2023-10-11T09:23:20.568 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-11T09:23:20.569 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi/7418830 --verbose --name vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi --owner scheduled_vshankar@teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/data-scan} -- /home/teuthworker/archive/vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi/7418830/orig.config.yaml 2023-10-11T09:23:20.628 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-11T09:23:20.715 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi/7418830 branch: wip-vshankar-testing-reef-20231009.131610 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/data-scan} email: vshankar@redhat.com first_in_suite: false job_id: '7418830' kernel: flavor: default kdb: true sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-vshankar-testing-reef-20231009.131610 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - bad backtrace - object missing on disk - error reading table object - error reading sessionmap - unmatched fragstat - unmatched rstat - was unreadable, recreating it now - Scrub error on inode - Metadata damage detected - inconsistent rstat on inode - Error recovering journal sha1: 9a774796f0ecdd11a4b771de396e627aa1a187af ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 9a774796f0ecdd11a4b771de396e627aa1a187af thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-reef-20231009.131610 sha1: 9a774796f0ecdd11a4b771de396e627aa1a187af owner: scheduled_vshankar@teuthology priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1116 sha1: 9a774796f0ecdd11a4b771de396e627aa1a187af sleep_before_teardown: 0 subset: 593/1024 suite: fs suite_branch: wip-vshankar-testing-reef-20231009.131610 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_9a774796f0ecdd11a4b771de396e627aa1a187af/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 9a774796f0ecdd11a4b771de396e627aa1a187af syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi078.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi133.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_data_scan teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-09_16:48:27 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-11T09:23:20.716 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_9a774796f0ecdd11a4b771de396e627aa1a187af/qa; will attempt to use it 2023-10-11T09:23:20.716 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_9a774796f0ecdd11a4b771de396e627aa1a187af/qa/tasks 2023-10-11T09:23:20.717 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-11T09:23:20.719 INFO:teuthology.task.internal:Checking packages... 2023-10-11T09:23:20.742 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '9a774796f0ecdd11a4b771de396e627aa1a187af' 2023-10-11T09:23:20.743 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-11T09:23:20.743 INFO:teuthology.packaging:ref: None 2023-10-11T09:23:20.743 INFO:teuthology.packaging:tag: None 2023-10-11T09:23:20.744 INFO:teuthology.packaging:branch: wip-vshankar-testing-reef-20231009.131610 2023-10-11T09:23:20.744 INFO:teuthology.packaging:sha1: 9a774796f0ecdd11a4b771de396e627aa1a187af 2023-10-11T09:23:20.744 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-vshankar-testing-reef-20231009.131610 2023-10-11T09:23:20.974 INFO:teuthology.task.internal:Found packages for ceph version 18.2.0-662-g9a774796-1focal 2023-10-11T09:23:20.975 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-11T09:23:20.982 INFO:teuthology.task.internal:no buildpackages task found 2023-10-11T09:23:20.982 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-11T09:23:20.988 INFO:teuthology.task.internal:Saving configuration 2023-10-11T09:23:21.006 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-11T09:23:21.014 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-11T09:23:21.040 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi078.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi/7418830', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-11 09:16:51.654493', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-10-11T09:23:21.068 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi133.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi/7418830', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-11 09:16:51.652908', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-10-11T09:23:21.068 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-11T09:23:21.075 INFO:teuthology.task.internal:roles: ubuntu@smithi078.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-11T09:23:21.076 INFO:teuthology.task.internal:roles: ubuntu@smithi133.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-11T09:23:21.076 INFO:teuthology.run_tasks:Running task console_log... 2023-10-11T09:23:21.149 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fed85817a60>, signals=[15]) 2023-10-11T09:23:21.151 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-11T09:23:21.157 INFO:teuthology.task.internal:Opening connections... 2023-10-11T09:23:21.158 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi078.front.sepia.ceph.com 2023-10-11T09:23:21.159 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-11T09:23:21.208 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi133.front.sepia.ceph.com 2023-10-11T09:23:21.209 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-11T09:23:21.257 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-11T09:23:21.264 DEBUG:teuthology.orchestra.run.smithi078:> uname -m 2023-10-11T09:23:21.284 INFO:teuthology.orchestra.run.smithi078.stdout:x86_64 2023-10-11T09:23:21.284 DEBUG:teuthology.orchestra.run.smithi078:> cat /etc/os-release 2023-10-11T09:23:21.332 INFO:teuthology.orchestra.run.smithi078.stdout:NAME="Ubuntu" 2023-10-11T09:23:21.333 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-10-11T09:23:21.333 INFO:teuthology.orchestra.run.smithi078.stdout:ID=ubuntu 2023-10-11T09:23:21.333 INFO:teuthology.orchestra.run.smithi078.stdout:ID_LIKE=debian 2023-10-11T09:23:21.334 INFO:teuthology.orchestra.run.smithi078.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-10-11T09:23:21.334 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION_ID="20.04" 2023-10-11T09:23:21.334 INFO:teuthology.orchestra.run.smithi078.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-11T09:23:21.334 INFO:teuthology.orchestra.run.smithi078.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-11T09:23:21.335 INFO:teuthology.orchestra.run.smithi078.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-11T09:23:21.335 INFO:teuthology.orchestra.run.smithi078.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-11T09:23:21.335 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION_CODENAME=focal 2023-10-11T09:23:21.335 INFO:teuthology.orchestra.run.smithi078.stdout:UBUNTU_CODENAME=focal 2023-10-11T09:23:21.336 INFO:teuthology.lock.ops:Updating smithi078.front.sepia.ceph.com on lock server 2023-10-11T09:23:21.357 DEBUG:teuthology.orchestra.run.smithi133:> uname -m 2023-10-11T09:23:21.364 INFO:teuthology.orchestra.run.smithi133.stdout:x86_64 2023-10-11T09:23:21.365 DEBUG:teuthology.orchestra.run.smithi133:> cat /etc/os-release 2023-10-11T09:23:21.414 INFO:teuthology.orchestra.run.smithi133.stdout:NAME="Ubuntu" 2023-10-11T09:23:21.414 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-10-11T09:23:21.414 INFO:teuthology.orchestra.run.smithi133.stdout:ID=ubuntu 2023-10-11T09:23:21.415 INFO:teuthology.orchestra.run.smithi133.stdout:ID_LIKE=debian 2023-10-11T09:23:21.415 INFO:teuthology.orchestra.run.smithi133.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-10-11T09:23:21.415 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_ID="20.04" 2023-10-11T09:23:21.415 INFO:teuthology.orchestra.run.smithi133.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-11T09:23:21.415 INFO:teuthology.orchestra.run.smithi133.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-11T09:23:21.416 INFO:teuthology.orchestra.run.smithi133.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-11T09:23:21.416 INFO:teuthology.orchestra.run.smithi133.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-11T09:23:21.416 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_CODENAME=focal 2023-10-11T09:23:21.416 INFO:teuthology.orchestra.run.smithi133.stdout:UBUNTU_CODENAME=focal 2023-10-11T09:23:21.417 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server 2023-10-11T09:23:21.450 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-11T09:23:21.460 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-11T09:23:21.465 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-11T09:23:21.466 DEBUG:teuthology.orchestra.run.smithi078:> test '!' -e /home/ubuntu/cephtest 2023-10-11T09:23:21.470 DEBUG:teuthology.orchestra.run.smithi133:> test '!' -e /home/ubuntu/cephtest 2023-10-11T09:23:21.475 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-11T09:23:21.481 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-11T09:23:21.482 DEBUG:teuthology.orchestra.run.smithi078:> test -z $(ls -A /var/lib/ceph) 2023-10-11T09:23:21.516 DEBUG:teuthology.orchestra.run.smithi133:> test -z $(ls -A /var/lib/ceph) 2023-10-11T09:23:21.541 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-11T09:23:21.594 INFO:teuthology.run_tasks:Running task kernel... 2023-10-11T09:23:21.608 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': 'fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c'} 2023-10-11T09:23:21.609 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': 'fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': 'fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c'}}, timeout 300 2023-10-11T09:23:21.610 DEBUG:teuthology.orchestra.run.smithi078:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-11T09:23:21.611 DEBUG:teuthology.orchestra.run.smithi133:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-11T09:23:21.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T09:23:21.619 DEBUG:teuthology.task.kernel:sha1 for mon.a is fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-11T09:23:21.619 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c"... 2023-10-11T09:23:21.620 DEBUG:teuthology.orchestra.run.smithi078:> uname -r 2023-10-11T09:23:21.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T09:23:21.621 DEBUG:teuthology.task.kernel:sha1 for mon.b is fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-11T09:23:21.621 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c"... 2023-10-11T09:23:21.622 DEBUG:teuthology.orchestra.run.smithi133:> uname -r 2023-10-11T09:23:21.668 INFO:teuthology.orchestra.run.smithi078.stdout:5.4.0-124-generic 2023-10-11T09:23:21.668 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-124-generic vs fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-11T09:23:21.669 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-11T09:23:21.669 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-11T09:23:21.673 INFO:teuthology.orchestra.run.smithi133.stdout:5.4.0-124-generic 2023-10-11T09:23:21.673 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-124-generic vs fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-11T09:23:21.674 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-11T09:23:21.674 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-11T09:23:21.817 INFO:teuthology.task.kernel:Installing linux-firmware on mon.a... 2023-10-11T09:23:21.817 DEBUG:teuthology.orchestra.run.smithi078:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2023-10-11T09:23:21.823 INFO:teuthology.task.kernel:Installing linux-firmware on mon.b... 2023-10-11T09:23:21.824 DEBUG:teuthology.orchestra.run.smithi133:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2023-10-11T09:23:21.839 INFO:teuthology.orchestra.run.smithi078.stderr:bash: line 0: cd: /lib/firmware/updates: No such file or directory 2023-10-11T09:23:21.844 INFO:teuthology.orchestra.run.smithi133.stderr:bash: line 0: cd: /lib/firmware/updates: No such file or directory 2023-10-11T09:23:21.954 INFO:teuthology.orchestra.run.smithi133.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2023-10-11T09:23:21.956 DEBUG:teuthology.orchestra.run.smithi133:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2023-10-11T09:23:21.970 INFO:teuthology.orchestra.run.smithi078.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2023-10-11T09:23:21.972 DEBUG:teuthology.orchestra.run.smithi078:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2023-10-11T09:23:21.987 DEBUG:teuthology.orchestra.run.smithi133:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2023-10-11T09:23:22.010 DEBUG:teuthology.orchestra.run.smithi078:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2023-10-11T09:23:22.026 DEBUG:teuthology.orchestra.run.smithi078:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2023-10-11T09:23:22.045 DEBUG:teuthology.orchestra.run.smithi133:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2023-10-11T09:25:18.675 INFO:teuthology.orchestra.run.smithi078.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2023-10-11T09:25:18.675 INFO:teuthology.orchestra.run.smithi078.stderr: * [new branch] main -> origin/main 2023-10-11T09:25:18.730 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20190312 -> 20190312 2023-10-11T09:25:18.731 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20190416 -> 20190416 2023-10-11T09:25:18.731 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20190514 -> 20190514 2023-10-11T09:25:18.731 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20190618 -> 20190618 2023-10-11T09:25:18.731 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20190717 -> 20190717 2023-10-11T09:25:18.731 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20190815 -> 20190815 2023-10-11T09:25:18.732 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20190923 -> 20190923 2023-10-11T09:25:18.732 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20191022 -> 20191022 2023-10-11T09:25:18.734 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20191215 -> 20191215 2023-10-11T09:25:18.734 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20200122 -> 20200122 2023-10-11T09:25:18.734 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20200316 -> 20200316 2023-10-11T09:25:18.734 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20200421 -> 20200421 2023-10-11T09:25:18.735 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20200519 -> 20200519 2023-10-11T09:25:18.735 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20200619 -> 20200619 2023-10-11T09:25:18.735 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20200721 -> 20200721 2023-10-11T09:25:18.735 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20200817 -> 20200817 2023-10-11T09:25:18.735 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20200918 -> 20200918 2023-10-11T09:25:18.735 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20201022 -> 20201022 2023-10-11T09:25:18.736 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20201118 -> 20201118 2023-10-11T09:25:18.736 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20201218 -> 20201218 2023-10-11T09:25:18.736 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20210208 -> 20210208 2023-10-11T09:25:18.736 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20210315 -> 20210315 2023-10-11T09:25:18.736 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20210511 -> 20210511 2023-10-11T09:25:18.737 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20210716 -> 20210716 2023-10-11T09:25:18.737 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20210818 -> 20210818 2023-10-11T09:25:18.739 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20210919 -> 20210919 2023-10-11T09:25:18.739 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20211027 -> 20211027 2023-10-11T09:25:18.740 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20211216 -> 20211216 2023-10-11T09:25:18.740 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20220209 -> 20220209 2023-10-11T09:25:18.740 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20220310 -> 20220310 2023-10-11T09:25:18.740 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20220411 -> 20220411 2023-10-11T09:25:18.740 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20220509 -> 20220509 2023-10-11T09:25:18.741 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20220610 -> 20220610 2023-10-11T09:25:18.741 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20220708 -> 20220708 2023-10-11T09:25:18.741 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20220815 -> 20220815 2023-10-11T09:25:18.741 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20220913 -> 20220913 2023-10-11T09:25:18.741 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20221012 -> 20221012 2023-10-11T09:25:18.741 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20221109 -> 20221109 2023-10-11T09:25:18.742 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20221214 -> 20221214 2023-10-11T09:25:18.742 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20230117 -> 20230117 2023-10-11T09:25:18.742 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20230210 -> 20230210 2023-10-11T09:25:18.742 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20230310 -> 20230310 2023-10-11T09:25:18.742 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20230404 -> 20230404 2023-10-11T09:25:18.743 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20230515 -> 20230515 2023-10-11T09:25:18.743 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20230625 -> 20230625 2023-10-11T09:25:18.743 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20230804 -> 20230804 2023-10-11T09:25:18.743 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20230919 -> 20230919 2023-10-11T09:25:22.401 INFO:teuthology.orchestra.run.smithi078.stderr:Updating files: 51% (1618/3158) Updating files: 52% (1643/3158) Updating files: 53% (1674/3158) Updating files: 54% (1706/3158) Updating files: 55% (1737/3158) Updating files: 56% (1769/3158) Updating files: 57% (1801/3158) Updating files: 58% (1832/3158) Updating files: 59% (1864/3158) Updating files: 60% (1895/3158) Updating files: 61% (1927/3158) Updating files: 62% (1958/3158) Updating files: 63% (1990/3158) Updating files: 64% (2022/3158) Updating files: 65% (2053/3158) Updating files: 66% (2085/3158) Updating files: 66% (2104/3158) Updating files: 67% (2116/3158) Updating files: 68% (2148/3158) Updating files: 69% (2180/3158) Updating files: 70% (2211/3158) Updating files: 71% (2243/3158) Updating files: 72% (2274/3158) Updating files: 73% (2306/3158) Updating files: 74% (2337/3158) Updating files: 75% (2369/3158) Updating files: 76% (2401/3158) Updating files: 77% (2432/3158) Updating files: 78% (2464/3158) Updating files: 79% (2495/3158) Updating files: 80% (2527/3158) Updating files: 81% (2558/3158) Updating files: 81% (2587/3158) Updating files: 82% (2590/3158) Updating files: 83% (2622/3158) Updating files: 84% (2653/3158) Updating files: 85% (2685/3158) Updating files: 86% (2716/3158) Updating files: 87% (2748/3158) Updating files: 88% (2780/3158) Updating files: 89% (2811/3158) Updating files: 90% (2843/3158) Updating files: 91% (2874/3158) Updating files: 92% (2906/3158) Updating files: 93% (2937/3158) Updating files: 94% (2969/3158) Updating files: 95% (3001/3158) Updating files: 96% (3032/3158) Updating files: 97% (3064/3158) Updating files: 98% (3095/3158) Updating files: 99% (3127/3158) Updating files: 100% (3158/3158) Updating files: 100% (3158/3158), done. 2023-10-11T09:25:22.406 INFO:teuthology.orchestra.run.smithi078.stdout:HEAD is now at 7727f7e3 Merge branch 'patch-1696561325' into 'main' 2023-10-11T09:25:22.422 INFO:teuthology.task.kernel:Downloading kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on mon.a... 2023-10-11T09:25:22.422 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-11T09:25:22.580 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/ubuntu/focal/flavors/default/pool/main/l/linux-6.5.0-rc7-gfef64f93da2b/ 2023-10-11T09:25:22.580 DEBUG:teuthology.orchestra.run.smithi078:> rm -f /tmp/linux-image.deb && echo linux-image-6.5.0-rc7-gfef64f93da2b_6.5.0-rc7-gfef64f93da2b-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/ubuntu/focal/flavors/default/pool/main/l/linux-6.5.0-rc7-gfef64f93da2b/ --input-file=- 2023-10-11T09:25:22.583 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi078.front.sepia.ceph.com to complete... 2023-10-11T09:25:24.074 INFO:teuthology.orchestra.run.smithi133.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2023-10-11T09:25:24.075 INFO:teuthology.orchestra.run.smithi133.stderr: * [new branch] main -> origin/main 2023-10-11T09:25:24.129 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20190312 -> 20190312 2023-10-11T09:25:24.130 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20190416 -> 20190416 2023-10-11T09:25:24.130 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20190514 -> 20190514 2023-10-11T09:25:24.130 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20190618 -> 20190618 2023-10-11T09:25:24.130 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20190717 -> 20190717 2023-10-11T09:25:24.130 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20190815 -> 20190815 2023-10-11T09:25:24.131 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20190923 -> 20190923 2023-10-11T09:25:24.131 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20191022 -> 20191022 2023-10-11T09:25:24.131 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20191215 -> 20191215 2023-10-11T09:25:24.133 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20200122 -> 20200122 2023-10-11T09:25:24.133 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20200316 -> 20200316 2023-10-11T09:25:24.133 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20200421 -> 20200421 2023-10-11T09:25:24.133 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20200519 -> 20200519 2023-10-11T09:25:24.134 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20200619 -> 20200619 2023-10-11T09:25:24.134 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20200721 -> 20200721 2023-10-11T09:25:24.134 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20200817 -> 20200817 2023-10-11T09:25:24.134 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20200918 -> 20200918 2023-10-11T09:25:24.135 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20201022 -> 20201022 2023-10-11T09:25:24.135 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20201118 -> 20201118 2023-10-11T09:25:24.135 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20201218 -> 20201218 2023-10-11T09:25:24.135 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20210208 -> 20210208 2023-10-11T09:25:24.135 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20210315 -> 20210315 2023-10-11T09:25:24.135 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20210511 -> 20210511 2023-10-11T09:25:24.136 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20210716 -> 20210716 2023-10-11T09:25:24.136 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20210818 -> 20210818 2023-10-11T09:25:24.138 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20210919 -> 20210919 2023-10-11T09:25:24.138 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20211027 -> 20211027 2023-10-11T09:25:24.138 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20211216 -> 20211216 2023-10-11T09:25:24.139 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20220209 -> 20220209 2023-10-11T09:25:24.139 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20220310 -> 20220310 2023-10-11T09:25:24.139 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20220411 -> 20220411 2023-10-11T09:25:24.139 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20220509 -> 20220509 2023-10-11T09:25:24.139 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20220610 -> 20220610 2023-10-11T09:25:24.140 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20220708 -> 20220708 2023-10-11T09:25:24.140 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20220815 -> 20220815 2023-10-11T09:25:24.140 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20220913 -> 20220913 2023-10-11T09:25:24.140 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20221012 -> 20221012 2023-10-11T09:25:24.140 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20221109 -> 20221109 2023-10-11T09:25:24.140 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20221214 -> 20221214 2023-10-11T09:25:24.141 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20230117 -> 20230117 2023-10-11T09:25:24.141 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20230210 -> 20230210 2023-10-11T09:25:24.141 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20230310 -> 20230310 2023-10-11T09:25:24.141 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20230404 -> 20230404 2023-10-11T09:25:24.141 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20230515 -> 20230515 2023-10-11T09:25:24.142 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20230625 -> 20230625 2023-10-11T09:25:24.142 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20230804 -> 20230804 2023-10-11T09:25:24.142 INFO:teuthology.orchestra.run.smithi133.stderr: * [new tag] 20230919 -> 20230919 2023-10-11T09:25:24.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11 09:25:24 URL:https://4.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/ubuntu/focal/flavors/default/pool/main/l/linux-6.5.0-rc7-gfef64f93da2b/linux-image-6.5.0-rc7-gfef64f93da2b_6.5.0-rc7-gfef64f93da2b-1_amd64.deb [38832758/38832758] -> "/tmp/linux-image.deb" [1] 2023-10-11T09:25:24.330 INFO:teuthology.orchestra.run.smithi078.stderr:FINISHED --2023-10-11 09:25:24-- 2023-10-11T09:25:24.330 INFO:teuthology.orchestra.run.smithi078.stderr:Total wall clock time: 1.7s 2023-10-11T09:25:24.330 INFO:teuthology.orchestra.run.smithi078.stderr:Downloaded: 1 files, 37M in 1.6s (23.3 MB/s) 2023-10-11T09:25:24.331 INFO:teuthology.task.kernel:Installing kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on mon.a... 2023-10-11T09:25:24.332 DEBUG:teuthology.orchestra.run.smithi078:> sudo dpkg -i /tmp/linux-image.deb 2023-10-11T09:25:24.590 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package linux-image-6.5.0-rc7-gfef64f93da2b. 2023-10-11T09:25:26.307 INFO:teuthology.orchestra.run.smithi078.stdout:(Reading database ... 104373 files and directories currently installed.) 2023-10-11T09:25:26.312 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack /tmp/linux-image.deb ... 2023-10-11T09:25:26.483 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking linux-image-6.5.0-rc7-gfef64f93da2b (6.5.0-rc7-gfef64f93da2b-1) ... 2023-10-11T09:25:27.804 INFO:teuthology.orchestra.run.smithi133.stderr:Updating files: 51% (1622/3158) Updating files: 52% (1643/3158) Updating files: 53% (1674/3158) Updating files: 54% (1706/3158) Updating files: 55% (1737/3158) Updating files: 56% (1769/3158) Updating files: 57% (1801/3158) Updating files: 58% (1832/3158) Updating files: 59% (1864/3158) Updating files: 60% (1895/3158) Updating files: 61% (1927/3158) Updating files: 62% (1958/3158) Updating files: 63% (1990/3158) Updating files: 64% (2022/3158) Updating files: 65% (2053/3158) Updating files: 66% (2085/3158) Updating files: 66% (2104/3158) Updating files: 67% (2116/3158) Updating files: 68% (2148/3158) Updating files: 69% (2180/3158) Updating files: 70% (2211/3158) Updating files: 71% (2243/3158) Updating files: 72% (2274/3158) Updating files: 73% (2306/3158) Updating files: 74% (2337/3158) Updating files: 75% (2369/3158) Updating files: 76% (2401/3158) Updating files: 77% (2432/3158) Updating files: 78% (2464/3158) Updating files: 79% (2495/3158) Updating files: 80% (2527/3158) Updating files: 81% (2558/3158) Updating files: 81% (2587/3158) Updating files: 82% (2590/3158) Updating files: 83% (2622/3158) Updating files: 84% (2653/3158) Updating files: 85% (2685/3158) Updating files: 86% (2716/3158) Updating files: 87% (2748/3158) Updating files: 88% (2780/3158) Updating files: 89% (2811/3158) Updating files: 90% (2843/3158) Updating files: 91% (2874/3158) Updating files: 92% (2906/3158) Updating files: 93% (2937/3158) Updating files: 94% (2969/3158) Updating files: 95% (3001/3158) Updating files: 96% (3032/3158) Updating files: 97% (3064/3158) Updating files: 98% (3095/3158) Updating files: 99% (3127/3158) Updating files: 100% (3158/3158) Updating files: 100% (3158/3158), done. 2023-10-11T09:25:27.808 INFO:teuthology.orchestra.run.smithi133.stdout:HEAD is now at 7727f7e3 Merge branch 'patch-1696561325' into 'main' 2023-10-11T09:25:27.824 INFO:teuthology.task.kernel:Downloading kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on mon.b... 2023-10-11T09:25:27.825 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-11T09:25:28.102 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/ubuntu/focal/flavors/default/pool/main/l/linux-6.5.0-rc7-gfef64f93da2b/ 2023-10-11T09:25:28.102 DEBUG:teuthology.orchestra.run.smithi133:> rm -f /tmp/linux-image.deb && echo linux-image-6.5.0-rc7-gfef64f93da2b_6.5.0-rc7-gfef64f93da2b-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/ubuntu/focal/flavors/default/pool/main/l/linux-6.5.0-rc7-gfef64f93da2b/ --input-file=- 2023-10-11T09:25:28.105 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi133.front.sepia.ceph.com to complete... 2023-10-11T09:25:29.514 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up linux-image-6.5.0-rc7-gfef64f93da2b (6.5.0-rc7-gfef64f93da2b-1) ... 2023-10-11T09:25:29.646 INFO:teuthology.orchestra.run.smithi078.stderr:update-initramfs: Generating /boot/initrd.img-6.5.0-rc7-gfef64f93da2b 2023-10-11T09:25:29.783 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11 09:25:30 URL:https://4.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/ubuntu/focal/flavors/default/pool/main/l/linux-6.5.0-rc7-gfef64f93da2b/linux-image-6.5.0-rc7-gfef64f93da2b_6.5.0-rc7-gfef64f93da2b-1_amd64.deb [38832758/38832758] -> "/tmp/linux-image.deb" [1] 2023-10-11T09:25:29.783 INFO:teuthology.orchestra.run.smithi133.stderr:FINISHED --2023-10-11 09:25:30-- 2023-10-11T09:25:29.784 INFO:teuthology.orchestra.run.smithi133.stderr:Total wall clock time: 1.6s 2023-10-11T09:25:29.784 INFO:teuthology.orchestra.run.smithi133.stderr:Downloaded: 1 files, 37M in 1.4s (25.9 MB/s) 2023-10-11T09:25:29.785 INFO:teuthology.task.kernel:Installing kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on mon.b... 2023-10-11T09:25:29.785 DEBUG:teuthology.orchestra.run.smithi133:> sudo dpkg -i /tmp/linux-image.deb 2023-10-11T09:25:30.099 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package linux-image-6.5.0-rc7-gfef64f93da2b. 2023-10-11T09:25:31.863 INFO:teuthology.orchestra.run.smithi133.stdout:(Reading database ... 104373 files and directories currently installed.) 2023-10-11T09:25:31.868 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack /tmp/linux-image.deb ... 2023-10-11T09:25:31.999 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking linux-image-6.5.0-rc7-gfef64f93da2b (6.5.0-rc7-gfef64f93da2b-1) ... 2023-10-11T09:25:35.010 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up linux-image-6.5.0-rc7-gfef64f93da2b (6.5.0-rc7-gfef64f93da2b-1) ... 2023-10-11T09:25:35.129 INFO:teuthology.orchestra.run.smithi133.stderr:update-initramfs: Generating /boot/initrd.img-6.5.0-rc7-gfef64f93da2b 2023-10-11T09:25:55.373 INFO:teuthology.orchestra.run.smithi078.stderr:Sourcing file `/etc/default/grub' 2023-10-11T09:25:55.378 INFO:teuthology.orchestra.run.smithi078.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-11T09:25:55.406 INFO:teuthology.orchestra.run.smithi078.stderr:Generating grub configuration file ... 2023-10-11T09:25:55.757 INFO:teuthology.orchestra.run.smithi078.stderr:Found linux image: /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b 2023-10-11T09:25:55.778 INFO:teuthology.orchestra.run.smithi078.stderr:Found initrd image: /boot/initrd.img-6.5.0-rc7-gfef64f93da2b 2023-10-11T09:25:56.188 INFO:teuthology.orchestra.run.smithi078.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-10-11T09:25:56.201 INFO:teuthology.orchestra.run.smithi078.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-10-11T09:25:58.848 INFO:teuthology.orchestra.run.smithi078.stderr:done 2023-10-11T09:25:59.025 DEBUG:teuthology.orchestra.run.smithi078:> dpkg-deb -c /tmp/linux-image.deb 2023-10-11T09:25:59.076 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./ 2023-10-11T09:25:59.076 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./boot/ 2023-10-11T09:25:59.076 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 4382168 2023-09-11 02:20 ./boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-11T09:25:59.093 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 168454 2023-09-11 02:20 ./boot/config-6.5.0-rc7-gfef64f93da2b 2023-10-11T09:25:59.094 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11294080 2023-09-11 02:20 ./boot/vmlinuz-6.5.0-rc7-gfef64f93da2b 2023-10-11T09:25:59.114 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./etc/ 2023-10-11T09:25:59.114 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./etc/kernel/ 2023-10-11T09:25:59.115 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./etc/kernel/postinst.d/ 2023-10-11T09:25:59.115 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./etc/kernel/postrm.d/ 2023-10-11T09:25:59.115 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./etc/kernel/preinst.d/ 2023-10-11T09:25:59.115 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./etc/kernel/prerm.d/ 2023-10-11T09:25:59.115 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/ 2023-10-11T09:25:59.116 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/ 2023-10-11T09:25:59.116 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/ 2023-10-11T09:25:59.116 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/ 2023-10-11T09:25:59.116 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/ 2023-10-11T09:25:59.116 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/ 2023-10-11T09:25:59.117 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/ 2023-10-11T09:25:59.117 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 490385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/aesni-intel.ko 2023-10-11T09:25:59.120 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-11T09:25:59.120 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-11T09:25:59.120 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-11T09:25:59.120 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 93113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-11T09:25:59.121 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 106729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-11T09:25:59.121 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 152241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-11T09:25:59.121 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-11T09:25:59.121 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13777 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-11T09:25:59.121 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 83889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-11T09:25:59.121 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-11T09:25:59.122 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 47225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-11T09:25:59.122 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 47209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-11T09:25:59.122 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 51801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-11T09:25:59.122 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 38937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-11T09:25:59.122 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 40529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-11T09:25:59.123 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 58585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-11T09:25:59.123 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 50497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-11T09:25:59.123 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 32065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-11T09:25:59.123 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-11T09:25:59.123 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/ 2023-10-11T09:25:59.124 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/ 2023-10-11T09:25:59.124 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce/ 2023-10-11T09:25:59.124 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 40113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-11T09:25:59.124 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpuid.ko 2023-10-11T09:25:59.124 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/msr.ko 2023-10-11T09:25:59.124 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/ 2023-10-11T09:25:59.125 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 348201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-11T09:25:59.125 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 882105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-11T09:25:59.125 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 2277369 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm.ko 2023-10-11T09:25:59.185 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ 2023-10-11T09:25:59.186 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 53585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/af_alg.ko 2023-10-11T09:25:59.186 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/algif_hash.ko 2023-10-11T09:25:59.186 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/algif_skcipher.ko 2023-10-11T09:25:59.186 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ansi_cprng.ko 2023-10-11T09:25:59.187 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/anubis.ko 2023-10-11T09:25:59.187 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/arc4.ko 2023-10-11T09:25:59.187 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/asymmetric_keys/ 2023-10-11T09:25:59.187 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2023-10-11T09:25:59.187 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/ 2023-10-11T09:25:59.188 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-11T09:25:59.188 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_pq.ko 2023-10-11T09:25:59.188 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21561 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-11T09:25:59.188 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_tx.ko 2023-10-11T09:25:59.188 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_xor.ko 2023-10-11T09:25:59.189 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/raid6test.ko 2023-10-11T09:25:59.189 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authenc.ko 2023-10-11T09:25:59.189 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authencesn.ko 2023-10-11T09:25:59.189 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blake2b_generic.ko 2023-10-11T09:25:59.190 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_common.ko 2023-10-11T09:25:59.190 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_generic.ko 2023-10-11T09:25:59.190 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 38873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/camellia_generic.ko 2023-10-11T09:25:59.190 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast5_generic.ko 2023-10-11T09:25:59.191 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast6_generic.ko 2023-10-11T09:25:59.191 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast_common.ko 2023-10-11T09:25:59.191 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 35097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ccm.ko 2023-10-11T09:25:59.191 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cmac.ko 2023-10-11T09:25:59.191 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crc32_generic.ko 2023-10-11T09:25:59.192 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 46961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cryptd.ko 2023-10-11T09:25:59.192 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_engine.ko 2023-10-11T09:25:59.192 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_simd.ko 2023-10-11T09:25:59.192 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_user.ko 2023-10-11T09:25:59.192 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cts.ko 2023-10-11T09:25:59.192 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/des_generic.ko 2023-10-11T09:25:59.193 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/dh_generic.ko 2023-10-11T09:25:59.193 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/echainiv.ko 2023-10-11T09:25:59.193 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/essiv.ko 2023-10-11T09:25:59.193 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/fcrypt.ko 2023-10-11T09:25:59.194 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/khazad.ko 2023-10-11T09:25:59.194 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lrw.ko 2023-10-11T09:25:59.195 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4.ko 2023-10-11T09:25:59.195 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4hc.ko 2023-10-11T09:25:59.196 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11761 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/md4.ko 2023-10-11T09:25:59.196 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/michael_mic.ko 2023-10-11T09:25:59.196 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcbc.ko 2023-10-11T09:25:59.197 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcrypt.ko 2023-10-11T09:25:59.197 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/rmd160.ko 2023-10-11T09:25:59.197 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/seed.ko 2023-10-11T09:25:59.197 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/serpent_generic.ko 2023-10-11T09:25:59.198 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 149193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tcrypt.ko 2023-10-11T09:25:59.198 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tea.ko 2023-10-11T09:25:59.198 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28777 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_common.ko 2023-10-11T09:25:59.199 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_generic.ko 2023-10-11T09:25:59.199 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/vmac.ko 2023-10-11T09:25:59.200 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/wp512.ko 2023-10-11T09:25:59.200 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xcbc.ko 2023-10-11T09:25:59.200 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xor.ko 2023-10-11T09:25:59.200 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xts.ko 2023-10-11T09:25:59.200 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9481 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xxhash_generic.ko 2023-10-11T09:25:59.201 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ 2023-10-11T09:25:59.201 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/ 2023-10-11T09:25:59.201 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_extlog.ko 2023-10-11T09:25:59.201 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-11T09:25:59.201 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_pad.ko 2023-10-11T09:25:59.202 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/apei/ 2023-10-11T09:25:59.202 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 31241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/apei/einj.ko 2023-10-11T09:25:59.202 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/ec_sys.ko 2023-10-11T09:25:59.202 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/platform_profile.ko 2023-10-11T09:25:59.202 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbs.ko 2023-10-11T09:25:59.203 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbshc.ko 2023-10-11T09:25:59.203 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 100401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/video.ko 2023-10-11T09:25:59.203 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ 2023-10-11T09:25:59.203 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/acard-ahci.ko 2023-10-11T09:25:59.203 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 88625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ahci.ko 2023-10-11T09:25:59.203 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ahci_platform.ko 2023-10-11T09:25:59.204 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 87841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/libahci.ko 2023-10-11T09:25:59.204 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 35049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/libahci_platform.ko 2023-10-11T09:25:59.204 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_acpi.ko 2023-10-11T09:25:59.204 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ali.ko 2023-10-11T09:25:59.204 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_amd.ko 2023-10-11T09:25:59.205 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_artop.ko 2023-10-11T09:25:59.205 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atiixp.ko 2023-10-11T09:25:59.205 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atp867x.ko 2023-10-11T09:25:59.205 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd640.ko 2023-10-11T09:25:59.205 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-11T09:25:59.205 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cypress.ko 2023-10-11T09:25:59.206 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_efar.ko 2023-10-11T09:25:59.206 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt366.ko 2023-10-11T09:25:59.206 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-11T09:25:59.206 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-11T09:25:59.206 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-11T09:25:59.207 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it8213.ko 2023-10-11T09:25:59.207 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 35177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it821x.ko 2023-10-11T09:25:59.207 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_jmicron.ko 2023-10-11T09:25:59.207 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 51737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_legacy.ko 2023-10-11T09:25:59.208 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_marvell.ko 2023-10-11T09:25:59.208 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_mpiix.ko 2023-10-11T09:25:59.208 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_netcell.ko 2023-10-11T09:25:59.208 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ninja32.ko 2023-10-11T09:25:59.208 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87410.ko 2023-10-11T09:25:59.208 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87415.ko 2023-10-11T09:25:59.209 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-11T09:25:59.209 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_opti.ko 2023-10-11T09:25:59.209 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_optidma.ko 2023-10-11T09:25:59.209 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 32521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-11T09:25:59.209 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 37729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-11T09:25:59.209 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-11T09:25:59.210 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_piccolo.ko 2023-10-11T09:25:59.210 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_radisys.ko 2023-10-11T09:25:59.210 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_rdc.ko 2023-10-11T09:25:59.210 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_rz1000.ko 2023-10-11T09:25:59.210 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sch.ko 2023-10-11T09:25:59.211 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_serverworks.ko 2023-10-11T09:25:59.211 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sil680.ko 2023-10-11T09:25:59.211 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-11T09:25:59.211 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_triflex.ko 2023-10-11T09:25:59.211 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_via.ko 2023-10-11T09:25:59.212 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pdc_adma.ko 2023-10-11T09:25:59.212 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_inic162x.ko 2023-10-11T09:25:59.212 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 83225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_mv.ko 2023-10-11T09:25:59.212 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 70537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_nv.ko 2023-10-11T09:25:59.212 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 37369 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_promise.ko 2023-10-11T09:25:59.212 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_qstor.ko 2023-10-11T09:25:59.213 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil.ko 2023-10-11T09:25:59.213 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 38273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil24.ko 2023-10-11T09:25:59.213 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sis.ko 2023-10-11T09:25:59.213 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_svw.ko 2023-10-11T09:25:59.213 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 44721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sx4.ko 2023-10-11T09:25:59.213 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_uli.ko 2023-10-11T09:25:59.214 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 35121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_via.ko 2023-10-11T09:25:59.214 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_vsc.ko 2023-10-11T09:25:59.214 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay/ 2023-10-11T09:25:59.214 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-11T09:25:59.214 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma/ 2023-10-11T09:25:59.215 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 124457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma/bcma.ko 2023-10-11T09:25:59.215 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/ 2023-10-11T09:25:59.215 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe/ 2023-10-11T09:25:59.215 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 113705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe/aoe.ko 2023-10-11T09:25:59.215 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd/ 2023-10-11T09:25:59.216 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 968433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd/drbd.ko 2023-10-11T09:25:59.216 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 173201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/floppy.ko 2023-10-11T09:25:59.216 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx/ 2023-10-11T09:25:59.216 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 94297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-11T09:25:59.216 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 115345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/nbd.ko 2023-10-11T09:25:59.216 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk/ 2023-10-11T09:25:59.217 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 110361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-11T09:25:59.217 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 98225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/pktcdvd.ko 2023-10-11T09:25:59.217 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 259401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/rbd.ko 2023-10-11T09:25:59.217 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback/ 2023-10-11T09:25:59.217 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 98129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-11T09:25:59.217 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram/ 2023-10-11T09:25:59.218 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 60265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram/zram.ko 2023-10-11T09:25:59.218 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ 2023-10-11T09:25:59.218 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/applicom.ko 2023-10-11T09:25:59.218 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hangcheck-timer.ko 2023-10-11T09:25:59.218 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/ 2023-10-11T09:25:59.219 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-11T09:25:59.219 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-11T09:25:59.219 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-11T09:25:59.219 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-11T09:25:59.219 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15633 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-11T09:25:59.219 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ 2023-10-11T09:25:59.220 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-11T09:25:59.220 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 150665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-11T09:25:59.220 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26561 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-11T09:25:59.220 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 139057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-11T09:25:59.221 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 70833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-11T09:25:59.221 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 53609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-11T09:25:59.221 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/lp.ko 2023-10-11T09:25:59.221 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave/ 2023-10-11T09:25:59.221 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 84121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave/mwave.ko 2023-10-11T09:25:59.221 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/nvram.ko 2023-10-11T09:25:59.222 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 32169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ppdev.ko 2023-10-11T09:25:59.222 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 51113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tlclk.ko 2023-10-11T09:25:59.222 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/ 2023-10-11T09:25:59.222 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-11T09:25:59.222 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2023-10-11T09:25:59.222 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2023-10-11T09:25:59.223 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 32985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2023-10-11T09:25:59.223 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-11T09:25:59.223 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-11T09:25:59.223 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23561 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/xen-tpmfront.ko 2023-10-11T09:25:59.223 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/xillybus/ 2023-10-11T09:25:59.223 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/xillybus/xillybus_class.ko 2023-10-11T09:25:59.224 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 50945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/xillybus/xillybus_core.ko 2023-10-11T09:25:59.224 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/xillybus/xillybus_pcie.ko 2023-10-11T09:25:59.224 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/clk/ 2023-10-11T09:25:59.224 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/clk/clk-palmas.ko 2023-10-11T09:25:59.224 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 47849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/clk/clk-si5351.ko 2023-10-11T09:25:59.224 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/clk/clk-twl6040.ko 2023-10-11T09:25:59.225 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/clk/clk-wm831x.ko 2023-10-11T09:25:59.225 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/ 2023-10-11T09:25:59.225 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-11T09:25:59.225 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-11T09:25:59.225 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-11T09:25:59.225 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ 2023-10-11T09:25:59.226 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ 2023-10-11T09:25:59.226 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 70785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-11T09:25:59.226 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 234473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-11T09:25:59.226 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/ 2023-10-11T09:25:59.226 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/ 2023-10-11T09:25:59.227 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common/ 2023-10-11T09:25:59.227 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 408489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-11T09:25:59.227 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2023-10-11T09:25:59.227 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-11T09:25:59.227 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-aes.ko 2023-10-11T09:25:59.227 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-sha.ko 2023-10-11T09:25:59.228 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio/ 2023-10-11T09:25:59.228 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 68937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-11T09:25:59.228 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca/ 2023-10-11T09:25:59.228 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca/dca.ko 2023-10-11T09:25:59.228 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ 2023-10-11T09:25:59.228 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw/ 2023-10-11T09:25:59.229 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-11T09:25:59.229 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat/ 2023-10-11T09:25:59.229 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 122641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-11T09:25:59.229 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/ 2023-10-11T09:25:59.229 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 80273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/amd64_edac.ko 2023-10-11T09:25:59.230 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 35769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/e752x_edac.ko 2023-10-11T09:25:59.230 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 63617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-11T09:25:59.230 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3000_edac.ko 2023-10-11T09:25:59.230 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3200_edac.ko 2023-10-11T09:25:59.230 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5100_edac.ko 2023-10-11T09:25:59.230 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5400_edac.ko 2023-10-11T09:25:59.231 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7300_edac.ko 2023-10-11T09:25:59.231 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 52025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7core_edac.ko 2023-10-11T09:25:59.231 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i82975x_edac.ko 2023-10-11T09:25:59.231 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/ie31200_edac.ko 2023-10-11T09:25:59.231 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 56809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/sb_edac.ko 2023-10-11T09:25:59.232 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/x38_edac.ko 2023-10-11T09:25:59.232 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/extcon/ 2023-10-11T09:25:59.232 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/extcon/extcon-gpio.ko 2023-10-11T09:25:59.232 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/extcon/extcon-max14577.ko 2023-10-11T09:25:59.232 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/extcon/extcon-max77693.ko 2023-10-11T09:25:59.233 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/extcon/extcon-max8997.ko 2023-10-11T09:25:59.233 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/extcon/extcon-palmas.ko 2023-10-11T09:25:59.233 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/extcon/extcon-rt8973a.ko 2023-10-11T09:25:59.233 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/extcon/extcon-sm5502.ko 2023-10-11T09:25:59.233 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/ 2023-10-11T09:25:59.234 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/dmi-sysfs.ko 2023-10-11T09:25:59.234 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/efi/ 2023-10-11T09:25:59.234 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/efi/efi-pstore.ko 2023-10-11T09:25:59.234 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-11T09:25:59.234 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/ 2023-10-11T09:25:59.234 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-adp5520.ko 2023-10-11T09:25:59.235 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-amd8111.ko 2023-10-11T09:25:59.235 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-arizona.ko 2023-10-11T09:25:59.235 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-da9052.ko 2023-10-11T09:25:59.235 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-da9055.ko 2023-10-11T09:25:59.235 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-dln2.ko 2023-10-11T09:25:59.235 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 77849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-f7188x.ko 2023-10-11T09:25:59.236 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-generic.ko 2023-10-11T09:25:59.236 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-ich.ko 2023-10-11T09:25:59.236 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-janz-ttl.ko 2023-10-11T09:25:59.236 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-kempld.ko 2023-10-11T09:25:59.236 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-lp3943.ko 2023-10-11T09:25:59.236 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-max7300.ko 2023-10-11T09:25:59.237 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-max730x.ko 2023-10-11T09:25:59.237 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-max732x.ko 2023-10-11T09:25:59.237 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-ml-ioh.ko 2023-10-11T09:25:59.237 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 46761 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-pca953x.ko 2023-10-11T09:25:59.237 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-pcf857x.ko 2023-10-11T09:25:59.238 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14377 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-rdc321x.ko 2023-10-11T09:25:59.238 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-sch.ko 2023-10-11T09:25:59.238 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-sch311x.ko 2023-10-11T09:25:59.238 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-tps65912.ko 2023-10-11T09:25:59.238 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-twl4030.ko 2023-10-11T09:25:59.239 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-twl6040.ko 2023-10-11T09:25:59.239 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-11T09:25:59.239 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-vx855.ko 2023-10-11T09:25:59.239 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-wm831x.ko 2023-10-11T09:25:59.239 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-wm8350.ko 2023-10-11T09:25:59.239 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-wm8994.ko 2023-10-11T09:25:59.240 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/ 2023-10-11T09:25:59.240 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-a4tech.ko 2023-10-11T09:25:59.240 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 39081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-apple.ko 2023-10-11T09:25:59.240 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14481 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-appleir.ko 2023-10-11T09:25:59.240 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-aureal.ko 2023-10-11T09:25:59.240 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-axff.ko 2023-10-11T09:25:59.241 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-belkin.ko 2023-10-11T09:25:59.241 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-cherry.ko 2023-10-11T09:25:59.241 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-chicony.ko 2023-10-11T09:25:59.241 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 59145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-cp2112.ko 2023-10-11T09:25:59.241 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-cypress.ko 2023-10-11T09:25:59.242 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-dr.ko 2023-10-11T09:25:59.242 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elecom.ko 2023-10-11T09:25:59.242 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elo.ko 2023-10-11T09:25:59.242 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-emsff.ko 2023-10-11T09:25:59.242 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-ezkey.ko 2023-10-11T09:25:59.243 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gaff.ko 2023-10-11T09:25:59.243 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-generic.ko 2023-10-11T09:25:59.243 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gt683r.ko 2023-10-11T09:25:59.243 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gyration.ko 2023-10-11T09:25:59.243 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-11T09:25:59.243 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-11T09:25:59.244 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtekff.ko 2023-10-11T09:25:59.244 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-hyperv.ko 2023-10-11T09:25:59.244 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-icade.ko 2023-10-11T09:25:59.244 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-kensington.ko 2023-10-11T09:25:59.244 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-keytouch.ko 2023-10-11T09:25:59.244 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-kye.ko 2023-10-11T09:25:59.245 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lcpower.ko 2023-10-11T09:25:59.245 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-led.ko 2023-10-11T09:25:59.245 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 42297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lenovo.ko 2023-10-11T09:25:59.245 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30761 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lg-g15.ko 2023-10-11T09:25:59.245 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 57809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-11T09:25:59.245 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 101009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-11T09:25:59.246 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 80993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech.ko 2023-10-11T09:25:59.246 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-magicmouse.ko 2023-10-11T09:25:59.246 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-microsoft.ko 2023-10-11T09:25:59.246 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-monterey.ko 2023-10-11T09:25:59.246 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 50153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-multitouch.ko 2023-10-11T09:25:59.247 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 32593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-ntrig.ko 2023-10-11T09:25:59.247 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-ortek.ko 2023-10-11T09:25:59.247 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-penmount.ko 2023-10-11T09:25:59.247 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-petalynx.ko 2023-10-11T09:25:59.247 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 72937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-picolcd.ko 2023-10-11T09:25:59.248 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-pl.ko 2023-10-11T09:25:59.248 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-plantronics.ko 2023-10-11T09:25:59.248 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-primax.ko 2023-10-11T09:25:59.248 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 35457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-rmi.ko 2023-10-11T09:25:59.248 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20761 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-11T09:25:59.248 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-11T09:25:59.249 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 31361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-11T09:25:59.249 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-11T09:25:59.249 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 31065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-11T09:25:59.249 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-11T09:25:59.249 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-11T09:25:59.250 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-11T09:25:59.250 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-11T09:25:59.250 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-11T09:25:59.250 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-11T09:25:59.250 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat.ko 2023-10-11T09:25:59.251 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-saitek.ko 2023-10-11T09:25:59.251 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-samsung.ko 2023-10-11T09:25:59.251 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-11T09:25:59.251 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sjoy.ko 2023-10-11T09:25:59.251 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 51201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sony.ko 2023-10-11T09:25:59.251 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-speedlink.ko 2023-10-11T09:25:59.252 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-steelseries.ko 2023-10-11T09:25:59.252 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sunplus.ko 2023-10-11T09:25:59.252 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-11T09:25:59.252 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tivo.ko 2023-10-11T09:25:59.252 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tmff.ko 2023-10-11T09:25:59.253 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-topseed.ko 2023-10-11T09:25:59.253 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-twinhan.ko 2023-10-11T09:25:59.253 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 71457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-uclogic.ko 2023-10-11T09:25:59.253 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-waltop.ko 2023-10-11T09:25:59.253 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 143585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-wiimote.ko 2023-10-11T09:25:59.253 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-xinmo.ko 2023-10-11T09:25:59.254 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zpff.ko 2023-10-11T09:25:59.254 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zydacron.ko 2023-10-11T09:25:59.254 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 267689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid.ko 2023-10-11T09:25:59.254 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 32321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/uhid.ko 2023-10-11T09:25:59.254 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/usbhid/ 2023-10-11T09:25:59.255 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 125721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/usbhid/usbhid.ko 2023-10-11T09:25:59.255 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/usbhid/usbkbd.ko 2023-10-11T09:25:59.255 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/usbhid/usbmouse.ko 2023-10-11T09:25:59.255 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 217401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/wacom.ko 2023-10-11T09:25:59.255 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/ 2023-10-11T09:25:59.255 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 63497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_balloon.ko 2023-10-11T09:25:59.256 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 89609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_utils.ko 2023-10-11T09:25:59.256 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 301449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_vmbus.ko 2023-10-11T09:25:59.256 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ 2023-10-11T09:25:59.256 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 66593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru.ko 2023-10-11T09:25:59.256 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 61393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru3.ko 2023-10-11T09:25:59.256 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-11T09:25:59.257 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7414.ko 2023-10-11T09:25:59.257 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7418.ko 2023-10-11T09:25:59.257 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adc128d818.ko 2023-10-11T09:25:59.257 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 31729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1025.ko 2023-10-11T09:25:59.257 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 97977 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1026.ko 2023-10-11T09:25:59.258 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1029.ko 2023-10-11T09:25:59.258 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 45553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1031.ko 2023-10-11T09:25:59.258 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm9240.ko 2023-10-11T09:25:59.258 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ads7828.ko 2023-10-11T09:25:59.258 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7410.ko 2023-10-11T09:25:59.259 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7411.ko 2023-10-11T09:25:59.259 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 74433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7462.ko 2023-10-11T09:25:59.259 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 39585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7470.ko 2023-10-11T09:25:59.259 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 67929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7475.ko 2023-10-11T09:25:59.259 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7x10.ko 2023-10-11T09:25:59.259 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 39073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/amc6821.ko 2023-10-11T09:25:59.260 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 55265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/applesmc.ko 2023-10-11T09:25:59.260 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 49913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asb100.ko 2023-10-11T09:25:59.260 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 58617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asc7621.ko 2023-10-11T09:25:59.260 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 40881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-11T09:25:59.260 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/atxp1.ko 2023-10-11T09:25:59.261 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/coretemp.ko 2023-10-11T09:25:59.261 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/da9052-hwmon.ko 2023-10-11T09:25:59.261 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/da9055-hwmon.ko 2023-10-11T09:25:59.261 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 44465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-11T09:25:59.261 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 99601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dme1737.ko 2023-10-11T09:25:59.261 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds1621.ko 2023-10-11T09:25:59.262 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds620.ko 2023-10-11T09:25:59.262 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29777 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc1403.ko 2023-10-11T09:25:59.262 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc2103.ko 2023-10-11T09:25:59.263 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc6w201.ko 2023-10-11T09:25:59.263 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 66985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71805f.ko 2023-10-11T09:25:59.263 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 94457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71882fg.ko 2023-10-11T09:25:59.263 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 41401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f75375s.ko 2023-10-11T09:25:59.263 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-11T09:25:59.263 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 55433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fschmd.ko 2023-10-11T09:25:59.264 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14481 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g760a.ko 2023-10-11T09:25:59.264 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g762.ko 2023-10-11T09:25:59.264 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 46801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl518sm.ko 2023-10-11T09:25:59.264 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 45529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl520sm.ko 2023-10-11T09:25:59.264 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/hih6130.ko 2023-10-11T09:25:59.264 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-11T09:25:59.265 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-11T09:25:59.265 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-11T09:25:59.265 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmaem.ko 2023-10-11T09:25:59.265 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmpex.ko 2023-10-11T09:25:59.265 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina209.ko 2023-10-11T09:25:59.265 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina2xx.ko 2023-10-11T09:25:59.266 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 129217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/it87.ko 2023-10-11T09:25:59.266 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/jc42.ko 2023-10-11T09:25:59.266 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k10temp.ko 2023-10-11T09:25:59.266 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k8temp.ko 2023-10-11T09:25:59.266 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-11T09:25:59.267 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 48601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm63.ko 2023-10-11T09:25:59.267 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16761 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm73.ko 2023-10-11T09:25:59.267 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36777 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm75.ko 2023-10-11T09:25:59.267 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm77.ko 2023-10-11T09:25:59.268 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm78.ko 2023-10-11T09:25:59.268 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 31745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm80.ko 2023-10-11T09:25:59.268 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm83.ko 2023-10-11T09:25:59.268 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 72833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm85.ko 2023-10-11T09:25:59.268 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 46153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm87.ko 2023-10-11T09:25:59.268 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 37913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm90.ko 2023-10-11T09:25:59.269 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm92.ko 2023-10-11T09:25:59.269 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 108841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm93.ko 2023-10-11T09:25:59.269 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95234.ko 2023-10-11T09:25:59.269 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95241.ko 2023-10-11T09:25:59.269 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95245.ko 2023-10-11T09:25:59.270 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc2945.ko 2023-10-11T09:25:59.270 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4151.ko 2023-10-11T09:25:59.270 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4215.ko 2023-10-11T09:25:59.270 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4222.ko 2023-10-11T09:25:59.270 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4245.ko 2023-10-11T09:25:59.271 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4260.ko 2023-10-11T09:25:59.271 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4261.ko 2023-10-11T09:25:59.271 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max16065.ko 2023-10-11T09:25:59.271 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1619.ko 2023-10-11T09:25:59.271 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1668.ko 2023-10-11T09:25:59.271 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max197.ko 2023-10-11T09:25:59.272 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6639.ko 2023-10-11T09:25:59.272 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6650.ko 2023-10-11T09:25:59.272 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 35777 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6697.ko 2023-10-11T09:25:59.272 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/mc13783-adc.ko 2023-10-11T09:25:59.272 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/mcp3021.ko 2023-10-11T09:25:59.272 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2023-10-11T09:25:59.273 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 44129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6683.ko 2023-10-11T09:25:59.273 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 116697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-11T09:25:59.273 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 60705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775.ko 2023-10-11T09:25:59.273 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 59737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct7802.ko 2023-10-11T09:25:59.273 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 82337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87360.ko 2023-10-11T09:25:59.274 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 54433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87427.ko 2023-10-11T09:25:59.274 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pcf8591.ko 2023-10-11T09:25:59.274 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ 2023-10-11T09:25:59.274 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-11T09:25:59.274 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-11T09:25:59.274 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-11T09:25:59.275 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-11T09:25:59.275 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-11T09:25:59.275 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-11T09:25:59.275 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-11T09:25:59.275 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 81841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-11T09:25:59.276 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-11T09:25:59.276 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-11T09:25:59.276 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-11T09:25:59.276 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-11T09:25:59.276 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/powr1220.ko 2023-10-11T09:25:59.276 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5627.ko 2023-10-11T09:25:59.276 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5636.ko 2023-10-11T09:25:59.277 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-11T09:25:59.277 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 32137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht15.ko 2023-10-11T09:25:59.277 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht21.ko 2023-10-11T09:25:59.277 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/shtc1.ko 2023-10-11T09:25:59.278 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 40105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sis5595.ko 2023-10-11T09:25:59.278 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 31929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smm665.ko 2023-10-11T09:25:59.278 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-11T09:25:59.279 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 39193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-11T09:25:59.279 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 37385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-11T09:25:59.279 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/thmc50.ko 2023-10-11T09:25:59.279 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp102.ko 2023-10-11T09:25:59.279 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp103.ko 2023-10-11T09:25:59.280 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp401.ko 2023-10-11T09:25:59.280 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp421.ko 2023-10-11T09:25:59.280 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-11T09:25:59.280 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 41945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via686a.ko 2023-10-11T09:25:59.280 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 52457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt1211.ko 2023-10-11T09:25:59.281 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 51545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt8231.ko 2023-10-11T09:25:59.281 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 57513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-11T09:25:59.281 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 78873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627hf.ko 2023-10-11T09:25:59.281 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 71345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83781d.ko 2023-10-11T09:25:59.281 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 63289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83791d.ko 2023-10-11T09:25:59.282 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 71321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83792d.ko 2023-10-11T09:25:59.282 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 101457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83793.ko 2023-10-11T09:25:59.282 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 119505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83795.ko 2023-10-11T09:25:59.282 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-11T09:25:59.282 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 35625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-11T09:25:59.283 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/wm831x-hwmon.ko 2023-10-11T09:25:59.283 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/wm8350-hwmon.ko 2023-10-11T09:25:59.283 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/ 2023-10-11T09:25:59.283 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/ 2023-10-11T09:25:59.284 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-11T09:25:59.284 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-11T09:25:59.284 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/ 2023-10-11T09:25:59.284 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ali1535.ko 2023-10-11T09:25:59.284 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ali1563.ko 2023-10-11T09:25:59.284 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27777 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2023-10-11T09:25:59.285 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-11T09:25:59.285 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-11T09:25:59.285 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-11T09:25:59.285 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2023-10-11T09:25:59.285 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 6929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-11T09:25:59.286 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2023-10-11T09:25:59.286 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 48641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-11T09:25:59.286 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-11T09:25:59.286 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-11T09:25:59.286 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-11T09:25:59.287 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-dln2.ko 2023-10-11T09:25:59.287 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-gpio.ko 2023-10-11T09:25:59.287 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 62729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-11T09:25:59.287 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-11T09:25:59.287 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 47889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-11T09:25:59.288 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17369 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-kempld.ko 2023-10-11T09:25:59.288 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-11T09:25:59.288 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-11T09:25:59.288 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ocores.ko 2023-10-11T09:25:59.289 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-11T09:25:59.289 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-11T09:25:59.289 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 47241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-11T09:25:59.289 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2023-10-11T09:25:59.289 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-11T09:25:59.290 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-11T09:25:59.290 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-sis5595.ko 2023-10-11T09:25:59.290 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-sis630.ko 2023-10-11T09:25:59.290 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-11T09:25:59.290 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2023-10-11T09:25:59.291 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-11T09:25:59.291 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-11T09:25:59.291 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-11T09:25:59.291 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-11T09:25:59.292 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 47593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-xiic.ko 2023-10-11T09:25:59.292 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-mux.ko 2023-10-11T09:25:59.292 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-11T09:25:59.292 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-stub.ko 2023-10-11T09:25:59.292 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/muxes/ 2023-10-11T09:25:59.293 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2023-10-11T09:25:59.293 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2023-10-11T09:25:59.293 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2023-10-11T09:25:59.293 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ 2023-10-11T09:25:59.294 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ 2023-10-11T09:25:59.294 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 273641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-11T09:25:59.294 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 876913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-11T09:25:59.294 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 69321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-11T09:25:59.294 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 324145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-11T09:25:59.295 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 97961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-11T09:25:59.295 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 279345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-11T09:25:59.295 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 66177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-11T09:25:59.295 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/ 2023-10-11T09:25:59.295 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4/ 2023-10-11T09:25:59.296 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 492569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-11T09:25:59.296 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4/ 2023-10-11T09:25:59.296 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 419193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-11T09:25:59.296 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5/ 2023-10-11T09:25:59.296 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 739705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-11T09:25:59.296 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca/ 2023-10-11T09:25:59.297 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 296649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-11T09:25:59.297 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/ocrdma/ 2023-10-11T09:25:59.297 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 153681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2023-10-11T09:25:59.297 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/usnic/ 2023-10-11T09:25:59.298 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 169017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2023-10-11T09:25:59.298 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ 2023-10-11T09:25:59.298 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib/ 2023-10-11T09:25:59.298 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 263177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-11T09:25:59.298 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser/ 2023-10-11T09:25:59.299 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 100225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-11T09:25:59.299 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert/ 2023-10-11T09:25:59.299 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 111993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-11T09:25:59.299 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp/ 2023-10-11T09:25:59.300 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 129081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-11T09:25:59.300 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt/ 2023-10-11T09:25:59.300 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 129553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-11T09:25:59.300 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/ 2023-10-11T09:25:59.300 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/evbug.ko 2023-10-11T09:25:59.301 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/ff-memless.ko 2023-10-11T09:25:59.301 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/ 2023-10-11T09:25:59.301 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-11T09:25:59.301 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-11T09:25:59.301 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 41217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/gameport.ko 2023-10-11T09:25:59.302 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/lightning.ko 2023-10-11T09:25:59.302 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/ns558.ko 2023-10-11T09:25:59.302 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 41825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/joydev.ko 2023-10-11T09:25:59.302 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/ 2023-10-11T09:25:59.303 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/adp5520-keys.ko 2023-10-11T09:25:59.303 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/adp5588-keys.ko 2023-10-11T09:25:59.303 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/adp5589-keys.ko 2023-10-11T09:25:59.303 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24761 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2023-10-11T09:25:59.303 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-11T09:25:59.304 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2023-10-11T09:25:59.304 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/lkkbd.ko 2023-10-11T09:25:59.304 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/lm8323.ko 2023-10-11T09:25:59.304 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/lm8333.ko 2023-10-11T09:25:59.305 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/matrix_keypad.ko 2023-10-11T09:25:59.305 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/max7359_keypad.ko 2023-10-11T09:25:59.305 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/mcs_touchkey.ko 2023-10-11T09:25:59.305 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2023-10-11T09:25:59.306 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/newtonkbd.ko 2023-10-11T09:25:59.306 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/opencores-kbd.ko 2023-10-11T09:25:59.306 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/qt1070.ko 2023-10-11T09:25:59.306 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/qt2160.ko 2023-10-11T09:25:59.307 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/samsung-keypad.ko 2023-10-11T09:25:59.307 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11481 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/stowaway.ko 2023-10-11T09:25:59.307 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/sunkbd.ko 2023-10-11T09:25:59.307 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/tca6416-keypad.ko 2023-10-11T09:25:59.309 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/tca8418_keypad.ko 2023-10-11T09:25:59.309 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/twl4030_keypad.ko 2023-10-11T09:25:59.310 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/xtkbd.ko 2023-10-11T09:25:59.310 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/matrix-keymap.ko 2023-10-11T09:25:59.310 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/ 2023-10-11T09:25:59.310 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/appletouch.ko 2023-10-11T09:25:59.310 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-11T09:25:59.311 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 123681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-11T09:25:59.311 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 89449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-11T09:25:59.311 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/gpio_mouse.ko 2023-10-11T09:25:59.311 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 335849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/psmouse.ko 2023-10-11T09:25:59.312 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/sermouse.ko 2023-10-11T09:25:59.312 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-11T09:25:59.312 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-11T09:25:59.312 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-11T09:25:59.313 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4/ 2023-10-11T09:25:59.313 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 109849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-11T09:25:59.313 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/ 2023-10-11T09:25:59.313 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-11T09:25:59.313 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-11T09:25:59.314 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/ct82c710.ko 2023-10-11T09:25:59.314 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-11T09:25:59.314 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/parkbd.ko 2023-10-11T09:25:59.314 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/pcips2.ko 2023-10-11T09:25:59.315 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/ps2mult.ko 2023-10-11T09:25:59.315 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/serio_raw.ko 2023-10-11T09:25:59.315 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17977 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/serport.ko 2023-10-11T09:25:59.315 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/sparse-keymap.ko 2023-10-11T09:25:59.316 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/ 2023-10-11T09:25:59.316 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd/ 2023-10-11T09:25:59.316 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 38857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-11T09:25:59.316 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/ 2023-10-11T09:25:59.316 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-88pm860x.ko 2023-10-11T09:25:59.317 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-adp5520.ko 2023-10-11T09:25:59.317 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 59753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-bd2802.ko 2023-10-11T09:25:59.317 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-blinkm.ko 2023-10-11T09:25:59.317 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-da903x.ko 2023-10-11T09:25:59.318 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-da9052.ko 2023-10-11T09:25:59.318 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16561 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-gpio.ko 2023-10-11T09:25:59.318 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm3530.ko 2023-10-11T09:25:59.318 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm3533.ko 2023-10-11T09:25:59.318 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm355x.ko 2023-10-11T09:25:59.319 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm3642.ko 2023-10-11T09:25:59.319 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lp3944.ko 2023-10-11T09:25:59.319 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lp8788.ko 2023-10-11T09:25:59.319 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-max8997.ko 2023-10-11T09:25:59.320 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-mc13783.ko 2023-10-11T09:25:59.320 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-menf21bmc.ko 2023-10-11T09:25:59.320 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-pca9532.ko 2023-10-11T09:25:59.320 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20481 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-pca955x.ko 2023-10-11T09:25:59.321 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-pca963x.ko 2023-10-11T09:25:59.321 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-ss4200.ko 2023-10-11T09:25:59.321 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-tca6507.ko 2023-10-11T09:25:59.321 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-wm831x-status.ko 2023-10-11T09:25:59.321 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-wm8350.ko 2023-10-11T09:25:59.322 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ 2023-10-11T09:25:59.322 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-11T09:25:59.322 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-11T09:25:59.322 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-11T09:25:59.323 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-11T09:25:59.323 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-11T09:25:59.323 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-11T09:25:59.323 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-11T09:25:59.323 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-11T09:25:59.324 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/ 2023-10-11T09:25:59.324 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache/ 2023-10-11T09:25:59.324 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 655473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache/bcache.ko 2023-10-11T09:25:59.324 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 40569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-bio-prison.ko 2023-10-11T09:25:59.324 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 89601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-bufio.ko 2023-10-11T09:25:59.325 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 47529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache-smq.ko 2023-10-11T09:25:59.325 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 155993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache.ko 2023-10-11T09:25:59.325 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 107049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-crypt.ko 2023-10-11T09:25:59.325 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-delay.ko 2023-10-11T09:25:59.326 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 53897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-era.ko 2023-10-11T09:25:59.326 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-flakey.ko 2023-10-11T09:25:59.326 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 41593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-log-userspace.ko 2023-10-11T09:25:59.326 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 32721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-log.ko 2023-10-11T09:25:59.327 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 49545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-mirror.ko 2023-10-11T09:25:59.327 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 78017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-multipath.ko 2023-10-11T09:25:59.327 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-queue-length.ko 2023-10-11T09:25:59.327 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 70953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-raid.ko 2023-10-11T09:25:59.328 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-region-hash.ko 2023-10-11T09:25:59.328 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-round-robin.ko 2023-10-11T09:25:59.328 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-service-time.ko 2023-10-11T09:25:59.328 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 109593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-snapshot.ko 2023-10-11T09:25:59.329 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-switch.ko 2023-10-11T09:25:59.329 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 181585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-thin-pool.ko 2023-10-11T09:25:59.329 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 51737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-verity.ko 2023-10-11T09:25:59.329 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-zero.ko 2023-10-11T09:25:59.329 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/faulty.ko 2023-10-11T09:25:59.330 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/linear.ko 2023-10-11T09:25:59.330 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 32305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/multipath.ko 2023-10-11T09:25:59.330 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data/ 2023-10-11T09:25:59.330 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 199937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-11T09:25:59.331 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 37321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid0.ko 2023-10-11T09:25:59.331 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 112609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid1.ko 2023-10-11T09:25:59.331 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 160553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid10.ko 2023-10-11T09:25:59.331 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 396153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid456.ko 2023-10-11T09:25:59.331 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/ 2023-10-11T09:25:59.332 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/ 2023-10-11T09:25:59.332 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 178113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptbase.ko 2023-10-11T09:25:59.332 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 63385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptctl.ko 2023-10-11T09:25:59.332 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 44649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptfc.ko 2023-10-11T09:25:59.332 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 37161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptlan.ko 2023-10-11T09:25:59.333 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 123353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptsas.ko 2023-10-11T09:25:59.333 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 79017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-11T09:25:59.333 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 53081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptspi.ko 2023-10-11T09:25:59.333 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/ 2023-10-11T09:25:59.333 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/88pm800.ko 2023-10-11T09:25:59.334 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12761 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/88pm805.ko 2023-10-11T09:25:59.334 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/88pm80x.ko 2023-10-11T09:25:59.334 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/arizona-i2c.ko 2023-10-11T09:25:59.334 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 123721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/arizona.ko 2023-10-11T09:25:59.335 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/bcm590xx.ko 2023-10-11T09:25:59.335 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/cros_ec_dev.ko 2023-10-11T09:25:59.335 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/dln2.ko 2023-10-11T09:25:59.335 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/janz-cmodio.ko 2023-10-11T09:25:59.335 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 53585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/kempld-core.ko 2023-10-11T09:25:59.336 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lm3533-core.ko 2023-10-11T09:25:59.336 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lm3533-ctrlbank.ko 2023-10-11T09:25:59.336 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lp3943.ko 2023-10-11T09:25:59.336 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 54569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_ich.ko 2023-10-11T09:25:59.336 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_sch.ko 2023-10-11T09:25:59.337 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/max8907.ko 2023-10-11T09:25:59.337 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/mc13xxx-core.ko 2023-10-11T09:25:59.337 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/mc13xxx-i2c.ko 2023-10-11T09:25:59.337 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/menf21bmc.ko 2023-10-11T09:25:59.337 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/pcf50633-adc.ko 2023-10-11T09:25:59.338 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/pcf50633-gpio.ko 2023-10-11T09:25:59.338 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/pcf50633.ko 2023-10-11T09:25:59.338 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/rdc321x-southbridge.ko 2023-10-11T09:25:59.338 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/retu-mfd.ko 2023-10-11T09:25:59.339 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 80425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/si476x-core.ko 2023-10-11T09:25:59.339 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 48441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/sm501.ko 2023-10-11T09:25:59.339 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/ti_am335x_tscadc.ko 2023-10-11T09:25:59.339 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 51137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/tps65010.ko 2023-10-11T09:25:59.339 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/tps6507x.ko 2023-10-11T09:25:59.340 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/viperboard.ko 2023-10-11T09:25:59.340 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/vx855.ko 2023-10-11T09:25:59.340 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/wl1273-core.ko 2023-10-11T09:25:59.340 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 44345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/wm8994.ko 2023-10-11T09:25:59.341 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/ 2023-10-11T09:25:59.341 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/ad525x_dpot-i2c.ko 2023-10-11T09:25:59.341 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 48489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/ad525x_dpot.ko 2023-10-11T09:25:59.341 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl/ 2023-10-11T09:25:59.341 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 46585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-11T09:25:59.342 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds9802als.ko 2023-10-11T09:25:59.342 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds990x.ko 2023-10-11T09:25:59.342 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 45593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/bh1770glc.ko 2023-10-11T09:25:59.342 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/c2port/ 2023-10-11T09:25:59.342 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2023-10-11T09:25:59.343 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 35305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/c2port/core.ko 2023-10-11T09:25:59.343 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710/ 2023-10-11T09:25:59.343 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710/cb710.ko 2023-10-11T09:25:59.343 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/ds1682.ko 2023-10-11T09:25:59.343 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/dummy-irq.ko 2023-10-11T09:25:59.344 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/echo/ 2023-10-11T09:25:59.344 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/echo/echo.ko 2023-10-11T09:25:59.344 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/ 2023-10-11T09:25:59.344 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 37849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/at24.ko 2023-10-11T09:25:59.345 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-11T09:25:59.345 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-11T09:25:59.345 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-11T09:25:59.345 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 32993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/enclosure.ko 2023-10-11T09:25:59.345 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/genwqe/ 2023-10-11T09:25:59.346 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 152137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/genwqe/genwqe_card.ko 2023-10-11T09:25:59.346 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/hmc6352.ko 2023-10-11T09:25:59.346 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 31193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/hpilo.ko 2023-10-11T09:25:59.346 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/ibmasm/ 2023-10-11T09:25:59.346 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 72601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/ibmasm/ibmasm.ko 2023-10-11T09:25:59.347 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/ics932s401.ko 2023-10-11T09:25:59.347 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29003.ko 2023-10-11T09:25:59.347 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29020.ko 2023-10-11T09:25:59.347 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/ 2023-10-11T09:25:59.348 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 42585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-11T09:25:59.348 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-11T09:25:59.348 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/ 2023-10-11T09:25:59.348 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 125209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-me.ko 2023-10-11T09:25:59.348 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 56521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-11T09:25:59.349 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 309857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei.ko 2023-10-11T09:25:59.349 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 31329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/phantom.ko 2023-10-11T09:25:59.349 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/ti-st/ 2023-10-11T09:25:59.349 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 74905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/ti-st/st_drv.ko 2023-10-11T09:25:59.349 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-11T09:25:59.350 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_core.ko 2023-10-11T09:25:59.350 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tsl2550.ko 2023-10-11T09:25:59.350 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 48673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_balloon.ko 2023-10-11T09:25:59.350 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci/ 2023-10-11T09:25:59.350 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 183857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-11T09:25:59.351 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ 2023-10-11T09:25:59.351 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding/ 2023-10-11T09:25:59.351 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 487865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding/bonding.ko 2023-10-11T09:25:59.352 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/ 2023-10-11T09:25:59.352 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53/ 2023-10-11T09:25:59.352 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 106009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-11T09:25:59.352 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 40281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-11T09:25:59.352 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-11T09:25:59.352 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx/ 2023-10-11T09:25:59.353 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 452681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-11T09:25:59.461 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dummy.ko 2023-10-11T09:25:59.462 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/eql.ko 2023-10-11T09:25:59.462 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ 2023-10-11T09:25:59.462 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/ 2023-10-11T09:25:59.462 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 45785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-11T09:25:59.463 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 42105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-11T09:25:59.463 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 130489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-11T09:25:59.463 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 68785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-11T09:25:59.463 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/ 2023-10-11T09:25:59.463 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 37649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-11T09:25:59.464 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 59689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-11T09:25:59.464 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-11T09:25:59.464 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 99113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-11T09:25:59.464 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec/ 2023-10-11T09:25:59.464 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 54521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-11T09:25:59.465 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere/ 2023-10-11T09:25:59.465 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 69145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-11T09:25:59.465 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon/ 2023-10-11T09:25:59.465 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 59521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-11T09:25:59.465 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera/ 2023-10-11T09:25:59.465 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 41393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-11T09:25:59.466 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/ 2023-10-11T09:25:59.466 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 54545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-11T09:25:59.466 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 49225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-11T09:25:59.466 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 87345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-11T09:25:59.466 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/ 2023-10-11T09:25:59.467 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx/ 2023-10-11T09:25:59.467 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 97321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-11T09:25:59.467 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c/ 2023-10-11T09:25:59.467 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 104609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-11T09:25:59.467 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e/ 2023-10-11T09:25:59.468 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 90257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-11T09:25:59.468 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/ 2023-10-11T09:25:59.468 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 81505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-11T09:25:59.468 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 69713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-11T09:25:59.469 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/ 2023-10-11T09:25:59.469 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 78401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-11T09:25:59.469 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 176881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-11T09:25:59.469 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2023-10-11T09:25:59.469 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 1422209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-11T09:25:59.470 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 128377 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-11T09:25:59.470 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet/ 2023-10-11T09:25:59.470 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 132289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-11T09:25:59.470 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 312145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-11T09:25:59.470 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/ 2023-10-11T09:25:59.471 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna/ 2023-10-11T09:25:59.471 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 323553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-11T09:25:59.471 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/ 2023-10-11T09:25:59.471 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb/ 2023-10-11T09:25:59.471 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 193433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-11T09:25:59.472 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2023-10-11T09:25:59.472 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 347641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-11T09:25:59.472 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2023-10-11T09:25:59.472 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 1044145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-11T09:25:59.472 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2023-10-11T09:25:59.473 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 166113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-11T09:25:59.473 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2023-10-11T09:25:59.473 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-11T09:25:59.473 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/ 2023-10-11T09:25:59.474 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic/ 2023-10-11T09:25:59.474 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 204329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-11T09:25:59.474 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/ 2023-10-11T09:25:59.474 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/ 2023-10-11T09:25:59.474 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 69449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-11T09:25:59.475 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 61489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-11T09:25:59.475 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 141161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-11T09:25:59.475 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 51433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-11T09:25:59.475 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 62865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-11T09:25:59.476 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 31305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-11T09:25:59.476 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/ 2023-10-11T09:25:59.476 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 60873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-11T09:25:59.476 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 62833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-11T09:25:59.476 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 45721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dnet.ko 2023-10-11T09:25:59.477 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ec_bhf.ko 2023-10-11T09:25:59.477 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/ 2023-10-11T09:25:59.477 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet/ 2023-10-11T09:25:59.477 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 281329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-11T09:25:59.477 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 49505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-11T09:25:59.478 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 51737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-11T09:25:59.478 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/fujitsu/ 2023-10-11T09:25:59.478 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 45281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2023-10-11T09:25:59.478 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ 2023-10-11T09:25:59.478 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 90465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-11T09:25:59.479 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000/ 2023-10-11T09:25:59.479 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 295713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-11T09:25:59.479 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e/ 2023-10-11T09:25:59.479 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 584249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-11T09:25:59.479 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k/ 2023-10-11T09:25:59.480 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 246321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-11T09:25:59.480 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e/ 2023-10-11T09:25:59.480 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 924017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-11T09:25:59.480 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf/ 2023-10-11T09:25:59.480 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 302577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-11T09:25:59.480 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb/ 2023-10-11T09:25:59.481 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 514889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-11T09:25:59.481 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf/ 2023-10-11T09:25:59.481 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 93593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-11T09:25:59.481 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe/ 2023-10-11T09:25:59.481 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 734297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-11T09:25:59.482 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf/ 2023-10-11T09:25:59.482 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 151841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-11T09:25:59.482 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 86921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/jme.ko 2023-10-11T09:25:59.482 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/ 2023-10-11T09:25:59.482 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20977 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-11T09:25:59.483 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 102385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-11T09:25:59.483 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 122681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-11T09:25:59.483 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/ 2023-10-11T09:25:59.483 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/ 2023-10-11T09:25:59.483 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 727433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-11T09:25:59.483 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 280513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-11T09:25:59.484 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/ 2023-10-11T09:25:59.484 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2023-10-11T09:25:59.484 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 1097537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-11T09:25:59.484 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ 2023-10-11T09:25:59.484 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 53041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ks8842.ko 2023-10-11T09:25:59.485 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 40993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2023-10-11T09:25:59.485 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2023-10-11T09:25:59.485 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 95665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-11T09:25:59.485 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/ 2023-10-11T09:25:59.485 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge/ 2023-10-11T09:25:59.486 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 108809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-11T09:25:59.486 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/ 2023-10-11T09:25:59.486 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 73377 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-11T09:25:59.486 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 47713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-11T09:25:59.486 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion/ 2023-10-11T09:25:59.487 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 150905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-11T09:25:59.487 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia/ 2023-10-11T09:25:59.487 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 136785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-11T09:25:59.487 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/ 2023-10-11T09:25:59.488 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 50481 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-11T09:25:59.488 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 43145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-11T09:25:59.488 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/ 2023-10-11T09:25:59.488 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen/ 2023-10-11T09:25:59.489 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 227113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-11T09:25:59.489 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 87081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-11T09:25:59.489 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2023-10-11T09:25:59.489 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 671393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-11T09:25:59.489 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc/ 2023-10-11T09:25:59.489 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 43017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-11T09:25:59.490 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/ 2023-10-11T09:25:59.490 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 67385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-11T09:25:59.490 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 83921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-11T09:25:59.490 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-11T09:25:59.491 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 189817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-11T09:25:59.491 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/samsung/ 2023-10-11T09:25:59.491 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/samsung/sxgbe/ 2023-10-11T09:25:59.491 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 104241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2023-10-11T09:25:59.491 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc/ 2023-10-11T09:25:59.492 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 952929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-11T09:25:59.492 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan/ 2023-10-11T09:25:59.492 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 47177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-11T09:25:59.492 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/ 2023-10-11T09:25:59.492 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 54809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-11T09:25:59.493 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 63273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-11T09:25:59.493 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/ 2023-10-11T09:25:59.493 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 61537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-11T09:25:59.493 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 67129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-11T09:25:59.493 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 73345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-11T09:25:59.494 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 68209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-11T09:25:59.494 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/ 2023-10-11T09:25:59.494 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/ 2023-10-11T09:25:59.494 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2023-10-11T09:25:59.494 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 40137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-11T09:25:59.495 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2023-10-11T09:25:59.495 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 481681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-11T09:25:59.495 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/ 2023-10-11T09:25:59.495 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 103321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-11T09:25:59.496 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 166121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-11T09:25:59.496 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 66473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-11T09:25:59.496 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 78737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-11T09:25:59.496 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti/ 2023-10-11T09:25:59.496 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 49993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-11T09:25:59.496 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti/ 2023-10-11T09:25:59.497 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 86465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-11T09:25:59.497 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/ 2023-10-11T09:25:59.497 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 80257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-11T09:25:59.497 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 83681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-11T09:25:59.498 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/ 2023-10-11T09:25:59.498 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 46241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-11T09:25:59.498 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-11T09:25:59.498 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom/ 2023-10-11T09:25:59.499 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 60041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-11T09:25:59.499 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/fddi/ 2023-10-11T09:25:59.499 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 41001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/fddi/defxx.ko 2023-10-11T09:25:59.499 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/fddi/skfp/ 2023-10-11T09:25:59.499 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 216977 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/fddi/skfp/skfp.ko 2023-10-11T09:25:59.500 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 77401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/geneve.ko 2023-10-11T09:25:59.500 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv/ 2023-10-11T09:25:59.500 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 235593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-11T09:25:59.500 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154/ 2023-10-11T09:25:59.500 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-11T09:25:59.501 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ifb.ko 2023-10-11T09:25:59.501 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan/ 2023-10-11T09:25:59.501 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 83993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-11T09:25:59.501 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 62065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvlan.ko 2023-10-11T09:25:59.501 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvtap.ko 2023-10-11T09:25:59.502 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/ 2023-10-11T09:25:59.502 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-11T09:25:59.502 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-11T09:25:59.502 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-gpio.ko 2023-10-11T09:25:59.502 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-11T09:25:59.503 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio.ko 2023-10-11T09:25:59.503 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mii.ko 2023-10-11T09:25:59.503 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 41697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/netconsole.ko 2023-10-11T09:25:59.503 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/nlmon.ko 2023-10-11T09:25:59.503 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/ 2023-10-11T09:25:59.503 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-11T09:25:59.504 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 32329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-11T09:25:59.504 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/ 2023-10-11T09:25:59.504 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/amd.ko 2023-10-11T09:25:59.504 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 50097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-11T09:25:59.504 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 43273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-11T09:25:59.505 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-11T09:25:59.505 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 41793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/broadcom.ko 2023-10-11T09:25:59.505 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10561 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/cicada.ko 2023-10-11T09:25:59.505 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/davicom.ko 2023-10-11T09:25:59.505 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/et1011c.ko 2023-10-11T09:25:59.506 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/icplus.ko 2023-10-11T09:25:59.506 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/lxt.ko 2023-10-11T09:25:59.506 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 78185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/marvell.ko 2023-10-11T09:25:59.506 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 75713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/micrel.ko 2023-10-11T09:25:59.506 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/national.ko 2023-10-11T09:25:59.506 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 97265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/phylink.ko 2023-10-11T09:25:59.507 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/qsemi.ko 2023-10-11T09:25:59.507 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 49553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/realtek.ko 2023-10-11T09:25:59.507 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/smsc.ko 2023-10-11T09:25:59.507 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-11T09:25:59.507 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/vitesse.ko 2023-10-11T09:25:59.508 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/plip/ 2023-10-11T09:25:59.508 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 43465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/plip/plip.ko 2023-10-11T09:25:59.508 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ 2023-10-11T09:25:59.508 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-11T09:25:59.509 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-11T09:25:59.509 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-11T09:25:59.509 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-11T09:25:59.509 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-11T09:25:59.509 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 42545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppoe.ko 2023-10-11T09:25:59.510 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppox.ko 2023-10-11T09:25:59.510 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 31969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pptp.ko 2023-10-11T09:25:59.510 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/sb1000.ko 2023-10-11T09:25:59.510 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/ 2023-10-11T09:25:59.510 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 42513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slip.ko 2023-10-11T09:25:59.513 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/sungem_phy.ko 2023-10-11T09:25:59.514 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 64937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tap.ko 2023-10-11T09:25:59.514 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/ 2023-10-11T09:25:59.514 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 106025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team.ko 2023-10-11T09:25:59.514 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-11T09:25:59.515 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-11T09:25:59.515 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-11T09:25:59.515 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_random.ko 2023-10-11T09:25:59.515 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-11T09:25:59.516 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 88257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/veth.ko 2023-10-11T09:25:59.516 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3/ 2023-10-11T09:25:59.516 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 120497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-11T09:25:59.516 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan/ 2023-10-11T09:25:59.516 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 293441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-11T09:25:59.517 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback/ 2023-10-11T09:25:59.517 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 143889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-11T09:25:59.517 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb/ 2023-10-11T09:25:59.517 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb/ntb.ko 2023-10-11T09:25:59.518 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/ 2023-10-11T09:25:59.518 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/ 2023-10-11T09:25:59.518 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 264585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-11T09:25:59.518 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 102393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme.ko 2023-10-11T09:25:59.519 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/ 2023-10-11T09:25:59.519 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 133817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport.ko 2023-10-11T09:25:59.519 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14561 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_cs.ko 2023-10-11T09:25:59.519 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 84337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_pc.ko 2023-10-11T09:25:59.519 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_serial.ko 2023-10-11T09:25:59.520 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/ 2023-10-11T09:25:59.520 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/ 2023-10-11T09:25:59.520 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-11T09:25:59.520 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2023-10-11T09:25:59.521 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2023-10-11T09:25:59.521 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pci-stub.ko 2023-10-11T09:25:59.521 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 47377 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/xen-pcifront.ko 2023-10-11T09:25:59.521 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/ 2023-10-11T09:25:59.521 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/i82092.ko 2023-10-11T09:25:59.522 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 145785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pcmcia.ko 2023-10-11T09:25:59.522 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 62081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pcmcia_core.ko 2023-10-11T09:25:59.522 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2023-10-11T09:25:59.522 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pd6729.ko 2023-10-11T09:25:59.523 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 112913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-11T09:25:59.523 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/phy/ 2023-10-11T09:25:59.523 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/phy/broadcom/ 2023-10-11T09:25:59.523 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2023-10-11T09:25:59.524 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/phy/samsung/ 2023-10-11T09:25:59.524 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2023-10-11T09:25:59.524 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/ 2023-10-11T09:25:59.524 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel/ 2023-10-11T09:25:59.524 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 71521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-11T09:25:59.525 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/ 2023-10-11T09:25:59.525 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/chrome/ 2023-10-11T09:25:59.525 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 50889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2023-10-11T09:25:59.525 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/chrome/cros_ec.ko 2023-10-11T09:25:59.525 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2023-10-11T09:25:59.526 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2023-10-11T09:25:59.526 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2023-10-11T09:25:59.526 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2023-10-11T09:25:59.526 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2023-10-11T09:25:59.526 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/ 2023-10-11T09:25:59.527 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 67097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-11T09:25:59.527 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 39145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-11T09:25:59.527 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-11T09:25:59.527 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 59665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-11T09:25:59.527 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-11T09:25:59.528 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 106873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-11T09:25:59.528 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-11T09:25:59.528 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 49121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-11T09:25:59.528 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-11T09:25:59.528 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 40265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-11T09:25:59.529 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-11T09:25:59.529 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/hdaps.ko 2023-10-11T09:25:59.529 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/ibm_rtl.ko 2023-10-11T09:25:59.529 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel/ 2023-10-11T09:25:59.530 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-11T09:25:59.530 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2023-10-11T09:25:59.530 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 43521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-11T09:25:59.530 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-11T09:25:59.530 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10377 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-11T09:25:59.531 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-11T09:25:59.531 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 42329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-11T09:25:59.531 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-11T09:25:59.531 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-11T09:25:59.531 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-11T09:25:59.532 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-11T09:25:59.532 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-11T09:25:59.532 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 57545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi.ko 2023-10-11T09:25:59.532 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/ 2023-10-11T09:25:59.532 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/ 2023-10-11T09:25:59.533 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/88pm860x_battery.ko 2023-10-11T09:25:59.533 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/88pm860x_charger.ko 2023-10-11T09:25:59.533 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 55057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/bq2415x_charger.ko 2023-10-11T09:25:59.533 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 53993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/bq24190_charger.ko 2023-10-11T09:25:59.533 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/bq24735-charger.ko 2023-10-11T09:25:59.534 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/cros_peripheral_charger.ko 2023-10-11T09:25:59.534 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/da9030_battery.ko 2023-10-11T09:25:59.534 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/da9052-battery.ko 2023-10-11T09:25:59.534 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/ds2760_battery.ko 2023-10-11T09:25:59.535 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/ds2780_battery.ko 2023-10-11T09:25:59.535 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/ds2781_battery.ko 2023-10-11T09:25:59.535 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/ds2782_battery.ko 2023-10-11T09:25:59.535 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/gpio-charger.ko 2023-10-11T09:25:59.535 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/isp1704_charger.ko 2023-10-11T09:25:59.536 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/lp8727_charger.ko 2023-10-11T09:25:59.536 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/max14577_charger.ko 2023-10-11T09:25:59.536 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/max17040_battery.ko 2023-10-11T09:25:59.537 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/max17042_battery.ko 2023-10-11T09:25:59.537 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/max8903_charger.ko 2023-10-11T09:25:59.537 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/max8925_power.ko 2023-10-11T09:25:59.537 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/pcf50633-charger.ko 2023-10-11T09:25:59.537 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 38009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/sbs-battery.ko 2023-10-11T09:25:59.538 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/test_power.ko 2023-10-11T09:25:59.538 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/tps65090-charger.ko 2023-10-11T09:25:59.538 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/wm831x_backup.ko 2023-10-11T09:25:59.538 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/wm831x_power.ko 2023-10-11T09:25:59.539 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/wm8350_power.ko 2023-10-11T09:25:59.539 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/ 2023-10-11T09:25:59.539 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 57737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-11T09:25:59.539 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-11T09:25:59.540 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/ 2023-10-11T09:25:59.540 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/ 2023-10-11T09:25:59.540 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-11T09:25:59.540 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-11T09:25:59.541 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-11T09:25:59.541 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 41921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/pps_core.ko 2023-10-11T09:25:59.541 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ 2023-10-11T09:25:59.541 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 66385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp.ko 2023-10-11T09:25:59.541 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-11T09:25:59.542 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/ 2023-10-11T09:25:59.542 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-88pm80x.ko 2023-10-11T09:25:59.542 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-88pm860x.ko 2023-10-11T09:25:59.542 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-11T09:25:59.543 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq4802.ko 2023-10-11T09:25:59.543 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-da9052.ko 2023-10-11T09:25:59.543 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-da9055.ko 2023-10-11T09:25:59.543 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-da9063.ko 2023-10-11T09:25:59.543 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-11T09:25:59.543 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 68577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-11T09:25:59.544 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-11T09:25:59.544 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-11T09:25:59.544 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-11T09:25:59.544 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-11T09:25:59.544 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13369 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-11T09:25:59.545 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-11T09:25:59.545 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-11T09:25:59.545 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-11T09:25:59.545 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-11T09:25:59.545 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-11T09:25:59.546 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-11T09:25:59.546 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-lp8788.ko 2023-10-11T09:25:59.546 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 38097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-11T09:25:59.546 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-11T09:25:59.547 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-11T09:25:59.547 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t86.ko 2023-10-11T09:25:59.547 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11633 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-11T09:25:59.547 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max8907.ko 2023-10-11T09:25:59.547 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max8925.ko 2023-10-11T09:25:59.548 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max8997.ko 2023-10-11T09:25:59.548 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max8998.ko 2023-10-11T09:25:59.548 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-mc13xxx.ko 2023-10-11T09:25:59.548 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-11T09:25:59.548 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-palmas.ko 2023-10-11T09:25:59.549 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 40121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-11T09:25:59.549 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17977 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf50633.ko 2023-10-11T09:25:59.549 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-11T09:25:59.549 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-11T09:25:59.549 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-11T09:25:59.550 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-11T09:25:59.550 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rc5t583.ko 2023-10-11T09:25:59.550 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-11T09:25:59.550 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-11T09:25:59.550 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-11T09:25:59.550 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-11T09:25:59.551 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-11T09:25:59.551 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-s35390a.ko 2023-10-11T09:25:59.551 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-11T09:25:59.551 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-tps6586x.ko 2023-10-11T09:25:59.551 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-tps65910.ko 2023-10-11T09:25:59.552 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-wm831x.ko 2023-10-11T09:25:59.552 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-wm8350.ko 2023-10-11T09:25:59.552 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-11T09:25:59.552 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ 2023-10-11T09:25:59.552 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 77617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-11T09:25:59.553 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 57665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-sas.ko 2023-10-11T09:25:59.553 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 55169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-11T09:25:59.553 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 135169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/BusLogic.ko 2023-10-11T09:25:59.553 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/a100u2w.ko 2023-10-11T09:25:59.553 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid/ 2023-10-11T09:25:59.554 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 272353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-11T09:25:59.554 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 100617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/advansys.ko 2023-10-11T09:25:59.554 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/ 2023-10-11T09:25:59.554 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 356265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-11T09:25:59.554 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 295801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-11T09:25:59.555 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic94xx/ 2023-10-11T09:25:59.555 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 189321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic94xx/aic94xx.ko 2023-10-11T09:25:59.555 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/am53c974.ko 2023-10-11T09:25:59.555 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr/ 2023-10-11T09:25:59.555 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 106569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-11T09:25:59.556 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 38185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/atp870u.ko 2023-10-11T09:25:59.556 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi/ 2023-10-11T09:25:59.556 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 246897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-11T09:25:59.556 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa/ 2023-10-11T09:25:59.556 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 1172185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-11T09:25:59.557 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc/ 2023-10-11T09:25:59.557 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 237169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-11T09:25:59.557 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i/ 2023-10-11T09:25:59.557 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 115393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-11T09:25:59.557 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 40609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ch.ko 2023-10-11T09:25:59.558 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor/ 2023-10-11T09:25:59.558 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 308889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-11T09:25:59.558 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/ 2023-10-11T09:25:59.558 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i/ 2023-10-11T09:25:59.558 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 76193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-11T09:25:59.559 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i/ 2023-10-11T09:25:59.559 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 101449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-11T09:25:59.559 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 146785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-11T09:25:59.559 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 75529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dc395x.ko 2023-10-11T09:25:59.560 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 38121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dmx3191d.ko 2023-10-11T09:25:59.560 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r/ 2023-10-11T09:25:59.560 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 212401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-11T09:25:59.560 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 60489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esp_scsi.ko 2023-10-11T09:25:59.560 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/ 2023-10-11T09:25:59.561 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 78289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-11T09:25:59.561 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 158193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-11T09:25:59.561 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fdomain.ko 2023-10-11T09:25:59.561 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic/ 2023-10-11T09:25:59.561 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 262729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-11T09:25:59.561 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 201793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hpsa.ko 2023-10-11T09:25:59.562 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 51145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hptiop.ko 2023-10-11T09:25:59.562 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 49041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-11T09:25:59.562 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 35321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/imm.ko 2023-10-11T09:25:59.562 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 44057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/initio.ko 2023-10-11T09:25:59.562 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 266521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ipr.ko 2023-10-11T09:25:59.563 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 101881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ips.ko 2023-10-11T09:25:59.563 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci/ 2023-10-11T09:25:59.563 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 317249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci/isci.ko 2023-10-11T09:25:59.563 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-11T09:25:59.563 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 52897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-11T09:25:59.564 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc/ 2023-10-11T09:25:59.564 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 314961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-11T09:25:59.564 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 145569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi.ko 2023-10-11T09:25:59.564 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 53521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-11T09:25:59.564 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas/ 2023-10-11T09:25:59.565 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 217209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-11T09:25:59.565 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc/ 2023-10-11T09:25:59.565 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 2192585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-11T09:25:59.565 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/ 2023-10-11T09:25:59.565 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 73545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-11T09:25:59.565 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-11T09:25:59.566 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 310729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-11T09:25:59.566 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 84697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid.ko 2023-10-11T09:25:59.566 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas/ 2023-10-11T09:25:59.566 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 659513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-11T09:25:59.566 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas/ 2023-10-11T09:25:59.567 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 119929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-11T09:25:59.567 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 78153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvumi.ko 2023-10-11T09:25:59.567 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pcmcia/ 2023-10-11T09:25:59.567 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 81601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2023-10-11T09:25:59.567 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2023-10-11T09:25:59.568 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2023-10-11T09:25:59.568 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2023-10-11T09:25:59.568 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001/ 2023-10-11T09:25:59.568 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 364257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-11T09:25:59.568 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 123489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pmcraid.ko 2023-10-11T09:25:59.569 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ppa.ko 2023-10-11T09:25:59.569 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 55345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla1280.ko 2023-10-11T09:25:59.569 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/ 2023-10-11T09:25:59.569 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 1860193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-11T09:25:59.569 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 85313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-11T09:25:59.570 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx/ 2023-10-11T09:25:59.570 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 558865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-11T09:25:59.570 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qlogicfas408.ko 2023-10-11T09:25:59.570 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/raid_class.ko 2023-10-11T09:25:59.570 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 238801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_debug.ko 2023-10-11T09:25:59.571 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 173921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-11T09:25:59.571 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 278865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-11T09:25:59.571 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 97705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-11T09:25:59.571 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 73881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-11T09:25:59.571 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 44993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-11T09:25:59.571 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ses.ko 2023-10-11T09:25:59.572 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 115849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/st.ko 2023-10-11T09:25:59.572 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 46825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/stex.ko 2023-10-11T09:25:59.572 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2/ 2023-10-11T09:25:59.572 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 162121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-11T09:25:59.572 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 42361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-11T09:25:59.573 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 50297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-11T09:25:59.573 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 40001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/wd719x.ko 2023-10-11T09:25:59.573 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 50025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/xen-scsifront.ko 2023-10-11T09:25:59.573 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb/ 2023-10-11T09:25:59.573 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 129417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb/ssb.ko 2023-10-11T09:25:59.574 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/ 2023-10-11T09:25:59.574 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi/ 2023-10-11T09:25:59.574 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 739089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-11T09:25:59.574 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback/ 2023-10-11T09:25:59.575 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 49985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-11T09:25:59.575 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 39713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_file.ko 2023-10-11T09:25:59.575 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 42961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_iblock.ko 2023-10-11T09:25:59.575 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 931721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_mod.ko 2023-10-11T09:25:59.575 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 44321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_pscsi.ko 2023-10-11T09:25:59.575 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 122545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_user.ko 2023-10-11T09:25:59.576 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc/ 2023-10-11T09:25:59.576 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 75641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-11T09:25:59.576 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/ 2023-10-11T09:25:59.576 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/ 2023-10-11T09:25:59.577 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/ 2023-10-11T09:25:59.577 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23481 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-11T09:25:59.577 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-11T09:25:59.577 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-11T09:25:59.577 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-11T09:25:59.578 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-11T09:25:59.578 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-11T09:25:59.578 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-11T09:25:59.578 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-11T09:25:59.578 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-11T09:25:59.579 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-11T09:25:59.579 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-11T09:25:59.579 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 37065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-11T09:25:59.579 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_powerclamp.ko 2023-10-11T09:25:59.579 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-11T09:25:59.580 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-11T09:25:59.580 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-11T09:25:59.580 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ 2023-10-11T09:25:59.580 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless/ 2023-10-11T09:25:59.580 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 78073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-11T09:25:59.580 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 57881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/moxa.ko 2023-10-11T09:25:59.581 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 50793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/mxser.ko 2023-10-11T09:25:59.581 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 107545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_gsm.ko 2023-10-11T09:25:59.581 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 32041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_hdlc.ko 2023-10-11T09:25:59.581 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 68137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/nozomi.ko 2023-10-11T09:25:59.581 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/ 2023-10-11T09:25:59.582 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250/ 2023-10-11T09:25:59.582 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 35745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250/8250_dw.ko 2023-10-11T09:25:59.582 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 59497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-11T09:25:59.582 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/altera_jtaguart.ko 2023-10-11T09:25:59.582 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/altera_uart.ko 2023-10-11T09:25:59.583 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-11T09:25:59.583 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 72193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/fsl_lpuart.ko 2023-10-11T09:25:59.583 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm/ 2023-10-11T09:25:59.583 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 103017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-11T09:25:59.583 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/rp2.ko 2023-10-11T09:25:59.584 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 52073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/sc16is7xx.ko 2023-10-11T09:25:59.584 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 100185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/synclink_gt.ko 2023-10-11T09:25:59.584 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/ 2023-10-11T09:25:59.584 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core/ 2023-10-11T09:25:59.584 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 536809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-11T09:25:59.585 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host/ 2023-10-11T09:25:59.585 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28377 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-11T09:25:59.585 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2023-10-11T09:25:59.585 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/ 2023-10-11T09:25:59.585 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 44585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio.ko 2023-10-11T09:25:59.586 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_aec.ko 2023-10-11T09:25:59.586 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_cif.ko 2023-10-11T09:25:59.586 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_dmem_genirq.ko 2023-10-11T09:25:59.586 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_mf624.ko 2023-10-11T09:25:59.586 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_netx.ko 2023-10-11T09:25:59.587 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-11T09:25:59.587 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_pdrv_genirq.ko 2023-10-11T09:25:59.587 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13633 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_sercos3.ko 2023-10-11T09:25:59.587 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/ 2023-10-11T09:25:59.587 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/c67x00/ 2023-10-11T09:25:59.588 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 73273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/c67x00/c67x00.ko 2023-10-11T09:25:59.588 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/chipidea/ 2023-10-11T09:25:59.588 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 143785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/chipidea/ci_hdrc.ko 2023-10-11T09:25:59.588 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2023-10-11T09:25:59.589 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2023-10-11T09:25:59.589 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2023-10-11T09:25:59.589 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/ 2023-10-11T09:25:59.589 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 85457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-11T09:25:59.589 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 51393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-11T09:25:59.590 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 47465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usblp.ko 2023-10-11T09:25:59.590 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 89833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usbtmc.ko 2023-10-11T09:25:59.590 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/common/ 2023-10-11T09:25:59.590 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/common/ulpi.ko 2023-10-11T09:25:59.590 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/dwc2/ 2023-10-11T09:25:59.591 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/dwc2/dwc2_pci.ko 2023-10-11T09:25:59.591 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/dwc3/ 2023-10-11T09:25:59.591 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/dwc3/dwc3-haps.ko 2023-10-11T09:25:59.591 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/dwc3/dwc3-pci.ko 2023-10-11T09:25:59.591 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 445217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/dwc3/dwc3.ko 2023-10-11T09:25:59.592 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/ 2023-10-11T09:25:59.592 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/ 2023-10-11T09:25:59.592 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 60089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/u_ether.ko 2023-10-11T09:25:59.592 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 51753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/u_serial.ko 2023-10-11T09:25:59.592 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2023-10-11T09:25:59.593 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 42649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2023-10-11T09:25:59.593 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2023-10-11T09:25:59.593 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2023-10-11T09:25:59.593 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 92377 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2023-10-11T09:25:59.593 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 51025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2023-10-11T09:25:59.594 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 122073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2023-10-11T09:25:59.594 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 57921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2023-10-11T09:25:59.594 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2023-10-11T09:25:59.594 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 31265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2023-10-11T09:25:59.594 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 57705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2023-10-11T09:25:59.594 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 85841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2023-10-11T09:25:59.595 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2023-10-11T09:25:59.595 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 58433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2023-10-11T09:25:59.595 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 67337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2023-10-11T09:25:59.595 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/ 2023-10-11T09:25:59.595 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2023-10-11T09:25:59.596 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2023-10-11T09:25:59.596 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2023-10-11T09:25:59.596 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_ether.ko 2023-10-11T09:25:59.596 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2023-10-11T09:25:59.596 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_hid.ko 2023-10-11T09:25:59.597 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2023-10-11T09:25:59.597 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16977 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2023-10-11T09:25:59.597 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 32073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2023-10-11T09:25:59.597 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_printer.ko 2023-10-11T09:25:59.597 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_serial.ko 2023-10-11T09:25:59.598 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_zero.ko 2023-10-11T09:25:59.598 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 62153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2023-10-11T09:25:59.598 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2023-10-11T09:25:59.598 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 161321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/libcomposite.ko 2023-10-11T09:25:59.598 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/ 2023-10-11T09:25:59.599 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2023-10-11T09:25:59.599 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/bdc/ 2023-10-11T09:25:59.599 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 157665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2023-10-11T09:25:59.599 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/goku_udc.ko 2023-10-11T09:25:59.600 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 52993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/gr_udc.ko 2023-10-11T09:25:59.600 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 69689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2023-10-11T09:25:59.600 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 56009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/mv_udc.ko 2023-10-11T09:25:59.600 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 76033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/net2272.ko 2023-10-11T09:25:59.600 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 75057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/net2280.ko 2023-10-11T09:25:59.600 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 73153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/pch_udc.ko 2023-10-11T09:25:59.601 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 65793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2023-10-11T09:25:59.601 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 66073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2023-10-11T09:25:59.601 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 81489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2023-10-11T09:25:59.601 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 156057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/udc-core.ko 2023-10-11T09:25:59.601 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/ 2023-10-11T09:25:59.602 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/bcma-hcd.ko 2023-10-11T09:25:59.602 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 80097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/isp116x-hcd.ko 2023-10-11T09:25:59.602 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 87337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/oxu210hp-hcd.ko 2023-10-11T09:25:59.602 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 78849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/r8a66597-hcd.ko 2023-10-11T09:25:59.603 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 51305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-11T09:25:59.603 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/sl811_cs.ko 2023-10-11T09:25:59.603 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/ssb-hcd.ko 2023-10-11T09:25:59.603 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/xhci-plat-hcd.ko 2023-10-11T09:25:59.603 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/ 2023-10-11T09:25:59.604 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/mdc800.ko 2023-10-11T09:25:59.604 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/microtek.ko 2023-10-11T09:25:59.604 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ 2023-10-11T09:25:59.604 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 41841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/adutux.ko 2023-10-11T09:25:59.604 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-11T09:25:59.605 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/cypress_cy7c63.ko 2023-10-11T09:25:59.605 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/cytherm.ko 2023-10-11T09:25:59.605 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ehset.ko 2023-10-11T09:25:59.605 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi26.ko 2023-10-11T09:25:59.605 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi62.ko 2023-10-11T09:25:59.606 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ezusb.ko 2023-10-11T09:25:59.606 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/idmouse.ko 2023-10-11T09:25:59.606 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-11T09:25:59.606 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-11T09:25:59.606 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ldusb.ko 2023-10-11T09:25:59.607 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-11T09:25:59.607 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/lvstest.ko 2023-10-11T09:25:59.607 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga/ 2023-10-11T09:25:59.607 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 64841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-11T09:25:59.607 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-11T09:25:59.608 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usb3503.ko 2023-10-11T09:25:59.608 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usblcd.ko 2023-10-11T09:25:59.608 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-11T09:25:59.608 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 70873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usbtest.ko 2023-10-11T09:25:59.608 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 41945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/uss720.ko 2023-10-11T09:25:59.609 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/yurex.ko 2023-10-11T09:25:59.609 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/mon/ 2023-10-11T09:25:59.609 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 73185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/mon/usbmon.ko 2023-10-11T09:25:59.609 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/musb/ 2023-10-11T09:25:59.609 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 301521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/musb/musb_hdrc.ko 2023-10-11T09:25:59.610 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/ 2023-10-11T09:25:59.610 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-11T09:25:59.610 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2023-10-11T09:25:59.610 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-isp1301.ko 2023-10-11T09:25:59.611 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-tahvo.ko 2023-10-11T09:25:59.611 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ 2023-10-11T09:25:59.611 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/aircable.ko 2023-10-11T09:25:59.611 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ark3116.ko 2023-10-11T09:25:59.611 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-11T09:25:59.612 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 38105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ch341.ko 2023-10-11T09:25:59.612 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 70209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cp210x.ko 2023-10-11T09:25:59.612 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 31169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-11T09:25:59.612 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 57937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-11T09:25:59.612 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 49753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-11T09:25:59.613 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/empeg.ko 2023-10-11T09:25:59.613 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 43897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/f81232.ko 2023-10-11T09:25:59.613 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 175145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-11T09:25:59.613 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 52713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-11T09:25:59.613 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 137105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-11T09:25:59.614 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 124673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_ti.ko 2023-10-11T09:25:59.614 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 67769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipaq.ko 2023-10-11T09:25:59.614 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipw.ko 2023-10-11T09:25:59.614 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-11T09:25:59.614 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 57937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-11T09:25:59.615 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 107417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan.ko 2023-10-11T09:25:59.615 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 35497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-11T09:25:59.615 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-11T09:25:59.615 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-11T09:25:59.615 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 32841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-11T09:25:59.616 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/metro-usb.ko 2023-10-11T09:25:59.616 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 66489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7720.ko 2023-10-11T09:25:59.616 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 78113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7840.ko 2023-10-11T09:25:59.616 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 56593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mxuport.ko 2023-10-11T09:25:59.616 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/navman.ko 2023-10-11T09:25:59.617 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/omninet.ko 2023-10-11T09:25:59.617 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/opticon.ko 2023-10-11T09:25:59.617 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 190233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/option.ko 2023-10-11T09:25:59.617 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 38905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/oti6858.ko 2023-10-11T09:25:59.617 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 54313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/pl2303.ko 2023-10-11T09:25:59.617 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcaux.ko 2023-10-11T09:25:59.618 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 37825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcserial.ko 2023-10-11T09:25:59.618 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34561 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/quatech2.ko 2023-10-11T09:25:59.618 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-11T09:25:59.618 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 51297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/sierra.ko 2023-10-11T09:25:59.619 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-11T09:25:59.619 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ssu100.ko 2023-10-11T09:25:59.619 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-11T09:25:59.619 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 61337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-11T09:25:59.619 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-11T09:25:59.620 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-11T09:25:59.620 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-11T09:25:59.620 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 125505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usbserial.ko 2023-10-11T09:25:59.620 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/visor.ko 2023-10-11T09:25:59.620 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 37409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-11T09:25:59.621 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/wishbone-serial.ko 2023-10-11T09:25:59.621 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-11T09:25:59.621 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ 2023-10-11T09:25:59.621 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 52913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/uas.ko 2023-10-11T09:25:59.621 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-11T09:25:59.621 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-11T09:25:59.622 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-11T09:25:59.622 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-11T09:25:59.622 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-11T09:25:59.622 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-11T09:25:59.622 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-11T09:25:59.623 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-11T09:25:59.623 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-11T09:25:59.623 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-11T09:25:59.623 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-11T09:25:59.623 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-11T09:25:59.624 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 35337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-11T09:25:59.624 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 180017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-11T09:25:59.624 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/ 2023-10-11T09:25:59.624 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 67457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-11T09:25:59.624 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 71345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-11T09:25:59.625 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 105401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-11T09:25:59.625 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/ 2023-10-11T09:25:59.625 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/ 2023-10-11T09:25:59.625 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 153697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-11T09:25:59.625 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-11T09:25:59.625 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 100569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio.ko 2023-10-11T09:25:59.626 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 75137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-11T09:25:59.626 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/ 2023-10-11T09:25:59.626 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 113889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost.ko 2023-10-11T09:25:59.626 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-11T09:25:59.626 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 58137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_net.ko 2023-10-11T09:25:59.627 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 89289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-11T09:25:59.627 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/ 2023-10-11T09:25:59.627 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/ 2023-10-11T09:25:59.627 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/88pm860x_bl.ko 2023-10-11T09:25:59.627 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/aat2870_bl.ko 2023-10-11T09:25:59.628 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/adp5520_bl.ko 2023-10-11T09:25:59.628 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 32609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/adp8860_bl.ko 2023-10-11T09:25:59.628 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 39993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/adp8870_bl.ko 2023-10-11T09:25:59.628 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-11T09:25:59.629 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16777 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/as3711_bl.ko 2023-10-11T09:25:59.629 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/bd6107.ko 2023-10-11T09:25:59.629 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/da903x_bl.ko 2023-10-11T09:25:59.629 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/da9052_bl.ko 2023-10-11T09:25:59.629 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/gpio_backlight.ko 2023-10-11T09:25:59.630 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/kb3886_bl.ko 2023-10-11T09:25:59.630 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lcd.ko 2023-10-11T09:25:59.630 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lm3533_bl.ko 2023-10-11T09:25:59.630 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lm3639_bl.ko 2023-10-11T09:25:59.630 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lv5207lp.ko 2023-10-11T09:25:59.631 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/max8925_bl.ko 2023-10-11T09:25:59.631 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/pandora_bl.ko 2023-10-11T09:25:59.631 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/pcf50633-backlight.ko 2023-10-11T09:25:59.631 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-11T09:25:59.631 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/wm831x_bl.ko 2023-10-11T09:25:59.632 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/ 2023-10-11T09:25:59.632 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ 2023-10-11T09:25:59.632 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2482.ko 2023-10-11T09:25:59.632 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 38961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2490.ko 2023-10-11T09:25:59.632 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/matrox_w1.ko 2023-10-11T09:25:59.633 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/w1-gpio.ko 2023-10-11T09:25:59.633 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/ 2023-10-11T09:25:59.633 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-11T09:25:59.633 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-11T09:25:59.633 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-11T09:25:59.633 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-11T09:25:59.634 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-11T09:25:59.634 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-11T09:25:59.634 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-11T09:25:59.634 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-11T09:25:59.634 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-11T09:25:59.635 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-11T09:25:59.635 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 56425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-11T09:25:59.635 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 102657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/wire.ko 2023-10-11T09:25:59.635 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ 2023-10-11T09:25:59.635 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/acquirewdt.ko 2023-10-11T09:25:59.636 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19977 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/advantechwdt.ko 2023-10-11T09:25:59.636 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18561 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-11T09:25:59.636 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-11T09:25:59.636 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/cpu5wdt.ko 2023-10-11T09:25:59.636 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/da9052_wdt.ko 2023-10-11T09:25:59.636 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12777 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/da9055_wdt.ko 2023-10-11T09:25:59.637 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/da9063_wdt.ko 2023-10-11T09:25:59.637 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/dw_wdt.ko 2023-10-11T09:25:59.637 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/eurotechwdt.ko 2023-10-11T09:25:59.637 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-11T09:25:59.637 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/hpwdt.ko 2023-10-11T09:25:59.638 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/i6300esb.ko 2023-10-11T09:25:59.638 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-11T09:25:59.638 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-11T09:25:59.638 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-11T09:25:59.639 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ibmasr.ko 2023-10-11T09:25:59.639 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-11T09:25:59.639 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-11T09:25:59.639 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-11T09:25:59.639 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/kempld_wdt.ko 2023-10-11T09:25:59.640 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/machzwd.ko 2023-10-11T09:25:59.640 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/mena21_wdt.ko 2023-10-11T09:25:59.640 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/menf21bmc_wdt.ko 2023-10-11T09:25:59.640 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/nv_tco.ko 2023-10-11T09:25:59.640 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/of_xilinx_wdt.ko 2023-10-11T09:25:59.640 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pc87413_wdt.ko 2023-10-11T09:25:59.641 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 38657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-11T09:25:59.641 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 31825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-11T09:25:59.641 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/retu_wdt.ko 2023-10-11T09:25:59.641 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sbc60xxwdt.ko 2023-10-11T09:25:59.641 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sbc_epx_c3.ko 2023-10-11T09:25:59.642 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-11T09:25:59.642 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sc1200wdt.ko 2023-10-11T09:25:59.642 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-11T09:25:59.642 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/smsc37b787_wdt.ko 2023-10-11T09:25:59.642 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/softdog.ko 2023-10-11T09:25:59.642 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-11T09:25:59.643 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/twl4030_wdt.ko 2023-10-11T09:25:59.643 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/via_wdt.ko 2023-10-11T09:25:59.643 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-11T09:25:59.643 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-11T09:25:59.643 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-11T09:25:59.644 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wafer5823wdt.ko 2023-10-11T09:25:59.644 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-11T09:25:59.644 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wm831x_wdt.ko 2023-10-11T09:25:59.644 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wm8350_wdt.ko 2023-10-11T09:25:59.644 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-11T09:25:59.644 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/ 2023-10-11T09:25:59.645 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-evtchn.ko 2023-10-11T09:25:59.645 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-11T09:25:59.645 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 47841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntdev.ko 2023-10-11T09:25:59.645 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback/ 2023-10-11T09:25:59.645 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 176417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-11T09:25:59.646 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 40105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-privcmd.ko 2023-10-11T09:25:59.646 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 71321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-scsiback.ko 2023-10-11T09:25:59.646 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs/ 2023-10-11T09:25:59.646 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-11T09:25:59.646 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ 2023-10-11T09:25:59.647 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p/ 2023-10-11T09:25:59.647 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 165625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p/9p.ko 2023-10-11T09:25:59.647 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/adfs/ 2023-10-11T09:25:59.647 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 60881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/adfs/adfs.ko 2023-10-11T09:25:59.647 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs/ 2023-10-11T09:25:59.648 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 162617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs/affs.ko 2023-10-11T09:25:59.648 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/afs/ 2023-10-11T09:25:59.648 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 931937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/afs/kafs.ko 2023-10-11T09:25:59.648 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/autofs/ 2023-10-11T09:25:59.648 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 100937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/autofs/autofs4.ko 2023-10-11T09:25:59.649 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs/ 2023-10-11T09:25:59.649 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 64481 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs/befs.ko 2023-10-11T09:25:59.649 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/bfs/ 2023-10-11T09:25:59.649 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 40993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/bfs/bfs.ko 2023-10-11T09:25:59.649 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/binfmt_misc.ko 2023-10-11T09:25:59.650 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs/ 2023-10-11T09:25:59.650 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 3474249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs/btrfs.ko 2023-10-11T09:25:59.650 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles/ 2023-10-11T09:25:59.650 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 305745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles/cachefiles.ko 2023-10-11T09:25:59.650 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph/ 2023-10-11T09:25:59.651 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 1316257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph/ceph.ko 2023-10-11T09:25:59.651 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda/ 2023-10-11T09:25:59.651 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 100497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda/coda.ko 2023-10-11T09:25:59.651 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/configfs/ 2023-10-11T09:25:59.651 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 108073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/configfs/configfs.ko 2023-10-11T09:25:59.651 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs/ 2023-10-11T09:25:59.652 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs/cramfs.ko 2023-10-11T09:25:59.652 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm/ 2023-10-11T09:25:59.652 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 633825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm/dlm.ko 2023-10-11T09:25:59.652 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/efs/ 2023-10-11T09:25:59.652 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/efs/efs.ko 2023-10-11T09:25:59.653 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs/ 2023-10-11T09:25:59.653 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 1500513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs/f2fs.ko 2023-10-11T09:25:59.653 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/ 2023-10-11T09:25:59.653 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/msdos.ko 2023-10-11T09:25:59.653 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/freevxfs/ 2023-10-11T09:25:59.653 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 42457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/freevxfs/freevxfs.ko 2023-10-11T09:25:59.654 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache/ 2023-10-11T09:25:59.654 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 226505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache/fscache.ko 2023-10-11T09:25:59.654 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/ 2023-10-11T09:25:59.654 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/cuse.ko 2023-10-11T09:25:59.654 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2/ 2023-10-11T09:25:59.655 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 1226385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2/gfs2.ko 2023-10-11T09:25:59.655 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs/ 2023-10-11T09:25:59.655 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 141241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs/hfs.ko 2023-10-11T09:25:59.655 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus/ 2023-10-11T09:25:59.655 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 221553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus/hfsplus.ko 2023-10-11T09:25:59.656 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hpfs/ 2023-10-11T09:25:59.656 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 193577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hpfs/hpfs.ko 2023-10-11T09:25:59.656 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs/ 2023-10-11T09:25:59.656 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 74393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs/isofs.ko 2023-10-11T09:25:59.656 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs/ 2023-10-11T09:25:59.656 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 421097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs/jfs.ko 2023-10-11T09:25:59.657 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd/ 2023-10-11T09:25:59.657 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 279041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd/lockd.ko 2023-10-11T09:25:59.657 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix/ 2023-10-11T09:25:59.657 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 87833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix/minix.ko 2023-10-11T09:25:59.657 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs/ 2023-10-11T09:25:59.657 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 124697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs/netfs.ko 2023-10-11T09:25:59.658 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/ 2023-10-11T09:25:59.658 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout/ 2023-10-11T09:25:59.658 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 81521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-11T09:25:59.658 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout/ 2023-10-11T09:25:59.659 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 59425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-11T09:25:59.659 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout/ 2023-10-11T09:25:59.659 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 117705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-11T09:25:59.659 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 1053889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfs.ko 2023-10-11T09:25:59.659 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 66089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv2.ko 2023-10-11T09:25:59.660 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 119801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv3.ko 2023-10-11T09:25:59.660 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 2041809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv4.ko 2023-10-11T09:25:59.660 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/ 2023-10-11T09:25:59.660 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/grace.ko 2023-10-11T09:25:59.660 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-11T09:25:59.661 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd/ 2023-10-11T09:25:59.661 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 1504929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd/nfsd.ko 2023-10-11T09:25:59.661 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2/ 2023-10-11T09:25:59.661 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 591401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2/nilfs2.ko 2023-10-11T09:25:59.661 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/ 2023-10-11T09:25:59.661 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-celtic.ko 2023-10-11T09:25:59.662 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-centeuro.ko 2023-10-11T09:25:59.662 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-croatian.ko 2023-10-11T09:25:59.662 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-cyrillic.ko 2023-10-11T09:25:59.662 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13481 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-gaelic.ko 2023-10-11T09:25:59.662 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-greek.ko 2023-10-11T09:25:59.663 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-iceland.ko 2023-10-11T09:25:59.663 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-inuit.ko 2023-10-11T09:25:59.663 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-roman.ko 2023-10-11T09:25:59.663 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-romanian.ko 2023-10-11T09:25:59.663 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-turkish.ko 2023-10-11T09:25:59.663 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_ascii.ko 2023-10-11T09:25:59.664 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1250.ko 2023-10-11T09:25:59.664 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1251.ko 2023-10-11T09:25:59.664 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1255.ko 2023-10-11T09:25:59.664 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp737.ko 2023-10-11T09:25:59.664 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp775.ko 2023-10-11T09:25:59.665 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp850.ko 2023-10-11T09:25:59.665 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp852.ko 2023-10-11T09:25:59.665 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp855.ko 2023-10-11T09:25:59.665 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp857.ko 2023-10-11T09:25:59.665 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp860.ko 2023-10-11T09:25:59.666 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp861.ko 2023-10-11T09:25:59.666 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp862.ko 2023-10-11T09:25:59.666 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp863.ko 2023-10-11T09:25:59.666 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp864.ko 2023-10-11T09:25:59.666 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp865.ko 2023-10-11T09:25:59.666 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp866.ko 2023-10-11T09:25:59.667 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp869.ko 2023-10-11T09:25:59.667 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp874.ko 2023-10-11T09:25:59.667 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 94137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp932.ko 2023-10-11T09:25:59.667 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 142673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp936.ko 2023-10-11T09:25:59.667 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 165745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp949.ko 2023-10-11T09:25:59.668 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 118121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp950.ko 2023-10-11T09:25:59.668 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_euc-jp.ko 2023-10-11T09:25:59.668 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-11T09:25:59.668 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-11T09:25:59.668 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-11T09:25:59.669 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-11T09:25:59.669 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-11T09:25:59.669 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-11T09:25:59.669 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-11T09:25:59.670 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-11T09:25:59.670 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-11T09:25:59.670 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-11T09:25:59.670 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-11T09:25:59.670 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-r.ko 2023-10-11T09:25:59.670 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-11T09:25:59.671 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-u.ko 2023-10-11T09:25:59.671 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_utf8.ko 2023-10-11T09:25:59.671 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ntfs/ 2023-10-11T09:25:59.671 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 219361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ntfs/ntfs.ko 2023-10-11T09:25:59.671 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ 2023-10-11T09:25:59.672 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster/ 2023-10-11T09:25:59.672 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 244177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-11T09:25:59.672 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm/ 2023-10-11T09:25:59.672 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 440785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-11T09:25:59.672 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs/ 2023-10-11T09:25:59.672 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 53841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-11T09:25:59.673 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 2589873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2.ko 2023-10-11T09:25:59.673 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-11T09:25:59.673 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 31585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-11T09:25:59.673 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 35881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-11T09:25:59.673 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/omfs/ 2023-10-11T09:25:59.674 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 52569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/omfs/omfs.ko 2023-10-11T09:25:59.674 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs/ 2023-10-11T09:25:59.674 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 336889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs/overlay.ko 2023-10-11T09:25:59.674 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore/ 2023-10-11T09:25:59.674 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 54609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore/ramoops.ko 2023-10-11T09:25:59.674 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/qnx4/ 2023-10-11T09:25:59.675 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/qnx4/qnx4.ko 2023-10-11T09:25:59.675 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/qnx6/ 2023-10-11T09:25:59.675 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 46649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/qnx6/qnx6.ko 2023-10-11T09:25:59.675 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/quota/ 2023-10-11T09:25:59.675 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/quota/quota_tree.ko 2023-10-11T09:25:59.676 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/quota/quota_v1.ko 2023-10-11T09:25:59.676 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/quota/quota_v2.ko 2023-10-11T09:25:59.676 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs/ 2023-10-11T09:25:59.676 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 530697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs/reiserfs.ko 2023-10-11T09:25:59.676 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs/ 2023-10-11T09:25:59.676 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs/romfs.ko 2023-10-11T09:25:59.677 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/ 2023-10-11T09:25:59.677 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client/ 2023-10-11T09:25:59.677 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 2661209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client/cifs.ko 2023-10-11T09:25:59.677 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/ 2023-10-11T09:25:59.677 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-11T09:25:59.678 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_md4.ko 2023-10-11T09:25:59.678 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs/ 2023-10-11T09:25:59.678 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 134313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs/squashfs.ko 2023-10-11T09:25:59.678 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv/ 2023-10-11T09:25:59.678 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 89737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv/sysv.ko 2023-10-11T09:25:59.679 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf/ 2023-10-11T09:25:59.679 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 248049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf/udf.ko 2023-10-11T09:25:59.679 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs/ 2023-10-11T09:25:59.679 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 160857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs/ufs.ko 2023-10-11T09:25:59.679 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs/ 2023-10-11T09:25:59.679 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 3998129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs/xfs.ko 2023-10-11T09:25:59.680 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/ 2023-10-11T09:25:59.680 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/locking/ 2023-10-11T09:25:59.680 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 74865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/locking/locktorture.ko 2023-10-11T09:25:59.680 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/time/ 2023-10-11T09:25:59.680 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/time/test_udelay.ko 2023-10-11T09:25:59.681 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 58473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/torture.ko 2023-10-11T09:25:59.681 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ 2023-10-11T09:25:59.681 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 6833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc-itu-t.ko 2023-10-11T09:25:59.681 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 6529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc7.ko 2023-10-11T09:25:59.681 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc8.ko 2023-10-11T09:25:59.682 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/ 2023-10-11T09:25:59.682 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libarc4.ko 2023-10-11T09:25:59.682 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 37081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libdes.ko 2023-10-11T09:25:59.682 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/interval_tree_test.ko 2023-10-11T09:25:59.682 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/libcrc32c.ko 2023-10-11T09:25:59.683 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lru_cache.ko 2023-10-11T09:25:59.683 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/ 2023-10-11T09:25:59.683 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4_compress.ko 2023-10-11T09:25:59.683 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22633 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-11T09:25:59.683 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math/ 2023-10-11T09:25:59.684 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 6617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math/cordic.ko 2023-10-11T09:25:59.684 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/memory-notifier-error-inject.ko 2023-10-11T09:25:59.684 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/notifier-error-inject.ko 2023-10-11T09:25:59.684 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/percpu_test.ko 2023-10-11T09:25:59.684 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/pm-notifier-error-inject.ko 2023-10-11T09:25:59.684 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6/ 2023-10-11T09:25:59.685 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 143433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6/raid6_pq.ko 2023-10-11T09:25:59.685 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/rbtree_test.ko 2023-10-11T09:25:59.685 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon/ 2023-10-11T09:25:59.685 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-11T09:25:59.685 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 38841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/test-kstrtox.ko 2023-10-11T09:25:59.686 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/test-string_helpers.ko 2023-10-11T09:25:59.686 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 4760825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/test_bpf.ko 2023-10-11T09:25:59.686 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 49081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/test_firmware.ko 2023-10-11T09:25:59.686 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 6369 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/test_module.ko 2023-10-11T09:25:59.686 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/test_user_copy.ko 2023-10-11T09:25:59.687 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_bm.ko 2023-10-11T09:25:59.687 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_fsm.ko 2023-10-11T09:25:59.687 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_kmp.ko 2023-10-11T09:25:59.687 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/xz/ 2023-10-11T09:25:59.687 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/xz/xz_dec_test.ko 2023-10-11T09:25:59.688 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm/ 2023-10-11T09:25:59.688 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm/hwpoison-inject.ko 2023-10-11T09:25:59.688 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ 2023-10-11T09:25:59.688 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/ 2023-10-11T09:25:59.688 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 55737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/6lowpan.ko 2023-10-11T09:25:59.689 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_dest.ko 2023-10-11T09:25:59.689 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-11T09:25:59.689 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_hop.ko 2023-10-11T09:25:59.689 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 6985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-11T09:25:59.689 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-11T09:25:59.690 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_routing.ko 2023-10-11T09:25:59.690 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_udp.ko 2023-10-11T09:25:59.690 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/ 2023-10-11T09:25:59.690 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/garp.ko 2023-10-11T09:25:59.691 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/mrp.ko 2023-10-11T09:25:59.691 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/p8022.ko 2023-10-11T09:25:59.691 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/psnap.ko 2023-10-11T09:25:59.691 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/stp.ko 2023-10-11T09:25:59.691 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q/ 2023-10-11T09:25:59.692 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 81017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q/8021q.ko 2023-10-11T09:25:59.692 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/ 2023-10-11T09:25:59.692 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 198465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet.ko 2023-10-11T09:25:59.692 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 35961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_fd.ko 2023-10-11T09:25:59.692 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_rdma.ko 2023-10-11T09:25:59.693 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 32321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_virtio.ko 2023-10-11T09:25:59.693 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv/ 2023-10-11T09:25:59.693 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 652553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv/batman-adv.ko 2023-10-11T09:25:59.693 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/ 2023-10-11T09:25:59.693 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 63825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/br_netfilter.ko 2023-10-11T09:25:59.694 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 772505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/bridge.ko 2023-10-11T09:25:59.694 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ 2023-10-11T09:25:59.694 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-11T09:25:59.694 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-11T09:25:59.694 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-11T09:25:59.695 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-11T09:25:59.695 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-11T09:25:59.695 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-11T09:25:59.695 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-11T09:25:59.695 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-11T09:25:59.696 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-11T09:25:59.696 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-11T09:25:59.696 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-11T09:25:59.696 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-11T09:25:59.696 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-11T09:25:59.696 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-11T09:25:59.697 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-11T09:25:59.697 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-11T09:25:59.697 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-11T09:25:59.697 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-11T09:25:59.697 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-11T09:25:59.698 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-11T09:25:59.698 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 55937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-11T09:25:59.698 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-11T09:25:59.698 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-11T09:25:59.699 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph/ 2023-10-11T09:25:59.699 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 1046921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph/libceph.ko 2023-10-11T09:25:59.699 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/ 2023-10-11T09:25:59.699 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 124793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/pktgen.ko 2023-10-11T09:25:59.699 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/ 2023-10-11T09:25:59.699 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 194321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp.ko 2023-10-11T09:25:59.700 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_diag.ko 2023-10-11T09:25:59.700 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 56945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv4.ko 2023-10-11T09:25:59.700 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 59457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv6.ko 2023-10-11T09:25:59.700 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/ 2023-10-11T09:25:59.700 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 390297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/dsa_core.ko 2023-10-11T09:25:59.701 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_brcm.ko 2023-10-11T09:25:59.701 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_dsa.ko 2023-10-11T09:25:59.701 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_none.ko 2023-10-11T09:25:59.701 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_trailer.ko 2023-10-11T09:25:59.701 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/hsr/ 2023-10-11T09:25:59.701 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 100257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/hsr/hsr.ko 2023-10-11T09:25:59.702 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ 2023-10-11T09:25:59.702 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan/ 2023-10-11T09:25:59.702 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 45169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-11T09:25:59.702 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 254601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154.ko 2023-10-11T09:25:59.702 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 47265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-11T09:25:59.703 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ 2023-10-11T09:25:59.703 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ah4.ko 2023-10-11T09:25:59.703 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 38905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/esp4.ko 2023-10-11T09:25:59.703 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 48345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/fou.ko 2023-10-11T09:25:59.703 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/gre.ko 2023-10-11T09:25:59.704 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 48529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/inet_diag.ko 2023-10-11T09:25:59.704 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 60593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_gre.ko 2023-10-11T09:25:59.704 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 61281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_tunnel.ko 2023-10-11T09:25:59.704 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_vti.ko 2023-10-11T09:25:59.704 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipcomp.ko 2023-10-11T09:25:59.705 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipip.ko 2023-10-11T09:25:59.705 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ 2023-10-11T09:25:59.705 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 37689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-11T09:25:59.705 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-11T09:25:59.705 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-11T09:25:59.706 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 41601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ip_tables.ko 2023-10-11T09:25:59.706 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-11T09:25:59.706 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2023-10-11T09:25:59.706 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-11T09:25:59.706 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-11T09:25:59.707 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-11T09:25:59.707 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_filter.ko 2023-10-11T09:25:59.707 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-11T09:25:59.707 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-11T09:25:59.707 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-11T09:25:59.708 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-11T09:25:59.708 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-11T09:25:59.708 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-11T09:25:59.708 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-11T09:25:59.708 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-11T09:25:59.709 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-11T09:25:59.709 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20561 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2023-10-11T09:25:59.709 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-11T09:25:59.709 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-11T09:25:59.710 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-11T09:25:59.710 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-11T09:25:59.710 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-11T09:25:59.710 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_bic.ko 2023-10-11T09:25:59.710 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-11T09:25:59.711 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_diag.ko 2023-10-11T09:25:59.711 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-11T09:25:59.711 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_htcp.ko 2023-10-11T09:25:59.711 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_hybla.ko 2023-10-11T09:25:59.711 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_illinois.ko 2023-10-11T09:25:59.712 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_lp.ko 2023-10-11T09:25:59.712 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_scalable.ko 2023-10-11T09:25:59.712 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_vegas.ko 2023-10-11T09:25:59.712 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_veno.ko 2023-10-11T09:25:59.712 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_westwood.ko 2023-10-11T09:25:59.712 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_yeah.ko 2023-10-11T09:25:59.713 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tunnel4.ko 2023-10-11T09:25:59.713 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_diag.ko 2023-10-11T09:25:59.713 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 41577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_tunnel.ko 2023-10-11T09:25:59.713 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-11T09:25:59.714 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ 2023-10-11T09:25:59.714 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ah6.ko 2023-10-11T09:25:59.714 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 39537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/esp6.ko 2023-10-11T09:25:59.714 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/fou6.ko 2023-10-11T09:25:59.714 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 80793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_gre.ko 2023-10-11T09:25:59.715 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 92817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-11T09:25:59.715 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-11T09:25:59.715 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 54089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_vti.ko 2023-10-11T09:25:59.715 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ipcomp6.ko 2023-10-11T09:25:59.715 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/mip6.ko 2023-10-11T09:25:59.715 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ 2023-10-11T09:25:59.717 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 42209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-11T09:25:59.717 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2023-10-11T09:25:59.717 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-11T09:25:59.718 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-11T09:25:59.718 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-11T09:25:59.718 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-11T09:25:59.718 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-11T09:25:59.718 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-11T09:25:59.719 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-11T09:25:59.719 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-11T09:25:59.719 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-11T09:25:59.719 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-11T09:25:59.719 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-11T09:25:59.720 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-11T09:25:59.720 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-11T09:25:59.720 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-11T09:25:59.720 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-11T09:25:59.720 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 38833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-11T09:25:59.721 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-11T09:25:59.721 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-11T09:25:59.721 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-11T09:25:59.721 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-11T09:25:59.721 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9633 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-11T09:25:59.721 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-11T09:25:59.722 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-11T09:25:59.722 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 73161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/sit.ko 2023-10-11T09:25:59.722 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/tunnel6.ko 2023-10-11T09:25:59.722 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-11T09:25:59.723 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key/ 2023-10-11T09:25:59.723 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 82929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key/af_key.ko 2023-10-11T09:25:59.723 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/ 2023-10-11T09:25:59.723 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 125873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_core.ko 2023-10-11T09:25:59.723 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-11T09:25:59.723 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_eth.ko 2023-10-11T09:25:59.724 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 35049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip.ko 2023-10-11T09:25:59.724 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-11T09:25:59.724 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-11T09:25:59.724 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 53153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-11T09:25:59.724 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/lapb/ 2023-10-11T09:25:59.725 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 48065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/lapb/lapb.ko 2023-10-11T09:25:59.725 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc/ 2023-10-11T09:25:59.725 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc/llc.ko 2023-10-11T09:25:59.725 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 267257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc/llc2.ko 2023-10-11T09:25:59.725 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154/ 2023-10-11T09:25:59.725 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 291041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154/mac802154.ko 2023-10-11T09:25:59.726 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls/ 2023-10-11T09:25:59.726 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls/mpls_gso.ko 2023-10-11T09:25:59.726 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ 2023-10-11T09:25:59.726 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ 2023-10-11T09:25:59.726 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 119649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-11T09:25:59.727 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-11T09:25:59.727 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-11T09:25:59.727 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-11T09:25:59.727 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 102033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-11T09:25:59.727 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 103121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-11T09:25:59.728 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 104593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-11T09:25:59.728 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 105129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-11T09:25:59.728 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 114393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-11T09:25:59.728 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 61161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-11T09:25:59.728 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 111161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-11T09:25:59.729 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 114673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-11T09:25:59.729 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 114329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-11T09:25:59.729 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 113217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-11T09:25:59.729 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 116217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-11T09:25:59.729 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-11T09:25:59.730 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ 2023-10-11T09:25:59.730 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 412889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-11T09:25:59.730 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-11T09:25:59.730 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-11T09:25:59.730 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-11T09:25:59.731 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-11T09:25:59.731 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-11T09:25:59.731 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-11T09:25:59.731 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-11T09:25:59.731 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-11T09:25:59.732 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-11T09:25:59.732 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-11T09:25:59.732 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-11T09:25:59.732 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-11T09:25:59.732 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-11T09:25:59.733 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 48897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conncount.ko 2023-10-11T09:25:59.733 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 408665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack.ko 2023-10-11T09:25:59.733 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-11T09:25:59.733 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-11T09:25:59.733 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33761 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-11T09:25:59.733 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 118145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-11T09:25:59.734 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-11T09:25:59.734 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-11T09:25:59.734 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 108393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-11T09:25:59.734 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-11T09:25:59.735 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-11T09:25:59.735 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 66641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-11T09:25:59.735 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-11T09:25:59.735 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-11T09:25:59.736 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-11T09:25:59.736 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 88513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table.ko 2023-10-11T09:25:59.736 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-11T09:25:59.736 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 37129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-11T09:25:59.736 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 95353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat.ko 2023-10-11T09:25:59.736 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-11T09:25:59.737 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-11T09:25:59.737 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-11T09:25:59.737 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-11T09:25:59.737 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-11T09:25:59.738 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 44025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-11T09:25:59.738 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 609257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_tables.ko 2023-10-11T09:25:59.738 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 43873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink.ko 2023-10-11T09:25:59.738 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-11T09:25:59.738 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_cttimeout.ko 2023-10-11T09:25:59.739 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 39905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-11T09:25:59.739 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-11T09:25:59.739 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 49673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-11T09:25:59.739 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-11T09:25:59.739 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_compat.ko 2023-10-11T09:25:59.740 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 40721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_ct.ko 2023-10-11T09:25:59.740 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-11T09:25:59.740 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fib.ko 2023-10-11T09:25:59.740 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-11T09:25:59.740 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_hash.ko 2023-10-11T09:25:59.741 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_limit.ko 2023-10-11T09:25:59.741 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_log.ko 2023-10-11T09:25:59.741 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_masq.ko 2023-10-11T09:25:59.741 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_nat.ko 2023-10-11T09:25:59.741 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_numgen.ko 2023-10-11T09:25:59.742 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_osf.ko 2023-10-11T09:25:59.742 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_queue.ko 2023-10-11T09:25:59.742 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_redir.ko 2023-10-11T09:25:59.742 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject.ko 2023-10-11T09:25:59.743 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-11T09:25:59.743 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_socket.ko 2023-10-11T09:25:59.743 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_synproxy.ko 2023-10-11T09:25:59.743 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_tproxy.ko 2023-10-11T09:25:59.743 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 89329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/x_tables.ko 2023-10-11T09:25:59.744 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-11T09:25:59.744 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9633 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-11T09:25:59.744 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-11T09:25:59.744 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-11T09:25:59.744 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CT.ko 2023-10-11T09:25:59.744 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_DSCP.ko 2023-10-11T09:25:59.745 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HL.ko 2023-10-11T09:25:59.745 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HMARK.ko 2023-10-11T09:25:59.745 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 38857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-11T09:25:59.745 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LED.ko 2023-10-11T09:25:59.745 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LOG.ko 2023-10-11T09:25:59.746 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-11T09:25:59.746 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-11T09:25:59.746 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-11T09:25:59.746 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-11T09:25:59.746 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-11T09:25:59.747 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-11T09:25:59.747 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-11T09:25:59.747 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-11T09:25:59.747 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-11T09:25:59.747 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17369 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TEE.ko 2023-10-11T09:25:59.748 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-11T09:25:59.748 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TRACE.ko 2023-10-11T09:25:59.748 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_addrtype.ko 2023-10-11T09:25:59.748 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_bpf.ko 2023-10-11T09:25:59.748 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cgroup.ko 2023-10-11T09:25:59.749 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cluster.ko 2023-10-11T09:25:59.749 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_comment.ko 2023-10-11T09:25:59.749 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connbytes.ko 2023-10-11T09:25:59.749 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlabel.ko 2023-10-11T09:25:59.750 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlimit.ko 2023-10-11T09:25:59.750 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connmark.ko 2023-10-11T09:25:59.750 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_conntrack.ko 2023-10-11T09:25:59.750 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cpu.ko 2023-10-11T09:25:59.750 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dccp.ko 2023-10-11T09:25:59.751 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_devgroup.ko 2023-10-11T09:25:59.751 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dscp.ko 2023-10-11T09:25:59.751 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ecn.ko 2023-10-11T09:25:59.751 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_esp.ko 2023-10-11T09:25:59.752 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 43457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-11T09:25:59.752 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_helper.ko 2023-10-11T09:25:59.752 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hl.ko 2023-10-11T09:25:59.752 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-11T09:25:59.752 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_iprange.ko 2023-10-11T09:25:59.753 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipvs.ko 2023-10-11T09:25:59.753 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_l2tp.ko 2023-10-11T09:25:59.753 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_length.ko 2023-10-11T09:25:59.753 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_limit.ko 2023-10-11T09:25:59.753 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mac.ko 2023-10-11T09:25:59.754 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mark.ko 2023-10-11T09:25:59.754 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_multiport.ko 2023-10-11T09:25:59.754 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nat.ko 2023-10-11T09:25:59.754 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nfacct.ko 2023-10-11T09:25:59.754 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_osf.ko 2023-10-11T09:25:59.755 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_owner.ko 2023-10-11T09:25:59.755 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_physdev.ko 2023-10-11T09:25:59.755 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7633 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_pkttype.ko 2023-10-11T09:25:59.755 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_policy.ko 2023-10-11T09:25:59.755 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_quota.ko 2023-10-11T09:25:59.756 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_rateest.ko 2023-10-11T09:25:59.756 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_realm.ko 2023-10-11T09:25:59.756 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_recent.ko 2023-10-11T09:25:59.756 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_sctp.ko 2023-10-11T09:25:59.756 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_set.ko 2023-10-11T09:25:59.756 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_socket.ko 2023-10-11T09:25:59.757 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_state.ko 2023-10-11T09:25:59.757 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_statistic.ko 2023-10-11T09:25:59.757 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_string.ko 2023-10-11T09:25:59.757 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-11T09:25:59.757 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15777 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_tcpudp.ko 2023-10-11T09:25:59.758 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_time.ko 2023-10-11T09:25:59.758 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_u32.ko 2023-10-11T09:25:59.758 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink/ 2023-10-11T09:25:59.758 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink/netlink_diag.ko 2023-10-11T09:25:59.758 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh/ 2023-10-11T09:25:59.759 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh/nsh.ko 2023-10-11T09:25:59.759 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/ 2023-10-11T09:25:59.759 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 379545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/openvswitch.ko 2023-10-11T09:25:59.759 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-geneve.ko 2023-10-11T09:25:59.759 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-gre.ko 2023-10-11T09:25:59.760 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-11T09:25:59.760 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet/ 2023-10-11T09:25:59.760 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet/af_packet_diag.ko 2023-10-11T09:25:59.760 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/phonet/ 2023-10-11T09:25:59.760 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 109961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/phonet/phonet.ko 2023-10-11T09:25:59.761 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 54761 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/phonet/pn_pep.ko 2023-10-11T09:25:59.761 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/ 2023-10-11T09:25:59.761 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 252889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds.ko 2023-10-11T09:25:59.761 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 165081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_rdma.ko 2023-10-11T09:25:59.761 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 67241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_tcp.ko 2023-10-11T09:25:59.762 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rxrpc/ 2023-10-11T09:25:59.762 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 820137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rxrpc/rxrpc.ko 2023-10-11T09:25:59.762 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/ 2023-10-11T09:25:59.762 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 32249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_csum.ko 2023-10-11T09:25:59.762 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_gact.ko 2023-10-11T09:25:59.763 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_ipt.ko 2023-10-11T09:25:59.763 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_mirred.ko 2023-10-11T09:25:59.763 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_nat.ko 2023-10-11T09:25:59.763 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 43817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_pedit.ko 2023-10-11T09:25:59.763 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 31985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_police.ko 2023-10-11T09:25:59.764 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_simple.ko 2023-10-11T09:25:59.764 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_skbedit.ko 2023-10-11T09:25:59.764 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_vlan.ko 2023-10-11T09:25:59.764 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_basic.ko 2023-10-11T09:25:59.764 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_bpf.ko 2023-10-11T09:25:59.765 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_cgroup.ko 2023-10-11T09:25:59.765 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 31377 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_flow.ko 2023-10-11T09:25:59.765 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 31769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_fw.ko 2023-10-11T09:25:59.765 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 38489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_route.ko 2023-10-11T09:25:59.765 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 61409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_u32.ko 2023-10-11T09:25:59.765 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_cmp.ko 2023-10-11T09:25:59.766 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_ipset.ko 2023-10-11T09:25:59.766 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_meta.ko 2023-10-11T09:25:59.766 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_nbyte.ko 2023-10-11T09:25:59.766 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_text.ko 2023-10-11T09:25:59.766 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_u32.ko 2023-10-11T09:25:59.767 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_choke.ko 2023-10-11T09:25:59.767 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_codel.ko 2023-10-11T09:25:59.767 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_drr.ko 2023-10-11T09:25:59.767 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_fq.ko 2023-10-11T09:25:59.767 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 31601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_fq_codel.ko 2023-10-11T09:25:59.768 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 35433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_gred.ko 2023-10-11T09:25:59.768 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 48409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hfsc.ko 2023-10-11T09:25:59.768 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hhf.ko 2023-10-11T09:25:59.768 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 65793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_htb.ko 2023-10-11T09:25:59.768 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_ingress.ko 2023-10-11T09:25:59.768 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio.ko 2023-10-11T09:25:59.769 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-11T09:25:59.769 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_multiq.ko 2023-10-11T09:25:59.769 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 35601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_netem.ko 2023-10-11T09:25:59.769 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_pie.ko 2023-10-11T09:25:59.769 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_plug.ko 2023-10-11T09:25:59.770 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_prio.ko 2023-10-11T09:25:59.770 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 47849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_qfq.ko 2023-10-11T09:25:59.770 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 32953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_red.ko 2023-10-11T09:25:59.770 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfb.ko 2023-10-11T09:25:59.770 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 31425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfq.ko 2023-10-11T09:25:59.771 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_tbf.ko 2023-10-11T09:25:59.771 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_teql.ko 2023-10-11T09:25:59.771 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/ 2023-10-11T09:25:59.771 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 930369 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp.ko 2023-10-11T09:25:59.771 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp_diag.ko 2023-10-11T09:25:59.772 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/ 2023-10-11T09:25:59.772 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/ 2023-10-11T09:25:59.772 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 335569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-11T09:25:59.772 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 61833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-11T09:25:59.772 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 1514673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/sunrpc.ko 2023-10-11T09:25:59.773 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma/ 2023-10-11T09:25:59.773 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 707401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-11T09:25:59.773 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/ 2023-10-11T09:25:59.773 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/diag.ko 2023-10-11T09:25:59.773 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 862385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/tipc.ko 2023-10-11T09:25:59.774 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix/ 2023-10-11T09:25:59.774 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix/unix_diag.ko 2023-10-11T09:25:59.774 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/ 2023-10-11T09:25:59.774 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 84841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-11T09:25:59.774 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 68033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-11T09:25:59.775 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 95985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock.ko 2023-10-11T09:25:59.775 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-11T09:25:59.775 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-11T09:25:59.775 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/x25/ 2023-10-11T09:25:59.775 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 125097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/x25/x25.ko 2023-10-11T09:25:59.776 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/ 2023-10-11T09:25:59.776 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/xfrm_algo.ko 2023-10-11T09:25:59.776 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-11T09:25:59.776 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 102353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/xfrm_user.ko 2023-10-11T09:25:59.776 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/ 2023-10-11T09:25:59.777 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib/ 2023-10-11T09:25:59.777 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib/irqbypass.ko 2023-10-11T09:25:59.777 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 638591 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias 2023-10-11T09:25:59.777 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 624324 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias.bin 2023-10-11T09:25:59.777 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7656 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin 2023-10-11T09:25:59.778 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.alias.bin 2023-10-11T09:25:59.778 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9821 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.bin 2023-10-11T09:25:59.778 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 69921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.modinfo 2023-10-11T09:25:59.778 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 146449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep 2023-10-11T09:25:59.778 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 232695 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep.bin 2023-10-11T09:25:59.779 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 210 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.devname 2023-10-11T09:25:59.779 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 72754 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.order 2023-10-11T09:25:59.779 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 879 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.softdep 2023-10-11T09:25:59.779 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 252880 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols 2023-10-11T09:25:59.779 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 307898 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols.bin 2023-10-11T09:25:59.780 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./usr/ 2023-10-11T09:25:59.780 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./usr/share/ 2023-10-11T09:25:59.780 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./usr/share/doc/ 2023-10-11T09:25:59.780 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./usr/share/doc/linux-image-6.5.0-rc7-gfef64f93da2b/ 2023-10-11T09:25:59.780 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 172 2023-09-11 02:20 ./usr/share/doc/linux-image-6.5.0-rc7-gfef64f93da2b/changelog.Debian.gz 2023-10-11T09:25:59.780 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 691 2023-09-11 02:20 ./usr/share/doc/linux-image-6.5.0-rc7-gfef64f93da2b/copyright 2023-10-11T09:25:59.781 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc7-gfef64f93da2b 2023-10-11T09:25:59.782 INFO:teuthology.task.kernel:searching for kernel 6.5.0-rc7-gfef64f93da2b 2023-10-11T09:25:59.782 DEBUG:teuthology.orchestra.run.smithi078:> egrep '(submenu|menuentry.*6.5.0-rc7-gfef64f93da2b).*{' /boot/grub/grub.cfg 2023-10-11T09:25:59.790 INFO:teuthology.orchestra.run.smithi078.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-10-11T09:25:59.790 INFO:teuthology.orchestra.run.smithi078.stdout: menuentry 'Ubuntu, with Linux 6.5.0-rc7-gfef64f93da2b' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.5.0-rc7-gfef64f93da2b-advanced-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-10-11T09:25:59.790 INFO:teuthology.orchestra.run.smithi078.stdout: menuentry 'Ubuntu, with Linux 6.5.0-rc7-gfef64f93da2b (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.5.0-rc7-gfef64f93da2b-recovery-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-10-11T09:25:59.791 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2023-10-11T09:25:59.791 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.5.0-rc7-gfef64f93da2b 2023-10-11T09:25:59.792 DEBUG:teuthology.orchestra.run.smithi078:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2023-10-11T09:25:59.794 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi078.front.sepia.ceph.com to complete... 2023-10-11T09:25:59.799 INFO:teuthology.orchestra.run.smithi133.stderr:Sourcing file `/etc/default/grub' 2023-10-11T09:25:59.809 INFO:teuthology.orchestra.run.smithi133.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-11T09:25:59.834 INFO:teuthology.orchestra.run.smithi133.stderr:Generating grub configuration file ... 2023-10-11T09:26:00.243 INFO:teuthology.orchestra.run.smithi133.stderr:Found linux image: /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b 2023-10-11T09:26:00.263 INFO:teuthology.orchestra.run.smithi133.stderr:Found initrd image: /boot/initrd.img-6.5.0-rc7-gfef64f93da2b 2023-10-11T09:26:00.521 INFO:teuthology.orchestra.run.smithi078.stderr:Sourcing file `/etc/default/grub' 2023-10-11T09:26:00.527 INFO:teuthology.orchestra.run.smithi078.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-11T09:26:00.547 INFO:teuthology.orchestra.run.smithi078.stderr:Generating grub configuration file ... 2023-10-11T09:26:00.639 INFO:teuthology.orchestra.run.smithi133.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-10-11T09:26:00.652 INFO:teuthology.orchestra.run.smithi133.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-10-11T09:26:00.831 INFO:teuthology.orchestra.run.smithi078.stderr:Found linux image: /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b 2023-10-11T09:26:00.842 INFO:teuthology.orchestra.run.smithi078.stderr:Found initrd image: /boot/initrd.img-6.5.0-rc7-gfef64f93da2b 2023-10-11T09:26:01.201 INFO:teuthology.orchestra.run.smithi078.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-10-11T09:26:01.213 INFO:teuthology.orchestra.run.smithi078.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-10-11T09:26:02.871 INFO:teuthology.orchestra.run.smithi078.stderr:done 2023-10-11T09:26:03.353 INFO:teuthology.orchestra.run.smithi133.stderr:done 2023-10-11T09:26:03.520 DEBUG:teuthology.orchestra.run.smithi133:> dpkg-deb -c /tmp/linux-image.deb 2023-10-11T09:26:03.530 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./ 2023-10-11T09:26:03.530 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./boot/ 2023-10-11T09:26:03.530 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 4382168 2023-09-11 02:20 ./boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-11T09:26:03.545 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 168454 2023-09-11 02:20 ./boot/config-6.5.0-rc7-gfef64f93da2b 2023-10-11T09:26:03.545 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11294080 2023-09-11 02:20 ./boot/vmlinuz-6.5.0-rc7-gfef64f93da2b 2023-10-11T09:26:03.566 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./etc/ 2023-10-11T09:26:03.566 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./etc/kernel/ 2023-10-11T09:26:03.567 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./etc/kernel/postinst.d/ 2023-10-11T09:26:03.567 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./etc/kernel/postrm.d/ 2023-10-11T09:26:03.567 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./etc/kernel/preinst.d/ 2023-10-11T09:26:03.567 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./etc/kernel/prerm.d/ 2023-10-11T09:26:03.568 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/ 2023-10-11T09:26:03.568 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/ 2023-10-11T09:26:03.568 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/ 2023-10-11T09:26:03.568 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/ 2023-10-11T09:26:03.568 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/ 2023-10-11T09:26:03.569 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/ 2023-10-11T09:26:03.569 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/ 2023-10-11T09:26:03.569 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 490385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/aesni-intel.ko 2023-10-11T09:26:03.571 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 25329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-11T09:26:03.572 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 29529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-11T09:26:03.572 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 30313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-11T09:26:03.572 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 93113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-11T09:26:03.572 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 106729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-11T09:26:03.572 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 152241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-11T09:26:03.572 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-11T09:26:03.573 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13777 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-11T09:26:03.573 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 83889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-11T09:26:03.573 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-11T09:26:03.573 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 47225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-11T09:26:03.573 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 47209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-11T09:26:03.574 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 51801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-11T09:26:03.574 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 38937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-11T09:26:03.574 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 40529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-11T09:26:03.574 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 58585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-11T09:26:03.574 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 50497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-11T09:26:03.575 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 32065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-11T09:26:03.575 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-11T09:26:03.575 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/ 2023-10-11T09:26:03.575 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/ 2023-10-11T09:26:03.575 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce/ 2023-10-11T09:26:03.576 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 40113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-11T09:26:03.576 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpuid.ko 2023-10-11T09:26:03.576 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/msr.ko 2023-10-11T09:26:03.576 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/ 2023-10-11T09:26:03.576 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 348201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-11T09:26:03.577 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 882105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-11T09:26:03.577 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 2277369 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm.ko 2023-10-11T09:26:03.639 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ 2023-10-11T09:26:03.640 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 53585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/af_alg.ko 2023-10-11T09:26:03.640 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 23097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/algif_hash.ko 2023-10-11T09:26:03.640 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/algif_skcipher.ko 2023-10-11T09:26:03.640 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ansi_cprng.ko 2023-10-11T09:26:03.640 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/anubis.ko 2023-10-11T09:26:03.641 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/arc4.ko 2023-10-11T09:26:03.641 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/asymmetric_keys/ 2023-10-11T09:26:03.641 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2023-10-11T09:26:03.641 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/ 2023-10-11T09:26:03.641 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-11T09:26:03.642 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_pq.ko 2023-10-11T09:26:03.642 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21561 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-11T09:26:03.642 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_tx.ko 2023-10-11T09:26:03.642 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_xor.ko 2023-10-11T09:26:03.642 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/raid6test.ko 2023-10-11T09:26:03.643 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authenc.ko 2023-10-11T09:26:03.643 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 22641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authencesn.ko 2023-10-11T09:26:03.643 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blake2b_generic.ko 2023-10-11T09:26:03.643 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_common.ko 2023-10-11T09:26:03.643 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_generic.ko 2023-10-11T09:26:03.644 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 38873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/camellia_generic.ko 2023-10-11T09:26:03.644 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 27321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast5_generic.ko 2023-10-11T09:26:03.644 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast6_generic.ko 2023-10-11T09:26:03.644 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast_common.ko 2023-10-11T09:26:03.644 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 35097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ccm.ko 2023-10-11T09:26:03.645 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cmac.ko 2023-10-11T09:26:03.645 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crc32_generic.ko 2023-10-11T09:26:03.645 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 46961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cryptd.ko 2023-10-11T09:26:03.645 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 25249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_engine.ko 2023-10-11T09:26:03.645 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_simd.ko 2023-10-11T09:26:03.646 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 24737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_user.ko 2023-10-11T09:26:03.646 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cts.ko 2023-10-11T09:26:03.646 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/des_generic.ko 2023-10-11T09:26:03.646 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/dh_generic.ko 2023-10-11T09:26:03.646 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/echainiv.ko 2023-10-11T09:26:03.647 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 25321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/essiv.ko 2023-10-11T09:26:03.647 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/fcrypt.ko 2023-10-11T09:26:03.647 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/khazad.ko 2023-10-11T09:26:03.647 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lrw.ko 2023-10-11T09:26:03.647 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4.ko 2023-10-11T09:26:03.648 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4hc.ko 2023-10-11T09:26:03.648 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11761 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/md4.ko 2023-10-11T09:26:03.648 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/michael_mic.ko 2023-10-11T09:26:03.648 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcbc.ko 2023-10-11T09:26:03.648 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcrypt.ko 2023-10-11T09:26:03.649 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/rmd160.ko 2023-10-11T09:26:03.649 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 29393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/seed.ko 2023-10-11T09:26:03.649 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 27081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/serpent_generic.ko 2023-10-11T09:26:03.649 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 149193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tcrypt.ko 2023-10-11T09:26:03.649 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tea.ko 2023-10-11T09:26:03.650 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 28777 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_common.ko 2023-10-11T09:26:03.650 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_generic.ko 2023-10-11T09:26:03.650 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/vmac.ko 2023-10-11T09:26:03.650 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 36185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/wp512.ko 2023-10-11T09:26:03.650 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xcbc.ko 2023-10-11T09:26:03.651 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xor.ko 2023-10-11T09:26:03.651 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xts.ko 2023-10-11T09:26:03.651 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9481 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xxhash_generic.ko 2023-10-11T09:26:03.651 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ 2023-10-11T09:26:03.651 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/ 2023-10-11T09:26:03.652 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_extlog.ko 2023-10-11T09:26:03.652 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-11T09:26:03.652 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 28793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_pad.ko 2023-10-11T09:26:03.652 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/apei/ 2023-10-11T09:26:03.652 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 31241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/apei/einj.ko 2023-10-11T09:26:03.652 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/ec_sys.ko 2023-10-11T09:26:03.653 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/platform_profile.ko 2023-10-11T09:26:03.653 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 24401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbs.ko 2023-10-11T09:26:03.653 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbshc.ko 2023-10-11T09:26:03.653 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 100401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/video.ko 2023-10-11T09:26:03.653 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ 2023-10-11T09:26:03.654 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/acard-ahci.ko 2023-10-11T09:26:03.654 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 88625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ahci.ko 2023-10-11T09:26:03.654 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ahci_platform.ko 2023-10-11T09:26:03.654 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 87841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/libahci.ko 2023-10-11T09:26:03.654 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 35049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/libahci_platform.ko 2023-10-11T09:26:03.655 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_acpi.ko 2023-10-11T09:26:03.655 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 30553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ali.ko 2023-10-11T09:26:03.655 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 33625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_amd.ko 2023-10-11T09:26:03.655 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_artop.ko 2023-10-11T09:26:03.655 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atiixp.ko 2023-10-11T09:26:03.656 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 24809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atp867x.ko 2023-10-11T09:26:03.656 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd640.ko 2023-10-11T09:26:03.656 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 25241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-11T09:26:03.656 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cypress.ko 2023-10-11T09:26:03.656 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_efar.ko 2023-10-11T09:26:03.657 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt366.ko 2023-10-11T09:26:03.657 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 34033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-11T09:26:03.657 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 23057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-11T09:26:03.657 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-11T09:26:03.657 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it8213.ko 2023-10-11T09:26:03.657 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 35177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it821x.ko 2023-10-11T09:26:03.658 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_jmicron.ko 2023-10-11T09:26:03.658 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 51737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_legacy.ko 2023-10-11T09:26:03.658 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_marvell.ko 2023-10-11T09:26:03.658 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_mpiix.ko 2023-10-11T09:26:03.658 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_netcell.ko 2023-10-11T09:26:03.659 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ninja32.ko 2023-10-11T09:26:03.659 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87410.ko 2023-10-11T09:26:03.659 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87415.ko 2023-10-11T09:26:03.659 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-11T09:26:03.659 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_opti.ko 2023-10-11T09:26:03.660 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_optidma.ko 2023-10-11T09:26:03.660 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 32521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-11T09:26:03.660 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 37729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-11T09:26:03.660 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-11T09:26:03.660 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_piccolo.ko 2023-10-11T09:26:03.661 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_radisys.ko 2023-10-11T09:26:03.661 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_rdc.ko 2023-10-11T09:26:03.661 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_rz1000.ko 2023-10-11T09:26:03.661 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sch.ko 2023-10-11T09:26:03.661 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 23001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_serverworks.ko 2023-10-11T09:26:03.661 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 23385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sil680.ko 2023-10-11T09:26:03.662 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-11T09:26:03.662 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_triflex.ko 2023-10-11T09:26:03.662 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 30425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_via.ko 2023-10-11T09:26:03.662 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 22729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pdc_adma.ko 2023-10-11T09:26:03.662 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 28993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_inic162x.ko 2023-10-11T09:26:03.663 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 83225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_mv.ko 2023-10-11T09:26:03.663 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 70537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_nv.ko 2023-10-11T09:26:03.663 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 37369 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_promise.ko 2023-10-11T09:26:03.663 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 22969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_qstor.ko 2023-10-11T09:26:03.663 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 29201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil.ko 2023-10-11T09:26:03.664 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 38273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil24.ko 2023-10-11T09:26:03.664 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sis.ko 2023-10-11T09:26:03.664 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_svw.ko 2023-10-11T09:26:03.664 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 44721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sx4.ko 2023-10-11T09:26:03.664 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_uli.ko 2023-10-11T09:26:03.665 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 35121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_via.ko 2023-10-11T09:26:03.665 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_vsc.ko 2023-10-11T09:26:03.665 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay/ 2023-10-11T09:26:03.665 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-11T09:26:03.665 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma/ 2023-10-11T09:26:03.666 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 124457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma/bcma.ko 2023-10-11T09:26:03.666 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/ 2023-10-11T09:26:03.666 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe/ 2023-10-11T09:26:03.666 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 113705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe/aoe.ko 2023-10-11T09:26:03.666 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd/ 2023-10-11T09:26:03.667 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 968433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd/drbd.ko 2023-10-11T09:26:03.667 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 173201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/floppy.ko 2023-10-11T09:26:03.667 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx/ 2023-10-11T09:26:03.667 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 94297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-11T09:26:03.667 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 115345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/nbd.ko 2023-10-11T09:26:03.668 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk/ 2023-10-11T09:26:03.668 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 110361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-11T09:26:03.668 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 98225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/pktcdvd.ko 2023-10-11T09:26:03.668 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 259401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/rbd.ko 2023-10-11T09:26:03.668 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback/ 2023-10-11T09:26:03.669 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 98129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-11T09:26:03.669 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram/ 2023-10-11T09:26:03.669 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 60265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram/zram.ko 2023-10-11T09:26:03.669 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ 2023-10-11T09:26:03.669 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 30849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/applicom.ko 2023-10-11T09:26:03.670 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hangcheck-timer.ko 2023-10-11T09:26:03.670 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/ 2023-10-11T09:26:03.670 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-11T09:26:03.670 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-11T09:26:03.670 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-11T09:26:03.671 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-11T09:26:03.671 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15633 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-11T09:26:03.671 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ 2023-10-11T09:26:03.671 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 30857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-11T09:26:03.671 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 150665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-11T09:26:03.672 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26561 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-11T09:26:03.672 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 139057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-11T09:26:03.672 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 70833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-11T09:26:03.672 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 53609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-11T09:26:03.672 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 36321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/lp.ko 2023-10-11T09:26:03.673 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave/ 2023-10-11T09:26:03.673 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 84121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave/mwave.ko 2023-10-11T09:26:03.673 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 25873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/nvram.ko 2023-10-11T09:26:03.673 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 32169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ppdev.ko 2023-10-11T09:26:03.673 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 51113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tlclk.ko 2023-10-11T09:26:03.674 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/ 2023-10-11T09:26:03.674 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-11T09:26:03.674 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2023-10-11T09:26:03.674 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 23473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2023-10-11T09:26:03.674 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 32985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2023-10-11T09:26:03.675 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 30121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-11T09:26:03.675 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 22697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-11T09:26:03.675 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 23561 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/xen-tpmfront.ko 2023-10-11T09:26:03.675 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/xillybus/ 2023-10-11T09:26:03.675 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/xillybus/xillybus_class.ko 2023-10-11T09:26:03.676 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 50945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/xillybus/xillybus_core.ko 2023-10-11T09:26:03.676 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/xillybus/xillybus_pcie.ko 2023-10-11T09:26:03.676 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/clk/ 2023-10-11T09:26:03.676 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/clk/clk-palmas.ko 2023-10-11T09:26:03.676 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 47849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/clk/clk-si5351.ko 2023-10-11T09:26:03.677 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/clk/clk-twl6040.ko 2023-10-11T09:26:03.677 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/clk/clk-wm831x.ko 2023-10-11T09:26:03.677 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/ 2023-10-11T09:26:03.677 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-11T09:26:03.677 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-11T09:26:03.677 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 25297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-11T09:26:03.678 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ 2023-10-11T09:26:03.678 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ 2023-10-11T09:26:03.678 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 70785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-11T09:26:03.678 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 234473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-11T09:26:03.678 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/ 2023-10-11T09:26:03.679 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/ 2023-10-11T09:26:03.679 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common/ 2023-10-11T09:26:03.679 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 408489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-11T09:26:03.679 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2023-10-11T09:26:03.679 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 23657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-11T09:26:03.680 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-aes.ko 2023-10-11T09:26:03.680 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-sha.ko 2023-10-11T09:26:03.680 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio/ 2023-10-11T09:26:03.680 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 68937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-11T09:26:03.680 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca/ 2023-10-11T09:26:03.681 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 29785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca/dca.ko 2023-10-11T09:26:03.681 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ 2023-10-11T09:26:03.681 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw/ 2023-10-11T09:26:03.681 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-11T09:26:03.681 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat/ 2023-10-11T09:26:03.681 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 122641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-11T09:26:03.682 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/ 2023-10-11T09:26:03.682 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 80273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/amd64_edac.ko 2023-10-11T09:26:03.682 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 35769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/e752x_edac.ko 2023-10-11T09:26:03.682 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 63617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-11T09:26:03.683 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3000_edac.ko 2023-10-11T09:26:03.683 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3200_edac.ko 2023-10-11T09:26:03.683 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 22537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5100_edac.ko 2023-10-11T09:26:03.683 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 25657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5400_edac.ko 2023-10-11T09:26:03.683 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 24913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7300_edac.ko 2023-10-11T09:26:03.683 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 52025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7core_edac.ko 2023-10-11T09:26:03.684 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i82975x_edac.ko 2023-10-11T09:26:03.684 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/ie31200_edac.ko 2023-10-11T09:26:03.684 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 56809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/sb_edac.ko 2023-10-11T09:26:03.684 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/x38_edac.ko 2023-10-11T09:26:03.684 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/extcon/ 2023-10-11T09:26:03.684 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/extcon/extcon-gpio.ko 2023-10-11T09:26:03.685 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 27185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/extcon/extcon-max14577.ko 2023-10-11T09:26:03.685 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 29569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/extcon/extcon-max77693.ko 2023-10-11T09:26:03.685 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 24745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/extcon/extcon-max8997.ko 2023-10-11T09:26:03.685 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/extcon/extcon-palmas.ko 2023-10-11T09:26:03.685 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 23713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/extcon/extcon-rt8973a.ko 2023-10-11T09:26:03.685 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 27785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/extcon/extcon-sm5502.ko 2023-10-11T09:26:03.686 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/ 2023-10-11T09:26:03.686 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 34953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/dmi-sysfs.ko 2023-10-11T09:26:03.686 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/efi/ 2023-10-11T09:26:03.686 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/efi/efi-pstore.ko 2023-10-11T09:26:03.686 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-11T09:26:03.687 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/ 2023-10-11T09:26:03.687 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-adp5520.ko 2023-10-11T09:26:03.687 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-amd8111.ko 2023-10-11T09:26:03.687 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-arizona.ko 2023-10-11T09:26:03.687 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-da9052.ko 2023-10-11T09:26:03.687 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-da9055.ko 2023-10-11T09:26:03.688 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 22065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-dln2.ko 2023-10-11T09:26:03.688 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 77849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-f7188x.ko 2023-10-11T09:26:03.688 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 28649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-generic.ko 2023-10-11T09:26:03.688 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-ich.ko 2023-10-11T09:26:03.688 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-janz-ttl.ko 2023-10-11T09:26:03.689 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-kempld.ko 2023-10-11T09:26:03.689 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-lp3943.ko 2023-10-11T09:26:03.689 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-max7300.ko 2023-10-11T09:26:03.689 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-max730x.ko 2023-10-11T09:26:03.689 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-max732x.ko 2023-10-11T09:26:03.690 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 25145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-ml-ioh.ko 2023-10-11T09:26:03.690 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 46761 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-pca953x.ko 2023-10-11T09:26:03.690 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-pcf857x.ko 2023-10-11T09:26:03.690 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14377 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-rdc321x.ko 2023-10-11T09:26:03.690 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-sch.ko 2023-10-11T09:26:03.691 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-sch311x.ko 2023-10-11T09:26:03.691 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-tps65912.ko 2023-10-11T09:26:03.691 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-twl4030.ko 2023-10-11T09:26:03.691 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-twl6040.ko 2023-10-11T09:26:03.691 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-11T09:26:03.691 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-vx855.ko 2023-10-11T09:26:03.692 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-wm831x.ko 2023-10-11T09:26:03.692 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-wm8350.ko 2023-10-11T09:26:03.692 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-wm8994.ko 2023-10-11T09:26:03.692 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/ 2023-10-11T09:26:03.692 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-a4tech.ko 2023-10-11T09:26:03.693 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 39081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-apple.ko 2023-10-11T09:26:03.693 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14481 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-appleir.ko 2023-10-11T09:26:03.693 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-aureal.ko 2023-10-11T09:26:03.693 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-axff.ko 2023-10-11T09:26:03.693 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-belkin.ko 2023-10-11T09:26:03.694 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-cherry.ko 2023-10-11T09:26:03.694 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-chicony.ko 2023-10-11T09:26:03.694 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 59145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-cp2112.ko 2023-10-11T09:26:03.694 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-cypress.ko 2023-10-11T09:26:03.694 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-dr.ko 2023-10-11T09:26:03.694 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elecom.ko 2023-10-11T09:26:03.695 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elo.ko 2023-10-11T09:26:03.695 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-emsff.ko 2023-10-11T09:26:03.695 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-ezkey.ko 2023-10-11T09:26:03.695 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gaff.ko 2023-10-11T09:26:03.695 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-generic.ko 2023-10-11T09:26:03.695 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gt683r.ko 2023-10-11T09:26:03.696 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gyration.ko 2023-10-11T09:26:03.696 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-11T09:26:03.696 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-11T09:26:03.696 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtekff.ko 2023-10-11T09:26:03.696 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-hyperv.ko 2023-10-11T09:26:03.696 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-icade.ko 2023-10-11T09:26:03.697 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-kensington.ko 2023-10-11T09:26:03.697 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-keytouch.ko 2023-10-11T09:26:03.697 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-kye.ko 2023-10-11T09:26:03.697 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lcpower.ko 2023-10-11T09:26:03.697 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-led.ko 2023-10-11T09:26:03.698 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 42297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lenovo.ko 2023-10-11T09:26:03.698 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 30761 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lg-g15.ko 2023-10-11T09:26:03.698 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 57809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-11T09:26:03.698 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 101009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-11T09:26:03.698 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 80993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech.ko 2023-10-11T09:26:03.699 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 30801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-magicmouse.ko 2023-10-11T09:26:03.699 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-microsoft.ko 2023-10-11T09:26:03.699 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-monterey.ko 2023-10-11T09:26:03.699 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 50153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-multitouch.ko 2023-10-11T09:26:03.699 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 32593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-ntrig.ko 2023-10-11T09:26:03.700 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-ortek.ko 2023-10-11T09:26:03.700 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-penmount.ko 2023-10-11T09:26:03.700 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-petalynx.ko 2023-10-11T09:26:03.700 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 72937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-picolcd.ko 2023-10-11T09:26:03.700 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-pl.ko 2023-10-11T09:26:03.701 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-plantronics.ko 2023-10-11T09:26:03.701 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-primax.ko 2023-10-11T09:26:03.701 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 35457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-rmi.ko 2023-10-11T09:26:03.701 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20761 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-11T09:26:03.701 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-11T09:26:03.701 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 31361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-11T09:26:03.702 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 29657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-11T09:26:03.702 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 31065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-11T09:26:03.702 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-11T09:26:03.702 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 29417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-11T09:26:03.702 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-11T09:26:03.703 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 29241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-11T09:26:03.703 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 27889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-11T09:26:03.703 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-11T09:26:03.703 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 23393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat.ko 2023-10-11T09:26:03.703 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-saitek.ko 2023-10-11T09:26:03.703 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-samsung.ko 2023-10-11T09:26:03.704 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 36801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-11T09:26:03.704 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sjoy.ko 2023-10-11T09:26:03.704 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 51201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sony.ko 2023-10-11T09:26:03.704 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-speedlink.ko 2023-10-11T09:26:03.704 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-steelseries.ko 2023-10-11T09:26:03.705 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sunplus.ko 2023-10-11T09:26:03.705 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-11T09:26:03.705 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tivo.ko 2023-10-11T09:26:03.705 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tmff.ko 2023-10-11T09:26:03.705 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-topseed.ko 2023-10-11T09:26:03.705 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-twinhan.ko 2023-10-11T09:26:03.706 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 71457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-uclogic.ko 2023-10-11T09:26:03.706 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-waltop.ko 2023-10-11T09:26:03.706 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 143585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-wiimote.ko 2023-10-11T09:26:03.706 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-xinmo.ko 2023-10-11T09:26:03.707 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zpff.ko 2023-10-11T09:26:03.707 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zydacron.ko 2023-10-11T09:26:03.707 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 267689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid.ko 2023-10-11T09:26:03.707 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 32321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/uhid.ko 2023-10-11T09:26:03.707 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/usbhid/ 2023-10-11T09:26:03.707 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 125721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/usbhid/usbhid.ko 2023-10-11T09:26:03.708 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/usbhid/usbkbd.ko 2023-10-11T09:26:03.708 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/usbhid/usbmouse.ko 2023-10-11T09:26:03.708 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 217401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/wacom.ko 2023-10-11T09:26:03.708 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/ 2023-10-11T09:26:03.708 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 63497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_balloon.ko 2023-10-11T09:26:03.709 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 89609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_utils.ko 2023-10-11T09:26:03.709 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 301449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_vmbus.ko 2023-10-11T09:26:03.709 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ 2023-10-11T09:26:03.709 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 66593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru.ko 2023-10-11T09:26:03.709 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 61393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru3.ko 2023-10-11T09:26:03.710 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 34745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-11T09:26:03.710 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7414.ko 2023-10-11T09:26:03.710 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7418.ko 2023-10-11T09:26:03.710 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 25257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adc128d818.ko 2023-10-11T09:26:03.710 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 31729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1025.ko 2023-10-11T09:26:03.711 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 97977 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1026.ko 2023-10-11T09:26:03.711 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1029.ko 2023-10-11T09:26:03.711 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 45553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1031.ko 2023-10-11T09:26:03.711 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm9240.ko 2023-10-11T09:26:03.711 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ads7828.ko 2023-10-11T09:26:03.712 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7410.ko 2023-10-11T09:26:03.712 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 22049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7411.ko 2023-10-11T09:26:03.712 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 74433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7462.ko 2023-10-11T09:26:03.712 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 39585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7470.ko 2023-10-11T09:26:03.712 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 67929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7475.ko 2023-10-11T09:26:03.713 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7x10.ko 2023-10-11T09:26:03.713 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 39073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/amc6821.ko 2023-10-11T09:26:03.713 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 55265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/applesmc.ko 2023-10-11T09:26:03.713 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 49913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asb100.ko 2023-10-11T09:26:03.713 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 58617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asc7621.ko 2023-10-11T09:26:03.713 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 40881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-11T09:26:03.714 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/atxp1.ko 2023-10-11T09:26:03.714 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 28281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/coretemp.ko 2023-10-11T09:26:03.714 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 24657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/da9052-hwmon.ko 2023-10-11T09:26:03.714 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/da9055-hwmon.ko 2023-10-11T09:26:03.714 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 44465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-11T09:26:03.715 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 99601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dme1737.ko 2023-10-11T09:26:03.715 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds1621.ko 2023-10-11T09:26:03.715 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds620.ko 2023-10-11T09:26:03.715 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 29777 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc1403.ko 2023-10-11T09:26:03.715 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 33185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc2103.ko 2023-10-11T09:26:03.716 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 30321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc6w201.ko 2023-10-11T09:26:03.716 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 66985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71805f.ko 2023-10-11T09:26:03.716 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 94457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71882fg.ko 2023-10-11T09:26:03.716 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 41401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f75375s.ko 2023-10-11T09:26:03.716 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-11T09:26:03.717 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 55433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fschmd.ko 2023-10-11T09:26:03.717 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14481 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g760a.ko 2023-10-11T09:26:03.717 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 24017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g762.ko 2023-10-11T09:26:03.717 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 46801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl518sm.ko 2023-10-11T09:26:03.717 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 45529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl520sm.ko 2023-10-11T09:26:03.718 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/hih6130.ko 2023-10-11T09:26:03.718 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-11T09:26:03.718 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-11T09:26:03.718 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 22601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-11T09:26:03.718 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 34505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmaem.ko 2023-10-11T09:26:03.718 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmpex.ko 2023-10-11T09:26:03.719 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 24873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina209.ko 2023-10-11T09:26:03.719 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina2xx.ko 2023-10-11T09:26:03.719 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 129217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/it87.ko 2023-10-11T09:26:03.719 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/jc42.ko 2023-10-11T09:26:03.719 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 24025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k10temp.ko 2023-10-11T09:26:03.720 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k8temp.ko 2023-10-11T09:26:03.720 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-11T09:26:03.720 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 48601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm63.ko 2023-10-11T09:26:03.720 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16761 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm73.ko 2023-10-11T09:26:03.720 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 36777 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm75.ko 2023-10-11T09:26:03.721 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm77.ko 2023-10-11T09:26:03.721 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 34657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm78.ko 2023-10-11T09:26:03.721 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 31745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm80.ko 2023-10-11T09:26:03.721 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm83.ko 2023-10-11T09:26:03.721 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 72833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm85.ko 2023-10-11T09:26:03.721 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 46153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm87.ko 2023-10-11T09:26:03.722 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 37913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm90.ko 2023-10-11T09:26:03.722 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm92.ko 2023-10-11T09:26:03.722 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 108841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm93.ko 2023-10-11T09:26:03.722 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 33393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95234.ko 2023-10-11T09:26:03.722 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95241.ko 2023-10-11T09:26:03.723 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95245.ko 2023-10-11T09:26:03.723 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 24017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc2945.ko 2023-10-11T09:26:03.723 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4151.ko 2023-10-11T09:26:03.723 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4215.ko 2023-10-11T09:26:03.723 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4222.ko 2023-10-11T09:26:03.724 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4245.ko 2023-10-11T09:26:03.724 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4260.ko 2023-10-11T09:26:03.724 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4261.ko 2023-10-11T09:26:03.724 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 33489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max16065.ko 2023-10-11T09:26:03.724 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1619.ko 2023-10-11T09:26:03.725 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 24353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1668.ko 2023-10-11T09:26:03.725 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max197.ko 2023-10-11T09:26:03.725 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6639.ko 2023-10-11T09:26:03.725 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 23321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6650.ko 2023-10-11T09:26:03.725 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 35777 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6697.ko 2023-10-11T09:26:03.725 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/mc13783-adc.ko 2023-10-11T09:26:03.726 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/mcp3021.ko 2023-10-11T09:26:03.726 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2023-10-11T09:26:03.726 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 44129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6683.ko 2023-10-11T09:26:03.726 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 116697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-11T09:26:03.726 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 60705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775.ko 2023-10-11T09:26:03.727 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 59737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct7802.ko 2023-10-11T09:26:03.727 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 82337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87360.ko 2023-10-11T09:26:03.727 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 54433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87427.ko 2023-10-11T09:26:03.727 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pcf8591.ko 2023-10-11T09:26:03.727 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ 2023-10-11T09:26:03.728 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-11T09:26:03.728 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-11T09:26:03.728 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 28329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-11T09:26:03.728 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-11T09:26:03.728 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-11T09:26:03.729 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-11T09:26:03.729 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-11T09:26:03.729 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 81841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-11T09:26:03.729 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-11T09:26:03.729 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 30729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-11T09:26:03.729 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-11T09:26:03.730 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-11T09:26:03.730 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/powr1220.ko 2023-10-11T09:26:03.730 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5627.ko 2023-10-11T09:26:03.730 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 28817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5636.ko 2023-10-11T09:26:03.730 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 27145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-11T09:26:03.731 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 32137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht15.ko 2023-10-11T09:26:03.731 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht21.ko 2023-10-11T09:26:03.731 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/shtc1.ko 2023-10-11T09:26:03.731 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 40105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sis5595.ko 2023-10-11T09:26:03.731 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 31929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smm665.ko 2023-10-11T09:26:03.731 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-11T09:26:03.732 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 39193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-11T09:26:03.732 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 37385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-11T09:26:03.732 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/thmc50.ko 2023-10-11T09:26:03.732 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp102.ko 2023-10-11T09:26:03.732 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp103.ko 2023-10-11T09:26:03.733 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp401.ko 2023-10-11T09:26:03.733 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp421.ko 2023-10-11T09:26:03.733 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-11T09:26:03.733 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 41945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via686a.ko 2023-10-11T09:26:03.733 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 52457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt1211.ko 2023-10-11T09:26:03.733 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 51545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt8231.ko 2023-10-11T09:26:03.734 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 57513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-11T09:26:03.734 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 78873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627hf.ko 2023-10-11T09:26:03.734 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 71345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83781d.ko 2023-10-11T09:26:03.734 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 63289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83791d.ko 2023-10-11T09:26:03.734 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 71321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83792d.ko 2023-10-11T09:26:03.735 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 101457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83793.ko 2023-10-11T09:26:03.735 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 119505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83795.ko 2023-10-11T09:26:03.735 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-11T09:26:03.735 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 35625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-11T09:26:03.735 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/wm831x-hwmon.ko 2023-10-11T09:26:03.735 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/wm8350-hwmon.ko 2023-10-11T09:26:03.736 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/ 2023-10-11T09:26:03.736 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/ 2023-10-11T09:26:03.736 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-11T09:26:03.736 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-11T09:26:03.736 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/ 2023-10-11T09:26:03.737 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 27969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ali1535.ko 2023-10-11T09:26:03.737 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 28801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ali1563.ko 2023-10-11T09:26:03.737 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 27777 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2023-10-11T09:26:03.737 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-11T09:26:03.737 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-11T09:26:03.738 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-11T09:26:03.738 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2023-10-11T09:26:03.738 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 6929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-11T09:26:03.738 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2023-10-11T09:26:03.738 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 48641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-11T09:26:03.739 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 23905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-11T09:26:03.739 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-11T09:26:03.739 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-11T09:26:03.739 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-dln2.ko 2023-10-11T09:26:03.739 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-gpio.ko 2023-10-11T09:26:03.740 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 62729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-11T09:26:03.740 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 22433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-11T09:26:03.740 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 47889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-11T09:26:03.740 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17369 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-kempld.ko 2023-10-11T09:26:03.740 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-11T09:26:03.741 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 22337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-11T09:26:03.741 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ocores.ko 2023-10-11T09:26:03.741 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 25121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-11T09:26:03.741 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-11T09:26:03.741 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 47241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-11T09:26:03.741 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2023-10-11T09:26:03.742 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-11T09:26:03.742 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-11T09:26:03.742 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 22313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-sis5595.ko 2023-10-11T09:26:03.742 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 29361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-sis630.ko 2023-10-11T09:26:03.742 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-11T09:26:03.743 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2023-10-11T09:26:03.743 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-11T09:26:03.743 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-11T09:26:03.743 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-11T09:26:03.743 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-11T09:26:03.744 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 47593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-xiic.ko 2023-10-11T09:26:03.744 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 22385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-mux.ko 2023-10-11T09:26:03.744 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-11T09:26:03.744 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 27065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-stub.ko 2023-10-11T09:26:03.744 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/muxes/ 2023-10-11T09:26:03.745 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2023-10-11T09:26:03.745 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2023-10-11T09:26:03.745 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2023-10-11T09:26:03.745 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ 2023-10-11T09:26:03.745 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ 2023-10-11T09:26:03.745 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 273641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-11T09:26:03.746 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 876913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-11T09:26:03.746 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 69321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-11T09:26:03.746 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 324145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-11T09:26:03.746 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 97961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-11T09:26:03.746 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 279345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-11T09:26:03.747 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 66177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-11T09:26:03.747 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/ 2023-10-11T09:26:03.747 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4/ 2023-10-11T09:26:03.747 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 492569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-11T09:26:03.747 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4/ 2023-10-11T09:26:03.747 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 419193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-11T09:26:03.748 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5/ 2023-10-11T09:26:03.748 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 739705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-11T09:26:03.748 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca/ 2023-10-11T09:26:03.748 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 296649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-11T09:26:03.748 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/ocrdma/ 2023-10-11T09:26:03.749 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 153681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2023-10-11T09:26:03.749 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/usnic/ 2023-10-11T09:26:03.749 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 169017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2023-10-11T09:26:03.749 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ 2023-10-11T09:26:03.749 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib/ 2023-10-11T09:26:03.750 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 263177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-11T09:26:03.750 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser/ 2023-10-11T09:26:03.750 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 100225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-11T09:26:03.750 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert/ 2023-10-11T09:26:03.750 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 111993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-11T09:26:03.751 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp/ 2023-10-11T09:26:03.751 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 129081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-11T09:26:03.751 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt/ 2023-10-11T09:26:03.751 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 129553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-11T09:26:03.751 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/ 2023-10-11T09:26:03.752 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/evbug.ko 2023-10-11T09:26:03.752 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 24129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/ff-memless.ko 2023-10-11T09:26:03.752 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/ 2023-10-11T09:26:03.752 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-11T09:26:03.752 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-11T09:26:03.753 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 41217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/gameport.ko 2023-10-11T09:26:03.753 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/lightning.ko 2023-10-11T09:26:03.753 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/ns558.ko 2023-10-11T09:26:03.753 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 41825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/joydev.ko 2023-10-11T09:26:03.753 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/ 2023-10-11T09:26:03.753 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/adp5520-keys.ko 2023-10-11T09:26:03.754 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 30441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/adp5588-keys.ko 2023-10-11T09:26:03.754 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/adp5589-keys.ko 2023-10-11T09:26:03.754 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 24761 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2023-10-11T09:26:03.754 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 36649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-11T09:26:03.754 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2023-10-11T09:26:03.755 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/lkkbd.ko 2023-10-11T09:26:03.755 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 28121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/lm8323.ko 2023-10-11T09:26:03.755 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/lm8333.ko 2023-10-11T09:26:03.755 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 24265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/matrix_keypad.ko 2023-10-11T09:26:03.755 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/max7359_keypad.ko 2023-10-11T09:26:03.756 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/mcs_touchkey.ko 2023-10-11T09:26:03.756 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2023-10-11T09:26:03.756 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/newtonkbd.ko 2023-10-11T09:26:03.756 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/opencores-kbd.ko 2023-10-11T09:26:03.756 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/qt1070.ko 2023-10-11T09:26:03.757 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/qt2160.ko 2023-10-11T09:26:03.757 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 23817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/samsung-keypad.ko 2023-10-11T09:26:03.757 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11481 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/stowaway.ko 2023-10-11T09:26:03.757 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/sunkbd.ko 2023-10-11T09:26:03.757 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/tca6416-keypad.ko 2023-10-11T09:26:03.757 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/tca8418_keypad.ko 2023-10-11T09:26:03.758 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/twl4030_keypad.ko 2023-10-11T09:26:03.758 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/xtkbd.ko 2023-10-11T09:26:03.758 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/matrix-keymap.ko 2023-10-11T09:26:03.758 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/ 2023-10-11T09:26:03.758 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 34329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/appletouch.ko 2023-10-11T09:26:03.759 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 34177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-11T09:26:03.759 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 123681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-11T09:26:03.759 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 89449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-11T09:26:03.759 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/gpio_mouse.ko 2023-10-11T09:26:03.759 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 335849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/psmouse.ko 2023-10-11T09:26:03.760 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/sermouse.ko 2023-10-11T09:26:03.760 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 23881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-11T09:26:03.760 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 22569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-11T09:26:03.760 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-11T09:26:03.760 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4/ 2023-10-11T09:26:03.760 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 109849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-11T09:26:03.761 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/ 2023-10-11T09:26:03.761 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-11T09:26:03.761 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-11T09:26:03.761 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/ct82c710.ko 2023-10-11T09:26:03.761 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-11T09:26:03.762 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/parkbd.ko 2023-10-11T09:26:03.762 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/pcips2.ko 2023-10-11T09:26:03.762 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/ps2mult.ko 2023-10-11T09:26:03.762 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 24129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/serio_raw.ko 2023-10-11T09:26:03.762 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17977 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/serport.ko 2023-10-11T09:26:03.763 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/sparse-keymap.ko 2023-10-11T09:26:03.763 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/ 2023-10-11T09:26:03.763 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd/ 2023-10-11T09:26:03.763 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 38857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-11T09:26:03.763 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/ 2023-10-11T09:26:03.764 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-88pm860x.ko 2023-10-11T09:26:03.764 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-adp5520.ko 2023-10-11T09:26:03.764 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 59753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-bd2802.ko 2023-10-11T09:26:03.764 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 24785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-blinkm.ko 2023-10-11T09:26:03.764 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-da903x.ko 2023-10-11T09:26:03.765 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-da9052.ko 2023-10-11T09:26:03.765 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16561 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-gpio.ko 2023-10-11T09:26:03.765 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm3530.ko 2023-10-11T09:26:03.765 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 30505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm3533.ko 2023-10-11T09:26:03.765 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm355x.ko 2023-10-11T09:26:03.765 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm3642.ko 2023-10-11T09:26:03.766 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lp3944.ko 2023-10-11T09:26:03.766 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lp8788.ko 2023-10-11T09:26:03.766 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-max8997.ko 2023-10-11T09:26:03.766 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-mc13783.ko 2023-10-11T09:26:03.766 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-menf21bmc.ko 2023-10-11T09:26:03.767 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-pca9532.ko 2023-10-11T09:26:03.767 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20481 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-pca955x.ko 2023-10-11T09:26:03.767 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-pca963x.ko 2023-10-11T09:26:03.767 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 34433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-ss4200.ko 2023-10-11T09:26:03.767 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 24865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-tca6507.ko 2023-10-11T09:26:03.768 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-wm831x-status.ko 2023-10-11T09:26:03.768 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-wm8350.ko 2023-10-11T09:26:03.768 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ 2023-10-11T09:26:03.768 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-11T09:26:03.768 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-11T09:26:03.769 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-11T09:26:03.769 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 7617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-11T09:26:03.769 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-11T09:26:03.769 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-11T09:26:03.769 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-11T09:26:03.769 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-11T09:26:03.770 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/ 2023-10-11T09:26:03.770 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache/ 2023-10-11T09:26:03.770 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 655473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache/bcache.ko 2023-10-11T09:26:03.770 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 40569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-bio-prison.ko 2023-10-11T09:26:03.770 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 89601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-bufio.ko 2023-10-11T09:26:03.771 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 47529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache-smq.ko 2023-10-11T09:26:03.771 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 155993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache.ko 2023-10-11T09:26:03.771 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 107049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-crypt.ko 2023-10-11T09:26:03.771 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-delay.ko 2023-10-11T09:26:03.771 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 53897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-era.ko 2023-10-11T09:26:03.772 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 28529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-flakey.ko 2023-10-11T09:26:03.772 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 41593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-log-userspace.ko 2023-10-11T09:26:03.772 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 32721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-log.ko 2023-10-11T09:26:03.772 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 49545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-mirror.ko 2023-10-11T09:26:03.772 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 78017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-multipath.ko 2023-10-11T09:26:03.773 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-queue-length.ko 2023-10-11T09:26:03.773 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 70953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-raid.ko 2023-10-11T09:26:03.773 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 36273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-region-hash.ko 2023-10-11T09:26:03.773 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-round-robin.ko 2023-10-11T09:26:03.773 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-service-time.ko 2023-10-11T09:26:03.773 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 109593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-snapshot.ko 2023-10-11T09:26:03.774 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-switch.ko 2023-10-11T09:26:03.774 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 181585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-thin-pool.ko 2023-10-11T09:26:03.774 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 51737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-verity.ko 2023-10-11T09:26:03.774 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-zero.ko 2023-10-11T09:26:03.774 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/faulty.ko 2023-10-11T09:26:03.775 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/linear.ko 2023-10-11T09:26:03.775 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 32305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/multipath.ko 2023-10-11T09:26:03.775 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data/ 2023-10-11T09:26:03.775 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 199937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-11T09:26:03.775 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 37321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid0.ko 2023-10-11T09:26:03.776 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 112609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid1.ko 2023-10-11T09:26:03.776 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 160553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid10.ko 2023-10-11T09:26:03.776 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 396153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid456.ko 2023-10-11T09:26:03.776 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/ 2023-10-11T09:26:03.776 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/ 2023-10-11T09:26:03.776 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 178113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptbase.ko 2023-10-11T09:26:03.777 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 63385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptctl.ko 2023-10-11T09:26:03.777 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 44649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptfc.ko 2023-10-11T09:26:03.777 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 37161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptlan.ko 2023-10-11T09:26:03.777 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 123353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptsas.ko 2023-10-11T09:26:03.777 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 79017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-11T09:26:03.778 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 53081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptspi.ko 2023-10-11T09:26:03.778 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/ 2023-10-11T09:26:03.778 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/88pm800.ko 2023-10-11T09:26:03.778 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12761 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/88pm805.ko 2023-10-11T09:26:03.778 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/88pm80x.ko 2023-10-11T09:26:03.779 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/arizona-i2c.ko 2023-10-11T09:26:03.779 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 123721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/arizona.ko 2023-10-11T09:26:03.779 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/bcm590xx.ko 2023-10-11T09:26:03.779 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/cros_ec_dev.ko 2023-10-11T09:26:03.779 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 34993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/dln2.ko 2023-10-11T09:26:03.780 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/janz-cmodio.ko 2023-10-11T09:26:03.780 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 53585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/kempld-core.ko 2023-10-11T09:26:03.780 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 27529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lm3533-core.ko 2023-10-11T09:26:03.780 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lm3533-ctrlbank.ko 2023-10-11T09:26:03.780 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lp3943.ko 2023-10-11T09:26:03.780 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 54569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_ich.ko 2023-10-11T09:26:03.781 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_sch.ko 2023-10-11T09:26:03.781 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/max8907.ko 2023-10-11T09:26:03.781 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 29857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/mc13xxx-core.ko 2023-10-11T09:26:03.781 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/mc13xxx-i2c.ko 2023-10-11T09:26:03.781 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/menf21bmc.ko 2023-10-11T09:26:03.782 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/pcf50633-adc.ko 2023-10-11T09:26:03.782 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/pcf50633-gpio.ko 2023-10-11T09:26:03.782 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 34873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/pcf50633.ko 2023-10-11T09:26:03.782 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/rdc321x-southbridge.ko 2023-10-11T09:26:03.782 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/retu-mfd.ko 2023-10-11T09:26:03.783 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 80425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/si476x-core.ko 2023-10-11T09:26:03.783 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 48441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/sm501.ko 2023-10-11T09:26:03.783 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/ti_am335x_tscadc.ko 2023-10-11T09:26:03.783 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 51137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/tps65010.ko 2023-10-11T09:26:03.783 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/tps6507x.ko 2023-10-11T09:26:03.784 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/viperboard.ko 2023-10-11T09:26:03.784 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/vx855.ko 2023-10-11T09:26:03.784 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/wl1273-core.ko 2023-10-11T09:26:03.784 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 44345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/wm8994.ko 2023-10-11T09:26:03.784 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/ 2023-10-11T09:26:03.784 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/ad525x_dpot-i2c.ko 2023-10-11T09:26:03.785 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 48489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/ad525x_dpot.ko 2023-10-11T09:26:03.785 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl/ 2023-10-11T09:26:03.785 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 46585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-11T09:26:03.785 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds9802als.ko 2023-10-11T09:26:03.785 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 36729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds990x.ko 2023-10-11T09:26:03.786 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 45593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/bh1770glc.ko 2023-10-11T09:26:03.786 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/c2port/ 2023-10-11T09:26:03.786 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2023-10-11T09:26:03.786 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 35305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/c2port/core.ko 2023-10-11T09:26:03.786 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710/ 2023-10-11T09:26:03.787 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 25161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710/cb710.ko 2023-10-11T09:26:03.787 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/ds1682.ko 2023-10-11T09:26:03.787 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/dummy-irq.ko 2023-10-11T09:26:03.787 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/echo/ 2023-10-11T09:26:03.787 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/echo/echo.ko 2023-10-11T09:26:03.787 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/ 2023-10-11T09:26:03.788 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 37849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/at24.ko 2023-10-11T09:26:03.788 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-11T09:26:03.788 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-11T09:26:03.788 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-11T09:26:03.788 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 32993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/enclosure.ko 2023-10-11T09:26:03.789 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/genwqe/ 2023-10-11T09:26:03.789 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 152137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/genwqe/genwqe_card.ko 2023-10-11T09:26:03.789 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/hmc6352.ko 2023-10-11T09:26:03.789 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 31193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/hpilo.ko 2023-10-11T09:26:03.789 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/ibmasm/ 2023-10-11T09:26:03.790 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 72601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/ibmasm/ibmasm.ko 2023-10-11T09:26:03.790 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/ics932s401.ko 2023-10-11T09:26:03.790 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29003.ko 2023-10-11T09:26:03.790 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29020.ko 2023-10-11T09:26:03.790 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/ 2023-10-11T09:26:03.791 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 42585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-11T09:26:03.791 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-11T09:26:03.791 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/ 2023-10-11T09:26:03.791 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 125209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-me.ko 2023-10-11T09:26:03.791 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 56521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-11T09:26:03.791 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 309857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei.ko 2023-10-11T09:26:03.792 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 31329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/phantom.ko 2023-10-11T09:26:03.792 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/ti-st/ 2023-10-11T09:26:03.792 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 74905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/ti-st/st_drv.ko 2023-10-11T09:26:03.792 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 24305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-11T09:26:03.792 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 28873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_core.ko 2023-10-11T09:26:03.793 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tsl2550.ko 2023-10-11T09:26:03.793 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 48673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_balloon.ko 2023-10-11T09:26:03.793 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci/ 2023-10-11T09:26:03.793 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 183857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-11T09:26:03.793 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ 2023-10-11T09:26:03.793 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding/ 2023-10-11T09:26:03.794 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 487865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding/bonding.ko 2023-10-11T09:26:03.794 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/ 2023-10-11T09:26:03.794 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53/ 2023-10-11T09:26:03.794 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 106009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-11T09:26:03.794 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 40281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-11T09:26:03.795 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-11T09:26:03.795 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx/ 2023-10-11T09:26:03.795 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 452681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-11T09:26:03.795 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dummy.ko 2023-10-11T09:26:03.795 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/eql.ko 2023-10-11T09:26:03.796 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ 2023-10-11T09:26:03.796 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/ 2023-10-11T09:26:03.796 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 45785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-11T09:26:03.796 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 42105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-11T09:26:03.796 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 130489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-11T09:26:03.797 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 68785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-11T09:26:03.797 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/ 2023-10-11T09:26:03.797 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 37649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-11T09:26:03.797 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 59689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-11T09:26:03.797 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 27193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-11T09:26:03.797 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 99113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-11T09:26:03.798 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec/ 2023-10-11T09:26:03.798 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 54521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-11T09:26:03.798 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere/ 2023-10-11T09:26:03.798 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 69145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-11T09:26:03.798 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon/ 2023-10-11T09:26:03.799 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 59521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-11T09:26:03.799 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera/ 2023-10-11T09:26:03.799 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 41393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-11T09:26:03.799 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/ 2023-10-11T09:26:03.799 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 54545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-11T09:26:03.800 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 49225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-11T09:26:03.800 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 87345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-11T09:26:03.800 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/ 2023-10-11T09:26:03.800 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx/ 2023-10-11T09:26:03.800 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 97321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-11T09:26:03.801 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c/ 2023-10-11T09:26:03.801 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 104609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-11T09:26:03.801 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e/ 2023-10-11T09:26:03.801 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 90257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-11T09:26:03.801 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/ 2023-10-11T09:26:03.801 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 81505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-11T09:26:03.907 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 69713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-11T09:26:03.907 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/ 2023-10-11T09:26:03.907 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 78401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-11T09:26:03.908 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 176881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-11T09:26:03.908 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2023-10-11T09:26:03.908 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 1422209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-11T09:26:03.908 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 128377 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-11T09:26:03.908 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet/ 2023-10-11T09:26:03.908 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 132289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-11T09:26:03.909 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 312145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-11T09:26:03.909 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/ 2023-10-11T09:26:03.909 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna/ 2023-10-11T09:26:03.909 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 323553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-11T09:26:03.909 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/ 2023-10-11T09:26:03.910 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb/ 2023-10-11T09:26:03.910 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 193433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-11T09:26:03.910 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2023-10-11T09:26:03.910 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 347641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-11T09:26:03.910 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2023-10-11T09:26:03.911 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 1044145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-11T09:26:03.911 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2023-10-11T09:26:03.911 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 166113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-11T09:26:03.911 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2023-10-11T09:26:03.911 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 34881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-11T09:26:03.911 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/ 2023-10-11T09:26:03.912 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic/ 2023-10-11T09:26:03.912 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 204329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-11T09:26:03.912 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/ 2023-10-11T09:26:03.912 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/ 2023-10-11T09:26:03.912 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 69449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-11T09:26:03.912 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 61489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-11T09:26:03.913 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 141161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-11T09:26:03.913 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 51433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-11T09:26:03.913 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 62865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-11T09:26:03.913 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 31305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-11T09:26:03.913 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/ 2023-10-11T09:26:03.914 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 60873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-11T09:26:03.914 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 62833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-11T09:26:03.914 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 45721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dnet.ko 2023-10-11T09:26:03.914 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 25233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ec_bhf.ko 2023-10-11T09:26:03.914 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/ 2023-10-11T09:26:03.914 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet/ 2023-10-11T09:26:03.915 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 281329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-11T09:26:03.915 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 49505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-11T09:26:03.915 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 51737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-11T09:26:03.915 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/fujitsu/ 2023-10-11T09:26:03.915 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 45281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2023-10-11T09:26:03.915 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ 2023-10-11T09:26:03.916 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 90465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-11T09:26:03.916 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000/ 2023-10-11T09:26:03.916 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 295713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-11T09:26:03.916 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e/ 2023-10-11T09:26:03.916 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 584249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-11T09:26:03.916 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k/ 2023-10-11T09:26:03.917 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 246321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-11T09:26:03.917 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e/ 2023-10-11T09:26:03.917 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 924017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-11T09:26:03.917 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf/ 2023-10-11T09:26:03.917 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 302577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-11T09:26:03.918 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb/ 2023-10-11T09:26:03.918 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 514889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-11T09:26:03.918 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf/ 2023-10-11T09:26:03.918 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 93593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-11T09:26:03.918 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe/ 2023-10-11T09:26:03.919 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 734297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-11T09:26:03.919 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf/ 2023-10-11T09:26:03.919 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 151841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-11T09:26:03.919 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 86921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/jme.ko 2023-10-11T09:26:03.919 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/ 2023-10-11T09:26:03.920 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20977 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-11T09:26:03.920 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 102385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-11T09:26:03.920 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 122681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-11T09:26:03.920 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/ 2023-10-11T09:26:03.920 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/ 2023-10-11T09:26:03.921 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 727433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-11T09:26:03.921 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 280513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-11T09:26:03.921 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/ 2023-10-11T09:26:03.921 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2023-10-11T09:26:03.921 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 1097537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-11T09:26:03.921 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ 2023-10-11T09:26:03.922 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 53041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ks8842.ko 2023-10-11T09:26:03.922 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 40993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2023-10-11T09:26:03.922 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2023-10-11T09:26:03.922 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 95665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-11T09:26:03.923 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/ 2023-10-11T09:26:03.923 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge/ 2023-10-11T09:26:03.923 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 108809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-11T09:26:03.923 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/ 2023-10-11T09:26:03.923 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 73377 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-11T09:26:03.923 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 47713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-11T09:26:03.924 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion/ 2023-10-11T09:26:03.924 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 150905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-11T09:26:03.924 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia/ 2023-10-11T09:26:03.924 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 136785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-11T09:26:03.924 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/ 2023-10-11T09:26:03.924 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 50481 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-11T09:26:03.925 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 43145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-11T09:26:03.925 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/ 2023-10-11T09:26:03.925 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen/ 2023-10-11T09:26:03.925 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 227113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-11T09:26:03.925 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 87081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-11T09:26:03.926 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2023-10-11T09:26:03.926 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 671393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-11T09:26:03.926 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc/ 2023-10-11T09:26:03.926 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 43017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-11T09:26:03.926 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/ 2023-10-11T09:26:03.927 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 67385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-11T09:26:03.927 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 83921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-11T09:26:03.927 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 29321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-11T09:26:03.927 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 189817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-11T09:26:03.927 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/samsung/ 2023-10-11T09:26:03.927 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/samsung/sxgbe/ 2023-10-11T09:26:03.928 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 104241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2023-10-11T09:26:03.928 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc/ 2023-10-11T09:26:03.928 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 952929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-11T09:26:03.928 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan/ 2023-10-11T09:26:03.928 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 47177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-11T09:26:03.929 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/ 2023-10-11T09:26:03.929 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 54809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-11T09:26:03.929 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 63273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-11T09:26:03.929 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/ 2023-10-11T09:26:03.929 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 61537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-11T09:26:03.930 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 67129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-11T09:26:03.930 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 73345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-11T09:26:03.930 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 68209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-11T09:26:03.930 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/ 2023-10-11T09:26:03.930 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/ 2023-10-11T09:26:03.931 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2023-10-11T09:26:03.931 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 40137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-11T09:26:03.931 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2023-10-11T09:26:03.931 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 481681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-11T09:26:03.931 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/ 2023-10-11T09:26:03.932 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 103321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-11T09:26:03.932 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 166121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-11T09:26:03.932 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 66473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-11T09:26:03.932 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 78737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-11T09:26:03.932 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti/ 2023-10-11T09:26:03.932 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 49993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-11T09:26:03.933 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti/ 2023-10-11T09:26:03.933 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 86465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-11T09:26:03.933 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/ 2023-10-11T09:26:03.933 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 80257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-11T09:26:03.933 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 83681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-11T09:26:03.934 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/ 2023-10-11T09:26:03.934 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 46241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-11T09:26:03.934 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 30849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-11T09:26:03.934 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom/ 2023-10-11T09:26:03.934 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 60041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-11T09:26:03.935 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/fddi/ 2023-10-11T09:26:03.935 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 41001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/fddi/defxx.ko 2023-10-11T09:26:03.935 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/fddi/skfp/ 2023-10-11T09:26:03.935 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 216977 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/fddi/skfp/skfp.ko 2023-10-11T09:26:03.935 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 77401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/geneve.ko 2023-10-11T09:26:03.935 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv/ 2023-10-11T09:26:03.936 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 235593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-11T09:26:03.936 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154/ 2023-10-11T09:26:03.936 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-11T09:26:03.936 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 25073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ifb.ko 2023-10-11T09:26:03.936 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan/ 2023-10-11T09:26:03.937 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 83993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-11T09:26:03.937 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 62065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvlan.ko 2023-10-11T09:26:03.937 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvtap.ko 2023-10-11T09:26:03.937 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/ 2023-10-11T09:26:03.937 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-11T09:26:03.937 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-11T09:26:03.938 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-gpio.ko 2023-10-11T09:26:03.938 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-11T09:26:03.938 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio.ko 2023-10-11T09:26:03.938 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mii.ko 2023-10-11T09:26:03.939 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 41697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/netconsole.ko 2023-10-11T09:26:03.939 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/nlmon.ko 2023-10-11T09:26:03.939 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/ 2023-10-11T09:26:03.939 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-11T09:26:03.939 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 32329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-11T09:26:03.939 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/ 2023-10-11T09:26:03.940 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/amd.ko 2023-10-11T09:26:03.940 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 50097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-11T09:26:03.940 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 43273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-11T09:26:03.940 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-11T09:26:03.940 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 41793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/broadcom.ko 2023-10-11T09:26:03.941 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10561 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/cicada.ko 2023-10-11T09:26:03.941 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/davicom.ko 2023-10-11T09:26:03.941 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/et1011c.ko 2023-10-11T09:26:03.941 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 24905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/icplus.ko 2023-10-11T09:26:03.941 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/lxt.ko 2023-10-11T09:26:03.941 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 78185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/marvell.ko 2023-10-11T09:26:03.942 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 75713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/micrel.ko 2023-10-11T09:26:03.942 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/national.ko 2023-10-11T09:26:03.942 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 97265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/phylink.ko 2023-10-11T09:26:03.942 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/qsemi.ko 2023-10-11T09:26:03.942 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 49553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/realtek.ko 2023-10-11T09:26:03.943 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 25753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/smsc.ko 2023-10-11T09:26:03.943 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-11T09:26:03.943 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 24033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/vitesse.ko 2023-10-11T09:26:03.943 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/plip/ 2023-10-11T09:26:03.943 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 43465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/plip/plip.ko 2023-10-11T09:26:03.943 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ 2023-10-11T09:26:03.944 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-11T09:26:03.944 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 29929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-11T09:26:03.944 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-11T09:26:03.944 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-11T09:26:03.944 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 28945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-11T09:26:03.944 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 42545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppoe.ko 2023-10-11T09:26:03.945 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppox.ko 2023-10-11T09:26:03.945 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 31969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pptp.ko 2023-10-11T09:26:03.945 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 28225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/sb1000.ko 2023-10-11T09:26:03.945 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/ 2023-10-11T09:26:03.945 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 42513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slip.ko 2023-10-11T09:26:03.945 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/sungem_phy.ko 2023-10-11T09:26:03.946 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 64937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tap.ko 2023-10-11T09:26:03.946 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/ 2023-10-11T09:26:03.946 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 106025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team.ko 2023-10-11T09:26:03.946 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-11T09:26:03.947 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-11T09:26:03.947 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 28993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-11T09:26:03.947 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_random.ko 2023-10-11T09:26:03.947 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-11T09:26:03.947 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 88257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/veth.ko 2023-10-11T09:26:03.947 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3/ 2023-10-11T09:26:03.948 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 120497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-11T09:26:03.948 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan/ 2023-10-11T09:26:03.948 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 293441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-11T09:26:03.948 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback/ 2023-10-11T09:26:03.948 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 143889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-11T09:26:03.949 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb/ 2023-10-11T09:26:03.949 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb/ntb.ko 2023-10-11T09:26:03.949 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/ 2023-10-11T09:26:03.949 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/ 2023-10-11T09:26:03.949 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 264585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-11T09:26:03.949 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 102393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme.ko 2023-10-11T09:26:03.950 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/ 2023-10-11T09:26:03.950 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 133817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport.ko 2023-10-11T09:26:03.950 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14561 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_cs.ko 2023-10-11T09:26:03.950 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 84337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_pc.ko 2023-10-11T09:26:03.950 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_serial.ko 2023-10-11T09:26:03.951 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/ 2023-10-11T09:26:03.951 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/ 2023-10-11T09:26:03.951 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 25009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-11T09:26:03.951 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2023-10-11T09:26:03.951 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2023-10-11T09:26:03.952 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pci-stub.ko 2023-10-11T09:26:03.952 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 47377 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/xen-pcifront.ko 2023-10-11T09:26:03.952 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/ 2023-10-11T09:26:03.952 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/i82092.ko 2023-10-11T09:26:03.952 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 145785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pcmcia.ko 2023-10-11T09:26:03.952 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 62081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pcmcia_core.ko 2023-10-11T09:26:03.953 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 34345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2023-10-11T09:26:03.953 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 34265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pd6729.ko 2023-10-11T09:26:03.953 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 112913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-11T09:26:03.953 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/phy/ 2023-10-11T09:26:03.953 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/phy/broadcom/ 2023-10-11T09:26:03.954 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2023-10-11T09:26:03.954 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/phy/samsung/ 2023-10-11T09:26:03.954 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2023-10-11T09:26:03.954 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/ 2023-10-11T09:26:03.954 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel/ 2023-10-11T09:26:03.954 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 71521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-11T09:26:03.955 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/ 2023-10-11T09:26:03.955 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/chrome/ 2023-10-11T09:26:03.955 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 50889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2023-10-11T09:26:03.955 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/chrome/cros_ec.ko 2023-10-11T09:26:03.955 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2023-10-11T09:26:03.956 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 25417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2023-10-11T09:26:03.956 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2023-10-11T09:26:03.956 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2023-10-11T09:26:03.956 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2023-10-11T09:26:03.956 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/ 2023-10-11T09:26:03.956 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 67097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-11T09:26:03.957 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 39145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-11T09:26:03.957 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 36913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-11T09:26:03.957 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 59665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-11T09:26:03.957 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 30705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-11T09:26:03.957 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 106873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-11T09:26:03.957 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 36913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-11T09:26:03.958 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 49121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-11T09:26:03.958 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-11T09:26:03.958 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 40265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-11T09:26:03.958 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 22521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-11T09:26:03.958 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 36657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/hdaps.ko 2023-10-11T09:26:03.959 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/ibm_rtl.ko 2023-10-11T09:26:03.959 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel/ 2023-10-11T09:26:03.959 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-11T09:26:03.959 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2023-10-11T09:26:03.959 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 43521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-11T09:26:03.960 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-11T09:26:03.960 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10377 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-11T09:26:03.960 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 34801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-11T09:26:03.960 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 42329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-11T09:26:03.960 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-11T09:26:03.960 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-11T09:26:03.961 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-11T09:26:03.961 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-11T09:26:03.961 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-11T09:26:03.961 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 57545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi.ko 2023-10-11T09:26:03.961 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/ 2023-10-11T09:26:03.961 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/ 2023-10-11T09:26:03.962 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 30345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/88pm860x_battery.ko 2023-10-11T09:26:03.962 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 33985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/88pm860x_charger.ko 2023-10-11T09:26:03.962 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 55057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/bq2415x_charger.ko 2023-10-11T09:26:03.962 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 53993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/bq24190_charger.ko 2023-10-11T09:26:03.962 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/bq24735-charger.ko 2023-10-11T09:26:03.963 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/cros_peripheral_charger.ko 2023-10-11T09:26:03.963 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/da9030_battery.ko 2023-10-11T09:26:03.963 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/da9052-battery.ko 2023-10-11T09:26:03.963 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 29281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/ds2760_battery.ko 2023-10-11T09:26:03.963 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 23217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/ds2780_battery.ko 2023-10-11T09:26:03.964 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 23353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/ds2781_battery.ko 2023-10-11T09:26:03.964 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 23649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/ds2782_battery.ko 2023-10-11T09:26:03.964 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/gpio-charger.ko 2023-10-11T09:26:03.964 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/isp1704_charger.ko 2023-10-11T09:26:03.964 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/lp8727_charger.ko 2023-10-11T09:26:03.965 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/max14577_charger.ko 2023-10-11T09:26:03.965 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 25897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/max17040_battery.ko 2023-10-11T09:26:03.965 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 30289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/max17042_battery.ko 2023-10-11T09:26:03.965 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/max8903_charger.ko 2023-10-11T09:26:03.965 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 23729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/max8925_power.ko 2023-10-11T09:26:03.966 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/pcf50633-charger.ko 2023-10-11T09:26:03.966 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 38009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/sbs-battery.ko 2023-10-11T09:26:03.966 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 28385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/test_power.ko 2023-10-11T09:26:03.966 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/tps65090-charger.ko 2023-10-11T09:26:03.966 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/wm831x_backup.ko 2023-10-11T09:26:03.967 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 24521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/wm831x_power.ko 2023-10-11T09:26:03.967 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 22417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/wm8350_power.ko 2023-10-11T09:26:03.967 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/ 2023-10-11T09:26:03.967 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 57737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-11T09:26:03.967 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-11T09:26:03.968 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/ 2023-10-11T09:26:03.968 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/ 2023-10-11T09:26:03.968 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-11T09:26:03.968 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-11T09:26:03.968 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-11T09:26:03.969 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 41921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/pps_core.ko 2023-10-11T09:26:03.969 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ 2023-10-11T09:26:03.969 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 66385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp.ko 2023-10-11T09:26:03.969 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-11T09:26:03.969 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/ 2023-10-11T09:26:03.969 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 23497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-88pm80x.ko 2023-10-11T09:26:03.970 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 24521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-88pm860x.ko 2023-10-11T09:26:03.970 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-11T09:26:03.970 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq4802.ko 2023-10-11T09:26:03.970 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-da9052.ko 2023-10-11T09:26:03.970 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-da9055.ko 2023-10-11T09:26:03.971 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-da9063.ko 2023-10-11T09:26:03.971 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-11T09:26:03.971 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 68577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-11T09:26:03.971 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-11T09:26:03.971 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-11T09:26:03.972 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-11T09:26:03.972 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-11T09:26:03.972 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13369 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-11T09:26:03.972 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-11T09:26:03.972 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 23705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-11T09:26:03.972 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-11T09:26:03.973 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 24329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-11T09:26:03.973 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-11T09:26:03.973 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 36057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-11T09:26:03.973 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-lp8788.ko 2023-10-11T09:26:03.973 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 38097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-11T09:26:03.974 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-11T09:26:03.974 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 25945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-11T09:26:03.974 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t86.ko 2023-10-11T09:26:03.974 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11633 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-11T09:26:03.975 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max8907.ko 2023-10-11T09:26:03.975 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max8925.ko 2023-10-11T09:26:03.975 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 25809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max8997.ko 2023-10-11T09:26:03.975 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max8998.ko 2023-10-11T09:26:03.975 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-mc13xxx.ko 2023-10-11T09:26:03.975 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-11T09:26:03.976 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-palmas.ko 2023-10-11T09:26:03.976 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 40121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-11T09:26:03.976 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17977 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf50633.ko 2023-10-11T09:26:03.976 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-11T09:26:03.976 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-11T09:26:03.977 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 29881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-11T09:26:03.977 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-11T09:26:03.977 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rc5t583.ko 2023-10-11T09:26:03.977 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-11T09:26:03.977 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 33753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-11T09:26:03.978 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 27833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-11T09:26:03.978 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 25257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-11T09:26:03.978 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-11T09:26:03.978 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-s35390a.ko 2023-10-11T09:26:03.978 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-11T09:26:03.979 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-tps6586x.ko 2023-10-11T09:26:03.979 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-tps65910.ko 2023-10-11T09:26:03.979 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-wm831x.ko 2023-10-11T09:26:03.979 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-wm8350.ko 2023-10-11T09:26:03.979 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 27873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-11T09:26:03.979 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ 2023-10-11T09:26:03.980 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 77617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-11T09:26:03.980 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 57665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-sas.ko 2023-10-11T09:26:03.980 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 55169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-11T09:26:03.980 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 135169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/BusLogic.ko 2023-10-11T09:26:03.980 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/a100u2w.ko 2023-10-11T09:26:03.981 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid/ 2023-10-11T09:26:03.981 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 272353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-11T09:26:03.981 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 100617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/advansys.ko 2023-10-11T09:26:03.981 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/ 2023-10-11T09:26:03.981 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 356265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-11T09:26:03.981 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 295801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-11T09:26:03.982 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic94xx/ 2023-10-11T09:26:03.982 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 189321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic94xx/aic94xx.ko 2023-10-11T09:26:03.982 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 24073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/am53c974.ko 2023-10-11T09:26:03.982 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr/ 2023-10-11T09:26:03.982 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 106569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-11T09:26:03.983 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 38185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/atp870u.ko 2023-10-11T09:26:03.983 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi/ 2023-10-11T09:26:03.983 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 246897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-11T09:26:03.983 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa/ 2023-10-11T09:26:03.983 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 1172185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-11T09:26:03.984 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc/ 2023-10-11T09:26:03.984 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 237169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-11T09:26:03.984 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i/ 2023-10-11T09:26:03.984 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 115393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-11T09:26:03.984 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 40609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ch.ko 2023-10-11T09:26:03.985 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor/ 2023-10-11T09:26:03.985 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 308889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-11T09:26:03.985 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/ 2023-10-11T09:26:03.985 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i/ 2023-10-11T09:26:03.985 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 76193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-11T09:26:03.986 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i/ 2023-10-11T09:26:03.986 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 101449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-11T09:26:03.986 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 146785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-11T09:26:03.986 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 75529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dc395x.ko 2023-10-11T09:26:03.986 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 38121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dmx3191d.ko 2023-10-11T09:26:03.987 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r/ 2023-10-11T09:26:03.987 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 212401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-11T09:26:03.987 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 60489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esp_scsi.ko 2023-10-11T09:26:03.987 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/ 2023-10-11T09:26:03.988 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 78289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-11T09:26:03.988 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 158193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-11T09:26:03.988 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fdomain.ko 2023-10-11T09:26:03.988 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic/ 2023-10-11T09:26:03.988 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 262729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-11T09:26:03.988 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 201793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hpsa.ko 2023-10-11T09:26:03.989 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 51145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hptiop.ko 2023-10-11T09:26:03.989 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 49041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-11T09:26:03.989 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 35321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/imm.ko 2023-10-11T09:26:03.989 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 44057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/initio.ko 2023-10-11T09:26:03.989 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 266521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ipr.ko 2023-10-11T09:26:03.990 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 101881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ips.ko 2023-10-11T09:26:03.990 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci/ 2023-10-11T09:26:03.990 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 317249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci/isci.ko 2023-10-11T09:26:03.990 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-11T09:26:03.990 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 52897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-11T09:26:03.991 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc/ 2023-10-11T09:26:03.991 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 314961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-11T09:26:03.991 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 145569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi.ko 2023-10-11T09:26:03.991 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 53521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-11T09:26:03.991 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas/ 2023-10-11T09:26:03.992 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 217209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-11T09:26:03.992 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc/ 2023-10-11T09:26:03.992 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 2192585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-11T09:26:03.992 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/ 2023-10-11T09:26:03.992 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 73545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-11T09:26:03.993 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 33505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-11T09:26:03.993 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 310729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-11T09:26:03.993 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 84697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid.ko 2023-10-11T09:26:03.993 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas/ 2023-10-11T09:26:03.993 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 659513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-11T09:26:03.993 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas/ 2023-10-11T09:26:03.994 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 119929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-11T09:26:03.994 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 78153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvumi.ko 2023-10-11T09:26:03.994 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pcmcia/ 2023-10-11T09:26:03.994 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 81601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2023-10-11T09:26:03.994 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2023-10-11T09:26:03.995 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2023-10-11T09:26:03.995 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 23857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2023-10-11T09:26:03.995 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001/ 2023-10-11T09:26:03.995 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 364257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-11T09:26:03.995 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 123489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pmcraid.ko 2023-10-11T09:26:03.996 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 33617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ppa.ko 2023-10-11T09:26:03.996 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 55345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla1280.ko 2023-10-11T09:26:03.996 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/ 2023-10-11T09:26:03.996 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 1860193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-11T09:26:03.996 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 85313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-11T09:26:03.997 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx/ 2023-10-11T09:26:03.997 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 558865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-11T09:26:03.997 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qlogicfas408.ko 2023-10-11T09:26:03.997 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/raid_class.ko 2023-10-11T09:26:03.997 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 238801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_debug.ko 2023-10-11T09:26:03.997 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 173921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-11T09:26:03.998 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 278865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-11T09:26:03.998 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 97705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-11T09:26:03.998 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 73881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-11T09:26:03.998 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 44993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-11T09:26:03.998 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ses.ko 2023-10-11T09:26:03.999 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 115849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/st.ko 2023-10-11T09:26:03.999 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 46825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/stex.ko 2023-10-11T09:26:03.999 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2/ 2023-10-11T09:26:03.999 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 162121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-11T09:26:03.999 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 42361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-11T09:26:04.000 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 50297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-11T09:26:04.000 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 40001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/wd719x.ko 2023-10-11T09:26:04.000 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 50025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/xen-scsifront.ko 2023-10-11T09:26:04.000 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb/ 2023-10-11T09:26:04.000 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 129417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb/ssb.ko 2023-10-11T09:26:04.000 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/ 2023-10-11T09:26:04.001 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi/ 2023-10-11T09:26:04.001 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 739089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-11T09:26:04.001 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback/ 2023-10-11T09:26:04.001 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 49985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-11T09:26:04.001 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 39713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_file.ko 2023-10-11T09:26:04.002 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 42961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_iblock.ko 2023-10-11T09:26:04.002 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 931721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_mod.ko 2023-10-11T09:26:04.002 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 44321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_pscsi.ko 2023-10-11T09:26:04.002 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 122545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_user.ko 2023-10-11T09:26:04.002 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc/ 2023-10-11T09:26:04.002 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 75641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-11T09:26:04.003 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/ 2023-10-11T09:26:04.003 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/ 2023-10-11T09:26:04.003 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/ 2023-10-11T09:26:04.003 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 23481 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-11T09:26:04.003 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 29153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-11T09:26:04.004 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-11T09:26:04.004 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-11T09:26:04.004 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-11T09:26:04.004 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-11T09:26:04.004 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-11T09:26:04.005 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-11T09:26:04.005 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-11T09:26:04.005 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-11T09:26:04.005 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-11T09:26:04.005 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 37065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-11T09:26:04.005 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 30729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_powerclamp.ko 2023-10-11T09:26:04.006 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-11T09:26:04.006 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-11T09:26:04.006 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-11T09:26:04.006 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ 2023-10-11T09:26:04.006 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless/ 2023-10-11T09:26:04.007 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 78073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-11T09:26:04.007 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 57881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/moxa.ko 2023-10-11T09:26:04.007 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 50793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/mxser.ko 2023-10-11T09:26:04.007 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 107545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_gsm.ko 2023-10-11T09:26:04.007 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 32041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_hdlc.ko 2023-10-11T09:26:04.008 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 68137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/nozomi.ko 2023-10-11T09:26:04.008 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/ 2023-10-11T09:26:04.008 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250/ 2023-10-11T09:26:04.008 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 35745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250/8250_dw.ko 2023-10-11T09:26:04.008 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 59497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-11T09:26:04.009 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/altera_jtaguart.ko 2023-10-11T09:26:04.009 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 22465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/altera_uart.ko 2023-10-11T09:26:04.009 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-11T09:26:04.009 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 72193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/fsl_lpuart.ko 2023-10-11T09:26:04.009 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm/ 2023-10-11T09:26:04.010 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 103017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-11T09:26:04.010 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 28785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/rp2.ko 2023-10-11T09:26:04.010 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 52073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/sc16is7xx.ko 2023-10-11T09:26:04.010 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 100185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/synclink_gt.ko 2023-10-11T09:26:04.010 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/ 2023-10-11T09:26:04.010 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core/ 2023-10-11T09:26:04.011 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 536809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-11T09:26:04.011 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host/ 2023-10-11T09:26:04.011 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 28377 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-11T09:26:04.011 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2023-10-11T09:26:04.011 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/ 2023-10-11T09:26:04.012 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 44585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio.ko 2023-10-11T09:26:04.012 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_aec.ko 2023-10-11T09:26:04.012 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_cif.ko 2023-10-11T09:26:04.012 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_dmem_genirq.ko 2023-10-11T09:26:04.012 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_mf624.ko 2023-10-11T09:26:04.013 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_netx.ko 2023-10-11T09:26:04.013 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-11T09:26:04.013 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_pdrv_genirq.ko 2023-10-11T09:26:04.013 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13633 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_sercos3.ko 2023-10-11T09:26:04.013 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/ 2023-10-11T09:26:04.013 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/c67x00/ 2023-10-11T09:26:04.014 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 73273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/c67x00/c67x00.ko 2023-10-11T09:26:04.014 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/chipidea/ 2023-10-11T09:26:04.014 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 143785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/chipidea/ci_hdrc.ko 2023-10-11T09:26:04.014 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2023-10-11T09:26:04.014 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2023-10-11T09:26:04.014 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2023-10-11T09:26:04.015 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/ 2023-10-11T09:26:04.015 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 85457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-11T09:26:04.015 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 51393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-11T09:26:04.015 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 47465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usblp.ko 2023-10-11T09:26:04.015 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 89833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usbtmc.ko 2023-10-11T09:26:04.016 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/common/ 2023-10-11T09:26:04.016 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 22953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/common/ulpi.ko 2023-10-11T09:26:04.016 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/dwc2/ 2023-10-11T09:26:04.016 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/dwc2/dwc2_pci.ko 2023-10-11T09:26:04.016 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/dwc3/ 2023-10-11T09:26:04.017 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/dwc3/dwc3-haps.ko 2023-10-11T09:26:04.017 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 30041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/dwc3/dwc3-pci.ko 2023-10-11T09:26:04.017 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 445217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/dwc3/dwc3.ko 2023-10-11T09:26:04.017 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/ 2023-10-11T09:26:04.017 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/ 2023-10-11T09:26:04.018 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 60089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/u_ether.ko 2023-10-11T09:26:04.018 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 51753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/u_serial.ko 2023-10-11T09:26:04.018 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 29049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2023-10-11T09:26:04.018 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 42649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2023-10-11T09:26:04.018 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 28089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2023-10-11T09:26:04.019 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 36585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2023-10-11T09:26:04.019 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 92377 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2023-10-11T09:26:04.019 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 51025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2023-10-11T09:26:04.019 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 122073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2023-10-11T09:26:04.019 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 57921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2023-10-11T09:26:04.019 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 22345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2023-10-11T09:26:04.020 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 31265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2023-10-11T09:26:04.020 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 57705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2023-10-11T09:26:04.020 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 85841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2023-10-11T09:26:04.020 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2023-10-11T09:26:04.020 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 58433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2023-10-11T09:26:04.021 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 67337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2023-10-11T09:26:04.021 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/ 2023-10-11T09:26:04.021 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2023-10-11T09:26:04.021 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2023-10-11T09:26:04.021 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2023-10-11T09:26:04.022 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_ether.ko 2023-10-11T09:26:04.022 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 28401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2023-10-11T09:26:04.022 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_hid.ko 2023-10-11T09:26:04.022 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2023-10-11T09:26:04.022 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16977 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2023-10-11T09:26:04.023 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 32073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2023-10-11T09:26:04.023 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_printer.ko 2023-10-11T09:26:04.023 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_serial.ko 2023-10-11T09:26:04.023 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 27201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_zero.ko 2023-10-11T09:26:04.023 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 62153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2023-10-11T09:26:04.023 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2023-10-11T09:26:04.024 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 161321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/libcomposite.ko 2023-10-11T09:26:04.024 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/ 2023-10-11T09:26:04.024 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2023-10-11T09:26:04.024 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/bdc/ 2023-10-11T09:26:04.024 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 157665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2023-10-11T09:26:04.025 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 36065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/goku_udc.ko 2023-10-11T09:26:04.025 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 52993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/gr_udc.ko 2023-10-11T09:26:04.025 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 69689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2023-10-11T09:26:04.025 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 56009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/mv_udc.ko 2023-10-11T09:26:04.025 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 76033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/net2272.ko 2023-10-11T09:26:04.025 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 75057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/net2280.ko 2023-10-11T09:26:04.026 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 73153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/pch_udc.ko 2023-10-11T09:26:04.026 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 65793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2023-10-11T09:26:04.026 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 66073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2023-10-11T09:26:04.026 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 81489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2023-10-11T09:26:04.026 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 156057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/udc-core.ko 2023-10-11T09:26:04.027 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/ 2023-10-11T09:26:04.027 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/bcma-hcd.ko 2023-10-11T09:26:04.027 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 80097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/isp116x-hcd.ko 2023-10-11T09:26:04.027 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 87337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/oxu210hp-hcd.ko 2023-10-11T09:26:04.027 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 78849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/r8a66597-hcd.ko 2023-10-11T09:26:04.028 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 51305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-11T09:26:04.028 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/sl811_cs.ko 2023-10-11T09:26:04.028 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/ssb-hcd.ko 2023-10-11T09:26:04.028 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 25945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/xhci-plat-hcd.ko 2023-10-11T09:26:04.028 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/ 2023-10-11T09:26:04.029 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 34585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/mdc800.ko 2023-10-11T09:26:04.029 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/microtek.ko 2023-10-11T09:26:04.029 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ 2023-10-11T09:26:04.029 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 41841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/adutux.ko 2023-10-11T09:26:04.029 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-11T09:26:04.029 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/cypress_cy7c63.ko 2023-10-11T09:26:04.030 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 23049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/cytherm.ko 2023-10-11T09:26:04.030 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ehset.ko 2023-10-11T09:26:04.030 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi26.ko 2023-10-11T09:26:04.030 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi62.ko 2023-10-11T09:26:04.030 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ezusb.ko 2023-10-11T09:26:04.031 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/idmouse.ko 2023-10-11T09:26:04.031 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 36617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-11T09:26:04.031 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-11T09:26:04.031 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 36529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ldusb.ko 2023-10-11T09:26:04.031 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 36265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-11T09:26:04.032 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 27793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/lvstest.ko 2023-10-11T09:26:04.032 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga/ 2023-10-11T09:26:04.032 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 64841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-11T09:26:04.032 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-11T09:26:04.032 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usb3503.ko 2023-10-11T09:26:04.032 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 25145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usblcd.ko 2023-10-11T09:26:04.033 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 22753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-11T09:26:04.033 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 70873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usbtest.ko 2023-10-11T09:26:04.033 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 41945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/uss720.ko 2023-10-11T09:26:04.033 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 27617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/yurex.ko 2023-10-11T09:26:04.033 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/mon/ 2023-10-11T09:26:04.034 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 73185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/mon/usbmon.ko 2023-10-11T09:26:04.034 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/musb/ 2023-10-11T09:26:04.034 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 301521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/musb/musb_hdrc.ko 2023-10-11T09:26:04.034 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/ 2023-10-11T09:26:04.034 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-11T09:26:04.035 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2023-10-11T09:26:04.035 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-isp1301.ko 2023-10-11T09:26:04.035 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-tahvo.ko 2023-10-11T09:26:04.035 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ 2023-10-11T09:26:04.035 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/aircable.ko 2023-10-11T09:26:04.035 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 33433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ark3116.ko 2023-10-11T09:26:04.036 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-11T09:26:04.036 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 38105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ch341.ko 2023-10-11T09:26:04.036 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 70209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cp210x.ko 2023-10-11T09:26:04.036 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 31169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-11T09:26:04.036 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 57937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-11T09:26:04.036 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 49753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-11T09:26:04.037 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/empeg.ko 2023-10-11T09:26:04.037 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 43897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/f81232.ko 2023-10-11T09:26:04.037 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 175145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-11T09:26:04.037 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 52713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-11T09:26:04.037 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 137105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-11T09:26:04.038 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 124673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_ti.ko 2023-10-11T09:26:04.038 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 67769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipaq.ko 2023-10-11T09:26:04.038 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipw.ko 2023-10-11T09:26:04.038 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-11T09:26:04.038 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 57937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-11T09:26:04.039 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 107417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan.ko 2023-10-11T09:26:04.039 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 35497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-11T09:26:04.039 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 25449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-11T09:26:04.039 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 33521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-11T09:26:04.039 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 32841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-11T09:26:04.040 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/metro-usb.ko 2023-10-11T09:26:04.040 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 66489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7720.ko 2023-10-11T09:26:04.040 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 78113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7840.ko 2023-10-11T09:26:04.040 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 56593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mxuport.ko 2023-10-11T09:26:04.040 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/navman.ko 2023-10-11T09:26:04.040 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/omninet.ko 2023-10-11T09:26:04.041 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 24281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/opticon.ko 2023-10-11T09:26:04.041 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 190233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/option.ko 2023-10-11T09:26:04.041 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 38905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/oti6858.ko 2023-10-11T09:26:04.041 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 54313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/pl2303.ko 2023-10-11T09:26:04.041 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcaux.ko 2023-10-11T09:26:04.042 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 37825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcserial.ko 2023-10-11T09:26:04.042 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 34561 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/quatech2.ko 2023-10-11T09:26:04.042 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-11T09:26:04.042 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 51297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/sierra.ko 2023-10-11T09:26:04.042 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-11T09:26:04.043 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 29913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ssu100.ko 2023-10-11T09:26:04.043 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-11T09:26:04.043 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 61337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-11T09:26:04.043 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 23729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-11T09:26:04.043 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-11T09:26:04.044 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 33313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-11T09:26:04.044 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 125505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usbserial.ko 2023-10-11T09:26:04.044 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 28753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/visor.ko 2023-10-11T09:26:04.044 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 37409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-11T09:26:04.044 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/wishbone-serial.ko 2023-10-11T09:26:04.044 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-11T09:26:04.045 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ 2023-10-11T09:26:04.045 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 52913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/uas.ko 2023-10-11T09:26:04.045 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-11T09:26:04.045 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-11T09:26:04.045 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 22049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-11T09:26:04.046 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 33497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-11T09:26:04.046 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-11T09:26:04.046 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-11T09:26:04.046 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-11T09:26:04.046 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-11T09:26:04.047 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-11T09:26:04.047 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 28441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-11T09:26:04.047 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 28905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-11T09:26:04.047 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-11T09:26:04.047 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 35337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-11T09:26:04.047 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 180017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-11T09:26:04.048 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/ 2023-10-11T09:26:04.048 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 67457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-11T09:26:04.048 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 71345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-11T09:26:04.048 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 105401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-11T09:26:04.048 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/ 2023-10-11T09:26:04.049 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/ 2023-10-11T09:26:04.049 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 153697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-11T09:26:04.049 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 29537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-11T09:26:04.049 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 100569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio.ko 2023-10-11T09:26:04.049 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 75137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-11T09:26:04.050 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/ 2023-10-11T09:26:04.050 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 113889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost.ko 2023-10-11T09:26:04.050 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-11T09:26:04.050 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 58137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_net.ko 2023-10-11T09:26:04.050 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 89289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-11T09:26:04.050 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/ 2023-10-11T09:26:04.051 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/ 2023-10-11T09:26:04.051 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/88pm860x_bl.ko 2023-10-11T09:26:04.051 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/aat2870_bl.ko 2023-10-11T09:26:04.051 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 23305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/adp5520_bl.ko 2023-10-11T09:26:04.051 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 32609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/adp8860_bl.ko 2023-10-11T09:26:04.052 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 39993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/adp8870_bl.ko 2023-10-11T09:26:04.052 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-11T09:26:04.052 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16777 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/as3711_bl.ko 2023-10-11T09:26:04.052 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/bd6107.ko 2023-10-11T09:26:04.052 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/da903x_bl.ko 2023-10-11T09:26:04.053 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/da9052_bl.ko 2023-10-11T09:26:04.053 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/gpio_backlight.ko 2023-10-11T09:26:04.053 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/kb3886_bl.ko 2023-10-11T09:26:04.053 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 22017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lcd.ko 2023-10-11T09:26:04.053 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 23657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lm3533_bl.ko 2023-10-11T09:26:04.053 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lm3639_bl.ko 2023-10-11T09:26:04.054 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lv5207lp.ko 2023-10-11T09:26:04.054 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/max8925_bl.ko 2023-10-11T09:26:04.054 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/pandora_bl.ko 2023-10-11T09:26:04.054 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/pcf50633-backlight.ko 2023-10-11T09:26:04.054 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-11T09:26:04.055 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/wm831x_bl.ko 2023-10-11T09:26:04.055 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/ 2023-10-11T09:26:04.055 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ 2023-10-11T09:26:04.055 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2482.ko 2023-10-11T09:26:04.055 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 38961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2490.ko 2023-10-11T09:26:04.056 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/matrox_w1.ko 2023-10-11T09:26:04.056 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/w1-gpio.ko 2023-10-11T09:26:04.056 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/ 2023-10-11T09:26:04.056 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-11T09:26:04.056 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 22809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-11T09:26:04.057 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-11T09:26:04.057 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-11T09:26:04.057 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-11T09:26:04.057 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-11T09:26:04.057 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-11T09:26:04.057 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-11T09:26:04.058 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-11T09:26:04.058 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 7385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-11T09:26:04.058 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 56425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-11T09:26:04.058 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 102657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/wire.ko 2023-10-11T09:26:04.058 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ 2023-10-11T09:26:04.059 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/acquirewdt.ko 2023-10-11T09:26:04.059 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19977 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/advantechwdt.ko 2023-10-11T09:26:04.059 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18561 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-11T09:26:04.059 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 22969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-11T09:26:04.059 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/cpu5wdt.ko 2023-10-11T09:26:04.060 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/da9052_wdt.ko 2023-10-11T09:26:04.060 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12777 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/da9055_wdt.ko 2023-10-11T09:26:04.060 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/da9063_wdt.ko 2023-10-11T09:26:04.060 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/dw_wdt.ko 2023-10-11T09:26:04.060 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/eurotechwdt.ko 2023-10-11T09:26:04.060 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 23865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-11T09:26:04.061 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 27401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/hpwdt.ko 2023-10-11T09:26:04.061 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/i6300esb.ko 2023-10-11T09:26:04.061 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-11T09:26:04.061 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-11T09:26:04.061 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-11T09:26:04.062 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ibmasr.ko 2023-10-11T09:26:04.062 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-11T09:26:04.062 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-11T09:26:04.062 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-11T09:26:04.062 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/kempld_wdt.ko 2023-10-11T09:26:04.063 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/machzwd.ko 2023-10-11T09:26:04.063 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/mena21_wdt.ko 2023-10-11T09:26:04.063 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/menf21bmc_wdt.ko 2023-10-11T09:26:04.063 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 24673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/nv_tco.ko 2023-10-11T09:26:04.063 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/of_xilinx_wdt.ko 2023-10-11T09:26:04.063 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 22265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pc87413_wdt.ko 2023-10-11T09:26:04.064 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 38657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-11T09:26:04.064 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 31825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-11T09:26:04.064 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/retu_wdt.ko 2023-10-11T09:26:04.064 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sbc60xxwdt.ko 2023-10-11T09:26:04.064 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sbc_epx_c3.ko 2023-10-11T09:26:04.065 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-11T09:26:04.065 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 23809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sc1200wdt.ko 2023-10-11T09:26:04.065 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 23321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-11T09:26:04.065 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/smsc37b787_wdt.ko 2023-10-11T09:26:04.065 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/softdog.ko 2023-10-11T09:26:04.065 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 22273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-11T09:26:04.066 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/twl4030_wdt.ko 2023-10-11T09:26:04.066 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/via_wdt.ko 2023-10-11T09:26:04.066 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-11T09:26:04.066 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-11T09:26:04.066 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 25409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-11T09:26:04.067 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wafer5823wdt.ko 2023-10-11T09:26:04.067 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 28089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-11T09:26:04.067 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wm831x_wdt.ko 2023-10-11T09:26:04.067 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wm8350_wdt.ko 2023-10-11T09:26:04.067 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-11T09:26:04.068 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/ 2023-10-11T09:26:04.068 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 24729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-evtchn.ko 2023-10-11T09:26:04.068 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-11T09:26:04.068 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 47841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntdev.ko 2023-10-11T09:26:04.068 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback/ 2023-10-11T09:26:04.068 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 176417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-11T09:26:04.069 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 40105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-privcmd.ko 2023-10-11T09:26:04.069 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 71321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-scsiback.ko 2023-10-11T09:26:04.069 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs/ 2023-10-11T09:26:04.069 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-11T09:26:04.069 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ 2023-10-11T09:26:04.070 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p/ 2023-10-11T09:26:04.070 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 165625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p/9p.ko 2023-10-11T09:26:04.070 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/adfs/ 2023-10-11T09:26:04.070 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 60881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/adfs/adfs.ko 2023-10-11T09:26:04.070 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs/ 2023-10-11T09:26:04.071 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 162617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs/affs.ko 2023-10-11T09:26:04.071 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/afs/ 2023-10-11T09:26:04.071 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 931937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/afs/kafs.ko 2023-10-11T09:26:04.071 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/autofs/ 2023-10-11T09:26:04.071 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 100937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/autofs/autofs4.ko 2023-10-11T09:26:04.071 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs/ 2023-10-11T09:26:04.072 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 64481 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs/befs.ko 2023-10-11T09:26:04.072 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/bfs/ 2023-10-11T09:26:04.072 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 40993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/bfs/bfs.ko 2023-10-11T09:26:04.072 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 34625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/binfmt_misc.ko 2023-10-11T09:26:04.072 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs/ 2023-10-11T09:26:04.073 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 3474249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs/btrfs.ko 2023-10-11T09:26:04.073 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles/ 2023-10-11T09:26:04.073 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 305745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles/cachefiles.ko 2023-10-11T09:26:04.073 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph/ 2023-10-11T09:26:04.073 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 1316257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph/ceph.ko 2023-10-11T09:26:04.073 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda/ 2023-10-11T09:26:04.074 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 100497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda/coda.ko 2023-10-11T09:26:04.074 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/configfs/ 2023-10-11T09:26:04.074 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 108073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/configfs/configfs.ko 2023-10-11T09:26:04.074 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs/ 2023-10-11T09:26:04.074 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 29273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs/cramfs.ko 2023-10-11T09:26:04.075 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm/ 2023-10-11T09:26:04.075 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 633825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm/dlm.ko 2023-10-11T09:26:04.075 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/efs/ 2023-10-11T09:26:04.075 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 36585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/efs/efs.ko 2023-10-11T09:26:04.075 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs/ 2023-10-11T09:26:04.076 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 1500513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs/f2fs.ko 2023-10-11T09:26:04.076 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/ 2023-10-11T09:26:04.076 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 27865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/msdos.ko 2023-10-11T09:26:04.076 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/freevxfs/ 2023-10-11T09:26:04.076 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 42457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/freevxfs/freevxfs.ko 2023-10-11T09:26:04.076 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache/ 2023-10-11T09:26:04.077 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 226505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache/fscache.ko 2023-10-11T09:26:04.077 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/ 2023-10-11T09:26:04.077 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 24473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/cuse.ko 2023-10-11T09:26:04.077 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2/ 2023-10-11T09:26:04.077 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 1226385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2/gfs2.ko 2023-10-11T09:26:04.078 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs/ 2023-10-11T09:26:04.078 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 141241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs/hfs.ko 2023-10-11T09:26:04.078 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus/ 2023-10-11T09:26:04.078 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 221553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus/hfsplus.ko 2023-10-11T09:26:04.078 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hpfs/ 2023-10-11T09:26:04.078 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 193577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hpfs/hpfs.ko 2023-10-11T09:26:04.079 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs/ 2023-10-11T09:26:04.079 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 74393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs/isofs.ko 2023-10-11T09:26:04.079 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs/ 2023-10-11T09:26:04.079 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 421097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs/jfs.ko 2023-10-11T09:26:04.079 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd/ 2023-10-11T09:26:04.080 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 279041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd/lockd.ko 2023-10-11T09:26:04.080 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix/ 2023-10-11T09:26:04.080 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 87833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix/minix.ko 2023-10-11T09:26:04.080 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs/ 2023-10-11T09:26:04.080 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 124697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs/netfs.ko 2023-10-11T09:26:04.081 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/ 2023-10-11T09:26:04.081 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout/ 2023-10-11T09:26:04.081 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 81521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-11T09:26:04.081 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout/ 2023-10-11T09:26:04.081 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 59425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-11T09:26:04.081 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout/ 2023-10-11T09:26:04.082 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 117705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-11T09:26:04.082 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 1053889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfs.ko 2023-10-11T09:26:04.082 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 66089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv2.ko 2023-10-11T09:26:04.082 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 119801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv3.ko 2023-10-11T09:26:04.082 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 2041809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv4.ko 2023-10-11T09:26:04.083 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/ 2023-10-11T09:26:04.083 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/grace.ko 2023-10-11T09:26:04.083 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-11T09:26:04.083 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd/ 2023-10-11T09:26:04.083 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 1504929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd/nfsd.ko 2023-10-11T09:26:04.084 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2/ 2023-10-11T09:26:04.084 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 591401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2/nilfs2.ko 2023-10-11T09:26:04.084 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/ 2023-10-11T09:26:04.084 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-celtic.ko 2023-10-11T09:26:04.084 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-centeuro.ko 2023-10-11T09:26:04.084 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-croatian.ko 2023-10-11T09:26:04.085 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-cyrillic.ko 2023-10-11T09:26:04.085 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13481 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-gaelic.ko 2023-10-11T09:26:04.085 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-greek.ko 2023-10-11T09:26:04.085 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-iceland.ko 2023-10-11T09:26:04.085 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-inuit.ko 2023-10-11T09:26:04.086 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-roman.ko 2023-10-11T09:26:04.086 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-romanian.ko 2023-10-11T09:26:04.086 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-turkish.ko 2023-10-11T09:26:04.086 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_ascii.ko 2023-10-11T09:26:04.086 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1250.ko 2023-10-11T09:26:04.087 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1251.ko 2023-10-11T09:26:04.087 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1255.ko 2023-10-11T09:26:04.087 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp737.ko 2023-10-11T09:26:04.087 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp775.ko 2023-10-11T09:26:04.087 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp850.ko 2023-10-11T09:26:04.087 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp852.ko 2023-10-11T09:26:04.088 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp855.ko 2023-10-11T09:26:04.088 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp857.ko 2023-10-11T09:26:04.088 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp860.ko 2023-10-11T09:26:04.088 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp861.ko 2023-10-11T09:26:04.088 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp862.ko 2023-10-11T09:26:04.089 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp863.ko 2023-10-11T09:26:04.089 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp864.ko 2023-10-11T09:26:04.089 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp865.ko 2023-10-11T09:26:04.089 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp866.ko 2023-10-11T09:26:04.089 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp869.ko 2023-10-11T09:26:04.089 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp874.ko 2023-10-11T09:26:04.090 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 94137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp932.ko 2023-10-11T09:26:04.090 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 142673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp936.ko 2023-10-11T09:26:04.090 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 165745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp949.ko 2023-10-11T09:26:04.090 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 118121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp950.ko 2023-10-11T09:26:04.090 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_euc-jp.ko 2023-10-11T09:26:04.091 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-11T09:26:04.091 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-11T09:26:04.091 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-11T09:26:04.091 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-11T09:26:04.091 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-11T09:26:04.091 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-11T09:26:04.092 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-11T09:26:04.092 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-11T09:26:04.092 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-11T09:26:04.092 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-11T09:26:04.092 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-11T09:26:04.093 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-r.ko 2023-10-11T09:26:04.093 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-11T09:26:04.093 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-u.ko 2023-10-11T09:26:04.093 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_utf8.ko 2023-10-11T09:26:04.093 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ntfs/ 2023-10-11T09:26:04.093 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 219361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ntfs/ntfs.ko 2023-10-11T09:26:04.094 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ 2023-10-11T09:26:04.094 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster/ 2023-10-11T09:26:04.094 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 244177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-11T09:26:04.094 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm/ 2023-10-11T09:26:04.094 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 440785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-11T09:26:04.095 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs/ 2023-10-11T09:26:04.095 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 53841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-11T09:26:04.095 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 2589873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2.ko 2023-10-11T09:26:04.095 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-11T09:26:04.095 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 31585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-11T09:26:04.095 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 35881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-11T09:26:04.096 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/omfs/ 2023-10-11T09:26:04.096 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 52569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/omfs/omfs.ko 2023-10-11T09:26:04.096 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs/ 2023-10-11T09:26:04.096 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 336889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs/overlay.ko 2023-10-11T09:26:04.096 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore/ 2023-10-11T09:26:04.096 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 54609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore/ramoops.ko 2023-10-11T09:26:04.097 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/qnx4/ 2023-10-11T09:26:04.097 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/qnx4/qnx4.ko 2023-10-11T09:26:04.097 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/qnx6/ 2023-10-11T09:26:04.097 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 46649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/qnx6/qnx6.ko 2023-10-11T09:26:04.097 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/quota/ 2023-10-11T09:26:04.098 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 30881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/quota/quota_tree.ko 2023-10-11T09:26:04.098 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/quota/quota_v1.ko 2023-10-11T09:26:04.098 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 22017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/quota/quota_v2.ko 2023-10-11T09:26:04.098 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs/ 2023-10-11T09:26:04.098 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 530697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs/reiserfs.ko 2023-10-11T09:26:04.099 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs/ 2023-10-11T09:26:04.099 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 28505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs/romfs.ko 2023-10-11T09:26:04.099 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/ 2023-10-11T09:26:04.099 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client/ 2023-10-11T09:26:04.099 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 2661209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client/cifs.ko 2023-10-11T09:26:04.100 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/ 2023-10-11T09:26:04.100 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 7161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-11T09:26:04.100 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_md4.ko 2023-10-11T09:26:04.100 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs/ 2023-10-11T09:26:04.100 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 134313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs/squashfs.ko 2023-10-11T09:26:04.100 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv/ 2023-10-11T09:26:04.101 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 89737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv/sysv.ko 2023-10-11T09:26:04.101 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf/ 2023-10-11T09:26:04.101 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 248049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf/udf.ko 2023-10-11T09:26:04.101 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs/ 2023-10-11T09:26:04.101 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 160857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs/ufs.ko 2023-10-11T09:26:04.102 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs/ 2023-10-11T09:26:04.102 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 3998129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs/xfs.ko 2023-10-11T09:26:04.102 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/ 2023-10-11T09:26:04.102 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/locking/ 2023-10-11T09:26:04.102 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 74865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/locking/locktorture.ko 2023-10-11T09:26:04.103 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/time/ 2023-10-11T09:26:04.103 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/time/test_udelay.ko 2023-10-11T09:26:04.103 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 58473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/torture.ko 2023-10-11T09:26:04.103 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ 2023-10-11T09:26:04.103 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 6833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc-itu-t.ko 2023-10-11T09:26:04.103 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 6529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc7.ko 2023-10-11T09:26:04.104 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 7025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc8.ko 2023-10-11T09:26:04.104 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/ 2023-10-11T09:26:04.104 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 7097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libarc4.ko 2023-10-11T09:26:04.104 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 37081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libdes.ko 2023-10-11T09:26:04.104 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/interval_tree_test.ko 2023-10-11T09:26:04.105 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/libcrc32c.ko 2023-10-11T09:26:04.105 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 24929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lru_cache.ko 2023-10-11T09:26:04.105 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/ 2023-10-11T09:26:04.105 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 33801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4_compress.ko 2023-10-11T09:26:04.105 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 22633 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-11T09:26:04.106 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math/ 2023-10-11T09:26:04.106 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 6617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math/cordic.ko 2023-10-11T09:26:04.106 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/memory-notifier-error-inject.ko 2023-10-11T09:26:04.106 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/notifier-error-inject.ko 2023-10-11T09:26:04.106 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/percpu_test.ko 2023-10-11T09:26:04.106 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/pm-notifier-error-inject.ko 2023-10-11T09:26:04.107 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6/ 2023-10-11T09:26:04.107 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 143433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6/raid6_pq.ko 2023-10-11T09:26:04.107 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/rbtree_test.ko 2023-10-11T09:26:04.107 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon/ 2023-10-11T09:26:04.107 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-11T09:26:04.108 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 38841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/test-kstrtox.ko 2023-10-11T09:26:04.108 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/test-string_helpers.ko 2023-10-11T09:26:04.108 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 4760825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/test_bpf.ko 2023-10-11T09:26:04.108 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 49081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/test_firmware.ko 2023-10-11T09:26:04.108 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 6369 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/test_module.ko 2023-10-11T09:26:04.109 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/test_user_copy.ko 2023-10-11T09:26:04.109 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_bm.ko 2023-10-11T09:26:04.109 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_fsm.ko 2023-10-11T09:26:04.109 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_kmp.ko 2023-10-11T09:26:04.109 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/xz/ 2023-10-11T09:26:04.110 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/xz/xz_dec_test.ko 2023-10-11T09:26:04.110 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm/ 2023-10-11T09:26:04.110 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm/hwpoison-inject.ko 2023-10-11T09:26:04.110 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ 2023-10-11T09:26:04.110 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/ 2023-10-11T09:26:04.110 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 55737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/6lowpan.ko 2023-10-11T09:26:04.111 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 7049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_dest.ko 2023-10-11T09:26:04.111 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 7065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-11T09:26:04.111 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 7049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_hop.ko 2023-10-11T09:26:04.111 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 6985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-11T09:26:04.111 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 7065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-11T09:26:04.112 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 7065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_routing.ko 2023-10-11T09:26:04.112 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_udp.ko 2023-10-11T09:26:04.112 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/ 2023-10-11T09:26:04.112 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 30385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/garp.ko 2023-10-11T09:26:04.112 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 34785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/mrp.ko 2023-10-11T09:26:04.113 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/p8022.ko 2023-10-11T09:26:04.113 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/psnap.ko 2023-10-11T09:26:04.113 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/stp.ko 2023-10-11T09:26:04.113 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q/ 2023-10-11T09:26:04.113 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 81017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q/8021q.ko 2023-10-11T09:26:04.114 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/ 2023-10-11T09:26:04.114 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 198465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet.ko 2023-10-11T09:26:04.114 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 35961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_fd.ko 2023-10-11T09:26:04.114 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_rdma.ko 2023-10-11T09:26:04.114 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 32321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_virtio.ko 2023-10-11T09:26:04.114 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv/ 2023-10-11T09:26:04.115 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 652553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv/batman-adv.ko 2023-10-11T09:26:04.115 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/ 2023-10-11T09:26:04.115 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 63825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/br_netfilter.ko 2023-10-11T09:26:04.115 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 772505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/bridge.ko 2023-10-11T09:26:04.115 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ 2023-10-11T09:26:04.116 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 7657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-11T09:26:04.116 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-11T09:26:04.116 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-11T09:26:04.116 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-11T09:26:04.116 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-11T09:26:04.117 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-11T09:26:04.117 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-11T09:26:04.117 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-11T09:26:04.117 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-11T09:26:04.117 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 7585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-11T09:26:04.117 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 7593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-11T09:26:04.118 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-11T09:26:04.118 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 7537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-11T09:26:04.118 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-11T09:26:04.118 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-11T09:26:04.118 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-11T09:26:04.119 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-11T09:26:04.119 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-11T09:26:04.119 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-11T09:26:04.119 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-11T09:26:04.119 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 55937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-11T09:26:04.120 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-11T09:26:04.120 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-11T09:26:04.120 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph/ 2023-10-11T09:26:04.120 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 1046921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph/libceph.ko 2023-10-11T09:26:04.120 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/ 2023-10-11T09:26:04.120 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 124793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/pktgen.ko 2023-10-11T09:26:04.121 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/ 2023-10-11T09:26:04.121 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 194321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp.ko 2023-10-11T09:26:04.121 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_diag.ko 2023-10-11T09:26:04.121 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 56945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv4.ko 2023-10-11T09:26:04.121 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 59457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv6.ko 2023-10-11T09:26:04.122 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/ 2023-10-11T09:26:04.122 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 390297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/dsa_core.ko 2023-10-11T09:26:04.122 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_brcm.ko 2023-10-11T09:26:04.122 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_dsa.ko 2023-10-11T09:26:04.122 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 7721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_none.ko 2023-10-11T09:26:04.123 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_trailer.ko 2023-10-11T09:26:04.123 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/hsr/ 2023-10-11T09:26:04.123 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 100257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/hsr/hsr.ko 2023-10-11T09:26:04.123 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ 2023-10-11T09:26:04.123 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan/ 2023-10-11T09:26:04.123 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 45169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-11T09:26:04.124 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 254601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154.ko 2023-10-11T09:26:04.124 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 47265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-11T09:26:04.124 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ 2023-10-11T09:26:04.124 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 24729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ah4.ko 2023-10-11T09:26:04.124 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 38905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/esp4.ko 2023-10-11T09:26:04.125 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 48345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/fou.ko 2023-10-11T09:26:04.125 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/gre.ko 2023-10-11T09:26:04.125 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 48529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/inet_diag.ko 2023-10-11T09:26:04.125 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 60593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_gre.ko 2023-10-11T09:26:04.125 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 61281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_tunnel.ko 2023-10-11T09:26:04.126 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 34457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_vti.ko 2023-10-11T09:26:04.126 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipcomp.ko 2023-10-11T09:26:04.126 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 28889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipip.ko 2023-10-11T09:26:04.126 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ 2023-10-11T09:26:04.126 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 37689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-11T09:26:04.126 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-11T09:26:04.127 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-11T09:26:04.127 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 41601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ip_tables.ko 2023-10-11T09:26:04.127 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-11T09:26:04.127 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2023-10-11T09:26:04.127 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-11T09:26:04.128 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-11T09:26:04.128 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-11T09:26:04.128 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_filter.ko 2023-10-11T09:26:04.128 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-11T09:26:04.128 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-11T09:26:04.129 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-11T09:26:04.129 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-11T09:26:04.129 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-11T09:26:04.129 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-11T09:26:04.129 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 27625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-11T09:26:04.130 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-11T09:26:04.130 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-11T09:26:04.130 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20561 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2023-10-11T09:26:04.130 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-11T09:26:04.130 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-11T09:26:04.131 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-11T09:26:04.131 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-11T09:26:04.131 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-11T09:26:04.131 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_bic.ko 2023-10-11T09:26:04.131 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-11T09:26:04.131 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_diag.ko 2023-10-11T09:26:04.132 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-11T09:26:04.132 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_htcp.ko 2023-10-11T09:26:04.132 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_hybla.ko 2023-10-11T09:26:04.132 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_illinois.ko 2023-10-11T09:26:04.132 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_lp.ko 2023-10-11T09:26:04.133 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_scalable.ko 2023-10-11T09:26:04.133 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_vegas.ko 2023-10-11T09:26:04.133 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_veno.ko 2023-10-11T09:26:04.133 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_westwood.ko 2023-10-11T09:26:04.133 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_yeah.ko 2023-10-11T09:26:04.133 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 22601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tunnel4.ko 2023-10-11T09:26:04.134 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_diag.ko 2023-10-11T09:26:04.134 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 41577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_tunnel.ko 2023-10-11T09:26:04.134 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-11T09:26:04.134 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ 2023-10-11T09:26:04.134 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ah6.ko 2023-10-11T09:26:04.135 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 39537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/esp6.ko 2023-10-11T09:26:04.135 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/fou6.ko 2023-10-11T09:26:04.135 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 80793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_gre.ko 2023-10-11T09:26:04.135 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 92817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-11T09:26:04.135 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-11T09:26:04.136 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 54089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_vti.ko 2023-10-11T09:26:04.136 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ipcomp6.ko 2023-10-11T09:26:04.136 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/mip6.ko 2023-10-11T09:26:04.136 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ 2023-10-11T09:26:04.136 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 42209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-11T09:26:04.136 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2023-10-11T09:26:04.137 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-11T09:26:04.137 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-11T09:26:04.137 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-11T09:26:04.137 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 7801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-11T09:26:04.137 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-11T09:26:04.137 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-11T09:26:04.138 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-11T09:26:04.138 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-11T09:26:04.138 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-11T09:26:04.138 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-11T09:26:04.138 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-11T09:26:04.139 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-11T09:26:04.139 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-11T09:26:04.139 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-11T09:26:04.139 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-11T09:26:04.139 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 38833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-11T09:26:04.140 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-11T09:26:04.140 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-11T09:26:04.140 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-11T09:26:04.140 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-11T09:26:04.140 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9633 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-11T09:26:04.141 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-11T09:26:04.141 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-11T09:26:04.141 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 73161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/sit.ko 2023-10-11T09:26:04.141 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 24193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/tunnel6.ko 2023-10-11T09:26:04.141 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 23305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-11T09:26:04.142 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key/ 2023-10-11T09:26:04.142 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 82929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key/af_key.ko 2023-10-11T09:26:04.142 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/ 2023-10-11T09:26:04.142 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 125873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_core.ko 2023-10-11T09:26:04.142 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-11T09:26:04.143 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 25865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_eth.ko 2023-10-11T09:26:04.143 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 35049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip.ko 2023-10-11T09:26:04.143 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 36745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-11T09:26:04.143 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 33001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-11T09:26:04.143 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 53153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-11T09:26:04.144 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/lapb/ 2023-10-11T09:26:04.144 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 48065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/lapb/lapb.ko 2023-10-11T09:26:04.144 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc/ 2023-10-11T09:26:04.144 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 23225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc/llc.ko 2023-10-11T09:26:04.144 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 267257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc/llc2.ko 2023-10-11T09:26:04.145 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154/ 2023-10-11T09:26:04.145 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 291041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154/mac802154.ko 2023-10-11T09:26:04.145 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls/ 2023-10-11T09:26:04.145 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls/mpls_gso.ko 2023-10-11T09:26:04.145 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ 2023-10-11T09:26:04.145 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ 2023-10-11T09:26:04.146 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 119649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-11T09:26:04.146 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 28449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-11T09:26:04.146 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 27033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-11T09:26:04.146 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-11T09:26:04.146 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 102033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-11T09:26:04.147 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 103121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-11T09:26:04.147 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 104593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-11T09:26:04.147 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 105129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-11T09:26:04.147 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 114393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-11T09:26:04.147 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 61161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-11T09:26:04.148 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 111161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-11T09:26:04.148 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 114673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-11T09:26:04.148 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 114329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-11T09:26:04.148 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 113217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-11T09:26:04.148 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 116217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-11T09:26:04.149 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 30145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-11T09:26:04.149 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ 2023-10-11T09:26:04.149 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 412889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-11T09:26:04.149 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-11T09:26:04.149 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 7625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-11T09:26:04.150 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-11T09:26:04.150 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-11T09:26:04.150 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-11T09:26:04.150 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 7689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-11T09:26:04.150 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 7689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-11T09:26:04.151 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-11T09:26:04.151 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-11T09:26:04.151 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 7697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-11T09:26:04.151 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-11T09:26:04.151 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 7697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-11T09:26:04.151 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-11T09:26:04.152 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 48897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conncount.ko 2023-10-11T09:26:04.152 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 408665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack.ko 2023-10-11T09:26:04.152 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-11T09:26:04.152 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-11T09:26:04.152 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 33761 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-11T09:26:04.153 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 118145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-11T09:26:04.153 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 22185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-11T09:26:04.153 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-11T09:26:04.153 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 108393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-11T09:26:04.153 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 33825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-11T09:26:04.153 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-11T09:26:04.154 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 66641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-11T09:26:04.154 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-11T09:26:04.154 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-11T09:26:04.154 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-11T09:26:04.154 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 88513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table.ko 2023-10-11T09:26:04.155 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-11T09:26:04.155 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 37129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-11T09:26:04.155 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 95353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat.ko 2023-10-11T09:26:04.155 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-11T09:26:04.155 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-11T09:26:04.156 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-11T09:26:04.156 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 28929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-11T09:26:04.156 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-11T09:26:04.156 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 44025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-11T09:26:04.156 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 609257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_tables.ko 2023-10-11T09:26:04.156 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 43873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink.ko 2023-10-11T09:26:04.157 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 30273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-11T09:26:04.157 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 30625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_cttimeout.ko 2023-10-11T09:26:04.157 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 39905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-11T09:26:04.157 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 22713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-11T09:26:04.157 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 49673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-11T09:26:04.157 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-11T09:26:04.158 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 33257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_compat.ko 2023-10-11T09:26:04.158 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 40721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_ct.ko 2023-10-11T09:26:04.158 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-11T09:26:04.158 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fib.ko 2023-10-11T09:26:04.158 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-11T09:26:04.159 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_hash.ko 2023-10-11T09:26:04.159 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_limit.ko 2023-10-11T09:26:04.159 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_log.ko 2023-10-11T09:26:04.159 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_masq.ko 2023-10-11T09:26:04.159 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_nat.ko 2023-10-11T09:26:04.160 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_numgen.ko 2023-10-11T09:26:04.160 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_osf.ko 2023-10-11T09:26:04.160 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_queue.ko 2023-10-11T09:26:04.160 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_redir.ko 2023-10-11T09:26:04.160 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject.ko 2023-10-11T09:26:04.161 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-11T09:26:04.161 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_socket.ko 2023-10-11T09:26:04.161 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_synproxy.ko 2023-10-11T09:26:04.161 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_tproxy.ko 2023-10-11T09:26:04.161 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 89329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/x_tables.ko 2023-10-11T09:26:04.162 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-11T09:26:04.162 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9633 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-11T09:26:04.162 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 7729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-11T09:26:04.162 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-11T09:26:04.162 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 18169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CT.ko 2023-10-11T09:26:04.163 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_DSCP.ko 2023-10-11T09:26:04.163 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HL.ko 2023-10-11T09:26:04.163 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HMARK.ko 2023-10-11T09:26:04.163 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 38857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-11T09:26:04.163 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LED.ko 2023-10-11T09:26:04.164 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LOG.ko 2023-10-11T09:26:04.164 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-11T09:26:04.164 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-11T09:26:04.164 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-11T09:26:04.164 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-11T09:26:04.164 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-11T09:26:04.165 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-11T09:26:04.165 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 14337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-11T09:26:04.165 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-11T09:26:04.165 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-11T09:26:04.165 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17369 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TEE.ko 2023-10-11T09:26:04.166 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-11T09:26:04.166 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TRACE.ko 2023-10-11T09:26:04.166 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_addrtype.ko 2023-10-11T09:26:04.166 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_bpf.ko 2023-10-11T09:26:04.166 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cgroup.ko 2023-10-11T09:26:04.167 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cluster.ko 2023-10-11T09:26:04.167 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 7505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_comment.ko 2023-10-11T09:26:04.167 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connbytes.ko 2023-10-11T09:26:04.167 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlabel.ko 2023-10-11T09:26:04.167 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlimit.ko 2023-10-11T09:26:04.167 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connmark.ko 2023-10-11T09:26:04.168 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_conntrack.ko 2023-10-11T09:26:04.168 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 7817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cpu.ko 2023-10-11T09:26:04.168 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dccp.ko 2023-10-11T09:26:04.168 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 7865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_devgroup.ko 2023-10-11T09:26:04.168 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dscp.ko 2023-10-11T09:26:04.169 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ecn.ko 2023-10-11T09:26:04.169 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_esp.ko 2023-10-11T09:26:04.169 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 43457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-11T09:26:04.169 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_helper.ko 2023-10-11T09:26:04.169 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 7865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hl.ko 2023-10-11T09:26:04.170 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-11T09:26:04.170 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_iprange.ko 2023-10-11T09:26:04.170 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipvs.ko 2023-10-11T09:26:04.170 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_l2tp.ko 2023-10-11T09:26:04.170 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_length.ko 2023-10-11T09:26:04.170 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_limit.ko 2023-10-11T09:26:04.171 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 7545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mac.ko 2023-10-11T09:26:04.171 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mark.ko 2023-10-11T09:26:04.171 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_multiport.ko 2023-10-11T09:26:04.171 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nat.ko 2023-10-11T09:26:04.171 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nfacct.ko 2023-10-11T09:26:04.172 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_osf.ko 2023-10-11T09:26:04.172 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_owner.ko 2023-10-11T09:26:04.172 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_physdev.ko 2023-10-11T09:26:04.172 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 7633 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_pkttype.ko 2023-10-11T09:26:04.172 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_policy.ko 2023-10-11T09:26:04.173 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_quota.ko 2023-10-11T09:26:04.173 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_rateest.ko 2023-10-11T09:26:04.173 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_realm.ko 2023-10-11T09:26:04.173 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 36441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_recent.ko 2023-10-11T09:26:04.173 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_sctp.ko 2023-10-11T09:26:04.174 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 30089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_set.ko 2023-10-11T09:26:04.174 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_socket.ko 2023-10-11T09:26:04.174 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_state.ko 2023-10-11T09:26:04.174 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_statistic.ko 2023-10-11T09:26:04.174 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_string.ko 2023-10-11T09:26:04.175 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 8553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-11T09:26:04.175 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15777 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_tcpudp.ko 2023-10-11T09:26:04.175 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_time.ko 2023-10-11T09:26:04.175 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_u32.ko 2023-10-11T09:26:04.175 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink/ 2023-10-11T09:26:04.176 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink/netlink_diag.ko 2023-10-11T09:26:04.176 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh/ 2023-10-11T09:26:04.176 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 12425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh/nsh.ko 2023-10-11T09:26:04.176 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/ 2023-10-11T09:26:04.176 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 379545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/openvswitch.ko 2023-10-11T09:26:04.176 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-geneve.ko 2023-10-11T09:26:04.177 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-gre.ko 2023-10-11T09:26:04.177 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 11793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-11T09:26:04.177 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet/ 2023-10-11T09:26:04.177 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet/af_packet_diag.ko 2023-10-11T09:26:04.177 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/phonet/ 2023-10-11T09:26:04.178 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 109961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/phonet/phonet.ko 2023-10-11T09:26:04.178 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 54761 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/phonet/pn_pep.ko 2023-10-11T09:26:04.178 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/ 2023-10-11T09:26:04.178 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 252889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds.ko 2023-10-11T09:26:04.178 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 165081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_rdma.ko 2023-10-11T09:26:04.179 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 67241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_tcp.ko 2023-10-11T09:26:04.179 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rxrpc/ 2023-10-11T09:26:04.179 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 820137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rxrpc/rxrpc.ko 2023-10-11T09:26:04.179 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/ 2023-10-11T09:26:04.179 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 32249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_csum.ko 2023-10-11T09:26:04.180 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 23537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_gact.ko 2023-10-11T09:26:04.180 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_ipt.ko 2023-10-11T09:26:04.180 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 34057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_mirred.ko 2023-10-11T09:26:04.180 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 23425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_nat.ko 2023-10-11T09:26:04.180 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 43817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_pedit.ko 2023-10-11T09:26:04.181 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 31985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_police.ko 2023-10-11T09:26:04.181 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_simple.ko 2023-10-11T09:26:04.181 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 36353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_skbedit.ko 2023-10-11T09:26:04.181 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 36057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_vlan.ko 2023-10-11T09:26:04.181 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_basic.ko 2023-10-11T09:26:04.181 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 33945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_bpf.ko 2023-10-11T09:26:04.182 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_cgroup.ko 2023-10-11T09:26:04.182 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 31377 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_flow.ko 2023-10-11T09:26:04.182 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 31769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_fw.ko 2023-10-11T09:26:04.182 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 38489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_route.ko 2023-10-11T09:26:04.182 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 61409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_u32.ko 2023-10-11T09:26:04.183 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 7441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_cmp.ko 2023-10-11T09:26:04.183 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 13457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_ipset.ko 2023-10-11T09:26:04.183 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 33337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_meta.ko 2023-10-11T09:26:04.183 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 7897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_nbyte.ko 2023-10-11T09:26:04.183 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_text.ko 2023-10-11T09:26:04.184 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 7249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_u32.ko 2023-10-11T09:26:04.184 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 21649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_choke.ko 2023-10-11T09:26:04.184 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_codel.ko 2023-10-11T09:26:04.184 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 33753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_drr.ko 2023-10-11T09:26:04.184 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 33609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_fq.ko 2023-10-11T09:26:04.185 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 31601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_fq_codel.ko 2023-10-11T09:26:04.185 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 35433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_gred.ko 2023-10-11T09:26:04.185 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 48409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hfsc.ko 2023-10-11T09:26:04.185 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 22073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hhf.ko 2023-10-11T09:26:04.185 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 65793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_htb.ko 2023-10-11T09:26:04.186 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 16065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_ingress.ko 2023-10-11T09:26:04.186 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 30273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio.ko 2023-10-11T09:26:04.186 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-11T09:26:04.186 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_multiq.ko 2023-10-11T09:26:04.186 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 35601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_netem.ko 2023-10-11T09:26:04.186 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 27545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_pie.ko 2023-10-11T09:26:04.187 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_plug.ko 2023-10-11T09:26:04.187 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 24729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_prio.ko 2023-10-11T09:26:04.187 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 47849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_qfq.ko 2023-10-11T09:26:04.187 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 32953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_red.ko 2023-10-11T09:26:04.187 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 26881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfb.ko 2023-10-11T09:26:04.188 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 31425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfq.ko 2023-10-11T09:26:04.188 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 28897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_tbf.ko 2023-10-11T09:26:04.188 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 27729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_teql.ko 2023-10-11T09:26:04.188 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/ 2023-10-11T09:26:04.188 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 930369 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp.ko 2023-10-11T09:26:04.189 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 19033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp_diag.ko 2023-10-11T09:26:04.189 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/ 2023-10-11T09:26:04.189 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/ 2023-10-11T09:26:04.189 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 335569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-11T09:26:04.189 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 61833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-11T09:26:04.190 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 1514673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/sunrpc.ko 2023-10-11T09:26:04.190 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma/ 2023-10-11T09:26:04.190 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 707401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-11T09:26:04.190 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/ 2023-10-11T09:26:04.190 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/diag.ko 2023-10-11T09:26:04.191 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 862385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/tipc.ko 2023-10-11T09:26:04.191 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix/ 2023-10-11T09:26:04.191 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix/unix_diag.ko 2023-10-11T09:26:04.191 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/ 2023-10-11T09:26:04.191 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 84841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-11T09:26:04.191 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 68033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-11T09:26:04.192 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 95985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock.ko 2023-10-11T09:26:04.192 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 10705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-11T09:26:04.192 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 17521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-11T09:26:04.192 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/x25/ 2023-10-11T09:26:04.192 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 125097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/x25/x25.ko 2023-10-11T09:26:04.192 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/ 2023-10-11T09:26:04.193 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 22657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/xfrm_algo.ko 2023-10-11T09:26:04.193 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 20705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-11T09:26:04.193 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 102353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/xfrm_user.ko 2023-10-11T09:26:04.193 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/ 2023-10-11T09:26:04.193 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib/ 2023-10-11T09:26:04.193 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 15513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib/irqbypass.ko 2023-10-11T09:26:04.194 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 638591 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias 2023-10-11T09:26:04.194 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 624324 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias.bin 2023-10-11T09:26:04.194 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 7656 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin 2023-10-11T09:26:04.194 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 23361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.alias.bin 2023-10-11T09:26:04.194 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 9821 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.bin 2023-10-11T09:26:04.195 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 69921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.modinfo 2023-10-11T09:26:04.195 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 146449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep 2023-10-11T09:26:04.195 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 232695 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep.bin 2023-10-11T09:26:04.195 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 210 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.devname 2023-10-11T09:26:04.195 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 72754 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.order 2023-10-11T09:26:04.195 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 879 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.softdep 2023-10-11T09:26:04.196 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 252880 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols 2023-10-11T09:26:04.196 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 307898 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols.bin 2023-10-11T09:26:04.196 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./usr/ 2023-10-11T09:26:04.196 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./usr/share/ 2023-10-11T09:26:04.196 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./usr/share/doc/ 2023-10-11T09:26:04.197 INFO:teuthology.orchestra.run.smithi133.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./usr/share/doc/linux-image-6.5.0-rc7-gfef64f93da2b/ 2023-10-11T09:26:04.197 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 172 2023-09-11 02:20 ./usr/share/doc/linux-image-6.5.0-rc7-gfef64f93da2b/changelog.Debian.gz 2023-10-11T09:26:04.197 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r-- root/root 691 2023-09-11 02:20 ./usr/share/doc/linux-image-6.5.0-rc7-gfef64f93da2b/copyright 2023-10-11T09:26:04.198 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc7-gfef64f93da2b 2023-10-11T09:26:04.198 INFO:teuthology.task.kernel:searching for kernel 6.5.0-rc7-gfef64f93da2b 2023-10-11T09:26:04.198 DEBUG:teuthology.orchestra.run.smithi133:> egrep '(submenu|menuentry.*6.5.0-rc7-gfef64f93da2b).*{' /boot/grub/grub.cfg 2023-10-11T09:26:04.208 INFO:teuthology.orchestra.run.smithi133.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-10-11T09:26:04.208 INFO:teuthology.orchestra.run.smithi133.stdout: menuentry 'Ubuntu, with Linux 6.5.0-rc7-gfef64f93da2b' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.5.0-rc7-gfef64f93da2b-advanced-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-10-11T09:26:04.208 INFO:teuthology.orchestra.run.smithi133.stdout: menuentry 'Ubuntu, with Linux 6.5.0-rc7-gfef64f93da2b (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.5.0-rc7-gfef64f93da2b-recovery-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-10-11T09:26:04.209 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2023-10-11T09:26:04.209 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.5.0-rc7-gfef64f93da2b 2023-10-11T09:26:04.209 DEBUG:teuthology.orchestra.run.smithi133:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2023-10-11T09:26:04.211 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi133.front.sepia.ceph.com to complete... 2023-10-11T09:26:04.902 INFO:teuthology.orchestra.run.smithi133.stderr:Sourcing file `/etc/default/grub' 2023-10-11T09:26:04.908 INFO:teuthology.orchestra.run.smithi133.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-11T09:26:04.925 INFO:teuthology.orchestra.run.smithi133.stderr:Generating grub configuration file ... 2023-10-11T09:26:05.245 INFO:teuthology.orchestra.run.smithi133.stderr:Found linux image: /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b 2023-10-11T09:26:05.257 INFO:teuthology.orchestra.run.smithi133.stderr:Found initrd image: /boot/initrd.img-6.5.0-rc7-gfef64f93da2b 2023-10-11T09:26:05.660 INFO:teuthology.orchestra.run.smithi133.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-10-11T09:26:05.671 INFO:teuthology.orchestra.run.smithi133.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-10-11T09:26:07.349 INFO:teuthology.orchestra.run.smithi133.stderr:done 2023-10-11T09:26:34.222 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-10-11T09:26:34.223 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2023-10-11T09:26:34.223 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-11T09:26:38.462 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-10-11T09:26:38.463 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2023-10-11T09:26:38.463 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-11T09:27:34.279 DEBUG:teuthology.orchestra.remote:timed out 2023-10-11T09:27:38.470 DEBUG:teuthology.orchestra.remote:timed out 2023-10-11T09:27:43.282 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2023-10-11T09:27:43.283 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-11T09:27:43.693 ERROR:teuthology.orchestra.connection:Error authenticating with smithi078.front.sepia.ceph.com: No SSH private key found! 2023-10-11T09:27:43.694 DEBUG:teuthology.orchestra.remote:No existing session 2023-10-11T09:27:47.475 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2023-10-11T09:27:47.475 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-11T09:27:48.181 DEBUG:teuthology.orchestra.run.smithi133:> true 2023-10-11T09:27:49.679 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi133.front.sepia.ceph.com' 2023-10-11T09:27:49.679 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.5.0-rc7-gfef64f93da2b"... 2023-10-11T09:27:49.679 DEBUG:teuthology.orchestra.run.smithi133:> uname -r 2023-10-11T09:27:49.737 INFO:teuthology.orchestra.run.smithi133.stdout:6.5.0-rc7-gfef64f93da2b 2023-10-11T09:27:49.738 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc7-gfef64f93da2b vs 6.5.0-rc7-gfef64f93da2b 2023-10-11T09:27:49.738 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-11T09:27:49.738 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-10-11T09:27:50.740 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-10-11T09:27:50.740 DEBUG:teuthology.orchestra.run.smithi133:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-11T09:27:51.333 INFO:teuthology.orchestra.run.smithi133.stdout:ttyS1 2023-10-11T09:27:51.400 DEBUG:teuthology.parallel:result is None 2023-10-11T09:27:55.695 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2023-10-11T09:27:55.696 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-11T09:27:55.801 DEBUG:teuthology.orchestra.run.smithi078:> true 2023-10-11T09:27:56.571 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi078.front.sepia.ceph.com' 2023-10-11T09:27:56.571 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.5.0-rc7-gfef64f93da2b"... 2023-10-11T09:27:56.572 DEBUG:teuthology.orchestra.run.smithi078:> uname -r 2023-10-11T09:27:56.628 INFO:teuthology.orchestra.run.smithi078.stdout:6.5.0-rc7-gfef64f93da2b 2023-10-11T09:27:56.628 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc7-gfef64f93da2b vs 6.5.0-rc7-gfef64f93da2b 2023-10-11T09:27:56.628 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-11T09:27:56.628 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-10-11T09:27:57.629 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-10-11T09:27:57.630 DEBUG:teuthology.orchestra.run.smithi078:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-11T09:27:57.758 INFO:teuthology.orchestra.run.smithi078.stdout:ttyS1 2023-10-11T09:27:57.805 DEBUG:teuthology.parallel:result is None 2023-10-11T09:27:57.806 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-11T09:27:57.813 INFO:teuthology.task.internal:Creating test directory... 2023-10-11T09:27:57.813 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-11T09:27:57.820 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-11T09:27:57.839 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-11T09:27:57.846 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-11T09:27:57.856 INFO:teuthology.task.internal:Creating archive directory... 2023-10-11T09:27:57.856 DEBUG:teuthology.orchestra.run.smithi078:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-11T09:27:57.879 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-11T09:27:57.921 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-11T09:27:57.928 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-11T09:27:57.928 DEBUG:teuthology.orchestra.run.smithi078:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-11T09:27:57.947 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-11T09:27:57.983 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-11T09:27:58.002 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-11T09:27:58.005 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-11T09:27:58.026 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-11T09:27:58.029 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-11T09:27:58.036 INFO:teuthology.task.internal:Configuring sudo... 2023-10-11T09:27:58.037 DEBUG:teuthology.orchestra.run.smithi078:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-11T09:27:58.059 DEBUG:teuthology.orchestra.run.smithi133:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-11T09:27:58.111 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-11T09:27:58.120 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-11T09:27:58.120 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-11T09:27:58.142 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-11T09:27:58.173 DEBUG:teuthology.orchestra.run.smithi078:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-11T09:27:58.213 DEBUG:teuthology.orchestra.run.smithi078:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-11T09:27:58.275 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:27:58.275 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-11T09:27:58.366 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-11T09:27:58.386 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-11T09:27:58.449 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:27:58.449 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-11T09:27:58.537 DEBUG:teuthology.orchestra.run.smithi078:> sudo service rsyslog restart 2023-10-11T09:27:58.545 DEBUG:teuthology.orchestra.run.smithi133:> sudo service rsyslog restart 2023-10-11T09:27:58.710 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-11T09:27:58.719 INFO:teuthology.task.internal:Starting timer... 2023-10-11T09:27:58.719 INFO:teuthology.run_tasks:Running task pcp... 2023-10-11T09:27:58.830 INFO:teuthology.run_tasks:Running task selinux... 2023-10-11T09:27:58.839 DEBUG:teuthology.task.selinux:Excluding smithi078: OS 'ubuntu' does not support SELinux 2023-10-11T09:27:58.839 DEBUG:teuthology.task.selinux:Excluding smithi133: OS 'ubuntu' does not support SELinux 2023-10-11T09:27:58.840 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-11T09:27:58.840 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-10-11T09:27:58.840 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-11T09:27:58.840 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-11T09:27:58.849 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-10-11T09:27:58.854 INFO:teuthology.repo_utils:Fetching main from origin 2023-10-11T09:27:58.982 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-11T09:27:58.999 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-11T09:27:59.001 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi078.front.sepia.ceph.com,smithi133.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-11T09:35:44.947 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi078.front.sepia.ceph.com'), Remote(name='ubuntu@smithi133.front.sepia.ceph.com')] 2023-10-11T09:35:44.949 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2023-10-11T09:35:44.950 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-11T09:35:45.053 DEBUG:teuthology.orchestra.run.smithi078:> true 2023-10-11T09:35:45.286 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi078.front.sepia.ceph.com' 2023-10-11T09:35:45.287 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2023-10-11T09:35:45.288 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-11T09:35:45.390 DEBUG:teuthology.orchestra.run.smithi133:> true 2023-10-11T09:35:45.622 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi133.front.sepia.ceph.com' 2023-10-11T09:35:45.622 INFO:teuthology.run_tasks:Running task clock... 2023-10-11T09:35:45.632 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-11T09:35:45.633 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-11T09:35:45.633 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-11T09:35:45.641 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-11T09:35:45.641 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-11T09:35:45.707 INFO:teuthology.orchestra.run.smithi078.stdout:11 Oct 09:35:45 ntpd[22805]: ntpd 4.2.8p12@1.3728-o (1): Starting 2023-10-11T09:35:45.708 INFO:teuthology.orchestra.run.smithi078.stdout:11 Oct 09:35:45 ntpd[22805]: Command line: ntpd -gq 2023-10-11T09:35:45.709 INFO:teuthology.orchestra.run.smithi078.stdout:11 Oct 09:35:45 ntpd[22805]: proto: precision = 0.066 usec (-24) 2023-10-11T09:35:45.709 INFO:teuthology.orchestra.run.smithi078.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-11T09:35:45.710 INFO:teuthology.orchestra.run.smithi078.stdout:11 Oct 09:35:45 ntpd[22805]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-11T09:35:45.710 INFO:teuthology.orchestra.run.smithi078.stderr:restrict ::: KOD does nothing without LIMITED. 2023-10-11T09:35:45.711 INFO:teuthology.orchestra.run.smithi078.stdout:11 Oct 09:35:45 ntpd[22805]: restrict ::: KOD does nothing without LIMITED. 2023-10-11T09:35:45.712 INFO:teuthology.orchestra.run.smithi078.stdout:11 Oct 09:35:45 ntpd[22805]: Listen and drop on 0 v6wildcard [::]:123 2023-10-11T09:35:45.712 INFO:teuthology.orchestra.run.smithi078.stdout:11 Oct 09:35:45 ntpd[22805]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-10-11T09:35:45.712 INFO:teuthology.orchestra.run.smithi078.stdout:11 Oct 09:35:45 ntpd[22805]: Listen normally on 2 lo 127.0.0.1:123 2023-10-11T09:35:45.712 INFO:teuthology.orchestra.run.smithi078.stdout:11 Oct 09:35:45 ntpd[22805]: Listen normally on 3 enp3s0f1 172.21.15.78:123 2023-10-11T09:35:45.712 INFO:teuthology.orchestra.run.smithi078.stdout:11 Oct 09:35:45 ntpd[22805]: Listen normally on 4 lo [::1]:123 2023-10-11T09:35:45.713 INFO:teuthology.orchestra.run.smithi078.stdout:11 Oct 09:35:45 ntpd[22805]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d4dd%5]:123 2023-10-11T09:35:45.713 INFO:teuthology.orchestra.run.smithi078.stdout:11 Oct 09:35:45 ntpd[22805]: Listening on routing socket on fd #22 for interface updates 2023-10-11T09:35:45.747 INFO:teuthology.orchestra.run.smithi133.stdout:11 Oct 09:35:45 ntpd[22729]: ntpd 4.2.8p12@1.3728-o (1): Starting 2023-10-11T09:35:45.747 INFO:teuthology.orchestra.run.smithi133.stdout:11 Oct 09:35:45 ntpd[22729]: Command line: ntpd -gq 2023-10-11T09:35:45.749 INFO:teuthology.orchestra.run.smithi133.stdout:11 Oct 09:35:45 ntpd[22729]: proto: precision = 0.066 usec (-24) 2023-10-11T09:35:45.749 INFO:teuthology.orchestra.run.smithi133.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-11T09:35:45.749 INFO:teuthology.orchestra.run.smithi133.stdout:11 Oct 09:35:45 ntpd[22729]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-11T09:35:45.750 INFO:teuthology.orchestra.run.smithi133.stderr:restrict ::: KOD does nothing without LIMITED. 2023-10-11T09:35:45.751 INFO:teuthology.orchestra.run.smithi133.stdout:11 Oct 09:35:45 ntpd[22729]: restrict ::: KOD does nothing without LIMITED. 2023-10-11T09:35:45.751 INFO:teuthology.orchestra.run.smithi133.stdout:11 Oct 09:35:45 ntpd[22729]: Listen and drop on 0 v6wildcard [::]:123 2023-10-11T09:35:45.751 INFO:teuthology.orchestra.run.smithi133.stdout:11 Oct 09:35:45 ntpd[22729]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-10-11T09:35:45.751 INFO:teuthology.orchestra.run.smithi133.stdout:11 Oct 09:35:45 ntpd[22729]: Listen normally on 2 lo 127.0.0.1:123 2023-10-11T09:35:45.752 INFO:teuthology.orchestra.run.smithi133.stdout:11 Oct 09:35:45 ntpd[22729]: Listen normally on 3 enp3s0f1 172.21.15.133:123 2023-10-11T09:35:45.752 INFO:teuthology.orchestra.run.smithi133.stdout:11 Oct 09:35:45 ntpd[22729]: Listen normally on 4 lo [::1]:123 2023-10-11T09:35:45.752 INFO:teuthology.orchestra.run.smithi133.stdout:11 Oct 09:35:45 ntpd[22729]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:8087%5]:123 2023-10-11T09:35:45.752 INFO:teuthology.orchestra.run.smithi133.stdout:11 Oct 09:35:45 ntpd[22729]: Listening on routing socket on fd #22 for interface updates 2023-10-11T09:35:46.709 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:46 ntpd[22805]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:46.709 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:46 ntpd[22805]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:46.710 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:46 ntpd[22805]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:46.710 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:46 ntpd[22805]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:46.711 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:46 ntpd[22805]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:46.711 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:46 ntpd[22805]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:46.712 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:46 ntpd[22805]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-11T09:35:46.712 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:46 ntpd[22805]: can't open /var/log/ntpstats/peerstats.20231011: Permission denied 2023-10-11T09:35:46.712 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:46 ntpd[22805]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:46.712 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:46 ntpd[22805]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:46.713 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:46 ntpd[22805]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-11T09:35:46.713 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:46 ntpd[22805]: can't open /var/log/ntpstats/peerstats.20231011: Permission denied 2023-10-11T09:35:46.748 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:46 ntpd[22729]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:46.748 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:46 ntpd[22729]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:47.749 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:47 ntpd[22729]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:47.749 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:47 ntpd[22729]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:47.749 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:47 ntpd[22729]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:47.750 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:47 ntpd[22729]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:47.750 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:47 ntpd[22729]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-11T09:35:47.751 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:47 ntpd[22729]: can't open /var/log/ntpstats/peerstats.20231011: Permission denied 2023-10-11T09:35:48.708 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:48 ntpd[22805]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:48.708 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:48 ntpd[22805]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:48.709 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:48 ntpd[22805]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:48.709 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:48 ntpd[22805]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:48.709 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:48 ntpd[22805]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:48.710 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:48 ntpd[22805]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:48.710 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:48 ntpd[22805]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-11T09:35:48.711 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:48 ntpd[22805]: can't open /var/log/ntpstats/peerstats.20231011: Permission denied 2023-10-11T09:35:48.711 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:48 ntpd[22805]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:48.711 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:48 ntpd[22805]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:48.711 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:48 ntpd[22805]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-11T09:35:48.711 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:48 ntpd[22805]: can't open /var/log/ntpstats/peerstats.20231011: Permission denied 2023-10-11T09:35:48.748 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:48 ntpd[22729]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:48.748 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:48 ntpd[22729]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:48.749 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:48 ntpd[22729]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:48.749 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:48 ntpd[22729]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:48.749 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:48 ntpd[22729]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-11T09:35:48.749 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:48 ntpd[22729]: can't open /var/log/ntpstats/peerstats.20231011: Permission denied 2023-10-11T09:35:49.708 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:49 ntpd[22805]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:49.709 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:49 ntpd[22805]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:49.709 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:49 ntpd[22805]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:49.709 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:49 ntpd[22805]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:49.709 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:49 ntpd[22805]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:49.710 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:49 ntpd[22805]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:49.710 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:49 ntpd[22805]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-11T09:35:49.710 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:49 ntpd[22805]: can't open /var/log/ntpstats/peerstats.20231011: Permission denied 2023-10-11T09:35:49.748 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:49 ntpd[22729]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:49.748 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:49 ntpd[22729]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:49.749 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:49 ntpd[22729]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:49.749 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:49 ntpd[22729]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:49.749 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:49 ntpd[22729]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-11T09:35:49.749 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:49 ntpd[22729]: can't open /var/log/ntpstats/peerstats.20231011: Permission denied 2023-10-11T09:35:50.709 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:50 ntpd[22805]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:50.709 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:50 ntpd[22805]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:50.709 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:50 ntpd[22805]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:50.710 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:50 ntpd[22805]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:50.710 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:50 ntpd[22805]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:50.710 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:50 ntpd[22805]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:50.710 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:50 ntpd[22805]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-11T09:35:50.710 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:50 ntpd[22805]: can't open /var/log/ntpstats/peerstats.20231011: Permission denied 2023-10-11T09:35:50.711 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:50 ntpd[22805]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:50.711 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:50 ntpd[22805]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:50.712 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:50 ntpd[22805]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-11T09:35:50.712 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:50 ntpd[22805]: can't open /var/log/ntpstats/peerstats.20231011: Permission denied 2023-10-11T09:35:50.748 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:50 ntpd[22729]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:50.749 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:50 ntpd[22729]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:50.749 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:50 ntpd[22729]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:50.749 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:50 ntpd[22729]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:50.749 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:50 ntpd[22729]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:50.750 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:50 ntpd[22729]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:50.750 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:50 ntpd[22729]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-11T09:35:50.750 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:50 ntpd[22729]: can't open /var/log/ntpstats/peerstats.20231011: Permission denied 2023-10-11T09:35:50.750 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:50 ntpd[22729]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:50.750 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:50 ntpd[22729]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:50.751 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:50 ntpd[22729]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-11T09:35:50.751 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:50 ntpd[22729]: can't open /var/log/ntpstats/peerstats.20231011: Permission denied 2023-10-11T09:35:51.710 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:51 ntpd[22805]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:51.710 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:51 ntpd[22805]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:51.710 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:51 ntpd[22805]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:51.710 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:51 ntpd[22805]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:51.711 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:51 ntpd[22805]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-11T09:35:51.711 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:51 ntpd[22805]: can't open /var/log/ntpstats/peerstats.20231011: Permission denied 2023-10-11T09:35:51.748 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:51 ntpd[22729]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:51.748 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:51 ntpd[22729]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:51.749 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:51 ntpd[22729]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:51.749 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:51 ntpd[22729]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:51.749 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:51 ntpd[22729]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-11T09:35:51.749 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:51 ntpd[22729]: can't open /var/log/ntpstats/peerstats.20231011: Permission denied 2023-10-11T09:35:52.708 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:52 ntpd[22805]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:52.709 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:52 ntpd[22805]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:52.709 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:52 ntpd[22805]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:52.709 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:52 ntpd[22805]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:52.709 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:52 ntpd[22805]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:52.710 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:52 ntpd[22805]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:52.710 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:52 ntpd[22805]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-11T09:35:52.710 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:52 ntpd[22805]: can't open /var/log/ntpstats/peerstats.20231011: Permission denied 2023-10-11T09:35:52.711 INFO:teuthology.orchestra.run.smithi078.stdout:11 Oct 09:35:52 ntpd[22805]: ntpd: time slew -0.001904 s 2023-10-11T09:35:52.711 INFO:teuthology.orchestra.run.smithi078.stdout:ntpd: time slew -0.001904s 2023-10-11T09:35:52.711 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:52 ntpd[22805]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-10-11T09:35:52.712 INFO:teuthology.orchestra.run.smithi078.stderr:11 Oct 09:35:52 ntpd[22805]: can't open /var/log/ntpstats/loopstats.20231011: Permission denied 2023-10-11T09:35:52.749 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:52 ntpd[22729]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:52.749 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:52 ntpd[22729]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:52.749 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:52 ntpd[22729]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:52.749 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:52 ntpd[22729]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:52.749 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:52 ntpd[22729]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:52.750 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:52 ntpd[22729]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:52.750 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:52 ntpd[22729]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-11T09:35:52.750 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:52 ntpd[22729]: can't open /var/log/ntpstats/peerstats.20231011: Permission denied 2023-10-11T09:35:52.750 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:52 ntpd[22729]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:52.750 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:52 ntpd[22729]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:52.751 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:52 ntpd[22729]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-11T09:35:52.751 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:52 ntpd[22729]: can't open /var/log/ntpstats/peerstats.20231011: Permission denied 2023-10-11T09:35:52.855 INFO:teuthology.orchestra.run.smithi078.stdout: remote refid st t when poll reach delay offset jitter 2023-10-11T09:35:52.855 INFO:teuthology.orchestra.run.smithi078.stdout:============================================================================== 2023-10-11T09:35:52.856 INFO:teuthology.orchestra.run.smithi078.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-11T09:35:52.856 INFO:teuthology.orchestra.run.smithi078.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-11T09:35:52.856 INFO:teuthology.orchestra.run.smithi078.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-11T09:35:52.856 INFO:teuthology.orchestra.run.smithi078.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-11T09:35:53.750 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:53 ntpd[22729]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:53.750 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:53 ntpd[22729]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:53.751 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:53 ntpd[22729]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-11T09:35:53.751 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:53 ntpd[22729]: can't open /var/log/ntpstats/rawstats.20231011: Permission denied 2023-10-11T09:35:53.751 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:53 ntpd[22729]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-11T09:35:53.751 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:53 ntpd[22729]: can't open /var/log/ntpstats/peerstats.20231011: Permission denied 2023-10-11T09:35:53.751 INFO:teuthology.orchestra.run.smithi133.stdout:11 Oct 09:35:53 ntpd[22729]: ntpd: time slew +0.000733 s 2023-10-11T09:35:53.752 INFO:teuthology.orchestra.run.smithi133.stdout:ntpd: time slew +0.000733s 2023-10-11T09:35:53.752 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:53 ntpd[22729]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-10-11T09:35:53.752 INFO:teuthology.orchestra.run.smithi133.stderr:11 Oct 09:35:53 ntpd[22729]: can't open /var/log/ntpstats/loopstats.20231011: Permission denied 2023-10-11T09:35:53.892 INFO:teuthology.orchestra.run.smithi133.stdout: remote refid st t when poll reach delay offset jitter 2023-10-11T09:35:53.892 INFO:teuthology.orchestra.run.smithi133.stdout:============================================================================== 2023-10-11T09:35:53.892 INFO:teuthology.orchestra.run.smithi133.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-11T09:35:53.893 INFO:teuthology.orchestra.run.smithi133.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-11T09:35:53.893 INFO:teuthology.orchestra.run.smithi133.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-11T09:35:53.893 INFO:teuthology.orchestra.run.smithi133.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-11T09:35:53.895 INFO:teuthology.run_tasks:Running task install... 2023-10-11T09:35:53.903 DEBUG:teuthology.task.install:project ceph 2023-10-11T09:35:53.904 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '9a774796f0ecdd11a4b771de396e627aa1a187af'}} 2023-10-11T09:35:53.904 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '9a774796f0ecdd11a4b771de396e627aa1a187af'} 2023-10-11T09:35:53.905 INFO:teuthology.task.install:Using flavor: default 2023-10-11T09:35:53.910 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-11T09:35:53.911 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-11T09:35:53.911 DEBUG:teuthology.orchestra.run.smithi078:> sudo apt-key list | grep Ceph 2023-10-11T09:35:53.912 DEBUG:teuthology.orchestra.run.smithi133:> sudo apt-key list | grep Ceph 2023-10-11T09:35:54.140 INFO:teuthology.orchestra.run.smithi078.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2023-10-11T09:35:54.145 INFO:teuthology.orchestra.run.smithi133.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2023-10-11T09:35:54.292 INFO:teuthology.orchestra.run.smithi078.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-10-11T09:35:54.293 INFO:teuthology.orchestra.run.smithi078.stdout:uid [ unknown] Ceph.com (release key) 2023-10-11T09:35:54.294 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-10-11T09:35:54.295 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-10-11T09:35:54.295 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=9a774796f0ecdd11a4b771de396e627aa1a187af 2023-10-11T09:35:54.299 INFO:teuthology.orchestra.run.smithi133.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-10-11T09:35:54.299 INFO:teuthology.orchestra.run.smithi133.stdout:uid [ unknown] Ceph.com (release key) 2023-10-11T09:35:54.300 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-10-11T09:35:54.301 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-10-11T09:35:54.301 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=9a774796f0ecdd11a4b771de396e627aa1a187af 2023-10-11T09:35:54.447 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default/ 2023-10-11T09:35:54.448 INFO:teuthology.task.install.deb:Package version is 18.2.0-662-g9a774796-1focal 2023-10-11T09:35:54.512 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default/ 2023-10-11T09:35:54.512 INFO:teuthology.task.install.deb:Package version is 18.2.0-662-g9a774796-1focal 2023-10-11T09:35:54.626 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:35:54.626 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-10-11T09:35:54.661 DEBUG:teuthology.orchestra.run.smithi133:> sudo apt-get update 2023-10-11T09:35:54.679 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:35:54.679 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-10-11T09:35:54.713 DEBUG:teuthology.orchestra.run.smithi078:> sudo apt-get update 2023-10-11T09:35:54.949 INFO:teuthology.orchestra.run.smithi133.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-10-11T09:35:54.949 INFO:teuthology.orchestra.run.smithi133.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-10-11T09:35:54.963 INFO:teuthology.orchestra.run.smithi133.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-10-11T09:35:54.988 INFO:teuthology.orchestra.run.smithi133.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal InRelease 2023-10-11T09:35:54.999 INFO:teuthology.orchestra.run.smithi078.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-10-11T09:35:55.000 INFO:teuthology.orchestra.run.smithi078.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-10-11T09:35:55.009 INFO:teuthology.orchestra.run.smithi133.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal Release [7,662 B] 2023-10-11T09:35:55.017 INFO:teuthology.orchestra.run.smithi078.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-10-11T09:35:55.033 INFO:teuthology.orchestra.run.smithi133.stdout:Ign:6 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal Release.gpg 2023-10-11T09:35:55.043 INFO:teuthology.orchestra.run.smithi078.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal InRelease 2023-10-11T09:35:55.054 INFO:teuthology.orchestra.run.smithi133.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main i386 Packages [3,429 B] 2023-10-11T09:35:55.065 INFO:teuthology.orchestra.run.smithi078.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal Release [7,662 B] 2023-10-11T09:35:55.078 INFO:teuthology.orchestra.run.smithi133.stdout:Hit:8 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-10-11T09:35:55.089 INFO:teuthology.orchestra.run.smithi078.stdout:Ign:6 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal Release.gpg 2023-10-11T09:35:55.100 INFO:teuthology.orchestra.run.smithi133.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2023-10-11T09:35:55.105 INFO:teuthology.orchestra.run.smithi078.stdout:Hit:7 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-10-11T09:35:55.111 INFO:teuthology.orchestra.run.smithi078.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2023-10-11T09:35:55.157 INFO:teuthology.orchestra.run.smithi078.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main i386 Packages [3,429 B] 2023-10-11T09:35:56.378 INFO:teuthology.orchestra.run.smithi133.stdout:Fetched 28.5 kB in 1s (20.6 kB/s) 2023-10-11T09:35:56.420 INFO:teuthology.orchestra.run.smithi078.stdout:Fetched 28.5 kB in 1s (20.8 kB/s) 2023-10-11T09:35:57.638 INFO:teuthology.orchestra.run.smithi133.stdout:Reading package lists... 2023-10-11T09:35:57.643 INFO:teuthology.orchestra.run.smithi078.stdout:Reading package lists... 2023-10-11T09:35:57.679 DEBUG:teuthology.orchestra.run.smithi133:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.0-662-g9a774796-1focal cephadm=18.2.0-662-g9a774796-1focal ceph-mds=18.2.0-662-g9a774796-1focal ceph-mgr=18.2.0-662-g9a774796-1focal ceph-common=18.2.0-662-g9a774796-1focal ceph-fuse=18.2.0-662-g9a774796-1focal ceph-test=18.2.0-662-g9a774796-1focal ceph-volume=18.2.0-662-g9a774796-1focal radosgw=18.2.0-662-g9a774796-1focal python3-rados=18.2.0-662-g9a774796-1focal python3-rgw=18.2.0-662-g9a774796-1focal python3-cephfs=18.2.0-662-g9a774796-1focal python3-rbd=18.2.0-662-g9a774796-1focal libcephfs2=18.2.0-662-g9a774796-1focal libcephfs-dev=18.2.0-662-g9a774796-1focal librados2=18.2.0-662-g9a774796-1focal librbd1=18.2.0-662-g9a774796-1focal rbd-fuse=18.2.0-662-g9a774796-1focal python3-cephfs=18.2.0-662-g9a774796-1focal cephfs-shell=18.2.0-662-g9a774796-1focal cephfs-top=18.2.0-662-g9a774796-1focal cephfs-mirror=18.2.0-662-g9a774796-1focal 2023-10-11T09:35:57.680 DEBUG:teuthology.orchestra.run.smithi078:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.0-662-g9a774796-1focal cephadm=18.2.0-662-g9a774796-1focal ceph-mds=18.2.0-662-g9a774796-1focal ceph-mgr=18.2.0-662-g9a774796-1focal ceph-common=18.2.0-662-g9a774796-1focal ceph-fuse=18.2.0-662-g9a774796-1focal ceph-test=18.2.0-662-g9a774796-1focal ceph-volume=18.2.0-662-g9a774796-1focal radosgw=18.2.0-662-g9a774796-1focal python3-rados=18.2.0-662-g9a774796-1focal python3-rgw=18.2.0-662-g9a774796-1focal python3-cephfs=18.2.0-662-g9a774796-1focal python3-rbd=18.2.0-662-g9a774796-1focal libcephfs2=18.2.0-662-g9a774796-1focal libcephfs-dev=18.2.0-662-g9a774796-1focal librados2=18.2.0-662-g9a774796-1focal librbd1=18.2.0-662-g9a774796-1focal rbd-fuse=18.2.0-662-g9a774796-1focal python3-cephfs=18.2.0-662-g9a774796-1focal cephfs-shell=18.2.0-662-g9a774796-1focal cephfs-top=18.2.0-662-g9a774796-1focal cephfs-mirror=18.2.0-662-g9a774796-1focal 2023-10-11T09:35:57.800 INFO:teuthology.orchestra.run.smithi078.stdout:Reading package lists... 2023-10-11T09:35:57.806 INFO:teuthology.orchestra.run.smithi133.stdout:Reading package lists... 2023-10-11T09:35:58.006 INFO:teuthology.orchestra.run.smithi133.stdout:Building dependency tree... 2023-10-11T09:35:58.006 INFO:teuthology.orchestra.run.smithi133.stdout:Reading state information... 2023-10-11T09:35:58.007 INFO:teuthology.orchestra.run.smithi078.stdout:Building dependency tree... 2023-10-11T09:35:58.008 INFO:teuthology.orchestra.run.smithi078.stdout:Reading state information... 2023-10-11T09:35:58.183 INFO:teuthology.orchestra.run.smithi133.stdout:The following packages were automatically installed and are no longer required: 2023-10-11T09:35:58.184 INFO:teuthology.orchestra.run.smithi133.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-10-11T09:35:58.184 INFO:teuthology.orchestra.run.smithi133.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-11T09:35:58.184 INFO:teuthology.orchestra.run.smithi133.stdout:The following additional packages will be installed: 2023-10-11T09:35:58.184 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-10-11T09:35:58.185 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2023-10-11T09:35:58.186 INFO:teuthology.orchestra.run.smithi133.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2023-10-11T09:35:58.186 INFO:teuthology.orchestra.run.smithi133.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2023-10-11T09:35:58.186 INFO:teuthology.orchestra.run.smithi133.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2023-10-11T09:35:58.186 INFO:teuthology.orchestra.run.smithi133.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2023-10-11T09:35:58.186 INFO:teuthology.orchestra.run.smithi133.stdout: python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2023-10-11T09:35:58.187 INFO:teuthology.orchestra.run.smithi133.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2023-10-11T09:35:58.187 INFO:teuthology.orchestra.run.smithi133.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2023-10-11T09:35:58.187 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-common python3-cherrypy3 python3-cmd2 python3-colorama 2023-10-11T09:35:58.187 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2023-10-11T09:35:58.187 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2023-10-11T09:35:58.188 INFO:teuthology.orchestra.run.smithi133.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2023-10-11T09:35:58.188 INFO:teuthology.orchestra.run.smithi133.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2023-10-11T09:35:58.188 INFO:teuthology.orchestra.run.smithi133.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-10-11T09:35:58.188 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2023-10-11T09:35:58.188 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2023-10-11T09:35:58.189 INFO:teuthology.orchestra.run.smithi133.stdout: python3-repoze.lru python3-requests-oauthlib python3-routes python3-rsa 2023-10-11T09:35:58.189 INFO:teuthology.orchestra.run.smithi133.stdout: python3-scipy python3-simplegeneric python3-simplejson 2023-10-11T09:35:58.189 INFO:teuthology.orchestra.run.smithi133.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-10-11T09:35:58.189 INFO:teuthology.orchestra.run.smithi133.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-10-11T09:35:58.189 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-10-11T09:35:58.190 INFO:teuthology.orchestra.run.smithi133.stdout: qttranslations5-l10n socat xmlstarlet 2023-10-11T09:35:58.190 INFO:teuthology.orchestra.run.smithi133.stdout:Suggested packages: 2023-10-11T09:35:58.190 INFO:teuthology.orchestra.run.smithi133.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2023-10-11T09:35:58.191 INFO:teuthology.orchestra.run.smithi133.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2023-10-11T09:35:58.191 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2023-10-11T09:35:58.191 INFO:teuthology.orchestra.run.smithi133.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2023-10-11T09:35:58.191 INFO:teuthology.orchestra.run.smithi133.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2023-10-11T09:35:58.191 INFO:teuthology.orchestra.run.smithi133.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2023-10-11T09:35:58.192 INFO:teuthology.orchestra.run.smithi133.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2023-10-11T09:35:58.192 INFO:teuthology.orchestra.run.smithi133.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-10-11T09:35:58.192 INFO:teuthology.orchestra.run.smithi133.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2023-10-11T09:35:58.192 INFO:teuthology.orchestra.run.smithi133.stdout:Recommended packages: 2023-10-11T09:35:58.192 INFO:teuthology.orchestra.run.smithi133.stdout: btrfs-tools 2023-10-11T09:35:58.194 INFO:teuthology.orchestra.run.smithi078.stdout:The following packages were automatically installed and are no longer required: 2023-10-11T09:35:58.194 INFO:teuthology.orchestra.run.smithi078.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-10-11T09:35:58.194 INFO:teuthology.orchestra.run.smithi078.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-11T09:35:58.194 INFO:teuthology.orchestra.run.smithi078.stdout:The following additional packages will be installed: 2023-10-11T09:35:58.194 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-10-11T09:35:58.195 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2023-10-11T09:35:58.195 INFO:teuthology.orchestra.run.smithi078.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2023-10-11T09:35:58.195 INFO:teuthology.orchestra.run.smithi078.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2023-10-11T09:35:58.195 INFO:teuthology.orchestra.run.smithi078.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2023-10-11T09:35:58.195 INFO:teuthology.orchestra.run.smithi078.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2023-10-11T09:35:58.196 INFO:teuthology.orchestra.run.smithi078.stdout: python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2023-10-11T09:35:58.196 INFO:teuthology.orchestra.run.smithi078.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2023-10-11T09:35:58.196 INFO:teuthology.orchestra.run.smithi078.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2023-10-11T09:35:58.196 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-common python3-cherrypy3 python3-cmd2 python3-colorama 2023-10-11T09:35:58.196 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2023-10-11T09:35:58.197 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2023-10-11T09:35:58.197 INFO:teuthology.orchestra.run.smithi078.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2023-10-11T09:35:58.197 INFO:teuthology.orchestra.run.smithi078.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2023-10-11T09:35:58.197 INFO:teuthology.orchestra.run.smithi078.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-10-11T09:35:58.197 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2023-10-11T09:35:58.198 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2023-10-11T09:35:58.198 INFO:teuthology.orchestra.run.smithi078.stdout: python3-repoze.lru python3-requests-oauthlib python3-routes python3-rsa 2023-10-11T09:35:58.198 INFO:teuthology.orchestra.run.smithi078.stdout: python3-scipy python3-simplegeneric python3-simplejson 2023-10-11T09:35:58.198 INFO:teuthology.orchestra.run.smithi078.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-10-11T09:35:58.198 INFO:teuthology.orchestra.run.smithi078.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-10-11T09:35:58.199 INFO:teuthology.orchestra.run.smithi078.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-10-11T09:35:58.199 INFO:teuthology.orchestra.run.smithi078.stdout: qttranslations5-l10n socat xmlstarlet 2023-10-11T09:35:58.199 INFO:teuthology.orchestra.run.smithi078.stdout:Suggested packages: 2023-10-11T09:35:58.199 INFO:teuthology.orchestra.run.smithi078.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2023-10-11T09:35:58.200 INFO:teuthology.orchestra.run.smithi078.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2023-10-11T09:35:58.200 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2023-10-11T09:35:58.200 INFO:teuthology.orchestra.run.smithi078.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2023-10-11T09:35:58.200 INFO:teuthology.orchestra.run.smithi078.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2023-10-11T09:35:58.200 INFO:teuthology.orchestra.run.smithi078.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2023-10-11T09:35:58.201 INFO:teuthology.orchestra.run.smithi078.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2023-10-11T09:35:58.201 INFO:teuthology.orchestra.run.smithi078.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-10-11T09:35:58.201 INFO:teuthology.orchestra.run.smithi078.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2023-10-11T09:35:58.201 INFO:teuthology.orchestra.run.smithi078.stdout:Recommended packages: 2023-10-11T09:35:58.201 INFO:teuthology.orchestra.run.smithi078.stdout: btrfs-tools 2023-10-11T09:35:58.389 INFO:teuthology.orchestra.run.smithi133.stdout:The following NEW packages will be installed: 2023-10-11T09:35:58.390 INFO:teuthology.orchestra.run.smithi133.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-10-11T09:35:58.390 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-10-11T09:35:58.390 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-10-11T09:35:58.390 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2023-10-11T09:35:58.390 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2023-10-11T09:35:58.391 INFO:teuthology.orchestra.run.smithi133.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2023-10-11T09:35:58.391 INFO:teuthology.orchestra.run.smithi133.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2023-10-11T09:35:58.392 INFO:teuthology.orchestra.run.smithi133.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2023-10-11T09:35:58.392 INFO:teuthology.orchestra.run.smithi133.stdout: nvme-cli python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2023-10-11T09:35:58.392 INFO:teuthology.orchestra.run.smithi133.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2023-10-11T09:35:58.393 INFO:teuthology.orchestra.run.smithi133.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2023-10-11T09:35:58.393 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cmd2 2023-10-11T09:35:58.393 INFO:teuthology.orchestra.run.smithi133.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2023-10-11T09:35:58.393 INFO:teuthology.orchestra.run.smithi133.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2023-10-11T09:35:58.393 INFO:teuthology.orchestra.run.smithi133.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2023-10-11T09:35:58.393 INFO:teuthology.orchestra.run.smithi133.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2023-10-11T09:35:58.394 INFO:teuthology.orchestra.run.smithi133.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-10-11T09:35:58.394 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-10-11T09:35:58.394 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-10-11T09:35:58.394 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2023-10-11T09:35:58.394 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-10-11T09:35:58.395 INFO:teuthology.orchestra.run.smithi133.stdout: python3-scipy python3-simplegeneric python3-simplejson 2023-10-11T09:35:58.395 INFO:teuthology.orchestra.run.smithi133.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-10-11T09:35:58.395 INFO:teuthology.orchestra.run.smithi133.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-10-11T09:35:58.395 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-10-11T09:35:58.395 INFO:teuthology.orchestra.run.smithi133.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2023-10-11T09:35:58.395 INFO:teuthology.orchestra.run.smithi133.stdout:The following packages will be upgraded: 2023-10-11T09:35:58.396 INFO:teuthology.orchestra.run.smithi133.stdout: librados2 librbd1 2023-10-11T09:35:58.398 INFO:teuthology.orchestra.run.smithi078.stdout:The following NEW packages will be installed: 2023-10-11T09:35:58.398 INFO:teuthology.orchestra.run.smithi078.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-10-11T09:35:58.398 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-10-11T09:35:58.398 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-10-11T09:35:58.398 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2023-10-11T09:35:58.399 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2023-10-11T09:35:58.399 INFO:teuthology.orchestra.run.smithi078.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2023-10-11T09:35:58.399 INFO:teuthology.orchestra.run.smithi078.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2023-10-11T09:35:58.400 INFO:teuthology.orchestra.run.smithi078.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2023-10-11T09:35:58.400 INFO:teuthology.orchestra.run.smithi078.stdout: nvme-cli python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2023-10-11T09:35:58.400 INFO:teuthology.orchestra.run.smithi078.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2023-10-11T09:35:58.401 INFO:teuthology.orchestra.run.smithi078.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2023-10-11T09:35:58.401 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cmd2 2023-10-11T09:35:58.401 INFO:teuthology.orchestra.run.smithi078.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2023-10-11T09:35:58.401 INFO:teuthology.orchestra.run.smithi078.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2023-10-11T09:35:58.402 INFO:teuthology.orchestra.run.smithi078.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2023-10-11T09:35:58.402 INFO:teuthology.orchestra.run.smithi078.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2023-10-11T09:35:58.402 INFO:teuthology.orchestra.run.smithi078.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-10-11T09:35:58.402 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-10-11T09:35:58.402 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-10-11T09:35:58.402 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2023-10-11T09:35:58.403 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-10-11T09:35:58.403 INFO:teuthology.orchestra.run.smithi078.stdout: python3-scipy python3-simplegeneric python3-simplejson 2023-10-11T09:35:58.403 INFO:teuthology.orchestra.run.smithi078.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-10-11T09:35:58.403 INFO:teuthology.orchestra.run.smithi078.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-10-11T09:35:58.404 INFO:teuthology.orchestra.run.smithi078.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-10-11T09:35:58.404 INFO:teuthology.orchestra.run.smithi078.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2023-10-11T09:35:58.404 INFO:teuthology.orchestra.run.smithi078.stdout:The following packages will be upgraded: 2023-10-11T09:35:58.404 INFO:teuthology.orchestra.run.smithi078.stdout: librados2 librbd1 2023-10-11T09:35:58.484 INFO:teuthology.orchestra.run.smithi133.stdout:2 upgraded, 116 newly installed, 0 to remove and 256 not upgraded. 2023-10-11T09:35:58.484 INFO:teuthology.orchestra.run.smithi133.stdout:Need to get 208 MB of archives. 2023-10-11T09:35:58.484 INFO:teuthology.orchestra.run.smithi133.stdout:After this operation, 1,029 MB of additional disk space will be used. 2023-10-11T09:35:58.484 INFO:teuthology.orchestra.run.smithi133.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2023-10-11T09:35:58.494 INFO:teuthology.orchestra.run.smithi078.stdout:2 upgraded, 116 newly installed, 0 to remove and 256 not upgraded. 2023-10-11T09:35:58.494 INFO:teuthology.orchestra.run.smithi078.stdout:Need to get 208 MB of archives. 2023-10-11T09:35:58.495 INFO:teuthology.orchestra.run.smithi078.stdout:After this operation, 1,029 MB of additional disk space will be used. 2023-10-11T09:35:58.495 INFO:teuthology.orchestra.run.smithi078.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2023-10-11T09:35:58.506 INFO:teuthology.orchestra.run.smithi133.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2023-10-11T09:35:58.515 INFO:teuthology.orchestra.run.smithi133.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2023-10-11T09:35:58.517 INFO:teuthology.orchestra.run.smithi078.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2023-10-11T09:35:58.526 INFO:teuthology.orchestra.run.smithi078.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2023-10-11T09:35:58.545 INFO:teuthology.orchestra.run.smithi133.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2023-10-11T09:35:58.556 INFO:teuthology.orchestra.run.smithi078.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2023-10-11T09:35:58.569 INFO:teuthology.orchestra.run.smithi133.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 18.2.0-662-g9a774796-1focal [3,431 kB] 2023-10-11T09:35:58.579 INFO:teuthology.orchestra.run.smithi133.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2023-10-11T09:35:58.583 INFO:teuthology.orchestra.run.smithi133.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2023-10-11T09:35:58.589 INFO:teuthology.orchestra.run.smithi078.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2023-10-11T09:35:58.594 INFO:teuthology.orchestra.run.smithi078.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2023-10-11T09:35:58.598 INFO:teuthology.orchestra.run.smithi133.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2023-10-11T09:35:58.609 INFO:teuthology.orchestra.run.smithi078.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2023-10-11T09:35:58.649 INFO:teuthology.orchestra.run.smithi133.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2023-10-11T09:35:58.652 INFO:teuthology.orchestra.run.smithi133.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2023-10-11T09:35:58.660 INFO:teuthology.orchestra.run.smithi078.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 18.2.0-662-g9a774796-1focal [3,431 kB] 2023-10-11T09:35:58.662 INFO:teuthology.orchestra.run.smithi133.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2023-10-11T09:35:58.665 INFO:teuthology.orchestra.run.smithi133.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2023-10-11T09:35:58.666 INFO:teuthology.orchestra.run.smithi078.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2023-10-11T09:35:58.666 INFO:teuthology.orchestra.run.smithi133.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2023-10-11T09:35:58.667 INFO:teuthology.orchestra.run.smithi133.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2023-10-11T09:35:58.668 INFO:teuthology.orchestra.run.smithi133.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2023-10-11T09:35:58.669 INFO:teuthology.orchestra.run.smithi078.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2023-10-11T09:35:58.674 INFO:teuthology.orchestra.run.smithi133.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2023-10-11T09:35:58.675 INFO:teuthology.orchestra.run.smithi133.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2023-10-11T09:35:58.681 INFO:teuthology.orchestra.run.smithi133.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2023-10-11T09:35:58.682 INFO:teuthology.orchestra.run.smithi078.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2023-10-11T09:35:58.683 INFO:teuthology.orchestra.run.smithi078.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2023-10-11T09:35:58.683 INFO:teuthology.orchestra.run.smithi078.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2023-10-11T09:35:58.685 INFO:teuthology.orchestra.run.smithi078.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2023-10-11T09:35:58.686 INFO:teuthology.orchestra.run.smithi078.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2023-10-11T09:35:58.690 INFO:teuthology.orchestra.run.smithi078.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2023-10-11T09:35:58.690 INFO:teuthology.orchestra.run.smithi078.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2023-10-11T09:35:58.692 INFO:teuthology.orchestra.run.smithi078.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2023-10-11T09:35:58.697 INFO:teuthology.orchestra.run.smithi133.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2023-10-11T09:35:58.699 INFO:teuthology.orchestra.run.smithi133.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2023-10-11T09:35:58.700 INFO:teuthology.orchestra.run.smithi133.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2023-10-11T09:35:58.702 INFO:teuthology.orchestra.run.smithi133.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-10-11T09:35:58.704 INFO:teuthology.orchestra.run.smithi133.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2023-10-11T09:35:58.705 INFO:teuthology.orchestra.run.smithi133.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2023-10-11T09:35:58.708 INFO:teuthology.orchestra.run.smithi133.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2023-10-11T09:35:58.708 INFO:teuthology.orchestra.run.smithi133.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2023-10-11T09:35:58.710 INFO:teuthology.orchestra.run.smithi133.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2023-10-11T09:35:58.712 INFO:teuthology.orchestra.run.smithi078.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2023-10-11T09:35:58.713 INFO:teuthology.orchestra.run.smithi078.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2023-10-11T09:35:58.713 INFO:teuthology.orchestra.run.smithi078.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2023-10-11T09:35:58.716 INFO:teuthology.orchestra.run.smithi078.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-10-11T09:35:58.716 INFO:teuthology.orchestra.run.smithi078.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2023-10-11T09:35:58.718 INFO:teuthology.orchestra.run.smithi078.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2023-10-11T09:35:58.720 INFO:teuthology.orchestra.run.smithi133.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2023-10-11T09:35:58.721 INFO:teuthology.orchestra.run.smithi133.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2023-10-11T09:35:58.731 INFO:teuthology.orchestra.run.smithi078.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2023-10-11T09:35:58.733 INFO:teuthology.orchestra.run.smithi078.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2023-10-11T09:35:58.734 INFO:teuthology.orchestra.run.smithi078.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2023-10-11T09:35:58.737 INFO:teuthology.orchestra.run.smithi078.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2023-10-11T09:35:58.740 INFO:teuthology.orchestra.run.smithi133.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2023-10-11T09:35:58.741 INFO:teuthology.orchestra.run.smithi133.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2023-10-11T09:35:58.742 INFO:teuthology.orchestra.run.smithi133.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2023-10-11T09:35:58.744 INFO:teuthology.orchestra.run.smithi133.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2023-10-11T09:35:58.745 INFO:teuthology.orchestra.run.smithi133.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.1 [183 kB] 2023-10-11T09:35:58.749 INFO:teuthology.orchestra.run.smithi133.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2023-10-11T09:35:58.751 INFO:teuthology.orchestra.run.smithi078.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2023-10-11T09:35:58.752 INFO:teuthology.orchestra.run.smithi133.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2023-10-11T09:35:58.753 INFO:teuthology.orchestra.run.smithi133.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-asyncssh all 1.12.2-1 [152 kB] 2023-10-11T09:35:58.760 INFO:teuthology.orchestra.run.smithi133.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2023-10-11T09:35:58.760 INFO:teuthology.orchestra.run.smithi078.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2023-10-11T09:35:58.761 INFO:teuthology.orchestra.run.smithi133.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2023-10-11T09:35:58.762 INFO:teuthology.orchestra.run.smithi078.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2023-10-11T09:35:58.763 INFO:teuthology.orchestra.run.smithi078.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2023-10-11T09:35:58.763 INFO:teuthology.orchestra.run.smithi078.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2023-10-11T09:35:58.764 INFO:teuthology.orchestra.run.smithi078.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.1 [183 kB] 2023-10-11T09:35:58.771 INFO:teuthology.orchestra.run.smithi078.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2023-10-11T09:35:58.774 INFO:teuthology.orchestra.run.smithi078.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2023-10-11T09:35:58.775 INFO:teuthology.orchestra.run.smithi078.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-asyncssh all 1.12.2-1 [152 kB] 2023-10-11T09:35:58.777 INFO:teuthology.orchestra.run.smithi078.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2023-10-11T09:35:58.778 INFO:teuthology.orchestra.run.smithi133.stdout:Get:40 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 18.2.0-662-g9a774796-1focal [3,989 kB] 2023-10-11T09:35:58.779 INFO:teuthology.orchestra.run.smithi133.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2023-10-11T09:35:58.781 INFO:teuthology.orchestra.run.smithi133.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2023-10-11T09:35:58.782 INFO:teuthology.orchestra.run.smithi133.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2023-10-11T09:35:58.784 INFO:teuthology.orchestra.run.smithi133.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2023-10-11T09:35:58.785 INFO:teuthology.orchestra.run.smithi133.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2023-10-11T09:35:58.786 INFO:teuthology.orchestra.run.smithi133.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2023-10-11T09:35:58.787 INFO:teuthology.orchestra.run.smithi133.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2023-10-11T09:35:58.790 INFO:teuthology.orchestra.run.smithi133.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-10-11T09:35:58.791 INFO:teuthology.orchestra.run.smithi078.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2023-10-11T09:35:58.791 INFO:teuthology.orchestra.run.smithi078.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2023-10-11T09:35:58.792 INFO:teuthology.orchestra.run.smithi078.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2023-10-11T09:35:58.793 INFO:teuthology.orchestra.run.smithi078.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2023-10-11T09:35:58.794 INFO:teuthology.orchestra.run.smithi078.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2023-10-11T09:35:58.796 INFO:teuthology.orchestra.run.smithi078.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2023-10-11T09:35:58.799 INFO:teuthology.orchestra.run.smithi133.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2023-10-11T09:35:58.801 INFO:teuthology.orchestra.run.smithi133.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2023-10-11T09:35:58.809 INFO:teuthology.orchestra.run.smithi078.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2023-10-11T09:35:58.810 INFO:teuthology.orchestra.run.smithi078.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2023-10-11T09:35:58.812 INFO:teuthology.orchestra.run.smithi078.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-10-11T09:35:58.813 INFO:teuthology.orchestra.run.smithi078.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2023-10-11T09:35:58.818 INFO:teuthology.orchestra.run.smithi133.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2023-10-11T09:35:58.821 INFO:teuthology.orchestra.run.smithi133.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2023-10-11T09:35:58.822 INFO:teuthology.orchestra.run.smithi133.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2023-10-11T09:35:58.824 INFO:teuthology.orchestra.run.smithi133.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2023-10-11T09:35:58.838 INFO:teuthology.orchestra.run.smithi078.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2023-10-11T09:35:58.841 INFO:teuthology.orchestra.run.smithi078.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2023-10-11T09:35:58.843 INFO:teuthology.orchestra.run.smithi078.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2023-10-11T09:35:58.843 INFO:teuthology.orchestra.run.smithi078.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2023-10-11T09:35:58.844 INFO:teuthology.orchestra.run.smithi078.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2023-10-11T09:35:58.908 INFO:teuthology.orchestra.run.smithi133.stdout:Get:55 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 18.2.0-662-g9a774796-1focal [728 kB] 2023-10-11T09:35:58.929 INFO:teuthology.orchestra.run.smithi133.stdout:Get:56 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 18.2.0-662-g9a774796-1focal [377 kB] 2023-10-11T09:35:58.941 INFO:teuthology.orchestra.run.smithi133.stdout:Get:57 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 18.2.0-662-g9a774796-1focal [22.5 kB] 2023-10-11T09:35:58.942 INFO:teuthology.orchestra.run.smithi133.stdout:Get:58 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 18.2.0-662-g9a774796-1focal [191 kB] 2023-10-11T09:35:58.949 INFO:teuthology.orchestra.run.smithi133.stdout:Get:59 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 18.2.0-662-g9a774796-1focal [50.8 kB] 2023-10-11T09:35:58.950 INFO:teuthology.orchestra.run.smithi133.stdout:Get:60 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 18.2.0-662-g9a774796-1focal [355 kB] 2023-10-11T09:35:58.965 INFO:teuthology.orchestra.run.smithi133.stdout:Get:61 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 18.2.0-662-g9a774796-1focal [6,567 kB] 2023-10-11T09:35:59.011 INFO:teuthology.orchestra.run.smithi133.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2023-10-11T09:35:59.036 INFO:teuthology.orchestra.run.smithi078.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2023-10-11T09:35:59.050 INFO:teuthology.orchestra.run.smithi133.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2023-10-11T09:35:59.052 INFO:teuthology.orchestra.run.smithi133.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2023-10-11T09:35:59.077 INFO:teuthology.orchestra.run.smithi078.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2023-10-11T09:35:59.082 INFO:teuthology.orchestra.run.smithi078.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2023-10-11T09:35:59.084 INFO:teuthology.orchestra.run.smithi133.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2023-10-11T09:35:59.084 INFO:teuthology.orchestra.run.smithi133.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2023-10-11T09:35:59.085 INFO:teuthology.orchestra.run.smithi133.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2023-10-11T09:35:59.086 INFO:teuthology.orchestra.run.smithi133.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2023-10-11T09:35:59.086 INFO:teuthology.orchestra.run.smithi133.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2023-10-11T09:35:59.087 INFO:teuthology.orchestra.run.smithi133.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2023-10-11T09:35:59.087 INFO:teuthology.orchestra.run.smithi133.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2023-10-11T09:35:59.102 INFO:teuthology.orchestra.run.smithi078.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2023-10-11T09:35:59.102 INFO:teuthology.orchestra.run.smithi078.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2023-10-11T09:35:59.103 INFO:teuthology.orchestra.run.smithi078.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2023-10-11T09:35:59.104 INFO:teuthology.orchestra.run.smithi078.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2023-10-11T09:35:59.104 INFO:teuthology.orchestra.run.smithi078.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2023-10-11T09:35:59.105 INFO:teuthology.orchestra.run.smithi078.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2023-10-11T09:35:59.105 INFO:teuthology.orchestra.run.smithi078.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2023-10-11T09:35:59.107 INFO:teuthology.orchestra.run.smithi133.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2023-10-11T09:35:59.108 INFO:teuthology.orchestra.run.smithi133.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2023-10-11T09:35:59.112 INFO:teuthology.orchestra.run.smithi078.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2023-10-11T09:35:59.113 INFO:teuthology.orchestra.run.smithi078.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2023-10-11T09:35:59.129 INFO:teuthology.orchestra.run.smithi133.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2023-10-11T09:35:59.137 INFO:teuthology.orchestra.run.smithi078.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2023-10-11T09:35:59.137 INFO:teuthology.orchestra.run.smithi133.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2023-10-11T09:35:59.141 INFO:teuthology.orchestra.run.smithi133.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2023-10-11T09:35:59.142 INFO:teuthology.orchestra.run.smithi133.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2023-10-11T09:35:59.143 INFO:teuthology.orchestra.run.smithi133.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2023-10-11T09:35:59.144 INFO:teuthology.orchestra.run.smithi078.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2023-10-11T09:35:59.147 INFO:teuthology.orchestra.run.smithi133.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2023-10-11T09:35:59.147 INFO:teuthology.orchestra.run.smithi078.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2023-10-11T09:35:59.149 INFO:teuthology.orchestra.run.smithi078.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2023-10-11T09:35:59.149 INFO:teuthology.orchestra.run.smithi133.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2023-10-11T09:35:59.150 INFO:teuthology.orchestra.run.smithi078.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2023-10-11T09:35:59.150 INFO:teuthology.orchestra.run.smithi133.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2023-10-11T09:35:59.153 INFO:teuthology.orchestra.run.smithi078.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2023-10-11T09:35:59.158 INFO:teuthology.orchestra.run.smithi078.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2023-10-11T09:35:59.159 INFO:teuthology.orchestra.run.smithi078.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2023-10-11T09:35:59.190 INFO:teuthology.orchestra.run.smithi133.stdout:Get:82 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 18.2.0-662-g9a774796-1focal [103 kB] 2023-10-11T09:35:59.192 INFO:teuthology.orchestra.run.smithi133.stdout:Get:83 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 18.2.0-662-g9a774796-1focal [467 kB] 2023-10-11T09:35:59.193 INFO:teuthology.orchestra.run.smithi133.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2023-10-11T09:35:59.197 INFO:teuthology.orchestra.run.smithi133.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-asyncssh-doc all 1.12.2-1 [258 kB] 2023-10-11T09:35:59.199 INFO:teuthology.orchestra.run.smithi078.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2023-10-11T09:35:59.203 INFO:teuthology.orchestra.run.smithi078.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-asyncssh-doc all 1.12.2-1 [258 kB] 2023-10-11T09:35:59.205 INFO:teuthology.orchestra.run.smithi133.stdout:Get:86 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 18.2.0-662-g9a774796-1focal [25.1 MB] 2023-10-11T09:35:59.212 INFO:teuthology.orchestra.run.smithi133.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2023-10-11T09:35:59.217 INFO:teuthology.orchestra.run.smithi078.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2023-10-11T09:35:59.307 INFO:teuthology.orchestra.run.smithi133.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2023-10-11T09:35:59.308 INFO:teuthology.orchestra.run.smithi133.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2023-10-11T09:35:59.309 INFO:teuthology.orchestra.run.smithi133.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2023-10-11T09:35:59.309 INFO:teuthology.orchestra.run.smithi133.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2023-10-11T09:35:59.322 INFO:teuthology.orchestra.run.smithi078.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2023-10-11T09:35:59.323 INFO:teuthology.orchestra.run.smithi078.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2023-10-11T09:35:59.324 INFO:teuthology.orchestra.run.smithi078.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2023-10-11T09:35:59.325 INFO:teuthology.orchestra.run.smithi078.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2023-10-11T09:35:59.341 INFO:teuthology.orchestra.run.smithi133.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-10-11T09:35:59.342 INFO:teuthology.orchestra.run.smithi133.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2023-10-11T09:35:59.344 INFO:teuthology.orchestra.run.smithi133.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2023-10-11T09:35:59.344 INFO:teuthology.orchestra.run.smithi133.stdout:Get:95 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2023-10-11T09:35:59.345 INFO:teuthology.orchestra.run.smithi133.stdout:Get:96 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2023-10-11T09:35:59.391 INFO:teuthology.orchestra.run.smithi078.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-10-11T09:35:59.392 INFO:teuthology.orchestra.run.smithi078.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2023-10-11T09:35:59.394 INFO:teuthology.orchestra.run.smithi078.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2023-10-11T09:35:59.395 INFO:teuthology.orchestra.run.smithi078.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2023-10-11T09:35:59.395 INFO:teuthology.orchestra.run.smithi078.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2023-10-11T09:36:00.096 INFO:teuthology.orchestra.run.smithi133.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 18.2.0-662-g9a774796-1focal [11.0 MB] 2023-10-11T09:36:00.493 INFO:teuthology.orchestra.run.smithi133.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 18.2.0-662-g9a774796-1focal [224 kB] 2023-10-11T09:36:00.496 INFO:teuthology.orchestra.run.smithi133.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 18.2.0-662-g9a774796-1focal [133 kB] 2023-10-11T09:36:00.498 INFO:teuthology.orchestra.run.smithi133.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 18.2.0-662-g9a774796-1focal [1,555 kB] 2023-10-11T09:36:00.530 INFO:teuthology.orchestra.run.smithi078.stdout:Get:86 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 18.2.0-662-g9a774796-1focal [3,989 kB] 2023-10-11T09:36:00.558 INFO:teuthology.orchestra.run.smithi133.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 18.2.0-662-g9a774796-1focal [9,392 kB] 2023-10-11T09:36:00.925 INFO:teuthology.orchestra.run.smithi133.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 18.2.0-662-g9a774796-1focal [31.3 MB] 2023-10-11T09:36:01.351 INFO:teuthology.orchestra.run.smithi078.stdout:Get:87 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 18.2.0-662-g9a774796-1focal [728 kB] 2023-10-11T09:36:01.479 INFO:teuthology.orchestra.run.smithi078.stdout:Get:88 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 18.2.0-662-g9a774796-1focal [377 kB] 2023-10-11T09:36:01.540 INFO:teuthology.orchestra.run.smithi078.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 18.2.0-662-g9a774796-1focal [22.5 kB] 2023-10-11T09:36:01.542 INFO:teuthology.orchestra.run.smithi078.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 18.2.0-662-g9a774796-1focal [191 kB] 2023-10-11T09:36:01.568 INFO:teuthology.orchestra.run.smithi078.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 18.2.0-662-g9a774796-1focal [50.8 kB] 2023-10-11T09:36:01.577 INFO:teuthology.orchestra.run.smithi078.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 18.2.0-662-g9a774796-1focal [355 kB] 2023-10-11T09:36:01.636 INFO:teuthology.orchestra.run.smithi078.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 18.2.0-662-g9a774796-1focal [6,567 kB] 2023-10-11T09:36:02.227 INFO:teuthology.orchestra.run.smithi133.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 18.2.0-662-g9a774796-1focal [3,868 B] 2023-10-11T09:36:02.228 INFO:teuthology.orchestra.run.smithi133.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 18.2.0-662-g9a774796-1focal [836 kB] 2023-10-11T09:36:02.249 INFO:teuthology.orchestra.run.smithi133.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 18.2.0-662-g9a774796-1focal [2,304 kB] 2023-10-11T09:36:02.353 INFO:teuthology.orchestra.run.smithi133.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 18.2.0-662-g9a774796-1focal [221 kB] 2023-10-11T09:36:02.362 INFO:teuthology.orchestra.run.smithi133.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 18.2.0-662-g9a774796-1focal [126 kB] 2023-10-11T09:36:02.369 INFO:teuthology.orchestra.run.smithi133.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 18.2.0-662-g9a774796-1focal [1,831 kB] 2023-10-11T09:36:02.482 INFO:teuthology.orchestra.run.smithi133.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 18.2.0-662-g9a774796-1focal [7,876 kB] 2023-10-11T09:36:02.518 INFO:teuthology.orchestra.run.smithi078.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 18.2.0-662-g9a774796-1focal [103 kB] 2023-10-11T09:36:02.519 INFO:teuthology.orchestra.run.smithi078.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 18.2.0-662-g9a774796-1focal [467 kB] 2023-10-11T09:36:02.572 INFO:teuthology.orchestra.run.smithi078.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 18.2.0-662-g9a774796-1focal [25.1 MB] 2023-10-11T09:36:02.811 INFO:teuthology.orchestra.run.smithi133.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 18.2.0-662-g9a774796-1focal [3,988 B] 2023-10-11T09:36:02.812 INFO:teuthology.orchestra.run.smithi133.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 18.2.0-662-g9a774796-1focal [50.0 MB] 2023-10-11T09:36:05.141 INFO:teuthology.orchestra.run.smithi133.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 18.2.0-662-g9a774796-1focal [223 kB] 2023-10-11T09:36:05.143 INFO:teuthology.orchestra.run.smithi133.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 18.2.0-662-g9a774796-1focal [256 kB] 2023-10-11T09:36:05.146 INFO:teuthology.orchestra.run.smithi133.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 18.2.0-662-g9a774796-1focal [23.3 kB] 2023-10-11T09:36:05.146 INFO:teuthology.orchestra.run.smithi133.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 18.2.0-662-g9a774796-1focal [12.9 kB] 2023-10-11T09:36:05.147 INFO:teuthology.orchestra.run.smithi133.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 18.2.0-662-g9a774796-1focal [28.0 kB] 2023-10-11T09:36:05.147 INFO:teuthology.orchestra.run.smithi133.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 18.2.0-662-g9a774796-1focal [13.4 MB] 2023-10-11T09:36:05.427 INFO:teuthology.orchestra.run.smithi078.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 18.2.0-662-g9a774796-1focal [11.0 MB] 2023-10-11T09:36:05.781 INFO:teuthology.orchestra.run.smithi133.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 18.2.0-662-g9a774796-1focal [97.4 kB] 2023-10-11T09:36:06.138 INFO:teuthology.orchestra.run.smithi078.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 18.2.0-662-g9a774796-1focal [224 kB] 2023-10-11T09:36:06.150 INFO:teuthology.orchestra.run.smithi078.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 18.2.0-662-g9a774796-1focal [133 kB] 2023-10-11T09:36:06.155 INFO:teuthology.orchestra.run.smithi078.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 18.2.0-662-g9a774796-1focal [1,555 kB] 2023-10-11T09:36:06.228 INFO:teuthology.orchestra.run.smithi078.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 18.2.0-662-g9a774796-1focal [9,392 kB] 2023-10-11T09:36:06.671 INFO:teuthology.orchestra.run.smithi078.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 18.2.0-662-g9a774796-1focal [31.3 MB] 2023-10-11T09:36:06.697 INFO:teuthology.orchestra.run.smithi133.stdout:Fetched 208 MB in 7s (28.2 MB/s) 2023-10-11T09:36:06.873 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-pyperclip. 2023-10-11T09:36:06.940 INFO:teuthology.orchestra.run.smithi133.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 107762 files and directories currently installed.) 2023-10-11T09:36:06.943 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2023-10-11T09:36:06.989 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2023-10-11T09:36:07.350 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-wcwidth. 2023-10-11T09:36:07.369 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2023-10-11T09:36:07.408 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2023-10-11T09:36:07.753 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-cmd2. 2023-10-11T09:36:07.772 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2023-10-11T09:36:07.811 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2023-10-11T09:36:08.085 INFO:teuthology.orchestra.run.smithi078.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 18.2.0-662-g9a774796-1focal [3,868 B] 2023-10-11T09:36:08.086 INFO:teuthology.orchestra.run.smithi078.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 18.2.0-662-g9a774796-1focal [836 kB] 2023-10-11T09:36:08.125 INFO:teuthology.orchestra.run.smithi078.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 18.2.0-662-g9a774796-1focal [2,304 kB] 2023-10-11T09:36:08.215 INFO:teuthology.orchestra.run.smithi078.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 18.2.0-662-g9a774796-1focal [221 kB] 2023-10-11T09:36:08.222 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2023-10-11T09:36:08.239 INFO:teuthology.orchestra.run.smithi078.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 18.2.0-662-g9a774796-1focal [126 kB] 2023-10-11T09:36:08.241 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2023-10-11T09:36:08.243 INFO:teuthology.orchestra.run.smithi078.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 18.2.0-662-g9a774796-1focal [1,831 kB] 2023-10-11T09:36:08.320 INFO:teuthology.orchestra.run.smithi078.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 18.2.0-662-g9a774796-1focal [7,876 kB] 2023-10-11T09:36:08.322 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2023-10-11T09:36:08.676 INFO:teuthology.orchestra.run.smithi078.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 18.2.0-662-g9a774796-1focal [3,988 B] 2023-10-11T09:36:08.677 INFO:teuthology.orchestra.run.smithi078.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 18.2.0-662-g9a774796-1focal [50.0 MB] 2023-10-11T09:36:08.719 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-10-11T09:36:08.737 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2023-10-11T09:36:08.776 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-10-11T09:36:09.172 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-10-11T09:36:09.190 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../005-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2023-10-11T09:36:09.263 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-10-11T09:36:09.791 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-10-11T09:36:09.810 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../006-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-11T09:36:09.873 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-11T09:36:10.428 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-10-11T09:36:10.447 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../007-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-11T09:36:10.486 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-11T09:36:10.914 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-10-11T09:36:10.933 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../008-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-11T09:36:10.943 INFO:teuthology.orchestra.run.smithi078.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 18.2.0-662-g9a774796-1focal [223 kB] 2023-10-11T09:36:10.946 INFO:teuthology.orchestra.run.smithi078.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 18.2.0-662-g9a774796-1focal [256 kB] 2023-10-11T09:36:10.948 INFO:teuthology.orchestra.run.smithi078.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 18.2.0-662-g9a774796-1focal [23.3 kB] 2023-10-11T09:36:10.949 INFO:teuthology.orchestra.run.smithi078.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 18.2.0-662-g9a774796-1focal [12.9 kB] 2023-10-11T09:36:10.949 INFO:teuthology.orchestra.run.smithi078.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 18.2.0-662-g9a774796-1focal [28.0 kB] 2023-10-11T09:36:10.950 INFO:teuthology.orchestra.run.smithi078.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 18.2.0-662-g9a774796-1focal [13.4 MB] 2023-10-11T09:36:10.972 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-11T09:36:11.384 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libthrift-0.13.0. 2023-10-11T09:36:11.401 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../009-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2023-10-11T09:36:11.442 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2023-10-11T09:36:11.568 INFO:teuthology.orchestra.run.smithi078.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 18.2.0-662-g9a774796-1focal [97.4 kB] 2023-10-11T09:36:11.848 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../010-librbd1_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:36:11.996 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking librbd1 (18.2.0-662-g9a774796-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-10-11T09:36:12.489 INFO:teuthology.orchestra.run.smithi078.stdout:Fetched 208 MB in 13s (15.8 MB/s) 2023-10-11T09:36:12.648 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-pyperclip. 2023-10-11T09:36:12.717 INFO:teuthology.orchestra.run.smithi078.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 107762 files and directories currently installed.) 2023-10-11T09:36:12.720 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2023-10-11T09:36:12.761 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2023-10-11T09:36:12.977 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../011-librados2_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:36:13.106 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-wcwidth. 2023-10-11T09:36:13.124 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2023-10-11T09:36:13.134 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking librados2 (18.2.0-662-g9a774796-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-10-11T09:36:13.164 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2023-10-11T09:36:13.450 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-cmd2. 2023-10-11T09:36:13.468 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2023-10-11T09:36:13.500 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2023-10-11T09:36:13.869 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2023-10-11T09:36:13.888 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2023-10-11T09:36:13.955 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libcephfs2. 2023-10-11T09:36:13.969 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2023-10-11T09:36:13.974 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../012-libcephfs2_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:36:14.013 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libcephfs2 (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:14.381 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-10-11T09:36:14.399 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-rados. 2023-10-11T09:36:14.400 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2023-10-11T09:36:14.418 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../013-python3-rados_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:36:14.439 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-10-11T09:36:14.457 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-rados (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:14.809 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-10-11T09:36:14.827 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../005-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2023-10-11T09:36:14.860 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-10-11T09:36:14.867 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-10-11T09:36:14.877 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../014-python3-ceph-argparse_18.2.0-662-g9a774796-1focal_all.deb ... 2023-10-11T09:36:14.918 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-ceph-argparse (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:15.229 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-cephfs. 2023-10-11T09:36:15.248 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../015-python3-cephfs_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:36:15.287 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-cephfs (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:15.304 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-10-11T09:36:15.323 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../006-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-11T09:36:15.386 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-11T09:36:15.665 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-ceph-common. 2023-10-11T09:36:15.685 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../016-python3-ceph-common_18.2.0-662-g9a774796-1focal_all.deb ... 2023-10-11T09:36:15.723 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-ceph-common (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:15.958 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-10-11T09:36:15.977 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../007-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-11T09:36:16.016 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-11T09:36:16.144 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-prettytable. 2023-10-11T09:36:16.161 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../017-python3-prettytable_0.7.2-5_all.deb ... 2023-10-11T09:36:16.226 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2023-10-11T09:36:16.444 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-10-11T09:36:16.463 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../008-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-11T09:36:16.494 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-11T09:36:16.521 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-rbd. 2023-10-11T09:36:16.539 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../018-python3-rbd_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:36:16.618 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-rbd (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:16.897 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libthrift-0.13.0. 2023-10-11T09:36:16.915 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../009-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2023-10-11T09:36:16.955 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2023-10-11T09:36:17.032 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2023-10-11T09:36:17.050 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../019-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2023-10-11T09:36:17.090 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-10-11T09:36:17.376 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../010-librbd1_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:36:17.500 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking librbd1 (18.2.0-662-g9a774796-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-10-11T09:36:17.501 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package librabbitmq4:amd64. 2023-10-11T09:36:17.521 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../020-librabbitmq4_0.10.0-1_amd64.deb ... 2023-10-11T09:36:17.559 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2023-10-11T09:36:17.963 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-10-11T09:36:17.982 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../021-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2023-10-11T09:36:18.020 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-10-11T09:36:18.263 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../011-librados2_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:36:18.382 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package librgw2. 2023-10-11T09:36:18.387 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking librados2 (18.2.0-662-g9a774796-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-10-11T09:36:18.400 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../022-librgw2_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:36:18.440 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking librgw2 (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:19.175 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libcephfs2. 2023-10-11T09:36:19.193 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../012-libcephfs2_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:36:19.233 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libcephfs2 (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:19.351 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-rgw. 2023-10-11T09:36:19.370 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../023-python3-rgw_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:36:19.409 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-rgw (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:19.594 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-rados. 2023-10-11T09:36:19.612 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../013-python3-rados_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:36:19.644 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-rados (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:19.804 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package liboath0. 2023-10-11T09:36:19.822 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2023-10-11T09:36:19.862 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2023-10-11T09:36:20.030 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-10-11T09:36:20.048 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../014-python3-ceph-argparse_18.2.0-662-g9a774796-1focal_all.deb ... 2023-10-11T09:36:20.113 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-ceph-argparse (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:20.257 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libradosstriper1. 2023-10-11T09:36:20.276 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../025-libradosstriper1_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:36:20.314 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libradosstriper1 (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:20.408 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-cephfs. 2023-10-11T09:36:20.427 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../015-python3-cephfs_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:36:20.465 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-cephfs (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:20.843 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-ceph-common. 2023-10-11T09:36:20.862 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../016-python3-ceph-common_18.2.0-662-g9a774796-1focal_all.deb ... 2023-10-11T09:36:20.868 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-common. 2023-10-11T09:36:20.887 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../026-ceph-common_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:36:20.901 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-ceph-common (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:20.926 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-common (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:21.296 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-prettytable. 2023-10-11T09:36:21.314 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../017-python3-prettytable_0.7.2-5_all.deb ... 2023-10-11T09:36:21.354 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2023-10-11T09:36:21.642 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-rbd. 2023-10-11T09:36:21.661 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../018-python3-rbd_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:36:21.734 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-rbd (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:22.103 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2023-10-11T09:36:22.123 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../019-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2023-10-11T09:36:22.161 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-10-11T09:36:22.589 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package librabbitmq4:amd64. 2023-10-11T09:36:22.608 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../020-librabbitmq4_0.10.0-1_amd64.deb ... 2023-10-11T09:36:22.648 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2023-10-11T09:36:23.051 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-10-11T09:36:23.071 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../021-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2023-10-11T09:36:23.109 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-10-11T09:36:23.529 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package librgw2. 2023-10-11T09:36:23.547 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../022-librgw2_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:36:23.587 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking librgw2 (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:23.773 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-base. 2023-10-11T09:36:23.793 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../027-ceph-base_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:36:23.852 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-base (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:24.523 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-rgw. 2023-10-11T09:36:24.540 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../023-python3-rgw_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:36:24.581 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-rgw (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:24.984 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package liboath0. 2023-10-11T09:36:24.996 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2023-10-11T09:36:25.033 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2023-10-11T09:36:25.336 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-cherrypy3. 2023-10-11T09:36:25.337 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libradosstriper1. 2023-10-11T09:36:25.355 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../028-python3-cherrypy3_8.9.1-6_all.deb ... 2023-10-11T09:36:25.356 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../025-libradosstriper1_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:36:25.394 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2023-10-11T09:36:25.395 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libradosstriper1 (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:25.781 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-natsort. 2023-10-11T09:36:25.798 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../029-python3-natsort_7.0.1-1_all.deb ... 2023-10-11T09:36:25.830 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-natsort (7.0.1-1) ... 2023-10-11T09:36:25.915 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package ceph-common. 2023-10-11T09:36:25.932 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../026-ceph-common_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:36:25.973 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking ceph-common (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:26.167 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-logutils. 2023-10-11T09:36:26.185 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../030-python3-logutils_0.3.3-7_all.deb ... 2023-10-11T09:36:26.225 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-logutils (0.3.3-7) ... 2023-10-11T09:36:26.587 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-markupsafe. 2023-10-11T09:36:26.605 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../031-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2023-10-11T09:36:26.644 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2023-10-11T09:36:27.006 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-mako. 2023-10-11T09:36:27.024 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../032-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2023-10-11T09:36:27.064 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-10-11T09:36:27.450 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-simplegeneric. 2023-10-11T09:36:27.463 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../033-python3-simplegeneric_0.8.1-3_all.deb ... 2023-10-11T09:36:27.529 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-10-11T09:36:27.845 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-singledispatch. 2023-10-11T09:36:27.858 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../034-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2023-10-11T09:36:27.936 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2023-10-11T09:36:28.948 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-webob. 2023-10-11T09:36:28.967 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../035-python3-webob_1%3a1.8.5-2_all.deb ... 2023-10-11T09:36:29.006 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2023-10-11T09:36:29.045 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package ceph-base. 2023-10-11T09:36:29.064 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../027-ceph-base_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:36:29.124 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking ceph-base (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:29.376 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-soupsieve. 2023-10-11T09:36:29.394 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../036-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2023-10-11T09:36:29.434 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2023-10-11T09:36:29.804 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-bs4. 2023-10-11T09:36:29.823 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../037-python3-bs4_4.8.2-1_all.deb ... 2023-10-11T09:36:29.878 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2023-10-11T09:36:30.307 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-waitress. 2023-10-11T09:36:30.325 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../038-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2023-10-11T09:36:30.372 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2023-10-11T09:36:30.583 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-cherrypy3. 2023-10-11T09:36:30.603 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../028-python3-cherrypy3_8.9.1-6_all.deb ... 2023-10-11T09:36:30.641 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2023-10-11T09:36:30.760 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-tempita. 2023-10-11T09:36:30.777 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../039-python3-tempita_0.5.2-6_all.deb ... 2023-10-11T09:36:30.817 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-tempita (0.5.2-6) ... 2023-10-11T09:36:31.003 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-natsort. 2023-10-11T09:36:31.015 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../029-python3-natsort_7.0.1-1_all.deb ... 2023-10-11T09:36:31.052 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-natsort (7.0.1-1) ... 2023-10-11T09:36:31.188 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-paste. 2023-10-11T09:36:31.206 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../040-python3-paste_3.4.0+dfsg1-1_all.deb ... 2023-10-11T09:36:31.245 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2023-10-11T09:36:31.397 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-logutils. 2023-10-11T09:36:31.410 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../030-python3-logutils_0.3.3-7_all.deb ... 2023-10-11T09:36:31.447 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-logutils (0.3.3-7) ... 2023-10-11T09:36:31.649 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-10-11T09:36:31.667 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../041-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2023-10-11T09:36:31.707 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2023-10-11T09:36:31.750 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-markupsafe. 2023-10-11T09:36:31.762 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../031-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2023-10-11T09:36:31.809 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2023-10-11T09:36:32.068 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-pastedeploy. 2023-10-11T09:36:32.087 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../042-python3-pastedeploy_2.1.0-1_all.deb ... 2023-10-11T09:36:32.136 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-mako. 2023-10-11T09:36:32.149 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../032-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2023-10-11T09:36:32.164 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2023-10-11T09:36:32.228 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-10-11T09:36:32.546 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-webtest. 2023-10-11T09:36:32.565 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../043-python3-webtest_2.0.32-2_all.deb ... 2023-10-11T09:36:32.598 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-simplegeneric. 2023-10-11T09:36:32.604 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-webtest (2.0.32-2) ... 2023-10-11T09:36:32.610 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../033-python3-simplegeneric_0.8.1-3_all.deb ... 2023-10-11T09:36:32.647 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-10-11T09:36:32.974 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-pecan. 2023-10-11T09:36:32.993 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../044-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2023-10-11T09:36:33.026 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-singledispatch. 2023-10-11T09:36:33.031 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2023-10-11T09:36:33.045 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../034-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2023-10-11T09:36:33.084 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2023-10-11T09:36:33.437 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-webob. 2023-10-11T09:36:33.453 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-werkzeug. 2023-10-11T09:36:33.455 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../035-python3-webob_1%3a1.8.5-2_all.deb ... 2023-10-11T09:36:33.473 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../045-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.1_all.deb ... 2023-10-11T09:36:33.495 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2023-10-11T09:36:33.528 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-10-11T09:36:33.890 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-soupsieve. 2023-10-11T09:36:33.898 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-10-11T09:36:33.908 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../036-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2023-10-11T09:36:33.916 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../046-ceph-mgr-modules-core_18.2.0-662-g9a774796-1focal_all.deb ... 2023-10-11T09:36:33.948 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2023-10-11T09:36:33.956 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-mgr-modules-core (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:34.293 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-bs4. 2023-10-11T09:36:34.311 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../037-python3-bs4_4.8.2-1_all.deb ... 2023-10-11T09:36:34.376 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2023-10-11T09:36:34.427 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-10-11T09:36:34.443 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../047-libsqlite3-mod-ceph_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:36:34.502 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libsqlite3-mod-ceph (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:34.704 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-waitress. 2023-10-11T09:36:34.722 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../038-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2023-10-11T09:36:34.760 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2023-10-11T09:36:34.863 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-cryptography. 2023-10-11T09:36:34.876 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../048-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2023-10-11T09:36:34.921 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2023-10-11T09:36:35.107 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-tempita. 2023-10-11T09:36:35.125 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../039-python3-tempita_0.5.2-6_all.deb ... 2023-10-11T09:36:35.156 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-tempita (0.5.2-6) ... 2023-10-11T09:36:35.375 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-openssl. 2023-10-11T09:36:35.394 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../049-python3-openssl_19.0.0-1build1_all.deb ... 2023-10-11T09:36:35.433 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2023-10-11T09:36:35.485 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-paste. 2023-10-11T09:36:35.503 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../040-python3-paste_3.4.0+dfsg1-1_all.deb ... 2023-10-11T09:36:35.542 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2023-10-11T09:36:35.862 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-mgr. 2023-10-11T09:36:35.880 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../050-ceph-mgr_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:36:35.896 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-10-11T09:36:35.914 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../041-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2023-10-11T09:36:35.919 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-mgr (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:35.954 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2023-10-11T09:36:36.307 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-pastedeploy. 2023-10-11T09:36:36.327 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../042-python3-pastedeploy_2.1.0-1_all.deb ... 2023-10-11T09:36:36.365 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2023-10-11T09:36:36.465 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-mon. 2023-10-11T09:36:36.483 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../051-ceph-mon_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:36:36.522 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-mon (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:36.744 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-webtest. 2023-10-11T09:36:36.763 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../043-python3-webtest_2.0.32-2_all.deb ... 2023-10-11T09:36:36.801 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-webtest (2.0.32-2) ... 2023-10-11T09:36:37.188 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-pecan. 2023-10-11T09:36:37.207 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../044-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2023-10-11T09:36:37.246 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2023-10-11T09:36:37.693 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-werkzeug. 2023-10-11T09:36:37.710 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../045-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.1_all.deb ... 2023-10-11T09:36:37.759 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-10-11T09:36:37.876 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-osd. 2023-10-11T09:36:37.895 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../052-ceph-osd_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:36:37.951 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-osd (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:38.071 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-10-11T09:36:38.089 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../046-ceph-mgr-modules-core_18.2.0-662-g9a774796-1focal_all.deb ... 2023-10-11T09:36:38.129 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking ceph-mgr-modules-core (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:38.542 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-10-11T09:36:38.560 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../047-libsqlite3-mod-ceph_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:36:38.600 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libsqlite3-mod-ceph (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:38.978 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-cryptography. 2023-10-11T09:36:38.997 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../048-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2023-10-11T09:36:39.036 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2023-10-11T09:36:39.482 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-openssl. 2023-10-11T09:36:39.500 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../049-python3-openssl_19.0.0-1build1_all.deb ... 2023-10-11T09:36:39.539 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2023-10-11T09:36:39.934 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package ceph-mgr. 2023-10-11T09:36:39.952 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../050-ceph-mgr_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:36:39.992 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking ceph-mgr (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:40.529 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package ceph-mon. 2023-10-11T09:36:40.548 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../051-ceph-mon_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:36:40.587 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking ceph-mon (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:40.704 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph. 2023-10-11T09:36:40.724 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../053-ceph_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:36:40.762 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:41.123 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-fuse. 2023-10-11T09:36:41.142 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../054-ceph-fuse_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:36:41.181 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-fuse (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:41.643 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-mds. 2023-10-11T09:36:41.661 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../055-ceph-mds_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:36:41.700 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-mds (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:41.807 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package ceph-osd. 2023-10-11T09:36:41.827 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../052-ceph-osd_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:36:41.857 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking ceph-osd (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:42.279 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package cephadm. 2023-10-11T09:36:42.298 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../056-cephadm_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:36:42.328 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking cephadm (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:42.706 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-asyncssh. 2023-10-11T09:36:42.724 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../057-python3-asyncssh_1.12.2-1_all.deb ... 2023-10-11T09:36:42.765 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-asyncssh (1.12.2-1) ... 2023-10-11T09:36:43.184 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-jinja2. 2023-10-11T09:36:43.202 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../058-python3-jinja2_2.10.1-2_all.deb ... 2023-10-11T09:36:43.258 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2023-10-11T09:36:43.637 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-10-11T09:36:43.654 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../059-ceph-mgr-cephadm_18.2.0-662-g9a774796-1focal_all.deb ... 2023-10-11T09:36:43.720 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-mgr-cephadm (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:44.106 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-bcrypt. 2023-10-11T09:36:44.127 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../060-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2023-10-11T09:36:44.164 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2023-10-11T09:36:44.534 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-jwt. 2023-10-11T09:36:44.553 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../061-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2023-10-11T09:36:44.593 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2023-10-11T09:36:44.635 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package ceph. 2023-10-11T09:36:44.654 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../053-ceph_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:36:44.693 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking ceph (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:44.972 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-attr. 2023-10-11T09:36:44.991 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../062-python3-attr_19.3.0-2_all.deb ... 2023-10-11T09:36:45.030 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-attr (19.3.0-2) ... 2023-10-11T09:36:45.080 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package ceph-fuse. 2023-10-11T09:36:45.099 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../054-ceph-fuse_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:36:45.138 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking ceph-fuse (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:45.416 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-pluggy. 2023-10-11T09:36:45.434 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../063-python3-pluggy_0.13.0-2_all.deb ... 2023-10-11T09:36:45.474 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2023-10-11T09:36:45.608 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package ceph-mds. 2023-10-11T09:36:45.627 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../055-ceph-mds_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:36:45.665 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking ceph-mds (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:45.844 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-py. 2023-10-11T09:36:45.862 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../064-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2023-10-11T09:36:45.901 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2023-10-11T09:36:46.261 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package cephadm. 2023-10-11T09:36:46.279 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../056-cephadm_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:36:46.297 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-atomicwrites. 2023-10-11T09:36:46.315 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../065-python3-atomicwrites_1.1.5-2build1_all.deb ... 2023-10-11T09:36:46.318 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking cephadm (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:46.354 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2023-10-11T09:36:46.691 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-packaging. 2023-10-11T09:36:46.697 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-asyncssh. 2023-10-11T09:36:46.710 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../066-python3-packaging_20.3-1_all.deb ... 2023-10-11T09:36:46.715 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../057-python3-asyncssh_1.12.2-1_all.deb ... 2023-10-11T09:36:46.749 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-packaging (20.3-1) ... 2023-10-11T09:36:46.754 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-asyncssh (1.12.2-1) ... 2023-10-11T09:36:47.110 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-pytest. 2023-10-11T09:36:47.128 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../067-python3-pytest_4.6.9-1_all.deb ... 2023-10-11T09:36:47.183 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-jinja2. 2023-10-11T09:36:47.192 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-pytest (4.6.9-1) ... 2023-10-11T09:36:47.202 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../058-python3-jinja2_2.10.1-2_all.deb ... 2023-10-11T09:36:47.240 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2023-10-11T09:36:47.588 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-repoze.lru. 2023-10-11T09:36:47.607 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../068-python3-repoze.lru_0.7-2_all.deb ... 2023-10-11T09:36:47.644 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-10-11T09:36:47.646 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-10-11T09:36:47.663 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../059-ceph-mgr-cephadm_18.2.0-662-g9a774796-1focal_all.deb ... 2023-10-11T09:36:47.701 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking ceph-mgr-cephadm (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:47.957 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libjs-underscore. 2023-10-11T09:36:47.969 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../069-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2023-10-11T09:36:48.006 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-10-11T09:36:48.088 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-bcrypt. 2023-10-11T09:36:48.107 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../060-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2023-10-11T09:36:48.171 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2023-10-11T09:36:48.343 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libjs-sphinxdoc. 2023-10-11T09:36:48.361 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../070-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2023-10-11T09:36:48.400 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-10-11T09:36:48.491 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-jwt. 2023-10-11T09:36:48.510 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../061-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2023-10-11T09:36:48.549 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2023-10-11T09:36:48.795 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-routes. 2023-10-11T09:36:48.813 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../071-python3-routes_2.4.1-2_all.deb ... 2023-10-11T09:36:48.852 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-routes (2.4.1-2) ... 2023-10-11T09:36:48.927 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-attr. 2023-10-11T09:36:48.947 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../062-python3-attr_19.3.0-2_all.deb ... 2023-10-11T09:36:48.985 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-attr (19.3.0-2) ... 2023-10-11T09:36:49.247 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-10-11T09:36:49.260 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../072-ceph-mgr-dashboard_18.2.0-662-g9a774796-1focal_all.deb ... 2023-10-11T09:36:49.305 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-mgr-dashboard (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:49.363 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-pluggy. 2023-10-11T09:36:49.382 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../063-python3-pluggy_0.13.0-2_all.deb ... 2023-10-11T09:36:49.421 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2023-10-11T09:36:49.816 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-py. 2023-10-11T09:36:49.835 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../064-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2023-10-11T09:36:49.874 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2023-10-11T09:36:49.925 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-decorator. 2023-10-11T09:36:49.938 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../073-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2023-10-11T09:36:49.983 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2023-10-11T09:36:50.236 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-atomicwrites. 2023-10-11T09:36:50.255 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../065-python3-atomicwrites_1.1.5-2build1_all.deb ... 2023-10-11T09:36:50.294 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2023-10-11T09:36:50.328 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2023-10-11T09:36:50.341 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../074-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2023-10-11T09:36:50.377 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-10-11T09:36:50.663 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-packaging. 2023-10-11T09:36:50.682 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../066-python3-packaging_20.3-1_all.deb ... 2023-10-11T09:36:50.721 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-packaging (20.3-1) ... 2023-10-11T09:36:50.739 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-scipy. 2023-10-11T09:36:50.752 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../075-python3-scipy_1.3.3-3build1_amd64.deb ... 2023-10-11T09:36:50.797 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2023-10-11T09:36:51.099 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-pytest. 2023-10-11T09:36:51.118 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../067-python3-pytest_4.6.9-1_all.deb ... 2023-10-11T09:36:51.157 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-pytest (4.6.9-1) ... 2023-10-11T09:36:51.536 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-repoze.lru. 2023-10-11T09:36:51.556 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../068-python3-repoze.lru_0.7-2_all.deb ... 2023-10-11T09:36:51.594 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-10-11T09:36:51.872 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libjs-underscore. 2023-10-11T09:36:51.891 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../069-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2023-10-11T09:36:51.929 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-10-11T09:36:52.220 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-sklearn-lib. 2023-10-11T09:36:52.239 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../076-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2023-10-11T09:36:52.266 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libjs-sphinxdoc. 2023-10-11T09:36:52.277 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-10-11T09:36:52.284 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../070-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2023-10-11T09:36:52.323 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-10-11T09:36:52.701 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-routes. 2023-10-11T09:36:52.720 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../071-python3-routes_2.4.1-2_all.deb ... 2023-10-11T09:36:52.759 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-routes (2.4.1-2) ... 2023-10-11T09:36:52.873 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-joblib. 2023-10-11T09:36:52.891 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../077-python3-joblib_0.14.0-3_all.deb ... 2023-10-11T09:36:52.968 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-joblib (0.14.0-3) ... 2023-10-11T09:36:53.163 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-10-11T09:36:53.181 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../072-ceph-mgr-dashboard_18.2.0-662-g9a774796-1focal_all.deb ... 2023-10-11T09:36:53.220 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking ceph-mgr-dashboard (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:53.392 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-sklearn. 2023-10-11T09:36:53.410 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../078-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2023-10-11T09:36:53.450 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-10-11T09:36:53.833 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-decorator. 2023-10-11T09:36:53.851 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../073-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2023-10-11T09:36:53.890 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2023-10-11T09:36:54.252 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2023-10-11T09:36:54.271 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../074-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2023-10-11T09:36:54.310 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-10-11T09:36:54.475 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-10-11T09:36:54.494 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../079-ceph-mgr-diskprediction-local_18.2.0-662-g9a774796-1focal_all.deb ... 2023-10-11T09:36:54.532 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:54.681 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-scipy. 2023-10-11T09:36:54.700 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../075-python3-scipy_1.3.3-3build1_amd64.deb ... 2023-10-11T09:36:54.743 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2023-10-11T09:36:55.761 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-cachetools. 2023-10-11T09:36:55.778 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../080-python3-cachetools_4.0.0-1_all.deb ... 2023-10-11T09:36:55.844 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2023-10-11T09:36:56.152 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-sklearn-lib. 2023-10-11T09:36:56.171 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../076-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2023-10-11T09:36:56.210 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-10-11T09:36:56.301 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-pyasn1. 2023-10-11T09:36:56.321 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../081-python3-pyasn1_0.4.2-3build1_all.deb ... 2023-10-11T09:36:56.361 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2023-10-11T09:36:56.772 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-joblib. 2023-10-11T09:36:56.773 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-10-11T09:36:56.792 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../077-python3-joblib_0.14.0-3_all.deb ... 2023-10-11T09:36:56.793 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../082-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2023-10-11T09:36:56.830 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-joblib (0.14.0-3) ... 2023-10-11T09:36:56.831 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-10-11T09:36:57.242 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-rsa. 2023-10-11T09:36:57.259 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-sklearn. 2023-10-11T09:36:57.261 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../083-python3-rsa_4.0-3_all.deb ... 2023-10-11T09:36:57.278 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../078-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2023-10-11T09:36:57.299 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-rsa (4.0-3) ... 2023-10-11T09:36:57.316 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-10-11T09:36:57.661 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-google-auth. 2023-10-11T09:36:57.681 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../084-python3-google-auth_1.5.1-2_all.deb ... 2023-10-11T09:36:57.761 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2023-10-11T09:36:58.105 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-10-11T09:36:58.123 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../079-ceph-mgr-diskprediction-local_18.2.0-662-g9a774796-1focal_all.deb ... 2023-10-11T09:36:58.139 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-blinker. 2023-10-11T09:36:58.158 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../085-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2023-10-11T09:36:58.162 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:36:58.197 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-10-11T09:36:58.559 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-oauthlib. 2023-10-11T09:36:58.578 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../086-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2023-10-11T09:36:58.616 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2023-10-11T09:36:58.979 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-10-11T09:36:58.998 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../087-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2023-10-11T09:36:59.037 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-10-11T09:36:59.407 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-cachetools. 2023-10-11T09:36:59.415 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-websocket. 2023-10-11T09:36:59.427 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../080-python3-cachetools_4.0.0-1_all.deb ... 2023-10-11T09:36:59.434 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../088-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2023-10-11T09:36:59.465 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2023-10-11T09:36:59.473 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2023-10-11T09:36:59.868 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-kubernetes. 2023-10-11T09:36:59.888 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../089-python3-kubernetes_7.0.0~a1-2_all.deb ... 2023-10-11T09:36:59.917 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2023-10-11T09:36:59.936 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-pyasn1. 2023-10-11T09:36:59.956 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../081-python3-pyasn1_0.4.2-3build1_all.deb ... 2023-10-11T09:36:59.994 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2023-10-11T09:37:00.374 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-10-11T09:37:00.393 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../082-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2023-10-11T09:37:00.431 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-10-11T09:37:00.590 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-10-11T09:37:00.610 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../090-ceph-mgr-k8sevents_18.2.0-662-g9a774796-1focal_all.deb ... 2023-10-11T09:37:00.647 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-mgr-k8sevents (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:00.810 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-rsa. 2023-10-11T09:37:00.828 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../083-python3-rsa_4.0-3_all.deb ... 2023-10-11T09:37:00.868 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-rsa (4.0-3) ... 2023-10-11T09:37:01.017 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libonig5:amd64. 2023-10-11T09:37:01.037 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../091-libonig5_6.9.4-1_amd64.deb ... 2023-10-11T09:37:01.075 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2023-10-11T09:37:01.222 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-google-auth. 2023-10-11T09:37:01.242 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../084-python3-google-auth_1.5.1-2_all.deb ... 2023-10-11T09:37:01.297 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2023-10-11T09:37:01.487 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libjq1:amd64. 2023-10-11T09:37:01.505 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../092-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-10-11T09:37:01.577 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-10-11T09:37:01.684 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-blinker. 2023-10-11T09:37:01.703 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../085-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2023-10-11T09:37:01.742 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-10-11T09:37:01.889 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package jq. 2023-10-11T09:37:01.908 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../093-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-10-11T09:37:01.980 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2023-10-11T09:37:02.103 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-oauthlib. 2023-10-11T09:37:02.122 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../086-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2023-10-11T09:37:02.163 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2023-10-11T09:37:02.267 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package socat. 2023-10-11T09:37:02.285 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../094-socat_1.7.3.3-2_amd64.deb ... 2023-10-11T09:37:02.358 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking socat (1.7.3.3-2) ... 2023-10-11T09:37:02.592 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-10-11T09:37:02.611 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../087-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2023-10-11T09:37:02.650 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-10-11T09:37:02.728 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package xmlstarlet. 2023-10-11T09:37:02.746 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../095-xmlstarlet_1.6.1-2build1_amd64.deb ... 2023-10-11T09:37:02.785 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2023-10-11T09:37:03.036 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-websocket. 2023-10-11T09:37:03.055 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../088-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2023-10-11T09:37:03.094 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2023-10-11T09:37:03.224 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-test. 2023-10-11T09:37:03.242 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../096-ceph-test_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:37:03.281 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-test (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:03.481 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-kubernetes. 2023-10-11T09:37:03.494 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../089-python3-kubernetes_7.0.0~a1-2_all.deb ... 2023-10-11T09:37:03.547 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2023-10-11T09:37:04.211 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-10-11T09:37:04.230 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../090-ceph-mgr-k8sevents_18.2.0-662-g9a774796-1focal_all.deb ... 2023-10-11T09:37:04.269 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking ceph-mgr-k8sevents (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:04.655 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libonig5:amd64. 2023-10-11T09:37:04.674 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../091-libonig5_6.9.4-1_amd64.deb ... 2023-10-11T09:37:04.712 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2023-10-11T09:37:05.125 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libjq1:amd64. 2023-10-11T09:37:05.143 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../092-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-10-11T09:37:05.182 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-10-11T09:37:05.478 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package jq. 2023-10-11T09:37:05.496 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../093-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-10-11T09:37:05.535 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2023-10-11T09:37:05.847 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package socat. 2023-10-11T09:37:05.865 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../094-socat_1.7.3.3-2_amd64.deb ... 2023-10-11T09:37:05.906 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking socat (1.7.3.3-2) ... 2023-10-11T09:37:06.285 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package xmlstarlet. 2023-10-11T09:37:06.304 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../095-xmlstarlet_1.6.1-2build1_amd64.deb ... 2023-10-11T09:37:06.342 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2023-10-11T09:37:06.755 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package ceph-test. 2023-10-11T09:37:06.774 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../096-ceph-test_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:37:06.813 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking ceph-test (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:07.461 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-volume. 2023-10-11T09:37:07.480 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../097-ceph-volume_18.2.0-662-g9a774796-1focal_all.deb ... 2023-10-11T09:37:07.544 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-volume (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:08.024 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package cephfs-mirror. 2023-10-11T09:37:08.044 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../098-cephfs-mirror_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:37:08.082 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking cephfs-mirror (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:08.460 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-colorama. 2023-10-11T09:37:08.478 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../099-python3-colorama_0.4.3-1build1_all.deb ... 2023-10-11T09:37:08.518 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2023-10-11T09:37:08.863 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package cephfs-shell. 2023-10-11T09:37:08.882 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../100-cephfs-shell_18.2.0-662-g9a774796-1focal_all.deb ... 2023-10-11T09:37:08.920 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking cephfs-shell (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:09.273 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package cephfs-top. 2023-10-11T09:37:09.287 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../101-cephfs-top_18.2.0-662-g9a774796-1focal_all.deb ... 2023-10-11T09:37:09.331 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking cephfs-top (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:09.659 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package fonts-mathjax. 2023-10-11T09:37:09.678 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../102-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2023-10-11T09:37:09.716 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2023-10-11T09:37:10.304 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libcephfs-dev. 2023-10-11T09:37:10.323 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../103-libcephfs-dev_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:37:10.361 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libcephfs-dev (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:10.824 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package nvme-cli. 2023-10-11T09:37:10.842 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../104-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2023-10-11T09:37:10.920 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2023-10-11T09:37:11.034 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package ceph-volume. 2023-10-11T09:37:11.053 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../097-ceph-volume_18.2.0-662-g9a774796-1focal_all.deb ... 2023-10-11T09:37:11.091 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking ceph-volume (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:11.286 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-10-11T09:37:11.305 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../105-python-asyncssh-doc_1.12.2-1_all.deb ... 2023-10-11T09:37:11.377 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python-asyncssh-doc (1.12.2-1) ... 2023-10-11T09:37:11.538 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package cephfs-mirror. 2023-10-11T09:37:11.558 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../098-cephfs-mirror_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:37:11.596 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking cephfs-mirror (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:11.780 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libjs-mathjax. 2023-10-11T09:37:11.798 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../106-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2023-10-11T09:37:11.838 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2023-10-11T09:37:11.983 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-colorama. 2023-10-11T09:37:12.002 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../099-python3-colorama_0.4.3-1build1_all.deb ... 2023-10-11T09:37:12.040 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2023-10-11T09:37:12.377 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package cephfs-shell. 2023-10-11T09:37:12.396 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../100-cephfs-shell_18.2.0-662-g9a774796-1focal_all.deb ... 2023-10-11T09:37:12.435 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking cephfs-shell (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:12.721 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package cephfs-top. 2023-10-11T09:37:12.740 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../101-cephfs-top_18.2.0-662-g9a774796-1focal_all.deb ... 2023-10-11T09:37:12.800 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking cephfs-top (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:13.082 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package fonts-mathjax. 2023-10-11T09:37:13.101 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../102-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2023-10-11T09:37:13.139 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2023-10-11T09:37:13.677 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libcephfs-dev. 2023-10-11T09:37:13.696 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../103-libcephfs-dev_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:37:13.735 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libcephfs-dev (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:13.918 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python-natsort-doc. 2023-10-11T09:37:13.940 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../107-python-natsort-doc_7.0.1-1_all.deb ... 2023-10-11T09:37:13.984 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2023-10-11T09:37:14.196 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package nvme-cli. 2023-10-11T09:37:14.209 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../104-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2023-10-11T09:37:14.278 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2023-10-11T09:37:14.371 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-webencodings. 2023-10-11T09:37:14.391 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../108-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2023-10-11T09:37:14.493 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2023-10-11T09:37:14.659 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-10-11T09:37:14.672 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../105-python-asyncssh-doc_1.12.2-1_all.deb ... 2023-10-11T09:37:14.717 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python-asyncssh-doc (1.12.2-1) ... 2023-10-11T09:37:14.857 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-html5lib. 2023-10-11T09:37:14.876 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../109-python3-html5lib_1.0.1-2_all.deb ... 2023-10-11T09:37:14.914 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2023-10-11T09:37:15.179 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libjs-mathjax. 2023-10-11T09:37:15.199 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../106-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2023-10-11T09:37:15.237 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2023-10-11T09:37:15.335 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-lxml:amd64. 2023-10-11T09:37:15.354 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../110-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2023-10-11T09:37:15.384 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-10-11T09:37:15.913 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-pastescript. 2023-10-11T09:37:15.933 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../111-python3-pastescript_2.0.2-4_all.deb ... 2023-10-11T09:37:16.015 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-10-11T09:37:16.407 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-psutil. 2023-10-11T09:37:16.427 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../112-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2023-10-11T09:37:16.464 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2023-10-11T09:37:16.910 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-pyinotify. 2023-10-11T09:37:16.930 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../113-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2023-10-11T09:37:16.976 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-10-11T09:37:17.327 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python-natsort-doc. 2023-10-11T09:37:17.329 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-simplejson. 2023-10-11T09:37:17.348 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../107-python-natsort-doc_7.0.1-1_all.deb ... 2023-10-11T09:37:17.348 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../114-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2023-10-11T09:37:17.385 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2023-10-11T09:37:17.387 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2023-10-11T09:37:17.751 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-10-11T09:37:17.771 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2023-10-11T09:37:17.771 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-webencodings. 2023-10-11T09:37:17.790 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../108-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2023-10-11T09:37:17.808 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-10-11T09:37:17.837 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2023-10-11T09:37:18.182 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-html5lib. 2023-10-11T09:37:18.202 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../109-python3-html5lib_1.0.1-2_all.deb ... 2023-10-11T09:37:18.239 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2023-10-11T09:37:18.588 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package radosgw. 2023-10-11T09:37:18.607 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../116-radosgw_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:37:18.627 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-lxml:amd64. 2023-10-11T09:37:18.645 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking radosgw (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:18.648 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../110-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2023-10-11T09:37:18.693 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-10-11T09:37:19.665 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-pastescript. 2023-10-11T09:37:19.679 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../111-python3-pastescript_2.0.2-4_all.deb ... 2023-10-11T09:37:19.723 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-10-11T09:37:20.086 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-psutil. 2023-10-11T09:37:20.106 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../112-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2023-10-11T09:37:20.118 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package rbd-fuse. 2023-10-11T09:37:20.138 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../117-rbd-fuse_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:37:20.161 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2023-10-11T09:37:20.176 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking rbd-fuse (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:20.539 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-pyinotify. 2023-10-11T09:37:20.543 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-attr (19.3.0-2) ... 2023-10-11T09:37:20.559 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../113-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2023-10-11T09:37:20.597 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-10-11T09:37:20.939 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2023-10-11T09:37:20.986 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-simplejson. 2023-10-11T09:37:21.000 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../114-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2023-10-11T09:37:21.044 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2023-10-11T09:37:21.098 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-10-11T09:37:21.224 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2023-10-11T09:37:21.380 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-10-11T09:37:21.394 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2023-10-11T09:37:21.438 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-10-11T09:37:21.461 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up cephadm (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:22.212 INFO:teuthology.orchestra.run.smithi133.stdout:Adding system user cephadm....done 2023-10-11T09:37:22.226 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package radosgw. 2023-10-11T09:37:22.255 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../116-radosgw_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:37:22.300 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking radosgw (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:22.409 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2023-10-11T09:37:22.560 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2023-10-11T09:37:22.995 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-10-11T09:37:23.354 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2023-10-11T09:37:23.480 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up liboath0 (2.6.1-1.3) ... 2023-10-11T09:37:23.606 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2023-10-11T09:37:23.697 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package rbd-fuse. 2023-10-11T09:37:23.712 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../117-rbd-fuse_18.2.0-662-g9a774796-1focal_amd64.deb ... 2023-10-11T09:37:23.755 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking rbd-fuse (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:24.033 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-joblib (0.14.0-3) ... 2023-10-11T09:37:24.106 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-attr (19.3.0-2) ... 2023-10-11T09:37:24.469 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2023-10-11T09:37:24.552 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2023-10-11T09:37:24.620 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-10-11T09:37:24.746 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2023-10-11T09:37:24.955 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up cephadm (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:25.078 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-cachetools (4.0.0-1) ... 2023-10-11T09:37:25.437 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2023-10-11T09:37:25.713 INFO:teuthology.orchestra.run.smithi078.stdout:Adding system user cephadm....done 2023-10-11T09:37:25.780 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2023-10-11T09:37:25.903 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2023-10-11T09:37:26.029 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2023-10-11T09:37:26.123 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-10-11T09:37:26.456 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-10-11T09:37:26.473 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-ceph-argparse (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:26.824 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2023-10-11T09:37:26.833 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-10-11T09:37:26.950 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up liboath0 (2.6.1-1.3) ... 2023-10-11T09:37:26.967 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2023-10-11T09:37:27.076 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2023-10-11T09:37:27.310 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2023-10-11T09:37:27.469 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-joblib (0.14.0-3) ... 2023-10-11T09:37:27.661 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2023-10-11T09:37:27.946 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2023-10-11T09:37:28.012 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-10-11T09:37:28.138 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2023-10-11T09:37:28.289 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-cachetools (4.0.0-1) ... 2023-10-11T09:37:28.572 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-natsort (7.0.1-1) ... 2023-10-11T09:37:28.640 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2023-10-11T09:37:28.915 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2023-10-11T09:37:28.992 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2023-10-11T09:37:29.249 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2023-10-11T09:37:29.335 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-10-11T09:37:29.651 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2023-10-11T09:37:29.711 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-ceph-argparse (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:29.993 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2023-10-11T09:37:30.095 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-10-11T09:37:30.229 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2023-10-11T09:37:30.403 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-tempita (0.5.2-6) ... 2023-10-11T09:37:30.597 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2023-10-11T09:37:30.762 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-packaging (20.3-1) ... 2023-10-11T09:37:30.948 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2023-10-11T09:37:31.113 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2023-10-11T09:37:31.239 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-10-11T09:37:31.299 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-10-11T09:37:31.398 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2023-10-11T09:37:31.433 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2023-10-11T09:37:31.774 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2023-10-11T09:37:31.885 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-natsort (7.0.1-1) ... 2023-10-11T09:37:32.251 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2023-10-11T09:37:32.261 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2023-10-11T09:37:32.595 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2023-10-11T09:37:32.619 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-10-11T09:37:33.005 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2023-10-11T09:37:33.062 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2023-10-11T09:37:33.348 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2023-10-11T09:37:33.497 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up socat (1.7.3.3-2) ... 2023-10-11T09:37:33.622 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-ceph-common (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:33.749 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-tempita (0.5.2-6) ... 2023-10-11T09:37:34.099 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-11T09:37:34.142 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-packaging (20.3-1) ... 2023-10-11T09:37:34.241 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-html5lib (1.0.1-2) ... 2023-10-11T09:37:34.568 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2023-10-11T09:37:34.668 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-10-11T09:37:34.694 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-10-11T09:37:34.794 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2023-10-11T09:37:34.820 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2023-10-11T09:37:34.961 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-pluggy (0.13.0-2) ... 2023-10-11T09:37:35.171 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2023-10-11T09:37:35.304 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-10-11T09:37:35.647 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2023-10-11T09:37:35.680 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-11T09:37:35.806 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-10-11T09:37:35.932 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2023-10-11T09:37:36.050 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-10-11T09:37:36.401 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2023-10-11T09:37:36.491 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2023-10-11T09:37:36.777 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-logutils (0.3.3-7) ... 2023-10-11T09:37:36.969 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up socat (1.7.3.3-2) ... 2023-10-11T09:37:37.132 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-ceph-common (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:37.169 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2023-10-11T09:37:37.295 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-10-11T09:37:37.520 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-11T09:37:37.638 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-prettytable (0.7.2-5) ... 2023-10-11T09:37:37.688 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-html5lib (1.0.1-2) ... 2023-10-11T09:37:37.997 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2023-10-11T09:37:38.098 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-10-11T09:37:38.249 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2023-10-11T09:37:38.365 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2023-10-11T09:37:38.377 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-pluggy (0.13.0-2) ... 2023-10-11T09:37:38.738 INFO:teuthology.orchestra.run.smithi133.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2023-10-11T09:37:38.750 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-10-11T09:37:39.093 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2023-10-11T09:37:39.129 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-11T09:37:39.219 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-10-11T09:37:39.255 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-10-11T09:37:39.414 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2023-10-11T09:37:39.620 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2023-10-11T09:37:39.807 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2023-10-11T09:37:40.046 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-10-11T09:37:40.172 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-10-11T09:37:40.175 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-logutils (0.3.3-7) ... 2023-10-11T09:37:40.326 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-webob (1:1.8.5-2) ... 2023-10-11T09:37:40.543 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2023-10-11T09:37:40.669 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-10-11T09:37:40.719 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-10-11T09:37:41.046 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-prettytable (0.7.2-5) ... 2023-10-11T09:37:41.070 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2023-10-11T09:37:41.399 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2023-10-11T09:37:41.792 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2023-10-11T09:37:42.183 INFO:teuthology.orchestra.run.smithi078.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2023-10-11T09:37:42.511 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2023-10-11T09:37:42.670 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-10-11T09:37:42.927 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-10-11T09:37:43.053 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-pytest (4.6.9-1) ... 2023-10-11T09:37:43.054 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2023-10-11T09:37:43.513 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2023-10-11T09:37:43.514 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-10-11T09:37:43.665 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-10-11T09:37:43.791 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-webob (1:1.8.5-2) ... 2023-10-11T09:37:43.889 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2023-10-11T09:37:44.217 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-10-11T09:37:44.240 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-11T09:37:44.368 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-bs4 (4.8.2-1) ... 2023-10-11T09:37:44.552 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2023-10-11T09:37:44.822 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2023-10-11T09:37:45.246 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-asyncssh (1.12.2-1) ... 2023-10-11T09:37:45.722 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-10-11T09:37:46.132 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-10-11T09:37:46.401 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-10-11T09:37:46.527 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-pytest (4.6.9-1) ... 2023-10-11T09:37:46.987 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2023-10-11T09:37:47.338 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2023-10-11T09:37:47.380 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-10-11T09:37:47.539 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2023-10-11T09:37:47.665 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2023-10-11T09:37:47.680 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-11T09:37:47.799 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-rsa (4.0-3) ... 2023-10-11T09:37:47.806 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-bs4 (4.8.2-1) ... 2023-10-11T09:37:48.184 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2023-10-11T09:37:48.224 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2023-10-11T09:37:48.351 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-webtest (2.0.32-2) ... 2023-10-11T09:37:48.619 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-asyncssh (1.12.2-1) ... 2023-10-11T09:37:48.719 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-10-11T09:37:49.079 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-10-11T09:37:49.087 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-10-11T09:37:49.480 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2023-10-11T09:37:49.497 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-10-11T09:37:49.923 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up librados2 (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:50.152 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python-asyncssh-doc (1.12.2-1) ... 2023-10-11T09:37:50.311 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up librgw2 (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:50.437 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libsqlite3-mod-ceph (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:50.587 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libcephfs2 (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:50.714 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-routes (2.4.1-2) ... 2023-10-11T09:37:50.718 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-10-11T09:37:50.861 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2023-10-11T09:37:50.987 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2023-10-11T09:37:51.090 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libradosstriper1 (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:51.113 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-rsa (4.0-3) ... 2023-10-11T09:37:51.216 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-google-auth (1.5.1-2) ... 2023-10-11T09:37:51.472 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2023-10-11T09:37:51.576 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up librbd1 (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:51.598 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-webtest (2.0.32-2) ... 2023-10-11T09:37:51.702 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-mgr-modules-core (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:51.853 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-fuse (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:51.967 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-10-11T09:37:52.111 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-11T09:37:52.111 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-11T09:37:52.317 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-10-11T09:37:52.760 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2023-10-11T09:37:52.996 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libcephfs-dev (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:53.122 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-rados (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:53.195 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up librados2 (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:53.280 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up cephfs-top (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:53.415 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python-asyncssh-doc (1.12.2-1) ... 2023-10-11T09:37:53.424 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2023-10-11T09:37:53.574 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up librgw2 (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:53.701 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libsqlite3-mod-ceph (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:53.826 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libcephfs2 (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:53.952 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-routes (2.4.1-2) ... 2023-10-11T09:37:54.362 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libradosstriper1 (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:54.488 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-google-auth (1.5.1-2) ... 2023-10-11T09:37:54.553 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-rbd (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:54.679 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up rbd-fuse (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:54.838 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-rgw (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:54.889 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up librbd1 (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:54.964 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-cephfs (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:55.022 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up ceph-mgr-modules-core (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:55.090 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up cephfs-shell (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:55.175 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up ceph-fuse (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:55.249 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-common (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:55.429 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-11T09:37:55.429 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-11T09:37:56.209 INFO:teuthology.orchestra.run.smithi133.stdout:Adding group ceph....done 2023-10-11T09:37:56.277 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libcephfs-dev (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:56.403 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-rados (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:56.529 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up cephfs-top (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:56.654 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2023-10-11T09:37:56.705 INFO:teuthology.orchestra.run.smithi133.stdout:Adding system user ceph....done 2023-10-11T09:37:56.961 INFO:teuthology.orchestra.run.smithi133.stdout:Setting system user ceph properties....done 2023-10-11T09:37:56.996 INFO:teuthology.orchestra.run.smithi133.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-10-11T09:37:57.197 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-10-11T09:37:57.737 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-10-11T09:37:57.858 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-rbd (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:57.984 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up rbd-fuse (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:58.119 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-rgw (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:58.244 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-cephfs (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:58.420 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up cephfs-shell (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:58.546 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up ceph-common (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:58.635 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up cephfs-mirror (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:58.893 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-11T09:37:58.894 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-11T09:37:59.169 INFO:teuthology.orchestra.run.smithi078.stdout:Adding group ceph....done 2023-10-11T09:37:59.651 INFO:teuthology.orchestra.run.smithi078.stdout:Adding system user ceph....done 2023-10-11T09:37:59.745 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-test (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:59.870 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up radosgw (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:37:59.871 INFO:teuthology.orchestra.run.smithi078.stdout:Setting system user ceph properties....done 2023-10-11T09:37:59.906 INFO:teuthology.orchestra.run.smithi078.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-10-11T09:38:00.100 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-10-11T09:38:00.220 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-11T09:38:00.221 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-11T09:38:00.633 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-10-11T09:38:01.514 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-base (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:38:01.556 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up cephfs-mirror (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:38:01.855 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-11T09:38:01.855 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-11T09:38:02.004 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-10-11T09:38:02.666 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up ceph-test (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:38:02.817 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up radosgw (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:38:02.916 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-mds (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:38:03.121 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-11T09:38:03.122 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-11T09:38:03.206 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-11T09:38:03.206 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-11T09:38:04.076 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-mgr (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:38:04.335 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up ceph-base (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:38:04.341 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-11T09:38:04.341 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-11T09:38:04.864 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-10-11T09:38:05.168 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-osd (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:38:05.770 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up ceph-mds (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:38:05.984 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-11T09:38:05.984 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-11T09:38:06.065 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-11T09:38:06.065 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-11T09:38:06.837 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-mgr-k8sevents (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:38:06.915 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up ceph-mgr (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:38:06.996 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-mgr-diskprediction-local (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:38:07.147 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-mon (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:38:07.185 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-11T09:38:07.185 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-11T09:38:07.405 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-11T09:38:07.406 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-11T09:38:08.007 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up ceph-osd (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:38:08.249 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-mgr-cephadm (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:38:08.433 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:38:08.559 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-mgr-dashboard (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:38:08.710 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-volume (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:38:08.755 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-11T09:38:08.756 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-11T09:38:08.870 INFO:teuthology.orchestra.run.smithi133.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-10-11T09:38:09.038 INFO:teuthology.orchestra.run.smithi133.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-10-11T09:38:09.523 INFO:teuthology.orchestra.run.smithi133.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-10-11T09:38:09.592 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up ceph-mgr-k8sevents (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:38:09.726 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up ceph-mgr-diskprediction-local (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:38:09.878 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up ceph-mon (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:38:10.154 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-11T09:38:10.155 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-11T09:38:10.979 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up ceph-mgr-cephadm (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:38:11.164 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up ceph (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:38:11.298 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up ceph-mgr-dashboard (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:38:11.449 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up ceph-volume (18.2.0-662-g9a774796-1focal) ... 2023-10-11T09:38:11.618 INFO:teuthology.orchestra.run.smithi078.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-10-11T09:38:11.778 INFO:teuthology.orchestra.run.smithi078.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-10-11T09:38:12.237 INFO:teuthology.orchestra.run.smithi078.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-10-11T09:38:14.625 INFO:teuthology.orchestra.run.smithi133.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2023-10-11T09:38:15.423 INFO:teuthology.orchestra.run.smithi133.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-11T09:38:15.428 DEBUG:teuthology.orchestra.run.smithi133:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-10-11T09:38:15.548 INFO:teuthology.orchestra.run.smithi133.stdout:Reading package lists... 2023-10-11T09:38:15.749 INFO:teuthology.orchestra.run.smithi133.stdout:Building dependency tree... 2023-10-11T09:38:15.749 INFO:teuthology.orchestra.run.smithi133.stdout:Reading state information... 2023-10-11T09:38:15.948 INFO:teuthology.orchestra.run.smithi133.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2023-10-11T09:38:15.948 INFO:teuthology.orchestra.run.smithi133.stdout:flex is already the newest version (2.6.4-6.2). 2023-10-11T09:38:15.948 INFO:teuthology.orchestra.run.smithi133.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2023-10-11T09:38:15.949 INFO:teuthology.orchestra.run.smithi133.stdout:libtool-bin is already the newest version (2.4.6-14). 2023-10-11T09:38:15.949 INFO:teuthology.orchestra.run.smithi133.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2023-10-11T09:38:15.949 INFO:teuthology.orchestra.run.smithi133.stdout:The following packages were automatically installed and are no longer required: 2023-10-11T09:38:15.950 INFO:teuthology.orchestra.run.smithi133.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-10-11T09:38:15.950 INFO:teuthology.orchestra.run.smithi133.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-11T09:38:15.951 INFO:teuthology.orchestra.run.smithi133.stdout:The following additional packages will be installed: 2023-10-11T09:38:15.951 INFO:teuthology.orchestra.run.smithi133.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2023-10-11T09:38:15.951 INFO:teuthology.orchestra.run.smithi133.stdout: libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl1.1 2023-10-11T09:38:15.951 INFO:teuthology.orchestra.run.smithi133.stdout: libteamdctl0 libuuid1 modemmanager network-manager-pptp postgresql-12 2023-10-11T09:38:15.952 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-client-12 postgresql-client-common ppp pptp-linux usb-modeswitch 2023-10-11T09:38:15.952 INFO:teuthology.orchestra.run.smithi133.stdout: usb-modeswitch-data wpasupplicant 2023-10-11T09:38:15.952 INFO:teuthology.orchestra.run.smithi133.stdout:Suggested packages: 2023-10-11T09:38:15.953 INFO:teuthology.orchestra.run.smithi133.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils postgresql-doc 2023-10-11T09:38:15.953 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-doc-12 libjson-perl comgt wvdial util-linux-locales wpagui 2023-10-11T09:38:15.953 INFO:teuthology.orchestra.run.smithi133.stdout: libengine-pkcs11-openssl 2023-10-11T09:38:16.052 INFO:teuthology.orchestra.run.smithi133.stdout:The following NEW packages will be installed: 2023-10-11T09:38:16.053 INFO:teuthology.orchestra.run.smithi133.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2023-10-11T09:38:16.053 INFO:teuthology.orchestra.run.smithi133.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy 2023-10-11T09:38:16.054 INFO:teuthology.orchestra.run.smithi133.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2023-10-11T09:38:16.054 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql postgresql-12 postgresql-client postgresql-client-12 2023-10-11T09:38:16.054 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-client-common postgresql-common postgresql-contrib ppp pptp-linux 2023-10-11T09:38:16.055 INFO:teuthology.orchestra.run.smithi133.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-11T09:38:16.055 INFO:teuthology.orchestra.run.smithi133.stdout:The following packages will be upgraded: 2023-10-11T09:38:16.056 INFO:teuthology.orchestra.run.smithi133.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2023-10-11T09:38:16.156 INFO:teuthology.orchestra.run.smithi133.stdout:6 upgraded, 31 newly installed, 0 to remove and 250 not upgraded. 2023-10-11T09:38:16.158 INFO:teuthology.orchestra.run.smithi133.stdout:Need to get 25.3 MB of archives. 2023-10-11T09:38:16.159 INFO:teuthology.orchestra.run.smithi133.stdout:After this operation, 79.9 MB of additional disk space will be used. 2023-10-11T09:38:16.159 INFO:teuthology.orchestra.run.smithi133.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.4 [1,021 kB] 2023-10-11T09:38:16.346 INFO:teuthology.orchestra.run.smithi133.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.4 [33.6 kB] 2023-10-11T09:38:16.348 INFO:teuthology.orchestra.run.smithi133.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.4 [20.0 kB] 2023-10-11T09:38:16.349 INFO:teuthology.orchestra.run.smithi133.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2023-10-11T09:38:16.357 INFO:teuthology.orchestra.run.smithi133.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2023-10-11T09:38:16.358 INFO:teuthology.orchestra.run.smithi133.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.19 [1,321 kB] 2023-10-11T09:38:16.403 INFO:teuthology.orchestra.run.smithi133.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2023-10-11T09:38:16.404 INFO:teuthology.orchestra.run.smithi133.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2023-10-11T09:38:16.408 INFO:teuthology.orchestra.run.smithi133.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2023-10-11T09:38:16.412 INFO:teuthology.orchestra.run.smithi133.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2023-10-11T09:38:16.412 INFO:teuthology.orchestra.run.smithi133.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.6 [60.6 kB] 2023-10-11T09:38:16.421 INFO:teuthology.orchestra.run.smithi133.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2023-10-11T09:38:16.422 INFO:teuthology.orchestra.run.smithi133.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2023-10-11T09:38:16.424 INFO:teuthology.orchestra.run.smithi133.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2023-10-11T09:38:16.425 INFO:teuthology.orchestra.run.smithi133.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2023-10-11T09:38:16.456 INFO:teuthology.orchestra.run.smithi133.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 libndp0 amd64 1.7-0ubuntu1 [10.9 kB] 2023-10-11T09:38:16.457 INFO:teuthology.orchestra.run.smithi133.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.3 [370 kB] 2023-10-11T09:38:16.465 INFO:teuthology.orchestra.run.smithi133.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpq5 amd64 12.16-0ubuntu0.20.04.1 [115 kB] 2023-10-11T09:38:16.468 INFO:teuthology.orchestra.run.smithi133.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2023-10-11T09:38:16.477 INFO:teuthology.orchestra.run.smithi133.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2023-10-11T09:38:16.478 INFO:teuthology.orchestra.run.smithi133.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.19 [1,584 kB] 2023-10-11T09:38:16.519 INFO:teuthology.orchestra.run.smithi133.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2023-10-11T09:38:16.519 INFO:teuthology.orchestra.run.smithi133.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2023-10-11T09:38:16.534 INFO:teuthology.orchestra.run.smithi133.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.3 [1,183 kB] 2023-10-11T09:38:16.568 INFO:teuthology.orchestra.run.smithi133.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.3 [1,855 kB] 2023-10-11T09:38:16.588 INFO:teuthology.orchestra.run.smithi133.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2023-10-11T09:38:16.598 INFO:teuthology.orchestra.run.smithi133.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2023-10-11T09:38:16.598 INFO:teuthology.orchestra.run.smithi133.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2023-10-11T09:38:16.599 INFO:teuthology.orchestra.run.smithi133.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-common all 214ubuntu0.1 [28.2 kB] 2023-10-11T09:38:16.600 INFO:teuthology.orchestra.run.smithi133.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-12 amd64 12.16-0ubuntu0.20.04.1 [1,053 kB] 2023-10-11T09:38:16.613 INFO:teuthology.orchestra.run.smithi133.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-common all 214ubuntu0.1 [169 kB] 2023-10-11T09:38:16.615 INFO:teuthology.orchestra.run.smithi133.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-12 amd64 12.16-0ubuntu0.20.04.1 [13.5 MB] 2023-10-11T09:38:17.060 INFO:teuthology.orchestra.run.smithi133.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql all 12+214ubuntu0.1 [3,924 B] 2023-10-11T09:38:17.060 INFO:teuthology.orchestra.run.smithi133.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client all 12+214ubuntu0.1 [3,940 B] 2023-10-11T09:38:17.061 INFO:teuthology.orchestra.run.smithi133.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-contrib all 12+214ubuntu0.1 [3,932 B] 2023-10-11T09:38:17.061 INFO:teuthology.orchestra.run.smithi133.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2023-10-11T09:38:17.061 INFO:teuthology.orchestra.run.smithi133.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2023-10-11T09:38:17.298 INFO:teuthology.orchestra.run.smithi078.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2023-10-11T09:38:17.608 INFO:teuthology.orchestra.run.smithi133.stdout:Preconfiguring packages ... 2023-10-11T09:38:17.850 INFO:teuthology.orchestra.run.smithi133.stdout:Fetched 25.3 MB in 1s (25.4 MB/s) 2023-10-11T09:38:18.113 INFO:teuthology.orchestra.run.smithi078.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-11T09:38:18.119 DEBUG:teuthology.orchestra.run.smithi078:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-10-11T09:38:18.206 INFO:teuthology.orchestra.run.smithi133.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 118874 files and directories currently installed.) 2023-10-11T09:38:18.210 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-11T09:38:18.245 INFO:teuthology.orchestra.run.smithi078.stdout:Reading package lists... 2023-10-11T09:38:18.391 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking util-linux (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-11T09:38:18.448 INFO:teuthology.orchestra.run.smithi078.stdout:Building dependency tree... 2023-10-11T09:38:18.449 INFO:teuthology.orchestra.run.smithi078.stdout:Reading state information... 2023-10-11T09:38:18.643 INFO:teuthology.orchestra.run.smithi078.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2023-10-11T09:38:18.643 INFO:teuthology.orchestra.run.smithi078.stdout:flex is already the newest version (2.6.4-6.2). 2023-10-11T09:38:18.643 INFO:teuthology.orchestra.run.smithi078.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2023-10-11T09:38:18.643 INFO:teuthology.orchestra.run.smithi078.stdout:libtool-bin is already the newest version (2.4.6-14). 2023-10-11T09:38:18.644 INFO:teuthology.orchestra.run.smithi078.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2023-10-11T09:38:18.644 INFO:teuthology.orchestra.run.smithi078.stdout:The following packages were automatically installed and are no longer required: 2023-10-11T09:38:18.644 INFO:teuthology.orchestra.run.smithi078.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-10-11T09:38:18.644 INFO:teuthology.orchestra.run.smithi078.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-11T09:38:18.644 INFO:teuthology.orchestra.run.smithi078.stdout:The following additional packages will be installed: 2023-10-11T09:38:18.645 INFO:teuthology.orchestra.run.smithi078.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2023-10-11T09:38:18.645 INFO:teuthology.orchestra.run.smithi078.stdout: libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl1.1 2023-10-11T09:38:18.645 INFO:teuthology.orchestra.run.smithi078.stdout: libteamdctl0 libuuid1 modemmanager network-manager-pptp postgresql-12 2023-10-11T09:38:18.645 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-client-12 postgresql-client-common ppp pptp-linux usb-modeswitch 2023-10-11T09:38:18.645 INFO:teuthology.orchestra.run.smithi078.stdout: usb-modeswitch-data wpasupplicant 2023-10-11T09:38:18.646 INFO:teuthology.orchestra.run.smithi078.stdout:Suggested packages: 2023-10-11T09:38:18.646 INFO:teuthology.orchestra.run.smithi078.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils postgresql-doc 2023-10-11T09:38:18.646 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-doc-12 libjson-perl comgt wvdial util-linux-locales wpagui 2023-10-11T09:38:18.647 INFO:teuthology.orchestra.run.smithi078.stdout: libengine-pkcs11-openssl 2023-10-11T09:38:18.736 INFO:teuthology.orchestra.run.smithi078.stdout:The following NEW packages will be installed: 2023-10-11T09:38:18.737 INFO:teuthology.orchestra.run.smithi078.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2023-10-11T09:38:18.737 INFO:teuthology.orchestra.run.smithi078.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy 2023-10-11T09:38:18.737 INFO:teuthology.orchestra.run.smithi078.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2023-10-11T09:38:18.737 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql postgresql-12 postgresql-client postgresql-client-12 2023-10-11T09:38:18.738 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-client-common postgresql-common postgresql-contrib ppp pptp-linux 2023-10-11T09:38:18.738 INFO:teuthology.orchestra.run.smithi078.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-11T09:38:18.738 INFO:teuthology.orchestra.run.smithi078.stdout:The following packages will be upgraded: 2023-10-11T09:38:18.739 INFO:teuthology.orchestra.run.smithi078.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2023-10-11T09:38:18.812 INFO:teuthology.orchestra.run.smithi078.stdout:6 upgraded, 31 newly installed, 0 to remove and 250 not upgraded. 2023-10-11T09:38:18.812 INFO:teuthology.orchestra.run.smithi078.stdout:Need to get 25.3 MB of archives. 2023-10-11T09:38:18.813 INFO:teuthology.orchestra.run.smithi078.stdout:After this operation, 79.9 MB of additional disk space will be used. 2023-10-11T09:38:18.813 INFO:teuthology.orchestra.run.smithi078.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.4 [1,021 kB] 2023-10-11T09:38:18.973 INFO:teuthology.orchestra.run.smithi078.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.4 [33.6 kB] 2023-10-11T09:38:18.975 INFO:teuthology.orchestra.run.smithi078.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.4 [20.0 kB] 2023-10-11T09:38:18.977 INFO:teuthology.orchestra.run.smithi078.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2023-10-11T09:38:18.983 INFO:teuthology.orchestra.run.smithi078.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2023-10-11T09:38:18.984 INFO:teuthology.orchestra.run.smithi078.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.19 [1,321 kB] 2023-10-11T09:38:19.003 INFO:teuthology.orchestra.run.smithi078.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2023-10-11T09:38:19.003 INFO:teuthology.orchestra.run.smithi078.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2023-10-11T09:38:19.005 INFO:teuthology.orchestra.run.smithi078.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2023-10-11T09:38:19.007 INFO:teuthology.orchestra.run.smithi078.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2023-10-11T09:38:19.007 INFO:teuthology.orchestra.run.smithi078.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.6 [60.6 kB] 2023-10-11T09:38:19.013 INFO:teuthology.orchestra.run.smithi078.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2023-10-11T09:38:19.014 INFO:teuthology.orchestra.run.smithi078.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2023-10-11T09:38:19.016 INFO:teuthology.orchestra.run.smithi078.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2023-10-11T09:38:19.016 INFO:teuthology.orchestra.run.smithi078.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2023-10-11T09:38:19.034 INFO:teuthology.orchestra.run.smithi078.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 libndp0 amd64 1.7-0ubuntu1 [10.9 kB] 2023-10-11T09:38:19.035 INFO:teuthology.orchestra.run.smithi078.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.3 [370 kB] 2023-10-11T09:38:19.043 INFO:teuthology.orchestra.run.smithi078.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpq5 amd64 12.16-0ubuntu0.20.04.1 [115 kB] 2023-10-11T09:38:19.045 INFO:teuthology.orchestra.run.smithi078.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2023-10-11T09:38:19.057 INFO:teuthology.orchestra.run.smithi078.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2023-10-11T09:38:19.057 INFO:teuthology.orchestra.run.smithi078.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.19 [1,584 kB] 2023-10-11T09:38:19.075 INFO:teuthology.orchestra.run.smithi078.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2023-10-11T09:38:19.075 INFO:teuthology.orchestra.run.smithi078.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2023-10-11T09:38:19.085 INFO:teuthology.orchestra.run.smithi078.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.3 [1,183 kB] 2023-10-11T09:38:19.097 INFO:teuthology.orchestra.run.smithi078.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.3 [1,855 kB] 2023-10-11T09:38:19.117 INFO:teuthology.orchestra.run.smithi078.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2023-10-11T09:38:19.120 INFO:teuthology.orchestra.run.smithi078.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2023-10-11T09:38:19.121 INFO:teuthology.orchestra.run.smithi078.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2023-10-11T09:38:19.121 INFO:teuthology.orchestra.run.smithi078.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-common all 214ubuntu0.1 [28.2 kB] 2023-10-11T09:38:19.122 INFO:teuthology.orchestra.run.smithi078.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-12 amd64 12.16-0ubuntu0.20.04.1 [1,053 kB] 2023-10-11T09:38:19.133 INFO:teuthology.orchestra.run.smithi078.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-common all 214ubuntu0.1 [169 kB] 2023-10-11T09:38:19.135 INFO:teuthology.orchestra.run.smithi078.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-12 amd64 12.16-0ubuntu0.20.04.1 [13.5 MB] 2023-10-11T09:38:19.271 INFO:teuthology.orchestra.run.smithi078.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql all 12+214ubuntu0.1 [3,924 B] 2023-10-11T09:38:19.271 INFO:teuthology.orchestra.run.smithi078.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client all 12+214ubuntu0.1 [3,940 B] 2023-10-11T09:38:19.272 INFO:teuthology.orchestra.run.smithi078.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-contrib all 12+214ubuntu0.1 [3,932 B] 2023-10-11T09:38:19.272 INFO:teuthology.orchestra.run.smithi078.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2023-10-11T09:38:19.272 INFO:teuthology.orchestra.run.smithi078.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2023-10-11T09:38:19.800 INFO:teuthology.orchestra.run.smithi078.stdout:Preconfiguring packages ... 2023-10-11T09:38:20.032 INFO:teuthology.orchestra.run.smithi078.stdout:Fetched 25.3 MB in 1s (48.2 MB/s) 2023-10-11T09:38:20.097 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up util-linux (2.34-0.1ubuntu9.4) ... 2023-10-11T09:38:20.354 INFO:teuthology.orchestra.run.smithi078.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 118874 files and directories currently installed.) 2023-10-11T09:38:20.357 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-11T09:38:20.536 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking util-linux (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-11T09:38:22.135 INFO:teuthology.orchestra.run.smithi133.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 118874 files and directories currently installed.) 2023-10-11T09:38:22.139 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-11T09:38:22.303 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-11T09:38:22.354 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up util-linux (2.34-0.1ubuntu9.4) ... 2023-10-11T09:38:22.763 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-11T09:38:22.934 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-11T09:38:23.332 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.4) ... 2023-10-11T09:38:23.906 INFO:teuthology.orchestra.run.smithi133.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 118874 files and directories currently installed.) 2023-10-11T09:38:23.910 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-11T09:38:24.050 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-10-11T09:38:24.330 INFO:teuthology.orchestra.run.smithi078.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 118874 files and directories currently installed.) 2023-10-11T09:38:24.334 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-11T09:38:24.465 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-11T09:38:24.591 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-11T09:38:24.722 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-10-11T09:38:24.942 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-11T09:38:25.112 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-11T09:38:25.264 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-10-11T09:38:25.435 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.19) over (1.1.1f-1ubuntu2.16) ... 2023-10-11T09:38:25.497 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.4) ... 2023-10-11T09:38:26.006 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-10-11T09:38:26.026 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2023-10-11T09:38:26.064 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2023-10-11T09:38:26.377 INFO:teuthology.orchestra.run.smithi078.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 118874 files and directories currently installed.) 2023-10-11T09:38:26.380 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-11T09:38:26.546 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-10-11T09:38:27.069 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package dump. 2023-10-11T09:38:27.078 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-11T09:38:27.089 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2023-10-11T09:38:27.127 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking dump (0.4b46-6) ... 2023-10-11T09:38:27.218 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-10-11T09:38:27.474 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package indent. 2023-10-11T09:38:27.491 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2023-10-11T09:38:27.530 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2023-10-11T09:38:27.758 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-10-11T09:38:27.897 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.19) over (1.1.1f-1ubuntu2.16) ... 2023-10-11T09:38:27.917 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-10-11T09:38:27.937 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2023-10-11T09:38:27.975 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2023-10-11T09:38:28.345 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-10-11T09:38:28.364 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.6_amd64.deb ... 2023-10-11T09:38:28.403 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-10-11T09:38:28.485 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-10-11T09:38:28.504 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2023-10-11T09:38:28.543 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2023-10-11T09:38:28.722 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-10-11T09:38:28.742 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-11T09:38:28.780 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-11T09:38:28.954 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package dump. 2023-10-11T09:38:28.974 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2023-10-11T09:38:29.012 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking dump (0.4b46-6) ... 2023-10-11T09:38:29.192 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-10-11T09:38:29.212 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-10-11T09:38:29.250 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-10-11T09:38:29.374 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package indent. 2023-10-11T09:38:29.394 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2023-10-11T09:38:29.432 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2023-10-11T09:38:29.586 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libmbim-proxy. 2023-10-11T09:38:29.607 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-10-11T09:38:29.652 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-10-11T09:38:29.794 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-10-11T09:38:29.813 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2023-10-11T09:38:29.852 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2023-10-11T09:38:30.056 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-10-11T09:38:30.076 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-10-11T09:38:30.138 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-10-11T09:38:30.222 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-10-11T09:38:30.241 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.6_amd64.deb ... 2023-10-11T09:38:30.305 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-10-11T09:38:30.567 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libndp0:amd64. 2023-10-11T09:38:30.588 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1_amd64.deb ... 2023-10-11T09:38:30.608 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-10-11T09:38:30.625 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1) ... 2023-10-11T09:38:30.626 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-11T09:38:30.666 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-11T09:38:30.986 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libnm0:amd64. 2023-10-11T09:38:31.007 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.3_amd64.deb ... 2023-10-11T09:38:31.053 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-10-11T09:38:31.069 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-10-11T09:38:31.089 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-10-11T09:38:31.227 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-10-11T09:38:31.472 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libpq5:amd64. 2023-10-11T09:38:31.493 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../14-libpq5_12.16-0ubuntu0.20.04.1_amd64.deb ... 2023-10-11T09:38:31.530 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libpq5:amd64 (12.16-0ubuntu0.20.04.1) ... 2023-10-11T09:38:31.723 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libmbim-proxy. 2023-10-11T09:38:31.744 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-10-11T09:38:31.781 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-10-11T09:38:31.934 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-10-11T09:38:31.955 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../15-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-10-11T09:38:32.000 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-10-11T09:38:32.126 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-10-11T09:38:32.146 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-10-11T09:38:32.184 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-10-11T09:38:32.361 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libqmi-proxy. 2023-10-11T09:38:32.374 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../16-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-10-11T09:38:32.452 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-10-11T09:38:32.596 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libndp0:amd64. 2023-10-11T09:38:32.616 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1_amd64.deb ... 2023-10-11T09:38:32.654 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1) ... 2023-10-11T09:38:32.730 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-10-11T09:38:32.744 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../17-libssl-dev_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-10-11T09:38:32.822 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-11T09:38:33.015 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libnm0:amd64. 2023-10-11T09:38:33.035 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.3_amd64.deb ... 2023-10-11T09:38:33.106 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-10-11T09:38:33.417 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-10-11T09:38:33.436 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../18-libteamdctl0_1.30-1_amd64.deb ... 2023-10-11T09:38:33.475 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2023-10-11T09:38:33.509 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libpq5:amd64. 2023-10-11T09:38:33.529 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../14-libpq5_12.16-0ubuntu0.20.04.1_amd64.deb ... 2023-10-11T09:38:33.567 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libpq5:amd64 (12.16-0ubuntu0.20.04.1) ... 2023-10-11T09:38:33.962 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package modemmanager. 2023-10-11T09:38:33.983 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../19-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-10-11T09:38:33.995 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-10-11T09:38:34.014 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../15-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-10-11T09:38:34.052 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-10-11T09:38:34.053 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-10-11T09:38:34.448 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libqmi-proxy. 2023-10-11T09:38:34.468 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../16-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-10-11T09:38:34.506 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-10-11T09:38:34.743 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package wpasupplicant. 2023-10-11T09:38:34.763 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../20-wpasupplicant_2%3a2.9-1ubuntu4.3_amd64.deb ... 2023-10-11T09:38:34.808 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-10-11T09:38:34.809 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-10-11T09:38:34.829 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../17-libssl-dev_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-10-11T09:38:34.866 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-11T09:38:35.420 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-10-11T09:38:35.430 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package network-manager. 2023-10-11T09:38:35.441 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../18-libteamdctl0_1.30-1_amd64.deb ... 2023-10-11T09:38:35.451 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../21-network-manager_1.22.10-1ubuntu2.3_amd64.deb ... 2023-10-11T09:38:35.486 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2023-10-11T09:38:35.564 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking network-manager (1.22.10-1ubuntu2.3) ... 2023-10-11T09:38:35.940 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package modemmanager. 2023-10-11T09:38:35.959 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../19-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-10-11T09:38:36.018 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-10-11T09:38:36.388 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ppp. 2023-10-11T09:38:36.407 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../22-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2023-10-11T09:38:36.712 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package wpasupplicant. 2023-10-11T09:38:36.731 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../20-wpasupplicant_2%3a2.9-1ubuntu4.3_amd64.deb ... 2023-10-11T09:38:36.770 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-10-11T09:38:36.877 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-10-11T09:38:37.293 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package pptp-linux. 2023-10-11T09:38:37.313 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../23-pptp-linux_1.10.0-1build1_amd64.deb ... 2023-10-11T09:38:37.351 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2023-10-11T09:38:37.367 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package network-manager. 2023-10-11T09:38:37.387 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../21-network-manager_1.22.10-1ubuntu2.3_amd64.deb ... 2023-10-11T09:38:37.467 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking network-manager (1.22.10-1ubuntu2.3) ... 2023-10-11T09:38:37.796 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package network-manager-pptp. 2023-10-11T09:38:37.815 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../24-network-manager-pptp_1.2.8-2_amd64.deb ... 2023-10-11T09:38:37.894 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2023-10-11T09:38:38.274 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package ppp. 2023-10-11T09:38:38.274 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package postgresql-client-common. 2023-10-11T09:38:38.293 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../22-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2023-10-11T09:38:38.294 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../25-postgresql-client-common_214ubuntu0.1_all.deb ... 2023-10-11T09:38:38.341 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking postgresql-client-common (214ubuntu0.1) ... 2023-10-11T09:38:38.720 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package postgresql-client-12. 2023-10-11T09:38:38.740 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../26-postgresql-client-12_12.16-0ubuntu0.20.04.1_amd64.deb ... 2023-10-11T09:38:38.762 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-10-11T09:38:38.778 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking postgresql-client-12 (12.16-0ubuntu0.20.04.1) ... 2023-10-11T09:38:39.156 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package pptp-linux. 2023-10-11T09:38:39.176 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../23-pptp-linux_1.10.0-1build1_amd64.deb ... 2023-10-11T09:38:39.214 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2023-10-11T09:38:39.458 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package postgresql-common. 2023-10-11T09:38:39.478 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../27-postgresql-common_214ubuntu0.1_all.deb ... 2023-10-11T09:38:39.545 INFO:teuthology.orchestra.run.smithi133.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-10-11T09:38:39.577 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking postgresql-common (214ubuntu0.1) ... 2023-10-11T09:38:39.618 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package network-manager-pptp. 2023-10-11T09:38:39.638 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../24-network-manager-pptp_1.2.8-2_amd64.deb ... 2023-10-11T09:38:39.727 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2023-10-11T09:38:40.063 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package postgresql-client-common. 2023-10-11T09:38:40.083 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../25-postgresql-client-common_214ubuntu0.1_all.deb ... 2023-10-11T09:38:40.121 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking postgresql-client-common (214ubuntu0.1) ... 2023-10-11T09:38:40.148 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package postgresql-12. 2023-10-11T09:38:40.168 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../28-postgresql-12_12.16-0ubuntu0.20.04.1_amd64.deb ... 2023-10-11T09:38:40.213 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking postgresql-12 (12.16-0ubuntu0.20.04.1) ... 2023-10-11T09:38:40.475 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package postgresql-client-12. 2023-10-11T09:38:40.496 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../26-postgresql-client-12_12.16-0ubuntu0.20.04.1_amd64.deb ... 2023-10-11T09:38:40.533 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking postgresql-client-12 (12.16-0ubuntu0.20.04.1) ... 2023-10-11T09:38:41.188 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package postgresql-common. 2023-10-11T09:38:41.209 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../27-postgresql-common_214ubuntu0.1_all.deb ... 2023-10-11T09:38:41.278 INFO:teuthology.orchestra.run.smithi078.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-10-11T09:38:41.333 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking postgresql-common (214ubuntu0.1) ... 2023-10-11T09:38:41.817 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package postgresql. 2023-10-11T09:38:41.836 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../29-postgresql_12+214ubuntu0.1_all.deb ... 2023-10-11T09:38:41.862 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package postgresql-12. 2023-10-11T09:38:41.875 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking postgresql (12+214ubuntu0.1) ... 2023-10-11T09:38:41.881 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../28-postgresql-12_12.16-0ubuntu0.20.04.1_amd64.deb ... 2023-10-11T09:38:41.927 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking postgresql-12 (12.16-0ubuntu0.20.04.1) ... 2023-10-11T09:38:42.169 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package postgresql-client. 2023-10-11T09:38:42.189 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../30-postgresql-client_12+214ubuntu0.1_all.deb ... 2023-10-11T09:38:42.227 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking postgresql-client (12+214ubuntu0.1) ... 2023-10-11T09:38:42.546 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package postgresql-contrib. 2023-10-11T09:38:42.567 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../31-postgresql-contrib_12+214ubuntu0.1_all.deb ... 2023-10-11T09:38:42.612 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking postgresql-contrib (12+214ubuntu0.1) ... 2023-10-11T09:38:42.923 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-10-11T09:38:42.943 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../32-usb-modeswitch-data_20191128-3_all.deb ... 2023-10-11T09:38:42.981 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2023-10-11T09:38:43.451 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package usb-modeswitch. 2023-10-11T09:38:43.471 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../33-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2023-10-11T09:38:43.529 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-10-11T09:38:43.548 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package postgresql. 2023-10-11T09:38:43.563 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../29-postgresql_12+214ubuntu0.1_all.deb ... 2023-10-11T09:38:43.607 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking postgresql (12+214ubuntu0.1) ... 2023-10-11T09:38:43.868 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package postgresql-client. 2023-10-11T09:38:43.870 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up postgresql-client-common (214ubuntu0.1) ... 2023-10-11T09:38:43.884 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../30-postgresql-client_12+214ubuntu0.1_all.deb ... 2023-10-11T09:38:43.926 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking postgresql-client (12+214ubuntu0.1) ... 2023-10-11T09:38:44.115 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-11T09:38:44.203 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package postgresql-contrib. 2023-10-11T09:38:44.217 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../31-postgresql-contrib_12+214ubuntu0.1_all.deb ... 2023-10-11T09:38:44.261 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking postgresql-contrib (12+214ubuntu0.1) ... 2023-10-11T09:38:44.475 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libpq5:amd64 (12.16-0ubuntu0.20.04.1) ... 2023-10-11T09:38:44.522 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-10-11T09:38:44.536 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../32-usb-modeswitch-data_20191128-3_all.deb ... 2023-10-11T09:38:44.601 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2023-10-11T09:38:44.605 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2023-10-11T09:38:44.760 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2023-10-11T09:38:44.886 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-10-11T09:38:45.034 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package usb-modeswitch. 2023-10-11T09:38:45.054 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../33-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2023-10-11T09:38:45.137 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-10-11T09:38:45.361 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-11T09:38:45.361 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-11T09:38:45.483 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up postgresql-client-common (214ubuntu0.1) ... 2023-10-11T09:38:45.715 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-11T09:38:46.083 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libpq5:amd64 (12.16-0ubuntu0.20.04.1) ... 2023-10-11T09:38:46.242 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2023-10-11T09:38:46.305 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up postgresql-client-12 (12.16-0ubuntu0.20.04.1) ... 2023-10-11T09:38:46.368 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2023-10-11T09:38:46.494 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-10-11T09:38:46.969 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-11T09:38:46.969 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-11T09:38:47.898 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up postgresql-client-12 (12.16-0ubuntu0.20.04.1) ... 2023-10-11T09:38:48.414 INFO:teuthology.orchestra.run.smithi133.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-10-11T09:38:48.552 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.4) ... 2023-10-11T09:38:48.679 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2023-10-11T09:38:48.804 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up postgresql-common (214ubuntu0.1) ... 2023-10-11T09:38:49.984 INFO:teuthology.orchestra.run.smithi078.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-10-11T09:38:50.103 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.4) ... 2023-10-11T09:38:50.104 INFO:teuthology.orchestra.run.smithi133.stdout:Adding user postgres to group ssl-cert 2023-10-11T09:38:50.238 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2023-10-11T09:38:50.364 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up postgresql-common (214ubuntu0.1) ... 2023-10-11T09:38:50.493 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:38:50.493 INFO:teuthology.orchestra.run.smithi133.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-10-11T09:38:50.988 INFO:teuthology.orchestra.run.smithi133.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-10-11T09:38:50.989 INFO:teuthology.orchestra.run.smithi133.stdout:Removing obsolete dictionary files: 2023-10-11T09:38:51.542 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-10-11T09:38:51.655 INFO:teuthology.orchestra.run.smithi078.stdout:Adding user postgres to group ssl-cert 2023-10-11T09:38:52.306 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:38:52.306 INFO:teuthology.orchestra.run.smithi078.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-10-11T09:38:52.800 INFO:teuthology.orchestra.run.smithi078.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-10-11T09:38:52.800 INFO:teuthology.orchestra.run.smithi078.stdout:Removing obsolete dictionary files: 2023-10-11T09:38:52.980 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-11T09:38:53.115 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2023-10-11T09:38:53.266 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-10-11T09:38:53.352 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-10-11T09:38:53.392 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-10-11T09:38:54.689 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2023-10-11T09:38:54.733 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-11T09:38:54.858 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2023-10-11T09:38:55.026 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-10-11T09:38:55.130 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up dump (0.4b46-6) ... 2023-10-11T09:38:55.152 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-10-11T09:38:55.240 INFO:teuthology.orchestra.run.smithi133.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-10-11T09:38:55.307 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-10-11T09:38:55.458 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-10-11T09:38:55.576 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2023-10-11T09:38:55.702 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-10-11T09:38:55.912 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1) ... 2023-10-11T09:38:56.037 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up postgresql-12 (12.16-0ubuntu0.20.04.1) ... 2023-10-11T09:38:56.386 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2023-10-11T09:38:56.523 INFO:teuthology.orchestra.run.smithi133.stdout:Creating new PostgreSQL cluster 12/main ... 2023-10-11T09:38:56.554 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/lib/postgresql/12/bin/initdb -D /var/lib/postgresql/12/main --auth-local peer --auth-host md5 2023-10-11T09:38:56.650 INFO:teuthology.orchestra.run.smithi133.stdout:The files belonging to this database system will be owned by user "postgres". 2023-10-11T09:38:56.650 INFO:teuthology.orchestra.run.smithi133.stdout:This user must also own the server process. 2023-10-11T09:38:56.650 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:38:56.650 INFO:teuthology.orchestra.run.smithi133.stdout:The database cluster will be initialized with locale "en_US". 2023-10-11T09:38:56.651 INFO:teuthology.orchestra.run.smithi133.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-10-11T09:38:56.651 INFO:teuthology.orchestra.run.smithi133.stdout:The default text search configuration will be set to "english". 2023-10-11T09:38:56.651 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:38:56.651 INFO:teuthology.orchestra.run.smithi133.stdout:Data page checksums are disabled. 2023-10-11T09:38:56.651 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:38:56.652 INFO:teuthology.orchestra.run.smithi133.stdout:fixing permissions on existing directory /var/lib/postgresql/12/main ... ok 2023-10-11T09:38:56.653 INFO:teuthology.orchestra.run.smithi133.stdout:creating subdirectories ... ok 2023-10-11T09:38:56.653 INFO:teuthology.orchestra.run.smithi133.stdout:selecting dynamic shared memory implementation ... posix 2023-10-11T09:38:56.691 INFO:teuthology.orchestra.run.smithi133.stdout:selecting default max_connections ... 100 2023-10-11T09:38:56.736 INFO:teuthology.orchestra.run.smithi133.stdout:selecting default shared_buffers ... 128MB 2023-10-11T09:38:56.757 INFO:teuthology.orchestra.run.smithi133.stdout:selecting default time zone ... Etc/UTC 2023-10-11T09:38:56.758 INFO:teuthology.orchestra.run.smithi133.stdout:creating configuration files ... ok 2023-10-11T09:38:56.791 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up dump (0.4b46-6) ... 2023-10-11T09:38:56.901 INFO:teuthology.orchestra.run.smithi078.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-10-11T09:38:56.950 INFO:teuthology.orchestra.run.smithi133.stdout:running bootstrap script ... ok 2023-10-11T09:38:57.015 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-10-11T09:38:57.478 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-10-11T09:38:57.549 INFO:teuthology.orchestra.run.smithi133.stdout:performing post-bootstrap initialization ... ok 2023-10-11T09:38:57.604 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2023-10-11T09:38:57.755 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-10-11T09:38:57.948 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1) ... 2023-10-11T09:38:58.074 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up postgresql-12 (12.16-0ubuntu0.20.04.1) ... 2023-10-11T09:38:58.418 INFO:teuthology.orchestra.run.smithi133.stdout:syncing data to disk ... ok 2023-10-11T09:38:58.418 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:38:58.419 INFO:teuthology.orchestra.run.smithi133.stdout:Success. You can now start the database server using: 2023-10-11T09:38:58.419 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:38:58.419 INFO:teuthology.orchestra.run.smithi133.stdout: pg_ctlcluster 12 main start 2023-10-11T09:38:58.421 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:38:58.576 INFO:teuthology.orchestra.run.smithi078.stdout:Creating new PostgreSQL cluster 12/main ... 2023-10-11T09:38:58.613 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/lib/postgresql/12/bin/initdb -D /var/lib/postgresql/12/main --auth-local peer --auth-host md5 2023-10-11T09:38:58.693 INFO:teuthology.orchestra.run.smithi078.stdout:The files belonging to this database system will be owned by user "postgres". 2023-10-11T09:38:58.693 INFO:teuthology.orchestra.run.smithi078.stdout:This user must also own the server process. 2023-10-11T09:38:58.694 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:38:58.694 INFO:teuthology.orchestra.run.smithi078.stdout:The database cluster will be initialized with locale "en_US". 2023-10-11T09:38:58.694 INFO:teuthology.orchestra.run.smithi078.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-10-11T09:38:58.694 INFO:teuthology.orchestra.run.smithi078.stdout:The default text search configuration will be set to "english". 2023-10-11T09:38:58.695 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:38:58.695 INFO:teuthology.orchestra.run.smithi078.stdout:Data page checksums are disabled. 2023-10-11T09:38:58.695 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:38:58.695 INFO:teuthology.orchestra.run.smithi078.stdout:fixing permissions on existing directory /var/lib/postgresql/12/main ... ok 2023-10-11T09:38:58.697 INFO:teuthology.orchestra.run.smithi078.stdout:creating subdirectories ... ok 2023-10-11T09:38:58.697 INFO:teuthology.orchestra.run.smithi078.stdout:selecting dynamic shared memory implementation ... posix 2023-10-11T09:38:58.730 INFO:teuthology.orchestra.run.smithi078.stdout:selecting default max_connections ... 100 2023-10-11T09:38:58.775 INFO:teuthology.orchestra.run.smithi078.stdout:selecting default shared_buffers ... 128MB 2023-10-11T09:38:58.797 INFO:teuthology.orchestra.run.smithi078.stdout:selecting default time zone ... Etc/UTC 2023-10-11T09:38:58.798 INFO:teuthology.orchestra.run.smithi078.stdout:creating configuration files ... ok 2023-10-11T09:38:58.986 INFO:teuthology.orchestra.run.smithi133.stdout:Ver Cluster Port Status Owner Data directory Log file 2023-10-11T09:38:58.986 INFO:teuthology.orchestra.run.smithi133.stdout:12 main 5432 down postgres /var/lib/postgresql/12/main /var/log/postgresql/postgresql-12-main.log 2023-10-11T09:38:58.988 INFO:teuthology.orchestra.run.smithi078.stdout:running bootstrap script ... ok 2023-10-11T09:38:59.199 INFO:teuthology.orchestra.run.smithi133.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-10-11T09:38:59.640 INFO:teuthology.orchestra.run.smithi078.stdout:performing post-bootstrap initialization ... ok 2023-10-11T09:39:00.180 INFO:teuthology.orchestra.run.smithi078.stdout:syncing data to disk ... ok 2023-10-11T09:39:00.180 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:39:00.180 INFO:teuthology.orchestra.run.smithi078.stdout:Success. You can now start the database server using: 2023-10-11T09:39:00.181 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:39:00.181 INFO:teuthology.orchestra.run.smithi078.stdout: pg_ctlcluster 12 main start 2023-10-11T09:39:00.181 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:39:00.741 INFO:teuthology.orchestra.run.smithi078.stdout:Ver Cluster Port Status Owner Data directory Log file 2023-10-11T09:39:00.741 INFO:teuthology.orchestra.run.smithi078.stdout:12 main 5432 down postgres /var/lib/postgresql/12/main /var/log/postgresql/postgresql-12-main.log 2023-10-11T09:39:00.965 INFO:teuthology.orchestra.run.smithi078.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-10-11T09:39:02.845 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-11T09:39:02.971 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-10-11T09:39:03.147 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2023-10-11T09:39:03.315 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-11T09:39:03.466 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up postgresql-client (12+214ubuntu0.1) ... 2023-10-11T09:39:03.592 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up network-manager (1.22.10-1ubuntu2.3) ... 2023-10-11T09:39:04.542 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-11T09:39:04.675 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-10-11T09:39:04.801 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2023-10-11T09:39:04.823 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:39:04.824 INFO:teuthology.orchestra.run.smithi133.stdout:The following network interfaces were found in /etc/network/interfaces 2023-10-11T09:39:04.824 INFO:teuthology.orchestra.run.smithi133.stdout:which means they are currently configured by ifupdown: 2023-10-11T09:39:04.824 INFO:teuthology.orchestra.run.smithi133.stdout:- enp3s0f1 2023-10-11T09:39:04.824 INFO:teuthology.orchestra.run.smithi133.stdout:If you want to manage those interfaces with NetworkManager instead 2023-10-11T09:39:04.824 INFO:teuthology.orchestra.run.smithi133.stdout:remove their configuration from /etc/network/interfaces. 2023-10-11T09:39:04.824 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:39:05.003 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-11T09:39:05.051 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-10-11T09:39:05.137 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up postgresql-client (12+214ubuntu0.1) ... 2023-10-11T09:39:05.263 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up network-manager (1.22.10-1ubuntu2.3) ... 2023-10-11T09:39:05.608 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-10-11T09:39:06.385 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:39:06.386 INFO:teuthology.orchestra.run.smithi078.stdout:The following network interfaces were found in /etc/network/interfaces 2023-10-11T09:39:06.386 INFO:teuthology.orchestra.run.smithi078.stdout:which means they are currently configured by ifupdown: 2023-10-11T09:39:06.386 INFO:teuthology.orchestra.run.smithi078.stdout:- enp3s0f1 2023-10-11T09:39:06.386 INFO:teuthology.orchestra.run.smithi078.stdout:If you want to manage those interfaces with NetworkManager instead 2023-10-11T09:39:06.386 INFO:teuthology.orchestra.run.smithi078.stdout:remove their configuration from /etc/network/interfaces. 2023-10-11T09:39:06.387 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:39:06.578 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-10-11T09:39:06.612 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-10-11T09:39:07.161 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-10-11T09:39:07.544 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-10-11T09:39:07.704 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2023-10-11T09:39:07.855 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-11T09:39:07.981 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up postgresql-contrib (12+214ubuntu0.1) ... 2023-10-11T09:39:08.132 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up postgresql (12+214ubuntu0.1) ... 2023-10-11T09:39:08.201 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-10-11T09:39:08.258 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-10-11T09:39:08.551 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-10-11T09:39:08.677 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-10-11T09:39:09.009 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-10-11T09:39:09.010 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-10-11T09:39:09.124 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-10-11T09:39:09.284 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2023-10-11T09:39:09.435 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-11T09:39:09.561 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up postgresql-contrib (12+214ubuntu0.1) ... 2023-10-11T09:39:09.687 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up postgresql (12+214ubuntu0.1) ... 2023-10-11T09:39:09.855 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-10-11T09:39:09.946 INFO:teuthology.orchestra.run.smithi133.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2023-10-11T09:39:09.981 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-10-11T09:39:10.073 INFO:teuthology.orchestra.run.smithi133.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-10-11T09:39:10.107 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-10-11T09:39:10.215 INFO:teuthology.orchestra.run.smithi133.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-10-11T09:39:10.400 INFO:teuthology.orchestra.run.smithi133.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-10-11T09:39:10.422 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-10-11T09:39:10.423 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-10-11T09:39:10.901 INFO:teuthology.orchestra.run.smithi133.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-10-11T09:39:11.393 INFO:teuthology.orchestra.run.smithi078.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2023-10-11T09:39:11.536 INFO:teuthology.orchestra.run.smithi078.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-10-11T09:39:11.679 INFO:teuthology.orchestra.run.smithi078.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-10-11T09:39:11.847 INFO:teuthology.orchestra.run.smithi078.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-10-11T09:39:12.340 INFO:teuthology.orchestra.run.smithi078.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-10-11T09:39:23.946 INFO:teuthology.orchestra.run.smithi133.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2023-10-11T09:39:24.063 INFO:teuthology.orchestra.run.smithi133.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2023-10-11T09:39:25.362 INFO:teuthology.orchestra.run.smithi133.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-11T09:39:25.368 DEBUG:teuthology.parallel:result is None 2023-10-11T09:39:25.578 INFO:teuthology.orchestra.run.smithi078.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2023-10-11T09:39:25.686 INFO:teuthology.orchestra.run.smithi078.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2023-10-11T09:39:26.966 INFO:teuthology.orchestra.run.smithi078.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-11T09:39:26.971 DEBUG:teuthology.parallel:result is None 2023-10-11T09:39:26.972 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=9a774796f0ecdd11a4b771de396e627aa1a187af 2023-10-11T09:39:27.201 DEBUG:teuthology.orchestra.run.smithi078:> dpkg-query -W -f '${Version}' ceph 2023-10-11T09:39:27.236 INFO:teuthology.orchestra.run.smithi078.stdout:18.2.0-662-g9a774796-1focal 2023-10-11T09:39:27.237 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-662-g9a774796-1focal 2023-10-11T09:39:27.237 INFO:teuthology.task.install:The correct ceph version 18.2.0-662-g9a774796-1focal is installed. 2023-10-11T09:39:27.238 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=9a774796f0ecdd11a4b771de396e627aa1a187af 2023-10-11T09:39:27.473 DEBUG:teuthology.orchestra.run.smithi133:> dpkg-query -W -f '${Version}' ceph 2023-10-11T09:39:27.510 INFO:teuthology.orchestra.run.smithi133.stdout:18.2.0-662-g9a774796-1focal 2023-10-11T09:39:27.511 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-662-g9a774796-1focal 2023-10-11T09:39:27.511 INFO:teuthology.task.install:The correct ceph version 18.2.0-662-g9a774796-1focal is installed. 2023-10-11T09:39:27.512 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-11T09:39:27.512 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:39:27.513 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-11T09:39:27.547 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:39:27.548 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-11T09:39:27.589 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-11T09:39:27.590 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:39:27.590 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/daemon-helper 2023-10-11T09:39:27.625 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-11T09:39:27.698 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:39:27.698 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/daemon-helper 2023-10-11T09:39:27.733 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-11T09:39:27.813 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-11T09:39:27.813 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:39:27.813 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-11T09:39:27.847 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-11T09:39:27.927 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:39:27.927 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-11T09:39:27.961 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-11T09:39:28.036 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-11T09:39:28.037 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:39:28.037 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/stdin-killer 2023-10-11T09:39:28.071 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-11T09:39:28.151 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:39:28.151 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/stdin-killer 2023-10-11T09:39:28.185 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-11T09:39:28.260 INFO:teuthology.run_tasks:Running task ceph... 2023-10-11T09:39:28.328 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-11T09:39:28.329 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 777 /var/log/ceph 2023-10-11T09:39:28.336 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 777 /var/log/ceph 2023-10-11T09:39:28.369 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-11T09:39:28.370 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-11T09:39:28.412 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-11T09:39:28.447 INFO:tasks.ceph:Creating extra log directories... 2023-10-11T09:39:28.448 DEBUG:teuthology.orchestra.run.smithi078:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-11T09:39:28.484 DEBUG:teuthology.orchestra.run.smithi133:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-11T09:39:28.534 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-11T09:39:28.535 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-11T09:39:28.535 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-11T09:39:28.536 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:39:28.536 DEBUG:teuthology.orchestra.run.smithi078:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-11T09:39:28.570 DEBUG:teuthology.orchestra.run.smithi078:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-11T09:39:28.709 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:39:28.710 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-11T09:39:28.730 DEBUG:teuthology.orchestra.run.smithi133:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-11T09:39:28.862 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-11T09:39:28.862 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'unmatched fragstat', 'unmatched rstat', 'was unreadable, recreating it now', 'Scrub error on inode', 'Metadata damage detected', 'inconsistent rstat on inode', 'Error recovering journal'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-11T09:39:28.863 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi/7418830', 'branch': 'wip-vshankar-testing-reef-20231009.131610', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/data-scan}', 'email': 'vshankar@redhat.com', 'first_in_suite': False, 'job_id': '7418830', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': 'fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-vshankar-testing-reef-20231009.131610'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'unmatched fragstat', 'unmatched rstat', 'was unreadable, recreating it now', 'Scrub error on inode', 'Metadata damage detected', 'inconsistent rstat on inode', 'Error recovering journal'], 'sha1': '9a774796f0ecdd11a4b771de396e627aa1a187af'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '9a774796f0ecdd11a4b771de396e627aa1a187af'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-vshankar-testing-reef-20231009.131610', 'sha1': '9a774796f0ecdd11a4b771de396e627aa1a187af'}}, 'owner': 'scheduled_vshankar@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1116, 'sha1': '9a774796f0ecdd11a4b771de396e627aa1a187af', 'sleep_before_teardown': 0, 'subset': '593/1024', 'suite': 'fs', 'suite_branch': 'wip-vshankar-testing-reef-20231009.131610', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_9a774796f0ecdd11a4b771de396e627aa1a187af/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '9a774796f0ecdd11a4b771de396e627aa1a187af', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi078.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi133.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': 'fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '9a774796f0ecdd11a4b771de396e627aa1a187af'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_data_scan']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-10-09_16:48:27', 'tube': 'smithi', 'user': 'vshankar', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-11T09:39:28.864 DEBUG:teuthology.orchestra.run.smithi078:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-11T09:39:28.871 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-11T09:39:28.889 DEBUG:teuthology.orchestra.run.smithi078:> sudo install -d -m0777 -- /var/run/ceph 2023-10-11T09:39:28.932 DEBUG:teuthology.orchestra.run.smithi133:> sudo install -d -m0777 -- /var/run/ceph 2023-10-11T09:39:28.966 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:39:28.966 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/scratch_devs of=/dev/stdout 2023-10-11T09:39:29.020 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-11T09:39:29.021 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_1 2023-10-11T09:39:29.084 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-11T09:39:29.084 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-11T09:39:29.085 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 497 Links: 1 2023-10-11T09:39:29.085 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-11T09:39:29.085 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2023-10-11 09:33:50.883127096 +0000 2023-10-11T09:39:29.085 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2023-10-11 09:33:50.435137422 +0000 2023-10-11T09:39:29.086 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2023-10-11 09:33:50.435137422 +0000 2023-10-11T09:39:29.086 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: - 2023-10-11T09:39:29.087 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-11T09:39:29.157 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2023-10-11T09:39:29.157 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2023-10-11T09:39:29.157 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000689549 s, 743 kB/s 2023-10-11T09:39:29.159 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-11T09:39:29.228 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_2 2023-10-11T09:39:29.288 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-11T09:39:29.288 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-11T09:39:29.288 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 503 Links: 1 2023-10-11T09:39:29.289 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-11T09:39:29.289 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2023-10-11 09:33:51.583110960 +0000 2023-10-11T09:39:29.289 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2023-10-11 09:33:51.151120918 +0000 2023-10-11T09:39:29.290 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2023-10-11 09:33:51.151120918 +0000 2023-10-11T09:39:29.290 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: - 2023-10-11T09:39:29.291 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-11T09:39:29.365 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2023-10-11T09:39:29.365 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2023-10-11T09:39:29.365 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000668515 s, 766 kB/s 2023-10-11T09:39:29.367 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-11T09:39:29.434 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_3 2023-10-11T09:39:29.496 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-11T09:39:29.496 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-11T09:39:29.496 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 511 Links: 1 2023-10-11T09:39:29.496 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-11T09:39:29.496 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2023-10-11 09:33:52.343093443 +0000 2023-10-11T09:39:29.497 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2023-10-11 09:33:51.903103584 +0000 2023-10-11T09:39:29.497 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2023-10-11 09:33:51.903103584 +0000 2023-10-11T09:39:29.497 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: - 2023-10-11T09:39:29.497 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-11T09:39:29.569 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2023-10-11T09:39:29.569 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2023-10-11T09:39:29.569 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000689182 s, 743 kB/s 2023-10-11T09:39:29.571 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-11T09:39:29.639 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_4 2023-10-11T09:39:29.699 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-11T09:39:29.700 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-11T09:39:29.700 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 518 Links: 1 2023-10-11T09:39:29.700 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-11T09:39:29.700 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2023-10-11 09:33:53.087076294 +0000 2023-10-11T09:39:29.701 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2023-10-11 09:33:52.615087173 +0000 2023-10-11T09:39:29.701 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2023-10-11 09:33:52.615087173 +0000 2023-10-11T09:39:29.701 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: - 2023-10-11T09:39:29.702 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-11T09:39:29.773 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2023-10-11T09:39:29.773 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2023-10-11T09:39:29.773 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000569305 s, 899 kB/s 2023-10-11T09:39:29.775 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-11T09:39:29.844 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-11T09:39:29.845 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:39:29.845 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/scratch_devs of=/dev/stdout 2023-10-11T09:39:29.862 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-11T09:39:29.863 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_1 2023-10-11T09:39:29.920 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-11T09:39:29.921 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-11T09:39:29.921 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 498 Links: 1 2023-10-11T09:39:29.921 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-11T09:39:29.921 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2023-10-11 09:33:49.782358065 +0000 2023-10-11T09:39:29.921 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2023-10-11 09:33:49.350368023 +0000 2023-10-11T09:39:29.922 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2023-10-11 09:33:49.350368023 +0000 2023-10-11T09:39:29.922 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: - 2023-10-11T09:39:29.922 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-11T09:39:29.994 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2023-10-11T09:39:29.995 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2023-10-11T09:39:29.995 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000617226 s, 830 kB/s 2023-10-11T09:39:29.996 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-11T09:39:30.069 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_2 2023-10-11T09:39:30.129 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-11T09:39:30.130 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-11T09:39:30.130 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 505 Links: 1 2023-10-11T09:39:30.130 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-11T09:39:30.130 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2023-10-11 09:33:50.530340825 +0000 2023-10-11T09:39:30.131 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2023-10-11 09:33:50.082351151 +0000 2023-10-11T09:39:30.131 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2023-10-11 09:33:50.082351151 +0000 2023-10-11T09:39:30.131 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: - 2023-10-11T09:39:30.131 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-11T09:39:30.202 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2023-10-11T09:39:30.202 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2023-10-11T09:39:30.202 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000571364 s, 896 kB/s 2023-10-11T09:39:30.204 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-11T09:39:30.274 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_3 2023-10-11T09:39:30.333 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-11T09:39:30.333 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-11T09:39:30.334 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 512 Links: 1 2023-10-11T09:39:30.334 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-11T09:39:30.334 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2023-10-11 09:33:51.258324044 +0000 2023-10-11T09:39:30.334 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2023-10-11 09:33:50.818334185 +0000 2023-10-11T09:39:30.334 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2023-10-11 09:33:50.818334185 +0000 2023-10-11T09:39:30.335 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: - 2023-10-11T09:39:30.335 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-11T09:39:30.406 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2023-10-11T09:39:30.407 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2023-10-11T09:39:30.407 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000600549 s, 853 kB/s 2023-10-11T09:39:30.408 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-11T09:39:30.480 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_4 2023-10-11T09:39:30.543 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-11T09:39:30.543 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-11T09:39:30.543 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 516 Links: 1 2023-10-11T09:39:30.543 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-11T09:39:30.544 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2023-10-11 09:33:51.994307080 +0000 2023-10-11T09:39:30.544 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2023-10-11 09:33:51.566316944 +0000 2023-10-11T09:39:30.544 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2023-10-11 09:33:51.566316944 +0000 2023-10-11T09:39:30.544 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: - 2023-10-11T09:39:30.545 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-11T09:39:30.618 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2023-10-11T09:39:30.618 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2023-10-11T09:39:30.619 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000671318 s, 763 kB/s 2023-10-11T09:39:30.620 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-11T09:39:30.691 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-11T09:39:30.691 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi078.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi133.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-11T09:39:30.692 INFO:tasks.ceph:Generating config... 2023-10-11T09:39:30.693 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-11T09:39:30.693 INFO:tasks.ceph:[client] debug client = 20 2023-10-11T09:39:30.693 INFO:tasks.ceph:[client] debug ms = 1 2023-10-11T09:39:30.694 INFO:tasks.ceph:[client] pid file = 2023-10-11T09:39:30.694 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-11T09:39:30.694 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-11T09:39:30.695 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-11T09:39:30.695 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-10-11T09:39:30.695 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-11T09:39:30.695 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-11T09:39:30.696 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-11T09:39:30.696 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-11T09:39:30.696 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-11T09:39:30.696 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-11T09:39:30.697 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-11T09:39:30.697 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-11T09:39:30.697 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-11T09:39:30.697 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-11T09:39:30.698 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-11T09:39:30.698 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-11T09:39:30.698 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-11T09:39:30.698 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-11T09:39:30.699 INFO:tasks.ceph:[osd] bdev async discard = True 2023-10-11T09:39:30.699 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-10-11T09:39:30.699 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-10-11T09:39:30.699 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-11T09:39:30.700 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-11T09:39:30.700 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-10-11T09:39:30.701 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-10-11T09:39:30.701 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-11T09:39:30.701 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-11T09:39:30.701 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-10-11T09:39:30.702 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-11T09:39:30.702 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-11T09:39:30.702 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-11T09:39:30.702 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-11T09:39:30.703 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-11T09:39:30.703 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-11T09:39:30.703 INFO:tasks.ceph:Setting up mon.a... 2023-10-11T09:39:30.704 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-11T09:39:30.774 INFO:teuthology.orchestra.run.smithi078.stdout:creating /etc/ceph/ceph.keyring 2023-10-11T09:39:30.791 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-11T09:39:30.864 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-11T09:39:30.895 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.78'), ('mon.b', '172.21.15.133'), ('mon.c', '[v2:172.21.15.133:3301,v1:172.21.15.133:6790]')] 2023-10-11T09:39:30.896 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.78,172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-11T09:39:30.897 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:39:30.897 DEBUG:teuthology.orchestra.run.smithi078:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-11T09:39:30.954 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.78 --add b 172.21.15.133 --addv c '[v2:172.21.15.133:3301,v1:172.21.15.133:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-11T09:39:31.070 INFO:teuthology.orchestra.run.smithi078.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-11T09:39:31.071 INFO:teuthology.orchestra.run.smithi078.stdout:monmaptool: generated fsid dee99db9-7a72-406f-a1e2-917943f2b13d 2023-10-11T09:39:31.071 INFO:teuthology.orchestra.run.smithi078.stdout:setting min_mon_release = pacific 2023-10-11T09:39:31.071 INFO:teuthology.orchestra.run.smithi078.stdout:epoch 0 2023-10-11T09:39:31.071 INFO:teuthology.orchestra.run.smithi078.stdout:fsid dee99db9-7a72-406f-a1e2-917943f2b13d 2023-10-11T09:39:31.071 INFO:teuthology.orchestra.run.smithi078.stdout:last_changed 2023-10-11T09:39:31.068440+0000 2023-10-11T09:39:31.072 INFO:teuthology.orchestra.run.smithi078.stdout:created 2023-10-11T09:39:31.068440+0000 2023-10-11T09:39:31.072 INFO:teuthology.orchestra.run.smithi078.stdout:min_mon_release 16 (pacific) 2023-10-11T09:39:31.072 INFO:teuthology.orchestra.run.smithi078.stdout:election_strategy: 1 2023-10-11T09:39:31.072 INFO:teuthology.orchestra.run.smithi078.stdout:0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2023-10-11T09:39:31.072 INFO:teuthology.orchestra.run.smithi078.stdout:1: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.b 2023-10-11T09:39:31.073 INFO:teuthology.orchestra.run.smithi078.stdout:2: [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] mon.c 2023-10-11T09:39:31.073 INFO:teuthology.orchestra.run.smithi078.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-11T09:39:31.074 DEBUG:teuthology.orchestra.run.smithi078:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-11T09:39:31.092 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID dee99db9-7a72-406f-a1e2-917943f2b13d... 2023-10-11T09:39:31.093 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-11T09:39:31.141 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-11T09:39:31.210 INFO:teuthology.orchestra.run.smithi078.stdout:[global] 2023-10-11T09:39:31.211 INFO:teuthology.orchestra.run.smithi078.stdout: chdir = "" 2023-10-11T09:39:31.211 INFO:teuthology.orchestra.run.smithi078.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-11T09:39:31.211 INFO:teuthology.orchestra.run.smithi078.stdout: auth supported = cephx 2023-10-11T09:39:31.211 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:39:31.212 INFO:teuthology.orchestra.run.smithi078.stdout: filestore xattr use omap = true 2023-10-11T09:39:31.212 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:39:31.212 INFO:teuthology.orchestra.run.smithi078.stdout: mon clock drift allowed = 1.000 2023-10-11T09:39:31.212 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:39:31.212 INFO:teuthology.orchestra.run.smithi078.stdout: osd crush chooseleaf type = 0 2023-10-11T09:39:31.212 INFO:teuthology.orchestra.run.smithi078.stdout: auth debug = true 2023-10-11T09:39:31.213 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:39:31.213 INFO:teuthology.orchestra.run.smithi078.stdout: ms die on old message = true 2023-10-11T09:39:31.213 INFO:teuthology.orchestra.run.smithi078.stdout: ms die on bug = true 2023-10-11T09:39:31.213 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:39:31.213 INFO:teuthology.orchestra.run.smithi078.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-11T09:39:31.214 INFO:teuthology.orchestra.run.smithi078.stdout: mon pg warn max object skew = 0 2023-10-11T09:39:31.214 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:39:31.214 INFO:teuthology.orchestra.run.smithi078.stdout: # disable pg_autoscaler by default for new pools 2023-10-11T09:39:31.214 INFO:teuthology.orchestra.run.smithi078.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-11T09:39:31.214 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:39:31.214 INFO:teuthology.orchestra.run.smithi078.stdout: osd pool default size = 2 2023-10-11T09:39:31.215 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:39:31.215 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd allow primary affinity = true 2023-10-11T09:39:31.215 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd allow pg remap = true 2023-10-11T09:39:31.215 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on legacy crush tunables = false 2023-10-11T09:39:31.215 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on crush straw calc version zero = false 2023-10-11T09:39:31.215 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on no sortbitwise = false 2023-10-11T09:39:31.216 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on osd down out interval zero = false 2023-10-11T09:39:31.216 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on too few osds = false 2023-10-11T09:39:31.216 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-11T09:39:31.216 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-11T09:39:31.216 INFO:teuthology.orchestra.run.smithi078.stdout: mon_allow_pool_size_one = true 2023-10-11T09:39:31.217 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:39:31.217 INFO:teuthology.orchestra.run.smithi078.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-11T09:39:31.217 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:39:31.217 INFO:teuthology.orchestra.run.smithi078.stdout: osd default data pool replay window = 5 2023-10-11T09:39:31.217 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:39:31.217 INFO:teuthology.orchestra.run.smithi078.stdout: mon allow pool delete = true 2023-10-11T09:39:31.218 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:39:31.218 INFO:teuthology.orchestra.run.smithi078.stdout: mon cluster log file level = debug 2023-10-11T09:39:31.218 INFO:teuthology.orchestra.run.smithi078.stdout: debug asserts on shutdown = true 2023-10-11T09:39:31.218 INFO:teuthology.orchestra.run.smithi078.stdout: mon health detail to clog = false 2023-10-11T09:39:31.218 INFO:teuthology.orchestra.run.smithi078.stdout: mon host = "172.21.15.78,172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790]" 2023-10-11T09:39:31.219 INFO:teuthology.orchestra.run.smithi078.stdout: fsid = dee99db9-7a72-406f-a1e2-917943f2b13d 2023-10-11T09:39:31.219 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:39:31.219 INFO:teuthology.orchestra.run.smithi078.stdout:[osd] 2023-10-11T09:39:31.219 INFO:teuthology.orchestra.run.smithi078.stdout: osd journal size = 100 2023-10-11T09:39:31.219 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:39:31.221 INFO:teuthology.orchestra.run.smithi078.stdout: osd scrub load threshold = 5.0 2023-10-11T09:39:31.221 INFO:teuthology.orchestra.run.smithi078.stdout: osd scrub max interval = 600 2023-10-11T09:39:31.221 INFO:teuthology.orchestra.run.smithi078.stdout: osd mclock profile = high_recovery_ops 2023-10-11T09:39:31.221 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:39:31.222 INFO:teuthology.orchestra.run.smithi078.stdout: osd recover clone overlap = true 2023-10-11T09:39:31.222 INFO:teuthology.orchestra.run.smithi078.stdout: osd recovery max chunk = 1048576 2023-10-11T09:39:31.222 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:39:31.222 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug shutdown = true 2023-10-11T09:39:31.222 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug op order = true 2023-10-11T09:39:31.223 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug verify stray on activate = true 2023-10-11T09:39:31.223 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:39:31.223 INFO:teuthology.orchestra.run.smithi078.stdout: osd open classes on start = true 2023-10-11T09:39:31.223 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug pg log writeout = true 2023-10-11T09:39:31.223 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:39:31.224 INFO:teuthology.orchestra.run.smithi078.stdout: osd deep scrub update digest min age = 30 2023-10-11T09:39:31.224 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:39:31.224 INFO:teuthology.orchestra.run.smithi078.stdout: osd map max advance = 10 2023-10-11T09:39:31.224 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:39:31.224 INFO:teuthology.orchestra.run.smithi078.stdout: journal zero on create = true 2023-10-11T09:39:31.224 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:39:31.225 INFO:teuthology.orchestra.run.smithi078.stdout: filestore ondisk finisher threads = 3 2023-10-11T09:39:31.225 INFO:teuthology.orchestra.run.smithi078.stdout: filestore apply finisher threads = 3 2023-10-11T09:39:31.225 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:39:31.225 INFO:teuthology.orchestra.run.smithi078.stdout: bdev debug aio = true 2023-10-11T09:39:31.225 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug misdirected ops = true 2023-10-11T09:39:31.226 INFO:teuthology.orchestra.run.smithi078.stdout: bdev async discard = True 2023-10-11T09:39:31.226 INFO:teuthology.orchestra.run.smithi078.stdout: bdev enable discard = True 2023-10-11T09:39:31.226 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore allocator = bitmap 2023-10-11T09:39:31.226 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore block size = 96636764160 2023-10-11T09:39:31.226 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore fsck on mount = True 2023-10-11T09:39:31.227 INFO:teuthology.orchestra.run.smithi078.stdout: debug bluefs = 1/20 2023-10-11T09:39:31.227 INFO:teuthology.orchestra.run.smithi078.stdout: debug bluestore = 1/20 2023-10-11T09:39:31.227 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:39:31.227 INFO:teuthology.orchestra.run.smithi078.stdout: debug osd = 20 2023-10-11T09:39:31.227 INFO:teuthology.orchestra.run.smithi078.stdout: debug rocksdb = 4/10 2023-10-11T09:39:31.227 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd backfillfull_ratio = 0.85 2023-10-11T09:39:31.228 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd full ratio = 0.9 2023-10-11T09:39:31.228 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd nearfull ratio = 0.8 2023-10-11T09:39:31.228 INFO:teuthology.orchestra.run.smithi078.stdout: osd failsafe full ratio = 0.95 2023-10-11T09:39:31.228 INFO:teuthology.orchestra.run.smithi078.stdout: osd objectstore = bluestore 2023-10-11T09:39:31.228 INFO:teuthology.orchestra.run.smithi078.stdout: osd op complaint time = 180 2023-10-11T09:39:31.229 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:39:31.229 INFO:teuthology.orchestra.run.smithi078.stdout:[mgr] 2023-10-11T09:39:31.229 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:39:31.229 INFO:teuthology.orchestra.run.smithi078.stdout: debug mgr = 20 2023-10-11T09:39:31.229 INFO:teuthology.orchestra.run.smithi078.stdout: debug mon = 20 2023-10-11T09:39:31.230 INFO:teuthology.orchestra.run.smithi078.stdout: debug auth = 20 2023-10-11T09:39:31.230 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:39:31.230 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:39:31.230 INFO:teuthology.orchestra.run.smithi078.stdout: mgr/telemetry/nag = false 2023-10-11T09:39:31.230 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:39:31.231 INFO:teuthology.orchestra.run.smithi078.stdout:[mon] 2023-10-11T09:39:31.231 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:39:31.231 INFO:teuthology.orchestra.run.smithi078.stdout: debug mon = 20 2023-10-11T09:39:31.231 INFO:teuthology.orchestra.run.smithi078.stdout: debug paxos = 20 2023-10-11T09:39:31.231 INFO:teuthology.orchestra.run.smithi078.stdout: debug auth = 20 2023-10-11T09:39:31.231 INFO:teuthology.orchestra.run.smithi078.stdout: mon data avail warn = 5 2023-10-11T09:39:31.232 INFO:teuthology.orchestra.run.smithi078.stdout: mon mgr mkfs grace = 240 2023-10-11T09:39:31.232 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:39:31.232 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd reporter subtree level = osd 2023-10-11T09:39:31.232 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd prime pg temp = true 2023-10-11T09:39:31.232 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:39:31.233 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:39:31.233 INFO:teuthology.orchestra.run.smithi078.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-11T09:39:31.233 INFO:teuthology.orchestra.run.smithi078.stdout: auth mon ticket ttl = 660 # 11m 2023-10-11T09:39:31.233 INFO:teuthology.orchestra.run.smithi078.stdout: auth service ticket ttl = 240 # 4m 2023-10-11T09:39:31.233 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:39:31.233 INFO:teuthology.orchestra.run.smithi078.stdout: # don't complain about insecure global_id in the test suite 2023-10-11T09:39:31.234 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-11T09:39:31.234 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-11T09:39:31.234 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:39:31.234 INFO:teuthology.orchestra.run.smithi078.stdout: # 1m isn't quite enough 2023-10-11T09:39:31.234 INFO:teuthology.orchestra.run.smithi078.stdout: mon_down_mkfs_grace = 2m 2023-10-11T09:39:31.235 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:39:31.235 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_filestore_osds = false 2023-10-11T09:39:31.235 INFO:teuthology.orchestra.run.smithi078.stdout: mon op complaint time = 120 2023-10-11T09:39:31.235 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:39:31.235 INFO:teuthology.orchestra.run.smithi078.stdout:[client] 2023-10-11T09:39:31.236 INFO:teuthology.orchestra.run.smithi078.stdout: rgw cache enabled = true 2023-10-11T09:39:31.236 INFO:teuthology.orchestra.run.smithi078.stdout: rgw enable ops log = true 2023-10-11T09:39:31.236 INFO:teuthology.orchestra.run.smithi078.stdout: rgw enable usage log = true 2023-10-11T09:39:31.236 INFO:teuthology.orchestra.run.smithi078.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-11T09:39:31.236 INFO:teuthology.orchestra.run.smithi078.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-11T09:39:31.237 INFO:teuthology.orchestra.run.smithi078.stdout: client mount timeout = 600 2023-10-11T09:39:31.237 INFO:teuthology.orchestra.run.smithi078.stdout: debug client = 20 2023-10-11T09:39:31.237 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:39:31.237 INFO:teuthology.orchestra.run.smithi078.stdout: pid file = "" 2023-10-11T09:39:31.237 INFO:teuthology.orchestra.run.smithi078.stdout: rados mon op timeout = 900 2023-10-11T09:39:31.238 INFO:teuthology.orchestra.run.smithi078.stdout: rados osd op timeout = 900 2023-10-11T09:39:31.238 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.a] 2023-10-11T09:39:31.238 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.b] 2023-10-11T09:39:31.238 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.c] 2023-10-11T09:39:31.238 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.a] 2023-10-11T09:39:31.239 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.c] 2023-10-11T09:39:31.239 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.b] 2023-10-11T09:39:31.239 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.d] 2023-10-11T09:39:31.239 INFO:teuthology.orchestra.run.smithi078.stdout:[mds] 2023-10-11T09:39:31.239 INFO:teuthology.orchestra.run.smithi078.stdout: debug mds = 20 2023-10-11T09:39:31.239 INFO:teuthology.orchestra.run.smithi078.stdout: debug mds balancer = 20 2023-10-11T09:39:31.240 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:39:31.240 INFO:teuthology.orchestra.run.smithi078.stdout: mds debug frag = True 2023-10-11T09:39:31.240 INFO:teuthology.orchestra.run.smithi078.stdout: mds debug scatterstat = True 2023-10-11T09:39:31.240 INFO:teuthology.orchestra.run.smithi078.stdout: mds op complaint time = 180 2023-10-11T09:39:31.240 INFO:teuthology.orchestra.run.smithi078.stdout: mds verify scatter = True 2023-10-11T09:39:31.241 INFO:teuthology.orchestra.run.smithi078.stdout: osd op complaint time = 180 2023-10-11T09:39:31.241 INFO:teuthology.orchestra.run.smithi078.stdout: rados mon op timeout = 900 2023-10-11T09:39:31.241 INFO:teuthology.orchestra.run.smithi078.stdout: rados osd op timeout = 900 2023-10-11T09:39:31.242 INFO:teuthology.orchestra.run.smithi133.stdout:[global] 2023-10-11T09:39:31.242 INFO:teuthology.orchestra.run.smithi133.stdout: chdir = "" 2023-10-11T09:39:31.242 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-11T09:39:31.242 INFO:teuthology.orchestra.run.smithi133.stdout: auth supported = cephx 2023-10-11T09:39:31.243 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:39:31.243 INFO:teuthology.orchestra.run.smithi133.stdout: filestore xattr use omap = true 2023-10-11T09:39:31.243 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:39:31.243 INFO:teuthology.orchestra.run.smithi133.stdout: mon clock drift allowed = 1.000 2023-10-11T09:39:31.243 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:39:31.244 INFO:teuthology.orchestra.run.smithi133.stdout: osd crush chooseleaf type = 0 2023-10-11T09:39:31.244 INFO:teuthology.orchestra.run.smithi133.stdout: auth debug = true 2023-10-11T09:39:31.244 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:39:31.244 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on old message = true 2023-10-11T09:39:31.244 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on bug = true 2023-10-11T09:39:31.245 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:39:31.245 INFO:teuthology.orchestra.run.smithi133.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-11T09:39:31.245 INFO:teuthology.orchestra.run.smithi133.stdout: mon pg warn max object skew = 0 2023-10-11T09:39:31.245 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:39:31.245 INFO:teuthology.orchestra.run.smithi133.stdout: # disable pg_autoscaler by default for new pools 2023-10-11T09:39:31.245 INFO:teuthology.orchestra.run.smithi133.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-11T09:39:31.246 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:39:31.246 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default size = 2 2023-10-11T09:39:31.246 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:39:31.246 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow primary affinity = true 2023-10-11T09:39:31.246 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow pg remap = true 2023-10-11T09:39:31.247 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on legacy crush tunables = false 2023-10-11T09:39:31.247 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on crush straw calc version zero = false 2023-10-11T09:39:31.247 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on no sortbitwise = false 2023-10-11T09:39:31.247 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on osd down out interval zero = false 2023-10-11T09:39:31.247 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on too few osds = false 2023-10-11T09:39:31.248 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-11T09:39:31.248 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-11T09:39:31.248 INFO:teuthology.orchestra.run.smithi133.stdout: mon_allow_pool_size_one = true 2023-10-11T09:39:31.248 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:39:31.248 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-11T09:39:31.249 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:39:31.249 INFO:teuthology.orchestra.run.smithi133.stdout: osd default data pool replay window = 5 2023-10-11T09:39:31.249 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:39:31.249 INFO:teuthology.orchestra.run.smithi133.stdout: mon allow pool delete = true 2023-10-11T09:39:31.249 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:39:31.250 INFO:teuthology.orchestra.run.smithi133.stdout: mon cluster log file level = debug 2023-10-11T09:39:31.250 INFO:teuthology.orchestra.run.smithi133.stdout: debug asserts on shutdown = true 2023-10-11T09:39:31.250 INFO:teuthology.orchestra.run.smithi133.stdout: mon health detail to clog = false 2023-10-11T09:39:31.250 INFO:teuthology.orchestra.run.smithi133.stdout: mon host = "172.21.15.78,172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790]" 2023-10-11T09:39:31.250 INFO:teuthology.orchestra.run.smithi133.stdout: fsid = dee99db9-7a72-406f-a1e2-917943f2b13d 2023-10-11T09:39:31.251 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:39:31.251 INFO:teuthology.orchestra.run.smithi133.stdout:[osd] 2023-10-11T09:39:31.251 INFO:teuthology.orchestra.run.smithi133.stdout: osd journal size = 100 2023-10-11T09:39:31.251 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:39:31.251 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub load threshold = 5.0 2023-10-11T09:39:31.251 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub max interval = 600 2023-10-11T09:39:31.252 INFO:teuthology.orchestra.run.smithi133.stdout: osd mclock profile = high_recovery_ops 2023-10-11T09:39:31.252 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:39:31.252 INFO:teuthology.orchestra.run.smithi133.stdout: osd recover clone overlap = true 2023-10-11T09:39:31.252 INFO:teuthology.orchestra.run.smithi133.stdout: osd recovery max chunk = 1048576 2023-10-11T09:39:31.252 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:39:31.253 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug shutdown = true 2023-10-11T09:39:31.253 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug op order = true 2023-10-11T09:39:31.253 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug verify stray on activate = true 2023-10-11T09:39:31.253 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:39:31.253 INFO:teuthology.orchestra.run.smithi133.stdout: osd open classes on start = true 2023-10-11T09:39:31.254 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug pg log writeout = true 2023-10-11T09:39:31.254 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:39:31.254 INFO:teuthology.orchestra.run.smithi133.stdout: osd deep scrub update digest min age = 30 2023-10-11T09:39:31.254 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:39:31.254 INFO:teuthology.orchestra.run.smithi133.stdout: osd map max advance = 10 2023-10-11T09:39:31.255 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:39:31.255 INFO:teuthology.orchestra.run.smithi133.stdout: journal zero on create = true 2023-10-11T09:39:31.255 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:39:31.255 INFO:teuthology.orchestra.run.smithi133.stdout: filestore ondisk finisher threads = 3 2023-10-11T09:39:31.255 INFO:teuthology.orchestra.run.smithi133.stdout: filestore apply finisher threads = 3 2023-10-11T09:39:31.255 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:39:31.256 INFO:teuthology.orchestra.run.smithi133.stdout: bdev debug aio = true 2023-10-11T09:39:31.256 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug misdirected ops = true 2023-10-11T09:39:31.256 INFO:teuthology.orchestra.run.smithi133.stdout: bdev async discard = True 2023-10-11T09:39:31.256 INFO:teuthology.orchestra.run.smithi133.stdout: bdev enable discard = True 2023-10-11T09:39:31.256 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore allocator = bitmap 2023-10-11T09:39:31.256 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore block size = 96636764160 2023-10-11T09:39:31.257 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore fsck on mount = True 2023-10-11T09:39:31.257 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluefs = 1/20 2023-10-11T09:39:31.257 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluestore = 1/20 2023-10-11T09:39:31.257 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:39:31.257 INFO:teuthology.orchestra.run.smithi133.stdout: debug osd = 20 2023-10-11T09:39:31.257 INFO:teuthology.orchestra.run.smithi133.stdout: debug rocksdb = 4/10 2023-10-11T09:39:31.258 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd backfillfull_ratio = 0.85 2023-10-11T09:39:31.258 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd full ratio = 0.9 2023-10-11T09:39:31.258 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd nearfull ratio = 0.8 2023-10-11T09:39:31.258 INFO:teuthology.orchestra.run.smithi133.stdout: osd failsafe full ratio = 0.95 2023-10-11T09:39:31.258 INFO:teuthology.orchestra.run.smithi133.stdout: osd objectstore = bluestore 2023-10-11T09:39:31.259 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2023-10-11T09:39:31.259 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:39:31.259 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr] 2023-10-11T09:39:31.259 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:39:31.259 INFO:teuthology.orchestra.run.smithi133.stdout: debug mgr = 20 2023-10-11T09:39:31.260 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2023-10-11T09:39:31.260 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2023-10-11T09:39:31.260 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:39:31.260 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:39:31.260 INFO:teuthology.orchestra.run.smithi133.stdout: mgr/telemetry/nag = false 2023-10-11T09:39:31.260 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:39:31.261 INFO:teuthology.orchestra.run.smithi133.stdout:[mon] 2023-10-11T09:39:31.261 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:39:31.261 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2023-10-11T09:39:31.261 INFO:teuthology.orchestra.run.smithi133.stdout: debug paxos = 20 2023-10-11T09:39:31.261 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2023-10-11T09:39:31.261 INFO:teuthology.orchestra.run.smithi133.stdout: mon data avail warn = 5 2023-10-11T09:39:31.262 INFO:teuthology.orchestra.run.smithi133.stdout: mon mgr mkfs grace = 240 2023-10-11T09:39:31.262 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:39:31.262 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd reporter subtree level = osd 2023-10-11T09:39:31.262 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd prime pg temp = true 2023-10-11T09:39:31.262 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:39:31.263 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:39:31.263 INFO:teuthology.orchestra.run.smithi133.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-11T09:39:31.263 INFO:teuthology.orchestra.run.smithi133.stdout: auth mon ticket ttl = 660 # 11m 2023-10-11T09:39:31.263 INFO:teuthology.orchestra.run.smithi133.stdout: auth service ticket ttl = 240 # 4m 2023-10-11T09:39:31.263 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:39:31.264 INFO:teuthology.orchestra.run.smithi133.stdout: # don't complain about insecure global_id in the test suite 2023-10-11T09:39:31.264 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-11T09:39:31.264 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-11T09:39:31.264 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:39:31.264 INFO:teuthology.orchestra.run.smithi133.stdout: # 1m isn't quite enough 2023-10-11T09:39:31.264 INFO:teuthology.orchestra.run.smithi133.stdout: mon_down_mkfs_grace = 2m 2023-10-11T09:39:31.265 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:39:31.265 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_filestore_osds = false 2023-10-11T09:39:31.265 INFO:teuthology.orchestra.run.smithi133.stdout: mon op complaint time = 120 2023-10-11T09:39:31.265 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:39:31.265 INFO:teuthology.orchestra.run.smithi133.stdout:[client] 2023-10-11T09:39:31.265 INFO:teuthology.orchestra.run.smithi133.stdout: rgw cache enabled = true 2023-10-11T09:39:31.266 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable ops log = true 2023-10-11T09:39:31.266 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable usage log = true 2023-10-11T09:39:31.266 INFO:teuthology.orchestra.run.smithi133.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-11T09:39:31.266 INFO:teuthology.orchestra.run.smithi133.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-11T09:39:31.266 INFO:teuthology.orchestra.run.smithi133.stdout: client mount timeout = 600 2023-10-11T09:39:31.267 INFO:teuthology.orchestra.run.smithi133.stdout: debug client = 20 2023-10-11T09:39:31.267 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:39:31.267 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = "" 2023-10-11T09:39:31.267 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2023-10-11T09:39:31.267 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2023-10-11T09:39:31.268 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.a] 2023-10-11T09:39:31.268 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.b] 2023-10-11T09:39:31.268 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.c] 2023-10-11T09:39:31.268 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.a] 2023-10-11T09:39:31.268 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.c] 2023-10-11T09:39:31.268 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.b] 2023-10-11T09:39:31.269 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.d] 2023-10-11T09:39:31.269 INFO:teuthology.orchestra.run.smithi133.stdout:[mds] 2023-10-11T09:39:31.269 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds = 20 2023-10-11T09:39:31.269 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds balancer = 20 2023-10-11T09:39:31.269 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:39:31.270 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug frag = True 2023-10-11T09:39:31.270 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug scatterstat = True 2023-10-11T09:39:31.270 INFO:teuthology.orchestra.run.smithi133.stdout: mds op complaint time = 180 2023-10-11T09:39:31.270 INFO:teuthology.orchestra.run.smithi133.stdout: mds verify scatter = True 2023-10-11T09:39:31.270 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2023-10-11T09:39:31.271 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2023-10-11T09:39:31.271 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2023-10-11T09:39:31.272 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-11T09:39:31.273 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-11T09:39:31.362 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-11T09:39:31.362 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:39:31.363 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-11T09:39:31.378 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:39:31.379 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-11T09:39:31.443 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi078.front.sepia.ceph.com 2023-10-11T09:39:31.444 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:39:31.444 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-11T09:39:31.444 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-11T09:39:31.549 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:39:31.550 DEBUG:teuthology.orchestra.run.smithi078:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-11T09:39:31.609 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi133.front.sepia.ceph.com 2023-10-11T09:39:31.609 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:39:31.610 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-11T09:39:31.610 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-11T09:39:31.669 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:39:31.670 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-11T09:39:31.729 INFO:tasks.ceph:Setting up mon nodes... 2023-10-11T09:39:31.730 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-11T09:39:31.730 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-11T09:39:31.827 INFO:teuthology.orchestra.run.smithi078.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-11T09:39:31.835 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-11T09:39:31.937 INFO:teuthology.orchestra.run.smithi133.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-11T09:39:31.954 INFO:tasks.ceph:Setting up mds nodes... 2023-10-11T09:39:31.954 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-11T09:39:32.054 INFO:teuthology.orchestra.run.smithi078.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-11T09:39:32.064 DEBUG:teuthology.orchestra.run.smithi078:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-11T09:39:32.098 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-11T09:39:32.241 INFO:teuthology.orchestra.run.smithi078.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-11T09:39:32.247 DEBUG:teuthology.orchestra.run.smithi078:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-11T09:39:32.282 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-11T09:39:32.384 INFO:teuthology.orchestra.run.smithi133.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-11T09:39:32.391 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-11T09:39:32.426 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-11T09:39:32.570 INFO:teuthology.orchestra.run.smithi133.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-11T09:39:32.576 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-11T09:39:32.607 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-11T09:39:32.608 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-11T09:39:32.681 INFO:teuthology.orchestra.run.smithi078.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-11T09:39:32.709 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-11T09:39:32.780 INFO:teuthology.orchestra.run.smithi078.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-11T09:39:32.807 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-11T09:39:32.885 INFO:teuthology.orchestra.run.smithi133.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-11T09:39:32.912 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-11T09:39:32.987 INFO:teuthology.orchestra.run.smithi133.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-11T09:39:33.012 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-11T09:39:33.012 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi078.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi133.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-11T09:39:33.013 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-11T09:39:33.047 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-11T09:39:33.048 INFO:tasks.ceph:role: osd.0 2023-10-11T09:39:33.048 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi078.front.sepia.ceph.com 2023-10-11T09:39:33.048 DEBUG:teuthology.orchestra.run.smithi078:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-11T09:39:33.615 INFO:teuthology.orchestra.run.smithi078.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-11T09:39:33.616 INFO:teuthology.orchestra.run.smithi078.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-11T09:39:33.616 INFO:teuthology.orchestra.run.smithi078.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-11T09:39:33.616 INFO:teuthology.orchestra.run.smithi078.stdout: = reflink=1 2023-10-11T09:39:33.616 INFO:teuthology.orchestra.run.smithi078.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-11T09:39:33.617 INFO:teuthology.orchestra.run.smithi078.stdout: = sunit=0 swidth=0 blks 2023-10-11T09:39:33.617 INFO:teuthology.orchestra.run.smithi078.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-11T09:39:33.617 INFO:teuthology.orchestra.run.smithi078.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-11T09:39:33.617 INFO:teuthology.orchestra.run.smithi078.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-11T09:39:33.617 INFO:teuthology.orchestra.run.smithi078.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-11T09:39:33.620 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi078.front.sepia.ceph.com -o noatime 2023-10-11T09:39:33.621 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-11T09:39:33.718 DEBUG:teuthology.orchestra.run.smithi078:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-11T09:39:33.739 INFO:teuthology.orchestra.run.smithi078.stderr:sudo: /sbin/restorecon: command not found 2023-10-11T09:39:33.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T09:39:33.740 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-11T09:39:33.816 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-11T09:39:33.817 INFO:tasks.ceph:role: osd.1 2023-10-11T09:39:33.817 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi078.front.sepia.ceph.com 2023-10-11T09:39:33.817 DEBUG:teuthology.orchestra.run.smithi078:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-11T09:39:34.349 INFO:teuthology.orchestra.run.smithi078.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-11T09:39:34.349 INFO:teuthology.orchestra.run.smithi078.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-11T09:39:34.350 INFO:teuthology.orchestra.run.smithi078.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-11T09:39:34.350 INFO:teuthology.orchestra.run.smithi078.stdout: = reflink=1 2023-10-11T09:39:34.350 INFO:teuthology.orchestra.run.smithi078.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-11T09:39:34.350 INFO:teuthology.orchestra.run.smithi078.stdout: = sunit=0 swidth=0 blks 2023-10-11T09:39:34.351 INFO:teuthology.orchestra.run.smithi078.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-11T09:39:34.351 INFO:teuthology.orchestra.run.smithi078.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-11T09:39:34.351 INFO:teuthology.orchestra.run.smithi078.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-11T09:39:34.351 INFO:teuthology.orchestra.run.smithi078.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-11T09:39:34.354 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi078.front.sepia.ceph.com -o noatime 2023-10-11T09:39:34.354 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-11T09:39:34.403 DEBUG:teuthology.orchestra.run.smithi078:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-11T09:39:34.469 INFO:teuthology.orchestra.run.smithi078.stderr:sudo: /sbin/restorecon: command not found 2023-10-11T09:39:34.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T09:39:34.471 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-11T09:39:34.548 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-11T09:39:34.548 INFO:tasks.ceph:role: osd.2 2023-10-11T09:39:34.549 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi078.front.sepia.ceph.com 2023-10-11T09:39:34.549 DEBUG:teuthology.orchestra.run.smithi078:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-11T09:39:35.144 INFO:teuthology.orchestra.run.smithi078.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-11T09:39:35.144 INFO:teuthology.orchestra.run.smithi078.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-11T09:39:35.144 INFO:teuthology.orchestra.run.smithi078.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-11T09:39:35.145 INFO:teuthology.orchestra.run.smithi078.stdout: = reflink=1 2023-10-11T09:39:35.145 INFO:teuthology.orchestra.run.smithi078.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-11T09:39:35.145 INFO:teuthology.orchestra.run.smithi078.stdout: = sunit=0 swidth=0 blks 2023-10-11T09:39:35.145 INFO:teuthology.orchestra.run.smithi078.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-11T09:39:35.145 INFO:teuthology.orchestra.run.smithi078.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-11T09:39:35.146 INFO:teuthology.orchestra.run.smithi078.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-11T09:39:35.146 INFO:teuthology.orchestra.run.smithi078.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-11T09:39:35.149 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi078.front.sepia.ceph.com -o noatime 2023-10-11T09:39:35.150 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-11T09:39:35.248 DEBUG:teuthology.orchestra.run.smithi078:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-11T09:39:35.270 INFO:teuthology.orchestra.run.smithi078.stderr:sudo: /sbin/restorecon: command not found 2023-10-11T09:39:35.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T09:39:35.271 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-11T09:39:35.344 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-11T09:39:35.344 INFO:tasks.ceph:role: osd.3 2023-10-11T09:39:35.345 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi078.front.sepia.ceph.com 2023-10-11T09:39:35.345 DEBUG:teuthology.orchestra.run.smithi078:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-11T09:39:35.900 INFO:teuthology.orchestra.run.smithi078.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-11T09:39:35.901 INFO:teuthology.orchestra.run.smithi078.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-11T09:39:35.901 INFO:teuthology.orchestra.run.smithi078.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-11T09:39:35.901 INFO:teuthology.orchestra.run.smithi078.stdout: = reflink=1 2023-10-11T09:39:35.901 INFO:teuthology.orchestra.run.smithi078.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-11T09:39:35.901 INFO:teuthology.orchestra.run.smithi078.stdout: = sunit=0 swidth=0 blks 2023-10-11T09:39:35.902 INFO:teuthology.orchestra.run.smithi078.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-11T09:39:35.902 INFO:teuthology.orchestra.run.smithi078.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-11T09:39:35.902 INFO:teuthology.orchestra.run.smithi078.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-11T09:39:35.902 INFO:teuthology.orchestra.run.smithi078.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-11T09:39:35.905 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi078.front.sepia.ceph.com -o noatime 2023-10-11T09:39:35.905 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-11T09:39:36.216 DEBUG:teuthology.orchestra.run.smithi078:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-11T09:39:36.233 INFO:teuthology.orchestra.run.smithi078.stderr:sudo: /sbin/restorecon: command not found 2023-10-11T09:39:36.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T09:39:36.235 DEBUG:teuthology.orchestra.run.smithi078:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-11T09:39:36.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:39:36.394+0000 7f7110901700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-11T09:39:36.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:39:36.402+0000 7f7110901700 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-11T09:39:36.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:39:36.402+0000 7f7110901700 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-11T09:39:36.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:39:36.402+0000 7f7110901700 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-11T09:39:36.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:39:36.402+0000 7f7110901700 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-11T09:39:36.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:39:36.410+0000 7f7110901700 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-11T09:39:39.493 DEBUG:teuthology.orchestra.run.smithi078:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-11T09:39:39.529 DEBUG:teuthology.orchestra.run.smithi078:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-11T09:39:39.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:39:39.650+0000 7f54124fd700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-11T09:39:39.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:39:39.650+0000 7f54124fd700 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-11T09:39:39.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:39:39.654+0000 7f54124fd700 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-11T09:39:39.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:39:39.654+0000 7f54124fd700 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-11T09:39:39.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:39:39.654+0000 7f54124fd700 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-11T09:39:39.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:39:39.654+0000 7f54124fd700 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-11T09:39:42.737 DEBUG:teuthology.orchestra.run.smithi078:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-11T09:39:42.770 DEBUG:teuthology.orchestra.run.smithi078:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-11T09:39:42.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:39:42.890+0000 7f12a3876700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-11T09:39:42.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:39:42.890+0000 7f12a3876700 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-11T09:39:42.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:39:42.890+0000 7f12a3876700 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-11T09:39:42.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:39:42.890+0000 7f12a3876700 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-11T09:39:42.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:39:42.890+0000 7f12a3876700 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-11T09:39:42.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:39:42.894+0000 7f12a3876700 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-11T09:39:45.967 DEBUG:teuthology.orchestra.run.smithi078:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-11T09:39:46.002 DEBUG:teuthology.orchestra.run.smithi078:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-11T09:39:46.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:39:46.122+0000 7fbedc81c700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-11T09:39:46.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:39:46.126+0000 7fbedc81c700 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-11T09:39:46.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:39:46.126+0000 7fbedc81c700 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-11T09:39:46.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:39:46.126+0000 7fbedc81c700 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-11T09:39:46.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:39:46.126+0000 7fbedc81c700 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-11T09:39:46.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:39:46.130+0000 7fbedc81c700 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-11T09:39:49.217 DEBUG:teuthology.orchestra.run.smithi078:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-11T09:39:49.253 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-11T09:39:49.289 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-11T09:39:49.290 INFO:tasks.ceph:role: osd.4 2023-10-11T09:39:49.290 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi133.front.sepia.ceph.com 2023-10-11T09:39:49.290 DEBUG:teuthology.orchestra.run.smithi133:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-11T09:39:49.895 INFO:teuthology.orchestra.run.smithi133.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-11T09:39:49.895 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-11T09:39:49.896 INFO:teuthology.orchestra.run.smithi133.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-11T09:39:49.897 INFO:teuthology.orchestra.run.smithi133.stdout: = reflink=1 2023-10-11T09:39:49.897 INFO:teuthology.orchestra.run.smithi133.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-11T09:39:49.897 INFO:teuthology.orchestra.run.smithi133.stdout: = sunit=0 swidth=0 blks 2023-10-11T09:39:49.898 INFO:teuthology.orchestra.run.smithi133.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-11T09:39:49.898 INFO:teuthology.orchestra.run.smithi133.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-11T09:39:49.898 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-11T09:39:49.898 INFO:teuthology.orchestra.run.smithi133.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-11T09:39:49.900 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi133.front.sepia.ceph.com -o noatime 2023-10-11T09:39:49.900 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-11T09:39:50.013 DEBUG:teuthology.orchestra.run.smithi133:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-11T09:39:50.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T09:39:50.036 INFO:teuthology.orchestra.run.smithi133.stderr:sudo: /sbin/restorecon: command not found 2023-10-11T09:39:50.037 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-11T09:39:50.119 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-11T09:39:50.120 INFO:tasks.ceph:role: osd.5 2023-10-11T09:39:50.120 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi133.front.sepia.ceph.com 2023-10-11T09:39:50.121 DEBUG:teuthology.orchestra.run.smithi133:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-11T09:39:50.638 INFO:teuthology.orchestra.run.smithi133.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-11T09:39:50.638 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-11T09:39:50.638 INFO:teuthology.orchestra.run.smithi133.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-11T09:39:50.639 INFO:teuthology.orchestra.run.smithi133.stdout: = reflink=1 2023-10-11T09:39:50.639 INFO:teuthology.orchestra.run.smithi133.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-11T09:39:50.639 INFO:teuthology.orchestra.run.smithi133.stdout: = sunit=0 swidth=0 blks 2023-10-11T09:39:50.639 INFO:teuthology.orchestra.run.smithi133.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-11T09:39:50.639 INFO:teuthology.orchestra.run.smithi133.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-11T09:39:50.640 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-11T09:39:50.640 INFO:teuthology.orchestra.run.smithi133.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-11T09:39:50.642 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi133.front.sepia.ceph.com -o noatime 2023-10-11T09:39:50.643 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-11T09:39:50.715 DEBUG:teuthology.orchestra.run.smithi133:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-11T09:39:50.737 INFO:teuthology.orchestra.run.smithi133.stderr:sudo: /sbin/restorecon: command not found 2023-10-11T09:39:50.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T09:39:50.738 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-11T09:39:50.812 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-11T09:39:50.812 INFO:tasks.ceph:role: osd.6 2023-10-11T09:39:50.813 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi133.front.sepia.ceph.com 2023-10-11T09:39:50.813 DEBUG:teuthology.orchestra.run.smithi133:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-11T09:39:51.377 INFO:teuthology.orchestra.run.smithi133.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-11T09:39:51.377 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-11T09:39:51.378 INFO:teuthology.orchestra.run.smithi133.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-11T09:39:51.378 INFO:teuthology.orchestra.run.smithi133.stdout: = reflink=1 2023-10-11T09:39:51.378 INFO:teuthology.orchestra.run.smithi133.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-11T09:39:51.378 INFO:teuthology.orchestra.run.smithi133.stdout: = sunit=0 swidth=0 blks 2023-10-11T09:39:51.379 INFO:teuthology.orchestra.run.smithi133.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-11T09:39:51.379 INFO:teuthology.orchestra.run.smithi133.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-11T09:39:51.379 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-11T09:39:51.379 INFO:teuthology.orchestra.run.smithi133.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-11T09:39:51.382 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi133.front.sepia.ceph.com -o noatime 2023-10-11T09:39:51.383 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-11T09:39:51.500 DEBUG:teuthology.orchestra.run.smithi133:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-11T09:39:51.521 INFO:teuthology.orchestra.run.smithi133.stderr:sudo: /sbin/restorecon: command not found 2023-10-11T09:39:51.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T09:39:51.522 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-11T09:39:51.596 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-11T09:39:51.597 INFO:tasks.ceph:role: osd.7 2023-10-11T09:39:51.597 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi133.front.sepia.ceph.com 2023-10-11T09:39:51.597 DEBUG:teuthology.orchestra.run.smithi133:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-11T09:39:52.162 INFO:teuthology.orchestra.run.smithi133.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-11T09:39:52.163 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-11T09:39:52.163 INFO:teuthology.orchestra.run.smithi133.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-11T09:39:52.163 INFO:teuthology.orchestra.run.smithi133.stdout: = reflink=1 2023-10-11T09:39:52.163 INFO:teuthology.orchestra.run.smithi133.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-11T09:39:52.164 INFO:teuthology.orchestra.run.smithi133.stdout: = sunit=0 swidth=0 blks 2023-10-11T09:39:52.164 INFO:teuthology.orchestra.run.smithi133.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-11T09:39:52.164 INFO:teuthology.orchestra.run.smithi133.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-11T09:39:52.164 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-11T09:39:52.164 INFO:teuthology.orchestra.run.smithi133.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-11T09:39:52.167 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi133.front.sepia.ceph.com -o noatime 2023-10-11T09:39:52.168 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-11T09:39:52.272 DEBUG:teuthology.orchestra.run.smithi133:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-11T09:39:52.292 INFO:teuthology.orchestra.run.smithi133.stderr:sudo: /sbin/restorecon: command not found 2023-10-11T09:39:52.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T09:39:52.294 DEBUG:teuthology.orchestra.run.smithi133:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-11T09:39:52.453 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:39:52.449+0000 7f5deb58c700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-11T09:39:52.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:39:52.449+0000 7f5deb58c700 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-11T09:39:52.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:39:52.449+0000 7f5deb58c700 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-11T09:39:52.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:39:52.449+0000 7f5deb58c700 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-11T09:39:52.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:39:52.449+0000 7f5deb58c700 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-11T09:39:52.457 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:39:52.453+0000 7f5deb58c700 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-11T09:39:55.546 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-11T09:39:55.579 DEBUG:teuthology.orchestra.run.smithi133:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-11T09:39:55.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:39:55.708+0000 7f865ffec700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-11T09:39:55.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:39:55.708+0000 7f865ffec700 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-11T09:39:55.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:39:55.708+0000 7f865ffec700 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-11T09:39:55.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:39:55.708+0000 7f865ffec700 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-11T09:39:55.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:39:55.708+0000 7f865ffec700 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-11T09:39:55.717 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:39:55.712+0000 7f865ffec700 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-11T09:39:58.769 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-11T09:39:58.802 DEBUG:teuthology.orchestra.run.smithi133:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-11T09:39:58.865 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:39:58.872 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:39:58.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:39:58.920+0000 7f2524a76700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-11T09:39:58.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:39:58.920+0000 7f2524a76700 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-11T09:39:58.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:39:58.920+0000 7f2524a76700 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-11T09:39:58.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:39:58.920+0000 7f2524a76700 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-11T09:39:58.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:39:58.920+0000 7f2524a76700 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-11T09:39:58.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:39:58.924+0000 7f2524a76700 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-11T09:40:01.997 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-11T09:40:02.031 DEBUG:teuthology.orchestra.run.smithi133:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-11T09:40:02.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:02.160+0000 7f27fe1b1700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-11T09:40:02.167 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:02.160+0000 7f27fe1b1700 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-11T09:40:02.168 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:02.164+0000 7f27fe1b1700 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-11T09:40:02.168 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:02.164+0000 7f27fe1b1700 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-11T09:40:02.168 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:02.164+0000 7f27fe1b1700 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-11T09:40:02.169 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:02.164+0000 7f27fe1b1700 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-11T09:40:05.261 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-11T09:40:05.296 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-11T09:40:05.296 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:40:05.297 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-11T09:40:05.332 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:40:05.332 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-11T09:40:05.405 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:40:05.406 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-11T09:40:05.482 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:40:05.482 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-11T09:40:05.558 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:40:05.558 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-11T09:40:05.634 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:40:05.635 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-11T09:40:05.710 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:40:05.710 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-11T09:40:05.786 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:40:05.786 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-11T09:40:05.820 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:40:05.821 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-11T09:40:05.895 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:40:05.896 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-11T09:40:05.979 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:40:05.979 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-11T09:40:06.056 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:40:06.056 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-11T09:40:06.133 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:40:06.133 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-11T09:40:06.206 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:40:06.207 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-11T09:40:06.280 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:40:06.281 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-11T09:40:06.298 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:40:06.298 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-11T09:40:06.359 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:40:06.360 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-11T09:40:06.378 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:40:06.378 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-11T09:40:06.441 INFO:tasks.ceph:Adding keys to all mons... 2023-10-11T09:40:06.441 DEBUG:teuthology.orchestra.run.smithi078:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-11T09:40:06.449 DEBUG:teuthology.orchestra.run.smithi133:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-11T09:40:06.495 INFO:teuthology.orchestra.run.smithi078.stdout:[mgr.y] 2023-10-11T09:40:06.495 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQBTbSZlnv4yMRAAQh8w18k76KRjdCvSDJ4QbA== 2023-10-11T09:40:06.495 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.a] 2023-10-11T09:40:06.495 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQBUbSZlL58cAxAALe+WtkV/aLopy1w9i6OVRg== 2023-10-11T09:40:06.496 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.c] 2023-10-11T09:40:06.496 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQBUbSZl9jJBDhAAjos7/02OqF0WldFm70/fVA== 2023-10-11T09:40:06.496 INFO:teuthology.orchestra.run.smithi078.stdout:[osd.0] 2023-10-11T09:40:06.496 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQBYbSZldbrVFxAA6ogUpCcFO8LhHn4zdkudNQ== 2023-10-11T09:40:06.496 INFO:teuthology.orchestra.run.smithi078.stdout:[osd.1] 2023-10-11T09:40:06.496 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQBbbSZl+mkPJxAA4Rrt0j5lzXapYjXhdulhNQ== 2023-10-11T09:40:06.497 INFO:teuthology.orchestra.run.smithi078.stdout:[osd.2] 2023-10-11T09:40:06.497 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQBebSZlZ6lSNRAA3ltw4ZTkFNpudCPc2zTYzQ== 2023-10-11T09:40:06.497 INFO:teuthology.orchestra.run.smithi078.stdout:[osd.3] 2023-10-11T09:40:06.497 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQBibSZlsE+jBxAA2Ix2JAFTUxljpHAwPpJeUA== 2023-10-11T09:40:06.497 INFO:teuthology.orchestra.run.smithi078.stdout:[mgr.x] 2023-10-11T09:40:06.498 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQBTbSZlS/TjNxAAMkq9xRMwgsbLlT4FdobATw== 2023-10-11T09:40:06.498 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.b] 2023-10-11T09:40:06.498 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQBUbSZliPLmFhAA+6wegaD6yTUsttFzo9OrIg== 2023-10-11T09:40:06.498 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.d] 2023-10-11T09:40:06.499 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQBUbSZlTb/4IRAAeXtcbWVB5TnSoYch+uMZWw== 2023-10-11T09:40:06.499 INFO:teuthology.orchestra.run.smithi078.stdout:[osd.4] 2023-10-11T09:40:06.499 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQBobSZlII4OGxAA0Z5wGuu4wOs/14tQRnHB1Q== 2023-10-11T09:40:06.499 INFO:teuthology.orchestra.run.smithi078.stdout:[osd.5] 2023-10-11T09:40:06.499 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQBrbSZlG6aSKhAAZq179SfpkelcJJLaUcLDCg== 2023-10-11T09:40:06.499 INFO:teuthology.orchestra.run.smithi078.stdout:[osd.6] 2023-10-11T09:40:06.500 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQBubSZl3W89NxAAgUrd0/yMOWlrhAOU1BpM9A== 2023-10-11T09:40:06.500 INFO:teuthology.orchestra.run.smithi078.stdout:[osd.7] 2023-10-11T09:40:06.500 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQBybSZloIjyCRAAjmvyK9zMSEsV8bVebwzPxg== 2023-10-11T09:40:06.500 INFO:teuthology.orchestra.run.smithi078.stdout:[client.0] 2023-10-11T09:40:06.500 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQBUbSZlmGmCKBAAOI8EvyRwg8HZt+q/vUy5Ow== 2023-10-11T09:40:06.501 INFO:teuthology.orchestra.run.smithi078.stdout:[client.2] 2023-10-11T09:40:06.501 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQBUbSZlbd9lLhAA9LAjreTcxwAjhiNlKndl+Q== 2023-10-11T09:40:06.501 INFO:teuthology.orchestra.run.smithi078.stdout:[client.1] 2023-10-11T09:40:06.501 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQBUbSZlYvfHNBAAReum0efx4MmqpLMxFnTROQ== 2023-10-11T09:40:06.501 INFO:teuthology.orchestra.run.smithi078.stdout:[client.3] 2023-10-11T09:40:06.502 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQBUbSZlHUjeOhAASPKTRvsillob4gjdW0akhQ== 2023-10-11T09:40:06.518 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr.y] 2023-10-11T09:40:06.518 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQBTbSZlnv4yMRAAQh8w18k76KRjdCvSDJ4QbA== 2023-10-11T09:40:06.518 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.a] 2023-10-11T09:40:06.518 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQBUbSZlL58cAxAALe+WtkV/aLopy1w9i6OVRg== 2023-10-11T09:40:06.518 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.c] 2023-10-11T09:40:06.519 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQBUbSZl9jJBDhAAjos7/02OqF0WldFm70/fVA== 2023-10-11T09:40:06.519 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.0] 2023-10-11T09:40:06.521 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQBYbSZldbrVFxAA6ogUpCcFO8LhHn4zdkudNQ== 2023-10-11T09:40:06.521 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.1] 2023-10-11T09:40:06.521 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQBbbSZl+mkPJxAA4Rrt0j5lzXapYjXhdulhNQ== 2023-10-11T09:40:06.521 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.2] 2023-10-11T09:40:06.522 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQBebSZlZ6lSNRAA3ltw4ZTkFNpudCPc2zTYzQ== 2023-10-11T09:40:06.522 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.3] 2023-10-11T09:40:06.522 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQBibSZlsE+jBxAA2Ix2JAFTUxljpHAwPpJeUA== 2023-10-11T09:40:06.522 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr.x] 2023-10-11T09:40:06.522 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQBTbSZlS/TjNxAAMkq9xRMwgsbLlT4FdobATw== 2023-10-11T09:40:06.523 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.b] 2023-10-11T09:40:06.523 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQBUbSZliPLmFhAA+6wegaD6yTUsttFzo9OrIg== 2023-10-11T09:40:06.523 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.d] 2023-10-11T09:40:06.523 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQBUbSZlTb/4IRAAeXtcbWVB5TnSoYch+uMZWw== 2023-10-11T09:40:06.523 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.4] 2023-10-11T09:40:06.523 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQBobSZlII4OGxAA0Z5wGuu4wOs/14tQRnHB1Q== 2023-10-11T09:40:06.524 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.5] 2023-10-11T09:40:06.524 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQBrbSZlG6aSKhAAZq179SfpkelcJJLaUcLDCg== 2023-10-11T09:40:06.524 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.6] 2023-10-11T09:40:06.524 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQBubSZl3W89NxAAgUrd0/yMOWlrhAOU1BpM9A== 2023-10-11T09:40:06.524 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.7] 2023-10-11T09:40:06.525 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQBybSZloIjyCRAAjmvyK9zMSEsV8bVebwzPxg== 2023-10-11T09:40:06.525 INFO:teuthology.orchestra.run.smithi133.stdout:[client.0] 2023-10-11T09:40:06.525 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQBUbSZlmGmCKBAAOI8EvyRwg8HZt+q/vUy5Ow== 2023-10-11T09:40:06.525 INFO:teuthology.orchestra.run.smithi133.stdout:[client.2] 2023-10-11T09:40:06.525 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQBUbSZlbd9lLhAA9LAjreTcxwAjhiNlKndl+Q== 2023-10-11T09:40:06.526 INFO:teuthology.orchestra.run.smithi133.stdout:[client.1] 2023-10-11T09:40:06.526 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQBUbSZlYvfHNBAAReum0efx4MmqpLMxFnTROQ== 2023-10-11T09:40:06.526 INFO:teuthology.orchestra.run.smithi133.stdout:[client.3] 2023-10-11T09:40:06.526 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQBUbSZlHUjeOhAASPKTRvsillob4gjdW0akhQ== 2023-10-11T09:40:06.527 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-11T09:40:06.551 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-11T09:40:06.653 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-11T09:40:06.661 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-11T09:40:06.746 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-11T09:40:06.753 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-11T09:40:06.839 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-11T09:40:06.846 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-11T09:40:06.927 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-11T09:40:06.934 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-11T09:40:07.017 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-11T09:40:07.024 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-11T09:40:07.105 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-11T09:40:07.112 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-11T09:40:07.192 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-11T09:40:07.198 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-11T09:40:07.281 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-11T09:40:07.288 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-11T09:40:07.372 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-11T09:40:07.379 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-11T09:40:07.464 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-11T09:40:07.469 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-11T09:40:07.548 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-11T09:40:07.554 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-11T09:40:07.636 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-11T09:40:07.643 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-11T09:40:07.732 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-11T09:40:07.740 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-11T09:40:07.826 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-11T09:40:07.832 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-11T09:40:07.914 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-11T09:40:07.921 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-11T09:40:08.009 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-11T09:40:08.016 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-11T09:40:08.105 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-11T09:40:08.113 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-11T09:40:08.198 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-11T09:40:08.199 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-11T09:40:08.231 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-11T09:40:08.375 DEBUG:teuthology.orchestra.run.smithi078:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-11T09:40:08.408 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-11T09:40:08.444 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-11T09:40:08.600 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-11T09:40:08.635 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-11T09:40:08.713 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-11T09:40:08.860 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-11T09:40:08.891 DEBUG:teuthology.orchestra.run.smithi078:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-11T09:40:08.898 DEBUG:teuthology.orchestra.run.smithi133:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-11T09:40:08.953 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-11T09:40:08.954 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-11T09:40:08.954 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-11T09:40:08.961 INFO:tasks.ceph.mon.a:Started 2023-10-11T09:40:08.962 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-11T09:40:08.962 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-11T09:40:09.002 INFO:tasks.ceph.mon.b:Started 2023-10-11T09:40:09.002 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-11T09:40:09.003 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-11T09:40:09.007 INFO:tasks.ceph.mon.c:Started 2023-10-11T09:40:09.007 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-11T09:40:09.008 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-11T09:40:09.008 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-11T09:40:09.016 INFO:tasks.ceph.mgr.y:Started 2023-10-11T09:40:09.016 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-11T09:40:09.016 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-11T09:40:09.022 INFO:tasks.ceph.mgr.x:Started 2023-10-11T09:40:09.022 INFO:tasks.ceph:Setting crush tunables to default 2023-10-11T09:40:09.023 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --cluster ceph osd crush tunables default 2023-10-11T09:40:09.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:09.186+0000 7faa1fc71700 1 Processor -- start 2023-10-11T09:40:09.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:09.186+0000 7faa1fc71700 1 -- start start 2023-10-11T09:40:09.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:09.186+0000 7faa1fc71700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faa1806d080 0x7faa180cb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:09.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:09.186+0000 7faa1fc71700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa18165570 con 0x7faa18160cb0 2023-10-11T09:40:09.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:09.186+0000 7faa1fc71700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa18164960 con 0x7faa180b1030 2023-10-11T09:40:09.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:09.186+0000 7faa1fc71700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa18161080 con 0x7faa1806d080 2023-10-11T09:40:09.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:09.186+0000 7faa1d20c700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7faa180b1030 0x7faa1806ffb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:42332/0 (socket says 172.21.15.78:42332) 2023-10-11T09:40:09.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:09.186+0000 7faa1d20c700 1 -- 172.21.15.78:0/2518364538 learned_addr learned my addr 172.21.15.78:0/2518364538 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:09.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:09.186+0000 7faa1e20e700 1 --2- 172.21.15.78:0/2518364538 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faa1806d080 0x7faa180cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:09.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:09.190+0000 7faa1e20e700 1 -- 172.21.15.78:0/2518364538 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faa1806d080 msgr2=0x7faa180cb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-11T09:40:09.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:09.190+0000 7faa1e20e700 1 -- 172.21.15.78:0/2518364538 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faa1806d080 msgr2=0x7faa180cb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-11T09:40:09.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:09.190+0000 7faa1e20e700 1 --2- 172.21.15.78:0/2518364538 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faa1806d080 0x7faa180cb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-11T09:40:09.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:09.190+0000 7faa1e20e700 1 --2- 172.21.15.78:0/2518364538 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faa1806d080 0x7faa180cb100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-11T09:40:09.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:09.394+0000 7faa1e20e700 1 --2- 172.21.15.78:0/2518364538 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faa1806d080 0x7faa180cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:09.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:09.394+0000 7faa1e20e700 1 -- 172.21.15.78:0/2518364538 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faa1806d080 msgr2=0x7faa180cb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-11T09:40:09.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:09.394+0000 7faa1e20e700 1 -- 172.21.15.78:0/2518364538 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faa1806d080 msgr2=0x7faa180cb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-11T09:40:09.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:09.394+0000 7faa1e20e700 1 --2- 172.21.15.78:0/2518364538 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faa1806d080 0x7faa180cb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-11T09:40:09.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:09.394+0000 7faa1e20e700 1 --2- 172.21.15.78:0/2518364538 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faa1806d080 0x7faa180cb100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-11T09:40:09.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:09.798+0000 7faa1e20e700 1 --2- 172.21.15.78:0/2518364538 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faa1806d080 0x7faa180cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:09.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:09.798+0000 7faa1e20e700 1 -- 172.21.15.78:0/2518364538 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faa1806d080 msgr2=0x7faa180cb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-11T09:40:09.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:09.798+0000 7faa1e20e700 1 -- 172.21.15.78:0/2518364538 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faa1806d080 msgr2=0x7faa180cb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-11T09:40:09.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:09.798+0000 7faa1e20e700 1 --2- 172.21.15.78:0/2518364538 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faa1806d080 0x7faa180cb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-11T09:40:09.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:09.798+0000 7faa1e20e700 1 --2- 172.21.15.78:0/2518364538 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faa1806d080 0x7faa180cb100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-11T09:40:10.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:10.602+0000 7faa1e20e700 1 --2- 172.21.15.78:0/2518364538 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faa1806d080 0x7faa180cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:10.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:10.602+0000 7faa1e20e700 1 -- 172.21.15.78:0/2518364538 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faa1806d080 msgr2=0x7faa180cb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-11T09:40:10.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:10.602+0000 7faa1e20e700 1 -- 172.21.15.78:0/2518364538 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faa1806d080 msgr2=0x7faa180cb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-11T09:40:10.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:10.602+0000 7faa1e20e700 1 --2- 172.21.15.78:0/2518364538 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faa1806d080 0x7faa180cb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-11T09:40:10.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:10.602+0000 7faa1e20e700 1 --2- 172.21.15.78:0/2518364538 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faa1806d080 0x7faa180cb100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-11T09:40:12.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:12.186+0000 7faa0f7fe700 1 -- 172.21.15.78:0/2518364538 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faa1806d080 msgr2=0x7faa180cb100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-11T09:40:12.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:12.186+0000 7faa0f7fe700 1 --2- 172.21.15.78:0/2518364538 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faa1806d080 0x7faa180cb100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:12.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:12.186+0000 7faa0f7fe700 1 -- 172.21.15.78:0/2518364538 >> v1:172.21.15.133:6789/0 conn(0x7faa180b1030 legacy=0x7faa1806ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:12.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:12.186+0000 7faa0f7fe700 1 -- 172.21.15.78:0/2518364538 >> v1:172.21.15.78:6789/0 conn(0x7faa18160cb0 legacy=0x7faa180b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:12.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:12.186+0000 7faa0f7fe700 1 --2- 172.21.15.78:0/2518364538 >> v2:172.21.15.78:3300/0 conn(0x7fa9fc003790 0x7fa9fc005c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:12.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:12.186+0000 7faa0f7fe700 1 --2- 172.21.15.78:0/2518364538 >> v2:172.21.15.133:3300/0 conn(0x7fa9fc006230 0x7fa9fc0086e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:12.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:12.186+0000 7faa0f7fe700 1 -- 172.21.15.78:0/2518364538 --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9fc008d70 con 0x7fa9fc003790 2023-10-11T09:40:12.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:12.186+0000 7faa0f7fe700 1 -- 172.21.15.78:0/2518364538 --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9fc008ee0 con 0x7fa9fc006230 2023-10-11T09:40:12.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:12.190+0000 7faa1da0d700 1 --2- 172.21.15.78:0/2518364538 >> v2:172.21.15.78:3300/0 conn(0x7fa9fc003790 0x7fa9fc005c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:12.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:12.190+0000 7faa1d20c700 1 --2- 172.21.15.78:0/2518364538 >> v2:172.21.15.133:3300/0 conn(0x7fa9fc006230 0x7fa9fc0086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:12.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:12.190+0000 7faa1d20c700 1 -- 172.21.15.78:0/2518364538 >> v2:172.21.15.133:3300/0 conn(0x7fa9fc006230 msgr2=0x7fa9fc0086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-11T09:40:12.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:12.190+0000 7faa1d20c700 1 -- 172.21.15.78:0/2518364538 >> v2:172.21.15.133:3300/0 conn(0x7fa9fc006230 msgr2=0x7fa9fc0086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-11T09:40:12.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:12.190+0000 7faa1d20c700 1 --2- 172.21.15.78:0/2518364538 >> v2:172.21.15.133:3300/0 conn(0x7fa9fc006230 0x7fa9fc0086e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-11T09:40:12.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:12.190+0000 7faa1da0d700 1 -- 172.21.15.78:0/2518364538 >> v2:172.21.15.78:3300/0 conn(0x7fa9fc003790 msgr2=0x7fa9fc005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-11T09:40:12.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:12.190+0000 7faa1da0d700 1 -- 172.21.15.78:0/2518364538 >> v2:172.21.15.78:3300/0 conn(0x7fa9fc003790 msgr2=0x7fa9fc005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-11T09:40:12.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:12.190+0000 7faa1da0d700 1 --2- 172.21.15.78:0/2518364538 >> v2:172.21.15.78:3300/0 conn(0x7fa9fc003790 0x7fa9fc005c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-11T09:40:12.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:12.190+0000 7faa1d20c700 1 --2- 172.21.15.78:0/2518364538 >> v2:172.21.15.133:3300/0 conn(0x7fa9fc006230 0x7fa9fc0086e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-11T09:40:12.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:12.190+0000 7faa1da0d700 1 --2- 172.21.15.78:0/2518364538 >> v2:172.21.15.78:3300/0 conn(0x7fa9fc003790 0x7fa9fc005c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-11T09:40:12.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:12.394+0000 7faa1da0d700 1 --2- 172.21.15.78:0/2518364538 >> v2:172.21.15.78:3300/0 conn(0x7fa9fc003790 0x7fa9fc005c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:12.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:12.394+0000 7faa1d20c700 1 --2- 172.21.15.78:0/2518364538 >> v2:172.21.15.133:3300/0 conn(0x7fa9fc006230 0x7fa9fc0086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:12.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:12.394+0000 7faa1da0d700 1 -- 172.21.15.78:0/2518364538 >> v2:172.21.15.78:3300/0 conn(0x7fa9fc003790 msgr2=0x7fa9fc005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-11T09:40:12.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:12.394+0000 7faa1da0d700 1 -- 172.21.15.78:0/2518364538 >> v2:172.21.15.78:3300/0 conn(0x7fa9fc003790 msgr2=0x7fa9fc005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-11T09:40:12.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:12.394+0000 7faa1da0d700 1 --2- 172.21.15.78:0/2518364538 >> v2:172.21.15.78:3300/0 conn(0x7fa9fc003790 0x7fa9fc005c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-11T09:40:12.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:12.394+0000 7faa1da0d700 1 --2- 172.21.15.78:0/2518364538 >> v2:172.21.15.78:3300/0 conn(0x7fa9fc003790 0x7fa9fc005c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-11T09:40:12.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:12.394+0000 7faa1d20c700 1 -- 172.21.15.78:0/2518364538 >> v2:172.21.15.133:3300/0 conn(0x7fa9fc006230 msgr2=0x7fa9fc0086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-11T09:40:12.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:12.394+0000 7faa1d20c700 1 -- 172.21.15.78:0/2518364538 >> v2:172.21.15.133:3300/0 conn(0x7fa9fc006230 msgr2=0x7fa9fc0086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-11T09:40:12.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:12.394+0000 7faa1d20c700 1 --2- 172.21.15.78:0/2518364538 >> v2:172.21.15.133:3300/0 conn(0x7fa9fc006230 0x7fa9fc0086e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-11T09:40:12.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:12.394+0000 7faa1d20c700 1 --2- 172.21.15.78:0/2518364538 >> v2:172.21.15.133:3300/0 conn(0x7fa9fc006230 0x7fa9fc0086e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-11T09:40:12.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:12.798+0000 7faa1da0d700 1 --2- 172.21.15.78:0/2518364538 >> v2:172.21.15.78:3300/0 conn(0x7fa9fc003790 0x7fa9fc005c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:12.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:12.798+0000 7faa1d20c700 1 --2- 172.21.15.78:0/2518364538 >> v2:172.21.15.133:3300/0 conn(0x7fa9fc006230 0x7fa9fc0086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:12.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:12.798+0000 7faa1da0d700 1 -- 172.21.15.78:0/2518364538 >> v2:172.21.15.78:3300/0 conn(0x7fa9fc003790 msgr2=0x7fa9fc005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-11T09:40:12.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:12.798+0000 7faa1da0d700 1 -- 172.21.15.78:0/2518364538 >> v2:172.21.15.78:3300/0 conn(0x7fa9fc003790 msgr2=0x7fa9fc005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-11T09:40:12.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:12.798+0000 7faa1da0d700 1 --2- 172.21.15.78:0/2518364538 >> v2:172.21.15.78:3300/0 conn(0x7fa9fc003790 0x7fa9fc005c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-11T09:40:12.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:12.798+0000 7faa1da0d700 1 --2- 172.21.15.78:0/2518364538 >> v2:172.21.15.78:3300/0 conn(0x7fa9fc003790 0x7fa9fc005c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-11T09:40:12.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:12.798+0000 7faa1d20c700 1 -- 172.21.15.78:0/2518364538 >> v2:172.21.15.133:3300/0 conn(0x7fa9fc006230 msgr2=0x7fa9fc0086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-11T09:40:12.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:12.798+0000 7faa1d20c700 1 -- 172.21.15.78:0/2518364538 >> v2:172.21.15.133:3300/0 conn(0x7fa9fc006230 msgr2=0x7fa9fc0086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-11T09:40:12.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:12.798+0000 7faa1d20c700 1 --2- 172.21.15.78:0/2518364538 >> v2:172.21.15.133:3300/0 conn(0x7fa9fc006230 0x7fa9fc0086e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-11T09:40:12.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:12.798+0000 7faa1d20c700 1 --2- 172.21.15.78:0/2518364538 >> v2:172.21.15.133:3300/0 conn(0x7fa9fc006230 0x7fa9fc0086e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-11T09:40:13.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:13.602+0000 7faa1da0d700 1 --2- 172.21.15.78:0/2518364538 >> v2:172.21.15.78:3300/0 conn(0x7fa9fc003790 0x7fa9fc005c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:13.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:13.602+0000 7faa1d20c700 1 --2- 172.21.15.78:0/2518364538 >> v2:172.21.15.133:3300/0 conn(0x7fa9fc006230 0x7fa9fc0086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:13.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:13.602+0000 7faa1da0d700 1 -- 172.21.15.78:0/2518364538 >> v2:172.21.15.78:3300/0 conn(0x7fa9fc003790 msgr2=0x7fa9fc005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-11T09:40:13.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:13.602+0000 7faa1da0d700 1 -- 172.21.15.78:0/2518364538 >> v2:172.21.15.78:3300/0 conn(0x7fa9fc003790 msgr2=0x7fa9fc005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-11T09:40:13.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:13.602+0000 7faa1da0d700 1 --2- 172.21.15.78:0/2518364538 >> v2:172.21.15.78:3300/0 conn(0x7fa9fc003790 0x7fa9fc005c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-11T09:40:13.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:13.602+0000 7faa1da0d700 1 --2- 172.21.15.78:0/2518364538 >> v2:172.21.15.78:3300/0 conn(0x7fa9fc003790 0x7fa9fc005c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-11T09:40:13.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:13.602+0000 7faa1d20c700 1 -- 172.21.15.78:0/2518364538 >> v2:172.21.15.133:3300/0 conn(0x7fa9fc006230 msgr2=0x7fa9fc0086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-11T09:40:13.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:13.602+0000 7faa1d20c700 1 -- 172.21.15.78:0/2518364538 >> v2:172.21.15.133:3300/0 conn(0x7fa9fc006230 msgr2=0x7fa9fc0086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-11T09:40:13.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:13.602+0000 7faa1d20c700 1 --2- 172.21.15.78:0/2518364538 >> v2:172.21.15.133:3300/0 conn(0x7fa9fc006230 0x7fa9fc0086e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-11T09:40:13.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:13.602+0000 7faa1d20c700 1 --2- 172.21.15.78:0/2518364538 >> v2:172.21.15.133:3300/0 conn(0x7fa9fc006230 0x7fa9fc0086e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-11T09:40:14.611 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-10-11T09:40:14.606+0000 7f4cf162c2c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-11T09:40:14.637 INFO:tasks.ceph.mgr.x.smithi133.stderr:2023-10-11T09:40:14.632+0000 7f8edb41d2c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-11T09:40:15.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.186+0000 7faa0f7fe700 1 -- 172.21.15.78:0/2518364538 >> v2:172.21.15.133:3300/0 conn(0x7fa9fc006230 msgr2=0x7fa9fc0086e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-11T09:40:15.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.186+0000 7faa0f7fe700 1 --2- 172.21.15.78:0/2518364538 >> v2:172.21.15.133:3300/0 conn(0x7fa9fc006230 0x7fa9fc0086e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:15.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.186+0000 7faa0f7fe700 1 -- 172.21.15.78:0/2518364538 >> v2:172.21.15.78:3300/0 conn(0x7fa9fc003790 msgr2=0x7fa9fc005c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-11T09:40:15.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.186+0000 7faa0f7fe700 1 --2- 172.21.15.78:0/2518364538 >> v2:172.21.15.78:3300/0 conn(0x7fa9fc003790 0x7fa9fc005c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:15.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.186+0000 7faa1e20e700 1 -- 172.21.15.78:0/2518364538 reap_dead start 2023-10-11T09:40:15.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.186+0000 7faa0f7fe700 1 --2- 172.21.15.78:0/2518364538 >> v2:172.21.15.133:3300/0 conn(0x7fa9fc009290 0x7fa9fc00b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:15.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.186+0000 7faa0f7fe700 1 --2- 172.21.15.78:0/2518364538 >> v2:172.21.15.78:3300/0 conn(0x7fa9fc003790 0x7fa9fc005be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:15.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.190+0000 7faa0f7fe700 1 -- 172.21.15.78:0/2518364538 --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa18164c40 con 0x7faa18160cb0 2023-10-11T09:40:15.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.190+0000 7faa0f7fe700 1 -- 172.21.15.78:0/2518364538 --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9fc008d70 con 0x7fa9fc003790 2023-10-11T09:40:15.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.190+0000 7faa0f7fe700 1 -- 172.21.15.78:0/2518364538 --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9fc008ee0 con 0x7fa9fc009290 2023-10-11T09:40:15.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.190+0000 7faa1d20c700 1 --2- 172.21.15.78:0/2518364538 >> v2:172.21.15.78:3300/0 conn(0x7fa9fc003790 0x7fa9fc005be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:15.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.190+0000 7faa1da0d700 1 --2- 172.21.15.78:0/2518364538 >> v2:172.21.15.133:3300/0 conn(0x7fa9fc009290 0x7fa9fc00b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:15.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.190+0000 7faa1ca0b700 1 -- 172.21.15.78:0/2518364538 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4279348109 0 0) 0x7faa18164c40 con 0x7faa18160cb0 2023-10-11T09:40:15.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.190+0000 7faa1ca0b700 1 -- 172.21.15.78:0/2518364538 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa040035d0 con 0x7faa18160cb0 2023-10-11T09:40:15.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.190+0000 7faa1da0d700 1 -- 172.21.15.78:0/2518364538 >> v2:172.21.15.78:3300/0 conn(0x7fa9fc003790 msgr2=0x7fa9fc005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:15.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.190+0000 7faa1da0d700 1 --2- 172.21.15.78:0/2518364538 >> v2:172.21.15.78:3300/0 conn(0x7fa9fc003790 0x7fa9fc005be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:15.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.190+0000 7faa1da0d700 1 -- 172.21.15.78:0/2518364538 >> v1:172.21.15.78:6789/0 conn(0x7faa18160cb0 legacy=0x7fa9fc00bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:15.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.190+0000 7faa1da0d700 1 -- 172.21.15.78:0/2518364538 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9fc008830 con 0x7fa9fc009290 2023-10-11T09:40:15.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.190+0000 7faa1d20c700 1 --2- 172.21.15.78:0/2518364538 >> v2:172.21.15.78:3300/0 conn(0x7fa9fc003790 0x7fa9fc005be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:40:15.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.190+0000 7faa1da0d700 1 --2- 172.21.15.78:0/2518364538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9fc009290 0x7fa9fc00b6b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faa180c9230 tx=0x7faa14001df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dc5fb6f8ae56efb server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:15.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.190+0000 7faa1ca0b700 1 -- 172.21.15.78:0/2518364538 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa14005400 con 0x7fa9fc009290 2023-10-11T09:40:15.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.190+0000 7faa1ca0b700 1 -- 172.21.15.78:0/2518364538 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa14005560 con 0x7fa9fc009290 2023-10-11T09:40:15.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.190+0000 7faa1ca0b700 1 -- 172.21.15.78:0/2518364538 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa140058d0 con 0x7fa9fc009290 2023-10-11T09:40:15.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.194+0000 7faa1fc71700 1 -- 172.21.15.78:0/2518364538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9fc009290 msgr2=0x7fa9fc00b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:15.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.194+0000 7faa1fc71700 1 --2- 172.21.15.78:0/2518364538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9fc009290 0x7fa9fc00b6b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faa180c9230 tx=0x7faa14001df0 comp rx=0 tx=0).stop 2023-10-11T09:40:15.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.194+0000 7faa1fc71700 1 -- 172.21.15.78:0/2518364538 shutdown_connections 2023-10-11T09:40:15.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.194+0000 7faa1fc71700 1 --2- 172.21.15.78:0/2518364538 >> v2:172.21.15.78:3300/0 conn(0x7fa9fc003790 0x7fa9fc005be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:15.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.194+0000 7faa1fc71700 1 --2- 172.21.15.78:0/2518364538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9fc009290 0x7fa9fc00b6b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:15.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.194+0000 7faa1fc71700 1 -- 172.21.15.78:0/2518364538 >> 172.21.15.78:0/2518364538 conn(0x7faa180cecc0 msgr2=0x7faa180cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:15.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.194+0000 7faa1fc71700 1 -- 172.21.15.78:0/2518364538 shutdown_connections 2023-10-11T09:40:15.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.194+0000 7faa1fc71700 1 -- 172.21.15.78:0/2518364538 wait complete. 2023-10-11T09:40:15.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.198+0000 7faa1fc71700 1 Processor -- start 2023-10-11T09:40:15.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.198+0000 7faa1fc71700 1 -- start start 2023-10-11T09:40:15.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.198+0000 7faa1fc71700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9fc003790 0x7faa1815a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:15.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.198+0000 7faa1fc71700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa9fc009290 0x7faa18161080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:15.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.198+0000 7faa1fc71700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa18160cb0 0x7faa181615c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:15.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.198+0000 7faa1fc71700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa181f6460 con 0x7faa18160cb0 2023-10-11T09:40:15.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.198+0000 7faa1fc71700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1815ca60 con 0x7fa9fc003790 2023-10-11T09:40:15.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.198+0000 7faa1fc71700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa18161bf0 con 0x7fa9fc009290 2023-10-11T09:40:15.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.198+0000 7faa1e20e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa18160cb0 0x7faa181615c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:15.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.198+0000 7faa1e20e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa18160cb0 0x7faa181615c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40944/0 (socket says 172.21.15.78:40944) 2023-10-11T09:40:15.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.198+0000 7faa1e20e700 1 -- 172.21.15.78:0/673938372 learned_addr learned my addr 172.21.15.78:0/673938372 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:15.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.198+0000 7faa1da0d700 1 --2- 172.21.15.78:0/673938372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9fc003790 0x7faa1815a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:15.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.198+0000 7faa1d20c700 1 --2- 172.21.15.78:0/673938372 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa9fc009290 0x7faa18161080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:15.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.198+0000 7faa1e20e700 1 -- 172.21.15.78:0/673938372 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa9fc009290 msgr2=0x7faa18161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:15.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.198+0000 7faa1e20e700 1 --2- 172.21.15.78:0/673938372 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa9fc009290 0x7faa18161080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:15.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.198+0000 7faa1e20e700 1 -- 172.21.15.78:0/673938372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9fc003790 msgr2=0x7faa1815a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:15.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.198+0000 7faa1e20e700 1 --2- 172.21.15.78:0/673938372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9fc003790 0x7faa1815a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:15.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.198+0000 7faa1e20e700 1 -- 172.21.15.78:0/673938372 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa0800f040 con 0x7faa18160cb0 2023-10-11T09:40:15.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.198+0000 7faa1da0d700 1 --2- 172.21.15.78:0/673938372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9fc003790 0x7faa1815a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:40:15.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.198+0000 7faa1e20e700 1 --2- 172.21.15.78:0/673938372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa18160cb0 0x7faa181615c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7faa180b4f50 tx=0x7faa10013a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:15.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.202+0000 7faa0e7fc700 1 -- 172.21.15.78:0/673938372 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa10004220 con 0x7faa18160cb0 2023-10-11T09:40:15.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.202+0000 7faa0e7fc700 1 -- 172.21.15.78:0/673938372 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa1000c040 con 0x7faa18160cb0 2023-10-11T09:40:15.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.202+0000 7faa0e7fc700 1 -- 172.21.15.78:0/673938372 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa100036c0 con 0x7faa18160cb0 2023-10-11T09:40:15.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.202+0000 7faa1fc71700 1 -- 172.21.15.78:0/673938372 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa1400f040 con 0x7faa18160cb0 2023-10-11T09:40:15.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.202+0000 7faa1fc71700 1 -- 172.21.15.78:0/673938372 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa1815ccb0 con 0x7faa18160cb0 2023-10-11T09:40:15.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.202+0000 7faa0e7fc700 1 -- 172.21.15.78:0/673938372 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7faa10003b20 con 0x7faa18160cb0 2023-10-11T09:40:15.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.202+0000 7faa0e7fc700 1 -- 172.21.15.78:0/673938372 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7faa10000e40 con 0x7faa18160cb0 2023-10-11T09:40:15.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.202+0000 7faa1fc71700 1 -- 172.21.15.78:0/673938372 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa1806eb80 con 0x7faa18160cb0 2023-10-11T09:40:15.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.206+0000 7faa0e7fc700 1 -- 172.21.15.78:0/673938372 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7faa1003fb00 con 0x7faa18160cb0 2023-10-11T09:40:15.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.278+0000 7faa1fc71700 1 -- 172.21.15.78:0/673938372 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7faa180b0e10 con 0x7faa18160cb0 2023-10-11T09:40:15.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.282+0000 7faa0e7fc700 1 -- 172.21.15.78:0/673938372 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7faa1002e020 con 0x7faa18160cb0 2023-10-11T09:40:15.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.282+0000 7faa1fc71700 1 -- 172.21.15.78:0/673938372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa18160cb0 msgr2=0x7faa181615c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:15.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.282+0000 7faa1fc71700 1 --2- 172.21.15.78:0/673938372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa18160cb0 0x7faa181615c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7faa180b4f50 tx=0x7faa10013a60 comp rx=0 tx=0).stop 2023-10-11T09:40:15.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.282+0000 7faa1fc71700 1 -- 172.21.15.78:0/673938372 shutdown_connections 2023-10-11T09:40:15.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.282+0000 7faa1fc71700 1 --2- 172.21.15.78:0/673938372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa18160cb0 0x7faa181615c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:15.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.282+0000 7faa1fc71700 1 --2- 172.21.15.78:0/673938372 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa9fc009290 0x7faa18161080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:15.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.282+0000 7faa1fc71700 1 --2- 172.21.15.78:0/673938372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9fc003790 0x7faa1815a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:15.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.282+0000 7faa1fc71700 1 -- 172.21.15.78:0/673938372 >> 172.21.15.78:0/673938372 conn(0x7faa180cecc0 msgr2=0x7faa180cdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:15.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.282+0000 7faa1fc71700 1 -- 172.21.15.78:0/673938372 shutdown_connections 2023-10-11T09:40:15.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:15.282+0000 7faa1fc71700 1 -- 172.21.15.78:0/673938372 wait complete. 2023-10-11T09:40:15.291 INFO:teuthology.orchestra.run.smithi078.stderr:adjusted tunables profile to default 2023-10-11T09:40:15.299 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-10-11T09:40:15.294+0000 7f4cf162c2c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-11T09:40:15.310 INFO:tasks.ceph:check_enable_crimson: False 2023-10-11T09:40:15.310 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-11T09:40:15.311 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:40:15.311 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-11T09:40:15.325 INFO:tasks.ceph.mgr.x.smithi133.stderr:2023-10-11T09:40:15.320+0000 7f8edb41d2c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-11T09:40:15.341 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:40:15.341 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-11T09:40:15.382 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-10-11T09:40:15.378+0000 7f4cf162c2c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-11T09:40:15.404 INFO:tasks.ceph.mgr.x.smithi133.stderr:2023-10-11T09:40:15.400+0000 7f8edb41d2c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-11T09:40:15.434 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:40:15.434 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-11T09:40:15.507 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:40:15.507 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-11T09:40:15.556 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-10-11T09:40:15.550+0000 7f4cf162c2c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-11T09:40:15.577 INFO:tasks.ceph.mgr.x.smithi133.stderr:2023-10-11T09:40:15.572+0000 7f8edb41d2c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-11T09:40:15.581 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:40:15.582 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-11T09:40:15.614 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:40:15.615 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-11T09:40:15.688 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:40:15.688 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-11T09:40:15.758 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:40:15.758 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-11T09:40:15.808 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-10-11T09:40:15.802+0000 7f4cf162c2c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-11T09:40:15.832 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph osd new 7b2760f0-8365-40e0-be27-80ea0140c601 0 2023-10-11T09:40:15.838 INFO:tasks.ceph.mgr.x.smithi133.stderr:2023-10-11T09:40:15.832+0000 7f8edb41d2c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-11T09:40:15.882 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-10-11T09:40:15.878+0000 7f4cf162c2c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-11T09:40:15.911 INFO:tasks.ceph.mgr.x.smithi133.stderr:2023-10-11T09:40:15.904+0000 7f8edb41d2c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-11T09:40:16.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:15.996+0000 7fce2d6fa700 1 Processor -- start 2023-10-11T09:40:16.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:15.996+0000 7fce2d6fa700 1 -- start start 2023-10-11T09:40:16.004 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.000+0000 7fce2d6fa700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fce2806fe50 0x7fce280cf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:16.004 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.000+0000 7fce2d6fa700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce2806f9c0 con 0x7fce280b3150 2023-10-11T09:40:16.005 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.000+0000 7fce2d6fa700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce2806edb0 con 0x7fce2815a810 2023-10-11T09:40:16.005 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.000+0000 7fce2d6fa700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce2815abe0 con 0x7fce2806fe50 2023-10-11T09:40:16.005 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.000+0000 7fce277fe700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fce2806fe50 0x7fce280cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:16.005 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.000+0000 7fce277fe700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fce2806fe50 0x7fce280cf310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:47780/0 (socket says 172.21.15.133:47780) 2023-10-11T09:40:16.005 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.000+0000 7fce277fe700 1 -- 172.21.15.133:0/1855199802 learned_addr learned my addr 172.21.15.133:0/1855199802 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-10-11T09:40:16.007 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.000+0000 7fce277fe700 1 -- 172.21.15.133:0/1855199802 >> v1:172.21.15.133:6789/0 conn(0x7fce2815a810 legacy=0x7fce280b2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:16.008 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.000+0000 7fce277fe700 1 -- 172.21.15.133:0/1855199802 >> v1:172.21.15.78:6789/0 conn(0x7fce280b3150 legacy=0x7fce280cec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:16.008 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.000+0000 7fce277fe700 1 -- 172.21.15.133:0/1855199802 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce2806f090 con 0x7fce2806fe50 2023-10-11T09:40:16.008 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.000+0000 7fce277fe700 1 --2- 172.21.15.133:0/1855199802 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fce2806fe50 0x7fce280cf310 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fce20009e80 tx=0x7fce2000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2be5f222db609772 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:16.008 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.000+0000 7fce25ffb700 1 -- 172.21.15.133:0/1855199802 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2000e070 con 0x7fce2806fe50 2023-10-11T09:40:16.008 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.000+0000 7fce25ffb700 1 -- 172.21.15.133:0/1855199802 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce2000b9c0 con 0x7fce2806fe50 2023-10-11T09:40:16.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.000+0000 7fce25ffb700 1 -- 172.21.15.133:0/1855199802 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2000bd50 con 0x7fce2806fe50 2023-10-11T09:40:16.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.000+0000 7fce2d6fa700 1 -- 172.21.15.133:0/1855199802 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fce2806fe50 msgr2=0x7fce280cf310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:16.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.000+0000 7fce2d6fa700 1 --2- 172.21.15.133:0/1855199802 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fce2806fe50 0x7fce280cf310 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fce20009e80 tx=0x7fce2000b330 comp rx=0 tx=0).stop 2023-10-11T09:40:16.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.000+0000 7fce2d6fa700 1 -- 172.21.15.133:0/1855199802 shutdown_connections 2023-10-11T09:40:16.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.000+0000 7fce2d6fa700 1 --2- 172.21.15.133:0/1855199802 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fce2806fe50 0x7fce280cf310 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:16.010 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.000+0000 7fce2d6fa700 1 -- 172.21.15.133:0/1855199802 >> 172.21.15.133:0/1855199802 conn(0x7fce2809a230 msgr2=0x7fce2808aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:16.010 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.004+0000 7fce2d6fa700 1 -- 172.21.15.133:0/1855199802 shutdown_connections 2023-10-11T09:40:16.010 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.004+0000 7fce2d6fa700 1 -- 172.21.15.133:0/1855199802 wait complete. 2023-10-11T09:40:16.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.004+0000 7fce2d6fa700 1 Processor -- start 2023-10-11T09:40:16.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.004+0000 7fce2d6fa700 1 -- start start 2023-10-11T09:40:16.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.004+0000 7fce2d6fa700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce2806fe50 0x7fce280c7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:16.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.008+0000 7fce2d6fa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce280b3150 0x7fce280b4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:16.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.008+0000 7fce2d6fa700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fce2815a810 0x7fce280b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:16.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.008+0000 7fce2d6fa700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce280c9760 con 0x7fce280b3150 2023-10-11T09:40:16.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.008+0000 7fce2d6fa700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce280c79b0 con 0x7fce2806fe50 2023-10-11T09:40:16.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.008+0000 7fce2d6fa700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce280b46d0 con 0x7fce2815a810 2023-10-11T09:40:16.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.008+0000 7fce26ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce2806fe50 0x7fce280c7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:16.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.008+0000 7fce277fe700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fce2815a810 0x7fce280b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:16.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.008+0000 7fce26ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce2806fe50 0x7fce280c7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36194/0 (socket says 172.21.15.133:36194) 2023-10-11T09:40:16.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.008+0000 7fce26ffd700 1 -- 172.21.15.133:0/1845882210 learned_addr learned my addr 172.21.15.133:0/1845882210 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-10-11T09:40:16.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.008+0000 7fce267fc700 1 --2- 172.21.15.133:0/1845882210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce280b3150 0x7fce280b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:16.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.008+0000 7fce26ffd700 1 -- 172.21.15.133:0/1845882210 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fce2815a810 msgr2=0x7fce280b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:16.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.008+0000 7fce26ffd700 1 --2- 172.21.15.133:0/1845882210 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fce2815a810 0x7fce280b19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:16.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.008+0000 7fce26ffd700 1 -- 172.21.15.133:0/1845882210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce280b3150 msgr2=0x7fce280b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:16.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.008+0000 7fce26ffd700 1 --2- 172.21.15.133:0/1845882210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce280b3150 0x7fce280b4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:16.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.008+0000 7fce26ffd700 1 -- 172.21.15.133:0/1845882210 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce20009b30 con 0x7fce2806fe50 2023-10-11T09:40:16.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.008+0000 7fce267fc700 1 --2- 172.21.15.133:0/1845882210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce280b3150 0x7fce280b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:40:16.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.008+0000 7fce277fe700 1 --2- 172.21.15.133:0/1845882210 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fce2815a810 0x7fce280b19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:40:16.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.008+0000 7fce26ffd700 1 --2- 172.21.15.133:0/1845882210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce2806fe50 0x7fce280c7470 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fce1800c450 tx=0x7fce1800c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:16.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.008+0000 7fce0f7fe700 1 -- 172.21.15.133:0/1845882210 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce18011790 con 0x7fce2806fe50 2023-10-11T09:40:16.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.008+0000 7fce0f7fe700 1 -- 172.21.15.133:0/1845882210 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce180118f0 con 0x7fce2806fe50 2023-10-11T09:40:16.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.008+0000 7fce2d6fa700 1 -- 172.21.15.133:0/1845882210 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce280b4970 con 0x7fce2806fe50 2023-10-11T09:40:16.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.008+0000 7fce0f7fe700 1 -- 172.21.15.133:0/1845882210 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce18011ca0 con 0x7fce2806fe50 2023-10-11T09:40:16.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.008+0000 7fce2d6fa700 1 -- 172.21.15.133:0/1845882210 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce280c9380 con 0x7fce2806fe50 2023-10-11T09:40:16.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.008+0000 7fce0f7fe700 1 -- 172.21.15.133:0/1845882210 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fce1801c760 con 0x7fce2806fe50 2023-10-11T09:40:16.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.008+0000 7fce0f7fe700 1 -- 172.21.15.133:0/1845882210 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fce18012620 con 0x7fce2806fe50 2023-10-11T09:40:16.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.008+0000 7fce2d6fa700 1 -- 172.21.15.133:0/1845882210 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce28070570 con 0x7fce2806fe50 2023-10-11T09:40:16.027 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.012+0000 7fce0f7fe700 1 -- 172.21.15.133:0/1845882210 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fce1801cba0 con 0x7fce2806fe50 2023-10-11T09:40:16.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.092+0000 7fce2d6fa700 1 -- 172.21.15.133:0/1845882210 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7b2760f0-8365-40e0-be27-80ea0140c601", "id": 0} v 0) v1 -- 0x7fce281653b0 con 0x7fce2806fe50 2023-10-11T09:40:16.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.100+0000 7fce0f7fe700 1 -- 172.21.15.133:0/1845882210 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7b2760f0-8365-40e0-be27-80ea0140c601", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fce1801cd40 con 0x7fce2806fe50 2023-10-11T09:40:16.107 INFO:teuthology.orchestra.run.smithi133.stdout:0 2023-10-11T09:40:16.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.104+0000 7fce2d6fa700 1 -- 172.21.15.133:0/1845882210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce2806fe50 msgr2=0x7fce280c7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:16.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.104+0000 7fce2d6fa700 1 --2- 172.21.15.133:0/1845882210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce2806fe50 0x7fce280c7470 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fce1800c450 tx=0x7fce1800c760 comp rx=0 tx=0).stop 2023-10-11T09:40:16.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.104+0000 7fce2d6fa700 1 -- 172.21.15.133:0/1845882210 shutdown_connections 2023-10-11T09:40:16.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.104+0000 7fce2d6fa700 1 --2- 172.21.15.133:0/1845882210 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fce2815a810 0x7fce280b19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:16.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.104+0000 7fce2d6fa700 1 --2- 172.21.15.133:0/1845882210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce280b3150 0x7fce280b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:16.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.104+0000 7fce2d6fa700 1 --2- 172.21.15.133:0/1845882210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce2806fe50 0x7fce280c7470 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:16.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.104+0000 7fce2d6fa700 1 -- 172.21.15.133:0/1845882210 >> 172.21.15.133:0/1845882210 conn(0x7fce2809a230 msgr2=0x7fce280ceb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:16.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.104+0000 7fce2d6fa700 1 -- 172.21.15.133:0/1845882210 shutdown_connections 2023-10-11T09:40:16.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.104+0000 7fce2d6fa700 1 -- 172.21.15.133:0/1845882210 wait complete. 2023-10-11T09:40:16.119 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-10-11T09:40:16.114+0000 7f4cf162c2c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-11T09:40:16.135 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph osd new 3de55db3-11d4-44f6-bf0b-41c5f97eb9c4 1 2023-10-11T09:40:16.163 INFO:tasks.ceph.mgr.x.smithi133.stderr:2023-10-11T09:40:16.156+0000 7f8edb41d2c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-11T09:40:16.199 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-10-11T09:40:16.194+0000 7f4cf162c2c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-11T09:40:16.247 INFO:tasks.ceph.mgr.x.smithi133.stderr:2023-10-11T09:40:16.240+0000 7f8edb41d2c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-11T09:40:16.286 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-10-11T09:40:16.282+0000 7f4cf162c2c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-11T09:40:16.296 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.292+0000 7f68e1a5e700 1 Processor -- start 2023-10-11T09:40:16.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.292+0000 7f68e1a5e700 1 -- start start 2023-10-11T09:40:16.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.292+0000 7f68e1a5e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f68dc0b84c0 0x7f68dc0cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:16.299 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.292+0000 7f68e1a5e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f68dc0b8890 0x7f68dc160370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:16.299 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.292+0000 7f68e1a5e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68dc0cd690 con 0x7f68dc1608b0 2023-10-11T09:40:16.299 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.292+0000 7f68e1a5e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68dc0cdea0 con 0x7f68dc0b84c0 2023-10-11T09:40:16.299 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.292+0000 7f68e1a5e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68dc0d0c10 con 0x7f68dc0b8890 2023-10-11T09:40:16.300 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.292+0000 7f68db7fe700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f68dc1608b0 0x7f68dc0d2fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:42332/0 (socket says 172.21.15.133:42332) 2023-10-11T09:40:16.300 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.292+0000 7f68daffd700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f68dc0b84c0 0x7f68dc0cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:16.300 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.292+0000 7f68db7fe700 1 -- 172.21.15.133:0/1652919132 learned_addr learned my addr 172.21.15.133:0/1652919132 (peer_addr_for_me v1:172.21.15.133:0/0) 2023-10-11T09:40:16.300 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.292+0000 7f68da7fc700 1 --2- 172.21.15.133:0/1652919132 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f68dc0b8890 0x7f68dc160370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:16.301 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.292+0000 7f68daffd700 1 -- 172.21.15.133:0/1652919132 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f68dc0b8890 msgr2=0x7f68dc160370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:16.302 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.292+0000 7f68daffd700 1 --2- 172.21.15.133:0/1652919132 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f68dc0b8890 0x7f68dc160370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:16.302 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.292+0000 7f68daffd700 1 -- 172.21.15.133:0/1652919132 >> v1:172.21.15.133:6789/0 conn(0x7f68dc1608b0 legacy=0x7f68dc0d2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:16.302 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.296+0000 7f68daffd700 1 -- 172.21.15.133:0/1652919132 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68dc0d02e0 con 0x7f68dc0b84c0 2023-10-11T09:40:16.302 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.296+0000 7f68daffd700 1 --2- 172.21.15.133:0/1652919132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68dc0b84c0 0x7f68dc0cfda0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f68cc009850 tx=0x7f68cc009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2555047c4bdf696f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:16.303 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.296+0000 7f68d9ffb700 1 -- 172.21.15.133:0/1652919132 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68cc011830 con 0x7f68dc0b84c0 2023-10-11T09:40:16.303 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.296+0000 7f68d9ffb700 1 -- 172.21.15.133:0/1652919132 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68cc011990 con 0x7f68dc0b84c0 2023-10-11T09:40:16.303 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.296+0000 7f68d9ffb700 1 -- 172.21.15.133:0/1652919132 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68cc011d20 con 0x7f68dc0b84c0 2023-10-11T09:40:16.303 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.296+0000 7f68e1a5e700 1 -- 172.21.15.133:0/1652919132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68dc0b84c0 msgr2=0x7f68dc0cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:16.303 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.296+0000 7f68e1a5e700 1 --2- 172.21.15.133:0/1652919132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68dc0b84c0 0x7f68dc0cfda0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f68cc009850 tx=0x7f68cc009b60 comp rx=0 tx=0).stop 2023-10-11T09:40:16.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.296+0000 7f68e1a5e700 1 -- 172.21.15.133:0/1652919132 shutdown_connections 2023-10-11T09:40:16.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.296+0000 7f68e1a5e700 1 --2- 172.21.15.133:0/1652919132 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f68dc0b8890 0x7f68dc160370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:16.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.296+0000 7f68e1a5e700 1 --2- 172.21.15.133:0/1652919132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68dc0b84c0 0x7f68dc0cfda0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:16.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.296+0000 7f68e1a5e700 1 -- 172.21.15.133:0/1652919132 >> 172.21.15.133:0/1652919132 conn(0x7f68dc06cb50 msgr2=0x7f68dc06cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:16.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.296+0000 7f68e1a5e700 1 -- 172.21.15.133:0/1652919132 shutdown_connections 2023-10-11T09:40:16.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.296+0000 7f68e1a5e700 1 -- 172.21.15.133:0/1652919132 wait complete. 2023-10-11T09:40:16.305 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.300+0000 7f68e1a5e700 1 Processor -- start 2023-10-11T09:40:16.305 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.300+0000 7f68e1a5e700 1 -- start start 2023-10-11T09:40:16.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.300+0000 7f68e1a5e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68dc0b84c0 0x7f68dc0d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:16.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.300+0000 7f68e1a5e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f68dc0b8890 0x7f68dc0ccf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:16.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.300+0000 7f68e1a5e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68dc1608b0 0x7f68dc0c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:16.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.300+0000 7f68e1a5e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68dc153a10 con 0x7f68dc0b84c0 2023-10-11T09:40:16.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.300+0000 7f68e1a5e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68dc0d17c0 con 0x7f68dc1608b0 2023-10-11T09:40:16.308 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.300+0000 7f68e1a5e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68dc0d1aa0 con 0x7f68dc0b8890 2023-10-11T09:40:16.308 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.300+0000 7f68da7fc700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f68dc0b8890 0x7f68dc0ccf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:16.308 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.300+0000 7f68db7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68dc1608b0 0x7f68dc0c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:16.308 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.300+0000 7f68db7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68dc1608b0 0x7f68dc0c5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36224/0 (socket says 172.21.15.133:36224) 2023-10-11T09:40:16.308 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.300+0000 7f68da7fc700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f68dc0b8890 0x7f68dc0ccf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:47824/0 (socket says 172.21.15.133:47824) 2023-10-11T09:40:16.309 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.300+0000 7f68db7fe700 1 -- 172.21.15.133:0/4263906336 learned_addr learned my addr 172.21.15.133:0/4263906336 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-10-11T09:40:16.309 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.300+0000 7f68daffd700 1 --2- 172.21.15.133:0/4263906336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68dc0b84c0 0x7f68dc0d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:16.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.300+0000 7f68db7fe700 1 -- 172.21.15.133:0/4263906336 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f68dc0b8890 msgr2=0x7f68dc0ccf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:16.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.300+0000 7f68db7fe700 1 --2- 172.21.15.133:0/4263906336 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f68dc0b8890 0x7f68dc0ccf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:16.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.304+0000 7f68db7fe700 1 -- 172.21.15.133:0/4263906336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68dc0b84c0 msgr2=0x7f68dc0d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:16.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.304+0000 7f68db7fe700 1 --2- 172.21.15.133:0/4263906336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68dc0b84c0 0x7f68dc0d11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:16.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.304+0000 7f68db7fe700 1 -- 172.21.15.133:0/4263906336 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68cc0094b0 con 0x7f68dc1608b0 2023-10-11T09:40:16.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.304+0000 7f68daffd700 1 --2- 172.21.15.133:0/4263906336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68dc0b84c0 0x7f68dc0d11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:40:16.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.304+0000 7f68db7fe700 1 --2- 172.21.15.133:0/4263906336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68dc1608b0 0x7f68dc0c5f30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f68d400cbc0 tx=0x7f68d400cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:16.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.304+0000 7f68c3fff700 1 -- 172.21.15.133:0/4263906336 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d400a7b0 con 0x7f68dc1608b0 2023-10-11T09:40:16.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.304+0000 7f68c3fff700 1 -- 172.21.15.133:0/4263906336 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d400a910 con 0x7f68dc1608b0 2023-10-11T09:40:16.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.304+0000 7f68c3fff700 1 -- 172.21.15.133:0/4263906336 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d400ac80 con 0x7f68dc1608b0 2023-10-11T09:40:16.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.304+0000 7f68e1a5e700 1 -- 172.21.15.133:0/4263906336 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68dc0d1d80 con 0x7f68dc1608b0 2023-10-11T09:40:16.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.304+0000 7f68e1a5e700 1 -- 172.21.15.133:0/4263906336 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68dc0d2340 con 0x7f68dc1608b0 2023-10-11T09:40:16.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.304+0000 7f68c3fff700 1 -- 172.21.15.133:0/4263906336 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f68d401c440 con 0x7f68dc1608b0 2023-10-11T09:40:16.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.304+0000 7f68c3fff700 1 -- 172.21.15.133:0/4263906336 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f68d40127b0 con 0x7f68dc1608b0 2023-10-11T09:40:16.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.304+0000 7f68e1a5e700 1 -- 172.21.15.133:0/4263906336 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68dc0d2580 con 0x7f68dc1608b0 2023-10-11T09:40:16.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.308+0000 7f68c3fff700 1 -- 172.21.15.133:0/4263906336 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f68dc0d2580 con 0x7f68dc1608b0 2023-10-11T09:40:16.336 INFO:tasks.ceph.mgr.x.smithi133.stderr:2023-10-11T09:40:16.332+0000 7f8edb41d2c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-11T09:40:16.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.388+0000 7f68e1a5e700 1 -- 172.21.15.133:0/4263906336 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3de55db3-11d4-44f6-bf0b-41c5f97eb9c4", "id": 1} v 0) v1 -- 0x7f68dc08bff0 con 0x7f68dc1608b0 2023-10-11T09:40:16.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.396+0000 7f68c3fff700 1 -- 172.21.15.133:0/4263906336 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3de55db3-11d4-44f6-bf0b-41c5f97eb9c4", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f68d4021060 con 0x7f68dc1608b0 2023-10-11T09:40:16.402 INFO:teuthology.orchestra.run.smithi133.stdout:1 2023-10-11T09:40:16.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.400+0000 7f68e1a5e700 1 -- 172.21.15.133:0/4263906336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68dc1608b0 msgr2=0x7f68dc0c5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:16.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.400+0000 7f68e1a5e700 1 --2- 172.21.15.133:0/4263906336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68dc1608b0 0x7f68dc0c5f30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f68d400cbc0 tx=0x7f68d400cbf0 comp rx=0 tx=0).stop 2023-10-11T09:40:16.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.400+0000 7f68e1a5e700 1 -- 172.21.15.133:0/4263906336 shutdown_connections 2023-10-11T09:40:16.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.400+0000 7f68e1a5e700 1 --2- 172.21.15.133:0/4263906336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68dc1608b0 0x7f68dc0c5f30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:16.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.400+0000 7f68e1a5e700 1 --2- 172.21.15.133:0/4263906336 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f68dc0b8890 0x7f68dc0ccf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:16.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.400+0000 7f68e1a5e700 1 --2- 172.21.15.133:0/4263906336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68dc0b84c0 0x7f68dc0d11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:16.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.400+0000 7f68e1a5e700 1 -- 172.21.15.133:0/4263906336 >> 172.21.15.133:0/4263906336 conn(0x7f68dc06cb50 msgr2=0x7f68dc15c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:16.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.400+0000 7f68e1a5e700 1 -- 172.21.15.133:0/4263906336 shutdown_connections 2023-10-11T09:40:16.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.400+0000 7f68e1a5e700 1 -- 172.21.15.133:0/4263906336 wait complete. 2023-10-11T09:40:16.425 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph osd new c1a86a2e-7754-445a-b955-2df5a647ed07 2 2023-10-11T09:40:16.632 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.628+0000 7fbcfcd97700 1 Processor -- start 2023-10-11T09:40:16.632 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.628+0000 7fbcfcd97700 1 -- start start 2023-10-11T09:40:16.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.628+0000 7fbcfcd97700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbcf815c850 0x7fbcf80c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:16.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.628+0000 7fbcfcd97700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fbcf806e990 0x7fbcf80cb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:16.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.628+0000 7fbcfcd97700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcf80cb760 con 0x7fbcf80cce10 2023-10-11T09:40:16.635 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.628+0000 7fbcfcd97700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcf80cc810 con 0x7fbcf806e990 2023-10-11T09:40:16.635 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.628+0000 7fbcfcd97700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcf80cb920 con 0x7fbcf815c850 2023-10-11T09:40:16.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.628+0000 7fbcf77fe700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbcf815c850 0x7fbcf80c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:16.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.628+0000 7fbcf77fe700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbcf815c850 0x7fbcf80c95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36238/0 (socket says 172.21.15.133:36238) 2023-10-11T09:40:16.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.628+0000 7fbceffff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fbcf806e990 0x7fbcf80cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:16.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.628+0000 7fbcf77fe700 1 -- 172.21.15.133:0/184738212 learned_addr learned my addr 172.21.15.133:0/184738212 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-10-11T09:40:16.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.632+0000 7fbceffff700 1 -- 172.21.15.133:0/184738212 >> v2:172.21.15.133:3300/0 conn(0x7fbcf815c850 msgr2=0x7fbcf80c95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:16.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.632+0000 7fbceffff700 1 --2- 172.21.15.133:0/184738212 >> v2:172.21.15.133:3300/0 conn(0x7fbcf815c850 0x7fbcf80c95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:16.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.632+0000 7fbceffff700 1 -- 172.21.15.133:0/184738212 >> v1:172.21.15.78:6789/0 conn(0x7fbcf80cce10 legacy=0x7fbcf80cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:16.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.632+0000 7fbceffff700 1 -- 172.21.15.133:0/184738212 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf80cbc00 con 0x7fbcf806e990 2023-10-11T09:40:16.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.632+0000 7fbceffff700 1 --2- 172.21.15.133:0/184738212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf806e990 0x7fbcf80cb220 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbcf806ff20 tx=0x7fbce0009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec1b97be5eb693a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:16.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.632+0000 7fbcf6ffd700 1 -- 172.21.15.133:0/184738212 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce0011830 con 0x7fbcf806e990 2023-10-11T09:40:16.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.632+0000 7fbcf6ffd700 1 -- 172.21.15.133:0/184738212 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce0011990 con 0x7fbcf806e990 2023-10-11T09:40:16.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.632+0000 7fbcf6ffd700 1 -- 172.21.15.133:0/184738212 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce0011d20 con 0x7fbcf806e990 2023-10-11T09:40:16.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.632+0000 7fbcfcd97700 1 -- 172.21.15.133:0/184738212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf806e990 msgr2=0x7fbcf80cb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:16.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.632+0000 7fbcfcd97700 1 --2- 172.21.15.133:0/184738212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf806e990 0x7fbcf80cb220 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbcf806ff20 tx=0x7fbce0009b30 comp rx=0 tx=0).stop 2023-10-11T09:40:16.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.632+0000 7fbcfcd97700 1 -- 172.21.15.133:0/184738212 shutdown_connections 2023-10-11T09:40:16.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.632+0000 7fbcfcd97700 1 --2- 172.21.15.133:0/184738212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf806e990 0x7fbcf80cb220 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:16.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.632+0000 7fbcfcd97700 1 --2- 172.21.15.133:0/184738212 >> v2:172.21.15.133:3300/0 conn(0x7fbcf815c850 0x7fbcf80c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:16.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.632+0000 7fbcfcd97700 1 -- 172.21.15.133:0/184738212 >> 172.21.15.133:0/184738212 conn(0x7fbcf806dd80 msgr2=0x7fbcf806e180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:16.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.632+0000 7fbcfcd97700 1 -- 172.21.15.133:0/184738212 shutdown_connections 2023-10-11T09:40:16.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.632+0000 7fbcfcd97700 1 -- 172.21.15.133:0/184738212 wait complete. 2023-10-11T09:40:16.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.636+0000 7fbcfcd97700 1 Processor -- start 2023-10-11T09:40:16.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.636+0000 7fbcfcd97700 1 -- start start 2023-10-11T09:40:16.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.636+0000 7fbcfcd97700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf80cce10 0x7fbcf8140590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:16.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.636+0000 7fbcfcd97700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcf815c850 0x7fbcf8070c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:16.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.636+0000 7fbcfcd97700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbcf8071180 0x7fbcf80b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:16.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.636+0000 7fbcfcd97700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf80715f0 con 0x7fbcf80cce10 2023-10-11T09:40:16.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.636+0000 7fbcfcd97700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf815cc20 con 0x7fbcf815c850 2023-10-11T09:40:16.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.636+0000 7fbcfcd97700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf815da80 con 0x7fbcf8071180 2023-10-11T09:40:16.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.636+0000 7fbceffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcf815c850 0x7fbcf8070c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:16.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.636+0000 7fbceffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcf815c850 0x7fbcf8070c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36250/0 (socket says 172.21.15.133:36250) 2023-10-11T09:40:16.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.636+0000 7fbceffff700 1 -- 172.21.15.133:0/2128723312 learned_addr learned my addr 172.21.15.133:0/2128723312 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-10-11T09:40:16.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.636+0000 7fbcf7fff700 1 --2- 172.21.15.133:0/2128723312 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbcf8071180 0x7fbcf80b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:16.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.640+0000 7fbcf77fe700 1 --2- 172.21.15.133:0/2128723312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf80cce10 0x7fbcf8140590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:16.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.640+0000 7fbceffff700 1 -- 172.21.15.133:0/2128723312 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbcf8071180 msgr2=0x7fbcf80b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:16.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.640+0000 7fbceffff700 1 --2- 172.21.15.133:0/2128723312 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbcf8071180 0x7fbcf80b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:16.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.640+0000 7fbceffff700 1 -- 172.21.15.133:0/2128723312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf80cce10 msgr2=0x7fbcf8140590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:16.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.640+0000 7fbceffff700 1 --2- 172.21.15.133:0/2128723312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf80cce10 0x7fbcf8140590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:16.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.640+0000 7fbceffff700 1 -- 172.21.15.133:0/2128723312 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce00094b0 con 0x7fbcf815c850 2023-10-11T09:40:16.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.640+0000 7fbcf77fe700 1 --2- 172.21.15.133:0/2128723312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf80cce10 0x7fbcf8140590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:40:16.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.640+0000 7fbcf7fff700 1 --2- 172.21.15.133:0/2128723312 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbcf8071180 0x7fbcf80b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:40:16.649 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.640+0000 7fbceffff700 1 --2- 172.21.15.133:0/2128723312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcf815c850 0x7fbcf8070c40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbce0009b70 tx=0x7fbce0004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:16.649 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.640+0000 7fbcf4ff9700 1 -- 172.21.15.133:0/2128723312 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce0004530 con 0x7fbcf815c850 2023-10-11T09:40:16.649 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.640+0000 7fbcf4ff9700 1 -- 172.21.15.133:0/2128723312 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce001f070 con 0x7fbcf815c850 2023-10-11T09:40:16.649 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.640+0000 7fbcfcd97700 1 -- 172.21.15.133:0/2128723312 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcf815ce70 con 0x7fbcf815c850 2023-10-11T09:40:16.649 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.640+0000 7fbcf4ff9700 1 -- 172.21.15.133:0/2128723312 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce00038e0 con 0x7fbcf815c850 2023-10-11T09:40:16.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.640+0000 7fbcfcd97700 1 -- 172.21.15.133:0/2128723312 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcf815d430 con 0x7fbcf815c850 2023-10-11T09:40:16.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.640+0000 7fbcf4ff9700 1 -- 172.21.15.133:0/2128723312 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fbce0021030 con 0x7fbcf815c850 2023-10-11T09:40:16.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.640+0000 7fbcf4ff9700 1 -- 172.21.15.133:0/2128723312 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fbce0003d40 con 0x7fbcf815c850 2023-10-11T09:40:16.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.640+0000 7fbcfcd97700 1 -- 172.21.15.133:0/2128723312 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcf8065d40 con 0x7fbcf815c850 2023-10-11T09:40:16.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.648+0000 7fbcf4ff9700 1 -- 172.21.15.133:0/2128723312 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fbce0015020 con 0x7fbcf815c850 2023-10-11T09:40:16.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.728+0000 7fbcfcd97700 1 -- 172.21.15.133:0/2128723312 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c1a86a2e-7754-445a-b955-2df5a647ed07", "id": 2} v 0) v1 -- 0x7fbcf815dbc0 con 0x7fbcf815c850 2023-10-11T09:40:16.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.740+0000 7fbcf4ff9700 1 -- 172.21.15.133:0/2128723312 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c1a86a2e-7754-445a-b955-2df5a647ed07", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbce001ac50 con 0x7fbcf815c850 2023-10-11T09:40:16.745 INFO:teuthology.orchestra.run.smithi133.stdout:2 2023-10-11T09:40:16.746 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.740+0000 7fbcfcd97700 1 -- 172.21.15.133:0/2128723312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcf815c850 msgr2=0x7fbcf8070c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:16.746 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.740+0000 7fbcfcd97700 1 --2- 172.21.15.133:0/2128723312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcf815c850 0x7fbcf8070c40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbce0009b70 tx=0x7fbce0004770 comp rx=0 tx=0).stop 2023-10-11T09:40:16.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.740+0000 7fbcfcd97700 1 -- 172.21.15.133:0/2128723312 shutdown_connections 2023-10-11T09:40:16.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.740+0000 7fbcfcd97700 1 --2- 172.21.15.133:0/2128723312 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbcf8071180 0x7fbcf80b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:16.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.740+0000 7fbcfcd97700 1 --2- 172.21.15.133:0/2128723312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcf815c850 0x7fbcf8070c40 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:16.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.740+0000 7fbcfcd97700 1 --2- 172.21.15.133:0/2128723312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf80cce10 0x7fbcf8140590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:16.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.740+0000 7fbcfcd97700 1 -- 172.21.15.133:0/2128723312 >> 172.21.15.133:0/2128723312 conn(0x7fbcf806dd80 msgr2=0x7fbcf80c82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:16.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.740+0000 7fbcfcd97700 1 -- 172.21.15.133:0/2128723312 shutdown_connections 2023-10-11T09:40:16.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.740+0000 7fbcfcd97700 1 -- 172.21.15.133:0/2128723312 wait complete. 2023-10-11T09:40:16.759 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-10-11T09:40:16.754+0000 7f4cf162c2c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-11T09:40:16.766 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph osd new 938b8437-85e7-4833-9e6f-36bcf287c784 3 2023-10-11T09:40:16.841 INFO:tasks.ceph.mgr.x.smithi133.stderr:2023-10-11T09:40:16.836+0000 7f8edb41d2c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-11T09:40:16.925 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-10-11T09:40:16.918+0000 7f4cf162c2c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-11T09:40:16.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.932+0000 7f02fbb4d700 1 Processor -- start 2023-10-11T09:40:16.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.932+0000 7f02fbb4d700 1 -- start start 2023-10-11T09:40:16.939 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.932+0000 7f02fbb4d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02f40b8890 0x7f02f40d2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:16.939 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.932+0000 7f02fbb4d700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f02f40d3510 0x7f02f40d3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:16.939 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.932+0000 7f02fbb4d700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02f40cd690 con 0x7f02f40b84c0 2023-10-11T09:40:16.939 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.932+0000 7f02fbb4d700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02f40d3eb0 con 0x7f02f40d3510 2023-10-11T09:40:16.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.932+0000 7f02fbb4d700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02f40cdea0 con 0x7f02f40b8890 2023-10-11T09:40:16.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.936+0000 7f02f98e9700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f02f40b84c0 0x7f02f40cfda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:42336/0 (socket says 172.21.15.133:42336) 2023-10-11T09:40:16.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.936+0000 7f02f98e9700 1 -- 172.21.15.133:0/268831128 learned_addr learned my addr 172.21.15.133:0/268831128 (peer_addr_for_me v1:172.21.15.133:0/0) 2023-10-11T09:40:16.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.936+0000 7f02fa0ea700 1 --2- 172.21.15.133:0/268831128 >> v2:172.21.15.133:3300/0 conn(0x7f02f40d3510 0x7f02f40d3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:16.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.936+0000 7f02f90e8700 1 --2- 172.21.15.133:0/268831128 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02f40b8890 0x7f02f40d2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:16.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.936+0000 7f02fa0ea700 1 -- 172.21.15.133:0/268831128 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02f40b8890 msgr2=0x7f02f40d2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:16.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.936+0000 7f02fa0ea700 1 --2- 172.21.15.133:0/268831128 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02f40b8890 0x7f02f40d2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:16.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.936+0000 7f02fa0ea700 1 -- 172.21.15.133:0/268831128 >> v1:172.21.15.133:6789/0 conn(0x7f02f40b84c0 legacy=0x7f02f40cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:16.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.936+0000 7f02fa0ea700 1 -- 172.21.15.133:0/268831128 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f40cd8e0 con 0x7f02f40d3510 2023-10-11T09:40:16.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.936+0000 7f02f88e7700 1 -- 172.21.15.133:0/268831128 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2942576673 0 0) 0x7f02f40cd690 con 0x7f02f40b84c0 2023-10-11T09:40:16.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.936+0000 7f02fa0ea700 1 --2- 172.21.15.133:0/268831128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02f40d3510 0x7f02f40d3970 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f02ec009d90 tx=0x7f02ec00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb754fdeb7cb00f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:16.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.936+0000 7f02f88e7700 1 -- 172.21.15.133:0/268831128 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ec00e070 con 0x7f02f40d3510 2023-10-11T09:40:16.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.936+0000 7f02f88e7700 1 -- 172.21.15.133:0/268831128 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02ec00bb20 con 0x7f02f40d3510 2023-10-11T09:40:16.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.936+0000 7f02f88e7700 1 -- 172.21.15.133:0/268831128 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ec012430 con 0x7f02f40d3510 2023-10-11T09:40:16.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.936+0000 7f02fbb4d700 1 -- 172.21.15.133:0/268831128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02f40d3510 msgr2=0x7f02f40d3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:16.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.936+0000 7f02fbb4d700 1 --2- 172.21.15.133:0/268831128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02f40d3510 0x7f02f40d3970 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f02ec009d90 tx=0x7f02ec00b330 comp rx=0 tx=0).stop 2023-10-11T09:40:16.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.936+0000 7f02fbb4d700 1 -- 172.21.15.133:0/268831128 shutdown_connections 2023-10-11T09:40:16.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.936+0000 7f02fbb4d700 1 --2- 172.21.15.133:0/268831128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02f40d3510 0x7f02f40d3970 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:16.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.936+0000 7f02fbb4d700 1 --2- 172.21.15.133:0/268831128 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02f40b8890 0x7f02f40d2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:16.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.936+0000 7f02fbb4d700 1 -- 172.21.15.133:0/268831128 >> 172.21.15.133:0/268831128 conn(0x7f02f406cb50 msgr2=0x7f02f406cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:16.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.936+0000 7f02fbb4d700 1 -- 172.21.15.133:0/268831128 shutdown_connections 2023-10-11T09:40:16.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.936+0000 7f02fbb4d700 1 -- 172.21.15.133:0/268831128 wait complete. 2023-10-11T09:40:16.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.940+0000 7f02fbb4d700 1 Processor -- start 2023-10-11T09:40:16.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.940+0000 7f02fbb4d700 1 -- start start 2023-10-11T09:40:16.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.940+0000 7f02fbb4d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02f40b84c0 0x7f02f40ca800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:16.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.940+0000 7f02fbb4d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02f40b8890 0x7f02f4150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:16.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.940+0000 7f02fbb4d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02f40d3510 0x7f02f40b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:16.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.940+0000 7f02fbb4d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f40cad40 con 0x7f02f40b84c0 2023-10-11T09:40:16.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.940+0000 7f02fbb4d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f41f5410 con 0x7f02f40d3510 2023-10-11T09:40:16.950 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.940+0000 7f02fbb4d700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02f41f56f0 con 0x7f02f40b8890 2023-10-11T09:40:16.950 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.940+0000 7f02fa0ea700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02f40d3510 0x7f02f40b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:16.950 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.940+0000 7f02fa0ea700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02f40d3510 0x7f02f40b7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36274/0 (socket says 172.21.15.133:36274) 2023-10-11T09:40:16.950 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.940+0000 7f02fa0ea700 1 -- 172.21.15.133:0/1095258984 learned_addr learned my addr 172.21.15.133:0/1095258984 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-10-11T09:40:16.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.940+0000 7f02f90e8700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02f40b8890 0x7f02f4150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:16.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.944+0000 7f02f98e9700 1 --2- 172.21.15.133:0/1095258984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02f40b84c0 0x7f02f40ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:16.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.944+0000 7f02f90e8700 1 -- 172.21.15.133:0/1095258984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02f40d3510 msgr2=0x7f02f40b7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:16.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.944+0000 7f02f90e8700 1 --2- 172.21.15.133:0/1095258984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02f40d3510 0x7f02f40b7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:16.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.944+0000 7f02f90e8700 1 -- 172.21.15.133:0/1095258984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02f40b84c0 msgr2=0x7f02f40ca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:16.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.944+0000 7f02f90e8700 1 --2- 172.21.15.133:0/1095258984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02f40b84c0 0x7f02f40ca800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:16.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.944+0000 7f02f90e8700 1 -- 172.21.15.133:0/1095258984 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02ec009a70 con 0x7f02f40b8890 2023-10-11T09:40:16.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.944+0000 7f02f90e8700 1 --2- 172.21.15.133:0/1095258984 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02f40b8890 0x7f02f4150280 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f02e400e910 tx=0x7f02e400ec20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:16.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.944+0000 7f02ea7fc700 1 -- 172.21.15.133:0/1095258984 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e400ee60 con 0x7f02f40b8890 2023-10-11T09:40:16.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.944+0000 7f02ea7fc700 1 -- 172.21.15.133:0/1095258984 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02e400c850 con 0x7f02f40b8890 2023-10-11T09:40:16.953 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.944+0000 7f02fbb4d700 1 -- 172.21.15.133:0/1095258984 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02f41f59d0 con 0x7f02f40b8890 2023-10-11T09:40:16.953 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.944+0000 7f02ea7fc700 1 -- 172.21.15.133:0/1095258984 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e400cc00 con 0x7f02f40b8890 2023-10-11T09:40:16.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.944+0000 7f02fbb4d700 1 -- 172.21.15.133:0/1095258984 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02f41f5f90 con 0x7f02f40b8890 2023-10-11T09:40:16.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.944+0000 7f02ea7fc700 1 -- 172.21.15.133:0/1095258984 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f02e4005d40 con 0x7f02f40b8890 2023-10-11T09:40:16.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.944+0000 7f02ea7fc700 1 -- 172.21.15.133:0/1095258984 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f02e400cd60 con 0x7f02f40b8890 2023-10-11T09:40:16.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.944+0000 7f02fbb4d700 1 -- 172.21.15.133:0/1095258984 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02f4150d40 con 0x7f02f40b8890 2023-10-11T09:40:16.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:16.948+0000 7f02ea7fc700 1 -- 172.21.15.133:0/1095258984 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f02e4021060 con 0x7f02f40b8890 2023-10-11T09:40:17.021 INFO:tasks.ceph.mgr.x.smithi133.stderr:2023-10-11T09:40:17.016+0000 7f8edb41d2c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-11T09:40:17.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.028+0000 7f02fbb4d700 1 -- 172.21.15.133:0/1095258984 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd new", "uuid": "938b8437-85e7-4833-9e6f-36bcf287c784", "id": 3} v 0) v1 -- 0x7f02f40d0df0 con 0x7f02f40b8890 2023-10-11T09:40:17.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.036+0000 7f02ea7fc700 1 -- 172.21.15.133:0/1095258984 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "938b8437-85e7-4833-9e6f-36bcf287c784", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f02e4029020 con 0x7f02f40b8890 2023-10-11T09:40:17.042 INFO:teuthology.orchestra.run.smithi133.stdout:3 2023-10-11T09:40:17.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.040+0000 7f02fbb4d700 1 -- 172.21.15.133:0/1095258984 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02f40b8890 msgr2=0x7f02f4150280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:17.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.040+0000 7f02fbb4d700 1 --2- 172.21.15.133:0/1095258984 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02f40b8890 0x7f02f4150280 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f02e400e910 tx=0x7f02e400ec20 comp rx=0 tx=0).stop 2023-10-11T09:40:17.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.040+0000 7f02fbb4d700 1 -- 172.21.15.133:0/1095258984 shutdown_connections 2023-10-11T09:40:17.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.040+0000 7f02fbb4d700 1 --2- 172.21.15.133:0/1095258984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02f40d3510 0x7f02f40b7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:17.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.040+0000 7f02fbb4d700 1 --2- 172.21.15.133:0/1095258984 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02f40b8890 0x7f02f4150280 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:17.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.040+0000 7f02fbb4d700 1 --2- 172.21.15.133:0/1095258984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02f40b84c0 0x7f02f40ca800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:17.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.040+0000 7f02fbb4d700 1 -- 172.21.15.133:0/1095258984 >> 172.21.15.133:0/1095258984 conn(0x7f02f406cb50 msgr2=0x7f02f415c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:17.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.040+0000 7f02fbb4d700 1 -- 172.21.15.133:0/1095258984 shutdown_connections 2023-10-11T09:40:17.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.040+0000 7f02fbb4d700 1 -- 172.21.15.133:0/1095258984 wait complete. 2023-10-11T09:40:17.063 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph osd new e8f66d61-bee7-4db0-b234-fa865e29d9fd 4 2023-10-11T09:40:17.224 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.220+0000 7f6a39d65700 1 Processor -- start 2023-10-11T09:40:17.224 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.220+0000 7f6a39d65700 1 -- start start 2023-10-11T09:40:17.225 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.220+0000 7f6a39d65700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6a34158430 0x7f6a340710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:17.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.220+0000 7f6a39d65700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f6a340caf90 0x7f6a3415afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:17.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.220+0000 7f6a39d65700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a34071610 con 0x7f6a3406d340 2023-10-11T09:40:17.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.220+0000 7f6a39d65700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a34070c10 con 0x7f6a340caf90 2023-10-11T09:40:17.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.220+0000 7f6a39d65700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a3415e280 con 0x7f6a34158430 2023-10-11T09:40:17.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.220+0000 7f6a337fe700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6a34158430 0x7f6a340710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:17.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.220+0000 7f6a337fe700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6a34158430 0x7f6a340710d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36282/0 (socket says 172.21.15.133:36282) 2023-10-11T09:40:17.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.220+0000 7f6a337fe700 1 -- 172.21.15.133:0/1991166617 learned_addr learned my addr 172.21.15.133:0/1991166617 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-10-11T09:40:17.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.220+0000 7f6a337fe700 1 -- 172.21.15.133:0/1991166617 >> v2:172.21.15.78:3300/0 conn(0x7f6a340caf90 msgr2=0x7f6a3415afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:17.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.220+0000 7f6a337fe700 1 --2- 172.21.15.133:0/1991166617 >> v2:172.21.15.78:3300/0 conn(0x7f6a340caf90 0x7f6a3415afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:17.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.220+0000 7f6a337fe700 1 -- 172.21.15.133:0/1991166617 >> v1:172.21.15.133:6789/0 conn(0x7f6a3406d340 legacy=0x7f6a340ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:17.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.220+0000 7f6a337fe700 1 -- 172.21.15.133:0/1991166617 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a3415e560 con 0x7f6a34158430 2023-10-11T09:40:17.230 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.220+0000 7f6a337fe700 1 --2- 172.21.15.133:0/1991166617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a34158430 0x7f6a340710d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6a3406fd00 tx=0x7f6a24009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7adc328831843b3a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:17.230 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.220+0000 7f6a32ffd700 1 -- 172.21.15.133:0/1991166617 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2400b8a0 con 0x7f6a34158430 2023-10-11T09:40:17.230 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.220+0000 7f6a32ffd700 1 -- 172.21.15.133:0/1991166617 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2400ba00 con 0x7f6a34158430 2023-10-11T09:40:17.230 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.220+0000 7f6a32ffd700 1 -- 172.21.15.133:0/1991166617 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2400bd90 con 0x7f6a34158430 2023-10-11T09:40:17.230 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.224+0000 7f6a39d65700 1 -- 172.21.15.133:0/1991166617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a34158430 msgr2=0x7f6a340710d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:17.231 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.224+0000 7f6a39d65700 1 --2- 172.21.15.133:0/1991166617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a34158430 0x7f6a340710d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6a3406fd00 tx=0x7f6a24009bf0 comp rx=0 tx=0).stop 2023-10-11T09:40:17.231 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.224+0000 7f6a39d65700 1 -- 172.21.15.133:0/1991166617 shutdown_connections 2023-10-11T09:40:17.231 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.224+0000 7f6a39d65700 1 --2- 172.21.15.133:0/1991166617 >> v2:172.21.15.78:3300/0 conn(0x7f6a340caf90 0x7f6a3415afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:17.231 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.224+0000 7f6a39d65700 1 --2- 172.21.15.133:0/1991166617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a34158430 0x7f6a340710d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:17.231 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.224+0000 7f6a39d65700 1 -- 172.21.15.133:0/1991166617 >> 172.21.15.133:0/1991166617 conn(0x7f6a340cecc0 msgr2=0x7f6a340cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:17.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.224+0000 7f6a39d65700 1 -- 172.21.15.133:0/1991166617 shutdown_connections 2023-10-11T09:40:17.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.224+0000 7f6a39d65700 1 -- 172.21.15.133:0/1991166617 wait complete. 2023-10-11T09:40:17.234 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.224+0000 7f6a39d65700 1 Processor -- start 2023-10-11T09:40:17.234 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.224+0000 7f6a39d65700 1 -- start start 2023-10-11T09:40:17.234 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.228+0000 7f6a39d65700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a3406d340 0x7f6a341ef7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:17.234 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.228+0000 7f6a39d65700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a340caf90 0x7f6a34159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:17.235 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.228+0000 7f6a39d65700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6a34158430 0x7f6a341562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:17.235 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.228+0000 7f6a39d65700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a341f0d00 con 0x7f6a3406d340 2023-10-11T09:40:17.235 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.228+0000 7f6a39d65700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a340b0240 con 0x7f6a340caf90 2023-10-11T09:40:17.235 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.228+0000 7f6a39d65700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a341f00f0 con 0x7f6a34158430 2023-10-11T09:40:17.235 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.228+0000 7f6a2affd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a340caf90 0x7f6a34159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:17.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.228+0000 7f6a33fff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6a34158430 0x7f6a341562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:17.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.228+0000 7f6a2affd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a340caf90 0x7f6a34159470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36290/0 (socket says 172.21.15.133:36290) 2023-10-11T09:40:17.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.228+0000 7f6a2affd700 1 -- 172.21.15.133:0/1945588438 learned_addr learned my addr 172.21.15.133:0/1945588438 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-10-11T09:40:17.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.228+0000 7f6a337fe700 1 --2- 172.21.15.133:0/1945588438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a3406d340 0x7f6a341ef7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:17.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.228+0000 7f6a2affd700 1 -- 172.21.15.133:0/1945588438 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6a34158430 msgr2=0x7f6a341562d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:17.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.228+0000 7f6a2affd700 1 --2- 172.21.15.133:0/1945588438 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6a34158430 0x7f6a341562d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:17.237 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.228+0000 7f6a2affd700 1 -- 172.21.15.133:0/1945588438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a3406d340 msgr2=0x7f6a341ef7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:17.237 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.228+0000 7f6a2affd700 1 --2- 172.21.15.133:0/1945588438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a3406d340 0x7f6a341ef7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:17.237 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.228+0000 7f6a2affd700 1 -- 172.21.15.133:0/1945588438 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a24009570 con 0x7f6a340caf90 2023-10-11T09:40:17.237 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.228+0000 7f6a33fff700 1 --2- 172.21.15.133:0/1945588438 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6a34158430 0x7f6a341562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:40:17.237 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.228+0000 7f6a2affd700 1 --2- 172.21.15.133:0/1945588438 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a340caf90 0x7f6a34159470 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c00c500 tx=0x7f6a1c00c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:17.238 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.228+0000 7f6a30ff9700 1 -- 172.21.15.133:0/1945588438 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c0052d0 con 0x7f6a340caf90 2023-10-11T09:40:17.238 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.228+0000 7f6a30ff9700 1 -- 172.21.15.133:0/1945588438 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a1c005430 con 0x7f6a340caf90 2023-10-11T09:40:17.238 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.228+0000 7f6a30ff9700 1 -- 172.21.15.133:0/1945588438 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c0055a0 con 0x7f6a340caf90 2023-10-11T09:40:17.239 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.228+0000 7f6a39d65700 1 -- 172.21.15.133:0/1945588438 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a341f03d0 con 0x7f6a340caf90 2023-10-11T09:40:17.239 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.228+0000 7f6a39d65700 1 -- 172.21.15.133:0/1945588438 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a341f0990 con 0x7f6a340caf90 2023-10-11T09:40:17.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.228+0000 7f6a30ff9700 1 -- 172.21.15.133:0/1945588438 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6a1c003be0 con 0x7f6a340caf90 2023-10-11T09:40:17.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.228+0000 7f6a30ff9700 1 -- 172.21.15.133:0/1945588438 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f6a1c011d60 con 0x7f6a340caf90 2023-10-11T09:40:17.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.232+0000 7f6a39d65700 1 -- 172.21.15.133:0/1945588438 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a3406eb80 con 0x7f6a340caf90 2023-10-11T09:40:17.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.236+0000 7f6a30ff9700 1 -- 172.21.15.133:0/1945588438 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f6a1c017070 con 0x7f6a340caf90 2023-10-11T09:40:17.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.312+0000 7f6a39d65700 1 -- 172.21.15.133:0/1945588438 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e8f66d61-bee7-4db0-b234-fa865e29d9fd", "id": 4} v 0) v1 -- 0x7f6a34065db0 con 0x7f6a340caf90 2023-10-11T09:40:17.320 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-10-11T09:40:17.314+0000 7f4cf162c2c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-11T09:40:17.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.324+0000 7f6a30ff9700 1 -- 172.21.15.133:0/1945588438 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e8f66d61-bee7-4db0-b234-fa865e29d9fd", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6a1c011590 con 0x7f6a340caf90 2023-10-11T09:40:17.329 INFO:teuthology.orchestra.run.smithi133.stdout:4 2023-10-11T09:40:17.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.324+0000 7f6a39d65700 1 -- 172.21.15.133:0/1945588438 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a340caf90 msgr2=0x7f6a34159470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:17.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.324+0000 7f6a39d65700 1 --2- 172.21.15.133:0/1945588438 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a340caf90 0x7f6a34159470 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c00c500 tx=0x7f6a1c00c810 comp rx=0 tx=0).stop 2023-10-11T09:40:17.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.324+0000 7f6a39d65700 1 -- 172.21.15.133:0/1945588438 shutdown_connections 2023-10-11T09:40:17.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.324+0000 7f6a39d65700 1 --2- 172.21.15.133:0/1945588438 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6a34158430 0x7f6a341562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:17.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.324+0000 7f6a39d65700 1 --2- 172.21.15.133:0/1945588438 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a340caf90 0x7f6a34159470 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:17.332 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.324+0000 7f6a39d65700 1 --2- 172.21.15.133:0/1945588438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a3406d340 0x7f6a341ef7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:17.332 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.324+0000 7f6a39d65700 1 -- 172.21.15.133:0/1945588438 >> 172.21.15.133:0/1945588438 conn(0x7f6a340cecc0 msgr2=0x7f6a340717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:17.332 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.324+0000 7f6a39d65700 1 -- 172.21.15.133:0/1945588438 shutdown_connections 2023-10-11T09:40:17.332 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.324+0000 7f6a39d65700 1 -- 172.21.15.133:0/1945588438 wait complete. 2023-10-11T09:40:17.351 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph osd new 91314bec-7752-484f-9df5-716e9fe253de 5 2023-10-11T09:40:17.459 INFO:tasks.ceph.mgr.x.smithi133.stderr:2023-10-11T09:40:17.452+0000 7f8edb41d2c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-11T09:40:17.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.504+0000 7fe01b78b700 1 Processor -- start 2023-10-11T09:40:17.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.504+0000 7fe01b78b700 1 -- start start 2023-10-11T09:40:17.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.508+0000 7fe01b78b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe01406d680 0x7fe01415db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:17.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.508+0000 7fe01b78b700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0140c9280 con 0x7fe0140c9b90 2023-10-11T09:40:17.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.508+0000 7fe01b78b700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0140cd430 con 0x7fe0141596b0 2023-10-11T09:40:17.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.508+0000 7fe01b78b700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0140c9fe0 con 0x7fe01406d680 2023-10-11T09:40:17.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.508+0000 7fe019527700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe0141596b0 0x7fe0140ccd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:42356/0 (socket says 172.21.15.133:42356) 2023-10-11T09:40:17.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.508+0000 7fe019527700 1 -- 172.21.15.133:0/2439611003 learned_addr learned my addr 172.21.15.133:0/2439611003 (peer_addr_for_me v1:172.21.15.133:0/0) 2023-10-11T09:40:17.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.508+0000 7fe00bfff700 1 -- 172.21.15.133:0/2439611003 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3756818006 0 0) 0x7fe0140cd430 con 0x7fe0141596b0 2023-10-11T09:40:17.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.508+0000 7fe019d28700 1 --2- 172.21.15.133:0/2439611003 >> v2:172.21.15.78:3300/0 conn(0x7fe01406d680 0x7fe01415db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:17.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.508+0000 7fe00bfff700 1 -- 172.21.15.133:0/2439611003 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0000035d0 con 0x7fe0141596b0 2023-10-11T09:40:17.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.508+0000 7fe00bfff700 1 -- 172.21.15.133:0/2439611003 <== mon.1 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2834497482 0 0) 0x7fe0000035d0 con 0x7fe0141596b0 2023-10-11T09:40:17.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.508+0000 7fe00bfff700 1 -- 172.21.15.133:0/2439611003 >> v2:172.21.15.78:3300/0 conn(0x7fe01406d680 msgr2=0x7fe01415db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:17.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.508+0000 7fe00bfff700 1 --2- 172.21.15.133:0/2439611003 >> v2:172.21.15.78:3300/0 conn(0x7fe01406d680 0x7fe01415db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:17.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.508+0000 7fe00bfff700 1 -- 172.21.15.133:0/2439611003 >> v1:172.21.15.78:6789/0 conn(0x7fe0140c9b90 legacy=0x7fe01406cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:17.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.508+0000 7fe00bfff700 1 -- 172.21.15.133:0/2439611003 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0140c94d0 con 0x7fe0141596b0 2023-10-11T09:40:17.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.508+0000 7fe00bfff700 1 -- 172.21.15.133:0/2439611003 <== mon.1 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7fe0100019f0 con 0x7fe0141596b0 2023-10-11T09:40:17.518 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.508+0000 7fe00bfff700 1 -- 172.21.15.133:0/2439611003 >> v1:172.21.15.133:6789/0 conn(0x7fe0141596b0 legacy=0x7fe0140ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:17.518 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.508+0000 7fe00bfff700 1 --2- 172.21.15.133:0/2439611003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe000003a70 0x7fe000003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:17.518 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.508+0000 7fe00bfff700 1 --2- 172.21.15.133:0/2439611003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe000004870 0x7fe000004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:17.518 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.508+0000 7fe00bfff700 1 --2- 172.21.15.133:0/2439611003 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe000005270 0x7fe00000fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:17.518 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.508+0000 7fe00bfff700 1 -- 172.21.15.133:0/2439611003 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0140c9fe0 con 0x7fe000004870 2023-10-11T09:40:17.519 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.508+0000 7fe00bfff700 1 -- 172.21.15.133:0/2439611003 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe000005c10 con 0x7fe000003a70 2023-10-11T09:40:17.519 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.508+0000 7fe00bfff700 1 -- 172.21.15.133:0/2439611003 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe000005dd0 con 0x7fe000005270 2023-10-11T09:40:17.519 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.508+0000 7fe019527700 1 --2- 172.21.15.133:0/2439611003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe000003a70 0x7fe000003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:17.519 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.508+0000 7fe019d28700 1 --2- 172.21.15.133:0/2439611003 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe000005270 0x7fe00000fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:17.519 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.508+0000 7fe018d26700 1 --2- 172.21.15.133:0/2439611003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe000004870 0x7fe000004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:17.520 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.508+0000 7fe019527700 1 -- 172.21.15.133:0/2439611003 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe000005270 msgr2=0x7fe00000fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:17.520 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.508+0000 7fe019527700 1 --2- 172.21.15.133:0/2439611003 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe000005270 0x7fe00000fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:17.520 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.508+0000 7fe019527700 1 -- 172.21.15.133:0/2439611003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe000004870 msgr2=0x7fe000004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:17.520 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.508+0000 7fe019527700 1 --2- 172.21.15.133:0/2439611003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe000004870 0x7fe000004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:17.520 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.508+0000 7fe019527700 1 -- 172.21.15.133:0/2439611003 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0000035d0 con 0x7fe000003a70 2023-10-11T09:40:17.521 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.508+0000 7fe018d26700 1 --2- 172.21.15.133:0/2439611003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe000004870 0x7fe000004ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:40:17.521 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.508+0000 7fe019d28700 1 -- 172.21.15.133:0/2439611003 reap_dead start 2023-10-11T09:40:17.521 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.508+0000 7fe019527700 1 --2- 172.21.15.133:0/2439611003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe000003a70 0x7fe000003e80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe01000a760 tx=0x7fe0000058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b764bb1f8ef5bde server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:17.521 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.512+0000 7fe00bfff700 1 -- 172.21.15.133:0/2439611003 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01000a250 con 0x7fe000003a70 2023-10-11T09:40:17.522 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.512+0000 7fe00bfff700 1 -- 172.21.15.133:0/2439611003 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0100070d0 con 0x7fe000003a70 2023-10-11T09:40:17.522 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.512+0000 7fe00bfff700 1 -- 172.21.15.133:0/2439611003 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe010007480 con 0x7fe000003a70 2023-10-11T09:40:17.525 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.512+0000 7fe01b78b700 1 -- 172.21.15.133:0/2439611003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe000003a70 msgr2=0x7fe000003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:17.525 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.512+0000 7fe01b78b700 1 --2- 172.21.15.133:0/2439611003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe000003a70 0x7fe000003e80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe01000a760 tx=0x7fe0000058a0 comp rx=0 tx=0).stop 2023-10-11T09:40:17.525 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.512+0000 7fe01b78b700 1 -- 172.21.15.133:0/2439611003 shutdown_connections 2023-10-11T09:40:17.525 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.512+0000 7fe01b78b700 1 --2- 172.21.15.133:0/2439611003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe000003a70 0x7fe000003e80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:17.526 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.512+0000 7fe01b78b700 1 -- 172.21.15.133:0/2439611003 >> 172.21.15.133:0/2439611003 conn(0x7fe0140cecc0 msgr2=0x7fe0140cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:17.526 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.512+0000 7fe01b78b700 1 -- 172.21.15.133:0/2439611003 shutdown_connections 2023-10-11T09:40:17.526 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.512+0000 7fe01b78b700 1 -- 172.21.15.133:0/2439611003 wait complete. 2023-10-11T09:40:17.526 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.516+0000 7fe01b78b700 1 Processor -- start 2023-10-11T09:40:17.526 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.516+0000 7fe01b78b700 1 -- start start 2023-10-11T09:40:17.527 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.516+0000 7fe01b78b700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe000003a70 0x7fe01414e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:17.527 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.516+0000 7fe01b78b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe014158110 0x7fe01414b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:17.527 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.516+0000 7fe01b78b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0140afdb0 0x7fe0140b0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:17.527 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.516+0000 7fe01b78b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0140af920 con 0x7fe014158110 2023-10-11T09:40:17.527 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.516+0000 7fe01b78b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01414e420 con 0x7fe0140afdb0 2023-10-11T09:40:17.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.516+0000 7fe01b78b700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe014158540 con 0x7fe000003a70 2023-10-11T09:40:17.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.516+0000 7fe019527700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe000003a70 0x7fe01414e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:17.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.516+0000 7fe019527700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe000003a70 0x7fe01414e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:47892/0 (socket says 172.21.15.133:47892) 2023-10-11T09:40:17.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.516+0000 7fe019527700 1 -- 172.21.15.133:0/1375902716 learned_addr learned my addr 172.21.15.133:0/1375902716 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-10-11T09:40:17.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.516+0000 7fe019d28700 1 --2- 172.21.15.133:0/1375902716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0140afdb0 0x7fe0140b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:17.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.516+0000 7fe018d26700 1 --2- 172.21.15.133:0/1375902716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe014158110 0x7fe01414b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:17.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.516+0000 7fe019527700 1 -- 172.21.15.133:0/1375902716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0140afdb0 msgr2=0x7fe0140b0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:17.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.516+0000 7fe019527700 1 --2- 172.21.15.133:0/1375902716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0140afdb0 0x7fe0140b0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:17.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.516+0000 7fe019527700 1 -- 172.21.15.133:0/1375902716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe014158110 msgr2=0x7fe01414b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:17.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.516+0000 7fe019527700 1 --2- 172.21.15.133:0/1375902716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe014158110 0x7fe01414b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:17.530 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.516+0000 7fe019527700 1 -- 172.21.15.133:0/1375902716 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0140af2f0 con 0x7fe000003a70 2023-10-11T09:40:17.530 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.516+0000 7fe019527700 1 --2- 172.21.15.133:0/1375902716 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe000003a70 0x7fe01414e640 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe0100074f0 tx=0x7fe010001cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:17.530 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.520+0000 7fe009ffb700 1 -- 172.21.15.133:0/1375902716 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe010024070 con 0x7fe000003a70 2023-10-11T09:40:17.530 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.520+0000 7fe009ffb700 1 -- 172.21.15.133:0/1375902716 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe010007990 con 0x7fe000003a70 2023-10-11T09:40:17.530 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.520+0000 7fe01b78b700 1 -- 172.21.15.133:0/1375902716 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0140b4970 con 0x7fe000003a70 2023-10-11T09:40:17.531 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.520+0000 7fe009ffb700 1 -- 172.21.15.133:0/1375902716 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01000a250 con 0x7fe000003a70 2023-10-11T09:40:17.531 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.520+0000 7fe01b78b700 1 -- 172.21.15.133:0/1375902716 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0140af5b0 con 0x7fe000003a70 2023-10-11T09:40:17.531 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.520+0000 7fe009ffb700 1 -- 172.21.15.133:0/1375902716 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe01001a070 con 0x7fe000003a70 2023-10-11T09:40:17.531 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.520+0000 7fe009ffb700 1 -- 172.21.15.133:0/1375902716 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fe010012400 con 0x7fe000003a70 2023-10-11T09:40:17.531 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.520+0000 7fe01b78b700 1 -- 172.21.15.133:0/1375902716 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0141586b0 con 0x7fe000003a70 2023-10-11T09:40:17.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.524+0000 7fe009ffb700 1 -- 172.21.15.133:0/1375902716 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fe010018070 con 0x7fe000003a70 2023-10-11T09:40:17.542 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-10-11T09:40:17.538+0000 7f4cf162c2c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-11T09:40:17.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.600+0000 7fe01b78b700 1 -- 172.21.15.133:0/1375902716 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd new", "uuid": "91314bec-7752-484f-9df5-716e9fe253de", "id": 5} v 0) v1 -- 0x7fe014065db0 con 0x7fe000003a70 2023-10-11T09:40:17.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.608+0000 7fe009ffb700 1 -- 172.21.15.133:0/1375902716 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "91314bec-7752-484f-9df5-716e9fe253de", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe0100380b0 con 0x7fe000003a70 2023-10-11T09:40:17.613 INFO:teuthology.orchestra.run.smithi133.stdout:5 2023-10-11T09:40:17.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.608+0000 7fe01b78b700 1 -- 172.21.15.133:0/1375902716 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe000003a70 msgr2=0x7fe01414e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:17.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.608+0000 7fe01b78b700 1 --2- 172.21.15.133:0/1375902716 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe000003a70 0x7fe01414e640 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe0100074f0 tx=0x7fe010001cb0 comp rx=0 tx=0).stop 2023-10-11T09:40:17.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.608+0000 7fe01b78b700 1 -- 172.21.15.133:0/1375902716 shutdown_connections 2023-10-11T09:40:17.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.608+0000 7fe01b78b700 1 --2- 172.21.15.133:0/1375902716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0140afdb0 0x7fe0140b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:17.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.608+0000 7fe01b78b700 1 --2- 172.21.15.133:0/1375902716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe014158110 0x7fe01414b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:17.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.608+0000 7fe01b78b700 1 --2- 172.21.15.133:0/1375902716 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe000003a70 0x7fe01414e640 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:17.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.608+0000 7fe01b78b700 1 -- 172.21.15.133:0/1375902716 >> 172.21.15.133:0/1375902716 conn(0x7fe0140cecc0 msgr2=0x7fe0140706f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:17.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.608+0000 7fe01b78b700 1 -- 172.21.15.133:0/1375902716 shutdown_connections 2023-10-11T09:40:17.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.608+0000 7fe01b78b700 1 -- 172.21.15.133:0/1375902716 wait complete. 2023-10-11T09:40:17.634 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph osd new 6cf9c466-c7e1-46a2-8b2e-feea51f253e3 6 2023-10-11T09:40:17.635 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-10-11T09:40:17.630+0000 7f4cf162c2c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-11T09:40:17.687 INFO:tasks.ceph.mgr.x.smithi133.stderr:2023-10-11T09:40:17.680+0000 7f8edb41d2c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-11T09:40:17.710 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-10-11T09:40:17.706+0000 7f4cf162c2c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-11T09:40:17.786 INFO:tasks.ceph.mgr.x.smithi133.stderr:2023-10-11T09:40:17.780+0000 7f8edb41d2c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-11T09:40:17.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.788+0000 7f9acbda7700 1 Processor -- start 2023-10-11T09:40:17.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.788+0000 7f9acbda7700 1 -- start start 2023-10-11T09:40:17.795 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.792+0000 7f9acbda7700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f9ac40b84c0 0x7f9ac40cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:17.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.792+0000 7f9acbda7700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ac40cd690 con 0x7f9ac4160a80 2023-10-11T09:40:17.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.792+0000 7f9acbda7700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ac40d02e0 con 0x7f9ac40b8890 2023-10-11T09:40:17.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.792+0000 7f9acbda7700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ac40cdea0 con 0x7f9ac40b84c0 2023-10-11T09:40:17.797 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.792+0000 7f9ac9342700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f9ac40b8890 0x7f9ac4160370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:42362/0 (socket says 172.21.15.133:42362) 2023-10-11T09:40:17.797 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.792+0000 7f9ac9342700 1 -- 172.21.15.133:0/1587570129 learned_addr learned my addr 172.21.15.133:0/1587570129 (peer_addr_for_me v1:172.21.15.133:0/0) 2023-10-11T09:40:17.797 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.792+0000 7f9ac9b43700 1 --2- 172.21.15.133:0/1587570129 >> v2:172.21.15.78:3300/0 conn(0x7f9ac40b84c0 0x7f9ac40cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:17.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.792+0000 7f9ac8b41700 1 -- 172.21.15.133:0/1587570129 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4239296516 0 0) 0x7f9ac40d02e0 con 0x7f9ac40b8890 2023-10-11T09:40:17.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.792+0000 7f9ac8b41700 1 -- 172.21.15.133:0/1587570129 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ab00035d0 con 0x7f9ac40b8890 2023-10-11T09:40:17.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.792+0000 7f9ac9b43700 1 -- 172.21.15.133:0/1587570129 >> v1:172.21.15.133:6789/0 conn(0x7f9ac40b8890 legacy=0x7f9ac4160370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:17.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.792+0000 7f9ac9b43700 1 -- 172.21.15.133:0/1587570129 >> v1:172.21.15.78:6789/0 conn(0x7f9ac4160a80 legacy=0x7f9ac40d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:17.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.792+0000 7f9ac9b43700 1 -- 172.21.15.133:0/1587570129 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac40cd8e0 con 0x7f9ac40b84c0 2023-10-11T09:40:17.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.792+0000 7f9ac9b43700 1 --2- 172.21.15.133:0/1587570129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ac40b84c0 0x7f9ac40cfda0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0009d90 tx=0x7f9ac000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=113f208b49e71e2b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:17.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.792+0000 7f9ac8b41700 1 -- 172.21.15.133:0/1587570129 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac000e070 con 0x7f9ac40b84c0 2023-10-11T09:40:17.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.792+0000 7f9ac8b41700 1 -- 172.21.15.133:0/1587570129 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac000bb20 con 0x7f9ac40b84c0 2023-10-11T09:40:17.802 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.792+0000 7f9ac8b41700 1 -- 172.21.15.133:0/1587570129 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0012430 con 0x7f9ac40b84c0 2023-10-11T09:40:17.804 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.792+0000 7f9acbda7700 1 -- 172.21.15.133:0/1587570129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ac40b84c0 msgr2=0x7f9ac40cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:17.804 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.792+0000 7f9acbda7700 1 --2- 172.21.15.133:0/1587570129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ac40b84c0 0x7f9ac40cfda0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0009d90 tx=0x7f9ac000b330 comp rx=0 tx=0).stop 2023-10-11T09:40:17.805 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.792+0000 7f9acbda7700 1 -- 172.21.15.133:0/1587570129 shutdown_connections 2023-10-11T09:40:17.805 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.792+0000 7f9acbda7700 1 --2- 172.21.15.133:0/1587570129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ac40b84c0 0x7f9ac40cfda0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:17.805 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.792+0000 7f9acbda7700 1 -- 172.21.15.133:0/1587570129 >> 172.21.15.133:0/1587570129 conn(0x7f9ac406cb50 msgr2=0x7f9ac406cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:17.806 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.796+0000 7f9acbda7700 1 -- 172.21.15.133:0/1587570129 shutdown_connections 2023-10-11T09:40:17.806 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.796+0000 7f9acbda7700 1 -- 172.21.15.133:0/1587570129 wait complete. 2023-10-11T09:40:17.806 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.796+0000 7f9acbda7700 1 Processor -- start 2023-10-11T09:40:17.806 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.796+0000 7f9acbda7700 1 -- start start 2023-10-11T09:40:17.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.796+0000 7f9acbda7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ac40b84c0 0x7f9ac414a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:17.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.796+0000 7f9acbda7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ac40b8890 0x7f9ac415ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:17.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.796+0000 7f9acbda7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9ac4160a80 0x7f9ac40cab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:17.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.796+0000 7f9acbda7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac414aa00 con 0x7f9ac40b8890 2023-10-11T09:40:17.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.796+0000 7f9acbda7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac40ca8f0 con 0x7f9ac40b84c0 2023-10-11T09:40:17.808 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.796+0000 7f9acbda7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac40ca610 con 0x7f9ac4160a80 2023-10-11T09:40:17.808 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.800+0000 7f9ac9b43700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ac40b84c0 0x7f9ac414a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:17.808 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.800+0000 7f9aca344700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9ac4160a80 0x7f9ac40cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:17.808 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.800+0000 7f9ac9b43700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ac40b84c0 0x7f9ac414a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36318/0 (socket says 172.21.15.133:36318) 2023-10-11T09:40:17.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.800+0000 7f9ac9b43700 1 -- 172.21.15.133:0/2784477844 learned_addr learned my addr 172.21.15.133:0/2784477844 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-10-11T09:40:17.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.800+0000 7f9ac9342700 1 --2- 172.21.15.133:0/2784477844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ac40b8890 0x7f9ac415ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:17.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.800+0000 7f9aca344700 1 -- 172.21.15.133:0/2784477844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ac40b84c0 msgr2=0x7f9ac414a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:17.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.800+0000 7f9aca344700 1 --2- 172.21.15.133:0/2784477844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ac40b84c0 0x7f9ac414a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:17.810 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.800+0000 7f9aca344700 1 -- 172.21.15.133:0/2784477844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ac40b8890 msgr2=0x7f9ac415ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:17.810 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.800+0000 7f9aca344700 1 --2- 172.21.15.133:0/2784477844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ac40b8890 0x7f9ac415ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:17.810 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.800+0000 7f9aca344700 1 -- 172.21.15.133:0/2784477844 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac0009a70 con 0x7f9ac4160a80 2023-10-11T09:40:17.811 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.800+0000 7f9aca344700 1 --2- 172.21.15.133:0/2784477844 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9ac4160a80 0x7f9ac40cab60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00c390 tx=0x7f9abc00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:17.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.800+0000 7f9aba7fc700 1 -- 172.21.15.133:0/2784477844 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc0052d0 con 0x7f9ac4160a80 2023-10-11T09:40:17.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.800+0000 7f9aba7fc700 1 -- 172.21.15.133:0/2784477844 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9abc005430 con 0x7f9ac4160a80 2023-10-11T09:40:17.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.800+0000 7f9aba7fc700 1 -- 172.21.15.133:0/2784477844 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc0055a0 con 0x7f9ac4160a80 2023-10-11T09:40:17.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.800+0000 7f9acbda7700 1 -- 172.21.15.133:0/2784477844 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ac415e270 con 0x7f9ac4160a80 2023-10-11T09:40:17.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.800+0000 7f9acbda7700 1 -- 172.21.15.133:0/2784477844 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac415e830 con 0x7f9ac4160a80 2023-10-11T09:40:17.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.800+0000 7f9aba7fc700 1 -- 172.21.15.133:0/2784477844 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9abc003be0 con 0x7f9ac4160a80 2023-10-11T09:40:17.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.800+0000 7f9aba7fc700 1 -- 172.21.15.133:0/2784477844 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f9abc012420 con 0x7f9ac4160a80 2023-10-11T09:40:17.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.800+0000 7f9acbda7700 1 -- 172.21.15.133:0/2784477844 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac415ea50 con 0x7f9ac4160a80 2023-10-11T09:40:17.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.804+0000 7f9aba7fc700 1 -- 172.21.15.133:0/2784477844 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f9abc017070 con 0x7f9ac4160a80 2023-10-11T09:40:17.872 INFO:tasks.ceph.mgr.x.smithi133.stderr:2023-10-11T09:40:17.868+0000 7f8edb41d2c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-11T09:40:17.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.876+0000 7f9acbda7700 1 -- 172.21.15.133:0/2784477844 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd new", "uuid": "6cf9c466-c7e1-46a2-8b2e-feea51f253e3", "id": 6} v 0) v1 -- 0x7f9ac408bff0 con 0x7f9ac4160a80 2023-10-11T09:40:17.893 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.888+0000 7f9aba7fc700 1 -- 172.21.15.133:0/2784477844 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6cf9c466-c7e1-46a2-8b2e-feea51f253e3", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9abc01c440 con 0x7f9ac4160a80 2023-10-11T09:40:17.893 INFO:teuthology.orchestra.run.smithi133.stdout:6 2023-10-11T09:40:17.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.888+0000 7f9acbda7700 1 -- 172.21.15.133:0/2784477844 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9ac4160a80 msgr2=0x7f9ac40cab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:17.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.888+0000 7f9acbda7700 1 --2- 172.21.15.133:0/2784477844 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9ac4160a80 0x7f9ac40cab60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00c390 tx=0x7f9abc00c6a0 comp rx=0 tx=0).stop 2023-10-11T09:40:17.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.888+0000 7f9acbda7700 1 -- 172.21.15.133:0/2784477844 shutdown_connections 2023-10-11T09:40:17.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.888+0000 7f9acbda7700 1 --2- 172.21.15.133:0/2784477844 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9ac4160a80 0x7f9ac40cab60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:17.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.888+0000 7f9acbda7700 1 --2- 172.21.15.133:0/2784477844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ac40b8890 0x7f9ac415ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:17.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.888+0000 7f9acbda7700 1 --2- 172.21.15.133:0/2784477844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ac40b84c0 0x7f9ac414a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:17.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.888+0000 7f9acbda7700 1 -- 172.21.15.133:0/2784477844 >> 172.21.15.133:0/2784477844 conn(0x7f9ac406cb50 msgr2=0x7f9ac40b3790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:17.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.888+0000 7f9acbda7700 1 -- 172.21.15.133:0/2784477844 shutdown_connections 2023-10-11T09:40:17.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:17.888+0000 7f9acbda7700 1 -- 172.21.15.133:0/2784477844 wait complete. 2023-10-11T09:40:17.914 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph osd new c62a10af-8966-42af-b799-3422ad19ae0d 7 2023-10-11T09:40:18.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.064+0000 7fe694cee700 1 Processor -- start 2023-10-11T09:40:18.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.064+0000 7fe694cee700 1 -- start start 2023-10-11T09:40:18.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.064+0000 7fe694cee700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6900d2350 0x7fe6900d36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:18.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.064+0000 7fe694cee700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6900cd690 con 0x7fe6900b8890 2023-10-11T09:40:18.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.064+0000 7fe694cee700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6900cd8e0 con 0x7fe6900b84c0 2023-10-11T09:40:18.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.064+0000 7fe694cee700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6900cdea0 con 0x7fe6900d2350 2023-10-11T09:40:18.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.068+0000 7fe68e59c700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe6900b84c0 0x7fe6900cfda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:42366/0 (socket says 172.21.15.133:42366) 2023-10-11T09:40:18.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.068+0000 7fe68e59c700 1 -- 172.21.15.133:0/3302206341 learned_addr learned my addr 172.21.15.133:0/3302206341 (peer_addr_for_me v1:172.21.15.133:0/0) 2023-10-11T09:40:18.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.068+0000 7fe68ed9d700 1 --2- 172.21.15.133:0/3302206341 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6900d2350 0x7fe6900d36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:18.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.068+0000 7fe68ed9d700 1 -- 172.21.15.133:0/3302206341 >> v1:172.21.15.133:6789/0 conn(0x7fe6900b84c0 legacy=0x7fe6900cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:18.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.068+0000 7fe68ed9d700 1 -- 172.21.15.133:0/3302206341 >> v1:172.21.15.78:6789/0 conn(0x7fe6900b8890 legacy=0x7fe6900d2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:18.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.068+0000 7fe68ed9d700 1 -- 172.21.15.133:0/3302206341 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6900cdbc0 con 0x7fe6900d2350 2023-10-11T09:40:18.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.068+0000 7fe68ed9d700 1 --2- 172.21.15.133:0/3302206341 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6900d2350 0x7fe6900d36e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe688009dc0 tx=0x7fe68800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f872992675b407 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:18.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.068+0000 7fe68d59a700 1 -- 172.21.15.133:0/3302206341 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68800e070 con 0x7fe6900d2350 2023-10-11T09:40:18.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.068+0000 7fe68d59a700 1 -- 172.21.15.133:0/3302206341 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe68800bbd0 con 0x7fe6900d2350 2023-10-11T09:40:18.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.068+0000 7fe68d59a700 1 -- 172.21.15.133:0/3302206341 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe688012430 con 0x7fe6900d2350 2023-10-11T09:40:18.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.068+0000 7fe694cee700 1 -- 172.21.15.133:0/3302206341 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6900d2350 msgr2=0x7fe6900d36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:18.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.068+0000 7fe694cee700 1 --2- 172.21.15.133:0/3302206341 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6900d2350 0x7fe6900d36e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe688009dc0 tx=0x7fe68800b330 comp rx=0 tx=0).stop 2023-10-11T09:40:18.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.068+0000 7fe694cee700 1 -- 172.21.15.133:0/3302206341 shutdown_connections 2023-10-11T09:40:18.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.068+0000 7fe694cee700 1 --2- 172.21.15.133:0/3302206341 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6900d2350 0x7fe6900d36e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:18.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.068+0000 7fe694cee700 1 -- 172.21.15.133:0/3302206341 >> 172.21.15.133:0/3302206341 conn(0x7fe69006cb50 msgr2=0x7fe69006cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:18.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.068+0000 7fe694cee700 1 -- 172.21.15.133:0/3302206341 shutdown_connections 2023-10-11T09:40:18.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.068+0000 7fe694cee700 1 -- 172.21.15.133:0/3302206341 wait complete. 2023-10-11T09:40:18.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.072+0000 7fe694cee700 1 Processor -- start 2023-10-11T09:40:18.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.072+0000 7fe694cee700 1 -- start start 2023-10-11T09:40:18.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.076+0000 7fe694cee700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6900b84c0 0x7fe690156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:18.080 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.076+0000 7fe694cee700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6900b8890 0x7fe690159440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:18.080 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.076+0000 7fe694cee700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6900d2350 0x7fe6900cc790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:18.080 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.076+0000 7fe694cee700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6900c85b0 con 0x7fe6900d2350 2023-10-11T09:40:18.080 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.076+0000 7fe694cee700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe690159980 con 0x7fe6900b84c0 2023-10-11T09:40:18.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.076+0000 7fe694cee700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe690156c30 con 0x7fe6900b8890 2023-10-11T09:40:18.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.076+0000 7fe68dd9b700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6900b8890 0x7fe690159440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:18.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.076+0000 7fe68e59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6900b84c0 0x7fe690156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:18.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.076+0000 7fe68dd9b700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6900b8890 0x7fe690159440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:47920/0 (socket says 172.21.15.133:47920) 2023-10-11T09:40:18.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.076+0000 7fe68dd9b700 1 -- 172.21.15.133:0/4105359378 learned_addr learned my addr 172.21.15.133:0/4105359378 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-10-11T09:40:18.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.076+0000 7fe68ed9d700 1 --2- 172.21.15.133:0/4105359378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6900d2350 0x7fe6900cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:18.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.076+0000 7fe68dd9b700 1 -- 172.21.15.133:0/4105359378 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6900b84c0 msgr2=0x7fe690156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:18.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.076+0000 7fe68dd9b700 1 --2- 172.21.15.133:0/4105359378 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6900b84c0 0x7fe690156650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:18.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.076+0000 7fe68dd9b700 1 -- 172.21.15.133:0/4105359378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6900d2350 msgr2=0x7fe6900cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:18.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.076+0000 7fe68dd9b700 1 --2- 172.21.15.133:0/4105359378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6900d2350 0x7fe6900cc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:18.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.076+0000 7fe68dd9b700 1 -- 172.21.15.133:0/4105359378 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe688009a70 con 0x7fe6900b8890 2023-10-11T09:40:18.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.076+0000 7fe68ed9d700 1 --2- 172.21.15.133:0/4105359378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6900d2350 0x7fe6900cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:40:18.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.076+0000 7fe68e59c700 1 --2- 172.21.15.133:0/4105359378 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6900b84c0 0x7fe690156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:40:18.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.076+0000 7fe68dd9b700 1 --2- 172.21.15.133:0/4105359378 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6900b8890 0x7fe690159440 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe68400c500 tx=0x7fe68400c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:18.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.076+0000 7fe67effd700 1 -- 172.21.15.133:0/4105359378 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6840052d0 con 0x7fe6900b8890 2023-10-11T09:40:18.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.076+0000 7fe67effd700 1 -- 172.21.15.133:0/4105359378 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe684005430 con 0x7fe6900b8890 2023-10-11T09:40:18.086 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.076+0000 7fe67effd700 1 -- 172.21.15.133:0/4105359378 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6840055a0 con 0x7fe6900b8890 2023-10-11T09:40:18.086 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.076+0000 7fe694cee700 1 -- 172.21.15.133:0/4105359378 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6900c8800 con 0x7fe6900b8890 2023-10-11T09:40:18.086 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.076+0000 7fe694cee700 1 -- 172.21.15.133:0/4105359378 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6900c8dc0 con 0x7fe6900b8890 2023-10-11T09:40:18.086 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.076+0000 7fe67effd700 1 -- 172.21.15.133:0/4105359378 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe684003be0 con 0x7fe6900b8890 2023-10-11T09:40:18.086 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.076+0000 7fe67effd700 1 -- 172.21.15.133:0/4105359378 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fe684011d60 con 0x7fe6900b8890 2023-10-11T09:40:18.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.080+0000 7fe694cee700 1 -- 172.21.15.133:0/4105359378 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6900c8fe0 con 0x7fe6900b8890 2023-10-11T09:40:18.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.084+0000 7fe67effd700 1 -- 172.21.15.133:0/4105359378 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fe684017070 con 0x7fe6900b8890 2023-10-11T09:40:18.165 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.160+0000 7fe694cee700 1 -- 172.21.15.133:0/4105359378 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c62a10af-8966-42af-b799-3422ad19ae0d", "id": 7} v 0) v1 -- 0x7fe69008bff0 con 0x7fe6900b8890 2023-10-11T09:40:18.176 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.172+0000 7fe67effd700 1 -- 172.21.15.133:0/4105359378 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c62a10af-8966-42af-b799-3422ad19ae0d", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe684011590 con 0x7fe6900b8890 2023-10-11T09:40:18.176 INFO:teuthology.orchestra.run.smithi133.stdout:7 2023-10-11T09:40:18.177 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.172+0000 7fe694cee700 1 -- 172.21.15.133:0/4105359378 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6900b8890 msgr2=0x7fe690159440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:18.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.172+0000 7fe694cee700 1 --2- 172.21.15.133:0/4105359378 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6900b8890 0x7fe690159440 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe68400c500 tx=0x7fe68400c810 comp rx=0 tx=0).stop 2023-10-11T09:40:18.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.172+0000 7fe694cee700 1 -- 172.21.15.133:0/4105359378 shutdown_connections 2023-10-11T09:40:18.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.172+0000 7fe694cee700 1 --2- 172.21.15.133:0/4105359378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6900d2350 0x7fe6900cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:18.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.172+0000 7fe694cee700 1 --2- 172.21.15.133:0/4105359378 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6900b8890 0x7fe690159440 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:18.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.172+0000 7fe694cee700 1 --2- 172.21.15.133:0/4105359378 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6900b84c0 0x7fe690156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:18.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.172+0000 7fe694cee700 1 -- 172.21.15.133:0/4105359378 >> 172.21.15.133:0/4105359378 conn(0x7fe69006cb50 msgr2=0x7fe6900d2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:18.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.172+0000 7fe694cee700 1 -- 172.21.15.133:0/4105359378 shutdown_connections 2023-10-11T09:40:18.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-10-11T09:40:18.172+0000 7fe694cee700 1 -- 172.21.15.133:0/4105359378 wait complete. 2023-10-11T09:40:18.197 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-11T09:40:18.198 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-11T09:40:18.205 INFO:tasks.ceph.osd.0:Started 2023-10-11T09:40:18.205 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-11T09:40:18.206 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-11T09:40:18.210 INFO:tasks.ceph.osd.1:Started 2023-10-11T09:40:18.210 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-11T09:40:18.211 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-11T09:40:18.217 INFO:tasks.ceph.osd.2:Started 2023-10-11T09:40:18.217 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-11T09:40:18.217 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-11T09:40:18.222 INFO:tasks.ceph.osd.3:Started 2023-10-11T09:40:18.222 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-11T09:40:18.222 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-11T09:40:18.230 INFO:tasks.ceph.osd.4:Started 2023-10-11T09:40:18.230 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-11T09:40:18.231 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-11T09:40:18.234 INFO:tasks.ceph.osd.5:Started 2023-10-11T09:40:18.235 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-11T09:40:18.235 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-11T09:40:18.239 INFO:tasks.ceph.osd.6:Started 2023-10-11T09:40:18.239 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-11T09:40:18.239 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-11T09:40:18.244 INFO:tasks.ceph.osd.7:Started 2023-10-11T09:40:18.244 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:40:18.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.414+0000 7f412359e700 1 Processor -- start 2023-10-11T09:40:18.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.414+0000 7f412359e700 1 -- start start 2023-10-11T09:40:18.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.418+0000 7f412359e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4124156ce0 0x7f4124160b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:18.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.418+0000 7f412359e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f412415dc10 con 0x7f41240d1660 2023-10-11T09:40:18.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.418+0000 7f412359e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4124160380 con 0x7f41240d1160 2023-10-11T09:40:18.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.418+0000 7f412359e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f412415d930 con 0x7f4124156ce0 2023-10-11T09:40:18.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.418+0000 7f4119d9b700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f41240d1660 0x7f41240d0a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:43140/0 (socket says 172.21.15.78:43140) 2023-10-11T09:40:18.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.418+0000 7f4119d9b700 1 -- 172.21.15.78:0/2192638181 learned_addr learned my addr 172.21.15.78:0/2192638181 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:18.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.418+0000 7f412259c700 1 --2- 172.21.15.78:0/2192638181 >> v2:172.21.15.133:3300/0 conn(0x7f4124156ce0 0x7f4124160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:18.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.418+0000 7f4121d9b700 1 -- 172.21.15.78:0/2192638181 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 327914374 0 0) 0x7f412415dc10 con 0x7f41240d1660 2023-10-11T09:40:18.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.418+0000 7f4121d9b700 1 -- 172.21.15.78:0/2192638181 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41100035d0 con 0x7f41240d1660 2023-10-11T09:40:18.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.418+0000 7f4121d9b700 1 -- 172.21.15.78:0/2192638181 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 446623784 0 0) 0x7f4124160380 con 0x7f41240d1160 2023-10-11T09:40:18.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.418+0000 7f4121d9b700 1 -- 172.21.15.78:0/2192638181 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f412415dc10 con 0x7f41240d1160 2023-10-11T09:40:18.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.418+0000 7f412259c700 1 -- 172.21.15.78:0/2192638181 >> v1:172.21.15.133:6789/0 conn(0x7f41240d1160 legacy=0x7f41240cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:18.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.418+0000 7f412259c700 1 -- 172.21.15.78:0/2192638181 >> v1:172.21.15.78:6789/0 conn(0x7f41240d1660 legacy=0x7f41240d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:18.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.418+0000 7f412259c700 1 -- 172.21.15.78:0/2192638181 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4124160620 con 0x7f4124156ce0 2023-10-11T09:40:18.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.418+0000 7f412259c700 1 --2- 172.21.15.78:0/2192638181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4124156ce0 0x7f4124160b40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4114009850 tx=0x7f4114009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c012ff2d3e2ec46 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:18.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.418+0000 7f4121d9b700 1 -- 172.21.15.78:0/2192638181 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4114011830 con 0x7f4124156ce0 2023-10-11T09:40:18.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.418+0000 7f4121d9b700 1 -- 172.21.15.78:0/2192638181 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4114011990 con 0x7f4124156ce0 2023-10-11T09:40:18.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.418+0000 7f4121d9b700 1 -- 172.21.15.78:0/2192638181 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4114011d40 con 0x7f4124156ce0 2023-10-11T09:40:18.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.422+0000 7f412359e700 1 -- 172.21.15.78:0/2192638181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4124156ce0 msgr2=0x7f4124160b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:18.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.422+0000 7f412359e700 1 --2- 172.21.15.78:0/2192638181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4124156ce0 0x7f4124160b40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4114009850 tx=0x7f4114009b60 comp rx=0 tx=0).stop 2023-10-11T09:40:18.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.422+0000 7f412359e700 1 -- 172.21.15.78:0/2192638181 shutdown_connections 2023-10-11T09:40:18.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.422+0000 7f412359e700 1 --2- 172.21.15.78:0/2192638181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4124156ce0 0x7f4124160b40 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:18.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.422+0000 7f412359e700 1 -- 172.21.15.78:0/2192638181 >> 172.21.15.78:0/2192638181 conn(0x7f412406afb0 msgr2=0x7f412406b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:18.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.422+0000 7f412359e700 1 -- 172.21.15.78:0/2192638181 shutdown_connections 2023-10-11T09:40:18.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.422+0000 7f412359e700 1 -- 172.21.15.78:0/2192638181 wait complete. 2023-10-11T09:40:18.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.422+0000 7f412359e700 1 Processor -- start 2023-10-11T09:40:18.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.422+0000 7f412359e700 1 -- start start 2023-10-11T09:40:18.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.426+0000 7f412359e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41240d1160 0x7f41240c69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:18.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.426+0000 7f412359e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41240d1660 0x7f4124154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:18.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.426+0000 7f412359e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4124156ce0 0x7f41240c8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:18.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.426+0000 7f412359e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41240c6f90 con 0x7f41240d1660 2023-10-11T09:40:18.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.426+0000 7f412359e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41240c7270 con 0x7f41240d1160 2023-10-11T09:40:18.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.426+0000 7f412359e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41240c7550 con 0x7f4124156ce0 2023-10-11T09:40:18.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.426+0000 7f4119d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41240d1660 0x7f4124154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:18.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.426+0000 7f412259c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41240d1160 0x7f41240c69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:18.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.426+0000 7f4119d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41240d1660 0x7f4124154f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40986/0 (socket says 172.21.15.78:40986) 2023-10-11T09:40:18.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.426+0000 7f412259c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41240d1160 0x7f41240c69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:55350/0 (socket says 172.21.15.78:55350) 2023-10-11T09:40:18.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.426+0000 7f4119d9b700 1 -- 172.21.15.78:0/17483728 learned_addr learned my addr 172.21.15.78:0/17483728 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:18.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.426+0000 7f4122d9d700 1 --2- 172.21.15.78:0/17483728 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4124156ce0 0x7f41240c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:18.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.426+0000 7f4119d9b700 1 -- 172.21.15.78:0/17483728 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4124156ce0 msgr2=0x7f41240c8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:18.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.426+0000 7f4119d9b700 1 --2- 172.21.15.78:0/17483728 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4124156ce0 0x7f41240c8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:18.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.426+0000 7f4119d9b700 1 -- 172.21.15.78:0/17483728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41240d1160 msgr2=0x7f41240c69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:18.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.426+0000 7f4119d9b700 1 --2- 172.21.15.78:0/17483728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41240d1160 0x7f41240c69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:18.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.426+0000 7f4119d9b700 1 -- 172.21.15.78:0/17483728 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41140094b0 con 0x7f41240d1660 2023-10-11T09:40:18.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.426+0000 7f4119d9b700 1 --2- 172.21.15.78:0/17483728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41240d1660 0x7f4124154f60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f410c00c530 tx=0x7f410c00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:18.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.426+0000 7f411b7fe700 1 -- 172.21.15.78:0/17483728 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c005e50 con 0x7f41240d1660 2023-10-11T09:40:18.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.426+0000 7f411b7fe700 1 -- 172.21.15.78:0/17483728 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410c0070d0 con 0x7f41240d1660 2023-10-11T09:40:18.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.426+0000 7f411b7fe700 1 -- 172.21.15.78:0/17483728 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c007460 con 0x7f41240d1660 2023-10-11T09:40:18.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.426+0000 7f412359e700 1 -- 172.21.15.78:0/17483728 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41240c7830 con 0x7f41240d1660 2023-10-11T09:40:18.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.426+0000 7f412359e700 1 -- 172.21.15.78:0/17483728 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41241574e0 con 0x7f41240d1660 2023-10-11T09:40:18.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.426+0000 7f411b7fe700 1 -- 172.21.15.78:0/17483728 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f410c01c440 con 0x7f41240d1660 2023-10-11T09:40:18.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.426+0000 7f411b7fe700 1 -- 172.21.15.78:0/17483728 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f410c012b20 con 0x7f41240d1660 2023-10-11T09:40:18.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.430+0000 7f412359e700 1 -- 172.21.15.78:0/17483728 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41240c7d50 con 0x7f41240d1660 2023-10-11T09:40:18.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.430+0000 7f411b7fe700 1 -- 172.21.15.78:0/17483728 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f41240c7d50 con 0x7f41240d1660 2023-10-11T09:40:18.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.498+0000 7f412359e700 1 -- 172.21.15.78:0/17483728 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4124161000 con 0x7f41240d1660 2023-10-11T09:40:18.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.502+0000 7f411b7fe700 1 -- 172.21.15.78:0/17483728 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f410c017070 con 0x7f41240d1660 2023-10-11T09:40:18.507 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:40:18.507 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":10,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:40:18.165644+0000","last_up_change":"0.000000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:40:18.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.502+0000 7f412359e700 1 -- 172.21.15.78:0/17483728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41240d1660 msgr2=0x7f4124154f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:18.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.502+0000 7f412359e700 1 --2- 172.21.15.78:0/17483728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41240d1660 0x7f4124154f60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f410c00c530 tx=0x7f410c00c840 comp rx=0 tx=0).stop 2023-10-11T09:40:18.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.502+0000 7f412359e700 1 -- 172.21.15.78:0/17483728 shutdown_connections 2023-10-11T09:40:18.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.502+0000 7f412359e700 1 --2- 172.21.15.78:0/17483728 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4124156ce0 0x7f41240c8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:18.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.502+0000 7f412359e700 1 --2- 172.21.15.78:0/17483728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41240d1660 0x7f4124154f60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:18.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.502+0000 7f412359e700 1 --2- 172.21.15.78:0/17483728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41240d1160 0x7f41240c69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:18.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.502+0000 7f412359e700 1 -- 172.21.15.78:0/17483728 >> 172.21.15.78:0/17483728 conn(0x7f412406afb0 msgr2=0x7f412415f640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:18.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.502+0000 7f412359e700 1 -- 172.21.15.78:0/17483728 shutdown_connections 2023-10-11T09:40:18.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:18.502+0000 7f412359e700 1 -- 172.21.15.78:0/17483728 wait complete. 2023-10-11T09:40:18.528 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-11T09:40:18.528 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-11T09:40:18.572 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-10-11T09:40:18.566+0000 7f4cf162c2c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-11T09:40:18.657 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-10-11T09:40:18.654+0000 7f4cf162c2c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-11T09:40:18.775 INFO:tasks.ceph.mgr.x.smithi133.stderr:2023-10-11T09:40:18.768+0000 7f8edb41d2c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-11T09:40:18.836 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-10-11T09:40:18.830+0000 7f4cf162c2c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-11T09:40:18.872 INFO:tasks.ceph.mgr.x.smithi133.stderr:2023-10-11T09:40:18.868+0000 7f8edb41d2c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-11T09:40:18.872 INFO:tasks.ceph.osd.0.smithi078.stderr:2023-10-11T09:40:18.866+0000 7fa75a616700 -1 Falling back to public interface 2023-10-11T09:40:18.886 INFO:tasks.ceph.osd.6.smithi133.stderr:2023-10-11T09:40:18.880+0000 7f65b7df4700 -1 Falling back to public interface 2023-10-11T09:40:18.891 INFO:tasks.ceph.osd.1.smithi078.stderr:2023-10-11T09:40:18.886+0000 7fe00ad0b700 -1 Falling back to public interface 2023-10-11T09:40:18.891 INFO:tasks.ceph.osd.2.smithi078.stderr:2023-10-11T09:40:18.886+0000 7fba1557c700 -1 Falling back to public interface 2023-10-11T09:40:18.894 INFO:tasks.ceph.osd.4.smithi133.stderr:2023-10-11T09:40:18.888+0000 7fcb596c3700 -1 Falling back to public interface 2023-10-11T09:40:18.906 INFO:tasks.ceph.osd.3.smithi078.stderr:2023-10-11T09:40:18.902+0000 7fc1acb47700 -1 Falling back to public interface 2023-10-11T09:40:18.923 INFO:tasks.ceph.osd.5.smithi133.stderr:2023-10-11T09:40:18.916+0000 7fee8f3b4700 -1 Falling back to public interface 2023-10-11T09:40:18.924 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-10-11T09:40:18.918+0000 7f4cf162c2c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-11T09:40:18.938 INFO:tasks.ceph.osd.7.smithi133.stderr:2023-10-11T09:40:18.932+0000 7f092262a700 -1 Falling back to public interface 2023-10-11T09:40:19.025 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-10-11T09:40:19.018+0000 7f4cf162c2c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-11T09:40:19.065 INFO:tasks.ceph.mgr.x.smithi133.stderr:2023-10-11T09:40:19.060+0000 7f8edb41d2c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-11T09:40:19.157 INFO:tasks.ceph.mgr.x.smithi133.stderr:2023-10-11T09:40:19.152+0000 7f8edb41d2c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-11T09:40:19.250 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-10-11T09:40:19.246+0000 7f4cf162c2c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-11T09:40:19.262 INFO:tasks.ceph.mgr.x.smithi133.stderr:2023-10-11T09:40:19.256+0000 7f8edb41d2c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-11T09:40:19.335 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-11T09:40:19.483 INFO:tasks.ceph.mgr.x.smithi133.stderr:2023-10-11T09:40:19.476+0000 7f8edb41d2c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-11T09:40:19.488 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.482+0000 7f48171bc700 1 Processor -- start 2023-10-11T09:40:19.489 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.482+0000 7f48171bc700 1 -- start start 2023-10-11T09:40:19.491 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.486+0000 7f48171bc700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f48100cab30 0x7f48100cc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:19.491 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.486+0000 7f48171bc700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f481015cff0 0x7f4810141bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:19.491 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.486+0000 7f48171bc700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48100c4330 con 0x7f48100ccad0 2023-10-11T09:40:19.491 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.486+0000 7f48171bc700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f481015d610 con 0x7f48100cab30 2023-10-11T09:40:19.491 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.486+0000 7f48171bc700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48100c3a40 con 0x7f481015cff0 2023-10-11T09:40:19.493 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.486+0000 7f48161ba700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f48100cab30 0x7f48100cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:19.493 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.486+0000 7f48169bb700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f481015cff0 0x7f4810141bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:19.493 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.486+0000 7f48161ba700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f48100cab30 0x7f48100cc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40990/0 (socket says 172.21.15.78:40990) 2023-10-11T09:40:19.493 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.486+0000 7f48161ba700 1 -- 172.21.15.78:0/3476666488 learned_addr learned my addr 172.21.15.78:0/3476666488 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:19.494 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.486+0000 7f48161ba700 1 -- 172.21.15.78:0/3476666488 >> v2:172.21.15.133:3300/0 conn(0x7f481015cff0 msgr2=0x7f4810141bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:19.494 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.486+0000 7f48161ba700 1 --2- 172.21.15.78:0/3476666488 >> v2:172.21.15.133:3300/0 conn(0x7f481015cff0 0x7f4810141bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:19.494 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.486+0000 7f48161ba700 1 -- 172.21.15.78:0/3476666488 >> v1:172.21.15.133:6789/0 conn(0x7f48100ccad0 legacy=0x7f481015c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:19.494 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.486+0000 7f48161ba700 1 -- 172.21.15.78:0/3476666488 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48100c9010 con 0x7f48100cab30 2023-10-11T09:40:19.494 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.486+0000 7f48161ba700 1 --2- 172.21.15.78:0/3476666488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48100cab30 0x7f48100cc590 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f480c009910 tx=0x7f480c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=347086e33d8e215a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:19.495 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.486+0000 7f48151b8700 1 -- 172.21.15.78:0/3476666488 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c011830 con 0x7f48100cab30 2023-10-11T09:40:19.495 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.486+0000 7f48151b8700 1 -- 172.21.15.78:0/3476666488 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480c011990 con 0x7f48100cab30 2023-10-11T09:40:19.495 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.486+0000 7f48151b8700 1 -- 172.21.15.78:0/3476666488 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c011d40 con 0x7f48100cab30 2023-10-11T09:40:19.496 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.490+0000 7f48171bc700 1 -- 172.21.15.78:0/3476666488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48100cab30 msgr2=0x7f48100cc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:19.497 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.490+0000 7f48171bc700 1 --2- 172.21.15.78:0/3476666488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48100cab30 0x7f48100cc590 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f480c009910 tx=0x7f480c009c20 comp rx=0 tx=0).stop 2023-10-11T09:40:19.497 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.490+0000 7f48171bc700 1 -- 172.21.15.78:0/3476666488 shutdown_connections 2023-10-11T09:40:19.497 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.490+0000 7f48171bc700 1 --2- 172.21.15.78:0/3476666488 >> v2:172.21.15.133:3300/0 conn(0x7f481015cff0 0x7f4810141bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:19.497 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.490+0000 7f48171bc700 1 --2- 172.21.15.78:0/3476666488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48100cab30 0x7f48100cc590 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:19.497 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.490+0000 7f48171bc700 1 -- 172.21.15.78:0/3476666488 >> 172.21.15.78:0/3476666488 conn(0x7f48100cf6a0 msgr2=0x7f48100cb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:19.498 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.490+0000 7f48171bc700 1 -- 172.21.15.78:0/3476666488 shutdown_connections 2023-10-11T09:40:19.498 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.490+0000 7f48171bc700 1 -- 172.21.15.78:0/3476666488 wait complete. 2023-10-11T09:40:19.498 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.494+0000 7f48171bc700 1 Processor -- start 2023-10-11T09:40:19.499 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.494+0000 7f48171bc700 1 -- start start 2023-10-11T09:40:19.500 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.494+0000 7f48171bc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48100cab30 0x7f4810159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:19.501 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.494+0000 7f48171bc700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48100ccad0 0x7f481015b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:19.501 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.494+0000 7f48171bc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f481015cff0 0x7f481015ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:19.501 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.494+0000 7f48171bc700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481015bfb0 con 0x7f481015cff0 2023-10-11T09:40:19.501 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.494+0000 7f48171bc700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481014fa60 con 0x7f48100cab30 2023-10-11T09:40:19.501 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.494+0000 7f48171bc700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4810070fe0 con 0x7f48100ccad0 2023-10-11T09:40:19.503 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.494+0000 7f48169bb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f481015cff0 0x7f481015ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:19.504 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.498+0000 7f48161ba700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48100cab30 0x7f4810159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:19.504 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.498+0000 7f48169bb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f481015cff0 0x7f481015ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41002/0 (socket says 172.21.15.78:41002) 2023-10-11T09:40:19.504 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.498+0000 7f48169bb700 1 -- 172.21.15.78:0/3785263682 learned_addr learned my addr 172.21.15.78:0/3785263682 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:19.504 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.498+0000 7f48159b9700 1 --2- 172.21.15.78:0/3785263682 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48100ccad0 0x7f481015b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:19.504 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.498+0000 7f48161ba700 1 -- 172.21.15.78:0/3785263682 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48100ccad0 msgr2=0x7f481015b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:19.504 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.498+0000 7f48161ba700 1 --2- 172.21.15.78:0/3785263682 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48100ccad0 0x7f481015b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:19.505 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.498+0000 7f48161ba700 1 -- 172.21.15.78:0/3785263682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f481015cff0 msgr2=0x7f481015ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:19.505 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.498+0000 7f48161ba700 1 --2- 172.21.15.78:0/3785263682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f481015cff0 0x7f481015ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:19.505 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.498+0000 7f48161ba700 1 -- 172.21.15.78:0/3785263682 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f480c009570 con 0x7f48100cab30 2023-10-11T09:40:19.505 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.498+0000 7f48169bb700 1 --2- 172.21.15.78:0/3785263682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f481015cff0 0x7f481015ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:40:19.505 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.498+0000 7f48161ba700 1 --2- 172.21.15.78:0/3785263682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48100cab30 0x7f4810159d20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f480c016000 tx=0x7f480c004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:19.506 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.498+0000 7f4806ffd700 1 -- 172.21.15.78:0/3785263682 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c004510 con 0x7f48100cab30 2023-10-11T09:40:19.506 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.498+0000 7f4806ffd700 1 -- 172.21.15.78:0/3785263682 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480c01e070 con 0x7f48100cab30 2023-10-11T09:40:19.506 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.498+0000 7f48171bc700 1 -- 172.21.15.78:0/3785263682 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48100712c0 con 0x7f48100cab30 2023-10-11T09:40:19.506 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.498+0000 7f4806ffd700 1 -- 172.21.15.78:0/3785263682 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c003900 con 0x7f48100cab30 2023-10-11T09:40:19.506 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.498+0000 7f48171bc700 1 -- 172.21.15.78:0/3785263682 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f481014dfb0 con 0x7f48100cab30 2023-10-11T09:40:19.507 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.498+0000 7f4806ffd700 1 -- 172.21.15.78:0/3785263682 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f480c01a420 con 0x7f48100cab30 2023-10-11T09:40:19.507 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.498+0000 7f4806ffd700 1 -- 172.21.15.78:0/3785263682 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f480c01ada0 con 0x7f48100cab30 2023-10-11T09:40:19.507 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.498+0000 7f48171bc700 1 -- 172.21.15.78:0/3785263682 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4810065d40 con 0x7f48100cab30 2023-10-11T09:40:19.511 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.506+0000 7f4806ffd700 1 -- 172.21.15.78:0/3785263682 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f480c020070 con 0x7f48100cab30 2023-10-11T09:40:19.515 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-10-11T09:40:19.510+0000 7f4cf162c2c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-11T09:40:19.580 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.574+0000 7f48171bc700 1 -- 172.21.15.78:0/3785263682 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4810072240 con 0x7f48100cab30 2023-10-11T09:40:19.581 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.574+0000 7f4806ffd700 1 -- 172.21.15.78:0/3785263682 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f480c015020 con 0x7f48100cab30 2023-10-11T09:40:19.581 INFO:teuthology.misc.health.smithi078.stdout: 2023-10-11T09:40:19.582 INFO:teuthology.misc.health.smithi078.stdout:{"epoch":10,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:40:18.165644+0000","last_up_change":"0.000000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:40:19.583 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.578+0000 7f48171bc700 1 -- 172.21.15.78:0/3785263682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48100cab30 msgr2=0x7f4810159d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:19.583 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.578+0000 7f48171bc700 1 --2- 172.21.15.78:0/3785263682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48100cab30 0x7f4810159d20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f480c016000 tx=0x7f480c004770 comp rx=0 tx=0).stop 2023-10-11T09:40:19.583 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.578+0000 7f48171bc700 1 -- 172.21.15.78:0/3785263682 shutdown_connections 2023-10-11T09:40:19.584 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.578+0000 7f48171bc700 1 --2- 172.21.15.78:0/3785263682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f481015cff0 0x7f481015ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:19.584 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.578+0000 7f48171bc700 1 --2- 172.21.15.78:0/3785263682 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48100ccad0 0x7f481015b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:19.584 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.578+0000 7f48171bc700 1 --2- 172.21.15.78:0/3785263682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48100cab30 0x7f4810159d20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:19.584 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.578+0000 7f48171bc700 1 -- 172.21.15.78:0/3785263682 >> 172.21.15.78:0/3785263682 conn(0x7f48100cf6a0 msgr2=0x7f4810153a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:19.584 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.578+0000 7f48171bc700 1 -- 172.21.15.78:0/3785263682 shutdown_connections 2023-10-11T09:40:19.584 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:19.578+0000 7f48171bc700 1 -- 172.21.15.78:0/3785263682 wait complete. 2023-10-11T09:40:19.600 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-11T09:40:19.753 INFO:tasks.ceph.mgr.x.smithi133.stderr:2023-10-11T09:40:19.748+0000 7f8edb41d2c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-11T09:40:20.295 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-10-11T09:40:20.290+0000 7f4cf162c2c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-11T09:40:20.598 INFO:tasks.ceph.mgr.x.smithi133.stderr:2023-10-11T09:40:20.592+0000 7f8edb41d2c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-11T09:40:21.338 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-10-11T09:40:21.334+0000 7f4cd86ad700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-11T09:40:21.342 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-10-11T09:40:21.338+0000 7f4cd86ad700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-11T09:40:22.829 INFO:tasks.ceph.osd.6.smithi133.stderr:2023-10-11T09:40:22.824+0000 7f65b7df4700 -1 osd.6 0 log_to_monitors true 2023-10-11T09:40:22.848 INFO:tasks.ceph.osd.0.smithi078.stderr:2023-10-11T09:40:22.842+0000 7fa75a616700 -1 osd.0 0 log_to_monitors true 2023-10-11T09:40:22.899 INFO:tasks.ceph.osd.4.smithi133.stderr:2023-10-11T09:40:22.892+0000 7fcb596c3700 -1 osd.4 0 log_to_monitors true 2023-10-11T09:40:22.934 INFO:tasks.ceph.osd.1.smithi078.stderr:2023-10-11T09:40:22.930+0000 7fe00ad0b700 -1 osd.1 0 log_to_monitors true 2023-10-11T09:40:23.104 INFO:tasks.ceph.osd.2.smithi078.stderr:2023-10-11T09:40:23.098+0000 7fba1557c700 -1 osd.2 0 log_to_monitors true 2023-10-11T09:40:23.207 INFO:tasks.ceph.osd.7.smithi133.stderr:2023-10-11T09:40:23.204+0000 7f092262a700 -1 osd.7 0 log_to_monitors true 2023-10-11T09:40:23.227 INFO:tasks.ceph.osd.3.smithi078.stderr:2023-10-11T09:40:23.222+0000 7fc1acb47700 -1 osd.3 0 log_to_monitors true 2023-10-11T09:40:23.229 INFO:tasks.ceph.osd.5.smithi133.stderr:2023-10-11T09:40:23.224+0000 7fee8f3b4700 -1 osd.5 0 log_to_monitors true 2023-10-11T09:40:26.407 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-11T09:40:26.584 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.578+0000 7f0ee5ee4700 1 Processor -- start 2023-10-11T09:40:26.585 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.578+0000 7f0ee5ee4700 1 -- start start 2023-10-11T09:40:26.586 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.582+0000 7f0ee5ee4700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0ee006c750 0x7f0ee006cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:26.586 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.582+0000 7f0ee5ee4700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ee00b2540 0x7f0ee00b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:26.587 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.582+0000 7f0ee5ee4700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ee006d0c0 con 0x7f0ee00b8580 2023-10-11T09:40:26.587 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.582+0000 7f0ee5ee4700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ee00d2a80 con 0x7f0ee006c750 2023-10-11T09:40:26.587 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.582+0000 7f0ee5ee4700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee00cd8e0 con 0x7f0ee00b2540 2023-10-11T09:40:26.587 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.582+0000 7f0ee4ee2700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f0ee00b8580 0x7f0ee00cf100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:50646/0 (socket says 172.21.15.78:50646) 2023-10-11T09:40:26.587 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.582+0000 7f0ee4ee2700 1 -- 172.21.15.78:0/1331467858 learned_addr learned my addr 172.21.15.78:0/1331467858 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:26.588 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.582+0000 7f0edffff700 1 --2- 172.21.15.78:0/1331467858 >> v2:172.21.15.78:3300/0 conn(0x7f0ee006c750 0x7f0ee006cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:26.589 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.582+0000 7f0ee56e3700 1 --2- 172.21.15.78:0/1331467858 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ee00b2540 0x7f0ee00b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:26.589 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.582+0000 7f0edf7fe700 1 -- 172.21.15.78:0/1331467858 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3881051471 0 0) 0x7f0ee006d0c0 con 0x7f0ee00b8580 2023-10-11T09:40:26.589 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.582+0000 7f0edf7fe700 1 -- 172.21.15.78:0/1331467858 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ec80035d0 con 0x7f0ee00b8580 2023-10-11T09:40:26.589 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.582+0000 7f0edffff700 1 -- 172.21.15.78:0/1331467858 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ee00b2540 msgr2=0x7f0ee00b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:26.590 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.582+0000 7f0edffff700 1 --2- 172.21.15.78:0/1331467858 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ee00b2540 0x7f0ee00b29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:26.590 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.582+0000 7f0edffff700 1 -- 172.21.15.78:0/1331467858 >> v1:172.21.15.78:6789/0 conn(0x7f0ee00b8580 legacy=0x7f0ee00cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:26.590 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.582+0000 7f0edffff700 1 -- 172.21.15.78:0/1331467858 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee00d1530 con 0x7f0ee006c750 2023-10-11T09:40:26.590 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.582+0000 7f0edffff700 1 --2- 172.21.15.78:0/1331467858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ee006c750 0x7f0ee006cb80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0009850 tx=0x7f0ed0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d6a5ec86f4165e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:26.590 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.582+0000 7f0edf7fe700 1 -- 172.21.15.78:0/1331467858 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0011830 con 0x7f0ee006c750 2023-10-11T09:40:26.591 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.582+0000 7f0edf7fe700 1 -- 172.21.15.78:0/1331467858 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed0011990 con 0x7f0ee006c750 2023-10-11T09:40:26.591 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.582+0000 7f0edf7fe700 1 -- 172.21.15.78:0/1331467858 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0011d40 con 0x7f0ee006c750 2023-10-11T09:40:26.591 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.586+0000 7f0ee5ee4700 1 -- 172.21.15.78:0/1331467858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ee006c750 msgr2=0x7f0ee006cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:26.592 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.586+0000 7f0ee5ee4700 1 --2- 172.21.15.78:0/1331467858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ee006c750 0x7f0ee006cb80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0009850 tx=0x7f0ed0009b60 comp rx=0 tx=0).stop 2023-10-11T09:40:26.592 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.586+0000 7f0ee5ee4700 1 -- 172.21.15.78:0/1331467858 shutdown_connections 2023-10-11T09:40:26.592 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.586+0000 7f0ee5ee4700 1 --2- 172.21.15.78:0/1331467858 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ee00b2540 0x7f0ee00b29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:26.592 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.586+0000 7f0ee5ee4700 1 --2- 172.21.15.78:0/1331467858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ee006c750 0x7f0ee006cb80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:26.592 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.586+0000 7f0ee5ee4700 1 -- 172.21.15.78:0/1331467858 >> 172.21.15.78:0/1331467858 conn(0x7f0ee00d2eb0 msgr2=0x7f0ee00d3280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:26.593 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.586+0000 7f0ee5ee4700 1 -- 172.21.15.78:0/1331467858 shutdown_connections 2023-10-11T09:40:26.593 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.586+0000 7f0ee5ee4700 1 -- 172.21.15.78:0/1331467858 wait complete. 2023-10-11T09:40:26.593 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.590+0000 7f0ee5ee4700 1 Processor -- start 2023-10-11T09:40:26.594 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.590+0000 7f0ee5ee4700 1 -- start start 2023-10-11T09:40:26.598 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.590+0000 7f0ee5ee4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ee006c750 0x7f0ee0151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:26.598 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.590+0000 7f0ee4ee2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ee006c750 0x7f0ee0151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:26.598 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.590+0000 7f0ee4ee2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ee006c750 0x7f0ee0151ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48176/0 (socket says 172.21.15.78:48176) 2023-10-11T09:40:26.598 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.590+0000 7f0ee4ee2700 1 -- 172.21.15.78:0/3274401915 learned_addr learned my addr 172.21.15.78:0/3274401915 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:26.599 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.594+0000 7f0ee5ee4700 1 --2- 172.21.15.78:0/3274401915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ee00b2540 0x7f0ee00c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:26.599 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.594+0000 7f0ee5ee4700 1 --2- 172.21.15.78:0/3274401915 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ee00b8580 0x7f0ee006a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:26.599 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.594+0000 7f0ee5ee4700 1 -- 172.21.15.78:0/3274401915 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee015fbb0 con 0x7f0ee006c750 2023-10-11T09:40:26.599 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.594+0000 7f0ee5ee4700 1 -- 172.21.15.78:0/3274401915 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee015f3a0 con 0x7f0ee00b2540 2023-10-11T09:40:26.599 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.594+0000 7f0ee5ee4700 1 -- 172.21.15.78:0/3274401915 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee015f5f0 con 0x7f0ee00b8580 2023-10-11T09:40:26.600 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.594+0000 7f0ee56e3700 1 --2- 172.21.15.78:0/3274401915 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ee00b8580 0x7f0ee006a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:26.601 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.594+0000 7f0ee56e3700 1 -- 172.21.15.78:0/3274401915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ee00b2540 msgr2=0x7f0ee00c9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:26.601 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.594+0000 7f0ee56e3700 1 --2- 172.21.15.78:0/3274401915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ee00b2540 0x7f0ee00c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:26.601 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.594+0000 7f0ee56e3700 1 -- 172.21.15.78:0/3274401915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ee006c750 msgr2=0x7f0ee0151ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:26.601 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.594+0000 7f0ee56e3700 1 --2- 172.21.15.78:0/3274401915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ee006c750 0x7f0ee0151ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:26.601 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.594+0000 7f0ee56e3700 1 -- 172.21.15.78:0/3274401915 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed00094b0 con 0x7f0ee00b8580 2023-10-11T09:40:26.602 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.594+0000 7f0ee4ee2700 1 --2- 172.21.15.78:0/3274401915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ee006c750 0x7f0ee0151ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:40:26.602 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.594+0000 7f0ee56e3700 1 --2- 172.21.15.78:0/3274401915 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ee00b8580 0x7f0ee006a5b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0ed800efc0 tx=0x7f0ed800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:26.602 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.594+0000 7f0edd7fa700 1 -- 172.21.15.78:0/3274401915 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed800cd40 con 0x7f0ee00b8580 2023-10-11T09:40:26.602 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.594+0000 7f0edd7fa700 1 -- 172.21.15.78:0/3274401915 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed800f040 con 0x7f0ee00b8580 2023-10-11T09:40:26.602 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.594+0000 7f0edd7fa700 1 -- 172.21.15.78:0/3274401915 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed800cea0 con 0x7f0ee00b8580 2023-10-11T09:40:26.603 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.594+0000 7f0ee5ee4700 1 -- 172.21.15.78:0/3274401915 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ee015f8d0 con 0x7f0ee00b8580 2023-10-11T09:40:26.603 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.594+0000 7f0ee5ee4700 1 -- 172.21.15.78:0/3274401915 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ee015d110 con 0x7f0ee00b8580 2023-10-11T09:40:26.605 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.598+0000 7f0ee5ee4700 1 -- 172.21.15.78:0/3274401915 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ee015cc70 con 0x7f0ee00b8580 2023-10-11T09:40:26.605 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.598+0000 7f0edd7fa700 1 -- 172.21.15.78:0/3274401915 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 4) v1 ==== 85480+0+0 (secure 0 0 0) 0x7f0ed80146c0 con 0x7f0ee00b8580 2023-10-11T09:40:26.605 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.602+0000 7f0edd7fa700 1 --2- 172.21.15.78:0/3274401915 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0ecc065e90 0x7f0ecc068340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:26.606 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.602+0000 7f0edd7fa700 1 -- 172.21.15.78:0/3274401915 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f0ed8083360 con 0x7f0ee00b8580 2023-10-11T09:40:26.606 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.602+0000 7f0ee4ee2700 1 --2- 172.21.15.78:0/3274401915 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0ecc065e90 0x7f0ecc068340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:26.606 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.602+0000 7f0ee4ee2700 1 --2- 172.21.15.78:0/3274401915 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0ecc065e90 0x7f0ecc068340 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0ed40011e0 tx=0x7f0ed4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:26.613 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.610+0000 7f0edd7fa700 1 -- 172.21.15.78:0/3274401915 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f0ed80519d0 con 0x7f0ee00b8580 2023-10-11T09:40:26.721 INFO:tasks.ceph.osd.5.smithi133.stderr:2023-10-11T09:40:26.716+0000 7fee8c37b700 -1 osd.5 0 waiting for initial osdmap 2023-10-11T09:40:26.730 INFO:tasks.ceph.osd.5.smithi133.stderr:2023-10-11T09:40:26.724+0000 7fee82bf1700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-11T09:40:26.761 INFO:tasks.ceph.osd.4.smithi133.stderr:2023-10-11T09:40:26.756+0000 7fcb5668a700 -1 osd.4 0 waiting for initial osdmap 2023-10-11T09:40:26.770 INFO:tasks.ceph.osd.4.smithi133.stderr:2023-10-11T09:40:26.764+0000 7fcb4cdf1700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-11T09:40:26.777 INFO:tasks.ceph.osd.7.smithi133.stderr:2023-10-11T09:40:26.772+0000 7f091f5f1700 -1 osd.7 0 waiting for initial osdmap 2023-10-11T09:40:26.786 INFO:tasks.ceph.osd.7.smithi133.stderr:2023-10-11T09:40:26.780+0000 7f0915df1700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-11T09:40:26.918 INFO:tasks.ceph.osd.6.smithi133.stderr:2023-10-11T09:40:26.912+0000 7f65b3410700 -1 osd.6 0 waiting for initial osdmap 2023-10-11T09:40:26.926 INFO:tasks.ceph.osd.6.smithi133.stderr:2023-10-11T09:40:26.920+0000 7f65ab5f1700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-11T09:40:26.950 INFO:tasks.ceph.osd.3.smithi078.stderr:2023-10-11T09:40:26.946+0000 7fc1a8163700 -1 osd.3 0 waiting for initial osdmap 2023-10-11T09:40:26.962 INFO:tasks.ceph.osd.3.smithi078.stderr:2023-10-11T09:40:26.958+0000 7fc1a03f1700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-11T09:40:26.966 INFO:tasks.ceph.osd.2.smithi078.stderr:2023-10-11T09:40:26.962+0000 7fba10b98700 -1 osd.2 0 waiting for initial osdmap 2023-10-11T09:40:26.977 INFO:tasks.ceph.osd.2.smithi078.stderr:2023-10-11T09:40:26.970+0000 7fba08df1700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-11T09:40:26.986 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.982+0000 7f0ee5ee4700 1 -- 172.21.15.78:0/3274401915 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ee015d300 con 0x7f0ee00b8580 2023-10-11T09:40:26.987 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.982+0000 7f0edd7fa700 1 -- 172.21.15.78:0/3274401915 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f0ed8018070 con 0x7f0ee00b8580 2023-10-11T09:40:26.988 INFO:teuthology.misc.health.smithi078.stdout: 2023-10-11T09:40:26.988 INFO:teuthology.misc.health.smithi078.stdout:{"epoch":12,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:40:24.408915+0000","last_up_change":"0.000000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:40:26.991 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.986+0000 7f0ee5ee4700 1 -- 172.21.15.78:0/3274401915 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0ecc065e90 msgr2=0x7f0ecc068340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:26.991 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.986+0000 7f0ee5ee4700 1 --2- 172.21.15.78:0/3274401915 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0ecc065e90 0x7f0ecc068340 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0ed40011e0 tx=0x7f0ed4009040 comp rx=0 tx=0).stop 2023-10-11T09:40:26.992 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.986+0000 7f0ee5ee4700 1 -- 172.21.15.78:0/3274401915 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ee00b8580 msgr2=0x7f0ee006a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:26.992 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.986+0000 7f0ee5ee4700 1 --2- 172.21.15.78:0/3274401915 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ee00b8580 0x7f0ee006a5b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0ed800efc0 tx=0x7f0ed800c3a0 comp rx=0 tx=0).stop 2023-10-11T09:40:26.992 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.986+0000 7f0ee5ee4700 1 -- 172.21.15.78:0/3274401915 shutdown_connections 2023-10-11T09:40:26.992 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.986+0000 7f0ee5ee4700 1 --2- 172.21.15.78:0/3274401915 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0ecc065e90 0x7f0ecc068340 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:26.992 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.986+0000 7f0ee5ee4700 1 --2- 172.21.15.78:0/3274401915 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ee00b8580 0x7f0ee006a5b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:26.993 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.986+0000 7f0ee5ee4700 1 --2- 172.21.15.78:0/3274401915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ee00b2540 0x7f0ee00c9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:26.993 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.986+0000 7f0ee5ee4700 1 --2- 172.21.15.78:0/3274401915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ee006c750 0x7f0ee0151ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:26.993 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.986+0000 7f0ee5ee4700 1 -- 172.21.15.78:0/3274401915 >> 172.21.15.78:0/3274401915 conn(0x7f0ee00d2eb0 msgr2=0x7f0ee00b5ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:26.993 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.986+0000 7f0ee5ee4700 1 -- 172.21.15.78:0/3274401915 shutdown_connections 2023-10-11T09:40:26.993 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:26.986+0000 7f0ee5ee4700 1 -- 172.21.15.78:0/3274401915 wait complete. 2023-10-11T09:40:27.011 INFO:tasks.ceph.osd.0.smithi078.stderr:2023-10-11T09:40:27.006+0000 7fa755c32700 -1 osd.0 0 waiting for initial osdmap 2023-10-11T09:40:27.015 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-11T09:40:27.021 INFO:tasks.ceph.osd.0.smithi078.stderr:2023-10-11T09:40:27.014+0000 7fa74ddf1700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-11T09:40:27.038 INFO:tasks.ceph.osd.1.smithi078.stderr:2023-10-11T09:40:27.034+0000 7fe006327700 -1 osd.1 0 waiting for initial osdmap 2023-10-11T09:40:27.046 INFO:tasks.ceph.osd.1.smithi078.stderr:2023-10-11T09:40:27.042+0000 7fdffe5f1700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-11T09:40:28.975 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:40:28.982 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:40:33.821 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-11T09:40:33.982 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.978+0000 7f4fe8b3e700 1 Processor -- start 2023-10-11T09:40:33.982 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.978+0000 7f4fe8b3e700 1 -- start start 2023-10-11T09:40:33.986 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.982+0000 7f4fe8b3e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4fe40cabf0 0x7f4fe40ad510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:33.986 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.982+0000 7f4fe8b3e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f4fe40cb040 0x7f4fe40ada50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:33.986 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.982+0000 7f4fe8b3e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fe406ba10 con 0x7f4fe415cf10 2023-10-11T09:40:33.986 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.982+0000 7f4fe8b3e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fe40ca7e0 con 0x7f4fe40cb040 2023-10-11T09:40:33.987 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.982+0000 7f4fe8b3e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fe40cd2a0 con 0x7f4fe40cabf0 2023-10-11T09:40:33.987 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.982+0000 7f4fe3fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f4fe40cb040 0x7f4fe40ada50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:33.987 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.982+0000 7f4fe3fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f4fe40cb040 0x7f4fe40ada50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48222/0 (socket says 172.21.15.78:48222) 2023-10-11T09:40:33.987 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.982+0000 7f4fe3fff700 1 -- 172.21.15.78:0/528786657 learned_addr learned my addr 172.21.15.78:0/528786657 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:33.989 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.982+0000 7f4fe2ffd700 1 --2- 172.21.15.78:0/528786657 >> v2:172.21.15.133:3300/0 conn(0x7f4fe40cabf0 0x7f4fe40ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:33.989 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.982+0000 7f4fe3fff700 1 -- 172.21.15.78:0/528786657 >> v2:172.21.15.133:3300/0 conn(0x7f4fe40cabf0 msgr2=0x7f4fe40ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:33.989 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.982+0000 7f4fe3fff700 1 --2- 172.21.15.78:0/528786657 >> v2:172.21.15.133:3300/0 conn(0x7f4fe40cabf0 0x7f4fe40ad510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:33.990 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.982+0000 7f4fe3fff700 1 -- 172.21.15.78:0/528786657 >> v1:172.21.15.133:6789/0 conn(0x7f4fe415cf10 legacy=0x7f4fe415d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:33.990 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.982+0000 7f4fe3fff700 1 -- 172.21.15.78:0/528786657 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe40cd580 con 0x7f4fe40cb040 2023-10-11T09:40:33.990 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.982+0000 7f4fe2ffd700 1 --2- 172.21.15.78:0/528786657 >> v2:172.21.15.133:3300/0 conn(0x7f4fe40cabf0 0x7f4fe40ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:40:33.990 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.982+0000 7f4fe3fff700 1 --2- 172.21.15.78:0/528786657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fe40cb040 0x7f4fe40ada50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4fe40cc220 tx=0x7f4fdc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6db1c72dae51946a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:33.990 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.982+0000 7f4fe27fc700 1 -- 172.21.15.78:0/528786657 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc00e070 con 0x7f4fe40cb040 2023-10-11T09:40:33.991 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.982+0000 7f4fe27fc700 1 -- 172.21.15.78:0/528786657 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fdc003a40 con 0x7f4fe40cb040 2023-10-11T09:40:33.991 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.982+0000 7f4fe27fc700 1 -- 172.21.15.78:0/528786657 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc003df0 con 0x7f4fe40cb040 2023-10-11T09:40:33.991 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.982+0000 7f4fe8b3e700 1 -- 172.21.15.78:0/528786657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fe40cb040 msgr2=0x7f4fe40ada50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:33.991 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.982+0000 7f4fe8b3e700 1 --2- 172.21.15.78:0/528786657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fe40cb040 0x7f4fe40ada50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4fe40cc220 tx=0x7f4fdc00b330 comp rx=0 tx=0).stop 2023-10-11T09:40:33.991 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.982+0000 7f4fe8b3e700 1 -- 172.21.15.78:0/528786657 shutdown_connections 2023-10-11T09:40:33.992 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.982+0000 7f4fe8b3e700 1 --2- 172.21.15.78:0/528786657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fe40cb040 0x7f4fe40ada50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:33.992 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.982+0000 7f4fe8b3e700 1 --2- 172.21.15.78:0/528786657 >> v2:172.21.15.133:3300/0 conn(0x7f4fe40cabf0 0x7f4fe40ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:33.992 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.982+0000 7f4fe8b3e700 1 -- 172.21.15.78:0/528786657 >> 172.21.15.78:0/528786657 conn(0x7f4fe40cf6a0 msgr2=0x7f4fe40cb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:33.992 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.982+0000 7f4fe8b3e700 1 -- 172.21.15.78:0/528786657 shutdown_connections 2023-10-11T09:40:33.992 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.982+0000 7f4fe8b3e700 1 -- 172.21.15.78:0/528786657 wait complete. 2023-10-11T09:40:33.993 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.986+0000 7f4fe8b3e700 1 Processor -- start 2023-10-11T09:40:33.993 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.990+0000 7f4fe8b3e700 1 -- start start 2023-10-11T09:40:33.995 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.990+0000 7f4fe8b3e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fe40cabf0 0x7f4fe40c79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:33.995 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.990+0000 7f4fe8b3e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fe40cb040 0x7f4fe4155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:33.995 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.990+0000 7f4fe8b3e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fe415cf10 0x7f4fe415c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:33.996 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.990+0000 7f4fe8b3e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe40c7520 con 0x7f4fe40cb040 2023-10-11T09:40:33.996 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.990+0000 7f4fe8b3e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe40c6910 con 0x7f4fe40cabf0 2023-10-11T09:40:33.996 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.990+0000 7f4fe8b3e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe40c6bf0 con 0x7f4fe415cf10 2023-10-11T09:40:33.996 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.990+0000 7f4fe2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fe40cb040 0x7f4fe4155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:33.996 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.990+0000 7f4fe2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fe40cb040 0x7f4fe4155e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48228/0 (socket says 172.21.15.78:48228) 2023-10-11T09:40:33.997 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.990+0000 7f4fe2ffd700 1 -- 172.21.15.78:0/523592778 learned_addr learned my addr 172.21.15.78:0/523592778 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:33.998 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.990+0000 7f4fe37fe700 1 --2- 172.21.15.78:0/523592778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fe40cabf0 0x7f4fe40c79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:33.998 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.990+0000 7f4fe3fff700 1 --2- 172.21.15.78:0/523592778 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fe415cf10 0x7f4fe415c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:33.998 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.990+0000 7f4fe2ffd700 1 -- 172.21.15.78:0/523592778 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fe415cf10 msgr2=0x7f4fe415c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:33.999 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.990+0000 7f4fe2ffd700 1 --2- 172.21.15.78:0/523592778 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fe415cf10 0x7f4fe415c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:33.999 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.990+0000 7f4fe2ffd700 1 -- 172.21.15.78:0/523592778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fe40cabf0 msgr2=0x7f4fe40c79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:33.999 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.990+0000 7f4fe2ffd700 1 --2- 172.21.15.78:0/523592778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fe40cabf0 0x7f4fe40c79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:33.999 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.990+0000 7f4fe2ffd700 1 -- 172.21.15.78:0/523592778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fdc009b30 con 0x7f4fe40cb040 2023-10-11T09:40:33.999 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.994+0000 7f4fe2ffd700 1 --2- 172.21.15.78:0/523592778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fe40cb040 0x7f4fe4155e00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4fd800c770 tx=0x7f4fd800ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:34.000 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.994+0000 7f4fcbfff700 1 -- 172.21.15.78:0/523592778 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8011800 con 0x7f4fe40cb040 2023-10-11T09:40:34.000 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.994+0000 7f4fcbfff700 1 -- 172.21.15.78:0/523592778 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd8011960 con 0x7f4fe40cb040 2023-10-11T09:40:34.000 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.994+0000 7f4fe8b3e700 1 -- 172.21.15.78:0/523592778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe40c6ed0 con 0x7f4fe40cb040 2023-10-11T09:40:34.000 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.994+0000 7f4fcbfff700 1 -- 172.21.15.78:0/523592778 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8011d10 con 0x7f4fe40cb040 2023-10-11T09:40:34.000 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.994+0000 7f4fe8b3e700 1 -- 172.21.15.78:0/523592778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe4156340 con 0x7f4fe40cb040 2023-10-11T09:40:34.001 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.994+0000 7f4fe8b3e700 1 -- 172.21.15.78:0/523592778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe40c76d0 con 0x7f4fe40cb040 2023-10-11T09:40:34.001 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.998+0000 7f4fcbfff700 1 -- 172.21.15.78:0/523592778 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f4fd8018360 con 0x7f4fe40cb040 2023-10-11T09:40:34.003 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.998+0000 7f4fcbfff700 1 --2- 172.21.15.78:0/523592778 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4fd0065ed0 0x7f4fd0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:34.003 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.998+0000 7f4fcbfff700 1 -- 172.21.15.78:0/523592778 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4fd80833c0 con 0x7f4fe40cb040 2023-10-11T09:40:34.003 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.998+0000 7f4fe37fe700 1 --2- 172.21.15.78:0/523592778 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4fd0065ed0 0x7f4fd0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:34.004 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:33.998+0000 7f4fe37fe700 1 --2- 172.21.15.78:0/523592778 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4fd0065ed0 0x7f4fd0068380 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4009980 tx=0x7f4fd40090e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:34.011 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:34.006+0000 7f4fcbfff700 1 -- 172.21.15.78:0/523592778 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f4fe40c76d0 con 0x7f4fe40cb040 2023-10-11T09:40:34.192 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:34.186+0000 7f4fe8b3e700 1 -- 172.21.15.78:0/523592778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4fe40c9fb0 con 0x7f4fe40cb040 2023-10-11T09:40:34.194 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:34.190+0000 7f4fcbfff700 1 -- 172.21.15.78:0/523592778 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12382 (secure 0 0 0) 0x7f4fd8050da0 con 0x7f4fe40cb040 2023-10-11T09:40:34.195 INFO:teuthology.misc.health.smithi078.stdout: 2023-10-11T09:40:34.195 INFO:teuthology.misc.health.smithi078.stdout:{"epoch":16,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:40:30.769235+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:40:34.198 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:34.194+0000 7f4fe8b3e700 1 -- 172.21.15.78:0/523592778 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4fd0065ed0 msgr2=0x7f4fd0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:34.198 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:34.194+0000 7f4fe8b3e700 1 --2- 172.21.15.78:0/523592778 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4fd0065ed0 0x7f4fd0068380 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4009980 tx=0x7f4fd40090e0 comp rx=0 tx=0).stop 2023-10-11T09:40:34.198 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:34.194+0000 7f4fe8b3e700 1 -- 172.21.15.78:0/523592778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fe40cb040 msgr2=0x7f4fe4155e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:34.198 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:34.194+0000 7f4fe8b3e700 1 --2- 172.21.15.78:0/523592778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fe40cb040 0x7f4fe4155e00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4fd800c770 tx=0x7f4fd800ca80 comp rx=0 tx=0).stop 2023-10-11T09:40:34.199 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:34.194+0000 7f4fe8b3e700 1 -- 172.21.15.78:0/523592778 shutdown_connections 2023-10-11T09:40:34.199 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:34.194+0000 7f4fe8b3e700 1 --2- 172.21.15.78:0/523592778 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4fd0065ed0 0x7f4fd0068380 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:34.199 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:34.194+0000 7f4fe8b3e700 1 --2- 172.21.15.78:0/523592778 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fe415cf10 0x7f4fe415c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:34.199 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:34.194+0000 7f4fe8b3e700 1 --2- 172.21.15.78:0/523592778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fe40cb040 0x7f4fe4155e00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:34.199 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:34.194+0000 7f4fe8b3e700 1 --2- 172.21.15.78:0/523592778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fe40cabf0 0x7f4fe40c79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:34.200 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:34.194+0000 7f4fe8b3e700 1 -- 172.21.15.78:0/523592778 >> 172.21.15.78:0/523592778 conn(0x7f4fe40cf6a0 msgr2=0x7f4fe4157dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:34.200 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:34.194+0000 7f4fe8b3e700 1 -- 172.21.15.78:0/523592778 shutdown_connections 2023-10-11T09:40:34.200 INFO:teuthology.misc.health.smithi078.stderr:2023-10-11T09:40:34.194+0000 7f4fe8b3e700 1 -- 172.21.15.78:0/523592778 wait complete. 2023-10-11T09:40:34.216 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-11T09:40:34.217 INFO:tasks.ceph:Creating RBD pool 2023-10-11T09:40:34.217 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --cluster ceph osd pool create rbd 8 2023-10-11T09:40:34.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.382+0000 7f610888c700 1 Processor -- start 2023-10-11T09:40:34.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.382+0000 7f610888c700 1 -- start start 2023-10-11T09:40:34.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.386+0000 7f610888c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f61040b2120 0x7f6104159200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:34.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.386+0000 7f610888c700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f610415d270 0x7f610406c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:34.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.386+0000 7f610888c700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f610415a1b0 con 0x7f61041533a0 2023-10-11T09:40:34.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.386+0000 7f610888c700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6104153770 con 0x7f61040b2120 2023-10-11T09:40:34.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.386+0000 7f610888c700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f610406bd10 con 0x7f610415d270 2023-10-11T09:40:34.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.386+0000 7f6102ffd700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f61040b2120 0x7f6104159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:34.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.386+0000 7f6102ffd700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f61040b2120 0x7f6104159200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48230/0 (socket says 172.21.15.78:48230) 2023-10-11T09:40:34.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.386+0000 7f6102ffd700 1 -- 172.21.15.78:0/305345799 learned_addr learned my addr 172.21.15.78:0/305345799 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:34.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.386+0000 7f6103fff700 1 --2- 172.21.15.78:0/305345799 >> v2:172.21.15.133:3300/0 conn(0x7f610415d270 0x7f610406c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:34.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.386+0000 7f6102ffd700 1 -- 172.21.15.78:0/305345799 >> v2:172.21.15.133:3300/0 conn(0x7f610415d270 msgr2=0x7f610406c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:34.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.386+0000 7f6102ffd700 1 --2- 172.21.15.78:0/305345799 >> v2:172.21.15.133:3300/0 conn(0x7f610415d270 0x7f610406c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:34.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.386+0000 7f6102ffd700 1 -- 172.21.15.78:0/305345799 >> v1:172.21.15.133:6789/0 conn(0x7f61041533a0 legacy=0x7f61040b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:34.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.386+0000 7f6102ffd700 1 -- 172.21.15.78:0/305345799 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6104159910 con 0x7f61040b2120 2023-10-11T09:40:34.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.386+0000 7f6102ffd700 1 --2- 172.21.15.78:0/305345799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61040b2120 0x7f6104159200 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f610415c720 tx=0x7f60f8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a61fb2e83dc2620 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:34.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.386+0000 7f61027fc700 1 -- 172.21.15.78:0/305345799 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f8011830 con 0x7f61040b2120 2023-10-11T09:40:34.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.386+0000 7f61027fc700 1 -- 172.21.15.78:0/305345799 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60f8011990 con 0x7f61040b2120 2023-10-11T09:40:34.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.386+0000 7f61027fc700 1 -- 172.21.15.78:0/305345799 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f8011d20 con 0x7f61040b2120 2023-10-11T09:40:34.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.390+0000 7f610888c700 1 -- 172.21.15.78:0/305345799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61040b2120 msgr2=0x7f6104159200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:34.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.390+0000 7f610888c700 1 --2- 172.21.15.78:0/305345799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61040b2120 0x7f6104159200 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f610415c720 tx=0x7f60f8009bf0 comp rx=0 tx=0).stop 2023-10-11T09:40:34.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.390+0000 7f610888c700 1 -- 172.21.15.78:0/305345799 shutdown_connections 2023-10-11T09:40:34.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.390+0000 7f610888c700 1 --2- 172.21.15.78:0/305345799 >> v2:172.21.15.133:3300/0 conn(0x7f610415d270 0x7f610406c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:34.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.390+0000 7f610888c700 1 --2- 172.21.15.78:0/305345799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61040b2120 0x7f6104159200 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:34.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.390+0000 7f610888c700 1 -- 172.21.15.78:0/305345799 >> 172.21.15.78:0/305345799 conn(0x7f610406a060 msgr2=0x7f610406a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:34.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.390+0000 7f610888c700 1 -- 172.21.15.78:0/305345799 shutdown_connections 2023-10-11T09:40:34.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.390+0000 7f610888c700 1 -- 172.21.15.78:0/305345799 wait complete. 2023-10-11T09:40:34.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.390+0000 7f610888c700 1 Processor -- start 2023-10-11T09:40:34.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.394+0000 7f610888c700 1 -- start start 2023-10-11T09:40:34.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.394+0000 7f610888c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61040b2120 0x7f610414fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:34.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.394+0000 7f610888c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61041533a0 0x7f610413b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:34.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.394+0000 7f610888c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f610415d270 0x7f61041354b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:34.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.394+0000 7f610888c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6104135090 con 0x7f61040b2120 2023-10-11T09:40:34.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.394+0000 7f610888c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6104150310 con 0x7f61041533a0 2023-10-11T09:40:34.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.394+0000 7f610888c700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6104134410 con 0x7f610415d270 2023-10-11T09:40:34.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.394+0000 7f61037fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61040b2120 0x7f610414fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:34.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.394+0000 7f61037fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61040b2120 0x7f610414fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48242/0 (socket says 172.21.15.78:48242) 2023-10-11T09:40:34.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.394+0000 7f61037fe700 1 -- 172.21.15.78:0/597519361 learned_addr learned my addr 172.21.15.78:0/597519361 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:34.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.394+0000 7f6102ffd700 1 --2- 172.21.15.78:0/597519361 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61041533a0 0x7f610413b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:34.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.394+0000 7f6103fff700 1 --2- 172.21.15.78:0/597519361 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f610415d270 0x7f61041354b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:34.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.394+0000 7f61037fe700 1 -- 172.21.15.78:0/597519361 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f610415d270 msgr2=0x7f61041354b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:34.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.394+0000 7f61037fe700 1 --2- 172.21.15.78:0/597519361 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f610415d270 0x7f61041354b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:34.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.394+0000 7f61037fe700 1 -- 172.21.15.78:0/597519361 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61041533a0 msgr2=0x7f610413b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:34.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.394+0000 7f61037fe700 1 --2- 172.21.15.78:0/597519361 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61041533a0 0x7f610413b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:34.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.394+0000 7f61037fe700 1 -- 172.21.15.78:0/597519361 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f8009570 con 0x7f61040b2120 2023-10-11T09:40:34.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.394+0000 7f61037fe700 1 --2- 172.21.15.78:0/597519361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61040b2120 0x7f610414fdd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f60f400c500 tx=0x7f60f400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:34.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.398+0000 7f60ebfff700 1 -- 172.21.15.78:0/597519361 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f40052d0 con 0x7f61040b2120 2023-10-11T09:40:34.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.398+0000 7f60ebfff700 1 -- 172.21.15.78:0/597519361 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60f4005430 con 0x7f61040b2120 2023-10-11T09:40:34.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.398+0000 7f610888c700 1 -- 172.21.15.78:0/597519361 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61041346f0 con 0x7f61040b2120 2023-10-11T09:40:34.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.398+0000 7f60ebfff700 1 -- 172.21.15.78:0/597519361 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f40055a0 con 0x7f61040b2120 2023-10-11T09:40:34.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.398+0000 7f610888c700 1 -- 172.21.15.78:0/597519361 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6104134cb0 con 0x7f61040b2120 2023-10-11T09:40:34.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.398+0000 7f610888c700 1 -- 172.21.15.78:0/597519361 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61041351d0 con 0x7f61040b2120 2023-10-11T09:40:34.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.398+0000 7f60ebfff700 1 -- 172.21.15.78:0/597519361 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f60f40038e0 con 0x7f61040b2120 2023-10-11T09:40:34.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.402+0000 7f60ebfff700 1 --2- 172.21.15.78:0/597519361 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f60f0065ed0 0x7f60f0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:34.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.402+0000 7f60ebfff700 1 -- 172.21.15.78:0/597519361 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f60f4082640 con 0x7f61040b2120 2023-10-11T09:40:34.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.402+0000 7f6102ffd700 1 --2- 172.21.15.78:0/597519361 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f60f0065ed0 0x7f60f0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:34.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.402+0000 7f6102ffd700 1 --2- 172.21.15.78:0/597519361 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f60f0065ed0 0x7f60f0068380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f610415c720 tx=0x7f60f8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:34.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.410+0000 7f60ebfff700 1 -- 172.21.15.78:0/597519361 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f61041351d0 con 0x7f61040b2120 2023-10-11T09:40:34.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:34.590+0000 7f610888c700 1 -- 172.21.15.78:0/597519361 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f6104065d40 con 0x7f61040b2120 2023-10-11T09:40:35.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.130+0000 7f60ebfff700 1 -- 172.21.15.78:0/597519361 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v17) v1 ==== 109+0+0 (secure 0 0 0) 0x7f60f40501d0 con 0x7f61040b2120 2023-10-11T09:40:35.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.134+0000 7f610888c700 1 -- 172.21.15.78:0/597519361 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f60f0065ed0 msgr2=0x7f60f0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:35.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.134+0000 7f610888c700 1 --2- 172.21.15.78:0/597519361 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f60f0065ed0 0x7f60f0068380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f610415c720 tx=0x7f60f8008030 comp rx=0 tx=0).stop 2023-10-11T09:40:35.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.134+0000 7f610888c700 1 -- 172.21.15.78:0/597519361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61040b2120 msgr2=0x7f610414fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:35.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.134+0000 7f610888c700 1 --2- 172.21.15.78:0/597519361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61040b2120 0x7f610414fdd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f60f400c500 tx=0x7f60f400c810 comp rx=0 tx=0).stop 2023-10-11T09:40:35.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.134+0000 7f610888c700 1 -- 172.21.15.78:0/597519361 shutdown_connections 2023-10-11T09:40:35.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.134+0000 7f610888c700 1 --2- 172.21.15.78:0/597519361 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f60f0065ed0 0x7f60f0068380 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:35.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.134+0000 7f610888c700 1 --2- 172.21.15.78:0/597519361 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f610415d270 0x7f61041354b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:35.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.134+0000 7f610888c700 1 --2- 172.21.15.78:0/597519361 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61041533a0 0x7f610413b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:35.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.134+0000 7f610888c700 1 --2- 172.21.15.78:0/597519361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61040b2120 0x7f610414fdd0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:35.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.134+0000 7f610888c700 1 -- 172.21.15.78:0/597519361 >> 172.21.15.78:0/597519361 conn(0x7f610406a060 msgr2=0x7f610413cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:35.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.134+0000 7f610888c700 1 -- 172.21.15.78:0/597519361 shutdown_connections 2023-10-11T09:40:35.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.134+0000 7f610888c700 1 -- 172.21.15.78:0/597519361 wait complete. 2023-10-11T09:40:35.140 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'rbd' created 2023-10-11T09:40:35.160 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-10-11T09:40:35.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.326+0000 7eff727b3700 1 Processor -- start 2023-10-11T09:40:35.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.326+0000 7eff727b3700 1 -- start start 2023-10-11T09:40:35.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.326+0000 7eff727b3700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7eff6c0cfbe0 0x7eff6c0d0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:35.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.326+0000 7eff727b3700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff6c15df90 0x7eff6c15e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:35.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.326+0000 7eff727b3700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff6c0cd720 con 0x7eff6c0b4ab0 2023-10-11T09:40:35.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.326+0000 7eff727b3700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff6c15e950 con 0x7eff6c0cfbe0 2023-10-11T09:40:35.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.330+0000 7eff727b3700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c0cda00 con 0x7eff6c15df90 2023-10-11T09:40:35.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.330+0000 7eff70d50700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff6c15df90 0x7eff6c15e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:35.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.330+0000 7eff6b7fe700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7eff6c0cfbe0 0x7eff6c0d0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:35.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.330+0000 7eff6bfff700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7eff6c0b4ab0 0x7eff6c160dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:43538/0 (socket says 172.21.15.78:43538) 2023-10-11T09:40:35.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.330+0000 7eff6bfff700 1 -- 172.21.15.78:0/2086366020 learned_addr learned my addr 172.21.15.78:0/2086366020 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:35.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.330+0000 7eff6affd700 1 -- 172.21.15.78:0/2086366020 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1048076085 0 0) 0x7eff6c0cd720 con 0x7eff6c0b4ab0 2023-10-11T09:40:35.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.330+0000 7eff6affd700 1 -- 172.21.15.78:0/2086366020 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff540035d0 con 0x7eff6c0b4ab0 2023-10-11T09:40:35.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.330+0000 7eff70d50700 1 -- 172.21.15.78:0/2086366020 >> v2:172.21.15.133:3300/0 conn(0x7eff6c0cfbe0 msgr2=0x7eff6c0d0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:35.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.330+0000 7eff70d50700 1 --2- 172.21.15.78:0/2086366020 >> v2:172.21.15.133:3300/0 conn(0x7eff6c0cfbe0 0x7eff6c0d0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:35.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.330+0000 7eff70d50700 1 -- 172.21.15.78:0/2086366020 >> v1:172.21.15.133:6789/0 conn(0x7eff6c0b4ab0 legacy=0x7eff6c160dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:35.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.330+0000 7eff70d50700 1 -- 172.21.15.78:0/2086366020 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff6c0cf3e0 con 0x7eff6c15df90 2023-10-11T09:40:35.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.330+0000 7eff70d50700 1 --2- 172.21.15.78:0/2086366020 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff6c15df90 0x7eff6c15e410 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7eff64009d90 tx=0x7eff6400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d7c995a687f5c36 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:35.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.330+0000 7eff6affd700 1 -- 172.21.15.78:0/2086366020 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6400e070 con 0x7eff6c15df90 2023-10-11T09:40:35.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.330+0000 7eff6affd700 1 -- 172.21.15.78:0/2086366020 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff6400bb20 con 0x7eff6c15df90 2023-10-11T09:40:35.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.330+0000 7eff6affd700 1 -- 172.21.15.78:0/2086366020 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff64012430 con 0x7eff6c15df90 2023-10-11T09:40:35.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.334+0000 7eff727b3700 1 -- 172.21.15.78:0/2086366020 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff6c15df90 msgr2=0x7eff6c15e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:35.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.334+0000 7eff727b3700 1 --2- 172.21.15.78:0/2086366020 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff6c15df90 0x7eff6c15e410 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7eff64009d90 tx=0x7eff6400b330 comp rx=0 tx=0).stop 2023-10-11T09:40:35.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.334+0000 7eff727b3700 1 -- 172.21.15.78:0/2086366020 shutdown_connections 2023-10-11T09:40:35.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.334+0000 7eff727b3700 1 --2- 172.21.15.78:0/2086366020 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff6c15df90 0x7eff6c15e410 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:35.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.334+0000 7eff727b3700 1 --2- 172.21.15.78:0/2086366020 >> v2:172.21.15.133:3300/0 conn(0x7eff6c0cfbe0 0x7eff6c0d0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:35.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.334+0000 7eff727b3700 1 -- 172.21.15.78:0/2086366020 >> 172.21.15.78:0/2086366020 conn(0x7eff6c06b1d0 msgr2=0x7eff6c06b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:35.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.334+0000 7eff727b3700 1 -- 172.21.15.78:0/2086366020 shutdown_connections 2023-10-11T09:40:35.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.334+0000 7eff727b3700 1 -- 172.21.15.78:0/2086366020 wait complete. 2023-10-11T09:40:35.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.334+0000 7eff727b3700 1 Processor -- start 2023-10-11T09:40:35.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.338+0000 7eff727b3700 1 -- start start 2023-10-11T09:40:35.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.338+0000 7eff727b3700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff6c0b4ab0 0x7eff6c0c9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:35.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.338+0000 7eff727b3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff6c0cfbe0 0x7eff6c0c6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:35.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.338+0000 7eff727b3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff6c15df90 0x7eff6c0d1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:35.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.338+0000 7eff727b3700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c15c110 con 0x7eff6c15df90 2023-10-11T09:40:35.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.338+0000 7eff727b3700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c0c97d0 con 0x7eff6c0cfbe0 2023-10-11T09:40:35.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.338+0000 7eff727b3700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c0d0d80 con 0x7eff6c0b4ab0 2023-10-11T09:40:35.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.338+0000 7eff70d50700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff6c15df90 0x7eff6c0d1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:35.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.338+0000 7eff6bfff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff6c0b4ab0 0x7eff6c0c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:35.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.338+0000 7eff70d50700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff6c15df90 0x7eff6c0d1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57994/0 (socket says 172.21.15.78:57994) 2023-10-11T09:40:35.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.338+0000 7eff70d50700 1 -- 172.21.15.78:0/1056676141 learned_addr learned my addr 172.21.15.78:0/1056676141 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:35.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.338+0000 7eff6b7fe700 1 --2- 172.21.15.78:0/1056676141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff6c0cfbe0 0x7eff6c0c6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:35.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.338+0000 7eff6bfff700 1 -- 172.21.15.78:0/1056676141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff6c0cfbe0 msgr2=0x7eff6c0c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:35.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.338+0000 7eff6bfff700 1 --2- 172.21.15.78:0/1056676141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff6c0cfbe0 0x7eff6c0c6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:35.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.338+0000 7eff6bfff700 1 -- 172.21.15.78:0/1056676141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff6c15df90 msgr2=0x7eff6c0d1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:35.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.338+0000 7eff6bfff700 1 --2- 172.21.15.78:0/1056676141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff6c15df90 0x7eff6c0d1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:35.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.338+0000 7eff6bfff700 1 -- 172.21.15.78:0/1056676141 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff64009a70 con 0x7eff6c0b4ab0 2023-10-11T09:40:35.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.338+0000 7eff70d50700 1 --2- 172.21.15.78:0/1056676141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff6c15df90 0x7eff6c0d1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:40:35.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.338+0000 7eff6bfff700 1 --2- 172.21.15.78:0/1056676141 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff6c0b4ab0 0x7eff6c0c9160 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00c410 tx=0x7eff5c00c720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:35.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.342+0000 7eff68ff9700 1 -- 172.21.15.78:0/1056676141 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c005d90 con 0x7eff6c0b4ab0 2023-10-11T09:40:35.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.342+0000 7eff68ff9700 1 -- 172.21.15.78:0/1056676141 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5c00ce20 con 0x7eff6c0b4ab0 2023-10-11T09:40:35.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.342+0000 7eff68ff9700 1 -- 172.21.15.78:0/1056676141 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c007310 con 0x7eff6c0b4ab0 2023-10-11T09:40:35.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.342+0000 7eff727b3700 1 -- 172.21.15.78:0/1056676141 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff6c0d1060 con 0x7eff6c0b4ab0 2023-10-11T09:40:35.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.342+0000 7eff727b3700 1 -- 172.21.15.78:0/1056676141 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff6c0c7180 con 0x7eff6c0b4ab0 2023-10-11T09:40:35.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.342+0000 7eff727b3700 1 -- 172.21.15.78:0/1056676141 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff6c04e530 con 0x7eff6c0b4ab0 2023-10-11T09:40:35.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.342+0000 7eff68ff9700 1 -- 172.21.15.78:0/1056676141 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7eff5c007470 con 0x7eff6c0b4ab0 2023-10-11T09:40:35.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.346+0000 7eff68ff9700 1 --2- 172.21.15.78:0/1056676141 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7eff58065ed0 0x7eff58068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:35.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.346+0000 7eff68ff9700 1 -- 172.21.15.78:0/1056676141 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5199+0+0 (secure 0 0 0) 0x7eff5c082470 con 0x7eff6c0b4ab0 2023-10-11T09:40:35.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.346+0000 7eff6b7fe700 1 --2- 172.21.15.78:0/1056676141 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7eff58065ed0 0x7eff58068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:35.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.346+0000 7eff6b7fe700 1 --2- 172.21.15.78:0/1056676141 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7eff58065ed0 0x7eff58068380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7eff6c15feb0 tx=0x7eff60009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:35.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.354+0000 7eff68ff9700 1 -- 172.21.15.78:0/1056676141 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7eff5c04fe10 con 0x7eff6c0b4ab0 2023-10-11T09:40:35.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:35.526+0000 7eff727b3700 1 -- 172.21.15.78:0/1056676141 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7eff6c0d0ec0 con 0x7eff6c0b4ab0 2023-10-11T09:40:36.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.138+0000 7eff68ff9700 1 -- 172.21.15.78:0/1056676141 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v18) v1 ==== 173+0+0 (secure 0 0 0) 0x7eff6c0d0ec0 con 0x7eff6c0b4ab0 2023-10-11T09:40:36.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.142+0000 7eff727b3700 1 -- 172.21.15.78:0/1056676141 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7eff58065ed0 msgr2=0x7eff58068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:36.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.142+0000 7eff727b3700 1 --2- 172.21.15.78:0/1056676141 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7eff58065ed0 0x7eff58068380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7eff6c15feb0 tx=0x7eff60009240 comp rx=0 tx=0).stop 2023-10-11T09:40:36.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.142+0000 7eff727b3700 1 -- 172.21.15.78:0/1056676141 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff6c0b4ab0 msgr2=0x7eff6c0c9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:36.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.142+0000 7eff727b3700 1 --2- 172.21.15.78:0/1056676141 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff6c0b4ab0 0x7eff6c0c9160 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00c410 tx=0x7eff5c00c720 comp rx=0 tx=0).stop 2023-10-11T09:40:36.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.146+0000 7eff727b3700 1 -- 172.21.15.78:0/1056676141 shutdown_connections 2023-10-11T09:40:36.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.146+0000 7eff727b3700 1 --2- 172.21.15.78:0/1056676141 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7eff58065ed0 0x7eff58068380 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:36.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.146+0000 7eff727b3700 1 --2- 172.21.15.78:0/1056676141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff6c15df90 0x7eff6c0d1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:36.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.146+0000 7eff727b3700 1 --2- 172.21.15.78:0/1056676141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff6c0cfbe0 0x7eff6c0c6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:36.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.146+0000 7eff727b3700 1 --2- 172.21.15.78:0/1056676141 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff6c0b4ab0 0x7eff6c0c9160 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:36.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.146+0000 7eff727b3700 1 -- 172.21.15.78:0/1056676141 >> 172.21.15.78:0/1056676141 conn(0x7eff6c06b1d0 msgr2=0x7eff6c0b7f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:36.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.146+0000 7eff727b3700 1 -- 172.21.15.78:0/1056676141 shutdown_connections 2023-10-11T09:40:36.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.146+0000 7eff727b3700 1 -- 172.21.15.78:0/1056676141 wait complete. 2023-10-11T09:40:36.154 INFO:teuthology.orchestra.run.smithi078.stderr:enabled application 'rbd' on pool 'rbd' 2023-10-11T09:40:36.172 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-11T09:40:36.173 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-11T09:40:36.173 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-11T09:40:36.181 INFO:tasks.ceph.mds.a:Started 2023-10-11T09:40:36.182 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-11T09:40:36.183 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-11T09:40:36.193 INFO:tasks.ceph.mds.c:Started 2023-10-11T09:40:36.194 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-11T09:40:36.195 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-11T09:40:36.205 INFO:tasks.ceph.mds.b:Started 2023-10-11T09:40:36.205 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-11T09:40:36.205 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-11T09:40:36.211 INFO:tasks.ceph.mds.d:Started 2023-10-11T09:40:36.212 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-11T09:40:36.213 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:40:36.305 INFO:tasks.ceph.mds.a.smithi078.stdout:starting mds.a at 2023-10-11T09:40:36.311 INFO:tasks.ceph.mds.c.smithi078.stdout:starting mds.c at 2023-10-11T09:40:36.328 INFO:tasks.ceph.mds.b.smithi133.stdout:starting mds.b at 2023-10-11T09:40:36.329 INFO:tasks.ceph.mds.d.smithi133.stdout:starting mds.d at 2023-10-11T09:40:36.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.374+0000 7f3a519b5700 1 Processor -- start 2023-10-11T09:40:36.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.374+0000 7f3a519b5700 1 -- start start 2023-10-11T09:40:36.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.374+0000 7f3a519b5700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3a4c160c50 0x7f3a4c0ad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:36.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.374+0000 7f3a519b5700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a4c0caf90 0x7f3a4c0cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:36.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.374+0000 7f3a519b5700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a4c0ae280 con 0x7f3a4c0cdcf0 2023-10-11T09:40:36.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.374+0000 7f3a519b5700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a4c0ce170 con 0x7f3a4c160c50 2023-10-11T09:40:36.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.374+0000 7f3a519b5700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c06be50 con 0x7f3a4c0caf90 2023-10-11T09:40:36.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.374+0000 7f3a4bfff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a4c0caf90 0x7f3a4c0cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:36.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.374+0000 7f3a509b3700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3a4c160c50 0x7f3a4c0ad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:36.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.374+0000 7f3a4bfff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a4c0caf90 0x7f3a4c0cd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:38032/0 (socket says 172.21.15.78:38032) 2023-10-11T09:40:36.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.374+0000 7f3a4bfff700 1 -- 172.21.15.78:0/190535261 learned_addr learned my addr 172.21.15.78:0/190535261 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:36.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.374+0000 7f3a509b3700 1 -- 172.21.15.78:0/190535261 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a4c0caf90 msgr2=0x7f3a4c0cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:36.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.374+0000 7f3a509b3700 1 --2- 172.21.15.78:0/190535261 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a4c0caf90 0x7f3a4c0cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:36.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.374+0000 7f3a509b3700 1 -- 172.21.15.78:0/190535261 >> v1:172.21.15.133:6789/0 conn(0x7f3a4c0cdcf0 legacy=0x7f3a4c0cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:36.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.378+0000 7f3a509b3700 1 -- 172.21.15.78:0/190535261 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a4c06b520 con 0x7f3a4c160c50 2023-10-11T09:40:36.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.378+0000 7f3a4bfff700 1 --2- 172.21.15.78:0/190535261 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a4c0caf90 0x7f3a4c0cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:40:36.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.378+0000 7f3a509b3700 1 --2- 172.21.15.78:0/190535261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a4c160c50 0x7f3a4c0ad9c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3a40009850 tx=0x7f3a40009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=541c60bdd5b68dcc server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:36.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.378+0000 7f3a4b7fe700 1 -- 172.21.15.78:0/190535261 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a40011830 con 0x7f3a4c160c50 2023-10-11T09:40:36.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.378+0000 7f3a4b7fe700 1 -- 172.21.15.78:0/190535261 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a40011990 con 0x7f3a4c160c50 2023-10-11T09:40:36.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.378+0000 7f3a4b7fe700 1 -- 172.21.15.78:0/190535261 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a40011d40 con 0x7f3a4c160c50 2023-10-11T09:40:36.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.378+0000 7f3a519b5700 1 -- 172.21.15.78:0/190535261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a4c160c50 msgr2=0x7f3a4c0ad9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:36.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.378+0000 7f3a519b5700 1 --2- 172.21.15.78:0/190535261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a4c160c50 0x7f3a4c0ad9c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3a40009850 tx=0x7f3a40009b60 comp rx=0 tx=0).stop 2023-10-11T09:40:36.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.378+0000 7f3a519b5700 1 -- 172.21.15.78:0/190535261 shutdown_connections 2023-10-11T09:40:36.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.378+0000 7f3a519b5700 1 --2- 172.21.15.78:0/190535261 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a4c0caf90 0x7f3a4c0cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:36.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.378+0000 7f3a519b5700 1 --2- 172.21.15.78:0/190535261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a4c160c50 0x7f3a4c0ad9c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:36.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.378+0000 7f3a519b5700 1 -- 172.21.15.78:0/190535261 >> 172.21.15.78:0/190535261 conn(0x7f3a4c0cc070 msgr2=0x7f3a4c0ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:36.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.378+0000 7f3a519b5700 1 -- 172.21.15.78:0/190535261 shutdown_connections 2023-10-11T09:40:36.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.378+0000 7f3a519b5700 1 -- 172.21.15.78:0/190535261 wait complete. 2023-10-11T09:40:36.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.382+0000 7f3a519b5700 1 Processor -- start 2023-10-11T09:40:36.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.382+0000 7f3a519b5700 1 -- start start 2023-10-11T09:40:36.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.386+0000 7f3a519b5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a4c0caf90 0x7f3a4c0c65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:36.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.386+0000 7f3a519b5700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a4c0cdcf0 0x7f3a4c15eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:36.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.386+0000 7f3a519b5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a4c160c50 0x7f3a4c1612b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:36.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.386+0000 7f3a519b5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c0c7990 con 0x7f3a4c160c50 2023-10-11T09:40:36.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.386+0000 7f3a519b5700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c0c6ae0 con 0x7f3a4c0caf90 2023-10-11T09:40:36.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.386+0000 7f3a519b5700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c0c7e20 con 0x7f3a4c0cdcf0 2023-10-11T09:40:36.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.386+0000 7f3a511b4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a4c160c50 0x7f3a4c1612b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:36.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.386+0000 7f3a511b4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a4c160c50 0x7f3a4c1612b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58030/0 (socket says 172.21.15.78:58030) 2023-10-11T09:40:36.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.386+0000 7f3a511b4700 1 -- 172.21.15.78:0/4227837908 learned_addr learned my addr 172.21.15.78:0/4227837908 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:36.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.386+0000 7f3a509b3700 1 --2- 172.21.15.78:0/4227837908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a4c0caf90 0x7f3a4c0c65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:36.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.386+0000 7f3a4bfff700 1 --2- 172.21.15.78:0/4227837908 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a4c0cdcf0 0x7f3a4c15eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:36.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.386+0000 7f3a511b4700 1 -- 172.21.15.78:0/4227837908 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a4c0cdcf0 msgr2=0x7f3a4c15eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:36.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.386+0000 7f3a511b4700 1 --2- 172.21.15.78:0/4227837908 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a4c0cdcf0 0x7f3a4c15eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:36.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.386+0000 7f3a511b4700 1 -- 172.21.15.78:0/4227837908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a4c0caf90 msgr2=0x7f3a4c0c65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:36.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.386+0000 7f3a511b4700 1 --2- 172.21.15.78:0/4227837908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a4c0caf90 0x7f3a4c0c65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:36.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.386+0000 7f3a511b4700 1 -- 172.21.15.78:0/4227837908 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a38009570 con 0x7f3a4c160c50 2023-10-11T09:40:36.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.386+0000 7f3a511b4700 1 --2- 172.21.15.78:0/4227837908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a4c160c50 0x7f3a4c1612b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3a4400cc50 tx=0x7f3a4400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:36.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.386+0000 7f3a497fa700 1 -- 172.21.15.78:0/4227837908 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a44003c20 con 0x7f3a4c160c50 2023-10-11T09:40:36.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.386+0000 7f3a497fa700 1 -- 172.21.15.78:0/4227837908 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a44003d80 con 0x7f3a4c160c50 2023-10-11T09:40:36.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.386+0000 7f3a519b5700 1 -- 172.21.15.78:0/4227837908 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a400094b0 con 0x7f3a4c160c50 2023-10-11T09:40:36.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.386+0000 7f3a497fa700 1 -- 172.21.15.78:0/4227837908 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4400aab0 con 0x7f3a4c160c50 2023-10-11T09:40:36.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.386+0000 7f3a519b5700 1 -- 172.21.15.78:0/4227837908 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a4c0c6d80 con 0x7f3a4c160c50 2023-10-11T09:40:36.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.390+0000 7f3a519b5700 1 -- 172.21.15.78:0/4227837908 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a4c06bac0 con 0x7f3a4c160c50 2023-10-11T09:40:36.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.390+0000 7f3a497fa700 1 -- 172.21.15.78:0/4227837908 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f3a4400ac10 con 0x7f3a4c160c50 2023-10-11T09:40:36.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.394+0000 7f3a497fa700 1 --2- 172.21.15.78:0/4227837908 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3a30065ed0 0x7f3a30068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:36.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.394+0000 7f3a497fa700 1 -- 172.21.15.78:0/4227837908 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3a440832f0 con 0x7f3a4c160c50 2023-10-11T09:40:36.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.394+0000 7f3a509b3700 1 --2- 172.21.15.78:0/4227837908 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3a30065ed0 0x7f3a30068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:36.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.394+0000 7f3a509b3700 1 --2- 172.21.15.78:0/4227837908 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3a30065ed0 0x7f3a30068380 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3a400090b0 tx=0x7f3a4001c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:36.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.402+0000 7f3a497fa700 1 -- 172.21.15.78:0/4227837908 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f3a4c06bac0 con 0x7f3a4c160c50 2023-10-11T09:40:36.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.582+0000 7f3a519b5700 1 -- 172.21.15.78:0/4227837908 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a4c0ae280 con 0x7f3a4c160c50 2023-10-11T09:40:36.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.586+0000 7f3a497fa700 1 -- 172.21.15.78:0/4227837908 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+14181 (secure 0 0 0) 0x7f3a4c0ae280 con 0x7f3a4c160c50 2023-10-11T09:40:36.590 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:40:36.591 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":18,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:40:36.130055+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:40:36.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.586+0000 7f3a519b5700 1 -- 172.21.15.78:0/4227837908 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3a30065ed0 msgr2=0x7f3a30068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:36.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.586+0000 7f3a519b5700 1 --2- 172.21.15.78:0/4227837908 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3a30065ed0 0x7f3a30068380 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3a400090b0 tx=0x7f3a4001c040 comp rx=0 tx=0).stop 2023-10-11T09:40:36.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.586+0000 7f3a519b5700 1 -- 172.21.15.78:0/4227837908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a4c160c50 msgr2=0x7f3a4c1612b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:36.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.586+0000 7f3a519b5700 1 --2- 172.21.15.78:0/4227837908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a4c160c50 0x7f3a4c1612b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3a4400cc50 tx=0x7f3a4400cf60 comp rx=0 tx=0).stop 2023-10-11T09:40:36.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.590+0000 7f3a519b5700 1 -- 172.21.15.78:0/4227837908 shutdown_connections 2023-10-11T09:40:36.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.590+0000 7f3a519b5700 1 --2- 172.21.15.78:0/4227837908 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3a30065ed0 0x7f3a30068380 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:36.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.590+0000 7f3a519b5700 1 --2- 172.21.15.78:0/4227837908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a4c160c50 0x7f3a4c1612b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:36.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.590+0000 7f3a519b5700 1 --2- 172.21.15.78:0/4227837908 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a4c0cdcf0 0x7f3a4c15eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:36.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.590+0000 7f3a519b5700 1 --2- 172.21.15.78:0/4227837908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a4c0caf90 0x7f3a4c0c65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:36.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.590+0000 7f3a519b5700 1 -- 172.21.15.78:0/4227837908 >> 172.21.15.78:0/4227837908 conn(0x7f3a4c0cc070 msgr2=0x7f3a4c06c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:36.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.590+0000 7f3a519b5700 1 -- 172.21.15.78:0/4227837908 shutdown_connections 2023-10-11T09:40:36.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.590+0000 7f3a519b5700 1 -- 172.21.15.78:0/4227837908 wait complete. 2023-10-11T09:40:36.611 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:40:36.612 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:40:36.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.782+0000 7fcaae445700 1 Processor -- start 2023-10-11T09:40:36.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.782+0000 7fcaae445700 1 -- start start 2023-10-11T09:40:36.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.786+0000 7fcaae445700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcaa80b3610 0x7fcaa80d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:36.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.786+0000 7fcaae445700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fcaa80b3a90 0x7fcaa80b85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:36.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.786+0000 7fcaae445700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaa80b47f0 con 0x7fcaa80b3240 2023-10-11T09:40:36.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.786+0000 7fcaae445700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaa80b8130 con 0x7fcaa80b3a90 2023-10-11T09:40:36.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.786+0000 7fcaae445700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa80b3f10 con 0x7fcaa80b3610 2023-10-11T09:40:36.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.786+0000 7fcaad443700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fcaa80b3240 0x7fcaa815d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:51534/0 (socket says 172.21.15.78:51534) 2023-10-11T09:40:36.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.786+0000 7fcaad443700 1 -- 172.21.15.78:0/26790041 learned_addr learned my addr 172.21.15.78:0/26790041 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:36.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.786+0000 7fcaadc44700 1 --2- 172.21.15.78:0/26790041 >> v2:172.21.15.78:3300/0 conn(0x7fcaa80b3a90 0x7fcaa80b85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:36.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.786+0000 7fcaacc42700 1 --2- 172.21.15.78:0/26790041 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcaa80b3610 0x7fcaa80d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:36.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.786+0000 7fca9ffff700 1 -- 172.21.15.78:0/26790041 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2523276635 0 0) 0x7fcaa4001110 con 0x7fcaa80b3240 2023-10-11T09:40:36.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.786+0000 7fca9ffff700 1 -- 172.21.15.78:0/26790041 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca940035d0 con 0x7fcaa80b3240 2023-10-11T09:40:36.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.786+0000 7fcaadc44700 1 -- 172.21.15.78:0/26790041 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcaa80b3610 msgr2=0x7fcaa80d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:36.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.786+0000 7fcaadc44700 1 --2- 172.21.15.78:0/26790041 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcaa80b3610 0x7fcaa80d29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:36.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.786+0000 7fcaadc44700 1 -- 172.21.15.78:0/26790041 >> v1:172.21.15.78:6789/0 conn(0x7fcaa80b3240 legacy=0x7fcaa815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:36.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.786+0000 7fcaadc44700 1 -- 172.21.15.78:0/26790041 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa80b41e0 con 0x7fcaa80b3a90 2023-10-11T09:40:36.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.786+0000 7fcaadc44700 1 --2- 172.21.15.78:0/26790041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa80b3a90 0x7fcaa80b85c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0009d90 tx=0x7fcaa000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b446897434cd473 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:36.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.786+0000 7fca9ffff700 1 -- 172.21.15.78:0/26790041 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa000e070 con 0x7fcaa80b3a90 2023-10-11T09:40:36.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.786+0000 7fca9ffff700 1 -- 172.21.15.78:0/26790041 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa000bb20 con 0x7fcaa80b3a90 2023-10-11T09:40:36.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.786+0000 7fca9ffff700 1 -- 172.21.15.78:0/26790041 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa000be90 con 0x7fcaa80b3a90 2023-10-11T09:40:36.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.790+0000 7fcaae445700 1 -- 172.21.15.78:0/26790041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa80b3a90 msgr2=0x7fcaa80b85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:36.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.790+0000 7fcaae445700 1 --2- 172.21.15.78:0/26790041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa80b3a90 0x7fcaa80b85c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0009d90 tx=0x7fcaa000b330 comp rx=0 tx=0).stop 2023-10-11T09:40:36.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.790+0000 7fcaae445700 1 -- 172.21.15.78:0/26790041 shutdown_connections 2023-10-11T09:40:36.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.790+0000 7fcaae445700 1 --2- 172.21.15.78:0/26790041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa80b3a90 0x7fcaa80b85c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:36.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.790+0000 7fcaae445700 1 --2- 172.21.15.78:0/26790041 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcaa80b3610 0x7fcaa80d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:36.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.790+0000 7fcaae445700 1 -- 172.21.15.78:0/26790041 >> 172.21.15.78:0/26790041 conn(0x7fcaa809c260 msgr2=0x7fcaa80d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:36.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.790+0000 7fcaae445700 1 -- 172.21.15.78:0/26790041 shutdown_connections 2023-10-11T09:40:36.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.790+0000 7fcaae445700 1 -- 172.21.15.78:0/26790041 wait complete. 2023-10-11T09:40:36.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.790+0000 7fcaae445700 1 Processor -- start 2023-10-11T09:40:36.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.790+0000 7fcaae445700 1 -- start start 2023-10-11T09:40:36.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.794+0000 7fcaae445700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcaa80b3240 0x7fcaa80c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:36.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.794+0000 7fcaae445700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcaa80b3610 0x7fcaa80c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:36.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.794+0000 7fcaae445700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa80b3a90 0x7fcaa814c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:36.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.794+0000 7fcaae445700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa809e210 con 0x7fcaa80b3a90 2023-10-11T09:40:36.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.794+0000 7fcaae445700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa815d1e0 con 0x7fcaa80b3610 2023-10-11T09:40:36.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.794+0000 7fcaae445700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa815d4c0 con 0x7fcaa80b3240 2023-10-11T09:40:36.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.794+0000 7fcaadc44700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa80b3a90 0x7fcaa814c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:36.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.794+0000 7fcaadc44700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa80b3a90 0x7fcaa814c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58046/0 (socket says 172.21.15.78:58046) 2023-10-11T09:40:36.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.794+0000 7fcaadc44700 1 -- 172.21.15.78:0/1586032569 learned_addr learned my addr 172.21.15.78:0/1586032569 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:36.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.794+0000 7fcaacc42700 1 --2- 172.21.15.78:0/1586032569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcaa80b3610 0x7fcaa80c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:36.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.794+0000 7fcaad443700 1 --2- 172.21.15.78:0/1586032569 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcaa80b3240 0x7fcaa80c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:36.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.794+0000 7fcaadc44700 1 -- 172.21.15.78:0/1586032569 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcaa80b3240 msgr2=0x7fcaa80c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:36.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.794+0000 7fcaadc44700 1 --2- 172.21.15.78:0/1586032569 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcaa80b3240 0x7fcaa80c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:36.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.794+0000 7fcaadc44700 1 -- 172.21.15.78:0/1586032569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcaa80b3610 msgr2=0x7fcaa80c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:36.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.794+0000 7fcaadc44700 1 --2- 172.21.15.78:0/1586032569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcaa80b3610 0x7fcaa80c95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:36.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.794+0000 7fcaadc44700 1 -- 172.21.15.78:0/1586032569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa0009a70 con 0x7fcaa80b3a90 2023-10-11T09:40:36.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.794+0000 7fcaadc44700 1 --2- 172.21.15.78:0/1586032569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa80b3a90 0x7fcaa814c000 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0014040 tx=0x7fcaa0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:36.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.794+0000 7fca9dffb700 1 -- 172.21.15.78:0/1586032569 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0004960 con 0x7fcaa80b3a90 2023-10-11T09:40:36.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.794+0000 7fca9dffb700 1 -- 172.21.15.78:0/1586032569 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa0020070 con 0x7fcaa80b3a90 2023-10-11T09:40:36.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.794+0000 7fca9dffb700 1 -- 172.21.15.78:0/1586032569 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa00276c0 con 0x7fcaa80b3a90 2023-10-11T09:40:36.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.794+0000 7fcaae445700 1 -- 172.21.15.78:0/1586032569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaa815d7a0 con 0x7fcaa80b3a90 2023-10-11T09:40:36.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.794+0000 7fcaae445700 1 -- 172.21.15.78:0/1586032569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa815dd60 con 0x7fcaa80b3a90 2023-10-11T09:40:36.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.798+0000 7fcaae445700 1 -- 172.21.15.78:0/1586032569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaa8065d40 con 0x7fcaa80b3a90 2023-10-11T09:40:36.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.798+0000 7fca9dffb700 1 -- 172.21.15.78:0/1586032569 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fcaa0012430 con 0x7fcaa80b3a90 2023-10-11T09:40:36.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.798+0000 7fca9dffb700 1 --2- 172.21.15.78:0/1586032569 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fca8c065ed0 0x7fca8c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:36.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.798+0000 7fca9dffb700 1 -- 172.21.15.78:0/1586032569 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fcaa0022070 con 0x7fcaa80b3a90 2023-10-11T09:40:36.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.798+0000 7fcaad443700 1 --2- 172.21.15.78:0/1586032569 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fca8c065ed0 0x7fca8c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:36.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.802+0000 7fcaad443700 1 --2- 172.21.15.78:0/1586032569 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fca8c065ed0 0x7fca8c068380 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4005930 tx=0x7fcaa4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:36.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.806+0000 7fca9dffb700 1 -- 172.21.15.78:0/1586032569 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fcaa0052ee0 con 0x7fcaa80b3a90 2023-10-11T09:40:36.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.978+0000 7fcaae445700 1 -- 172.21.15.78:0/1586032569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcaa80b7300 con 0x7fcaa80b3a90 2023-10-11T09:40:36.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.978+0000 7fca9dffb700 1 -- 172.21.15.78:0/1586032569 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcaa002c020 con 0x7fcaa80b3a90 2023-10-11T09:40:36.985 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:40:36.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.986+0000 7fcaae445700 1 -- 172.21.15.78:0/1586032569 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fca8c065ed0 msgr2=0x7fca8c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:36.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.986+0000 7fcaae445700 1 --2- 172.21.15.78:0/1586032569 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fca8c065ed0 0x7fca8c068380 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4005930 tx=0x7fcaa4009040 comp rx=0 tx=0).stop 2023-10-11T09:40:36.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.986+0000 7fcaae445700 1 -- 172.21.15.78:0/1586032569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa80b3a90 msgr2=0x7fcaa814c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:36.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.986+0000 7fcaae445700 1 --2- 172.21.15.78:0/1586032569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa80b3a90 0x7fcaa814c000 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0014040 tx=0x7fcaa0004af0 comp rx=0 tx=0).stop 2023-10-11T09:40:36.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.986+0000 7fcaae445700 1 -- 172.21.15.78:0/1586032569 shutdown_connections 2023-10-11T09:40:36.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.986+0000 7fcaae445700 1 --2- 172.21.15.78:0/1586032569 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fca8c065ed0 0x7fca8c068380 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:36.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.986+0000 7fcaae445700 1 --2- 172.21.15.78:0/1586032569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa80b3a90 0x7fcaa814c000 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:36.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.986+0000 7fcaae445700 1 --2- 172.21.15.78:0/1586032569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcaa80b3610 0x7fcaa80c95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:36.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.986+0000 7fcaae445700 1 --2- 172.21.15.78:0/1586032569 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcaa80b3240 0x7fcaa80c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:36.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.986+0000 7fcaae445700 1 -- 172.21.15.78:0/1586032569 >> 172.21.15.78:0/1586032569 conn(0x7fcaa809c260 msgr2=0x7fcaa80b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:36.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.986+0000 7fcaae445700 1 -- 172.21.15.78:0/1586032569 shutdown_connections 2023-10-11T09:40:36.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:36.986+0000 7fcaae445700 1 -- 172.21.15.78:0/1586032569 wait complete. 2023-10-11T09:40:37.011 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:40:37.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.174+0000 7f6719ebd700 1 Processor -- start 2023-10-11T09:40:37.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.174+0000 7f6719ebd700 1 -- start start 2023-10-11T09:40:37.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.174+0000 7f6719ebd700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f671415c900 0x7f67140b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:37.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.174+0000 7f6719ebd700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f67140ad290 0x7f6714071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:37.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.174+0000 7f6719ebd700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671406fde0 con 0x7f67140ce7f0 2023-10-11T09:40:37.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.174+0000 7f6719ebd700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67140705f0 con 0x7f67140ad290 2023-10-11T09:40:37.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.174+0000 7f6719ebd700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6714071fc0 con 0x7f671415c900 2023-10-11T09:40:37.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.178+0000 7f6713fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f67140ad290 0x7f6714071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:37.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.178+0000 7f6718ebb700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f671415c900 0x7f67140b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:37.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.178+0000 7f6713fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f67140ad290 0x7f6714071a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58048/0 (socket says 172.21.15.78:58048) 2023-10-11T09:40:37.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.178+0000 7f67196bc700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f67140ce7f0 0x7f671406bfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:43564/0 (socket says 172.21.15.78:43564) 2023-10-11T09:40:37.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.178+0000 7f6713fff700 1 -- 172.21.15.78:0/2698158464 learned_addr learned my addr 172.21.15.78:0/2698158464 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:37.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.178+0000 7f6713fff700 1 -- 172.21.15.78:0/2698158464 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f671415c900 msgr2=0x7f67140b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:37.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.178+0000 7f6713fff700 1 --2- 172.21.15.78:0/2698158464 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f671415c900 0x7f67140b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:37.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.178+0000 7f6713fff700 1 -- 172.21.15.78:0/2698158464 >> v1:172.21.15.133:6789/0 conn(0x7f67140ce7f0 legacy=0x7f671406bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:37.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.178+0000 7f6713fff700 1 -- 172.21.15.78:0/2698158464 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67140ae000 con 0x7f67140ad290 2023-10-11T09:40:37.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.178+0000 7f6713fff700 1 --2- 172.21.15.78:0/2698158464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67140ad290 0x7f6714071a80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6700009940 tx=0x7f6700009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a438c6d48a0da7 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:37.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.178+0000 7f67137fe700 1 -- 172.21.15.78:0/2698158464 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6700011830 con 0x7f67140ad290 2023-10-11T09:40:37.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.178+0000 7f67137fe700 1 -- 172.21.15.78:0/2698158464 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6700011990 con 0x7f67140ad290 2023-10-11T09:40:37.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.178+0000 7f67137fe700 1 -- 172.21.15.78:0/2698158464 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6700011d20 con 0x7f67140ad290 2023-10-11T09:40:37.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.178+0000 7f6719ebd700 1 -- 172.21.15.78:0/2698158464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67140ad290 msgr2=0x7f6714071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:37.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.178+0000 7f6719ebd700 1 --2- 172.21.15.78:0/2698158464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67140ad290 0x7f6714071a80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6700009940 tx=0x7f6700009c50 comp rx=0 tx=0).stop 2023-10-11T09:40:37.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.178+0000 7f6719ebd700 1 -- 172.21.15.78:0/2698158464 shutdown_connections 2023-10-11T09:40:37.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.178+0000 7f6719ebd700 1 --2- 172.21.15.78:0/2698158464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67140ad290 0x7f6714071a80 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:37.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.178+0000 7f6719ebd700 1 --2- 172.21.15.78:0/2698158464 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f671415c900 0x7f67140b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:37.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.178+0000 7f6719ebd700 1 -- 172.21.15.78:0/2698158464 >> 172.21.15.78:0/2698158464 conn(0x7f671406a060 msgr2=0x7f671406a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:37.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.178+0000 7f6719ebd700 1 -- 172.21.15.78:0/2698158464 shutdown_connections 2023-10-11T09:40:37.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.178+0000 7f6719ebd700 1 -- 172.21.15.78:0/2698158464 wait complete. 2023-10-11T09:40:37.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.182+0000 7f6719ebd700 1 Processor -- start 2023-10-11T09:40:37.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.182+0000 7f6719ebd700 1 -- start start 2023-10-11T09:40:37.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.182+0000 7f6719ebd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67140ad290 0x7f671415d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:37.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.182+0000 7f6719ebd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67140ce7f0 0x7f6714151d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:37.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.182+0000 7f6719ebd700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f671415c900 0x7f6714152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:37.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.182+0000 7f6719ebd700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671415f9e0 con 0x7f67140ad290 2023-10-11T09:40:37.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.182+0000 7f6719ebd700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67141541e0 con 0x7f67140ce7f0 2023-10-11T09:40:37.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.182+0000 7f6719ebd700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f671415e270 con 0x7f671415c900 2023-10-11T09:40:37.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.182+0000 7f6718ebb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67140ad290 0x7f671415d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:37.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.182+0000 7f6718ebb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67140ad290 0x7f671415d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58064/0 (socket says 172.21.15.78:58064) 2023-10-11T09:40:37.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.182+0000 7f6718ebb700 1 -- 172.21.15.78:0/2126747610 learned_addr learned my addr 172.21.15.78:0/2126747610 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:37.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.182+0000 7f67196bc700 1 --2- 172.21.15.78:0/2126747610 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f671415c900 0x7f6714152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:37.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.182+0000 7f6713fff700 1 --2- 172.21.15.78:0/2126747610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67140ce7f0 0x7f6714151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:37.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.182+0000 7f6718ebb700 1 -- 172.21.15.78:0/2126747610 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f671415c900 msgr2=0x7f6714152260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:37.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.182+0000 7f6718ebb700 1 --2- 172.21.15.78:0/2126747610 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f671415c900 0x7f6714152260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:37.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.182+0000 7f6718ebb700 1 -- 172.21.15.78:0/2126747610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67140ce7f0 msgr2=0x7f6714151d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:37.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.182+0000 7f6718ebb700 1 --2- 172.21.15.78:0/2126747610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67140ce7f0 0x7f6714151d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:37.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.182+0000 7f6718ebb700 1 -- 172.21.15.78:0/2126747610 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6700009570 con 0x7f67140ad290 2023-10-11T09:40:37.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.182+0000 7f67196bc700 1 --2- 172.21.15.78:0/2126747610 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f671415c900 0x7f6714152260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:40:37.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.182+0000 7f6718ebb700 1 --2- 172.21.15.78:0/2126747610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67140ad290 0x7f671415d1a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f670800ea00 tx=0x7f670800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:37.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.182+0000 7f67117fa700 1 -- 172.21.15.78:0/2126747610 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6708009720 con 0x7f67140ad290 2023-10-11T09:40:37.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.182+0000 7f67117fa700 1 -- 172.21.15.78:0/2126747610 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f670800c850 con 0x7f67140ad290 2023-10-11T09:40:37.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.182+0000 7f6719ebd700 1 -- 172.21.15.78:0/2126747610 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67141544a0 con 0x7f67140ad290 2023-10-11T09:40:37.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.182+0000 7f67117fa700 1 -- 172.21.15.78:0/2126747610 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670800cbc0 con 0x7f67140ad290 2023-10-11T09:40:37.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.186+0000 7f6719ebd700 1 -- 172.21.15.78:0/2126747610 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f671415dcb0 con 0x7f67140ad290 2023-10-11T09:40:37.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.186+0000 7f6719ebd700 1 -- 172.21.15.78:0/2126747610 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6714065d40 con 0x7f67140ad290 2023-10-11T09:40:37.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.186+0000 7f67117fa700 1 -- 172.21.15.78:0/2126747610 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f670800cd20 con 0x7f67140ad290 2023-10-11T09:40:37.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.190+0000 7f67117fa700 1 --2- 172.21.15.78:0/2126747610 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f66f8065ed0 0x7f66f8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:37.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.190+0000 7f67117fa700 1 -- 172.21.15.78:0/2126747610 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f67080835c0 con 0x7f67140ad290 2023-10-11T09:40:37.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.190+0000 7f6713fff700 1 --2- 172.21.15.78:0/2126747610 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f66f8065ed0 0x7f66f8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:37.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.190+0000 7f6713fff700 1 --2- 172.21.15.78:0/2126747610 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f66f8065ed0 0x7f66f8068380 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6700014040 tx=0x7f6700008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:37.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.194+0000 7f67117fa700 1 -- 172.21.15.78:0/2126747610 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f6708050f50 con 0x7f67140ad290 2023-10-11T09:40:37.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.370+0000 7f6719ebd700 1 -- 172.21.15.78:0/2126747610 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f671415cf80 con 0x7f67140ad290 2023-10-11T09:40:37.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.370+0000 7f67117fa700 1 -- 172.21.15.78:0/2126747610 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6708054570 con 0x7f67140ad290 2023-10-11T09:40:37.375 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:40:37.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.374+0000 7f6719ebd700 1 -- 172.21.15.78:0/2126747610 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f66f8065ed0 msgr2=0x7f66f8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:37.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.374+0000 7f6719ebd700 1 --2- 172.21.15.78:0/2126747610 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f66f8065ed0 0x7f66f8068380 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6700014040 tx=0x7f6700008030 comp rx=0 tx=0).stop 2023-10-11T09:40:37.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.374+0000 7f6719ebd700 1 -- 172.21.15.78:0/2126747610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67140ad290 msgr2=0x7f671415d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:37.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.374+0000 7f6719ebd700 1 --2- 172.21.15.78:0/2126747610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67140ad290 0x7f671415d1a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f670800ea00 tx=0x7f670800edc0 comp rx=0 tx=0).stop 2023-10-11T09:40:37.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.374+0000 7f6719ebd700 1 -- 172.21.15.78:0/2126747610 shutdown_connections 2023-10-11T09:40:37.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.374+0000 7f6719ebd700 1 --2- 172.21.15.78:0/2126747610 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f66f8065ed0 0x7f66f8068380 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:37.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.374+0000 7f6719ebd700 1 --2- 172.21.15.78:0/2126747610 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f671415c900 0x7f6714152260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:37.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.374+0000 7f6719ebd700 1 --2- 172.21.15.78:0/2126747610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67140ce7f0 0x7f6714151d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:37.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.374+0000 7f6719ebd700 1 --2- 172.21.15.78:0/2126747610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67140ad290 0x7f671415d1a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:37.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.374+0000 7f6719ebd700 1 -- 172.21.15.78:0/2126747610 >> 172.21.15.78:0/2126747610 conn(0x7f671406a060 msgr2=0x7f671406ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:37.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.374+0000 7f6719ebd700 1 -- 172.21.15.78:0/2126747610 shutdown_connections 2023-10-11T09:40:37.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.374+0000 7f6719ebd700 1 -- 172.21.15.78:0/2126747610 wait complete. 2023-10-11T09:40:37.402 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:40:37.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.566+0000 7fe1febd7700 1 Processor -- start 2023-10-11T09:40:37.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.566+0000 7fe1febd7700 1 -- start start 2023-10-11T09:40:37.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.570+0000 7fe1febd7700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe1f80ce500 0x7fe1f816bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:37.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.570+0000 7fe1febd7700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1f806ffa0 con 0x7fe1f8071040 2023-10-11T09:40:37.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.570+0000 7fe1febd7700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1f8070280 con 0x7fe1f8160d40 2023-10-11T09:40:37.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.570+0000 7fe1febd7700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1f8070c20 con 0x7fe1f80ce500 2023-10-11T09:40:37.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.570+0000 7fe1f7fff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fe1f8071040 0x7fe1f80cddf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:51540/0 (socket says 172.21.15.78:51540) 2023-10-11T09:40:37.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.570+0000 7fe1f7fff700 1 -- 172.21.15.78:0/3062699827 learned_addr learned my addr 172.21.15.78:0/3062699827 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:37.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.570+0000 7fe1fd174700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe1f80ce500 0x7fe1f816bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:37.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.570+0000 7fe1f77fe700 1 -- 172.21.15.78:0/3062699827 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2615378934 0 0) 0x7fe1f806ffa0 con 0x7fe1f8071040 2023-10-11T09:40:37.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.570+0000 7fe1f77fe700 1 -- 172.21.15.78:0/3062699827 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1e80035d0 con 0x7fe1f8071040 2023-10-11T09:40:37.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.570+0000 7fe1fd174700 1 -- 172.21.15.78:0/3062699827 >> v1:172.21.15.133:6789/0 conn(0x7fe1f8160d40 legacy=0x7fe1f80cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:37.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.570+0000 7fe1fd174700 1 -- 172.21.15.78:0/3062699827 >> v1:172.21.15.78:6789/0 conn(0x7fe1f8071040 legacy=0x7fe1f80cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:37.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.570+0000 7fe1fd174700 1 -- 172.21.15.78:0/3062699827 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f8070560 con 0x7fe1f80ce500 2023-10-11T09:40:37.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.570+0000 7fe1fd174700 1 --2- 172.21.15.78:0/3062699827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1f80ce500 0x7fe1f816bb80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0009d90 tx=0x7fe1f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e79444a44fff8315 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:37.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.570+0000 7fe1f77fe700 1 -- 172.21.15.78:0/3062699827 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f000e070 con 0x7fe1f80ce500 2023-10-11T09:40:37.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.570+0000 7fe1f77fe700 1 -- 172.21.15.78:0/3062699827 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f000bb20 con 0x7fe1f80ce500 2023-10-11T09:40:37.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.570+0000 7fe1f77fe700 1 -- 172.21.15.78:0/3062699827 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f000be90 con 0x7fe1f80ce500 2023-10-11T09:40:37.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.570+0000 7fe1febd7700 1 -- 172.21.15.78:0/3062699827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1f80ce500 msgr2=0x7fe1f816bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:37.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.570+0000 7fe1febd7700 1 --2- 172.21.15.78:0/3062699827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1f80ce500 0x7fe1f816bb80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0009d90 tx=0x7fe1f000b330 comp rx=0 tx=0).stop 2023-10-11T09:40:37.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.570+0000 7fe1febd7700 1 -- 172.21.15.78:0/3062699827 shutdown_connections 2023-10-11T09:40:37.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.570+0000 7fe1febd7700 1 --2- 172.21.15.78:0/3062699827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1f80ce500 0x7fe1f816bb80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:37.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.570+0000 7fe1febd7700 1 -- 172.21.15.78:0/3062699827 >> 172.21.15.78:0/3062699827 conn(0x7fe1f806aca0 msgr2=0x7fe1f80b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:37.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.570+0000 7fe1febd7700 1 -- 172.21.15.78:0/3062699827 shutdown_connections 2023-10-11T09:40:37.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.570+0000 7fe1febd7700 1 -- 172.21.15.78:0/3062699827 wait complete. 2023-10-11T09:40:37.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.574+0000 7fe1febd7700 1 Processor -- start 2023-10-11T09:40:37.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.574+0000 7fe1febd7700 1 -- start start 2023-10-11T09:40:37.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.574+0000 7fe1febd7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1f8071040 0x7fe1f81f75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:37.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.574+0000 7fe1febd7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1f80ce500 0x7fe1f80ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:37.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.574+0000 7fe1febd7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1f8160d40 0x7fe1f80cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:37.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.574+0000 7fe1febd7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f80c6900 con 0x7fe1f80ce500 2023-10-11T09:40:37.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.574+0000 7fe1febd7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f81f7b10 con 0x7fe1f8160d40 2023-10-11T09:40:37.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.574+0000 7fe1febd7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f80c5cf0 con 0x7fe1f8071040 2023-10-11T09:40:37.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.578+0000 7fe1f7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1f80ce500 0x7fe1f80ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:37.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.578+0000 7fe1f7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1f80ce500 0x7fe1f80ceca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58082/0 (socket says 172.21.15.78:58082) 2023-10-11T09:40:37.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.578+0000 7fe1f7fff700 1 -- 172.21.15.78:0/2122858498 learned_addr learned my addr 172.21.15.78:0/2122858498 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:37.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.578+0000 7fe1fd174700 1 --2- 172.21.15.78:0/2122858498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1f8160d40 0x7fe1f80cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:37.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.578+0000 7fe1fc973700 1 --2- 172.21.15.78:0/2122858498 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1f8071040 0x7fe1f81f75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:37.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.578+0000 7fe1f7fff700 1 -- 172.21.15.78:0/2122858498 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1f8071040 msgr2=0x7fe1f81f75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:37.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.578+0000 7fe1f7fff700 1 --2- 172.21.15.78:0/2122858498 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1f8071040 0x7fe1f81f75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:37.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.578+0000 7fe1f7fff700 1 -- 172.21.15.78:0/2122858498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1f8160d40 msgr2=0x7fe1f80cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:37.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.578+0000 7fe1f7fff700 1 --2- 172.21.15.78:0/2122858498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1f8160d40 0x7fe1f80cf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:37.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.578+0000 7fe1f7fff700 1 -- 172.21.15.78:0/2122858498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f0009a70 con 0x7fe1f80ce500 2023-10-11T09:40:37.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.578+0000 7fe1f7fff700 1 --2- 172.21.15.78:0/2122858498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1f80ce500 0x7fe1f80ceca0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe1e400c4d0 tx=0x7fe1e400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:37.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.578+0000 7fe1f57fa700 1 -- 172.21.15.78:0/2122858498 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4005d90 con 0x7fe1f80ce500 2023-10-11T09:40:37.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.578+0000 7fe1f57fa700 1 -- 172.21.15.78:0/2122858498 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e40070d0 con 0x7fe1f80ce500 2023-10-11T09:40:37.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.578+0000 7fe1febd7700 1 -- 172.21.15.78:0/2122858498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1f80c5fd0 con 0x7fe1f80ce500 2023-10-11T09:40:37.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.578+0000 7fe1f57fa700 1 -- 172.21.15.78:0/2122858498 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4007480 con 0x7fe1f80ce500 2023-10-11T09:40:37.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.578+0000 7fe1febd7700 1 -- 172.21.15.78:0/2122858498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1f80c6590 con 0x7fe1f80ce500 2023-10-11T09:40:37.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.578+0000 7fe1febd7700 1 -- 172.21.15.78:0/2122858498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1f80673e0 con 0x7fe1f80ce500 2023-10-11T09:40:37.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.582+0000 7fe1f57fa700 1 -- 172.21.15.78:0/2122858498 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fe1e40075e0 con 0x7fe1f80ce500 2023-10-11T09:40:37.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.582+0000 7fe1f57fa700 1 --2- 172.21.15.78:0/2122858498 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe1dc065ed0 0x7fe1dc068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:37.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.582+0000 7fe1f57fa700 1 -- 172.21.15.78:0/2122858498 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe1e4082940 con 0x7fe1f80ce500 2023-10-11T09:40:37.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.582+0000 7fe1fc973700 1 --2- 172.21.15.78:0/2122858498 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe1dc065ed0 0x7fe1dc068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:37.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.586+0000 7fe1fc973700 1 --2- 172.21.15.78:0/2122858498 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe1dc065ed0 0x7fe1dc068380 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec0016e0 tx=0x7fe1ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:37.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.590+0000 7fe1f57fa700 1 -- 172.21.15.78:0/2122858498 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fe1e4050370 con 0x7fe1f80ce500 2023-10-11T09:40:37.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.798+0000 7fe1febd7700 1 -- 172.21.15.78:0/2122858498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1f8072250 con 0x7fe1f80ce500 2023-10-11T09:40:37.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.802+0000 7fe1f57fa700 1 -- 172.21.15.78:0/2122858498 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3530 (secure 0 0 0) 0x7fe1e4017de0 con 0x7fe1f80ce500 2023-10-11T09:40:37.806 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:40:37.806 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/4282378510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4282378510},{"type":"v1","addr":"172.21.15.133:6835","nonce":4282378510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3986631249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3986631249},{"type":"v1","addr":"172.21.15.78:6835","nonce":3986631249}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3135421902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3135421902},{"type":"v1","addr":"172.21.15.78:6837","nonce":3135421902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4306,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/3925157967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":3925157967},{"type":"v1","addr":"172.21.15.133:6834","nonce":3925157967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-10-11T09:40:37.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.802+0000 7fe1febd7700 1 -- 172.21.15.78:0/2122858498 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe1dc065ed0 msgr2=0x7fe1dc068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:37.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.802+0000 7fe1febd7700 1 --2- 172.21.15.78:0/2122858498 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe1dc065ed0 0x7fe1dc068380 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec0016e0 tx=0x7fe1ec009040 comp rx=0 tx=0).stop 2023-10-11T09:40:37.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.802+0000 7fe1febd7700 1 -- 172.21.15.78:0/2122858498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1f80ce500 msgr2=0x7fe1f80ceca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:37.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.802+0000 7fe1febd7700 1 --2- 172.21.15.78:0/2122858498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1f80ce500 0x7fe1f80ceca0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe1e400c4d0 tx=0x7fe1e400c890 comp rx=0 tx=0).stop 2023-10-11T09:40:37.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.802+0000 7fe1febd7700 1 -- 172.21.15.78:0/2122858498 shutdown_connections 2023-10-11T09:40:37.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.802+0000 7fe1febd7700 1 --2- 172.21.15.78:0/2122858498 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe1dc065ed0 0x7fe1dc068380 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:37.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.802+0000 7fe1febd7700 1 --2- 172.21.15.78:0/2122858498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1f8160d40 0x7fe1f80cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:37.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.802+0000 7fe1febd7700 1 --2- 172.21.15.78:0/2122858498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1f80ce500 0x7fe1f80ceca0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:37.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.802+0000 7fe1febd7700 1 --2- 172.21.15.78:0/2122858498 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1f8071040 0x7fe1f81f75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:37.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.802+0000 7fe1febd7700 1 -- 172.21.15.78:0/2122858498 >> 172.21.15.78:0/2122858498 conn(0x7fe1f806aca0 msgr2=0x7fe1f80cb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:37.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.802+0000 7fe1febd7700 1 -- 172.21.15.78:0/2122858498 shutdown_connections 2023-10-11T09:40:37.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.802+0000 7fe1febd7700 1 -- 172.21.15.78:0/2122858498 wait complete. 2023-10-11T09:40:37.811 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 3 2023-10-11T09:40:37.830 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:40:38.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.994+0000 7ff7d9957700 1 Processor -- start 2023-10-11T09:40:38.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.998+0000 7ff7d9957700 1 -- start start 2023-10-11T09:40:38.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.998+0000 7ff7d9957700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff7d413f740 0x7ff7d40ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:38.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.998+0000 7ff7d9957700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ff7d406d180 0x7ff7d40b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:38.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.998+0000 7ff7d9957700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7d4070c80 con 0x7ff7d406cd00 2023-10-11T09:40:38.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.998+0000 7ff7d9957700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7d40ce4c0 con 0x7ff7d406d180 2023-10-11T09:40:38.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:37.998+0000 7ff7d9957700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7d406d600 con 0x7ff7d413f740 2023-10-11T09:40:38.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.002+0000 7ff7d37fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ff7d406d180 0x7ff7d40b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:38.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.002+0000 7ff7d37fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ff7d406d180 0x7ff7d40b0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58088/0 (socket says 172.21.15.78:58088) 2023-10-11T09:40:38.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.002+0000 7ff7d37fe700 1 -- 172.21.15.78:0/2734412580 learned_addr learned my addr 172.21.15.78:0/2734412580 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:38.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.002+0000 7ff7d2ffd700 1 --2- 172.21.15.78:0/2734412580 >> v2:172.21.15.133:3300/0 conn(0x7ff7d413f740 0x7ff7d40ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:38.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.002+0000 7ff7d37fe700 1 -- 172.21.15.78:0/2734412580 >> v2:172.21.15.133:3300/0 conn(0x7ff7d413f740 msgr2=0x7ff7d40ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:38.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.002+0000 7ff7d37fe700 1 --2- 172.21.15.78:0/2734412580 >> v2:172.21.15.133:3300/0 conn(0x7ff7d413f740 0x7ff7d40ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:38.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.002+0000 7ff7d37fe700 1 -- 172.21.15.78:0/2734412580 >> v1:172.21.15.78:6789/0 conn(0x7ff7d406cd00 legacy=0x7ff7d40b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:38.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.002+0000 7ff7d37fe700 1 -- 172.21.15.78:0/2734412580 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d40cd420 con 0x7ff7d406d180 2023-10-11T09:40:38.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.002+0000 7ff7d37fe700 1 --2- 172.21.15.78:0/2734412580 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7d406d180 0x7ff7d40b0db0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff7d406fb70 tx=0x7ff7cc00d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d170240aece4948 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:38.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.002+0000 7ff7d27fc700 1 -- 172.21.15.78:0/2734412580 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc010070 con 0x7ff7d406d180 2023-10-11T09:40:38.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.002+0000 7ff7d27fc700 1 -- 172.21.15.78:0/2734412580 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7cc00dd90 con 0x7ff7d406d180 2023-10-11T09:40:38.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.002+0000 7ff7d27fc700 1 -- 172.21.15.78:0/2734412580 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc00b310 con 0x7ff7d406d180 2023-10-11T09:40:38.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.002+0000 7ff7d9957700 1 -- 172.21.15.78:0/2734412580 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7d406d180 msgr2=0x7ff7d40b0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:38.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.002+0000 7ff7d9957700 1 --2- 172.21.15.78:0/2734412580 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7d406d180 0x7ff7d40b0db0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff7d406fb70 tx=0x7ff7cc00d430 comp rx=0 tx=0).stop 2023-10-11T09:40:38.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.002+0000 7ff7d9957700 1 -- 172.21.15.78:0/2734412580 shutdown_connections 2023-10-11T09:40:38.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.002+0000 7ff7d9957700 1 --2- 172.21.15.78:0/2734412580 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7d406d180 0x7ff7d40b0db0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:38.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.002+0000 7ff7d9957700 1 --2- 172.21.15.78:0/2734412580 >> v2:172.21.15.133:3300/0 conn(0x7ff7d413f740 0x7ff7d40ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:38.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.002+0000 7ff7d9957700 1 -- 172.21.15.78:0/2734412580 >> 172.21.15.78:0/2734412580 conn(0x7ff7d40cea60 msgr2=0x7ff7d40cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:38.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.002+0000 7ff7d9957700 1 -- 172.21.15.78:0/2734412580 shutdown_connections 2023-10-11T09:40:38.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.002+0000 7ff7d9957700 1 -- 172.21.15.78:0/2734412580 wait complete. 2023-10-11T09:40:38.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.006+0000 7ff7d9957700 1 Processor -- start 2023-10-11T09:40:38.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.006+0000 7ff7d9957700 1 -- start start 2023-10-11T09:40:38.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.006+0000 7ff7d9957700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7d406cd00 0x7ff7d40b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:38.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.006+0000 7ff7d9957700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7d406d180 0x7ff7d40c5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:38.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.006+0000 7ff7d9957700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff7d413f740 0x7ff7d40c62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:38.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.006+0000 7ff7d9957700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d40b4130 con 0x7ff7d406cd00 2023-10-11T09:40:38.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.006+0000 7ff7d9957700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d40b3520 con 0x7ff7d406d180 2023-10-11T09:40:38.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.006+0000 7ff7d9957700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d40b3800 con 0x7ff7d413f740 2023-10-11T09:40:38.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.006+0000 7ff7d2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7d406cd00 0x7ff7d40b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:38.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.010+0000 7ff7d2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7d406cd00 0x7ff7d40b2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58092/0 (socket says 172.21.15.78:58092) 2023-10-11T09:40:38.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.010+0000 7ff7d2ffd700 1 -- 172.21.15.78:0/2177207897 learned_addr learned my addr 172.21.15.78:0/2177207897 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:38.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.010+0000 7ff7ca7fc700 1 --2- 172.21.15.78:0/2177207897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7d406d180 0x7ff7d40c5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:38.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.010+0000 7ff7d2ffd700 1 -- 172.21.15.78:0/2177207897 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff7d413f740 msgr2=0x7ff7d40c62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:38.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.010+0000 7ff7d37fe700 1 --2- 172.21.15.78:0/2177207897 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff7d413f740 0x7ff7d40c62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:38.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.010+0000 7ff7d2ffd700 1 --2- 172.21.15.78:0/2177207897 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff7d413f740 0x7ff7d40c62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:38.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.010+0000 7ff7d2ffd700 1 -- 172.21.15.78:0/2177207897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7d406d180 msgr2=0x7ff7d40c5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:38.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.010+0000 7ff7d2ffd700 1 --2- 172.21.15.78:0/2177207897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7d406d180 0x7ff7d40c5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:38.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.010+0000 7ff7d2ffd700 1 -- 172.21.15.78:0/2177207897 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7cc00bd40 con 0x7ff7d406cd00 2023-10-11T09:40:38.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.010+0000 7ff7ca7fc700 1 --2- 172.21.15.78:0/2177207897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7d406d180 0x7ff7d40c5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:40:38.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.010+0000 7ff7d37fe700 1 --2- 172.21.15.78:0/2177207897 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff7d413f740 0x7ff7d40c62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:40:38.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.010+0000 7ff7d2ffd700 1 --2- 172.21.15.78:0/2177207897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7d406cd00 0x7ff7d40b2a60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff7c400d800 tx=0x7ff7c400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:38.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.010+0000 7ff7d0ff9700 1 -- 172.21.15.78:0/2177207897 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c40125f0 con 0x7ff7d406cd00 2023-10-11T09:40:38.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.010+0000 7ff7d0ff9700 1 -- 172.21.15.78:0/2177207897 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c4012750 con 0x7ff7d406cd00 2023-10-11T09:40:38.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.010+0000 7ff7d0ff9700 1 -- 172.21.15.78:0/2177207897 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c4012b00 con 0x7ff7d406cd00 2023-10-11T09:40:38.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.010+0000 7ff7d9957700 1 -- 172.21.15.78:0/2177207897 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d40b3ae0 con 0x7ff7d406cd00 2023-10-11T09:40:38.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.010+0000 7ff7d9957700 1 -- 172.21.15.78:0/2177207897 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d4141710 con 0x7ff7d406cd00 2023-10-11T09:40:38.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.010+0000 7ff7d9957700 1 -- 172.21.15.78:0/2177207897 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7d4065d40 con 0x7ff7d406cd00 2023-10-11T09:40:38.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.010+0000 7ff7d0ff9700 1 -- 172.21.15.78:0/2177207897 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7ff7c4012c60 con 0x7ff7d406cd00 2023-10-11T09:40:38.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.014+0000 7ff7d0ff9700 1 --2- 172.21.15.78:0/2177207897 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff7b4065ed0 0x7ff7b4068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:38.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.014+0000 7ff7d0ff9700 1 -- 172.21.15.78:0/2177207897 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff7c4083850 con 0x7ff7d406cd00 2023-10-11T09:40:38.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.014+0000 7ff7ca7fc700 1 --2- 172.21.15.78:0/2177207897 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff7b4065ed0 0x7ff7b4068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:38.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.014+0000 7ff7ca7fc700 1 --2- 172.21.15.78:0/2177207897 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff7b4065ed0 0x7ff7b4068380 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff7d40b4cb0 tx=0x7ff7bc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:38.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.018+0000 7ff7d0ff9700 1 -- 172.21.15.78:0/2177207897 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7ff7c40511f0 con 0x7ff7d406cd00 2023-10-11T09:40:38.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.194+0000 7ff7d9957700 1 -- 172.21.15.78:0/2177207897 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7d40c5b70 con 0x7ff7d406cd00 2023-10-11T09:40:38.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.194+0000 7ff7d0ff9700 1 -- 172.21.15.78:0/2177207897 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+14168 (secure 0 0 0) 0x7ff7c4054810 con 0x7ff7d406cd00 2023-10-11T09:40:38.200 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:40:38.201 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":19,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:40:37.145567+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:40:38.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.198+0000 7ff7d9957700 1 -- 172.21.15.78:0/2177207897 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff7b4065ed0 msgr2=0x7ff7b4068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:38.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.198+0000 7ff7d9957700 1 --2- 172.21.15.78:0/2177207897 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff7b4065ed0 0x7ff7b4068380 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff7d40b4cb0 tx=0x7ff7bc009040 comp rx=0 tx=0).stop 2023-10-11T09:40:38.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.198+0000 7ff7d9957700 1 -- 172.21.15.78:0/2177207897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7d406cd00 msgr2=0x7ff7d40b2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:38.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.198+0000 7ff7d9957700 1 --2- 172.21.15.78:0/2177207897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7d406cd00 0x7ff7d40b2a60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff7c400d800 tx=0x7ff7c400dbc0 comp rx=0 tx=0).stop 2023-10-11T09:40:38.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.198+0000 7ff7d9957700 1 -- 172.21.15.78:0/2177207897 shutdown_connections 2023-10-11T09:40:38.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.198+0000 7ff7d9957700 1 --2- 172.21.15.78:0/2177207897 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff7b4065ed0 0x7ff7b4068380 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:38.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.198+0000 7ff7d9957700 1 --2- 172.21.15.78:0/2177207897 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff7d413f740 0x7ff7d40c62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:38.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.198+0000 7ff7d9957700 1 --2- 172.21.15.78:0/2177207897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7d406d180 0x7ff7d40c5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:38.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.198+0000 7ff7d9957700 1 --2- 172.21.15.78:0/2177207897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7d406cd00 0x7ff7d40b2a60 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:38.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.198+0000 7ff7d9957700 1 -- 172.21.15.78:0/2177207897 >> 172.21.15.78:0/2177207897 conn(0x7ff7d40cea60 msgr2=0x7ff7d4070ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:38.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.198+0000 7ff7d9957700 1 -- 172.21.15.78:0/2177207897 shutdown_connections 2023-10-11T09:40:38.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.198+0000 7ff7d9957700 1 -- 172.21.15.78:0/2177207897 wait complete. 2023-10-11T09:40:38.223 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:40:38.224 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:40:38.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.394+0000 7f78654c6700 1 Processor -- start 2023-10-11T09:40:38.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.394+0000 7f78654c6700 1 -- start start 2023-10-11T09:40:38.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.398+0000 7f78654c6700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f78600b84c0 0x7f78600cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:38.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.398+0000 7f78654c6700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f7860160a80 0x7f78600d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:38.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.398+0000 7f78654c6700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78600cd690 con 0x7f78600b8890 2023-10-11T09:40:38.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.398+0000 7f78654c6700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78600cdea0 con 0x7f7860160a80 2023-10-11T09:40:38.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.398+0000 7f78654c6700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78600d0c10 con 0x7f78600b84c0 2023-10-11T09:40:38.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.398+0000 7f785f7fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f7860160a80 0x7f78600d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:38.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.398+0000 7f785effd700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f78600b84c0 0x7f78600cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:38.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.398+0000 7f785f7fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f7860160a80 0x7f78600d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58104/0 (socket says 172.21.15.78:58104) 2023-10-11T09:40:38.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.398+0000 7f785f7fe700 1 -- 172.21.15.78:0/1230916075 learned_addr learned my addr 172.21.15.78:0/1230916075 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:38.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.398+0000 7f785f7fe700 1 -- 172.21.15.78:0/1230916075 >> v2:172.21.15.133:3300/0 conn(0x7f78600b84c0 msgr2=0x7f78600cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:38.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.398+0000 7f785f7fe700 1 --2- 172.21.15.78:0/1230916075 >> v2:172.21.15.133:3300/0 conn(0x7f78600b84c0 0x7f78600cfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:38.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.398+0000 7f785f7fe700 1 -- 172.21.15.78:0/1230916075 >> v1:172.21.15.133:6789/0 conn(0x7f78600b8890 legacy=0x7f7860160370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:38.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.398+0000 7f785f7fe700 1 -- 172.21.15.78:0/1230916075 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78600d02e0 con 0x7f7860160a80 2023-10-11T09:40:38.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.402+0000 7f785f7fe700 1 --2- 172.21.15.78:0/1230916075 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7860160a80 0x7f78600d1340 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f78600675c0 tx=0x7f785800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf6b2e0d4d8f9da7 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:38.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.402+0000 7f785dffb700 1 -- 172.21.15.78:0/1230916075 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785800e070 con 0x7f7860160a80 2023-10-11T09:40:38.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.402+0000 7f785dffb700 1 -- 172.21.15.78:0/1230916075 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7858003a40 con 0x7f7860160a80 2023-10-11T09:40:38.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.402+0000 7f785dffb700 1 -- 172.21.15.78:0/1230916075 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7858003df0 con 0x7f7860160a80 2023-10-11T09:40:38.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.402+0000 7f78654c6700 1 -- 172.21.15.78:0/1230916075 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7860160a80 msgr2=0x7f78600d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:38.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.402+0000 7f78654c6700 1 --2- 172.21.15.78:0/1230916075 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7860160a80 0x7f78600d1340 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f78600675c0 tx=0x7f785800b330 comp rx=0 tx=0).stop 2023-10-11T09:40:38.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.402+0000 7f78654c6700 1 -- 172.21.15.78:0/1230916075 shutdown_connections 2023-10-11T09:40:38.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.402+0000 7f78654c6700 1 --2- 172.21.15.78:0/1230916075 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7860160a80 0x7f78600d1340 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:38.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.402+0000 7f78654c6700 1 --2- 172.21.15.78:0/1230916075 >> v2:172.21.15.133:3300/0 conn(0x7f78600b84c0 0x7f78600cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:38.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.402+0000 7f78654c6700 1 -- 172.21.15.78:0/1230916075 >> 172.21.15.78:0/1230916075 conn(0x7f786006cb50 msgr2=0x7f786006cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:38.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.402+0000 7f78654c6700 1 -- 172.21.15.78:0/1230916075 shutdown_connections 2023-10-11T09:40:38.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.402+0000 7f78654c6700 1 -- 172.21.15.78:0/1230916075 wait complete. 2023-10-11T09:40:38.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.406+0000 7f78654c6700 1 Processor -- start 2023-10-11T09:40:38.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.406+0000 7f78654c6700 1 -- start start 2023-10-11T09:40:38.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.406+0000 7f78654c6700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f78600b84c0 0x7f78600c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:38.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.406+0000 7f78654c6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78600b8890 0x7f78600cc800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:38.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.406+0000 7f78654c6700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7860160a80 0x7f78600ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:38.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.406+0000 7f78654c6700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78600c9020 con 0x7f78600b8890 2023-10-11T09:40:38.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.406+0000 7f78654c6700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78600ccd40 con 0x7f7860160a80 2023-10-11T09:40:38.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.406+0000 7f78654c6700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78600c7d20 con 0x7f78600b84c0 2023-10-11T09:40:38.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.406+0000 7f785e7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78600b8890 0x7f78600cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:38.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.406+0000 7f785e7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78600b8890 0x7f78600cc800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58116/0 (socket says 172.21.15.78:58116) 2023-10-11T09:40:38.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.406+0000 7f785e7fc700 1 -- 172.21.15.78:0/3365916632 learned_addr learned my addr 172.21.15.78:0/3365916632 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:38.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.406+0000 7f785e7fc700 1 -- 172.21.15.78:0/3365916632 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f78600b84c0 msgr2=0x7f78600c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:38.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.406+0000 7f785e7fc700 1 --2- 172.21.15.78:0/3365916632 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f78600b84c0 0x7f78600c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:38.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.406+0000 7f785e7fc700 1 -- 172.21.15.78:0/3365916632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7860160a80 msgr2=0x7f78600ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:38.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.406+0000 7f785e7fc700 1 --2- 172.21.15.78:0/3365916632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7860160a80 0x7f78600ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:38.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.406+0000 7f785e7fc700 1 -- 172.21.15.78:0/3365916632 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7858009b30 con 0x7f78600b8890 2023-10-11T09:40:38.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.406+0000 7f785e7fc700 1 --2- 172.21.15.78:0/3365916632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78600b8890 0x7f78600cc800 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f785400c4e0 tx=0x7f785400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:38.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.406+0000 7f78477fe700 1 -- 172.21.15.78:0/3365916632 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7854011790 con 0x7f78600b8890 2023-10-11T09:40:38.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.406+0000 7f78477fe700 1 -- 172.21.15.78:0/3365916632 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7854011910 con 0x7f78600b8890 2023-10-11T09:40:38.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.406+0000 7f78477fe700 1 -- 172.21.15.78:0/3365916632 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7854011cc0 con 0x7f78600b8890 2023-10-11T09:40:38.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.406+0000 7f78654c6700 1 -- 172.21.15.78:0/3365916632 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78600c8000 con 0x7f78600b8890 2023-10-11T09:40:38.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.406+0000 7f78654c6700 1 -- 172.21.15.78:0/3365916632 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78600c85c0 con 0x7f78600b8890 2023-10-11T09:40:38.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.410+0000 7f78654c6700 1 -- 172.21.15.78:0/3365916632 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78600c87e0 con 0x7f78600b8890 2023-10-11T09:40:38.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.410+0000 7f78477fe700 1 -- 172.21.15.78:0/3365916632 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f785401c460 con 0x7f78600b8890 2023-10-11T09:40:38.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.410+0000 7f78477fe700 1 --2- 172.21.15.78:0/3365916632 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f784c065ed0 0x7f784c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:38.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.410+0000 7f78477fe700 1 -- 172.21.15.78:0/3365916632 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7854083330 con 0x7f78600b8890 2023-10-11T09:40:38.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.414+0000 7f785effd700 1 --2- 172.21.15.78:0/3365916632 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f784c065ed0 0x7f784c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:38.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.414+0000 7f785effd700 1 --2- 172.21.15.78:0/3365916632 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f784c065ed0 0x7f784c068380 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7850006b40 tx=0x7f7850006ad0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:38.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.418+0000 7f78477fe700 1 -- 172.21.15.78:0/3365916632 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f7854050cc0 con 0x7f78600b8890 2023-10-11T09:40:38.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.594+0000 7f78654c6700 1 -- 172.21.15.78:0/3365916632 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f786008bff0 con 0x7f78600b8890 2023-10-11T09:40:38.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.594+0000 7f78477fe700 1 -- 172.21.15.78:0/3365916632 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7f78540542e0 con 0x7f78600b8890 2023-10-11T09:40:38.599 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:40:38.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.598+0000 7f78654c6700 1 -- 172.21.15.78:0/3365916632 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f784c065ed0 msgr2=0x7f784c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:38.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.598+0000 7f78654c6700 1 --2- 172.21.15.78:0/3365916632 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f784c065ed0 0x7f784c068380 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7850006b40 tx=0x7f7850006ad0 comp rx=0 tx=0).stop 2023-10-11T09:40:38.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.598+0000 7f78654c6700 1 -- 172.21.15.78:0/3365916632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78600b8890 msgr2=0x7f78600cc800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:38.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.598+0000 7f78654c6700 1 --2- 172.21.15.78:0/3365916632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78600b8890 0x7f78600cc800 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f785400c4e0 tx=0x7f785400c8a0 comp rx=0 tx=0).stop 2023-10-11T09:40:38.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.598+0000 7f78654c6700 1 -- 172.21.15.78:0/3365916632 shutdown_connections 2023-10-11T09:40:38.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.598+0000 7f78654c6700 1 --2- 172.21.15.78:0/3365916632 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f784c065ed0 0x7f784c068380 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:38.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.598+0000 7f78654c6700 1 --2- 172.21.15.78:0/3365916632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7860160a80 0x7f78600ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:38.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.598+0000 7f78654c6700 1 --2- 172.21.15.78:0/3365916632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78600b8890 0x7f78600cc800 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:38.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.598+0000 7f78654c6700 1 --2- 172.21.15.78:0/3365916632 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f78600b84c0 0x7f78600c8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:38.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.598+0000 7f78654c6700 1 -- 172.21.15.78:0/3365916632 >> 172.21.15.78:0/3365916632 conn(0x7f786006cb50 msgr2=0x7f786006b140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:38.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.598+0000 7f78654c6700 1 -- 172.21.15.78:0/3365916632 shutdown_connections 2023-10-11T09:40:38.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.598+0000 7f78654c6700 1 -- 172.21.15.78:0/3365916632 wait complete. 2023-10-11T09:40:38.627 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:40:38.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.798+0000 7f983f45d700 1 Processor -- start 2023-10-11T09:40:38.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.798+0000 7f983f45d700 1 -- start start 2023-10-11T09:40:38.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.802+0000 7f983f45d700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f98380b3240 0x7f98380d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:38.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.802+0000 7f983f45d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f98380b3a90 0x7f98380b85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:38.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.802+0000 7f983f45d700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98380b47f0 con 0x7f98380b3610 2023-10-11T09:40:38.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.802+0000 7f983f45d700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98380b8130 con 0x7f98380b3240 2023-10-11T09:40:38.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.802+0000 7f983f45d700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98380b3f30 con 0x7f98380b3a90 2023-10-11T09:40:38.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.802+0000 7f983dc5a700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f98380b3610 0x7f983815d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:51552/0 (socket says 172.21.15.78:51552) 2023-10-11T09:40:38.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.802+0000 7f983dc5a700 1 -- 172.21.15.78:0/2606356156 learned_addr learned my addr 172.21.15.78:0/2606356156 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:38.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.802+0000 7f983e45b700 1 --2- 172.21.15.78:0/2606356156 >> v2:172.21.15.133:3300/0 conn(0x7f98380b3240 0x7f98380d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:38.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.802+0000 7f983ec5c700 1 --2- 172.21.15.78:0/2606356156 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f98380b3a90 0x7f98380b85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:38.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.802+0000 7f983d459700 1 -- 172.21.15.78:0/2606356156 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 701924322 0 0) 0x7f9828001110 con 0x7f98380b3610 2023-10-11T09:40:38.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.802+0000 7f983d459700 1 -- 172.21.15.78:0/2606356156 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98240035d0 con 0x7f98380b3610 2023-10-11T09:40:38.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.802+0000 7f983ec5c700 1 -- 172.21.15.78:0/2606356156 >> v2:172.21.15.133:3300/0 conn(0x7f98380b3240 msgr2=0x7f98380d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:38.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.802+0000 7f983ec5c700 1 --2- 172.21.15.78:0/2606356156 >> v2:172.21.15.133:3300/0 conn(0x7f98380b3240 0x7f98380d29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:38.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.802+0000 7f983ec5c700 1 -- 172.21.15.78:0/2606356156 >> v1:172.21.15.78:6789/0 conn(0x7f98380b3610 legacy=0x7f983815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:38.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.802+0000 7f983ec5c700 1 -- 172.21.15.78:0/2606356156 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98380b41e0 con 0x7f98380b3a90 2023-10-11T09:40:38.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.802+0000 7f983e45b700 1 --2- 172.21.15.78:0/2606356156 >> v2:172.21.15.133:3300/0 conn(0x7f98380b3240 0x7f98380d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:40:38.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.806+0000 7f983ec5c700 1 --2- 172.21.15.78:0/2606356156 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f98380b3a90 0x7f98380b85c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9830009d90 tx=0x7f983000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5395f7fae7407be4 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:38.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.806+0000 7f983d459700 1 -- 172.21.15.78:0/2606356156 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983000e070 con 0x7f98380b3a90 2023-10-11T09:40:38.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.806+0000 7f983d459700 1 -- 172.21.15.78:0/2606356156 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f983000bb20 con 0x7f98380b3a90 2023-10-11T09:40:38.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.806+0000 7f983d459700 1 -- 172.21.15.78:0/2606356156 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9830012430 con 0x7f98380b3a90 2023-10-11T09:40:38.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.806+0000 7f983f45d700 1 -- 172.21.15.78:0/2606356156 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f98380b3a90 msgr2=0x7f98380b85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:38.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.806+0000 7f983f45d700 1 --2- 172.21.15.78:0/2606356156 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f98380b3a90 0x7f98380b85c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9830009d90 tx=0x7f983000b330 comp rx=0 tx=0).stop 2023-10-11T09:40:38.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.806+0000 7f983f45d700 1 -- 172.21.15.78:0/2606356156 shutdown_connections 2023-10-11T09:40:38.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.806+0000 7f983f45d700 1 --2- 172.21.15.78:0/2606356156 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f98380b3a90 0x7f98380b85c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:38.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.806+0000 7f983f45d700 1 --2- 172.21.15.78:0/2606356156 >> v2:172.21.15.133:3300/0 conn(0x7f98380b3240 0x7f98380d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:38.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.806+0000 7f983f45d700 1 -- 172.21.15.78:0/2606356156 >> 172.21.15.78:0/2606356156 conn(0x7f983809c260 msgr2=0x7f98380d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:38.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.806+0000 7f983f45d700 1 -- 172.21.15.78:0/2606356156 shutdown_connections 2023-10-11T09:40:38.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.806+0000 7f983f45d700 1 -- 172.21.15.78:0/2606356156 wait complete. 2023-10-11T09:40:38.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.810+0000 7f983f45d700 1 Processor -- start 2023-10-11T09:40:38.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.810+0000 7f983f45d700 1 -- start start 2023-10-11T09:40:38.818 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.814+0000 7f983f45d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98380b3240 0x7f98381f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:38.819 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.814+0000 7f983f45d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98380b3610 0x7f98380d0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:38.819 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.814+0000 7f983f45d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f98380b3a90 0x7f98380d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:38.819 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.814+0000 7f983f45d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983815ed80 con 0x7f98380b3240 2023-10-11T09:40:38.819 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.814+0000 7f983f45d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983815b6f0 con 0x7f98380b3610 2023-10-11T09:40:38.819 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.814+0000 7f983f45d700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f983815eaa0 con 0x7f98380b3a90 2023-10-11T09:40:38.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.814+0000 7f983e45b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98380b3240 0x7f98381f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:38.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.814+0000 7f983e45b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98380b3240 0x7f98381f5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58128/0 (socket says 172.21.15.78:58128) 2023-10-11T09:40:38.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.814+0000 7f983e45b700 1 -- 172.21.15.78:0/1498363217 learned_addr learned my addr 172.21.15.78:0/1498363217 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:38.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.814+0000 7f983ec5c700 1 --2- 172.21.15.78:0/1498363217 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f98380b3a90 0x7f98380d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:38.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.814+0000 7f983dc5a700 1 --2- 172.21.15.78:0/1498363217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98380b3610 0x7f98380d0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:38.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.814+0000 7f983e45b700 1 -- 172.21.15.78:0/1498363217 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f98380b3a90 msgr2=0x7f98380d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:38.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.814+0000 7f983e45b700 1 --2- 172.21.15.78:0/1498363217 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f98380b3a90 0x7f98380d10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:38.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.814+0000 7f983e45b700 1 -- 172.21.15.78:0/1498363217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98380b3610 msgr2=0x7f98380d0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:38.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.814+0000 7f983e45b700 1 --2- 172.21.15.78:0/1498363217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98380b3610 0x7f98380d0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:38.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.814+0000 7f983e45b700 1 -- 172.21.15.78:0/1498363217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98340094b0 con 0x7f98380b3240 2023-10-11T09:40:38.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.814+0000 7f983dc5a700 1 --2- 172.21.15.78:0/1498363217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98380b3610 0x7f98380d0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:40:38.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.814+0000 7f983ec5c700 1 --2- 172.21.15.78:0/1498363217 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f98380b3a90 0x7f98380d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:40:38.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.814+0000 7f983e45b700 1 --2- 172.21.15.78:0/1498363217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98380b3240 0x7f98381f5cc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f983400ebe0 tx=0x7f9834009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:38.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.814+0000 7f982effd700 1 -- 172.21.15.78:0/1498363217 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983400c8b0 con 0x7f98380b3240 2023-10-11T09:40:38.824 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.814+0000 7f982effd700 1 -- 172.21.15.78:0/1498363217 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f983400ca10 con 0x7f98380b3240 2023-10-11T09:40:38.824 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.814+0000 7f983f45d700 1 -- 172.21.15.78:0/1498363217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9830009a70 con 0x7f98380b3240 2023-10-11T09:40:38.824 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.814+0000 7f982effd700 1 -- 172.21.15.78:0/1498363217 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983400cda0 con 0x7f98380b3240 2023-10-11T09:40:38.824 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.814+0000 7f983f45d700 1 -- 172.21.15.78:0/1498363217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98381f9e80 con 0x7f98380b3240 2023-10-11T09:40:38.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.818+0000 7f983f45d700 1 -- 172.21.15.78:0/1498363217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98381f9a00 con 0x7f98380b3240 2023-10-11T09:40:38.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.818+0000 7f982effd700 1 -- 172.21.15.78:0/1498363217 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f9834013470 con 0x7f98380b3240 2023-10-11T09:40:38.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.822+0000 7f982effd700 1 --2- 172.21.15.78:0/1498363217 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f981c065ed0 0x7f981c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:38.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.822+0000 7f982effd700 1 -- 172.21.15.78:0/1498363217 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f98340835f0 con 0x7f98380b3240 2023-10-11T09:40:38.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.822+0000 7f983dc5a700 1 --2- 172.21.15.78:0/1498363217 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f981c065ed0 0x7f981c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:38.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.822+0000 7f983dc5a700 1 --2- 172.21.15.78:0/1498363217 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f981c065ed0 0x7f981c068380 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9828001270 tx=0x7f9828009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:38.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:38.826+0000 7f982effd700 1 -- 172.21.15.78:0/1498363217 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f98381f9a00 con 0x7f98380b3240 2023-10-11T09:40:39.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.006+0000 7f983f45d700 1 -- 172.21.15.78:0/1498363217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f98381f9a00 con 0x7f98380b3240 2023-10-11T09:40:39.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.010+0000 7f982effd700 1 -- 172.21.15.78:0/1498363217 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7f98381f9a00 con 0x7f98380b3240 2023-10-11T09:40:39.013 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:40:39.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.010+0000 7f983f45d700 1 -- 172.21.15.78:0/1498363217 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f981c065ed0 msgr2=0x7f981c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:39.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.010+0000 7f983f45d700 1 --2- 172.21.15.78:0/1498363217 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f981c065ed0 0x7f981c068380 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9828001270 tx=0x7f9828009040 comp rx=0 tx=0).stop 2023-10-11T09:40:39.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.010+0000 7f983f45d700 1 -- 172.21.15.78:0/1498363217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98380b3240 msgr2=0x7f98381f5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:39.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.010+0000 7f983f45d700 1 --2- 172.21.15.78:0/1498363217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98380b3240 0x7f98381f5cc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f983400ebe0 tx=0x7f9834009e30 comp rx=0 tx=0).stop 2023-10-11T09:40:39.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.010+0000 7f983f45d700 1 -- 172.21.15.78:0/1498363217 shutdown_connections 2023-10-11T09:40:39.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.010+0000 7f983f45d700 1 --2- 172.21.15.78:0/1498363217 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f981c065ed0 0x7f981c068380 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:39.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.010+0000 7f983f45d700 1 --2- 172.21.15.78:0/1498363217 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f98380b3a90 0x7f98380d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:39.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.010+0000 7f983f45d700 1 --2- 172.21.15.78:0/1498363217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98380b3610 0x7f98380d0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:39.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.010+0000 7f983f45d700 1 --2- 172.21.15.78:0/1498363217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98380b3240 0x7f98381f5cc0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:39.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.010+0000 7f983f45d700 1 -- 172.21.15.78:0/1498363217 >> 172.21.15.78:0/1498363217 conn(0x7f983809c260 msgr2=0x7f98380afb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:39.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.010+0000 7f983f45d700 1 -- 172.21.15.78:0/1498363217 shutdown_connections 2023-10-11T09:40:39.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.010+0000 7f983f45d700 1 -- 172.21.15.78:0/1498363217 wait complete. 2023-10-11T09:40:39.036 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-11T09:40:39.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.206+0000 7f91348b1700 1 Processor -- start 2023-10-11T09:40:39.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.206+0000 7f91348b1700 1 -- start start 2023-10-11T09:40:39.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.206+0000 7f91348b1700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f9130156ce0 0x7f9130160b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:39.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.206+0000 7f91348b1700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f91300d1660 0x7f91300d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:39.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.206+0000 7f91348b1700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f91300d28e0 0x7f91300d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:39.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.206+0000 7f91348b1700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f913015dc10 con 0x7f9130156ce0 2023-10-11T09:40:39.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.206+0000 7f91348b1700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9130160380 con 0x7f91300d1660 2023-10-11T09:40:39.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.206+0000 7f91348b1700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f913015d930 con 0x7f91300d28e0 2023-10-11T09:40:39.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.210+0000 7f912f7fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f9130156ce0 0x7f9130160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:39.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.210+0000 7f9127fff700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f91300d1660 0x7f91300d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:39.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.210+0000 7f912ffff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f91300d28e0 0x7f91300d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:39.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.210+0000 7f9127fff700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f91300d1660 0x7f91300d0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:36568/0 (socket says 172.21.15.78:36568) 2023-10-11T09:40:39.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.210+0000 7f912ffff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f91300d28e0 0x7f91300d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:38140/0 (socket says 172.21.15.78:38140) 2023-10-11T09:40:39.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.210+0000 7f9127fff700 1 -- 172.21.15.78:0/2076110827 learned_addr learned my addr 172.21.15.78:0/2076110827 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:39.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.210+0000 7f912ffff700 1 -- 172.21.15.78:0/2076110827 >> v2:172.21.15.133:3300/0 conn(0x7f91300d1660 msgr2=0x7f91300d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:39.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.210+0000 7f912ffff700 1 --2- 172.21.15.78:0/2076110827 >> v2:172.21.15.133:3300/0 conn(0x7f91300d1660 0x7f91300d0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:39.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.210+0000 7f912ffff700 1 -- 172.21.15.78:0/2076110827 >> v2:172.21.15.78:3300/0 conn(0x7f9130156ce0 msgr2=0x7f9130160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:39.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.210+0000 7f912ffff700 1 --2- 172.21.15.78:0/2076110827 >> v2:172.21.15.78:3300/0 conn(0x7f9130156ce0 0x7f9130160b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:39.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.210+0000 7f912ffff700 1 -- 172.21.15.78:0/2076110827 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9130160620 con 0x7f91300d28e0 2023-10-11T09:40:39.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.210+0000 7f9127fff700 1 --2- 172.21.15.78:0/2076110827 >> v2:172.21.15.133:3300/0 conn(0x7f91300d1660 0x7f91300d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:40:39.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.210+0000 7f912f7fe700 1 --2- 172.21.15.78:0/2076110827 >> v2:172.21.15.78:3300/0 conn(0x7f9130156ce0 0x7f9130160b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:40:39.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.210+0000 7f912ffff700 1 --2- 172.21.15.78:0/2076110827 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f91300d28e0 0x7f91300d0f90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9130160550 tx=0x7f912800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d755a759137ee93 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:39.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.210+0000 7f912effd700 1 -- 172.21.15.78:0/2076110827 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9128016070 con 0x7f91300d28e0 2023-10-11T09:40:39.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.210+0000 7f912effd700 1 -- 172.21.15.78:0/2076110827 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912800baf0 con 0x7f91300d28e0 2023-10-11T09:40:39.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.210+0000 7f912effd700 1 -- 172.21.15.78:0/2076110827 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912800e070 con 0x7f91300d28e0 2023-10-11T09:40:39.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.210+0000 7f91348b1700 1 -- 172.21.15.78:0/2076110827 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f91300d28e0 msgr2=0x7f91300d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:39.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.210+0000 7f91348b1700 1 --2- 172.21.15.78:0/2076110827 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f91300d28e0 0x7f91300d0f90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9130160550 tx=0x7f912800b330 comp rx=0 tx=0).stop 2023-10-11T09:40:39.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.210+0000 7f91348b1700 1 -- 172.21.15.78:0/2076110827 shutdown_connections 2023-10-11T09:40:39.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.210+0000 7f91348b1700 1 --2- 172.21.15.78:0/2076110827 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f91300d28e0 0x7f91300d0f90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:39.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.210+0000 7f91348b1700 1 --2- 172.21.15.78:0/2076110827 >> v2:172.21.15.133:3300/0 conn(0x7f91300d1660 0x7f91300d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:39.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.210+0000 7f91348b1700 1 --2- 172.21.15.78:0/2076110827 >> v2:172.21.15.78:3300/0 conn(0x7f9130156ce0 0x7f9130160b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:39.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.210+0000 7f91348b1700 1 -- 172.21.15.78:0/2076110827 >> 172.21.15.78:0/2076110827 conn(0x7f913006afb0 msgr2=0x7f913006b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:39.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.214+0000 7f91348b1700 1 -- 172.21.15.78:0/2076110827 shutdown_connections 2023-10-11T09:40:39.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.214+0000 7f91348b1700 1 -- 172.21.15.78:0/2076110827 wait complete. 2023-10-11T09:40:39.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.214+0000 7f91348b1700 1 Processor -- start 2023-10-11T09:40:39.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.214+0000 7f91348b1700 1 -- start start 2023-10-11T09:40:39.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.214+0000 7f91348b1700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f91300d1660 0x7f91300cc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:39.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.214+0000 7f91348b1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91300d28e0 0x7f91300ca960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:39.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.214+0000 7f91348b1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9130156ce0 0x7f91300c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:39.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.214+0000 7f91348b1700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91300c9420 con 0x7f9130156ce0 2023-10-11T09:40:39.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.214+0000 7f91348b1700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91300c69b0 con 0x7f91300d28e0 2023-10-11T09:40:39.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.214+0000 7f91348b1700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91300c6c90 con 0x7f91300d1660 2023-10-11T09:40:39.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.218+0000 7f912ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9130156ce0 0x7f91300c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:39.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.218+0000 7f912ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9130156ce0 0x7f91300c7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58150/0 (socket says 172.21.15.78:58150) 2023-10-11T09:40:39.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.218+0000 7f912ffff700 1 -- 172.21.15.78:0/3858490196 learned_addr learned my addr 172.21.15.78:0/3858490196 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:39.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.218+0000 7f912f7fe700 1 --2- 172.21.15.78:0/3858490196 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f91300d1660 0x7f91300cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:39.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.218+0000 7f9127fff700 1 --2- 172.21.15.78:0/3858490196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91300d28e0 0x7f91300ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:39.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.218+0000 7f912ffff700 1 -- 172.21.15.78:0/3858490196 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f91300d1660 msgr2=0x7f91300cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:39.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.218+0000 7f912ffff700 1 --2- 172.21.15.78:0/3858490196 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f91300d1660 0x7f91300cc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:39.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.218+0000 7f912ffff700 1 -- 172.21.15.78:0/3858490196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91300d28e0 msgr2=0x7f91300ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:39.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.218+0000 7f912ffff700 1 --2- 172.21.15.78:0/3858490196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91300d28e0 0x7f91300ca960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:39.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.218+0000 7f912ffff700 1 -- 172.21.15.78:0/3858490196 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9118009570 con 0x7f9130156ce0 2023-10-11T09:40:39.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.218+0000 7f912ffff700 1 --2- 172.21.15.78:0/3858490196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9130156ce0 0x7f91300c7d30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9128014040 tx=0x7f9128004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:39.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.218+0000 7f912cff9700 1 -- 172.21.15.78:0/3858490196 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9128016040 con 0x7f9130156ce0 2023-10-11T09:40:39.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.218+0000 7f912cff9700 1 -- 172.21.15.78:0/3858490196 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91280044f0 con 0x7f9130156ce0 2023-10-11T09:40:39.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.218+0000 7f91348b1700 1 -- 172.21.15.78:0/3858490196 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91200094b0 con 0x7f9130156ce0 2023-10-11T09:40:39.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.218+0000 7f912cff9700 1 -- 172.21.15.78:0/3858490196 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91280048a0 con 0x7f9130156ce0 2023-10-11T09:40:39.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.218+0000 7f91348b1700 1 -- 172.21.15.78:0/3858490196 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9128009a70 con 0x7f9130156ce0 2023-10-11T09:40:39.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.218+0000 7f91348b1700 1 -- 172.21.15.78:0/3858490196 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9130065d40 con 0x7f9130156ce0 2023-10-11T09:40:39.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.222+0000 7f912cff9700 1 -- 172.21.15.78:0/3858490196 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f912801b440 con 0x7f9130156ce0 2023-10-11T09:40:39.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.222+0000 7f912cff9700 1 --2- 172.21.15.78:0/3858490196 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9110065f00 0x7f91100683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:39.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.222+0000 7f912cff9700 1 -- 172.21.15.78:0/3858490196 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9128084b30 con 0x7f9130156ce0 2023-10-11T09:40:39.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.222+0000 7f912f7fe700 1 --2- 172.21.15.78:0/3858490196 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9110065f00 0x7f91100683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:39.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.226+0000 7f912f7fe700 1 --2- 172.21.15.78:0/3858490196 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9110065f00 0x7f91100683b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9120009bf0 tx=0x7f9120008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:39.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.230+0000 7f912cff9700 1 -- 172.21.15.78:0/3858490196 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f9128052440 con 0x7f9130156ce0 2023-10-11T09:40:39.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.406+0000 7f91348b1700 1 -- 172.21.15.78:0/3858490196 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9130160bb0 con 0x7f9130156ce0 2023-10-11T09:40:39.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.406+0000 7f912cff9700 1 -- 172.21.15.78:0/3858490196 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) v1 ==== 84+0+126 (secure 0 0 0) 0x7f9128055a60 con 0x7f9130156ce0 2023-10-11T09:40:39.411 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:40:39.411 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2023-10-11T09:40:39.411 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:40:39.411 INFO:teuthology.orchestra.run.smithi078.stdout: "poolnum": 1, 2023-10-11T09:40:39.411 INFO:teuthology.orchestra.run.smithi078.stdout: "poolname": ".mgr" 2023-10-11T09:40:39.412 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:40:39.412 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:40:39.412 INFO:teuthology.orchestra.run.smithi078.stdout: "poolnum": 2, 2023-10-11T09:40:39.412 INFO:teuthology.orchestra.run.smithi078.stdout: "poolname": "rbd" 2023-10-11T09:40:39.412 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:40:39.413 INFO:teuthology.orchestra.run.smithi078.stdout:] 2023-10-11T09:40:39.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.406+0000 7f91348b1700 1 -- 172.21.15.78:0/3858490196 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9110065f00 msgr2=0x7f91100683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:39.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.410+0000 7f91348b1700 1 --2- 172.21.15.78:0/3858490196 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9110065f00 0x7f91100683b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9120009bf0 tx=0x7f9120008040 comp rx=0 tx=0).stop 2023-10-11T09:40:39.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.410+0000 7f91348b1700 1 -- 172.21.15.78:0/3858490196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9130156ce0 msgr2=0x7f91300c7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:39.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.410+0000 7f91348b1700 1 --2- 172.21.15.78:0/3858490196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9130156ce0 0x7f91300c7d30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9128014040 tx=0x7f9128004af0 comp rx=0 tx=0).stop 2023-10-11T09:40:39.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.410+0000 7f91348b1700 1 -- 172.21.15.78:0/3858490196 shutdown_connections 2023-10-11T09:40:39.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.410+0000 7f91348b1700 1 --2- 172.21.15.78:0/3858490196 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9110065f00 0x7f91100683b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:39.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.410+0000 7f91348b1700 1 --2- 172.21.15.78:0/3858490196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9130156ce0 0x7f91300c7d30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:39.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.410+0000 7f91348b1700 1 --2- 172.21.15.78:0/3858490196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91300d28e0 0x7f91300ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:39.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.410+0000 7f91348b1700 1 --2- 172.21.15.78:0/3858490196 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f91300d1660 0x7f91300cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:39.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.410+0000 7f91348b1700 1 -- 172.21.15.78:0/3858490196 >> 172.21.15.78:0/3858490196 conn(0x7f913006afb0 msgr2=0x7f913015e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:39.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.410+0000 7f91348b1700 1 -- 172.21.15.78:0/3858490196 shutdown_connections 2023-10-11T09:40:39.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.410+0000 7f91348b1700 1 -- 172.21.15.78:0/3858490196 wait complete. 2023-10-11T09:40:39.434 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-11T09:40:39.434 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-11T09:40:39.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.602+0000 7f32b45d9700 1 Processor -- start 2023-10-11T09:40:39.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.606+0000 7f32b45d9700 1 -- start start 2023-10-11T09:40:39.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.606+0000 7f32b45d9700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32ac0b4ab0 0x7f32ac160dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:39.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.606+0000 7f32b45d9700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f32ac0cfbe0 0x7f32ac0d0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:39.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.606+0000 7f32b45d9700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32ac161310 con 0x7f32ac15df90 2023-10-11T09:40:39.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.606+0000 7f32b45d9700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32ac0cd720 con 0x7f32ac0cfbe0 2023-10-11T09:40:39.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.606+0000 7f32b45d9700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac0cda00 con 0x7f32ac0b4ab0 2023-10-11T09:40:39.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.606+0000 7f32b2375700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32ac0b4ab0 0x7f32ac160dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:39.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.606+0000 7f32b1b74700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f32ac0cfbe0 0x7f32ac0d0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:39.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.606+0000 7f32b2375700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32ac0b4ab0 0x7f32ac160dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:38152/0 (socket says 172.21.15.78:38152) 2023-10-11T09:40:39.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.606+0000 7f32b2375700 1 -- 172.21.15.78:0/3811729961 learned_addr learned my addr 172.21.15.78:0/3811729961 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:39.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.606+0000 7f32b2375700 1 -- 172.21.15.78:0/3811729961 >> v2:172.21.15.133:3300/0 conn(0x7f32ac0cfbe0 msgr2=0x7f32ac0d0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:39.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.606+0000 7f32b2375700 1 --2- 172.21.15.78:0/3811729961 >> v2:172.21.15.133:3300/0 conn(0x7f32ac0cfbe0 0x7f32ac0d0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:39.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.606+0000 7f32b2375700 1 -- 172.21.15.78:0/3811729961 >> v1:172.21.15.133:6789/0 conn(0x7f32ac15df90 legacy=0x7f32ac0b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:39.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.606+0000 7f32b1373700 1 -- 172.21.15.78:0/3811729961 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4242537447 0 0) 0x7f32ac161310 con 0x7f32ac15df90 2023-10-11T09:40:39.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.606+0000 7f32b2375700 1 -- 172.21.15.78:0/3811729961 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ac0cf3e0 con 0x7f32ac0b4ab0 2023-10-11T09:40:39.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.610+0000 7f32b2375700 1 --2- 172.21.15.78:0/3811729961 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32ac0b4ab0 0x7f32ac160dd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f32a8009910 tx=0x7f32a8009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=266e2db07d3b313d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:39.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.610+0000 7f32b1373700 1 -- 172.21.15.78:0/3811729961 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a8011830 con 0x7f32ac0b4ab0 2023-10-11T09:40:39.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.610+0000 7f32b1373700 1 -- 172.21.15.78:0/3811729961 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a8011990 con 0x7f32ac0b4ab0 2023-10-11T09:40:39.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.610+0000 7f32b1373700 1 -- 172.21.15.78:0/3811729961 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a8011d40 con 0x7f32ac0b4ab0 2023-10-11T09:40:39.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.610+0000 7f32b45d9700 1 -- 172.21.15.78:0/3811729961 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32ac0b4ab0 msgr2=0x7f32ac160dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:39.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.610+0000 7f32b45d9700 1 --2- 172.21.15.78:0/3811729961 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32ac0b4ab0 0x7f32ac160dd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f32a8009910 tx=0x7f32a8009c20 comp rx=0 tx=0).stop 2023-10-11T09:40:39.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.610+0000 7f32b45d9700 1 -- 172.21.15.78:0/3811729961 shutdown_connections 2023-10-11T09:40:39.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.610+0000 7f32b45d9700 1 --2- 172.21.15.78:0/3811729961 >> v2:172.21.15.133:3300/0 conn(0x7f32ac0cfbe0 0x7f32ac0d0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:39.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.610+0000 7f32b45d9700 1 --2- 172.21.15.78:0/3811729961 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32ac0b4ab0 0x7f32ac160dd0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:39.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.610+0000 7f32b45d9700 1 -- 172.21.15.78:0/3811729961 >> 172.21.15.78:0/3811729961 conn(0x7f32ac06b1d0 msgr2=0x7f32ac06b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:39.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.610+0000 7f32b45d9700 1 -- 172.21.15.78:0/3811729961 shutdown_connections 2023-10-11T09:40:39.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.610+0000 7f32b45d9700 1 -- 172.21.15.78:0/3811729961 wait complete. 2023-10-11T09:40:39.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.614+0000 7f32b45d9700 1 Processor -- start 2023-10-11T09:40:39.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.614+0000 7f32b45d9700 1 -- start start 2023-10-11T09:40:39.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.614+0000 7f32b45d9700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32ac0b4ab0 0x7f32ac0cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:39.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.614+0000 7f32b45d9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32ac0cfbe0 0x7f32ac0cc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:39.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.614+0000 7f32b45d9700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32ac15df90 0x7f32ac0c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:39.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.614+0000 7f32b45d9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac0c7bb0 con 0x7f32ac0cfbe0 2023-10-11T09:40:39.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.614+0000 7f32b45d9700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac0cc980 con 0x7f32ac0b4ab0 2023-10-11T09:40:39.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.614+0000 7f32b45d9700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac1f68e0 con 0x7f32ac15df90 2023-10-11T09:40:39.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.614+0000 7f32b1b74700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32ac0cfbe0 0x7f32ac0cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:39.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.614+0000 7f32b2375700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32ac0b4ab0 0x7f32ac0cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:39.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.614+0000 7f32b1b74700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32ac0cfbe0 0x7f32ac0cc440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58154/0 (socket says 172.21.15.78:58154) 2023-10-11T09:40:39.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.614+0000 7f32b2375700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32ac0b4ab0 0x7f32ac0cbf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:36606/0 (socket says 172.21.15.78:36606) 2023-10-11T09:40:39.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.614+0000 7f32b1b74700 1 -- 172.21.15.78:0/1763143411 learned_addr learned my addr 172.21.15.78:0/1763143411 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:39.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.614+0000 7f32b2b76700 1 --2- 172.21.15.78:0/1763143411 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32ac15df90 0x7f32ac0c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:39.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.614+0000 7f32b1b74700 1 -- 172.21.15.78:0/1763143411 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32ac15df90 msgr2=0x7f32ac0c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:39.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.614+0000 7f32b1b74700 1 --2- 172.21.15.78:0/1763143411 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32ac15df90 0x7f32ac0c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:39.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.614+0000 7f32b1b74700 1 -- 172.21.15.78:0/1763143411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32ac0b4ab0 msgr2=0x7f32ac0cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:39.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.614+0000 7f32b1b74700 1 --2- 172.21.15.78:0/1763143411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32ac0b4ab0 0x7f32ac0cbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:39.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.614+0000 7f32b1b74700 1 -- 172.21.15.78:0/1763143411 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a8009570 con 0x7f32ac0cfbe0 2023-10-11T09:40:39.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.618+0000 7f32b1b74700 1 --2- 172.21.15.78:0/1763143411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32ac0cfbe0 0x7f32ac0cc440 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f329c00e9d0 tx=0x7f329c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:39.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.618+0000 7f32a2ffd700 1 -- 172.21.15.78:0/1763143411 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c009720 con 0x7f32ac0cfbe0 2023-10-11T09:40:39.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.618+0000 7f32a2ffd700 1 -- 172.21.15.78:0/1763143411 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329c00ee70 con 0x7f32ac0cfbe0 2023-10-11T09:40:39.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.618+0000 7f32a2ffd700 1 -- 172.21.15.78:0/1763143411 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c00ca50 con 0x7f32ac0cfbe0 2023-10-11T09:40:39.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.618+0000 7f32b45d9700 1 -- 172.21.15.78:0/1763143411 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32ac152a90 con 0x7f32ac0cfbe0 2023-10-11T09:40:39.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.618+0000 7f32b45d9700 1 -- 172.21.15.78:0/1763143411 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32ac0c7280 con 0x7f32ac0cfbe0 2023-10-11T09:40:39.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.618+0000 7f32b45d9700 1 -- 172.21.15.78:0/1763143411 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32ac04e530 con 0x7f32ac0cfbe0 2023-10-11T09:40:39.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.622+0000 7f32a2ffd700 1 -- 172.21.15.78:0/1763143411 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f329c00cbb0 con 0x7f32ac0cfbe0 2023-10-11T09:40:39.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.622+0000 7f32a2ffd700 1 --2- 172.21.15.78:0/1763143411 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3290065ed0 0x7f3290068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:39.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.622+0000 7f32a2ffd700 1 -- 172.21.15.78:0/1763143411 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f329c083570 con 0x7f32ac0cfbe0 2023-10-11T09:40:39.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.622+0000 7f32b2375700 1 --2- 172.21.15.78:0/1763143411 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3290065ed0 0x7f3290068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:39.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.622+0000 7f32b2375700 1 --2- 172.21.15.78:0/1763143411 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3290065ed0 0x7f3290068380 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f32a8000c00 tx=0x7f32a8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:39.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.626+0000 7f32a2ffd700 1 -- 172.21.15.78:0/1763143411 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f329c050e70 con 0x7f32ac0cfbe0 2023-10-11T09:40:39.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:39.798+0000 7f32b45d9700 1 -- 172.21.15.78:0/1763143411 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f32ac0c7cf0 con 0x7f32ac0cfbe0 2023-10-11T09:40:40.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.186+0000 7f32a2ffd700 1 -- 172.21.15.78:0/1763143411 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) v1 ==== 138+0+0 (secure 0 0 0) 0x7f329c00ce60 con 0x7f32ac0cfbe0 2023-10-11T09:40:40.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.190+0000 7f32b45d9700 1 -- 172.21.15.78:0/1763143411 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3290065ed0 msgr2=0x7f3290068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:40.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.190+0000 7f32b45d9700 1 --2- 172.21.15.78:0/1763143411 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3290065ed0 0x7f3290068380 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f32a8000c00 tx=0x7f32a8008030 comp rx=0 tx=0).stop 2023-10-11T09:40:40.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.190+0000 7f32b45d9700 1 -- 172.21.15.78:0/1763143411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32ac0cfbe0 msgr2=0x7f32ac0cc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:40.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.190+0000 7f32b45d9700 1 --2- 172.21.15.78:0/1763143411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32ac0cfbe0 0x7f32ac0cc440 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f329c00e9d0 tx=0x7f329c00ece0 comp rx=0 tx=0).stop 2023-10-11T09:40:40.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.194+0000 7f32b45d9700 1 -- 172.21.15.78:0/1763143411 shutdown_connections 2023-10-11T09:40:40.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.194+0000 7f32b45d9700 1 --2- 172.21.15.78:0/1763143411 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3290065ed0 0x7f3290068380 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:40.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.194+0000 7f32b45d9700 1 --2- 172.21.15.78:0/1763143411 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32ac15df90 0x7f32ac0c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:40.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.194+0000 7f32b45d9700 1 --2- 172.21.15.78:0/1763143411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32ac0cfbe0 0x7f32ac0cc440 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:40.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.194+0000 7f32b45d9700 1 --2- 172.21.15.78:0/1763143411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32ac0b4ab0 0x7f32ac0cbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:40.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.194+0000 7f32b45d9700 1 -- 172.21.15.78:0/1763143411 >> 172.21.15.78:0/1763143411 conn(0x7f32ac06b1d0 msgr2=0x7f32ac0b7280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:40.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.194+0000 7f32b45d9700 1 -- 172.21.15.78:0/1763143411 shutdown_connections 2023-10-11T09:40:40.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.194+0000 7f32b45d9700 1 -- 172.21.15.78:0/1763143411 wait complete. 2023-10-11T09:40:40.205 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' created 2023-10-11T09:40:40.220 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-11T09:40:40.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.394+0000 7fc69e441700 1 Processor -- start 2023-10-11T09:40:40.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.394+0000 7fc69e441700 1 -- start start 2023-10-11T09:40:40.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.398+0000 7fc69e441700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc6980ce8d0 0x7fc6980cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:40.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.398+0000 7fc69e441700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6980ced30 con 0x7fc69806fa10 2023-10-11T09:40:40.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.398+0000 7fc69e441700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6980707e0 con 0x7fc69815ac70 2023-10-11T09:40:40.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.398+0000 7fc69e441700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6980cefd0 con 0x7fc6980ce8d0 2023-10-11T09:40:40.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.398+0000 7fc69d43f700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fc69806fa10 0x7fc69806fde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:51564/0 (socket says 172.21.15.78:51564) 2023-10-11T09:40:40.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.398+0000 7fc69d43f700 1 -- 172.21.15.78:0/2359934739 learned_addr learned my addr 172.21.15.78:0/2359934739 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:40.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.398+0000 7fc69dc40700 1 --2- 172.21.15.78:0/2359934739 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc6980ce8d0 0x7fc6980cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:40.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.398+0000 7fc68ffff700 1 -- 172.21.15.78:0/2359934739 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 475422699 0 0) 0x7fc6980ced30 con 0x7fc69806fa10 2023-10-11T09:40:40.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.398+0000 7fc68ffff700 1 -- 172.21.15.78:0/2359934739 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6840035d0 con 0x7fc69806fa10 2023-10-11T09:40:40.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.398+0000 7fc69dc40700 1 -- 172.21.15.78:0/2359934739 >> v1:172.21.15.133:6789/0 conn(0x7fc69815ac70 legacy=0x7fc6980d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:40.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.398+0000 7fc68ffff700 1 -- 172.21.15.78:0/2359934739 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2934471575 0 0) 0x7fc6980707e0 con 0x7fc69815ac70 2023-10-11T09:40:40.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.398+0000 7fc69dc40700 1 -- 172.21.15.78:0/2359934739 >> v1:172.21.15.78:6789/0 conn(0x7fc69806fa10 legacy=0x7fc69806fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:40.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.398+0000 7fc69dc40700 1 -- 172.21.15.78:0/2359934739 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6980cf2b0 con 0x7fc6980ce8d0 2023-10-11T09:40:40.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.398+0000 7fc69dc40700 1 --2- 172.21.15.78:0/2359934739 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc6980ce8d0 0x7fc6980cc1b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc690009dc0 tx=0x7fc69000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae50c61ab8bb9327 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:40.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.398+0000 7fc68ffff700 1 -- 172.21.15.78:0/2359934739 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69000e070 con 0x7fc6980ce8d0 2023-10-11T09:40:40.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.398+0000 7fc68ffff700 1 -- 172.21.15.78:0/2359934739 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc69000bb20 con 0x7fc6980ce8d0 2023-10-11T09:40:40.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.398+0000 7fc68ffff700 1 -- 172.21.15.78:0/2359934739 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69000be90 con 0x7fc6980ce8d0 2023-10-11T09:40:40.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.402+0000 7fc69e441700 1 -- 172.21.15.78:0/2359934739 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc6980ce8d0 msgr2=0x7fc6980cc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:40.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.402+0000 7fc69e441700 1 --2- 172.21.15.78:0/2359934739 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc6980ce8d0 0x7fc6980cc1b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc690009dc0 tx=0x7fc69000b330 comp rx=0 tx=0).stop 2023-10-11T09:40:40.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.402+0000 7fc69e441700 1 -- 172.21.15.78:0/2359934739 shutdown_connections 2023-10-11T09:40:40.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.402+0000 7fc69e441700 1 --2- 172.21.15.78:0/2359934739 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc6980ce8d0 0x7fc6980cc1b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:40.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.402+0000 7fc69e441700 1 -- 172.21.15.78:0/2359934739 >> 172.21.15.78:0/2359934739 conn(0x7fc69806ace0 msgr2=0x7fc6980b2830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:40.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.402+0000 7fc69e441700 1 -- 172.21.15.78:0/2359934739 shutdown_connections 2023-10-11T09:40:40.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.402+0000 7fc69e441700 1 -- 172.21.15.78:0/2359934739 wait complete. 2023-10-11T09:40:40.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.402+0000 7fc69e441700 1 Processor -- start 2023-10-11T09:40:40.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.402+0000 7fc69e441700 1 -- start start 2023-10-11T09:40:40.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.406+0000 7fc69e441700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc6980ce8d0 0x7fc6981585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:40.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.406+0000 7fc69e441700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc69815ac70 0x7fc698159110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:40.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.406+0000 7fc69e441700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc69806cae0 0x7fc6980aff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:40.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.406+0000 7fc69e441700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc698158cb0 con 0x7fc69815ac70 2023-10-11T09:40:40.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.406+0000 7fc69e441700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc698159650 con 0x7fc69806cae0 2023-10-11T09:40:40.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.406+0000 7fc69e441700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc69806bd20 con 0x7fc6980ce8d0 2023-10-11T09:40:40.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.406+0000 7fc69cc3e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc69815ac70 0x7fc698159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:40.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.406+0000 7fc69cc3e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc69815ac70 0x7fc698159110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58162/0 (socket says 172.21.15.78:58162) 2023-10-11T09:40:40.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.406+0000 7fc69cc3e700 1 -- 172.21.15.78:0/1046397526 learned_addr learned my addr 172.21.15.78:0/1046397526 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:40.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.406+0000 7fc69d43f700 1 --2- 172.21.15.78:0/1046397526 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc6980ce8d0 0x7fc6981585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:40.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.406+0000 7fc69dc40700 1 --2- 172.21.15.78:0/1046397526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc69806cae0 0x7fc6980aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:40.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.406+0000 7fc69cc3e700 1 -- 172.21.15.78:0/1046397526 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc6980ce8d0 msgr2=0x7fc6981585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:40.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.406+0000 7fc69cc3e700 1 --2- 172.21.15.78:0/1046397526 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc6980ce8d0 0x7fc6981585b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:40.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.406+0000 7fc69cc3e700 1 -- 172.21.15.78:0/1046397526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc69806cae0 msgr2=0x7fc6980aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:40.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.406+0000 7fc69cc3e700 1 --2- 172.21.15.78:0/1046397526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc69806cae0 0x7fc6980aff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:40.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.406+0000 7fc69cc3e700 1 -- 172.21.15.78:0/1046397526 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc690009a70 con 0x7fc69815ac70 2023-10-11T09:40:40.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.406+0000 7fc69d43f700 1 --2- 172.21.15.78:0/1046397526 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc6980ce8d0 0x7fc6981585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:40:40.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.406+0000 7fc69cc3e700 1 --2- 172.21.15.78:0/1046397526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc69815ac70 0x7fc698159110 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc68800c520 tx=0x7fc68800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:40.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.406+0000 7fc68dffb700 1 -- 172.21.15.78:0/1046397526 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6880054e0 con 0x7fc69815ac70 2023-10-11T09:40:40.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.406+0000 7fc68dffb700 1 -- 172.21.15.78:0/1046397526 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6880036a0 con 0x7fc69815ac70 2023-10-11T09:40:40.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.406+0000 7fc69e441700 1 -- 172.21.15.78:0/1046397526 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc69806c000 con 0x7fc69815ac70 2023-10-11T09:40:40.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.406+0000 7fc68dffb700 1 -- 172.21.15.78:0/1046397526 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc688003a50 con 0x7fc69815ac70 2023-10-11T09:40:40.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.406+0000 7fc69e441700 1 -- 172.21.15.78:0/1046397526 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc69806c5c0 con 0x7fc69815ac70 2023-10-11T09:40:40.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.406+0000 7fc69e441700 1 -- 172.21.15.78:0/1046397526 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc698065d40 con 0x7fc69815ac70 2023-10-11T09:40:40.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.410+0000 7fc68dffb700 1 -- 172.21.15.78:0/1046397526 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fc688003bb0 con 0x7fc69815ac70 2023-10-11T09:40:40.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.410+0000 7fc68dffb700 1 --2- 172.21.15.78:0/1046397526 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc67c065f00 0x7fc67c0683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:40.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.410+0000 7fc68dffb700 1 -- 172.21.15.78:0/1046397526 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5580+0+0 (secure 0 0 0) 0x7fc688082ab0 con 0x7fc69815ac70 2023-10-11T09:40:40.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.410+0000 7fc69d43f700 1 --2- 172.21.15.78:0/1046397526 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc67c065f00 0x7fc67c0683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:40.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.414+0000 7fc69d43f700 1 --2- 172.21.15.78:0/1046397526 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc67c065f00 0x7fc67c0683b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc694005930 tx=0x7fc694009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:40.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.418+0000 7fc68dffb700 1 -- 172.21.15.78:0/1046397526 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fc688050240 con 0x7fc69815ac70 2023-10-11T09:40:40.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:40.594+0000 7fc69e441700 1 -- 172.21.15.78:0/1046397526 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc6980b3620 con 0x7fc69815ac70 2023-10-11T09:40:41.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.202+0000 7fc68dffb700 1 -- 172.21.15.78:0/1046397526 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v21) v1 ==== 170+0+0 (secure 0 0 0) 0x7fc688017020 con 0x7fc69815ac70 2023-10-11T09:40:41.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.214+0000 7fc69e441700 1 -- 172.21.15.78:0/1046397526 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc67c065f00 msgr2=0x7fc67c0683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:41.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.214+0000 7fc69e441700 1 --2- 172.21.15.78:0/1046397526 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc67c065f00 0x7fc67c0683b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc694005930 tx=0x7fc694009040 comp rx=0 tx=0).stop 2023-10-11T09:40:41.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.214+0000 7fc69e441700 1 -- 172.21.15.78:0/1046397526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc69815ac70 msgr2=0x7fc698159110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:41.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.214+0000 7fc69e441700 1 --2- 172.21.15.78:0/1046397526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc69815ac70 0x7fc698159110 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc68800c520 tx=0x7fc68800c830 comp rx=0 tx=0).stop 2023-10-11T09:40:41.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.226+0000 7fc69e441700 1 -- 172.21.15.78:0/1046397526 shutdown_connections 2023-10-11T09:40:41.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.226+0000 7fc69e441700 1 --2- 172.21.15.78:0/1046397526 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc67c065f00 0x7fc67c0683b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:41.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.226+0000 7fc69e441700 1 --2- 172.21.15.78:0/1046397526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc69806cae0 0x7fc6980aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:41.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.226+0000 7fc69e441700 1 --2- 172.21.15.78:0/1046397526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc69815ac70 0x7fc698159110 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:41.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.226+0000 7fc69e441700 1 --2- 172.21.15.78:0/1046397526 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc6980ce8d0 0x7fc6981585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:41.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.226+0000 7fc69e441700 1 -- 172.21.15.78:0/1046397526 >> 172.21.15.78:0/1046397526 conn(0x7fc69806fa10 msgr2=0x7fc6980b2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:41.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.226+0000 7fc69e441700 1 -- 172.21.15.78:0/1046397526 shutdown_connections 2023-10-11T09:40:41.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.226+0000 7fc69e441700 1 -- 172.21.15.78:0/1046397526 wait complete. 2023-10-11T09:40:41.235 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' created 2023-10-11T09:40:41.252 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-11T09:40:41.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.422+0000 7fe761f00700 1 Processor -- start 2023-10-11T09:40:41.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.422+0000 7fe761f00700 1 -- start start 2023-10-11T09:40:41.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.426+0000 7fe761f00700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe75c0ad290 0x7fe75c0b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:41.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.426+0000 7fe761f00700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe75c06fde0 con 0x7fe75c15c900 2023-10-11T09:40:41.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.426+0000 7fe761f00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe75c0ae000 con 0x7fe75c0ce7f0 2023-10-11T09:40:41.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.426+0000 7fe761f00700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe75c0705f0 con 0x7fe75c0ad290 2023-10-11T09:40:41.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.426+0000 7fe753fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe75c0ad290 0x7fe75c0b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:41.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.426+0000 7fe753fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe75c0ad290 0x7fe75c0b1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58168/0 (socket says 172.21.15.78:58168) 2023-10-11T09:40:41.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.426+0000 7fe753fff700 1 -- 172.21.15.78:0/3146710554 learned_addr learned my addr 172.21.15.78:0/3146710554 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:41.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.426+0000 7fe753fff700 1 -- 172.21.15.78:0/3146710554 >> v1:172.21.15.133:6789/0 conn(0x7fe75c0ce7f0 legacy=0x7fe75c06bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:41.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.426+0000 7fe753fff700 1 -- 172.21.15.78:0/3146710554 >> v1:172.21.15.78:6789/0 conn(0x7fe75c15c900 legacy=0x7fe75c071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:41.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.426+0000 7fe753fff700 1 -- 172.21.15.78:0/3146710554 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe75c0ad7e0 con 0x7fe75c0ad290 2023-10-11T09:40:41.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.430+0000 7fe753fff700 1 --2- 172.21.15.78:0/3146710554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe75c0ad290 0x7fe75c0b1a10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe74c009910 tx=0x7fe74c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186bf76e4ed36c25 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:41.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.430+0000 7fe7537fe700 1 -- 172.21.15.78:0/3146710554 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c004230 con 0x7fe75c0ad290 2023-10-11T09:40:41.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.430+0000 7fe7537fe700 1 -- 172.21.15.78:0/3146710554 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74c004390 con 0x7fe75c0ad290 2023-10-11T09:40:41.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.430+0000 7fe7537fe700 1 -- 172.21.15.78:0/3146710554 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c009e60 con 0x7fe75c0ad290 2023-10-11T09:40:41.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.430+0000 7fe761f00700 1 -- 172.21.15.78:0/3146710554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe75c0ad290 msgr2=0x7fe75c0b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:41.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.430+0000 7fe761f00700 1 --2- 172.21.15.78:0/3146710554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe75c0ad290 0x7fe75c0b1a10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe74c009910 tx=0x7fe74c009c20 comp rx=0 tx=0).stop 2023-10-11T09:40:41.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.430+0000 7fe761f00700 1 -- 172.21.15.78:0/3146710554 shutdown_connections 2023-10-11T09:40:41.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.430+0000 7fe761f00700 1 --2- 172.21.15.78:0/3146710554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe75c0ad290 0x7fe75c0b1a10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:41.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.430+0000 7fe761f00700 1 -- 172.21.15.78:0/3146710554 >> 172.21.15.78:0/3146710554 conn(0x7fe75c06a060 msgr2=0x7fe75c06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:41.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.430+0000 7fe761f00700 1 -- 172.21.15.78:0/3146710554 shutdown_connections 2023-10-11T09:40:41.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.430+0000 7fe761f00700 1 -- 172.21.15.78:0/3146710554 wait complete. 2023-10-11T09:40:41.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.434+0000 7fe761f00700 1 Processor -- start 2023-10-11T09:40:41.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.434+0000 7fe761f00700 1 -- start start 2023-10-11T09:40:41.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.434+0000 7fe761f00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe75c0ad290 0x7fe75c14c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:41.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.434+0000 7fe761f00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe75c0ce7f0 0x7fe75c156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:41.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.434+0000 7fe761f00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe75c15c900 0x7fe75c157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:41.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.434+0000 7fe761f00700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c15ded0 con 0x7fe75c15c900 2023-10-11T09:40:41.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.434+0000 7fe761f00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c0cdfd0 con 0x7fe75c0ad290 2023-10-11T09:40:41.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.434+0000 7fe761f00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c15dcd0 con 0x7fe75c0ce7f0 2023-10-11T09:40:41.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.434+0000 7fe7616ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe75c15c900 0x7fe75c157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:41.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.438+0000 7fe7616ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe75c15c900 0x7fe75c157330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58176/0 (socket says 172.21.15.78:58176) 2023-10-11T09:40:41.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.438+0000 7fe7616ff700 1 -- 172.21.15.78:0/1022308679 learned_addr learned my addr 172.21.15.78:0/1022308679 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:41.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.438+0000 7fe753fff700 1 --2- 172.21.15.78:0/1022308679 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe75c0ce7f0 0x7fe75c156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:41.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.438+0000 7fe760efe700 1 --2- 172.21.15.78:0/1022308679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe75c0ad290 0x7fe75c14c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:41.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.438+0000 7fe7616ff700 1 -- 172.21.15.78:0/1022308679 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe75c0ce7f0 msgr2=0x7fe75c156df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:41.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.438+0000 7fe7616ff700 1 --2- 172.21.15.78:0/1022308679 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe75c0ce7f0 0x7fe75c156df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:41.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.438+0000 7fe7616ff700 1 -- 172.21.15.78:0/1022308679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe75c0ad290 msgr2=0x7fe75c14c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:41.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.438+0000 7fe7616ff700 1 --2- 172.21.15.78:0/1022308679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe75c0ad290 0x7fe75c14c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:41.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.438+0000 7fe7616ff700 1 -- 172.21.15.78:0/1022308679 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe74c009570 con 0x7fe75c15c900 2023-10-11T09:40:41.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.438+0000 7fe7616ff700 1 --2- 172.21.15.78:0/1022308679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe75c15c900 0x7fe75c157330 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe75400cc10 tx=0x7fe75400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:41.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.438+0000 7fe7517fa700 1 -- 172.21.15.78:0/1022308679 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75400ab10 con 0x7fe75c15c900 2023-10-11T09:40:41.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.438+0000 7fe7517fa700 1 -- 172.21.15.78:0/1022308679 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75400ac70 con 0x7fe75c15c900 2023-10-11T09:40:41.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.438+0000 7fe7517fa700 1 -- 172.21.15.78:0/1022308679 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe754012420 con 0x7fe75c15c900 2023-10-11T09:40:41.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.438+0000 7fe761f00700 1 -- 172.21.15.78:0/1022308679 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe75c14cdf0 con 0x7fe75c15c900 2023-10-11T09:40:41.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.438+0000 7fe761f00700 1 -- 172.21.15.78:0/1022308679 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe75c0af3b0 con 0x7fe75c15c900 2023-10-11T09:40:41.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.442+0000 7fe761f00700 1 -- 172.21.15.78:0/1022308679 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe75c065d40 con 0x7fe75c15c900 2023-10-11T09:40:41.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.442+0000 7fe7517fa700 1 -- 172.21.15.78:0/1022308679 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fe754012580 con 0x7fe75c15c900 2023-10-11T09:40:41.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.442+0000 7fe7517fa700 1 --2- 172.21.15.78:0/1022308679 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe748065ed0 0x7fe748068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:41.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.442+0000 7fe7517fa700 1 -- 172.21.15.78:0/1022308679 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fe754083ae0 con 0x7fe75c15c900 2023-10-11T09:40:41.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.442+0000 7fe760efe700 1 --2- 172.21.15.78:0/1022308679 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe748065ed0 0x7fe748068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:41.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.446+0000 7fe760efe700 1 --2- 172.21.15.78:0/1022308679 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe748065ed0 0x7fe748068380 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe758001640 tx=0x7fe758009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:41.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.454+0000 7fe7517fa700 1 -- 172.21.15.78:0/1022308679 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fe754051180 con 0x7fe75c15c900 2023-10-11T09:40:41.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.666+0000 7fe761f00700 1 -- 172.21.15.78:0/1022308679 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe75c0ce340 con 0x7fe75c15c900 2023-10-11T09:40:41.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.682+0000 7fe7517fa700 1 -- 172.21.15.78:0/1022308679 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fe7540547a0 con 0x7fe75c15c900 2023-10-11T09:40:41.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.706+0000 7fe761f00700 1 -- 172.21.15.78:0/1022308679 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe748065ed0 msgr2=0x7fe748068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:41.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.706+0000 7fe761f00700 1 --2- 172.21.15.78:0/1022308679 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe748065ed0 0x7fe748068380 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe758001640 tx=0x7fe758009040 comp rx=0 tx=0).stop 2023-10-11T09:40:41.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.706+0000 7fe761f00700 1 -- 172.21.15.78:0/1022308679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe75c15c900 msgr2=0x7fe75c157330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:41.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.706+0000 7fe761f00700 1 --2- 172.21.15.78:0/1022308679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe75c15c900 0x7fe75c157330 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe75400cc10 tx=0x7fe75400cf20 comp rx=0 tx=0).stop 2023-10-11T09:40:41.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.706+0000 7fe761f00700 1 -- 172.21.15.78:0/1022308679 shutdown_connections 2023-10-11T09:40:41.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.706+0000 7fe761f00700 1 --2- 172.21.15.78:0/1022308679 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe748065ed0 0x7fe748068380 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:41.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.706+0000 7fe761f00700 1 --2- 172.21.15.78:0/1022308679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe75c15c900 0x7fe75c157330 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:41.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.706+0000 7fe761f00700 1 --2- 172.21.15.78:0/1022308679 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe75c0ce7f0 0x7fe75c156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:41.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.706+0000 7fe761f00700 1 --2- 172.21.15.78:0/1022308679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe75c0ad290 0x7fe75c14c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:41.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.706+0000 7fe761f00700 1 -- 172.21.15.78:0/1022308679 >> 172.21.15.78:0/1022308679 conn(0x7fe75c06a060 msgr2=0x7fe75c06a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:41.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.710+0000 7fe761f00700 1 -- 172.21.15.78:0/1022308679 shutdown_connections 2023-10-11T09:40:41.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.710+0000 7fe761f00700 1 -- 172.21.15.78:0/1022308679 wait complete. 2023-10-11T09:40:41.718 INFO:teuthology.orchestra.run.smithi078.stderr:new fs with metadata pool 3 and data pool 4 2023-10-11T09:40:41.738 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:40:41.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.898+0000 7f6574e5e700 1 Processor -- start 2023-10-11T09:40:41.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.898+0000 7f6574e5e700 1 -- start start 2023-10-11T09:40:41.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.902+0000 7f6574e5e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65700b2600 0x7f65700d3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:41.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.902+0000 7f6574e5e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f65700d0df0 0x7f65700d1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:41.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.902+0000 7f6574e5e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65700cdcc0 con 0x7f65700d2fb0 2023-10-11T09:40:41.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.902+0000 7f6574e5e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65700d2d20 con 0x7f65700d0df0 2023-10-11T09:40:41.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.902+0000 7f6574e5e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65700cfd30 con 0x7f65700b2600 2023-10-11T09:40:41.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.902+0000 7f656e59c700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f65700d2fb0 0x7f65700d3380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:51572/0 (socket says 172.21.15.78:51572) 2023-10-11T09:40:41.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.902+0000 7f656e59c700 1 -- 172.21.15.78:0/2561022053 learned_addr learned my addr 172.21.15.78:0/2561022053 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:41.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.902+0000 7f656ed9d700 1 --2- 172.21.15.78:0/2561022053 >> v2:172.21.15.133:3300/0 conn(0x7f65700d0df0 0x7f65700d1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:41.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.902+0000 7f6567fff700 1 --2- 172.21.15.78:0/2561022053 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65700b2600 0x7f65700d3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:41.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.902+0000 7f656dd9b700 1 -- 172.21.15.78:0/2561022053 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2435387371 0 0) 0x7f65700cdcc0 con 0x7f65700d2fb0 2023-10-11T09:40:41.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.902+0000 7f656dd9b700 1 -- 172.21.15.78:0/2561022053 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f655c0035d0 con 0x7f65700d2fb0 2023-10-11T09:40:41.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.902+0000 7f656dd9b700 1 -- 172.21.15.78:0/2561022053 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3946860391 0 0) 0x7f655c0035d0 con 0x7f65700d2fb0 2023-10-11T09:40:41.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.902+0000 7f656dd9b700 1 -- 172.21.15.78:0/2561022053 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65700b2600 msgr2=0x7f65700d3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:41.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.902+0000 7f656dd9b700 1 --2- 172.21.15.78:0/2561022053 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65700b2600 0x7f65700d3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:41.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.902+0000 7f656dd9b700 1 -- 172.21.15.78:0/2561022053 >> v2:172.21.15.133:3300/0 conn(0x7f65700d0df0 msgr2=0x7f65700d1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:41.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.902+0000 7f656dd9b700 1 --2- 172.21.15.78:0/2561022053 >> v2:172.21.15.133:3300/0 conn(0x7f65700d0df0 0x7f65700d1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:41.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.902+0000 7f656dd9b700 1 -- 172.21.15.78:0/2561022053 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65700cffb0 con 0x7f65700d2fb0 2023-10-11T09:40:41.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.902+0000 7f656dd9b700 1 -- 172.21.15.78:0/2561022053 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f6560001670 con 0x7f65700d2fb0 2023-10-11T09:40:41.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.902+0000 7f656dd9b700 1 -- 172.21.15.78:0/2561022053 >> v1:172.21.15.78:6789/0 conn(0x7f65700d2fb0 legacy=0x7f65700d3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:41.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.902+0000 7f656dd9b700 1 --2- 172.21.15.78:0/2561022053 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65700b2600 0x7f655c003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:41.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.902+0000 7f656dd9b700 1 --2- 172.21.15.78:0/2561022053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f655c003fa0 0x7f655c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:41.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.902+0000 7f656dd9b700 1 --2- 172.21.15.78:0/2561022053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f655c004db0 0x7f655c005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:41.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.902+0000 7f656dd9b700 1 -- 172.21.15.78:0/2561022053 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65700d2d20 con 0x7f655c003fa0 2023-10-11T09:40:41.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.902+0000 7f656dd9b700 1 -- 172.21.15.78:0/2561022053 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65700cfd30 con 0x7f655c004db0 2023-10-11T09:40:41.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.902+0000 7f656dd9b700 1 -- 172.21.15.78:0/2561022053 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f655c00ba30 con 0x7f65700b2600 2023-10-11T09:40:41.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.902+0000 7f6567fff700 1 --2- 172.21.15.78:0/2561022053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f655c003fa0 0x7f655c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:41.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.906+0000 7f656e59c700 1 --2- 172.21.15.78:0/2561022053 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65700b2600 0x7f655c003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:41.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.906+0000 7f656ed9d700 1 --2- 172.21.15.78:0/2561022053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f655c004db0 0x7f655c005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:41.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.906+0000 7f6567fff700 1 -- 172.21.15.78:0/2561022053 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65700b2600 msgr2=0x7f655c003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:41.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.906+0000 7f6567fff700 1 --2- 172.21.15.78:0/2561022053 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65700b2600 0x7f655c003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:41.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.906+0000 7f6567fff700 1 -- 172.21.15.78:0/2561022053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f655c004db0 msgr2=0x7f655c005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:41.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.906+0000 7f6567fff700 1 --2- 172.21.15.78:0/2561022053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f655c004db0 0x7f655c005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:41.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.906+0000 7f6567fff700 1 -- 172.21.15.78:0/2561022053 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f655c0035d0 con 0x7f655c003fa0 2023-10-11T09:40:41.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.906+0000 7f656ed9d700 1 --2- 172.21.15.78:0/2561022053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f655c004db0 0x7f655c005220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:40:41.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.906+0000 7f6567fff700 1 --2- 172.21.15.78:0/2561022053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f655c003fa0 0x7f655c004870 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6558009b70 tx=0x7f655c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d2ec28dfd3a100 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:41.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.906+0000 7f656dd9b700 1 -- 172.21.15.78:0/2561022053 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655800e6c0 con 0x7f655c003fa0 2023-10-11T09:40:41.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.906+0000 7f656dd9b700 1 -- 172.21.15.78:0/2561022053 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f655800e820 con 0x7f655c003fa0 2023-10-11T09:40:41.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.906+0000 7f656dd9b700 1 -- 172.21.15.78:0/2561022053 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655800eb90 con 0x7f655c003fa0 2023-10-11T09:40:41.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.906+0000 7f6574e5e700 1 -- 172.21.15.78:0/2561022053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f655c003fa0 msgr2=0x7f655c004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:41.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.906+0000 7f6574e5e700 1 --2- 172.21.15.78:0/2561022053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f655c003fa0 0x7f655c004870 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6558009b70 tx=0x7f655c0058a0 comp rx=0 tx=0).stop 2023-10-11T09:40:41.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.906+0000 7f656ed9d700 1 -- 172.21.15.78:0/2561022053 reap_dead start 2023-10-11T09:40:41.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.906+0000 7f6574e5e700 1 -- 172.21.15.78:0/2561022053 shutdown_connections 2023-10-11T09:40:41.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.906+0000 7f6574e5e700 1 -- 172.21.15.78:0/2561022053 >> 172.21.15.78:0/2561022053 conn(0x7f657006cb50 msgr2=0x7f657006cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:41.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.906+0000 7f6574e5e700 1 -- 172.21.15.78:0/2561022053 shutdown_connections 2023-10-11T09:40:41.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.906+0000 7f6574e5e700 1 -- 172.21.15.78:0/2561022053 wait complete. 2023-10-11T09:40:41.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.910+0000 7f6574e5e700 1 Processor -- start 2023-10-11T09:40:41.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.910+0000 7f6574e5e700 1 -- start start 2023-10-11T09:40:41.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.910+0000 7f6574e5e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f655c003fa0 0x7f65701f1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:41.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.910+0000 7f6574e5e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f657015ecf0 0x7f657015f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:41.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.910+0000 7f6574e5e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65700ab1d0 0x7f65700ab640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:41.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.910+0000 7f6574e5e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65700aabf0 con 0x7f655c003fa0 2023-10-11T09:40:41.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.910+0000 7f6574e5e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65700abb80 con 0x7f65700ab1d0 2023-10-11T09:40:41.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.910+0000 7f6574e5e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65700aa3e0 con 0x7f657015ecf0 2023-10-11T09:40:41.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.910+0000 7f656e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f655c003fa0 0x7f65701f1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:41.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.910+0000 7f656e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f655c003fa0 0x7f65701f1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58192/0 (socket says 172.21.15.78:58192) 2023-10-11T09:40:41.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.910+0000 7f656e59c700 1 -- 172.21.15.78:0/3040762351 learned_addr learned my addr 172.21.15.78:0/3040762351 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:41.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.910+0000 7f6567fff700 1 --2- 172.21.15.78:0/3040762351 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f657015ecf0 0x7f657015f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:41.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.910+0000 7f656ed9d700 1 --2- 172.21.15.78:0/3040762351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65700ab1d0 0x7f65700ab640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:41.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.910+0000 7f656e59c700 1 -- 172.21.15.78:0/3040762351 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f657015ecf0 msgr2=0x7f657015f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:41.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.910+0000 7f656e59c700 1 --2- 172.21.15.78:0/3040762351 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f657015ecf0 0x7f657015f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:41.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.910+0000 7f656e59c700 1 -- 172.21.15.78:0/3040762351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65700ab1d0 msgr2=0x7f65700ab640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:41.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.910+0000 7f656e59c700 1 --2- 172.21.15.78:0/3040762351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65700ab1d0 0x7f65700ab640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:41.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.910+0000 7f656e59c700 1 -- 172.21.15.78:0/3040762351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657015f1f0 con 0x7f655c003fa0 2023-10-11T09:40:41.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.910+0000 7f656e59c700 1 --2- 172.21.15.78:0/3040762351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f655c003fa0 0x7f65701f1700 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6560007370 tx=0x7f6560007730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:41.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.910+0000 7f6566ffd700 1 -- 172.21.15.78:0/3040762351 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6560007cc0 con 0x7f655c003fa0 2023-10-11T09:40:41.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.910+0000 7f6574e5e700 1 -- 172.21.15.78:0/3040762351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65700aa630 con 0x7f655c003fa0 2023-10-11T09:40:41.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.910+0000 7f6566ffd700 1 -- 172.21.15.78:0/3040762351 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f656000f870 con 0x7f655c003fa0 2023-10-11T09:40:41.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.910+0000 7f6566ffd700 1 -- 172.21.15.78:0/3040762351 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656000fc00 con 0x7f655c003fa0 2023-10-11T09:40:41.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.910+0000 7f6574e5e700 1 -- 172.21.15.78:0/3040762351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65701f1c40 con 0x7f655c003fa0 2023-10-11T09:40:41.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.914+0000 7f6574e5e700 1 -- 172.21.15.78:0/3040762351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f657015f4d0 con 0x7f655c003fa0 2023-10-11T09:40:41.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.914+0000 7f6566ffd700 1 -- 172.21.15.78:0/3040762351 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f656000fd60 con 0x7f655c003fa0 2023-10-11T09:40:41.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.914+0000 7f6566ffd700 1 --2- 172.21.15.78:0/3040762351 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6550065ed0 0x7f6550068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:41.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.914+0000 7f6566ffd700 1 -- 172.21.15.78:0/3040762351 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f65600832c0 con 0x7f655c003fa0 2023-10-11T09:40:41.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.914+0000 7f6567fff700 1 --2- 172.21.15.78:0/3040762351 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6550065ed0 0x7f6550068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:41.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.918+0000 7f6567fff700 1 --2- 172.21.15.78:0/3040762351 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6550065ed0 0x7f6550068380 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f65580090b0 tx=0x7f6558008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:41.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:41.922+0000 7f6566ffd700 1 -- 172.21.15.78:0/3040762351 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f6560050900 con 0x7f655c003fa0 2023-10-11T09:40:42.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.098+0000 7f6574e5e700 1 -- 172.21.15.78:0/3040762351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f657004e530 con 0x7f655c003fa0 2023-10-11T09:40:42.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.102+0000 7f6566ffd700 1 -- 172.21.15.78:0/3040762351 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+17824 (secure 0 0 0) 0x7f6560053f20 con 0x7f655c003fa0 2023-10-11T09:40:42.108 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:40:42.109 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":22,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:40:41.672768+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:40:39.862581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T09:40:40.663641+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:40:42.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.106+0000 7f6574e5e700 1 -- 172.21.15.78:0/3040762351 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6550065ed0 msgr2=0x7f6550068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:42.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.106+0000 7f6574e5e700 1 --2- 172.21.15.78:0/3040762351 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6550065ed0 0x7f6550068380 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f65580090b0 tx=0x7f6558008030 comp rx=0 tx=0).stop 2023-10-11T09:40:42.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.106+0000 7f6574e5e700 1 -- 172.21.15.78:0/3040762351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f655c003fa0 msgr2=0x7f65701f1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:42.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.106+0000 7f6574e5e700 1 --2- 172.21.15.78:0/3040762351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f655c003fa0 0x7f65701f1700 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6560007370 tx=0x7f6560007730 comp rx=0 tx=0).stop 2023-10-11T09:40:42.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.106+0000 7f6574e5e700 1 -- 172.21.15.78:0/3040762351 shutdown_connections 2023-10-11T09:40:42.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.106+0000 7f6574e5e700 1 --2- 172.21.15.78:0/3040762351 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6550065ed0 0x7f6550068380 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:42.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.106+0000 7f6574e5e700 1 --2- 172.21.15.78:0/3040762351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65700ab1d0 0x7f65700ab640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:42.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.106+0000 7f6574e5e700 1 --2- 172.21.15.78:0/3040762351 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f657015ecf0 0x7f657015f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:42.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.106+0000 7f6574e5e700 1 --2- 172.21.15.78:0/3040762351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f655c003fa0 0x7f65701f1700 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:42.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.106+0000 7f6574e5e700 1 -- 172.21.15.78:0/3040762351 >> 172.21.15.78:0/3040762351 conn(0x7f657006cb50 msgr2=0x7f65700d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:42.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.106+0000 7f6574e5e700 1 -- 172.21.15.78:0/3040762351 shutdown_connections 2023-10-11T09:40:42.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.106+0000 7f6574e5e700 1 -- 172.21.15.78:0/3040762351 wait complete. 2023-10-11T09:40:42.133 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:40:42.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.298+0000 7f0eb5af7700 1 Processor -- start 2023-10-11T09:40:42.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.298+0000 7f0eb5af7700 1 -- start start 2023-10-11T09:40:42.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.302+0000 7f0eb5af7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0eb013f740 0x7f0eb00ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:42.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.302+0000 7f0eb5af7700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eb00b14c0 con 0x7f0eb006cd00 2023-10-11T09:40:42.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.302+0000 7f0eb5af7700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eb0070c80 con 0x7f0eb006d150 2023-10-11T09:40:42.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.302+0000 7f0eb5af7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb00ce4c0 con 0x7f0eb013f740 2023-10-11T09:40:42.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.302+0000 7f0ea7fff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f0eb006cd00 0x7f0eb00b06a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:51588/0 (socket says 172.21.15.78:51588) 2023-10-11T09:40:42.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.302+0000 7f0ea7fff700 1 -- 172.21.15.78:0/1604936253 learned_addr learned my addr 172.21.15.78:0/1604936253 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:42.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.302+0000 7f0eb4af5700 1 --2- 172.21.15.78:0/1604936253 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0eb013f740 0x7f0eb00ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:42.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.302+0000 7f0ea77fe700 1 -- 172.21.15.78:0/1604936253 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4163379986 0 0) 0x7f0eb00b14c0 con 0x7f0eb006cd00 2023-10-11T09:40:42.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.302+0000 7f0ea77fe700 1 -- 172.21.15.78:0/1604936253 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e980035d0 con 0x7f0eb006cd00 2023-10-11T09:40:42.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.302+0000 7f0ea77fe700 1 -- 172.21.15.78:0/1604936253 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1324843697 0 0) 0x7f0e980035d0 con 0x7f0eb006cd00 2023-10-11T09:40:42.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.302+0000 7f0ea77fe700 1 -- 172.21.15.78:0/1604936253 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0eb013f740 msgr2=0x7f0eb00ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:42.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.302+0000 7f0ea77fe700 1 --2- 172.21.15.78:0/1604936253 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0eb013f740 0x7f0eb00ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:42.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.302+0000 7f0ea77fe700 1 -- 172.21.15.78:0/1604936253 >> v1:172.21.15.133:6789/0 conn(0x7f0eb006d150 legacy=0x7f0eb00b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:42.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.302+0000 7f0ea77fe700 1 -- 172.21.15.78:0/1604936253 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb00cd420 con 0x7f0eb006cd00 2023-10-11T09:40:42.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.302+0000 7f0ea77fe700 1 -- 172.21.15.78:0/1604936253 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f0ea0001670 con 0x7f0eb006cd00 2023-10-11T09:40:42.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.302+0000 7f0ea77fe700 1 -- 172.21.15.78:0/1604936253 >> v1:172.21.15.78:6789/0 conn(0x7f0eb006cd00 legacy=0x7f0eb00b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:42.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.302+0000 7f0ea77fe700 1 --2- 172.21.15.78:0/1604936253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e98003a50 0x7f0e98003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:42.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.302+0000 7f0ea77fe700 1 --2- 172.21.15.78:0/1604936253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e98004870 0x7f0e98004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:42.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.302+0000 7f0ea77fe700 1 --2- 172.21.15.78:0/1604936253 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0eb013f740 0x7f0e98005260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:42.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.302+0000 7f0ea77fe700 1 -- 172.21.15.78:0/1604936253 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb00ce4c0 con 0x7f0e98003a50 2023-10-11T09:40:42.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.302+0000 7f0ea77fe700 1 -- 172.21.15.78:0/1604936253 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9800bb60 con 0x7f0e98004870 2023-10-11T09:40:42.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.302+0000 7f0ea77fe700 1 -- 172.21.15.78:0/1604936253 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9800bd20 con 0x7f0eb013f740 2023-10-11T09:40:42.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.302+0000 7f0eb4af5700 1 --2- 172.21.15.78:0/1604936253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e98003a50 0x7f0e98003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:42.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.302+0000 7f0ea7fff700 1 --2- 172.21.15.78:0/1604936253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e98004870 0x7f0e98004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:42.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.302+0000 7f0eb52f6700 1 --2- 172.21.15.78:0/1604936253 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0eb013f740 0x7f0e98005260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:42.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.306+0000 7f0eb4af5700 1 -- 172.21.15.78:0/1604936253 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0eb013f740 msgr2=0x7f0e98005260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:42.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.306+0000 7f0eb4af5700 1 --2- 172.21.15.78:0/1604936253 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0eb013f740 0x7f0e98005260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:42.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.306+0000 7f0eb4af5700 1 -- 172.21.15.78:0/1604936253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e98004870 msgr2=0x7f0e98004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:42.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.306+0000 7f0eb4af5700 1 --2- 172.21.15.78:0/1604936253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e98004870 0x7f0e98004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:42.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.306+0000 7f0eb4af5700 1 -- 172.21.15.78:0/1604936253 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e980035d0 con 0x7f0e98003a50 2023-10-11T09:40:42.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.306+0000 7f0ea7fff700 1 --2- 172.21.15.78:0/1604936253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e98004870 0x7f0e98004ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:40:42.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.306+0000 7f0eb4af5700 1 --2- 172.21.15.78:0/1604936253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e98003a50 0x7f0e98003e60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0eac007360 tx=0x7f0e980058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=950850c955c0db29 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:42.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.306+0000 7f0ea77fe700 1 -- 172.21.15.78:0/1604936253 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac007dc0 con 0x7f0e98003a50 2023-10-11T09:40:42.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.306+0000 7f0ea77fe700 1 -- 172.21.15.78:0/1604936253 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eac014400 con 0x7f0e98003a50 2023-10-11T09:40:42.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.306+0000 7f0ea77fe700 1 -- 172.21.15.78:0/1604936253 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac0147b0 con 0x7f0e98003a50 2023-10-11T09:40:42.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.306+0000 7f0eb5af7700 1 -- 172.21.15.78:0/1604936253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e98003a50 msgr2=0x7f0e98003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:42.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.306+0000 7f0eb5af7700 1 --2- 172.21.15.78:0/1604936253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e98003a50 0x7f0e98003e60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0eac007360 tx=0x7f0e980058a0 comp rx=0 tx=0).stop 2023-10-11T09:40:42.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.306+0000 7f0eb52f6700 1 -- 172.21.15.78:0/1604936253 reap_dead start 2023-10-11T09:40:42.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.306+0000 7f0eb5af7700 1 -- 172.21.15.78:0/1604936253 shutdown_connections 2023-10-11T09:40:42.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.306+0000 7f0eb5af7700 1 -- 172.21.15.78:0/1604936253 >> 172.21.15.78:0/1604936253 conn(0x7f0eb00cea60 msgr2=0x7f0eb00cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:42.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.306+0000 7f0eb5af7700 1 -- 172.21.15.78:0/1604936253 shutdown_connections 2023-10-11T09:40:42.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.306+0000 7f0eb5af7700 1 -- 172.21.15.78:0/1604936253 wait complete. 2023-10-11T09:40:42.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.310+0000 7f0eb5af7700 1 Processor -- start 2023-10-11T09:40:42.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.310+0000 7f0eb5af7700 1 -- start start 2023-10-11T09:40:42.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.310+0000 7f0eb5af7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e98003a50 0x7f0eb0152060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:42.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.310+0000 7f0eb5af7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0eb0159870 0x7f0eb0159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:42.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.310+0000 7f0eb5af7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0eb00b2a60 0x7f0eb00b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:42.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.310+0000 7f0eb5af7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb015a220 con 0x7f0eb0159870 2023-10-11T09:40:42.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.310+0000 7f0eb5af7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb00b3770 con 0x7f0eb00b2a60 2023-10-11T09:40:42.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.310+0000 7f0eb5af7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb00b3a50 con 0x7f0e98003a50 2023-10-11T09:40:42.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.310+0000 7f0ea7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0eb0159870 0x7f0eb0159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:42.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.310+0000 7f0ea7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0eb0159870 0x7f0eb0159ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58204/0 (socket says 172.21.15.78:58204) 2023-10-11T09:40:42.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.310+0000 7f0ea7fff700 1 -- 172.21.15.78:0/1562692023 learned_addr learned my addr 172.21.15.78:0/1562692023 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:42.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.310+0000 7f0eb4af5700 1 --2- 172.21.15.78:0/1562692023 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e98003a50 0x7f0eb0152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:42.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.310+0000 7f0eb52f6700 1 --2- 172.21.15.78:0/1562692023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0eb00b2a60 0x7f0eb00b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:42.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.314+0000 7f0ea7fff700 1 -- 172.21.15.78:0/1562692023 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e98003a50 msgr2=0x7f0eb0152060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:42.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.314+0000 7f0ea7fff700 1 --2- 172.21.15.78:0/1562692023 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e98003a50 0x7f0eb0152060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:42.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.314+0000 7f0ea7fff700 1 -- 172.21.15.78:0/1562692023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0eb00b2a60 msgr2=0x7f0eb00b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:42.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.314+0000 7f0ea7fff700 1 --2- 172.21.15.78:0/1562692023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0eb00b2a60 0x7f0eb00b2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:42.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.314+0000 7f0ea7fff700 1 -- 172.21.15.78:0/1562692023 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb00b3d30 con 0x7f0eb0159870 2023-10-11T09:40:42.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.314+0000 7f0eb4af5700 1 --2- 172.21.15.78:0/1562692023 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e98003a50 0x7f0eb0152060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:40:42.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.314+0000 7f0ea7fff700 1 --2- 172.21.15.78:0/1562692023 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0eb0159870 0x7f0eb0159ce0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0000f60 tx=0x7f0ea000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:42.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.314+0000 7f0ea57fa700 1 -- 172.21.15.78:0/1562692023 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea000c780 con 0x7f0eb0159870 2023-10-11T09:40:42.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.314+0000 7f0ea57fa700 1 -- 172.21.15.78:0/1562692023 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea000c8e0 con 0x7f0eb0159870 2023-10-11T09:40:42.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.314+0000 7f0eb5af7700 1 -- 172.21.15.78:0/1562692023 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb00b4010 con 0x7f0eb0159870 2023-10-11T09:40:42.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.314+0000 7f0ea57fa700 1 -- 172.21.15.78:0/1562692023 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea000cc50 con 0x7f0eb0159870 2023-10-11T09:40:42.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.314+0000 7f0eb5af7700 1 -- 172.21.15.78:0/1562692023 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb00b3410 con 0x7f0eb0159870 2023-10-11T09:40:42.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.314+0000 7f0eb5af7700 1 -- 172.21.15.78:0/1562692023 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb01e4540 con 0x7f0eb0159870 2023-10-11T09:40:42.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.318+0000 7f0ea57fa700 1 -- 172.21.15.78:0/1562692023 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f0ea001d460 con 0x7f0eb0159870 2023-10-11T09:40:42.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.318+0000 7f0ea57fa700 1 --2- 172.21.15.78:0/1562692023 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0e9c065f00 0x7f0e9c0683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:42.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.318+0000 7f0ea57fa700 1 -- 172.21.15.78:0/1562692023 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0ea00837c0 con 0x7f0eb0159870 2023-10-11T09:40:42.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.318+0000 7f0eb4af5700 1 --2- 172.21.15.78:0/1562692023 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0e9c065f00 0x7f0e9c0683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:42.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.318+0000 7f0eb4af5700 1 --2- 172.21.15.78:0/1562692023 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0e9c065f00 0x7f0e9c0683b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0eac017040 tx=0x7f0eac00b030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:42.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.322+0000 7f0ea57fa700 1 -- 172.21.15.78:0/1562692023 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f0ea0050d70 con 0x7f0eb0159870 2023-10-11T09:40:42.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.498+0000 7f0eb5af7700 1 -- 172.21.15.78:0/1562692023 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0eb0065d40 con 0x7f0eb0159870 2023-10-11T09:40:42.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.502+0000 7f0ea57fa700 1 -- 172.21.15.78:0/1562692023 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+17824 (secure 0 0 0) 0x7f0ea000cdb0 con 0x7f0eb0159870 2023-10-11T09:40:42.508 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:40:42.508 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":22,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:40:41.672768+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:40:39.862581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T09:40:40.663641+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:40:42.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.506+0000 7f0eb5af7700 1 -- 172.21.15.78:0/1562692023 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0e9c065f00 msgr2=0x7f0e9c0683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:42.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.506+0000 7f0eb5af7700 1 --2- 172.21.15.78:0/1562692023 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0e9c065f00 0x7f0e9c0683b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0eac017040 tx=0x7f0eac00b030 comp rx=0 tx=0).stop 2023-10-11T09:40:42.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.506+0000 7f0eb5af7700 1 -- 172.21.15.78:0/1562692023 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0eb0159870 msgr2=0x7f0eb0159ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:42.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.506+0000 7f0eb5af7700 1 --2- 172.21.15.78:0/1562692023 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0eb0159870 0x7f0eb0159ce0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0000f60 tx=0x7f0ea000ee80 comp rx=0 tx=0).stop 2023-10-11T09:40:42.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.506+0000 7f0eb5af7700 1 -- 172.21.15.78:0/1562692023 shutdown_connections 2023-10-11T09:40:42.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.506+0000 7f0eb5af7700 1 --2- 172.21.15.78:0/1562692023 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0e9c065f00 0x7f0e9c0683b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:42.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.506+0000 7f0eb5af7700 1 --2- 172.21.15.78:0/1562692023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0eb00b2a60 0x7f0eb00b2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:42.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.506+0000 7f0eb5af7700 1 --2- 172.21.15.78:0/1562692023 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0eb0159870 0x7f0eb0159ce0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:42.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.506+0000 7f0eb5af7700 1 --2- 172.21.15.78:0/1562692023 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e98003a50 0x7f0eb0152060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:42.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.506+0000 7f0eb5af7700 1 -- 172.21.15.78:0/1562692023 >> 172.21.15.78:0/1562692023 conn(0x7f0eb00cea60 msgr2=0x7f0eb006c520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:42.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.506+0000 7f0eb5af7700 1 -- 172.21.15.78:0/1562692023 shutdown_connections 2023-10-11T09:40:42.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.506+0000 7f0eb5af7700 1 -- 172.21.15.78:0/1562692023 wait complete. 2023-10-11T09:40:42.533 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-11T09:40:42.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.702+0000 7f03cf59e700 1 Processor -- start 2023-10-11T09:40:42.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.702+0000 7f03cf59e700 1 -- start start 2023-10-11T09:40:42.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.702+0000 7f03cf59e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f03d00b3610 0x7f03d00d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:42.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.702+0000 7f03cf59e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f03d00b3a90 0x7f03d00b85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:42.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.702+0000 7f03cf59e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03d00b47f0 con 0x7f03d00b3240 2023-10-11T09:40:42.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.702+0000 7f03cf59e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03d00b8130 con 0x7f03d00b3610 2023-10-11T09:40:42.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.702+0000 7f03cf59e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d00b3f30 con 0x7f03d00b3a90 2023-10-11T09:40:42.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.702+0000 7f03ce59c700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f03d00b3240 0x7f03d015d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:51602/0 (socket says 172.21.15.78:51602) 2023-10-11T09:40:42.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.702+0000 7f03ce59c700 1 -- 172.21.15.78:0/4281714089 learned_addr learned my addr 172.21.15.78:0/4281714089 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:42.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.702+0000 7f03cdd9b700 1 --2- 172.21.15.78:0/4281714089 >> v2:172.21.15.78:3300/0 conn(0x7f03d00b3610 0x7f03d00d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:42.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.702+0000 7f03ced9d700 1 --2- 172.21.15.78:0/4281714089 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f03d00b3a90 0x7f03d00b85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:42.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.706+0000 7f03cdd9b700 1 -- 172.21.15.78:0/4281714089 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f03d00b3a90 msgr2=0x7f03d00b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:42.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.706+0000 7f03cdd9b700 1 --2- 172.21.15.78:0/4281714089 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f03d00b3a90 0x7f03d00b85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:42.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.706+0000 7f03cdd9b700 1 -- 172.21.15.78:0/4281714089 >> v1:172.21.15.78:6789/0 conn(0x7f03d00b3240 legacy=0x7f03d015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:42.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.706+0000 7f03cdd9b700 1 -- 172.21.15.78:0/4281714089 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d00b41e0 con 0x7f03d00b3610 2023-10-11T09:40:42.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.706+0000 7f03cdd9b700 1 --2- 172.21.15.78:0/4281714089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03d00b3610 0x7f03d00d29d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f03c4009850 tx=0x7f03c4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aa8c256503b2ff9 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:42.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.706+0000 7f03cd59a700 1 -- 172.21.15.78:0/4281714089 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c4011830 con 0x7f03d00b3610 2023-10-11T09:40:42.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.706+0000 7f03cd59a700 1 -- 172.21.15.78:0/4281714089 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c4011990 con 0x7f03d00b3610 2023-10-11T09:40:42.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.706+0000 7f03cd59a700 1 -- 172.21.15.78:0/4281714089 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c4011d20 con 0x7f03d00b3610 2023-10-11T09:40:42.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.706+0000 7f03cf59e700 1 -- 172.21.15.78:0/4281714089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03d00b3610 msgr2=0x7f03d00d29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:42.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.706+0000 7f03cf59e700 1 --2- 172.21.15.78:0/4281714089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03d00b3610 0x7f03d00d29d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f03c4009850 tx=0x7f03c4009b60 comp rx=0 tx=0).stop 2023-10-11T09:40:42.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.706+0000 7f03cf59e700 1 -- 172.21.15.78:0/4281714089 shutdown_connections 2023-10-11T09:40:42.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.706+0000 7f03cf59e700 1 --2- 172.21.15.78:0/4281714089 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f03d00b3a90 0x7f03d00b85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:42.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.706+0000 7f03cf59e700 1 --2- 172.21.15.78:0/4281714089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03d00b3610 0x7f03d00d29d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:42.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.706+0000 7f03cf59e700 1 -- 172.21.15.78:0/4281714089 >> 172.21.15.78:0/4281714089 conn(0x7f03d009c260 msgr2=0x7f03d00d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:42.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.706+0000 7f03cf59e700 1 -- 172.21.15.78:0/4281714089 shutdown_connections 2023-10-11T09:40:42.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.706+0000 7f03cf59e700 1 -- 172.21.15.78:0/4281714089 wait complete. 2023-10-11T09:40:42.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.710+0000 7f03cf59e700 1 Processor -- start 2023-10-11T09:40:42.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.710+0000 7f03cf59e700 1 -- start start 2023-10-11T09:40:42.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.710+0000 7f03cf59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03d00b3240 0x7f03d015e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:42.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.710+0000 7f03cf59e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f03d00b3610 0x7f03d0159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:42.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.710+0000 7f03cf59e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03d00b3a90 0x7f03d015ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:42.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.710+0000 7f03cf59e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d00c78b0 con 0x7f03d00b3240 2023-10-11T09:40:42.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.710+0000 7f03cf59e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d015f7d0 con 0x7f03d00b3a90 2023-10-11T09:40:42.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.710+0000 7f03cf59e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d00b0ec0 con 0x7f03d00b3610 2023-10-11T09:40:42.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.710+0000 7f03ce59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03d00b3240 0x7f03d015e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:42.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.710+0000 7f03ce59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03d00b3240 0x7f03d015e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58218/0 (socket says 172.21.15.78:58218) 2023-10-11T09:40:42.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.710+0000 7f03ce59c700 1 -- 172.21.15.78:0/1023535633 learned_addr learned my addr 172.21.15.78:0/1023535633 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:42.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.710+0000 7f03cdd9b700 1 --2- 172.21.15.78:0/1023535633 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f03d00b3610 0x7f03d0159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:42.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.710+0000 7f03ced9d700 1 --2- 172.21.15.78:0/1023535633 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03d00b3a90 0x7f03d015ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:42.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.710+0000 7f03ce59c700 1 -- 172.21.15.78:0/1023535633 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f03d00b3610 msgr2=0x7f03d0159040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:42.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.710+0000 7f03ce59c700 1 --2- 172.21.15.78:0/1023535633 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f03d00b3610 0x7f03d0159040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:42.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.710+0000 7f03ce59c700 1 -- 172.21.15.78:0/1023535633 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03d00b3a90 msgr2=0x7f03d015ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:42.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.710+0000 7f03ce59c700 1 --2- 172.21.15.78:0/1023535633 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03d00b3a90 0x7f03d015ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:42.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.710+0000 7f03ce59c700 1 -- 172.21.15.78:0/1023535633 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c40094b0 con 0x7f03d00b3240 2023-10-11T09:40:42.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.710+0000 7f03ce59c700 1 --2- 172.21.15.78:0/1023535633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03d00b3240 0x7f03d015e790 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f03c000c4d0 tx=0x7f03c000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:42.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.710+0000 7f03beffd700 1 -- 172.21.15.78:0/1023535633 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c0005d90 con 0x7f03d00b3240 2023-10-11T09:40:42.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.710+0000 7f03beffd700 1 -- 172.21.15.78:0/1023535633 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c00070d0 con 0x7f03d00b3240 2023-10-11T09:40:42.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.710+0000 7f03cf59e700 1 -- 172.21.15.78:0/1023535633 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d0159580 con 0x7f03d00b3240 2023-10-11T09:40:42.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.714+0000 7f03beffd700 1 -- 172.21.15.78:0/1023535633 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c0007440 con 0x7f03d00b3240 2023-10-11T09:40:42.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.714+0000 7f03cf59e700 1 -- 172.21.15.78:0/1023535633 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d015f4f0 con 0x7f03d00b3240 2023-10-11T09:40:42.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.714+0000 7f03cf59e700 1 -- 172.21.15.78:0/1023535633 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03d00b77c0 con 0x7f03d00b3240 2023-10-11T09:40:42.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.714+0000 7f03beffd700 1 -- 172.21.15.78:0/1023535633 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f03c00075a0 con 0x7f03d00b3240 2023-10-11T09:40:42.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.718+0000 7f03beffd700 1 --2- 172.21.15.78:0/1023535633 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f03b0065ed0 0x7f03b0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:42.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.718+0000 7f03beffd700 1 -- 172.21.15.78:0/1023535633 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f03c00829e0 con 0x7f03d00b3240 2023-10-11T09:40:42.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.718+0000 7f03cdd9b700 1 --2- 172.21.15.78:0/1023535633 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f03b0065ed0 0x7f03b0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:42.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.718+0000 7f03cdd9b700 1 --2- 172.21.15.78:0/1023535633 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f03b0065ed0 0x7f03b0068380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f03c4000c00 tx=0x7f03c4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:42.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.722+0000 7f03beffd700 1 -- 172.21.15.78:0/1023535633 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f03c00500d0 con 0x7f03d00b3240 2023-10-11T09:40:42.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:42.942+0000 7f03cf59e700 1 -- 172.21.15.78:0/1023535633 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f03d0065d40 con 0x7f03d00b3240 2023-10-11T09:40:43.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.706+0000 7f03beffd700 1 -- 172.21.15.78:0/1023535633 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f03c00170b0 con 0x7f03d00b3240 2023-10-11T09:40:43.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.710+0000 7f03cf59e700 1 -- 172.21.15.78:0/1023535633 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f03b0065ed0 msgr2=0x7f03b0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:43.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.710+0000 7f03cf59e700 1 --2- 172.21.15.78:0/1023535633 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f03b0065ed0 0x7f03b0068380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f03c4000c00 tx=0x7f03c4008030 comp rx=0 tx=0).stop 2023-10-11T09:40:43.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.710+0000 7f03cf59e700 1 -- 172.21.15.78:0/1023535633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03d00b3240 msgr2=0x7f03d015e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:43.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.710+0000 7f03cf59e700 1 --2- 172.21.15.78:0/1023535633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03d00b3240 0x7f03d015e790 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f03c000c4d0 tx=0x7f03c000c890 comp rx=0 tx=0).stop 2023-10-11T09:40:43.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.710+0000 7f03cf59e700 1 -- 172.21.15.78:0/1023535633 shutdown_connections 2023-10-11T09:40:43.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.710+0000 7f03cf59e700 1 --2- 172.21.15.78:0/1023535633 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f03b0065ed0 0x7f03b0068380 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:43.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.710+0000 7f03cf59e700 1 --2- 172.21.15.78:0/1023535633 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03d00b3a90 0x7f03d015ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:43.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.710+0000 7f03cf59e700 1 --2- 172.21.15.78:0/1023535633 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f03d00b3610 0x7f03d0159040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:43.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.710+0000 7f03cf59e700 1 --2- 172.21.15.78:0/1023535633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03d00b3240 0x7f03d015e790 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:43.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.710+0000 7f03cf59e700 1 -- 172.21.15.78:0/1023535633 >> 172.21.15.78:0/1023535633 conn(0x7f03d009c260 msgr2=0x7f03d006c720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:43.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.710+0000 7f03cf59e700 1 -- 172.21.15.78:0/1023535633 shutdown_connections 2023-10-11T09:40:43.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.710+0000 7f03cf59e700 1 -- 172.21.15.78:0/1023535633 wait complete. 2023-10-11T09:40:43.738 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-11T09:40:43.739 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-11T09:40:43.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.910+0000 7fa265712700 1 Processor -- start 2023-10-11T09:40:43.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.910+0000 7fa265712700 1 -- start start 2023-10-11T09:40:43.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.914+0000 7fa265712700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fa2600b3240 0x7fa2600d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:43.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.914+0000 7fa265712700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa2600b3a90 0x7fa2600b85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:43.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.914+0000 7fa265712700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2600b47f0 con 0x7fa2600b3610 2023-10-11T09:40:43.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.914+0000 7fa265712700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2600b8130 con 0x7fa2600b3240 2023-10-11T09:40:43.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.914+0000 7fa265712700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2600b41e0 con 0x7fa2600b3a90 2023-10-11T09:40:43.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.914+0000 7fa2567fc700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fa2600b3610 0x7fa26015d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:51618/0 (socket says 172.21.15.78:51618) 2023-10-11T09:40:43.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.914+0000 7fa2567fc700 1 -- 172.21.15.78:0/498920751 learned_addr learned my addr 172.21.15.78:0/498920751 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:43.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.914+0000 7fa25effd700 1 --2- 172.21.15.78:0/498920751 >> v2:172.21.15.78:3300/0 conn(0x7fa2600b3240 0x7fa2600d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:43.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.914+0000 7fa25f7fe700 1 --2- 172.21.15.78:0/498920751 >> v2:172.21.15.133:3300/0 conn(0x7fa2600b3a90 0x7fa2600b85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:43.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.914+0000 7fa25e7fc700 1 -- 172.21.15.78:0/498920751 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2316365744 0 0) 0x7fa248001110 con 0x7fa2600b3610 2023-10-11T09:40:43.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.914+0000 7fa25e7fc700 1 -- 172.21.15.78:0/498920751 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa24c0035d0 con 0x7fa2600b3610 2023-10-11T09:40:43.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.914+0000 7fa25effd700 1 -- 172.21.15.78:0/498920751 >> v2:172.21.15.133:3300/0 conn(0x7fa2600b3a90 msgr2=0x7fa2600b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:43.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.914+0000 7fa25effd700 1 --2- 172.21.15.78:0/498920751 >> v2:172.21.15.133:3300/0 conn(0x7fa2600b3a90 0x7fa2600b85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:43.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.914+0000 7fa25effd700 1 -- 172.21.15.78:0/498920751 >> v1:172.21.15.78:6789/0 conn(0x7fa2600b3610 legacy=0x7fa26015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:43.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.914+0000 7fa25effd700 1 -- 172.21.15.78:0/498920751 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2600b7800 con 0x7fa2600b3240 2023-10-11T09:40:43.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.918+0000 7fa25effd700 1 --2- 172.21.15.78:0/498920751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2600b3240 0x7fa2600d29d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa260072b40 tx=0x7fa250009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d81d19df602faa server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:43.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.918+0000 7fa25e7fc700 1 -- 172.21.15.78:0/498920751 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa250011830 con 0x7fa2600b3240 2023-10-11T09:40:43.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.918+0000 7fa25e7fc700 1 -- 172.21.15.78:0/498920751 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa250011990 con 0x7fa2600b3240 2023-10-11T09:40:43.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.918+0000 7fa25e7fc700 1 -- 172.21.15.78:0/498920751 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa250011d20 con 0x7fa2600b3240 2023-10-11T09:40:43.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.918+0000 7fa265712700 1 -- 172.21.15.78:0/498920751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2600b3240 msgr2=0x7fa2600d29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:43.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.918+0000 7fa265712700 1 --2- 172.21.15.78:0/498920751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2600b3240 0x7fa2600d29d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa260072b40 tx=0x7fa250009b30 comp rx=0 tx=0).stop 2023-10-11T09:40:43.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.918+0000 7fa265712700 1 -- 172.21.15.78:0/498920751 shutdown_connections 2023-10-11T09:40:43.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.918+0000 7fa265712700 1 --2- 172.21.15.78:0/498920751 >> v2:172.21.15.133:3300/0 conn(0x7fa2600b3a90 0x7fa2600b85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:43.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.918+0000 7fa265712700 1 --2- 172.21.15.78:0/498920751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2600b3240 0x7fa2600d29d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:43.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.918+0000 7fa265712700 1 -- 172.21.15.78:0/498920751 >> 172.21.15.78:0/498920751 conn(0x7fa26009c260 msgr2=0x7fa2600d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:43.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.918+0000 7fa265712700 1 -- 172.21.15.78:0/498920751 shutdown_connections 2023-10-11T09:40:43.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.918+0000 7fa265712700 1 -- 172.21.15.78:0/498920751 wait complete. 2023-10-11T09:40:43.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.922+0000 7fa265712700 1 Processor -- start 2023-10-11T09:40:43.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.922+0000 7fa265712700 1 -- start start 2023-10-11T09:40:43.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.922+0000 7fa265712700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa2600b3240 0x7fa26015f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:43.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.922+0000 7fa265712700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2600b3610 0x7fa260148230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:43.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.926+0000 7fa265712700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2600b3a90 0x7fa2600d2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:43.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.926+0000 7fa265712700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26015f3e0 con 0x7fa2600b3610 2023-10-11T09:40:43.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.926+0000 7fa265712700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26015e4c0 con 0x7fa2600b3a90 2023-10-11T09:40:43.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.926+0000 7fa265712700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa26015e760 con 0x7fa2600b3240 2023-10-11T09:40:43.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.926+0000 7fa2567fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2600b3610 0x7fa260148230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:43.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.926+0000 7fa2567fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2600b3610 0x7fa260148230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58230/0 (socket says 172.21.15.78:58230) 2023-10-11T09:40:43.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.926+0000 7fa2567fc700 1 -- 172.21.15.78:0/2487105965 learned_addr learned my addr 172.21.15.78:0/2487105965 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:43.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.926+0000 7fa25effd700 1 --2- 172.21.15.78:0/2487105965 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa2600b3240 0x7fa26015f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:43.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.926+0000 7fa25f7fe700 1 --2- 172.21.15.78:0/2487105965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2600b3a90 0x7fa2600d2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:43.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.926+0000 7fa2567fc700 1 -- 172.21.15.78:0/2487105965 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa2600b3240 msgr2=0x7fa26015f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:43.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.926+0000 7fa2567fc700 1 --2- 172.21.15.78:0/2487105965 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa2600b3240 0x7fa26015f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:43.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.926+0000 7fa2567fc700 1 -- 172.21.15.78:0/2487105965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2600b3a90 msgr2=0x7fa2600d2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:43.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.926+0000 7fa2567fc700 1 --2- 172.21.15.78:0/2487105965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2600b3a90 0x7fa2600d2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:43.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.926+0000 7fa2567fc700 1 -- 172.21.15.78:0/2487105965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2500094b0 con 0x7fa2600b3610 2023-10-11T09:40:43.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.926+0000 7fa25f7fe700 1 --2- 172.21.15.78:0/2487105965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2600b3a90 0x7fa2600d2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:40:43.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.926+0000 7fa2567fc700 1 --2- 172.21.15.78:0/2487105965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2600b3610 0x7fa260148230 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa248001270 tx=0x7fa24800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:43.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.926+0000 7fa257fff700 1 -- 172.21.15.78:0/2487105965 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa248005d90 con 0x7fa2600b3610 2023-10-11T09:40:43.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.926+0000 7fa257fff700 1 -- 172.21.15.78:0/2487105965 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2480070d0 con 0x7fa2600b3610 2023-10-11T09:40:43.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.926+0000 7fa257fff700 1 -- 172.21.15.78:0/2487105965 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa248007480 con 0x7fa2600b3610 2023-10-11T09:40:43.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.926+0000 7fa265712700 1 -- 172.21.15.78:0/2487105965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa26015ea40 con 0x7fa2600b3610 2023-10-11T09:40:43.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.926+0000 7fa265712700 1 -- 172.21.15.78:0/2487105965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa26015f000 con 0x7fa2600b3610 2023-10-11T09:40:43.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.926+0000 7fa265712700 1 -- 172.21.15.78:0/2487105965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2600b7da0 con 0x7fa2600b3610 2023-10-11T09:40:43.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.930+0000 7fa257fff700 1 -- 172.21.15.78:0/2487105965 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fa2480075e0 con 0x7fa2600b3610 2023-10-11T09:40:43.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.930+0000 7fa257fff700 1 --2- 172.21.15.78:0/2487105965 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa240065ed0 0x7fa240068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:43.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.930+0000 7fa257fff700 1 -- 172.21.15.78:0/2487105965 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa248082bb0 con 0x7fa2600b3610 2023-10-11T09:40:43.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.930+0000 7fa25effd700 1 --2- 172.21.15.78:0/2487105965 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa240065ed0 0x7fa240068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:43.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.934+0000 7fa25effd700 1 --2- 172.21.15.78:0/2487105965 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa240065ed0 0x7fa240068380 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa250000c00 tx=0x7fa250008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:43.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:43.938+0000 7fa257fff700 1 -- 172.21.15.78:0/2487105965 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fa2600b7da0 con 0x7fa2600b3610 2023-10-11T09:40:44.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:44.150+0000 7fa265712700 1 -- 172.21.15.78:0/2487105965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa26015f520 con 0x7fa2600b3610 2023-10-11T09:40:45.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.134+0000 7fa257fff700 1 -- 172.21.15.78:0/2487105965 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa26015f520 con 0x7fa2600b3610 2023-10-11T09:40:45.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.138+0000 7fa265712700 1 -- 172.21.15.78:0/2487105965 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa240065ed0 msgr2=0x7fa240068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:45.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.138+0000 7fa265712700 1 --2- 172.21.15.78:0/2487105965 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa240065ed0 0x7fa240068380 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa250000c00 tx=0x7fa250008030 comp rx=0 tx=0).stop 2023-10-11T09:40:45.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.138+0000 7fa265712700 1 -- 172.21.15.78:0/2487105965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2600b3610 msgr2=0x7fa260148230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:45.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.138+0000 7fa265712700 1 --2- 172.21.15.78:0/2487105965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2600b3610 0x7fa260148230 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa248001270 tx=0x7fa24800c880 comp rx=0 tx=0).stop 2023-10-11T09:40:45.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.138+0000 7fa265712700 1 -- 172.21.15.78:0/2487105965 shutdown_connections 2023-10-11T09:40:45.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.138+0000 7fa265712700 1 --2- 172.21.15.78:0/2487105965 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa240065ed0 0x7fa240068380 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:45.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.138+0000 7fa265712700 1 --2- 172.21.15.78:0/2487105965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2600b3a90 0x7fa2600d2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:45.145 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.138+0000 7fa265712700 1 --2- 172.21.15.78:0/2487105965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2600b3610 0x7fa260148230 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:45.145 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.138+0000 7fa265712700 1 --2- 172.21.15.78:0/2487105965 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa2600b3240 0x7fa26015f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:45.145 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.138+0000 7fa265712700 1 -- 172.21.15.78:0/2487105965 >> 172.21.15.78:0/2487105965 conn(0x7fa26009c260 msgr2=0x7fa2600b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:45.145 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.138+0000 7fa265712700 1 -- 172.21.15.78:0/2487105965 shutdown_connections 2023-10-11T09:40:45.145 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.138+0000 7fa265712700 1 -- 172.21.15.78:0/2487105965 wait complete. 2023-10-11T09:40:45.168 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-11T09:40:45.168 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:40:45.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.338+0000 7fc5ab28c700 1 Processor -- start 2023-10-11T09:40:45.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.342+0000 7fc5ab28c700 1 -- start start 2023-10-11T09:40:45.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.342+0000 7fc5ab28c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fc5a415d050 0x7fc5a40cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:45.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.342+0000 7fc5ab28c700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc5a4160c60 0x7fc5a415d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:45.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.342+0000 7fc5ab28c700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a4160210 con 0x7fc5a415d960 2023-10-11T09:40:45.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.342+0000 7fc5ab28c700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a40ccd60 con 0x7fc5a415d050 2023-10-11T09:40:45.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.342+0000 7fc5ab28c700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a40ccfc0 con 0x7fc5a4160c60 2023-10-11T09:40:45.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.346+0000 7fc5aa28a700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fc5a415d050 0x7fc5a40cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:45.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.346+0000 7fc5a9a89700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc5a4160c60 0x7fc5a415d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:45.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.346+0000 7fc5a9a89700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc5a4160c60 0x7fc5a415d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:49766/0 (socket says 172.21.15.78:49766) 2023-10-11T09:40:45.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.346+0000 7fc5a9a89700 1 -- 172.21.15.78:0/3466742193 learned_addr learned my addr 172.21.15.78:0/3466742193 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:45.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.346+0000 7fc5aa28a700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fc5a415d050 0x7fc5a40cc820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56734/0 (socket says 172.21.15.78:56734) 2023-10-11T09:40:45.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.346+0000 7fc5aa28a700 1 -- 172.21.15.78:0/3466742193 >> v2:172.21.15.133:3300/0 conn(0x7fc5a4160c60 msgr2=0x7fc5a415d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:45.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.346+0000 7fc5aa28a700 1 --2- 172.21.15.78:0/3466742193 >> v2:172.21.15.133:3300/0 conn(0x7fc5a4160c60 0x7fc5a415d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:45.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.346+0000 7fc5aa28a700 1 -- 172.21.15.78:0/3466742193 >> v1:172.21.15.133:6789/0 conn(0x7fc5a415d960 legacy=0x7fc5a40b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:45.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.346+0000 7fc5aa28a700 1 -- 172.21.15.78:0/3466742193 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a40cd2a0 con 0x7fc5a415d050 2023-10-11T09:40:45.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.346+0000 7fc5aa28a700 1 --2- 172.21.15.78:0/3466742193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5a415d050 0x7fc5a40cc820 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc5a406e260 tx=0x7fc5a0009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60bbeae71f87c73e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:45.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.346+0000 7fc5a9288700 1 -- 172.21.15.78:0/3466742193 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0011830 con 0x7fc5a415d050 2023-10-11T09:40:45.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.346+0000 7fc5a9288700 1 -- 172.21.15.78:0/3466742193 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a0011990 con 0x7fc5a415d050 2023-10-11T09:40:45.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.346+0000 7fc5a9288700 1 -- 172.21.15.78:0/3466742193 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0011d20 con 0x7fc5a415d050 2023-10-11T09:40:45.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.346+0000 7fc5ab28c700 1 -- 172.21.15.78:0/3466742193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5a415d050 msgr2=0x7fc5a40cc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:45.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.346+0000 7fc5ab28c700 1 --2- 172.21.15.78:0/3466742193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5a415d050 0x7fc5a40cc820 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc5a406e260 tx=0x7fc5a0009b30 comp rx=0 tx=0).stop 2023-10-11T09:40:45.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.346+0000 7fc5ab28c700 1 -- 172.21.15.78:0/3466742193 shutdown_connections 2023-10-11T09:40:45.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.346+0000 7fc5ab28c700 1 --2- 172.21.15.78:0/3466742193 >> v2:172.21.15.133:3300/0 conn(0x7fc5a4160c60 0x7fc5a415d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:45.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.346+0000 7fc5ab28c700 1 --2- 172.21.15.78:0/3466742193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5a415d050 0x7fc5a40cc820 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:45.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.346+0000 7fc5ab28c700 1 -- 172.21.15.78:0/3466742193 >> 172.21.15.78:0/3466742193 conn(0x7fc5a40cea60 msgr2=0x7fc5a40cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:45.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.346+0000 7fc5ab28c700 1 -- 172.21.15.78:0/3466742193 shutdown_connections 2023-10-11T09:40:45.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.346+0000 7fc5ab28c700 1 -- 172.21.15.78:0/3466742193 wait complete. 2023-10-11T09:40:45.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.350+0000 7fc5ab28c700 1 Processor -- start 2023-10-11T09:40:45.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.350+0000 7fc5ab28c700 1 -- start start 2023-10-11T09:40:45.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.350+0000 7fc5ab28c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5a415d050 0x7fc5a41526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:45.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.350+0000 7fc5ab28c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc5a415d960 0x7fc5a41f5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:45.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.350+0000 7fc5aa28a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5a415d050 0x7fc5a41526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:45.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.350+0000 7fc5aa28a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5a415d050 0x7fc5a41526e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56736/0 (socket says 172.21.15.78:56736) 2023-10-11T09:40:45.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.350+0000 7fc5ab28c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5a4160c60 0x7fc5a4151e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:45.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.350+0000 7fc5ab28c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a41523a0 con 0x7fc5a415d050 2023-10-11T09:40:45.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.354+0000 7fc5ab28c700 1 -- 172.21.15.78:0/499641379 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a4152de0 con 0x7fc5a4160c60 2023-10-11T09:40:45.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.354+0000 7fc5aa28a700 1 -- 172.21.15.78:0/499641379 learned_addr learned my addr 172.21.15.78:0/499641379 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:45.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.354+0000 7fc5ab28c700 1 -- 172.21.15.78:0/499641379 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a41530c0 con 0x7fc5a415d960 2023-10-11T09:40:45.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.354+0000 7fc5a9a89700 1 --2- 172.21.15.78:0/499641379 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc5a415d960 0x7fc5a41f5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:45.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.354+0000 7fc5aa28a700 1 -- 172.21.15.78:0/499641379 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc5a415d960 msgr2=0x7fc5a41f5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:45.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.354+0000 7fc5aa28a700 1 --2- 172.21.15.78:0/499641379 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc5a415d960 0x7fc5a41f5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:45.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.354+0000 7fc5aa28a700 1 -- 172.21.15.78:0/499641379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5a4160c60 msgr2=0x7fc5a4151e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:45.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.354+0000 7fc5aa28a700 1 --2- 172.21.15.78:0/499641379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5a4160c60 0x7fc5a4151e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:45.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.354+0000 7fc5aa28a700 1 -- 172.21.15.78:0/499641379 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a00094b0 con 0x7fc5a415d050 2023-10-11T09:40:45.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.354+0000 7fc5aa28a700 1 --2- 172.21.15.78:0/499641379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5a415d050 0x7fc5a41526e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0014040 tx=0x7fc5a0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:45.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.354+0000 7fc59affd700 1 -- 172.21.15.78:0/499641379 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a00036c0 con 0x7fc5a415d050 2023-10-11T09:40:45.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.354+0000 7fc59affd700 1 -- 172.21.15.78:0/499641379 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a001e070 con 0x7fc5a415d050 2023-10-11T09:40:45.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.354+0000 7fc5ab28c700 1 -- 172.21.15.78:0/499641379 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a41533a0 con 0x7fc5a415d050 2023-10-11T09:40:45.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.354+0000 7fc59affd700 1 -- 172.21.15.78:0/499641379 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0003a40 con 0x7fc5a415d050 2023-10-11T09:40:45.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.354+0000 7fc5ab28c700 1 -- 172.21.15.78:0/499641379 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a4153960 con 0x7fc5a415d050 2023-10-11T09:40:45.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.354+0000 7fc5ab28c700 1 -- 172.21.15.78:0/499641379 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a4065d40 con 0x7fc5a415d050 2023-10-11T09:40:45.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.358+0000 7fc59affd700 1 -- 172.21.15.78:0/499641379 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fc5a0003ba0 con 0x7fc5a415d050 2023-10-11T09:40:45.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.358+0000 7fc59affd700 1 --2- 172.21.15.78:0/499641379 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc588065ed0 0x7fc588068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:45.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.358+0000 7fc59affd700 1 -- 172.21.15.78:0/499641379 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc5a0084900 con 0x7fc5a415d050 2023-10-11T09:40:45.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.358+0000 7fc5a9a89700 1 --2- 172.21.15.78:0/499641379 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc588065ed0 0x7fc588068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:45.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.358+0000 7fc5a9a89700 1 --2- 172.21.15.78:0/499641379 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc588065ed0 0x7fc588068380 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4140f10 tx=0x7fc594008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:45.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.366+0000 7fc59affd700 1 -- 172.21.15.78:0/499641379 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fc5a0051f40 con 0x7fc5a415d050 2023-10-11T09:40:45.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.574+0000 7fc5ab28c700 1 -- 172.21.15.78:0/499641379 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5a40b1310 con 0x7fc5a415d050 2023-10-11T09:40:45.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.574+0000 7fc59affd700 1 -- 172.21.15.78:0/499641379 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4648 (secure 0 0 0) 0x7fc5a0055560 con 0x7fc5a415d050 2023-10-11T09:40:45.581 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:40:45.581 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/4282378510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4282378510},{"type":"v1","addr":"172.21.15.133:6835","nonce":4282378510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3986631249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3986631249},{"type":"v1","addr":"172.21.15.78:6835","nonce":3986631249}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3135421902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3135421902},{"type":"v1","addr":"172.21.15.78:6837","nonce":3135421902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:40:41.673556+0000","modified":"2023-10-11T09:40:45.122852+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6834/3925157967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":3925157967},{"type":"v1","addr":"172.21.15.133:6834","nonce":3925157967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-11T09:40:45.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.578+0000 7fc5ab28c700 1 -- 172.21.15.78:0/499641379 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc588065ed0 msgr2=0x7fc588068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:45.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.578+0000 7fc5ab28c700 1 --2- 172.21.15.78:0/499641379 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc588065ed0 0x7fc588068380 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4140f10 tx=0x7fc594008040 comp rx=0 tx=0).stop 2023-10-11T09:40:45.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.578+0000 7fc5ab28c700 1 -- 172.21.15.78:0/499641379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5a415d050 msgr2=0x7fc5a41526e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:45.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.578+0000 7fc5ab28c700 1 --2- 172.21.15.78:0/499641379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5a415d050 0x7fc5a41526e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0014040 tx=0x7fc5a0004770 comp rx=0 tx=0).stop 2023-10-11T09:40:45.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.578+0000 7fc5ab28c700 1 -- 172.21.15.78:0/499641379 shutdown_connections 2023-10-11T09:40:45.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.578+0000 7fc5ab28c700 1 --2- 172.21.15.78:0/499641379 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc588065ed0 0x7fc588068380 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:45.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.578+0000 7fc5ab28c700 1 --2- 172.21.15.78:0/499641379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5a4160c60 0x7fc5a4151e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:45.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.578+0000 7fc5ab28c700 1 --2- 172.21.15.78:0/499641379 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc5a415d960 0x7fc5a41f5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:45.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.578+0000 7fc5ab28c700 1 --2- 172.21.15.78:0/499641379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5a415d050 0x7fc5a41526e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:45.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.578+0000 7fc5ab28c700 1 -- 172.21.15.78:0/499641379 >> 172.21.15.78:0/499641379 conn(0x7fc5a40cea60 msgr2=0x7fc5a415de60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:45.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.578+0000 7fc5ab28c700 1 -- 172.21.15.78:0/499641379 shutdown_connections 2023-10-11T09:40:45.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.578+0000 7fc5ab28c700 1 -- 172.21.15.78:0/499641379 wait complete. 2023-10-11T09:40:45.587 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 8 2023-10-11T09:40:45.605 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:40:45.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.766+0000 7feca105e700 1 Processor -- start 2023-10-11T09:40:45.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.766+0000 7feca105e700 1 -- start start 2023-10-11T09:40:45.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.770+0000 7feca105e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fec9c158b90 0x7fec9c0cb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:45.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.770+0000 7feca105e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fec9c15d260 0x7fec9c06c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:45.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.770+0000 7feca105e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec9c0cbe40 con 0x7fec9c06cbe0 2023-10-11T09:40:45.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.770+0000 7feca105e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec9c06d060 con 0x7fec9c158b90 2023-10-11T09:40:45.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.770+0000 7feca105e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c15d820 con 0x7fec9c15d260 2023-10-11T09:40:45.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.770+0000 7fec9b59e700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fec9c06cbe0 0x7fec9c0730c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:58680/0 (socket says 172.21.15.78:58680) 2023-10-11T09:40:45.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.770+0000 7fec9b59e700 1 -- 172.21.15.78:0/4219745870 learned_addr learned my addr 172.21.15.78:0/4219745870 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:45.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.770+0000 7fec9ad9d700 1 --2- 172.21.15.78:0/4219745870 >> v2:172.21.15.133:3300/0 conn(0x7fec9c158b90 0x7fec9c0cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:45.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.770+0000 7fec93fff700 1 --2- 172.21.15.78:0/4219745870 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fec9c15d260 0x7fec9c06c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:45.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.770+0000 7fec9a59c700 1 -- 172.21.15.78:0/4219745870 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 131801005 0 0) 0x7fec94001490 con 0x7fec9c06cbe0 2023-10-11T09:40:45.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.770+0000 7fec9a59c700 1 -- 172.21.15.78:0/4219745870 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec880035d0 con 0x7fec9c06cbe0 2023-10-11T09:40:45.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.770+0000 7fec93fff700 1 -- 172.21.15.78:0/4219745870 >> v2:172.21.15.133:3300/0 conn(0x7fec9c158b90 msgr2=0x7fec9c0cb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:45.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.770+0000 7fec93fff700 1 --2- 172.21.15.78:0/4219745870 >> v2:172.21.15.133:3300/0 conn(0x7fec9c158b90 0x7fec9c0cb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:45.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.770+0000 7fec93fff700 1 -- 172.21.15.78:0/4219745870 >> v1:172.21.15.78:6789/0 conn(0x7fec9c06cbe0 legacy=0x7fec9c0730c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:45.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.770+0000 7fec93fff700 1 -- 172.21.15.78:0/4219745870 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec9c0cd2d0 con 0x7fec9c15d260 2023-10-11T09:40:45.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.774+0000 7fec93fff700 1 --2- 172.21.15.78:0/4219745870 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fec9c15d260 0x7fec9c06c6a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fec84009850 tx=0x7fec84009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bb7005e4816e54f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:45.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.774+0000 7fec9a59c700 1 -- 172.21.15.78:0/4219745870 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec84011830 con 0x7fec9c15d260 2023-10-11T09:40:45.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.774+0000 7fec9a59c700 1 -- 172.21.15.78:0/4219745870 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec84011990 con 0x7fec9c15d260 2023-10-11T09:40:45.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.774+0000 7fec9a59c700 1 -- 172.21.15.78:0/4219745870 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec84011d40 con 0x7fec9c15d260 2023-10-11T09:40:45.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.774+0000 7feca105e700 1 -- 172.21.15.78:0/4219745870 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fec9c15d260 msgr2=0x7fec9c06c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:45.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.774+0000 7feca105e700 1 --2- 172.21.15.78:0/4219745870 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fec9c15d260 0x7fec9c06c6a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fec84009850 tx=0x7fec84009b60 comp rx=0 tx=0).stop 2023-10-11T09:40:45.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.774+0000 7feca105e700 1 -- 172.21.15.78:0/4219745870 shutdown_connections 2023-10-11T09:40:45.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.774+0000 7feca105e700 1 --2- 172.21.15.78:0/4219745870 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fec9c15d260 0x7fec9c06c6a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:45.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.774+0000 7feca105e700 1 --2- 172.21.15.78:0/4219745870 >> v2:172.21.15.133:3300/0 conn(0x7fec9c158b90 0x7fec9c0cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:45.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.774+0000 7feca105e700 1 -- 172.21.15.78:0/4219745870 >> 172.21.15.78:0/4219745870 conn(0x7fec9c0b1d50 msgr2=0x7fec9c0cf5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:45.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.774+0000 7feca105e700 1 -- 172.21.15.78:0/4219745870 shutdown_connections 2023-10-11T09:40:45.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.774+0000 7feca105e700 1 -- 172.21.15.78:0/4219745870 wait complete. 2023-10-11T09:40:45.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.778+0000 7feca105e700 1 Processor -- start 2023-10-11T09:40:45.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.778+0000 7feca105e700 1 -- start start 2023-10-11T09:40:45.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.778+0000 7feca105e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fec9c06cbe0 0x7fec9c14d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:45.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.778+0000 7feca105e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec9c158b90 0x7fec9c070680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:45.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.778+0000 7feca105e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec9c15d260 0x7fec9c157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:45.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.778+0000 7feca105e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c14daa0 con 0x7fec9c158b90 2023-10-11T09:40:45.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.778+0000 7feca105e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c14d750 con 0x7fec9c15d260 2023-10-11T09:40:45.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.778+0000 7feca105e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c0ad5a0 con 0x7fec9c06cbe0 2023-10-11T09:40:45.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.778+0000 7fec93fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec9c158b90 0x7fec9c070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:45.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.778+0000 7fec93fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec9c158b90 0x7fec9c070680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56750/0 (socket says 172.21.15.78:56750) 2023-10-11T09:40:45.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.778+0000 7fec93fff700 1 -- 172.21.15.78:0/3040181500 learned_addr learned my addr 172.21.15.78:0/3040181500 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:45.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.778+0000 7fec9ad9d700 1 --2- 172.21.15.78:0/3040181500 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fec9c06cbe0 0x7fec9c14d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:45.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.778+0000 7fec9b59e700 1 --2- 172.21.15.78:0/3040181500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec9c15d260 0x7fec9c157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:45.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.778+0000 7fec93fff700 1 -- 172.21.15.78:0/3040181500 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fec9c06cbe0 msgr2=0x7fec9c14d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:45.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.778+0000 7fec93fff700 1 --2- 172.21.15.78:0/3040181500 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fec9c06cbe0 0x7fec9c14d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:45.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.778+0000 7fec93fff700 1 -- 172.21.15.78:0/3040181500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec9c15d260 msgr2=0x7fec9c157400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:45.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.778+0000 7fec93fff700 1 --2- 172.21.15.78:0/3040181500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec9c15d260 0x7fec9c157400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:45.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.778+0000 7fec93fff700 1 -- 172.21.15.78:0/3040181500 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec840094b0 con 0x7fec9c158b90 2023-10-11T09:40:45.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.778+0000 7fec9b59e700 1 --2- 172.21.15.78:0/3040181500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec9c15d260 0x7fec9c157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:40:45.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.782+0000 7fec93fff700 1 --2- 172.21.15.78:0/3040181500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec9c158b90 0x7fec9c070680 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fec84014040 tx=0x7fec84004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:45.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.782+0000 7fec937fe700 1 -- 172.21.15.78:0/3040181500 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec84004590 con 0x7fec9c158b90 2023-10-11T09:40:45.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.782+0000 7fec937fe700 1 -- 172.21.15.78:0/3040181500 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec8401e070 con 0x7fec9c158b90 2023-10-11T09:40:45.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.782+0000 7feca105e700 1 -- 172.21.15.78:0/3040181500 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec9c0ad880 con 0x7fec9c158b90 2023-10-11T09:40:45.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.782+0000 7fec937fe700 1 -- 172.21.15.78:0/3040181500 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec84003900 con 0x7fec9c158b90 2023-10-11T09:40:45.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.782+0000 7feca105e700 1 -- 172.21.15.78:0/3040181500 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec9c070bc0 con 0x7fec9c158b90 2023-10-11T09:40:45.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.782+0000 7feca105e700 1 -- 172.21.15.78:0/3040181500 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec9c14dc50 con 0x7fec9c158b90 2023-10-11T09:40:45.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.786+0000 7fec937fe700 1 -- 172.21.15.78:0/3040181500 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fec84003a60 con 0x7fec9c158b90 2023-10-11T09:40:45.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.786+0000 7fec937fe700 1 --2- 172.21.15.78:0/3040181500 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fec7c065f00 0x7fec7c0683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:45.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.786+0000 7fec937fe700 1 -- 172.21.15.78:0/3040181500 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fec84020070 con 0x7fec9c158b90 2023-10-11T09:40:45.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.786+0000 7fec9ad9d700 1 --2- 172.21.15.78:0/3040181500 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fec7c065f00 0x7fec7c0683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:45.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.786+0000 7fec9ad9d700 1 --2- 172.21.15.78:0/3040181500 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fec7c065f00 0x7fec7c0683b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fec8c00b240 tx=0x7fec8c00b450 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:45.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.790+0000 7fec937fe700 1 -- 172.21.15.78:0/3040181500 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fec9c14dc50 con 0x7fec9c158b90 2023-10-11T09:40:45.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.962+0000 7feca105e700 1 -- 172.21.15.78:0/3040181500 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec9c04e530 con 0x7fec9c158b90 2023-10-11T09:40:45.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.966+0000 7fec937fe700 1 -- 172.21.15.78:0/3040181500 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17811 (secure 0 0 0) 0x7fec8402a090 con 0x7fec9c158b90 2023-10-11T09:40:45.970 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:40:45.971 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":23,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:40:42.678477+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:40:39.862581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T09:40:40.663641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:40:45.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.970+0000 7feca105e700 1 -- 172.21.15.78:0/3040181500 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fec7c065f00 msgr2=0x7fec7c0683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:45.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.970+0000 7feca105e700 1 --2- 172.21.15.78:0/3040181500 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fec7c065f00 0x7fec7c0683b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fec8c00b240 tx=0x7fec8c00b450 comp rx=0 tx=0).stop 2023-10-11T09:40:45.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.970+0000 7feca105e700 1 -- 172.21.15.78:0/3040181500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec9c158b90 msgr2=0x7fec9c070680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:45.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.970+0000 7feca105e700 1 --2- 172.21.15.78:0/3040181500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec9c158b90 0x7fec9c070680 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fec84014040 tx=0x7fec84004770 comp rx=0 tx=0).stop 2023-10-11T09:40:45.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.970+0000 7feca105e700 1 -- 172.21.15.78:0/3040181500 shutdown_connections 2023-10-11T09:40:45.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.970+0000 7feca105e700 1 --2- 172.21.15.78:0/3040181500 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fec7c065f00 0x7fec7c0683b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:45.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.970+0000 7feca105e700 1 --2- 172.21.15.78:0/3040181500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec9c15d260 0x7fec9c157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:45.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.970+0000 7feca105e700 1 --2- 172.21.15.78:0/3040181500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec9c158b90 0x7fec9c070680 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:45.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.970+0000 7feca105e700 1 --2- 172.21.15.78:0/3040181500 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fec9c06cbe0 0x7fec9c14d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:45.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.970+0000 7feca105e700 1 -- 172.21.15.78:0/3040181500 >> 172.21.15.78:0/3040181500 conn(0x7fec9c0b1d50 msgr2=0x7fec9c0b0640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:45.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.970+0000 7feca105e700 1 -- 172.21.15.78:0/3040181500 shutdown_connections 2023-10-11T09:40:45.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:45.970+0000 7feca105e700 1 -- 172.21.15.78:0/3040181500 wait complete. 2023-10-11T09:40:45.999 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-11T09:40:46.000 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-11T09:40:46.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.166+0000 7fe01cea7700 1 Processor -- start 2023-10-11T09:40:46.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.166+0000 7fe01cea7700 1 -- start start 2023-10-11T09:40:46.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.170+0000 7fe01cea7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe018160d40 0x7fe0180cd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:46.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.170+0000 7fe01cea7700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe0180ce500 0x7fe01806be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:46.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.170+0000 7fe01cea7700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe01806ffa0 con 0x7fe018071040 2023-10-11T09:40:46.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.170+0000 7fe01cea7700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe018070c20 con 0x7fe0180ce500 2023-10-11T09:40:46.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.170+0000 7fe01cea7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe018161110 con 0x7fe018160d40 2023-10-11T09:40:46.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.170+0000 7fe016d9d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe0180ce500 0x7fe01806be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:46.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.170+0000 7fe016d9d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe0180ce500 0x7fe01806be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56762/0 (socket says 172.21.15.78:56762) 2023-10-11T09:40:46.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.170+0000 7fe016d9d700 1 -- 172.21.15.78:0/3106213877 learned_addr learned my addr 172.21.15.78:0/3106213877 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:46.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.170+0000 7fe01659c700 1 --2- 172.21.15.78:0/3106213877 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe018160d40 0x7fe0180cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:46.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.170+0000 7fe016d9d700 1 -- 172.21.15.78:0/3106213877 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe018160d40 msgr2=0x7fe0180cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:46.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.170+0000 7fe016d9d700 1 --2- 172.21.15.78:0/3106213877 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe018160d40 0x7fe0180cd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:46.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.170+0000 7fe016d9d700 1 -- 172.21.15.78:0/3106213877 >> v1:172.21.15.133:6789/0 conn(0x7fe018071040 legacy=0x7fe0180cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:46.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.170+0000 7fe016d9d700 1 -- 172.21.15.78:0/3106213877 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe018070280 con 0x7fe0180ce500 2023-10-11T09:40:46.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.170+0000 7fe01659c700 1 --2- 172.21.15.78:0/3106213877 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe018160d40 0x7fe0180cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:40:46.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.170+0000 7fe016d9d700 1 --2- 172.21.15.78:0/3106213877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0180ce500 0x7fe01806be60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe010009e50 tx=0x7fe01000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a2c3a78b2b52e7a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:46.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.170+0000 7fe015d9b700 1 -- 172.21.15.78:0/3106213877 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01000e070 con 0x7fe0180ce500 2023-10-11T09:40:46.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.170+0000 7fe015d9b700 1 -- 172.21.15.78:0/3106213877 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01000b9c0 con 0x7fe0180ce500 2023-10-11T09:40:46.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.170+0000 7fe015d9b700 1 -- 172.21.15.78:0/3106213877 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01000bd70 con 0x7fe0180ce500 2023-10-11T09:40:46.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.170+0000 7fe01cea7700 1 -- 172.21.15.78:0/3106213877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0180ce500 msgr2=0x7fe01806be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:46.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.170+0000 7fe01cea7700 1 --2- 172.21.15.78:0/3106213877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0180ce500 0x7fe01806be60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe010009e50 tx=0x7fe01000b330 comp rx=0 tx=0).stop 2023-10-11T09:40:46.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.174+0000 7fe01cea7700 1 -- 172.21.15.78:0/3106213877 shutdown_connections 2023-10-11T09:40:46.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.174+0000 7fe01cea7700 1 --2- 172.21.15.78:0/3106213877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0180ce500 0x7fe01806be60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:46.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.174+0000 7fe01cea7700 1 --2- 172.21.15.78:0/3106213877 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe018160d40 0x7fe0180cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:46.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.174+0000 7fe01cea7700 1 -- 172.21.15.78:0/3106213877 >> 172.21.15.78:0/3106213877 conn(0x7fe01806aca0 msgr2=0x7fe0180b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:46.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.174+0000 7fe01cea7700 1 -- 172.21.15.78:0/3106213877 shutdown_connections 2023-10-11T09:40:46.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.174+0000 7fe01cea7700 1 -- 172.21.15.78:0/3106213877 wait complete. 2023-10-11T09:40:46.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.178+0000 7fe01cea7700 1 Processor -- start 2023-10-11T09:40:46.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.178+0000 7fe01cea7700 1 -- start start 2023-10-11T09:40:46.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.178+0000 7fe01cea7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe018071040 0x7fe01814ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:46.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.178+0000 7fe01cea7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe0180ce500 0x7fe0180c3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:46.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.178+0000 7fe01cea7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe018160d40 0x7fe0180ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:46.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.178+0000 7fe01cea7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0180b3890 con 0x7fe018160d40 2023-10-11T09:40:46.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.178+0000 7fe01cea7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0181f6970 con 0x7fe018071040 2023-10-11T09:40:46.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.178+0000 7fe01cea7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0181f6c50 con 0x7fe0180ce500 2023-10-11T09:40:46.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.178+0000 7fe016d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe018160d40 0x7fe0180ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:46.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.178+0000 7fe016d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe018160d40 0x7fe0180ceca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56772/0 (socket says 172.21.15.78:56772) 2023-10-11T09:40:46.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.178+0000 7fe016d9d700 1 -- 172.21.15.78:0/2590048329 learned_addr learned my addr 172.21.15.78:0/2590048329 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:46.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.178+0000 7fe01659c700 1 --2- 172.21.15.78:0/2590048329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe018071040 0x7fe01814ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:46.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.178+0000 7fe00dd9b700 1 --2- 172.21.15.78:0/2590048329 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe0180ce500 0x7fe0180c3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:46.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.178+0000 7fe016d9d700 1 -- 172.21.15.78:0/2590048329 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe0180ce500 msgr2=0x7fe0180c3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:46.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.178+0000 7fe016d9d700 1 --2- 172.21.15.78:0/2590048329 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe0180ce500 0x7fe0180c3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:46.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.178+0000 7fe016d9d700 1 -- 172.21.15.78:0/2590048329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe018071040 msgr2=0x7fe01814ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:46.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.178+0000 7fe016d9d700 1 --2- 172.21.15.78:0/2590048329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe018071040 0x7fe01814ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:46.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.178+0000 7fe016d9d700 1 -- 172.21.15.78:0/2590048329 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe010009b30 con 0x7fe018160d40 2023-10-11T09:40:46.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.178+0000 7fe00dd9b700 1 --2- 172.21.15.78:0/2590048329 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe0180ce500 0x7fe0180c3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:40:46.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.178+0000 7fe016d9d700 1 --2- 172.21.15.78:0/2590048329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe018160d40 0x7fe0180ceca0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe010012000 tx=0x7fe010004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:46.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.182+0000 7fe00f7fe700 1 -- 172.21.15.78:0/2590048329 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0100048b0 con 0x7fe018160d40 2023-10-11T09:40:46.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.182+0000 7fe00f7fe700 1 -- 172.21.15.78:0/2590048329 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0100099b0 con 0x7fe018160d40 2023-10-11T09:40:46.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.182+0000 7fe01cea7700 1 -- 172.21.15.78:0/2590048329 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0181f6f30 con 0x7fe018160d40 2023-10-11T09:40:46.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.182+0000 7fe00f7fe700 1 -- 172.21.15.78:0/2590048329 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01001c680 con 0x7fe018160d40 2023-10-11T09:40:46.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.182+0000 7fe01cea7700 1 -- 172.21.15.78:0/2590048329 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe01814b3c0 con 0x7fe018160d40 2023-10-11T09:40:46.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.182+0000 7fe01cea7700 1 -- 172.21.15.78:0/2590048329 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe018072250 con 0x7fe018160d40 2023-10-11T09:40:46.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.182+0000 7fe00f7fe700 1 -- 172.21.15.78:0/2590048329 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fe01001c7e0 con 0x7fe018160d40 2023-10-11T09:40:46.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.186+0000 7fe00f7fe700 1 --2- 172.21.15.78:0/2590048329 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdffc065ed0 0x7fdffc068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:46.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.186+0000 7fe00f7fe700 1 -- 172.21.15.78:0/2590048329 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe0100853d0 con 0x7fe018160d40 2023-10-11T09:40:46.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.186+0000 7fe01659c700 1 --2- 172.21.15.78:0/2590048329 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdffc065ed0 0x7fdffc068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:46.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.186+0000 7fe01659c700 1 --2- 172.21.15.78:0/2590048329 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdffc065ed0 0x7fdffc068380 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe00800c960 tx=0x7fe008008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:46.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.190+0000 7fe00f7fe700 1 -- 172.21.15.78:0/2590048329 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fe010052ab0 con 0x7fe018160d40 2023-10-11T09:40:46.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.354+0000 7fe01cea7700 1 -- 172.21.15.78:0/2590048329 --> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe018070840 con 0x7fdffc065ed0 2023-10-11T09:40:46.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.374+0000 7fe00f7fe700 1 -- 172.21.15.78:0/2590048329 <== mgr.4101 v2:172.21.15.78:6832/35606 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286517 (secure 0 0 0) 0x7fe00800cdd0 con 0x7fdffc065ed0 2023-10-11T09:40:46.380 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:40:46.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.378+0000 7fe01cea7700 1 -- 172.21.15.78:0/2590048329 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdffc065ed0 msgr2=0x7fdffc068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:46.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.378+0000 7fe01cea7700 1 --2- 172.21.15.78:0/2590048329 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdffc065ed0 0x7fdffc068380 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe00800c960 tx=0x7fe008008040 comp rx=0 tx=0).stop 2023-10-11T09:40:46.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.378+0000 7fe01cea7700 1 -- 172.21.15.78:0/2590048329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe018160d40 msgr2=0x7fe0180ceca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:46.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.378+0000 7fe01cea7700 1 --2- 172.21.15.78:0/2590048329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe018160d40 0x7fe0180ceca0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe010012000 tx=0x7fe010004af0 comp rx=0 tx=0).stop 2023-10-11T09:40:46.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.378+0000 7fe01cea7700 1 -- 172.21.15.78:0/2590048329 shutdown_connections 2023-10-11T09:40:46.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.378+0000 7fe01cea7700 1 --2- 172.21.15.78:0/2590048329 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdffc065ed0 0x7fdffc068380 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:46.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.378+0000 7fe01cea7700 1 --2- 172.21.15.78:0/2590048329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe018160d40 0x7fe0180ceca0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:46.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.378+0000 7fe01cea7700 1 --2- 172.21.15.78:0/2590048329 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe0180ce500 0x7fe0180c3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:46.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.378+0000 7fe01cea7700 1 --2- 172.21.15.78:0/2590048329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe018071040 0x7fe01814ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:46.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.378+0000 7fe01cea7700 1 -- 172.21.15.78:0/2590048329 >> 172.21.15.78:0/2590048329 conn(0x7fe01806aca0 msgr2=0x7fe0180b0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:46.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.382+0000 7fe01cea7700 1 -- 172.21.15.78:0/2590048329 shutdown_connections 2023-10-11T09:40:46.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.382+0000 7fe01cea7700 1 -- 172.21.15.78:0/2590048329 wait complete. 2023-10-11T09:40:46.387 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2023-10-11T09:40:46.407 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-11T09:40:46.324161+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218680,"kb_used_data":3064,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754756040,"statfs":{"total":773094113280,"available":772870184960,"internally_reserved":0,"allocated":3137536,"data_stored":1875640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001919"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144669+0000","last_change":"2023-10-11T09:40:41.696400+0000","last_active":"2023-10-11T09:40:43.144669+0000","last_peered":"2023-10-11T09:40:43.144669+0000","last_clean":"2023-10-11T09:40:43.144669+0000","last_became_active":"2023-10-11T09:40:41.696101+0000","last_became_peered":"2023-10-11T09:40:41.696101+0000","last_unstale":"2023-10-11T09:40:43.144669+0000","last_undegraded":"2023-10-11T09:40:43.144669+0000","last_fullsized":"2023-10-11T09:40:43.144669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:43:01.331013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.694262+0000","last_change":"2023-10-11T09:40:41.719611+0000","last_active":"2023-10-11T09:40:42.694262+0000","last_peered":"2023-10-11T09:40:42.694262+0000","last_clean":"2023-10-11T09:40:42.694262+0000","last_became_active":"2023-10-11T09:40:41.719319+0000","last_became_peered":"2023-10-11T09:40:41.719319+0000","last_unstale":"2023-10-11T09:40:42.694262+0000","last_undegraded":"2023-10-11T09:40:42.694262+0000","last_fullsized":"2023-10-11T09:40:42.694262+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:07:02.302344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.912066+0000","last_change":"2023-10-11T09:40:41.701063+0000","last_active":"2023-10-11T09:40:42.912066+0000","last_peered":"2023-10-11T09:40:42.912066+0000","last_clean":"2023-10-11T09:40:42.912066+0000","last_became_active":"2023-10-11T09:40:41.699445+0000","last_became_peered":"2023-10-11T09:40:41.699445+0000","last_unstale":"2023-10-11T09:40:42.912066+0000","last_undegraded":"2023-10-11T09:40:42.912066+0000","last_fullsized":"2023-10-11T09:40:42.912066+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:19:44.476799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144161+0000","last_change":"2023-10-11T09:40:41.721932+0000","last_active":"2023-10-11T09:40:43.144161+0000","last_peered":"2023-10-11T09:40:43.144161+0000","last_clean":"2023-10-11T09:40:43.144161+0000","last_became_active":"2023-10-11T09:40:41.720602+0000","last_became_peered":"2023-10-11T09:40:41.720602+0000","last_unstale":"2023-10-11T09:40:43.144161+0000","last_undegraded":"2023-10-11T09:40:43.144161+0000","last_fullsized":"2023-10-11T09:40:43.144161+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:17:36.655491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145802+0000","last_change":"2023-10-11T09:40:41.710213+0000","last_active":"2023-10-11T09:40:43.145802+0000","last_peered":"2023-10-11T09:40:43.145802+0000","last_clean":"2023-10-11T09:40:43.145802+0000","last_became_active":"2023-10-11T09:40:41.707488+0000","last_became_peered":"2023-10-11T09:40:41.707488+0000","last_unstale":"2023-10-11T09:40:43.145802+0000","last_undegraded":"2023-10-11T09:40:43.145802+0000","last_fullsized":"2023-10-11T09:40:43.145802+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:42:34.821513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911583+0000","last_change":"2023-10-11T09:40:41.700595+0000","last_active":"2023-10-11T09:40:42.911583+0000","last_peered":"2023-10-11T09:40:42.911583+0000","last_clean":"2023-10-11T09:40:42.911583+0000","last_became_active":"2023-10-11T09:40:41.700239+0000","last_became_peered":"2023-10-11T09:40:41.700239+0000","last_unstale":"2023-10-11T09:40:42.911583+0000","last_undegraded":"2023-10-11T09:40:42.911583+0000","last_fullsized":"2023-10-11T09:40:42.911583+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:48:14.871150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.691894+0000","last_change":"2023-10-11T09:40:41.712870+0000","last_active":"2023-10-11T09:40:42.691894+0000","last_peered":"2023-10-11T09:40:42.691894+0000","last_clean":"2023-10-11T09:40:42.691894+0000","last_became_active":"2023-10-11T09:40:41.712593+0000","last_became_peered":"2023-10-11T09:40:41.712593+0000","last_unstale":"2023-10-11T09:40:42.691894+0000","last_undegraded":"2023-10-11T09:40:42.691894+0000","last_fullsized":"2023-10-11T09:40:42.691894+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:50:15.829941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.693734+0000","last_change":"2023-10-11T09:40:41.714890+0000","last_active":"2023-10-11T09:40:42.693734+0000","last_peered":"2023-10-11T09:40:42.693734+0000","last_clean":"2023-10-11T09:40:42.693734+0000","last_became_active":"2023-10-11T09:40:41.711116+0000","last_became_peered":"2023-10-11T09:40:41.711116+0000","last_unstale":"2023-10-11T09:40:42.693734+0000","last_undegraded":"2023-10-11T09:40:42.693734+0000","last_fullsized":"2023-10-11T09:40:42.693734+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:00:05.422788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.146063+0000","last_change":"2023-10-11T09:40:41.700768+0000","last_active":"2023-10-11T09:40:43.146063+0000","last_peered":"2023-10-11T09:40:43.146063+0000","last_clean":"2023-10-11T09:40:43.146063+0000","last_became_active":"2023-10-11T09:40:41.700440+0000","last_became_peered":"2023-10-11T09:40:41.700440+0000","last_unstale":"2023-10-11T09:40:43.146063+0000","last_undegraded":"2023-10-11T09:40:43.146063+0000","last_fullsized":"2023-10-11T09:40:43.146063+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:56:50.343276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.694372+0000","last_change":"2023-10-11T09:40:41.712701+0000","last_active":"2023-10-11T09:40:42.694372+0000","last_peered":"2023-10-11T09:40:42.694372+0000","last_clean":"2023-10-11T09:40:42.694372+0000","last_became_active":"2023-10-11T09:40:41.712349+0000","last_became_peered":"2023-10-11T09:40:41.712349+0000","last_unstale":"2023-10-11T09:40:42.694372+0000","last_undegraded":"2023-10-11T09:40:42.694372+0000","last_fullsized":"2023-10-11T09:40:42.694372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:42:34.821513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.912276+0000","last_change":"2023-10-11T09:40:41.706310+0000","last_active":"2023-10-11T09:40:42.912276+0000","last_peered":"2023-10-11T09:40:42.912276+0000","last_clean":"2023-10-11T09:40:42.912276+0000","last_became_active":"2023-10-11T09:40:41.705986+0000","last_became_peered":"2023-10-11T09:40:41.705986+0000","last_unstale":"2023-10-11T09:40:42.912276+0000","last_undegraded":"2023-10-11T09:40:42.912276+0000","last_fullsized":"2023-10-11T09:40:42.912276+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:59:46.864358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144038+0000","last_change":"2023-10-11T09:40:41.700251+0000","last_active":"2023-10-11T09:40:43.144038+0000","last_peered":"2023-10-11T09:40:43.144038+0000","last_clean":"2023-10-11T09:40:43.144038+0000","last_became_active":"2023-10-11T09:40:41.699961+0000","last_became_peered":"2023-10-11T09:40:41.699961+0000","last_unstale":"2023-10-11T09:40:43.144038+0000","last_undegraded":"2023-10-11T09:40:43.144038+0000","last_fullsized":"2023-10-11T09:40:43.144038+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:12:05.737223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.143594+0000","last_change":"2023-10-11T09:40:41.697448+0000","last_active":"2023-10-11T09:40:43.143594+0000","last_peered":"2023-10-11T09:40:43.143594+0000","last_clean":"2023-10-11T09:40:43.143594+0000","last_became_active":"2023-10-11T09:40:41.697160+0000","last_became_peered":"2023-10-11T09:40:41.697160+0000","last_unstale":"2023-10-11T09:40:43.143594+0000","last_undegraded":"2023-10-11T09:40:43.143594+0000","last_fullsized":"2023-10-11T09:40:43.143594+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:00:57.063025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.693705+0000","last_change":"2023-10-11T09:40:41.717312+0000","last_active":"2023-10-11T09:40:42.693705+0000","last_peered":"2023-10-11T09:40:42.693705+0000","last_clean":"2023-10-11T09:40:42.693705+0000","last_became_active":"2023-10-11T09:40:41.716622+0000","last_became_peered":"2023-10-11T09:40:41.716622+0000","last_unstale":"2023-10-11T09:40:42.693705+0000","last_undegraded":"2023-10-11T09:40:42.693705+0000","last_fullsized":"2023-10-11T09:40:42.693705+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:35:04.646798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.692225+0000","last_change":"2023-10-11T09:40:41.715325+0000","last_active":"2023-10-11T09:40:42.692225+0000","last_peered":"2023-10-11T09:40:42.692225+0000","last_clean":"2023-10-11T09:40:42.692225+0000","last_became_active":"2023-10-11T09:40:41.715033+0000","last_became_peered":"2023-10-11T09:40:41.715033+0000","last_unstale":"2023-10-11T09:40:42.692225+0000","last_undegraded":"2023-10-11T09:40:42.692225+0000","last_fullsized":"2023-10-11T09:40:42.692225+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:35:36.436236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911235+0000","last_change":"2023-10-11T09:40:41.699851+0000","last_active":"2023-10-11T09:40:42.911235+0000","last_peered":"2023-10-11T09:40:42.911235+0000","last_clean":"2023-10-11T09:40:42.911235+0000","last_became_active":"2023-10-11T09:40:41.699530+0000","last_became_peered":"2023-10-11T09:40:41.699530+0000","last_unstale":"2023-10-11T09:40:42.911235+0000","last_undegraded":"2023-10-11T09:40:42.911235+0000","last_fullsized":"2023-10-11T09:40:42.911235+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:23:30.814623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.146112+0000","last_change":"2023-10-11T09:40:41.700611+0000","last_active":"2023-10-11T09:40:43.146112+0000","last_peered":"2023-10-11T09:40:43.146112+0000","last_clean":"2023-10-11T09:40:43.146112+0000","last_became_active":"2023-10-11T09:40:41.700354+0000","last_became_peered":"2023-10-11T09:40:41.700354+0000","last_unstale":"2023-10-11T09:40:43.146112+0000","last_undegraded":"2023-10-11T09:40:43.146112+0000","last_fullsized":"2023-10-11T09:40:43.146112+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:48:15.689750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.912114+0000","last_change":"2023-10-11T09:40:41.700742+0000","last_active":"2023-10-11T09:40:42.912114+0000","last_peered":"2023-10-11T09:40:42.912114+0000","last_clean":"2023-10-11T09:40:42.912114+0000","last_became_active":"2023-10-11T09:40:41.700432+0000","last_became_peered":"2023-10-11T09:40:41.700432+0000","last_unstale":"2023-10-11T09:40:42.912114+0000","last_undegraded":"2023-10-11T09:40:42.912114+0000","last_fullsized":"2023-10-11T09:40:42.912114+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:33:13.162340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.143660+0000","last_change":"2023-10-11T09:40:41.719737+0000","last_active":"2023-10-11T09:40:43.143660+0000","last_peered":"2023-10-11T09:40:43.143660+0000","last_clean":"2023-10-11T09:40:43.143660+0000","last_became_active":"2023-10-11T09:40:41.719484+0000","last_became_peered":"2023-10-11T09:40:41.719484+0000","last_unstale":"2023-10-11T09:40:43.143660+0000","last_undegraded":"2023-10-11T09:40:43.143660+0000","last_fullsized":"2023-10-11T09:40:43.143660+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:48:52.897052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.912175+0000","last_change":"2023-10-11T09:40:41.728139+0000","last_active":"2023-10-11T09:40:42.912175+0000","last_peered":"2023-10-11T09:40:42.912175+0000","last_clean":"2023-10-11T09:40:42.912175+0000","last_became_active":"2023-10-11T09:40:41.727790+0000","last_became_peered":"2023-10-11T09:40:41.727790+0000","last_unstale":"2023-10-11T09:40:42.912175+0000","last_undegraded":"2023-10-11T09:40:42.912175+0000","last_fullsized":"2023-10-11T09:40:42.912175+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:07:58.571258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.692070+0000","last_change":"2023-10-11T09:40:41.713375+0000","last_active":"2023-10-11T09:40:42.692070+0000","last_peered":"2023-10-11T09:40:42.692070+0000","last_clean":"2023-10-11T09:40:42.692070+0000","last_became_active":"2023-10-11T09:40:41.713114+0000","last_became_peered":"2023-10-11T09:40:41.713114+0000","last_unstale":"2023-10-11T09:40:42.692070+0000","last_undegraded":"2023-10-11T09:40:42.692070+0000","last_fullsized":"2023-10-11T09:40:42.692070+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:07:02.302344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145102+0000","last_change":"2023-10-11T09:40:41.718540+0000","last_active":"2023-10-11T09:40:43.145102+0000","last_peered":"2023-10-11T09:40:43.145102+0000","last_clean":"2023-10-11T09:40:43.145102+0000","last_became_active":"2023-10-11T09:40:41.718309+0000","last_became_peered":"2023-10-11T09:40:41.718309+0000","last_unstale":"2023-10-11T09:40:43.145102+0000","last_undegraded":"2023-10-11T09:40:43.145102+0000","last_fullsized":"2023-10-11T09:40:43.145102+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:09:27.825336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687447+0000","last_change":"2023-10-11T09:40:41.699075+0000","last_active":"2023-10-11T09:40:42.687447+0000","last_peered":"2023-10-11T09:40:42.687447+0000","last_clean":"2023-10-11T09:40:42.687447+0000","last_became_active":"2023-10-11T09:40:41.698747+0000","last_became_peered":"2023-10-11T09:40:41.698747+0000","last_unstale":"2023-10-11T09:40:42.687447+0000","last_undegraded":"2023-10-11T09:40:42.687447+0000","last_fullsized":"2023-10-11T09:40:42.687447+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:09:27.825336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.691897+0000","last_change":"2023-10-11T09:40:41.714225+0000","last_active":"2023-10-11T09:40:42.691897+0000","last_peered":"2023-10-11T09:40:42.691897+0000","last_clean":"2023-10-11T09:40:42.691897+0000","last_became_active":"2023-10-11T09:40:41.710262+0000","last_became_peered":"2023-10-11T09:40:41.710262+0000","last_unstale":"2023-10-11T09:40:42.691897+0000","last_undegraded":"2023-10-11T09:40:42.691897+0000","last_fullsized":"2023-10-11T09:40:42.691897+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:42:34.821513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144675+0000","last_change":"2023-10-11T09:40:41.729722+0000","last_active":"2023-10-11T09:40:43.144675+0000","last_peered":"2023-10-11T09:40:43.144675+0000","last_clean":"2023-10-11T09:40:43.144675+0000","last_became_active":"2023-10-11T09:40:41.729360+0000","last_became_peered":"2023-10-11T09:40:41.729360+0000","last_unstale":"2023-10-11T09:40:43.144675+0000","last_undegraded":"2023-10-11T09:40:43.144675+0000","last_fullsized":"2023-10-11T09:40:43.144675+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:07:02.302344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145219+0000","last_change":"2023-10-11T09:40:41.709354+0000","last_active":"2023-10-11T09:40:43.145219+0000","last_peered":"2023-10-11T09:40:43.145219+0000","last_clean":"2023-10-11T09:40:43.145219+0000","last_became_active":"2023-10-11T09:40:41.709096+0000","last_became_peered":"2023-10-11T09:40:41.709096+0000","last_unstale":"2023-10-11T09:40:43.145219+0000","last_undegraded":"2023-10-11T09:40:43.145219+0000","last_fullsized":"2023-10-11T09:40:43.145219+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:35:04.646798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.694072+0000","last_change":"2023-10-11T09:40:41.723510+0000","last_active":"2023-10-11T09:40:42.694072+0000","last_peered":"2023-10-11T09:40:42.694072+0000","last_clean":"2023-10-11T09:40:42.694072+0000","last_became_active":"2023-10-11T09:40:41.723110+0000","last_became_peered":"2023-10-11T09:40:41.723110+0000","last_unstale":"2023-10-11T09:40:42.694072+0000","last_undegraded":"2023-10-11T09:40:42.694072+0000","last_fullsized":"2023-10-11T09:40:42.694072+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:50:15.829941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145120+0000","last_change":"2023-10-11T09:40:41.714481+0000","last_active":"2023-10-11T09:40:43.145120+0000","last_peered":"2023-10-11T09:40:43.145120+0000","last_clean":"2023-10-11T09:40:43.145120+0000","last_became_active":"2023-10-11T09:40:41.714219+0000","last_became_peered":"2023-10-11T09:40:41.714219+0000","last_unstale":"2023-10-11T09:40:43.145120+0000","last_undegraded":"2023-10-11T09:40:43.145120+0000","last_fullsized":"2023-10-11T09:40:43.145120+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:19:07.089979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.688002+0000","last_change":"2023-10-11T09:40:41.709612+0000","last_active":"2023-10-11T09:40:42.688002+0000","last_peered":"2023-10-11T09:40:42.688002+0000","last_clean":"2023-10-11T09:40:42.688002+0000","last_became_active":"2023-10-11T09:40:41.709317+0000","last_became_peered":"2023-10-11T09:40:41.709317+0000","last_unstale":"2023-10-11T09:40:42.688002+0000","last_undegraded":"2023-10-11T09:40:42.688002+0000","last_fullsized":"2023-10-11T09:40:42.688002+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:35:36.436236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911908+0000","last_change":"2023-10-11T09:40:41.727354+0000","last_active":"2023-10-11T09:40:42.911908+0000","last_peered":"2023-10-11T09:40:42.911908+0000","last_clean":"2023-10-11T09:40:42.911908+0000","last_became_active":"2023-10-11T09:40:41.727039+0000","last_became_peered":"2023-10-11T09:40:41.727039+0000","last_unstale":"2023-10-11T09:40:42.911908+0000","last_undegraded":"2023-10-11T09:40:42.911908+0000","last_fullsized":"2023-10-11T09:40:42.911908+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:01:36.923947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687557+0000","last_change":"2023-10-11T09:40:41.712971+0000","last_active":"2023-10-11T09:40:42.687557+0000","last_peered":"2023-10-11T09:40:42.687557+0000","last_clean":"2023-10-11T09:40:42.687557+0000","last_became_active":"2023-10-11T09:40:41.712707+0000","last_became_peered":"2023-10-11T09:40:41.712707+0000","last_unstale":"2023-10-11T09:40:42.687557+0000","last_undegraded":"2023-10-11T09:40:42.687557+0000","last_fullsized":"2023-10-11T09:40:42.687557+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:48:15.689750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.143927+0000","last_change":"2023-10-11T09:40:41.728941+0000","last_active":"2023-10-11T09:40:43.143927+0000","last_peered":"2023-10-11T09:40:43.143927+0000","last_clean":"2023-10-11T09:40:43.143927+0000","last_became_active":"2023-10-11T09:40:41.728657+0000","last_became_peered":"2023-10-11T09:40:41.728657+0000","last_unstale":"2023-10-11T09:40:43.143927+0000","last_undegraded":"2023-10-11T09:40:43.143927+0000","last_fullsized":"2023-10-11T09:40:43.143927+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:12:05.737223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.688259+0000","last_change":"2023-10-11T09:40:41.718662+0000","last_active":"2023-10-11T09:40:42.688259+0000","last_peered":"2023-10-11T09:40:42.688259+0000","last_clean":"2023-10-11T09:40:42.688259+0000","last_became_active":"2023-10-11T09:40:41.718402+0000","last_became_peered":"2023-10-11T09:40:41.718402+0000","last_unstale":"2023-10-11T09:40:42.688259+0000","last_undegraded":"2023-10-11T09:40:42.688259+0000","last_fullsized":"2023-10-11T09:40:42.688259+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:56:50.343276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.692319+0000","last_change":"2023-10-11T09:40:41.234649+0000","last_active":"2023-10-11T09:40:42.692319+0000","last_peered":"2023-10-11T09:40:42.692319+0000","last_clean":"2023-10-11T09:40:42.692319+0000","last_became_active":"2023-10-11T09:40:41.234006+0000","last_became_peered":"2023-10-11T09:40:41.234006+0000","last_unstale":"2023-10-11T09:40:42.692319+0000","last_undegraded":"2023-10-11T09:40:42.692319+0000","last_fullsized":"2023-10-11T09:40:42.692319+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:45:39.083797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.694114+0000","last_change":"2023-10-11T09:40:41.723234+0000","last_active":"2023-10-11T09:40:42.694114+0000","last_peered":"2023-10-11T09:40:42.694114+0000","last_clean":"2023-10-11T09:40:42.694114+0000","last_became_active":"2023-10-11T09:40:41.722928+0000","last_became_peered":"2023-10-11T09:40:41.722928+0000","last_unstale":"2023-10-11T09:40:42.694114+0000","last_undegraded":"2023-10-11T09:40:42.694114+0000","last_fullsized":"2023-10-11T09:40:42.694114+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:09:27.825336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145323+0000","last_change":"2023-10-11T09:40:41.225247+0000","last_active":"2023-10-11T09:40:43.145323+0000","last_peered":"2023-10-11T09:40:43.145323+0000","last_clean":"2023-10-11T09:40:43.145323+0000","last_became_active":"2023-10-11T09:40:41.224967+0000","last_became_peered":"2023-10-11T09:40:41.224967+0000","last_unstale":"2023-10-11T09:40:43.145323+0000","last_undegraded":"2023-10-11T09:40:43.145323+0000","last_fullsized":"2023-10-11T09:40:43.145323+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:33:10.649755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145747+0000","last_change":"2023-10-11T09:40:41.703018+0000","last_active":"2023-10-11T09:40:43.145747+0000","last_peered":"2023-10-11T09:40:43.145747+0000","last_clean":"2023-10-11T09:40:43.145747+0000","last_became_active":"2023-10-11T09:40:41.702729+0000","last_became_peered":"2023-10-11T09:40:41.702729+0000","last_unstale":"2023-10-11T09:40:43.145747+0000","last_undegraded":"2023-10-11T09:40:43.145747+0000","last_fullsized":"2023-10-11T09:40:43.145747+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:07:02.302344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.692298+0000","last_change":"2023-10-11T09:40:41.234929+0000","last_active":"2023-10-11T09:40:42.692298+0000","last_peered":"2023-10-11T09:40:42.692298+0000","last_clean":"2023-10-11T09:40:42.692298+0000","last_became_active":"2023-10-11T09:40:41.234149+0000","last_became_peered":"2023-10-11T09:40:41.234149+0000","last_unstale":"2023-10-11T09:40:42.692298+0000","last_undegraded":"2023-10-11T09:40:42.692298+0000","last_fullsized":"2023-10-11T09:40:42.692298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:17:35.640470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.688569+0000","last_change":"2023-10-11T09:40:41.716719+0000","last_active":"2023-10-11T09:40:42.688569+0000","last_peered":"2023-10-11T09:40:42.688569+0000","last_clean":"2023-10-11T09:40:42.688569+0000","last_became_active":"2023-10-11T09:40:41.716181+0000","last_became_peered":"2023-10-11T09:40:41.716181+0000","last_unstale":"2023-10-11T09:40:42.688569+0000","last_undegraded":"2023-10-11T09:40:42.688569+0000","last_fullsized":"2023-10-11T09:40:42.688569+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:50:15.829941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.143863+0000","last_change":"2023-10-11T09:40:41.229549+0000","last_active":"2023-10-11T09:40:43.143863+0000","last_peered":"2023-10-11T09:40:43.143863+0000","last_clean":"2023-10-11T09:40:43.143863+0000","last_became_active":"2023-10-11T09:40:41.229259+0000","last_became_peered":"2023-10-11T09:40:41.229259+0000","last_unstale":"2023-10-11T09:40:43.143863+0000","last_undegraded":"2023-10-11T09:40:43.143863+0000","last_fullsized":"2023-10-11T09:40:43.143863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:16:37.265056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.692895+0000","last_change":"2023-10-11T09:40:41.234477+0000","last_active":"2023-10-11T09:40:42.692895+0000","last_peered":"2023-10-11T09:40:42.692895+0000","last_clean":"2023-10-11T09:40:42.692895+0000","last_became_active":"2023-10-11T09:40:41.233977+0000","last_became_peered":"2023-10-11T09:40:41.233977+0000","last_unstale":"2023-10-11T09:40:42.692895+0000","last_undegraded":"2023-10-11T09:40:42.692895+0000","last_fullsized":"2023-10-11T09:40:42.692895+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:47:25.022675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.693696+0000","last_change":"2023-10-11T09:40:41.717799+0000","last_active":"2023-10-11T09:40:42.693696+0000","last_peered":"2023-10-11T09:40:42.693696+0000","last_clean":"2023-10-11T09:40:42.693696+0000","last_became_active":"2023-10-11T09:40:41.717520+0000","last_became_peered":"2023-10-11T09:40:41.717520+0000","last_unstale":"2023-10-11T09:40:42.693696+0000","last_undegraded":"2023-10-11T09:40:42.693696+0000","last_fullsized":"2023-10-11T09:40:42.693696+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:35:36.436236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.692703+0000","last_change":"2023-10-11T09:40:41.234979+0000","last_active":"2023-10-11T09:40:42.692703+0000","last_peered":"2023-10-11T09:40:42.692703+0000","last_clean":"2023-10-11T09:40:42.692703+0000","last_became_active":"2023-10-11T09:40:41.234245+0000","last_became_peered":"2023-10-11T09:40:41.234245+0000","last_unstale":"2023-10-11T09:40:42.692703+0000","last_undegraded":"2023-10-11T09:40:42.692703+0000","last_fullsized":"2023-10-11T09:40:42.692703+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:31:56.724263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911330+0000","last_change":"2023-10-11T09:40:41.706936+0000","last_active":"2023-10-11T09:40:42.911330+0000","last_peered":"2023-10-11T09:40:42.911330+0000","last_clean":"2023-10-11T09:40:42.911330+0000","last_became_active":"2023-10-11T09:40:41.706440+0000","last_became_peered":"2023-10-11T09:40:41.706440+0000","last_unstale":"2023-10-11T09:40:42.911330+0000","last_undegraded":"2023-10-11T09:40:42.911330+0000","last_fullsized":"2023-10-11T09:40:42.911330+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:12:14.951424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.692744+0000","last_change":"2023-10-11T09:40:41.228740+0000","last_active":"2023-10-11T09:40:42.692744+0000","last_peered":"2023-10-11T09:40:42.692744+0000","last_clean":"2023-10-11T09:40:42.692744+0000","last_became_active":"2023-10-11T09:40:41.228327+0000","last_became_peered":"2023-10-11T09:40:41.228327+0000","last_unstale":"2023-10-11T09:40:42.692744+0000","last_undegraded":"2023-10-11T09:40:42.692744+0000","last_fullsized":"2023-10-11T09:40:42.692744+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:59:14.883485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144214+0000","last_change":"2023-10-11T09:40:41.721661+0000","last_active":"2023-10-11T09:40:43.144214+0000","last_peered":"2023-10-11T09:40:43.144214+0000","last_clean":"2023-10-11T09:40:43.144214+0000","last_became_active":"2023-10-11T09:40:41.720154+0000","last_became_peered":"2023-10-11T09:40:41.720154+0000","last_unstale":"2023-10-11T09:40:43.144214+0000","last_undegraded":"2023-10-11T09:40:43.144214+0000","last_fullsized":"2023-10-11T09:40:43.144214+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:45:40.098819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.693986+0000","last_change":"2023-10-11T09:40:41.225444+0000","last_active":"2023-10-11T09:40:42.693986+0000","last_peered":"2023-10-11T09:40:42.693986+0000","last_clean":"2023-10-11T09:40:42.693986+0000","last_became_active":"2023-10-11T09:40:41.225159+0000","last_became_peered":"2023-10-11T09:40:41.225159+0000","last_unstale":"2023-10-11T09:40:42.693986+0000","last_undegraded":"2023-10-11T09:40:42.693986+0000","last_fullsized":"2023-10-11T09:40:42.693986+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:00:56.048004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145496+0000","last_change":"2023-10-11T09:40:41.712920+0000","last_active":"2023-10-11T09:40:43.145496+0000","last_peered":"2023-10-11T09:40:43.145496+0000","last_clean":"2023-10-11T09:40:43.145496+0000","last_became_active":"2023-10-11T09:40:41.712654+0000","last_became_peered":"2023-10-11T09:40:41.712654+0000","last_unstale":"2023-10-11T09:40:43.145496+0000","last_undegraded":"2023-10-11T09:40:43.145496+0000","last_fullsized":"2023-10-11T09:40:43.145496+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:23:30.814623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.688154+0000","last_change":"2023-10-11T09:40:41.229300+0000","last_active":"2023-10-11T09:40:42.688154+0000","last_peered":"2023-10-11T09:40:42.688154+0000","last_clean":"2023-10-11T09:40:42.688154+0000","last_became_active":"2023-10-11T09:40:41.228539+0000","last_became_peered":"2023-10-11T09:40:41.228539+0000","last_unstale":"2023-10-11T09:40:42.688154+0000","last_undegraded":"2023-10-11T09:40:42.688154+0000","last_fullsized":"2023-10-11T09:40:42.688154+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:09:03.936717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.912449+0000","last_change":"2023-10-11T09:40:41.705540+0000","last_active":"2023-10-11T09:40:42.912449+0000","last_peered":"2023-10-11T09:40:42.912449+0000","last_clean":"2023-10-11T09:40:42.912449+0000","last_became_active":"2023-10-11T09:40:41.705163+0000","last_became_peered":"2023-10-11T09:40:41.705163+0000","last_unstale":"2023-10-11T09:40:42.912449+0000","last_undegraded":"2023-10-11T09:40:42.912449+0000","last_fullsized":"2023-10-11T09:40:42.912449+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:56:50.343276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144408+0000","last_change":"2023-10-11T09:40:41.224108+0000","last_active":"2023-10-11T09:40:43.144408+0000","last_peered":"2023-10-11T09:40:43.144408+0000","last_clean":"2023-10-11T09:40:43.144408+0000","last_became_active":"2023-10-11T09:40:41.223643+0000","last_became_peered":"2023-10-11T09:40:41.223643+0000","last_unstale":"2023-10-11T09:40:43.144408+0000","last_undegraded":"2023-10-11T09:40:43.144408+0000","last_fullsized":"2023-10-11T09:40:43.144408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:36:34.713602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.912393+0000","last_change":"2023-10-11T09:40:41.700784+0000","last_active":"2023-10-11T09:40:42.912393+0000","last_peered":"2023-10-11T09:40:42.912393+0000","last_clean":"2023-10-11T09:40:42.912393+0000","last_became_active":"2023-10-11T09:40:41.700495+0000","last_became_peered":"2023-10-11T09:40:41.700495+0000","last_unstale":"2023-10-11T09:40:42.912393+0000","last_undegraded":"2023-10-11T09:40:42.912393+0000","last_fullsized":"2023-10-11T09:40:42.912393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:57:16.072435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911546+0000","last_change":"2023-10-11T09:40:41.233232+0000","last_active":"2023-10-11T09:40:42.911546+0000","last_peered":"2023-10-11T09:40:42.911546+0000","last_clean":"2023-10-11T09:40:42.911546+0000","last_became_active":"2023-10-11T09:40:41.232092+0000","last_became_peered":"2023-10-11T09:40:41.232092+0000","last_unstale":"2023-10-11T09:40:42.911546+0000","last_undegraded":"2023-10-11T09:40:42.911546+0000","last_fullsized":"2023-10-11T09:40:42.911546+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:16:37.265056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.912288+0000","last_change":"2023-10-11T09:40:41.721557+0000","last_active":"2023-10-11T09:40:42.912288+0000","last_peered":"2023-10-11T09:40:42.912288+0000","last_clean":"2023-10-11T09:40:42.912288+0000","last_became_active":"2023-10-11T09:40:41.721314+0000","last_became_peered":"2023-10-11T09:40:41.721314+0000","last_unstale":"2023-10-11T09:40:42.912288+0000","last_undegraded":"2023-10-11T09:40:42.912288+0000","last_fullsized":"2023-10-11T09:40:42.912288+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:59:15.898506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687786+0000","last_change":"2023-10-11T09:40:41.228065+0000","last_active":"2023-10-11T09:40:42.687786+0000","last_peered":"2023-10-11T09:40:42.687786+0000","last_clean":"2023-10-11T09:40:42.687786+0000","last_became_active":"2023-10-11T09:40:41.227765+0000","last_became_peered":"2023-10-11T09:40:41.227765+0000","last_unstale":"2023-10-11T09:40:42.687786+0000","last_undegraded":"2023-10-11T09:40:42.687786+0000","last_fullsized":"2023-10-11T09:40:42.687786+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:36:34.713602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144923+0000","last_change":"2023-10-11T09:40:41.708864+0000","last_active":"2023-10-11T09:40:43.144923+0000","last_peered":"2023-10-11T09:40:43.144923+0000","last_clean":"2023-10-11T09:40:43.144923+0000","last_became_active":"2023-10-11T09:40:41.708555+0000","last_became_peered":"2023-10-11T09:40:41.708555+0000","last_unstale":"2023-10-11T09:40:43.144923+0000","last_undegraded":"2023-10-11T09:40:43.144923+0000","last_fullsized":"2023-10-11T09:40:43.144923+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:43:30.691251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":167,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-11T09:40:41.695042+0000","last_change":"2023-10-11T09:40:29.874372+0000","last_active":"2023-10-11T09:40:41.695042+0000","last_peered":"2023-10-11T09:40:41.695042+0000","last_clean":"2023-10-11T09:40:41.695042+0000","last_became_active":"2023-10-11T09:40:29.873621+0000","last_became_peered":"2023-10-11T09:40:29.873621+0000","last_unstale":"2023-10-11T09:40:41.695042+0000","last_undegraded":"2023-10-11T09:40:41.695042+0000","last_fullsized":"2023-10-11T09:40:41.695042+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_clean_scrub_stamp":"2023-10-11T09:40:28.742276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:22:07.985311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-11T09:40:41.705703+0000","last_change":"2023-10-11T09:40:36.148773+0000","last_active":"2023-10-11T09:40:41.705703+0000","last_peered":"2023-10-11T09:40:41.705703+0000","last_clean":"2023-10-11T09:40:41.705703+0000","last_became_active":"2023-10-11T09:40:36.148489+0000","last_became_peered":"2023-10-11T09:40:36.148489+0000","last_unstale":"2023-10-11T09:40:41.705703+0000","last_undegraded":"2023-10-11T09:40:41.705703+0000","last_fullsized":"2023-10-11T09:40:41.705703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687547+0000","last_change":"2023-10-11T09:40:41.227668+0000","last_active":"2023-10-11T09:40:42.687547+0000","last_peered":"2023-10-11T09:40:42.687547+0000","last_clean":"2023-10-11T09:40:42.687547+0000","last_became_active":"2023-10-11T09:40:41.227357+0000","last_became_peered":"2023-10-11T09:40:41.227357+0000","last_unstale":"2023-10-11T09:40:42.687547+0000","last_undegraded":"2023-10-11T09:40:42.687547+0000","last_fullsized":"2023-10-11T09:40:42.687547+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:16:37.265056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144334+0000","last_change":"2023-10-11T09:40:41.726684+0000","last_active":"2023-10-11T09:40:43.144334+0000","last_peered":"2023-10-11T09:40:43.144334+0000","last_clean":"2023-10-11T09:40:43.144334+0000","last_became_active":"2023-10-11T09:40:41.726323+0000","last_became_peered":"2023-10-11T09:40:41.726323+0000","last_unstale":"2023-10-11T09:40:43.144334+0000","last_undegraded":"2023-10-11T09:40:43.144334+0000","last_fullsized":"2023-10-11T09:40:43.144334+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:47:26.037696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-11T09:40:41.694934+0000","last_change":"2023-10-11T09:40:36.149739+0000","last_active":"2023-10-11T09:40:41.694934+0000","last_peered":"2023-10-11T09:40:41.694934+0000","last_clean":"2023-10-11T09:40:41.694934+0000","last_became_active":"2023-10-11T09:40:36.149419+0000","last_became_peered":"2023-10-11T09:40:36.149419+0000","last_unstale":"2023-10-11T09:40:41.694934+0000","last_undegraded":"2023-10-11T09:40:41.694934+0000","last_fullsized":"2023-10-11T09:40:41.694934+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:54:50.640362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.143706+0000","last_change":"2023-10-11T09:40:41.224529+0000","last_active":"2023-10-11T09:40:43.143706+0000","last_peered":"2023-10-11T09:40:43.143706+0000","last_clean":"2023-10-11T09:40:43.143706+0000","last_became_active":"2023-10-11T09:40:41.224252+0000","last_became_peered":"2023-10-11T09:40:41.224252+0000","last_unstale":"2023-10-11T09:40:43.143706+0000","last_undegraded":"2023-10-11T09:40:43.143706+0000","last_fullsized":"2023-10-11T09:40:43.143706+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:09:03.936717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145788+0000","last_change":"2023-10-11T09:40:41.718539+0000","last_active":"2023-10-11T09:40:43.145788+0000","last_peered":"2023-10-11T09:40:43.145788+0000","last_clean":"2023-10-11T09:40:43.145788+0000","last_became_active":"2023-10-11T09:40:41.718231+0000","last_became_peered":"2023-10-11T09:40:41.718231+0000","last_unstale":"2023-10-11T09:40:43.145788+0000","last_undegraded":"2023-10-11T09:40:43.145788+0000","last_fullsized":"2023-10-11T09:40:43.145788+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:50:15.829941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-11T09:40:41.682824+0000","last_change":"2023-10-11T09:40:36.216248+0000","last_active":"2023-10-11T09:40:41.682824+0000","last_peered":"2023-10-11T09:40:41.682824+0000","last_clean":"2023-10-11T09:40:41.682824+0000","last_became_active":"2023-10-11T09:40:36.215890+0000","last_became_peered":"2023-10-11T09:40:36.215890+0000","last_unstale":"2023-10-11T09:40:41.682824+0000","last_undegraded":"2023-10-11T09:40:41.682824+0000","last_fullsized":"2023-10-11T09:40:41.682824+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.143998+0000","last_change":"2023-10-11T09:40:41.220987+0000","last_active":"2023-10-11T09:40:43.143998+0000","last_peered":"2023-10-11T09:40:43.143998+0000","last_clean":"2023-10-11T09:40:43.143998+0000","last_became_active":"2023-10-11T09:40:41.220035+0000","last_became_peered":"2023-10-11T09:40:41.220035+0000","last_unstale":"2023-10-11T09:40:43.143998+0000","last_undegraded":"2023-10-11T09:40:43.143998+0000","last_fullsized":"2023-10-11T09:40:43.143998+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:00:56.048004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144958+0000","last_change":"2023-10-11T09:40:41.718356+0000","last_active":"2023-10-11T09:40:43.144958+0000","last_peered":"2023-10-11T09:40:43.144958+0000","last_clean":"2023-10-11T09:40:43.144958+0000","last_became_active":"2023-10-11T09:40:41.718069+0000","last_became_peered":"2023-10-11T09:40:41.718069+0000","last_unstale":"2023-10-11T09:40:43.144958+0000","last_undegraded":"2023-10-11T09:40:43.144958+0000","last_fullsized":"2023-10-11T09:40:43.144958+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:42:34.821513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-11T09:40:41.705511+0000","last_change":"2023-10-11T09:40:36.151710+0000","last_active":"2023-10-11T09:40:41.705511+0000","last_peered":"2023-10-11T09:40:41.705511+0000","last_clean":"2023-10-11T09:40:41.705511+0000","last_became_active":"2023-10-11T09:40:36.151219+0000","last_became_peered":"2023-10-11T09:40:36.151219+0000","last_unstale":"2023-10-11T09:40:41.705511+0000","last_undegraded":"2023-10-11T09:40:41.705511+0000","last_fullsized":"2023-10-11T09:40:41.705511+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911785+0000","last_change":"2023-10-11T09:40:41.231464+0000","last_active":"2023-10-11T09:40:42.911785+0000","last_peered":"2023-10-11T09:40:42.911785+0000","last_clean":"2023-10-11T09:40:42.911785+0000","last_became_active":"2023-10-11T09:40:41.229072+0000","last_became_peered":"2023-10-11T09:40:41.229072+0000","last_unstale":"2023-10-11T09:40:42.911785+0000","last_undegraded":"2023-10-11T09:40:42.911785+0000","last_fullsized":"2023-10-11T09:40:42.911785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:33:10.649755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144540+0000","last_change":"2023-10-11T09:40:41.715638+0000","last_active":"2023-10-11T09:40:43.144540+0000","last_peered":"2023-10-11T09:40:43.144540+0000","last_clean":"2023-10-11T09:40:43.144540+0000","last_became_active":"2023-10-11T09:40:41.715399+0000","last_became_peered":"2023-10-11T09:40:41.715399+0000","last_unstale":"2023-10-11T09:40:43.144540+0000","last_undegraded":"2023-10-11T09:40:43.144540+0000","last_fullsized":"2023-10-11T09:40:43.144540+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:00:05.422788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.694217+0000","last_change":"2023-10-11T09:40:41.231735+0000","last_active":"2023-10-11T09:40:42.694217+0000","last_peered":"2023-10-11T09:40:42.694217+0000","last_clean":"2023-10-11T09:40:42.694217+0000","last_became_active":"2023-10-11T09:40:41.229777+0000","last_became_peered":"2023-10-11T09:40:41.229777+0000","last_unstale":"2023-10-11T09:40:42.694217+0000","last_undegraded":"2023-10-11T09:40:42.694217+0000","last_fullsized":"2023-10-11T09:40:42.694217+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:09:03.936717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-11T09:40:41.695209+0000","last_change":"2023-10-11T09:40:36.160620+0000","last_active":"2023-10-11T09:40:41.695209+0000","last_peered":"2023-10-11T09:40:41.695209+0000","last_clean":"2023-10-11T09:40:41.695209+0000","last_became_active":"2023-10-11T09:40:36.160171+0000","last_became_peered":"2023-10-11T09:40:36.160171+0000","last_unstale":"2023-10-11T09:40:41.695209+0000","last_undegraded":"2023-10-11T09:40:41.695209+0000","last_fullsized":"2023-10-11T09:40:41.695209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145219+0000","last_change":"2023-10-11T09:40:41.718869+0000","last_active":"2023-10-11T09:40:43.145219+0000","last_peered":"2023-10-11T09:40:43.145219+0000","last_clean":"2023-10-11T09:40:43.145219+0000","last_became_active":"2023-10-11T09:40:41.718628+0000","last_became_peered":"2023-10-11T09:40:41.718628+0000","last_unstale":"2023-10-11T09:40:43.145219+0000","last_undegraded":"2023-10-11T09:40:43.145219+0000","last_fullsized":"2023-10-11T09:40:43.145219+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:50:15.829941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144700+0000","last_change":"2023-10-11T09:40:41.225562+0000","last_active":"2023-10-11T09:40:43.144700+0000","last_peered":"2023-10-11T09:40:43.144700+0000","last_clean":"2023-10-11T09:40:43.144700+0000","last_became_active":"2023-10-11T09:40:41.222909+0000","last_became_peered":"2023-10-11T09:40:41.222909+0000","last_unstale":"2023-10-11T09:40:43.144700+0000","last_undegraded":"2023-10-11T09:40:43.144700+0000","last_fullsized":"2023-10-11T09:40:43.144700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:09:03.936717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-11T09:40:41.696680+0000","last_change":"2023-10-11T09:40:36.162174+0000","last_active":"2023-10-11T09:40:41.696680+0000","last_peered":"2023-10-11T09:40:41.696680+0000","last_clean":"2023-10-11T09:40:41.696680+0000","last_became_active":"2023-10-11T09:40:36.161710+0000","last_became_peered":"2023-10-11T09:40:36.161710+0000","last_unstale":"2023-10-11T09:40:41.696680+0000","last_undegraded":"2023-10-11T09:40:41.696680+0000","last_fullsized":"2023-10-11T09:40:41.696680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:57:03.630878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145842+0000","last_change":"2023-10-11T09:40:41.716886+0000","last_active":"2023-10-11T09:40:43.145842+0000","last_peered":"2023-10-11T09:40:43.145842+0000","last_clean":"2023-10-11T09:40:43.145842+0000","last_became_active":"2023-10-11T09:40:41.716601+0000","last_became_peered":"2023-10-11T09:40:41.716601+0000","last_unstale":"2023-10-11T09:40:43.145842+0000","last_undegraded":"2023-10-11T09:40:43.145842+0000","last_fullsized":"2023-10-11T09:40:43.145842+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:00:05.422788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.688690+0000","last_change":"2023-10-11T09:40:41.224655+0000","last_active":"2023-10-11T09:40:42.688690+0000","last_peered":"2023-10-11T09:40:42.688690+0000","last_clean":"2023-10-11T09:40:42.688690+0000","last_became_active":"2023-10-11T09:40:41.224374+0000","last_became_peered":"2023-10-11T09:40:41.224374+0000","last_unstale":"2023-10-11T09:40:42.688690+0000","last_undegraded":"2023-10-11T09:40:42.688690+0000","last_fullsized":"2023-10-11T09:40:42.688690+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:33:10.649755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-11T09:40:41.699405+0000","last_change":"2023-10-11T09:40:36.150224+0000","last_active":"2023-10-11T09:40:41.699405+0000","last_peered":"2023-10-11T09:40:41.699405+0000","last_clean":"2023-10-11T09:40:41.699405+0000","last_became_active":"2023-10-11T09:40:36.149919+0000","last_became_peered":"2023-10-11T09:40:36.149919+0000","last_unstale":"2023-10-11T09:40:41.699405+0000","last_undegraded":"2023-10-11T09:40:41.699405+0000","last_fullsized":"2023-10-11T09:40:41.699405+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687621+0000","last_change":"2023-10-11T09:40:41.696509+0000","last_active":"2023-10-11T09:40:42.687621+0000","last_peered":"2023-10-11T09:40:42.687621+0000","last_clean":"2023-10-11T09:40:42.687621+0000","last_became_active":"2023-10-11T09:40:41.696149+0000","last_became_peered":"2023-10-11T09:40:41.696149+0000","last_unstale":"2023-10-11T09:40:42.687621+0000","last_undegraded":"2023-10-11T09:40:42.687621+0000","last_fullsized":"2023-10-11T09:40:42.687621+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:00:05.422788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145899+0000","last_change":"2023-10-11T09:40:41.230379+0000","last_active":"2023-10-11T09:40:43.145899+0000","last_peered":"2023-10-11T09:40:43.145899+0000","last_clean":"2023-10-11T09:40:43.145899+0000","last_became_active":"2023-10-11T09:40:41.226481+0000","last_became_peered":"2023-10-11T09:40:41.226481+0000","last_unstale":"2023-10-11T09:40:43.145899+0000","last_undegraded":"2023-10-11T09:40:43.145899+0000","last_fullsized":"2023-10-11T09:40:43.145899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:48:51.882031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-11T09:40:41.708126+0000","last_change":"2023-10-11T09:40:36.151839+0000","last_active":"2023-10-11T09:40:41.708126+0000","last_peered":"2023-10-11T09:40:41.708126+0000","last_clean":"2023-10-11T09:40:41.708126+0000","last_became_active":"2023-10-11T09:40:36.151179+0000","last_became_peered":"2023-10-11T09:40:36.151179+0000","last_unstale":"2023-10-11T09:40:41.708126+0000","last_undegraded":"2023-10-11T09:40:41.708126+0000","last_fullsized":"2023-10-11T09:40:41.708126+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911579+0000","last_change":"2023-10-11T09:40:41.712383+0000","last_active":"2023-10-11T09:40:42.911579+0000","last_peered":"2023-10-11T09:40:42.911579+0000","last_clean":"2023-10-11T09:40:42.911579+0000","last_became_active":"2023-10-11T09:40:41.711978+0000","last_became_peered":"2023-10-11T09:40:41.711978+0000","last_unstale":"2023-10-11T09:40:42.911579+0000","last_undegraded":"2023-10-11T09:40:42.911579+0000","last_fullsized":"2023-10-11T09:40:42.911579+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:45:40.098819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911910+0000","last_change":"2023-10-11T09:40:41.720785+0000","last_active":"2023-10-11T09:40:42.911910+0000","last_peered":"2023-10-11T09:40:42.911910+0000","last_clean":"2023-10-11T09:40:42.911910+0000","last_became_active":"2023-10-11T09:40:41.720508+0000","last_became_peered":"2023-10-11T09:40:41.720508+0000","last_unstale":"2023-10-11T09:40:42.911910+0000","last_undegraded":"2023-10-11T09:40:42.911910+0000","last_fullsized":"2023-10-11T09:40:42.911910+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:35:04.646798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.912435+0000","last_change":"2023-10-11T09:40:41.229054+0000","last_active":"2023-10-11T09:40:42.912435+0000","last_peered":"2023-10-11T09:40:42.912435+0000","last_clean":"2023-10-11T09:40:42.912435+0000","last_became_active":"2023-10-11T09:40:41.227552+0000","last_became_peered":"2023-10-11T09:40:41.227552+0000","last_unstale":"2023-10-11T09:40:42.912435+0000","last_undegraded":"2023-10-11T09:40:42.912435+0000","last_fullsized":"2023-10-11T09:40:42.912435+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:36:34.713602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144970+0000","last_change":"2023-10-11T09:40:41.730718+0000","last_active":"2023-10-11T09:40:43.144970+0000","last_peered":"2023-10-11T09:40:43.144970+0000","last_clean":"2023-10-11T09:40:43.144970+0000","last_became_active":"2023-10-11T09:40:41.730499+0000","last_became_peered":"2023-10-11T09:40:41.730499+0000","last_unstale":"2023-10-11T09:40:43.144970+0000","last_undegraded":"2023-10-11T09:40:43.144970+0000","last_fullsized":"2023-10-11T09:40:43.144970+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:35:36.436236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.692532+0000","last_change":"2023-10-11T09:40:41.230822+0000","last_active":"2023-10-11T09:40:42.692532+0000","last_peered":"2023-10-11T09:40:42.692532+0000","last_clean":"2023-10-11T09:40:42.692532+0000","last_became_active":"2023-10-11T09:40:41.230417+0000","last_became_peered":"2023-10-11T09:40:41.230417+0000","last_unstale":"2023-10-11T09:40:42.692532+0000","last_undegraded":"2023-10-11T09:40:42.692532+0000","last_fullsized":"2023-10-11T09:40:42.692532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:20:53.471012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.694445+0000","last_change":"2023-10-11T09:40:41.231668+0000","last_active":"2023-10-11T09:40:42.694445+0000","last_peered":"2023-10-11T09:40:42.694445+0000","last_clean":"2023-10-11T09:40:42.694445+0000","last_became_active":"2023-10-11T09:40:41.226956+0000","last_became_peered":"2023-10-11T09:40:41.226956+0000","last_unstale":"2023-10-11T09:40:42.694445+0000","last_undegraded":"2023-10-11T09:40:42.694445+0000","last_fullsized":"2023-10-11T09:40:42.694445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:43:00.315992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687648+0000","last_change":"2023-10-11T09:40:41.709493+0000","last_active":"2023-10-11T09:40:42.687648+0000","last_peered":"2023-10-11T09:40:42.687648+0000","last_clean":"2023-10-11T09:40:42.687648+0000","last_became_active":"2023-10-11T09:40:41.709217+0000","last_became_peered":"2023-10-11T09:40:41.709217+0000","last_unstale":"2023-10-11T09:40:42.687648+0000","last_undegraded":"2023-10-11T09:40:42.687648+0000","last_fullsized":"2023-10-11T09:40:42.687648+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:07:02.302344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.146199+0000","last_change":"2023-10-11T09:40:41.228934+0000","last_active":"2023-10-11T09:40:43.146199+0000","last_peered":"2023-10-11T09:40:43.146199+0000","last_clean":"2023-10-11T09:40:43.146199+0000","last_became_active":"2023-10-11T09:40:41.226833+0000","last_became_peered":"2023-10-11T09:40:41.226833+0000","last_unstale":"2023-10-11T09:40:43.146199+0000","last_undegraded":"2023-10-11T09:40:43.146199+0000","last_fullsized":"2023-10-11T09:40:43.146199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:00:56.048004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687536+0000","last_change":"2023-10-11T09:40:41.715540+0000","last_active":"2023-10-11T09:40:42.687536+0000","last_peered":"2023-10-11T09:40:42.687536+0000","last_clean":"2023-10-11T09:40:42.687536+0000","last_became_active":"2023-10-11T09:40:41.715254+0000","last_became_peered":"2023-10-11T09:40:41.715254+0000","last_unstale":"2023-10-11T09:40:42.687536+0000","last_undegraded":"2023-10-11T09:40:42.687536+0000","last_fullsized":"2023-10-11T09:40:42.687536+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:23:30.814623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144670+0000","last_change":"2023-10-11T09:40:41.722871+0000","last_active":"2023-10-11T09:40:43.144670+0000","last_peered":"2023-10-11T09:40:43.144670+0000","last_clean":"2023-10-11T09:40:43.144670+0000","last_became_active":"2023-10-11T09:40:41.722538+0000","last_became_peered":"2023-10-11T09:40:41.722538+0000","last_unstale":"2023-10-11T09:40:43.144670+0000","last_undegraded":"2023-10-11T09:40:43.144670+0000","last_fullsized":"2023-10-11T09:40:43.144670+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:55:31.027029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687761+0000","last_change":"2023-10-11T09:40:41.230916+0000","last_active":"2023-10-11T09:40:42.687761+0000","last_peered":"2023-10-11T09:40:42.687761+0000","last_clean":"2023-10-11T09:40:42.687761+0000","last_became_active":"2023-10-11T09:40:41.230639+0000","last_became_peered":"2023-10-11T09:40:41.230639+0000","last_unstale":"2023-10-11T09:40:42.687761+0000","last_undegraded":"2023-10-11T09:40:42.687761+0000","last_fullsized":"2023-10-11T09:40:42.687761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:48:51.882031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145531+0000","last_change":"2023-10-11T09:40:41.705483+0000","last_active":"2023-10-11T09:40:43.145531+0000","last_peered":"2023-10-11T09:40:43.145531+0000","last_clean":"2023-10-11T09:40:43.145531+0000","last_became_active":"2023-10-11T09:40:41.705060+0000","last_became_peered":"2023-10-11T09:40:41.705060+0000","last_unstale":"2023-10-11T09:40:43.145531+0000","last_undegraded":"2023-10-11T09:40:43.145531+0000","last_fullsized":"2023-10-11T09:40:43.145531+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:35:36.436236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144834+0000","last_change":"2023-10-11T09:40:41.227035+0000","last_active":"2023-10-11T09:40:43.144834+0000","last_peered":"2023-10-11T09:40:43.144834+0000","last_clean":"2023-10-11T09:40:43.144834+0000","last_became_active":"2023-10-11T09:40:41.226593+0000","last_became_peered":"2023-10-11T09:40:41.226593+0000","last_unstale":"2023-10-11T09:40:43.144834+0000","last_undegraded":"2023-10-11T09:40:43.144834+0000","last_fullsized":"2023-10-11T09:40:43.144834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:48:51.882031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.688081+0000","last_change":"2023-10-11T09:40:41.715379+0000","last_active":"2023-10-11T09:40:42.688081+0000","last_peered":"2023-10-11T09:40:42.688081+0000","last_clean":"2023-10-11T09:40:42.688081+0000","last_became_active":"2023-10-11T09:40:41.715091+0000","last_became_peered":"2023-10-11T09:40:41.715091+0000","last_unstale":"2023-10-11T09:40:42.688081+0000","last_undegraded":"2023-10-11T09:40:42.688081+0000","last_fullsized":"2023-10-11T09:40:42.688081+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:42:34.821513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911962+0000","last_change":"2023-10-11T09:40:41.235100+0000","last_active":"2023-10-11T09:40:42.911962+0000","last_peered":"2023-10-11T09:40:42.911962+0000","last_clean":"2023-10-11T09:40:42.911962+0000","last_became_active":"2023-10-11T09:40:41.233361+0000","last_became_peered":"2023-10-11T09:40:41.233361+0000","last_unstale":"2023-10-11T09:40:42.911962+0000","last_undegraded":"2023-10-11T09:40:42.911962+0000","last_fullsized":"2023-10-11T09:40:42.911962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:12:09.780799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144249+0000","last_change":"2023-10-11T09:40:41.705452+0000","last_active":"2023-10-11T09:40:43.144249+0000","last_peered":"2023-10-11T09:40:43.144249+0000","last_clean":"2023-10-11T09:40:43.144249+0000","last_became_active":"2023-10-11T09:40:41.705028+0000","last_became_peered":"2023-10-11T09:40:41.705028+0000","last_unstale":"2023-10-11T09:40:43.144249+0000","last_undegraded":"2023-10-11T09:40:43.144249+0000","last_fullsized":"2023-10-11T09:40:43.144249+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:09:04.951739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145980+0000","last_change":"2023-10-11T09:40:41.232125+0000","last_active":"2023-10-11T09:40:43.145980+0000","last_peered":"2023-10-11T09:40:43.145980+0000","last_clean":"2023-10-11T09:40:43.145980+0000","last_became_active":"2023-10-11T09:40:41.231730+0000","last_became_peered":"2023-10-11T09:40:41.231730+0000","last_unstale":"2023-10-11T09:40:43.145980+0000","last_undegraded":"2023-10-11T09:40:43.145980+0000","last_fullsized":"2023-10-11T09:40:43.145980+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:12:04.722202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911689+0000","last_change":"2023-10-11T09:40:41.702062+0000","last_active":"2023-10-11T09:40:42.911689+0000","last_peered":"2023-10-11T09:40:42.911689+0000","last_clean":"2023-10-11T09:40:42.911689+0000","last_became_active":"2023-10-11T09:40:41.701665+0000","last_became_peered":"2023-10-11T09:40:41.701665+0000","last_unstale":"2023-10-11T09:40:42.911689+0000","last_undegraded":"2023-10-11T09:40:42.911689+0000","last_fullsized":"2023-10-11T09:40:42.911689+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:03:52.325886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.693906+0000","last_change":"2023-10-11T09:40:41.227957+0000","last_active":"2023-10-11T09:40:42.693906+0000","last_peered":"2023-10-11T09:40:42.693906+0000","last_clean":"2023-10-11T09:40:42.693906+0000","last_became_active":"2023-10-11T09:40:41.226855+0000","last_became_peered":"2023-10-11T09:40:41.226855+0000","last_unstale":"2023-10-11T09:40:42.693906+0000","last_undegraded":"2023-10-11T09:40:42.693906+0000","last_fullsized":"2023-10-11T09:40:42.693906+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:12:04.722202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911772+0000","last_change":"2023-10-11T09:40:41.699151+0000","last_active":"2023-10-11T09:40:42.911772+0000","last_peered":"2023-10-11T09:40:42.911772+0000","last_clean":"2023-10-11T09:40:42.911772+0000","last_became_active":"2023-10-11T09:40:41.698188+0000","last_became_peered":"2023-10-11T09:40:41.698188+0000","last_unstale":"2023-10-11T09:40:42.911772+0000","last_undegraded":"2023-10-11T09:40:42.911772+0000","last_fullsized":"2023-10-11T09:40:42.911772+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:48:15.689750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.694192+0000","last_change":"2023-10-11T09:40:41.224843+0000","last_active":"2023-10-11T09:40:42.694192+0000","last_peered":"2023-10-11T09:40:42.694192+0000","last_clean":"2023-10-11T09:40:42.694192+0000","last_became_active":"2023-10-11T09:40:41.224415+0000","last_became_peered":"2023-10-11T09:40:41.224415+0000","last_unstale":"2023-10-11T09:40:42.694192+0000","last_undegraded":"2023-10-11T09:40:42.694192+0000","last_fullsized":"2023-10-11T09:40:42.694192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:31:56.724263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687814+0000","last_change":"2023-10-11T09:40:41.695801+0000","last_active":"2023-10-11T09:40:42.687814+0000","last_peered":"2023-10-11T09:40:42.687814+0000","last_clean":"2023-10-11T09:40:42.687814+0000","last_became_active":"2023-10-11T09:40:41.694814+0000","last_became_peered":"2023-10-11T09:40:41.694814+0000","last_unstale":"2023-10-11T09:40:42.687814+0000","last_undegraded":"2023-10-11T09:40:42.687814+0000","last_fullsized":"2023-10-11T09:40:42.687814+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:19:07.089979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687947+0000","last_change":"2023-10-11T09:40:41.222156+0000","last_active":"2023-10-11T09:40:42.687947+0000","last_peered":"2023-10-11T09:40:42.687947+0000","last_clean":"2023-10-11T09:40:42.687947+0000","last_became_active":"2023-10-11T09:40:41.221758+0000","last_became_peered":"2023-10-11T09:40:41.221758+0000","last_unstale":"2023-10-11T09:40:42.687947+0000","last_undegraded":"2023-10-11T09:40:42.687947+0000","last_fullsized":"2023-10-11T09:40:42.687947+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:00:56.048004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911260+0000","last_change":"2023-10-11T09:40:41.729793+0000","last_active":"2023-10-11T09:40:42.911260+0000","last_peered":"2023-10-11T09:40:42.911260+0000","last_clean":"2023-10-11T09:40:42.911260+0000","last_became_active":"2023-10-11T09:40:41.729468+0000","last_became_peered":"2023-10-11T09:40:41.729468+0000","last_unstale":"2023-10-11T09:40:42.911260+0000","last_undegraded":"2023-10-11T09:40:42.911260+0000","last_fullsized":"2023-10-11T09:40:42.911260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:20:54.486033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.146157+0000","last_change":"2023-10-11T09:40:41.231067+0000","last_active":"2023-10-11T09:40:43.146157+0000","last_peered":"2023-10-11T09:40:43.146157+0000","last_clean":"2023-10-11T09:40:43.146157+0000","last_became_active":"2023-10-11T09:40:41.229252+0000","last_became_peered":"2023-10-11T09:40:41.229252+0000","last_unstale":"2023-10-11T09:40:43.146157+0000","last_undegraded":"2023-10-11T09:40:43.146157+0000","last_fullsized":"2023-10-11T09:40:43.146157+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:43:00.315992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":864256,"data_stored":681747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95699999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.363}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4430000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3360000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.468}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":229376,"data_stored":82946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1679999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3740000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2629999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4510000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.347}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":249856,"data_stored":90251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":262144,"data_stored":91219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.149}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3919999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.478}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.518}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":245760,"data_stored":89689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.478}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7390000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.587}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.637}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":229376,"data_stored":82946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.294}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.365}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.139}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.488}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3260000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":229376,"data_stored":82946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1519999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6890000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7470000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.714}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":827392,"data_stored":673896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.335}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3640000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.29}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.153}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-11T09:40:46.414 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-11T09:40:46.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.614+0000 7fe9b359e700 1 Processor -- start 2023-10-11T09:40:46.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.614+0000 7fe9b359e700 1 -- start start 2023-10-11T09:40:46.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.618+0000 7fe9b359e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe9b41644f0 0x7fe9b4160ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:46.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.618+0000 7fe9b359e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe9b40d20b0 0x7fe9b40d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:46.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.618+0000 7fe9b359e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9b4164a40 con 0x7fe9b4160ad0 2023-10-11T09:40:46.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.618+0000 7fe9b359e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9b40d2a90 con 0x7fe9b41644f0 2023-10-11T09:40:46.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.618+0000 7fe9b359e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b40cef90 con 0x7fe9b40d20b0 2023-10-11T09:40:46.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.618+0000 7fe9b1d9b700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe9b41644f0 0x7fe9b4160ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:46.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.618+0000 7fe9b2d9d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe9b40d20b0 0x7fe9b40d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:46.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.618+0000 7fe9b1d9b700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe9b41644f0 0x7fe9b4160ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:49820/0 (socket says 172.21.15.78:49820) 2023-10-11T09:40:46.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.618+0000 7fe9b1d9b700 1 -- 172.21.15.78:0/2394131536 learned_addr learned my addr 172.21.15.78:0/2394131536 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:46.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.618+0000 7fe9b1d9b700 1 -- 172.21.15.78:0/2394131536 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe9b40d20b0 msgr2=0x7fe9b40d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:46.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.618+0000 7fe9b1d9b700 1 --2- 172.21.15.78:0/2394131536 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe9b40d20b0 0x7fe9b40d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:46.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.618+0000 7fe9b1d9b700 1 -- 172.21.15.78:0/2394131536 >> v1:172.21.15.133:6789/0 conn(0x7fe9b4160ad0 legacy=0x7fe9b40ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:46.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.618+0000 7fe9b1d9b700 1 -- 172.21.15.78:0/2394131536 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b4164d20 con 0x7fe9b41644f0 2023-10-11T09:40:46.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.618+0000 7fe9b2d9d700 1 --2- 172.21.15.78:0/2394131536 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe9b40d20b0 0x7fe9b40d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:40:46.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.622+0000 7fe9b1d9b700 1 --2- 172.21.15.78:0/2394131536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9b41644f0 0x7fe9b4160ea0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8009910 tx=0x7fe9a8009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33844d3917c3359e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:46.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.622+0000 7fe9b159a700 1 -- 172.21.15.78:0/2394131536 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8011830 con 0x7fe9b41644f0 2023-10-11T09:40:46.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.622+0000 7fe9b159a700 1 -- 172.21.15.78:0/2394131536 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a8011990 con 0x7fe9b41644f0 2023-10-11T09:40:46.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.622+0000 7fe9b159a700 1 -- 172.21.15.78:0/2394131536 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8011d40 con 0x7fe9b41644f0 2023-10-11T09:40:46.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.622+0000 7fe9b359e700 1 -- 172.21.15.78:0/2394131536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9b41644f0 msgr2=0x7fe9b4160ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:46.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.622+0000 7fe9b359e700 1 --2- 172.21.15.78:0/2394131536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9b41644f0 0x7fe9b4160ea0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8009910 tx=0x7fe9a8009c20 comp rx=0 tx=0).stop 2023-10-11T09:40:46.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.622+0000 7fe9b359e700 1 -- 172.21.15.78:0/2394131536 shutdown_connections 2023-10-11T09:40:46.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.622+0000 7fe9b359e700 1 --2- 172.21.15.78:0/2394131536 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe9b40d20b0 0x7fe9b40d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:46.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.622+0000 7fe9b359e700 1 --2- 172.21.15.78:0/2394131536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9b41644f0 0x7fe9b4160ea0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:46.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.622+0000 7fe9b359e700 1 -- 172.21.15.78:0/2394131536 >> 172.21.15.78:0/2394131536 conn(0x7fe9b406afb0 msgr2=0x7fe9b406b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:46.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.622+0000 7fe9b359e700 1 -- 172.21.15.78:0/2394131536 shutdown_connections 2023-10-11T09:40:46.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.622+0000 7fe9b359e700 1 -- 172.21.15.78:0/2394131536 wait complete. 2023-10-11T09:40:46.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.626+0000 7fe9b359e700 1 Processor -- start 2023-10-11T09:40:46.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.626+0000 7fe9b359e700 1 -- start start 2023-10-11T09:40:46.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.626+0000 7fe9b359e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9b40d20b0 0x7fe9b414c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:46.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.626+0000 7fe9b359e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe9b4160ad0 0x7fe9b40c69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:46.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.626+0000 7fe9b359e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9b41644f0 0x7fe9b40c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:46.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.626+0000 7fe9b359e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b40b2bd0 con 0x7fe9b41644f0 2023-10-11T09:40:46.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.626+0000 7fe9b359e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b414c7f0 con 0x7fe9b40d20b0 2023-10-11T09:40:46.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.626+0000 7fe9b359e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b40b2880 con 0x7fe9b4160ad0 2023-10-11T09:40:46.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.626+0000 7fe9b2d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9b41644f0 0x7fe9b40c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:46.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.626+0000 7fe9b2d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9b41644f0 0x7fe9b40c84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56774/0 (socket says 172.21.15.78:56774) 2023-10-11T09:40:46.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.626+0000 7fe9b2d9d700 1 -- 172.21.15.78:0/649592246 learned_addr learned my addr 172.21.15.78:0/649592246 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:46.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.630+0000 7fe9b259c700 1 --2- 172.21.15.78:0/649592246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9b40d20b0 0x7fe9b414c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:46.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.630+0000 7fe9b1d9b700 1 --2- 172.21.15.78:0/649592246 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe9b4160ad0 0x7fe9b40c69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:46.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.630+0000 7fe9b2d9d700 1 -- 172.21.15.78:0/649592246 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe9b4160ad0 msgr2=0x7fe9b40c69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:46.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.630+0000 7fe9b2d9d700 1 --2- 172.21.15.78:0/649592246 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe9b4160ad0 0x7fe9b40c69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:46.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.630+0000 7fe9b2d9d700 1 -- 172.21.15.78:0/649592246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9b40d20b0 msgr2=0x7fe9b414c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:46.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.630+0000 7fe9b2d9d700 1 --2- 172.21.15.78:0/649592246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9b40d20b0 0x7fe9b414c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:46.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.630+0000 7fe9b2d9d700 1 -- 172.21.15.78:0/649592246 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ac009a70 con 0x7fe9b41644f0 2023-10-11T09:40:46.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.630+0000 7fe9b2d9d700 1 --2- 172.21.15.78:0/649592246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9b41644f0 0x7fe9b40c84a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac009f00 tx=0x7fe9ac00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:46.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.630+0000 7fe9a2ffd700 1 -- 172.21.15.78:0/649592246 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac009d70 con 0x7fe9b41644f0 2023-10-11T09:40:46.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.630+0000 7fe9a2ffd700 1 -- 172.21.15.78:0/649592246 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9ac00f040 con 0x7fe9b41644f0 2023-10-11T09:40:46.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.630+0000 7fe9b359e700 1 -- 172.21.15.78:0/649592246 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9a8009570 con 0x7fe9b41644f0 2023-10-11T09:40:46.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.630+0000 7fe9a2ffd700 1 -- 172.21.15.78:0/649592246 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac00cd70 con 0x7fe9b41644f0 2023-10-11T09:40:46.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.630+0000 7fe9b359e700 1 -- 172.21.15.78:0/649592246 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9b40c95c0 con 0x7fe9b41644f0 2023-10-11T09:40:46.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.630+0000 7fe9b359e700 1 -- 172.21.15.78:0/649592246 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9b40b2d10 con 0x7fe9b41644f0 2023-10-11T09:40:46.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.634+0000 7fe9a2ffd700 1 -- 172.21.15.78:0/649592246 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fe9ac014480 con 0x7fe9b41644f0 2023-10-11T09:40:46.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.634+0000 7fe9a2ffd700 1 --2- 172.21.15.78:0/649592246 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe994065ed0 0x7fe994068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:46.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.634+0000 7fe9a2ffd700 1 -- 172.21.15.78:0/649592246 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe9ac083de0 con 0x7fe9b41644f0 2023-10-11T09:40:46.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.634+0000 7fe9b259c700 1 --2- 172.21.15.78:0/649592246 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe994065ed0 0x7fe994068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:46.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.638+0000 7fe9b259c700 1 --2- 172.21.15.78:0/649592246 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe994065ed0 0x7fe994068380 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe9b40b57b0 tx=0x7fe9a4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:46.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.638+0000 7fe9a2ffd700 1 -- 172.21.15.78:0/649592246 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fe9b40b2d10 con 0x7fe9b41644f0 2023-10-11T09:40:46.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.806+0000 7fe9b359e700 1 -- 172.21.15.78:0/649592246 --> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe9b41652e0 con 0x7fe994065ed0 2023-10-11T09:40:46.819 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.814+0000 7fe9a2ffd700 1 -- 172.21.15.78:0/649592246 <== mgr.4101 v2:172.21.15.78:6832/35606 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286517 (secure 0 0 0) 0x7fe9a4001600 con 0x7fe994065ed0 2023-10-11T09:40:46.824 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:40:46.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.822+0000 7fe9b359e700 1 -- 172.21.15.78:0/649592246 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe994065ed0 msgr2=0x7fe994068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:46.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.822+0000 7fe9b359e700 1 --2- 172.21.15.78:0/649592246 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe994065ed0 0x7fe994068380 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe9b40b57b0 tx=0x7fe9a4009040 comp rx=0 tx=0).stop 2023-10-11T09:40:46.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.822+0000 7fe9b359e700 1 -- 172.21.15.78:0/649592246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9b41644f0 msgr2=0x7fe9b40c84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:46.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.822+0000 7fe9b359e700 1 --2- 172.21.15.78:0/649592246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9b41644f0 0x7fe9b40c84a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac009f00 tx=0x7fe9ac00c3a0 comp rx=0 tx=0).stop 2023-10-11T09:40:46.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.822+0000 7fe9b359e700 1 -- 172.21.15.78:0/649592246 shutdown_connections 2023-10-11T09:40:46.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.822+0000 7fe9b359e700 1 --2- 172.21.15.78:0/649592246 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe994065ed0 0x7fe994068380 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:46.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.822+0000 7fe9b359e700 1 --2- 172.21.15.78:0/649592246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9b41644f0 0x7fe9b40c84a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:46.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.822+0000 7fe9b359e700 1 --2- 172.21.15.78:0/649592246 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe9b4160ad0 0x7fe9b40c69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:46.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.822+0000 7fe9b359e700 1 --2- 172.21.15.78:0/649592246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9b40d20b0 0x7fe9b414c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:46.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.822+0000 7fe9b359e700 1 -- 172.21.15.78:0/649592246 >> 172.21.15.78:0/649592246 conn(0x7fe9b406afb0 msgr2=0x7fe9b415e680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:46.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.822+0000 7fe9b359e700 1 -- 172.21.15.78:0/649592246 shutdown_connections 2023-10-11T09:40:46.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:46.822+0000 7fe9b359e700 1 -- 172.21.15.78:0/649592246 wait complete. 2023-10-11T09:40:46.829 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2023-10-11T09:40:46.848 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-11T09:40:46.324161+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218680,"kb_used_data":3064,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754756040,"statfs":{"total":773094113280,"available":772870184960,"internally_reserved":0,"allocated":3137536,"data_stored":1875640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001919"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144669+0000","last_change":"2023-10-11T09:40:41.696400+0000","last_active":"2023-10-11T09:40:43.144669+0000","last_peered":"2023-10-11T09:40:43.144669+0000","last_clean":"2023-10-11T09:40:43.144669+0000","last_became_active":"2023-10-11T09:40:41.696101+0000","last_became_peered":"2023-10-11T09:40:41.696101+0000","last_unstale":"2023-10-11T09:40:43.144669+0000","last_undegraded":"2023-10-11T09:40:43.144669+0000","last_fullsized":"2023-10-11T09:40:43.144669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:43:01.331013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.694262+0000","last_change":"2023-10-11T09:40:41.719611+0000","last_active":"2023-10-11T09:40:42.694262+0000","last_peered":"2023-10-11T09:40:42.694262+0000","last_clean":"2023-10-11T09:40:42.694262+0000","last_became_active":"2023-10-11T09:40:41.719319+0000","last_became_peered":"2023-10-11T09:40:41.719319+0000","last_unstale":"2023-10-11T09:40:42.694262+0000","last_undegraded":"2023-10-11T09:40:42.694262+0000","last_fullsized":"2023-10-11T09:40:42.694262+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:07:02.302344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.912066+0000","last_change":"2023-10-11T09:40:41.701063+0000","last_active":"2023-10-11T09:40:42.912066+0000","last_peered":"2023-10-11T09:40:42.912066+0000","last_clean":"2023-10-11T09:40:42.912066+0000","last_became_active":"2023-10-11T09:40:41.699445+0000","last_became_peered":"2023-10-11T09:40:41.699445+0000","last_unstale":"2023-10-11T09:40:42.912066+0000","last_undegraded":"2023-10-11T09:40:42.912066+0000","last_fullsized":"2023-10-11T09:40:42.912066+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:19:44.476799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144161+0000","last_change":"2023-10-11T09:40:41.721932+0000","last_active":"2023-10-11T09:40:43.144161+0000","last_peered":"2023-10-11T09:40:43.144161+0000","last_clean":"2023-10-11T09:40:43.144161+0000","last_became_active":"2023-10-11T09:40:41.720602+0000","last_became_peered":"2023-10-11T09:40:41.720602+0000","last_unstale":"2023-10-11T09:40:43.144161+0000","last_undegraded":"2023-10-11T09:40:43.144161+0000","last_fullsized":"2023-10-11T09:40:43.144161+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:17:36.655491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145802+0000","last_change":"2023-10-11T09:40:41.710213+0000","last_active":"2023-10-11T09:40:43.145802+0000","last_peered":"2023-10-11T09:40:43.145802+0000","last_clean":"2023-10-11T09:40:43.145802+0000","last_became_active":"2023-10-11T09:40:41.707488+0000","last_became_peered":"2023-10-11T09:40:41.707488+0000","last_unstale":"2023-10-11T09:40:43.145802+0000","last_undegraded":"2023-10-11T09:40:43.145802+0000","last_fullsized":"2023-10-11T09:40:43.145802+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:42:34.821513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911583+0000","last_change":"2023-10-11T09:40:41.700595+0000","last_active":"2023-10-11T09:40:42.911583+0000","last_peered":"2023-10-11T09:40:42.911583+0000","last_clean":"2023-10-11T09:40:42.911583+0000","last_became_active":"2023-10-11T09:40:41.700239+0000","last_became_peered":"2023-10-11T09:40:41.700239+0000","last_unstale":"2023-10-11T09:40:42.911583+0000","last_undegraded":"2023-10-11T09:40:42.911583+0000","last_fullsized":"2023-10-11T09:40:42.911583+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:48:14.871150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.691894+0000","last_change":"2023-10-11T09:40:41.712870+0000","last_active":"2023-10-11T09:40:42.691894+0000","last_peered":"2023-10-11T09:40:42.691894+0000","last_clean":"2023-10-11T09:40:42.691894+0000","last_became_active":"2023-10-11T09:40:41.712593+0000","last_became_peered":"2023-10-11T09:40:41.712593+0000","last_unstale":"2023-10-11T09:40:42.691894+0000","last_undegraded":"2023-10-11T09:40:42.691894+0000","last_fullsized":"2023-10-11T09:40:42.691894+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:50:15.829941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.693734+0000","last_change":"2023-10-11T09:40:41.714890+0000","last_active":"2023-10-11T09:40:42.693734+0000","last_peered":"2023-10-11T09:40:42.693734+0000","last_clean":"2023-10-11T09:40:42.693734+0000","last_became_active":"2023-10-11T09:40:41.711116+0000","last_became_peered":"2023-10-11T09:40:41.711116+0000","last_unstale":"2023-10-11T09:40:42.693734+0000","last_undegraded":"2023-10-11T09:40:42.693734+0000","last_fullsized":"2023-10-11T09:40:42.693734+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:00:05.422788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.146063+0000","last_change":"2023-10-11T09:40:41.700768+0000","last_active":"2023-10-11T09:40:43.146063+0000","last_peered":"2023-10-11T09:40:43.146063+0000","last_clean":"2023-10-11T09:40:43.146063+0000","last_became_active":"2023-10-11T09:40:41.700440+0000","last_became_peered":"2023-10-11T09:40:41.700440+0000","last_unstale":"2023-10-11T09:40:43.146063+0000","last_undegraded":"2023-10-11T09:40:43.146063+0000","last_fullsized":"2023-10-11T09:40:43.146063+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:56:50.343276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.694372+0000","last_change":"2023-10-11T09:40:41.712701+0000","last_active":"2023-10-11T09:40:42.694372+0000","last_peered":"2023-10-11T09:40:42.694372+0000","last_clean":"2023-10-11T09:40:42.694372+0000","last_became_active":"2023-10-11T09:40:41.712349+0000","last_became_peered":"2023-10-11T09:40:41.712349+0000","last_unstale":"2023-10-11T09:40:42.694372+0000","last_undegraded":"2023-10-11T09:40:42.694372+0000","last_fullsized":"2023-10-11T09:40:42.694372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:42:34.821513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.912276+0000","last_change":"2023-10-11T09:40:41.706310+0000","last_active":"2023-10-11T09:40:42.912276+0000","last_peered":"2023-10-11T09:40:42.912276+0000","last_clean":"2023-10-11T09:40:42.912276+0000","last_became_active":"2023-10-11T09:40:41.705986+0000","last_became_peered":"2023-10-11T09:40:41.705986+0000","last_unstale":"2023-10-11T09:40:42.912276+0000","last_undegraded":"2023-10-11T09:40:42.912276+0000","last_fullsized":"2023-10-11T09:40:42.912276+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:59:46.864358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144038+0000","last_change":"2023-10-11T09:40:41.700251+0000","last_active":"2023-10-11T09:40:43.144038+0000","last_peered":"2023-10-11T09:40:43.144038+0000","last_clean":"2023-10-11T09:40:43.144038+0000","last_became_active":"2023-10-11T09:40:41.699961+0000","last_became_peered":"2023-10-11T09:40:41.699961+0000","last_unstale":"2023-10-11T09:40:43.144038+0000","last_undegraded":"2023-10-11T09:40:43.144038+0000","last_fullsized":"2023-10-11T09:40:43.144038+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:12:05.737223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.143594+0000","last_change":"2023-10-11T09:40:41.697448+0000","last_active":"2023-10-11T09:40:43.143594+0000","last_peered":"2023-10-11T09:40:43.143594+0000","last_clean":"2023-10-11T09:40:43.143594+0000","last_became_active":"2023-10-11T09:40:41.697160+0000","last_became_peered":"2023-10-11T09:40:41.697160+0000","last_unstale":"2023-10-11T09:40:43.143594+0000","last_undegraded":"2023-10-11T09:40:43.143594+0000","last_fullsized":"2023-10-11T09:40:43.143594+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:00:57.063025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.693705+0000","last_change":"2023-10-11T09:40:41.717312+0000","last_active":"2023-10-11T09:40:42.693705+0000","last_peered":"2023-10-11T09:40:42.693705+0000","last_clean":"2023-10-11T09:40:42.693705+0000","last_became_active":"2023-10-11T09:40:41.716622+0000","last_became_peered":"2023-10-11T09:40:41.716622+0000","last_unstale":"2023-10-11T09:40:42.693705+0000","last_undegraded":"2023-10-11T09:40:42.693705+0000","last_fullsized":"2023-10-11T09:40:42.693705+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:35:04.646798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.692225+0000","last_change":"2023-10-11T09:40:41.715325+0000","last_active":"2023-10-11T09:40:42.692225+0000","last_peered":"2023-10-11T09:40:42.692225+0000","last_clean":"2023-10-11T09:40:42.692225+0000","last_became_active":"2023-10-11T09:40:41.715033+0000","last_became_peered":"2023-10-11T09:40:41.715033+0000","last_unstale":"2023-10-11T09:40:42.692225+0000","last_undegraded":"2023-10-11T09:40:42.692225+0000","last_fullsized":"2023-10-11T09:40:42.692225+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:35:36.436236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911235+0000","last_change":"2023-10-11T09:40:41.699851+0000","last_active":"2023-10-11T09:40:42.911235+0000","last_peered":"2023-10-11T09:40:42.911235+0000","last_clean":"2023-10-11T09:40:42.911235+0000","last_became_active":"2023-10-11T09:40:41.699530+0000","last_became_peered":"2023-10-11T09:40:41.699530+0000","last_unstale":"2023-10-11T09:40:42.911235+0000","last_undegraded":"2023-10-11T09:40:42.911235+0000","last_fullsized":"2023-10-11T09:40:42.911235+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:23:30.814623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.146112+0000","last_change":"2023-10-11T09:40:41.700611+0000","last_active":"2023-10-11T09:40:43.146112+0000","last_peered":"2023-10-11T09:40:43.146112+0000","last_clean":"2023-10-11T09:40:43.146112+0000","last_became_active":"2023-10-11T09:40:41.700354+0000","last_became_peered":"2023-10-11T09:40:41.700354+0000","last_unstale":"2023-10-11T09:40:43.146112+0000","last_undegraded":"2023-10-11T09:40:43.146112+0000","last_fullsized":"2023-10-11T09:40:43.146112+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:48:15.689750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.912114+0000","last_change":"2023-10-11T09:40:41.700742+0000","last_active":"2023-10-11T09:40:42.912114+0000","last_peered":"2023-10-11T09:40:42.912114+0000","last_clean":"2023-10-11T09:40:42.912114+0000","last_became_active":"2023-10-11T09:40:41.700432+0000","last_became_peered":"2023-10-11T09:40:41.700432+0000","last_unstale":"2023-10-11T09:40:42.912114+0000","last_undegraded":"2023-10-11T09:40:42.912114+0000","last_fullsized":"2023-10-11T09:40:42.912114+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:33:13.162340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.143660+0000","last_change":"2023-10-11T09:40:41.719737+0000","last_active":"2023-10-11T09:40:43.143660+0000","last_peered":"2023-10-11T09:40:43.143660+0000","last_clean":"2023-10-11T09:40:43.143660+0000","last_became_active":"2023-10-11T09:40:41.719484+0000","last_became_peered":"2023-10-11T09:40:41.719484+0000","last_unstale":"2023-10-11T09:40:43.143660+0000","last_undegraded":"2023-10-11T09:40:43.143660+0000","last_fullsized":"2023-10-11T09:40:43.143660+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:48:52.897052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.912175+0000","last_change":"2023-10-11T09:40:41.728139+0000","last_active":"2023-10-11T09:40:42.912175+0000","last_peered":"2023-10-11T09:40:42.912175+0000","last_clean":"2023-10-11T09:40:42.912175+0000","last_became_active":"2023-10-11T09:40:41.727790+0000","last_became_peered":"2023-10-11T09:40:41.727790+0000","last_unstale":"2023-10-11T09:40:42.912175+0000","last_undegraded":"2023-10-11T09:40:42.912175+0000","last_fullsized":"2023-10-11T09:40:42.912175+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:07:58.571258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.692070+0000","last_change":"2023-10-11T09:40:41.713375+0000","last_active":"2023-10-11T09:40:42.692070+0000","last_peered":"2023-10-11T09:40:42.692070+0000","last_clean":"2023-10-11T09:40:42.692070+0000","last_became_active":"2023-10-11T09:40:41.713114+0000","last_became_peered":"2023-10-11T09:40:41.713114+0000","last_unstale":"2023-10-11T09:40:42.692070+0000","last_undegraded":"2023-10-11T09:40:42.692070+0000","last_fullsized":"2023-10-11T09:40:42.692070+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:07:02.302344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145102+0000","last_change":"2023-10-11T09:40:41.718540+0000","last_active":"2023-10-11T09:40:43.145102+0000","last_peered":"2023-10-11T09:40:43.145102+0000","last_clean":"2023-10-11T09:40:43.145102+0000","last_became_active":"2023-10-11T09:40:41.718309+0000","last_became_peered":"2023-10-11T09:40:41.718309+0000","last_unstale":"2023-10-11T09:40:43.145102+0000","last_undegraded":"2023-10-11T09:40:43.145102+0000","last_fullsized":"2023-10-11T09:40:43.145102+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:09:27.825336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687447+0000","last_change":"2023-10-11T09:40:41.699075+0000","last_active":"2023-10-11T09:40:42.687447+0000","last_peered":"2023-10-11T09:40:42.687447+0000","last_clean":"2023-10-11T09:40:42.687447+0000","last_became_active":"2023-10-11T09:40:41.698747+0000","last_became_peered":"2023-10-11T09:40:41.698747+0000","last_unstale":"2023-10-11T09:40:42.687447+0000","last_undegraded":"2023-10-11T09:40:42.687447+0000","last_fullsized":"2023-10-11T09:40:42.687447+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:09:27.825336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.691897+0000","last_change":"2023-10-11T09:40:41.714225+0000","last_active":"2023-10-11T09:40:42.691897+0000","last_peered":"2023-10-11T09:40:42.691897+0000","last_clean":"2023-10-11T09:40:42.691897+0000","last_became_active":"2023-10-11T09:40:41.710262+0000","last_became_peered":"2023-10-11T09:40:41.710262+0000","last_unstale":"2023-10-11T09:40:42.691897+0000","last_undegraded":"2023-10-11T09:40:42.691897+0000","last_fullsized":"2023-10-11T09:40:42.691897+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:42:34.821513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144675+0000","last_change":"2023-10-11T09:40:41.729722+0000","last_active":"2023-10-11T09:40:43.144675+0000","last_peered":"2023-10-11T09:40:43.144675+0000","last_clean":"2023-10-11T09:40:43.144675+0000","last_became_active":"2023-10-11T09:40:41.729360+0000","last_became_peered":"2023-10-11T09:40:41.729360+0000","last_unstale":"2023-10-11T09:40:43.144675+0000","last_undegraded":"2023-10-11T09:40:43.144675+0000","last_fullsized":"2023-10-11T09:40:43.144675+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:07:02.302344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145219+0000","last_change":"2023-10-11T09:40:41.709354+0000","last_active":"2023-10-11T09:40:43.145219+0000","last_peered":"2023-10-11T09:40:43.145219+0000","last_clean":"2023-10-11T09:40:43.145219+0000","last_became_active":"2023-10-11T09:40:41.709096+0000","last_became_peered":"2023-10-11T09:40:41.709096+0000","last_unstale":"2023-10-11T09:40:43.145219+0000","last_undegraded":"2023-10-11T09:40:43.145219+0000","last_fullsized":"2023-10-11T09:40:43.145219+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:35:04.646798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.694072+0000","last_change":"2023-10-11T09:40:41.723510+0000","last_active":"2023-10-11T09:40:42.694072+0000","last_peered":"2023-10-11T09:40:42.694072+0000","last_clean":"2023-10-11T09:40:42.694072+0000","last_became_active":"2023-10-11T09:40:41.723110+0000","last_became_peered":"2023-10-11T09:40:41.723110+0000","last_unstale":"2023-10-11T09:40:42.694072+0000","last_undegraded":"2023-10-11T09:40:42.694072+0000","last_fullsized":"2023-10-11T09:40:42.694072+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:50:15.829941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145120+0000","last_change":"2023-10-11T09:40:41.714481+0000","last_active":"2023-10-11T09:40:43.145120+0000","last_peered":"2023-10-11T09:40:43.145120+0000","last_clean":"2023-10-11T09:40:43.145120+0000","last_became_active":"2023-10-11T09:40:41.714219+0000","last_became_peered":"2023-10-11T09:40:41.714219+0000","last_unstale":"2023-10-11T09:40:43.145120+0000","last_undegraded":"2023-10-11T09:40:43.145120+0000","last_fullsized":"2023-10-11T09:40:43.145120+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:19:07.089979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.688002+0000","last_change":"2023-10-11T09:40:41.709612+0000","last_active":"2023-10-11T09:40:42.688002+0000","last_peered":"2023-10-11T09:40:42.688002+0000","last_clean":"2023-10-11T09:40:42.688002+0000","last_became_active":"2023-10-11T09:40:41.709317+0000","last_became_peered":"2023-10-11T09:40:41.709317+0000","last_unstale":"2023-10-11T09:40:42.688002+0000","last_undegraded":"2023-10-11T09:40:42.688002+0000","last_fullsized":"2023-10-11T09:40:42.688002+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:35:36.436236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911908+0000","last_change":"2023-10-11T09:40:41.727354+0000","last_active":"2023-10-11T09:40:42.911908+0000","last_peered":"2023-10-11T09:40:42.911908+0000","last_clean":"2023-10-11T09:40:42.911908+0000","last_became_active":"2023-10-11T09:40:41.727039+0000","last_became_peered":"2023-10-11T09:40:41.727039+0000","last_unstale":"2023-10-11T09:40:42.911908+0000","last_undegraded":"2023-10-11T09:40:42.911908+0000","last_fullsized":"2023-10-11T09:40:42.911908+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:01:36.923947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687557+0000","last_change":"2023-10-11T09:40:41.712971+0000","last_active":"2023-10-11T09:40:42.687557+0000","last_peered":"2023-10-11T09:40:42.687557+0000","last_clean":"2023-10-11T09:40:42.687557+0000","last_became_active":"2023-10-11T09:40:41.712707+0000","last_became_peered":"2023-10-11T09:40:41.712707+0000","last_unstale":"2023-10-11T09:40:42.687557+0000","last_undegraded":"2023-10-11T09:40:42.687557+0000","last_fullsized":"2023-10-11T09:40:42.687557+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:48:15.689750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.143927+0000","last_change":"2023-10-11T09:40:41.728941+0000","last_active":"2023-10-11T09:40:43.143927+0000","last_peered":"2023-10-11T09:40:43.143927+0000","last_clean":"2023-10-11T09:40:43.143927+0000","last_became_active":"2023-10-11T09:40:41.728657+0000","last_became_peered":"2023-10-11T09:40:41.728657+0000","last_unstale":"2023-10-11T09:40:43.143927+0000","last_undegraded":"2023-10-11T09:40:43.143927+0000","last_fullsized":"2023-10-11T09:40:43.143927+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:12:05.737223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.688259+0000","last_change":"2023-10-11T09:40:41.718662+0000","last_active":"2023-10-11T09:40:42.688259+0000","last_peered":"2023-10-11T09:40:42.688259+0000","last_clean":"2023-10-11T09:40:42.688259+0000","last_became_active":"2023-10-11T09:40:41.718402+0000","last_became_peered":"2023-10-11T09:40:41.718402+0000","last_unstale":"2023-10-11T09:40:42.688259+0000","last_undegraded":"2023-10-11T09:40:42.688259+0000","last_fullsized":"2023-10-11T09:40:42.688259+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:56:50.343276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.692319+0000","last_change":"2023-10-11T09:40:41.234649+0000","last_active":"2023-10-11T09:40:42.692319+0000","last_peered":"2023-10-11T09:40:42.692319+0000","last_clean":"2023-10-11T09:40:42.692319+0000","last_became_active":"2023-10-11T09:40:41.234006+0000","last_became_peered":"2023-10-11T09:40:41.234006+0000","last_unstale":"2023-10-11T09:40:42.692319+0000","last_undegraded":"2023-10-11T09:40:42.692319+0000","last_fullsized":"2023-10-11T09:40:42.692319+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:45:39.083797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.694114+0000","last_change":"2023-10-11T09:40:41.723234+0000","last_active":"2023-10-11T09:40:42.694114+0000","last_peered":"2023-10-11T09:40:42.694114+0000","last_clean":"2023-10-11T09:40:42.694114+0000","last_became_active":"2023-10-11T09:40:41.722928+0000","last_became_peered":"2023-10-11T09:40:41.722928+0000","last_unstale":"2023-10-11T09:40:42.694114+0000","last_undegraded":"2023-10-11T09:40:42.694114+0000","last_fullsized":"2023-10-11T09:40:42.694114+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:09:27.825336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145323+0000","last_change":"2023-10-11T09:40:41.225247+0000","last_active":"2023-10-11T09:40:43.145323+0000","last_peered":"2023-10-11T09:40:43.145323+0000","last_clean":"2023-10-11T09:40:43.145323+0000","last_became_active":"2023-10-11T09:40:41.224967+0000","last_became_peered":"2023-10-11T09:40:41.224967+0000","last_unstale":"2023-10-11T09:40:43.145323+0000","last_undegraded":"2023-10-11T09:40:43.145323+0000","last_fullsized":"2023-10-11T09:40:43.145323+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:33:10.649755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145747+0000","last_change":"2023-10-11T09:40:41.703018+0000","last_active":"2023-10-11T09:40:43.145747+0000","last_peered":"2023-10-11T09:40:43.145747+0000","last_clean":"2023-10-11T09:40:43.145747+0000","last_became_active":"2023-10-11T09:40:41.702729+0000","last_became_peered":"2023-10-11T09:40:41.702729+0000","last_unstale":"2023-10-11T09:40:43.145747+0000","last_undegraded":"2023-10-11T09:40:43.145747+0000","last_fullsized":"2023-10-11T09:40:43.145747+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:07:02.302344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.692298+0000","last_change":"2023-10-11T09:40:41.234929+0000","last_active":"2023-10-11T09:40:42.692298+0000","last_peered":"2023-10-11T09:40:42.692298+0000","last_clean":"2023-10-11T09:40:42.692298+0000","last_became_active":"2023-10-11T09:40:41.234149+0000","last_became_peered":"2023-10-11T09:40:41.234149+0000","last_unstale":"2023-10-11T09:40:42.692298+0000","last_undegraded":"2023-10-11T09:40:42.692298+0000","last_fullsized":"2023-10-11T09:40:42.692298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:17:35.640470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.688569+0000","last_change":"2023-10-11T09:40:41.716719+0000","last_active":"2023-10-11T09:40:42.688569+0000","last_peered":"2023-10-11T09:40:42.688569+0000","last_clean":"2023-10-11T09:40:42.688569+0000","last_became_active":"2023-10-11T09:40:41.716181+0000","last_became_peered":"2023-10-11T09:40:41.716181+0000","last_unstale":"2023-10-11T09:40:42.688569+0000","last_undegraded":"2023-10-11T09:40:42.688569+0000","last_fullsized":"2023-10-11T09:40:42.688569+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:50:15.829941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.143863+0000","last_change":"2023-10-11T09:40:41.229549+0000","last_active":"2023-10-11T09:40:43.143863+0000","last_peered":"2023-10-11T09:40:43.143863+0000","last_clean":"2023-10-11T09:40:43.143863+0000","last_became_active":"2023-10-11T09:40:41.229259+0000","last_became_peered":"2023-10-11T09:40:41.229259+0000","last_unstale":"2023-10-11T09:40:43.143863+0000","last_undegraded":"2023-10-11T09:40:43.143863+0000","last_fullsized":"2023-10-11T09:40:43.143863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:16:37.265056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.692895+0000","last_change":"2023-10-11T09:40:41.234477+0000","last_active":"2023-10-11T09:40:42.692895+0000","last_peered":"2023-10-11T09:40:42.692895+0000","last_clean":"2023-10-11T09:40:42.692895+0000","last_became_active":"2023-10-11T09:40:41.233977+0000","last_became_peered":"2023-10-11T09:40:41.233977+0000","last_unstale":"2023-10-11T09:40:42.692895+0000","last_undegraded":"2023-10-11T09:40:42.692895+0000","last_fullsized":"2023-10-11T09:40:42.692895+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:47:25.022675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.693696+0000","last_change":"2023-10-11T09:40:41.717799+0000","last_active":"2023-10-11T09:40:42.693696+0000","last_peered":"2023-10-11T09:40:42.693696+0000","last_clean":"2023-10-11T09:40:42.693696+0000","last_became_active":"2023-10-11T09:40:41.717520+0000","last_became_peered":"2023-10-11T09:40:41.717520+0000","last_unstale":"2023-10-11T09:40:42.693696+0000","last_undegraded":"2023-10-11T09:40:42.693696+0000","last_fullsized":"2023-10-11T09:40:42.693696+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:35:36.436236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.692703+0000","last_change":"2023-10-11T09:40:41.234979+0000","last_active":"2023-10-11T09:40:42.692703+0000","last_peered":"2023-10-11T09:40:42.692703+0000","last_clean":"2023-10-11T09:40:42.692703+0000","last_became_active":"2023-10-11T09:40:41.234245+0000","last_became_peered":"2023-10-11T09:40:41.234245+0000","last_unstale":"2023-10-11T09:40:42.692703+0000","last_undegraded":"2023-10-11T09:40:42.692703+0000","last_fullsized":"2023-10-11T09:40:42.692703+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:31:56.724263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911330+0000","last_change":"2023-10-11T09:40:41.706936+0000","last_active":"2023-10-11T09:40:42.911330+0000","last_peered":"2023-10-11T09:40:42.911330+0000","last_clean":"2023-10-11T09:40:42.911330+0000","last_became_active":"2023-10-11T09:40:41.706440+0000","last_became_peered":"2023-10-11T09:40:41.706440+0000","last_unstale":"2023-10-11T09:40:42.911330+0000","last_undegraded":"2023-10-11T09:40:42.911330+0000","last_fullsized":"2023-10-11T09:40:42.911330+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:12:14.951424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.692744+0000","last_change":"2023-10-11T09:40:41.228740+0000","last_active":"2023-10-11T09:40:42.692744+0000","last_peered":"2023-10-11T09:40:42.692744+0000","last_clean":"2023-10-11T09:40:42.692744+0000","last_became_active":"2023-10-11T09:40:41.228327+0000","last_became_peered":"2023-10-11T09:40:41.228327+0000","last_unstale":"2023-10-11T09:40:42.692744+0000","last_undegraded":"2023-10-11T09:40:42.692744+0000","last_fullsized":"2023-10-11T09:40:42.692744+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:59:14.883485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144214+0000","last_change":"2023-10-11T09:40:41.721661+0000","last_active":"2023-10-11T09:40:43.144214+0000","last_peered":"2023-10-11T09:40:43.144214+0000","last_clean":"2023-10-11T09:40:43.144214+0000","last_became_active":"2023-10-11T09:40:41.720154+0000","last_became_peered":"2023-10-11T09:40:41.720154+0000","last_unstale":"2023-10-11T09:40:43.144214+0000","last_undegraded":"2023-10-11T09:40:43.144214+0000","last_fullsized":"2023-10-11T09:40:43.144214+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:45:40.098819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.693986+0000","last_change":"2023-10-11T09:40:41.225444+0000","last_active":"2023-10-11T09:40:42.693986+0000","last_peered":"2023-10-11T09:40:42.693986+0000","last_clean":"2023-10-11T09:40:42.693986+0000","last_became_active":"2023-10-11T09:40:41.225159+0000","last_became_peered":"2023-10-11T09:40:41.225159+0000","last_unstale":"2023-10-11T09:40:42.693986+0000","last_undegraded":"2023-10-11T09:40:42.693986+0000","last_fullsized":"2023-10-11T09:40:42.693986+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:00:56.048004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145496+0000","last_change":"2023-10-11T09:40:41.712920+0000","last_active":"2023-10-11T09:40:43.145496+0000","last_peered":"2023-10-11T09:40:43.145496+0000","last_clean":"2023-10-11T09:40:43.145496+0000","last_became_active":"2023-10-11T09:40:41.712654+0000","last_became_peered":"2023-10-11T09:40:41.712654+0000","last_unstale":"2023-10-11T09:40:43.145496+0000","last_undegraded":"2023-10-11T09:40:43.145496+0000","last_fullsized":"2023-10-11T09:40:43.145496+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:23:30.814623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.688154+0000","last_change":"2023-10-11T09:40:41.229300+0000","last_active":"2023-10-11T09:40:42.688154+0000","last_peered":"2023-10-11T09:40:42.688154+0000","last_clean":"2023-10-11T09:40:42.688154+0000","last_became_active":"2023-10-11T09:40:41.228539+0000","last_became_peered":"2023-10-11T09:40:41.228539+0000","last_unstale":"2023-10-11T09:40:42.688154+0000","last_undegraded":"2023-10-11T09:40:42.688154+0000","last_fullsized":"2023-10-11T09:40:42.688154+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:09:03.936717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.912449+0000","last_change":"2023-10-11T09:40:41.705540+0000","last_active":"2023-10-11T09:40:42.912449+0000","last_peered":"2023-10-11T09:40:42.912449+0000","last_clean":"2023-10-11T09:40:42.912449+0000","last_became_active":"2023-10-11T09:40:41.705163+0000","last_became_peered":"2023-10-11T09:40:41.705163+0000","last_unstale":"2023-10-11T09:40:42.912449+0000","last_undegraded":"2023-10-11T09:40:42.912449+0000","last_fullsized":"2023-10-11T09:40:42.912449+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:56:50.343276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144408+0000","last_change":"2023-10-11T09:40:41.224108+0000","last_active":"2023-10-11T09:40:43.144408+0000","last_peered":"2023-10-11T09:40:43.144408+0000","last_clean":"2023-10-11T09:40:43.144408+0000","last_became_active":"2023-10-11T09:40:41.223643+0000","last_became_peered":"2023-10-11T09:40:41.223643+0000","last_unstale":"2023-10-11T09:40:43.144408+0000","last_undegraded":"2023-10-11T09:40:43.144408+0000","last_fullsized":"2023-10-11T09:40:43.144408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:36:34.713602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.912393+0000","last_change":"2023-10-11T09:40:41.700784+0000","last_active":"2023-10-11T09:40:42.912393+0000","last_peered":"2023-10-11T09:40:42.912393+0000","last_clean":"2023-10-11T09:40:42.912393+0000","last_became_active":"2023-10-11T09:40:41.700495+0000","last_became_peered":"2023-10-11T09:40:41.700495+0000","last_unstale":"2023-10-11T09:40:42.912393+0000","last_undegraded":"2023-10-11T09:40:42.912393+0000","last_fullsized":"2023-10-11T09:40:42.912393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:57:16.072435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911546+0000","last_change":"2023-10-11T09:40:41.233232+0000","last_active":"2023-10-11T09:40:42.911546+0000","last_peered":"2023-10-11T09:40:42.911546+0000","last_clean":"2023-10-11T09:40:42.911546+0000","last_became_active":"2023-10-11T09:40:41.232092+0000","last_became_peered":"2023-10-11T09:40:41.232092+0000","last_unstale":"2023-10-11T09:40:42.911546+0000","last_undegraded":"2023-10-11T09:40:42.911546+0000","last_fullsized":"2023-10-11T09:40:42.911546+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:16:37.265056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.912288+0000","last_change":"2023-10-11T09:40:41.721557+0000","last_active":"2023-10-11T09:40:42.912288+0000","last_peered":"2023-10-11T09:40:42.912288+0000","last_clean":"2023-10-11T09:40:42.912288+0000","last_became_active":"2023-10-11T09:40:41.721314+0000","last_became_peered":"2023-10-11T09:40:41.721314+0000","last_unstale":"2023-10-11T09:40:42.912288+0000","last_undegraded":"2023-10-11T09:40:42.912288+0000","last_fullsized":"2023-10-11T09:40:42.912288+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:59:15.898506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687786+0000","last_change":"2023-10-11T09:40:41.228065+0000","last_active":"2023-10-11T09:40:42.687786+0000","last_peered":"2023-10-11T09:40:42.687786+0000","last_clean":"2023-10-11T09:40:42.687786+0000","last_became_active":"2023-10-11T09:40:41.227765+0000","last_became_peered":"2023-10-11T09:40:41.227765+0000","last_unstale":"2023-10-11T09:40:42.687786+0000","last_undegraded":"2023-10-11T09:40:42.687786+0000","last_fullsized":"2023-10-11T09:40:42.687786+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:36:34.713602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144923+0000","last_change":"2023-10-11T09:40:41.708864+0000","last_active":"2023-10-11T09:40:43.144923+0000","last_peered":"2023-10-11T09:40:43.144923+0000","last_clean":"2023-10-11T09:40:43.144923+0000","last_became_active":"2023-10-11T09:40:41.708555+0000","last_became_peered":"2023-10-11T09:40:41.708555+0000","last_unstale":"2023-10-11T09:40:43.144923+0000","last_undegraded":"2023-10-11T09:40:43.144923+0000","last_fullsized":"2023-10-11T09:40:43.144923+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:43:30.691251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":167,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-11T09:40:41.695042+0000","last_change":"2023-10-11T09:40:29.874372+0000","last_active":"2023-10-11T09:40:41.695042+0000","last_peered":"2023-10-11T09:40:41.695042+0000","last_clean":"2023-10-11T09:40:41.695042+0000","last_became_active":"2023-10-11T09:40:29.873621+0000","last_became_peered":"2023-10-11T09:40:29.873621+0000","last_unstale":"2023-10-11T09:40:41.695042+0000","last_undegraded":"2023-10-11T09:40:41.695042+0000","last_fullsized":"2023-10-11T09:40:41.695042+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_clean_scrub_stamp":"2023-10-11T09:40:28.742276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:22:07.985311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-11T09:40:41.705703+0000","last_change":"2023-10-11T09:40:36.148773+0000","last_active":"2023-10-11T09:40:41.705703+0000","last_peered":"2023-10-11T09:40:41.705703+0000","last_clean":"2023-10-11T09:40:41.705703+0000","last_became_active":"2023-10-11T09:40:36.148489+0000","last_became_peered":"2023-10-11T09:40:36.148489+0000","last_unstale":"2023-10-11T09:40:41.705703+0000","last_undegraded":"2023-10-11T09:40:41.705703+0000","last_fullsized":"2023-10-11T09:40:41.705703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687547+0000","last_change":"2023-10-11T09:40:41.227668+0000","last_active":"2023-10-11T09:40:42.687547+0000","last_peered":"2023-10-11T09:40:42.687547+0000","last_clean":"2023-10-11T09:40:42.687547+0000","last_became_active":"2023-10-11T09:40:41.227357+0000","last_became_peered":"2023-10-11T09:40:41.227357+0000","last_unstale":"2023-10-11T09:40:42.687547+0000","last_undegraded":"2023-10-11T09:40:42.687547+0000","last_fullsized":"2023-10-11T09:40:42.687547+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:16:37.265056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144334+0000","last_change":"2023-10-11T09:40:41.726684+0000","last_active":"2023-10-11T09:40:43.144334+0000","last_peered":"2023-10-11T09:40:43.144334+0000","last_clean":"2023-10-11T09:40:43.144334+0000","last_became_active":"2023-10-11T09:40:41.726323+0000","last_became_peered":"2023-10-11T09:40:41.726323+0000","last_unstale":"2023-10-11T09:40:43.144334+0000","last_undegraded":"2023-10-11T09:40:43.144334+0000","last_fullsized":"2023-10-11T09:40:43.144334+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:47:26.037696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-11T09:40:41.694934+0000","last_change":"2023-10-11T09:40:36.149739+0000","last_active":"2023-10-11T09:40:41.694934+0000","last_peered":"2023-10-11T09:40:41.694934+0000","last_clean":"2023-10-11T09:40:41.694934+0000","last_became_active":"2023-10-11T09:40:36.149419+0000","last_became_peered":"2023-10-11T09:40:36.149419+0000","last_unstale":"2023-10-11T09:40:41.694934+0000","last_undegraded":"2023-10-11T09:40:41.694934+0000","last_fullsized":"2023-10-11T09:40:41.694934+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:54:50.640362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.143706+0000","last_change":"2023-10-11T09:40:41.224529+0000","last_active":"2023-10-11T09:40:43.143706+0000","last_peered":"2023-10-11T09:40:43.143706+0000","last_clean":"2023-10-11T09:40:43.143706+0000","last_became_active":"2023-10-11T09:40:41.224252+0000","last_became_peered":"2023-10-11T09:40:41.224252+0000","last_unstale":"2023-10-11T09:40:43.143706+0000","last_undegraded":"2023-10-11T09:40:43.143706+0000","last_fullsized":"2023-10-11T09:40:43.143706+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:09:03.936717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145788+0000","last_change":"2023-10-11T09:40:41.718539+0000","last_active":"2023-10-11T09:40:43.145788+0000","last_peered":"2023-10-11T09:40:43.145788+0000","last_clean":"2023-10-11T09:40:43.145788+0000","last_became_active":"2023-10-11T09:40:41.718231+0000","last_became_peered":"2023-10-11T09:40:41.718231+0000","last_unstale":"2023-10-11T09:40:43.145788+0000","last_undegraded":"2023-10-11T09:40:43.145788+0000","last_fullsized":"2023-10-11T09:40:43.145788+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:50:15.829941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-11T09:40:41.682824+0000","last_change":"2023-10-11T09:40:36.216248+0000","last_active":"2023-10-11T09:40:41.682824+0000","last_peered":"2023-10-11T09:40:41.682824+0000","last_clean":"2023-10-11T09:40:41.682824+0000","last_became_active":"2023-10-11T09:40:36.215890+0000","last_became_peered":"2023-10-11T09:40:36.215890+0000","last_unstale":"2023-10-11T09:40:41.682824+0000","last_undegraded":"2023-10-11T09:40:41.682824+0000","last_fullsized":"2023-10-11T09:40:41.682824+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.143998+0000","last_change":"2023-10-11T09:40:41.220987+0000","last_active":"2023-10-11T09:40:43.143998+0000","last_peered":"2023-10-11T09:40:43.143998+0000","last_clean":"2023-10-11T09:40:43.143998+0000","last_became_active":"2023-10-11T09:40:41.220035+0000","last_became_peered":"2023-10-11T09:40:41.220035+0000","last_unstale":"2023-10-11T09:40:43.143998+0000","last_undegraded":"2023-10-11T09:40:43.143998+0000","last_fullsized":"2023-10-11T09:40:43.143998+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:00:56.048004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144958+0000","last_change":"2023-10-11T09:40:41.718356+0000","last_active":"2023-10-11T09:40:43.144958+0000","last_peered":"2023-10-11T09:40:43.144958+0000","last_clean":"2023-10-11T09:40:43.144958+0000","last_became_active":"2023-10-11T09:40:41.718069+0000","last_became_peered":"2023-10-11T09:40:41.718069+0000","last_unstale":"2023-10-11T09:40:43.144958+0000","last_undegraded":"2023-10-11T09:40:43.144958+0000","last_fullsized":"2023-10-11T09:40:43.144958+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:42:34.821513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-11T09:40:41.705511+0000","last_change":"2023-10-11T09:40:36.151710+0000","last_active":"2023-10-11T09:40:41.705511+0000","last_peered":"2023-10-11T09:40:41.705511+0000","last_clean":"2023-10-11T09:40:41.705511+0000","last_became_active":"2023-10-11T09:40:36.151219+0000","last_became_peered":"2023-10-11T09:40:36.151219+0000","last_unstale":"2023-10-11T09:40:41.705511+0000","last_undegraded":"2023-10-11T09:40:41.705511+0000","last_fullsized":"2023-10-11T09:40:41.705511+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911785+0000","last_change":"2023-10-11T09:40:41.231464+0000","last_active":"2023-10-11T09:40:42.911785+0000","last_peered":"2023-10-11T09:40:42.911785+0000","last_clean":"2023-10-11T09:40:42.911785+0000","last_became_active":"2023-10-11T09:40:41.229072+0000","last_became_peered":"2023-10-11T09:40:41.229072+0000","last_unstale":"2023-10-11T09:40:42.911785+0000","last_undegraded":"2023-10-11T09:40:42.911785+0000","last_fullsized":"2023-10-11T09:40:42.911785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:33:10.649755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144540+0000","last_change":"2023-10-11T09:40:41.715638+0000","last_active":"2023-10-11T09:40:43.144540+0000","last_peered":"2023-10-11T09:40:43.144540+0000","last_clean":"2023-10-11T09:40:43.144540+0000","last_became_active":"2023-10-11T09:40:41.715399+0000","last_became_peered":"2023-10-11T09:40:41.715399+0000","last_unstale":"2023-10-11T09:40:43.144540+0000","last_undegraded":"2023-10-11T09:40:43.144540+0000","last_fullsized":"2023-10-11T09:40:43.144540+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:00:05.422788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.694217+0000","last_change":"2023-10-11T09:40:41.231735+0000","last_active":"2023-10-11T09:40:42.694217+0000","last_peered":"2023-10-11T09:40:42.694217+0000","last_clean":"2023-10-11T09:40:42.694217+0000","last_became_active":"2023-10-11T09:40:41.229777+0000","last_became_peered":"2023-10-11T09:40:41.229777+0000","last_unstale":"2023-10-11T09:40:42.694217+0000","last_undegraded":"2023-10-11T09:40:42.694217+0000","last_fullsized":"2023-10-11T09:40:42.694217+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:09:03.936717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-11T09:40:41.695209+0000","last_change":"2023-10-11T09:40:36.160620+0000","last_active":"2023-10-11T09:40:41.695209+0000","last_peered":"2023-10-11T09:40:41.695209+0000","last_clean":"2023-10-11T09:40:41.695209+0000","last_became_active":"2023-10-11T09:40:36.160171+0000","last_became_peered":"2023-10-11T09:40:36.160171+0000","last_unstale":"2023-10-11T09:40:41.695209+0000","last_undegraded":"2023-10-11T09:40:41.695209+0000","last_fullsized":"2023-10-11T09:40:41.695209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145219+0000","last_change":"2023-10-11T09:40:41.718869+0000","last_active":"2023-10-11T09:40:43.145219+0000","last_peered":"2023-10-11T09:40:43.145219+0000","last_clean":"2023-10-11T09:40:43.145219+0000","last_became_active":"2023-10-11T09:40:41.718628+0000","last_became_peered":"2023-10-11T09:40:41.718628+0000","last_unstale":"2023-10-11T09:40:43.145219+0000","last_undegraded":"2023-10-11T09:40:43.145219+0000","last_fullsized":"2023-10-11T09:40:43.145219+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:50:15.829941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144700+0000","last_change":"2023-10-11T09:40:41.225562+0000","last_active":"2023-10-11T09:40:43.144700+0000","last_peered":"2023-10-11T09:40:43.144700+0000","last_clean":"2023-10-11T09:40:43.144700+0000","last_became_active":"2023-10-11T09:40:41.222909+0000","last_became_peered":"2023-10-11T09:40:41.222909+0000","last_unstale":"2023-10-11T09:40:43.144700+0000","last_undegraded":"2023-10-11T09:40:43.144700+0000","last_fullsized":"2023-10-11T09:40:43.144700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:09:03.936717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-11T09:40:41.696680+0000","last_change":"2023-10-11T09:40:36.162174+0000","last_active":"2023-10-11T09:40:41.696680+0000","last_peered":"2023-10-11T09:40:41.696680+0000","last_clean":"2023-10-11T09:40:41.696680+0000","last_became_active":"2023-10-11T09:40:36.161710+0000","last_became_peered":"2023-10-11T09:40:36.161710+0000","last_unstale":"2023-10-11T09:40:41.696680+0000","last_undegraded":"2023-10-11T09:40:41.696680+0000","last_fullsized":"2023-10-11T09:40:41.696680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:57:03.630878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145842+0000","last_change":"2023-10-11T09:40:41.716886+0000","last_active":"2023-10-11T09:40:43.145842+0000","last_peered":"2023-10-11T09:40:43.145842+0000","last_clean":"2023-10-11T09:40:43.145842+0000","last_became_active":"2023-10-11T09:40:41.716601+0000","last_became_peered":"2023-10-11T09:40:41.716601+0000","last_unstale":"2023-10-11T09:40:43.145842+0000","last_undegraded":"2023-10-11T09:40:43.145842+0000","last_fullsized":"2023-10-11T09:40:43.145842+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:00:05.422788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.688690+0000","last_change":"2023-10-11T09:40:41.224655+0000","last_active":"2023-10-11T09:40:42.688690+0000","last_peered":"2023-10-11T09:40:42.688690+0000","last_clean":"2023-10-11T09:40:42.688690+0000","last_became_active":"2023-10-11T09:40:41.224374+0000","last_became_peered":"2023-10-11T09:40:41.224374+0000","last_unstale":"2023-10-11T09:40:42.688690+0000","last_undegraded":"2023-10-11T09:40:42.688690+0000","last_fullsized":"2023-10-11T09:40:42.688690+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:33:10.649755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-11T09:40:41.699405+0000","last_change":"2023-10-11T09:40:36.150224+0000","last_active":"2023-10-11T09:40:41.699405+0000","last_peered":"2023-10-11T09:40:41.699405+0000","last_clean":"2023-10-11T09:40:41.699405+0000","last_became_active":"2023-10-11T09:40:36.149919+0000","last_became_peered":"2023-10-11T09:40:36.149919+0000","last_unstale":"2023-10-11T09:40:41.699405+0000","last_undegraded":"2023-10-11T09:40:41.699405+0000","last_fullsized":"2023-10-11T09:40:41.699405+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687621+0000","last_change":"2023-10-11T09:40:41.696509+0000","last_active":"2023-10-11T09:40:42.687621+0000","last_peered":"2023-10-11T09:40:42.687621+0000","last_clean":"2023-10-11T09:40:42.687621+0000","last_became_active":"2023-10-11T09:40:41.696149+0000","last_became_peered":"2023-10-11T09:40:41.696149+0000","last_unstale":"2023-10-11T09:40:42.687621+0000","last_undegraded":"2023-10-11T09:40:42.687621+0000","last_fullsized":"2023-10-11T09:40:42.687621+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:00:05.422788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145899+0000","last_change":"2023-10-11T09:40:41.230379+0000","last_active":"2023-10-11T09:40:43.145899+0000","last_peered":"2023-10-11T09:40:43.145899+0000","last_clean":"2023-10-11T09:40:43.145899+0000","last_became_active":"2023-10-11T09:40:41.226481+0000","last_became_peered":"2023-10-11T09:40:41.226481+0000","last_unstale":"2023-10-11T09:40:43.145899+0000","last_undegraded":"2023-10-11T09:40:43.145899+0000","last_fullsized":"2023-10-11T09:40:43.145899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:48:51.882031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-11T09:40:41.708126+0000","last_change":"2023-10-11T09:40:36.151839+0000","last_active":"2023-10-11T09:40:41.708126+0000","last_peered":"2023-10-11T09:40:41.708126+0000","last_clean":"2023-10-11T09:40:41.708126+0000","last_became_active":"2023-10-11T09:40:36.151179+0000","last_became_peered":"2023-10-11T09:40:36.151179+0000","last_unstale":"2023-10-11T09:40:41.708126+0000","last_undegraded":"2023-10-11T09:40:41.708126+0000","last_fullsized":"2023-10-11T09:40:41.708126+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911579+0000","last_change":"2023-10-11T09:40:41.712383+0000","last_active":"2023-10-11T09:40:42.911579+0000","last_peered":"2023-10-11T09:40:42.911579+0000","last_clean":"2023-10-11T09:40:42.911579+0000","last_became_active":"2023-10-11T09:40:41.711978+0000","last_became_peered":"2023-10-11T09:40:41.711978+0000","last_unstale":"2023-10-11T09:40:42.911579+0000","last_undegraded":"2023-10-11T09:40:42.911579+0000","last_fullsized":"2023-10-11T09:40:42.911579+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:45:40.098819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911910+0000","last_change":"2023-10-11T09:40:41.720785+0000","last_active":"2023-10-11T09:40:42.911910+0000","last_peered":"2023-10-11T09:40:42.911910+0000","last_clean":"2023-10-11T09:40:42.911910+0000","last_became_active":"2023-10-11T09:40:41.720508+0000","last_became_peered":"2023-10-11T09:40:41.720508+0000","last_unstale":"2023-10-11T09:40:42.911910+0000","last_undegraded":"2023-10-11T09:40:42.911910+0000","last_fullsized":"2023-10-11T09:40:42.911910+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:35:04.646798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.912435+0000","last_change":"2023-10-11T09:40:41.229054+0000","last_active":"2023-10-11T09:40:42.912435+0000","last_peered":"2023-10-11T09:40:42.912435+0000","last_clean":"2023-10-11T09:40:42.912435+0000","last_became_active":"2023-10-11T09:40:41.227552+0000","last_became_peered":"2023-10-11T09:40:41.227552+0000","last_unstale":"2023-10-11T09:40:42.912435+0000","last_undegraded":"2023-10-11T09:40:42.912435+0000","last_fullsized":"2023-10-11T09:40:42.912435+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:36:34.713602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144970+0000","last_change":"2023-10-11T09:40:41.730718+0000","last_active":"2023-10-11T09:40:43.144970+0000","last_peered":"2023-10-11T09:40:43.144970+0000","last_clean":"2023-10-11T09:40:43.144970+0000","last_became_active":"2023-10-11T09:40:41.730499+0000","last_became_peered":"2023-10-11T09:40:41.730499+0000","last_unstale":"2023-10-11T09:40:43.144970+0000","last_undegraded":"2023-10-11T09:40:43.144970+0000","last_fullsized":"2023-10-11T09:40:43.144970+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:35:36.436236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.692532+0000","last_change":"2023-10-11T09:40:41.230822+0000","last_active":"2023-10-11T09:40:42.692532+0000","last_peered":"2023-10-11T09:40:42.692532+0000","last_clean":"2023-10-11T09:40:42.692532+0000","last_became_active":"2023-10-11T09:40:41.230417+0000","last_became_peered":"2023-10-11T09:40:41.230417+0000","last_unstale":"2023-10-11T09:40:42.692532+0000","last_undegraded":"2023-10-11T09:40:42.692532+0000","last_fullsized":"2023-10-11T09:40:42.692532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:20:53.471012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.694445+0000","last_change":"2023-10-11T09:40:41.231668+0000","last_active":"2023-10-11T09:40:42.694445+0000","last_peered":"2023-10-11T09:40:42.694445+0000","last_clean":"2023-10-11T09:40:42.694445+0000","last_became_active":"2023-10-11T09:40:41.226956+0000","last_became_peered":"2023-10-11T09:40:41.226956+0000","last_unstale":"2023-10-11T09:40:42.694445+0000","last_undegraded":"2023-10-11T09:40:42.694445+0000","last_fullsized":"2023-10-11T09:40:42.694445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:43:00.315992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687648+0000","last_change":"2023-10-11T09:40:41.709493+0000","last_active":"2023-10-11T09:40:42.687648+0000","last_peered":"2023-10-11T09:40:42.687648+0000","last_clean":"2023-10-11T09:40:42.687648+0000","last_became_active":"2023-10-11T09:40:41.709217+0000","last_became_peered":"2023-10-11T09:40:41.709217+0000","last_unstale":"2023-10-11T09:40:42.687648+0000","last_undegraded":"2023-10-11T09:40:42.687648+0000","last_fullsized":"2023-10-11T09:40:42.687648+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:07:02.302344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.146199+0000","last_change":"2023-10-11T09:40:41.228934+0000","last_active":"2023-10-11T09:40:43.146199+0000","last_peered":"2023-10-11T09:40:43.146199+0000","last_clean":"2023-10-11T09:40:43.146199+0000","last_became_active":"2023-10-11T09:40:41.226833+0000","last_became_peered":"2023-10-11T09:40:41.226833+0000","last_unstale":"2023-10-11T09:40:43.146199+0000","last_undegraded":"2023-10-11T09:40:43.146199+0000","last_fullsized":"2023-10-11T09:40:43.146199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:00:56.048004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687536+0000","last_change":"2023-10-11T09:40:41.715540+0000","last_active":"2023-10-11T09:40:42.687536+0000","last_peered":"2023-10-11T09:40:42.687536+0000","last_clean":"2023-10-11T09:40:42.687536+0000","last_became_active":"2023-10-11T09:40:41.715254+0000","last_became_peered":"2023-10-11T09:40:41.715254+0000","last_unstale":"2023-10-11T09:40:42.687536+0000","last_undegraded":"2023-10-11T09:40:42.687536+0000","last_fullsized":"2023-10-11T09:40:42.687536+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:23:30.814623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144670+0000","last_change":"2023-10-11T09:40:41.722871+0000","last_active":"2023-10-11T09:40:43.144670+0000","last_peered":"2023-10-11T09:40:43.144670+0000","last_clean":"2023-10-11T09:40:43.144670+0000","last_became_active":"2023-10-11T09:40:41.722538+0000","last_became_peered":"2023-10-11T09:40:41.722538+0000","last_unstale":"2023-10-11T09:40:43.144670+0000","last_undegraded":"2023-10-11T09:40:43.144670+0000","last_fullsized":"2023-10-11T09:40:43.144670+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:55:31.027029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687761+0000","last_change":"2023-10-11T09:40:41.230916+0000","last_active":"2023-10-11T09:40:42.687761+0000","last_peered":"2023-10-11T09:40:42.687761+0000","last_clean":"2023-10-11T09:40:42.687761+0000","last_became_active":"2023-10-11T09:40:41.230639+0000","last_became_peered":"2023-10-11T09:40:41.230639+0000","last_unstale":"2023-10-11T09:40:42.687761+0000","last_undegraded":"2023-10-11T09:40:42.687761+0000","last_fullsized":"2023-10-11T09:40:42.687761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:48:51.882031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145531+0000","last_change":"2023-10-11T09:40:41.705483+0000","last_active":"2023-10-11T09:40:43.145531+0000","last_peered":"2023-10-11T09:40:43.145531+0000","last_clean":"2023-10-11T09:40:43.145531+0000","last_became_active":"2023-10-11T09:40:41.705060+0000","last_became_peered":"2023-10-11T09:40:41.705060+0000","last_unstale":"2023-10-11T09:40:43.145531+0000","last_undegraded":"2023-10-11T09:40:43.145531+0000","last_fullsized":"2023-10-11T09:40:43.145531+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:35:36.436236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144834+0000","last_change":"2023-10-11T09:40:41.227035+0000","last_active":"2023-10-11T09:40:43.144834+0000","last_peered":"2023-10-11T09:40:43.144834+0000","last_clean":"2023-10-11T09:40:43.144834+0000","last_became_active":"2023-10-11T09:40:41.226593+0000","last_became_peered":"2023-10-11T09:40:41.226593+0000","last_unstale":"2023-10-11T09:40:43.144834+0000","last_undegraded":"2023-10-11T09:40:43.144834+0000","last_fullsized":"2023-10-11T09:40:43.144834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:48:51.882031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.688081+0000","last_change":"2023-10-11T09:40:41.715379+0000","last_active":"2023-10-11T09:40:42.688081+0000","last_peered":"2023-10-11T09:40:42.688081+0000","last_clean":"2023-10-11T09:40:42.688081+0000","last_became_active":"2023-10-11T09:40:41.715091+0000","last_became_peered":"2023-10-11T09:40:41.715091+0000","last_unstale":"2023-10-11T09:40:42.688081+0000","last_undegraded":"2023-10-11T09:40:42.688081+0000","last_fullsized":"2023-10-11T09:40:42.688081+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:42:34.821513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911962+0000","last_change":"2023-10-11T09:40:41.235100+0000","last_active":"2023-10-11T09:40:42.911962+0000","last_peered":"2023-10-11T09:40:42.911962+0000","last_clean":"2023-10-11T09:40:42.911962+0000","last_became_active":"2023-10-11T09:40:41.233361+0000","last_became_peered":"2023-10-11T09:40:41.233361+0000","last_unstale":"2023-10-11T09:40:42.911962+0000","last_undegraded":"2023-10-11T09:40:42.911962+0000","last_fullsized":"2023-10-11T09:40:42.911962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:12:09.780799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144249+0000","last_change":"2023-10-11T09:40:41.705452+0000","last_active":"2023-10-11T09:40:43.144249+0000","last_peered":"2023-10-11T09:40:43.144249+0000","last_clean":"2023-10-11T09:40:43.144249+0000","last_became_active":"2023-10-11T09:40:41.705028+0000","last_became_peered":"2023-10-11T09:40:41.705028+0000","last_unstale":"2023-10-11T09:40:43.144249+0000","last_undegraded":"2023-10-11T09:40:43.144249+0000","last_fullsized":"2023-10-11T09:40:43.144249+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:09:04.951739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145980+0000","last_change":"2023-10-11T09:40:41.232125+0000","last_active":"2023-10-11T09:40:43.145980+0000","last_peered":"2023-10-11T09:40:43.145980+0000","last_clean":"2023-10-11T09:40:43.145980+0000","last_became_active":"2023-10-11T09:40:41.231730+0000","last_became_peered":"2023-10-11T09:40:41.231730+0000","last_unstale":"2023-10-11T09:40:43.145980+0000","last_undegraded":"2023-10-11T09:40:43.145980+0000","last_fullsized":"2023-10-11T09:40:43.145980+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:12:04.722202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911689+0000","last_change":"2023-10-11T09:40:41.702062+0000","last_active":"2023-10-11T09:40:42.911689+0000","last_peered":"2023-10-11T09:40:42.911689+0000","last_clean":"2023-10-11T09:40:42.911689+0000","last_became_active":"2023-10-11T09:40:41.701665+0000","last_became_peered":"2023-10-11T09:40:41.701665+0000","last_unstale":"2023-10-11T09:40:42.911689+0000","last_undegraded":"2023-10-11T09:40:42.911689+0000","last_fullsized":"2023-10-11T09:40:42.911689+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:03:52.325886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.693906+0000","last_change":"2023-10-11T09:40:41.227957+0000","last_active":"2023-10-11T09:40:42.693906+0000","last_peered":"2023-10-11T09:40:42.693906+0000","last_clean":"2023-10-11T09:40:42.693906+0000","last_became_active":"2023-10-11T09:40:41.226855+0000","last_became_peered":"2023-10-11T09:40:41.226855+0000","last_unstale":"2023-10-11T09:40:42.693906+0000","last_undegraded":"2023-10-11T09:40:42.693906+0000","last_fullsized":"2023-10-11T09:40:42.693906+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:12:04.722202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911772+0000","last_change":"2023-10-11T09:40:41.699151+0000","last_active":"2023-10-11T09:40:42.911772+0000","last_peered":"2023-10-11T09:40:42.911772+0000","last_clean":"2023-10-11T09:40:42.911772+0000","last_became_active":"2023-10-11T09:40:41.698188+0000","last_became_peered":"2023-10-11T09:40:41.698188+0000","last_unstale":"2023-10-11T09:40:42.911772+0000","last_undegraded":"2023-10-11T09:40:42.911772+0000","last_fullsized":"2023-10-11T09:40:42.911772+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:48:15.689750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.694192+0000","last_change":"2023-10-11T09:40:41.224843+0000","last_active":"2023-10-11T09:40:42.694192+0000","last_peered":"2023-10-11T09:40:42.694192+0000","last_clean":"2023-10-11T09:40:42.694192+0000","last_became_active":"2023-10-11T09:40:41.224415+0000","last_became_peered":"2023-10-11T09:40:41.224415+0000","last_unstale":"2023-10-11T09:40:42.694192+0000","last_undegraded":"2023-10-11T09:40:42.694192+0000","last_fullsized":"2023-10-11T09:40:42.694192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:31:56.724263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687814+0000","last_change":"2023-10-11T09:40:41.695801+0000","last_active":"2023-10-11T09:40:42.687814+0000","last_peered":"2023-10-11T09:40:42.687814+0000","last_clean":"2023-10-11T09:40:42.687814+0000","last_became_active":"2023-10-11T09:40:41.694814+0000","last_became_peered":"2023-10-11T09:40:41.694814+0000","last_unstale":"2023-10-11T09:40:42.687814+0000","last_undegraded":"2023-10-11T09:40:42.687814+0000","last_fullsized":"2023-10-11T09:40:42.687814+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:19:07.089979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687947+0000","last_change":"2023-10-11T09:40:41.222156+0000","last_active":"2023-10-11T09:40:42.687947+0000","last_peered":"2023-10-11T09:40:42.687947+0000","last_clean":"2023-10-11T09:40:42.687947+0000","last_became_active":"2023-10-11T09:40:41.221758+0000","last_became_peered":"2023-10-11T09:40:41.221758+0000","last_unstale":"2023-10-11T09:40:42.687947+0000","last_undegraded":"2023-10-11T09:40:42.687947+0000","last_fullsized":"2023-10-11T09:40:42.687947+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:00:56.048004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911260+0000","last_change":"2023-10-11T09:40:41.729793+0000","last_active":"2023-10-11T09:40:42.911260+0000","last_peered":"2023-10-11T09:40:42.911260+0000","last_clean":"2023-10-11T09:40:42.911260+0000","last_became_active":"2023-10-11T09:40:41.729468+0000","last_became_peered":"2023-10-11T09:40:41.729468+0000","last_unstale":"2023-10-11T09:40:42.911260+0000","last_undegraded":"2023-10-11T09:40:42.911260+0000","last_fullsized":"2023-10-11T09:40:42.911260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:20:54.486033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.146157+0000","last_change":"2023-10-11T09:40:41.231067+0000","last_active":"2023-10-11T09:40:43.146157+0000","last_peered":"2023-10-11T09:40:43.146157+0000","last_clean":"2023-10-11T09:40:43.146157+0000","last_became_active":"2023-10-11T09:40:41.229252+0000","last_became_peered":"2023-10-11T09:40:41.229252+0000","last_unstale":"2023-10-11T09:40:43.146157+0000","last_undegraded":"2023-10-11T09:40:43.146157+0000","last_fullsized":"2023-10-11T09:40:43.146157+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:43:00.315992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":864256,"data_stored":681747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95699999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.363}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4430000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3360000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.468}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":229376,"data_stored":82946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1679999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3740000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2629999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4510000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.347}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":249856,"data_stored":90251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":262144,"data_stored":91219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.149}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3919999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.478}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.518}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":245760,"data_stored":89689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.478}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7390000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.587}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.637}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":229376,"data_stored":82946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.294}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.365}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.139}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.488}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3260000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":229376,"data_stored":82946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1519999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6890000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7470000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.714}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":827392,"data_stored":673896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.335}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3640000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.29}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.153}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-11T09:40:46.855 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-11T09:40:46.856 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-11T09:40:46.856 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-11T09:40:46.856 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-11T09:40:46.857 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-11T09:40:47.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.066+0000 7fd79d63e700 1 Processor -- start 2023-10-11T09:40:47.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.070+0000 7fd79d63e700 1 -- start start 2023-10-11T09:40:47.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.070+0000 7fd79d63e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fd798154e00 0x7fd79815b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:47.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.070+0000 7fd79d63e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd798071040 0x7fd7980cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:47.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.070+0000 7fd79d63e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7980720f0 con 0x7fd7980cba00 2023-10-11T09:40:47.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.070+0000 7fd79d63e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd79815bb90 con 0x7fd798154e00 2023-10-11T09:40:47.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.070+0000 7fd79d63e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7980714e0 con 0x7fd798071040 2023-10-11T09:40:47.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.070+0000 7fd796ffd700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fd798154e00 0x7fd79815b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:47.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.070+0000 7fd796ffd700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fd798154e00 0x7fd79815b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56776/0 (socket says 172.21.15.78:56776) 2023-10-11T09:40:47.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.070+0000 7fd796ffd700 1 -- 172.21.15.78:0/1175358004 learned_addr learned my addr 172.21.15.78:0/1175358004 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:47.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.070+0000 7fd78ffff700 1 --2- 172.21.15.78:0/1175358004 >> v2:172.21.15.133:3300/0 conn(0x7fd798071040 0x7fd7980cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:47.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.070+0000 7fd796ffd700 1 -- 172.21.15.78:0/1175358004 >> v2:172.21.15.133:3300/0 conn(0x7fd798071040 msgr2=0x7fd7980cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:47.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.070+0000 7fd796ffd700 1 --2- 172.21.15.78:0/1175358004 >> v2:172.21.15.133:3300/0 conn(0x7fd798071040 0x7fd7980cb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:47.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.070+0000 7fd796ffd700 1 -- 172.21.15.78:0/1175358004 >> v1:172.21.15.78:6789/0 conn(0x7fd7980cba00 legacy=0x7fd79815e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:47.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.070+0000 7fd796ffd700 1 -- 172.21.15.78:0/1175358004 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7980717c0 con 0x7fd798154e00 2023-10-11T09:40:47.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.074+0000 7fd796ffd700 1 --2- 172.21.15.78:0/1175358004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd798154e00 0x7fd79815b650 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd7980707f0 tx=0x7fd788009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a112b9cf249a76d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:47.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.074+0000 7fd7967fc700 1 -- 172.21.15.78:0/1175358004 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd788011830 con 0x7fd798154e00 2023-10-11T09:40:47.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.074+0000 7fd7967fc700 1 -- 172.21.15.78:0/1175358004 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd788011990 con 0x7fd798154e00 2023-10-11T09:40:47.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.074+0000 7fd7967fc700 1 -- 172.21.15.78:0/1175358004 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd788011d20 con 0x7fd798154e00 2023-10-11T09:40:47.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.074+0000 7fd79d63e700 1 -- 172.21.15.78:0/1175358004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd798154e00 msgr2=0x7fd79815b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:47.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.074+0000 7fd79d63e700 1 --2- 172.21.15.78:0/1175358004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd798154e00 0x7fd79815b650 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd7980707f0 tx=0x7fd788009bf0 comp rx=0 tx=0).stop 2023-10-11T09:40:47.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.074+0000 7fd79d63e700 1 -- 172.21.15.78:0/1175358004 shutdown_connections 2023-10-11T09:40:47.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.074+0000 7fd79d63e700 1 --2- 172.21.15.78:0/1175358004 >> v2:172.21.15.133:3300/0 conn(0x7fd798071040 0x7fd7980cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:47.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.074+0000 7fd79d63e700 1 --2- 172.21.15.78:0/1175358004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd798154e00 0x7fd79815b650 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:47.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.074+0000 7fd79d63e700 1 -- 172.21.15.78:0/1175358004 >> 172.21.15.78:0/1175358004 conn(0x7fd79806aca0 msgr2=0x7fd7980b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:47.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.074+0000 7fd79d63e700 1 -- 172.21.15.78:0/1175358004 shutdown_connections 2023-10-11T09:40:47.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.074+0000 7fd79d63e700 1 -- 172.21.15.78:0/1175358004 wait complete. 2023-10-11T09:40:47.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.078+0000 7fd79d63e700 1 Processor -- start 2023-10-11T09:40:47.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.078+0000 7fd79d63e700 1 -- start start 2023-10-11T09:40:47.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.078+0000 7fd79d63e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd798071040 0x7fd7980b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:47.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.078+0000 7fd79d63e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7980cba00 0x7fd798144790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:47.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.078+0000 7fd79d63e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd798154e00 0x7fd798145050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:47.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.078+0000 7fd79d63e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79814bc50 con 0x7fd798154e00 2023-10-11T09:40:47.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.078+0000 7fd79d63e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7980ccb80 con 0x7fd7980cba00 2023-10-11T09:40:47.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.078+0000 7fd79d63e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd798144cd0 con 0x7fd798071040 2023-10-11T09:40:47.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.078+0000 7fd7977fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd798154e00 0x7fd798145050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:47.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.078+0000 7fd7977fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd798154e00 0x7fd798145050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56780/0 (socket says 172.21.15.78:56780) 2023-10-11T09:40:47.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.078+0000 7fd7977fe700 1 -- 172.21.15.78:0/1660647435 learned_addr learned my addr 172.21.15.78:0/1660647435 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:47.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.078+0000 7fd796ffd700 1 --2- 172.21.15.78:0/1660647435 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd798071040 0x7fd7980b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:47.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.078+0000 7fd78ffff700 1 --2- 172.21.15.78:0/1660647435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7980cba00 0x7fd798144790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:47.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.078+0000 7fd7977fe700 1 -- 172.21.15.78:0/1660647435 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd798071040 msgr2=0x7fd7980b36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:47.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.078+0000 7fd7977fe700 1 --2- 172.21.15.78:0/1660647435 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd798071040 0x7fd7980b36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:47.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.078+0000 7fd7977fe700 1 -- 172.21.15.78:0/1660647435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7980cba00 msgr2=0x7fd798144790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:47.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.078+0000 7fd7977fe700 1 --2- 172.21.15.78:0/1660647435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7980cba00 0x7fd798144790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:47.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.078+0000 7fd7977fe700 1 -- 172.21.15.78:0/1660647435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd788009570 con 0x7fd798154e00 2023-10-11T09:40:47.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.078+0000 7fd7977fe700 1 --2- 172.21.15.78:0/1660647435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd798154e00 0x7fd798145050 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd79000cba0 tx=0x7fd79000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:47.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.078+0000 7fd78f7fe700 1 -- 172.21.15.78:0/1660647435 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd790003c20 con 0x7fd798154e00 2023-10-11T09:40:47.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.078+0000 7fd78f7fe700 1 -- 172.21.15.78:0/1660647435 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd790003d80 con 0x7fd798154e00 2023-10-11T09:40:47.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.078+0000 7fd79d63e700 1 -- 172.21.15.78:0/1660647435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd798145590 con 0x7fd798154e00 2023-10-11T09:40:47.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.078+0000 7fd78f7fe700 1 -- 172.21.15.78:0/1660647435 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79000abc0 con 0x7fd798154e00 2023-10-11T09:40:47.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.078+0000 7fd79d63e700 1 -- 172.21.15.78:0/1660647435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd798145aa0 con 0x7fd798154e00 2023-10-11T09:40:47.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.082+0000 7fd79d63e700 1 -- 172.21.15.78:0/1660647435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd79814bd90 con 0x7fd798154e00 2023-10-11T09:40:47.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.082+0000 7fd78f7fe700 1 -- 172.21.15.78:0/1660647435 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fd79000ad20 con 0x7fd798154e00 2023-10-11T09:40:47.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.086+0000 7fd78f7fe700 1 --2- 172.21.15.78:0/1660647435 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd778065ed0 0x7fd778068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:47.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.086+0000 7fd78f7fe700 1 -- 172.21.15.78:0/1660647435 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd790083950 con 0x7fd798154e00 2023-10-11T09:40:47.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.086+0000 7fd796ffd700 1 --2- 172.21.15.78:0/1660647435 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd778065ed0 0x7fd778068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:47.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.086+0000 7fd796ffd700 1 --2- 172.21.15.78:0/1660647435 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd778065ed0 0x7fd778068380 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd788000c00 tx=0x7fd788008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:47.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.090+0000 7fd78f7fe700 1 -- 172.21.15.78:0/1660647435 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fd79814bd90 con 0x7fd798154e00 2023-10-11T09:40:47.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.302+0000 7fd79d63e700 1 -- 172.21.15.78:0/1660647435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fd79814bd90 con 0x7fd798154e00 2023-10-11T09:40:47.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.310+0000 7fd78f7fe700 1 -- 172.21.15.78:0/1660647435 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+163891 (secure 0 0 0) 0x7fd79814bd90 con 0x7fd798154e00 2023-10-11T09:40:47.317 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:40:47.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.314+0000 7fd79d63e700 1 -- 172.21.15.78:0/1660647435 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd778065ed0 msgr2=0x7fd778068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:47.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.314+0000 7fd79d63e700 1 --2- 172.21.15.78:0/1660647435 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd778065ed0 0x7fd778068380 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd788000c00 tx=0x7fd788008030 comp rx=0 tx=0).stop 2023-10-11T09:40:47.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.314+0000 7fd79d63e700 1 -- 172.21.15.78:0/1660647435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd798154e00 msgr2=0x7fd798145050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:47.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.314+0000 7fd79d63e700 1 --2- 172.21.15.78:0/1660647435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd798154e00 0x7fd798145050 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd79000cba0 tx=0x7fd79000cf60 comp rx=0 tx=0).stop 2023-10-11T09:40:47.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.314+0000 7fd79d63e700 1 -- 172.21.15.78:0/1660647435 shutdown_connections 2023-10-11T09:40:47.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.314+0000 7fd79d63e700 1 --2- 172.21.15.78:0/1660647435 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd778065ed0 0x7fd778068380 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:47.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.314+0000 7fd79d63e700 1 --2- 172.21.15.78:0/1660647435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd798154e00 0x7fd798145050 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:47.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.314+0000 7fd79d63e700 1 --2- 172.21.15.78:0/1660647435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7980cba00 0x7fd798144790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:47.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.314+0000 7fd79d63e700 1 --2- 172.21.15.78:0/1660647435 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd798071040 0x7fd7980b36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:47.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.314+0000 7fd79d63e700 1 -- 172.21.15.78:0/1660647435 >> 172.21.15.78:0/1660647435 conn(0x7fd79806aca0 msgr2=0x7fd7980afce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:47.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.314+0000 7fd79d63e700 1 -- 172.21.15.78:0/1660647435 shutdown_connections 2023-10-11T09:40:47.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.314+0000 7fd79d63e700 1 -- 172.21.15.78:0/1660647435 wait complete. 2023-10-11T09:40:47.342 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":5,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":35606},{"type":"v1","addr":"172.21.15.78:6833","nonce":35606}]},"active_addr":"172.21.15.78:6833/35606","active_change":"2023-10-11T09:40:20.296969+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4103,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph:v18","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph:v18","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":1471198500}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":3778679848}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":1082076356}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":4236242220}]}]} 2023-10-11T09:40:47.346 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-11T09:40:47.347 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-11T09:40:47.347 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:40:47.348 INFO:tasks.ceph.mds.d.smithi133.stderr:2023-10-11T09:40:47.340+0000 7f550bbd1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-11T09:40:47.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.518+0000 7f9c8d733700 1 Processor -- start 2023-10-11T09:40:47.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.518+0000 7f9c8d733700 1 -- start start 2023-10-11T09:40:47.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.518+0000 7f9c8d733700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f9c880b2310 0x7f9c88161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:47.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.518+0000 7f9c8d733700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9c880b4b60 0x7f9c880d1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:47.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.518+0000 7f9c8d733700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c880d21e0 0x7f9c880b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:47.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.518+0000 7f9c8d733700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c880d11e0 con 0x7f9c880b2310 2023-10-11T09:40:47.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.518+0000 7f9c8d733700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c880b5370 con 0x7f9c880b4b60 2023-10-11T09:40:47.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.518+0000 7f9c8d733700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c880d14a0 con 0x7f9c880d21e0 2023-10-11T09:40:47.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.518+0000 7f9c86ffd700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f9c880b2310 0x7f9c88161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:47.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.518+0000 7f9c86ffd700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f9c880b2310 0x7f9c88161150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56794/0 (socket says 172.21.15.78:56794) 2023-10-11T09:40:47.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.518+0000 7f9c86ffd700 1 -- 172.21.15.78:0/1117257640 learned_addr learned my addr 172.21.15.78:0/1117257640 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:47.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.518+0000 7f9c7ffff700 1 --2- 172.21.15.78:0/1117257640 >> v2:172.21.15.133:3300/0 conn(0x7f9c880b4b60 0x7f9c880d1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:47.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.518+0000 7f9c877fe700 1 --2- 172.21.15.78:0/1117257640 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c880d21e0 0x7f9c880b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:47.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.518+0000 7f9c86ffd700 1 -- 172.21.15.78:0/1117257640 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c880d21e0 msgr2=0x7f9c880b5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:47.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.518+0000 7f9c86ffd700 1 --2- 172.21.15.78:0/1117257640 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c880d21e0 0x7f9c880b5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:47.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.518+0000 7f9c86ffd700 1 -- 172.21.15.78:0/1117257640 >> v2:172.21.15.133:3300/0 conn(0x7f9c880b4b60 msgr2=0x7f9c880d1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:47.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.518+0000 7f9c86ffd700 1 --2- 172.21.15.78:0/1117257640 >> v2:172.21.15.133:3300/0 conn(0x7f9c880b4b60 0x7f9c880d1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:47.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.518+0000 7f9c86ffd700 1 -- 172.21.15.78:0/1117257640 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c880d1780 con 0x7f9c880b2310 2023-10-11T09:40:47.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.522+0000 7f9c7ffff700 1 --2- 172.21.15.78:0/1117257640 >> v2:172.21.15.133:3300/0 conn(0x7f9c880b4b60 0x7f9c880d1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:40:47.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.522+0000 7f9c86ffd700 1 --2- 172.21.15.78:0/1117257640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c880b2310 0x7f9c88161150 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9c880b5ba0 tx=0x7f9c78009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e8784c0cc45e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:47.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.522+0000 7f9c867fc700 1 -- 172.21.15.78:0/1117257640 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c78011830 con 0x7f9c880b2310 2023-10-11T09:40:47.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.522+0000 7f9c867fc700 1 -- 172.21.15.78:0/1117257640 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c78011990 con 0x7f9c880b2310 2023-10-11T09:40:47.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.522+0000 7f9c867fc700 1 -- 172.21.15.78:0/1117257640 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c78011d20 con 0x7f9c880b2310 2023-10-11T09:40:47.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.522+0000 7f9c8d733700 1 -- 172.21.15.78:0/1117257640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c880b2310 msgr2=0x7f9c88161150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:47.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.522+0000 7f9c8d733700 1 --2- 172.21.15.78:0/1117257640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c880b2310 0x7f9c88161150 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9c880b5ba0 tx=0x7f9c78009bf0 comp rx=0 tx=0).stop 2023-10-11T09:40:47.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.522+0000 7f9c8d733700 1 -- 172.21.15.78:0/1117257640 shutdown_connections 2023-10-11T09:40:47.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.522+0000 7f9c8d733700 1 --2- 172.21.15.78:0/1117257640 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c880d21e0 0x7f9c880b5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:47.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.522+0000 7f9c8d733700 1 --2- 172.21.15.78:0/1117257640 >> v2:172.21.15.133:3300/0 conn(0x7f9c880b4b60 0x7f9c880d1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:47.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.522+0000 7f9c8d733700 1 --2- 172.21.15.78:0/1117257640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c880b2310 0x7f9c88161150 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:47.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.522+0000 7f9c8d733700 1 -- 172.21.15.78:0/1117257640 >> 172.21.15.78:0/1117257640 conn(0x7f9c880cfb60 msgr2=0x7f9c880cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:47.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.522+0000 7f9c8d733700 1 -- 172.21.15.78:0/1117257640 shutdown_connections 2023-10-11T09:40:47.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.522+0000 7f9c8d733700 1 -- 172.21.15.78:0/1117257640 wait complete. 2023-10-11T09:40:47.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.526+0000 7f9c8d733700 1 Processor -- start 2023-10-11T09:40:47.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.526+0000 7f9c8d733700 1 -- start start 2023-10-11T09:40:47.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.526+0000 7f9c8d733700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c880b2310 0x7f9c880c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:47.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.526+0000 7f9c8d733700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c880b4b60 0x7f9c881f25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:47.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.526+0000 7f9c8d733700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c880d21e0 0x7f9c8814d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:47.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.526+0000 7f9c8d733700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c880b64d0 con 0x7f9c880b4b60 2023-10-11T09:40:47.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.526+0000 7f9c8d733700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c880b67b0 con 0x7f9c880b2310 2023-10-11T09:40:47.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.526+0000 7f9c8d733700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c880b6a90 con 0x7f9c880d21e0 2023-10-11T09:40:47.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.526+0000 7f9c7ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c880b4b60 0x7f9c881f25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:47.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.526+0000 7f9c7ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c880b4b60 0x7f9c881f25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56808/0 (socket says 172.21.15.78:56808) 2023-10-11T09:40:47.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.526+0000 7f9c7ffff700 1 -- 172.21.15.78:0/1150732980 learned_addr learned my addr 172.21.15.78:0/1150732980 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:47.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.526+0000 7f9c877fe700 1 --2- 172.21.15.78:0/1150732980 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c880d21e0 0x7f9c8814d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:47.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.530+0000 7f9c86ffd700 1 --2- 172.21.15.78:0/1150732980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c880b2310 0x7f9c880c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:47.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.530+0000 7f9c7ffff700 1 -- 172.21.15.78:0/1150732980 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c880d21e0 msgr2=0x7f9c8814d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:47.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.530+0000 7f9c7ffff700 1 --2- 172.21.15.78:0/1150732980 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c880d21e0 0x7f9c8814d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:47.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.530+0000 7f9c7ffff700 1 -- 172.21.15.78:0/1150732980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c880b2310 msgr2=0x7f9c880c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:47.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.530+0000 7f9c7ffff700 1 --2- 172.21.15.78:0/1150732980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c880b2310 0x7f9c880c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:47.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.530+0000 7f9c7ffff700 1 -- 172.21.15.78:0/1150732980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c700094b0 con 0x7f9c880b4b60 2023-10-11T09:40:47.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.530+0000 7f9c7ffff700 1 --2- 172.21.15.78:0/1150732980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c880b4b60 0x7f9c881f25b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9c7000ebe0 tx=0x7f9c70009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:47.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.530+0000 7f9c7f7fe700 1 -- 172.21.15.78:0/1150732980 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c70009790 con 0x7f9c880b4b60 2023-10-11T09:40:47.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.530+0000 7f9c7f7fe700 1 -- 172.21.15.78:0/1150732980 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c7000c8b0 con 0x7f9c880b4b60 2023-10-11T09:40:47.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.530+0000 7f9c7f7fe700 1 -- 172.21.15.78:0/1150732980 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7000cc40 con 0x7f9c880b4b60 2023-10-11T09:40:47.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.530+0000 7f9c8d733700 1 -- 172.21.15.78:0/1150732980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c78009570 con 0x7f9c880b4b60 2023-10-11T09:40:47.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.530+0000 7f9c8d733700 1 -- 172.21.15.78:0/1150732980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c880b5140 con 0x7f9c880b4b60 2023-10-11T09:40:47.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.530+0000 7f9c8d733700 1 -- 172.21.15.78:0/1150732980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c8804e530 con 0x7f9c880b4b60 2023-10-11T09:40:47.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.534+0000 7f9c7f7fe700 1 -- 172.21.15.78:0/1150732980 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f9c70013470 con 0x7f9c880b4b60 2023-10-11T09:40:47.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.534+0000 7f9c7f7fe700 1 --2- 172.21.15.78:0/1150732980 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9c68065ed0 0x7f9c68068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:47.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.534+0000 7f9c7f7fe700 1 -- 172.21.15.78:0/1150732980 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9c700835c0 con 0x7f9c880b4b60 2023-10-11T09:40:47.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.534+0000 7f9c86ffd700 1 --2- 172.21.15.78:0/1150732980 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9c68065ed0 0x7f9c68068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:47.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.534+0000 7f9c86ffd700 1 --2- 172.21.15.78:0/1150732980 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9c68065ed0 0x7f9c68068380 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9c78000c00 tx=0x7f9c78008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:47.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.542+0000 7f9c7f7fe700 1 -- 172.21.15.78:0/1150732980 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f9c70050c10 con 0x7f9c880b4b60 2023-10-11T09:40:47.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.714+0000 7f9c8d733700 1 -- 172.21.15.78:0/1150732980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c880d1fc0 con 0x7f9c880b4b60 2023-10-11T09:40:47.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.718+0000 7f9c7f7fe700 1 -- 172.21.15.78:0/1150732980 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17811 (secure 0 0 0) 0x7f9c70054230 con 0x7f9c880b4b60 2023-10-11T09:40:47.724 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:40:47.724 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":23,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:40:42.678477+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:40:39.862581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T09:40:40.663641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:40:47.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.722+0000 7f9c8d733700 1 -- 172.21.15.78:0/1150732980 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9c68065ed0 msgr2=0x7f9c68068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:47.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.722+0000 7f9c8d733700 1 --2- 172.21.15.78:0/1150732980 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9c68065ed0 0x7f9c68068380 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9c78000c00 tx=0x7f9c78008030 comp rx=0 tx=0).stop 2023-10-11T09:40:47.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.722+0000 7f9c8d733700 1 -- 172.21.15.78:0/1150732980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c880b4b60 msgr2=0x7f9c881f25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:47.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.722+0000 7f9c8d733700 1 --2- 172.21.15.78:0/1150732980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c880b4b60 0x7f9c881f25b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9c7000ebe0 tx=0x7f9c70009e50 comp rx=0 tx=0).stop 2023-10-11T09:40:47.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.722+0000 7f9c8d733700 1 -- 172.21.15.78:0/1150732980 shutdown_connections 2023-10-11T09:40:47.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.722+0000 7f9c8d733700 1 --2- 172.21.15.78:0/1150732980 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9c68065ed0 0x7f9c68068380 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:47.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.722+0000 7f9c8d733700 1 --2- 172.21.15.78:0/1150732980 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c880d21e0 0x7f9c8814d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:47.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.722+0000 7f9c8d733700 1 --2- 172.21.15.78:0/1150732980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c880b4b60 0x7f9c881f25b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:47.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.722+0000 7f9c8d733700 1 --2- 172.21.15.78:0/1150732980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c880b2310 0x7f9c880c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:47.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.722+0000 7f9c8d733700 1 -- 172.21.15.78:0/1150732980 >> 172.21.15.78:0/1150732980 conn(0x7f9c880cfb60 msgr2=0x7f9c8815c400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:47.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.722+0000 7f9c8d733700 1 -- 172.21.15.78:0/1150732980 shutdown_connections 2023-10-11T09:40:47.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.722+0000 7f9c8d733700 1 -- 172.21.15.78:0/1150732980 wait complete. 2023-10-11T09:40:47.753 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-11T09:40:47.753 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:40:47.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.922+0000 7f593e6aa700 1 Processor -- start 2023-10-11T09:40:47.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.922+0000 7f593e6aa700 1 -- start start 2023-10-11T09:40:47.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.926+0000 7f593e6aa700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5938161150 0x7f59380d1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:47.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.926+0000 7f593e6aa700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59380d21e0 0x7f59380b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:47.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.926+0000 7f593e6aa700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59380d11e0 con 0x7f59380b2310 2023-10-11T09:40:47.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.926+0000 7f593e6aa700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59381615b0 con 0x7f5938161150 2023-10-11T09:40:47.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.926+0000 7f593e6aa700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59380b5370 con 0x7f59380d21e0 2023-10-11T09:40:47.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.926+0000 7f5937fff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f59380b2310 0x7f59380b4b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:58696/0 (socket says 172.21.15.78:58696) 2023-10-11T09:40:47.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.926+0000 7f5937fff700 1 -- 172.21.15.78:0/761317872 learned_addr learned my addr 172.21.15.78:0/761317872 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:47.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.926+0000 7f593cc47700 1 --2- 172.21.15.78:0/761317872 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59380d21e0 0x7f59380b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:47.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.926+0000 7f59377fe700 1 -- 172.21.15.78:0/761317872 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2688218284 0 0) 0x7f59380d11e0 con 0x7f59380b2310 2023-10-11T09:40:47.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.926+0000 7f59377fe700 1 -- 172.21.15.78:0/761317872 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59240035d0 con 0x7f59380b2310 2023-10-11T09:40:47.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.926+0000 7f592f7fe700 1 --2- 172.21.15.78:0/761317872 >> v2:172.21.15.133:3300/0 conn(0x7f5938161150 0x7f59380d1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:47.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.926+0000 7f59377fe700 1 -- 172.21.15.78:0/761317872 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3674971783 0 0) 0x7f59240035d0 con 0x7f59380b2310 2023-10-11T09:40:47.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.926+0000 7f59377fe700 1 -- 172.21.15.78:0/761317872 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59380d21e0 msgr2=0x7f59380b5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:47.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.926+0000 7f59377fe700 1 --2- 172.21.15.78:0/761317872 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59380d21e0 0x7f59380b5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:47.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.926+0000 7f59377fe700 1 -- 172.21.15.78:0/761317872 >> v2:172.21.15.133:3300/0 conn(0x7f5938161150 msgr2=0x7f59380d1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:47.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.926+0000 7f59377fe700 1 --2- 172.21.15.78:0/761317872 >> v2:172.21.15.133:3300/0 conn(0x7f5938161150 0x7f59380d1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:47.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.926+0000 7f59377fe700 1 -- 172.21.15.78:0/761317872 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59380d14a0 con 0x7f59380b2310 2023-10-11T09:40:47.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.926+0000 7f59377fe700 1 -- 172.21.15.78:0/761317872 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f59280019f0 con 0x7f59380b2310 2023-10-11T09:40:47.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.930+0000 7f59377fe700 1 -- 172.21.15.78:0/761317872 >> v1:172.21.15.78:6789/0 conn(0x7f59380b2310 legacy=0x7f59380b4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:47.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.930+0000 7f59377fe700 1 --2- 172.21.15.78:0/761317872 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59380d21e0 0x7f5924003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:47.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.930+0000 7f59377fe700 1 --2- 172.21.15.78:0/761317872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5924003f60 0x7f5924004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:47.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.930+0000 7f59377fe700 1 --2- 172.21.15.78:0/761317872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5924004db0 0x7f5924005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:47.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.930+0000 7f59377fe700 1 -- 172.21.15.78:0/761317872 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59380b5370 con 0x7f5924004db0 2023-10-11T09:40:47.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.930+0000 7f59377fe700 1 -- 172.21.15.78:0/761317872 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5924004690 con 0x7f5924003f60 2023-10-11T09:40:47.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.930+0000 7f59377fe700 1 -- 172.21.15.78:0/761317872 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f592400ba60 con 0x7f59380d21e0 2023-10-11T09:40:47.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.930+0000 7f593cc47700 1 --2- 172.21.15.78:0/761317872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5924004db0 0x7f5924005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:47.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.930+0000 7f5937fff700 1 --2- 172.21.15.78:0/761317872 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59380d21e0 0x7f5924003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:47.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.930+0000 7f592f7fe700 1 --2- 172.21.15.78:0/761317872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5924003f60 0x7f5924004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:47.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.930+0000 7f593cc47700 1 -- 172.21.15.78:0/761317872 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59380d21e0 msgr2=0x7f5924003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:47.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.930+0000 7f593cc47700 1 --2- 172.21.15.78:0/761317872 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59380d21e0 0x7f5924003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:47.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.930+0000 7f593cc47700 1 -- 172.21.15.78:0/761317872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5924003f60 msgr2=0x7f5924004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:47.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.930+0000 7f593cc47700 1 --2- 172.21.15.78:0/761317872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5924003f60 0x7f5924004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:47.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.930+0000 7f593cc47700 1 -- 172.21.15.78:0/761317872 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59240035d0 con 0x7f5924004db0 2023-10-11T09:40:47.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.930+0000 7f5937fff700 1 --2- 172.21.15.78:0/761317872 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59380d21e0 0x7f5924003a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:40:47.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.930+0000 7f593cc47700 1 --2- 172.21.15.78:0/761317872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5924004db0 0x7f5924005220 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5930009b70 tx=0x7f59240058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210a56fc5780be08 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:47.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.930+0000 7f59377fe700 1 -- 172.21.15.78:0/761317872 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593000e6c0 con 0x7f5924004db0 2023-10-11T09:40:47.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.930+0000 7f59377fe700 1 -- 172.21.15.78:0/761317872 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593000e820 con 0x7f5924004db0 2023-10-11T09:40:47.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.930+0000 7f59377fe700 1 -- 172.21.15.78:0/761317872 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593000eb90 con 0x7f5924004db0 2023-10-11T09:40:47.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.930+0000 7f593e6aa700 1 -- 172.21.15.78:0/761317872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5924004db0 msgr2=0x7f5924005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:47.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.930+0000 7f593e6aa700 1 --2- 172.21.15.78:0/761317872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5924004db0 0x7f5924005220 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5930009b70 tx=0x7f59240058a0 comp rx=0 tx=0).stop 2023-10-11T09:40:47.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.930+0000 7f593cc47700 1 -- 172.21.15.78:0/761317872 reap_dead start 2023-10-11T09:40:47.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.930+0000 7f593e6aa700 1 -- 172.21.15.78:0/761317872 shutdown_connections 2023-10-11T09:40:47.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.930+0000 7f593e6aa700 1 -- 172.21.15.78:0/761317872 >> 172.21.15.78:0/761317872 conn(0x7f59380cfb60 msgr2=0x7f59380cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:47.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.930+0000 7f593e6aa700 1 -- 172.21.15.78:0/761317872 shutdown_connections 2023-10-11T09:40:47.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.930+0000 7f593e6aa700 1 -- 172.21.15.78:0/761317872 wait complete. 2023-10-11T09:40:47.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.934+0000 7f593e6aa700 1 Processor -- start 2023-10-11T09:40:47.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.934+0000 7f593e6aa700 1 -- start start 2023-10-11T09:40:47.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.938+0000 7f593e6aa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5924004db0 0x7f59380c9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:47.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.938+0000 7f593e6aa700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59380c7620 0x7f593814e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:47.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.938+0000 7f593e6aa700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f593806af50 0x7f593806b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:47.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.938+0000 7f593e6aa700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f593806b930 con 0x7f5924004db0 2023-10-11T09:40:47.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.938+0000 7f593e6aa700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59380c7e00 con 0x7f59380c7620 2023-10-11T09:40:47.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.938+0000 7f593e6aa700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59380c80e0 con 0x7f593806af50 2023-10-11T09:40:47.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.938+0000 7f5937fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5924004db0 0x7f59380c9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:47.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.938+0000 7f5937fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5924004db0 0x7f59380c9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56828/0 (socket says 172.21.15.78:56828) 2023-10-11T09:40:47.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.938+0000 7f5937fff700 1 -- 172.21.15.78:0/1452042149 learned_addr learned my addr 172.21.15.78:0/1452042149 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:47.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.938+0000 7f593cc47700 1 --2- 172.21.15.78:0/1452042149 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f593806af50 0x7f593806b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:47.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.938+0000 7f592f7fe700 1 --2- 172.21.15.78:0/1452042149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59380c7620 0x7f593814e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:47.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.938+0000 7f5937fff700 1 -- 172.21.15.78:0/1452042149 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f593806af50 msgr2=0x7f593806b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:47.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.938+0000 7f5937fff700 1 --2- 172.21.15.78:0/1452042149 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f593806af50 0x7f593806b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:47.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.938+0000 7f5937fff700 1 -- 172.21.15.78:0/1452042149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59380c7620 msgr2=0x7f593814e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:47.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.938+0000 7f5937fff700 1 --2- 172.21.15.78:0/1452042149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59380c7620 0x7f593814e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:47.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.938+0000 7f5937fff700 1 -- 172.21.15.78:0/1452042149 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59380c83c0 con 0x7f5924004db0 2023-10-11T09:40:47.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.938+0000 7f5937fff700 1 --2- 172.21.15.78:0/1452042149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5924004db0 0x7f59380c9be0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5928008c90 tx=0x7f5928008cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:47.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.938+0000 7f59357fa700 1 -- 172.21.15.78:0/1452042149 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5928006e10 con 0x7f5924004db0 2023-10-11T09:40:47.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.938+0000 7f59357fa700 1 -- 172.21.15.78:0/1452042149 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5928015480 con 0x7f5924004db0 2023-10-11T09:40:47.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.938+0000 7f593e6aa700 1 -- 172.21.15.78:0/1452042149 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59380c86a0 con 0x7f5924004db0 2023-10-11T09:40:47.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.938+0000 7f59357fa700 1 -- 172.21.15.78:0/1452042149 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5928015830 con 0x7f5924004db0 2023-10-11T09:40:47.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.938+0000 7f593e6aa700 1 -- 172.21.15.78:0/1452042149 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59380c7a90 con 0x7f5924004db0 2023-10-11T09:40:47.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.942+0000 7f593e6aa700 1 -- 172.21.15.78:0/1452042149 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59380c8bc0 con 0x7f5924004db0 2023-10-11T09:40:47.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.942+0000 7f59357fa700 1 -- 172.21.15.78:0/1452042149 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f5928015990 con 0x7f5924004db0 2023-10-11T09:40:47.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.942+0000 7f59357fa700 1 --2- 172.21.15.78:0/1452042149 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5918065ed0 0x7f5918068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:47.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.942+0000 7f59357fa700 1 -- 172.21.15.78:0/1452042149 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5928083c00 con 0x7f5924004db0 2023-10-11T09:40:47.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.942+0000 7f592f7fe700 1 --2- 172.21.15.78:0/1452042149 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5918065ed0 0x7f5918068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:47.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.946+0000 7f592f7fe700 1 --2- 172.21.15.78:0/1452042149 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5918065ed0 0x7f5918068380 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f593815c9e0 tx=0x7f5920008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:47.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:47.950+0000 7f59357fa700 1 -- 172.21.15.78:0/1452042149 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f59380c8bc0 con 0x7f5924004db0 2023-10-11T09:40:48.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.126+0000 7f593e6aa700 1 -- 172.21.15.78:0/1452042149 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59380c8bc0 con 0x7f5924004db0 2023-10-11T09:40:48.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.126+0000 7f59357fa700 1 -- 172.21.15.78:0/1452042149 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17811 (secure 0 0 0) 0x7f59380c8bc0 con 0x7f5924004db0 2023-10-11T09:40:48.132 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:40:48.133 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":23,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:40:42.678477+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:40:39.862581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T09:40:40.663641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:40:48.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.130+0000 7f593e6aa700 1 -- 172.21.15.78:0/1452042149 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5918065ed0 msgr2=0x7f5918068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.130+0000 7f593e6aa700 1 --2- 172.21.15.78:0/1452042149 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5918065ed0 0x7f5918068380 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f593815c9e0 tx=0x7f5920008040 comp rx=0 tx=0).stop 2023-10-11T09:40:48.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.130+0000 7f593e6aa700 1 -- 172.21.15.78:0/1452042149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5924004db0 msgr2=0x7f59380c9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.130+0000 7f593e6aa700 1 --2- 172.21.15.78:0/1452042149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5924004db0 0x7f59380c9be0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5928008c90 tx=0x7f5928008cc0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.130+0000 7f593e6aa700 1 -- 172.21.15.78:0/1452042149 shutdown_connections 2023-10-11T09:40:48.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.130+0000 7f593e6aa700 1 --2- 172.21.15.78:0/1452042149 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5918065ed0 0x7f5918068380 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.130+0000 7f593e6aa700 1 --2- 172.21.15.78:0/1452042149 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f593806af50 0x7f593806b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.130+0000 7f593e6aa700 1 --2- 172.21.15.78:0/1452042149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59380c7620 0x7f593814e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.130+0000 7f593e6aa700 1 --2- 172.21.15.78:0/1452042149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5924004db0 0x7f59380c9be0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.130+0000 7f593e6aa700 1 -- 172.21.15.78:0/1452042149 >> 172.21.15.78:0/1452042149 conn(0x7f59380cfb60 msgr2=0x7f59380b3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:48.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.130+0000 7f593e6aa700 1 -- 172.21.15.78:0/1452042149 shutdown_connections 2023-10-11T09:40:48.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.130+0000 7f593e6aa700 1 -- 172.21.15.78:0/1452042149 wait complete. 2023-10-11T09:40:48.155 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-11T09:40:48.156 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-11T09:40:48.156 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-11T09:40:48.157 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-11T09:40:48.157 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-11T09:40:48.158 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-11T09:40:48.159 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-11T09:40:48.159 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-11T09:40:48.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.382+0000 7f224ea82700 1 Processor -- start 2023-10-11T09:40:48.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.382+0000 7f224ea82700 1 -- start start 2023-10-11T09:40:48.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.386+0000 7f224ea82700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f22480cd030 0x7f22480b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.386+0000 7f224ea82700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22480c9020 0x7f22480b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.386+0000 7f224ea82700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22480cc5a0 con 0x7f224815c6b0 2023-10-11T09:40:48.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.386+0000 7f224ea82700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f224806d230 con 0x7f22480cd030 2023-10-11T09:40:48.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.386+0000 7f224ea82700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22480cc7f0 con 0x7f22480c9020 2023-10-11T09:40:48.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.386+0000 7f224c81e700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f224815c6b0 0x7f2248160ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:58704/0 (socket says 172.21.15.78:58704) 2023-10-11T09:40:48.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.386+0000 7f224c81e700 1 -- 172.21.15.78:0/427495737 learned_addr learned my addr 172.21.15.78:0/427495737 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:48.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.386+0000 7f4f69237700 1 Processor -- start 2023-10-11T09:40:48.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.386+0000 7f4f69237700 1 -- start start 2023-10-11T09:40:48.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.386+0000 7f2247fff700 1 --2- 172.21.15.78:0/427495737 >> v2:172.21.15.133:3300/0 conn(0x7f22480cd030 0x7f22480b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.386+0000 7f224d01f700 1 --2- 172.21.15.78:0/427495737 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22480c9020 0x7f22480b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.386+0000 7f2247fff700 1 -- 172.21.15.78:0/427495737 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22480c9020 msgr2=0x7f22480b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:48.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.386+0000 7f2247fff700 1 --2- 172.21.15.78:0/427495737 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22480c9020 0x7f22480b0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.386+0000 7f2247fff700 1 -- 172.21.15.78:0/427495737 >> v1:172.21.15.78:6789/0 conn(0x7f224815c6b0 legacy=0x7f2248160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.386+0000 7f2247fff700 1 -- 172.21.15.78:0/427495737 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22480ccad0 con 0x7f22480cd030 2023-10-11T09:40:48.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.386+0000 7f2247fff700 1 --2- 172.21.15.78:0/427495737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22480cd030 0x7f22480b06a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f223c009850 tx=0x7f223c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5083aeaaa8a61e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.386+0000 7f4f69237700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f4f640cd030 0x7f4f640b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.386+0000 7f4f69237700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f640cc5a0 con 0x7f4f640c9020 2023-10-11T09:40:48.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.386+0000 7f4f69237700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f6406d230 con 0x7f4f6415c6b0 2023-10-11T09:40:48.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.386+0000 7f4f69237700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f640cc7f0 con 0x7f4f640cd030 2023-10-11T09:40:48.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.386+0000 7f22477fe700 1 -- 172.21.15.78:0/427495737 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223c004230 con 0x7f22480cd030 2023-10-11T09:40:48.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.386+0000 7f22477fe700 1 -- 172.21.15.78:0/427495737 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f223c004390 con 0x7f22480cd030 2023-10-11T09:40:48.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.386+0000 7f22477fe700 1 -- 172.21.15.78:0/427495737 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223c0036c0 con 0x7f22480cd030 2023-10-11T09:40:48.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.390+0000 7f224ea82700 1 -- 172.21.15.78:0/427495737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22480cd030 msgr2=0x7f22480b06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.390+0000 7f224ea82700 1 --2- 172.21.15.78:0/427495737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22480cd030 0x7f22480b06a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f223c009850 tx=0x7f223c009b60 comp rx=0 tx=0).stop 2023-10-11T09:40:48.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.390+0000 7f4f62d9d700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f4f6415c6b0 0x7f4f64160ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:33742/0 (socket says 172.21.15.78:33742) 2023-10-11T09:40:48.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.390+0000 7f4f62d9d700 1 -- 172.21.15.78:0/4080849000 learned_addr learned my addr 172.21.15.78:0/4080849000 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:48.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.390+0000 7f224ea82700 1 -- 172.21.15.78:0/427495737 shutdown_connections 2023-10-11T09:40:48.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.390+0000 7f224ea82700 1 --2- 172.21.15.78:0/427495737 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22480c9020 0x7f22480b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.390+0000 7f224ea82700 1 --2- 172.21.15.78:0/427495737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22480cd030 0x7f22480b06a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.390+0000 7f224ea82700 1 -- 172.21.15.78:0/427495737 >> 172.21.15.78:0/427495737 conn(0x7f224806aca0 msgr2=0x7f22480cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:48.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.390+0000 7f224ea82700 1 -- 172.21.15.78:0/427495737 shutdown_connections 2023-10-11T09:40:48.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.390+0000 7f224ea82700 1 -- 172.21.15.78:0/427495737 wait complete. 2023-10-11T09:40:48.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.390+0000 7f4f6259c700 1 --2- 172.21.15.78:0/4080849000 >> v2:172.21.15.78:3300/0 conn(0x7f4f640cd030 0x7f4f640b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.390+0000 7f98ba71f700 1 Processor -- start 2023-10-11T09:40:48.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.390+0000 7f4f61d9b700 1 -- 172.21.15.78:0/4080849000 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1024830466 0 0) 0x7f4f6406d230 con 0x7f4f6415c6b0 2023-10-11T09:40:48.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.390+0000 7f4f61d9b700 1 -- 172.21.15.78:0/4080849000 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f4c0035d0 con 0x7f4f6415c6b0 2023-10-11T09:40:48.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.390+0000 7f98ba71f700 1 -- start start 2023-10-11T09:40:48.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.390+0000 7f4f6259c700 1 -- 172.21.15.78:0/4080849000 >> v1:172.21.15.133:6789/0 conn(0x7f4f6415c6b0 legacy=0x7f4f64160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.390+0000 7f4f6259c700 1 -- 172.21.15.78:0/4080849000 >> v1:172.21.15.78:6789/0 conn(0x7f4f640c9020 legacy=0x7f4f6406c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:48.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.390+0000 7f4f6259c700 1 -- 172.21.15.78:0/4080849000 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f640ccad0 con 0x7f4f640cd030 2023-10-11T09:40:48.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.390+0000 7f4f6259c700 1 --2- 172.21.15.78:0/4080849000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f640cd030 0x7f4f640b06a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c009d90 tx=0x7f4f5c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91171dc044ea7b84 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.390+0000 7f224ea82700 1 Processor -- start 2023-10-11T09:40:48.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.390+0000 7f224ea82700 1 -- start start 2023-10-11T09:40:48.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.390+0000 7f98ba71f700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f98b41586e0 0x7f98b406bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.390+0000 7f4f61d9b700 1 -- 172.21.15.78:0/4080849000 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c00e070 con 0x7f4f640cd030 2023-10-11T09:40:48.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f4f61d9b700 1 -- 172.21.15.78:0/4080849000 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f5c00bb20 con 0x7f4f640cd030 2023-10-11T09:40:48.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f4f61d9b700 1 -- 172.21.15.78:0/4080849000 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c012430 con 0x7f4f640cd030 2023-10-11T09:40:48.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f98ba71f700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98b406c440 con 0x7f98b40b0db0 2023-10-11T09:40:48.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f98ba71f700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98b406c720 con 0x7f98b406d380 2023-10-11T09:40:48.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f98b3fff700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f98b41586e0 0x7f98b406bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f4f69237700 1 -- 172.21.15.78:0/4080849000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f640cd030 msgr2=0x7f4f640b06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f4f69237700 1 --2- 172.21.15.78:0/4080849000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f640cd030 0x7f4f640b06a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c009d90 tx=0x7f4f5c00b330 comp rx=0 tx=0).stop 2023-10-11T09:40:48.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f224ea82700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22480c9020 0x7f224814df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f224ea82700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22480cd030 0x7f224814fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f224ea82700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f224815c6b0 0x7f22480b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f224ea82700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f224815cd50 con 0x7f22480cd030 2023-10-11T09:40:48.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f224ea82700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f224814e440 con 0x7f22480c9020 2023-10-11T09:40:48.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f224ea82700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f224815cfa0 con 0x7f224815c6b0 2023-10-11T09:40:48.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f98ab7fe700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f98b406d380 0x7f98b40b06a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:33750/0 (socket says 172.21.15.78:33750) 2023-10-11T09:40:48.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f98ab7fe700 1 -- 172.21.15.78:0/2972431039 learned_addr learned my addr 172.21.15.78:0/2972431039 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:48.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f98ba71f700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98b406ca00 con 0x7f98b41586e0 2023-10-11T09:40:48.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f4f69237700 1 -- 172.21.15.78:0/4080849000 shutdown_connections 2023-10-11T09:40:48.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f4f69237700 1 --2- 172.21.15.78:0/4080849000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f640cd030 0x7f4f640b06a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f4f69237700 1 -- 172.21.15.78:0/4080849000 >> 172.21.15.78:0/4080849000 conn(0x7f4f6406aca0 msgr2=0x7f4f640cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:48.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f4f69237700 1 -- 172.21.15.78:0/4080849000 shutdown_connections 2023-10-11T09:40:48.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f4f69237700 1 -- 172.21.15.78:0/4080849000 wait complete. 2023-10-11T09:40:48.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f98b3fff700 1 -- 172.21.15.78:0/2972431039 >> v1:172.21.15.133:6789/0 conn(0x7f98b406d380 legacy=0x7f98b40b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f98b3fff700 1 -- 172.21.15.78:0/2972431039 >> v1:172.21.15.78:6789/0 conn(0x7f98b40b0db0 legacy=0x7f98b4070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:48.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f98b3fff700 1 -- 172.21.15.78:0/2972431039 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b406fd10 con 0x7f98b41586e0 2023-10-11T09:40:48.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f224c81e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22480c9020 0x7f224814df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f224c81e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22480c9020 0x7f224814df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:49934/0 (socket says 172.21.15.78:49934) 2023-10-11T09:40:48.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f224c81e700 1 -- 172.21.15.78:0/2823643740 learned_addr learned my addr 172.21.15.78:0/2823643740 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:48.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f224d01f700 1 --2- 172.21.15.78:0/2823643740 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f224815c6b0 0x7f22480b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f2247fff700 1 --2- 172.21.15.78:0/2823643740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22480cd030 0x7f224814fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f98b3fff700 1 --2- 172.21.15.78:0/2972431039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98b41586e0 0x7f98b406bbf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f989c00a970 tx=0x7f989c00ac80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10bb626bb987c863 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f224d01f700 1 -- 172.21.15.78:0/2823643740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22480c9020 msgr2=0x7f224814df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f224d01f700 1 --2- 172.21.15.78:0/2823643740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22480c9020 0x7f224814df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f224d01f700 1 -- 172.21.15.78:0/2823643740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22480cd030 msgr2=0x7f224814fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f224d01f700 1 --2- 172.21.15.78:0/2823643740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22480cd030 0x7f224814fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f224d01f700 1 -- 172.21.15.78:0/2823643740 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f223c0094b0 con 0x7f224815c6b0 2023-10-11T09:40:48.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f98b37fe700 1 -- 172.21.15.78:0/2972431039 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c005e50 con 0x7f98b41586e0 2023-10-11T09:40:48.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f98b37fe700 1 -- 172.21.15.78:0/2972431039 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989c004030 con 0x7f98b41586e0 2023-10-11T09:40:48.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f4fc61eb700 1 Processor -- start 2023-10-11T09:40:48.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.394+0000 7f4fc61eb700 1 -- start start 2023-10-11T09:40:48.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.398+0000 7f98ba71f700 1 -- 172.21.15.78:0/2972431039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98b41586e0 msgr2=0x7f98b406bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.398+0000 7f98ba71f700 1 --2- 172.21.15.78:0/2972431039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98b41586e0 0x7f98b406bbf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f989c00a970 tx=0x7f989c00ac80 comp rx=0 tx=0).stop 2023-10-11T09:40:48.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.398+0000 7f4f69237700 1 Processor -- start 2023-10-11T09:40:48.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.398+0000 7f224d01f700 1 --2- 172.21.15.78:0/2823643740 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f224815c6b0 0x7f22480b1a10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f223800e7e0 tx=0x7f223800eaf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.398+0000 7f98ba71f700 1 -- 172.21.15.78:0/2972431039 shutdown_connections 2023-10-11T09:40:48.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.398+0000 7f98ba71f700 1 --2- 172.21.15.78:0/2972431039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98b41586e0 0x7f98b406bbf0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.398+0000 7f98ba71f700 1 -- 172.21.15.78:0/2972431039 >> 172.21.15.78:0/2972431039 conn(0x7f98b406aca0 msgr2=0x7f98b40cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:48.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.398+0000 7f98ba71f700 1 -- 172.21.15.78:0/2972431039 shutdown_connections 2023-10-11T09:40:48.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.398+0000 7f22457fa700 1 -- 172.21.15.78:0/2823643740 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223800c070 con 0x7f224815c6b0 2023-10-11T09:40:48.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.398+0000 7f22457fa700 1 -- 172.21.15.78:0/2823643740 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2238013610 con 0x7f224815c6b0 2023-10-11T09:40:48.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.398+0000 7f22457fa700 1 -- 172.21.15.78:0/2823643740 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22380139c0 con 0x7f224815c6b0 2023-10-11T09:40:48.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.398+0000 7f224ea82700 1 -- 172.21.15.78:0/2823643740 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f224815d280 con 0x7f224815c6b0 2023-10-11T09:40:48.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.398+0000 7f4f69237700 1 -- start start 2023-10-11T09:40:48.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.398+0000 7f4fc61eb700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f4fc006d5f0 0x7f4fc00b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.398+0000 7f4fc61eb700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fc006d9c0 con 0x7f4fc0148250 2023-10-11T09:40:48.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.398+0000 7f4fc61eb700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fc0161520 con 0x7f4fc0161090 2023-10-11T09:40:48.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.398+0000 7f224ea82700 1 -- 172.21.15.78:0/2823643740 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f224815d840 con 0x7f224815c6b0 2023-10-11T09:40:48.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.398+0000 7f4fc61eb700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fc0148690 con 0x7f4fc006d5f0 2023-10-11T09:40:48.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.398+0000 7f4fbffff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f4fc0148250 0x7f4fc00ca210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:58730/0 (socket says 172.21.15.78:58730) 2023-10-11T09:40:48.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.398+0000 7f4fbffff700 1 -- 172.21.15.78:0/1367114020 learned_addr learned my addr 172.21.15.78:0/1367114020 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:48.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.398+0000 7f4fbf7fe700 1 --2- 172.21.15.78:0/1367114020 >> v2:172.21.15.78:3300/0 conn(0x7f4fc006d5f0 0x7f4fc00b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.398+0000 7f4f69237700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4f640c9020 0x7f4f6414e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.398+0000 7f4f69237700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f640cd030 0x7f4f640affa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.398+0000 7f4f69237700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f6415c6b0 0x7f4f6414a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.398+0000 7f4f69237700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f641e5c00 con 0x7f4f640cd030 2023-10-11T09:40:48.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.398+0000 7f4f69237700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6414ea80 con 0x7f4f6415c6b0 2023-10-11T09:40:48.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.398+0000 7f4f69237700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f641e5e50 con 0x7f4f640c9020 2023-10-11T09:40:48.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.398+0000 7f98ba71f700 1 -- 172.21.15.78:0/2972431039 wait complete. 2023-10-11T09:40:48.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.398+0000 7f98ba71f700 1 Processor -- start 2023-10-11T09:40:48.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4f6259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f640cd030 0x7f4f640affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4f62d9d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4f640c9020 0x7f4f6414e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4f6259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f640cd030 0x7f4f640affa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56872/0 (socket says 172.21.15.78:56872) 2023-10-11T09:40:48.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4f6259c700 1 -- 172.21.15.78:0/2496708753 learned_addr learned my addr 172.21.15.78:0/2496708753 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:48.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4f6359e700 1 --2- 172.21.15.78:0/2496708753 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f6415c6b0 0x7f4f6414a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4fbe7fc700 1 -- 172.21.15.78:0/1367114020 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3899062662 0 0) 0x7f4fb8001490 con 0x7f4fc0148250 2023-10-11T09:40:48.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4fbe7fc700 1 -- 172.21.15.78:0/1367114020 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fa80035d0 con 0x7f4fc0148250 2023-10-11T09:40:48.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4fbe7fc700 1 -- 172.21.15.78:0/1367114020 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 224283973 0 0) 0x7f4fc0161520 con 0x7f4fc0161090 2023-10-11T09:40:48.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4fbe7fc700 1 -- 172.21.15.78:0/1367114020 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fb8001490 con 0x7f4fc0161090 2023-10-11T09:40:48.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f224ea82700 1 -- 172.21.15.78:0/2823643740 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f224806d760 con 0x7f224815c6b0 2023-10-11T09:40:48.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4fbe7fc700 1 -- 172.21.15.78:0/1367114020 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2793522525 0 0) 0x7f4fa80035d0 con 0x7f4fc0148250 2023-10-11T09:40:48.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4fbe7fc700 1 -- 172.21.15.78:0/1367114020 >> v2:172.21.15.78:3300/0 conn(0x7f4fc006d5f0 msgr2=0x7f4fc00b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:48.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f98ba71f700 1 -- start start 2023-10-11T09:40:48.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4f62d9d700 1 -- 172.21.15.78:0/2496708753 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f6415c6b0 msgr2=0x7f4f6414a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4f62d9d700 1 --2- 172.21.15.78:0/2496708753 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f6415c6b0 0x7f4f6414a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4f62d9d700 1 -- 172.21.15.78:0/2496708753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f640cd030 msgr2=0x7f4f640affa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4f62d9d700 1 --2- 172.21.15.78:0/2496708753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f640cd030 0x7f4f640affa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4f62d9d700 1 -- 172.21.15.78:0/2496708753 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f5c009a70 con 0x7f4f640c9020 2023-10-11T09:40:48.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4fbe7fc700 1 --2- 172.21.15.78:0/1367114020 >> v2:172.21.15.78:3300/0 conn(0x7f4fc006d5f0 0x7f4fc00b06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4fbe7fc700 1 -- 172.21.15.78:0/1367114020 >> v1:172.21.15.133:6789/0 conn(0x7f4fc0161090 legacy=0x7f4fc0147b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4fbe7fc700 1 -- 172.21.15.78:0/1367114020 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc0161800 con 0x7f4fc0148250 2023-10-11T09:40:48.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4fbe7fc700 1 -- 172.21.15.78:0/1367114020 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f4fb8001b80 con 0x7f4fc0148250 2023-10-11T09:40:48.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4fbe7fc700 1 -- 172.21.15.78:0/1367114020 >> v1:172.21.15.78:6789/0 conn(0x7f4fc0148250 legacy=0x7f4fc00ca210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4fbe7fc700 1 --2- 172.21.15.78:0/1367114020 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fa8004c90 0x7f4fa8005060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4fbe7fc700 1 --2- 172.21.15.78:0/1367114020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fa8003810 0x7f4fa8003c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4fbe7fc700 1 --2- 172.21.15.78:0/1367114020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fa8004210 0x7f4fa800bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4fbe7fc700 1 -- 172.21.15.78:0/1367114020 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb00010e0 con 0x7f4fa8003810 2023-10-11T09:40:48.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4fbe7fc700 1 -- 172.21.15.78:0/1367114020 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc0148690 con 0x7f4fa8004210 2023-10-11T09:40:48.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4fbe7fc700 1 -- 172.21.15.78:0/1367114020 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa800c280 con 0x7f4fa8004c90 2023-10-11T09:40:48.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4fbf7fe700 1 --2- 172.21.15.78:0/1367114020 >> v2:172.21.15.78:3300/0 conn(0x7f4fc006d5f0 0x7f4fc00b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:40:48.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4fbeffd700 1 --2- 172.21.15.78:0/1367114020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fa8003810 0x7f4fa8003c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4f62d9d700 1 --2- 172.21.15.78:0/2496708753 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4f640c9020 0x7f4f6414e180 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4f5400c410 tx=0x7f4f5400c720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4fbffff700 1 --2- 172.21.15.78:0/1367114020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fa8004210 0x7f4fa800bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4fbf7fe700 1 --2- 172.21.15.78:0/1367114020 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fa8004c90 0x7f4fa8005060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f22457fa700 1 -- 172.21.15.78:0/2823643740 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f2238013b20 con 0x7f224815c6b0 2023-10-11T09:40:48.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4f4b7fe700 1 -- 172.21.15.78:0/2496708753 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f54005d90 con 0x7f4f640c9020 2023-10-11T09:40:48.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4fbeffd700 1 -- 172.21.15.78:0/1367114020 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fa8004c90 msgr2=0x7f4fa8005060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:48.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4fbeffd700 1 --2- 172.21.15.78:0/1367114020 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fa8004c90 0x7f4fa8005060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4fbeffd700 1 -- 172.21.15.78:0/1367114020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fa8004210 msgr2=0x7f4fa800bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:48.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4fbeffd700 1 --2- 172.21.15.78:0/1367114020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fa8004210 0x7f4fa800bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4fbeffd700 1 -- 172.21.15.78:0/1367114020 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa80035d0 con 0x7f4fa8003810 2023-10-11T09:40:48.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4f4b7fe700 1 -- 172.21.15.78:0/2496708753 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f5400ce20 con 0x7f4f640c9020 2023-10-11T09:40:48.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4fbffff700 1 -- 172.21.15.78:0/1367114020 reap_dead start 2023-10-11T09:40:48.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f8a5f330700 1 Processor -- start 2023-10-11T09:40:48.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f8a5f330700 1 -- start start 2023-10-11T09:40:48.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f22457fa700 1 --2- 172.21.15.78:0/2823643740 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2234065fa0 0x7f2234068450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4fbeffd700 1 --2- 172.21.15.78:0/1367114020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fa8003810 0x7f4fa8003c80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0007e20 tx=0x7f4fa80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e8d8c9f63e1fe40 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f98ba71f700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f98b406d380 0x7f98b414bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f98ba71f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98b40b0db0 0x7f98b4155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f22457fa700 1 -- 172.21.15.78:0/2823643740 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2238083460 con 0x7f224815c6b0 2023-10-11T09:40:48.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f22457fa700 1 --2- 172.21.15.78:0/2823643740 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x7f223406e850 0x7f2234070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f22457fa700 1 -- 172.21.15.78:0/2823643740 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2234071310 con 0x7f223406e850 2023-10-11T09:40:48.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f22457fa700 1 -- 172.21.15.78:0/2823643740 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2238085a80 con 0x7f224815c6b0 2023-10-11T09:40:48.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4f4b7fe700 1 -- 172.21.15.78:0/2496708753 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f540072d0 con 0x7f4f640c9020 2023-10-11T09:40:48.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4f69237700 1 -- 172.21.15.78:0/2496708753 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f641e6130 con 0x7f4f640c9020 2023-10-11T09:40:48.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4f69237700 1 -- 172.21.15.78:0/2496708753 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f640b1bf0 con 0x7f4f640c9020 2023-10-11T09:40:48.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4fbe7fc700 1 -- 172.21.15.78:0/1367114020 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb00052d0 con 0x7f4fa8003810 2023-10-11T09:40:48.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4fbe7fc700 1 -- 172.21.15.78:0/1367114020 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb0005430 con 0x7f4fa8003810 2023-10-11T09:40:48.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f4fbe7fc700 1 -- 172.21.15.78:0/1367114020 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb000cb50 con 0x7f4fa8003810 2023-10-11T09:40:48.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f243cd0c700 1 Processor -- start 2023-10-11T09:40:48.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.402+0000 7f2247fff700 1 --2- 172.21.15.78:0/2823643740 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x7f223406e850 0x7f2234070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f243cd0c700 1 -- start start 2023-10-11T09:40:48.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f98ba71f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98b41586e0 0x7f98b40b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f98ab7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98b40b0db0 0x7f98b4155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f98ba71f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b4156ce0 con 0x7f98b40b0db0 2023-10-11T09:40:48.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f98ba71f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b41560d0 con 0x7f98b41586e0 2023-10-11T09:40:48.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f98ab7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98b40b0db0 0x7f98b4155630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56894/0 (socket says 172.21.15.78:56894) 2023-10-11T09:40:48.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f98ba71f700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b41563b0 con 0x7f98b406d380 2023-10-11T09:40:48.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f98ab7fe700 1 -- 172.21.15.78:0/1543056024 learned_addr learned my addr 172.21.15.78:0/1543056024 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:48.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f98b8cbc700 1 --2- 172.21.15.78:0/1543056024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98b41586e0 0x7f98b40b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f98ab7fe700 1 -- 172.21.15.78:0/1543056024 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f98b406d380 msgr2=0x7f98b414bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f98ab7fe700 1 --2- 172.21.15.78:0/1543056024 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f98b406d380 0x7f98b414bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f98ab7fe700 1 -- 172.21.15.78:0/1543056024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98b41586e0 msgr2=0x7f98b40b2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f98ab7fe700 1 --2- 172.21.15.78:0/1543056024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98b41586e0 0x7f98b40b2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f98ab7fe700 1 -- 172.21.15.78:0/1543056024 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989c00a5d0 con 0x7f98b40b0db0 2023-10-11T09:40:48.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f2247fff700 1 --2- 172.21.15.78:0/2823643740 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x7f223406e850 0x7f2234070c60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f224c81e700 1 --2- 172.21.15.78:0/2823643740 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2234065fa0 0x7f2234068450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f224c81e700 1 --2- 172.21.15.78:0/2823643740 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2234065fa0 0x7f2234068450 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2240002230 tx=0x7f2240001e10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f4fc61eb700 1 -- 172.21.15.78:0/1367114020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fa8003810 msgr2=0x7f4fa8003c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f4fc61eb700 1 --2- 172.21.15.78:0/1367114020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fa8003810 0x7f4fa8003c80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0007e20 tx=0x7f4fa80058a0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f98ab7fe700 1 --2- 172.21.15.78:0/1543056024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98b40b0db0 0x7f98b4155630 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f98a400c410 tx=0x7f98a400c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f22457fa700 1 -- 172.21.15.78:0/2823643740 <== osd.0 v2:172.21.15.78:6800/35735 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f2234071310 con 0x7f223406e850 2023-10-11T09:40:48.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f4fc61eb700 1 -- 172.21.15.78:0/1367114020 shutdown_connections 2023-10-11T09:40:48.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f4fc61eb700 1 --2- 172.21.15.78:0/1367114020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fa8003810 0x7f4fa8003c80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f4fc61eb700 1 -- 172.21.15.78:0/1367114020 >> 172.21.15.78:0/1367114020 conn(0x7f4fc006aca0 msgr2=0x7f4fc00cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:48.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f4f69237700 1 -- 172.21.15.78:0/2496708753 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4f6406d760 con 0x7f4f640c9020 2023-10-11T09:40:48.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f8a5f330700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8a5806d5f0 0x7f8a580b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f8a5f330700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f8a58161090 0x7f8a580b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f8a5f330700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a5806d9c0 con 0x7f8a58147b40 2023-10-11T09:40:48.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f8a5f330700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a58148690 con 0x7f8a58161090 2023-10-11T09:40:48.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f8a5f330700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a580cb220 con 0x7f8a5806d5f0 2023-10-11T09:40:48.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f4fc61eb700 1 -- 172.21.15.78:0/1367114020 shutdown_connections 2023-10-11T09:40:48.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f8a5eb2f700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f8a58147b40 0x7f8a58147f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:33770/0 (socket says 172.21.15.78:33770) 2023-10-11T09:40:48.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f243cd0c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f2430036d40 0x7f24300f1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f8a5eb2f700 1 -- 172.21.15.78:0/3187889575 learned_addr learned my addr 172.21.15.78:0/3187889575 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:48.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f243659c700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f24300e6d20 0x7f243004f200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:58738/0 (socket says 172.21.15.78:58738) 2023-10-11T09:40:48.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f8a5e32e700 1 --2- 172.21.15.78:0/3187889575 >> v2:172.21.15.133:3300/0 conn(0x7f8a5806d5f0 0x7f8a580b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f8a5db2d700 1 --2- 172.21.15.78:0/3187889575 >> v2:172.21.15.78:3300/0 conn(0x7f8a58161090 0x7f8a580b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f2435d9b700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f2430051170 0x7f2430036630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:33780/0 (socket says 172.21.15.78:33780) 2023-10-11T09:40:48.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f243659c700 1 -- 172.21.15.78:0/2955997057 learned_addr learned my addr 172.21.15.78:0/2955997057 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:48.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f243cd0c700 1 -- 172.21.15.78:0/2955997057 --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2430053cb0 con 0x7f24300e6d20 2023-10-11T09:40:48.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f4f4b7fe700 1 -- 172.21.15.78:0/2496708753 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f4f54007430 con 0x7f4f640c9020 2023-10-11T09:40:48.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f8a5e32e700 1 -- 172.21.15.78:0/3187889575 >> v2:172.21.15.78:3300/0 conn(0x7f8a58161090 msgr2=0x7f8a580b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:48.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f8a5e32e700 1 --2- 172.21.15.78:0/3187889575 >> v2:172.21.15.78:3300/0 conn(0x7f8a58161090 0x7f8a580b0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f8a5e32e700 1 -- 172.21.15.78:0/3187889575 >> v1:172.21.15.133:6789/0 conn(0x7f8a58147b40 legacy=0x7f8a58147f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f8a5e32e700 1 -- 172.21.15.78:0/3187889575 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a58161520 con 0x7f8a5806d5f0 2023-10-11T09:40:48.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f98b17fa700 1 -- 172.21.15.78:0/1543056024 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a4005d90 con 0x7f98b40b0db0 2023-10-11T09:40:48.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f98b17fa700 1 -- 172.21.15.78:0/1543056024 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a40070d0 con 0x7f98b40b0db0 2023-10-11T09:40:48.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f98b17fa700 1 -- 172.21.15.78:0/1543056024 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a4007480 con 0x7f98b40b0db0 2023-10-11T09:40:48.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f2436d9d700 1 --2- 172.21.15.78:0/2955997057 >> v2:172.21.15.78:3300/0 conn(0x7f2430036d40 0x7f24300f1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f243cd0c700 1 -- 172.21.15.78:0/2955997057 --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f243004f910 con 0x7f2430051170 2023-10-11T09:40:48.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f243cd0c700 1 -- 172.21.15.78:0/2955997057 --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2430048570 con 0x7f2430036d40 2023-10-11T09:40:48.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f8a5e32e700 1 --2- 172.21.15.78:0/3187889575 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a5806d5f0 0x7f8a580b06a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8a580cb440 tx=0x7f8a5000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d15a19ee9f53f1ac server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.406+0000 7f4fc61eb700 1 -- 172.21.15.78:0/1367114020 wait complete. 2023-10-11T09:40:48.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f4f4b7fe700 1 --2- 172.21.15.78:0/2496708753 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4f50065ed0 0x7f4f50068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f243559a700 1 -- 172.21.15.78:0/2955997057 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4243879698 0 0) 0x7f2430053cb0 con 0x7f24300e6d20 2023-10-11T09:40:48.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f243559a700 1 -- 172.21.15.78:0/2955997057 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24200035d0 con 0x7f24300e6d20 2023-10-11T09:40:48.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f4f4b7fe700 1 -- 172.21.15.78:0/2496708753 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4f540825d0 con 0x7f4f640c9020 2023-10-11T09:40:48.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f4f4b7fe700 1 --2- 172.21.15.78:0/2496708753 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x7f4f5006e700 0x7f4f50070b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f4f4b7fe700 1 -- 172.21.15.78:0/2496708753 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4f500711c0 con 0x7f4f5006e700 2023-10-11T09:40:48.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f4f4b7fe700 1 -- 172.21.15.78:0/2496708753 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f54084c90 con 0x7f4f640c9020 2023-10-11T09:40:48.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f4f6259c700 1 --2- 172.21.15.78:0/2496708753 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4f50065ed0 0x7f4f50068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f8a5d32c700 1 -- 172.21.15.78:0/3187889575 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5000e070 con 0x7f8a5806d5f0 2023-10-11T09:40:48.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f8a5d32c700 1 -- 172.21.15.78:0/3187889575 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a5000bad0 con 0x7f8a5806d5f0 2023-10-11T09:40:48.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f243559a700 1 -- 172.21.15.78:0/2955997057 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1451157607 0 0) 0x7f24200035d0 con 0x7f24300e6d20 2023-10-11T09:40:48.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f243559a700 1 -- 172.21.15.78:0/2955997057 >> v2:172.21.15.78:3300/0 conn(0x7f2430036d40 msgr2=0x7f24300f1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:48.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f243559a700 1 --2- 172.21.15.78:0/2955997057 >> v2:172.21.15.78:3300/0 conn(0x7f2430036d40 0x7f24300f1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f243559a700 1 -- 172.21.15.78:0/2955997057 >> v1:172.21.15.133:6789/0 conn(0x7f2430051170 legacy=0x7f2430036630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f243559a700 1 -- 172.21.15.78:0/2955997057 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2430033940 con 0x7f24300e6d20 2023-10-11T09:40:48.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f243559a700 1 -- 172.21.15.78:0/2955997057 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f2428001670 con 0x7f24300e6d20 2023-10-11T09:40:48.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f243559a700 1 -- 172.21.15.78:0/2955997057 >> v1:172.21.15.78:6789/0 conn(0x7f24300e6d20 legacy=0x7f243004f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f243559a700 1 --2- 172.21.15.78:0/2955997057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2420003a20 0x7f2420003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f243559a700 1 --2- 172.21.15.78:0/2955997057 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2420004410 0x7f24200048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f243559a700 1 --2- 172.21.15.78:0/2955997057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24200050d0 0x7f242000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f243559a700 1 -- 172.21.15.78:0/2955997057 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2430048570 con 0x7f2420003a20 2023-10-11T09:40:48.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f243559a700 1 -- 172.21.15.78:0/2955997057 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242000c020 con 0x7f24200050d0 2023-10-11T09:40:48.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f243559a700 1 -- 172.21.15.78:0/2955997057 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f242000c1e0 con 0x7f2420004410 2023-10-11T09:40:48.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f4f6359e700 1 --2- 172.21.15.78:0/2496708753 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x7f4f5006e700 0x7f4f50070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f4f6259c700 1 --2- 172.21.15.78:0/2496708753 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4f50065ed0 0x7f4f50068380 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4f6414d150 tx=0x7f4f5c011000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f8a5d32c700 1 -- 172.21.15.78:0/3187889575 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5000be80 con 0x7f8a5806d5f0 2023-10-11T09:40:48.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f4fc61eb700 1 Processor -- start 2023-10-11T09:40:48.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7ffb2dc8c700 1 Processor -- start 2023-10-11T09:40:48.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f2436d9d700 1 --2- 172.21.15.78:0/2955997057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24200050d0 0x7f242000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f2435d9b700 1 --2- 172.21.15.78:0/2955997057 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2420004410 0x7f24200048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f8a5f330700 1 -- 172.21.15.78:0/3187889575 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a5806d5f0 msgr2=0x7f8a580b06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f8a5f330700 1 --2- 172.21.15.78:0/3187889575 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a5806d5f0 0x7f8a580b06a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8a580cb440 tx=0x7f8a5000b330 comp rx=0 tx=0).stop 2023-10-11T09:40:48.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f98ba71f700 1 -- 172.21.15.78:0/1543056024 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98b4156690 con 0x7f98b40b0db0 2023-10-11T09:40:48.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f98ba71f700 1 -- 172.21.15.78:0/1543056024 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98b40cd170 con 0x7f98b40b0db0 2023-10-11T09:40:48.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7ffb2dc8c700 1 -- start start 2023-10-11T09:40:48.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f2436d9d700 1 -- 172.21.15.78:0/2955997057 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2420004410 msgr2=0x7f24200048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:48.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f2436d9d700 1 --2- 172.21.15.78:0/2955997057 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2420004410 0x7f24200048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f2436d9d700 1 -- 172.21.15.78:0/2955997057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2420003a20 msgr2=0x7f2420003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:48.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f2436d9d700 1 --2- 172.21.15.78:0/2955997057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2420003a20 0x7f2420003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f2436d9d700 1 -- 172.21.15.78:0/2955997057 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2430033940 con 0x7f24200050d0 2023-10-11T09:40:48.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f4fc61eb700 1 -- start start 2023-10-11T09:40:48.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f2436d9d700 1 -- 172.21.15.78:0/2955997057 reap_dead start 2023-10-11T09:40:48.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f8a5f330700 1 -- 172.21.15.78:0/3187889575 shutdown_connections 2023-10-11T09:40:48.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f8a5f330700 1 --2- 172.21.15.78:0/3187889575 >> v2:172.21.15.78:3300/0 conn(0x7f8a58161090 0x7f8a580b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f8a5f330700 1 --2- 172.21.15.78:0/3187889575 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a5806d5f0 0x7f8a580b06a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.410+0000 7f8a5f330700 1 -- 172.21.15.78:0/3187889575 >> 172.21.15.78:0/3187889575 conn(0x7f8a5806aca0 msgr2=0x7f8a580cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:48.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f4f6359e700 1 --2- 172.21.15.78:0/2496708753 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x7f4f5006e700 0x7f4f50070b10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f2436d9d700 1 --2- 172.21.15.78:0/2955997057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24200050d0 0x7f242000ba30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f24380691c0 tx=0x7f24200058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81751d2dd9cb07fc server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f8a5f330700 1 -- 172.21.15.78:0/3187889575 shutdown_connections 2023-10-11T09:40:48.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f243559a700 1 -- 172.21.15.78:0/2955997057 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2438068150 con 0x7f24200050d0 2023-10-11T09:40:48.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f243559a700 1 -- 172.21.15.78:0/2955997057 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243808f070 con 0x7f24200050d0 2023-10-11T09:40:48.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f243559a700 1 -- 172.21.15.78:0/2955997057 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2438067300 con 0x7f24200050d0 2023-10-11T09:40:48.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f8a5f330700 1 -- 172.21.15.78:0/3187889575 wait complete. 2023-10-11T09:40:48.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f4fc61eb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fa8003810 0x7f4fc01ffb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f4fc61eb700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fc01fbf60 0x7f4fc01ff3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f4fc61eb700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fc01fc580 0x7f4fc00ce980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f4fc61eb700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc0200e10 con 0x7f4fa8003810 2023-10-11T09:40:48.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f4fc61eb700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc01fc3d0 con 0x7f4fc01fc580 2023-10-11T09:40:48.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f4fc61eb700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc01ff920 con 0x7f4fc01fbf60 2023-10-11T09:40:48.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f8b85990700 1 Processor -- start 2023-10-11T09:40:48.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f8b85990700 1 -- start start 2023-10-11T09:40:48.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7ffb2dc8c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffb200e1780 0x7ffb200e4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7ffb2dc8c700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ffb200e0c80 0x7ffb200512e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7ffb2dc8c700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb200e4fe0 con 0x7ffb20036bc0 2023-10-11T09:40:48.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7ffb2dc8c700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb200daf70 con 0x7ffb200e0c80 2023-10-11T09:40:48.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7ffb2dc8c700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb200e4a20 con 0x7ffb200e1780 2023-10-11T09:40:48.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f4fbffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fc01fc580 0x7f4fc00ce980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f4f4b7fe700 1 -- 172.21.15.78:0/2496708753 <== osd.2 v2:172.21.15.78:6808/35732 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4f500711c0 con 0x7f4f5006e700 2023-10-11T09:40:48.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f4fbeffd700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fc01fbf60 0x7f4fc01ff3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f4fbeffd700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fc01fbf60 0x7f4fc01ff3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:44112/0 (socket says 172.21.15.78:44112) 2023-10-11T09:40:48.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f4fbeffd700 1 -- 172.21.15.78:0/628420573 learned_addr learned my addr 172.21.15.78:0/628420573 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:48.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7ffb2cc8a700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffb200e1780 0x7ffb200e4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7ffb2d48b700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ffb20036bc0 0x7ffb20036fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:33796/0 (socket says 172.21.15.78:33796) 2023-10-11T09:40:48.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7ffb27fff700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ffb200e0c80 0x7ffb200512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7ffb2d48b700 1 -- 172.21.15.78:0/1522288976 learned_addr learned my addr 172.21.15.78:0/1522288976 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:48.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f4fbeffd700 1 -- 172.21.15.78:0/628420573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fc01fc580 msgr2=0x7f4fc00ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f4fbeffd700 1 --2- 172.21.15.78:0/628420573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fc01fc580 0x7f4fc00ce980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f4fbeffd700 1 -- 172.21.15.78:0/628420573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fa8003810 msgr2=0x7f4fc01ffb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f4fbeffd700 1 --2- 172.21.15.78:0/628420573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fa8003810 0x7f4fc01ffb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f4fbeffd700 1 -- 172.21.15.78:0/628420573 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc0200190 con 0x7f4fc01fbf60 2023-10-11T09:40:48.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f243cd0c700 1 -- 172.21.15.78:0/2955997057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24200050d0 msgr2=0x7f242000ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f243cd0c700 1 --2- 172.21.15.78:0/2955997057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24200050d0 0x7f242000ba30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f24380691c0 tx=0x7f24200058a0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f8a5f330700 1 Processor -- start 2023-10-11T09:40:48.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f8a5f330700 1 -- start start 2023-10-11T09:40:48.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f4fbeffd700 1 --2- 172.21.15.78:0/628420573 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fc01fbf60 0x7f4fc01ff3e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0005300 tx=0x7f4fb0001600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f243cd0c700 1 -- 172.21.15.78:0/2955997057 shutdown_connections 2023-10-11T09:40:48.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f243cd0c700 1 --2- 172.21.15.78:0/2955997057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24200050d0 0x7f242000ba30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f243cd0c700 1 -- 172.21.15.78:0/2955997057 >> 172.21.15.78:0/2955997057 conn(0x7f2430051630 msgr2=0x7f2430051a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:48.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f98b17fa700 1 -- 172.21.15.78:0/1543056024 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f98a40075e0 con 0x7f98b40b0db0 2023-10-11T09:40:48.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f98ba71f700 1 -- 172.21.15.78:0/1543056024 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f98b406cce0 con 0x7f98b40b0db0 2023-10-11T09:40:48.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f98b17fa700 1 --2- 172.21.15.78:0/1543056024 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9894065fa0 0x7f9894068450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f4fa7fff700 1 -- 172.21.15.78:0/628420573 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0023070 con 0x7f4fc01fbf60 2023-10-11T09:40:48.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f4fa7fff700 1 -- 172.21.15.78:0/628420573 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb0006840 con 0x7f4fc01fbf60 2023-10-11T09:40:48.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f4fa7fff700 1 -- 172.21.15.78:0/628420573 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0006bb0 con 0x7f4fc01fbf60 2023-10-11T09:40:48.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f4fc61eb700 1 -- 172.21.15.78:0/628420573 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc0200470 con 0x7f4fc01fbf60 2023-10-11T09:40:48.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f4fc61eb700 1 -- 172.21.15.78:0/628420573 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc0200a30 con 0x7f4fc01fbf60 2023-10-11T09:40:48.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7ffb277fe700 1 -- 172.21.15.78:0/1522288976 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3186680328 0 0) 0x7ffb200e4fe0 con 0x7ffb20036bc0 2023-10-11T09:40:48.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7ffb277fe700 1 -- 172.21.15.78:0/1522288976 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb180035d0 con 0x7ffb20036bc0 2023-10-11T09:40:48.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7ffb2cc8a700 1 -- 172.21.15.78:0/1522288976 >> v2:172.21.15.133:3300/0 conn(0x7ffb200e0c80 msgr2=0x7ffb200512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:48.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7ffb2cc8a700 1 --2- 172.21.15.78:0/1522288976 >> v2:172.21.15.133:3300/0 conn(0x7ffb200e0c80 0x7ffb200512e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7ffb2cc8a700 1 -- 172.21.15.78:0/1522288976 >> v1:172.21.15.133:6789/0 conn(0x7ffb20036bc0 legacy=0x7ffb20036fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7ffb2cc8a700 1 -- 172.21.15.78:0/1522288976 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb200e4d00 con 0x7ffb200e1780 2023-10-11T09:40:48.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.414+0000 7f243cd0c700 1 -- 172.21.15.78:0/2955997057 shutdown_connections 2023-10-11T09:40:48.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f243cd0c700 1 -- 172.21.15.78:0/2955997057 wait complete. 2023-10-11T09:40:48.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f8a5f330700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a5806d5f0 0x7f8a58145c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f8a5f330700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a58147b40 0x7f8a580af5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f8a5f330700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a58161090 0x7f8a580afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f8a5f330700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a580b4760 con 0x7f8a58161090 2023-10-11T09:40:48.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f8a5f330700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a580b3b50 con 0x7f8a5806d5f0 2023-10-11T09:40:48.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f8a5f330700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a580b3e30 con 0x7f8a58147b40 2023-10-11T09:40:48.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f8a5e32e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a5806d5f0 0x7f8a58145c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f8a5e32e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a5806d5f0 0x7f8a58145c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:49998/0 (socket says 172.21.15.78:49998) 2023-10-11T09:40:48.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f8a5e32e700 1 -- 172.21.15.78:0/2303273943 learned_addr learned my addr 172.21.15.78:0/2303273943 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:48.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f8b85990700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8b8015b8e0 0x7f8b8006d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f8b85990700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8b800ca950 0x7f8b8013fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f8b85990700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b8015e010 con 0x7f8b8006d610 2023-10-11T09:40:48.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f8b85990700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b8015e280 con 0x7f8b800ca950 2023-10-11T09:40:48.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f8b85990700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8015e560 con 0x7f8b8015b8e0 2023-10-11T09:40:48.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f8b8498e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8b8015b8e0 0x7f8b8006d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f8b8518f700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8b800ca950 0x7f8b8013fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f8b8518f700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8b800ca950 0x7f8b8013fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:49996/0 (socket says 172.21.15.78:49996) 2023-10-11T09:40:48.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f8b8518f700 1 -- 172.21.15.78:0/1278929966 learned_addr learned my addr 172.21.15.78:0/1278929966 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:48.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f8b8498e700 1 -- 172.21.15.78:0/1278929966 >> v2:172.21.15.133:3300/0 conn(0x7f8b800ca950 msgr2=0x7f8b8013fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:48.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f8b8498e700 1 --2- 172.21.15.78:0/1278929966 >> v2:172.21.15.133:3300/0 conn(0x7f8b800ca950 0x7f8b8013fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f8b8498e700 1 -- 172.21.15.78:0/1278929966 >> v1:172.21.15.78:6789/0 conn(0x7f8b8006d610 legacy=0x7f8b800ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:48.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f8b8498e700 1 -- 172.21.15.78:0/1278929966 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8015e840 con 0x7f8b8015b8e0 2023-10-11T09:40:48.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f8a5db2d700 1 --2- 172.21.15.78:0/2303273943 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a58147b40 0x7f8a580af5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f4fc61eb700 1 -- 172.21.15.78:0/628420573 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4fa0000fc0 con 0x7f4fc01fbf60 2023-10-11T09:40:48.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7ffb2cc8a700 1 --2- 172.21.15.78:0/1522288976 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffb200e1780 0x7ffb200e4120 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c009850 tx=0x7ffb1c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f29aedb1686e54cc server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f8a5e32e700 1 -- 172.21.15.78:0/2303273943 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a58147b40 msgr2=0x7f8a580af5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f8a5e32e700 1 --2- 172.21.15.78:0/2303273943 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a58147b40 0x7f8a580af5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f8a5e32e700 1 -- 172.21.15.78:0/2303273943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a58161090 msgr2=0x7f8a580afaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f8a5e32e700 1 --2- 172.21.15.78:0/2303273943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a58161090 0x7f8a580afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f8a5e32e700 1 -- 172.21.15.78:0/2303273943 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a50009a70 con 0x7f8a5806d5f0 2023-10-11T09:40:48.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f8b8498e700 1 --2- 172.21.15.78:0/1278929966 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8b8015b8e0 0x7f8b8006d0d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8b74009850 tx=0x7f8b74009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f1e6ae68f8083f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f98b17fa700 1 -- 172.21.15.78:0/1543056024 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f98a4082bc0 con 0x7f98b40b0db0 2023-10-11T09:40:48.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f98b17fa700 1 --2- 172.21.15.78:0/1543056024 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x7f989406e7d0 0x7f9894070be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f98b17fa700 1 -- 172.21.15.78:0/1543056024 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9894071290 con 0x7f989406e7d0 2023-10-11T09:40:48.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f98b3fff700 1 --2- 172.21.15.78:0/1543056024 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9894065fa0 0x7f9894068450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f98b8cbc700 1 --2- 172.21.15.78:0/1543056024 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x7f989406e7d0 0x7f9894070be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f8a5e32e700 1 --2- 172.21.15.78:0/2303273943 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a5806d5f0 0x7f8a58145c80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8a5000b370 tx=0x7f8a50004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7ffb277fe700 1 -- 172.21.15.78:0/1522288976 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c011830 con 0x7ffb200e1780 2023-10-11T09:40:48.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7ffb277fe700 1 -- 172.21.15.78:0/1522288976 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb1c011990 con 0x7ffb200e1780 2023-10-11T09:40:48.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7ffb277fe700 1 -- 172.21.15.78:0/1522288976 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c011d00 con 0x7ffb200e1780 2023-10-11T09:40:48.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f8b7f7fe700 1 -- 172.21.15.78:0/1278929966 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b74004230 con 0x7f8b8015b8e0 2023-10-11T09:40:48.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f8b7f7fe700 1 -- 172.21.15.78:0/1278929966 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b74004390 con 0x7f8b8015b8e0 2023-10-11T09:40:48.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f8b7f7fe700 1 -- 172.21.15.78:0/1278929966 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b740036c0 con 0x7f8b8015b8e0 2023-10-11T09:40:48.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f8a56ffd700 1 -- 172.21.15.78:0/2303273943 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a500048d0 con 0x7f8a5806d5f0 2023-10-11T09:40:48.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f8a56ffd700 1 -- 172.21.15.78:0/2303273943 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a5001e430 con 0x7f8a5806d5f0 2023-10-11T09:40:48.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f8a56ffd700 1 -- 172.21.15.78:0/2303273943 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5001e7e0 con 0x7f8a5806d5f0 2023-10-11T09:40:48.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.418+0000 7f8a5f330700 1 -- 172.21.15.78:0/2303273943 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a580b4110 con 0x7f8a5806d5f0 2023-10-11T09:40:48.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.422+0000 7f8a5f330700 1 -- 172.21.15.78:0/2303273943 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a581461c0 con 0x7f8a5806d5f0 2023-10-11T09:40:48.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.422+0000 7ffb2dc8c700 1 -- 172.21.15.78:0/1522288976 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffb200e1780 msgr2=0x7ffb200e4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.422+0000 7ffb2dc8c700 1 --2- 172.21.15.78:0/1522288976 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffb200e1780 0x7ffb200e4120 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c009850 tx=0x7ffb1c009b60 comp rx=0 tx=0).stop 2023-10-11T09:40:48.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.422+0000 7ffb2dc8c700 1 -- 172.21.15.78:0/1522288976 shutdown_connections 2023-10-11T09:40:48.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.422+0000 7ffb2dc8c700 1 --2- 172.21.15.78:0/1522288976 >> v2:172.21.15.133:3300/0 conn(0x7ffb200e0c80 0x7ffb200512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.422+0000 7ffb2dc8c700 1 --2- 172.21.15.78:0/1522288976 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffb200e1780 0x7ffb200e4120 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.422+0000 7ffb2dc8c700 1 -- 172.21.15.78:0/1522288976 >> 172.21.15.78:0/1522288976 conn(0x7ffb20051bc0 msgr2=0x7ffb20051fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:48.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.422+0000 7f8b85990700 1 -- 172.21.15.78:0/1278929966 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8b8015b8e0 msgr2=0x7f8b8006d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.422+0000 7f8b85990700 1 --2- 172.21.15.78:0/1278929966 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8b8015b8e0 0x7f8b8006d0d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8b74009850 tx=0x7f8b74009b60 comp rx=0 tx=0).stop 2023-10-11T09:40:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.422+0000 7ffb2dc8c700 1 -- 172.21.15.78:0/1522288976 shutdown_connections 2023-10-11T09:40:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.422+0000 7ffb2dc8c700 1 -- 172.21.15.78:0/1522288976 wait complete. 2023-10-11T09:40:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.422+0000 7f8b85990700 1 -- 172.21.15.78:0/1278929966 shutdown_connections 2023-10-11T09:40:48.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.422+0000 7f8b85990700 1 --2- 172.21.15.78:0/1278929966 >> v2:172.21.15.133:3300/0 conn(0x7f8b800ca950 0x7f8b8013fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.422+0000 7f8b85990700 1 --2- 172.21.15.78:0/1278929966 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8b8015b8e0 0x7f8b8006d0d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.422+0000 7f8b85990700 1 -- 172.21.15.78:0/1278929966 >> 172.21.15.78:0/1278929966 conn(0x7f8b800cea60 msgr2=0x7f8b800cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:48.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.422+0000 7f8b85990700 1 -- 172.21.15.78:0/1278929966 shutdown_connections 2023-10-11T09:40:48.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.422+0000 7f8b85990700 1 -- 172.21.15.78:0/1278929966 wait complete. 2023-10-11T09:40:48.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.422+0000 7f98b8cbc700 1 --2- 172.21.15.78:0/1543056024 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x7f989406e7d0 0x7f9894070be0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.422+0000 7f98b3fff700 1 --2- 172.21.15.78:0/1543056024 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9894065fa0 0x7f9894068450 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f989c00a970 tx=0x7f989c010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.422+0000 7f243cd0c700 1 Processor -- start 2023-10-11T09:40:48.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.422+0000 7f243cd0c700 1 -- start start 2023-10-11T09:40:48.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.422+0000 7f98b17fa700 1 -- 172.21.15.78:0/1543056024 <== osd.1 v2:172.21.15.78:6809/35734 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9894071290 con 0x7f989406e7d0 2023-10-11T09:40:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.422+0000 7f98b17fa700 1 -- 172.21.15.78:0/1543056024 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98a40502b0 con 0x7f98b40b0db0 2023-10-11T09:40:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.422+0000 7ffb2dc8c700 1 Processor -- start 2023-10-11T09:40:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.422+0000 7f4fa7fff700 1 -- 172.21.15.78:0/628420573 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f4fb0006d10 con 0x7f4fc01fbf60 2023-10-11T09:40:48.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.422+0000 7ffb2dc8c700 1 -- start start 2023-10-11T09:40:48.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.422+0000 7f8b85990700 1 Processor -- start 2023-10-11T09:40:48.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7f8a56ffd700 1 -- 172.21.15.78:0/2303273943 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f8a5001e940 con 0x7f8a5806d5f0 2023-10-11T09:40:48.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7f4fa7fff700 1 --2- 172.21.15.78:0/628420573 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4fac065fd0 0x7f4fac068480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7f4fa7fff700 1 -- 172.21.15.78:0/628420573 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4fb0084b70 con 0x7f4fc01fbf60 2023-10-11T09:40:48.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7f4fa7fff700 1 --2- 172.21.15.78:0/628420573 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x7f4fac06e880 0x7f4fac070c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7f4fa7fff700 1 -- 172.21.15.78:0/628420573 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4fac071340 con 0x7f4fac06e880 2023-10-11T09:40:48.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7f4fa7fff700 1 -- 172.21.15.78:0/628420573 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fb0084f20 con 0x7f4fc01fbf60 2023-10-11T09:40:48.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7ffb2dc8c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb20036bc0 0x7ffb20035c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7f8b85990700 1 -- start start 2023-10-11T09:40:48.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7f4fbf7fe700 1 --2- 172.21.15.78:0/628420573 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4fac065fd0 0x7f4fac068480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7f8b85990700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b8006d610 0x7f8b800b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7f4fbffff700 1 --2- 172.21.15.78:0/628420573 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x7f4fac06e880 0x7f4fac070c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.430+0000 7f4fbffff700 1 --2- 172.21.15.78:0/628420573 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x7f4fac06e880 0x7f4fac070c90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.430+0000 7f4fbf7fe700 1 --2- 172.21.15.78:0/628420573 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4fac065fd0 0x7f4fac068480 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4006df0 tx=0x7f4fb4006d60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7ffb2dc8c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffb200e0c80 0x7ffb200d3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7ffb2dc8c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb200e1780 0x7ffb200d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7ffb2dc8c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb20049920 con 0x7ffb20036bc0 2023-10-11T09:40:48.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7ffb2dc8c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb20048d10 con 0x7ffb200e1780 2023-10-11T09:40:48.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7ffb2dc8c700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb20048ff0 con 0x7ffb200e0c80 2023-10-11T09:40:48.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7ffb27fff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffb200e0c80 0x7ffb200d3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7ffb27fff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffb200e0c80 0x7ffb200d3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:44152/0 (socket says 172.21.15.78:44152) 2023-10-11T09:40:48.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7ffb27fff700 1 -- 172.21.15.78:0/3593205207 learned_addr learned my addr 172.21.15.78:0/3593205207 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:48.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7ffb2d48b700 1 --2- 172.21.15.78:0/3593205207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb200e1780 0x7ffb200d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7ffb2d48b700 1 -- 172.21.15.78:0/3593205207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffb200e0c80 msgr2=0x7ffb200d3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7ffb2d48b700 1 --2- 172.21.15.78:0/3593205207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffb200e0c80 0x7ffb200d3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7ffb2d48b700 1 -- 172.21.15.78:0/3593205207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb20036bc0 msgr2=0x7ffb20035c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7ffb2d48b700 1 --2- 172.21.15.78:0/3593205207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb20036bc0 0x7ffb20035c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7ffb2d48b700 1 -- 172.21.15.78:0/3593205207 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb1c0094b0 con 0x7ffb200e1780 2023-10-11T09:40:48.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7f243cd0c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24200050d0 0x7f24300e9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7f243cd0c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f24300df130 0x7f24300dc050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7f243cd0c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24300e4ab0 0x7f24300e4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7f243cd0c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24300dff40 con 0x7f24200050d0 2023-10-11T09:40:48.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7f243cd0c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24300e5460 con 0x7f24300e4ab0 2023-10-11T09:40:48.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7f243cd0c700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24300e7570 con 0x7f24300df130 2023-10-11T09:40:48.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7f2436d9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24300e4ab0 0x7f24300e4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.430+0000 7f2436d9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24300e4ab0 0x7f24300e4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:50014/0 (socket says 172.21.15.78:50014) 2023-10-11T09:40:48.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.430+0000 7f2436d9d700 1 -- 172.21.15.78:0/275786440 learned_addr learned my addr 172.21.15.78:0/275786440 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:48.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7f2435d9b700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f24300df130 0x7f24300dc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.430+0000 7f2436d9d700 1 -- 172.21.15.78:0/275786440 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f24300df130 msgr2=0x7f24300dc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.430+0000 7f2436d9d700 1 --2- 172.21.15.78:0/275786440 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f24300df130 0x7f24300dc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.430+0000 7f2436d9d700 1 -- 172.21.15.78:0/275786440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24200050d0 msgr2=0x7f24300e9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.430+0000 7f2436d9d700 1 --2- 172.21.15.78:0/275786440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24200050d0 0x7f24300e9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.430+0000 7f2436d9d700 1 -- 172.21.15.78:0/275786440 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24300df6a0 con 0x7f24300e4ab0 2023-10-11T09:40:48.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7f8a56ffd700 1 --2- 172.21.15.78:0/2303273943 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8a44065ed0 0x7f8a44068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7f8a56ffd700 1 -- 172.21.15.78:0/2303273943 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8a50085cd0 con 0x7f8a5806d5f0 2023-10-11T09:40:48.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7f8a5db2d700 1 --2- 172.21.15.78:0/2303273943 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8a44065ed0 0x7f8a44068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.430+0000 7f8a5f330700 1 --2- 172.21.15.78:0/2303273943 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x7f8a580b2e60 0x7f8a580b3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.430+0000 7f8a5f330700 1 -- 172.21.15.78:0/2303273943 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8a58161800 con 0x7f8a580b2e60 2023-10-11T09:40:48.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7f8b85990700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8b800ca950 0x7f8b8013f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7f8b85990700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b8015b8e0 0x7f8b80159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7f8b85990700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8013fb80 con 0x7f8b8015b8e0 2023-10-11T09:40:48.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7f8b85990700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b800b3170 con 0x7f8b8006d610 2023-10-11T09:40:48.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7f8b85990700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b800b3450 con 0x7f8b800ca950 2023-10-11T09:40:48.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7f8b7ffff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8b800ca950 0x7f8b8013f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7f8b7ffff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8b800ca950 0x7f8b8013f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:44166/0 (socket says 172.21.15.78:44166) 2023-10-11T09:40:48.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7f8b7ffff700 1 -- 172.21.15.78:0/528895759 learned_addr learned my addr 172.21.15.78:0/528895759 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:48.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7f8b8498e700 1 --2- 172.21.15.78:0/528895759 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b8006d610 0x7f8b800b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7f8b7ffff700 1 -- 172.21.15.78:0/528895759 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b8006d610 msgr2=0x7f8b800b2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7f8b7ffff700 1 --2- 172.21.15.78:0/528895759 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b8006d610 0x7f8b800b2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7f8b7ffff700 1 -- 172.21.15.78:0/528895759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b8015b8e0 msgr2=0x7f8b80159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7f8b7ffff700 1 --2- 172.21.15.78:0/528895759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b8015b8e0 0x7f8b80159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.426+0000 7f8b7ffff700 1 -- 172.21.15.78:0/528895759 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b740094b0 con 0x7f8b800ca950 2023-10-11T09:40:48.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.430+0000 7f8b7ffff700 1 --2- 172.21.15.78:0/528895759 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8b800ca950 0x7f8b8013f640 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c00c500 tx=0x7f8b6c00c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.430+0000 7f2436d9d700 1 --2- 172.21.15.78:0/275786440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24300e4ab0 0x7f24300e4f20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2438069ad0 tx=0x7f2438069f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.430+0000 7f2426ffd700 1 -- 172.21.15.78:0/275786440 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2438098040 con 0x7f24300e4ab0 2023-10-11T09:40:48.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.430+0000 7f2426ffd700 1 -- 172.21.15.78:0/275786440 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243806d260 con 0x7f24300e4ab0 2023-10-11T09:40:48.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.430+0000 7f2426ffd700 1 -- 172.21.15.78:0/275786440 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2438067510 con 0x7f24300e4ab0 2023-10-11T09:40:48.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.430+0000 7f243cd0c700 1 -- 172.21.15.78:0/275786440 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24300df980 con 0x7f24300e4ab0 2023-10-11T09:40:48.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.430+0000 7f243cd0c700 1 -- 172.21.15.78:0/275786440 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24300e9fd0 con 0x7f24300e4ab0 2023-10-11T09:40:48.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.430+0000 7f8a5eb2f700 1 --2- 172.21.15.78:0/2303273943 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x7f8a580b2e60 0x7f8a580b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.430+0000 7ffb2d48b700 1 --2- 172.21.15.78:0/3593205207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb200e1780 0x7ffb200d35c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ffb28061d20 tx=0x7ffb28068230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.430+0000 7f8a5db2d700 1 --2- 172.21.15.78:0/2303273943 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8a44065ed0 0x7f8a44068380 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8a581f9780 tx=0x7f8a4c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.430+0000 7f8b7d7fa700 1 -- 172.21.15.78:0/528895759 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c0054e0 con 0x7f8b800ca950 2023-10-11T09:40:48.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.430+0000 7f8a5eb2f700 1 --2- 172.21.15.78:0/2303273943 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x7f8a580b2e60 0x7f8a580b3230 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.430+0000 7ffb257fa700 1 -- 172.21.15.78:0/3593205207 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2806d500 con 0x7ffb200e1780 2023-10-11T09:40:48.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.430+0000 7f8b85990700 1 -- 172.21.15.78:0/528895759 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b800b3730 con 0x7f8b800ca950 2023-10-11T09:40:48.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.430+0000 7f8b85990700 1 -- 172.21.15.78:0/528895759 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b800b3cf0 con 0x7f8b800ca950 2023-10-11T09:40:48.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.430+0000 7f243cd0c700 1 -- 172.21.15.78:0/275786440 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2430033f00 con 0x7f24300e4ab0 2023-10-11T09:40:48.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.430+0000 7f4fa7fff700 1 -- 172.21.15.78:0/628420573 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4fac071340 con 0x7f4fac06e880 2023-10-11T09:40:48.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.430+0000 7f8b7d7fa700 1 -- 172.21.15.78:0/528895759 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6c0036a0 con 0x7f8b800ca950 2023-10-11T09:40:48.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.430+0000 7ffb2dc8c700 1 -- 172.21.15.78:0/3593205207 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb200492d0 con 0x7ffb200e1780 2023-10-11T09:40:48.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.430+0000 7ffb2dc8c700 1 -- 172.21.15.78:0/3593205207 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb20036190 con 0x7ffb200e1780 2023-10-11T09:40:48.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.430+0000 7ffb257fa700 1 -- 172.21.15.78:0/3593205207 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb2806d660 con 0x7ffb200e1780 2023-10-11T09:40:48.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.430+0000 7ffb257fa700 1 -- 172.21.15.78:0/3593205207 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2806da10 con 0x7ffb200e1780 2023-10-11T09:40:48.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.430+0000 7ffb2dc8c700 1 -- 172.21.15.78:0/3593205207 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffb20050dc0 con 0x7ffb200e1780 2023-10-11T09:40:48.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.434+0000 7f8b85990700 1 -- 172.21.15.78:0/528895759 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8b60000fc0 con 0x7f8b800ca950 2023-10-11T09:40:48.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.434+0000 7f8a56ffd700 1 -- 172.21.15.78:0/2303273943 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f8a480019c0 con 0x7f8a580b2e60 2023-10-11T09:40:48.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.434+0000 7f8b7d7fa700 1 -- 172.21.15.78:0/528895759 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c003a50 con 0x7f8b800ca950 2023-10-11T09:40:48.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.434+0000 7f8b7d7fa700 1 -- 172.21.15.78:0/528895759 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f8b6c003c30 con 0x7f8b800ca950 2023-10-11T09:40:48.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.434+0000 7f224ea82700 1 -- 172.21.15.78:0/2823643740 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f224806d480 con 0x7f223406e850 2023-10-11T09:40:48.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.434+0000 7f22457fa700 1 -- 172.21.15.78:0/2823643740 <== osd.0 v2:172.21.15.78:6800/35735 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f223c006ca0 con 0x7f223406e850 2023-10-11T09:40:48.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.434+0000 7f8b7d7fa700 1 --2- 172.21.15.78:0/528895759 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8b64065f50 0x7f8b64068400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.434+0000 7f8b7d7fa700 1 -- 172.21.15.78:0/528895759 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8b6c083370 con 0x7f8b800ca950 2023-10-11T09:40:48.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.434+0000 7f8b7d7fa700 1 --2- 172.21.15.78:0/528895759 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x7f8b6406e690 0x7f8b64070aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.434+0000 7f8b7d7fa700 1 -- 172.21.15.78:0/528895759 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8b64071150 con 0x7f8b6406e690 2023-10-11T09:40:48.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.434+0000 7f8b7d7fa700 1 -- 172.21.15.78:0/528895759 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b6c0836e0 con 0x7f8b800ca950 2023-10-11T09:40:48.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.434+0000 7f8b8498e700 1 --2- 172.21.15.78:0/528895759 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8b64065f50 0x7f8b64068400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.438+0000 7f224ea82700 1 -- 172.21.15.78:0/2823643740 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x7f223406e850 msgr2=0x7f2234070c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.438+0000 7f224ea82700 1 --2- 172.21.15.78:0/2823643740 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x7f223406e850 0x7f2234070c60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.438+0000 7f224ea82700 1 -- 172.21.15.78:0/2823643740 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2234065fa0 msgr2=0x7f2234068450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.438+0000 7f224ea82700 1 --2- 172.21.15.78:0/2823643740 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2234065fa0 0x7f2234068450 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2240002230 tx=0x7f2240001e10 comp rx=0 tx=0).stop 2023-10-11T09:40:48.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.434+0000 7ffb257fa700 1 -- 172.21.15.78:0/3593205207 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7ffb28094440 con 0x7ffb200e1780 2023-10-11T09:40:48.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.438+0000 7ffb257fa700 1 --2- 172.21.15.78:0/3593205207 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffb0c065fd0 0x7ffb0c068480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.438+0000 7ffb257fa700 1 -- 172.21.15.78:0/3593205207 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffb280fd610 con 0x7ffb200e1780 2023-10-11T09:40:48.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.438+0000 7ffb257fa700 1 --2- 172.21.15.78:0/3593205207 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x7ffb0c06e880 0x7ffb0c070c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.438+0000 7f224ea82700 1 -- 172.21.15.78:0/2823643740 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f224815c6b0 msgr2=0x7f22480b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.438+0000 7ffb2cc8a700 1 --2- 172.21.15.78:0/3593205207 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffb0c065fd0 0x7ffb0c068480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.438+0000 7f224ea82700 1 --2- 172.21.15.78:0/2823643740 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f224815c6b0 0x7f22480b1a10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f223800e7e0 tx=0x7f223800eaf0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.438+0000 7f224d01f700 1 -- 172.21.15.78:0/2823643740 reap_dead start 2023-10-11T09:40:48.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.438+0000 7f224ea82700 1 -- 172.21.15.78:0/2823643740 shutdown_connections 2023-10-11T09:40:48.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.438+0000 7ffb257fa700 1 -- 172.21.15.78:0/3593205207 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffb0c071340 con 0x7ffb0c06e880 2023-10-11T09:40:48.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.438+0000 7ffb257fa700 1 -- 172.21.15.78:0/3593205207 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb280fd9c0 con 0x7ffb200e1780 2023-10-11T09:40:48.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.438+0000 7f224ea82700 1 -- 172.21.15.78:0/2823643740 >> 172.21.15.78:0/2823643740 conn(0x7f224806aca0 msgr2=0x7f22480c9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:48.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.438+0000 7f224ea82700 1 -- 172.21.15.78:0/2823643740 shutdown_connections 2023-10-11T09:40:48.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.438+0000 7f224ea82700 1 -- 172.21.15.78:0/2823643740 wait complete. 2023-10-11T09:40:48.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.438+0000 7ffb27fff700 1 --2- 172.21.15.78:0/3593205207 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x7ffb0c06e880 0x7ffb0c070c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.438+0000 7f8b8518f700 1 --2- 172.21.15.78:0/528895759 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x7f8b6406e690 0x7f8b64070aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.438+0000 7f8b8498e700 1 --2- 172.21.15.78:0/528895759 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8b64065f50 0x7f8b64068400 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8b74009850 tx=0x7f8b74008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.438+0000 7f8b8518f700 1 --2- 172.21.15.78:0/528895759 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x7f8b6406e690 0x7f8b64070aa0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.438+0000 7ffb2cc8a700 1 --2- 172.21.15.78:0/3593205207 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffb0c065fd0 0x7ffb0c068480 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c009ba0 tx=0x7ffb1c010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.442+0000 7ffb27fff700 1 --2- 172.21.15.78:0/3593205207 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x7ffb0c06e880 0x7ffb0c070c90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.442+0000 7f8b7d7fa700 1 -- 172.21.15.78:0/528895759 <== osd.7 v2:172.21.15.133:6824/35901 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f8b64071150 con 0x7f8b6406e690 2023-10-11T09:40:48.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.442+0000 7ffb257fa700 1 -- 172.21.15.78:0/3593205207 <== osd.4 v2:172.21.15.133:6808/35898 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7ffb0c071340 con 0x7ffb0c06e880 2023-10-11T09:40:48.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.442+0000 7f4f69237700 1 -- 172.21.15.78:0/2496708753 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4f6406d480 con 0x7f4f5006e700 2023-10-11T09:40:48.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.442+0000 7f2426ffd700 1 -- 172.21.15.78:0/275786440 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f2438091070 con 0x7f24300e4ab0 2023-10-11T09:40:48.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.442+0000 7f2426ffd700 1 --2- 172.21.15.78:0/275786440 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2418065f00 0x7f24180683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.442+0000 7f2426ffd700 1 -- 172.21.15.78:0/275786440 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f24380fe430 con 0x7f24300e4ab0 2023-10-11T09:40:48.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.442+0000 7f2426ffd700 1 --2- 172.21.15.78:0/275786440 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x7f241806e730 0x7f2418070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.442+0000 7f2426ffd700 1 -- 172.21.15.78:0/275786440 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f24180711f0 con 0x7f241806e730 2023-10-11T09:40:48.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.442+0000 7f2426ffd700 1 -- 172.21.15.78:0/275786440 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24380fe7a0 con 0x7f24300e4ab0 2023-10-11T09:40:48.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.442+0000 7f4f4b7fe700 1 -- 172.21.15.78:0/2496708753 <== osd.2 v2:172.21.15.78:6808/35732 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4f58001480 con 0x7f4f5006e700 2023-10-11T09:40:48.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.442+0000 7f243659c700 1 --2- 172.21.15.78:0/275786440 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2418065f00 0x7f24180683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.442+0000 7f2435d9b700 1 --2- 172.21.15.78:0/275786440 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x7f241806e730 0x7f2418070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.442+0000 7f4f497fa700 1 -- 172.21.15.78:0/2496708753 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x7f4f5006e700 msgr2=0x7f4f50070b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.446+0000 7f4f497fa700 1 --2- 172.21.15.78:0/2496708753 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x7f4f5006e700 0x7f4f50070b10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.446+0000 7f4f497fa700 1 -- 172.21.15.78:0/2496708753 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4f50065ed0 msgr2=0x7f4f50068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.446+0000 7f4f497fa700 1 --2- 172.21.15.78:0/2496708753 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4f50065ed0 0x7f4f50068380 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4f6414d150 tx=0x7f4f5c011000 comp rx=0 tx=0).stop 2023-10-11T09:40:48.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.446+0000 7f4f497fa700 1 -- 172.21.15.78:0/2496708753 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4f640c9020 msgr2=0x7f4f6414e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.446+0000 7f4f497fa700 1 --2- 172.21.15.78:0/2496708753 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4f640c9020 0x7f4f6414e180 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4f5400c410 tx=0x7f4f5400c720 comp rx=0 tx=0).stop 2023-10-11T09:40:48.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.446+0000 7f4f6359e700 1 -- 172.21.15.78:0/2496708753 reap_dead start 2023-10-11T09:40:48.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.446+0000 7f4f497fa700 1 -- 172.21.15.78:0/2496708753 shutdown_connections 2023-10-11T09:40:48.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.446+0000 7f4f497fa700 1 -- 172.21.15.78:0/2496708753 >> 172.21.15.78:0/2496708753 conn(0x7f4f6406aca0 msgr2=0x7f4f6406fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:48.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.446+0000 7f2435d9b700 1 --2- 172.21.15.78:0/275786440 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x7f241806e730 0x7f2418070b40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.446+0000 7f243659c700 1 --2- 172.21.15.78:0/275786440 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2418065f00 0x7f24180683b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2428000fd0 tx=0x7f24280015f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.446+0000 7f4f497fa700 1 -- 172.21.15.78:0/2496708753 shutdown_connections 2023-10-11T09:40:48.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.446+0000 7f4f497fa700 1 -- 172.21.15.78:0/2496708753 wait complete. 2023-10-11T09:40:48.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.446+0000 7f2426ffd700 1 -- 172.21.15.78:0/275786440 <== osd.5 v2:172.21.15.133:6816/35899 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f24180711f0 con 0x7f241806e730 2023-10-11T09:40:48.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.446+0000 7f98ba71f700 1 -- 172.21.15.78:0/1543056024 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f98b406c130 con 0x7f989406e7d0 2023-10-11T09:40:48.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.446+0000 7f98b17fa700 1 -- 172.21.15.78:0/1543056024 <== osd.1 v2:172.21.15.78:6809/35734 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f98ac005040 con 0x7f989406e7d0 2023-10-11T09:40:48.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.458+0000 7f98aa7fc700 1 -- 172.21.15.78:0/1543056024 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x7f989406e7d0 msgr2=0x7f9894070be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.458+0000 7f98aa7fc700 1 --2- 172.21.15.78:0/1543056024 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x7f989406e7d0 0x7f9894070be0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.458+0000 7f98aa7fc700 1 -- 172.21.15.78:0/1543056024 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9894065fa0 msgr2=0x7f9894068450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.458+0000 7f98aa7fc700 1 --2- 172.21.15.78:0/1543056024 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9894065fa0 0x7f9894068450 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f989c00a970 tx=0x7f989c010000 comp rx=0 tx=0).stop 2023-10-11T09:40:48.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.458+0000 7f98aa7fc700 1 -- 172.21.15.78:0/1543056024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98b40b0db0 msgr2=0x7f98b4155630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.458+0000 7f98aa7fc700 1 --2- 172.21.15.78:0/1543056024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98b40b0db0 0x7f98b4155630 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f98a400c410 tx=0x7f98a400c720 comp rx=0 tx=0).stop 2023-10-11T09:40:48.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.458+0000 7f98b8cbc700 1 -- 172.21.15.78:0/1543056024 reap_dead start 2023-10-11T09:40:48.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.458+0000 7f98aa7fc700 1 -- 172.21.15.78:0/1543056024 shutdown_connections 2023-10-11T09:40:48.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.458+0000 7f98aa7fc700 1 -- 172.21.15.78:0/1543056024 >> 172.21.15.78:0/1543056024 conn(0x7f98b406aca0 msgr2=0x7f98b40b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:48.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.458+0000 7f98aa7fc700 1 -- 172.21.15.78:0/1543056024 shutdown_connections 2023-10-11T09:40:48.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.458+0000 7f98aa7fc700 1 -- 172.21.15.78:0/1543056024 wait complete. 2023-10-11T09:40:48.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.458+0000 7f4fc61eb700 1 -- 172.21.15.78:0/628420573 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4fa0002d90 con 0x7f4fac06e880 2023-10-11T09:40:48.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.458+0000 7f4fa7fff700 1 -- 172.21.15.78:0/628420573 <== osd.3 v2:172.21.15.78:6824/35738 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4fa0002d90 con 0x7f4fac06e880 2023-10-11T09:40:48.554 INFO:teuthology.orchestra.run.smithi078.stdout:55834574854 2023-10-11T09:40:48.554 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-11T09:40:48.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.462+0000 7f4fa5ffb700 1 -- 172.21.15.78:0/628420573 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x7f4fac06e880 msgr2=0x7f4fac070c90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.462+0000 7f4fa5ffb700 1 --2- 172.21.15.78:0/628420573 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x7f4fac06e880 0x7f4fac070c90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.462+0000 7f4fa5ffb700 1 -- 172.21.15.78:0/628420573 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4fac065fd0 msgr2=0x7f4fac068480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.462+0000 7f4fa5ffb700 1 --2- 172.21.15.78:0/628420573 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4fac065fd0 0x7f4fac068480 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4006df0 tx=0x7f4fb4006d60 comp rx=0 tx=0).stop 2023-10-11T09:40:48.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.462+0000 7f4fa5ffb700 1 -- 172.21.15.78:0/628420573 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fc01fbf60 msgr2=0x7f4fc01ff3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.462+0000 7f4fa5ffb700 1 --2- 172.21.15.78:0/628420573 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fc01fbf60 0x7f4fc01ff3e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0005300 tx=0x7f4fb0001600 comp rx=0 tx=0).stop 2023-10-11T09:40:48.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.462+0000 7f4fbffff700 1 -- 172.21.15.78:0/628420573 reap_dead start 2023-10-11T09:40:48.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.462+0000 7f4fa5ffb700 1 -- 172.21.15.78:0/628420573 shutdown_connections 2023-10-11T09:40:48.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.462+0000 7f4fa5ffb700 1 -- 172.21.15.78:0/628420573 >> 172.21.15.78:0/628420573 conn(0x7f4fc006aca0 msgr2=0x7f4fc0146380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:48.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.466+0000 7f4fa5ffb700 1 -- 172.21.15.78:0/628420573 shutdown_connections 2023-10-11T09:40:48.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.466+0000 7f4fa5ffb700 1 -- 172.21.15.78:0/628420573 wait complete. 2023-10-11T09:40:48.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.466+0000 7f8a5f330700 1 -- 172.21.15.78:0/2303273943 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8a58161dc0 con 0x7f8a580b2e60 2023-10-11T09:40:48.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.466+0000 7f8a56ffd700 1 -- 172.21.15.78:0/2303273943 <== osd.6 v2:172.21.15.133:6800/35900 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8a48007ea0 con 0x7f8a580b2e60 2023-10-11T09:40:48.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.466+0000 7f8b85990700 1 -- 172.21.15.78:0/528895759 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8b60002d90 con 0x7f8b6406e690 2023-10-11T09:40:48.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.466+0000 7f8a5f330700 1 -- 172.21.15.78:0/2303273943 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x7f8a580b2e60 msgr2=0x7f8a580b3230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.466+0000 7f8a5f330700 1 --2- 172.21.15.78:0/2303273943 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x7f8a580b2e60 0x7f8a580b3230 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.466+0000 7f243cd0c700 1 -- 172.21.15.78:0/275786440 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2430033c20 con 0x7f241806e730 2023-10-11T09:40:48.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.466+0000 7f8a5f330700 1 -- 172.21.15.78:0/2303273943 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8a44065ed0 msgr2=0x7f8a44068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.466+0000 7f8a5f330700 1 --2- 172.21.15.78:0/2303273943 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8a44065ed0 0x7f8a44068380 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8a581f9780 tx=0x7f8a4c008040 comp rx=0 tx=0).stop 2023-10-11T09:40:48.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.466+0000 7f8a5f330700 1 -- 172.21.15.78:0/2303273943 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a5806d5f0 msgr2=0x7f8a58145c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.466+0000 7f8a5f330700 1 --2- 172.21.15.78:0/2303273943 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a5806d5f0 0x7f8a58145c80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8a5000b370 tx=0x7f8a50004af0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.466+0000 7f8a5eb2f700 1 -- 172.21.15.78:0/2303273943 reap_dead start 2023-10-11T09:40:48.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7f8a5f330700 1 -- 172.21.15.78:0/2303273943 shutdown_connections 2023-10-11T09:40:48.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7f8a5f330700 1 -- 172.21.15.78:0/2303273943 >> 172.21.15.78:0/2303273943 conn(0x7f8a5806aca0 msgr2=0x7f8a580ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:48.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7f8a5f330700 1 -- 172.21.15.78:0/2303273943 shutdown_connections 2023-10-11T09:40:48.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7f8a5f330700 1 -- 172.21.15.78:0/2303273943 wait complete. 2023-10-11T09:40:48.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7ffb2dc8c700 1 -- 172.21.15.78:0/3593205207 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffb20050ae0 con 0x7ffb0c06e880 2023-10-11T09:40:48.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7f8b7d7fa700 1 -- 172.21.15.78:0/528895759 <== osd.7 v2:172.21.15.133:6824/35901 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8b60002d90 con 0x7f8b6406e690 2023-10-11T09:40:48.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7f2426ffd700 1 -- 172.21.15.78:0/275786440 <== osd.5 v2:172.21.15.133:6816/35899 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f242c007a30 con 0x7f241806e730 2023-10-11T09:40:48.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7f8b85990700 1 -- 172.21.15.78:0/528895759 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x7f8b6406e690 msgr2=0x7f8b64070aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7f8b85990700 1 --2- 172.21.15.78:0/528895759 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x7f8b6406e690 0x7f8b64070aa0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7ffb257fa700 1 -- 172.21.15.78:0/3593205207 <== osd.4 v2:172.21.15.133:6808/35898 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ffb14009b40 con 0x7ffb0c06e880 2023-10-11T09:40:48.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7f8b85990700 1 -- 172.21.15.78:0/528895759 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8b64065f50 msgr2=0x7f8b64068400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7f8b85990700 1 --2- 172.21.15.78:0/528895759 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8b64065f50 0x7f8b64068400 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8b74009850 tx=0x7f8b74008030 comp rx=0 tx=0).stop 2023-10-11T09:40:48.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7f8b85990700 1 -- 172.21.15.78:0/528895759 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8b800ca950 msgr2=0x7f8b8013f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7f8b85990700 1 --2- 172.21.15.78:0/528895759 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8b800ca950 0x7f8b8013f640 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c00c500 tx=0x7f8b6c00c810 comp rx=0 tx=0).stop 2023-10-11T09:40:48.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7f8b8518f700 1 -- 172.21.15.78:0/528895759 reap_dead start 2023-10-11T09:40:48.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7f8b85990700 1 -- 172.21.15.78:0/528895759 shutdown_connections 2023-10-11T09:40:48.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7f8b85990700 1 -- 172.21.15.78:0/528895759 >> 172.21.15.78:0/528895759 conn(0x7f8b800cea60 msgr2=0x7f8b800cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:48.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7f8b85990700 1 -- 172.21.15.78:0/528895759 shutdown_connections 2023-10-11T09:40:48.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7f8b85990700 1 -- 172.21.15.78:0/528895759 wait complete. 2023-10-11T09:40:48.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7f243cd0c700 1 -- 172.21.15.78:0/275786440 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x7f241806e730 msgr2=0x7f2418070b40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7f243cd0c700 1 --2- 172.21.15.78:0/275786440 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x7f241806e730 0x7f2418070b40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.568 INFO:teuthology.orchestra.run.smithi078.stdout:55834574856 2023-10-11T09:40:48.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7f243cd0c700 1 -- 172.21.15.78:0/275786440 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2418065f00 msgr2=0x7f24180683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7f243cd0c700 1 --2- 172.21.15.78:0/275786440 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2418065f00 0x7f24180683b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2428000fd0 tx=0x7f24280015f0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7f243cd0c700 1 -- 172.21.15.78:0/275786440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24300e4ab0 msgr2=0x7f24300e4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7f243cd0c700 1 --2- 172.21.15.78:0/275786440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24300e4ab0 0x7f24300e4f20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2438069ad0 tx=0x7f2438069f90 comp rx=0 tx=0).stop 2023-10-11T09:40:48.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7f2436d9d700 1 -- 172.21.15.78:0/275786440 reap_dead start 2023-10-11T09:40:48.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7f243cd0c700 1 -- 172.21.15.78:0/275786440 shutdown_connections 2023-10-11T09:40:48.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7f243cd0c700 1 -- 172.21.15.78:0/275786440 >> 172.21.15.78:0/275786440 conn(0x7f2430051630 msgr2=0x7f2430036430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:48.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7f243cd0c700 1 -- 172.21.15.78:0/275786440 shutdown_connections 2023-10-11T09:40:48.571 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-11T09:40:48.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7f243cd0c700 1 -- 172.21.15.78:0/275786440 wait complete. 2023-10-11T09:40:48.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7ffb2dc8c700 1 -- 172.21.15.78:0/3593205207 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x7ffb0c06e880 msgr2=0x7ffb0c070c90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7ffb2dc8c700 1 --2- 172.21.15.78:0/3593205207 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x7ffb0c06e880 0x7ffb0c070c90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7ffb2dc8c700 1 -- 172.21.15.78:0/3593205207 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffb0c065fd0 msgr2=0x7ffb0c068480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7ffb2dc8c700 1 --2- 172.21.15.78:0/3593205207 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffb0c065fd0 0x7ffb0c068480 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c009ba0 tx=0x7ffb1c010000 comp rx=0 tx=0).stop 2023-10-11T09:40:48.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7ffb2dc8c700 1 -- 172.21.15.78:0/3593205207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb200e1780 msgr2=0x7ffb200d35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7ffb2dc8c700 1 --2- 172.21.15.78:0/3593205207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb200e1780 0x7ffb200d35c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ffb28061d20 tx=0x7ffb28068230 comp rx=0 tx=0).stop 2023-10-11T09:40:48.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7ffb2d48b700 1 -- 172.21.15.78:0/3593205207 reap_dead start 2023-10-11T09:40:48.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7ffb2dc8c700 1 -- 172.21.15.78:0/3593205207 shutdown_connections 2023-10-11T09:40:48.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.470+0000 7ffb2dc8c700 1 -- 172.21.15.78:0/3593205207 >> 172.21.15.78:0/3593205207 conn(0x7ffb20051bc0 msgr2=0x7ffb20054410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:48.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.474+0000 7ffb2dc8c700 1 -- 172.21.15.78:0/3593205207 shutdown_connections 2023-10-11T09:40:48.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.474+0000 7ffb2dc8c700 1 -- 172.21.15.78:0/3593205207 wait complete. 2023-10-11T09:40:48.575 INFO:teuthology.orchestra.run.smithi078.stdout:55834574855 2023-10-11T09:40:48.575 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-11T09:40:48.576 INFO:teuthology.orchestra.run.smithi078.stdout:55834574854 2023-10-11T09:40:48.577 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-11T09:40:48.578 INFO:teuthology.orchestra.run.smithi078.stdout:55834574855 2023-10-11T09:40:48.578 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-11T09:40:48.579 INFO:teuthology.orchestra.run.smithi078.stdout:55834574855 2023-10-11T09:40:48.580 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-11T09:40:48.581 INFO:teuthology.orchestra.run.smithi078.stdout:55834574856 2023-10-11T09:40:48.581 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-11T09:40:48.582 INFO:teuthology.orchestra.run.smithi078.stdout:55834574855 2023-10-11T09:40:48.582 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-11T09:40:48.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.830+0000 7f364cb1f700 1 Processor -- start 2023-10-11T09:40:48.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.830+0000 7f364cb1f700 1 -- start start 2023-10-11T09:40:48.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.834+0000 7f498371f700 1 Processor -- start 2023-10-11T09:40:48.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.834+0000 7fdec54bf700 1 Processor -- start 2023-10-11T09:40:48.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.834+0000 7fdec54bf700 1 -- start start 2023-10-11T09:40:48.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.834+0000 7f498371f700 1 -- start start 2023-10-11T09:40:48.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.834+0000 7f364cb1f700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f364815a990 0x7f36480b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.834+0000 7f364cb1f700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f36480b0be0 0x7f364813fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.834+0000 7f364cb1f700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36480b1040 con 0x7f364815b7f0 2023-10-11T09:40:48.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.834+0000 7f364cb1f700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f364815bbc0 con 0x7f364815a990 2023-10-11T09:40:48.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.834+0000 7f364cb1f700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f364815e050 con 0x7f36480b0be0 2023-10-11T09:40:48.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.834+0000 7f3645d9b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f364815a990 0x7f36480b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.834+0000 7f3645d9b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f364815a990 0x7f36480b06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56960/0 (socket says 172.21.15.78:56960) 2023-10-11T09:40:48.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.834+0000 7f3645d9b700 1 -- 172.21.15.78:0/2039487900 learned_addr learned my addr 172.21.15.78:0/2039487900 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:48.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.834+0000 7f3646d9d700 1 --2- 172.21.15.78:0/2039487900 >> v2:172.21.15.133:3300/0 conn(0x7f36480b0be0 0x7f364813fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.834+0000 7fdec54bf700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdec00c6f30 0x7fdec00b2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.834+0000 7f498371f700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f497c0cda10 0x7f497c0caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.834+0000 7fdec54bf700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fdec00b2590 0x7fdec00ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.834+0000 7f498371f700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f497c164d20 0x7f497c06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.834+0000 7fdec54bf700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdec00c7300 con 0x7fdec00ce560 2023-10-11T09:40:48.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.834+0000 7fdec54bf700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdec00b29e0 con 0x7fdec00b2590 2023-10-11T09:40:48.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.834+0000 7fdec54bf700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec0159dc0 con 0x7fdec00c6f30 2023-10-11T09:40:48.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.834+0000 7f3645d9b700 1 -- 172.21.15.78:0/2039487900 >> v2:172.21.15.133:3300/0 conn(0x7f36480b0be0 msgr2=0x7f364813fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:48.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.834+0000 7f3645d9b700 1 --2- 172.21.15.78:0/2039487900 >> v2:172.21.15.133:3300/0 conn(0x7f36480b0be0 0x7f364813fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.834+0000 7f3645d9b700 1 -- 172.21.15.78:0/2039487900 >> v1:172.21.15.133:6789/0 conn(0x7f364815b7f0 legacy=0x7f364806d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.834+0000 7f498371f700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f497c165390 con 0x7f497c0cdde0 2023-10-11T09:40:48.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.834+0000 7f498371f700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f497c1651a0 con 0x7f497c164d20 2023-10-11T09:40:48.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.834+0000 7f498371f700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f497c0ca440 con 0x7f497c0cda10 2023-10-11T09:40:48.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.834+0000 7f364559a700 1 -- 172.21.15.78:0/2039487900 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3609745483 0 0) 0x7f36480b1040 con 0x7f364815b7f0 2023-10-11T09:40:48.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.834+0000 7f3645d9b700 1 -- 172.21.15.78:0/2039487900 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364815e330 con 0x7f364815a990 2023-10-11T09:40:48.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.834+0000 7fdebffff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fdec00ce560 0x7fdec00715e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:58766/0 (socket says 172.21.15.78:58766) 2023-10-11T09:40:48.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.834+0000 7fdebffff700 1 -- 172.21.15.78:0/4122978210 learned_addr learned my addr 172.21.15.78:0/4122978210 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:48.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.834+0000 7f3645d9b700 1 --2- 172.21.15.78:0/2039487900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f364815a990 0x7f36480b06a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3648143b20 tx=0x7f363c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f451027c17422 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.834+0000 7f4981cbc700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f497c164d20 0x7f497c06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.834+0000 7f4981cbc700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f497c164d20 0x7f497c06bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56964/0 (socket says 172.21.15.78:56964) 2023-10-11T09:40:48.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.834+0000 7f4981cbc700 1 -- 172.21.15.78:0/831835807 learned_addr learned my addr 172.21.15.78:0/831835807 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:48.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.834+0000 7f49814bb700 1 --2- 172.21.15.78:0/831835807 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f497c0cda10 0x7f497c0caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.838+0000 7fdebe7fc700 1 --2- 172.21.15.78:0/4122978210 >> v2:172.21.15.78:3300/0 conn(0x7fdec00b2590 0x7fdec00ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.838+0000 7f364559a700 1 -- 172.21.15.78:0/2039487900 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c011830 con 0x7f364815a990 2023-10-11T09:40:48.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.838+0000 7f364559a700 1 -- 172.21.15.78:0/2039487900 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f363c011990 con 0x7f364815a990 2023-10-11T09:40:48.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.838+0000 7f364559a700 1 -- 172.21.15.78:0/2039487900 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c011d20 con 0x7f364815a990 2023-10-11T09:40:48.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.838+0000 7f4981cbc700 1 -- 172.21.15.78:0/831835807 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f497c0cda10 msgr2=0x7f497c0caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:48.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.838+0000 7f4981cbc700 1 --2- 172.21.15.78:0/831835807 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f497c0cda10 0x7f497c0caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.838+0000 7f4981cbc700 1 -- 172.21.15.78:0/831835807 >> v1:172.21.15.78:6789/0 conn(0x7f497c0cdde0 legacy=0x7f497c164610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:48.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.838+0000 7f4981cbc700 1 -- 172.21.15.78:0/831835807 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f497c0ca720 con 0x7f497c164d20 2023-10-11T09:40:48.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.838+0000 7f5c4ff8b700 1 Processor -- start 2023-10-11T09:40:48.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.838+0000 7fdebdffb700 1 -- 172.21.15.78:0/4122978210 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3754895166 0 0) 0x7fdec00c7300 con 0x7fdec00ce560 2023-10-11T09:40:48.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.838+0000 7fdebdffb700 1 -- 172.21.15.78:0/4122978210 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdea80035d0 con 0x7fdec00ce560 2023-10-11T09:40:48.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.838+0000 7fdebeffd700 1 --2- 172.21.15.78:0/4122978210 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdec00c6f30 0x7fdec00b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.838+0000 7fdebe7fc700 1 -- 172.21.15.78:0/4122978210 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdec00c6f30 msgr2=0x7fdec00b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:48.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.838+0000 7fdebe7fc700 1 --2- 172.21.15.78:0/4122978210 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdec00c6f30 0x7fdec00b2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.838+0000 7fdebe7fc700 1 -- 172.21.15.78:0/4122978210 >> v1:172.21.15.78:6789/0 conn(0x7fdec00ce560 legacy=0x7fdec00715e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.838+0000 7fdebe7fc700 1 -- 172.21.15.78:0/4122978210 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec01591f0 con 0x7fdec00b2590 2023-10-11T09:40:48.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.838+0000 7f5c4ff8b700 1 -- start start 2023-10-11T09:40:48.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.838+0000 7f364cb1f700 1 -- 172.21.15.78:0/2039487900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f364815a990 msgr2=0x7f36480b06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.838+0000 7f364cb1f700 1 --2- 172.21.15.78:0/2039487900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f364815a990 0x7f36480b06a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3648143b20 tx=0x7f363c009bf0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.838+0000 7f4981cbc700 1 --2- 172.21.15.78:0/831835807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f497c164d20 0x7f497c06bbf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4974009e50 tx=0x7f497400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c6aac00ccda9a17 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.838+0000 7fdebe7fc700 1 --2- 172.21.15.78:0/4122978210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdec00b2590 0x7fdec00ce020 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4009850 tx=0x7fdeb4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=846eda5301c6ad6 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.838+0000 7f496ffff700 1 -- 172.21.15.78:0/831835807 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497400e070 con 0x7f497c164d20 2023-10-11T09:40:48.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.838+0000 7f496ffff700 1 -- 172.21.15.78:0/831835807 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4974003a40 con 0x7f497c164d20 2023-10-11T09:40:48.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.838+0000 7f496ffff700 1 -- 172.21.15.78:0/831835807 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4974003df0 con 0x7f497c164d20 2023-10-11T09:40:48.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.838+0000 7f364cb1f700 1 -- 172.21.15.78:0/2039487900 shutdown_connections 2023-10-11T09:40:48.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.838+0000 7f364cb1f700 1 --2- 172.21.15.78:0/2039487900 >> v2:172.21.15.133:3300/0 conn(0x7f36480b0be0 0x7f364813fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.838+0000 7f364cb1f700 1 --2- 172.21.15.78:0/2039487900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f364815a990 0x7f36480b06a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.838+0000 7f364cb1f700 1 -- 172.21.15.78:0/2039487900 >> 172.21.15.78:0/2039487900 conn(0x7f364806aca0 msgr2=0x7f36480cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:48.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.838+0000 7f364cb1f700 1 -- 172.21.15.78:0/2039487900 shutdown_connections 2023-10-11T09:40:48.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.838+0000 7fdebdffb700 1 -- 172.21.15.78:0/4122978210 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4011830 con 0x7fdec00b2590 2023-10-11T09:40:48.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.838+0000 7fdebdffb700 1 -- 172.21.15.78:0/4122978210 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb4011990 con 0x7fdec00b2590 2023-10-11T09:40:48.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.838+0000 7fdebdffb700 1 -- 172.21.15.78:0/4122978210 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4011d40 con 0x7fdec00b2590 2023-10-11T09:40:48.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.838+0000 7f498371f700 1 -- 172.21.15.78:0/831835807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f497c164d20 msgr2=0x7f497c06bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.838+0000 7f498371f700 1 --2- 172.21.15.78:0/831835807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f497c164d20 0x7f497c06bbf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4974009e50 tx=0x7f497400b330 comp rx=0 tx=0).stop 2023-10-11T09:40:48.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.838+0000 7f364cb1f700 1 -- 172.21.15.78:0/2039487900 wait complete. 2023-10-11T09:40:48.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.838+0000 7f5c4ff8b700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c400de730 0x7f5c40036630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.838+0000 7fdec54bf700 1 -- 172.21.15.78:0/4122978210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdec00b2590 msgr2=0x7fdec00ce020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.838+0000 7fdec54bf700 1 --2- 172.21.15.78:0/4122978210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdec00b2590 0x7fdec00ce020 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4009850 tx=0x7fdeb4009b60 comp rx=0 tx=0).stop 2023-10-11T09:40:48.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7f5c4ff8b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f5c400deb00 0x7f5c40036b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7f498371f700 1 -- 172.21.15.78:0/831835807 shutdown_connections 2023-10-11T09:40:48.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7f498371f700 1 --2- 172.21.15.78:0/831835807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f497c164d20 0x7f497c06bbf0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7f498371f700 1 --2- 172.21.15.78:0/831835807 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f497c0cda10 0x7f497c0caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7f498371f700 1 -- 172.21.15.78:0/831835807 >> 172.21.15.78:0/831835807 conn(0x7f497c0cea60 msgr2=0x7f497c0cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:48.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7fdec54bf700 1 -- 172.21.15.78:0/4122978210 shutdown_connections 2023-10-11T09:40:48.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7fdec54bf700 1 --2- 172.21.15.78:0/4122978210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdec00b2590 0x7fdec00ce020 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7fdec54bf700 1 --2- 172.21.15.78:0/4122978210 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdec00c6f30 0x7fdec00b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7fdec54bf700 1 -- 172.21.15.78:0/4122978210 >> 172.21.15.78:0/4122978210 conn(0x7fdec006dd80 msgr2=0x7fdec006e150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:48.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7fdec54bf700 1 -- 172.21.15.78:0/4122978210 shutdown_connections 2023-10-11T09:40:48.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7fa0b31b7700 1 Processor -- start 2023-10-11T09:40:48.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7f498371f700 1 -- 172.21.15.78:0/831835807 shutdown_connections 2023-10-11T09:40:48.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7f498371f700 1 -- 172.21.15.78:0/831835807 wait complete. 2023-10-11T09:40:48.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7f5c4e528700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f5c4004f200 0x7f5c4004f640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:58782/0 (socket says 172.21.15.78:58782) 2023-10-11T09:40:48.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7fa0b31b7700 1 -- start start 2023-10-11T09:40:48.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7f5c4d526700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f5c400deb00 0x7f5c40036b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7f5c4d526700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f5c400deb00 0x7f5c40036b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56980/0 (socket says 172.21.15.78:56980) 2023-10-11T09:40:48.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7f5c4e528700 1 -- 172.21.15.78:0/3839050592 learned_addr learned my addr 172.21.15.78:0/3839050592 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:48.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7f364cb1f700 1 Processor -- start 2023-10-11T09:40:48.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7f5c4dd27700 1 --2- 172.21.15.78:0/3839050592 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c400de730 0x7f5c40036630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7f5c4ff8b700 1 -- 172.21.15.78:0/3839050592 --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c40034460 con 0x7f5c4004f200 2023-10-11T09:40:48.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7f364cb1f700 1 -- start start 2023-10-11T09:40:48.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7f5c4cd25700 1 -- 172.21.15.78:0/3839050592 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1048482616 0 0) 0x7f5c40034460 con 0x7f5c4004f200 2023-10-11T09:40:48.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7f5c4ff8b700 1 -- 172.21.15.78:0/3839050592 --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c400df090 con 0x7f5c400deb00 2023-10-11T09:40:48.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7f5c4ff8b700 1 -- 172.21.15.78:0/3839050592 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c400df370 con 0x7f5c400de730 2023-10-11T09:40:48.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7fa0b31b7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0a40dee50 0x7fa0a404f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7f5c4cd25700 1 -- 172.21.15.78:0/3839050592 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c340035d0 con 0x7f5c4004f200 2023-10-11T09:40:48.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7fdec54bf700 1 -- 172.21.15.78:0/4122978210 wait complete. 2023-10-11T09:40:48.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7fdec54bf700 1 Processor -- start 2023-10-11T09:40:48.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7fa0b31b7700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fa0a404f880 0x7fa0a404fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7f498371f700 1 Processor -- start 2023-10-11T09:40:48.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7f5c4dd27700 1 -- 172.21.15.78:0/3839050592 >> v2:172.21.15.78:3300/0 conn(0x7f5c400deb00 msgr2=0x7f5c40036b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:48.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7f5c4dd27700 1 --2- 172.21.15.78:0/3839050592 >> v2:172.21.15.78:3300/0 conn(0x7f5c400deb00 0x7f5c40036b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7f5c4dd27700 1 -- 172.21.15.78:0/3839050592 >> v1:172.21.15.78:6789/0 conn(0x7f5c4004f200 legacy=0x7f5c4004f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7f5c4dd27700 1 -- 172.21.15.78:0/3839050592 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c400df650 con 0x7f5c400de730 2023-10-11T09:40:48.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7f364cb1f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36480b0be0 0x7f364813a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7f364cb1f700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f364815a990 0x7f36480cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7f364cb1f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f364815b7f0 0x7f36480ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7f364cb1f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3648138f70 con 0x7f364815b7f0 2023-10-11T09:40:48.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7f364cb1f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364813a7f0 con 0x7f36480b0be0 2023-10-11T09:40:48.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7f364cb1f700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36481391c0 con 0x7f364815a990 2023-10-11T09:40:48.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7f498371f700 1 -- start start 2023-10-11T09:40:48.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7fdec54bf700 1 -- start start 2023-10-11T09:40:48.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.842+0000 7fa0b21b5700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0a40dee50 0x7fa0a404f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7fa0b21b5700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0a40dee50 0x7fa0a404f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:44200/0 (socket says 172.21.15.78:44200) 2023-10-11T09:40:48.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7fa0b21b5700 1 -- 172.21.15.78:0/1154459262 learned_addr learned my addr 172.21.15.78:0/1154459262 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:48.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f5c4dd27700 1 --2- 172.21.15.78:0/3839050592 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c400de730 0x7f5c40036630 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5c4804f770 tx=0x7f5c4804f580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c76dd9f8dc4e8862 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7fa0b31b7700 1 -- 172.21.15.78:0/1154459262 --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0a40064d0 con 0x7fa0a4006960 2023-10-11T09:40:48.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7fa0b31b7700 1 -- 172.21.15.78:0/1154459262 --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0a4050210 con 0x7fa0a404f880 2023-10-11T09:40:48.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7fa0b31b7700 1 -- 172.21.15.78:0/1154459262 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a40df220 con 0x7fa0a40dee50 2023-10-11T09:40:48.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f364659c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36480b0be0 0x7f364813a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f364659c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36480b0be0 0x7f364813a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:50034/0 (socket says 172.21.15.78:50034) 2023-10-11T09:40:48.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f364659c700 1 -- 172.21.15.78:0/1080473021 learned_addr learned my addr 172.21.15.78:0/1080473021 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:48.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f3645d9b700 1 --2- 172.21.15.78:0/1080473021 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f364815a990 0x7f36480cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f364659c700 1 -- 172.21.15.78:0/1080473021 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f364815a990 msgr2=0x7f36480cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f364659c700 1 --2- 172.21.15.78:0/1080473021 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f364815a990 0x7f36480cc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f364659c700 1 -- 172.21.15.78:0/1080473021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f364815b7f0 msgr2=0x7f36480ccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f364659c700 1 --2- 172.21.15.78:0/1080473021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f364815b7f0 0x7f36480ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f364659c700 1 -- 172.21.15.78:0/1080473021 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363c009570 con 0x7f36480b0be0 2023-10-11T09:40:48.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f7aea04a700 1 Processor -- start 2023-10-11T09:40:48.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7fdec54bf700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdec00b2590 0x7fdec00c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f498371f700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f497c0cda10 0x7f497c146e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f7aea04a700 1 -- start start 2023-10-11T09:40:48.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f364659c700 1 --2- 172.21.15.78:0/1080473021 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36480b0be0 0x7f364813a2b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f364000cc50 tx=0x7f364000cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7fdec54bf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdec00c6f30 0x7fdec01fdda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f5c4cd25700 1 -- 172.21.15.78:0/3839050592 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c48067f50 con 0x7f5c400de730 2023-10-11T09:40:48.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f5c4cd25700 1 -- 172.21.15.78:0/3839050592 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c480680b0 con 0x7f5c400de730 2023-10-11T09:40:48.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f5c4cd25700 1 -- 172.21.15.78:0/3839050592 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c48067340 con 0x7f5c400de730 2023-10-11T09:40:48.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f5c4ff8b700 1 -- 172.21.15.78:0/3839050592 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c400de730 msgr2=0x7f5c40036630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f5c4ff8b700 1 --2- 172.21.15.78:0/3839050592 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c400de730 0x7f5c40036630 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5c4804f770 tx=0x7f5c4804f580 comp rx=0 tx=0).stop 2023-10-11T09:40:48.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7fa0b21b5700 1 -- 172.21.15.78:0/1154459262 >> v2:172.21.15.78:3300/0 conn(0x7fa0a404f880 msgr2=0x7fa0a404fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:48.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7fa0b21b5700 1 --2- 172.21.15.78:0/1154459262 >> v2:172.21.15.78:3300/0 conn(0x7fa0a404f880 0x7fa0a404fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7fa0b21b5700 1 -- 172.21.15.78:0/1154459262 >> v1:172.21.15.78:6789/0 conn(0x7fa0a4006960 legacy=0x7fa0a4006de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7fdebeffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdec00b2590 0x7fdec00c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7fdebeffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdec00b2590 0x7fdec00c7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:50040/0 (socket says 172.21.15.78:50040) 2023-10-11T09:40:48.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f5c4ff8b700 1 -- 172.21.15.78:0/3839050592 shutdown_connections 2023-10-11T09:40:48.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f5c4ff8b700 1 --2- 172.21.15.78:0/3839050592 >> v2:172.21.15.78:3300/0 conn(0x7f5c400deb00 0x7f5c40036b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f5c4ff8b700 1 --2- 172.21.15.78:0/3839050592 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c400de730 0x7f5c40036630 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f5c4ff8b700 1 -- 172.21.15.78:0/3839050592 >> 172.21.15.78:0/3839050592 conn(0x7f5c40051630 msgr2=0x7f5c40051a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:48.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f3636ffd700 1 -- 172.21.15.78:0/1080473021 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3640003e50 con 0x7f36480b0be0 2023-10-11T09:40:48.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f3636ffd700 1 -- 172.21.15.78:0/1080473021 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f364000a770 con 0x7f36480b0be0 2023-10-11T09:40:48.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f3636ffd700 1 -- 172.21.15.78:0/1080473021 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364000aae0 con 0x7f36480b0be0 2023-10-11T09:40:48.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f364cb1f700 1 -- 172.21.15.78:0/1080473021 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36481394a0 con 0x7f36480b0be0 2023-10-11T09:40:48.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f49814bb700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f497c0cda10 0x7f497c146e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7fdebe7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdec00c6f30 0x7fdec01fdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7fdebe7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdec00c6f30 0x7fdec01fdda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57008/0 (socket says 172.21.15.78:57008) 2023-10-11T09:40:48.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7fa0b21b5700 1 -- 172.21.15.78:0/1154459262 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a40340b0 con 0x7fa0a40dee50 2023-10-11T09:40:48.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f5c4ff8b700 1 -- 172.21.15.78:0/3839050592 shutdown_connections 2023-10-11T09:40:48.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7fdebeffd700 1 -- 172.21.15.78:0/1848472170 learned_addr learned my addr 172.21.15.78:0/1848472170 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:48.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f49814bb700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f497c0cda10 0x7f497c146e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:44218/0 (socket says 172.21.15.78:44218) 2023-10-11T09:40:48.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f498371f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f497c0cdde0 0x7f497c162770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7fdec54bf700 1 --2- 172.21.15.78:0/1848472170 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdec00ce560 0x7fdec01fe2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7fdec54bf700 1 -- 172.21.15.78:0/1848472170 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec01fe820 con 0x7fdec00c6f30 2023-10-11T09:40:48.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7fa0b21b5700 1 --2- 172.21.15.78:0/1154459262 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0a40dee50 0x7fa0a404f340 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8009910 tx=0x7fa0a8009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6327e822158735e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7fdec54bf700 1 -- 172.21.15.78:0/1848472170 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec00cb1c0 con 0x7fdec00b2590 2023-10-11T09:40:48.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f364cb1f700 1 -- 172.21.15.78:0/1080473021 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3648139a60 con 0x7f36480b0be0 2023-10-11T09:40:48.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f498371f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f497c164d20 0x7f497c06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f498371f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497c1f7a40 con 0x7f497c0cdde0 2023-10-11T09:40:48.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f498371f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497c1f7d20 con 0x7f497c164d20 2023-10-11T09:40:48.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f498371f700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f497c1f8000 con 0x7f497c0cda10 2023-10-11T09:40:48.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f49814bb700 1 -- 172.21.15.78:0/2970240399 learned_addr learned my addr 172.21.15.78:0/2970240399 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:48.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7fdec54bf700 1 -- 172.21.15.78:0/1848472170 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec00cabb0 con 0x7fdec00ce560 2023-10-11T09:40:48.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7fdebffff700 1 --2- 172.21.15.78:0/1848472170 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdec00ce560 0x7fdec01fe2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7fdebeffd700 1 -- 172.21.15.78:0/1848472170 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdec00ce560 msgr2=0x7fdec01fe2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7fdebeffd700 1 --2- 172.21.15.78:0/1848472170 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdec00ce560 0x7fdec01fe2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7fdebeffd700 1 -- 172.21.15.78:0/1848472170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdec00c6f30 msgr2=0x7fdec01fdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7fdebeffd700 1 --2- 172.21.15.78:0/1848472170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdec00c6f30 0x7fdec01fdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7fdebeffd700 1 -- 172.21.15.78:0/1848472170 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb40094b0 con 0x7fdec00b2590 2023-10-11T09:40:48.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7fa0b11b3700 1 -- 172.21.15.78:0/1154459262 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a800ba60 con 0x7fa0a40dee50 2023-10-11T09:40:48.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7fa0b11b3700 1 -- 172.21.15.78:0/1154459262 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a800bbc0 con 0x7fa0a40dee50 2023-10-11T09:40:48.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f7aea04a700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7adc038430 0x7f7adc006c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f7aea04a700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7adc03b250 0x7f7adc056c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f7aea04a700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7adc0077c0 con 0x7f7adc055690 2023-10-11T09:40:48.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f7aea04a700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7adc038800 con 0x7f7adc038430 2023-10-11T09:40:48.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f7aea04a700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc007160 con 0x7f7adc03b250 2023-10-11T09:40:48.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7f7ae37fe700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7adc038430 0x7f7adc006c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7f7ae37fe700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7adc038430 0x7f7adc006c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:50066/0 (socket says 172.21.15.78:50066) 2023-10-11T09:40:48.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7f7ae37fe700 1 -- 172.21.15.78:0/90528879 learned_addr learned my addr 172.21.15.78:0/90528879 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:48.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f4980cba700 1 --2- 172.21.15.78:0/2970240399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f497c0cdde0 0x7f497c162770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f4981cbc700 1 --2- 172.21.15.78:0/2970240399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f497c164d20 0x7f497c06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7f49814bb700 1 -- 172.21.15.78:0/2970240399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f497c164d20 msgr2=0x7f497c06bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7f49814bb700 1 --2- 172.21.15.78:0/2970240399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f497c164d20 0x7f497c06bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7f49814bb700 1 -- 172.21.15.78:0/2970240399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f497c0cdde0 msgr2=0x7f497c162770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7f49814bb700 1 --2- 172.21.15.78:0/2970240399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f497c0cdde0 0x7f497c162770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7f49814bb700 1 -- 172.21.15.78:0/2970240399 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4974009b30 con 0x7f497c0cda10 2023-10-11T09:40:48.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7fdebeffd700 1 --2- 172.21.15.78:0/1848472170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdec00b2590 0x7fdec00c7e10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdeb000e760 tx=0x7fdeb000ea70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7f7ae3fff700 1 --2- 172.21.15.78:0/90528879 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7adc03b250 0x7f7adc056c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7fa0b31b7700 1 -- 172.21.15.78:0/1154459262 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0a40dee50 msgr2=0x7fa0a404f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7fa0b31b7700 1 --2- 172.21.15.78:0/1154459262 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0a40dee50 0x7fa0a404f340 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8009910 tx=0x7fa0a8009c20 comp rx=0 tx=0).stop 2023-10-11T09:40:48.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7fa0b11b3700 1 -- 172.21.15.78:0/1154459262 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8009e60 con 0x7fa0a40dee50 2023-10-11T09:40:48.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.846+0000 7f5c4ff8b700 1 -- 172.21.15.78:0/3839050592 wait complete. 2023-10-11T09:40:48.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7fc3754f7700 1 Processor -- start 2023-10-11T09:40:48.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7f7ae37fe700 1 -- 172.21.15.78:0/90528879 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7adc03b250 msgr2=0x7f7adc056c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:48.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7f7ae37fe700 1 --2- 172.21.15.78:0/90528879 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7adc03b250 0x7f7adc056c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7f7ae37fe700 1 -- 172.21.15.78:0/90528879 >> v1:172.21.15.78:6789/0 conn(0x7f7adc055690 legacy=0x7f7adc03ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:48.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7f7ae37fe700 1 -- 172.21.15.78:0/90528879 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7adc053840 con 0x7f7adc038430 2023-10-11T09:40:48.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7f5c4ff8b700 1 Processor -- start 2023-10-11T09:40:48.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7fdea77fe700 1 -- 172.21.15.78:0/1848472170 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb000c070 con 0x7fdec00b2590 2023-10-11T09:40:48.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7fdea77fe700 1 -- 172.21.15.78:0/1848472170 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb0013610 con 0x7fdec00b2590 2023-10-11T09:40:48.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7fdea77fe700 1 -- 172.21.15.78:0/1848472170 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0013980 con 0x7fdec00b2590 2023-10-11T09:40:48.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7fdec54bf700 1 -- 172.21.15.78:0/1848472170 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdec00ca010 con 0x7fdec00b2590 2023-10-11T09:40:48.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7fdec54bf700 1 -- 172.21.15.78:0/1848472170 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec00ca2f0 con 0x7fdec00b2590 2023-10-11T09:40:48.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7fa0b31b7700 1 -- 172.21.15.78:0/1154459262 shutdown_connections 2023-10-11T09:40:48.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7fa0b31b7700 1 --2- 172.21.15.78:0/1154459262 >> v2:172.21.15.78:3300/0 conn(0x7fa0a404f880 0x7fa0a404fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7fa0b31b7700 1 --2- 172.21.15.78:0/1154459262 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0a40dee50 0x7fa0a404f340 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7fa0b31b7700 1 -- 172.21.15.78:0/1154459262 >> 172.21.15.78:0/1154459262 conn(0x7fa0a4051770 msgr2=0x7fa0a4051b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:48.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7fa0b31b7700 1 -- 172.21.15.78:0/1154459262 shutdown_connections 2023-10-11T09:40:48.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7fa0b31b7700 1 -- 172.21.15.78:0/1154459262 wait complete. 2023-10-11T09:40:48.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7f7ae37fe700 1 --2- 172.21.15.78:0/90528879 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7adc038430 0x7f7adc006c20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8009910 tx=0x7f7ad8009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d425378ede33a66f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7f7ae27fc700 1 -- 172.21.15.78:0/90528879 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8004230 con 0x7f7adc038430 2023-10-11T09:40:48.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7f71bb239700 1 Processor -- start 2023-10-11T09:40:48.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7f71bb239700 1 -- start start 2023-10-11T09:40:48.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7f7ae27fc700 1 -- 172.21.15.78:0/90528879 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad8004390 con 0x7f7adc038430 2023-10-11T09:40:48.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7f7ae27fc700 1 -- 172.21.15.78:0/90528879 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8009e60 con 0x7f7adc038430 2023-10-11T09:40:48.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7f5c4ff8b700 1 -- start start 2023-10-11T09:40:48.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7fdec54bf700 1 -- 172.21.15.78:0/1848472170 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdec004e530 con 0x7fdec00b2590 2023-10-11T09:40:48.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7f5c4ff8b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c4004f200 0x7f5c400d5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7f49814bb700 1 --2- 172.21.15.78:0/2970240399 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f497c0cda10 0x7f497c146e40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f497800e9d0 tx=0x7f497800ece0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7f3636ffd700 1 -- 172.21.15.78:0/1080473021 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f364000ac40 con 0x7f36480b0be0 2023-10-11T09:40:48.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7f496dffb700 1 -- 172.21.15.78:0/2970240399 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4978009720 con 0x7f497c0cda10 2023-10-11T09:40:48.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7f498371f700 1 -- 172.21.15.78:0/2970240399 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f497c0ce3b0 con 0x7f497c0cda10 2023-10-11T09:40:48.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7f496dffb700 1 -- 172.21.15.78:0/2970240399 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497800c850 con 0x7f497c0cda10 2023-10-11T09:40:48.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7f496dffb700 1 -- 172.21.15.78:0/2970240399 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497800cbc0 con 0x7f497c0cda10 2023-10-11T09:40:48.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7f5c4ff8b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c400de730 0x7f5c400499c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7f5c4dd27700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c4004f200 0x7f5c400d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7f5c4dd27700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c4004f200 0x7f5c400d5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57026/0 (socket says 172.21.15.78:57026) 2023-10-11T09:40:48.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7f7aea04a700 1 -- 172.21.15.78:0/90528879 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7adc038430 msgr2=0x7f7adc006c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7f7aea04a700 1 --2- 172.21.15.78:0/90528879 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7adc038430 0x7f7adc006c20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8009910 tx=0x7f7ad8009c20 comp rx=0 tx=0).stop 2023-10-11T09:40:48.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7f7aea04a700 1 -- 172.21.15.78:0/90528879 shutdown_connections 2023-10-11T09:40:48.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7f7aea04a700 1 --2- 172.21.15.78:0/90528879 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7adc03b250 0x7f7adc056c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7f5c4dd27700 1 -- 172.21.15.78:0/910380996 learned_addr learned my addr 172.21.15.78:0/910380996 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:48.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7f7aea04a700 1 --2- 172.21.15.78:0/90528879 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7adc038430 0x7f7adc006c20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7f7aea04a700 1 -- 172.21.15.78:0/90528879 >> 172.21.15.78:0/90528879 conn(0x7f7adc055b40 msgr2=0x7f7adc055f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:48.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7f5c4ff8b700 1 --2- 172.21.15.78:0/910380996 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c400deb00 0x7f5c40049f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.850+0000 7fc3754f7700 1 -- start start 2023-10-11T09:40:48.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7f7aea04a700 1 -- 172.21.15.78:0/90528879 shutdown_connections 2023-10-11T09:40:48.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7f5c4d526700 1 --2- 172.21.15.78:0/910380996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c400de730 0x7f5c400499c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7f71bb239700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f71ac0e29f0 0x7f71ac055490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7f5c4ff8b700 1 -- 172.21.15.78:0/910380996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4004bc80 con 0x7f5c4004f200 2023-10-11T09:40:48.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7f71bb239700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f71ac03b330 0x7f71ac057210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7f71bb239700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71ac007360 con 0x7f71ac038560 2023-10-11T09:40:48.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7f71bb239700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71ac0e2dc0 con 0x7f71ac03b330 2023-10-11T09:40:48.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7f71bb239700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71ac053f70 con 0x7f71ac0e29f0 2023-10-11T09:40:48.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7f3636ffd700 1 --2- 172.21.15.78:0/1080473021 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3628065fa0 0x7f3628068450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7f3636ffd700 1 -- 172.21.15.78:0/1080473021 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3640083240 con 0x7f36480b0be0 2023-10-11T09:40:48.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7f5c4e528700 1 --2- 172.21.15.78:0/910380996 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c400deb00 0x7f5c40049f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7f71ba237700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f71ac0e29f0 0x7f71ac055490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7f71ba237700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f71ac0e29f0 0x7f71ac055490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:50090/0 (socket says 172.21.15.78:50090) 2023-10-11T09:40:48.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7f71ba237700 1 -- 172.21.15.78:0/1158985610 learned_addr learned my addr 172.21.15.78:0/1158985610 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:48.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7f71baa38700 1 --2- 172.21.15.78:0/1158985610 >> v2:172.21.15.78:3300/0 conn(0x7f71ac03b330 0x7f71ac057210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7f7aea04a700 1 -- 172.21.15.78:0/90528879 wait complete. 2023-10-11T09:40:48.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7f364cb1f700 1 -- 172.21.15.78:0/1080473021 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36480b1a10 con 0x7f36480b0be0 2023-10-11T09:40:48.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7f3645d9b700 1 --2- 172.21.15.78:0/1080473021 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3628065fa0 0x7f3628068450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7f498371f700 1 -- 172.21.15.78:0/2970240399 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f497c1f85c0 con 0x7f497c0cda10 2023-10-11T09:40:48.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7f498371f700 1 -- 172.21.15.78:0/2970240399 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f497c1f87e0 con 0x7f497c0cda10 2023-10-11T09:40:48.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7f5c4ff8b700 1 -- 172.21.15.78:0/910380996 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4004c520 con 0x7f5c400de730 2023-10-11T09:40:48.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7f5c4ff8b700 1 -- 172.21.15.78:0/910380996 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c400d61e0 con 0x7f5c400deb00 2023-10-11T09:40:48.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7fa0b31b7700 1 Processor -- start 2023-10-11T09:40:48.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7fa0b31b7700 1 -- start start 2023-10-11T09:40:48.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7fc3754f7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc3700cda10 0x7fc3700caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7fc3754f7700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc370165390 con 0x7fc370164d20 2023-10-11T09:40:48.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7fc3754f7700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3700ca440 con 0x7fc3700cdde0 2023-10-11T09:40:48.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7fc3754f7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3701651a0 con 0x7fc3700cda10 2023-10-11T09:40:48.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7fc36ffff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc3700cda10 0x7fc3700caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7fc36f7fe700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fc3700cdde0 0x7fc370164610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:33816/0 (socket says 172.21.15.78:33816) 2023-10-11T09:40:48.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7fc36f7fe700 1 -- 172.21.15.78:0/1793620657 learned_addr learned my addr 172.21.15.78:0/1793620657 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:48.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7f7aea04a700 1 Processor -- start 2023-10-11T09:40:48.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7f71ba237700 1 -- 172.21.15.78:0/1158985610 >> v2:172.21.15.78:3300/0 conn(0x7f71ac03b330 msgr2=0x7f71ac057210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:48.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7f71ba237700 1 --2- 172.21.15.78:0/1158985610 >> v2:172.21.15.78:3300/0 conn(0x7f71ac03b330 0x7f71ac057210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7f71ba237700 1 -- 172.21.15.78:0/1158985610 >> v1:172.21.15.133:6789/0 conn(0x7f71ac038560 legacy=0x7f71ac03ac20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-11T09:40:48.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7f71ba237700 1 -- 172.21.15.78:0/1158985610 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71ac006dc0 con 0x7f71ac0e29f0 2023-10-11T09:40:48.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7f71ba237700 1 --2- 172.21.15.78:0/1158985610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71ac0e29f0 0x7f71ac055490 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f71ac012210 tx=0x7f71b0009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=730d87ffae82cc9d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7f7aea04a700 1 -- start start 2023-10-11T09:40:48.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7f5c4d526700 1 -- 172.21.15.78:0/910380996 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c400deb00 msgr2=0x7f5c40049f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7f5c4d526700 1 --2- 172.21.15.78:0/910380996 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c400deb00 0x7f5c40049f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7f5c4d526700 1 -- 172.21.15.78:0/910380996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c4004f200 msgr2=0x7f5c400d5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7f5c4d526700 1 --2- 172.21.15.78:0/910380996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c4004f200 0x7f5c400d5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7f5c4d526700 1 -- 172.21.15.78:0/910380996 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c4804f3f0 con 0x7f5c400de730 2023-10-11T09:40:48.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7f5c4d526700 1 --2- 172.21.15.78:0/910380996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c400de730 0x7f5c400499c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5c3800e870 tx=0x7f5c3800eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7fa0b31b7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0a4006960 0x7fa0a40dccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7f7aea04a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7adc038430 0x7f7adc17c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.854+0000 7fdea77fe700 1 -- 172.21.15.78:0/1848472170 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fdeb0013ae0 con 0x7fdec00b2590 2023-10-11T09:40:48.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7fc36effd700 1 -- 172.21.15.78:0/1793620657 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2692948662 0 0) 0x7fc370165390 con 0x7fc370164d20 2023-10-11T09:40:48.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7fc36effd700 1 -- 172.21.15.78:0/1793620657 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc35c0035d0 con 0x7fc370164d20 2023-10-11T09:40:48.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7fa0b31b7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0a404f880 0x7fa0a4176a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7fa0b21b5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0a4006960 0x7fa0a40dccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7fa0b21b5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0a4006960 0x7fa0a40dccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57056/0 (socket says 172.21.15.78:57056) 2023-10-11T09:40:48.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7fc36ffff700 1 -- 172.21.15.78:0/1793620657 >> v1:172.21.15.133:6789/0 conn(0x7fc3700cdde0 legacy=0x7fc370164610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7fc36ffff700 1 -- 172.21.15.78:0/1793620657 >> v1:172.21.15.78:6789/0 conn(0x7fc370164d20 legacy=0x7fc37006bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7fc36ffff700 1 -- 172.21.15.78:0/1793620657 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3700ca720 con 0x7fc3700cda10 2023-10-11T09:40:48.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7f7ae37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7adc038430 0x7f7adc17c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7f7ae37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7adc038430 0x7f7adc17c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57044/0 (socket says 172.21.15.78:57044) 2023-10-11T09:40:48.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7f7aea04a700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7adc03b250 0x7f7adc0da080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7fa0b21b5700 1 -- 172.21.15.78:0/195245663 learned_addr learned my addr 172.21.15.78:0/195245663 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:48.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7f7ae37fe700 1 -- 172.21.15.78:0/539272325 learned_addr learned my addr 172.21.15.78:0/539272325 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:48.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7f7aea04a700 1 --2- 172.21.15.78:0/539272325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7adc055690 0x7f7adc0dc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7fa0b31b7700 1 --2- 172.21.15.78:0/195245663 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0a40dee50 0x7fa0a40d9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7f3645d9b700 1 --2- 172.21.15.78:0/1080473021 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3628065fa0 0x7f3628068450 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f363c000c00 tx=0x7f363c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7f496dffb700 1 -- 172.21.15.78:0/2970240399 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f497800cd20 con 0x7f497c0cda10 2023-10-11T09:40:48.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7f7aea04a700 1 -- 172.21.15.78:0/539272325 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc17c660 con 0x7f7adc038430 2023-10-11T09:40:48.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7fdea77fe700 1 --2- 172.21.15.78:0/1848472170 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdeac065f20 0x7fdeac0683d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7f7aea04a700 1 -- 172.21.15.78:0/539272325 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc1790d0 con 0x7f7adc055690 2023-10-11T09:40:48.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7fdea77fe700 1 -- 172.21.15.78:0/1848472170 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdeb00840e0 con 0x7fdec00b2590 2023-10-11T09:40:48.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7f7aea04a700 1 -- 172.21.15.78:0/539272325 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc0de560 con 0x7f7adc03b250 2023-10-11T09:40:48.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7fc36ffff700 1 --2- 172.21.15.78:0/1793620657 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc3700cda10 0x7fc3700caf20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc368009880 tx=0x7fc368009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12ac37c28650fa87 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7fa0b19b4700 1 --2- 172.21.15.78:0/195245663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0a404f880 0x7fa0a4176a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7fa0b31b7700 1 -- 172.21.15.78:0/195245663 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a40dd200 con 0x7fa0a4006960 2023-10-11T09:40:48.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7fa0b31b7700 1 -- 172.21.15.78:0/195245663 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a40497b0 con 0x7fa0a404f880 2023-10-11T09:40:48.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7f7ae2ffd700 1 --2- 172.21.15.78:0/539272325 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7adc03b250 0x7f7adc0da080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7fa0b29b6700 1 --2- 172.21.15.78:0/195245663 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0a40dee50 0x7fa0a40d9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7fa0b31b7700 1 -- 172.21.15.78:0/195245663 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a4049a90 con 0x7fa0a40dee50 2023-10-11T09:40:48.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7fdebe7fc700 1 --2- 172.21.15.78:0/1848472170 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdeac065f20 0x7fdeac0683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7f7ae37fe700 1 -- 172.21.15.78:0/539272325 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7adc03b250 msgr2=0x7f7adc0da080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7f7ae37fe700 1 --2- 172.21.15.78:0/539272325 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7adc03b250 0x7f7adc0da080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7f7ae37fe700 1 -- 172.21.15.78:0/539272325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7adc055690 msgr2=0x7f7adc0dc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7f7ae37fe700 1 --2- 172.21.15.78:0/539272325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7adc055690 0x7f7adc0dc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7f7ae37fe700 1 -- 172.21.15.78:0/539272325 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad8009570 con 0x7f7adc038430 2023-10-11T09:40:48.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7f7ae37fe700 1 --2- 172.21.15.78:0/539272325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7adc038430 0x7f7adc17c120 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8004000 tx=0x7f7ad8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7f496dffb700 1 --2- 172.21.15.78:0/2970240399 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4960065ed0 0x7f4960068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7fdebe7fc700 1 --2- 172.21.15.78:0/1848472170 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdeac065f20 0x7fdeac0683d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40090b0 tx=0x7fdeb4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7f7acffff700 1 -- 172.21.15.78:0/539272325 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8009e80 con 0x7f7adc038430 2023-10-11T09:40:48.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7f7acffff700 1 -- 172.21.15.78:0/539272325 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad80036c0 con 0x7f7adc038430 2023-10-11T09:40:48.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7f7acffff700 1 -- 172.21.15.78:0/539272325 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8003a30 con 0x7f7adc038430 2023-10-11T09:40:48.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7f7aea04a700 1 -- 172.21.15.78:0/539272325 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7adc178830 con 0x7f7adc038430 2023-10-11T09:40:48.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7f496dffb700 1 -- 172.21.15.78:0/2970240399 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f49780832b0 con 0x7f497c0cda10 2023-10-11T09:40:48.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7fa0b21b5700 1 -- 172.21.15.78:0/195245663 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0a40dee50 msgr2=0x7fa0a40d9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7fa0b21b5700 1 --2- 172.21.15.78:0/195245663 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0a40dee50 0x7fa0a40d9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7fa0b21b5700 1 -- 172.21.15.78:0/195245663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0a404f880 msgr2=0x7fa0a4176a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7fa0b21b5700 1 --2- 172.21.15.78:0/195245663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0a404f880 0x7fa0a4176a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7fa0b21b5700 1 -- 172.21.15.78:0/195245663 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a8009570 con 0x7fa0a4006960 2023-10-11T09:40:48.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7fa0b21b5700 1 --2- 172.21.15.78:0/195245663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0a4006960 0x7fa0a40dccc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8000c00 tx=0x7fa0a8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7f7aea04a700 1 -- 172.21.15.78:0/539272325 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7adc178df0 con 0x7f7adc038430 2023-10-11T09:40:48.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7f4980cba700 1 --2- 172.21.15.78:0/2970240399 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4960065ed0 0x7f4960068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7fa0a2ffd700 1 -- 172.21.15.78:0/195245663 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8003e00 con 0x7fa0a4006960 2023-10-11T09:40:48.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7fa0a2ffd700 1 -- 172.21.15.78:0/195245663 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a8020470 con 0x7fa0a4006960 2023-10-11T09:40:48.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7fa0b31b7700 1 -- 172.21.15.78:0/195245663 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0a4049d70 con 0x7fa0a4006960 2023-10-11T09:40:48.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7fa0b31b7700 1 -- 172.21.15.78:0/195245663 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0a404a330 con 0x7fa0a4006960 2023-10-11T09:40:48.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.858+0000 7f7aea04a700 1 -- 172.21.15.78:0/539272325 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7adc005840 con 0x7f7adc038430 2023-10-11T09:40:48.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.862+0000 7f4980cba700 1 --2- 172.21.15.78:0/2970240399 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4960065ed0 0x7f4960068380 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f497c0c9180 tx=0x7f4970007ee0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.862+0000 7fa0a2ffd700 1 -- 172.21.15.78:0/195245663 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8020820 con 0x7fa0a4006960 2023-10-11T09:40:48.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.862+0000 7f7acffff700 1 -- 172.21.15.78:0/539272325 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f7ad8003b90 con 0x7f7adc038430 2023-10-11T09:40:48.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.862+0000 7fa0b31b7700 1 -- 172.21.15.78:0/195245663 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa094005310 con 0x7fa0a4006960 2023-10-11T09:40:48.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.862+0000 7f7acffff700 1 --2- 172.21.15.78:0/539272325 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7ac8065f20 0x7f7ac80683d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.862+0000 7fa0a2ffd700 1 -- 172.21.15.78:0/195245663 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fa0a8020980 con 0x7fa0a4006960 2023-10-11T09:40:48.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.862+0000 7f7acffff700 1 -- 172.21.15.78:0/539272325 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7ad8085060 con 0x7f7adc038430 2023-10-11T09:40:48.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.862+0000 7f7ae2ffd700 1 --2- 172.21.15.78:0/539272325 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7ac8065f20 0x7f7ac80683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.862+0000 7f7ae2ffd700 1 --2- 172.21.15.78:0/539272325 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7ac8065f20 0x7f7ac80683d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7adc0cf950 tx=0x7f7ad0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.862+0000 7fa0a2ffd700 1 --2- 172.21.15.78:0/195245663 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa090065f20 0x7fa0900683d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.862+0000 7fdea77fe700 1 -- 172.21.15.78:0/1848472170 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fdeb00517a0 con 0x7fdec00b2590 2023-10-11T09:40:48.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.862+0000 7fa0a2ffd700 1 -- 172.21.15.78:0/195245663 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa0a8013070 con 0x7fa0a4006960 2023-10-11T09:40:48.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.862+0000 7fa0b19b4700 1 --2- 172.21.15.78:0/195245663 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa090065f20 0x7fa0900683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.862+0000 7fa0b19b4700 1 --2- 172.21.15.78:0/195245663 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa090065f20 0x7fa0900683d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4048520 tx=0x7fa09c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.866+0000 7f7acffff700 1 -- 172.21.15.78:0/539272325 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f7ad80b5fa0 con 0x7f7adc038430 2023-10-11T09:40:48.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.866+0000 7f496dffb700 1 -- 172.21.15.78:0/2970240399 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f4978050900 con 0x7f497c0cda10 2023-10-11T09:40:48.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.866+0000 7f5c3e7fc700 1 -- 172.21.15.78:0/910380996 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c38013680 con 0x7f5c400de730 2023-10-11T09:40:48.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.866+0000 7f5c3e7fc700 1 -- 172.21.15.78:0/910380996 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c380137e0 con 0x7f5c400de730 2023-10-11T09:40:48.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.866+0000 7f5c3e7fc700 1 -- 172.21.15.78:0/910380996 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c38013b70 con 0x7f5c400de730 2023-10-11T09:40:48.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.866+0000 7fc36effd700 1 -- 172.21.15.78:0/1793620657 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc368011830 con 0x7fc3700cda10 2023-10-11T09:40:48.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.866+0000 7fc36effd700 1 -- 172.21.15.78:0/1793620657 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc368011990 con 0x7fc3700cda10 2023-10-11T09:40:48.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.866+0000 7f3636ffd700 1 -- 172.21.15.78:0/1080473021 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f3640050920 con 0x7f36480b0be0 2023-10-11T09:40:48.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.866+0000 7fc36effd700 1 -- 172.21.15.78:0/1793620657 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc368011d20 con 0x7fc3700cda10 2023-10-11T09:40:48.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.866+0000 7fc3754f7700 1 -- 172.21.15.78:0/1793620657 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc3700cda10 msgr2=0x7fc3700caf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.866+0000 7fc3754f7700 1 --2- 172.21.15.78:0/1793620657 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc3700cda10 0x7fc3700caf20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc368009880 tx=0x7fc368009b90 comp rx=0 tx=0).stop 2023-10-11T09:40:48.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.866+0000 7f71b9235700 1 -- 172.21.15.78:0/1158985610 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b0004230 con 0x7f71ac0e29f0 2023-10-11T09:40:48.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.866+0000 7f71b9235700 1 -- 172.21.15.78:0/1158985610 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b0004390 con 0x7f71ac0e29f0 2023-10-11T09:40:48.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.866+0000 7f71b9235700 1 -- 172.21.15.78:0/1158985610 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b00036c0 con 0x7f71ac0e29f0 2023-10-11T09:40:48.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.870+0000 7fc3754f7700 1 -- 172.21.15.78:0/1793620657 shutdown_connections 2023-10-11T09:40:48.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.870+0000 7fc3754f7700 1 --2- 172.21.15.78:0/1793620657 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc3700cda10 0x7fc3700caf20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.870+0000 7fc3754f7700 1 -- 172.21.15.78:0/1793620657 >> 172.21.15.78:0/1793620657 conn(0x7fc3700cea60 msgr2=0x7fc3700cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:48.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.870+0000 7fc3754f7700 1 -- 172.21.15.78:0/1793620657 shutdown_connections 2023-10-11T09:40:48.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.870+0000 7fc3754f7700 1 -- 172.21.15.78:0/1793620657 wait complete. 2023-10-11T09:40:48.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.870+0000 7f71bb239700 1 -- 172.21.15.78:0/1158985610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71ac0e29f0 msgr2=0x7f71ac055490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.870+0000 7f71bb239700 1 --2- 172.21.15.78:0/1158985610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71ac0e29f0 0x7f71ac055490 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f71ac012210 tx=0x7f71b0009bf0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.870+0000 7f71bb239700 1 -- 172.21.15.78:0/1158985610 shutdown_connections 2023-10-11T09:40:48.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.870+0000 7f71bb239700 1 --2- 172.21.15.78:0/1158985610 >> v2:172.21.15.78:3300/0 conn(0x7f71ac03b330 0x7f71ac057210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.870+0000 7f71bb239700 1 --2- 172.21.15.78:0/1158985610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71ac0e29f0 0x7f71ac055490 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f71ac012210 tx=0x7f71b0009bf0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.870+0000 7f71bb239700 1 -- 172.21.15.78:0/1158985610 >> 172.21.15.78:0/1158985610 conn(0x7f71ac055c20 msgr2=0x7f71ac056020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:48.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.870+0000 7f5c4ff8b700 1 -- 172.21.15.78:0/910380996 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c40035810 con 0x7f5c400de730 2023-10-11T09:40:48.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.870+0000 7f5c4ff8b700 1 -- 172.21.15.78:0/910380996 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c4004c140 con 0x7f5c400de730 2023-10-11T09:40:48.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.870+0000 7f71bb239700 1 -- 172.21.15.78:0/1158985610 shutdown_connections 2023-10-11T09:40:48.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.870+0000 7f71bb239700 1 -- 172.21.15.78:0/1158985610 wait complete. 2023-10-11T09:40:48.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.870+0000 7fc3754f7700 1 Processor -- start 2023-10-11T09:40:48.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.870+0000 7f5c4ff8b700 1 -- 172.21.15.78:0/910380996 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c40036f40 con 0x7f5c400de730 2023-10-11T09:40:48.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.870+0000 7fc3754f7700 1 -- start start 2023-10-11T09:40:48.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.870+0000 7f71bb239700 1 Processor -- start 2023-10-11T09:40:48.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.874+0000 7f71bb239700 1 -- start start 2023-10-11T09:40:48.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.874+0000 7f71bb239700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71ac038560 0x7f71ac0dc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.874+0000 7f71bb239700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71ac03b330 0x7f71ac0e1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.874+0000 7fc3754f7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3700cda10 0x7fc3701fed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.874+0000 7fc3754f7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc3700cdde0 0x7fc3701632d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.874+0000 7f71bb239700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f71ac052440 0x7f71ac0528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.874+0000 7f71b9a36700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71ac03b330 0x7f71ac0e1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.874+0000 7f71ba237700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71ac038560 0x7f71ac0dc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.874+0000 7f71ba237700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71ac038560 0x7f71ac0dc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:50110/0 (socket says 172.21.15.78:50110) 2023-10-11T09:40:48.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.874+0000 7fc3754f7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc370164d20 0x7fc37015fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.874+0000 7f71b9a36700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71ac03b330 0x7f71ac0e1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57066/0 (socket says 172.21.15.78:57066) 2023-10-11T09:40:48.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.874+0000 7fc36ffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3700cda10 0x7fc3701fed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.874+0000 7fc36ffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3700cda10 0x7fc3701fed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:50116/0 (socket says 172.21.15.78:50116) 2023-10-11T09:40:48.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.874+0000 7f5c3e7fc700 1 -- 172.21.15.78:0/910380996 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f5c38013cd0 con 0x7f5c400de730 2023-10-11T09:40:48.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.874+0000 7f5c3e7fc700 1 --2- 172.21.15.78:0/910380996 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5c2c065f20 0x7f5c2c0683d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.874+0000 7fc36f7fe700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc3700cdde0 0x7fc3701632d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.874+0000 7fc374cf6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc370164d20 0x7fc37015fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.874+0000 7fc36f7fe700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc3700cdde0 0x7fc3701632d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:44278/0 (socket says 172.21.15.78:44278) 2023-10-11T09:40:48.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.874+0000 7fc374cf6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc370164d20 0x7fc37015fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57074/0 (socket says 172.21.15.78:57074) 2023-10-11T09:40:48.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.874+0000 7fc36ffff700 1 -- 172.21.15.78:0/4020301898 learned_addr learned my addr 172.21.15.78:0/4020301898 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:48.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.874+0000 7fc3754f7700 1 -- 172.21.15.78:0/4020301898 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3701ff240 con 0x7fc370164d20 2023-10-11T09:40:48.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.874+0000 7fc3754f7700 1 -- 172.21.15.78:0/4020301898 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc370202080 con 0x7fc3700cda10 2023-10-11T09:40:48.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.874+0000 7fc3754f7700 1 -- 172.21.15.78:0/4020301898 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc370146ee0 con 0x7fc3700cdde0 2023-10-11T09:40:48.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.878+0000 7fa0a2ffd700 1 -- 172.21.15.78:0/195245663 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fa0a8052d80 con 0x7fa0a4006960 2023-10-11T09:40:48.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.878+0000 7fc36f7fe700 1 -- 172.21.15.78:0/4020301898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3700cda10 msgr2=0x7fc3701fed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.878+0000 7fc36f7fe700 1 --2- 172.21.15.78:0/4020301898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3700cda10 0x7fc3701fed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.878+0000 7fc36f7fe700 1 -- 172.21.15.78:0/4020301898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc370164d20 msgr2=0x7fc37015fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.878+0000 7fc36f7fe700 1 --2- 172.21.15.78:0/4020301898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc370164d20 0x7fc37015fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.878+0000 7fc36f7fe700 1 -- 172.21.15.78:0/4020301898 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3680094b0 con 0x7fc3700cdde0 2023-10-11T09:40:48.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.878+0000 7f5c3e7fc700 1 -- 172.21.15.78:0/910380996 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5c38084100 con 0x7f5c400de730 2023-10-11T09:40:48.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.878+0000 7f5c4dd27700 1 --2- 172.21.15.78:0/910380996 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5c2c065f20 0x7f5c2c0683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.878+0000 7fc36f7fe700 1 --2- 172.21.15.78:0/4020301898 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc3700cdde0 0x7fc3701632d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc36000cb10 tx=0x7fc36000cb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.878+0000 7f71ba237700 1 -- 172.21.15.78:0/3618472622 learned_addr learned my addr 172.21.15.78:0/3618472622 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:48.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.878+0000 7f71bb239700 1 -- 172.21.15.78:0/3618472622 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac0dccb0 con 0x7f71ac03b330 2023-10-11T09:40:48.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.878+0000 7f71bb239700 1 -- 172.21.15.78:0/3618472622 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac052e00 con 0x7f71ac038560 2023-10-11T09:40:48.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.878+0000 7f71bb239700 1 -- 172.21.15.78:0/3618472622 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac180430 con 0x7f71ac052440 2023-10-11T09:40:48.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.878+0000 7fc36cff9700 1 -- 172.21.15.78:0/4020301898 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36000a770 con 0x7fc3700cdde0 2023-10-11T09:40:48.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.878+0000 7fc36cff9700 1 -- 172.21.15.78:0/4020301898 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc36000a8d0 con 0x7fc3700cdde0 2023-10-11T09:40:48.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.878+0000 7fc3754f7700 1 -- 172.21.15.78:0/4020301898 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3700ce3b0 con 0x7fc3700cdde0 2023-10-11T09:40:48.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.878+0000 7fc3754f7700 1 -- 172.21.15.78:0/4020301898 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3701474a0 con 0x7fc3700cdde0 2023-10-11T09:40:48.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.878+0000 7fc36cff9700 1 -- 172.21.15.78:0/4020301898 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36000ac40 con 0x7fc3700cdde0 2023-10-11T09:40:48.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.878+0000 7f5c4dd27700 1 --2- 172.21.15.78:0/910380996 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5c2c065f20 0x7f5c2c0683d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5c48069ad0 tx=0x7f5c48071000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.878+0000 7f71b9a36700 1 -- 172.21.15.78:0/3618472622 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f71ac052440 msgr2=0x7f71ac0528c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.878+0000 7f71b9a36700 1 --2- 172.21.15.78:0/3618472622 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f71ac052440 0x7f71ac0528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.878+0000 7f71b9a36700 1 -- 172.21.15.78:0/3618472622 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71ac038560 msgr2=0x7f71ac0dc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:48.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.878+0000 7f71b9a36700 1 --2- 172.21.15.78:0/3618472622 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71ac038560 0x7f71ac0dc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:48.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.878+0000 7f71b9a36700 1 -- 172.21.15.78:0/3618472622 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b0009570 con 0x7f71ac03b330 2023-10-11T09:40:48.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.878+0000 7f71b9a36700 1 --2- 172.21.15.78:0/3618472622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71ac03b330 0x7f71ac0e1b50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f71a400b470 tx=0x7f71a400b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.878+0000 7f71aaffd700 1 -- 172.21.15.78:0/3618472622 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a4011830 con 0x7f71ac03b330 2023-10-11T09:40:48.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.878+0000 7f71aaffd700 1 -- 172.21.15.78:0/3618472622 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a4011990 con 0x7f71ac03b330 2023-10-11T09:40:48.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.878+0000 7f71aaffd700 1 -- 172.21.15.78:0/3618472622 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a4011d00 con 0x7f71ac03b330 2023-10-11T09:40:48.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.878+0000 7fc3754f7700 1 -- 172.21.15.78:0/4020301898 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc370065d40 con 0x7fc3700cdde0 2023-10-11T09:40:48.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.882+0000 7f71bb239700 1 -- 172.21.15.78:0/3618472622 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71ac180710 con 0x7f71ac03b330 2023-10-11T09:40:48.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.882+0000 7f71bb239700 1 -- 172.21.15.78:0/3618472622 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71ac180cd0 con 0x7f71ac03b330 2023-10-11T09:40:48.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.882+0000 7fc36cff9700 1 -- 172.21.15.78:0/4020301898 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fc36001b3b0 con 0x7fc3700cdde0 2023-10-11T09:40:48.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.882+0000 7fc36cff9700 1 --2- 172.21.15.78:0/4020301898 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc350065ed0 0x7fc350068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:48.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.882+0000 7fc36cff9700 1 -- 172.21.15.78:0/4020301898 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc360083020 con 0x7fc3700cdde0 2023-10-11T09:40:48.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.882+0000 7fc36ffff700 1 --2- 172.21.15.78:0/4020301898 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc350065ed0 0x7fc350068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:48.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.882+0000 7fc36ffff700 1 --2- 172.21.15.78:0/4020301898 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc350065ed0 0x7fc350068380 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc3680097d0 tx=0x7fc36801c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:48.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.882+0000 7f71bb239700 1 -- 172.21.15.78:0/3618472622 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71ac005530 con 0x7f71ac03b330 2023-10-11T09:40:48.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.882+0000 7f71aaffd700 1 -- 172.21.15.78:0/3618472622 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f71a401a400 con 0x7f71ac03b330 2023-10-11T09:40:48.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.886+0000 7f5c3e7fc700 1 -- 172.21.15.78:0/910380996 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f5c38051730 con 0x7f5c400de730 2023-10-11T09:40:49.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.886+0000 7f71aaffd700 1 --2- 172.21.15.78:0/3618472622 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7198065ed0 0x7f7198068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:49.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.886+0000 7f71ba237700 1 --2- 172.21.15.78:0/3618472622 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7198065ed0 0x7f7198068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:49.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.886+0000 7f71aaffd700 1 -- 172.21.15.78:0/3618472622 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f71a4051090 con 0x7f71ac03b330 2023-10-11T09:40:49.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.886+0000 7f71ba237700 1 --2- 172.21.15.78:0/3618472622 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7198065ed0 0x7f7198068380 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f71b0009fd0 tx=0x7f71b0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:49.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.890+0000 7fc36cff9700 1 -- 172.21.15.78:0/4020301898 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fc3600b3fa0 con 0x7fc3700cdde0 2023-10-11T09:40:49.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:48.898+0000 7f71aaffd700 1 -- 172.21.15.78:0/3618472622 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f71a4015080 con 0x7f71ac03b330 2023-10-11T09:40:49.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.166+0000 7fdec54bf700 1 -- 172.21.15.78:0/1848472170 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fdec00cda90 con 0x7fdec00b2590 2023-10-11T09:40:49.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.166+0000 7fdea77fe700 1 -- 172.21.15.78:0/1848472170 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdeb0054dc0 con 0x7fdec00b2590 2023-10-11T09:40:49.172 INFO:teuthology.orchestra.run.smithi078.stdout:55834574852 2023-10-11T09:40:49.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.170+0000 7fdea57fa700 1 -- 172.21.15.78:0/1848472170 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdeac065f20 msgr2=0x7fdeac0683d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:49.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.170+0000 7fdea57fa700 1 --2- 172.21.15.78:0/1848472170 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdeac065f20 0x7fdeac0683d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40090b0 tx=0x7fdeb4008030 comp rx=0 tx=0).stop 2023-10-11T09:40:49.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.170+0000 7fdea57fa700 1 -- 172.21.15.78:0/1848472170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdec00b2590 msgr2=0x7fdec00c7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:49.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.170+0000 7fdea57fa700 1 --2- 172.21.15.78:0/1848472170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdec00b2590 0x7fdec00c7e10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdeb000e760 tx=0x7fdeb000ea70 comp rx=0 tx=0).stop 2023-10-11T09:40:49.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.170+0000 7fdea57fa700 1 -- 172.21.15.78:0/1848472170 shutdown_connections 2023-10-11T09:40:49.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.170+0000 7fdea57fa700 1 --2- 172.21.15.78:0/1848472170 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdeac065f20 0x7fdeac0683d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:49.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.170+0000 7fdea57fa700 1 --2- 172.21.15.78:0/1848472170 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdec00ce560 0x7fdec01fe2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:49.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.170+0000 7fdea57fa700 1 --2- 172.21.15.78:0/1848472170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdec00c6f30 0x7fdec01fdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:49.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.170+0000 7fdea57fa700 1 --2- 172.21.15.78:0/1848472170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdec00b2590 0x7fdec00c7e10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:49.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.170+0000 7fdea57fa700 1 -- 172.21.15.78:0/1848472170 >> 172.21.15.78:0/1848472170 conn(0x7fdec006dd80 msgr2=0x7fdec006a060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:49.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.170+0000 7fdea57fa700 1 -- 172.21.15.78:0/1848472170 shutdown_connections 2023-10-11T09:40:49.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.170+0000 7fdea57fa700 1 -- 172.21.15.78:0/1848472170 wait complete. 2023-10-11T09:40:49.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.190+0000 7f498371f700 1 -- 172.21.15.78:0/2970240399 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f497c065d40 con 0x7f497c0cda10 2023-10-11T09:40:49.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.194+0000 7f496dffb700 1 -- 172.21.15.78:0/2970240399 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4978053f20 con 0x7f497c0cda10 2023-10-11T09:40:49.198 INFO:teuthology.orchestra.run.smithi078.stdout:55834574854 2023-10-11T09:40:49.199 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.3 2023-10-11T09:40:49.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.194+0000 7f7aea04a700 1 -- 172.21.15.78:0/539272325 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f7adc057c20 con 0x7f7adc038430 2023-10-11T09:40:49.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.194+0000 7f7acffff700 1 -- 172.21.15.78:0/539272325 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7ad8052720 con 0x7f7adc038430 2023-10-11T09:40:49.201 INFO:teuthology.orchestra.run.smithi078.stdout:55834574854 2023-10-11T09:40:49.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.198+0000 7f498371f700 1 -- 172.21.15.78:0/2970240399 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4960065ed0 msgr2=0x7f4960068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:49.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.198+0000 7f498371f700 1 --2- 172.21.15.78:0/2970240399 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4960065ed0 0x7f4960068380 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f497c0c9180 tx=0x7f4970007ee0 comp rx=0 tx=0).stop 2023-10-11T09:40:49.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.198+0000 7f498371f700 1 -- 172.21.15.78:0/2970240399 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f497c0cda10 msgr2=0x7f497c146e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:49.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.198+0000 7f498371f700 1 --2- 172.21.15.78:0/2970240399 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f497c0cda10 0x7f497c146e40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f497800e9d0 tx=0x7f497800ece0 comp rx=0 tx=0).stop 2023-10-11T09:40:49.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.198+0000 7f498371f700 1 -- 172.21.15.78:0/2970240399 shutdown_connections 2023-10-11T09:40:49.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.198+0000 7f498371f700 1 --2- 172.21.15.78:0/2970240399 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4960065ed0 0x7f4960068380 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:49.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.198+0000 7f498371f700 1 --2- 172.21.15.78:0/2970240399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f497c164d20 0x7f497c06bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:49.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.198+0000 7f498371f700 1 --2- 172.21.15.78:0/2970240399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f497c0cdde0 0x7f497c162770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:49.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.198+0000 7f498371f700 1 --2- 172.21.15.78:0/2970240399 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f497c0cda10 0x7f497c146e40 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:49.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.198+0000 7f498371f700 1 -- 172.21.15.78:0/2970240399 >> 172.21.15.78:0/2970240399 conn(0x7f497c0cea60 msgr2=0x7f497c0cee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:49.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.198+0000 7f364cb1f700 1 -- 172.21.15.78:0/1080473021 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3648065d40 con 0x7f36480b0be0 2023-10-11T09:40:49.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.198+0000 7f498371f700 1 -- 172.21.15.78:0/2970240399 shutdown_connections 2023-10-11T09:40:49.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.198+0000 7f498371f700 1 -- 172.21.15.78:0/2970240399 wait complete. 2023-10-11T09:40:49.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.198+0000 7f3636ffd700 1 -- 172.21.15.78:0/1080473021 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3640017020 con 0x7f36480b0be0 2023-10-11T09:40:49.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.198+0000 7f7aea04a700 1 -- 172.21.15.78:0/539272325 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7ac8065f20 msgr2=0x7f7ac80683d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:49.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.198+0000 7f7aea04a700 1 --2- 172.21.15.78:0/539272325 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7ac8065f20 0x7f7ac80683d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7adc0cf950 tx=0x7f7ad0009040 comp rx=0 tx=0).stop 2023-10-11T09:40:49.207 INFO:teuthology.orchestra.run.smithi078.stdout:55834574852 2023-10-11T09:40:49.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.198+0000 7f7aea04a700 1 -- 172.21.15.78:0/539272325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7adc038430 msgr2=0x7f7adc17c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:49.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.198+0000 7f7aea04a700 1 --2- 172.21.15.78:0/539272325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7adc038430 0x7f7adc17c120 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8004000 tx=0x7f7ad8004770 comp rx=0 tx=0).stop 2023-10-11T09:40:49.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.198+0000 7f7aea04a700 1 -- 172.21.15.78:0/539272325 shutdown_connections 2023-10-11T09:40:49.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.198+0000 7f7aea04a700 1 --2- 172.21.15.78:0/539272325 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7ac8065f20 0x7f7ac80683d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:49.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.198+0000 7f7aea04a700 1 --2- 172.21.15.78:0/539272325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7adc055690 0x7f7adc0dc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:49.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.198+0000 7f7aea04a700 1 --2- 172.21.15.78:0/539272325 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7adc03b250 0x7f7adc0da080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:49.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.198+0000 7f7aea04a700 1 --2- 172.21.15.78:0/539272325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7adc038430 0x7f7adc17c120 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:49.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.198+0000 7f7aea04a700 1 -- 172.21.15.78:0/539272325 >> 172.21.15.78:0/539272325 conn(0x7f7adc055b40 msgr2=0x7f7adc058e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:49.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.202+0000 7f7aea04a700 1 -- 172.21.15.78:0/539272325 shutdown_connections 2023-10-11T09:40:49.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.202+0000 7f7aea04a700 1 -- 172.21.15.78:0/539272325 wait complete. 2023-10-11T09:40:49.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.202+0000 7f364cb1f700 1 -- 172.21.15.78:0/1080473021 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3628065fa0 msgr2=0x7f3628068450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:49.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.202+0000 7f364cb1f700 1 --2- 172.21.15.78:0/1080473021 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3628065fa0 0x7f3628068450 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f363c000c00 tx=0x7f363c008030 comp rx=0 tx=0).stop 2023-10-11T09:40:49.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.202+0000 7f364cb1f700 1 -- 172.21.15.78:0/1080473021 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36480b0be0 msgr2=0x7f364813a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:49.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.202+0000 7f364cb1f700 1 --2- 172.21.15.78:0/1080473021 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36480b0be0 0x7f364813a2b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f364000cc50 tx=0x7f364000cf60 comp rx=0 tx=0).stop 2023-10-11T09:40:49.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.202+0000 7f364cb1f700 1 -- 172.21.15.78:0/1080473021 shutdown_connections 2023-10-11T09:40:49.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.202+0000 7f364cb1f700 1 --2- 172.21.15.78:0/1080473021 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3628065fa0 0x7f3628068450 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:49.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.202+0000 7f364cb1f700 1 --2- 172.21.15.78:0/1080473021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f364815b7f0 0x7f36480ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:49.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.202+0000 7f364cb1f700 1 --2- 172.21.15.78:0/1080473021 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f364815a990 0x7f36480cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:49.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.202+0000 7f364cb1f700 1 --2- 172.21.15.78:0/1080473021 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36480b0be0 0x7f364813a2b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:49.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.202+0000 7f364cb1f700 1 -- 172.21.15.78:0/1080473021 >> 172.21.15.78:0/1080473021 conn(0x7f364806aca0 msgr2=0x7f3648070b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:49.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.202+0000 7f364cb1f700 1 -- 172.21.15.78:0/1080473021 shutdown_connections 2023-10-11T09:40:49.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.202+0000 7f364cb1f700 1 -- 172.21.15.78:0/1080473021 wait complete. 2023-10-11T09:40:49.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.210+0000 7f5c4ff8b700 1 -- 172.21.15.78:0/910380996 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5c400053f0 con 0x7f5c400de730 2023-10-11T09:40:49.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.210+0000 7f5c3e7fc700 1 -- 172.21.15.78:0/910380996 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5c38017090 con 0x7f5c400de730 2023-10-11T09:40:49.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.210+0000 7fa0b31b7700 1 -- 172.21.15.78:0/195245663 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fa0940055c0 con 0x7fa0a4006960 2023-10-11T09:40:49.216 INFO:teuthology.orchestra.run.smithi078.stdout:55834574853 2023-10-11T09:40:49.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.210+0000 7fa0a2ffd700 1 -- 172.21.15.78:0/195245663 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa0a80563a0 con 0x7fa0a4006960 2023-10-11T09:40:49.217 INFO:teuthology.orchestra.run.smithi078.stdout:55834574853 2023-10-11T09:40:49.218 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.214+0000 7fc3754f7700 1 -- 172.21.15.78:0/4020301898 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fc3700b1480 con 0x7fc3700cdde0 2023-10-11T09:40:49.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.214+0000 7f5c4ff8b700 1 -- 172.21.15.78:0/910380996 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5c2c065f20 msgr2=0x7f5c2c0683d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:49.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.214+0000 7f5c4ff8b700 1 --2- 172.21.15.78:0/910380996 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5c2c065f20 0x7f5c2c0683d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5c48069ad0 tx=0x7f5c48071000 comp rx=0 tx=0).stop 2023-10-11T09:40:49.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.214+0000 7f5c4ff8b700 1 -- 172.21.15.78:0/910380996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c400de730 msgr2=0x7f5c400499c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:49.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.214+0000 7f5c4ff8b700 1 --2- 172.21.15.78:0/910380996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c400de730 0x7f5c400499c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5c3800e870 tx=0x7f5c3800eb80 comp rx=0 tx=0).stop 2023-10-11T09:40:49.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.214+0000 7f5c4ff8b700 1 -- 172.21.15.78:0/910380996 shutdown_connections 2023-10-11T09:40:49.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.214+0000 7f5c4ff8b700 1 --2- 172.21.15.78:0/910380996 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5c2c065f20 0x7f5c2c0683d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:49.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.214+0000 7f5c4ff8b700 1 --2- 172.21.15.78:0/910380996 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c400deb00 0x7f5c40049f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:49.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.214+0000 7f5c4ff8b700 1 --2- 172.21.15.78:0/910380996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c400de730 0x7f5c400499c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:49.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.214+0000 7f5c4ff8b700 1 --2- 172.21.15.78:0/910380996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c4004f200 0x7f5c400d5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:49.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.214+0000 7f5c4ff8b700 1 -- 172.21.15.78:0/910380996 >> 172.21.15.78:0/910380996 conn(0x7f5c40051630 msgr2=0x7f5c400486e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:49.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.214+0000 7f5c4ff8b700 1 -- 172.21.15.78:0/910380996 shutdown_connections 2023-10-11T09:40:49.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.214+0000 7f5c4ff8b700 1 -- 172.21.15.78:0/910380996 wait complete. 2023-10-11T09:40:49.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.214+0000 7fc36cff9700 1 -- 172.21.15.78:0/4020301898 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc360017020 con 0x7fc3700cdde0 2023-10-11T09:40:49.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.214+0000 7fa0b31b7700 1 -- 172.21.15.78:0/195245663 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa090065f20 msgr2=0x7fa0900683d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:49.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.214+0000 7fa0b31b7700 1 --2- 172.21.15.78:0/195245663 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa090065f20 0x7fa0900683d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4048520 tx=0x7fa09c008040 comp rx=0 tx=0).stop 2023-10-11T09:40:49.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.214+0000 7fa0b31b7700 1 -- 172.21.15.78:0/195245663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0a4006960 msgr2=0x7fa0a40dccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:49.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.214+0000 7fa0b31b7700 1 --2- 172.21.15.78:0/195245663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0a4006960 0x7fa0a40dccc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8000c00 tx=0x7fa0a8004770 comp rx=0 tx=0).stop 2023-10-11T09:40:49.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.214+0000 7fa0b31b7700 1 -- 172.21.15.78:0/195245663 shutdown_connections 2023-10-11T09:40:49.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.214+0000 7fa0b31b7700 1 --2- 172.21.15.78:0/195245663 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa090065f20 0x7fa0900683d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:49.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.214+0000 7fa0b31b7700 1 --2- 172.21.15.78:0/195245663 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0a40dee50 0x7fa0a40d9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:49.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.214+0000 7fa0b31b7700 1 --2- 172.21.15.78:0/195245663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0a404f880 0x7fa0a4176a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:49.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.214+0000 7fa0b31b7700 1 --2- 172.21.15.78:0/195245663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0a4006960 0x7fa0a40dccc0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:49.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.214+0000 7fa0b31b7700 1 -- 172.21.15.78:0/195245663 >> 172.21.15.78:0/195245663 conn(0x7fa0a4051770 msgr2=0x7fa0a4033950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:49.225 INFO:teuthology.orchestra.run.smithi078.stdout:55834574853 2023-10-11T09:40:49.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.214+0000 7fa0b31b7700 1 -- 172.21.15.78:0/195245663 shutdown_connections 2023-10-11T09:40:49.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.214+0000 7fa0b31b7700 1 -- 172.21.15.78:0/195245663 wait complete. 2023-10-11T09:40:49.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.218+0000 7fc3754f7700 1 -- 172.21.15.78:0/4020301898 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc350065ed0 msgr2=0x7fc350068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:49.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.218+0000 7fc3754f7700 1 --2- 172.21.15.78:0/4020301898 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc350065ed0 0x7fc350068380 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc3680097d0 tx=0x7fc36801c040 comp rx=0 tx=0).stop 2023-10-11T09:40:49.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.218+0000 7fc3754f7700 1 -- 172.21.15.78:0/4020301898 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc3700cdde0 msgr2=0x7fc3701632d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:49.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.218+0000 7fc3754f7700 1 --2- 172.21.15.78:0/4020301898 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc3700cdde0 0x7fc3701632d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc36000cb10 tx=0x7fc36000cb40 comp rx=0 tx=0).stop 2023-10-11T09:40:49.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.222+0000 7fc3754f7700 1 -- 172.21.15.78:0/4020301898 shutdown_connections 2023-10-11T09:40:49.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.222+0000 7fc3754f7700 1 --2- 172.21.15.78:0/4020301898 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc350065ed0 0x7fc350068380 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:49.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.222+0000 7fc3754f7700 1 --2- 172.21.15.78:0/4020301898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc370164d20 0x7fc37015fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:49.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.222+0000 7fc3754f7700 1 --2- 172.21.15.78:0/4020301898 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc3700cdde0 0x7fc3701632d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:49.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.222+0000 7fc3754f7700 1 --2- 172.21.15.78:0/4020301898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3700cda10 0x7fc3701fed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:49.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.222+0000 7fc3754f7700 1 -- 172.21.15.78:0/4020301898 >> 172.21.15.78:0/4020301898 conn(0x7fc3700cea60 msgr2=0x7fc370145fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:49.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.222+0000 7fc3754f7700 1 -- 172.21.15.78:0/4020301898 shutdown_connections 2023-10-11T09:40:49.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.222+0000 7fc3754f7700 1 -- 172.21.15.78:0/4020301898 wait complete. 2023-10-11T09:40:49.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.226+0000 7f71bb239700 1 -- 172.21.15.78:0/3618472622 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f71ac03c500 con 0x7f71ac03b330 2023-10-11T09:40:49.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.226+0000 7f71aaffd700 1 -- 172.21.15.78:0/3618472622 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f71a4050cc0 con 0x7f71ac03b330 2023-10-11T09:40:49.231 INFO:teuthology.orchestra.run.smithi078.stdout:55834574853 2023-10-11T09:40:49.232 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574854 for osd.2 2023-10-11T09:40:49.233 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574854 for osd.7 2023-10-11T09:40:49.236 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.0 2023-10-11T09:40:49.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.230+0000 7f71bb239700 1 -- 172.21.15.78:0/3618472622 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7198065ed0 msgr2=0x7f7198068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:49.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.230+0000 7f71bb239700 1 --2- 172.21.15.78:0/3618472622 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7198065ed0 0x7f7198068380 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f71b0009fd0 tx=0x7f71b0008030 comp rx=0 tx=0).stop 2023-10-11T09:40:49.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.230+0000 7f71bb239700 1 -- 172.21.15.78:0/3618472622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71ac03b330 msgr2=0x7f71ac0e1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:49.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.230+0000 7f71bb239700 1 --2- 172.21.15.78:0/3618472622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71ac03b330 0x7f71ac0e1b50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f71a400b470 tx=0x7f71a400b780 comp rx=0 tx=0).stop 2023-10-11T09:40:49.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.230+0000 7f71bb239700 1 -- 172.21.15.78:0/3618472622 shutdown_connections 2023-10-11T09:40:49.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.230+0000 7f71bb239700 1 --2- 172.21.15.78:0/3618472622 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7198065ed0 0x7f7198068380 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:49.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.230+0000 7f71bb239700 1 --2- 172.21.15.78:0/3618472622 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f71ac052440 0x7f71ac0528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:49.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.230+0000 7f71bb239700 1 --2- 172.21.15.78:0/3618472622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71ac03b330 0x7f71ac0e1b50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:49.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.230+0000 7f71bb239700 1 --2- 172.21.15.78:0/3618472622 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71ac038560 0x7f71ac0dc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:49.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.230+0000 7f71bb239700 1 -- 172.21.15.78:0/3618472622 >> 172.21.15.78:0/3618472622 conn(0x7f71ac055c20 msgr2=0x7f71ac03a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:49.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.230+0000 7f71bb239700 1 -- 172.21.15.78:0/3618472622 shutdown_connections 2023-10-11T09:40:49.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:49.230+0000 7f71bb239700 1 -- 172.21.15.78:0/3618472622 wait complete. 2023-10-11T09:40:49.243 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574853 for osd.1 2023-10-11T09:40:49.245 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574853 for osd.6 2023-10-11T09:40:49.256 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574853 for osd.4 2023-10-11T09:40:49.260 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574853 for osd.5 2023-10-11T09:40:50.200 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-11T09:40:50.233 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-11T09:40:50.234 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-11T09:40:50.237 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-11T09:40:50.244 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-11T09:40:50.246 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-11T09:40:50.257 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-11T09:40:50.261 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-11T09:40:50.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.410+0000 7f2360dba700 1 Processor -- start 2023-10-11T09:40:50.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.410+0000 7f2360dba700 1 -- start start 2023-10-11T09:40:50.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.414+0000 7f2360dba700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f235c1586e0 0x7f235c06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.414+0000 7f2360dba700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f235c06d380 0x7f235c0b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.414+0000 7f2360dba700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f235c0b0fe0 con 0x7f235c0b0be0 2023-10-11T09:40:50.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.414+0000 7f2360dba700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f235c06c440 con 0x7f235c1586e0 2023-10-11T09:40:50.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.414+0000 7f2360dba700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f235c06c720 con 0x7f235c06d380 2023-10-11T09:40:50.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.414+0000 7f235a59c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f235c1586e0 0x7f235c06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.414+0000 7f2359d9b700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f235c06d380 0x7f235c0b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.414+0000 7f235a59c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f235c1586e0 0x7f235c06bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57084/0 (socket says 172.21.15.78:57084) 2023-10-11T09:40:50.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.414+0000 7f235a59c700 1 -- 172.21.15.78:0/142549911 learned_addr learned my addr 172.21.15.78:0/142549911 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:50.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.414+0000 7f235a59c700 1 -- 172.21.15.78:0/142549911 >> v2:172.21.15.133:3300/0 conn(0x7f235c06d380 msgr2=0x7f235c0b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:50.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.414+0000 7f235a59c700 1 --2- 172.21.15.78:0/142549911 >> v2:172.21.15.133:3300/0 conn(0x7f235c06d380 0x7f235c0b06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.414+0000 7f235a59c700 1 -- 172.21.15.78:0/142549911 >> v1:172.21.15.78:6789/0 conn(0x7f235c0b0be0 legacy=0x7f235c15c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:50.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.414+0000 7f235a59c700 1 -- 172.21.15.78:0/142549911 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235c06ca00 con 0x7f235c1586e0 2023-10-11T09:40:50.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.414+0000 7f235a59c700 1 --2- 172.21.15.78:0/142549911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f235c1586e0 0x7f235c06bbf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f235c0b13f0 tx=0x7f234c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed858b4dfee6f452 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:50.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.418+0000 7f235959a700 1 -- 172.21.15.78:0/142549911 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234c004230 con 0x7f235c1586e0 2023-10-11T09:40:50.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.418+0000 7f235959a700 1 -- 172.21.15.78:0/142549911 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f234c004390 con 0x7f235c1586e0 2023-10-11T09:40:50.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.418+0000 7f235959a700 1 -- 172.21.15.78:0/142549911 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234c00c700 con 0x7f235c1586e0 2023-10-11T09:40:50.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.418+0000 7f2360dba700 1 -- 172.21.15.78:0/142549911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f235c1586e0 msgr2=0x7f235c06bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.418+0000 7f2360dba700 1 --2- 172.21.15.78:0/142549911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f235c1586e0 0x7f235c06bbf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f235c0b13f0 tx=0x7f234c009f90 comp rx=0 tx=0).stop 2023-10-11T09:40:50.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.418+0000 7f2360dba700 1 -- 172.21.15.78:0/142549911 shutdown_connections 2023-10-11T09:40:50.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.418+0000 7f2360dba700 1 --2- 172.21.15.78:0/142549911 >> v2:172.21.15.133:3300/0 conn(0x7f235c06d380 0x7f235c0b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.418+0000 7f2360dba700 1 --2- 172.21.15.78:0/142549911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f235c1586e0 0x7f235c06bbf0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.418+0000 7f2360dba700 1 -- 172.21.15.78:0/142549911 >> 172.21.15.78:0/142549911 conn(0x7f235c06aca0 msgr2=0x7f235c0cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:50.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.418+0000 7f2360dba700 1 -- 172.21.15.78:0/142549911 shutdown_connections 2023-10-11T09:40:50.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.418+0000 7f2360dba700 1 -- 172.21.15.78:0/142549911 wait complete. 2023-10-11T09:40:50.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.422+0000 7f2360dba700 1 Processor -- start 2023-10-11T09:40:50.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.422+0000 7f2360dba700 1 -- start start 2023-10-11T09:40:50.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.422+0000 7f2360dba700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f235c06d380 0x7f235c13f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.422+0000 7f2360dba700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f235c0b0be0 0x7f235c147fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.422+0000 7f2360dba700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f235c1586e0 0x7f235c0af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.422+0000 7f2360dba700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c0b3380 con 0x7f235c0b0be0 2023-10-11T09:40:50.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.422+0000 7f2360dba700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c0b2420 con 0x7f235c06d380 2023-10-11T09:40:50.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.422+0000 7f2360dba700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f235c0b2700 con 0x7f235c1586e0 2023-10-11T09:40:50.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.422+0000 7f2359d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f235c0b0be0 0x7f235c147fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.422+0000 7f2359d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f235c0b0be0 0x7f235c147fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57088/0 (socket says 172.21.15.78:57088) 2023-10-11T09:40:50.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.422+0000 7f2359d9b700 1 -- 172.21.15.78:0/1924996578 learned_addr learned my addr 172.21.15.78:0/1924996578 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:50.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.422+0000 7f235a59c700 1 --2- 172.21.15.78:0/1924996578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f235c06d380 0x7f235c13f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.422+0000 7f235ad9d700 1 --2- 172.21.15.78:0/1924996578 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f235c1586e0 0x7f235c0af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.426+0000 7f235a59c700 1 -- 172.21.15.78:0/1924996578 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f235c1586e0 msgr2=0x7f235c0af2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.426+0000 7f235a59c700 1 --2- 172.21.15.78:0/1924996578 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f235c1586e0 0x7f235c0af2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.426+0000 7f235a59c700 1 -- 172.21.15.78:0/1924996578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f235c0b0be0 msgr2=0x7f235c147fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.426+0000 7f235a59c700 1 --2- 172.21.15.78:0/1924996578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f235c0b0be0 0x7f235c147fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.426+0000 7f235a59c700 1 -- 172.21.15.78:0/1924996578 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f234c009e00 con 0x7f235c06d380 2023-10-11T09:40:50.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.426+0000 7f235a59c700 1 --2- 172.21.15.78:0/1924996578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f235c06d380 0x7f235c13f690 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f234c016000 tx=0x7f234c004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:50.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.426+0000 7f234affd700 1 -- 172.21.15.78:0/1924996578 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234c01e070 con 0x7f235c06d380 2023-10-11T09:40:50.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.426+0000 7f2360dba700 1 -- 172.21.15.78:0/1924996578 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f235c0b29e0 con 0x7f235c06d380 2023-10-11T09:40:50.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.426+0000 7f2360dba700 1 -- 172.21.15.78:0/1924996578 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f235c0b2fa0 con 0x7f235c06d380 2023-10-11T09:40:50.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.426+0000 7f234affd700 1 -- 172.21.15.78:0/1924996578 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f234c004500 con 0x7f235c06d380 2023-10-11T09:40:50.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.426+0000 7f234affd700 1 -- 172.21.15.78:0/1924996578 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234c003ad0 con 0x7f235c06d380 2023-10-11T09:40:50.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.426+0000 7f2360dba700 1 -- 172.21.15.78:0/1924996578 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f235c065d40 con 0x7f235c06d380 2023-10-11T09:40:50.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.430+0000 7f234affd700 1 -- 172.21.15.78:0/1924996578 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f234c003c30 con 0x7f235c06d380 2023-10-11T09:40:50.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.434+0000 7f234affd700 1 --2- 172.21.15.78:0/1924996578 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f233c065fa0 0x7f233c068450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.434+0000 7f2359d9b700 1 --2- 172.21.15.78:0/1924996578 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f233c065fa0 0x7f233c068450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.434+0000 7f2359d9b700 1 --2- 172.21.15.78:0/1924996578 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f233c065fa0 0x7f233c068450 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f235c0c9630 tx=0x7f235400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:50.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.434+0000 7f234affd700 1 -- 172.21.15.78:0/1924996578 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f234c084f50 con 0x7f235c06d380 2023-10-11T09:40:50.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.438+0000 7f234affd700 1 -- 172.21.15.78:0/1924996578 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f234c0526c0 con 0x7f235c06d380 2023-10-11T09:40:50.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.462+0000 7f9c23019700 1 Processor -- start 2023-10-11T09:40:50.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.462+0000 7f9c23019700 1 -- start start 2023-10-11T09:40:50.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.466+0000 7f9c23019700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f9c1c1586e0 0x7f9c1c06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.466+0000 7f9c23019700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c1c06c440 con 0x7f9c1c0b0db0 2023-10-11T09:40:50.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.466+0000 7f9c23019700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c1c06c720 con 0x7f9c1c06d380 2023-10-11T09:40:50.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.466+0000 7f9c23019700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c1c06ca00 con 0x7f9c1c1586e0 2023-10-11T09:40:50.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.466+0000 7f9c20db5700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f9c1c1586e0 0x7f9c1c06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.466+0000 7f9c20db5700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f9c1c1586e0 0x7f9c1c06bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57102/0 (socket says 172.21.15.78:57102) 2023-10-11T09:40:50.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.466+0000 7f9c20db5700 1 -- 172.21.15.78:0/1217036489 learned_addr learned my addr 172.21.15.78:0/1217036489 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:50.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.466+0000 7f9c20db5700 1 -- 172.21.15.78:0/1217036489 >> v1:172.21.15.133:6789/0 conn(0x7f9c1c06d380 legacy=0x7f9c1c0b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.466+0000 7f9c20db5700 1 -- 172.21.15.78:0/1217036489 >> v1:172.21.15.78:6789/0 conn(0x7f9c1c0b0db0 legacy=0x7f9c1c070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.466+0000 7f9c20db5700 1 -- 172.21.15.78:0/1217036489 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c1c06fd10 con 0x7f9c1c1586e0 2023-10-11T09:40:50.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.466+0000 7f9c20db5700 1 --2- 172.21.15.78:0/1217036489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c1c1586e0 0x7f9c1c06bbf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9c10009200 tx=0x7f9c10009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c3a550ca26f3c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:50.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.470+0000 7f9c1b7fe700 1 -- 172.21.15.78:0/1217036489 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1000b900 con 0x7f9c1c1586e0 2023-10-11T09:40:50.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.470+0000 7f9c1b7fe700 1 -- 172.21.15.78:0/1217036489 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c1000ba60 con 0x7f9c1c1586e0 2023-10-11T09:40:50.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.470+0000 7f9c1b7fe700 1 -- 172.21.15.78:0/1217036489 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1000bdf0 con 0x7f9c1c1586e0 2023-10-11T09:40:50.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.470+0000 7f9c23019700 1 -- 172.21.15.78:0/1217036489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c1c1586e0 msgr2=0x7f9c1c06bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.470+0000 7f9c23019700 1 --2- 172.21.15.78:0/1217036489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c1c1586e0 0x7f9c1c06bbf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9c10009200 tx=0x7f9c10009f90 comp rx=0 tx=0).stop 2023-10-11T09:40:50.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.470+0000 7f9c23019700 1 -- 172.21.15.78:0/1217036489 shutdown_connections 2023-10-11T09:40:50.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.470+0000 7f9c23019700 1 --2- 172.21.15.78:0/1217036489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c1c1586e0 0x7f9c1c06bbf0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.470+0000 7f9c23019700 1 -- 172.21.15.78:0/1217036489 >> 172.21.15.78:0/1217036489 conn(0x7f9c1c06aca0 msgr2=0x7f9c1c0cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:50.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.470+0000 7f9c23019700 1 -- 172.21.15.78:0/1217036489 shutdown_connections 2023-10-11T09:40:50.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.470+0000 7f9c23019700 1 -- 172.21.15.78:0/1217036489 wait complete. 2023-10-11T09:40:50.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.474+0000 7f9c23019700 1 Processor -- start 2023-10-11T09:40:50.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.474+0000 7f9c23019700 1 -- start start 2023-10-11T09:40:50.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.474+0000 7f89793a7700 1 Processor -- start 2023-10-11T09:40:50.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.474+0000 7f89793a7700 1 -- start start 2023-10-11T09:40:50.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.474+0000 7f9c23019700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c1c06d380 0x7f9c1c158e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.474+0000 7f9c23019700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c1c0b0db0 0x7f9c1c0cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7f9c23019700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c1c1586e0 0x7f9c1c2022f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7f9c20db5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c1c06d380 0x7f9c1c158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7f9c20db5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c1c06d380 0x7f9c1c158e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:50154/0 (socket says 172.21.15.78:50154) 2023-10-11T09:40:50.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7f9c1bfff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c1c0b0db0 0x7f9c1c0cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7f9c1bfff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c1c0b0db0 0x7f9c1c0cc5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:44304/0 (socket says 172.21.15.78:44304) 2023-10-11T09:40:50.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7f9c215b6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c1c1586e0 0x7f9c1c2022f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7f9c215b6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c1c1586e0 0x7f9c1c2022f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57118/0 (socket says 172.21.15.78:57118) 2023-10-11T09:40:50.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7f9c20db5700 1 -- 172.21.15.78:0/1370849818 learned_addr learned my addr 172.21.15.78:0/1370849818 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:50.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7f9c23019700 1 -- 172.21.15.78:0/1370849818 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1c0b3c80 con 0x7f9c1c1586e0 2023-10-11T09:40:50.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7fbfd6526700 1 Processor -- start 2023-10-11T09:40:50.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7fbfd6526700 1 -- start start 2023-10-11T09:40:50.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7f9c23019700 1 -- 172.21.15.78:0/1370849818 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1c0b3e90 con 0x7f9c1c06d380 2023-10-11T09:40:50.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7f89793a7700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f896c037b40 0x7f896c0e4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7f89793a7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f896c0e4f50 0x7f896c036770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7f89793a7700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f896c0e0af0 con 0x7f896c0e12a0 2023-10-11T09:40:50.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7f89793a7700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f896c0e45f0 con 0x7f896c037b40 2023-10-11T09:40:50.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7f89793a7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f896c050f00 con 0x7f896c0e4f50 2023-10-11T09:40:50.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7f9c23019700 1 -- 172.21.15.78:0/1370849818 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1c0b4170 con 0x7f9c1c0b0db0 2023-10-11T09:40:50.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7f89727fc700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f896c037b40 0x7f896c0e4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7f8972ffd700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f896c0e12a0 0x7f896c0e1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:33858/0 (socket says 172.21.15.78:33858) 2023-10-11T09:40:50.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7f8972ffd700 1 -- 172.21.15.78:0/2721031615 learned_addr learned my addr 172.21.15.78:0/2721031615 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:50.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7fbfd6526700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbfc80e6e60 0x7fbfc80060c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7fbfd6526700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbfc80e7230 0x7fbfc8034040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7fbfd6526700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc8006ff0 con 0x7fbfc8036770 2023-10-11T09:40:50.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7fbfd6526700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfc8034580 con 0x7fbfc80e7230 2023-10-11T09:40:50.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7fbfd6526700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc8006750 con 0x7fbfc80e6e60 2023-10-11T09:40:50.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7f89737fe700 1 --2- 172.21.15.78:0/2721031615 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f896c0e4f50 0x7f896c036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7f9c20db5700 1 -- 172.21.15.78:0/1370849818 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c1c0b0db0 msgr2=0x7f9c1c0cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7f9c20db5700 1 --2- 172.21.15.78:0/1370849818 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c1c0b0db0 0x7f9c1c0cc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7f9c20db5700 1 -- 172.21.15.78:0/1370849818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c1c1586e0 msgr2=0x7f9c1c2022f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7f9c20db5700 1 --2- 172.21.15.78:0/1370849818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c1c1586e0 0x7f9c1c2022f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7f9c20db5700 1 -- 172.21.15.78:0/1370849818 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c10009e00 con 0x7f9c1c06d380 2023-10-11T09:40:50.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7f8971ffb700 1 -- 172.21.15.78:0/2721031615 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1146670398 0 0) 0x7f8964001110 con 0x7f896c0e12a0 2023-10-11T09:40:50.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7f8971ffb700 1 -- 172.21.15.78:0/2721031615 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f895c0035d0 con 0x7f896c0e12a0 2023-10-11T09:40:50.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7fbfcf7fe700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbfc80e7230 0x7fbfc8034040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7fbfcffff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbfc80e6e60 0x7fbfc80060c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7fbfcffff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbfc80e6e60 0x7fbfc80060c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:44330/0 (socket says 172.21.15.78:44330) 2023-10-11T09:40:50.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7fbfcf7fe700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbfc80e7230 0x7fbfc8034040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:50162/0 (socket says 172.21.15.78:50162) 2023-10-11T09:40:50.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7fbfcffff700 1 -- 172.21.15.78:0/3512261349 learned_addr learned my addr 172.21.15.78:0/3512261349 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:50.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7f89727fc700 1 -- 172.21.15.78:0/2721031615 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f896c0e4f50 msgr2=0x7f896c036770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:50.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7f89727fc700 1 --2- 172.21.15.78:0/2721031615 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f896c0e4f50 0x7f896c036770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7f89727fc700 1 -- 172.21.15.78:0/2721031615 >> v1:172.21.15.133:6789/0 conn(0x7f896c0e12a0 legacy=0x7f896c0e1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7f89727fc700 1 -- 172.21.15.78:0/2721031615 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f896c037fc0 con 0x7f896c037b40 2023-10-11T09:40:50.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.478+0000 7f9c20db5700 1 --2- 172.21.15.78:0/1370849818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c1c06d380 0x7f9c1c158e20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9c10009fd0 tx=0x7f9c10004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:50.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.482+0000 7fbfcffff700 1 -- 172.21.15.78:0/3512261349 >> v2:172.21.15.133:3300/0 conn(0x7fbfc80e7230 msgr2=0x7fbfc8034040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:50.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.482+0000 7fbfcffff700 1 --2- 172.21.15.78:0/3512261349 >> v2:172.21.15.133:3300/0 conn(0x7fbfc80e7230 0x7fbfc8034040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.482+0000 7f10c0d13700 1 Processor -- start 2023-10-11T09:40:50.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.482+0000 7fbfcffff700 1 -- 172.21.15.78:0/3512261349 >> v1:172.21.15.78:6789/0 conn(0x7fbfc8036770 legacy=0x7fbfc8036b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:50.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.482+0000 7fbfcffff700 1 -- 172.21.15.78:0/3512261349 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc8006a30 con 0x7fbfc80e6e60 2023-10-11T09:40:50.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.482+0000 7f9c197fa700 1 -- 172.21.15.78:0/1370849818 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c10003ad0 con 0x7f9c1c06d380 2023-10-11T09:40:50.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.482+0000 7f9c197fa700 1 -- 172.21.15.78:0/1370849818 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c10003c30 con 0x7f9c1c06d380 2023-10-11T09:40:50.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.482+0000 7f9c197fa700 1 -- 172.21.15.78:0/1370849818 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c10018400 con 0x7f9c1c06d380 2023-10-11T09:40:50.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.482+0000 7f89727fc700 1 --2- 172.21.15.78:0/2721031615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f896c037b40 0x7f896c0e4a10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8968009850 tx=0x7f8968009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec0ccfe94a1c189d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:50.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.482+0000 7f9c23019700 1 -- 172.21.15.78:0/1370849818 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c1c0b4450 con 0x7f9c1c06d380 2023-10-11T09:40:50.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.482+0000 7f9c23019700 1 -- 172.21.15.78:0/1370849818 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c1c0b4730 con 0x7f9c1c06d380 2023-10-11T09:40:50.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.482+0000 7f8971ffb700 1 -- 172.21.15.78:0/2721031615 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8968011830 con 0x7f896c037b40 2023-10-11T09:40:50.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.482+0000 7f8971ffb700 1 -- 172.21.15.78:0/2721031615 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8968011990 con 0x7f896c037b40 2023-10-11T09:40:50.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.482+0000 7f8971ffb700 1 -- 172.21.15.78:0/2721031615 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8968011d40 con 0x7f896c037b40 2023-10-11T09:40:50.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.482+0000 7f10c0d13700 1 -- start start 2023-10-11T09:40:50.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.482+0000 7f89793a7700 1 -- 172.21.15.78:0/2721031615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f896c037b40 msgr2=0x7f896c0e4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.482+0000 7f89793a7700 1 --2- 172.21.15.78:0/2721031615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f896c037b40 0x7f896c0e4a10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8968009850 tx=0x7f8968009b60 comp rx=0 tx=0).stop 2023-10-11T09:40:50.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.482+0000 7f9c23019700 1 -- 172.21.15.78:0/1370849818 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c1c15bc70 con 0x7f9c1c06d380 2023-10-11T09:40:50.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.482+0000 7f89793a7700 1 -- 172.21.15.78:0/2721031615 shutdown_connections 2023-10-11T09:40:50.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.482+0000 7f89793a7700 1 --2- 172.21.15.78:0/2721031615 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f896c0e4f50 0x7f896c036770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.482+0000 7f89793a7700 1 --2- 172.21.15.78:0/2721031615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f896c037b40 0x7f896c0e4a10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.482+0000 7f89793a7700 1 -- 172.21.15.78:0/2721031615 >> 172.21.15.78:0/2721031615 conn(0x7f896c051770 msgr2=0x7f896c051b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:50.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.482+0000 7f89793a7700 1 -- 172.21.15.78:0/2721031615 shutdown_connections 2023-10-11T09:40:50.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.482+0000 7f89793a7700 1 -- 172.21.15.78:0/2721031615 wait complete. 2023-10-11T09:40:50.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.482+0000 7fbfcffff700 1 --2- 172.21.15.78:0/3512261349 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbfc80e6e60 0x7fbfc80060c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc009910 tx=0x7fbfbc009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e51ce234021dbc2e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:50.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.482+0000 7fbfceffd700 1 -- 172.21.15.78:0/3512261349 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc004230 con 0x7fbfc80e6e60 2023-10-11T09:40:50.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.482+0000 7fbfceffd700 1 -- 172.21.15.78:0/3512261349 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfbc004390 con 0x7fbfc80e6e60 2023-10-11T09:40:50.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.482+0000 7fbfceffd700 1 -- 172.21.15.78:0/3512261349 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc0036c0 con 0x7fbfc80e6e60 2023-10-11T09:40:50.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.482+0000 7f10c0d13700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f10bc0b06a0 0x7f10bc0b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.482+0000 7f10c0d13700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10bc0b0fe0 con 0x7f10bc0cf170 2023-10-11T09:40:50.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.482+0000 7f10c0d13700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10bc06c720 con 0x7f10bc06d380 2023-10-11T09:40:50.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.482+0000 7f10c0d13700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10bc06ca00 con 0x7f10bc0b06a0 2023-10-11T09:40:50.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.482+0000 7fbfd6526700 1 -- 172.21.15.78:0/3512261349 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbfc80e6e60 msgr2=0x7fbfc80060c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.482+0000 7fbfd6526700 1 --2- 172.21.15.78:0/3512261349 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbfc80e6e60 0x7fbfc80060c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc009910 tx=0x7fbfbc009c20 comp rx=0 tx=0).stop 2023-10-11T09:40:50.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.486+0000 7f10b9d9b700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f10bc06d380 0x7f10bc158470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:33868/0 (socket says 172.21.15.78:33868) 2023-10-11T09:40:50.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.486+0000 7f10b9d9b700 1 -- 172.21.15.78:0/2351257343 learned_addr learned my addr 172.21.15.78:0/2351257343 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:50.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.486+0000 7fbfd6526700 1 -- 172.21.15.78:0/3512261349 shutdown_connections 2023-10-11T09:40:50.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.486+0000 7fbfd6526700 1 --2- 172.21.15.78:0/3512261349 >> v2:172.21.15.133:3300/0 conn(0x7fbfc80e7230 0x7fbfc8034040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.486+0000 7fbfd6526700 1 --2- 172.21.15.78:0/3512261349 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbfc80e6e60 0x7fbfc80060c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.486+0000 7fbfd6526700 1 -- 172.21.15.78:0/3512261349 >> 172.21.15.78:0/3512261349 conn(0x7fbfc8051770 msgr2=0x7fbfc8051b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:50.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.486+0000 7fbfd6526700 1 -- 172.21.15.78:0/3512261349 shutdown_connections 2023-10-11T09:40:50.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.486+0000 7f10bad9d700 1 --2- 172.21.15.78:0/2351257343 >> v2:172.21.15.133:3300/0 conn(0x7f10bc0b06a0 0x7f10bc0b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.486+0000 7f10bad9d700 1 -- 172.21.15.78:0/2351257343 >> v1:172.21.15.133:6789/0 conn(0x7f10bc06d380 legacy=0x7f10bc158470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.486+0000 7f10bad9d700 1 -- 172.21.15.78:0/2351257343 >> v1:172.21.15.78:6789/0 conn(0x7f10bc0cf170 legacy=0x7f10bc06bbf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-11T09:40:50.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.486+0000 7fbfd6526700 1 -- 172.21.15.78:0/3512261349 wait complete. 2023-10-11T09:40:50.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.486+0000 7f89793a7700 1 Processor -- start 2023-10-11T09:40:50.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.486+0000 7f10bad9d700 1 -- 172.21.15.78:0/2351257343 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10bc06fd10 con 0x7f10bc0b06a0 2023-10-11T09:40:50.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.486+0000 7f89793a7700 1 -- start start 2023-10-11T09:40:50.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.486+0000 7f10bad9d700 1 --2- 172.21.15.78:0/2351257343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10bc0b06a0 0x7f10bc0b0aa0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f10b0009850 tx=0x7f10b0009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4b1550299c63478 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:50.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.486+0000 7fbfd6526700 1 Processor -- start 2023-10-11T09:40:50.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.486+0000 7f9c197fa700 1 -- 172.21.15.78:0/1370849818 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f9c10018560 con 0x7f9c1c06d380 2023-10-11T09:40:50.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.486+0000 7f9c197fa700 1 --2- 172.21.15.78:0/1370849818 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9c08065fa0 0x7f9c08068450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.486+0000 7f9c197fa700 1 -- 172.21.15.78:0/1370849818 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9c10084fb0 con 0x7f9c1c06d380 2023-10-11T09:40:50.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.486+0000 7fbfd6526700 1 -- start start 2023-10-11T09:40:50.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.486+0000 7f0d6c792700 1 Processor -- start 2023-10-11T09:40:50.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.486+0000 7f9c1bfff700 1 --2- 172.21.15.78:0/1370849818 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9c08065fa0 0x7f9c08068450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7f0d6c792700 1 -- start start 2023-10-11T09:40:50.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7f9c1bfff700 1 --2- 172.21.15.78:0/1370849818 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9c08065fa0 0x7f9c08068450 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c159360 tx=0x7f9c0c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:50.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7fbfd6526700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfc8036770 0x7fbfc8048750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7fbfd6526700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfc80e6e60 0x7fbfc8039f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7f89793a7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f896c037b40 0x7f896c0448b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7f89793a7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f896c0e12a0 0x7f896c044df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7f0d6c792700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0d640cd030 0x7f0d640b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7fbfcffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfc8036770 0x7fbfc8048750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7fbfcffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfc8036770 0x7fbfc8048750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:50192/0 (socket says 172.21.15.78:50192) 2023-10-11T09:40:50.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7fbfcf7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfc80e6e60 0x7fbfc8039f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7fbfcf7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfc80e6e60 0x7fbfc8039f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57136/0 (socket says 172.21.15.78:57136) 2023-10-11T09:40:50.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7f0d6c792700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d640c9020 0x7f0d640b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7f0d6c792700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d640cc5a0 con 0x7f0d6415c6b0 2023-10-11T09:40:50.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7fbfcffff700 1 -- 172.21.15.78:0/476496036 learned_addr learned my addr 172.21.15.78:0/476496036 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:50.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7f0d6c792700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d6406d230 con 0x7f0d640cd030 2023-10-11T09:40:50.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7f0d6c792700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d640cc7f0 con 0x7f0d640c9020 2023-10-11T09:40:50.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7fbfd6526700 1 --2- 172.21.15.78:0/476496036 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbfc80e7230 0x7fbfc80e43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7f8972ffd700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f896c037b40 0x7f896c0448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7f8972ffd700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f896c037b40 0x7f896c0448b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:44342/0 (socket says 172.21.15.78:44342) 2023-10-11T09:40:50.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7f89727fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f896c0e12a0 0x7f896c044df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7f89727fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f896c0e12a0 0x7f896c044df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57152/0 (socket says 172.21.15.78:57152) 2023-10-11T09:40:50.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7fbfd6526700 1 -- 172.21.15.78:0/476496036 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc8048c90 con 0x7fbfc80e6e60 2023-10-11T09:40:50.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7fbfd6526700 1 -- 172.21.15.78:0/476496036 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc8048f30 con 0x7fbfc8036770 2023-10-11T09:40:50.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7fbfd6526700 1 -- 172.21.15.78:0/476496036 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc8049210 con 0x7fbfc80e7230 2023-10-11T09:40:50.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7f0d6a52e700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f0d6415c6b0 0x7f0d64160ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:58846/0 (socket says 172.21.15.78:58846) 2023-10-11T09:40:50.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7f0d6a52e700 1 -- 172.21.15.78:0/545058690 learned_addr learned my addr 172.21.15.78:0/545058690 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:50.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7f89793a7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f896c0e4f50 0x7f896c0425f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7f89793a7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f896c0d3770 con 0x7f896c0e12a0 2023-10-11T09:40:50.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7f89793a7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f896c0cdf20 con 0x7f896c0e4f50 2023-10-11T09:40:50.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7f89793a7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f896c0d3490 con 0x7f896c037b40 2023-10-11T09:40:50.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7f8972ffd700 1 -- 172.21.15.78:0/1286113761 learned_addr learned my addr 172.21.15.78:0/1286113761 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:50.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7fbfd4ac3700 1 --2- 172.21.15.78:0/476496036 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbfc80e7230 0x7fbfc80e43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7f0d6ad2f700 1 --2- 172.21.15.78:0/545058690 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d640c9020 0x7f0d640b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7f0d69d2d700 1 --2- 172.21.15.78:0/545058690 >> v2:172.21.15.133:3300/0 conn(0x7f0d640cd030 0x7f0d640b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7f10b959a700 1 -- 172.21.15.78:0/2351257343 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b0004230 con 0x7f10bc0b06a0 2023-10-11T09:40:50.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7f10b959a700 1 -- 172.21.15.78:0/2351257343 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b0004390 con 0x7f10bc0b06a0 2023-10-11T09:40:50.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7f10b959a700 1 -- 172.21.15.78:0/2351257343 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b00036c0 con 0x7f10bc0b06a0 2023-10-11T09:40:50.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7f89727fc700 1 -- 172.21.15.78:0/1286113761 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f896c037b40 msgr2=0x7f896c0448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7f89727fc700 1 --2- 172.21.15.78:0/1286113761 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f896c037b40 0x7f896c0448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7f89727fc700 1 -- 172.21.15.78:0/1286113761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f896c0e4f50 msgr2=0x7f896c0425f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-11T09:40:50.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7f89727fc700 1 --2- 172.21.15.78:0/1286113761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f896c0e4f50 0x7f896c0425f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7f89727fc700 1 -- 172.21.15.78:0/1286113761 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89680094b0 con 0x7f896c0e12a0 2023-10-11T09:40:50.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7f0d6ad2f700 1 -- 172.21.15.78:0/545058690 >> v2:172.21.15.133:3300/0 conn(0x7f0d640cd030 msgr2=0x7f0d640b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:50.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7f0d6ad2f700 1 --2- 172.21.15.78:0/545058690 >> v2:172.21.15.133:3300/0 conn(0x7f0d640cd030 0x7f0d640b06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7f0d6ad2f700 1 -- 172.21.15.78:0/545058690 >> v1:172.21.15.78:6789/0 conn(0x7f0d6415c6b0 legacy=0x7f0d64160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7f0d6952c700 1 -- 172.21.15.78:0/545058690 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4079835074 0 0) 0x7f0d640cc5a0 con 0x7f0d6415c6b0 2023-10-11T09:40:50.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.490+0000 7f0d6ad2f700 1 -- 172.21.15.78:0/545058690 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d640ccad0 con 0x7f0d640c9020 2023-10-11T09:40:50.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.494+0000 7f89727fc700 1 --2- 172.21.15.78:0/1286113761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f896c0e12a0 0x7f896c044df0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8968016000 tx=0x7f8968004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:50.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.494+0000 7f10c0d13700 1 -- 172.21.15.78:0/2351257343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10bc0b06a0 msgr2=0x7f10bc0b0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.494+0000 7f10c0d13700 1 --2- 172.21.15.78:0/2351257343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10bc0b06a0 0x7f10bc0b0aa0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f10b0009850 tx=0x7f10b0009b60 comp rx=0 tx=0).stop 2023-10-11T09:40:50.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.494+0000 7f9c197fa700 1 -- 172.21.15.78:0/1370849818 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f9c10052670 con 0x7f9c1c06d380 2023-10-11T09:40:50.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.494+0000 7f895b7fe700 1 -- 172.21.15.78:0/1286113761 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8968004590 con 0x7f896c0e12a0 2023-10-11T09:40:50.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.494+0000 7f895b7fe700 1 -- 172.21.15.78:0/1286113761 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f896801e070 con 0x7f896c0e12a0 2023-10-11T09:40:50.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.494+0000 7f895b7fe700 1 -- 172.21.15.78:0/1286113761 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89680038c0 con 0x7f896c0e12a0 2023-10-11T09:40:50.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.494+0000 7f89793a7700 1 -- 172.21.15.78:0/1286113761 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f896c053450 con 0x7f896c0e12a0 2023-10-11T09:40:50.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.494+0000 7f0d6ad2f700 1 --2- 172.21.15.78:0/545058690 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d640c9020 0x7f0d640b0be0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c009d90 tx=0x7f0d5c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4799da006d3ca46 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:50.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.494+0000 7f10c0d13700 1 -- 172.21.15.78:0/2351257343 shutdown_connections 2023-10-11T09:40:50.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.494+0000 7f10c0d13700 1 --2- 172.21.15.78:0/2351257343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10bc0b06a0 0x7f10bc0b0aa0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.494+0000 7f10c0d13700 1 -- 172.21.15.78:0/2351257343 >> 172.21.15.78:0/2351257343 conn(0x7f10bc06aca0 msgr2=0x7f10bc0cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:50.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.494+0000 7fbfcf7fe700 1 -- 172.21.15.78:0/476496036 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbfc80e7230 msgr2=0x7fbfc80e43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.494+0000 7fbfcf7fe700 1 --2- 172.21.15.78:0/476496036 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbfc80e7230 0x7fbfc80e43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.494+0000 7fbfcf7fe700 1 -- 172.21.15.78:0/476496036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfc8036770 msgr2=0x7fbfc8048750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.494+0000 7fbfcf7fe700 1 --2- 172.21.15.78:0/476496036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfc8036770 0x7fbfc8048750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.494+0000 7f0d6952c700 1 -- 172.21.15.78:0/545058690 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c00e070 con 0x7f0d640c9020 2023-10-11T09:40:50.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.494+0000 7fbfcf7fe700 1 -- 172.21.15.78:0/476496036 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfbc009570 con 0x7fbfc80e6e60 2023-10-11T09:40:50.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.494+0000 7f0d6952c700 1 -- 172.21.15.78:0/545058690 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d5c00bbd0 con 0x7f0d640c9020 2023-10-11T09:40:50.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.494+0000 7f0d6952c700 1 -- 172.21.15.78:0/545058690 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c012430 con 0x7f0d640c9020 2023-10-11T09:40:50.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.494+0000 7fbfcf7fe700 1 --2- 172.21.15.78:0/476496036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfc80e6e60 0x7fbfc8039f60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbfc400e870 tx=0x7fbfc400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:50.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.494+0000 7f0d6c792700 1 -- 172.21.15.78:0/545058690 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d640c9020 msgr2=0x7f0d640b0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.494+0000 7f0d6c792700 1 --2- 172.21.15.78:0/545058690 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d640c9020 0x7f0d640b0be0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c009d90 tx=0x7f0d5c00b330 comp rx=0 tx=0).stop 2023-10-11T09:40:50.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.494+0000 7f0d6c792700 1 -- 172.21.15.78:0/545058690 shutdown_connections 2023-10-11T09:40:50.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.494+0000 7f0d6c792700 1 --2- 172.21.15.78:0/545058690 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d640c9020 0x7f0d640b0be0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.494+0000 7f0d6c792700 1 --2- 172.21.15.78:0/545058690 >> v2:172.21.15.133:3300/0 conn(0x7f0d640cd030 0x7f0d640b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.494+0000 7f0d6c792700 1 -- 172.21.15.78:0/545058690 >> 172.21.15.78:0/545058690 conn(0x7f0d6406aca0 msgr2=0x7f0d640cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:50.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.494+0000 7f0d6c792700 1 -- 172.21.15.78:0/545058690 shutdown_connections 2023-10-11T09:40:50.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.494+0000 7f0d6c792700 1 -- 172.21.15.78:0/545058690 wait complete. 2023-10-11T09:40:50.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.494+0000 7f89793a7700 1 -- 172.21.15.78:0/1286113761 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f896c053a10 con 0x7f896c0e12a0 2023-10-11T09:40:50.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.494+0000 7fbfccff9700 1 -- 172.21.15.78:0/476496036 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4013680 con 0x7fbfc80e6e60 2023-10-11T09:40:50.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.494+0000 7fbfccff9700 1 -- 172.21.15.78:0/476496036 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc4013800 con 0x7fbfc80e6e60 2023-10-11T09:40:50.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.494+0000 7fbfccff9700 1 -- 172.21.15.78:0/476496036 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4013bb0 con 0x7fbfc80e6e60 2023-10-11T09:40:50.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.494+0000 7fbfd6526700 1 -- 172.21.15.78:0/476496036 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc80494f0 con 0x7fbfc80e6e60 2023-10-11T09:40:50.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.494+0000 7fbfd6526700 1 -- 172.21.15.78:0/476496036 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc804cf90 con 0x7fbfc80e6e60 2023-10-11T09:40:50.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.494+0000 7f10c0d13700 1 -- 172.21.15.78:0/2351257343 shutdown_connections 2023-10-11T09:40:50.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.494+0000 7f10c0d13700 1 -- 172.21.15.78:0/2351257343 wait complete. 2023-10-11T09:40:50.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.498+0000 7f89793a7700 1 -- 172.21.15.78:0/1286113761 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f896c0532b0 con 0x7f896c0e12a0 2023-10-11T09:40:50.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.498+0000 7fbfd6526700 1 -- 172.21.15.78:0/476496036 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfc804d180 con 0x7fbfc80e6e60 2023-10-11T09:40:50.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.498+0000 7f10c0d13700 1 Processor -- start 2023-10-11T09:40:50.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.498+0000 7f0d6c792700 1 Processor -- start 2023-10-11T09:40:50.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.498+0000 7f0d6c792700 1 -- start start 2023-10-11T09:40:50.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.498+0000 7f895b7fe700 1 -- 172.21.15.78:0/1286113761 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f8968003a20 con 0x7f896c0e12a0 2023-10-11T09:40:50.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.502+0000 7f0d6c792700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d640c9020 0x7f0d6414c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.502+0000 7f895b7fe700 1 --2- 172.21.15.78:0/1286113761 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8960065f20 0x7f89600683d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.502+0000 7f0d6c792700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d640cd030 0x7f0d6415eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.502+0000 7f0d6c792700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d6415c6b0 0x7f0d641514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.502+0000 7f0d6c792700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6415a5d0 con 0x7f0d640c9020 2023-10-11T09:40:50.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.502+0000 7f0d6c792700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6414cba0 con 0x7f0d6415c6b0 2023-10-11T09:40:50.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.502+0000 7f0d6c792700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6415f480 con 0x7f0d640cd030 2023-10-11T09:40:50.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.502+0000 7f8972ffd700 1 --2- 172.21.15.78:0/1286113761 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8960065f20 0x7f89600683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.502+0000 7f0d6ad2f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d6415c6b0 0x7f0d641514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.502+0000 7f0d6ad2f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d6415c6b0 0x7f0d641514a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:50218/0 (socket says 172.21.15.78:50218) 2023-10-11T09:40:50.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.502+0000 7f0d6ad2f700 1 -- 172.21.15.78:0/584555442 learned_addr learned my addr 172.21.15.78:0/584555442 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:50.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.502+0000 7f10c0d13700 1 -- start start 2023-10-11T09:40:50.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.506+0000 7f0d69d2d700 1 --2- 172.21.15.78:0/584555442 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d640cd030 0x7f0d6415eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.506+0000 7f0d6ad2f700 1 -- 172.21.15.78:0/584555442 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d640cd030 msgr2=0x7f0d6415eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.506+0000 7f0d6ad2f700 1 --2- 172.21.15.78:0/584555442 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d640cd030 0x7f0d6415eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.506+0000 7f0d6ad2f700 1 -- 172.21.15.78:0/584555442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d640c9020 msgr2=0x7f0d6414c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.506+0000 7f0d6ad2f700 1 --2- 172.21.15.78:0/584555442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d640c9020 0x7f0d6414c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.506+0000 7f0d6ad2f700 1 -- 172.21.15.78:0/584555442 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d5c009a70 con 0x7f0d6415c6b0 2023-10-11T09:40:50.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.506+0000 7f0d69d2d700 1 --2- 172.21.15.78:0/584555442 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d640cd030 0x7f0d6415eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:40:50.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.506+0000 7f0d6ad2f700 1 --2- 172.21.15.78:0/584555442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d6415c6b0 0x7f0d641514a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c014040 tx=0x7f0d5c004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:50.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.506+0000 7f10c0d13700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10bc06d380 0x7f10bc14fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.506+0000 7f10c0d13700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10bc0b06a0 0x7f10bc1445b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.506+0000 7f10c0d13700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10bc0cf170 0x7f10bc144af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.506+0000 7f10c0d13700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc14f9c0 con 0x7f10bc0cf170 2023-10-11T09:40:50.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.506+0000 7f10c0d13700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc1f7230 con 0x7f10bc06d380 2023-10-11T09:40:50.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.506+0000 7f10c0d13700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc14f370 con 0x7f10bc0b06a0 2023-10-11T09:40:50.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.506+0000 7f0d5affd700 1 -- 172.21.15.78:0/584555442 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c0048e0 con 0x7f0d6415c6b0 2023-10-11T09:40:50.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.506+0000 7f0d5affd700 1 -- 172.21.15.78:0/584555442 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d5c020070 con 0x7f0d6415c6b0 2023-10-11T09:40:50.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.506+0000 7f0d5affd700 1 -- 172.21.15.78:0/584555442 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c0276c0 con 0x7f0d6415c6b0 2023-10-11T09:40:50.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.506+0000 7f0d6c792700 1 -- 172.21.15.78:0/584555442 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d64159ca0 con 0x7f0d6415c6b0 2023-10-11T09:40:50.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.506+0000 7f0d6c792700 1 -- 172.21.15.78:0/584555442 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d6415a260 con 0x7f0d6415c6b0 2023-10-11T09:40:50.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.506+0000 7fbfccff9700 1 -- 172.21.15.78:0/476496036 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fbfc4013d10 con 0x7fbfc80e6e60 2023-10-11T09:40:50.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.506+0000 7fbfccff9700 1 --2- 172.21.15.78:0/476496036 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbfb4065f50 0x7fbfb4068400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.506+0000 7fbfccff9700 1 -- 172.21.15.78:0/476496036 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbfc40835b0 con 0x7fbfc80e6e60 2023-10-11T09:40:50.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.506+0000 7fbfcffff700 1 --2- 172.21.15.78:0/476496036 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbfb4065f50 0x7fbfb4068400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.506+0000 7fbfcffff700 1 --2- 172.21.15.78:0/476496036 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbfb4065f50 0x7fbfb4068400 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc009c60 tx=0x7fbfbc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:50.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.506+0000 7f10ba59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10bc06d380 0x7f10bc14fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.506+0000 7f10b9d9b700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10bc0b06a0 0x7f10bc1445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.506+0000 7f10ba59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10bc06d380 0x7f10bc14fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:50234/0 (socket says 172.21.15.78:50234) 2023-10-11T09:40:50.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.506+0000 7f10ba59c700 1 -- 172.21.15.78:0/20729074 learned_addr learned my addr 172.21.15.78:0/20729074 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:50.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.506+0000 7f10bad9d700 1 --2- 172.21.15.78:0/20729074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10bc0cf170 0x7f10bc144af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.506+0000 7f10b9d9b700 1 -- 172.21.15.78:0/20729074 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10bc06d380 msgr2=0x7f10bc14fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.506+0000 7f10b9d9b700 1 --2- 172.21.15.78:0/20729074 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10bc06d380 0x7f10bc14fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.506+0000 7f10b9d9b700 1 -- 172.21.15.78:0/20729074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10bc0cf170 msgr2=0x7f10bc144af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.506+0000 7f10b9d9b700 1 --2- 172.21.15.78:0/20729074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10bc0cf170 0x7f10bc144af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.506+0000 7f10b9d9b700 1 -- 172.21.15.78:0/20729074 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b00094b0 con 0x7f10bc0b06a0 2023-10-11T09:40:50.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.510+0000 7f0d6c792700 1 -- 172.21.15.78:0/584555442 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d4c005310 con 0x7f0d6415c6b0 2023-10-11T09:40:50.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.510+0000 7f895b7fe700 1 -- 172.21.15.78:0/1286113761 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f89680558a0 con 0x7f896c0e12a0 2023-10-11T09:40:50.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.510+0000 7f10b9d9b700 1 --2- 172.21.15.78:0/20729074 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10bc0b06a0 0x7f10bc1445b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f10b400caa0 tx=0x7f10b400cad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:50.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.510+0000 7f8972ffd700 1 --2- 172.21.15.78:0/1286113761 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8960065f20 0x7f89600683d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8964001370 tx=0x7f8964009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:50.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.510+0000 7f10aaffd700 1 -- 172.21.15.78:0/20729074 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b4007480 con 0x7f10bc0b06a0 2023-10-11T09:40:50.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.510+0000 7f10c0d13700 1 -- 172.21.15.78:0/20729074 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10bc14f650 con 0x7f10bc0b06a0 2023-10-11T09:40:50.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.510+0000 7f10aaffd700 1 -- 172.21.15.78:0/20729074 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b40075e0 con 0x7f10bc0b06a0 2023-10-11T09:40:50.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.510+0000 7f10aaffd700 1 -- 172.21.15.78:0/20729074 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b4007950 con 0x7f10bc0b06a0 2023-10-11T09:40:50.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.510+0000 7f10c0d13700 1 -- 172.21.15.78:0/20729074 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10bc14c600 con 0x7f10bc0b06a0 2023-10-11T09:40:50.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.510+0000 7f10c0d13700 1 -- 172.21.15.78:0/20729074 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10bc14f1b0 con 0x7f10bc0b06a0 2023-10-11T09:40:50.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.510+0000 7f895b7fe700 1 -- 172.21.15.78:0/1286113761 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f896c0532b0 con 0x7f896c0e12a0 2023-10-11T09:40:50.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.510+0000 7f0d5affd700 1 -- 172.21.15.78:0/584555442 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f0d5c012430 con 0x7f0d6415c6b0 2023-10-11T09:40:50.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.514+0000 7f10aaffd700 1 -- 172.21.15.78:0/20729074 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f10b401b3b0 con 0x7f10bc0b06a0 2023-10-11T09:40:50.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.514+0000 7f10aaffd700 1 --2- 172.21.15.78:0/20729074 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f109c065ed0 0x7f109c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.514+0000 7f10aaffd700 1 -- 172.21.15.78:0/20729074 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f10b4082fe0 con 0x7f10bc0b06a0 2023-10-11T09:40:50.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.514+0000 7f0d5affd700 1 --2- 172.21.15.78:0/584555442 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0d48065f20 0x7f0d480683d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.514+0000 7f0d5affd700 1 -- 172.21.15.78:0/584555442 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0d5c022070 con 0x7f0d6415c6b0 2023-10-11T09:40:50.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.514+0000 7f0d6a52e700 1 --2- 172.21.15.78:0/584555442 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0d48065f20 0x7f0d480683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.514+0000 7f0d6a52e700 1 --2- 172.21.15.78:0/584555442 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0d48065f20 0x7f0d480683d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0d60000c00 tx=0x7f0d60007a70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:50.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.518+0000 7f10ba59c700 1 --2- 172.21.15.78:0/20729074 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f109c065ed0 0x7f109c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.518+0000 7f10ba59c700 1 --2- 172.21.15.78:0/20729074 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f109c065ed0 0x7f109c068380 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f10ac001960 tx=0x7f10ac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:50.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.518+0000 7f0d5affd700 1 -- 172.21.15.78:0/584555442 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f0d5c053330 con 0x7f0d6415c6b0 2023-10-11T09:40:50.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.518+0000 7fbfccff9700 1 -- 172.21.15.78:0/476496036 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fbfc4050bf0 con 0x7fbfc80e6e60 2023-10-11T09:40:50.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.522+0000 7f10aaffd700 1 -- 172.21.15.78:0/20729074 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f10b4050640 con 0x7f10bc0b06a0 2023-10-11T09:40:50.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.526+0000 7fe17ddb3700 1 Processor -- start 2023-10-11T09:40:50.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.526+0000 7fe17ddb3700 1 -- start start 2023-10-11T09:40:50.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.530+0000 7fe17ddb3700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe17815c830 0x7fe1780d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.530+0000 7fe17ddb3700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1780b0930 con 0x7fe1780d2a30 2023-10-11T09:40:50.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.530+0000 7fe17ddb3700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1780b18a0 con 0x7fe1780b40a0 2023-10-11T09:40:50.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.530+0000 7fe17ddb3700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1780d3ec0 con 0x7fe17815c830 2023-10-11T09:40:50.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.530+0000 7fe1777fe700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe17815c830 0x7fe1780d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.530+0000 7fe1777fe700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe17815c830 0x7fe1780d0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:50238/0 (socket says 172.21.15.78:50238) 2023-10-11T09:40:50.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.530+0000 7fe1777fe700 1 -- 172.21.15.78:0/3144382697 learned_addr learned my addr 172.21.15.78:0/3144382697 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:50.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.530+0000 7fe1777fe700 1 -- 172.21.15.78:0/3144382697 >> v1:172.21.15.133:6789/0 conn(0x7fe1780b40a0 legacy=0x7fe1780b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.530+0000 7fe1777fe700 1 -- 172.21.15.78:0/3144382697 >> v1:172.21.15.78:6789/0 conn(0x7fe1780d2a30 legacy=0x7fe1780afb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.530+0000 7fe1777fe700 1 -- 172.21.15.78:0/3144382697 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1780b25b0 con 0x7fe17815c830 2023-10-11T09:40:50.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.534+0000 7fe1777fe700 1 --2- 172.21.15.78:0/3144382697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe17815c830 0x7fe1780d0ab0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe16c009890 tx=0x7fe16c009ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=148c8910dac3a8a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:50.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.534+0000 7fe1767fc700 1 -- 172.21.15.78:0/3144382697 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16c004230 con 0x7fe17815c830 2023-10-11T09:40:50.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.534+0000 7fe1767fc700 1 -- 172.21.15.78:0/3144382697 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe16c004390 con 0x7fe17815c830 2023-10-11T09:40:50.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.534+0000 7fe17ddb3700 1 -- 172.21.15.78:0/3144382697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe17815c830 msgr2=0x7fe1780d0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.534+0000 7fe17ddb3700 1 --2- 172.21.15.78:0/3144382697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe17815c830 0x7fe1780d0ab0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe16c009890 tx=0x7fe16c009ba0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.534+0000 7fe17ddb3700 1 -- 172.21.15.78:0/3144382697 shutdown_connections 2023-10-11T09:40:50.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.534+0000 7fe17ddb3700 1 --2- 172.21.15.78:0/3144382697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe17815c830 0x7fe1780d0ab0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.534+0000 7fe17ddb3700 1 -- 172.21.15.78:0/3144382697 >> 172.21.15.78:0/3144382697 conn(0x7fe17809c1b0 msgr2=0x7fe1780d2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:50.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.534+0000 7fe17ddb3700 1 -- 172.21.15.78:0/3144382697 shutdown_connections 2023-10-11T09:40:50.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.534+0000 7fe17ddb3700 1 -- 172.21.15.78:0/3144382697 wait complete. 2023-10-11T09:40:50.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.538+0000 7fe17ddb3700 1 Processor -- start 2023-10-11T09:40:50.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.538+0000 7fe17ddb3700 1 -- start start 2023-10-11T09:40:50.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.542+0000 7fe17ddb3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1780b40a0 0x7fe1780c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.542+0000 7fe17ddb3700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1780d2a30 0x7fe178201ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.542+0000 7fe17ddb3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe17815c830 0x7fe1781fdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.542+0000 7fe177fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe17815c830 0x7fe1781fdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.542+0000 7fe177fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe17815c830 0x7fe1781fdbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57176/0 (socket says 172.21.15.78:57176) 2023-10-11T09:40:50.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.542+0000 7fe1777fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1780b40a0 0x7fe1780c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.542+0000 7fe1777fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1780b40a0 0x7fe1780c9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:50254/0 (socket says 172.21.15.78:50254) 2023-10-11T09:40:50.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.542+0000 7fe176ffd700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1780d2a30 0x7fe178201ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.542+0000 7fe176ffd700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1780d2a30 0x7fe178201ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:44378/0 (socket says 172.21.15.78:44378) 2023-10-11T09:40:50.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.542+0000 7fe177fff700 1 -- 172.21.15.78:0/724862941 learned_addr learned my addr 172.21.15.78:0/724862941 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:50.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.542+0000 7fe17ddb3700 1 -- 172.21.15.78:0/724862941 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1780cc6a0 con 0x7fe17815c830 2023-10-11T09:40:50.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.542+0000 7fe17ddb3700 1 -- 172.21.15.78:0/724862941 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe178202850 con 0x7fe1780b40a0 2023-10-11T09:40:50.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.542+0000 7fe17ddb3700 1 -- 172.21.15.78:0/724862941 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1780cc940 con 0x7fe1780d2a30 2023-10-11T09:40:50.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.546+0000 7f80e30e0700 1 Processor -- start 2023-10-11T09:40:50.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.546+0000 7f80e30e0700 1 -- start start 2023-10-11T09:40:50.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.546+0000 7f80e30e0700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f80d4036e80 0x7f80d403a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.546+0000 7f80e30e0700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f80d403a640 0x7f80d40528c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.546+0000 7f80e30e0700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80d40060c0 con 0x7f80d4054800 2023-10-11T09:40:50.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.546+0000 7f80e30e0700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80d40070b0 con 0x7f80d4036e80 2023-10-11T09:40:50.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.546+0000 7f80e30e0700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80d40ded00 con 0x7f80d403a640 2023-10-11T09:40:50.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.546+0000 7f80e167d700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f80d403a640 0x7f80d40528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.546+0000 7f80e167d700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f80d403a640 0x7f80d40528c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:50258/0 (socket says 172.21.15.78:50258) 2023-10-11T09:40:50.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.546+0000 7f80e167d700 1 -- 172.21.15.78:0/1962237107 learned_addr learned my addr 172.21.15.78:0/1962237107 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:50.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.546+0000 7fe177fff700 1 -- 172.21.15.78:0/724862941 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1780d2a30 msgr2=0x7fe178201ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.546+0000 7fe177fff700 1 --2- 172.21.15.78:0/724862941 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1780d2a30 0x7fe178201ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.546+0000 7fe177fff700 1 -- 172.21.15.78:0/724862941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1780b40a0 msgr2=0x7fe1780c9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.546+0000 7fe177fff700 1 --2- 172.21.15.78:0/724862941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1780b40a0 0x7fe1780c9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.546+0000 7fe177fff700 1 -- 172.21.15.78:0/724862941 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe16c009520 con 0x7fe17815c830 2023-10-11T09:40:50.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.546+0000 7fe176ffd700 1 --2- 172.21.15.78:0/724862941 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1780d2a30 0x7fe178201ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:40:50.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.546+0000 7fe177fff700 1 --2- 172.21.15.78:0/724862941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe17815c830 0x7fe1781fdbc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe17000cba0 tx=0x7fe17000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:50.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.546+0000 7fe163fff700 1 -- 172.21.15.78:0/724862941 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17000aae0 con 0x7fe17815c830 2023-10-11T09:40:50.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.546+0000 7fe163fff700 1 -- 172.21.15.78:0/724862941 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17000ac40 con 0x7fe17815c830 2023-10-11T09:40:50.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.546+0000 7fe163fff700 1 -- 172.21.15.78:0/724862941 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe170011430 con 0x7fe17815c830 2023-10-11T09:40:50.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.546+0000 7fe17ddb3700 1 -- 172.21.15.78:0/724862941 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1780ccc20 con 0x7fe17815c830 2023-10-11T09:40:50.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.546+0000 7fe17ddb3700 1 -- 172.21.15.78:0/724862941 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe178202b30 con 0x7fe17815c830 2023-10-11T09:40:50.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.550+0000 7f80e167d700 1 -- 172.21.15.78:0/1962237107 >> v2:172.21.15.78:3300/0 conn(0x7f80d4036e80 msgr2=0x7f80d403a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:50.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.550+0000 7f80e167d700 1 --2- 172.21.15.78:0/1962237107 >> v2:172.21.15.78:3300/0 conn(0x7f80d4036e80 0x7f80d403a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.550+0000 7f80e167d700 1 -- 172.21.15.78:0/1962237107 >> v1:172.21.15.78:6789/0 conn(0x7f80d4054800 legacy=0x7f80d4036770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:50.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.550+0000 7f80e167d700 1 -- 172.21.15.78:0/1962237107 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d403aac0 con 0x7f80d403a640 2023-10-11T09:40:50.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.550+0000 7fe17ddb3700 1 -- 172.21.15.78:0/724862941 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe154005310 con 0x7fe17815c830 2023-10-11T09:40:50.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.550+0000 7f80e167d700 1 --2- 172.21.15.78:0/1962237107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80d403a640 0x7f80d40528c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f80dc068d70 tx=0x7f80dc04ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea2685d265a128aa server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:50.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.554+0000 7f80db7fe700 1 -- 172.21.15.78:0/1962237107 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc068150 con 0x7f80d403a640 2023-10-11T09:40:50.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.554+0000 7f80db7fe700 1 -- 172.21.15.78:0/1962237107 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80dc08f070 con 0x7f80d403a640 2023-10-11T09:40:50.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.554+0000 7f80e30e0700 1 -- 172.21.15.78:0/1962237107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80d403a640 msgr2=0x7f80d40528c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.554+0000 7f80e30e0700 1 --2- 172.21.15.78:0/1962237107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80d403a640 0x7f80d40528c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f80dc068d70 tx=0x7f80dc04ec20 comp rx=0 tx=0).stop 2023-10-11T09:40:50.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.554+0000 7fe163fff700 1 -- 172.21.15.78:0/724862941 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fe1700115b0 con 0x7fe17815c830 2023-10-11T09:40:50.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.554+0000 7fe163fff700 1 --2- 172.21.15.78:0/724862941 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe15c065fa0 0x7fe15c068450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.554+0000 7fe163fff700 1 -- 172.21.15.78:0/724862941 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe17000d040 con 0x7fe17815c830 2023-10-11T09:40:50.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.554+0000 7fe1777fe700 1 --2- 172.21.15.78:0/724862941 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe15c065fa0 0x7fe15c068450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.558+0000 7fe1777fe700 1 --2- 172.21.15.78:0/724862941 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe15c065fa0 0x7fe15c068450 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe16c009890 tx=0x7fe16c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:50.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.558+0000 7f80e30e0700 1 -- 172.21.15.78:0/1962237107 shutdown_connections 2023-10-11T09:40:50.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.558+0000 7f80e30e0700 1 --2- 172.21.15.78:0/1962237107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80d403a640 0x7f80d40528c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.558+0000 7f80e30e0700 1 --2- 172.21.15.78:0/1962237107 >> v2:172.21.15.78:3300/0 conn(0x7f80d4036e80 0x7f80d403a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.558+0000 7f80e30e0700 1 -- 172.21.15.78:0/1962237107 >> 172.21.15.78:0/1962237107 conn(0x7f80d4051770 msgr2=0x7f80d4051b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:50.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.558+0000 7f80e30e0700 1 -- 172.21.15.78:0/1962237107 shutdown_connections 2023-10-11T09:40:50.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.558+0000 7f80e30e0700 1 -- 172.21.15.78:0/1962237107 wait complete. 2023-10-11T09:40:50.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.558+0000 7fe163fff700 1 -- 172.21.15.78:0/724862941 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fe170051700 con 0x7fe17815c830 2023-10-11T09:40:50.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.562+0000 7f80e30e0700 1 Processor -- start 2023-10-11T09:40:50.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.562+0000 7f80e30e0700 1 -- start start 2023-10-11T09:40:50.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.562+0000 7f80e30e0700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80d4036e80 0x7f80d4176670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.562+0000 7f80e30e0700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f80d403a640 0x7f80d4038b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.562+0000 7f80e30e0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80d4054800 0x7f80d40390b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.562+0000 7f80e0e7c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80d4036e80 0x7f80d4176670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.562+0000 7f80e0e7c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80d4036e80 0x7f80d4176670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:50260/0 (socket says 172.21.15.78:50260) 2023-10-11T09:40:50.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.562+0000 7f80e167d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80d4054800 0x7f80d40390b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.562+0000 7f80e167d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80d4054800 0x7f80d40390b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57186/0 (socket says 172.21.15.78:57186) 2023-10-11T09:40:50.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.566+0000 7f80e0e7c700 1 -- 172.21.15.78:0/1811764959 learned_addr learned my addr 172.21.15.78:0/1811764959 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:50.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.566+0000 7f80e30e0700 1 -- 172.21.15.78:0/1811764959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d4176bb0 con 0x7f80d4054800 2023-10-11T09:40:50.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.566+0000 7f80e30e0700 1 -- 172.21.15.78:0/1811764959 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d417c750 con 0x7f80d4036e80 2023-10-11T09:40:50.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.566+0000 7f80e30e0700 1 -- 172.21.15.78:0/1811764959 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d417c9f0 con 0x7f80d403a640 2023-10-11T09:40:50.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.566+0000 7f80e167d700 1 -- 172.21.15.78:0/1811764959 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f80d403a640 msgr2=0x7f80d4038b70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-11T09:40:50.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.566+0000 7f80e167d700 1 --2- 172.21.15.78:0/1811764959 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f80d403a640 0x7f80d4038b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.566+0000 7f80e167d700 1 -- 172.21.15.78:0/1811764959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80d4036e80 msgr2=0x7f80d4176670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.566+0000 7f80e167d700 1 --2- 172.21.15.78:0/1811764959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80d4036e80 0x7f80d4176670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.566+0000 7f80e167d700 1 -- 172.21.15.78:0/1811764959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d417ccd0 con 0x7f80d4054800 2023-10-11T09:40:50.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.566+0000 7f80e167d700 1 --2- 172.21.15.78:0/1811764959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80d4054800 0x7f80d40390b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f80dc069760 tx=0x7f80dc069f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:50.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.566+0000 7f80d97fa700 1 -- 172.21.15.78:0/1811764959 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc06fea0 con 0x7f80d4054800 2023-10-11T09:40:50.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.566+0000 7f80e30e0700 1 -- 172.21.15.78:0/1811764959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80d417cfb0 con 0x7f80d4054800 2023-10-11T09:40:50.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.570+0000 7f80e30e0700 1 -- 172.21.15.78:0/1811764959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80d40d8c60 con 0x7f80d4054800 2023-10-11T09:40:50.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.570+0000 7f80d97fa700 1 -- 172.21.15.78:0/1811764959 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80dc06d460 con 0x7f80d4054800 2023-10-11T09:40:50.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.574+0000 7f80d97fa700 1 -- 172.21.15.78:0/1811764959 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc06d7d0 con 0x7f80d4054800 2023-10-11T09:40:50.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.574+0000 7f80d97fa700 1 -- 172.21.15.78:0/1811764959 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f80dc09b040 con 0x7f80d4054800 2023-10-11T09:40:50.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.574+0000 7f80d97fa700 1 --2- 172.21.15.78:0/1811764959 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f80c0066020 0x7f80c00684d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:50.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.574+0000 7f80d97fa700 1 -- 172.21.15.78:0/1811764959 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f80dc0fe3b0 con 0x7f80d4054800 2023-10-11T09:40:50.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.574+0000 7f80e0e7c700 1 --2- 172.21.15.78:0/1811764959 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f80c0066020 0x7f80c00684d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:50.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.574+0000 7f80e0e7c700 1 --2- 172.21.15.78:0/1811764959 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f80c0066020 0x7f80c00684d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f80d0009a50 tx=0x7f80d0009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:50.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.578+0000 7f80e30e0700 1 -- 172.21.15.78:0/1811764959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80bc005310 con 0x7f80d4054800 2023-10-11T09:40:50.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.586+0000 7f80d97fa700 1 -- 172.21.15.78:0/1811764959 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f80dc0cba70 con 0x7f80d4054800 2023-10-11T09:40:50.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.770+0000 7f2360dba700 1 -- 172.21.15.78:0/1924996578 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f235c06cfb0 con 0x7f235c06d380 2023-10-11T09:40:50.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.770+0000 7f234affd700 1 -- 172.21.15.78:0/1924996578 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f234c027020 con 0x7f235c06d380 2023-10-11T09:40:50.778 INFO:teuthology.orchestra.run.smithi078.stdout:55834574852 2023-10-11T09:40:50.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.774+0000 7f2348ff9700 1 -- 172.21.15.78:0/1924996578 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f233c065fa0 msgr2=0x7f233c068450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.774+0000 7f2348ff9700 1 --2- 172.21.15.78:0/1924996578 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f233c065fa0 0x7f233c068450 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f235c0c9630 tx=0x7f235400b040 comp rx=0 tx=0).stop 2023-10-11T09:40:50.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.774+0000 7f2348ff9700 1 -- 172.21.15.78:0/1924996578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f235c06d380 msgr2=0x7f235c13f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.774+0000 7f2348ff9700 1 --2- 172.21.15.78:0/1924996578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f235c06d380 0x7f235c13f690 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f234c016000 tx=0x7f234c004770 comp rx=0 tx=0).stop 2023-10-11T09:40:50.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.774+0000 7f2348ff9700 1 -- 172.21.15.78:0/1924996578 shutdown_connections 2023-10-11T09:40:50.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.774+0000 7f2348ff9700 1 --2- 172.21.15.78:0/1924996578 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f233c065fa0 0x7f233c068450 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.774+0000 7f2348ff9700 1 --2- 172.21.15.78:0/1924996578 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f235c1586e0 0x7f235c0af2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.774+0000 7f2348ff9700 1 --2- 172.21.15.78:0/1924996578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f235c0b0be0 0x7f235c147fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.774+0000 7f2348ff9700 1 --2- 172.21.15.78:0/1924996578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f235c06d380 0x7f235c13f690 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.778+0000 7f2348ff9700 1 -- 172.21.15.78:0/1924996578 >> 172.21.15.78:0/1924996578 conn(0x7f235c06aca0 msgr2=0x7f235c0ce9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:50.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.778+0000 7f2348ff9700 1 -- 172.21.15.78:0/1924996578 shutdown_connections 2023-10-11T09:40:50.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.778+0000 7f2348ff9700 1 -- 172.21.15.78:0/1924996578 wait complete. 2023-10-11T09:40:50.807 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.3 2023-10-11T09:40:50.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.810+0000 7f9c23019700 1 -- 172.21.15.78:0/1370849818 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9c1c15bc70 con 0x7f9c1c06d380 2023-10-11T09:40:50.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.810+0000 7f9c197fa700 1 -- 172.21.15.78:0/1370849818 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9c1c15bc70 con 0x7f9c1c06d380 2023-10-11T09:40:50.816 INFO:teuthology.orchestra.run.smithi078.stdout:55834574854 2023-10-11T09:40:50.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.814+0000 7f9c23019700 1 -- 172.21.15.78:0/1370849818 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9c08065fa0 msgr2=0x7f9c08068450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.814+0000 7f9c23019700 1 --2- 172.21.15.78:0/1370849818 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9c08065fa0 0x7f9c08068450 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c159360 tx=0x7f9c0c009040 comp rx=0 tx=0).stop 2023-10-11T09:40:50.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.814+0000 7f9c23019700 1 -- 172.21.15.78:0/1370849818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c1c06d380 msgr2=0x7f9c1c158e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.814+0000 7f9c23019700 1 --2- 172.21.15.78:0/1370849818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c1c06d380 0x7f9c1c158e20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9c10009fd0 tx=0x7f9c10004770 comp rx=0 tx=0).stop 2023-10-11T09:40:50.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.814+0000 7f9c23019700 1 -- 172.21.15.78:0/1370849818 shutdown_connections 2023-10-11T09:40:50.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.814+0000 7f9c23019700 1 --2- 172.21.15.78:0/1370849818 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9c08065fa0 0x7f9c08068450 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.818+0000 7f9c23019700 1 --2- 172.21.15.78:0/1370849818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c1c1586e0 0x7f9c1c2022f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.818+0000 7f9c23019700 1 --2- 172.21.15.78:0/1370849818 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c1c0b0db0 0x7f9c1c0cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.818+0000 7f9c23019700 1 --2- 172.21.15.78:0/1370849818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c1c06d380 0x7f9c1c158e20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.818+0000 7f9c23019700 1 -- 172.21.15.78:0/1370849818 >> 172.21.15.78:0/1370849818 conn(0x7f9c1c06aca0 msgr2=0x7f9c1c0cd220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:50.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.818+0000 7f9c23019700 1 -- 172.21.15.78:0/1370849818 shutdown_connections 2023-10-11T09:40:50.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.818+0000 7f9c23019700 1 -- 172.21.15.78:0/1370849818 wait complete. 2023-10-11T09:40:50.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.834+0000 7fbfd6526700 1 -- 172.21.15.78:0/476496036 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fbfc8005530 con 0x7fbfc80e6e60 2023-10-11T09:40:50.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.834+0000 7fbfccff9700 1 -- 172.21.15.78:0/476496036 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbfc4054210 con 0x7fbfc80e6e60 2023-10-11T09:40:50.841 INFO:teuthology.orchestra.run.smithi078.stdout:55834574854 2023-10-11T09:40:50.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.838+0000 7f89793a7700 1 -- 172.21.15.78:0/1286113761 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f896c0532b0 con 0x7f896c0e12a0 2023-10-11T09:40:50.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.838+0000 7f895b7fe700 1 -- 172.21.15.78:0/1286113761 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f896c0532b0 con 0x7f896c0e12a0 2023-10-11T09:40:50.843 INFO:teuthology.orchestra.run.smithi078.stdout:55834574852 2023-10-11T09:40:50.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.838+0000 7f0d6c792700 1 -- 172.21.15.78:0/584555442 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f0d4c005890 con 0x7f0d6415c6b0 2023-10-11T09:40:50.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.838+0000 7fbfd6526700 1 -- 172.21.15.78:0/476496036 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbfb4065f50 msgr2=0x7fbfb4068400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.838+0000 7fbfd6526700 1 --2- 172.21.15.78:0/476496036 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbfb4065f50 0x7fbfb4068400 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc009c60 tx=0x7fbfbc008030 comp rx=0 tx=0).stop 2023-10-11T09:40:50.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.838+0000 7fbfd6526700 1 -- 172.21.15.78:0/476496036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfc80e6e60 msgr2=0x7fbfc8039f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.838+0000 7fbfd6526700 1 --2- 172.21.15.78:0/476496036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfc80e6e60 0x7fbfc8039f60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbfc400e870 tx=0x7fbfc400eb80 comp rx=0 tx=0).stop 2023-10-11T09:40:50.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.838+0000 7fbfd6526700 1 -- 172.21.15.78:0/476496036 shutdown_connections 2023-10-11T09:40:50.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.838+0000 7fbfd6526700 1 --2- 172.21.15.78:0/476496036 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbfb4065f50 0x7fbfb4068400 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.838+0000 7fbfd6526700 1 --2- 172.21.15.78:0/476496036 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbfc80e7230 0x7fbfc80e43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.838+0000 7fbfd6526700 1 --2- 172.21.15.78:0/476496036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfc80e6e60 0x7fbfc8039f60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.838+0000 7fbfd6526700 1 --2- 172.21.15.78:0/476496036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfc8036770 0x7fbfc8048750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.838+0000 7fbfd6526700 1 -- 172.21.15.78:0/476496036 >> 172.21.15.78:0/476496036 conn(0x7fbfc8051770 msgr2=0x7fbfc8035920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:50.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.838+0000 7f89793a7700 1 -- 172.21.15.78:0/1286113761 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8960065f20 msgr2=0x7f89600683d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.838+0000 7f89793a7700 1 --2- 172.21.15.78:0/1286113761 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8960065f20 0x7f89600683d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8964001370 tx=0x7f8964009040 comp rx=0 tx=0).stop 2023-10-11T09:40:50.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.838+0000 7f89793a7700 1 -- 172.21.15.78:0/1286113761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f896c0e12a0 msgr2=0x7f896c044df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.838+0000 7f89793a7700 1 --2- 172.21.15.78:0/1286113761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f896c0e12a0 0x7f896c044df0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8968016000 tx=0x7f8968004770 comp rx=0 tx=0).stop 2023-10-11T09:40:50.850 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574854 for osd.2 2023-10-11T09:40:50.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.842+0000 7f0d5affd700 1 -- 172.21.15.78:0/584555442 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0d5c056950 con 0x7f0d6415c6b0 2023-10-11T09:40:50.851 INFO:teuthology.orchestra.run.smithi078.stdout:55834574853 2023-10-11T09:40:50.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.842+0000 7f89793a7700 1 -- 172.21.15.78:0/1286113761 shutdown_connections 2023-10-11T09:40:50.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.842+0000 7f89793a7700 1 --2- 172.21.15.78:0/1286113761 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8960065f20 0x7f89600683d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.842+0000 7f89793a7700 1 --2- 172.21.15.78:0/1286113761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f896c0e4f50 0x7f896c0425f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.842+0000 7f89793a7700 1 --2- 172.21.15.78:0/1286113761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f896c0e12a0 0x7f896c044df0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.842+0000 7f89793a7700 1 --2- 172.21.15.78:0/1286113761 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f896c037b40 0x7f896c0448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.842+0000 7f89793a7700 1 -- 172.21.15.78:0/1286113761 >> 172.21.15.78:0/1286113761 conn(0x7f896c051770 msgr2=0x7f896c0e2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:50.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.842+0000 7f89793a7700 1 -- 172.21.15.78:0/1286113761 shutdown_connections 2023-10-11T09:40:50.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.842+0000 7f89793a7700 1 -- 172.21.15.78:0/1286113761 wait complete. 2023-10-11T09:40:50.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.842+0000 7fbfd6526700 1 -- 172.21.15.78:0/476496036 shutdown_connections 2023-10-11T09:40:50.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.842+0000 7fbfd6526700 1 -- 172.21.15.78:0/476496036 wait complete. 2023-10-11T09:40:50.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.842+0000 7f0d6c792700 1 -- 172.21.15.78:0/584555442 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0d48065f20 msgr2=0x7f0d480683d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.842+0000 7f0d6c792700 1 --2- 172.21.15.78:0/584555442 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0d48065f20 0x7f0d480683d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0d60000c00 tx=0x7f0d60007a70 comp rx=0 tx=0).stop 2023-10-11T09:40:50.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.842+0000 7f0d6c792700 1 -- 172.21.15.78:0/584555442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d6415c6b0 msgr2=0x7f0d641514a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.842+0000 7f0d6c792700 1 --2- 172.21.15.78:0/584555442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d6415c6b0 0x7f0d641514a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c014040 tx=0x7f0d5c004af0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.842+0000 7f0d6c792700 1 -- 172.21.15.78:0/584555442 shutdown_connections 2023-10-11T09:40:50.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.842+0000 7f0d6c792700 1 --2- 172.21.15.78:0/584555442 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0d48065f20 0x7f0d480683d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.842+0000 7f0d6c792700 1 --2- 172.21.15.78:0/584555442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d6415c6b0 0x7f0d641514a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.842+0000 7f0d6c792700 1 --2- 172.21.15.78:0/584555442 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d640cd030 0x7f0d6415eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.842+0000 7f0d6c792700 1 --2- 172.21.15.78:0/584555442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d640c9020 0x7f0d6414c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.842+0000 7f0d6c792700 1 -- 172.21.15.78:0/584555442 >> 172.21.15.78:0/584555442 conn(0x7f0d6406aca0 msgr2=0x7f0d640c94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:50.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.842+0000 7f0d6c792700 1 -- 172.21.15.78:0/584555442 shutdown_connections 2023-10-11T09:40:50.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.842+0000 7f0d6c792700 1 -- 172.21.15.78:0/584555442 wait complete. 2023-10-11T09:40:50.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.846+0000 7f10c0d13700 1 -- 172.21.15.78:0/20729074 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f10bc14f4b0 con 0x7f10bc0b06a0 2023-10-11T09:40:50.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.846+0000 7f10aaffd700 1 -- 172.21.15.78:0/20729074 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f10bc14f4b0 con 0x7f10bc0b06a0 2023-10-11T09:40:50.859 INFO:teuthology.orchestra.run.smithi078.stdout:55834574853 2023-10-11T09:40:50.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.850+0000 7f10c0d13700 1 -- 172.21.15.78:0/20729074 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f109c065ed0 msgr2=0x7f109c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.850+0000 7f10c0d13700 1 --2- 172.21.15.78:0/20729074 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f109c065ed0 0x7f109c068380 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f10ac001960 tx=0x7f10ac009040 comp rx=0 tx=0).stop 2023-10-11T09:40:50.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.850+0000 7f10c0d13700 1 -- 172.21.15.78:0/20729074 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10bc0b06a0 msgr2=0x7f10bc1445b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.850+0000 7f10c0d13700 1 --2- 172.21.15.78:0/20729074 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10bc0b06a0 0x7f10bc1445b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f10b400caa0 tx=0x7f10b400cad0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.850+0000 7f10c0d13700 1 -- 172.21.15.78:0/20729074 shutdown_connections 2023-10-11T09:40:50.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.850+0000 7f10c0d13700 1 --2- 172.21.15.78:0/20729074 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f109c065ed0 0x7f109c068380 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.850+0000 7f10c0d13700 1 --2- 172.21.15.78:0/20729074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10bc0cf170 0x7f10bc144af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.850+0000 7f10c0d13700 1 --2- 172.21.15.78:0/20729074 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10bc0b06a0 0x7f10bc1445b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.850+0000 7f10c0d13700 1 --2- 172.21.15.78:0/20729074 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10bc06d380 0x7f10bc14fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.850+0000 7f10c0d13700 1 -- 172.21.15.78:0/20729074 >> 172.21.15.78:0/20729074 conn(0x7f10bc06aca0 msgr2=0x7f10bc14e610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:50.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.850+0000 7f10c0d13700 1 -- 172.21.15.78:0/20729074 shutdown_connections 2023-10-11T09:40:50.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.850+0000 7f10c0d13700 1 -- 172.21.15.78:0/20729074 wait complete. 2023-10-11T09:40:50.865 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.0 2023-10-11T09:40:50.866 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574854 for osd.7 2023-10-11T09:40:50.874 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574853 for osd.6 2023-10-11T09:40:50.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.870+0000 7f80e30e0700 1 -- 172.21.15.78:0/1811764959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f80bc0055c0 con 0x7f80d4054800 2023-10-11T09:40:50.876 INFO:teuthology.orchestra.run.smithi078.stdout:55834574853 2023-10-11T09:40:50.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.870+0000 7f80d97fa700 1 -- 172.21.15.78:0/1811764959 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f80dc0a1030 con 0x7f80d4054800 2023-10-11T09:40:50.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.874+0000 7f80e30e0700 1 -- 172.21.15.78:0/1811764959 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f80c0066020 msgr2=0x7f80c00684d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.874+0000 7f80e30e0700 1 --2- 172.21.15.78:0/1811764959 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f80c0066020 0x7f80c00684d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f80d0009a50 tx=0x7f80d0009190 comp rx=0 tx=0).stop 2023-10-11T09:40:50.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.874+0000 7f80e30e0700 1 -- 172.21.15.78:0/1811764959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80d4054800 msgr2=0x7f80d40390b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.874+0000 7f80e30e0700 1 --2- 172.21.15.78:0/1811764959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80d4054800 0x7f80d40390b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f80dc069760 tx=0x7f80dc069f90 comp rx=0 tx=0).stop 2023-10-11T09:40:50.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.874+0000 7f80e30e0700 1 -- 172.21.15.78:0/1811764959 shutdown_connections 2023-10-11T09:40:50.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.874+0000 7f80e30e0700 1 --2- 172.21.15.78:0/1811764959 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f80c0066020 0x7f80c00684d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.874+0000 7f80e30e0700 1 --2- 172.21.15.78:0/1811764959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80d4054800 0x7f80d40390b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.874+0000 7f80e30e0700 1 --2- 172.21.15.78:0/1811764959 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f80d403a640 0x7f80d4038b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.874+0000 7f80e30e0700 1 --2- 172.21.15.78:0/1811764959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80d4036e80 0x7f80d4176670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.874+0000 7f80e30e0700 1 -- 172.21.15.78:0/1811764959 >> 172.21.15.78:0/1811764959 conn(0x7f80d4051770 msgr2=0x7f80d40e15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:50.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.874+0000 7f80e30e0700 1 -- 172.21.15.78:0/1811764959 shutdown_connections 2023-10-11T09:40:50.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.874+0000 7f80e30e0700 1 -- 172.21.15.78:0/1811764959 wait complete. 2023-10-11T09:40:50.883 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574853 for osd.1 2023-10-11T09:40:50.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.878+0000 7fe17ddb3700 1 -- 172.21.15.78:0/724862941 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fe154005180 con 0x7fe17815c830 2023-10-11T09:40:50.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.878+0000 7fe163fff700 1 -- 172.21.15.78:0/724862941 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe170015090 con 0x7fe17815c830 2023-10-11T09:40:50.885 INFO:teuthology.orchestra.run.smithi078.stdout:55834574853 2023-10-11T09:40:50.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.878+0000 7fe17ddb3700 1 -- 172.21.15.78:0/724862941 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe15c065fa0 msgr2=0x7fe15c068450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.878+0000 7fe17ddb3700 1 --2- 172.21.15.78:0/724862941 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe15c065fa0 0x7fe15c068450 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe16c009890 tx=0x7fe16c008030 comp rx=0 tx=0).stop 2023-10-11T09:40:50.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.878+0000 7fe17ddb3700 1 -- 172.21.15.78:0/724862941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe17815c830 msgr2=0x7fe1781fdbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:50.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.878+0000 7fe17ddb3700 1 --2- 172.21.15.78:0/724862941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe17815c830 0x7fe1781fdbc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe17000cba0 tx=0x7fe17000ceb0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.878+0000 7fe17ddb3700 1 -- 172.21.15.78:0/724862941 shutdown_connections 2023-10-11T09:40:50.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.878+0000 7fe17ddb3700 1 --2- 172.21.15.78:0/724862941 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe15c065fa0 0x7fe15c068450 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.878+0000 7fe17ddb3700 1 --2- 172.21.15.78:0/724862941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe17815c830 0x7fe1781fdbc0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.878+0000 7fe17ddb3700 1 --2- 172.21.15.78:0/724862941 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1780d2a30 0x7fe178201ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.878+0000 7fe17ddb3700 1 --2- 172.21.15.78:0/724862941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1780b40a0 0x7fe1780c9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:50.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.878+0000 7fe17ddb3700 1 -- 172.21.15.78:0/724862941 >> 172.21.15.78:0/724862941 conn(0x7fe17809c1b0 msgr2=0x7fe17806ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:50.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.878+0000 7fe17ddb3700 1 -- 172.21.15.78:0/724862941 shutdown_connections 2023-10-11T09:40:50.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:50.878+0000 7fe17ddb3700 1 -- 172.21.15.78:0/724862941 wait complete. 2023-10-11T09:40:50.901 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574853 for osd.4 2023-10-11T09:40:50.902 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574853 for osd.5 2023-10-11T09:40:51.808 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-11T09:40:51.851 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-11T09:40:51.866 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-11T09:40:51.867 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-11T09:40:51.876 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-11T09:40:51.884 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-11T09:40:51.903 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-11T09:40:51.903 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-11T09:40:51.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:51.994+0000 7fa37f619700 1 Processor -- start 2023-10-11T09:40:52.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:51.994+0000 7fa37f619700 1 -- start start 2023-10-11T09:40:52.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:51.994+0000 7fa37f619700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa37806d380 0x7fa3780b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:51.994+0000 7fa37f619700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3780b0be0 0x7fa378070230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:51.994+0000 7fa37f619700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3780b0fe0 con 0x7fa3781586e0 2023-10-11T09:40:52.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:51.994+0000 7fa37f619700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa37806c440 con 0x7fa37806d380 2023-10-11T09:40:52.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:51.994+0000 7fa37f619700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa37806c720 con 0x7fa3780b0be0 2023-10-11T09:40:52.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:51.998+0000 7fa37d3b5700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fa3781586e0 0x7fa37806bbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:58856/0 (socket says 172.21.15.78:58856) 2023-10-11T09:40:52.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:51.998+0000 7fa37d3b5700 1 -- 172.21.15.78:0/2219787323 learned_addr learned my addr 172.21.15.78:0/2219787323 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:52.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:51.998+0000 7fa37cbb4700 1 --2- 172.21.15.78:0/2219787323 >> v2:172.21.15.133:3300/0 conn(0x7fa37806d380 0x7fa3780b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:51.998+0000 7fa37dbb6700 1 --2- 172.21.15.78:0/2219787323 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3780b0be0 0x7fa378070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:51.998+0000 7fa37cbb4700 1 -- 172.21.15.78:0/2219787323 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3780b0be0 msgr2=0x7fa378070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:52.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:51.998+0000 7fa37cbb4700 1 --2- 172.21.15.78:0/2219787323 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3780b0be0 0x7fa378070230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:51.998+0000 7fa37cbb4700 1 -- 172.21.15.78:0/2219787323 >> v1:172.21.15.78:6789/0 conn(0x7fa3781586e0 legacy=0x7fa37806bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:51.998+0000 7fa37cbb4700 1 -- 172.21.15.78:0/2219787323 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa37806ca00 con 0x7fa37806d380 2023-10-11T09:40:52.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:51.998+0000 7fa37cbb4700 1 --2- 172.21.15.78:0/2219787323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa37806d380 0x7fa3780b06a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa374009850 tx=0x7fa374009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=222127a6d4e04fef server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:52.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:51.998+0000 7fa36ffff700 1 -- 172.21.15.78:0/2219787323 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa374004230 con 0x7fa37806d380 2023-10-11T09:40:52.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:51.998+0000 7fa36ffff700 1 -- 172.21.15.78:0/2219787323 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa374004390 con 0x7fa37806d380 2023-10-11T09:40:52.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.002+0000 7fa37f619700 1 -- 172.21.15.78:0/2219787323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa37806d380 msgr2=0x7fa3780b06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.002+0000 7fa37f619700 1 --2- 172.21.15.78:0/2219787323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa37806d380 0x7fa3780b06a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa374009850 tx=0x7fa374009b60 comp rx=0 tx=0).stop 2023-10-11T09:40:52.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.002+0000 7fa37f619700 1 -- 172.21.15.78:0/2219787323 shutdown_connections 2023-10-11T09:40:52.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.002+0000 7fa37f619700 1 --2- 172.21.15.78:0/2219787323 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3780b0be0 0x7fa378070230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.002+0000 7fa37f619700 1 --2- 172.21.15.78:0/2219787323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa37806d380 0x7fa3780b06a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.002+0000 7fa37f619700 1 -- 172.21.15.78:0/2219787323 >> 172.21.15.78:0/2219787323 conn(0x7fa37806aca0 msgr2=0x7fa3780cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:52.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.002+0000 7fa37f619700 1 -- 172.21.15.78:0/2219787323 shutdown_connections 2023-10-11T09:40:52.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.002+0000 7fa37f619700 1 -- 172.21.15.78:0/2219787323 wait complete. 2023-10-11T09:40:52.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.006+0000 7fa37f619700 1 Processor -- start 2023-10-11T09:40:52.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.006+0000 7fa37f619700 1 -- start start 2023-10-11T09:40:52.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.006+0000 7fa37f619700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3780b0be0 0x7fa3781fe6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.006+0000 7fa37f619700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3781586e0 0x7fa378202cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.006+0000 7fa37f619700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa378151dd0 0x7fa37814d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.006+0000 7fa37d3b5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3780b0be0 0x7fa3781fe6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.010+0000 7fa37cbb4700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3781586e0 0x7fa378202cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.010+0000 7fa37cbb4700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3781586e0 0x7fa378202cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:44388/0 (socket says 172.21.15.78:44388) 2023-10-11T09:40:52.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.010+0000 7fa37d3b5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3780b0be0 0x7fa3781fe6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:50272/0 (socket says 172.21.15.78:50272) 2023-10-11T09:40:52.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.010+0000 7fa37cbb4700 1 -- 172.21.15.78:0/4055854975 learned_addr learned my addr 172.21.15.78:0/4055854975 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:52.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.010+0000 7fa37dbb6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa378151dd0 0x7fa37814d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.010+0000 7fa37f619700 1 -- 172.21.15.78:0/4055854975 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3780cd2c0 con 0x7fa378151dd0 2023-10-11T09:40:52.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.010+0000 7fa37f619700 1 -- 172.21.15.78:0/4055854975 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3780cc640 con 0x7fa3780b0be0 2023-10-11T09:40:52.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.010+0000 7fa37f619700 1 -- 172.21.15.78:0/4055854975 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3780cc920 con 0x7fa3781586e0 2023-10-11T09:40:52.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.010+0000 7fa37dbb6700 1 -- 172.21.15.78:0/4055854975 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3781586e0 msgr2=0x7fa378202cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.010+0000 7fa37dbb6700 1 --2- 172.21.15.78:0/4055854975 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3781586e0 0x7fa378202cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.010+0000 7fa37dbb6700 1 -- 172.21.15.78:0/4055854975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3780b0be0 msgr2=0x7fa3781fe6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.010+0000 7fa37dbb6700 1 --2- 172.21.15.78:0/4055854975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3780b0be0 0x7fa3781fe6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.010+0000 7fa37dbb6700 1 -- 172.21.15.78:0/4055854975 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3740094b0 con 0x7fa378151dd0 2023-10-11T09:40:52.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.010+0000 7fa37d3b5700 1 --2- 172.21.15.78:0/4055854975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3780b0be0 0x7fa3781fe6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:40:52.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.010+0000 7fa37cbb4700 1 --2- 172.21.15.78:0/4055854975 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3781586e0 0x7fa378202cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:40:52.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.010+0000 7fa37dbb6700 1 --2- 172.21.15.78:0/4055854975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa378151dd0 0x7fa37814d880 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa36800e870 tx=0x7fa36800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:52.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.014+0000 7fa36dffb700 1 -- 172.21.15.78:0/4055854975 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa368013610 con 0x7fa378151dd0 2023-10-11T09:40:52.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.014+0000 7fa37f619700 1 -- 172.21.15.78:0/4055854975 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3780ccc00 con 0x7fa378151dd0 2023-10-11T09:40:52.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.018+0000 7fa37f619700 1 -- 172.21.15.78:0/4055854975 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa378152ef0 con 0x7fa378151dd0 2023-10-11T09:40:52.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.018+0000 7fa36dffb700 1 -- 172.21.15.78:0/4055854975 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa368013770 con 0x7fa378151dd0 2023-10-11T09:40:52.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.018+0000 7fa36dffb700 1 -- 172.21.15.78:0/4055854975 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa368013b00 con 0x7fa378151dd0 2023-10-11T09:40:52.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.022+0000 7fa36dffb700 1 -- 172.21.15.78:0/4055854975 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fa368013c60 con 0x7fa378151dd0 2023-10-11T09:40:52.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.022+0000 7fa36dffb700 1 --2- 172.21.15.78:0/4055854975 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa35c065fa0 0x7fa35c068450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.022+0000 7fa36dffb700 1 -- 172.21.15.78:0/4055854975 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa368083330 con 0x7fa378151dd0 2023-10-11T09:40:52.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.022+0000 7fa37d3b5700 1 --2- 172.21.15.78:0/4055854975 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa35c065fa0 0x7fa35c068450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.022+0000 7fa37f619700 1 -- 172.21.15.78:0/4055854975 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa358005310 con 0x7fa378151dd0 2023-10-11T09:40:52.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.030+0000 7fa36dffb700 1 -- 172.21.15.78:0/4055854975 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fa3680509f0 con 0x7fa378151dd0 2023-10-11T09:40:52.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.034+0000 7fa37d3b5700 1 --2- 172.21.15.78:0/4055854975 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa35c065fa0 0x7fa35c068450 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa3700076e0 tx=0x7fa3700071a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:52.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.082+0000 7f50a0243700 1 Processor -- start 2023-10-11T09:40:52.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.082+0000 7f50a0243700 1 -- start start 2023-10-11T09:40:52.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.082+0000 7f50a0243700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f50981670b0 0x7f50980b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.082+0000 7f50a0243700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f509816dd60 con 0x7f509816d720 2023-10-11T09:40:52.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.082+0000 7f50a0243700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5098169d00 con 0x7f50980b0be0 2023-10-11T09:40:52.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.082+0000 7f50a0243700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f509816a3c0 con 0x7f50981670b0 2023-10-11T09:40:52.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.082+0000 7f509e7e0700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f509816d720 0x7f50980ca210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:58858/0 (socket says 172.21.15.78:58858) 2023-10-11T09:40:52.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.082+0000 7f509e7e0700 1 -- 172.21.15.78:0/1941136810 learned_addr learned my addr 172.21.15.78:0/1941136810 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:52.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.082+0000 7f509dfdf700 1 --2- 172.21.15.78:0/1941136810 >> v2:172.21.15.133:3300/0 conn(0x7f50981670b0 0x7f50980b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.090+0000 7f509cfdd700 1 -- 172.21.15.78:0/1941136810 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4176267881 0 0) 0x7f509816dd60 con 0x7f509816d720 2023-10-11T09:40:52.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.090+0000 7f509cfdd700 1 -- 172.21.15.78:0/1941136810 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50840035d0 con 0x7f509816d720 2023-10-11T09:40:52.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.090+0000 7f509cfdd700 1 -- 172.21.15.78:0/1941136810 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1878507492 0 0) 0x7f5098169d00 con 0x7f50980b0be0 2023-10-11T09:40:52.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.090+0000 7f509cfdd700 1 -- 172.21.15.78:0/1941136810 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f509816dd60 con 0x7f50980b0be0 2023-10-11T09:40:52.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.090+0000 7f509cfdd700 1 -- 172.21.15.78:0/1941136810 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3846060671 0 0) 0x7f50840035d0 con 0x7f509816d720 2023-10-11T09:40:52.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.090+0000 7f509dfdf700 1 -- 172.21.15.78:0/1941136810 >> v1:172.21.15.133:6789/0 conn(0x7f50980b0be0 legacy=0x7f509816d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.090+0000 7f509dfdf700 1 -- 172.21.15.78:0/1941136810 >> v1:172.21.15.78:6789/0 conn(0x7f509816d720 legacy=0x7f50980ca210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.090+0000 7f509dfdf700 1 -- 172.21.15.78:0/1941136810 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5098169fe0 con 0x7f50981670b0 2023-10-11T09:40:52.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.090+0000 7f509dfdf700 1 --2- 172.21.15.78:0/1941136810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50981670b0 0x7f50980b06a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5088007360 tx=0x7f5088009c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21808cfc774e5a81 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:52.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.090+0000 7f509cfdd700 1 -- 172.21.15.78:0/1941136810 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508800e070 con 0x7f50981670b0 2023-10-11T09:40:52.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.090+0000 7f509cfdd700 1 -- 172.21.15.78:0/1941136810 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50880092e0 con 0x7f50981670b0 2023-10-11T09:40:52.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.090+0000 7f50a0243700 1 -- 172.21.15.78:0/1941136810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50981670b0 msgr2=0x7f50980b06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.090+0000 7f50a0243700 1 --2- 172.21.15.78:0/1941136810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50981670b0 0x7f50980b06a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5088007360 tx=0x7f5088009c70 comp rx=0 tx=0).stop 2023-10-11T09:40:52.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.094+0000 7f50a0243700 1 -- 172.21.15.78:0/1941136810 shutdown_connections 2023-10-11T09:40:52.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.094+0000 7f50a0243700 1 --2- 172.21.15.78:0/1941136810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50981670b0 0x7f50980b06a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.094+0000 7f50a0243700 1 -- 172.21.15.78:0/1941136810 >> 172.21.15.78:0/1941136810 conn(0x7f509806aca0 msgr2=0x7f50980cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:52.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.094+0000 7f50a0243700 1 -- 172.21.15.78:0/1941136810 shutdown_connections 2023-10-11T09:40:52.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.094+0000 7f50a0243700 1 -- 172.21.15.78:0/1941136810 wait complete. 2023-10-11T09:40:52.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.094+0000 7f50a0243700 1 Processor -- start 2023-10-11T09:40:52.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.094+0000 7f50a0243700 1 -- start start 2023-10-11T09:40:52.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.098+0000 7f50a0243700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50980b0be0 0x7f50980b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.098+0000 7f509dfdf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50980b0be0 0x7f50980b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.098+0000 7f509dfdf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50980b0be0 0x7f50980b1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57198/0 (socket says 172.21.15.78:57198) 2023-10-11T09:40:52.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.098+0000 7f509dfdf700 1 -- 172.21.15.78:0/2648874592 learned_addr learned my addr 172.21.15.78:0/2648874592 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:52.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.098+0000 7f50a0243700 1 --2- 172.21.15.78:0/2648874592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50981670b0 0x7f509820a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.098+0000 7f50a0243700 1 --2- 172.21.15.78:0/2648874592 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f509816d720 0x7f509820ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.098+0000 7f509d7de700 1 --2- 172.21.15.78:0/2648874592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50981670b0 0x7f509820a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.098+0000 7f50a0243700 1 -- 172.21.15.78:0/2648874592 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50980cbaf0 con 0x7f50980b0be0 2023-10-11T09:40:52.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.102+0000 7f509e7e0700 1 --2- 172.21.15.78:0/2648874592 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f509816d720 0x7f509820ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.102+0000 7f50a0243700 1 -- 172.21.15.78:0/2648874592 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50980b23f0 con 0x7f50981670b0 2023-10-11T09:40:52.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.102+0000 7f50a0243700 1 -- 172.21.15.78:0/2648874592 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50980caee0 con 0x7f509816d720 2023-10-11T09:40:52.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.102+0000 7f509dfdf700 1 -- 172.21.15.78:0/2648874592 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f509816d720 msgr2=0x7f509820ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.102+0000 7f509dfdf700 1 --2- 172.21.15.78:0/2648874592 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f509816d720 0x7f509820ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.102+0000 7f509dfdf700 1 -- 172.21.15.78:0/2648874592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50981670b0 msgr2=0x7f509820a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.102+0000 7f509dfdf700 1 --2- 172.21.15.78:0/2648874592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50981670b0 0x7f509820a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.102+0000 7f509dfdf700 1 -- 172.21.15.78:0/2648874592 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5088007040 con 0x7f50980b0be0 2023-10-11T09:40:52.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.102+0000 7f509dfdf700 1 --2- 172.21.15.78:0/2648874592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50980b0be0 0x7f50980b1eb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5088000c00 tx=0x7f5088004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:52.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.102+0000 7f508e7fc700 1 -- 172.21.15.78:0/2648874592 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5088003c90 con 0x7f50980b0be0 2023-10-11T09:40:52.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.102+0000 7f50a0243700 1 -- 172.21.15.78:0/2648874592 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f509816a960 con 0x7f50980b0be0 2023-10-11T09:40:52.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.102+0000 7f50a0243700 1 -- 172.21.15.78:0/2648874592 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50980cb4a0 con 0x7f50980b0be0 2023-10-11T09:40:52.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.102+0000 7f508e7fc700 1 -- 172.21.15.78:0/2648874592 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5088003e10 con 0x7f50980b0be0 2023-10-11T09:40:52.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.102+0000 7f508e7fc700 1 -- 172.21.15.78:0/2648874592 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50880076d0 con 0x7f50980b0be0 2023-10-11T09:40:52.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.106+0000 7f50a0243700 1 -- 172.21.15.78:0/2648874592 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5080005310 con 0x7f50980b0be0 2023-10-11T09:40:52.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.114+0000 7f508e7fc700 1 -- 172.21.15.78:0/2648874592 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f5088007830 con 0x7f50980b0be0 2023-10-11T09:40:52.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.114+0000 7f508e7fc700 1 --2- 172.21.15.78:0/2648874592 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f507c065fa0 0x7f507c068450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.114+0000 7f509d7de700 1 --2- 172.21.15.78:0/2648874592 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f507c065fa0 0x7f507c068450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.114+0000 7f508e7fc700 1 -- 172.21.15.78:0/2648874592 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5088084320 con 0x7f50980b0be0 2023-10-11T09:40:52.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.114+0000 7f509d7de700 1 --2- 172.21.15.78:0/2648874592 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f507c065fa0 0x7f507c068450 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f50980b0b70 tx=0x7f5094009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:52.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.118+0000 7f508e7fc700 1 -- 172.21.15.78:0/2648874592 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f5088051b00 con 0x7f50980b0be0 2023-10-11T09:40:52.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.126+0000 7f3772b16700 1 Processor -- start 2023-10-11T09:40:52.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.126+0000 7fafa869b700 1 Processor -- start 2023-10-11T09:40:52.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.126+0000 7f3772b16700 1 -- start start 2023-10-11T09:40:52.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.126+0000 7fafa869b700 1 -- start start 2023-10-11T09:40:52.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.130+0000 7fafa869b700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faf9803c740 0x7faf98053080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.130+0000 7f3772b16700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f37640383e0 0x7f3764006d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.130+0000 7fafa869b700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7faf980e4c50 0x7faf98038840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.130+0000 7fafa869b700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf980391e0 con 0x7faf98038d80 2023-10-11T09:40:52.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.130+0000 7fafa869b700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf980e51a0 con 0x7faf980e4c50 2023-10-11T09:40:52.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.130+0000 7fafa869b700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf98056c80 con 0x7faf9803c740 2023-10-11T09:40:52.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.130+0000 7f3772b16700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37640079e0 con 0x7f376403ac80 2023-10-11T09:40:52.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.130+0000 7f3772b16700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3764053850 con 0x7f376403b0d0 2023-10-11T09:40:52.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.130+0000 7fba877f9700 1 Processor -- start 2023-10-11T09:40:52.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.130+0000 7f3771313700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f376403ac80 0x7f37640e6530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:58874/0 (socket says 172.21.15.78:58874) 2023-10-11T09:40:52.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.130+0000 7f3772b16700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f376403c050 con 0x7f37640383e0 2023-10-11T09:40:52.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.130+0000 7f3771313700 1 -- 172.21.15.78:0/1257455045 learned_addr learned my addr 172.21.15.78:0/1257455045 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:52.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.130+0000 7f3771b14700 1 --2- 172.21.15.78:0/1257455045 >> v2:172.21.15.78:3300/0 conn(0x7f37640383e0 0x7f3764006d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.130+0000 7fafa6c38700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7faf98038d80 0x7faf9800dd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:33886/0 (socket says 172.21.15.78:33886) 2023-10-11T09:40:52.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.130+0000 7fafa6c38700 1 -- 172.21.15.78:0/1994230678 learned_addr learned my addr 172.21.15.78:0/1994230678 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:52.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.130+0000 7fba877f9700 1 -- start start 2023-10-11T09:40:52.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.130+0000 7fafa6437700 1 --2- 172.21.15.78:0/1994230678 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faf9803c740 0x7faf98053080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.130+0000 7f3771b14700 1 -- 172.21.15.78:0/1257455045 >> v1:172.21.15.133:6789/0 conn(0x7f376403b0d0 legacy=0x7f37640e6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:52.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.130+0000 7f3771b14700 1 -- 172.21.15.78:0/1257455045 >> v1:172.21.15.78:6789/0 conn(0x7f376403ac80 legacy=0x7f37640e6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.130+0000 7f3771b14700 1 -- 172.21.15.78:0/1257455045 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3764007700 con 0x7f37640383e0 2023-10-11T09:40:52.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.130+0000 7fafa5435700 1 -- 172.21.15.78:0/1994230678 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2510362784 0 0) 0x7faf980391e0 con 0x7faf98038d80 2023-10-11T09:40:52.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.130+0000 7fafa5435700 1 -- 172.21.15.78:0/1994230678 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf8c0035d0 con 0x7faf98038d80 2023-10-11T09:40:52.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.130+0000 7fafa6437700 1 -- 172.21.15.78:0/1994230678 >> v2:172.21.15.133:3300/0 conn(0x7faf980e4c50 msgr2=0x7faf98038840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-11T09:40:52.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.130+0000 7f3771b14700 1 --2- 172.21.15.78:0/1257455045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37640383e0 0x7f3764006d30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f376c04f2b0 tx=0x7f376c074690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fde4740f4b94b51 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:52.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.130+0000 7fafa5435700 1 -- 172.21.15.78:0/1994230678 <== mon.1 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2285304259 0 0) 0x7faf8c0035d0 con 0x7faf98038d80 2023-10-11T09:40:52.145 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7f3770b12700 1 -- 172.21.15.78:0/1257455045 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376c068150 con 0x7f37640383e0 2023-10-11T09:40:52.145 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fafa6437700 1 --2- 172.21.15.78:0/1994230678 >> v2:172.21.15.133:3300/0 conn(0x7faf980e4c50 0x7faf98038840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.145 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fafa6437700 1 -- 172.21.15.78:0/1994230678 >> v1:172.21.15.133:6789/0 conn(0x7faf98038d80 legacy=0x7faf9800dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fafa6437700 1 -- 172.21.15.78:0/1994230678 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf980e0ba0 con 0x7faf9803c740 2023-10-11T09:40:52.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7f3770b12700 1 -- 172.21.15.78:0/1257455045 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f376c067100 con 0x7f37640383e0 2023-10-11T09:40:52.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fafa6437700 1 --2- 172.21.15.78:0/1994230678 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faf9803c740 0x7faf98053080 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7faf9c009850 tx=0x7faf9c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca5e21c7aa823a3c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:52.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fba877f9700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fba78038e00 0x7fba780091d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fba877f9700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba78054650 con 0x7fba78038980 2023-10-11T09:40:52.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fba877f9700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba780e7190 con 0x7fba780e2aa0 2023-10-11T09:40:52.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fba877f9700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba78007fe0 con 0x7fba78038e00 2023-10-11T09:40:52.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fafa5435700 1 -- 172.21.15.78:0/1994230678 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa0074190 con 0x7faf9803c740 2023-10-11T09:40:52.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fafa5435700 1 -- 172.21.15.78:0/1994230678 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf9c004030 con 0x7faf9803c740 2023-10-11T09:40:52.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fafa5435700 1 -- 172.21.15.78:0/1994230678 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9c0043e0 con 0x7faf9803c740 2023-10-11T09:40:52.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7f3772b16700 1 -- 172.21.15.78:0/1257455045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37640383e0 msgr2=0x7f3764006d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7f3772b16700 1 --2- 172.21.15.78:0/1257455045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37640383e0 0x7f3764006d30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f376c04f2b0 tx=0x7f376c074690 comp rx=0 tx=0).stop 2023-10-11T09:40:52.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fba85595700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fba780e2aa0 0x7fba780e6620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:33904/0 (socket says 172.21.15.78:33904) 2023-10-11T09:40:52.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fba85595700 1 -- 172.21.15.78:0/1844916915 learned_addr learned my addr 172.21.15.78:0/1844916915 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:52.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fba85d96700 1 --2- 172.21.15.78:0/1844916915 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fba78038e00 0x7fba780091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7f3772b16700 1 -- 172.21.15.78:0/1257455045 shutdown_connections 2023-10-11T09:40:52.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7f3772b16700 1 --2- 172.21.15.78:0/1257455045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37640383e0 0x7f3764006d30 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7f3772b16700 1 -- 172.21.15.78:0/1257455045 >> 172.21.15.78:0/1257455045 conn(0x7f3764055c80 msgr2=0x7f3764056050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:52.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fba77fff700 1 -- 172.21.15.78:0/1844916915 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 601693199 0 0) 0x7fba780e7190 con 0x7fba780e2aa0 2023-10-11T09:40:52.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fba77fff700 1 -- 172.21.15.78:0/1844916915 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba6c0035d0 con 0x7fba780e2aa0 2023-10-11T09:40:52.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7f3772b16700 1 -- 172.21.15.78:0/1257455045 shutdown_connections 2023-10-11T09:40:52.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fba77fff700 1 -- 172.21.15.78:0/1844916915 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 203641749 0 0) 0x7fba78054650 con 0x7fba78038980 2023-10-11T09:40:52.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fba77fff700 1 -- 172.21.15.78:0/1844916915 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba780e7190 con 0x7fba78038980 2023-10-11T09:40:52.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fafa869b700 1 -- 172.21.15.78:0/1994230678 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faf9803c740 msgr2=0x7faf98053080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fafa869b700 1 --2- 172.21.15.78:0/1994230678 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faf9803c740 0x7faf98053080 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7faf9c009850 tx=0x7faf9c009b60 comp rx=0 tx=0).stop 2023-10-11T09:40:52.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fba77fff700 1 -- 172.21.15.78:0/1844916915 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3740126379 0 0) 0x7fba780e7190 con 0x7fba78038980 2023-10-11T09:40:52.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fba77fff700 1 -- 172.21.15.78:0/1844916915 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fba78038e00 msgr2=0x7fba780091d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:52.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fba77fff700 1 --2- 172.21.15.78:0/1844916915 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fba78038e00 0x7fba780091d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fba77fff700 1 -- 172.21.15.78:0/1844916915 >> v1:172.21.15.133:6789/0 conn(0x7fba780e2aa0 legacy=0x7fba780e6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fba77fff700 1 -- 172.21.15.78:0/1844916915 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba780082c0 con 0x7fba78038980 2023-10-11T09:40:52.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fba77fff700 1 -- 172.21.15.78:0/1844916915 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7fba7c001670 con 0x7fba78038980 2023-10-11T09:40:52.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fba77fff700 1 -- 172.21.15.78:0/1844916915 >> v1:172.21.15.78:6789/0 conn(0x7fba78038980 legacy=0x7fba78008ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fba77fff700 1 --2- 172.21.15.78:0/1844916915 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fba6c004c40 0x7fba6c005030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fba77fff700 1 --2- 172.21.15.78:0/1844916915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba6c003810 0x7fba6c003c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fba77fff700 1 --2- 172.21.15.78:0/1844916915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba6c0041c0 0x7fba6c00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fba77fff700 1 -- 172.21.15.78:0/1844916915 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba78007fe0 con 0x7fba6c003810 2023-10-11T09:40:52.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fba77fff700 1 -- 172.21.15.78:0/1844916915 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba6c00bf70 con 0x7fba6c0041c0 2023-10-11T09:40:52.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fafa869b700 1 -- 172.21.15.78:0/1994230678 shutdown_connections 2023-10-11T09:40:52.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fafa869b700 1 --2- 172.21.15.78:0/1994230678 >> v2:172.21.15.133:3300/0 conn(0x7faf980e4c50 0x7faf98038840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fafa869b700 1 --2- 172.21.15.78:0/1994230678 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faf9803c740 0x7faf98053080 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fafa869b700 1 -- 172.21.15.78:0/1994230678 >> 172.21.15.78:0/1994230678 conn(0x7faf9800cf60 msgr2=0x7faf98053840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:52.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fba77fff700 1 -- 172.21.15.78:0/1844916915 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba6c00c0b0 con 0x7fba6c004c40 2023-10-11T09:40:52.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fafa869b700 1 -- 172.21.15.78:0/1994230678 shutdown_connections 2023-10-11T09:40:52.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fba84d94700 1 --2- 172.21.15.78:0/1844916915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba6c003810 0x7fba6c003c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fba85595700 1 --2- 172.21.15.78:0/1844916915 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fba6c004c40 0x7fba6c005030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fba85d96700 1 --2- 172.21.15.78:0/1844916915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba6c0041c0 0x7fba6c00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7fba84d94700 1 -- 172.21.15.78:0/1844916915 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fba6c004c40 msgr2=0x7fba6c005030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:52.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7fba84d94700 1 --2- 172.21.15.78:0/1844916915 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fba6c004c40 0x7fba6c005030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7fba84d94700 1 -- 172.21.15.78:0/1844916915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba6c0041c0 msgr2=0x7fba6c00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:52.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7fba84d94700 1 --2- 172.21.15.78:0/1844916915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba6c0041c0 0x7fba6c00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7fba84d94700 1 -- 172.21.15.78:0/1844916915 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba780082c0 con 0x7fba6c003810 2023-10-11T09:40:52.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7fafa869b700 1 -- 172.21.15.78:0/1994230678 wait complete. 2023-10-11T09:40:52.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7fba84d94700 1 --2- 172.21.15.78:0/1844916915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba6c003810 0x7fba6c003c80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fba7c00c290 tx=0x7fba6c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61643776ef5b57e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:52.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.134+0000 7f3772b16700 1 -- 172.21.15.78:0/1257455045 wait complete. 2023-10-11T09:40:52.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7f3772b16700 1 Processor -- start 2023-10-11T09:40:52.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7ff1288c7700 1 Processor -- start 2023-10-11T09:40:52.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7ff1288c7700 1 -- start start 2023-10-11T09:40:52.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7ff1288c7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff12406d5f0 0x7ff1240b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7fba77fff700 1 -- 172.21.15.78:0/1844916915 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7c007b90 con 0x7fba6c003810 2023-10-11T09:40:52.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7fba77fff700 1 -- 172.21.15.78:0/1844916915 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba7c00cd30 con 0x7fba6c003810 2023-10-11T09:40:52.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7ff1288c7700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff124148250 0x7ff1240b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7ff1288c7700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff12406d9c0 con 0x7ff124161090 2023-10-11T09:40:52.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7ff1288c7700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1240cb220 con 0x7ff124148250 2023-10-11T09:40:52.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7ff1288c7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff124161520 con 0x7ff12406d5f0 2023-10-11T09:40:52.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7fba77fff700 1 -- 172.21.15.78:0/1844916915 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7c00cea0 con 0x7fba6c003810 2023-10-11T09:40:52.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7f3772b16700 1 -- start start 2023-10-11T09:40:52.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7f3772b16700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37640383e0 0x7f37640dd670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7fafa869b700 1 Processor -- start 2023-10-11T09:40:52.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7fafa869b700 1 -- start start 2023-10-11T09:40:52.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7f3772b16700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f376403ac80 0x7f3764050b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7fafa869b700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faf98038d80 0x7faf980d8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7fafa869b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf9803c740 0x7faf981795d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7fafa869b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf980e4c50 0x7faf980e15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7fafa869b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf980e2d30 con 0x7faf980e4c50 2023-10-11T09:40:52.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7fafa869b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf98179b10 con 0x7faf9803c740 2023-10-11T09:40:52.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7fafa869b700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf980e2460 con 0x7faf98038d80 2023-10-11T09:40:52.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7f3772b16700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f376403b0d0 0x7f3764051090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7f3771b14700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37640383e0 0x7f37640dd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7f3771b14700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37640383e0 0x7f37640dd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:50316/0 (socket says 172.21.15.78:50316) 2023-10-11T09:40:52.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7ff122d9d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff12406d5f0 0x7ff1240b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7fba877f9700 1 -- 172.21.15.78:0/1844916915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba6c003810 msgr2=0x7fba6c003c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7fba877f9700 1 --2- 172.21.15.78:0/1844916915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba6c003810 0x7fba6c003c80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fba7c00c290 tx=0x7fba6c0058a0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7f3771313700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f376403ac80 0x7f3764050b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7f3771313700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f376403ac80 0x7f3764050b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:44436/0 (socket says 172.21.15.78:44436) 2023-10-11T09:40:52.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7fafa6c38700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf980e4c50 0x7faf980e15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7fafa6437700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faf98038d80 0x7faf980d8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.138+0000 7fafa6c38700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf980e4c50 0x7faf980e15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57224/0 (socket says 172.21.15.78:57224) 2023-10-11T09:40:52.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7fafa6c38700 1 -- 172.21.15.78:0/1534578069 learned_addr learned my addr 172.21.15.78:0/1534578069 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:52.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7fba85d96700 1 -- 172.21.15.78:0/1844916915 reap_dead start 2023-10-11T09:40:52.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7fba877f9700 1 -- 172.21.15.78:0/1844916915 shutdown_connections 2023-10-11T09:40:52.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7fba877f9700 1 -- 172.21.15.78:0/1844916915 >> 172.21.15.78:0/1844916915 conn(0x7fba78057b90 msgr2=0x7fba78053980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:52.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7f3772315700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f376403b0d0 0x7f3764051090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7f3772315700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f376403b0d0 0x7f3764051090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57226/0 (socket says 172.21.15.78:57226) 2023-10-11T09:40:52.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7fafa6437700 1 -- 172.21.15.78:0/1534578069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf9803c740 msgr2=0x7faf981795d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7fafa5c36700 1 --2- 172.21.15.78:0/1534578069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf9803c740 0x7faf981795d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7fafa6437700 1 --2- 172.21.15.78:0/1534578069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf9803c740 0x7faf981795d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7fafa6437700 1 -- 172.21.15.78:0/1534578069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf980e4c50 msgr2=0x7faf980e15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7fafa6437700 1 --2- 172.21.15.78:0/1534578069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf980e4c50 0x7faf980e15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7fafa6437700 1 -- 172.21.15.78:0/1534578069 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9c0094b0 con 0x7faf98038d80 2023-10-11T09:40:52.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7fc4b1959700 1 Processor -- start 2023-10-11T09:40:52.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7ff122d9d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff12406d5f0 0x7ff1240b06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:44428/0 (socket says 172.21.15.78:44428) 2023-10-11T09:40:52.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7fba877f9700 1 -- 172.21.15.78:0/1844916915 shutdown_connections 2023-10-11T09:40:52.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7fba877f9700 1 -- 172.21.15.78:0/1844916915 wait complete. 2023-10-11T09:40:52.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7ff12259c700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7ff124161090 0x7ff124147b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:58894/0 (socket says 172.21.15.78:58894) 2023-10-11T09:40:52.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7ff12259c700 1 -- 172.21.15.78:0/50775865 learned_addr learned my addr 172.21.15.78:0/50775865 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:52.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7ff12359e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff124148250 0x7ff1240b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7fafa6437700 1 --2- 172.21.15.78:0/1534578069 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faf98038d80 0x7faf980d8db0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7faf9c015000 tx=0x7faf9c004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:52.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7fc4b1959700 1 -- start start 2023-10-11T09:40:52.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7f3771b14700 1 -- 172.21.15.78:0/3961077610 learned_addr learned my addr 172.21.15.78:0/3961077610 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:52.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7f3772b16700 1 -- 172.21.15.78:0/3961077610 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f376403e110 con 0x7f376403b0d0 2023-10-11T09:40:52.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7f3772b16700 1 -- 172.21.15.78:0/3961077610 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f376403e3f0 con 0x7f37640383e0 2023-10-11T09:40:52.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7faf96ffd700 1 -- 172.21.15.78:0/1534578069 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9c011630 con 0x7faf98038d80 2023-10-11T09:40:52.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7faf96ffd700 1 -- 172.21.15.78:0/1534578069 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf9c01e070 con 0x7faf98038d80 2023-10-11T09:40:52.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7faf96ffd700 1 -- 172.21.15.78:0/1534578069 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9c0119f0 con 0x7faf98038d80 2023-10-11T09:40:52.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7fafa869b700 1 -- 172.21.15.78:0/1534578069 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf980e1ea0 con 0x7faf98038d80 2023-10-11T09:40:52.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7fafa869b700 1 -- 172.21.15.78:0/1534578069 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf980e2950 con 0x7faf98038d80 2023-10-11T09:40:52.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7f3772b16700 1 -- 172.21.15.78:0/3961077610 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f376403e6d0 con 0x7f376403ac80 2023-10-11T09:40:52.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7fa7e9a8a700 1 Processor -- start 2023-10-11T09:40:52.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7ff121d9b700 1 -- 172.21.15.78:0/50775865 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 777063483 0 0) 0x7ff114001110 con 0x7ff124161090 2023-10-11T09:40:52.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7ff121d9b700 1 -- 172.21.15.78:0/50775865 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff10c0035d0 con 0x7ff124161090 2023-10-11T09:40:52.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7fba877f9700 1 Processor -- start 2023-10-11T09:40:52.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7fba877f9700 1 -- start start 2023-10-11T09:40:52.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7fba877f9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba6c003810 0x7fba780e4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7fba877f9700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba780e2e70 0x7fba780e32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7fba877f9700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fba780d5aa0 0x7fba780d5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7fba877f9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba78050b50 con 0x7fba6c003810 2023-10-11T09:40:52.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7fba877f9700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba780e3820 con 0x7fba780e2e70 2023-10-11T09:40:52.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7fba877f9700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba7804ad10 con 0x7fba780d5aa0 2023-10-11T09:40:52.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7fba85d96700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fba780d5aa0 0x7fba780d5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7fba85d96700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fba780d5aa0 0x7fba780d5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:44450/0 (socket says 172.21.15.78:44450) 2023-10-11T09:40:52.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7fba85d96700 1 -- 172.21.15.78:0/1438777946 learned_addr learned my addr 172.21.15.78:0/1438777946 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:52.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7fba84d94700 1 --2- 172.21.15.78:0/1438777946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba780e2e70 0x7fba780e32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7fba85595700 1 --2- 172.21.15.78:0/1438777946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba6c003810 0x7fba780e4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7f3772315700 1 -- 172.21.15.78:0/3961077610 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f376403ac80 msgr2=0x7f3764050b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7f3772315700 1 --2- 172.21.15.78:0/3961077610 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f376403ac80 0x7f3764050b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7f3772315700 1 -- 172.21.15.78:0/3961077610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37640383e0 msgr2=0x7f37640dd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7f3772315700 1 --2- 172.21.15.78:0/3961077610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37640383e0 0x7f37640dd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7f3772315700 1 -- 172.21.15.78:0/3961077610 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f376c04eaa0 con 0x7f376403b0d0 2023-10-11T09:40:52.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7f3771313700 1 --2- 172.21.15.78:0/3961077610 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f376403ac80 0x7f3764050b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:40:52.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7f3771b14700 1 --2- 172.21.15.78:0/3961077610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37640383e0 0x7f37640dd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:40:52.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7f3772315700 1 --2- 172.21.15.78:0/3961077610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f376403b0d0 0x7f3764051090 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f376800b410 tx=0x7f376800b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:52.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7f37627fc700 1 -- 172.21.15.78:0/3961077610 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3768011620 con 0x7f376403b0d0 2023-10-11T09:40:52.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7f37627fc700 1 -- 172.21.15.78:0/3961077610 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37680117a0 con 0x7f376403b0d0 2023-10-11T09:40:52.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7f37627fc700 1 -- 172.21.15.78:0/3961077610 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3768011b50 con 0x7f376403b0d0 2023-10-11T09:40:52.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7f3772b16700 1 -- 172.21.15.78:0/3961077610 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f376403e9b0 con 0x7f376403b0d0 2023-10-11T09:40:52.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7f3772b16700 1 -- 172.21.15.78:0/3961077610 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37640ddbb0 con 0x7f376403b0d0 2023-10-11T09:40:52.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7f3772b16700 1 -- 172.21.15.78:0/3961077610 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3754005310 con 0x7f376403b0d0 2023-10-11T09:40:52.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7fafa869b700 1 -- 172.21.15.78:0/1534578069 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf88005310 con 0x7faf98038d80 2023-10-11T09:40:52.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7faf96ffd700 1 -- 172.21.15.78:0/1534578069 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7faf9c011b50 con 0x7faf98038d80 2023-10-11T09:40:52.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.150+0000 7faf96ffd700 1 --2- 172.21.15.78:0/1534578069 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7faf84065fa0 0x7faf84068450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.150+0000 7faf96ffd700 1 -- 172.21.15.78:0/1534578069 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faf9c0845f0 con 0x7faf98038d80 2023-10-11T09:40:52.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.150+0000 7fafa5c36700 1 --2- 172.21.15.78:0/1534578069 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7faf84065fa0 0x7faf84068450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7ff121d9b700 1 -- 172.21.15.78:0/50775865 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 223454291 0 0) 0x7ff10c0035d0 con 0x7ff124161090 2023-10-11T09:40:52.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7ff122d9d700 1 -- 172.21.15.78:0/50775865 >> v2:172.21.15.133:3300/0 conn(0x7ff124148250 msgr2=0x7ff1240b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:52.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7ff122d9d700 1 --2- 172.21.15.78:0/50775865 >> v2:172.21.15.133:3300/0 conn(0x7ff124148250 0x7ff1240b0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7ff122d9d700 1 -- 172.21.15.78:0/50775865 >> v1:172.21.15.78:6789/0 conn(0x7ff124161090 legacy=0x7ff124147b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7ff122d9d700 1 -- 172.21.15.78:0/50775865 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff124161800 con 0x7ff12406d5f0 2023-10-11T09:40:52.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7ff122d9d700 1 --2- 172.21.15.78:0/50775865 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff12406d5f0 0x7ff1240b06a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff118009880 tx=0x7ff118009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b3ce36007f8102 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:52.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7ff121d9b700 1 -- 172.21.15.78:0/50775865 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff114001800 con 0x7ff12406d5f0 2023-10-11T09:40:52.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7ff121d9b700 1 -- 172.21.15.78:0/50775865 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff118004030 con 0x7ff12406d5f0 2023-10-11T09:40:52.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7ff121d9b700 1 -- 172.21.15.78:0/50775865 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1180043c0 con 0x7ff12406d5f0 2023-10-11T09:40:52.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7ff1288c7700 1 -- 172.21.15.78:0/50775865 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff12406d5f0 msgr2=0x7ff1240b06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7ff1288c7700 1 --2- 172.21.15.78:0/50775865 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff12406d5f0 0x7ff1240b06a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff118009880 tx=0x7ff118009b90 comp rx=0 tx=0).stop 2023-10-11T09:40:52.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7ff1288c7700 1 -- 172.21.15.78:0/50775865 shutdown_connections 2023-10-11T09:40:52.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7ff1288c7700 1 --2- 172.21.15.78:0/50775865 >> v2:172.21.15.133:3300/0 conn(0x7ff124148250 0x7ff1240b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7ff1288c7700 1 --2- 172.21.15.78:0/50775865 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff12406d5f0 0x7ff1240b06a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7ff1288c7700 1 -- 172.21.15.78:0/50775865 >> 172.21.15.78:0/50775865 conn(0x7ff12406aca0 msgr2=0x7ff1240cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:52.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.150+0000 7ff1288c7700 1 -- 172.21.15.78:0/50775865 shutdown_connections 2023-10-11T09:40:52.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.150+0000 7ff1288c7700 1 -- 172.21.15.78:0/50775865 wait complete. 2023-10-11T09:40:52.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7fa7e9a8a700 1 -- start start 2023-10-11T09:40:52.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7fa7e9a8a700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa7e406d380 0x7fa7e40b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7fa7e9a8a700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7e40b0fe0 con 0x7fa7e40b0be0 2023-10-11T09:40:52.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7fa7e9a8a700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7e406c440 con 0x7fa7e41586e0 2023-10-11T09:40:52.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7fa7e9a8a700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7e406c720 con 0x7fa7e406d380 2023-10-11T09:40:52.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7fa7e37fe700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa7e41586e0 0x7fa7e406bbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:33912/0 (socket says 172.21.15.78:33912) 2023-10-11T09:40:52.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7fa7e37fe700 1 -- 172.21.15.78:0/1628865631 learned_addr learned my addr 172.21.15.78:0/1628865631 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:52.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7fa7e2ffd700 1 --2- 172.21.15.78:0/1628865631 >> v2:172.21.15.133:3300/0 conn(0x7fa7e406d380 0x7fa7e40b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7fa7e27fc700 1 -- 172.21.15.78:0/1628865631 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1144238438 0 0) 0x7fa7e406c440 con 0x7fa7e41586e0 2023-10-11T09:40:52.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7fa7e27fc700 1 -- 172.21.15.78:0/1628865631 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7cc0035d0 con 0x7fa7e41586e0 2023-10-11T09:40:52.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7fa7e2ffd700 1 -- 172.21.15.78:0/1628865631 >> v1:172.21.15.133:6789/0 conn(0x7fa7e41586e0 legacy=0x7fa7e406bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7fa7e2ffd700 1 -- 172.21.15.78:0/1628865631 >> v1:172.21.15.78:6789/0 conn(0x7fa7e40b0be0 legacy=0x7fa7e4070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:52.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7fa7e2ffd700 1 -- 172.21.15.78:0/1628865631 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e406ca00 con 0x7fa7e406d380 2023-10-11T09:40:52.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.150+0000 7fa7e2ffd700 1 --2- 172.21.15.78:0/1628865631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7e406d380 0x7fa7e40b06a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4009850 tx=0x7fa7d4009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ce61186d57aee62 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:52.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.150+0000 7fa7e27fc700 1 -- 172.21.15.78:0/1628865631 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d4004230 con 0x7fa7e406d380 2023-10-11T09:40:52.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.150+0000 7fa7e27fc700 1 -- 172.21.15.78:0/1628865631 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d4004390 con 0x7fa7e406d380 2023-10-11T09:40:52.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.150+0000 7fa7e27fc700 1 -- 172.21.15.78:0/1628865631 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d40036c0 con 0x7fa7e406d380 2023-10-11T09:40:52.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.150+0000 7fa7e9a8a700 1 -- 172.21.15.78:0/1628865631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7e406d380 msgr2=0x7fa7e40b06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.150+0000 7fa7e9a8a700 1 --2- 172.21.15.78:0/1628865631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7e406d380 0x7fa7e40b06a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4009850 tx=0x7fa7d4009b60 comp rx=0 tx=0).stop 2023-10-11T09:40:52.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.150+0000 7fa7e9a8a700 1 -- 172.21.15.78:0/1628865631 shutdown_connections 2023-10-11T09:40:52.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.150+0000 7fa7e9a8a700 1 --2- 172.21.15.78:0/1628865631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7e406d380 0x7fa7e40b06a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.150+0000 7fa7e9a8a700 1 -- 172.21.15.78:0/1628865631 >> 172.21.15.78:0/1628865631 conn(0x7fa7e406aca0 msgr2=0x7fa7e40cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:52.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.150+0000 7fa7e9a8a700 1 -- 172.21.15.78:0/1628865631 shutdown_connections 2023-10-11T09:40:52.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7fc4b1959700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc4ac0cf170 0x7fc4ac06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7fc4b1959700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4ac06c720 con 0x7fc4ac06d380 2023-10-11T09:40:52.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7fc4b1959700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4ac06ca00 con 0x7fc4ac0b06a0 2023-10-11T09:40:52.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.142+0000 7fc4b1959700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4ac15f710 con 0x7fc4ac0cf170 2023-10-11T09:40:52.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7fc4aaffd700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc4ac0cf170 0x7fc4ac06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7fc4aa7fc700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fc4ac06d380 0x7fc4ac158470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:58910/0 (socket says 172.21.15.78:58910) 2023-10-11T09:40:52.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7fc4aa7fc700 1 -- 172.21.15.78:0/1483698307 learned_addr learned my addr 172.21.15.78:0/1483698307 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:52.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7fc4aaffd700 1 -- 172.21.15.78:0/1483698307 >> v1:172.21.15.133:6789/0 conn(0x7fc4ac0b06a0 legacy=0x7fc4ac0b0aa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-11T09:40:52.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7fc4aaffd700 1 -- 172.21.15.78:0/1483698307 >> v1:172.21.15.78:6789/0 conn(0x7fc4ac06d380 legacy=0x7fc4ac158470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7fc4aaffd700 1 -- 172.21.15.78:0/1483698307 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4ac06fd10 con 0x7fc4ac0cf170 2023-10-11T09:40:52.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.150+0000 7f37627fc700 1 -- 172.21.15.78:0/3961077610 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f3768011cb0 con 0x7f376403b0d0 2023-10-11T09:40:52.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.150+0000 7f37627fc700 1 --2- 172.21.15.78:0/3961077610 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3750065fa0 0x7f3750068450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.150+0000 7f37627fc700 1 -- 172.21.15.78:0/3961077610 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3768083990 con 0x7f376403b0d0 2023-10-11T09:40:52.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.150+0000 7fafa5c36700 1 --2- 172.21.15.78:0/1534578069 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7faf84065fa0 0x7faf84068450 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7faf98056060 tx=0x7faf90008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:52.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.150+0000 7ff1288c7700 1 Processor -- start 2023-10-11T09:40:52.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.150+0000 7fa7e9a8a700 1 -- 172.21.15.78:0/1628865631 wait complete. 2023-10-11T09:40:52.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.150+0000 7ff1288c7700 1 -- start start 2023-10-11T09:40:52.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7fba85d96700 1 -- 172.21.15.78:0/1438777946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba780e2e70 msgr2=0x7fba780e32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7fba85d96700 1 --2- 172.21.15.78:0/1438777946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba780e2e70 0x7fba780e32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7fba85d96700 1 -- 172.21.15.78:0/1438777946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba6c003810 msgr2=0x7fba780e4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7fba85d96700 1 --2- 172.21.15.78:0/1438777946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba6c003810 0x7fba780e4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7fba85d96700 1 -- 172.21.15.78:0/1438777946 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba780d6af0 con 0x7fba780d5aa0 2023-10-11T09:40:52.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7fba85d96700 1 --2- 172.21.15.78:0/1438777946 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fba780d5aa0 0x7fba780d5f10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fba7803ce50 tx=0x7fba70011bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:52.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.150+0000 7fba75ffb700 1 -- 172.21.15.78:0/1438777946 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba70004300 con 0x7fba780d5aa0 2023-10-11T09:40:52.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.150+0000 7fba75ffb700 1 -- 172.21.15.78:0/1438777946 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba70004480 con 0x7fba780d5aa0 2023-10-11T09:40:52.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.150+0000 7fba75ffb700 1 -- 172.21.15.78:0/1438777946 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba70003900 con 0x7fba780d5aa0 2023-10-11T09:40:52.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.150+0000 7fba877f9700 1 -- 172.21.15.78:0/1438777946 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba780e53e0 con 0x7fba780d5aa0 2023-10-11T09:40:52.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.150+0000 7fba877f9700 1 -- 172.21.15.78:0/1438777946 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba780e5980 con 0x7fba780d5aa0 2023-10-11T09:40:52.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.150+0000 7fba877f9700 1 -- 172.21.15.78:0/1438777946 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba78005980 con 0x7fba780d5aa0 2023-10-11T09:40:52.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.146+0000 7fc4aaffd700 1 --2- 172.21.15.78:0/1483698307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4ac0cf170 0x7fc4ac06bbf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4009d90 tx=0x7fc4a400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7f0b0e7561c6349 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:52.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.154+0000 7fc4a9ffb700 1 -- 172.21.15.78:0/1483698307 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a400e070 con 0x7fc4ac0cf170 2023-10-11T09:40:52.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.154+0000 7fc4a9ffb700 1 -- 172.21.15.78:0/1483698307 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4a400bb90 con 0x7fc4ac0cf170 2023-10-11T09:40:52.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.154+0000 7fc4a9ffb700 1 -- 172.21.15.78:0/1483698307 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4003a40 con 0x7fc4ac0cf170 2023-10-11T09:40:52.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.154+0000 7ff1288c7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff12406d5f0 0x7ff1240ae560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.154+0000 7ff1288c7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff124148250 0x7ff124144610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.154+0000 7ff1288c7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff124161090 0x7ff124144b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.154+0000 7ff1288c7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1240706c0 con 0x7ff124148250 2023-10-11T09:40:52.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.154+0000 7ff1288c7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1241441b0 con 0x7ff124161090 2023-10-11T09:40:52.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.154+0000 7ff1288c7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1240af730 con 0x7ff12406d5f0 2023-10-11T09:40:52.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.154+0000 7fa7e9a8a700 1 Processor -- start 2023-10-11T09:40:52.218 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.154+0000 7f3771b14700 1 --2- 172.21.15.78:0/3961077610 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3750065fa0 0x7f3750068450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.218 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.154+0000 7fc4b1959700 1 -- 172.21.15.78:0/1483698307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4ac0cf170 msgr2=0x7fc4ac06bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.154+0000 7fc4b1959700 1 --2- 172.21.15.78:0/1483698307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4ac0cf170 0x7fc4ac06bbf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4009d90 tx=0x7fc4a400b330 comp rx=0 tx=0).stop 2023-10-11T09:40:52.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.154+0000 7ff12259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff124148250 0x7ff124144610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.154+0000 7ff122d9d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff12406d5f0 0x7ff1240ae560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.154+0000 7ff12259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff124148250 0x7ff124144610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57234/0 (socket says 172.21.15.78:57234) 2023-10-11T09:40:52.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.154+0000 7ff12259c700 1 -- 172.21.15.78:0/643009385 learned_addr learned my addr 172.21.15.78:0/643009385 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:52.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.154+0000 7fa7e9a8a700 1 -- start start 2023-10-11T09:40:52.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.154+0000 7ff122d9d700 1 -- 172.21.15.78:0/643009385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff124161090 msgr2=0x7ff124144b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.154+0000 7ff122d9d700 1 --2- 172.21.15.78:0/643009385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff124161090 0x7ff124144b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.154+0000 7ff122d9d700 1 -- 172.21.15.78:0/643009385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff124148250 msgr2=0x7ff124144610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.154+0000 7ff122d9d700 1 --2- 172.21.15.78:0/643009385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff124148250 0x7ff124144610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.154+0000 7ff122d9d700 1 -- 172.21.15.78:0/643009385 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1180094b0 con 0x7ff12406d5f0 2023-10-11T09:40:52.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.154+0000 7ff122d9d700 1 --2- 172.21.15.78:0/643009385 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff12406d5f0 0x7ff1240ae560 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff118013000 tx=0x7ff118004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:52.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.154+0000 7f3771b14700 1 --2- 172.21.15.78:0/3961077610 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3750065fa0 0x7f3750068450 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f376c063e50 tx=0x7f376c061ce0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:52.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.154+0000 7fc4b1959700 1 -- 172.21.15.78:0/1483698307 shutdown_connections 2023-10-11T09:40:52.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.154+0000 7fc4b1959700 1 --2- 172.21.15.78:0/1483698307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4ac0cf170 0x7fc4ac06bbf0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.154+0000 7fc4b1959700 1 -- 172.21.15.78:0/1483698307 >> 172.21.15.78:0/1483698307 conn(0x7fc4ac06aca0 msgr2=0x7fc4ac0cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:52.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.154+0000 7fc4b1959700 1 -- 172.21.15.78:0/1483698307 shutdown_connections 2023-10-11T09:40:52.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.154+0000 7faf96ffd700 1 -- 172.21.15.78:0/1534578069 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7faf9c051c60 con 0x7faf98038d80 2023-10-11T09:40:52.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.154+0000 7fc4b1959700 1 -- 172.21.15.78:0/1483698307 wait complete. 2023-10-11T09:40:52.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.154+0000 7fa7e9a8a700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa7e406d380 0x7fa7e4154a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.154+0000 7fa7e9a8a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7e40b0be0 0x7fa7e413c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.154+0000 7fa7e37fe700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa7e406d380 0x7fa7e4154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.154+0000 7fa7e37fe700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa7e406d380 0x7fa7e4154a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:44462/0 (socket says 172.21.15.78:44462) 2023-10-11T09:40:52.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.154+0000 7fa7e9a8a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7e41586e0 0x7fa7e4144860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.158+0000 7ff10b7fe700 1 -- 172.21.15.78:0/643009385 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff118003dc0 con 0x7ff12406d5f0 2023-10-11T09:40:52.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.158+0000 7ff10b7fe700 1 -- 172.21.15.78:0/643009385 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff118011630 con 0x7ff12406d5f0 2023-10-11T09:40:52.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.158+0000 7ff10b7fe700 1 -- 172.21.15.78:0/643009385 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1180119a0 con 0x7ff12406d5f0 2023-10-11T09:40:52.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.158+0000 7fa7e2ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7e40b0be0 0x7fa7e413c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.158+0000 7fa7e2ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7e40b0be0 0x7fa7e413c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:50370/0 (socket says 172.21.15.78:50370) 2023-10-11T09:40:52.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.158+0000 7fa7e37fe700 1 -- 172.21.15.78:0/2938010791 learned_addr learned my addr 172.21.15.78:0/2938010791 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:52.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.158+0000 7fa7e9a8a700 1 -- 172.21.15.78:0/2938010791 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e41545d0 con 0x7fa7e41586e0 2023-10-11T09:40:52.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.158+0000 7fa7e9a8a700 1 -- 172.21.15.78:0/2938010791 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e41539c0 con 0x7fa7e40b0be0 2023-10-11T09:40:52.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.158+0000 7fa7e9a8a700 1 -- 172.21.15.78:0/2938010791 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e4153ca0 con 0x7fa7e406d380 2023-10-11T09:40:52.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.158+0000 7fa7e3fff700 1 --2- 172.21.15.78:0/2938010791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7e41586e0 0x7fa7e4144860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.158+0000 7ff1288c7700 1 -- 172.21.15.78:0/643009385 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1240af450 con 0x7ff12406d5f0 2023-10-11T09:40:52.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.158+0000 7f37627fc700 1 -- 172.21.15.78:0/3961077610 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f3768051050 con 0x7f376403b0d0 2023-10-11T09:40:52.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.158+0000 7fba75ffb700 1 -- 172.21.15.78:0/1438777946 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fba70003a60 con 0x7fba780d5aa0 2023-10-11T09:40:52.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.158+0000 7ff1288c7700 1 -- 172.21.15.78:0/643009385 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1240702e0 con 0x7ff12406d5f0 2023-10-11T09:40:52.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.158+0000 7ff1288c7700 1 -- 172.21.15.78:0/643009385 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff12406fe60 con 0x7ff12406d5f0 2023-10-11T09:40:52.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.158+0000 7fa7e3fff700 1 -- 172.21.15.78:0/2938010791 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa7e406d380 msgr2=0x7fa7e4154a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.158+0000 7fa7e3fff700 1 --2- 172.21.15.78:0/2938010791 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa7e406d380 0x7fa7e4154a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.158+0000 7fa7e3fff700 1 -- 172.21.15.78:0/2938010791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7e40b0be0 msgr2=0x7fa7e413c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.158+0000 7fa7e3fff700 1 --2- 172.21.15.78:0/2938010791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7e40b0be0 0x7fa7e413c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.158+0000 7fa7e3fff700 1 -- 172.21.15.78:0/2938010791 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d40094b0 con 0x7fa7e41586e0 2023-10-11T09:40:52.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.158+0000 7fa7e3fff700 1 --2- 172.21.15.78:0/2938010791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7e41586e0 0x7fa7e4144860 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa7d800b410 tx=0x7fa7d800b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:52.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.158+0000 7fa7cbfff700 1 -- 172.21.15.78:0/2938010791 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8011620 con 0x7fa7e41586e0 2023-10-11T09:40:52.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.158+0000 7fa7cbfff700 1 -- 172.21.15.78:0/2938010791 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d80117a0 con 0x7fa7e41586e0 2023-10-11T09:40:52.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.158+0000 7fa7cbfff700 1 -- 172.21.15.78:0/2938010791 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8011b50 con 0x7fa7e41586e0 2023-10-11T09:40:52.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.158+0000 7fc4b1959700 1 Processor -- start 2023-10-11T09:40:52.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.162+0000 7fa7e9a8a700 1 -- 172.21.15.78:0/2938010791 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e4153f80 con 0x7fa7e41586e0 2023-10-11T09:40:52.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.162+0000 7fc4b1959700 1 -- start start 2023-10-11T09:40:52.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.162+0000 7fba75ffb700 1 --2- 172.21.15.78:0/1438777946 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fba64065fd0 0x7fba64068480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.162+0000 7fba75ffb700 1 -- 172.21.15.78:0/1438777946 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fba70083860 con 0x7fba780d5aa0 2023-10-11T09:40:52.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.162+0000 7fba85595700 1 --2- 172.21.15.78:0/1438777946 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fba64065fd0 0x7fba64068480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.162+0000 7fc4b1959700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4ac06d380 0x7fc4ac141cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.162+0000 7fc4b1959700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc4ac0b06a0 0x7fc4ac1576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.162+0000 7fc4b1959700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4ac0cf170 0x7fc4ac0cdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.162+0000 7fc4aaffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4ac06d380 0x7fc4ac141cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.162+0000 7fc4aaffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4ac06d380 0x7fc4ac141cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:50380/0 (socket says 172.21.15.78:50380) 2023-10-11T09:40:52.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.162+0000 7fba85595700 1 --2- 172.21.15.78:0/1438777946 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fba64065fd0 0x7fba64068480 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fba8004f2b0 tx=0x7fba80071040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:52.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.162+0000 7fc4ab7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4ac0cf170 0x7fc4ac0cdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.162+0000 7fc4ab7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4ac0cf170 0x7fc4ac0cdfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57240/0 (socket says 172.21.15.78:57240) 2023-10-11T09:40:52.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.162+0000 7fc4b1959700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ac1418d0 con 0x7fc4ac0cf170 2023-10-11T09:40:52.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.162+0000 7fc4b1959700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ac142230 con 0x7fc4ac06d380 2023-10-11T09:40:52.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.162+0000 7fc4b1959700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ac157c30 con 0x7fc4ac0b06a0 2023-10-11T09:40:52.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.162+0000 7fc4aaffd700 1 -- 172.21.15.78:0/3843458171 learned_addr learned my addr 172.21.15.78:0/3843458171 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:52.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.162+0000 7fc4aaffd700 1 -- 172.21.15.78:0/3843458171 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc4ac0b06a0 msgr2=0x7fc4ac1576f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-11T09:40:52.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.162+0000 7fc4aaffd700 1 --2- 172.21.15.78:0/3843458171 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc4ac0b06a0 0x7fc4ac1576f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.162+0000 7fc4aaffd700 1 -- 172.21.15.78:0/3843458171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4ac0cf170 msgr2=0x7fc4ac0cdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.162+0000 7fc4aaffd700 1 --2- 172.21.15.78:0/3843458171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4ac0cf170 0x7fc4ac0cdfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.162+0000 7fc4aaffd700 1 -- 172.21.15.78:0/3843458171 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a4009a70 con 0x7fc4ac06d380 2023-10-11T09:40:52.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.162+0000 7fba75ffb700 1 -- 172.21.15.78:0/1438777946 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fba70050e90 con 0x7fba780d5aa0 2023-10-11T09:40:52.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.162+0000 7fc4aaffd700 1 --2- 172.21.15.78:0/3843458171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4ac06d380 0x7fc4ac141cf0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4000f80 tx=0x7fc4a4004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:52.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.162+0000 7ff10b7fe700 1 -- 172.21.15.78:0/643009385 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7ff118018050 con 0x7ff12406d5f0 2023-10-11T09:40:52.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.166+0000 7fc493fff700 1 -- 172.21.15.78:0/3843458171 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4012630 con 0x7fc4ac06d380 2023-10-11T09:40:52.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.162+0000 7fa7cbfff700 1 -- 172.21.15.78:0/2938010791 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fa7d8011cb0 con 0x7fa7e41586e0 2023-10-11T09:40:52.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.166+0000 7fa7cbfff700 1 --2- 172.21.15.78:0/2938010791 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa7d0065f20 0x7fa7d00683d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.166+0000 7fc493fff700 1 -- 172.21.15.78:0/3843458171 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4a4012790 con 0x7fc4ac06d380 2023-10-11T09:40:52.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.166+0000 7fc493fff700 1 -- 172.21.15.78:0/3843458171 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4012b00 con 0x7fc4ac06d380 2023-10-11T09:40:52.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.166+0000 7fc4b1959700 1 -- 172.21.15.78:0/3843458171 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4ac0ce4e0 con 0x7fc4ac06d380 2023-10-11T09:40:52.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.166+0000 7fc4b1959700 1 -- 172.21.15.78:0/3843458171 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4ac13fab0 con 0x7fc4ac06d380 2023-10-11T09:40:52.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.166+0000 7fa7e9a8a700 1 -- 172.21.15.78:0/2938010791 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e4153650 con 0x7fa7e41586e0 2023-10-11T09:40:52.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.166+0000 7fa7e37fe700 1 --2- 172.21.15.78:0/2938010791 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa7d0065f20 0x7fa7d00683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.166+0000 7ff10b7fe700 1 --2- 172.21.15.78:0/643009385 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff110065f20 0x7ff1100683d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.166+0000 7ff10b7fe700 1 -- 172.21.15.78:0/643009385 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff1180843f0 con 0x7ff12406d5f0 2023-10-11T09:40:52.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.166+0000 7fa7cbfff700 1 -- 172.21.15.78:0/2938010791 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa7d8054250 con 0x7fa7e41586e0 2023-10-11T09:40:52.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.166+0000 7ff12259c700 1 --2- 172.21.15.78:0/643009385 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff110065f20 0x7ff1100683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.166+0000 7ff10b7fe700 1 -- 172.21.15.78:0/643009385 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7ff118051ab0 con 0x7ff12406d5f0 2023-10-11T09:40:52.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.166+0000 7ff12259c700 1 --2- 172.21.15.78:0/643009385 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff110065f20 0x7ff1100683d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff124153930 tx=0x7ff114001230 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:52.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.166+0000 7fa7e37fe700 1 --2- 172.21.15.78:0/2938010791 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa7d0065f20 0x7fa7d00683d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc004e00 tx=0x7fa7dc00a040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:52.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.170+0000 7fc4b1959700 1 -- 172.21.15.78:0/3843458171 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4ac141b40 con 0x7fc4ac06d380 2023-10-11T09:40:52.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.170+0000 7fa7c9ffb700 1 -- 172.21.15.78:0/2938010791 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e4154740 con 0x7fa7e41586e0 2023-10-11T09:40:52.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.170+0000 7fc493fff700 1 -- 172.21.15.78:0/3843458171 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fc4a4012c60 con 0x7fc4ac06d380 2023-10-11T09:40:52.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.170+0000 7fc493fff700 1 --2- 172.21.15.78:0/3843458171 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc498065ed0 0x7fc498068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.170+0000 7fc493fff700 1 -- 172.21.15.78:0/3843458171 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc4a4085740 con 0x7fc4ac06d380 2023-10-11T09:40:52.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.170+0000 7fc4aa7fc700 1 --2- 172.21.15.78:0/3843458171 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc498065ed0 0x7fc498068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.174+0000 7fc4aa7fc700 1 --2- 172.21.15.78:0/3843458171 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc498065ed0 0x7fc498068380 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac0cdaa0 tx=0x7fc4a0007af0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:52.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.178+0000 7fa7cbfff700 1 -- 172.21.15.78:0/2938010791 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fa7d8015070 con 0x7fa7e41586e0 2023-10-11T09:40:52.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.178+0000 7fc493fff700 1 -- 172.21.15.78:0/3843458171 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fc4a4056780 con 0x7fc4ac06d380 2023-10-11T09:40:52.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.366+0000 7fa37f619700 1 -- 172.21.15.78:0/4055854975 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa3580055c0 con 0x7fa378151dd0 2023-10-11T09:40:52.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.366+0000 7fa36dffb700 1 -- 172.21.15.78:0/4055854975 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa368021070 con 0x7fa378151dd0 2023-10-11T09:40:52.376 INFO:teuthology.orchestra.run.smithi078.stdout:55834574854 2023-10-11T09:40:52.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.370+0000 7fa37f619700 1 -- 172.21.15.78:0/4055854975 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa35c065fa0 msgr2=0x7fa35c068450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.370+0000 7fa37f619700 1 --2- 172.21.15.78:0/4055854975 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa35c065fa0 0x7fa35c068450 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa3700076e0 tx=0x7fa3700071a0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.370+0000 7fa37f619700 1 -- 172.21.15.78:0/4055854975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa378151dd0 msgr2=0x7fa37814d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.370+0000 7fa37f619700 1 --2- 172.21.15.78:0/4055854975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa378151dd0 0x7fa37814d880 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa36800e870 tx=0x7fa36800eb80 comp rx=0 tx=0).stop 2023-10-11T09:40:52.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.370+0000 7fa37f619700 1 -- 172.21.15.78:0/4055854975 shutdown_connections 2023-10-11T09:40:52.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.370+0000 7fa37f619700 1 --2- 172.21.15.78:0/4055854975 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa35c065fa0 0x7fa35c068450 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.370+0000 7fa37f619700 1 --2- 172.21.15.78:0/4055854975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa378151dd0 0x7fa37814d880 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.374+0000 7fa37f619700 1 --2- 172.21.15.78:0/4055854975 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3781586e0 0x7fa378202cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.374+0000 7fa37f619700 1 --2- 172.21.15.78:0/4055854975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3780b0be0 0x7fa3781fe6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.374+0000 7fa37f619700 1 -- 172.21.15.78:0/4055854975 >> 172.21.15.78:0/4055854975 conn(0x7fa37806aca0 msgr2=0x7fa3780afef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:52.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.374+0000 7fa37f619700 1 -- 172.21.15.78:0/4055854975 shutdown_connections 2023-10-11T09:40:52.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.374+0000 7fa37f619700 1 -- 172.21.15.78:0/4055854975 wait complete. 2023-10-11T09:40:52.403 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2023-10-11T09:40:52.404 DEBUG:teuthology.parallel:result is None 2023-10-11T09:40:52.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.454+0000 7fa7c9ffb700 1 -- 172.21.15.78:0/2938010791 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fa7e4065d40 con 0x7fa7e41586e0 2023-10-11T09:40:52.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.454+0000 7fa7cbfff700 1 -- 172.21.15.78:0/2938010791 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa7d804fe90 con 0x7fa7e41586e0 2023-10-11T09:40:52.460 INFO:teuthology.orchestra.run.smithi078.stdout:55834574855 2023-10-11T09:40:52.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.458+0000 7fa7c9ffb700 1 -- 172.21.15.78:0/2938010791 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa7d0065f20 msgr2=0x7fa7d00683d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.458+0000 7fa7c9ffb700 1 --2- 172.21.15.78:0/2938010791 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa7d0065f20 0x7fa7d00683d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc004e00 tx=0x7fa7dc00a040 comp rx=0 tx=0).stop 2023-10-11T09:40:52.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.458+0000 7fa7c9ffb700 1 -- 172.21.15.78:0/2938010791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7e41586e0 msgr2=0x7fa7e4144860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.458+0000 7fa7c9ffb700 1 --2- 172.21.15.78:0/2938010791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7e41586e0 0x7fa7e4144860 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa7d800b410 tx=0x7fa7d800b720 comp rx=0 tx=0).stop 2023-10-11T09:40:52.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.458+0000 7fa7c9ffb700 1 -- 172.21.15.78:0/2938010791 shutdown_connections 2023-10-11T09:40:52.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.458+0000 7fa7c9ffb700 1 --2- 172.21.15.78:0/2938010791 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa7d0065f20 0x7fa7d00683d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.458+0000 7fa7c9ffb700 1 --2- 172.21.15.78:0/2938010791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7e41586e0 0x7fa7e4144860 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.458+0000 7fa7c9ffb700 1 --2- 172.21.15.78:0/2938010791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7e40b0be0 0x7fa7e413c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.458+0000 7fa7c9ffb700 1 --2- 172.21.15.78:0/2938010791 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa7e406d380 0x7fa7e4154a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.458+0000 7fa7c9ffb700 1 -- 172.21.15.78:0/2938010791 >> 172.21.15.78:0/2938010791 conn(0x7fa7e406aca0 msgr2=0x7fa7e406bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:52.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.458+0000 7fa7c9ffb700 1 -- 172.21.15.78:0/2938010791 shutdown_connections 2023-10-11T09:40:52.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.458+0000 7fa7c9ffb700 1 -- 172.21.15.78:0/2938010791 wait complete. 2023-10-11T09:40:52.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.458+0000 7f50a0243700 1 -- 172.21.15.78:0/2648874592 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f50800055c0 con 0x7f50980b0be0 2023-10-11T09:40:52.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.462+0000 7f508e7fc700 1 -- 172.21.15.78:0/2648874592 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5088055120 con 0x7f50980b0be0 2023-10-11T09:40:52.466 INFO:teuthology.orchestra.run.smithi078.stdout:55834574856 2023-10-11T09:40:52.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.462+0000 7f50a0243700 1 -- 172.21.15.78:0/2648874592 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f507c065fa0 msgr2=0x7f507c068450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.466+0000 7f50a0243700 1 --2- 172.21.15.78:0/2648874592 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f507c065fa0 0x7f507c068450 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f50980b0b70 tx=0x7f5094009040 comp rx=0 tx=0).stop 2023-10-11T09:40:52.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.466+0000 7f50a0243700 1 -- 172.21.15.78:0/2648874592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50980b0be0 msgr2=0x7f50980b1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.466+0000 7f50a0243700 1 --2- 172.21.15.78:0/2648874592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50980b0be0 0x7f50980b1eb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5088000c00 tx=0x7f5088004770 comp rx=0 tx=0).stop 2023-10-11T09:40:52.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.466+0000 7f50a0243700 1 -- 172.21.15.78:0/2648874592 shutdown_connections 2023-10-11T09:40:52.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.466+0000 7f50a0243700 1 --2- 172.21.15.78:0/2648874592 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f507c065fa0 0x7f507c068450 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.466+0000 7f50a0243700 1 --2- 172.21.15.78:0/2648874592 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f509816d720 0x7f509820ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.466+0000 7f50a0243700 1 --2- 172.21.15.78:0/2648874592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50981670b0 0x7f509820a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.466+0000 7f50a0243700 1 --2- 172.21.15.78:0/2648874592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50980b0be0 0x7f50980b1eb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.466+0000 7f50a0243700 1 -- 172.21.15.78:0/2648874592 >> 172.21.15.78:0/2648874592 conn(0x7f509806aca0 msgr2=0x7f50981662e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:52.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.466+0000 7f50a0243700 1 -- 172.21.15.78:0/2648874592 shutdown_connections 2023-10-11T09:40:52.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.466+0000 7f50a0243700 1 -- 172.21.15.78:0/2648874592 wait complete. 2023-10-11T09:40:52.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.466+0000 7ff1288c7700 1 -- 172.21.15.78:0/643009385 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff124070800 con 0x7ff12406d5f0 2023-10-11T09:40:52.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.470+0000 7ff10b7fe700 1 -- 172.21.15.78:0/643009385 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff11802b020 con 0x7ff12406d5f0 2023-10-11T09:40:52.474 INFO:teuthology.orchestra.run.smithi078.stdout:55834574855 2023-10-11T09:40:52.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.470+0000 7ff1288c7700 1 -- 172.21.15.78:0/643009385 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff110065f20 msgr2=0x7ff1100683d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.470+0000 7ff1288c7700 1 --2- 172.21.15.78:0/643009385 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff110065f20 0x7ff1100683d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff124153930 tx=0x7ff114001230 comp rx=0 tx=0).stop 2023-10-11T09:40:52.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.470+0000 7ff1288c7700 1 -- 172.21.15.78:0/643009385 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff12406d5f0 msgr2=0x7ff1240ae560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.470+0000 7ff1288c7700 1 --2- 172.21.15.78:0/643009385 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff12406d5f0 0x7ff1240ae560 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff118013000 tx=0x7ff118004770 comp rx=0 tx=0).stop 2023-10-11T09:40:52.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.470+0000 7ff1288c7700 1 -- 172.21.15.78:0/643009385 shutdown_connections 2023-10-11T09:40:52.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.470+0000 7ff1288c7700 1 --2- 172.21.15.78:0/643009385 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff110065f20 0x7ff1100683d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.470+0000 7ff1288c7700 1 --2- 172.21.15.78:0/643009385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff124161090 0x7ff124144b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.470+0000 7ff1288c7700 1 --2- 172.21.15.78:0/643009385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff124148250 0x7ff124144610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.470+0000 7ff1288c7700 1 --2- 172.21.15.78:0/643009385 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff12406d5f0 0x7ff1240ae560 secure :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff118013000 tx=0x7ff118004770 comp rx=0 tx=0).stop 2023-10-11T09:40:52.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.470+0000 7ff1288c7700 1 -- 172.21.15.78:0/643009385 >> 172.21.15.78:0/643009385 conn(0x7ff12406aca0 msgr2=0x7ff124146a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:52.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.470+0000 7ff1288c7700 1 -- 172.21.15.78:0/643009385 shutdown_connections 2023-10-11T09:40:52.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.470+0000 7ff1288c7700 1 -- 172.21.15.78:0/643009385 wait complete. 2023-10-11T09:40:52.484 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2023-10-11T09:40:52.485 DEBUG:teuthology.parallel:result is None 2023-10-11T09:40:52.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.486+0000 7fba877f9700 1 -- 172.21.15.78:0/1438777946 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fba7804ae50 con 0x7fba780d5aa0 2023-10-11T09:40:52.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.486+0000 7fba75ffb700 1 -- 172.21.15.78:0/1438777946 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fba700544b0 con 0x7fba780d5aa0 2023-10-11T09:40:52.491 INFO:teuthology.orchestra.run.smithi078.stdout:55834574854 2023-10-11T09:40:52.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.490+0000 7f3772b16700 1 -- 172.21.15.78:0/3961077610 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f3754005890 con 0x7f376403b0d0 2023-10-11T09:40:52.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.490+0000 7fafa869b700 1 -- 172.21.15.78:0/1534578069 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7faf880055c0 con 0x7faf98038d80 2023-10-11T09:40:52.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.490+0000 7f37627fc700 1 -- 172.21.15.78:0/3961077610 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3768015020 con 0x7f376403b0d0 2023-10-11T09:40:52.495 INFO:teuthology.orchestra.run.smithi078.stdout:55834574856 2023-10-11T09:40:52.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.490+0000 7fba877f9700 1 -- 172.21.15.78:0/1438777946 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fba64065fd0 msgr2=0x7fba64068480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.490+0000 7fba877f9700 1 --2- 172.21.15.78:0/1438777946 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fba64065fd0 0x7fba64068480 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fba8004f2b0 tx=0x7fba80071040 comp rx=0 tx=0).stop 2023-10-11T09:40:52.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.490+0000 7fba877f9700 1 -- 172.21.15.78:0/1438777946 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fba780d5aa0 msgr2=0x7fba780d5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.490+0000 7fba877f9700 1 --2- 172.21.15.78:0/1438777946 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fba780d5aa0 0x7fba780d5f10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fba7803ce50 tx=0x7fba70011bf0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.490+0000 7fba877f9700 1 -- 172.21.15.78:0/1438777946 shutdown_connections 2023-10-11T09:40:52.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.490+0000 7fba877f9700 1 --2- 172.21.15.78:0/1438777946 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fba64065fd0 0x7fba64068480 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.490+0000 7fba877f9700 1 --2- 172.21.15.78:0/1438777946 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fba780d5aa0 0x7fba780d5f10 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.490+0000 7fba877f9700 1 --2- 172.21.15.78:0/1438777946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba780e2e70 0x7fba780e32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.490+0000 7fba877f9700 1 --2- 172.21.15.78:0/1438777946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba6c003810 0x7fba780e4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.490+0000 7fba877f9700 1 -- 172.21.15.78:0/1438777946 >> 172.21.15.78:0/1438777946 conn(0x7fba78057b90 msgr2=0x7fba780dc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:52.499 INFO:teuthology.orchestra.run.smithi078.stdout:55834574855 2023-10-11T09:40:52.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.490+0000 7faf96ffd700 1 -- 172.21.15.78:0/1534578069 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7faf9c055280 con 0x7faf98038d80 2023-10-11T09:40:52.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.490+0000 7fba877f9700 1 -- 172.21.15.78:0/1438777946 shutdown_connections 2023-10-11T09:40:52.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.494+0000 7fba877f9700 1 -- 172.21.15.78:0/1438777946 wait complete. 2023-10-11T09:40:52.501 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.2 2023-10-11T09:40:52.502 DEBUG:teuthology.parallel:result is None 2023-10-11T09:40:52.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.494+0000 7f3772b16700 1 -- 172.21.15.78:0/3961077610 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3750065fa0 msgr2=0x7f3750068450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.494+0000 7f3772b16700 1 --2- 172.21.15.78:0/3961077610 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3750065fa0 0x7f3750068450 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f376c063e50 tx=0x7f376c061ce0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.494+0000 7f3772b16700 1 -- 172.21.15.78:0/3961077610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f376403b0d0 msgr2=0x7f3764051090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.494+0000 7f3772b16700 1 --2- 172.21.15.78:0/3961077610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f376403b0d0 0x7f3764051090 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f376800b410 tx=0x7f376800b720 comp rx=0 tx=0).stop 2023-10-11T09:40:52.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.494+0000 7f3772b16700 1 -- 172.21.15.78:0/3961077610 shutdown_connections 2023-10-11T09:40:52.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.494+0000 7f3772b16700 1 --2- 172.21.15.78:0/3961077610 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3750065fa0 0x7f3750068450 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.494+0000 7f3772b16700 1 --2- 172.21.15.78:0/3961077610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f376403b0d0 0x7f3764051090 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.494+0000 7f3772b16700 1 --2- 172.21.15.78:0/3961077610 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f376403ac80 0x7f3764050b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.494+0000 7f3772b16700 1 --2- 172.21.15.78:0/3961077610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37640383e0 0x7f37640dd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.494+0000 7f3772b16700 1 -- 172.21.15.78:0/3961077610 >> 172.21.15.78:0/3961077610 conn(0x7f3764055c80 msgr2=0x7f3764039e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:52.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.494+0000 7f3772b16700 1 -- 172.21.15.78:0/3961077610 shutdown_connections 2023-10-11T09:40:52.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.494+0000 7f3772b16700 1 -- 172.21.15.78:0/3961077610 wait complete. 2023-10-11T09:40:52.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.494+0000 7fafa869b700 1 -- 172.21.15.78:0/1534578069 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7faf84065fa0 msgr2=0x7faf84068450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.494+0000 7fafa869b700 1 --2- 172.21.15.78:0/1534578069 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7faf84065fa0 0x7faf84068450 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7faf98056060 tx=0x7faf90008040 comp rx=0 tx=0).stop 2023-10-11T09:40:52.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.494+0000 7fafa869b700 1 -- 172.21.15.78:0/1534578069 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faf98038d80 msgr2=0x7faf980d8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.494+0000 7fafa869b700 1 --2- 172.21.15.78:0/1534578069 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faf98038d80 0x7faf980d8db0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7faf9c015000 tx=0x7faf9c004770 comp rx=0 tx=0).stop 2023-10-11T09:40:52.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.494+0000 7fafa869b700 1 -- 172.21.15.78:0/1534578069 shutdown_connections 2023-10-11T09:40:52.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.494+0000 7fafa869b700 1 --2- 172.21.15.78:0/1534578069 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7faf84065fa0 0x7faf84068450 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.494+0000 7fafa869b700 1 --2- 172.21.15.78:0/1534578069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf980e4c50 0x7faf980e15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.494+0000 7fafa869b700 1 --2- 172.21.15.78:0/1534578069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf9803c740 0x7faf981795d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.494+0000 7fafa869b700 1 --2- 172.21.15.78:0/1534578069 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faf98038d80 0x7faf980d8db0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.494+0000 7fafa869b700 1 -- 172.21.15.78:0/1534578069 >> 172.21.15.78:0/1534578069 conn(0x7faf9800cf60 msgr2=0x7faf9803b540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:52.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.494+0000 7fafa869b700 1 -- 172.21.15.78:0/1534578069 shutdown_connections 2023-10-11T09:40:52.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.494+0000 7fafa869b700 1 -- 172.21.15.78:0/1534578069 wait complete. 2023-10-11T09:40:52.511 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2023-10-11T09:40:52.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.498+0000 7fc4b1959700 1 -- 172.21.15.78:0/3843458171 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fc4ac065d40 con 0x7fc4ac06d380 2023-10-11T09:40:52.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.498+0000 7fc493fff700 1 -- 172.21.15.78:0/3843458171 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc4a4056310 con 0x7fc4ac06d380 2023-10-11T09:40:52.512 DEBUG:teuthology.parallel:result is None 2023-10-11T09:40:52.513 INFO:teuthology.orchestra.run.smithi078.stdout:55834574855 2023-10-11T09:40:52.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.502+0000 7fc4b1959700 1 -- 172.21.15.78:0/3843458171 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc498065ed0 msgr2=0x7fc498068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.502+0000 7fc4b1959700 1 --2- 172.21.15.78:0/3843458171 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc498065ed0 0x7fc498068380 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac0cdaa0 tx=0x7fc4a0007af0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.502+0000 7fc4b1959700 1 -- 172.21.15.78:0/3843458171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4ac06d380 msgr2=0x7fc4ac141cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.502+0000 7fc4b1959700 1 --2- 172.21.15.78:0/3843458171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4ac06d380 0x7fc4ac141cf0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4000f80 tx=0x7fc4a4004af0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.502+0000 7fc4b1959700 1 -- 172.21.15.78:0/3843458171 shutdown_connections 2023-10-11T09:40:52.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.502+0000 7fc4b1959700 1 --2- 172.21.15.78:0/3843458171 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc498065ed0 0x7fc498068380 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.502+0000 7fc4b1959700 1 --2- 172.21.15.78:0/3843458171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4ac0cf170 0x7fc4ac0cdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.502+0000 7fc4b1959700 1 --2- 172.21.15.78:0/3843458171 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc4ac0b06a0 0x7fc4ac1576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.502+0000 7fc4b1959700 1 --2- 172.21.15.78:0/3843458171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4ac06d380 0x7fc4ac141cf0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.502+0000 7fc4b1959700 1 -- 172.21.15.78:0/3843458171 >> 172.21.15.78:0/3843458171 conn(0x7fc4ac06aca0 msgr2=0x7fc4ac14e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:52.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.502+0000 7fc4b1959700 1 -- 172.21.15.78:0/3843458171 shutdown_connections 2023-10-11T09:40:52.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.502+0000 7fc4b1959700 1 -- 172.21.15.78:0/3843458171 wait complete. 2023-10-11T09:40:52.521 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2023-10-11T09:40:52.521 DEBUG:teuthology.parallel:result is None 2023-10-11T09:40:52.524 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.7 2023-10-11T09:40:52.526 DEBUG:teuthology.parallel:result is None 2023-10-11T09:40:52.527 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2023-10-11T09:40:52.527 DEBUG:teuthology.parallel:result is None 2023-10-11T09:40:52.528 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2023-10-11T09:40:52.529 DEBUG:teuthology.parallel:result is None 2023-10-11T09:40:52.529 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-11T09:40:52.530 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-11T09:40:52.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.726+0000 7f9230b2c700 1 Processor -- start 2023-10-11T09:40:52.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.726+0000 7f9230b2c700 1 -- start start 2023-10-11T09:40:52.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.730+0000 7f9230b2c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f922c0cbd30 0x7f922c06be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.730+0000 7f9230b2c700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f922c0c93a0 con 0x7f922c15c880 2023-10-11T09:40:52.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.730+0000 7f9230b2c700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f922c0cca10 con 0x7f922c0cd210 2023-10-11T09:40:52.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.730+0000 7f9230b2c700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f922c161420 con 0x7f922c0cbd30 2023-10-11T09:40:52.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.730+0000 7f922a59c700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f922c15c880 0x7f922c0cb620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:58926/0 (socket says 172.21.15.78:58926) 2023-10-11T09:40:52.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.730+0000 7f922a59c700 1 -- 172.21.15.78:0/2083736120 learned_addr learned my addr 172.21.15.78:0/2083736120 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:52.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.730+0000 7f9221d9b700 1 --2- 172.21.15.78:0/2083736120 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f922c0cbd30 0x7f922c06be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.730+0000 7f9229d9b700 1 -- 172.21.15.78:0/2083736120 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 525061322 0 0) 0x7f922c0c93a0 con 0x7f922c15c880 2023-10-11T09:40:52.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.730+0000 7f9229d9b700 1 -- 172.21.15.78:0/2083736120 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92180035d0 con 0x7f922c15c880 2023-10-11T09:40:52.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.730+0000 7f9221d9b700 1 -- 172.21.15.78:0/2083736120 >> v1:172.21.15.133:6789/0 conn(0x7f922c0cd210 legacy=0x7f922c0c9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.730+0000 7f9221d9b700 1 -- 172.21.15.78:0/2083736120 >> v1:172.21.15.78:6789/0 conn(0x7f922c15c880 legacy=0x7f922c0cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.730+0000 7f9221d9b700 1 -- 172.21.15.78:0/2083736120 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f922c0cccb0 con 0x7f922c0cbd30 2023-10-11T09:40:52.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.730+0000 7f9221d9b700 1 --2- 172.21.15.78:0/2083736120 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f922c0cbd30 0x7f922c06be60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9214009880 tx=0x7f9214009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63e6c058f522d372 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:52.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.730+0000 7f9229d9b700 1 -- 172.21.15.78:0/2083736120 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9214011830 con 0x7f922c0cbd30 2023-10-11T09:40:52.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.730+0000 7f9229d9b700 1 -- 172.21.15.78:0/2083736120 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9214011990 con 0x7f922c0cbd30 2023-10-11T09:40:52.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.730+0000 7f9229d9b700 1 -- 172.21.15.78:0/2083736120 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9214011d20 con 0x7f922c0cbd30 2023-10-11T09:40:52.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.734+0000 7f9230b2c700 1 -- 172.21.15.78:0/2083736120 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f922c0cbd30 msgr2=0x7f922c06be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.734+0000 7f9230b2c700 1 --2- 172.21.15.78:0/2083736120 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f922c0cbd30 0x7f922c06be60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9214009880 tx=0x7f9214009b90 comp rx=0 tx=0).stop 2023-10-11T09:40:52.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.734+0000 7f9230b2c700 1 -- 172.21.15.78:0/2083736120 shutdown_connections 2023-10-11T09:40:52.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.734+0000 7f9230b2c700 1 --2- 172.21.15.78:0/2083736120 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f922c0cbd30 0x7f922c06be60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.734+0000 7f9230b2c700 1 -- 172.21.15.78:0/2083736120 >> 172.21.15.78:0/2083736120 conn(0x7f922c06aca0 msgr2=0x7f922c0b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:52.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.734+0000 7f9230b2c700 1 -- 172.21.15.78:0/2083736120 shutdown_connections 2023-10-11T09:40:52.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.734+0000 7f9230b2c700 1 -- 172.21.15.78:0/2083736120 wait complete. 2023-10-11T09:40:52.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.738+0000 7f9230b2c700 1 Processor -- start 2023-10-11T09:40:52.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.738+0000 7f9230b2c700 1 -- start start 2023-10-11T09:40:52.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.738+0000 7f9230b2c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f922c0cbd30 0x7f922c0b1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.738+0000 7f9230b2c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f922c0cd210 0x7f922c14ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.738+0000 7f9230b2c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f922c15c880 0x7f922c0b4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.738+0000 7f9230b2c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922c157a00 con 0x7f922c0cbd30 2023-10-11T09:40:52.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.738+0000 7f9230b2c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922c156df0 con 0x7f922c0cd210 2023-10-11T09:40:52.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.738+0000 7f9230b2c700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f922c1570d0 con 0x7f922c15c880 2023-10-11T09:40:52.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.738+0000 7f922a59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f922c0cbd30 0x7f922c0b1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.738+0000 7f922a59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f922c0cbd30 0x7f922c0b1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57242/0 (socket says 172.21.15.78:57242) 2023-10-11T09:40:52.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.738+0000 7f922a59c700 1 -- 172.21.15.78:0/1480928373 learned_addr learned my addr 172.21.15.78:0/1480928373 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:52.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.738+0000 7f9221d9b700 1 --2- 172.21.15.78:0/1480928373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f922c0cd210 0x7f922c14ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.738+0000 7f922ad9d700 1 --2- 172.21.15.78:0/1480928373 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f922c15c880 0x7f922c0b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.738+0000 7f922a59c700 1 -- 172.21.15.78:0/1480928373 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f922c15c880 msgr2=0x7f922c0b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.738+0000 7f922a59c700 1 --2- 172.21.15.78:0/1480928373 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f922c15c880 0x7f922c0b4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.738+0000 7f922a59c700 1 -- 172.21.15.78:0/1480928373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f922c0cd210 msgr2=0x7f922c14ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.738+0000 7f922a59c700 1 --2- 172.21.15.78:0/1480928373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f922c0cd210 0x7f922c14ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.738+0000 7f922a59c700 1 -- 172.21.15.78:0/1480928373 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92140094b0 con 0x7f922c0cbd30 2023-10-11T09:40:52.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.738+0000 7f922a59c700 1 --2- 172.21.15.78:0/1480928373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f922c0cbd30 0x7f922c0b1da0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f921c00cbc0 tx=0x7f921c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:52.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.742+0000 7f92237fe700 1 -- 172.21.15.78:0/1480928373 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c007480 con 0x7f922c0cbd30 2023-10-11T09:40:52.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.742+0000 7f92237fe700 1 -- 172.21.15.78:0/1480928373 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f921c0075e0 con 0x7f922c0cbd30 2023-10-11T09:40:52.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.742+0000 7f9230b2c700 1 -- 172.21.15.78:0/1480928373 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f922c1573b0 con 0x7f922c0cbd30 2023-10-11T09:40:52.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.742+0000 7f92237fe700 1 -- 172.21.15.78:0/1480928373 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c007950 con 0x7f922c0cbd30 2023-10-11T09:40:52.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.742+0000 7f9230b2c700 1 -- 172.21.15.78:0/1480928373 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f922c0b22e0 con 0x7f922c0cbd30 2023-10-11T09:40:52.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.742+0000 7f9230b2c700 1 -- 172.21.15.78:0/1480928373 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f922c04e530 con 0x7f922c0cbd30 2023-10-11T09:40:52.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.742+0000 7f92237fe700 1 -- 172.21.15.78:0/1480928373 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f921c01b3b0 con 0x7f922c0cbd30 2023-10-11T09:40:52.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.746+0000 7f92237fe700 1 --2- 172.21.15.78:0/1480928373 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f920c065f00 0x7f920c0683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:52.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.746+0000 7f92237fe700 1 -- 172.21.15.78:0/1480928373 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f921c082fb0 con 0x7f922c0cbd30 2023-10-11T09:40:52.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.746+0000 7f9221d9b700 1 --2- 172.21.15.78:0/1480928373 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f920c065f00 0x7f920c0683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:52.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.746+0000 7f9221d9b700 1 --2- 172.21.15.78:0/1480928373 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f920c065f00 0x7f920c0683b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f922c0cc120 tx=0x7f9214010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:52.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.754+0000 7f92237fe700 1 -- 172.21.15.78:0/1480928373 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f921c0b3fa0 con 0x7f922c0cbd30 2023-10-11T09:40:52.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.918+0000 7f9230b2c700 1 -- 172.21.15.78:0/1480928373 --> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f922c0cb360 con 0x7f920c065f00 2023-10-11T09:40:52.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.930+0000 7f92237fe700 1 -- 172.21.15.78:0/1480928373 <== mgr.4101 v2:172.21.15.78:6832/35606 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286426 (secure 0 0 0) 0x7f9214011b80 con 0x7f920c065f00 2023-10-11T09:40:52.941 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:40:52.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.938+0000 7f9230b2c700 1 -- 172.21.15.78:0/1480928373 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f920c065f00 msgr2=0x7f920c0683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.938+0000 7f9230b2c700 1 --2- 172.21.15.78:0/1480928373 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f920c065f00 0x7f920c0683b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f922c0cc120 tx=0x7f9214010000 comp rx=0 tx=0).stop 2023-10-11T09:40:52.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.938+0000 7f9230b2c700 1 -- 172.21.15.78:0/1480928373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f922c0cbd30 msgr2=0x7f922c0b1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:52.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.938+0000 7f9230b2c700 1 --2- 172.21.15.78:0/1480928373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f922c0cbd30 0x7f922c0b1da0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f921c00cbc0 tx=0x7f921c00cbf0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.938+0000 7f9230b2c700 1 -- 172.21.15.78:0/1480928373 shutdown_connections 2023-10-11T09:40:52.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.938+0000 7f9230b2c700 1 --2- 172.21.15.78:0/1480928373 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f920c065f00 0x7f920c0683b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.938+0000 7f9230b2c700 1 --2- 172.21.15.78:0/1480928373 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f922c15c880 0x7f922c0b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.938+0000 7f9230b2c700 1 --2- 172.21.15.78:0/1480928373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f922c0cd210 0x7f922c14ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.938+0000 7f9230b2c700 1 --2- 172.21.15.78:0/1480928373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f922c0cbd30 0x7f922c0b1da0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:52.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.938+0000 7f9230b2c700 1 -- 172.21.15.78:0/1480928373 >> 172.21.15.78:0/1480928373 conn(0x7f922c06aca0 msgr2=0x7f922c06cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:52.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.938+0000 7f9230b2c700 1 -- 172.21.15.78:0/1480928373 shutdown_connections 2023-10-11T09:40:52.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:52.938+0000 7f9230b2c700 1 -- 172.21.15.78:0/1480928373 wait complete. 2023-10-11T09:40:52.947 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2023-10-11T09:40:52.967 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-10-11T09:40:52.325814+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218728,"kb_used_data":3112,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755992,"statfs":{"total":773094113280,"available":772870135808,"internally_reserved":0,"allocated":3186688,"data_stored":1902252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003045"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144669+0000","last_change":"2023-10-11T09:40:41.696400+0000","last_active":"2023-10-11T09:40:43.144669+0000","last_peered":"2023-10-11T09:40:43.144669+0000","last_clean":"2023-10-11T09:40:43.144669+0000","last_became_active":"2023-10-11T09:40:41.696101+0000","last_became_peered":"2023-10-11T09:40:41.696101+0000","last_unstale":"2023-10-11T09:40:43.144669+0000","last_undegraded":"2023-10-11T09:40:43.144669+0000","last_fullsized":"2023-10-11T09:40:43.144669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:43:01.331013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.694262+0000","last_change":"2023-10-11T09:40:41.719611+0000","last_active":"2023-10-11T09:40:42.694262+0000","last_peered":"2023-10-11T09:40:42.694262+0000","last_clean":"2023-10-11T09:40:42.694262+0000","last_became_active":"2023-10-11T09:40:41.719319+0000","last_became_peered":"2023-10-11T09:40:41.719319+0000","last_unstale":"2023-10-11T09:40:42.694262+0000","last_undegraded":"2023-10-11T09:40:42.694262+0000","last_fullsized":"2023-10-11T09:40:42.694262+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:07:02.302344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.912066+0000","last_change":"2023-10-11T09:40:41.701063+0000","last_active":"2023-10-11T09:40:42.912066+0000","last_peered":"2023-10-11T09:40:42.912066+0000","last_clean":"2023-10-11T09:40:42.912066+0000","last_became_active":"2023-10-11T09:40:41.699445+0000","last_became_peered":"2023-10-11T09:40:41.699445+0000","last_unstale":"2023-10-11T09:40:42.912066+0000","last_undegraded":"2023-10-11T09:40:42.912066+0000","last_fullsized":"2023-10-11T09:40:42.912066+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:19:44.476799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144161+0000","last_change":"2023-10-11T09:40:41.721932+0000","last_active":"2023-10-11T09:40:43.144161+0000","last_peered":"2023-10-11T09:40:43.144161+0000","last_clean":"2023-10-11T09:40:43.144161+0000","last_became_active":"2023-10-11T09:40:41.720602+0000","last_became_peered":"2023-10-11T09:40:41.720602+0000","last_unstale":"2023-10-11T09:40:43.144161+0000","last_undegraded":"2023-10-11T09:40:43.144161+0000","last_fullsized":"2023-10-11T09:40:43.144161+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:17:36.655491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145802+0000","last_change":"2023-10-11T09:40:41.710213+0000","last_active":"2023-10-11T09:40:43.145802+0000","last_peered":"2023-10-11T09:40:43.145802+0000","last_clean":"2023-10-11T09:40:43.145802+0000","last_became_active":"2023-10-11T09:40:41.707488+0000","last_became_peered":"2023-10-11T09:40:41.707488+0000","last_unstale":"2023-10-11T09:40:43.145802+0000","last_undegraded":"2023-10-11T09:40:43.145802+0000","last_fullsized":"2023-10-11T09:40:43.145802+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:42:34.821513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911583+0000","last_change":"2023-10-11T09:40:41.700595+0000","last_active":"2023-10-11T09:40:42.911583+0000","last_peered":"2023-10-11T09:40:42.911583+0000","last_clean":"2023-10-11T09:40:42.911583+0000","last_became_active":"2023-10-11T09:40:41.700239+0000","last_became_peered":"2023-10-11T09:40:41.700239+0000","last_unstale":"2023-10-11T09:40:42.911583+0000","last_undegraded":"2023-10-11T09:40:42.911583+0000","last_fullsized":"2023-10-11T09:40:42.911583+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:48:14.871150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.691894+0000","last_change":"2023-10-11T09:40:41.712870+0000","last_active":"2023-10-11T09:40:42.691894+0000","last_peered":"2023-10-11T09:40:42.691894+0000","last_clean":"2023-10-11T09:40:42.691894+0000","last_became_active":"2023-10-11T09:40:41.712593+0000","last_became_peered":"2023-10-11T09:40:41.712593+0000","last_unstale":"2023-10-11T09:40:42.691894+0000","last_undegraded":"2023-10-11T09:40:42.691894+0000","last_fullsized":"2023-10-11T09:40:42.691894+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:50:15.829941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.693734+0000","last_change":"2023-10-11T09:40:41.714890+0000","last_active":"2023-10-11T09:40:42.693734+0000","last_peered":"2023-10-11T09:40:42.693734+0000","last_clean":"2023-10-11T09:40:42.693734+0000","last_became_active":"2023-10-11T09:40:41.711116+0000","last_became_peered":"2023-10-11T09:40:41.711116+0000","last_unstale":"2023-10-11T09:40:42.693734+0000","last_undegraded":"2023-10-11T09:40:42.693734+0000","last_fullsized":"2023-10-11T09:40:42.693734+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:00:05.422788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.146063+0000","last_change":"2023-10-11T09:40:41.700768+0000","last_active":"2023-10-11T09:40:43.146063+0000","last_peered":"2023-10-11T09:40:43.146063+0000","last_clean":"2023-10-11T09:40:43.146063+0000","last_became_active":"2023-10-11T09:40:41.700440+0000","last_became_peered":"2023-10-11T09:40:41.700440+0000","last_unstale":"2023-10-11T09:40:43.146063+0000","last_undegraded":"2023-10-11T09:40:43.146063+0000","last_fullsized":"2023-10-11T09:40:43.146063+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:56:50.343276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.694372+0000","last_change":"2023-10-11T09:40:41.712701+0000","last_active":"2023-10-11T09:40:42.694372+0000","last_peered":"2023-10-11T09:40:42.694372+0000","last_clean":"2023-10-11T09:40:42.694372+0000","last_became_active":"2023-10-11T09:40:41.712349+0000","last_became_peered":"2023-10-11T09:40:41.712349+0000","last_unstale":"2023-10-11T09:40:42.694372+0000","last_undegraded":"2023-10-11T09:40:42.694372+0000","last_fullsized":"2023-10-11T09:40:42.694372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:42:34.821513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.912276+0000","last_change":"2023-10-11T09:40:41.706310+0000","last_active":"2023-10-11T09:40:42.912276+0000","last_peered":"2023-10-11T09:40:42.912276+0000","last_clean":"2023-10-11T09:40:42.912276+0000","last_became_active":"2023-10-11T09:40:41.705986+0000","last_became_peered":"2023-10-11T09:40:41.705986+0000","last_unstale":"2023-10-11T09:40:42.912276+0000","last_undegraded":"2023-10-11T09:40:42.912276+0000","last_fullsized":"2023-10-11T09:40:42.912276+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:59:46.864358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144038+0000","last_change":"2023-10-11T09:40:41.700251+0000","last_active":"2023-10-11T09:40:43.144038+0000","last_peered":"2023-10-11T09:40:43.144038+0000","last_clean":"2023-10-11T09:40:43.144038+0000","last_became_active":"2023-10-11T09:40:41.699961+0000","last_became_peered":"2023-10-11T09:40:41.699961+0000","last_unstale":"2023-10-11T09:40:43.144038+0000","last_undegraded":"2023-10-11T09:40:43.144038+0000","last_fullsized":"2023-10-11T09:40:43.144038+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:12:05.737223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.143594+0000","last_change":"2023-10-11T09:40:41.697448+0000","last_active":"2023-10-11T09:40:43.143594+0000","last_peered":"2023-10-11T09:40:43.143594+0000","last_clean":"2023-10-11T09:40:43.143594+0000","last_became_active":"2023-10-11T09:40:41.697160+0000","last_became_peered":"2023-10-11T09:40:41.697160+0000","last_unstale":"2023-10-11T09:40:43.143594+0000","last_undegraded":"2023-10-11T09:40:43.143594+0000","last_fullsized":"2023-10-11T09:40:43.143594+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:00:57.063025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.693705+0000","last_change":"2023-10-11T09:40:41.717312+0000","last_active":"2023-10-11T09:40:42.693705+0000","last_peered":"2023-10-11T09:40:42.693705+0000","last_clean":"2023-10-11T09:40:42.693705+0000","last_became_active":"2023-10-11T09:40:41.716622+0000","last_became_peered":"2023-10-11T09:40:41.716622+0000","last_unstale":"2023-10-11T09:40:42.693705+0000","last_undegraded":"2023-10-11T09:40:42.693705+0000","last_fullsized":"2023-10-11T09:40:42.693705+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:35:04.646798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.692225+0000","last_change":"2023-10-11T09:40:41.715325+0000","last_active":"2023-10-11T09:40:42.692225+0000","last_peered":"2023-10-11T09:40:42.692225+0000","last_clean":"2023-10-11T09:40:42.692225+0000","last_became_active":"2023-10-11T09:40:41.715033+0000","last_became_peered":"2023-10-11T09:40:41.715033+0000","last_unstale":"2023-10-11T09:40:42.692225+0000","last_undegraded":"2023-10-11T09:40:42.692225+0000","last_fullsized":"2023-10-11T09:40:42.692225+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:35:36.436236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911235+0000","last_change":"2023-10-11T09:40:41.699851+0000","last_active":"2023-10-11T09:40:42.911235+0000","last_peered":"2023-10-11T09:40:42.911235+0000","last_clean":"2023-10-11T09:40:42.911235+0000","last_became_active":"2023-10-11T09:40:41.699530+0000","last_became_peered":"2023-10-11T09:40:41.699530+0000","last_unstale":"2023-10-11T09:40:42.911235+0000","last_undegraded":"2023-10-11T09:40:42.911235+0000","last_fullsized":"2023-10-11T09:40:42.911235+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:23:30.814623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.146112+0000","last_change":"2023-10-11T09:40:41.700611+0000","last_active":"2023-10-11T09:40:43.146112+0000","last_peered":"2023-10-11T09:40:43.146112+0000","last_clean":"2023-10-11T09:40:43.146112+0000","last_became_active":"2023-10-11T09:40:41.700354+0000","last_became_peered":"2023-10-11T09:40:41.700354+0000","last_unstale":"2023-10-11T09:40:43.146112+0000","last_undegraded":"2023-10-11T09:40:43.146112+0000","last_fullsized":"2023-10-11T09:40:43.146112+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:48:15.689750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.912114+0000","last_change":"2023-10-11T09:40:41.700742+0000","last_active":"2023-10-11T09:40:42.912114+0000","last_peered":"2023-10-11T09:40:42.912114+0000","last_clean":"2023-10-11T09:40:42.912114+0000","last_became_active":"2023-10-11T09:40:41.700432+0000","last_became_peered":"2023-10-11T09:40:41.700432+0000","last_unstale":"2023-10-11T09:40:42.912114+0000","last_undegraded":"2023-10-11T09:40:42.912114+0000","last_fullsized":"2023-10-11T09:40:42.912114+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:33:13.162340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.143660+0000","last_change":"2023-10-11T09:40:41.719737+0000","last_active":"2023-10-11T09:40:43.143660+0000","last_peered":"2023-10-11T09:40:43.143660+0000","last_clean":"2023-10-11T09:40:43.143660+0000","last_became_active":"2023-10-11T09:40:41.719484+0000","last_became_peered":"2023-10-11T09:40:41.719484+0000","last_unstale":"2023-10-11T09:40:43.143660+0000","last_undegraded":"2023-10-11T09:40:43.143660+0000","last_fullsized":"2023-10-11T09:40:43.143660+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:48:52.897052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.912175+0000","last_change":"2023-10-11T09:40:41.728139+0000","last_active":"2023-10-11T09:40:42.912175+0000","last_peered":"2023-10-11T09:40:42.912175+0000","last_clean":"2023-10-11T09:40:42.912175+0000","last_became_active":"2023-10-11T09:40:41.727790+0000","last_became_peered":"2023-10-11T09:40:41.727790+0000","last_unstale":"2023-10-11T09:40:42.912175+0000","last_undegraded":"2023-10-11T09:40:42.912175+0000","last_fullsized":"2023-10-11T09:40:42.912175+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:07:58.571258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.692070+0000","last_change":"2023-10-11T09:40:41.713375+0000","last_active":"2023-10-11T09:40:42.692070+0000","last_peered":"2023-10-11T09:40:42.692070+0000","last_clean":"2023-10-11T09:40:42.692070+0000","last_became_active":"2023-10-11T09:40:41.713114+0000","last_became_peered":"2023-10-11T09:40:41.713114+0000","last_unstale":"2023-10-11T09:40:42.692070+0000","last_undegraded":"2023-10-11T09:40:42.692070+0000","last_fullsized":"2023-10-11T09:40:42.692070+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:07:02.302344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145102+0000","last_change":"2023-10-11T09:40:41.718540+0000","last_active":"2023-10-11T09:40:43.145102+0000","last_peered":"2023-10-11T09:40:43.145102+0000","last_clean":"2023-10-11T09:40:43.145102+0000","last_became_active":"2023-10-11T09:40:41.718309+0000","last_became_peered":"2023-10-11T09:40:41.718309+0000","last_unstale":"2023-10-11T09:40:43.145102+0000","last_undegraded":"2023-10-11T09:40:43.145102+0000","last_fullsized":"2023-10-11T09:40:43.145102+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:09:27.825336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687447+0000","last_change":"2023-10-11T09:40:41.699075+0000","last_active":"2023-10-11T09:40:42.687447+0000","last_peered":"2023-10-11T09:40:42.687447+0000","last_clean":"2023-10-11T09:40:42.687447+0000","last_became_active":"2023-10-11T09:40:41.698747+0000","last_became_peered":"2023-10-11T09:40:41.698747+0000","last_unstale":"2023-10-11T09:40:42.687447+0000","last_undegraded":"2023-10-11T09:40:42.687447+0000","last_fullsized":"2023-10-11T09:40:42.687447+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:09:27.825336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.691897+0000","last_change":"2023-10-11T09:40:41.714225+0000","last_active":"2023-10-11T09:40:42.691897+0000","last_peered":"2023-10-11T09:40:42.691897+0000","last_clean":"2023-10-11T09:40:42.691897+0000","last_became_active":"2023-10-11T09:40:41.710262+0000","last_became_peered":"2023-10-11T09:40:41.710262+0000","last_unstale":"2023-10-11T09:40:42.691897+0000","last_undegraded":"2023-10-11T09:40:42.691897+0000","last_fullsized":"2023-10-11T09:40:42.691897+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:42:34.821513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144675+0000","last_change":"2023-10-11T09:40:41.729722+0000","last_active":"2023-10-11T09:40:43.144675+0000","last_peered":"2023-10-11T09:40:43.144675+0000","last_clean":"2023-10-11T09:40:43.144675+0000","last_became_active":"2023-10-11T09:40:41.729360+0000","last_became_peered":"2023-10-11T09:40:41.729360+0000","last_unstale":"2023-10-11T09:40:43.144675+0000","last_undegraded":"2023-10-11T09:40:43.144675+0000","last_fullsized":"2023-10-11T09:40:43.144675+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:07:02.302344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145219+0000","last_change":"2023-10-11T09:40:41.709354+0000","last_active":"2023-10-11T09:40:43.145219+0000","last_peered":"2023-10-11T09:40:43.145219+0000","last_clean":"2023-10-11T09:40:43.145219+0000","last_became_active":"2023-10-11T09:40:41.709096+0000","last_became_peered":"2023-10-11T09:40:41.709096+0000","last_unstale":"2023-10-11T09:40:43.145219+0000","last_undegraded":"2023-10-11T09:40:43.145219+0000","last_fullsized":"2023-10-11T09:40:43.145219+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:35:04.646798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.694072+0000","last_change":"2023-10-11T09:40:41.723510+0000","last_active":"2023-10-11T09:40:42.694072+0000","last_peered":"2023-10-11T09:40:42.694072+0000","last_clean":"2023-10-11T09:40:42.694072+0000","last_became_active":"2023-10-11T09:40:41.723110+0000","last_became_peered":"2023-10-11T09:40:41.723110+0000","last_unstale":"2023-10-11T09:40:42.694072+0000","last_undegraded":"2023-10-11T09:40:42.694072+0000","last_fullsized":"2023-10-11T09:40:42.694072+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:50:15.829941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145120+0000","last_change":"2023-10-11T09:40:41.714481+0000","last_active":"2023-10-11T09:40:43.145120+0000","last_peered":"2023-10-11T09:40:43.145120+0000","last_clean":"2023-10-11T09:40:43.145120+0000","last_became_active":"2023-10-11T09:40:41.714219+0000","last_became_peered":"2023-10-11T09:40:41.714219+0000","last_unstale":"2023-10-11T09:40:43.145120+0000","last_undegraded":"2023-10-11T09:40:43.145120+0000","last_fullsized":"2023-10-11T09:40:43.145120+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:19:07.089979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.688002+0000","last_change":"2023-10-11T09:40:41.709612+0000","last_active":"2023-10-11T09:40:42.688002+0000","last_peered":"2023-10-11T09:40:42.688002+0000","last_clean":"2023-10-11T09:40:42.688002+0000","last_became_active":"2023-10-11T09:40:41.709317+0000","last_became_peered":"2023-10-11T09:40:41.709317+0000","last_unstale":"2023-10-11T09:40:42.688002+0000","last_undegraded":"2023-10-11T09:40:42.688002+0000","last_fullsized":"2023-10-11T09:40:42.688002+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:35:36.436236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911908+0000","last_change":"2023-10-11T09:40:41.727354+0000","last_active":"2023-10-11T09:40:42.911908+0000","last_peered":"2023-10-11T09:40:42.911908+0000","last_clean":"2023-10-11T09:40:42.911908+0000","last_became_active":"2023-10-11T09:40:41.727039+0000","last_became_peered":"2023-10-11T09:40:41.727039+0000","last_unstale":"2023-10-11T09:40:42.911908+0000","last_undegraded":"2023-10-11T09:40:42.911908+0000","last_fullsized":"2023-10-11T09:40:42.911908+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:01:36.923947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687557+0000","last_change":"2023-10-11T09:40:41.712971+0000","last_active":"2023-10-11T09:40:42.687557+0000","last_peered":"2023-10-11T09:40:42.687557+0000","last_clean":"2023-10-11T09:40:42.687557+0000","last_became_active":"2023-10-11T09:40:41.712707+0000","last_became_peered":"2023-10-11T09:40:41.712707+0000","last_unstale":"2023-10-11T09:40:42.687557+0000","last_undegraded":"2023-10-11T09:40:42.687557+0000","last_fullsized":"2023-10-11T09:40:42.687557+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:48:15.689750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.143927+0000","last_change":"2023-10-11T09:40:41.728941+0000","last_active":"2023-10-11T09:40:43.143927+0000","last_peered":"2023-10-11T09:40:43.143927+0000","last_clean":"2023-10-11T09:40:43.143927+0000","last_became_active":"2023-10-11T09:40:41.728657+0000","last_became_peered":"2023-10-11T09:40:41.728657+0000","last_unstale":"2023-10-11T09:40:43.143927+0000","last_undegraded":"2023-10-11T09:40:43.143927+0000","last_fullsized":"2023-10-11T09:40:43.143927+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:12:05.737223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.688259+0000","last_change":"2023-10-11T09:40:41.718662+0000","last_active":"2023-10-11T09:40:42.688259+0000","last_peered":"2023-10-11T09:40:42.688259+0000","last_clean":"2023-10-11T09:40:42.688259+0000","last_became_active":"2023-10-11T09:40:41.718402+0000","last_became_peered":"2023-10-11T09:40:41.718402+0000","last_unstale":"2023-10-11T09:40:42.688259+0000","last_undegraded":"2023-10-11T09:40:42.688259+0000","last_fullsized":"2023-10-11T09:40:42.688259+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:56:50.343276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.692319+0000","last_change":"2023-10-11T09:40:41.234649+0000","last_active":"2023-10-11T09:40:42.692319+0000","last_peered":"2023-10-11T09:40:42.692319+0000","last_clean":"2023-10-11T09:40:42.692319+0000","last_became_active":"2023-10-11T09:40:41.234006+0000","last_became_peered":"2023-10-11T09:40:41.234006+0000","last_unstale":"2023-10-11T09:40:42.692319+0000","last_undegraded":"2023-10-11T09:40:42.692319+0000","last_fullsized":"2023-10-11T09:40:42.692319+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:45:39.083797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.694114+0000","last_change":"2023-10-11T09:40:41.723234+0000","last_active":"2023-10-11T09:40:42.694114+0000","last_peered":"2023-10-11T09:40:42.694114+0000","last_clean":"2023-10-11T09:40:42.694114+0000","last_became_active":"2023-10-11T09:40:41.722928+0000","last_became_peered":"2023-10-11T09:40:41.722928+0000","last_unstale":"2023-10-11T09:40:42.694114+0000","last_undegraded":"2023-10-11T09:40:42.694114+0000","last_fullsized":"2023-10-11T09:40:42.694114+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:09:27.825336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145323+0000","last_change":"2023-10-11T09:40:41.225247+0000","last_active":"2023-10-11T09:40:43.145323+0000","last_peered":"2023-10-11T09:40:43.145323+0000","last_clean":"2023-10-11T09:40:43.145323+0000","last_became_active":"2023-10-11T09:40:41.224967+0000","last_became_peered":"2023-10-11T09:40:41.224967+0000","last_unstale":"2023-10-11T09:40:43.145323+0000","last_undegraded":"2023-10-11T09:40:43.145323+0000","last_fullsized":"2023-10-11T09:40:43.145323+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:33:10.649755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145747+0000","last_change":"2023-10-11T09:40:41.703018+0000","last_active":"2023-10-11T09:40:43.145747+0000","last_peered":"2023-10-11T09:40:43.145747+0000","last_clean":"2023-10-11T09:40:43.145747+0000","last_became_active":"2023-10-11T09:40:41.702729+0000","last_became_peered":"2023-10-11T09:40:41.702729+0000","last_unstale":"2023-10-11T09:40:43.145747+0000","last_undegraded":"2023-10-11T09:40:43.145747+0000","last_fullsized":"2023-10-11T09:40:43.145747+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:07:02.302344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.692298+0000","last_change":"2023-10-11T09:40:41.234929+0000","last_active":"2023-10-11T09:40:42.692298+0000","last_peered":"2023-10-11T09:40:42.692298+0000","last_clean":"2023-10-11T09:40:42.692298+0000","last_became_active":"2023-10-11T09:40:41.234149+0000","last_became_peered":"2023-10-11T09:40:41.234149+0000","last_unstale":"2023-10-11T09:40:42.692298+0000","last_undegraded":"2023-10-11T09:40:42.692298+0000","last_fullsized":"2023-10-11T09:40:42.692298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:17:35.640470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.688569+0000","last_change":"2023-10-11T09:40:41.716719+0000","last_active":"2023-10-11T09:40:42.688569+0000","last_peered":"2023-10-11T09:40:42.688569+0000","last_clean":"2023-10-11T09:40:42.688569+0000","last_became_active":"2023-10-11T09:40:41.716181+0000","last_became_peered":"2023-10-11T09:40:41.716181+0000","last_unstale":"2023-10-11T09:40:42.688569+0000","last_undegraded":"2023-10-11T09:40:42.688569+0000","last_fullsized":"2023-10-11T09:40:42.688569+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:50:15.829941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.143863+0000","last_change":"2023-10-11T09:40:41.229549+0000","last_active":"2023-10-11T09:40:43.143863+0000","last_peered":"2023-10-11T09:40:43.143863+0000","last_clean":"2023-10-11T09:40:43.143863+0000","last_became_active":"2023-10-11T09:40:41.229259+0000","last_became_peered":"2023-10-11T09:40:41.229259+0000","last_unstale":"2023-10-11T09:40:43.143863+0000","last_undegraded":"2023-10-11T09:40:43.143863+0000","last_fullsized":"2023-10-11T09:40:43.143863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:16:37.265056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.692895+0000","last_change":"2023-10-11T09:40:41.234477+0000","last_active":"2023-10-11T09:40:42.692895+0000","last_peered":"2023-10-11T09:40:42.692895+0000","last_clean":"2023-10-11T09:40:42.692895+0000","last_became_active":"2023-10-11T09:40:41.233977+0000","last_became_peered":"2023-10-11T09:40:41.233977+0000","last_unstale":"2023-10-11T09:40:42.692895+0000","last_undegraded":"2023-10-11T09:40:42.692895+0000","last_fullsized":"2023-10-11T09:40:42.692895+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:47:25.022675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.693696+0000","last_change":"2023-10-11T09:40:41.717799+0000","last_active":"2023-10-11T09:40:42.693696+0000","last_peered":"2023-10-11T09:40:42.693696+0000","last_clean":"2023-10-11T09:40:42.693696+0000","last_became_active":"2023-10-11T09:40:41.717520+0000","last_became_peered":"2023-10-11T09:40:41.717520+0000","last_unstale":"2023-10-11T09:40:42.693696+0000","last_undegraded":"2023-10-11T09:40:42.693696+0000","last_fullsized":"2023-10-11T09:40:42.693696+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:35:36.436236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.692703+0000","last_change":"2023-10-11T09:40:41.234979+0000","last_active":"2023-10-11T09:40:42.692703+0000","last_peered":"2023-10-11T09:40:42.692703+0000","last_clean":"2023-10-11T09:40:42.692703+0000","last_became_active":"2023-10-11T09:40:41.234245+0000","last_became_peered":"2023-10-11T09:40:41.234245+0000","last_unstale":"2023-10-11T09:40:42.692703+0000","last_undegraded":"2023-10-11T09:40:42.692703+0000","last_fullsized":"2023-10-11T09:40:42.692703+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:31:56.724263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911330+0000","last_change":"2023-10-11T09:40:41.706936+0000","last_active":"2023-10-11T09:40:42.911330+0000","last_peered":"2023-10-11T09:40:42.911330+0000","last_clean":"2023-10-11T09:40:42.911330+0000","last_became_active":"2023-10-11T09:40:41.706440+0000","last_became_peered":"2023-10-11T09:40:41.706440+0000","last_unstale":"2023-10-11T09:40:42.911330+0000","last_undegraded":"2023-10-11T09:40:42.911330+0000","last_fullsized":"2023-10-11T09:40:42.911330+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:12:14.951424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.692744+0000","last_change":"2023-10-11T09:40:41.228740+0000","last_active":"2023-10-11T09:40:42.692744+0000","last_peered":"2023-10-11T09:40:42.692744+0000","last_clean":"2023-10-11T09:40:42.692744+0000","last_became_active":"2023-10-11T09:40:41.228327+0000","last_became_peered":"2023-10-11T09:40:41.228327+0000","last_unstale":"2023-10-11T09:40:42.692744+0000","last_undegraded":"2023-10-11T09:40:42.692744+0000","last_fullsized":"2023-10-11T09:40:42.692744+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:59:14.883485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144214+0000","last_change":"2023-10-11T09:40:41.721661+0000","last_active":"2023-10-11T09:40:43.144214+0000","last_peered":"2023-10-11T09:40:43.144214+0000","last_clean":"2023-10-11T09:40:43.144214+0000","last_became_active":"2023-10-11T09:40:41.720154+0000","last_became_peered":"2023-10-11T09:40:41.720154+0000","last_unstale":"2023-10-11T09:40:43.144214+0000","last_undegraded":"2023-10-11T09:40:43.144214+0000","last_fullsized":"2023-10-11T09:40:43.144214+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:45:40.098819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.693986+0000","last_change":"2023-10-11T09:40:41.225444+0000","last_active":"2023-10-11T09:40:42.693986+0000","last_peered":"2023-10-11T09:40:42.693986+0000","last_clean":"2023-10-11T09:40:42.693986+0000","last_became_active":"2023-10-11T09:40:41.225159+0000","last_became_peered":"2023-10-11T09:40:41.225159+0000","last_unstale":"2023-10-11T09:40:42.693986+0000","last_undegraded":"2023-10-11T09:40:42.693986+0000","last_fullsized":"2023-10-11T09:40:42.693986+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:00:56.048004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145496+0000","last_change":"2023-10-11T09:40:41.712920+0000","last_active":"2023-10-11T09:40:43.145496+0000","last_peered":"2023-10-11T09:40:43.145496+0000","last_clean":"2023-10-11T09:40:43.145496+0000","last_became_active":"2023-10-11T09:40:41.712654+0000","last_became_peered":"2023-10-11T09:40:41.712654+0000","last_unstale":"2023-10-11T09:40:43.145496+0000","last_undegraded":"2023-10-11T09:40:43.145496+0000","last_fullsized":"2023-10-11T09:40:43.145496+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:23:30.814623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.688154+0000","last_change":"2023-10-11T09:40:41.229300+0000","last_active":"2023-10-11T09:40:42.688154+0000","last_peered":"2023-10-11T09:40:42.688154+0000","last_clean":"2023-10-11T09:40:42.688154+0000","last_became_active":"2023-10-11T09:40:41.228539+0000","last_became_peered":"2023-10-11T09:40:41.228539+0000","last_unstale":"2023-10-11T09:40:42.688154+0000","last_undegraded":"2023-10-11T09:40:42.688154+0000","last_fullsized":"2023-10-11T09:40:42.688154+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:09:03.936717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.912449+0000","last_change":"2023-10-11T09:40:41.705540+0000","last_active":"2023-10-11T09:40:42.912449+0000","last_peered":"2023-10-11T09:40:42.912449+0000","last_clean":"2023-10-11T09:40:42.912449+0000","last_became_active":"2023-10-11T09:40:41.705163+0000","last_became_peered":"2023-10-11T09:40:41.705163+0000","last_unstale":"2023-10-11T09:40:42.912449+0000","last_undegraded":"2023-10-11T09:40:42.912449+0000","last_fullsized":"2023-10-11T09:40:42.912449+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:56:50.343276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144408+0000","last_change":"2023-10-11T09:40:41.224108+0000","last_active":"2023-10-11T09:40:43.144408+0000","last_peered":"2023-10-11T09:40:43.144408+0000","last_clean":"2023-10-11T09:40:43.144408+0000","last_became_active":"2023-10-11T09:40:41.223643+0000","last_became_peered":"2023-10-11T09:40:41.223643+0000","last_unstale":"2023-10-11T09:40:43.144408+0000","last_undegraded":"2023-10-11T09:40:43.144408+0000","last_fullsized":"2023-10-11T09:40:43.144408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:36:34.713602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.912393+0000","last_change":"2023-10-11T09:40:41.700784+0000","last_active":"2023-10-11T09:40:42.912393+0000","last_peered":"2023-10-11T09:40:42.912393+0000","last_clean":"2023-10-11T09:40:42.912393+0000","last_became_active":"2023-10-11T09:40:41.700495+0000","last_became_peered":"2023-10-11T09:40:41.700495+0000","last_unstale":"2023-10-11T09:40:42.912393+0000","last_undegraded":"2023-10-11T09:40:42.912393+0000","last_fullsized":"2023-10-11T09:40:42.912393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:57:16.072435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911546+0000","last_change":"2023-10-11T09:40:41.233232+0000","last_active":"2023-10-11T09:40:42.911546+0000","last_peered":"2023-10-11T09:40:42.911546+0000","last_clean":"2023-10-11T09:40:42.911546+0000","last_became_active":"2023-10-11T09:40:41.232092+0000","last_became_peered":"2023-10-11T09:40:41.232092+0000","last_unstale":"2023-10-11T09:40:42.911546+0000","last_undegraded":"2023-10-11T09:40:42.911546+0000","last_fullsized":"2023-10-11T09:40:42.911546+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:16:37.265056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.912288+0000","last_change":"2023-10-11T09:40:41.721557+0000","last_active":"2023-10-11T09:40:42.912288+0000","last_peered":"2023-10-11T09:40:42.912288+0000","last_clean":"2023-10-11T09:40:42.912288+0000","last_became_active":"2023-10-11T09:40:41.721314+0000","last_became_peered":"2023-10-11T09:40:41.721314+0000","last_unstale":"2023-10-11T09:40:42.912288+0000","last_undegraded":"2023-10-11T09:40:42.912288+0000","last_fullsized":"2023-10-11T09:40:42.912288+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:59:15.898506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687786+0000","last_change":"2023-10-11T09:40:41.228065+0000","last_active":"2023-10-11T09:40:42.687786+0000","last_peered":"2023-10-11T09:40:42.687786+0000","last_clean":"2023-10-11T09:40:42.687786+0000","last_became_active":"2023-10-11T09:40:41.227765+0000","last_became_peered":"2023-10-11T09:40:41.227765+0000","last_unstale":"2023-10-11T09:40:42.687786+0000","last_undegraded":"2023-10-11T09:40:42.687786+0000","last_fullsized":"2023-10-11T09:40:42.687786+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:36:34.713602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144923+0000","last_change":"2023-10-11T09:40:41.708864+0000","last_active":"2023-10-11T09:40:43.144923+0000","last_peered":"2023-10-11T09:40:43.144923+0000","last_clean":"2023-10-11T09:40:43.144923+0000","last_became_active":"2023-10-11T09:40:41.708555+0000","last_became_peered":"2023-10-11T09:40:41.708555+0000","last_unstale":"2023-10-11T09:40:43.144923+0000","last_undegraded":"2023-10-11T09:40:43.144923+0000","last_fullsized":"2023-10-11T09:40:43.144923+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:43:30.691251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":167,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-11T09:40:41.695042+0000","last_change":"2023-10-11T09:40:29.874372+0000","last_active":"2023-10-11T09:40:41.695042+0000","last_peered":"2023-10-11T09:40:41.695042+0000","last_clean":"2023-10-11T09:40:41.695042+0000","last_became_active":"2023-10-11T09:40:29.873621+0000","last_became_peered":"2023-10-11T09:40:29.873621+0000","last_unstale":"2023-10-11T09:40:41.695042+0000","last_undegraded":"2023-10-11T09:40:41.695042+0000","last_fullsized":"2023-10-11T09:40:41.695042+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_clean_scrub_stamp":"2023-10-11T09:40:28.742276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:22:07.985311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-11T09:40:41.705703+0000","last_change":"2023-10-11T09:40:36.148773+0000","last_active":"2023-10-11T09:40:41.705703+0000","last_peered":"2023-10-11T09:40:41.705703+0000","last_clean":"2023-10-11T09:40:41.705703+0000","last_became_active":"2023-10-11T09:40:36.148489+0000","last_became_peered":"2023-10-11T09:40:36.148489+0000","last_unstale":"2023-10-11T09:40:41.705703+0000","last_undegraded":"2023-10-11T09:40:41.705703+0000","last_fullsized":"2023-10-11T09:40:41.705703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687547+0000","last_change":"2023-10-11T09:40:41.227668+0000","last_active":"2023-10-11T09:40:42.687547+0000","last_peered":"2023-10-11T09:40:42.687547+0000","last_clean":"2023-10-11T09:40:42.687547+0000","last_became_active":"2023-10-11T09:40:41.227357+0000","last_became_peered":"2023-10-11T09:40:41.227357+0000","last_unstale":"2023-10-11T09:40:42.687547+0000","last_undegraded":"2023-10-11T09:40:42.687547+0000","last_fullsized":"2023-10-11T09:40:42.687547+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:16:37.265056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144334+0000","last_change":"2023-10-11T09:40:41.726684+0000","last_active":"2023-10-11T09:40:43.144334+0000","last_peered":"2023-10-11T09:40:43.144334+0000","last_clean":"2023-10-11T09:40:43.144334+0000","last_became_active":"2023-10-11T09:40:41.726323+0000","last_became_peered":"2023-10-11T09:40:41.726323+0000","last_unstale":"2023-10-11T09:40:43.144334+0000","last_undegraded":"2023-10-11T09:40:43.144334+0000","last_fullsized":"2023-10-11T09:40:43.144334+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:47:26.037696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-11T09:40:41.694934+0000","last_change":"2023-10-11T09:40:36.149739+0000","last_active":"2023-10-11T09:40:41.694934+0000","last_peered":"2023-10-11T09:40:41.694934+0000","last_clean":"2023-10-11T09:40:41.694934+0000","last_became_active":"2023-10-11T09:40:36.149419+0000","last_became_peered":"2023-10-11T09:40:36.149419+0000","last_unstale":"2023-10-11T09:40:41.694934+0000","last_undegraded":"2023-10-11T09:40:41.694934+0000","last_fullsized":"2023-10-11T09:40:41.694934+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:54:50.640362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.143706+0000","last_change":"2023-10-11T09:40:41.224529+0000","last_active":"2023-10-11T09:40:43.143706+0000","last_peered":"2023-10-11T09:40:43.143706+0000","last_clean":"2023-10-11T09:40:43.143706+0000","last_became_active":"2023-10-11T09:40:41.224252+0000","last_became_peered":"2023-10-11T09:40:41.224252+0000","last_unstale":"2023-10-11T09:40:43.143706+0000","last_undegraded":"2023-10-11T09:40:43.143706+0000","last_fullsized":"2023-10-11T09:40:43.143706+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:09:03.936717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145788+0000","last_change":"2023-10-11T09:40:41.718539+0000","last_active":"2023-10-11T09:40:43.145788+0000","last_peered":"2023-10-11T09:40:43.145788+0000","last_clean":"2023-10-11T09:40:43.145788+0000","last_became_active":"2023-10-11T09:40:41.718231+0000","last_became_peered":"2023-10-11T09:40:41.718231+0000","last_unstale":"2023-10-11T09:40:43.145788+0000","last_undegraded":"2023-10-11T09:40:43.145788+0000","last_fullsized":"2023-10-11T09:40:43.145788+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:50:15.829941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-11T09:40:41.682824+0000","last_change":"2023-10-11T09:40:36.216248+0000","last_active":"2023-10-11T09:40:41.682824+0000","last_peered":"2023-10-11T09:40:41.682824+0000","last_clean":"2023-10-11T09:40:41.682824+0000","last_became_active":"2023-10-11T09:40:36.215890+0000","last_became_peered":"2023-10-11T09:40:36.215890+0000","last_unstale":"2023-10-11T09:40:41.682824+0000","last_undegraded":"2023-10-11T09:40:41.682824+0000","last_fullsized":"2023-10-11T09:40:41.682824+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.143998+0000","last_change":"2023-10-11T09:40:41.220987+0000","last_active":"2023-10-11T09:40:43.143998+0000","last_peered":"2023-10-11T09:40:43.143998+0000","last_clean":"2023-10-11T09:40:43.143998+0000","last_became_active":"2023-10-11T09:40:41.220035+0000","last_became_peered":"2023-10-11T09:40:41.220035+0000","last_unstale":"2023-10-11T09:40:43.143998+0000","last_undegraded":"2023-10-11T09:40:43.143998+0000","last_fullsized":"2023-10-11T09:40:43.143998+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:00:56.048004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144958+0000","last_change":"2023-10-11T09:40:41.718356+0000","last_active":"2023-10-11T09:40:43.144958+0000","last_peered":"2023-10-11T09:40:43.144958+0000","last_clean":"2023-10-11T09:40:43.144958+0000","last_became_active":"2023-10-11T09:40:41.718069+0000","last_became_peered":"2023-10-11T09:40:41.718069+0000","last_unstale":"2023-10-11T09:40:43.144958+0000","last_undegraded":"2023-10-11T09:40:43.144958+0000","last_fullsized":"2023-10-11T09:40:43.144958+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:42:34.821513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-11T09:40:41.705511+0000","last_change":"2023-10-11T09:40:36.151710+0000","last_active":"2023-10-11T09:40:41.705511+0000","last_peered":"2023-10-11T09:40:41.705511+0000","last_clean":"2023-10-11T09:40:41.705511+0000","last_became_active":"2023-10-11T09:40:36.151219+0000","last_became_peered":"2023-10-11T09:40:36.151219+0000","last_unstale":"2023-10-11T09:40:41.705511+0000","last_undegraded":"2023-10-11T09:40:41.705511+0000","last_fullsized":"2023-10-11T09:40:41.705511+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911785+0000","last_change":"2023-10-11T09:40:41.231464+0000","last_active":"2023-10-11T09:40:42.911785+0000","last_peered":"2023-10-11T09:40:42.911785+0000","last_clean":"2023-10-11T09:40:42.911785+0000","last_became_active":"2023-10-11T09:40:41.229072+0000","last_became_peered":"2023-10-11T09:40:41.229072+0000","last_unstale":"2023-10-11T09:40:42.911785+0000","last_undegraded":"2023-10-11T09:40:42.911785+0000","last_fullsized":"2023-10-11T09:40:42.911785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:33:10.649755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144540+0000","last_change":"2023-10-11T09:40:41.715638+0000","last_active":"2023-10-11T09:40:43.144540+0000","last_peered":"2023-10-11T09:40:43.144540+0000","last_clean":"2023-10-11T09:40:43.144540+0000","last_became_active":"2023-10-11T09:40:41.715399+0000","last_became_peered":"2023-10-11T09:40:41.715399+0000","last_unstale":"2023-10-11T09:40:43.144540+0000","last_undegraded":"2023-10-11T09:40:43.144540+0000","last_fullsized":"2023-10-11T09:40:43.144540+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:00:05.422788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.694217+0000","last_change":"2023-10-11T09:40:41.231735+0000","last_active":"2023-10-11T09:40:42.694217+0000","last_peered":"2023-10-11T09:40:42.694217+0000","last_clean":"2023-10-11T09:40:42.694217+0000","last_became_active":"2023-10-11T09:40:41.229777+0000","last_became_peered":"2023-10-11T09:40:41.229777+0000","last_unstale":"2023-10-11T09:40:42.694217+0000","last_undegraded":"2023-10-11T09:40:42.694217+0000","last_fullsized":"2023-10-11T09:40:42.694217+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:09:03.936717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-11T09:40:41.695209+0000","last_change":"2023-10-11T09:40:36.160620+0000","last_active":"2023-10-11T09:40:41.695209+0000","last_peered":"2023-10-11T09:40:41.695209+0000","last_clean":"2023-10-11T09:40:41.695209+0000","last_became_active":"2023-10-11T09:40:36.160171+0000","last_became_peered":"2023-10-11T09:40:36.160171+0000","last_unstale":"2023-10-11T09:40:41.695209+0000","last_undegraded":"2023-10-11T09:40:41.695209+0000","last_fullsized":"2023-10-11T09:40:41.695209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145219+0000","last_change":"2023-10-11T09:40:41.718869+0000","last_active":"2023-10-11T09:40:43.145219+0000","last_peered":"2023-10-11T09:40:43.145219+0000","last_clean":"2023-10-11T09:40:43.145219+0000","last_became_active":"2023-10-11T09:40:41.718628+0000","last_became_peered":"2023-10-11T09:40:41.718628+0000","last_unstale":"2023-10-11T09:40:43.145219+0000","last_undegraded":"2023-10-11T09:40:43.145219+0000","last_fullsized":"2023-10-11T09:40:43.145219+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:50:15.829941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144700+0000","last_change":"2023-10-11T09:40:41.225562+0000","last_active":"2023-10-11T09:40:43.144700+0000","last_peered":"2023-10-11T09:40:43.144700+0000","last_clean":"2023-10-11T09:40:43.144700+0000","last_became_active":"2023-10-11T09:40:41.222909+0000","last_became_peered":"2023-10-11T09:40:41.222909+0000","last_unstale":"2023-10-11T09:40:43.144700+0000","last_undegraded":"2023-10-11T09:40:43.144700+0000","last_fullsized":"2023-10-11T09:40:43.144700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:09:03.936717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-11T09:40:41.696680+0000","last_change":"2023-10-11T09:40:36.162174+0000","last_active":"2023-10-11T09:40:41.696680+0000","last_peered":"2023-10-11T09:40:41.696680+0000","last_clean":"2023-10-11T09:40:41.696680+0000","last_became_active":"2023-10-11T09:40:36.161710+0000","last_became_peered":"2023-10-11T09:40:36.161710+0000","last_unstale":"2023-10-11T09:40:41.696680+0000","last_undegraded":"2023-10-11T09:40:41.696680+0000","last_fullsized":"2023-10-11T09:40:41.696680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:57:03.630878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145842+0000","last_change":"2023-10-11T09:40:41.716886+0000","last_active":"2023-10-11T09:40:43.145842+0000","last_peered":"2023-10-11T09:40:43.145842+0000","last_clean":"2023-10-11T09:40:43.145842+0000","last_became_active":"2023-10-11T09:40:41.716601+0000","last_became_peered":"2023-10-11T09:40:41.716601+0000","last_unstale":"2023-10-11T09:40:43.145842+0000","last_undegraded":"2023-10-11T09:40:43.145842+0000","last_fullsized":"2023-10-11T09:40:43.145842+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:00:05.422788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.688690+0000","last_change":"2023-10-11T09:40:41.224655+0000","last_active":"2023-10-11T09:40:42.688690+0000","last_peered":"2023-10-11T09:40:42.688690+0000","last_clean":"2023-10-11T09:40:42.688690+0000","last_became_active":"2023-10-11T09:40:41.224374+0000","last_became_peered":"2023-10-11T09:40:41.224374+0000","last_unstale":"2023-10-11T09:40:42.688690+0000","last_undegraded":"2023-10-11T09:40:42.688690+0000","last_fullsized":"2023-10-11T09:40:42.688690+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:33:10.649755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-11T09:40:41.699405+0000","last_change":"2023-10-11T09:40:36.150224+0000","last_active":"2023-10-11T09:40:41.699405+0000","last_peered":"2023-10-11T09:40:41.699405+0000","last_clean":"2023-10-11T09:40:41.699405+0000","last_became_active":"2023-10-11T09:40:36.149919+0000","last_became_peered":"2023-10-11T09:40:36.149919+0000","last_unstale":"2023-10-11T09:40:41.699405+0000","last_undegraded":"2023-10-11T09:40:41.699405+0000","last_fullsized":"2023-10-11T09:40:41.699405+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687621+0000","last_change":"2023-10-11T09:40:41.696509+0000","last_active":"2023-10-11T09:40:42.687621+0000","last_peered":"2023-10-11T09:40:42.687621+0000","last_clean":"2023-10-11T09:40:42.687621+0000","last_became_active":"2023-10-11T09:40:41.696149+0000","last_became_peered":"2023-10-11T09:40:41.696149+0000","last_unstale":"2023-10-11T09:40:42.687621+0000","last_undegraded":"2023-10-11T09:40:42.687621+0000","last_fullsized":"2023-10-11T09:40:42.687621+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:00:05.422788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.438621+0000","last_change":"2023-10-11T09:40:41.230379+0000","last_active":"2023-10-11T09:40:43.438621+0000","last_peered":"2023-10-11T09:40:43.438621+0000","last_clean":"2023-10-11T09:40:43.438621+0000","last_became_active":"2023-10-11T09:40:41.226481+0000","last_became_peered":"2023-10-11T09:40:41.226481+0000","last_unstale":"2023-10-11T09:40:43.438621+0000","last_undegraded":"2023-10-11T09:40:43.438621+0000","last_fullsized":"2023-10-11T09:40:43.438621+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:48:51.882031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-11T09:40:41.708126+0000","last_change":"2023-10-11T09:40:36.151839+0000","last_active":"2023-10-11T09:40:41.708126+0000","last_peered":"2023-10-11T09:40:41.708126+0000","last_clean":"2023-10-11T09:40:41.708126+0000","last_became_active":"2023-10-11T09:40:36.151179+0000","last_became_peered":"2023-10-11T09:40:36.151179+0000","last_unstale":"2023-10-11T09:40:41.708126+0000","last_undegraded":"2023-10-11T09:40:41.708126+0000","last_fullsized":"2023-10-11T09:40:41.708126+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911579+0000","last_change":"2023-10-11T09:40:41.712383+0000","last_active":"2023-10-11T09:40:42.911579+0000","last_peered":"2023-10-11T09:40:42.911579+0000","last_clean":"2023-10-11T09:40:42.911579+0000","last_became_active":"2023-10-11T09:40:41.711978+0000","last_became_peered":"2023-10-11T09:40:41.711978+0000","last_unstale":"2023-10-11T09:40:42.911579+0000","last_undegraded":"2023-10-11T09:40:42.911579+0000","last_fullsized":"2023-10-11T09:40:42.911579+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:45:40.098819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911910+0000","last_change":"2023-10-11T09:40:41.720785+0000","last_active":"2023-10-11T09:40:42.911910+0000","last_peered":"2023-10-11T09:40:42.911910+0000","last_clean":"2023-10-11T09:40:42.911910+0000","last_became_active":"2023-10-11T09:40:41.720508+0000","last_became_peered":"2023-10-11T09:40:41.720508+0000","last_unstale":"2023-10-11T09:40:42.911910+0000","last_undegraded":"2023-10-11T09:40:42.911910+0000","last_fullsized":"2023-10-11T09:40:42.911910+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:35:04.646798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.912435+0000","last_change":"2023-10-11T09:40:41.229054+0000","last_active":"2023-10-11T09:40:42.912435+0000","last_peered":"2023-10-11T09:40:42.912435+0000","last_clean":"2023-10-11T09:40:42.912435+0000","last_became_active":"2023-10-11T09:40:41.227552+0000","last_became_peered":"2023-10-11T09:40:41.227552+0000","last_unstale":"2023-10-11T09:40:42.912435+0000","last_undegraded":"2023-10-11T09:40:42.912435+0000","last_fullsized":"2023-10-11T09:40:42.912435+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:36:34.713602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144970+0000","last_change":"2023-10-11T09:40:41.730718+0000","last_active":"2023-10-11T09:40:43.144970+0000","last_peered":"2023-10-11T09:40:43.144970+0000","last_clean":"2023-10-11T09:40:43.144970+0000","last_became_active":"2023-10-11T09:40:41.730499+0000","last_became_peered":"2023-10-11T09:40:41.730499+0000","last_unstale":"2023-10-11T09:40:43.144970+0000","last_undegraded":"2023-10-11T09:40:43.144970+0000","last_fullsized":"2023-10-11T09:40:43.144970+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:35:36.436236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.692532+0000","last_change":"2023-10-11T09:40:41.230822+0000","last_active":"2023-10-11T09:40:42.692532+0000","last_peered":"2023-10-11T09:40:42.692532+0000","last_clean":"2023-10-11T09:40:42.692532+0000","last_became_active":"2023-10-11T09:40:41.230417+0000","last_became_peered":"2023-10-11T09:40:41.230417+0000","last_unstale":"2023-10-11T09:40:42.692532+0000","last_undegraded":"2023-10-11T09:40:42.692532+0000","last_fullsized":"2023-10-11T09:40:42.692532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:20:53.471012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.694445+0000","last_change":"2023-10-11T09:40:41.231668+0000","last_active":"2023-10-11T09:40:42.694445+0000","last_peered":"2023-10-11T09:40:42.694445+0000","last_clean":"2023-10-11T09:40:42.694445+0000","last_became_active":"2023-10-11T09:40:41.226956+0000","last_became_peered":"2023-10-11T09:40:41.226956+0000","last_unstale":"2023-10-11T09:40:42.694445+0000","last_undegraded":"2023-10-11T09:40:42.694445+0000","last_fullsized":"2023-10-11T09:40:42.694445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:43:00.315992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687648+0000","last_change":"2023-10-11T09:40:41.709493+0000","last_active":"2023-10-11T09:40:42.687648+0000","last_peered":"2023-10-11T09:40:42.687648+0000","last_clean":"2023-10-11T09:40:42.687648+0000","last_became_active":"2023-10-11T09:40:41.709217+0000","last_became_peered":"2023-10-11T09:40:41.709217+0000","last_unstale":"2023-10-11T09:40:42.687648+0000","last_undegraded":"2023-10-11T09:40:42.687648+0000","last_fullsized":"2023-10-11T09:40:42.687648+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:07:02.302344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.146199+0000","last_change":"2023-10-11T09:40:41.228934+0000","last_active":"2023-10-11T09:40:43.146199+0000","last_peered":"2023-10-11T09:40:43.146199+0000","last_clean":"2023-10-11T09:40:43.146199+0000","last_became_active":"2023-10-11T09:40:41.226833+0000","last_became_peered":"2023-10-11T09:40:41.226833+0000","last_unstale":"2023-10-11T09:40:43.146199+0000","last_undegraded":"2023-10-11T09:40:43.146199+0000","last_fullsized":"2023-10-11T09:40:43.146199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:00:56.048004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687536+0000","last_change":"2023-10-11T09:40:41.715540+0000","last_active":"2023-10-11T09:40:42.687536+0000","last_peered":"2023-10-11T09:40:42.687536+0000","last_clean":"2023-10-11T09:40:42.687536+0000","last_became_active":"2023-10-11T09:40:41.715254+0000","last_became_peered":"2023-10-11T09:40:41.715254+0000","last_unstale":"2023-10-11T09:40:42.687536+0000","last_undegraded":"2023-10-11T09:40:42.687536+0000","last_fullsized":"2023-10-11T09:40:42.687536+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:23:30.814623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144670+0000","last_change":"2023-10-11T09:40:41.722871+0000","last_active":"2023-10-11T09:40:43.144670+0000","last_peered":"2023-10-11T09:40:43.144670+0000","last_clean":"2023-10-11T09:40:43.144670+0000","last_became_active":"2023-10-11T09:40:41.722538+0000","last_became_peered":"2023-10-11T09:40:41.722538+0000","last_unstale":"2023-10-11T09:40:43.144670+0000","last_undegraded":"2023-10-11T09:40:43.144670+0000","last_fullsized":"2023-10-11T09:40:43.144670+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:55:31.027029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687761+0000","last_change":"2023-10-11T09:40:41.230916+0000","last_active":"2023-10-11T09:40:42.687761+0000","last_peered":"2023-10-11T09:40:42.687761+0000","last_clean":"2023-10-11T09:40:42.687761+0000","last_became_active":"2023-10-11T09:40:41.230639+0000","last_became_peered":"2023-10-11T09:40:41.230639+0000","last_unstale":"2023-10-11T09:40:42.687761+0000","last_undegraded":"2023-10-11T09:40:42.687761+0000","last_fullsized":"2023-10-11T09:40:42.687761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:48:51.882031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145531+0000","last_change":"2023-10-11T09:40:41.705483+0000","last_active":"2023-10-11T09:40:43.145531+0000","last_peered":"2023-10-11T09:40:43.145531+0000","last_clean":"2023-10-11T09:40:43.145531+0000","last_became_active":"2023-10-11T09:40:41.705060+0000","last_became_peered":"2023-10-11T09:40:41.705060+0000","last_unstale":"2023-10-11T09:40:43.145531+0000","last_undegraded":"2023-10-11T09:40:43.145531+0000","last_fullsized":"2023-10-11T09:40:43.145531+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:35:36.436236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144834+0000","last_change":"2023-10-11T09:40:41.227035+0000","last_active":"2023-10-11T09:40:43.144834+0000","last_peered":"2023-10-11T09:40:43.144834+0000","last_clean":"2023-10-11T09:40:43.144834+0000","last_became_active":"2023-10-11T09:40:41.226593+0000","last_became_peered":"2023-10-11T09:40:41.226593+0000","last_unstale":"2023-10-11T09:40:43.144834+0000","last_undegraded":"2023-10-11T09:40:43.144834+0000","last_fullsized":"2023-10-11T09:40:43.144834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:48:51.882031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.688081+0000","last_change":"2023-10-11T09:40:41.715379+0000","last_active":"2023-10-11T09:40:42.688081+0000","last_peered":"2023-10-11T09:40:42.688081+0000","last_clean":"2023-10-11T09:40:42.688081+0000","last_became_active":"2023-10-11T09:40:41.715091+0000","last_became_peered":"2023-10-11T09:40:41.715091+0000","last_unstale":"2023-10-11T09:40:42.688081+0000","last_undegraded":"2023-10-11T09:40:42.688081+0000","last_fullsized":"2023-10-11T09:40:42.688081+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:42:34.821513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911962+0000","last_change":"2023-10-11T09:40:41.235100+0000","last_active":"2023-10-11T09:40:42.911962+0000","last_peered":"2023-10-11T09:40:42.911962+0000","last_clean":"2023-10-11T09:40:42.911962+0000","last_became_active":"2023-10-11T09:40:41.233361+0000","last_became_peered":"2023-10-11T09:40:41.233361+0000","last_unstale":"2023-10-11T09:40:42.911962+0000","last_undegraded":"2023-10-11T09:40:42.911962+0000","last_fullsized":"2023-10-11T09:40:42.911962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:12:09.780799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144249+0000","last_change":"2023-10-11T09:40:41.705452+0000","last_active":"2023-10-11T09:40:43.144249+0000","last_peered":"2023-10-11T09:40:43.144249+0000","last_clean":"2023-10-11T09:40:43.144249+0000","last_became_active":"2023-10-11T09:40:41.705028+0000","last_became_peered":"2023-10-11T09:40:41.705028+0000","last_unstale":"2023-10-11T09:40:43.144249+0000","last_undegraded":"2023-10-11T09:40:43.144249+0000","last_fullsized":"2023-10-11T09:40:43.144249+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:09:04.951739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145980+0000","last_change":"2023-10-11T09:40:41.232125+0000","last_active":"2023-10-11T09:40:43.145980+0000","last_peered":"2023-10-11T09:40:43.145980+0000","last_clean":"2023-10-11T09:40:43.145980+0000","last_became_active":"2023-10-11T09:40:41.231730+0000","last_became_peered":"2023-10-11T09:40:41.231730+0000","last_unstale":"2023-10-11T09:40:43.145980+0000","last_undegraded":"2023-10-11T09:40:43.145980+0000","last_fullsized":"2023-10-11T09:40:43.145980+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:12:04.722202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911689+0000","last_change":"2023-10-11T09:40:41.702062+0000","last_active":"2023-10-11T09:40:42.911689+0000","last_peered":"2023-10-11T09:40:42.911689+0000","last_clean":"2023-10-11T09:40:42.911689+0000","last_became_active":"2023-10-11T09:40:41.701665+0000","last_became_peered":"2023-10-11T09:40:41.701665+0000","last_unstale":"2023-10-11T09:40:42.911689+0000","last_undegraded":"2023-10-11T09:40:42.911689+0000","last_fullsized":"2023-10-11T09:40:42.911689+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:03:52.325886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.693906+0000","last_change":"2023-10-11T09:40:41.227957+0000","last_active":"2023-10-11T09:40:42.693906+0000","last_peered":"2023-10-11T09:40:42.693906+0000","last_clean":"2023-10-11T09:40:42.693906+0000","last_became_active":"2023-10-11T09:40:41.226855+0000","last_became_peered":"2023-10-11T09:40:41.226855+0000","last_unstale":"2023-10-11T09:40:42.693906+0000","last_undegraded":"2023-10-11T09:40:42.693906+0000","last_fullsized":"2023-10-11T09:40:42.693906+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:12:04.722202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911772+0000","last_change":"2023-10-11T09:40:41.699151+0000","last_active":"2023-10-11T09:40:42.911772+0000","last_peered":"2023-10-11T09:40:42.911772+0000","last_clean":"2023-10-11T09:40:42.911772+0000","last_became_active":"2023-10-11T09:40:41.698188+0000","last_became_peered":"2023-10-11T09:40:41.698188+0000","last_unstale":"2023-10-11T09:40:42.911772+0000","last_undegraded":"2023-10-11T09:40:42.911772+0000","last_fullsized":"2023-10-11T09:40:42.911772+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:48:15.689750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.694192+0000","last_change":"2023-10-11T09:40:41.224843+0000","last_active":"2023-10-11T09:40:42.694192+0000","last_peered":"2023-10-11T09:40:42.694192+0000","last_clean":"2023-10-11T09:40:42.694192+0000","last_became_active":"2023-10-11T09:40:41.224415+0000","last_became_peered":"2023-10-11T09:40:41.224415+0000","last_unstale":"2023-10-11T09:40:42.694192+0000","last_undegraded":"2023-10-11T09:40:42.694192+0000","last_fullsized":"2023-10-11T09:40:42.694192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:31:56.724263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687814+0000","last_change":"2023-10-11T09:40:41.695801+0000","last_active":"2023-10-11T09:40:42.687814+0000","last_peered":"2023-10-11T09:40:42.687814+0000","last_clean":"2023-10-11T09:40:42.687814+0000","last_became_active":"2023-10-11T09:40:41.694814+0000","last_became_peered":"2023-10-11T09:40:41.694814+0000","last_unstale":"2023-10-11T09:40:42.687814+0000","last_undegraded":"2023-10-11T09:40:42.687814+0000","last_fullsized":"2023-10-11T09:40:42.687814+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:19:07.089979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687947+0000","last_change":"2023-10-11T09:40:41.222156+0000","last_active":"2023-10-11T09:40:42.687947+0000","last_peered":"2023-10-11T09:40:42.687947+0000","last_clean":"2023-10-11T09:40:42.687947+0000","last_became_active":"2023-10-11T09:40:41.221758+0000","last_became_peered":"2023-10-11T09:40:41.221758+0000","last_unstale":"2023-10-11T09:40:42.687947+0000","last_undegraded":"2023-10-11T09:40:42.687947+0000","last_fullsized":"2023-10-11T09:40:42.687947+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:00:56.048004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911260+0000","last_change":"2023-10-11T09:40:41.729793+0000","last_active":"2023-10-11T09:40:42.911260+0000","last_peered":"2023-10-11T09:40:42.911260+0000","last_clean":"2023-10-11T09:40:42.911260+0000","last_became_active":"2023-10-11T09:40:41.729468+0000","last_became_peered":"2023-10-11T09:40:41.729468+0000","last_unstale":"2023-10-11T09:40:42.911260+0000","last_undegraded":"2023-10-11T09:40:42.911260+0000","last_fullsized":"2023-10-11T09:40:42.911260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:20:54.486033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.146157+0000","last_change":"2023-10-11T09:40:41.231067+0000","last_active":"2023-10-11T09:40:43.146157+0000","last_peered":"2023-10-11T09:40:43.146157+0000","last_clean":"2023-10-11T09:40:43.146157+0000","last_became_active":"2023-10-11T09:40:41.229252+0000","last_became_peered":"2023-10-11T09:40:41.229252+0000","last_unstale":"2023-10-11T09:40:43.146157+0000","last_undegraded":"2023-10-11T09:40:43.146157+0000","last_fullsized":"2023-10-11T09:40:43.146157+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:43:00.315992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":864256,"data_stored":681747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1479999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.389}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0740000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.333}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.446}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.373}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.407}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.585}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":249856,"data_stored":90251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.238}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0529999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.093}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3620000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.321}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":262144,"data_stored":91219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0860000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.298}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":245760,"data_stored":89689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3540000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.542}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2969999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.474}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.786}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.538}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2689999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27772,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344068,"statfs":{"total":96636764160,"available":96608325632,"internally_reserved":0,"allocated":839680,"data_stored":680549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1879999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.125}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.744}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5449999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.825}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8759999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-11T09:40:52.974 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-11T09:40:53.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.174+0000 7fa9e6be8700 1 Processor -- start 2023-10-11T09:40:53.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.174+0000 7fa9e6be8700 1 -- start start 2023-10-11T09:40:53.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.174+0000 7fa9e6be8700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa9e00b0db0 0x7fa9e0160c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:53.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.174+0000 7fa9e6be8700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9e0160210 con 0x7fa9e015d050 2023-10-11T09:40:53.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.174+0000 7fa9e6be8700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9e01611a0 con 0x7fa9e00cc820 2023-10-11T09:40:53.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.174+0000 7fa9e6be8700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9e00ccfc0 con 0x7fa9e00b0db0 2023-10-11T09:40:53.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.174+0000 7fa9e5be6700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fa9e015d050 0x7fa9e015d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:58928/0 (socket says 172.21.15.78:58928) 2023-10-11T09:40:53.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.174+0000 7fa9e5be6700 1 -- 172.21.15.78:0/3231126969 learned_addr learned my addr 172.21.15.78:0/3231126969 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:53.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.178+0000 7fa9e63e7700 1 --2- 172.21.15.78:0/3231126969 >> v2:172.21.15.133:3300/0 conn(0x7fa9e00b0db0 0x7fa9e0160c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:53.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.178+0000 7fa9e4be4700 1 -- 172.21.15.78:0/3231126969 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2589035226 0 0) 0x7fa9e0160210 con 0x7fa9e015d050 2023-10-11T09:40:53.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.178+0000 7fa9e4be4700 1 -- 172.21.15.78:0/3231126969 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9cc0035d0 con 0x7fa9e015d050 2023-10-11T09:40:53.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.178+0000 7fa9e4be4700 1 -- 172.21.15.78:0/3231126969 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3230684660 0 0) 0x7fa9e01611a0 con 0x7fa9e00cc820 2023-10-11T09:40:53.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.178+0000 7fa9e4be4700 1 -- 172.21.15.78:0/3231126969 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9e0160210 con 0x7fa9e00cc820 2023-10-11T09:40:53.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.178+0000 7fa9e4be4700 1 -- 172.21.15.78:0/3231126969 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3819927748 0 0) 0x7fa9cc0035d0 con 0x7fa9e015d050 2023-10-11T09:40:53.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.178+0000 7fa9e4be4700 1 -- 172.21.15.78:0/3231126969 >> v2:172.21.15.133:3300/0 conn(0x7fa9e00b0db0 msgr2=0x7fa9e0160c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:53.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.178+0000 7fa9e4be4700 1 --2- 172.21.15.78:0/3231126969 >> v2:172.21.15.133:3300/0 conn(0x7fa9e00b0db0 0x7fa9e0160c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:53.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.178+0000 7fa9e4be4700 1 -- 172.21.15.78:0/3231126969 >> v1:172.21.15.133:6789/0 conn(0x7fa9e00cc820 legacy=0x7fa9e00b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:53.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.178+0000 7fa9e4be4700 1 -- 172.21.15.78:0/3231126969 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e00cd2a0 con 0x7fa9e015d050 2023-10-11T09:40:53.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.178+0000 7fa9e4be4700 1 -- 172.21.15.78:0/3231126969 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7fa9dc001670 con 0x7fa9e015d050 2023-10-11T09:40:53.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.178+0000 7fa9e4be4700 1 -- 172.21.15.78:0/3231126969 >> v1:172.21.15.78:6789/0 conn(0x7fa9e015d050 legacy=0x7fa9e015d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:53.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.178+0000 7fa9e4be4700 1 --2- 172.21.15.78:0/3231126969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9cc003c00 0x7fa9cc003ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:53.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.178+0000 7fa9e4be4700 1 --2- 172.21.15.78:0/3231126969 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa9cc004870 0x7fa9cc004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:53.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.178+0000 7fa9e4be4700 1 --2- 172.21.15.78:0/3231126969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9cc005220 0x7fa9cc00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:53.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.178+0000 7fa9e4be4700 1 -- 172.21.15.78:0/3231126969 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e00ccfc0 con 0x7fa9cc005220 2023-10-11T09:40:53.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.178+0000 7fa9e4be4700 1 -- 172.21.15.78:0/3231126969 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc00c100 con 0x7fa9cc003c00 2023-10-11T09:40:53.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.178+0000 7fa9e4be4700 1 -- 172.21.15.78:0/3231126969 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc00c290 con 0x7fa9cc004870 2023-10-11T09:40:53.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.178+0000 7fa9e63e7700 1 --2- 172.21.15.78:0/3231126969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9cc005220 0x7fa9cc00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:53.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.178+0000 7fa9e5be6700 1 --2- 172.21.15.78:0/3231126969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9cc003c00 0x7fa9cc003ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:53.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.178+0000 7fa9e53e5700 1 --2- 172.21.15.78:0/3231126969 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa9cc004870 0x7fa9cc004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:53.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.178+0000 7fa9e63e7700 1 -- 172.21.15.78:0/3231126969 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa9cc004870 msgr2=0x7fa9cc004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:53.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.178+0000 7fa9e63e7700 1 --2- 172.21.15.78:0/3231126969 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa9cc004870 0x7fa9cc004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:53.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.178+0000 7fa9e63e7700 1 -- 172.21.15.78:0/3231126969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9cc003c00 msgr2=0x7fa9cc003ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:53.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.178+0000 7fa9e63e7700 1 --2- 172.21.15.78:0/3231126969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9cc003c00 0x7fa9cc003ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:53.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.178+0000 7fa9e63e7700 1 -- 172.21.15.78:0/3231126969 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9cc0035d0 con 0x7fa9cc005220 2023-10-11T09:40:53.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.178+0000 7fa9e53e5700 1 --2- 172.21.15.78:0/3231126969 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa9cc004870 0x7fa9cc004ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:40:53.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.178+0000 7fa9e63e7700 1 -- 172.21.15.78:0/3231126969 reap_dead start 2023-10-11T09:40:53.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.178+0000 7fa9e63e7700 1 --2- 172.21.15.78:0/3231126969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9cc005220 0x7fa9cc00ba30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8009f90 tx=0x7fa9cc0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd06f51dd387918 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:53.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.178+0000 7fa9e4be4700 1 -- 172.21.15.78:0/3231126969 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8003a40 con 0x7fa9cc005220 2023-10-11T09:40:53.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.178+0000 7fa9e4be4700 1 -- 172.21.15.78:0/3231126969 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d800f040 con 0x7fa9cc005220 2023-10-11T09:40:53.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.178+0000 7fa9e4be4700 1 -- 172.21.15.78:0/3231126969 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8003de0 con 0x7fa9cc005220 2023-10-11T09:40:53.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.178+0000 7fa9e6be8700 1 -- 172.21.15.78:0/3231126969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9cc005220 msgr2=0x7fa9cc00ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:53.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.178+0000 7fa9e6be8700 1 --2- 172.21.15.78:0/3231126969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9cc005220 0x7fa9cc00ba30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8009f90 tx=0x7fa9cc0058a0 comp rx=0 tx=0).stop 2023-10-11T09:40:53.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.178+0000 7fa9e6be8700 1 -- 172.21.15.78:0/3231126969 shutdown_connections 2023-10-11T09:40:53.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.178+0000 7fa9e6be8700 1 --2- 172.21.15.78:0/3231126969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9cc005220 0x7fa9cc00ba30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:53.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.178+0000 7fa9e6be8700 1 -- 172.21.15.78:0/3231126969 >> 172.21.15.78:0/3231126969 conn(0x7fa9e00cea60 msgr2=0x7fa9e00cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:53.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.182+0000 7fa9e6be8700 1 -- 172.21.15.78:0/3231126969 shutdown_connections 2023-10-11T09:40:53.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.182+0000 7fa9e6be8700 1 -- 172.21.15.78:0/3231126969 wait complete. 2023-10-11T09:40:53.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.182+0000 7fa9e6be8700 1 Processor -- start 2023-10-11T09:40:53.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.182+0000 7fa9e6be8700 1 -- start start 2023-10-11T09:40:53.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.186+0000 7fa9e6be8700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa9cc005220 0x7fa9e014e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:53.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.186+0000 7fa9e6be8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9e00aeee0 0x7fa9e00af330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:53.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.186+0000 7fa9e6be8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9e0155130 0x7fa9e01555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:53.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.186+0000 7fa9e6be8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e0155ae0 con 0x7fa9e0155130 2023-10-11T09:40:53.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.186+0000 7fa9e6be8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e00b4990 con 0x7fa9e00aeee0 2023-10-11T09:40:53.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.186+0000 7fa9e6be8700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e00af870 con 0x7fa9cc005220 2023-10-11T09:40:53.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.186+0000 7fa9e63e7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9e0155130 0x7fa9e01555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:53.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.186+0000 7fa9e63e7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9e0155130 0x7fa9e01555a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57250/0 (socket says 172.21.15.78:57250) 2023-10-11T09:40:53.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.186+0000 7fa9e63e7700 1 -- 172.21.15.78:0/4194240640 learned_addr learned my addr 172.21.15.78:0/4194240640 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:53.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.186+0000 7fa9e5be6700 1 --2- 172.21.15.78:0/4194240640 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa9cc005220 0x7fa9e014e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:53.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.186+0000 7fa9e53e5700 1 --2- 172.21.15.78:0/4194240640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9e00aeee0 0x7fa9e00af330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:53.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.186+0000 7fa9e63e7700 1 -- 172.21.15.78:0/4194240640 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa9cc005220 msgr2=0x7fa9e014e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:53.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.186+0000 7fa9e63e7700 1 --2- 172.21.15.78:0/4194240640 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa9cc005220 0x7fa9e014e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:53.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.186+0000 7fa9e63e7700 1 -- 172.21.15.78:0/4194240640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9e00aeee0 msgr2=0x7fa9e00af330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:53.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.186+0000 7fa9e63e7700 1 --2- 172.21.15.78:0/4194240640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9e00aeee0 0x7fa9e00af330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:53.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.186+0000 7fa9e63e7700 1 -- 172.21.15.78:0/4194240640 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e014eee0 con 0x7fa9e0155130 2023-10-11T09:40:53.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.186+0000 7fa9e63e7700 1 --2- 172.21.15.78:0/4194240640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9e0155130 0x7fa9e01555a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa9d800ef30 tx=0x7fa9d8004b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:53.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.186+0000 7fa9d67fc700 1 -- 172.21.15.78:0/4194240640 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8004230 con 0x7fa9e0155130 2023-10-11T09:40:53.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.186+0000 7fa9d67fc700 1 -- 172.21.15.78:0/4194240640 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d80043b0 con 0x7fa9e0155130 2023-10-11T09:40:53.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.186+0000 7fa9e6be8700 1 -- 172.21.15.78:0/4194240640 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9e00afaa0 con 0x7fa9e0155130 2023-10-11T09:40:53.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.186+0000 7fa9d67fc700 1 -- 172.21.15.78:0/4194240640 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8004760 con 0x7fa9e0155130 2023-10-11T09:40:53.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.186+0000 7fa9e6be8700 1 -- 172.21.15.78:0/4194240640 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9e00b0060 con 0x7fa9e0155130 2023-10-11T09:40:53.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.190+0000 7fa9e6be8700 1 -- 172.21.15.78:0/4194240640 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9e0065d40 con 0x7fa9e0155130 2023-10-11T09:40:53.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.190+0000 7fa9d67fc700 1 -- 172.21.15.78:0/4194240640 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fa9d8014480 con 0x7fa9e0155130 2023-10-11T09:40:53.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.190+0000 7fa9d67fc700 1 --2- 172.21.15.78:0/4194240640 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa9c4065ed0 0x7fa9c4068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:53.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.190+0000 7fa9d67fc700 1 -- 172.21.15.78:0/4194240640 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa9d8084fd0 con 0x7fa9e0155130 2023-10-11T09:40:53.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.190+0000 7fa9e5be6700 1 --2- 172.21.15.78:0/4194240640 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa9c4065ed0 0x7fa9c4068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:53.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.194+0000 7fa9e5be6700 1 --2- 172.21.15.78:0/4194240640 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa9c4065ed0 0x7fa9c4068380 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc001050 tx=0x7fa9dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:53.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.198+0000 7fa9d67fc700 1 -- 172.21.15.78:0/4194240640 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fa9d8052610 con 0x7fa9e0155130 2023-10-11T09:40:53.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.354+0000 7fa9e6be8700 1 -- 172.21.15.78:0/4194240640 --> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa9e0160740 con 0x7fa9c4065ed0 2023-10-11T09:40:53.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.362+0000 7fa9d67fc700 1 -- 172.21.15.78:0/4194240640 <== mgr.4101 v2:172.21.15.78:6832/35606 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286426 (secure 0 0 0) 0x7fa9dc005e00 con 0x7fa9c4065ed0 2023-10-11T09:40:53.371 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:40:53.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.366+0000 7fa9e6be8700 1 -- 172.21.15.78:0/4194240640 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa9c4065ed0 msgr2=0x7fa9c4068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:53.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.366+0000 7fa9e6be8700 1 --2- 172.21.15.78:0/4194240640 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa9c4065ed0 0x7fa9c4068380 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc001050 tx=0x7fa9dc009040 comp rx=0 tx=0).stop 2023-10-11T09:40:53.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.366+0000 7fa9e6be8700 1 -- 172.21.15.78:0/4194240640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9e0155130 msgr2=0x7fa9e01555a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:53.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.366+0000 7fa9e6be8700 1 --2- 172.21.15.78:0/4194240640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9e0155130 0x7fa9e01555a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa9d800ef30 tx=0x7fa9d8004b60 comp rx=0 tx=0).stop 2023-10-11T09:40:53.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.370+0000 7fa9e6be8700 1 -- 172.21.15.78:0/4194240640 shutdown_connections 2023-10-11T09:40:53.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.370+0000 7fa9e6be8700 1 --2- 172.21.15.78:0/4194240640 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa9c4065ed0 0x7fa9c4068380 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:53.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.370+0000 7fa9e6be8700 1 --2- 172.21.15.78:0/4194240640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9e0155130 0x7fa9e01555a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:53.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.370+0000 7fa9e6be8700 1 --2- 172.21.15.78:0/4194240640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9e00aeee0 0x7fa9e00af330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:53.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.370+0000 7fa9e6be8700 1 --2- 172.21.15.78:0/4194240640 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa9cc005220 0x7fa9e014e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:53.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.370+0000 7fa9e6be8700 1 -- 172.21.15.78:0/4194240640 >> 172.21.15.78:0/4194240640 conn(0x7fa9e00cea60 msgr2=0x7fa9e015f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:53.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.370+0000 7fa9e6be8700 1 -- 172.21.15.78:0/4194240640 shutdown_connections 2023-10-11T09:40:53.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.370+0000 7fa9e6be8700 1 -- 172.21.15.78:0/4194240640 wait complete. 2023-10-11T09:40:53.376 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2023-10-11T09:40:53.394 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-10-11T09:40:52.325814+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218728,"kb_used_data":3112,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755992,"statfs":{"total":773094113280,"available":772870135808,"internally_reserved":0,"allocated":3186688,"data_stored":1902252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003045"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144669+0000","last_change":"2023-10-11T09:40:41.696400+0000","last_active":"2023-10-11T09:40:43.144669+0000","last_peered":"2023-10-11T09:40:43.144669+0000","last_clean":"2023-10-11T09:40:43.144669+0000","last_became_active":"2023-10-11T09:40:41.696101+0000","last_became_peered":"2023-10-11T09:40:41.696101+0000","last_unstale":"2023-10-11T09:40:43.144669+0000","last_undegraded":"2023-10-11T09:40:43.144669+0000","last_fullsized":"2023-10-11T09:40:43.144669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:43:01.331013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.694262+0000","last_change":"2023-10-11T09:40:41.719611+0000","last_active":"2023-10-11T09:40:42.694262+0000","last_peered":"2023-10-11T09:40:42.694262+0000","last_clean":"2023-10-11T09:40:42.694262+0000","last_became_active":"2023-10-11T09:40:41.719319+0000","last_became_peered":"2023-10-11T09:40:41.719319+0000","last_unstale":"2023-10-11T09:40:42.694262+0000","last_undegraded":"2023-10-11T09:40:42.694262+0000","last_fullsized":"2023-10-11T09:40:42.694262+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:07:02.302344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.912066+0000","last_change":"2023-10-11T09:40:41.701063+0000","last_active":"2023-10-11T09:40:42.912066+0000","last_peered":"2023-10-11T09:40:42.912066+0000","last_clean":"2023-10-11T09:40:42.912066+0000","last_became_active":"2023-10-11T09:40:41.699445+0000","last_became_peered":"2023-10-11T09:40:41.699445+0000","last_unstale":"2023-10-11T09:40:42.912066+0000","last_undegraded":"2023-10-11T09:40:42.912066+0000","last_fullsized":"2023-10-11T09:40:42.912066+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:19:44.476799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144161+0000","last_change":"2023-10-11T09:40:41.721932+0000","last_active":"2023-10-11T09:40:43.144161+0000","last_peered":"2023-10-11T09:40:43.144161+0000","last_clean":"2023-10-11T09:40:43.144161+0000","last_became_active":"2023-10-11T09:40:41.720602+0000","last_became_peered":"2023-10-11T09:40:41.720602+0000","last_unstale":"2023-10-11T09:40:43.144161+0000","last_undegraded":"2023-10-11T09:40:43.144161+0000","last_fullsized":"2023-10-11T09:40:43.144161+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:17:36.655491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145802+0000","last_change":"2023-10-11T09:40:41.710213+0000","last_active":"2023-10-11T09:40:43.145802+0000","last_peered":"2023-10-11T09:40:43.145802+0000","last_clean":"2023-10-11T09:40:43.145802+0000","last_became_active":"2023-10-11T09:40:41.707488+0000","last_became_peered":"2023-10-11T09:40:41.707488+0000","last_unstale":"2023-10-11T09:40:43.145802+0000","last_undegraded":"2023-10-11T09:40:43.145802+0000","last_fullsized":"2023-10-11T09:40:43.145802+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:42:34.821513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911583+0000","last_change":"2023-10-11T09:40:41.700595+0000","last_active":"2023-10-11T09:40:42.911583+0000","last_peered":"2023-10-11T09:40:42.911583+0000","last_clean":"2023-10-11T09:40:42.911583+0000","last_became_active":"2023-10-11T09:40:41.700239+0000","last_became_peered":"2023-10-11T09:40:41.700239+0000","last_unstale":"2023-10-11T09:40:42.911583+0000","last_undegraded":"2023-10-11T09:40:42.911583+0000","last_fullsized":"2023-10-11T09:40:42.911583+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:48:14.871150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.691894+0000","last_change":"2023-10-11T09:40:41.712870+0000","last_active":"2023-10-11T09:40:42.691894+0000","last_peered":"2023-10-11T09:40:42.691894+0000","last_clean":"2023-10-11T09:40:42.691894+0000","last_became_active":"2023-10-11T09:40:41.712593+0000","last_became_peered":"2023-10-11T09:40:41.712593+0000","last_unstale":"2023-10-11T09:40:42.691894+0000","last_undegraded":"2023-10-11T09:40:42.691894+0000","last_fullsized":"2023-10-11T09:40:42.691894+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:50:15.829941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.693734+0000","last_change":"2023-10-11T09:40:41.714890+0000","last_active":"2023-10-11T09:40:42.693734+0000","last_peered":"2023-10-11T09:40:42.693734+0000","last_clean":"2023-10-11T09:40:42.693734+0000","last_became_active":"2023-10-11T09:40:41.711116+0000","last_became_peered":"2023-10-11T09:40:41.711116+0000","last_unstale":"2023-10-11T09:40:42.693734+0000","last_undegraded":"2023-10-11T09:40:42.693734+0000","last_fullsized":"2023-10-11T09:40:42.693734+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:00:05.422788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.146063+0000","last_change":"2023-10-11T09:40:41.700768+0000","last_active":"2023-10-11T09:40:43.146063+0000","last_peered":"2023-10-11T09:40:43.146063+0000","last_clean":"2023-10-11T09:40:43.146063+0000","last_became_active":"2023-10-11T09:40:41.700440+0000","last_became_peered":"2023-10-11T09:40:41.700440+0000","last_unstale":"2023-10-11T09:40:43.146063+0000","last_undegraded":"2023-10-11T09:40:43.146063+0000","last_fullsized":"2023-10-11T09:40:43.146063+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:56:50.343276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.694372+0000","last_change":"2023-10-11T09:40:41.712701+0000","last_active":"2023-10-11T09:40:42.694372+0000","last_peered":"2023-10-11T09:40:42.694372+0000","last_clean":"2023-10-11T09:40:42.694372+0000","last_became_active":"2023-10-11T09:40:41.712349+0000","last_became_peered":"2023-10-11T09:40:41.712349+0000","last_unstale":"2023-10-11T09:40:42.694372+0000","last_undegraded":"2023-10-11T09:40:42.694372+0000","last_fullsized":"2023-10-11T09:40:42.694372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:42:34.821513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.912276+0000","last_change":"2023-10-11T09:40:41.706310+0000","last_active":"2023-10-11T09:40:42.912276+0000","last_peered":"2023-10-11T09:40:42.912276+0000","last_clean":"2023-10-11T09:40:42.912276+0000","last_became_active":"2023-10-11T09:40:41.705986+0000","last_became_peered":"2023-10-11T09:40:41.705986+0000","last_unstale":"2023-10-11T09:40:42.912276+0000","last_undegraded":"2023-10-11T09:40:42.912276+0000","last_fullsized":"2023-10-11T09:40:42.912276+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:59:46.864358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144038+0000","last_change":"2023-10-11T09:40:41.700251+0000","last_active":"2023-10-11T09:40:43.144038+0000","last_peered":"2023-10-11T09:40:43.144038+0000","last_clean":"2023-10-11T09:40:43.144038+0000","last_became_active":"2023-10-11T09:40:41.699961+0000","last_became_peered":"2023-10-11T09:40:41.699961+0000","last_unstale":"2023-10-11T09:40:43.144038+0000","last_undegraded":"2023-10-11T09:40:43.144038+0000","last_fullsized":"2023-10-11T09:40:43.144038+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:12:05.737223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.143594+0000","last_change":"2023-10-11T09:40:41.697448+0000","last_active":"2023-10-11T09:40:43.143594+0000","last_peered":"2023-10-11T09:40:43.143594+0000","last_clean":"2023-10-11T09:40:43.143594+0000","last_became_active":"2023-10-11T09:40:41.697160+0000","last_became_peered":"2023-10-11T09:40:41.697160+0000","last_unstale":"2023-10-11T09:40:43.143594+0000","last_undegraded":"2023-10-11T09:40:43.143594+0000","last_fullsized":"2023-10-11T09:40:43.143594+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:00:57.063025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.693705+0000","last_change":"2023-10-11T09:40:41.717312+0000","last_active":"2023-10-11T09:40:42.693705+0000","last_peered":"2023-10-11T09:40:42.693705+0000","last_clean":"2023-10-11T09:40:42.693705+0000","last_became_active":"2023-10-11T09:40:41.716622+0000","last_became_peered":"2023-10-11T09:40:41.716622+0000","last_unstale":"2023-10-11T09:40:42.693705+0000","last_undegraded":"2023-10-11T09:40:42.693705+0000","last_fullsized":"2023-10-11T09:40:42.693705+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:35:04.646798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.692225+0000","last_change":"2023-10-11T09:40:41.715325+0000","last_active":"2023-10-11T09:40:42.692225+0000","last_peered":"2023-10-11T09:40:42.692225+0000","last_clean":"2023-10-11T09:40:42.692225+0000","last_became_active":"2023-10-11T09:40:41.715033+0000","last_became_peered":"2023-10-11T09:40:41.715033+0000","last_unstale":"2023-10-11T09:40:42.692225+0000","last_undegraded":"2023-10-11T09:40:42.692225+0000","last_fullsized":"2023-10-11T09:40:42.692225+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:35:36.436236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911235+0000","last_change":"2023-10-11T09:40:41.699851+0000","last_active":"2023-10-11T09:40:42.911235+0000","last_peered":"2023-10-11T09:40:42.911235+0000","last_clean":"2023-10-11T09:40:42.911235+0000","last_became_active":"2023-10-11T09:40:41.699530+0000","last_became_peered":"2023-10-11T09:40:41.699530+0000","last_unstale":"2023-10-11T09:40:42.911235+0000","last_undegraded":"2023-10-11T09:40:42.911235+0000","last_fullsized":"2023-10-11T09:40:42.911235+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:23:30.814623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.146112+0000","last_change":"2023-10-11T09:40:41.700611+0000","last_active":"2023-10-11T09:40:43.146112+0000","last_peered":"2023-10-11T09:40:43.146112+0000","last_clean":"2023-10-11T09:40:43.146112+0000","last_became_active":"2023-10-11T09:40:41.700354+0000","last_became_peered":"2023-10-11T09:40:41.700354+0000","last_unstale":"2023-10-11T09:40:43.146112+0000","last_undegraded":"2023-10-11T09:40:43.146112+0000","last_fullsized":"2023-10-11T09:40:43.146112+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:48:15.689750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.912114+0000","last_change":"2023-10-11T09:40:41.700742+0000","last_active":"2023-10-11T09:40:42.912114+0000","last_peered":"2023-10-11T09:40:42.912114+0000","last_clean":"2023-10-11T09:40:42.912114+0000","last_became_active":"2023-10-11T09:40:41.700432+0000","last_became_peered":"2023-10-11T09:40:41.700432+0000","last_unstale":"2023-10-11T09:40:42.912114+0000","last_undegraded":"2023-10-11T09:40:42.912114+0000","last_fullsized":"2023-10-11T09:40:42.912114+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:33:13.162340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.143660+0000","last_change":"2023-10-11T09:40:41.719737+0000","last_active":"2023-10-11T09:40:43.143660+0000","last_peered":"2023-10-11T09:40:43.143660+0000","last_clean":"2023-10-11T09:40:43.143660+0000","last_became_active":"2023-10-11T09:40:41.719484+0000","last_became_peered":"2023-10-11T09:40:41.719484+0000","last_unstale":"2023-10-11T09:40:43.143660+0000","last_undegraded":"2023-10-11T09:40:43.143660+0000","last_fullsized":"2023-10-11T09:40:43.143660+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:48:52.897052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.912175+0000","last_change":"2023-10-11T09:40:41.728139+0000","last_active":"2023-10-11T09:40:42.912175+0000","last_peered":"2023-10-11T09:40:42.912175+0000","last_clean":"2023-10-11T09:40:42.912175+0000","last_became_active":"2023-10-11T09:40:41.727790+0000","last_became_peered":"2023-10-11T09:40:41.727790+0000","last_unstale":"2023-10-11T09:40:42.912175+0000","last_undegraded":"2023-10-11T09:40:42.912175+0000","last_fullsized":"2023-10-11T09:40:42.912175+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:07:58.571258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.692070+0000","last_change":"2023-10-11T09:40:41.713375+0000","last_active":"2023-10-11T09:40:42.692070+0000","last_peered":"2023-10-11T09:40:42.692070+0000","last_clean":"2023-10-11T09:40:42.692070+0000","last_became_active":"2023-10-11T09:40:41.713114+0000","last_became_peered":"2023-10-11T09:40:41.713114+0000","last_unstale":"2023-10-11T09:40:42.692070+0000","last_undegraded":"2023-10-11T09:40:42.692070+0000","last_fullsized":"2023-10-11T09:40:42.692070+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:07:02.302344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145102+0000","last_change":"2023-10-11T09:40:41.718540+0000","last_active":"2023-10-11T09:40:43.145102+0000","last_peered":"2023-10-11T09:40:43.145102+0000","last_clean":"2023-10-11T09:40:43.145102+0000","last_became_active":"2023-10-11T09:40:41.718309+0000","last_became_peered":"2023-10-11T09:40:41.718309+0000","last_unstale":"2023-10-11T09:40:43.145102+0000","last_undegraded":"2023-10-11T09:40:43.145102+0000","last_fullsized":"2023-10-11T09:40:43.145102+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:09:27.825336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687447+0000","last_change":"2023-10-11T09:40:41.699075+0000","last_active":"2023-10-11T09:40:42.687447+0000","last_peered":"2023-10-11T09:40:42.687447+0000","last_clean":"2023-10-11T09:40:42.687447+0000","last_became_active":"2023-10-11T09:40:41.698747+0000","last_became_peered":"2023-10-11T09:40:41.698747+0000","last_unstale":"2023-10-11T09:40:42.687447+0000","last_undegraded":"2023-10-11T09:40:42.687447+0000","last_fullsized":"2023-10-11T09:40:42.687447+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:09:27.825336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.691897+0000","last_change":"2023-10-11T09:40:41.714225+0000","last_active":"2023-10-11T09:40:42.691897+0000","last_peered":"2023-10-11T09:40:42.691897+0000","last_clean":"2023-10-11T09:40:42.691897+0000","last_became_active":"2023-10-11T09:40:41.710262+0000","last_became_peered":"2023-10-11T09:40:41.710262+0000","last_unstale":"2023-10-11T09:40:42.691897+0000","last_undegraded":"2023-10-11T09:40:42.691897+0000","last_fullsized":"2023-10-11T09:40:42.691897+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:42:34.821513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144675+0000","last_change":"2023-10-11T09:40:41.729722+0000","last_active":"2023-10-11T09:40:43.144675+0000","last_peered":"2023-10-11T09:40:43.144675+0000","last_clean":"2023-10-11T09:40:43.144675+0000","last_became_active":"2023-10-11T09:40:41.729360+0000","last_became_peered":"2023-10-11T09:40:41.729360+0000","last_unstale":"2023-10-11T09:40:43.144675+0000","last_undegraded":"2023-10-11T09:40:43.144675+0000","last_fullsized":"2023-10-11T09:40:43.144675+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:07:02.302344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145219+0000","last_change":"2023-10-11T09:40:41.709354+0000","last_active":"2023-10-11T09:40:43.145219+0000","last_peered":"2023-10-11T09:40:43.145219+0000","last_clean":"2023-10-11T09:40:43.145219+0000","last_became_active":"2023-10-11T09:40:41.709096+0000","last_became_peered":"2023-10-11T09:40:41.709096+0000","last_unstale":"2023-10-11T09:40:43.145219+0000","last_undegraded":"2023-10-11T09:40:43.145219+0000","last_fullsized":"2023-10-11T09:40:43.145219+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:35:04.646798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.694072+0000","last_change":"2023-10-11T09:40:41.723510+0000","last_active":"2023-10-11T09:40:42.694072+0000","last_peered":"2023-10-11T09:40:42.694072+0000","last_clean":"2023-10-11T09:40:42.694072+0000","last_became_active":"2023-10-11T09:40:41.723110+0000","last_became_peered":"2023-10-11T09:40:41.723110+0000","last_unstale":"2023-10-11T09:40:42.694072+0000","last_undegraded":"2023-10-11T09:40:42.694072+0000","last_fullsized":"2023-10-11T09:40:42.694072+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:50:15.829941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145120+0000","last_change":"2023-10-11T09:40:41.714481+0000","last_active":"2023-10-11T09:40:43.145120+0000","last_peered":"2023-10-11T09:40:43.145120+0000","last_clean":"2023-10-11T09:40:43.145120+0000","last_became_active":"2023-10-11T09:40:41.714219+0000","last_became_peered":"2023-10-11T09:40:41.714219+0000","last_unstale":"2023-10-11T09:40:43.145120+0000","last_undegraded":"2023-10-11T09:40:43.145120+0000","last_fullsized":"2023-10-11T09:40:43.145120+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:19:07.089979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.688002+0000","last_change":"2023-10-11T09:40:41.709612+0000","last_active":"2023-10-11T09:40:42.688002+0000","last_peered":"2023-10-11T09:40:42.688002+0000","last_clean":"2023-10-11T09:40:42.688002+0000","last_became_active":"2023-10-11T09:40:41.709317+0000","last_became_peered":"2023-10-11T09:40:41.709317+0000","last_unstale":"2023-10-11T09:40:42.688002+0000","last_undegraded":"2023-10-11T09:40:42.688002+0000","last_fullsized":"2023-10-11T09:40:42.688002+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:35:36.436236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911908+0000","last_change":"2023-10-11T09:40:41.727354+0000","last_active":"2023-10-11T09:40:42.911908+0000","last_peered":"2023-10-11T09:40:42.911908+0000","last_clean":"2023-10-11T09:40:42.911908+0000","last_became_active":"2023-10-11T09:40:41.727039+0000","last_became_peered":"2023-10-11T09:40:41.727039+0000","last_unstale":"2023-10-11T09:40:42.911908+0000","last_undegraded":"2023-10-11T09:40:42.911908+0000","last_fullsized":"2023-10-11T09:40:42.911908+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:01:36.923947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687557+0000","last_change":"2023-10-11T09:40:41.712971+0000","last_active":"2023-10-11T09:40:42.687557+0000","last_peered":"2023-10-11T09:40:42.687557+0000","last_clean":"2023-10-11T09:40:42.687557+0000","last_became_active":"2023-10-11T09:40:41.712707+0000","last_became_peered":"2023-10-11T09:40:41.712707+0000","last_unstale":"2023-10-11T09:40:42.687557+0000","last_undegraded":"2023-10-11T09:40:42.687557+0000","last_fullsized":"2023-10-11T09:40:42.687557+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:48:15.689750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.143927+0000","last_change":"2023-10-11T09:40:41.728941+0000","last_active":"2023-10-11T09:40:43.143927+0000","last_peered":"2023-10-11T09:40:43.143927+0000","last_clean":"2023-10-11T09:40:43.143927+0000","last_became_active":"2023-10-11T09:40:41.728657+0000","last_became_peered":"2023-10-11T09:40:41.728657+0000","last_unstale":"2023-10-11T09:40:43.143927+0000","last_undegraded":"2023-10-11T09:40:43.143927+0000","last_fullsized":"2023-10-11T09:40:43.143927+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:12:05.737223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.688259+0000","last_change":"2023-10-11T09:40:41.718662+0000","last_active":"2023-10-11T09:40:42.688259+0000","last_peered":"2023-10-11T09:40:42.688259+0000","last_clean":"2023-10-11T09:40:42.688259+0000","last_became_active":"2023-10-11T09:40:41.718402+0000","last_became_peered":"2023-10-11T09:40:41.718402+0000","last_unstale":"2023-10-11T09:40:42.688259+0000","last_undegraded":"2023-10-11T09:40:42.688259+0000","last_fullsized":"2023-10-11T09:40:42.688259+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:56:50.343276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.692319+0000","last_change":"2023-10-11T09:40:41.234649+0000","last_active":"2023-10-11T09:40:42.692319+0000","last_peered":"2023-10-11T09:40:42.692319+0000","last_clean":"2023-10-11T09:40:42.692319+0000","last_became_active":"2023-10-11T09:40:41.234006+0000","last_became_peered":"2023-10-11T09:40:41.234006+0000","last_unstale":"2023-10-11T09:40:42.692319+0000","last_undegraded":"2023-10-11T09:40:42.692319+0000","last_fullsized":"2023-10-11T09:40:42.692319+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:45:39.083797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.694114+0000","last_change":"2023-10-11T09:40:41.723234+0000","last_active":"2023-10-11T09:40:42.694114+0000","last_peered":"2023-10-11T09:40:42.694114+0000","last_clean":"2023-10-11T09:40:42.694114+0000","last_became_active":"2023-10-11T09:40:41.722928+0000","last_became_peered":"2023-10-11T09:40:41.722928+0000","last_unstale":"2023-10-11T09:40:42.694114+0000","last_undegraded":"2023-10-11T09:40:42.694114+0000","last_fullsized":"2023-10-11T09:40:42.694114+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:09:27.825336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145323+0000","last_change":"2023-10-11T09:40:41.225247+0000","last_active":"2023-10-11T09:40:43.145323+0000","last_peered":"2023-10-11T09:40:43.145323+0000","last_clean":"2023-10-11T09:40:43.145323+0000","last_became_active":"2023-10-11T09:40:41.224967+0000","last_became_peered":"2023-10-11T09:40:41.224967+0000","last_unstale":"2023-10-11T09:40:43.145323+0000","last_undegraded":"2023-10-11T09:40:43.145323+0000","last_fullsized":"2023-10-11T09:40:43.145323+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:33:10.649755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145747+0000","last_change":"2023-10-11T09:40:41.703018+0000","last_active":"2023-10-11T09:40:43.145747+0000","last_peered":"2023-10-11T09:40:43.145747+0000","last_clean":"2023-10-11T09:40:43.145747+0000","last_became_active":"2023-10-11T09:40:41.702729+0000","last_became_peered":"2023-10-11T09:40:41.702729+0000","last_unstale":"2023-10-11T09:40:43.145747+0000","last_undegraded":"2023-10-11T09:40:43.145747+0000","last_fullsized":"2023-10-11T09:40:43.145747+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:07:02.302344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.692298+0000","last_change":"2023-10-11T09:40:41.234929+0000","last_active":"2023-10-11T09:40:42.692298+0000","last_peered":"2023-10-11T09:40:42.692298+0000","last_clean":"2023-10-11T09:40:42.692298+0000","last_became_active":"2023-10-11T09:40:41.234149+0000","last_became_peered":"2023-10-11T09:40:41.234149+0000","last_unstale":"2023-10-11T09:40:42.692298+0000","last_undegraded":"2023-10-11T09:40:42.692298+0000","last_fullsized":"2023-10-11T09:40:42.692298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:17:35.640470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.688569+0000","last_change":"2023-10-11T09:40:41.716719+0000","last_active":"2023-10-11T09:40:42.688569+0000","last_peered":"2023-10-11T09:40:42.688569+0000","last_clean":"2023-10-11T09:40:42.688569+0000","last_became_active":"2023-10-11T09:40:41.716181+0000","last_became_peered":"2023-10-11T09:40:41.716181+0000","last_unstale":"2023-10-11T09:40:42.688569+0000","last_undegraded":"2023-10-11T09:40:42.688569+0000","last_fullsized":"2023-10-11T09:40:42.688569+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:50:15.829941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.143863+0000","last_change":"2023-10-11T09:40:41.229549+0000","last_active":"2023-10-11T09:40:43.143863+0000","last_peered":"2023-10-11T09:40:43.143863+0000","last_clean":"2023-10-11T09:40:43.143863+0000","last_became_active":"2023-10-11T09:40:41.229259+0000","last_became_peered":"2023-10-11T09:40:41.229259+0000","last_unstale":"2023-10-11T09:40:43.143863+0000","last_undegraded":"2023-10-11T09:40:43.143863+0000","last_fullsized":"2023-10-11T09:40:43.143863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:16:37.265056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.692895+0000","last_change":"2023-10-11T09:40:41.234477+0000","last_active":"2023-10-11T09:40:42.692895+0000","last_peered":"2023-10-11T09:40:42.692895+0000","last_clean":"2023-10-11T09:40:42.692895+0000","last_became_active":"2023-10-11T09:40:41.233977+0000","last_became_peered":"2023-10-11T09:40:41.233977+0000","last_unstale":"2023-10-11T09:40:42.692895+0000","last_undegraded":"2023-10-11T09:40:42.692895+0000","last_fullsized":"2023-10-11T09:40:42.692895+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:47:25.022675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.693696+0000","last_change":"2023-10-11T09:40:41.717799+0000","last_active":"2023-10-11T09:40:42.693696+0000","last_peered":"2023-10-11T09:40:42.693696+0000","last_clean":"2023-10-11T09:40:42.693696+0000","last_became_active":"2023-10-11T09:40:41.717520+0000","last_became_peered":"2023-10-11T09:40:41.717520+0000","last_unstale":"2023-10-11T09:40:42.693696+0000","last_undegraded":"2023-10-11T09:40:42.693696+0000","last_fullsized":"2023-10-11T09:40:42.693696+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:35:36.436236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.692703+0000","last_change":"2023-10-11T09:40:41.234979+0000","last_active":"2023-10-11T09:40:42.692703+0000","last_peered":"2023-10-11T09:40:42.692703+0000","last_clean":"2023-10-11T09:40:42.692703+0000","last_became_active":"2023-10-11T09:40:41.234245+0000","last_became_peered":"2023-10-11T09:40:41.234245+0000","last_unstale":"2023-10-11T09:40:42.692703+0000","last_undegraded":"2023-10-11T09:40:42.692703+0000","last_fullsized":"2023-10-11T09:40:42.692703+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:31:56.724263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911330+0000","last_change":"2023-10-11T09:40:41.706936+0000","last_active":"2023-10-11T09:40:42.911330+0000","last_peered":"2023-10-11T09:40:42.911330+0000","last_clean":"2023-10-11T09:40:42.911330+0000","last_became_active":"2023-10-11T09:40:41.706440+0000","last_became_peered":"2023-10-11T09:40:41.706440+0000","last_unstale":"2023-10-11T09:40:42.911330+0000","last_undegraded":"2023-10-11T09:40:42.911330+0000","last_fullsized":"2023-10-11T09:40:42.911330+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:12:14.951424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.692744+0000","last_change":"2023-10-11T09:40:41.228740+0000","last_active":"2023-10-11T09:40:42.692744+0000","last_peered":"2023-10-11T09:40:42.692744+0000","last_clean":"2023-10-11T09:40:42.692744+0000","last_became_active":"2023-10-11T09:40:41.228327+0000","last_became_peered":"2023-10-11T09:40:41.228327+0000","last_unstale":"2023-10-11T09:40:42.692744+0000","last_undegraded":"2023-10-11T09:40:42.692744+0000","last_fullsized":"2023-10-11T09:40:42.692744+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:59:14.883485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144214+0000","last_change":"2023-10-11T09:40:41.721661+0000","last_active":"2023-10-11T09:40:43.144214+0000","last_peered":"2023-10-11T09:40:43.144214+0000","last_clean":"2023-10-11T09:40:43.144214+0000","last_became_active":"2023-10-11T09:40:41.720154+0000","last_became_peered":"2023-10-11T09:40:41.720154+0000","last_unstale":"2023-10-11T09:40:43.144214+0000","last_undegraded":"2023-10-11T09:40:43.144214+0000","last_fullsized":"2023-10-11T09:40:43.144214+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:45:40.098819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.693986+0000","last_change":"2023-10-11T09:40:41.225444+0000","last_active":"2023-10-11T09:40:42.693986+0000","last_peered":"2023-10-11T09:40:42.693986+0000","last_clean":"2023-10-11T09:40:42.693986+0000","last_became_active":"2023-10-11T09:40:41.225159+0000","last_became_peered":"2023-10-11T09:40:41.225159+0000","last_unstale":"2023-10-11T09:40:42.693986+0000","last_undegraded":"2023-10-11T09:40:42.693986+0000","last_fullsized":"2023-10-11T09:40:42.693986+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:00:56.048004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145496+0000","last_change":"2023-10-11T09:40:41.712920+0000","last_active":"2023-10-11T09:40:43.145496+0000","last_peered":"2023-10-11T09:40:43.145496+0000","last_clean":"2023-10-11T09:40:43.145496+0000","last_became_active":"2023-10-11T09:40:41.712654+0000","last_became_peered":"2023-10-11T09:40:41.712654+0000","last_unstale":"2023-10-11T09:40:43.145496+0000","last_undegraded":"2023-10-11T09:40:43.145496+0000","last_fullsized":"2023-10-11T09:40:43.145496+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:23:30.814623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.688154+0000","last_change":"2023-10-11T09:40:41.229300+0000","last_active":"2023-10-11T09:40:42.688154+0000","last_peered":"2023-10-11T09:40:42.688154+0000","last_clean":"2023-10-11T09:40:42.688154+0000","last_became_active":"2023-10-11T09:40:41.228539+0000","last_became_peered":"2023-10-11T09:40:41.228539+0000","last_unstale":"2023-10-11T09:40:42.688154+0000","last_undegraded":"2023-10-11T09:40:42.688154+0000","last_fullsized":"2023-10-11T09:40:42.688154+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:09:03.936717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.912449+0000","last_change":"2023-10-11T09:40:41.705540+0000","last_active":"2023-10-11T09:40:42.912449+0000","last_peered":"2023-10-11T09:40:42.912449+0000","last_clean":"2023-10-11T09:40:42.912449+0000","last_became_active":"2023-10-11T09:40:41.705163+0000","last_became_peered":"2023-10-11T09:40:41.705163+0000","last_unstale":"2023-10-11T09:40:42.912449+0000","last_undegraded":"2023-10-11T09:40:42.912449+0000","last_fullsized":"2023-10-11T09:40:42.912449+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:56:50.343276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144408+0000","last_change":"2023-10-11T09:40:41.224108+0000","last_active":"2023-10-11T09:40:43.144408+0000","last_peered":"2023-10-11T09:40:43.144408+0000","last_clean":"2023-10-11T09:40:43.144408+0000","last_became_active":"2023-10-11T09:40:41.223643+0000","last_became_peered":"2023-10-11T09:40:41.223643+0000","last_unstale":"2023-10-11T09:40:43.144408+0000","last_undegraded":"2023-10-11T09:40:43.144408+0000","last_fullsized":"2023-10-11T09:40:43.144408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:36:34.713602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.912393+0000","last_change":"2023-10-11T09:40:41.700784+0000","last_active":"2023-10-11T09:40:42.912393+0000","last_peered":"2023-10-11T09:40:42.912393+0000","last_clean":"2023-10-11T09:40:42.912393+0000","last_became_active":"2023-10-11T09:40:41.700495+0000","last_became_peered":"2023-10-11T09:40:41.700495+0000","last_unstale":"2023-10-11T09:40:42.912393+0000","last_undegraded":"2023-10-11T09:40:42.912393+0000","last_fullsized":"2023-10-11T09:40:42.912393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:57:16.072435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911546+0000","last_change":"2023-10-11T09:40:41.233232+0000","last_active":"2023-10-11T09:40:42.911546+0000","last_peered":"2023-10-11T09:40:42.911546+0000","last_clean":"2023-10-11T09:40:42.911546+0000","last_became_active":"2023-10-11T09:40:41.232092+0000","last_became_peered":"2023-10-11T09:40:41.232092+0000","last_unstale":"2023-10-11T09:40:42.911546+0000","last_undegraded":"2023-10-11T09:40:42.911546+0000","last_fullsized":"2023-10-11T09:40:42.911546+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:16:37.265056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.912288+0000","last_change":"2023-10-11T09:40:41.721557+0000","last_active":"2023-10-11T09:40:42.912288+0000","last_peered":"2023-10-11T09:40:42.912288+0000","last_clean":"2023-10-11T09:40:42.912288+0000","last_became_active":"2023-10-11T09:40:41.721314+0000","last_became_peered":"2023-10-11T09:40:41.721314+0000","last_unstale":"2023-10-11T09:40:42.912288+0000","last_undegraded":"2023-10-11T09:40:42.912288+0000","last_fullsized":"2023-10-11T09:40:42.912288+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:59:15.898506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687786+0000","last_change":"2023-10-11T09:40:41.228065+0000","last_active":"2023-10-11T09:40:42.687786+0000","last_peered":"2023-10-11T09:40:42.687786+0000","last_clean":"2023-10-11T09:40:42.687786+0000","last_became_active":"2023-10-11T09:40:41.227765+0000","last_became_peered":"2023-10-11T09:40:41.227765+0000","last_unstale":"2023-10-11T09:40:42.687786+0000","last_undegraded":"2023-10-11T09:40:42.687786+0000","last_fullsized":"2023-10-11T09:40:42.687786+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:36:34.713602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144923+0000","last_change":"2023-10-11T09:40:41.708864+0000","last_active":"2023-10-11T09:40:43.144923+0000","last_peered":"2023-10-11T09:40:43.144923+0000","last_clean":"2023-10-11T09:40:43.144923+0000","last_became_active":"2023-10-11T09:40:41.708555+0000","last_became_peered":"2023-10-11T09:40:41.708555+0000","last_unstale":"2023-10-11T09:40:43.144923+0000","last_undegraded":"2023-10-11T09:40:43.144923+0000","last_fullsized":"2023-10-11T09:40:43.144923+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:43:30.691251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":167,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-11T09:40:41.695042+0000","last_change":"2023-10-11T09:40:29.874372+0000","last_active":"2023-10-11T09:40:41.695042+0000","last_peered":"2023-10-11T09:40:41.695042+0000","last_clean":"2023-10-11T09:40:41.695042+0000","last_became_active":"2023-10-11T09:40:29.873621+0000","last_became_peered":"2023-10-11T09:40:29.873621+0000","last_unstale":"2023-10-11T09:40:41.695042+0000","last_undegraded":"2023-10-11T09:40:41.695042+0000","last_fullsized":"2023-10-11T09:40:41.695042+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_clean_scrub_stamp":"2023-10-11T09:40:28.742276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:22:07.985311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-11T09:40:41.705703+0000","last_change":"2023-10-11T09:40:36.148773+0000","last_active":"2023-10-11T09:40:41.705703+0000","last_peered":"2023-10-11T09:40:41.705703+0000","last_clean":"2023-10-11T09:40:41.705703+0000","last_became_active":"2023-10-11T09:40:36.148489+0000","last_became_peered":"2023-10-11T09:40:36.148489+0000","last_unstale":"2023-10-11T09:40:41.705703+0000","last_undegraded":"2023-10-11T09:40:41.705703+0000","last_fullsized":"2023-10-11T09:40:41.705703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687547+0000","last_change":"2023-10-11T09:40:41.227668+0000","last_active":"2023-10-11T09:40:42.687547+0000","last_peered":"2023-10-11T09:40:42.687547+0000","last_clean":"2023-10-11T09:40:42.687547+0000","last_became_active":"2023-10-11T09:40:41.227357+0000","last_became_peered":"2023-10-11T09:40:41.227357+0000","last_unstale":"2023-10-11T09:40:42.687547+0000","last_undegraded":"2023-10-11T09:40:42.687547+0000","last_fullsized":"2023-10-11T09:40:42.687547+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:16:37.265056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144334+0000","last_change":"2023-10-11T09:40:41.726684+0000","last_active":"2023-10-11T09:40:43.144334+0000","last_peered":"2023-10-11T09:40:43.144334+0000","last_clean":"2023-10-11T09:40:43.144334+0000","last_became_active":"2023-10-11T09:40:41.726323+0000","last_became_peered":"2023-10-11T09:40:41.726323+0000","last_unstale":"2023-10-11T09:40:43.144334+0000","last_undegraded":"2023-10-11T09:40:43.144334+0000","last_fullsized":"2023-10-11T09:40:43.144334+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:47:26.037696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-11T09:40:41.694934+0000","last_change":"2023-10-11T09:40:36.149739+0000","last_active":"2023-10-11T09:40:41.694934+0000","last_peered":"2023-10-11T09:40:41.694934+0000","last_clean":"2023-10-11T09:40:41.694934+0000","last_became_active":"2023-10-11T09:40:36.149419+0000","last_became_peered":"2023-10-11T09:40:36.149419+0000","last_unstale":"2023-10-11T09:40:41.694934+0000","last_undegraded":"2023-10-11T09:40:41.694934+0000","last_fullsized":"2023-10-11T09:40:41.694934+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:54:50.640362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.143706+0000","last_change":"2023-10-11T09:40:41.224529+0000","last_active":"2023-10-11T09:40:43.143706+0000","last_peered":"2023-10-11T09:40:43.143706+0000","last_clean":"2023-10-11T09:40:43.143706+0000","last_became_active":"2023-10-11T09:40:41.224252+0000","last_became_peered":"2023-10-11T09:40:41.224252+0000","last_unstale":"2023-10-11T09:40:43.143706+0000","last_undegraded":"2023-10-11T09:40:43.143706+0000","last_fullsized":"2023-10-11T09:40:43.143706+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:09:03.936717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145788+0000","last_change":"2023-10-11T09:40:41.718539+0000","last_active":"2023-10-11T09:40:43.145788+0000","last_peered":"2023-10-11T09:40:43.145788+0000","last_clean":"2023-10-11T09:40:43.145788+0000","last_became_active":"2023-10-11T09:40:41.718231+0000","last_became_peered":"2023-10-11T09:40:41.718231+0000","last_unstale":"2023-10-11T09:40:43.145788+0000","last_undegraded":"2023-10-11T09:40:43.145788+0000","last_fullsized":"2023-10-11T09:40:43.145788+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:50:15.829941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-11T09:40:41.682824+0000","last_change":"2023-10-11T09:40:36.216248+0000","last_active":"2023-10-11T09:40:41.682824+0000","last_peered":"2023-10-11T09:40:41.682824+0000","last_clean":"2023-10-11T09:40:41.682824+0000","last_became_active":"2023-10-11T09:40:36.215890+0000","last_became_peered":"2023-10-11T09:40:36.215890+0000","last_unstale":"2023-10-11T09:40:41.682824+0000","last_undegraded":"2023-10-11T09:40:41.682824+0000","last_fullsized":"2023-10-11T09:40:41.682824+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.143998+0000","last_change":"2023-10-11T09:40:41.220987+0000","last_active":"2023-10-11T09:40:43.143998+0000","last_peered":"2023-10-11T09:40:43.143998+0000","last_clean":"2023-10-11T09:40:43.143998+0000","last_became_active":"2023-10-11T09:40:41.220035+0000","last_became_peered":"2023-10-11T09:40:41.220035+0000","last_unstale":"2023-10-11T09:40:43.143998+0000","last_undegraded":"2023-10-11T09:40:43.143998+0000","last_fullsized":"2023-10-11T09:40:43.143998+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:00:56.048004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144958+0000","last_change":"2023-10-11T09:40:41.718356+0000","last_active":"2023-10-11T09:40:43.144958+0000","last_peered":"2023-10-11T09:40:43.144958+0000","last_clean":"2023-10-11T09:40:43.144958+0000","last_became_active":"2023-10-11T09:40:41.718069+0000","last_became_peered":"2023-10-11T09:40:41.718069+0000","last_unstale":"2023-10-11T09:40:43.144958+0000","last_undegraded":"2023-10-11T09:40:43.144958+0000","last_fullsized":"2023-10-11T09:40:43.144958+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:42:34.821513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-11T09:40:41.705511+0000","last_change":"2023-10-11T09:40:36.151710+0000","last_active":"2023-10-11T09:40:41.705511+0000","last_peered":"2023-10-11T09:40:41.705511+0000","last_clean":"2023-10-11T09:40:41.705511+0000","last_became_active":"2023-10-11T09:40:36.151219+0000","last_became_peered":"2023-10-11T09:40:36.151219+0000","last_unstale":"2023-10-11T09:40:41.705511+0000","last_undegraded":"2023-10-11T09:40:41.705511+0000","last_fullsized":"2023-10-11T09:40:41.705511+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911785+0000","last_change":"2023-10-11T09:40:41.231464+0000","last_active":"2023-10-11T09:40:42.911785+0000","last_peered":"2023-10-11T09:40:42.911785+0000","last_clean":"2023-10-11T09:40:42.911785+0000","last_became_active":"2023-10-11T09:40:41.229072+0000","last_became_peered":"2023-10-11T09:40:41.229072+0000","last_unstale":"2023-10-11T09:40:42.911785+0000","last_undegraded":"2023-10-11T09:40:42.911785+0000","last_fullsized":"2023-10-11T09:40:42.911785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:33:10.649755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144540+0000","last_change":"2023-10-11T09:40:41.715638+0000","last_active":"2023-10-11T09:40:43.144540+0000","last_peered":"2023-10-11T09:40:43.144540+0000","last_clean":"2023-10-11T09:40:43.144540+0000","last_became_active":"2023-10-11T09:40:41.715399+0000","last_became_peered":"2023-10-11T09:40:41.715399+0000","last_unstale":"2023-10-11T09:40:43.144540+0000","last_undegraded":"2023-10-11T09:40:43.144540+0000","last_fullsized":"2023-10-11T09:40:43.144540+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:00:05.422788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.694217+0000","last_change":"2023-10-11T09:40:41.231735+0000","last_active":"2023-10-11T09:40:42.694217+0000","last_peered":"2023-10-11T09:40:42.694217+0000","last_clean":"2023-10-11T09:40:42.694217+0000","last_became_active":"2023-10-11T09:40:41.229777+0000","last_became_peered":"2023-10-11T09:40:41.229777+0000","last_unstale":"2023-10-11T09:40:42.694217+0000","last_undegraded":"2023-10-11T09:40:42.694217+0000","last_fullsized":"2023-10-11T09:40:42.694217+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:09:03.936717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-11T09:40:41.695209+0000","last_change":"2023-10-11T09:40:36.160620+0000","last_active":"2023-10-11T09:40:41.695209+0000","last_peered":"2023-10-11T09:40:41.695209+0000","last_clean":"2023-10-11T09:40:41.695209+0000","last_became_active":"2023-10-11T09:40:36.160171+0000","last_became_peered":"2023-10-11T09:40:36.160171+0000","last_unstale":"2023-10-11T09:40:41.695209+0000","last_undegraded":"2023-10-11T09:40:41.695209+0000","last_fullsized":"2023-10-11T09:40:41.695209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145219+0000","last_change":"2023-10-11T09:40:41.718869+0000","last_active":"2023-10-11T09:40:43.145219+0000","last_peered":"2023-10-11T09:40:43.145219+0000","last_clean":"2023-10-11T09:40:43.145219+0000","last_became_active":"2023-10-11T09:40:41.718628+0000","last_became_peered":"2023-10-11T09:40:41.718628+0000","last_unstale":"2023-10-11T09:40:43.145219+0000","last_undegraded":"2023-10-11T09:40:43.145219+0000","last_fullsized":"2023-10-11T09:40:43.145219+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:50:15.829941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144700+0000","last_change":"2023-10-11T09:40:41.225562+0000","last_active":"2023-10-11T09:40:43.144700+0000","last_peered":"2023-10-11T09:40:43.144700+0000","last_clean":"2023-10-11T09:40:43.144700+0000","last_became_active":"2023-10-11T09:40:41.222909+0000","last_became_peered":"2023-10-11T09:40:41.222909+0000","last_unstale":"2023-10-11T09:40:43.144700+0000","last_undegraded":"2023-10-11T09:40:43.144700+0000","last_fullsized":"2023-10-11T09:40:43.144700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:09:03.936717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-11T09:40:41.696680+0000","last_change":"2023-10-11T09:40:36.162174+0000","last_active":"2023-10-11T09:40:41.696680+0000","last_peered":"2023-10-11T09:40:41.696680+0000","last_clean":"2023-10-11T09:40:41.696680+0000","last_became_active":"2023-10-11T09:40:36.161710+0000","last_became_peered":"2023-10-11T09:40:36.161710+0000","last_unstale":"2023-10-11T09:40:41.696680+0000","last_undegraded":"2023-10-11T09:40:41.696680+0000","last_fullsized":"2023-10-11T09:40:41.696680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:57:03.630878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145842+0000","last_change":"2023-10-11T09:40:41.716886+0000","last_active":"2023-10-11T09:40:43.145842+0000","last_peered":"2023-10-11T09:40:43.145842+0000","last_clean":"2023-10-11T09:40:43.145842+0000","last_became_active":"2023-10-11T09:40:41.716601+0000","last_became_peered":"2023-10-11T09:40:41.716601+0000","last_unstale":"2023-10-11T09:40:43.145842+0000","last_undegraded":"2023-10-11T09:40:43.145842+0000","last_fullsized":"2023-10-11T09:40:43.145842+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:00:05.422788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.688690+0000","last_change":"2023-10-11T09:40:41.224655+0000","last_active":"2023-10-11T09:40:42.688690+0000","last_peered":"2023-10-11T09:40:42.688690+0000","last_clean":"2023-10-11T09:40:42.688690+0000","last_became_active":"2023-10-11T09:40:41.224374+0000","last_became_peered":"2023-10-11T09:40:41.224374+0000","last_unstale":"2023-10-11T09:40:42.688690+0000","last_undegraded":"2023-10-11T09:40:42.688690+0000","last_fullsized":"2023-10-11T09:40:42.688690+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:33:10.649755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-11T09:40:41.699405+0000","last_change":"2023-10-11T09:40:36.150224+0000","last_active":"2023-10-11T09:40:41.699405+0000","last_peered":"2023-10-11T09:40:41.699405+0000","last_clean":"2023-10-11T09:40:41.699405+0000","last_became_active":"2023-10-11T09:40:36.149919+0000","last_became_peered":"2023-10-11T09:40:36.149919+0000","last_unstale":"2023-10-11T09:40:41.699405+0000","last_undegraded":"2023-10-11T09:40:41.699405+0000","last_fullsized":"2023-10-11T09:40:41.699405+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687621+0000","last_change":"2023-10-11T09:40:41.696509+0000","last_active":"2023-10-11T09:40:42.687621+0000","last_peered":"2023-10-11T09:40:42.687621+0000","last_clean":"2023-10-11T09:40:42.687621+0000","last_became_active":"2023-10-11T09:40:41.696149+0000","last_became_peered":"2023-10-11T09:40:41.696149+0000","last_unstale":"2023-10-11T09:40:42.687621+0000","last_undegraded":"2023-10-11T09:40:42.687621+0000","last_fullsized":"2023-10-11T09:40:42.687621+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:00:05.422788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.438621+0000","last_change":"2023-10-11T09:40:41.230379+0000","last_active":"2023-10-11T09:40:43.438621+0000","last_peered":"2023-10-11T09:40:43.438621+0000","last_clean":"2023-10-11T09:40:43.438621+0000","last_became_active":"2023-10-11T09:40:41.226481+0000","last_became_peered":"2023-10-11T09:40:41.226481+0000","last_unstale":"2023-10-11T09:40:43.438621+0000","last_undegraded":"2023-10-11T09:40:43.438621+0000","last_fullsized":"2023-10-11T09:40:43.438621+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:48:51.882031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-11T09:40:41.708126+0000","last_change":"2023-10-11T09:40:36.151839+0000","last_active":"2023-10-11T09:40:41.708126+0000","last_peered":"2023-10-11T09:40:41.708126+0000","last_clean":"2023-10-11T09:40:41.708126+0000","last_became_active":"2023-10-11T09:40:36.151179+0000","last_became_peered":"2023-10-11T09:40:36.151179+0000","last_unstale":"2023-10-11T09:40:41.708126+0000","last_undegraded":"2023-10-11T09:40:41.708126+0000","last_fullsized":"2023-10-11T09:40:41.708126+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911579+0000","last_change":"2023-10-11T09:40:41.712383+0000","last_active":"2023-10-11T09:40:42.911579+0000","last_peered":"2023-10-11T09:40:42.911579+0000","last_clean":"2023-10-11T09:40:42.911579+0000","last_became_active":"2023-10-11T09:40:41.711978+0000","last_became_peered":"2023-10-11T09:40:41.711978+0000","last_unstale":"2023-10-11T09:40:42.911579+0000","last_undegraded":"2023-10-11T09:40:42.911579+0000","last_fullsized":"2023-10-11T09:40:42.911579+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:45:40.098819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911910+0000","last_change":"2023-10-11T09:40:41.720785+0000","last_active":"2023-10-11T09:40:42.911910+0000","last_peered":"2023-10-11T09:40:42.911910+0000","last_clean":"2023-10-11T09:40:42.911910+0000","last_became_active":"2023-10-11T09:40:41.720508+0000","last_became_peered":"2023-10-11T09:40:41.720508+0000","last_unstale":"2023-10-11T09:40:42.911910+0000","last_undegraded":"2023-10-11T09:40:42.911910+0000","last_fullsized":"2023-10-11T09:40:42.911910+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:35:04.646798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.912435+0000","last_change":"2023-10-11T09:40:41.229054+0000","last_active":"2023-10-11T09:40:42.912435+0000","last_peered":"2023-10-11T09:40:42.912435+0000","last_clean":"2023-10-11T09:40:42.912435+0000","last_became_active":"2023-10-11T09:40:41.227552+0000","last_became_peered":"2023-10-11T09:40:41.227552+0000","last_unstale":"2023-10-11T09:40:42.912435+0000","last_undegraded":"2023-10-11T09:40:42.912435+0000","last_fullsized":"2023-10-11T09:40:42.912435+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:36:34.713602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144970+0000","last_change":"2023-10-11T09:40:41.730718+0000","last_active":"2023-10-11T09:40:43.144970+0000","last_peered":"2023-10-11T09:40:43.144970+0000","last_clean":"2023-10-11T09:40:43.144970+0000","last_became_active":"2023-10-11T09:40:41.730499+0000","last_became_peered":"2023-10-11T09:40:41.730499+0000","last_unstale":"2023-10-11T09:40:43.144970+0000","last_undegraded":"2023-10-11T09:40:43.144970+0000","last_fullsized":"2023-10-11T09:40:43.144970+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:35:36.436236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.692532+0000","last_change":"2023-10-11T09:40:41.230822+0000","last_active":"2023-10-11T09:40:42.692532+0000","last_peered":"2023-10-11T09:40:42.692532+0000","last_clean":"2023-10-11T09:40:42.692532+0000","last_became_active":"2023-10-11T09:40:41.230417+0000","last_became_peered":"2023-10-11T09:40:41.230417+0000","last_unstale":"2023-10-11T09:40:42.692532+0000","last_undegraded":"2023-10-11T09:40:42.692532+0000","last_fullsized":"2023-10-11T09:40:42.692532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:20:53.471012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.694445+0000","last_change":"2023-10-11T09:40:41.231668+0000","last_active":"2023-10-11T09:40:42.694445+0000","last_peered":"2023-10-11T09:40:42.694445+0000","last_clean":"2023-10-11T09:40:42.694445+0000","last_became_active":"2023-10-11T09:40:41.226956+0000","last_became_peered":"2023-10-11T09:40:41.226956+0000","last_unstale":"2023-10-11T09:40:42.694445+0000","last_undegraded":"2023-10-11T09:40:42.694445+0000","last_fullsized":"2023-10-11T09:40:42.694445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:43:00.315992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687648+0000","last_change":"2023-10-11T09:40:41.709493+0000","last_active":"2023-10-11T09:40:42.687648+0000","last_peered":"2023-10-11T09:40:42.687648+0000","last_clean":"2023-10-11T09:40:42.687648+0000","last_became_active":"2023-10-11T09:40:41.709217+0000","last_became_peered":"2023-10-11T09:40:41.709217+0000","last_unstale":"2023-10-11T09:40:42.687648+0000","last_undegraded":"2023-10-11T09:40:42.687648+0000","last_fullsized":"2023-10-11T09:40:42.687648+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:07:02.302344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.146199+0000","last_change":"2023-10-11T09:40:41.228934+0000","last_active":"2023-10-11T09:40:43.146199+0000","last_peered":"2023-10-11T09:40:43.146199+0000","last_clean":"2023-10-11T09:40:43.146199+0000","last_became_active":"2023-10-11T09:40:41.226833+0000","last_became_peered":"2023-10-11T09:40:41.226833+0000","last_unstale":"2023-10-11T09:40:43.146199+0000","last_undegraded":"2023-10-11T09:40:43.146199+0000","last_fullsized":"2023-10-11T09:40:43.146199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:00:56.048004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687536+0000","last_change":"2023-10-11T09:40:41.715540+0000","last_active":"2023-10-11T09:40:42.687536+0000","last_peered":"2023-10-11T09:40:42.687536+0000","last_clean":"2023-10-11T09:40:42.687536+0000","last_became_active":"2023-10-11T09:40:41.715254+0000","last_became_peered":"2023-10-11T09:40:41.715254+0000","last_unstale":"2023-10-11T09:40:42.687536+0000","last_undegraded":"2023-10-11T09:40:42.687536+0000","last_fullsized":"2023-10-11T09:40:42.687536+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:23:30.814623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144670+0000","last_change":"2023-10-11T09:40:41.722871+0000","last_active":"2023-10-11T09:40:43.144670+0000","last_peered":"2023-10-11T09:40:43.144670+0000","last_clean":"2023-10-11T09:40:43.144670+0000","last_became_active":"2023-10-11T09:40:41.722538+0000","last_became_peered":"2023-10-11T09:40:41.722538+0000","last_unstale":"2023-10-11T09:40:43.144670+0000","last_undegraded":"2023-10-11T09:40:43.144670+0000","last_fullsized":"2023-10-11T09:40:43.144670+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:55:31.027029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687761+0000","last_change":"2023-10-11T09:40:41.230916+0000","last_active":"2023-10-11T09:40:42.687761+0000","last_peered":"2023-10-11T09:40:42.687761+0000","last_clean":"2023-10-11T09:40:42.687761+0000","last_became_active":"2023-10-11T09:40:41.230639+0000","last_became_peered":"2023-10-11T09:40:41.230639+0000","last_unstale":"2023-10-11T09:40:42.687761+0000","last_undegraded":"2023-10-11T09:40:42.687761+0000","last_fullsized":"2023-10-11T09:40:42.687761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:48:51.882031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145531+0000","last_change":"2023-10-11T09:40:41.705483+0000","last_active":"2023-10-11T09:40:43.145531+0000","last_peered":"2023-10-11T09:40:43.145531+0000","last_clean":"2023-10-11T09:40:43.145531+0000","last_became_active":"2023-10-11T09:40:41.705060+0000","last_became_peered":"2023-10-11T09:40:41.705060+0000","last_unstale":"2023-10-11T09:40:43.145531+0000","last_undegraded":"2023-10-11T09:40:43.145531+0000","last_fullsized":"2023-10-11T09:40:43.145531+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:35:36.436236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144834+0000","last_change":"2023-10-11T09:40:41.227035+0000","last_active":"2023-10-11T09:40:43.144834+0000","last_peered":"2023-10-11T09:40:43.144834+0000","last_clean":"2023-10-11T09:40:43.144834+0000","last_became_active":"2023-10-11T09:40:41.226593+0000","last_became_peered":"2023-10-11T09:40:41.226593+0000","last_unstale":"2023-10-11T09:40:43.144834+0000","last_undegraded":"2023-10-11T09:40:43.144834+0000","last_fullsized":"2023-10-11T09:40:43.144834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:48:51.882031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.688081+0000","last_change":"2023-10-11T09:40:41.715379+0000","last_active":"2023-10-11T09:40:42.688081+0000","last_peered":"2023-10-11T09:40:42.688081+0000","last_clean":"2023-10-11T09:40:42.688081+0000","last_became_active":"2023-10-11T09:40:41.715091+0000","last_became_peered":"2023-10-11T09:40:41.715091+0000","last_unstale":"2023-10-11T09:40:42.688081+0000","last_undegraded":"2023-10-11T09:40:42.688081+0000","last_fullsized":"2023-10-11T09:40:42.688081+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:42:34.821513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911962+0000","last_change":"2023-10-11T09:40:41.235100+0000","last_active":"2023-10-11T09:40:42.911962+0000","last_peered":"2023-10-11T09:40:42.911962+0000","last_clean":"2023-10-11T09:40:42.911962+0000","last_became_active":"2023-10-11T09:40:41.233361+0000","last_became_peered":"2023-10-11T09:40:41.233361+0000","last_unstale":"2023-10-11T09:40:42.911962+0000","last_undegraded":"2023-10-11T09:40:42.911962+0000","last_fullsized":"2023-10-11T09:40:42.911962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:12:09.780799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.144249+0000","last_change":"2023-10-11T09:40:41.705452+0000","last_active":"2023-10-11T09:40:43.144249+0000","last_peered":"2023-10-11T09:40:43.144249+0000","last_clean":"2023-10-11T09:40:43.144249+0000","last_became_active":"2023-10-11T09:40:41.705028+0000","last_became_peered":"2023-10-11T09:40:41.705028+0000","last_unstale":"2023-10-11T09:40:43.144249+0000","last_undegraded":"2023-10-11T09:40:43.144249+0000","last_fullsized":"2023-10-11T09:40:43.144249+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:09:04.951739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.145980+0000","last_change":"2023-10-11T09:40:41.232125+0000","last_active":"2023-10-11T09:40:43.145980+0000","last_peered":"2023-10-11T09:40:43.145980+0000","last_clean":"2023-10-11T09:40:43.145980+0000","last_became_active":"2023-10-11T09:40:41.231730+0000","last_became_peered":"2023-10-11T09:40:41.231730+0000","last_unstale":"2023-10-11T09:40:43.145980+0000","last_undegraded":"2023-10-11T09:40:43.145980+0000","last_fullsized":"2023-10-11T09:40:43.145980+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:12:04.722202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911689+0000","last_change":"2023-10-11T09:40:41.702062+0000","last_active":"2023-10-11T09:40:42.911689+0000","last_peered":"2023-10-11T09:40:42.911689+0000","last_clean":"2023-10-11T09:40:42.911689+0000","last_became_active":"2023-10-11T09:40:41.701665+0000","last_became_peered":"2023-10-11T09:40:41.701665+0000","last_unstale":"2023-10-11T09:40:42.911689+0000","last_undegraded":"2023-10-11T09:40:42.911689+0000","last_fullsized":"2023-10-11T09:40:42.911689+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:03:52.325886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.693906+0000","last_change":"2023-10-11T09:40:41.227957+0000","last_active":"2023-10-11T09:40:42.693906+0000","last_peered":"2023-10-11T09:40:42.693906+0000","last_clean":"2023-10-11T09:40:42.693906+0000","last_became_active":"2023-10-11T09:40:41.226855+0000","last_became_peered":"2023-10-11T09:40:41.226855+0000","last_unstale":"2023-10-11T09:40:42.693906+0000","last_undegraded":"2023-10-11T09:40:42.693906+0000","last_fullsized":"2023-10-11T09:40:42.693906+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:12:04.722202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911772+0000","last_change":"2023-10-11T09:40:41.699151+0000","last_active":"2023-10-11T09:40:42.911772+0000","last_peered":"2023-10-11T09:40:42.911772+0000","last_clean":"2023-10-11T09:40:42.911772+0000","last_became_active":"2023-10-11T09:40:41.698188+0000","last_became_peered":"2023-10-11T09:40:41.698188+0000","last_unstale":"2023-10-11T09:40:42.911772+0000","last_undegraded":"2023-10-11T09:40:42.911772+0000","last_fullsized":"2023-10-11T09:40:42.911772+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:48:15.689750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.694192+0000","last_change":"2023-10-11T09:40:41.224843+0000","last_active":"2023-10-11T09:40:42.694192+0000","last_peered":"2023-10-11T09:40:42.694192+0000","last_clean":"2023-10-11T09:40:42.694192+0000","last_became_active":"2023-10-11T09:40:41.224415+0000","last_became_peered":"2023-10-11T09:40:41.224415+0000","last_unstale":"2023-10-11T09:40:42.694192+0000","last_undegraded":"2023-10-11T09:40:42.694192+0000","last_fullsized":"2023-10-11T09:40:42.694192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:31:56.724263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687814+0000","last_change":"2023-10-11T09:40:41.695801+0000","last_active":"2023-10-11T09:40:42.687814+0000","last_peered":"2023-10-11T09:40:42.687814+0000","last_clean":"2023-10-11T09:40:42.687814+0000","last_became_active":"2023-10-11T09:40:41.694814+0000","last_became_peered":"2023-10-11T09:40:41.694814+0000","last_unstale":"2023-10-11T09:40:42.687814+0000","last_undegraded":"2023-10-11T09:40:42.687814+0000","last_fullsized":"2023-10-11T09:40:42.687814+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:19:07.089979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.687947+0000","last_change":"2023-10-11T09:40:41.222156+0000","last_active":"2023-10-11T09:40:42.687947+0000","last_peered":"2023-10-11T09:40:42.687947+0000","last_clean":"2023-10-11T09:40:42.687947+0000","last_became_active":"2023-10-11T09:40:41.221758+0000","last_became_peered":"2023-10-11T09:40:41.221758+0000","last_unstale":"2023-10-11T09:40:42.687947+0000","last_undegraded":"2023-10-11T09:40:42.687947+0000","last_fullsized":"2023-10-11T09:40:42.687947+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:00:56.048004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:42.911260+0000","last_change":"2023-10-11T09:40:41.729793+0000","last_active":"2023-10-11T09:40:42.911260+0000","last_peered":"2023-10-11T09:40:42.911260+0000","last_clean":"2023-10-11T09:40:42.911260+0000","last_became_active":"2023-10-11T09:40:41.729468+0000","last_became_peered":"2023-10-11T09:40:41.729468+0000","last_unstale":"2023-10-11T09:40:42.911260+0000","last_undegraded":"2023-10-11T09:40:42.911260+0000","last_fullsized":"2023-10-11T09:40:42.911260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:41.190152+0000","last_clean_scrub_stamp":"2023-10-11T09:40:41.190152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:20:54.486033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T09:40:43.146157+0000","last_change":"2023-10-11T09:40:41.231067+0000","last_active":"2023-10-11T09:40:43.146157+0000","last_peered":"2023-10-11T09:40:43.146157+0000","last_clean":"2023-10-11T09:40:43.146157+0000","last_became_active":"2023-10-11T09:40:41.229252+0000","last_became_peered":"2023-10-11T09:40:41.229252+0000","last_unstale":"2023-10-11T09:40:43.146157+0000","last_undegraded":"2023-10-11T09:40:43.146157+0000","last_fullsized":"2023-10-11T09:40:43.146157+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:40.175130+0000","last_clean_scrub_stamp":"2023-10-11T09:40:40.175130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:43:00.315992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":864256,"data_stored":681747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1479999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.389}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0740000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.333}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.446}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.373}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.407}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.585}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":249856,"data_stored":90251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.238}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0529999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.093}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3620000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.321}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":262144,"data_stored":91219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0860000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.298}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":245760,"data_stored":89689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3540000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.542}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2969999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.474}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.786}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.538}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2689999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27772,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344068,"statfs":{"total":96636764160,"available":96608325632,"internally_reserved":0,"allocated":839680,"data_stored":680549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1879999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.125}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.744}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5449999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.825}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8759999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-11T09:40:53.400 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-11T09:40:53.400 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-11T09:40:53.401 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-11T09:40:53.401 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-11T09:40:53.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.598+0000 7f410e7b7700 1 Processor -- start 2023-10-11T09:40:53.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.598+0000 7f410e7b7700 1 -- start start 2023-10-11T09:40:53.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.602+0000 7f410e7b7700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4108156530 0x7f4108159b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:53.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.602+0000 7f410e7b7700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f410815c830 con 0x7f41080cc280 2023-10-11T09:40:53.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.602+0000 7f410e7b7700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f410815a070 con 0x7f4108156900 2023-10-11T09:40:53.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.602+0000 7f410e7b7700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41080cd750 con 0x7f4108156530 2023-10-11T09:40:53.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.602+0000 7f410dfb6700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f41080cc280 0x7f41080cc690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:58934/0 (socket says 172.21.15.78:58934) 2023-10-11T09:40:53.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.602+0000 7f410dfb6700 1 -- 172.21.15.78:0/4130998991 learned_addr learned my addr 172.21.15.78:0/4130998991 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:53.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.602+0000 7f410d7b5700 1 --2- 172.21.15.78:0/4130998991 >> v2:172.21.15.133:3300/0 conn(0x7f4108156530 0x7f4108159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:53.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.602+0000 7f40fffff700 1 -- 172.21.15.78:0/4130998991 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1291000384 0 0) 0x7f410815c830 con 0x7f41080cc280 2023-10-11T09:40:53.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.602+0000 7f40fffff700 1 -- 172.21.15.78:0/4130998991 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40f40035d0 con 0x7f41080cc280 2023-10-11T09:40:53.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.602+0000 7f410d7b5700 1 -- 172.21.15.78:0/4130998991 >> v1:172.21.15.133:6789/0 conn(0x7f4108156900 legacy=0x7f41080cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:53.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.602+0000 7f410d7b5700 1 -- 172.21.15.78:0/4130998991 >> v1:172.21.15.78:6789/0 conn(0x7f41080cc280 legacy=0x7f41080cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:53.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.602+0000 7f410d7b5700 1 -- 172.21.15.78:0/4130998991 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41080c9e80 con 0x7f4108156530 2023-10-11T09:40:53.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.602+0000 7f410d7b5700 1 --2- 172.21.15.78:0/4130998991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4108156530 0x7f4108159b30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f410400a9f0 tx=0x7f410400ad00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32e0892f1b1b6c12 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:53.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.602+0000 7f40fffff700 1 -- 172.21.15.78:0/4130998991 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410400cb70 con 0x7f4108156530 2023-10-11T09:40:53.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.602+0000 7f40fffff700 1 -- 172.21.15.78:0/4130998991 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410400ae90 con 0x7f4108156530 2023-10-11T09:40:53.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.602+0000 7f40fffff700 1 -- 172.21.15.78:0/4130998991 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4104018400 con 0x7f4108156530 2023-10-11T09:40:53.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.606+0000 7f410e7b7700 1 -- 172.21.15.78:0/4130998991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4108156530 msgr2=0x7f4108159b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:53.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.606+0000 7f410e7b7700 1 --2- 172.21.15.78:0/4130998991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4108156530 0x7f4108159b30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f410400a9f0 tx=0x7f410400ad00 comp rx=0 tx=0).stop 2023-10-11T09:40:53.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.606+0000 7f410e7b7700 1 -- 172.21.15.78:0/4130998991 shutdown_connections 2023-10-11T09:40:53.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.606+0000 7f410e7b7700 1 --2- 172.21.15.78:0/4130998991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4108156530 0x7f4108159b30 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:53.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.606+0000 7f410e7b7700 1 -- 172.21.15.78:0/4130998991 >> 172.21.15.78:0/4130998991 conn(0x7f410806c590 msgr2=0x7f410806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:53.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.606+0000 7f410e7b7700 1 -- 172.21.15.78:0/4130998991 shutdown_connections 2023-10-11T09:40:53.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.606+0000 7f410e7b7700 1 -- 172.21.15.78:0/4130998991 wait complete. 2023-10-11T09:40:53.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.610+0000 7f410e7b7700 1 Processor -- start 2023-10-11T09:40:53.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.610+0000 7f410e7b7700 1 -- start start 2023-10-11T09:40:53.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.610+0000 7f410e7b7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f41080cc280 0x7f41080b0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:53.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.610+0000 7f410e7b7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4108156530 0x7f41080b1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:53.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.610+0000 7f410e7b7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4108156900 0x7f41080aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:53.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.610+0000 7f410e7b7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41081f2df0 con 0x7f4108156900 2023-10-11T09:40:53.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.610+0000 7f410e7b7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41080b11f0 con 0x7f4108156530 2023-10-11T09:40:53.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.610+0000 7f410e7b7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41081f21e0 con 0x7f41080cc280 2023-10-11T09:40:53.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.610+0000 7f410dfb6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4108156900 0x7f41080aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:53.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.610+0000 7f410dfb6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4108156900 0x7f41080aef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57266/0 (socket says 172.21.15.78:57266) 2023-10-11T09:40:53.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.610+0000 7f410dfb6700 1 -- 172.21.15.78:0/2775998236 learned_addr learned my addr 172.21.15.78:0/2775998236 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:53.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.610+0000 7f410d7b5700 1 --2- 172.21.15.78:0/2775998236 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f41080cc280 0x7f41080b0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:53.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.610+0000 7f410cfb4700 1 --2- 172.21.15.78:0/2775998236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4108156530 0x7f41080b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:53.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.610+0000 7f410dfb6700 1 -- 172.21.15.78:0/2775998236 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f41080cc280 msgr2=0x7f41080b0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:53.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.610+0000 7f410dfb6700 1 --2- 172.21.15.78:0/2775998236 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f41080cc280 0x7f41080b0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:53.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.610+0000 7f410dfb6700 1 -- 172.21.15.78:0/2775998236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4108156530 msgr2=0x7f41080b1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:53.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.610+0000 7f410dfb6700 1 --2- 172.21.15.78:0/2775998236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4108156530 0x7f41080b1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:53.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.610+0000 7f410dfb6700 1 -- 172.21.15.78:0/2775998236 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f410400a650 con 0x7f4108156900 2023-10-11T09:40:53.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.610+0000 7f410dfb6700 1 --2- 172.21.15.78:0/2775998236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4108156900 0x7f41080aef40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f410000cb40 tx=0x7f410000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:53.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.610+0000 7f40fdffb700 1 -- 172.21.15.78:0/2775998236 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410000a9e0 con 0x7f4108156900 2023-10-11T09:40:53.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.610+0000 7f40fdffb700 1 -- 172.21.15.78:0/2775998236 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410000ab40 con 0x7f4108156900 2023-10-11T09:40:53.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.614+0000 7f40fdffb700 1 -- 172.21.15.78:0/2775998236 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410000ce60 con 0x7f4108156900 2023-10-11T09:40:53.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.614+0000 7f410e7b7700 1 -- 172.21.15.78:0/2775998236 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41081f24c0 con 0x7f4108156900 2023-10-11T09:40:53.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.614+0000 7f410e7b7700 1 -- 172.21.15.78:0/2775998236 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41081f2a80 con 0x7f4108156900 2023-10-11T09:40:53.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.614+0000 7f410e7b7700 1 -- 172.21.15.78:0/2775998236 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4108067f50 con 0x7f4108156900 2023-10-11T09:40:53.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.614+0000 7f40fdffb700 1 -- 172.21.15.78:0/2775998236 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f410001b3b0 con 0x7f4108156900 2023-10-11T09:40:53.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.618+0000 7f40fdffb700 1 --2- 172.21.15.78:0/2775998236 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f40ec065ed0 0x7f40ec068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:53.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.618+0000 7f40fdffb700 1 -- 172.21.15.78:0/2775998236 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f41000832e0 con 0x7f4108156900 2023-10-11T09:40:53.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.618+0000 7f410d7b5700 1 --2- 172.21.15.78:0/2775998236 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f40ec065ed0 0x7f40ec068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:53.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.618+0000 7f410d7b5700 1 --2- 172.21.15.78:0/2775998236 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f40ec065ed0 0x7f40ec068380 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f410400a1c0 tx=0x7f410401a040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:53.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.622+0000 7f40fdffb700 1 -- 172.21.15.78:0/2775998236 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f4100050930 con 0x7f4108156900 2023-10-11T09:40:53.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.834+0000 7f410e7b7700 1 -- 172.21.15.78:0/2775998236 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f4108065d40 con 0x7f4108156900 2023-10-11T09:40:53.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.834+0000 7f40fdffb700 1 -- 172.21.15.78:0/2775998236 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f4100053f50 con 0x7f4108156900 2023-10-11T09:40:53.840 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:40:53.840 INFO:teuthology.orchestra.run.smithi078.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-11T09:40:53.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.838+0000 7f410e7b7700 1 -- 172.21.15.78:0/2775998236 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f40ec065ed0 msgr2=0x7f40ec068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:53.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.838+0000 7f410e7b7700 1 --2- 172.21.15.78:0/2775998236 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f40ec065ed0 0x7f40ec068380 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f410400a1c0 tx=0x7f410401a040 comp rx=0 tx=0).stop 2023-10-11T09:40:53.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.838+0000 7f410e7b7700 1 -- 172.21.15.78:0/2775998236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4108156900 msgr2=0x7f41080aef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:53.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.838+0000 7f410e7b7700 1 --2- 172.21.15.78:0/2775998236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4108156900 0x7f41080aef40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f410000cb40 tx=0x7f410000cc20 comp rx=0 tx=0).stop 2023-10-11T09:40:53.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.838+0000 7f410e7b7700 1 -- 172.21.15.78:0/2775998236 shutdown_connections 2023-10-11T09:40:53.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.838+0000 7f410e7b7700 1 --2- 172.21.15.78:0/2775998236 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f40ec065ed0 0x7f40ec068380 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:53.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.838+0000 7f410e7b7700 1 --2- 172.21.15.78:0/2775998236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4108156900 0x7f41080aef40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:53.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.838+0000 7f410e7b7700 1 --2- 172.21.15.78:0/2775998236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4108156530 0x7f41080b1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:53.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.838+0000 7f410e7b7700 1 --2- 172.21.15.78:0/2775998236 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f41080cc280 0x7f41080b0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:53.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.838+0000 7f410e7b7700 1 -- 172.21.15.78:0/2775998236 >> 172.21.15.78:0/2775998236 conn(0x7f410806c590 msgr2=0x7f410815ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:53.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.838+0000 7f410e7b7700 1 -- 172.21.15.78:0/2775998236 shutdown_connections 2023-10-11T09:40:53.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:53.838+0000 7f410e7b7700 1 -- 172.21.15.78:0/2775998236 wait complete. 2023-10-11T09:40:53.862 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-11T09:40:53.863 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:40:54.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.026+0000 7f4e8759e700 1 Processor -- start 2023-10-11T09:40:54.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.030+0000 7f4e8759e700 1 -- start start 2023-10-11T09:40:54.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.030+0000 7f4e8759e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4e8813f740 0x7f4e880ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:54.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.030+0000 7f4e8759e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f4e8806d180 0x7f4e880b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:54.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.030+0000 7f4e8759e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e88070c80 con 0x7f4e8806cd00 2023-10-11T09:40:54.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.030+0000 7f4e8759e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e880ce4c0 con 0x7f4e8806d180 2023-10-11T09:40:54.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.030+0000 7f4e8759e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e8806d600 con 0x7f4e8813f740 2023-10-11T09:40:54.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.030+0000 7f4e7dd9b700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f4e8806cd00 0x7f4e880b06a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:58942/0 (socket says 172.21.15.78:58942) 2023-10-11T09:40:54.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.030+0000 7f4e7dd9b700 1 -- 172.21.15.78:0/649697140 learned_addr learned my addr 172.21.15.78:0/649697140 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:54.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.034+0000 7f4e86d9d700 1 --2- 172.21.15.78:0/649697140 >> v2:172.21.15.78:3300/0 conn(0x7f4e8806d180 0x7f4e880b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:54.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.034+0000 7f4e8659c700 1 --2- 172.21.15.78:0/649697140 >> v2:172.21.15.133:3300/0 conn(0x7f4e8813f740 0x7f4e880ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:54.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.034+0000 7f4e85d9b700 1 -- 172.21.15.78:0/649697140 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2597451 0 0) 0x7f4e88070c80 con 0x7f4e8806cd00 2023-10-11T09:40:54.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.034+0000 7f4e85d9b700 1 -- 172.21.15.78:0/649697140 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e740035d0 con 0x7f4e8806cd00 2023-10-11T09:40:54.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.034+0000 7f4e8659c700 1 -- 172.21.15.78:0/649697140 >> v2:172.21.15.78:3300/0 conn(0x7f4e8806d180 msgr2=0x7f4e880b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:54.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.034+0000 7f4e8659c700 1 --2- 172.21.15.78:0/649697140 >> v2:172.21.15.78:3300/0 conn(0x7f4e8806d180 0x7f4e880b0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:54.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.034+0000 7f4e8659c700 1 -- 172.21.15.78:0/649697140 >> v1:172.21.15.78:6789/0 conn(0x7f4e8806cd00 legacy=0x7f4e880b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:54.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.034+0000 7f4e8659c700 1 -- 172.21.15.78:0/649697140 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e880cd420 con 0x7f4e8813f740 2023-10-11T09:40:54.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.034+0000 7f4e8659c700 1 --2- 172.21.15.78:0/649697140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e8813f740 0x7f4e880ccab0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4e8806fb70 tx=0x7f4e78009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62d0ee1a7a3e10c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:54.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.034+0000 7f4e85d9b700 1 -- 172.21.15.78:0/649697140 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e78011830 con 0x7f4e8813f740 2023-10-11T09:40:54.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.034+0000 7f4e85d9b700 1 -- 172.21.15.78:0/649697140 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e78011990 con 0x7f4e8813f740 2023-10-11T09:40:54.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.034+0000 7f4e85d9b700 1 -- 172.21.15.78:0/649697140 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e78011d20 con 0x7f4e8813f740 2023-10-11T09:40:54.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.034+0000 7f4e8759e700 1 -- 172.21.15.78:0/649697140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e8813f740 msgr2=0x7f4e880ccab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:54.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.034+0000 7f4e8759e700 1 --2- 172.21.15.78:0/649697140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e8813f740 0x7f4e880ccab0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4e8806fb70 tx=0x7f4e78009b30 comp rx=0 tx=0).stop 2023-10-11T09:40:54.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.034+0000 7f4e8759e700 1 -- 172.21.15.78:0/649697140 shutdown_connections 2023-10-11T09:40:54.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.034+0000 7f4e8759e700 1 --2- 172.21.15.78:0/649697140 >> v2:172.21.15.78:3300/0 conn(0x7f4e8806d180 0x7f4e880b0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:54.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.034+0000 7f4e8759e700 1 --2- 172.21.15.78:0/649697140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e8813f740 0x7f4e880ccab0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:54.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.034+0000 7f4e8759e700 1 -- 172.21.15.78:0/649697140 >> 172.21.15.78:0/649697140 conn(0x7f4e880cea60 msgr2=0x7f4e880cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:54.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.034+0000 7f4e8759e700 1 -- 172.21.15.78:0/649697140 shutdown_connections 2023-10-11T09:40:54.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.034+0000 7f4e8759e700 1 -- 172.21.15.78:0/649697140 wait complete. 2023-10-11T09:40:54.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.038+0000 7f4e8759e700 1 Processor -- start 2023-10-11T09:40:54.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.038+0000 7f4e8759e700 1 -- start start 2023-10-11T09:40:54.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.038+0000 7f4e8759e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e8806cd00 0x7f4e8806c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:54.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.038+0000 7f4e8759e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e8806d180 0x7f4e881e4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:54.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.038+0000 7f4e8759e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e8813f740 0x7f4e881565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:54.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.038+0000 7f4e8759e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e881558b0 con 0x7f4e8806d180 2023-10-11T09:40:54.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.038+0000 7f4e8759e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e88155680 con 0x7f4e8806cd00 2023-10-11T09:40:54.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.038+0000 7f4e8759e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e881e4bb0 con 0x7f4e8813f740 2023-10-11T09:40:54.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.038+0000 7f4e7dd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e8806d180 0x7f4e881e4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:54.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.038+0000 7f4e7dd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e8806d180 0x7f4e881e4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57286/0 (socket says 172.21.15.78:57286) 2023-10-11T09:40:54.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.038+0000 7f4e7dd9b700 1 -- 172.21.15.78:0/3290107396 learned_addr learned my addr 172.21.15.78:0/3290107396 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:54.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.038+0000 7f4e8659c700 1 --2- 172.21.15.78:0/3290107396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e8806cd00 0x7f4e8806c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:54.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.038+0000 7f4e86d9d700 1 --2- 172.21.15.78:0/3290107396 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e8813f740 0x7f4e881565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:54.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.042+0000 7f4e7dd9b700 1 -- 172.21.15.78:0/3290107396 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e8813f740 msgr2=0x7f4e881565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:54.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.042+0000 7f4e7dd9b700 1 --2- 172.21.15.78:0/3290107396 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e8813f740 0x7f4e881565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:54.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.042+0000 7f4e7dd9b700 1 -- 172.21.15.78:0/3290107396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e8806cd00 msgr2=0x7f4e8806c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:54.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.042+0000 7f4e7dd9b700 1 --2- 172.21.15.78:0/3290107396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e8806cd00 0x7f4e8806c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:54.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.042+0000 7f4e7dd9b700 1 -- 172.21.15.78:0/3290107396 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e780094b0 con 0x7f4e8806d180 2023-10-11T09:40:54.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.042+0000 7f4e7dd9b700 1 --2- 172.21.15.78:0/3290107396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e8806d180 0x7f4e881e4670 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4e7000c4d0 tx=0x7f4e7000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:54.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.042+0000 7f4e7f7fe700 1 -- 172.21.15.78:0/3290107396 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e70005d90 con 0x7f4e8806d180 2023-10-11T09:40:54.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.042+0000 7f4e7f7fe700 1 -- 172.21.15.78:0/3290107396 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e700070d0 con 0x7f4e8806d180 2023-10-11T09:40:54.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.042+0000 7f4e7f7fe700 1 -- 172.21.15.78:0/3290107396 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e70007480 con 0x7f4e8806d180 2023-10-11T09:40:54.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.042+0000 7f4e8759e700 1 -- 172.21.15.78:0/3290107396 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e881e4e80 con 0x7f4e8806d180 2023-10-11T09:40:54.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.042+0000 7f4e8759e700 1 -- 172.21.15.78:0/3290107396 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e88155de0 con 0x7f4e8806d180 2023-10-11T09:40:54.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.042+0000 7f4e8759e700 1 -- 172.21.15.78:0/3290107396 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e88065d40 con 0x7f4e8806d180 2023-10-11T09:40:54.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.046+0000 7f4e7f7fe700 1 -- 172.21.15.78:0/3290107396 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f4e700075e0 con 0x7f4e8806d180 2023-10-11T09:40:54.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.046+0000 7f4e7f7fe700 1 --2- 172.21.15.78:0/3290107396 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4e68065ed0 0x7f4e68068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:54.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.046+0000 7f4e7f7fe700 1 -- 172.21.15.78:0/3290107396 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4e70082bd0 con 0x7f4e8806d180 2023-10-11T09:40:54.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.046+0000 7f4e8659c700 1 --2- 172.21.15.78:0/3290107396 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4e68065ed0 0x7f4e68068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:54.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.046+0000 7f4e8659c700 1 --2- 172.21.15.78:0/3290107396 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4e68065ed0 0x7f4e68068380 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4e78000c00 tx=0x7f4e78008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:54.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.050+0000 7f4e7f7fe700 1 -- 172.21.15.78:0/3290107396 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f4e70050220 con 0x7f4e8806d180 2023-10-11T09:40:54.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.230+0000 7f4e8759e700 1 -- 172.21.15.78:0/3290107396 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e88159290 con 0x7f4e8806d180 2023-10-11T09:40:54.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.230+0000 7f4e7f7fe700 1 -- 172.21.15.78:0/3290107396 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17811 (secure 0 0 0) 0x7f4e70017090 con 0x7f4e8806d180 2023-10-11T09:40:54.237 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:40:54.237 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":23,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:40:42.678477+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:40:39.862581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T09:40:40.663641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:40:54.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.234+0000 7f4e8759e700 1 -- 172.21.15.78:0/3290107396 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4e68065ed0 msgr2=0x7f4e68068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:54.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.234+0000 7f4e8759e700 1 --2- 172.21.15.78:0/3290107396 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4e68065ed0 0x7f4e68068380 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4e78000c00 tx=0x7f4e78008030 comp rx=0 tx=0).stop 2023-10-11T09:40:54.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.234+0000 7f4e8759e700 1 -- 172.21.15.78:0/3290107396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e8806d180 msgr2=0x7f4e881e4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:54.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.234+0000 7f4e8759e700 1 --2- 172.21.15.78:0/3290107396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e8806d180 0x7f4e881e4670 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4e7000c4d0 tx=0x7f4e7000c7e0 comp rx=0 tx=0).stop 2023-10-11T09:40:54.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.234+0000 7f4e8759e700 1 -- 172.21.15.78:0/3290107396 shutdown_connections 2023-10-11T09:40:54.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.234+0000 7f4e8759e700 1 --2- 172.21.15.78:0/3290107396 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4e68065ed0 0x7f4e68068380 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:54.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.234+0000 7f4e8759e700 1 --2- 172.21.15.78:0/3290107396 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e8813f740 0x7f4e881565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:54.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.234+0000 7f4e8759e700 1 --2- 172.21.15.78:0/3290107396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e8806d180 0x7f4e881e4670 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:54.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.234+0000 7f4e8759e700 1 --2- 172.21.15.78:0/3290107396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e8806cd00 0x7f4e8806c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:54.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.234+0000 7f4e8759e700 1 -- 172.21.15.78:0/3290107396 >> 172.21.15.78:0/3290107396 conn(0x7f4e880cea60 msgr2=0x7f4e8813d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:54.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.234+0000 7f4e8759e700 1 -- 172.21.15.78:0/3290107396 shutdown_connections 2023-10-11T09:40:54.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.234+0000 7f4e8759e700 1 -- 172.21.15.78:0/3290107396 wait complete. 2023-10-11T09:40:54.259 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T09:40:39.862581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T09:40:40.663641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:40:54.260 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:40:54.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.430+0000 7fde247a7700 1 Processor -- start 2023-10-11T09:40:54.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.430+0000 7fde247a7700 1 -- start start 2023-10-11T09:40:54.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.434+0000 7fde247a7700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fde1c06d340 0x7fde1c0ca880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:54.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.434+0000 7fde247a7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fde1c0cadc0 0x7fde1c15afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:54.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.434+0000 7fde247a7700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde1c15e280 con 0x7fde1c158430 2023-10-11T09:40:54.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.434+0000 7fde247a7700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde1c070c10 con 0x7fde1c06d340 2023-10-11T09:40:54.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.434+0000 7fde247a7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c0cb240 con 0x7fde1c0cadc0 2023-10-11T09:40:54.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.434+0000 7fde21d42700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fde1c06d340 0x7fde1c0ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:54.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.434+0000 7fde22543700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fde1c158430 0x7fde1c0710d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:58954/0 (socket says 172.21.15.78:58954) 2023-10-11T09:40:54.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.434+0000 7fde22543700 1 -- 172.21.15.78:0/4059141407 learned_addr learned my addr 172.21.15.78:0/4059141407 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:54.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.434+0000 7fde22d44700 1 --2- 172.21.15.78:0/4059141407 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fde1c0cadc0 0x7fde1c15afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:54.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.434+0000 7fde21d42700 1 -- 172.21.15.78:0/4059141407 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fde1c0cadc0 msgr2=0x7fde1c15afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:54.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.434+0000 7fde21d42700 1 --2- 172.21.15.78:0/4059141407 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fde1c0cadc0 0x7fde1c15afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:54.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.434+0000 7fde21d42700 1 -- 172.21.15.78:0/4059141407 >> v1:172.21.15.78:6789/0 conn(0x7fde1c158430 legacy=0x7fde1c0710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:54.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.434+0000 7fde21d42700 1 -- 172.21.15.78:0/4059141407 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1c15e560 con 0x7fde1c06d340 2023-10-11T09:40:54.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.434+0000 7fde21d42700 1 --2- 172.21.15.78:0/4059141407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde1c06d340 0x7fde1c0ca880 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fde0c009850 tx=0x7fde0c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d09affd39dba528 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:54.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.434+0000 7fde21541700 1 -- 172.21.15.78:0/4059141407 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0c011830 con 0x7fde1c06d340 2023-10-11T09:40:54.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.434+0000 7fde21541700 1 -- 172.21.15.78:0/4059141407 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde0c011990 con 0x7fde1c06d340 2023-10-11T09:40:54.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.434+0000 7fde21541700 1 -- 172.21.15.78:0/4059141407 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0c011d40 con 0x7fde1c06d340 2023-10-11T09:40:54.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.434+0000 7fde247a7700 1 -- 172.21.15.78:0/4059141407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde1c06d340 msgr2=0x7fde1c0ca880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:54.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.434+0000 7fde247a7700 1 --2- 172.21.15.78:0/4059141407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde1c06d340 0x7fde1c0ca880 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fde0c009850 tx=0x7fde0c009b60 comp rx=0 tx=0).stop 2023-10-11T09:40:54.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.434+0000 7fde247a7700 1 -- 172.21.15.78:0/4059141407 shutdown_connections 2023-10-11T09:40:54.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.434+0000 7fde247a7700 1 --2- 172.21.15.78:0/4059141407 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fde1c0cadc0 0x7fde1c15afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:54.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.434+0000 7fde247a7700 1 --2- 172.21.15.78:0/4059141407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde1c06d340 0x7fde1c0ca880 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:54.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.434+0000 7fde247a7700 1 -- 172.21.15.78:0/4059141407 >> 172.21.15.78:0/4059141407 conn(0x7fde1c0cecc0 msgr2=0x7fde1c0cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:54.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.438+0000 7fde247a7700 1 -- 172.21.15.78:0/4059141407 shutdown_connections 2023-10-11T09:40:54.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.438+0000 7fde247a7700 1 -- 172.21.15.78:0/4059141407 wait complete. 2023-10-11T09:40:54.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.438+0000 7fde247a7700 1 Processor -- start 2023-10-11T09:40:54.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.438+0000 7fde247a7700 1 -- start start 2023-10-11T09:40:54.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.438+0000 7fde247a7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fde1c06d340 0x7fde1c156d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:54.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.438+0000 7fde247a7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde1c0cadc0 0x7fde1c0b2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:54.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.438+0000 7fde247a7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde1c158430 0x7fde1c1f55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:54.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.438+0000 7fde247a7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c1f6090 con 0x7fde1c158430 2023-10-11T09:40:54.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.438+0000 7fde247a7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c1f6370 con 0x7fde1c0cadc0 2023-10-11T09:40:54.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.438+0000 7fde247a7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c0b1c90 con 0x7fde1c06d340 2023-10-11T09:40:54.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.438+0000 7fde22d44700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde1c158430 0x7fde1c1f55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:54.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.438+0000 7fde22d44700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde1c158430 0x7fde1c1f55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57300/0 (socket says 172.21.15.78:57300) 2023-10-11T09:40:54.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.438+0000 7fde22d44700 1 -- 172.21.15.78:0/2747767099 learned_addr learned my addr 172.21.15.78:0/2747767099 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:54.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.438+0000 7fde22543700 1 --2- 172.21.15.78:0/2747767099 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fde1c06d340 0x7fde1c156d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:54.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.438+0000 7fde21d42700 1 --2- 172.21.15.78:0/2747767099 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde1c0cadc0 0x7fde1c0b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:54.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.442+0000 7fde22d44700 1 -- 172.21.15.78:0/2747767099 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fde1c06d340 msgr2=0x7fde1c156d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:54.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.442+0000 7fde22d44700 1 --2- 172.21.15.78:0/2747767099 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fde1c06d340 0x7fde1c156d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:54.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.442+0000 7fde22d44700 1 -- 172.21.15.78:0/2747767099 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde1c0cadc0 msgr2=0x7fde1c0b2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:54.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.442+0000 7fde22d44700 1 --2- 172.21.15.78:0/2747767099 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde1c0cadc0 0x7fde1c0b2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:54.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.442+0000 7fde22d44700 1 -- 172.21.15.78:0/2747767099 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde0c0094b0 con 0x7fde1c158430 2023-10-11T09:40:54.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.442+0000 7fde21d42700 1 --2- 172.21.15.78:0/2747767099 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde1c0cadc0 0x7fde1c0b2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:40:54.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.442+0000 7fde22d44700 1 --2- 172.21.15.78:0/2747767099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde1c158430 0x7fde1c1f55f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fde1400efc0 tx=0x7fde1400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:54.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.442+0000 7fde12ffd700 1 -- 172.21.15.78:0/2747767099 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde14009ea0 con 0x7fde1c158430 2023-10-11T09:40:54.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.442+0000 7fde12ffd700 1 -- 172.21.15.78:0/2747767099 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde1400f040 con 0x7fde1c158430 2023-10-11T09:40:54.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.442+0000 7fde247a7700 1 -- 172.21.15.78:0/2747767099 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde1c0b1f70 con 0x7fde1c158430 2023-10-11T09:40:54.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.442+0000 7fde12ffd700 1 -- 172.21.15.78:0/2747767099 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde14014480 con 0x7fde1c158430 2023-10-11T09:40:54.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.442+0000 7fde247a7700 1 -- 172.21.15.78:0/2747767099 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde1c0cf860 con 0x7fde1c158430 2023-10-11T09:40:54.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.442+0000 7fde247a7700 1 -- 172.21.15.78:0/2747767099 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde1c06eb80 con 0x7fde1c158430 2023-10-11T09:40:54.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.442+0000 7fde12ffd700 1 -- 172.21.15.78:0/2747767099 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fde140145e0 con 0x7fde1c158430 2023-10-11T09:40:54.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.446+0000 7fde12ffd700 1 --2- 172.21.15.78:0/2747767099 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fde00065ed0 0x7fde00068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:54.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.446+0000 7fde12ffd700 1 -- 172.21.15.78:0/2747767099 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fde14083fe0 con 0x7fde1c158430 2023-10-11T09:40:54.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.446+0000 7fde22543700 1 --2- 172.21.15.78:0/2747767099 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fde00065ed0 0x7fde00068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:54.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.446+0000 7fde22543700 1 --2- 172.21.15.78:0/2747767099 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fde00065ed0 0x7fde00068380 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fde18005930 tx=0x7fde18009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:54.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.450+0000 7fde12ffd700 1 -- 172.21.15.78:0/2747767099 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fde140516a0 con 0x7fde1c158430 2023-10-11T09:40:54.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.622+0000 7fde247a7700 1 -- 172.21.15.78:0/2747767099 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fde1c157de0 con 0x7fde1c158430 2023-10-11T09:40:54.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.626+0000 7fde12ffd700 1 -- 172.21.15.78:0/2747767099 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fde14018020 con 0x7fde1c158430 2023-10-11T09:40:54.630 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:40:54.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.626+0000 7fde247a7700 1 -- 172.21.15.78:0/2747767099 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fde00065ed0 msgr2=0x7fde00068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:54.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.626+0000 7fde247a7700 1 --2- 172.21.15.78:0/2747767099 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fde00065ed0 0x7fde00068380 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fde18005930 tx=0x7fde18009040 comp rx=0 tx=0).stop 2023-10-11T09:40:54.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.626+0000 7fde247a7700 1 -- 172.21.15.78:0/2747767099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde1c158430 msgr2=0x7fde1c1f55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:54.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.626+0000 7fde247a7700 1 --2- 172.21.15.78:0/2747767099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde1c158430 0x7fde1c1f55f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fde1400efc0 tx=0x7fde1400c3a0 comp rx=0 tx=0).stop 2023-10-11T09:40:54.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.626+0000 7fde247a7700 1 -- 172.21.15.78:0/2747767099 shutdown_connections 2023-10-11T09:40:54.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.626+0000 7fde247a7700 1 --2- 172.21.15.78:0/2747767099 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fde00065ed0 0x7fde00068380 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:54.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.626+0000 7fde247a7700 1 --2- 172.21.15.78:0/2747767099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde1c158430 0x7fde1c1f55f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:54.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.626+0000 7fde247a7700 1 --2- 172.21.15.78:0/2747767099 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde1c0cadc0 0x7fde1c0b2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:54.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.626+0000 7fde247a7700 1 --2- 172.21.15.78:0/2747767099 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fde1c06d340 0x7fde1c156d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:54.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.626+0000 7fde247a7700 1 -- 172.21.15.78:0/2747767099 >> 172.21.15.78:0/2747767099 conn(0x7fde1c0cecc0 msgr2=0x7fde1c15d670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:54.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.626+0000 7fde247a7700 1 -- 172.21.15.78:0/2747767099 shutdown_connections 2023-10-11T09:40:54.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.630+0000 7fde247a7700 1 -- 172.21.15.78:0/2747767099 wait complete. 2023-10-11T09:40:54.652 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:40:54.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.822+0000 7f492759e700 1 Processor -- start 2023-10-11T09:40:54.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.822+0000 7f492759e700 1 -- start start 2023-10-11T09:40:54.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.826+0000 7f492759e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f49280b3610 0x7f49280d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:54.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.826+0000 7f492759e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49280b47f0 con 0x7f49280b3240 2023-10-11T09:40:54.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.826+0000 7f492759e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49280b8130 con 0x7f49280b3a90 2023-10-11T09:40:54.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.826+0000 7f492759e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49280b41e0 con 0x7f49280b3610 2023-10-11T09:40:54.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.826+0000 7f492659c700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f49280b3240 0x7f492815d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:34556/0 (socket says 172.21.15.78:34556) 2023-10-11T09:40:54.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.826+0000 7f492659c700 1 -- 172.21.15.78:0/278955663 learned_addr learned my addr 172.21.15.78:0/278955663 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:54.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.826+0000 7f491dd9b700 1 --2- 172.21.15.78:0/278955663 >> v2:172.21.15.133:3300/0 conn(0x7f49280b3610 0x7f49280d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:54.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.826+0000 7f4925d9b700 1 -- 172.21.15.78:0/278955663 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1085680003 0 0) 0x7f4918001490 con 0x7f49280b3240 2023-10-11T09:40:54.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.826+0000 7f4925d9b700 1 -- 172.21.15.78:0/278955663 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49140035d0 con 0x7f49280b3240 2023-10-11T09:40:54.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.826+0000 7f4925d9b700 1 -- 172.21.15.78:0/278955663 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3792590110 0 0) 0x7f49280b8130 con 0x7f49280b3a90 2023-10-11T09:40:54.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.826+0000 7f4925d9b700 1 -- 172.21.15.78:0/278955663 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4918001490 con 0x7f49280b3a90 2023-10-11T09:40:54.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.826+0000 7f491dd9b700 1 -- 172.21.15.78:0/278955663 >> v1:172.21.15.133:6789/0 conn(0x7f49280b3a90 legacy=0x7f49280b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:54.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.826+0000 7f491dd9b700 1 -- 172.21.15.78:0/278955663 >> v1:172.21.15.78:6789/0 conn(0x7f49280b3240 legacy=0x7f492815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:54.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.826+0000 7f491dd9b700 1 -- 172.21.15.78:0/278955663 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49280b7800 con 0x7f49280b3610 2023-10-11T09:40:54.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.826+0000 7f491dd9b700 1 --2- 172.21.15.78:0/278955663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49280b3610 0x7f49280d29d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4910009850 tx=0x7f4910009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36afc6d0c18d26cf server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:54.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.826+0000 7f4925d9b700 1 -- 172.21.15.78:0/278955663 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4910011830 con 0x7f49280b3610 2023-10-11T09:40:54.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.826+0000 7f4925d9b700 1 -- 172.21.15.78:0/278955663 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4910011990 con 0x7f49280b3610 2023-10-11T09:40:54.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.826+0000 7f4925d9b700 1 -- 172.21.15.78:0/278955663 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4910011d40 con 0x7f49280b3610 2023-10-11T09:40:54.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.830+0000 7f492759e700 1 -- 172.21.15.78:0/278955663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49280b3610 msgr2=0x7f49280d29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:54.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.830+0000 7f492759e700 1 --2- 172.21.15.78:0/278955663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49280b3610 0x7f49280d29d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4910009850 tx=0x7f4910009b60 comp rx=0 tx=0).stop 2023-10-11T09:40:54.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.830+0000 7f492759e700 1 -- 172.21.15.78:0/278955663 shutdown_connections 2023-10-11T09:40:54.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.830+0000 7f492759e700 1 --2- 172.21.15.78:0/278955663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49280b3610 0x7f49280d29d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:54.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.830+0000 7f492759e700 1 -- 172.21.15.78:0/278955663 >> 172.21.15.78:0/278955663 conn(0x7f492809c260 msgr2=0x7f49280d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:54.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.830+0000 7f492759e700 1 -- 172.21.15.78:0/278955663 shutdown_connections 2023-10-11T09:40:54.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.830+0000 7f492759e700 1 -- 172.21.15.78:0/278955663 wait complete. 2023-10-11T09:40:54.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.834+0000 7f492759e700 1 Processor -- start 2023-10-11T09:40:54.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.834+0000 7f492759e700 1 -- start start 2023-10-11T09:40:54.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.834+0000 7f492759e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49280b3240 0x7f49281563c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:54.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.834+0000 7f492759e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f49280b3610 0x7f49281522f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:54.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.834+0000 7f492759e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49280b3a90 0x7f4928152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:54.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.834+0000 7f492759e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4928156900 con 0x7f49280b3240 2023-10-11T09:40:54.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.834+0000 7f492759e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4928156b30 con 0x7f49280b3a90 2023-10-11T09:40:54.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.834+0000 7f492759e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4928156e10 con 0x7f49280b3610 2023-10-11T09:40:54.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.834+0000 7f492659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49280b3240 0x7f49281563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:54.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.834+0000 7f492659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49280b3240 0x7f49281563c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56594/0 (socket says 172.21.15.78:56594) 2023-10-11T09:40:54.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.834+0000 7f492659c700 1 -- 172.21.15.78:0/1308499800 learned_addr learned my addr 172.21.15.78:0/1308499800 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:54.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.834+0000 7f491dd9b700 1 --2- 172.21.15.78:0/1308499800 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f49280b3610 0x7f49281522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:54.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.834+0000 7f492659c700 1 -- 172.21.15.78:0/1308499800 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f49280b3610 msgr2=0x7f49281522f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:54.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.834+0000 7f492659c700 1 --2- 172.21.15.78:0/1308499800 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f49280b3610 0x7f49281522f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:54.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.834+0000 7f492659c700 1 -- 172.21.15.78:0/1308499800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49280b3a90 msgr2=0x7f4928152eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:54.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.834+0000 7f4926d9d700 1 --2- 172.21.15.78:0/1308499800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49280b3a90 0x7f4928152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:54.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.834+0000 7f492659c700 1 --2- 172.21.15.78:0/1308499800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49280b3a90 0x7f4928152eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:54.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.834+0000 7f492659c700 1 -- 172.21.15.78:0/1308499800 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49100094b0 con 0x7f49280b3240 2023-10-11T09:40:54.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.834+0000 7f4926d9d700 1 --2- 172.21.15.78:0/1308499800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49280b3a90 0x7f4928152eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:40:54.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.834+0000 7f492659c700 1 --2- 172.21.15.78:0/1308499800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49280b3240 0x7f49281563c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4918001f30 tx=0x7f4918002010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:54.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.834+0000 7f491f7fe700 1 -- 172.21.15.78:0/1308499800 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491800a9e0 con 0x7f49280b3240 2023-10-11T09:40:54.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.834+0000 7f491f7fe700 1 -- 172.21.15.78:0/1308499800 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f491800ab40 con 0x7f49280b3240 2023-10-11T09:40:54.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.834+0000 7f492759e700 1 -- 172.21.15.78:0/1308499800 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49281570f0 con 0x7f49280b3240 2023-10-11T09:40:54.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.834+0000 7f491f7fe700 1 -- 172.21.15.78:0/1308499800 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4918012400 con 0x7f49280b3240 2023-10-11T09:40:54.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.838+0000 7f492759e700 1 -- 172.21.15.78:0/1308499800 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49281576b0 con 0x7f49280b3240 2023-10-11T09:40:54.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.838+0000 7f492759e700 1 -- 172.21.15.78:0/1308499800 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49281578d0 con 0x7f49280b3240 2023-10-11T09:40:54.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.838+0000 7f491f7fe700 1 -- 172.21.15.78:0/1308499800 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f491801b3b0 con 0x7f49280b3240 2023-10-11T09:40:54.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.842+0000 7f491f7fe700 1 --2- 172.21.15.78:0/1308499800 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4908065ed0 0x7f4908068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:54.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.842+0000 7f491f7fe700 1 -- 172.21.15.78:0/1308499800 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4918083400 con 0x7f49280b3240 2023-10-11T09:40:54.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.842+0000 7f491dd9b700 1 --2- 172.21.15.78:0/1308499800 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4908065ed0 0x7f4908068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:54.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.842+0000 7f491dd9b700 1 --2- 172.21.15.78:0/1308499800 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4908065ed0 0x7f4908068380 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f49100090b0 tx=0x7f491001c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:54.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:54.846+0000 7f491f7fe700 1 -- 172.21.15.78:0/1308499800 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f4918050950 con 0x7f49280b3240 2023-10-11T09:40:55.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.018+0000 7f492759e700 1 -- 172.21.15.78:0/1308499800 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f49280b7ab0 con 0x7f49280b3240 2023-10-11T09:40:55.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.018+0000 7f491f7fe700 1 -- 172.21.15.78:0/1308499800 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4918053f70 con 0x7f49280b3240 2023-10-11T09:40:55.023 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:40:55.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.022+0000 7f492759e700 1 -- 172.21.15.78:0/1308499800 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4908065ed0 msgr2=0x7f4908068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:55.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.022+0000 7f492759e700 1 --2- 172.21.15.78:0/1308499800 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4908065ed0 0x7f4908068380 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f49100090b0 tx=0x7f491001c040 comp rx=0 tx=0).stop 2023-10-11T09:40:55.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.022+0000 7f492759e700 1 -- 172.21.15.78:0/1308499800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49280b3240 msgr2=0x7f49281563c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:55.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.022+0000 7f492759e700 1 --2- 172.21.15.78:0/1308499800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49280b3240 0x7f49281563c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4918001f30 tx=0x7f4918002010 comp rx=0 tx=0).stop 2023-10-11T09:40:55.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.022+0000 7f492759e700 1 -- 172.21.15.78:0/1308499800 shutdown_connections 2023-10-11T09:40:55.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.022+0000 7f492759e700 1 --2- 172.21.15.78:0/1308499800 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4908065ed0 0x7f4908068380 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:55.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.022+0000 7f492759e700 1 --2- 172.21.15.78:0/1308499800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49280b3a90 0x7f4928152eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:55.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.022+0000 7f492759e700 1 --2- 172.21.15.78:0/1308499800 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f49280b3610 0x7f49281522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:55.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.022+0000 7f492759e700 1 --2- 172.21.15.78:0/1308499800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49280b3240 0x7f49281563c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:55.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.022+0000 7f492759e700 1 -- 172.21.15.78:0/1308499800 >> 172.21.15.78:0/1308499800 conn(0x7f492809c260 msgr2=0x7f492815b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:55.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.022+0000 7f492759e700 1 -- 172.21.15.78:0/1308499800 shutdown_connections 2023-10-11T09:40:55.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.022+0000 7f492759e700 1 -- 172.21.15.78:0/1308499800 wait complete. 2023-10-11T09:40:55.044 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T09:40:55.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.218+0000 7f67b7513700 1 Processor -- start 2023-10-11T09:40:55.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.218+0000 7f67b7513700 1 -- start start 2023-10-11T09:40:55.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.218+0000 7f67b7513700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f67b00b84c0 0x7f67b00cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:55.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.218+0000 7f67b7513700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f67b00b8890 0x7f67b0160370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:55.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.222+0000 7f67b7513700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67b00cd690 con 0x7f67b01608b0 2023-10-11T09:40:55.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.222+0000 7f67b7513700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67b00cdea0 con 0x7f67b00b84c0 2023-10-11T09:40:55.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.222+0000 7f67b7513700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67b00d0c10 con 0x7f67b00b8890 2023-10-11T09:40:55.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.222+0000 7f67b52af700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f67b00b84c0 0x7f67b00cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:55.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.222+0000 7f67b52af700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f67b00b84c0 0x7f67b00cfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56606/0 (socket says 172.21.15.78:56606) 2023-10-11T09:40:55.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.222+0000 7f67b5ab0700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f67b01608b0 0x7f67b00d2fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:34572/0 (socket says 172.21.15.78:34572) 2023-10-11T09:40:55.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.222+0000 7f67b52af700 1 -- 172.21.15.78:0/1969488367 learned_addr learned my addr 172.21.15.78:0/1969488367 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:55.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.222+0000 7f67b4aae700 1 --2- 172.21.15.78:0/1969488367 >> v2:172.21.15.133:3300/0 conn(0x7f67b00b8890 0x7f67b0160370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:55.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.222+0000 7f67b52af700 1 -- 172.21.15.78:0/1969488367 >> v2:172.21.15.133:3300/0 conn(0x7f67b00b8890 msgr2=0x7f67b0160370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:55.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.222+0000 7f67b52af700 1 --2- 172.21.15.78:0/1969488367 >> v2:172.21.15.133:3300/0 conn(0x7f67b00b8890 0x7f67b0160370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:55.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.222+0000 7f67b52af700 1 -- 172.21.15.78:0/1969488367 >> v1:172.21.15.78:6789/0 conn(0x7f67b01608b0 legacy=0x7f67b00d2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:55.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.222+0000 7f67b52af700 1 -- 172.21.15.78:0/1969488367 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b00d02e0 con 0x7f67b00b84c0 2023-10-11T09:40:55.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.222+0000 7f67b4aae700 1 --2- 172.21.15.78:0/1969488367 >> v2:172.21.15.133:3300/0 conn(0x7f67b00b8890 0x7f67b0160370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:40:55.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.222+0000 7f67b52af700 1 --2- 172.21.15.78:0/1969488367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67b00b84c0 0x7f67b00cfda0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f67b008bfc0 tx=0x7f67ac009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15d6e1444563835b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:55.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.222+0000 7f67a7fff700 1 -- 172.21.15.78:0/1969488367 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac00b8a0 con 0x7f67b00b84c0 2023-10-11T09:40:55.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.222+0000 7f67a7fff700 1 -- 172.21.15.78:0/1969488367 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67ac00ba00 con 0x7f67b00b84c0 2023-10-11T09:40:55.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.222+0000 7f67a7fff700 1 -- 172.21.15.78:0/1969488367 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac00bd90 con 0x7f67b00b84c0 2023-10-11T09:40:55.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.222+0000 7f67b7513700 1 -- 172.21.15.78:0/1969488367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67b00b84c0 msgr2=0x7f67b00cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:55.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.222+0000 7f67b7513700 1 --2- 172.21.15.78:0/1969488367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67b00b84c0 0x7f67b00cfda0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f67b008bfc0 tx=0x7f67ac009bf0 comp rx=0 tx=0).stop 2023-10-11T09:40:55.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.222+0000 7f67b7513700 1 -- 172.21.15.78:0/1969488367 shutdown_connections 2023-10-11T09:40:55.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.222+0000 7f67b7513700 1 --2- 172.21.15.78:0/1969488367 >> v2:172.21.15.133:3300/0 conn(0x7f67b00b8890 0x7f67b0160370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:55.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.222+0000 7f67b7513700 1 --2- 172.21.15.78:0/1969488367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67b00b84c0 0x7f67b00cfda0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:55.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.222+0000 7f67b7513700 1 -- 172.21.15.78:0/1969488367 >> 172.21.15.78:0/1969488367 conn(0x7f67b006cb50 msgr2=0x7f67b006cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:55.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.222+0000 7f67b7513700 1 -- 172.21.15.78:0/1969488367 shutdown_connections 2023-10-11T09:40:55.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.222+0000 7f67b7513700 1 -- 172.21.15.78:0/1969488367 wait complete. 2023-10-11T09:40:55.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.226+0000 7f67b7513700 1 Processor -- start 2023-10-11T09:40:55.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.226+0000 7f67b7513700 1 -- start start 2023-10-11T09:40:55.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.230+0000 7f67b7513700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67b00b84c0 0x7f67b01f1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:55.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.230+0000 7f67b7513700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67b00b8890 0x7f67b00c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:55.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.230+0000 7f67b7513700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67b01608b0 0x7f67b00c9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:55.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.230+0000 7f67b7513700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b00c8250 con 0x7f67b00b8890 2023-10-11T09:40:55.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.230+0000 7f67b7513700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b00c8530 con 0x7f67b01608b0 2023-10-11T09:40:55.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.230+0000 7f67b7513700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b014bf50 con 0x7f67b00b84c0 2023-10-11T09:40:55.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.230+0000 7f67b4aae700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67b00b8890 0x7f67b00c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:55.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.230+0000 7f67b4aae700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67b00b8890 0x7f67b00c8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56614/0 (socket says 172.21.15.78:56614) 2023-10-11T09:40:55.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.230+0000 7f67b4aae700 1 -- 172.21.15.78:0/3966228838 learned_addr learned my addr 172.21.15.78:0/3966228838 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:55.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.230+0000 7f67b52af700 1 --2- 172.21.15.78:0/3966228838 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67b00b84c0 0x7f67b01f1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:55.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.230+0000 7f67b5ab0700 1 --2- 172.21.15.78:0/3966228838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67b01608b0 0x7f67b00c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:55.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.230+0000 7f67b4aae700 1 -- 172.21.15.78:0/3966228838 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67b00b84c0 msgr2=0x7f67b01f1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:55.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.230+0000 7f67b4aae700 1 --2- 172.21.15.78:0/3966228838 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67b00b84c0 0x7f67b01f1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:55.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.230+0000 7f67b4aae700 1 -- 172.21.15.78:0/3966228838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67b01608b0 msgr2=0x7f67b00c9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:55.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.230+0000 7f67b4aae700 1 --2- 172.21.15.78:0/3966228838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67b01608b0 0x7f67b00c9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:55.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.230+0000 7f67b4aae700 1 -- 172.21.15.78:0/3966228838 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ac009570 con 0x7f67b00b8890 2023-10-11T09:40:55.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.230+0000 7f67b52af700 1 --2- 172.21.15.78:0/3966228838 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67b00b84c0 0x7f67b01f1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:40:55.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.230+0000 7f67b4aae700 1 --2- 172.21.15.78:0/3966228838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67b00b8890 0x7f67b00c8c10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f67a000e880 tx=0x7f67a000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:55.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.230+0000 7f67a5ffb700 1 -- 172.21.15.78:0/3966228838 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a000c780 con 0x7f67b00b8890 2023-10-11T09:40:55.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.230+0000 7f67a5ffb700 1 -- 172.21.15.78:0/3966228838 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a000c8e0 con 0x7f67b00b8890 2023-10-11T09:40:55.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.230+0000 7f67b7513700 1 -- 172.21.15.78:0/3966228838 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67b014c230 con 0x7f67b00b8890 2023-10-11T09:40:55.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.230+0000 7f67a5ffb700 1 -- 172.21.15.78:0/3966228838 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a000cc70 con 0x7f67b00b8890 2023-10-11T09:40:55.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.230+0000 7f67b7513700 1 -- 172.21.15.78:0/3966228838 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b01f1950 con 0x7f67b00b8890 2023-10-11T09:40:55.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.234+0000 7f67b7513700 1 -- 172.21.15.78:0/3966228838 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67b008bff0 con 0x7f67b00b8890 2023-10-11T09:40:55.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.234+0000 7f67a5ffb700 1 -- 172.21.15.78:0/3966228838 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f67a0013410 con 0x7f67b00b8890 2023-10-11T09:40:55.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.234+0000 7f67a5ffb700 1 --2- 172.21.15.78:0/3966228838 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6794065ed0 0x7f6794068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:55.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.234+0000 7f67a5ffb700 1 -- 172.21.15.78:0/3966228838 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f67a0083510 con 0x7f67b00b8890 2023-10-11T09:40:55.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.238+0000 7f67b52af700 1 --2- 172.21.15.78:0/3966228838 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6794065ed0 0x7f6794068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:55.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.238+0000 7f67b52af700 1 --2- 172.21.15.78:0/3966228838 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6794065ed0 0x7f6794068380 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f67ac000c00 tx=0x7f67ac008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:55.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.242+0000 7f67a5ffb700 1 -- 172.21.15.78:0/3966228838 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f67a0050ad0 con 0x7f67b00b8890 2023-10-11T09:40:55.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.418+0000 7f67b7513700 1 -- 172.21.15.78:0/3966228838 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f67b0068070 con 0x7f67b00b8890 2023-10-11T09:40:55.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.418+0000 7f67a5ffb700 1 -- 172.21.15.78:0/3966228838 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f67a00540f0 con 0x7f67b00b8890 2023-10-11T09:40:55.426 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:40:55.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.422+0000 7f67b7513700 1 -- 172.21.15.78:0/3966228838 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6794065ed0 msgr2=0x7f6794068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:55.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.422+0000 7f67b7513700 1 --2- 172.21.15.78:0/3966228838 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6794065ed0 0x7f6794068380 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f67ac000c00 tx=0x7f67ac008030 comp rx=0 tx=0).stop 2023-10-11T09:40:55.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.422+0000 7f67b7513700 1 -- 172.21.15.78:0/3966228838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67b00b8890 msgr2=0x7f67b00c8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:55.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.422+0000 7f67b7513700 1 --2- 172.21.15.78:0/3966228838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67b00b8890 0x7f67b00c8c10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f67a000e880 tx=0x7f67a000ec40 comp rx=0 tx=0).stop 2023-10-11T09:40:55.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.422+0000 7f67b7513700 1 -- 172.21.15.78:0/3966228838 shutdown_connections 2023-10-11T09:40:55.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.422+0000 7f67b7513700 1 --2- 172.21.15.78:0/3966228838 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6794065ed0 0x7f6794068380 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:55.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.422+0000 7f67b7513700 1 --2- 172.21.15.78:0/3966228838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67b01608b0 0x7f67b00c9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:55.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.422+0000 7f67b7513700 1 --2- 172.21.15.78:0/3966228838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67b00b8890 0x7f67b00c8c10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:55.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.422+0000 7f67b7513700 1 --2- 172.21.15.78:0/3966228838 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67b00b84c0 0x7f67b01f1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:55.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.422+0000 7f67b7513700 1 -- 172.21.15.78:0/3966228838 >> 172.21.15.78:0/3966228838 conn(0x7f67b006cb50 msgr2=0x7f67b00b2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:55.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.422+0000 7f67b7513700 1 -- 172.21.15.78:0/3966228838 shutdown_connections 2023-10-11T09:40:55.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.422+0000 7f67b7513700 1 -- 172.21.15.78:0/3966228838 wait complete. 2023-10-11T09:40:55.447 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T09:40:55.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.614+0000 7f851eb51700 1 Processor -- start 2023-10-11T09:40:55.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.614+0000 7f851eb51700 1 -- start start 2023-10-11T09:40:55.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.614+0000 7f851eb51700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8518146fe0 0x7f851806bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:55.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.614+0000 7f851eb51700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f85180b36f0 0x7f85181629e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:55.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.614+0000 7f851eb51700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f851806ccb0 con 0x7f8518161d90 2023-10-11T09:40:55.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.614+0000 7f851eb51700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f851806c410 con 0x7f85180b36f0 2023-10-11T09:40:55.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.614+0000 7f851eb51700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f851806c6f0 con 0x7f8518146fe0 2023-10-11T09:40:55.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.618+0000 7f851e350700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f85180b36f0 0x7f85181629e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:55.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.618+0000 7f851e350700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f85180b36f0 0x7f85181629e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56626/0 (socket says 172.21.15.78:56626) 2023-10-11T09:40:55.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.618+0000 7f851e350700 1 -- 172.21.15.78:0/2446503545 learned_addr learned my addr 172.21.15.78:0/2446503545 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:55.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.618+0000 7f851db4f700 1 --2- 172.21.15.78:0/2446503545 >> v2:172.21.15.133:3300/0 conn(0x7f8518146fe0 0x7f851806bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:55.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.618+0000 7f851e350700 1 -- 172.21.15.78:0/2446503545 >> v2:172.21.15.133:3300/0 conn(0x7f8518146fe0 msgr2=0x7f851806bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:55.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.618+0000 7f851e350700 1 --2- 172.21.15.78:0/2446503545 >> v2:172.21.15.133:3300/0 conn(0x7f8518146fe0 0x7f851806bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:55.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.618+0000 7f851e350700 1 -- 172.21.15.78:0/2446503545 >> v1:172.21.15.133:6789/0 conn(0x7f8518161d90 legacy=0x7f85180b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:55.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.618+0000 7f851e350700 1 -- 172.21.15.78:0/2446503545 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851806c9d0 con 0x7f85180b36f0 2023-10-11T09:40:55.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.618+0000 7f851db4f700 1 --2- 172.21.15.78:0/2446503545 >> v2:172.21.15.133:3300/0 conn(0x7f8518146fe0 0x7f851806bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:40:55.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.618+0000 7f851e350700 1 --2- 172.21.15.78:0/2446503545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85180b36f0 0x7f85181629e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f851806f7f0 tx=0x7f851000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3869a7fb5f80fe5d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:55.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.618+0000 7f851cb4d700 1 -- 172.21.15.78:0/2446503545 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851000e070 con 0x7f85180b36f0 2023-10-11T09:40:55.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.618+0000 7f851cb4d700 1 -- 172.21.15.78:0/2446503545 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8510003a40 con 0x7f85180b36f0 2023-10-11T09:40:55.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.618+0000 7f851cb4d700 1 -- 172.21.15.78:0/2446503545 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8510003df0 con 0x7f85180b36f0 2023-10-11T09:40:55.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.618+0000 7f851eb51700 1 -- 172.21.15.78:0/2446503545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85180b36f0 msgr2=0x7f85181629e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:55.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.618+0000 7f851eb51700 1 --2- 172.21.15.78:0/2446503545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85180b36f0 0x7f85181629e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f851806f7f0 tx=0x7f851000b330 comp rx=0 tx=0).stop 2023-10-11T09:40:55.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.618+0000 7f851eb51700 1 -- 172.21.15.78:0/2446503545 shutdown_connections 2023-10-11T09:40:55.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.618+0000 7f851eb51700 1 --2- 172.21.15.78:0/2446503545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85180b36f0 0x7f85181629e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:55.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.618+0000 7f851eb51700 1 --2- 172.21.15.78:0/2446503545 >> v2:172.21.15.133:3300/0 conn(0x7f8518146fe0 0x7f851806bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:55.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.618+0000 7f851eb51700 1 -- 172.21.15.78:0/2446503545 >> 172.21.15.78:0/2446503545 conn(0x7f851806a060 msgr2=0x7f851806a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:55.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.618+0000 7f851eb51700 1 -- 172.21.15.78:0/2446503545 shutdown_connections 2023-10-11T09:40:55.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.618+0000 7f851eb51700 1 -- 172.21.15.78:0/2446503545 wait complete. 2023-10-11T09:40:55.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.622+0000 7f851eb51700 1 Processor -- start 2023-10-11T09:40:55.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.622+0000 7f851eb51700 1 -- start start 2023-10-11T09:40:55.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.626+0000 7f851eb51700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f85180b36f0 0x7f85180a84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:55.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.626+0000 7f851eb51700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8518146fe0 0x7f851815f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:55.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.626+0000 7f851eb51700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8518161d90 0x7f85180adeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:55.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.626+0000 7f851eb51700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85180ae3f0 con 0x7f8518161d90 2023-10-11T09:40:55.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.626+0000 7f851eb51700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85180acb70 con 0x7f8518146fe0 2023-10-11T09:40:55.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.626+0000 7f851eb51700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85180ace10 con 0x7f85180b36f0 2023-10-11T09:40:55.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.626+0000 7f851e350700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8518161d90 0x7f85180adeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:55.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.626+0000 7f851e350700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8518161d90 0x7f85180adeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56628/0 (socket says 172.21.15.78:56628) 2023-10-11T09:40:55.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.626+0000 7f851e350700 1 -- 172.21.15.78:0/1419607501 learned_addr learned my addr 172.21.15.78:0/1419607501 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:55.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.626+0000 7f851d34e700 1 --2- 172.21.15.78:0/1419607501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8518146fe0 0x7f851815f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:55.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.626+0000 7f851db4f700 1 --2- 172.21.15.78:0/1419607501 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f85180b36f0 0x7f85180a84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:55.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.626+0000 7f851e350700 1 -- 172.21.15.78:0/1419607501 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f85180b36f0 msgr2=0x7f85180a84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:55.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.626+0000 7f851e350700 1 --2- 172.21.15.78:0/1419607501 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f85180b36f0 0x7f85180a84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:55.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.626+0000 7f851e350700 1 -- 172.21.15.78:0/1419607501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8518146fe0 msgr2=0x7f851815f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:55.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.626+0000 7f851e350700 1 --2- 172.21.15.78:0/1419607501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8518146fe0 0x7f851815f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:55.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.626+0000 7f851e350700 1 -- 172.21.15.78:0/1419607501 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8510009b30 con 0x7f8518161d90 2023-10-11T09:40:55.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.626+0000 7f851e350700 1 --2- 172.21.15.78:0/1419607501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8518161d90 0x7f85180adeb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f851000b370 tx=0x7f8510004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:55.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.626+0000 7f850e7fc700 1 -- 172.21.15.78:0/1419607501 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8510004440 con 0x7f8518161d90 2023-10-11T09:40:55.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.626+0000 7f850e7fc700 1 -- 172.21.15.78:0/1419607501 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85100045c0 con 0x7f8518161d90 2023-10-11T09:40:55.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.626+0000 7f851eb51700 1 -- 172.21.15.78:0/1419607501 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85180ad0f0 con 0x7f8518161d90 2023-10-11T09:40:55.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.626+0000 7f850e7fc700 1 -- 172.21.15.78:0/1419607501 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8510004970 con 0x7f8518161d90 2023-10-11T09:40:55.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.626+0000 7f851eb51700 1 -- 172.21.15.78:0/1419607501 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85180ad6b0 con 0x7f8518161d90 2023-10-11T09:40:55.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.630+0000 7f851eb51700 1 -- 172.21.15.78:0/1419607501 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8518065d40 con 0x7f8518161d90 2023-10-11T09:40:55.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.630+0000 7f850e7fc700 1 -- 172.21.15.78:0/1419607501 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f8510012430 con 0x7f8518161d90 2023-10-11T09:40:55.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.630+0000 7f850e7fc700 1 --2- 172.21.15.78:0/1419607501 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f84fc065ed0 0x7f84fc068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:55.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.630+0000 7f850e7fc700 1 -- 172.21.15.78:0/1419607501 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8510084940 con 0x7f8518161d90 2023-10-11T09:40:55.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.630+0000 7f851db4f700 1 --2- 172.21.15.78:0/1419607501 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f84fc065ed0 0x7f84fc068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:55.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.634+0000 7f851db4f700 1 --2- 172.21.15.78:0/1419607501 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f84fc065ed0 0x7f84fc068380 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f85140044b0 tx=0x7f8514008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:55.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.638+0000 7f850e7fc700 1 -- 172.21.15.78:0/1419607501 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f8510055a10 con 0x7f8518161d90 2023-10-11T09:40:55.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.810+0000 7f851eb51700 1 -- 172.21.15.78:0/1419607501 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f851806b570 con 0x7f8518161d90 2023-10-11T09:40:55.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.810+0000 7f850e7fc700 1 -- 172.21.15.78:0/1419607501 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f85100170a0 con 0x7f8518161d90 2023-10-11T09:40:55.816 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2023-10-11T09:40:55.819 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.814+0000 7f851eb51700 1 -- 172.21.15.78:0/1419607501 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f84fc065ed0 msgr2=0x7f84fc068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:55.819 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.814+0000 7f851eb51700 1 --2- 172.21.15.78:0/1419607501 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f84fc065ed0 0x7f84fc068380 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f85140044b0 tx=0x7f8514008040 comp rx=0 tx=0).stop 2023-10-11T09:40:55.819 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.814+0000 7f851eb51700 1 -- 172.21.15.78:0/1419607501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8518161d90 msgr2=0x7f85180adeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:55.819 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.814+0000 7f851eb51700 1 --2- 172.21.15.78:0/1419607501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8518161d90 0x7f85180adeb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f851000b370 tx=0x7f8510004af0 comp rx=0 tx=0).stop 2023-10-11T09:40:55.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.814+0000 7f851eb51700 1 -- 172.21.15.78:0/1419607501 shutdown_connections 2023-10-11T09:40:55.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.814+0000 7f851eb51700 1 --2- 172.21.15.78:0/1419607501 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f84fc065ed0 0x7f84fc068380 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:55.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.814+0000 7f851eb51700 1 --2- 172.21.15.78:0/1419607501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8518161d90 0x7f85180adeb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:55.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.814+0000 7f851eb51700 1 --2- 172.21.15.78:0/1419607501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8518146fe0 0x7f851815f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:55.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.814+0000 7f851eb51700 1 --2- 172.21.15.78:0/1419607501 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f85180b36f0 0x7f85180a84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:55.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.814+0000 7f851eb51700 1 -- 172.21.15.78:0/1419607501 >> 172.21.15.78:0/1419607501 conn(0x7f851806a060 msgr2=0x7f8518070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:55.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.814+0000 7f851eb51700 1 -- 172.21.15.78:0/1419607501 shutdown_connections 2023-10-11T09:40:55.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:55.814+0000 7f851eb51700 1 -- 172.21.15.78:0/1419607501 wait complete. 2023-10-11T09:40:55.839 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:40:56.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.002+0000 7f7b32e82700 1 Processor -- start 2023-10-11T09:40:56.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.002+0000 7f7b32e82700 1 -- start start 2023-10-11T09:40:56.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.006+0000 7f7b32e82700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f7b2c0ce500 0x7f7b2c15d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:56.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.006+0000 7f7b32e82700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b2c159c50 con 0x7f7b2c070100 2023-10-11T09:40:56.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.006+0000 7f7b32e82700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b2c159ef0 con 0x7f7b2c159490 2023-10-11T09:40:56.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.006+0000 7f7b32e82700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b2c15a1d0 con 0x7f7b2c0ce500 2023-10-11T09:40:56.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.006+0000 7f7b2bfff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f7b2c0ce500 0x7f7b2c15d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:56.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.006+0000 7f7b2bfff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f7b2c0ce500 0x7f7b2c15d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56634/0 (socket says 172.21.15.78:56634) 2023-10-11T09:40:56.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.006+0000 7f7b2bfff700 1 -- 172.21.15.78:0/1163737287 learned_addr learned my addr 172.21.15.78:0/1163737287 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:56.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.006+0000 7f7b2bfff700 1 -- 172.21.15.78:0/1163737287 >> v1:172.21.15.133:6789/0 conn(0x7f7b2c159490 legacy=0x7f7b2c15d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:56.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.006+0000 7f7b2bfff700 1 -- 172.21.15.78:0/1163737287 >> v1:172.21.15.78:6789/0 conn(0x7f7b2c070100 legacy=0x7f7b2c070560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:56.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.006+0000 7f7b2bfff700 1 -- 172.21.15.78:0/1163737287 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b2c15a4b0 con 0x7f7b2c0ce500 2023-10-11T09:40:56.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.010+0000 7f7b2bfff700 1 --2- 172.21.15.78:0/1163737287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b2c0ce500 0x7f7b2c15d740 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7b18009910 tx=0x7f7b18009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e056ab55353454 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:56.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.010+0000 7f7b2b7fe700 1 -- 172.21.15.78:0/1163737287 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1800ba60 con 0x7f7b2c0ce500 2023-10-11T09:40:56.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.010+0000 7f7b2b7fe700 1 -- 172.21.15.78:0/1163737287 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b1800bbc0 con 0x7f7b2c0ce500 2023-10-11T09:40:56.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.010+0000 7f7b2b7fe700 1 -- 172.21.15.78:0/1163737287 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b18009e60 con 0x7f7b2c0ce500 2023-10-11T09:40:56.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.010+0000 7f7b32e82700 1 -- 172.21.15.78:0/1163737287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b2c0ce500 msgr2=0x7f7b2c15d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:56.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.010+0000 7f7b32e82700 1 --2- 172.21.15.78:0/1163737287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b2c0ce500 0x7f7b2c15d740 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7b18009910 tx=0x7f7b18009c20 comp rx=0 tx=0).stop 2023-10-11T09:40:56.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.010+0000 7f7b32e82700 1 -- 172.21.15.78:0/1163737287 shutdown_connections 2023-10-11T09:40:56.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.010+0000 7f7b32e82700 1 --2- 172.21.15.78:0/1163737287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b2c0ce500 0x7f7b2c15d740 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:56.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.010+0000 7f7b32e82700 1 -- 172.21.15.78:0/1163737287 >> 172.21.15.78:0/1163737287 conn(0x7f7b2c06aca0 msgr2=0x7f7b2c0b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:56.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.010+0000 7f7b32e82700 1 -- 172.21.15.78:0/1163737287 shutdown_connections 2023-10-11T09:40:56.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.010+0000 7f7b32e82700 1 -- 172.21.15.78:0/1163737287 wait complete. 2023-10-11T09:40:56.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.014+0000 7f7b32e82700 1 Processor -- start 2023-10-11T09:40:56.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.014+0000 7f7b32e82700 1 -- start start 2023-10-11T09:40:56.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.014+0000 7f7b32e82700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b2c070100 0x7f7b2c151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:56.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.014+0000 7f7b32e82700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b2c0ce500 0x7f7b2c1f6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:56.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.014+0000 7f7b32e82700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7b2c159490 0x7f7b2c06c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:56.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.014+0000 7f7b32e82700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2c152be0 con 0x7f7b2c0ce500 2023-10-11T09:40:56.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.014+0000 7f7b32e82700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2c1f7330 con 0x7f7b2c070100 2023-10-11T09:40:56.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.014+0000 7f7b32e82700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2c152060 con 0x7f7b2c159490 2023-10-11T09:40:56.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.014+0000 7f7b2bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b2c0ce500 0x7f7b2c1f6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:56.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.014+0000 7f7b2bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b2c0ce500 0x7f7b2c1f6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56646/0 (socket says 172.21.15.78:56646) 2023-10-11T09:40:56.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.014+0000 7f7b2bfff700 1 -- 172.21.15.78:0/344406826 learned_addr learned my addr 172.21.15.78:0/344406826 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:56.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.014+0000 7f7b30c1e700 1 --2- 172.21.15.78:0/344406826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b2c070100 0x7f7b2c151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:56.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.014+0000 7f7b3141f700 1 --2- 172.21.15.78:0/344406826 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7b2c159490 0x7f7b2c06c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:56.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.014+0000 7f7b2bfff700 1 -- 172.21.15.78:0/344406826 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7b2c159490 msgr2=0x7f7b2c06c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:56.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.014+0000 7f7b2bfff700 1 --2- 172.21.15.78:0/344406826 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7b2c159490 0x7f7b2c06c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:56.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.014+0000 7f7b2bfff700 1 -- 172.21.15.78:0/344406826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b2c070100 msgr2=0x7f7b2c151a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:56.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.014+0000 7f7b2bfff700 1 --2- 172.21.15.78:0/344406826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b2c070100 0x7f7b2c151a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:56.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.014+0000 7f7b2bfff700 1 -- 172.21.15.78:0/344406826 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b18009570 con 0x7f7b2c0ce500 2023-10-11T09:40:56.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.014+0000 7f7b3141f700 1 --2- 172.21.15.78:0/344406826 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7b2c159490 0x7f7b2c06c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:40:56.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.014+0000 7f7b30c1e700 1 --2- 172.21.15.78:0/344406826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b2c070100 0x7f7b2c151a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:40:56.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.014+0000 7f7b2bfff700 1 --2- 172.21.15.78:0/344406826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b2c0ce500 0x7f7b2c1f6df0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7b18009c60 tx=0x7f7b18004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:56.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.018+0000 7f7b297fa700 1 -- 172.21.15.78:0/344406826 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b18003ad0 con 0x7f7b2c0ce500 2023-10-11T09:40:56.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.018+0000 7f7b297fa700 1 -- 172.21.15.78:0/344406826 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b18003c50 con 0x7f7b2c0ce500 2023-10-11T09:40:56.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.018+0000 7f7b32e82700 1 -- 172.21.15.78:0/344406826 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b2c152340 con 0x7f7b2c0ce500 2023-10-11T09:40:56.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.018+0000 7f7b297fa700 1 -- 172.21.15.78:0/344406826 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b18010470 con 0x7f7b2c0ce500 2023-10-11T09:40:56.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.018+0000 7f7b32e82700 1 -- 172.21.15.78:0/344406826 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b2c152900 con 0x7f7b2c0ce500 2023-10-11T09:40:56.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.018+0000 7f7b32e82700 1 -- 172.21.15.78:0/344406826 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b2c04e530 con 0x7f7b2c0ce500 2023-10-11T09:40:56.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.022+0000 7f7b297fa700 1 -- 172.21.15.78:0/344406826 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f7b1802c440 con 0x7f7b2c0ce500 2023-10-11T09:40:56.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.022+0000 7f7b297fa700 1 --2- 172.21.15.78:0/344406826 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7b10065f00 0x7f7b100683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:56.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.022+0000 7f7b297fa700 1 -- 172.21.15.78:0/344406826 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7b18084ec0 con 0x7f7b2c0ce500 2023-10-11T09:40:56.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.022+0000 7f7b30c1e700 1 --2- 172.21.15.78:0/344406826 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7b10065f00 0x7f7b100683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:56.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.022+0000 7f7b30c1e700 1 --2- 172.21.15.78:0/344406826 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7b10065f00 0x7f7b100683b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7b20000f60 tx=0x7f7b20009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:56.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.026+0000 7f7b297fa700 1 -- 172.21.15.78:0/344406826 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f7b18052500 con 0x7f7b2c0ce500 2023-10-11T09:40:56.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.234+0000 7f7b32e82700 1 -- 172.21.15.78:0/344406826 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b2c071c80 con 0x7f7b2c0ce500 2023-10-11T09:40:56.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.234+0000 7f7b297fa700 1 -- 172.21.15.78:0/344406826 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4648 (secure 0 0 0) 0x7f7b18055b20 con 0x7f7b2c0ce500 2023-10-11T09:40:56.238 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:40:56.238 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/4282378510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4282378510},{"type":"v1","addr":"172.21.15.133:6835","nonce":4282378510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3986631249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3986631249},{"type":"v1","addr":"172.21.15.78:6835","nonce":3986631249}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3135421902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3135421902},{"type":"v1","addr":"172.21.15.78:6837","nonce":3135421902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:40:41.673556+0000","modified":"2023-10-11T09:40:45.122852+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6834/3925157967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":3925157967},{"type":"v1","addr":"172.21.15.133:6834","nonce":3925157967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-11T09:40:56.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.234+0000 7f7b32e82700 1 -- 172.21.15.78:0/344406826 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7b10065f00 msgr2=0x7f7b100683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:56.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.234+0000 7f7b32e82700 1 --2- 172.21.15.78:0/344406826 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7b10065f00 0x7f7b100683b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7b20000f60 tx=0x7f7b20009040 comp rx=0 tx=0).stop 2023-10-11T09:40:56.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.234+0000 7f7b32e82700 1 -- 172.21.15.78:0/344406826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b2c0ce500 msgr2=0x7f7b2c1f6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:56.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.234+0000 7f7b32e82700 1 --2- 172.21.15.78:0/344406826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b2c0ce500 0x7f7b2c1f6df0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7b18009c60 tx=0x7f7b18004770 comp rx=0 tx=0).stop 2023-10-11T09:40:56.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.238+0000 7f7b32e82700 1 -- 172.21.15.78:0/344406826 shutdown_connections 2023-10-11T09:40:56.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.238+0000 7f7b32e82700 1 --2- 172.21.15.78:0/344406826 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7b10065f00 0x7f7b100683b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:56.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.238+0000 7f7b32e82700 1 --2- 172.21.15.78:0/344406826 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7b2c159490 0x7f7b2c06c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:56.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.238+0000 7f7b32e82700 1 --2- 172.21.15.78:0/344406826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b2c0ce500 0x7f7b2c1f6df0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:56.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.238+0000 7f7b32e82700 1 --2- 172.21.15.78:0/344406826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b2c070100 0x7f7b2c151a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:56.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.238+0000 7f7b32e82700 1 -- 172.21.15.78:0/344406826 >> 172.21.15.78:0/344406826 conn(0x7f7b2c06aca0 msgr2=0x7f7b2c0cddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:56.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.238+0000 7f7b32e82700 1 -- 172.21.15.78:0/344406826 shutdown_connections 2023-10-11T09:40:56.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.238+0000 7f7b32e82700 1 -- 172.21.15.78:0/344406826 wait complete. 2023-10-11T09:40:56.244 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 8 2023-10-11T09:40:56.265 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:40:56.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.430+0000 7ff4b5397700 1 Processor -- start 2023-10-11T09:40:56.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.434+0000 7ff4b5397700 1 -- start start 2023-10-11T09:40:56.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.434+0000 7ff4b5397700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4b015b8e0 0x7ff4b006d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:56.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.434+0000 7ff4b5397700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ff4b006d610 0x7ff4b00ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:56.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.434+0000 7ff4b5397700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff4b015a940 0x7ff4b00ca780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:56.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.434+0000 7ff4b5397700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b015e010 con 0x7ff4b006d610 2023-10-11T09:40:56.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.434+0000 7ff4b5397700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b015e280 con 0x7ff4b015a940 2023-10-11T09:40:56.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.434+0000 7ff4b5397700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b015e560 con 0x7ff4b015b8e0 2023-10-11T09:40:56.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.434+0000 7ff4af7fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ff4b006d610 0x7ff4b00ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:56.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.434+0000 7ff4af7fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ff4b006d610 0x7ff4b00ca240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56650/0 (socket says 172.21.15.78:56650) 2023-10-11T09:40:56.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.434+0000 7ff4af7fe700 1 -- 172.21.15.78:0/3225327973 learned_addr learned my addr 172.21.15.78:0/3225327973 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:56.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.434+0000 7ff4affff700 1 --2- 172.21.15.78:0/3225327973 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4b015b8e0 0x7ff4b006d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:56.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.434+0000 7ff4b4b96700 1 --2- 172.21.15.78:0/3225327973 >> v2:172.21.15.133:3300/0 conn(0x7ff4b015a940 0x7ff4b00ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:56.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.434+0000 7ff4af7fe700 1 -- 172.21.15.78:0/3225327973 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4b015b8e0 msgr2=0x7ff4b006d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:56.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.434+0000 7ff4af7fe700 1 --2- 172.21.15.78:0/3225327973 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4b015b8e0 0x7ff4b006d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:56.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.434+0000 7ff4af7fe700 1 -- 172.21.15.78:0/3225327973 >> v2:172.21.15.133:3300/0 conn(0x7ff4b015a940 msgr2=0x7ff4b00ca780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:56.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.434+0000 7ff4af7fe700 1 --2- 172.21.15.78:0/3225327973 >> v2:172.21.15.133:3300/0 conn(0x7ff4b015a940 0x7ff4b00ca780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:56.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.434+0000 7ff4af7fe700 1 -- 172.21.15.78:0/3225327973 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b015e840 con 0x7ff4b006d610 2023-10-11T09:40:56.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.438+0000 7ff4af7fe700 1 --2- 172.21.15.78:0/3225327973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4b006d610 0x7ff4b00ca240 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff4b00cfab0 tx=0x7ff4a4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f047b93e28a2ee7 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:56.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.438+0000 7ff4aeffd700 1 -- 172.21.15.78:0/3225327973 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a400b8a0 con 0x7ff4b006d610 2023-10-11T09:40:56.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.438+0000 7ff4aeffd700 1 -- 172.21.15.78:0/3225327973 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a400ba00 con 0x7ff4b006d610 2023-10-11T09:40:56.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.438+0000 7ff4aeffd700 1 -- 172.21.15.78:0/3225327973 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a400bd90 con 0x7ff4b006d610 2023-10-11T09:40:56.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.438+0000 7ff4b5397700 1 -- 172.21.15.78:0/3225327973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4b006d610 msgr2=0x7ff4b00ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:56.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.438+0000 7ff4b5397700 1 --2- 172.21.15.78:0/3225327973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4b006d610 0x7ff4b00ca240 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff4b00cfab0 tx=0x7ff4a4009bf0 comp rx=0 tx=0).stop 2023-10-11T09:40:56.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.442+0000 7ff4b5397700 1 -- 172.21.15.78:0/3225327973 shutdown_connections 2023-10-11T09:40:56.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.442+0000 7ff4b5397700 1 --2- 172.21.15.78:0/3225327973 >> v2:172.21.15.133:3300/0 conn(0x7ff4b015a940 0x7ff4b00ca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:56.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.442+0000 7ff4b5397700 1 --2- 172.21.15.78:0/3225327973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4b006d610 0x7ff4b00ca240 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:56.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.442+0000 7ff4b5397700 1 --2- 172.21.15.78:0/3225327973 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4b015b8e0 0x7ff4b006d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:56.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.442+0000 7ff4b5397700 1 -- 172.21.15.78:0/3225327973 >> 172.21.15.78:0/3225327973 conn(0x7ff4b00cea60 msgr2=0x7ff4b00cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:56.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.442+0000 7ff4b5397700 1 -- 172.21.15.78:0/3225327973 shutdown_connections 2023-10-11T09:40:56.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.442+0000 7ff4b5397700 1 -- 172.21.15.78:0/3225327973 wait complete. 2023-10-11T09:40:56.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.442+0000 7ff4b5397700 1 Processor -- start 2023-10-11T09:40:56.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.442+0000 7ff4b5397700 1 -- start start 2023-10-11T09:40:56.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.446+0000 7ff4b5397700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4b006d610 0x7ff4b00b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:56.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.446+0000 7ff4b5397700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4b015a940 0x7ff4b013c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:56.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.446+0000 7ff4b5397700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4b015b8e0 0x7ff4b013d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:56.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.446+0000 7ff4b5397700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b013c010 con 0x7ff4b015b8e0 2023-10-11T09:40:56.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.446+0000 7ff4b5397700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b013b400 con 0x7ff4b015a940 2023-10-11T09:40:56.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.446+0000 7ff4b5397700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b013b6e0 con 0x7ff4b006d610 2023-10-11T09:40:56.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.446+0000 7ff4b4b96700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4b015b8e0 0x7ff4b013d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:56.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.446+0000 7ff4af7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4b015a940 0x7ff4b013c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:56.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.446+0000 7ff4b4b96700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4b015b8e0 0x7ff4b013d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56654/0 (socket says 172.21.15.78:56654) 2023-10-11T09:40:56.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.446+0000 7ff4af7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4b015a940 0x7ff4b013c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:36964/0 (socket says 172.21.15.78:36964) 2023-10-11T09:40:56.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.446+0000 7ff4b4b96700 1 -- 172.21.15.78:0/2005924060 learned_addr learned my addr 172.21.15.78:0/2005924060 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:56.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.446+0000 7ff4affff700 1 --2- 172.21.15.78:0/2005924060 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4b006d610 0x7ff4b00b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:56.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.446+0000 7ff4b4b96700 1 -- 172.21.15.78:0/2005924060 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4b006d610 msgr2=0x7ff4b00b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:56.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.446+0000 7ff4b4b96700 1 --2- 172.21.15.78:0/2005924060 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4b006d610 0x7ff4b00b4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:56.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.446+0000 7ff4b4b96700 1 -- 172.21.15.78:0/2005924060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4b015a940 msgr2=0x7ff4b013c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:56.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.446+0000 7ff4b4b96700 1 --2- 172.21.15.78:0/2005924060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4b015a940 0x7ff4b013c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:56.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.446+0000 7ff4b4b96700 1 -- 172.21.15.78:0/2005924060 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a4009570 con 0x7ff4b015b8e0 2023-10-11T09:40:56.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.446+0000 7ff4b4b96700 1 --2- 172.21.15.78:0/2005924060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4b015b8e0 0x7ff4b013d170 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8007e40 tx=0x7ff4a800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:56.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.446+0000 7ff4acff9700 1 -- 172.21.15.78:0/2005924060 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a800ed30 con 0x7ff4b015b8e0 2023-10-11T09:40:56.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.446+0000 7ff4acff9700 1 -- 172.21.15.78:0/2005924060 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a8014420 con 0x7ff4b015b8e0 2023-10-11T09:40:56.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.446+0000 7ff4acff9700 1 -- 172.21.15.78:0/2005924060 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a80147d0 con 0x7ff4b015b8e0 2023-10-11T09:40:56.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.446+0000 7ff4b5397700 1 -- 172.21.15.78:0/2005924060 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b013b9c0 con 0x7ff4b015b8e0 2023-10-11T09:40:56.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.446+0000 7ff4b5397700 1 -- 172.21.15.78:0/2005924060 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b00b48b0 con 0x7ff4b015b8e0 2023-10-11T09:40:56.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.450+0000 7ff4b5397700 1 -- 172.21.15.78:0/2005924060 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b00b4a40 con 0x7ff4b015b8e0 2023-10-11T09:40:56.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.450+0000 7ff4acff9700 1 -- 172.21.15.78:0/2005924060 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7ff4a8014930 con 0x7ff4b015b8e0 2023-10-11T09:40:56.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.450+0000 7ff4acff9700 1 --2- 172.21.15.78:0/2005924060 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff49c065ed0 0x7ff49c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:56.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.450+0000 7ff4acff9700 1 -- 172.21.15.78:0/2005924060 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff4a8083f90 con 0x7ff4b015b8e0 2023-10-11T09:40:56.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.454+0000 7ff4affff700 1 --2- 172.21.15.78:0/2005924060 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff49c065ed0 0x7ff49c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:56.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.454+0000 7ff4affff700 1 --2- 172.21.15.78:0/2005924060 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff49c065ed0 0x7ff49c068380 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0006b40 tx=0x7ff4a0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:56.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.458+0000 7ff4acff9700 1 -- 172.21.15.78:0/2005924060 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7ff4a8051540 con 0x7ff4b015b8e0 2023-10-11T09:40:56.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.634+0000 7ff4b5397700 1 -- 172.21.15.78:0/2005924060 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4b0065d40 con 0x7ff4b015b8e0 2023-10-11T09:40:56.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.638+0000 7ff4acff9700 1 -- 172.21.15.78:0/2005924060 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17811 (secure 0 0 0) 0x7ff4a8054b60 con 0x7ff4b015b8e0 2023-10-11T09:40:56.641 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:40:56.641 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":23,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:40:42.678477+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:40:39.862581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T09:40:40.663641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:40:56.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.642+0000 7ff4b5397700 1 -- 172.21.15.78:0/2005924060 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff49c065ed0 msgr2=0x7ff49c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:56.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.642+0000 7ff4b5397700 1 --2- 172.21.15.78:0/2005924060 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff49c065ed0 0x7ff49c068380 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0006b40 tx=0x7ff4a0008040 comp rx=0 tx=0).stop 2023-10-11T09:40:56.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.642+0000 7ff4b5397700 1 -- 172.21.15.78:0/2005924060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4b015b8e0 msgr2=0x7ff4b013d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:56.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.642+0000 7ff4b5397700 1 --2- 172.21.15.78:0/2005924060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4b015b8e0 0x7ff4b013d170 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8007e40 tx=0x7ff4a800e3c0 comp rx=0 tx=0).stop 2023-10-11T09:40:56.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.642+0000 7ff4b5397700 1 -- 172.21.15.78:0/2005924060 shutdown_connections 2023-10-11T09:40:56.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.642+0000 7ff4b5397700 1 --2- 172.21.15.78:0/2005924060 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff49c065ed0 0x7ff49c068380 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:56.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.642+0000 7ff4b5397700 1 --2- 172.21.15.78:0/2005924060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4b015b8e0 0x7ff4b013d170 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:56.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.642+0000 7ff4b5397700 1 --2- 172.21.15.78:0/2005924060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4b015a940 0x7ff4b013c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:56.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.642+0000 7ff4b5397700 1 --2- 172.21.15.78:0/2005924060 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4b006d610 0x7ff4b00b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:56.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.642+0000 7ff4b5397700 1 -- 172.21.15.78:0/2005924060 >> 172.21.15.78:0/2005924060 conn(0x7ff4b00cea60 msgr2=0x7ff4b0071810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:56.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.642+0000 7ff4b5397700 1 -- 172.21.15.78:0/2005924060 shutdown_connections 2023-10-11T09:40:56.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.642+0000 7ff4b5397700 1 -- 172.21.15.78:0/2005924060 wait complete. 2023-10-11T09:40:56.669 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T09:40:39.862581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T09:40:40.663641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:40:56.670 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:40:56.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.850+0000 7fc99ede6700 1 Processor -- start 2023-10-11T09:40:56.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.850+0000 7fc99ede6700 1 -- start start 2023-10-11T09:40:56.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.850+0000 7fc99ede6700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc99806d340 0x7fc9980ca880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:56.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.850+0000 7fc99ede6700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc99815e280 con 0x7fc998158430 2023-10-11T09:40:56.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.850+0000 7fc99ede6700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc99815e560 con 0x7fc9980cadc0 2023-10-11T09:40:56.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.850+0000 7fc99ede6700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc998070c10 con 0x7fc99806d340 2023-10-11T09:40:56.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.850+0000 7fc99cb82700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fc998158430 0x7fc9980710d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:34584/0 (socket says 172.21.15.78:34584) 2023-10-11T09:40:56.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.850+0000 7fc99cb82700 1 -- 172.21.15.78:0/2220461392 learned_addr learned my addr 172.21.15.78:0/2220461392 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:56.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.850+0000 7fc997fff700 1 --2- 172.21.15.78:0/2220461392 >> v2:172.21.15.133:3300/0 conn(0x7fc99806d340 0x7fc9980ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:56.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.854+0000 7fc9977fe700 1 -- 172.21.15.78:0/2220461392 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1835309995 0 0) 0x7fc99815e280 con 0x7fc998158430 2023-10-11T09:40:56.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.854+0000 7fc9977fe700 1 -- 172.21.15.78:0/2220461392 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9880035d0 con 0x7fc998158430 2023-10-11T09:40:56.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.854+0000 7fc997fff700 1 -- 172.21.15.78:0/2220461392 >> v1:172.21.15.133:6789/0 conn(0x7fc9980cadc0 legacy=0x7fc99815afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:56.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.854+0000 7fc997fff700 1 -- 172.21.15.78:0/2220461392 >> v1:172.21.15.78:6789/0 conn(0x7fc998158430 legacy=0x7fc9980710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:56.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.854+0000 7fc997fff700 1 -- 172.21.15.78:0/2220461392 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99815e840 con 0x7fc99806d340 2023-10-11T09:40:56.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.854+0000 7fc997fff700 1 --2- 172.21.15.78:0/2220461392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc99806d340 0x7fc9980ca880 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc984009850 tx=0x7fc984009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=743077a9634012ec server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:56.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.854+0000 7fc9977fe700 1 -- 172.21.15.78:0/2220461392 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc984011830 con 0x7fc99806d340 2023-10-11T09:40:56.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.854+0000 7fc9977fe700 1 -- 172.21.15.78:0/2220461392 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc984011990 con 0x7fc99806d340 2023-10-11T09:40:56.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.854+0000 7fc9977fe700 1 -- 172.21.15.78:0/2220461392 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc984011d40 con 0x7fc99806d340 2023-10-11T09:40:56.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.854+0000 7fc99ede6700 1 -- 172.21.15.78:0/2220461392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc99806d340 msgr2=0x7fc9980ca880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:56.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.854+0000 7fc99ede6700 1 --2- 172.21.15.78:0/2220461392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc99806d340 0x7fc9980ca880 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc984009850 tx=0x7fc984009b60 comp rx=0 tx=0).stop 2023-10-11T09:40:56.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.854+0000 7fc99ede6700 1 -- 172.21.15.78:0/2220461392 shutdown_connections 2023-10-11T09:40:56.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.854+0000 7fc99ede6700 1 --2- 172.21.15.78:0/2220461392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc99806d340 0x7fc9980ca880 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:56.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.854+0000 7fc99ede6700 1 -- 172.21.15.78:0/2220461392 >> 172.21.15.78:0/2220461392 conn(0x7fc9980cecc0 msgr2=0x7fc9980cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:56.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.854+0000 7fc99ede6700 1 -- 172.21.15.78:0/2220461392 shutdown_connections 2023-10-11T09:40:56.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.854+0000 7fc99ede6700 1 -- 172.21.15.78:0/2220461392 wait complete. 2023-10-11T09:40:56.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.858+0000 7fc99ede6700 1 Processor -- start 2023-10-11T09:40:56.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.858+0000 7fc99ede6700 1 -- start start 2023-10-11T09:40:56.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.858+0000 7fc99ede6700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc99806d340 0x7fc9981f3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:56.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.858+0000 7fc99ede6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9980cadc0 0x7fc9981f2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:56.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.858+0000 7fc99ede6700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc998158430 0x7fc9981f4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:56.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.858+0000 7fc99ede6700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9980cc800 con 0x7fc9980cadc0 2023-10-11T09:40:56.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.858+0000 7fc99ede6700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9981f3280 con 0x7fc99806d340 2023-10-11T09:40:56.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.858+0000 7fc99ede6700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9980cca50 con 0x7fc998158430 2023-10-11T09:40:56.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.858+0000 7fc997fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9980cadc0 0x7fc9981f2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:56.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.858+0000 7fc997fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9980cadc0 0x7fc9981f2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56656/0 (socket says 172.21.15.78:56656) 2023-10-11T09:40:56.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.858+0000 7fc997fff700 1 -- 172.21.15.78:0/2126352504 learned_addr learned my addr 172.21.15.78:0/2126352504 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:56.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.858+0000 7fc99d383700 1 --2- 172.21.15.78:0/2126352504 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc998158430 0x7fc9981f4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:56.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.858+0000 7fc99cb82700 1 --2- 172.21.15.78:0/2126352504 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc99806d340 0x7fc9981f3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:56.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.862+0000 7fc997fff700 1 -- 172.21.15.78:0/2126352504 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc998158430 msgr2=0x7fc9981f4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:56.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.862+0000 7fc997fff700 1 --2- 172.21.15.78:0/2126352504 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc998158430 0x7fc9981f4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:56.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.862+0000 7fc997fff700 1 -- 172.21.15.78:0/2126352504 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc99806d340 msgr2=0x7fc9981f3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:56.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.862+0000 7fc997fff700 1 --2- 172.21.15.78:0/2126352504 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc99806d340 0x7fc9981f3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:56.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.862+0000 7fc997fff700 1 -- 172.21.15.78:0/2126352504 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9840094b0 con 0x7fc9980cadc0 2023-10-11T09:40:56.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.862+0000 7fc997fff700 1 --2- 172.21.15.78:0/2126352504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9980cadc0 0x7fc9981f2d40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc984000c00 tx=0x7fc984004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:56.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.862+0000 7fc9957fa700 1 -- 172.21.15.78:0/2126352504 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98401a990 con 0x7fc9980cadc0 2023-10-11T09:40:56.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.862+0000 7fc9957fa700 1 -- 172.21.15.78:0/2126352504 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc98401f070 con 0x7fc9980cadc0 2023-10-11T09:40:56.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.862+0000 7fc99ede6700 1 -- 172.21.15.78:0/2126352504 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9980ccd30 con 0x7fc9980cadc0 2023-10-11T09:40:56.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.862+0000 7fc9957fa700 1 -- 172.21.15.78:0/2126352504 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98401ad30 con 0x7fc9980cadc0 2023-10-11T09:40:56.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.862+0000 7fc99ede6700 1 -- 172.21.15.78:0/2126352504 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9980cd2f0 con 0x7fc9980cadc0 2023-10-11T09:40:56.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.862+0000 7fc99ede6700 1 -- 172.21.15.78:0/2126352504 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc99806eb80 con 0x7fc9980cadc0 2023-10-11T09:40:56.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.862+0000 7fc9957fa700 1 -- 172.21.15.78:0/2126352504 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fc98401b430 con 0x7fc9980cadc0 2023-10-11T09:40:56.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.866+0000 7fc9957fa700 1 --2- 172.21.15.78:0/2126352504 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc97c065ed0 0x7fc97c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:56.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.866+0000 7fc9957fa700 1 -- 172.21.15.78:0/2126352504 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc984084810 con 0x7fc9980cadc0 2023-10-11T09:40:56.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.866+0000 7fc99cb82700 1 --2- 172.21.15.78:0/2126352504 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc97c065ed0 0x7fc97c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:56.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.866+0000 7fc99cb82700 1 --2- 172.21.15.78:0/2126352504 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc97c065ed0 0x7fc97c068380 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc98c007fd0 tx=0x7fc98c007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:56.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:56.874+0000 7fc9957fa700 1 -- 172.21.15.78:0/2126352504 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fc984051e50 con 0x7fc9980cadc0 2023-10-11T09:40:57.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.046+0000 7fc99ede6700 1 -- 172.21.15.78:0/2126352504 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc99815eaf0 con 0x7fc9980cadc0 2023-10-11T09:40:57.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.046+0000 7fc9957fa700 1 -- 172.21.15.78:0/2126352504 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc984015020 con 0x7fc9980cadc0 2023-10-11T09:40:57.051 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:40:57.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.050+0000 7fc99ede6700 1 -- 172.21.15.78:0/2126352504 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc97c065ed0 msgr2=0x7fc97c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:57.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.050+0000 7fc99ede6700 1 --2- 172.21.15.78:0/2126352504 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc97c065ed0 0x7fc97c068380 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc98c007fd0 tx=0x7fc98c007de0 comp rx=0 tx=0).stop 2023-10-11T09:40:57.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.050+0000 7fc99ede6700 1 -- 172.21.15.78:0/2126352504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9980cadc0 msgr2=0x7fc9981f2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:57.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.050+0000 7fc99ede6700 1 --2- 172.21.15.78:0/2126352504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9980cadc0 0x7fc9981f2d40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc984000c00 tx=0x7fc984004770 comp rx=0 tx=0).stop 2023-10-11T09:40:57.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.050+0000 7fc99ede6700 1 -- 172.21.15.78:0/2126352504 shutdown_connections 2023-10-11T09:40:57.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.050+0000 7fc99ede6700 1 --2- 172.21.15.78:0/2126352504 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc97c065ed0 0x7fc97c068380 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:57.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.050+0000 7fc99ede6700 1 --2- 172.21.15.78:0/2126352504 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc998158430 0x7fc9981f4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:57.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.050+0000 7fc99ede6700 1 --2- 172.21.15.78:0/2126352504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9980cadc0 0x7fc9981f2d40 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:57.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.050+0000 7fc99ede6700 1 --2- 172.21.15.78:0/2126352504 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc99806d340 0x7fc9981f3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:57.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.050+0000 7fc99ede6700 1 -- 172.21.15.78:0/2126352504 >> 172.21.15.78:0/2126352504 conn(0x7fc9980cecc0 msgr2=0x7fc9980cc030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:57.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.050+0000 7fc99ede6700 1 -- 172.21.15.78:0/2126352504 shutdown_connections 2023-10-11T09:40:57.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.050+0000 7fc99ede6700 1 -- 172.21.15.78:0/2126352504 wait complete. 2023-10-11T09:40:57.077 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:40:57.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.242+0000 7fa5eb59e700 1 Processor -- start 2023-10-11T09:40:57.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.246+0000 7fa5eb59e700 1 -- start start 2023-10-11T09:40:57.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.246+0000 7fa5eb59e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa5ec0b3240 0x7fa5ec0d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:57.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.246+0000 7fa5eb59e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fa5ec0b3610 0x7fa5ec0b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:57.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.246+0000 7fa5eb59e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5ec0b47f0 con 0x7fa5ec15d040 2023-10-11T09:40:57.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.246+0000 7fa5eb59e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5ec0b3fd0 con 0x7fa5ec0b3610 2023-10-11T09:40:57.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.246+0000 7fa5eb59e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5ec0b8130 con 0x7fa5ec0b3240 2023-10-11T09:40:57.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.246+0000 7fa5ea59c700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa5ec0b3240 0x7fa5ec0d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:57.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.250+0000 7fa5ea59c700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa5ec0b3240 0x7fa5ec0d29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:36990/0 (socket says 172.21.15.78:36990) 2023-10-11T09:40:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.250+0000 7fa5ea59c700 1 -- 172.21.15.78:0/2287592749 learned_addr learned my addr 172.21.15.78:0/2287592749 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.250+0000 7fa5e9d9b700 1 --2- 172.21.15.78:0/2287592749 >> v2:172.21.15.78:3300/0 conn(0x7fa5ec0b3610 0x7fa5ec0b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.250+0000 7fa5ea59c700 1 -- 172.21.15.78:0/2287592749 >> v2:172.21.15.78:3300/0 conn(0x7fa5ec0b3610 msgr2=0x7fa5ec0b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.250+0000 7fa5ea59c700 1 --2- 172.21.15.78:0/2287592749 >> v2:172.21.15.78:3300/0 conn(0x7fa5ec0b3610 0x7fa5ec0b3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:57.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.250+0000 7fa5ea59c700 1 -- 172.21.15.78:0/2287592749 >> v1:172.21.15.133:6789/0 conn(0x7fa5ec15d040 legacy=0x7fa5ec0b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:57.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.250+0000 7fa5ea59c700 1 -- 172.21.15.78:0/2287592749 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5ec0b41e0 con 0x7fa5ec0b3240 2023-10-11T09:40:57.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.250+0000 7fa5ea59c700 1 --2- 172.21.15.78:0/2287592749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5ec0b3240 0x7fa5ec0d29d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec06aa40 tx=0x7fa5dc009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b735fb8f956bae1 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:57.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.250+0000 7fa5e959a700 1 -- 172.21.15.78:0/2287592749 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc00b8a0 con 0x7fa5ec0b3240 2023-10-11T09:40:57.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.250+0000 7fa5e959a700 1 -- 172.21.15.78:0/2287592749 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5dc00ba00 con 0x7fa5ec0b3240 2023-10-11T09:40:57.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.250+0000 7fa5e959a700 1 -- 172.21.15.78:0/2287592749 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc00bd90 con 0x7fa5ec0b3240 2023-10-11T09:40:57.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.250+0000 7fa5eb59e700 1 -- 172.21.15.78:0/2287592749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5ec0b3240 msgr2=0x7fa5ec0d29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:57.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.250+0000 7fa5eb59e700 1 --2- 172.21.15.78:0/2287592749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5ec0b3240 0x7fa5ec0d29d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec06aa40 tx=0x7fa5dc009bf0 comp rx=0 tx=0).stop 2023-10-11T09:40:57.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.250+0000 7fa5eb59e700 1 -- 172.21.15.78:0/2287592749 shutdown_connections 2023-10-11T09:40:57.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.250+0000 7fa5eb59e700 1 --2- 172.21.15.78:0/2287592749 >> v2:172.21.15.78:3300/0 conn(0x7fa5ec0b3610 0x7fa5ec0b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:57.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.250+0000 7fa5eb59e700 1 --2- 172.21.15.78:0/2287592749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5ec0b3240 0x7fa5ec0d29d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:57.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.250+0000 7fa5eb59e700 1 -- 172.21.15.78:0/2287592749 >> 172.21.15.78:0/2287592749 conn(0x7fa5ec09c260 msgr2=0x7fa5ec0d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:57.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.250+0000 7fa5eb59e700 1 -- 172.21.15.78:0/2287592749 shutdown_connections 2023-10-11T09:40:57.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.250+0000 7fa5eb59e700 1 -- 172.21.15.78:0/2287592749 wait complete. 2023-10-11T09:40:57.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.254+0000 7fa5eb59e700 1 Processor -- start 2023-10-11T09:40:57.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.254+0000 7fa5eb59e700 1 -- start start 2023-10-11T09:40:57.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.254+0000 7fa5eb59e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5ec0b3240 0x7fa5ec1f9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:57.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.254+0000 7fa5eb59e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa5ec0b3610 0x7fa5ec071730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:57.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.254+0000 7fa5eb59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5ec15d040 0x7fa5ec0b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:57.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.254+0000 7fa5eb59e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec1f67e0 con 0x7fa5ec15d040 2023-10-11T09:40:57.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.254+0000 7fa5eb59e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec071c70 con 0x7fa5ec0b3240 2023-10-11T09:40:57.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.254+0000 7fa5eb59e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec1fada0 con 0x7fa5ec0b3610 2023-10-11T09:40:57.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.254+0000 7fa5ead9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5ec15d040 0x7fa5ec0b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:57.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.254+0000 7fa5ea59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5ec0b3240 0x7fa5ec1f9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:57.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.254+0000 7fa5ead9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5ec15d040 0x7fa5ec0b1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56672/0 (socket says 172.21.15.78:56672) 2023-10-11T09:40:57.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.254+0000 7fa5ead9d700 1 -- 172.21.15.78:0/3919344372 learned_addr learned my addr 172.21.15.78:0/3919344372 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:57.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.254+0000 7fa5e9d9b700 1 --2- 172.21.15.78:0/3919344372 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa5ec0b3610 0x7fa5ec071730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:57.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.254+0000 7fa5ea59c700 1 -- 172.21.15.78:0/3919344372 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa5ec0b3610 msgr2=0x7fa5ec071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:57.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.254+0000 7fa5ea59c700 1 --2- 172.21.15.78:0/3919344372 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa5ec0b3610 0x7fa5ec071730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:57.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.254+0000 7fa5ea59c700 1 -- 172.21.15.78:0/3919344372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5ec15d040 msgr2=0x7fa5ec0b1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:57.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.254+0000 7fa5ea59c700 1 --2- 172.21.15.78:0/3919344372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5ec15d040 0x7fa5ec0b1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:57.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.254+0000 7fa5ea59c700 1 -- 172.21.15.78:0/3919344372 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5dc009570 con 0x7fa5ec0b3240 2023-10-11T09:40:57.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.254+0000 7fa5e9d9b700 1 --2- 172.21.15.78:0/3919344372 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa5ec0b3610 0x7fa5ec071730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:40:57.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.254+0000 7fa5ea59c700 1 --2- 172.21.15.78:0/3919344372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5ec0b3240 0x7fa5ec1f9e20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc016000 tx=0x7fa5dc004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:57.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.258+0000 7fa5daffd700 1 -- 172.21.15.78:0/3919344372 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc0038c0 con 0x7fa5ec0b3240 2023-10-11T09:40:57.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.258+0000 7fa5daffd700 1 -- 172.21.15.78:0/3919344372 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5dc01e070 con 0x7fa5ec0b3240 2023-10-11T09:40:57.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.258+0000 7fa5eb59e700 1 -- 172.21.15.78:0/3919344372 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5ec1fa500 con 0x7fa5ec0b3240 2023-10-11T09:40:57.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.258+0000 7fa5daffd700 1 -- 172.21.15.78:0/3919344372 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc003c60 con 0x7fa5ec0b3240 2023-10-11T09:40:57.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.258+0000 7fa5eb59e700 1 -- 172.21.15.78:0/3919344372 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5ec1faac0 con 0x7fa5ec0b3240 2023-10-11T09:40:57.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.258+0000 7fa5eb59e700 1 -- 172.21.15.78:0/3919344372 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5ec065d40 con 0x7fa5ec0b3240 2023-10-11T09:40:57.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.262+0000 7fa5daffd700 1 -- 172.21.15.78:0/3919344372 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fa5dc011420 con 0x7fa5ec0b3240 2023-10-11T09:40:57.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.262+0000 7fa5daffd700 1 --2- 172.21.15.78:0/3919344372 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa5cc065ed0 0x7fa5cc068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:57.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.262+0000 7fa5daffd700 1 -- 172.21.15.78:0/3919344372 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa5dc020070 con 0x7fa5ec0b3240 2023-10-11T09:40:57.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.262+0000 7fa5e9d9b700 1 --2- 172.21.15.78:0/3919344372 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa5cc065ed0 0x7fa5cc068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:57.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.262+0000 7fa5e9d9b700 1 --2- 172.21.15.78:0/3919344372 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa5cc065ed0 0x7fa5cc068380 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec0d0a90 tx=0x7fa5e0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:57.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.270+0000 7fa5daffd700 1 -- 172.21.15.78:0/3919344372 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fa5dc052560 con 0x7fa5ec0b3240 2023-10-11T09:40:57.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.446+0000 7fa5eb59e700 1 -- 172.21.15.78:0/3919344372 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa5ec0b7d90 con 0x7fa5ec0b3240 2023-10-11T09:40:57.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.450+0000 7fa5daffd700 1 -- 172.21.15.78:0/3919344372 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa5dc055b80 con 0x7fa5ec0b3240 2023-10-11T09:40:57.456 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:40:57.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.454+0000 7fa5eb59e700 1 -- 172.21.15.78:0/3919344372 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa5cc065ed0 msgr2=0x7fa5cc068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:57.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.454+0000 7fa5eb59e700 1 --2- 172.21.15.78:0/3919344372 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa5cc065ed0 0x7fa5cc068380 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec0d0a90 tx=0x7fa5e0008040 comp rx=0 tx=0).stop 2023-10-11T09:40:57.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.454+0000 7fa5eb59e700 1 -- 172.21.15.78:0/3919344372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5ec0b3240 msgr2=0x7fa5ec1f9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:57.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.454+0000 7fa5eb59e700 1 --2- 172.21.15.78:0/3919344372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5ec0b3240 0x7fa5ec1f9e20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc016000 tx=0x7fa5dc004770 comp rx=0 tx=0).stop 2023-10-11T09:40:57.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.454+0000 7fa5eb59e700 1 -- 172.21.15.78:0/3919344372 shutdown_connections 2023-10-11T09:40:57.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.454+0000 7fa5eb59e700 1 --2- 172.21.15.78:0/3919344372 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa5cc065ed0 0x7fa5cc068380 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:57.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.454+0000 7fa5eb59e700 1 --2- 172.21.15.78:0/3919344372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5ec15d040 0x7fa5ec0b1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:57.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.454+0000 7fa5eb59e700 1 --2- 172.21.15.78:0/3919344372 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa5ec0b3610 0x7fa5ec071730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:57.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.454+0000 7fa5eb59e700 1 --2- 172.21.15.78:0/3919344372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5ec0b3240 0x7fa5ec1f9e20 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:57.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.454+0000 7fa5eb59e700 1 -- 172.21.15.78:0/3919344372 >> 172.21.15.78:0/3919344372 conn(0x7fa5ec09c260 msgr2=0x7fa5ec0b85c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:57.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.454+0000 7fa5eb59e700 1 -- 172.21.15.78:0/3919344372 shutdown_connections 2023-10-11T09:40:57.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.454+0000 7fa5eb59e700 1 -- 172.21.15.78:0/3919344372 wait complete. 2023-10-11T09:40:57.479 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T09:40:57.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.654+0000 7fa5fa84c700 1 Processor -- start 2023-10-11T09:40:57.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.654+0000 7fa5fa84c700 1 -- start start 2023-10-11T09:40:57.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.654+0000 7fa5fa84c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fa5f40b3610 0x7fa5f40d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:57.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.654+0000 7fa5fa84c700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa5f40b3a90 0x7fa5f40b85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:57.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.654+0000 7fa5fa84c700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5f40b47f0 con 0x7fa5f40b3240 2023-10-11T09:40:57.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.654+0000 7fa5fa84c700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5f40b8130 con 0x7fa5f40b3610 2023-10-11T09:40:57.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.654+0000 7fa5fa84c700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5f40b41e0 con 0x7fa5f40b3a90 2023-10-11T09:40:57.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.658+0000 7fa5f9049700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fa5f40b3610 0x7fa5f40d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:57.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.658+0000 7fa5f984a700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa5f40b3240 0x7fa5f415d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:39400/0 (socket says 172.21.15.78:39400) 2023-10-11T09:40:57.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.658+0000 7fa5f984a700 1 -- 172.21.15.78:0/1031265627 learned_addr learned my addr 172.21.15.78:0/1031265627 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:57.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.658+0000 7fa5fa04b700 1 --2- 172.21.15.78:0/1031265627 >> v2:172.21.15.133:3300/0 conn(0x7fa5f40b3a90 0x7fa5f40b85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:57.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.658+0000 7fa5f9049700 1 -- 172.21.15.78:0/1031265627 >> v2:172.21.15.133:3300/0 conn(0x7fa5f40b3a90 msgr2=0x7fa5f40b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:57.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.658+0000 7fa5f9049700 1 --2- 172.21.15.78:0/1031265627 >> v2:172.21.15.133:3300/0 conn(0x7fa5f40b3a90 0x7fa5f40b85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:57.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.658+0000 7fa5f9049700 1 -- 172.21.15.78:0/1031265627 >> v1:172.21.15.133:6789/0 conn(0x7fa5f40b3240 legacy=0x7fa5f415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:57.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.658+0000 7fa5f9049700 1 -- 172.21.15.78:0/1031265627 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f40b7800 con 0x7fa5f40b3610 2023-10-11T09:40:57.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.658+0000 7fa5f8848700 1 -- 172.21.15.78:0/1031265627 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1600083111 0 0) 0x7fa5f0001110 con 0x7fa5f40b3240 2023-10-11T09:40:57.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.658+0000 7fa5f9049700 1 --2- 172.21.15.78:0/1031265627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5f40b3610 0x7fa5f40d29d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4072b40 tx=0x7fa5e4009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20151cf193ea9684 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:57.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.658+0000 7fa5f8848700 1 -- 172.21.15.78:0/1031265627 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4011830 con 0x7fa5f40b3610 2023-10-11T09:40:57.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.658+0000 7fa5f8848700 1 -- 172.21.15.78:0/1031265627 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e4011990 con 0x7fa5f40b3610 2023-10-11T09:40:57.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.658+0000 7fa5f8848700 1 -- 172.21.15.78:0/1031265627 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4011d20 con 0x7fa5f40b3610 2023-10-11T09:40:57.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.658+0000 7fa5fa84c700 1 -- 172.21.15.78:0/1031265627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5f40b3610 msgr2=0x7fa5f40d29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:57.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.658+0000 7fa5fa84c700 1 --2- 172.21.15.78:0/1031265627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5f40b3610 0x7fa5f40d29d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4072b40 tx=0x7fa5e4009b30 comp rx=0 tx=0).stop 2023-10-11T09:40:57.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.658+0000 7fa5fa84c700 1 -- 172.21.15.78:0/1031265627 shutdown_connections 2023-10-11T09:40:57.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.658+0000 7fa5fa84c700 1 --2- 172.21.15.78:0/1031265627 >> v2:172.21.15.133:3300/0 conn(0x7fa5f40b3a90 0x7fa5f40b85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:57.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.658+0000 7fa5fa84c700 1 --2- 172.21.15.78:0/1031265627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5f40b3610 0x7fa5f40d29d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:57.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.658+0000 7fa5fa84c700 1 -- 172.21.15.78:0/1031265627 >> 172.21.15.78:0/1031265627 conn(0x7fa5f409c260 msgr2=0x7fa5f40d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:57.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.658+0000 7fa5fa84c700 1 -- 172.21.15.78:0/1031265627 shutdown_connections 2023-10-11T09:40:57.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.658+0000 7fa5fa84c700 1 -- 172.21.15.78:0/1031265627 wait complete. 2023-10-11T09:40:57.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.662+0000 7fa5fa84c700 1 Processor -- start 2023-10-11T09:40:57.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.662+0000 7fa5fa84c700 1 -- start start 2023-10-11T09:40:57.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.662+0000 7fa5fa84c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5f40b3240 0x7fa5f4158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:57.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.662+0000 7fa5fa84c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5f40b3610 0x7fa5f409d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:57.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.662+0000 7fa5fa84c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa5f40b3a90 0x7fa5f415ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:57.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.662+0000 7fa5fa84c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f409e2c0 con 0x7fa5f40b3240 2023-10-11T09:40:57.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.662+0000 7fa5fa84c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f409e5a0 con 0x7fa5f40b3610 2023-10-11T09:40:57.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.662+0000 7fa5fa84c700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f409e880 con 0x7fa5f40b3a90 2023-10-11T09:40:57.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.662+0000 7fa5f984a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5f40b3240 0x7fa5f4158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:57.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.662+0000 7fa5f984a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5f40b3240 0x7fa5f4158f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56686/0 (socket says 172.21.15.78:56686) 2023-10-11T09:40:57.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.662+0000 7fa5f984a700 1 -- 172.21.15.78:0/2246996781 learned_addr learned my addr 172.21.15.78:0/2246996781 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:57.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.662+0000 7fa5f9049700 1 --2- 172.21.15.78:0/2246996781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5f40b3610 0x7fa5f409d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:57.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.662+0000 7fa5fa04b700 1 --2- 172.21.15.78:0/2246996781 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa5f40b3a90 0x7fa5f415ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:57.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.662+0000 7fa5f984a700 1 -- 172.21.15.78:0/2246996781 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa5f40b3a90 msgr2=0x7fa5f415ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:57.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.662+0000 7fa5f984a700 1 --2- 172.21.15.78:0/2246996781 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa5f40b3a90 0x7fa5f415ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:57.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.662+0000 7fa5f984a700 1 -- 172.21.15.78:0/2246996781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5f40b3610 msgr2=0x7fa5f409d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:57.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.662+0000 7fa5f984a700 1 --2- 172.21.15.78:0/2246996781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5f40b3610 0x7fa5f409d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:57.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.662+0000 7fa5f984a700 1 -- 172.21.15.78:0/2246996781 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e40094b0 con 0x7fa5f40b3240 2023-10-11T09:40:57.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.662+0000 7fa5f9049700 1 --2- 172.21.15.78:0/2246996781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5f40b3610 0x7fa5f409d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:40:57.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.666+0000 7fa5f984a700 1 --2- 172.21.15.78:0/2246996781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5f40b3240 0x7fa5f4158f90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa5f000c4e0 tx=0x7fa5f000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:57.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.666+0000 7fa5ea7fc700 1 -- 172.21.15.78:0/2246996781 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0004ca0 con 0x7fa5f40b3240 2023-10-11T09:40:57.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.666+0000 7fa5ea7fc700 1 -- 172.21.15.78:0/2246996781 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f0004e20 con 0x7fa5f40b3240 2023-10-11T09:40:57.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.666+0000 7fa5fa84c700 1 -- 172.21.15.78:0/2246996781 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5f409eb60 con 0x7fa5f40b3240 2023-10-11T09:40:57.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.666+0000 7fa5ea7fc700 1 -- 172.21.15.78:0/2246996781 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0007310 con 0x7fa5f40b3240 2023-10-11T09:40:57.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.666+0000 7fa5fa84c700 1 -- 172.21.15.78:0/2246996781 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5f41594d0 con 0x7fa5f40b3240 2023-10-11T09:40:57.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.666+0000 7fa5fa84c700 1 -- 172.21.15.78:0/2246996781 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5f4065d40 con 0x7fa5f40b3240 2023-10-11T09:40:57.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.670+0000 7fa5ea7fc700 1 -- 172.21.15.78:0/2246996781 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fa5f0007470 con 0x7fa5f40b3240 2023-10-11T09:40:57.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.670+0000 7fa5ea7fc700 1 --2- 172.21.15.78:0/2246996781 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa5d8065ed0 0x7fa5d8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:57.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.670+0000 7fa5ea7fc700 1 -- 172.21.15.78:0/2246996781 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa5f0083380 con 0x7fa5f40b3240 2023-10-11T09:40:57.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.670+0000 7fa5f9049700 1 --2- 172.21.15.78:0/2246996781 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa5d8065ed0 0x7fa5d8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:57.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.670+0000 7fa5f9049700 1 --2- 172.21.15.78:0/2246996781 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa5d8065ed0 0x7fa5d8068380 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4009fd0 tx=0x7fa5e4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:57.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.678+0000 7fa5ea7fc700 1 -- 172.21.15.78:0/2246996781 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fa5f00508a0 con 0x7fa5f40b3240 2023-10-11T09:40:57.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.850+0000 7fa5fa84c700 1 -- 172.21.15.78:0/2246996781 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa5f40b47f0 con 0x7fa5f40b3240 2023-10-11T09:40:57.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.850+0000 7fa5ea7fc700 1 -- 172.21.15.78:0/2246996781 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa5f0007720 con 0x7fa5f40b3240 2023-10-11T09:40:57.857 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:40:57.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.858+0000 7fa5fa84c700 1 -- 172.21.15.78:0/2246996781 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa5d8065ed0 msgr2=0x7fa5d8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:57.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.858+0000 7fa5fa84c700 1 --2- 172.21.15.78:0/2246996781 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa5d8065ed0 0x7fa5d8068380 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4009fd0 tx=0x7fa5e4008030 comp rx=0 tx=0).stop 2023-10-11T09:40:57.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.858+0000 7fa5fa84c700 1 -- 172.21.15.78:0/2246996781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5f40b3240 msgr2=0x7fa5f4158f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:57.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.858+0000 7fa5fa84c700 1 --2- 172.21.15.78:0/2246996781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5f40b3240 0x7fa5f4158f90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa5f000c4e0 tx=0x7fa5f000c7f0 comp rx=0 tx=0).stop 2023-10-11T09:40:57.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.858+0000 7fa5fa84c700 1 -- 172.21.15.78:0/2246996781 shutdown_connections 2023-10-11T09:40:57.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.858+0000 7fa5fa84c700 1 --2- 172.21.15.78:0/2246996781 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa5d8065ed0 0x7fa5d8068380 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:57.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.858+0000 7fa5fa84c700 1 --2- 172.21.15.78:0/2246996781 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa5f40b3a90 0x7fa5f415ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:57.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.858+0000 7fa5fa84c700 1 --2- 172.21.15.78:0/2246996781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5f40b3610 0x7fa5f409d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:57.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.858+0000 7fa5fa84c700 1 --2- 172.21.15.78:0/2246996781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5f40b3240 0x7fa5f4158f90 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:57.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.858+0000 7fa5fa84c700 1 -- 172.21.15.78:0/2246996781 >> 172.21.15.78:0/2246996781 conn(0x7fa5f409c260 msgr2=0x7fa5f40d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:57.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.858+0000 7fa5fa84c700 1 -- 172.21.15.78:0/2246996781 shutdown_connections 2023-10-11T09:40:57.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:57.858+0000 7fa5fa84c700 1 -- 172.21.15.78:0/2246996781 wait complete. 2023-10-11T09:40:57.881 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T09:40:58.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.050+0000 7f8807fff700 1 Processor -- start 2023-10-11T09:40:58.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.050+0000 7f8807fff700 1 -- start start 2023-10-11T09:40:58.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.054+0000 7f8807fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f880815c900 0x7f88080b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:58.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.054+0000 7f8807fff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88080ad290 0x7f8808071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:58.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.054+0000 7f8807fff700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f880806fde0 con 0x7f88080ce7f0 2023-10-11T09:40:58.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.054+0000 7f8807fff700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88080705f0 con 0x7f880815c900 2023-10-11T09:40:58.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.054+0000 7f8807fff700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8808071fc0 con 0x7f88080ad290 2023-10-11T09:40:58.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.054+0000 7f8806ffd700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f880815c900 0x7f88080b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:58.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.054+0000 7f88077fe700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f88080ce7f0 0x7f880806bfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:34598/0 (socket says 172.21.15.78:34598) 2023-10-11T09:40:58.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.054+0000 7f88077fe700 1 -- 172.21.15.78:0/3886433404 learned_addr learned my addr 172.21.15.78:0/3886433404 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:58.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.054+0000 7f88067fc700 1 --2- 172.21.15.78:0/3886433404 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88080ad290 0x7f8808071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:58.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.054+0000 7f8806ffd700 1 -- 172.21.15.78:0/3886433404 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88080ad290 msgr2=0x7f8808071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:58.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.054+0000 7f8806ffd700 1 --2- 172.21.15.78:0/3886433404 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88080ad290 0x7f8808071a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:58.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.054+0000 7f8806ffd700 1 -- 172.21.15.78:0/3886433404 >> v1:172.21.15.78:6789/0 conn(0x7f88080ce7f0 legacy=0x7f880806bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:58.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.054+0000 7f8806ffd700 1 -- 172.21.15.78:0/3886433404 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88080ae000 con 0x7f880815c900 2023-10-11T09:40:58.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.054+0000 7f88067fc700 1 --2- 172.21.15.78:0/3886433404 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88080ad290 0x7f8808071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:40:58.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.054+0000 7f8806ffd700 1 --2- 172.21.15.78:0/3886433404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f880815c900 0x7f88080b1a10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009880 tx=0x7f87f8009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bf0d1fa8f10a2b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:58.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.054+0000 7f8805ffb700 1 -- 172.21.15.78:0/3886433404 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f8011830 con 0x7f880815c900 2023-10-11T09:40:58.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.054+0000 7f8805ffb700 1 -- 172.21.15.78:0/3886433404 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f8011990 con 0x7f880815c900 2023-10-11T09:40:58.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.054+0000 7f8805ffb700 1 -- 172.21.15.78:0/3886433404 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f8011d20 con 0x7f880815c900 2023-10-11T09:40:58.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.054+0000 7f8807fff700 1 -- 172.21.15.78:0/3886433404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f880815c900 msgr2=0x7f88080b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:58.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.054+0000 7f8807fff700 1 --2- 172.21.15.78:0/3886433404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f880815c900 0x7f88080b1a10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009880 tx=0x7f87f8009b90 comp rx=0 tx=0).stop 2023-10-11T09:40:58.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.058+0000 7f8807fff700 1 -- 172.21.15.78:0/3886433404 shutdown_connections 2023-10-11T09:40:58.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.058+0000 7f8807fff700 1 --2- 172.21.15.78:0/3886433404 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88080ad290 0x7f8808071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:58.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.058+0000 7f8807fff700 1 --2- 172.21.15.78:0/3886433404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f880815c900 0x7f88080b1a10 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:58.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.058+0000 7f8807fff700 1 -- 172.21.15.78:0/3886433404 >> 172.21.15.78:0/3886433404 conn(0x7f880806a060 msgr2=0x7f880806a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:58.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.058+0000 7f8807fff700 1 -- 172.21.15.78:0/3886433404 shutdown_connections 2023-10-11T09:40:58.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.058+0000 7f8807fff700 1 -- 172.21.15.78:0/3886433404 wait complete. 2023-10-11T09:40:58.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.058+0000 7f8807fff700 1 Processor -- start 2023-10-11T09:40:58.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.058+0000 7f8807fff700 1 -- start start 2023-10-11T09:40:58.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.062+0000 7f8807fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88080ad290 0x7f88081e8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:58.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.062+0000 7f8807fff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88080ce7f0 0x7f880814cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:58.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.062+0000 7f8807fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f880815c900 0x7f880814e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:58.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.062+0000 7f8807fff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88081e8e70 con 0x7f880815c900 2023-10-11T09:40:58.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.062+0000 7f8807fff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88081e5170 con 0x7f88080ad290 2023-10-11T09:40:58.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.062+0000 7f8807fff700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88081e5450 con 0x7f88080ce7f0 2023-10-11T09:40:58.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.062+0000 7f88077fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f880815c900 0x7f880814e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:58.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.062+0000 7f88077fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f880815c900 0x7f880814e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56700/0 (socket says 172.21.15.78:56700) 2023-10-11T09:40:58.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.062+0000 7f88077fe700 1 -- 172.21.15.78:0/3808165141 learned_addr learned my addr 172.21.15.78:0/3808165141 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:58.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.062+0000 7f8806ffd700 1 --2- 172.21.15.78:0/3808165141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88080ad290 0x7f88081e8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:58.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.062+0000 7f88067fc700 1 --2- 172.21.15.78:0/3808165141 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88080ce7f0 0x7f880814cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:58.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.062+0000 7f88077fe700 1 -- 172.21.15.78:0/3808165141 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88080ce7f0 msgr2=0x7f880814cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:58.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.062+0000 7f88077fe700 1 --2- 172.21.15.78:0/3808165141 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88080ce7f0 0x7f880814cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:58.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.062+0000 7f88077fe700 1 -- 172.21.15.78:0/3808165141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88080ad290 msgr2=0x7f88081e8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:58.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.062+0000 7f88077fe700 1 --2- 172.21.15.78:0/3808165141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88080ad290 0x7f88081e8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:58.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.062+0000 7f88077fe700 1 -- 172.21.15.78:0/3808165141 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f80094b0 con 0x7f880815c900 2023-10-11T09:40:58.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.062+0000 7f88077fe700 1 --2- 172.21.15.78:0/3808165141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f880815c900 0x7f880814e070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f880000ca80 tx=0x7f880000cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:58.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.062+0000 7f87effff700 1 -- 172.21.15.78:0/3808165141 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8800007480 con 0x7f880815c900 2023-10-11T09:40:58.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.062+0000 7f87effff700 1 -- 172.21.15.78:0/3808165141 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88000075e0 con 0x7f880815c900 2023-10-11T09:40:58.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.062+0000 7f8807fff700 1 -- 172.21.15.78:0/3808165141 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88081e5730 con 0x7f880815c900 2023-10-11T09:40:58.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.062+0000 7f87effff700 1 -- 172.21.15.78:0/3808165141 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8800007950 con 0x7f880815c900 2023-10-11T09:40:58.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.062+0000 7f8807fff700 1 -- 172.21.15.78:0/3808165141 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f880814d190 con 0x7f880815c900 2023-10-11T09:40:58.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.066+0000 7f8807fff700 1 -- 172.21.15.78:0/3808165141 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88081e5c50 con 0x7f880815c900 2023-10-11T09:40:58.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.066+0000 7f87effff700 1 -- 172.21.15.78:0/3808165141 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f880001b3b0 con 0x7f880815c900 2023-10-11T09:40:58.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.066+0000 7f87effff700 1 --2- 172.21.15.78:0/3808165141 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f87f4065ed0 0x7f87f4068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:58.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.070+0000 7f87effff700 1 -- 172.21.15.78:0/3808165141 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8800082fb0 con 0x7f880815c900 2023-10-11T09:40:58.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.070+0000 7f8806ffd700 1 --2- 172.21.15.78:0/3808165141 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f87f4065ed0 0x7f87f4068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:58.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.070+0000 7f8806ffd700 1 --2- 172.21.15.78:0/3808165141 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f87f4065ed0 0x7f87f4068380 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f87f8000c00 tx=0x7f87f8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:58.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.074+0000 7f87effff700 1 -- 172.21.15.78:0/3808165141 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f88081e5c50 con 0x7f880815c900 2023-10-11T09:40:58.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.246+0000 7f8807fff700 1 -- 172.21.15.78:0/3808165141 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f88081e5c50 con 0x7f880815c900 2023-10-11T09:40:58.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.246+0000 7f87effff700 1 -- 172.21.15.78:0/3808165141 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f88081e5c50 con 0x7f880815c900 2023-10-11T09:40:58.250 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2023-10-11T09:40:58.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.250+0000 7f8807fff700 1 -- 172.21.15.78:0/3808165141 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f87f4065ed0 msgr2=0x7f87f4068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:58.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.250+0000 7f8807fff700 1 --2- 172.21.15.78:0/3808165141 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f87f4065ed0 0x7f87f4068380 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f87f8000c00 tx=0x7f87f8008030 comp rx=0 tx=0).stop 2023-10-11T09:40:58.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.250+0000 7f8807fff700 1 -- 172.21.15.78:0/3808165141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f880815c900 msgr2=0x7f880814e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:58.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.250+0000 7f8807fff700 1 --2- 172.21.15.78:0/3808165141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f880815c900 0x7f880814e070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f880000ca80 tx=0x7f880000cab0 comp rx=0 tx=0).stop 2023-10-11T09:40:58.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.250+0000 7f8807fff700 1 -- 172.21.15.78:0/3808165141 shutdown_connections 2023-10-11T09:40:58.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.250+0000 7f8807fff700 1 --2- 172.21.15.78:0/3808165141 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f87f4065ed0 0x7f87f4068380 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:58.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.250+0000 7f8807fff700 1 --2- 172.21.15.78:0/3808165141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f880815c900 0x7f880814e070 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:58.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.250+0000 7f8807fff700 1 --2- 172.21.15.78:0/3808165141 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88080ce7f0 0x7f880814cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:58.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.250+0000 7f8807fff700 1 --2- 172.21.15.78:0/3808165141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88080ad290 0x7f88081e8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:58.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.250+0000 7f8807fff700 1 -- 172.21.15.78:0/3808165141 >> 172.21.15.78:0/3808165141 conn(0x7f880806a060 msgr2=0x7f88080b0480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:58.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.250+0000 7f8807fff700 1 -- 172.21.15.78:0/3808165141 shutdown_connections 2023-10-11T09:40:58.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.250+0000 7f8807fff700 1 -- 172.21.15.78:0/3808165141 wait complete. 2023-10-11T09:40:58.271 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:40:58.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.442+0000 7f1314df9700 1 Processor -- start 2023-10-11T09:40:58.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.442+0000 7f1314df9700 1 -- start start 2023-10-11T09:40:58.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.446+0000 7f1314df9700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f13100ce500 0x7f131015d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:58.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.446+0000 7f1314df9700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1310159c50 con 0x7f1310070100 2023-10-11T09:40:58.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.446+0000 7f1314df9700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1310159ef0 con 0x7f1310159490 2023-10-11T09:40:58.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.446+0000 7f1314df9700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f131015a1d0 con 0x7f13100ce500 2023-10-11T09:40:58.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.446+0000 7f130dd9b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f13100ce500 0x7f131015d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:58.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.446+0000 7f130dd9b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f13100ce500 0x7f131015d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56706/0 (socket says 172.21.15.78:56706) 2023-10-11T09:40:58.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.446+0000 7f130dd9b700 1 -- 172.21.15.78:0/4288390802 learned_addr learned my addr 172.21.15.78:0/4288390802 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:58.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.446+0000 7f130ed9d700 1 --1- 172.21.15.78:0/4288390802 >> v1:172.21.15.78:6789/0 conn(0x7f1310070100 0x7f1310070560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:34608/0 (socket says 172.21.15.78:34608) 2023-10-11T09:40:58.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.446+0000 7f130dd9b700 1 -- 172.21.15.78:0/4288390802 >> v1:172.21.15.133:6789/0 conn(0x7f1310159490 legacy=0x7f131015d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:58.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.446+0000 7f130dd9b700 1 -- 172.21.15.78:0/4288390802 >> v1:172.21.15.78:6789/0 conn(0x7f1310070100 legacy=0x7f1310070560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:58.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.446+0000 7f130dd9b700 1 -- 172.21.15.78:0/4288390802 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f131015a4b0 con 0x7f13100ce500 2023-10-11T09:40:58.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.446+0000 7f130dd9b700 1 --2- 172.21.15.78:0/4288390802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13100ce500 0x7f131015d740 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1304009910 tx=0x7f1304009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f99917677d57ae2 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:58.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.446+0000 7f130d59a700 1 -- 172.21.15.78:0/4288390802 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130400ba60 con 0x7f13100ce500 2023-10-11T09:40:58.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.446+0000 7f130d59a700 1 -- 172.21.15.78:0/4288390802 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f130400bbc0 con 0x7f13100ce500 2023-10-11T09:40:58.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.446+0000 7f130d59a700 1 -- 172.21.15.78:0/4288390802 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1304009e60 con 0x7f13100ce500 2023-10-11T09:40:58.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.446+0000 7f1314df9700 1 -- 172.21.15.78:0/4288390802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13100ce500 msgr2=0x7f131015d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:58.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.446+0000 7f1314df9700 1 --2- 172.21.15.78:0/4288390802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13100ce500 0x7f131015d740 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1304009910 tx=0x7f1304009c20 comp rx=0 tx=0).stop 2023-10-11T09:40:58.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.446+0000 7f1314df9700 1 -- 172.21.15.78:0/4288390802 shutdown_connections 2023-10-11T09:40:58.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.446+0000 7f1314df9700 1 --2- 172.21.15.78:0/4288390802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13100ce500 0x7f131015d740 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:58.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.446+0000 7f1314df9700 1 -- 172.21.15.78:0/4288390802 >> 172.21.15.78:0/4288390802 conn(0x7f131006aca0 msgr2=0x7f13100b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:58.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.446+0000 7f1314df9700 1 -- 172.21.15.78:0/4288390802 shutdown_connections 2023-10-11T09:40:58.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.446+0000 7f1314df9700 1 -- 172.21.15.78:0/4288390802 wait complete. 2023-10-11T09:40:58.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.450+0000 7f1314df9700 1 Processor -- start 2023-10-11T09:40:58.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.450+0000 7f1314df9700 1 -- start start 2023-10-11T09:40:58.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.454+0000 7f1314df9700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1310070100 0x7f1310145c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:58.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.454+0000 7f1314df9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13100ce500 0x7f1310153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:58.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.454+0000 7f1314df9700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1310159490 0x7f13101462e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:58.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.454+0000 7f1314df9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1310156a80 con 0x7f13100ce500 2023-10-11T09:40:58.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.454+0000 7f1314df9700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1310156d60 con 0x7f1310070100 2023-10-11T09:40:58.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.454+0000 7f1314df9700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1310156050 con 0x7f1310159490 2023-10-11T09:40:58.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.454+0000 7f130dd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13100ce500 0x7f1310153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:58.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.454+0000 7f130dd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13100ce500 0x7f1310153500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56720/0 (socket says 172.21.15.78:56720) 2023-10-11T09:40:58.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.454+0000 7f130dd9b700 1 -- 172.21.15.78:0/2956249573 learned_addr learned my addr 172.21.15.78:0/2956249573 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:58.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.454+0000 7f130ed9d700 1 --2- 172.21.15.78:0/2956249573 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1310159490 0x7f13101462e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:58.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.454+0000 7f130e59c700 1 --2- 172.21.15.78:0/2956249573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1310070100 0x7f1310145c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:58.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.454+0000 7f130dd9b700 1 -- 172.21.15.78:0/2956249573 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1310159490 msgr2=0x7f13101462e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:58.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.454+0000 7f130dd9b700 1 --2- 172.21.15.78:0/2956249573 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1310159490 0x7f13101462e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:58.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.454+0000 7f130dd9b700 1 -- 172.21.15.78:0/2956249573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1310070100 msgr2=0x7f1310145c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:58.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.454+0000 7f130dd9b700 1 --2- 172.21.15.78:0/2956249573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1310070100 0x7f1310145c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:58.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.454+0000 7f130dd9b700 1 -- 172.21.15.78:0/2956249573 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1304009570 con 0x7f13100ce500 2023-10-11T09:40:58.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.454+0000 7f130dd9b700 1 --2- 172.21.15.78:0/2956249573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13100ce500 0x7f1310153500 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1304000c00 tx=0x7f1304004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:58.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.454+0000 7f12feffd700 1 -- 172.21.15.78:0/2956249573 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1304004610 con 0x7f13100ce500 2023-10-11T09:40:58.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.454+0000 7f12feffd700 1 -- 172.21.15.78:0/2956249573 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f130401f070 con 0x7f13100ce500 2023-10-11T09:40:58.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.454+0000 7f12feffd700 1 -- 172.21.15.78:0/2956249573 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13040038e0 con 0x7f13100ce500 2023-10-11T09:40:58.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.454+0000 7f1314df9700 1 -- 172.21.15.78:0/2956249573 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1310156330 con 0x7f13100ce500 2023-10-11T09:40:58.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.454+0000 7f1314df9700 1 -- 172.21.15.78:0/2956249573 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1310153a40 con 0x7f13100ce500 2023-10-11T09:40:58.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.454+0000 7f1314df9700 1 -- 172.21.15.78:0/2956249573 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1310153bd0 con 0x7f13100ce500 2023-10-11T09:40:58.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.458+0000 7f12feffd700 1 -- 172.21.15.78:0/2956249573 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f1304003a40 con 0x7f13100ce500 2023-10-11T09:40:58.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.462+0000 7f12feffd700 1 --2- 172.21.15.78:0/2956249573 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f12f0065ed0 0x7f12f0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:58.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.462+0000 7f12feffd700 1 -- 172.21.15.78:0/2956249573 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1304021070 con 0x7f13100ce500 2023-10-11T09:40:58.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.462+0000 7f130e59c700 1 --2- 172.21.15.78:0/2956249573 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f12f0065ed0 0x7f12f0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:58.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.462+0000 7f130e59c700 1 --2- 172.21.15.78:0/2956249573 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f12f0065ed0 0x7f12f0068380 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1300000f60 tx=0x7f1300009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:58.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.466+0000 7f12feffd700 1 -- 172.21.15.78:0/2956249573 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f1304052900 con 0x7f13100ce500 2023-10-11T09:40:58.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.670+0000 7f1314df9700 1 -- 172.21.15.78:0/2956249573 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f131004e530 con 0x7f13100ce500 2023-10-11T09:40:58.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.674+0000 7f12feffd700 1 -- 172.21.15.78:0/2956249573 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4648 (secure 0 0 0) 0x7f1304055f20 con 0x7f13100ce500 2023-10-11T09:40:58.678 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:40:58.678 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/4282378510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4282378510},{"type":"v1","addr":"172.21.15.133:6835","nonce":4282378510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3986631249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3986631249},{"type":"v1","addr":"172.21.15.78:6835","nonce":3986631249}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3135421902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3135421902},{"type":"v1","addr":"172.21.15.78:6837","nonce":3135421902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:40:41.673556+0000","modified":"2023-10-11T09:40:45.122852+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6834/3925157967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":3925157967},{"type":"v1","addr":"172.21.15.133:6834","nonce":3925157967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-11T09:40:58.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.674+0000 7f1314df9700 1 -- 172.21.15.78:0/2956249573 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f12f0065ed0 msgr2=0x7f12f0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:58.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.674+0000 7f1314df9700 1 --2- 172.21.15.78:0/2956249573 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f12f0065ed0 0x7f12f0068380 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1300000f60 tx=0x7f1300009040 comp rx=0 tx=0).stop 2023-10-11T09:40:58.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.674+0000 7f1314df9700 1 -- 172.21.15.78:0/2956249573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13100ce500 msgr2=0x7f1310153500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:58.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.674+0000 7f1314df9700 1 --2- 172.21.15.78:0/2956249573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13100ce500 0x7f1310153500 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1304000c00 tx=0x7f1304004770 comp rx=0 tx=0).stop 2023-10-11T09:40:58.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.678+0000 7f1314df9700 1 -- 172.21.15.78:0/2956249573 shutdown_connections 2023-10-11T09:40:58.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.678+0000 7f1314df9700 1 --2- 172.21.15.78:0/2956249573 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f12f0065ed0 0x7f12f0068380 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:58.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.678+0000 7f1314df9700 1 --2- 172.21.15.78:0/2956249573 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1310159490 0x7f13101462e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:58.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.678+0000 7f1314df9700 1 --2- 172.21.15.78:0/2956249573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13100ce500 0x7f1310153500 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:58.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.678+0000 7f1314df9700 1 --2- 172.21.15.78:0/2956249573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1310070100 0x7f1310145c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:58.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.678+0000 7f1314df9700 1 -- 172.21.15.78:0/2956249573 >> 172.21.15.78:0/2956249573 conn(0x7f131006aca0 msgr2=0x7f131014f750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:58.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.678+0000 7f1314df9700 1 -- 172.21.15.78:0/2956249573 shutdown_connections 2023-10-11T09:40:58.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.678+0000 7f1314df9700 1 -- 172.21.15.78:0/2956249573 wait complete. 2023-10-11T09:40:58.684 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 8 2023-10-11T09:40:58.701 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:40:58.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.870+0000 7f8f596a7700 1 Processor -- start 2023-10-11T09:40:58.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.870+0000 7f8f596a7700 1 -- start start 2023-10-11T09:40:58.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.870+0000 7f8f596a7700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8f5415b8e0 0x7f8f5406d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:58.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.874+0000 7f8f596a7700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f5415e010 con 0x7f8f5406d610 2023-10-11T09:40:58.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.874+0000 7f8f596a7700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f5415e280 con 0x7f8f540ca950 2023-10-11T09:40:58.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.874+0000 7f8f596a7700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f5415e560 con 0x7f8f5415b8e0 2023-10-11T09:40:58.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.874+0000 7f8f4a7fc700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f8f5406d610 0x7f8f540ca240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:34612/0 (socket says 172.21.15.78:34612) 2023-10-11T09:40:58.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.874+0000 7f8f4a7fc700 1 -- 172.21.15.78:0/1803148409 learned_addr learned my addr 172.21.15.78:0/1803148409 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:58.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.874+0000 7f8f52ffd700 1 --2- 172.21.15.78:0/1803148409 >> v2:172.21.15.133:3300/0 conn(0x7f8f5415b8e0 0x7f8f5406d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:58.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.874+0000 7f8f527fc700 1 -- 172.21.15.78:0/1803148409 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2826177534 0 0) 0x7f8f5415e010 con 0x7f8f5406d610 2023-10-11T09:40:58.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.874+0000 7f8f527fc700 1 -- 172.21.15.78:0/1803148409 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f400035d0 con 0x7f8f5406d610 2023-10-11T09:40:58.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.874+0000 7f8f52ffd700 1 -- 172.21.15.78:0/1803148409 >> v1:172.21.15.133:6789/0 conn(0x7f8f540ca950 legacy=0x7f8f5413fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:58.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.874+0000 7f8f52ffd700 1 -- 172.21.15.78:0/1803148409 >> v1:172.21.15.78:6789/0 conn(0x7f8f5406d610 legacy=0x7f8f540ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:58.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.874+0000 7f8f52ffd700 1 -- 172.21.15.78:0/1803148409 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f5415e840 con 0x7f8f5415b8e0 2023-10-11T09:40:58.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.874+0000 7f8f52ffd700 1 --2- 172.21.15.78:0/1803148409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f5415b8e0 0x7f8f5406d0d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8f44009850 tx=0x7f8f44009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c8677f94e441c01 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:58.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.874+0000 7f8f527fc700 1 -- 172.21.15.78:0/1803148409 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f44011830 con 0x7f8f5415b8e0 2023-10-11T09:40:58.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.874+0000 7f8f527fc700 1 -- 172.21.15.78:0/1803148409 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f44011990 con 0x7f8f5415b8e0 2023-10-11T09:40:58.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.874+0000 7f8f527fc700 1 -- 172.21.15.78:0/1803148409 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f44011d40 con 0x7f8f5415b8e0 2023-10-11T09:40:58.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.874+0000 7f8f596a7700 1 -- 172.21.15.78:0/1803148409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f5415b8e0 msgr2=0x7f8f5406d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:58.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.874+0000 7f8f596a7700 1 --2- 172.21.15.78:0/1803148409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f5415b8e0 0x7f8f5406d0d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8f44009850 tx=0x7f8f44009b60 comp rx=0 tx=0).stop 2023-10-11T09:40:58.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.878+0000 7f8f596a7700 1 -- 172.21.15.78:0/1803148409 shutdown_connections 2023-10-11T09:40:58.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.878+0000 7f8f596a7700 1 --2- 172.21.15.78:0/1803148409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f5415b8e0 0x7f8f5406d0d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:58.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.878+0000 7f8f596a7700 1 -- 172.21.15.78:0/1803148409 >> 172.21.15.78:0/1803148409 conn(0x7f8f540cea60 msgr2=0x7f8f540cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:58.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.878+0000 7f8f596a7700 1 -- 172.21.15.78:0/1803148409 shutdown_connections 2023-10-11T09:40:58.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.878+0000 7f8f596a7700 1 -- 172.21.15.78:0/1803148409 wait complete. 2023-10-11T09:40:58.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.878+0000 7f8f596a7700 1 Processor -- start 2023-10-11T09:40:58.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.878+0000 7f8f596a7700 1 -- start start 2023-10-11T09:40:58.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.882+0000 7f8f596a7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f5406d610 0x7f8f54141a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:58.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.882+0000 7f8f596a7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8f540ca950 0x7f8f541f5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:58.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.882+0000 7f8f596a7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f5415b8e0 0x7f8f54159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:58.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.882+0000 7f8f596a7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f541f59c0 con 0x7f8f5406d610 2023-10-11T09:40:58.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.882+0000 7f8f596a7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f541f5ca0 con 0x7f8f5415b8e0 2023-10-11T09:40:58.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.882+0000 7f8f596a7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f541f5f80 con 0x7f8f540ca950 2023-10-11T09:40:58.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.882+0000 7f8f52ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f5406d610 0x7f8f54141a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:58.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.882+0000 7f8f52ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f5406d610 0x7f8f54141a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56738/0 (socket says 172.21.15.78:56738) 2023-10-11T09:40:58.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.882+0000 7f8f52ffd700 1 -- 172.21.15.78:0/2924175846 learned_addr learned my addr 172.21.15.78:0/2924175846 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:58.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.882+0000 7f8f537fe700 1 --2- 172.21.15.78:0/2924175846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f5415b8e0 0x7f8f54159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:58.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.882+0000 7f8f4a7fc700 1 --2- 172.21.15.78:0/2924175846 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8f540ca950 0x7f8f541f5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:58.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.882+0000 7f8f52ffd700 1 -- 172.21.15.78:0/2924175846 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8f540ca950 msgr2=0x7f8f541f5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:58.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.882+0000 7f8f52ffd700 1 --2- 172.21.15.78:0/2924175846 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8f540ca950 0x7f8f541f5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:58.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.882+0000 7f8f52ffd700 1 -- 172.21.15.78:0/2924175846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f5415b8e0 msgr2=0x7f8f54159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:58.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.882+0000 7f8f52ffd700 1 --2- 172.21.15.78:0/2924175846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f5415b8e0 0x7f8f54159bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:58.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.882+0000 7f8f52ffd700 1 -- 172.21.15.78:0/2924175846 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f440094b0 con 0x7f8f5406d610 2023-10-11T09:40:58.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.882+0000 7f8f537fe700 1 --2- 172.21.15.78:0/2924175846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f5415b8e0 0x7f8f54159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:40:58.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.882+0000 7f8f52ffd700 1 --2- 172.21.15.78:0/2924175846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f5406d610 0x7f8f54141a20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8f44014040 tx=0x7f8f44004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:58.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.882+0000 7f8f4bfff700 1 -- 172.21.15.78:0/2924175846 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f44019ab0 con 0x7f8f5406d610 2023-10-11T09:40:58.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.882+0000 7f8f4bfff700 1 -- 172.21.15.78:0/2924175846 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f4401e070 con 0x7f8f5406d610 2023-10-11T09:40:58.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.882+0000 7f8f4bfff700 1 -- 172.21.15.78:0/2924175846 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f44019e50 con 0x7f8f5406d610 2023-10-11T09:40:58.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.882+0000 7f8f596a7700 1 -- 172.21.15.78:0/2924175846 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f541f1e90 con 0x7f8f5406d610 2023-10-11T09:40:58.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.882+0000 7f8f596a7700 1 -- 172.21.15.78:0/2924175846 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f5415a100 con 0x7f8f5406d610 2023-10-11T09:40:58.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.886+0000 7f8f596a7700 1 -- 172.21.15.78:0/2924175846 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f54065d40 con 0x7f8f5406d610 2023-10-11T09:40:58.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.886+0000 7f8f4bfff700 1 -- 172.21.15.78:0/2924175846 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f8f4402d450 con 0x7f8f5406d610 2023-10-11T09:40:58.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.886+0000 7f8f4bfff700 1 --2- 172.21.15.78:0/2924175846 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8f34065f00 0x7f8f340683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:58.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.890+0000 7f8f4bfff700 1 -- 172.21.15.78:0/2924175846 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8f44084880 con 0x7f8f5406d610 2023-10-11T09:40:58.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.890+0000 7f8f4a7fc700 1 --2- 172.21.15.78:0/2924175846 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8f34065f00 0x7f8f340683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:58.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.890+0000 7f8f4a7fc700 1 --2- 172.21.15.78:0/2924175846 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8f34065f00 0x7f8f340683b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8f540cf7c0 tx=0x7f8f3c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:58.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:58.894+0000 7f8f4bfff700 1 -- 172.21.15.78:0/2924175846 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f8f44051ee0 con 0x7f8f5406d610 2023-10-11T09:40:59.050 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:40:59.056 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:40:59.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.066+0000 7f8f596a7700 1 -- 172.21.15.78:0/2924175846 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f540c9b60 con 0x7f8f5406d610 2023-10-11T09:40:59.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.066+0000 7f8f4bfff700 1 -- 172.21.15.78:0/2924175846 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17811 (secure 0 0 0) 0x7f8f44015090 con 0x7f8f5406d610 2023-10-11T09:40:59.072 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:40:59.072 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":23,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:40:42.678477+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:40:39.862581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T09:40:40.663641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:40:59.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.070+0000 7f8f596a7700 1 -- 172.21.15.78:0/2924175846 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8f34065f00 msgr2=0x7f8f340683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:59.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.070+0000 7f8f596a7700 1 --2- 172.21.15.78:0/2924175846 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8f34065f00 0x7f8f340683b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8f540cf7c0 tx=0x7f8f3c009040 comp rx=0 tx=0).stop 2023-10-11T09:40:59.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.070+0000 7f8f596a7700 1 -- 172.21.15.78:0/2924175846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f5406d610 msgr2=0x7f8f54141a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:59.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.070+0000 7f8f596a7700 1 --2- 172.21.15.78:0/2924175846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f5406d610 0x7f8f54141a20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8f44014040 tx=0x7f8f44004770 comp rx=0 tx=0).stop 2023-10-11T09:40:59.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.070+0000 7f8f596a7700 1 -- 172.21.15.78:0/2924175846 shutdown_connections 2023-10-11T09:40:59.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.070+0000 7f8f596a7700 1 --2- 172.21.15.78:0/2924175846 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8f34065f00 0x7f8f340683b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:59.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.070+0000 7f8f596a7700 1 --2- 172.21.15.78:0/2924175846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f5415b8e0 0x7f8f54159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:59.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.070+0000 7f8f596a7700 1 --2- 172.21.15.78:0/2924175846 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8f540ca950 0x7f8f541f5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:59.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.070+0000 7f8f596a7700 1 --2- 172.21.15.78:0/2924175846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f5406d610 0x7f8f54141a20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:59.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.070+0000 7f8f596a7700 1 -- 172.21.15.78:0/2924175846 >> 172.21.15.78:0/2924175846 conn(0x7f8f540cea60 msgr2=0x7f8f540c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:59.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.070+0000 7f8f596a7700 1 -- 172.21.15.78:0/2924175846 shutdown_connections 2023-10-11T09:40:59.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.070+0000 7f8f596a7700 1 -- 172.21.15.78:0/2924175846 wait complete. 2023-10-11T09:40:59.092 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:40:59.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.318+0000 7f4cdaaf5700 1 Processor -- start 2023-10-11T09:40:59.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.322+0000 7f4cdaaf5700 1 -- start start 2023-10-11T09:40:59.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.322+0000 7f4cdaaf5700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4cd4156ce0 0x7f4cd4160b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:59.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.322+0000 7f4cdaaf5700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4cd40d1160 0x7f4cd40cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:59.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.322+0000 7f4cdaaf5700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cd415dc10 con 0x7f4cd40d1660 2023-10-11T09:40:59.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.322+0000 7f4cdaaf5700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cd4160380 con 0x7f4cd40d1160 2023-10-11T09:40:59.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.322+0000 7f4cdaaf5700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd415d930 con 0x7f4cd4156ce0 2023-10-11T09:40:59.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.322+0000 7f4cd92f2700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f4cd40d1660 0x7f4cd40d0a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:34622/0 (socket says 172.21.15.78:34622) 2023-10-11T09:40:59.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.322+0000 7f4cd92f2700 1 -- 172.21.15.78:0/2290965197 learned_addr learned my addr 172.21.15.78:0/2290965197 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:40:59.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.322+0000 7f4cd9af3700 1 --2- 172.21.15.78:0/2290965197 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4cd4156ce0 0x7f4cd4160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:59.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.322+0000 7f4cda2f4700 1 --2- 172.21.15.78:0/2290965197 >> v2:172.21.15.133:3300/0 conn(0x7f4cd40d1160 0x7f4cd40cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:59.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.322+0000 7f4cd8af1700 1 -- 172.21.15.78:0/2290965197 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 71011166 0 0) 0x7f4cd415dc10 con 0x7f4cd40d1660 2023-10-11T09:40:59.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.322+0000 7f4cd8af1700 1 -- 172.21.15.78:0/2290965197 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cc00035d0 con 0x7f4cd40d1660 2023-10-11T09:40:59.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.322+0000 7f4cd8af1700 1 -- 172.21.15.78:0/2290965197 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3706386050 0 0) 0x7f4cc00035d0 con 0x7f4cd40d1660 2023-10-11T09:40:59.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.322+0000 7f4cd8af1700 1 -- 172.21.15.78:0/2290965197 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4cd4156ce0 msgr2=0x7f4cd4160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:59.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.322+0000 7f4cd8af1700 1 --2- 172.21.15.78:0/2290965197 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4cd4156ce0 0x7f4cd4160b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:59.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.322+0000 7f4cd8af1700 1 -- 172.21.15.78:0/2290965197 >> v2:172.21.15.133:3300/0 conn(0x7f4cd40d1160 msgr2=0x7f4cd40cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:59.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.322+0000 7f4cd8af1700 1 --2- 172.21.15.78:0/2290965197 >> v2:172.21.15.133:3300/0 conn(0x7f4cd40d1160 0x7f4cd40cefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:59.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.322+0000 7f4cd8af1700 1 -- 172.21.15.78:0/2290965197 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd4160620 con 0x7f4cd40d1660 2023-10-11T09:40:59.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.322+0000 7f4cd8af1700 1 -- 172.21.15.78:0/2290965197 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f4cc4001670 con 0x7f4cd40d1660 2023-10-11T09:40:59.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.322+0000 7f4cd8af1700 1 -- 172.21.15.78:0/2290965197 >> v1:172.21.15.78:6789/0 conn(0x7f4cd40d1660 legacy=0x7f4cd40d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:59.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.322+0000 7f4cd8af1700 1 --2- 172.21.15.78:0/2290965197 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cc0003a20 0x7f4cc0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:59.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.322+0000 7f4cd8af1700 1 --2- 172.21.15.78:0/2290965197 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4cd4156ce0 0x7f4cc0004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:59.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.322+0000 7f4cd8af1700 1 --2- 172.21.15.78:0/2290965197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cc0004db0 0x7f4cc0005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:59.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.322+0000 7f4cd8af1700 1 -- 172.21.15.78:0/2290965197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd4160380 con 0x7f4cc0004db0 2023-10-11T09:40:59.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.322+0000 7f4cd8af1700 1 -- 172.21.15.78:0/2290965197 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd415d930 con 0x7f4cc0003a20 2023-10-11T09:40:59.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.322+0000 7f4cd8af1700 1 -- 172.21.15.78:0/2290965197 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc000ba60 con 0x7f4cd4156ce0 2023-10-11T09:40:59.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.322+0000 7f4cda2f4700 1 --2- 172.21.15.78:0/2290965197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cc0004db0 0x7f4cc0005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:59.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.326+0000 7f4cd9af3700 1 --2- 172.21.15.78:0/2290965197 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cc0003a20 0x7f4cc0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:59.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.326+0000 7f4cd92f2700 1 --2- 172.21.15.78:0/2290965197 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4cd4156ce0 0x7f4cc0004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:59.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.326+0000 7f4cda2f4700 1 -- 172.21.15.78:0/2290965197 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4cd4156ce0 msgr2=0x7f4cc0004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:59.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.326+0000 7f4cda2f4700 1 --2- 172.21.15.78:0/2290965197 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4cd4156ce0 0x7f4cc0004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:59.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.326+0000 7f4cda2f4700 1 -- 172.21.15.78:0/2290965197 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cc0003a20 msgr2=0x7f4cc0003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:40:59.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.326+0000 7f4cda2f4700 1 --2- 172.21.15.78:0/2290965197 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cc0003a20 0x7f4cc0003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:59.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.326+0000 7f4cda2f4700 1 -- 172.21.15.78:0/2290965197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc00035d0 con 0x7f4cc0004db0 2023-10-11T09:40:59.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.326+0000 7f4cda2f4700 1 --2- 172.21.15.78:0/2290965197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cc0004db0 0x7f4cc0005220 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc009f90 tx=0x7f4cc00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c0b6e71d4d49e26 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:59.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.326+0000 7f4cd8af1700 1 -- 172.21.15.78:0/2290965197 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc00ebd0 con 0x7f4cc0004db0 2023-10-11T09:40:59.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.326+0000 7f4cd8af1700 1 -- 172.21.15.78:0/2290965197 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ccc00f040 con 0x7f4cc0004db0 2023-10-11T09:40:59.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.326+0000 7f4cd8af1700 1 -- 172.21.15.78:0/2290965197 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc014480 con 0x7f4cc0004db0 2023-10-11T09:40:59.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.326+0000 7f4cdaaf5700 1 -- 172.21.15.78:0/2290965197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cc0004db0 msgr2=0x7f4cc0005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:59.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.326+0000 7f4cdaaf5700 1 --2- 172.21.15.78:0/2290965197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cc0004db0 0x7f4cc0005220 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc009f90 tx=0x7f4cc00058a0 comp rx=0 tx=0).stop 2023-10-11T09:40:59.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.326+0000 7f4cda2f4700 1 -- 172.21.15.78:0/2290965197 reap_dead start 2023-10-11T09:40:59.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.326+0000 7f4cdaaf5700 1 -- 172.21.15.78:0/2290965197 shutdown_connections 2023-10-11T09:40:59.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.326+0000 7f4cdaaf5700 1 -- 172.21.15.78:0/2290965197 >> 172.21.15.78:0/2290965197 conn(0x7f4cd406afb0 msgr2=0x7f4cd406b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:59.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.326+0000 7f4cdaaf5700 1 -- 172.21.15.78:0/2290965197 shutdown_connections 2023-10-11T09:40:59.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.326+0000 7f4cdaaf5700 1 -- 172.21.15.78:0/2290965197 wait complete. 2023-10-11T09:40:59.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.330+0000 7f4cdaaf5700 1 Processor -- start 2023-10-11T09:40:59.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.330+0000 7f4cdaaf5700 1 -- start start 2023-10-11T09:40:59.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.330+0000 7f4cdaaf5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cc0004db0 0x7f4cd41e84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:59.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.330+0000 7f4cdaaf5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cd40c89c0 0x7f4cd40c94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:59.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.330+0000 7f4cdaaf5700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4cd41538f0 0x7f4cd4153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:59.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.330+0000 7f4cdaaf5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd41e89f0 con 0x7f4cc0004db0 2023-10-11T09:40:59.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.330+0000 7f4cdaaf5700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd414de30 con 0x7f4cd40c89c0 2023-10-11T09:40:59.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.330+0000 7f4cdaaf5700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd41542a0 con 0x7f4cd41538f0 2023-10-11T09:40:59.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.330+0000 7f4cd9af3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cc0004db0 0x7f4cd41e84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:59.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.330+0000 7f4cd9af3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cc0004db0 0x7f4cd41e84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56758/0 (socket says 172.21.15.78:56758) 2023-10-11T09:40:59.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.330+0000 7f4cd9af3700 1 -- 172.21.15.78:0/1084767499 learned_addr learned my addr 172.21.15.78:0/1084767499 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:40:59.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.330+0000 7f4cd92f2700 1 --2- 172.21.15.78:0/1084767499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cd40c89c0 0x7f4cd40c94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:59.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.330+0000 7f4cda2f4700 1 --2- 172.21.15.78:0/1084767499 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4cd41538f0 0x7f4cd4153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:59.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.330+0000 7f4cd9af3700 1 -- 172.21.15.78:0/1084767499 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4cd41538f0 msgr2=0x7f4cd4153d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:59.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.330+0000 7f4cd9af3700 1 --2- 172.21.15.78:0/1084767499 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4cd41538f0 0x7f4cd4153d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:59.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.330+0000 7f4cd9af3700 1 -- 172.21.15.78:0/1084767499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cd40c89c0 msgr2=0x7f4cd40c94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:59.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.330+0000 7f4cd9af3700 1 --2- 172.21.15.78:0/1084767499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cd40c89c0 0x7f4cd40c94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:59.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.330+0000 7f4cd9af3700 1 -- 172.21.15.78:0/1084767499 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd40c99f0 con 0x7f4cc0004db0 2023-10-11T09:40:59.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.330+0000 7f4cda2f4700 1 --2- 172.21.15.78:0/1084767499 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4cd41538f0 0x7f4cd4153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:40:59.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.330+0000 7f4cd92f2700 1 --2- 172.21.15.78:0/1084767499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cd40c89c0 0x7f4cd40c94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:40:59.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.330+0000 7f4cd9af3700 1 --2- 172.21.15.78:0/1084767499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cc0004db0 0x7f4cd41e84b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4cd00119e0 tx=0x7f4cd0011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:59.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.334+0000 7f4cca7fc700 1 -- 172.21.15.78:0/1084767499 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd000f940 con 0x7f4cc0004db0 2023-10-11T09:40:59.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.334+0000 7f4cca7fc700 1 -- 172.21.15.78:0/1084767499 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd000faa0 con 0x7f4cc0004db0 2023-10-11T09:40:59.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.334+0000 7f4cca7fc700 1 -- 172.21.15.78:0/1084767499 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd000fe30 con 0x7f4cc0004db0 2023-10-11T09:40:59.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.334+0000 7f4cdaaf5700 1 -- 172.21.15.78:0/1084767499 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cd40c7640 con 0x7f4cc0004db0 2023-10-11T09:40:59.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.334+0000 7f4cdaaf5700 1 -- 172.21.15.78:0/1084767499 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cd40c7c00 con 0x7f4cc0004db0 2023-10-11T09:40:59.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.334+0000 7f4cdaaf5700 1 -- 172.21.15.78:0/1084767499 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cd4065d40 con 0x7f4cc0004db0 2023-10-11T09:40:59.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.334+0000 7f4cca7fc700 1 -- 172.21.15.78:0/1084767499 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f4cd0014440 con 0x7f4cc0004db0 2023-10-11T09:40:59.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.338+0000 7f4cca7fc700 1 --2- 172.21.15.78:0/1084767499 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4cb8065ed0 0x7f4cb8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:40:59.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.338+0000 7f4cca7fc700 1 -- 172.21.15.78:0/1084767499 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4cd0083f10 con 0x7f4cc0004db0 2023-10-11T09:40:59.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.338+0000 7f4cd92f2700 1 --2- 172.21.15.78:0/1084767499 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4cb8065ed0 0x7f4cb8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:40:59.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.338+0000 7f4cd92f2700 1 --2- 172.21.15.78:0/1084767499 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4cb8065ed0 0x7f4cb8068380 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4000fb0 tx=0x7f4cc4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:40:59.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.346+0000 7f4cca7fc700 1 -- 172.21.15.78:0/1084767499 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f4cd0051550 con 0x7f4cc0004db0 2023-10-11T09:40:59.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.554+0000 7f4cdaaf5700 1 -- 172.21.15.78:0/1084767499 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4cd406d310 con 0x7f4cc0004db0 2023-10-11T09:40:59.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.554+0000 7f4cca7fc700 1 -- 172.21.15.78:0/1084767499 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4648 (secure 0 0 0) 0x7f4cd0054b70 con 0x7f4cc0004db0 2023-10-11T09:40:59.560 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:40:59.560 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/4282378510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4282378510},{"type":"v1","addr":"172.21.15.133:6835","nonce":4282378510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3986631249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3986631249},{"type":"v1","addr":"172.21.15.78:6835","nonce":3986631249}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3135421902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3135421902},{"type":"v1","addr":"172.21.15.78:6837","nonce":3135421902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:40:41.673556+0000","modified":"2023-10-11T09:40:45.122852+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6834/3925157967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":3925157967},{"type":"v1","addr":"172.21.15.133:6834","nonce":3925157967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-11T09:40:59.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.558+0000 7f4cdaaf5700 1 -- 172.21.15.78:0/1084767499 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4cb8065ed0 msgr2=0x7f4cb8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:59.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.558+0000 7f4cdaaf5700 1 --2- 172.21.15.78:0/1084767499 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4cb8065ed0 0x7f4cb8068380 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4000fb0 tx=0x7f4cc4009040 comp rx=0 tx=0).stop 2023-10-11T09:40:59.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.558+0000 7f4cdaaf5700 1 -- 172.21.15.78:0/1084767499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cc0004db0 msgr2=0x7f4cd41e84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:40:59.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.558+0000 7f4cdaaf5700 1 --2- 172.21.15.78:0/1084767499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cc0004db0 0x7f4cd41e84b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4cd00119e0 tx=0x7f4cd0011da0 comp rx=0 tx=0).stop 2023-10-11T09:40:59.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.558+0000 7f4cdaaf5700 1 -- 172.21.15.78:0/1084767499 shutdown_connections 2023-10-11T09:40:59.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.558+0000 7f4cdaaf5700 1 --2- 172.21.15.78:0/1084767499 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4cb8065ed0 0x7f4cb8068380 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:59.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.558+0000 7f4cdaaf5700 1 --2- 172.21.15.78:0/1084767499 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4cd41538f0 0x7f4cd4153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:59.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.558+0000 7f4cdaaf5700 1 --2- 172.21.15.78:0/1084767499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cd40c89c0 0x7f4cd40c94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:59.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.558+0000 7f4cdaaf5700 1 --2- 172.21.15.78:0/1084767499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cc0004db0 0x7f4cd41e84b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:40:59.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.558+0000 7f4cdaaf5700 1 -- 172.21.15.78:0/1084767499 >> 172.21.15.78:0/1084767499 conn(0x7f4cd406afb0 msgr2=0x7f4cd40b6250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:40:59.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.558+0000 7f4cdaaf5700 1 -- 172.21.15.78:0/1084767499 shutdown_connections 2023-10-11T09:40:59.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:40:59.558+0000 7f4cdaaf5700 1 -- 172.21.15.78:0/1084767499 wait complete. 2023-10-11T09:40:59.565 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 8 2023-10-11T09:40:59.986 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-11T09:40:41.673556+0000', 'modified': '2023-10-11T09:40:45.122852+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4306}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4306': {'gid': 4306, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.133:6834/3925157967', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6833', 'nonce': 3925157967}, {'type': 'v1', 'addr': '172.21.15.133:6834', 'nonce': 3925157967}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-11T09:40:59.987 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-11T09:40:59.987 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-11T09:41:00.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.166+0000 7f403df7f700 1 Processor -- start 2023-10-11T09:41:00.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.166+0000 7f403df7f700 1 -- start start 2023-10-11T09:41:00.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.166+0000 7f403df7f700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f403806fb90 0x7f40380cfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:00.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.166+0000 7f403df7f700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4038160dc0 0x7f40381611b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:00.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.166+0000 7f403df7f700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40380d0620 con 0x7f403806e1c0 2023-10-11T09:41:00.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.170+0000 7f403df7f700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f403806f140 con 0x7f4038160dc0 2023-10-11T09:41:00.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.170+0000 7f403df7f700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4038160690 con 0x7f403806fb90 2023-10-11T09:41:00.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.170+0000 7f4037fff700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4038160dc0 0x7f40381611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:00.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.170+0000 7f4037fff700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4038160dc0 0x7f40381611b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:37082/0 (socket says 172.21.15.78:37082) 2023-10-11T09:41:00.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.170+0000 7f4037fff700 1 -- 172.21.15.78:0/1013987570 learned_addr learned my addr 172.21.15.78:0/1013987570 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:00.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.170+0000 7f4036ffd700 1 --2- 172.21.15.78:0/1013987570 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f403806fb90 0x7f40380cfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:00.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.170+0000 7f4037fff700 1 -- 172.21.15.78:0/1013987570 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f403806fb90 msgr2=0x7f40380cfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:00.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.170+0000 7f4037fff700 1 --2- 172.21.15.78:0/1013987570 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f403806fb90 0x7f40380cfdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.170+0000 7f4037fff700 1 -- 172.21.15.78:0/1013987570 >> v1:172.21.15.133:6789/0 conn(0x7f403806e1c0 legacy=0x7f403806e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:00.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.170+0000 7f4037fff700 1 -- 172.21.15.78:0/1013987570 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40380d0900 con 0x7f4038160dc0 2023-10-11T09:41:00.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.170+0000 7f4037fff700 1 --2- 172.21.15.78:0/1013987570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4038160dc0 0x7f40381611b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4030009e50 tx=0x7f403000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e850a335ead7900b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:00.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.170+0000 7f40367fc700 1 -- 172.21.15.78:0/1013987570 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403000e070 con 0x7f4038160dc0 2023-10-11T09:41:00.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.170+0000 7f40367fc700 1 -- 172.21.15.78:0/1013987570 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f403000b910 con 0x7f4038160dc0 2023-10-11T09:41:00.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.170+0000 7f40367fc700 1 -- 172.21.15.78:0/1013987570 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403000bcc0 con 0x7f4038160dc0 2023-10-11T09:41:00.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.170+0000 7f403df7f700 1 -- 172.21.15.78:0/1013987570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4038160dc0 msgr2=0x7f40381611b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:00.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.170+0000 7f403df7f700 1 --2- 172.21.15.78:0/1013987570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4038160dc0 0x7f40381611b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4030009e50 tx=0x7f403000b330 comp rx=0 tx=0).stop 2023-10-11T09:41:00.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.170+0000 7f403df7f700 1 -- 172.21.15.78:0/1013987570 shutdown_connections 2023-10-11T09:41:00.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.170+0000 7f403df7f700 1 --2- 172.21.15.78:0/1013987570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4038160dc0 0x7f40381611b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.170+0000 7f403df7f700 1 --2- 172.21.15.78:0/1013987570 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f403806fb90 0x7f40380cfdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.170+0000 7f403df7f700 1 -- 172.21.15.78:0/1013987570 >> 172.21.15.78:0/1013987570 conn(0x7f40380d0f70 msgr2=0x7f40380d1370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:00.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.170+0000 7f403df7f700 1 -- 172.21.15.78:0/1013987570 shutdown_connections 2023-10-11T09:41:00.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.170+0000 7f403df7f700 1 -- 172.21.15.78:0/1013987570 wait complete. 2023-10-11T09:41:00.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.174+0000 7f403df7f700 1 Processor -- start 2023-10-11T09:41:00.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.174+0000 7f403df7f700 1 -- start start 2023-10-11T09:41:00.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.178+0000 7f403df7f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f403806e1c0 0x7f40380b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:00.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.178+0000 7f403df7f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f403806fb90 0x7f40380b5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:00.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.178+0000 7f403df7f700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4038160dc0 0x7f40381581a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:00.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.178+0000 7f403df7f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40380b1b80 con 0x7f403806e1c0 2023-10-11T09:41:00.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.178+0000 7f403df7f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40380b1e60 con 0x7f403806fb90 2023-10-11T09:41:00.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.178+0000 7f403df7f700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40380b2140 con 0x7f4038160dc0 2023-10-11T09:41:00.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.178+0000 7f40377fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f403806e1c0 0x7f40380b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:00.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.178+0000 7f40377fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f403806e1c0 0x7f40380b1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56764/0 (socket says 172.21.15.78:56764) 2023-10-11T09:41:00.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.178+0000 7f40377fe700 1 -- 172.21.15.78:0/2094989675 learned_addr learned my addr 172.21.15.78:0/2094989675 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:00.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.178+0000 7f4036ffd700 1 --2- 172.21.15.78:0/2094989675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f403806fb90 0x7f40380b5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:00.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.178+0000 7f4037fff700 1 --2- 172.21.15.78:0/2094989675 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4038160dc0 0x7f40381581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:00.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.178+0000 7f40377fe700 1 -- 172.21.15.78:0/2094989675 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4038160dc0 msgr2=0x7f40381581a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:00.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.178+0000 7f40377fe700 1 --2- 172.21.15.78:0/2094989675 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4038160dc0 0x7f40381581a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.178+0000 7f40377fe700 1 -- 172.21.15.78:0/2094989675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f403806fb90 msgr2=0x7f40380b5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:00.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.178+0000 7f40377fe700 1 --2- 172.21.15.78:0/2094989675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f403806fb90 0x7f40380b5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.178+0000 7f40377fe700 1 -- 172.21.15.78:0/2094989675 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4030009b30 con 0x7f403806e1c0 2023-10-11T09:41:00.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.178+0000 7f4037fff700 1 --2- 172.21.15.78:0/2094989675 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4038160dc0 0x7f40381581a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:00.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.178+0000 7f40377fe700 1 --2- 172.21.15.78:0/2094989675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f403806e1c0 0x7f40380b1250 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f402800c500 tx=0x7f402800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:00.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.178+0000 7f401ffff700 1 -- 172.21.15.78:0/2094989675 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40280052d0 con 0x7f403806e1c0 2023-10-11T09:41:00.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.178+0000 7f403df7f700 1 -- 172.21.15.78:0/2094989675 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40380ad790 con 0x7f403806e1c0 2023-10-11T09:41:00.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.178+0000 7f401ffff700 1 -- 172.21.15.78:0/2094989675 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4028005430 con 0x7f403806e1c0 2023-10-11T09:41:00.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.178+0000 7f401ffff700 1 -- 172.21.15.78:0/2094989675 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40280055a0 con 0x7f403806e1c0 2023-10-11T09:41:00.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.178+0000 7f403df7f700 1 -- 172.21.15.78:0/2094989675 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40380b1790 con 0x7f403806e1c0 2023-10-11T09:41:00.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.182+0000 7f401ffff700 1 -- 172.21.15.78:0/2094989675 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f40280038e0 con 0x7f403806e1c0 2023-10-11T09:41:00.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.182+0000 7f4041211740 1 Processor -- start 2023-10-11T09:41:00.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.182+0000 7f4041211740 1 -- start start 2023-10-11T09:41:00.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.186+0000 7f401ffff700 1 --2- 172.21.15.78:0/2094989675 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4024065ed0 0x7f4024068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:00.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.186+0000 7f401ffff700 1 -- 172.21.15.78:0/2094989675 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4028082bd0 con 0x7f403806e1c0 2023-10-11T09:41:00.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.186+0000 7f4041211740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x116ceb0 0x141c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:00.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.186+0000 7f4041211740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x116cae0 0x1424d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:00.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.186+0000 7f4041211740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x116c710 0x1425290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:00.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.186+0000 7f4037fff700 1 --2- 172.21.15.78:0/2094989675 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4024065ed0 0x7f4024068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:00.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.186+0000 7f4041211740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x134c4a0 con 0x116cae0 2023-10-11T09:41:00.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.186+0000 7f4041211740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x1349c30 con 0x116ceb0 2023-10-11T09:41:00.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.186+0000 7f4041211740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x1345520 con 0x116c710 2023-10-11T09:41:00.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.186+0000 7f4036ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x116cae0 0x1424d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:00.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.186+0000 7f4036ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x116cae0 0x1424d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56778/0 (socket says 172.21.15.78:56778) 2023-10-11T09:41:00.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.186+0000 7f4036ffd700 1 -- 172.21.15.78:0/1012539181 learned_addr learned my addr 172.21.15.78:0/1012539181 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:00.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.186+0000 7f40377fe700 1 --2- 172.21.15.78:0/1012539181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x116ceb0 0x141c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:00.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.186+0000 7f4037fff700 1 --2- 172.21.15.78:0/1012539181 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x116c710 0x1425290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:00.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.186+0000 7f4036ffd700 1 -- 172.21.15.78:0/1012539181 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x116c710 msgr2=0x1425290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:00.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.186+0000 7f4036ffd700 1 --2- 172.21.15.78:0/1012539181 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x116c710 0x1425290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.186+0000 7f4036ffd700 1 -- 172.21.15.78:0/1012539181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x116ceb0 msgr2=0x141c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:00.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.186+0000 7f4036ffd700 1 --2- 172.21.15.78:0/1012539181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x116ceb0 0x141c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.186+0000 7f4036ffd700 1 -- 172.21.15.78:0/1012539181 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x11b89e0 con 0x116cae0 2023-10-11T09:41:00.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.186+0000 7f4037fff700 1 --2- 172.21.15.78:0/2094989675 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4024065ed0 0x7f4024068380 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f403000b370 tx=0x7f403000e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:00.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.186+0000 7f4036ffd700 1 --2- 172.21.15.78:0/1012539181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x116cae0 0x1424d50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f402c009ba0 tx=0x7f402c009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f526e21fc3106ac server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:00.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.186+0000 7f401d7fa700 1 -- 172.21.15.78:0/1012539181 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c004300 con 0x116cae0 2023-10-11T09:41:00.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.186+0000 7f401d7fa700 1 -- 172.21.15.78:0/1012539181 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f402c004460 con 0x116cae0 2023-10-11T09:41:00.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.186+0000 7f401d7fa700 1 -- 172.21.15.78:0/1012539181 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c0045d0 con 0x116cae0 2023-10-11T09:41:00.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.186+0000 7f4041211740 1 -- 172.21.15.78:0/1012539181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x116cae0 msgr2=0x1424d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:00.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.186+0000 7f4041211740 1 --2- 172.21.15.78:0/1012539181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x116cae0 0x1424d50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f402c009ba0 tx=0x7f402c009eb0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.186+0000 7f4041211740 1 -- 172.21.15.78:0/1012539181 shutdown_connections 2023-10-11T09:41:00.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.186+0000 7f4041211740 1 --2- 172.21.15.78:0/1012539181 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x116c710 0x1425290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.186+0000 7f4041211740 1 --2- 172.21.15.78:0/1012539181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x116cae0 0x1424d50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.186+0000 7f4041211740 1 --2- 172.21.15.78:0/1012539181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x116ceb0 0x141c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.186+0000 7f4041211740 1 -- 172.21.15.78:0/1012539181 >> 172.21.15.78:0/1012539181 conn(0x13d4fa0 msgr2=0x1406210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:00.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.186+0000 7f4041211740 1 -- 172.21.15.78:0/1012539181 shutdown_connections 2023-10-11T09:41:00.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.186+0000 7f4041211740 1 -- 172.21.15.78:0/1012539181 wait complete. 2023-10-11T09:41:00.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.190+0000 7f4041211740 1 -- start start 2023-10-11T09:41:00.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.190+0000 7f4041211740 1 Processor -- start 2023-10-11T09:41:00.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.190+0000 7f4041211740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x116c710 0x141af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:00.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.190+0000 7f4041211740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x116cae0 0x13d88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:00.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.190+0000 7f4041211740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x116ceb0 0x13d8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:00.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.190+0000 7f4041211740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x1308b00 con 0x116ceb0 2023-10-11T09:41:00.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.190+0000 7f4041211740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x13087a0 con 0x116cae0 2023-10-11T09:41:00.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.190+0000 7f4041211740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x11f1b20 con 0x116c710 2023-10-11T09:41:00.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.190+0000 7f4041211740 10 client.0 _ll_register_callbacks cb 0x133ee60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:41:00.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.190+0000 7f4037fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x116ceb0 0x13d8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:00.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.190+0000 7f4037fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x116ceb0 0x13d8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56786/0 (socket says 172.21.15.78:56786) 2023-10-11T09:41:00.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.190+0000 7f4037fff700 1 -- 172.21.15.78:0/1729503476 learned_addr learned my addr 172.21.15.78:0/1729503476 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:00.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.190+0000 7f4036ffd700 1 --2- 172.21.15.78:0/1729503476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x116cae0 0x13d88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:00.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.190+0000 7f40377fe700 1 --2- 172.21.15.78:0/1729503476 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x116c710 0x141af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:00.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.190+0000 7f4037fff700 1 -- 172.21.15.78:0/1729503476 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x116c710 msgr2=0x141af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:00.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.190+0000 7f4037fff700 1 --2- 172.21.15.78:0/1729503476 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x116c710 0x141af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.190+0000 7f4037fff700 1 -- 172.21.15.78:0/1729503476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x116cae0 msgr2=0x13d88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:00.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.190+0000 7f4037fff700 1 --2- 172.21.15.78:0/1729503476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x116cae0 0x13d88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.190+0000 7f4037fff700 1 -- 172.21.15.78:0/1729503476 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f402c0097d0 con 0x116ceb0 2023-10-11T09:41:00.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.190+0000 7f4037fff700 1 --2- 172.21.15.78:0/1729503476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x116ceb0 0x13d8e10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4030020b80 tx=0x7f4030004b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2434607f44b1a5b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:00.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.190+0000 7f40167fc700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:41:00.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.190+0000 7f40167fc700 1 -- 172.21.15.78:0/1729503476 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403001fa40 con 0x116ceb0 2023-10-11T09:41:00.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.190+0000 7f4041211740 1 -- 172.21.15.78:0/1729503476 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x133e5a0 con 0x116ceb0 2023-10-11T09:41:00.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.190+0000 7f40167fc700 1 -- 172.21.15.78:0/1729503476 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f403001fba0 con 0x116ceb0 2023-10-11T09:41:00.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.190+0000 7f40167fc700 1 -- 172.21.15.78:0/1729503476 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f403001fd60 con 0x116ceb0 2023-10-11T09:41:00.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.190+0000 7f4041211740 10 client.4725 fetch_fsmap learned FSMap version 8 2023-10-11T09:41:00.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.190+0000 7f4041211740 1 -- 172.21.15.78:0/1729503476 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x130b750 con 0x116ceb0 2023-10-11T09:41:00.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.190+0000 7f40167fc700 1 -- 172.21.15.78:0/1729503476 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x133e5a0 con 0x116ceb0 2023-10-11T09:41:00.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.190+0000 7f40167fc700 1 -- 172.21.15.78:0/1729503476 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 8) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f403002c400 con 0x116ceb0 2023-10-11T09:41:00.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.190+0000 7f4041211740 10 client.4725 fetch_fsmap finished waiting for FSMap version 8 2023-10-11T09:41:00.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.190+0000 7f4041211740 10 client.4725 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-11T09:41:00.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.190+0000 7f4041211740 20 client.4725 populate_metadata read hostname 'smithi078' 2023-10-11T09:41:00.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.190+0000 7f4041211740 1 --2- 172.21.15.78:0/1729503476 >> [v2:172.21.15.133:6833/3925157967,v1:172.21.15.133:6834/3925157967] conn(0x11b7790 0x1438020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:00.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.190+0000 7f4041211740 4 client.4725 mds_command: new command op to 4306 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-11T09:41:00.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.190+0000 7f4041211740 1 -- 172.21.15.78:0/1729503476 --> [v2:172.21.15.133:6833/3925157967,v1:172.21.15.133:6834/3925157967] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x133e2c0 con 0x11b7790 2023-10-11T09:41:00.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.194+0000 7f40377fe700 1 --2- 172.21.15.78:0/1729503476 >> [v2:172.21.15.133:6833/3925157967,v1:172.21.15.133:6834/3925157967] conn(0x11b7790 0x1438020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:00.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.194+0000 7f40377fe700 1 --2- 172.21.15.78:0/1729503476 >> [v2:172.21.15.133:6833/3925157967,v1:172.21.15.133:6834/3925157967] conn(0x11b7790 0x1438020 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee166f504d762520 server_cookie=7a9116ddaca5fc24 in_seq=0 out_seq=0 2023-10-11T09:41:00.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.194+0000 7f40167fc700 10 client.4725 ms_handle_connect on v2:172.21.15.133:6833/3925157967 2023-10-11T09:41:00.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.194+0000 7f40167fc700 1 -- 172.21.15.78:0/1729503476 <== mds.0 v2:172.21.15.133:6833/3925157967 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x133e2c0 con 0x11b7790 2023-10-11T09:41:00.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.194+0000 7f40167fc700 10 client.4725 handle_command_reply: tid=0 2023-10-11T09:41:00.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.194+0000 7f4041211740 1 client.4725 shutdown 2023-10-11T09:41:00.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.194+0000 7f4041211740 1 -- 172.21.15.78:0/1729503476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x116ceb0 msgr2=0x13d8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:00.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.194+0000 7f4041211740 1 --2- 172.21.15.78:0/1729503476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x116ceb0 0x13d8e10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4030020b80 tx=0x7f4030004b60 comp rx=0 tx=0).stop 2023-10-11T09:41:00.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.194+0000 7f4041211740 1 -- 172.21.15.78:0/1729503476 shutdown_connections 2023-10-11T09:41:00.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.194+0000 7f4041211740 1 --2- 172.21.15.78:0/1729503476 >> [v2:172.21.15.133:6833/3925157967,v1:172.21.15.133:6834/3925157967] conn(0x11b7790 0x1438020 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.194+0000 7f4041211740 1 --2- 172.21.15.78:0/1729503476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x116ceb0 0x13d8e10 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.194+0000 7f4041211740 1 --2- 172.21.15.78:0/1729503476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x116cae0 0x13d88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.194+0000 7f40167fc700 0 client.4725 ms_handle_reset on v2:172.21.15.133:6833/3925157967 2023-10-11T09:41:00.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.194+0000 7f4041211740 1 --2- 172.21.15.78:0/1729503476 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x116c710 0x141af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.194+0000 7f4041211740 1 -- 172.21.15.78:0/1729503476 >> 172.21.15.78:0/1729503476 conn(0x13d4fa0 msgr2=0x14240d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:00.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.194+0000 7f4041211740 1 -- 172.21.15.78:0/1729503476 shutdown_connections 2023-10-11T09:41:00.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.194+0000 7f4041211740 1 -- 172.21.15.78:0/1729503476 wait complete. 2023-10-11T09:41:00.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.194+0000 7f4041211740 20 client.4725 trim_cache size 0 max 16384 2023-10-11T09:41:00.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.214+0000 7f4041211740 1 Processor -- start 2023-10-11T09:41:00.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.214+0000 7f4041211740 1 -- start start 2023-10-11T09:41:00.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.218+0000 7f4041211740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x116c710 0x13fd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:00.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.218+0000 7f4041211740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x116cae0 0x13fadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:00.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.218+0000 7f4041211740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x116ceb0 0x13fb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:00.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.218+0000 7f4041211740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x1314cd0 con 0x116c710 2023-10-11T09:41:00.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.218+0000 7f4041211740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x1344d30 con 0x116ceb0 2023-10-11T09:41:00.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.218+0000 7f4041211740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x130fa20 con 0x116cae0 2023-10-11T09:41:00.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.218+0000 7f40377fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x116c710 0x13fd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:00.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.218+0000 7f40377fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x116c710 0x13fd950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56790/0 (socket says 172.21.15.78:56790) 2023-10-11T09:41:00.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.218+0000 7f40377fe700 1 -- 172.21.15.78:0/1426695142 learned_addr learned my addr 172.21.15.78:0/1426695142 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:00.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.218+0000 7f4036ffd700 1 --2- 172.21.15.78:0/1426695142 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x116cae0 0x13fadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:00.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.218+0000 7f4037fff700 1 --2- 172.21.15.78:0/1426695142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x116ceb0 0x13fb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:00.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.218+0000 7f40377fe700 1 -- 172.21.15.78:0/1426695142 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x116cae0 msgr2=0x13fadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:00.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.218+0000 7f40377fe700 1 --2- 172.21.15.78:0/1426695142 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x116cae0 0x13fadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.218+0000 7f40377fe700 1 -- 172.21.15.78:0/1426695142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x116ceb0 msgr2=0x13fb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:00.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.218+0000 7f40377fe700 1 --2- 172.21.15.78:0/1426695142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x116ceb0 0x13fb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.218+0000 7f40377fe700 1 -- 172.21.15.78:0/1426695142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x13d8ff0 con 0x116c710 2023-10-11T09:41:00.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.218+0000 7f40377fe700 1 --2- 172.21.15.78:0/1426695142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x116c710 0x13fd950 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4028053bd0 tx=0x7f4028005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=547bdf87cf329292 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:00.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.218+0000 7f401dffb700 1 -- 172.21.15.78:0/1426695142 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4028052010 con 0x116c710 2023-10-11T09:41:00.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.218+0000 7f401dffb700 1 -- 172.21.15.78:0/1426695142 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4028081010 con 0x116c710 2023-10-11T09:41:00.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.218+0000 7f401dffb700 1 -- 172.21.15.78:0/1426695142 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40280533c0 con 0x116c710 2023-10-11T09:41:00.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.218+0000 7f4041211740 1 -- 172.21.15.78:0/1426695142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x116c710 msgr2=0x13fd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:00.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.218+0000 7f4041211740 1 --2- 172.21.15.78:0/1426695142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x116c710 0x13fd950 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4028053bd0 tx=0x7f4028005040 comp rx=0 tx=0).stop 2023-10-11T09:41:00.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.218+0000 7f4041211740 1 -- 172.21.15.78:0/1426695142 shutdown_connections 2023-10-11T09:41:00.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.218+0000 7f4041211740 1 --2- 172.21.15.78:0/1426695142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x116ceb0 0x13fb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.218+0000 7f4041211740 1 --2- 172.21.15.78:0/1426695142 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x116cae0 0x13fadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.218+0000 7f4041211740 1 --2- 172.21.15.78:0/1426695142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x116c710 0x13fd950 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.218+0000 7f4041211740 1 -- 172.21.15.78:0/1426695142 >> 172.21.15.78:0/1426695142 conn(0x13d4fa0 msgr2=0x14016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:00.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.222+0000 7f4041211740 1 -- 172.21.15.78:0/1426695142 shutdown_connections 2023-10-11T09:41:00.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.222+0000 7f4041211740 1 -- 172.21.15.78:0/1426695142 wait complete. 2023-10-11T09:41:00.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.222+0000 7f4041211740 1 -- start start 2023-10-11T09:41:00.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.222+0000 7f4041211740 1 Processor -- start 2023-10-11T09:41:00.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.222+0000 7f4041211740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x116c710 0x1480630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:00.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.222+0000 7f4041211740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x116cae0 0x13db620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:00.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.222+0000 7f4041211740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x116ceb0 0x13dbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:00.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.222+0000 7f4041211740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x13fcf00 con 0x116ceb0 2023-10-11T09:41:00.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.222+0000 7f4041211740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x13dc0a0 con 0x116cae0 2023-10-11T09:41:00.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.222+0000 7f4041211740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x13dd6c0 con 0x116c710 2023-10-11T09:41:00.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.222+0000 7f4041211740 10 client.0 _ll_register_callbacks cb 0x7f402c009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:41:00.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.222+0000 7f4037fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x116ceb0 0x13dbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:00.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.226+0000 7f4036ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x116cae0 0x13db620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:00.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.226+0000 7f4037fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x116ceb0 0x13dbb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56804/0 (socket says 172.21.15.78:56804) 2023-10-11T09:41:00.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.226+0000 7f4037fff700 1 -- 172.21.15.78:0/1432311628 learned_addr learned my addr 172.21.15.78:0/1432311628 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:00.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.226+0000 7f40377fe700 1 --2- 172.21.15.78:0/1432311628 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x116c710 0x1480630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:00.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.226+0000 7f4037fff700 1 -- 172.21.15.78:0/1432311628 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x116c710 msgr2=0x1480630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:00.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.226+0000 7f4037fff700 1 --2- 172.21.15.78:0/1432311628 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x116c710 0x1480630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.226+0000 7f4037fff700 1 -- 172.21.15.78:0/1432311628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x116cae0 msgr2=0x13db620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:00.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.226+0000 7f4037fff700 1 --2- 172.21.15.78:0/1432311628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x116cae0 0x13db620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.226+0000 7f4037fff700 1 -- 172.21.15.78:0/1432311628 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40380b1790 con 0x116ceb0 2023-10-11T09:41:00.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.226+0000 7f4037fff700 1 --2- 172.21.15.78:0/1432311628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x116ceb0 0x13dbb60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f403001f4f0 tx=0x7f4030028000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9728c7480fc5d968 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:00.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.226+0000 7f40167fc700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:41:00.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.226+0000 7f40167fc700 1 -- 172.21.15.78:0/1432311628 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403001fc50 con 0x116ceb0 2023-10-11T09:41:00.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.226+0000 7f4041211740 1 -- 172.21.15.78:0/1432311628 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x143c9a0 con 0x116ceb0 2023-10-11T09:41:00.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.226+0000 7f40167fc700 1 -- 172.21.15.78:0/1432311628 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4030040070 con 0x116ceb0 2023-10-11T09:41:00.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.226+0000 7f40167fc700 1 -- 172.21.15.78:0/1432311628 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4030020760 con 0x116ceb0 2023-10-11T09:41:00.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.226+0000 7f4041211740 10 client.4731 fetch_fsmap learned FSMap version 8 2023-10-11T09:41:00.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.226+0000 7f4041211740 1 -- 172.21.15.78:0/1432311628 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x7f4038158a40 con 0x116ceb0 2023-10-11T09:41:00.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.226+0000 7f40167fc700 1 -- 172.21.15.78:0/1432311628 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x143c9a0 con 0x116ceb0 2023-10-11T09:41:00.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.226+0000 7f40167fc700 1 -- 172.21.15.78:0/1432311628 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 8) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f403003a3f0 con 0x116ceb0 2023-10-11T09:41:00.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.226+0000 7f4041211740 10 client.4731 fetch_fsmap finished waiting for FSMap version 8 2023-10-11T09:41:00.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.226+0000 7f4041211740 10 client.4731 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-11T09:41:00.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.226+0000 7f4041211740 20 client.4731 populate_metadata read hostname 'smithi078' 2023-10-11T09:41:00.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.226+0000 7f4041211740 1 --2- 172.21.15.78:0/1432311628 >> [v2:172.21.15.133:6833/3925157967,v1:172.21.15.133:6834/3925157967] conn(0x13f8870 0x13f8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:00.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.226+0000 7f4041211740 4 client.4731 mds_command: new command op to 4306 tid=0[{"prefix": "status"}] 2023-10-11T09:41:00.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.226+0000 7f4041211740 1 -- 172.21.15.78:0/1432311628 --> [v2:172.21.15.133:6833/3925157967,v1:172.21.15.133:6834/3925157967] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f402c004460 con 0x13f8870 2023-10-11T09:41:00.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.226+0000 7f40377fe700 1 --2- 172.21.15.78:0/1432311628 >> [v2:172.21.15.133:6833/3925157967,v1:172.21.15.133:6834/3925157967] conn(0x13f8870 0x13f8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:00.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.226+0000 7f40377fe700 1 --2- 172.21.15.78:0/1432311628 >> [v2:172.21.15.133:6833/3925157967,v1:172.21.15.133:6834/3925157967] conn(0x13f8870 0x13f8c40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64ddc6e8a4e6286a server_cookie=f94d48e39ca78fc4 in_seq=0 out_seq=0 2023-10-11T09:41:00.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.226+0000 7f40167fc700 10 client.4731 ms_handle_connect on v2:172.21.15.133:6833/3925157967 2023-10-11T09:41:00.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.230+0000 7f40167fc700 1 -- 172.21.15.78:0/1432311628 <== mds.0 v2:172.21.15.133:6833/3925157967 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x7f402c004460 con 0x13f8870 2023-10-11T09:41:00.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.230+0000 7f40167fc700 10 client.4731 handle_command_reply: tid=0 2023-10-11T09:41:00.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.230+0000 7f4041211740 1 client.4731 shutdown 2023-10-11T09:41:00.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.230+0000 7f4041211740 1 -- 172.21.15.78:0/1432311628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x116ceb0 msgr2=0x13dbb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:00.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.230+0000 7f4041211740 1 --2- 172.21.15.78:0/1432311628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x116ceb0 0x13dbb60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f403001f4f0 tx=0x7f4030028000 comp rx=0 tx=0).stop 2023-10-11T09:41:00.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.230+0000 7f4041211740 1 -- 172.21.15.78:0/1432311628 shutdown_connections 2023-10-11T09:41:00.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.230+0000 7f4041211740 1 --2- 172.21.15.78:0/1432311628 >> [v2:172.21.15.133:6833/3925157967,v1:172.21.15.133:6834/3925157967] conn(0x13f8870 0x13f8c40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.230+0000 7f4041211740 1 --2- 172.21.15.78:0/1432311628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x116ceb0 0x13dbb60 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.230+0000 7f4041211740 1 --2- 172.21.15.78:0/1432311628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x116cae0 0x13db620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.230+0000 7f4041211740 1 --2- 172.21.15.78:0/1432311628 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x116c710 0x1480630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.230+0000 7f40167fc700 0 client.4731 ms_handle_reset on v2:172.21.15.133:6833/3925157967 2023-10-11T09:41:00.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.230+0000 7f4041211740 1 -- 172.21.15.78:0/1432311628 >> 172.21.15.78:0/1432311628 conn(0x13d4fa0 msgr2=0x13fc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:00.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.230+0000 7f4041211740 1 -- 172.21.15.78:0/1432311628 shutdown_connections 2023-10-11T09:41:00.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.230+0000 7f4041211740 1 -- 172.21.15.78:0/1432311628 wait complete. 2023-10-11T09:41:00.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.230+0000 7f4041211740 20 client.4731 trim_cache size 0 max 16384 2023-10-11T09:41:00.246 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:41:00.246 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "dee99db9-7a72-406f-a1e2-917943f2b13d", 2023-10-11T09:41:00.246 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2023-10-11T09:41:00.247 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 4306, 2023-10-11T09:41:00.247 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2023-10-11T09:41:00.247 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2023-10-11T09:41:00.247 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2023-10-11T09:41:00.247 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 18.550065336999999, 2023-10-11T09:41:00.248 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 8, 2023-10-11T09:41:00.248 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 23, 2023-10-11T09:41:00.248 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 23, 2023-10-11T09:41:00.249 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 23.901271783999999 2023-10-11T09:41:00.249 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:41:00.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.230+0000 7f403df7f700 1 -- 172.21.15.78:0/2094989675 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4024065ed0 msgr2=0x7f4024068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:00.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.230+0000 7f403df7f700 1 --2- 172.21.15.78:0/2094989675 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4024065ed0 0x7f4024068380 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f403000b370 tx=0x7f403000e000 comp rx=0 tx=0).stop 2023-10-11T09:41:00.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.230+0000 7f403df7f700 1 -- 172.21.15.78:0/2094989675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f403806e1c0 msgr2=0x7f40380b1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:00.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.230+0000 7f403df7f700 1 --2- 172.21.15.78:0/2094989675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f403806e1c0 0x7f40380b1250 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f402800c500 tx=0x7f402800c810 comp rx=0 tx=0).stop 2023-10-11T09:41:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.230+0000 7f403df7f700 1 -- 172.21.15.78:0/2094989675 shutdown_connections 2023-10-11T09:41:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.230+0000 7f403df7f700 1 --2- 172.21.15.78:0/2094989675 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4024065ed0 0x7f4024068380 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.230+0000 7f403df7f700 1 --2- 172.21.15.78:0/2094989675 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4038160dc0 0x7f40381581a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.230+0000 7f403df7f700 1 --2- 172.21.15.78:0/2094989675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f403806fb90 0x7f40380b5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.230+0000 7f403df7f700 1 --2- 172.21.15.78:0/2094989675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f403806e1c0 0x7f40380b1250 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.230+0000 7f403df7f700 1 -- 172.21.15.78:0/2094989675 >> 172.21.15.78:0/2094989675 conn(0x7f40380d0f70 msgr2=0x7f403815b310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:00.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.230+0000 7f403df7f700 1 -- 172.21.15.78:0/2094989675 shutdown_connections 2023-10-11T09:41:00.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.230+0000 7f403df7f700 1 -- 172.21.15.78:0/2094989675 wait complete. 2023-10-11T09:41:00.254 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-10-11T09:41:00.277 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-10-11T09:41:00.278 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-11T09:41:00.278 INFO:tasks.ceph_fuse:client.0 config is {} 2023-10-11T09:41:00.279 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-11T09:41:00.279 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-11T09:41:00.280 INFO:tasks.ceph_fuse:client.2 config is {} 2023-10-11T09:41:00.280 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-11T09:41:00.280 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-11T09:41:00.280 INFO:tasks.ceph_fuse:client.1 config is {} 2023-10-11T09:41:00.281 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-11T09:41:00.281 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-11T09:41:00.281 INFO:tasks.ceph_fuse:client.3 config is {} 2023-10-11T09:41:00.282 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-11T09:41:00.282 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-11T09:41:00.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:00.283 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2023-10-11T09:41:00.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:00.302 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link delete ceph-brx 2023-10-11T09:41:00.383 INFO:teuthology.orchestra.run.smithi133.stderr:Cannot find device "ceph-brx" 2023-10-11T09:41:00.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T09:41:00.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:00.387 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2023-10-11T09:41:00.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:00.408 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link delete ceph-brx 2023-10-11T09:41:00.487 INFO:teuthology.orchestra.run.smithi078.stderr:Cannot find device "ceph-brx" 2023-10-11T09:41:00.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T09:41:00.490 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-10-11T09:41:00.490 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-11T09:41:00.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.694+0000 7f9efb419700 1 Processor -- start 2023-10-11T09:41:00.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.694+0000 7f9efb419700 1 -- start start 2023-10-11T09:41:00.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.694+0000 7f9efb419700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9ef40add00 0x7f9ef4070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:00.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.694+0000 7f9efb419700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9ef415e120 0x7f9ef40ca230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:00.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.694+0000 7f9efb419700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f9ef4071150 0x7f9ef40ca770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:00.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.694+0000 7f9efb419700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ef415a9f0 con 0x7f9ef4071150 2023-10-11T09:41:00.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.694+0000 7f9efb419700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ef40ae0d0 con 0x7f9ef415e120 2023-10-11T09:41:00.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.694+0000 7f9efb419700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef4071f20 con 0x7f9ef40add00 2023-10-11T09:41:00.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.698+0000 7f9efac18700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f9ef4071150 0x7f9ef40ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:00.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.698+0000 7f9efac18700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f9ef4071150 0x7f9ef40ca770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56820/0 (socket says 172.21.15.78:56820) 2023-10-11T09:41:00.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.698+0000 7f9efac18700 1 -- 172.21.15.78:0/3568733665 learned_addr learned my addr 172.21.15.78:0/3568733665 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:00.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.698+0000 7f9ef9c16700 1 --2- 172.21.15.78:0/3568733665 >> v2:172.21.15.133:3300/0 conn(0x7f9ef415e120 0x7f9ef40ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:00.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.698+0000 7f9efa417700 1 --2- 172.21.15.78:0/3568733665 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9ef40add00 0x7f9ef4070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:00.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.698+0000 7f9efac18700 1 -- 172.21.15.78:0/3568733665 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9ef40add00 msgr2=0x7f9ef4070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:00.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.698+0000 7f9efac18700 1 --2- 172.21.15.78:0/3568733665 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9ef40add00 0x7f9ef4070c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.698+0000 7f9efac18700 1 -- 172.21.15.78:0/3568733665 >> v2:172.21.15.133:3300/0 conn(0x7f9ef415e120 msgr2=0x7f9ef40ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:00.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.698+0000 7f9efac18700 1 --2- 172.21.15.78:0/3568733665 >> v2:172.21.15.133:3300/0 conn(0x7f9ef415e120 0x7f9ef40ca230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.698+0000 7f9efac18700 1 -- 172.21.15.78:0/3568733665 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef4071680 con 0x7f9ef4071150 2023-10-11T09:41:00.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.698+0000 7f9ef9c16700 1 --2- 172.21.15.78:0/3568733665 >> v2:172.21.15.133:3300/0 conn(0x7f9ef415e120 0x7f9ef40ca230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:00.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.698+0000 7f9efac18700 1 --2- 172.21.15.78:0/3568733665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ef4071150 0x7f9ef40ca770 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9ef40668a0 tx=0x7f9eec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2287d8fdff4f5a24 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:00.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.698+0000 7f9ef9415700 1 -- 172.21.15.78:0/3568733665 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eec00e070 con 0x7f9ef4071150 2023-10-11T09:41:00.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.698+0000 7f9ef9415700 1 -- 172.21.15.78:0/3568733665 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eec00b9c0 con 0x7f9ef4071150 2023-10-11T09:41:00.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.698+0000 7f9ef9415700 1 -- 172.21.15.78:0/3568733665 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eec00bd70 con 0x7f9ef4071150 2023-10-11T09:41:00.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.698+0000 7f9efb419700 1 -- 172.21.15.78:0/3568733665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ef4071150 msgr2=0x7f9ef40ca770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:00.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.698+0000 7f9efb419700 1 --2- 172.21.15.78:0/3568733665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ef4071150 0x7f9ef40ca770 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9ef40668a0 tx=0x7f9eec00b330 comp rx=0 tx=0).stop 2023-10-11T09:41:00.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.698+0000 7f9efb419700 1 -- 172.21.15.78:0/3568733665 shutdown_connections 2023-10-11T09:41:00.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.698+0000 7f9efb419700 1 --2- 172.21.15.78:0/3568733665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ef4071150 0x7f9ef40ca770 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.698+0000 7f9efb419700 1 --2- 172.21.15.78:0/3568733665 >> v2:172.21.15.133:3300/0 conn(0x7f9ef415e120 0x7f9ef40ca230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.698+0000 7f9efb419700 1 --2- 172.21.15.78:0/3568733665 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9ef40add00 0x7f9ef4070c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.698+0000 7f9efb419700 1 -- 172.21.15.78:0/3568733665 >> 172.21.15.78:0/3568733665 conn(0x7f9ef40cc070 msgr2=0x7f9ef40ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:00.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.698+0000 7f9efb419700 1 -- 172.21.15.78:0/3568733665 shutdown_connections 2023-10-11T09:41:00.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.698+0000 7f9efb419700 1 -- 172.21.15.78:0/3568733665 wait complete. 2023-10-11T09:41:00.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.702+0000 7f9efb419700 1 Processor -- start 2023-10-11T09:41:00.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.702+0000 7f9efb419700 1 -- start start 2023-10-11T09:41:00.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.706+0000 7f9efb419700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ef4071150 0x7f9ef40c2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:00.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.706+0000 7f9efb419700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ef40add00 0x7f9ef414b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:00.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.706+0000 7f9efb419700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9ef415e120 0x7f9ef4150450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:00.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.706+0000 7f9efb419700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef40c4110 con 0x7f9ef40add00 2023-10-11T09:41:00.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.706+0000 7f9efb419700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef40c3500 con 0x7f9ef4071150 2023-10-11T09:41:00.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.706+0000 7f9efb419700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef40c37e0 con 0x7f9ef415e120 2023-10-11T09:41:00.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.706+0000 7f9efa417700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ef4071150 0x7f9ef40c2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:00.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.706+0000 7f9efa417700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ef4071150 0x7f9ef40c2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:37136/0 (socket says 172.21.15.78:37136) 2023-10-11T09:41:00.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.706+0000 7f9efa417700 1 -- 172.21.15.78:0/985540878 learned_addr learned my addr 172.21.15.78:0/985540878 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:00.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.706+0000 7f9ef9c16700 1 --2- 172.21.15.78:0/985540878 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ef40add00 0x7f9ef414b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:00.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.706+0000 7f9efac18700 1 --2- 172.21.15.78:0/985540878 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9ef415e120 0x7f9ef4150450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:00.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.706+0000 7f9efa417700 1 -- 172.21.15.78:0/985540878 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9ef415e120 msgr2=0x7f9ef4150450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:00.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.706+0000 7f9efa417700 1 --2- 172.21.15.78:0/985540878 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9ef415e120 0x7f9ef4150450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.706+0000 7f9efa417700 1 -- 172.21.15.78:0/985540878 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ef40add00 msgr2=0x7f9ef414b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:00.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.706+0000 7f9efa417700 1 --2- 172.21.15.78:0/985540878 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ef40add00 0x7f9ef414b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.706+0000 7f9efa417700 1 -- 172.21.15.78:0/985540878 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eec009b30 con 0x7f9ef4071150 2023-10-11T09:41:00.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.706+0000 7f9ef9c16700 1 --2- 172.21.15.78:0/985540878 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ef40add00 0x7f9ef414b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:00.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.706+0000 7f9efa417700 1 --2- 172.21.15.78:0/985540878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ef4071150 0x7f9ef40c2b00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9ef000e810 tx=0x7f9ef000ebd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:00.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.706+0000 7f9eeaffd700 1 -- 172.21.15.78:0/985540878 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0013610 con 0x7f9ef4071150 2023-10-11T09:41:00.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.706+0000 7f9eeaffd700 1 -- 172.21.15.78:0/985540878 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef0013790 con 0x7f9ef4071150 2023-10-11T09:41:00.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.706+0000 7f9eeaffd700 1 -- 172.21.15.78:0/985540878 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0013b40 con 0x7f9ef4071150 2023-10-11T09:41:00.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.706+0000 7f9efb419700 1 -- 172.21.15.78:0/985540878 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ef40c3ac0 con 0x7f9ef4071150 2023-10-11T09:41:00.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.706+0000 7f9efb419700 1 -- 172.21.15.78:0/985540878 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ef40c3040 con 0x7f9ef4071150 2023-10-11T09:41:00.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.710+0000 7f9efb419700 1 -- 172.21.15.78:0/985540878 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ef4070000 con 0x7f9ef4071150 2023-10-11T09:41:00.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.710+0000 7f9eeaffd700 1 -- 172.21.15.78:0/985540878 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f9ef0013ca0 con 0x7f9ef4071150 2023-10-11T09:41:00.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.710+0000 7f9eeaffd700 1 --2- 172.21.15.78:0/985540878 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9ed8065ed0 0x7f9ed8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:00.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.710+0000 7f9eeaffd700 1 -- 172.21.15.78:0/985540878 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9ef00834e0 con 0x7f9ef4071150 2023-10-11T09:41:00.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.710+0000 7f9ef9c16700 1 --2- 172.21.15.78:0/985540878 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9ed8065ed0 0x7f9ed8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:00.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.714+0000 7f9ef9c16700 1 --2- 172.21.15.78:0/985540878 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9ed8065ed0 0x7f9ed8068380 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4165340 tx=0x7f9ee4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:00.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.718+0000 7f9eeaffd700 1 -- 172.21.15.78:0/985540878 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f9ef0050a90 con 0x7f9ef4071150 2023-10-11T09:41:00.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.934+0000 7f9efb419700 1 -- 172.21.15.78:0/985540878 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9ef4071c20 con 0x7f9ef4071150 2023-10-11T09:41:00.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.934+0000 7f9eeaffd700 1 -- 172.21.15.78:0/985540878 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9ef4071c20 con 0x7f9ef4071150 2023-10-11T09:41:00.940 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-11T09:41:00.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.938+0000 7f9efb419700 1 -- 172.21.15.78:0/985540878 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9ed8065ed0 msgr2=0x7f9ed8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:00.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.938+0000 7f9efb419700 1 --2- 172.21.15.78:0/985540878 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9ed8065ed0 0x7f9ed8068380 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4165340 tx=0x7f9ee4008040 comp rx=0 tx=0).stop 2023-10-11T09:41:00.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.938+0000 7f9efb419700 1 -- 172.21.15.78:0/985540878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ef4071150 msgr2=0x7f9ef40c2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:00.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.938+0000 7f9efb419700 1 --2- 172.21.15.78:0/985540878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ef4071150 0x7f9ef40c2b00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9ef000e810 tx=0x7f9ef000ebd0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.938+0000 7f9efb419700 1 -- 172.21.15.78:0/985540878 shutdown_connections 2023-10-11T09:41:00.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.938+0000 7f9efb419700 1 --2- 172.21.15.78:0/985540878 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9ed8065ed0 0x7f9ed8068380 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.938+0000 7f9efb419700 1 --2- 172.21.15.78:0/985540878 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9ef415e120 0x7f9ef4150450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.938+0000 7f9efb419700 1 --2- 172.21.15.78:0/985540878 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ef40add00 0x7f9ef414b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.938+0000 7f9efb419700 1 --2- 172.21.15.78:0/985540878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ef4071150 0x7f9ef40c2b00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:00.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.938+0000 7f9efb419700 1 -- 172.21.15.78:0/985540878 >> 172.21.15.78:0/985540878 conn(0x7f9ef40cc070 msgr2=0x7f9ef40b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:00.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.938+0000 7f9efb419700 1 -- 172.21.15.78:0/985540878 shutdown_connections 2023-10-11T09:41:00.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:00.938+0000 7f9efb419700 1 -- 172.21.15.78:0/985540878 wait complete. 2023-10-11T09:41:00.961 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-11T09:41:00.961 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-11T09:41:00.961 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi078.front.sepia.ceph.com 2023-10-11T09:41:00.962 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-11T09:41:00.962 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-11T09:41:00.962 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-11T09:41:00.962 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-11T09:41:00.963 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-11T09:41:00.963 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-11T09:41:00.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:00.964 DEBUG:teuthology.orchestra.run.smithi078:> ip addr 2023-10-11T09:41:00.981 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-11T09:41:00.981 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-11T09:41:00.982 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2023-10-11T09:41:00.982 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:41:00.982 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2023-10-11T09:41:00.982 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:41:00.982 INFO:teuthology.orchestra.run.smithi078.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:41:00.983 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:22 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:41:00.983 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f0 2023-10-11T09:41:00.983 INFO:teuthology.orchestra.run.smithi078.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:41:00.983 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:23 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:41:00.983 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f1 2023-10-11T09:41:00.984 INFO:teuthology.orchestra.run.smithi078.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:41:00.984 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dc brd ff:ff:ff:ff:ff:ff 2023-10-11T09:41:00.984 INFO:teuthology.orchestra.run.smithi078.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-11T09:41:00.984 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dd brd ff:ff:ff:ff:ff:ff 2023-10-11T09:41:00.985 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.21.15.78/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-11T09:41:00.985 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft 42403sec preferred_lft 42403sec 2023-10-11T09:41:00.985 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::ec4:7aff:fe8f:d4dd/64 scope link 2023-10-11T09:41:00.985 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:41:00.985 INFO:teuthology.orchestra.run.smithi078.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-11T09:41:00.986 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 02:42:b6:e4:f8:0d brd ff:ff:ff:ff:ff:ff 2023-10-11T09:41:00.986 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-11T09:41:00.986 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:41:00.987 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-11T09:41:00.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:00.988 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec bash -c ' 2023-10-11T09:41:00.988 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2023-10-11T09:41:00.989 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link add name ceph-brx type bridge 2023-10-11T09:41:00.989 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip addr flush dev ceph-brx 2023-10-11T09:41:00.989 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set ceph-brx up 2023-10-11T09:41:00.989 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-11T09:41:00.990 DEBUG:teuthology.orchestra.run.smithi078:> ') 2023-10-11T09:41:01.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:01.140 DEBUG:teuthology.orchestra.run.smithi078:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-11T09:41:01.173 INFO:teuthology.orchestra.run.smithi078.stdout:1 2023-10-11T09:41:01.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:01.176 DEBUG:teuthology.orchestra.run.smithi078:> route 2023-10-11T09:41:01.254 INFO:teuthology.orchestra.run.smithi078.stdout:Kernel IP routing table 2023-10-11T09:41:01.254 INFO:teuthology.orchestra.run.smithi078.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-11T09:41:01.254 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-11T09:41:01.255 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-11T09:41:01.255 INFO:teuthology.orchestra.run.smithi078.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-11T09:41:01.255 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-10-11T09:41:01.255 INFO:teuthology.orchestra.run.smithi078.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-11T09:41:01.255 INFO:teuthology.orchestra.run.smithi078.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-11T09:41:01.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:01.256 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec bash -c ' 2023-10-11T09:41:01.258 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2023-10-11T09:41:01.258 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-11T09:41:01.259 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-11T09:41:01.259 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-11T09:41:01.260 DEBUG:teuthology.orchestra.run.smithi078:> ') 2023-10-11T09:41:01.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:01.382 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2023-10-11T09:41:01.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:01.399 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list-id 2023-10-11T09:41:01.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:01.464 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec bash -c ' 2023-10-11T09:41:01.464 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2023-10-11T09:41:01.464 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-11T09:41:01.465 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-11T09:41:01.465 DEBUG:teuthology.orchestra.run.smithi078:> ') 2023-10-11T09:41:01.585 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-11T09:41:01.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:01.586 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec bash -c ' 2023-10-11T09:41:01.587 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2023-10-11T09:41:01.587 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-11T09:41:01.587 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-11T09:41:01.588 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-11T09:41:01.588 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-11T09:41:01.588 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-11T09:41:01.589 DEBUG:teuthology.orchestra.run.smithi078:> ') 2023-10-11T09:41:01.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:01.763 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec bash -c ' 2023-10-11T09:41:01.763 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2023-10-11T09:41:01.763 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set brx.0 up 2023-10-11T09:41:01.764 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set dev brx.0 master ceph-brx 2023-10-11T09:41:01.764 DEBUG:teuthology.orchestra.run.smithi078:> ') 2023-10-11T09:41:01.837 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-11T09:41:01.838 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T09:41:01.838 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-11T09:41:01.853 INFO:teuthology.orchestra.run.smithi078.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-11T09:41:01.854 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T09:41:01.854 DEBUG:teuthology.orchestra.run.smithi078:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-11T09:41:01.917 DEBUG:teuthology.orchestra.run.smithi078:> sudo modprobe fuse 2023-10-11T09:41:02.004 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T09:41:02.004 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T09:41:02.078 INFO:teuthology.orchestra.run.smithi078.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T09:41:02.082 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T09:41:02.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:02.083 DEBUG:teuthology.orchestra.run.smithi078:> ls /sys/fs/fuse/connections 2023-10-11T09:41:02.141 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-11T09:41:02.142 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-11T09:41:02.191 DEBUG:teuthology.orchestra.run.smithi078:> sudo modprobe fuse 2023-10-11T09:41:02.226 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T09:41:02.226 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T09:41:02.306 INFO:teuthology.orchestra.run.smithi078.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T09:41:02.309 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T09:41:02.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:02.310 DEBUG:teuthology.orchestra.run.smithi078:> ls /sys/fs/fuse/connections 2023-10-11T09:41:02.334 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:2023-10-11T09:41:02.330+0000 7fc1a336d640 -1 init, newargv = 0x56182d7e0820 newargc=15 2023-10-11T09:41:02.334 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:ceph-fuse[42904]: starting ceph client 2023-10-11T09:41:02.348 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:ceph-fuse[42904]: starting fuse 2023-10-11T09:41:03.448 DEBUG:teuthology.orchestra.run.smithi078:> sudo modprobe fuse 2023-10-11T09:41:03.486 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T09:41:03.488 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T09:41:03.560 INFO:teuthology.orchestra.run.smithi078.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T09:41:03.562 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T09:41:03.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:03.564 DEBUG:teuthology.orchestra.run.smithi078:> ls /sys/fs/fuse/connections 2023-10-11T09:41:03.627 INFO:teuthology.orchestra.run.smithi078.stdout:49 2023-10-11T09:41:03.627 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2023-10-11T09:41:03.628 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:41:03.628 DEBUG:teuthology.orchestra.run.smithi078:> import glob 2023-10-11T09:41:03.629 DEBUG:teuthology.orchestra.run.smithi078:> import re 2023-10-11T09:41:03.629 DEBUG:teuthology.orchestra.run.smithi078:> import os 2023-10-11T09:41:03.629 DEBUG:teuthology.orchestra.run.smithi078:> import subprocess 2023-10-11T09:41:03.629 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:41:03.630 DEBUG:teuthology.orchestra.run.smithi078:> def _find_admin_socket(client_name): 2023-10-11T09:41:03.630 DEBUG:teuthology.orchestra.run.smithi078:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-11T09:41:03.631 DEBUG:teuthology.orchestra.run.smithi078:> files = glob.glob(asok_path) 2023-10-11T09:41:03.631 DEBUG:teuthology.orchestra.run.smithi078:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-11T09:41:03.631 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:41:03.632 DEBUG:teuthology.orchestra.run.smithi078:> # Given a non-glob path, it better be there 2023-10-11T09:41:03.632 DEBUG:teuthology.orchestra.run.smithi078:> if "*" not in asok_path: 2023-10-11T09:41:03.632 DEBUG:teuthology.orchestra.run.smithi078:> assert(len(files) == 1) 2023-10-11T09:41:03.632 DEBUG:teuthology.orchestra.run.smithi078:> return files[0] 2023-10-11T09:41:03.633 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:41:03.633 DEBUG:teuthology.orchestra.run.smithi078:> for f in files: 2023-10-11T09:41:03.633 DEBUG:teuthology.orchestra.run.smithi078:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-11T09:41:03.634 DEBUG:teuthology.orchestra.run.smithi078:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-11T09:41:03.634 DEBUG:teuthology.orchestra.run.smithi078:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-11T09:41:03.635 DEBUG:teuthology.orchestra.run.smithi078:> contents = proc_f.read() 2023-10-11T09:41:03.635 DEBUG:teuthology.orchestra.run.smithi078:> if mountpoint in contents: 2023-10-11T09:41:03.635 DEBUG:teuthology.orchestra.run.smithi078:> return f 2023-10-11T09:41:03.636 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-11T09:41:03.636 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:41:03.636 DEBUG:teuthology.orchestra.run.smithi078:> print(_find_admin_socket("client.0")) 2023-10-11T09:41:03.636 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:41:03.787 INFO:teuthology.orchestra.run.smithi078.stdout:/var/run/ceph/ceph-client.0.42904.asok 2023-10-11T09:41:03.972 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.42904.asok 2023-10-11T09:41:03.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:03.972 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.42904.asok status 2023-10-11T09:41:04.171 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:41:04.171 INFO:teuthology.orchestra.run.smithi078.stdout: "metadata": { 2023-10-11T09:41:04.171 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_sha1": "9a774796f0ecdd11a4b771de396e627aa1a187af", 2023-10-11T09:41:04.171 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_version": "ceph version 18.2.0-662-g9a774796 (9a774796f0ecdd11a4b771de396e627aa1a187af) reef (stable)", 2023-10-11T09:41:04.172 INFO:teuthology.orchestra.run.smithi078.stdout: "entity_id": "0", 2023-10-11T09:41:04.172 INFO:teuthology.orchestra.run.smithi078.stdout: "hostname": "smithi078", 2023-10-11T09:41:04.172 INFO:teuthology.orchestra.run.smithi078.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-11T09:41:04.172 INFO:teuthology.orchestra.run.smithi078.stdout: "pid": "42904", 2023-10-11T09:41:04.172 INFO:teuthology.orchestra.run.smithi078.stdout: "root": "/" 2023-10-11T09:41:04.173 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:41:04.173 INFO:teuthology.orchestra.run.smithi078.stdout: "dentry_count": 0, 2023-10-11T09:41:04.173 INFO:teuthology.orchestra.run.smithi078.stdout: "dentry_pinned_count": 0, 2023-10-11T09:41:04.173 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 4746, 2023-10-11T09:41:04.173 INFO:teuthology.orchestra.run.smithi078.stdout: "inst": { 2023-10-11T09:41:04.174 INFO:teuthology.orchestra.run.smithi078.stdout: "name": { 2023-10-11T09:41:04.174 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "client", 2023-10-11T09:41:04.174 INFO:teuthology.orchestra.run.smithi078.stdout: "num": 4746 2023-10-11T09:41:04.174 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:41:04.174 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": { 2023-10-11T09:41:04.175 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2023-10-11T09:41:04.175 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "192.168.0.1:0", 2023-10-11T09:41:04.175 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 1308706829 2023-10-11T09:41:04.175 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:41:04.175 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:41:04.175 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": { 2023-10-11T09:41:04.176 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2023-10-11T09:41:04.176 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "192.168.0.1:0", 2023-10-11T09:41:04.176 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 1308706829 2023-10-11T09:41:04.176 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:41:04.176 INFO:teuthology.orchestra.run.smithi078.stdout: "inst_str": "client.4746 192.168.0.1:0/1308706829", 2023-10-11T09:41:04.177 INFO:teuthology.orchestra.run.smithi078.stdout: "addr_str": "192.168.0.1:0/1308706829", 2023-10-11T09:41:04.177 INFO:teuthology.orchestra.run.smithi078.stdout: "inode_count": 1, 2023-10-11T09:41:04.177 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_epoch": 8, 2023-10-11T09:41:04.177 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_epoch": 23, 2023-10-11T09:41:04.177 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_epoch_barrier": 0, 2023-10-11T09:41:04.177 INFO:teuthology.orchestra.run.smithi078.stdout: "blocklisted": false, 2023-10-11T09:41:04.178 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs" 2023-10-11T09:41:04.178 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:41:04.185 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-11T09:41:04.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.346+0000 7f6c8359e700 1 Processor -- start 2023-10-11T09:41:04.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.346+0000 7f6c8359e700 1 -- start start 2023-10-11T09:41:04.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.350+0000 7f6c8359e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6c8415b8e0 0x7f6c8406d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:04.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.350+0000 7f6c8359e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6c8406d610 0x7f6c840ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:04.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.350+0000 7f6c8359e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f6c8415a940 0x7f6c840ca780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:04.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.350+0000 7f6c8359e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c8415e010 con 0x7f6c8415a940 2023-10-11T09:41:04.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.350+0000 7f6c8359e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c8415e280 con 0x7f6c8415b8e0 2023-10-11T09:41:04.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.350+0000 7f6c8359e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8415e560 con 0x7f6c8406d610 2023-10-11T09:41:04.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.350+0000 7f6c82d9d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f6c8415a940 0x7f6c840ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:04.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.350+0000 7f6c8259c700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6c8415b8e0 0x7f6c8406d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:04.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.350+0000 7f6c7bfff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6c8406d610 0x7f6c840ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:04.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.350+0000 7f6c8259c700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6c8415b8e0 0x7f6c8406d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:37152/0 (socket says 172.21.15.78:37152) 2023-10-11T09:41:04.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.350+0000 7f6c8259c700 1 -- 172.21.15.78:0/558955582 learned_addr learned my addr 172.21.15.78:0/558955582 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:04.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.350+0000 7f6c82d9d700 1 -- 172.21.15.78:0/558955582 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6c8406d610 msgr2=0x7f6c840ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:04.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.350+0000 7f6c82d9d700 1 --2- 172.21.15.78:0/558955582 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6c8406d610 0x7f6c840ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:04.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.350+0000 7f6c82d9d700 1 -- 172.21.15.78:0/558955582 >> v2:172.21.15.133:3300/0 conn(0x7f6c8415b8e0 msgr2=0x7f6c8406d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:04.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.350+0000 7f6c82d9d700 1 --2- 172.21.15.78:0/558955582 >> v2:172.21.15.133:3300/0 conn(0x7f6c8415b8e0 0x7f6c8406d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:04.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.350+0000 7f6c82d9d700 1 -- 172.21.15.78:0/558955582 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c8415e840 con 0x7f6c8415a940 2023-10-11T09:41:04.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.350+0000 7f6c8259c700 1 --2- 172.21.15.78:0/558955582 >> v2:172.21.15.133:3300/0 conn(0x7f6c8415b8e0 0x7f6c8406d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:41:04.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.350+0000 7f6c82d9d700 1 --2- 172.21.15.78:0/558955582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c8415a940 0x7f6c840ca780 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6c840b4cd0 tx=0x7f6c7c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e793cc199239976 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:04.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.354+0000 7f6c81d9b700 1 -- 172.21.15.78:0/558955582 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c00e070 con 0x7f6c8415a940 2023-10-11T09:41:04.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.354+0000 7f6c81d9b700 1 -- 172.21.15.78:0/558955582 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c7c00bb20 con 0x7f6c8415a940 2023-10-11T09:41:04.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.354+0000 7f6c81d9b700 1 -- 172.21.15.78:0/558955582 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c012430 con 0x7f6c8415a940 2023-10-11T09:41:04.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.354+0000 7f6c8359e700 1 -- 172.21.15.78:0/558955582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c8415a940 msgr2=0x7f6c840ca780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:04.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.354+0000 7f6c8359e700 1 --2- 172.21.15.78:0/558955582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c8415a940 0x7f6c840ca780 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6c840b4cd0 tx=0x7f6c7c00b330 comp rx=0 tx=0).stop 2023-10-11T09:41:04.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.354+0000 7f6c8359e700 1 -- 172.21.15.78:0/558955582 shutdown_connections 2023-10-11T09:41:04.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.354+0000 7f6c8359e700 1 --2- 172.21.15.78:0/558955582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c8415a940 0x7f6c840ca780 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:04.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.354+0000 7f6c8359e700 1 --2- 172.21.15.78:0/558955582 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6c8406d610 0x7f6c840ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:04.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.354+0000 7f6c8359e700 1 --2- 172.21.15.78:0/558955582 >> v2:172.21.15.133:3300/0 conn(0x7f6c8415b8e0 0x7f6c8406d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:04.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.354+0000 7f6c8359e700 1 -- 172.21.15.78:0/558955582 >> 172.21.15.78:0/558955582 conn(0x7f6c840cea60 msgr2=0x7f6c840cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:04.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.354+0000 7f6c8359e700 1 -- 172.21.15.78:0/558955582 shutdown_connections 2023-10-11T09:41:04.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.354+0000 7f6c8359e700 1 -- 172.21.15.78:0/558955582 wait complete. 2023-10-11T09:41:04.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.358+0000 7f6c8359e700 1 Processor -- start 2023-10-11T09:41:04.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.358+0000 7f6c8359e700 1 -- start start 2023-10-11T09:41:04.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.358+0000 7f6c8359e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c8406d610 0x7f6c840aae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:04.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.358+0000 7f6c8359e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c8415a940 0x7f6c840b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:04.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.358+0000 7f6c8359e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6c8415b8e0 0x7f6c840a9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:04.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.358+0000 7f6c8359e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c840b24d0 con 0x7f6c8415a940 2023-10-11T09:41:04.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.358+0000 7f6c8359e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c840b27b0 con 0x7f6c8406d610 2023-10-11T09:41:04.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.358+0000 7f6c8359e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c840b2a90 con 0x7f6c8415b8e0 2023-10-11T09:41:04.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.358+0000 7f6c7bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c8415a940 0x7f6c840b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:04.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.358+0000 7f6c7bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c8415a940 0x7f6c840b1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56854/0 (socket says 172.21.15.78:56854) 2023-10-11T09:41:04.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.358+0000 7f6c7bfff700 1 -- 172.21.15.78:0/2257752228 learned_addr learned my addr 172.21.15.78:0/2257752228 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:04.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.358+0000 7f6c82d9d700 1 --2- 172.21.15.78:0/2257752228 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6c8415b8e0 0x7f6c840a9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:04.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.358+0000 7f6c8259c700 1 --2- 172.21.15.78:0/2257752228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c8406d610 0x7f6c840aae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:04.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.358+0000 7f6c7bfff700 1 -- 172.21.15.78:0/2257752228 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6c8415b8e0 msgr2=0x7f6c840a9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:04.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.358+0000 7f6c7bfff700 1 --2- 172.21.15.78:0/2257752228 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6c8415b8e0 0x7f6c840a9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:04.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.358+0000 7f6c7bfff700 1 -- 172.21.15.78:0/2257752228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c8406d610 msgr2=0x7f6c840aae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:04.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.358+0000 7f6c7bfff700 1 --2- 172.21.15.78:0/2257752228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c8406d610 0x7f6c840aae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:04.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.358+0000 7f6c7bfff700 1 -- 172.21.15.78:0/2257752228 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c74009570 con 0x7f6c8415a940 2023-10-11T09:41:04.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.358+0000 7f6c8259c700 1 --2- 172.21.15.78:0/2257752228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c8406d610 0x7f6c840aae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:04.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.358+0000 7f6c7bfff700 1 --2- 172.21.15.78:0/2257752228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c8415a940 0x7f6c840b1a10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00e870 tx=0x7f6c6c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:04.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.358+0000 7f6c7affd700 1 -- 172.21.15.78:0/2257752228 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c0095c0 con 0x7f6c8415a940 2023-10-11T09:41:04.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.358+0000 7f6c7affd700 1 -- 172.21.15.78:0/2257752228 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c6c00ee70 con 0x7f6c8415a940 2023-10-11T09:41:04.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.358+0000 7f6c8359e700 1 -- 172.21.15.78:0/2257752228 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c7c009a70 con 0x7f6c8415a940 2023-10-11T09:41:04.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.358+0000 7f6c7affd700 1 -- 172.21.15.78:0/2257752228 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c00ca70 con 0x7f6c8415a940 2023-10-11T09:41:04.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.362+0000 7f6c8359e700 1 -- 172.21.15.78:0/2257752228 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c840b3050 con 0x7f6c8415a940 2023-10-11T09:41:04.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.362+0000 7f6c8359e700 1 -- 172.21.15.78:0/2257752228 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c84065d40 con 0x7f6c8415a940 2023-10-11T09:41:04.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.362+0000 7f6c7affd700 1 -- 172.21.15.78:0/2257752228 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f6c6c00cbd0 con 0x7f6c8415a940 2023-10-11T09:41:04.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.366+0000 7f6c7affd700 1 --2- 172.21.15.78:0/2257752228 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6c64065ed0 0x7f6c64068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:04.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.366+0000 7f6c7affd700 1 -- 172.21.15.78:0/2257752228 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6c6c083910 con 0x7f6c8415a940 2023-10-11T09:41:04.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.366+0000 7f6c8259c700 1 --2- 172.21.15.78:0/2257752228 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6c64065ed0 0x7f6c64068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:04.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.366+0000 7f6c8259c700 1 --2- 172.21.15.78:0/2257752228 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6c64065ed0 0x7f6c64068380 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6c8415b140 tx=0x7f6c74008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:04.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.374+0000 7f6c7affd700 1 -- 172.21.15.78:0/2257752228 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f6c6c050ec0 con 0x7f6c8415a940 2023-10-11T09:41:04.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.586+0000 7f6c8359e700 1 -- 172.21.15.78:0/2257752228 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6c840cb3f0 con 0x7f6c8415a940 2023-10-11T09:41:04.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.586+0000 7f6c7affd700 1 -- 172.21.15.78:0/2257752228 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6c6c017070 con 0x7f6c8415a940 2023-10-11T09:41:04.592 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-11T09:41:04.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.590+0000 7f6c8359e700 1 -- 172.21.15.78:0/2257752228 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6c64065ed0 msgr2=0x7f6c64068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:04.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.590+0000 7f6c8359e700 1 --2- 172.21.15.78:0/2257752228 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6c64065ed0 0x7f6c64068380 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6c8415b140 tx=0x7f6c74008040 comp rx=0 tx=0).stop 2023-10-11T09:41:04.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.590+0000 7f6c8359e700 1 -- 172.21.15.78:0/2257752228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c8415a940 msgr2=0x7f6c840b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:04.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.590+0000 7f6c8359e700 1 --2- 172.21.15.78:0/2257752228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c8415a940 0x7f6c840b1a10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00e870 tx=0x7f6c6c00ec30 comp rx=0 tx=0).stop 2023-10-11T09:41:04.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.590+0000 7f6c8359e700 1 -- 172.21.15.78:0/2257752228 shutdown_connections 2023-10-11T09:41:04.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.590+0000 7f6c8359e700 1 --2- 172.21.15.78:0/2257752228 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6c64065ed0 0x7f6c64068380 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:04.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.590+0000 7f6c8359e700 1 --2- 172.21.15.78:0/2257752228 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6c8415b8e0 0x7f6c840a9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:04.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.590+0000 7f6c8359e700 1 --2- 172.21.15.78:0/2257752228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c8415a940 0x7f6c840b1a10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:04.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.590+0000 7f6c8359e700 1 --2- 172.21.15.78:0/2257752228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c8406d610 0x7f6c840aae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:04.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.590+0000 7f6c8359e700 1 -- 172.21.15.78:0/2257752228 >> 172.21.15.78:0/2257752228 conn(0x7f6c840cea60 msgr2=0x7f6c8415bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:04.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.590+0000 7f6c8359e700 1 -- 172.21.15.78:0/2257752228 shutdown_connections 2023-10-11T09:41:04.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:04.590+0000 7f6c8359e700 1 -- 172.21.15.78:0/2257752228 wait complete. 2023-10-11T09:41:04.615 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-11T09:41:04.615 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-11T09:41:04.616 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi078.front.sepia.ceph.com 2023-10-11T09:41:04.616 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-11T09:41:04.616 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-11T09:41:04.616 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-11T09:41:04.617 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-11T09:41:04.617 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-11T09:41:04.617 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-11T09:41:04.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:04.618 DEBUG:teuthology.orchestra.run.smithi078:> ip addr 2023-10-11T09:41:04.636 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-11T09:41:04.636 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-11T09:41:04.637 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2023-10-11T09:41:04.637 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:41:04.637 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2023-10-11T09:41:04.637 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:41:04.637 INFO:teuthology.orchestra.run.smithi078.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:41:04.638 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:22 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:41:04.638 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f0 2023-10-11T09:41:04.638 INFO:teuthology.orchestra.run.smithi078.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:41:04.638 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:23 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:41:04.638 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f1 2023-10-11T09:41:04.639 INFO:teuthology.orchestra.run.smithi078.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:41:04.639 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dc brd ff:ff:ff:ff:ff:ff 2023-10-11T09:41:04.639 INFO:teuthology.orchestra.run.smithi078.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-11T09:41:04.639 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dd brd ff:ff:ff:ff:ff:ff 2023-10-11T09:41:04.639 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.21.15.78/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-11T09:41:04.640 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft 42399sec preferred_lft 42399sec 2023-10-11T09:41:04.640 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::ec4:7aff:fe8f:d4dd/64 scope link 2023-10-11T09:41:04.640 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:41:04.640 INFO:teuthology.orchestra.run.smithi078.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-11T09:41:04.640 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 02:42:b6:e4:f8:0d brd ff:ff:ff:ff:ff:ff 2023-10-11T09:41:04.641 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-11T09:41:04.641 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:41:04.641 INFO:teuthology.orchestra.run.smithi078.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-11T09:41:04.641 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 7a:89:8a:a9:e1:23 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:41:04.641 INFO:teuthology.orchestra.run.smithi078.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-11T09:41:04.641 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:41:04.642 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::8ca:ebff:fea9:13f1/64 scope link 2023-10-11T09:41:04.642 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:41:04.642 INFO:teuthology.orchestra.run.smithi078.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-11T09:41:04.642 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 7a:89:8a:a9:e1:23 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-11T09:41:04.642 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::7889:8aff:fea9:e123/64 scope link 2023-10-11T09:41:04.643 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:41:04.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:04.644 DEBUG:teuthology.orchestra.run.smithi078:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-11T09:41:04.720 INFO:teuthology.orchestra.run.smithi078.stdout:1 2023-10-11T09:41:04.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:04.723 DEBUG:teuthology.orchestra.run.smithi078:> route 2023-10-11T09:41:04.788 INFO:teuthology.orchestra.run.smithi078.stdout:Kernel IP routing table 2023-10-11T09:41:04.788 INFO:teuthology.orchestra.run.smithi078.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-11T09:41:04.789 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-11T09:41:04.789 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-11T09:41:04.789 INFO:teuthology.orchestra.run.smithi078.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-11T09:41:04.789 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-10-11T09:41:04.790 INFO:teuthology.orchestra.run.smithi078.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-11T09:41:04.790 INFO:teuthology.orchestra.run.smithi078.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-11T09:41:04.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:04.791 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec bash -c ' 2023-10-11T09:41:04.791 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2023-10-11T09:41:04.791 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-11T09:41:04.792 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-11T09:41:04.792 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-11T09:41:04.792 DEBUG:teuthology.orchestra.run.smithi078:> ') 2023-10-11T09:41:04.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:04.934 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2023-10-11T09:41:04.949 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-11T09:41:04.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:04.951 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list-id 2023-10-11T09:41:05.014 INFO:teuthology.orchestra.run.smithi078.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-11T09:41:05.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:05.016 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec bash -c ' 2023-10-11T09:41:05.016 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2023-10-11T09:41:05.017 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-11T09:41:05.017 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-11T09:41:05.017 DEBUG:teuthology.orchestra.run.smithi078:> ') 2023-10-11T09:41:05.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:05.135 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-11T09:41:05.164 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-11T09:41:05.164 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-11T09:41:05.165 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2023-10-11T09:41:05.165 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:41:05.165 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2023-10-11T09:41:05.165 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:41:05.166 INFO:teuthology.orchestra.run.smithi078.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-11T09:41:05.166 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 22:dc:c1:4a:eb:85 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-11T09:41:05.166 INFO:teuthology.orchestra.run.smithi078.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-11T09:41:05.166 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:41:05.166 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::20dc:c1ff:fe4a:eb85/64 scope link 2023-10-11T09:41:05.167 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:41:05.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:05.168 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-11T09:41:05.247 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-11T09:41:05.248 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-11T09:41:05.248 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2023-10-11T09:41:05.248 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:41:05.248 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2023-10-11T09:41:05.248 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:41:05.249 INFO:teuthology.orchestra.run.smithi078.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-11T09:41:05.249 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 22:dc:c1:4a:eb:85 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-11T09:41:05.249 INFO:teuthology.orchestra.run.smithi078.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-11T09:41:05.249 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:41:05.249 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::20dc:c1ff:fe4a:eb85/64 scope link 2023-10-11T09:41:05.249 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:41:05.252 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-11T09:41:05.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:05.252 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec bash -c ' 2023-10-11T09:41:05.253 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2023-10-11T09:41:05.253 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-11T09:41:05.253 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-11T09:41:05.253 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-11T09:41:05.254 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-11T09:41:05.254 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-11T09:41:05.254 DEBUG:teuthology.orchestra.run.smithi078:> ') 2023-10-11T09:41:05.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:05.449 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec bash -c ' 2023-10-11T09:41:05.449 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2023-10-11T09:41:05.449 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set brx.1 up 2023-10-11T09:41:05.450 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set dev brx.1 master ceph-brx 2023-10-11T09:41:05.450 DEBUG:teuthology.orchestra.run.smithi078:> ') 2023-10-11T09:41:05.529 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-10-11T09:41:05.529 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T09:41:05.529 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-11T09:41:05.546 INFO:teuthology.orchestra.run.smithi078.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-11T09:41:05.547 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T09:41:05.547 DEBUG:teuthology.orchestra.run.smithi078:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-11T09:41:05.608 DEBUG:teuthology.orchestra.run.smithi078:> sudo modprobe fuse 2023-10-11T09:41:05.687 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T09:41:05.687 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T09:41:05.765 INFO:teuthology.orchestra.run.smithi078.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T09:41:05.768 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T09:41:05.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:05.769 DEBUG:teuthology.orchestra.run.smithi078:> ls /sys/fs/fuse/connections 2023-10-11T09:41:05.834 INFO:teuthology.orchestra.run.smithi078.stdout:49 2023-10-11T09:41:05.835 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [49] 2023-10-11T09:41:05.835 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-10-11T09:41:05.883 DEBUG:teuthology.orchestra.run.smithi078:> sudo modprobe fuse 2023-10-11T09:41:05.916 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T09:41:05.917 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T09:41:05.998 INFO:teuthology.orchestra.run.smithi078.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T09:41:06.001 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T09:41:06.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:06.002 DEBUG:teuthology.orchestra.run.smithi078:> ls /sys/fs/fuse/connections 2023-10-11T09:41:06.024 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi078.stderr:2023-10-11T09:41:06.018+0000 7f3d35a38640 -1 init, newargv = 0x55e10852a800 newargc=15 2023-10-11T09:41:06.025 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi078.stderr:ceph-fuse[43046]: starting ceph client 2023-10-11T09:41:06.037 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi078.stderr:ceph-fuse[43046]: starting fuse 2023-10-11T09:41:06.038 INFO:teuthology.orchestra.run.smithi078.stdout:49 2023-10-11T09:41:06.038 INFO:teuthology.orchestra.run.smithi078.stdout:50 2023-10-11T09:41:06.039 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49, 50] 2023-10-11T09:41:06.040 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:41:06.040 DEBUG:teuthology.orchestra.run.smithi078:> import glob 2023-10-11T09:41:06.040 DEBUG:teuthology.orchestra.run.smithi078:> import re 2023-10-11T09:41:06.041 DEBUG:teuthology.orchestra.run.smithi078:> import os 2023-10-11T09:41:06.041 DEBUG:teuthology.orchestra.run.smithi078:> import subprocess 2023-10-11T09:41:06.041 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:41:06.041 DEBUG:teuthology.orchestra.run.smithi078:> def _find_admin_socket(client_name): 2023-10-11T09:41:06.042 DEBUG:teuthology.orchestra.run.smithi078:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-10-11T09:41:06.042 DEBUG:teuthology.orchestra.run.smithi078:> files = glob.glob(asok_path) 2023-10-11T09:41:06.042 DEBUG:teuthology.orchestra.run.smithi078:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-10-11T09:41:06.043 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:41:06.043 DEBUG:teuthology.orchestra.run.smithi078:> # Given a non-glob path, it better be there 2023-10-11T09:41:06.043 DEBUG:teuthology.orchestra.run.smithi078:> if "*" not in asok_path: 2023-10-11T09:41:06.044 DEBUG:teuthology.orchestra.run.smithi078:> assert(len(files) == 1) 2023-10-11T09:41:06.044 DEBUG:teuthology.orchestra.run.smithi078:> return files[0] 2023-10-11T09:41:06.044 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:41:06.045 DEBUG:teuthology.orchestra.run.smithi078:> for f in files: 2023-10-11T09:41:06.045 DEBUG:teuthology.orchestra.run.smithi078:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-11T09:41:06.045 DEBUG:teuthology.orchestra.run.smithi078:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-11T09:41:06.046 DEBUG:teuthology.orchestra.run.smithi078:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-11T09:41:06.046 DEBUG:teuthology.orchestra.run.smithi078:> contents = proc_f.read() 2023-10-11T09:41:06.046 DEBUG:teuthology.orchestra.run.smithi078:> if mountpoint in contents: 2023-10-11T09:41:06.046 DEBUG:teuthology.orchestra.run.smithi078:> return f 2023-10-11T09:41:06.047 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-11T09:41:06.047 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:41:06.047 DEBUG:teuthology.orchestra.run.smithi078:> print(_find_admin_socket("client.2")) 2023-10-11T09:41:06.048 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:41:06.199 INFO:teuthology.orchestra.run.smithi078.stdout:/var/run/ceph/ceph-client.2.43046.asok 2023-10-11T09:41:06.383 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.43046.asok 2023-10-11T09:41:06.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:06.384 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.43046.asok status 2023-10-11T09:41:06.580 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:41:06.716 INFO:teuthology.orchestra.run.smithi078.stdout: "metadata": { 2023-10-11T09:41:06.716 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_sha1": "9a774796f0ecdd11a4b771de396e627aa1a187af", 2023-10-11T09:41:06.716 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_version": "ceph version 18.2.0-662-g9a774796 (9a774796f0ecdd11a4b771de396e627aa1a187af) reef (stable)", 2023-10-11T09:41:06.717 INFO:teuthology.orchestra.run.smithi078.stdout: "entity_id": "2", 2023-10-11T09:41:06.717 INFO:teuthology.orchestra.run.smithi078.stdout: "hostname": "smithi078", 2023-10-11T09:41:06.717 INFO:teuthology.orchestra.run.smithi078.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-10-11T09:41:06.718 INFO:teuthology.orchestra.run.smithi078.stdout: "pid": "43046", 2023-10-11T09:41:06.718 INFO:teuthology.orchestra.run.smithi078.stdout: "root": "/" 2023-10-11T09:41:06.718 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:41:06.718 INFO:teuthology.orchestra.run.smithi078.stdout: "dentry_count": 0, 2023-10-11T09:41:06.719 INFO:teuthology.orchestra.run.smithi078.stdout: "dentry_pinned_count": 0, 2023-10-11T09:41:06.719 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 4758, 2023-10-11T09:41:06.719 INFO:teuthology.orchestra.run.smithi078.stdout: "inst": { 2023-10-11T09:41:06.720 INFO:teuthology.orchestra.run.smithi078.stdout: "name": { 2023-10-11T09:41:06.720 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "client", 2023-10-11T09:41:06.720 INFO:teuthology.orchestra.run.smithi078.stdout: "num": 4758 2023-10-11T09:41:06.720 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:41:06.720 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": { 2023-10-11T09:41:06.721 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2023-10-11T09:41:06.721 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "192.168.0.2:0", 2023-10-11T09:41:06.721 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 515195044 2023-10-11T09:41:06.721 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:41:06.721 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:41:06.722 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": { 2023-10-11T09:41:06.722 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2023-10-11T09:41:06.722 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "192.168.0.2:0", 2023-10-11T09:41:06.722 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 515195044 2023-10-11T09:41:06.723 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:41:06.723 INFO:teuthology.orchestra.run.smithi078.stdout: "inst_str": "client.4758 192.168.0.2:0/515195044", 2023-10-11T09:41:06.723 INFO:teuthology.orchestra.run.smithi078.stdout: "addr_str": "192.168.0.2:0/515195044", 2023-10-11T09:41:06.723 INFO:teuthology.orchestra.run.smithi078.stdout: "inode_count": 1, 2023-10-11T09:41:06.724 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_epoch": 8, 2023-10-11T09:41:06.724 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_epoch": 23, 2023-10-11T09:41:06.724 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_epoch_barrier": 0, 2023-10-11T09:41:06.724 INFO:teuthology.orchestra.run.smithi078.stdout: "blocklisted": false, 2023-10-11T09:41:06.724 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs" 2023-10-11T09:41:06.725 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:41:06.726 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-11T09:41:06.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.894+0000 7ffbf359e700 1 Processor -- start 2023-10-11T09:41:06.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.894+0000 7ffbf359e700 1 -- start start 2023-10-11T09:41:06.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.894+0000 7ffbf359e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ffbf415b8e0 0x7ffbf406d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:06.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.894+0000 7ffbf359e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ffbf406d610 0x7ffbf40ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:06.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.894+0000 7ffbf359e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbf415e010 con 0x7ffbf415a940 2023-10-11T09:41:06.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.894+0000 7ffbf359e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbf415e280 con 0x7ffbf406d610 2023-10-11T09:41:06.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.894+0000 7ffbf359e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbf415e560 con 0x7ffbf415b8e0 2023-10-11T09:41:06.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.894+0000 7ffbe9d9b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ffbf406d610 0x7ffbf40ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:06.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.894+0000 7ffbf2d9d700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7ffbf415a940 0x7ffbf40c9020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:42060/0 (socket says 172.21.15.78:42060) 2023-10-11T09:41:06.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.894+0000 7ffbf2d9d700 1 -- 172.21.15.78:0/809174445 learned_addr learned my addr 172.21.15.78:0/809174445 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:06.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.894+0000 7ffbf259c700 1 --2- 172.21.15.78:0/809174445 >> v2:172.21.15.133:3300/0 conn(0x7ffbf415b8e0 0x7ffbf406d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:06.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.898+0000 7ffbe9d9b700 1 -- 172.21.15.78:0/809174445 >> v2:172.21.15.133:3300/0 conn(0x7ffbf415b8e0 msgr2=0x7ffbf406d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:06.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.898+0000 7ffbe9d9b700 1 --2- 172.21.15.78:0/809174445 >> v2:172.21.15.133:3300/0 conn(0x7ffbf415b8e0 0x7ffbf406d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:06.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.898+0000 7ffbe9d9b700 1 -- 172.21.15.78:0/809174445 >> v1:172.21.15.78:6789/0 conn(0x7ffbf415a940 legacy=0x7ffbf40c9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:06.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.898+0000 7ffbe9d9b700 1 -- 172.21.15.78:0/809174445 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbf415e840 con 0x7ffbf406d610 2023-10-11T09:41:06.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.898+0000 7ffbf1d9b700 1 -- 172.21.15.78:0/809174445 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 429674170 0 0) 0x7ffbf415e010 con 0x7ffbf415a940 2023-10-11T09:41:06.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.898+0000 7ffbe9d9b700 1 --2- 172.21.15.78:0/809174445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbf406d610 0x7ffbf40ca240 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ffbf40b4cd0 tx=0x7ffbdc009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51db9b4889f25705 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.898+0000 7ffbf1d9b700 1 -- 172.21.15.78:0/809174445 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc011830 con 0x7ffbf406d610 2023-10-11T09:41:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.898+0000 7ffbf1d9b700 1 -- 172.21.15.78:0/809174445 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbdc011990 con 0x7ffbf406d610 2023-10-11T09:41:06.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.898+0000 7ffbf1d9b700 1 -- 172.21.15.78:0/809174445 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc011d20 con 0x7ffbf406d610 2023-10-11T09:41:06.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.898+0000 7ffbf359e700 1 -- 172.21.15.78:0/809174445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbf406d610 msgr2=0x7ffbf40ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:06.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.898+0000 7ffbf359e700 1 --2- 172.21.15.78:0/809174445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbf406d610 0x7ffbf40ca240 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ffbf40b4cd0 tx=0x7ffbdc009b30 comp rx=0 tx=0).stop 2023-10-11T09:41:06.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.898+0000 7ffbf359e700 1 -- 172.21.15.78:0/809174445 shutdown_connections 2023-10-11T09:41:06.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.898+0000 7ffbf359e700 1 --2- 172.21.15.78:0/809174445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbf406d610 0x7ffbf40ca240 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:06.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.898+0000 7ffbf359e700 1 --2- 172.21.15.78:0/809174445 >> v2:172.21.15.133:3300/0 conn(0x7ffbf415b8e0 0x7ffbf406d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:06.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.898+0000 7ffbf359e700 1 -- 172.21.15.78:0/809174445 >> 172.21.15.78:0/809174445 conn(0x7ffbf40cea60 msgr2=0x7ffbf40cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:06.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.898+0000 7ffbf359e700 1 -- 172.21.15.78:0/809174445 shutdown_connections 2023-10-11T09:41:06.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.898+0000 7ffbf359e700 1 -- 172.21.15.78:0/809174445 wait complete. 2023-10-11T09:41:06.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.902+0000 7ffbf359e700 1 Processor -- start 2023-10-11T09:41:06.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.902+0000 7ffbf359e700 1 -- start start 2023-10-11T09:41:06.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.906+0000 7ffbf359e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbf406d610 0x7ffbf4139de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:06.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.906+0000 7ffbf359e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbf415a940 0x7ffbf41f1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:06.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.906+0000 7ffbf359e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffbf415b8e0 0x7ffbf40b4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:06.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.906+0000 7ffbf359e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbf40b4a70 con 0x7ffbf406d610 2023-10-11T09:41:06.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.906+0000 7ffbf359e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbf40cda80 con 0x7ffbf415a940 2023-10-11T09:41:06.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.906+0000 7ffbf359e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbf40cd150 con 0x7ffbf415b8e0 2023-10-11T09:41:06.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.906+0000 7ffbf259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbf406d610 0x7ffbf4139de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:06.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.906+0000 7ffbf259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbf406d610 0x7ffbf4139de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53112/0 (socket says 172.21.15.78:53112) 2023-10-11T09:41:06.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.906+0000 7ffbf259c700 1 -- 172.21.15.78:0/2833610897 learned_addr learned my addr 172.21.15.78:0/2833610897 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:06.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.906+0000 7ffbf2d9d700 1 --2- 172.21.15.78:0/2833610897 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffbf415b8e0 0x7ffbf40b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:06.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.906+0000 7ffbe9d9b700 1 --2- 172.21.15.78:0/2833610897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbf415a940 0x7ffbf41f1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:06.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.906+0000 7ffbf259c700 1 -- 172.21.15.78:0/2833610897 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffbf415b8e0 msgr2=0x7ffbf40b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:06.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.906+0000 7ffbf259c700 1 --2- 172.21.15.78:0/2833610897 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffbf415b8e0 0x7ffbf40b4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:06.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.906+0000 7ffbf259c700 1 -- 172.21.15.78:0/2833610897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbf415a940 msgr2=0x7ffbf41f1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:06.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.906+0000 7ffbf259c700 1 --2- 172.21.15.78:0/2833610897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbf415a940 0x7ffbf41f1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:06.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.906+0000 7ffbf259c700 1 -- 172.21.15.78:0/2833610897 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbdc0094b0 con 0x7ffbf406d610 2023-10-11T09:41:06.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.906+0000 7ffbf259c700 1 --2- 172.21.15.78:0/2833610897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbf406d610 0x7ffbf4139de0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ffbe400e870 tx=0x7ffbe400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:06.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.906+0000 7ffbeb7fe700 1 -- 172.21.15.78:0/2833610897 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe40095a0 con 0x7ffbf406d610 2023-10-11T09:41:06.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.906+0000 7ffbeb7fe700 1 -- 172.21.15.78:0/2833610897 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbe400ee70 con 0x7ffbf406d610 2023-10-11T09:41:06.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.906+0000 7ffbeb7fe700 1 -- 172.21.15.78:0/2833610897 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe400ca50 con 0x7ffbf406d610 2023-10-11T09:41:06.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.906+0000 7ffbf359e700 1 -- 172.21.15.78:0/2833610897 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbf40cd430 con 0x7ffbf406d610 2023-10-11T09:41:06.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.906+0000 7ffbf359e700 1 -- 172.21.15.78:0/2833610897 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbf40b46d0 con 0x7ffbf406d610 2023-10-11T09:41:06.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.910+0000 7ffbf359e700 1 -- 172.21.15.78:0/2833610897 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbf4065d40 con 0x7ffbf406d610 2023-10-11T09:41:06.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.910+0000 7ffbeb7fe700 1 -- 172.21.15.78:0/2833610897 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7ffbe400cbb0 con 0x7ffbf406d610 2023-10-11T09:41:06.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.910+0000 7ffbeb7fe700 1 --2- 172.21.15.78:0/2833610897 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffbd4065ed0 0x7ffbd4068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:06.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.914+0000 7ffbeb7fe700 1 -- 172.21.15.78:0/2833610897 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffbe4083780 con 0x7ffbf406d610 2023-10-11T09:41:06.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.914+0000 7ffbe9d9b700 1 --2- 172.21.15.78:0/2833610897 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffbd4065ed0 0x7ffbd4068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:06.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.914+0000 7ffbe9d9b700 1 --2- 172.21.15.78:0/2833610897 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffbd4065ed0 0x7ffbd4068380 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc0090b0 tx=0x7ffbdc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:06.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:06.918+0000 7ffbeb7fe700 1 -- 172.21.15.78:0/2833610897 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7ffbe4050dc0 con 0x7ffbf406d610 2023-10-11T09:41:07.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:07.130+0000 7ffbf359e700 1 -- 172.21.15.78:0/2833610897 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ffbf40cc140 con 0x7ffbf406d610 2023-10-11T09:41:07.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:07.130+0000 7ffbeb7fe700 1 -- 172.21.15.78:0/2833610897 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7ffbe40543e0 con 0x7ffbf406d610 2023-10-11T09:41:07.136 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-11T09:41:07.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:07.134+0000 7ffbf359e700 1 -- 172.21.15.78:0/2833610897 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffbd4065ed0 msgr2=0x7ffbd4068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:07.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:07.134+0000 7ffbf359e700 1 --2- 172.21.15.78:0/2833610897 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffbd4065ed0 0x7ffbd4068380 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc0090b0 tx=0x7ffbdc008030 comp rx=0 tx=0).stop 2023-10-11T09:41:07.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:07.134+0000 7ffbf359e700 1 -- 172.21.15.78:0/2833610897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbf406d610 msgr2=0x7ffbf4139de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:07.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:07.134+0000 7ffbf359e700 1 --2- 172.21.15.78:0/2833610897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbf406d610 0x7ffbf4139de0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ffbe400e870 tx=0x7ffbe400ec30 comp rx=0 tx=0).stop 2023-10-11T09:41:07.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:07.134+0000 7ffbf359e700 1 -- 172.21.15.78:0/2833610897 shutdown_connections 2023-10-11T09:41:07.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:07.134+0000 7ffbf359e700 1 --2- 172.21.15.78:0/2833610897 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffbd4065ed0 0x7ffbd4068380 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:07.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:07.134+0000 7ffbf359e700 1 --2- 172.21.15.78:0/2833610897 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffbf415b8e0 0x7ffbf40b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:07.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:07.134+0000 7ffbf359e700 1 --2- 172.21.15.78:0/2833610897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbf415a940 0x7ffbf41f1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:07.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:07.134+0000 7ffbf359e700 1 --2- 172.21.15.78:0/2833610897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbf406d610 0x7ffbf4139de0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:07.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:07.134+0000 7ffbf359e700 1 -- 172.21.15.78:0/2833610897 >> 172.21.15.78:0/2833610897 conn(0x7ffbf40cea60 msgr2=0x7ffbf40cba30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:07.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:07.134+0000 7ffbf359e700 1 -- 172.21.15.78:0/2833610897 shutdown_connections 2023-10-11T09:41:07.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:07.134+0000 7ffbf359e700 1 -- 172.21.15.78:0/2833610897 wait complete. 2023-10-11T09:41:07.160 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-11T09:41:07.160 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-11T09:41:07.161 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi133.front.sepia.ceph.com 2023-10-11T09:41:07.161 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-11T09:41:07.162 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-11T09:41:07.162 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-11T09:41:07.163 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-11T09:41:07.163 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-11T09:41:07.163 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-11T09:41:07.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:07.164 DEBUG:teuthology.orchestra.run.smithi133:> ip addr 2023-10-11T09:41:07.186 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-11T09:41:07.187 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-11T09:41:07.187 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2023-10-11T09:41:07.187 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:41:07.187 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2023-10-11T09:41:07.188 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:41:07.188 INFO:teuthology.orchestra.run.smithi133.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:41:07.189 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b2 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:41:07.189 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f0 2023-10-11T09:41:07.189 INFO:teuthology.orchestra.run.smithi133.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:41:07.190 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b3 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:41:07.190 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f1 2023-10-11T09:41:07.190 INFO:teuthology.orchestra.run.smithi133.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:41:07.191 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:86 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:41:07.191 INFO:teuthology.orchestra.run.smithi133.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-11T09:41:07.191 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:87 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:41:07.191 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.21.15.133/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-11T09:41:07.192 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft 42399sec preferred_lft 42399sec 2023-10-11T09:41:07.192 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::ec4:7aff:fe88:8087/64 scope link 2023-10-11T09:41:07.193 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:41:07.193 INFO:teuthology.orchestra.run.smithi133.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-11T09:41:07.193 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 02:42:dc:fa:bd:f4 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:41:07.194 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-11T09:41:07.194 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:41:07.195 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-11T09:41:07.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:07.195 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2023-10-11T09:41:07.196 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2023-10-11T09:41:07.196 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link add name ceph-brx type bridge 2023-10-11T09:41:07.196 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip addr flush dev ceph-brx 2023-10-11T09:41:07.197 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set ceph-brx up 2023-10-11T09:41:07.197 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-11T09:41:07.198 DEBUG:teuthology.orchestra.run.smithi133:> ') 2023-10-11T09:41:07.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:07.338 DEBUG:teuthology.orchestra.run.smithi133:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-11T09:41:07.372 INFO:teuthology.orchestra.run.smithi133.stdout:1 2023-10-11T09:41:07.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:07.375 DEBUG:teuthology.orchestra.run.smithi133:> route 2023-10-11T09:41:07.455 INFO:teuthology.orchestra.run.smithi133.stdout:Kernel IP routing table 2023-10-11T09:41:07.455 INFO:teuthology.orchestra.run.smithi133.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-11T09:41:07.455 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-11T09:41:07.456 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-11T09:41:07.456 INFO:teuthology.orchestra.run.smithi133.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-11T09:41:07.456 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-10-11T09:41:07.456 INFO:teuthology.orchestra.run.smithi133.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-11T09:41:07.457 INFO:teuthology.orchestra.run.smithi133.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-11T09:41:07.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:07.458 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2023-10-11T09:41:07.459 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2023-10-11T09:41:07.459 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-11T09:41:07.459 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-11T09:41:07.460 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-11T09:41:07.460 DEBUG:teuthology.orchestra.run.smithi133:> ') 2023-10-11T09:41:07.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:07.593 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2023-10-11T09:41:07.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:07.609 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list-id 2023-10-11T09:41:07.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:07.670 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2023-10-11T09:41:07.671 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2023-10-11T09:41:07.671 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-11T09:41:07.671 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-11T09:41:07.671 DEBUG:teuthology.orchestra.run.smithi133:> ') 2023-10-11T09:41:07.786 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-11T09:41:07.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:07.787 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2023-10-11T09:41:07.787 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2023-10-11T09:41:07.787 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-11T09:41:07.788 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-11T09:41:07.788 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-11T09:41:07.788 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-11T09:41:07.788 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-11T09:41:07.789 DEBUG:teuthology.orchestra.run.smithi133:> ') 2023-10-11T09:41:07.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:07.966 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2023-10-11T09:41:07.967 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2023-10-11T09:41:07.967 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set brx.0 up 2023-10-11T09:41:07.967 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set dev brx.0 master ceph-brx 2023-10-11T09:41:07.967 DEBUG:teuthology.orchestra.run.smithi133:> ') 2023-10-11T09:41:08.041 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-11T09:41:08.042 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T09:41:08.042 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-11T09:41:08.061 INFO:teuthology.orchestra.run.smithi133.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-11T09:41:08.061 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T09:41:08.062 DEBUG:teuthology.orchestra.run.smithi133:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-11T09:41:08.121 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2023-10-11T09:41:08.200 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T09:41:08.201 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T09:41:08.278 INFO:teuthology.orchestra.run.smithi133.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T09:41:08.281 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T09:41:08.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:08.283 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2023-10-11T09:41:08.340 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-11T09:41:08.341 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-11T09:41:08.390 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2023-10-11T09:41:08.421 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T09:41:08.422 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T09:41:08.496 INFO:teuthology.orchestra.run.smithi133.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T09:41:08.497 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T09:41:08.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:08.499 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2023-10-11T09:41:08.526 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi133.stderr:2023-10-11T09:41:08.523+0000 7f0bf180e640 -1 init, newargv = 0x5588fa2b7f20 newargc=15 2023-10-11T09:41:08.527 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi133.stderr:ceph-fuse[40858]: starting ceph client 2023-10-11T09:41:08.538 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi133.stderr:ceph-fuse[40858]: starting fuse 2023-10-11T09:41:08.539 INFO:teuthology.orchestra.run.smithi133.stdout:49 2023-10-11T09:41:08.539 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2023-10-11T09:41:08.540 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:41:08.540 DEBUG:teuthology.orchestra.run.smithi133:> import glob 2023-10-11T09:41:08.540 DEBUG:teuthology.orchestra.run.smithi133:> import re 2023-10-11T09:41:08.541 DEBUG:teuthology.orchestra.run.smithi133:> import os 2023-10-11T09:41:08.541 DEBUG:teuthology.orchestra.run.smithi133:> import subprocess 2023-10-11T09:41:08.541 DEBUG:teuthology.orchestra.run.smithi133:> 2023-10-11T09:41:08.542 DEBUG:teuthology.orchestra.run.smithi133:> def _find_admin_socket(client_name): 2023-10-11T09:41:08.542 DEBUG:teuthology.orchestra.run.smithi133:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-11T09:41:08.542 DEBUG:teuthology.orchestra.run.smithi133:> files = glob.glob(asok_path) 2023-10-11T09:41:08.542 DEBUG:teuthology.orchestra.run.smithi133:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-11T09:41:08.543 DEBUG:teuthology.orchestra.run.smithi133:> 2023-10-11T09:41:08.543 DEBUG:teuthology.orchestra.run.smithi133:> # Given a non-glob path, it better be there 2023-10-11T09:41:08.543 DEBUG:teuthology.orchestra.run.smithi133:> if "*" not in asok_path: 2023-10-11T09:41:08.544 DEBUG:teuthology.orchestra.run.smithi133:> assert(len(files) == 1) 2023-10-11T09:41:08.544 DEBUG:teuthology.orchestra.run.smithi133:> return files[0] 2023-10-11T09:41:08.544 DEBUG:teuthology.orchestra.run.smithi133:> 2023-10-11T09:41:08.545 DEBUG:teuthology.orchestra.run.smithi133:> for f in files: 2023-10-11T09:41:08.545 DEBUG:teuthology.orchestra.run.smithi133:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-11T09:41:08.545 DEBUG:teuthology.orchestra.run.smithi133:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-11T09:41:08.546 DEBUG:teuthology.orchestra.run.smithi133:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-11T09:41:08.546 DEBUG:teuthology.orchestra.run.smithi133:> contents = proc_f.read() 2023-10-11T09:41:08.546 DEBUG:teuthology.orchestra.run.smithi133:> if mountpoint in contents: 2023-10-11T09:41:08.546 DEBUG:teuthology.orchestra.run.smithi133:> return f 2023-10-11T09:41:08.547 DEBUG:teuthology.orchestra.run.smithi133:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-11T09:41:08.547 DEBUG:teuthology.orchestra.run.smithi133:> 2023-10-11T09:41:08.547 DEBUG:teuthology.orchestra.run.smithi133:> print(_find_admin_socket("client.1")) 2023-10-11T09:41:08.548 DEBUG:teuthology.orchestra.run.smithi133:> ' 2023-10-11T09:41:08.698 INFO:teuthology.orchestra.run.smithi133.stdout:/var/run/ceph/ceph-client.1.40858.asok 2023-10-11T09:41:08.883 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.40858.asok 2023-10-11T09:41:08.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:08.883 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.40858.asok status 2023-10-11T09:41:09.081 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-10-11T09:41:09.081 INFO:teuthology.orchestra.run.smithi133.stdout: "metadata": { 2023-10-11T09:41:09.082 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_sha1": "9a774796f0ecdd11a4b771de396e627aa1a187af", 2023-10-11T09:41:09.082 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_version": "ceph version 18.2.0-662-g9a774796 (9a774796f0ecdd11a4b771de396e627aa1a187af) reef (stable)", 2023-10-11T09:41:09.082 INFO:teuthology.orchestra.run.smithi133.stdout: "entity_id": "1", 2023-10-11T09:41:09.082 INFO:teuthology.orchestra.run.smithi133.stdout: "hostname": "smithi133", 2023-10-11T09:41:09.082 INFO:teuthology.orchestra.run.smithi133.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-11T09:41:09.083 INFO:teuthology.orchestra.run.smithi133.stdout: "pid": "40858", 2023-10-11T09:41:09.083 INFO:teuthology.orchestra.run.smithi133.stdout: "root": "/" 2023-10-11T09:41:09.083 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-10-11T09:41:09.083 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_count": 0, 2023-10-11T09:41:09.083 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_pinned_count": 0, 2023-10-11T09:41:09.084 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 4592, 2023-10-11T09:41:09.084 INFO:teuthology.orchestra.run.smithi133.stdout: "inst": { 2023-10-11T09:41:09.084 INFO:teuthology.orchestra.run.smithi133.stdout: "name": { 2023-10-11T09:41:09.084 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "client", 2023-10-11T09:41:09.084 INFO:teuthology.orchestra.run.smithi133.stdout: "num": 4592 2023-10-11T09:41:09.085 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-10-11T09:41:09.085 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2023-10-11T09:41:09.085 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2023-10-11T09:41:09.085 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2023-10-11T09:41:09.085 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 293804017 2023-10-11T09:41:09.085 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-10-11T09:41:09.086 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-10-11T09:41:09.086 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2023-10-11T09:41:09.086 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2023-10-11T09:41:09.086 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2023-10-11T09:41:09.086 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 293804017 2023-10-11T09:41:09.087 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-10-11T09:41:09.087 INFO:teuthology.orchestra.run.smithi133.stdout: "inst_str": "client.4592 192.168.0.1:0/293804017", 2023-10-11T09:41:09.087 INFO:teuthology.orchestra.run.smithi133.stdout: "addr_str": "192.168.0.1:0/293804017", 2023-10-11T09:41:09.087 INFO:teuthology.orchestra.run.smithi133.stdout: "inode_count": 1, 2023-10-11T09:41:09.087 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_epoch": 8, 2023-10-11T09:41:09.088 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch": 23, 2023-10-11T09:41:09.088 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch_barrier": 0, 2023-10-11T09:41:09.088 INFO:teuthology.orchestra.run.smithi133.stdout: "blocklisted": false, 2023-10-11T09:41:09.088 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs" 2023-10-11T09:41:09.088 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-10-11T09:41:09.095 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-11T09:41:09.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.262+0000 7fc85933d700 1 Processor -- start 2023-10-11T09:41:09.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.262+0000 7fc85933d700 1 -- start start 2023-10-11T09:41:09.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.262+0000 7fc85933d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc8540b2050 0x7fc854072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:09.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.262+0000 7fc85933d700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc8540b2420 0x7fc85415dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:09.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.262+0000 7fc85933d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fc85415e530 0x7fc85415e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:09.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.262+0000 7fc85933d700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8540cac50 con 0x7fc85415e530 2023-10-11T09:41:09.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.262+0000 7fc85933d700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc85415eec0 con 0x7fc8540b2420 2023-10-11T09:41:09.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.262+0000 7fc85933d700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc854072870 con 0x7fc8540b2050 2023-10-11T09:41:09.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.262+0000 7fc853fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fc85415e530 0x7fc85415e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:09.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.266+0000 7fc853fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fc85415e530 0x7fc85415e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53114/0 (socket says 172.21.15.78:53114) 2023-10-11T09:41:09.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.266+0000 7fc852ffd700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc8540b2050 0x7fc854072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:09.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.266+0000 7fc853fff700 1 -- 172.21.15.78:0/1479899907 learned_addr learned my addr 172.21.15.78:0/1479899907 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:09.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.266+0000 7fc8527fc700 1 --2- 172.21.15.78:0/1479899907 >> v2:172.21.15.133:3300/0 conn(0x7fc8540b2420 0x7fc85415dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:09.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.266+0000 7fc853fff700 1 -- 172.21.15.78:0/1479899907 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc8540b2050 msgr2=0x7fc854072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:09.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.266+0000 7fc853fff700 1 --2- 172.21.15.78:0/1479899907 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc8540b2050 0x7fc854072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:09.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.266+0000 7fc853fff700 1 -- 172.21.15.78:0/1479899907 >> v2:172.21.15.133:3300/0 conn(0x7fc8540b2420 msgr2=0x7fc85415dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:09.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.266+0000 7fc853fff700 1 --2- 172.21.15.78:0/1479899907 >> v2:172.21.15.133:3300/0 conn(0x7fc8540b2420 0x7fc85415dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:09.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.266+0000 7fc853fff700 1 -- 172.21.15.78:0/1479899907 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8540c6a30 con 0x7fc85415e530 2023-10-11T09:41:09.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.266+0000 7fc853fff700 1 --2- 172.21.15.78:0/1479899907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc85415e530 0x7fc85415e980 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc85406cd10 tx=0x7fc84c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97fc9eddb453ed9e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:09.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.266+0000 7fc851ffb700 1 -- 172.21.15.78:0/1479899907 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c00e070 con 0x7fc85415e530 2023-10-11T09:41:09.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.266+0000 7fc851ffb700 1 -- 172.21.15.78:0/1479899907 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc84c00b8d0 con 0x7fc85415e530 2023-10-11T09:41:09.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.266+0000 7fc851ffb700 1 -- 172.21.15.78:0/1479899907 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c00bc80 con 0x7fc85415e530 2023-10-11T09:41:09.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.266+0000 7fc85933d700 1 -- 172.21.15.78:0/1479899907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc85415e530 msgr2=0x7fc85415e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:09.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.266+0000 7fc85933d700 1 --2- 172.21.15.78:0/1479899907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc85415e530 0x7fc85415e980 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc85406cd10 tx=0x7fc84c00b330 comp rx=0 tx=0).stop 2023-10-11T09:41:09.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.266+0000 7fc85933d700 1 -- 172.21.15.78:0/1479899907 shutdown_connections 2023-10-11T09:41:09.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.266+0000 7fc85933d700 1 --2- 172.21.15.78:0/1479899907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc85415e530 0x7fc85415e980 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:09.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.266+0000 7fc85933d700 1 --2- 172.21.15.78:0/1479899907 >> v2:172.21.15.133:3300/0 conn(0x7fc8540b2420 0x7fc85415dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:09.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.266+0000 7fc85933d700 1 --2- 172.21.15.78:0/1479899907 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc8540b2050 0x7fc854072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:09.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.266+0000 7fc85933d700 1 -- 172.21.15.78:0/1479899907 >> 172.21.15.78:0/1479899907 conn(0x7fc85406dd80 msgr2=0x7fc85406e150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:09.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.266+0000 7fc85933d700 1 -- 172.21.15.78:0/1479899907 shutdown_connections 2023-10-11T09:41:09.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.266+0000 7fc85933d700 1 -- 172.21.15.78:0/1479899907 wait complete. 2023-10-11T09:41:09.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.270+0000 7fc85933d700 1 Processor -- start 2023-10-11T09:41:09.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.270+0000 7fc85933d700 1 -- start start 2023-10-11T09:41:09.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.274+0000 7fc85933d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8540b2050 0x7fc8540c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:09.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.274+0000 7fc85933d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc8540b2420 0x7fc8540c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:09.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.274+0000 7fc85933d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc85415e530 0x7fc8540c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:09.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.274+0000 7fc85933d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc854153900 con 0x7fc8540b2050 2023-10-11T09:41:09.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.274+0000 7fc85933d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8540add20 con 0x7fc85415e530 2023-10-11T09:41:09.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.274+0000 7fc85933d700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8540cebc0 con 0x7fc8540b2420 2023-10-11T09:41:09.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.274+0000 7fc852ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8540b2050 0x7fc8540c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:09.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.274+0000 7fc852ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8540b2050 0x7fc8540c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53120/0 (socket says 172.21.15.78:53120) 2023-10-11T09:41:09.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.274+0000 7fc852ffd700 1 -- 172.21.15.78:0/621759625 learned_addr learned my addr 172.21.15.78:0/621759625 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:09.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.274+0000 7fc853fff700 1 --2- 172.21.15.78:0/621759625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc85415e530 0x7fc8540c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:09.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.274+0000 7fc8527fc700 1 --2- 172.21.15.78:0/621759625 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc8540b2420 0x7fc8540c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:09.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.274+0000 7fc852ffd700 1 -- 172.21.15.78:0/621759625 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc8540b2420 msgr2=0x7fc8540c8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:09.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.274+0000 7fc852ffd700 1 --2- 172.21.15.78:0/621759625 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc8540b2420 0x7fc8540c8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:09.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.274+0000 7fc852ffd700 1 -- 172.21.15.78:0/621759625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc85415e530 msgr2=0x7fc8540c55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:09.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.274+0000 7fc852ffd700 1 --2- 172.21.15.78:0/621759625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc85415e530 0x7fc8540c55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:09.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.274+0000 7fc852ffd700 1 -- 172.21.15.78:0/621759625 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc84c009b30 con 0x7fc8540b2050 2023-10-11T09:41:09.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.274+0000 7fc852ffd700 1 --2- 172.21.15.78:0/621759625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8540b2050 0x7fc8540c9dc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc84400e870 tx=0x7fc84400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:09.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.274+0000 7fc83b7fe700 1 -- 172.21.15.78:0/621759625 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc844009540 con 0x7fc8540b2050 2023-10-11T09:41:09.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.274+0000 7fc83b7fe700 1 -- 172.21.15.78:0/621759625 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc844013680 con 0x7fc8540b2050 2023-10-11T09:41:09.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.274+0000 7fc85933d700 1 -- 172.21.15.78:0/621759625 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8540ce290 con 0x7fc8540b2050 2023-10-11T09:41:09.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.274+0000 7fc83b7fe700 1 -- 172.21.15.78:0/621759625 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc844013a10 con 0x7fc8540b2050 2023-10-11T09:41:09.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.274+0000 7fc85933d700 1 -- 172.21.15.78:0/621759625 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8540ce850 con 0x7fc8540b2050 2023-10-11T09:41:09.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.278+0000 7fc85933d700 1 -- 172.21.15.78:0/621759625 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8540c6fb0 con 0x7fc8540b2050 2023-10-11T09:41:09.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.278+0000 7fc83b7fe700 1 -- 172.21.15.78:0/621759625 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fc844013b70 con 0x7fc8540b2050 2023-10-11T09:41:09.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.282+0000 7fc83b7fe700 1 --2- 172.21.15.78:0/621759625 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc840065f00 0x7fc8400683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:09.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.282+0000 7fc83b7fe700 1 -- 172.21.15.78:0/621759625 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc844083620 con 0x7fc8540b2050 2023-10-11T09:41:09.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.282+0000 7fc8527fc700 1 --2- 172.21.15.78:0/621759625 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc840065f00 0x7fc8400683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:09.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.282+0000 7fc8527fc700 1 --2- 172.21.15.78:0/621759625 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc840065f00 0x7fc8400683b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc8541544b0 tx=0x7fc848009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:09.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.286+0000 7fc83b7fe700 1 -- 172.21.15.78:0/621759625 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fc8540c6fb0 con 0x7fc8540b2050 2023-10-11T09:41:09.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.506+0000 7fc85933d700 1 -- 172.21.15.78:0/621759625 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc8540c6fb0 con 0x7fc8540b2050 2023-10-11T09:41:09.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.506+0000 7fc83b7fe700 1 -- 172.21.15.78:0/621759625 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc8540c6fb0 con 0x7fc8540b2050 2023-10-11T09:41:09.511 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-11T09:41:09.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.510+0000 7fc85933d700 1 -- 172.21.15.78:0/621759625 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc840065f00 msgr2=0x7fc8400683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:09.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.510+0000 7fc85933d700 1 --2- 172.21.15.78:0/621759625 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc840065f00 0x7fc8400683b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc8541544b0 tx=0x7fc848009240 comp rx=0 tx=0).stop 2023-10-11T09:41:09.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.510+0000 7fc85933d700 1 -- 172.21.15.78:0/621759625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8540b2050 msgr2=0x7fc8540c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:09.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.510+0000 7fc85933d700 1 --2- 172.21.15.78:0/621759625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8540b2050 0x7fc8540c9dc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc84400e870 tx=0x7fc84400eb80 comp rx=0 tx=0).stop 2023-10-11T09:41:09.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.510+0000 7fc85933d700 1 -- 172.21.15.78:0/621759625 shutdown_connections 2023-10-11T09:41:09.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.510+0000 7fc85933d700 1 --2- 172.21.15.78:0/621759625 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc840065f00 0x7fc8400683b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:09.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.510+0000 7fc85933d700 1 --2- 172.21.15.78:0/621759625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc85415e530 0x7fc8540c55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:09.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.510+0000 7fc85933d700 1 --2- 172.21.15.78:0/621759625 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc8540b2420 0x7fc8540c8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:09.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.510+0000 7fc85933d700 1 --2- 172.21.15.78:0/621759625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8540b2050 0x7fc8540c9dc0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:09.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.510+0000 7fc85933d700 1 -- 172.21.15.78:0/621759625 >> 172.21.15.78:0/621759625 conn(0x7fc85406dd80 msgr2=0x7fc85406e150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:09.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.510+0000 7fc85933d700 1 -- 172.21.15.78:0/621759625 shutdown_connections 2023-10-11T09:41:09.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:09.510+0000 7fc85933d700 1 -- 172.21.15.78:0/621759625 wait complete. 2023-10-11T09:41:09.534 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-11T09:41:09.534 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-11T09:41:09.535 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi133.front.sepia.ceph.com 2023-10-11T09:41:09.535 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-11T09:41:09.535 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-11T09:41:09.535 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-11T09:41:09.536 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-11T09:41:09.536 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-11T09:41:09.536 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-11T09:41:09.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:09.537 DEBUG:teuthology.orchestra.run.smithi133:> ip addr 2023-10-11T09:41:09.557 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-11T09:41:09.557 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-11T09:41:09.558 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2023-10-11T09:41:09.558 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:41:09.558 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2023-10-11T09:41:09.558 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:41:09.558 INFO:teuthology.orchestra.run.smithi133.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:41:09.559 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b2 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:41:09.559 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f0 2023-10-11T09:41:09.559 INFO:teuthology.orchestra.run.smithi133.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:41:09.559 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b3 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:41:09.559 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f1 2023-10-11T09:41:09.560 INFO:teuthology.orchestra.run.smithi133.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:41:09.560 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:86 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:41:09.560 INFO:teuthology.orchestra.run.smithi133.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-11T09:41:09.560 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:87 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:41:09.560 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.21.15.133/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-11T09:41:09.560 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft 42397sec preferred_lft 42397sec 2023-10-11T09:41:09.561 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::ec4:7aff:fe88:8087/64 scope link 2023-10-11T09:41:09.561 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:41:09.561 INFO:teuthology.orchestra.run.smithi133.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-11T09:41:09.561 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 02:42:dc:fa:bd:f4 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:41:09.561 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-11T09:41:09.562 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:41:09.562 INFO:teuthology.orchestra.run.smithi133.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-11T09:41:09.562 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 92:06:0a:27:61:ed brd ff:ff:ff:ff:ff:ff 2023-10-11T09:41:09.562 INFO:teuthology.orchestra.run.smithi133.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-11T09:41:09.562 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:41:09.563 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::d031:feff:fee9:57d5/64 scope link 2023-10-11T09:41:09.563 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:41:09.563 INFO:teuthology.orchestra.run.smithi133.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-11T09:41:09.563 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 92:06:0a:27:61:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-11T09:41:09.563 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::9006:aff:fe27:61ed/64 scope link 2023-10-11T09:41:09.564 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:41:09.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:09.565 DEBUG:teuthology.orchestra.run.smithi133:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-11T09:41:09.642 INFO:teuthology.orchestra.run.smithi133.stdout:1 2023-10-11T09:41:09.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:09.645 DEBUG:teuthology.orchestra.run.smithi133:> route 2023-10-11T09:41:09.708 INFO:teuthology.orchestra.run.smithi133.stdout:Kernel IP routing table 2023-10-11T09:41:09.709 INFO:teuthology.orchestra.run.smithi133.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-11T09:41:09.709 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-11T09:41:09.709 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-11T09:41:09.710 INFO:teuthology.orchestra.run.smithi133.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-11T09:41:09.710 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-10-11T09:41:09.710 INFO:teuthology.orchestra.run.smithi133.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-11T09:41:09.710 INFO:teuthology.orchestra.run.smithi133.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-11T09:41:09.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:09.711 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2023-10-11T09:41:09.711 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2023-10-11T09:41:09.712 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-11T09:41:09.712 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-11T09:41:09.712 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-11T09:41:09.712 DEBUG:teuthology.orchestra.run.smithi133:> ') 2023-10-11T09:41:09.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:09.844 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2023-10-11T09:41:09.860 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-11T09:41:09.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:09.861 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list-id 2023-10-11T09:41:09.919 INFO:teuthology.orchestra.run.smithi133.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-11T09:41:09.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:09.921 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2023-10-11T09:41:09.921 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2023-10-11T09:41:09.922 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-11T09:41:09.922 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-11T09:41:09.922 DEBUG:teuthology.orchestra.run.smithi133:> ') 2023-10-11T09:41:10.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:10.033 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-11T09:41:10.067 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-11T09:41:10.067 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-11T09:41:10.068 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2023-10-11T09:41:10.068 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:41:10.068 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2023-10-11T09:41:10.068 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:41:10.068 INFO:teuthology.orchestra.run.smithi133.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-11T09:41:10.069 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 62:74:42:bf:1b:80 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-11T09:41:10.069 INFO:teuthology.orchestra.run.smithi133.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-11T09:41:10.069 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:41:10.069 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::6074:42ff:febf:1b80/64 scope link 2023-10-11T09:41:10.069 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:41:10.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:10.071 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-11T09:41:10.151 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-11T09:41:10.152 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-11T09:41:10.152 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2023-10-11T09:41:10.152 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:41:10.152 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2023-10-11T09:41:10.152 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:41:10.153 INFO:teuthology.orchestra.run.smithi133.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-11T09:41:10.153 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 62:74:42:bf:1b:80 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-11T09:41:10.153 INFO:teuthology.orchestra.run.smithi133.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-11T09:41:10.153 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:41:10.153 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::6074:42ff:febf:1b80/64 scope link 2023-10-11T09:41:10.154 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:41:10.155 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-11T09:41:10.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:10.155 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2023-10-11T09:41:10.156 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2023-10-11T09:41:10.156 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-11T09:41:10.156 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-11T09:41:10.157 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-11T09:41:10.157 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-11T09:41:10.157 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-11T09:41:10.157 DEBUG:teuthology.orchestra.run.smithi133:> ') 2023-10-11T09:41:10.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:10.352 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2023-10-11T09:41:10.352 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2023-10-11T09:41:10.353 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set brx.1 up 2023-10-11T09:41:10.353 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set dev brx.1 master ceph-brx 2023-10-11T09:41:10.353 DEBUG:teuthology.orchestra.run.smithi133:> ') 2023-10-11T09:41:10.426 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-10-11T09:41:10.427 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T09:41:10.427 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-11T09:41:10.441 INFO:teuthology.orchestra.run.smithi133.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-11T09:41:10.441 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T09:41:10.442 DEBUG:teuthology.orchestra.run.smithi133:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-11T09:41:10.503 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2023-10-11T09:41:10.581 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T09:41:10.581 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T09:41:10.656 INFO:teuthology.orchestra.run.smithi133.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T09:41:10.659 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T09:41:10.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:10.660 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2023-10-11T09:41:10.721 INFO:teuthology.orchestra.run.smithi133.stdout:49 2023-10-11T09:41:10.722 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [49] 2023-10-11T09:41:10.722 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-10-11T09:41:10.774 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2023-10-11T09:41:10.815 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T09:41:10.815 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T09:41:10.888 INFO:teuthology.orchestra.run.smithi133.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T09:41:10.889 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T09:41:10.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:10.891 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2023-10-11T09:41:10.917 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi133.stderr:2023-10-11T09:41:10.915+0000 7fea89028640 -1 init, newargv = 0x55630410e7c0 newargc=15 2023-10-11T09:41:10.917 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi133.stderr:ceph-fuse[40967]: starting ceph client 2023-10-11T09:41:10.928 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi133.stderr:ceph-fuse[40967]: starting fuse 2023-10-11T09:41:10.933 INFO:teuthology.orchestra.run.smithi133.stdout:49 2023-10-11T09:41:10.933 INFO:teuthology.orchestra.run.smithi133.stdout:50 2023-10-11T09:41:10.933 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49, 50] 2023-10-11T09:41:10.934 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:41:10.934 DEBUG:teuthology.orchestra.run.smithi133:> import glob 2023-10-11T09:41:10.935 DEBUG:teuthology.orchestra.run.smithi133:> import re 2023-10-11T09:41:10.935 DEBUG:teuthology.orchestra.run.smithi133:> import os 2023-10-11T09:41:10.935 DEBUG:teuthology.orchestra.run.smithi133:> import subprocess 2023-10-11T09:41:10.935 DEBUG:teuthology.orchestra.run.smithi133:> 2023-10-11T09:41:10.936 DEBUG:teuthology.orchestra.run.smithi133:> def _find_admin_socket(client_name): 2023-10-11T09:41:10.936 DEBUG:teuthology.orchestra.run.smithi133:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-10-11T09:41:10.936 DEBUG:teuthology.orchestra.run.smithi133:> files = glob.glob(asok_path) 2023-10-11T09:41:10.937 DEBUG:teuthology.orchestra.run.smithi133:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-10-11T09:41:10.937 DEBUG:teuthology.orchestra.run.smithi133:> 2023-10-11T09:41:10.937 DEBUG:teuthology.orchestra.run.smithi133:> # Given a non-glob path, it better be there 2023-10-11T09:41:10.938 DEBUG:teuthology.orchestra.run.smithi133:> if "*" not in asok_path: 2023-10-11T09:41:10.938 DEBUG:teuthology.orchestra.run.smithi133:> assert(len(files) == 1) 2023-10-11T09:41:10.938 DEBUG:teuthology.orchestra.run.smithi133:> return files[0] 2023-10-11T09:41:10.938 DEBUG:teuthology.orchestra.run.smithi133:> 2023-10-11T09:41:10.939 DEBUG:teuthology.orchestra.run.smithi133:> for f in files: 2023-10-11T09:41:10.939 DEBUG:teuthology.orchestra.run.smithi133:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-11T09:41:10.939 DEBUG:teuthology.orchestra.run.smithi133:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-11T09:41:10.940 DEBUG:teuthology.orchestra.run.smithi133:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-11T09:41:10.940 DEBUG:teuthology.orchestra.run.smithi133:> contents = proc_f.read() 2023-10-11T09:41:10.940 DEBUG:teuthology.orchestra.run.smithi133:> if mountpoint in contents: 2023-10-11T09:41:10.941 DEBUG:teuthology.orchestra.run.smithi133:> return f 2023-10-11T09:41:10.941 DEBUG:teuthology.orchestra.run.smithi133:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-11T09:41:10.941 DEBUG:teuthology.orchestra.run.smithi133:> 2023-10-11T09:41:10.941 DEBUG:teuthology.orchestra.run.smithi133:> print(_find_admin_socket("client.3")) 2023-10-11T09:41:10.942 DEBUG:teuthology.orchestra.run.smithi133:> ' 2023-10-11T09:41:11.089 INFO:teuthology.orchestra.run.smithi133.stdout:/var/run/ceph/ceph-client.3.40967.asok 2023-10-11T09:41:11.275 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.40967.asok 2023-10-11T09:41:11.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:11.276 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.40967.asok status 2023-10-11T09:41:11.471 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-10-11T09:41:11.471 INFO:teuthology.orchestra.run.smithi133.stdout: "metadata": { 2023-10-11T09:41:11.471 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_sha1": "9a774796f0ecdd11a4b771de396e627aa1a187af", 2023-10-11T09:41:11.472 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_version": "ceph version 18.2.0-662-g9a774796 (9a774796f0ecdd11a4b771de396e627aa1a187af) reef (stable)", 2023-10-11T09:41:11.472 INFO:teuthology.orchestra.run.smithi133.stdout: "entity_id": "3", 2023-10-11T09:41:11.472 INFO:teuthology.orchestra.run.smithi133.stdout: "hostname": "smithi133", 2023-10-11T09:41:11.472 INFO:teuthology.orchestra.run.smithi133.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-10-11T09:41:11.472 INFO:teuthology.orchestra.run.smithi133.stdout: "pid": "40967", 2023-10-11T09:41:11.473 INFO:teuthology.orchestra.run.smithi133.stdout: "root": "/" 2023-10-11T09:41:11.473 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-10-11T09:41:11.473 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_count": 0, 2023-10-11T09:41:11.473 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_pinned_count": 0, 2023-10-11T09:41:11.473 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 4604, 2023-10-11T09:41:11.474 INFO:teuthology.orchestra.run.smithi133.stdout: "inst": { 2023-10-11T09:41:11.474 INFO:teuthology.orchestra.run.smithi133.stdout: "name": { 2023-10-11T09:41:11.474 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "client", 2023-10-11T09:41:11.474 INFO:teuthology.orchestra.run.smithi133.stdout: "num": 4604 2023-10-11T09:41:11.474 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-10-11T09:41:11.475 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2023-10-11T09:41:11.475 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2023-10-11T09:41:11.475 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.2:0", 2023-10-11T09:41:11.475 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 1135414738 2023-10-11T09:41:11.476 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-10-11T09:41:11.476 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-10-11T09:41:11.476 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2023-10-11T09:41:11.476 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2023-10-11T09:41:11.476 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.2:0", 2023-10-11T09:41:11.477 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 1135414738 2023-10-11T09:41:11.477 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-10-11T09:41:11.477 INFO:teuthology.orchestra.run.smithi133.stdout: "inst_str": "client.4604 192.168.0.2:0/1135414738", 2023-10-11T09:41:11.477 INFO:teuthology.orchestra.run.smithi133.stdout: "addr_str": "192.168.0.2:0/1135414738", 2023-10-11T09:41:11.477 INFO:teuthology.orchestra.run.smithi133.stdout: "inode_count": 1, 2023-10-11T09:41:11.477 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_epoch": 8, 2023-10-11T09:41:11.478 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch": 23, 2023-10-11T09:41:11.478 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch_barrier": 0, 2023-10-11T09:41:11.478 INFO:teuthology.orchestra.run.smithi133.stdout: "blocklisted": false, 2023-10-11T09:41:11.478 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs" 2023-10-11T09:41:11.478 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-10-11T09:41:11.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:11.486 DEBUG:teuthology.orchestra.run.smithi078:> stat --file-system '--printf=%T 2023-10-11T09:41:11.487 DEBUG:teuthology.orchestra.run.smithi078:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-11T09:41:11.506 INFO:teuthology.orchestra.run.smithi078.stdout:fuseblk 2023-10-11T09:41:11.506 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-11T09:41:11.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:11.507 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-11T09:41:11.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:11.586 DEBUG:teuthology.orchestra.run.smithi078:> stat --file-system '--printf=%T 2023-10-11T09:41:11.587 DEBUG:teuthology.orchestra.run.smithi078:> ' -- /home/ubuntu/cephtest/mnt.2 2023-10-11T09:41:11.653 INFO:teuthology.orchestra.run.smithi078.stdout:fuseblk 2023-10-11T09:41:11.653 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-10-11T09:41:11.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:11.654 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-11T09:41:11.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:11.740 DEBUG:teuthology.orchestra.run.smithi133:> stat --file-system '--printf=%T 2023-10-11T09:41:11.740 DEBUG:teuthology.orchestra.run.smithi133:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-11T09:41:11.757 INFO:teuthology.orchestra.run.smithi133.stdout:fuseblk 2023-10-11T09:41:11.758 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-11T09:41:11.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:11.758 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-11T09:41:11.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:11.837 DEBUG:teuthology.orchestra.run.smithi133:> stat --file-system '--printf=%T 2023-10-11T09:41:11.837 DEBUG:teuthology.orchestra.run.smithi133:> ' -- /home/ubuntu/cephtest/mnt.3 2023-10-11T09:41:11.899 INFO:teuthology.orchestra.run.smithi133.stdout:fuseblk 2023-10-11T09:41:11.899 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-10-11T09:41:11.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:11.900 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-11T09:41:11.977 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-11T09:41:12.068 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:41:12.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.246+0000 7ff4c0b46700 1 Processor -- start 2023-10-11T09:41:12.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.246+0000 7ff4c0b46700 1 -- start start 2023-10-11T09:41:12.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.246+0000 7ff4c0b46700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ff4bc06a770 0x7ff4bc1608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:12.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.246+0000 7ff4c0b46700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4bc0ceab0 con 0x7ff4bc0af3e0 2023-10-11T09:41:12.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.246+0000 7ff4c0b46700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4bc06d780 con 0x7ff4bc0af010 2023-10-11T09:41:12.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.246+0000 7ff4c0b46700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4bc0cfaf0 con 0x7ff4bc06a770 2023-10-11T09:41:12.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.246+0000 7ff4b9d9b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ff4bc06a770 0x7ff4bc1608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:12.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.246+0000 7ff4bad9d700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7ff4bc0af3e0 0x7ff4bc160df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:42072/0 (socket says 172.21.15.78:42072) 2023-10-11T09:41:12.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.246+0000 7ff4bad9d700 1 -- 172.21.15.78:0/1971919952 learned_addr learned my addr 172.21.15.78:0/1971919952 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:12.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.250+0000 7ff4b9d9b700 1 -- 172.21.15.78:0/1971919952 >> v1:172.21.15.133:6789/0 conn(0x7ff4bc0af010 legacy=0x7ff4bc06a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:12.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.250+0000 7ff4b9d9b700 1 -- 172.21.15.78:0/1971919952 >> v1:172.21.15.78:6789/0 conn(0x7ff4bc0af3e0 legacy=0x7ff4bc160df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:12.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.250+0000 7ff4b959a700 1 -- 172.21.15.78:0/1971919952 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 393416885 0 0) 0x7ff4bc06d780 con 0x7ff4bc0af010 2023-10-11T09:41:12.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.250+0000 7ff4b9d9b700 1 -- 172.21.15.78:0/1971919952 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4bc0ced30 con 0x7ff4bc06a770 2023-10-11T09:41:12.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.250+0000 7ff4b9d9b700 1 --2- 172.21.15.78:0/1971919952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4bc06a770 0x7ff4bc1608b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0009850 tx=0x7ff4b0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56c69058b14734e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:12.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.250+0000 7ff4b959a700 1 -- 172.21.15.78:0/1971919952 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b0011830 con 0x7ff4bc06a770 2023-10-11T09:41:12.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.250+0000 7ff4b959a700 1 -- 172.21.15.78:0/1971919952 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b0011990 con 0x7ff4bc06a770 2023-10-11T09:41:12.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.250+0000 7ff4b959a700 1 -- 172.21.15.78:0/1971919952 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b0011d20 con 0x7ff4bc06a770 2023-10-11T09:41:12.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.250+0000 7ff4c0b46700 1 -- 172.21.15.78:0/1971919952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4bc06a770 msgr2=0x7ff4bc1608b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:12.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.250+0000 7ff4c0b46700 1 --2- 172.21.15.78:0/1971919952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4bc06a770 0x7ff4bc1608b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0009850 tx=0x7ff4b0009b60 comp rx=0 tx=0).stop 2023-10-11T09:41:12.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.250+0000 7ff4c0b46700 1 -- 172.21.15.78:0/1971919952 shutdown_connections 2023-10-11T09:41:12.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.250+0000 7ff4c0b46700 1 --2- 172.21.15.78:0/1971919952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4bc06a770 0x7ff4bc1608b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:12.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.250+0000 7ff4c0b46700 1 -- 172.21.15.78:0/1971919952 >> 172.21.15.78:0/1971919952 conn(0x7ff4bc0cc820 msgr2=0x7ff4bc0ccc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:12.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.250+0000 7ff4c0b46700 1 -- 172.21.15.78:0/1971919952 shutdown_connections 2023-10-11T09:41:12.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.250+0000 7ff4c0b46700 1 -- 172.21.15.78:0/1971919952 wait complete. 2023-10-11T09:41:12.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.254+0000 7ff4c0b46700 1 Processor -- start 2023-10-11T09:41:12.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.254+0000 7ff4c0b46700 1 -- start start 2023-10-11T09:41:12.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.258+0000 7ff4c0b46700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4bc06a770 0x7ff4bc0c6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:12.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.258+0000 7ff4c0b46700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4bc0af010 0x7ff4bc150e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:12.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.258+0000 7ff4c0b46700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4bc0af3e0 0x7ff4bc0c7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:12.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.258+0000 7ff4c0b46700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4bc0c8c90 con 0x7ff4bc0af3e0 2023-10-11T09:41:12.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.258+0000 7ff4c0b46700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4bc1513b0 con 0x7ff4bc06a770 2023-10-11T09:41:12.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.258+0000 7ff4c0b46700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4bc0c8f00 con 0x7ff4bc0af010 2023-10-11T09:41:12.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.258+0000 7ff4bad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4bc0af3e0 0x7ff4bc0c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:12.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.258+0000 7ff4bad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4bc0af3e0 0x7ff4bc0c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53136/0 (socket says 172.21.15.78:53136) 2023-10-11T09:41:12.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.258+0000 7ff4bad9d700 1 -- 172.21.15.78:0/2973271007 learned_addr learned my addr 172.21.15.78:0/2973271007 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:12.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.258+0000 7ff4b9d9b700 1 --2- 172.21.15.78:0/2973271007 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4bc0af010 0x7ff4bc150e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:12.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.258+0000 7ff4ba59c700 1 --2- 172.21.15.78:0/2973271007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4bc06a770 0x7ff4bc0c6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:12.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.258+0000 7ff4bad9d700 1 -- 172.21.15.78:0/2973271007 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4bc0af010 msgr2=0x7ff4bc150e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:12.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.258+0000 7ff4bad9d700 1 --2- 172.21.15.78:0/2973271007 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4bc0af010 0x7ff4bc150e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:12.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.258+0000 7ff4bad9d700 1 -- 172.21.15.78:0/2973271007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4bc06a770 msgr2=0x7ff4bc0c6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:12.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.258+0000 7ff4bad9d700 1 --2- 172.21.15.78:0/2973271007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4bc06a770 0x7ff4bc0c6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:12.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.258+0000 7ff4bad9d700 1 -- 172.21.15.78:0/2973271007 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b00094b0 con 0x7ff4bc0af3e0 2023-10-11T09:41:12.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.258+0000 7ff4ba59c700 1 --2- 172.21.15.78:0/2973271007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4bc06a770 0x7ff4bc0c6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:12.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.258+0000 7ff4bad9d700 1 --2- 172.21.15.78:0/2973271007 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4bc0af3e0 0x7ff4bc0c7650 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff4b400cbc0 tx=0x7ff4b400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:12.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.258+0000 7ff4aaffd700 1 -- 172.21.15.78:0/2973271007 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b400a7b0 con 0x7ff4bc0af3e0 2023-10-11T09:41:12.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.258+0000 7ff4aaffd700 1 -- 172.21.15.78:0/2973271007 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b400a910 con 0x7ff4bc0af3e0 2023-10-11T09:41:12.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.258+0000 7ff4aaffd700 1 -- 172.21.15.78:0/2973271007 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b400ac80 con 0x7ff4bc0af3e0 2023-10-11T09:41:12.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.258+0000 7ff4c0b46700 1 -- 172.21.15.78:0/2973271007 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4bc0c91e0 con 0x7ff4bc0af3e0 2023-10-11T09:41:12.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.258+0000 7ff4c0b46700 1 -- 172.21.15.78:0/2973271007 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4bc1f2d00 con 0x7ff4bc0af3e0 2023-10-11T09:41:12.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.262+0000 7ff4c0b46700 1 -- 172.21.15.78:0/2973271007 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4bc0726b0 con 0x7ff4bc0af3e0 2023-10-11T09:41:12.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.262+0000 7ff4aaffd700 1 -- 172.21.15.78:0/2973271007 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7ff4b401b3b0 con 0x7ff4bc0af3e0 2023-10-11T09:41:12.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.266+0000 7ff4aaffd700 1 --2- 172.21.15.78:0/2973271007 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff49c065f00 0x7ff49c0683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:12.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.266+0000 7ff4aaffd700 1 -- 172.21.15.78:0/2973271007 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff4b4082ff0 con 0x7ff4bc0af3e0 2023-10-11T09:41:12.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.266+0000 7ff4ba59c700 1 --2- 172.21.15.78:0/2973271007 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff49c065f00 0x7ff49c0683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:12.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.266+0000 7ff4ba59c700 1 --2- 172.21.15.78:0/2973271007 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff49c065f00 0x7ff49c0683b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac007bd0 tx=0x7ff4ac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:12.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.270+0000 7ff4aaffd700 1 -- 172.21.15.78:0/2973271007 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7ff4b4050740 con 0x7ff4bc0af3e0 2023-10-11T09:41:12.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.450+0000 7ff4c0b46700 1 -- 172.21.15.78:0/2973271007 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4bc04e530 con 0x7ff4bc0af3e0 2023-10-11T09:41:12.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.454+0000 7ff4aaffd700 1 -- 172.21.15.78:0/2973271007 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17811 (secure 0 0 0) 0x7ff4b4053d60 con 0x7ff4bc0af3e0 2023-10-11T09:41:12.459 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:41:12.459 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":23,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:40:42.678477+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:40:39.862581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T09:40:40.663641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:41:12.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.458+0000 7ff4c0b46700 1 -- 172.21.15.78:0/2973271007 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff49c065f00 msgr2=0x7ff49c0683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:12.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.458+0000 7ff4c0b46700 1 --2- 172.21.15.78:0/2973271007 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff49c065f00 0x7ff49c0683b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac007bd0 tx=0x7ff4ac009040 comp rx=0 tx=0).stop 2023-10-11T09:41:12.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.458+0000 7ff4c0b46700 1 -- 172.21.15.78:0/2973271007 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4bc0af3e0 msgr2=0x7ff4bc0c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:12.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.458+0000 7ff4c0b46700 1 --2- 172.21.15.78:0/2973271007 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4bc0af3e0 0x7ff4bc0c7650 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff4b400cbc0 tx=0x7ff4b400cbf0 comp rx=0 tx=0).stop 2023-10-11T09:41:12.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.458+0000 7ff4c0b46700 1 -- 172.21.15.78:0/2973271007 shutdown_connections 2023-10-11T09:41:12.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.458+0000 7ff4c0b46700 1 --2- 172.21.15.78:0/2973271007 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff49c065f00 0x7ff49c0683b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:12.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.458+0000 7ff4c0b46700 1 --2- 172.21.15.78:0/2973271007 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4bc0af3e0 0x7ff4bc0c7650 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:12.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.458+0000 7ff4c0b46700 1 --2- 172.21.15.78:0/2973271007 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4bc0af010 0x7ff4bc150e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:12.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.458+0000 7ff4c0b46700 1 --2- 172.21.15.78:0/2973271007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4bc06a770 0x7ff4bc0c6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:12.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.458+0000 7ff4c0b46700 1 -- 172.21.15.78:0/2973271007 >> 172.21.15.78:0/2973271007 conn(0x7ff4bc0cc820 msgr2=0x7ff4bc0cbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:12.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.458+0000 7ff4c0b46700 1 -- 172.21.15.78:0/2973271007 shutdown_connections 2023-10-11T09:41:12.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.458+0000 7ff4c0b46700 1 -- 172.21.15.78:0/2973271007 wait complete. 2023-10-11T09:41:12.486 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T09:40:39.862581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T09:40:40.663641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:41:12.486 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:41:12.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.666+0000 7f3fb16ae700 1 Processor -- start 2023-10-11T09:41:12.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.666+0000 7f3fb16ae700 1 -- start start 2023-10-11T09:41:12.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.670+0000 7f3fb16ae700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3fac0b84c0 0x7f3fac0cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:12.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.670+0000 7f3fb16ae700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f3fac160a80 0x7f3fac0d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:12.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.670+0000 7f3fb16ae700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fac0cd690 con 0x7f3fac0b8890 2023-10-11T09:41:12.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.670+0000 7f3fb16ae700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fac0cdea0 con 0x7f3fac160a80 2023-10-11T09:41:12.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.670+0000 7f3fb16ae700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fac0d0c10 con 0x7f3fac0b84c0 2023-10-11T09:41:12.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.670+0000 7f3fab7fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f3fac160a80 0x7f3fac0d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:12.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.670+0000 7f3faaffd700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3fac0b84c0 0x7f3fac0cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:12.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.670+0000 7f3fab7fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f3fac160a80 0x7f3fac0d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53146/0 (socket says 172.21.15.78:53146) 2023-10-11T09:41:12.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.670+0000 7f3fab7fe700 1 -- 172.21.15.78:0/2646268090 learned_addr learned my addr 172.21.15.78:0/2646268090 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:12.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.670+0000 7f3faaffd700 1 -- 172.21.15.78:0/2646268090 >> v2:172.21.15.78:3300/0 conn(0x7f3fac160a80 msgr2=0x7f3fac0d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:12.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.670+0000 7f3faaffd700 1 --2- 172.21.15.78:0/2646268090 >> v2:172.21.15.78:3300/0 conn(0x7f3fac160a80 0x7f3fac0d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:12.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.670+0000 7f3faaffd700 1 -- 172.21.15.78:0/2646268090 >> v1:172.21.15.133:6789/0 conn(0x7f3fac0b8890 legacy=0x7f3fac160370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:12.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.670+0000 7f3faaffd700 1 -- 172.21.15.78:0/2646268090 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fac0d02e0 con 0x7f3fac0b84c0 2023-10-11T09:41:12.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.670+0000 7f3fa9ffb700 1 -- 172.21.15.78:0/2646268090 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4218724675 0 0) 0x7f3fac0cd690 con 0x7f3fac0b8890 2023-10-11T09:41:12.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.670+0000 7f3faaffd700 1 --2- 172.21.15.78:0/2646268090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fac0b84c0 0x7f3fac0cfda0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3fac0675c0 tx=0x7f3f9c009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48254ae07019d6ff server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:12.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.670+0000 7f3fa9ffb700 1 -- 172.21.15.78:0/2646268090 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c011830 con 0x7f3fac0b84c0 2023-10-11T09:41:12.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.670+0000 7f3fa9ffb700 1 -- 172.21.15.78:0/2646268090 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f9c011990 con 0x7f3fac0b84c0 2023-10-11T09:41:12.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.670+0000 7f3fa9ffb700 1 -- 172.21.15.78:0/2646268090 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c011d20 con 0x7f3fac0b84c0 2023-10-11T09:41:12.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.674+0000 7f3fb16ae700 1 -- 172.21.15.78:0/2646268090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fac0b84c0 msgr2=0x7f3fac0cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:12.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.674+0000 7f3fb16ae700 1 --2- 172.21.15.78:0/2646268090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fac0b84c0 0x7f3fac0cfda0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3fac0675c0 tx=0x7f3f9c009b30 comp rx=0 tx=0).stop 2023-10-11T09:41:12.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.674+0000 7f3fb16ae700 1 -- 172.21.15.78:0/2646268090 shutdown_connections 2023-10-11T09:41:12.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.674+0000 7f3fb16ae700 1 --2- 172.21.15.78:0/2646268090 >> v2:172.21.15.78:3300/0 conn(0x7f3fac160a80 0x7f3fac0d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:12.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.674+0000 7f3fb16ae700 1 --2- 172.21.15.78:0/2646268090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fac0b84c0 0x7f3fac0cfda0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:12.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.674+0000 7f3fb16ae700 1 -- 172.21.15.78:0/2646268090 >> 172.21.15.78:0/2646268090 conn(0x7f3fac06cb50 msgr2=0x7f3fac06cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:12.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.674+0000 7f3fb16ae700 1 -- 172.21.15.78:0/2646268090 shutdown_connections 2023-10-11T09:41:12.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.674+0000 7f3fb16ae700 1 -- 172.21.15.78:0/2646268090 wait complete. 2023-10-11T09:41:12.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.674+0000 7f3fb16ae700 1 Processor -- start 2023-10-11T09:41:12.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.674+0000 7f3fb16ae700 1 -- start start 2023-10-11T09:41:12.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.678+0000 7f3fb16ae700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3fac0b84c0 0x7f3fac0c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:12.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.678+0000 7f3fb16ae700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fac0b8890 0x7f3fac151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:12.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.678+0000 7f3fb16ae700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fac160a80 0x7f3fac159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:12.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.678+0000 7f3fb16ae700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac15d220 con 0x7f3fac160a80 2023-10-11T09:41:12.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.678+0000 7f3fb16ae700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac152350 con 0x7f3fac0b8890 2023-10-11T09:41:12.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.678+0000 7f3fb16ae700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac0c7e10 con 0x7f3fac0b84c0 2023-10-11T09:41:12.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.678+0000 7f3fab7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fac160a80 0x7f3fac159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:12.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.678+0000 7f3fab7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fac160a80 0x7f3fac159d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53150/0 (socket says 172.21.15.78:53150) 2023-10-11T09:41:12.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.678+0000 7f3fab7fe700 1 -- 172.21.15.78:0/2937909539 learned_addr learned my addr 172.21.15.78:0/2937909539 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:12.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.678+0000 7f3faa7fc700 1 --2- 172.21.15.78:0/2937909539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fac0b8890 0x7f3fac151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:12.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.678+0000 7f3fab7fe700 1 -- 172.21.15.78:0/2937909539 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3fac0b84c0 msgr2=0x7f3fac0c8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:12.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.678+0000 7f3fab7fe700 1 --2- 172.21.15.78:0/2937909539 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3fac0b84c0 0x7f3fac0c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:12.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.678+0000 7f3fab7fe700 1 -- 172.21.15.78:0/2937909539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fac0b8890 msgr2=0x7f3fac151e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:12.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.678+0000 7f3fab7fe700 1 --2- 172.21.15.78:0/2937909539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fac0b8890 0x7f3fac151e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:12.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.678+0000 7f3fab7fe700 1 -- 172.21.15.78:0/2937909539 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f9c0094b0 con 0x7f3fac160a80 2023-10-11T09:41:12.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.678+0000 7f3faa7fc700 1 --2- 172.21.15.78:0/2937909539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fac0b8890 0x7f3fac151e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:12.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.678+0000 7f3fab7fe700 1 --2- 172.21.15.78:0/2937909539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fac160a80 0x7f3fac159d30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3fa400efc0 tx=0x7f3fa400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:12.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.678+0000 7f3f937fe700 1 -- 172.21.15.78:0/2937909539 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4003c40 con 0x7f3fac160a80 2023-10-11T09:41:12.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.678+0000 7f3f937fe700 1 -- 172.21.15.78:0/2937909539 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa400f040 con 0x7f3fac160a80 2023-10-11T09:41:12.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.678+0000 7f3fb16ae700 1 -- 172.21.15.78:0/2937909539 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fac0c8150 con 0x7f3fac160a80 2023-10-11T09:41:12.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.678+0000 7f3f937fe700 1 -- 172.21.15.78:0/2937909539 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa400c930 con 0x7f3fac160a80 2023-10-11T09:41:12.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.678+0000 7f3fb16ae700 1 -- 172.21.15.78:0/2937909539 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fac1571a0 con 0x7f3fac160a80 2023-10-11T09:41:12.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.682+0000 7f3fb16ae700 1 -- 172.21.15.78:0/2937909539 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fac0c88f0 con 0x7f3fac160a80 2023-10-11T09:41:12.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.682+0000 7f3f937fe700 1 -- 172.21.15.78:0/2937909539 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f3fa400ca90 con 0x7f3fac160a80 2023-10-11T09:41:12.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.682+0000 7f3f937fe700 1 --2- 172.21.15.78:0/2937909539 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3f98065f00 0x7f3f980683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:12.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.686+0000 7f3f937fe700 1 -- 172.21.15.78:0/2937909539 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3fa4083ca0 con 0x7f3fac160a80 2023-10-11T09:41:12.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.686+0000 7f3faaffd700 1 --2- 172.21.15.78:0/2937909539 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3f98065f00 0x7f3f980683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:12.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.686+0000 7f3faaffd700 1 --2- 172.21.15.78:0/2937909539 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3f98065f00 0x7f3f980683b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c014040 tx=0x7f3f9c010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:12.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.686+0000 7f3f937fe700 1 -- 172.21.15.78:0/2937909539 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f3fa40512e0 con 0x7f3fac160a80 2023-10-11T09:41:12.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.858+0000 7f3fb16ae700 1 -- 172.21.15.78:0/2937909539 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3fac0c7f80 con 0x7f3fac160a80 2023-10-11T09:41:12.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.862+0000 7f3f937fe700 1 -- 172.21.15.78:0/2937909539 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3fa4054900 con 0x7f3fac160a80 2023-10-11T09:41:12.865 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:41:12.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.866+0000 7f3fb16ae700 1 -- 172.21.15.78:0/2937909539 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3f98065f00 msgr2=0x7f3f980683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:12.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.866+0000 7f3fb16ae700 1 --2- 172.21.15.78:0/2937909539 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3f98065f00 0x7f3f980683b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c014040 tx=0x7f3f9c010000 comp rx=0 tx=0).stop 2023-10-11T09:41:12.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.866+0000 7f3fb16ae700 1 -- 172.21.15.78:0/2937909539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fac160a80 msgr2=0x7f3fac159d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:12.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.866+0000 7f3fb16ae700 1 --2- 172.21.15.78:0/2937909539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fac160a80 0x7f3fac159d30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3fa400efc0 tx=0x7f3fa400c3a0 comp rx=0 tx=0).stop 2023-10-11T09:41:12.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.866+0000 7f3fb16ae700 1 -- 172.21.15.78:0/2937909539 shutdown_connections 2023-10-11T09:41:12.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.866+0000 7f3fb16ae700 1 --2- 172.21.15.78:0/2937909539 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3f98065f00 0x7f3f980683b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:12.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.866+0000 7f3fb16ae700 1 --2- 172.21.15.78:0/2937909539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fac160a80 0x7f3fac159d30 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:12.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.866+0000 7f3fb16ae700 1 --2- 172.21.15.78:0/2937909539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fac0b8890 0x7f3fac151e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:12.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.866+0000 7f3fb16ae700 1 --2- 172.21.15.78:0/2937909539 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3fac0b84c0 0x7f3fac0c8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:12.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.866+0000 7f3fb16ae700 1 -- 172.21.15.78:0/2937909539 >> 172.21.15.78:0/2937909539 conn(0x7f3fac06cb50 msgr2=0x7f3fac066010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:12.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.866+0000 7f3fb16ae700 1 -- 172.21.15.78:0/2937909539 shutdown_connections 2023-10-11T09:41:12.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:12.866+0000 7f3fb16ae700 1 -- 172.21.15.78:0/2937909539 wait complete. 2023-10-11T09:41:12.892 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:41:13.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.070+0000 7faefed29700 1 Processor -- start 2023-10-11T09:41:13.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.070+0000 7faefed29700 1 -- start start 2023-10-11T09:41:13.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.070+0000 7faefed29700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7faef8146fe0 0x7faef806bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:13.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.070+0000 7faefed29700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faef80b36f0 0x7faef81629e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:13.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.070+0000 7faefed29700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faef806ccb0 con 0x7faef8161d90 2023-10-11T09:41:13.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.070+0000 7faefed29700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faef806c410 con 0x7faef8146fe0 2023-10-11T09:41:13.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.070+0000 7faefed29700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faef806c6f0 con 0x7faef80b36f0 2023-10-11T09:41:13.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.070+0000 7faefdd27700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7faef8146fe0 0x7faef806bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:13.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.070+0000 7faefdd27700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7faef8146fe0 0x7faef806bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53164/0 (socket says 172.21.15.78:53164) 2023-10-11T09:41:13.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.070+0000 7faefdd27700 1 -- 172.21.15.78:0/2805453511 learned_addr learned my addr 172.21.15.78:0/2805453511 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:13.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.070+0000 7faefe528700 1 --2- 172.21.15.78:0/2805453511 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faef80b36f0 0x7faef81629e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:13.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.070+0000 7faefdd27700 1 -- 172.21.15.78:0/2805453511 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faef80b36f0 msgr2=0x7faef81629e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:13.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.070+0000 7faefdd27700 1 --2- 172.21.15.78:0/2805453511 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faef80b36f0 0x7faef81629e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:13.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.070+0000 7faefdd27700 1 -- 172.21.15.78:0/2805453511 >> v1:172.21.15.133:6789/0 conn(0x7faef8161d90 legacy=0x7faef80b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:13.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.070+0000 7faefdd27700 1 -- 172.21.15.78:0/2805453511 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef806c9d0 con 0x7faef8146fe0 2023-10-11T09:41:13.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.070+0000 7faefe528700 1 --2- 172.21.15.78:0/2805453511 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faef80b36f0 0x7faef81629e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:13.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.074+0000 7faefdd27700 1 --2- 172.21.15.78:0/2805453511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faef8146fe0 0x7faef806bbe0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7faef4009910 tx=0x7faef4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bde5d5b380cdb21 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:13.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.074+0000 7faefcd25700 1 -- 172.21.15.78:0/2805453511 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef400ba60 con 0x7faef8146fe0 2023-10-11T09:41:13.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.074+0000 7faefcd25700 1 -- 172.21.15.78:0/2805453511 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef400bbc0 con 0x7faef8146fe0 2023-10-11T09:41:13.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.074+0000 7faefcd25700 1 -- 172.21.15.78:0/2805453511 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef4009e60 con 0x7faef8146fe0 2023-10-11T09:41:13.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.074+0000 7faefed29700 1 -- 172.21.15.78:0/2805453511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faef8146fe0 msgr2=0x7faef806bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:13.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.074+0000 7faefed29700 1 --2- 172.21.15.78:0/2805453511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faef8146fe0 0x7faef806bbe0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7faef4009910 tx=0x7faef4009c20 comp rx=0 tx=0).stop 2023-10-11T09:41:13.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.074+0000 7faefed29700 1 -- 172.21.15.78:0/2805453511 shutdown_connections 2023-10-11T09:41:13.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.074+0000 7faefed29700 1 --2- 172.21.15.78:0/2805453511 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faef80b36f0 0x7faef81629e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:13.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.074+0000 7faefed29700 1 --2- 172.21.15.78:0/2805453511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faef8146fe0 0x7faef806bbe0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:13.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.074+0000 7faefed29700 1 -- 172.21.15.78:0/2805453511 >> 172.21.15.78:0/2805453511 conn(0x7faef806a060 msgr2=0x7faef806a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:13.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.074+0000 7faefed29700 1 -- 172.21.15.78:0/2805453511 shutdown_connections 2023-10-11T09:41:13.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.074+0000 7faefed29700 1 -- 172.21.15.78:0/2805453511 wait complete. 2023-10-11T09:41:13.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.078+0000 7faefed29700 1 Processor -- start 2023-10-11T09:41:13.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.078+0000 7faefed29700 1 -- start start 2023-10-11T09:41:13.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.078+0000 7faefed29700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faef80b36f0 0x7faef80ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:13.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.078+0000 7faefed29700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faef8146fe0 0x7faef81f9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:13.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.078+0000 7faefed29700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faef8161d90 0x7faef81f9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:13.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.078+0000 7faefed29700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef81fa850 con 0x7faef80b36f0 2023-10-11T09:41:13.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.078+0000 7faefed29700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef81fa300 con 0x7faef8161d90 2023-10-11T09:41:13.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.078+0000 7faefed29700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faef81fa570 con 0x7faef8146fe0 2023-10-11T09:41:13.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.078+0000 7faefdd27700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faef80b36f0 0x7faef80ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:13.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.078+0000 7faefdd27700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faef80b36f0 0x7faef80ce4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53176/0 (socket says 172.21.15.78:53176) 2023-10-11T09:41:13.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.078+0000 7faefdd27700 1 -- 172.21.15.78:0/1655030544 learned_addr learned my addr 172.21.15.78:0/1655030544 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:13.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.082+0000 7faefd526700 1 --2- 172.21.15.78:0/1655030544 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faef8146fe0 0x7faef81f9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:13.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.082+0000 7faefe528700 1 --2- 172.21.15.78:0/1655030544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faef8161d90 0x7faef81f9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:13.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.082+0000 7faefdd27700 1 -- 172.21.15.78:0/1655030544 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faef8146fe0 msgr2=0x7faef81f9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:13.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.082+0000 7faefdd27700 1 --2- 172.21.15.78:0/1655030544 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faef8146fe0 0x7faef81f9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:13.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.082+0000 7faefdd27700 1 -- 172.21.15.78:0/1655030544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faef8161d90 msgr2=0x7faef81f9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:13.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.082+0000 7faefdd27700 1 --2- 172.21.15.78:0/1655030544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faef8161d90 0x7faef81f9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:13.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.082+0000 7faefdd27700 1 -- 172.21.15.78:0/1655030544 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef4009570 con 0x7faef80b36f0 2023-10-11T09:41:13.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.082+0000 7faefdd27700 1 --2- 172.21.15.78:0/1655030544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faef80b36f0 0x7faef80ce4b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faef4000c00 tx=0x7faef4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:13.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.082+0000 7faeee7fc700 1 -- 172.21.15.78:0/1655030544 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef4004610 con 0x7faef80b36f0 2023-10-11T09:41:13.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.082+0000 7faeee7fc700 1 -- 172.21.15.78:0/1655030544 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef401e070 con 0x7faef80b36f0 2023-10-11T09:41:13.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.082+0000 7faeee7fc700 1 -- 172.21.15.78:0/1655030544 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef4003920 con 0x7faef80b36f0 2023-10-11T09:41:13.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.082+0000 7faefed29700 1 -- 172.21.15.78:0/1655030544 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faef80a8640 con 0x7faef80b36f0 2023-10-11T09:41:13.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.082+0000 7faefed29700 1 -- 172.21.15.78:0/1655030544 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faef80a8c00 con 0x7faef80b36f0 2023-10-11T09:41:13.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.082+0000 7faefed29700 1 -- 172.21.15.78:0/1655030544 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faef8065d40 con 0x7faef80b36f0 2023-10-11T09:41:13.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.086+0000 7faeee7fc700 1 -- 172.21.15.78:0/1655030544 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7faef4003a80 con 0x7faef80b36f0 2023-10-11T09:41:13.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.086+0000 7faeee7fc700 1 --2- 172.21.15.78:0/1655030544 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7faedc065ed0 0x7faedc068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:13.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.086+0000 7faeee7fc700 1 -- 172.21.15.78:0/1655030544 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faef4020070 con 0x7faef80b36f0 2023-10-11T09:41:13.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.086+0000 7faefd526700 1 --2- 172.21.15.78:0/1655030544 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7faedc065ed0 0x7faedc068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:13.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.086+0000 7faefd526700 1 --2- 172.21.15.78:0/1655030544 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7faedc065ed0 0x7faedc068380 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7faef81561a0 tx=0x7faee8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:13.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.090+0000 7faeee7fc700 1 -- 172.21.15.78:0/1655030544 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7faef4052380 con 0x7faef80b36f0 2023-10-11T09:41:13.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.266+0000 7faefed29700 1 -- 172.21.15.78:0/1655030544 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7faef806ce50 con 0x7faef80b36f0 2023-10-11T09:41:13.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.266+0000 7faeee7fc700 1 -- 172.21.15.78:0/1655030544 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7faef40150a0 con 0x7faef80b36f0 2023-10-11T09:41:13.270 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:41:13.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.270+0000 7faefed29700 1 -- 172.21.15.78:0/1655030544 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7faedc065ed0 msgr2=0x7faedc068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:13.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.270+0000 7faefed29700 1 --2- 172.21.15.78:0/1655030544 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7faedc065ed0 0x7faedc068380 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7faef81561a0 tx=0x7faee8009040 comp rx=0 tx=0).stop 2023-10-11T09:41:13.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.270+0000 7faefed29700 1 -- 172.21.15.78:0/1655030544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faef80b36f0 msgr2=0x7faef80ce4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:13.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.270+0000 7faefed29700 1 --2- 172.21.15.78:0/1655030544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faef80b36f0 0x7faef80ce4b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faef4000c00 tx=0x7faef4004770 comp rx=0 tx=0).stop 2023-10-11T09:41:13.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.270+0000 7faefed29700 1 -- 172.21.15.78:0/1655030544 shutdown_connections 2023-10-11T09:41:13.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.270+0000 7faefed29700 1 --2- 172.21.15.78:0/1655030544 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7faedc065ed0 0x7faedc068380 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:13.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.270+0000 7faefed29700 1 --2- 172.21.15.78:0/1655030544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faef8161d90 0x7faef81f9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:13.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.270+0000 7faefed29700 1 --2- 172.21.15.78:0/1655030544 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faef8146fe0 0x7faef81f9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:13.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.270+0000 7faefed29700 1 --2- 172.21.15.78:0/1655030544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faef80b36f0 0x7faef80ce4b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:13.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.270+0000 7faefed29700 1 -- 172.21.15.78:0/1655030544 >> 172.21.15.78:0/1655030544 conn(0x7faef806a060 msgr2=0x7faef806fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:13.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.270+0000 7faefed29700 1 -- 172.21.15.78:0/1655030544 shutdown_connections 2023-10-11T09:41:13.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.270+0000 7faefed29700 1 -- 172.21.15.78:0/1655030544 wait complete. 2023-10-11T09:41:13.296 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T09:41:13.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.466+0000 7f6346d46700 1 Processor -- start 2023-10-11T09:41:13.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.466+0000 7f6346d46700 1 -- start start 2023-10-11T09:41:13.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.470+0000 7f6346d46700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63400b84c0 0x7f63400cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:13.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.470+0000 7f6346d46700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f63400b8890 0x7f6340160370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:13.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.470+0000 7f6346d46700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63400cd690 con 0x7f63401608b0 2023-10-11T09:41:13.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.470+0000 7f6346d46700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63400cdea0 con 0x7f63400b8890 2023-10-11T09:41:13.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.470+0000 7f6346d46700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63400d0c10 con 0x7f63400b84c0 2023-10-11T09:41:13.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.470+0000 7f63452e3700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f63401608b0 0x7f63400d2fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:42084/0 (socket says 172.21.15.78:42084) 2023-10-11T09:41:13.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.470+0000 7f63452e3700 1 -- 172.21.15.78:0/2942006829 learned_addr learned my addr 172.21.15.78:0/2942006829 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:13.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.470+0000 7f633ffff700 1 --2- 172.21.15.78:0/2942006829 >> v2:172.21.15.133:3300/0 conn(0x7f63400b8890 0x7f6340160370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:13.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.470+0000 7f6344ae2700 1 --2- 172.21.15.78:0/2942006829 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63400b84c0 0x7f63400cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:13.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.470+0000 7f633f7fe700 1 -- 172.21.15.78:0/2942006829 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1062694719 0 0) 0x7f63400cd690 con 0x7f63401608b0 2023-10-11T09:41:13.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.470+0000 7f633f7fe700 1 -- 172.21.15.78:0/2942006829 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63300035d0 con 0x7f63401608b0 2023-10-11T09:41:13.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.470+0000 7f633ffff700 1 -- 172.21.15.78:0/2942006829 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63400b84c0 msgr2=0x7f63400cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:13.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.470+0000 7f633ffff700 1 --2- 172.21.15.78:0/2942006829 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63400b84c0 0x7f63400cfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:13.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.470+0000 7f633ffff700 1 -- 172.21.15.78:0/2942006829 >> v1:172.21.15.78:6789/0 conn(0x7f63401608b0 legacy=0x7f63400d2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:13.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.470+0000 7f633ffff700 1 -- 172.21.15.78:0/2942006829 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63400d02e0 con 0x7f63400b8890 2023-10-11T09:41:13.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.470+0000 7f6344ae2700 1 --2- 172.21.15.78:0/2942006829 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63400b84c0 0x7f63400cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:41:13.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.470+0000 7f633ffff700 1 --2- 172.21.15.78:0/2942006829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63400b8890 0x7f6340160370 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f632c009850 tx=0x7f632c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8972eff48c05d2b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:13.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.470+0000 7f633f7fe700 1 -- 172.21.15.78:0/2942006829 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c011830 con 0x7f63400b8890 2023-10-11T09:41:13.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.470+0000 7f633f7fe700 1 -- 172.21.15.78:0/2942006829 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f632c011990 con 0x7f63400b8890 2023-10-11T09:41:13.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.470+0000 7f633f7fe700 1 -- 172.21.15.78:0/2942006829 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c011d40 con 0x7f63400b8890 2023-10-11T09:41:13.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.474+0000 7f6346d46700 1 -- 172.21.15.78:0/2942006829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63400b8890 msgr2=0x7f6340160370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:13.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.474+0000 7f6346d46700 1 --2- 172.21.15.78:0/2942006829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63400b8890 0x7f6340160370 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f632c009850 tx=0x7f632c009b60 comp rx=0 tx=0).stop 2023-10-11T09:41:13.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.474+0000 7f6346d46700 1 -- 172.21.15.78:0/2942006829 shutdown_connections 2023-10-11T09:41:13.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.474+0000 7f6346d46700 1 --2- 172.21.15.78:0/2942006829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63400b8890 0x7f6340160370 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:13.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.474+0000 7f6346d46700 1 --2- 172.21.15.78:0/2942006829 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63400b84c0 0x7f63400cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:13.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.474+0000 7f6346d46700 1 -- 172.21.15.78:0/2942006829 >> 172.21.15.78:0/2942006829 conn(0x7f634006cb50 msgr2=0x7f634006cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:13.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.474+0000 7f6346d46700 1 -- 172.21.15.78:0/2942006829 shutdown_connections 2023-10-11T09:41:13.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.474+0000 7f6346d46700 1 -- 172.21.15.78:0/2942006829 wait complete. 2023-10-11T09:41:13.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.478+0000 7f6346d46700 1 Processor -- start 2023-10-11T09:41:13.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.478+0000 7f6346d46700 1 -- start start 2023-10-11T09:41:13.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.482+0000 7f6346d46700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63400b84c0 0x7f634015b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:13.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.482+0000 7f6346d46700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63400b8890 0x7f63401f2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:13.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.482+0000 7f6346d46700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63401608b0 0x7f63401e8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:13.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.482+0000 7f6346d46700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63401e87b0 con 0x7f63400b84c0 2023-10-11T09:41:13.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.482+0000 7f6346d46700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63401e8320 con 0x7f63400b8890 2023-10-11T09:41:13.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.482+0000 7f6346d46700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63401e7750 con 0x7f63401608b0 2023-10-11T09:41:13.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.482+0000 7f6344ae2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63400b84c0 0x7f634015b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:13.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.482+0000 7f6344ae2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63400b84c0 0x7f634015b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53178/0 (socket says 172.21.15.78:53178) 2023-10-11T09:41:13.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.482+0000 7f6344ae2700 1 -- 172.21.15.78:0/1368582251 learned_addr learned my addr 172.21.15.78:0/1368582251 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:13.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.482+0000 7f633ffff700 1 --2- 172.21.15.78:0/1368582251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63400b8890 0x7f63401f2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:13.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.482+0000 7f63452e3700 1 --2- 172.21.15.78:0/1368582251 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63401608b0 0x7f63401e8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:13.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.482+0000 7f633ffff700 1 -- 172.21.15.78:0/1368582251 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63401608b0 msgr2=0x7f63401e8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:13.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.482+0000 7f633ffff700 1 --2- 172.21.15.78:0/1368582251 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63401608b0 0x7f63401e8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:13.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.482+0000 7f633ffff700 1 -- 172.21.15.78:0/1368582251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63400b84c0 msgr2=0x7f634015b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:13.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.482+0000 7f633ffff700 1 --2- 172.21.15.78:0/1368582251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63400b84c0 0x7f634015b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:13.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.482+0000 7f633ffff700 1 -- 172.21.15.78:0/1368582251 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63340094b0 con 0x7f63400b8890 2023-10-11T09:41:13.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.482+0000 7f6344ae2700 1 --2- 172.21.15.78:0/1368582251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63400b84c0 0x7f634015b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:41:13.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.482+0000 7f633ffff700 1 --2- 172.21.15.78:0/1368582251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63400b8890 0x7f63401f2e40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f63400c6ab0 tx=0x7f632c004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:13.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.482+0000 7f633d7fa700 1 -- 172.21.15.78:0/1368582251 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c004510 con 0x7f63400b8890 2023-10-11T09:41:13.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.482+0000 7f633d7fa700 1 -- 172.21.15.78:0/1368582251 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f632c0036c0 con 0x7f63400b8890 2023-10-11T09:41:13.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.482+0000 7f6346d46700 1 -- 172.21.15.78:0/1368582251 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f632c0094b0 con 0x7f63400b8890 2023-10-11T09:41:13.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.482+0000 7f633d7fa700 1 -- 172.21.15.78:0/1368582251 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c003a70 con 0x7f63400b8890 2023-10-11T09:41:13.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.482+0000 7f6346d46700 1 -- 172.21.15.78:0/1368582251 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63401e79f0 con 0x7f63400b8890 2023-10-11T09:41:13.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.486+0000 7f6346d46700 1 -- 172.21.15.78:0/1368582251 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f634008bff0 con 0x7f63400b8890 2023-10-11T09:41:13.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.486+0000 7f633d7fa700 1 -- 172.21.15.78:0/1368582251 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f632c003bd0 con 0x7f63400b8890 2023-10-11T09:41:13.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.486+0000 7f633d7fa700 1 --2- 172.21.15.78:0/1368582251 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6324065ed0 0x7f6324068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:13.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.486+0000 7f633d7fa700 1 -- 172.21.15.78:0/1368582251 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f632c021030 con 0x7f63400b8890 2023-10-11T09:41:13.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.486+0000 7f6344ae2700 1 --2- 172.21.15.78:0/1368582251 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6324065ed0 0x7f6324068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:13.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.490+0000 7f6344ae2700 1 --2- 172.21.15.78:0/1368582251 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6324065ed0 0x7f6324068380 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f63340114b0 tx=0x7f6334009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:13.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.494+0000 7f633d7fa700 1 -- 172.21.15.78:0/1368582251 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f632c0522d0 con 0x7f63400b8890 2023-10-11T09:41:13.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.670+0000 7f6346d46700 1 -- 172.21.15.78:0/1368582251 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f63400b5a40 con 0x7f63400b8890 2023-10-11T09:41:13.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.674+0000 7f633d7fa700 1 -- 172.21.15.78:0/1368582251 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f632c015020 con 0x7f63400b8890 2023-10-11T09:41:13.679 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:41:13.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.678+0000 7f6346d46700 1 -- 172.21.15.78:0/1368582251 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6324065ed0 msgr2=0x7f6324068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:13.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.678+0000 7f6346d46700 1 --2- 172.21.15.78:0/1368582251 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6324065ed0 0x7f6324068380 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f63340114b0 tx=0x7f6334009240 comp rx=0 tx=0).stop 2023-10-11T09:41:13.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.678+0000 7f6346d46700 1 -- 172.21.15.78:0/1368582251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63400b8890 msgr2=0x7f63401f2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:13.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.678+0000 7f6346d46700 1 --2- 172.21.15.78:0/1368582251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63400b8890 0x7f63401f2e40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f63400c6ab0 tx=0x7f632c004770 comp rx=0 tx=0).stop 2023-10-11T09:41:13.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.678+0000 7f6346d46700 1 -- 172.21.15.78:0/1368582251 shutdown_connections 2023-10-11T09:41:13.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.678+0000 7f6346d46700 1 --2- 172.21.15.78:0/1368582251 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6324065ed0 0x7f6324068380 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:13.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.678+0000 7f6346d46700 1 --2- 172.21.15.78:0/1368582251 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63401608b0 0x7f63401e8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:13.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.678+0000 7f6346d46700 1 --2- 172.21.15.78:0/1368582251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63400b8890 0x7f63401f2e40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:13.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.678+0000 7f6346d46700 1 --2- 172.21.15.78:0/1368582251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63400b84c0 0x7f634015b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:13.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.678+0000 7f6346d46700 1 -- 172.21.15.78:0/1368582251 >> 172.21.15.78:0/1368582251 conn(0x7f634006cb50 msgr2=0x7f63400b3890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:13.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.678+0000 7f6346d46700 1 -- 172.21.15.78:0/1368582251 shutdown_connections 2023-10-11T09:41:13.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.678+0000 7f6346d46700 1 -- 172.21.15.78:0/1368582251 wait complete. 2023-10-11T09:41:13.704 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T09:41:13.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.870+0000 7f43693dc700 1 Processor -- start 2023-10-11T09:41:13.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.870+0000 7f43693dc700 1 -- start start 2023-10-11T09:41:13.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.874+0000 7f43693dc700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f43640b3610 0x7f43640d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:13.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.874+0000 7f43693dc700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43640b47f0 con 0x7f43640b3a90 2023-10-11T09:41:13.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.874+0000 7f43693dc700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43640b8130 con 0x7f43640b3240 2023-10-11T09:41:13.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.874+0000 7f43693dc700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43640b41e0 con 0x7f43640b3610 2023-10-11T09:41:13.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.874+0000 7f4368bdb700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f43640b3a90 0x7f43640b85c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:42092/0 (socket says 172.21.15.78:42092) 2023-10-11T09:41:13.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.874+0000 7f4368bdb700 1 -- 172.21.15.78:0/402309261 learned_addr learned my addr 172.21.15.78:0/402309261 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:13.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.874+0000 7f43637fe700 1 --2- 172.21.15.78:0/402309261 >> v2:172.21.15.133:3300/0 conn(0x7f43640b3610 0x7f43640d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:13.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.874+0000 7f43637fe700 1 -- 172.21.15.78:0/402309261 >> v1:172.21.15.133:6789/0 conn(0x7f43640b3240 legacy=0x7f436415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:13.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.874+0000 7f43637fe700 1 -- 172.21.15.78:0/402309261 >> v1:172.21.15.78:6789/0 conn(0x7f43640b3a90 legacy=0x7f43640b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:13.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.874+0000 7f43637fe700 1 -- 172.21.15.78:0/402309261 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43640b7800 con 0x7f43640b3610 2023-10-11T09:41:13.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.874+0000 7f43637fe700 1 --2- 172.21.15.78:0/402309261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43640b3610 0x7f43640d29d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4358009850 tx=0x7f4358009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0950f5611f2d74d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:13.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.878+0000 7f4362ffd700 1 -- 172.21.15.78:0/402309261 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4358004230 con 0x7f43640b3610 2023-10-11T09:41:13.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.878+0000 7f4362ffd700 1 -- 172.21.15.78:0/402309261 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4358004390 con 0x7f43640b3610 2023-10-11T09:41:13.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.878+0000 7f4362ffd700 1 -- 172.21.15.78:0/402309261 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43580036c0 con 0x7f43640b3610 2023-10-11T09:41:13.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.878+0000 7f43693dc700 1 -- 172.21.15.78:0/402309261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43640b3610 msgr2=0x7f43640d29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:13.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.878+0000 7f43693dc700 1 --2- 172.21.15.78:0/402309261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43640b3610 0x7f43640d29d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4358009850 tx=0x7f4358009b60 comp rx=0 tx=0).stop 2023-10-11T09:41:13.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.878+0000 7f43693dc700 1 -- 172.21.15.78:0/402309261 shutdown_connections 2023-10-11T09:41:13.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.878+0000 7f43693dc700 1 --2- 172.21.15.78:0/402309261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43640b3610 0x7f43640d29d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:13.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.878+0000 7f43693dc700 1 -- 172.21.15.78:0/402309261 >> 172.21.15.78:0/402309261 conn(0x7f436409c260 msgr2=0x7f43640d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:13.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.878+0000 7f43693dc700 1 -- 172.21.15.78:0/402309261 shutdown_connections 2023-10-11T09:41:13.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.878+0000 7f43693dc700 1 -- 172.21.15.78:0/402309261 wait complete. 2023-10-11T09:41:13.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.878+0000 7f43693dc700 1 Processor -- start 2023-10-11T09:41:13.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.882+0000 7f43693dc700 1 -- start start 2023-10-11T09:41:13.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.882+0000 7f43693dc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43640b3240 0x7f43641f4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:13.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.882+0000 7f43693dc700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43640b3610 0x7f43641f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:13.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.882+0000 7f43693dc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43640b3a90 0x7f4364149460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:13.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.882+0000 7f43693dc700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436414cfb0 con 0x7f43640b3240 2023-10-11T09:41:13.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.882+0000 7f43693dc700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436415e060 con 0x7f43640b3a90 2023-10-11T09:41:13.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.882+0000 7f43693dc700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f436415e340 con 0x7f43640b3610 2023-10-11T09:41:13.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.882+0000 7f4363fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43640b3240 0x7f43641f4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:13.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.882+0000 7f4363fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43640b3240 0x7f43641f4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53190/0 (socket says 172.21.15.78:53190) 2023-10-11T09:41:13.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.882+0000 7f4363fff700 1 -- 172.21.15.78:0/577114922 learned_addr learned my addr 172.21.15.78:0/577114922 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:13.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.882+0000 7f43637fe700 1 --2- 172.21.15.78:0/577114922 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43640b3610 0x7f43641f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:13.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.882+0000 7f4363fff700 1 -- 172.21.15.78:0/577114922 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43640b3610 msgr2=0x7f43641f51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:13.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.882+0000 7f4363fff700 1 --2- 172.21.15.78:0/577114922 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43640b3610 0x7f43641f51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:13.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.882+0000 7f4368bdb700 1 --2- 172.21.15.78:0/577114922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43640b3a90 0x7f4364149460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:13.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.882+0000 7f4363fff700 1 -- 172.21.15.78:0/577114922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43640b3a90 msgr2=0x7f4364149460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:13.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.882+0000 7f4363fff700 1 --2- 172.21.15.78:0/577114922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43640b3a90 0x7f4364149460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:13.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.882+0000 7f4363fff700 1 -- 172.21.15.78:0/577114922 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43580094b0 con 0x7f43640b3240 2023-10-11T09:41:13.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.882+0000 7f4368bdb700 1 --2- 172.21.15.78:0/577114922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43640b3a90 0x7f4364149460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:41:13.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.882+0000 7f4363fff700 1 --2- 172.21.15.78:0/577114922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43640b3240 0x7f43641f4c80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f435400c500 tx=0x7f435400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:13.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.882+0000 7f4360ff9700 1 -- 172.21.15.78:0/577114922 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43540052d0 con 0x7f43640b3240 2023-10-11T09:41:13.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.882+0000 7f4360ff9700 1 -- 172.21.15.78:0/577114922 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4354005430 con 0x7f43640b3240 2023-10-11T09:41:13.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.882+0000 7f4360ff9700 1 -- 172.21.15.78:0/577114922 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43540055a0 con 0x7f43640b3240 2023-10-11T09:41:13.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.882+0000 7f43693dc700 1 -- 172.21.15.78:0/577114922 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f436415e620 con 0x7f43640b3240 2023-10-11T09:41:13.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.886+0000 7f43693dc700 1 -- 172.21.15.78:0/577114922 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43641f5700 con 0x7f43640b3240 2023-10-11T09:41:13.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.886+0000 7f43693dc700 1 -- 172.21.15.78:0/577114922 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4364065d40 con 0x7f43640b3240 2023-10-11T09:41:13.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.886+0000 7f4360ff9700 1 -- 172.21.15.78:0/577114922 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f43540038e0 con 0x7f43640b3240 2023-10-11T09:41:13.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.890+0000 7f4360ff9700 1 --2- 172.21.15.78:0/577114922 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4350065ed0 0x7f4350068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:13.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.890+0000 7f4360ff9700 1 -- 172.21.15.78:0/577114922 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4354082c60 con 0x7f43640b3240 2023-10-11T09:41:13.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.890+0000 7f43637fe700 1 --2- 172.21.15.78:0/577114922 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4350065ed0 0x7f4350068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:13.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.890+0000 7f43637fe700 1 --2- 172.21.15.78:0/577114922 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4350065ed0 0x7f4350068380 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4358009850 tx=0x7f4358010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:13.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:13.894+0000 7f4360ff9700 1 -- 172.21.15.78:0/577114922 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f43540502c0 con 0x7f43640b3240 2023-10-11T09:41:14.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.066+0000 7f43693dc700 1 -- 172.21.15.78:0/577114922 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f436408bdb0 con 0x7f43640b3240 2023-10-11T09:41:14.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.066+0000 7f4360ff9700 1 -- 172.21.15.78:0/577114922 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f43540538e0 con 0x7f43640b3240 2023-10-11T09:41:14.072 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2023-10-11T09:41:14.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.070+0000 7f43693dc700 1 -- 172.21.15.78:0/577114922 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4350065ed0 msgr2=0x7f4350068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:14.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.070+0000 7f43693dc700 1 --2- 172.21.15.78:0/577114922 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4350065ed0 0x7f4350068380 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4358009850 tx=0x7f4358010000 comp rx=0 tx=0).stop 2023-10-11T09:41:14.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.070+0000 7f43693dc700 1 -- 172.21.15.78:0/577114922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43640b3240 msgr2=0x7f43641f4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:14.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.070+0000 7f43693dc700 1 --2- 172.21.15.78:0/577114922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43640b3240 0x7f43641f4c80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f435400c500 tx=0x7f435400c8c0 comp rx=0 tx=0).stop 2023-10-11T09:41:14.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.070+0000 7f43693dc700 1 -- 172.21.15.78:0/577114922 shutdown_connections 2023-10-11T09:41:14.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.070+0000 7f43693dc700 1 --2- 172.21.15.78:0/577114922 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4350065ed0 0x7f4350068380 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:14.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.070+0000 7f43693dc700 1 --2- 172.21.15.78:0/577114922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43640b3a90 0x7f4364149460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:14.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.070+0000 7f43693dc700 1 --2- 172.21.15.78:0/577114922 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43640b3610 0x7f43641f51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:14.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.070+0000 7f43693dc700 1 --2- 172.21.15.78:0/577114922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43640b3240 0x7f43641f4c80 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:14.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.070+0000 7f43693dc700 1 -- 172.21.15.78:0/577114922 >> 172.21.15.78:0/577114922 conn(0x7f436409c260 msgr2=0x7f43640d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:14.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.070+0000 7f43693dc700 1 -- 172.21.15.78:0/577114922 shutdown_connections 2023-10-11T09:41:14.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.070+0000 7f43693dc700 1 -- 172.21.15.78:0/577114922 wait complete. 2023-10-11T09:41:14.095 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:41:14.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.266+0000 7ff741d51700 1 Processor -- start 2023-10-11T09:41:14.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.266+0000 7ff741d51700 1 -- start start 2023-10-11T09:41:14.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.270+0000 7ff741d51700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ff73c15b8e0 0x7ff73c06d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:14.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.270+0000 7ff741d51700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff73c15e010 con 0x7ff73c06d610 2023-10-11T09:41:14.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.270+0000 7ff741d51700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff73c15e280 con 0x7ff73c0ca950 2023-10-11T09:41:14.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.270+0000 7ff741d51700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff73c15e560 con 0x7ff73c15b8e0 2023-10-11T09:41:14.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.270+0000 7ff740d4f700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ff73c15b8e0 0x7ff73c06d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:14.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.270+0000 7ff733fff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7ff73c06d610 0x7ff73c0ca240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:42094/0 (socket says 172.21.15.78:42094) 2023-10-11T09:41:14.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.270+0000 7ff733fff700 1 -- 172.21.15.78:0/2985060609 learned_addr learned my addr 172.21.15.78:0/2985060609 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:14.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.270+0000 7ff7337fe700 1 -- 172.21.15.78:0/2985060609 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 648510173 0 0) 0x7ff73c15e010 con 0x7ff73c06d610 2023-10-11T09:41:14.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.270+0000 7ff7337fe700 1 -- 172.21.15.78:0/2985060609 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7240035d0 con 0x7ff73c06d610 2023-10-11T09:41:14.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.270+0000 7ff740d4f700 1 -- 172.21.15.78:0/2985060609 >> v1:172.21.15.133:6789/0 conn(0x7ff73c0ca950 legacy=0x7ff73c13fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:14.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.270+0000 7ff740d4f700 1 -- 172.21.15.78:0/2985060609 >> v1:172.21.15.78:6789/0 conn(0x7ff73c06d610 legacy=0x7ff73c0ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:14.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.270+0000 7ff740d4f700 1 -- 172.21.15.78:0/2985060609 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff73c15e840 con 0x7ff73c15b8e0 2023-10-11T09:41:14.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.270+0000 7ff740d4f700 1 --2- 172.21.15.78:0/2985060609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff73c15b8e0 0x7ff73c06d0d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff738009850 tx=0x7ff738009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3301e436b691599 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:14.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.270+0000 7ff7337fe700 1 -- 172.21.15.78:0/2985060609 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff738011830 con 0x7ff73c15b8e0 2023-10-11T09:41:14.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.270+0000 7ff7337fe700 1 -- 172.21.15.78:0/2985060609 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff738011990 con 0x7ff73c15b8e0 2023-10-11T09:41:14.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.270+0000 7ff7337fe700 1 -- 172.21.15.78:0/2985060609 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff738011d40 con 0x7ff73c15b8e0 2023-10-11T09:41:14.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.274+0000 7ff741d51700 1 -- 172.21.15.78:0/2985060609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff73c15b8e0 msgr2=0x7ff73c06d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:14.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.274+0000 7ff741d51700 1 --2- 172.21.15.78:0/2985060609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff73c15b8e0 0x7ff73c06d0d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff738009850 tx=0x7ff738009b60 comp rx=0 tx=0).stop 2023-10-11T09:41:14.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.274+0000 7ff741d51700 1 -- 172.21.15.78:0/2985060609 shutdown_connections 2023-10-11T09:41:14.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.274+0000 7ff741d51700 1 --2- 172.21.15.78:0/2985060609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff73c15b8e0 0x7ff73c06d0d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:14.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.274+0000 7ff741d51700 1 -- 172.21.15.78:0/2985060609 >> 172.21.15.78:0/2985060609 conn(0x7ff73c0cea60 msgr2=0x7ff73c0cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:14.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.274+0000 7ff741d51700 1 -- 172.21.15.78:0/2985060609 shutdown_connections 2023-10-11T09:41:14.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.274+0000 7ff741d51700 1 -- 172.21.15.78:0/2985060609 wait complete. 2023-10-11T09:41:14.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.274+0000 7ff741d51700 1 Processor -- start 2023-10-11T09:41:14.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.274+0000 7ff741d51700 1 -- start start 2023-10-11T09:41:14.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.278+0000 7ff741d51700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff73c06d610 0x7ff73c1efa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:14.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.278+0000 7ff741d51700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff73c0ca950 0x7ff73c141700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:14.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.278+0000 7ff741d51700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff73c15b8e0 0x7ff73c0c4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:14.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.278+0000 7ff741d51700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c13dd20 con 0x7ff73c06d610 2023-10-11T09:41:14.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.278+0000 7ff741d51700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c142b40 con 0x7ff73c15b8e0 2023-10-11T09:41:14.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.278+0000 7ff741d51700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c141f30 con 0x7ff73c0ca950 2023-10-11T09:41:14.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.278+0000 7ff740d4f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff73c06d610 0x7ff73c1efa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:14.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.278+0000 7ff740d4f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff73c06d610 0x7ff73c1efa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53210/0 (socket says 172.21.15.78:53210) 2023-10-11T09:41:14.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.278+0000 7ff740d4f700 1 -- 172.21.15.78:0/3557677305 learned_addr learned my addr 172.21.15.78:0/3557677305 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:14.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.278+0000 7ff733fff700 1 --2- 172.21.15.78:0/3557677305 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff73c0ca950 0x7ff73c141700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:14.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.278+0000 7ff740d4f700 1 -- 172.21.15.78:0/3557677305 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff73c0ca950 msgr2=0x7ff73c141700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:14.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.278+0000 7ff740d4f700 1 --2- 172.21.15.78:0/3557677305 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff73c0ca950 0x7ff73c141700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:14.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.278+0000 7ff740d4f700 1 -- 172.21.15.78:0/3557677305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff73c15b8e0 msgr2=0x7ff73c0c4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:14.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.278+0000 7ff741550700 1 --2- 172.21.15.78:0/3557677305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff73c15b8e0 0x7ff73c0c4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:14.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.278+0000 7ff740d4f700 1 --2- 172.21.15.78:0/3557677305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff73c15b8e0 0x7ff73c0c4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:14.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.278+0000 7ff740d4f700 1 -- 172.21.15.78:0/3557677305 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7380094b0 con 0x7ff73c06d610 2023-10-11T09:41:14.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.278+0000 7ff741550700 1 --2- 172.21.15.78:0/3557677305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff73c15b8e0 0x7ff73c0c4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:41:14.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.278+0000 7ff740d4f700 1 --2- 172.21.15.78:0/3557677305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff73c06d610 0x7ff73c1efa20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff738009ba0 tx=0x7ff738004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:14.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.278+0000 7ff7317fa700 1 -- 172.21.15.78:0/3557677305 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7380036c0 con 0x7ff73c06d610 2023-10-11T09:41:14.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.278+0000 7ff7317fa700 1 -- 172.21.15.78:0/3557677305 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff73801e070 con 0x7ff73c06d610 2023-10-11T09:41:14.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.278+0000 7ff741d51700 1 -- 172.21.15.78:0/3557677305 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff73c142210 con 0x7ff73c06d610 2023-10-11T09:41:14.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.278+0000 7ff7317fa700 1 -- 172.21.15.78:0/3557677305 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff738003a80 con 0x7ff73c06d610 2023-10-11T09:41:14.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.278+0000 7ff741d51700 1 -- 172.21.15.78:0/3557677305 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff73c1427d0 con 0x7ff73c06d610 2023-10-11T09:41:14.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.282+0000 7ff741d51700 1 -- 172.21.15.78:0/3557677305 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff73c067490 con 0x7ff73c06d610 2023-10-11T09:41:14.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.282+0000 7ff7317fa700 1 -- 172.21.15.78:0/3557677305 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7ff738003be0 con 0x7ff73c06d610 2023-10-11T09:41:14.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.282+0000 7ff7317fa700 1 --2- 172.21.15.78:0/3557677305 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff728065ed0 0x7ff728068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:14.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.282+0000 7ff7317fa700 1 -- 172.21.15.78:0/3557677305 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff738084790 con 0x7ff73c06d610 2023-10-11T09:41:14.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.282+0000 7ff733fff700 1 --2- 172.21.15.78:0/3557677305 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff728065ed0 0x7ff728068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:14.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.286+0000 7ff733fff700 1 --2- 172.21.15.78:0/3557677305 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff728065ed0 0x7ff728068380 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0cf6e0 tx=0x7ff72c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:14.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.290+0000 7ff7317fa700 1 -- 172.21.15.78:0/3557677305 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7ff738051dd0 con 0x7ff73c06d610 2023-10-11T09:41:14.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.470+0000 7ff741d51700 1 -- 172.21.15.78:0/3557677305 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff73c065d40 con 0x7ff73c06d610 2023-10-11T09:41:14.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.474+0000 7ff7317fa700 1 -- 172.21.15.78:0/3557677305 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17811 (secure 0 0 0) 0x7ff7380553f0 con 0x7ff73c06d610 2023-10-11T09:41:14.477 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:41:14.477 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":23,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:40:42.678477+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:40:39.862581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T09:40:40.663641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:41:14.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.478+0000 7ff741d51700 1 -- 172.21.15.78:0/3557677305 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff728065ed0 msgr2=0x7ff728068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:14.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.478+0000 7ff741d51700 1 --2- 172.21.15.78:0/3557677305 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff728065ed0 0x7ff728068380 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0cf6e0 tx=0x7ff72c009040 comp rx=0 tx=0).stop 2023-10-11T09:41:14.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.478+0000 7ff741d51700 1 -- 172.21.15.78:0/3557677305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff73c06d610 msgr2=0x7ff73c1efa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:14.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.478+0000 7ff741d51700 1 --2- 172.21.15.78:0/3557677305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff73c06d610 0x7ff73c1efa20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff738009ba0 tx=0x7ff738004770 comp rx=0 tx=0).stop 2023-10-11T09:41:14.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.478+0000 7ff741d51700 1 -- 172.21.15.78:0/3557677305 shutdown_connections 2023-10-11T09:41:14.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.478+0000 7ff741d51700 1 --2- 172.21.15.78:0/3557677305 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff728065ed0 0x7ff728068380 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:14.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.478+0000 7ff741d51700 1 --2- 172.21.15.78:0/3557677305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff73c15b8e0 0x7ff73c0c4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:14.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.478+0000 7ff741d51700 1 --2- 172.21.15.78:0/3557677305 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff73c0ca950 0x7ff73c141700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:14.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.478+0000 7ff741d51700 1 --2- 172.21.15.78:0/3557677305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff73c06d610 0x7ff73c1efa20 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:14.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.478+0000 7ff741d51700 1 -- 172.21.15.78:0/3557677305 >> 172.21.15.78:0/3557677305 conn(0x7ff73c0cea60 msgr2=0x7ff73c0cb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:14.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.478+0000 7ff741d51700 1 -- 172.21.15.78:0/3557677305 shutdown_connections 2023-10-11T09:41:14.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.478+0000 7ff741d51700 1 -- 172.21.15.78:0/3557677305 wait complete. 2023-10-11T09:41:14.504 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T09:40:39.862581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T09:40:40.663641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:41:14.505 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:41:14.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.678+0000 7fbbecabf700 1 Processor -- start 2023-10-11T09:41:14.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.678+0000 7fbbecabf700 1 -- start start 2023-10-11T09:41:14.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.678+0000 7fbbecabf700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbbe80caf90 0x7fbbe815afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:14.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.678+0000 7fbbecabf700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe815e280 con 0x7fbbe8158430 2023-10-11T09:41:14.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.678+0000 7fbbecabf700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe815e560 con 0x7fbbe806d340 2023-10-11T09:41:14.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.678+0000 7fbbecabf700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe8070c10 con 0x7fbbe80caf90 2023-10-11T09:41:14.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.678+0000 7fbbe659c700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fbbe8158430 0x7fbbe80710d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:55770/0 (socket says 172.21.15.78:55770) 2023-10-11T09:41:14.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.678+0000 7fbbe659c700 1 -- 172.21.15.78:0/447627350 learned_addr learned my addr 172.21.15.78:0/447627350 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:14.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.678+0000 7fbbe6d9d700 1 --2- 172.21.15.78:0/447627350 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbbe80caf90 0x7fbbe815afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:14.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.682+0000 7fbbe559a700 1 -- 172.21.15.78:0/447627350 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3184744560 0 0) 0x7fbbe815e280 con 0x7fbbe8158430 2023-10-11T09:41:14.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.682+0000 7fbbe559a700 1 -- 172.21.15.78:0/447627350 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbd00035d0 con 0x7fbbe8158430 2023-10-11T09:41:14.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.682+0000 7fbbe559a700 1 -- 172.21.15.78:0/447627350 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3274481875 0 0) 0x7fbbe815e560 con 0x7fbbe806d340 2023-10-11T09:41:14.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.682+0000 7fbbe559a700 1 -- 172.21.15.78:0/447627350 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbe815e280 con 0x7fbbe806d340 2023-10-11T09:41:14.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.682+0000 7fbbe6d9d700 1 -- 172.21.15.78:0/447627350 >> v1:172.21.15.133:6789/0 conn(0x7fbbe806d340 legacy=0x7fbbe80ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:14.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.682+0000 7fbbe6d9d700 1 -- 172.21.15.78:0/447627350 >> v1:172.21.15.78:6789/0 conn(0x7fbbe8158430 legacy=0x7fbbe80710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:14.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.682+0000 7fbbe6d9d700 1 -- 172.21.15.78:0/447627350 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe815e840 con 0x7fbbe80caf90 2023-10-11T09:41:14.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.682+0000 7fbbe6d9d700 1 --2- 172.21.15.78:0/447627350 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbbe80caf90 0x7fbbe815afe0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0009dc0 tx=0x7fbbe000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7cf5eed1f88776c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:14.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.682+0000 7fbbe559a700 1 -- 172.21.15.78:0/447627350 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe000e070 con 0x7fbbe80caf90 2023-10-11T09:41:14.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.682+0000 7fbbe559a700 1 -- 172.21.15.78:0/447627350 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe000bb20 con 0x7fbbe80caf90 2023-10-11T09:41:14.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.682+0000 7fbbe559a700 1 -- 172.21.15.78:0/447627350 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0012430 con 0x7fbbe80caf90 2023-10-11T09:41:14.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.682+0000 7fbbecabf700 1 -- 172.21.15.78:0/447627350 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbbe80caf90 msgr2=0x7fbbe815afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:14.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.682+0000 7fbbecabf700 1 --2- 172.21.15.78:0/447627350 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbbe80caf90 0x7fbbe815afe0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0009dc0 tx=0x7fbbe000b330 comp rx=0 tx=0).stop 2023-10-11T09:41:14.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.682+0000 7fbbecabf700 1 -- 172.21.15.78:0/447627350 shutdown_connections 2023-10-11T09:41:14.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.682+0000 7fbbecabf700 1 --2- 172.21.15.78:0/447627350 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbbe80caf90 0x7fbbe815afe0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:14.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.682+0000 7fbbecabf700 1 -- 172.21.15.78:0/447627350 >> 172.21.15.78:0/447627350 conn(0x7fbbe80cecc0 msgr2=0x7fbbe80cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:14.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.682+0000 7fbbecabf700 1 -- 172.21.15.78:0/447627350 shutdown_connections 2023-10-11T09:41:14.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.682+0000 7fbbecabf700 1 -- 172.21.15.78:0/447627350 wait complete. 2023-10-11T09:41:14.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.686+0000 7fbbecabf700 1 Processor -- start 2023-10-11T09:41:14.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.686+0000 7fbbecabf700 1 -- start start 2023-10-11T09:41:14.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.690+0000 7fbbecabf700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbbe806d340 0x7fbbe81498c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:14.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.690+0000 7fbbecabf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbe80caf90 0x7fbbe80ce370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:14.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.690+0000 7fbbecabf700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbe8158430 0x7fbbe80b0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:14.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.690+0000 7fbbecabf700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe80b10b0 con 0x7fbbe80caf90 2023-10-11T09:41:14.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.690+0000 7fbbecabf700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe8155870 con 0x7fbbe8158430 2023-10-11T09:41:14.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.690+0000 7fbbecabf700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe8155b50 con 0x7fbbe806d340 2023-10-11T09:41:14.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.690+0000 7fbbe5d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbe80caf90 0x7fbbe80ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:14.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.690+0000 7fbbe5d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbe80caf90 0x7fbbe80ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50630/0 (socket says 172.21.15.78:50630) 2023-10-11T09:41:14.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.690+0000 7fbbe5d9b700 1 -- 172.21.15.78:0/996444626 learned_addr learned my addr 172.21.15.78:0/996444626 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:14.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.690+0000 7fbbe659c700 1 --2- 172.21.15.78:0/996444626 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbbe806d340 0x7fbbe81498c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:14.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.690+0000 7fbbe6d9d700 1 --2- 172.21.15.78:0/996444626 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbe8158430 0x7fbbe80b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:14.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.690+0000 7fbbe5d9b700 1 -- 172.21.15.78:0/996444626 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbbe806d340 msgr2=0x7fbbe81498c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:14.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.690+0000 7fbbe5d9b700 1 --2- 172.21.15.78:0/996444626 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbbe806d340 0x7fbbe81498c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:14.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.690+0000 7fbbe5d9b700 1 -- 172.21.15.78:0/996444626 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbe8158430 msgr2=0x7fbbe80b0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:14.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.690+0000 7fbbe5d9b700 1 --2- 172.21.15.78:0/996444626 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbe8158430 0x7fbbe80b0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:14.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.690+0000 7fbbe5d9b700 1 -- 172.21.15.78:0/996444626 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe0009a70 con 0x7fbbe80caf90 2023-10-11T09:41:14.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.690+0000 7fbbe5d9b700 1 --2- 172.21.15.78:0/996444626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbe80caf90 0x7fbbe80ce370 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00c730 tx=0x7fbbdc00ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:14.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.690+0000 7fbbd6ffd700 1 -- 172.21.15.78:0/996444626 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc0052d0 con 0x7fbbe80caf90 2023-10-11T09:41:14.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.690+0000 7fbbd6ffd700 1 -- 172.21.15.78:0/996444626 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbdc005450 con 0x7fbbe80caf90 2023-10-11T09:41:14.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.690+0000 7fbbecabf700 1 -- 172.21.15.78:0/996444626 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe8155e30 con 0x7fbbe80caf90 2023-10-11T09:41:14.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.694+0000 7fbbd6ffd700 1 -- 172.21.15.78:0/996444626 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc0038e0 con 0x7fbbe80caf90 2023-10-11T09:41:14.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.694+0000 7fbbecabf700 1 -- 172.21.15.78:0/996444626 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe80b0d80 con 0x7fbbe80caf90 2023-10-11T09:41:14.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.694+0000 7fbbecabf700 1 -- 172.21.15.78:0/996444626 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe806d790 con 0x7fbbe80caf90 2023-10-11T09:41:14.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.698+0000 7fbbd6ffd700 1 -- 172.21.15.78:0/996444626 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fbbdc003a40 con 0x7fbbe80caf90 2023-10-11T09:41:14.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.698+0000 7fbbd6ffd700 1 --2- 172.21.15.78:0/996444626 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbbc8065ed0 0x7fbbc8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:14.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.698+0000 7fbbd6ffd700 1 -- 172.21.15.78:0/996444626 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbbdc082c60 con 0x7fbbe80caf90 2023-10-11T09:41:14.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.698+0000 7fbbe659c700 1 --2- 172.21.15.78:0/996444626 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbbc8065ed0 0x7fbbc8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:14.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.698+0000 7fbbe659c700 1 --2- 172.21.15.78:0/996444626 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbbc8065ed0 0x7fbbc8068380 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8007fd0 tx=0x7fbbd8007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:14.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.706+0000 7fbbd6ffd700 1 -- 172.21.15.78:0/996444626 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fbbdc0502a0 con 0x7fbbe80caf90 2023-10-11T09:41:14.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.874+0000 7fbbecabf700 1 -- 172.21.15.78:0/996444626 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbbe8157de0 con 0x7fbbe80caf90 2023-10-11T09:41:14.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.874+0000 7fbbd6ffd700 1 -- 172.21.15.78:0/996444626 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbbdc0170a0 con 0x7fbbe80caf90 2023-10-11T09:41:14.881 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:41:14.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.882+0000 7fbbecabf700 1 -- 172.21.15.78:0/996444626 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbbc8065ed0 msgr2=0x7fbbc8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:14.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.882+0000 7fbbecabf700 1 --2- 172.21.15.78:0/996444626 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbbc8065ed0 0x7fbbc8068380 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8007fd0 tx=0x7fbbd8007de0 comp rx=0 tx=0).stop 2023-10-11T09:41:14.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.882+0000 7fbbecabf700 1 -- 172.21.15.78:0/996444626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbe80caf90 msgr2=0x7fbbe80ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:14.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.882+0000 7fbbecabf700 1 --2- 172.21.15.78:0/996444626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbe80caf90 0x7fbbe80ce370 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00c730 tx=0x7fbbdc00ca40 comp rx=0 tx=0).stop 2023-10-11T09:41:14.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.882+0000 7fbbecabf700 1 -- 172.21.15.78:0/996444626 shutdown_connections 2023-10-11T09:41:14.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.882+0000 7fbbecabf700 1 --2- 172.21.15.78:0/996444626 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbbc8065ed0 0x7fbbc8068380 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:14.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.882+0000 7fbbecabf700 1 --2- 172.21.15.78:0/996444626 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbe8158430 0x7fbbe80b0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:14.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.882+0000 7fbbecabf700 1 --2- 172.21.15.78:0/996444626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbe80caf90 0x7fbbe80ce370 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:14.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.882+0000 7fbbecabf700 1 --2- 172.21.15.78:0/996444626 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbbe806d340 0x7fbbe81498c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:14.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.882+0000 7fbbecabf700 1 -- 172.21.15.78:0/996444626 >> 172.21.15.78:0/996444626 conn(0x7fbbe80cecc0 msgr2=0x7fbbe8071810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:14.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.882+0000 7fbbecabf700 1 -- 172.21.15.78:0/996444626 shutdown_connections 2023-10-11T09:41:14.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:14.882+0000 7fbbecabf700 1 -- 172.21.15.78:0/996444626 wait complete. 2023-10-11T09:41:14.907 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:41:15.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.078+0000 7f6d9ce82700 1 Processor -- start 2023-10-11T09:41:15.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.078+0000 7f6d9ce82700 1 -- start start 2023-10-11T09:41:15.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.082+0000 7f6d9ce82700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6d9806d340 0x7f6d980ca880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:15.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.082+0000 7f6d9ce82700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d9815e280 con 0x7f6d980cadc0 2023-10-11T09:41:15.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.082+0000 7f6d9ce82700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d9815e560 con 0x7f6d98158430 2023-10-11T09:41:15.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.082+0000 7f6d9ce82700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d98070c10 con 0x7f6d9806d340 2023-10-11T09:41:15.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.082+0000 7f6d96d9d700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f6d980cadc0 0x7f6d9815afe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:55778/0 (socket says 172.21.15.78:55778) 2023-10-11T09:41:15.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.082+0000 7f6d96d9d700 1 -- 172.21.15.78:0/40531037 learned_addr learned my addr 172.21.15.78:0/40531037 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:15.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.082+0000 7f6d95d9b700 1 --2- 172.21.15.78:0/40531037 >> v2:172.21.15.133:3300/0 conn(0x7f6d9806d340 0x7f6d980ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:15.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.082+0000 7f6d9559a700 1 -- 172.21.15.78:0/40531037 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 238373084 0 0) 0x7f6d9815e280 con 0x7f6d980cadc0 2023-10-11T09:41:15.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.086+0000 7f6d9559a700 1 -- 172.21.15.78:0/40531037 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d800035d0 con 0x7f6d980cadc0 2023-10-11T09:41:15.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.086+0000 7f6d9559a700 1 -- 172.21.15.78:0/40531037 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 999359505 0 0) 0x7f6d800035d0 con 0x7f6d980cadc0 2023-10-11T09:41:15.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.086+0000 7f6d9559a700 1 -- 172.21.15.78:0/40531037 >> v2:172.21.15.133:3300/0 conn(0x7f6d9806d340 msgr2=0x7f6d980ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:15.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.086+0000 7f6d9559a700 1 --2- 172.21.15.78:0/40531037 >> v2:172.21.15.133:3300/0 conn(0x7f6d9806d340 0x7f6d980ca880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:15.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.086+0000 7f6d9559a700 1 -- 172.21.15.78:0/40531037 >> v1:172.21.15.133:6789/0 conn(0x7f6d98158430 legacy=0x7f6d980710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:15.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.086+0000 7f6d9559a700 1 -- 172.21.15.78:0/40531037 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d9815e840 con 0x7f6d980cadc0 2023-10-11T09:41:15.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.086+0000 7f6d9559a700 1 -- 172.21.15.78:0/40531037 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f6d900019f0 con 0x7f6d980cadc0 2023-10-11T09:41:15.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.086+0000 7f6d95d9b700 1 --2- 172.21.15.78:0/40531037 >> v2:172.21.15.133:3300/0 conn(0x7f6d9806d340 0x7f6d980ca880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:41:15.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.086+0000 7f6d9559a700 1 -- 172.21.15.78:0/40531037 >> v1:172.21.15.78:6789/0 conn(0x7f6d980cadc0 legacy=0x7f6d9815afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:15.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.086+0000 7f6d9559a700 1 --2- 172.21.15.78:0/40531037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d80003a20 0x7f6d80003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:15.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.086+0000 7f6d9559a700 1 --2- 172.21.15.78:0/40531037 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d80004410 0x7f6d800048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:15.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.086+0000 7f6d9559a700 1 --2- 172.21.15.78:0/40531037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d800050d0 0x7f6d8000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:15.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.086+0000 7f6d9559a700 1 -- 172.21.15.78:0/40531037 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8000c020 con 0x7f6d80003a20 2023-10-11T09:41:15.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.086+0000 7f6d9559a700 1 -- 172.21.15.78:0/40531037 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8000c160 con 0x7f6d800050d0 2023-10-11T09:41:15.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.086+0000 7f6d9559a700 1 -- 172.21.15.78:0/40531037 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8000c320 con 0x7f6d80004410 2023-10-11T09:41:15.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.086+0000 7f6d9659c700 1 --2- 172.21.15.78:0/40531037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d80003a20 0x7f6d80003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:15.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.086+0000 7f6d95d9b700 1 --2- 172.21.15.78:0/40531037 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d80004410 0x7f6d800048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:15.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.086+0000 7f6d96d9d700 1 --2- 172.21.15.78:0/40531037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d800050d0 0x7f6d8000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:15.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.086+0000 7f6d9659c700 1 -- 172.21.15.78:0/40531037 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d80004410 msgr2=0x7f6d800048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:15.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.086+0000 7f6d9659c700 1 --2- 172.21.15.78:0/40531037 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d80004410 0x7f6d800048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:15.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.086+0000 7f6d9659c700 1 -- 172.21.15.78:0/40531037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d800050d0 msgr2=0x7f6d8000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:15.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.086+0000 7f6d9659c700 1 --2- 172.21.15.78:0/40531037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d800050d0 0x7f6d8000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:15.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.086+0000 7f6d9659c700 1 -- 172.21.15.78:0/40531037 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d800035d0 con 0x7f6d80003a20 2023-10-11T09:41:15.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.086+0000 7f6d96d9d700 1 -- 172.21.15.78:0/40531037 reap_dead start 2023-10-11T09:41:15.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.086+0000 7f6d9659c700 1 --2- 172.21.15.78:0/40531037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d80003a20 0x7f6d80003df0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6d880018f0 tx=0x7f6d800058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e266261fb70cddf8 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:15.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.086+0000 7f6d9559a700 1 -- 172.21.15.78:0/40531037 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d88001390 con 0x7f6d80003a20 2023-10-11T09:41:15.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.086+0000 7f6d9559a700 1 -- 172.21.15.78:0/40531037 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d880052d0 con 0x7f6d80003a20 2023-10-11T09:41:15.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.086+0000 7f6d9559a700 1 -- 172.21.15.78:0/40531037 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d880036a0 con 0x7f6d80003a20 2023-10-11T09:41:15.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.086+0000 7f6d9ce82700 1 -- 172.21.15.78:0/40531037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d80003a20 msgr2=0x7f6d80003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:15.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.086+0000 7f6d9ce82700 1 --2- 172.21.15.78:0/40531037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d80003a20 0x7f6d80003df0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6d880018f0 tx=0x7f6d800058a0 comp rx=0 tx=0).stop 2023-10-11T09:41:15.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.086+0000 7f6d9ce82700 1 -- 172.21.15.78:0/40531037 shutdown_connections 2023-10-11T09:41:15.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.086+0000 7f6d9ce82700 1 --2- 172.21.15.78:0/40531037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d80003a20 0x7f6d80003df0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:15.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.086+0000 7f6d9ce82700 1 -- 172.21.15.78:0/40531037 >> 172.21.15.78:0/40531037 conn(0x7f6d980cecc0 msgr2=0x7f6d980cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:15.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.086+0000 7f6d9ce82700 1 -- 172.21.15.78:0/40531037 shutdown_connections 2023-10-11T09:41:15.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.086+0000 7f6d9ce82700 1 -- 172.21.15.78:0/40531037 wait complete. 2023-10-11T09:41:15.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.090+0000 7f6d9ce82700 1 Processor -- start 2023-10-11T09:41:15.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.090+0000 7f6d9ce82700 1 -- start start 2023-10-11T09:41:15.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.094+0000 7f6d9ce82700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d80003a20 0x7f6d9814c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:15.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.094+0000 7f6d9ce82700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d980b3100 0x7f6d980b3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:15.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.094+0000 7f6d9ce82700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d98154df0 0x7f6d98155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:15.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.094+0000 7f6d9ce82700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d98155780 con 0x7f6d980b3100 2023-10-11T09:41:15.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.094+0000 7f6d9ce82700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d981f0f10 con 0x7f6d80003a20 2023-10-11T09:41:15.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.094+0000 7f6d9ce82700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d981f0130 con 0x7f6d98154df0 2023-10-11T09:41:15.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.094+0000 7f6d95d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d980b3100 0x7f6d980b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:15.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.094+0000 7f6d95d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d980b3100 0x7f6d980b3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50648/0 (socket says 172.21.15.78:50648) 2023-10-11T09:41:15.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.094+0000 7f6d95d9b700 1 -- 172.21.15.78:0/1487796653 learned_addr learned my addr 172.21.15.78:0/1487796653 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:15.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.094+0000 7f6d9659c700 1 --2- 172.21.15.78:0/1487796653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d80003a20 0x7f6d9814c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:15.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.094+0000 7f6d96d9d700 1 --2- 172.21.15.78:0/1487796653 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d98154df0 0x7f6d98155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:15.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.094+0000 7f6d95d9b700 1 -- 172.21.15.78:0/1487796653 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d98154df0 msgr2=0x7f6d98155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:15.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.094+0000 7f6d95d9b700 1 --2- 172.21.15.78:0/1487796653 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d98154df0 0x7f6d98155240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:15.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.094+0000 7f6d95d9b700 1 -- 172.21.15.78:0/1487796653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d80003a20 msgr2=0x7f6d9814c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:15.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.094+0000 7f6d95d9b700 1 --2- 172.21.15.78:0/1487796653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d80003a20 0x7f6d9814c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:15.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.094+0000 7f6d95d9b700 1 -- 172.21.15.78:0/1487796653 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d981f0410 con 0x7f6d980b3100 2023-10-11T09:41:15.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.094+0000 7f6d96d9d700 1 --2- 172.21.15.78:0/1487796653 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d98154df0 0x7f6d98155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:15.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.094+0000 7f6d95d9b700 1 --2- 172.21.15.78:0/1487796653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d980b3100 0x7f6d980b3570 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c011c00 tx=0x7f6d8c011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:15.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.094+0000 7f6d86ffd700 1 -- 172.21.15.78:0/1487796653 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c00f9c0 con 0x7f6d980b3100 2023-10-11T09:41:15.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.094+0000 7f6d86ffd700 1 -- 172.21.15.78:0/1487796653 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8c00fb20 con 0x7f6d980b3100 2023-10-11T09:41:15.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.094+0000 7f6d86ffd700 1 -- 172.21.15.78:0/1487796653 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c00fed0 con 0x7f6d980b3100 2023-10-11T09:41:15.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.094+0000 7f6d9ce82700 1 -- 172.21.15.78:0/1487796653 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d981f06f0 con 0x7f6d980b3100 2023-10-11T09:41:15.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.094+0000 7f6d9ce82700 1 -- 172.21.15.78:0/1487796653 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d981f0cb0 con 0x7f6d980b3100 2023-10-11T09:41:15.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.098+0000 7f6d9ce82700 1 -- 172.21.15.78:0/1487796653 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d9806eb80 con 0x7f6d980b3100 2023-10-11T09:41:15.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.098+0000 7f6d86ffd700 1 -- 172.21.15.78:0/1487796653 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f6d8c014440 con 0x7f6d980b3100 2023-10-11T09:41:15.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.102+0000 7f6d86ffd700 1 --2- 172.21.15.78:0/1487796653 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6d78065ed0 0x7f6d78068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:15.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.102+0000 7f6d86ffd700 1 -- 172.21.15.78:0/1487796653 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6d8c083e20 con 0x7f6d980b3100 2023-10-11T09:41:15.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.102+0000 7f6d9659c700 1 --2- 172.21.15.78:0/1487796653 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6d78065ed0 0x7f6d78068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:15.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.102+0000 7f6d9659c700 1 --2- 172.21.15.78:0/1487796653 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6d78065ed0 0x7f6d78068380 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6d88000c00 tx=0x7f6d88005000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:15.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.106+0000 7f6d86ffd700 1 -- 172.21.15.78:0/1487796653 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f6d8c051480 con 0x7f6d980b3100 2023-10-11T09:41:15.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.278+0000 7f6d9ce82700 1 -- 172.21.15.78:0/1487796653 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6d98065db0 con 0x7f6d980b3100 2023-10-11T09:41:15.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.278+0000 7f6d86ffd700 1 -- 172.21.15.78:0/1487796653 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6d8c006ce0 con 0x7f6d980b3100 2023-10-11T09:41:15.284 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:41:15.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.282+0000 7f6d9ce82700 1 -- 172.21.15.78:0/1487796653 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6d78065ed0 msgr2=0x7f6d78068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:15.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.282+0000 7f6d9ce82700 1 --2- 172.21.15.78:0/1487796653 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6d78065ed0 0x7f6d78068380 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6d88000c00 tx=0x7f6d88005000 comp rx=0 tx=0).stop 2023-10-11T09:41:15.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.282+0000 7f6d9ce82700 1 -- 172.21.15.78:0/1487796653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d980b3100 msgr2=0x7f6d980b3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:15.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.282+0000 7f6d9ce82700 1 --2- 172.21.15.78:0/1487796653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d980b3100 0x7f6d980b3570 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c011c00 tx=0x7f6d8c011f10 comp rx=0 tx=0).stop 2023-10-11T09:41:15.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.282+0000 7f6d9ce82700 1 -- 172.21.15.78:0/1487796653 shutdown_connections 2023-10-11T09:41:15.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.282+0000 7f6d9ce82700 1 --2- 172.21.15.78:0/1487796653 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6d78065ed0 0x7f6d78068380 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:15.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.282+0000 7f6d9ce82700 1 --2- 172.21.15.78:0/1487796653 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d98154df0 0x7f6d98155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:15.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.282+0000 7f6d9ce82700 1 --2- 172.21.15.78:0/1487796653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d980b3100 0x7f6d980b3570 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:15.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.282+0000 7f6d9ce82700 1 --2- 172.21.15.78:0/1487796653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d80003a20 0x7f6d9814c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:15.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.282+0000 7f6d9ce82700 1 -- 172.21.15.78:0/1487796653 >> 172.21.15.78:0/1487796653 conn(0x7f6d980cecc0 msgr2=0x7f6d980ca060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:15.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.282+0000 7f6d9ce82700 1 -- 172.21.15.78:0/1487796653 shutdown_connections 2023-10-11T09:41:15.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.282+0000 7f6d9ce82700 1 -- 172.21.15.78:0/1487796653 wait complete. 2023-10-11T09:41:15.307 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T09:41:15.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.478+0000 7f97c5737700 1 Processor -- start 2023-10-11T09:41:15.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.478+0000 7f97c5737700 1 -- start start 2023-10-11T09:41:15.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.482+0000 7f97c5737700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f97c00b3610 0x7f97c00d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:15.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.482+0000 7f97c5737700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f97c00b3a90 0x7f97c00b85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:15.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.482+0000 7f97c5737700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97c00b47f0 con 0x7f97c00b3240 2023-10-11T09:41:15.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.482+0000 7f97c5737700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97c00b8130 con 0x7f97c00b3a90 2023-10-11T09:41:15.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.482+0000 7f97c5737700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c00b3f10 con 0x7f97c00b3610 2023-10-11T09:41:15.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.482+0000 7f97bffff700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f97c00b3240 0x7f97c015d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:44730/0 (socket says 172.21.15.78:44730) 2023-10-11T09:41:15.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.482+0000 7f97bffff700 1 -- 172.21.15.78:0/3541665004 learned_addr learned my addr 172.21.15.78:0/3541665004 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:15.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.482+0000 7f97bf7fe700 1 --2- 172.21.15.78:0/3541665004 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f97c00b3610 0x7f97c00d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:15.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.482+0000 7f97c4f36700 1 --2- 172.21.15.78:0/3541665004 >> v2:172.21.15.133:3300/0 conn(0x7f97c00b3a90 0x7f97c00b85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:15.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.482+0000 7f97beffd700 1 -- 172.21.15.78:0/3541665004 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2705417504 0 0) 0x7f97b0001110 con 0x7f97c00b3240 2023-10-11T09:41:15.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.482+0000 7f97beffd700 1 -- 172.21.15.78:0/3541665004 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97a80035d0 con 0x7f97c00b3240 2023-10-11T09:41:15.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.482+0000 7f97c4f36700 1 -- 172.21.15.78:0/3541665004 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f97c00b3610 msgr2=0x7f97c00d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:15.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.482+0000 7f97c4f36700 1 --2- 172.21.15.78:0/3541665004 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f97c00b3610 0x7f97c00d29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:15.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.482+0000 7f97c4f36700 1 -- 172.21.15.78:0/3541665004 >> v1:172.21.15.133:6789/0 conn(0x7f97c00b3240 legacy=0x7f97c015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:15.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.482+0000 7f97c4f36700 1 -- 172.21.15.78:0/3541665004 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c00b41e0 con 0x7f97c00b3a90 2023-10-11T09:41:15.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.482+0000 7f97c4f36700 1 --2- 172.21.15.78:0/3541665004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97c00b3a90 0x7f97c00b85c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f97b8009d90 tx=0x7f97b800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9251898404f2adc9 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:15.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.482+0000 7f97beffd700 1 -- 172.21.15.78:0/3541665004 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b800e070 con 0x7f97c00b3a90 2023-10-11T09:41:15.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.482+0000 7f97beffd700 1 -- 172.21.15.78:0/3541665004 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b800bb20 con 0x7f97c00b3a90 2023-10-11T09:41:15.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.482+0000 7f97beffd700 1 -- 172.21.15.78:0/3541665004 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b8012430 con 0x7f97c00b3a90 2023-10-11T09:41:15.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.486+0000 7f97c5737700 1 -- 172.21.15.78:0/3541665004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97c00b3a90 msgr2=0x7f97c00b85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:15.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.486+0000 7f97c5737700 1 --2- 172.21.15.78:0/3541665004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97c00b3a90 0x7f97c00b85c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f97b8009d90 tx=0x7f97b800b330 comp rx=0 tx=0).stop 2023-10-11T09:41:15.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.486+0000 7f97c5737700 1 -- 172.21.15.78:0/3541665004 shutdown_connections 2023-10-11T09:41:15.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.486+0000 7f97c5737700 1 --2- 172.21.15.78:0/3541665004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97c00b3a90 0x7f97c00b85c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:15.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.486+0000 7f97c5737700 1 --2- 172.21.15.78:0/3541665004 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f97c00b3610 0x7f97c00d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:15.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.486+0000 7f97c5737700 1 -- 172.21.15.78:0/3541665004 >> 172.21.15.78:0/3541665004 conn(0x7f97c009c260 msgr2=0x7f97c00d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:15.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.486+0000 7f97c5737700 1 -- 172.21.15.78:0/3541665004 shutdown_connections 2023-10-11T09:41:15.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.486+0000 7f97c5737700 1 -- 172.21.15.78:0/3541665004 wait complete. 2023-10-11T09:41:15.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.486+0000 7f97c5737700 1 Processor -- start 2023-10-11T09:41:15.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.486+0000 7f97c5737700 1 -- start start 2023-10-11T09:41:15.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.486+0000 7f97c5737700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f97c00b3240 0x7f97c014f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:15.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.486+0000 7f97c5737700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97c00b3610 0x7f97c015c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:15.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.486+0000 7f97c5737700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97c00b3a90 0x7f97c015c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:15.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.486+0000 7f97c5737700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c00c6ac0 con 0x7f97c00b3a90 2023-10-11T09:41:15.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.486+0000 7f97c5737700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c00c6d10 con 0x7f97c00b3610 2023-10-11T09:41:15.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.486+0000 7f97c5737700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c00c6ff0 con 0x7f97c00b3240 2023-10-11T09:41:15.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.490+0000 7f97c4f36700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97c00b3a90 0x7f97c015c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:15.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.490+0000 7f97c4f36700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97c00b3a90 0x7f97c015c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50658/0 (socket says 172.21.15.78:50658) 2023-10-11T09:41:15.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.490+0000 7f97c4f36700 1 -- 172.21.15.78:0/927438821 learned_addr learned my addr 172.21.15.78:0/927438821 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:15.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.490+0000 7f97bf7fe700 1 --2- 172.21.15.78:0/927438821 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97c00b3610 0x7f97c015c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:15.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.490+0000 7f97c4f36700 1 -- 172.21.15.78:0/927438821 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f97c00b3240 msgr2=0x7f97c014f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:15.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.490+0000 7f97c4f36700 1 --2- 172.21.15.78:0/927438821 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f97c00b3240 0x7f97c014f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:15.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.490+0000 7f97c4f36700 1 -- 172.21.15.78:0/927438821 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97c00b3610 msgr2=0x7f97c015c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:15.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.490+0000 7f97c4f36700 1 --2- 172.21.15.78:0/927438821 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97c00b3610 0x7f97c015c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:15.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.490+0000 7f97c4f36700 1 -- 172.21.15.78:0/927438821 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b8009a70 con 0x7f97c00b3a90 2023-10-11T09:41:15.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.490+0000 7f97bf7fe700 1 --2- 172.21.15.78:0/927438821 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97c00b3610 0x7f97c015c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:41:15.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.490+0000 7f97c4f36700 1 --2- 172.21.15.78:0/927438821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97c00b3a90 0x7f97c015c660 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f97b8014040 tx=0x7f97b8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:15.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.490+0000 7f97bcff9700 1 -- 172.21.15.78:0/927438821 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b8012430 con 0x7f97c00b3a90 2023-10-11T09:41:15.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.490+0000 7f97bcff9700 1 -- 172.21.15.78:0/927438821 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b8020070 con 0x7f97c00b3a90 2023-10-11T09:41:15.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.490+0000 7f97c5737700 1 -- 172.21.15.78:0/927438821 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97c00c72d0 con 0x7f97c00b3a90 2023-10-11T09:41:15.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.490+0000 7f97bcff9700 1 -- 172.21.15.78:0/927438821 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b8004280 con 0x7f97c00b3a90 2023-10-11T09:41:15.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.490+0000 7f97c5737700 1 -- 172.21.15.78:0/927438821 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c00c7890 con 0x7f97c00b3a90 2023-10-11T09:41:15.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.490+0000 7f97c5737700 1 -- 172.21.15.78:0/927438821 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97c0065d40 con 0x7f97c00b3a90 2023-10-11T09:41:15.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.494+0000 7f97bcff9700 1 -- 172.21.15.78:0/927438821 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f97b80043e0 con 0x7f97c00b3a90 2023-10-11T09:41:15.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.494+0000 7f97bcff9700 1 --2- 172.21.15.78:0/927438821 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f97ac065ed0 0x7f97ac068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:15.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.494+0000 7f97bcff9700 1 -- 172.21.15.78:0/927438821 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f97b8022070 con 0x7f97c00b3a90 2023-10-11T09:41:15.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.494+0000 7f97bffff700 1 --2- 172.21.15.78:0/927438821 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f97ac065ed0 0x7f97ac068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:15.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.494+0000 7f97bffff700 1 --2- 172.21.15.78:0/927438821 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f97ac065ed0 0x7f97ac068380 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f97b0005930 tx=0x7f97b0001150 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:15.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.498+0000 7f97bcff9700 1 -- 172.21.15.78:0/927438821 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f97b8053170 con 0x7f97c00b3a90 2023-10-11T09:41:15.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.666+0000 7f97c5737700 1 -- 172.21.15.78:0/927438821 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f97c006b280 con 0x7f97c00b3a90 2023-10-11T09:41:15.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.666+0000 7f97bcff9700 1 -- 172.21.15.78:0/927438821 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f97b802c0a0 con 0x7f97c00b3a90 2023-10-11T09:41:15.672 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:41:15.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.670+0000 7f97c5737700 1 -- 172.21.15.78:0/927438821 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f97ac065ed0 msgr2=0x7f97ac068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:15.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.670+0000 7f97c5737700 1 --2- 172.21.15.78:0/927438821 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f97ac065ed0 0x7f97ac068380 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f97b0005930 tx=0x7f97b0001150 comp rx=0 tx=0).stop 2023-10-11T09:41:15.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.670+0000 7f97c5737700 1 -- 172.21.15.78:0/927438821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97c00b3a90 msgr2=0x7f97c015c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:15.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.670+0000 7f97c5737700 1 --2- 172.21.15.78:0/927438821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97c00b3a90 0x7f97c015c660 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f97b8014040 tx=0x7f97b8004af0 comp rx=0 tx=0).stop 2023-10-11T09:41:15.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.670+0000 7f97c5737700 1 -- 172.21.15.78:0/927438821 shutdown_connections 2023-10-11T09:41:15.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.670+0000 7f97c5737700 1 --2- 172.21.15.78:0/927438821 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f97ac065ed0 0x7f97ac068380 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:15.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.670+0000 7f97c5737700 1 --2- 172.21.15.78:0/927438821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97c00b3a90 0x7f97c015c660 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:15.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.670+0000 7f97c5737700 1 --2- 172.21.15.78:0/927438821 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97c00b3610 0x7f97c015c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:15.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.670+0000 7f97c5737700 1 --2- 172.21.15.78:0/927438821 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f97c00b3240 0x7f97c014f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:15.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.670+0000 7f97c5737700 1 -- 172.21.15.78:0/927438821 >> 172.21.15.78:0/927438821 conn(0x7f97c009c260 msgr2=0x7f97c00afb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:15.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.674+0000 7f97c5737700 1 -- 172.21.15.78:0/927438821 shutdown_connections 2023-10-11T09:41:15.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.674+0000 7f97c5737700 1 -- 172.21.15.78:0/927438821 wait complete. 2023-10-11T09:41:15.698 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T09:41:15.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.870+0000 7fbc63f69700 1 Processor -- start 2023-10-11T09:41:15.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.870+0000 7fbc63f69700 1 -- start start 2023-10-11T09:41:15.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.874+0000 7fbc63f69700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbc5c0b84c0 0x7fbc5c0cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:15.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.874+0000 7fbc63f69700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fbc5c0b8890 0x7fbc5c160370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:15.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.874+0000 7fbc63f69700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc5c0cd690 con 0x7fbc5c1608b0 2023-10-11T09:41:15.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.874+0000 7fbc63f69700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc5c0cdea0 con 0x7fbc5c0b8890 2023-10-11T09:41:15.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.874+0000 7fbc63f69700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c0d0c10 con 0x7fbc5c0b84c0 2023-10-11T09:41:15.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.874+0000 7fbc61504700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fbc5c0b8890 0x7fbc5c160370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:15.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.874+0000 7fbc62506700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fbc5c1608b0 0x7fbc5c0d2fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:55792/0 (socket says 172.21.15.78:55792) 2023-10-11T09:41:15.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.874+0000 7fbc62506700 1 -- 172.21.15.78:0/383944179 learned_addr learned my addr 172.21.15.78:0/383944179 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:15.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.874+0000 7fbc61d05700 1 --2- 172.21.15.78:0/383944179 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbc5c0b84c0 0x7fbc5c0cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:15.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.874+0000 7fbc61504700 1 -- 172.21.15.78:0/383944179 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbc5c0b84c0 msgr2=0x7fbc5c0cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:15.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.874+0000 7fbc61504700 1 --2- 172.21.15.78:0/383944179 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbc5c0b84c0 0x7fbc5c0cfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:15.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.874+0000 7fbc61504700 1 -- 172.21.15.78:0/383944179 >> v1:172.21.15.78:6789/0 conn(0x7fbc5c1608b0 legacy=0x7fbc5c0d2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:15.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.874+0000 7fbc61504700 1 -- 172.21.15.78:0/383944179 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc5c0d02e0 con 0x7fbc5c0b8890 2023-10-11T09:41:15.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.874+0000 7fbc61504700 1 --2- 172.21.15.78:0/383944179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc5c0b8890 0x7fbc5c160370 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c009850 tx=0x7fbc4c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22f9c89bee04fcc server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:15.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.874+0000 7fbc60d03700 1 -- 172.21.15.78:0/383944179 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c011830 con 0x7fbc5c0b8890 2023-10-11T09:41:15.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.874+0000 7fbc60d03700 1 -- 172.21.15.78:0/383944179 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4c011990 con 0x7fbc5c0b8890 2023-10-11T09:41:15.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.874+0000 7fbc60d03700 1 -- 172.21.15.78:0/383944179 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c011d20 con 0x7fbc5c0b8890 2023-10-11T09:41:15.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.878+0000 7fbc63f69700 1 -- 172.21.15.78:0/383944179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc5c0b8890 msgr2=0x7fbc5c160370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:15.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.878+0000 7fbc63f69700 1 --2- 172.21.15.78:0/383944179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc5c0b8890 0x7fbc5c160370 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c009850 tx=0x7fbc4c009b60 comp rx=0 tx=0).stop 2023-10-11T09:41:15.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.878+0000 7fbc63f69700 1 -- 172.21.15.78:0/383944179 shutdown_connections 2023-10-11T09:41:15.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.878+0000 7fbc63f69700 1 --2- 172.21.15.78:0/383944179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc5c0b8890 0x7fbc5c160370 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:15.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.878+0000 7fbc63f69700 1 --2- 172.21.15.78:0/383944179 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbc5c0b84c0 0x7fbc5c0cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:15.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.878+0000 7fbc63f69700 1 -- 172.21.15.78:0/383944179 >> 172.21.15.78:0/383944179 conn(0x7fbc5c06cb50 msgr2=0x7fbc5c06cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:15.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.878+0000 7fbc63f69700 1 -- 172.21.15.78:0/383944179 shutdown_connections 2023-10-11T09:41:15.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.878+0000 7fbc63f69700 1 -- 172.21.15.78:0/383944179 wait complete. 2023-10-11T09:41:15.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.878+0000 7fbc63f69700 1 Processor -- start 2023-10-11T09:41:15.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.878+0000 7fbc63f69700 1 -- start start 2023-10-11T09:41:15.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.882+0000 7fbc63f69700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc5c0b84c0 0x7fbc5c15b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:15.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.882+0000 7fbc63f69700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc5c0b8890 0x7fbc5c15ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:15.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.882+0000 7fbc63f69700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbc5c1608b0 0x7fbc5c06b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:15.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.882+0000 7fbc63f69700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c15e2f0 con 0x7fbc5c0b8890 2023-10-11T09:41:15.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.882+0000 7fbc63f69700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c15ba30 con 0x7fbc5c0b84c0 2023-10-11T09:41:15.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.882+0000 7fbc63f69700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c15d770 con 0x7fbc5c1608b0 2023-10-11T09:41:15.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.882+0000 7fbc61504700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc5c0b8890 0x7fbc5c15ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:15.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.882+0000 7fbc61504700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc5c0b8890 0x7fbc5c15ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50670/0 (socket says 172.21.15.78:50670) 2023-10-11T09:41:15.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.882+0000 7fbc61504700 1 -- 172.21.15.78:0/3316940071 learned_addr learned my addr 172.21.15.78:0/3316940071 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:15.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.882+0000 7fbc61d05700 1 --2- 172.21.15.78:0/3316940071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc5c0b84c0 0x7fbc5c15b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:15.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.882+0000 7fbc62506700 1 --2- 172.21.15.78:0/3316940071 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbc5c1608b0 0x7fbc5c06b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:15.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.882+0000 7fbc61504700 1 -- 172.21.15.78:0/3316940071 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbc5c1608b0 msgr2=0x7fbc5c06b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:15.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.882+0000 7fbc61504700 1 --2- 172.21.15.78:0/3316940071 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbc5c1608b0 0x7fbc5c06b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:15.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.882+0000 7fbc61504700 1 -- 172.21.15.78:0/3316940071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc5c0b84c0 msgr2=0x7fbc5c15b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:15.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.882+0000 7fbc61504700 1 --2- 172.21.15.78:0/3316940071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc5c0b84c0 0x7fbc5c15b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:15.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.882+0000 7fbc61504700 1 -- 172.21.15.78:0/3316940071 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc4c0094b0 con 0x7fbc5c0b8890 2023-10-11T09:41:15.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.882+0000 7fbc61504700 1 --2- 172.21.15.78:0/3316940071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc5c0b8890 0x7fbc5c15ce70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c000c00 tx=0x7fbc4c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:15.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.882+0000 7fbc527fc700 1 -- 172.21.15.78:0/3316940071 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c004570 con 0x7fbc5c0b8890 2023-10-11T09:41:15.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.882+0000 7fbc527fc700 1 -- 172.21.15.78:0/3316940071 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4c01f070 con 0x7fbc5c0b8890 2023-10-11T09:41:15.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.882+0000 7fbc63f69700 1 -- 172.21.15.78:0/3316940071 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc5c15da50 con 0x7fbc5c0b8890 2023-10-11T09:41:15.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.882+0000 7fbc527fc700 1 -- 172.21.15.78:0/3316940071 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c003900 con 0x7fbc5c0b8890 2023-10-11T09:41:15.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.882+0000 7fbc63f69700 1 -- 172.21.15.78:0/3316940071 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc5c15e010 con 0x7fbc5c0b8890 2023-10-11T09:41:15.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.886+0000 7fbc63f69700 1 -- 172.21.15.78:0/3316940071 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc5c08bff0 con 0x7fbc5c0b8890 2023-10-11T09:41:15.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.886+0000 7fbc527fc700 1 -- 172.21.15.78:0/3316940071 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fbc4c021030 con 0x7fbc5c0b8890 2023-10-11T09:41:15.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.886+0000 7fbc527fc700 1 --2- 172.21.15.78:0/3316940071 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbc40065ed0 0x7fbc40068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:15.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.886+0000 7fbc527fc700 1 -- 172.21.15.78:0/3316940071 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbc4c084cf0 con 0x7fbc5c0b8890 2023-10-11T09:41:15.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.886+0000 7fbc61d05700 1 --2- 172.21.15.78:0/3316940071 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbc40065ed0 0x7fbc40068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:15.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.890+0000 7fbc61d05700 1 --2- 172.21.15.78:0/3316940071 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbc40065ed0 0x7fbc40068380 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbc5800b9d0 tx=0x7fbc58008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:15.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:15.890+0000 7fbc527fc700 1 -- 172.21.15.78:0/3316940071 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fbc4c052330 con 0x7fbc5c0b8890 2023-10-11T09:41:16.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.062+0000 7fbc63f69700 1 -- 172.21.15.78:0/3316940071 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbc5c0ceb50 con 0x7fbc5c0b8890 2023-10-11T09:41:16.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.062+0000 7fbc527fc700 1 -- 172.21.15.78:0/3316940071 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbc4c055950 con 0x7fbc5c0b8890 2023-10-11T09:41:16.067 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2023-10-11T09:41:16.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.066+0000 7fbc63f69700 1 -- 172.21.15.78:0/3316940071 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbc40065ed0 msgr2=0x7fbc40068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:16.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.066+0000 7fbc63f69700 1 --2- 172.21.15.78:0/3316940071 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbc40065ed0 0x7fbc40068380 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbc5800b9d0 tx=0x7fbc58008040 comp rx=0 tx=0).stop 2023-10-11T09:41:16.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.066+0000 7fbc63f69700 1 -- 172.21.15.78:0/3316940071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc5c0b8890 msgr2=0x7fbc5c15ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:16.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.066+0000 7fbc63f69700 1 --2- 172.21.15.78:0/3316940071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc5c0b8890 0x7fbc5c15ce70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c000c00 tx=0x7fbc4c004770 comp rx=0 tx=0).stop 2023-10-11T09:41:16.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.066+0000 7fbc63f69700 1 -- 172.21.15.78:0/3316940071 shutdown_connections 2023-10-11T09:41:16.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.066+0000 7fbc63f69700 1 --2- 172.21.15.78:0/3316940071 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbc40065ed0 0x7fbc40068380 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:16.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.066+0000 7fbc63f69700 1 --2- 172.21.15.78:0/3316940071 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbc5c1608b0 0x7fbc5c06b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:16.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.066+0000 7fbc63f69700 1 --2- 172.21.15.78:0/3316940071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc5c0b8890 0x7fbc5c15ce70 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:16.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.066+0000 7fbc63f69700 1 --2- 172.21.15.78:0/3316940071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc5c0b84c0 0x7fbc5c15b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:16.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.066+0000 7fbc63f69700 1 -- 172.21.15.78:0/3316940071 >> 172.21.15.78:0/3316940071 conn(0x7fbc5c06cb50 msgr2=0x7fbc5c15c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:16.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.066+0000 7fbc63f69700 1 -- 172.21.15.78:0/3316940071 shutdown_connections 2023-10-11T09:41:16.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.066+0000 7fbc63f69700 1 -- 172.21.15.78:0/3316940071 wait complete. 2023-10-11T09:41:16.089 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:41:16.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.250+0000 7fe7ab59e700 1 Processor -- start 2023-10-11T09:41:16.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.250+0000 7fe7ab59e700 1 -- start start 2023-10-11T09:41:16.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.250+0000 7fe7ab59e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe7ac0d1660 0x7fe7ac0d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:16.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.250+0000 7fe7ab59e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe7ac0d28e0 0x7fe7ac0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:16.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.250+0000 7fe7ab59e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7ac15dc10 con 0x7fe7ac156ce0 2023-10-11T09:41:16.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.250+0000 7fe7ab59e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7ac160380 con 0x7fe7ac0d28e0 2023-10-11T09:41:16.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.250+0000 7fe7ab59e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac15d930 con 0x7fe7ac0d1660 2023-10-11T09:41:16.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.254+0000 7fe7aad9d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe7ac0d28e0 0x7fe7ac0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:16.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.254+0000 7fe7aa59c700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe7ac156ce0 0x7fe7ac160b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:44736/0 (socket says 172.21.15.78:44736) 2023-10-11T09:41:16.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.254+0000 7fe7aa59c700 1 -- 172.21.15.78:0/2139570316 learned_addr learned my addr 172.21.15.78:0/2139570316 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:16.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.254+0000 7fe7a1d9b700 1 --2- 172.21.15.78:0/2139570316 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe7ac0d1660 0x7fe7ac0d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:16.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.254+0000 7fe7a9d9b700 1 -- 172.21.15.78:0/2139570316 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1948482916 0 0) 0x7fe7ac15dc10 con 0x7fe7ac156ce0 2023-10-11T09:41:16.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.254+0000 7fe7a9d9b700 1 -- 172.21.15.78:0/2139570316 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7980035d0 con 0x7fe7ac156ce0 2023-10-11T09:41:16.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.254+0000 7fe7aad9d700 1 -- 172.21.15.78:0/2139570316 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe7ac0d1660 msgr2=0x7fe7ac0d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:16.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.254+0000 7fe7aad9d700 1 --2- 172.21.15.78:0/2139570316 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe7ac0d1660 0x7fe7ac0d0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:16.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.254+0000 7fe7aad9d700 1 -- 172.21.15.78:0/2139570316 >> v1:172.21.15.133:6789/0 conn(0x7fe7ac156ce0 legacy=0x7fe7ac160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:16.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.254+0000 7fe7aad9d700 1 -- 172.21.15.78:0/2139570316 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ac160620 con 0x7fe7ac0d28e0 2023-10-11T09:41:16.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.254+0000 7fe7aad9d700 1 --2- 172.21.15.78:0/2139570316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7ac0d28e0 0x7fe7ac0d0f90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4009850 tx=0x7fe7a4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f417886e99b43eb server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:16.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.254+0000 7fe7a9d9b700 1 -- 172.21.15.78:0/2139570316 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a4011830 con 0x7fe7ac0d28e0 2023-10-11T09:41:16.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.254+0000 7fe7a9d9b700 1 -- 172.21.15.78:0/2139570316 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a4011990 con 0x7fe7ac0d28e0 2023-10-11T09:41:16.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.254+0000 7fe7a9d9b700 1 -- 172.21.15.78:0/2139570316 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a4011d00 con 0x7fe7ac0d28e0 2023-10-11T09:41:16.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.254+0000 7fe7ab59e700 1 -- 172.21.15.78:0/2139570316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7ac0d28e0 msgr2=0x7fe7ac0d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:16.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.254+0000 7fe7ab59e700 1 --2- 172.21.15.78:0/2139570316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7ac0d28e0 0x7fe7ac0d0f90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4009850 tx=0x7fe7a4009b60 comp rx=0 tx=0).stop 2023-10-11T09:41:16.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.258+0000 7fe7ab59e700 1 -- 172.21.15.78:0/2139570316 shutdown_connections 2023-10-11T09:41:16.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.258+0000 7fe7ab59e700 1 --2- 172.21.15.78:0/2139570316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7ac0d28e0 0x7fe7ac0d0f90 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:16.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.258+0000 7fe7ab59e700 1 --2- 172.21.15.78:0/2139570316 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe7ac0d1660 0x7fe7ac0d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:16.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.258+0000 7fe7ab59e700 1 -- 172.21.15.78:0/2139570316 >> 172.21.15.78:0/2139570316 conn(0x7fe7ac06afb0 msgr2=0x7fe7ac06b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:16.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.258+0000 7fe7ab59e700 1 -- 172.21.15.78:0/2139570316 shutdown_connections 2023-10-11T09:41:16.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.258+0000 7fe7ab59e700 1 -- 172.21.15.78:0/2139570316 wait complete. 2023-10-11T09:41:16.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.258+0000 7fe7ab59e700 1 Processor -- start 2023-10-11T09:41:16.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.258+0000 7fe7ab59e700 1 -- start start 2023-10-11T09:41:16.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.262+0000 7fe7ab59e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7ac0d1660 0x7fe7ac0ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:16.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.262+0000 7fe7ab59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7ac0d28e0 0x7fe7ac144bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:16.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.262+0000 7fe7ab59e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe7ac156ce0 0x7fe7ac145130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:16.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.262+0000 7fe7ab59e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac0ca5a0 con 0x7fe7ac0d28e0 2023-10-11T09:41:16.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.262+0000 7fe7ab59e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac148eb0 con 0x7fe7ac0d1660 2023-10-11T09:41:16.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.262+0000 7fe7ab59e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac145760 con 0x7fe7ac156ce0 2023-10-11T09:41:16.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.262+0000 7fe7a1d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7ac0d28e0 0x7fe7ac144bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:16.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.262+0000 7fe7a1d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7ac0d28e0 0x7fe7ac144bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50690/0 (socket says 172.21.15.78:50690) 2023-10-11T09:41:16.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.262+0000 7fe7a1d9b700 1 -- 172.21.15.78:0/1387337193 learned_addr learned my addr 172.21.15.78:0/1387337193 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:16.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.262+0000 7fe7aad9d700 1 --2- 172.21.15.78:0/1387337193 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe7ac156ce0 0x7fe7ac145130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:16.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.262+0000 7fe7aa59c700 1 --2- 172.21.15.78:0/1387337193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7ac0d1660 0x7fe7ac0ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:16.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.262+0000 7fe7a1d9b700 1 -- 172.21.15.78:0/1387337193 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe7ac156ce0 msgr2=0x7fe7ac145130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:16.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.262+0000 7fe7a1d9b700 1 --2- 172.21.15.78:0/1387337193 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe7ac156ce0 0x7fe7ac145130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:16.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.262+0000 7fe7a1d9b700 1 -- 172.21.15.78:0/1387337193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7ac0d1660 msgr2=0x7fe7ac0ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:16.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.262+0000 7fe7a1d9b700 1 --2- 172.21.15.78:0/1387337193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7ac0d1660 0x7fe7ac0ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:16.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.262+0000 7fe7a1d9b700 1 -- 172.21.15.78:0/1387337193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a40094b0 con 0x7fe7ac0d28e0 2023-10-11T09:41:16.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.262+0000 7fe7aa59c700 1 --2- 172.21.15.78:0/1387337193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7ac0d1660 0x7fe7ac0ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:16.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.262+0000 7fe7a1d9b700 1 --2- 172.21.15.78:0/1387337193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7ac0d28e0 0x7fe7ac144bf0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe79400e9a0 tx=0x7fe79400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:16.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.262+0000 7fe7a37fe700 1 -- 172.21.15.78:0/1387337193 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe794009720 con 0x7fe7ac0d28e0 2023-10-11T09:41:16.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.262+0000 7fe7a37fe700 1 -- 172.21.15.78:0/1387337193 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe79400c850 con 0x7fe7ac0d28e0 2023-10-11T09:41:16.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.262+0000 7fe7ab59e700 1 -- 172.21.15.78:0/1387337193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7ac145a40 con 0x7fe7ac0d28e0 2023-10-11T09:41:16.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.262+0000 7fe7a37fe700 1 -- 172.21.15.78:0/1387337193 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79400cc00 con 0x7fe7ac0d28e0 2023-10-11T09:41:16.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.262+0000 7fe7ab59e700 1 -- 172.21.15.78:0/1387337193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7ac0ca230 con 0x7fe7ac0d28e0 2023-10-11T09:41:16.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.266+0000 7fe7ab59e700 1 -- 172.21.15.78:0/1387337193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7ac0ca710 con 0x7fe7ac0d28e0 2023-10-11T09:41:16.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.266+0000 7fe7a37fe700 1 -- 172.21.15.78:0/1387337193 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fe794013470 con 0x7fe7ac0d28e0 2023-10-11T09:41:16.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.270+0000 7fe7a37fe700 1 --2- 172.21.15.78:0/1387337193 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe78c065ed0 0x7fe78c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:16.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.270+0000 7fe7a37fe700 1 -- 172.21.15.78:0/1387337193 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe794083cb0 con 0x7fe7ac0d28e0 2023-10-11T09:41:16.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.270+0000 7fe7aa59c700 1 --2- 172.21.15.78:0/1387337193 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe78c065ed0 0x7fe78c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:16.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.270+0000 7fe7aa59c700 1 --2- 172.21.15.78:0/1387337193 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe78c065ed0 0x7fe78c068380 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe79c005cb0 tx=0x7fe79c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:16.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.274+0000 7fe7a37fe700 1 -- 172.21.15.78:0/1387337193 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fe794051260 con 0x7fe7ac0d28e0 2023-10-11T09:41:16.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.458+0000 7fe7ab59e700 1 -- 172.21.15.78:0/1387337193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7ac065d40 con 0x7fe7ac0d28e0 2023-10-11T09:41:16.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.462+0000 7fe7a37fe700 1 -- 172.21.15.78:0/1387337193 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17811 (secure 0 0 0) 0x7fe794017090 con 0x7fe7ac0d28e0 2023-10-11T09:41:16.466 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:41:16.467 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":23,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:40:42.678477+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:40:39.862581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T09:40:40.663641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:41:16.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.462+0000 7fe7ab59e700 1 -- 172.21.15.78:0/1387337193 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe78c065ed0 msgr2=0x7fe78c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:16.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.462+0000 7fe7ab59e700 1 --2- 172.21.15.78:0/1387337193 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe78c065ed0 0x7fe78c068380 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe79c005cb0 tx=0x7fe79c009040 comp rx=0 tx=0).stop 2023-10-11T09:41:16.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.466+0000 7fe7ab59e700 1 -- 172.21.15.78:0/1387337193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7ac0d28e0 msgr2=0x7fe7ac144bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:16.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.466+0000 7fe7ab59e700 1 --2- 172.21.15.78:0/1387337193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7ac0d28e0 0x7fe7ac144bf0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe79400e9a0 tx=0x7fe79400ecb0 comp rx=0 tx=0).stop 2023-10-11T09:41:16.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.466+0000 7fe7ab59e700 1 -- 172.21.15.78:0/1387337193 shutdown_connections 2023-10-11T09:41:16.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.466+0000 7fe7ab59e700 1 --2- 172.21.15.78:0/1387337193 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe78c065ed0 0x7fe78c068380 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:16.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.466+0000 7fe7ab59e700 1 --2- 172.21.15.78:0/1387337193 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe7ac156ce0 0x7fe7ac145130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:16.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.466+0000 7fe7ab59e700 1 --2- 172.21.15.78:0/1387337193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7ac0d28e0 0x7fe7ac144bf0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:16.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.466+0000 7fe7ab59e700 1 --2- 172.21.15.78:0/1387337193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7ac0d1660 0x7fe7ac0ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:16.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.466+0000 7fe7ab59e700 1 -- 172.21.15.78:0/1387337193 >> 172.21.15.78:0/1387337193 conn(0x7fe7ac06afb0 msgr2=0x7fe7ac15e970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:16.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.466+0000 7fe7ab59e700 1 -- 172.21.15.78:0/1387337193 shutdown_connections 2023-10-11T09:41:16.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.466+0000 7fe7ab59e700 1 -- 172.21.15.78:0/1387337193 wait complete. 2023-10-11T09:41:16.490 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T09:40:39.862581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T09:40:40.663641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:41:16.491 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:41:16.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.666+0000 7f1e051a0700 1 Processor -- start 2023-10-11T09:41:16.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.666+0000 7f1e051a0700 1 -- start start 2023-10-11T09:41:16.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.670+0000 7f1e051a0700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1e00158430 0x7f1e000710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:16.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.670+0000 7f1e051a0700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f1e0006d340 0x7f1e000ca880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:16.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.670+0000 7f1e051a0700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e000cadc0 0x7f1e0015afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:16.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.670+0000 7f1e051a0700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e00070c10 con 0x7f1e0006d340 2023-10-11T09:41:16.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.670+0000 7f1e051a0700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e000cb240 con 0x7f1e00158430 2023-10-11T09:41:16.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.670+0000 7f1e051a0700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e00071610 con 0x7f1e000cadc0 2023-10-11T09:41:16.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.670+0000 7f1dfe59c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f1e0006d340 0x7f1e000ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:16.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.670+0000 7f1dfe59c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f1e0006d340 0x7f1e000ca880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50698/0 (socket says 172.21.15.78:50698) 2023-10-11T09:41:16.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.670+0000 7f1dfe59c700 1 -- 172.21.15.78:0/3527697619 learned_addr learned my addr 172.21.15.78:0/3527697619 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:16.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.670+0000 7f1dfed9d700 1 --2- 172.21.15.78:0/3527697619 >> v2:172.21.15.133:3300/0 conn(0x7f1e00158430 0x7f1e000710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:16.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.670+0000 7f1dff59e700 1 --2- 172.21.15.78:0/3527697619 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e000cadc0 0x7f1e0015afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:16.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.670+0000 7f1dfe59c700 1 -- 172.21.15.78:0/3527697619 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e000cadc0 msgr2=0x7f1e0015afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:16.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.670+0000 7f1dfe59c700 1 --2- 172.21.15.78:0/3527697619 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e000cadc0 0x7f1e0015afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:16.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.670+0000 7f1dfe59c700 1 -- 172.21.15.78:0/3527697619 >> v2:172.21.15.133:3300/0 conn(0x7f1e00158430 msgr2=0x7f1e000710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:16.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.670+0000 7f1dfe59c700 1 --2- 172.21.15.78:0/3527697619 >> v2:172.21.15.133:3300/0 conn(0x7f1e00158430 0x7f1e000710d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:16.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.670+0000 7f1dfe59c700 1 -- 172.21.15.78:0/3527697619 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0015e280 con 0x7f1e0006d340 2023-10-11T09:41:16.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.670+0000 7f1dfe59c700 1 --2- 172.21.15.78:0/3527697619 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e0006d340 0x7f1e000ca880 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1e0006fd00 tx=0x7f1df4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6306d7dda0726c59 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:16.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.670+0000 7f1dfdd9b700 1 -- 172.21.15.78:0/3527697619 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df400b8a0 con 0x7f1e0006d340 2023-10-11T09:41:16.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.670+0000 7f1dfdd9b700 1 -- 172.21.15.78:0/3527697619 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df400ba00 con 0x7f1e0006d340 2023-10-11T09:41:16.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.670+0000 7f1dfdd9b700 1 -- 172.21.15.78:0/3527697619 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df400bd90 con 0x7f1e0006d340 2023-10-11T09:41:16.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.670+0000 7f1e051a0700 1 -- 172.21.15.78:0/3527697619 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e0006d340 msgr2=0x7f1e000ca880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:16.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.670+0000 7f1e051a0700 1 --2- 172.21.15.78:0/3527697619 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e0006d340 0x7f1e000ca880 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1e0006fd00 tx=0x7f1df4009bf0 comp rx=0 tx=0).stop 2023-10-11T09:41:16.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.670+0000 7f1e051a0700 1 -- 172.21.15.78:0/3527697619 shutdown_connections 2023-10-11T09:41:16.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.670+0000 7f1e051a0700 1 --2- 172.21.15.78:0/3527697619 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e000cadc0 0x7f1e0015afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:16.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.670+0000 7f1e051a0700 1 --2- 172.21.15.78:0/3527697619 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e0006d340 0x7f1e000ca880 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:16.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.670+0000 7f1e051a0700 1 --2- 172.21.15.78:0/3527697619 >> v2:172.21.15.133:3300/0 conn(0x7f1e00158430 0x7f1e000710d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:16.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.670+0000 7f1e051a0700 1 -- 172.21.15.78:0/3527697619 >> 172.21.15.78:0/3527697619 conn(0x7f1e000cecc0 msgr2=0x7f1e000cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:16.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.670+0000 7f1e051a0700 1 -- 172.21.15.78:0/3527697619 shutdown_connections 2023-10-11T09:41:16.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.670+0000 7f1e051a0700 1 -- 172.21.15.78:0/3527697619 wait complete. 2023-10-11T09:41:16.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.674+0000 7f1e051a0700 1 Processor -- start 2023-10-11T09:41:16.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.674+0000 7f1e051a0700 1 -- start start 2023-10-11T09:41:16.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.674+0000 7f1e051a0700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e0006d340 0x7f1e000ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:16.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.674+0000 7f1e051a0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e000cadc0 0x7f1e0015afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:16.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.674+0000 7f1e051a0700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e00158430 0x7f1e000b36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:16.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.674+0000 7f1e051a0700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e000b0960 con 0x7f1e000cadc0 2023-10-11T09:41:16.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.674+0000 7f1e051a0700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e000b0120 con 0x7f1e0006d340 2023-10-11T09:41:16.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.674+0000 7f1e051a0700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e000b3bf0 con 0x7f1e00158430 2023-10-11T09:41:16.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.678+0000 7f1dfe59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e000cadc0 0x7f1e0015afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:16.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.678+0000 7f1dfe59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e000cadc0 0x7f1e0015afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50708/0 (socket says 172.21.15.78:50708) 2023-10-11T09:41:16.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.678+0000 7f1dfe59c700 1 -- 172.21.15.78:0/4270755022 learned_addr learned my addr 172.21.15.78:0/4270755022 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:16.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.678+0000 7f1dfed9d700 1 --2- 172.21.15.78:0/4270755022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e0006d340 0x7f1e000ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:16.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.678+0000 7f1dff59e700 1 --2- 172.21.15.78:0/4270755022 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e00158430 0x7f1e000b36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:16.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.678+0000 7f1dfe59c700 1 -- 172.21.15.78:0/4270755022 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e00158430 msgr2=0x7f1e000b36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:16.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.678+0000 7f1dfe59c700 1 --2- 172.21.15.78:0/4270755022 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e00158430 0x7f1e000b36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:16.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.678+0000 7f1dfe59c700 1 -- 172.21.15.78:0/4270755022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e0006d340 msgr2=0x7f1e000ade90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:16.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.678+0000 7f1dfe59c700 1 --2- 172.21.15.78:0/4270755022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e0006d340 0x7f1e000ade90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:16.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.678+0000 7f1dfe59c700 1 -- 172.21.15.78:0/4270755022 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df4009570 con 0x7f1e000cadc0 2023-10-11T09:41:16.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.678+0000 7f1dfed9d700 1 --2- 172.21.15.78:0/4270755022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e0006d340 0x7f1e000ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:16.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.678+0000 7f1dff59e700 1 --2- 172.21.15.78:0/4270755022 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e00158430 0x7f1e000b36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:16.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.678+0000 7f1dfe59c700 1 --2- 172.21.15.78:0/4270755022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e000cadc0 0x7f1e0015afe0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1e0006fbc0 tx=0x7f1df4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:16.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.678+0000 7f1de77fe700 1 -- 172.21.15.78:0/4270755022 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df40036c0 con 0x7f1e000cadc0 2023-10-11T09:41:16.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.678+0000 7f1e051a0700 1 -- 172.21.15.78:0/4270755022 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e00157080 con 0x7f1e000cadc0 2023-10-11T09:41:16.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.678+0000 7f1de77fe700 1 -- 172.21.15.78:0/4270755022 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df401f070 con 0x7f1e000cadc0 2023-10-11T09:41:16.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.678+0000 7f1de77fe700 1 -- 172.21.15.78:0/4270755022 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df4003a80 con 0x7f1e000cadc0 2023-10-11T09:41:16.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.678+0000 7f1e051a0700 1 -- 172.21.15.78:0/4270755022 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e000b0680 con 0x7f1e000cadc0 2023-10-11T09:41:16.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.678+0000 7f1e051a0700 1 -- 172.21.15.78:0/4270755022 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e00156ee0 con 0x7f1e000cadc0 2023-10-11T09:41:16.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.682+0000 7f1de77fe700 1 -- 172.21.15.78:0/4270755022 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f1df4021030 con 0x7f1e000cadc0 2023-10-11T09:41:16.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.682+0000 7f1de77fe700 1 --2- 172.21.15.78:0/4270755022 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1dec065ed0 0x7f1dec068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:16.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.682+0000 7f1de77fe700 1 -- 172.21.15.78:0/4270755022 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1df4084e70 con 0x7f1e000cadc0 2023-10-11T09:41:16.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.682+0000 7f1dfed9d700 1 --2- 172.21.15.78:0/4270755022 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1dec065ed0 0x7f1dec068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:16.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.682+0000 7f1dfed9d700 1 --2- 172.21.15.78:0/4270755022 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1dec065ed0 0x7f1dec068380 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1e0014d430 tx=0x7f1df0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:16.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.690+0000 7f1de77fe700 1 -- 172.21.15.78:0/4270755022 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f1e00156ee0 con 0x7f1e000cadc0 2023-10-11T09:41:16.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.858+0000 7f1e051a0700 1 -- 172.21.15.78:0/4270755022 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1e00156ee0 con 0x7f1e000cadc0 2023-10-11T09:41:16.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.858+0000 7f1de77fe700 1 -- 172.21.15.78:0/4270755022 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1e00156ee0 con 0x7f1e000cadc0 2023-10-11T09:41:16.864 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:41:16.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.862+0000 7f1e051a0700 1 -- 172.21.15.78:0/4270755022 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1dec065ed0 msgr2=0x7f1dec068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:16.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.862+0000 7f1e051a0700 1 --2- 172.21.15.78:0/4270755022 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1dec065ed0 0x7f1dec068380 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1e0014d430 tx=0x7f1df0008040 comp rx=0 tx=0).stop 2023-10-11T09:41:16.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.862+0000 7f1e051a0700 1 -- 172.21.15.78:0/4270755022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e000cadc0 msgr2=0x7f1e0015afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:16.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.862+0000 7f1e051a0700 1 --2- 172.21.15.78:0/4270755022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e000cadc0 0x7f1e0015afe0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1e0006fbc0 tx=0x7f1df4004770 comp rx=0 tx=0).stop 2023-10-11T09:41:16.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.866+0000 7f1e051a0700 1 -- 172.21.15.78:0/4270755022 shutdown_connections 2023-10-11T09:41:16.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.866+0000 7f1e051a0700 1 --2- 172.21.15.78:0/4270755022 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1dec065ed0 0x7f1dec068380 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:16.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.866+0000 7f1e051a0700 1 --2- 172.21.15.78:0/4270755022 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e00158430 0x7f1e000b36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:16.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.866+0000 7f1e051a0700 1 --2- 172.21.15.78:0/4270755022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e000cadc0 0x7f1e0015afe0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:16.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.866+0000 7f1e051a0700 1 --2- 172.21.15.78:0/4270755022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e0006d340 0x7f1e000ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:16.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.866+0000 7f1e051a0700 1 -- 172.21.15.78:0/4270755022 >> 172.21.15.78:0/4270755022 conn(0x7f1e000cecc0 msgr2=0x7f1e00157a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:16.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.866+0000 7f1e051a0700 1 -- 172.21.15.78:0/4270755022 shutdown_connections 2023-10-11T09:41:16.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:16.866+0000 7f1e051a0700 1 -- 172.21.15.78:0/4270755022 wait complete. 2023-10-11T09:41:16.891 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:41:17.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.062+0000 7f23b45e6700 1 Processor -- start 2023-10-11T09:41:17.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.062+0000 7f23b45e6700 1 -- start start 2023-10-11T09:41:17.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.066+0000 7f23b45e6700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23ac158430 0x7f23ac0710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:17.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.066+0000 7f23b45e6700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f23ac0caf90 0x7f23ac15afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:17.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.066+0000 7f23b45e6700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23ac071610 con 0x7f23ac06d340 2023-10-11T09:41:17.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.066+0000 7f23b45e6700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23ac070c10 con 0x7f23ac0caf90 2023-10-11T09:41:17.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.066+0000 7f23b45e6700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23ac15e280 con 0x7f23ac158430 2023-10-11T09:41:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.066+0000 7f23b2b83700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f23ac0caf90 0x7f23ac15afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.066+0000 7f23b2382700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23ac158430 0x7f23ac0710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.066+0000 7f23b2b83700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f23ac0caf90 0x7f23ac15afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50716/0 (socket says 172.21.15.78:50716) 2023-10-11T09:41:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.066+0000 7f23b2b83700 1 -- 172.21.15.78:0/565963147 learned_addr learned my addr 172.21.15.78:0/565963147 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:17.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.066+0000 7f23b2b83700 1 -- 172.21.15.78:0/565963147 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23ac158430 msgr2=0x7f23ac0710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:17.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.066+0000 7f23b2b83700 1 --2- 172.21.15.78:0/565963147 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23ac158430 0x7f23ac0710d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:17.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.066+0000 7f23b2b83700 1 -- 172.21.15.78:0/565963147 >> v1:172.21.15.133:6789/0 conn(0x7f23ac06d340 legacy=0x7f23ac0ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:17.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.066+0000 7f23b2b83700 1 -- 172.21.15.78:0/565963147 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23ac15e560 con 0x7f23ac0caf90 2023-10-11T09:41:17.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.066+0000 7f23b2b83700 1 --2- 172.21.15.78:0/565963147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23ac0caf90 0x7f23ac15afe0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f23a4009e50 tx=0x7f23a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d73514255eaa7094 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:17.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.066+0000 7f23b1380700 1 -- 172.21.15.78:0/565963147 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a400e070 con 0x7f23ac0caf90 2023-10-11T09:41:17.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.070+0000 7f23b1380700 1 -- 172.21.15.78:0/565963147 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a400b9c0 con 0x7f23ac0caf90 2023-10-11T09:41:17.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.070+0000 7f23b1380700 1 -- 172.21.15.78:0/565963147 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a400bd70 con 0x7f23ac0caf90 2023-10-11T09:41:17.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.070+0000 7f23b45e6700 1 -- 172.21.15.78:0/565963147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23ac0caf90 msgr2=0x7f23ac15afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:17.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.070+0000 7f23b45e6700 1 --2- 172.21.15.78:0/565963147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23ac0caf90 0x7f23ac15afe0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f23a4009e50 tx=0x7f23a400b330 comp rx=0 tx=0).stop 2023-10-11T09:41:17.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.070+0000 7f23b45e6700 1 -- 172.21.15.78:0/565963147 shutdown_connections 2023-10-11T09:41:17.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.070+0000 7f23b45e6700 1 --2- 172.21.15.78:0/565963147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23ac0caf90 0x7f23ac15afe0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:17.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.070+0000 7f23b45e6700 1 --2- 172.21.15.78:0/565963147 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23ac158430 0x7f23ac0710d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:17.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.070+0000 7f23b45e6700 1 -- 172.21.15.78:0/565963147 >> 172.21.15.78:0/565963147 conn(0x7f23ac0cecc0 msgr2=0x7f23ac0cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:17.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.070+0000 7f23b45e6700 1 -- 172.21.15.78:0/565963147 shutdown_connections 2023-10-11T09:41:17.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.070+0000 7f23b45e6700 1 -- 172.21.15.78:0/565963147 wait complete. 2023-10-11T09:41:17.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.070+0000 7f23b45e6700 1 Processor -- start 2023-10-11T09:41:17.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.070+0000 7f23b45e6700 1 -- start start 2023-10-11T09:41:17.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.074+0000 7f23b45e6700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23ac06d340 0x7f23ac0c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:17.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.074+0000 7f23b45e6700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23ac0caf90 0x7f23ac0b1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:17.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.074+0000 7f23b45e6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23ac158430 0x7f23ac0b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:17.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.074+0000 7f23b45e6700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23ac0ad4a0 con 0x7f23ac158430 2023-10-11T09:41:17.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.074+0000 7f23b45e6700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23ac0b34b0 con 0x7f23ac0caf90 2023-10-11T09:41:17.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.074+0000 7f23b45e6700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23ac0b28a0 con 0x7f23ac06d340 2023-10-11T09:41:17.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.074+0000 7f23b2b83700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23ac158430 0x7f23ac0b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:17.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.074+0000 7f23b2b83700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23ac158430 0x7f23ac0b21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50732/0 (socket says 172.21.15.78:50732) 2023-10-11T09:41:17.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.074+0000 7f23b2b83700 1 -- 172.21.15.78:0/2660736655 learned_addr learned my addr 172.21.15.78:0/2660736655 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:17.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.074+0000 7f23b2382700 1 --2- 172.21.15.78:0/2660736655 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23ac06d340 0x7f23ac0c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:17.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.074+0000 7f23b1b81700 1 --2- 172.21.15.78:0/2660736655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23ac0caf90 0x7f23ac0b1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:17.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.074+0000 7f23b2b83700 1 -- 172.21.15.78:0/2660736655 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23ac06d340 msgr2=0x7f23ac0c95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:17.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.074+0000 7f23b2b83700 1 --2- 172.21.15.78:0/2660736655 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23ac06d340 0x7f23ac0c95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:17.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.074+0000 7f23b2b83700 1 -- 172.21.15.78:0/2660736655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23ac0caf90 msgr2=0x7f23ac0b1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:17.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.074+0000 7f23b2b83700 1 --2- 172.21.15.78:0/2660736655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23ac0caf90 0x7f23ac0b1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:17.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.074+0000 7f23b2b83700 1 -- 172.21.15.78:0/2660736655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a4009b30 con 0x7f23ac158430 2023-10-11T09:41:17.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.074+0000 7f23b2382700 1 --2- 172.21.15.78:0/2660736655 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23ac06d340 0x7f23ac0c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:17.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.074+0000 7f23b2b83700 1 --2- 172.21.15.78:0/2660736655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23ac158430 0x7f23ac0b21d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f23ac06fbc0 tx=0x7f23a4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:17.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.074+0000 7f23a2ffd700 1 -- 172.21.15.78:0/2660736655 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a40099b0 con 0x7f23ac158430 2023-10-11T09:41:17.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.074+0000 7f23a2ffd700 1 -- 172.21.15.78:0/2660736655 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a40092e0 con 0x7f23ac158430 2023-10-11T09:41:17.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.074+0000 7f23b45e6700 1 -- 172.21.15.78:0/2660736655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23ac0b2b80 con 0x7f23ac158430 2023-10-11T09:41:17.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.074+0000 7f23a2ffd700 1 -- 172.21.15.78:0/2660736655 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a401c640 con 0x7f23ac158430 2023-10-11T09:41:17.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.074+0000 7f23b45e6700 1 -- 172.21.15.78:0/2660736655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23ac0b3140 con 0x7f23ac158430 2023-10-11T09:41:17.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.078+0000 7f23b45e6700 1 -- 172.21.15.78:0/2660736655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23ac0b3620 con 0x7f23ac158430 2023-10-11T09:41:17.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.078+0000 7f23a2ffd700 1 -- 172.21.15.78:0/2660736655 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f23a401c7a0 con 0x7f23ac158430 2023-10-11T09:41:17.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.082+0000 7f23a2ffd700 1 --2- 172.21.15.78:0/2660736655 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2390065ed0 0x7f2390068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:17.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.082+0000 7f23a2ffd700 1 -- 172.21.15.78:0/2660736655 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f23a40854d0 con 0x7f23ac158430 2023-10-11T09:41:17.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.082+0000 7f23b2382700 1 --2- 172.21.15.78:0/2660736655 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2390065ed0 0x7f2390068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:17.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.082+0000 7f23b2382700 1 --2- 172.21.15.78:0/2660736655 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2390065ed0 0x7f2390068380 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f23ac1f30b0 tx=0x7f23a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:17.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.086+0000 7f23a2ffd700 1 -- 172.21.15.78:0/2660736655 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f23a4052bb0 con 0x7f23ac158430 2023-10-11T09:41:17.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.258+0000 7f23b45e6700 1 -- 172.21.15.78:0/2660736655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f23ac065db0 con 0x7f23ac158430 2023-10-11T09:41:17.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.258+0000 7f23a2ffd700 1 -- 172.21.15.78:0/2660736655 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f23a40561d0 con 0x7f23ac158430 2023-10-11T09:41:17.263 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:41:17.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.262+0000 7f23b45e6700 1 -- 172.21.15.78:0/2660736655 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2390065ed0 msgr2=0x7f2390068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:17.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.262+0000 7f23b45e6700 1 --2- 172.21.15.78:0/2660736655 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2390065ed0 0x7f2390068380 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f23ac1f30b0 tx=0x7f23a8008040 comp rx=0 tx=0).stop 2023-10-11T09:41:17.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.262+0000 7f23b45e6700 1 -- 172.21.15.78:0/2660736655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23ac158430 msgr2=0x7f23ac0b21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:17.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.262+0000 7f23b45e6700 1 --2- 172.21.15.78:0/2660736655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23ac158430 0x7f23ac0b21d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f23ac06fbc0 tx=0x7f23a4004af0 comp rx=0 tx=0).stop 2023-10-11T09:41:17.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.262+0000 7f23b45e6700 1 -- 172.21.15.78:0/2660736655 shutdown_connections 2023-10-11T09:41:17.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.262+0000 7f23b45e6700 1 --2- 172.21.15.78:0/2660736655 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2390065ed0 0x7f2390068380 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:17.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.262+0000 7f23b45e6700 1 --2- 172.21.15.78:0/2660736655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23ac158430 0x7f23ac0b21d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:17.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.262+0000 7f23b45e6700 1 --2- 172.21.15.78:0/2660736655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23ac0caf90 0x7f23ac0b1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:17.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.262+0000 7f23b45e6700 1 --2- 172.21.15.78:0/2660736655 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23ac06d340 0x7f23ac0c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:17.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.262+0000 7f23b45e6700 1 -- 172.21.15.78:0/2660736655 >> 172.21.15.78:0/2660736655 conn(0x7f23ac0cecc0 msgr2=0x7f23ac0aeb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:17.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.262+0000 7f23b45e6700 1 -- 172.21.15.78:0/2660736655 shutdown_connections 2023-10-11T09:41:17.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.262+0000 7f23b45e6700 1 -- 172.21.15.78:0/2660736655 wait complete. 2023-10-11T09:41:17.289 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T09:41:17.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.462+0000 7f87b82cf700 1 Processor -- start 2023-10-11T09:41:17.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.462+0000 7f87b82cf700 1 -- start start 2023-10-11T09:41:17.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.466+0000 7f87b82cf700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f87b00d2350 0x7f87b00d36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:17.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.466+0000 7f87b82cf700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b00cd690 con 0x7f87b00b84c0 2023-10-11T09:41:17.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.466+0000 7f87b82cf700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b00cd8e0 con 0x7f87b00b8890 2023-10-11T09:41:17.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.466+0000 7f87b82cf700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87b00cdea0 con 0x7f87b00d2350 2023-10-11T09:41:17.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.466+0000 7f87b686c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f87b00d2350 0x7f87b00d36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:17.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.466+0000 7f87b686c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f87b00d2350 0x7f87b00d36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50734/0 (socket says 172.21.15.78:50734) 2023-10-11T09:41:17.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.466+0000 7f87b686c700 1 -- 172.21.15.78:0/1793655903 learned_addr learned my addr 172.21.15.78:0/1793655903 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:17.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.466+0000 7f87b606b700 1 --1- 172.21.15.78:0/1793655903 >> v1:172.21.15.78:6789/0 conn(0x7f87b00b84c0 0x7f87b00cfda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:55800/0 (socket says 172.21.15.78:55800) 2023-10-11T09:41:17.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.466+0000 7f87b686c700 1 -- 172.21.15.78:0/1793655903 >> v1:172.21.15.133:6789/0 conn(0x7f87b00b8890 legacy=0x7f87b00d2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:17.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.466+0000 7f87b686c700 1 -- 172.21.15.78:0/1793655903 >> v1:172.21.15.78:6789/0 conn(0x7f87b00b84c0 legacy=0x7f87b00cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:17.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.466+0000 7f87b686c700 1 -- 172.21.15.78:0/1793655903 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b00cdbc0 con 0x7f87b00d2350 2023-10-11T09:41:17.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.466+0000 7f87b686c700 1 --2- 172.21.15.78:0/1793655903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87b00d2350 0x7f87b00d36e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f87a8009e50 tx=0x7f87a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d217b13d5a8f1c45 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:17.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.466+0000 7f87b5069700 1 -- 172.21.15.78:0/1793655903 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a800e070 con 0x7f87b00d2350 2023-10-11T09:41:17.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.466+0000 7f87b5069700 1 -- 172.21.15.78:0/1793655903 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a8003a40 con 0x7f87b00d2350 2023-10-11T09:41:17.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.466+0000 7f87b5069700 1 -- 172.21.15.78:0/1793655903 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a8003df0 con 0x7f87b00d2350 2023-10-11T09:41:17.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.470+0000 7f87b82cf700 1 -- 172.21.15.78:0/1793655903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87b00d2350 msgr2=0x7f87b00d36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:17.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.470+0000 7f87b82cf700 1 --2- 172.21.15.78:0/1793655903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87b00d2350 0x7f87b00d36e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f87a8009e50 tx=0x7f87a800b330 comp rx=0 tx=0).stop 2023-10-11T09:41:17.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.470+0000 7f87b82cf700 1 -- 172.21.15.78:0/1793655903 shutdown_connections 2023-10-11T09:41:17.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.470+0000 7f87b82cf700 1 --2- 172.21.15.78:0/1793655903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87b00d2350 0x7f87b00d36e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:17.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.470+0000 7f87b82cf700 1 -- 172.21.15.78:0/1793655903 >> 172.21.15.78:0/1793655903 conn(0x7f87b006cb50 msgr2=0x7f87b006cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:17.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.470+0000 7f87b82cf700 1 -- 172.21.15.78:0/1793655903 shutdown_connections 2023-10-11T09:41:17.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.470+0000 7f87b82cf700 1 -- 172.21.15.78:0/1793655903 wait complete. 2023-10-11T09:41:17.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.470+0000 7f87b82cf700 1 Processor -- start 2023-10-11T09:41:17.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.470+0000 7f87b82cf700 1 -- start start 2023-10-11T09:41:17.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.474+0000 7f87b82cf700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87b00b84c0 0x7f87b01578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:17.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.474+0000 7f87b82cf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87b00b8890 0x7f87b01485c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:17.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.474+0000 7f87b82cf700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f87b00d2350 0x7f87b00ca110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:17.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.474+0000 7f87b82cf700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b0156720 con 0x7f87b00b8890 2023-10-11T09:41:17.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.474+0000 7f87b82cf700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b00c8c20 con 0x7f87b00b84c0 2023-10-11T09:41:17.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.474+0000 7f87b82cf700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b00c7d30 con 0x7f87b00d2350 2023-10-11T09:41:17.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.474+0000 7f87b586a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87b00b8890 0x7f87b01485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:17.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.474+0000 7f87b606b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87b00b84c0 0x7f87b01578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:17.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.474+0000 7f87b606b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87b00b84c0 0x7f87b01578b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:53538/0 (socket says 172.21.15.78:53538) 2023-10-11T09:41:17.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.474+0000 7f87b586a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87b00b8890 0x7f87b01485c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50748/0 (socket says 172.21.15.78:50748) 2023-10-11T09:41:17.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.474+0000 7f87b606b700 1 -- 172.21.15.78:0/3683603743 learned_addr learned my addr 172.21.15.78:0/3683603743 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:17.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.474+0000 7f87b686c700 1 --2- 172.21.15.78:0/3683603743 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f87b00d2350 0x7f87b00ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:17.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.474+0000 7f87b586a700 1 -- 172.21.15.78:0/3683603743 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f87b00d2350 msgr2=0x7f87b00ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:17.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.474+0000 7f87b586a700 1 --2- 172.21.15.78:0/3683603743 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f87b00d2350 0x7f87b00ca110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:17.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.474+0000 7f87b586a700 1 -- 172.21.15.78:0/3683603743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87b00b84c0 msgr2=0x7f87b01578b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:17.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.474+0000 7f87b586a700 1 --2- 172.21.15.78:0/3683603743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87b00b84c0 0x7f87b01578b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:17.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.474+0000 7f87b586a700 1 -- 172.21.15.78:0/3683603743 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a8009b30 con 0x7f87b00b8890 2023-10-11T09:41:17.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.474+0000 7f87b606b700 1 --2- 172.21.15.78:0/3683603743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87b00b84c0 0x7f87b01578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:41:17.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.474+0000 7f87b586a700 1 --2- 172.21.15.78:0/3683603743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87b00b8890 0x7f87b01485c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f87a000c490 tx=0x7f87a000c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:17.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.474+0000 7f87a6ffd700 1 -- 172.21.15.78:0/3683603743 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a000cea0 con 0x7f87b00b8890 2023-10-11T09:41:17.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.474+0000 7f87a6ffd700 1 -- 172.21.15.78:0/3683603743 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a00117b0 con 0x7f87b00b8890 2023-10-11T09:41:17.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.474+0000 7f87b82cf700 1 -- 172.21.15.78:0/3683603743 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b00c8010 con 0x7f87b00b8890 2023-10-11T09:41:17.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.474+0000 7f87a6ffd700 1 -- 172.21.15.78:0/3683603743 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a0011b60 con 0x7f87b00b8890 2023-10-11T09:41:17.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.474+0000 7f87b82cf700 1 -- 172.21.15.78:0/3683603743 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b00c85d0 con 0x7f87b00b8890 2023-10-11T09:41:17.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.478+0000 7f87b82cf700 1 -- 172.21.15.78:0/3683603743 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b00c8dd0 con 0x7f87b00b8890 2023-10-11T09:41:17.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.478+0000 7f87a6ffd700 1 -- 172.21.15.78:0/3683603743 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f87a0011cc0 con 0x7f87b00b8890 2023-10-11T09:41:17.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.482+0000 7f87a6ffd700 1 --2- 172.21.15.78:0/3683603743 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8794065ed0 0x7f8794068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:17.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.482+0000 7f87a6ffd700 1 -- 172.21.15.78:0/3683603743 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f87a00834e0 con 0x7f87b00b8890 2023-10-11T09:41:17.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.482+0000 7f87b606b700 1 --2- 172.21.15.78:0/3683603743 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8794065ed0 0x7f8794068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:17.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.482+0000 7f87b606b700 1 --2- 172.21.15.78:0/3683603743 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8794065ed0 0x7f8794068380 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f87ac000f60 tx=0x7f87ac0010f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:17.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.486+0000 7f87a6ffd700 1 -- 172.21.15.78:0/3683603743 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f87b00c8dd0 con 0x7f87b00b8890 2023-10-11T09:41:17.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.662+0000 7f87b82cf700 1 -- 172.21.15.78:0/3683603743 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f87b00c8d90 con 0x7f87b00b8890 2023-10-11T09:41:17.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.662+0000 7f87a6ffd700 1 -- 172.21.15.78:0/3683603743 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f87a0050a90 con 0x7f87b00b8890 2023-10-11T09:41:17.666 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:41:17.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.666+0000 7f87b82cf700 1 -- 172.21.15.78:0/3683603743 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8794065ed0 msgr2=0x7f8794068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:17.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.666+0000 7f87b82cf700 1 --2- 172.21.15.78:0/3683603743 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8794065ed0 0x7f8794068380 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f87ac000f60 tx=0x7f87ac0010f0 comp rx=0 tx=0).stop 2023-10-11T09:41:17.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.666+0000 7f87b82cf700 1 -- 172.21.15.78:0/3683603743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87b00b8890 msgr2=0x7f87b01485c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:17.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.666+0000 7f87b82cf700 1 --2- 172.21.15.78:0/3683603743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87b00b8890 0x7f87b01485c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f87a000c490 tx=0x7f87a000c7a0 comp rx=0 tx=0).stop 2023-10-11T09:41:17.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.666+0000 7f87b82cf700 1 -- 172.21.15.78:0/3683603743 shutdown_connections 2023-10-11T09:41:17.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.666+0000 7f87b82cf700 1 --2- 172.21.15.78:0/3683603743 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8794065ed0 0x7f8794068380 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:17.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.666+0000 7f87b82cf700 1 --2- 172.21.15.78:0/3683603743 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f87b00d2350 0x7f87b00ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:17.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.666+0000 7f87b82cf700 1 --2- 172.21.15.78:0/3683603743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87b00b8890 0x7f87b01485c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:17.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.666+0000 7f87b82cf700 1 --2- 172.21.15.78:0/3683603743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87b00b84c0 0x7f87b01578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:17.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.666+0000 7f87b82cf700 1 -- 172.21.15.78:0/3683603743 >> 172.21.15.78:0/3683603743 conn(0x7f87b006cb50 msgr2=0x7f87b00d2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:17.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.666+0000 7f87b82cf700 1 -- 172.21.15.78:0/3683603743 shutdown_connections 2023-10-11T09:41:17.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.666+0000 7f87b82cf700 1 -- 172.21.15.78:0/3683603743 wait complete. 2023-10-11T09:41:17.687 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T09:41:17.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.862+0000 7fa1573f2700 1 Processor -- start 2023-10-11T09:41:17.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.862+0000 7fa1573f2700 1 -- start start 2023-10-11T09:41:17.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.866+0000 7fa1573f2700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fa15006c6a0 0x7fa150159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:17.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.866+0000 7fa1573f2700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa1500cb8e0 0x7fa15006ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:17.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.866+0000 7fa1573f2700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa15015a130 con 0x7fa150070780 2023-10-11T09:41:17.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.866+0000 7fa1573f2700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa15015a370 con 0x7fa15006c6a0 2023-10-11T09:41:17.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.866+0000 7fa1573f2700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa15006cfb0 con 0x7fa1500cb8e0 2023-10-11T09:41:17.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.866+0000 7fa15598f700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fa150070780 0x7fa15006a620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:55810/0 (socket says 172.21.15.78:55810) 2023-10-11T09:41:17.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.866+0000 7fa15598f700 1 -- 172.21.15.78:0/471721106 learned_addr learned my addr 172.21.15.78:0/471721106 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:17.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.866+0000 7fa15518e700 1 --2- 172.21.15.78:0/471721106 >> v2:172.21.15.78:3300/0 conn(0x7fa15006c6a0 0x7fa150159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:17.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.866+0000 7fa15498d700 1 --2- 172.21.15.78:0/471721106 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa1500cb8e0 0x7fa15006ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:17.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.866+0000 7fa147fff700 1 -- 172.21.15.78:0/471721106 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2265551676 0 0) 0x7fa148001490 con 0x7fa150070780 2023-10-11T09:41:17.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.866+0000 7fa147fff700 1 -- 172.21.15.78:0/471721106 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa13c0035d0 con 0x7fa150070780 2023-10-11T09:41:17.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.866+0000 7fa15518e700 1 -- 172.21.15.78:0/471721106 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa1500cb8e0 msgr2=0x7fa15006ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:17.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.866+0000 7fa15518e700 1 --2- 172.21.15.78:0/471721106 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa1500cb8e0 0x7fa15006ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:17.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.866+0000 7fa15518e700 1 -- 172.21.15.78:0/471721106 >> v1:172.21.15.78:6789/0 conn(0x7fa150070780 legacy=0x7fa15006a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:17.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.866+0000 7fa15518e700 1 -- 172.21.15.78:0/471721106 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1500c9de0 con 0x7fa15006c6a0 2023-10-11T09:41:17.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.866+0000 7fa15498d700 1 --2- 172.21.15.78:0/471721106 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa1500cb8e0 0x7fa15006ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:17.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.866+0000 7fa15518e700 1 --2- 172.21.15.78:0/471721106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa15006c6a0 0x7fa150159bf0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009880 tx=0x7fa14c009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef598309fdf7293 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:17.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.870+0000 7fa147fff700 1 -- 172.21.15.78:0/471721106 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c011830 con 0x7fa15006c6a0 2023-10-11T09:41:17.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.870+0000 7fa147fff700 1 -- 172.21.15.78:0/471721106 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa14c011990 con 0x7fa15006c6a0 2023-10-11T09:41:17.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.870+0000 7fa147fff700 1 -- 172.21.15.78:0/471721106 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c011d20 con 0x7fa15006c6a0 2023-10-11T09:41:17.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.870+0000 7fa1573f2700 1 -- 172.21.15.78:0/471721106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa15006c6a0 msgr2=0x7fa150159bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:17.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.870+0000 7fa1573f2700 1 --2- 172.21.15.78:0/471721106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa15006c6a0 0x7fa150159bf0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009880 tx=0x7fa14c009b90 comp rx=0 tx=0).stop 2023-10-11T09:41:17.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.870+0000 7fa1573f2700 1 -- 172.21.15.78:0/471721106 shutdown_connections 2023-10-11T09:41:17.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.870+0000 7fa1573f2700 1 --2- 172.21.15.78:0/471721106 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa1500cb8e0 0x7fa15006ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:17.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.870+0000 7fa1573f2700 1 --2- 172.21.15.78:0/471721106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa15006c6a0 0x7fa150159bf0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:17.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.870+0000 7fa1573f2700 1 -- 172.21.15.78:0/471721106 >> 172.21.15.78:0/471721106 conn(0x7fa1500b1d60 msgr2=0x7fa1500cf5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:17.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.870+0000 7fa1573f2700 1 -- 172.21.15.78:0/471721106 shutdown_connections 2023-10-11T09:41:17.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.870+0000 7fa1573f2700 1 -- 172.21.15.78:0/471721106 wait complete. 2023-10-11T09:41:17.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.874+0000 7fa1573f2700 1 Processor -- start 2023-10-11T09:41:17.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.874+0000 7fa1573f2700 1 -- start start 2023-10-11T09:41:17.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.874+0000 7fa1573f2700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa15006c6a0 0x7fa1501f1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:17.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.874+0000 7fa1573f2700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa150070780 0x7fa1500c30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:17.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.874+0000 7fa1573f2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1500cb8e0 0x7fa1500c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:17.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.874+0000 7fa1573f2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1501f2210 con 0x7fa1500cb8e0 2023-10-11T09:41:17.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.874+0000 7fa1573f2700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1500c3a10 con 0x7fa150070780 2023-10-11T09:41:17.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.874+0000 7fa1573f2700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1500c3cf0 con 0x7fa15006c6a0 2023-10-11T09:41:17.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.874+0000 7fa15598f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1500cb8e0 0x7fa1500c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:17.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.874+0000 7fa15498d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa150070780 0x7fa1500c30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:17.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.874+0000 7fa15518e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa15006c6a0 0x7fa1501f1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:17.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.874+0000 7fa15598f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1500cb8e0 0x7fa1500c4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50756/0 (socket says 172.21.15.78:50756) 2023-10-11T09:41:17.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.874+0000 7fa15498d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa150070780 0x7fa1500c30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:53544/0 (socket says 172.21.15.78:53544) 2023-10-11T09:41:17.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.874+0000 7fa15598f700 1 -- 172.21.15.78:0/3761737431 learned_addr learned my addr 172.21.15.78:0/3761737431 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:17.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.878+0000 7fa15598f700 1 -- 172.21.15.78:0/3761737431 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa15006c6a0 msgr2=0x7fa1501f1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:17.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.878+0000 7fa15598f700 1 --2- 172.21.15.78:0/3761737431 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa15006c6a0 0x7fa1501f1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:17.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.878+0000 7fa15598f700 1 -- 172.21.15.78:0/3761737431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa150070780 msgr2=0x7fa1500c30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:17.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.878+0000 7fa15598f700 1 --2- 172.21.15.78:0/3761737431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa150070780 0x7fa1500c30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:17.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.878+0000 7fa15598f700 1 -- 172.21.15.78:0/3761737431 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa14c0094b0 con 0x7fa1500cb8e0 2023-10-11T09:41:17.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.878+0000 7fa15598f700 1 --2- 172.21.15.78:0/3761737431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1500cb8e0 0x7fa1500c4ab0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa14800cb40 tx=0x7fa14800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:17.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.878+0000 7fa145ffb700 1 -- 172.21.15.78:0/3761737431 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa148007500 con 0x7fa1500cb8e0 2023-10-11T09:41:17.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.878+0000 7fa145ffb700 1 -- 172.21.15.78:0/3761737431 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa148007660 con 0x7fa1500cb8e0 2023-10-11T09:41:17.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.878+0000 7fa1573f2700 1 -- 172.21.15.78:0/3761737431 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1500c3fd0 con 0x7fa1500cb8e0 2023-10-11T09:41:17.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.878+0000 7fa145ffb700 1 -- 172.21.15.78:0/3761737431 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1480079f0 con 0x7fa1500cb8e0 2023-10-11T09:41:17.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.878+0000 7fa1573f2700 1 -- 172.21.15.78:0/3761737431 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1500c4590 con 0x7fa1500cb8e0 2023-10-11T09:41:17.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.878+0000 7fa1573f2700 1 -- 172.21.15.78:0/3761737431 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1500c4780 con 0x7fa1500cb8e0 2023-10-11T09:41:17.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.882+0000 7fa145ffb700 1 -- 172.21.15.78:0/3761737431 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fa14801b3b0 con 0x7fa1500cb8e0 2023-10-11T09:41:17.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.882+0000 7fa145ffb700 1 --2- 172.21.15.78:0/3761737431 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa134065ed0 0x7fa134068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:17.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.882+0000 7fa145ffb700 1 -- 172.21.15.78:0/3761737431 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa148083060 con 0x7fa1500cb8e0 2023-10-11T09:41:17.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.882+0000 7fa15518e700 1 --2- 172.21.15.78:0/3761737431 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa134065ed0 0x7fa134068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:17.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.886+0000 7fa15518e700 1 --2- 172.21.15.78:0/3761737431 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa134065ed0 0x7fa134068380 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009bd0 tx=0x7fa14c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:17.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:17.886+0000 7fa145ffb700 1 -- 172.21.15.78:0/3761737431 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fa1480506b0 con 0x7fa1500cb8e0 2023-10-11T09:41:18.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.058+0000 7fa1573f2700 1 -- 172.21.15.78:0/3761737431 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa15015a130 con 0x7fa1500cb8e0 2023-10-11T09:41:18.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.058+0000 7fa145ffb700 1 -- 172.21.15.78:0/3761737431 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa148017020 con 0x7fa1500cb8e0 2023-10-11T09:41:18.063 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2023-10-11T09:41:18.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.062+0000 7fa1573f2700 1 -- 172.21.15.78:0/3761737431 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa134065ed0 msgr2=0x7fa134068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:18.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.062+0000 7fa1573f2700 1 --2- 172.21.15.78:0/3761737431 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa134065ed0 0x7fa134068380 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009bd0 tx=0x7fa14c008030 comp rx=0 tx=0).stop 2023-10-11T09:41:18.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.062+0000 7fa1573f2700 1 -- 172.21.15.78:0/3761737431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1500cb8e0 msgr2=0x7fa1500c4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:18.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.062+0000 7fa1573f2700 1 --2- 172.21.15.78:0/3761737431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1500cb8e0 0x7fa1500c4ab0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa14800cb40 tx=0x7fa14800cb70 comp rx=0 tx=0).stop 2023-10-11T09:41:18.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.062+0000 7fa1573f2700 1 -- 172.21.15.78:0/3761737431 shutdown_connections 2023-10-11T09:41:18.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.062+0000 7fa1573f2700 1 --2- 172.21.15.78:0/3761737431 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa134065ed0 0x7fa134068380 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:18.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.062+0000 7fa1573f2700 1 --2- 172.21.15.78:0/3761737431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1500cb8e0 0x7fa1500c4ab0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:18.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.062+0000 7fa1573f2700 1 --2- 172.21.15.78:0/3761737431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa150070780 0x7fa1500c30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:18.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.062+0000 7fa1573f2700 1 --2- 172.21.15.78:0/3761737431 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa15006c6a0 0x7fa1501f1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:18.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.062+0000 7fa1573f2700 1 -- 172.21.15.78:0/3761737431 >> 172.21.15.78:0/3761737431 conn(0x7fa1500b1d60 msgr2=0x7fa150157a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:18.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.062+0000 7fa1573f2700 1 -- 172.21.15.78:0/3761737431 shutdown_connections 2023-10-11T09:41:18.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.062+0000 7fa1573f2700 1 -- 172.21.15.78:0/3761737431 wait complete. 2023-10-11T09:41:18.091 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:41:18.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.258+0000 7f8c9cb70700 1 Processor -- start 2023-10-11T09:41:18.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.258+0000 7f8c9cb70700 1 -- start start 2023-10-11T09:41:18.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.262+0000 7f8c9cb70700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f8c9815a940 0x7f8c980ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:18.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.262+0000 7f8c9cb70700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8c980ca780 0x7f8c9813fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:18.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.262+0000 7f8c9cb70700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c9815e010 con 0x7f8c9815b8e0 2023-10-11T09:41:18.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.262+0000 7f8c9cb70700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c980cab70 con 0x7f8c9815a940 2023-10-11T09:41:18.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.262+0000 7f8c9cb70700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c9815e280 con 0x7f8c980ca780 2023-10-11T09:41:18.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.262+0000 7f8c977fe700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f8c9815b8e0 0x7f8c9806d0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:55812/0 (socket says 172.21.15.78:55812) 2023-10-11T09:41:18.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.262+0000 7f8c977fe700 1 -- 172.21.15.78:0/1977708770 learned_addr learned my addr 172.21.15.78:0/1977708770 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:18.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.262+0000 7f8c96ffd700 1 --2- 172.21.15.78:0/1977708770 >> v2:172.21.15.78:3300/0 conn(0x7f8c9815a940 0x7f8c980ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:18.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.262+0000 7f8c97fff700 1 --2- 172.21.15.78:0/1977708770 >> v2:172.21.15.133:3300/0 conn(0x7f8c980ca780 0x7f8c9813fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:18.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.262+0000 7f8c967fc700 1 -- 172.21.15.78:0/1977708770 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3132961095 0 0) 0x7f8c9815e010 con 0x7f8c9815b8e0 2023-10-11T09:41:18.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.262+0000 7f8c967fc700 1 -- 172.21.15.78:0/1977708770 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c800035d0 con 0x7f8c9815b8e0 2023-10-11T09:41:18.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.262+0000 7f8c967fc700 1 -- 172.21.15.78:0/1977708770 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2802049135 0 0) 0x7f8c800035d0 con 0x7f8c9815b8e0 2023-10-11T09:41:18.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.262+0000 7f8c967fc700 1 -- 172.21.15.78:0/1977708770 >> v2:172.21.15.133:3300/0 conn(0x7f8c980ca780 msgr2=0x7f8c9813fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:18.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.262+0000 7f8c967fc700 1 --2- 172.21.15.78:0/1977708770 >> v2:172.21.15.133:3300/0 conn(0x7f8c980ca780 0x7f8c9813fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:18.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.262+0000 7f8c967fc700 1 -- 172.21.15.78:0/1977708770 >> v2:172.21.15.78:3300/0 conn(0x7f8c9815a940 msgr2=0x7f8c980ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:18.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.262+0000 7f8c967fc700 1 --2- 172.21.15.78:0/1977708770 >> v2:172.21.15.78:3300/0 conn(0x7f8c9815a940 0x7f8c980ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:18.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.262+0000 7f8c967fc700 1 -- 172.21.15.78:0/1977708770 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c9815e560 con 0x7f8c9815b8e0 2023-10-11T09:41:18.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.262+0000 7f8c967fc700 1 -- 172.21.15.78:0/1977708770 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f8c88001670 con 0x7f8c9815b8e0 2023-10-11T09:41:18.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.262+0000 7f8c967fc700 1 -- 172.21.15.78:0/1977708770 >> v1:172.21.15.78:6789/0 conn(0x7f8c9815b8e0 legacy=0x7f8c9806d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:18.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.262+0000 7f8c967fc700 1 --2- 172.21.15.78:0/1977708770 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c80003a70 0x7f8c80003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:18.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.262+0000 7f8c967fc700 1 --2- 172.21.15.78:0/1977708770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c80004870 0x7f8c80004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:18.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.262+0000 7f8c967fc700 1 --2- 172.21.15.78:0/1977708770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c80005220 0x7f8c8000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:18.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.262+0000 7f8c967fc700 1 -- 172.21.15.78:0/1977708770 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c980cab70 con 0x7f8c80004870 2023-10-11T09:41:18.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.262+0000 7f8c967fc700 1 -- 172.21.15.78:0/1977708770 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9815e280 con 0x7f8c80005220 2023-10-11T09:41:18.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.262+0000 7f8c967fc700 1 -- 172.21.15.78:0/1977708770 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8000bff0 con 0x7f8c80003a70 2023-10-11T09:41:18.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.266+0000 7f8c977fe700 1 --2- 172.21.15.78:0/1977708770 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c80003a70 0x7f8c80003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:18.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.266+0000 7f8c96ffd700 1 --2- 172.21.15.78:0/1977708770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c80004870 0x7f8c80004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:18.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.266+0000 7f8c97fff700 1 --2- 172.21.15.78:0/1977708770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c80005220 0x7f8c8000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:18.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.266+0000 7f8c96ffd700 1 -- 172.21.15.78:0/1977708770 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c80003a70 msgr2=0x7f8c80003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:18.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.266+0000 7f8c96ffd700 1 --2- 172.21.15.78:0/1977708770 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c80003a70 0x7f8c80003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:18.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.266+0000 7f8c96ffd700 1 -- 172.21.15.78:0/1977708770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c80005220 msgr2=0x7f8c8000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:18.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.266+0000 7f8c96ffd700 1 --2- 172.21.15.78:0/1977708770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c80005220 0x7f8c8000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:18.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.266+0000 7f8c96ffd700 1 -- 172.21.15.78:0/1977708770 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c800035d0 con 0x7f8c80004870 2023-10-11T09:41:18.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.266+0000 7f8c97fff700 1 -- 172.21.15.78:0/1977708770 reap_dead start 2023-10-11T09:41:18.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.266+0000 7f8c96ffd700 1 --2- 172.21.15.78:0/1977708770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c80004870 0x7f8c80004ce0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c009b70 tx=0x7f8c800058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a7ef739e13a7b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:18.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.266+0000 7f8c967fc700 1 -- 172.21.15.78:0/1977708770 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c00e6c0 con 0x7f8c80004870 2023-10-11T09:41:18.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.266+0000 7f8c967fc700 1 -- 172.21.15.78:0/1977708770 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8c00e820 con 0x7f8c80004870 2023-10-11T09:41:18.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.266+0000 7f8c967fc700 1 -- 172.21.15.78:0/1977708770 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c00eb90 con 0x7f8c80004870 2023-10-11T09:41:18.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.266+0000 7f8c9cb70700 1 -- 172.21.15.78:0/1977708770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c80004870 msgr2=0x7f8c80004ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:18.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.266+0000 7f8c9cb70700 1 --2- 172.21.15.78:0/1977708770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c80004870 0x7f8c80004ce0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c009b70 tx=0x7f8c800058a0 comp rx=0 tx=0).stop 2023-10-11T09:41:18.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.266+0000 7f8c9cb70700 1 -- 172.21.15.78:0/1977708770 shutdown_connections 2023-10-11T09:41:18.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.266+0000 7f8c9cb70700 1 --2- 172.21.15.78:0/1977708770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c80004870 0x7f8c80004ce0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:18.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.266+0000 7f8c9cb70700 1 -- 172.21.15.78:0/1977708770 >> 172.21.15.78:0/1977708770 conn(0x7f8c980cea60 msgr2=0x7f8c980cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:18.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.266+0000 7f8c9cb70700 1 -- 172.21.15.78:0/1977708770 shutdown_connections 2023-10-11T09:41:18.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.266+0000 7f8c9cb70700 1 -- 172.21.15.78:0/1977708770 wait complete. 2023-10-11T09:41:18.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.270+0000 7f8c9cb70700 1 Processor -- start 2023-10-11T09:41:18.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.270+0000 7f8c9cb70700 1 -- start start 2023-10-11T09:41:18.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.274+0000 7f8c9cb70700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c80004870 0x7f8c98151180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:18.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.274+0000 7f8c9cb70700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c9813d000 0x7f8c9813d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:18.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.274+0000 7f8c9cb70700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c9813af00 0x7f8c9813b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:18.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.274+0000 7f8c9cb70700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9813d990 con 0x7f8c9813d000 2023-10-11T09:41:18.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.274+0000 7f8c9cb70700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9813c840 con 0x7f8c80004870 2023-10-11T09:41:18.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.274+0000 7f8c9cb70700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9813ccd0 con 0x7f8c9813af00 2023-10-11T09:41:18.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.274+0000 7f8c96ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c9813d000 0x7f8c9813d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:18.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.274+0000 7f8c96ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c9813d000 0x7f8c9813d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50796/0 (socket says 172.21.15.78:50796) 2023-10-11T09:41:18.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.274+0000 7f8c96ffd700 1 -- 172.21.15.78:0/3331139344 learned_addr learned my addr 172.21.15.78:0/3331139344 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:18.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.274+0000 7f8c977fe700 1 --2- 172.21.15.78:0/3331139344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c80004870 0x7f8c98151180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:18.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.274+0000 7f8c97fff700 1 --2- 172.21.15.78:0/3331139344 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c9813af00 0x7f8c9813b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:18.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.274+0000 7f8c96ffd700 1 -- 172.21.15.78:0/3331139344 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c9813af00 msgr2=0x7f8c9813b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:18.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.274+0000 7f8c96ffd700 1 --2- 172.21.15.78:0/3331139344 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c9813af00 0x7f8c9813b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:18.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.274+0000 7f8c96ffd700 1 -- 172.21.15.78:0/3331139344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c80004870 msgr2=0x7f8c98151180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:18.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.274+0000 7f8c96ffd700 1 --2- 172.21.15.78:0/3331139344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c80004870 0x7f8c98151180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:18.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.274+0000 7f8c96ffd700 1 -- 172.21.15.78:0/3331139344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c981516c0 con 0x7f8c9813d000 2023-10-11T09:41:18.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.274+0000 7f8c96ffd700 1 --2- 172.21.15.78:0/3331139344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c9813d000 0x7f8c9813d450 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8c800047d0 tx=0x7f8c8c0047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:18.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.274+0000 7f8c7ffff700 1 -- 172.21.15.78:0/3331139344 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c00e6c0 con 0x7f8c9813d000 2023-10-11T09:41:18.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.274+0000 7f8c7ffff700 1 -- 172.21.15.78:0/3331139344 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8c003b10 con 0x7f8c9813d000 2023-10-11T09:41:18.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.274+0000 7f8c9cb70700 1 -- 172.21.15.78:0/3331139344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c9813bc30 con 0x7f8c9813d000 2023-10-11T09:41:18.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.274+0000 7f8c7ffff700 1 -- 172.21.15.78:0/3331139344 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c013470 con 0x7f8c9813d000 2023-10-11T09:41:18.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.274+0000 7f8c9cb70700 1 -- 172.21.15.78:0/3331139344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c9813c1f0 con 0x7f8c9813d000 2023-10-11T09:41:18.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.278+0000 7f8c9cb70700 1 -- 172.21.15.78:0/3331139344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c98067f50 con 0x7f8c9813d000 2023-10-11T09:41:18.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.278+0000 7f8c7ffff700 1 -- 172.21.15.78:0/3331139344 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f8c8c0135d0 con 0x7f8c9813d000 2023-10-11T09:41:18.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.282+0000 7f8c7ffff700 1 --2- 172.21.15.78:0/3331139344 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8c84065ed0 0x7f8c84068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:18.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.282+0000 7f8c7ffff700 1 -- 172.21.15.78:0/3331139344 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8c8c084d70 con 0x7f8c9813d000 2023-10-11T09:41:18.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.282+0000 7f8c977fe700 1 --2- 172.21.15.78:0/3331139344 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8c84065ed0 0x7f8c84068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:18.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.282+0000 7f8c977fe700 1 --2- 172.21.15.78:0/3331139344 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8c84065ed0 0x7f8c84068380 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8c88000c00 tx=0x7f8c88009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:18.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.286+0000 7f8c7ffff700 1 -- 172.21.15.78:0/3331139344 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f8c8c052460 con 0x7f8c9813d000 2023-10-11T09:41:18.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.466+0000 7f8c9cb70700 1 -- 172.21.15.78:0/3331139344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c98065d40 con 0x7f8c9813d000 2023-10-11T09:41:18.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.466+0000 7f8c7ffff700 1 -- 172.21.15.78:0/3331139344 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17811 (secure 0 0 0) 0x7f8c8c055a80 con 0x7f8c9813d000 2023-10-11T09:41:18.473 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:41:18.474 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":23,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:40:42.678477+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:40:39.862581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T09:40:40.663641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:41:18.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.470+0000 7f8c9cb70700 1 -- 172.21.15.78:0/3331139344 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8c84065ed0 msgr2=0x7f8c84068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:18.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.470+0000 7f8c9cb70700 1 --2- 172.21.15.78:0/3331139344 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8c84065ed0 0x7f8c84068380 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8c88000c00 tx=0x7f8c88009040 comp rx=0 tx=0).stop 2023-10-11T09:41:18.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.470+0000 7f8c9cb70700 1 -- 172.21.15.78:0/3331139344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c9813d000 msgr2=0x7f8c9813d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:18.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.470+0000 7f8c9cb70700 1 --2- 172.21.15.78:0/3331139344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c9813d000 0x7f8c9813d450 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8c800047d0 tx=0x7f8c8c0047e0 comp rx=0 tx=0).stop 2023-10-11T09:41:18.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.470+0000 7f8c9cb70700 1 -- 172.21.15.78:0/3331139344 shutdown_connections 2023-10-11T09:41:18.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.470+0000 7f8c9cb70700 1 --2- 172.21.15.78:0/3331139344 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8c84065ed0 0x7f8c84068380 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:18.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.470+0000 7f8c9cb70700 1 --2- 172.21.15.78:0/3331139344 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c9813af00 0x7f8c9813b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:18.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.470+0000 7f8c9cb70700 1 --2- 172.21.15.78:0/3331139344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c9813d000 0x7f8c9813d450 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:18.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.470+0000 7f8c9cb70700 1 --2- 172.21.15.78:0/3331139344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c80004870 0x7f8c98151180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:18.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.470+0000 7f8c9cb70700 1 -- 172.21.15.78:0/3331139344 >> 172.21.15.78:0/3331139344 conn(0x7f8c980cea60 msgr2=0x7f8c980cbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:18.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.470+0000 7f8c9cb70700 1 -- 172.21.15.78:0/3331139344 shutdown_connections 2023-10-11T09:41:18.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.470+0000 7f8c9cb70700 1 -- 172.21.15.78:0/3331139344 wait complete. 2023-10-11T09:41:18.497 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T09:40:39.862581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T09:40:40.663641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:41:18.498 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:41:18.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.670+0000 7fc8a297f700 1 Processor -- start 2023-10-11T09:41:18.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.670+0000 7fc8a297f700 1 -- start start 2023-10-11T09:41:18.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.674+0000 7fc8a297f700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc89c0b84c0 0x7fc89c0cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:18.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.674+0000 7fc8a297f700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc89c0b8890 0x7fc89c160370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:18.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.674+0000 7fc8a297f700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc89c0cd690 con 0x7fc89c1608b0 2023-10-11T09:41:18.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.674+0000 7fc8a297f700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc89c0cdea0 con 0x7fc89c0b84c0 2023-10-11T09:41:18.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.674+0000 7fc8a297f700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c0d0c10 con 0x7fc89c0b8890 2023-10-11T09:41:18.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.674+0000 7fc8a0f1c700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fc89c1608b0 0x7fc89c0d2fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:55826/0 (socket says 172.21.15.78:55826) 2023-10-11T09:41:18.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.674+0000 7fc8a0f1c700 1 -- 172.21.15.78:0/4122206384 learned_addr learned my addr 172.21.15.78:0/4122206384 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:18.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.674+0000 7fc89bfff700 1 --2- 172.21.15.78:0/4122206384 >> v2:172.21.15.133:3300/0 conn(0x7fc89c0b84c0 0x7fc89c0cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:18.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.674+0000 7fc8937fe700 1 --2- 172.21.15.78:0/4122206384 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc89c0b8890 0x7fc89c160370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:18.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.674+0000 7fc89b7fe700 1 -- 172.21.15.78:0/4122206384 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1935160426 0 0) 0x7fc89c0cd690 con 0x7fc89c1608b0 2023-10-11T09:41:18.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.674+0000 7fc89b7fe700 1 -- 172.21.15.78:0/4122206384 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8880035d0 con 0x7fc89c1608b0 2023-10-11T09:41:18.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.674+0000 7fc8937fe700 1 -- 172.21.15.78:0/4122206384 >> v2:172.21.15.133:3300/0 conn(0x7fc89c0b84c0 msgr2=0x7fc89c0cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:18.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.674+0000 7fc8937fe700 1 --2- 172.21.15.78:0/4122206384 >> v2:172.21.15.133:3300/0 conn(0x7fc89c0b84c0 0x7fc89c0cfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:18.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.674+0000 7fc8937fe700 1 -- 172.21.15.78:0/4122206384 >> v1:172.21.15.78:6789/0 conn(0x7fc89c1608b0 legacy=0x7fc89c0d2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:18.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.674+0000 7fc8937fe700 1 -- 172.21.15.78:0/4122206384 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc89c0d02e0 con 0x7fc89c0b8890 2023-10-11T09:41:18.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.678+0000 7fc8937fe700 1 --2- 172.21.15.78:0/4122206384 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc89c0b8890 0x7fc89c160370 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc884009850 tx=0x7fc884009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=462d30c7e907c065 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:18.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.678+0000 7fc89b7fe700 1 -- 172.21.15.78:0/4122206384 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc884011830 con 0x7fc89c0b8890 2023-10-11T09:41:18.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.678+0000 7fc89b7fe700 1 -- 172.21.15.78:0/4122206384 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc884011990 con 0x7fc89c0b8890 2023-10-11T09:41:18.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.678+0000 7fc89b7fe700 1 -- 172.21.15.78:0/4122206384 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc884011d40 con 0x7fc89c0b8890 2023-10-11T09:41:18.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.678+0000 7fc8a297f700 1 -- 172.21.15.78:0/4122206384 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc89c0b8890 msgr2=0x7fc89c160370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:18.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.678+0000 7fc8a297f700 1 --2- 172.21.15.78:0/4122206384 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc89c0b8890 0x7fc89c160370 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc884009850 tx=0x7fc884009b60 comp rx=0 tx=0).stop 2023-10-11T09:41:18.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.678+0000 7fc8a297f700 1 -- 172.21.15.78:0/4122206384 shutdown_connections 2023-10-11T09:41:18.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.678+0000 7fc8a297f700 1 --2- 172.21.15.78:0/4122206384 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc89c0b8890 0x7fc89c160370 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:18.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.678+0000 7fc8a297f700 1 --2- 172.21.15.78:0/4122206384 >> v2:172.21.15.133:3300/0 conn(0x7fc89c0b84c0 0x7fc89c0cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:18.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.678+0000 7fc8a297f700 1 -- 172.21.15.78:0/4122206384 >> 172.21.15.78:0/4122206384 conn(0x7fc89c06cb50 msgr2=0x7fc89c06cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:18.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.678+0000 7fc8a297f700 1 -- 172.21.15.78:0/4122206384 shutdown_connections 2023-10-11T09:41:18.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.678+0000 7fc8a297f700 1 -- 172.21.15.78:0/4122206384 wait complete. 2023-10-11T09:41:18.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.678+0000 7fc8a297f700 1 Processor -- start 2023-10-11T09:41:18.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.678+0000 7fc8a297f700 1 -- start start 2023-10-11T09:41:18.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.682+0000 7fc8a297f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc89c0b84c0 0x7fc89c1494b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:18.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.682+0000 7fc8a297f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc89c0b8890 0x7fc89c15f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:18.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.682+0000 7fc8a297f700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc89c1608b0 0x7fc89c15e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:18.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.682+0000 7fc8a297f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c1499f0 con 0x7fc89c0b8890 2023-10-11T09:41:18.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.682+0000 7fc8a297f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c1e59f0 con 0x7fc89c0b84c0 2023-10-11T09:41:18.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.682+0000 7fc8a297f700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c1e5c90 con 0x7fc89c1608b0 2023-10-11T09:41:18.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.682+0000 7fc8937fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc89c0b8890 0x7fc89c15f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:18.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.682+0000 7fc8937fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc89c0b8890 0x7fc89c15f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50802/0 (socket says 172.21.15.78:50802) 2023-10-11T09:41:18.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.682+0000 7fc8937fe700 1 -- 172.21.15.78:0/3537991357 learned_addr learned my addr 172.21.15.78:0/3537991357 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:18.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.682+0000 7fc89bfff700 1 --2- 172.21.15.78:0/3537991357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc89c0b84c0 0x7fc89c1494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:18.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.682+0000 7fc8a0f1c700 1 --2- 172.21.15.78:0/3537991357 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc89c1608b0 0x7fc89c15e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:18.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.682+0000 7fc8937fe700 1 -- 172.21.15.78:0/3537991357 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc89c1608b0 msgr2=0x7fc89c15e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:18.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.682+0000 7fc8937fe700 1 --2- 172.21.15.78:0/3537991357 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc89c1608b0 0x7fc89c15e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:18.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.682+0000 7fc8937fe700 1 -- 172.21.15.78:0/3537991357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc89c0b84c0 msgr2=0x7fc89c1494b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:18.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.682+0000 7fc8937fe700 1 --2- 172.21.15.78:0/3537991357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc89c0b84c0 0x7fc89c1494b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:18.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.682+0000 7fc8937fe700 1 -- 172.21.15.78:0/3537991357 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8840094b0 con 0x7fc89c0b8890 2023-10-11T09:41:18.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.682+0000 7fc8937fe700 1 --2- 172.21.15.78:0/3537991357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc89c0b8890 0x7fc89c15f710 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc8840090b0 tx=0x7fc884004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:18.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.682+0000 7fc8997fa700 1 -- 172.21.15.78:0/3537991357 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc884004480 con 0x7fc89c0b8890 2023-10-11T09:41:18.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.682+0000 7fc8997fa700 1 -- 172.21.15.78:0/3537991357 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc88401f070 con 0x7fc89c0b8890 2023-10-11T09:41:18.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.682+0000 7fc8a297f700 1 -- 172.21.15.78:0/3537991357 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc89c1e5f70 con 0x7fc89c0b8890 2023-10-11T09:41:18.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.682+0000 7fc8997fa700 1 -- 172.21.15.78:0/3537991357 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc884003900 con 0x7fc89c0b8890 2023-10-11T09:41:18.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.682+0000 7fc8a297f700 1 -- 172.21.15.78:0/3537991357 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc89c15e440 con 0x7fc89c0b8890 2023-10-11T09:41:18.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.686+0000 7fc8a297f700 1 -- 172.21.15.78:0/3537991357 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc89c15e630 con 0x7fc89c0b8890 2023-10-11T09:41:18.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.686+0000 7fc8997fa700 1 -- 172.21.15.78:0/3537991357 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fc884021030 con 0x7fc89c0b8890 2023-10-11T09:41:18.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.686+0000 7fc8997fa700 1 --2- 172.21.15.78:0/3537991357 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc87c065ed0 0x7fc87c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:18.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.686+0000 7fc8997fa700 1 -- 172.21.15.78:0/3537991357 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc884084b30 con 0x7fc89c0b8890 2023-10-11T09:41:18.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.686+0000 7fc89bfff700 1 --2- 172.21.15.78:0/3537991357 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc87c065ed0 0x7fc87c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:18.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.690+0000 7fc89bfff700 1 --2- 172.21.15.78:0/3537991357 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc87c065ed0 0x7fc87c068380 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc89c1f53e0 tx=0x7fc88c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:18.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.694+0000 7fc8997fa700 1 -- 172.21.15.78:0/3537991357 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fc8840520f0 con 0x7fc89c0b8890 2023-10-11T09:41:18.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.866+0000 7fc8a297f700 1 -- 172.21.15.78:0/3537991357 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc89c08bff0 con 0x7fc89c0b8890 2023-10-11T09:41:18.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.866+0000 7fc8997fa700 1 -- 172.21.15.78:0/3537991357 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc884006ce0 con 0x7fc89c0b8890 2023-10-11T09:41:18.870 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:41:18.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.866+0000 7fc8a297f700 1 -- 172.21.15.78:0/3537991357 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc87c065ed0 msgr2=0x7fc87c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:18.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.866+0000 7fc8a297f700 1 --2- 172.21.15.78:0/3537991357 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc87c065ed0 0x7fc87c068380 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc89c1f53e0 tx=0x7fc88c008040 comp rx=0 tx=0).stop 2023-10-11T09:41:18.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.866+0000 7fc8a297f700 1 -- 172.21.15.78:0/3537991357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc89c0b8890 msgr2=0x7fc89c15f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:18.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.866+0000 7fc8a297f700 1 --2- 172.21.15.78:0/3537991357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc89c0b8890 0x7fc89c15f710 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc8840090b0 tx=0x7fc884004770 comp rx=0 tx=0).stop 2023-10-11T09:41:18.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.870+0000 7fc8a297f700 1 -- 172.21.15.78:0/3537991357 shutdown_connections 2023-10-11T09:41:18.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.870+0000 7fc8a297f700 1 --2- 172.21.15.78:0/3537991357 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc87c065ed0 0x7fc87c068380 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:18.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.870+0000 7fc8a297f700 1 --2- 172.21.15.78:0/3537991357 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc89c1608b0 0x7fc89c15e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:18.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.870+0000 7fc8a297f700 1 --2- 172.21.15.78:0/3537991357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc89c0b8890 0x7fc89c15f710 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:18.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.870+0000 7fc8a297f700 1 --2- 172.21.15.78:0/3537991357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc89c0b84c0 0x7fc89c1494b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:18.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.870+0000 7fc8a297f700 1 -- 172.21.15.78:0/3537991357 >> 172.21.15.78:0/3537991357 conn(0x7fc89c06cb50 msgr2=0x7fc89c0b5f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:18.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.870+0000 7fc8a297f700 1 -- 172.21.15.78:0/3537991357 shutdown_connections 2023-10-11T09:41:18.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:18.870+0000 7fc8a297f700 1 -- 172.21.15.78:0/3537991357 wait complete. 2023-10-11T09:41:18.894 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:41:19.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.066+0000 7f6d8823e700 1 Processor -- start 2023-10-11T09:41:19.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.070+0000 7f6d8823e700 1 -- start start 2023-10-11T09:41:19.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.070+0000 7f6d8823e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f6d8006d080 0x7f6d800cb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:19.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.070+0000 7f6d8823e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d80165570 con 0x7f6d800b1030 2023-10-11T09:41:19.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.070+0000 7f6d8823e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d80164960 con 0x7f6d80160cb0 2023-10-11T09:41:19.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.070+0000 7f6d8823e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d80161080 con 0x7f6d8006d080 2023-10-11T09:41:19.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.070+0000 7f6d857d9700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f6d800b1030 0x7f6d8006ffb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:55830/0 (socket says 172.21.15.78:55830) 2023-10-11T09:41:19.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.070+0000 7f6d857d9700 1 -- 172.21.15.78:0/3332831678 learned_addr learned my addr 172.21.15.78:0/3332831678 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:19.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.070+0000 7f6d867db700 1 --2- 172.21.15.78:0/3332831678 >> v2:172.21.15.78:3300/0 conn(0x7f6d8006d080 0x7f6d800cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:19.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.070+0000 7f6d84fd8700 1 -- 172.21.15.78:0/3332831678 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2014254211 0 0) 0x7f6d80165570 con 0x7f6d800b1030 2023-10-11T09:41:19.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.070+0000 7f6d84fd8700 1 -- 172.21.15.78:0/3332831678 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d6c0035d0 con 0x7f6d800b1030 2023-10-11T09:41:19.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.074+0000 7f6d867db700 1 -- 172.21.15.78:0/3332831678 >> v1:172.21.15.133:6789/0 conn(0x7f6d80160cb0 legacy=0x7f6d800b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:19.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.074+0000 7f6d84fd8700 1 -- 172.21.15.78:0/3332831678 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3923107382 0 0) 0x7f6d80164960 con 0x7f6d80160cb0 2023-10-11T09:41:19.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.074+0000 7f6d867db700 1 -- 172.21.15.78:0/3332831678 >> v1:172.21.15.78:6789/0 conn(0x7f6d800b1030 legacy=0x7f6d8006ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:19.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.074+0000 7f6d867db700 1 -- 172.21.15.78:0/3332831678 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d80164c40 con 0x7f6d8006d080 2023-10-11T09:41:19.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.074+0000 7f6d867db700 1 --2- 172.21.15.78:0/3332831678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d8006d080 0x7f6d800cb100 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6d78009d90 tx=0x7f6d7800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616dc617561e095 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:19.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.074+0000 7f6d84fd8700 1 -- 172.21.15.78:0/3332831678 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7800e070 con 0x7f6d8006d080 2023-10-11T09:41:19.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.074+0000 7f6d84fd8700 1 -- 172.21.15.78:0/3332831678 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d7800bb20 con 0x7f6d8006d080 2023-10-11T09:41:19.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.074+0000 7f6d84fd8700 1 -- 172.21.15.78:0/3332831678 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7800bed0 con 0x7f6d8006d080 2023-10-11T09:41:19.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.074+0000 7f6d8823e700 1 -- 172.21.15.78:0/3332831678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d8006d080 msgr2=0x7f6d800cb100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:19.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.074+0000 7f6d8823e700 1 --2- 172.21.15.78:0/3332831678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d8006d080 0x7f6d800cb100 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6d78009d90 tx=0x7f6d7800b330 comp rx=0 tx=0).stop 2023-10-11T09:41:19.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.074+0000 7f6d8823e700 1 -- 172.21.15.78:0/3332831678 shutdown_connections 2023-10-11T09:41:19.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.074+0000 7f6d8823e700 1 --2- 172.21.15.78:0/3332831678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d8006d080 0x7f6d800cb100 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:19.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.074+0000 7f6d8823e700 1 -- 172.21.15.78:0/3332831678 >> 172.21.15.78:0/3332831678 conn(0x7f6d800cecc0 msgr2=0x7f6d800cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:19.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.074+0000 7f6d8823e700 1 -- 172.21.15.78:0/3332831678 shutdown_connections 2023-10-11T09:41:19.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.074+0000 7f6d8823e700 1 -- 172.21.15.78:0/3332831678 wait complete. 2023-10-11T09:41:19.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.078+0000 7f6d8823e700 1 Processor -- start 2023-10-11T09:41:19.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.078+0000 7f6d8823e700 1 -- start start 2023-10-11T09:41:19.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.078+0000 7f6d8823e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d8006d080 0x7f6d8015c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:19.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.078+0000 7f6d8823e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d800b1030 0x7f6d80154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:19.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.078+0000 7f6d8823e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d80160cb0 0x7f6d80163a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:19.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.078+0000 7f6d8823e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d801f9830 con 0x7f6d8006d080 2023-10-11T09:41:19.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.078+0000 7f6d8823e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8015cea0 con 0x7f6d80160cb0 2023-10-11T09:41:19.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.078+0000 7f6d8823e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d80155460 con 0x7f6d800b1030 2023-10-11T09:41:19.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.078+0000 7f6d85fda700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d8006d080 0x7f6d8015c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:19.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.078+0000 7f6d85fda700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d8006d080 0x7f6d8015c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50816/0 (socket says 172.21.15.78:50816) 2023-10-11T09:41:19.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.078+0000 7f6d85fda700 1 -- 172.21.15.78:0/737384262 learned_addr learned my addr 172.21.15.78:0/737384262 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:19.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.078+0000 7f6d857d9700 1 --2- 172.21.15.78:0/737384262 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d800b1030 0x7f6d80154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:19.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.078+0000 7f6d867db700 1 --2- 172.21.15.78:0/737384262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d80160cb0 0x7f6d80163a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:19.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.078+0000 7f6d85fda700 1 -- 172.21.15.78:0/737384262 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d800b1030 msgr2=0x7f6d80154f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:19.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.078+0000 7f6d85fda700 1 --2- 172.21.15.78:0/737384262 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d800b1030 0x7f6d80154f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:19.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.078+0000 7f6d85fda700 1 -- 172.21.15.78:0/737384262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d80160cb0 msgr2=0x7f6d80163a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:19.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.078+0000 7f6d85fda700 1 --2- 172.21.15.78:0/737384262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d80160cb0 0x7f6d80163a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:19.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.078+0000 7f6d85fda700 1 -- 172.21.15.78:0/737384262 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d78009a70 con 0x7f6d8006d080 2023-10-11T09:41:19.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.078+0000 7f6d867db700 1 --2- 172.21.15.78:0/737384262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d80160cb0 0x7f6d80163a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:19.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.082+0000 7f6d85fda700 1 --2- 172.21.15.78:0/737384262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d8006d080 0x7f6d8015c960 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c00c520 tx=0x7f6d7c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:19.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.082+0000 7f6d767fc700 1 -- 172.21.15.78:0/737384262 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c0054e0 con 0x7f6d8006d080 2023-10-11T09:41:19.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.082+0000 7f6d767fc700 1 -- 172.21.15.78:0/737384262 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d7c0036a0 con 0x7f6d8006d080 2023-10-11T09:41:19.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.082+0000 7f6d8823e700 1 -- 172.21.15.78:0/737384262 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d80163fb0 con 0x7f6d8006d080 2023-10-11T09:41:19.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.082+0000 7f6d767fc700 1 -- 172.21.15.78:0/737384262 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c003a50 con 0x7f6d8006d080 2023-10-11T09:41:19.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.082+0000 7f6d8823e700 1 -- 172.21.15.78:0/737384262 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d801f9d60 con 0x7f6d8006d080 2023-10-11T09:41:19.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.082+0000 7f6d8823e700 1 -- 172.21.15.78:0/737384262 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d800b1c90 con 0x7f6d8006d080 2023-10-11T09:41:19.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.082+0000 7f6d767fc700 1 -- 172.21.15.78:0/737384262 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f6d7c003bb0 con 0x7f6d8006d080 2023-10-11T09:41:19.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.086+0000 7f6d767fc700 1 --2- 172.21.15.78:0/737384262 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6d64065ed0 0x7f6d64068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:19.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.086+0000 7f6d767fc700 1 -- 172.21.15.78:0/737384262 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6d7c082d10 con 0x7f6d8006d080 2023-10-11T09:41:19.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.086+0000 7f6d857d9700 1 --2- 172.21.15.78:0/737384262 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6d64065ed0 0x7f6d64068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:19.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.086+0000 7f6d857d9700 1 --2- 172.21.15.78:0/737384262 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6d64065ed0 0x7f6d64068380 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6d8015f940 tx=0x7f6d70009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:19.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.090+0000 7f6d767fc700 1 -- 172.21.15.78:0/737384262 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f6d800b1c90 con 0x7f6d8006d080 2023-10-11T09:41:19.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.258+0000 7f6d8823e700 1 -- 172.21.15.78:0/737384262 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6d800b1c90 con 0x7f6d8006d080 2023-10-11T09:41:19.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.258+0000 7f6d767fc700 1 -- 172.21.15.78:0/737384262 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6d800b1c90 con 0x7f6d8006d080 2023-10-11T09:41:19.265 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:41:19.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.262+0000 7f6d8823e700 1 -- 172.21.15.78:0/737384262 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6d64065ed0 msgr2=0x7f6d64068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:19.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.262+0000 7f6d8823e700 1 --2- 172.21.15.78:0/737384262 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6d64065ed0 0x7f6d64068380 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6d8015f940 tx=0x7f6d70009040 comp rx=0 tx=0).stop 2023-10-11T09:41:19.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.262+0000 7f6d8823e700 1 -- 172.21.15.78:0/737384262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d8006d080 msgr2=0x7f6d8015c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:19.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.262+0000 7f6d8823e700 1 --2- 172.21.15.78:0/737384262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d8006d080 0x7f6d8015c960 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c00c520 tx=0x7f6d7c00c8e0 comp rx=0 tx=0).stop 2023-10-11T09:41:19.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.262+0000 7f6d8823e700 1 -- 172.21.15.78:0/737384262 shutdown_connections 2023-10-11T09:41:19.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.262+0000 7f6d8823e700 1 --2- 172.21.15.78:0/737384262 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6d64065ed0 0x7f6d64068380 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:19.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.262+0000 7f6d8823e700 1 --2- 172.21.15.78:0/737384262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d80160cb0 0x7f6d80163a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:19.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.262+0000 7f6d8823e700 1 --2- 172.21.15.78:0/737384262 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d800b1030 0x7f6d80154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:19.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.262+0000 7f6d8823e700 1 --2- 172.21.15.78:0/737384262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d8006d080 0x7f6d8015c960 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:19.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.262+0000 7f6d8823e700 1 -- 172.21.15.78:0/737384262 >> 172.21.15.78:0/737384262 conn(0x7f6d800cecc0 msgr2=0x7f6d800cc800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:19.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.262+0000 7f6d8823e700 1 -- 172.21.15.78:0/737384262 shutdown_connections 2023-10-11T09:41:19.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.262+0000 7f6d8823e700 1 -- 172.21.15.78:0/737384262 wait complete. 2023-10-11T09:41:19.288 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T09:41:19.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.462+0000 7f29acae7700 1 Processor -- start 2023-10-11T09:41:19.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.462+0000 7f29acae7700 1 -- start start 2023-10-11T09:41:19.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.466+0000 7f29acae7700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f29a81596b0 0x7f29a80c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:19.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.466+0000 7f29acae7700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f29a806cf70 0x7f29a806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:19.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.466+0000 7f29acae7700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29a80c9280 con 0x7f29a815db30 2023-10-11T09:41:19.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.466+0000 7f29acae7700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29a806d930 con 0x7f29a81596b0 2023-10-11T09:41:19.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.466+0000 7f29acae7700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29a80cd430 con 0x7f29a806cf70 2023-10-11T09:41:19.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.466+0000 7f29a659c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f29a81596b0 0x7f29a80c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:19.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.466+0000 7f29a659c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f29a81596b0 0x7f29a80c9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50818/0 (socket says 172.21.15.78:50818) 2023-10-11T09:41:19.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.466+0000 7f29a659c700 1 -- 172.21.15.78:0/3993170360 learned_addr learned my addr 172.21.15.78:0/3993170360 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:19.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.466+0000 7f29a6d9d700 1 --2- 172.21.15.78:0/3993170360 >> v2:172.21.15.133:3300/0 conn(0x7f29a806cf70 0x7f29a806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:19.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.466+0000 7f29a659c700 1 -- 172.21.15.78:0/3993170360 >> v2:172.21.15.133:3300/0 conn(0x7f29a806cf70 msgr2=0x7f29a806d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:19.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.466+0000 7f29a659c700 1 --2- 172.21.15.78:0/3993170360 >> v2:172.21.15.133:3300/0 conn(0x7f29a806cf70 0x7f29a806d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:19.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.466+0000 7f29a659c700 1 -- 172.21.15.78:0/3993170360 >> v1:172.21.15.133:6789/0 conn(0x7f29a815db30 legacy=0x7f29a80ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:19.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.466+0000 7f29a659c700 1 -- 172.21.15.78:0/3993170360 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a815df80 con 0x7f29a81596b0 2023-10-11T09:41:19.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.466+0000 7f29a659c700 1 --2- 172.21.15.78:0/3993170360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a81596b0 0x7f29a80c9b90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f29a806b020 tx=0x7f299800be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8bc13cb84829c30 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:19.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.466+0000 7f29a559a700 1 -- 172.21.15.78:0/3993170360 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299800dd30 con 0x7f29a81596b0 2023-10-11T09:41:19.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.466+0000 7f29a559a700 1 -- 172.21.15.78:0/3993170360 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f299800de90 con 0x7f29a81596b0 2023-10-11T09:41:19.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.466+0000 7f29a559a700 1 -- 172.21.15.78:0/3993170360 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2998010690 con 0x7f29a81596b0 2023-10-11T09:41:19.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.466+0000 7f29acae7700 1 -- 172.21.15.78:0/3993170360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a81596b0 msgr2=0x7f29a80c9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:19.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.466+0000 7f29acae7700 1 --2- 172.21.15.78:0/3993170360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a81596b0 0x7f29a80c9b90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f29a806b020 tx=0x7f299800be90 comp rx=0 tx=0).stop 2023-10-11T09:41:19.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.466+0000 7f29acae7700 1 -- 172.21.15.78:0/3993170360 shutdown_connections 2023-10-11T09:41:19.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.466+0000 7f29acae7700 1 --2- 172.21.15.78:0/3993170360 >> v2:172.21.15.133:3300/0 conn(0x7f29a806cf70 0x7f29a806d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:19.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.466+0000 7f29acae7700 1 --2- 172.21.15.78:0/3993170360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a81596b0 0x7f29a80c9b90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:19.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.466+0000 7f29acae7700 1 -- 172.21.15.78:0/3993170360 >> 172.21.15.78:0/3993170360 conn(0x7f29a80cecc0 msgr2=0x7f29a80cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:19.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.466+0000 7f29acae7700 1 -- 172.21.15.78:0/3993170360 shutdown_connections 2023-10-11T09:41:19.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.466+0000 7f29acae7700 1 -- 172.21.15.78:0/3993170360 wait complete. 2023-10-11T09:41:19.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.470+0000 7f29acae7700 1 Processor -- start 2023-10-11T09:41:19.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.470+0000 7f29acae7700 1 -- start start 2023-10-11T09:41:19.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.470+0000 7f29acae7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a806cf70 0x7f29a8145750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:19.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.470+0000 7f29acae7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29a81596b0 0x7f29a8156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:19.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.470+0000 7f29acae7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29a815db30 0x7f29a8157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:19.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.470+0000 7f29acae7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a80af7b0 con 0x7f29a806cf70 2023-10-11T09:41:19.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.470+0000 7f29acae7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a80b1fe0 con 0x7f29a81596b0 2023-10-11T09:41:19.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.474+0000 7f29acae7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a80ae8e0 con 0x7f29a815db30 2023-10-11T09:41:19.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.474+0000 7f29a659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a806cf70 0x7f29a8145750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:19.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.474+0000 7f29a659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a806cf70 0x7f29a8145750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50832/0 (socket says 172.21.15.78:50832) 2023-10-11T09:41:19.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.474+0000 7f29a659c700 1 -- 172.21.15.78:0/864607509 learned_addr learned my addr 172.21.15.78:0/864607509 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:19.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.474+0000 7f29a6d9d700 1 --2- 172.21.15.78:0/864607509 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29a815db30 0x7f29a8157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:19.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.474+0000 7f29a5d9b700 1 --2- 172.21.15.78:0/864607509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29a81596b0 0x7f29a8156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:19.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.474+0000 7f29a659c700 1 -- 172.21.15.78:0/864607509 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29a815db30 msgr2=0x7f29a8157090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:19.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.474+0000 7f29a659c700 1 --2- 172.21.15.78:0/864607509 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29a815db30 0x7f29a8157090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:19.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.474+0000 7f29a659c700 1 -- 172.21.15.78:0/864607509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29a81596b0 msgr2=0x7f29a8156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:19.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.474+0000 7f29a659c700 1 --2- 172.21.15.78:0/864607509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29a81596b0 0x7f29a8156b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:19.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.474+0000 7f29a659c700 1 -- 172.21.15.78:0/864607509 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f299800b810 con 0x7f29a806cf70 2023-10-11T09:41:19.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.474+0000 7f29a5d9b700 1 --2- 172.21.15.78:0/864607509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29a81596b0 0x7f29a8156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:19.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.474+0000 7f29a659c700 1 --2- 172.21.15.78:0/864607509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a806cf70 0x7f29a8145750 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f29a806f8e0 tx=0x7f2998004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:19.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.474+0000 7f2996ffd700 1 -- 172.21.15.78:0/864607509 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2998003c90 con 0x7f29a806cf70 2023-10-11T09:41:19.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.474+0000 7f2996ffd700 1 -- 172.21.15.78:0/864607509 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2998003e10 con 0x7f29a806cf70 2023-10-11T09:41:19.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.474+0000 7f29acae7700 1 -- 172.21.15.78:0/864607509 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29a8145c90 con 0x7f29a806cf70 2023-10-11T09:41:19.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.474+0000 7f2996ffd700 1 -- 172.21.15.78:0/864607509 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29980106b0 con 0x7f29a806cf70 2023-10-11T09:41:19.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.474+0000 7f29acae7700 1 -- 172.21.15.78:0/864607509 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a80aee10 con 0x7f29a806cf70 2023-10-11T09:41:19.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.478+0000 7f29acae7700 1 -- 172.21.15.78:0/864607509 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29a806eb80 con 0x7f29a806cf70 2023-10-11T09:41:19.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.478+0000 7f2996ffd700 1 -- 172.21.15.78:0/864607509 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f2998019400 con 0x7f29a806cf70 2023-10-11T09:41:19.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.478+0000 7f2996ffd700 1 --2- 172.21.15.78:0/864607509 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2988065ed0 0x7f2988068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:19.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.478+0000 7f2996ffd700 1 -- 172.21.15.78:0/864607509 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2998085280 con 0x7f29a806cf70 2023-10-11T09:41:19.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.478+0000 7f29a5d9b700 1 --2- 172.21.15.78:0/864607509 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2988065ed0 0x7f2988068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:19.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.482+0000 7f29a5d9b700 1 --2- 172.21.15.78:0/864607509 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2988065ed0 0x7f2988068380 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f29a8071920 tx=0x7f299c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:19.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.486+0000 7f2996ffd700 1 -- 172.21.15.78:0/864607509 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f29980529e0 con 0x7f29a806cf70 2023-10-11T09:41:19.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.662+0000 7f29acae7700 1 -- 172.21.15.78:0/864607509 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f29a815c900 con 0x7f29a806cf70 2023-10-11T09:41:19.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.662+0000 7f2996ffd700 1 -- 172.21.15.78:0/864607509 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2998056000 con 0x7f29a806cf70 2023-10-11T09:41:19.667 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:41:19.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.666+0000 7f29acae7700 1 -- 172.21.15.78:0/864607509 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2988065ed0 msgr2=0x7f2988068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:19.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.666+0000 7f29acae7700 1 --2- 172.21.15.78:0/864607509 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2988065ed0 0x7f2988068380 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f29a8071920 tx=0x7f299c009040 comp rx=0 tx=0).stop 2023-10-11T09:41:19.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.666+0000 7f29acae7700 1 -- 172.21.15.78:0/864607509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a806cf70 msgr2=0x7f29a8145750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:19.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.666+0000 7f29acae7700 1 --2- 172.21.15.78:0/864607509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a806cf70 0x7f29a8145750 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f29a806f8e0 tx=0x7f2998004770 comp rx=0 tx=0).stop 2023-10-11T09:41:19.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.666+0000 7f29acae7700 1 -- 172.21.15.78:0/864607509 shutdown_connections 2023-10-11T09:41:19.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.666+0000 7f29acae7700 1 --2- 172.21.15.78:0/864607509 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2988065ed0 0x7f2988068380 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:19.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.666+0000 7f29acae7700 1 --2- 172.21.15.78:0/864607509 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29a815db30 0x7f29a8157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:19.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.666+0000 7f29acae7700 1 --2- 172.21.15.78:0/864607509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29a81596b0 0x7f29a8156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:19.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.666+0000 7f29acae7700 1 --2- 172.21.15.78:0/864607509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a806cf70 0x7f29a8145750 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:19.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.666+0000 7f29acae7700 1 -- 172.21.15.78:0/864607509 >> 172.21.15.78:0/864607509 conn(0x7f29a80cecc0 msgr2=0x7f29a814f980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:19.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.666+0000 7f29acae7700 1 -- 172.21.15.78:0/864607509 shutdown_connections 2023-10-11T09:41:19.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.666+0000 7f29acae7700 1 -- 172.21.15.78:0/864607509 wait complete. 2023-10-11T09:41:19.691 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T09:41:19.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.854+0000 7f5f7933d700 1 Processor -- start 2023-10-11T09:41:19.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.858+0000 7f5f7933d700 1 -- start start 2023-10-11T09:41:19.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.858+0000 7f5f7933d700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5f74146fe0 0x7f5f7406bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:19.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.858+0000 7f5f7933d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5f74161d90 0x7f5f740b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:19.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.858+0000 7f5f7933d700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f7406ccb0 con 0x7f5f740b3520 2023-10-11T09:41:19.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.858+0000 7f5f7933d700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f7406c410 con 0x7f5f74146fe0 2023-10-11T09:41:19.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.858+0000 7f5f7933d700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7406c6f0 con 0x7f5f74161d90 2023-10-11T09:41:19.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.862+0000 7f5f737fe700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f5f740b3520 0x7f5f741647e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:55846/0 (socket says 172.21.15.78:55846) 2023-10-11T09:41:19.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.862+0000 7f5f737fe700 1 -- 172.21.15.78:0/2575169662 learned_addr learned my addr 172.21.15.78:0/2575169662 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:19.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.862+0000 7f5f72ffd700 1 --2- 172.21.15.78:0/2575169662 >> v2:172.21.15.133:3300/0 conn(0x7f5f74146fe0 0x7f5f7406bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:19.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.862+0000 7f5f6a7fc700 1 --2- 172.21.15.78:0/2575169662 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5f74161d90 0x7f5f740b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:19.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.862+0000 7f5f727fc700 1 -- 172.21.15.78:0/2575169662 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3619325953 0 0) 0x7f5f6c001110 con 0x7f5f740b3520 2023-10-11T09:41:19.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.862+0000 7f5f727fc700 1 -- 172.21.15.78:0/2575169662 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f600035d0 con 0x7f5f740b3520 2023-10-11T09:41:19.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.862+0000 7f5f6a7fc700 1 -- 172.21.15.78:0/2575169662 >> v2:172.21.15.133:3300/0 conn(0x7f5f74146fe0 msgr2=0x7f5f7406bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:19.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.862+0000 7f5f6a7fc700 1 --2- 172.21.15.78:0/2575169662 >> v2:172.21.15.133:3300/0 conn(0x7f5f74146fe0 0x7f5f7406bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:19.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.862+0000 7f5f6a7fc700 1 -- 172.21.15.78:0/2575169662 >> v1:172.21.15.78:6789/0 conn(0x7f5f740b3520 legacy=0x7f5f741647e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:19.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.862+0000 7f5f6a7fc700 1 -- 172.21.15.78:0/2575169662 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7406c9d0 con 0x7f5f74161d90 2023-10-11T09:41:19.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.862+0000 7f5f6a7fc700 1 --2- 172.21.15.78:0/2575169662 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5f74161d90 0x7f5f740b2fe0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c009850 tx=0x7f5f5c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b77df11e7ce67af server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:19.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.862+0000 7f5f727fc700 1 -- 172.21.15.78:0/2575169662 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c011830 con 0x7f5f74161d90 2023-10-11T09:41:19.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.862+0000 7f5f727fc700 1 -- 172.21.15.78:0/2575169662 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f5c011990 con 0x7f5f74161d90 2023-10-11T09:41:19.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.862+0000 7f5f727fc700 1 -- 172.21.15.78:0/2575169662 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c011d40 con 0x7f5f74161d90 2023-10-11T09:41:19.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.862+0000 7f5f7933d700 1 -- 172.21.15.78:0/2575169662 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5f74161d90 msgr2=0x7f5f740b2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:19.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.862+0000 7f5f7933d700 1 --2- 172.21.15.78:0/2575169662 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5f74161d90 0x7f5f740b2fe0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c009850 tx=0x7f5f5c009b60 comp rx=0 tx=0).stop 2023-10-11T09:41:19.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.862+0000 7f5f7933d700 1 -- 172.21.15.78:0/2575169662 shutdown_connections 2023-10-11T09:41:19.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.862+0000 7f5f7933d700 1 --2- 172.21.15.78:0/2575169662 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5f74161d90 0x7f5f740b2fe0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:19.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.862+0000 7f5f7933d700 1 --2- 172.21.15.78:0/2575169662 >> v2:172.21.15.133:3300/0 conn(0x7f5f74146fe0 0x7f5f7406bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:19.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.862+0000 7f5f7933d700 1 -- 172.21.15.78:0/2575169662 >> 172.21.15.78:0/2575169662 conn(0x7f5f7406a060 msgr2=0x7f5f7406a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:19.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.862+0000 7f5f7933d700 1 -- 172.21.15.78:0/2575169662 shutdown_connections 2023-10-11T09:41:19.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.866+0000 7f5f7933d700 1 -- 172.21.15.78:0/2575169662 wait complete. 2023-10-11T09:41:19.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.866+0000 7f5f7933d700 1 Processor -- start 2023-10-11T09:41:19.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.866+0000 7f5f7933d700 1 -- start start 2023-10-11T09:41:19.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.870+0000 7f5f7933d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5f740b3520 0x7f5f740adf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:19.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.870+0000 7f5f7933d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f74146fe0 0x7f5f7415f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:19.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.870+0000 7f5f7933d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f74161d90 0x7f5f741f6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:19.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.870+0000 7f5f7933d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f741f72e0 con 0x7f5f74161d90 2023-10-11T09:41:19.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.870+0000 7f5f7933d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7415fc40 con 0x7f5f74146fe0 2023-10-11T09:41:19.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.870+0000 7f5f7933d700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f741f67c0 con 0x7f5f740b3520 2023-10-11T09:41:19.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.870+0000 7f5f737fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f74161d90 0x7f5f741f6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:19.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.870+0000 7f5f737fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f74161d90 0x7f5f741f6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50836/0 (socket says 172.21.15.78:50836) 2023-10-11T09:41:19.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.870+0000 7f5f737fe700 1 -- 172.21.15.78:0/4070703116 learned_addr learned my addr 172.21.15.78:0/4070703116 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:19.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.870+0000 7f5f72ffd700 1 --2- 172.21.15.78:0/4070703116 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5f740b3520 0x7f5f740adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:19.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.870+0000 7f5f737fe700 1 -- 172.21.15.78:0/4070703116 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5f740b3520 msgr2=0x7f5f740adf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:19.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.870+0000 7f5f6a7fc700 1 --2- 172.21.15.78:0/4070703116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f74146fe0 0x7f5f7415f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:19.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.870+0000 7f5f737fe700 1 --2- 172.21.15.78:0/4070703116 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5f740b3520 0x7f5f740adf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:19.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.870+0000 7f5f737fe700 1 -- 172.21.15.78:0/4070703116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f74146fe0 msgr2=0x7f5f7415f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:19.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.870+0000 7f5f737fe700 1 --2- 172.21.15.78:0/4070703116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f74146fe0 0x7f5f7415f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:19.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.870+0000 7f5f737fe700 1 -- 172.21.15.78:0/4070703116 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f5c0094b0 con 0x7f5f74161d90 2023-10-11T09:41:19.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.870+0000 7f5f6a7fc700 1 --2- 172.21.15.78:0/4070703116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f74146fe0 0x7f5f7415f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:41:19.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.870+0000 7f5f737fe700 1 --2- 172.21.15.78:0/4070703116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f74161d90 0x7f5f741f6280 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c00c520 tx=0x7f5f6c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:19.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.870+0000 7f5f6bfff700 1 -- 172.21.15.78:0/4070703116 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c005d90 con 0x7f5f74161d90 2023-10-11T09:41:19.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.870+0000 7f5f6bfff700 1 -- 172.21.15.78:0/4070703116 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6c0070d0 con 0x7f5f74161d90 2023-10-11T09:41:19.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.870+0000 7f5f7933d700 1 -- 172.21.15.78:0/4070703116 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f741f6a40 con 0x7f5f74161d90 2023-10-11T09:41:19.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.870+0000 7f5f6bfff700 1 -- 172.21.15.78:0/4070703116 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c007460 con 0x7f5f74161d90 2023-10-11T09:41:19.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.870+0000 7f5f7933d700 1 -- 172.21.15.78:0/4070703116 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f741f7000 con 0x7f5f74161d90 2023-10-11T09:41:19.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.874+0000 7f5f7933d700 1 -- 172.21.15.78:0/4070703116 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f74065d40 con 0x7f5f74161d90 2023-10-11T09:41:19.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.874+0000 7f5f6bfff700 1 -- 172.21.15.78:0/4070703116 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f5f6c0075c0 con 0x7f5f74161d90 2023-10-11T09:41:19.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.874+0000 7f5f6bfff700 1 --2- 172.21.15.78:0/4070703116 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5f54065ed0 0x7f5f54068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:19.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.874+0000 7f5f6bfff700 1 -- 172.21.15.78:0/4070703116 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5f6c082d00 con 0x7f5f74161d90 2023-10-11T09:41:19.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.878+0000 7f5f72ffd700 1 --2- 172.21.15.78:0/4070703116 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5f54065ed0 0x7f5f54068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:19.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.878+0000 7f5f72ffd700 1 --2- 172.21.15.78:0/4070703116 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5f54065ed0 0x7f5f54068380 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5f74071b50 tx=0x7f5f6400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:19.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:19.882+0000 7f5f6bfff700 1 -- 172.21.15.78:0/4070703116 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f5f6c050230 con 0x7f5f74161d90 2023-10-11T09:41:20.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.058+0000 7f5f7933d700 1 -- 172.21.15.78:0/4070703116 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5f7415fd80 con 0x7f5f74161d90 2023-10-11T09:41:20.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.058+0000 7f5f6bfff700 1 -- 172.21.15.78:0/4070703116 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5f6c007dd0 con 0x7f5f74161d90 2023-10-11T09:41:20.063 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2023-10-11T09:41:20.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.058+0000 7f5f7933d700 1 -- 172.21.15.78:0/4070703116 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5f54065ed0 msgr2=0x7f5f54068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:20.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.058+0000 7f5f7933d700 1 --2- 172.21.15.78:0/4070703116 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5f54065ed0 0x7f5f54068380 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5f74071b50 tx=0x7f5f6400b040 comp rx=0 tx=0).stop 2023-10-11T09:41:20.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.058+0000 7f5f7933d700 1 -- 172.21.15.78:0/4070703116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f74161d90 msgr2=0x7f5f741f6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:20.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.058+0000 7f5f7933d700 1 --2- 172.21.15.78:0/4070703116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f74161d90 0x7f5f741f6280 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c00c520 tx=0x7f5f6c00c8e0 comp rx=0 tx=0).stop 2023-10-11T09:41:20.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.062+0000 7f5f7933d700 1 -- 172.21.15.78:0/4070703116 shutdown_connections 2023-10-11T09:41:20.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.062+0000 7f5f7933d700 1 --2- 172.21.15.78:0/4070703116 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5f54065ed0 0x7f5f54068380 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:20.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.062+0000 7f5f7933d700 1 --2- 172.21.15.78:0/4070703116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f74161d90 0x7f5f741f6280 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:20.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.062+0000 7f5f7933d700 1 --2- 172.21.15.78:0/4070703116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f74146fe0 0x7f5f7415f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:20.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.062+0000 7f5f7933d700 1 --2- 172.21.15.78:0/4070703116 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5f740b3520 0x7f5f740adf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:20.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.062+0000 7f5f7933d700 1 -- 172.21.15.78:0/4070703116 >> 172.21.15.78:0/4070703116 conn(0x7f5f7406a060 msgr2=0x7f5f741609e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:20.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.062+0000 7f5f7933d700 1 -- 172.21.15.78:0/4070703116 shutdown_connections 2023-10-11T09:41:20.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.062+0000 7f5f7933d700 1 -- 172.21.15.78:0/4070703116 wait complete. 2023-10-11T09:41:20.124 INFO:tasks.cephfs_test_runner:Starting test: test_extra_data_pool_rebuild_few_files (tasks.cephfs.test_data_scan.TestDataScan) 2023-10-11T09:41:20.125 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_few_files' 2023-10-11T09:41:20.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.298+0000 7fe9669fc700 1 Processor -- start 2023-10-11T09:41:20.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.298+0000 7fe9669fc700 1 -- start start 2023-10-11T09:41:20.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.298+0000 7fe9669fc700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe960142770 0x7fe96006c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:20.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.298+0000 7fe9669fc700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9600ce3b0 con 0x7fe96006ff00 2023-10-11T09:41:20.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.298+0000 7fe9669fc700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9600cc870 con 0x7fe9600cd9d0 2023-10-11T09:41:20.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.298+0000 7fe9669fc700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9600cce30 con 0x7fe960142770 2023-10-11T09:41:20.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.298+0000 7fe9577fe700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fe96006ff00 0x7fe9600cd2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:55860/0 (socket says 172.21.15.78:55860) 2023-10-11T09:41:20.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.298+0000 7fe9577fe700 1 -- 172.21.15.78:0/1743284685 learned_addr learned my addr 172.21.15.78:0/1743284685 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:20.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.298+0000 7fe95ffff700 1 --2- 172.21.15.78:0/1743284685 >> v2:172.21.15.78:3300/0 conn(0x7fe960142770 0x7fe96006c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:20.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.298+0000 7fe95f7fe700 1 -- 172.21.15.78:0/1743284685 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1938660482 0 0) 0x7fe9600ce3b0 con 0x7fe96006ff00 2023-10-11T09:41:20.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.302+0000 7fe95f7fe700 1 -- 172.21.15.78:0/1743284685 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe94c0035d0 con 0x7fe96006ff00 2023-10-11T09:41:20.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.302+0000 7fe95ffff700 1 -- 172.21.15.78:0/1743284685 >> v1:172.21.15.133:6789/0 conn(0x7fe9600cd9d0 legacy=0x7fe9600cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:20.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.302+0000 7fe95f7fe700 1 -- 172.21.15.78:0/1743284685 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2297214308 0 0) 0x7fe94c0035d0 con 0x7fe96006ff00 2023-10-11T09:41:20.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.302+0000 7fe95ffff700 1 -- 172.21.15.78:0/1743284685 >> v1:172.21.15.78:6789/0 conn(0x7fe96006ff00 legacy=0x7fe9600cd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:20.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.302+0000 7fe95ffff700 1 -- 172.21.15.78:0/1743284685 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9600ce600 con 0x7fe960142770 2023-10-11T09:41:20.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.302+0000 7fe95ffff700 1 --2- 172.21.15.78:0/1743284685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe960142770 0x7fe96006c870 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe950009d90 tx=0x7fe95000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3400a7f25e225c3d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:20.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.302+0000 7fe95f7fe700 1 -- 172.21.15.78:0/1743284685 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95000e070 con 0x7fe960142770 2023-10-11T09:41:20.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.302+0000 7fe95f7fe700 1 -- 172.21.15.78:0/1743284685 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe950003a40 con 0x7fe960142770 2023-10-11T09:41:20.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.302+0000 7fe95f7fe700 1 -- 172.21.15.78:0/1743284685 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe948001670 con 0x7fe960142770 2023-10-11T09:41:20.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.302+0000 7fe9669fc700 1 -- 172.21.15.78:0/1743284685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe960142770 msgr2=0x7fe96006c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:20.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.302+0000 7fe9669fc700 1 --2- 172.21.15.78:0/1743284685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe960142770 0x7fe96006c870 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe950009d90 tx=0x7fe95000b330 comp rx=0 tx=0).stop 2023-10-11T09:41:20.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.302+0000 7fe9669fc700 1 -- 172.21.15.78:0/1743284685 shutdown_connections 2023-10-11T09:41:20.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.302+0000 7fe9669fc700 1 --2- 172.21.15.78:0/1743284685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe960142770 0x7fe96006c870 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:20.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.302+0000 7fe9669fc700 1 -- 172.21.15.78:0/1743284685 >> 172.21.15.78:0/1743284685 conn(0x7fe96006aca0 msgr2=0x7fe9600b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:20.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.302+0000 7fe9669fc700 1 -- 172.21.15.78:0/1743284685 shutdown_connections 2023-10-11T09:41:20.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.302+0000 7fe9669fc700 1 -- 172.21.15.78:0/1743284685 wait complete. 2023-10-11T09:41:20.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.306+0000 7fe9669fc700 1 Processor -- start 2023-10-11T09:41:20.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.306+0000 7fe9669fc700 1 -- start start 2023-10-11T09:41:20.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.306+0000 7fe9669fc700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe96006ff00 0x7fe96013b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:20.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.306+0000 7fe9669fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9600cd9d0 0x7fe960139420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:20.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.306+0000 7fe9669fc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe960142770 0x7fe960152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:20.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.306+0000 7fe9669fc700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe960139960 con 0x7fe9600cd9d0 2023-10-11T09:41:20.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.306+0000 7fe9669fc700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe960138660 con 0x7fe960142770 2023-10-11T09:41:20.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.306+0000 7fe9669fc700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe960138940 con 0x7fe96006ff00 2023-10-11T09:41:20.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.306+0000 7fe9577fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9600cd9d0 0x7fe960139420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:20.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.310+0000 7fe9577fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9600cd9d0 0x7fe960139420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50862/0 (socket says 172.21.15.78:50862) 2023-10-11T09:41:20.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.310+0000 7fe9577fe700 1 -- 172.21.15.78:0/3239731419 learned_addr learned my addr 172.21.15.78:0/3239731419 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:20.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.310+0000 7fe95ffff700 1 --2- 172.21.15.78:0/3239731419 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe96006ff00 0x7fe96013b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:20.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.310+0000 7fe964f99700 1 --2- 172.21.15.78:0/3239731419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe960142770 0x7fe960152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:20.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.310+0000 7fe9577fe700 1 -- 172.21.15.78:0/3239731419 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe96006ff00 msgr2=0x7fe96013b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:20.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.310+0000 7fe9577fe700 1 --2- 172.21.15.78:0/3239731419 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe96006ff00 0x7fe96013b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:20.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.310+0000 7fe9577fe700 1 -- 172.21.15.78:0/3239731419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe960142770 msgr2=0x7fe960152740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:20.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.310+0000 7fe9577fe700 1 --2- 172.21.15.78:0/3239731419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe960142770 0x7fe960152740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:20.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.310+0000 7fe9577fe700 1 -- 172.21.15.78:0/3239731419 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe950009a70 con 0x7fe9600cd9d0 2023-10-11T09:41:20.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.310+0000 7fe9577fe700 1 --2- 172.21.15.78:0/3239731419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9600cd9d0 0x7fe960139420 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe94800c780 tx=0x7fe94800ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:20.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.310+0000 7fe95d7fa700 1 -- 172.21.15.78:0/3239731419 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe948004ca0 con 0x7fe9600cd9d0 2023-10-11T09:41:20.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.310+0000 7fe95d7fa700 1 -- 172.21.15.78:0/3239731419 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe948004e20 con 0x7fe9600cd9d0 2023-10-11T09:41:20.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.310+0000 7fe95d7fa700 1 -- 172.21.15.78:0/3239731419 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe948007310 con 0x7fe9600cd9d0 2023-10-11T09:41:20.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.310+0000 7fe9669fc700 1 -- 172.21.15.78:0/3239731419 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe960138c20 con 0x7fe9600cd9d0 2023-10-11T09:41:20.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.310+0000 7fe9669fc700 1 -- 172.21.15.78:0/3239731419 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe960152c80 con 0x7fe9600cd9d0 2023-10-11T09:41:20.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.310+0000 7fe9669fc700 1 -- 172.21.15.78:0/3239731419 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe960139140 con 0x7fe9600cd9d0 2023-10-11T09:41:20.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.314+0000 7fe95d7fa700 1 -- 172.21.15.78:0/3239731419 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fe948007470 con 0x7fe9600cd9d0 2023-10-11T09:41:20.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.314+0000 7fe95d7fa700 1 --2- 172.21.15.78:0/3239731419 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe940065f00 0x7fe9400683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:20.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.314+0000 7fe95d7fa700 1 -- 172.21.15.78:0/3239731419 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe9480834e0 con 0x7fe9600cd9d0 2023-10-11T09:41:20.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.314+0000 7fe95ffff700 1 --2- 172.21.15.78:0/3239731419 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe940065f00 0x7fe9400683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:20.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.314+0000 7fe95ffff700 1 --2- 172.21.15.78:0/3239731419 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe940065f00 0x7fe9400683b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe950000f80 tx=0x7fe950010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:20.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.318+0000 7fe95d7fa700 1 -- 172.21.15.78:0/3239731419 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fe960139140 con 0x7fe9600cd9d0 2023-10-11T09:41:20.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:20.566+0000 7fe9669fc700 1 -- 172.21.15.78:0/3239731419 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_few_files"]} v 0) v1 -- 0x7fe960139140 con 0x7fe9600cd9d0 2023-10-11T09:41:21.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.478+0000 7fe95d7fa700 1 -- 172.21.15.78:0/3239731419 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_few_files"]}]=0 v61) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe960139140 con 0x7fe9600cd9d0 2023-10-11T09:41:21.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.482+0000 7fe9669fc700 1 -- 172.21.15.78:0/3239731419 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe940065f00 msgr2=0x7fe9400683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:21.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.482+0000 7fe9669fc700 1 --2- 172.21.15.78:0/3239731419 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe940065f00 0x7fe9400683b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe950000f80 tx=0x7fe950010000 comp rx=0 tx=0).stop 2023-10-11T09:41:21.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.482+0000 7fe9669fc700 1 -- 172.21.15.78:0/3239731419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9600cd9d0 msgr2=0x7fe960139420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:21.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.482+0000 7fe9669fc700 1 --2- 172.21.15.78:0/3239731419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9600cd9d0 0x7fe960139420 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe94800c780 tx=0x7fe94800ca90 comp rx=0 tx=0).stop 2023-10-11T09:41:21.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.482+0000 7fe9669fc700 1 -- 172.21.15.78:0/3239731419 shutdown_connections 2023-10-11T09:41:21.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.482+0000 7fe9669fc700 1 --2- 172.21.15.78:0/3239731419 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe940065f00 0x7fe9400683b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:21.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.482+0000 7fe9669fc700 1 --2- 172.21.15.78:0/3239731419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe960142770 0x7fe960152740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:21.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.482+0000 7fe9669fc700 1 --2- 172.21.15.78:0/3239731419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9600cd9d0 0x7fe960139420 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:21.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.482+0000 7fe9669fc700 1 --2- 172.21.15.78:0/3239731419 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe96006ff00 0x7fe96013b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:21.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.482+0000 7fe9669fc700 1 -- 172.21.15.78:0/3239731419 >> 172.21.15.78:0/3239731419 conn(0x7fe96006aca0 msgr2=0x7fe96015d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:21.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.486+0000 7fe9669fc700 1 -- 172.21.15.78:0/3239731419 shutdown_connections 2023-10-11T09:41:21.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.486+0000 7fe9669fc700 1 -- 172.21.15.78:0/3239731419 wait complete. 2023-10-11T09:41:21.512 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-11T09:41:21.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.686+0000 7f1df8de1700 1 Processor -- start 2023-10-11T09:41:21.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.686+0000 7f1df8de1700 1 -- start start 2023-10-11T09:41:21.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.690+0000 7f1df8de1700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1df40cfc30 0x7f1df40ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:21.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.690+0000 7f1df8de1700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f1df406cda0 0x7f1df40afb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:21.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.690+0000 7f1df8de1700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1df4161550 con 0x7f1df40cc890 2023-10-11T09:41:21.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.690+0000 7f1df8de1700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1df406d200 con 0x7f1df406cda0 2023-10-11T09:41:21.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.690+0000 7f1df8de1700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df40ce700 con 0x7f1df40cfc30 2023-10-11T09:41:21.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.690+0000 7f1df3fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f1df406cda0 0x7f1df40afb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:21.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.690+0000 7f1df37fe700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f1df40cc890 0x7f1df40b28f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:55864/0 (socket says 172.21.15.78:55864) 2023-10-11T09:41:21.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.690+0000 7f1df37fe700 1 -- 172.21.15.78:0/80551055 learned_addr learned my addr 172.21.15.78:0/80551055 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:21.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.690+0000 7f1df3fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f1df406cda0 0x7f1df40afb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50870/0 (socket says 172.21.15.78:50870) 2023-10-11T09:41:21.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.690+0000 7f1debfff700 1 --2- 172.21.15.78:0/80551055 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1df40cfc30 0x7f1df40ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:21.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.690+0000 7f1df3fff700 1 -- 172.21.15.78:0/80551055 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1df40cfc30 msgr2=0x7f1df40ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:21.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.690+0000 7f1df3fff700 1 --2- 172.21.15.78:0/80551055 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1df40cfc30 0x7f1df40ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:21.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.690+0000 7f1df3fff700 1 -- 172.21.15.78:0/80551055 >> v1:172.21.15.78:6789/0 conn(0x7f1df40cc890 legacy=0x7f1df40b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:21.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.690+0000 7f1df3fff700 1 -- 172.21.15.78:0/80551055 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df4160940 con 0x7f1df406cda0 2023-10-11T09:41:21.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.690+0000 7f1df2ffd700 1 -- 172.21.15.78:0/80551055 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3303417665 0 0) 0x7f1df4161550 con 0x7f1df40cc890 2023-10-11T09:41:21.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.690+0000 7f1df3fff700 1 --2- 172.21.15.78:0/80551055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1df406cda0 0x7f1df40afb20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1dec009d90 tx=0x7f1dec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9585a5c91c7a2c04 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:21.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.690+0000 7f1df2ffd700 1 -- 172.21.15.78:0/80551055 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec00e070 con 0x7f1df406cda0 2023-10-11T09:41:21.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.690+0000 7f1df2ffd700 1 -- 172.21.15.78:0/80551055 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dec00bb20 con 0x7f1df406cda0 2023-10-11T09:41:21.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.690+0000 7f1df2ffd700 1 -- 172.21.15.78:0/80551055 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec012430 con 0x7f1df406cda0 2023-10-11T09:41:21.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.690+0000 7f1df8de1700 1 -- 172.21.15.78:0/80551055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1df406cda0 msgr2=0x7f1df40afb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:21.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.690+0000 7f1df8de1700 1 --2- 172.21.15.78:0/80551055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1df406cda0 0x7f1df40afb20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1dec009d90 tx=0x7f1dec00b330 comp rx=0 tx=0).stop 2023-10-11T09:41:21.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.694+0000 7f1df8de1700 1 -- 172.21.15.78:0/80551055 shutdown_connections 2023-10-11T09:41:21.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.694+0000 7f1df8de1700 1 --2- 172.21.15.78:0/80551055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1df406cda0 0x7f1df40afb20 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:21.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.694+0000 7f1df8de1700 1 --2- 172.21.15.78:0/80551055 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1df40cfc30 0x7f1df40ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:21.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.694+0000 7f1df8de1700 1 -- 172.21.15.78:0/80551055 >> 172.21.15.78:0/80551055 conn(0x7f1df406a060 msgr2=0x7f1df406a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:21.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.694+0000 7f1df8de1700 1 -- 172.21.15.78:0/80551055 shutdown_connections 2023-10-11T09:41:21.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.694+0000 7f1df8de1700 1 -- 172.21.15.78:0/80551055 wait complete. 2023-10-11T09:41:21.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.694+0000 7f1df8de1700 1 Processor -- start 2023-10-11T09:41:21.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.698+0000 7f1df8de1700 1 -- start start 2023-10-11T09:41:21.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.698+0000 7f1df8de1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1df406cda0 0x7f1df40cc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:21.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.698+0000 7f1df8de1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1df40cc890 0x7f1df40c50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:21.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.698+0000 7f1df8de1700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1df40cfc30 0x7f1df415f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:21.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.698+0000 7f1df8de1700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df40d1950 con 0x7f1df40cc890 2023-10-11T09:41:21.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.698+0000 7f1df8de1700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df414e940 con 0x7f1df406cda0 2023-10-11T09:41:21.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.698+0000 7f1df8de1700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df40cc660 con 0x7f1df40cfc30 2023-10-11T09:41:21.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.698+0000 7f1debfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1df40cc890 0x7f1df40c50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:21.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.698+0000 7f1debfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1df40cc890 0x7f1df40c50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50886/0 (socket says 172.21.15.78:50886) 2023-10-11T09:41:21.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.698+0000 7f1debfff700 1 -- 172.21.15.78:0/1006205320 learned_addr learned my addr 172.21.15.78:0/1006205320 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:21.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.698+0000 7f1df37fe700 1 --2- 172.21.15.78:0/1006205320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1df406cda0 0x7f1df40cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:21.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.698+0000 7f1df3fff700 1 --2- 172.21.15.78:0/1006205320 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1df40cfc30 0x7f1df415f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:21.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.698+0000 7f1debfff700 1 -- 172.21.15.78:0/1006205320 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1df40cfc30 msgr2=0x7f1df415f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:21.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.698+0000 7f1debfff700 1 --2- 172.21.15.78:0/1006205320 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1df40cfc30 0x7f1df415f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:21.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.698+0000 7f1debfff700 1 -- 172.21.15.78:0/1006205320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1df406cda0 msgr2=0x7f1df40cc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:21.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.698+0000 7f1debfff700 1 --2- 172.21.15.78:0/1006205320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1df406cda0 0x7f1df40cc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:21.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.698+0000 7f1debfff700 1 -- 172.21.15.78:0/1006205320 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dec009a70 con 0x7f1df40cc890 2023-10-11T09:41:21.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.698+0000 7f1debfff700 1 --2- 172.21.15.78:0/1006205320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1df40cc890 0x7f1df40c50d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc00e9b0 tx=0x7f1ddc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:21.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.698+0000 7f1df0ff9700 1 -- 172.21.15.78:0/1006205320 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc009670 con 0x7f1df40cc890 2023-10-11T09:41:21.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.702+0000 7f1df0ff9700 1 -- 172.21.15.78:0/1006205320 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ddc00c850 con 0x7f1df40cc890 2023-10-11T09:41:21.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.702+0000 7f1df8de1700 1 -- 172.21.15.78:0/1006205320 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df40d1670 con 0x7f1df40cc890 2023-10-11T09:41:21.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.702+0000 7f1df0ff9700 1 -- 172.21.15.78:0/1006205320 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc00cbc0 con 0x7f1df40cc890 2023-10-11T09:41:21.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.702+0000 7f1df8de1700 1 -- 172.21.15.78:0/1006205320 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df414e5d0 con 0x7f1df40cc890 2023-10-11T09:41:21.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.702+0000 7f1df8de1700 1 -- 172.21.15.78:0/1006205320 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1df414e150 con 0x7f1df40cc890 2023-10-11T09:41:21.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.702+0000 7f1df0ff9700 1 -- 172.21.15.78:0/1006205320 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f1ddc00cd20 con 0x7f1df40cc890 2023-10-11T09:41:21.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.706+0000 7f1df0ff9700 1 --2- 172.21.15.78:0/1006205320 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1dd4065ed0 0x7f1dd4068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:21.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.706+0000 7f1df0ff9700 1 -- 172.21.15.78:0/1006205320 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f1ddc083cd0 con 0x7f1df40cc890 2023-10-11T09:41:21.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.706+0000 7f1df37fe700 1 --2- 172.21.15.78:0/1006205320 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1dd4065ed0 0x7f1dd4068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:21.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.706+0000 7f1df37fe700 1 --2- 172.21.15.78:0/1006205320 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1dd4065ed0 0x7f1dd4068380 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1de4000f60 tx=0x7f1de4007c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:21.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.710+0000 7f1df0ff9700 1 -- 172.21.15.78:0/1006205320 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f1df414e150 con 0x7f1df40cc890 2023-10-11T09:41:21.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.882+0000 7f1df8de1700 1 -- 172.21.15.78:0/1006205320 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1df414e150 con 0x7f1df40cc890 2023-10-11T09:41:21.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.894+0000 7f1df0ff9700 1 -- 172.21.15.78:0/1006205320 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1df414e150 con 0x7f1df40cc890 2023-10-11T09:41:21.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.894+0000 7f1df8de1700 1 -- 172.21.15.78:0/1006205320 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1dd4065ed0 msgr2=0x7f1dd4068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:21.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.894+0000 7f1df8de1700 1 --2- 172.21.15.78:0/1006205320 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1dd4065ed0 0x7f1dd4068380 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1de4000f60 tx=0x7f1de4007c00 comp rx=0 tx=0).stop 2023-10-11T09:41:21.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.894+0000 7f1df8de1700 1 -- 172.21.15.78:0/1006205320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1df40cc890 msgr2=0x7f1df40c50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:21.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.894+0000 7f1df8de1700 1 --2- 172.21.15.78:0/1006205320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1df40cc890 0x7f1df40c50d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc00e9b0 tx=0x7f1ddc00ed70 comp rx=0 tx=0).stop 2023-10-11T09:41:21.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.898+0000 7f1df8de1700 1 -- 172.21.15.78:0/1006205320 shutdown_connections 2023-10-11T09:41:21.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.898+0000 7f1df8de1700 1 --2- 172.21.15.78:0/1006205320 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1dd4065ed0 0x7f1dd4068380 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:21.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.898+0000 7f1df8de1700 1 --2- 172.21.15.78:0/1006205320 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1df40cfc30 0x7f1df415f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:21.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.898+0000 7f1df8de1700 1 --2- 172.21.15.78:0/1006205320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1df40cc890 0x7f1df40c50d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:21.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.898+0000 7f1df8de1700 1 --2- 172.21.15.78:0/1006205320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1df406cda0 0x7f1df40cc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:21.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.898+0000 7f1df8de1700 1 -- 172.21.15.78:0/1006205320 >> 172.21.15.78:0/1006205320 conn(0x7f1df406a060 msgr2=0x7f1df415c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:21.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.898+0000 7f1df8de1700 1 -- 172.21.15.78:0/1006205320 shutdown_connections 2023-10-11T09:41:21.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:21.898+0000 7f1df8de1700 1 -- 172.21.15.78:0/1006205320 wait complete. 2023-10-11T09:41:21.923 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-11T09:41:21.924 DEBUG:teuthology.orchestra.run.smithi078:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-11T09:41:21.970 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-11T09:41:22.016 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-11T09:41:22.016 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:41:22.017 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:41:22.035 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:41:22.036 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:41:22.093 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:41:22.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.310+0000 7fd202008700 1 Processor -- start 2023-10-11T09:41:22.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.310+0000 7fd202008700 1 -- start start 2023-10-11T09:41:22.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.310+0000 7fd202008700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd1fc0d1660 0x7fd1fc0d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:22.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.310+0000 7fd202008700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fd1fc0d28e0 0x7fd1fc0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:22.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.310+0000 7fd202008700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1fc15dc10 con 0x7fd1fc156ce0 2023-10-11T09:41:22.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.310+0000 7fd202008700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1fc160380 con 0x7fd1fc0d28e0 2023-10-11T09:41:22.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.310+0000 7fd202008700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1fc15d930 con 0x7fd1fc0d1660 2023-10-11T09:41:22.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.314+0000 7fd201807700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fd1fc0d28e0 0x7fd1fc0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:22.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.314+0000 7fd201807700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fd1fc0d28e0 0x7fd1fc0d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50902/0 (socket says 172.21.15.78:50902) 2023-10-11T09:41:22.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.314+0000 7fd201807700 1 -- 172.21.15.78:0/51460719 learned_addr learned my addr 172.21.15.78:0/51460719 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:22.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.314+0000 7fd200805700 1 --2- 172.21.15.78:0/51460719 >> v2:172.21.15.133:3300/0 conn(0x7fd1fc0d1660 0x7fd1fc0d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:22.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.314+0000 7fd201807700 1 -- 172.21.15.78:0/51460719 >> v2:172.21.15.133:3300/0 conn(0x7fd1fc0d1660 msgr2=0x7fd1fc0d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:22.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.314+0000 7fd201807700 1 --2- 172.21.15.78:0/51460719 >> v2:172.21.15.133:3300/0 conn(0x7fd1fc0d1660 0x7fd1fc0d0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:22.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.314+0000 7fd201807700 1 -- 172.21.15.78:0/51460719 >> v1:172.21.15.133:6789/0 conn(0x7fd1fc156ce0 legacy=0x7fd1fc160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:22.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.314+0000 7fd201807700 1 -- 172.21.15.78:0/51460719 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1fc160620 con 0x7fd1fc0d28e0 2023-10-11T09:41:22.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.314+0000 7fd201807700 1 --2- 172.21.15.78:0/51460719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1fc0d28e0 0x7fd1fc0d0f90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc06d2e0 tx=0x7fd1f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f59df63cd74495 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:22.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.314+0000 7fd1f3fff700 1 -- 172.21.15.78:0/51460719 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f400e070 con 0x7fd1fc0d28e0 2023-10-11T09:41:22.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.314+0000 7fd1f3fff700 1 -- 172.21.15.78:0/51460719 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f400b910 con 0x7fd1fc0d28e0 2023-10-11T09:41:22.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.314+0000 7fd1f3fff700 1 -- 172.21.15.78:0/51460719 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f400bcc0 con 0x7fd1fc0d28e0 2023-10-11T09:41:22.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.314+0000 7fd202008700 1 -- 172.21.15.78:0/51460719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1fc0d28e0 msgr2=0x7fd1fc0d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:22.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.314+0000 7fd202008700 1 --2- 172.21.15.78:0/51460719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1fc0d28e0 0x7fd1fc0d0f90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc06d2e0 tx=0x7fd1f400b330 comp rx=0 tx=0).stop 2023-10-11T09:41:22.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.314+0000 7fd202008700 1 -- 172.21.15.78:0/51460719 shutdown_connections 2023-10-11T09:41:22.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.314+0000 7fd202008700 1 --2- 172.21.15.78:0/51460719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1fc0d28e0 0x7fd1fc0d0f90 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:22.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.314+0000 7fd202008700 1 --2- 172.21.15.78:0/51460719 >> v2:172.21.15.133:3300/0 conn(0x7fd1fc0d1660 0x7fd1fc0d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:22.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.314+0000 7fd202008700 1 -- 172.21.15.78:0/51460719 >> 172.21.15.78:0/51460719 conn(0x7fd1fc06afb0 msgr2=0x7fd1fc06b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:22.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.314+0000 7fd202008700 1 -- 172.21.15.78:0/51460719 shutdown_connections 2023-10-11T09:41:22.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.314+0000 7fd202008700 1 -- 172.21.15.78:0/51460719 wait complete. 2023-10-11T09:41:22.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.318+0000 7fd202008700 1 Processor -- start 2023-10-11T09:41:22.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.318+0000 7fd202008700 1 -- start start 2023-10-11T09:41:22.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.322+0000 7fd202008700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1fc0d1660 0x7fd1fc157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:22.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.322+0000 7fd202008700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1fc0d28e0 0x7fd1fc146550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:22.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.322+0000 7fd202008700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd1fc156ce0 0x7fd1fc148440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:22.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.322+0000 7fd202008700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1fc146a90 con 0x7fd1fc0d1660 2023-10-11T09:41:22.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.322+0000 7fd202008700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1fc148980 con 0x7fd1fc0d28e0 2023-10-11T09:41:22.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.322+0000 7fd202008700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1fc143200 con 0x7fd1fc156ce0 2023-10-11T09:41:22.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.322+0000 7fd201006700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1fc0d1660 0x7fd1fc157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:22.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.322+0000 7fd201006700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1fc0d1660 0x7fd1fc157230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50908/0 (socket says 172.21.15.78:50908) 2023-10-11T09:41:22.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.322+0000 7fd201006700 1 -- 172.21.15.78:0/3397122766 learned_addr learned my addr 172.21.15.78:0/3397122766 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:22.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.322+0000 7fd201006700 1 -- 172.21.15.78:0/3397122766 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd1fc156ce0 msgr2=0x7fd1fc148440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:22.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.322+0000 7fd201006700 1 --2- 172.21.15.78:0/3397122766 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd1fc156ce0 0x7fd1fc148440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:22.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.322+0000 7fd201006700 1 -- 172.21.15.78:0/3397122766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1fc0d28e0 msgr2=0x7fd1fc146550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:22.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.322+0000 7fd201006700 1 --2- 172.21.15.78:0/3397122766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1fc0d28e0 0x7fd1fc146550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:22.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.322+0000 7fd201006700 1 -- 172.21.15.78:0/3397122766 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f4009b30 con 0x7fd1fc0d1660 2023-10-11T09:41:22.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.322+0000 7fd201006700 1 --2- 172.21.15.78:0/3397122766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1fc0d1660 0x7fd1fc157230 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd1f800c500 tx=0x7fd1f800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:22.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.322+0000 7fd1f1ffb700 1 -- 172.21.15.78:0/3397122766 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f80052d0 con 0x7fd1fc0d1660 2023-10-11T09:41:22.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.322+0000 7fd202008700 1 -- 172.21.15.78:0/3397122766 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1fc1434e0 con 0x7fd1fc0d1660 2023-10-11T09:41:22.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.322+0000 7fd1f1ffb700 1 -- 172.21.15.78:0/3397122766 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f8005430 con 0x7fd1fc0d1660 2023-10-11T09:41:22.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.322+0000 7fd1f1ffb700 1 -- 172.21.15.78:0/3397122766 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f80055a0 con 0x7fd1fc0d1660 2023-10-11T09:41:22.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.322+0000 7fd202008700 1 -- 172.21.15.78:0/3397122766 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1fc143aa0 con 0x7fd1fc0d1660 2023-10-11T09:41:22.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.326+0000 7fd202008700 1 -- 172.21.15.78:0/3397122766 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1fc065d40 con 0x7fd1fc0d1660 2023-10-11T09:41:22.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.326+0000 7fd1f1ffb700 1 -- 172.21.15.78:0/3397122766 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fd1f80038e0 con 0x7fd1fc0d1660 2023-10-11T09:41:22.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.326+0000 7fd1f1ffb700 1 --2- 172.21.15.78:0/3397122766 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd1e0065ed0 0x7fd1e0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:22.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.326+0000 7fd1f1ffb700 1 -- 172.21.15.78:0/3397122766 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fd1f8082ce0 con 0x7fd1fc0d1660 2023-10-11T09:41:22.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.326+0000 7fd200805700 1 --2- 172.21.15.78:0/3397122766 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd1e0065ed0 0x7fd1e0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:22.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.326+0000 7fd200805700 1 --2- 172.21.15.78:0/3397122766 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd1e0065ed0 0x7fd1e0068380 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec006b40 tx=0x7fd1ec006ad0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:22.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.334+0000 7fd1f1ffb700 1 -- 172.21.15.78:0/3397122766 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fd1f8050270 con 0x7fd1fc0d1660 2023-10-11T09:41:22.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.505+0000 7fd202008700 1 -- 172.21.15.78:0/3397122766 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1fc06d310 con 0x7fd1fc0d1660 2023-10-11T09:41:22.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.509+0000 7fd1f1ffb700 1 -- 172.21.15.78:0/3397122766 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17907 (secure 0 0 0) 0x7fd1f8053890 con 0x7fd1fc0d1660 2023-10-11T09:41:22.518 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:41:22.518 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":25,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:41:21.473566+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:40:39.862581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T09:40:40.663641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.37","mappings":[{"from":3,"to":2}]},{"pgid":"4.3e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:41:22.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.509+0000 7fd202008700 1 -- 172.21.15.78:0/3397122766 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd1e0065ed0 msgr2=0x7fd1e0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:22.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.509+0000 7fd202008700 1 --2- 172.21.15.78:0/3397122766 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd1e0065ed0 0x7fd1e0068380 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec006b40 tx=0x7fd1ec006ad0 comp rx=0 tx=0).stop 2023-10-11T09:41:22.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.509+0000 7fd202008700 1 -- 172.21.15.78:0/3397122766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1fc0d1660 msgr2=0x7fd1fc157230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:22.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.509+0000 7fd202008700 1 --2- 172.21.15.78:0/3397122766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1fc0d1660 0x7fd1fc157230 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd1f800c500 tx=0x7fd1f800c8c0 comp rx=0 tx=0).stop 2023-10-11T09:41:22.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.509+0000 7fd202008700 1 -- 172.21.15.78:0/3397122766 shutdown_connections 2023-10-11T09:41:22.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.509+0000 7fd202008700 1 --2- 172.21.15.78:0/3397122766 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd1e0065ed0 0x7fd1e0068380 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:22.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.509+0000 7fd202008700 1 --2- 172.21.15.78:0/3397122766 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd1fc156ce0 0x7fd1fc148440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:22.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.509+0000 7fd202008700 1 --2- 172.21.15.78:0/3397122766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1fc0d28e0 0x7fd1fc146550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:22.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.509+0000 7fd202008700 1 --2- 172.21.15.78:0/3397122766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1fc0d1660 0x7fd1fc157230 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:22.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.509+0000 7fd202008700 1 -- 172.21.15.78:0/3397122766 >> 172.21.15.78:0/3397122766 conn(0x7fd1fc06afb0 msgr2=0x7fd1fc15f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:22.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.513+0000 7fd202008700 1 -- 172.21.15.78:0/3397122766 shutdown_connections 2023-10-11T09:41:22.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.513+0000 7fd202008700 1 -- 172.21.15.78:0/3397122766 wait complete. 2023-10-11T09:41:22.538 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T09:40:39.862581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T09:40:40.663641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:41:22.539 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:41:22.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.717+0000 7fc9b237d700 1 Processor -- start 2023-10-11T09:41:22.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.717+0000 7fc9b237d700 1 -- start start 2023-10-11T09:41:22.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.721+0000 7fc9b237d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc9ac146fe0 0x7fc9ac06bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:22.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.721+0000 7fc9b237d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fc9ac0b36f0 0x7fc9ac1629e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:22.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.721+0000 7fc9b237d700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9ac06ccb0 con 0x7fc9ac161d90 2023-10-11T09:41:22.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.721+0000 7fc9b237d700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9ac06c410 con 0x7fc9ac0b36f0 2023-10-11T09:41:22.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.721+0000 7fc9b237d700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac06c6f0 con 0x7fc9ac146fe0 2023-10-11T09:41:22.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.721+0000 7fc9b1b7c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fc9ac0b36f0 0x7fc9ac1629e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:22.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.721+0000 7fc9b1b7c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fc9ac0b36f0 0x7fc9ac1629e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50912/0 (socket says 172.21.15.78:50912) 2023-10-11T09:41:22.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.721+0000 7fc9b1b7c700 1 -- 172.21.15.78:0/548004722 learned_addr learned my addr 172.21.15.78:0/548004722 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:22.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.721+0000 7fc9b137b700 1 --2- 172.21.15.78:0/548004722 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc9ac146fe0 0x7fc9ac06bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:22.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.721+0000 7fc9b1b7c700 1 -- 172.21.15.78:0/548004722 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc9ac146fe0 msgr2=0x7fc9ac06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:22.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.721+0000 7fc9b1b7c700 1 --2- 172.21.15.78:0/548004722 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc9ac146fe0 0x7fc9ac06bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:22.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.721+0000 7fc9b1b7c700 1 -- 172.21.15.78:0/548004722 >> v1:172.21.15.133:6789/0 conn(0x7fc9ac161d90 legacy=0x7fc9ac0b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:22.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.721+0000 7fc9b1b7c700 1 -- 172.21.15.78:0/548004722 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ac06c9d0 con 0x7fc9ac0b36f0 2023-10-11T09:41:22.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.725+0000 7fc9b1b7c700 1 --2- 172.21.15.78:0/548004722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9ac0b36f0 0x7fc9ac1629e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4009e50 tx=0x7fc9a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d071445836ba2fbc server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:22.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.725+0000 7fc9a3fff700 1 -- 172.21.15.78:0/548004722 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a400e070 con 0x7fc9ac0b36f0 2023-10-11T09:41:22.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.725+0000 7fc9a3fff700 1 -- 172.21.15.78:0/548004722 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a400b9c0 con 0x7fc9ac0b36f0 2023-10-11T09:41:22.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.725+0000 7fc9a3fff700 1 -- 172.21.15.78:0/548004722 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a400bd70 con 0x7fc9ac0b36f0 2023-10-11T09:41:22.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.725+0000 7fc9b237d700 1 -- 172.21.15.78:0/548004722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9ac0b36f0 msgr2=0x7fc9ac1629e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:22.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.725+0000 7fc9b237d700 1 --2- 172.21.15.78:0/548004722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9ac0b36f0 0x7fc9ac1629e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4009e50 tx=0x7fc9a400b330 comp rx=0 tx=0).stop 2023-10-11T09:41:22.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.725+0000 7fc9b237d700 1 -- 172.21.15.78:0/548004722 shutdown_connections 2023-10-11T09:41:22.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.725+0000 7fc9b237d700 1 --2- 172.21.15.78:0/548004722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9ac0b36f0 0x7fc9ac1629e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:22.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.725+0000 7fc9b237d700 1 --2- 172.21.15.78:0/548004722 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc9ac146fe0 0x7fc9ac06bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:22.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.725+0000 7fc9b237d700 1 -- 172.21.15.78:0/548004722 >> 172.21.15.78:0/548004722 conn(0x7fc9ac06a060 msgr2=0x7fc9ac06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:22.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.725+0000 7fc9b237d700 1 -- 172.21.15.78:0/548004722 shutdown_connections 2023-10-11T09:41:22.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.725+0000 7fc9b237d700 1 -- 172.21.15.78:0/548004722 wait complete. 2023-10-11T09:41:22.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.729+0000 7fc9b237d700 1 Processor -- start 2023-10-11T09:41:22.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.729+0000 7fc9b237d700 1 -- start start 2023-10-11T09:41:22.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.729+0000 7fc9b237d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9ac0b36f0 0x7fc9ac1f73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:22.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.729+0000 7fc9b237d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9ac146fe0 0x7fc9ac15ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:22.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.729+0000 7fc9b237d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc9ac161d90 0x7fc9ac15f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:22.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.729+0000 7fc9b237d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac0cd5f0 con 0x7fc9ac0b36f0 2023-10-11T09:41:22.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.729+0000 7fc9b237d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac0cd8d0 con 0x7fc9ac146fe0 2023-10-11T09:41:22.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.729+0000 7fc9b237d700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac0cdbb0 con 0x7fc9ac161d90 2023-10-11T09:41:22.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.729+0000 7fc9b137b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9ac0b36f0 0x7fc9ac1f73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:22.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.729+0000 7fc9b137b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9ac0b36f0 0x7fc9ac1f73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50924/0 (socket says 172.21.15.78:50924) 2023-10-11T09:41:22.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.729+0000 7fc9b137b700 1 -- 172.21.15.78:0/3223569291 learned_addr learned my addr 172.21.15.78:0/3223569291 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:22.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.729+0000 7fc9b1b7c700 1 --2- 172.21.15.78:0/3223569291 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc9ac161d90 0x7fc9ac15f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:22.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.729+0000 7fc9b0b7a700 1 --2- 172.21.15.78:0/3223569291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9ac146fe0 0x7fc9ac15ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:22.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.729+0000 7fc9b137b700 1 -- 172.21.15.78:0/3223569291 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc9ac161d90 msgr2=0x7fc9ac15f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:22.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.729+0000 7fc9b137b700 1 --2- 172.21.15.78:0/3223569291 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc9ac161d90 0x7fc9ac15f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:22.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.729+0000 7fc9b137b700 1 -- 172.21.15.78:0/3223569291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9ac146fe0 msgr2=0x7fc9ac15ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:22.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.729+0000 7fc9b137b700 1 --2- 172.21.15.78:0/3223569291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9ac146fe0 0x7fc9ac15ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:22.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.729+0000 7fc9b137b700 1 -- 172.21.15.78:0/3223569291 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a4009b30 con 0x7fc9ac0b36f0 2023-10-11T09:41:22.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.729+0000 7fc9b137b700 1 --2- 172.21.15.78:0/3223569291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9ac0b36f0 0x7fc9ac1f73c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800e810 tx=0x7fc9a800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:22.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.729+0000 7fc9a1ffb700 1 -- 172.21.15.78:0/3223569291 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a8013610 con 0x7fc9ac0b36f0 2023-10-11T09:41:22.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.729+0000 7fc9a1ffb700 1 -- 172.21.15.78:0/3223569291 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a8013770 con 0x7fc9ac0b36f0 2023-10-11T09:41:22.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.729+0000 7fc9b237d700 1 -- 172.21.15.78:0/3223569291 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9ac0cde90 con 0x7fc9ac0b36f0 2023-10-11T09:41:22.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.729+0000 7fc9a1ffb700 1 -- 172.21.15.78:0/3223569291 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a8013b20 con 0x7fc9ac0b36f0 2023-10-11T09:41:22.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.729+0000 7fc9b237d700 1 -- 172.21.15.78:0/3223569291 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9ac1f7900 con 0x7fc9ac0b36f0 2023-10-11T09:41:22.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.733+0000 7fc9b237d700 1 -- 172.21.15.78:0/3223569291 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9ac065d40 con 0x7fc9ac0b36f0 2023-10-11T09:41:22.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.733+0000 7fc9a1ffb700 1 -- 172.21.15.78:0/3223569291 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fc9a8013c80 con 0x7fc9ac0b36f0 2023-10-11T09:41:22.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.737+0000 7fc9a1ffb700 1 --2- 172.21.15.78:0/3223569291 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc990065ed0 0x7fc990068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:22.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.737+0000 7fc9a1ffb700 1 -- 172.21.15.78:0/3223569291 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fc9a8083650 con 0x7fc9ac0b36f0 2023-10-11T09:41:22.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.737+0000 7fc9b0b7a700 1 --2- 172.21.15.78:0/3223569291 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc990065ed0 0x7fc990068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:22.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.737+0000 7fc9b0b7a700 1 --2- 172.21.15.78:0/3223569291 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc990065ed0 0x7fc990068380 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac0cf8b0 tx=0x7fc99c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:22.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.741+0000 7fc9a1ffb700 1 -- 172.21.15.78:0/3223569291 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fc9a8050bd0 con 0x7fc9ac0b36f0 2023-10-11T09:41:22.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.913+0000 7fc9b237d700 1 -- 172.21.15.78:0/3223569291 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc9ac0cd730 con 0x7fc9ac0b36f0 2023-10-11T09:41:22.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.913+0000 7fc9a1ffb700 1 -- 172.21.15.78:0/3223569291 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc9ac0cd730 con 0x7fc9ac0b36f0 2023-10-11T09:41:22.920 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:41:22.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.917+0000 7fc9b237d700 1 -- 172.21.15.78:0/3223569291 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc990065ed0 msgr2=0x7fc990068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:22.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.917+0000 7fc9b237d700 1 --2- 172.21.15.78:0/3223569291 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc990065ed0 0x7fc990068380 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac0cf8b0 tx=0x7fc99c009040 comp rx=0 tx=0).stop 2023-10-11T09:41:22.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.917+0000 7fc9b237d700 1 -- 172.21.15.78:0/3223569291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9ac0b36f0 msgr2=0x7fc9ac1f73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:22.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.917+0000 7fc9b237d700 1 --2- 172.21.15.78:0/3223569291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9ac0b36f0 0x7fc9ac1f73c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800e810 tx=0x7fc9a800ebd0 comp rx=0 tx=0).stop 2023-10-11T09:41:22.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.917+0000 7fc9b237d700 1 -- 172.21.15.78:0/3223569291 shutdown_connections 2023-10-11T09:41:22.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.917+0000 7fc9b237d700 1 --2- 172.21.15.78:0/3223569291 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc990065ed0 0x7fc990068380 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:22.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.917+0000 7fc9b237d700 1 --2- 172.21.15.78:0/3223569291 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc9ac161d90 0x7fc9ac15f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:22.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.917+0000 7fc9b237d700 1 --2- 172.21.15.78:0/3223569291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9ac146fe0 0x7fc9ac15ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:22.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.917+0000 7fc9b237d700 1 --2- 172.21.15.78:0/3223569291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9ac0b36f0 0x7fc9ac1f73c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:22.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.917+0000 7fc9b237d700 1 -- 172.21.15.78:0/3223569291 >> 172.21.15.78:0/3223569291 conn(0x7fc9ac06a060 msgr2=0x7fc9ac0af810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:22.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.917+0000 7fc9b237d700 1 -- 172.21.15.78:0/3223569291 shutdown_connections 2023-10-11T09:41:22.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:22.917+0000 7fc9b237d700 1 -- 172.21.15.78:0/3223569291 wait complete. 2023-10-11T09:41:22.941 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:41:23.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.113+0000 7f899c1c6700 1 Processor -- start 2023-10-11T09:41:23.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.113+0000 7f899c1c6700 1 -- start start 2023-10-11T09:41:23.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.117+0000 7f899c1c6700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f89940b8890 0x7f89940d2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:23.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.117+0000 7f899c1c6700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89940cd690 con 0x7f89940d3510 2023-10-11T09:41:23.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.117+0000 7f899c1c6700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89940cd8e0 con 0x7f89940b84c0 2023-10-11T09:41:23.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.117+0000 7f899c1c6700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89940cdea0 con 0x7f89940b8890 2023-10-11T09:41:23.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.117+0000 7f899a763700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f89940d3510 0x7f89940d1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:55870/0 (socket says 172.21.15.78:55870) 2023-10-11T09:41:23.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.117+0000 7f8999f62700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f89940b84c0 0x7f89940cfda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:44802/0 (socket says 172.21.15.78:44802) 2023-10-11T09:41:23.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.117+0000 7f899a763700 1 -- 172.21.15.78:0/1811683402 learned_addr learned my addr 172.21.15.78:0/1811683402 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:23.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.117+0000 7f8999761700 1 --2- 172.21.15.78:0/1811683402 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f89940b8890 0x7f89940d2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:23.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.117+0000 7f8998f60700 1 -- 172.21.15.78:0/1811683402 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3642307980 0 0) 0x7f89940cd690 con 0x7f89940d3510 2023-10-11T09:41:23.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.117+0000 7f8998f60700 1 -- 172.21.15.78:0/1811683402 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89800035d0 con 0x7f89940d3510 2023-10-11T09:41:23.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.117+0000 7f8999761700 1 -- 172.21.15.78:0/1811683402 >> v1:172.21.15.133:6789/0 conn(0x7f89940b84c0 legacy=0x7f89940cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:23.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.117+0000 7f8998f60700 1 -- 172.21.15.78:0/1811683402 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1662199574 0 0) 0x7f89940cd8e0 con 0x7f89940b84c0 2023-10-11T09:41:23.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.117+0000 7f8999761700 1 -- 172.21.15.78:0/1811683402 >> v1:172.21.15.78:6789/0 conn(0x7f89940d3510 legacy=0x7f89940d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:23.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.117+0000 7f8999761700 1 -- 172.21.15.78:0/1811683402 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89940cdbc0 con 0x7f89940b8890 2023-10-11T09:41:23.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.117+0000 7f8999761700 1 --2- 172.21.15.78:0/1811683402 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f89940b8890 0x7f89940d2fd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8984009880 tx=0x7f8984009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3839bda6ccbe043f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:23.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.117+0000 7f8998f60700 1 -- 172.21.15.78:0/1811683402 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8984011830 con 0x7f89940b8890 2023-10-11T09:41:23.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.117+0000 7f8998f60700 1 -- 172.21.15.78:0/1811683402 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8984011990 con 0x7f89940b8890 2023-10-11T09:41:23.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.117+0000 7f8998f60700 1 -- 172.21.15.78:0/1811683402 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8984011d20 con 0x7f89940b8890 2023-10-11T09:41:23.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.121+0000 7f899c1c6700 1 -- 172.21.15.78:0/1811683402 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f89940b8890 msgr2=0x7f89940d2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:23.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.121+0000 7f899c1c6700 1 --2- 172.21.15.78:0/1811683402 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f89940b8890 0x7f89940d2fd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8984009880 tx=0x7f8984009b90 comp rx=0 tx=0).stop 2023-10-11T09:41:23.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.121+0000 7f899c1c6700 1 -- 172.21.15.78:0/1811683402 shutdown_connections 2023-10-11T09:41:23.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.121+0000 7f899c1c6700 1 --2- 172.21.15.78:0/1811683402 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f89940b8890 0x7f89940d2fd0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:23.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.121+0000 7f899c1c6700 1 -- 172.21.15.78:0/1811683402 >> 172.21.15.78:0/1811683402 conn(0x7f899406cb50 msgr2=0x7f899406cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:23.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.121+0000 7f899c1c6700 1 -- 172.21.15.78:0/1811683402 shutdown_connections 2023-10-11T09:41:23.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.121+0000 7f899c1c6700 1 -- 172.21.15.78:0/1811683402 wait complete. 2023-10-11T09:41:23.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.121+0000 7f899c1c6700 1 Processor -- start 2023-10-11T09:41:23.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.121+0000 7f899c1c6700 1 -- start start 2023-10-11T09:41:23.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.121+0000 7f899c1c6700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89940b84c0 0x7f89940c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:23.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.125+0000 7f899c1c6700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f89940b8890 0x7f899414b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:23.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.125+0000 7f899c1c6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89940d3510 0x7f89941f1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:23.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.125+0000 7f899c1c6700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899414b320 con 0x7f89940d3510 2023-10-11T09:41:23.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.125+0000 7f899c1c6700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899414ad20 con 0x7f89940b84c0 2023-10-11T09:41:23.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.125+0000 7f899c1c6700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89941f2e60 con 0x7f89940b8890 2023-10-11T09:41:23.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.125+0000 7f899a763700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89940d3510 0x7f89941f1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:23.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.125+0000 7f899a763700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89940d3510 0x7f89941f1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50934/0 (socket says 172.21.15.78:50934) 2023-10-11T09:41:23.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.125+0000 7f899a763700 1 -- 172.21.15.78:0/322062357 learned_addr learned my addr 172.21.15.78:0/322062357 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:23.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.125+0000 7f8999761700 1 --2- 172.21.15.78:0/322062357 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f89940b8890 0x7f899414b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:23.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.125+0000 7f8999f62700 1 --2- 172.21.15.78:0/322062357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89940b84c0 0x7f89940c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:23.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.125+0000 7f899a763700 1 -- 172.21.15.78:0/322062357 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f89940b8890 msgr2=0x7f899414b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:23.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.125+0000 7f899a763700 1 --2- 172.21.15.78:0/322062357 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f89940b8890 0x7f899414b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:23.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.125+0000 7f899a763700 1 -- 172.21.15.78:0/322062357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89940b84c0 msgr2=0x7f89940c9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:23.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.125+0000 7f899a763700 1 --2- 172.21.15.78:0/322062357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89940b84c0 0x7f89940c9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:23.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.125+0000 7f899a763700 1 -- 172.21.15.78:0/322062357 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89840094b0 con 0x7f89940d3510 2023-10-11T09:41:23.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.125+0000 7f8999f62700 1 --2- 172.21.15.78:0/322062357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89940b84c0 0x7f89940c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:23.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.125+0000 7f899a763700 1 --2- 172.21.15.78:0/322062357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89940d3510 0x7f89941f1f30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f898c00cad0 tx=0x7f898c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:23.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.125+0000 7f898a7fc700 1 -- 172.21.15.78:0/322062357 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898c007480 con 0x7f89940d3510 2023-10-11T09:41:23.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.125+0000 7f898a7fc700 1 -- 172.21.15.78:0/322062357 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f898c0075e0 con 0x7f89940d3510 2023-10-11T09:41:23.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.125+0000 7f899c1c6700 1 -- 172.21.15.78:0/322062357 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89941f28a0 con 0x7f89940d3510 2023-10-11T09:41:23.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.125+0000 7f898a7fc700 1 -- 172.21.15.78:0/322062357 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898c007950 con 0x7f89940d3510 2023-10-11T09:41:23.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.125+0000 7f899c1c6700 1 -- 172.21.15.78:0/322062357 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f899414af40 con 0x7f89940d3510 2023-10-11T09:41:23.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.125+0000 7f899c1c6700 1 -- 172.21.15.78:0/322062357 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f899414b460 con 0x7f89940d3510 2023-10-11T09:41:23.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.129+0000 7f898a7fc700 1 -- 172.21.15.78:0/322062357 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f898c01b3b0 con 0x7f89940d3510 2023-10-11T09:41:23.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.129+0000 7f898a7fc700 1 --2- 172.21.15.78:0/322062357 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8978065ed0 0x7f8978068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:23.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.129+0000 7f898a7fc700 1 -- 172.21.15.78:0/322062357 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f898c083180 con 0x7f89940d3510 2023-10-11T09:41:23.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.129+0000 7f8999f62700 1 --2- 172.21.15.78:0/322062357 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8978065ed0 0x7f8978068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:23.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.129+0000 7f8999f62700 1 --2- 172.21.15.78:0/322062357 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8978065ed0 0x7f8978068380 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8990000f60 tx=0x7f8990007cc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:23.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.137+0000 7f898a7fc700 1 -- 172.21.15.78:0/322062357 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f899414b460 con 0x7f89940d3510 2023-10-11T09:41:23.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.309+0000 7f899c1c6700 1 -- 172.21.15.78:0/322062357 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f899414b460 con 0x7f89940d3510 2023-10-11T09:41:23.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.309+0000 7f898a7fc700 1 -- 172.21.15.78:0/322062357 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f899414b460 con 0x7f89940d3510 2023-10-11T09:41:23.314 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:41:23.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.309+0000 7f899c1c6700 1 -- 172.21.15.78:0/322062357 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8978065ed0 msgr2=0x7f8978068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:23.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.309+0000 7f899c1c6700 1 --2- 172.21.15.78:0/322062357 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8978065ed0 0x7f8978068380 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8990000f60 tx=0x7f8990007cc0 comp rx=0 tx=0).stop 2023-10-11T09:41:23.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.309+0000 7f899c1c6700 1 -- 172.21.15.78:0/322062357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89940d3510 msgr2=0x7f89941f1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:23.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.309+0000 7f899c1c6700 1 --2- 172.21.15.78:0/322062357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89940d3510 0x7f89941f1f30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f898c00cad0 tx=0x7f898c00cbb0 comp rx=0 tx=0).stop 2023-10-11T09:41:23.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.313+0000 7f899c1c6700 1 -- 172.21.15.78:0/322062357 shutdown_connections 2023-10-11T09:41:23.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.313+0000 7f899c1c6700 1 --2- 172.21.15.78:0/322062357 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8978065ed0 0x7f8978068380 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:23.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.313+0000 7f899c1c6700 1 --2- 172.21.15.78:0/322062357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89940d3510 0x7f89941f1f30 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:23.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.313+0000 7f899c1c6700 1 --2- 172.21.15.78:0/322062357 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f89940b8890 0x7f899414b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:23.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.313+0000 7f899c1c6700 1 --2- 172.21.15.78:0/322062357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89940b84c0 0x7f89940c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:23.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.313+0000 7f899c1c6700 1 -- 172.21.15.78:0/322062357 >> 172.21.15.78:0/322062357 conn(0x7f899406cb50 msgr2=0x7f89940b67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:23.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.313+0000 7f899c1c6700 1 -- 172.21.15.78:0/322062357 shutdown_connections 2023-10-11T09:41:23.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.313+0000 7f899c1c6700 1 -- 172.21.15.78:0/322062357 wait complete. 2023-10-11T09:41:23.338 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T09:41:23.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.505+0000 7f501e9e1700 1 Processor -- start 2023-10-11T09:41:23.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.505+0000 7f501e9e1700 1 -- start start 2023-10-11T09:41:23.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.505+0000 7f501e9e1700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f50180d29d0 0x7f50180b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:23.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.505+0000 7f501e9e1700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50180b47f0 con 0x7f50180b3610 2023-10-11T09:41:23.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.505+0000 7f501e9e1700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50180b8130 con 0x7f50180b3240 2023-10-11T09:41:23.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.505+0000 7f501e9e1700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50180b3fd0 con 0x7f50180d29d0 2023-10-11T09:41:23.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.505+0000 7f501d1de700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f50180b3610 0x7f50180b85c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:55880/0 (socket says 172.21.15.78:55880) 2023-10-11T09:41:23.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.509+0000 7f501d1de700 1 -- 172.21.15.78:0/1497337708 learned_addr learned my addr 172.21.15.78:0/1497337708 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:23.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.509+0000 7f501e1e0700 1 --2- 172.21.15.78:0/1497337708 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f50180d29d0 0x7f50180b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:23.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.509+0000 7f501c9dd700 1 -- 172.21.15.78:0/1497337708 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3977966548 0 0) 0x7f5008001110 con 0x7f50180b3610 2023-10-11T09:41:23.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.509+0000 7f501c9dd700 1 -- 172.21.15.78:0/1497337708 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50040035d0 con 0x7f50180b3610 2023-10-11T09:41:23.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.509+0000 7f501c9dd700 1 -- 172.21.15.78:0/1497337708 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1693102730 0 0) 0x7f50180b8130 con 0x7f50180b3240 2023-10-11T09:41:23.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.509+0000 7f501c9dd700 1 -- 172.21.15.78:0/1497337708 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5008001110 con 0x7f50180b3240 2023-10-11T09:41:23.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.509+0000 7f501e1e0700 1 -- 172.21.15.78:0/1497337708 >> v1:172.21.15.133:6789/0 conn(0x7f50180b3240 legacy=0x7f501815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:23.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.509+0000 7f501e1e0700 1 -- 172.21.15.78:0/1497337708 >> v1:172.21.15.78:6789/0 conn(0x7f50180b3610 legacy=0x7f50180b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:23.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.509+0000 7f501c9dd700 1 -- 172.21.15.78:0/1497337708 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3822774064 0 0) 0x7f50040035d0 con 0x7f50180b3610 2023-10-11T09:41:23.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.509+0000 7f501e1e0700 1 -- 172.21.15.78:0/1497337708 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50180b41e0 con 0x7f50180d29d0 2023-10-11T09:41:23.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.509+0000 7f501e1e0700 1 --2- 172.21.15.78:0/1497337708 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f50180d29d0 0x7f50180b3a90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5010009dc0 tx=0x7f501000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61a1b9ebfae52e9c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:23.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.509+0000 7f501c9dd700 1 -- 172.21.15.78:0/1497337708 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501000e070 con 0x7f50180d29d0 2023-10-11T09:41:23.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.509+0000 7f501c9dd700 1 -- 172.21.15.78:0/1497337708 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5010003a40 con 0x7f50180d29d0 2023-10-11T09:41:23.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.509+0000 7f501c9dd700 1 -- 172.21.15.78:0/1497337708 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5008001800 con 0x7f50180d29d0 2023-10-11T09:41:23.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.509+0000 7f501e9e1700 1 -- 172.21.15.78:0/1497337708 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f50180d29d0 msgr2=0x7f50180b3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:23.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.509+0000 7f501e9e1700 1 --2- 172.21.15.78:0/1497337708 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f50180d29d0 0x7f50180b3a90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5010009dc0 tx=0x7f501000b330 comp rx=0 tx=0).stop 2023-10-11T09:41:23.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.509+0000 7f501e9e1700 1 -- 172.21.15.78:0/1497337708 shutdown_connections 2023-10-11T09:41:23.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.509+0000 7f501e9e1700 1 --2- 172.21.15.78:0/1497337708 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f50180d29d0 0x7f50180b3a90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:23.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.509+0000 7f501e9e1700 1 -- 172.21.15.78:0/1497337708 >> 172.21.15.78:0/1497337708 conn(0x7f501809c260 msgr2=0x7f50180d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:23.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.509+0000 7f501e9e1700 1 -- 172.21.15.78:0/1497337708 shutdown_connections 2023-10-11T09:41:23.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.509+0000 7f501e9e1700 1 -- 172.21.15.78:0/1497337708 wait complete. 2023-10-11T09:41:23.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.513+0000 7f501e9e1700 1 Processor -- start 2023-10-11T09:41:23.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.513+0000 7f501e9e1700 1 -- start start 2023-10-11T09:41:23.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.517+0000 7f501e9e1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50180b3240 0x7f501815a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:23.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.517+0000 7f501e9e1700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f50180b3610 0x7f501815ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:23.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.517+0000 7f501e9e1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50180d29d0 0x7f50180c9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:23.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.517+0000 7f501e9e1700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501815d340 con 0x7f50180d29d0 2023-10-11T09:41:23.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.517+0000 7f501e9e1700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50181f3010 con 0x7f50180b3240 2023-10-11T09:41:23.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.517+0000 7f501e9e1700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50181f32f0 con 0x7f50180b3610 2023-10-11T09:41:23.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.517+0000 7f501e1e0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50180d29d0 0x7f50180c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:23.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.517+0000 7f501e1e0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50180d29d0 0x7f50180c9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50948/0 (socket says 172.21.15.78:50948) 2023-10-11T09:41:23.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.517+0000 7f501e1e0700 1 -- 172.21.15.78:0/2048769736 learned_addr learned my addr 172.21.15.78:0/2048769736 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:23.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.517+0000 7f501d1de700 1 --2- 172.21.15.78:0/2048769736 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f50180b3610 0x7f501815ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:23.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.517+0000 7f501d9df700 1 --2- 172.21.15.78:0/2048769736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50180b3240 0x7f501815a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:23.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.517+0000 7f501e1e0700 1 -- 172.21.15.78:0/2048769736 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f50180b3610 msgr2=0x7f501815ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:23.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.517+0000 7f501e1e0700 1 --2- 172.21.15.78:0/2048769736 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f50180b3610 0x7f501815ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:23.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.517+0000 7f501e1e0700 1 -- 172.21.15.78:0/2048769736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50180b3240 msgr2=0x7f501815a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:23.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.517+0000 7f501e1e0700 1 --2- 172.21.15.78:0/2048769736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50180b3240 0x7f501815a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:23.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.517+0000 7f501e1e0700 1 -- 172.21.15.78:0/2048769736 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5010009a70 con 0x7f50180d29d0 2023-10-11T09:41:23.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.517+0000 7f501d1de700 1 --2- 172.21.15.78:0/2048769736 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f50180b3610 0x7f501815ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:23.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.517+0000 7f501e1e0700 1 --2- 172.21.15.78:0/2048769736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50180d29d0 0x7f50180c9260 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f501000b370 tx=0x7f5010004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:23.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.517+0000 7f500e7fc700 1 -- 172.21.15.78:0/2048769736 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5010004600 con 0x7f50180d29d0 2023-10-11T09:41:23.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.517+0000 7f500e7fc700 1 -- 172.21.15.78:0/2048769736 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5010004760 con 0x7f50180d29d0 2023-10-11T09:41:23.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.517+0000 7f500e7fc700 1 -- 172.21.15.78:0/2048769736 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501001d470 con 0x7f50180d29d0 2023-10-11T09:41:23.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.517+0000 7f501e9e1700 1 -- 172.21.15.78:0/2048769736 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50181f35d0 con 0x7f50180d29d0 2023-10-11T09:41:23.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.517+0000 7f501e9e1700 1 -- 172.21.15.78:0/2048769736 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50181f3b90 con 0x7f50180d29d0 2023-10-11T09:41:23.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.517+0000 7f501e9e1700 1 -- 172.21.15.78:0/2048769736 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50181f3d80 con 0x7f50180d29d0 2023-10-11T09:41:23.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.521+0000 7f500e7fc700 1 -- 172.21.15.78:0/2048769736 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f501001d5d0 con 0x7f50180d29d0 2023-10-11T09:41:23.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.521+0000 7f500e7fc700 1 --2- 172.21.15.78:0/2048769736 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4ffc065f00 0x7f4ffc0683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:23.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.521+0000 7f500e7fc700 1 -- 172.21.15.78:0/2048769736 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f5010084eb0 con 0x7f50180d29d0 2023-10-11T09:41:23.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.521+0000 7f501d9df700 1 --2- 172.21.15.78:0/2048769736 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4ffc065f00 0x7f4ffc0683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:23.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.521+0000 7f501d9df700 1 --2- 172.21.15.78:0/2048769736 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4ffc065f00 0x7f4ffc0683b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f50140011e0 tx=0x7f5014009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:23.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.529+0000 7f500e7fc700 1 -- 172.21.15.78:0/2048769736 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f5010055fe0 con 0x7f50180d29d0 2023-10-11T09:41:23.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.697+0000 7f501e9e1700 1 -- 172.21.15.78:0/2048769736 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5018065d40 con 0x7f50180d29d0 2023-10-11T09:41:23.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.701+0000 7f500e7fc700 1 -- 172.21.15.78:0/2048769736 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5010055b70 con 0x7f50180d29d0 2023-10-11T09:41:23.706 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:41:23.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.701+0000 7f501e9e1700 1 -- 172.21.15.78:0/2048769736 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4ffc065f00 msgr2=0x7f4ffc0683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:23.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.701+0000 7f501e9e1700 1 --2- 172.21.15.78:0/2048769736 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4ffc065f00 0x7f4ffc0683b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f50140011e0 tx=0x7f5014009040 comp rx=0 tx=0).stop 2023-10-11T09:41:23.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.701+0000 7f501e9e1700 1 -- 172.21.15.78:0/2048769736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50180d29d0 msgr2=0x7f50180c9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:23.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.701+0000 7f501e9e1700 1 --2- 172.21.15.78:0/2048769736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50180d29d0 0x7f50180c9260 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f501000b370 tx=0x7f5010004af0 comp rx=0 tx=0).stop 2023-10-11T09:41:23.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.701+0000 7f501e9e1700 1 -- 172.21.15.78:0/2048769736 shutdown_connections 2023-10-11T09:41:23.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.701+0000 7f501e9e1700 1 --2- 172.21.15.78:0/2048769736 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4ffc065f00 0x7f4ffc0683b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:23.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.701+0000 7f501e9e1700 1 --2- 172.21.15.78:0/2048769736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50180d29d0 0x7f50180c9260 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:23.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.701+0000 7f501e9e1700 1 --2- 172.21.15.78:0/2048769736 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f50180b3610 0x7f501815ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:23.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.701+0000 7f501e9e1700 1 --2- 172.21.15.78:0/2048769736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50180b3240 0x7f501815a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:23.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.701+0000 7f501e9e1700 1 -- 172.21.15.78:0/2048769736 >> 172.21.15.78:0/2048769736 conn(0x7f501809c260 msgr2=0x7f501806c530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:23.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.701+0000 7f501e9e1700 1 -- 172.21.15.78:0/2048769736 shutdown_connections 2023-10-11T09:41:23.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.701+0000 7f501e9e1700 1 -- 172.21.15.78:0/2048769736 wait complete. 2023-10-11T09:41:23.730 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T09:41:23.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.897+0000 7fb7c1e34700 1 Processor -- start 2023-10-11T09:41:23.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.897+0000 7fb7c1e34700 1 -- start start 2023-10-11T09:41:23.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.897+0000 7fb7c1e34700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7bc158430 0x7fb7bc0710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:23.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.897+0000 7fb7c1e34700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb7bc06d340 0x7fb7bc0ca880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:23.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.897+0000 7fb7c1e34700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7bc071610 con 0x7fb7bc0cadc0 2023-10-11T09:41:23.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.897+0000 7fb7c1e34700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7bc070c10 con 0x7fb7bc06d340 2023-10-11T09:41:23.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.897+0000 7fb7c1e34700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7bc0cb240 con 0x7fb7bc158430 2023-10-11T09:41:23.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.901+0000 7fb7baffd700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb7bc06d340 0x7fb7bc0ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:23.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.901+0000 7fb7bb7fe700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7bc158430 0x7fb7bc0710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:23.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.901+0000 7fb7baffd700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb7bc06d340 0x7fb7bc0ca880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:53720/0 (socket says 172.21.15.78:53720) 2023-10-11T09:41:23.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.901+0000 7fb7baffd700 1 -- 172.21.15.78:0/2143062358 learned_addr learned my addr 172.21.15.78:0/2143062358 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:23.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.901+0000 7fb7baffd700 1 -- 172.21.15.78:0/2143062358 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7bc158430 msgr2=0x7fb7bc0710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:23.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.901+0000 7fb7baffd700 1 --2- 172.21.15.78:0/2143062358 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7bc158430 0x7fb7bc0710d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:23.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.901+0000 7fb7baffd700 1 -- 172.21.15.78:0/2143062358 >> v1:172.21.15.133:6789/0 conn(0x7fb7bc0cadc0 legacy=0x7fb7bc15afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:23.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.901+0000 7fb7baffd700 1 -- 172.21.15.78:0/2143062358 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7bc15e280 con 0x7fb7bc06d340 2023-10-11T09:41:23.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.901+0000 7fb7baffd700 1 --2- 172.21.15.78:0/2143062358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7bc06d340 0x7fb7bc0ca880 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0009910 tx=0x7fb7b0009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66f81df33b0db31b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:23.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.901+0000 7fb7ba7fc700 1 -- 172.21.15.78:0/2143062358 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b0011830 con 0x7fb7bc06d340 2023-10-11T09:41:23.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.901+0000 7fb7ba7fc700 1 -- 172.21.15.78:0/2143062358 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b0011990 con 0x7fb7bc06d340 2023-10-11T09:41:23.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.901+0000 7fb7ba7fc700 1 -- 172.21.15.78:0/2143062358 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b0011d40 con 0x7fb7bc06d340 2023-10-11T09:41:23.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.901+0000 7fb7c1e34700 1 -- 172.21.15.78:0/2143062358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7bc06d340 msgr2=0x7fb7bc0ca880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:23.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.901+0000 7fb7c1e34700 1 --2- 172.21.15.78:0/2143062358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7bc06d340 0x7fb7bc0ca880 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0009910 tx=0x7fb7b0009c20 comp rx=0 tx=0).stop 2023-10-11T09:41:23.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.901+0000 7fb7c1e34700 1 -- 172.21.15.78:0/2143062358 shutdown_connections 2023-10-11T09:41:23.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.901+0000 7fb7c1e34700 1 --2- 172.21.15.78:0/2143062358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7bc06d340 0x7fb7bc0ca880 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:23.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.901+0000 7fb7c1e34700 1 --2- 172.21.15.78:0/2143062358 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7bc158430 0x7fb7bc0710d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:23.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.901+0000 7fb7c1e34700 1 -- 172.21.15.78:0/2143062358 >> 172.21.15.78:0/2143062358 conn(0x7fb7bc0cecc0 msgr2=0x7fb7bc0cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:23.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.901+0000 7fb7c1e34700 1 -- 172.21.15.78:0/2143062358 shutdown_connections 2023-10-11T09:41:23.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.901+0000 7fb7c1e34700 1 -- 172.21.15.78:0/2143062358 wait complete. 2023-10-11T09:41:23.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.905+0000 7fb7c1e34700 1 Processor -- start 2023-10-11T09:41:23.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.905+0000 7fb7c1e34700 1 -- start start 2023-10-11T09:41:23.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.909+0000 7fb7c1e34700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7bc06d340 0x7fb7bc149f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:23.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.909+0000 7fb7c1e34700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7bc0cadc0 0x7fb7bc15bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:23.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.909+0000 7fb7c1e34700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7bc158430 0x7fb7bc15c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:23.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.909+0000 7fb7c1e34700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7bc0a9e00 con 0x7fb7bc0cadc0 2023-10-11T09:41:23.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.909+0000 7fb7c1e34700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7bc0a91f0 con 0x7fb7bc06d340 2023-10-11T09:41:23.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.909+0000 7fb7c1e34700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7bc0a94d0 con 0x7fb7bc158430 2023-10-11T09:41:23.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.909+0000 7fb7baffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7bc0cadc0 0x7fb7bc15bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:23.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.909+0000 7fb7baffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7bc0cadc0 0x7fb7bc15bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50964/0 (socket says 172.21.15.78:50964) 2023-10-11T09:41:23.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.909+0000 7fb7baffd700 1 -- 172.21.15.78:0/2605390690 learned_addr learned my addr 172.21.15.78:0/2605390690 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:23.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.909+0000 7fb7bb7fe700 1 --2- 172.21.15.78:0/2605390690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7bc06d340 0x7fb7bc149f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:23.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.909+0000 7fb7bbfff700 1 --2- 172.21.15.78:0/2605390690 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7bc158430 0x7fb7bc15c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:23.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.909+0000 7fb7baffd700 1 -- 172.21.15.78:0/2605390690 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7bc158430 msgr2=0x7fb7bc15c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:23.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.909+0000 7fb7baffd700 1 --2- 172.21.15.78:0/2605390690 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7bc158430 0x7fb7bc15c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:23.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.909+0000 7fb7baffd700 1 -- 172.21.15.78:0/2605390690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7bc06d340 msgr2=0x7fb7bc149f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:23.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.909+0000 7fb7baffd700 1 --2- 172.21.15.78:0/2605390690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7bc06d340 0x7fb7bc149f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:23.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.909+0000 7fb7baffd700 1 -- 172.21.15.78:0/2605390690 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b0009570 con 0x7fb7bc0cadc0 2023-10-11T09:41:23.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.909+0000 7fb7baffd700 1 --2- 172.21.15.78:0/2605390690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7bc0cadc0 0x7fb7bc15bf30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0000c00 tx=0x7fb7b0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:23.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.909+0000 7fb7a3fff700 1 -- 172.21.15.78:0/2605390690 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b0004510 con 0x7fb7bc0cadc0 2023-10-11T09:41:23.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.909+0000 7fb7c1e34700 1 -- 172.21.15.78:0/2605390690 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7bc0a97b0 con 0x7fb7bc0cadc0 2023-10-11T09:41:23.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.909+0000 7fb7a3fff700 1 -- 172.21.15.78:0/2605390690 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b00036c0 con 0x7fb7bc0cadc0 2023-10-11T09:41:23.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.909+0000 7fb7a3fff700 1 -- 172.21.15.78:0/2605390690 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b0003a70 con 0x7fb7bc0cadc0 2023-10-11T09:41:23.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.909+0000 7fb7c1e34700 1 -- 172.21.15.78:0/2605390690 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7bc14a460 con 0x7fb7bc0cadc0 2023-10-11T09:41:23.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.913+0000 7fb7c1e34700 1 -- 172.21.15.78:0/2605390690 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7bc0a9fb0 con 0x7fb7bc0cadc0 2023-10-11T09:41:23.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.913+0000 7fb7a3fff700 1 -- 172.21.15.78:0/2605390690 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fb7b0003bd0 con 0x7fb7bc0cadc0 2023-10-11T09:41:23.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.913+0000 7fb7a3fff700 1 --2- 172.21.15.78:0/2605390690 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb7a8065ed0 0x7fb7a8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:23.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.913+0000 7fb7a3fff700 1 -- 172.21.15.78:0/2605390690 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb7b0021030 con 0x7fb7bc0cadc0 2023-10-11T09:41:23.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.917+0000 7fb7bb7fe700 1 --2- 172.21.15.78:0/2605390690 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb7a8065ed0 0x7fb7a8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:23.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.917+0000 7fb7bb7fe700 1 --2- 172.21.15.78:0/2605390690 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb7a8065ed0 0x7fb7a8068380 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac009b50 tx=0x7fb7ac009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:23.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:23.921+0000 7fb7a3fff700 1 -- 172.21.15.78:0/2605390690 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fb7bc0a9fb0 con 0x7fb7bc0cadc0 2023-10-11T09:41:24.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.093+0000 7fb7c1e34700 1 -- 172.21.15.78:0/2605390690 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb7bc0a9f40 con 0x7fb7bc0cadc0 2023-10-11T09:41:24.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.093+0000 7fb7a3fff700 1 -- 172.21.15.78:0/2605390690 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb7b008b080 con 0x7fb7bc0cadc0 2023-10-11T09:41:24.101 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2023-10-11T09:41:24.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.097+0000 7fb7c1e34700 1 -- 172.21.15.78:0/2605390690 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb7a8065ed0 msgr2=0x7fb7a8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:24.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.097+0000 7fb7c1e34700 1 --2- 172.21.15.78:0/2605390690 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb7a8065ed0 0x7fb7a8068380 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac009b50 tx=0x7fb7ac009240 comp rx=0 tx=0).stop 2023-10-11T09:41:24.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.097+0000 7fb7c1e34700 1 -- 172.21.15.78:0/2605390690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7bc0cadc0 msgr2=0x7fb7bc15bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:24.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.097+0000 7fb7c1e34700 1 --2- 172.21.15.78:0/2605390690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7bc0cadc0 0x7fb7bc15bf30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0000c00 tx=0x7fb7b0004770 comp rx=0 tx=0).stop 2023-10-11T09:41:24.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.097+0000 7fb7c1e34700 1 -- 172.21.15.78:0/2605390690 shutdown_connections 2023-10-11T09:41:24.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.097+0000 7fb7c1e34700 1 --2- 172.21.15.78:0/2605390690 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb7a8065ed0 0x7fb7a8068380 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:24.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.097+0000 7fb7c1e34700 1 --2- 172.21.15.78:0/2605390690 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7bc158430 0x7fb7bc15c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:24.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.097+0000 7fb7c1e34700 1 --2- 172.21.15.78:0/2605390690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7bc0cadc0 0x7fb7bc15bf30 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:24.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.097+0000 7fb7c1e34700 1 --2- 172.21.15.78:0/2605390690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7bc06d340 0x7fb7bc149f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:24.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.097+0000 7fb7c1e34700 1 -- 172.21.15.78:0/2605390690 >> 172.21.15.78:0/2605390690 conn(0x7fb7bc0cecc0 msgr2=0x7fb7bc15d220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:24.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.097+0000 7fb7c1e34700 1 -- 172.21.15.78:0/2605390690 shutdown_connections 2023-10-11T09:41:24.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.097+0000 7fb7c1e34700 1 -- 172.21.15.78:0/2605390690 wait complete. 2023-10-11T09:41:24.126 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T09:41:24.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.293+0000 7fec3359e700 1 Processor -- start 2023-10-11T09:41:24.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.293+0000 7fec3359e700 1 -- start start 2023-10-11T09:41:24.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.293+0000 7fec3359e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fec340cda10 0x7fec340caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:24.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.293+0000 7fec3359e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fec34164d20 0x7fec3406bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:24.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.297+0000 7fec3359e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec34165390 con 0x7fec340cdde0 2023-10-11T09:41:24.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.297+0000 7fec3359e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec341651a0 con 0x7fec340cda10 2023-10-11T09:41:24.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.297+0000 7fec3359e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec340ca440 con 0x7fec34164d20 2023-10-11T09:41:24.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.297+0000 7fec3259c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fec340cda10 0x7fec340caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:24.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.297+0000 7fec31d9b700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fec340cdde0 0x7fec34164610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:55894/0 (socket says 172.21.15.78:55894) 2023-10-11T09:41:24.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.297+0000 7fec3259c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fec340cda10 0x7fec340caf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50976/0 (socket says 172.21.15.78:50976) 2023-10-11T09:41:24.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.297+0000 7fec31d9b700 1 -- 172.21.15.78:0/783489431 learned_addr learned my addr 172.21.15.78:0/783489431 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:24.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.297+0000 7fec32d9d700 1 --2- 172.21.15.78:0/783489431 >> v2:172.21.15.133:3300/0 conn(0x7fec34164d20 0x7fec3406bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:24.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.297+0000 7fec3159a700 1 -- 172.21.15.78:0/783489431 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3390933726 0 0) 0x7fec34165390 con 0x7fec340cdde0 2023-10-11T09:41:24.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.297+0000 7fec3159a700 1 -- 172.21.15.78:0/783489431 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec1c0035d0 con 0x7fec340cdde0 2023-10-11T09:41:24.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.297+0000 7fec3259c700 1 -- 172.21.15.78:0/783489431 >> v2:172.21.15.133:3300/0 conn(0x7fec34164d20 msgr2=0x7fec3406bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:24.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.297+0000 7fec3259c700 1 --2- 172.21.15.78:0/783489431 >> v2:172.21.15.133:3300/0 conn(0x7fec34164d20 0x7fec3406bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:24.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.297+0000 7fec3259c700 1 -- 172.21.15.78:0/783489431 >> v1:172.21.15.78:6789/0 conn(0x7fec340cdde0 legacy=0x7fec34164610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:24.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.297+0000 7fec3259c700 1 -- 172.21.15.78:0/783489431 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec340ca720 con 0x7fec340cda10 2023-10-11T09:41:24.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.297+0000 7fec32d9d700 1 --2- 172.21.15.78:0/783489431 >> v2:172.21.15.133:3300/0 conn(0x7fec34164d20 0x7fec3406bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:24.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.297+0000 7fec3259c700 1 --2- 172.21.15.78:0/783489431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec340cda10 0x7fec340caf20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fec34165670 tx=0x7fec24009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19db93c6d64f8e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:24.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.297+0000 7fec3159a700 1 -- 172.21.15.78:0/783489431 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec24011830 con 0x7fec340cda10 2023-10-11T09:41:24.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.297+0000 7fec3159a700 1 -- 172.21.15.78:0/783489431 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec24011990 con 0x7fec340cda10 2023-10-11T09:41:24.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.297+0000 7fec3159a700 1 -- 172.21.15.78:0/783489431 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec24011d20 con 0x7fec340cda10 2023-10-11T09:41:24.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.297+0000 7fec3359e700 1 -- 172.21.15.78:0/783489431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec340cda10 msgr2=0x7fec340caf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:24.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.297+0000 7fec3359e700 1 --2- 172.21.15.78:0/783489431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec340cda10 0x7fec340caf20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fec34165670 tx=0x7fec24009b30 comp rx=0 tx=0).stop 2023-10-11T09:41:24.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.297+0000 7fec3359e700 1 -- 172.21.15.78:0/783489431 shutdown_connections 2023-10-11T09:41:24.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.297+0000 7fec3359e700 1 --2- 172.21.15.78:0/783489431 >> v2:172.21.15.133:3300/0 conn(0x7fec34164d20 0x7fec3406bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:24.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.297+0000 7fec3359e700 1 --2- 172.21.15.78:0/783489431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec340cda10 0x7fec340caf20 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:24.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.297+0000 7fec3359e700 1 -- 172.21.15.78:0/783489431 >> 172.21.15.78:0/783489431 conn(0x7fec340cea60 msgr2=0x7fec340cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:24.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.301+0000 7fec3359e700 1 -- 172.21.15.78:0/783489431 shutdown_connections 2023-10-11T09:41:24.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.301+0000 7fec3359e700 1 -- 172.21.15.78:0/783489431 wait complete. 2023-10-11T09:41:24.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.301+0000 7fec3359e700 1 Processor -- start 2023-10-11T09:41:24.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.301+0000 7fec3359e700 1 -- start start 2023-10-11T09:41:24.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.301+0000 7fec3359e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fec340cda10 0x7fec3415fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:24.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.301+0000 7fec3359e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec340cdde0 0x7fec34146bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:24.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.301+0000 7fec3359e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec34164d20 0x7fec34070ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:24.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.301+0000 7fec3359e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec340c8a50 con 0x7fec340cdde0 2023-10-11T09:41:24.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.301+0000 7fec3359e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec34160110 con 0x7fec34164d20 2023-10-11T09:41:24.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.301+0000 7fec3359e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec340c3b20 con 0x7fec340cda10 2023-10-11T09:41:24.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.301+0000 7fec31d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec340cdde0 0x7fec34146bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:24.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.301+0000 7fec31d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec340cdde0 0x7fec34146bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50984/0 (socket says 172.21.15.78:50984) 2023-10-11T09:41:24.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.301+0000 7fec31d9b700 1 -- 172.21.15.78:0/4045498128 learned_addr learned my addr 172.21.15.78:0/4045498128 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:24.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.305+0000 7fec32d9d700 1 --2- 172.21.15.78:0/4045498128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec34164d20 0x7fec34070ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:24.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.305+0000 7fec3259c700 1 --2- 172.21.15.78:0/4045498128 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fec340cda10 0x7fec3415fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:24.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.305+0000 7fec31d9b700 1 -- 172.21.15.78:0/4045498128 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fec340cda10 msgr2=0x7fec3415fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:24.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.305+0000 7fec31d9b700 1 --2- 172.21.15.78:0/4045498128 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fec340cda10 0x7fec3415fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:24.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.305+0000 7fec31d9b700 1 -- 172.21.15.78:0/4045498128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec34164d20 msgr2=0x7fec34070ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:24.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.305+0000 7fec31d9b700 1 --2- 172.21.15.78:0/4045498128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec34164d20 0x7fec34070ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:24.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.305+0000 7fec31d9b700 1 -- 172.21.15.78:0/4045498128 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec240094b0 con 0x7fec340cdde0 2023-10-11T09:41:24.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.305+0000 7fec3259c700 1 --2- 172.21.15.78:0/4045498128 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fec340cda10 0x7fec3415fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:24.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.305+0000 7fec31d9b700 1 --2- 172.21.15.78:0/4045498128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec340cdde0 0x7fec34146bf0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fec2800c4d0 tx=0x7fec2800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:24.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.305+0000 7fec22ffd700 1 -- 172.21.15.78:0/4045498128 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec28005d90 con 0x7fec340cdde0 2023-10-11T09:41:24.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.305+0000 7fec22ffd700 1 -- 172.21.15.78:0/4045498128 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec280070d0 con 0x7fec340cdde0 2023-10-11T09:41:24.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.305+0000 7fec3359e700 1 -- 172.21.15.78:0/4045498128 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec341474f0 con 0x7fec340cdde0 2023-10-11T09:41:24.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.305+0000 7fec22ffd700 1 -- 172.21.15.78:0/4045498128 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec28007480 con 0x7fec340cdde0 2023-10-11T09:41:24.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.305+0000 7fec3359e700 1 -- 172.21.15.78:0/4045498128 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec34147880 con 0x7fec340cdde0 2023-10-11T09:41:24.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.305+0000 7fec3359e700 1 -- 172.21.15.78:0/4045498128 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec34065d40 con 0x7fec340cdde0 2023-10-11T09:41:24.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.309+0000 7fec22ffd700 1 -- 172.21.15.78:0/4045498128 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fec280075e0 con 0x7fec340cdde0 2023-10-11T09:41:24.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.309+0000 7fec22ffd700 1 --2- 172.21.15.78:0/4045498128 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fec14065ed0 0x7fec14068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:24.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.309+0000 7fec22ffd700 1 -- 172.21.15.78:0/4045498128 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fec28082d00 con 0x7fec340cdde0 2023-10-11T09:41:24.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.309+0000 7fec3259c700 1 --2- 172.21.15.78:0/4045498128 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fec14065ed0 0x7fec14068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:24.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.309+0000 7fec3259c700 1 --2- 172.21.15.78:0/4045498128 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fec14065ed0 0x7fec14068380 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fec24000c00 tx=0x7fec24008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:24.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.317+0000 7fec22ffd700 1 -- 172.21.15.78:0/4045498128 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fec28050310 con 0x7fec340cdde0 2023-10-11T09:41:24.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.501+0000 7fec3359e700 1 -- 172.21.15.78:0/4045498128 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fec340ca9d0 con 0x7fec340cdde0 2023-10-11T09:41:24.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.501+0000 7fec22ffd700 1 -- 172.21.15.78:0/4045498128 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7fec28017080 con 0x7fec340cdde0 2023-10-11T09:41:24.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.505+0000 7fec3359e700 1 -- 172.21.15.78:0/4045498128 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fec14065ed0 msgr2=0x7fec14068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:24.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.505+0000 7fec3359e700 1 --2- 172.21.15.78:0/4045498128 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fec14065ed0 0x7fec14068380 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fec24000c00 tx=0x7fec24008030 comp rx=0 tx=0).stop 2023-10-11T09:41:24.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.505+0000 7fec3359e700 1 -- 172.21.15.78:0/4045498128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec340cdde0 msgr2=0x7fec34146bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:24.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.505+0000 7fec3359e700 1 --2- 172.21.15.78:0/4045498128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec340cdde0 0x7fec34146bf0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fec2800c4d0 tx=0x7fec2800c7e0 comp rx=0 tx=0).stop 2023-10-11T09:41:24.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.505+0000 7fec3359e700 1 -- 172.21.15.78:0/4045498128 shutdown_connections 2023-10-11T09:41:24.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.505+0000 7fec3359e700 1 --2- 172.21.15.78:0/4045498128 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fec14065ed0 0x7fec14068380 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:24.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.505+0000 7fec3359e700 1 --2- 172.21.15.78:0/4045498128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec34164d20 0x7fec34070ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:24.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.505+0000 7fec3359e700 1 --2- 172.21.15.78:0/4045498128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec340cdde0 0x7fec34146bf0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:24.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.505+0000 7fec3359e700 1 --2- 172.21.15.78:0/4045498128 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fec340cda10 0x7fec3415fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:24.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.505+0000 7fec3359e700 1 -- 172.21.15.78:0/4045498128 >> 172.21.15.78:0/4045498128 conn(0x7fec340cea60 msgr2=0x7fec341640b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:24.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.509+0000 7fec3359e700 1 -- 172.21.15.78:0/4045498128 shutdown_connections 2023-10-11T09:41:24.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.509+0000 7fec3359e700 1 -- 172.21.15.78:0/4045498128 wait complete. 2023-10-11T09:41:24.515 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2023-10-11T09:41:24.534 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:41:24.534 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:41:24.549 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:41:24.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.761+0000 7f279f00d700 1 Processor -- start 2023-10-11T09:41:24.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.761+0000 7f279f00d700 1 -- start start 2023-10-11T09:41:24.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.761+0000 7f279f00d700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f27980d1160 0x7f27980cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:24.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.761+0000 7f279f00d700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f279815dc10 con 0x7f27980d1660 2023-10-11T09:41:24.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.761+0000 7f279f00d700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2798160380 con 0x7f2798156ce0 2023-10-11T09:41:24.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.761+0000 7f279f00d700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f279815d930 con 0x7f27980d1160 2023-10-11T09:41:24.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.761+0000 7f279d80a700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f27980d1660 0x7f27980d0a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:57496/0 (socket says 172.21.15.78:57496) 2023-10-11T09:41:24.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.761+0000 7f279d80a700 1 -- 172.21.15.78:0/690748820 learned_addr learned my addr 172.21.15.78:0/690748820 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:24.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.761+0000 7f279e80c700 1 --2- 172.21.15.78:0/690748820 >> v2:172.21.15.133:3300/0 conn(0x7f27980d1160 0x7f27980cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:24.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.761+0000 7f279d009700 1 -- 172.21.15.78:0/690748820 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1685800294 0 0) 0x7f279815dc10 con 0x7f27980d1660 2023-10-11T09:41:24.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.761+0000 7f279d009700 1 -- 172.21.15.78:0/690748820 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27840035d0 con 0x7f27980d1660 2023-10-11T09:41:24.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.761+0000 7f279d009700 1 -- 172.21.15.78:0/690748820 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3041405109 0 0) 0x7f27840035d0 con 0x7f27980d1660 2023-10-11T09:41:24.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.761+0000 7f279d009700 1 -- 172.21.15.78:0/690748820 >> v2:172.21.15.133:3300/0 conn(0x7f27980d1160 msgr2=0x7f27980cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:24.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.761+0000 7f279d009700 1 --2- 172.21.15.78:0/690748820 >> v2:172.21.15.133:3300/0 conn(0x7f27980d1160 0x7f27980cefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:24.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.761+0000 7f279d009700 1 -- 172.21.15.78:0/690748820 >> v1:172.21.15.133:6789/0 conn(0x7f2798156ce0 legacy=0x7f2798160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:24.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.761+0000 7f279d009700 1 -- 172.21.15.78:0/690748820 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2798160620 con 0x7f27980d1660 2023-10-11T09:41:24.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.761+0000 7f279d009700 1 -- 172.21.15.78:0/690748820 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f2788001670 con 0x7f27980d1660 2023-10-11T09:41:24.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.765+0000 7f279d009700 1 -- 172.21.15.78:0/690748820 >> v1:172.21.15.78:6789/0 conn(0x7f27980d1660 legacy=0x7f27980d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:24.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.765+0000 7f279d009700 1 --2- 172.21.15.78:0/690748820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2784003a20 0x7f2784003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:24.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.765+0000 7f279d009700 1 --2- 172.21.15.78:0/690748820 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2784004870 0x7f2784004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:24.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.765+0000 7f279d009700 1 --2- 172.21.15.78:0/690748820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2784005340 0x7f278400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:24.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.765+0000 7f279d009700 1 -- 172.21.15.78:0/690748820 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279815d930 con 0x7f2784003a20 2023-10-11T09:41:24.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.765+0000 7f279d009700 1 -- 172.21.15.78:0/690748820 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278400c200 con 0x7f2784005340 2023-10-11T09:41:24.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.765+0000 7f279d009700 1 -- 172.21.15.78:0/690748820 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f278400c3c0 con 0x7f2784004870 2023-10-11T09:41:24.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.765+0000 7f279e80c700 1 --2- 172.21.15.78:0/690748820 >> v2:172.21.15.133:3300/0 conn(0x7f27980d1160 0x7f27980cefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:24.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.765+0000 7f279e00b700 1 --2- 172.21.15.78:0/690748820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2784003a20 0x7f2784003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:24.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.765+0000 7f279e00b700 1 -- 172.21.15.78:0/690748820 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2784004870 msgr2=0x7f2784004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:24.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.765+0000 7f279e00b700 1 --2- 172.21.15.78:0/690748820 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2784004870 0x7f2784004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:24.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.765+0000 7f279e00b700 1 -- 172.21.15.78:0/690748820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2784005340 msgr2=0x7f278400ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:24.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.765+0000 7f279e00b700 1 --2- 172.21.15.78:0/690748820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2784005340 0x7f278400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:24.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.765+0000 7f279e00b700 1 -- 172.21.15.78:0/690748820 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27840035d0 con 0x7f2784003a20 2023-10-11T09:41:24.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.765+0000 7f279e80c700 1 -- 172.21.15.78:0/690748820 reap_dead start 2023-10-11T09:41:24.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.765+0000 7f279e00b700 1 --2- 172.21.15.78:0/690748820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2784003a20 0x7f2784003e30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f27940017e0 tx=0x7f27840058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=716aa518db5140cf server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:24.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.765+0000 7f279d009700 1 -- 172.21.15.78:0/690748820 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27940052d0 con 0x7f2784003a20 2023-10-11T09:41:24.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.765+0000 7f279d009700 1 -- 172.21.15.78:0/690748820 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2794005430 con 0x7f2784003a20 2023-10-11T09:41:24.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.765+0000 7f279d009700 1 -- 172.21.15.78:0/690748820 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27940055a0 con 0x7f2784003a20 2023-10-11T09:41:24.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.765+0000 7f279f00d700 1 -- 172.21.15.78:0/690748820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2784003a20 msgr2=0x7f2784003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:24.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.765+0000 7f279f00d700 1 --2- 172.21.15.78:0/690748820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2784003a20 0x7f2784003e30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f27940017e0 tx=0x7f27840058a0 comp rx=0 tx=0).stop 2023-10-11T09:41:24.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.765+0000 7f279f00d700 1 -- 172.21.15.78:0/690748820 shutdown_connections 2023-10-11T09:41:24.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.765+0000 7f279f00d700 1 --2- 172.21.15.78:0/690748820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2784003a20 0x7f2784003e30 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:24.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.765+0000 7f279f00d700 1 -- 172.21.15.78:0/690748820 >> 172.21.15.78:0/690748820 conn(0x7f279806afb0 msgr2=0x7f279806b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:24.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.765+0000 7f279f00d700 1 -- 172.21.15.78:0/690748820 shutdown_connections 2023-10-11T09:41:24.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.765+0000 7f279f00d700 1 -- 172.21.15.78:0/690748820 wait complete. 2023-10-11T09:41:24.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.769+0000 7f279f00d700 1 Processor -- start 2023-10-11T09:41:24.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.769+0000 7f279f00d700 1 -- start start 2023-10-11T09:41:24.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.769+0000 7f279f00d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2784003a20 0x7f279814ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:24.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.769+0000 7f279f00d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f27980c9f50 0x7f27980ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:24.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.769+0000 7f279f00d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27981e72f0 0x7f27981e7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:24.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.769+0000 7f279f00d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279814b390 con 0x7f27981e72f0 2023-10-11T09:41:24.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.769+0000 7f279f00d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27981e6250 con 0x7f2784003a20 2023-10-11T09:41:24.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.769+0000 7f279f00d700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27981e6530 con 0x7f27980c9f50 2023-10-11T09:41:24.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.769+0000 7f279e80c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27981e72f0 0x7f27981e7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:24.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.769+0000 7f279e80c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27981e72f0 0x7f27981e7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59322/0 (socket says 172.21.15.78:59322) 2023-10-11T09:41:24.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.769+0000 7f279e80c700 1 -- 172.21.15.78:0/4061946243 learned_addr learned my addr 172.21.15.78:0/4061946243 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:24.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.769+0000 7f279e00b700 1 --2- 172.21.15.78:0/4061946243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2784003a20 0x7f279814ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:24.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.769+0000 7f279d80a700 1 --2- 172.21.15.78:0/4061946243 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f27980c9f50 0x7f27980ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:24.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.769+0000 7f279e80c700 1 -- 172.21.15.78:0/4061946243 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f27980c9f50 msgr2=0x7f27980ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:24.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.769+0000 7f279e80c700 1 --2- 172.21.15.78:0/4061946243 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f27980c9f50 0x7f27980ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:24.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.769+0000 7f279e80c700 1 -- 172.21.15.78:0/4061946243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2784003a20 msgr2=0x7f279814ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:24.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.769+0000 7f279e80c700 1 --2- 172.21.15.78:0/4061946243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2784003a20 0x7f279814ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:24.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.769+0000 7f279e80c700 1 -- 172.21.15.78:0/4061946243 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27981e6810 con 0x7f27981e72f0 2023-10-11T09:41:24.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.769+0000 7f279e00b700 1 --2- 172.21.15.78:0/4061946243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2784003a20 0x7f279814ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:24.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.773+0000 7f279e80c700 1 --2- 172.21.15.78:0/4061946243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27981e72f0 0x7f27981e7790 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f27980ce820 tx=0x7f279000d0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:24.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.773+0000 7f278effd700 1 -- 172.21.15.78:0/4061946243 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2790003c40 con 0x7f27981e72f0 2023-10-11T09:41:24.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.773+0000 7f278effd700 1 -- 172.21.15.78:0/4061946243 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f279000f040 con 0x7f27981e72f0 2023-10-11T09:41:24.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.773+0000 7f278effd700 1 -- 172.21.15.78:0/4061946243 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279000d960 con 0x7f27981e72f0 2023-10-11T09:41:24.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.773+0000 7f279f00d700 1 -- 172.21.15.78:0/4061946243 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27981e6af0 con 0x7f27981e72f0 2023-10-11T09:41:24.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.773+0000 7f279f00d700 1 -- 172.21.15.78:0/4061946243 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27980b7b00 con 0x7f27981e72f0 2023-10-11T09:41:24.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.773+0000 7f279f00d700 1 -- 172.21.15.78:0/4061946243 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27981e7010 con 0x7f27981e72f0 2023-10-11T09:41:24.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.773+0000 7f278effd700 1 -- 172.21.15.78:0/4061946243 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f279000dae0 con 0x7f27981e72f0 2023-10-11T09:41:24.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.777+0000 7f278effd700 1 --2- 172.21.15.78:0/4061946243 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f277c065ed0 0x7f277c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:24.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.777+0000 7f278effd700 1 -- 172.21.15.78:0/4061946243 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f2790083610 con 0x7f27981e72f0 2023-10-11T09:41:24.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.777+0000 7f279e00b700 1 --2- 172.21.15.78:0/4061946243 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f277c065ed0 0x7f277c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:24.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.777+0000 7f279e00b700 1 --2- 172.21.15.78:0/4061946243 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f277c065ed0 0x7f277c068380 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2794005040 tx=0x7f2794009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:24.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.781+0000 7f278effd700 1 -- 172.21.15.78:0/4061946243 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f27981e7010 con 0x7f27981e72f0 2023-10-11T09:41:24.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.957+0000 7f279f00d700 1 -- 172.21.15.78:0/4061946243 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f27981e7010 con 0x7f27981e72f0 2023-10-11T09:41:24.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.961+0000 7f278effd700 1 -- 172.21.15.78:0/4061946243 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17907 (secure 0 0 0) 0x7f27981e7010 con 0x7f27981e72f0 2023-10-11T09:41:24.966 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:41:24.966 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":25,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:41:21.473566+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:40:39.862581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T09:40:40.663641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.37","mappings":[{"from":3,"to":2}]},{"pgid":"4.3e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:41:24.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.961+0000 7f279f00d700 1 -- 172.21.15.78:0/4061946243 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f277c065ed0 msgr2=0x7f277c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:24.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.961+0000 7f279f00d700 1 --2- 172.21.15.78:0/4061946243 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f277c065ed0 0x7f277c068380 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2794005040 tx=0x7f2794009030 comp rx=0 tx=0).stop 2023-10-11T09:41:24.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.961+0000 7f279f00d700 1 -- 172.21.15.78:0/4061946243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27981e72f0 msgr2=0x7f27981e7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:24.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.961+0000 7f279f00d700 1 --2- 172.21.15.78:0/4061946243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27981e72f0 0x7f27981e7790 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f27980ce820 tx=0x7f279000d0d0 comp rx=0 tx=0).stop 2023-10-11T09:41:24.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.961+0000 7f279f00d700 1 -- 172.21.15.78:0/4061946243 shutdown_connections 2023-10-11T09:41:24.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.961+0000 7f279f00d700 1 --2- 172.21.15.78:0/4061946243 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f277c065ed0 0x7f277c068380 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:24.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.961+0000 7f279f00d700 1 --2- 172.21.15.78:0/4061946243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27981e72f0 0x7f27981e7790 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:24.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.961+0000 7f279f00d700 1 --2- 172.21.15.78:0/4061946243 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f27980c9f50 0x7f27980ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:24.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.961+0000 7f279f00d700 1 --2- 172.21.15.78:0/4061946243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2784003a20 0x7f279814ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:24.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.961+0000 7f279f00d700 1 -- 172.21.15.78:0/4061946243 >> 172.21.15.78:0/4061946243 conn(0x7f279806afb0 msgr2=0x7f27981554c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:24.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.961+0000 7f279f00d700 1 -- 172.21.15.78:0/4061946243 shutdown_connections 2023-10-11T09:41:24.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:24.961+0000 7f279f00d700 1 -- 172.21.15.78:0/4061946243 wait complete. 2023-10-11T09:41:24.988 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T09:40:39.862581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T09:40:40.663641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:41:24.989 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:41:25.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.153+0000 7f81ca628700 1 Processor -- start 2023-10-11T09:41:25.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.153+0000 7f81ca628700 1 -- start start 2023-10-11T09:41:25.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.157+0000 7f81ca628700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f81c415a6b0 0x7f81c40ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:25.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.157+0000 7f81ca628700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f81c406d730 0x7f81c406bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:25.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.157+0000 7f81ca628700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81c40b2d30 con 0x7f81c40b3a60 2023-10-11T09:41:25.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.157+0000 7f81ca628700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81c40cf360 con 0x7f81c415a6b0 2023-10-11T09:41:25.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.157+0000 7f81ca628700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81c40b2f80 con 0x7f81c406d730 2023-10-11T09:41:25.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.157+0000 7f81c9626700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f81c415a6b0 0x7f81c40ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:25.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.157+0000 7f81c9626700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f81c415a6b0 0x7f81c40ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59338/0 (socket says 172.21.15.78:59338) 2023-10-11T09:41:25.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.157+0000 7f81c9626700 1 -- 172.21.15.78:0/1352184514 learned_addr learned my addr 172.21.15.78:0/1352184514 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:25.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.157+0000 7f81c9e27700 1 --2- 172.21.15.78:0/1352184514 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f81c406d730 0x7f81c406bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:25.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.157+0000 7f81c9626700 1 -- 172.21.15.78:0/1352184514 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f81c406d730 msgr2=0x7f81c406bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:25.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.157+0000 7f81c9626700 1 --2- 172.21.15.78:0/1352184514 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f81c406d730 0x7f81c406bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:25.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.157+0000 7f81c9626700 1 -- 172.21.15.78:0/1352184514 >> v1:172.21.15.133:6789/0 conn(0x7f81c40b3a60 legacy=0x7f81c406d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:25.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.157+0000 7f81c9626700 1 -- 172.21.15.78:0/1352184514 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c40b3260 con 0x7f81c415a6b0 2023-10-11T09:41:25.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.157+0000 7f81c9e27700 1 --2- 172.21.15.78:0/1352184514 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f81c406d730 0x7f81c406bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:25.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.157+0000 7f81c9626700 1 --2- 172.21.15.78:0/1352184514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81c415a6b0 0x7f81c40ce8a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f81c0009910 tx=0x7f81c0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6536e1f857b4f87 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:25.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.157+0000 7f81bbfff700 1 -- 172.21.15.78:0/1352184514 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c000ba60 con 0x7f81c415a6b0 2023-10-11T09:41:25.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.157+0000 7f81bbfff700 1 -- 172.21.15.78:0/1352184514 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81c000bbc0 con 0x7f81c415a6b0 2023-10-11T09:41:25.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.157+0000 7f81bbfff700 1 -- 172.21.15.78:0/1352184514 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c0009e60 con 0x7f81c415a6b0 2023-10-11T09:41:25.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.161+0000 7f81ca628700 1 -- 172.21.15.78:0/1352184514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81c415a6b0 msgr2=0x7f81c40ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:25.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.161+0000 7f81ca628700 1 --2- 172.21.15.78:0/1352184514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81c415a6b0 0x7f81c40ce8a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f81c0009910 tx=0x7f81c0009c20 comp rx=0 tx=0).stop 2023-10-11T09:41:25.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.161+0000 7f81ca628700 1 -- 172.21.15.78:0/1352184514 shutdown_connections 2023-10-11T09:41:25.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.161+0000 7f81ca628700 1 --2- 172.21.15.78:0/1352184514 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f81c406d730 0x7f81c406bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:25.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.161+0000 7f81ca628700 1 --2- 172.21.15.78:0/1352184514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81c415a6b0 0x7f81c40ce8a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:25.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.161+0000 7f81ca628700 1 -- 172.21.15.78:0/1352184514 >> 172.21.15.78:0/1352184514 conn(0x7f81c406a060 msgr2=0x7f81c406a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:25.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.161+0000 7f81ca628700 1 -- 172.21.15.78:0/1352184514 shutdown_connections 2023-10-11T09:41:25.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.161+0000 7f81ca628700 1 -- 172.21.15.78:0/1352184514 wait complete. 2023-10-11T09:41:25.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.165+0000 7f81ca628700 1 Processor -- start 2023-10-11T09:41:25.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.165+0000 7f81ca628700 1 -- start start 2023-10-11T09:41:25.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.165+0000 7f81ca628700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81c406d730 0x7f81c41539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:25.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.165+0000 7f81ca628700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81c40b3a60 0x7f81c414f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:25.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.165+0000 7f81ca628700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f81c415a6b0 0x7f81c4155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:25.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.165+0000 7f81ca628700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c41511a0 con 0x7f81c406d730 2023-10-11T09:41:25.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.165+0000 7f81ca628700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c4151fc0 con 0x7f81c40b3a60 2023-10-11T09:41:25.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.165+0000 7f81ca628700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81c41513b0 con 0x7f81c415a6b0 2023-10-11T09:41:25.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.165+0000 7f81c9626700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81c406d730 0x7f81c41539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:25.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.165+0000 7f81c9626700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81c406d730 0x7f81c41539a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59342/0 (socket says 172.21.15.78:59342) 2023-10-11T09:41:25.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.165+0000 7f81c9626700 1 -- 172.21.15.78:0/121216603 learned_addr learned my addr 172.21.15.78:0/121216603 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:25.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.169+0000 7f81c9e27700 1 --2- 172.21.15.78:0/121216603 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f81c415a6b0 0x7f81c4155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:25.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.169+0000 7f81c8e25700 1 --2- 172.21.15.78:0/121216603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81c40b3a60 0x7f81c414f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:25.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.169+0000 7f81c9626700 1 -- 172.21.15.78:0/121216603 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f81c415a6b0 msgr2=0x7f81c4155270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:25.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.169+0000 7f81c9626700 1 --2- 172.21.15.78:0/121216603 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f81c415a6b0 0x7f81c4155270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:25.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.169+0000 7f81c9626700 1 -- 172.21.15.78:0/121216603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81c40b3a60 msgr2=0x7f81c414f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:25.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.169+0000 7f81c9626700 1 --2- 172.21.15.78:0/121216603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81c40b3a60 0x7f81c414f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:25.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.169+0000 7f81c9626700 1 -- 172.21.15.78:0/121216603 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c0009570 con 0x7f81c406d730 2023-10-11T09:41:25.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.169+0000 7f81c9e27700 1 --2- 172.21.15.78:0/121216603 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f81c415a6b0 0x7f81c4155270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:25.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.169+0000 7f81c8e25700 1 --2- 172.21.15.78:0/121216603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81c40b3a60 0x7f81c414f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:25.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.169+0000 7f81c9626700 1 --2- 172.21.15.78:0/121216603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81c406d730 0x7f81c41539a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f81c0014040 tx=0x7f81c0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:25.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.169+0000 7f81b9ffb700 1 -- 172.21.15.78:0/121216603 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c0004610 con 0x7f81c406d730 2023-10-11T09:41:25.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.169+0000 7f81b9ffb700 1 -- 172.21.15.78:0/121216603 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81c001e070 con 0x7f81c406d730 2023-10-11T09:41:25.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.169+0000 7f81b9ffb700 1 -- 172.21.15.78:0/121216603 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c0003920 con 0x7f81c406d730 2023-10-11T09:41:25.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.169+0000 7f81ca628700 1 -- 172.21.15.78:0/121216603 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81c4151690 con 0x7f81c406d730 2023-10-11T09:41:25.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.169+0000 7f81ca628700 1 -- 172.21.15.78:0/121216603 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81c4151c50 con 0x7f81c406d730 2023-10-11T09:41:25.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.169+0000 7f81ca628700 1 -- 172.21.15.78:0/121216603 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81c4156150 con 0x7f81c406d730 2023-10-11T09:41:25.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.173+0000 7f81b9ffb700 1 -- 172.21.15.78:0/121216603 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f81c0003a80 con 0x7f81c406d730 2023-10-11T09:41:25.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.173+0000 7f81b9ffb700 1 --2- 172.21.15.78:0/121216603 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f81a8065f00 0x7f81a80683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:25.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.173+0000 7f81b9ffb700 1 -- 172.21.15.78:0/121216603 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f81c0020070 con 0x7f81c406d730 2023-10-11T09:41:25.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.173+0000 7f81c8e25700 1 --2- 172.21.15.78:0/121216603 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f81a8065f00 0x7f81a80683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:25.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.177+0000 7f81c8e25700 1 --2- 172.21.15.78:0/121216603 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f81a8065f00 0x7f81a80683b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f81c41f33a0 tx=0x7f81b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:25.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.181+0000 7f81b9ffb700 1 -- 172.21.15.78:0/121216603 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f81c0052690 con 0x7f81c406d730 2023-10-11T09:41:25.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.357+0000 7f81ca628700 1 -- 172.21.15.78:0/121216603 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f81c4065d40 con 0x7f81c406d730 2023-10-11T09:41:25.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.357+0000 7f81b9ffb700 1 -- 172.21.15.78:0/121216603 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f81c00150a0 con 0x7f81c406d730 2023-10-11T09:41:25.363 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:41:25.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.361+0000 7f81ca628700 1 -- 172.21.15.78:0/121216603 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f81a8065f00 msgr2=0x7f81a80683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:25.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.361+0000 7f81ca628700 1 --2- 172.21.15.78:0/121216603 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f81a8065f00 0x7f81a80683b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f81c41f33a0 tx=0x7f81b4009040 comp rx=0 tx=0).stop 2023-10-11T09:41:25.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.361+0000 7f81ca628700 1 -- 172.21.15.78:0/121216603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81c406d730 msgr2=0x7f81c41539a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:25.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.361+0000 7f81ca628700 1 --2- 172.21.15.78:0/121216603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81c406d730 0x7f81c41539a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f81c0014040 tx=0x7f81c0004770 comp rx=0 tx=0).stop 2023-10-11T09:41:25.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.361+0000 7f81ca628700 1 -- 172.21.15.78:0/121216603 shutdown_connections 2023-10-11T09:41:25.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.361+0000 7f81ca628700 1 --2- 172.21.15.78:0/121216603 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f81a8065f00 0x7f81a80683b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:25.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.361+0000 7f81ca628700 1 --2- 172.21.15.78:0/121216603 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f81c415a6b0 0x7f81c4155270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:25.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.361+0000 7f81ca628700 1 --2- 172.21.15.78:0/121216603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81c40b3a60 0x7f81c414f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:25.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.361+0000 7f81ca628700 1 --2- 172.21.15.78:0/121216603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81c406d730 0x7f81c41539a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:25.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.361+0000 7f81ca628700 1 -- 172.21.15.78:0/121216603 >> 172.21.15.78:0/121216603 conn(0x7f81c406a060 msgr2=0x7f81c415ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:25.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.361+0000 7f81ca628700 1 -- 172.21.15.78:0/121216603 shutdown_connections 2023-10-11T09:41:25.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.361+0000 7f81ca628700 1 -- 172.21.15.78:0/121216603 wait complete. 2023-10-11T09:41:25.387 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:41:25.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.553+0000 7fcfb5241700 1 Processor -- start 2023-10-11T09:41:25.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.553+0000 7fcfb5241700 1 -- start start 2023-10-11T09:41:25.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.557+0000 7fcfb5241700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcfb00b3610 0x7fcfb00d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:25.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.557+0000 7fcfb5241700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfb00b47f0 con 0x7fcfb00b3240 2023-10-11T09:41:25.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.557+0000 7fcfb5241700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfb00b8130 con 0x7fcfb00b3a90 2023-10-11T09:41:25.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.557+0000 7fcfb5241700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb00b3f10 con 0x7fcfb00b3610 2023-10-11T09:41:25.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.557+0000 7fcfaffff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fcfb00b3240 0x7fcfb015d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:57502/0 (socket says 172.21.15.78:57502) 2023-10-11T09:41:25.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.557+0000 7fcfaffff700 1 -- 172.21.15.78:0/339183207 learned_addr learned my addr 172.21.15.78:0/339183207 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:25.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.557+0000 7fcfaf7fe700 1 --2- 172.21.15.78:0/339183207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcfb00b3610 0x7fcfb00d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:25.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.557+0000 7fcfaeffd700 1 -- 172.21.15.78:0/339183207 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3495194321 0 0) 0x7fcfa0001110 con 0x7fcfb00b3240 2023-10-11T09:41:25.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.557+0000 7fcfaeffd700 1 -- 172.21.15.78:0/339183207 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf980035d0 con 0x7fcfb00b3240 2023-10-11T09:41:25.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.557+0000 7fcfaeffd700 1 -- 172.21.15.78:0/339183207 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3227359969 0 0) 0x7fcfb00b8130 con 0x7fcfb00b3a90 2023-10-11T09:41:25.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.557+0000 7fcfaeffd700 1 -- 172.21.15.78:0/339183207 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfa0001110 con 0x7fcfb00b3a90 2023-10-11T09:41:25.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.557+0000 7fcfaeffd700 1 -- 172.21.15.78:0/339183207 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2483877982 0 0) 0x7fcf980035d0 con 0x7fcfb00b3240 2023-10-11T09:41:25.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.557+0000 7fcfaeffd700 1 -- 172.21.15.78:0/339183207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcfb00b3610 msgr2=0x7fcfb00d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:25.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.557+0000 7fcfaeffd700 1 --2- 172.21.15.78:0/339183207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcfb00b3610 0x7fcfb00d29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:25.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.557+0000 7fcfaeffd700 1 -- 172.21.15.78:0/339183207 >> v1:172.21.15.133:6789/0 conn(0x7fcfb00b3a90 legacy=0x7fcfb00b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:25.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.557+0000 7fcfaeffd700 1 -- 172.21.15.78:0/339183207 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb00b41e0 con 0x7fcfb00b3240 2023-10-11T09:41:25.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.557+0000 7fcfaeffd700 1 -- 172.21.15.78:0/339183207 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7fcfa0001800 con 0x7fcfb00b3240 2023-10-11T09:41:25.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.557+0000 7fcfaf7fe700 1 --2- 172.21.15.78:0/339183207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcfb00b3610 0x7fcfb00d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:41:25.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.557+0000 7fcfaeffd700 1 -- 172.21.15.78:0/339183207 >> v1:172.21.15.78:6789/0 conn(0x7fcfb00b3240 legacy=0x7fcfb015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:25.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.557+0000 7fcfaeffd700 1 --2- 172.21.15.78:0/339183207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf98003be0 0x7fcf98003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:25.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.557+0000 7fcfaeffd700 1 --2- 172.21.15.78:0/339183207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf98004870 0x7fcf98004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:25.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.557+0000 7fcfaeffd700 1 --2- 172.21.15.78:0/339183207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcfb00b3610 0x7fcf98005240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:25.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.557+0000 7fcfaeffd700 1 -- 172.21.15.78:0/339183207 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb00b3f10 con 0x7fcf98004870 2023-10-11T09:41:25.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.557+0000 7fcfaeffd700 1 -- 172.21.15.78:0/339183207 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9800bb80 con 0x7fcf98003be0 2023-10-11T09:41:25.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.557+0000 7fcfaeffd700 1 -- 172.21.15.78:0/339183207 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9800bd40 con 0x7fcfb00b3610 2023-10-11T09:41:25.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.557+0000 7fcfaf7fe700 1 --2- 172.21.15.78:0/339183207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf98004870 0x7fcf98004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:25.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.557+0000 7fcfaffff700 1 --2- 172.21.15.78:0/339183207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf98003be0 0x7fcf98003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:25.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.557+0000 7fcfb4a40700 1 --2- 172.21.15.78:0/339183207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcfb00b3610 0x7fcf98005240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:25.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.561+0000 7fcfaf7fe700 1 -- 172.21.15.78:0/339183207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcfb00b3610 msgr2=0x7fcf98005240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:25.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.561+0000 7fcfaf7fe700 1 --2- 172.21.15.78:0/339183207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcfb00b3610 0x7fcf98005240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:25.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.561+0000 7fcfaf7fe700 1 -- 172.21.15.78:0/339183207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf98003be0 msgr2=0x7fcf98003fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:25.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.561+0000 7fcfaf7fe700 1 --2- 172.21.15.78:0/339183207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf98003be0 0x7fcf98003fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:25.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.561+0000 7fcfaf7fe700 1 -- 172.21.15.78:0/339183207 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa40094b0 con 0x7fcf98004870 2023-10-11T09:41:25.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.561+0000 7fcfaf7fe700 1 --2- 172.21.15.78:0/339183207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf98004870 0x7fcf98004ce0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4009d10 tx=0x7fcf980058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5b91172db973a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:25.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.561+0000 7fcfaeffd700 1 -- 172.21.15.78:0/339183207 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4009e60 con 0x7fcf98004870 2023-10-11T09:41:25.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.561+0000 7fcfaeffd700 1 -- 172.21.15.78:0/339183207 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa400e9a0 con 0x7fcf98004870 2023-10-11T09:41:25.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.561+0000 7fcfaeffd700 1 -- 172.21.15.78:0/339183207 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa400ed10 con 0x7fcf98004870 2023-10-11T09:41:25.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.561+0000 7fcfb5241700 1 -- 172.21.15.78:0/339183207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf98004870 msgr2=0x7fcf98004ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:25.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.561+0000 7fcfb5241700 1 --2- 172.21.15.78:0/339183207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf98004870 0x7fcf98004ce0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4009d10 tx=0x7fcf980058a0 comp rx=0 tx=0).stop 2023-10-11T09:41:25.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.561+0000 7fcfb4a40700 1 -- 172.21.15.78:0/339183207 reap_dead start 2023-10-11T09:41:25.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.561+0000 7fcfb5241700 1 -- 172.21.15.78:0/339183207 shutdown_connections 2023-10-11T09:41:25.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.561+0000 7fcfb5241700 1 -- 172.21.15.78:0/339183207 >> 172.21.15.78:0/339183207 conn(0x7fcfb009c260 msgr2=0x7fcfb00d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:25.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.561+0000 7fcfb5241700 1 -- 172.21.15.78:0/339183207 shutdown_connections 2023-10-11T09:41:25.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.561+0000 7fcfb5241700 1 -- 172.21.15.78:0/339183207 wait complete. 2023-10-11T09:41:25.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.565+0000 7fcfb5241700 1 Processor -- start 2023-10-11T09:41:25.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.565+0000 7fcfb5241700 1 -- start start 2023-10-11T09:41:25.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.565+0000 7fcfb5241700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf98004870 0x7fcfb015f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:25.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.565+0000 7fcfb5241700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcfb009ccd0 0x7fcfb015a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:25.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.565+0000 7fcfb5241700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfb015ceb0 0x7fcfb00b6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:25.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.565+0000 7fcfb5241700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb0160300 con 0x7fcfb015ceb0 2023-10-11T09:41:25.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.565+0000 7fcfb5241700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb015aa30 con 0x7fcf98004870 2023-10-11T09:41:25.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.565+0000 7fcfb5241700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb015fd40 con 0x7fcfb009ccd0 2023-10-11T09:41:25.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.565+0000 7fcfb4a40700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfb015ceb0 0x7fcfb00b6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:25.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.565+0000 7fcfb4a40700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfb015ceb0 0x7fcfb00b6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59364/0 (socket says 172.21.15.78:59364) 2023-10-11T09:41:25.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.569+0000 7fcfb4a40700 1 -- 172.21.15.78:0/3956346854 learned_addr learned my addr 172.21.15.78:0/3956346854 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:25.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.569+0000 7fcfaf7fe700 1 --2- 172.21.15.78:0/3956346854 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcfb009ccd0 0x7fcfb015a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:25.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.569+0000 7fcfaffff700 1 --2- 172.21.15.78:0/3956346854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf98004870 0x7fcfb015f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:25.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.569+0000 7fcfb4a40700 1 -- 172.21.15.78:0/3956346854 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcfb009ccd0 msgr2=0x7fcfb015a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:25.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.569+0000 7fcfb4a40700 1 --2- 172.21.15.78:0/3956346854 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcfb009ccd0 0x7fcfb015a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:25.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.569+0000 7fcfb4a40700 1 -- 172.21.15.78:0/3956346854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf98004870 msgr2=0x7fcfb015f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:25.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.569+0000 7fcfb4a40700 1 --2- 172.21.15.78:0/3956346854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf98004870 0x7fcfb015f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:25.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.569+0000 7fcfb4a40700 1 -- 172.21.15.78:0/3956346854 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb0160020 con 0x7fcfb015ceb0 2023-10-11T09:41:25.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.569+0000 7fcfaffff700 1 --2- 172.21.15.78:0/3956346854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf98004870 0x7fcfb015f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:25.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.569+0000 7fcfb4a40700 1 --2- 172.21.15.78:0/3956346854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfb015ceb0 0x7fcfb00b6710 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcfb00b3e60 tx=0x7fcfa800d660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:25.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.569+0000 7fcfacff9700 1 -- 172.21.15.78:0/3956346854 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa800dbc0 con 0x7fcfb015ceb0 2023-10-11T09:41:25.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.569+0000 7fcfacff9700 1 -- 172.21.15.78:0/3956346854 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa800dd20 con 0x7fcfb015ceb0 2023-10-11T09:41:25.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.569+0000 7fcfb5241700 1 -- 172.21.15.78:0/3956346854 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfb00c79f0 con 0x7fcfb015ceb0 2023-10-11T09:41:25.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.569+0000 7fcfacff9700 1 -- 172.21.15.78:0/3956346854 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8003c60 con 0x7fcfb015ceb0 2023-10-11T09:41:25.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.569+0000 7fcfb5241700 1 -- 172.21.15.78:0/3956346854 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfb015d320 con 0x7fcfb015ceb0 2023-10-11T09:41:25.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.569+0000 7fcfb5241700 1 -- 172.21.15.78:0/3956346854 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfb0065d40 con 0x7fcfb015ceb0 2023-10-11T09:41:25.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.573+0000 7fcfacff9700 1 -- 172.21.15.78:0/3956346854 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fcfa80050d0 con 0x7fcfb015ceb0 2023-10-11T09:41:25.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.573+0000 7fcfacff9700 1 --2- 172.21.15.78:0/3956346854 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fcf9c065f00 0x7fcf9c0683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:25.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.573+0000 7fcfacff9700 1 -- 172.21.15.78:0/3956346854 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fcfa8083ba0 con 0x7fcfb015ceb0 2023-10-11T09:41:25.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.573+0000 7fcfaffff700 1 --2- 172.21.15.78:0/3956346854 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fcf9c065f00 0x7fcf9c0683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:25.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.577+0000 7fcfaffff700 1 --2- 172.21.15.78:0/3956346854 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fcf9c065f00 0x7fcf9c0683b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcfa00017a0 tx=0x7fcfa0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:25.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.581+0000 7fcfacff9700 1 -- 172.21.15.78:0/3956346854 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fcfa80511a0 con 0x7fcfb015ceb0 2023-10-11T09:41:25.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.753+0000 7fcfb5241700 1 -- 172.21.15.78:0/3956346854 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fcfb00b7d90 con 0x7fcfb015ceb0 2023-10-11T09:41:25.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.753+0000 7fcfacff9700 1 -- 172.21.15.78:0/3956346854 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fcfa80547c0 con 0x7fcfb015ceb0 2023-10-11T09:41:25.761 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:41:25.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.757+0000 7fcfb5241700 1 -- 172.21.15.78:0/3956346854 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fcf9c065f00 msgr2=0x7fcf9c0683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:25.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.757+0000 7fcfb5241700 1 --2- 172.21.15.78:0/3956346854 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fcf9c065f00 0x7fcf9c0683b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcfa00017a0 tx=0x7fcfa0009040 comp rx=0 tx=0).stop 2023-10-11T09:41:25.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.757+0000 7fcfb5241700 1 -- 172.21.15.78:0/3956346854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfb015ceb0 msgr2=0x7fcfb00b6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:25.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.757+0000 7fcfb5241700 1 --2- 172.21.15.78:0/3956346854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfb015ceb0 0x7fcfb00b6710 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcfb00b3e60 tx=0x7fcfa800d660 comp rx=0 tx=0).stop 2023-10-11T09:41:25.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.757+0000 7fcfb5241700 1 -- 172.21.15.78:0/3956346854 shutdown_connections 2023-10-11T09:41:25.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.757+0000 7fcfb5241700 1 --2- 172.21.15.78:0/3956346854 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fcf9c065f00 0x7fcf9c0683b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:25.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.757+0000 7fcfb5241700 1 --2- 172.21.15.78:0/3956346854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfb015ceb0 0x7fcfb00b6710 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:25.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.757+0000 7fcfb5241700 1 --2- 172.21.15.78:0/3956346854 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcfb009ccd0 0x7fcfb015a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:25.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.757+0000 7fcfb5241700 1 --2- 172.21.15.78:0/3956346854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf98004870 0x7fcfb015f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:25.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.757+0000 7fcfb5241700 1 -- 172.21.15.78:0/3956346854 >> 172.21.15.78:0/3956346854 conn(0x7fcfb009c260 msgr2=0x7fcfb00b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:25.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.757+0000 7fcfb5241700 1 -- 172.21.15.78:0/3956346854 shutdown_connections 2023-10-11T09:41:25.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.757+0000 7fcfb5241700 1 -- 172.21.15.78:0/3956346854 wait complete. 2023-10-11T09:41:25.785 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T09:41:25.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.949+0000 7f945b239700 1 Processor -- start 2023-10-11T09:41:25.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.949+0000 7f945b239700 1 -- start start 2023-10-11T09:41:25.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.949+0000 7f945b239700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f945413f9e0 0x7f94540ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:25.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.949+0000 7f945b239700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f94540b1820 0x7f94540b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:25.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.953+0000 7f945b239700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94540cc420 con 0x7f94541601f0 2023-10-11T09:41:25.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.953+0000 7f945b239700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f945413fdb0 con 0x7f945413f9e0 2023-10-11T09:41:25.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.953+0000 7f945b239700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94540b21e0 con 0x7f94540b1820 2023-10-11T09:41:25.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.953+0000 7f945aa38700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f94540b1820 0x7f94540b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:25.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.953+0000 7f945aa38700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f94540b1820 0x7f94540b1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:33392/0 (socket says 172.21.15.78:33392) 2023-10-11T09:41:25.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.953+0000 7f945a237700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f945413f9e0 0x7f94540ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:25.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.953+0000 7f945aa38700 1 -- 172.21.15.78:0/1453703916 learned_addr learned my addr 172.21.15.78:0/1453703916 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:25.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.953+0000 7f945aa38700 1 -- 172.21.15.78:0/1453703916 >> v2:172.21.15.133:3300/0 conn(0x7f945413f9e0 msgr2=0x7f94540ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:25.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.953+0000 7f945aa38700 1 --2- 172.21.15.78:0/1453703916 >> v2:172.21.15.133:3300/0 conn(0x7f945413f9e0 0x7f94540ca1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:25.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.953+0000 7f945aa38700 1 -- 172.21.15.78:0/1453703916 >> v1:172.21.15.133:6789/0 conn(0x7f94541601f0 legacy=0x7f94540c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:25.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.953+0000 7f945aa38700 1 -- 172.21.15.78:0/1453703916 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9454160680 con 0x7f94540b1820 2023-10-11T09:41:25.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.953+0000 7f945aa38700 1 --2- 172.21.15.78:0/1453703916 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f94540b1820 0x7f94540b1ca0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f944c009910 tx=0x7f944c009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb3eb8e018bfd430 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:25.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.953+0000 7f9459235700 1 -- 172.21.15.78:0/1453703916 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944c011830 con 0x7f94540b1820 2023-10-11T09:41:25.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.953+0000 7f9459235700 1 -- 172.21.15.78:0/1453703916 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f944c011990 con 0x7f94540b1820 2023-10-11T09:41:25.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.953+0000 7f9459235700 1 -- 172.21.15.78:0/1453703916 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944c011d40 con 0x7f94540b1820 2023-10-11T09:41:25.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.953+0000 7f945b239700 1 -- 172.21.15.78:0/1453703916 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f94540b1820 msgr2=0x7f94540b1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:25.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.953+0000 7f945b239700 1 --2- 172.21.15.78:0/1453703916 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f94540b1820 0x7f94540b1ca0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f944c009910 tx=0x7f944c009c20 comp rx=0 tx=0).stop 2023-10-11T09:41:25.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.957+0000 7f945b239700 1 -- 172.21.15.78:0/1453703916 shutdown_connections 2023-10-11T09:41:25.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.957+0000 7f945b239700 1 --2- 172.21.15.78:0/1453703916 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f94540b1820 0x7f94540b1ca0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:25.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.957+0000 7f945b239700 1 --2- 172.21.15.78:0/1453703916 >> v2:172.21.15.133:3300/0 conn(0x7f945413f9e0 0x7f94540ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:25.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.957+0000 7f945b239700 1 -- 172.21.15.78:0/1453703916 >> 172.21.15.78:0/1453703916 conn(0x7f945406dd80 msgr2=0x7f945406e180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:25.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.957+0000 7f945b239700 1 -- 172.21.15.78:0/1453703916 shutdown_connections 2023-10-11T09:41:25.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.957+0000 7f945b239700 1 -- 172.21.15.78:0/1453703916 wait complete. 2023-10-11T09:41:25.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.957+0000 7f945b239700 1 Processor -- start 2023-10-11T09:41:25.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.957+0000 7f945b239700 1 -- start start 2023-10-11T09:41:25.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.961+0000 7f945b239700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94540b1820 0x7f945415d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:25.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.961+0000 7f945b239700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f945413f9e0 0x7f94541f3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:25.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.961+0000 7f945b239700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94541601f0 0x7f9454157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:25.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.961+0000 7f945b239700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94541f37b0 con 0x7f94540b1820 2023-10-11T09:41:25.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.961+0000 7f945b239700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94541f39e0 con 0x7f94541601f0 2023-10-11T09:41:25.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.961+0000 7f945b239700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94541f3cc0 con 0x7f945413f9e0 2023-10-11T09:41:25.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.961+0000 7f945a237700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94540b1820 0x7f945415d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:25.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.961+0000 7f945a237700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94540b1820 0x7f945415d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59380/0 (socket says 172.21.15.78:59380) 2023-10-11T09:41:25.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.961+0000 7f945a237700 1 -- 172.21.15.78:0/101650948 learned_addr learned my addr 172.21.15.78:0/101650948 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:25.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.961+0000 7f945aa38700 1 --2- 172.21.15.78:0/101650948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94541601f0 0x7f9454157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:25.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.961+0000 7f9459a36700 1 --2- 172.21.15.78:0/101650948 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f945413f9e0 0x7f94541f3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:25.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.961+0000 7f945a237700 1 -- 172.21.15.78:0/101650948 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f945413f9e0 msgr2=0x7f94541f3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:25.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.961+0000 7f945a237700 1 --2- 172.21.15.78:0/101650948 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f945413f9e0 0x7f94541f3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:25.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.961+0000 7f945a237700 1 -- 172.21.15.78:0/101650948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94541601f0 msgr2=0x7f9454157a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:25.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.961+0000 7f945a237700 1 --2- 172.21.15.78:0/101650948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94541601f0 0x7f9454157a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:25.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.961+0000 7f945a237700 1 -- 172.21.15.78:0/101650948 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f944c009570 con 0x7f94540b1820 2023-10-11T09:41:25.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.961+0000 7f945a237700 1 --2- 172.21.15.78:0/101650948 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94540b1820 0x7f945415d890 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f945000efc0 tx=0x7f945000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:25.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.961+0000 7f944affd700 1 -- 172.21.15.78:0/101650948 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945000cdc0 con 0x7f94540b1820 2023-10-11T09:41:25.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.961+0000 7f944affd700 1 -- 172.21.15.78:0/101650948 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945000f040 con 0x7f94540b1820 2023-10-11T09:41:25.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.961+0000 7f945b239700 1 -- 172.21.15.78:0/101650948 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94541f3fa0 con 0x7f94540b1820 2023-10-11T09:41:25.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.961+0000 7f944affd700 1 -- 172.21.15.78:0/101650948 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94500146a0 con 0x7f94540b1820 2023-10-11T09:41:25.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.961+0000 7f945b239700 1 -- 172.21.15.78:0/101650948 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94541f4560 con 0x7f94540b1820 2023-10-11T09:41:25.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.965+0000 7f945b239700 1 -- 172.21.15.78:0/101650948 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9454065d40 con 0x7f94540b1820 2023-10-11T09:41:25.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.965+0000 7f944affd700 1 -- 172.21.15.78:0/101650948 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f9450014800 con 0x7f94540b1820 2023-10-11T09:41:25.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.965+0000 7f944affd700 1 --2- 172.21.15.78:0/101650948 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9438065ed0 0x7f9438068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:25.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.965+0000 7f944affd700 1 -- 172.21.15.78:0/101650948 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f9450084220 con 0x7f94540b1820 2023-10-11T09:41:25.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.965+0000 7f9459a36700 1 --2- 172.21.15.78:0/101650948 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9438065ed0 0x7f9438068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:25.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.965+0000 7f9459a36700 1 --2- 172.21.15.78:0/101650948 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9438065ed0 0x7f9438068380 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f94540cf7b0 tx=0x7f9444009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:25.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:25.973+0000 7f944affd700 1 -- 172.21.15.78:0/101650948 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f94500517a0 con 0x7f94540b1820 2023-10-11T09:41:26.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.149+0000 7f945b239700 1 -- 172.21.15.78:0/101650948 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f94540cfa10 con 0x7f94540b1820 2023-10-11T09:41:26.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.149+0000 7f944affd700 1 -- 172.21.15.78:0/101650948 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9450054dc0 con 0x7f94540b1820 2023-10-11T09:41:26.157 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:41:26.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.153+0000 7f945b239700 1 -- 172.21.15.78:0/101650948 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9438065ed0 msgr2=0x7f9438068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:26.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.153+0000 7f945b239700 1 --2- 172.21.15.78:0/101650948 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9438065ed0 0x7f9438068380 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f94540cf7b0 tx=0x7f9444009040 comp rx=0 tx=0).stop 2023-10-11T09:41:26.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.153+0000 7f945b239700 1 -- 172.21.15.78:0/101650948 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94540b1820 msgr2=0x7f945415d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:26.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.153+0000 7f945b239700 1 --2- 172.21.15.78:0/101650948 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94540b1820 0x7f945415d890 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f945000efc0 tx=0x7f945000c3a0 comp rx=0 tx=0).stop 2023-10-11T09:41:26.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.153+0000 7f945b239700 1 -- 172.21.15.78:0/101650948 shutdown_connections 2023-10-11T09:41:26.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.153+0000 7f945b239700 1 --2- 172.21.15.78:0/101650948 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9438065ed0 0x7f9438068380 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:26.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.153+0000 7f945b239700 1 --2- 172.21.15.78:0/101650948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94541601f0 0x7f9454157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:26.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.153+0000 7f945b239700 1 --2- 172.21.15.78:0/101650948 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f945413f9e0 0x7f94541f3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:26.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.153+0000 7f945b239700 1 --2- 172.21.15.78:0/101650948 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94540b1820 0x7f945415d890 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:26.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.153+0000 7f945b239700 1 -- 172.21.15.78:0/101650948 >> 172.21.15.78:0/101650948 conn(0x7f945406dd80 msgr2=0x7f94540ca060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:26.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.153+0000 7f945b239700 1 -- 172.21.15.78:0/101650948 shutdown_connections 2023-10-11T09:41:26.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.153+0000 7f945b239700 1 -- 172.21.15.78:0/101650948 wait complete. 2023-10-11T09:41:26.179 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T09:41:26.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.353+0000 7fc7a0d97700 1 Processor -- start 2023-10-11T09:41:26.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.353+0000 7fc7a0d97700 1 -- start start 2023-10-11T09:41:26.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.357+0000 7fc7a0d97700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc79c0b3610 0x7fc79c0d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:26.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.357+0000 7fc7a0d97700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc79c0b47f0 con 0x7fc79c0b3a90 2023-10-11T09:41:26.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.357+0000 7fc7a0d97700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc79c0b8130 con 0x7fc79c0b3240 2023-10-11T09:41:26.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.357+0000 7fc7a0d97700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc79c0b41e0 con 0x7fc79c0b3610 2023-10-11T09:41:26.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.357+0000 7fc79bfff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fc79c0b3a90 0x7fc79c0b85c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:57504/0 (socket says 172.21.15.78:57504) 2023-10-11T09:41:26.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.357+0000 7fc79bfff700 1 -- 172.21.15.78:0/2150458704 learned_addr learned my addr 172.21.15.78:0/2150458704 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:26.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.357+0000 7fc79affd700 1 --2- 172.21.15.78:0/2150458704 >> v2:172.21.15.133:3300/0 conn(0x7fc79c0b3610 0x7fc79c0d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:26.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.357+0000 7fc79a7fc700 1 -- 172.21.15.78:0/2150458704 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1961903489 0 0) 0x7fc794001490 con 0x7fc79c0b3a90 2023-10-11T09:41:26.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.357+0000 7fc79a7fc700 1 -- 172.21.15.78:0/2150458704 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7840035d0 con 0x7fc79c0b3a90 2023-10-11T09:41:26.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.357+0000 7fc79affd700 1 -- 172.21.15.78:0/2150458704 >> v1:172.21.15.133:6789/0 conn(0x7fc79c0b3240 legacy=0x7fc79c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:26.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.357+0000 7fc79affd700 1 -- 172.21.15.78:0/2150458704 >> v1:172.21.15.78:6789/0 conn(0x7fc79c0b3a90 legacy=0x7fc79c0b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:26.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.357+0000 7fc79affd700 1 -- 172.21.15.78:0/2150458704 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79c0b7800 con 0x7fc79c0b3610 2023-10-11T09:41:26.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.357+0000 7fc79affd700 1 --2- 172.21.15.78:0/2150458704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc79c0b3610 0x7fc79c0d29d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc790009850 tx=0x7fc790009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=690209c165968a64 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:26.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.357+0000 7fc79a7fc700 1 -- 172.21.15.78:0/2150458704 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc790011830 con 0x7fc79c0b3610 2023-10-11T09:41:26.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.357+0000 7fc79a7fc700 1 -- 172.21.15.78:0/2150458704 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc790011990 con 0x7fc79c0b3610 2023-10-11T09:41:26.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.357+0000 7fc79a7fc700 1 -- 172.21.15.78:0/2150458704 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc790011d40 con 0x7fc79c0b3610 2023-10-11T09:41:26.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.361+0000 7fc7a0d97700 1 -- 172.21.15.78:0/2150458704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc79c0b3610 msgr2=0x7fc79c0d29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:26.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.361+0000 7fc7a0d97700 1 --2- 172.21.15.78:0/2150458704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc79c0b3610 0x7fc79c0d29d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc790009850 tx=0x7fc790009b60 comp rx=0 tx=0).stop 2023-10-11T09:41:26.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.361+0000 7fc7a0d97700 1 -- 172.21.15.78:0/2150458704 shutdown_connections 2023-10-11T09:41:26.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.361+0000 7fc7a0d97700 1 --2- 172.21.15.78:0/2150458704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc79c0b3610 0x7fc79c0d29d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:26.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.361+0000 7fc7a0d97700 1 -- 172.21.15.78:0/2150458704 >> 172.21.15.78:0/2150458704 conn(0x7fc79c09c260 msgr2=0x7fc79c0d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:26.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.361+0000 7fc7a0d97700 1 -- 172.21.15.78:0/2150458704 shutdown_connections 2023-10-11T09:41:26.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.361+0000 7fc7a0d97700 1 -- 172.21.15.78:0/2150458704 wait complete. 2023-10-11T09:41:26.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.361+0000 7fc7a0d97700 1 Processor -- start 2023-10-11T09:41:26.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.361+0000 7fc7a0d97700 1 -- start start 2023-10-11T09:41:26.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.365+0000 7fc7a0d97700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc79c0b3240 0x7fc79c15f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:26.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.365+0000 7fc7a0d97700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc79c0b3610 0x7fc79c1f7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:26.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.365+0000 7fc7a0d97700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc79c0b3a90 0x7fc79c0c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:26.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.365+0000 7fc7a0d97700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79c15bdf0 con 0x7fc79c0b3240 2023-10-11T09:41:26.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.365+0000 7fc7a0d97700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79c0c7590 con 0x7fc79c0b3a90 2023-10-11T09:41:26.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.365+0000 7fc7a0d97700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc79c0c6c60 con 0x7fc79c0b3610 2023-10-11T09:41:26.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.365+0000 7fc79b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc79c0b3240 0x7fc79c15f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:26.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.365+0000 7fc79b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc79c0b3240 0x7fc79c15f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59390/0 (socket says 172.21.15.78:59390) 2023-10-11T09:41:26.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.365+0000 7fc79b7fe700 1 -- 172.21.15.78:0/745134670 learned_addr learned my addr 172.21.15.78:0/745134670 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:26.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.365+0000 7fc79bfff700 1 --2- 172.21.15.78:0/745134670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc79c0b3a90 0x7fc79c0c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:26.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.365+0000 7fc79affd700 1 --2- 172.21.15.78:0/745134670 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc79c0b3610 0x7fc79c1f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:26.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.365+0000 7fc79b7fe700 1 -- 172.21.15.78:0/745134670 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc79c0b3610 msgr2=0x7fc79c1f7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:26.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.365+0000 7fc79b7fe700 1 --2- 172.21.15.78:0/745134670 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc79c0b3610 0x7fc79c1f7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:26.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.365+0000 7fc79b7fe700 1 -- 172.21.15.78:0/745134670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc79c0b3a90 msgr2=0x7fc79c0c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:26.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.365+0000 7fc79b7fe700 1 --2- 172.21.15.78:0/745134670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc79c0b3a90 0x7fc79c0c8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:26.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.365+0000 7fc79b7fe700 1 -- 172.21.15.78:0/745134670 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7900094b0 con 0x7fc79c0b3240 2023-10-11T09:41:26.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.365+0000 7fc79b7fe700 1 --2- 172.21.15.78:0/745134670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc79c0b3240 0x7fc79c15f260 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc78c00c500 tx=0x7fc78c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:26.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.365+0000 7fc783fff700 1 -- 172.21.15.78:0/745134670 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c0052d0 con 0x7fc79c0b3240 2023-10-11T09:41:26.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.365+0000 7fc783fff700 1 -- 172.21.15.78:0/745134670 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78c005430 con 0x7fc79c0b3240 2023-10-11T09:41:26.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.365+0000 7fc7a0d97700 1 -- 172.21.15.78:0/745134670 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc79c0c6f40 con 0x7fc79c0b3240 2023-10-11T09:41:26.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.365+0000 7fc783fff700 1 -- 172.21.15.78:0/745134670 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c0055a0 con 0x7fc79c0b3240 2023-10-11T09:41:26.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.365+0000 7fc7a0d97700 1 -- 172.21.15.78:0/745134670 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc79c1f7990 con 0x7fc79c0b3240 2023-10-11T09:41:26.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.369+0000 7fc7a0d97700 1 -- 172.21.15.78:0/745134670 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc79c065d40 con 0x7fc79c0b3240 2023-10-11T09:41:26.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.369+0000 7fc783fff700 1 -- 172.21.15.78:0/745134670 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fc78c0038e0 con 0x7fc79c0b3240 2023-10-11T09:41:26.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.369+0000 7fc783fff700 1 --2- 172.21.15.78:0/745134670 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc788065f00 0x7fc7880683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:26.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.369+0000 7fc783fff700 1 -- 172.21.15.78:0/745134670 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fc78c082ac0 con 0x7fc79c0b3240 2023-10-11T09:41:26.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.369+0000 7fc79affd700 1 --2- 172.21.15.78:0/745134670 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc788065f00 0x7fc7880683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:26.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.373+0000 7fc79affd700 1 --2- 172.21.15.78:0/745134670 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc788065f00 0x7fc7880683b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc7900090b0 tx=0x7fc79001c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:26.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.377+0000 7fc783fff700 1 -- 172.21.15.78:0/745134670 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fc78c050160 con 0x7fc79c0b3240 2023-10-11T09:41:26.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.549+0000 7fc7a0d97700 1 -- 172.21.15.78:0/745134670 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc79c0b2db0 con 0x7fc79c0b3240 2023-10-11T09:41:26.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.549+0000 7fc783fff700 1 -- 172.21.15.78:0/745134670 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc78c0170a0 con 0x7fc79c0b3240 2023-10-11T09:41:26.555 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2023-10-11T09:41:26.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.553+0000 7fc7a0d97700 1 -- 172.21.15.78:0/745134670 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc788065f00 msgr2=0x7fc7880683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:26.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.553+0000 7fc7a0d97700 1 --2- 172.21.15.78:0/745134670 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc788065f00 0x7fc7880683b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc7900090b0 tx=0x7fc79001c040 comp rx=0 tx=0).stop 2023-10-11T09:41:26.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.553+0000 7fc7a0d97700 1 -- 172.21.15.78:0/745134670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc79c0b3240 msgr2=0x7fc79c15f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:26.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.553+0000 7fc7a0d97700 1 --2- 172.21.15.78:0/745134670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc79c0b3240 0x7fc79c15f260 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc78c00c500 tx=0x7fc78c00c810 comp rx=0 tx=0).stop 2023-10-11T09:41:26.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.553+0000 7fc7a0d97700 1 -- 172.21.15.78:0/745134670 shutdown_connections 2023-10-11T09:41:26.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.553+0000 7fc7a0d97700 1 --2- 172.21.15.78:0/745134670 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc788065f00 0x7fc7880683b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:26.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.553+0000 7fc7a0d97700 1 --2- 172.21.15.78:0/745134670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc79c0b3a90 0x7fc79c0c8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:26.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.553+0000 7fc7a0d97700 1 --2- 172.21.15.78:0/745134670 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc79c0b3610 0x7fc79c1f7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:26.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.553+0000 7fc7a0d97700 1 --2- 172.21.15.78:0/745134670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc79c0b3240 0x7fc79c15f260 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:26.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.553+0000 7fc7a0d97700 1 -- 172.21.15.78:0/745134670 >> 172.21.15.78:0/745134670 conn(0x7fc79c09c260 msgr2=0x7fc79c0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:26.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.553+0000 7fc7a0d97700 1 -- 172.21.15.78:0/745134670 shutdown_connections 2023-10-11T09:41:26.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.553+0000 7fc7a0d97700 1 -- 172.21.15.78:0/745134670 wait complete. 2023-10-11T09:41:26.583 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T09:41:26.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.749+0000 7f414cff9700 1 Processor -- start 2023-10-11T09:41:26.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.753+0000 7f414cff9700 1 -- start start 2023-10-11T09:41:26.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.753+0000 7f414cff9700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4148071040 0x7f41480cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:26.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.753+0000 7f414cff9700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f414806ffa0 con 0x7f4148160d40 2023-10-11T09:41:26.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.753+0000 7f414cff9700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4148070280 con 0x7f41480ce330 2023-10-11T09:41:26.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.753+0000 7f414cff9700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4148070c20 con 0x7f4148071040 2023-10-11T09:41:26.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.753+0000 7f414659c700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f4148160d40 0x7f41480cd280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:57512/0 (socket says 172.21.15.78:57512) 2023-10-11T09:41:26.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.753+0000 7f414659c700 1 -- 172.21.15.78:0/979088907 learned_addr learned my addr 172.21.15.78:0/979088907 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:26.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.753+0000 7f4145d9b700 1 --2- 172.21.15.78:0/979088907 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4148071040 0x7f41480cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:26.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.757+0000 7f414559a700 1 -- 172.21.15.78:0/979088907 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 73631941 0 0) 0x7f414806ffa0 con 0x7f4148160d40 2023-10-11T09:41:26.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.757+0000 7f414559a700 1 -- 172.21.15.78:0/979088907 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41300035d0 con 0x7f4148160d40 2023-10-11T09:41:26.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.757+0000 7f4145d9b700 1 -- 172.21.15.78:0/979088907 >> v1:172.21.15.133:6789/0 conn(0x7f41480ce330 legacy=0x7f414806be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:26.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.757+0000 7f4145d9b700 1 -- 172.21.15.78:0/979088907 >> v1:172.21.15.78:6789/0 conn(0x7f4148160d40 legacy=0x7f41480cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:26.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.757+0000 7f4145d9b700 1 -- 172.21.15.78:0/979088907 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4148070560 con 0x7f4148071040 2023-10-11T09:41:26.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.757+0000 7f4145d9b700 1 --2- 172.21.15.78:0/979088907 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4148071040 0x7f41480cddf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f413c009880 tx=0x7f413c009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e73209296f2aa1c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:26.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.757+0000 7f414559a700 1 -- 172.21.15.78:0/979088907 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c011830 con 0x7f4148071040 2023-10-11T09:41:26.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.757+0000 7f414559a700 1 -- 172.21.15.78:0/979088907 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f413c011990 con 0x7f4148071040 2023-10-11T09:41:26.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.757+0000 7f414559a700 1 -- 172.21.15.78:0/979088907 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c011d20 con 0x7f4148071040 2023-10-11T09:41:26.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.757+0000 7f414cff9700 1 -- 172.21.15.78:0/979088907 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4148071040 msgr2=0x7f41480cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:26.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.757+0000 7f414cff9700 1 --2- 172.21.15.78:0/979088907 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4148071040 0x7f41480cddf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f413c009880 tx=0x7f413c009b90 comp rx=0 tx=0).stop 2023-10-11T09:41:26.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.757+0000 7f414cff9700 1 -- 172.21.15.78:0/979088907 shutdown_connections 2023-10-11T09:41:26.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.757+0000 7f414cff9700 1 --2- 172.21.15.78:0/979088907 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4148071040 0x7f41480cddf0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:26.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.757+0000 7f414cff9700 1 -- 172.21.15.78:0/979088907 >> 172.21.15.78:0/979088907 conn(0x7f414806aca0 msgr2=0x7f41480b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:26.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.757+0000 7f414cff9700 1 -- 172.21.15.78:0/979088907 shutdown_connections 2023-10-11T09:41:26.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.757+0000 7f414cff9700 1 -- 172.21.15.78:0/979088907 wait complete. 2023-10-11T09:41:26.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.761+0000 7f414cff9700 1 Processor -- start 2023-10-11T09:41:26.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.761+0000 7f414cff9700 1 -- start start 2023-10-11T09:41:26.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.765+0000 7f414cff9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4148071040 0x7f41480c3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:26.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.765+0000 7f414cff9700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41480ce330 0x7f41481fa640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:26.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.765+0000 7f414cff9700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4148160d40 0x7f41481ec5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:26.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.765+0000 7f414cff9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41480c74a0 con 0x7f4148071040 2023-10-11T09:41:26.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.765+0000 7f414cff9700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41481ecae0 con 0x7f41480ce330 2023-10-11T09:41:26.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.765+0000 7f414cff9700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41481ecd80 con 0x7f4148160d40 2023-10-11T09:41:26.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.765+0000 7f414659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4148071040 0x7f41480c3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:26.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.765+0000 7f414659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4148071040 0x7f41480c3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59396/0 (socket says 172.21.15.78:59396) 2023-10-11T09:41:26.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.765+0000 7f414659c700 1 -- 172.21.15.78:0/2595123902 learned_addr learned my addr 172.21.15.78:0/2595123902 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:26.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.765+0000 7f4145d9b700 1 --2- 172.21.15.78:0/2595123902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41480ce330 0x7f41481fa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:26.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.765+0000 7f4146d9d700 1 --2- 172.21.15.78:0/2595123902 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4148160d40 0x7f41481ec5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:26.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.765+0000 7f414659c700 1 -- 172.21.15.78:0/2595123902 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4148160d40 msgr2=0x7f41481ec5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:26.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.765+0000 7f414659c700 1 --2- 172.21.15.78:0/2595123902 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4148160d40 0x7f41481ec5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:26.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.765+0000 7f414659c700 1 -- 172.21.15.78:0/2595123902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41480ce330 msgr2=0x7f41481fa640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:26.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.765+0000 7f414659c700 1 --2- 172.21.15.78:0/2595123902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41480ce330 0x7f41481fa640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:26.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.765+0000 7f414659c700 1 -- 172.21.15.78:0/2595123902 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f413c0094b0 con 0x7f4148071040 2023-10-11T09:41:26.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.765+0000 7f414659c700 1 --2- 172.21.15.78:0/2595123902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4148071040 0x7f41480c3c00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f413800c700 tx=0x7f413800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:26.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.765+0000 7f4136ffd700 1 -- 172.21.15.78:0/2595123902 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4138005d90 con 0x7f4148071040 2023-10-11T09:41:26.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.765+0000 7f4136ffd700 1 -- 172.21.15.78:0/2595123902 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41380070d0 con 0x7f4148071040 2023-10-11T09:41:26.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.765+0000 7f414cff9700 1 -- 172.21.15.78:0/2595123902 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41481ed060 con 0x7f4148071040 2023-10-11T09:41:26.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.765+0000 7f4136ffd700 1 -- 172.21.15.78:0/2595123902 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4138007480 con 0x7f4148071040 2023-10-11T09:41:26.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.765+0000 7f414cff9700 1 -- 172.21.15.78:0/2595123902 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41480c7720 con 0x7f4148071040 2023-10-11T09:41:26.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.769+0000 7f414cff9700 1 -- 172.21.15.78:0/2595123902 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4148072250 con 0x7f4148071040 2023-10-11T09:41:26.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.769+0000 7f4136ffd700 1 -- 172.21.15.78:0/2595123902 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f41380075e0 con 0x7f4148071040 2023-10-11T09:41:26.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.773+0000 7f4136ffd700 1 --2- 172.21.15.78:0/2595123902 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4128065ed0 0x7f4128068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:26.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.773+0000 7f4136ffd700 1 -- 172.21.15.78:0/2595123902 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f4138082d20 con 0x7f4148071040 2023-10-11T09:41:26.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.773+0000 7f4145d9b700 1 --2- 172.21.15.78:0/2595123902 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4128065ed0 0x7f4128068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:26.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.773+0000 7f4145d9b700 1 --2- 172.21.15.78:0/2595123902 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4128065ed0 0x7f4128068380 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4148071430 tx=0x7f413c010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:26.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.781+0000 7f4136ffd700 1 -- 172.21.15.78:0/2595123902 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f4138050340 con 0x7f4148071040 2023-10-11T09:41:26.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.965+0000 7f414cff9700 1 -- 172.21.15.78:0/2595123902 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f41481ed580 con 0x7f4148071040 2023-10-11T09:41:26.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.969+0000 7f4136ffd700 1 -- 172.21.15.78:0/2595123902 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4138053960 con 0x7f4148071040 2023-10-11T09:41:26.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.969+0000 7f414cff9700 1 -- 172.21.15.78:0/2595123902 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4128065ed0 msgr2=0x7f4128068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:26.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.969+0000 7f414cff9700 1 --2- 172.21.15.78:0/2595123902 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4128065ed0 0x7f4128068380 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4148071430 tx=0x7f413c010000 comp rx=0 tx=0).stop 2023-10-11T09:41:26.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.969+0000 7f414cff9700 1 -- 172.21.15.78:0/2595123902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4148071040 msgr2=0x7f41480c3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:26.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.969+0000 7f414cff9700 1 --2- 172.21.15.78:0/2595123902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4148071040 0x7f41480c3c00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f413800c700 tx=0x7f413800ca10 comp rx=0 tx=0).stop 2023-10-11T09:41:26.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.969+0000 7f414cff9700 1 -- 172.21.15.78:0/2595123902 shutdown_connections 2023-10-11T09:41:26.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.969+0000 7f414cff9700 1 --2- 172.21.15.78:0/2595123902 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4128065ed0 0x7f4128068380 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:26.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.969+0000 7f414cff9700 1 --2- 172.21.15.78:0/2595123902 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4148160d40 0x7f41481ec5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:26.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.969+0000 7f414cff9700 1 --2- 172.21.15.78:0/2595123902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41480ce330 0x7f41481fa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:26.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.969+0000 7f414cff9700 1 --2- 172.21.15.78:0/2595123902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4148071040 0x7f41480c3c00 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:26.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.969+0000 7f414cff9700 1 -- 172.21.15.78:0/2595123902 >> 172.21.15.78:0/2595123902 conn(0x7f414806aca0 msgr2=0x7f4148161310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:26.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.969+0000 7f414cff9700 1 -- 172.21.15.78:0/2595123902 shutdown_connections 2023-10-11T09:41:26.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:26.973+0000 7f414cff9700 1 -- 172.21.15.78:0/2595123902 wait complete. 2023-10-11T09:41:26.979 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2023-10-11T09:41:27.000 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi078.front.sepia.ceph.com... 2023-10-11T09:41:27.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:27.001 DEBUG:teuthology.orchestra.run.smithi078:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-11T09:41:27.041 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:41:27.042 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2023-10-11T09:41:27.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:27.043 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-11T09:41:27.115 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:41:27.116 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:41:27.135 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:41:27.135 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:41:27.196 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:41:27.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.369+0000 7f6cc0898700 1 Processor -- start 2023-10-11T09:41:27.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.369+0000 7f6cc0898700 1 -- start start 2023-10-11T09:41:27.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.373+0000 7f6cc0898700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6cbc15a940 0x7f6cbc0ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:27.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.373+0000 7f6cc0898700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6cbc0ca780 0x7f6cbc13fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:27.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.373+0000 7f6cc0898700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cbc0cabc0 con 0x7f6cbc15b8e0 2023-10-11T09:41:27.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.373+0000 7f6cc0898700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cbc15e010 con 0x7f6cbc15a940 2023-10-11T09:41:27.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.373+0000 7f6cc0898700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc15e280 con 0x7f6cbc0ca780 2023-10-11T09:41:27.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.373+0000 7f6cb2ffd700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6cbc15a940 0x7f6cbc0ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:27.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.373+0000 7f6cbbfff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6cbc0ca780 0x7f6cbc13fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:27.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.373+0000 7f6cbb7fe700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f6cbc15b8e0 0x7f6cbc06d0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:44630/0 (socket says 172.21.15.78:44630) 2023-10-11T09:41:27.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.373+0000 7f6cb2ffd700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6cbc15a940 0x7f6cbc0ca240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:36180/0 (socket says 172.21.15.78:36180) 2023-10-11T09:41:27.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.373+0000 7f6cbb7fe700 1 -- 172.21.15.78:0/3769082347 learned_addr learned my addr 172.21.15.78:0/3769082347 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:27.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.373+0000 7f6cbaffd700 1 -- 172.21.15.78:0/3769082347 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2280361028 0 0) 0x7f6cbc0cabc0 con 0x7f6cbc15b8e0 2023-10-11T09:41:27.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.373+0000 7f6cbaffd700 1 -- 172.21.15.78:0/3769082347 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ca80035d0 con 0x7f6cbc15b8e0 2023-10-11T09:41:27.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.373+0000 7f6cb2ffd700 1 -- 172.21.15.78:0/3769082347 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6cbc0ca780 msgr2=0x7f6cbc13fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:27.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.373+0000 7f6cb2ffd700 1 --2- 172.21.15.78:0/3769082347 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6cbc0ca780 0x7f6cbc13fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:27.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.373+0000 7f6cb2ffd700 1 -- 172.21.15.78:0/3769082347 >> v1:172.21.15.133:6789/0 conn(0x7f6cbc15b8e0 legacy=0x7f6cbc06d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:27.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.373+0000 7f6cb2ffd700 1 -- 172.21.15.78:0/3769082347 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cbc15e560 con 0x7f6cbc15a940 2023-10-11T09:41:27.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.373+0000 7f6cbbfff700 1 --2- 172.21.15.78:0/3769082347 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6cbc0ca780 0x7f6cbc13fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:41:27.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.373+0000 7f6cb2ffd700 1 --2- 172.21.15.78:0/3769082347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6cbc15a940 0x7f6cbc0ca240 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4009850 tx=0x7f6ca4009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da5916c9346d2335 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:27.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.377+0000 7f6cbaffd700 1 -- 172.21.15.78:0/3769082347 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4011830 con 0x7f6cbc15a940 2023-10-11T09:41:27.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.377+0000 7f6cbaffd700 1 -- 172.21.15.78:0/3769082347 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca4011990 con 0x7f6cbc15a940 2023-10-11T09:41:27.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.377+0000 7f6cbaffd700 1 -- 172.21.15.78:0/3769082347 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4011d40 con 0x7f6cbc15a940 2023-10-11T09:41:27.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.377+0000 7f6cc0898700 1 -- 172.21.15.78:0/3769082347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6cbc15a940 msgr2=0x7f6cbc0ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:27.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.377+0000 7f6cc0898700 1 --2- 172.21.15.78:0/3769082347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6cbc15a940 0x7f6cbc0ca240 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4009850 tx=0x7f6ca4009b60 comp rx=0 tx=0).stop 2023-10-11T09:41:27.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.377+0000 7f6cc0898700 1 -- 172.21.15.78:0/3769082347 shutdown_connections 2023-10-11T09:41:27.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.377+0000 7f6cc0898700 1 --2- 172.21.15.78:0/3769082347 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6cbc0ca780 0x7f6cbc13fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:27.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.377+0000 7f6cc0898700 1 --2- 172.21.15.78:0/3769082347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6cbc15a940 0x7f6cbc0ca240 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:27.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.377+0000 7f6cc0898700 1 -- 172.21.15.78:0/3769082347 >> 172.21.15.78:0/3769082347 conn(0x7f6cbc0cea60 msgr2=0x7f6cbc0cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:27.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.377+0000 7f6cc0898700 1 -- 172.21.15.78:0/3769082347 shutdown_connections 2023-10-11T09:41:27.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.377+0000 7f6cc0898700 1 -- 172.21.15.78:0/3769082347 wait complete. 2023-10-11T09:41:27.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.377+0000 7f6cc0898700 1 Processor -- start 2023-10-11T09:41:27.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.381+0000 7f6cc0898700 1 -- start start 2023-10-11T09:41:27.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.381+0000 7f6cc0898700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6cbc0ca780 0x7f6cbc0b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:27.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.381+0000 7f6cc0898700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6cbc15a940 0x7f6cbc1ef490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:27.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.381+0000 7f6cc0898700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6cbc15b8e0 0x7f6cbc0c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:27.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.381+0000 7f6cc0898700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc0b4460 con 0x7f6cbc0ca780 2023-10-11T09:41:27.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.381+0000 7f6cc0898700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc1ef9d0 con 0x7f6cbc15b8e0 2023-10-11T09:41:27.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.381+0000 7f6cc0898700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc1f2c30 con 0x7f6cbc15a940 2023-10-11T09:41:27.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.381+0000 7f6cbb7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6cbc0ca780 0x7f6cbc0b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:27.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.381+0000 7f6cbb7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6cbc0ca780 0x7f6cbc0b35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59402/0 (socket says 172.21.15.78:59402) 2023-10-11T09:41:27.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.381+0000 7f6cbb7fe700 1 -- 172.21.15.78:0/3996025356 learned_addr learned my addr 172.21.15.78:0/3996025356 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:27.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.381+0000 7f6cbbfff700 1 --2- 172.21.15.78:0/3996025356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6cbc15b8e0 0x7f6cbc0c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:27.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.381+0000 7f6cb2ffd700 1 --2- 172.21.15.78:0/3996025356 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6cbc15a940 0x7f6cbc1ef490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:27.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.381+0000 7f6cbb7fe700 1 -- 172.21.15.78:0/3996025356 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6cbc15a940 msgr2=0x7f6cbc1ef490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:27.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.381+0000 7f6cbb7fe700 1 --2- 172.21.15.78:0/3996025356 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6cbc15a940 0x7f6cbc1ef490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:27.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.381+0000 7f6cbb7fe700 1 -- 172.21.15.78:0/3996025356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6cbc15b8e0 msgr2=0x7f6cbc0c7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:27.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.381+0000 7f6cbb7fe700 1 --2- 172.21.15.78:0/3996025356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6cbc15b8e0 0x7f6cbc0c7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:27.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.381+0000 7f6cbb7fe700 1 -- 172.21.15.78:0/3996025356 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb40094b0 con 0x7f6cbc0ca780 2023-10-11T09:41:27.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.381+0000 7f6cbbfff700 1 --2- 172.21.15.78:0/3996025356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6cbc15b8e0 0x7f6cbc0c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:27.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.381+0000 7f6cbb7fe700 1 --2- 172.21.15.78:0/3996025356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6cbc0ca780 0x7f6cbc0b35e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6cac00cbc0 tx=0x7f6cac00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:27.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.381+0000 7f6cb8ff9700 1 -- 172.21.15.78:0/3996025356 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac00a9e0 con 0x7f6cbc0ca780 2023-10-11T09:41:27.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.385+0000 7f6cc0898700 1 -- 172.21.15.78:0/3996025356 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca40094b0 con 0x7f6cbc0ca780 2023-10-11T09:41:27.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.385+0000 7f6cb8ff9700 1 -- 172.21.15.78:0/3996025356 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cac00ab40 con 0x7f6cbc0ca780 2023-10-11T09:41:27.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.385+0000 7f6cb8ff9700 1 -- 172.21.15.78:0/3996025356 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac012400 con 0x7f6cbc0ca780 2023-10-11T09:41:27.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.385+0000 7f6cc0898700 1 -- 172.21.15.78:0/3996025356 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cbc0b2b00 con 0x7f6cbc0ca780 2023-10-11T09:41:27.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.385+0000 7f6cc0898700 1 -- 172.21.15.78:0/3996025356 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cbc065d40 con 0x7f6cbc0ca780 2023-10-11T09:41:27.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.385+0000 7f6cb8ff9700 1 -- 172.21.15.78:0/3996025356 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f6cac01b3b0 con 0x7f6cbc0ca780 2023-10-11T09:41:27.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.389+0000 7f6cb8ff9700 1 --2- 172.21.15.78:0/3996025356 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6c9c065ed0 0x7f6c9c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:27.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.389+0000 7f6cb8ff9700 1 -- 172.21.15.78:0/3996025356 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f6cac0833e0 con 0x7f6cbc0ca780 2023-10-11T09:41:27.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.389+0000 7f6cb2ffd700 1 --2- 172.21.15.78:0/3996025356 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6c9c065ed0 0x7f6c9c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:27.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.389+0000 7f6cb2ffd700 1 --2- 172.21.15.78:0/3996025356 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6c9c065ed0 0x7f6c9c068380 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6ca40090b0 tx=0x7f6ca4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:27.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.393+0000 7f6cb8ff9700 1 -- 172.21.15.78:0/3996025356 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f6cac050970 con 0x7f6cbc0ca780 2023-10-11T09:41:27.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.569+0000 7f6cc0898700 1 -- 172.21.15.78:0/3996025356 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6cbc06ffa0 con 0x7f6cbc0ca780 2023-10-11T09:41:27.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.573+0000 7f6cb8ff9700 1 -- 172.21.15.78:0/3996025356 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17907 (secure 0 0 0) 0x7f6cac053f90 con 0x7f6cbc0ca780 2023-10-11T09:41:27.577 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:41:27.577 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":25,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:41:21.473566+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:40:39.862581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T09:40:40.663641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.37","mappings":[{"from":3,"to":2}]},{"pgid":"4.3e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:41:27.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.573+0000 7f6cc0898700 1 -- 172.21.15.78:0/3996025356 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6c9c065ed0 msgr2=0x7f6c9c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:27.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.573+0000 7f6cc0898700 1 --2- 172.21.15.78:0/3996025356 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6c9c065ed0 0x7f6c9c068380 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6ca40090b0 tx=0x7f6ca4008030 comp rx=0 tx=0).stop 2023-10-11T09:41:27.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.573+0000 7f6cc0898700 1 -- 172.21.15.78:0/3996025356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6cbc0ca780 msgr2=0x7f6cbc0b35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:27.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.573+0000 7f6cc0898700 1 --2- 172.21.15.78:0/3996025356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6cbc0ca780 0x7f6cbc0b35e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6cac00cbc0 tx=0x7f6cac00cca0 comp rx=0 tx=0).stop 2023-10-11T09:41:27.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.573+0000 7f6cc0898700 1 -- 172.21.15.78:0/3996025356 shutdown_connections 2023-10-11T09:41:27.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.573+0000 7f6cc0898700 1 --2- 172.21.15.78:0/3996025356 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6c9c065ed0 0x7f6c9c068380 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:27.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.573+0000 7f6cc0898700 1 --2- 172.21.15.78:0/3996025356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6cbc15b8e0 0x7f6cbc0c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:27.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.573+0000 7f6cc0898700 1 --2- 172.21.15.78:0/3996025356 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6cbc15a940 0x7f6cbc1ef490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:27.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.573+0000 7f6cc0898700 1 --2- 172.21.15.78:0/3996025356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6cbc0ca780 0x7f6cbc0b35e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:27.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.573+0000 7f6cc0898700 1 -- 172.21.15.78:0/3996025356 >> 172.21.15.78:0/3996025356 conn(0x7f6cbc0cea60 msgr2=0x7f6cbc070c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:27.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.573+0000 7f6cc0898700 1 -- 172.21.15.78:0/3996025356 shutdown_connections 2023-10-11T09:41:27.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.573+0000 7f6cc0898700 1 -- 172.21.15.78:0/3996025356 wait complete. 2023-10-11T09:41:27.603 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T09:40:39.862581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T09:40:40.663641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:41:27.604 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:41:27.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.777+0000 7fe9255ef700 1 Processor -- start 2023-10-11T09:41:27.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.777+0000 7fe9255ef700 1 -- start start 2023-10-11T09:41:27.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.777+0000 7fe9255ef700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe9200d29d0 0x7fe9200b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:27.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.777+0000 7fe9255ef700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9200b47f0 con 0x7fe9200b3240 2023-10-11T09:41:27.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.777+0000 7fe9255ef700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9200b8130 con 0x7fe9200b3610 2023-10-11T09:41:27.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.777+0000 7fe9255ef700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9200d2e30 con 0x7fe9200d29d0 2023-10-11T09:41:27.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.777+0000 7fe91ffff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fe9200b3240 0x7fe92015d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:57520/0 (socket says 172.21.15.78:57520) 2023-10-11T09:41:27.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.777+0000 7fe91ffff700 1 -- 172.21.15.78:0/3362907095 learned_addr learned my addr 172.21.15.78:0/3362907095 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:27.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.777+0000 7fe924dee700 1 --2- 172.21.15.78:0/3362907095 >> v2:172.21.15.133:3300/0 conn(0x7fe9200d29d0 0x7fe9200b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:27.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.777+0000 7fe91effd700 1 -- 172.21.15.78:0/3362907095 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3936966016 0 0) 0x7fe910001110 con 0x7fe9200b3240 2023-10-11T09:41:27.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.781+0000 7fe91effd700 1 -- 172.21.15.78:0/3362907095 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9080035d0 con 0x7fe9200b3240 2023-10-11T09:41:27.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.781+0000 7fe924dee700 1 -- 172.21.15.78:0/3362907095 >> v1:172.21.15.133:6789/0 conn(0x7fe9200b3610 legacy=0x7fe9200b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:27.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.781+0000 7fe924dee700 1 -- 172.21.15.78:0/3362907095 >> v1:172.21.15.78:6789/0 conn(0x7fe9200b3240 legacy=0x7fe92015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:27.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.781+0000 7fe924dee700 1 -- 172.21.15.78:0/3362907095 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9200b41e0 con 0x7fe9200d29d0 2023-10-11T09:41:27.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.781+0000 7fe924dee700 1 --2- 172.21.15.78:0/3362907095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9200d29d0 0x7fe9200b3a90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe918009d90 tx=0x7fe91800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=952fdfb39f51e8f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:27.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.781+0000 7fe91effd700 1 -- 172.21.15.78:0/3362907095 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91800e070 con 0x7fe9200d29d0 2023-10-11T09:41:27.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.781+0000 7fe91effd700 1 -- 172.21.15.78:0/3362907095 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe91800bb20 con 0x7fe9200d29d0 2023-10-11T09:41:27.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.781+0000 7fe91effd700 1 -- 172.21.15.78:0/3362907095 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe918012430 con 0x7fe9200d29d0 2023-10-11T09:41:27.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.781+0000 7fe9255ef700 1 -- 172.21.15.78:0/3362907095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9200d29d0 msgr2=0x7fe9200b3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:27.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.781+0000 7fe9255ef700 1 --2- 172.21.15.78:0/3362907095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9200d29d0 0x7fe9200b3a90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe918009d90 tx=0x7fe91800b330 comp rx=0 tx=0).stop 2023-10-11T09:41:27.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.781+0000 7fe9255ef700 1 -- 172.21.15.78:0/3362907095 shutdown_connections 2023-10-11T09:41:27.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.781+0000 7fe9255ef700 1 --2- 172.21.15.78:0/3362907095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9200d29d0 0x7fe9200b3a90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:27.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.781+0000 7fe9255ef700 1 -- 172.21.15.78:0/3362907095 >> 172.21.15.78:0/3362907095 conn(0x7fe92009c260 msgr2=0x7fe9200d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:27.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.781+0000 7fe9255ef700 1 -- 172.21.15.78:0/3362907095 shutdown_connections 2023-10-11T09:41:27.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.781+0000 7fe9255ef700 1 -- 172.21.15.78:0/3362907095 wait complete. 2023-10-11T09:41:27.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.785+0000 7fe9255ef700 1 Processor -- start 2023-10-11T09:41:27.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.785+0000 7fe9255ef700 1 -- start start 2023-10-11T09:41:27.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.789+0000 7fe9255ef700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9200b3240 0x7fe92015ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:27.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.789+0000 7fe9255ef700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9200b3610 0x7fe9200c7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:27.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.789+0000 7fe9255ef700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe9200d29d0 0x7fe9201e68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:27.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.789+0000 7fe9255ef700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9200c8590 con 0x7fe9200b3610 2023-10-11T09:41:27.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.789+0000 7fe9255ef700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9200c8870 con 0x7fe9200b3240 2023-10-11T09:41:27.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.789+0000 7fe9255ef700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9200c8b50 con 0x7fe9200d29d0 2023-10-11T09:41:27.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.789+0000 7fe91f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9200b3610 0x7fe9200c7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:27.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.789+0000 7fe91f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9200b3610 0x7fe9200c7b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59414/0 (socket says 172.21.15.78:59414) 2023-10-11T09:41:27.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.789+0000 7fe91f7fe700 1 -- 172.21.15.78:0/3737053108 learned_addr learned my addr 172.21.15.78:0/3737053108 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:27.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.789+0000 7fe924dee700 1 --2- 172.21.15.78:0/3737053108 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe9200d29d0 0x7fe9201e68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:27.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.789+0000 7fe91ffff700 1 --2- 172.21.15.78:0/3737053108 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9200b3240 0x7fe92015ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:27.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.789+0000 7fe91f7fe700 1 -- 172.21.15.78:0/3737053108 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe9200d29d0 msgr2=0x7fe9201e68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:27.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.789+0000 7fe91f7fe700 1 --2- 172.21.15.78:0/3737053108 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe9200d29d0 0x7fe9201e68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:27.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.789+0000 7fe91f7fe700 1 -- 172.21.15.78:0/3737053108 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9200b3240 msgr2=0x7fe92015ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:27.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.789+0000 7fe91f7fe700 1 --2- 172.21.15.78:0/3737053108 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9200b3240 0x7fe92015ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:27.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.789+0000 7fe91f7fe700 1 -- 172.21.15.78:0/3737053108 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe918009a70 con 0x7fe9200b3610 2023-10-11T09:41:27.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.789+0000 7fe91f7fe700 1 --2- 172.21.15.78:0/3737053108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9200b3610 0x7fe9200c7b10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe91400c500 tx=0x7fe91400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:27.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.789+0000 7fe91cff9700 1 -- 172.21.15.78:0/3737053108 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9140052d0 con 0x7fe9200b3610 2023-10-11T09:41:27.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.789+0000 7fe91cff9700 1 -- 172.21.15.78:0/3737053108 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe914005430 con 0x7fe9200b3610 2023-10-11T09:41:27.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.789+0000 7fe91cff9700 1 -- 172.21.15.78:0/3737053108 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9140055a0 con 0x7fe9200b3610 2023-10-11T09:41:27.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.789+0000 7fe9255ef700 1 -- 172.21.15.78:0/3737053108 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9200c8e30 con 0x7fe9200b3610 2023-10-11T09:41:27.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.789+0000 7fe9255ef700 1 -- 172.21.15.78:0/3737053108 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe92015f230 con 0x7fe9200b3610 2023-10-11T09:41:27.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.793+0000 7fe9255ef700 1 -- 172.21.15.78:0/3737053108 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe920065d40 con 0x7fe9200b3610 2023-10-11T09:41:27.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.793+0000 7fe91cff9700 1 -- 172.21.15.78:0/3737053108 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fe9140038e0 con 0x7fe9200b3610 2023-10-11T09:41:27.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.797+0000 7fe91cff9700 1 --2- 172.21.15.78:0/3737053108 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe90c065ed0 0x7fe90c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:27.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.797+0000 7fe91cff9700 1 -- 172.21.15.78:0/3737053108 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fe914082b70 con 0x7fe9200b3610 2023-10-11T09:41:27.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.797+0000 7fe91ffff700 1 --2- 172.21.15.78:0/3737053108 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe90c065ed0 0x7fe90c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:27.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.797+0000 7fe91ffff700 1 --2- 172.21.15.78:0/3737053108 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe90c065ed0 0x7fe90c068380 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe910007fd0 tx=0x7fe910007c60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:27.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.801+0000 7fe91cff9700 1 -- 172.21.15.78:0/3737053108 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fe9140500f0 con 0x7fe9200b3610 2023-10-11T09:41:27.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.977+0000 7fe9255ef700 1 -- 172.21.15.78:0/3737053108 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe92006c2c0 con 0x7fe9200b3610 2023-10-11T09:41:27.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.977+0000 7fe91cff9700 1 -- 172.21.15.78:0/3737053108 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe914017020 con 0x7fe9200b3610 2023-10-11T09:41:27.983 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:41:27.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.981+0000 7fe9255ef700 1 -- 172.21.15.78:0/3737053108 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe90c065ed0 msgr2=0x7fe90c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:27.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.981+0000 7fe9255ef700 1 --2- 172.21.15.78:0/3737053108 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe90c065ed0 0x7fe90c068380 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe910007fd0 tx=0x7fe910007c60 comp rx=0 tx=0).stop 2023-10-11T09:41:27.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.981+0000 7fe9255ef700 1 -- 172.21.15.78:0/3737053108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9200b3610 msgr2=0x7fe9200c7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:27.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.981+0000 7fe9255ef700 1 --2- 172.21.15.78:0/3737053108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9200b3610 0x7fe9200c7b10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe91400c500 tx=0x7fe91400c810 comp rx=0 tx=0).stop 2023-10-11T09:41:27.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.981+0000 7fe9255ef700 1 -- 172.21.15.78:0/3737053108 shutdown_connections 2023-10-11T09:41:27.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.981+0000 7fe9255ef700 1 --2- 172.21.15.78:0/3737053108 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe90c065ed0 0x7fe90c068380 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:27.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.981+0000 7fe9255ef700 1 --2- 172.21.15.78:0/3737053108 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe9200d29d0 0x7fe9201e68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:27.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.981+0000 7fe9255ef700 1 --2- 172.21.15.78:0/3737053108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9200b3610 0x7fe9200c7b10 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:27.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.981+0000 7fe9255ef700 1 --2- 172.21.15.78:0/3737053108 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9200b3240 0x7fe92015ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:27.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.981+0000 7fe9255ef700 1 -- 172.21.15.78:0/3737053108 >> 172.21.15.78:0/3737053108 conn(0x7fe92009c260 msgr2=0x7fe92006b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:27.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.981+0000 7fe9255ef700 1 -- 172.21.15.78:0/3737053108 shutdown_connections 2023-10-11T09:41:27.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:27.981+0000 7fe9255ef700 1 -- 172.21.15.78:0/3737053108 wait complete. 2023-10-11T09:41:28.010 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:41:28.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.177+0000 7f70a6916700 1 Processor -- start 2023-10-11T09:41:28.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.177+0000 7f70a6916700 1 -- start start 2023-10-11T09:41:28.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.181+0000 7f70a6916700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f70a015a6b0 0x7f70a00ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:28.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.181+0000 7f70a6916700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70a00b2d30 con 0x7f70a00b3a60 2023-10-11T09:41:28.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.181+0000 7f70a6916700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70a00cf360 con 0x7f70a006d730 2023-10-11T09:41:28.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.181+0000 7f70a6916700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70a00b2f80 con 0x7f70a015a6b0 2023-10-11T09:41:28.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.181+0000 7f70a5113700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f70a00b3a60 0x7f70a006d020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:57528/0 (socket says 172.21.15.78:57528) 2023-10-11T09:41:28.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.181+0000 7f70a5113700 1 -- 172.21.15.78:0/1398703111 learned_addr learned my addr 172.21.15.78:0/1398703111 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:28.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.181+0000 7f70a5914700 1 --2- 172.21.15.78:0/1398703111 >> v2:172.21.15.133:3300/0 conn(0x7f70a015a6b0 0x7f70a00ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:28.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.181+0000 7f70a4912700 1 -- 172.21.15.78:0/1398703111 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 260487514 0 0) 0x7f70a00b2d30 con 0x7f70a00b3a60 2023-10-11T09:41:28.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.181+0000 7f70a4912700 1 -- 172.21.15.78:0/1398703111 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f708c0035d0 con 0x7f70a00b3a60 2023-10-11T09:41:28.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.181+0000 7f70a5914700 1 -- 172.21.15.78:0/1398703111 >> v1:172.21.15.133:6789/0 conn(0x7f70a006d730 legacy=0x7f70a006bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:28.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.181+0000 7f70a5914700 1 -- 172.21.15.78:0/1398703111 >> v1:172.21.15.78:6789/0 conn(0x7f70a00b3a60 legacy=0x7f70a006d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:28.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.181+0000 7f70a5914700 1 -- 172.21.15.78:0/1398703111 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a00b3260 con 0x7f70a015a6b0 2023-10-11T09:41:28.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.185+0000 7f70a5914700 1 --2- 172.21.15.78:0/1398703111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70a015a6b0 0x7f70a00ce8a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f709c009d90 tx=0x7f709c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3230420c33bff23f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:28.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.185+0000 7f70a4912700 1 -- 172.21.15.78:0/1398703111 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c00e070 con 0x7f70a015a6b0 2023-10-11T09:41:28.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.185+0000 7f70a4912700 1 -- 172.21.15.78:0/1398703111 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f709c00bb20 con 0x7f70a015a6b0 2023-10-11T09:41:28.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.185+0000 7f70a4912700 1 -- 172.21.15.78:0/1398703111 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c012430 con 0x7f70a015a6b0 2023-10-11T09:41:28.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.185+0000 7f70a6916700 1 -- 172.21.15.78:0/1398703111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70a015a6b0 msgr2=0x7f70a00ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:28.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.185+0000 7f70a6916700 1 --2- 172.21.15.78:0/1398703111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70a015a6b0 0x7f70a00ce8a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f709c009d90 tx=0x7f709c00b330 comp rx=0 tx=0).stop 2023-10-11T09:41:28.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.185+0000 7f70a6916700 1 -- 172.21.15.78:0/1398703111 shutdown_connections 2023-10-11T09:41:28.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.185+0000 7f70a6916700 1 --2- 172.21.15.78:0/1398703111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70a015a6b0 0x7f70a00ce8a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:28.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.185+0000 7f70a6916700 1 -- 172.21.15.78:0/1398703111 >> 172.21.15.78:0/1398703111 conn(0x7f70a006a060 msgr2=0x7f70a006a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:28.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.185+0000 7f70a6916700 1 -- 172.21.15.78:0/1398703111 shutdown_connections 2023-10-11T09:41:28.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.185+0000 7f70a6916700 1 -- 172.21.15.78:0/1398703111 wait complete. 2023-10-11T09:41:28.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.189+0000 7f70a6916700 1 Processor -- start 2023-10-11T09:41:28.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.189+0000 7f70a6916700 1 -- start start 2023-10-11T09:41:28.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.189+0000 7f70a6916700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70a006d730 0x7f70a014e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:28.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.189+0000 7f70a6916700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f70a00b3a60 0x7f70a015dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:28.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.189+0000 7f70a6916700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70a015a6b0 0x7f70a015e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:28.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.189+0000 7f70a6916700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a0070660 con 0x7f70a006d730 2023-10-11T09:41:28.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.189+0000 7f70a6916700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a01f2b70 con 0x7f70a015a6b0 2023-10-11T09:41:28.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.189+0000 7f70a6916700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70a0070010 con 0x7f70a00b3a60 2023-10-11T09:41:28.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.189+0000 7f70a5914700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70a006d730 0x7f70a014e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:28.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.189+0000 7f70a5914700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70a006d730 0x7f70a014e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59420/0 (socket says 172.21.15.78:59420) 2023-10-11T09:41:28.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.189+0000 7f70a5914700 1 -- 172.21.15.78:0/2189319069 learned_addr learned my addr 172.21.15.78:0/2189319069 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:28.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.189+0000 7f70a5113700 1 --2- 172.21.15.78:0/2189319069 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f70a00b3a60 0x7f70a015dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:28.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.189+0000 7f70a5914700 1 -- 172.21.15.78:0/2189319069 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f70a00b3a60 msgr2=0x7f70a015dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:28.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.189+0000 7f70a5914700 1 --2- 172.21.15.78:0/2189319069 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f70a00b3a60 0x7f70a015dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:28.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.189+0000 7f70a5914700 1 -- 172.21.15.78:0/2189319069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70a015a6b0 msgr2=0x7f70a015e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:28.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.189+0000 7f70a6115700 1 --2- 172.21.15.78:0/2189319069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70a015a6b0 0x7f70a015e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:28.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.189+0000 7f70a5914700 1 --2- 172.21.15.78:0/2189319069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70a015a6b0 0x7f70a015e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:28.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.189+0000 7f70a5914700 1 -- 172.21.15.78:0/2189319069 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f709c009a70 con 0x7f70a006d730 2023-10-11T09:41:28.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.189+0000 7f70a5113700 1 --2- 172.21.15.78:0/2189319069 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f70a00b3a60 0x7f70a015dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:41:28.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.189+0000 7f70a6115700 1 --2- 172.21.15.78:0/2189319069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70a015a6b0 0x7f70a015e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:41:28.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.189+0000 7f70a5914700 1 --2- 172.21.15.78:0/2189319069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70a006d730 0x7f70a014e940 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f709c014040 tx=0x7f709c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:28.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.189+0000 7f70967fc700 1 -- 172.21.15.78:0/2189319069 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c012430 con 0x7f70a006d730 2023-10-11T09:41:28.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.189+0000 7f70967fc700 1 -- 172.21.15.78:0/2189319069 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f709c020070 con 0x7f70a006d730 2023-10-11T09:41:28.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.189+0000 7f70967fc700 1 -- 172.21.15.78:0/2189319069 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c004260 con 0x7f70a006d730 2023-10-11T09:41:28.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.189+0000 7f70a6916700 1 -- 172.21.15.78:0/2189319069 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70a00702f0 con 0x7f70a006d730 2023-10-11T09:41:28.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.193+0000 7f70a6916700 1 -- 172.21.15.78:0/2189319069 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a01f2e50 con 0x7f70a006d730 2023-10-11T09:41:28.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.193+0000 7f70a6916700 1 -- 172.21.15.78:0/2189319069 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70a0065d40 con 0x7f70a006d730 2023-10-11T09:41:28.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.193+0000 7f70967fc700 1 -- 172.21.15.78:0/2189319069 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f709c0043c0 con 0x7f70a006d730 2023-10-11T09:41:28.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.197+0000 7f70967fc700 1 --2- 172.21.15.78:0/2189319069 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7084065ed0 0x7f7084068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:28.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.197+0000 7f70967fc700 1 -- 172.21.15.78:0/2189319069 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f709c022070 con 0x7f70a006d730 2023-10-11T09:41:28.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.197+0000 7f70a5113700 1 --2- 172.21.15.78:0/2189319069 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7084065ed0 0x7f7084068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:28.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.197+0000 7f70a5113700 1 --2- 172.21.15.78:0/2189319069 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7084065ed0 0x7f7084068380 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f70a00b0e80 tx=0x7f7090009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:28.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.201+0000 7f70967fc700 1 -- 172.21.15.78:0/2189319069 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f709c052be0 con 0x7f70a006d730 2023-10-11T09:41:28.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.381+0000 7f70a6916700 1 -- 172.21.15.78:0/2189319069 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f70a006b570 con 0x7f70a006d730 2023-10-11T09:41:28.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.381+0000 7f70967fc700 1 -- 172.21.15.78:0/2189319069 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f709c056200 con 0x7f70a006d730 2023-10-11T09:41:28.386 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:41:28.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.381+0000 7f70a6916700 1 -- 172.21.15.78:0/2189319069 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7084065ed0 msgr2=0x7f7084068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:28.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.381+0000 7f70a6916700 1 --2- 172.21.15.78:0/2189319069 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7084065ed0 0x7f7084068380 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f70a00b0e80 tx=0x7f7090009040 comp rx=0 tx=0).stop 2023-10-11T09:41:28.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.381+0000 7f70a6916700 1 -- 172.21.15.78:0/2189319069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70a006d730 msgr2=0x7f70a014e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:28.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.381+0000 7f70a6916700 1 --2- 172.21.15.78:0/2189319069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70a006d730 0x7f70a014e940 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f709c014040 tx=0x7f709c004af0 comp rx=0 tx=0).stop 2023-10-11T09:41:28.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.381+0000 7f70a6916700 1 -- 172.21.15.78:0/2189319069 shutdown_connections 2023-10-11T09:41:28.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.381+0000 7f70a6916700 1 --2- 172.21.15.78:0/2189319069 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7084065ed0 0x7f7084068380 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:28.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.381+0000 7f70a6916700 1 --2- 172.21.15.78:0/2189319069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70a015a6b0 0x7f70a015e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:28.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.381+0000 7f70a6916700 1 --2- 172.21.15.78:0/2189319069 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f70a00b3a60 0x7f70a015dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:28.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.381+0000 7f70a6916700 1 --2- 172.21.15.78:0/2189319069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70a006d730 0x7f70a014e940 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:28.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.381+0000 7f70a6916700 1 -- 172.21.15.78:0/2189319069 >> 172.21.15.78:0/2189319069 conn(0x7f70a006a060 msgr2=0x7f70a014f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:28.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.385+0000 7f70a6916700 1 -- 172.21.15.78:0/2189319069 shutdown_connections 2023-10-11T09:41:28.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.385+0000 7f70a6916700 1 -- 172.21.15.78:0/2189319069 wait complete. 2023-10-11T09:41:28.409 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T09:41:28.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.581+0000 7f0cabfff700 1 Processor -- start 2023-10-11T09:41:28.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.581+0000 7f0cabfff700 1 -- start start 2023-10-11T09:41:28.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.585+0000 7f0cabfff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0cac15a6b0 0x7f0cac0ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:28.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.585+0000 7f0cabfff700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0cac06d730 0x7f0cac06bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:28.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.585+0000 7f0cabfff700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cac0b2d30 con 0x7f0cac0b3a60 2023-10-11T09:41:28.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.585+0000 7f0cabfff700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cac0cf360 con 0x7f0cac06d730 2023-10-11T09:41:28.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.585+0000 7f0cabfff700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cac0b2f80 con 0x7f0cac15a6b0 2023-10-11T09:41:28.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.585+0000 7f0cab7fe700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0cac06d730 0x7f0cac06bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:28.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.585+0000 7f0caaffd700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0cac15a6b0 0x7f0cac0ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:28.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.589+0000 7f0cab7fe700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0cac06d730 0x7f0cac06bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:36236/0 (socket says 172.21.15.78:36236) 2023-10-11T09:41:28.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.589+0000 7f0cab7fe700 1 -- 172.21.15.78:0/1782978356 learned_addr learned my addr 172.21.15.78:0/1782978356 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:28.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.589+0000 7f0ca27fc700 1 --1- 172.21.15.78:0/1782978356 >> v1:172.21.15.133:6789/0 conn(0x7f0cac0b3a60 0x7f0cac06d020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:44640/0 (socket says 172.21.15.78:44640) 2023-10-11T09:41:28.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.589+0000 7f0caaffd700 1 -- 172.21.15.78:0/1782978356 >> v2:172.21.15.133:3300/0 conn(0x7f0cac06d730 msgr2=0x7f0cac06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:28.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.589+0000 7f0caaffd700 1 --2- 172.21.15.78:0/1782978356 >> v2:172.21.15.133:3300/0 conn(0x7f0cac06d730 0x7f0cac06bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:28.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.589+0000 7f0caaffd700 1 -- 172.21.15.78:0/1782978356 >> v1:172.21.15.133:6789/0 conn(0x7f0cac0b3a60 legacy=0x7f0cac06d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:28.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.589+0000 7f0caaffd700 1 -- 172.21.15.78:0/1782978356 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cac0b3260 con 0x7f0cac15a6b0 2023-10-11T09:41:28.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.589+0000 7f0cab7fe700 1 --2- 172.21.15.78:0/1782978356 >> v2:172.21.15.133:3300/0 conn(0x7f0cac06d730 0x7f0cac06bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:41:28.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.589+0000 7f0caaffd700 1 --2- 172.21.15.78:0/1782978356 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0cac15a6b0 0x7f0cac0ce8a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c009850 tx=0x7f0c9c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6bdf9f769d7a6dd server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:28.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.589+0000 7f0caa7fc700 1 -- 172.21.15.78:0/1782978356 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c004230 con 0x7f0cac15a6b0 2023-10-11T09:41:28.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.589+0000 7f0caa7fc700 1 -- 172.21.15.78:0/1782978356 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c9c004390 con 0x7f0cac15a6b0 2023-10-11T09:41:28.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.589+0000 7f0caa7fc700 1 -- 172.21.15.78:0/1782978356 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c0036c0 con 0x7f0cac15a6b0 2023-10-11T09:41:28.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.589+0000 7f0cabfff700 1 -- 172.21.15.78:0/1782978356 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0cac15a6b0 msgr2=0x7f0cac0ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:28.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.589+0000 7f0cabfff700 1 --2- 172.21.15.78:0/1782978356 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0cac15a6b0 0x7f0cac0ce8a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c009850 tx=0x7f0c9c009b60 comp rx=0 tx=0).stop 2023-10-11T09:41:28.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.589+0000 7f0cabfff700 1 -- 172.21.15.78:0/1782978356 shutdown_connections 2023-10-11T09:41:28.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.589+0000 7f0cabfff700 1 --2- 172.21.15.78:0/1782978356 >> v2:172.21.15.133:3300/0 conn(0x7f0cac06d730 0x7f0cac06bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:28.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.589+0000 7f0cabfff700 1 --2- 172.21.15.78:0/1782978356 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0cac15a6b0 0x7f0cac0ce8a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:28.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.589+0000 7f0cabfff700 1 -- 172.21.15.78:0/1782978356 >> 172.21.15.78:0/1782978356 conn(0x7f0cac06a060 msgr2=0x7f0cac06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:28.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.589+0000 7f0cabfff700 1 -- 172.21.15.78:0/1782978356 shutdown_connections 2023-10-11T09:41:28.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.593+0000 7f0cabfff700 1 -- 172.21.15.78:0/1782978356 wait complete. 2023-10-11T09:41:28.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.593+0000 7f0cabfff700 1 Processor -- start 2023-10-11T09:41:28.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.593+0000 7f0cabfff700 1 -- start start 2023-10-11T09:41:28.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.597+0000 7f0cabfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cac06d730 0x7f0cac1457b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:28.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.597+0000 7f0cabfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0cac0b3a60 0x7f0cac148fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:28.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.597+0000 7f0cabfff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0cac15a6b0 0x7f0cac149510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:28.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.597+0000 7f0cabfff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cac1f6370 con 0x7f0cac06d730 2023-10-11T09:41:28.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.597+0000 7f0cabfff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cac0709e0 con 0x7f0cac0b3a60 2023-10-11T09:41:28.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.597+0000 7f0cabfff700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cac1f56f0 con 0x7f0cac15a6b0 2023-10-11T09:41:28.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.597+0000 7f0caaffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cac06d730 0x7f0cac1457b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:28.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.597+0000 7f0caaffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cac06d730 0x7f0cac1457b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59428/0 (socket says 172.21.15.78:59428) 2023-10-11T09:41:28.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.597+0000 7f0caaffd700 1 -- 172.21.15.78:0/707126569 learned_addr learned my addr 172.21.15.78:0/707126569 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:28.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.597+0000 7f0ca27fc700 1 --2- 172.21.15.78:0/707126569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0cac0b3a60 0x7f0cac148fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:28.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.597+0000 7f0cab7fe700 1 --2- 172.21.15.78:0/707126569 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0cac15a6b0 0x7f0cac149510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:28.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.597+0000 7f0caaffd700 1 -- 172.21.15.78:0/707126569 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0cac15a6b0 msgr2=0x7f0cac149510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:28.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.597+0000 7f0caaffd700 1 --2- 172.21.15.78:0/707126569 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0cac15a6b0 0x7f0cac149510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:28.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.597+0000 7f0caaffd700 1 -- 172.21.15.78:0/707126569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0cac0b3a60 msgr2=0x7f0cac148fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:28.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.597+0000 7f0caaffd700 1 --2- 172.21.15.78:0/707126569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0cac0b3a60 0x7f0cac148fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:28.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.597+0000 7f0caaffd700 1 -- 172.21.15.78:0/707126569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca4009b30 con 0x7f0cac06d730 2023-10-11T09:41:28.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.597+0000 7f0cab7fe700 1 --2- 172.21.15.78:0/707126569 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0cac15a6b0 0x7f0cac149510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:28.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.597+0000 7f0ca27fc700 1 --2- 172.21.15.78:0/707126569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0cac0b3a60 0x7f0cac148fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:28.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.597+0000 7f0caaffd700 1 --2- 172.21.15.78:0/707126569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cac06d730 0x7f0cac1457b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c0090b0 tx=0x7f0c9c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:28.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.597+0000 7f0ca3fff700 1 -- 172.21.15.78:0/707126569 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c003e50 con 0x7f0cac06d730 2023-10-11T09:41:28.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.597+0000 7f0cabfff700 1 -- 172.21.15.78:0/707126569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c9c0094b0 con 0x7f0cac06d730 2023-10-11T09:41:28.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.597+0000 7f0ca3fff700 1 -- 172.21.15.78:0/707126569 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c9c01f070 con 0x7f0cac06d730 2023-10-11T09:41:28.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.597+0000 7f0ca3fff700 1 -- 172.21.15.78:0/707126569 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c011870 con 0x7f0cac06d730 2023-10-11T09:41:28.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.597+0000 7f0cabfff700 1 -- 172.21.15.78:0/707126569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cac1f5cb0 con 0x7f0cac06d730 2023-10-11T09:41:28.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.601+0000 7f0cabfff700 1 -- 172.21.15.78:0/707126569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cac1f64b0 con 0x7f0cac06d730 2023-10-11T09:41:28.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.601+0000 7f0ca3fff700 1 -- 172.21.15.78:0/707126569 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f0c9c021030 con 0x7f0cac06d730 2023-10-11T09:41:28.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.605+0000 7f0ca3fff700 1 --2- 172.21.15.78:0/707126569 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0c8c065ed0 0x7f0c8c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:28.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.605+0000 7f0ca3fff700 1 -- 172.21.15.78:0/707126569 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f0c9c084e40 con 0x7f0cac06d730 2023-10-11T09:41:28.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.605+0000 7f0ca27fc700 1 --2- 172.21.15.78:0/707126569 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0c8c065ed0 0x7f0c8c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:28.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.605+0000 7f0ca27fc700 1 --2- 172.21.15.78:0/707126569 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0c8c065ed0 0x7f0c8c068380 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0cac06cd20 tx=0x7f0c94007e70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:28.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.613+0000 7f0ca3fff700 1 -- 172.21.15.78:0/707126569 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f0c9c0523c0 con 0x7f0cac06d730 2023-10-11T09:41:28.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.789+0000 7f0cabfff700 1 -- 172.21.15.78:0/707126569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0cac1f64b0 con 0x7f0cac06d730 2023-10-11T09:41:28.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.789+0000 7f0ca3fff700 1 -- 172.21.15.78:0/707126569 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0c9c022e60 con 0x7f0cac06d730 2023-10-11T09:41:28.795 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:41:28.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.793+0000 7f0cabfff700 1 -- 172.21.15.78:0/707126569 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0c8c065ed0 msgr2=0x7f0c8c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:28.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.793+0000 7f0cabfff700 1 --2- 172.21.15.78:0/707126569 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0c8c065ed0 0x7f0c8c068380 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0cac06cd20 tx=0x7f0c94007e70 comp rx=0 tx=0).stop 2023-10-11T09:41:28.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.793+0000 7f0cabfff700 1 -- 172.21.15.78:0/707126569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cac06d730 msgr2=0x7f0cac1457b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:28.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.793+0000 7f0cabfff700 1 --2- 172.21.15.78:0/707126569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cac06d730 0x7f0cac1457b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c0090b0 tx=0x7f0c9c004770 comp rx=0 tx=0).stop 2023-10-11T09:41:28.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.793+0000 7f0cabfff700 1 -- 172.21.15.78:0/707126569 shutdown_connections 2023-10-11T09:41:28.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.793+0000 7f0cabfff700 1 --2- 172.21.15.78:0/707126569 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0c8c065ed0 0x7f0c8c068380 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:28.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.793+0000 7f0cabfff700 1 --2- 172.21.15.78:0/707126569 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0cac15a6b0 0x7f0cac149510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:28.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.793+0000 7f0cabfff700 1 --2- 172.21.15.78:0/707126569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0cac0b3a60 0x7f0cac148fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:28.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.793+0000 7f0cabfff700 1 --2- 172.21.15.78:0/707126569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cac06d730 0x7f0cac1457b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:28.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.793+0000 7f0cabfff700 1 -- 172.21.15.78:0/707126569 >> 172.21.15.78:0/707126569 conn(0x7f0cac06a060 msgr2=0x7f0cac06bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:28.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.793+0000 7f0cabfff700 1 -- 172.21.15.78:0/707126569 shutdown_connections 2023-10-11T09:41:28.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.793+0000 7f0cabfff700 1 -- 172.21.15.78:0/707126569 wait complete. 2023-10-11T09:41:28.817 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T09:41:28.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.977+0000 7f6c3b59e700 1 Processor -- start 2023-10-11T09:41:28.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.977+0000 7f6c3b59e700 1 -- start start 2023-10-11T09:41:28.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.981+0000 7f6c3b59e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6c3c0b3240 0x7f6c3c0d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:28.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.981+0000 7f6c3b59e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f6c3c0b3610 0x7f6c3c0b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:28.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.981+0000 7f6c3b59e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6c3c15d040 0x7f6c3c15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:28.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.981+0000 7f6c3b59e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c3c0b3fd0 con 0x7f6c3c0b3610 2023-10-11T09:41:28.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.981+0000 7f6c3b59e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c3c0b47f0 con 0x7f6c3c15d040 2023-10-11T09:41:28.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.981+0000 7f6c3b59e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c0b8130 con 0x7f6c3c0b3240 2023-10-11T09:41:28.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.981+0000 7f6c39d9b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f6c3c0b3610 0x7f6c3c0b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:28.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.981+0000 7f6c39d9b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f6c3c0b3610 0x7f6c3c0b3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59438/0 (socket says 172.21.15.78:59438) 2023-10-11T09:41:28.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.981+0000 7f6c39d9b700 1 -- 172.21.15.78:0/220946521 learned_addr learned my addr 172.21.15.78:0/220946521 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:28.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.981+0000 7f6c3ad9d700 1 --2- 172.21.15.78:0/220946521 >> v2:172.21.15.133:3300/0 conn(0x7f6c3c15d040 0x7f6c3c15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:28.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.981+0000 7f6c3a59c700 1 --2- 172.21.15.78:0/220946521 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6c3c0b3240 0x7f6c3c0d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:28.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.981+0000 7f6c39d9b700 1 -- 172.21.15.78:0/220946521 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6c3c0b3240 msgr2=0x7f6c3c0d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:28.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.981+0000 7f6c39d9b700 1 --2- 172.21.15.78:0/220946521 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6c3c0b3240 0x7f6c3c0d29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:28.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.981+0000 7f6c39d9b700 1 -- 172.21.15.78:0/220946521 >> v2:172.21.15.133:3300/0 conn(0x7f6c3c15d040 msgr2=0x7f6c3c15d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:28.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.981+0000 7f6c39d9b700 1 --2- 172.21.15.78:0/220946521 >> v2:172.21.15.133:3300/0 conn(0x7f6c3c15d040 0x7f6c3c15d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:28.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.981+0000 7f6c39d9b700 1 -- 172.21.15.78:0/220946521 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3c0b41e0 con 0x7f6c3c0b3610 2023-10-11T09:41:28.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.981+0000 7f6c39d9b700 1 --2- 172.21.15.78:0/220946521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c3c0b3610 0x7f6c3c0b3a90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c0672a0 tx=0x7f6c30009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677167646eb202e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:28.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.981+0000 7f6c3959a700 1 -- 172.21.15.78:0/220946521 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c30011830 con 0x7f6c3c0b3610 2023-10-11T09:41:28.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.981+0000 7f6c3959a700 1 -- 172.21.15.78:0/220946521 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c30011990 con 0x7f6c3c0b3610 2023-10-11T09:41:28.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.981+0000 7f6c3959a700 1 -- 172.21.15.78:0/220946521 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c30011d20 con 0x7f6c3c0b3610 2023-10-11T09:41:28.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.985+0000 7f6c3b59e700 1 -- 172.21.15.78:0/220946521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c3c0b3610 msgr2=0x7f6c3c0b3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:28.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.985+0000 7f6c3b59e700 1 --2- 172.21.15.78:0/220946521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c3c0b3610 0x7f6c3c0b3a90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c0672a0 tx=0x7f6c30009bf0 comp rx=0 tx=0).stop 2023-10-11T09:41:28.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.985+0000 7f6c3b59e700 1 -- 172.21.15.78:0/220946521 shutdown_connections 2023-10-11T09:41:28.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.985+0000 7f6c3b59e700 1 --2- 172.21.15.78:0/220946521 >> v2:172.21.15.133:3300/0 conn(0x7f6c3c15d040 0x7f6c3c15d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:28.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.985+0000 7f6c3b59e700 1 --2- 172.21.15.78:0/220946521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c3c0b3610 0x7f6c3c0b3a90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:28.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.985+0000 7f6c3b59e700 1 --2- 172.21.15.78:0/220946521 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6c3c0b3240 0x7f6c3c0d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:28.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.985+0000 7f6c3b59e700 1 -- 172.21.15.78:0/220946521 >> 172.21.15.78:0/220946521 conn(0x7f6c3c09c260 msgr2=0x7f6c3c0d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:28.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.985+0000 7f6c3b59e700 1 -- 172.21.15.78:0/220946521 shutdown_connections 2023-10-11T09:41:28.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.985+0000 7f6c3b59e700 1 -- 172.21.15.78:0/220946521 wait complete. 2023-10-11T09:41:28.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.989+0000 7f6c3b59e700 1 Processor -- start 2023-10-11T09:41:28.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.989+0000 7f6c3b59e700 1 -- start start 2023-10-11T09:41:28.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.989+0000 7f6c3b59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c3c0b3240 0x7f6c3c0a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:28.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.989+0000 7f6c3b59e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6c3c0b3610 0x7f6c3c0c9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:28.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.989+0000 7f6c3b59e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c3c15d040 0x7f6c3c0c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:28.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.989+0000 7f6c3b59e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c0c7730 con 0x7f6c3c0b3240 2023-10-11T09:41:28.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.989+0000 7f6c3b59e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c0c6b60 con 0x7f6c3c15d040 2023-10-11T09:41:28.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.989+0000 7f6c3b59e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c0c6e00 con 0x7f6c3c0b3610 2023-10-11T09:41:28.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.989+0000 7f6c3a59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c3c0b3240 0x7f6c3c0a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:28.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.989+0000 7f6c3a59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c3c0b3240 0x7f6c3c0a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59448/0 (socket says 172.21.15.78:59448) 2023-10-11T09:41:28.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.989+0000 7f6c3a59c700 1 -- 172.21.15.78:0/4177207126 learned_addr learned my addr 172.21.15.78:0/4177207126 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:28.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.989+0000 7f6c3ad9d700 1 --2- 172.21.15.78:0/4177207126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c3c15d040 0x7f6c3c0c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:28.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.989+0000 7f6c39d9b700 1 --2- 172.21.15.78:0/4177207126 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6c3c0b3610 0x7f6c3c0c9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:28.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.989+0000 7f6c3a59c700 1 -- 172.21.15.78:0/4177207126 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6c3c0b3610 msgr2=0x7f6c3c0c9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:28.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.989+0000 7f6c3a59c700 1 --2- 172.21.15.78:0/4177207126 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6c3c0b3610 0x7f6c3c0c9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:28.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.989+0000 7f6c3a59c700 1 -- 172.21.15.78:0/4177207126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c3c15d040 msgr2=0x7f6c3c0c96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:29.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.989+0000 7f6c3a59c700 1 --2- 172.21.15.78:0/4177207126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c3c15d040 0x7f6c3c0c96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:29.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.989+0000 7f6c3a59c700 1 -- 172.21.15.78:0/4177207126 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c30009570 con 0x7f6c3c0b3240 2023-10-11T09:41:29.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.993+0000 7f6c3a59c700 1 --2- 172.21.15.78:0/4177207126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c3c0b3240 0x7f6c3c0a0640 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c00d800 tx=0x7f6c2c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:29.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.993+0000 7f6c2affd700 1 -- 172.21.15.78:0/4177207126 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c014620 con 0x7f6c3c0b3240 2023-10-11T09:41:29.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.993+0000 7f6c2affd700 1 -- 172.21.15.78:0/4177207126 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2c014780 con 0x7f6c3c0b3240 2023-10-11T09:41:29.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.993+0000 7f6c2affd700 1 -- 172.21.15.78:0/4177207126 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c014b10 con 0x7f6c3c0b3240 2023-10-11T09:41:29.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.993+0000 7f6c3b59e700 1 -- 172.21.15.78:0/4177207126 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c3c0c70e0 con 0x7f6c3c0b3240 2023-10-11T09:41:29.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.993+0000 7f6c3b59e700 1 -- 172.21.15.78:0/4177207126 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c3c0a0b80 con 0x7f6c3c0b3240 2023-10-11T09:41:29.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.993+0000 7f6c3b59e700 1 -- 172.21.15.78:0/4177207126 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c3c065d40 con 0x7f6c3c0b3240 2023-10-11T09:41:29.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.997+0000 7f6c2affd700 1 -- 172.21.15.78:0/4177207126 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f6c2c014c70 con 0x7f6c3c0b3240 2023-10-11T09:41:29.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.997+0000 7f6c2affd700 1 --2- 172.21.15.78:0/4177207126 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6c1c065f00 0x7f6c1c0683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:29.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.997+0000 7f6c2affd700 1 -- 172.21.15.78:0/4177207126 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f6c2c083f00 con 0x7f6c3c0b3240 2023-10-11T09:41:29.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.997+0000 7f6c39d9b700 1 --2- 172.21.15.78:0/4177207126 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6c1c065f00 0x7f6c1c0683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:29.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:28.997+0000 7f6c39d9b700 1 --2- 172.21.15.78:0/4177207126 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6c1c065f00 0x7f6c1c0683b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c0b3a40 tx=0x7f6c30008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:29.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.005+0000 7f6c2affd700 1 -- 172.21.15.78:0/4177207126 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f6c2c051360 con 0x7f6c3c0b3240 2023-10-11T09:41:29.145 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:41:29.151 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:41:29.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.177+0000 7f6c3b59e700 1 -- 172.21.15.78:0/4177207126 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6c3c0b2db0 con 0x7f6c3c0b3240 2023-10-11T09:41:29.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.177+0000 7f6c2affd700 1 -- 172.21.15.78:0/4177207126 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6c2c054980 con 0x7f6c3c0b3240 2023-10-11T09:41:29.184 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2023-10-11T09:41:29.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.181+0000 7f6c3b59e700 1 -- 172.21.15.78:0/4177207126 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6c1c065f00 msgr2=0x7f6c1c0683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:29.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.181+0000 7f6c3b59e700 1 --2- 172.21.15.78:0/4177207126 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6c1c065f00 0x7f6c1c0683b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c0b3a40 tx=0x7f6c30008030 comp rx=0 tx=0).stop 2023-10-11T09:41:29.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.181+0000 7f6c3b59e700 1 -- 172.21.15.78:0/4177207126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c3c0b3240 msgr2=0x7f6c3c0a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:29.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.181+0000 7f6c3b59e700 1 --2- 172.21.15.78:0/4177207126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c3c0b3240 0x7f6c3c0a0640 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c00d800 tx=0x7f6c2c00dbc0 comp rx=0 tx=0).stop 2023-10-11T09:41:29.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.181+0000 7f6c3b59e700 1 -- 172.21.15.78:0/4177207126 shutdown_connections 2023-10-11T09:41:29.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.181+0000 7f6c3b59e700 1 --2- 172.21.15.78:0/4177207126 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6c1c065f00 0x7f6c1c0683b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:29.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.181+0000 7f6c3b59e700 1 --2- 172.21.15.78:0/4177207126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c3c15d040 0x7f6c3c0c96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:29.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.181+0000 7f6c3b59e700 1 --2- 172.21.15.78:0/4177207126 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6c3c0b3610 0x7f6c3c0c9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:29.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.181+0000 7f6c3b59e700 1 --2- 172.21.15.78:0/4177207126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c3c0b3240 0x7f6c3c0a0640 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:29.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.181+0000 7f6c3b59e700 1 -- 172.21.15.78:0/4177207126 >> 172.21.15.78:0/4177207126 conn(0x7f6c3c09c260 msgr2=0x7f6c3c0d0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:29.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.181+0000 7f6c3b59e700 1 -- 172.21.15.78:0/4177207126 shutdown_connections 2023-10-11T09:41:29.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.181+0000 7f6c3b59e700 1 -- 172.21.15.78:0/4177207126 wait complete. 2023-10-11T09:41:29.206 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T09:41:29.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.445+0000 7f17fe516700 1 Processor -- start 2023-10-11T09:41:29.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.449+0000 7f17fe516700 1 -- start start 2023-10-11T09:41:29.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.449+0000 7f17fe516700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f17f80cda10 0x7f17f80caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:29.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.449+0000 7f17fe516700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f17f80cdde0 0x7f17f8164610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:29.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.449+0000 7f17fe516700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17f8165390 con 0x7f17f8164b50 2023-10-11T09:41:29.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.449+0000 7f17fe516700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17f80ca440 con 0x7f17f80cdde0 2023-10-11T09:41:29.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.449+0000 7f17fe516700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17f80ca720 con 0x7f17f80cda10 2023-10-11T09:41:29.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.449+0000 7f17fcd13700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f17f80cdde0 0x7f17f8164610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:29.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.449+0000 7f17fcd13700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f17f80cdde0 0x7f17f8164610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59456/0 (socket says 172.21.15.78:59456) 2023-10-11T09:41:29.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.449+0000 7f17fcd13700 1 -- 172.21.15.78:0/242585169 learned_addr learned my addr 172.21.15.78:0/242585169 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:29.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.449+0000 7f17fd514700 1 --2- 172.21.15.78:0/242585169 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f17f80cda10 0x7f17f80caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:29.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.449+0000 7f17fcd13700 1 -- 172.21.15.78:0/242585169 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f17f80cda10 msgr2=0x7f17f80caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:29.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.449+0000 7f17fcd13700 1 --2- 172.21.15.78:0/242585169 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f17f80cda10 0x7f17f80caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:29.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.449+0000 7f17fcd13700 1 -- 172.21.15.78:0/242585169 >> v1:172.21.15.78:6789/0 conn(0x7f17f8164b50 legacy=0x7f17f806bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:29.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.453+0000 7f17fcd13700 1 -- 172.21.15.78:0/242585169 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f80caa00 con 0x7f17f80cdde0 2023-10-11T09:41:29.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.453+0000 7f17fd514700 1 --2- 172.21.15.78:0/242585169 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f17f80cda10 0x7f17f80caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:29.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.453+0000 7f17fcd13700 1 --2- 172.21.15.78:0/242585169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f80cdde0 0x7f17f8164610 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f17e8009910 tx=0x7f17e8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16985084793b6ca1 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:29.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.453+0000 7f17effff700 1 -- 172.21.15.78:0/242585169 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e800ba60 con 0x7f17f80cdde0 2023-10-11T09:41:29.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.453+0000 7f17effff700 1 -- 172.21.15.78:0/242585169 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17e800bbc0 con 0x7f17f80cdde0 2023-10-11T09:41:29.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.453+0000 7f17effff700 1 -- 172.21.15.78:0/242585169 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e8009e60 con 0x7f17f80cdde0 2023-10-11T09:41:29.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.453+0000 7f17fe516700 1 -- 172.21.15.78:0/242585169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f80cdde0 msgr2=0x7f17f8164610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:29.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.453+0000 7f17fe516700 1 --2- 172.21.15.78:0/242585169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f80cdde0 0x7f17f8164610 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f17e8009910 tx=0x7f17e8009c20 comp rx=0 tx=0).stop 2023-10-11T09:41:29.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.453+0000 7f17fe516700 1 -- 172.21.15.78:0/242585169 shutdown_connections 2023-10-11T09:41:29.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.453+0000 7f17fe516700 1 --2- 172.21.15.78:0/242585169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f80cdde0 0x7f17f8164610 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:29.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.453+0000 7f17fe516700 1 --2- 172.21.15.78:0/242585169 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f17f80cda10 0x7f17f80caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:29.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.453+0000 7f17fe516700 1 -- 172.21.15.78:0/242585169 >> 172.21.15.78:0/242585169 conn(0x7f17f80cea60 msgr2=0x7f17f80cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:29.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.453+0000 7f17fe516700 1 -- 172.21.15.78:0/242585169 shutdown_connections 2023-10-11T09:41:29.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.453+0000 7f17fe516700 1 -- 172.21.15.78:0/242585169 wait complete. 2023-10-11T09:41:29.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.457+0000 7f17fe516700 1 Processor -- start 2023-10-11T09:41:29.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.457+0000 7f17fe516700 1 -- start start 2023-10-11T09:41:29.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.457+0000 7f17fe516700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17f80cda10 0x7f17f8146790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:29.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.457+0000 7f17fe516700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f80cdde0 0x7f17f80b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:29.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.457+0000 7f17fe516700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f17f8164b50 0x7f17f81634b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:29.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.457+0000 7f17fe516700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f8163090 con 0x7f17f80cdde0 2023-10-11T09:41:29.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.457+0000 7f17fe516700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f8162410 con 0x7f17f80cda10 2023-10-11T09:41:29.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.457+0000 7f17fe516700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17f81626f0 con 0x7f17f8164b50 2023-10-11T09:41:29.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.457+0000 7f17fcd13700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f80cdde0 0x7f17f80b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:29.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.457+0000 7f17fcd13700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f80cdde0 0x7f17f80b2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59472/0 (socket says 172.21.15.78:59472) 2023-10-11T09:41:29.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.457+0000 7f17fcd13700 1 -- 172.21.15.78:0/2584511766 learned_addr learned my addr 172.21.15.78:0/2584511766 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:29.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.457+0000 7f17fd514700 1 --2- 172.21.15.78:0/2584511766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17f80cda10 0x7f17f8146790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:29.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.457+0000 7f17fcd13700 1 -- 172.21.15.78:0/2584511766 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f17f8164b50 msgr2=0x7f17f81634b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:29.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.457+0000 7f17fdd15700 1 --2- 172.21.15.78:0/2584511766 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f17f8164b50 0x7f17f81634b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:29.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.457+0000 7f17fcd13700 1 --2- 172.21.15.78:0/2584511766 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f17f8164b50 0x7f17f81634b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:29.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.457+0000 7f17fcd13700 1 -- 172.21.15.78:0/2584511766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17f80cda10 msgr2=0x7f17f8146790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:29.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.457+0000 7f17fcd13700 1 --2- 172.21.15.78:0/2584511766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17f80cda10 0x7f17f8146790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:29.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.457+0000 7f17fcd13700 1 -- 172.21.15.78:0/2584511766 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e8009570 con 0x7f17f80cdde0 2023-10-11T09:41:29.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.457+0000 7f17fd514700 1 --2- 172.21.15.78:0/2584511766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17f80cda10 0x7f17f8146790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:41:29.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.457+0000 7f17fdd15700 1 --2- 172.21.15.78:0/2584511766 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f17f8164b50 0x7f17f81634b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:41:29.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.457+0000 7f17fcd13700 1 --2- 172.21.15.78:0/2584511766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f80cdde0 0x7f17f80b2db0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f17f8072690 tx=0x7f17e8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:29.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.461+0000 7f17edffb700 1 -- 172.21.15.78:0/2584511766 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e8004610 con 0x7f17f80cdde0 2023-10-11T09:41:29.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.461+0000 7f17edffb700 1 -- 172.21.15.78:0/2584511766 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17e801f070 con 0x7f17f80cdde0 2023-10-11T09:41:29.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.461+0000 7f17edffb700 1 -- 172.21.15.78:0/2584511766 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e8003900 con 0x7f17f80cdde0 2023-10-11T09:41:29.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.461+0000 7f17fe516700 1 -- 172.21.15.78:0/2584511766 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17f80ce3b0 con 0x7f17f80cdde0 2023-10-11T09:41:29.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.461+0000 7f17fe516700 1 -- 172.21.15.78:0/2584511766 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17f8162cb0 con 0x7f17f80cdde0 2023-10-11T09:41:29.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.461+0000 7f17fe516700 1 -- 172.21.15.78:0/2584511766 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17f81631d0 con 0x7f17f80cdde0 2023-10-11T09:41:29.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.461+0000 7f17edffb700 1 -- 172.21.15.78:0/2584511766 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f17e8003a60 con 0x7f17f80cdde0 2023-10-11T09:41:29.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.465+0000 7f17edffb700 1 --2- 172.21.15.78:0/2584511766 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f17dc065ed0 0x7f17dc068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:29.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.465+0000 7f17edffb700 1 -- 172.21.15.78:0/2584511766 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f17e8021070 con 0x7f17f80cdde0 2023-10-11T09:41:29.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.465+0000 7f17fd514700 1 --2- 172.21.15.78:0/2584511766 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f17dc065ed0 0x7f17dc068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:29.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.465+0000 7f17fd514700 1 --2- 172.21.15.78:0/2584511766 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f17dc065ed0 0x7f17dc068380 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f17f81f9440 tx=0x7f17f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:29.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.469+0000 7f17edffb700 1 -- 172.21.15.78:0/2584511766 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f17f81631d0 con 0x7f17f80cdde0 2023-10-11T09:41:29.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.653+0000 7f17fe516700 1 -- 172.21.15.78:0/2584511766 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f17f81631d0 con 0x7f17f80cdde0 2023-10-11T09:41:29.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.653+0000 7f17edffb700 1 -- 172.21.15.78:0/2584511766 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f17f81631d0 con 0x7f17f80cdde0 2023-10-11T09:41:29.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.657+0000 7f17fe516700 1 -- 172.21.15.78:0/2584511766 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f17dc065ed0 msgr2=0x7f17dc068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:29.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.657+0000 7f17fe516700 1 --2- 172.21.15.78:0/2584511766 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f17dc065ed0 0x7f17dc068380 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f17f81f9440 tx=0x7f17f4008040 comp rx=0 tx=0).stop 2023-10-11T09:41:29.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.657+0000 7f17fe516700 1 -- 172.21.15.78:0/2584511766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f80cdde0 msgr2=0x7f17f80b2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:29.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.657+0000 7f17fe516700 1 --2- 172.21.15.78:0/2584511766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f80cdde0 0x7f17f80b2db0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f17f8072690 tx=0x7f17e8004770 comp rx=0 tx=0).stop 2023-10-11T09:41:29.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.657+0000 7f17fe516700 1 -- 172.21.15.78:0/2584511766 shutdown_connections 2023-10-11T09:41:29.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.657+0000 7f17fe516700 1 --2- 172.21.15.78:0/2584511766 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f17dc065ed0 0x7f17dc068380 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:29.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.657+0000 7f17fe516700 1 --2- 172.21.15.78:0/2584511766 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f17f8164b50 0x7f17f81634b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:29.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.657+0000 7f17fe516700 1 --2- 172.21.15.78:0/2584511766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f80cdde0 0x7f17f80b2db0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:29.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.657+0000 7f17fe516700 1 --2- 172.21.15.78:0/2584511766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17f80cda10 0x7f17f8146790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:29.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.657+0000 7f17fe516700 1 -- 172.21.15.78:0/2584511766 >> 172.21.15.78:0/2584511766 conn(0x7f17f80cea60 msgr2=0x7f17f8149990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:29.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.657+0000 7f17fe516700 1 -- 172.21.15.78:0/2584511766 shutdown_connections 2023-10-11T09:41:29.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.657+0000 7f17fe516700 1 -- 172.21.15.78:0/2584511766 wait complete. 2023-10-11T09:41:29.664 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2023-10-11T09:41:29.680 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:41:29.680 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:41:29.699 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:41:29.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.857+0000 7f12ab757700 1 Processor -- start 2023-10-11T09:41:29.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.857+0000 7f12ab757700 1 -- start start 2023-10-11T09:41:29.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.857+0000 7f12ab757700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f12a415c880 0x7f12a406be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:29.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.857+0000 7f12ab757700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f12a40cbd30 0x7f12a40c9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:29.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.857+0000 7f12ab757700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12a40c93a0 con 0x7f12a40cd210 2023-10-11T09:41:29.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.857+0000 7f12ab757700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12a4161420 con 0x7f12a40cbd30 2023-10-11T09:41:29.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.857+0000 7f12ab757700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a40cca10 con 0x7f12a415c880 2023-10-11T09:41:29.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.857+0000 7f12a9cf4700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f12a40cbd30 0x7f12a40c9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:29.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.857+0000 7f12a9cf4700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f12a40cbd30 0x7f12a40c9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59488/0 (socket says 172.21.15.78:59488) 2023-10-11T09:41:29.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.857+0000 7f12a9cf4700 1 -- 172.21.15.78:0/1167191419 learned_addr learned my addr 172.21.15.78:0/1167191419 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:29.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.857+0000 7f12a94f3700 1 --2- 172.21.15.78:0/1167191419 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f12a415c880 0x7f12a406be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:29.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.857+0000 7f12a9cf4700 1 -- 172.21.15.78:0/1167191419 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f12a415c880 msgr2=0x7f12a406be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:29.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.857+0000 7f12a9cf4700 1 --2- 172.21.15.78:0/1167191419 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f12a415c880 0x7f12a406be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:29.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.857+0000 7f12a9cf4700 1 -- 172.21.15.78:0/1167191419 >> v1:172.21.15.133:6789/0 conn(0x7f12a40cd210 legacy=0x7f12a40cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:29.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.857+0000 7f12a9cf4700 1 -- 172.21.15.78:0/1167191419 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a40cccb0 con 0x7f12a40cbd30 2023-10-11T09:41:29.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.861+0000 7f12a9cf4700 1 --2- 172.21.15.78:0/1167191419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12a40cbd30 0x7f12a40c9b40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f129c009e50 tx=0x7f129c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35f7138108a1d8d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:29.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.861+0000 7f129bfff700 1 -- 172.21.15.78:0/1167191419 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c00e070 con 0x7f12a40cbd30 2023-10-11T09:41:29.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.861+0000 7f129bfff700 1 -- 172.21.15.78:0/1167191419 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129c00b910 con 0x7f12a40cbd30 2023-10-11T09:41:29.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.861+0000 7f129bfff700 1 -- 172.21.15.78:0/1167191419 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c00bcc0 con 0x7f12a40cbd30 2023-10-11T09:41:29.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.861+0000 7f12ab757700 1 -- 172.21.15.78:0/1167191419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12a40cbd30 msgr2=0x7f12a40c9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:29.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.861+0000 7f12ab757700 1 --2- 172.21.15.78:0/1167191419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12a40cbd30 0x7f12a40c9b40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f129c009e50 tx=0x7f129c00b330 comp rx=0 tx=0).stop 2023-10-11T09:41:29.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.861+0000 7f12ab757700 1 -- 172.21.15.78:0/1167191419 shutdown_connections 2023-10-11T09:41:29.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.861+0000 7f12ab757700 1 --2- 172.21.15.78:0/1167191419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12a40cbd30 0x7f12a40c9b40 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:29.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.861+0000 7f12ab757700 1 --2- 172.21.15.78:0/1167191419 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f12a415c880 0x7f12a406be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:29.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.861+0000 7f12ab757700 1 -- 172.21.15.78:0/1167191419 >> 172.21.15.78:0/1167191419 conn(0x7f12a406aca0 msgr2=0x7f12a40b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:29.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.861+0000 7f12ab757700 1 -- 172.21.15.78:0/1167191419 shutdown_connections 2023-10-11T09:41:29.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.861+0000 7f12ab757700 1 -- 172.21.15.78:0/1167191419 wait complete. 2023-10-11T09:41:29.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.865+0000 7f12ab757700 1 Processor -- start 2023-10-11T09:41:29.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.865+0000 7f12ab757700 1 -- start start 2023-10-11T09:41:29.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.865+0000 7f12ab757700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12a40cbd30 0x7f12a4150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:29.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.865+0000 7f12ab757700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12a40cd210 0x7f12a415ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:29.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.865+0000 7f12ab757700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f12a415c880 0x7f12a4153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:29.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.865+0000 7f12ab757700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a4159fa0 con 0x7f12a40cbd30 2023-10-11T09:41:29.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.865+0000 7f12ab757700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a415a280 con 0x7f12a40cd210 2023-10-11T09:41:29.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.865+0000 7f12ab757700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a415a560 con 0x7f12a415c880 2023-10-11T09:41:29.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.865+0000 7f12a94f3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12a40cbd30 0x7f12a4150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:29.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.865+0000 7f12a94f3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12a40cbd30 0x7f12a4150950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59502/0 (socket says 172.21.15.78:59502) 2023-10-11T09:41:29.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.865+0000 7f12a94f3700 1 -- 172.21.15.78:0/4139351556 learned_addr learned my addr 172.21.15.78:0/4139351556 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:29.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.865+0000 7f12a8cf2700 1 --2- 172.21.15.78:0/4139351556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12a40cd210 0x7f12a415ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:29.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.865+0000 7f12a9cf4700 1 --2- 172.21.15.78:0/4139351556 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f12a415c880 0x7f12a4153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:29.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.865+0000 7f12a94f3700 1 -- 172.21.15.78:0/4139351556 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f12a415c880 msgr2=0x7f12a4153490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:29.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.865+0000 7f12a94f3700 1 --2- 172.21.15.78:0/4139351556 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f12a415c880 0x7f12a4153490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:29.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.865+0000 7f12a94f3700 1 -- 172.21.15.78:0/4139351556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12a40cd210 msgr2=0x7f12a415ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:29.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.865+0000 7f12a94f3700 1 --2- 172.21.15.78:0/4139351556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12a40cd210 0x7f12a415ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:29.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.865+0000 7f12a94f3700 1 -- 172.21.15.78:0/4139351556 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f129c009b30 con 0x7f12a40cbd30 2023-10-11T09:41:29.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.865+0000 7f12a94f3700 1 --2- 172.21.15.78:0/4139351556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12a40cbd30 0x7f12a4150950 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f12a000e840 tx=0x7f12a000eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:29.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.869+0000 7f1299ffb700 1 -- 172.21.15.78:0/4139351556 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a00095a0 con 0x7f12a40cbd30 2023-10-11T09:41:29.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.869+0000 7f1299ffb700 1 -- 172.21.15.78:0/4139351556 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a0013680 con 0x7f12a40cbd30 2023-10-11T09:41:29.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.869+0000 7f1299ffb700 1 -- 172.21.15.78:0/4139351556 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a00139f0 con 0x7f12a40cbd30 2023-10-11T09:41:29.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.869+0000 7f12ab757700 1 -- 172.21.15.78:0/4139351556 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a415a840 con 0x7f12a40cbd30 2023-10-11T09:41:29.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.869+0000 7f12ab757700 1 -- 172.21.15.78:0/4139351556 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a4153f80 con 0x7f12a40cbd30 2023-10-11T09:41:29.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.869+0000 7f12ab757700 1 -- 172.21.15.78:0/4139351556 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12a415a9d0 con 0x7f12a40cbd30 2023-10-11T09:41:29.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.869+0000 7f1299ffb700 1 -- 172.21.15.78:0/4139351556 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f12a0013b50 con 0x7f12a40cbd30 2023-10-11T09:41:29.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.873+0000 7f1299ffb700 1 --2- 172.21.15.78:0/4139351556 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1288065ed0 0x7f1288068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:29.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.873+0000 7f1299ffb700 1 -- 172.21.15.78:0/4139351556 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f12a00836d0 con 0x7f12a40cbd30 2023-10-11T09:41:29.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.873+0000 7f12a8cf2700 1 --2- 172.21.15.78:0/4139351556 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1288065ed0 0x7f1288068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:29.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.873+0000 7f12a8cf2700 1 --2- 172.21.15.78:0/4139351556 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1288065ed0 0x7f1288068380 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f12a41f5920 tx=0x7f1294009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:29.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:29.881+0000 7f1299ffb700 1 -- 172.21.15.78:0/4139351556 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f12a0050d70 con 0x7f12a40cbd30 2023-10-11T09:41:30.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.061+0000 7f12ab757700 1 -- 172.21.15.78:0/4139351556 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12a404e530 con 0x7f12a40cbd30 2023-10-11T09:41:30.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.061+0000 7f1299ffb700 1 -- 172.21.15.78:0/4139351556 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17907 (secure 0 0 0) 0x7f12a0054390 con 0x7f12a40cbd30 2023-10-11T09:41:30.068 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:41:30.068 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":25,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:41:21.473566+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:40:39.862581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T09:40:40.663641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.37","mappings":[{"from":3,"to":2}]},{"pgid":"4.3e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:41:30.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.065+0000 7f12ab757700 1 -- 172.21.15.78:0/4139351556 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1288065ed0 msgr2=0x7f1288068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:30.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.065+0000 7f12ab757700 1 --2- 172.21.15.78:0/4139351556 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1288065ed0 0x7f1288068380 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f12a41f5920 tx=0x7f1294009040 comp rx=0 tx=0).stop 2023-10-11T09:41:30.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.065+0000 7f12ab757700 1 -- 172.21.15.78:0/4139351556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12a40cbd30 msgr2=0x7f12a4150950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:30.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.065+0000 7f12ab757700 1 --2- 172.21.15.78:0/4139351556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12a40cbd30 0x7f12a4150950 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f12a000e840 tx=0x7f12a000eb50 comp rx=0 tx=0).stop 2023-10-11T09:41:30.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.065+0000 7f12ab757700 1 -- 172.21.15.78:0/4139351556 shutdown_connections 2023-10-11T09:41:30.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.065+0000 7f12ab757700 1 --2- 172.21.15.78:0/4139351556 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1288065ed0 0x7f1288068380 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:30.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.065+0000 7f12ab757700 1 --2- 172.21.15.78:0/4139351556 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f12a415c880 0x7f12a4153490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:30.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.065+0000 7f12ab757700 1 --2- 172.21.15.78:0/4139351556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12a40cd210 0x7f12a415ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:30.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.065+0000 7f12ab757700 1 --2- 172.21.15.78:0/4139351556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12a40cbd30 0x7f12a4150950 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:30.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.065+0000 7f12ab757700 1 -- 172.21.15.78:0/4139351556 >> 172.21.15.78:0/4139351556 conn(0x7f12a406aca0 msgr2=0x7f12a406fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:30.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.065+0000 7f12ab757700 1 -- 172.21.15.78:0/4139351556 shutdown_connections 2023-10-11T09:41:30.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.065+0000 7f12ab757700 1 -- 172.21.15.78:0/4139351556 wait complete. 2023-10-11T09:41:30.094 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T09:40:39.862581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T09:40:40.663641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:41:30.095 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:41:30.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.269+0000 7f10abaa3700 1 Processor -- start 2023-10-11T09:41:30.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.269+0000 7f10abaa3700 1 -- start start 2023-10-11T09:41:30.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.273+0000 7f10abaa3700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f10a4160cb0 0x7f10a40cb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:30.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.273+0000 7f10abaa3700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10a406d080 0x7f10a40b0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:30.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.273+0000 7f10abaa3700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f10a40b0e60 0x7f10a406be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:30.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.273+0000 7f10abaa3700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10a4161080 con 0x7f10a40b0e60 2023-10-11T09:41:30.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.273+0000 7f10abaa3700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10a4164750 con 0x7f10a4160cb0 2023-10-11T09:41:30.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.273+0000 7f10abaa3700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a40b12e0 con 0x7f10a406d080 2023-10-11T09:41:30.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.273+0000 7f10aa040700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f10a40b0e60 0x7f10a406be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:30.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.273+0000 7f10aa040700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f10a40b0e60 0x7f10a406be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59514/0 (socket says 172.21.15.78:59514) 2023-10-11T09:41:30.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.273+0000 7f10aa040700 1 -- 172.21.15.78:0/1104272523 learned_addr learned my addr 172.21.15.78:0/1104272523 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:30.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.273+0000 7f10a903e700 1 --2- 172.21.15.78:0/1104272523 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10a406d080 0x7f10a40b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:30.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.273+0000 7f10aa040700 1 -- 172.21.15.78:0/1104272523 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10a406d080 msgr2=0x7f10a40b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:30.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.273+0000 7f10aa040700 1 --2- 172.21.15.78:0/1104272523 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10a406d080 0x7f10a40b0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:30.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.273+0000 7f10a983f700 1 --2- 172.21.15.78:0/1104272523 >> v2:172.21.15.133:3300/0 conn(0x7f10a4160cb0 0x7f10a40cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:30.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.273+0000 7f10aa040700 1 -- 172.21.15.78:0/1104272523 >> v2:172.21.15.133:3300/0 conn(0x7f10a4160cb0 msgr2=0x7f10a40cb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:30.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.273+0000 7f10aa040700 1 --2- 172.21.15.78:0/1104272523 >> v2:172.21.15.133:3300/0 conn(0x7f10a4160cb0 0x7f10a40cb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:30.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.273+0000 7f10aa040700 1 -- 172.21.15.78:0/1104272523 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a4165570 con 0x7f10a40b0e60 2023-10-11T09:41:30.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.273+0000 7f10a903e700 1 --2- 172.21.15.78:0/1104272523 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10a406d080 0x7f10a40b0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:41:30.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.273+0000 7f10a983f700 1 --2- 172.21.15.78:0/1104272523 >> v2:172.21.15.133:3300/0 conn(0x7f10a4160cb0 0x7f10a40cb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:41:30.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.273+0000 7f10aa040700 1 --2- 172.21.15.78:0/1104272523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10a40b0e60 0x7f10a406be60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f10a40726f0 tx=0x7f109c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9909028b58ea8758 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:30.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.277+0000 7f10a883d700 1 -- 172.21.15.78:0/1104272523 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c00e070 con 0x7f10a40b0e60 2023-10-11T09:41:30.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.277+0000 7f10a883d700 1 -- 172.21.15.78:0/1104272523 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109c00b9c0 con 0x7f10a40b0e60 2023-10-11T09:41:30.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.277+0000 7f10a883d700 1 -- 172.21.15.78:0/1104272523 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c00bd70 con 0x7f10a40b0e60 2023-10-11T09:41:30.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.277+0000 7f10abaa3700 1 -- 172.21.15.78:0/1104272523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10a40b0e60 msgr2=0x7f10a406be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:30.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.277+0000 7f10abaa3700 1 --2- 172.21.15.78:0/1104272523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10a40b0e60 0x7f10a406be60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f10a40726f0 tx=0x7f109c00b330 comp rx=0 tx=0).stop 2023-10-11T09:41:30.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.277+0000 7f10abaa3700 1 -- 172.21.15.78:0/1104272523 shutdown_connections 2023-10-11T09:41:30.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.277+0000 7f10abaa3700 1 --2- 172.21.15.78:0/1104272523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10a40b0e60 0x7f10a406be60 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:30.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.277+0000 7f10abaa3700 1 --2- 172.21.15.78:0/1104272523 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10a406d080 0x7f10a40b0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:30.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.277+0000 7f10abaa3700 1 --2- 172.21.15.78:0/1104272523 >> v2:172.21.15.133:3300/0 conn(0x7f10a4160cb0 0x7f10a40cb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:30.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.277+0000 7f10abaa3700 1 -- 172.21.15.78:0/1104272523 >> 172.21.15.78:0/1104272523 conn(0x7f10a40cecc0 msgr2=0x7f10a40cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:30.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.277+0000 7f10abaa3700 1 -- 172.21.15.78:0/1104272523 shutdown_connections 2023-10-11T09:41:30.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.277+0000 7f10abaa3700 1 -- 172.21.15.78:0/1104272523 wait complete. 2023-10-11T09:41:30.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.277+0000 7f10abaa3700 1 Processor -- start 2023-10-11T09:41:30.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.277+0000 7f10abaa3700 1 -- start start 2023-10-11T09:41:30.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.281+0000 7f10abaa3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10a406d080 0x7f10a41f6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:30.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.281+0000 7f10abaa3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10a40b0e60 0x7f10a414be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:30.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.281+0000 7f10abaa3700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10a4160cb0 0x7f10a4156cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:30.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.281+0000 7f10abaa3700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a41f6e70 con 0x7f10a406d080 2023-10-11T09:41:30.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.281+0000 7f10abaa3700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a4161aa0 con 0x7f10a40b0e60 2023-10-11T09:41:30.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.281+0000 7f10abaa3700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a4161d80 con 0x7f10a4160cb0 2023-10-11T09:41:30.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.281+0000 7f10a983f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10a406d080 0x7f10a41f6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:30.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.281+0000 7f10a983f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10a406d080 0x7f10a41f6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59530/0 (socket says 172.21.15.78:59530) 2023-10-11T09:41:30.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.281+0000 7f10a983f700 1 -- 172.21.15.78:0/4244467110 learned_addr learned my addr 172.21.15.78:0/4244467110 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:30.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.281+0000 7f10a903e700 1 --2- 172.21.15.78:0/4244467110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10a40b0e60 0x7f10a414be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:30.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.281+0000 7f10aa040700 1 --2- 172.21.15.78:0/4244467110 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10a4160cb0 0x7f10a4156cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:30.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.281+0000 7f10a983f700 1 -- 172.21.15.78:0/4244467110 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10a4160cb0 msgr2=0x7f10a4156cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:30.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.281+0000 7f10a983f700 1 --2- 172.21.15.78:0/4244467110 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10a4160cb0 0x7f10a4156cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:30.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.281+0000 7f10a983f700 1 -- 172.21.15.78:0/4244467110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10a40b0e60 msgr2=0x7f10a414be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:30.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.281+0000 7f10a983f700 1 --2- 172.21.15.78:0/4244467110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10a40b0e60 0x7f10a414be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:30.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.281+0000 7f10a983f700 1 -- 172.21.15.78:0/4244467110 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f109c009b30 con 0x7f10a406d080 2023-10-11T09:41:30.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.281+0000 7f10a903e700 1 --2- 172.21.15.78:0/4244467110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10a40b0e60 0x7f10a414be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:30.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.281+0000 7f10a983f700 1 --2- 172.21.15.78:0/4244467110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10a406d080 0x7f10a41f6930 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f10a000c7c0 tx=0x7f10a000cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:30.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.281+0000 7f109a7fc700 1 -- 172.21.15.78:0/4244467110 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a0011800 con 0x7f10a406d080 2023-10-11T09:41:30.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.281+0000 7f109a7fc700 1 -- 172.21.15.78:0/4244467110 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a0011960 con 0x7f10a406d080 2023-10-11T09:41:30.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.281+0000 7f10abaa3700 1 -- 172.21.15.78:0/4244467110 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10a4162060 con 0x7f10a406d080 2023-10-11T09:41:30.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.281+0000 7f109a7fc700 1 -- 172.21.15.78:0/4244467110 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a0011cf0 con 0x7f10a406d080 2023-10-11T09:41:30.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.281+0000 7f10abaa3700 1 -- 172.21.15.78:0/4244467110 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a4162620 con 0x7f10a406d080 2023-10-11T09:41:30.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.285+0000 7f10abaa3700 1 -- 172.21.15.78:0/4244467110 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10a406eb80 con 0x7f10a406d080 2023-10-11T09:41:30.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.285+0000 7f109a7fc700 1 -- 172.21.15.78:0/4244467110 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f10a0021350 con 0x7f10a406d080 2023-10-11T09:41:30.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.289+0000 7f109a7fc700 1 --2- 172.21.15.78:0/4244467110 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1088065ed0 0x7f1088068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:30.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.289+0000 7f109a7fc700 1 -- 172.21.15.78:0/4244467110 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f10a00837c0 con 0x7f10a406d080 2023-10-11T09:41:30.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.289+0000 7f10a903e700 1 --2- 172.21.15.78:0/4244467110 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1088065ed0 0x7f1088068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:30.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.289+0000 7f10a903e700 1 --2- 172.21.15.78:0/4244467110 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1088065ed0 0x7f1088068380 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f10a41644e0 tx=0x7f1094008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:30.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.293+0000 7f109a7fc700 1 -- 172.21.15.78:0/4244467110 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f10a0050d40 con 0x7f10a406d080 2023-10-11T09:41:30.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.465+0000 7f10abaa3700 1 -- 172.21.15.78:0/4244467110 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f10a40b4cc0 con 0x7f10a406d080 2023-10-11T09:41:30.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.469+0000 7f109a7fc700 1 -- 172.21.15.78:0/4244467110 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f10a0054360 con 0x7f10a406d080 2023-10-11T09:41:30.473 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:41:30.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.473+0000 7f10abaa3700 1 -- 172.21.15.78:0/4244467110 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1088065ed0 msgr2=0x7f1088068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:30.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.473+0000 7f10abaa3700 1 --2- 172.21.15.78:0/4244467110 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1088065ed0 0x7f1088068380 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f10a41644e0 tx=0x7f1094008040 comp rx=0 tx=0).stop 2023-10-11T09:41:30.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.473+0000 7f10abaa3700 1 -- 172.21.15.78:0/4244467110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10a406d080 msgr2=0x7f10a41f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:30.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.473+0000 7f10abaa3700 1 --2- 172.21.15.78:0/4244467110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10a406d080 0x7f10a41f6930 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f10a000c7c0 tx=0x7f10a000cad0 comp rx=0 tx=0).stop 2023-10-11T09:41:30.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.473+0000 7f10abaa3700 1 -- 172.21.15.78:0/4244467110 shutdown_connections 2023-10-11T09:41:30.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.473+0000 7f10abaa3700 1 --2- 172.21.15.78:0/4244467110 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1088065ed0 0x7f1088068380 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:30.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.473+0000 7f10abaa3700 1 --2- 172.21.15.78:0/4244467110 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10a4160cb0 0x7f10a4156cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:30.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.473+0000 7f10abaa3700 1 --2- 172.21.15.78:0/4244467110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10a40b0e60 0x7f10a414be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:30.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.473+0000 7f10abaa3700 1 --2- 172.21.15.78:0/4244467110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10a406d080 0x7f10a41f6930 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:30.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.473+0000 7f10abaa3700 1 -- 172.21.15.78:0/4244467110 >> 172.21.15.78:0/4244467110 conn(0x7f10a40cecc0 msgr2=0x7f10a40af8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:30.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.473+0000 7f10abaa3700 1 -- 172.21.15.78:0/4244467110 shutdown_connections 2023-10-11T09:41:30.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.473+0000 7f10abaa3700 1 -- 172.21.15.78:0/4244467110 wait complete. 2023-10-11T09:41:30.498 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:41:30.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.665+0000 7f0ba40cf700 1 Processor -- start 2023-10-11T09:41:30.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.669+0000 7f0ba40cf700 1 -- start start 2023-10-11T09:41:30.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.669+0000 7f0ba40cf700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0b9c0b84c0 0x7f0b9c0cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:30.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.669+0000 7f0ba40cf700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0b9c160a80 0x7f0b9c0d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:30.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.669+0000 7f0ba40cf700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b9c0cd690 con 0x7f0b9c0b8890 2023-10-11T09:41:30.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.669+0000 7f0ba40cf700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b9c0cdea0 con 0x7f0b9c160a80 2023-10-11T09:41:30.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.669+0000 7f0ba40cf700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c0d0c10 con 0x7f0b9c0b84c0 2023-10-11T09:41:30.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.669+0000 7f0ba166a700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f0b9c0b8890 0x7f0b9c160370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:57548/0 (socket says 172.21.15.78:57548) 2023-10-11T09:41:30.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.669+0000 7f0ba166a700 1 -- 172.21.15.78:0/83496197 learned_addr learned my addr 172.21.15.78:0/83496197 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:30.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.669+0000 7f0ba266c700 1 --2- 172.21.15.78:0/83496197 >> v2:172.21.15.78:3300/0 conn(0x7f0b9c160a80 0x7f0b9c0d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:30.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.669+0000 7f0ba1e6b700 1 --2- 172.21.15.78:0/83496197 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0b9c0b84c0 0x7f0b9c0cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:30.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.673+0000 7f0ba0e69700 1 -- 172.21.15.78:0/83496197 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1000786008 0 0) 0x7f0b9c0cd690 con 0x7f0b9c0b8890 2023-10-11T09:41:30.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.673+0000 7f0ba0e69700 1 -- 172.21.15.78:0/83496197 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b880035d0 con 0x7f0b9c0b8890 2023-10-11T09:41:30.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.673+0000 7f0ba266c700 1 -- 172.21.15.78:0/83496197 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0b9c0b84c0 msgr2=0x7f0b9c0cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:30.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.673+0000 7f0ba266c700 1 --2- 172.21.15.78:0/83496197 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0b9c0b84c0 0x7f0b9c0cfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:30.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.673+0000 7f0ba266c700 1 -- 172.21.15.78:0/83496197 >> v1:172.21.15.78:6789/0 conn(0x7f0b9c0b8890 legacy=0x7f0b9c160370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:30.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.673+0000 7f0ba266c700 1 -- 172.21.15.78:0/83496197 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b9c0d02e0 con 0x7f0b9c160a80 2023-10-11T09:41:30.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.673+0000 7f0ba1e6b700 1 --2- 172.21.15.78:0/83496197 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0b9c0b84c0 0x7f0b9c0cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:41:30.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.673+0000 7f0ba266c700 1 --2- 172.21.15.78:0/83496197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b9c160a80 0x7f0b9c0d1340 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0b94009d90 tx=0x7f0b9400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d26149569c9a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:30.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.673+0000 7f0ba0e69700 1 -- 172.21.15.78:0/83496197 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9400e070 con 0x7f0b9c160a80 2023-10-11T09:41:30.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.673+0000 7f0ba0e69700 1 -- 172.21.15.78:0/83496197 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b9400bb20 con 0x7f0b9c160a80 2023-10-11T09:41:30.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.673+0000 7f0ba0e69700 1 -- 172.21.15.78:0/83496197 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b94012430 con 0x7f0b9c160a80 2023-10-11T09:41:30.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.673+0000 7f0ba40cf700 1 -- 172.21.15.78:0/83496197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b9c160a80 msgr2=0x7f0b9c0d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:30.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.673+0000 7f0ba40cf700 1 --2- 172.21.15.78:0/83496197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b9c160a80 0x7f0b9c0d1340 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0b94009d90 tx=0x7f0b9400b330 comp rx=0 tx=0).stop 2023-10-11T09:41:30.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.673+0000 7f0ba40cf700 1 -- 172.21.15.78:0/83496197 shutdown_connections 2023-10-11T09:41:30.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.673+0000 7f0ba40cf700 1 --2- 172.21.15.78:0/83496197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b9c160a80 0x7f0b9c0d1340 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:30.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.673+0000 7f0ba40cf700 1 --2- 172.21.15.78:0/83496197 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0b9c0b84c0 0x7f0b9c0cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:30.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.673+0000 7f0ba40cf700 1 -- 172.21.15.78:0/83496197 >> 172.21.15.78:0/83496197 conn(0x7f0b9c06cb50 msgr2=0x7f0b9c06cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:30.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.673+0000 7f0ba40cf700 1 -- 172.21.15.78:0/83496197 shutdown_connections 2023-10-11T09:41:30.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.673+0000 7f0ba40cf700 1 -- 172.21.15.78:0/83496197 wait complete. 2023-10-11T09:41:30.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.677+0000 7f0ba40cf700 1 Processor -- start 2023-10-11T09:41:30.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.677+0000 7f0ba40cf700 1 -- start start 2023-10-11T09:41:30.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.677+0000 7f0ba40cf700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0b9c0b84c0 0x7f0b9c0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:30.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.677+0000 7f0ba40cf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b9c0b8890 0x7f0b9c159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:30.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.677+0000 7f0ba40cf700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b9c160a80 0x7f0b9c1f5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:30.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.677+0000 7f0ba40cf700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c0c9d20 con 0x7f0b9c0b8890 2023-10-11T09:41:30.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.677+0000 7f0ba40cf700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c15a060 con 0x7f0b9c160a80 2023-10-11T09:41:30.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.677+0000 7f0ba40cf700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c1f63c0 con 0x7f0b9c0b84c0 2023-10-11T09:41:30.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.677+0000 7f0ba166a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b9c0b8890 0x7f0b9c159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:30.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.677+0000 7f0ba166a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b9c0b8890 0x7f0b9c159b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59544/0 (socket says 172.21.15.78:59544) 2023-10-11T09:41:30.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.677+0000 7f0ba166a700 1 -- 172.21.15.78:0/28724275 learned_addr learned my addr 172.21.15.78:0/28724275 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:30.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.677+0000 7f0ba266c700 1 --2- 172.21.15.78:0/28724275 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b9c160a80 0x7f0b9c1f5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:30.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.677+0000 7f0ba1e6b700 1 --2- 172.21.15.78:0/28724275 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0b9c0b84c0 0x7f0b9c0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:30.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.677+0000 7f0ba166a700 1 -- 172.21.15.78:0/28724275 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0b9c0b84c0 msgr2=0x7f0b9c0c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:30.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.677+0000 7f0ba166a700 1 --2- 172.21.15.78:0/28724275 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0b9c0b84c0 0x7f0b9c0c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:30.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.677+0000 7f0ba166a700 1 -- 172.21.15.78:0/28724275 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b9c160a80 msgr2=0x7f0b9c1f5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:30.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.677+0000 7f0ba166a700 1 --2- 172.21.15.78:0/28724275 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b9c160a80 0x7f0b9c1f5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:30.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.677+0000 7f0ba166a700 1 -- 172.21.15.78:0/28724275 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b980094b0 con 0x7f0b9c0b8890 2023-10-11T09:41:30.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.677+0000 7f0ba166a700 1 --2- 172.21.15.78:0/28724275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b9c0b8890 0x7f0b9c159b20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c00c700 tx=0x7f0b8c00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:30.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.677+0000 7f0b927fc700 1 -- 172.21.15.78:0/28724275 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c005d90 con 0x7f0b9c0b8890 2023-10-11T09:41:30.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.677+0000 7f0b927fc700 1 -- 172.21.15.78:0/28724275 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b8c0070d0 con 0x7f0b9c0b8890 2023-10-11T09:41:30.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.677+0000 7f0b927fc700 1 -- 172.21.15.78:0/28724275 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c007480 con 0x7f0b9c0b8890 2023-10-11T09:41:30.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.677+0000 7f0ba40cf700 1 -- 172.21.15.78:0/28724275 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b94009a70 con 0x7f0b9c0b8890 2023-10-11T09:41:30.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.677+0000 7f0ba40cf700 1 -- 172.21.15.78:0/28724275 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b9c0c9380 con 0x7f0b9c0b8890 2023-10-11T09:41:30.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.681+0000 7f0ba40cf700 1 -- 172.21.15.78:0/28724275 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b9c0c9e60 con 0x7f0b9c0b8890 2023-10-11T09:41:30.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.681+0000 7f0b927fc700 1 -- 172.21.15.78:0/28724275 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f0b8c0075e0 con 0x7f0b9c0b8890 2023-10-11T09:41:30.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.681+0000 7f0b927fc700 1 --2- 172.21.15.78:0/28724275 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0b80065ed0 0x7f0b80068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:30.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.681+0000 7f0b927fc700 1 -- 172.21.15.78:0/28724275 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f0b8c082e00 con 0x7f0b9c0b8890 2023-10-11T09:41:30.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.681+0000 7f0ba1e6b700 1 --2- 172.21.15.78:0/28724275 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0b80065ed0 0x7f0b80068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:30.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.685+0000 7f0ba1e6b700 1 --2- 172.21.15.78:0/28724275 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0b80065ed0 0x7f0b80068380 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0b98009c70 tx=0x7f0b98008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:30.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.689+0000 7f0b927fc700 1 -- 172.21.15.78:0/28724275 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f0b9c0c9e60 con 0x7f0b9c0b8890 2023-10-11T09:41:30.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.861+0000 7f0ba40cf700 1 -- 172.21.15.78:0/28724275 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0b9c0c9e60 con 0x7f0b9c0b8890 2023-10-11T09:41:30.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.865+0000 7f0b927fc700 1 -- 172.21.15.78:0/28724275 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0b9c0c9e60 con 0x7f0b9c0b8890 2023-10-11T09:41:30.870 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:41:30.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.865+0000 7f0ba40cf700 1 -- 172.21.15.78:0/28724275 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0b80065ed0 msgr2=0x7f0b80068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:30.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.865+0000 7f0ba40cf700 1 --2- 172.21.15.78:0/28724275 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0b80065ed0 0x7f0b80068380 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0b98009c70 tx=0x7f0b98008040 comp rx=0 tx=0).stop 2023-10-11T09:41:30.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.865+0000 7f0ba40cf700 1 -- 172.21.15.78:0/28724275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b9c0b8890 msgr2=0x7f0b9c159b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:30.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.865+0000 7f0ba40cf700 1 --2- 172.21.15.78:0/28724275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b9c0b8890 0x7f0b9c159b20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c00c700 tx=0x7f0b8c00ca10 comp rx=0 tx=0).stop 2023-10-11T09:41:30.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.865+0000 7f0ba40cf700 1 -- 172.21.15.78:0/28724275 shutdown_connections 2023-10-11T09:41:30.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.865+0000 7f0ba40cf700 1 --2- 172.21.15.78:0/28724275 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0b80065ed0 0x7f0b80068380 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:30.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.869+0000 7f0ba40cf700 1 --2- 172.21.15.78:0/28724275 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b9c160a80 0x7f0b9c1f5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:30.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.869+0000 7f0ba40cf700 1 --2- 172.21.15.78:0/28724275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b9c0b8890 0x7f0b9c159b20 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:30.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.869+0000 7f0ba40cf700 1 --2- 172.21.15.78:0/28724275 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0b9c0b84c0 0x7f0b9c0c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:30.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.869+0000 7f0ba40cf700 1 -- 172.21.15.78:0/28724275 >> 172.21.15.78:0/28724275 conn(0x7f0b9c06cb50 msgr2=0x7f0b9c06b040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:30.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.869+0000 7f0ba40cf700 1 -- 172.21.15.78:0/28724275 shutdown_connections 2023-10-11T09:41:30.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:30.869+0000 7f0ba40cf700 1 -- 172.21.15.78:0/28724275 wait complete. 2023-10-11T09:41:30.896 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T09:41:31.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.061+0000 7f51c248c700 1 Processor -- start 2023-10-11T09:41:31.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.061+0000 7f51c248c700 1 -- start start 2023-10-11T09:41:31.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.065+0000 7f51c248c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f51bc146fe0 0x7f51bc06bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:31.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.065+0000 7f51c248c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51bc0b36f0 0x7f51bc1629e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:31.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.065+0000 7f51c248c700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51bc06ccb0 con 0x7f51bc161d90 2023-10-11T09:41:31.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.065+0000 7f51c248c700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51bc06c410 con 0x7f51bc146fe0 2023-10-11T09:41:31.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.065+0000 7f51c248c700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc06c6f0 con 0x7f51bc0b36f0 2023-10-11T09:41:31.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.065+0000 7f51c148a700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f51bc146fe0 0x7f51bc06bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:31.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.065+0000 7f51c148a700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f51bc146fe0 0x7f51bc06bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59558/0 (socket says 172.21.15.78:59558) 2023-10-11T09:41:31.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.065+0000 7f51c148a700 1 -- 172.21.15.78:0/95503624 learned_addr learned my addr 172.21.15.78:0/95503624 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:31.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.065+0000 7f51c1c8b700 1 --2- 172.21.15.78:0/95503624 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51bc0b36f0 0x7f51bc1629e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:31.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.065+0000 7f51c148a700 1 -- 172.21.15.78:0/95503624 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51bc0b36f0 msgr2=0x7f51bc1629e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:31.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.065+0000 7f51c148a700 1 --2- 172.21.15.78:0/95503624 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51bc0b36f0 0x7f51bc1629e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:31.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.065+0000 7f51c148a700 1 -- 172.21.15.78:0/95503624 >> v1:172.21.15.133:6789/0 conn(0x7f51bc161d90 legacy=0x7f51bc0b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:31.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.065+0000 7f51c148a700 1 -- 172.21.15.78:0/95503624 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51bc06c9d0 con 0x7f51bc146fe0 2023-10-11T09:41:31.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.065+0000 7f51c1c8b700 1 --2- 172.21.15.78:0/95503624 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51bc0b36f0 0x7f51bc1629e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:41:31.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.065+0000 7f51c148a700 1 --2- 172.21.15.78:0/95503624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51bc146fe0 0x7f51bc06bbe0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f51b8009910 tx=0x7f51b8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad48497b1481ed7d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:31.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.065+0000 7f51b3fff700 1 -- 172.21.15.78:0/95503624 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b8011830 con 0x7f51bc146fe0 2023-10-11T09:41:31.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.065+0000 7f51b3fff700 1 -- 172.21.15.78:0/95503624 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b8011990 con 0x7f51bc146fe0 2023-10-11T09:41:31.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.065+0000 7f51b3fff700 1 -- 172.21.15.78:0/95503624 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b8011d40 con 0x7f51bc146fe0 2023-10-11T09:41:31.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.069+0000 7f51c248c700 1 -- 172.21.15.78:0/95503624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51bc146fe0 msgr2=0x7f51bc06bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:31.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.069+0000 7f51c248c700 1 --2- 172.21.15.78:0/95503624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51bc146fe0 0x7f51bc06bbe0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f51b8009910 tx=0x7f51b8009c20 comp rx=0 tx=0).stop 2023-10-11T09:41:31.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.069+0000 7f51c248c700 1 -- 172.21.15.78:0/95503624 shutdown_connections 2023-10-11T09:41:31.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.069+0000 7f51c248c700 1 --2- 172.21.15.78:0/95503624 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51bc0b36f0 0x7f51bc1629e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:31.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.069+0000 7f51c248c700 1 --2- 172.21.15.78:0/95503624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51bc146fe0 0x7f51bc06bbe0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:31.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.069+0000 7f51c248c700 1 -- 172.21.15.78:0/95503624 >> 172.21.15.78:0/95503624 conn(0x7f51bc06a060 msgr2=0x7f51bc06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:31.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.069+0000 7f51c248c700 1 -- 172.21.15.78:0/95503624 shutdown_connections 2023-10-11T09:41:31.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.069+0000 7f51c248c700 1 -- 172.21.15.78:0/95503624 wait complete. 2023-10-11T09:41:31.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.069+0000 7f51c248c700 1 Processor -- start 2023-10-11T09:41:31.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.069+0000 7f51c248c700 1 -- start start 2023-10-11T09:41:31.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.073+0000 7f51c248c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51bc0b36f0 0x7f51bc0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:31.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.073+0000 7f51c248c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51bc146fe0 0x7f51bc1f5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:31.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.073+0000 7f51c248c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51bc161d90 0x7f51bc0cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:31.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.073+0000 7f51c248c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc1f63e0 con 0x7f51bc0b36f0 2023-10-11T09:41:31.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.073+0000 7f51c248c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc15b0f0 con 0x7f51bc146fe0 2023-10-11T09:41:31.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.073+0000 7f51c248c700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc0cb820 con 0x7f51bc161d90 2023-10-11T09:41:31.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.073+0000 7f51c148a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51bc0b36f0 0x7f51bc0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:31.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.073+0000 7f51c148a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51bc0b36f0 0x7f51bc0cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59564/0 (socket says 172.21.15.78:59564) 2023-10-11T09:41:31.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.073+0000 7f51c148a700 1 -- 172.21.15.78:0/2157874895 learned_addr learned my addr 172.21.15.78:0/2157874895 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:31.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.073+0000 7f51c0c89700 1 --2- 172.21.15.78:0/2157874895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51bc146fe0 0x7f51bc1f5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:31.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.073+0000 7f51c1c8b700 1 --2- 172.21.15.78:0/2157874895 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51bc161d90 0x7f51bc0cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:31.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.073+0000 7f51c148a700 1 -- 172.21.15.78:0/2157874895 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51bc161d90 msgr2=0x7f51bc0cb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:31.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.073+0000 7f51c148a700 1 --2- 172.21.15.78:0/2157874895 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51bc161d90 0x7f51bc0cb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:31.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.073+0000 7f51c148a700 1 -- 172.21.15.78:0/2157874895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51bc146fe0 msgr2=0x7f51bc1f5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:31.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.073+0000 7f51c148a700 1 --2- 172.21.15.78:0/2157874895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51bc146fe0 0x7f51bc1f5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:31.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.073+0000 7f51c148a700 1 -- 172.21.15.78:0/2157874895 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b4009a70 con 0x7f51bc0b36f0 2023-10-11T09:41:31.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.073+0000 7f51c0c89700 1 --2- 172.21.15.78:0/2157874895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51bc146fe0 0x7f51bc1f5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:31.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.073+0000 7f51c148a700 1 --2- 172.21.15.78:0/2157874895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51bc0b36f0 0x7f51bc0cf290 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f51b8009f70 tx=0x7f51b8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:31.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.073+0000 7f51b1ffb700 1 -- 172.21.15.78:0/2157874895 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b8004590 con 0x7f51bc0b36f0 2023-10-11T09:41:31.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.073+0000 7f51b1ffb700 1 -- 172.21.15.78:0/2157874895 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b801e070 con 0x7f51bc0b36f0 2023-10-11T09:41:31.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.073+0000 7f51c248c700 1 -- 172.21.15.78:0/2157874895 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b8009570 con 0x7f51bc0b36f0 2023-10-11T09:41:31.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.073+0000 7f51b1ffb700 1 -- 172.21.15.78:0/2157874895 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b8003900 con 0x7f51bc0b36f0 2023-10-11T09:41:31.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.073+0000 7f51c248c700 1 -- 172.21.15.78:0/2157874895 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51bc146810 con 0x7f51bc0b36f0 2023-10-11T09:41:31.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.077+0000 7f51c248c700 1 -- 172.21.15.78:0/2157874895 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51bc065d40 con 0x7f51bc0b36f0 2023-10-11T09:41:31.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.077+0000 7f51b1ffb700 1 -- 172.21.15.78:0/2157874895 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f51b8003a60 con 0x7f51bc0b36f0 2023-10-11T09:41:31.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.077+0000 7f51b1ffb700 1 --2- 172.21.15.78:0/2157874895 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f51a0065f00 0x7f51a00683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:31.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.081+0000 7f51b1ffb700 1 -- 172.21.15.78:0/2157874895 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f51b8084670 con 0x7f51bc0b36f0 2023-10-11T09:41:31.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.081+0000 7f51c0c89700 1 --2- 172.21.15.78:0/2157874895 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f51a0065f00 0x7f51a00683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:31.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.081+0000 7f51c0c89700 1 --2- 172.21.15.78:0/2157874895 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f51a0065f00 0x7f51a00683b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f51bc15a080 tx=0x7f51ac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:31.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.085+0000 7f51b1ffb700 1 -- 172.21.15.78:0/2157874895 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f51b8051c70 con 0x7f51bc0b36f0 2023-10-11T09:41:31.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.257+0000 7f51c248c700 1 -- 172.21.15.78:0/2157874895 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f51bc06ce50 con 0x7f51bc0b36f0 2023-10-11T09:41:31.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.257+0000 7f51b1ffb700 1 -- 172.21.15.78:0/2157874895 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f51b8055290 con 0x7f51bc0b36f0 2023-10-11T09:41:31.264 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:41:31.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.261+0000 7f51c248c700 1 -- 172.21.15.78:0/2157874895 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f51a0065f00 msgr2=0x7f51a00683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:31.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.261+0000 7f51c248c700 1 --2- 172.21.15.78:0/2157874895 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f51a0065f00 0x7f51a00683b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f51bc15a080 tx=0x7f51ac009040 comp rx=0 tx=0).stop 2023-10-11T09:41:31.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.261+0000 7f51c248c700 1 -- 172.21.15.78:0/2157874895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51bc0b36f0 msgr2=0x7f51bc0cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:31.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.261+0000 7f51c248c700 1 --2- 172.21.15.78:0/2157874895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51bc0b36f0 0x7f51bc0cf290 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f51b8009f70 tx=0x7f51b8004770 comp rx=0 tx=0).stop 2023-10-11T09:41:31.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.261+0000 7f51c248c700 1 -- 172.21.15.78:0/2157874895 shutdown_connections 2023-10-11T09:41:31.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.261+0000 7f51c248c700 1 --2- 172.21.15.78:0/2157874895 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f51a0065f00 0x7f51a00683b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:31.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.261+0000 7f51c248c700 1 --2- 172.21.15.78:0/2157874895 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51bc161d90 0x7f51bc0cb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:31.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.261+0000 7f51c248c700 1 --2- 172.21.15.78:0/2157874895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51bc146fe0 0x7f51bc1f5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:31.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.261+0000 7f51c248c700 1 --2- 172.21.15.78:0/2157874895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51bc0b36f0 0x7f51bc0cf290 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:31.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.261+0000 7f51c248c700 1 -- 172.21.15.78:0/2157874895 >> 172.21.15.78:0/2157874895 conn(0x7f51bc06a060 msgr2=0x7f51bc1637e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:31.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.261+0000 7f51c248c700 1 -- 172.21.15.78:0/2157874895 shutdown_connections 2023-10-11T09:41:31.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.261+0000 7f51c248c700 1 -- 172.21.15.78:0/2157874895 wait complete. 2023-10-11T09:41:31.288 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T09:41:31.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.457+0000 7f81828e0700 1 Processor -- start 2023-10-11T09:41:31.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.461+0000 7f81828e0700 1 -- start start 2023-10-11T09:41:31.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.461+0000 7f81828e0700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f817c0caf90 0x7f817c15afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:31.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.461+0000 7f81828e0700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f817c15e280 con 0x7f817c158430 2023-10-11T09:41:31.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.461+0000 7f81828e0700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f817c15e560 con 0x7f817c06d340 2023-10-11T09:41:31.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.461+0000 7f81828e0700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f817c070c10 con 0x7f817c0caf90 2023-10-11T09:41:31.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.461+0000 7f817bfff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f817c158430 0x7f817c0710d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:57564/0 (socket says 172.21.15.78:57564) 2023-10-11T09:41:31.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.461+0000 7f817bfff700 1 -- 172.21.15.78:0/1554752373 learned_addr learned my addr 172.21.15.78:0/1554752373 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:31.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.461+0000 7f8180e7d700 1 --2- 172.21.15.78:0/1554752373 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f817c0caf90 0x7f817c15afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:31.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.461+0000 7f817affd700 1 -- 172.21.15.78:0/1554752373 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3846209061 0 0) 0x7f817c15e280 con 0x7f817c158430 2023-10-11T09:41:31.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.461+0000 7f817affd700 1 -- 172.21.15.78:0/1554752373 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81640035d0 con 0x7f817c158430 2023-10-11T09:41:31.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.465+0000 7f817affd700 1 -- 172.21.15.78:0/1554752373 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3476842040 0 0) 0x7f817c15e560 con 0x7f817c06d340 2023-10-11T09:41:31.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.465+0000 7f817affd700 1 -- 172.21.15.78:0/1554752373 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f817c15e280 con 0x7f817c06d340 2023-10-11T09:41:31.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.465+0000 7f817affd700 1 -- 172.21.15.78:0/1554752373 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 825640942 0 0) 0x7f81640035d0 con 0x7f817c158430 2023-10-11T09:41:31.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.465+0000 7f817affd700 1 -- 172.21.15.78:0/1554752373 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f817c0caf90 msgr2=0x7f817c15afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:31.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.465+0000 7f817affd700 1 --2- 172.21.15.78:0/1554752373 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f817c0caf90 0x7f817c15afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:31.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.465+0000 7f817affd700 1 -- 172.21.15.78:0/1554752373 >> v1:172.21.15.133:6789/0 conn(0x7f817c06d340 legacy=0x7f817c0ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:31.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.465+0000 7f817affd700 1 -- 172.21.15.78:0/1554752373 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f817c15e840 con 0x7f817c158430 2023-10-11T09:41:31.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.465+0000 7f817affd700 1 -- 172.21.15.78:0/1554752373 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f816c0019f0 con 0x7f817c158430 2023-10-11T09:41:31.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.465+0000 7f817affd700 1 -- 172.21.15.78:0/1554752373 >> v1:172.21.15.78:6789/0 conn(0x7f817c158430 legacy=0x7f817c0710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:31.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.465+0000 7f817affd700 1 --2- 172.21.15.78:0/1554752373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8164003be0 0x7f8164003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:31.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.465+0000 7f817affd700 1 --2- 172.21.15.78:0/1554752373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8164004870 0x7f8164004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:31.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.465+0000 7f817affd700 1 --2- 172.21.15.78:0/1554752373 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f817c0caf90 0x7f8164005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:31.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.465+0000 7f817affd700 1 -- 172.21.15.78:0/1554752373 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816400bae0 con 0x7f8164004870 2023-10-11T09:41:31.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.465+0000 7f817affd700 1 -- 172.21.15.78:0/1554752373 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816400bc20 con 0x7f8164003be0 2023-10-11T09:41:31.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.465+0000 7f817affd700 1 -- 172.21.15.78:0/1554752373 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f816400bde0 con 0x7f817c0caf90 2023-10-11T09:41:31.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.465+0000 7f817b7fe700 1 --2- 172.21.15.78:0/1554752373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8164004870 0x7f8164004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:31.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.465+0000 7f817bfff700 1 --2- 172.21.15.78:0/1554752373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8164003be0 0x7f8164003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:31.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.465+0000 7f8180e7d700 1 --2- 172.21.15.78:0/1554752373 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f817c0caf90 0x7f8164005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:31.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.465+0000 7f817b7fe700 1 -- 172.21.15.78:0/1554752373 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f817c0caf90 msgr2=0x7f8164005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:31.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.465+0000 7f817b7fe700 1 --2- 172.21.15.78:0/1554752373 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f817c0caf90 0x7f8164005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:31.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.465+0000 7f817b7fe700 1 -- 172.21.15.78:0/1554752373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8164003be0 msgr2=0x7f8164003fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:31.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.465+0000 7f817b7fe700 1 --2- 172.21.15.78:0/1554752373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8164003be0 0x7f8164003fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:31.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.465+0000 7f817b7fe700 1 -- 172.21.15.78:0/1554752373 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81640035d0 con 0x7f8164004870 2023-10-11T09:41:31.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.465+0000 7f817b7fe700 1 --2- 172.21.15.78:0/1554752373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8164004870 0x7f8164004ce0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8170001960 tx=0x7f81640058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5af5e00c1ce76d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:31.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.465+0000 7f817affd700 1 -- 172.21.15.78:0/1554752373 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8170001450 con 0x7f8164004870 2023-10-11T09:41:31.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.465+0000 7f817affd700 1 -- 172.21.15.78:0/1554752373 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81700052f0 con 0x7f8164004870 2023-10-11T09:41:31.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.465+0000 7f817affd700 1 -- 172.21.15.78:0/1554752373 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81700036a0 con 0x7f8164004870 2023-10-11T09:41:31.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.465+0000 7f81828e0700 1 -- 172.21.15.78:0/1554752373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8164004870 msgr2=0x7f8164004ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:31.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.465+0000 7f81828e0700 1 --2- 172.21.15.78:0/1554752373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8164004870 0x7f8164004ce0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8170001960 tx=0x7f81640058a0 comp rx=0 tx=0).stop 2023-10-11T09:41:31.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.465+0000 7f8180e7d700 1 -- 172.21.15.78:0/1554752373 reap_dead start 2023-10-11T09:41:31.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.469+0000 7f81828e0700 1 -- 172.21.15.78:0/1554752373 shutdown_connections 2023-10-11T09:41:31.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.469+0000 7f81828e0700 1 -- 172.21.15.78:0/1554752373 >> 172.21.15.78:0/1554752373 conn(0x7f817c0cecc0 msgr2=0x7f817c0cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:31.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.469+0000 7f81828e0700 1 -- 172.21.15.78:0/1554752373 shutdown_connections 2023-10-11T09:41:31.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.469+0000 7f81828e0700 1 -- 172.21.15.78:0/1554752373 wait complete. 2023-10-11T09:41:31.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.469+0000 7f81828e0700 1 Processor -- start 2023-10-11T09:41:31.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.469+0000 7f81828e0700 1 -- start start 2023-10-11T09:41:31.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.473+0000 7f81828e0700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8164004870 0x7f817c0c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:31.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.473+0000 7f81828e0700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f817c1568f0 0x7f817c1542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:31.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.473+0000 7f81828e0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f817c0c8010 0x7f817c0c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:31.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.473+0000 7f81828e0700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817c0ccf60 con 0x7f817c0c8010 2023-10-11T09:41:31.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.473+0000 7f81828e0700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817c0c89c0 con 0x7f817c1568f0 2023-10-11T09:41:31.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.473+0000 7f81828e0700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f817c157320 con 0x7f8164004870 2023-10-11T09:41:31.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.473+0000 7f8180e7d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f817c0c8010 0x7f817c0c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:31.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.473+0000 7f8180e7d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f817c0c8010 0x7f817c0c8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59592/0 (socket says 172.21.15.78:59592) 2023-10-11T09:41:31.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.473+0000 7f8180e7d700 1 -- 172.21.15.78:0/2481810281 learned_addr learned my addr 172.21.15.78:0/2481810281 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:31.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.473+0000 7f817bfff700 1 --2- 172.21.15.78:0/2481810281 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8164004870 0x7f817c0c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:31.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.473+0000 7f817b7fe700 1 --2- 172.21.15.78:0/2481810281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f817c1568f0 0x7f817c1542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:31.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.473+0000 7f8180e7d700 1 -- 172.21.15.78:0/2481810281 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8164004870 msgr2=0x7f817c0c8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:31.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.473+0000 7f8180e7d700 1 --2- 172.21.15.78:0/2481810281 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8164004870 0x7f817c0c8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:31.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.473+0000 7f8180e7d700 1 -- 172.21.15.78:0/2481810281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f817c1568f0 msgr2=0x7f817c1542d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:31.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.473+0000 7f8180e7d700 1 --2- 172.21.15.78:0/2481810281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f817c1568f0 0x7f817c1542d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:31.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.473+0000 7f8180e7d700 1 -- 172.21.15.78:0/2481810281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f817c157040 con 0x7f817c0c8010 2023-10-11T09:41:31.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.473+0000 7f8180e7d700 1 --2- 172.21.15.78:0/2481810281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f817c0c8010 0x7f817c0c8480 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f817c0cec90 tx=0x7f8174011770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:31.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.473+0000 7f8178ff9700 1 -- 172.21.15.78:0/2481810281 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8174004440 con 0x7f817c0c8010 2023-10-11T09:41:31.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.473+0000 7f8178ff9700 1 -- 172.21.15.78:0/2481810281 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8174011eb0 con 0x7f817c0c8010 2023-10-11T09:41:31.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.473+0000 7f8178ff9700 1 -- 172.21.15.78:0/2481810281 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81740038e0 con 0x7f817c0c8010 2023-10-11T09:41:31.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.473+0000 7f81828e0700 1 -- 172.21.15.78:0/2481810281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f817c0cc8a0 con 0x7f817c0c8010 2023-10-11T09:41:31.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.477+0000 7f81828e0700 1 -- 172.21.15.78:0/2481810281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f817c154810 con 0x7f817c0c8010 2023-10-11T09:41:31.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.477+0000 7f81828e0700 1 -- 172.21.15.78:0/2481810281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f817c06eb80 con 0x7f817c0c8010 2023-10-11T09:41:31.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.477+0000 7f8178ff9700 1 -- 172.21.15.78:0/2481810281 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f8174003a40 con 0x7f817c0c8010 2023-10-11T09:41:31.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.481+0000 7f8178ff9700 1 --2- 172.21.15.78:0/2481810281 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8168065ed0 0x7f8168068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:31.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.481+0000 7f8178ff9700 1 -- 172.21.15.78:0/2481810281 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f8174083510 con 0x7f817c0c8010 2023-10-11T09:41:31.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.481+0000 7f817bfff700 1 --2- 172.21.15.78:0/2481810281 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8168065ed0 0x7f8168068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:31.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.481+0000 7f817bfff700 1 --2- 172.21.15.78:0/2481810281 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8168065ed0 0x7f8168068380 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f816c002230 tx=0x7f816c006040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:31.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.485+0000 7f8178ff9700 1 -- 172.21.15.78:0/2481810281 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f81740509f0 con 0x7f817c0c8010 2023-10-11T09:41:31.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.657+0000 7f81828e0700 1 -- 172.21.15.78:0/2481810281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f817c06d0d0 con 0x7f817c0c8010 2023-10-11T09:41:31.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.661+0000 7f8178ff9700 1 -- 172.21.15.78:0/2481810281 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8174054010 con 0x7f817c0c8010 2023-10-11T09:41:31.666 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2023-10-11T09:41:31.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.665+0000 7f81828e0700 1 -- 172.21.15.78:0/2481810281 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8168065ed0 msgr2=0x7f8168068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:31.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.665+0000 7f81828e0700 1 --2- 172.21.15.78:0/2481810281 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8168065ed0 0x7f8168068380 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f816c002230 tx=0x7f816c006040 comp rx=0 tx=0).stop 2023-10-11T09:41:31.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.665+0000 7f81828e0700 1 -- 172.21.15.78:0/2481810281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f817c0c8010 msgr2=0x7f817c0c8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:31.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.665+0000 7f81828e0700 1 --2- 172.21.15.78:0/2481810281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f817c0c8010 0x7f817c0c8480 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f817c0cec90 tx=0x7f8174011770 comp rx=0 tx=0).stop 2023-10-11T09:41:31.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.665+0000 7f81828e0700 1 -- 172.21.15.78:0/2481810281 shutdown_connections 2023-10-11T09:41:31.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.665+0000 7f81828e0700 1 --2- 172.21.15.78:0/2481810281 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8168065ed0 0x7f8168068380 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:31.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.665+0000 7f81828e0700 1 --2- 172.21.15.78:0/2481810281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f817c0c8010 0x7f817c0c8480 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:31.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.665+0000 7f81828e0700 1 --2- 172.21.15.78:0/2481810281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f817c1568f0 0x7f817c1542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:31.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.665+0000 7f81828e0700 1 --2- 172.21.15.78:0/2481810281 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8164004870 0x7f817c0c8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:31.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.665+0000 7f81828e0700 1 -- 172.21.15.78:0/2481810281 >> 172.21.15.78:0/2481810281 conn(0x7f817c0cecc0 msgr2=0x7f817c0b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:31.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.665+0000 7f81828e0700 1 -- 172.21.15.78:0/2481810281 shutdown_connections 2023-10-11T09:41:31.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.665+0000 7f81828e0700 1 -- 172.21.15.78:0/2481810281 wait complete. 2023-10-11T09:41:31.691 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T09:41:31.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.861+0000 7f73e6b64700 1 Processor -- start 2023-10-11T09:41:31.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.861+0000 7f73e6b64700 1 -- start start 2023-10-11T09:41:31.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.865+0000 7f73e6b64700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f73e00caf20 0x7f73e0164610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:31.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.865+0000 7f73e6b64700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f73e0164b50 0x7f73e006bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:31.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.865+0000 7f73e6b64700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73e0165390 con 0x7f73e00cda10 2023-10-11T09:41:31.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.865+0000 7f73e6b64700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73e00cb3a0 con 0x7f73e00caf20 2023-10-11T09:41:31.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.865+0000 7f73e6b64700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e00ca440 con 0x7f73e0164b50 2023-10-11T09:41:31.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.865+0000 7f73e5361700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f73e00caf20 0x7f73e0164610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:31.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.865+0000 7f73e5361700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f73e00caf20 0x7f73e0164610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59608/0 (socket says 172.21.15.78:59608) 2023-10-11T09:41:31.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.865+0000 7f73e5361700 1 -- 172.21.15.78:0/898185141 learned_addr learned my addr 172.21.15.78:0/898185141 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:31.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.865+0000 7f73e5b62700 1 --1- 172.21.15.78:0/898185141 >> v1:172.21.15.133:6789/0 conn(0x7f73e00cda10 0x7f73e00cdde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:44670/0 (socket says 172.21.15.78:44670) 2023-10-11T09:41:31.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.865+0000 7f73e6363700 1 --2- 172.21.15.78:0/898185141 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f73e0164b50 0x7f73e006bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:31.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.865+0000 7f73e5361700 1 -- 172.21.15.78:0/898185141 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f73e0164b50 msgr2=0x7f73e006bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:31.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.865+0000 7f73e5361700 1 --2- 172.21.15.78:0/898185141 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f73e0164b50 0x7f73e006bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:31.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.865+0000 7f73e5361700 1 -- 172.21.15.78:0/898185141 >> v1:172.21.15.133:6789/0 conn(0x7f73e00cda10 legacy=0x7f73e00cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:31.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.865+0000 7f73e5361700 1 -- 172.21.15.78:0/898185141 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e00ca720 con 0x7f73e00caf20 2023-10-11T09:41:31.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.865+0000 7f73e5361700 1 --2- 172.21.15.78:0/898185141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73e00caf20 0x7f73e0164610 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f73d0009910 tx=0x7f73d0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd158391eaa01e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:31.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.865+0000 7f73e4b60700 1 -- 172.21.15.78:0/898185141 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d000ba60 con 0x7f73e00caf20 2023-10-11T09:41:31.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.869+0000 7f73e4b60700 1 -- 172.21.15.78:0/898185141 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d000bbc0 con 0x7f73e00caf20 2023-10-11T09:41:31.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.869+0000 7f73e4b60700 1 -- 172.21.15.78:0/898185141 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d0009e60 con 0x7f73e00caf20 2023-10-11T09:41:31.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.869+0000 7f73e6b64700 1 -- 172.21.15.78:0/898185141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73e00caf20 msgr2=0x7f73e0164610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:31.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.869+0000 7f73e6b64700 1 --2- 172.21.15.78:0/898185141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73e00caf20 0x7f73e0164610 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f73d0009910 tx=0x7f73d0009c20 comp rx=0 tx=0).stop 2023-10-11T09:41:31.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.869+0000 7f73e6b64700 1 -- 172.21.15.78:0/898185141 shutdown_connections 2023-10-11T09:41:31.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.869+0000 7f73e6b64700 1 --2- 172.21.15.78:0/898185141 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f73e0164b50 0x7f73e006bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:31.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.869+0000 7f73e6b64700 1 --2- 172.21.15.78:0/898185141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73e00caf20 0x7f73e0164610 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:31.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.869+0000 7f73e6b64700 1 -- 172.21.15.78:0/898185141 >> 172.21.15.78:0/898185141 conn(0x7f73e00cea60 msgr2=0x7f73e00cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:31.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.869+0000 7f73e6b64700 1 -- 172.21.15.78:0/898185141 shutdown_connections 2023-10-11T09:41:31.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.869+0000 7f73e6b64700 1 -- 172.21.15.78:0/898185141 wait complete. 2023-10-11T09:41:31.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.873+0000 7f73e6b64700 1 Processor -- start 2023-10-11T09:41:31.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.873+0000 7f73e6b64700 1 -- start start 2023-10-11T09:41:31.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.873+0000 7f73e6b64700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73e00caf20 0x7f73e0144e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:31.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.873+0000 7f73e6b64700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73e00cda10 0x7f73e00aef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:31.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.873+0000 7f73e6b64700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f73e0164b50 0x7f73e00af450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:31.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.873+0000 7f73e6b64700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e0161fe0 con 0x7f73e00cda10 2023-10-11T09:41:31.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.873+0000 7f73e6b64700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e01622c0 con 0x7f73e00caf20 2023-10-11T09:41:31.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.873+0000 7f73e6b64700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e01625a0 con 0x7f73e0164b50 2023-10-11T09:41:31.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.873+0000 7f73e5361700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73e00cda10 0x7f73e00aef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:31.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.873+0000 7f73e5361700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73e00cda10 0x7f73e00aef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59618/0 (socket says 172.21.15.78:59618) 2023-10-11T09:41:31.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.873+0000 7f73e5361700 1 -- 172.21.15.78:0/1121021932 learned_addr learned my addr 172.21.15.78:0/1121021932 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:31.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.873+0000 7f73e5b62700 1 --2- 172.21.15.78:0/1121021932 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73e00caf20 0x7f73e0144e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:31.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.873+0000 7f73e6363700 1 --2- 172.21.15.78:0/1121021932 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f73e0164b50 0x7f73e00af450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:31.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.873+0000 7f73e5361700 1 -- 172.21.15.78:0/1121021932 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f73e0164b50 msgr2=0x7f73e00af450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:31.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.873+0000 7f73e5361700 1 --2- 172.21.15.78:0/1121021932 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f73e0164b50 0x7f73e00af450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:31.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.873+0000 7f73e5361700 1 -- 172.21.15.78:0/1121021932 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73e00caf20 msgr2=0x7f73e0144e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:31.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.873+0000 7f73e5361700 1 --2- 172.21.15.78:0/1121021932 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73e00caf20 0x7f73e0144e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:31.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.873+0000 7f73e5361700 1 -- 172.21.15.78:0/1121021932 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d0009570 con 0x7f73e00cda10 2023-10-11T09:41:31.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.877+0000 7f73e5361700 1 --2- 172.21.15.78:0/1121021932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73e00cda10 0x7f73e00aef10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f73e0072690 tx=0x7f73d0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:31.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.877+0000 7f73d67fc700 1 -- 172.21.15.78:0/1121021932 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d0004610 con 0x7f73e00cda10 2023-10-11T09:41:31.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.877+0000 7f73d67fc700 1 -- 172.21.15.78:0/1121021932 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d001f070 con 0x7f73e00cda10 2023-10-11T09:41:31.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.877+0000 7f73d67fc700 1 -- 172.21.15.78:0/1121021932 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d0003900 con 0x7f73e00cda10 2023-10-11T09:41:31.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.877+0000 7f73e6b64700 1 -- 172.21.15.78:0/1121021932 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73e0162880 con 0x7f73e00cda10 2023-10-11T09:41:31.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.877+0000 7f73e6b64700 1 -- 172.21.15.78:0/1121021932 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73e0145340 con 0x7f73e00cda10 2023-10-11T09:41:31.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.877+0000 7f73e6b64700 1 -- 172.21.15.78:0/1121021932 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73e0162da0 con 0x7f73e00cda10 2023-10-11T09:41:31.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.881+0000 7f73d67fc700 1 -- 172.21.15.78:0/1121021932 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f73d0003a60 con 0x7f73e00cda10 2023-10-11T09:41:31.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.881+0000 7f73d67fc700 1 --2- 172.21.15.78:0/1121021932 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f73c4065ed0 0x7f73c4068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:31.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.881+0000 7f73d67fc700 1 -- 172.21.15.78:0/1121021932 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f73d0021070 con 0x7f73e00cda10 2023-10-11T09:41:31.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.881+0000 7f73e5b62700 1 --2- 172.21.15.78:0/1121021932 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f73c4065ed0 0x7f73c4068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:31.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.881+0000 7f73e5b62700 1 --2- 172.21.15.78:0/1121021932 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f73c4065ed0 0x7f73c4068380 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f73dc000f60 tx=0x7f73dc007d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:31.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:31.889+0000 7f73d67fc700 1 -- 172.21.15.78:0/1121021932 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f73e0162da0 con 0x7f73e00cda10 2023-10-11T09:41:32.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.069+0000 7f73e6b64700 1 -- 172.21.15.78:0/1121021932 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f73e0162da0 con 0x7f73e00cda10 2023-10-11T09:41:32.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.069+0000 7f73d67fc700 1 -- 172.21.15.78:0/1121021932 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f73e0162da0 con 0x7f73e00cda10 2023-10-11T09:41:32.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.073+0000 7f73e6b64700 1 -- 172.21.15.78:0/1121021932 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f73c4065ed0 msgr2=0x7f73c4068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:32.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.073+0000 7f73e6b64700 1 --2- 172.21.15.78:0/1121021932 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f73c4065ed0 0x7f73c4068380 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f73dc000f60 tx=0x7f73dc007d50 comp rx=0 tx=0).stop 2023-10-11T09:41:32.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.073+0000 7f73e6b64700 1 -- 172.21.15.78:0/1121021932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73e00cda10 msgr2=0x7f73e00aef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:32.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.073+0000 7f73e6b64700 1 --2- 172.21.15.78:0/1121021932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73e00cda10 0x7f73e00aef10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f73e0072690 tx=0x7f73d0004770 comp rx=0 tx=0).stop 2023-10-11T09:41:32.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.073+0000 7f73e6b64700 1 -- 172.21.15.78:0/1121021932 shutdown_connections 2023-10-11T09:41:32.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.073+0000 7f73e6b64700 1 --2- 172.21.15.78:0/1121021932 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f73c4065ed0 0x7f73c4068380 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:32.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.073+0000 7f73e6b64700 1 --2- 172.21.15.78:0/1121021932 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f73e0164b50 0x7f73e00af450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:32.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.073+0000 7f73e6b64700 1 --2- 172.21.15.78:0/1121021932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73e00cda10 0x7f73e00aef10 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:32.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.073+0000 7f73e6b64700 1 --2- 172.21.15.78:0/1121021932 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73e00caf20 0x7f73e0144e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:32.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.073+0000 7f73e6b64700 1 -- 172.21.15.78:0/1121021932 >> 172.21.15.78:0/1121021932 conn(0x7f73e00cea60 msgr2=0x7f73e00714a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:32.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.073+0000 7f73e6b64700 1 -- 172.21.15.78:0/1121021932 shutdown_connections 2023-10-11T09:41:32.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.073+0000 7f73e6b64700 1 -- 172.21.15.78:0/1121021932 wait complete. 2023-10-11T09:41:32.081 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2023-10-11T09:41:32.100 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi133.front.sepia.ceph.com... 2023-10-11T09:41:32.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:32.101 DEBUG:teuthology.orchestra.run.smithi133:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-11T09:41:32.150 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:41:32.151 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2023-10-11T09:41:32.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:32.152 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-11T09:41:32.231 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:41:32.231 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:41:32.249 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:41:32.250 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:41:32.311 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:41:32.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.517+0000 7ffa5b178700 1 Processor -- start 2023-10-11T09:41:32.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.517+0000 7ffa5b178700 1 -- start start 2023-10-11T09:41:32.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.521+0000 7ffa5b178700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ffa54160d40 0x7ffa540cd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:32.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.521+0000 7ffa5b178700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffa540ce500 0x7ffa5406be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:32.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.521+0000 7ffa5b178700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa5406ffa0 con 0x7ffa54071040 2023-10-11T09:41:32.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.521+0000 7ffa5b178700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa54070c20 con 0x7ffa54160d40 2023-10-11T09:41:32.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.521+0000 7ffa5b178700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa54161110 con 0x7ffa540ce500 2023-10-11T09:41:32.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.521+0000 7ffa4bfff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7ffa54071040 0x7ffa540cddf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:57572/0 (socket says 172.21.15.78:57572) 2023-10-11T09:41:32.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.521+0000 7ffa4bfff700 1 -- 172.21.15.78:0/414412880 learned_addr learned my addr 172.21.15.78:0/414412880 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:32.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.521+0000 7ffa58f14700 1 --2- 172.21.15.78:0/414412880 >> v2:172.21.15.78:3300/0 conn(0x7ffa54160d40 0x7ffa540cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:32.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.521+0000 7ffa59715700 1 --2- 172.21.15.78:0/414412880 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffa540ce500 0x7ffa5406be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:32.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.521+0000 7ffa4b7fe700 1 -- 172.21.15.78:0/414412880 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1455587388 0 0) 0x7ffa5406ffa0 con 0x7ffa54071040 2023-10-11T09:41:32.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.521+0000 7ffa4b7fe700 1 -- 172.21.15.78:0/414412880 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa3c0035d0 con 0x7ffa54071040 2023-10-11T09:41:32.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.521+0000 7ffa58f14700 1 -- 172.21.15.78:0/414412880 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffa540ce500 msgr2=0x7ffa5406be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:32.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.521+0000 7ffa58f14700 1 --2- 172.21.15.78:0/414412880 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffa540ce500 0x7ffa5406be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:32.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.521+0000 7ffa58f14700 1 -- 172.21.15.78:0/414412880 >> v1:172.21.15.78:6789/0 conn(0x7ffa54071040 legacy=0x7ffa540cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:32.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.521+0000 7ffa58f14700 1 -- 172.21.15.78:0/414412880 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa54070280 con 0x7ffa54160d40 2023-10-11T09:41:32.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.521+0000 7ffa58f14700 1 --2- 172.21.15.78:0/414412880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa54160d40 0x7ffa540cd280 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ffa50009880 tx=0x7ffa50009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e883e3f0b3cddd4 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:32.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.525+0000 7ffa4b7fe700 1 -- 172.21.15.78:0/414412880 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa50011830 con 0x7ffa54160d40 2023-10-11T09:41:32.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.525+0000 7ffa4b7fe700 1 -- 172.21.15.78:0/414412880 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa50011990 con 0x7ffa54160d40 2023-10-11T09:41:32.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.525+0000 7ffa4b7fe700 1 -- 172.21.15.78:0/414412880 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa50011d20 con 0x7ffa54160d40 2023-10-11T09:41:32.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.525+0000 7ffa5b178700 1 -- 172.21.15.78:0/414412880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa54160d40 msgr2=0x7ffa540cd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:32.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.525+0000 7ffa5b178700 1 --2- 172.21.15.78:0/414412880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa54160d40 0x7ffa540cd280 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ffa50009880 tx=0x7ffa50009b90 comp rx=0 tx=0).stop 2023-10-11T09:41:32.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.525+0000 7ffa5b178700 1 -- 172.21.15.78:0/414412880 shutdown_connections 2023-10-11T09:41:32.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.525+0000 7ffa5b178700 1 --2- 172.21.15.78:0/414412880 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffa540ce500 0x7ffa5406be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:32.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.525+0000 7ffa5b178700 1 --2- 172.21.15.78:0/414412880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa54160d40 0x7ffa540cd280 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:32.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.525+0000 7ffa5b178700 1 -- 172.21.15.78:0/414412880 >> 172.21.15.78:0/414412880 conn(0x7ffa5406aca0 msgr2=0x7ffa540b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:32.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.525+0000 7ffa5b178700 1 -- 172.21.15.78:0/414412880 shutdown_connections 2023-10-11T09:41:32.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.525+0000 7ffa5b178700 1 -- 172.21.15.78:0/414412880 wait complete. 2023-10-11T09:41:32.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.529+0000 7ffa5b178700 1 Processor -- start 2023-10-11T09:41:32.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.529+0000 7ffa5b178700 1 -- start start 2023-10-11T09:41:32.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.529+0000 7ffa5b178700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffa54071040 0x7ffa541587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:32.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.529+0000 7ffa5b178700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa540ce500 0x7ffa54154780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:32.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.529+0000 7ffa5b178700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa54160d40 0x7ffa540cd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:32.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.529+0000 7ffa5b178700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa540cba30 con 0x7ffa54160d40 2023-10-11T09:41:32.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.529+0000 7ffa5b178700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa541fa230 con 0x7ffa540ce500 2023-10-11T09:41:32.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.529+0000 7ffa5b178700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa541fa510 con 0x7ffa54071040 2023-10-11T09:41:32.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.529+0000 7ffa59715700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa54160d40 0x7ffa540cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:32.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.529+0000 7ffa59715700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa54160d40 0x7ffa540cd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59630/0 (socket says 172.21.15.78:59630) 2023-10-11T09:41:32.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.529+0000 7ffa59715700 1 -- 172.21.15.78:0/2963724597 learned_addr learned my addr 172.21.15.78:0/2963724597 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:32.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.529+0000 7ffa4bfff700 1 --2- 172.21.15.78:0/2963724597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa540ce500 0x7ffa54154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:32.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.529+0000 7ffa58f14700 1 --2- 172.21.15.78:0/2963724597 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffa54071040 0x7ffa541587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:32.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.529+0000 7ffa59715700 1 -- 172.21.15.78:0/2963724597 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffa54071040 msgr2=0x7ffa541587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:32.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.529+0000 7ffa59715700 1 --2- 172.21.15.78:0/2963724597 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffa54071040 0x7ffa541587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:32.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.529+0000 7ffa59715700 1 -- 172.21.15.78:0/2963724597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa540ce500 msgr2=0x7ffa54154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:32.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.529+0000 7ffa59715700 1 --2- 172.21.15.78:0/2963724597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa540ce500 0x7ffa54154780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:32.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.529+0000 7ffa59715700 1 -- 172.21.15.78:0/2963724597 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa500094b0 con 0x7ffa54160d40 2023-10-11T09:41:32.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.529+0000 7ffa58f14700 1 --2- 172.21.15.78:0/2963724597 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffa54071040 0x7ffa541587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:32.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.529+0000 7ffa4bfff700 1 --2- 172.21.15.78:0/2963724597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa540ce500 0x7ffa54154780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:32.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.533+0000 7ffa59715700 1 --2- 172.21.15.78:0/2963724597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa54160d40 0x7ffa540cd280 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c00ef90 tx=0x7ffa4c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:32.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.533+0000 7ffa497fa700 1 -- 172.21.15.78:0/2963724597 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c009ca0 con 0x7ffa54160d40 2023-10-11T09:41:32.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.533+0000 7ffa5b178700 1 -- 172.21.15.78:0/2963724597 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa541fa7f0 con 0x7ffa54160d40 2023-10-11T09:41:32.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.533+0000 7ffa497fa700 1 -- 172.21.15.78:0/2963724597 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa4c00f040 con 0x7ffa54160d40 2023-10-11T09:41:32.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.533+0000 7ffa497fa700 1 -- 172.21.15.78:0/2963724597 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c014480 con 0x7ffa54160d40 2023-10-11T09:41:32.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.533+0000 7ffa5b178700 1 -- 172.21.15.78:0/2963724597 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa541faff0 con 0x7ffa54160d40 2023-10-11T09:41:32.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.533+0000 7ffa5b178700 1 -- 172.21.15.78:0/2963724597 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa54072250 con 0x7ffa54160d40 2023-10-11T09:41:32.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.537+0000 7ffa497fa700 1 -- 172.21.15.78:0/2963724597 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7ffa4c0145e0 con 0x7ffa54160d40 2023-10-11T09:41:32.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.537+0000 7ffa497fa700 1 --2- 172.21.15.78:0/2963724597 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffa40065ed0 0x7ffa40068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:32.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.537+0000 7ffa497fa700 1 -- 172.21.15.78:0/2963724597 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ffa4c084c30 con 0x7ffa54160d40 2023-10-11T09:41:32.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.537+0000 7ffa58f14700 1 --2- 172.21.15.78:0/2963724597 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffa40065ed0 0x7ffa40068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:32.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.537+0000 7ffa58f14700 1 --2- 172.21.15.78:0/2963724597 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffa40065ed0 0x7ffa40068380 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ffa50000c00 tx=0x7ffa50008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:32.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.545+0000 7ffa497fa700 1 -- 172.21.15.78:0/2963724597 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7ffa4c0518a0 con 0x7ffa54160d40 2023-10-11T09:41:32.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.713+0000 7ffa5b178700 1 -- 172.21.15.78:0/2963724597 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa541fad10 con 0x7ffa54160d40 2023-10-11T09:41:32.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.717+0000 7ffa497fa700 1 -- 172.21.15.78:0/2963724597 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17907 (secure 0 0 0) 0x7ffa541fad10 con 0x7ffa54160d40 2023-10-11T09:41:32.722 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:41:32.722 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":25,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:41:21.473566+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:40:39.862581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T09:40:40.663641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.37","mappings":[{"from":3,"to":2}]},{"pgid":"4.3e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:41:32.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.717+0000 7ffa5b178700 1 -- 172.21.15.78:0/2963724597 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffa40065ed0 msgr2=0x7ffa40068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:32.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.717+0000 7ffa5b178700 1 --2- 172.21.15.78:0/2963724597 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffa40065ed0 0x7ffa40068380 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ffa50000c00 tx=0x7ffa50008030 comp rx=0 tx=0).stop 2023-10-11T09:41:32.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.717+0000 7ffa5b178700 1 -- 172.21.15.78:0/2963724597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa54160d40 msgr2=0x7ffa540cd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:32.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.717+0000 7ffa5b178700 1 --2- 172.21.15.78:0/2963724597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa54160d40 0x7ffa540cd280 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c00ef90 tx=0x7ffa4c00c3a0 comp rx=0 tx=0).stop 2023-10-11T09:41:32.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.717+0000 7ffa5b178700 1 -- 172.21.15.78:0/2963724597 shutdown_connections 2023-10-11T09:41:32.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.717+0000 7ffa5b178700 1 --2- 172.21.15.78:0/2963724597 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffa40065ed0 0x7ffa40068380 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:32.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.717+0000 7ffa5b178700 1 --2- 172.21.15.78:0/2963724597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa54160d40 0x7ffa540cd280 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:32.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.717+0000 7ffa5b178700 1 --2- 172.21.15.78:0/2963724597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa540ce500 0x7ffa54154780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:32.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.717+0000 7ffa5b178700 1 --2- 172.21.15.78:0/2963724597 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffa54071040 0x7ffa541587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:32.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.717+0000 7ffa5b178700 1 -- 172.21.15.78:0/2963724597 >> 172.21.15.78:0/2963724597 conn(0x7ffa5406aca0 msgr2=0x7ffa540c9350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:32.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.721+0000 7ffa5b178700 1 -- 172.21.15.78:0/2963724597 shutdown_connections 2023-10-11T09:41:32.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.721+0000 7ffa5b178700 1 -- 172.21.15.78:0/2963724597 wait complete. 2023-10-11T09:41:32.747 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T09:40:39.862581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T09:40:40.663641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:41:32.748 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:41:32.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.913+0000 7fddc92ee700 1 Processor -- start 2023-10-11T09:41:32.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.913+0000 7fddc92ee700 1 -- start start 2023-10-11T09:41:32.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.917+0000 7fddc92ee700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fddc4146fe0 0x7fddc406bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:32.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.917+0000 7fddc92ee700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fddc40b36f0 0x7fddc41629e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:32.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.917+0000 7fddc92ee700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddc406ccb0 con 0x7fddc4161d90 2023-10-11T09:41:32.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.917+0000 7fddc92ee700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddc406c410 con 0x7fddc4146fe0 2023-10-11T09:41:32.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.917+0000 7fddc92ee700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc406c6f0 con 0x7fddc40b36f0 2023-10-11T09:41:32.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.917+0000 7fddc37fe700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fddc40b36f0 0x7fddc41629e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:32.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.917+0000 7fddc27fc700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fddc4161d90 0x7fddc40b2fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:57584/0 (socket says 172.21.15.78:57584) 2023-10-11T09:41:32.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.917+0000 7fddc27fc700 1 -- 172.21.15.78:0/3220794624 learned_addr learned my addr 172.21.15.78:0/3220794624 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:32.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.917+0000 7fddc2ffd700 1 --2- 172.21.15.78:0/3220794624 >> v2:172.21.15.133:3300/0 conn(0x7fddc4146fe0 0x7fddc406bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:32.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.917+0000 7fddc1ffb700 1 -- 172.21.15.78:0/3220794624 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 964987999 0 0) 0x7fddb8001110 con 0x7fddc4161d90 2023-10-11T09:41:32.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.917+0000 7fddc1ffb700 1 -- 172.21.15.78:0/3220794624 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddac0035d0 con 0x7fddc4161d90 2023-10-11T09:41:32.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.917+0000 7fddc37fe700 1 -- 172.21.15.78:0/3220794624 >> v2:172.21.15.133:3300/0 conn(0x7fddc4146fe0 msgr2=0x7fddc406bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:32.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.917+0000 7fddc37fe700 1 --2- 172.21.15.78:0/3220794624 >> v2:172.21.15.133:3300/0 conn(0x7fddc4146fe0 0x7fddc406bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:32.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.917+0000 7fddc37fe700 1 -- 172.21.15.78:0/3220794624 >> v1:172.21.15.78:6789/0 conn(0x7fddc4161d90 legacy=0x7fddc40b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:32.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.917+0000 7fddc37fe700 1 -- 172.21.15.78:0/3220794624 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc406c9d0 con 0x7fddc40b36f0 2023-10-11T09:41:32.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.917+0000 7fddc2ffd700 1 --2- 172.21.15.78:0/3220794624 >> v2:172.21.15.133:3300/0 conn(0x7fddc4146fe0 0x7fddc406bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:41:32.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.917+0000 7fddc37fe700 1 --2- 172.21.15.78:0/3220794624 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fddc40b36f0 0x7fddc41629e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fddbc009d90 tx=0x7fddbc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef44275c09db3c28 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:32.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.917+0000 7fddc1ffb700 1 -- 172.21.15.78:0/3220794624 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddbc00e070 con 0x7fddc40b36f0 2023-10-11T09:41:32.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.917+0000 7fddc1ffb700 1 -- 172.21.15.78:0/3220794624 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddbc00bb20 con 0x7fddc40b36f0 2023-10-11T09:41:32.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.917+0000 7fddc1ffb700 1 -- 172.21.15.78:0/3220794624 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddbc012430 con 0x7fddc40b36f0 2023-10-11T09:41:32.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.921+0000 7fddc92ee700 1 -- 172.21.15.78:0/3220794624 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fddc40b36f0 msgr2=0x7fddc41629e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:32.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.921+0000 7fddc92ee700 1 --2- 172.21.15.78:0/3220794624 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fddc40b36f0 0x7fddc41629e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fddbc009d90 tx=0x7fddbc00b330 comp rx=0 tx=0).stop 2023-10-11T09:41:32.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.921+0000 7fddc92ee700 1 -- 172.21.15.78:0/3220794624 shutdown_connections 2023-10-11T09:41:32.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.921+0000 7fddc92ee700 1 --2- 172.21.15.78:0/3220794624 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fddc40b36f0 0x7fddc41629e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:32.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.921+0000 7fddc92ee700 1 --2- 172.21.15.78:0/3220794624 >> v2:172.21.15.133:3300/0 conn(0x7fddc4146fe0 0x7fddc406bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:32.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.921+0000 7fddc92ee700 1 -- 172.21.15.78:0/3220794624 >> 172.21.15.78:0/3220794624 conn(0x7fddc406a060 msgr2=0x7fddc406a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:32.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.921+0000 7fddc92ee700 1 -- 172.21.15.78:0/3220794624 shutdown_connections 2023-10-11T09:41:32.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.921+0000 7fddc92ee700 1 -- 172.21.15.78:0/3220794624 wait complete. 2023-10-11T09:41:32.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.921+0000 7fddc92ee700 1 Processor -- start 2023-10-11T09:41:32.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.921+0000 7fddc92ee700 1 -- start start 2023-10-11T09:41:32.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.925+0000 7fddc92ee700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fddc40b36f0 0x7fddc41570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:32.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.925+0000 7fddc92ee700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fddc4146fe0 0x7fddc4137040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:32.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.925+0000 7fddc92ee700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fddc4161d90 0x7fddc4137580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:32.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.925+0000 7fddc92ee700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc41575e0 con 0x7fddc4161d90 2023-10-11T09:41:32.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.925+0000 7fddc92ee700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc4135fa0 con 0x7fddc40b36f0 2023-10-11T09:41:32.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.925+0000 7fddc92ee700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc4136280 con 0x7fddc4146fe0 2023-10-11T09:41:32.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.925+0000 7fddc37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fddc4161d90 0x7fddc4137580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:32.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.925+0000 7fddc37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fddc4161d90 0x7fddc4137580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59640/0 (socket says 172.21.15.78:59640) 2023-10-11T09:41:32.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.925+0000 7fddc37fe700 1 -- 172.21.15.78:0/1018485720 learned_addr learned my addr 172.21.15.78:0/1018485720 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:32.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.925+0000 7fddc27fc700 1 --2- 172.21.15.78:0/1018485720 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fddc4146fe0 0x7fddc4137040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:32.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.925+0000 7fddc2ffd700 1 --2- 172.21.15.78:0/1018485720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fddc40b36f0 0x7fddc41570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:32.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.925+0000 7fddc37fe700 1 -- 172.21.15.78:0/1018485720 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fddc4146fe0 msgr2=0x7fddc4137040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.925+0000 7fddc37fe700 1 --2- 172.21.15.78:0/1018485720 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fddc4146fe0 0x7fddc4137040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.925+0000 7fddc37fe700 1 -- 172.21.15.78:0/1018485720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fddc40b36f0 msgr2=0x7fddc41570a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.925+0000 7fddc37fe700 1 --2- 172.21.15.78:0/1018485720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fddc40b36f0 0x7fddc41570a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.925+0000 7fddc37fe700 1 -- 172.21.15.78:0/1018485720 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb40094b0 con 0x7fddc4161d90 2023-10-11T09:41:32.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.925+0000 7fddc27fc700 1 --2- 172.21.15.78:0/1018485720 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fddc4146fe0 0x7fddc4137040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:32.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.925+0000 7fddc37fe700 1 --2- 172.21.15.78:0/1018485720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fddc4161d90 0x7fddc4137580 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fddbc014040 tx=0x7fddbc004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:32.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.925+0000 7fddab7fe700 1 -- 172.21.15.78:0/1018485720 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddbc012430 con 0x7fddc4161d90 2023-10-11T09:41:32.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.925+0000 7fddab7fe700 1 -- 172.21.15.78:0/1018485720 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddbc020070 con 0x7fddc4161d90 2023-10-11T09:41:32.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.925+0000 7fddc92ee700 1 -- 172.21.15.78:0/1018485720 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddbc009a70 con 0x7fddc4161d90 2023-10-11T09:41:32.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.925+0000 7fddab7fe700 1 -- 172.21.15.78:0/1018485720 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddbc004280 con 0x7fddc4161d90 2023-10-11T09:41:32.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.925+0000 7fddc92ee700 1 -- 172.21.15.78:0/1018485720 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc4136840 con 0x7fddc4161d90 2023-10-11T09:41:32.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.929+0000 7fddc92ee700 1 -- 172.21.15.78:0/1018485720 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc4065d40 con 0x7fddc4161d90 2023-10-11T09:41:32.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.929+0000 7fddab7fe700 1 -- 172.21.15.78:0/1018485720 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fddbc0043e0 con 0x7fddc4161d90 2023-10-11T09:41:32.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.929+0000 7fddab7fe700 1 --2- 172.21.15.78:0/1018485720 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fddb0065ed0 0x7fddb0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:32.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.933+0000 7fddab7fe700 1 -- 172.21.15.78:0/1018485720 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fddbc022070 con 0x7fddc4161d90 2023-10-11T09:41:32.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.933+0000 7fddc2ffd700 1 --2- 172.21.15.78:0/1018485720 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fddb0065ed0 0x7fddb0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:32.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.933+0000 7fddc2ffd700 1 --2- 172.21.15.78:0/1018485720 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fddb0065ed0 0x7fddb0068380 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fddc40cf860 tx=0x7fddb4009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:32.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:32.937+0000 7fddab7fe700 1 -- 172.21.15.78:0/1018485720 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fddbc052a50 con 0x7fddc4161d90 2023-10-11T09:41:33.114 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.109+0000 7fddc92ee700 1 -- 172.21.15.78:0/1018485720 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fddc40717d0 con 0x7fddc4161d90 2023-10-11T09:41:33.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.109+0000 7fddab7fe700 1 -- 172.21.15.78:0/1018485720 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fddbc02c020 con 0x7fddc4161d90 2023-10-11T09:41:33.115 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:41:33.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.113+0000 7fddc92ee700 1 -- 172.21.15.78:0/1018485720 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fddb0065ed0 msgr2=0x7fddb0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:33.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.113+0000 7fddc92ee700 1 --2- 172.21.15.78:0/1018485720 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fddb0065ed0 0x7fddb0068380 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fddc40cf860 tx=0x7fddb4009240 comp rx=0 tx=0).stop 2023-10-11T09:41:33.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.113+0000 7fddc92ee700 1 -- 172.21.15.78:0/1018485720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fddc4161d90 msgr2=0x7fddc4137580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:33.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.113+0000 7fddc92ee700 1 --2- 172.21.15.78:0/1018485720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fddc4161d90 0x7fddc4137580 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fddbc014040 tx=0x7fddbc004af0 comp rx=0 tx=0).stop 2023-10-11T09:41:33.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.113+0000 7fddc92ee700 1 -- 172.21.15.78:0/1018485720 shutdown_connections 2023-10-11T09:41:33.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.113+0000 7fddc92ee700 1 --2- 172.21.15.78:0/1018485720 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fddb0065ed0 0x7fddb0068380 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:33.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.113+0000 7fddc92ee700 1 --2- 172.21.15.78:0/1018485720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fddc4161d90 0x7fddc4137580 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:33.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.113+0000 7fddc92ee700 1 --2- 172.21.15.78:0/1018485720 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fddc4146fe0 0x7fddc4137040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:33.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.113+0000 7fddc92ee700 1 --2- 172.21.15.78:0/1018485720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fddc40b36f0 0x7fddc41570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:33.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.113+0000 7fddc92ee700 1 -- 172.21.15.78:0/1018485720 >> 172.21.15.78:0/1018485720 conn(0x7fddc406a060 msgr2=0x7fddc40ae620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:33.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.113+0000 7fddc92ee700 1 -- 172.21.15.78:0/1018485720 shutdown_connections 2023-10-11T09:41:33.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.113+0000 7fddc92ee700 1 -- 172.21.15.78:0/1018485720 wait complete. 2023-10-11T09:41:33.143 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:41:33.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.309+0000 7ff29ca7e700 1 Processor -- start 2023-10-11T09:41:33.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.309+0000 7ff29ca7e700 1 -- start start 2023-10-11T09:41:33.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.309+0000 7ff29ca7e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff298161d90 0x7ff2980b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:33.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.309+0000 7ff29ca7e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff29806ccb0 con 0x7ff2980b3520 2023-10-11T09:41:33.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.309+0000 7ff29ca7e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff29806c410 con 0x7ff298146fe0 2023-10-11T09:41:33.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.309+0000 7ff29ca7e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff29806c6f0 con 0x7ff298161d90 2023-10-11T09:41:33.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.313+0000 7ff2977fe700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff298146fe0 0x7ff29806bbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:44684/0 (socket says 172.21.15.78:44684) 2023-10-11T09:41:33.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.313+0000 7ff2977fe700 1 -- 172.21.15.78:0/3801187003 learned_addr learned my addr 172.21.15.78:0/3801187003 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:33.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.313+0000 7ff296ffd700 1 --2- 172.21.15.78:0/3801187003 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff298161d90 0x7ff2980b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:33.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.313+0000 7ff2967fc700 1 -- 172.21.15.78:0/3801187003 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4238036984 0 0) 0x7ff29806c410 con 0x7ff298146fe0 2023-10-11T09:41:33.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.313+0000 7ff2967fc700 1 -- 172.21.15.78:0/3801187003 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2800035d0 con 0x7ff298146fe0 2023-10-11T09:41:33.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.313+0000 7ff296ffd700 1 -- 172.21.15.78:0/3801187003 >> v1:172.21.15.133:6789/0 conn(0x7ff298146fe0 legacy=0x7ff29806bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:33.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.313+0000 7ff2967fc700 1 -- 172.21.15.78:0/3801187003 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4024671451 0 0) 0x7ff290001490 con 0x7ff2980b3520 2023-10-11T09:41:33.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.313+0000 7ff296ffd700 1 -- 172.21.15.78:0/3801187003 >> v1:172.21.15.78:6789/0 conn(0x7ff2980b3520 legacy=0x7ff2981629e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:33.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.313+0000 7ff296ffd700 1 -- 172.21.15.78:0/3801187003 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29806c9d0 con 0x7ff298161d90 2023-10-11T09:41:33.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.313+0000 7ff296ffd700 1 --2- 172.21.15.78:0/3801187003 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff298161d90 0x7ff2980b2fe0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff28c009880 tx=0x7ff28c009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16666eabe43e5a59 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:33.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.313+0000 7ff2967fc700 1 -- 172.21.15.78:0/3801187003 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c011830 con 0x7ff298161d90 2023-10-11T09:41:33.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.313+0000 7ff2967fc700 1 -- 172.21.15.78:0/3801187003 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28c011990 con 0x7ff298161d90 2023-10-11T09:41:33.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.313+0000 7ff2967fc700 1 -- 172.21.15.78:0/3801187003 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c011d20 con 0x7ff298161d90 2023-10-11T09:41:33.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.313+0000 7ff29ca7e700 1 -- 172.21.15.78:0/3801187003 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff298161d90 msgr2=0x7ff2980b2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:33.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.313+0000 7ff29ca7e700 1 --2- 172.21.15.78:0/3801187003 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff298161d90 0x7ff2980b2fe0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff28c009880 tx=0x7ff28c009b90 comp rx=0 tx=0).stop 2023-10-11T09:41:33.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.313+0000 7ff29ca7e700 1 -- 172.21.15.78:0/3801187003 shutdown_connections 2023-10-11T09:41:33.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.313+0000 7ff29ca7e700 1 --2- 172.21.15.78:0/3801187003 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff298161d90 0x7ff2980b2fe0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:33.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.313+0000 7ff29ca7e700 1 -- 172.21.15.78:0/3801187003 >> 172.21.15.78:0/3801187003 conn(0x7ff29806a060 msgr2=0x7ff29806a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:33.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.313+0000 7ff29ca7e700 1 -- 172.21.15.78:0/3801187003 shutdown_connections 2023-10-11T09:41:33.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.313+0000 7ff29ca7e700 1 -- 172.21.15.78:0/3801187003 wait complete. 2023-10-11T09:41:33.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.317+0000 7ff29ca7e700 1 Processor -- start 2023-10-11T09:41:33.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.317+0000 7ff29ca7e700 1 -- start start 2023-10-11T09:41:33.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.321+0000 7ff29ca7e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2980b3520 0x7ff298145eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:33.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.321+0000 7ff29ca7e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff298146fe0 0x7ff2980aaae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:33.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.321+0000 7ff29ca7e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff298161d90 0x7ff2980ab020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:33.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.321+0000 7ff29ca7e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29815f820 con 0x7ff2980b3520 2023-10-11T09:41:33.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.321+0000 7ff29ca7e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2980ce130 con 0x7ff298161d90 2023-10-11T09:41:33.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.321+0000 7ff29ca7e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2980ce410 con 0x7ff298146fe0 2023-10-11T09:41:33.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.321+0000 7ff2977fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2980b3520 0x7ff298145eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:33.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.321+0000 7ff2977fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2980b3520 0x7ff298145eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59650/0 (socket says 172.21.15.78:59650) 2023-10-11T09:41:33.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.321+0000 7ff2977fe700 1 -- 172.21.15.78:0/3894953779 learned_addr learned my addr 172.21.15.78:0/3894953779 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:33.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.321+0000 7ff296ffd700 1 --2- 172.21.15.78:0/3894953779 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff298146fe0 0x7ff2980aaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:33.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.321+0000 7ff297fff700 1 --2- 172.21.15.78:0/3894953779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff298161d90 0x7ff2980ab020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:33.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.321+0000 7ff2977fe700 1 -- 172.21.15.78:0/3894953779 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff298146fe0 msgr2=0x7ff2980aaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:33.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.321+0000 7ff2977fe700 1 --2- 172.21.15.78:0/3894953779 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff298146fe0 0x7ff2980aaae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:33.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.321+0000 7ff2977fe700 1 -- 172.21.15.78:0/3894953779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff298161d90 msgr2=0x7ff2980ab020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:33.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.321+0000 7ff2977fe700 1 --2- 172.21.15.78:0/3894953779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff298161d90 0x7ff2980ab020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:33.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.321+0000 7ff2977fe700 1 -- 172.21.15.78:0/3894953779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff28c0094b0 con 0x7ff2980b3520 2023-10-11T09:41:33.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.321+0000 7ff2977fe700 1 --2- 172.21.15.78:0/3894953779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2980b3520 0x7ff298145eb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff28800c700 tx=0x7ff28800cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:33.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.321+0000 7ff27ffff700 1 -- 172.21.15.78:0/3894953779 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff288005d90 con 0x7ff2980b3520 2023-10-11T09:41:33.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.321+0000 7ff27ffff700 1 -- 172.21.15.78:0/3894953779 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2880070d0 con 0x7ff2980b3520 2023-10-11T09:41:33.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.321+0000 7ff29ca7e700 1 -- 172.21.15.78:0/3894953779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2980ce6f0 con 0x7ff2980b3520 2023-10-11T09:41:33.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.321+0000 7ff27ffff700 1 -- 172.21.15.78:0/3894953779 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff288007480 con 0x7ff2980b3520 2023-10-11T09:41:33.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.321+0000 7ff29ca7e700 1 -- 172.21.15.78:0/3894953779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2980cecb0 con 0x7ff2980b3520 2023-10-11T09:41:33.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.321+0000 7ff29ca7e700 1 -- 172.21.15.78:0/3894953779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff298065d40 con 0x7ff2980b3520 2023-10-11T09:41:33.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.325+0000 7ff27ffff700 1 -- 172.21.15.78:0/3894953779 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7ff2880075e0 con 0x7ff2980b3520 2023-10-11T09:41:33.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.325+0000 7ff27ffff700 1 --2- 172.21.15.78:0/3894953779 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff284065ed0 0x7ff284068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:33.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.325+0000 7ff27ffff700 1 -- 172.21.15.78:0/3894953779 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff288082e40 con 0x7ff2980b3520 2023-10-11T09:41:33.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.325+0000 7ff296ffd700 1 --2- 172.21.15.78:0/3894953779 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff284065ed0 0x7ff284068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:33.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.329+0000 7ff296ffd700 1 --2- 172.21.15.78:0/3894953779 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff284065ed0 0x7ff284068380 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff28c0090b0 tx=0x7ff28c01c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:33.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.333+0000 7ff27ffff700 1 -- 172.21.15.78:0/3894953779 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7ff288050460 con 0x7ff2980b3520 2023-10-11T09:41:33.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.501+0000 7ff29ca7e700 1 -- 172.21.15.78:0/3894953779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff29806ce50 con 0x7ff2980b3520 2023-10-11T09:41:33.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.501+0000 7ff27ffff700 1 -- 172.21.15.78:0/3894953779 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff288053a80 con 0x7ff2980b3520 2023-10-11T09:41:33.508 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:41:33.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.505+0000 7ff29ca7e700 1 -- 172.21.15.78:0/3894953779 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff284065ed0 msgr2=0x7ff284068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:33.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.505+0000 7ff29ca7e700 1 --2- 172.21.15.78:0/3894953779 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff284065ed0 0x7ff284068380 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff28c0090b0 tx=0x7ff28c01c040 comp rx=0 tx=0).stop 2023-10-11T09:41:33.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.505+0000 7ff29ca7e700 1 -- 172.21.15.78:0/3894953779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2980b3520 msgr2=0x7ff298145eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:33.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.505+0000 7ff29ca7e700 1 --2- 172.21.15.78:0/3894953779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2980b3520 0x7ff298145eb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff28800c700 tx=0x7ff28800cac0 comp rx=0 tx=0).stop 2023-10-11T09:41:33.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.505+0000 7ff29ca7e700 1 -- 172.21.15.78:0/3894953779 shutdown_connections 2023-10-11T09:41:33.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.505+0000 7ff29ca7e700 1 --2- 172.21.15.78:0/3894953779 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff284065ed0 0x7ff284068380 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:33.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.505+0000 7ff29ca7e700 1 --2- 172.21.15.78:0/3894953779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff298161d90 0x7ff2980ab020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:33.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.505+0000 7ff29ca7e700 1 --2- 172.21.15.78:0/3894953779 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff298146fe0 0x7ff2980aaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:33.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.505+0000 7ff29ca7e700 1 --2- 172.21.15.78:0/3894953779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2980b3520 0x7ff298145eb0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:33.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.505+0000 7ff29ca7e700 1 -- 172.21.15.78:0/3894953779 >> 172.21.15.78:0/3894953779 conn(0x7ff29806a060 msgr2=0x7ff2981628e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:33.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.505+0000 7ff29ca7e700 1 -- 172.21.15.78:0/3894953779 shutdown_connections 2023-10-11T09:41:33.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.505+0000 7ff29ca7e700 1 -- 172.21.15.78:0/3894953779 wait complete. 2023-10-11T09:41:33.533 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T09:41:33.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.697+0000 7f36989b9700 1 Processor -- start 2023-10-11T09:41:33.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.697+0000 7f36989b9700 1 -- start start 2023-10-11T09:41:33.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.701+0000 7f36989b9700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f369415a6b0 0x7f36940ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:33.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.701+0000 7f36989b9700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36940b3a60 0x7f369406d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:33.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.701+0000 7f36989b9700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f36940cede0 0x7f369406d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:33.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.701+0000 7f36989b9700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36940b2d30 con 0x7f369415a6b0 2023-10-11T09:41:33.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.701+0000 7f36989b9700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36940cf360 con 0x7f36940cede0 2023-10-11T09:41:33.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.701+0000 7f36989b9700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36940b2f80 con 0x7f36940b3a60 2023-10-11T09:41:33.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.701+0000 7f3692d9d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f369415a6b0 0x7f36940ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:33.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.701+0000 7f3692d9d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f369415a6b0 0x7f36940ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59654/0 (socket says 172.21.15.78:59654) 2023-10-11T09:41:33.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.701+0000 7f3692d9d700 1 -- 172.21.15.78:0/801349586 learned_addr learned my addr 172.21.15.78:0/801349586 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:33.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.701+0000 7f369359e700 1 --2- 172.21.15.78:0/801349586 >> v2:172.21.15.133:3300/0 conn(0x7f36940cede0 0x7f369406d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:33.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.701+0000 7f368a59c700 1 --2- 172.21.15.78:0/801349586 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36940b3a60 0x7f369406d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:33.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.701+0000 7f3692d9d700 1 -- 172.21.15.78:0/801349586 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36940b3a60 msgr2=0x7f369406d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:33.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.701+0000 7f3692d9d700 1 --2- 172.21.15.78:0/801349586 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36940b3a60 0x7f369406d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:33.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.701+0000 7f3692d9d700 1 -- 172.21.15.78:0/801349586 >> v2:172.21.15.133:3300/0 conn(0x7f36940cede0 msgr2=0x7f369406d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:33.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.701+0000 7f3692d9d700 1 --2- 172.21.15.78:0/801349586 >> v2:172.21.15.133:3300/0 conn(0x7f36940cede0 0x7f369406d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:33.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.701+0000 7f3692d9d700 1 -- 172.21.15.78:0/801349586 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36940b3260 con 0x7f369415a6b0 2023-10-11T09:41:33.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.705+0000 7f3692d9d700 1 --2- 172.21.15.78:0/801349586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f369415a6b0 0x7f36940ce8a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f36940b2ab0 tx=0x7f3684009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=720b589e45a2768b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:33.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.705+0000 7f369259c700 1 -- 172.21.15.78:0/801349586 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3684011830 con 0x7f369415a6b0 2023-10-11T09:41:33.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.705+0000 7f369259c700 1 -- 172.21.15.78:0/801349586 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3684011990 con 0x7f369415a6b0 2023-10-11T09:41:33.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.705+0000 7f369259c700 1 -- 172.21.15.78:0/801349586 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3684011d20 con 0x7f369415a6b0 2023-10-11T09:41:33.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.705+0000 7f36989b9700 1 -- 172.21.15.78:0/801349586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f369415a6b0 msgr2=0x7f36940ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:33.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.705+0000 7f36989b9700 1 --2- 172.21.15.78:0/801349586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f369415a6b0 0x7f36940ce8a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f36940b2ab0 tx=0x7f3684009bf0 comp rx=0 tx=0).stop 2023-10-11T09:41:33.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.705+0000 7f36989b9700 1 -- 172.21.15.78:0/801349586 shutdown_connections 2023-10-11T09:41:33.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.705+0000 7f36989b9700 1 --2- 172.21.15.78:0/801349586 >> v2:172.21.15.133:3300/0 conn(0x7f36940cede0 0x7f369406d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:33.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.705+0000 7f36989b9700 1 --2- 172.21.15.78:0/801349586 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36940b3a60 0x7f369406d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:33.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.705+0000 7f36989b9700 1 --2- 172.21.15.78:0/801349586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f369415a6b0 0x7f36940ce8a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:33.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.705+0000 7f36989b9700 1 -- 172.21.15.78:0/801349586 >> 172.21.15.78:0/801349586 conn(0x7f369406a060 msgr2=0x7f369406a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:33.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.705+0000 7f36989b9700 1 -- 172.21.15.78:0/801349586 shutdown_connections 2023-10-11T09:41:33.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.705+0000 7f36989b9700 1 -- 172.21.15.78:0/801349586 wait complete. 2023-10-11T09:41:33.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.709+0000 7f36989b9700 1 Processor -- start 2023-10-11T09:41:33.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.709+0000 7f36989b9700 1 -- start start 2023-10-11T09:41:33.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.709+0000 7f36989b9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36940b3a60 0x7f3694158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:33.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.709+0000 7f36989b9700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36940cede0 0x7f3694155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:33.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.709+0000 7f36989b9700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f369415a6b0 0x7f369413d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:33.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.709+0000 7f36989b9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3694159410 con 0x7f36940b3a60 2023-10-11T09:41:33.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.709+0000 7f36989b9700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3694158b70 con 0x7f369415a6b0 2023-10-11T09:41:33.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.709+0000 7f36989b9700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3694158e50 con 0x7f36940cede0 2023-10-11T09:41:33.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.709+0000 7f3692d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36940b3a60 0x7f3694158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:33.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.709+0000 7f3692d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36940b3a60 0x7f3694158120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59658/0 (socket says 172.21.15.78:59658) 2023-10-11T09:41:33.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.709+0000 7f3692d9d700 1 -- 172.21.15.78:0/3598890955 learned_addr learned my addr 172.21.15.78:0/3598890955 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:33.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.709+0000 7f3692d9d700 1 -- 172.21.15.78:0/3598890955 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36940cede0 msgr2=0x7f3694155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:33.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.709+0000 7f3692d9d700 1 --2- 172.21.15.78:0/3598890955 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36940cede0 0x7f3694155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:33.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.709+0000 7f3692d9d700 1 -- 172.21.15.78:0/3598890955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f369415a6b0 msgr2=0x7f369413d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:33.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.709+0000 7f3692d9d700 1 --2- 172.21.15.78:0/3598890955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f369415a6b0 0x7f369413d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:33.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.709+0000 7f3692d9d700 1 -- 172.21.15.78:0/3598890955 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3684009570 con 0x7f36940b3a60 2023-10-11T09:41:33.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.709+0000 7f3692d9d700 1 --2- 172.21.15.78:0/3598890955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36940b3a60 0x7f3694158120 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3684000c00 tx=0x7f3684004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:33.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.713+0000 7f368bfff700 1 -- 172.21.15.78:0/3598890955 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3684004590 con 0x7f36940b3a60 2023-10-11T09:41:33.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.713+0000 7f368bfff700 1 -- 172.21.15.78:0/3598890955 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f368401e070 con 0x7f36940b3a60 2023-10-11T09:41:33.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.713+0000 7f368bfff700 1 -- 172.21.15.78:0/3598890955 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3684003900 con 0x7f36940b3a60 2023-10-11T09:41:33.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.713+0000 7f36989b9700 1 -- 172.21.15.78:0/3598890955 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3694159130 con 0x7f36940b3a60 2023-10-11T09:41:33.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.713+0000 7f36989b9700 1 -- 172.21.15.78:0/3598890955 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36941418b0 con 0x7f36940b3a60 2023-10-11T09:41:33.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.713+0000 7f36989b9700 1 -- 172.21.15.78:0/3598890955 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f369413b9c0 con 0x7f36940b3a60 2023-10-11T09:41:33.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.713+0000 7f368bfff700 1 -- 172.21.15.78:0/3598890955 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f3684003a60 con 0x7f36940b3a60 2023-10-11T09:41:33.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.717+0000 7f368bfff700 1 --2- 172.21.15.78:0/3598890955 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3674065ed0 0x7f3674068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:33.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.717+0000 7f368bfff700 1 -- 172.21.15.78:0/3598890955 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f3684084240 con 0x7f36940b3a60 2023-10-11T09:41:33.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.717+0000 7f368a59c700 1 --2- 172.21.15.78:0/3598890955 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3674065ed0 0x7f3674068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:33.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.717+0000 7f368a59c700 1 --2- 172.21.15.78:0/3598890955 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3674065ed0 0x7f3674068380 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f367c006b40 tx=0x7f367c006ad0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:33.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.721+0000 7f368bfff700 1 -- 172.21.15.78:0/3598890955 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f369413b9c0 con 0x7f36940b3a60 2023-10-11T09:41:33.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.897+0000 7f36989b9700 1 -- 172.21.15.78:0/3598890955 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f369413b9c0 con 0x7f36940b3a60 2023-10-11T09:41:33.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.897+0000 7f368bfff700 1 -- 172.21.15.78:0/3598890955 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3684051960 con 0x7f36940b3a60 2023-10-11T09:41:33.902 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:41:33.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.901+0000 7f36989b9700 1 -- 172.21.15.78:0/3598890955 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3674065ed0 msgr2=0x7f3674068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:33.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.901+0000 7f36989b9700 1 --2- 172.21.15.78:0/3598890955 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3674065ed0 0x7f3674068380 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f367c006b40 tx=0x7f367c006ad0 comp rx=0 tx=0).stop 2023-10-11T09:41:33.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.901+0000 7f36989b9700 1 -- 172.21.15.78:0/3598890955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36940b3a60 msgr2=0x7f3694158120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:33.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.901+0000 7f36989b9700 1 --2- 172.21.15.78:0/3598890955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36940b3a60 0x7f3694158120 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3684000c00 tx=0x7f3684004770 comp rx=0 tx=0).stop 2023-10-11T09:41:33.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.901+0000 7f36989b9700 1 -- 172.21.15.78:0/3598890955 shutdown_connections 2023-10-11T09:41:33.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.901+0000 7f36989b9700 1 --2- 172.21.15.78:0/3598890955 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3674065ed0 0x7f3674068380 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:33.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.901+0000 7f36989b9700 1 --2- 172.21.15.78:0/3598890955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f369415a6b0 0x7f369413d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:33.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.901+0000 7f36989b9700 1 --2- 172.21.15.78:0/3598890955 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36940cede0 0x7f3694155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:33.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.901+0000 7f36989b9700 1 --2- 172.21.15.78:0/3598890955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36940b3a60 0x7f3694158120 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:33.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.901+0000 7f36989b9700 1 -- 172.21.15.78:0/3598890955 >> 172.21.15.78:0/3598890955 conn(0x7f369406a060 msgr2=0x7f369406ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:33.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.901+0000 7f36989b9700 1 -- 172.21.15.78:0/3598890955 shutdown_connections 2023-10-11T09:41:33.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:33.901+0000 7f36989b9700 1 -- 172.21.15.78:0/3598890955 wait complete. 2023-10-11T09:41:33.927 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T09:41:34.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.097+0000 7f928bc9b700 1 Processor -- start 2023-10-11T09:41:34.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.097+0000 7f928bc9b700 1 -- start start 2023-10-11T09:41:34.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.097+0000 7f928bc9b700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f92840b8890 0x7f92840d2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:34.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.097+0000 7f928bc9b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f92840d3510 0x7f92840d3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:34.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.097+0000 7f928bc9b700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92840cd690 con 0x7f92840b84c0 2023-10-11T09:41:34.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.097+0000 7f928bc9b700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92840d3eb0 con 0x7f92840d3510 2023-10-11T09:41:34.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.097+0000 7f928bc9b700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92840cdea0 con 0x7f92840b8890 2023-10-11T09:41:34.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.097+0000 7f928a238700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f92840d3510 0x7f92840d3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:34.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.097+0000 7f928a238700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f92840d3510 0x7f92840d3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59662/0 (socket says 172.21.15.78:59662) 2023-10-11T09:41:34.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.097+0000 7f9289a37700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f92840b84c0 0x7f92840cfda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:57606/0 (socket says 172.21.15.78:57606) 2023-10-11T09:41:34.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.097+0000 7f928a238700 1 -- 172.21.15.78:0/3816261026 learned_addr learned my addr 172.21.15.78:0/3816261026 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:34.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.101+0000 7f9289236700 1 --2- 172.21.15.78:0/3816261026 >> v2:172.21.15.133:3300/0 conn(0x7f92840b8890 0x7f92840d2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:34.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.101+0000 7f928a238700 1 -- 172.21.15.78:0/3816261026 >> v2:172.21.15.133:3300/0 conn(0x7f92840b8890 msgr2=0x7f92840d2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:34.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.101+0000 7f928a238700 1 --2- 172.21.15.78:0/3816261026 >> v2:172.21.15.133:3300/0 conn(0x7f92840b8890 0x7f92840d2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:34.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.101+0000 7f928a238700 1 -- 172.21.15.78:0/3816261026 >> v1:172.21.15.78:6789/0 conn(0x7f92840b84c0 legacy=0x7f92840cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:34.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.101+0000 7f928a238700 1 -- 172.21.15.78:0/3816261026 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92840cd8e0 con 0x7f92840d3510 2023-10-11T09:41:34.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.101+0000 7f928a238700 1 --2- 172.21.15.78:0/3816261026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92840d3510 0x7f92840d3970 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f928408c380 tx=0x7f927c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d159ce8578a76a52 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:34.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.101+0000 7f9288a35700 1 -- 172.21.15.78:0/3816261026 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c00e070 con 0x7f92840d3510 2023-10-11T09:41:34.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.101+0000 7f9288a35700 1 -- 172.21.15.78:0/3816261026 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927c00b910 con 0x7f92840d3510 2023-10-11T09:41:34.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.101+0000 7f9288a35700 1 -- 172.21.15.78:0/3816261026 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c00bcc0 con 0x7f92840d3510 2023-10-11T09:41:34.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.101+0000 7f928bc9b700 1 -- 172.21.15.78:0/3816261026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92840d3510 msgr2=0x7f92840d3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:34.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.101+0000 7f928bc9b700 1 --2- 172.21.15.78:0/3816261026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92840d3510 0x7f92840d3970 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f928408c380 tx=0x7f927c00b330 comp rx=0 tx=0).stop 2023-10-11T09:41:34.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.101+0000 7f928bc9b700 1 -- 172.21.15.78:0/3816261026 shutdown_connections 2023-10-11T09:41:34.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.101+0000 7f928bc9b700 1 --2- 172.21.15.78:0/3816261026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92840d3510 0x7f92840d3970 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:34.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.101+0000 7f928bc9b700 1 --2- 172.21.15.78:0/3816261026 >> v2:172.21.15.133:3300/0 conn(0x7f92840b8890 0x7f92840d2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:34.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.101+0000 7f928bc9b700 1 -- 172.21.15.78:0/3816261026 >> 172.21.15.78:0/3816261026 conn(0x7f928406cb50 msgr2=0x7f928406cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:34.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.101+0000 7f928bc9b700 1 -- 172.21.15.78:0/3816261026 shutdown_connections 2023-10-11T09:41:34.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.101+0000 7f928bc9b700 1 -- 172.21.15.78:0/3816261026 wait complete. 2023-10-11T09:41:34.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.105+0000 7f928bc9b700 1 Processor -- start 2023-10-11T09:41:34.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.105+0000 7f928bc9b700 1 -- start start 2023-10-11T09:41:34.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.105+0000 7f928bc9b700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92840b84c0 0x7f92840c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:34.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.105+0000 7f928bc9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92840b8890 0x7f92841e2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:34.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.105+0000 7f928bc9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92840d3510 0x7f928414e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:34.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.105+0000 7f928bc9b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92841e2ad0 con 0x7f92840d3510 2023-10-11T09:41:34.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.105+0000 7f928bc9b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92840c6ff0 con 0x7f92840b8890 2023-10-11T09:41:34.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.105+0000 7f928bc9b700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92841e32c0 con 0x7f92840b84c0 2023-10-11T09:41:34.114 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.105+0000 7f928a238700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92840d3510 0x7f928414e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:34.114 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.109+0000 7f928a238700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92840d3510 0x7f928414e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59668/0 (socket says 172.21.15.78:59668) 2023-10-11T09:41:34.114 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.109+0000 7f928a238700 1 -- 172.21.15.78:0/2284523632 learned_addr learned my addr 172.21.15.78:0/2284523632 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:34.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.109+0000 7f9289236700 1 --2- 172.21.15.78:0/2284523632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92840b8890 0x7f92841e2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:34.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.109+0000 7f9289a37700 1 --2- 172.21.15.78:0/2284523632 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92840b84c0 0x7f92840c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:34.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.109+0000 7f928a238700 1 -- 172.21.15.78:0/2284523632 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92840b84c0 msgr2=0x7f92840c95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:34.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.109+0000 7f928a238700 1 --2- 172.21.15.78:0/2284523632 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92840b84c0 0x7f92840c95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:34.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.109+0000 7f928a238700 1 -- 172.21.15.78:0/2284523632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92840b8890 msgr2=0x7f92841e2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:34.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.109+0000 7f928a238700 1 --2- 172.21.15.78:0/2284523632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92840b8890 0x7f92841e2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:34.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.109+0000 7f928a238700 1 -- 172.21.15.78:0/2284523632 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f927c009b30 con 0x7f92840d3510 2023-10-11T09:41:34.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.109+0000 7f928a238700 1 --2- 172.21.15.78:0/2284523632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92840d3510 0x7f928414e470 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9284067640 tx=0x7f927c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:34.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.109+0000 7f927a7fc700 1 -- 172.21.15.78:0/2284523632 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c0099b0 con 0x7f92840d3510 2023-10-11T09:41:34.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.109+0000 7f927a7fc700 1 -- 172.21.15.78:0/2284523632 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927c0092e0 con 0x7f92840d3510 2023-10-11T09:41:34.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.109+0000 7f927a7fc700 1 -- 172.21.15.78:0/2284523632 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c01b630 con 0x7f92840d3510 2023-10-11T09:41:34.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.109+0000 7f928bc9b700 1 -- 172.21.15.78:0/2284523632 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f928414e9b0 con 0x7f92840d3510 2023-10-11T09:41:34.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.109+0000 7f928bc9b700 1 -- 172.21.15.78:0/2284523632 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92841e2fe0 con 0x7f92840d3510 2023-10-11T09:41:34.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.109+0000 7f928bc9b700 1 -- 172.21.15.78:0/2284523632 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92840c7180 con 0x7f92840d3510 2023-10-11T09:41:34.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.113+0000 7f927a7fc700 1 -- 172.21.15.78:0/2284523632 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f927c01b790 con 0x7f92840d3510 2023-10-11T09:41:34.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.113+0000 7f927a7fc700 1 --2- 172.21.15.78:0/2284523632 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9268065ed0 0x7f9268068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:34.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.113+0000 7f927a7fc700 1 -- 172.21.15.78:0/2284523632 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f927c084dc0 con 0x7f92840d3510 2023-10-11T09:41:34.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.113+0000 7f9289a37700 1 --2- 172.21.15.78:0/2284523632 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9268065ed0 0x7f9268068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:34.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.113+0000 7f9289a37700 1 --2- 172.21.15.78:0/2284523632 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9268065ed0 0x7f9268068380 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9280000f60 tx=0x7f9280007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:34.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.117+0000 7f927a7fc700 1 -- 172.21.15.78:0/2284523632 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f927c052510 con 0x7f92840d3510 2023-10-11T09:41:34.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.289+0000 7f928bc9b700 1 -- 172.21.15.78:0/2284523632 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f928408bff0 con 0x7f92840d3510 2023-10-11T09:41:34.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.293+0000 7f927a7fc700 1 -- 172.21.15.78:0/2284523632 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f927c055b30 con 0x7f92840d3510 2023-10-11T09:41:34.298 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2023-10-11T09:41:34.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.293+0000 7f928bc9b700 1 -- 172.21.15.78:0/2284523632 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9268065ed0 msgr2=0x7f9268068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:34.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.293+0000 7f928bc9b700 1 --2- 172.21.15.78:0/2284523632 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9268065ed0 0x7f9268068380 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9280000f60 tx=0x7f9280007d30 comp rx=0 tx=0).stop 2023-10-11T09:41:34.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.293+0000 7f928bc9b700 1 -- 172.21.15.78:0/2284523632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92840d3510 msgr2=0x7f928414e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:34.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.293+0000 7f928bc9b700 1 --2- 172.21.15.78:0/2284523632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92840d3510 0x7f928414e470 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9284067640 tx=0x7f927c004af0 comp rx=0 tx=0).stop 2023-10-11T09:41:34.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.297+0000 7f928bc9b700 1 -- 172.21.15.78:0/2284523632 shutdown_connections 2023-10-11T09:41:34.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.297+0000 7f928bc9b700 1 --2- 172.21.15.78:0/2284523632 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9268065ed0 0x7f9268068380 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:34.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.297+0000 7f928bc9b700 1 --2- 172.21.15.78:0/2284523632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92840d3510 0x7f928414e470 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:34.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.297+0000 7f928bc9b700 1 --2- 172.21.15.78:0/2284523632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92840b8890 0x7f92841e2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:34.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.297+0000 7f928bc9b700 1 --2- 172.21.15.78:0/2284523632 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92840b84c0 0x7f92840c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:34.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.297+0000 7f928bc9b700 1 -- 172.21.15.78:0/2284523632 >> 172.21.15.78:0/2284523632 conn(0x7f928406cb50 msgr2=0x7f92840cc120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:34.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.297+0000 7f928bc9b700 1 -- 172.21.15.78:0/2284523632 shutdown_connections 2023-10-11T09:41:34.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.297+0000 7f928bc9b700 1 -- 172.21.15.78:0/2284523632 wait complete. 2023-10-11T09:41:34.320 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T09:41:34.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.489+0000 7f2306149700 1 Processor -- start 2023-10-11T09:41:34.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.489+0000 7f2306149700 1 -- start start 2023-10-11T09:41:34.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.489+0000 7f2306149700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2300072740 0x7f230006b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:34.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.489+0000 7f2306149700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23000cbe50 con 0x7f230015c980 2023-10-11T09:41:34.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.489+0000 7f2306149700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f230015cd50 con 0x7f230006cdb0 2023-10-11T09:41:34.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.489+0000 7f2306149700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2300160610 con 0x7f2300072740 2023-10-11T09:41:34.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.489+0000 7f22ff7fe700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f230015c980 0x7f230006c6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:57608/0 (socket says 172.21.15.78:57608) 2023-10-11T09:41:34.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.489+0000 7f22ff7fe700 1 -- 172.21.15.78:0/1606791537 learned_addr learned my addr 172.21.15.78:0/1606791537 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:34.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.489+0000 7f22fffff700 1 --2- 172.21.15.78:0/1606791537 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2300072740 0x7f230006b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:34.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.493+0000 7f22fe7fc700 1 -- 172.21.15.78:0/1606791537 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 808590076 0 0) 0x7f230015cd50 con 0x7f230006cdb0 2023-10-11T09:41:34.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.493+0000 7f22fe7fc700 1 -- 172.21.15.78:0/1606791537 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22e80035d0 con 0x7f230006cdb0 2023-10-11T09:41:34.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.493+0000 7f22fe7fc700 1 -- 172.21.15.78:0/1606791537 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3661215678 0 0) 0x7f23000cbe50 con 0x7f230015c980 2023-10-11T09:41:34.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.493+0000 7f22fe7fc700 1 -- 172.21.15.78:0/1606791537 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f230015cd50 con 0x7f230015c980 2023-10-11T09:41:34.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.493+0000 7f22fffff700 1 -- 172.21.15.78:0/1606791537 >> v1:172.21.15.133:6789/0 conn(0x7f230006cdb0 legacy=0x7f2300070930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:34.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.493+0000 7f22fffff700 1 -- 172.21.15.78:0/1606791537 >> v1:172.21.15.78:6789/0 conn(0x7f230015c980 legacy=0x7f230006c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:34.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.493+0000 7f22fe7fc700 1 -- 172.21.15.78:0/1606791537 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 438046889 0 0) 0x7f230015cd50 con 0x7f230015c980 2023-10-11T09:41:34.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.493+0000 7f22fffff700 1 -- 172.21.15.78:0/1606791537 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23001608f0 con 0x7f2300072740 2023-10-11T09:41:34.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.493+0000 7f22fffff700 1 --2- 172.21.15.78:0/1606791537 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2300072740 0x7f230006b500 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f22f8009880 tx=0x7f22f8009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a64beaeceb2be747 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:34.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.493+0000 7f22fe7fc700 1 -- 172.21.15.78:0/1606791537 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f00019f0 con 0x7f2300072740 2023-10-11T09:41:34.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.493+0000 7f22fe7fc700 1 -- 172.21.15.78:0/1606791537 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f8004030 con 0x7f2300072740 2023-10-11T09:41:34.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.493+0000 7f22fe7fc700 1 -- 172.21.15.78:0/1606791537 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f80043c0 con 0x7f2300072740 2023-10-11T09:41:34.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.493+0000 7f2306149700 1 -- 172.21.15.78:0/1606791537 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2300072740 msgr2=0x7f230006b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:34.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.493+0000 7f2306149700 1 --2- 172.21.15.78:0/1606791537 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2300072740 0x7f230006b500 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f22f8009880 tx=0x7f22f8009b90 comp rx=0 tx=0).stop 2023-10-11T09:41:34.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.493+0000 7f2306149700 1 -- 172.21.15.78:0/1606791537 shutdown_connections 2023-10-11T09:41:34.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.493+0000 7f2306149700 1 --2- 172.21.15.78:0/1606791537 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2300072740 0x7f230006b500 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:34.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.493+0000 7f2306149700 1 -- 172.21.15.78:0/1606791537 >> 172.21.15.78:0/1606791537 conn(0x7f23000b1d50 msgr2=0x7f23000cf5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:34.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.493+0000 7f2306149700 1 -- 172.21.15.78:0/1606791537 shutdown_connections 2023-10-11T09:41:34.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.493+0000 7f2306149700 1 -- 172.21.15.78:0/1606791537 wait complete. 2023-10-11T09:41:34.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.497+0000 7f2306149700 1 Processor -- start 2023-10-11T09:41:34.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.497+0000 7f2306149700 1 -- start start 2023-10-11T09:41:34.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.497+0000 7f2306149700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f230006cdb0 0x7f23001f5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:34.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.497+0000 7f2306149700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2300072740 0x7f230014ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:34.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.497+0000 7f2306149700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f230015c980 0x7f230015ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:34.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.497+0000 7f2306149700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230015f9e0 con 0x7f2300072740 2023-10-11T09:41:34.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.497+0000 7f2306149700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230015fcc0 con 0x7f230015c980 2023-10-11T09:41:34.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.497+0000 7f2306149700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23001f5400 con 0x7f230006cdb0 2023-10-11T09:41:34.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.497+0000 7f22feffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2300072740 0x7f230014ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:34.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.497+0000 7f22feffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2300072740 0x7f230014ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59674/0 (socket says 172.21.15.78:59674) 2023-10-11T09:41:34.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.497+0000 7f22feffd700 1 -- 172.21.15.78:0/3706342892 learned_addr learned my addr 172.21.15.78:0/3706342892 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:34.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.501+0000 7f22fffff700 1 --2- 172.21.15.78:0/3706342892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f230015c980 0x7f230015ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:34.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.501+0000 7f22ff7fe700 1 --2- 172.21.15.78:0/3706342892 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f230006cdb0 0x7f23001f5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:34.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.501+0000 7f22feffd700 1 -- 172.21.15.78:0/3706342892 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f230006cdb0 msgr2=0x7f23001f5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:34.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.501+0000 7f22feffd700 1 --2- 172.21.15.78:0/3706342892 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f230006cdb0 0x7f23001f5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:34.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.501+0000 7f22feffd700 1 -- 172.21.15.78:0/3706342892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f230015c980 msgr2=0x7f230015ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:34.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.501+0000 7f22feffd700 1 --2- 172.21.15.78:0/3706342892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f230015c980 0x7f230015ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:34.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.501+0000 7f22feffd700 1 -- 172.21.15.78:0/3706342892 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f80094b0 con 0x7f2300072740 2023-10-11T09:41:34.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.501+0000 7f22ff7fe700 1 --2- 172.21.15.78:0/3706342892 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f230006cdb0 0x7f23001f5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:34.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.501+0000 7f22feffd700 1 --2- 172.21.15.78:0/3706342892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2300072740 0x7f230014ac40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f22f400c730 tx=0x7f22f400ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:34.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.501+0000 7f22e7fff700 1 -- 172.21.15.78:0/3706342892 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f40052d0 con 0x7f2300072740 2023-10-11T09:41:34.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.501+0000 7f22e7fff700 1 -- 172.21.15.78:0/3706342892 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f4005450 con 0x7f2300072740 2023-10-11T09:41:34.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.501+0000 7f22e7fff700 1 -- 172.21.15.78:0/3706342892 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f40038e0 con 0x7f2300072740 2023-10-11T09:41:34.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.501+0000 7f2306149700 1 -- 172.21.15.78:0/3706342892 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23001f56e0 con 0x7f2300072740 2023-10-11T09:41:34.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.501+0000 7f2306149700 1 -- 172.21.15.78:0/3706342892 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23000cf1e0 con 0x7f2300072740 2023-10-11T09:41:34.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.501+0000 7f2306149700 1 -- 172.21.15.78:0/3706342892 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23001f5c00 con 0x7f2300072740 2023-10-11T09:41:34.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.505+0000 7f22e7fff700 1 -- 172.21.15.78:0/3706342892 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f22f4003a40 con 0x7f2300072740 2023-10-11T09:41:34.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.505+0000 7f22e7fff700 1 --2- 172.21.15.78:0/3706342892 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f22ec065ed0 0x7f22ec068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:34.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.505+0000 7f22e7fff700 1 -- 172.21.15.78:0/3706342892 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f22f4082db0 con 0x7f2300072740 2023-10-11T09:41:34.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.505+0000 7f22ff7fe700 1 --2- 172.21.15.78:0/3706342892 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f22ec065ed0 0x7f22ec068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:34.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.509+0000 7f22ff7fe700 1 --2- 172.21.15.78:0/3706342892 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f22ec065ed0 0x7f22ec068380 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f22f0001560 tx=0x7f22f0001970 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:34.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.509+0000 7f22e7fff700 1 -- 172.21.15.78:0/3706342892 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f23001f5c00 con 0x7f2300072740 2023-10-11T09:41:34.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.697+0000 7f2306149700 1 -- 172.21.15.78:0/3706342892 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f23001f5c00 con 0x7f2300072740 2023-10-11T09:41:34.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.697+0000 7f22e7fff700 1 -- 172.21.15.78:0/3706342892 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f23001f5c00 con 0x7f2300072740 2023-10-11T09:41:34.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.697+0000 7f2306149700 1 -- 172.21.15.78:0/3706342892 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f22ec065ed0 msgr2=0x7f22ec068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:34.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.697+0000 7f2306149700 1 --2- 172.21.15.78:0/3706342892 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f22ec065ed0 0x7f22ec068380 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f22f0001560 tx=0x7f22f0001970 comp rx=0 tx=0).stop 2023-10-11T09:41:34.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.697+0000 7f2306149700 1 -- 172.21.15.78:0/3706342892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2300072740 msgr2=0x7f230014ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:34.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.697+0000 7f2306149700 1 --2- 172.21.15.78:0/3706342892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2300072740 0x7f230014ac40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f22f400c730 tx=0x7f22f400ca40 comp rx=0 tx=0).stop 2023-10-11T09:41:34.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.701+0000 7f2306149700 1 -- 172.21.15.78:0/3706342892 shutdown_connections 2023-10-11T09:41:34.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.701+0000 7f2306149700 1 --2- 172.21.15.78:0/3706342892 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f22ec065ed0 0x7f22ec068380 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:34.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.701+0000 7f2306149700 1 --2- 172.21.15.78:0/3706342892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f230015c980 0x7f230015ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:34.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.701+0000 7f2306149700 1 --2- 172.21.15.78:0/3706342892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2300072740 0x7f230014ac40 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:34.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.701+0000 7f2306149700 1 --2- 172.21.15.78:0/3706342892 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f230006cdb0 0x7f23001f5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:34.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.701+0000 7f2306149700 1 -- 172.21.15.78:0/3706342892 >> 172.21.15.78:0/3706342892 conn(0x7f23000b1d50 msgr2=0x7f2300071820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:34.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.701+0000 7f2306149700 1 -- 172.21.15.78:0/3706342892 shutdown_connections 2023-10-11T09:41:34.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.701+0000 7f2306149700 1 -- 172.21.15.78:0/3706342892 wait complete. 2023-10-11T09:41:34.708 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2023-10-11T09:41:34.725 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:41:34.726 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:41:34.744 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:41:34.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.953+0000 7ff7e7276700 1 Processor -- start 2023-10-11T09:41:34.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.953+0000 7ff7e7276700 1 -- start start 2023-10-11T09:41:34.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.957+0000 7ff7e7276700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff7e00cda10 0x7ff7e00caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:34.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.957+0000 7ff7e7276700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ff7e00cdde0 0x7ff7e0164610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:34.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.957+0000 7ff7e7276700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7e0165390 con 0x7ff7e0164b50 2023-10-11T09:41:34.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.957+0000 7ff7e7276700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7e00ca440 con 0x7ff7e00cdde0 2023-10-11T09:41:34.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.957+0000 7ff7e7276700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7e00ca720 con 0x7ff7e00cda10 2023-10-11T09:41:34.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.957+0000 7ff7e5a73700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ff7e00cdde0 0x7ff7e0164610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:34.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.957+0000 7ff7e6274700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff7e00cda10 0x7ff7e00caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:34.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.957+0000 7ff7e5a73700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ff7e00cdde0 0x7ff7e0164610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54270/0 (socket says 172.21.15.78:54270) 2023-10-11T09:41:34.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.957+0000 7ff7e5a73700 1 -- 172.21.15.78:0/2647928756 learned_addr learned my addr 172.21.15.78:0/2647928756 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:34.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.957+0000 7ff7e5a73700 1 -- 172.21.15.78:0/2647928756 >> v2:172.21.15.133:3300/0 conn(0x7ff7e00cda10 msgr2=0x7ff7e00caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:34.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.957+0000 7ff7e5a73700 1 --2- 172.21.15.78:0/2647928756 >> v2:172.21.15.133:3300/0 conn(0x7ff7e00cda10 0x7ff7e00caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:34.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.957+0000 7ff7e5a73700 1 -- 172.21.15.78:0/2647928756 >> v1:172.21.15.133:6789/0 conn(0x7ff7e0164b50 legacy=0x7ff7e006bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:34.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.957+0000 7ff7e5a73700 1 -- 172.21.15.78:0/2647928756 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e00caa00 con 0x7ff7e00cdde0 2023-10-11T09:41:34.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.957+0000 7ff7e5a73700 1 --2- 172.21.15.78:0/2647928756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7e00cdde0 0x7ff7e0164610 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff7e006afb0 tx=0x7ff7d0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc2084d9058dcaf server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:34.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.957+0000 7ff7e5272700 1 -- 172.21.15.78:0/2647928756 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0011830 con 0x7ff7e00cdde0 2023-10-11T09:41:34.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.957+0000 7ff7e5272700 1 -- 172.21.15.78:0/2647928756 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d0011990 con 0x7ff7e00cdde0 2023-10-11T09:41:34.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.957+0000 7ff7e5272700 1 -- 172.21.15.78:0/2647928756 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0011d20 con 0x7ff7e00cdde0 2023-10-11T09:41:34.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.961+0000 7ff7e7276700 1 -- 172.21.15.78:0/2647928756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7e00cdde0 msgr2=0x7ff7e0164610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:34.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.961+0000 7ff7e7276700 1 --2- 172.21.15.78:0/2647928756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7e00cdde0 0x7ff7e0164610 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff7e006afb0 tx=0x7ff7d0009bf0 comp rx=0 tx=0).stop 2023-10-11T09:41:34.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.961+0000 7ff7e7276700 1 -- 172.21.15.78:0/2647928756 shutdown_connections 2023-10-11T09:41:34.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.961+0000 7ff7e7276700 1 --2- 172.21.15.78:0/2647928756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7e00cdde0 0x7ff7e0164610 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:34.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.961+0000 7ff7e7276700 1 --2- 172.21.15.78:0/2647928756 >> v2:172.21.15.133:3300/0 conn(0x7ff7e00cda10 0x7ff7e00caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:34.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.961+0000 7ff7e7276700 1 -- 172.21.15.78:0/2647928756 >> 172.21.15.78:0/2647928756 conn(0x7ff7e00cea60 msgr2=0x7ff7e00cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:34.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.961+0000 7ff7e7276700 1 -- 172.21.15.78:0/2647928756 shutdown_connections 2023-10-11T09:41:34.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.961+0000 7ff7e7276700 1 -- 172.21.15.78:0/2647928756 wait complete. 2023-10-11T09:41:34.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.965+0000 7ff7e7276700 1 Processor -- start 2023-10-11T09:41:34.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.965+0000 7ff7e7276700 1 -- start start 2023-10-11T09:41:34.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.965+0000 7ff7e7276700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7e00cda10 0x7ff7e00b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:34.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.965+0000 7ff7e7276700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7e00cdde0 0x7ff7e01f6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:34.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.965+0000 7ff7e7276700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff7e0164b50 0x7ff7e0163070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:34.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.965+0000 7ff7e7276700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e01f7060 con 0x7ff7e00cda10 2023-10-11T09:41:34.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.965+0000 7ff7e7276700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e0160730 con 0x7ff7e00cdde0 2023-10-11T09:41:34.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.965+0000 7ff7e7276700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e00b3ee0 con 0x7ff7e0164b50 2023-10-11T09:41:34.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.965+0000 7ff7e6274700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7e00cda10 0x7ff7e00b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:34.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.965+0000 7ff7e6274700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7e00cda10 0x7ff7e00b4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54284/0 (socket says 172.21.15.78:54284) 2023-10-11T09:41:34.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.965+0000 7ff7e6274700 1 -- 172.21.15.78:0/629731532 learned_addr learned my addr 172.21.15.78:0/629731532 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:34.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.965+0000 7ff7e5a73700 1 --2- 172.21.15.78:0/629731532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7e00cdde0 0x7ff7e01f6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:34.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.965+0000 7ff7e6a75700 1 --2- 172.21.15.78:0/629731532 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff7e0164b50 0x7ff7e0163070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:34.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.965+0000 7ff7e6274700 1 -- 172.21.15.78:0/629731532 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff7e0164b50 msgr2=0x7ff7e0163070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:34.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.965+0000 7ff7e6274700 1 --2- 172.21.15.78:0/629731532 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff7e0164b50 0x7ff7e0163070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:34.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.965+0000 7ff7e6274700 1 -- 172.21.15.78:0/629731532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7e00cdde0 msgr2=0x7ff7e01f6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:34.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.965+0000 7ff7e6274700 1 --2- 172.21.15.78:0/629731532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7e00cdde0 0x7ff7e01f6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:34.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.965+0000 7ff7e6274700 1 -- 172.21.15.78:0/629731532 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d0009570 con 0x7ff7e00cda10 2023-10-11T09:41:34.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.965+0000 7ff7e6274700 1 --2- 172.21.15.78:0/629731532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7e00cda10 0x7ff7e00b4370 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc00e9d0 tx=0x7ff7dc00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:34.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.965+0000 7ff7d6ffd700 1 -- 172.21.15.78:0/629731532 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc009720 con 0x7ff7e00cda10 2023-10-11T09:41:34.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.969+0000 7ff7d6ffd700 1 -- 172.21.15.78:0/629731532 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7dc00ee70 con 0x7ff7e00cda10 2023-10-11T09:41:34.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.969+0000 7ff7e7276700 1 -- 172.21.15.78:0/629731532 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7e0163e50 con 0x7ff7e00cda10 2023-10-11T09:41:34.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.969+0000 7ff7d6ffd700 1 -- 172.21.15.78:0/629731532 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc00ca50 con 0x7ff7e00cda10 2023-10-11T09:41:34.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.969+0000 7ff7e7276700 1 -- 172.21.15.78:0/629731532 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7e00b3890 con 0x7ff7e00cda10 2023-10-11T09:41:34.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.969+0000 7ff7e7276700 1 -- 172.21.15.78:0/629731532 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7e0065d40 con 0x7ff7e00cda10 2023-10-11T09:41:34.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.969+0000 7ff7d6ffd700 1 -- 172.21.15.78:0/629731532 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7ff7dc00cbb0 con 0x7ff7e00cda10 2023-10-11T09:41:34.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.973+0000 7ff7d6ffd700 1 --2- 172.21.15.78:0/629731532 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff7c4065f00 0x7ff7c40683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:34.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.973+0000 7ff7d6ffd700 1 -- 172.21.15.78:0/629731532 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff7dc0838c0 con 0x7ff7e00cda10 2023-10-11T09:41:34.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.973+0000 7ff7e5a73700 1 --2- 172.21.15.78:0/629731532 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff7c4065f00 0x7ff7c40683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:34.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.973+0000 7ff7e5a73700 1 --2- 172.21.15.78:0/629731532 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff7c4065f00 0x7ff7c40683b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0000c00 tx=0x7ff7d0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:34.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:34.977+0000 7ff7d6ffd700 1 -- 172.21.15.78:0/629731532 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7ff7dc050e50 con 0x7ff7e00cda10 2023-10-11T09:41:35.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.153+0000 7ff7e7276700 1 -- 172.21.15.78:0/629731532 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7e00b1480 con 0x7ff7e00cda10 2023-10-11T09:41:35.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.157+0000 7ff7d6ffd700 1 -- 172.21.15.78:0/629731532 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17907 (secure 0 0 0) 0x7ff7dc00ce60 con 0x7ff7e00cda10 2023-10-11T09:41:35.162 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:41:35.163 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":25,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:41:21.473566+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:40:39.862581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T09:40:40.663641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.37","mappings":[{"from":3,"to":2}]},{"pgid":"4.3e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:41:35.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.157+0000 7ff7e7276700 1 -- 172.21.15.78:0/629731532 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff7c4065f00 msgr2=0x7ff7c40683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:35.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.157+0000 7ff7e7276700 1 --2- 172.21.15.78:0/629731532 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff7c4065f00 0x7ff7c40683b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0000c00 tx=0x7ff7d0008030 comp rx=0 tx=0).stop 2023-10-11T09:41:35.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.157+0000 7ff7e7276700 1 -- 172.21.15.78:0/629731532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7e00cda10 msgr2=0x7ff7e00b4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:35.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.157+0000 7ff7e7276700 1 --2- 172.21.15.78:0/629731532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7e00cda10 0x7ff7e00b4370 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc00e9d0 tx=0x7ff7dc00ece0 comp rx=0 tx=0).stop 2023-10-11T09:41:35.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.161+0000 7ff7e7276700 1 -- 172.21.15.78:0/629731532 shutdown_connections 2023-10-11T09:41:35.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.161+0000 7ff7e7276700 1 --2- 172.21.15.78:0/629731532 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff7c4065f00 0x7ff7c40683b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:35.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.161+0000 7ff7e7276700 1 --2- 172.21.15.78:0/629731532 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff7e0164b50 0x7ff7e0163070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:35.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.161+0000 7ff7e7276700 1 --2- 172.21.15.78:0/629731532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7e00cdde0 0x7ff7e01f6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:35.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.161+0000 7ff7e7276700 1 --2- 172.21.15.78:0/629731532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7e00cda10 0x7ff7e00b4370 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:35.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.161+0000 7ff7e7276700 1 -- 172.21.15.78:0/629731532 >> 172.21.15.78:0/629731532 conn(0x7ff7e00cea60 msgr2=0x7ff7e00ae540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:35.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.161+0000 7ff7e7276700 1 -- 172.21.15.78:0/629731532 shutdown_connections 2023-10-11T09:41:35.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.161+0000 7ff7e7276700 1 -- 172.21.15.78:0/629731532 wait complete. 2023-10-11T09:41:35.185 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T09:40:39.862581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T09:40:40.663641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:41:35.186 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:41:35.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.357+0000 7f9ad1378700 1 Processor -- start 2023-10-11T09:41:35.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.361+0000 7f9ad1378700 1 -- start start 2023-10-11T09:41:35.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.361+0000 7f9ad1378700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9acc15cb60 0x7f9acc0b2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:35.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.361+0000 7f9ad1378700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9acc0cfa80 0x7f9acc0b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:35.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.361+0000 7f9ad1378700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f9acc0d0720 0x7f9acc0cb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:35.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.361+0000 7f9ad1378700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9acc15cf30 con 0x7f9acc0d0720 2023-10-11T09:41:35.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.361+0000 7f9ad1378700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9acc0d0520 con 0x7f9acc15cb60 2023-10-11T09:41:35.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.361+0000 7f9ad1378700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc0cf5f0 con 0x7f9acc0cfa80 2023-10-11T09:41:35.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.365+0000 7f9acb7fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f9acc0d0720 0x7f9acc0cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:35.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.365+0000 7f9acb7fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f9acc0d0720 0x7f9acc0cb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54292/0 (socket says 172.21.15.78:54292) 2023-10-11T09:41:35.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.365+0000 7f9acb7fe700 1 -- 172.21.15.78:0/3044156084 learned_addr learned my addr 172.21.15.78:0/3044156084 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:35.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.365+0000 7f9aca7fc700 1 --2- 172.21.15.78:0/3044156084 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9acc0cfa80 0x7f9acc0b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:35.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.365+0000 7f9acaffd700 1 --2- 172.21.15.78:0/3044156084 >> v2:172.21.15.133:3300/0 conn(0x7f9acc15cb60 0x7f9acc0b2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:35.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.365+0000 7f9acb7fe700 1 -- 172.21.15.78:0/3044156084 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9acc0cfa80 msgr2=0x7f9acc0b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:35.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.365+0000 7f9acb7fe700 1 --2- 172.21.15.78:0/3044156084 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9acc0cfa80 0x7f9acc0b2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:35.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.365+0000 7f9acb7fe700 1 -- 172.21.15.78:0/3044156084 >> v2:172.21.15.133:3300/0 conn(0x7f9acc15cb60 msgr2=0x7f9acc0b2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:35.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.365+0000 7f9acb7fe700 1 --2- 172.21.15.78:0/3044156084 >> v2:172.21.15.133:3300/0 conn(0x7f9acc15cb60 0x7f9acc0b2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:35.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.365+0000 7f9acb7fe700 1 -- 172.21.15.78:0/3044156084 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9acc0cecc0 con 0x7f9acc0d0720 2023-10-11T09:41:35.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.365+0000 7f9acb7fe700 1 --2- 172.21.15.78:0/3044156084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9acc0d0720 0x7f9acc0cb610 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4009e50 tx=0x7f9ac400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d107d0cf6c27e48 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:35.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.365+0000 7f9ac9ffb700 1 -- 172.21.15.78:0/3044156084 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac400e070 con 0x7f9acc0d0720 2023-10-11T09:41:35.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.365+0000 7f9ac9ffb700 1 -- 172.21.15.78:0/3044156084 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac400b8d0 con 0x7f9acc0d0720 2023-10-11T09:41:35.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.365+0000 7f9ac9ffb700 1 -- 172.21.15.78:0/3044156084 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac400bc80 con 0x7f9acc0d0720 2023-10-11T09:41:35.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.365+0000 7f9ad1378700 1 -- 172.21.15.78:0/3044156084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9acc0d0720 msgr2=0x7f9acc0cb610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:35.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.365+0000 7f9ad1378700 1 --2- 172.21.15.78:0/3044156084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9acc0d0720 0x7f9acc0cb610 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4009e50 tx=0x7f9ac400b330 comp rx=0 tx=0).stop 2023-10-11T09:41:35.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.365+0000 7f9ad1378700 1 -- 172.21.15.78:0/3044156084 shutdown_connections 2023-10-11T09:41:35.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.365+0000 7f9ad1378700 1 --2- 172.21.15.78:0/3044156084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9acc0d0720 0x7f9acc0cb610 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:35.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.365+0000 7f9ad1378700 1 --2- 172.21.15.78:0/3044156084 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9acc0cfa80 0x7f9acc0b2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:35.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.365+0000 7f9ad1378700 1 --2- 172.21.15.78:0/3044156084 >> v2:172.21.15.133:3300/0 conn(0x7f9acc15cb60 0x7f9acc0b2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:35.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.365+0000 7f9ad1378700 1 -- 172.21.15.78:0/3044156084 >> 172.21.15.78:0/3044156084 conn(0x7f9acc09a230 msgr2=0x7f9acc08aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:35.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.365+0000 7f9ad1378700 1 -- 172.21.15.78:0/3044156084 shutdown_connections 2023-10-11T09:41:35.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.369+0000 7f9ad1378700 1 -- 172.21.15.78:0/3044156084 wait complete. 2023-10-11T09:41:35.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.369+0000 7f9ad1378700 1 Processor -- start 2023-10-11T09:41:35.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.369+0000 7f9ad1378700 1 -- start start 2023-10-11T09:41:35.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.373+0000 7f9ad1378700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9acc0cfa80 0x7f9acc0b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:35.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.373+0000 7f9ad1378700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9acc0d0720 0x7f9acc1f19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:35.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.373+0000 7f9ad1378700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9acc15cb60 0x7f9acc0c9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:35.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.373+0000 7f9ad1378700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc0b32d0 con 0x7f9acc0d0720 2023-10-11T09:41:35.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.373+0000 7f9ad1378700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc0b07b0 con 0x7f9acc0cfa80 2023-10-11T09:41:35.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.373+0000 7f9ad1378700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc0b0a90 con 0x7f9acc15cb60 2023-10-11T09:41:35.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.373+0000 7f9aca7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9acc0d0720 0x7f9acc1f19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:35.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.373+0000 7f9aca7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9acc0d0720 0x7f9acc1f19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54296/0 (socket says 172.21.15.78:54296) 2023-10-11T09:41:35.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.373+0000 7f9aca7fc700 1 -- 172.21.15.78:0/229216860 learned_addr learned my addr 172.21.15.78:0/229216860 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:35.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.373+0000 7f9acaffd700 1 --2- 172.21.15.78:0/229216860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9acc0cfa80 0x7f9acc0b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:35.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.373+0000 7f9aca7fc700 1 -- 172.21.15.78:0/229216860 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9acc15cb60 msgr2=0x7f9acc0c9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:35.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.373+0000 7f9aca7fc700 1 --2- 172.21.15.78:0/229216860 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9acc15cb60 0x7f9acc0c9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:35.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.373+0000 7f9aca7fc700 1 -- 172.21.15.78:0/229216860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9acc0cfa80 msgr2=0x7f9acc0b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:35.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.373+0000 7f9aca7fc700 1 --2- 172.21.15.78:0/229216860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9acc0cfa80 0x7f9acc0b3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:35.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.373+0000 7f9aca7fc700 1 -- 172.21.15.78:0/229216860 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac4009b30 con 0x7f9acc0d0720 2023-10-11T09:41:35.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.373+0000 7f9acaffd700 1 --2- 172.21.15.78:0/229216860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9acc0cfa80 0x7f9acc0b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:41:35.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.373+0000 7f9aca7fc700 1 --2- 172.21.15.78:0/229216860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9acc0d0720 0x7f9acc1f19f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000e7c0 tx=0x7f9ac000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:35.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.373+0000 7f9ab37fe700 1 -- 172.21.15.78:0/229216860 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0013680 con 0x7f9acc0d0720 2023-10-11T09:41:35.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.373+0000 7f9ab37fe700 1 -- 172.21.15.78:0/229216860 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac00137e0 con 0x7f9acc0d0720 2023-10-11T09:41:35.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.373+0000 7f9ad1378700 1 -- 172.21.15.78:0/229216860 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9acc0b0d70 con 0x7f9acc0d0720 2023-10-11T09:41:35.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.373+0000 7f9ab37fe700 1 -- 172.21.15.78:0/229216860 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0013b90 con 0x7f9acc0d0720 2023-10-11T09:41:35.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.373+0000 7f9ad1378700 1 -- 172.21.15.78:0/229216860 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9acc0b1330 con 0x7f9acc0d0720 2023-10-11T09:41:35.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.377+0000 7f9ad1378700 1 -- 172.21.15.78:0/229216860 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9acc0cef50 con 0x7f9acc0d0720 2023-10-11T09:41:35.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.377+0000 7f9ab37fe700 1 -- 172.21.15.78:0/229216860 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f9ac0013cf0 con 0x7f9acc0d0720 2023-10-11T09:41:35.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.381+0000 7f9ab37fe700 1 --2- 172.21.15.78:0/229216860 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9ab8065ed0 0x7f9ab8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:35.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.381+0000 7f9ab37fe700 1 -- 172.21.15.78:0/229216860 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f9ac0083820 con 0x7f9acc0d0720 2023-10-11T09:41:35.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.381+0000 7f9acaffd700 1 --2- 172.21.15.78:0/229216860 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9ab8065ed0 0x7f9ab8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:35.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.381+0000 7f9acaffd700 1 --2- 172.21.15.78:0/229216860 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9ab8065ed0 0x7f9ab8068380 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9abc009b60 tx=0x7f9abc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:35.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.385+0000 7f9ab37fe700 1 -- 172.21.15.78:0/229216860 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f9ac0050d00 con 0x7f9acc0d0720 2023-10-11T09:41:35.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.557+0000 7f9ad1378700 1 -- 172.21.15.78:0/229216860 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9acc070570 con 0x7f9acc0d0720 2023-10-11T09:41:35.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.561+0000 7f9ab37fe700 1 -- 172.21.15.78:0/229216860 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9ac00170a0 con 0x7f9acc0d0720 2023-10-11T09:41:35.567 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:41:35.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.565+0000 7f9ad1378700 1 -- 172.21.15.78:0/229216860 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9ab8065ed0 msgr2=0x7f9ab8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:35.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.565+0000 7f9ad1378700 1 --2- 172.21.15.78:0/229216860 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9ab8065ed0 0x7f9ab8068380 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9abc009b60 tx=0x7f9abc008040 comp rx=0 tx=0).stop 2023-10-11T09:41:35.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.565+0000 7f9ad1378700 1 -- 172.21.15.78:0/229216860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9acc0d0720 msgr2=0x7f9acc1f19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:35.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.565+0000 7f9ad1378700 1 --2- 172.21.15.78:0/229216860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9acc0d0720 0x7f9acc1f19f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000e7c0 tx=0x7f9ac000eb80 comp rx=0 tx=0).stop 2023-10-11T09:41:35.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.565+0000 7f9ad1378700 1 -- 172.21.15.78:0/229216860 shutdown_connections 2023-10-11T09:41:35.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.565+0000 7f9ad1378700 1 --2- 172.21.15.78:0/229216860 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9ab8065ed0 0x7f9ab8068380 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:35.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.565+0000 7f9ad1378700 1 --2- 172.21.15.78:0/229216860 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9acc15cb60 0x7f9acc0c9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:35.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.565+0000 7f9ad1378700 1 --2- 172.21.15.78:0/229216860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9acc0d0720 0x7f9acc1f19f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:35.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.565+0000 7f9ad1378700 1 --2- 172.21.15.78:0/229216860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9acc0cfa80 0x7f9acc0b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:35.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.565+0000 7f9ad1378700 1 -- 172.21.15.78:0/229216860 >> 172.21.15.78:0/229216860 conn(0x7f9acc09a230 msgr2=0x7f9acc06b850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:35.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.565+0000 7f9ad1378700 1 -- 172.21.15.78:0/229216860 shutdown_connections 2023-10-11T09:41:35.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.565+0000 7f9ad1378700 1 -- 172.21.15.78:0/229216860 wait complete. 2023-10-11T09:41:35.593 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:41:35.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.757+0000 7fdc450f7700 1 Processor -- start 2023-10-11T09:41:35.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.757+0000 7fdc450f7700 1 -- start start 2023-10-11T09:41:35.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.761+0000 7fdc450f7700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdc400b0230 0x7fdc400ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:35.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.761+0000 7fdc450f7700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc40161550 con 0x7fdc400cfc30 2023-10-11T09:41:35.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.761+0000 7fdc450f7700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc400ce700 con 0x7fdc400cc890 2023-10-11T09:41:35.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.761+0000 7fdc450f7700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc40160940 con 0x7fdc400b0230 2023-10-11T09:41:35.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.761+0000 7fdc3f7fe700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fdc400cfc30 0x7fdc400afb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33130/0 (socket says 172.21.15.78:33130) 2023-10-11T09:41:35.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.761+0000 7fdc3f7fe700 1 -- 172.21.15.78:0/1959015307 learned_addr learned my addr 172.21.15.78:0/1959015307 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:35.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.761+0000 7fdc448f6700 1 --2- 172.21.15.78:0/1959015307 >> v2:172.21.15.133:3300/0 conn(0x7fdc400b0230 0x7fdc400ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:35.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.761+0000 7fdc3effd700 1 -- 172.21.15.78:0/1959015307 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 782723223 0 0) 0x7fdc40161550 con 0x7fdc400cfc30 2023-10-11T09:41:35.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.761+0000 7fdc3effd700 1 -- 172.21.15.78:0/1959015307 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc280035d0 con 0x7fdc400cfc30 2023-10-11T09:41:35.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.761+0000 7fdc448f6700 1 -- 172.21.15.78:0/1959015307 >> v1:172.21.15.133:6789/0 conn(0x7fdc400cc890 legacy=0x7fdc400b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:35.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.761+0000 7fdc448f6700 1 -- 172.21.15.78:0/1959015307 >> v1:172.21.15.78:6789/0 conn(0x7fdc400cfc30 legacy=0x7fdc400afb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:35.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.761+0000 7fdc448f6700 1 -- 172.21.15.78:0/1959015307 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc40160c20 con 0x7fdc400b0230 2023-10-11T09:41:35.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.761+0000 7fdc448f6700 1 --2- 172.21.15.78:0/1959015307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc400b0230 0x7fdc400ccc60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdc38009d90 tx=0x7fdc3800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cc14fcddbb93359 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:35.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.761+0000 7fdc3effd700 1 -- 172.21.15.78:0/1959015307 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3800e070 con 0x7fdc400b0230 2023-10-11T09:41:35.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.761+0000 7fdc3effd700 1 -- 172.21.15.78:0/1959015307 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc3800bb20 con 0x7fdc400b0230 2023-10-11T09:41:35.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.761+0000 7fdc3effd700 1 -- 172.21.15.78:0/1959015307 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc38012430 con 0x7fdc400b0230 2023-10-11T09:41:35.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.761+0000 7fdc450f7700 1 -- 172.21.15.78:0/1959015307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc400b0230 msgr2=0x7fdc400ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:35.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.761+0000 7fdc450f7700 1 --2- 172.21.15.78:0/1959015307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc400b0230 0x7fdc400ccc60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdc38009d90 tx=0x7fdc3800b330 comp rx=0 tx=0).stop 2023-10-11T09:41:35.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.765+0000 7fdc450f7700 1 -- 172.21.15.78:0/1959015307 shutdown_connections 2023-10-11T09:41:35.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.765+0000 7fdc450f7700 1 --2- 172.21.15.78:0/1959015307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc400b0230 0x7fdc400ccc60 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:35.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.765+0000 7fdc450f7700 1 -- 172.21.15.78:0/1959015307 >> 172.21.15.78:0/1959015307 conn(0x7fdc4006a060 msgr2=0x7fdc4006a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:35.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.765+0000 7fdc450f7700 1 -- 172.21.15.78:0/1959015307 shutdown_connections 2023-10-11T09:41:35.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.765+0000 7fdc450f7700 1 -- 172.21.15.78:0/1959015307 wait complete. 2023-10-11T09:41:35.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.765+0000 7fdc450f7700 1 Processor -- start 2023-10-11T09:41:35.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.765+0000 7fdc450f7700 1 -- start start 2023-10-11T09:41:35.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.765+0000 7fdc450f7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc400b0230 0x7fdc400aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:35.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.769+0000 7fdc450f7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdc400cc890 0x7fdc4006bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:35.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.769+0000 7fdc450f7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc400cfc30 0x7fdc40158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:35.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.769+0000 7fdc450f7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc401601a0 con 0x7fdc400cfc30 2023-10-11T09:41:35.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.769+0000 7fdc450f7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4015f590 con 0x7fdc400b0230 2023-10-11T09:41:35.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.769+0000 7fdc450f7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4015f870 con 0x7fdc400cc890 2023-10-11T09:41:35.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.769+0000 7fdc448f6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc400cfc30 0x7fdc40158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:35.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.769+0000 7fdc448f6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc400cfc30 0x7fdc40158320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54302/0 (socket says 172.21.15.78:54302) 2023-10-11T09:41:35.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.769+0000 7fdc448f6700 1 -- 172.21.15.78:0/4073040114 learned_addr learned my addr 172.21.15.78:0/4073040114 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:35.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.769+0000 7fdc3ffff700 1 --2- 172.21.15.78:0/4073040114 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc400b0230 0x7fdc400aeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:35.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.769+0000 7fdc3f7fe700 1 --2- 172.21.15.78:0/4073040114 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdc400cc890 0x7fdc4006bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:35.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.769+0000 7fdc448f6700 1 -- 172.21.15.78:0/4073040114 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdc400cc890 msgr2=0x7fdc4006bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:35.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.769+0000 7fdc448f6700 1 --2- 172.21.15.78:0/4073040114 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdc400cc890 0x7fdc4006bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:35.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.769+0000 7fdc448f6700 1 -- 172.21.15.78:0/4073040114 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc400b0230 msgr2=0x7fdc400aeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:35.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.769+0000 7fdc448f6700 1 --2- 172.21.15.78:0/4073040114 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc400b0230 0x7fdc400aeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:35.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.769+0000 7fdc448f6700 1 -- 172.21.15.78:0/4073040114 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc38009a70 con 0x7fdc400cfc30 2023-10-11T09:41:35.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.769+0000 7fdc3f7fe700 1 --2- 172.21.15.78:0/4073040114 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdc400cc890 0x7fdc4006bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:35.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.769+0000 7fdc448f6700 1 --2- 172.21.15.78:0/4073040114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc400cfc30 0x7fdc40158320 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdc38013000 tx=0x7fdc38004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:35.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.769+0000 7fdc3cff9700 1 -- 172.21.15.78:0/4073040114 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc38012430 con 0x7fdc400cfc30 2023-10-11T09:41:35.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.769+0000 7fdc3cff9700 1 -- 172.21.15.78:0/4073040114 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc38020070 con 0x7fdc400cfc30 2023-10-11T09:41:35.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.769+0000 7fdc450f7700 1 -- 172.21.15.78:0/4073040114 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc4015fb50 con 0x7fdc400cfc30 2023-10-11T09:41:35.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.769+0000 7fdc3cff9700 1 -- 172.21.15.78:0/4073040114 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc38004280 con 0x7fdc400cfc30 2023-10-11T09:41:35.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.769+0000 7fdc450f7700 1 -- 172.21.15.78:0/4073040114 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc400aeff0 con 0x7fdc400cfc30 2023-10-11T09:41:35.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.769+0000 7fdc450f7700 1 -- 172.21.15.78:0/4073040114 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc400c4910 con 0x7fdc400cfc30 2023-10-11T09:41:35.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.773+0000 7fdc3cff9700 1 -- 172.21.15.78:0/4073040114 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fdc380043e0 con 0x7fdc400cfc30 2023-10-11T09:41:35.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.773+0000 7fdc3cff9700 1 --2- 172.21.15.78:0/4073040114 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdc2c065ed0 0x7fdc2c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:35.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.773+0000 7fdc3cff9700 1 -- 172.21.15.78:0/4073040114 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fdc38022070 con 0x7fdc400cfc30 2023-10-11T09:41:35.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.773+0000 7fdc3ffff700 1 --2- 172.21.15.78:0/4073040114 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdc2c065ed0 0x7fdc2c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:35.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.777+0000 7fdc3ffff700 1 --2- 172.21.15.78:0/4073040114 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdc2c065ed0 0x7fdc2c068380 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdc30000f60 tx=0x7fdc30009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:35.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.781+0000 7fdc3cff9700 1 -- 172.21.15.78:0/4073040114 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fdc400c4910 con 0x7fdc400cfc30 2023-10-11T09:41:35.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.949+0000 7fdc450f7700 1 -- 172.21.15.78:0/4073040114 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdc400c4910 con 0x7fdc400cfc30 2023-10-11T09:41:35.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.953+0000 7fdc3cff9700 1 -- 172.21.15.78:0/4073040114 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdc400c4910 con 0x7fdc400cfc30 2023-10-11T09:41:35.958 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:41:35.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.957+0000 7fdc450f7700 1 -- 172.21.15.78:0/4073040114 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdc2c065ed0 msgr2=0x7fdc2c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:35.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.957+0000 7fdc450f7700 1 --2- 172.21.15.78:0/4073040114 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdc2c065ed0 0x7fdc2c068380 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdc30000f60 tx=0x7fdc30009040 comp rx=0 tx=0).stop 2023-10-11T09:41:35.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.957+0000 7fdc450f7700 1 -- 172.21.15.78:0/4073040114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc400cfc30 msgr2=0x7fdc40158320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:35.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.957+0000 7fdc450f7700 1 --2- 172.21.15.78:0/4073040114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc400cfc30 0x7fdc40158320 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdc38013000 tx=0x7fdc38004af0 comp rx=0 tx=0).stop 2023-10-11T09:41:35.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.957+0000 7fdc450f7700 1 -- 172.21.15.78:0/4073040114 shutdown_connections 2023-10-11T09:41:35.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.957+0000 7fdc450f7700 1 --2- 172.21.15.78:0/4073040114 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdc2c065ed0 0x7fdc2c068380 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:35.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.957+0000 7fdc450f7700 1 --2- 172.21.15.78:0/4073040114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc400cfc30 0x7fdc40158320 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:35.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.957+0000 7fdc450f7700 1 --2- 172.21.15.78:0/4073040114 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdc400cc890 0x7fdc4006bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:35.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.957+0000 7fdc450f7700 1 --2- 172.21.15.78:0/4073040114 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc400b0230 0x7fdc400aeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:35.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.957+0000 7fdc450f7700 1 -- 172.21.15.78:0/4073040114 >> 172.21.15.78:0/4073040114 conn(0x7fdc4006a060 msgr2=0x7fdc400b2410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:35.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.957+0000 7fdc450f7700 1 -- 172.21.15.78:0/4073040114 shutdown_connections 2023-10-11T09:41:35.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:35.957+0000 7fdc450f7700 1 -- 172.21.15.78:0/4073040114 wait complete. 2023-10-11T09:41:35.985 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T09:41:36.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.153+0000 7f89b9cb5700 1 Processor -- start 2023-10-11T09:41:36.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.153+0000 7f89b9cb5700 1 -- start start 2023-10-11T09:41:36.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.157+0000 7f89b9cb5700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f89b40cd3a0 0x7f89b406c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:36.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.157+0000 7f89b9cb5700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89b4159c00 con 0x7f89b4159830 2023-10-11T09:41:36.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.157+0000 7f89b9cb5700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89b40cd0f0 con 0x7f89b406e490 2023-10-11T09:41:36.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.157+0000 7f89b9cb5700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89b406cd70 con 0x7f89b40cd3a0 2023-10-11T09:41:36.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.157+0000 7f89b37fe700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f89b4159830 0x7f89b406dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33134/0 (socket says 172.21.15.78:33134) 2023-10-11T09:41:36.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.157+0000 7f89b37fe700 1 -- 172.21.15.78:0/2861946669 learned_addr learned my addr 172.21.15.78:0/2861946669 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:36.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.157+0000 7f89b3fff700 1 --2- 172.21.15.78:0/2861946669 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f89b40cd3a0 0x7f89b406c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:36.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.157+0000 7f89b2ffd700 1 -- 172.21.15.78:0/2861946669 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3114095928 0 0) 0x7f89a4001110 con 0x7f89b4159830 2023-10-11T09:41:36.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.157+0000 7f89b2ffd700 1 -- 172.21.15.78:0/2861946669 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89a00035d0 con 0x7f89b4159830 2023-10-11T09:41:36.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.157+0000 7f89b2ffd700 1 -- 172.21.15.78:0/2861946669 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1013476553 0 0) 0x7f89b40cd0f0 con 0x7f89b406e490 2023-10-11T09:41:36.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.157+0000 7f89b2ffd700 1 -- 172.21.15.78:0/2861946669 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89a4001110 con 0x7f89b406e490 2023-10-11T09:41:36.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.157+0000 7f89b2ffd700 1 -- 172.21.15.78:0/2861946669 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 387433313 0 0) 0x7f89a00035d0 con 0x7f89b4159830 2023-10-11T09:41:36.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.157+0000 7f89b2ffd700 1 -- 172.21.15.78:0/2861946669 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f89b40cd3a0 msgr2=0x7f89b406c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:36.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.157+0000 7f89b2ffd700 1 --2- 172.21.15.78:0/2861946669 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f89b40cd3a0 0x7f89b406c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:36.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.157+0000 7f89b2ffd700 1 -- 172.21.15.78:0/2861946669 >> v1:172.21.15.133:6789/0 conn(0x7f89b406e490 legacy=0x7f89b4071590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:36.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.157+0000 7f89b2ffd700 1 -- 172.21.15.78:0/2861946669 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b406d050 con 0x7f89b4159830 2023-10-11T09:41:36.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.157+0000 7f89b2ffd700 1 -- 172.21.15.78:0/2861946669 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f89a4001800 con 0x7f89b4159830 2023-10-11T09:41:36.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.161+0000 7f89b2ffd700 1 -- 172.21.15.78:0/2861946669 >> v1:172.21.15.78:6789/0 conn(0x7f89b4159830 legacy=0x7f89b406dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:36.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.161+0000 7f89b2ffd700 1 --2- 172.21.15.78:0/2861946669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89a0003be0 0x7f89a0003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:36.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.161+0000 7f89b2ffd700 1 --2- 172.21.15.78:0/2861946669 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f89b40cd3a0 0x7f89a0004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:36.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.161+0000 7f89b2ffd700 1 --2- 172.21.15.78:0/2861946669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89a0004db0 0x7f89a0005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:36.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.161+0000 7f89b2ffd700 1 -- 172.21.15.78:0/2861946669 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b406cd70 con 0x7f89a0004db0 2023-10-11T09:41:36.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.161+0000 7f89b2ffd700 1 -- 172.21.15.78:0/2861946669 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a000bae0 con 0x7f89a0003be0 2023-10-11T09:41:36.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.161+0000 7f89b2ffd700 1 -- 172.21.15.78:0/2861946669 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a000bc50 con 0x7f89b40cd3a0 2023-10-11T09:41:36.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.161+0000 7f89b3fff700 1 --2- 172.21.15.78:0/2861946669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89a0004db0 0x7f89a0005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:36.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.161+0000 7f89b37fe700 1 --2- 172.21.15.78:0/2861946669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89a0003be0 0x7f89a0003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:36.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.161+0000 7f89abfff700 1 --2- 172.21.15.78:0/2861946669 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f89b40cd3a0 0x7f89a0004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:36.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.161+0000 7f89b3fff700 1 -- 172.21.15.78:0/2861946669 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f89b40cd3a0 msgr2=0x7f89a0004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:36.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.161+0000 7f89b3fff700 1 --2- 172.21.15.78:0/2861946669 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f89b40cd3a0 0x7f89a0004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:36.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.161+0000 7f89b3fff700 1 -- 172.21.15.78:0/2861946669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89a0003be0 msgr2=0x7f89a0003fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:36.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.161+0000 7f89b3fff700 1 --2- 172.21.15.78:0/2861946669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89a0003be0 0x7f89a0003fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:36.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.161+0000 7f89b3fff700 1 -- 172.21.15.78:0/2861946669 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a00035d0 con 0x7f89a0004db0 2023-10-11T09:41:36.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.161+0000 7f89b3fff700 1 --2- 172.21.15.78:0/2861946669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89a0004db0 0x7f89a0005220 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f89ac009fc0 tx=0x7f89a00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2976c98945780a6e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:36.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.161+0000 7f89b2ffd700 1 -- 172.21.15.78:0/2861946669 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac00ebd0 con 0x7f89a0004db0 2023-10-11T09:41:36.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.161+0000 7f89b2ffd700 1 -- 172.21.15.78:0/2861946669 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89ac00f040 con 0x7f89a0004db0 2023-10-11T09:41:36.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.161+0000 7f89b2ffd700 1 -- 172.21.15.78:0/2861946669 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac014480 con 0x7f89a0004db0 2023-10-11T09:41:36.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.161+0000 7f89b9cb5700 1 -- 172.21.15.78:0/2861946669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89a0004db0 msgr2=0x7f89a0005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:36.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.161+0000 7f89b9cb5700 1 --2- 172.21.15.78:0/2861946669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89a0004db0 0x7f89a0005220 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f89ac009fc0 tx=0x7f89a00058a0 comp rx=0 tx=0).stop 2023-10-11T09:41:36.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.161+0000 7f89b3fff700 1 -- 172.21.15.78:0/2861946669 reap_dead start 2023-10-11T09:41:36.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.161+0000 7f89b9cb5700 1 -- 172.21.15.78:0/2861946669 shutdown_connections 2023-10-11T09:41:36.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.161+0000 7f89b9cb5700 1 -- 172.21.15.78:0/2861946669 >> 172.21.15.78:0/2861946669 conn(0x7f89b40b24c0 msgr2=0x7f89b40cc7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:36.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.161+0000 7f89b9cb5700 1 -- 172.21.15.78:0/2861946669 shutdown_connections 2023-10-11T09:41:36.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.161+0000 7f89b9cb5700 1 -- 172.21.15.78:0/2861946669 wait complete. 2023-10-11T09:41:36.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.165+0000 7f89b9cb5700 1 Processor -- start 2023-10-11T09:41:36.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.165+0000 7f89b9cb5700 1 -- start start 2023-10-11T09:41:36.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.165+0000 7f89b9cb5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89a0004db0 0x7f89b40caac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:36.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.165+0000 7f89b9cb5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89b4156e20 0x7f89b40cc710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:36.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.165+0000 7f89b9cb5700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f89b40ccc50 0x7f89b40adbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:36.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.165+0000 7f89b9cb5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b40cd0c0 con 0x7f89a0004db0 2023-10-11T09:41:36.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.165+0000 7f89b9cb5700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b4157290 con 0x7f89b4156e20 2023-10-11T09:41:36.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.165+0000 7f89b9cb5700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89b40ca8a0 con 0x7f89b40ccc50 2023-10-11T09:41:36.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.169+0000 7f89b37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89a0004db0 0x7f89b40caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:36.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.169+0000 7f89b37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89a0004db0 0x7f89b40caac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54318/0 (socket says 172.21.15.78:54318) 2023-10-11T09:41:36.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.169+0000 7f89b37fe700 1 -- 172.21.15.78:0/813238940 learned_addr learned my addr 172.21.15.78:0/813238940 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:36.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.169+0000 7f89abfff700 1 --2- 172.21.15.78:0/813238940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89b4156e20 0x7f89b40cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:36.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.169+0000 7f89b3fff700 1 --2- 172.21.15.78:0/813238940 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f89b40ccc50 0x7f89b40adbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:36.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.169+0000 7f89b37fe700 1 -- 172.21.15.78:0/813238940 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f89b40ccc50 msgr2=0x7f89b40adbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:36.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.169+0000 7f89b37fe700 1 --2- 172.21.15.78:0/813238940 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f89b40ccc50 0x7f89b40adbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:36.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.169+0000 7f89b37fe700 1 -- 172.21.15.78:0/813238940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89b4156e20 msgr2=0x7f89b40cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:36.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.169+0000 7f89b37fe700 1 --2- 172.21.15.78:0/813238940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89b4156e20 0x7f89b40cc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:36.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.169+0000 7f89b37fe700 1 -- 172.21.15.78:0/813238940 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b40ca2e0 con 0x7f89a0004db0 2023-10-11T09:41:36.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.169+0000 7f89b37fe700 1 --2- 172.21.15.78:0/813238940 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89a0004db0 0x7f89b40caac0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f89a400eb20 tx=0x7f89a400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:36.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.169+0000 7f89b0ff9700 1 -- 172.21.15.78:0/813238940 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a4005e80 con 0x7f89a0004db0 2023-10-11T09:41:36.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.169+0000 7f89b0ff9700 1 -- 172.21.15.78:0/813238940 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a4004aa0 con 0x7f89a0004db0 2023-10-11T09:41:36.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.169+0000 7f89b9cb5700 1 -- 172.21.15.78:0/813238940 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89b40ca5c0 con 0x7f89a0004db0 2023-10-11T09:41:36.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.169+0000 7f89b0ff9700 1 -- 172.21.15.78:0/813238940 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a4004e30 con 0x7f89a0004db0 2023-10-11T09:41:36.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.169+0000 7f89b9cb5700 1 -- 172.21.15.78:0/813238940 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b414ad80 con 0x7f89a0004db0 2023-10-11T09:41:36.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.169+0000 7f89b9cb5700 1 -- 172.21.15.78:0/813238940 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89b40ada20 con 0x7f89a0004db0 2023-10-11T09:41:36.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.173+0000 7f89b0ff9700 1 -- 172.21.15.78:0/813238940 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f89a4005050 con 0x7f89a0004db0 2023-10-11T09:41:36.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.173+0000 7f89b0ff9700 1 --2- 172.21.15.78:0/813238940 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8994065ed0 0x7f8994068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:36.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.173+0000 7f89b0ff9700 1 -- 172.21.15.78:0/813238940 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f89a4082ff0 con 0x7f89a0004db0 2023-10-11T09:41:36.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.173+0000 7f89abfff700 1 --2- 172.21.15.78:0/813238940 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8994065ed0 0x7f8994068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:36.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.173+0000 7f89abfff700 1 --2- 172.21.15.78:0/813238940 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8994065ed0 0x7f8994068380 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f899c007fd0 tx=0x7f899c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:36.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.181+0000 7f89b0ff9700 1 -- 172.21.15.78:0/813238940 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f89a4050690 con 0x7f89a0004db0 2023-10-11T09:41:36.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.353+0000 7f89b9cb5700 1 -- 172.21.15.78:0/813238940 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f89b4159c00 con 0x7f89a0004db0 2023-10-11T09:41:36.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.353+0000 7f89b0ff9700 1 -- 172.21.15.78:0/813238940 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f89a4019020 con 0x7f89a0004db0 2023-10-11T09:41:36.361 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:41:36.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.357+0000 7f89b9cb5700 1 -- 172.21.15.78:0/813238940 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8994065ed0 msgr2=0x7f8994068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:36.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.357+0000 7f89b9cb5700 1 --2- 172.21.15.78:0/813238940 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8994065ed0 0x7f8994068380 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f899c007fd0 tx=0x7f899c009040 comp rx=0 tx=0).stop 2023-10-11T09:41:36.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.357+0000 7f89b9cb5700 1 -- 172.21.15.78:0/813238940 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89a0004db0 msgr2=0x7f89b40caac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:36.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.357+0000 7f89b9cb5700 1 --2- 172.21.15.78:0/813238940 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89a0004db0 0x7f89b40caac0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f89a400eb20 tx=0x7f89a400ee30 comp rx=0 tx=0).stop 2023-10-11T09:41:36.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.357+0000 7f89b9cb5700 1 -- 172.21.15.78:0/813238940 shutdown_connections 2023-10-11T09:41:36.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.357+0000 7f89b9cb5700 1 --2- 172.21.15.78:0/813238940 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8994065ed0 0x7f8994068380 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:36.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.357+0000 7f89b9cb5700 1 --2- 172.21.15.78:0/813238940 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f89b40ccc50 0x7f89b40adbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:36.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.357+0000 7f89b9cb5700 1 --2- 172.21.15.78:0/813238940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89b4156e20 0x7f89b40cc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:36.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.357+0000 7f89b9cb5700 1 --2- 172.21.15.78:0/813238940 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89a0004db0 0x7f89b40caac0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:36.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.357+0000 7f89b9cb5700 1 -- 172.21.15.78:0/813238940 >> 172.21.15.78:0/813238940 conn(0x7f89b40b24c0 msgr2=0x7f89b414bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:36.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.361+0000 7f89b9cb5700 1 -- 172.21.15.78:0/813238940 shutdown_connections 2023-10-11T09:41:36.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.361+0000 7f89b9cb5700 1 -- 172.21.15.78:0/813238940 wait complete. 2023-10-11T09:41:36.385 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T09:41:36.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.553+0000 7fe744bfb700 1 Processor -- start 2023-10-11T09:41:36.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.553+0000 7fe744bfb700 1 -- start start 2023-10-11T09:41:36.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.557+0000 7fe744bfb700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe7400b3240 0x7fe7400d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:36.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.557+0000 7fe744bfb700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7400b47f0 con 0x7fe7400b3a90 2023-10-11T09:41:36.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.557+0000 7fe744bfb700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7400b8130 con 0x7fe7400b3610 2023-10-11T09:41:36.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.557+0000 7fe744bfb700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7400b41e0 con 0x7fe7400b3240 2023-10-11T09:41:36.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.557+0000 7fe73ffff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fe7400b3a90 0x7fe7400b85c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33146/0 (socket says 172.21.15.78:33146) 2023-10-11T09:41:36.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.557+0000 7fe73ffff700 1 -- 172.21.15.78:0/110976134 learned_addr learned my addr 172.21.15.78:0/110976134 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:36.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.557+0000 7fe73f7fe700 1 --2- 172.21.15.78:0/110976134 >> v2:172.21.15.133:3300/0 conn(0x7fe7400b3240 0x7fe7400d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:36.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.557+0000 7fe73e7fc700 1 -- 172.21.15.78:0/110976134 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1236061652 0 0) 0x7fe738001490 con 0x7fe7400b3a90 2023-10-11T09:41:36.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.557+0000 7fe73e7fc700 1 -- 172.21.15.78:0/110976134 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7280035d0 con 0x7fe7400b3a90 2023-10-11T09:41:36.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.557+0000 7fe73f7fe700 1 -- 172.21.15.78:0/110976134 >> v1:172.21.15.133:6789/0 conn(0x7fe7400b3610 legacy=0x7fe74015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:36.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.557+0000 7fe73f7fe700 1 -- 172.21.15.78:0/110976134 >> v1:172.21.15.78:6789/0 conn(0x7fe7400b3a90 legacy=0x7fe7400b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:36.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.557+0000 7fe73f7fe700 1 -- 172.21.15.78:0/110976134 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7400b7800 con 0x7fe7400b3240 2023-10-11T09:41:36.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.557+0000 7fe73f7fe700 1 --2- 172.21.15.78:0/110976134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7400b3240 0x7fe7400d29d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe730009850 tx=0x7fe730009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e01eedad9c9434ae server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:36.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.557+0000 7fe73e7fc700 1 -- 172.21.15.78:0/110976134 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe730011830 con 0x7fe7400b3240 2023-10-11T09:41:36.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.557+0000 7fe73e7fc700 1 -- 172.21.15.78:0/110976134 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe730011990 con 0x7fe7400b3240 2023-10-11T09:41:36.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.557+0000 7fe73e7fc700 1 -- 172.21.15.78:0/110976134 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe730011d40 con 0x7fe7400b3240 2023-10-11T09:41:36.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.561+0000 7fe744bfb700 1 -- 172.21.15.78:0/110976134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7400b3240 msgr2=0x7fe7400d29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:36.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.561+0000 7fe744bfb700 1 --2- 172.21.15.78:0/110976134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7400b3240 0x7fe7400d29d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe730009850 tx=0x7fe730009b60 comp rx=0 tx=0).stop 2023-10-11T09:41:36.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.561+0000 7fe744bfb700 1 -- 172.21.15.78:0/110976134 shutdown_connections 2023-10-11T09:41:36.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.561+0000 7fe744bfb700 1 --2- 172.21.15.78:0/110976134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7400b3240 0x7fe7400d29d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:36.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.561+0000 7fe744bfb700 1 -- 172.21.15.78:0/110976134 >> 172.21.15.78:0/110976134 conn(0x7fe74009c260 msgr2=0x7fe7400d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:36.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.561+0000 7fe744bfb700 1 -- 172.21.15.78:0/110976134 shutdown_connections 2023-10-11T09:41:36.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.561+0000 7fe744bfb700 1 -- 172.21.15.78:0/110976134 wait complete. 2023-10-11T09:41:36.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.565+0000 7fe744bfb700 1 Processor -- start 2023-10-11T09:41:36.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.565+0000 7fe744bfb700 1 -- start start 2023-10-11T09:41:36.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.565+0000 7fe744bfb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7400b3240 0x7fe7401f63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:36.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.565+0000 7fe744bfb700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe7400b3610 0x7fe74014e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:36.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.565+0000 7fe744bfb700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7400b3a90 0x7fe7400c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:36.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.565+0000 7fe744bfb700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7401f6900 con 0x7fe7400b3240 2023-10-11T09:41:36.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.565+0000 7fe744bfb700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7401f2760 con 0x7fe7400b3a90 2023-10-11T09:41:36.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.565+0000 7fe744bfb700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7401f2a40 con 0x7fe7400b3610 2023-10-11T09:41:36.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.565+0000 7fe73f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7400b3240 0x7fe7401f63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:36.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.565+0000 7fe73f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7400b3240 0x7fe7401f63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54330/0 (socket says 172.21.15.78:54330) 2023-10-11T09:41:36.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.565+0000 7fe73f7fe700 1 -- 172.21.15.78:0/2572238584 learned_addr learned my addr 172.21.15.78:0/2572238584 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:36.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.565+0000 7fe73ffff700 1 --2- 172.21.15.78:0/2572238584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7400b3a90 0x7fe7400c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:36.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.565+0000 7fe73effd700 1 --2- 172.21.15.78:0/2572238584 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe7400b3610 0x7fe74014e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:36.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.569+0000 7fe73f7fe700 1 -- 172.21.15.78:0/2572238584 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe7400b3610 msgr2=0x7fe74014e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:36.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.569+0000 7fe73f7fe700 1 --2- 172.21.15.78:0/2572238584 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe7400b3610 0x7fe74014e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:36.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.569+0000 7fe73f7fe700 1 -- 172.21.15.78:0/2572238584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7400b3a90 msgr2=0x7fe7400c80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:36.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.569+0000 7fe73f7fe700 1 --2- 172.21.15.78:0/2572238584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7400b3a90 0x7fe7400c80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:36.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.569+0000 7fe73f7fe700 1 -- 172.21.15.78:0/2572238584 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7300094b0 con 0x7fe7400b3240 2023-10-11T09:41:36.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.569+0000 7fe73ffff700 1 --2- 172.21.15.78:0/2572238584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7400b3a90 0x7fe7400c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:36.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.569+0000 7fe73effd700 1 --2- 172.21.15.78:0/2572238584 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe7400b3610 0x7fe74014e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:36.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.569+0000 7fe73f7fe700 1 --2- 172.21.15.78:0/2572238584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7400b3240 0x7fe7401f63c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe730014040 tx=0x7fe730004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:36.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.569+0000 7fe727fff700 1 -- 172.21.15.78:0/2572238584 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe730019ab0 con 0x7fe7400b3240 2023-10-11T09:41:36.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.569+0000 7fe727fff700 1 -- 172.21.15.78:0/2572238584 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe73001e070 con 0x7fe7400b3240 2023-10-11T09:41:36.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.569+0000 7fe727fff700 1 -- 172.21.15.78:0/2572238584 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe730019e50 con 0x7fe7400b3240 2023-10-11T09:41:36.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.569+0000 7fe744bfb700 1 -- 172.21.15.78:0/2572238584 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7401f2d20 con 0x7fe7400b3240 2023-10-11T09:41:36.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.569+0000 7fe744bfb700 1 -- 172.21.15.78:0/2572238584 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe74014e8b0 con 0x7fe7400b3240 2023-10-11T09:41:36.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.569+0000 7fe744bfb700 1 -- 172.21.15.78:0/2572238584 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe740065d40 con 0x7fe7400b3240 2023-10-11T09:41:36.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.573+0000 7fe727fff700 1 -- 172.21.15.78:0/2572238584 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fe73002d450 con 0x7fe7400b3240 2023-10-11T09:41:36.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.573+0000 7fe727fff700 1 --2- 172.21.15.78:0/2572238584 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe72c065ed0 0x7fe72c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:36.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.573+0000 7fe727fff700 1 -- 172.21.15.78:0/2572238584 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fe730084810 con 0x7fe7400b3240 2023-10-11T09:41:36.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.573+0000 7fe73effd700 1 --2- 172.21.15.78:0/2572238584 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe72c065ed0 0x7fe72c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:36.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.573+0000 7fe73effd700 1 --2- 172.21.15.78:0/2572238584 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe72c065ed0 0x7fe72c068380 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe74015b810 tx=0x7fe734009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:36.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.577+0000 7fe727fff700 1 -- 172.21.15.78:0/2572238584 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fe730051eb0 con 0x7fe7400b3240 2023-10-11T09:41:36.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.749+0000 7fe744bfb700 1 -- 172.21.15.78:0/2572238584 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe7400b6dd0 con 0x7fe7400b3240 2023-10-11T09:41:36.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.753+0000 7fe727fff700 1 -- 172.21.15.78:0/2572238584 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe7300150a0 con 0x7fe7400b3240 2023-10-11T09:41:36.757 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2023-10-11T09:41:36.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.753+0000 7fe744bfb700 1 -- 172.21.15.78:0/2572238584 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe72c065ed0 msgr2=0x7fe72c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:36.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.753+0000 7fe744bfb700 1 --2- 172.21.15.78:0/2572238584 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe72c065ed0 0x7fe72c068380 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe74015b810 tx=0x7fe734009040 comp rx=0 tx=0).stop 2023-10-11T09:41:36.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.753+0000 7fe744bfb700 1 -- 172.21.15.78:0/2572238584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7400b3240 msgr2=0x7fe7401f63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:36.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.753+0000 7fe744bfb700 1 --2- 172.21.15.78:0/2572238584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7400b3240 0x7fe7401f63c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe730014040 tx=0x7fe730004770 comp rx=0 tx=0).stop 2023-10-11T09:41:36.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.753+0000 7fe744bfb700 1 -- 172.21.15.78:0/2572238584 shutdown_connections 2023-10-11T09:41:36.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.753+0000 7fe744bfb700 1 --2- 172.21.15.78:0/2572238584 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe72c065ed0 0x7fe72c068380 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:36.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.753+0000 7fe744bfb700 1 --2- 172.21.15.78:0/2572238584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7400b3a90 0x7fe7400c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:36.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.753+0000 7fe744bfb700 1 --2- 172.21.15.78:0/2572238584 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe7400b3610 0x7fe74014e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:36.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.753+0000 7fe744bfb700 1 --2- 172.21.15.78:0/2572238584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7400b3240 0x7fe7401f63c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:36.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.753+0000 7fe744bfb700 1 -- 172.21.15.78:0/2572238584 >> 172.21.15.78:0/2572238584 conn(0x7fe74009c260 msgr2=0x7fe74006b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:36.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.753+0000 7fe744bfb700 1 -- 172.21.15.78:0/2572238584 shutdown_connections 2023-10-11T09:41:36.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.753+0000 7fe744bfb700 1 -- 172.21.15.78:0/2572238584 wait complete. 2023-10-11T09:41:36.780 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T09:41:36.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.941+0000 7f5de560a700 1 Processor -- start 2023-10-11T09:41:36.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.941+0000 7f5de560a700 1 -- start start 2023-10-11T09:41:36.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.941+0000 7f5de560a700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5de00d1660 0x7f5de00d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:36.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.941+0000 7f5de560a700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5de00d28e0 0x7f5de00d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:36.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.941+0000 7f5de560a700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5de015dc10 con 0x7f5de0156ce0 2023-10-11T09:41:36.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.941+0000 7f5de560a700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5de0160380 con 0x7f5de00d28e0 2023-10-11T09:41:36.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.941+0000 7f5de560a700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5de015d930 con 0x7f5de00d1660 2023-10-11T09:41:36.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.945+0000 7f5ddeffd700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f5de0156ce0 0x7f5de0160b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33150/0 (socket says 172.21.15.78:33150) 2023-10-11T09:41:36.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.945+0000 7f5ddeffd700 1 -- 172.21.15.78:0/2150801067 learned_addr learned my addr 172.21.15.78:0/2150801067 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:36.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.945+0000 7f5ddf7fe700 1 --2- 172.21.15.78:0/2150801067 >> v2:172.21.15.133:3300/0 conn(0x7f5de00d28e0 0x7f5de00d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:36.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.945+0000 7f5dde7fc700 1 --2- 172.21.15.78:0/2150801067 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5de00d1660 0x7f5de00d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:36.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.945+0000 7f5dddffb700 1 -- 172.21.15.78:0/2150801067 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3784528626 0 0) 0x7f5de015dc10 con 0x7f5de0156ce0 2023-10-11T09:41:36.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.945+0000 7f5dddffb700 1 -- 172.21.15.78:0/2150801067 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5dc80035d0 con 0x7f5de0156ce0 2023-10-11T09:41:36.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.945+0000 7f5ddf7fe700 1 -- 172.21.15.78:0/2150801067 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5de00d1660 msgr2=0x7f5de00d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:36.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.945+0000 7f5ddf7fe700 1 --2- 172.21.15.78:0/2150801067 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5de00d1660 0x7f5de00d0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:36.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.945+0000 7f5ddf7fe700 1 -- 172.21.15.78:0/2150801067 >> v1:172.21.15.78:6789/0 conn(0x7f5de0156ce0 legacy=0x7f5de0160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:36.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.945+0000 7f5ddf7fe700 1 -- 172.21.15.78:0/2150801067 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de0160620 con 0x7f5de00d28e0 2023-10-11T09:41:36.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.945+0000 7f5ddf7fe700 1 --2- 172.21.15.78:0/2150801067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5de00d28e0 0x7f5de00d0f90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009d90 tx=0x7f5dd800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b4e125ab7f7f818 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:36.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.945+0000 7f5dddffb700 1 -- 172.21.15.78:0/2150801067 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd800e070 con 0x7f5de00d28e0 2023-10-11T09:41:36.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.945+0000 7f5dddffb700 1 -- 172.21.15.78:0/2150801067 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd800bb20 con 0x7f5de00d28e0 2023-10-11T09:41:36.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.945+0000 7f5dddffb700 1 -- 172.21.15.78:0/2150801067 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8012430 con 0x7f5de00d28e0 2023-10-11T09:41:36.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.945+0000 7f5de560a700 1 -- 172.21.15.78:0/2150801067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5de00d28e0 msgr2=0x7f5de00d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:36.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.945+0000 7f5de560a700 1 --2- 172.21.15.78:0/2150801067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5de00d28e0 0x7f5de00d0f90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009d90 tx=0x7f5dd800b330 comp rx=0 tx=0).stop 2023-10-11T09:41:36.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.949+0000 7f5de560a700 1 -- 172.21.15.78:0/2150801067 shutdown_connections 2023-10-11T09:41:36.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.949+0000 7f5de560a700 1 --2- 172.21.15.78:0/2150801067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5de00d28e0 0x7f5de00d0f90 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:36.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.949+0000 7f5de560a700 1 --2- 172.21.15.78:0/2150801067 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5de00d1660 0x7f5de00d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:36.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.949+0000 7f5de560a700 1 -- 172.21.15.78:0/2150801067 >> 172.21.15.78:0/2150801067 conn(0x7f5de006afb0 msgr2=0x7f5de006b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:36.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.949+0000 7f5de560a700 1 -- 172.21.15.78:0/2150801067 shutdown_connections 2023-10-11T09:41:36.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.949+0000 7f5de560a700 1 -- 172.21.15.78:0/2150801067 wait complete. 2023-10-11T09:41:36.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.949+0000 7f5de560a700 1 Processor -- start 2023-10-11T09:41:36.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.949+0000 7f5de560a700 1 -- start start 2023-10-11T09:41:36.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.953+0000 7f5de560a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5de00d1660 0x7f5de00c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:36.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.953+0000 7f5de560a700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5de00d28e0 0x7f5de00cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:36.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.953+0000 7f5de560a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5de0156ce0 0x7f5de00cac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:36.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.953+0000 7f5de560a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de00c62b0 con 0x7f5de00d1660 2023-10-11T09:41:36.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.953+0000 7f5de560a700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de00c9f50 con 0x7f5de0156ce0 2023-10-11T09:41:36.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.953+0000 7f5de560a700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5de00cc420 con 0x7f5de00d28e0 2023-10-11T09:41:36.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.953+0000 7f5ddeffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5de00d1660 0x7f5de00c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:36.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.953+0000 7f5ddeffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5de00d1660 0x7f5de00c5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54336/0 (socket says 172.21.15.78:54336) 2023-10-11T09:41:36.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.953+0000 7f5ddeffd700 1 -- 172.21.15.78:0/2341689546 learned_addr learned my addr 172.21.15.78:0/2341689546 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:36.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.953+0000 7f5ddf7fe700 1 --2- 172.21.15.78:0/2341689546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5de0156ce0 0x7f5de00cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:36.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.953+0000 7f5dde7fc700 1 --2- 172.21.15.78:0/2341689546 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5de00d28e0 0x7f5de00cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:36.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.953+0000 7f5ddf7fe700 1 -- 172.21.15.78:0/2341689546 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5de00d28e0 msgr2=0x7f5de00cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:36.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.953+0000 7f5ddf7fe700 1 --2- 172.21.15.78:0/2341689546 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5de00d28e0 0x7f5de00cbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:36.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.953+0000 7f5ddf7fe700 1 -- 172.21.15.78:0/2341689546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5de00d1660 msgr2=0x7f5de00c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:36.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.953+0000 7f5ddf7fe700 1 --2- 172.21.15.78:0/2341689546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5de00d1660 0x7f5de00c5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:36.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.953+0000 7f5ddf7fe700 1 -- 172.21.15.78:0/2341689546 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd8009a70 con 0x7f5de0156ce0 2023-10-11T09:41:36.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.953+0000 7f5ddeffd700 1 --2- 172.21.15.78:0/2341689546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5de00d1660 0x7f5de00c5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:41:36.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.953+0000 7f5ddf7fe700 1 --2- 172.21.15.78:0/2341689546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5de0156ce0 0x7f5de00cac80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8016000 tx=0x7f5dd8004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:36.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.953+0000 7f5dc77fe700 1 -- 172.21.15.78:0/2341689546 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8012430 con 0x7f5de0156ce0 2023-10-11T09:41:36.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.953+0000 7f5dc77fe700 1 -- 172.21.15.78:0/2341689546 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd8020070 con 0x7f5de0156ce0 2023-10-11T09:41:36.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.953+0000 7f5dc77fe700 1 -- 172.21.15.78:0/2341689546 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd80098f0 con 0x7f5de0156ce0 2023-10-11T09:41:36.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.953+0000 7f5de560a700 1 -- 172.21.15.78:0/2341689546 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5de00ca1a0 con 0x7f5de0156ce0 2023-10-11T09:41:36.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.953+0000 7f5de560a700 1 -- 172.21.15.78:0/2341689546 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5de00ca760 con 0x7f5de0156ce0 2023-10-11T09:41:36.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.957+0000 7f5de560a700 1 -- 172.21.15.78:0/2341689546 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5de0065d40 con 0x7f5de0156ce0 2023-10-11T09:41:36.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.957+0000 7f5dc77fe700 1 -- 172.21.15.78:0/2341689546 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f5dd8004040 con 0x7f5de0156ce0 2023-10-11T09:41:36.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.961+0000 7f5dc77fe700 1 --2- 172.21.15.78:0/2341689546 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5dcc065ed0 0x7f5dcc068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:36.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.961+0000 7f5dc77fe700 1 -- 172.21.15.78:0/2341689546 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f5dd8022070 con 0x7f5de0156ce0 2023-10-11T09:41:36.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.961+0000 7f5ddeffd700 1 --2- 172.21.15.78:0/2341689546 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5dcc065ed0 0x7f5dcc068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:36.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.961+0000 7f5ddeffd700 1 --2- 172.21.15.78:0/2341689546 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5dcc065ed0 0x7f5dcc068380 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5dd00011e0 tx=0x7f5dd0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:36.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:36.969+0000 7f5dc77fe700 1 -- 172.21.15.78:0/2341689546 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f5dd8052ff0 con 0x7f5de0156ce0 2023-10-11T09:41:37.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.149+0000 7f5de560a700 1 -- 172.21.15.78:0/2341689546 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5de006d310 con 0x7f5de0156ce0 2023-10-11T09:41:37.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.153+0000 7f5dc77fe700 1 -- 172.21.15.78:0/2341689546 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5dd802c020 con 0x7f5de0156ce0 2023-10-11T09:41:37.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.153+0000 7f5de560a700 1 -- 172.21.15.78:0/2341689546 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5dcc065ed0 msgr2=0x7f5dcc068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:37.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.153+0000 7f5de560a700 1 --2- 172.21.15.78:0/2341689546 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5dcc065ed0 0x7f5dcc068380 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5dd00011e0 tx=0x7f5dd0009040 comp rx=0 tx=0).stop 2023-10-11T09:41:37.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.153+0000 7f5de560a700 1 -- 172.21.15.78:0/2341689546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5de0156ce0 msgr2=0x7f5de00cac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:37.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.153+0000 7f5de560a700 1 --2- 172.21.15.78:0/2341689546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5de0156ce0 0x7f5de00cac80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8016000 tx=0x7f5dd8004af0 comp rx=0 tx=0).stop 2023-10-11T09:41:37.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.153+0000 7f5de560a700 1 -- 172.21.15.78:0/2341689546 shutdown_connections 2023-10-11T09:41:37.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.153+0000 7f5de560a700 1 --2- 172.21.15.78:0/2341689546 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5dcc065ed0 0x7f5dcc068380 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:37.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.153+0000 7f5de560a700 1 --2- 172.21.15.78:0/2341689546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5de0156ce0 0x7f5de00cac80 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:37.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.153+0000 7f5de560a700 1 --2- 172.21.15.78:0/2341689546 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5de00d28e0 0x7f5de00cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:37.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.153+0000 7f5de560a700 1 --2- 172.21.15.78:0/2341689546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5de00d1660 0x7f5de00c5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:37.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.153+0000 7f5de560a700 1 -- 172.21.15.78:0/2341689546 >> 172.21.15.78:0/2341689546 conn(0x7f5de006afb0 msgr2=0x7f5de00b8030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:37.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.153+0000 7f5de560a700 1 -- 172.21.15.78:0/2341689546 shutdown_connections 2023-10-11T09:41:37.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.157+0000 7f5de560a700 1 -- 172.21.15.78:0/2341689546 wait complete. 2023-10-11T09:41:37.163 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2023-10-11T09:41:37.181 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi078.front.sepia.ceph.com... 2023-10-11T09:41:37.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:37.182 DEBUG:teuthology.orchestra.run.smithi078:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-10-11T09:41:37.215 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:41:37.215 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2023-10-11T09:41:37.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:37.216 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-11T09:41:37.276 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:41:37.277 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:41:37.297 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:41:37.298 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:41:37.359 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:41:37.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.529+0000 7f40e7199700 1 Processor -- start 2023-10-11T09:41:37.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.529+0000 7f40e7199700 1 -- start start 2023-10-11T09:41:37.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.533+0000 7f40e7199700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f40e0154e00 0x7f40e015b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:37.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.533+0000 7f40e7199700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40e0071040 0x7f40e00cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:37.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.533+0000 7f40e7199700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f40e00cba00 0x7f40e015e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:37.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.533+0000 7f40e7199700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40e00720f0 con 0x7f40e00cba00 2023-10-11T09:41:37.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.533+0000 7f40e7199700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40e015bb90 con 0x7f40e0154e00 2023-10-11T09:41:37.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.533+0000 7f40e7199700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e00714e0 con 0x7f40e0071040 2023-10-11T09:41:37.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.533+0000 7f40e5736700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f40e00cba00 0x7f40e015e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:37.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.533+0000 7f40e5736700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f40e00cba00 0x7f40e015e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54340/0 (socket says 172.21.15.78:54340) 2023-10-11T09:41:37.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.533+0000 7f40e4f35700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f40e0154e00 0x7f40e015b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:37.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.533+0000 7f40e5736700 1 -- 172.21.15.78:0/2971518103 learned_addr learned my addr 172.21.15.78:0/2971518103 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:37.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.533+0000 7f40dffff700 1 --2- 172.21.15.78:0/2971518103 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40e0071040 0x7f40e00cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:37.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.533+0000 7f40e5736700 1 -- 172.21.15.78:0/2971518103 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40e0071040 msgr2=0x7f40e00cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:37.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.533+0000 7f40e5736700 1 --2- 172.21.15.78:0/2971518103 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40e0071040 0x7f40e00cb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:37.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.533+0000 7f40e5736700 1 -- 172.21.15.78:0/2971518103 >> v2:172.21.15.133:3300/0 conn(0x7f40e0154e00 msgr2=0x7f40e015b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:37.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.533+0000 7f40e5736700 1 --2- 172.21.15.78:0/2971518103 >> v2:172.21.15.133:3300/0 conn(0x7f40e0154e00 0x7f40e015b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:37.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.533+0000 7f40e5736700 1 -- 172.21.15.78:0/2971518103 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e00717c0 con 0x7f40e00cba00 2023-10-11T09:41:37.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.537+0000 7f40e5736700 1 --2- 172.21.15.78:0/2971518103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40e00cba00 0x7f40e015e1c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f40e006f7c0 tx=0x7f40d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44cd08e4cefa8c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:37.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.537+0000 7f40df7fe700 1 -- 172.21.15.78:0/2971518103 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d800e070 con 0x7f40e00cba00 2023-10-11T09:41:37.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.537+0000 7f40df7fe700 1 -- 172.21.15.78:0/2971518103 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d800b910 con 0x7f40e00cba00 2023-10-11T09:41:37.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.537+0000 7f40df7fe700 1 -- 172.21.15.78:0/2971518103 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d800bcc0 con 0x7f40e00cba00 2023-10-11T09:41:37.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.537+0000 7f40e7199700 1 -- 172.21.15.78:0/2971518103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40e00cba00 msgr2=0x7f40e015e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:37.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.537+0000 7f40e7199700 1 --2- 172.21.15.78:0/2971518103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40e00cba00 0x7f40e015e1c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f40e006f7c0 tx=0x7f40d800b330 comp rx=0 tx=0).stop 2023-10-11T09:41:37.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.537+0000 7f40e7199700 1 -- 172.21.15.78:0/2971518103 shutdown_connections 2023-10-11T09:41:37.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.537+0000 7f40e7199700 1 --2- 172.21.15.78:0/2971518103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40e00cba00 0x7f40e015e1c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:37.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.537+0000 7f40e7199700 1 --2- 172.21.15.78:0/2971518103 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40e0071040 0x7f40e00cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:37.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.537+0000 7f40e7199700 1 --2- 172.21.15.78:0/2971518103 >> v2:172.21.15.133:3300/0 conn(0x7f40e0154e00 0x7f40e015b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:37.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.537+0000 7f40e7199700 1 -- 172.21.15.78:0/2971518103 >> 172.21.15.78:0/2971518103 conn(0x7f40e006aca0 msgr2=0x7f40e00b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:37.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.537+0000 7f40e7199700 1 -- 172.21.15.78:0/2971518103 shutdown_connections 2023-10-11T09:41:37.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.537+0000 7f40e7199700 1 -- 172.21.15.78:0/2971518103 wait complete. 2023-10-11T09:41:37.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.541+0000 7f40e7199700 1 Processor -- start 2023-10-11T09:41:37.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.541+0000 7f40e7199700 1 -- start start 2023-10-11T09:41:37.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.541+0000 7f40e7199700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40e0071040 0x7f40e00ca190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:37.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.541+0000 7f40e7199700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40e00cba00 0x7f40e01f1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:37.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.541+0000 7f40e7199700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40e0154e00 0x7f40e00b2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:37.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.541+0000 7f40e7199700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e00b0a80 con 0x7f40e00cba00 2023-10-11T09:41:37.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.541+0000 7f40e7199700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e00b1e30 con 0x7f40e0071040 2023-10-11T09:41:37.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.541+0000 7f40e7199700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e00b20d0 con 0x7f40e0154e00 2023-10-11T09:41:37.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.545+0000 7f40dffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40e00cba00 0x7f40e01f1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:37.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.545+0000 7f40dffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40e00cba00 0x7f40e01f1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54354/0 (socket says 172.21.15.78:54354) 2023-10-11T09:41:37.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.545+0000 7f40dffff700 1 -- 172.21.15.78:0/3346692354 learned_addr learned my addr 172.21.15.78:0/3346692354 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:37.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.545+0000 7f40e5736700 1 --2- 172.21.15.78:0/3346692354 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40e0154e00 0x7f40e00b2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:37.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.545+0000 7f40dffff700 1 -- 172.21.15.78:0/3346692354 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40e0154e00 msgr2=0x7f40e00b2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:37.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.545+0000 7f40e4f35700 1 --2- 172.21.15.78:0/3346692354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40e0071040 0x7f40e00ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:37.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.545+0000 7f40dffff700 1 --2- 172.21.15.78:0/3346692354 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40e0154e00 0x7f40e00b2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:37.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.545+0000 7f40dffff700 1 -- 172.21.15.78:0/3346692354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40e0071040 msgr2=0x7f40e00ca190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:37.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.545+0000 7f40dffff700 1 --2- 172.21.15.78:0/3346692354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40e0071040 0x7f40e00ca190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:37.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.545+0000 7f40dffff700 1 -- 172.21.15.78:0/3346692354 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d8009b30 con 0x7f40e00cba00 2023-10-11T09:41:37.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.545+0000 7f40e5736700 1 --2- 172.21.15.78:0/3346692354 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40e0154e00 0x7f40e00b2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:41:37.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.545+0000 7f40e4f35700 1 --2- 172.21.15.78:0/3346692354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40e0071040 0x7f40e00ca190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:41:37.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.545+0000 7f40dffff700 1 --2- 172.21.15.78:0/3346692354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40e00cba00 0x7f40e01f1200 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f40cc00e900 tx=0x7f40cc00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:37.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.545+0000 7f40dd7fa700 1 -- 172.21.15.78:0/3346692354 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc009660 con 0x7f40e00cba00 2023-10-11T09:41:37.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.545+0000 7f40dd7fa700 1 -- 172.21.15.78:0/3346692354 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40cc00c850 con 0x7f40e00cba00 2023-10-11T09:41:37.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.545+0000 7f40e7199700 1 -- 172.21.15.78:0/3346692354 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40e00b23b0 con 0x7f40e00cba00 2023-10-11T09:41:37.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.545+0000 7f40dd7fa700 1 -- 172.21.15.78:0/3346692354 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc00cbe0 con 0x7f40e00cba00 2023-10-11T09:41:37.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.545+0000 7f40e7199700 1 -- 172.21.15.78:0/3346692354 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40e00ca6d0 con 0x7f40e00cba00 2023-10-11T09:41:37.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.545+0000 7f40e7199700 1 -- 172.21.15.78:0/3346692354 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40e00b28d0 con 0x7f40e00cba00 2023-10-11T09:41:37.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.549+0000 7f40dd7fa700 1 -- 172.21.15.78:0/3346692354 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f40cc00cd40 con 0x7f40e00cba00 2023-10-11T09:41:37.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.549+0000 7f40dd7fa700 1 --2- 172.21.15.78:0/3346692354 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f40c4065ed0 0x7f40c4068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:37.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.549+0000 7f40dd7fa700 1 -- 172.21.15.78:0/3346692354 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f40cc083950 con 0x7f40e00cba00 2023-10-11T09:41:37.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.549+0000 7f40e4f35700 1 --2- 172.21.15.78:0/3346692354 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f40c4065ed0 0x7f40c4068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:37.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.553+0000 7f40e4f35700 1 --2- 172.21.15.78:0/3346692354 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f40c4065ed0 0x7f40c4068380 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f40e015e430 tx=0x7f40d4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:37.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.557+0000 7f40dd7fa700 1 -- 172.21.15.78:0/3346692354 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f40e00b28d0 con 0x7f40e00cba00 2023-10-11T09:41:37.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.741+0000 7f40e7199700 1 -- 172.21.15.78:0/3346692354 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40e00b28d0 con 0x7f40e00cba00 2023-10-11T09:41:37.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.741+0000 7f40dd7fa700 1 -- 172.21.15.78:0/3346692354 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17907 (secure 0 0 0) 0x7f40e00b28d0 con 0x7f40e00cba00 2023-10-11T09:41:37.748 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:41:37.749 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":25,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:41:21.473566+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:40:39.862581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T09:40:40.663641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.37","mappings":[{"from":3,"to":2}]},{"pgid":"4.3e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:41:37.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.749+0000 7f40e7199700 1 -- 172.21.15.78:0/3346692354 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f40c4065ed0 msgr2=0x7f40c4068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:37.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.749+0000 7f40e7199700 1 --2- 172.21.15.78:0/3346692354 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f40c4065ed0 0x7f40c4068380 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f40e015e430 tx=0x7f40d4008040 comp rx=0 tx=0).stop 2023-10-11T09:41:37.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.749+0000 7f40e7199700 1 -- 172.21.15.78:0/3346692354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40e00cba00 msgr2=0x7f40e01f1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:37.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.749+0000 7f40e7199700 1 --2- 172.21.15.78:0/3346692354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40e00cba00 0x7f40e01f1200 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f40cc00e900 tx=0x7f40cc00ecc0 comp rx=0 tx=0).stop 2023-10-11T09:41:37.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.749+0000 7f40e7199700 1 -- 172.21.15.78:0/3346692354 shutdown_connections 2023-10-11T09:41:37.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.749+0000 7f40e7199700 1 --2- 172.21.15.78:0/3346692354 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f40c4065ed0 0x7f40c4068380 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:37.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.749+0000 7f40e7199700 1 --2- 172.21.15.78:0/3346692354 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40e0154e00 0x7f40e00b2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:37.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.749+0000 7f40e7199700 1 --2- 172.21.15.78:0/3346692354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40e00cba00 0x7f40e01f1200 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:37.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.749+0000 7f40e7199700 1 --2- 172.21.15.78:0/3346692354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40e0071040 0x7f40e00ca190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:37.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.749+0000 7f40e7199700 1 -- 172.21.15.78:0/3346692354 >> 172.21.15.78:0/3346692354 conn(0x7f40e006aca0 msgr2=0x7f40e015e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:37.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.749+0000 7f40e7199700 1 -- 172.21.15.78:0/3346692354 shutdown_connections 2023-10-11T09:41:37.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.749+0000 7f40e7199700 1 -- 172.21.15.78:0/3346692354 wait complete. 2023-10-11T09:41:37.775 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T09:40:39.862581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T09:40:40.663641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:41:37.775 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:41:37.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.945+0000 7fa692ae4700 1 Processor -- start 2023-10-11T09:41:37.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.945+0000 7fa692ae4700 1 -- start start 2023-10-11T09:41:37.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.949+0000 7fa692ae4700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa68c0b3240 0x7fa68c0d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:37.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.949+0000 7fa692ae4700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa68c0b47f0 con 0x7fa68c0b3a90 2023-10-11T09:41:37.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.949+0000 7fa692ae4700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa68c0b8130 con 0x7fa68c0b3610 2023-10-11T09:41:37.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.949+0000 7fa692ae4700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa68c0b41e0 con 0x7fa68c0b3240 2023-10-11T09:41:37.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.949+0000 7fa691ae2700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa68c0b3240 0x7fa68c0d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:37.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.949+0000 7fa6922e3700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fa68c0b3a90 0x7fa68c0b85c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33158/0 (socket says 172.21.15.78:33158) 2023-10-11T09:41:37.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.949+0000 7fa6922e3700 1 -- 172.21.15.78:0/264907446 learned_addr learned my addr 172.21.15.78:0/264907446 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:37.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.949+0000 7fa690ae0700 1 -- 172.21.15.78:0/264907446 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3605152499 0 0) 0x7fa684001110 con 0x7fa68c0b3a90 2023-10-11T09:41:37.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.949+0000 7fa690ae0700 1 -- 172.21.15.78:0/264907446 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6780035d0 con 0x7fa68c0b3a90 2023-10-11T09:41:37.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.949+0000 7fa691ae2700 1 -- 172.21.15.78:0/264907446 >> v1:172.21.15.133:6789/0 conn(0x7fa68c0b3610 legacy=0x7fa68c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:37.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.949+0000 7fa691ae2700 1 -- 172.21.15.78:0/264907446 >> v1:172.21.15.78:6789/0 conn(0x7fa68c0b3a90 legacy=0x7fa68c0b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:37.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.949+0000 7fa691ae2700 1 -- 172.21.15.78:0/264907446 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa68c0b7800 con 0x7fa68c0b3240 2023-10-11T09:41:37.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.949+0000 7fa691ae2700 1 --2- 172.21.15.78:0/264907446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa68c0b3240 0x7fa68c0d29d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa688009d90 tx=0x7fa68800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c003ac6b62530e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:37.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.949+0000 7fa690ae0700 1 -- 172.21.15.78:0/264907446 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68800e070 con 0x7fa68c0b3240 2023-10-11T09:41:37.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.949+0000 7fa690ae0700 1 -- 172.21.15.78:0/264907446 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa68800bb20 con 0x7fa68c0b3240 2023-10-11T09:41:37.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.949+0000 7fa690ae0700 1 -- 172.21.15.78:0/264907446 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68800be90 con 0x7fa68c0b3240 2023-10-11T09:41:37.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.953+0000 7fa692ae4700 1 -- 172.21.15.78:0/264907446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa68c0b3240 msgr2=0x7fa68c0d29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:37.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.953+0000 7fa692ae4700 1 --2- 172.21.15.78:0/264907446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa68c0b3240 0x7fa68c0d29d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa688009d90 tx=0x7fa68800b330 comp rx=0 tx=0).stop 2023-10-11T09:41:37.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.953+0000 7fa692ae4700 1 -- 172.21.15.78:0/264907446 shutdown_connections 2023-10-11T09:41:37.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.953+0000 7fa692ae4700 1 --2- 172.21.15.78:0/264907446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa68c0b3240 0x7fa68c0d29d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:37.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.953+0000 7fa692ae4700 1 -- 172.21.15.78:0/264907446 >> 172.21.15.78:0/264907446 conn(0x7fa68c09c260 msgr2=0x7fa68c0d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:37.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.953+0000 7fa692ae4700 1 -- 172.21.15.78:0/264907446 shutdown_connections 2023-10-11T09:41:37.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.953+0000 7fa692ae4700 1 -- 172.21.15.78:0/264907446 wait complete. 2023-10-11T09:41:37.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.953+0000 7fa692ae4700 1 Processor -- start 2023-10-11T09:41:37.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.957+0000 7fa692ae4700 1 -- start start 2023-10-11T09:41:37.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.957+0000 7fa692ae4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa68c0b3240 0x7fa68c0c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:37.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.957+0000 7fa692ae4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa68c0b3610 0x7fa68c0c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:37.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.957+0000 7fa692ae4700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa68c0b3a90 0x7fa68c1f5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:37.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.957+0000 7fa692ae4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68c09d080 con 0x7fa68c0b3610 2023-10-11T09:41:37.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.957+0000 7fa692ae4700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68c0c92f0 con 0x7fa68c0b3240 2023-10-11T09:41:37.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.957+0000 7fa692ae4700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa68c0c7de0 con 0x7fa68c0b3a90 2023-10-11T09:41:37.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.957+0000 7fa6912e1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa68c0b3610 0x7fa68c0c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:37.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.957+0000 7fa6912e1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa68c0b3610 0x7fa68c0c6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54362/0 (socket says 172.21.15.78:54362) 2023-10-11T09:41:37.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.957+0000 7fa6912e1700 1 -- 172.21.15.78:0/3967821847 learned_addr learned my addr 172.21.15.78:0/3967821847 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:37.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.957+0000 7fa691ae2700 1 --2- 172.21.15.78:0/3967821847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa68c0b3240 0x7fa68c0c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:37.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.957+0000 7fa6922e3700 1 --2- 172.21.15.78:0/3967821847 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa68c0b3a90 0x7fa68c1f5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:37.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.957+0000 7fa6912e1700 1 -- 172.21.15.78:0/3967821847 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa68c0b3a90 msgr2=0x7fa68c1f5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:37.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.957+0000 7fa6912e1700 1 --2- 172.21.15.78:0/3967821847 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa68c0b3a90 0x7fa68c1f5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:37.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.957+0000 7fa6912e1700 1 -- 172.21.15.78:0/3967821847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa68c0b3240 msgr2=0x7fa68c0c77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:37.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.957+0000 7fa6912e1700 1 --2- 172.21.15.78:0/3967821847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa68c0b3240 0x7fa68c0c77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:37.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.957+0000 7fa6912e1700 1 -- 172.21.15.78:0/3967821847 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa688009a70 con 0x7fa68c0b3610 2023-10-11T09:41:37.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.957+0000 7fa6912e1700 1 --2- 172.21.15.78:0/3967821847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa68c0b3610 0x7fa68c0c6ac0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa67c00c440 tx=0x7fa67c00c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:37.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.957+0000 7fa6827fc700 1 -- 172.21.15.78:0/3967821847 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67c0052d0 con 0x7fa68c0b3610 2023-10-11T09:41:37.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.957+0000 7fa6827fc700 1 -- 172.21.15.78:0/3967821847 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa67c005430 con 0x7fa68c0b3610 2023-10-11T09:41:37.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.957+0000 7fa692ae4700 1 -- 172.21.15.78:0/3967821847 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa68c0c9560 con 0x7fa68c0b3610 2023-10-11T09:41:37.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.957+0000 7fa6827fc700 1 -- 172.21.15.78:0/3967821847 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67c0055a0 con 0x7fa68c0b3610 2023-10-11T09:41:37.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.961+0000 7fa692ae4700 1 -- 172.21.15.78:0/3967821847 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa68c0c9840 con 0x7fa68c0b3610 2023-10-11T09:41:37.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.961+0000 7fa692ae4700 1 -- 172.21.15.78:0/3967821847 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa68c0c7150 con 0x7fa68c0b3610 2023-10-11T09:41:37.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.961+0000 7fa6827fc700 1 -- 172.21.15.78:0/3967821847 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fa67c0036a0 con 0x7fa68c0b3610 2023-10-11T09:41:37.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.961+0000 7fa6827fc700 1 --2- 172.21.15.78:0/3967821847 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa670065ed0 0x7fa670068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:37.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.961+0000 7fa6827fc700 1 -- 172.21.15.78:0/3967821847 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa67c082ae0 con 0x7fa68c0b3610 2023-10-11T09:41:37.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.965+0000 7fa691ae2700 1 --2- 172.21.15.78:0/3967821847 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa670065ed0 0x7fa670068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:37.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.965+0000 7fa691ae2700 1 --2- 172.21.15.78:0/3967821847 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa670065ed0 0x7fa670068380 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa68800b370 tx=0x7fa688011000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:37.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:37.969+0000 7fa6827fc700 1 -- 172.21.15.78:0/3967821847 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fa67c04ffd0 con 0x7fa68c0b3610 2023-10-11T09:41:38.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.141+0000 7fa692ae4700 1 -- 172.21.15.78:0/3967821847 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa68c06cf50 con 0x7fa68c0b3610 2023-10-11T09:41:38.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.141+0000 7fa6827fc700 1 -- 172.21.15.78:0/3967821847 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa67c0535f0 con 0x7fa68c0b3610 2023-10-11T09:41:38.148 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:41:38.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.145+0000 7fa692ae4700 1 -- 172.21.15.78:0/3967821847 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa670065ed0 msgr2=0x7fa670068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:38.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.145+0000 7fa692ae4700 1 --2- 172.21.15.78:0/3967821847 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa670065ed0 0x7fa670068380 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa68800b370 tx=0x7fa688011000 comp rx=0 tx=0).stop 2023-10-11T09:41:38.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.145+0000 7fa692ae4700 1 -- 172.21.15.78:0/3967821847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa68c0b3610 msgr2=0x7fa68c0c6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:38.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.145+0000 7fa692ae4700 1 --2- 172.21.15.78:0/3967821847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa68c0b3610 0x7fa68c0c6ac0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa67c00c440 tx=0x7fa67c00c750 comp rx=0 tx=0).stop 2023-10-11T09:41:38.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.145+0000 7fa692ae4700 1 -- 172.21.15.78:0/3967821847 shutdown_connections 2023-10-11T09:41:38.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.145+0000 7fa692ae4700 1 --2- 172.21.15.78:0/3967821847 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa670065ed0 0x7fa670068380 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:38.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.145+0000 7fa692ae4700 1 --2- 172.21.15.78:0/3967821847 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa68c0b3a90 0x7fa68c1f5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:38.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.145+0000 7fa692ae4700 1 --2- 172.21.15.78:0/3967821847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa68c0b3610 0x7fa68c0c6ac0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:38.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.145+0000 7fa692ae4700 1 --2- 172.21.15.78:0/3967821847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa68c0b3240 0x7fa68c0c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:38.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.145+0000 7fa692ae4700 1 -- 172.21.15.78:0/3967821847 >> 172.21.15.78:0/3967821847 conn(0x7fa68c09c260 msgr2=0x7fa68c0d0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:38.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.145+0000 7fa692ae4700 1 -- 172.21.15.78:0/3967821847 shutdown_connections 2023-10-11T09:41:38.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.145+0000 7fa692ae4700 1 -- 172.21.15.78:0/3967821847 wait complete. 2023-10-11T09:41:38.172 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:41:38.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.337+0000 7fb5c759e700 1 Processor -- start 2023-10-11T09:41:38.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.341+0000 7fb5c759e700 1 -- start start 2023-10-11T09:41:38.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.341+0000 7fb5c759e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb5c80b3610 0x7fb5c80d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:38.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.341+0000 7fb5c759e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5c80b47f0 con 0x7fb5c80b3240 2023-10-11T09:41:38.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.341+0000 7fb5c759e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5c80b8130 con 0x7fb5c80b3a90 2023-10-11T09:41:38.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.341+0000 7fb5c759e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c80b3f10 con 0x7fb5c80b3610 2023-10-11T09:41:38.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.341+0000 7fb5c659c700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fb5c80b3240 0x7fb5c815d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33160/0 (socket says 172.21.15.78:33160) 2023-10-11T09:41:38.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.341+0000 7fb5c659c700 1 -- 172.21.15.78:0/1676074988 learned_addr learned my addr 172.21.15.78:0/1676074988 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:38.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.341+0000 7fb5c5d9b700 1 --2- 172.21.15.78:0/1676074988 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb5c80b3610 0x7fb5c80d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:38.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.341+0000 7fb5c559a700 1 -- 172.21.15.78:0/1676074988 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2623237110 0 0) 0x7fb5b8001110 con 0x7fb5c80b3240 2023-10-11T09:41:38.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.341+0000 7fb5c559a700 1 -- 172.21.15.78:0/1676074988 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5b00035d0 con 0x7fb5c80b3240 2023-10-11T09:41:38.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.345+0000 7fb5c559a700 1 -- 172.21.15.78:0/1676074988 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2255491600 0 0) 0x7fb5b00035d0 con 0x7fb5c80b3240 2023-10-11T09:41:38.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.345+0000 7fb5c559a700 1 -- 172.21.15.78:0/1676074988 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb5c80b3610 msgr2=0x7fb5c80d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:38.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.345+0000 7fb5c559a700 1 --2- 172.21.15.78:0/1676074988 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb5c80b3610 0x7fb5c80d29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:38.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.345+0000 7fb5c559a700 1 -- 172.21.15.78:0/1676074988 >> v1:172.21.15.133:6789/0 conn(0x7fb5c80b3a90 legacy=0x7fb5c80b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:38.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.345+0000 7fb5c559a700 1 -- 172.21.15.78:0/1676074988 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c80b41e0 con 0x7fb5c80b3240 2023-10-11T09:41:38.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.345+0000 7fb5c559a700 1 -- 172.21.15.78:0/1676074988 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7fb5b8001800 con 0x7fb5c80b3240 2023-10-11T09:41:38.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.345+0000 7fb5c559a700 1 -- 172.21.15.78:0/1676074988 >> v1:172.21.15.78:6789/0 conn(0x7fb5c80b3240 legacy=0x7fb5c815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:38.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.345+0000 7fb5c559a700 1 --2- 172.21.15.78:0/1676074988 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb5c80b3610 0x7fb5b0003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:38.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.345+0000 7fb5c559a700 1 --2- 172.21.15.78:0/1676074988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5b0003fd0 0x7fb5b0004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:38.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.345+0000 7fb5c559a700 1 --2- 172.21.15.78:0/1676074988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5b0004db0 0x7fb5b0005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:38.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.345+0000 7fb5c559a700 1 -- 172.21.15.78:0/1676074988 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c80b3f10 con 0x7fb5b0004db0 2023-10-11T09:41:38.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.345+0000 7fb5c559a700 1 -- 172.21.15.78:0/1676074988 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b000bbc0 con 0x7fb5b0003fd0 2023-10-11T09:41:38.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.345+0000 7fb5c559a700 1 -- 172.21.15.78:0/1676074988 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b000bd80 con 0x7fb5c80b3610 2023-10-11T09:41:38.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.345+0000 7fb5c6d9d700 1 --2- 172.21.15.78:0/1676074988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5b0004db0 0x7fb5b0005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:38.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.345+0000 7fb5c659c700 1 --2- 172.21.15.78:0/1676074988 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb5c80b3610 0x7fb5b0003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:38.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.345+0000 7fb5c5d9b700 1 --2- 172.21.15.78:0/1676074988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5b0003fd0 0x7fb5b0004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:38.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.345+0000 7fb5c6d9d700 1 -- 172.21.15.78:0/1676074988 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb5c80b3610 msgr2=0x7fb5b0003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:38.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.345+0000 7fb5c6d9d700 1 --2- 172.21.15.78:0/1676074988 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb5c80b3610 0x7fb5b0003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:38.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.345+0000 7fb5c6d9d700 1 -- 172.21.15.78:0/1676074988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5b0003fd0 msgr2=0x7fb5b0004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:38.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.345+0000 7fb5c6d9d700 1 --2- 172.21.15.78:0/1676074988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5b0003fd0 0x7fb5b0004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:38.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.345+0000 7fb5c6d9d700 1 -- 172.21.15.78:0/1676074988 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b00035d0 con 0x7fb5b0004db0 2023-10-11T09:41:38.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.345+0000 7fb5c6d9d700 1 --2- 172.21.15.78:0/1676074988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5b0004db0 0x7fb5b0005220 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb5c000c160 tx=0x7fb5b00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a69c4e7c280d32 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:38.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.345+0000 7fb5c559a700 1 -- 172.21.15.78:0/1676074988 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0001db0 con 0x7fb5b0004db0 2023-10-11T09:41:38.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.345+0000 7fb5c559a700 1 -- 172.21.15.78:0/1676074988 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c000cca0 con 0x7fb5b0004db0 2023-10-11T09:41:38.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.345+0000 7fb5c559a700 1 -- 172.21.15.78:0/1676074988 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0003c60 con 0x7fb5b0004db0 2023-10-11T09:41:38.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.345+0000 7fb5c759e700 1 -- 172.21.15.78:0/1676074988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5b0004db0 msgr2=0x7fb5b0005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:38.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.345+0000 7fb5c759e700 1 --2- 172.21.15.78:0/1676074988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5b0004db0 0x7fb5b0005220 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb5c000c160 tx=0x7fb5b00058a0 comp rx=0 tx=0).stop 2023-10-11T09:41:38.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.345+0000 7fb5c6d9d700 1 -- 172.21.15.78:0/1676074988 reap_dead start 2023-10-11T09:41:38.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.349+0000 7fb5c759e700 1 -- 172.21.15.78:0/1676074988 shutdown_connections 2023-10-11T09:41:38.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.349+0000 7fb5c759e700 1 -- 172.21.15.78:0/1676074988 >> 172.21.15.78:0/1676074988 conn(0x7fb5c809c260 msgr2=0x7fb5c80d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:38.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.349+0000 7fb5c759e700 1 -- 172.21.15.78:0/1676074988 shutdown_connections 2023-10-11T09:41:38.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.349+0000 7fb5c759e700 1 -- 172.21.15.78:0/1676074988 wait complete. 2023-10-11T09:41:38.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.349+0000 7fb5c759e700 1 Processor -- start 2023-10-11T09:41:38.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.353+0000 7fb5c759e700 1 -- start start 2023-10-11T09:41:38.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.353+0000 7fb5c759e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5b0004db0 0x7fb5c815f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:38.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.353+0000 7fb5c759e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb5c815f9a0 0x7fb5c815e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:38.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.353+0000 7fb5c759e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5c814c940 0x7fb5c815e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:38.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.353+0000 7fb5c759e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c814b8a0 con 0x7fb5b0004db0 2023-10-11T09:41:38.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.353+0000 7fb5c759e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c814bb80 con 0x7fb5c814c940 2023-10-11T09:41:38.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.353+0000 7fb5c759e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c814be60 con 0x7fb5c815f9a0 2023-10-11T09:41:38.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.353+0000 7fb5c659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5b0004db0 0x7fb5c815f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:38.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.353+0000 7fb5c659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5b0004db0 0x7fb5c815f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54386/0 (socket says 172.21.15.78:54386) 2023-10-11T09:41:38.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.353+0000 7fb5c659c700 1 -- 172.21.15.78:0/137354106 learned_addr learned my addr 172.21.15.78:0/137354106 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:38.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.353+0000 7fb5c5d9b700 1 --2- 172.21.15.78:0/137354106 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb5c815f9a0 0x7fb5c815e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:38.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.353+0000 7fb5c6d9d700 1 --2- 172.21.15.78:0/137354106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5c814c940 0x7fb5c815e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:38.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.353+0000 7fb5c659c700 1 -- 172.21.15.78:0/137354106 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb5c815f9a0 msgr2=0x7fb5c815e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:38.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.353+0000 7fb5c659c700 1 --2- 172.21.15.78:0/137354106 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb5c815f9a0 0x7fb5c815e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:38.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.353+0000 7fb5c659c700 1 -- 172.21.15.78:0/137354106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5c814c940 msgr2=0x7fb5c815e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:38.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.353+0000 7fb5c659c700 1 --2- 172.21.15.78:0/137354106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5c814c940 0x7fb5c815e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:38.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.353+0000 7fb5c659c700 1 -- 172.21.15.78:0/137354106 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c814c140 con 0x7fb5b0004db0 2023-10-11T09:41:38.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.353+0000 7fb5c659c700 1 --2- 172.21.15.78:0/137354106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5b0004db0 0x7fb5c815f460 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8007310 tx=0x7fb5b8007620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:38.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.353+0000 7fb5b6ffd700 1 -- 172.21.15.78:0/137354106 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b800f660 con 0x7fb5b0004db0 2023-10-11T09:41:38.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.353+0000 7fb5b6ffd700 1 -- 172.21.15.78:0/137354106 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b800f800 con 0x7fb5b0004db0 2023-10-11T09:41:38.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.353+0000 7fb5c759e700 1 -- 172.21.15.78:0/137354106 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5c814c420 con 0x7fb5b0004db0 2023-10-11T09:41:38.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.353+0000 7fb5b6ffd700 1 -- 172.21.15.78:0/137354106 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b800fbb0 con 0x7fb5b0004db0 2023-10-11T09:41:38.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.353+0000 7fb5c759e700 1 -- 172.21.15.78:0/137354106 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c8150fe0 con 0x7fb5b0004db0 2023-10-11T09:41:38.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.357+0000 7fb5c759e700 1 -- 172.21.15.78:0/137354106 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5c81511d0 con 0x7fb5b0004db0 2023-10-11T09:41:38.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.357+0000 7fb5b6ffd700 1 -- 172.21.15.78:0/137354106 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fb5b800fd10 con 0x7fb5b0004db0 2023-10-11T09:41:38.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.357+0000 7fb5b6ffd700 1 --2- 172.21.15.78:0/137354106 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb5a8065ed0 0x7fb5a8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:38.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.357+0000 7fb5b6ffd700 1 -- 172.21.15.78:0/137354106 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb5b80835e0 con 0x7fb5b0004db0 2023-10-11T09:41:38.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.357+0000 7fb5c5d9b700 1 --2- 172.21.15.78:0/137354106 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb5a8065ed0 0x7fb5a8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:38.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.361+0000 7fb5c5d9b700 1 --2- 172.21.15.78:0/137354106 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb5a8065ed0 0x7fb5a8068380 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb5c80b7660 tx=0x7fb5bc009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:38.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.365+0000 7fb5b6ffd700 1 -- 172.21.15.78:0/137354106 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fb5c81511d0 con 0x7fb5b0004db0 2023-10-11T09:41:38.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.537+0000 7fb5c759e700 1 -- 172.21.15.78:0/137354106 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb5c81511d0 con 0x7fb5b0004db0 2023-10-11T09:41:38.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.537+0000 7fb5b6ffd700 1 -- 172.21.15.78:0/137354106 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb5c81511d0 con 0x7fb5b0004db0 2023-10-11T09:41:38.544 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:41:38.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.541+0000 7fb5c759e700 1 -- 172.21.15.78:0/137354106 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb5a8065ed0 msgr2=0x7fb5a8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:38.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.541+0000 7fb5c759e700 1 --2- 172.21.15.78:0/137354106 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb5a8065ed0 0x7fb5a8068380 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb5c80b7660 tx=0x7fb5bc009240 comp rx=0 tx=0).stop 2023-10-11T09:41:38.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.541+0000 7fb5c759e700 1 -- 172.21.15.78:0/137354106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5b0004db0 msgr2=0x7fb5c815f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:38.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.541+0000 7fb5c759e700 1 --2- 172.21.15.78:0/137354106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5b0004db0 0x7fb5c815f460 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8007310 tx=0x7fb5b8007620 comp rx=0 tx=0).stop 2023-10-11T09:41:38.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.541+0000 7fb5c759e700 1 -- 172.21.15.78:0/137354106 shutdown_connections 2023-10-11T09:41:38.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.541+0000 7fb5c759e700 1 --2- 172.21.15.78:0/137354106 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb5a8065ed0 0x7fb5a8068380 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:38.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.541+0000 7fb5c759e700 1 --2- 172.21.15.78:0/137354106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5c814c940 0x7fb5c815e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:38.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.541+0000 7fb5c759e700 1 --2- 172.21.15.78:0/137354106 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb5c815f9a0 0x7fb5c815e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:38.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.541+0000 7fb5c759e700 1 --2- 172.21.15.78:0/137354106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5b0004db0 0x7fb5c815f460 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:38.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.541+0000 7fb5c759e700 1 -- 172.21.15.78:0/137354106 >> 172.21.15.78:0/137354106 conn(0x7fb5c809c260 msgr2=0x7fb5c80d0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:38.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.541+0000 7fb5c759e700 1 -- 172.21.15.78:0/137354106 shutdown_connections 2023-10-11T09:41:38.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.541+0000 7fb5c759e700 1 -- 172.21.15.78:0/137354106 wait complete. 2023-10-11T09:41:38.570 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T09:41:38.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.737+0000 7f0ed8def700 1 Processor -- start 2023-10-11T09:41:38.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.737+0000 7f0ed8def700 1 -- start start 2023-10-11T09:41:38.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.741+0000 7f0ed8def700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0ed40ceca0 0x7f0ed4159c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:38.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.741+0000 7f0ed8def700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ed415c860 con 0x7f0ed40cf6f0 2023-10-11T09:41:38.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.741+0000 7f0ed8def700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ed40cd630 con 0x7f0ed40cfac0 2023-10-11T09:41:38.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.741+0000 7f0ed37fe700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f0ed40cf6f0 0x7f0ed40ccf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33174/0 (socket says 172.21.15.78:33174) 2023-10-11T09:41:38.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.741+0000 7f0ed8def700 1 -- 172.21.15.78:0/1045916022 --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ed40cf150 con 0x7f0ed40ceca0 2023-10-11T09:41:38.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.741+0000 7f0ed37fe700 1 -- 172.21.15.78:0/1045916022 learned_addr learned my addr 172.21.15.78:0/1045916022 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:38.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.741+0000 7f0ed2ffd700 1 --2- 172.21.15.78:0/1045916022 >> v2:172.21.15.78:3300/0 conn(0x7f0ed40ceca0 0x7f0ed4159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:38.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.741+0000 7f0ed27fc700 1 -- 172.21.15.78:0/1045916022 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 369253372 0 0) 0x7f0ed415c860 con 0x7f0ed40cf6f0 2023-10-11T09:41:38.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.741+0000 7f0ed27fc700 1 -- 172.21.15.78:0/1045916022 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ebc0057e0 con 0x7f0ed40cf6f0 2023-10-11T09:41:38.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.741+0000 7f0ed2ffd700 1 -- 172.21.15.78:0/1045916022 >> v1:172.21.15.133:6789/0 conn(0x7f0ed40cfac0 legacy=0x7f0ed415cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:38.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.741+0000 7f0ed2ffd700 1 -- 172.21.15.78:0/1045916022 >> v1:172.21.15.78:6789/0 conn(0x7f0ed40cf6f0 legacy=0x7f0ed40ccf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:38.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.741+0000 7f0ed2ffd700 1 -- 172.21.15.78:0/1045916022 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed40cb4f0 con 0x7f0ed40ceca0 2023-10-11T09:41:38.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.741+0000 7f0ed2ffd700 1 --2- 172.21.15.78:0/1045916022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ed40ceca0 0x7f0ed4159c50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800a9f0 tx=0x7f0ec800ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212cfe5b1d2abca2 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:38.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.741+0000 7f0ed27fc700 1 -- 172.21.15.78:0/1045916022 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec800cb70 con 0x7f0ed40ceca0 2023-10-11T09:41:38.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.741+0000 7f0ed27fc700 1 -- 172.21.15.78:0/1045916022 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec800ccd0 con 0x7f0ed40ceca0 2023-10-11T09:41:38.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.741+0000 7f0ed27fc700 1 -- 172.21.15.78:0/1045916022 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8010620 con 0x7f0ed40ceca0 2023-10-11T09:41:38.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.745+0000 7f0ed8def700 1 -- 172.21.15.78:0/1045916022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ed40ceca0 msgr2=0x7f0ed4159c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:38.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.745+0000 7f0ed8def700 1 --2- 172.21.15.78:0/1045916022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ed40ceca0 0x7f0ed4159c50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800a9f0 tx=0x7f0ec800ad00 comp rx=0 tx=0).stop 2023-10-11T09:41:38.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.745+0000 7f0ed8def700 1 -- 172.21.15.78:0/1045916022 shutdown_connections 2023-10-11T09:41:38.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.745+0000 7f0ed8def700 1 --2- 172.21.15.78:0/1045916022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ed40ceca0 0x7f0ed4159c50 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:38.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.745+0000 7f0ed8def700 1 -- 172.21.15.78:0/1045916022 >> 172.21.15.78:0/1045916022 conn(0x7f0ed40b0f40 msgr2=0x7f0ed40b1310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:38.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.745+0000 7f0ed8def700 1 -- 172.21.15.78:0/1045916022 shutdown_connections 2023-10-11T09:41:38.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.745+0000 7f0ed8def700 1 -- 172.21.15.78:0/1045916022 wait complete. 2023-10-11T09:41:38.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.745+0000 7f0ed8def700 1 Processor -- start 2023-10-11T09:41:38.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.745+0000 7f0ed8def700 1 -- start start 2023-10-11T09:41:38.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.749+0000 7f0ed8def700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ed40ceca0 0x7f0ed41494d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:38.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.749+0000 7f0ed8def700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ed40cf6f0 0x7f0ed40b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:38.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.749+0000 7f0ed8def700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ed40cfac0 0x7f0ed415cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:38.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.749+0000 7f0ed8def700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed40b2be0 con 0x7f0ed40cf6f0 2023-10-11T09:41:38.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.749+0000 7f0ed8def700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed40b4930 con 0x7f0ed40ceca0 2023-10-11T09:41:38.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.749+0000 7f0ed8def700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed40b2240 con 0x7f0ed40cfac0 2023-10-11T09:41:38.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.749+0000 7f0ed2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ed40cf6f0 0x7f0ed40b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:38.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.749+0000 7f0ed2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ed40cf6f0 0x7f0ed40b3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54402/0 (socket says 172.21.15.78:54402) 2023-10-11T09:41:38.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.749+0000 7f0ed2ffd700 1 -- 172.21.15.78:0/2654728402 learned_addr learned my addr 172.21.15.78:0/2654728402 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:38.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.749+0000 7f0ed37fe700 1 --2- 172.21.15.78:0/2654728402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ed40ceca0 0x7f0ed41494d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:38.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.749+0000 7f0ed3fff700 1 --2- 172.21.15.78:0/2654728402 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ed40cfac0 0x7f0ed415cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:38.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.749+0000 7f0ed2ffd700 1 -- 172.21.15.78:0/2654728402 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ed40cfac0 msgr2=0x7f0ed415cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:38.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.749+0000 7f0ed2ffd700 1 --2- 172.21.15.78:0/2654728402 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ed40cfac0 0x7f0ed415cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:38.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.749+0000 7f0ed2ffd700 1 -- 172.21.15.78:0/2654728402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ed40ceca0 msgr2=0x7f0ed41494d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:38.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.749+0000 7f0ed2ffd700 1 --2- 172.21.15.78:0/2654728402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ed40ceca0 0x7f0ed41494d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:38.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.749+0000 7f0ed2ffd700 1 -- 172.21.15.78:0/2654728402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec800a650 con 0x7f0ed40cf6f0 2023-10-11T09:41:38.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.749+0000 7f0ed2ffd700 1 --2- 172.21.15.78:0/2654728402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ed40cf6f0 0x7f0ed40b3e70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800ad40 tx=0x7f0ec8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:38.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.749+0000 7f0ebbfff700 1 -- 172.21.15.78:0/2654728402 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec801e070 con 0x7f0ed40cf6f0 2023-10-11T09:41:38.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.749+0000 7f0ebbfff700 1 -- 172.21.15.78:0/2654728402 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec800cec0 con 0x7f0ed40cf6f0 2023-10-11T09:41:38.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.749+0000 7f0ed8def700 1 -- 172.21.15.78:0/2654728402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed40b2520 con 0x7f0ed40cf6f0 2023-10-11T09:41:38.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.749+0000 7f0ebbfff700 1 -- 172.21.15.78:0/2654728402 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8004480 con 0x7f0ed40cf6f0 2023-10-11T09:41:38.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.749+0000 7f0ed8def700 1 -- 172.21.15.78:0/2654728402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed4157810 con 0x7f0ed40cf6f0 2023-10-11T09:41:38.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.753+0000 7f0ed8def700 1 -- 172.21.15.78:0/2654728402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ed40b2d20 con 0x7f0ed40cf6f0 2023-10-11T09:41:38.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.753+0000 7f0ebbfff700 1 -- 172.21.15.78:0/2654728402 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f0ec80107d0 con 0x7f0ed40cf6f0 2023-10-11T09:41:38.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.757+0000 7f0ebbfff700 1 --2- 172.21.15.78:0/2654728402 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0ec0065ed0 0x7f0ec0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:38.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.757+0000 7f0ebbfff700 1 -- 172.21.15.78:0/2654728402 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f0ec8084ca0 con 0x7f0ed40cf6f0 2023-10-11T09:41:38.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.757+0000 7f0ed37fe700 1 --2- 172.21.15.78:0/2654728402 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0ec0065ed0 0x7f0ec0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:38.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.757+0000 7f0ed37fe700 1 --2- 172.21.15.78:0/2654728402 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0ec0065ed0 0x7f0ec0068380 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0ec40011e0 tx=0x7f0ec4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:38.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.761+0000 7f0ebbfff700 1 -- 172.21.15.78:0/2654728402 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f0ed40b2d20 con 0x7f0ed40cf6f0 2023-10-11T09:41:38.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.933+0000 7f0ed8def700 1 -- 172.21.15.78:0/2654728402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0ed40b2d20 con 0x7f0ed40cf6f0 2023-10-11T09:41:38.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.933+0000 7f0ebbfff700 1 -- 172.21.15.78:0/2654728402 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0ed40b2d20 con 0x7f0ed40cf6f0 2023-10-11T09:41:38.940 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:41:38.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.937+0000 7f0ed8def700 1 -- 172.21.15.78:0/2654728402 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0ec0065ed0 msgr2=0x7f0ec0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:38.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.937+0000 7f0ed8def700 1 --2- 172.21.15.78:0/2654728402 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0ec0065ed0 0x7f0ec0068380 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0ec40011e0 tx=0x7f0ec4009040 comp rx=0 tx=0).stop 2023-10-11T09:41:38.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.937+0000 7f0ed8def700 1 -- 172.21.15.78:0/2654728402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ed40cf6f0 msgr2=0x7f0ed40b3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:38.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.937+0000 7f0ed8def700 1 --2- 172.21.15.78:0/2654728402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ed40cf6f0 0x7f0ed40b3e70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800ad40 tx=0x7f0ec8004770 comp rx=0 tx=0).stop 2023-10-11T09:41:38.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.937+0000 7f0ed8def700 1 -- 172.21.15.78:0/2654728402 shutdown_connections 2023-10-11T09:41:38.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.937+0000 7f0ed8def700 1 --2- 172.21.15.78:0/2654728402 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0ec0065ed0 0x7f0ec0068380 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:38.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.937+0000 7f0ed8def700 1 --2- 172.21.15.78:0/2654728402 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ed40cfac0 0x7f0ed415cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:38.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.937+0000 7f0ed8def700 1 --2- 172.21.15.78:0/2654728402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ed40cf6f0 0x7f0ed40b3e70 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:38.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.937+0000 7f0ed8def700 1 --2- 172.21.15.78:0/2654728402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ed40ceca0 0x7f0ed41494d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:38.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.937+0000 7f0ed8def700 1 -- 172.21.15.78:0/2654728402 >> 172.21.15.78:0/2654728402 conn(0x7f0ed40b0f40 msgr2=0x7f0ed40ce260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:38.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.937+0000 7f0ed8def700 1 -- 172.21.15.78:0/2654728402 shutdown_connections 2023-10-11T09:41:38.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:38.937+0000 7f0ed8def700 1 -- 172.21.15.78:0/2654728402 wait complete. 2023-10-11T09:41:38.963 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T09:41:39.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.129+0000 7f0491d64700 1 Processor -- start 2023-10-11T09:41:39.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.129+0000 7f0491d64700 1 -- start start 2023-10-11T09:41:39.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.133+0000 7f0491d64700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f048c0b3240 0x7f048c0d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:39.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.133+0000 7f0491d64700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f048c0b3610 0x7f048c0b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:39.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.133+0000 7f0491d64700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f048c0b47f0 con 0x7f048c15d040 2023-10-11T09:41:39.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.133+0000 7f0491d64700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f048c0b3fd0 con 0x7f048c0b3610 2023-10-11T09:41:39.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.133+0000 7f0491d64700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f048c0b8130 con 0x7f048c0b3240 2023-10-11T09:41:39.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.133+0000 7f048bfff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f048c0b3610 0x7f048c0b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:39.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.133+0000 7f0491563700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f048c15d040 0x7f048c0b85c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33186/0 (socket says 172.21.15.78:33186) 2023-10-11T09:41:39.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.133+0000 7f0491563700 1 -- 172.21.15.78:0/2869281471 learned_addr learned my addr 172.21.15.78:0/2869281471 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:39.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.133+0000 7f0490d62700 1 --2- 172.21.15.78:0/2869281471 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f048c0b3240 0x7f048c0d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:39.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.137+0000 7f048bfff700 1 -- 172.21.15.78:0/2869281471 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f048c0b3240 msgr2=0x7f048c0d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:39.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.137+0000 7f048bfff700 1 --2- 172.21.15.78:0/2869281471 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f048c0b3240 0x7f048c0d29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:39.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.137+0000 7f048bfff700 1 -- 172.21.15.78:0/2869281471 >> v1:172.21.15.78:6789/0 conn(0x7f048c15d040 legacy=0x7f048c0b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:39.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.137+0000 7f048b7fe700 1 -- 172.21.15.78:0/2869281471 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1125279226 0 0) 0x7f0484001490 con 0x7f048c15d040 2023-10-11T09:41:39.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.137+0000 7f048bfff700 1 -- 172.21.15.78:0/2869281471 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f048c0b41e0 con 0x7f048c0b3610 2023-10-11T09:41:39.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.137+0000 7f048bfff700 1 --2- 172.21.15.78:0/2869281471 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f048c0b3610 0x7f048c0b3a90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0478009850 tx=0x7f0478009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48934952591978be server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:39.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.141+0000 7f048b7fe700 1 -- 172.21.15.78:0/2869281471 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0478011830 con 0x7f048c0b3610 2023-10-11T09:41:39.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.141+0000 7f048b7fe700 1 -- 172.21.15.78:0/2869281471 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0478011990 con 0x7f048c0b3610 2023-10-11T09:41:39.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.141+0000 7f048b7fe700 1 -- 172.21.15.78:0/2869281471 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0478011d40 con 0x7f048c0b3610 2023-10-11T09:41:39.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.141+0000 7f0491d64700 1 -- 172.21.15.78:0/2869281471 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f048c0b3610 msgr2=0x7f048c0b3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:39.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.141+0000 7f0491d64700 1 --2- 172.21.15.78:0/2869281471 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f048c0b3610 0x7f048c0b3a90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0478009850 tx=0x7f0478009b60 comp rx=0 tx=0).stop 2023-10-11T09:41:39.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.141+0000 7f0491d64700 1 -- 172.21.15.78:0/2869281471 shutdown_connections 2023-10-11T09:41:39.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.141+0000 7f0491d64700 1 --2- 172.21.15.78:0/2869281471 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f048c0b3610 0x7f048c0b3a90 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:39.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.141+0000 7f0491d64700 1 --2- 172.21.15.78:0/2869281471 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f048c0b3240 0x7f048c0d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:39.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.141+0000 7f0491d64700 1 -- 172.21.15.78:0/2869281471 >> 172.21.15.78:0/2869281471 conn(0x7f048c09c260 msgr2=0x7f048c0d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:39.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.141+0000 7f0491d64700 1 -- 172.21.15.78:0/2869281471 shutdown_connections 2023-10-11T09:41:39.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.141+0000 7f0491d64700 1 -- 172.21.15.78:0/2869281471 wait complete. 2023-10-11T09:41:39.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.145+0000 7f0491d64700 1 Processor -- start 2023-10-11T09:41:39.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.145+0000 7f0491d64700 1 -- start start 2023-10-11T09:41:39.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.145+0000 7f0491d64700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f048c0b3240 0x7f048c09f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:39.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.145+0000 7f0491d64700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f048c0b3610 0x7f048c1f5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:39.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.145+0000 7f0491d64700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f048c15d040 0x7f048c15ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:39.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.145+0000 7f0491d64700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048c1f6410 con 0x7f048c0b3240 2023-10-11T09:41:39.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.145+0000 7f0491d64700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048c09d880 con 0x7f048c0b3610 2023-10-11T09:41:39.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.145+0000 7f0491d64700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f048c09f880 con 0x7f048c15d040 2023-10-11T09:41:39.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.145+0000 7f0490d62700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f048c0b3240 0x7f048c09f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:39.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.145+0000 7f048bfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f048c0b3610 0x7f048c1f5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:39.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.145+0000 7f0490d62700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f048c0b3240 0x7f048c09f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54424/0 (socket says 172.21.15.78:54424) 2023-10-11T09:41:39.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.145+0000 7f0490d62700 1 -- 172.21.15.78:0/1733321382 learned_addr learned my addr 172.21.15.78:0/1733321382 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:39.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.145+0000 7f0491563700 1 --2- 172.21.15.78:0/1733321382 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f048c15d040 0x7f048c15ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:39.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.149+0000 7f0490d62700 1 -- 172.21.15.78:0/1733321382 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f048c15d040 msgr2=0x7f048c15ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:39.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.149+0000 7f0490d62700 1 --2- 172.21.15.78:0/1733321382 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f048c15d040 0x7f048c15ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:39.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.149+0000 7f0490d62700 1 -- 172.21.15.78:0/1733321382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f048c0b3610 msgr2=0x7f048c1f5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:39.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.149+0000 7f0490d62700 1 --2- 172.21.15.78:0/1733321382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f048c0b3610 0x7f048c1f5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:39.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.149+0000 7f0490d62700 1 -- 172.21.15.78:0/1733321382 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04780094b0 con 0x7f048c0b3240 2023-10-11T09:41:39.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.149+0000 7f0490d62700 1 --2- 172.21.15.78:0/1733321382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f048c0b3240 0x7f048c09f250 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f048000e840 tx=0x7f048000eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:39.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.149+0000 7f04897fa700 1 -- 172.21.15.78:0/1733321382 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04800095a0 con 0x7f048c0b3240 2023-10-11T09:41:39.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.149+0000 7f04897fa700 1 -- 172.21.15.78:0/1733321382 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0480013680 con 0x7f048c0b3240 2023-10-11T09:41:39.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.149+0000 7f0491d64700 1 -- 172.21.15.78:0/1733321382 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f048c09fb60 con 0x7f048c0b3240 2023-10-11T09:41:39.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.149+0000 7f04897fa700 1 -- 172.21.15.78:0/1733321382 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04800139f0 con 0x7f048c0b3240 2023-10-11T09:41:39.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.149+0000 7f0491d64700 1 -- 172.21.15.78:0/1733321382 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f048c0a0120 con 0x7f048c0b3240 2023-10-11T09:41:39.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.149+0000 7f0491d64700 1 -- 172.21.15.78:0/1733321382 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f048c0d14d0 con 0x7f048c0b3240 2023-10-11T09:41:39.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.153+0000 7f04897fa700 1 -- 172.21.15.78:0/1733321382 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f0480013b50 con 0x7f048c0b3240 2023-10-11T09:41:39.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.153+0000 7f04897fa700 1 --2- 172.21.15.78:0/1733321382 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0470065ed0 0x7f0470068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:39.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.153+0000 7f04897fa700 1 -- 172.21.15.78:0/1733321382 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f0480083710 con 0x7f048c0b3240 2023-10-11T09:41:39.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.153+0000 7f048bfff700 1 --2- 172.21.15.78:0/1733321382 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0470065ed0 0x7f0470068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:39.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.153+0000 7f048bfff700 1 --2- 172.21.15.78:0/1733321382 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0470065ed0 0x7f0470068380 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f048c0b3a40 tx=0x7f047801c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:39.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.161+0000 7f04897fa700 1 -- 172.21.15.78:0/1733321382 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f0480050c80 con 0x7f048c0b3240 2023-10-11T09:41:39.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.333+0000 7f0491d64700 1 -- 172.21.15.78:0/1733321382 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f048c065d40 con 0x7f048c0b3240 2023-10-11T09:41:39.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.333+0000 7f04897fa700 1 -- 172.21.15.78:0/1733321382 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f04800542a0 con 0x7f048c0b3240 2023-10-11T09:41:39.339 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2023-10-11T09:41:39.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.337+0000 7f0491d64700 1 -- 172.21.15.78:0/1733321382 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0470065ed0 msgr2=0x7f0470068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:39.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.337+0000 7f0491d64700 1 --2- 172.21.15.78:0/1733321382 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0470065ed0 0x7f0470068380 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f048c0b3a40 tx=0x7f047801c040 comp rx=0 tx=0).stop 2023-10-11T09:41:39.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.337+0000 7f0491d64700 1 -- 172.21.15.78:0/1733321382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f048c0b3240 msgr2=0x7f048c09f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:39.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.337+0000 7f0491d64700 1 --2- 172.21.15.78:0/1733321382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f048c0b3240 0x7f048c09f250 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f048000e840 tx=0x7f048000eb50 comp rx=0 tx=0).stop 2023-10-11T09:41:39.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.337+0000 7f0491d64700 1 -- 172.21.15.78:0/1733321382 shutdown_connections 2023-10-11T09:41:39.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.337+0000 7f0491d64700 1 --2- 172.21.15.78:0/1733321382 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0470065ed0 0x7f0470068380 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:39.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.337+0000 7f0491d64700 1 --2- 172.21.15.78:0/1733321382 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f048c15d040 0x7f048c15ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:39.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.337+0000 7f0491d64700 1 --2- 172.21.15.78:0/1733321382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f048c0b3610 0x7f048c1f5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:39.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.337+0000 7f0491d64700 1 --2- 172.21.15.78:0/1733321382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f048c0b3240 0x7f048c09f250 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:39.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.337+0000 7f0491d64700 1 -- 172.21.15.78:0/1733321382 >> 172.21.15.78:0/1733321382 conn(0x7f048c09c260 msgr2=0x7f048c0d2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:39.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.337+0000 7f0491d64700 1 -- 172.21.15.78:0/1733321382 shutdown_connections 2023-10-11T09:41:39.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.337+0000 7f0491d64700 1 -- 172.21.15.78:0/1733321382 wait complete. 2023-10-11T09:41:39.365 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T09:41:39.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.529+0000 7f8770cd8700 1 Processor -- start 2023-10-11T09:41:39.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.529+0000 7f8770cd8700 1 -- start start 2023-10-11T09:41:39.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.533+0000 7f8770cd8700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f876c06e1c0 0x7f876c06e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:39.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.533+0000 7f8770cd8700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f876c06ead0 con 0x7f876c160dc0 2023-10-11T09:41:39.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.533+0000 7f8770cd8700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f876c0d0620 con 0x7f876c06fb90 2023-10-11T09:41:39.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.533+0000 7f8770cd8700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f876c06f140 con 0x7f876c06e1c0 2023-10-11T09:41:39.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.533+0000 7f876ad9d700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f876c160dc0 0x7f876c15cfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33196/0 (socket says 172.21.15.78:33196) 2023-10-11T09:41:39.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.533+0000 7f876ad9d700 1 -- 172.21.15.78:0/2627495399 learned_addr learned my addr 172.21.15.78:0/2627495399 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:39.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.533+0000 7f876a59c700 1 --2- 172.21.15.78:0/2627495399 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f876c06e1c0 0x7f876c06e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:39.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.533+0000 7f8769d9b700 1 -- 172.21.15.78:0/2627495399 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2646395189 0 0) 0x7f876c06ead0 con 0x7f876c160dc0 2023-10-11T09:41:39.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.533+0000 7f8769d9b700 1 -- 172.21.15.78:0/2627495399 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87580035d0 con 0x7f876c160dc0 2023-10-11T09:41:39.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.533+0000 7f876a59c700 1 -- 172.21.15.78:0/2627495399 >> v1:172.21.15.133:6789/0 conn(0x7f876c06fb90 legacy=0x7f876c0cfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:39.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.533+0000 7f876a59c700 1 -- 172.21.15.78:0/2627495399 >> v1:172.21.15.78:6789/0 conn(0x7f876c160dc0 legacy=0x7f876c15cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:39.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.533+0000 7f876a59c700 1 -- 172.21.15.78:0/2627495399 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f876c0d0900 con 0x7f876c06e1c0 2023-10-11T09:41:39.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.537+0000 7f876a59c700 1 --2- 172.21.15.78:0/2627495399 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f876c06e1c0 0x7f876c06e590 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f875c009880 tx=0x7f875c009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=109f21a13c356e3d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:39.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.537+0000 7f8769d9b700 1 -- 172.21.15.78:0/2627495399 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c011830 con 0x7f876c06e1c0 2023-10-11T09:41:39.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.537+0000 7f8769d9b700 1 -- 172.21.15.78:0/2627495399 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875c011990 con 0x7f876c06e1c0 2023-10-11T09:41:39.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.537+0000 7f8769d9b700 1 -- 172.21.15.78:0/2627495399 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c011d20 con 0x7f876c06e1c0 2023-10-11T09:41:39.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.537+0000 7f8770cd8700 1 -- 172.21.15.78:0/2627495399 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f876c06e1c0 msgr2=0x7f876c06e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:39.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.537+0000 7f8770cd8700 1 --2- 172.21.15.78:0/2627495399 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f876c06e1c0 0x7f876c06e590 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f875c009880 tx=0x7f875c009b90 comp rx=0 tx=0).stop 2023-10-11T09:41:39.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.537+0000 7f8770cd8700 1 -- 172.21.15.78:0/2627495399 shutdown_connections 2023-10-11T09:41:39.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.537+0000 7f8770cd8700 1 --2- 172.21.15.78:0/2627495399 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f876c06e1c0 0x7f876c06e590 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:39.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.537+0000 7f8770cd8700 1 -- 172.21.15.78:0/2627495399 >> 172.21.15.78:0/2627495399 conn(0x7f876c0d0f70 msgr2=0x7f876c0d1370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:39.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.537+0000 7f8770cd8700 1 -- 172.21.15.78:0/2627495399 shutdown_connections 2023-10-11T09:41:39.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.537+0000 7f8770cd8700 1 -- 172.21.15.78:0/2627495399 wait complete. 2023-10-11T09:41:39.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.541+0000 7f8770cd8700 1 Processor -- start 2023-10-11T09:41:39.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.541+0000 7f8770cd8700 1 -- start start 2023-10-11T09:41:39.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.541+0000 7f8770cd8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f876c06e1c0 0x7f876c0caa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:39.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.541+0000 7f8770cd8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f876c06fb90 0x7f876c148d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:39.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.541+0000 7f8770cd8700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f876c160dc0 0x7f876c0c6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:39.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.541+0000 7f8770cd8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876c0b0e90 con 0x7f876c06e1c0 2023-10-11T09:41:39.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.541+0000 7f8770cd8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876c0b0910 con 0x7f876c06fb90 2023-10-11T09:41:39.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.541+0000 7f8770cd8700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f876c149250 con 0x7f876c160dc0 2023-10-11T09:41:39.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.545+0000 7f876a59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f876c06e1c0 0x7f876c0caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:39.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.545+0000 7f876a59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f876c06e1c0 0x7f876c0caa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54436/0 (socket says 172.21.15.78:54436) 2023-10-11T09:41:39.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.545+0000 7f876a59c700 1 -- 172.21.15.78:0/3251589820 learned_addr learned my addr 172.21.15.78:0/3251589820 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:39.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.545+0000 7f8763fff700 1 --2- 172.21.15.78:0/3251589820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f876c06fb90 0x7f876c148d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:39.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.545+0000 7f876ad9d700 1 --2- 172.21.15.78:0/3251589820 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f876c160dc0 0x7f876c0c6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:39.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.545+0000 7f876a59c700 1 -- 172.21.15.78:0/3251589820 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f876c160dc0 msgr2=0x7f876c0c6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:39.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.545+0000 7f876a59c700 1 --2- 172.21.15.78:0/3251589820 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f876c160dc0 0x7f876c0c6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:39.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.545+0000 7f876a59c700 1 -- 172.21.15.78:0/3251589820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f876c06fb90 msgr2=0x7f876c148d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:39.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.545+0000 7f876a59c700 1 --2- 172.21.15.78:0/3251589820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f876c06fb90 0x7f876c148d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:39.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.545+0000 7f876a59c700 1 -- 172.21.15.78:0/3251589820 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875c0094b0 con 0x7f876c06e1c0 2023-10-11T09:41:39.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.545+0000 7f876a59c700 1 --2- 172.21.15.78:0/3251589820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f876c06e1c0 0x7f876c0caa20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f875c009bd0 tx=0x7f875c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:39.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.545+0000 7f8762ffd700 1 -- 172.21.15.78:0/3251589820 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c0036c0 con 0x7f876c06e1c0 2023-10-11T09:41:39.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.545+0000 7f8762ffd700 1 -- 172.21.15.78:0/3251589820 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875c01e070 con 0x7f876c06e1c0 2023-10-11T09:41:39.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.545+0000 7f8770cd8700 1 -- 172.21.15.78:0/3251589820 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f876c0b0bb0 con 0x7f876c06e1c0 2023-10-11T09:41:39.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.545+0000 7f8762ffd700 1 -- 172.21.15.78:0/3251589820 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c003a80 con 0x7f876c06e1c0 2023-10-11T09:41:39.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.545+0000 7f8770cd8700 1 -- 172.21.15.78:0/3251589820 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f876c14c4a0 con 0x7f876c06e1c0 2023-10-11T09:41:39.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.545+0000 7f8770cd8700 1 -- 172.21.15.78:0/3251589820 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f876c0c62c0 con 0x7f876c06e1c0 2023-10-11T09:41:39.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.549+0000 7f8762ffd700 1 -- 172.21.15.78:0/3251589820 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f875c003be0 con 0x7f876c06e1c0 2023-10-11T09:41:39.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.549+0000 7f8762ffd700 1 --2- 172.21.15.78:0/3251589820 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f874c065ed0 0x7f874c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:39.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.549+0000 7f8762ffd700 1 -- 172.21.15.78:0/3251589820 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f875c084740 con 0x7f876c06e1c0 2023-10-11T09:41:39.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.549+0000 7f8763fff700 1 --2- 172.21.15.78:0/3251589820 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f874c065ed0 0x7f874c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:39.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.553+0000 7f8763fff700 1 --2- 172.21.15.78:0/3251589820 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f874c065ed0 0x7f874c068380 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f876c0c7f60 tx=0x7f8754009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:39.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.557+0000 7f8762ffd700 1 -- 172.21.15.78:0/3251589820 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f876c0c62c0 con 0x7f876c06e1c0 2023-10-11T09:41:39.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.741+0000 7f8770cd8700 1 -- 172.21.15.78:0/3251589820 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f876c0c66a0 con 0x7f876c06e1c0 2023-10-11T09:41:39.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.741+0000 7f8762ffd700 1 -- 172.21.15.78:0/3251589820 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f876c0c66a0 con 0x7f876c06e1c0 2023-10-11T09:41:39.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.745+0000 7f8770cd8700 1 -- 172.21.15.78:0/3251589820 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f874c065ed0 msgr2=0x7f874c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:39.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.745+0000 7f8770cd8700 1 --2- 172.21.15.78:0/3251589820 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f874c065ed0 0x7f874c068380 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f876c0c7f60 tx=0x7f8754009040 comp rx=0 tx=0).stop 2023-10-11T09:41:39.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.745+0000 7f8770cd8700 1 -- 172.21.15.78:0/3251589820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f876c06e1c0 msgr2=0x7f876c0caa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:39.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.745+0000 7f8770cd8700 1 --2- 172.21.15.78:0/3251589820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f876c06e1c0 0x7f876c0caa20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f875c009bd0 tx=0x7f875c004770 comp rx=0 tx=0).stop 2023-10-11T09:41:39.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.745+0000 7f8770cd8700 1 -- 172.21.15.78:0/3251589820 shutdown_connections 2023-10-11T09:41:39.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.745+0000 7f8770cd8700 1 --2- 172.21.15.78:0/3251589820 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f874c065ed0 0x7f874c068380 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:39.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.745+0000 7f8770cd8700 1 --2- 172.21.15.78:0/3251589820 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f876c160dc0 0x7f876c0c6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:39.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.745+0000 7f8770cd8700 1 --2- 172.21.15.78:0/3251589820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f876c06fb90 0x7f876c148d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:39.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.745+0000 7f8770cd8700 1 --2- 172.21.15.78:0/3251589820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f876c06e1c0 0x7f876c0caa20 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:39.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.745+0000 7f8770cd8700 1 -- 172.21.15.78:0/3251589820 >> 172.21.15.78:0/3251589820 conn(0x7f876c0d0f70 msgr2=0x7f876c0d1340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:39.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.745+0000 7f8770cd8700 1 -- 172.21.15.78:0/3251589820 shutdown_connections 2023-10-11T09:41:39.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.745+0000 7f8770cd8700 1 -- 172.21.15.78:0/3251589820 wait complete. 2023-10-11T09:41:39.753 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2023-10-11T09:41:39.773 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:41:39.774 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:41:39.792 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:41:39.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.957+0000 7fdeccaaa700 1 Processor -- start 2023-10-11T09:41:39.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.957+0000 7fdeccaaa700 1 -- start start 2023-10-11T09:41:39.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.961+0000 7fdeccaaa700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdec80caf20 0x7fdec8164610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:39.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.961+0000 7fdeccaaa700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fdec8164b50 0x7fdec806bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:39.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.961+0000 7fdeccaaa700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdec8165390 con 0x7fdec80cda10 2023-10-11T09:41:39.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.961+0000 7fdeccaaa700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdec80cb3a0 con 0x7fdec8164b50 2023-10-11T09:41:39.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.961+0000 7fdeccaaa700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdec80ca440 con 0x7fdec80caf20 2023-10-11T09:41:39.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.961+0000 7fdec7fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fdec8164b50 0x7fdec806bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:39.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.961+0000 7fdec77fe700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fdec80cda10 0x7fdec80cdde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33204/0 (socket says 172.21.15.78:33204) 2023-10-11T09:41:39.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.961+0000 7fdec77fe700 1 -- 172.21.15.78:0/3825562602 learned_addr learned my addr 172.21.15.78:0/3825562602 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:39.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.961+0000 7fdec7fff700 1 -- 172.21.15.78:0/3825562602 >> v2:172.21.15.133:3300/0 conn(0x7fdec80caf20 msgr2=0x7fdec8164610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:39.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.961+0000 7fdec7fff700 1 --2- 172.21.15.78:0/3825562602 >> v2:172.21.15.133:3300/0 conn(0x7fdec80caf20 0x7fdec8164610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:39.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.961+0000 7fdec7fff700 1 -- 172.21.15.78:0/3825562602 >> v1:172.21.15.78:6789/0 conn(0x7fdec80cda10 legacy=0x7fdec80cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:39.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.961+0000 7fdec7fff700 1 -- 172.21.15.78:0/3825562602 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec80ca720 con 0x7fdec8164b50 2023-10-11T09:41:39.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.961+0000 7fdec7fff700 1 --2- 172.21.15.78:0/3825562602 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdec8164b50 0x7fdec806bbf0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdec80cea10 tx=0x7fdec000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc33fca70154521b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:39.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.961+0000 7fdec67fc700 1 -- 172.21.15.78:0/3825562602 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec000e070 con 0x7fdec8164b50 2023-10-11T09:41:39.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.961+0000 7fdec67fc700 1 -- 172.21.15.78:0/3825562602 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec000bbd0 con 0x7fdec8164b50 2023-10-11T09:41:39.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.961+0000 7fdec67fc700 1 -- 172.21.15.78:0/3825562602 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec0012430 con 0x7fdec8164b50 2023-10-11T09:41:39.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.961+0000 7fdeccaaa700 1 -- 172.21.15.78:0/3825562602 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdec8164b50 msgr2=0x7fdec806bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:39.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.961+0000 7fdeccaaa700 1 --2- 172.21.15.78:0/3825562602 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdec8164b50 0x7fdec806bbf0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdec80cea10 tx=0x7fdec000b330 comp rx=0 tx=0).stop 2023-10-11T09:41:39.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.961+0000 7fdeccaaa700 1 -- 172.21.15.78:0/3825562602 shutdown_connections 2023-10-11T09:41:39.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.961+0000 7fdeccaaa700 1 --2- 172.21.15.78:0/3825562602 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdec8164b50 0x7fdec806bbf0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:39.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.961+0000 7fdeccaaa700 1 --2- 172.21.15.78:0/3825562602 >> v2:172.21.15.133:3300/0 conn(0x7fdec80caf20 0x7fdec8164610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:39.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.961+0000 7fdeccaaa700 1 -- 172.21.15.78:0/3825562602 >> 172.21.15.78:0/3825562602 conn(0x7fdec80cea60 msgr2=0x7fdec80cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:39.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.965+0000 7fdeccaaa700 1 -- 172.21.15.78:0/3825562602 shutdown_connections 2023-10-11T09:41:39.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.965+0000 7fdeccaaa700 1 -- 172.21.15.78:0/3825562602 wait complete. 2023-10-11T09:41:39.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.969+0000 7fdeccaaa700 1 Processor -- start 2023-10-11T09:41:39.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.969+0000 7fdeccaaa700 1 -- start start 2023-10-11T09:41:39.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.969+0000 7fdeccaaa700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdec80caf20 0x7fdec8144cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:39.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.969+0000 7fdeccaaa700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdec80cda10 0x7fdec8154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:39.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.969+0000 7fdeccaaa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdec8164b50 0x7fdec80b2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:39.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.969+0000 7fdeccaaa700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec8145200 con 0x7fdec8164b50 2023-10-11T09:41:39.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.969+0000 7fdeccaaa700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec8162680 con 0x7fdec80cda10 2023-10-11T09:41:39.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.969+0000 7fdeccaaa700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec8155910 con 0x7fdec80caf20 2023-10-11T09:41:39.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.969+0000 7fdec7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdec8164b50 0x7fdec80b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:39.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.969+0000 7fdec7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdec8164b50 0x7fdec80b2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54460/0 (socket says 172.21.15.78:54460) 2023-10-11T09:41:39.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.969+0000 7fdec7fff700 1 -- 172.21.15.78:0/3090354339 learned_addr learned my addr 172.21.15.78:0/3090354339 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:39.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.969+0000 7fdec77fe700 1 --2- 172.21.15.78:0/3090354339 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdec80caf20 0x7fdec8144cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:39.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.969+0000 7fdec7fff700 1 -- 172.21.15.78:0/3090354339 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdec80caf20 msgr2=0x7fdec8144cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:39.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.969+0000 7fdec7fff700 1 --2- 172.21.15.78:0/3090354339 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdec80caf20 0x7fdec8144cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:39.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.969+0000 7fdec7fff700 1 -- 172.21.15.78:0/3090354339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdec80cda10 msgr2=0x7fdec8154fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:39.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.969+0000 7fdec6ffd700 1 --2- 172.21.15.78:0/3090354339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdec80cda10 0x7fdec8154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:39.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.969+0000 7fdec7fff700 1 --2- 172.21.15.78:0/3090354339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdec80cda10 0x7fdec8154fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:39.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.969+0000 7fdec7fff700 1 -- 172.21.15.78:0/3090354339 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec0009a70 con 0x7fdec8164b50 2023-10-11T09:41:39.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.969+0000 7fdec77fe700 1 --2- 172.21.15.78:0/3090354339 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdec80caf20 0x7fdec8144cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:41:39.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.969+0000 7fdec6ffd700 1 --2- 172.21.15.78:0/3090354339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdec80cda10 0x7fdec8154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:41:39.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.969+0000 7fdec7fff700 1 --2- 172.21.15.78:0/3090354339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdec8164b50 0x7fdec80b2560 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fdec0014040 tx=0x7fdec0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:39.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.969+0000 7fdeaffff700 1 -- 172.21.15.78:0/3090354339 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec00098f0 con 0x7fdec8164b50 2023-10-11T09:41:39.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.969+0000 7fdeccaaa700 1 -- 172.21.15.78:0/3090354339 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdec8155bf0 con 0x7fdec8164b50 2023-10-11T09:41:39.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.969+0000 7fdeaffff700 1 -- 172.21.15.78:0/3090354339 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec0020070 con 0x7fdec8164b50 2023-10-11T09:41:39.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.969+0000 7fdeaffff700 1 -- 172.21.15.78:0/3090354339 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec0004280 con 0x7fdec8164b50 2023-10-11T09:41:39.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.969+0000 7fdeccaaa700 1 -- 172.21.15.78:0/3090354339 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec8162310 con 0x7fdec8164b50 2023-10-11T09:41:39.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.973+0000 7fdeccaaa700 1 -- 172.21.15.78:0/3090354339 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdec8065d40 con 0x7fdec8164b50 2023-10-11T09:41:39.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.973+0000 7fdeaffff700 1 -- 172.21.15.78:0/3090354339 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fdec00043e0 con 0x7fdec8164b50 2023-10-11T09:41:39.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.977+0000 7fdeaffff700 1 --2- 172.21.15.78:0/3090354339 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdeb4065ed0 0x7fdeb4068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:39.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.977+0000 7fdeaffff700 1 -- 172.21.15.78:0/3090354339 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fdec0022070 con 0x7fdec8164b50 2023-10-11T09:41:39.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.977+0000 7fdec77fe700 1 --2- 172.21.15.78:0/3090354339 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdeb4065ed0 0x7fdeb4068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:39.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.977+0000 7fdec77fe700 1 --2- 172.21.15.78:0/3090354339 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdeb4065ed0 0x7fdeb4068380 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8005930 tx=0x7fdeb8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:39.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:39.981+0000 7fdeaffff700 1 -- 172.21.15.78:0/3090354339 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fdec0052cf0 con 0x7fdec8164b50 2023-10-11T09:41:40.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.153+0000 7fdeccaaa700 1 -- 172.21.15.78:0/3090354339 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdec81627c0 con 0x7fdec8164b50 2023-10-11T09:41:40.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.157+0000 7fdeaffff700 1 -- 172.21.15.78:0/3090354339 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17907 (secure 0 0 0) 0x7fdec0056310 con 0x7fdec8164b50 2023-10-11T09:41:40.163 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:41:40.164 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":25,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:41:21.473566+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:40:39.862581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T09:40:40.663641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.37","mappings":[{"from":3,"to":2}]},{"pgid":"4.3e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:41:40.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.161+0000 7fdeccaaa700 1 -- 172.21.15.78:0/3090354339 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdeb4065ed0 msgr2=0x7fdeb4068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:40.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.161+0000 7fdeccaaa700 1 --2- 172.21.15.78:0/3090354339 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdeb4065ed0 0x7fdeb4068380 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8005930 tx=0x7fdeb8009040 comp rx=0 tx=0).stop 2023-10-11T09:41:40.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.161+0000 7fdeccaaa700 1 -- 172.21.15.78:0/3090354339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdec8164b50 msgr2=0x7fdec80b2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:40.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.161+0000 7fdeccaaa700 1 --2- 172.21.15.78:0/3090354339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdec8164b50 0x7fdec80b2560 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fdec0014040 tx=0x7fdec0004af0 comp rx=0 tx=0).stop 2023-10-11T09:41:40.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.161+0000 7fdeccaaa700 1 -- 172.21.15.78:0/3090354339 shutdown_connections 2023-10-11T09:41:40.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.161+0000 7fdeccaaa700 1 --2- 172.21.15.78:0/3090354339 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdeb4065ed0 0x7fdeb4068380 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:40.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.161+0000 7fdeccaaa700 1 --2- 172.21.15.78:0/3090354339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdec8164b50 0x7fdec80b2560 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:40.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.161+0000 7fdeccaaa700 1 --2- 172.21.15.78:0/3090354339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdec80cda10 0x7fdec8154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:40.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.161+0000 7fdeccaaa700 1 --2- 172.21.15.78:0/3090354339 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdec80caf20 0x7fdec8144cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:40.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.161+0000 7fdeccaaa700 1 -- 172.21.15.78:0/3090354339 >> 172.21.15.78:0/3090354339 conn(0x7fdec80cea60 msgr2=0x7fdec806bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:40.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.161+0000 7fdeccaaa700 1 -- 172.21.15.78:0/3090354339 shutdown_connections 2023-10-11T09:41:40.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.161+0000 7fdeccaaa700 1 -- 172.21.15.78:0/3090354339 wait complete. 2023-10-11T09:41:40.189 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T09:40:39.862581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T09:40:40.663641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:41:40.190 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:41:40.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.353+0000 7f63f5345700 1 Processor -- start 2023-10-11T09:41:40.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.353+0000 7f63f5345700 1 -- start start 2023-10-11T09:41:40.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.357+0000 7f63f5345700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63f00b3240 0x7f63f00d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:40.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.357+0000 7f63f5345700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f63f00b3a90 0x7f63f00b85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:40.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.357+0000 7f63f5345700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63f00b47f0 con 0x7f63f00b3610 2023-10-11T09:41:40.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.357+0000 7f63f5345700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63f00b8130 con 0x7f63f00b3a90 2023-10-11T09:41:40.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.357+0000 7f63f5345700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f00b3f10 con 0x7f63f00b3240 2023-10-11T09:41:40.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.357+0000 7f63effff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63f00b3240 0x7f63f00d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:40.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.357+0000 7f63f4b44700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f63f00b3a90 0x7f63f00b85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:40.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.357+0000 7f63effff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63f00b3240 0x7f63f00d29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:33694/0 (socket says 172.21.15.78:33694) 2023-10-11T09:41:40.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.357+0000 7f63effff700 1 -- 172.21.15.78:0/1696043343 learned_addr learned my addr 172.21.15.78:0/1696043343 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:40.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.357+0000 7f63effff700 1 -- 172.21.15.78:0/1696043343 >> v2:172.21.15.133:3300/0 conn(0x7f63f00b3a90 msgr2=0x7f63f00b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:40.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.357+0000 7f63effff700 1 --2- 172.21.15.78:0/1696043343 >> v2:172.21.15.133:3300/0 conn(0x7f63f00b3a90 0x7f63f00b85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:40.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.357+0000 7f63effff700 1 -- 172.21.15.78:0/1696043343 >> v1:172.21.15.133:6789/0 conn(0x7f63f00b3610 legacy=0x7f63f015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:40.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.357+0000 7f63effff700 1 -- 172.21.15.78:0/1696043343 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f00b41e0 con 0x7f63f00b3240 2023-10-11T09:41:40.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.361+0000 7f63effff700 1 --2- 172.21.15.78:0/1696043343 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63f00b3240 0x7f63f00d29d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f63e0009910 tx=0x7f63e0009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a72d6b66911937cd server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:40.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.361+0000 7f63ef7fe700 1 -- 172.21.15.78:0/1696043343 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e0011830 con 0x7f63f00b3240 2023-10-11T09:41:40.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.361+0000 7f63ef7fe700 1 -- 172.21.15.78:0/1696043343 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e0011990 con 0x7f63f00b3240 2023-10-11T09:41:40.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.361+0000 7f63ef7fe700 1 -- 172.21.15.78:0/1696043343 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e0011d40 con 0x7f63f00b3240 2023-10-11T09:41:40.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.361+0000 7f63f5345700 1 -- 172.21.15.78:0/1696043343 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63f00b3240 msgr2=0x7f63f00d29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:40.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.361+0000 7f63f5345700 1 --2- 172.21.15.78:0/1696043343 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63f00b3240 0x7f63f00d29d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f63e0009910 tx=0x7f63e0009c20 comp rx=0 tx=0).stop 2023-10-11T09:41:40.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.361+0000 7f63f5345700 1 -- 172.21.15.78:0/1696043343 shutdown_connections 2023-10-11T09:41:40.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.361+0000 7f63f5345700 1 --2- 172.21.15.78:0/1696043343 >> v2:172.21.15.133:3300/0 conn(0x7f63f00b3a90 0x7f63f00b85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:40.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.361+0000 7f63f5345700 1 --2- 172.21.15.78:0/1696043343 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63f00b3240 0x7f63f00d29d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:40.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.361+0000 7f63f5345700 1 -- 172.21.15.78:0/1696043343 >> 172.21.15.78:0/1696043343 conn(0x7f63f009c260 msgr2=0x7f63f00d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:40.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.361+0000 7f63f5345700 1 -- 172.21.15.78:0/1696043343 shutdown_connections 2023-10-11T09:41:40.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.361+0000 7f63f5345700 1 -- 172.21.15.78:0/1696043343 wait complete. 2023-10-11T09:41:40.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.365+0000 7f63f5345700 1 Processor -- start 2023-10-11T09:41:40.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.365+0000 7f63f5345700 1 -- start start 2023-10-11T09:41:40.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.365+0000 7f63f5345700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63f00b3240 0x7f63f01f5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:40.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.365+0000 7f63f5345700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63f00b3610 0x7f63f015e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:40.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.365+0000 7f63f5345700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63f00b3a90 0x7f63f00a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:40.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.365+0000 7f63f5345700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f009f640 con 0x7f63f00b3610 2023-10-11T09:41:40.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.365+0000 7f63f5345700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f009f920 con 0x7f63f00b3a90 2023-10-11T09:41:40.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.365+0000 7f63f5345700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f009fc00 con 0x7f63f00b3240 2023-10-11T09:41:40.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.365+0000 7f63e77fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63f00b3610 0x7f63f015e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:40.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.365+0000 7f63e77fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63f00b3610 0x7f63f015e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54476/0 (socket says 172.21.15.78:54476) 2023-10-11T09:41:40.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.365+0000 7f63e77fe700 1 -- 172.21.15.78:0/2336898219 learned_addr learned my addr 172.21.15.78:0/2336898219 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:40.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.365+0000 7f63effff700 1 --2- 172.21.15.78:0/2336898219 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63f00b3240 0x7f63f01f5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:40.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.365+0000 7f63f4b44700 1 --2- 172.21.15.78:0/2336898219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63f00b3a90 0x7f63f00a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:40.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.365+0000 7f63e77fe700 1 -- 172.21.15.78:0/2336898219 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63f00b3240 msgr2=0x7f63f01f5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:40.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.365+0000 7f63e77fe700 1 --2- 172.21.15.78:0/2336898219 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63f00b3240 0x7f63f01f5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:40.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.365+0000 7f63e77fe700 1 -- 172.21.15.78:0/2336898219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63f00b3a90 msgr2=0x7f63f00a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:40.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.365+0000 7f63e77fe700 1 --2- 172.21.15.78:0/2336898219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63f00b3a90 0x7f63f00a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:40.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.365+0000 7f63e77fe700 1 -- 172.21.15.78:0/2336898219 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e0009570 con 0x7f63f00b3610 2023-10-11T09:41:40.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.369+0000 7f63e77fe700 1 --2- 172.21.15.78:0/2336898219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63f00b3610 0x7f63f015e5d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f63d800c430 tx=0x7f63d800c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:40.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.369+0000 7f63ed7fa700 1 -- 172.21.15.78:0/2336898219 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d80052d0 con 0x7f63f00b3610 2023-10-11T09:41:40.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.369+0000 7f63ed7fa700 1 -- 172.21.15.78:0/2336898219 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d8005430 con 0x7f63f00b3610 2023-10-11T09:41:40.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.369+0000 7f63f5345700 1 -- 172.21.15.78:0/2336898219 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63f009fee0 con 0x7f63f00b3610 2023-10-11T09:41:40.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.369+0000 7f63ed7fa700 1 -- 172.21.15.78:0/2336898219 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d80055a0 con 0x7f63f00b3610 2023-10-11T09:41:40.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.369+0000 7f63f5345700 1 -- 172.21.15.78:0/2336898219 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f009cf20 con 0x7f63f00b3610 2023-10-11T09:41:40.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.369+0000 7f63f5345700 1 -- 172.21.15.78:0/2336898219 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63f00a00d0 con 0x7f63f00b3610 2023-10-11T09:41:40.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.373+0000 7f63ed7fa700 1 -- 172.21.15.78:0/2336898219 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f63d80036a0 con 0x7f63f00b3610 2023-10-11T09:41:40.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.373+0000 7f63ed7fa700 1 --2- 172.21.15.78:0/2336898219 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f63d4065ed0 0x7f63d4068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:40.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.373+0000 7f63ed7fa700 1 -- 172.21.15.78:0/2336898219 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f63d8082850 con 0x7f63f00b3610 2023-10-11T09:41:40.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.373+0000 7f63effff700 1 --2- 172.21.15.78:0/2336898219 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f63d4065ed0 0x7f63d4068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:40.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.373+0000 7f63effff700 1 --2- 172.21.15.78:0/2336898219 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f63d4065ed0 0x7f63d4068380 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f63e0000c00 tx=0x7f63e0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:40.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.381+0000 7f63ed7fa700 1 -- 172.21.15.78:0/2336898219 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f63d804fdc0 con 0x7f63f00b3610 2023-10-11T09:41:40.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.553+0000 7f63f5345700 1 -- 172.21.15.78:0/2336898219 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f63f009f780 con 0x7f63f00b3610 2023-10-11T09:41:40.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.553+0000 7f63ed7fa700 1 -- 172.21.15.78:0/2336898219 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f63f009f780 con 0x7f63f00b3610 2023-10-11T09:41:40.558 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:41:40.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.553+0000 7f63f5345700 1 -- 172.21.15.78:0/2336898219 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f63d4065ed0 msgr2=0x7f63d4068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:40.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.553+0000 7f63f5345700 1 --2- 172.21.15.78:0/2336898219 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f63d4065ed0 0x7f63d4068380 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f63e0000c00 tx=0x7f63e0008030 comp rx=0 tx=0).stop 2023-10-11T09:41:40.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.553+0000 7f63f5345700 1 -- 172.21.15.78:0/2336898219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63f00b3610 msgr2=0x7f63f015e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:40.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.553+0000 7f63f5345700 1 --2- 172.21.15.78:0/2336898219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63f00b3610 0x7f63f015e5d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f63d800c430 tx=0x7f63d800c740 comp rx=0 tx=0).stop 2023-10-11T09:41:40.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.557+0000 7f63f5345700 1 -- 172.21.15.78:0/2336898219 shutdown_connections 2023-10-11T09:41:40.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.557+0000 7f63f5345700 1 --2- 172.21.15.78:0/2336898219 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f63d4065ed0 0x7f63d4068380 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:40.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.557+0000 7f63f5345700 1 --2- 172.21.15.78:0/2336898219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63f00b3a90 0x7f63f00a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:40.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.557+0000 7f63f5345700 1 --2- 172.21.15.78:0/2336898219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63f00b3610 0x7f63f015e5d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:40.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.557+0000 7f63f5345700 1 --2- 172.21.15.78:0/2336898219 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63f00b3240 0x7f63f01f5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:40.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.557+0000 7f63f5345700 1 -- 172.21.15.78:0/2336898219 >> 172.21.15.78:0/2336898219 conn(0x7f63f009c260 msgr2=0x7f63f00d0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:40.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.557+0000 7f63f5345700 1 -- 172.21.15.78:0/2336898219 shutdown_connections 2023-10-11T09:41:40.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.557+0000 7f63f5345700 1 -- 172.21.15.78:0/2336898219 wait complete. 2023-10-11T09:41:40.584 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:41:40.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.757+0000 7f069f190700 1 Processor -- start 2023-10-11T09:41:40.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.757+0000 7f069f190700 1 -- start start 2023-10-11T09:41:40.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.761+0000 7f069f190700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0698157130 0x7f069815a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:40.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.761+0000 7f069f190700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f069806d400 0x7f06980705b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:40.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.761+0000 7f069f190700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0698070af0 con 0x7f06980ce240 2023-10-11T09:41:40.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.761+0000 7f069f190700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06980b1aa0 con 0x7f069806d400 2023-10-11T09:41:40.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.761+0000 7f069f190700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06980ce690 con 0x7f0698157130 2023-10-11T09:41:40.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.761+0000 7f069e98f700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f069806d400 0x7f06980705b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:40.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.761+0000 7f069e98f700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f069806d400 0x7f06980705b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:49418/0 (socket says 172.21.15.78:49418) 2023-10-11T09:41:40.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.761+0000 7f069d98d700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f06980ce240 0x7f069806ccf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:34932/0 (socket says 172.21.15.78:34932) 2023-10-11T09:41:40.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.761+0000 7f069e98f700 1 -- 172.21.15.78:0/573520255 learned_addr learned my addr 172.21.15.78:0/573520255 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:40.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.761+0000 7f069e18e700 1 --2- 172.21.15.78:0/573520255 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0698157130 0x7f069815a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:40.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.761+0000 7f069e98f700 1 -- 172.21.15.78:0/573520255 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0698157130 msgr2=0x7f069815a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:40.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.761+0000 7f069e98f700 1 --2- 172.21.15.78:0/573520255 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0698157130 0x7f069815a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:40.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.761+0000 7f069e98f700 1 -- 172.21.15.78:0/573520255 >> v1:172.21.15.133:6789/0 conn(0x7f06980ce240 legacy=0x7f069806ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:40.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.761+0000 7f069e98f700 1 -- 172.21.15.78:0/573520255 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0698157500 con 0x7f069806d400 2023-10-11T09:41:40.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.761+0000 7f069d18c700 1 -- 172.21.15.78:0/573520255 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3021936744 0 0) 0x7f0698070af0 con 0x7f06980ce240 2023-10-11T09:41:40.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.761+0000 7f069e98f700 1 --2- 172.21.15.78:0/573520255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f069806d400 0x7f06980705b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0690009e50 tx=0x7f069000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21c8e3e390aff037 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:40.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.761+0000 7f069d18c700 1 -- 172.21.15.78:0/573520255 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069000e070 con 0x7f069806d400 2023-10-11T09:41:40.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.761+0000 7f069d18c700 1 -- 172.21.15.78:0/573520255 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f069000b910 con 0x7f069806d400 2023-10-11T09:41:40.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.761+0000 7f069d18c700 1 -- 172.21.15.78:0/573520255 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069000bcc0 con 0x7f069806d400 2023-10-11T09:41:40.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.765+0000 7f069f190700 1 -- 172.21.15.78:0/573520255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f069806d400 msgr2=0x7f06980705b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:40.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.765+0000 7f069f190700 1 --2- 172.21.15.78:0/573520255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f069806d400 0x7f06980705b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0690009e50 tx=0x7f069000b330 comp rx=0 tx=0).stop 2023-10-11T09:41:40.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.765+0000 7f069f190700 1 -- 172.21.15.78:0/573520255 shutdown_connections 2023-10-11T09:41:40.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.765+0000 7f069f190700 1 --2- 172.21.15.78:0/573520255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f069806d400 0x7f06980705b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:40.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.765+0000 7f069f190700 1 --2- 172.21.15.78:0/573520255 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0698157130 0x7f069815a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:40.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.765+0000 7f069f190700 1 -- 172.21.15.78:0/573520255 >> 172.21.15.78:0/573520255 conn(0x7f069806a060 msgr2=0x7f069806a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:40.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.765+0000 7f069f190700 1 -- 172.21.15.78:0/573520255 shutdown_connections 2023-10-11T09:41:40.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.765+0000 7f069f190700 1 -- 172.21.15.78:0/573520255 wait complete. 2023-10-11T09:41:40.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.769+0000 7f069f190700 1 Processor -- start 2023-10-11T09:41:40.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.769+0000 7f069f190700 1 -- start start 2023-10-11T09:41:40.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.769+0000 7f069f190700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f069806d400 0x7f06981f1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:40.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.769+0000 7f069f190700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06980ce240 0x7f069806bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:40.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.769+0000 7f069f190700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0698157130 0x7f069806c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:40.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.769+0000 7f069f190700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06980b0ed0 con 0x7f06980ce240 2023-10-11T09:41:40.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.769+0000 7f069f190700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06980aedc0 con 0x7f0698157130 2023-10-11T09:41:40.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.769+0000 7f069f190700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06981edb30 con 0x7f069806d400 2023-10-11T09:41:40.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.769+0000 7f069d98d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06980ce240 0x7f069806bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:40.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.769+0000 7f069d98d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06980ce240 0x7f069806bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54478/0 (socket says 172.21.15.78:54478) 2023-10-11T09:41:40.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.769+0000 7f069e18e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f069806d400 0x7f06981f1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:40.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.769+0000 7f069d98d700 1 -- 172.21.15.78:0/2701472417 learned_addr learned my addr 172.21.15.78:0/2701472417 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:40.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.769+0000 7f069e98f700 1 --2- 172.21.15.78:0/2701472417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0698157130 0x7f069806c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:40.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.773+0000 7f069d98d700 1 -- 172.21.15.78:0/2701472417 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f069806d400 msgr2=0x7f06981f1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:40.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.773+0000 7f069d98d700 1 --2- 172.21.15.78:0/2701472417 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f069806d400 0x7f06981f1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:40.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.773+0000 7f069d98d700 1 -- 172.21.15.78:0/2701472417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0698157130 msgr2=0x7f069806c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:40.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.773+0000 7f069d98d700 1 --2- 172.21.15.78:0/2701472417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0698157130 0x7f069806c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:40.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.773+0000 7f069d98d700 1 -- 172.21.15.78:0/2701472417 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0690009b30 con 0x7f06980ce240 2023-10-11T09:41:40.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.773+0000 7f069d98d700 1 --2- 172.21.15.78:0/2701472417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06980ce240 0x7f069806bbe0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f068800c4e0 tx=0x7f068800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:40.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.773+0000 7f068effd700 1 -- 172.21.15.78:0/2701472417 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0688004ca0 con 0x7f06980ce240 2023-10-11T09:41:40.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.773+0000 7f068effd700 1 -- 172.21.15.78:0/2701472417 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f068800ce90 con 0x7f06980ce240 2023-10-11T09:41:40.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.773+0000 7f069f190700 1 -- 172.21.15.78:0/2701472417 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06981f2050 con 0x7f06980ce240 2023-10-11T09:41:40.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.773+0000 7f068effd700 1 -- 172.21.15.78:0/2701472417 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0688007310 con 0x7f06980ce240 2023-10-11T09:41:40.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.773+0000 7f069f190700 1 -- 172.21.15.78:0/2701472417 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06980ae800 con 0x7f06980ce240 2023-10-11T09:41:40.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.773+0000 7f069f190700 1 -- 172.21.15.78:0/2701472417 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06981edcd0 con 0x7f06980ce240 2023-10-11T09:41:40.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.777+0000 7f068effd700 1 -- 172.21.15.78:0/2701472417 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f0688007470 con 0x7f06980ce240 2023-10-11T09:41:40.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.777+0000 7f068effd700 1 --2- 172.21.15.78:0/2701472417 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f067c065f00 0x7f067c0683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:40.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.777+0000 7f068effd700 1 -- 172.21.15.78:0/2701472417 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f0688083320 con 0x7f06980ce240 2023-10-11T09:41:40.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.777+0000 7f069e18e700 1 --2- 172.21.15.78:0/2701472417 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f067c065f00 0x7f067c0683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:40.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.777+0000 7f069e18e700 1 --2- 172.21.15.78:0/2701472417 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f067c065f00 0x7f067c0683b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0698151190 tx=0x7f069400b450 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:40.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.781+0000 7f068effd700 1 -- 172.21.15.78:0/2701472417 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f06981edcd0 con 0x7f06980ce240 2023-10-11T09:41:40.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.957+0000 7f069f190700 1 -- 172.21.15.78:0/2701472417 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f06981edc70 con 0x7f06980ce240 2023-10-11T09:41:40.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.957+0000 7f068effd700 1 -- 172.21.15.78:0/2701472417 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0688050920 con 0x7f06980ce240 2023-10-11T09:41:40.962 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:41:40.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.957+0000 7f069f190700 1 -- 172.21.15.78:0/2701472417 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f067c065f00 msgr2=0x7f067c0683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:40.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.957+0000 7f069f190700 1 --2- 172.21.15.78:0/2701472417 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f067c065f00 0x7f067c0683b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0698151190 tx=0x7f069400b450 comp rx=0 tx=0).stop 2023-10-11T09:41:40.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.957+0000 7f069f190700 1 -- 172.21.15.78:0/2701472417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06980ce240 msgr2=0x7f069806bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:40.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.957+0000 7f069f190700 1 --2- 172.21.15.78:0/2701472417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06980ce240 0x7f069806bbe0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f068800c4e0 tx=0x7f068800c7f0 comp rx=0 tx=0).stop 2023-10-11T09:41:40.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.961+0000 7f069f190700 1 -- 172.21.15.78:0/2701472417 shutdown_connections 2023-10-11T09:41:40.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.961+0000 7f069f190700 1 --2- 172.21.15.78:0/2701472417 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f067c065f00 0x7f067c0683b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:40.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.961+0000 7f069f190700 1 --2- 172.21.15.78:0/2701472417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0698157130 0x7f069806c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:40.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.961+0000 7f069f190700 1 --2- 172.21.15.78:0/2701472417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06980ce240 0x7f069806bbe0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:40.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.961+0000 7f069f190700 1 --2- 172.21.15.78:0/2701472417 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f069806d400 0x7f06981f1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:40.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.961+0000 7f069f190700 1 -- 172.21.15.78:0/2701472417 >> 172.21.15.78:0/2701472417 conn(0x7f069806a060 msgr2=0x7f069815b060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:40.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.961+0000 7f069f190700 1 -- 172.21.15.78:0/2701472417 shutdown_connections 2023-10-11T09:41:40.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:40.961+0000 7f069f190700 1 -- 172.21.15.78:0/2701472417 wait complete. 2023-10-11T09:41:40.987 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T09:41:41.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.149+0000 7f5475f58700 1 Processor -- start 2023-10-11T09:41:41.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.149+0000 7f5475f58700 1 -- start start 2023-10-11T09:41:41.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.153+0000 7f5475f58700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f54701533a0 0x7f5470159200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:41.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.153+0000 7f5475f58700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f54700b2120 0x7f547006c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:41.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.153+0000 7f5475f58700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f547015a1b0 con 0x7f547015d270 2023-10-11T09:41:41.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.153+0000 7f5475f58700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5470153770 con 0x7f54700b2120 2023-10-11T09:41:41.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.153+0000 7f5475f58700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f547006bd10 con 0x7f54701533a0 2023-10-11T09:41:41.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.153+0000 7f5467fff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f547015d270 0x7f54700b1a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33210/0 (socket says 172.21.15.78:33210) 2023-10-11T09:41:41.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.153+0000 7f5467fff700 1 -- 172.21.15.78:0/1161688413 learned_addr learned my addr 172.21.15.78:0/1161688413 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:41.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.153+0000 7f5474f56700 1 --2- 172.21.15.78:0/1161688413 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f54701533a0 0x7f5470159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:41.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.153+0000 7f5475757700 1 --2- 172.21.15.78:0/1161688413 >> v2:172.21.15.133:3300/0 conn(0x7f54700b2120 0x7f547006c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:41.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.153+0000 7f54677fe700 1 -- 172.21.15.78:0/1161688413 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2298224750 0 0) 0x7f547015a1b0 con 0x7f547015d270 2023-10-11T09:41:41.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.153+0000 7f54677fe700 1 -- 172.21.15.78:0/1161688413 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54580057e0 con 0x7f547015d270 2023-10-11T09:41:41.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.153+0000 7f54677fe700 1 -- 172.21.15.78:0/1161688413 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3863936647 0 0) 0x7f54580057e0 con 0x7f547015d270 2023-10-11T09:41:41.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.153+0000 7f5474f56700 1 -- 172.21.15.78:0/1161688413 >> v2:172.21.15.133:3300/0 conn(0x7f54700b2120 msgr2=0x7f547006c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:41.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.153+0000 7f5474f56700 1 --2- 172.21.15.78:0/1161688413 >> v2:172.21.15.133:3300/0 conn(0x7f54700b2120 0x7f547006c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:41.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.153+0000 7f5474f56700 1 -- 172.21.15.78:0/1161688413 >> v1:172.21.15.78:6789/0 conn(0x7f547015d270 legacy=0x7f54700b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:41.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.153+0000 7f5474f56700 1 -- 172.21.15.78:0/1161688413 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5470159910 con 0x7f54701533a0 2023-10-11T09:41:41.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.153+0000 7f5475757700 1 --2- 172.21.15.78:0/1161688413 >> v2:172.21.15.133:3300/0 conn(0x7f54700b2120 0x7f547006c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:41:41.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.153+0000 7f5474f56700 1 --2- 172.21.15.78:0/1161688413 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f54701533a0 0x7f5470159200 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5460009d90 tx=0x7f546000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1c07ec942893acd server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:41.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.157+0000 7f54677fe700 1 -- 172.21.15.78:0/1161688413 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546000e070 con 0x7f54701533a0 2023-10-11T09:41:41.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.157+0000 7f54677fe700 1 -- 172.21.15.78:0/1161688413 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5460003a40 con 0x7f54701533a0 2023-10-11T09:41:41.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.157+0000 7f54677fe700 1 -- 172.21.15.78:0/1161688413 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546c001670 con 0x7f54701533a0 2023-10-11T09:41:41.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.157+0000 7f5475f58700 1 -- 172.21.15.78:0/1161688413 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f54701533a0 msgr2=0x7f5470159200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:41.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.157+0000 7f5475f58700 1 --2- 172.21.15.78:0/1161688413 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f54701533a0 0x7f5470159200 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5460009d90 tx=0x7f546000b330 comp rx=0 tx=0).stop 2023-10-11T09:41:41.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.157+0000 7f5475f58700 1 -- 172.21.15.78:0/1161688413 shutdown_connections 2023-10-11T09:41:41.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.157+0000 7f5475f58700 1 --2- 172.21.15.78:0/1161688413 >> v2:172.21.15.133:3300/0 conn(0x7f54700b2120 0x7f547006c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:41.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.157+0000 7f5475f58700 1 --2- 172.21.15.78:0/1161688413 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f54701533a0 0x7f5470159200 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:41.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.157+0000 7f5475f58700 1 -- 172.21.15.78:0/1161688413 >> 172.21.15.78:0/1161688413 conn(0x7f547006a060 msgr2=0x7f547006a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:41.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.157+0000 7f5475f58700 1 -- 172.21.15.78:0/1161688413 shutdown_connections 2023-10-11T09:41:41.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.157+0000 7f5475f58700 1 -- 172.21.15.78:0/1161688413 wait complete. 2023-10-11T09:41:41.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.157+0000 7f5475f58700 1 Processor -- start 2023-10-11T09:41:41.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.157+0000 7f5475f58700 1 -- start start 2023-10-11T09:41:41.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.161+0000 7f5475f58700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f54700b2120 0x7f54701f2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:41.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.161+0000 7f5475f58700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54701533a0 0x7f54700a9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:41.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.161+0000 7f5475f58700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f547015d270 0x7f5470157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:41.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.161+0000 7f5475f58700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547013ec90 con 0x7f54701533a0 2023-10-11T09:41:41.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.161+0000 7f5475f58700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54701f2d50 con 0x7f547015d270 2023-10-11T09:41:41.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.161+0000 7f5475f58700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f547013d010 con 0x7f54700b2120 2023-10-11T09:41:41.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.161+0000 7f5467fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54701533a0 0x7f54700a9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:41.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.161+0000 7f5467fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54701533a0 0x7f54700a9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54494/0 (socket says 172.21.15.78:54494) 2023-10-11T09:41:41.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.161+0000 7f5467fff700 1 -- 172.21.15.78:0/2327638598 learned_addr learned my addr 172.21.15.78:0/2327638598 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:41.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.161+0000 7f5474f56700 1 --2- 172.21.15.78:0/2327638598 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f54700b2120 0x7f54701f2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:41.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.161+0000 7f5475757700 1 --2- 172.21.15.78:0/2327638598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f547015d270 0x7f5470157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:41.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.161+0000 7f5467fff700 1 -- 172.21.15.78:0/2327638598 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f54700b2120 msgr2=0x7f54701f2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:41.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.161+0000 7f5467fff700 1 --2- 172.21.15.78:0/2327638598 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f54700b2120 0x7f54701f2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:41.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.161+0000 7f5467fff700 1 -- 172.21.15.78:0/2327638598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f547015d270 msgr2=0x7f5470157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:41.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.161+0000 7f5467fff700 1 --2- 172.21.15.78:0/2327638598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f547015d270 0x7f5470157cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:41.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.161+0000 7f5467fff700 1 -- 172.21.15.78:0/2327638598 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54680094b0 con 0x7f54701533a0 2023-10-11T09:41:41.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.161+0000 7f5475757700 1 --2- 172.21.15.78:0/2327638598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f547015d270 0x7f5470157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:41.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.161+0000 7f5467fff700 1 --2- 172.21.15.78:0/2327638598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54701533a0 0x7f54700a9c20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f546c0011e0 tx=0x7f546c00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:41.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.161+0000 7f54657fa700 1 -- 172.21.15.78:0/2327638598 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546c004ca0 con 0x7f54701533a0 2023-10-11T09:41:41.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.161+0000 7f54657fa700 1 -- 172.21.15.78:0/2327638598 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f546c004e20 con 0x7f54701533a0 2023-10-11T09:41:41.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.161+0000 7f54657fa700 1 -- 172.21.15.78:0/2327638598 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546c007310 con 0x7f54701533a0 2023-10-11T09:41:41.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.161+0000 7f5475f58700 1 -- 172.21.15.78:0/2327638598 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5460009a70 con 0x7f54701533a0 2023-10-11T09:41:41.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.161+0000 7f5475f58700 1 -- 172.21.15.78:0/2327638598 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f547013d5d0 con 0x7f54701533a0 2023-10-11T09:41:41.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.161+0000 7f5475f58700 1 -- 172.21.15.78:0/2327638598 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5470065d40 con 0x7f54701533a0 2023-10-11T09:41:41.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.165+0000 7f54657fa700 1 -- 172.21.15.78:0/2327638598 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f546c007470 con 0x7f54701533a0 2023-10-11T09:41:41.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.165+0000 7f54657fa700 1 --2- 172.21.15.78:0/2327638598 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f545c065ed0 0x7f545c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:41.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.165+0000 7f54657fa700 1 -- 172.21.15.78:0/2327638598 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f546c0835c0 con 0x7f54701533a0 2023-10-11T09:41:41.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.165+0000 7f5474f56700 1 --2- 172.21.15.78:0/2327638598 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f545c065ed0 0x7f545c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:41.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.165+0000 7f5474f56700 1 --2- 172.21.15.78:0/2327638598 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f545c065ed0 0x7f545c068380 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f546000b370 tx=0x7f5460011000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:41.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.173+0000 7f54657fa700 1 -- 172.21.15.78:0/2327638598 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f546c050be0 con 0x7f54701533a0 2023-10-11T09:41:41.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.345+0000 7f5475f58700 1 -- 172.21.15.78:0/2327638598 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f547006b570 con 0x7f54701533a0 2023-10-11T09:41:41.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.345+0000 7f54657fa700 1 -- 172.21.15.78:0/2327638598 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f546c007720 con 0x7f54701533a0 2023-10-11T09:41:41.352 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:41:41.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.349+0000 7f5475f58700 1 -- 172.21.15.78:0/2327638598 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f545c065ed0 msgr2=0x7f545c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:41.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.349+0000 7f5475f58700 1 --2- 172.21.15.78:0/2327638598 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f545c065ed0 0x7f545c068380 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f546000b370 tx=0x7f5460011000 comp rx=0 tx=0).stop 2023-10-11T09:41:41.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.349+0000 7f5475f58700 1 -- 172.21.15.78:0/2327638598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54701533a0 msgr2=0x7f54700a9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:41.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.349+0000 7f5475f58700 1 --2- 172.21.15.78:0/2327638598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54701533a0 0x7f54700a9c20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f546c0011e0 tx=0x7f546c00c9e0 comp rx=0 tx=0).stop 2023-10-11T09:41:41.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.349+0000 7f5475f58700 1 -- 172.21.15.78:0/2327638598 shutdown_connections 2023-10-11T09:41:41.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.349+0000 7f5475f58700 1 --2- 172.21.15.78:0/2327638598 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f545c065ed0 0x7f545c068380 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:41.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.349+0000 7f5475f58700 1 --2- 172.21.15.78:0/2327638598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f547015d270 0x7f5470157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:41.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.349+0000 7f5475f58700 1 --2- 172.21.15.78:0/2327638598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54701533a0 0x7f54700a9c20 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:41.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.349+0000 7f5475f58700 1 --2- 172.21.15.78:0/2327638598 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f54700b2120 0x7f54701f2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:41.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.349+0000 7f5475f58700 1 -- 172.21.15.78:0/2327638598 >> 172.21.15.78:0/2327638598 conn(0x7f547006a060 msgr2=0x7f547006c090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:41.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.353+0000 7f5475f58700 1 -- 172.21.15.78:0/2327638598 shutdown_connections 2023-10-11T09:41:41.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.353+0000 7f5475f58700 1 -- 172.21.15.78:0/2327638598 wait complete. 2023-10-11T09:41:41.378 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T09:41:41.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.549+0000 7f99ac9ef700 1 Processor -- start 2023-10-11T09:41:41.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.549+0000 7f99ac9ef700 1 -- start start 2023-10-11T09:41:41.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.549+0000 7f99ac9ef700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f99a80d29d0 0x7f99a80b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:41.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.549+0000 7f99ac9ef700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a80b47f0 con 0x7f99a80b3610 2023-10-11T09:41:41.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.549+0000 7f99ac9ef700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a80b8130 con 0x7f99a80b3240 2023-10-11T09:41:41.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.549+0000 7f99ac9ef700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99a80d2e30 con 0x7f99a80d29d0 2023-10-11T09:41:41.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.549+0000 7f99a7fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f99a80d29d0 0x7f99a80b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:41.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.553+0000 7f99a6ffd700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f99a80b3610 0x7f99a80b85c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33220/0 (socket says 172.21.15.78:33220) 2023-10-11T09:41:41.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.553+0000 7f99a7fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f99a80d29d0 0x7f99a80b3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54496/0 (socket says 172.21.15.78:54496) 2023-10-11T09:41:41.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.553+0000 7f99a6ffd700 1 -- 172.21.15.78:0/1618883964 learned_addr learned my addr 172.21.15.78:0/1618883964 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:41.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.553+0000 7f99a7fff700 1 -- 172.21.15.78:0/1618883964 >> v1:172.21.15.133:6789/0 conn(0x7f99a80b3240 legacy=0x7f99a815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:41.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.553+0000 7f99a7fff700 1 -- 172.21.15.78:0/1618883964 >> v1:172.21.15.78:6789/0 conn(0x7f99a80b3610 legacy=0x7f99a80b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:41.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.553+0000 7f99a7fff700 1 -- 172.21.15.78:0/1618883964 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a80b41e0 con 0x7f99a80d29d0 2023-10-11T09:41:41.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.553+0000 7f99a7fff700 1 --2- 172.21.15.78:0/1618883964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99a80d29d0 0x7f99a80b3a90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f99a0009d90 tx=0x7f99a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f8833d50ff72f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:41.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.553+0000 7f99a67fc700 1 -- 172.21.15.78:0/1618883964 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a000e070 con 0x7f99a80d29d0 2023-10-11T09:41:41.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.553+0000 7f99a67fc700 1 -- 172.21.15.78:0/1618883964 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a000bbd0 con 0x7f99a80d29d0 2023-10-11T09:41:41.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.553+0000 7f99a67fc700 1 -- 172.21.15.78:0/1618883964 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a0012430 con 0x7f99a80d29d0 2023-10-11T09:41:41.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.553+0000 7f99ac9ef700 1 -- 172.21.15.78:0/1618883964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99a80d29d0 msgr2=0x7f99a80b3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:41.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.553+0000 7f99ac9ef700 1 --2- 172.21.15.78:0/1618883964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99a80d29d0 0x7f99a80b3a90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f99a0009d90 tx=0x7f99a000b330 comp rx=0 tx=0).stop 2023-10-11T09:41:41.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.553+0000 7f99ac9ef700 1 -- 172.21.15.78:0/1618883964 shutdown_connections 2023-10-11T09:41:41.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.553+0000 7f99ac9ef700 1 --2- 172.21.15.78:0/1618883964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99a80d29d0 0x7f99a80b3a90 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:41.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.553+0000 7f99ac9ef700 1 -- 172.21.15.78:0/1618883964 >> 172.21.15.78:0/1618883964 conn(0x7f99a809c260 msgr2=0x7f99a80d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:41.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.553+0000 7f99ac9ef700 1 -- 172.21.15.78:0/1618883964 shutdown_connections 2023-10-11T09:41:41.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.553+0000 7f99ac9ef700 1 -- 172.21.15.78:0/1618883964 wait complete. 2023-10-11T09:41:41.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.557+0000 7f99ac9ef700 1 Processor -- start 2023-10-11T09:41:41.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.557+0000 7f99ac9ef700 1 -- start start 2023-10-11T09:41:41.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.561+0000 7f99ac9ef700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99a80b3240 0x7f99a80c6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:41.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.561+0000 7f99ac9ef700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99a80b3610 0x7f99a814be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:41.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.561+0000 7f99ac9ef700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f99a80d29d0 0x7f99a815f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:41.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.561+0000 7f99ac9ef700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a815e610 con 0x7f99a80b3610 2023-10-11T09:41:41.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.561+0000 7f99ac9ef700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a815e8f0 con 0x7f99a80b3240 2023-10-11T09:41:41.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.561+0000 7f99ac9ef700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a815ebd0 con 0x7f99a80d29d0 2023-10-11T09:41:41.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.561+0000 7f99a6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99a80b3610 0x7f99a814be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:41.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.561+0000 7f99a6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99a80b3610 0x7f99a814be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54500/0 (socket says 172.21.15.78:54500) 2023-10-11T09:41:41.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.561+0000 7f99a6ffd700 1 -- 172.21.15.78:0/1414202490 learned_addr learned my addr 172.21.15.78:0/1414202490 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:41.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.561+0000 7f99a77fe700 1 --2- 172.21.15.78:0/1414202490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99a80b3240 0x7f99a80c6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:41.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.561+0000 7f99a7fff700 1 --2- 172.21.15.78:0/1414202490 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f99a80d29d0 0x7f99a815f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:41.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.561+0000 7f99a6ffd700 1 -- 172.21.15.78:0/1414202490 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f99a80d29d0 msgr2=0x7f99a815f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:41.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.561+0000 7f99a6ffd700 1 --2- 172.21.15.78:0/1414202490 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f99a80d29d0 0x7f99a815f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:41.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.561+0000 7f99a6ffd700 1 -- 172.21.15.78:0/1414202490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99a80b3240 msgr2=0x7f99a80c6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:41.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.561+0000 7f99a6ffd700 1 --2- 172.21.15.78:0/1414202490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99a80b3240 0x7f99a80c6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:41.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.561+0000 7f99a6ffd700 1 -- 172.21.15.78:0/1414202490 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a0009a70 con 0x7f99a80b3610 2023-10-11T09:41:41.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.561+0000 7f99a6ffd700 1 --2- 172.21.15.78:0/1414202490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99a80b3610 0x7f99a814be00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f999c001eb0 tx=0x7f999c00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:41.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.561+0000 7f998ffff700 1 -- 172.21.15.78:0/1414202490 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c005d90 con 0x7f99a80b3610 2023-10-11T09:41:41.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.561+0000 7f998ffff700 1 -- 172.21.15.78:0/1414202490 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f999c0070d0 con 0x7f99a80b3610 2023-10-11T09:41:41.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.561+0000 7f99ac9ef700 1 -- 172.21.15.78:0/1414202490 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a815eeb0 con 0x7f99a80b3610 2023-10-11T09:41:41.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.561+0000 7f998ffff700 1 -- 172.21.15.78:0/1414202490 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c007440 con 0x7f99a80b3610 2023-10-11T09:41:41.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.561+0000 7f99ac9ef700 1 -- 172.21.15.78:0/1414202490 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a80c7270 con 0x7f99a80b3610 2023-10-11T09:41:41.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.565+0000 7f99ac9ef700 1 -- 172.21.15.78:0/1414202490 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99a80c7430 con 0x7f99a80b3610 2023-10-11T09:41:41.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.565+0000 7f998ffff700 1 -- 172.21.15.78:0/1414202490 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f999c0075a0 con 0x7f99a80b3610 2023-10-11T09:41:41.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.569+0000 7f998ffff700 1 --2- 172.21.15.78:0/1414202490 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9994065ed0 0x7f9994068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:41.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.569+0000 7f998ffff700 1 -- 172.21.15.78:0/1414202490 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f999c0829f0 con 0x7f99a80b3610 2023-10-11T09:41:41.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.569+0000 7f99a77fe700 1 --2- 172.21.15.78:0/1414202490 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9994065ed0 0x7f9994068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:41.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.569+0000 7f99a77fe700 1 --2- 172.21.15.78:0/1414202490 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9994065ed0 0x7f9994068380 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9998000f60 tx=0x7f9998009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:41.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.573+0000 7f998ffff700 1 -- 172.21.15.78:0/1414202490 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f999c050010 con 0x7f99a80b3610 2023-10-11T09:41:41.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.753+0000 7f99ac9ef700 1 -- 172.21.15.78:0/1414202490 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f99a8065d40 con 0x7f99a80b3610 2023-10-11T09:41:41.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.753+0000 7f998ffff700 1 -- 172.21.15.78:0/1414202490 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f999c0170a0 con 0x7f99a80b3610 2023-10-11T09:41:41.759 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2023-10-11T09:41:41.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.757+0000 7f99ac9ef700 1 -- 172.21.15.78:0/1414202490 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9994065ed0 msgr2=0x7f9994068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:41.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.757+0000 7f99ac9ef700 1 --2- 172.21.15.78:0/1414202490 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9994065ed0 0x7f9994068380 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9998000f60 tx=0x7f9998009040 comp rx=0 tx=0).stop 2023-10-11T09:41:41.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.757+0000 7f99ac9ef700 1 -- 172.21.15.78:0/1414202490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99a80b3610 msgr2=0x7f99a814be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:41.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.757+0000 7f99ac9ef700 1 --2- 172.21.15.78:0/1414202490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99a80b3610 0x7f99a814be00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f999c001eb0 tx=0x7f999c00c7b0 comp rx=0 tx=0).stop 2023-10-11T09:41:41.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.757+0000 7f99ac9ef700 1 -- 172.21.15.78:0/1414202490 shutdown_connections 2023-10-11T09:41:41.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.757+0000 7f99ac9ef700 1 --2- 172.21.15.78:0/1414202490 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9994065ed0 0x7f9994068380 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:41.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.757+0000 7f99ac9ef700 1 --2- 172.21.15.78:0/1414202490 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f99a80d29d0 0x7f99a815f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:41.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.757+0000 7f99ac9ef700 1 --2- 172.21.15.78:0/1414202490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99a80b3610 0x7f99a814be00 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:41.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.757+0000 7f99ac9ef700 1 --2- 172.21.15.78:0/1414202490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99a80b3240 0x7f99a80c6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:41.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.757+0000 7f99ac9ef700 1 -- 172.21.15.78:0/1414202490 >> 172.21.15.78:0/1414202490 conn(0x7f99a809c260 msgr2=0x7f99a806c610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:41.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.757+0000 7f99ac9ef700 1 -- 172.21.15.78:0/1414202490 shutdown_connections 2023-10-11T09:41:41.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.757+0000 7f99ac9ef700 1 -- 172.21.15.78:0/1414202490 wait complete. 2023-10-11T09:41:41.785 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T09:41:41.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.957+0000 7f779a1fc700 1 Processor -- start 2023-10-11T09:41:41.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.957+0000 7f779a1fc700 1 -- start start 2023-10-11T09:41:41.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.961+0000 7f779a1fc700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77940b2800 0x7f77940cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:41.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.961+0000 7f779a1fc700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77940cf6e0 con 0x7f779406dd80 2023-10-11T09:41:41.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.961+0000 7f779a1fc700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77940cead0 con 0x7f77940d1a40 2023-10-11T09:41:41.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.961+0000 7f779a1fc700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77940ce8c0 con 0x7f77940b2800 2023-10-11T09:41:41.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.961+0000 7f77991fa700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f779406dd80 0x7f779406e150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33228/0 (socket says 172.21.15.78:33228) 2023-10-11T09:41:41.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.961+0000 7f77991fa700 1 -- 172.21.15.78:0/128164321 learned_addr learned my addr 172.21.15.78:0/128164321 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:41.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.961+0000 7f77999fb700 1 --2- 172.21.15.78:0/128164321 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77940b2800 0x7f77940cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:41.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.961+0000 7f778bfff700 1 -- 172.21.15.78:0/128164321 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1557357558 0 0) 0x7f77940cead0 con 0x7f77940d1a40 2023-10-11T09:41:41.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.961+0000 7f778bfff700 1 -- 172.21.15.78:0/128164321 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77800035d0 con 0x7f77940d1a40 2023-10-11T09:41:41.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.961+0000 7f778bfff700 1 -- 172.21.15.78:0/128164321 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1687922014 0 0) 0x7f77940cf6e0 con 0x7f779406dd80 2023-10-11T09:41:41.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.961+0000 7f778bfff700 1 -- 172.21.15.78:0/128164321 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77940cead0 con 0x7f779406dd80 2023-10-11T09:41:41.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.961+0000 7f77999fb700 1 -- 172.21.15.78:0/128164321 >> v1:172.21.15.133:6789/0 conn(0x7f77940d1a40 legacy=0x7f77940cb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:41.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.961+0000 7f77999fb700 1 -- 172.21.15.78:0/128164321 >> v1:172.21.15.78:6789/0 conn(0x7f779406dd80 legacy=0x7f779406e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:41.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.961+0000 7f77999fb700 1 -- 172.21.15.78:0/128164321 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77940cedb0 con 0x7f77940b2800 2023-10-11T09:41:41.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.961+0000 7f77999fb700 1 --2- 172.21.15.78:0/128164321 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77940b2800 0x7f77940cfb70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f778c009880 tx=0x7f778c009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=440ff17e553094d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:41.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.965+0000 7f778bfff700 1 -- 172.21.15.78:0/128164321 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778c011830 con 0x7f77940b2800 2023-10-11T09:41:41.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.965+0000 7f778bfff700 1 -- 172.21.15.78:0/128164321 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f778c011990 con 0x7f77940b2800 2023-10-11T09:41:41.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.965+0000 7f778bfff700 1 -- 172.21.15.78:0/128164321 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778c011d20 con 0x7f77940b2800 2023-10-11T09:41:41.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.965+0000 7f779a1fc700 1 -- 172.21.15.78:0/128164321 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77940b2800 msgr2=0x7f77940cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:41.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.965+0000 7f779a1fc700 1 --2- 172.21.15.78:0/128164321 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77940b2800 0x7f77940cfb70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f778c009880 tx=0x7f778c009b90 comp rx=0 tx=0).stop 2023-10-11T09:41:41.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.965+0000 7f779a1fc700 1 -- 172.21.15.78:0/128164321 shutdown_connections 2023-10-11T09:41:41.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.965+0000 7f779a1fc700 1 --2- 172.21.15.78:0/128164321 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77940b2800 0x7f77940cfb70 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:41.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.965+0000 7f779a1fc700 1 -- 172.21.15.78:0/128164321 >> 172.21.15.78:0/128164321 conn(0x7f779409a130 msgr2=0x7f77940d0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:41.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.965+0000 7f779a1fc700 1 -- 172.21.15.78:0/128164321 shutdown_connections 2023-10-11T09:41:41.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.965+0000 7f779a1fc700 1 -- 172.21.15.78:0/128164321 wait complete. 2023-10-11T09:41:41.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.965+0000 7f779a1fc700 1 Processor -- start 2023-10-11T09:41:41.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.965+0000 7f779a1fc700 1 -- start start 2023-10-11T09:41:41.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.969+0000 7f779a1fc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f779406dd80 0x7f779414bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:41.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.969+0000 7f779a1fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77940b2800 0x7f779414f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:41.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.969+0000 7f779a1fc700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7794148980 0x7f779414b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:41.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.969+0000 7f779a1fc700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779414f8b0 con 0x7f77940b2800 2023-10-11T09:41:41.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.969+0000 7f779a1fc700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77940b5f10 con 0x7f779406dd80 2023-10-11T09:41:41.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.969+0000 7f779a1fc700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77940c7250 con 0x7f7794148980 2023-10-11T09:41:41.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.969+0000 7f77989f9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77940b2800 0x7f779414f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:41.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.969+0000 7f77989f9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77940b2800 0x7f779414f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54508/0 (socket says 172.21.15.78:54508) 2023-10-11T09:41:41.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.969+0000 7f77989f9700 1 -- 172.21.15.78:0/1137051222 learned_addr learned my addr 172.21.15.78:0/1137051222 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:41.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.969+0000 7f77991fa700 1 --2- 172.21.15.78:0/1137051222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f779406dd80 0x7f779414bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:41.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.969+0000 7f77999fb700 1 --2- 172.21.15.78:0/1137051222 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7794148980 0x7f779414b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:41.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.969+0000 7f77989f9700 1 -- 172.21.15.78:0/1137051222 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7794148980 msgr2=0x7f779414b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:41.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.969+0000 7f77989f9700 1 --2- 172.21.15.78:0/1137051222 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7794148980 0x7f779414b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:41.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.969+0000 7f77989f9700 1 -- 172.21.15.78:0/1137051222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f779406dd80 msgr2=0x7f779414bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:41.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.969+0000 7f77989f9700 1 --2- 172.21.15.78:0/1137051222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f779406dd80 0x7f779414bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:41.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.969+0000 7f77989f9700 1 -- 172.21.15.78:0/1137051222 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f778c0094b0 con 0x7f77940b2800 2023-10-11T09:41:41.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.969+0000 7f77989f9700 1 --2- 172.21.15.78:0/1137051222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77940b2800 0x7f779414f370 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f778400c4e0 tx=0x7f778400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:41.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.969+0000 7f7789ffb700 1 -- 172.21.15.78:0/1137051222 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77840052d0 con 0x7f77940b2800 2023-10-11T09:41:41.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.969+0000 7f7789ffb700 1 -- 172.21.15.78:0/1137051222 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7784005430 con 0x7f77940b2800 2023-10-11T09:41:41.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.969+0000 7f7789ffb700 1 -- 172.21.15.78:0/1137051222 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77840055a0 con 0x7f77940b2800 2023-10-11T09:41:41.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.969+0000 7f779a1fc700 1 -- 172.21.15.78:0/1137051222 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77940c7530 con 0x7f77940b2800 2023-10-11T09:41:41.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.969+0000 7f779a1fc700 1 -- 172.21.15.78:0/1137051222 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77940b6490 con 0x7f77940b2800 2023-10-11T09:41:41.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.973+0000 7f779a1fc700 1 -- 172.21.15.78:0/1137051222 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77940b66b0 con 0x7f77940b2800 2023-10-11T09:41:41.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.973+0000 7f7789ffb700 1 -- 172.21.15.78:0/1137051222 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f77840038e0 con 0x7f77940b2800 2023-10-11T09:41:41.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.977+0000 7f7789ffb700 1 --2- 172.21.15.78:0/1137051222 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7778065ed0 0x7f7778068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:41.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.977+0000 7f7789ffb700 1 -- 172.21.15.78:0/1137051222 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f7784082c90 con 0x7f77940b2800 2023-10-11T09:41:41.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.977+0000 7f77991fa700 1 --2- 172.21.15.78:0/1137051222 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7778065ed0 0x7f7778068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:41.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.977+0000 7f77991fa700 1 --2- 172.21.15.78:0/1137051222 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7778065ed0 0x7f7778068380 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7790001460 tx=0x7f7790009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:41.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:41.981+0000 7f7789ffb700 1 -- 172.21.15.78:0/1137051222 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f77840502c0 con 0x7f77940b2800 2023-10-11T09:41:42.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.169+0000 7f779a1fc700 1 -- 172.21.15.78:0/1137051222 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7794065d40 con 0x7f77940b2800 2023-10-11T09:41:42.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.169+0000 7f7789ffb700 1 -- 172.21.15.78:0/1137051222 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7784017080 con 0x7f77940b2800 2023-10-11T09:41:42.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.173+0000 7f779a1fc700 1 -- 172.21.15.78:0/1137051222 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7778065ed0 msgr2=0x7f7778068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:42.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.173+0000 7f779a1fc700 1 --2- 172.21.15.78:0/1137051222 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7778065ed0 0x7f7778068380 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7790001460 tx=0x7f7790009040 comp rx=0 tx=0).stop 2023-10-11T09:41:42.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.173+0000 7f779a1fc700 1 -- 172.21.15.78:0/1137051222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77940b2800 msgr2=0x7f779414f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:42.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.173+0000 7f779a1fc700 1 --2- 172.21.15.78:0/1137051222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77940b2800 0x7f779414f370 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f778400c4e0 tx=0x7f778400c7f0 comp rx=0 tx=0).stop 2023-10-11T09:41:42.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.173+0000 7f779a1fc700 1 -- 172.21.15.78:0/1137051222 shutdown_connections 2023-10-11T09:41:42.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.173+0000 7f779a1fc700 1 --2- 172.21.15.78:0/1137051222 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7778065ed0 0x7f7778068380 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:42.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.173+0000 7f779a1fc700 1 --2- 172.21.15.78:0/1137051222 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7794148980 0x7f779414b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:42.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.173+0000 7f779a1fc700 1 --2- 172.21.15.78:0/1137051222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77940b2800 0x7f779414f370 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:42.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.173+0000 7f779a1fc700 1 --2- 172.21.15.78:0/1137051222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f779406dd80 0x7f779414bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:42.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.173+0000 7f779a1fc700 1 -- 172.21.15.78:0/1137051222 >> 172.21.15.78:0/1137051222 conn(0x7f779409a130 msgr2=0x7f779406b510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:42.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.173+0000 7f779a1fc700 1 -- 172.21.15.78:0/1137051222 shutdown_connections 2023-10-11T09:41:42.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.173+0000 7f779a1fc700 1 -- 172.21.15.78:0/1137051222 wait complete. 2023-10-11T09:41:42.181 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2023-10-11T09:41:42.200 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi133.front.sepia.ceph.com... 2023-10-11T09:41:42.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:42.201 DEBUG:teuthology.orchestra.run.smithi133:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-10-11T09:41:42.234 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:41:42.235 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2023-10-11T09:41:42.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:41:42.236 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-11T09:41:42.295 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-11T09:41:42.295 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:41:42.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.461+0000 7f1bbd666700 1 Processor -- start 2023-10-11T09:41:42.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.461+0000 7f1bbd666700 1 -- start start 2023-10-11T09:41:42.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.465+0000 7f1bbd666700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1bb80cda10 0x7f1bb80caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:42.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.465+0000 7f1bbd666700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f1bb80cdde0 0x7f1bb8164610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:42.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.465+0000 7f1bbd666700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bb8165390 con 0x7f1bb8164b50 2023-10-11T09:41:42.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.465+0000 7f1bbd666700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bb80ca440 con 0x7f1bb80cdde0 2023-10-11T09:41:42.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.465+0000 7f1bbd666700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb80ca720 con 0x7f1bb80cda10 2023-10-11T09:41:42.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.465+0000 7f1baf7fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f1bb80cdde0 0x7f1bb8164610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:42.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.465+0000 7f1baf7fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f1bb80cdde0 0x7f1bb8164610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54510/0 (socket says 172.21.15.78:54510) 2023-10-11T09:41:42.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.465+0000 7f1baf7fe700 1 -- 172.21.15.78:0/2305552424 learned_addr learned my addr 172.21.15.78:0/2305552424 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:42.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.465+0000 7f1bb7fff700 1 --2- 172.21.15.78:0/2305552424 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1bb80cda10 0x7f1bb80caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:42.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.465+0000 7f1baf7fe700 1 -- 172.21.15.78:0/2305552424 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1bb80cda10 msgr2=0x7f1bb80caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:42.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.465+0000 7f1baf7fe700 1 --2- 172.21.15.78:0/2305552424 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1bb80cda10 0x7f1bb80caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:42.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.465+0000 7f1baf7fe700 1 -- 172.21.15.78:0/2305552424 >> v1:172.21.15.133:6789/0 conn(0x7f1bb8164b50 legacy=0x7f1bb806bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:42.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.465+0000 7f1baf7fe700 1 -- 172.21.15.78:0/2305552424 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb80caa00 con 0x7f1bb80cdde0 2023-10-11T09:41:42.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.465+0000 7f1baf7fe700 1 --2- 172.21.15.78:0/2305552424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bb80cdde0 0x7f1bb8164610 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0009910 tx=0x7f1ba0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8906841bf1bfbe server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:42.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.465+0000 7f1bb77fe700 1 -- 172.21.15.78:0/2305552424 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba000ba60 con 0x7f1bb80cdde0 2023-10-11T09:41:42.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.465+0000 7f1bb77fe700 1 -- 172.21.15.78:0/2305552424 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba000bbc0 con 0x7f1bb80cdde0 2023-10-11T09:41:42.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.465+0000 7f1bb77fe700 1 -- 172.21.15.78:0/2305552424 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0009e60 con 0x7f1bb80cdde0 2023-10-11T09:41:42.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.469+0000 7f1bbd666700 1 -- 172.21.15.78:0/2305552424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bb80cdde0 msgr2=0x7f1bb8164610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:42.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.469+0000 7f1bbd666700 1 --2- 172.21.15.78:0/2305552424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bb80cdde0 0x7f1bb8164610 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0009910 tx=0x7f1ba0009c20 comp rx=0 tx=0).stop 2023-10-11T09:41:42.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.469+0000 7f1bbd666700 1 -- 172.21.15.78:0/2305552424 shutdown_connections 2023-10-11T09:41:42.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.469+0000 7f1bbd666700 1 --2- 172.21.15.78:0/2305552424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bb80cdde0 0x7f1bb8164610 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:42.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.469+0000 7f1bbd666700 1 --2- 172.21.15.78:0/2305552424 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1bb80cda10 0x7f1bb80caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:42.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.469+0000 7f1bbd666700 1 -- 172.21.15.78:0/2305552424 >> 172.21.15.78:0/2305552424 conn(0x7f1bb80cea60 msgr2=0x7f1bb80cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:42.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.469+0000 7f1bbd666700 1 -- 172.21.15.78:0/2305552424 shutdown_connections 2023-10-11T09:41:42.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.469+0000 7f1bbd666700 1 -- 172.21.15.78:0/2305552424 wait complete. 2023-10-11T09:41:42.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.469+0000 7f1bbd666700 1 Processor -- start 2023-10-11T09:41:42.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.469+0000 7f1bbd666700 1 -- start start 2023-10-11T09:41:42.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.473+0000 7f1bbd666700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bb80cda10 0x7f1bb815cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:42.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.473+0000 7f1bbd666700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bb80cdde0 0x7f1bb8162bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:42.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.473+0000 7f1bbd666700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1bb8164b50 0x7f1bb81eadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:42.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.473+0000 7f1bbd666700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb8163dc0 con 0x7f1bb80cda10 2023-10-11T09:41:42.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.473+0000 7f1bbd666700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb8163520 con 0x7f1bb80cdde0 2023-10-11T09:41:42.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.473+0000 7f1bbd666700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb8163800 con 0x7f1bb8164b50 2023-10-11T09:41:42.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.473+0000 7f1bb7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bb80cda10 0x7f1bb815cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:42.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.473+0000 7f1bb7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bb80cda10 0x7f1bb815cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54514/0 (socket says 172.21.15.78:54514) 2023-10-11T09:41:42.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.473+0000 7f1bb7fff700 1 -- 172.21.15.78:0/1948291431 learned_addr learned my addr 172.21.15.78:0/1948291431 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:42.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.473+0000 7f1baf7fe700 1 --2- 172.21.15.78:0/1948291431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bb80cdde0 0x7f1bb8162bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:42.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.473+0000 7f1bbce65700 1 --2- 172.21.15.78:0/1948291431 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1bb8164b50 0x7f1bb81eadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:42.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.473+0000 7f1bb7fff700 1 -- 172.21.15.78:0/1948291431 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1bb8164b50 msgr2=0x7f1bb81eadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:42.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.473+0000 7f1bb7fff700 1 --2- 172.21.15.78:0/1948291431 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1bb8164b50 0x7f1bb81eadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:42.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.473+0000 7f1bb7fff700 1 -- 172.21.15.78:0/1948291431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bb80cdde0 msgr2=0x7f1bb8162bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:42.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.473+0000 7f1bb7fff700 1 --2- 172.21.15.78:0/1948291431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bb80cdde0 0x7f1bb8162bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:42.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.473+0000 7f1bb7fff700 1 -- 172.21.15.78:0/1948291431 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba0009570 con 0x7f1bb80cda10 2023-10-11T09:41:42.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.473+0000 7f1bbce65700 1 --2- 172.21.15.78:0/1948291431 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1bb8164b50 0x7f1bb81eadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:42.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.473+0000 7f1bb7fff700 1 --2- 172.21.15.78:0/1948291431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bb80cda10 0x7f1bb815cef0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1ba800d800 tx=0x7f1ba800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:42.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.473+0000 7f1bb57fa700 1 -- 172.21.15.78:0/1948291431 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba80125f0 con 0x7f1bb80cda10 2023-10-11T09:41:42.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.473+0000 7f1bb57fa700 1 -- 172.21.15.78:0/1948291431 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba8012750 con 0x7f1bb80cda10 2023-10-11T09:41:42.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.473+0000 7f1bbd666700 1 -- 172.21.15.78:0/1948291431 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bb80ce3b0 con 0x7f1bb80cda10 2023-10-11T09:41:42.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.473+0000 7f1bb57fa700 1 -- 172.21.15.78:0/1948291431 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba8012ac0 con 0x7f1bb80cda10 2023-10-11T09:41:42.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.473+0000 7f1bbd666700 1 -- 172.21.15.78:0/1948291431 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb80b1ab0 con 0x7f1bb80cda10 2023-10-11T09:41:42.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.473+0000 7f1bbd666700 1 -- 172.21.15.78:0/1948291431 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bb8065d40 con 0x7f1bb80cda10 2023-10-11T09:41:42.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.477+0000 7f1bb57fa700 1 -- 172.21.15.78:0/1948291431 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f1ba8012c20 con 0x7f1bb80cda10 2023-10-11T09:41:42.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.477+0000 7f1bb57fa700 1 --2- 172.21.15.78:0/1948291431 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1b9c065ed0 0x7f1b9c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:42.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.477+0000 7f1bb57fa700 1 -- 172.21.15.78:0/1948291431 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f1ba80839c0 con 0x7f1bb80cda10 2023-10-11T09:41:42.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.477+0000 7f1baf7fe700 1 --2- 172.21.15.78:0/1948291431 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1b9c065ed0 0x7f1b9c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:42.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.481+0000 7f1baf7fe700 1 --2- 172.21.15.78:0/1948291431 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1b9c065ed0 0x7f1b9c068380 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0000c00 tx=0x7f1ba0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:42.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.485+0000 7f1bb57fa700 1 -- 172.21.15.78:0/1948291431 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f1ba8050f30 con 0x7f1bb80cda10 2023-10-11T09:41:42.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.693+0000 7f1bbd666700 1 -- 172.21.15.78:0/1948291431 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1bb80b1480 con 0x7f1bb80cda10 2023-10-11T09:41:42.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.693+0000 7f1bb57fa700 1 -- 172.21.15.78:0/1948291431 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4648 (secure 0 0 0) 0x7f1ba8054550 con 0x7f1bb80cda10 2023-10-11T09:41:42.699 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:41:42.700 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/4282378510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4282378510},{"type":"v1","addr":"172.21.15.133:6835","nonce":4282378510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3986631249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3986631249},{"type":"v1","addr":"172.21.15.78:6835","nonce":3986631249}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3135421902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3135421902},{"type":"v1","addr":"172.21.15.78:6837","nonce":3135421902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:40:41.673556+0000","modified":"2023-10-11T09:40:45.122852+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6834/3925157967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":3925157967},{"type":"v1","addr":"172.21.15.133:6834","nonce":3925157967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-11T09:41:42.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.697+0000 7f1bbd666700 1 -- 172.21.15.78:0/1948291431 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1b9c065ed0 msgr2=0x7f1b9c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:42.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.697+0000 7f1bbd666700 1 --2- 172.21.15.78:0/1948291431 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1b9c065ed0 0x7f1b9c068380 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0000c00 tx=0x7f1ba0008030 comp rx=0 tx=0).stop 2023-10-11T09:41:42.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.697+0000 7f1bbd666700 1 -- 172.21.15.78:0/1948291431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bb80cda10 msgr2=0x7f1bb815cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:42.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.697+0000 7f1bbd666700 1 --2- 172.21.15.78:0/1948291431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bb80cda10 0x7f1bb815cef0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1ba800d800 tx=0x7f1ba800db10 comp rx=0 tx=0).stop 2023-10-11T09:41:42.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.697+0000 7f1bbd666700 1 -- 172.21.15.78:0/1948291431 shutdown_connections 2023-10-11T09:41:42.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.697+0000 7f1bbd666700 1 --2- 172.21.15.78:0/1948291431 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1b9c065ed0 0x7f1b9c068380 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:42.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.697+0000 7f1bbd666700 1 --2- 172.21.15.78:0/1948291431 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1bb8164b50 0x7f1bb81eadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:42.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.697+0000 7f1bbd666700 1 --2- 172.21.15.78:0/1948291431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bb80cdde0 0x7f1bb8162bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:42.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.697+0000 7f1bbd666700 1 --2- 172.21.15.78:0/1948291431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bb80cda10 0x7f1bb815cef0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:42.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.697+0000 7f1bbd666700 1 -- 172.21.15.78:0/1948291431 >> 172.21.15.78:0/1948291431 conn(0x7f1bb80cea60 msgr2=0x7f1bb8149990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:42.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.697+0000 7f1bbd666700 1 -- 172.21.15.78:0/1948291431 shutdown_connections 2023-10-11T09:41:42.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.697+0000 7f1bbd666700 1 -- 172.21.15.78:0/1948291431 wait complete. 2023-10-11T09:41:42.705 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 8 2023-10-11T09:41:42.726 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:41:42.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.889+0000 7fae2759e700 1 Processor -- start 2023-10-11T09:41:42.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.889+0000 7fae2759e700 1 -- start start 2023-10-11T09:41:42.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.893+0000 7fae2759e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae280ca950 0x7fae281650f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:42.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.893+0000 7fae2759e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae2815e010 con 0x7fae2815a940 2023-10-11T09:41:42.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.893+0000 7fae2759e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae2815e280 con 0x7fae2815b8e0 2023-10-11T09:41:42.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.893+0000 7fae2759e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae2815e560 con 0x7fae280ca950 2023-10-11T09:41:42.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.893+0000 7fae25d9b700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fae2815a940 0x7fae280ca240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33242/0 (socket says 172.21.15.78:33242) 2023-10-11T09:41:42.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.893+0000 7fae25d9b700 1 -- 172.21.15.78:0/381824511 learned_addr learned my addr 172.21.15.78:0/381824511 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:42.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.893+0000 7fae26d9d700 1 --2- 172.21.15.78:0/381824511 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae280ca950 0x7fae281650f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:42.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.893+0000 7fae2559a700 1 -- 172.21.15.78:0/381824511 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1411612730 0 0) 0x7fae2815e010 con 0x7fae2815a940 2023-10-11T09:41:42.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.893+0000 7fae2559a700 1 -- 172.21.15.78:0/381824511 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae100035d0 con 0x7fae2815a940 2023-10-11T09:41:42.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.893+0000 7fae2559a700 1 -- 172.21.15.78:0/381824511 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2370690006 0 0) 0x7fae100035d0 con 0x7fae2815a940 2023-10-11T09:41:42.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.893+0000 7fae2559a700 1 -- 172.21.15.78:0/381824511 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae280ca950 msgr2=0x7fae281650f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:42.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.893+0000 7fae2559a700 1 --2- 172.21.15.78:0/381824511 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae280ca950 0x7fae281650f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:42.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.893+0000 7fae2559a700 1 -- 172.21.15.78:0/381824511 >> v1:172.21.15.133:6789/0 conn(0x7fae2815b8e0 legacy=0x7fae2806d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:42.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.893+0000 7fae2559a700 1 -- 172.21.15.78:0/381824511 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae2815e840 con 0x7fae2815a940 2023-10-11T09:41:42.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.893+0000 7fae2559a700 1 -- 172.21.15.78:0/381824511 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7fae1c001670 con 0x7fae2815a940 2023-10-11T09:41:42.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.893+0000 7fae2559a700 1 -- 172.21.15.78:0/381824511 >> v1:172.21.15.78:6789/0 conn(0x7fae2815a940 legacy=0x7fae280ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:42.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.893+0000 7fae2559a700 1 --2- 172.21.15.78:0/381824511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae10003a50 0x7fae10003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:42.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.897+0000 7fae2559a700 1 --2- 172.21.15.78:0/381824511 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae280ca950 0x7fae10004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:42.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.897+0000 7fae2559a700 1 --2- 172.21.15.78:0/381824511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae10004db0 0x7fae10005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:42.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.897+0000 7fae2559a700 1 -- 172.21.15.78:0/381824511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2815e560 con 0x7fae10003a50 2023-10-11T09:41:42.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.897+0000 7fae2559a700 1 -- 172.21.15.78:0/381824511 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1000fef0 con 0x7fae10004db0 2023-10-11T09:41:42.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.897+0000 7fae2559a700 1 -- 172.21.15.78:0/381824511 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae10010030 con 0x7fae280ca950 2023-10-11T09:41:42.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.897+0000 7fae2659c700 1 --2- 172.21.15.78:0/381824511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae10003a50 0x7fae10003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:42.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.897+0000 7fae26d9d700 1 --2- 172.21.15.78:0/381824511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae10004db0 0x7fae10005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:42.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.897+0000 7fae25d9b700 1 --2- 172.21.15.78:0/381824511 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae280ca950 0x7fae10004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:42.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.897+0000 7fae2659c700 1 -- 172.21.15.78:0/381824511 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae280ca950 msgr2=0x7fae10004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:42.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.897+0000 7fae2659c700 1 --2- 172.21.15.78:0/381824511 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae280ca950 0x7fae10004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:42.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.897+0000 7fae2659c700 1 -- 172.21.15.78:0/381824511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae10004db0 msgr2=0x7fae10005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:42.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.897+0000 7fae2659c700 1 --2- 172.21.15.78:0/381824511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae10004db0 0x7fae10005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:42.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.897+0000 7fae2659c700 1 -- 172.21.15.78:0/381824511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae100035d0 con 0x7fae10003a50 2023-10-11T09:41:42.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.897+0000 7fae26d9d700 1 --2- 172.21.15.78:0/381824511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae10004db0 0x7fae10005220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:41:42.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.897+0000 7fae2659c700 1 --2- 172.21.15.78:0/381824511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae10003a50 0x7fae10003e20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fae180018f0 tx=0x7fae100058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae5e07d905350fc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:42.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.897+0000 7fae2559a700 1 -- 172.21.15.78:0/381824511 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae18001390 con 0x7fae10003a50 2023-10-11T09:41:42.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.897+0000 7fae2559a700 1 -- 172.21.15.78:0/381824511 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae1800c690 con 0x7fae10003a50 2023-10-11T09:41:42.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.897+0000 7fae2559a700 1 -- 172.21.15.78:0/381824511 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1800ca40 con 0x7fae10003a50 2023-10-11T09:41:42.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.897+0000 7fae2759e700 1 -- 172.21.15.78:0/381824511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae10003a50 msgr2=0x7fae10003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:42.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.897+0000 7fae2759e700 1 --2- 172.21.15.78:0/381824511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae10003a50 0x7fae10003e20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fae180018f0 tx=0x7fae100058a0 comp rx=0 tx=0).stop 2023-10-11T09:41:42.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.897+0000 7fae26d9d700 1 -- 172.21.15.78:0/381824511 reap_dead start 2023-10-11T09:41:42.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.901+0000 7fae2759e700 1 -- 172.21.15.78:0/381824511 shutdown_connections 2023-10-11T09:41:42.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.901+0000 7fae2759e700 1 -- 172.21.15.78:0/381824511 >> 172.21.15.78:0/381824511 conn(0x7fae280cea60 msgr2=0x7fae280cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:42.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.901+0000 7fae2759e700 1 -- 172.21.15.78:0/381824511 shutdown_connections 2023-10-11T09:41:42.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.901+0000 7fae2759e700 1 -- 172.21.15.78:0/381824511 wait complete. 2023-10-11T09:41:42.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.901+0000 7fae2759e700 1 Processor -- start 2023-10-11T09:41:42.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.901+0000 7fae2759e700 1 -- start start 2023-10-11T09:41:42.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.905+0000 7fae2759e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae10003a50 0x7fae280c5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:42.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.905+0000 7fae2759e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae28159990 0x7fae280cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:42.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.905+0000 7fae2759e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae280cb5f0 0x7fae280c3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:42.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.905+0000 7fae2759e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae280cba60 con 0x7fae280cb5f0 2023-10-11T09:41:42.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.905+0000 7fae2759e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae280c45d0 con 0x7fae10003a50 2023-10-11T09:41:42.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.905+0000 7fae2759e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae280c48b0 con 0x7fae28159990 2023-10-11T09:41:42.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.905+0000 7fae26d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae280cb5f0 0x7fae280c3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:42.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.905+0000 7fae26d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae280cb5f0 0x7fae280c3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54534/0 (socket says 172.21.15.78:54534) 2023-10-11T09:41:42.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.905+0000 7fae26d9d700 1 -- 172.21.15.78:0/3936867582 learned_addr learned my addr 172.21.15.78:0/3936867582 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:42.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.905+0000 7fae2659c700 1 --2- 172.21.15.78:0/3936867582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae10003a50 0x7fae280c5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:42.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.905+0000 7fae25d9b700 1 --2- 172.21.15.78:0/3936867582 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae28159990 0x7fae280cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:42.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.905+0000 7fae26d9d700 1 -- 172.21.15.78:0/3936867582 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae28159990 msgr2=0x7fae280cb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:42.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.905+0000 7fae26d9d700 1 --2- 172.21.15.78:0/3936867582 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae28159990 0x7fae280cb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:42.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.905+0000 7fae26d9d700 1 -- 172.21.15.78:0/3936867582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae10003a50 msgr2=0x7fae280c5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:42.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.905+0000 7fae26d9d700 1 --2- 172.21.15.78:0/3936867582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae10003a50 0x7fae280c5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:42.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.905+0000 7fae26d9d700 1 -- 172.21.15.78:0/3936867582 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae280c4b90 con 0x7fae280cb5f0 2023-10-11T09:41:42.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.905+0000 7fae26d9d700 1 --2- 172.21.15.78:0/3936867582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae280cb5f0 0x7fae280c3980 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fae280cf890 tx=0x7fae2000f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:42.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.905+0000 7fae16ffd700 1 -- 172.21.15.78:0/3936867582 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2000e040 con 0x7fae280cb5f0 2023-10-11T09:41:42.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.905+0000 7fae16ffd700 1 -- 172.21.15.78:0/3936867582 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2000f040 con 0x7fae280cb5f0 2023-10-11T09:41:42.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.905+0000 7fae16ffd700 1 -- 172.21.15.78:0/3936867582 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2000fbd0 con 0x7fae280cb5f0 2023-10-11T09:41:42.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.905+0000 7fae2759e700 1 -- 172.21.15.78:0/3936867582 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae280c4e70 con 0x7fae280cb5f0 2023-10-11T09:41:42.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.905+0000 7fae2759e700 1 -- 172.21.15.78:0/3936867582 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae28159e00 con 0x7fae280cb5f0 2023-10-11T09:41:42.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.909+0000 7fae2759e700 1 -- 172.21.15.78:0/3936867582 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae280b3c70 con 0x7fae280cb5f0 2023-10-11T09:41:42.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.909+0000 7fae16ffd700 1 -- 172.21.15.78:0/3936867582 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fae2000fd30 con 0x7fae280cb5f0 2023-10-11T09:41:42.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.909+0000 7fae16ffd700 1 --2- 172.21.15.78:0/3936867582 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fae08065f00 0x7fae080683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:42.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.909+0000 7fae16ffd700 1 -- 172.21.15.78:0/3936867582 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fae20083d20 con 0x7fae280cb5f0 2023-10-11T09:41:42.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.909+0000 7fae2659c700 1 --2- 172.21.15.78:0/3936867582 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fae08065f00 0x7fae080683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:42.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.913+0000 7fae2659c700 1 --2- 172.21.15.78:0/3936867582 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fae08065f00 0x7fae080683b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fae18014040 tx=0x7fae18005040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:42.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:42.917+0000 7fae16ffd700 1 -- 172.21.15.78:0/3936867582 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fae20051290 con 0x7fae280cb5f0 2023-10-11T09:41:43.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.089+0000 7fae2759e700 1 -- 172.21.15.78:0/3936867582 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae28065d40 con 0x7fae280cb5f0 2023-10-11T09:41:43.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.093+0000 7fae16ffd700 1 -- 172.21.15.78:0/3936867582 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17907 (secure 0 0 0) 0x7fae20013310 con 0x7fae280cb5f0 2023-10-11T09:41:43.099 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:41:43.099 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":25,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:41:21.473566+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:40:39.862581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T09:40:40.663641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.37","mappings":[{"from":3,"to":2}]},{"pgid":"4.3e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:41:43.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.097+0000 7fae2759e700 1 -- 172.21.15.78:0/3936867582 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fae08065f00 msgr2=0x7fae080683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:43.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.097+0000 7fae2759e700 1 --2- 172.21.15.78:0/3936867582 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fae08065f00 0x7fae080683b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fae18014040 tx=0x7fae18005040 comp rx=0 tx=0).stop 2023-10-11T09:41:43.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.097+0000 7fae2759e700 1 -- 172.21.15.78:0/3936867582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae280cb5f0 msgr2=0x7fae280c3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:43.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.097+0000 7fae2759e700 1 --2- 172.21.15.78:0/3936867582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae280cb5f0 0x7fae280c3980 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fae280cf890 tx=0x7fae2000f490 comp rx=0 tx=0).stop 2023-10-11T09:41:43.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.097+0000 7fae2759e700 1 -- 172.21.15.78:0/3936867582 shutdown_connections 2023-10-11T09:41:43.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.097+0000 7fae2759e700 1 --2- 172.21.15.78:0/3936867582 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fae08065f00 0x7fae080683b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:43.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.097+0000 7fae2759e700 1 --2- 172.21.15.78:0/3936867582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae280cb5f0 0x7fae280c3980 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:43.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.097+0000 7fae2759e700 1 --2- 172.21.15.78:0/3936867582 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae28159990 0x7fae280cb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:43.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.097+0000 7fae2759e700 1 --2- 172.21.15.78:0/3936867582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae10003a50 0x7fae280c5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:43.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.097+0000 7fae2759e700 1 -- 172.21.15.78:0/3936867582 >> 172.21.15.78:0/3936867582 conn(0x7fae280cea60 msgr2=0x7fae280cdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:43.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.097+0000 7fae2759e700 1 -- 172.21.15.78:0/3936867582 shutdown_connections 2023-10-11T09:41:43.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.097+0000 7fae2759e700 1 -- 172.21.15.78:0/3936867582 wait complete. 2023-10-11T09:41:43.126 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T09:40:39.862581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T09:40:40.663641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:41:43.127 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:41:43.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.297+0000 7f37118d5700 1 Processor -- start 2023-10-11T09:41:43.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.301+0000 7f37118d5700 1 -- start start 2023-10-11T09:41:43.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.301+0000 7f37118d5700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f370c06d730 0x7f370c165240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:43.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.301+0000 7f37118d5700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f370c0b2d30 con 0x7f370c15a6b0 2023-10-11T09:41:43.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.301+0000 7f37118d5700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f370c0cf360 con 0x7f370c0b3a60 2023-10-11T09:41:43.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.301+0000 7f37118d5700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f370c0b2f80 con 0x7f370c06d730 2023-10-11T09:41:43.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.301+0000 7f37108d3700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f370c15a6b0 0x7f370c0ce8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33248/0 (socket says 172.21.15.78:33248) 2023-10-11T09:41:43.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.301+0000 7f37108d3700 1 -- 172.21.15.78:0/3606829543 learned_addr learned my addr 172.21.15.78:0/3606829543 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:43.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.301+0000 7f37110d4700 1 --2- 172.21.15.78:0/3606829543 >> v2:172.21.15.133:3300/0 conn(0x7f370c06d730 0x7f370c165240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:43.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.301+0000 7f370b7fe700 1 -- 172.21.15.78:0/3606829543 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4144847631 0 0) 0x7f370c0b2d30 con 0x7f370c15a6b0 2023-10-11T09:41:43.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.301+0000 7f370b7fe700 1 -- 172.21.15.78:0/3606829543 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36fc0035d0 con 0x7f370c15a6b0 2023-10-11T09:41:43.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.301+0000 7f370b7fe700 1 -- 172.21.15.78:0/3606829543 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 922893335 0 0) 0x7f370c0cf360 con 0x7f370c0b3a60 2023-10-11T09:41:43.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.301+0000 7f370b7fe700 1 -- 172.21.15.78:0/3606829543 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f370c0b2d30 con 0x7f370c0b3a60 2023-10-11T09:41:43.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.301+0000 7f370b7fe700 1 -- 172.21.15.78:0/3606829543 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1001832019 0 0) 0x7f36fc0035d0 con 0x7f370c15a6b0 2023-10-11T09:41:43.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.301+0000 7f370b7fe700 1 -- 172.21.15.78:0/3606829543 >> v2:172.21.15.133:3300/0 conn(0x7f370c06d730 msgr2=0x7f370c165240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:43.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.301+0000 7f370b7fe700 1 --2- 172.21.15.78:0/3606829543 >> v2:172.21.15.133:3300/0 conn(0x7f370c06d730 0x7f370c165240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:43.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.301+0000 7f370b7fe700 1 -- 172.21.15.78:0/3606829543 >> v1:172.21.15.133:6789/0 conn(0x7f370c0b3a60 legacy=0x7f370c06d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:43.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.301+0000 7f370b7fe700 1 -- 172.21.15.78:0/3606829543 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f370c0b3260 con 0x7f370c15a6b0 2023-10-11T09:41:43.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.301+0000 7f370b7fe700 1 -- 172.21.15.78:0/3606829543 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f3700001670 con 0x7f370c15a6b0 2023-10-11T09:41:43.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.301+0000 7f370b7fe700 1 -- 172.21.15.78:0/3606829543 >> v1:172.21.15.78:6789/0 conn(0x7f370c15a6b0 legacy=0x7f370c0ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:43.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.301+0000 7f370b7fe700 1 --2- 172.21.15.78:0/3606829543 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36fc003bb0 0x7f36fc003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:43.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.301+0000 7f370b7fe700 1 --2- 172.21.15.78:0/3606829543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36fc004870 0x7f36fc004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:43.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.301+0000 7f370b7fe700 1 --2- 172.21.15.78:0/3606829543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36fc005220 0x7f36fc00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:43.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.301+0000 7f370b7fe700 1 -- 172.21.15.78:0/3606829543 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370c0b2f80 con 0x7f36fc004870 2023-10-11T09:41:43.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.301+0000 7f370b7fe700 1 -- 172.21.15.78:0/3606829543 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc00c130 con 0x7f36fc005220 2023-10-11T09:41:43.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.301+0000 7f370b7fe700 1 -- 172.21.15.78:0/3606829543 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc00c2f0 con 0x7f36fc003bb0 2023-10-11T09:41:43.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.305+0000 7f370bfff700 1 --2- 172.21.15.78:0/3606829543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36fc004870 0x7f36fc004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:43.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.305+0000 7f37110d4700 1 --2- 172.21.15.78:0/3606829543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36fc005220 0x7f36fc00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:43.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.305+0000 7f37108d3700 1 --2- 172.21.15.78:0/3606829543 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36fc003bb0 0x7f36fc003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:43.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.305+0000 7f370bfff700 1 -- 172.21.15.78:0/3606829543 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36fc003bb0 msgr2=0x7f36fc003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:43.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.305+0000 7f370bfff700 1 --2- 172.21.15.78:0/3606829543 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36fc003bb0 0x7f36fc003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:43.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.305+0000 7f370bfff700 1 -- 172.21.15.78:0/3606829543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36fc005220 msgr2=0x7f36fc00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:43.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.305+0000 7f370bfff700 1 --2- 172.21.15.78:0/3606829543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36fc005220 0x7f36fc00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:43.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.305+0000 7f370bfff700 1 -- 172.21.15.78:0/3606829543 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36fc0035d0 con 0x7f36fc004870 2023-10-11T09:41:43.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.305+0000 7f37110d4700 1 -- 172.21.15.78:0/3606829543 reap_dead start 2023-10-11T09:41:43.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.305+0000 7f370bfff700 1 --2- 172.21.15.78:0/3606829543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36fc004870 0x7f36fc004ce0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f36f8007fd0 tx=0x7f36fc0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee72ad6f9be9f774 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:43.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.305+0000 7f370b7fe700 1 -- 172.21.15.78:0/3606829543 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f8001450 con 0x7f36fc004870 2023-10-11T09:41:43.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.305+0000 7f370b7fe700 1 -- 172.21.15.78:0/3606829543 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f8006aa0 con 0x7f36fc004870 2023-10-11T09:41:43.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.305+0000 7f370b7fe700 1 -- 172.21.15.78:0/3606829543 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f8006e50 con 0x7f36fc004870 2023-10-11T09:41:43.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.305+0000 7f37118d5700 1 -- 172.21.15.78:0/3606829543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36fc004870 msgr2=0x7f36fc004ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:43.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.305+0000 7f37118d5700 1 --2- 172.21.15.78:0/3606829543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36fc004870 0x7f36fc004ce0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f36f8007fd0 tx=0x7f36fc0058a0 comp rx=0 tx=0).stop 2023-10-11T09:41:43.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.305+0000 7f37118d5700 1 -- 172.21.15.78:0/3606829543 shutdown_connections 2023-10-11T09:41:43.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.305+0000 7f37118d5700 1 --2- 172.21.15.78:0/3606829543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36fc004870 0x7f36fc004ce0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:43.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.305+0000 7f37118d5700 1 -- 172.21.15.78:0/3606829543 >> 172.21.15.78:0/3606829543 conn(0x7f370c06a060 msgr2=0x7f370c06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:43.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.305+0000 7f37118d5700 1 -- 172.21.15.78:0/3606829543 shutdown_connections 2023-10-11T09:41:43.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.305+0000 7f37118d5700 1 -- 172.21.15.78:0/3606829543 wait complete. 2023-10-11T09:41:43.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.309+0000 7f37118d5700 1 Processor -- start 2023-10-11T09:41:43.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.309+0000 7f37118d5700 1 -- start start 2023-10-11T09:41:43.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.309+0000 7f37118d5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36fc004870 0x7f370c1458d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:43.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.309+0000 7f37118d5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f370c149200 0x7f370c0b0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:43.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.309+0000 7f37118d5700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f370c149670 0x7f370c0b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:43.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.309+0000 7f37118d5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370c14b410 con 0x7f36fc004870 2023-10-11T09:41:43.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.309+0000 7f37118d5700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370c1f3dc0 con 0x7f370c149200 2023-10-11T09:41:43.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.309+0000 7f37118d5700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f370c149dc0 con 0x7f370c149670 2023-10-11T09:41:43.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.309+0000 7f37108d3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36fc004870 0x7f370c1458d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:43.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.313+0000 7f37108d3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36fc004870 0x7f370c1458d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54562/0 (socket says 172.21.15.78:54562) 2023-10-11T09:41:43.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.313+0000 7f37108d3700 1 -- 172.21.15.78:0/874986885 learned_addr learned my addr 172.21.15.78:0/874986885 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:43.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.313+0000 7f370bfff700 1 --2- 172.21.15.78:0/874986885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f370c149200 0x7f370c0b0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:43.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.313+0000 7f37110d4700 1 --2- 172.21.15.78:0/874986885 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f370c149670 0x7f370c0b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:43.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.313+0000 7f37108d3700 1 -- 172.21.15.78:0/874986885 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f370c149670 msgr2=0x7f370c0b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:43.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.313+0000 7f37108d3700 1 --2- 172.21.15.78:0/874986885 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f370c149670 0x7f370c0b0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:43.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.313+0000 7f37108d3700 1 -- 172.21.15.78:0/874986885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f370c149200 msgr2=0x7f370c0b0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:43.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.313+0000 7f37108d3700 1 --2- 172.21.15.78:0/874986885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f370c149200 0x7f370c0b0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:43.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.313+0000 7f37108d3700 1 -- 172.21.15.78:0/874986885 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f370c1f3240 con 0x7f36fc004870 2023-10-11T09:41:43.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.313+0000 7f370bfff700 1 --2- 172.21.15.78:0/874986885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f370c149200 0x7f370c0b0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:41:43.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.313+0000 7f37110d4700 1 --2- 172.21.15.78:0/874986885 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f370c149670 0x7f370c0b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:41:43.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.313+0000 7f37108d3700 1 --2- 172.21.15.78:0/874986885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36fc004870 0x7f370c1458d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3700001200 tx=0x7f370000eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:43.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.313+0000 7f37097fa700 1 -- 172.21.15.78:0/874986885 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3700012590 con 0x7f36fc004870 2023-10-11T09:41:43.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.313+0000 7f37097fa700 1 -- 172.21.15.78:0/874986885 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37000126f0 con 0x7f36fc004870 2023-10-11T09:41:43.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.313+0000 7f37118d5700 1 -- 172.21.15.78:0/874986885 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f370c1f3520 con 0x7f36fc004870 2023-10-11T09:41:43.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.313+0000 7f37097fa700 1 -- 172.21.15.78:0/874986885 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3700012a60 con 0x7f36fc004870 2023-10-11T09:41:43.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.313+0000 7f37118d5700 1 -- 172.21.15.78:0/874986885 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f370c1f3ae0 con 0x7f36fc004870 2023-10-11T09:41:43.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.313+0000 7f37118d5700 1 -- 172.21.15.78:0/874986885 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f370c065d40 con 0x7f36fc004870 2023-10-11T09:41:43.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.317+0000 7f37097fa700 1 -- 172.21.15.78:0/874986885 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f3700012bc0 con 0x7f36fc004870 2023-10-11T09:41:43.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.317+0000 7f37097fa700 1 --2- 172.21.15.78:0/874986885 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f36f0065ed0 0x7f36f0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:43.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.317+0000 7f37097fa700 1 -- 172.21.15.78:0/874986885 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f3700083670 con 0x7f36fc004870 2023-10-11T09:41:43.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.317+0000 7f370bfff700 1 --2- 172.21.15.78:0/874986885 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f36f0065ed0 0x7f36f0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:43.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.317+0000 7f370bfff700 1 --2- 172.21.15.78:0/874986885 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f36f0065ed0 0x7f36f0068380 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f370c14f100 tx=0x7f36f8005040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:43.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.325+0000 7f37097fa700 1 -- 172.21.15.78:0/874986885 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f3700050be0 con 0x7f36fc004870 2023-10-11T09:41:43.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.497+0000 7f37118d5700 1 -- 172.21.15.78:0/874986885 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f370c0cb6a0 con 0x7f36fc004870 2023-10-11T09:41:43.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.501+0000 7f37097fa700 1 -- 172.21.15.78:0/874986885 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3700054200 con 0x7f36fc004870 2023-10-11T09:41:43.506 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:41:43.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.501+0000 7f37118d5700 1 -- 172.21.15.78:0/874986885 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f36f0065ed0 msgr2=0x7f36f0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:43.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.501+0000 7f37118d5700 1 --2- 172.21.15.78:0/874986885 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f36f0065ed0 0x7f36f0068380 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f370c14f100 tx=0x7f36f8005040 comp rx=0 tx=0).stop 2023-10-11T09:41:43.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.501+0000 7f37118d5700 1 -- 172.21.15.78:0/874986885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36fc004870 msgr2=0x7f370c1458d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:43.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.501+0000 7f37118d5700 1 --2- 172.21.15.78:0/874986885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36fc004870 0x7f370c1458d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3700001200 tx=0x7f370000eb10 comp rx=0 tx=0).stop 2023-10-11T09:41:43.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.501+0000 7f37118d5700 1 -- 172.21.15.78:0/874986885 shutdown_connections 2023-10-11T09:41:43.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.501+0000 7f37118d5700 1 --2- 172.21.15.78:0/874986885 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f36f0065ed0 0x7f36f0068380 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:43.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.501+0000 7f37118d5700 1 --2- 172.21.15.78:0/874986885 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f370c149670 0x7f370c0b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:43.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.501+0000 7f37118d5700 1 --2- 172.21.15.78:0/874986885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f370c149200 0x7f370c0b0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:43.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.501+0000 7f37118d5700 1 --2- 172.21.15.78:0/874986885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36fc004870 0x7f370c1458d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:43.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.501+0000 7f37118d5700 1 -- 172.21.15.78:0/874986885 >> 172.21.15.78:0/874986885 conn(0x7f370c06a060 msgr2=0x7f370c06bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:43.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.501+0000 7f37118d5700 1 -- 172.21.15.78:0/874986885 shutdown_connections 2023-10-11T09:41:43.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.505+0000 7f37118d5700 1 -- 172.21.15.78:0/874986885 wait complete. 2023-10-11T09:41:43.532 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:41:43.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.705+0000 7f48f9a19700 1 Processor -- start 2023-10-11T09:41:43.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.709+0000 7f48f9a19700 1 -- start start 2023-10-11T09:41:43.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.709+0000 7f48f9a19700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f48f40b3610 0x7f48f40d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:43.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.709+0000 7f48f9a19700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48f40b3a90 0x7f48f40b85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:43.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.709+0000 7f48f9a19700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48f40b47f0 con 0x7f48f40b3240 2023-10-11T09:41:43.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.709+0000 7f48f9a19700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48f40b8130 con 0x7f48f40b3610 2023-10-11T09:41:43.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.709+0000 7f48f9a19700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48f40b3f30 con 0x7f48f40b3a90 2023-10-11T09:41:43.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.713+0000 7f48f3fff700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f48f40b3610 0x7f48f40d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:43.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.713+0000 7f48f9218700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48f40b3a90 0x7f48f40b85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:43.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.713+0000 7f48f8a17700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f48f40b3240 0x7f48f415d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:34986/0 (socket says 172.21.15.78:34986) 2023-10-11T09:41:43.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.713+0000 7f48f8a17700 1 -- 172.21.15.78:0/3800299031 learned_addr learned my addr 172.21.15.78:0/3800299031 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:43.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.713+0000 7f48f37fe700 1 -- 172.21.15.78:0/3800299031 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1666838424 0 0) 0x7f48e8001110 con 0x7f48f40b3240 2023-10-11T09:41:43.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.713+0000 7f48f37fe700 1 -- 172.21.15.78:0/3800299031 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48e40035d0 con 0x7f48f40b3240 2023-10-11T09:41:43.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.713+0000 7f48f3fff700 1 -- 172.21.15.78:0/3800299031 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48f40b3a90 msgr2=0x7f48f40b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:43.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.713+0000 7f48f3fff700 1 --2- 172.21.15.78:0/3800299031 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48f40b3a90 0x7f48f40b85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:43.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.713+0000 7f48f3fff700 1 -- 172.21.15.78:0/3800299031 >> v1:172.21.15.133:6789/0 conn(0x7f48f40b3240 legacy=0x7f48f415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:43.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.713+0000 7f48f3fff700 1 -- 172.21.15.78:0/3800299031 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f40b41e0 con 0x7f48f40b3610 2023-10-11T09:41:43.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.713+0000 7f48f9218700 1 --2- 172.21.15.78:0/3800299031 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48f40b3a90 0x7f48f40b85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:41:43.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.713+0000 7f48f3fff700 1 --2- 172.21.15.78:0/3800299031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48f40b3610 0x7f48f40d29d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f48e0009850 tx=0x7f48e0009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d3d592bd85daa5d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:43.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.713+0000 7f48f37fe700 1 -- 172.21.15.78:0/3800299031 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e0011830 con 0x7f48f40b3610 2023-10-11T09:41:43.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.713+0000 7f48f37fe700 1 -- 172.21.15.78:0/3800299031 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e0011990 con 0x7f48f40b3610 2023-10-11T09:41:43.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.713+0000 7f48f37fe700 1 -- 172.21.15.78:0/3800299031 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e0011d40 con 0x7f48f40b3610 2023-10-11T09:41:43.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.713+0000 7f48f9a19700 1 -- 172.21.15.78:0/3800299031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48f40b3610 msgr2=0x7f48f40d29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:43.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.713+0000 7f48f9a19700 1 --2- 172.21.15.78:0/3800299031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48f40b3610 0x7f48f40d29d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f48e0009850 tx=0x7f48e0009b60 comp rx=0 tx=0).stop 2023-10-11T09:41:43.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.713+0000 7f48f9a19700 1 -- 172.21.15.78:0/3800299031 shutdown_connections 2023-10-11T09:41:43.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.713+0000 7f48f9a19700 1 --2- 172.21.15.78:0/3800299031 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48f40b3a90 0x7f48f40b85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:43.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.713+0000 7f48f9a19700 1 --2- 172.21.15.78:0/3800299031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48f40b3610 0x7f48f40d29d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:43.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.713+0000 7f48f9a19700 1 -- 172.21.15.78:0/3800299031 >> 172.21.15.78:0/3800299031 conn(0x7f48f409c260 msgr2=0x7f48f40d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:43.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.717+0000 7f48f9a19700 1 -- 172.21.15.78:0/3800299031 shutdown_connections 2023-10-11T09:41:43.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.717+0000 7f48f9a19700 1 -- 172.21.15.78:0/3800299031 wait complete. 2023-10-11T09:41:43.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.717+0000 7f48f9a19700 1 Processor -- start 2023-10-11T09:41:43.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.717+0000 7f48f9a19700 1 -- start start 2023-10-11T09:41:43.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.717+0000 7f48f9a19700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48f40b3240 0x7f48f415df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:43.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.717+0000 7f48f9a19700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48f40b3610 0x7f48f40c8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:43.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.717+0000 7f48f9a19700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48f40b3a90 0x7f48f4159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:43.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.717+0000 7f48f9a19700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f40c9a80 con 0x7f48f40b3240 2023-10-11T09:41:43.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.717+0000 7f48f9a19700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f40c6ac0 con 0x7f48f40b3610 2023-10-11T09:41:43.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.717+0000 7f48f9a19700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48f415e630 con 0x7f48f40b3a90 2023-10-11T09:41:43.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.717+0000 7f48f8a17700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48f40b3240 0x7f48f415df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:43.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.717+0000 7f48f8a17700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48f40b3240 0x7f48f415df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54576/0 (socket says 172.21.15.78:54576) 2023-10-11T09:41:43.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.717+0000 7f48f8a17700 1 -- 172.21.15.78:0/1575836991 learned_addr learned my addr 172.21.15.78:0/1575836991 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:43.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.721+0000 7f48f3fff700 1 --2- 172.21.15.78:0/1575836991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48f40b3610 0x7f48f40c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:43.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.721+0000 7f48f9218700 1 --2- 172.21.15.78:0/1575836991 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48f40b3a90 0x7f48f4159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:43.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.721+0000 7f48f8a17700 1 -- 172.21.15.78:0/1575836991 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48f40b3a90 msgr2=0x7f48f4159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:43.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.721+0000 7f48f8a17700 1 --2- 172.21.15.78:0/1575836991 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48f40b3a90 0x7f48f4159d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:43.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.721+0000 7f48f8a17700 1 -- 172.21.15.78:0/1575836991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48f40b3610 msgr2=0x7f48f40c8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:43.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.721+0000 7f48f8a17700 1 --2- 172.21.15.78:0/1575836991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48f40b3610 0x7f48f40c8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:43.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.721+0000 7f48f8a17700 1 -- 172.21.15.78:0/1575836991 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ec009a70 con 0x7f48f40b3240 2023-10-11T09:41:43.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.721+0000 7f48f3fff700 1 --2- 172.21.15.78:0/1575836991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48f40b3610 0x7f48f40c8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:43.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.721+0000 7f48f8a17700 1 --2- 172.21.15.78:0/1575836991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48f40b3240 0x7f48f415df70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f48e8001270 tx=0x7f48e800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:43.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.721+0000 7f48f17fa700 1 -- 172.21.15.78:0/1575836991 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e8005d90 con 0x7f48f40b3240 2023-10-11T09:41:43.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.721+0000 7f48f17fa700 1 -- 172.21.15.78:0/1575836991 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e80070d0 con 0x7f48f40b3240 2023-10-11T09:41:43.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.721+0000 7f48f9a19700 1 -- 172.21.15.78:0/1575836991 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48e00094b0 con 0x7f48f40b3240 2023-10-11T09:41:43.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.721+0000 7f48f17fa700 1 -- 172.21.15.78:0/1575836991 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e8007480 con 0x7f48f40b3240 2023-10-11T09:41:43.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.721+0000 7f48f9a19700 1 -- 172.21.15.78:0/1575836991 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48f40c6d10 con 0x7f48f40b3240 2023-10-11T09:41:43.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.721+0000 7f48f9a19700 1 -- 172.21.15.78:0/1575836991 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48f4065d40 con 0x7f48f40b3240 2023-10-11T09:41:43.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.725+0000 7f48f17fa700 1 -- 172.21.15.78:0/1575836991 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f48e80075e0 con 0x7f48f40b3240 2023-10-11T09:41:43.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.725+0000 7f48f17fa700 1 --2- 172.21.15.78:0/1575836991 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f48d8065ed0 0x7f48d8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:43.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.725+0000 7f48f17fa700 1 -- 172.21.15.78:0/1575836991 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f48e8082c10 con 0x7f48f40b3240 2023-10-11T09:41:43.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.725+0000 7f48f3fff700 1 --2- 172.21.15.78:0/1575836991 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f48d8065ed0 0x7f48d8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:43.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.725+0000 7f48f3fff700 1 --2- 172.21.15.78:0/1575836991 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f48d8065ed0 0x7f48d8068380 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f48e00090b0 tx=0x7f48e0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:43.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.729+0000 7f48f17fa700 1 -- 172.21.15.78:0/1575836991 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f48e8050210 con 0x7f48f40b3240 2023-10-11T09:41:43.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.901+0000 7f48f9a19700 1 -- 172.21.15.78:0/1575836991 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f48f40b7ab0 con 0x7f48f40b3240 2023-10-11T09:41:43.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.901+0000 7f48f17fa700 1 -- 172.21.15.78:0/1575836991 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f48e80170a0 con 0x7f48f40b3240 2023-10-11T09:41:43.907 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:41:43.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.901+0000 7f48f9a19700 1 -- 172.21.15.78:0/1575836991 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f48d8065ed0 msgr2=0x7f48d8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:43.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.901+0000 7f48f9a19700 1 --2- 172.21.15.78:0/1575836991 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f48d8065ed0 0x7f48d8068380 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f48e00090b0 tx=0x7f48e0008030 comp rx=0 tx=0).stop 2023-10-11T09:41:43.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.901+0000 7f48f9a19700 1 -- 172.21.15.78:0/1575836991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48f40b3240 msgr2=0x7f48f415df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:43.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.901+0000 7f48f9a19700 1 --2- 172.21.15.78:0/1575836991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48f40b3240 0x7f48f415df70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f48e8001270 tx=0x7f48e800c880 comp rx=0 tx=0).stop 2023-10-11T09:41:43.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.905+0000 7f48f9a19700 1 -- 172.21.15.78:0/1575836991 shutdown_connections 2023-10-11T09:41:43.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.905+0000 7f48f9a19700 1 --2- 172.21.15.78:0/1575836991 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f48d8065ed0 0x7f48d8068380 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:43.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.905+0000 7f48f9a19700 1 --2- 172.21.15.78:0/1575836991 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48f40b3a90 0x7f48f4159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:43.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.905+0000 7f48f9a19700 1 --2- 172.21.15.78:0/1575836991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48f40b3610 0x7f48f40c8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:43.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.905+0000 7f48f9a19700 1 --2- 172.21.15.78:0/1575836991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48f40b3240 0x7f48f415df70 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:43.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.905+0000 7f48f9a19700 1 -- 172.21.15.78:0/1575836991 >> 172.21.15.78:0/1575836991 conn(0x7f48f409c260 msgr2=0x7f48f40b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:43.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.905+0000 7f48f9a19700 1 -- 172.21.15.78:0/1575836991 shutdown_connections 2023-10-11T09:41:43.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:43.905+0000 7f48f9a19700 1 -- 172.21.15.78:0/1575836991 wait complete. 2023-10-11T09:41:43.928 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T09:41:44.114 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.109+0000 7f0cea243700 1 Processor -- start 2023-10-11T09:41:44.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.109+0000 7f0cea243700 1 -- start start 2023-10-11T09:41:44.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.113+0000 7f0cea243700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0ce4160cb0 0x7f0ce40cb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:44.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.113+0000 7f0cea243700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ce40b1030 0x7f0ce406ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:44.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.113+0000 7f0cea243700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ce4165570 con 0x7f0ce406d080 2023-10-11T09:41:44.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.113+0000 7f0cea243700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce4161080 con 0x7f0ce4160cb0 2023-10-11T09:41:44.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.113+0000 7f0cea243700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce4164750 con 0x7f0ce40b1030 2023-10-11T09:41:44.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.113+0000 7f0ce2ffd700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f0ce406d080 0x7f0ce40b0920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33262/0 (socket says 172.21.15.78:33262) 2023-10-11T09:41:44.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.113+0000 7f0ce2ffd700 1 -- 172.21.15.78:0/3329074999 learned_addr learned my addr 172.21.15.78:0/3329074999 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:44.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.113+0000 7f0ce37fe700 1 --2- 172.21.15.78:0/3329074999 >> v2:172.21.15.133:3300/0 conn(0x7f0ce4160cb0 0x7f0ce40cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:44.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.113+0000 7f0ce3fff700 1 --2- 172.21.15.78:0/3329074999 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ce40b1030 0x7f0ce406ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:44.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.113+0000 7f0ce27fc700 1 -- 172.21.15.78:0/3329074999 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2042322670 0 0) 0x7f0ce4165570 con 0x7f0ce406d080 2023-10-11T09:41:44.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.113+0000 7f0ce27fc700 1 -- 172.21.15.78:0/3329074999 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ccc0035d0 con 0x7f0ce406d080 2023-10-11T09:41:44.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.113+0000 7f0ce37fe700 1 -- 172.21.15.78:0/3329074999 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ce40b1030 msgr2=0x7f0ce406ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:44.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.113+0000 7f0ce37fe700 1 --2- 172.21.15.78:0/3329074999 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ce40b1030 0x7f0ce406ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:44.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.113+0000 7f0ce37fe700 1 -- 172.21.15.78:0/3329074999 >> v1:172.21.15.78:6789/0 conn(0x7f0ce406d080 legacy=0x7f0ce40b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:44.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.113+0000 7f0ce37fe700 1 -- 172.21.15.78:0/3329074999 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce4164960 con 0x7f0ce4160cb0 2023-10-11T09:41:44.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.113+0000 7f0ce37fe700 1 --2- 172.21.15.78:0/3329074999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ce4160cb0 0x7f0ce40cb100 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4009850 tx=0x7f0cd4009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=335e5860207f63bd server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:44.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.117+0000 7f0ce27fc700 1 -- 172.21.15.78:0/3329074999 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4011830 con 0x7f0ce4160cb0 2023-10-11T09:41:44.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.117+0000 7f0ce27fc700 1 -- 172.21.15.78:0/3329074999 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd4011990 con 0x7f0ce4160cb0 2023-10-11T09:41:44.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.117+0000 7f0ce27fc700 1 -- 172.21.15.78:0/3329074999 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4011d40 con 0x7f0ce4160cb0 2023-10-11T09:41:44.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.117+0000 7f0cea243700 1 -- 172.21.15.78:0/3329074999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ce4160cb0 msgr2=0x7f0ce40cb100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:44.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.117+0000 7f0cea243700 1 --2- 172.21.15.78:0/3329074999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ce4160cb0 0x7f0ce40cb100 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4009850 tx=0x7f0cd4009b60 comp rx=0 tx=0).stop 2023-10-11T09:41:44.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.117+0000 7f0cea243700 1 -- 172.21.15.78:0/3329074999 shutdown_connections 2023-10-11T09:41:44.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.117+0000 7f0cea243700 1 --2- 172.21.15.78:0/3329074999 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ce40b1030 0x7f0ce406ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:44.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.117+0000 7f0cea243700 1 --2- 172.21.15.78:0/3329074999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ce4160cb0 0x7f0ce40cb100 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:44.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.117+0000 7f0cea243700 1 -- 172.21.15.78:0/3329074999 >> 172.21.15.78:0/3329074999 conn(0x7f0ce40cecc0 msgr2=0x7f0ce40cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:44.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.117+0000 7f0cea243700 1 -- 172.21.15.78:0/3329074999 shutdown_connections 2023-10-11T09:41:44.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.117+0000 7f0cea243700 1 -- 172.21.15.78:0/3329074999 wait complete. 2023-10-11T09:41:44.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.117+0000 7f0cea243700 1 Processor -- start 2023-10-11T09:41:44.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.121+0000 7f0cea243700 1 -- start start 2023-10-11T09:41:44.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.121+0000 7f0cea243700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ce406d080 0x7f0ce40b3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:44.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.121+0000 7f0cea243700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ce40b1030 0x7f0ce4163230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:44.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.121+0000 7f0cea243700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ce4160cb0 0x7f0ce41f7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:44.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.121+0000 7f0cea243700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce41626d0 con 0x7f0ce406d080 2023-10-11T09:41:44.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.121+0000 7f0cea243700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce40b2b80 con 0x7f0ce4160cb0 2023-10-11T09:41:44.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.121+0000 7f0cea243700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce4155880 con 0x7f0ce40b1030 2023-10-11T09:41:44.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.121+0000 7f0ce37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ce406d080 0x7f0ce40b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:44.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.121+0000 7f0ce37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ce406d080 0x7f0ce40b3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54584/0 (socket says 172.21.15.78:54584) 2023-10-11T09:41:44.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.121+0000 7f0ce37fe700 1 -- 172.21.15.78:0/3931772438 learned_addr learned my addr 172.21.15.78:0/3931772438 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:44.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.121+0000 7f0ce37fe700 1 -- 172.21.15.78:0/3931772438 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ce40b1030 msgr2=0x7f0ce4163230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:44.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.121+0000 7f0ce2ffd700 1 --2- 172.21.15.78:0/3931772438 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ce40b1030 0x7f0ce4163230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:44.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.121+0000 7f0ce3fff700 1 --2- 172.21.15.78:0/3931772438 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ce4160cb0 0x7f0ce41f7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:44.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.121+0000 7f0ce37fe700 1 --2- 172.21.15.78:0/3931772438 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ce40b1030 0x7f0ce4163230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:44.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.121+0000 7f0ce37fe700 1 -- 172.21.15.78:0/3931772438 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ce4160cb0 msgr2=0x7f0ce41f7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:44.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.121+0000 7f0ce37fe700 1 --2- 172.21.15.78:0/3931772438 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ce4160cb0 0x7f0ce41f7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:44.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.121+0000 7f0ce37fe700 1 -- 172.21.15.78:0/3931772438 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd40094b0 con 0x7f0ce406d080 2023-10-11T09:41:44.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.121+0000 7f0ce3fff700 1 --2- 172.21.15.78:0/3931772438 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ce4160cb0 0x7f0ce41f7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:41:44.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.121+0000 7f0ce37fe700 1 --2- 172.21.15.78:0/3931772438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ce406d080 0x7f0ce40b3010 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0cd40090b0 tx=0x7f0cd4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:44.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.121+0000 7f0ccbfff700 1 -- 172.21.15.78:0/3931772438 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd40036c0 con 0x7f0ce406d080 2023-10-11T09:41:44.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.121+0000 7f0ccbfff700 1 -- 172.21.15.78:0/3931772438 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd401e070 con 0x7f0ce406d080 2023-10-11T09:41:44.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.121+0000 7f0cea243700 1 -- 172.21.15.78:0/3931772438 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce40b1c90 con 0x7f0ce406d080 2023-10-11T09:41:44.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.121+0000 7f0ccbfff700 1 -- 172.21.15.78:0/3931772438 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4003a80 con 0x7f0ce406d080 2023-10-11T09:41:44.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.121+0000 7f0cea243700 1 -- 172.21.15.78:0/3931772438 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce40b2250 con 0x7f0ce406d080 2023-10-11T09:41:44.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.125+0000 7f0cea243700 1 -- 172.21.15.78:0/3931772438 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ce406eb80 con 0x7f0ce406d080 2023-10-11T09:41:44.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.125+0000 7f0ccbfff700 1 -- 172.21.15.78:0/3931772438 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f0cd4003be0 con 0x7f0ce406d080 2023-10-11T09:41:44.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.125+0000 7f0ccbfff700 1 --2- 172.21.15.78:0/3931772438 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0cd0065ed0 0x7f0cd0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:44.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.125+0000 7f0ccbfff700 1 -- 172.21.15.78:0/3931772438 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f0cd4020070 con 0x7f0ce406d080 2023-10-11T09:41:44.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.125+0000 7f0ce2ffd700 1 --2- 172.21.15.78:0/3931772438 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0cd0065ed0 0x7f0cd0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:44.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.129+0000 7f0ce2ffd700 1 --2- 172.21.15.78:0/3931772438 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0cd0065ed0 0x7f0cd0068380 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4163e50 tx=0x7f0cd8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:44.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.133+0000 7f0ccbfff700 1 -- 172.21.15.78:0/3931772438 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f0cd4052290 con 0x7f0ce406d080 2023-10-11T09:41:44.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.305+0000 7f0cea243700 1 -- 172.21.15.78:0/3931772438 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0ce4164ef0 con 0x7f0ce406d080 2023-10-11T09:41:44.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.305+0000 7f0ccbfff700 1 -- 172.21.15.78:0/3931772438 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0cd4015020 con 0x7f0ce406d080 2023-10-11T09:41:44.313 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:41:44.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.313+0000 7f0cea243700 1 -- 172.21.15.78:0/3931772438 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0cd0065ed0 msgr2=0x7f0cd0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:44.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.313+0000 7f0cea243700 1 --2- 172.21.15.78:0/3931772438 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0cd0065ed0 0x7f0cd0068380 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4163e50 tx=0x7f0cd8009040 comp rx=0 tx=0).stop 2023-10-11T09:41:44.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.313+0000 7f0cea243700 1 -- 172.21.15.78:0/3931772438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ce406d080 msgr2=0x7f0ce40b3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:44.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.313+0000 7f0cea243700 1 --2- 172.21.15.78:0/3931772438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ce406d080 0x7f0ce40b3010 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0cd40090b0 tx=0x7f0cd4004770 comp rx=0 tx=0).stop 2023-10-11T09:41:44.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.313+0000 7f0cea243700 1 -- 172.21.15.78:0/3931772438 shutdown_connections 2023-10-11T09:41:44.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.313+0000 7f0cea243700 1 --2- 172.21.15.78:0/3931772438 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0cd0065ed0 0x7f0cd0068380 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:44.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.313+0000 7f0cea243700 1 --2- 172.21.15.78:0/3931772438 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ce4160cb0 0x7f0ce41f7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:44.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.313+0000 7f0cea243700 1 --2- 172.21.15.78:0/3931772438 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ce40b1030 0x7f0ce4163230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:44.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.313+0000 7f0cea243700 1 --2- 172.21.15.78:0/3931772438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ce406d080 0x7f0ce40b3010 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:44.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.313+0000 7f0cea243700 1 -- 172.21.15.78:0/3931772438 >> 172.21.15.78:0/3931772438 conn(0x7f0ce40cecc0 msgr2=0x7f0ce40cbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:44.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.313+0000 7f0cea243700 1 -- 172.21.15.78:0/3931772438 shutdown_connections 2023-10-11T09:41:44.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.313+0000 7f0cea243700 1 -- 172.21.15.78:0/3931772438 wait complete. 2023-10-11T09:41:44.340 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T09:41:44.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.509+0000 7fcad14a3700 1 Processor -- start 2023-10-11T09:41:44.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.509+0000 7fcad14a3700 1 -- start start 2023-10-11T09:41:44.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.509+0000 7fcad14a3700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcacc06d730 0x7fcacc165240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:44.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.509+0000 7fcad14a3700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcacc0b2d30 con 0x7fcacc15a6b0 2023-10-11T09:41:44.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.509+0000 7fcad14a3700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcacc0cf360 con 0x7fcacc0b3a60 2023-10-11T09:41:44.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.509+0000 7fcad14a3700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc0b2f80 con 0x7fcacc06d730 2023-10-11T09:41:44.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.513+0000 7fcacaffd700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fcacc15a6b0 0x7fcacc0ce8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33268/0 (socket says 172.21.15.78:33268) 2023-10-11T09:41:44.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.513+0000 7fcacaffd700 1 -- 172.21.15.78:0/55655811 learned_addr learned my addr 172.21.15.78:0/55655811 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:44.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.513+0000 7fcacb7fe700 1 --2- 172.21.15.78:0/55655811 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcacc06d730 0x7fcacc165240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:44.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.513+0000 7fcac9ffb700 1 -- 172.21.15.78:0/55655811 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3196552704 0 0) 0x7fcacc0b2d30 con 0x7fcacc15a6b0 2023-10-11T09:41:44.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.513+0000 7fcac9ffb700 1 -- 172.21.15.78:0/55655811 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcab40035d0 con 0x7fcacc15a6b0 2023-10-11T09:41:44.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.513+0000 7fcacb7fe700 1 -- 172.21.15.78:0/55655811 >> v1:172.21.15.133:6789/0 conn(0x7fcacc0b3a60 legacy=0x7fcacc06d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:44.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.513+0000 7fcacb7fe700 1 -- 172.21.15.78:0/55655811 >> v1:172.21.15.78:6789/0 conn(0x7fcacc15a6b0 legacy=0x7fcacc0ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:44.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.513+0000 7fcacb7fe700 1 -- 172.21.15.78:0/55655811 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcacc0b3260 con 0x7fcacc06d730 2023-10-11T09:41:44.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.513+0000 7fcacb7fe700 1 --2- 172.21.15.78:0/55655811 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcacc06d730 0x7fcacc165240 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcac4009dc0 tx=0x7fcac400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc5eef16c7cd542f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:44.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.513+0000 7fcac9ffb700 1 -- 172.21.15.78:0/55655811 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac400e070 con 0x7fcacc06d730 2023-10-11T09:41:44.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.513+0000 7fcac9ffb700 1 -- 172.21.15.78:0/55655811 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcac400bb20 con 0x7fcacc06d730 2023-10-11T09:41:44.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.513+0000 7fcac9ffb700 1 -- 172.21.15.78:0/55655811 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac4012430 con 0x7fcacc06d730 2023-10-11T09:41:44.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.517+0000 7fcad14a3700 1 -- 172.21.15.78:0/55655811 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcacc06d730 msgr2=0x7fcacc165240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:44.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.517+0000 7fcad14a3700 1 --2- 172.21.15.78:0/55655811 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcacc06d730 0x7fcacc165240 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcac4009dc0 tx=0x7fcac400b330 comp rx=0 tx=0).stop 2023-10-11T09:41:44.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.517+0000 7fcad14a3700 1 -- 172.21.15.78:0/55655811 shutdown_connections 2023-10-11T09:41:44.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.517+0000 7fcad14a3700 1 --2- 172.21.15.78:0/55655811 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcacc06d730 0x7fcacc165240 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:44.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.517+0000 7fcad14a3700 1 -- 172.21.15.78:0/55655811 >> 172.21.15.78:0/55655811 conn(0x7fcacc06a060 msgr2=0x7fcacc06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:44.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.517+0000 7fcad14a3700 1 -- 172.21.15.78:0/55655811 shutdown_connections 2023-10-11T09:41:44.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.517+0000 7fcad14a3700 1 -- 172.21.15.78:0/55655811 wait complete. 2023-10-11T09:41:44.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.517+0000 7fcad14a3700 1 Processor -- start 2023-10-11T09:41:44.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.521+0000 7fcad14a3700 1 -- start start 2023-10-11T09:41:44.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.521+0000 7fcad14a3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcacc06d730 0x7fcacc1f3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:44.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.521+0000 7fcad14a3700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcacc0b3a60 0x7fcacc145080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:44.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.521+0000 7fcad14a3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcacc15a6b0 0x7fcacc0afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:44.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.521+0000 7fcad14a3700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc146a00 con 0x7fcacc15a6b0 2023-10-11T09:41:44.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.521+0000 7fcad14a3700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc1f4170 con 0x7fcacc06d730 2023-10-11T09:41:44.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.521+0000 7fcad14a3700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc146c50 con 0x7fcacc0b3a60 2023-10-11T09:41:44.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.521+0000 7fcacb7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcacc15a6b0 0x7fcacc0afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:44.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.521+0000 7fcacb7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcacc15a6b0 0x7fcacc0afdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54590/0 (socket says 172.21.15.78:54590) 2023-10-11T09:41:44.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.521+0000 7fcacb7fe700 1 -- 172.21.15.78:0/2533626317 learned_addr learned my addr 172.21.15.78:0/2533626317 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:44.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.521+0000 7fcacaffd700 1 --2- 172.21.15.78:0/2533626317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcacc06d730 0x7fcacc1f3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:44.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.521+0000 7fcaca7fc700 1 --2- 172.21.15.78:0/2533626317 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcacc0b3a60 0x7fcacc145080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:44.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.521+0000 7fcacb7fe700 1 -- 172.21.15.78:0/2533626317 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcacc0b3a60 msgr2=0x7fcacc145080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:44.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.521+0000 7fcacb7fe700 1 --2- 172.21.15.78:0/2533626317 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcacc0b3a60 0x7fcacc145080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:44.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.521+0000 7fcacb7fe700 1 -- 172.21.15.78:0/2533626317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcacc06d730 msgr2=0x7fcacc1f3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:44.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.521+0000 7fcacb7fe700 1 --2- 172.21.15.78:0/2533626317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcacc06d730 0x7fcacc1f3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:44.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.521+0000 7fcacb7fe700 1 -- 172.21.15.78:0/2533626317 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac4009a70 con 0x7fcacc15a6b0 2023-10-11T09:41:44.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.521+0000 7fcacb7fe700 1 --2- 172.21.15.78:0/2533626317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcacc15a6b0 0x7fcacc0afdd0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fcacc0725d0 tx=0x7fcac4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:44.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.521+0000 7fcab37fe700 1 -- 172.21.15.78:0/2533626317 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac4012430 con 0x7fcacc15a6b0 2023-10-11T09:41:44.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.521+0000 7fcab37fe700 1 -- 172.21.15.78:0/2533626317 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcac4020070 con 0x7fcacc15a6b0 2023-10-11T09:41:44.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.521+0000 7fcad14a3700 1 -- 172.21.15.78:0/2533626317 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcacc146f30 con 0x7fcacc15a6b0 2023-10-11T09:41:44.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.525+0000 7fcab37fe700 1 -- 172.21.15.78:0/2533626317 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac4004280 con 0x7fcacc15a6b0 2023-10-11T09:41:44.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.525+0000 7fcad14a3700 1 -- 172.21.15.78:0/2533626317 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcacc1474f0 con 0x7fcacc15a6b0 2023-10-11T09:41:44.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.525+0000 7fcad14a3700 1 -- 172.21.15.78:0/2533626317 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcacc065d40 con 0x7fcacc15a6b0 2023-10-11T09:41:44.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.525+0000 7fcab37fe700 1 -- 172.21.15.78:0/2533626317 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fcac40043e0 con 0x7fcacc15a6b0 2023-10-11T09:41:44.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.529+0000 7fcab37fe700 1 --2- 172.21.15.78:0/2533626317 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fcab8065ed0 0x7fcab8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:44.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.529+0000 7fcab37fe700 1 -- 172.21.15.78:0/2533626317 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fcac4022070 con 0x7fcacc15a6b0 2023-10-11T09:41:44.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.529+0000 7fcacaffd700 1 --2- 172.21.15.78:0/2533626317 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fcab8065ed0 0x7fcab8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:44.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.529+0000 7fcacaffd700 1 --2- 172.21.15.78:0/2533626317 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fcab8065ed0 0x7fcab8068380 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcabc0011e0 tx=0x7fcabc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:44.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.533+0000 7fcab37fe700 1 -- 172.21.15.78:0/2533626317 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fcac4052fa0 con 0x7fcacc15a6b0 2023-10-11T09:41:44.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.709+0000 7fcad14a3700 1 -- 172.21.15.78:0/2533626317 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcacc156360 con 0x7fcacc15a6b0 2023-10-11T09:41:44.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.709+0000 7fcab37fe700 1 -- 172.21.15.78:0/2533626317 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcac40565c0 con 0x7fcacc15a6b0 2023-10-11T09:41:44.716 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2023-10-11T09:41:44.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.713+0000 7fcad14a3700 1 -- 172.21.15.78:0/2533626317 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fcab8065ed0 msgr2=0x7fcab8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:44.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.713+0000 7fcad14a3700 1 --2- 172.21.15.78:0/2533626317 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fcab8065ed0 0x7fcab8068380 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcabc0011e0 tx=0x7fcabc009040 comp rx=0 tx=0).stop 2023-10-11T09:41:44.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.713+0000 7fcad14a3700 1 -- 172.21.15.78:0/2533626317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcacc15a6b0 msgr2=0x7fcacc0afdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:44.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.713+0000 7fcad14a3700 1 --2- 172.21.15.78:0/2533626317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcacc15a6b0 0x7fcacc0afdd0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fcacc0725d0 tx=0x7fcac4004af0 comp rx=0 tx=0).stop 2023-10-11T09:41:44.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.713+0000 7fcad14a3700 1 -- 172.21.15.78:0/2533626317 shutdown_connections 2023-10-11T09:41:44.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.713+0000 7fcad14a3700 1 --2- 172.21.15.78:0/2533626317 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fcab8065ed0 0x7fcab8068380 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:44.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.713+0000 7fcad14a3700 1 --2- 172.21.15.78:0/2533626317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcacc15a6b0 0x7fcacc0afdd0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:44.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.713+0000 7fcad14a3700 1 --2- 172.21.15.78:0/2533626317 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcacc0b3a60 0x7fcacc145080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:44.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.713+0000 7fcad14a3700 1 --2- 172.21.15.78:0/2533626317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcacc06d730 0x7fcacc1f3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:44.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.713+0000 7fcad14a3700 1 -- 172.21.15.78:0/2533626317 >> 172.21.15.78:0/2533626317 conn(0x7fcacc06a060 msgr2=0x7fcacc06bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:44.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.713+0000 7fcad14a3700 1 -- 172.21.15.78:0/2533626317 shutdown_connections 2023-10-11T09:41:44.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.713+0000 7fcad14a3700 1 -- 172.21.15.78:0/2533626317 wait complete. 2023-10-11T09:41:44.739 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:41:44.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.905+0000 7f91801a7700 1 Processor -- start 2023-10-11T09:41:44.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.909+0000 7f91801a7700 1 -- start start 2023-10-11T09:41:44.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.909+0000 7f91801a7700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9178071040 0x7f91780cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:44.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.909+0000 7f91801a7700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f91780cba00 0x7f917806c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:44.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.909+0000 7f91801a7700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91780720f0 con 0x7f9178154e00 2023-10-11T09:41:44.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.909+0000 7f91801a7700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91780714e0 con 0x7f91780cba00 2023-10-11T09:41:44.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.909+0000 7f91801a7700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91780717c0 con 0x7f9178071040 2023-10-11T09:41:44.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.909+0000 7f917e744700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f91780cba00 0x7f917806c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:44.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.913+0000 7f917d742700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9178071040 0x7f91780cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:44.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.913+0000 7f917e744700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f91780cba00 0x7f917806c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40030/0 (socket says 172.21.15.78:40030) 2023-10-11T09:41:44.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.913+0000 7f917e744700 1 -- 172.21.15.78:0/737677588 learned_addr learned my addr 172.21.15.78:0/737677588 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:44.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.913+0000 7f917d742700 1 -- 172.21.15.78:0/737677588 >> v2:172.21.15.78:3300/0 conn(0x7f91780cba00 msgr2=0x7f917806c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:44.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.913+0000 7f917d742700 1 --2- 172.21.15.78:0/737677588 >> v2:172.21.15.78:3300/0 conn(0x7f91780cba00 0x7f917806c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:44.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.913+0000 7f917d742700 1 -- 172.21.15.78:0/737677588 >> v1:172.21.15.133:6789/0 conn(0x7f9178154e00 legacy=0x7f917815b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:44.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.913+0000 7f917d742700 1 -- 172.21.15.78:0/737677588 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9178071aa0 con 0x7f9178071040 2023-10-11T09:41:44.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.913+0000 7f917e744700 1 --2- 172.21.15.78:0/737677588 >> v2:172.21.15.78:3300/0 conn(0x7f91780cba00 0x7f917806c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:41:44.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.913+0000 7f917d742700 1 --2- 172.21.15.78:0/737677588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9178071040 0x7f91780cb4c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f91780707f0 tx=0x7f9168009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44c5da769ffd16e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:44.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.913+0000 7f917cf41700 1 -- 172.21.15.78:0/737677588 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9168011830 con 0x7f9178071040 2023-10-11T09:41:44.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.913+0000 7f917cf41700 1 -- 172.21.15.78:0/737677588 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9168011990 con 0x7f9178071040 2023-10-11T09:41:44.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.913+0000 7f917cf41700 1 -- 172.21.15.78:0/737677588 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9168011d20 con 0x7f9178071040 2023-10-11T09:41:44.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.913+0000 7f91801a7700 1 -- 172.21.15.78:0/737677588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9178071040 msgr2=0x7f91780cb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:44.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.913+0000 7f91801a7700 1 --2- 172.21.15.78:0/737677588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9178071040 0x7f91780cb4c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f91780707f0 tx=0x7f9168009b30 comp rx=0 tx=0).stop 2023-10-11T09:41:44.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.913+0000 7f91801a7700 1 -- 172.21.15.78:0/737677588 shutdown_connections 2023-10-11T09:41:44.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.913+0000 7f91801a7700 1 --2- 172.21.15.78:0/737677588 >> v2:172.21.15.78:3300/0 conn(0x7f91780cba00 0x7f917806c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:44.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.913+0000 7f91801a7700 1 --2- 172.21.15.78:0/737677588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9178071040 0x7f91780cb4c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:44.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.913+0000 7f91801a7700 1 -- 172.21.15.78:0/737677588 >> 172.21.15.78:0/737677588 conn(0x7f917806aca0 msgr2=0x7f91780b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:44.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.913+0000 7f91801a7700 1 -- 172.21.15.78:0/737677588 shutdown_connections 2023-10-11T09:41:44.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.913+0000 7f91801a7700 1 -- 172.21.15.78:0/737677588 wait complete. 2023-10-11T09:41:44.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.917+0000 7f91801a7700 1 Processor -- start 2023-10-11T09:41:44.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.917+0000 7f91801a7700 1 -- start start 2023-10-11T09:41:44.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.917+0000 7f91801a7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9178071040 0x7f9178155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:44.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.917+0000 7f91801a7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91780cba00 0x7f91780b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:44.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.917+0000 7f91801a7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9178154e00 0x7f9178156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:44.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.917+0000 7f91801a7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9178156410 con 0x7f91780cba00 2023-10-11T09:41:44.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.917+0000 7f91801a7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9178155430 con 0x7f9178154e00 2023-10-11T09:41:44.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.917+0000 7f91801a7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9178157770 con 0x7f9178071040 2023-10-11T09:41:44.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.917+0000 7f917d742700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91780cba00 0x7f91780b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:44.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.921+0000 7f917d742700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91780cba00 0x7f91780b34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40046/0 (socket says 172.21.15.78:40046) 2023-10-11T09:41:44.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.921+0000 7f917d742700 1 -- 172.21.15.78:0/1275815492 learned_addr learned my addr 172.21.15.78:0/1275815492 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:44.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.921+0000 7f917e744700 1 --2- 172.21.15.78:0/1275815492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9178154e00 0x7f9178156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:44.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.921+0000 7f917df43700 1 --2- 172.21.15.78:0/1275815492 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9178071040 0x7f9178155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:44.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.921+0000 7f917d742700 1 -- 172.21.15.78:0/1275815492 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9178071040 msgr2=0x7f9178155ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:44.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.921+0000 7f917d742700 1 --2- 172.21.15.78:0/1275815492 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9178071040 0x7f9178155ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:44.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.921+0000 7f917d742700 1 -- 172.21.15.78:0/1275815492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9178154e00 msgr2=0x7f9178156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:44.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.921+0000 7f917d742700 1 --2- 172.21.15.78:0/1275815492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9178154e00 0x7f9178156920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:44.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.921+0000 7f917d742700 1 -- 172.21.15.78:0/1275815492 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9170009b30 con 0x7f91780cba00 2023-10-11T09:41:44.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.921+0000 7f917e744700 1 --2- 172.21.15.78:0/1275815492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9178154e00 0x7f9178156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:41:44.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.921+0000 7f917d742700 1 --2- 172.21.15.78:0/1275815492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91780cba00 0x7f91780b34f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f91680090b0 tx=0x7f9168004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:44.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.921+0000 7f916e7fc700 1 -- 172.21.15.78:0/1275815492 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9168004400 con 0x7f91780cba00 2023-10-11T09:41:44.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.921+0000 7f916e7fc700 1 -- 172.21.15.78:0/1275815492 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f916801f070 con 0x7f91780cba00 2023-10-11T09:41:44.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.921+0000 7f91801a7700 1 -- 172.21.15.78:0/1275815492 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91680094b0 con 0x7f91780cba00 2023-10-11T09:41:44.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.921+0000 7f916e7fc700 1 -- 172.21.15.78:0/1275815492 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91680036c0 con 0x7f91780cba00 2023-10-11T09:41:44.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.921+0000 7f91801a7700 1 -- 172.21.15.78:0/1275815492 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91781556d0 con 0x7f91780cba00 2023-10-11T09:41:44.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.921+0000 7f91801a7700 1 -- 172.21.15.78:0/1275815492 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9178155290 con 0x7f91780cba00 2023-10-11T09:41:44.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.925+0000 7f916e7fc700 1 -- 172.21.15.78:0/1275815492 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f9168021030 con 0x7f91780cba00 2023-10-11T09:41:44.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.925+0000 7f916e7fc700 1 --2- 172.21.15.78:0/1275815492 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f915c065ed0 0x7f915c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:44.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.925+0000 7f916e7fc700 1 -- 172.21.15.78:0/1275815492 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f9168084c20 con 0x7f91780cba00 2023-10-11T09:41:44.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.929+0000 7f917df43700 1 --2- 172.21.15.78:0/1275815492 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f915c065ed0 0x7f915c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:44.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.929+0000 7f917df43700 1 --2- 172.21.15.78:0/1275815492 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f915c065ed0 0x7f915c068380 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9174000f60 tx=0x7f9174009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:44.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:44.933+0000 7f916e7fc700 1 -- 172.21.15.78:0/1275815492 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f9178155290 con 0x7f91780cba00 2023-10-11T09:41:45.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.137+0000 7f91801a7700 1 -- 172.21.15.78:0/1275815492 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9178155260 con 0x7f91780cba00 2023-10-11T09:41:45.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.137+0000 7f916e7fc700 1 -- 172.21.15.78:0/1275815492 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4648 (secure 0 0 0) 0x7f9168052340 con 0x7f91780cba00 2023-10-11T09:41:45.143 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:41:45.143 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/4282378510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4282378510},{"type":"v1","addr":"172.21.15.133:6835","nonce":4282378510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3986631249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3986631249},{"type":"v1","addr":"172.21.15.78:6835","nonce":3986631249}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3135421902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3135421902},{"type":"v1","addr":"172.21.15.78:6837","nonce":3135421902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:40:41.673556+0000","modified":"2023-10-11T09:40:45.122852+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6834/3925157967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":3925157967},{"type":"v1","addr":"172.21.15.133:6834","nonce":3925157967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-11T09:41:45.145 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.141+0000 7f91801a7700 1 -- 172.21.15.78:0/1275815492 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f915c065ed0 msgr2=0x7f915c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:45.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.141+0000 7f91801a7700 1 --2- 172.21.15.78:0/1275815492 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f915c065ed0 0x7f915c068380 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9174000f60 tx=0x7f9174009040 comp rx=0 tx=0).stop 2023-10-11T09:41:45.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.141+0000 7f91801a7700 1 -- 172.21.15.78:0/1275815492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91780cba00 msgr2=0x7f91780b34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:45.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.141+0000 7f91801a7700 1 --2- 172.21.15.78:0/1275815492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91780cba00 0x7f91780b34f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f91680090b0 tx=0x7f9168004770 comp rx=0 tx=0).stop 2023-10-11T09:41:45.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.141+0000 7f91801a7700 1 -- 172.21.15.78:0/1275815492 shutdown_connections 2023-10-11T09:41:45.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.141+0000 7f91801a7700 1 --2- 172.21.15.78:0/1275815492 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f915c065ed0 0x7f915c068380 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:45.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.141+0000 7f91801a7700 1 --2- 172.21.15.78:0/1275815492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9178154e00 0x7f9178156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:45.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.141+0000 7f91801a7700 1 --2- 172.21.15.78:0/1275815492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91780cba00 0x7f91780b34f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:45.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.141+0000 7f91801a7700 1 --2- 172.21.15.78:0/1275815492 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9178071040 0x7f9178155ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:45.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.141+0000 7f91801a7700 1 -- 172.21.15.78:0/1275815492 >> 172.21.15.78:0/1275815492 conn(0x7f917806aca0 msgr2=0x7f917815dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:45.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.141+0000 7f91801a7700 1 -- 172.21.15.78:0/1275815492 shutdown_connections 2023-10-11T09:41:45.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.141+0000 7f91801a7700 1 -- 172.21.15.78:0/1275815492 wait complete. 2023-10-11T09:41:45.148 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 8 2023-10-11T09:41:45.169 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:41:45.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.333+0000 7fe6474d8700 1 Processor -- start 2023-10-11T09:41:45.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.333+0000 7fe6474d8700 1 -- start start 2023-10-11T09:41:45.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.337+0000 7fe6474d8700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe640154e00 0x7fe64015b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:45.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.337+0000 7fe6474d8700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe6400cbbd0 0x7fe64006c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:45.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.337+0000 7fe6474d8700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6400720f0 con 0x7fe640071040 2023-10-11T09:41:45.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.337+0000 7fe6474d8700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe64015bb90 con 0x7fe640154e00 2023-10-11T09:41:45.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.337+0000 7fe6474d8700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6400714e0 con 0x7fe6400cbbd0 2023-10-11T09:41:45.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.337+0000 7fe645274700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe640154e00 0x7fe64015b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:45.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.337+0000 7fe645274700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe640154e00 0x7fe64015b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40060/0 (socket says 172.21.15.78:40060) 2023-10-11T09:41:45.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.337+0000 7fe645274700 1 -- 172.21.15.78:0/1964952029 learned_addr learned my addr 172.21.15.78:0/1964952029 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:45.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.337+0000 7fe645a75700 1 --2- 172.21.15.78:0/1964952029 >> v2:172.21.15.133:3300/0 conn(0x7fe6400cbbd0 0x7fe64006c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:45.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.337+0000 7fe645274700 1 -- 172.21.15.78:0/1964952029 >> v2:172.21.15.133:3300/0 conn(0x7fe6400cbbd0 msgr2=0x7fe64006c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:45.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.337+0000 7fe645274700 1 --2- 172.21.15.78:0/1964952029 >> v2:172.21.15.133:3300/0 conn(0x7fe6400cbbd0 0x7fe64006c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:45.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.337+0000 7fe645274700 1 -- 172.21.15.78:0/1964952029 >> v1:172.21.15.133:6789/0 conn(0x7fe640071040 legacy=0x7fe6400cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:45.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.337+0000 7fe645274700 1 -- 172.21.15.78:0/1964952029 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6400717c0 con 0x7fe640154e00 2023-10-11T09:41:45.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.341+0000 7fe645274700 1 --2- 172.21.15.78:0/1964952029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe640154e00 0x7fe64015b650 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe64006f700 tx=0x7fe63c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8613e5a170445a42 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:45.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.341+0000 7fe637fff700 1 -- 172.21.15.78:0/1964952029 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63c00b8a0 con 0x7fe640154e00 2023-10-11T09:41:45.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.341+0000 7fe637fff700 1 -- 172.21.15.78:0/1964952029 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe63c00ba00 con 0x7fe640154e00 2023-10-11T09:41:45.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.341+0000 7fe637fff700 1 -- 172.21.15.78:0/1964952029 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63c00bd90 con 0x7fe640154e00 2023-10-11T09:41:45.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.341+0000 7fe6474d8700 1 -- 172.21.15.78:0/1964952029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe640154e00 msgr2=0x7fe64015b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:45.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.341+0000 7fe6474d8700 1 --2- 172.21.15.78:0/1964952029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe640154e00 0x7fe64015b650 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe64006f700 tx=0x7fe63c009bf0 comp rx=0 tx=0).stop 2023-10-11T09:41:45.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.341+0000 7fe6474d8700 1 -- 172.21.15.78:0/1964952029 shutdown_connections 2023-10-11T09:41:45.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.341+0000 7fe6474d8700 1 --2- 172.21.15.78:0/1964952029 >> v2:172.21.15.133:3300/0 conn(0x7fe6400cbbd0 0x7fe64006c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:45.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.341+0000 7fe6474d8700 1 --2- 172.21.15.78:0/1964952029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe640154e00 0x7fe64015b650 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:45.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.341+0000 7fe6474d8700 1 -- 172.21.15.78:0/1964952029 >> 172.21.15.78:0/1964952029 conn(0x7fe64006aca0 msgr2=0x7fe6400b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:45.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.341+0000 7fe6474d8700 1 -- 172.21.15.78:0/1964952029 shutdown_connections 2023-10-11T09:41:45.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.341+0000 7fe6474d8700 1 -- 172.21.15.78:0/1964952029 wait complete. 2023-10-11T09:41:45.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.345+0000 7fe6474d8700 1 Processor -- start 2023-10-11T09:41:45.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.345+0000 7fe6474d8700 1 -- start start 2023-10-11T09:41:45.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.345+0000 7fe6474d8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe640071040 0x7fe6400ce310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:45.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.345+0000 7fe6474d8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6400cbbd0 0x7fe64014e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:45.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.345+0000 7fe6474d8700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe640154e00 0x7fe6400b2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:45.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.345+0000 7fe6474d8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6400b1ef0 con 0x7fe640071040 2023-10-11T09:41:45.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.345+0000 7fe6474d8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6400ce850 con 0x7fe6400cbbd0 2023-10-11T09:41:45.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.345+0000 7fe6474d8700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6400b3380 con 0x7fe640154e00 2023-10-11T09:41:45.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.349+0000 7fe645274700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe640071040 0x7fe6400ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:45.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.349+0000 7fe645274700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe640071040 0x7fe6400ce310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40064/0 (socket says 172.21.15.78:40064) 2023-10-11T09:41:45.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.349+0000 7fe645274700 1 -- 172.21.15.78:0/985949757 learned_addr learned my addr 172.21.15.78:0/985949757 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:45.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.349+0000 7fe644a73700 1 --2- 172.21.15.78:0/985949757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6400cbbd0 0x7fe64014e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:45.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.349+0000 7fe645a75700 1 --2- 172.21.15.78:0/985949757 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe640154e00 0x7fe6400b2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:45.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.349+0000 7fe645274700 1 -- 172.21.15.78:0/985949757 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe640154e00 msgr2=0x7fe6400b2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:45.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.349+0000 7fe645274700 1 --2- 172.21.15.78:0/985949757 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe640154e00 0x7fe6400b2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:45.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.349+0000 7fe645274700 1 -- 172.21.15.78:0/985949757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6400cbbd0 msgr2=0x7fe64014e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:45.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.349+0000 7fe645274700 1 --2- 172.21.15.78:0/985949757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6400cbbd0 0x7fe64014e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:45.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.349+0000 7fe645274700 1 -- 172.21.15.78:0/985949757 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe63c009570 con 0x7fe640071040 2023-10-11T09:41:45.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.349+0000 7fe644a73700 1 --2- 172.21.15.78:0/985949757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6400cbbd0 0x7fe64014e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:41:45.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.349+0000 7fe645a75700 1 --2- 172.21.15.78:0/985949757 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe640154e00 0x7fe6400b2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:41:45.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.349+0000 7fe645274700 1 --2- 172.21.15.78:0/985949757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe640071040 0x7fe6400ce310 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe63c000c00 tx=0x7fe63c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:45.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.349+0000 7fe635ffb700 1 -- 172.21.15.78:0/985949757 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63c003ad0 con 0x7fe640071040 2023-10-11T09:41:45.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.349+0000 7fe635ffb700 1 -- 172.21.15.78:0/985949757 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe63c01e070 con 0x7fe640071040 2023-10-11T09:41:45.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.349+0000 7fe6474d8700 1 -- 172.21.15.78:0/985949757 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6400b3660 con 0x7fe640071040 2023-10-11T09:41:45.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.349+0000 7fe635ffb700 1 -- 172.21.15.78:0/985949757 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63c003e90 con 0x7fe640071040 2023-10-11T09:41:45.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.349+0000 7fe6474d8700 1 -- 172.21.15.78:0/985949757 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe64014ef20 con 0x7fe640071040 2023-10-11T09:41:45.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.353+0000 7fe6474d8700 1 -- 172.21.15.78:0/985949757 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6400b3b80 con 0x7fe640071040 2023-10-11T09:41:45.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.353+0000 7fe635ffb700 1 -- 172.21.15.78:0/985949757 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fe63c011420 con 0x7fe640071040 2023-10-11T09:41:45.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.353+0000 7fe635ffb700 1 --2- 172.21.15.78:0/985949757 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe624065ed0 0x7fe624068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:45.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.353+0000 7fe635ffb700 1 -- 172.21.15.78:0/985949757 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fe63c020070 con 0x7fe640071040 2023-10-11T09:41:45.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.353+0000 7fe644a73700 1 --2- 172.21.15.78:0/985949757 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe624065ed0 0x7fe624068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:45.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.357+0000 7fe644a73700 1 --2- 172.21.15.78:0/985949757 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe624065ed0 0x7fe624068380 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe6400cb450 tx=0x7fe630009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:45.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.361+0000 7fe635ffb700 1 -- 172.21.15.78:0/985949757 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fe63c052600 con 0x7fe640071040 2023-10-11T09:41:45.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.533+0000 7fe6474d8700 1 -- 172.21.15.78:0/985949757 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6400b1600 con 0x7fe640071040 2023-10-11T09:41:45.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.537+0000 7fe635ffb700 1 -- 172.21.15.78:0/985949757 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17907 (secure 0 0 0) 0x7fe63c055c20 con 0x7fe640071040 2023-10-11T09:41:45.544 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:41:45.544 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":25,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:41:21.473566+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:40:39.862581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T09:40:40.663641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.37","mappings":[{"from":3,"to":2}]},{"pgid":"4.3e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:41:45.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.541+0000 7fe6474d8700 1 -- 172.21.15.78:0/985949757 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe624065ed0 msgr2=0x7fe624068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:45.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.541+0000 7fe6474d8700 1 --2- 172.21.15.78:0/985949757 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe624065ed0 0x7fe624068380 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe6400cb450 tx=0x7fe630009040 comp rx=0 tx=0).stop 2023-10-11T09:41:45.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.541+0000 7fe6474d8700 1 -- 172.21.15.78:0/985949757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe640071040 msgr2=0x7fe6400ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:45.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.541+0000 7fe6474d8700 1 --2- 172.21.15.78:0/985949757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe640071040 0x7fe6400ce310 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe63c000c00 tx=0x7fe63c004770 comp rx=0 tx=0).stop 2023-10-11T09:41:45.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.541+0000 7fe6474d8700 1 -- 172.21.15.78:0/985949757 shutdown_connections 2023-10-11T09:41:45.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.541+0000 7fe6474d8700 1 --2- 172.21.15.78:0/985949757 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe624065ed0 0x7fe624068380 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:45.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.541+0000 7fe6474d8700 1 --2- 172.21.15.78:0/985949757 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe640154e00 0x7fe6400b2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:45.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.541+0000 7fe6474d8700 1 --2- 172.21.15.78:0/985949757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6400cbbd0 0x7fe64014e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:45.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.541+0000 7fe6474d8700 1 --2- 172.21.15.78:0/985949757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe640071040 0x7fe6400ce310 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:45.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.541+0000 7fe6474d8700 1 -- 172.21.15.78:0/985949757 >> 172.21.15.78:0/985949757 conn(0x7fe64006aca0 msgr2=0x7fe64014f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:45.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.541+0000 7fe6474d8700 1 -- 172.21.15.78:0/985949757 shutdown_connections 2023-10-11T09:41:45.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.541+0000 7fe6474d8700 1 -- 172.21.15.78:0/985949757 wait complete. 2023-10-11T09:41:45.570 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-11T09:41:45.571 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:41:45.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.733+0000 7fb601643700 1 Processor -- start 2023-10-11T09:41:45.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.733+0000 7fb601643700 1 -- start start 2023-10-11T09:41:45.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.733+0000 7fb601643700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fb5fc0cda10 0x7fb5fc0caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:45.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.733+0000 7fb601643700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb5fc0cdde0 0x7fb5fc164610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:45.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.733+0000 7fb601643700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5fc165390 con 0x7fb5fc164b50 2023-10-11T09:41:45.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.733+0000 7fb601643700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5fc0ca440 con 0x7fb5fc0cda10 2023-10-11T09:41:45.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.733+0000 7fb601643700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5fc0ca720 con 0x7fb5fc0cdde0 2023-10-11T09:41:45.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.733+0000 7fb5faffd700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fb5fc0cda10 0x7fb5fc0caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:45.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.733+0000 7fb5faffd700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fb5fc0cda10 0x7fb5fc0caf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40066/0 (socket says 172.21.15.78:40066) 2023-10-11T09:41:45.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.733+0000 7fb5faffd700 1 -- 172.21.15.78:0/133029921 learned_addr learned my addr 172.21.15.78:0/133029921 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:45.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.733+0000 7fb5fa7fc700 1 --2- 172.21.15.78:0/133029921 >> v2:172.21.15.133:3300/0 conn(0x7fb5fc0cdde0 0x7fb5fc164610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:45.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.733+0000 7fb5faffd700 1 -- 172.21.15.78:0/133029921 >> v2:172.21.15.133:3300/0 conn(0x7fb5fc0cdde0 msgr2=0x7fb5fc164610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:45.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.733+0000 7fb5faffd700 1 --2- 172.21.15.78:0/133029921 >> v2:172.21.15.133:3300/0 conn(0x7fb5fc0cdde0 0x7fb5fc164610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:45.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.733+0000 7fb5faffd700 1 -- 172.21.15.78:0/133029921 >> v1:172.21.15.78:6789/0 conn(0x7fb5fc164b50 legacy=0x7fb5fc06bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:45.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.733+0000 7fb5faffd700 1 -- 172.21.15.78:0/133029921 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5fc0caa00 con 0x7fb5fc0cda10 2023-10-11T09:41:45.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.733+0000 7fb5faffd700 1 --2- 172.21.15.78:0/133029921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5fc0cda10 0x7fb5fc0caf20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc0cea10 tx=0x7fb5ec009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d1275d999a6678e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:45.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.737+0000 7fb5f9ffb700 1 -- 172.21.15.78:0/133029921 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec011830 con 0x7fb5fc0cda10 2023-10-11T09:41:45.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.737+0000 7fb5f9ffb700 1 -- 172.21.15.78:0/133029921 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5ec011990 con 0x7fb5fc0cda10 2023-10-11T09:41:45.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.737+0000 7fb5f9ffb700 1 -- 172.21.15.78:0/133029921 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec011d20 con 0x7fb5fc0cda10 2023-10-11T09:41:45.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.737+0000 7fb601643700 1 -- 172.21.15.78:0/133029921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5fc0cda10 msgr2=0x7fb5fc0caf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:45.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.737+0000 7fb601643700 1 --2- 172.21.15.78:0/133029921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5fc0cda10 0x7fb5fc0caf20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc0cea10 tx=0x7fb5ec009bf0 comp rx=0 tx=0).stop 2023-10-11T09:41:45.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.737+0000 7fb601643700 1 -- 172.21.15.78:0/133029921 shutdown_connections 2023-10-11T09:41:45.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.737+0000 7fb601643700 1 --2- 172.21.15.78:0/133029921 >> v2:172.21.15.133:3300/0 conn(0x7fb5fc0cdde0 0x7fb5fc164610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:45.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.737+0000 7fb601643700 1 --2- 172.21.15.78:0/133029921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5fc0cda10 0x7fb5fc0caf20 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:45.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.737+0000 7fb601643700 1 -- 172.21.15.78:0/133029921 >> 172.21.15.78:0/133029921 conn(0x7fb5fc0cea60 msgr2=0x7fb5fc0cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:45.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.737+0000 7fb601643700 1 -- 172.21.15.78:0/133029921 shutdown_connections 2023-10-11T09:41:45.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.737+0000 7fb601643700 1 -- 172.21.15.78:0/133029921 wait complete. 2023-10-11T09:41:45.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.741+0000 7fb601643700 1 Processor -- start 2023-10-11T09:41:45.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.741+0000 7fb601643700 1 -- start start 2023-10-11T09:41:45.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.741+0000 7fb601643700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5fc0cda10 0x7fb5fc1f6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:45.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.741+0000 7fb601643700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb5fc0cdde0 0x7fb5fc0b3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:45.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.741+0000 7fb601643700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5fc164b50 0x7fb5fc1f9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:45.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.741+0000 7fb601643700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc1f71f0 con 0x7fb5fc164b50 2023-10-11T09:41:45.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.741+0000 7fb601643700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc1f6950 con 0x7fb5fc0cda10 2023-10-11T09:41:45.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.741+0000 7fb601643700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc1f6c30 con 0x7fb5fc0cdde0 2023-10-11T09:41:45.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.741+0000 7fb5fb7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5fc164b50 0x7fb5fc1f9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:45.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.741+0000 7fb5fb7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5fc164b50 0x7fb5fc1f9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40070/0 (socket says 172.21.15.78:40070) 2023-10-11T09:41:45.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.741+0000 7fb5fb7fe700 1 -- 172.21.15.78:0/1637953641 learned_addr learned my addr 172.21.15.78:0/1637953641 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:45.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.741+0000 7fb5faffd700 1 --2- 172.21.15.78:0/1637953641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5fc0cda10 0x7fb5fc1f6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:45.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.741+0000 7fb5fa7fc700 1 --2- 172.21.15.78:0/1637953641 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb5fc0cdde0 0x7fb5fc0b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:45.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.745+0000 7fb5fb7fe700 1 -- 172.21.15.78:0/1637953641 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb5fc0cdde0 msgr2=0x7fb5fc0b3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:45.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.745+0000 7fb5fb7fe700 1 --2- 172.21.15.78:0/1637953641 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb5fc0cdde0 0x7fb5fc0b3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:45.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.745+0000 7fb5fb7fe700 1 -- 172.21.15.78:0/1637953641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5fc0cda10 msgr2=0x7fb5fc1f6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:45.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.745+0000 7fb5fb7fe700 1 --2- 172.21.15.78:0/1637953641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5fc0cda10 0x7fb5fc1f6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:45.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.745+0000 7fb5fb7fe700 1 -- 172.21.15.78:0/1637953641 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ec009570 con 0x7fb5fc164b50 2023-10-11T09:41:45.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.745+0000 7fb5fb7fe700 1 --2- 172.21.15.78:0/1637953641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5fc164b50 0x7fb5fc1f9160 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb5f400cb50 tx=0x7fb5f400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:45.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.745+0000 7fb5e37fe700 1 -- 172.21.15.78:0/1637953641 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4003c20 con 0x7fb5fc164b50 2023-10-11T09:41:45.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.745+0000 7fb5e37fe700 1 -- 172.21.15.78:0/1637953641 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f4003d80 con 0x7fb5fc164b50 2023-10-11T09:41:45.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.745+0000 7fb601643700 1 -- 172.21.15.78:0/1637953641 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5fc1f6f10 con 0x7fb5fc164b50 2023-10-11T09:41:45.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.745+0000 7fb5e37fe700 1 -- 172.21.15.78:0/1637953641 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f400aa20 con 0x7fb5fc164b50 2023-10-11T09:41:45.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.745+0000 7fb601643700 1 -- 172.21.15.78:0/1637953641 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5fc1f7410 con 0x7fb5fc164b50 2023-10-11T09:41:45.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.745+0000 7fb601643700 1 -- 172.21.15.78:0/1637953641 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5fc0b1d80 con 0x7fb5fc164b50 2023-10-11T09:41:45.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.749+0000 7fb5e37fe700 1 -- 172.21.15.78:0/1637953641 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fb5f400ab80 con 0x7fb5fc164b50 2023-10-11T09:41:45.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.749+0000 7fb5e37fe700 1 --2- 172.21.15.78:0/1637953641 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb5e8065ed0 0x7fb5e8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:45.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.749+0000 7fb5e37fe700 1 -- 172.21.15.78:0/1637953641 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb5f4083780 con 0x7fb5fc164b50 2023-10-11T09:41:45.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.749+0000 7fb5faffd700 1 --2- 172.21.15.78:0/1637953641 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb5e8065ed0 0x7fb5e8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:45.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.753+0000 7fb5faffd700 1 --2- 172.21.15.78:0/1637953641 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb5e8065ed0 0x7fb5e8068380 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec000c00 tx=0x7fb5ec008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:45.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.757+0000 7fb5e37fe700 1 -- 172.21.15.78:0/1637953641 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fb5fc0b1d80 con 0x7fb5fc164b50 2023-10-11T09:41:45.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.965+0000 7fb601643700 1 -- 172.21.15.78:0/1637953641 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb5fc0b1d80 con 0x7fb5fc164b50 2023-10-11T09:41:45.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.965+0000 7fb5e37fe700 1 -- 172.21.15.78:0/1637953641 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4648 (secure 0 0 0) 0x7fb5fc0b1d80 con 0x7fb5fc164b50 2023-10-11T09:41:45.971 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:41:45.971 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/4282378510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4282378510},{"type":"v1","addr":"172.21.15.133:6835","nonce":4282378510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3986631249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3986631249},{"type":"v1","addr":"172.21.15.78:6835","nonce":3986631249}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3135421902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3135421902},{"type":"v1","addr":"172.21.15.78:6837","nonce":3135421902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:40:41.673556+0000","modified":"2023-10-11T09:40:45.122852+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6834/3925157967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":3925157967},{"type":"v1","addr":"172.21.15.133:6834","nonce":3925157967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-11T09:41:45.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.969+0000 7fb601643700 1 -- 172.21.15.78:0/1637953641 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb5e8065ed0 msgr2=0x7fb5e8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:45.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.969+0000 7fb601643700 1 --2- 172.21.15.78:0/1637953641 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb5e8065ed0 0x7fb5e8068380 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec000c00 tx=0x7fb5ec008030 comp rx=0 tx=0).stop 2023-10-11T09:41:45.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.969+0000 7fb601643700 1 -- 172.21.15.78:0/1637953641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5fc164b50 msgr2=0x7fb5fc1f9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:45.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.969+0000 7fb601643700 1 --2- 172.21.15.78:0/1637953641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5fc164b50 0x7fb5fc1f9160 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb5f400cb50 tx=0x7fb5f400ce60 comp rx=0 tx=0).stop 2023-10-11T09:41:45.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.969+0000 7fb601643700 1 -- 172.21.15.78:0/1637953641 shutdown_connections 2023-10-11T09:41:45.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.969+0000 7fb601643700 1 --2- 172.21.15.78:0/1637953641 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb5e8065ed0 0x7fb5e8068380 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:45.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.969+0000 7fb601643700 1 --2- 172.21.15.78:0/1637953641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5fc164b50 0x7fb5fc1f9160 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:45.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.969+0000 7fb601643700 1 --2- 172.21.15.78:0/1637953641 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb5fc0cdde0 0x7fb5fc0b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:45.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.969+0000 7fb601643700 1 --2- 172.21.15.78:0/1637953641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5fc0cda10 0x7fb5fc1f6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:45.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.969+0000 7fb601643700 1 -- 172.21.15.78:0/1637953641 >> 172.21.15.78:0/1637953641 conn(0x7fb5fc0cea60 msgr2=0x7fb5fc06fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:45.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.969+0000 7fb601643700 1 -- 172.21.15.78:0/1637953641 shutdown_connections 2023-10-11T09:41:45.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:45.969+0000 7fb601643700 1 -- 172.21.15.78:0/1637953641 wait complete. 2023-10-11T09:41:45.979 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 8 2023-10-11T09:41:45.998 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:41:46.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.165+0000 7f8f2ea39700 1 Processor -- start 2023-10-11T09:41:46.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.165+0000 7f8f2ea39700 1 -- start start 2023-10-11T09:41:46.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.165+0000 7f8f2ea39700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8f280cda10 0x7f8f280caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:46.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.165+0000 7f8f2ea39700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8f280cdde0 0x7f8f28164610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:46.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.165+0000 7f8f2ea39700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f28165390 con 0x7f8f28164b50 2023-10-11T09:41:46.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.165+0000 7f8f2ea39700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f280ca440 con 0x7f8f280cda10 2023-10-11T09:41:46.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.165+0000 7f8f2ea39700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f280ca720 con 0x7f8f280cdde0 2023-10-11T09:41:46.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.165+0000 7f8f2e238700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f8f28164b50 0x7f8f2806bbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:35686/0 (socket says 172.21.15.78:35686) 2023-10-11T09:41:46.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.165+0000 7f8f2e238700 1 -- 172.21.15.78:0/207824126 learned_addr learned my addr 172.21.15.78:0/207824126 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:46.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.165+0000 7f8f2d236700 1 --2- 172.21.15.78:0/207824126 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8f280cdde0 0x7f8f28164610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:46.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.169+0000 7f8f2da37700 1 --2- 172.21.15.78:0/207824126 >> v2:172.21.15.133:3300/0 conn(0x7f8f280cda10 0x7f8f280caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:46.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.169+0000 7f8f2d236700 1 -- 172.21.15.78:0/207824126 >> v2:172.21.15.133:3300/0 conn(0x7f8f280cda10 msgr2=0x7f8f280caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:46.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.169+0000 7f8f2d236700 1 --2- 172.21.15.78:0/207824126 >> v2:172.21.15.133:3300/0 conn(0x7f8f280cda10 0x7f8f280caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:46.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.169+0000 7f8f2d236700 1 -- 172.21.15.78:0/207824126 >> v1:172.21.15.78:6789/0 conn(0x7f8f28164b50 legacy=0x7f8f2806bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:46.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.169+0000 7f8f2d236700 1 -- 172.21.15.78:0/207824126 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f280caa00 con 0x7f8f280cdde0 2023-10-11T09:41:46.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.169+0000 7f8f2ca35700 1 -- 172.21.15.78:0/207824126 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1771786068 0 0) 0x7f8f28165390 con 0x7f8f28164b50 2023-10-11T09:41:46.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.169+0000 7f8f2d236700 1 --2- 172.21.15.78:0/207824126 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8f280cdde0 0x7f8f28164610 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8f18009850 tx=0x7f8f18009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b77a6bc2ae8ab8c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:46.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.169+0000 7f8f2ca35700 1 -- 172.21.15.78:0/207824126 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f18011830 con 0x7f8f280cdde0 2023-10-11T09:41:46.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.169+0000 7f8f2ca35700 1 -- 172.21.15.78:0/207824126 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f18011990 con 0x7f8f280cdde0 2023-10-11T09:41:46.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.169+0000 7f8f2ca35700 1 -- 172.21.15.78:0/207824126 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f18011d40 con 0x7f8f280cdde0 2023-10-11T09:41:46.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.169+0000 7f8f2ea39700 1 -- 172.21.15.78:0/207824126 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8f280cdde0 msgr2=0x7f8f28164610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:46.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.169+0000 7f8f2ea39700 1 --2- 172.21.15.78:0/207824126 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8f280cdde0 0x7f8f28164610 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8f18009850 tx=0x7f8f18009b60 comp rx=0 tx=0).stop 2023-10-11T09:41:46.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.169+0000 7f8f2ea39700 1 -- 172.21.15.78:0/207824126 shutdown_connections 2023-10-11T09:41:46.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.169+0000 7f8f2ea39700 1 --2- 172.21.15.78:0/207824126 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8f280cdde0 0x7f8f28164610 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:46.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.169+0000 7f8f2ea39700 1 --2- 172.21.15.78:0/207824126 >> v2:172.21.15.133:3300/0 conn(0x7f8f280cda10 0x7f8f280caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:46.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.169+0000 7f8f2ea39700 1 -- 172.21.15.78:0/207824126 >> 172.21.15.78:0/207824126 conn(0x7f8f280cea60 msgr2=0x7f8f280cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:46.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.169+0000 7f8f2ea39700 1 -- 172.21.15.78:0/207824126 shutdown_connections 2023-10-11T09:41:46.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.169+0000 7f8f2ea39700 1 -- 172.21.15.78:0/207824126 wait complete. 2023-10-11T09:41:46.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.173+0000 7f8f2ea39700 1 Processor -- start 2023-10-11T09:41:46.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.173+0000 7f8f2ea39700 1 -- start start 2023-10-11T09:41:46.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.173+0000 7f8f2ea39700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8f280cda10 0x7f8f280b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:46.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.173+0000 7f8f2ea39700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f280cdde0 0x7f8f28144c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:46.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.173+0000 7f8f2ea39700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f28164b50 0x7f8f2815ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:46.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.173+0000 7f8f2ea39700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f28146cb0 con 0x7f8f280cdde0 2023-10-11T09:41:46.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.173+0000 7f8f2ea39700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f28147270 con 0x7f8f28164b50 2023-10-11T09:41:46.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.173+0000 7f8f2ea39700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f281451b0 con 0x7f8f280cda10 2023-10-11T09:41:46.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.173+0000 7f8f2d236700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f280cdde0 0x7f8f28144c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:46.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.173+0000 7f8f2d236700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f280cdde0 0x7f8f28144c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40084/0 (socket says 172.21.15.78:40084) 2023-10-11T09:41:46.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.173+0000 7f8f2d236700 1 -- 172.21.15.78:0/231190947 learned_addr learned my addr 172.21.15.78:0/231190947 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:46.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.173+0000 7f8f2da37700 1 --2- 172.21.15.78:0/231190947 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8f280cda10 0x7f8f280b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:46.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.177+0000 7f8f2e238700 1 --2- 172.21.15.78:0/231190947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f28164b50 0x7f8f2815ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:46.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.177+0000 7f8f2d236700 1 -- 172.21.15.78:0/231190947 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8f280cda10 msgr2=0x7f8f280b1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:46.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.177+0000 7f8f2d236700 1 --2- 172.21.15.78:0/231190947 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8f280cda10 0x7f8f280b1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:46.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.177+0000 7f8f2d236700 1 -- 172.21.15.78:0/231190947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f28164b50 msgr2=0x7f8f2815ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:46.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.177+0000 7f8f2d236700 1 --2- 172.21.15.78:0/231190947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f28164b50 0x7f8f2815ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:46.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.177+0000 7f8f2d236700 1 -- 172.21.15.78:0/231190947 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f180094b0 con 0x7f8f280cdde0 2023-10-11T09:41:46.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.177+0000 7f8f2e238700 1 --2- 172.21.15.78:0/231190947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f28164b50 0x7f8f2815ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:41:46.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.177+0000 7f8f2d236700 1 --2- 172.21.15.78:0/231190947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f280cdde0 0x7f8f28144c70 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8f180090b0 tx=0x7f8f18004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:46.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.177+0000 7f8f1e7fc700 1 -- 172.21.15.78:0/231190947 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1801aac0 con 0x7f8f280cdde0 2023-10-11T09:41:46.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.177+0000 7f8f1e7fc700 1 -- 172.21.15.78:0/231190947 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f1801f070 con 0x7f8f280cdde0 2023-10-11T09:41:46.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.177+0000 7f8f1e7fc700 1 -- 172.21.15.78:0/231190947 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1801ae60 con 0x7f8f280cdde0 2023-10-11T09:41:46.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.177+0000 7f8f2ea39700 1 -- 172.21.15.78:0/231190947 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f280ce3b0 con 0x7f8f280cdde0 2023-10-11T09:41:46.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.177+0000 7f8f2ea39700 1 -- 172.21.15.78:0/231190947 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f28162f20 con 0x7f8f280cdde0 2023-10-11T09:41:46.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.177+0000 7f8f2ea39700 1 -- 172.21.15.78:0/231190947 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f28147420 con 0x7f8f280cdde0 2023-10-11T09:41:46.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.177+0000 7f8f1e7fc700 1 -- 172.21.15.78:0/231190947 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f8f1801b430 con 0x7f8f280cdde0 2023-10-11T09:41:46.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.181+0000 7f8f1e7fc700 1 --2- 172.21.15.78:0/231190947 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8f0c065ed0 0x7f8f0c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:46.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.181+0000 7f8f1e7fc700 1 -- 172.21.15.78:0/231190947 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f8f18084a80 con 0x7f8f280cdde0 2023-10-11T09:41:46.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.181+0000 7f8f2da37700 1 --2- 172.21.15.78:0/231190947 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8f0c065ed0 0x7f8f0c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:46.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.181+0000 7f8f2da37700 1 --2- 172.21.15.78:0/231190947 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8f0c065ed0 0x7f8f0c068380 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8f24009ae0 tx=0x7f8f24008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:46.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.185+0000 7f8f1e7fc700 1 -- 172.21.15.78:0/231190947 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f8f18052100 con 0x7f8f280cdde0 2023-10-11T09:41:46.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.397+0000 7f8f2ea39700 1 -- 172.21.15.78:0/231190947 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f281473e0 con 0x7f8f280cdde0 2023-10-11T09:41:46.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.401+0000 7f8f1e7fc700 1 -- 172.21.15.78:0/231190947 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4648 (secure 0 0 0) 0x7f8f18015090 con 0x7f8f280cdde0 2023-10-11T09:41:46.405 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:41:46.405 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/4282378510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4282378510},{"type":"v1","addr":"172.21.15.133:6835","nonce":4282378510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3986631249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3986631249},{"type":"v1","addr":"172.21.15.78:6835","nonce":3986631249}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3135421902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3135421902},{"type":"v1","addr":"172.21.15.78:6837","nonce":3135421902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:40:41.673556+0000","modified":"2023-10-11T09:40:45.122852+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6834/3925157967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":3925157967},{"type":"v1","addr":"172.21.15.133:6834","nonce":3925157967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-11T09:41:46.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.405+0000 7f8f2ea39700 1 -- 172.21.15.78:0/231190947 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8f0c065ed0 msgr2=0x7f8f0c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:46.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.405+0000 7f8f2ea39700 1 --2- 172.21.15.78:0/231190947 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8f0c065ed0 0x7f8f0c068380 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8f24009ae0 tx=0x7f8f24008040 comp rx=0 tx=0).stop 2023-10-11T09:41:46.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.405+0000 7f8f2ea39700 1 -- 172.21.15.78:0/231190947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f280cdde0 msgr2=0x7f8f28144c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:46.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.405+0000 7f8f2ea39700 1 --2- 172.21.15.78:0/231190947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f280cdde0 0x7f8f28144c70 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8f180090b0 tx=0x7f8f18004770 comp rx=0 tx=0).stop 2023-10-11T09:41:46.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.405+0000 7f8f2ea39700 1 -- 172.21.15.78:0/231190947 shutdown_connections 2023-10-11T09:41:46.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.405+0000 7f8f2ea39700 1 --2- 172.21.15.78:0/231190947 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8f0c065ed0 0x7f8f0c068380 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:46.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.405+0000 7f8f2ea39700 1 --2- 172.21.15.78:0/231190947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f28164b50 0x7f8f2815ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:46.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.405+0000 7f8f2ea39700 1 --2- 172.21.15.78:0/231190947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f280cdde0 0x7f8f28144c70 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:46.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.405+0000 7f8f2ea39700 1 --2- 172.21.15.78:0/231190947 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8f280cda10 0x7f8f280b1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:46.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.405+0000 7f8f2ea39700 1 -- 172.21.15.78:0/231190947 >> 172.21.15.78:0/231190947 conn(0x7f8f280cea60 msgr2=0x7f8f28070680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:46.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.405+0000 7f8f2ea39700 1 -- 172.21.15.78:0/231190947 shutdown_connections 2023-10-11T09:41:46.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.405+0000 7f8f2ea39700 1 -- 172.21.15.78:0/231190947 wait complete. 2023-10-11T09:41:46.413 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 8 2023-10-11T09:41:46.431 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:41:46.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.601+0000 7f0e19634700 1 Processor -- start 2023-10-11T09:41:46.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.601+0000 7f0e19634700 1 -- start start 2023-10-11T09:41:46.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.605+0000 7f0e19634700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0e14154e00 0x7f0e1415b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:46.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.605+0000 7f0e19634700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0e14071040 0x7f0e140cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:46.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.605+0000 7f0e19634700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e140cba00 0x7f0e1415e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:46.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.605+0000 7f0e19634700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e140720f0 con 0x7f0e14071040 2023-10-11T09:41:46.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.605+0000 7f0e19634700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e1415bb90 con 0x7f0e14154e00 2023-10-11T09:41:46.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.605+0000 7f0e19634700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e140714e0 con 0x7f0e140cba00 2023-10-11T09:41:46.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.605+0000 7f0e127fc700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0e14071040 0x7f0e140cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:46.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.605+0000 7f0e127fc700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0e14071040 0x7f0e140cb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40090/0 (socket says 172.21.15.78:40090) 2023-10-11T09:41:46.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.605+0000 7f0e127fc700 1 -- 172.21.15.78:0/1551163418 learned_addr learned my addr 172.21.15.78:0/1551163418 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:46.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.605+0000 7f0e12ffd700 1 --2- 172.21.15.78:0/1551163418 >> v2:172.21.15.133:3300/0 conn(0x7f0e14154e00 0x7f0e1415b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:46.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.605+0000 7f0e137fe700 1 --2- 172.21.15.78:0/1551163418 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e140cba00 0x7f0e1415e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:46.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.605+0000 7f0e12ffd700 1 -- 172.21.15.78:0/1551163418 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e140cba00 msgr2=0x7f0e1415e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:46.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.605+0000 7f0e12ffd700 1 --2- 172.21.15.78:0/1551163418 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e140cba00 0x7f0e1415e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:46.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.605+0000 7f0e12ffd700 1 -- 172.21.15.78:0/1551163418 >> v2:172.21.15.78:3300/0 conn(0x7f0e14071040 msgr2=0x7f0e140cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:46.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.605+0000 7f0e12ffd700 1 --2- 172.21.15.78:0/1551163418 >> v2:172.21.15.78:3300/0 conn(0x7f0e14071040 0x7f0e140cb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:46.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.605+0000 7f0e12ffd700 1 -- 172.21.15.78:0/1551163418 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e140717c0 con 0x7f0e14154e00 2023-10-11T09:41:46.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.605+0000 7f0e127fc700 1 --2- 172.21.15.78:0/1551163418 >> v2:172.21.15.78:3300/0 conn(0x7f0e14071040 0x7f0e140cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:41:46.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.609+0000 7f0e12ffd700 1 --2- 172.21.15.78:0/1551163418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e14154e00 0x7f0e1415b650 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0e1406f7c0 tx=0x7f0e0400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a74d7b580286683d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:46.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.609+0000 7f0e11ffb700 1 -- 172.21.15.78:0/1551163418 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0400e070 con 0x7f0e14154e00 2023-10-11T09:41:46.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.609+0000 7f0e11ffb700 1 -- 172.21.15.78:0/1551163418 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e0400bb20 con 0x7f0e14154e00 2023-10-11T09:41:46.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.609+0000 7f0e11ffb700 1 -- 172.21.15.78:0/1551163418 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e04012430 con 0x7f0e14154e00 2023-10-11T09:41:46.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.609+0000 7f0e19634700 1 -- 172.21.15.78:0/1551163418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e14154e00 msgr2=0x7f0e1415b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:46.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.609+0000 7f0e19634700 1 --2- 172.21.15.78:0/1551163418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e14154e00 0x7f0e1415b650 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0e1406f7c0 tx=0x7f0e0400b330 comp rx=0 tx=0).stop 2023-10-11T09:41:46.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.609+0000 7f0e19634700 1 -- 172.21.15.78:0/1551163418 shutdown_connections 2023-10-11T09:41:46.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.609+0000 7f0e19634700 1 --2- 172.21.15.78:0/1551163418 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e140cba00 0x7f0e1415e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:46.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.609+0000 7f0e19634700 1 --2- 172.21.15.78:0/1551163418 >> v2:172.21.15.78:3300/0 conn(0x7f0e14071040 0x7f0e140cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:46.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.609+0000 7f0e19634700 1 --2- 172.21.15.78:0/1551163418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e14154e00 0x7f0e1415b650 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:46.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.609+0000 7f0e19634700 1 -- 172.21.15.78:0/1551163418 >> 172.21.15.78:0/1551163418 conn(0x7f0e1406aca0 msgr2=0x7f0e140b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:46.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.609+0000 7f0e19634700 1 -- 172.21.15.78:0/1551163418 shutdown_connections 2023-10-11T09:41:46.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.609+0000 7f0e19634700 1 -- 172.21.15.78:0/1551163418 wait complete. 2023-10-11T09:41:46.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.613+0000 7f0e19634700 1 Processor -- start 2023-10-11T09:41:46.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.613+0000 7f0e19634700 1 -- start start 2023-10-11T09:41:46.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.613+0000 7f0e19634700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e14071040 0x7f0e14143520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:46.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.613+0000 7f0e19634700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e140cba00 0x7f0e14140420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:46.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.617+0000 7f0e19634700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e14154e00 0x7f0e14159840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:46.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.617+0000 7f0e19634700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e14143b10 con 0x7f0e14071040 2023-10-11T09:41:46.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.617+0000 7f0e19634700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e140cb7f0 con 0x7f0e14154e00 2023-10-11T09:41:46.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.617+0000 7f0e19634700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e140cafb0 con 0x7f0e140cba00 2023-10-11T09:41:46.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.617+0000 7f0e12ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e14071040 0x7f0e14143520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:46.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.617+0000 7f0e12ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e14071040 0x7f0e14143520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40094/0 (socket says 172.21.15.78:40094) 2023-10-11T09:41:46.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.617+0000 7f0e12ffd700 1 -- 172.21.15.78:0/4178577450 learned_addr learned my addr 172.21.15.78:0/4178577450 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:46.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.617+0000 7f0e12ffd700 1 -- 172.21.15.78:0/4178577450 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e140cba00 msgr2=0x7f0e14140420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:46.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.617+0000 7f0e12ffd700 1 --2- 172.21.15.78:0/4178577450 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e140cba00 0x7f0e14140420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:46.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.617+0000 7f0e12ffd700 1 -- 172.21.15.78:0/4178577450 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e14154e00 msgr2=0x7f0e14159840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:46.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.617+0000 7f0e12ffd700 1 --2- 172.21.15.78:0/4178577450 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e14154e00 0x7f0e14159840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:46.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.617+0000 7f0e12ffd700 1 -- 172.21.15.78:0/4178577450 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e08009570 con 0x7f0e14071040 2023-10-11T09:41:46.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.617+0000 7f0e12ffd700 1 --2- 172.21.15.78:0/4178577450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e14071040 0x7f0e14143520 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0e0400b370 tx=0x7f0e04004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:46.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.617+0000 7f0dfb7fe700 1 -- 172.21.15.78:0/4178577450 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e04004960 con 0x7f0e14071040 2023-10-11T09:41:46.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.617+0000 7f0dfb7fe700 1 -- 172.21.15.78:0/4178577450 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e04020070 con 0x7f0e14071040 2023-10-11T09:41:46.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.617+0000 7f0e19634700 1 -- 172.21.15.78:0/4178577450 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e04009a70 con 0x7f0e14071040 2023-10-11T09:41:46.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.617+0000 7f0dfb7fe700 1 -- 172.21.15.78:0/4178577450 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e040276e0 con 0x7f0e14071040 2023-10-11T09:41:46.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.617+0000 7f0e19634700 1 -- 172.21.15.78:0/4178577450 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e140cb510 con 0x7f0e14071040 2023-10-11T09:41:46.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.621+0000 7f0e19634700 1 -- 172.21.15.78:0/4178577450 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e1404e530 con 0x7f0e14071040 2023-10-11T09:41:46.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.621+0000 7f0dfb7fe700 1 -- 172.21.15.78:0/4178577450 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f0e04012430 con 0x7f0e14071040 2023-10-11T09:41:46.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.621+0000 7f0dfb7fe700 1 --2- 172.21.15.78:0/4178577450 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0e00065ed0 0x7f0e00068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:46.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.621+0000 7f0dfb7fe700 1 -- 172.21.15.78:0/4178577450 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f0e04022070 con 0x7f0e14071040 2023-10-11T09:41:46.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.621+0000 7f0e127fc700 1 --2- 172.21.15.78:0/4178577450 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0e00065ed0 0x7f0e00068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:46.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.625+0000 7f0e127fc700 1 --2- 172.21.15.78:0/4178577450 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0e00065ed0 0x7f0e00068380 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0e14150190 tx=0x7f0e080097a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:46.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.629+0000 7f0dfb7fe700 1 -- 172.21.15.78:0/4178577450 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f0e04053290 con 0x7f0e14071040 2023-10-11T09:41:46.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.801+0000 7f0e19634700 1 -- 172.21.15.78:0/4178577450 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e140b3cb0 con 0x7f0e14071040 2023-10-11T09:41:46.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.805+0000 7f0dfb7fe700 1 -- 172.21.15.78:0/4178577450 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17907 (secure 0 0 0) 0x7f0e040568b0 con 0x7f0e14071040 2023-10-11T09:41:46.811 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:41:46.812 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":25,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:41:21.473566+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:40:39.862581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T09:40:40.663641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.37","mappings":[{"from":3,"to":2}]},{"pgid":"4.3e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:41:46.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.809+0000 7f0e19634700 1 -- 172.21.15.78:0/4178577450 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0e00065ed0 msgr2=0x7f0e00068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:46.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.809+0000 7f0e19634700 1 --2- 172.21.15.78:0/4178577450 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0e00065ed0 0x7f0e00068380 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0e14150190 tx=0x7f0e080097a0 comp rx=0 tx=0).stop 2023-10-11T09:41:46.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.809+0000 7f0e19634700 1 -- 172.21.15.78:0/4178577450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e14071040 msgr2=0x7f0e14143520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:46.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.809+0000 7f0e19634700 1 --2- 172.21.15.78:0/4178577450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e14071040 0x7f0e14143520 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0e0400b370 tx=0x7f0e04004af0 comp rx=0 tx=0).stop 2023-10-11T09:41:46.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.809+0000 7f0e19634700 1 -- 172.21.15.78:0/4178577450 shutdown_connections 2023-10-11T09:41:46.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.809+0000 7f0e19634700 1 --2- 172.21.15.78:0/4178577450 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0e00065ed0 0x7f0e00068380 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:46.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.809+0000 7f0e19634700 1 --2- 172.21.15.78:0/4178577450 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e14154e00 0x7f0e14159840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:46.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.809+0000 7f0e19634700 1 --2- 172.21.15.78:0/4178577450 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e140cba00 0x7f0e14140420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:46.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.809+0000 7f0e19634700 1 --2- 172.21.15.78:0/4178577450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e14071040 0x7f0e14143520 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:46.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.809+0000 7f0e19634700 1 -- 172.21.15.78:0/4178577450 >> 172.21.15.78:0/4178577450 conn(0x7f0e1406aca0 msgr2=0x7f0e140c8010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:46.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.809+0000 7f0e19634700 1 -- 172.21.15.78:0/4178577450 shutdown_connections 2023-10-11T09:41:46.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:46.809+0000 7f0e19634700 1 -- 172.21.15.78:0/4178577450 wait complete. 2023-10-11T09:41:46.835 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-11T09:41:47.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.005+0000 7f3958d84700 1 Processor -- start 2023-10-11T09:41:47.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.005+0000 7f3958d84700 1 -- start start 2023-10-11T09:41:47.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.009+0000 7f3958d84700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f3954156ce0 0x7f3954160b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:47.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.009+0000 7f3958d84700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f395415dc10 con 0x7f39540d1160 2023-10-11T09:41:47.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.009+0000 7f3958d84700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3954160380 con 0x7f39540d1660 2023-10-11T09:41:47.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.009+0000 7f3958d84700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f395415d930 con 0x7f3954156ce0 2023-10-11T09:41:47.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.009+0000 7f39537fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f3954156ce0 0x7f3954160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:47.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.009+0000 7f39537fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f3954156ce0 0x7f3954160b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40102/0 (socket says 172.21.15.78:40102) 2023-10-11T09:41:47.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.009+0000 7f39537fe700 1 -- 172.21.15.78:0/741397727 learned_addr learned my addr 172.21.15.78:0/741397727 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:47.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.009+0000 7f39537fe700 1 -- 172.21.15.78:0/741397727 >> v1:172.21.15.133:6789/0 conn(0x7f39540d1660 legacy=0x7f39540d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:47.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.009+0000 7f39537fe700 1 -- 172.21.15.78:0/741397727 >> v1:172.21.15.78:6789/0 conn(0x7f39540d1160 legacy=0x7f39540cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:47.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.009+0000 7f39537fe700 1 -- 172.21.15.78:0/741397727 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3954160620 con 0x7f3954156ce0 2023-10-11T09:41:47.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.009+0000 7f3952ffd700 1 -- 172.21.15.78:0/741397727 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2140802392 0 0) 0x7f395415dc10 con 0x7f39540d1160 2023-10-11T09:41:47.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.009+0000 7f39537fe700 1 --2- 172.21.15.78:0/741397727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3954156ce0 0x7f3954160b40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3944009910 tx=0x7f3944009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6588c8c05870cc server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:47.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.009+0000 7f3952ffd700 1 -- 172.21.15.78:0/741397727 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394400ba60 con 0x7f3954156ce0 2023-10-11T09:41:47.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.009+0000 7f3952ffd700 1 -- 172.21.15.78:0/741397727 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f394400bbc0 con 0x7f3954156ce0 2023-10-11T09:41:47.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.009+0000 7f3952ffd700 1 -- 172.21.15.78:0/741397727 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3944009e60 con 0x7f3954156ce0 2023-10-11T09:41:47.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.009+0000 7f3958d84700 1 -- 172.21.15.78:0/741397727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3954156ce0 msgr2=0x7f3954160b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:47.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.009+0000 7f3958d84700 1 --2- 172.21.15.78:0/741397727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3954156ce0 0x7f3954160b40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3944009910 tx=0x7f3944009c20 comp rx=0 tx=0).stop 2023-10-11T09:41:47.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.013+0000 7f3958d84700 1 -- 172.21.15.78:0/741397727 shutdown_connections 2023-10-11T09:41:47.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.013+0000 7f3958d84700 1 --2- 172.21.15.78:0/741397727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3954156ce0 0x7f3954160b40 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:47.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.013+0000 7f3958d84700 1 -- 172.21.15.78:0/741397727 >> 172.21.15.78:0/741397727 conn(0x7f395406afb0 msgr2=0x7f395406b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:47.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.013+0000 7f3958d84700 1 -- 172.21.15.78:0/741397727 shutdown_connections 2023-10-11T09:41:47.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.013+0000 7f3958d84700 1 -- 172.21.15.78:0/741397727 wait complete. 2023-10-11T09:41:47.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.013+0000 7f3958d84700 1 Processor -- start 2023-10-11T09:41:47.030 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.013+0000 7f3958d84700 1 -- start start 2023-10-11T09:41:47.030 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.017+0000 7f3958d84700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39540d1160 0x7f39541f27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:47.030 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.017+0000 7f3958d84700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f39540d1660 0x7f39540c69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:47.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.017+0000 7f3958d84700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3954156ce0 0x7f39540cc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:47.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.017+0000 7f3958d84700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39540c7e10 con 0x7f39540d1160 2023-10-11T09:41:47.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.017+0000 7f3958d84700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39540c6ef0 con 0x7f3954156ce0 2023-10-11T09:41:47.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.017+0000 7f3958d84700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39540c7190 con 0x7f39540d1660 2023-10-11T09:41:47.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.017+0000 7f39537fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39540d1160 0x7f39541f27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:47.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.017+0000 7f39537fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39540d1160 0x7f39541f27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40114/0 (socket says 172.21.15.78:40114) 2023-10-11T09:41:47.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.017+0000 7f39537fe700 1 -- 172.21.15.78:0/1350541660 learned_addr learned my addr 172.21.15.78:0/1350541660 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:47.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.017+0000 7f394affd700 1 --2- 172.21.15.78:0/1350541660 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f39540d1660 0x7f39540c69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:47.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.017+0000 7f3953fff700 1 --2- 172.21.15.78:0/1350541660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3954156ce0 0x7f39540cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:47.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.017+0000 7f39537fe700 1 -- 172.21.15.78:0/1350541660 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f39540d1660 msgr2=0x7f39540c69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:47.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.017+0000 7f39537fe700 1 --2- 172.21.15.78:0/1350541660 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f39540d1660 0x7f39540c69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:47.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.017+0000 7f39537fe700 1 -- 172.21.15.78:0/1350541660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3954156ce0 msgr2=0x7f39540cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:47.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.017+0000 7f39537fe700 1 --2- 172.21.15.78:0/1350541660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3954156ce0 0x7f39540cc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:47.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.017+0000 7f39537fe700 1 -- 172.21.15.78:0/1350541660 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3944009570 con 0x7f39540d1160 2023-10-11T09:41:47.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.017+0000 7f394affd700 1 --2- 172.21.15.78:0/1350541660 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f39540d1660 0x7f39540c69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:41:47.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.017+0000 7f3953fff700 1 --2- 172.21.15.78:0/1350541660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3954156ce0 0x7f39540cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:41:47.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.017+0000 7f39537fe700 1 --2- 172.21.15.78:0/1350541660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39540d1160 0x7f39541f27c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3944009c60 tx=0x7f3944004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:47.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.017+0000 7f3950ff9700 1 -- 172.21.15.78:0/1350541660 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3944011ab0 con 0x7f39540d1160 2023-10-11T09:41:47.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.017+0000 7f3950ff9700 1 -- 172.21.15.78:0/1350541660 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f394401f070 con 0x7f39540d1160 2023-10-11T09:41:47.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.017+0000 7f3958d84700 1 -- 172.21.15.78:0/1350541660 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39540c7470 con 0x7f39540d1160 2023-10-11T09:41:47.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.017+0000 7f3950ff9700 1 -- 172.21.15.78:0/1350541660 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3944011e10 con 0x7f39540d1160 2023-10-11T09:41:47.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.017+0000 7f3958d84700 1 -- 172.21.15.78:0/1350541660 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39540c7a30 con 0x7f39540d1160 2023-10-11T09:41:47.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.021+0000 7f3958d84700 1 -- 172.21.15.78:0/1350541660 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3954065d40 con 0x7f39540d1160 2023-10-11T09:41:47.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.021+0000 7f3950ff9700 1 -- 172.21.15.78:0/1350541660 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f394401a430 con 0x7f39540d1160 2023-10-11T09:41:47.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.025+0000 7f3950ff9700 1 --2- 172.21.15.78:0/1350541660 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3934065ed0 0x7f3934068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:47.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.025+0000 7f3950ff9700 1 -- 172.21.15.78:0/1350541660 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f3944021070 con 0x7f39540d1160 2023-10-11T09:41:47.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.025+0000 7f394affd700 1 --2- 172.21.15.78:0/1350541660 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3934065ed0 0x7f3934068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:47.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.025+0000 7f394affd700 1 --2- 172.21.15.78:0/1350541660 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3934065ed0 0x7f3934068380 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3954153ed0 tx=0x7f393c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:47.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.029+0000 7f3950ff9700 1 -- 172.21.15.78:0/1350541660 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f39440528d0 con 0x7f39540d1160 2023-10-11T09:41:47.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.241+0000 7f3958d84700 1 -- 172.21.15.78:0/1350541660 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f395415cf20 con 0x7f39540d1160 2023-10-11T09:41:47.251 INFO:tasks.ceph.mon.a.smithi078.stderr:2023-10-11T09:41:47.245+0000 7f90131f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-11T09:41:47.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.261+0000 7f3950ff9700 1 -- 172.21.15.78:0/1350541660 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3944015090 con 0x7f39540d1160 2023-10-11T09:41:47.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.265+0000 7f3958d84700 1 -- 172.21.15.78:0/1350541660 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3934065ed0 msgr2=0x7f3934068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:47.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.265+0000 7f3958d84700 1 --2- 172.21.15.78:0/1350541660 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3934065ed0 0x7f3934068380 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3954153ed0 tx=0x7f393c009040 comp rx=0 tx=0).stop 2023-10-11T09:41:47.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.265+0000 7f3958d84700 1 -- 172.21.15.78:0/1350541660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39540d1160 msgr2=0x7f39541f27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:47.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.265+0000 7f3958d84700 1 --2- 172.21.15.78:0/1350541660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39540d1160 0x7f39541f27c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3944009c60 tx=0x7f3944004770 comp rx=0 tx=0).stop 2023-10-11T09:41:47.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.265+0000 7f3958d84700 1 -- 172.21.15.78:0/1350541660 shutdown_connections 2023-10-11T09:41:47.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.265+0000 7f3958d84700 1 --2- 172.21.15.78:0/1350541660 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3934065ed0 0x7f3934068380 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:47.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.265+0000 7f3958d84700 1 --2- 172.21.15.78:0/1350541660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3954156ce0 0x7f39540cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:47.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.265+0000 7f3958d84700 1 --2- 172.21.15.78:0/1350541660 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f39540d1660 0x7f39540c69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:47.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.265+0000 7f3958d84700 1 --2- 172.21.15.78:0/1350541660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39540d1160 0x7f39541f27c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:47.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.265+0000 7f3958d84700 1 -- 172.21.15.78:0/1350541660 >> 172.21.15.78:0/1350541660 conn(0x7f395406afb0 msgr2=0x7f39540d0860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:47.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.265+0000 7f3958d84700 1 -- 172.21.15.78:0/1350541660 shutdown_connections 2023-10-11T09:41:47.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.265+0000 7f3958d84700 1 -- 172.21.15.78:0/1350541660 wait complete. 2023-10-11T09:41:47.274 INFO:teuthology.orchestra.run.smithi078.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-11T09:41:47.276 INFO:tasks.ceph.mds.d.smithi133.stderr: -3465> 2023-10-11T09:40:47.340+0000 7f550bbd1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-11T09:41:47.293 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-11T09:41:47.335 INFO:tasks.ceph.mds.d.smithi133.stdout:starting mds.d at 2023-10-11T09:41:47.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.461+0000 7fb6af59e700 1 Processor -- start 2023-10-11T09:41:47.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.461+0000 7fb6af59e700 1 -- start start 2023-10-11T09:41:47.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.465+0000 7fb6af59e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fb6b0157130 0x7fb6b015a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:47.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.465+0000 7fb6af59e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6b00ce240 0x7fb6b00705b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:47.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.465+0000 7fb6af59e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6b0070af0 con 0x7fb6b006ccf0 2023-10-11T09:41:47.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.465+0000 7fb6af59e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6b00b1aa0 con 0x7fb6b0157130 2023-10-11T09:41:47.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.465+0000 7fb6af59e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b00ce690 con 0x7fb6b00ce240 2023-10-11T09:41:47.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.465+0000 7fb6ae59c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fb6b0157130 0x7fb6b015a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:47.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.465+0000 7fb6add9b700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6b00ce240 0x7fb6b00705b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:47.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.465+0000 7fb6ae59c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fb6b0157130 0x7fb6b015a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40118/0 (socket says 172.21.15.78:40118) 2023-10-11T09:41:47.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.465+0000 7fb6ae59c700 1 -- 172.21.15.78:0/3262475605 learned_addr learned my addr 172.21.15.78:0/3262475605 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:47.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.465+0000 7fb6ae59c700 1 -- 172.21.15.78:0/3262475605 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6b00ce240 msgr2=0x7fb6b00705b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:47.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.465+0000 7fb6ae59c700 1 --2- 172.21.15.78:0/3262475605 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6b00ce240 0x7fb6b00705b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:47.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.465+0000 7fb6ae59c700 1 -- 172.21.15.78:0/3262475605 >> v1:172.21.15.133:6789/0 conn(0x7fb6b006ccf0 legacy=0x7fb6b00b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:47.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.465+0000 7fb6ae59c700 1 -- 172.21.15.78:0/3262475605 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b0157500 con 0x7fb6b0157130 2023-10-11T09:41:47.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.469+0000 7fb6ae59c700 1 --2- 172.21.15.78:0/3262475605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6b0157130 0x7fb6b015a680 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009910 tx=0x7fb6a0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf09968640cf7997 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:47.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.469+0000 7fb6ad59a700 1 -- 172.21.15.78:0/3262475605 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0011830 con 0x7fb6b0157130 2023-10-11T09:41:47.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.469+0000 7fb6ad59a700 1 -- 172.21.15.78:0/3262475605 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a0011990 con 0x7fb6b0157130 2023-10-11T09:41:47.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.469+0000 7fb6ad59a700 1 -- 172.21.15.78:0/3262475605 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0011d40 con 0x7fb6b0157130 2023-10-11T09:41:47.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.469+0000 7fb6af59e700 1 -- 172.21.15.78:0/3262475605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6b0157130 msgr2=0x7fb6b015a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:47.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.469+0000 7fb6af59e700 1 --2- 172.21.15.78:0/3262475605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6b0157130 0x7fb6b015a680 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009910 tx=0x7fb6a0009c20 comp rx=0 tx=0).stop 2023-10-11T09:41:47.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.469+0000 7fb6af59e700 1 -- 172.21.15.78:0/3262475605 shutdown_connections 2023-10-11T09:41:47.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.469+0000 7fb6af59e700 1 --2- 172.21.15.78:0/3262475605 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6b00ce240 0x7fb6b00705b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:47.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.469+0000 7fb6af59e700 1 --2- 172.21.15.78:0/3262475605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6b0157130 0x7fb6b015a680 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:47.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.469+0000 7fb6af59e700 1 -- 172.21.15.78:0/3262475605 >> 172.21.15.78:0/3262475605 conn(0x7fb6b006a060 msgr2=0x7fb6b006a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:47.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.469+0000 7fb6af59e700 1 -- 172.21.15.78:0/3262475605 shutdown_connections 2023-10-11T09:41:47.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.469+0000 7fb6af59e700 1 -- 172.21.15.78:0/3262475605 wait complete. 2023-10-11T09:41:47.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.473+0000 7fb6af59e700 1 Processor -- start 2023-10-11T09:41:47.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.473+0000 7fb6af59e700 1 -- start start 2023-10-11T09:41:47.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.473+0000 7fb6af59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6b006ccf0 0x7fb6b00b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:47.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.477+0000 7fb6af59e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6b00ce240 0x7fb6b014e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:47.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.477+0000 7fb6af59e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6b0157130 0x7fb6b014bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:47.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.477+0000 7fb6af59e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b0147000 con 0x7fb6b006ccf0 2023-10-11T09:41:47.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.477+0000 7fb6af59e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b00b0430 con 0x7fb6b00ce240 2023-10-11T09:41:47.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.477+0000 7fb6af59e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b014e9a0 con 0x7fb6b0157130 2023-10-11T09:41:47.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.477+0000 7fb6ae59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6b006ccf0 0x7fb6b00b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:47.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.477+0000 7fb6ae59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6b006ccf0 0x7fb6b00b08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40120/0 (socket says 172.21.15.78:40120) 2023-10-11T09:41:47.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.477+0000 7fb6ae59c700 1 -- 172.21.15.78:0/864673535 learned_addr learned my addr 172.21.15.78:0/864673535 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:47.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.477+0000 7fb6add9b700 1 --2- 172.21.15.78:0/864673535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6b00ce240 0x7fb6b014e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:47.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.477+0000 7fb6aed9d700 1 --2- 172.21.15.78:0/864673535 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6b0157130 0x7fb6b014bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:47.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.477+0000 7fb6ae59c700 1 -- 172.21.15.78:0/864673535 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6b0157130 msgr2=0x7fb6b014bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:47.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.477+0000 7fb6ae59c700 1 --2- 172.21.15.78:0/864673535 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6b0157130 0x7fb6b014bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:47.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.477+0000 7fb6ae59c700 1 -- 172.21.15.78:0/864673535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6b00ce240 msgr2=0x7fb6b014e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:47.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.477+0000 7fb6ae59c700 1 --2- 172.21.15.78:0/864673535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6b00ce240 0x7fb6b014e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:47.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.477+0000 7fb6ae59c700 1 -- 172.21.15.78:0/864673535 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a0009570 con 0x7fb6b006ccf0 2023-10-11T09:41:47.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.477+0000 7fb6add9b700 1 --2- 172.21.15.78:0/864673535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6b00ce240 0x7fb6b014e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:41:47.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.477+0000 7fb6aed9d700 1 --2- 172.21.15.78:0/864673535 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6b0157130 0x7fb6b014bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:41:47.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.477+0000 7fb6ae59c700 1 --2- 172.21.15.78:0/864673535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6b006ccf0 0x7fb6b00b08c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb6b00b4930 tx=0x7fb6a0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:47.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.477+0000 7fb69effd700 1 -- 172.21.15.78:0/864673535 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0004590 con 0x7fb6b006ccf0 2023-10-11T09:41:47.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.477+0000 7fb69effd700 1 -- 172.21.15.78:0/864673535 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a001e070 con 0x7fb6b006ccf0 2023-10-11T09:41:47.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.477+0000 7fb6af59e700 1 -- 172.21.15.78:0/864673535 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6b014c610 con 0x7fb6b006ccf0 2023-10-11T09:41:47.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.477+0000 7fb69effd700 1 -- 172.21.15.78:0/864673535 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0003900 con 0x7fb6b006ccf0 2023-10-11T09:41:47.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.477+0000 7fb6af59e700 1 -- 172.21.15.78:0/864673535 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6b00b00c0 con 0x7fb6b006ccf0 2023-10-11T09:41:47.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.481+0000 7fb6af59e700 1 -- 172.21.15.78:0/864673535 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6b0065d40 con 0x7fb6b006ccf0 2023-10-11T09:41:47.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.481+0000 7fb69effd700 1 -- 172.21.15.78:0/864673535 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fb6a0003a60 con 0x7fb6b006ccf0 2023-10-11T09:41:47.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.481+0000 7fb69effd700 1 --2- 172.21.15.78:0/864673535 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb690065ed0 0x7fb690068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:47.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.481+0000 7fb69effd700 1 -- 172.21.15.78:0/864673535 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6206+0+0 (secure 0 0 0) 0x7fb6a00846f0 con 0x7fb6b006ccf0 2023-10-11T09:41:47.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.481+0000 7fb6add9b700 1 --2- 172.21.15.78:0/864673535 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb690065ed0 0x7fb690068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:47.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.485+0000 7fb6add9b700 1 --2- 172.21.15.78:0/864673535 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb690065ed0 0x7fb690068380 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0156660 tx=0x7fb6a4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:47.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.489+0000 7fb69effd700 1 -- 172.21.15.78:0/864673535 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fb6a0051ca0 con 0x7fb6b006ccf0 2023-10-11T09:41:47.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:47.709+0000 7fb6af59e700 1 -- 172.21.15.78:0/864673535 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb6b0070160 con 0x7fb6b006ccf0 2023-10-11T09:41:48.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.265+0000 7fb69effd700 1 -- 172.21.15.78:0/864673535 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb6a00552c0 con 0x7fb6b006ccf0 2023-10-11T09:41:48.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.269+0000 7fb6af59e700 1 -- 172.21.15.78:0/864673535 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb690065ed0 msgr2=0x7fb690068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:48.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.269+0000 7fb6af59e700 1 --2- 172.21.15.78:0/864673535 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb690065ed0 0x7fb690068380 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0156660 tx=0x7fb6a4008040 comp rx=0 tx=0).stop 2023-10-11T09:41:48.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.269+0000 7fb6af59e700 1 -- 172.21.15.78:0/864673535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6b006ccf0 msgr2=0x7fb6b00b08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:48.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.269+0000 7fb6af59e700 1 --2- 172.21.15.78:0/864673535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6b006ccf0 0x7fb6b00b08c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb6b00b4930 tx=0x7fb6a0004770 comp rx=0 tx=0).stop 2023-10-11T09:41:48.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.269+0000 7fb6af59e700 1 -- 172.21.15.78:0/864673535 shutdown_connections 2023-10-11T09:41:48.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.269+0000 7fb6af59e700 1 --2- 172.21.15.78:0/864673535 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb690065ed0 0x7fb690068380 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:48.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.269+0000 7fb6af59e700 1 --2- 172.21.15.78:0/864673535 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6b0157130 0x7fb6b014bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:48.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.269+0000 7fb6af59e700 1 --2- 172.21.15.78:0/864673535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6b00ce240 0x7fb6b014e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:48.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.269+0000 7fb6af59e700 1 --2- 172.21.15.78:0/864673535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6b006ccf0 0x7fb6b00b08c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:48.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.269+0000 7fb6af59e700 1 -- 172.21.15.78:0/864673535 >> 172.21.15.78:0/864673535 conn(0x7fb6b006a060 msgr2=0x7fb6b015d640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:48.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.269+0000 7fb6af59e700 1 -- 172.21.15.78:0/864673535 shutdown_connections 2023-10-11T09:41:48.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.269+0000 7fb6af59e700 1 -- 172.21.15.78:0/864673535 wait complete. 2023-10-11T09:41:48.302 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-11T09:41:48.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.477+0000 7f5e093e6700 1 Processor -- start 2023-10-11T09:41:48.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.477+0000 7f5e093e6700 1 -- start start 2023-10-11T09:41:48.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.477+0000 7f5e093e6700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5e0415a0f0 0x7f5e040cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:48.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.477+0000 7f5e093e6700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f5e040b52e0 0x7f5e041608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:48.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.477+0000 7f5e093e6700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e040d2bd0 con 0x7f5e04160de0 2023-10-11T09:41:48.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.477+0000 7f5e093e6700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e040cf630 con 0x7f5e040b52e0 2023-10-11T09:41:48.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.477+0000 7f5e093e6700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e04160410 con 0x7f5e0415a0f0 2023-10-11T09:41:48.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.481+0000 7f5e037fe700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f5e04160de0 0x7f5e0415cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:35690/0 (socket says 172.21.15.78:35690) 2023-10-11T09:41:48.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.481+0000 7f5e037fe700 1 -- 172.21.15.78:0/2039021709 learned_addr learned my addr 172.21.15.78:0/2039021709 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:48.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.481+0000 7f5e027fc700 1 --2- 172.21.15.78:0/2039021709 >> v2:172.21.15.78:3300/0 conn(0x7f5e040b52e0 0x7f5e041608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:48.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.481+0000 7f5e02ffd700 1 --2- 172.21.15.78:0/2039021709 >> v2:172.21.15.133:3300/0 conn(0x7f5e0415a0f0 0x7f5e040cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:48.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.481+0000 7f5e01ffb700 1 -- 172.21.15.78:0/2039021709 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2710315533 0 0) 0x7f5e040d2bd0 con 0x7f5e04160de0 2023-10-11T09:41:48.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.481+0000 7f5e01ffb700 1 -- 172.21.15.78:0/2039021709 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5dec0035d0 con 0x7f5e04160de0 2023-10-11T09:41:48.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.481+0000 7f5e027fc700 1 -- 172.21.15.78:0/2039021709 >> v2:172.21.15.133:3300/0 conn(0x7f5e0415a0f0 msgr2=0x7f5e040cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:48.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.481+0000 7f5e027fc700 1 --2- 172.21.15.78:0/2039021709 >> v2:172.21.15.133:3300/0 conn(0x7f5e0415a0f0 0x7f5e040cefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:48.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.481+0000 7f5e027fc700 1 -- 172.21.15.78:0/2039021709 >> v1:172.21.15.78:6789/0 conn(0x7f5e04160de0 legacy=0x7f5e0415cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:48.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.481+0000 7f5e027fc700 1 -- 172.21.15.78:0/2039021709 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e040cdb30 con 0x7f5e040b52e0 2023-10-11T09:41:48.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.481+0000 7f5e027fc700 1 --2- 172.21.15.78:0/2039021709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e040b52e0 0x7f5e041608a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5e0406cbb0 tx=0x7f5df8009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e368259a07829d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:48.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.481+0000 7f5e01ffb700 1 -- 172.21.15.78:0/2039021709 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df8011830 con 0x7f5e040b52e0 2023-10-11T09:41:48.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.481+0000 7f5e01ffb700 1 -- 172.21.15.78:0/2039021709 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5df8011990 con 0x7f5e040b52e0 2023-10-11T09:41:48.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.481+0000 7f5e01ffb700 1 -- 172.21.15.78:0/2039021709 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df8011d20 con 0x7f5e040b52e0 2023-10-11T09:41:48.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.481+0000 7f5e093e6700 1 -- 172.21.15.78:0/2039021709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e040b52e0 msgr2=0x7f5e041608a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:48.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.481+0000 7f5e093e6700 1 --2- 172.21.15.78:0/2039021709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e040b52e0 0x7f5e041608a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5e0406cbb0 tx=0x7f5df8009b30 comp rx=0 tx=0).stop 2023-10-11T09:41:48.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.485+0000 7f5e093e6700 1 -- 172.21.15.78:0/2039021709 shutdown_connections 2023-10-11T09:41:48.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.485+0000 7f5e093e6700 1 --2- 172.21.15.78:0/2039021709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e040b52e0 0x7f5e041608a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:48.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.485+0000 7f5e093e6700 1 --2- 172.21.15.78:0/2039021709 >> v2:172.21.15.133:3300/0 conn(0x7f5e0415a0f0 0x7f5e040cefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:48.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.485+0000 7f5e093e6700 1 -- 172.21.15.78:0/2039021709 >> 172.21.15.78:0/2039021709 conn(0x7f5e0406afd0 msgr2=0x7f5e0406b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:48.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.485+0000 7f5e093e6700 1 -- 172.21.15.78:0/2039021709 shutdown_connections 2023-10-11T09:41:48.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.485+0000 7f5e093e6700 1 -- 172.21.15.78:0/2039021709 wait complete. 2023-10-11T09:41:48.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.485+0000 7f5e093e6700 1 Processor -- start 2023-10-11T09:41:48.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.485+0000 7f5e093e6700 1 -- start start 2023-10-11T09:41:48.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.489+0000 7f5e093e6700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e0415a0f0 0x7f5e04150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:48.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.489+0000 7f5e093e6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e04160de0 0x7f5e040c69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:48.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.489+0000 7f5e093e6700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e040c84a0 0x7f5e0415b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:48.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.489+0000 7f5e093e6700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e040c7400 con 0x7f5e04160de0 2023-10-11T09:41:48.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.489+0000 7f5e093e6700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e040c76e0 con 0x7f5e0415a0f0 2023-10-11T09:41:48.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.489+0000 7f5e093e6700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e040c79c0 con 0x7f5e040c84a0 2023-10-11T09:41:48.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.489+0000 7f5e027fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e04160de0 0x7f5e040c69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:48.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.489+0000 7f5e027fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e04160de0 0x7f5e040c69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40140/0 (socket says 172.21.15.78:40140) 2023-10-11T09:41:48.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.489+0000 7f5e027fc700 1 -- 172.21.15.78:0/1890648135 learned_addr learned my addr 172.21.15.78:0/1890648135 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:48.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.489+0000 7f5e02ffd700 1 --2- 172.21.15.78:0/1890648135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e0415a0f0 0x7f5e04150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:48.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.489+0000 7f5e037fe700 1 --2- 172.21.15.78:0/1890648135 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e040c84a0 0x7f5e0415b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:48.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.489+0000 7f5e027fc700 1 -- 172.21.15.78:0/1890648135 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e040c84a0 msgr2=0x7f5e0415b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:48.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.489+0000 7f5e027fc700 1 --2- 172.21.15.78:0/1890648135 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e040c84a0 0x7f5e0415b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:48.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.489+0000 7f5e027fc700 1 -- 172.21.15.78:0/1890648135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e0415a0f0 msgr2=0x7f5e04150fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:48.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.489+0000 7f5e027fc700 1 --2- 172.21.15.78:0/1890648135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e0415a0f0 0x7f5e04150fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:48.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.489+0000 7f5e027fc700 1 -- 172.21.15.78:0/1890648135 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df80094b0 con 0x7f5e04160de0 2023-10-11T09:41:48.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.489+0000 7f5e027fc700 1 --2- 172.21.15.78:0/1890648135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e04160de0 0x7f5e040c69b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5df8000c00 tx=0x7f5df8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:48.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.489+0000 7f5deb7fe700 1 -- 172.21.15.78:0/1890648135 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df8004340 con 0x7f5e04160de0 2023-10-11T09:41:48.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.489+0000 7f5deb7fe700 1 -- 172.21.15.78:0/1890648135 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5df801f070 con 0x7f5e04160de0 2023-10-11T09:41:48.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.489+0000 7f5e093e6700 1 -- 172.21.15.78:0/1890648135 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e040c7ca0 con 0x7f5e04160de0 2023-10-11T09:41:48.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.489+0000 7f5deb7fe700 1 -- 172.21.15.78:0/1890648135 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df80036c0 con 0x7f5e04160de0 2023-10-11T09:41:48.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.489+0000 7f5e093e6700 1 -- 172.21.15.78:0/1890648135 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e041514f0 con 0x7f5e04160de0 2023-10-11T09:41:48.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.493+0000 7f5e093e6700 1 -- 172.21.15.78:0/1890648135 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e040c81c0 con 0x7f5e04160de0 2023-10-11T09:41:48.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.493+0000 7f5deb7fe700 1 -- 172.21.15.78:0/1890648135 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f5df8021030 con 0x7f5e04160de0 2023-10-11T09:41:48.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.493+0000 7f5deb7fe700 1 --2- 172.21.15.78:0/1890648135 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5df0065f00 0x7f5df00683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:48.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.493+0000 7f5deb7fe700 1 -- 172.21.15.78:0/1890648135 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f5df8084b70 con 0x7f5e04160de0 2023-10-11T09:41:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.493+0000 7f5e02ffd700 1 --2- 172.21.15.78:0/1890648135 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5df0065f00 0x7f5df00683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.497+0000 7f5e02ffd700 1 --2- 172.21.15.78:0/1890648135 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5df0065f00 0x7f5df00683b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5e040c99b0 tx=0x7f5df4009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:48.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.501+0000 7f5deb7fe700 1 -- 172.21.15.78:0/1890648135 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f5e040c81c0 con 0x7f5e04160de0 2023-10-11T09:41:48.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:48.673+0000 7f5e093e6700 1 -- 172.21.15.78:0/1890648135 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5e040c81c0 con 0x7f5e04160de0 2023-10-11T09:41:49.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.665+0000 7f5deb7fe700 1 -- 172.21.15.78:0/1890648135 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v27) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5e040c81c0 con 0x7f5e04160de0 2023-10-11T09:41:49.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.669+0000 7f5e093e6700 1 -- 172.21.15.78:0/1890648135 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5df0065f00 msgr2=0x7f5df00683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:49.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.669+0000 7f5e093e6700 1 --2- 172.21.15.78:0/1890648135 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5df0065f00 0x7f5df00683b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5e040c99b0 tx=0x7f5df4009240 comp rx=0 tx=0).stop 2023-10-11T09:41:49.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.669+0000 7f5e093e6700 1 -- 172.21.15.78:0/1890648135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e04160de0 msgr2=0x7f5e040c69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:49.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.669+0000 7f5e093e6700 1 --2- 172.21.15.78:0/1890648135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e04160de0 0x7f5e040c69b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5df8000c00 tx=0x7f5df8004770 comp rx=0 tx=0).stop 2023-10-11T09:41:49.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.669+0000 7f5e093e6700 1 -- 172.21.15.78:0/1890648135 shutdown_connections 2023-10-11T09:41:49.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.669+0000 7f5e093e6700 1 --2- 172.21.15.78:0/1890648135 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5df0065f00 0x7f5df00683b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:49.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.669+0000 7f5e093e6700 1 --2- 172.21.15.78:0/1890648135 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e040c84a0 0x7f5e0415b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:49.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.669+0000 7f5e093e6700 1 --2- 172.21.15.78:0/1890648135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e04160de0 0x7f5e040c69b0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:49.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.669+0000 7f5e093e6700 1 --2- 172.21.15.78:0/1890648135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e0415a0f0 0x7f5e04150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:49.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.669+0000 7f5e093e6700 1 -- 172.21.15.78:0/1890648135 >> 172.21.15.78:0/1890648135 conn(0x7f5e0406afd0 msgr2=0x7f5e04157db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:49.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.669+0000 7f5e093e6700 1 -- 172.21.15.78:0/1890648135 shutdown_connections 2023-10-11T09:41:49.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.669+0000 7f5e093e6700 1 -- 172.21.15.78:0/1890648135 wait complete. 2023-10-11T09:41:49.679 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' removed 2023-10-11T09:41:49.699 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-11T09:41:49.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.869+0000 7f84d99f2700 1 Processor -- start 2023-10-11T09:41:49.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.869+0000 7f84d99f2700 1 -- start start 2023-10-11T09:41:49.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.873+0000 7f84d99f2700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f84d40cefc0 0x7f84d40b52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:49.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.873+0000 7f84d99f2700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84d415cbf0 0x7f84d415d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:49.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.873+0000 7f84d99f2700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84d40d2bd0 con 0x7f84d415a0f0 2023-10-11T09:41:49.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.873+0000 7f84d99f2700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84d415d5b0 con 0x7f84d40cefc0 2023-10-11T09:41:49.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.873+0000 7f84d99f2700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84d40cf430 con 0x7f84d415cbf0 2023-10-11T09:41:49.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.873+0000 7f84d27fc700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f84d40cefc0 0x7f84d40b52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:49.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.873+0000 7f84d27fc700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f84d40cefc0 0x7f84d40b52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40142/0 (socket says 172.21.15.78:40142) 2023-10-11T09:41:49.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.873+0000 7f84d27fc700 1 -- 172.21.15.78:0/966939359 learned_addr learned my addr 172.21.15.78:0/966939359 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:49.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.873+0000 7f84d37fe700 1 --2- 172.21.15.78:0/966939359 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84d415cbf0 0x7f84d415d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:49.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.873+0000 7f84d27fc700 1 -- 172.21.15.78:0/966939359 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84d415cbf0 msgr2=0x7f84d415d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:49.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.873+0000 7f84d27fc700 1 --2- 172.21.15.78:0/966939359 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84d415cbf0 0x7f84d415d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:49.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.873+0000 7f84d27fc700 1 -- 172.21.15.78:0/966939359 >> v1:172.21.15.133:6789/0 conn(0x7f84d415a0f0 legacy=0x7f84d41608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:49.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.873+0000 7f84d27fc700 1 -- 172.21.15.78:0/966939359 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d40cf630 con 0x7f84d40cefc0 2023-10-11T09:41:49.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.873+0000 7f84d27fc700 1 --2- 172.21.15.78:0/966939359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84d40cefc0 0x7f84d40b52e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f84c8009910 tx=0x7f84c8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=203a8794c44cd2c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:49.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.873+0000 7f84d1ffb700 1 -- 172.21.15.78:0/966939359 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c8011830 con 0x7f84d40cefc0 2023-10-11T09:41:49.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.873+0000 7f84d1ffb700 1 -- 172.21.15.78:0/966939359 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c8011990 con 0x7f84d40cefc0 2023-10-11T09:41:49.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.873+0000 7f84d1ffb700 1 -- 172.21.15.78:0/966939359 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c8011d40 con 0x7f84d40cefc0 2023-10-11T09:41:49.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.873+0000 7f84d99f2700 1 -- 172.21.15.78:0/966939359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84d40cefc0 msgr2=0x7f84d40b52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:49.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.873+0000 7f84d99f2700 1 --2- 172.21.15.78:0/966939359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84d40cefc0 0x7f84d40b52e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f84c8009910 tx=0x7f84c8009c20 comp rx=0 tx=0).stop 2023-10-11T09:41:49.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.877+0000 7f84d99f2700 1 -- 172.21.15.78:0/966939359 shutdown_connections 2023-10-11T09:41:49.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.877+0000 7f84d99f2700 1 --2- 172.21.15.78:0/966939359 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84d415cbf0 0x7f84d415d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:49.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.877+0000 7f84d99f2700 1 --2- 172.21.15.78:0/966939359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84d40cefc0 0x7f84d40b52e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:49.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.877+0000 7f84d99f2700 1 -- 172.21.15.78:0/966939359 >> 172.21.15.78:0/966939359 conn(0x7f84d406afd0 msgr2=0x7f84d406b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:49.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.877+0000 7f84d99f2700 1 -- 172.21.15.78:0/966939359 shutdown_connections 2023-10-11T09:41:49.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.877+0000 7f84d99f2700 1 -- 172.21.15.78:0/966939359 wait complete. 2023-10-11T09:41:49.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.877+0000 7f84d99f2700 1 Processor -- start 2023-10-11T09:41:49.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.877+0000 7f84d99f2700 1 -- start start 2023-10-11T09:41:49.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.881+0000 7f84d99f2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84d40cefc0 0x7f84d40b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:49.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.881+0000 7f84d99f2700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84d415a0f0 0x7f84d40ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:49.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.881+0000 7f84d99f2700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84d415cbf0 0x7f84d40ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:49.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.881+0000 7f84d99f2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d40b72e0 con 0x7f84d40cefc0 2023-10-11T09:41:49.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.881+0000 7f84d99f2700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d40b75c0 con 0x7f84d415cbf0 2023-10-11T09:41:49.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.881+0000 7f84d99f2700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84d40b78a0 con 0x7f84d415a0f0 2023-10-11T09:41:49.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.881+0000 7f84d2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84d40cefc0 0x7f84d40b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:49.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.881+0000 7f84d2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84d40cefc0 0x7f84d40b6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40158/0 (socket says 172.21.15.78:40158) 2023-10-11T09:41:49.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.881+0000 7f84d2ffd700 1 -- 172.21.15.78:0/2982947696 learned_addr learned my addr 172.21.15.78:0/2982947696 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:49.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.881+0000 7f84d27fc700 1 --2- 172.21.15.78:0/2982947696 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84d415a0f0 0x7f84d40ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:49.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.881+0000 7f84d37fe700 1 --2- 172.21.15.78:0/2982947696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84d415cbf0 0x7f84d40ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:49.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.881+0000 7f84d2ffd700 1 -- 172.21.15.78:0/2982947696 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84d415a0f0 msgr2=0x7f84d40ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:49.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.881+0000 7f84d2ffd700 1 --2- 172.21.15.78:0/2982947696 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84d415a0f0 0x7f84d40ccba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:49.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.881+0000 7f84d2ffd700 1 -- 172.21.15.78:0/2982947696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84d415cbf0 msgr2=0x7f84d40ca1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:49.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.881+0000 7f84d2ffd700 1 --2- 172.21.15.78:0/2982947696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84d415cbf0 0x7f84d40ca1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:49.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.881+0000 7f84d2ffd700 1 -- 172.21.15.78:0/2982947696 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c8009570 con 0x7f84d40cefc0 2023-10-11T09:41:49.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.881+0000 7f84d2ffd700 1 --2- 172.21.15.78:0/2982947696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84d40cefc0 0x7f84d40b6d00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f84c400c500 tx=0x7f84c400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:49.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.881+0000 7f84bbfff700 1 -- 172.21.15.78:0/2982947696 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c40052d0 con 0x7f84d40cefc0 2023-10-11T09:41:49.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.881+0000 7f84bbfff700 1 -- 172.21.15.78:0/2982947696 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c4005430 con 0x7f84d40cefc0 2023-10-11T09:41:49.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.881+0000 7f84bbfff700 1 -- 172.21.15.78:0/2982947696 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c40055a0 con 0x7f84d40cefc0 2023-10-11T09:41:49.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.881+0000 7f84d99f2700 1 -- 172.21.15.78:0/2982947696 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84d40b7b80 con 0x7f84d40cefc0 2023-10-11T09:41:49.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.881+0000 7f84d99f2700 1 -- 172.21.15.78:0/2982947696 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84d40cd0e0 con 0x7f84d40cefc0 2023-10-11T09:41:49.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.885+0000 7f84d99f2700 1 -- 172.21.15.78:0/2982947696 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84d40b80a0 con 0x7f84d40cefc0 2023-10-11T09:41:49.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.885+0000 7f84bbfff700 1 -- 172.21.15.78:0/2982947696 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f84c40038e0 con 0x7f84d40cefc0 2023-10-11T09:41:49.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.889+0000 7f84bbfff700 1 --2- 172.21.15.78:0/2982947696 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f84c0065ed0 0x7f84c0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:49.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.889+0000 7f84bbfff700 1 -- 172.21.15.78:0/2982947696 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 5768+0+0 (secure 0 0 0) 0x7f84c4082b90 con 0x7f84d40cefc0 2023-10-11T09:41:49.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.889+0000 7f84d27fc700 1 --2- 172.21.15.78:0/2982947696 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f84c0065ed0 0x7f84c0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:49.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.889+0000 7f84d27fc700 1 --2- 172.21.15.78:0/2982947696 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f84c0065ed0 0x7f84c0068380 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f84c8000c00 tx=0x7f84c8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:49.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:49.893+0000 7f84bbfff700 1 -- 172.21.15.78:0/2982947696 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f84d40b80a0 con 0x7f84d40cefc0 2023-10-11T09:41:50.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.089+0000 7f84d99f2700 1 -- 172.21.15.78:0/2982947696 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f84d40b80a0 con 0x7f84d40cefc0 2023-10-11T09:41:50.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.673+0000 7f84bbfff700 1 -- 172.21.15.78:0/2982947696 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v28) v1 ==== 169+0+0 (secure 0 0 0) 0x7f84c4017020 con 0x7f84d40cefc0 2023-10-11T09:41:50.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.681+0000 7f84d99f2700 1 -- 172.21.15.78:0/2982947696 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f84c0065ed0 msgr2=0x7f84c0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:50.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.681+0000 7f84d99f2700 1 --2- 172.21.15.78:0/2982947696 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f84c0065ed0 0x7f84c0068380 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f84c8000c00 tx=0x7f84c8008030 comp rx=0 tx=0).stop 2023-10-11T09:41:50.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.681+0000 7f84d99f2700 1 -- 172.21.15.78:0/2982947696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84d40cefc0 msgr2=0x7f84d40b6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:50.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.681+0000 7f84d99f2700 1 --2- 172.21.15.78:0/2982947696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84d40cefc0 0x7f84d40b6d00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f84c400c500 tx=0x7f84c400c810 comp rx=0 tx=0).stop 2023-10-11T09:41:50.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.681+0000 7f84d99f2700 1 -- 172.21.15.78:0/2982947696 shutdown_connections 2023-10-11T09:41:50.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.681+0000 7f84d99f2700 1 --2- 172.21.15.78:0/2982947696 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f84c0065ed0 0x7f84c0068380 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:50.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.681+0000 7f84d99f2700 1 --2- 172.21.15.78:0/2982947696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84d415cbf0 0x7f84d40ca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:50.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.681+0000 7f84d99f2700 1 --2- 172.21.15.78:0/2982947696 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84d415a0f0 0x7f84d40ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:50.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.681+0000 7f84d99f2700 1 --2- 172.21.15.78:0/2982947696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84d40cefc0 0x7f84d40b6d00 secure :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f84c400c500 tx=0x7f84c400c810 comp rx=0 tx=0).stop 2023-10-11T09:41:50.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.681+0000 7f84d99f2700 1 -- 172.21.15.78:0/2982947696 >> 172.21.15.78:0/2982947696 conn(0x7f84d406afd0 msgr2=0x7f84d415a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:50.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.681+0000 7f84d99f2700 1 -- 172.21.15.78:0/2982947696 shutdown_connections 2023-10-11T09:41:50.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.681+0000 7f84d99f2700 1 -- 172.21.15.78:0/2982947696 wait complete. 2023-10-11T09:41:50.689 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' removed 2023-10-11T09:41:50.709 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-11T09:41:50.709 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-11T09:41:50.709 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-11T09:41:50.710 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:41:50.710 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-11T09:41:50.710 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-11T09:41:50.711 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-11T09:41:50.711 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:41:50.711 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-11T09:41:50.711 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-11T09:41:50.712 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-11T09:41:50.712 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:41:50.712 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-11T09:41:50.712 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-11T09:41:50.713 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-11T09:41:50.713 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:41:50.727 INFO:tasks.ceph.mds.a:Stopped 2023-10-11T09:41:50.727 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-11T09:41:50.730 INFO:tasks.ceph.mds.c:Stopped 2023-10-11T09:41:50.730 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-11T09:41:50.731 INFO:tasks.ceph.mds.b:Stopped 2023-10-11T09:41:50.732 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-11T09:41:50.733 INFO:tasks.ceph.mds.d:Stopped 2023-10-11T09:41:50.733 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-11T09:41:50.778 INFO:tasks.ceph.mds.a:Started 2023-10-11T09:41:50.779 DEBUG:teuthology.parallel:result is None 2023-10-11T09:41:50.784 INFO:tasks.ceph.mds.c:Started 2023-10-11T09:41:50.785 DEBUG:teuthology.parallel:result is None 2023-10-11T09:41:50.785 INFO:tasks.ceph.mds.b:Started 2023-10-11T09:41:50.786 DEBUG:teuthology.parallel:result is None 2023-10-11T09:41:50.791 INFO:tasks.ceph.mds.d:Started 2023-10-11T09:41:50.791 DEBUG:teuthology.parallel:result is None 2023-10-11T09:41:50.792 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-11T09:41:50.892 INFO:tasks.ceph.mds.a.smithi078.stdout:starting mds.a at 2023-10-11T09:41:50.895 INFO:tasks.ceph.mds.c.smithi078.stdout:starting mds.c at 2023-10-11T09:41:50.913 INFO:tasks.ceph.mds.b.smithi133.stdout:starting mds.b at 2023-10-11T09:41:50.919 INFO:tasks.ceph.mds.d.smithi133.stdout:starting mds.d at 2023-10-11T09:41:50.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.973+0000 7efc5131e700 1 Processor -- start 2023-10-11T09:41:50.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.973+0000 7efc5131e700 1 -- start start 2023-10-11T09:41:50.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.977+0000 7efc5131e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7efc4c0d1660 0x7efc4c0d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:50.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.977+0000 7efc5131e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc4c15dc10 con 0x7efc4c156ce0 2023-10-11T09:41:50.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.977+0000 7efc5131e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc4c160380 con 0x7efc4c0d28e0 2023-10-11T09:41:50.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.977+0000 7efc5131e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc4c15d930 con 0x7efc4c0d1660 2023-10-11T09:41:50.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.977+0000 7efc4a7fc700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7efc4c0d1660 0x7efc4c0d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:50.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.977+0000 7efc4affd700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7efc4c156ce0 0x7efc4c160b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:35708/0 (socket says 172.21.15.78:35708) 2023-10-11T09:41:50.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.977+0000 7efc4affd700 1 -- 172.21.15.78:0/463232719 learned_addr learned my addr 172.21.15.78:0/463232719 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:50.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.977+0000 7efc4a7fc700 1 -- 172.21.15.78:0/463232719 >> v1:172.21.15.133:6789/0 conn(0x7efc4c0d28e0 legacy=0x7efc4c0cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:50.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.977+0000 7efc4a7fc700 1 -- 172.21.15.78:0/463232719 >> v1:172.21.15.78:6789/0 conn(0x7efc4c156ce0 legacy=0x7efc4c160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.977+0000 7efc4a7fc700 1 -- 172.21.15.78:0/463232719 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc4c160620 con 0x7efc4c0d1660 2023-10-11T09:41:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.977+0000 7efc4a7fc700 1 --2- 172.21.15.78:0/463232719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc4c0d1660 0x7efc4c0d0a50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7efc40009850 tx=0x7efc40009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7714fd162c33c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.977+0000 7efc49ffb700 1 -- 172.21.15.78:0/463232719 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc40004230 con 0x7efc4c0d1660 2023-10-11T09:41:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.977+0000 7efc49ffb700 1 -- 172.21.15.78:0/463232719 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc40004390 con 0x7efc4c0d1660 2023-10-11T09:41:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.977+0000 7efc49ffb700 1 -- 172.21.15.78:0/463232719 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc400036c0 con 0x7efc4c0d1660 2023-10-11T09:41:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.977+0000 7efc5131e700 1 -- 172.21.15.78:0/463232719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc4c0d1660 msgr2=0x7efc4c0d0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.977+0000 7efc5131e700 1 --2- 172.21.15.78:0/463232719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc4c0d1660 0x7efc4c0d0a50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7efc40009850 tx=0x7efc40009b60 comp rx=0 tx=0).stop 2023-10-11T09:41:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.977+0000 7efc5131e700 1 -- 172.21.15.78:0/463232719 shutdown_connections 2023-10-11T09:41:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.977+0000 7efc5131e700 1 --2- 172.21.15.78:0/463232719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc4c0d1660 0x7efc4c0d0a50 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.977+0000 7efc5131e700 1 -- 172.21.15.78:0/463232719 >> 172.21.15.78:0/463232719 conn(0x7efc4c06afb0 msgr2=0x7efc4c06b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:50.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.977+0000 7efc5131e700 1 -- 172.21.15.78:0/463232719 shutdown_connections 2023-10-11T09:41:50.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.977+0000 7efc5131e700 1 -- 172.21.15.78:0/463232719 wait complete. 2023-10-11T09:41:50.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.981+0000 7efc5131e700 1 Processor -- start 2023-10-11T09:41:50.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.981+0000 7efc5131e700 1 -- start start 2023-10-11T09:41:50.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.981+0000 7efc5131e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc4c0d1660 0x7efc4c1e8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:50.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.981+0000 7efc5131e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc4c0d28e0 0x7efc4c153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:50.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.981+0000 7efc5131e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efc4c156ce0 0x7efc4c0cb010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:50.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.981+0000 7efc5131e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c1e8d70 con 0x7efc4c0d28e0 2023-10-11T09:41:50.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.981+0000 7efc5131e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c154470 con 0x7efc4c0d1660 2023-10-11T09:41:50.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.981+0000 7efc5131e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c1e8f80 con 0x7efc4c156ce0 2023-10-11T09:41:50.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.981+0000 7efc4a7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc4c0d28e0 0x7efc4c153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:50.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.981+0000 7efc4a7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc4c0d28e0 0x7efc4c153f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40204/0 (socket says 172.21.15.78:40204) 2023-10-11T09:41:50.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.981+0000 7efc4a7fc700 1 -- 172.21.15.78:0/719886058 learned_addr learned my addr 172.21.15.78:0/719886058 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:50.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.985+0000 7efc4affd700 1 --2- 172.21.15.78:0/719886058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc4c0d1660 0x7efc4c1e8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:50.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.985+0000 7efc4b7fe700 1 --2- 172.21.15.78:0/719886058 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efc4c156ce0 0x7efc4c0cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:50.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.985+0000 7efc4a7fc700 1 -- 172.21.15.78:0/719886058 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efc4c156ce0 msgr2=0x7efc4c0cb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:50.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.985+0000 7efc4a7fc700 1 --2- 172.21.15.78:0/719886058 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efc4c156ce0 0x7efc4c0cb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:50.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.985+0000 7efc4a7fc700 1 -- 172.21.15.78:0/719886058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc4c0d1660 msgr2=0x7efc4c1e8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:50.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.985+0000 7efc4a7fc700 1 --2- 172.21.15.78:0/719886058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc4c0d1660 0x7efc4c1e8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:50.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.985+0000 7efc4a7fc700 1 -- 172.21.15.78:0/719886058 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc400094b0 con 0x7efc4c0d28e0 2023-10-11T09:41:50.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.985+0000 7efc4a7fc700 1 --2- 172.21.15.78:0/719886058 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc4c0d28e0 0x7efc4c153f30 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7efc40000c00 tx=0x7efc40004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:50.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.985+0000 7efc337fe700 1 -- 172.21.15.78:0/719886058 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc40006ea0 con 0x7efc4c0d28e0 2023-10-11T09:41:50.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.985+0000 7efc337fe700 1 -- 172.21.15.78:0/719886058 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4001f070 con 0x7efc4c0d28e0 2023-10-11T09:41:50.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.985+0000 7efc337fe700 1 -- 172.21.15.78:0/719886058 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc40003ea0 con 0x7efc4c0d28e0 2023-10-11T09:41:50.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.985+0000 7efc5131e700 1 -- 172.21.15.78:0/719886058 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc4c1e9260 con 0x7efc4c0d28e0 2023-10-11T09:41:50.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.985+0000 7efc5131e700 1 -- 172.21.15.78:0/719886058 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc4c1e9820 con 0x7efc4c0d28e0 2023-10-11T09:41:50.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.985+0000 7efc5131e700 1 -- 172.21.15.78:0/719886058 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc4c065d40 con 0x7efc4c0d28e0 2023-10-11T09:41:50.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.989+0000 7efc337fe700 1 -- 172.21.15.78:0/719886058 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7efc40021030 con 0x7efc4c0d28e0 2023-10-11T09:41:50.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.989+0000 7efc337fe700 1 --2- 172.21.15.78:0/719886058 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efc38065ed0 0x7efc38068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:50.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.989+0000 7efc337fe700 1 -- 172.21.15.78:0/719886058 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5296+0+0 (secure 0 0 0) 0x7efc40084b20 con 0x7efc4c0d28e0 2023-10-11T09:41:50.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.989+0000 7efc4affd700 1 --2- 172.21.15.78:0/719886058 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efc38065ed0 0x7efc38068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:50.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.993+0000 7efc4affd700 1 --2- 172.21.15.78:0/719886058 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efc38065ed0 0x7efc38068380 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efc3c000f60 tx=0x7efc3c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:51.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:50.997+0000 7efc337fe700 1 -- 172.21.15.78:0/719886058 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7efc40052500 con 0x7efc4c0d28e0 2023-10-11T09:41:51.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:51.177+0000 7efc5131e700 1 -- 172.21.15.78:0/719886058 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7efc4c065d40 con 0x7efc4c0d28e0 2023-10-11T09:41:51.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:51.953+0000 7efc337fe700 1 -- 172.21.15.78:0/719886058 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v33) v1 ==== 97+0+0 (secure 0 0 0) 0x7efc40055b20 con 0x7efc4c0d28e0 2023-10-11T09:41:51.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:51.957+0000 7efc5131e700 1 -- 172.21.15.78:0/719886058 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efc38065ed0 msgr2=0x7efc38068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:51.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:51.957+0000 7efc5131e700 1 --2- 172.21.15.78:0/719886058 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efc38065ed0 0x7efc38068380 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efc3c000f60 tx=0x7efc3c009040 comp rx=0 tx=0).stop 2023-10-11T09:41:51.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:51.957+0000 7efc5131e700 1 -- 172.21.15.78:0/719886058 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc4c0d28e0 msgr2=0x7efc4c153f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:51.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:51.957+0000 7efc5131e700 1 --2- 172.21.15.78:0/719886058 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc4c0d28e0 0x7efc4c153f30 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7efc40000c00 tx=0x7efc40004770 comp rx=0 tx=0).stop 2023-10-11T09:41:51.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:51.957+0000 7efc5131e700 1 -- 172.21.15.78:0/719886058 shutdown_connections 2023-10-11T09:41:51.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:51.957+0000 7efc5131e700 1 --2- 172.21.15.78:0/719886058 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efc38065ed0 0x7efc38068380 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:51.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:51.957+0000 7efc5131e700 1 --2- 172.21.15.78:0/719886058 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efc4c156ce0 0x7efc4c0cb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:51.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:51.957+0000 7efc5131e700 1 --2- 172.21.15.78:0/719886058 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc4c0d28e0 0x7efc4c153f30 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:51.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:51.957+0000 7efc5131e700 1 --2- 172.21.15.78:0/719886058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc4c0d1660 0x7efc4c1e8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:51.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:51.957+0000 7efc5131e700 1 -- 172.21.15.78:0/719886058 >> 172.21.15.78:0/719886058 conn(0x7efc4c06afb0 msgr2=0x7efc4c0b8280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:51.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:51.957+0000 7efc5131e700 1 -- 172.21.15.78:0/719886058 shutdown_connections 2023-10-11T09:41:51.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:51.957+0000 7efc5131e700 1 -- 172.21.15.78:0/719886058 wait complete. 2023-10-11T09:41:51.967 INFO:teuthology.orchestra.run.smithi078.stderr: removed all blocklist entries 2023-10-11T09:41:51.987 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-11T09:41:52.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.153+0000 7fc2b6b45700 1 Processor -- start 2023-10-11T09:41:52.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.153+0000 7fc2b6b45700 1 -- start start 2023-10-11T09:41:52.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.153+0000 7fc2b6b45700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc2b0156ce0 0x7fc2b0160b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:52.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.153+0000 7fc2b6b45700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fc2b00d1660 0x7fc2b00d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:52.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.157+0000 7fc2b6b45700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2b00d28e0 0x7fc2b00d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:52.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.157+0000 7fc2b6b45700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2b015dc10 con 0x7fc2b00d1660 2023-10-11T09:41:52.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.157+0000 7fc2b6b45700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2b0160380 con 0x7fc2b0156ce0 2023-10-11T09:41:52.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.157+0000 7fc2b6b45700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b015d930 con 0x7fc2b00d28e0 2023-10-11T09:41:52.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.157+0000 7fc2b5342700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fc2b00d1660 0x7fc2b00d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:52.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.157+0000 7fc2b5342700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fc2b00d1660 0x7fc2b00d0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40220/0 (socket says 172.21.15.78:40220) 2023-10-11T09:41:52.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.157+0000 7fc2b5342700 1 -- 172.21.15.78:0/2189859685 learned_addr learned my addr 172.21.15.78:0/2189859685 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:52.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.157+0000 7fc2b6344700 1 --2- 172.21.15.78:0/2189859685 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2b00d28e0 0x7fc2b00d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:52.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.157+0000 7fc2b5b43700 1 --2- 172.21.15.78:0/2189859685 >> v2:172.21.15.133:3300/0 conn(0x7fc2b0156ce0 0x7fc2b0160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:52.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.157+0000 7fc2b5342700 1 -- 172.21.15.78:0/2189859685 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2b00d28e0 msgr2=0x7fc2b00d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:52.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.157+0000 7fc2b5342700 1 --2- 172.21.15.78:0/2189859685 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2b00d28e0 0x7fc2b00d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:52.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.157+0000 7fc2b5342700 1 -- 172.21.15.78:0/2189859685 >> v2:172.21.15.133:3300/0 conn(0x7fc2b0156ce0 msgr2=0x7fc2b0160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:52.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.157+0000 7fc2b5342700 1 --2- 172.21.15.78:0/2189859685 >> v2:172.21.15.133:3300/0 conn(0x7fc2b0156ce0 0x7fc2b0160b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:52.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.157+0000 7fc2b5342700 1 -- 172.21.15.78:0/2189859685 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b0160620 con 0x7fc2b00d1660 2023-10-11T09:41:52.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.157+0000 7fc2b5342700 1 --2- 172.21.15.78:0/2189859685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b00d1660 0x7fc2b00d0a50 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0160550 tx=0x7fc2a0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd074467dc26439 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:52.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.157+0000 7fc2b4b41700 1 -- 172.21.15.78:0/2189859685 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0004230 con 0x7fc2b00d1660 2023-10-11T09:41:52.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.157+0000 7fc2b4b41700 1 -- 172.21.15.78:0/2189859685 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a0004390 con 0x7fc2b00d1660 2023-10-11T09:41:52.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.157+0000 7fc2b4b41700 1 -- 172.21.15.78:0/2189859685 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a00036c0 con 0x7fc2b00d1660 2023-10-11T09:41:52.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.161+0000 7fc2b6b45700 1 -- 172.21.15.78:0/2189859685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b00d1660 msgr2=0x7fc2b00d0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:52.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.161+0000 7fc2b6b45700 1 --2- 172.21.15.78:0/2189859685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b00d1660 0x7fc2b00d0a50 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0160550 tx=0x7fc2a0009bf0 comp rx=0 tx=0).stop 2023-10-11T09:41:52.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.161+0000 7fc2b6b45700 1 -- 172.21.15.78:0/2189859685 shutdown_connections 2023-10-11T09:41:52.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.161+0000 7fc2b6b45700 1 --2- 172.21.15.78:0/2189859685 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2b00d28e0 0x7fc2b00d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:52.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.161+0000 7fc2b6b45700 1 --2- 172.21.15.78:0/2189859685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b00d1660 0x7fc2b00d0a50 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:52.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.161+0000 7fc2b6b45700 1 --2- 172.21.15.78:0/2189859685 >> v2:172.21.15.133:3300/0 conn(0x7fc2b0156ce0 0x7fc2b0160b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:52.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.161+0000 7fc2b6b45700 1 -- 172.21.15.78:0/2189859685 >> 172.21.15.78:0/2189859685 conn(0x7fc2b006afb0 msgr2=0x7fc2b006b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:52.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.161+0000 7fc2b6b45700 1 -- 172.21.15.78:0/2189859685 shutdown_connections 2023-10-11T09:41:52.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.161+0000 7fc2b6b45700 1 -- 172.21.15.78:0/2189859685 wait complete. 2023-10-11T09:41:52.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.161+0000 7fc2b6b45700 1 Processor -- start 2023-10-11T09:41:52.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.161+0000 7fc2b6b45700 1 -- start start 2023-10-11T09:41:52.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.165+0000 7fc2b6b45700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2b00d1660 0x7fc2b0152d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:52.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.165+0000 7fc2b6b45700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b00d28e0 0x7fc2b00ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:52.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.165+0000 7fc2b6b45700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2b0156ce0 0x7fc2b00ca730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:52.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.165+0000 7fc2b6b45700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b0142aa0 con 0x7fc2b00d28e0 2023-10-11T09:41:52.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.165+0000 7fc2b6b45700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b0143620 con 0x7fc2b00d1660 2023-10-11T09:41:52.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.165+0000 7fc2b6b45700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b0142cf0 con 0x7fc2b0156ce0 2023-10-11T09:41:52.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.165+0000 7fc2b5342700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b00d28e0 0x7fc2b00ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:52.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.165+0000 7fc2b5342700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b00d28e0 0x7fc2b00ca1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40224/0 (socket says 172.21.15.78:40224) 2023-10-11T09:41:52.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.165+0000 7fc2b5342700 1 -- 172.21.15.78:0/2394633606 learned_addr learned my addr 172.21.15.78:0/2394633606 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:52.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.165+0000 7fc2b6344700 1 --2- 172.21.15.78:0/2394633606 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2b0156ce0 0x7fc2b00ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:52.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.165+0000 7fc2b5b43700 1 --2- 172.21.15.78:0/2394633606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2b00d1660 0x7fc2b0152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:52.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.165+0000 7fc2b5342700 1 -- 172.21.15.78:0/2394633606 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2b0156ce0 msgr2=0x7fc2b00ca730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:52.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.165+0000 7fc2b5342700 1 --2- 172.21.15.78:0/2394633606 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2b0156ce0 0x7fc2b00ca730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:52.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.165+0000 7fc2b5342700 1 -- 172.21.15.78:0/2394633606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2b00d1660 msgr2=0x7fc2b0152d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:52.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.165+0000 7fc2b5342700 1 --2- 172.21.15.78:0/2394633606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2b00d1660 0x7fc2b0152d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:52.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.165+0000 7fc2b5342700 1 -- 172.21.15.78:0/2394633606 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a0009570 con 0x7fc2b00d28e0 2023-10-11T09:41:52.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.165+0000 7fc2b5342700 1 --2- 172.21.15.78:0/2394633606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b00d28e0 0x7fc2b00ca1f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0004000 tx=0x7fc2a0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:52.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.165+0000 7fc2a67fc700 1 -- 172.21.15.78:0/2394633606 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0011630 con 0x7fc2b00d28e0 2023-10-11T09:41:52.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.165+0000 7fc2a67fc700 1 -- 172.21.15.78:0/2394633606 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a00117b0 con 0x7fc2b00d28e0 2023-10-11T09:41:52.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.165+0000 7fc2a67fc700 1 -- 172.21.15.78:0/2394633606 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0011b60 con 0x7fc2b00d28e0 2023-10-11T09:41:52.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.165+0000 7fc2b6b45700 1 -- 172.21.15.78:0/2394633606 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2b0142fd0 con 0x7fc2b00d28e0 2023-10-11T09:41:52.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.165+0000 7fc2b6b45700 1 -- 172.21.15.78:0/2394633606 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b01532b0 con 0x7fc2b00d28e0 2023-10-11T09:41:52.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.169+0000 7fc2b6b45700 1 -- 172.21.15.78:0/2394633606 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2b00ce8c0 con 0x7fc2b00d28e0 2023-10-11T09:41:52.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.169+0000 7fc2a67fc700 1 -- 172.21.15.78:0/2394633606 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fc2a0021070 con 0x7fc2b00d28e0 2023-10-11T09:41:52.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.169+0000 7fc2a67fc700 1 --2- 172.21.15.78:0/2394633606 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc294065ed0 0x7fc294068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:52.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.169+0000 7fc2a67fc700 1 -- 172.21.15.78:0/2394633606 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc2a0084b60 con 0x7fc2b00d28e0 2023-10-11T09:41:52.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.173+0000 7fc2b5b43700 1 --2- 172.21.15.78:0/2394633606 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc294065ed0 0x7fc294068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:52.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.173+0000 7fc2b5b43700 1 --2- 172.21.15.78:0/2394633606 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc294065ed0 0x7fc294068380 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac011340 tx=0x7fc2ac00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:52.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.177+0000 7fc2a67fc700 1 -- 172.21.15.78:0/2394633606 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fc2a00526b0 con 0x7fc2b00d28e0 2023-10-11T09:41:52.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.409+0000 7fc2b6b45700 1 -- 172.21.15.78:0/2394633606 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc2b0065d40 con 0x7fc2b00d28e0 2023-10-11T09:41:52.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.413+0000 7fc2a67fc700 1 -- 172.21.15.78:0/2394633606 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fc2a0055cd0 con 0x7fc2b00d28e0 2023-10-11T09:41:52.417 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:41:52.418 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:41:52.418 INFO:teuthology.orchestra.run.smithi078.stdout: "auth_dump": [ 2023-10-11T09:41:52.418 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:41:52.418 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.a", 2023-10-11T09:41:52.419 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZlL58cAxAALe+WtkV/aLopy1w9i6OVRg==", 2023-10-11T09:41:52.419 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:41:52.419 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:41:52.419 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2023-10-11T09:41:52.419 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2023-10-11T09:41:52.419 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:41:52.420 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:41:52.420 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:41:52.420 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:41:52.420 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.b", 2023-10-11T09:41:52.420 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZliPLmFhAA+6wegaD6yTUsttFzo9OrIg==", 2023-10-11T09:41:52.421 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:41:52.421 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:41:52.421 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2023-10-11T09:41:52.421 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2023-10-11T09:41:52.421 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:41:52.422 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:41:52.422 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:41:52.422 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:41:52.422 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.c", 2023-10-11T09:41:52.422 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZl9jJBDhAAjos7/02OqF0WldFm70/fVA==", 2023-10-11T09:41:52.423 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:41:52.423 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:41:52.423 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2023-10-11T09:41:52.423 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2023-10-11T09:41:52.423 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:41:52.424 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:41:52.424 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:41:52.424 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:41:52.424 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.d", 2023-10-11T09:41:52.424 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZlTb/4IRAAeXtcbWVB5TnSoYch+uMZWw==", 2023-10-11T09:41:52.425 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:41:52.425 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:41:52.425 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2023-10-11T09:41:52.425 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2023-10-11T09:41:52.425 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:41:52.426 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:41:52.426 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:41:52.426 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:41:52.426 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.0", 2023-10-11T09:41:52.426 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBYbSZldbrVFxAA6ogUpCcFO8LhHn4zdkudNQ==", 2023-10-11T09:41:52.427 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:41:52.427 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:41:52.427 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:41:52.427 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:41:52.427 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:41:52.428 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:41:52.428 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:41:52.428 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.1", 2023-10-11T09:41:52.428 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBbbSZl+mkPJxAA4Rrt0j5lzXapYjXhdulhNQ==", 2023-10-11T09:41:52.428 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:41:52.429 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:41:52.429 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:41:52.429 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:41:52.429 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:41:52.429 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:41:52.429 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:41:52.430 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.2", 2023-10-11T09:41:52.430 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBebSZlZ6lSNRAA3ltw4ZTkFNpudCPc2zTYzQ==", 2023-10-11T09:41:52.430 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:41:52.430 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:41:52.431 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:41:52.431 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:41:52.431 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:41:52.431 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:41:52.431 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:41:52.431 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.3", 2023-10-11T09:41:52.432 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBibSZlsE+jBxAA2Ix2JAFTUxljpHAwPpJeUA==", 2023-10-11T09:41:52.432 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:41:52.432 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:41:52.432 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:41:52.432 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:41:52.433 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:41:52.433 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:41:52.433 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:41:52.433 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.4", 2023-10-11T09:41:52.433 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBobSZlII4OGxAA0Z5wGuu4wOs/14tQRnHB1Q==", 2023-10-11T09:41:52.434 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:41:52.434 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:41:52.434 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:41:52.434 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:41:52.434 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:41:52.435 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:41:52.435 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:41:52.435 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.5", 2023-10-11T09:41:52.435 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBrbSZlG6aSKhAAZq179SfpkelcJJLaUcLDCg==", 2023-10-11T09:41:52.435 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:41:52.436 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:41:52.436 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:41:52.436 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:41:52.436 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:41:52.436 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:41:52.437 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:41:52.437 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.6", 2023-10-11T09:41:52.437 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBubSZl3W89NxAAgUrd0/yMOWlrhAOU1BpM9A==", 2023-10-11T09:41:52.437 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:41:52.437 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:41:52.438 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:41:52.438 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:41:52.438 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:41:52.438 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:41:52.438 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:41:52.439 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.7", 2023-10-11T09:41:52.439 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBybSZloIjyCRAAjmvyK9zMSEsV8bVebwzPxg==", 2023-10-11T09:41:52.439 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:41:52.439 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:41:52.439 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:41:52.440 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:41:52.440 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:41:52.440 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:41:52.440 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:41:52.440 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.0", 2023-10-11T09:41:52.440 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZlmGmCKBAAOI8EvyRwg8HZt+q/vUy5Ow==", 2023-10-11T09:41:52.441 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:41:52.441 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:41:52.441 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2023-10-11T09:41:52.441 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2023-10-11T09:41:52.441 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2023-10-11T09:41:52.442 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:41:52.442 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:41:52.442 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:41:52.442 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.1", 2023-10-11T09:41:52.442 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZlYvfHNBAAReum0efx4MmqpLMxFnTROQ==", 2023-10-11T09:41:52.443 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:41:52.443 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:41:52.443 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2023-10-11T09:41:52.443 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2023-10-11T09:41:52.443 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2023-10-11T09:41:52.443 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:41:52.444 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:41:52.444 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:41:52.444 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.2", 2023-10-11T09:41:52.444 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZlbd9lLhAA9LAjreTcxwAjhiNlKndl+Q==", 2023-10-11T09:41:52.444 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:41:52.445 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:41:52.445 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2023-10-11T09:41:52.445 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2023-10-11T09:41:52.445 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2023-10-11T09:41:52.445 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:41:52.446 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:41:52.446 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:41:52.446 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.3", 2023-10-11T09:41:52.446 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZlHUjeOhAASPKTRvsillob4gjdW0akhQ==", 2023-10-11T09:41:52.446 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:41:52.446 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:41:52.447 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2023-10-11T09:41:52.447 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2023-10-11T09:41:52.447 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2023-10-11T09:41:52.447 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:41:52.447 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:41:52.448 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:41:52.448 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.admin", 2023-10-11T09:41:52.448 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBTbSZlhqcOFRAA/lkGYL03++trOuIqNLS0kQ==", 2023-10-11T09:41:52.448 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:41:52.448 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2023-10-11T09:41:52.449 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2023-10-11T09:41:52.449 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2023-10-11T09:41:52.449 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:41:52.449 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:41:52.449 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:41:52.450 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:41:52.450 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.bootstrap-mds", 2023-10-11T09:41:52.450 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQB+bSZlBVlBFRAAVPvmhlaVDMvQBrOPlIdXgQ==", 2023-10-11T09:41:52.450 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:41:52.451 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile bootstrap-mds" 2023-10-11T09:41:52.451 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:41:52.451 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:41:52.451 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:41:52.452 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.bootstrap-mgr", 2023-10-11T09:41:52.452 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQB+bSZl/3hBFRAAkxQbJkPIBI34Pbv4zpWKJQ==", 2023-10-11T09:41:52.452 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:41:52.452 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile bootstrap-mgr" 2023-10-11T09:41:52.452 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:41:52.452 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:41:52.453 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:41:52.453 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.bootstrap-osd", 2023-10-11T09:41:52.453 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQB+bSZl4pBBFRAAYS0h6soeV2eXa7fapNzlzA==", 2023-10-11T09:41:52.453 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:41:52.453 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile bootstrap-osd" 2023-10-11T09:41:52.454 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:41:52.454 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:41:52.454 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:41:52.454 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.bootstrap-rbd", 2023-10-11T09:41:52.455 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQB+bSZlradBFRAAGZy2g1ZNW/wrXCfDcPDFAg==", 2023-10-11T09:41:52.455 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:41:52.455 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile bootstrap-rbd" 2023-10-11T09:41:52.455 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:41:52.456 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:41:52.456 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:41:52.456 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-10-11T09:41:52.456 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQB+bSZlcsBBFRAADURjgkdvz49Qgj8UQrfKZw==", 2023-10-11T09:41:52.456 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:41:52.457 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-10-11T09:41:52.457 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:41:52.457 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:41:52.457 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:41:52.457 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.bootstrap-rgw", 2023-10-11T09:41:52.458 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQB+bSZlsdhBFRAAMGqouBLI6cy2LcA/anQGTQ==", 2023-10-11T09:41:52.458 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:41:52.458 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile bootstrap-rgw" 2023-10-11T09:41:52.458 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:41:52.458 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:41:52.459 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:41:52.459 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mgr.x", 2023-10-11T09:41:52.459 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBTbSZlS/TjNxAAMkq9xRMwgsbLlT4FdobATw==", 2023-10-11T09:41:52.459 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:41:52.459 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2023-10-11T09:41:52.460 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile mgr", 2023-10-11T09:41:52.460 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:41:52.460 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:41:52.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:41:52.460 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:41:52.460 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mgr.y", 2023-10-11T09:41:52.461 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBTbSZlnv4yMRAAQh8w18k76KRjdCvSDJ4QbA==", 2023-10-11T09:41:52.461 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:41:52.461 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2023-10-11T09:41:52.461 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile mgr", 2023-10-11T09:41:52.461 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:41:52.462 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:41:52.462 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:41:52.462 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2023-10-11T09:41:52.462 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:41:52.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.417+0000 7fc2b6b45700 1 -- 172.21.15.78:0/2394633606 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc294065ed0 msgr2=0x7fc294068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:52.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.417+0000 7fc2b6b45700 1 --2- 172.21.15.78:0/2394633606 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc294065ed0 0x7fc294068380 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac011340 tx=0x7fc2ac00b040 comp rx=0 tx=0).stop 2023-10-11T09:41:52.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.417+0000 7fc2b6b45700 1 -- 172.21.15.78:0/2394633606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b00d28e0 msgr2=0x7fc2b00ca1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:52.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.417+0000 7fc2b6b45700 1 --2- 172.21.15.78:0/2394633606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b00d28e0 0x7fc2b00ca1f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0004000 tx=0x7fc2a0004770 comp rx=0 tx=0).stop 2023-10-11T09:41:52.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.417+0000 7fc2b6b45700 1 -- 172.21.15.78:0/2394633606 shutdown_connections 2023-10-11T09:41:52.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.417+0000 7fc2b6b45700 1 --2- 172.21.15.78:0/2394633606 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc294065ed0 0x7fc294068380 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:52.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.417+0000 7fc2b6b45700 1 --2- 172.21.15.78:0/2394633606 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2b0156ce0 0x7fc2b00ca730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:52.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.417+0000 7fc2b6b45700 1 --2- 172.21.15.78:0/2394633606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b00d28e0 0x7fc2b00ca1f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:52.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.417+0000 7fc2b6b45700 1 --2- 172.21.15.78:0/2394633606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2b00d1660 0x7fc2b0152d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:52.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.417+0000 7fc2b6b45700 1 -- 172.21.15.78:0/2394633606 >> 172.21.15.78:0/2394633606 conn(0x7fc2b006afb0 msgr2=0x7fc2b00d0780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:52.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.417+0000 7fc2b6b45700 1 -- 172.21.15.78:0/2394633606 shutdown_connections 2023-10-11T09:41:52.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.417+0000 7fc2b6b45700 1 -- 172.21.15.78:0/2394633606 wait complete. 2023-10-11T09:41:52.467 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-10-11T09:41:52.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.641+0000 7fe726b97700 1 Processor -- start 2023-10-11T09:41:52.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.641+0000 7fe726b97700 1 -- start start 2023-10-11T09:41:52.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.645+0000 7fe726b97700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe720159490 0x7fe7200ce500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:52.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.645+0000 7fe726b97700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe72015d030 0x7fe72015d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:52.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.645+0000 7fe726b97700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe720159c50 con 0x7fe72015d9c0 2023-10-11T09:41:52.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.645+0000 7fe726b97700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7200cea40 con 0x7fe72015d030 2023-10-11T09:41:52.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.645+0000 7fe726b97700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe72015de20 con 0x7fe720159490 2023-10-11T09:41:52.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.645+0000 7fe717fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe72015d030 0x7fe72015d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:52.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.645+0000 7fe717fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe72015d030 0x7fe72015d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40240/0 (socket says 172.21.15.78:40240) 2023-10-11T09:41:52.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.645+0000 7fe717fff700 1 -- 172.21.15.78:0/3746301773 learned_addr learned my addr 172.21.15.78:0/3746301773 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:52.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.645+0000 7fe724933700 1 --2- 172.21.15.78:0/3746301773 >> v2:172.21.15.133:3300/0 conn(0x7fe720159490 0x7fe7200ce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:52.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.645+0000 7fe717fff700 1 -- 172.21.15.78:0/3746301773 >> v2:172.21.15.133:3300/0 conn(0x7fe720159490 msgr2=0x7fe7200ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:52.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.645+0000 7fe717fff700 1 --2- 172.21.15.78:0/3746301773 >> v2:172.21.15.133:3300/0 conn(0x7fe720159490 0x7fe7200ce500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:52.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.645+0000 7fe717fff700 1 -- 172.21.15.78:0/3746301773 >> v1:172.21.15.133:6789/0 conn(0x7fe72015d9c0 legacy=0x7fe7200c92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:52.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.645+0000 7fe717fff700 1 -- 172.21.15.78:0/3746301773 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe720159ef0 con 0x7fe72015d030 2023-10-11T09:41:52.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.645+0000 7fe717fff700 1 --2- 172.21.15.78:0/3746301773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe72015d030 0x7fe72015d480 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe72006f760 tx=0x7fe710009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ea9ed5a719d154 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:52.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.649+0000 7fe7177fe700 1 -- 172.21.15.78:0/3746301773 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71000b8a0 con 0x7fe72015d030 2023-10-11T09:41:52.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.649+0000 7fe7177fe700 1 -- 172.21.15.78:0/3746301773 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71000ba00 con 0x7fe72015d030 2023-10-11T09:41:52.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.649+0000 7fe7177fe700 1 -- 172.21.15.78:0/3746301773 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71000bd90 con 0x7fe72015d030 2023-10-11T09:41:52.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.649+0000 7fe726b97700 1 -- 172.21.15.78:0/3746301773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe72015d030 msgr2=0x7fe72015d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:52.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.649+0000 7fe726b97700 1 --2- 172.21.15.78:0/3746301773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe72015d030 0x7fe72015d480 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe72006f760 tx=0x7fe710009bf0 comp rx=0 tx=0).stop 2023-10-11T09:41:52.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.649+0000 7fe726b97700 1 -- 172.21.15.78:0/3746301773 shutdown_connections 2023-10-11T09:41:52.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.649+0000 7fe726b97700 1 --2- 172.21.15.78:0/3746301773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe72015d030 0x7fe72015d480 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:52.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.649+0000 7fe726b97700 1 --2- 172.21.15.78:0/3746301773 >> v2:172.21.15.133:3300/0 conn(0x7fe720159490 0x7fe7200ce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:52.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.649+0000 7fe726b97700 1 -- 172.21.15.78:0/3746301773 >> 172.21.15.78:0/3746301773 conn(0x7fe72006aca0 msgr2=0x7fe7200b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:52.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.649+0000 7fe726b97700 1 -- 172.21.15.78:0/3746301773 shutdown_connections 2023-10-11T09:41:52.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.649+0000 7fe726b97700 1 -- 172.21.15.78:0/3746301773 wait complete. 2023-10-11T09:41:52.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.649+0000 7fe726b97700 1 Processor -- start 2023-10-11T09:41:52.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.649+0000 7fe726b97700 1 -- start start 2023-10-11T09:41:52.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.653+0000 7fe726b97700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe720159490 0x7fe7201470d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:52.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.653+0000 7fe726b97700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe72015d030 0x7fe7201585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:52.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.653+0000 7fe726b97700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe72015d9c0 0x7fe72015abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:52.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.653+0000 7fe726b97700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe720157560 con 0x7fe72015d9c0 2023-10-11T09:41:52.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.653+0000 7fe726b97700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe720157800 con 0x7fe72015d030 2023-10-11T09:41:52.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.653+0000 7fe726b97700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe720157ae0 con 0x7fe720159490 2023-10-11T09:41:52.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.653+0000 7fe725134700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe72015d9c0 0x7fe72015abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:52.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.653+0000 7fe725134700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe72015d9c0 0x7fe72015abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40242/0 (socket says 172.21.15.78:40242) 2023-10-11T09:41:52.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.653+0000 7fe725134700 1 -- 172.21.15.78:0/1966731426 learned_addr learned my addr 172.21.15.78:0/1966731426 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:52.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.653+0000 7fe717fff700 1 --2- 172.21.15.78:0/1966731426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe72015d030 0x7fe7201585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:52.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.653+0000 7fe724933700 1 --2- 172.21.15.78:0/1966731426 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe720159490 0x7fe7201470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:52.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.653+0000 7fe725134700 1 -- 172.21.15.78:0/1966731426 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe720159490 msgr2=0x7fe7201470d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:52.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.653+0000 7fe725134700 1 --2- 172.21.15.78:0/1966731426 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe720159490 0x7fe7201470d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:52.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.653+0000 7fe725134700 1 -- 172.21.15.78:0/1966731426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe72015d030 msgr2=0x7fe7201585c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:52.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.653+0000 7fe725134700 1 --2- 172.21.15.78:0/1966731426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe72015d030 0x7fe7201585c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:52.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.653+0000 7fe725134700 1 -- 172.21.15.78:0/1966731426 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe710009570 con 0x7fe72015d9c0 2023-10-11T09:41:52.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.653+0000 7fe724933700 1 --2- 172.21.15.78:0/1966731426 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe720159490 0x7fe7201470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:52.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.653+0000 7fe725134700 1 --2- 172.21.15.78:0/1966731426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe72015d9c0 0x7fe72015abb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe71800cb50 tx=0x7fe71800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:52.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.653+0000 7fe7157fa700 1 -- 172.21.15.78:0/1966731426 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe718003c20 con 0x7fe72015d9c0 2023-10-11T09:41:52.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.653+0000 7fe7157fa700 1 -- 172.21.15.78:0/1966731426 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe718003d80 con 0x7fe72015d9c0 2023-10-11T09:41:52.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.653+0000 7fe726b97700 1 -- 172.21.15.78:0/1966731426 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe720157dc0 con 0x7fe72015d9c0 2023-10-11T09:41:52.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.653+0000 7fe7157fa700 1 -- 172.21.15.78:0/1966731426 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71800aac0 con 0x7fe72015d9c0 2023-10-11T09:41:52.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.653+0000 7fe726b97700 1 -- 172.21.15.78:0/1966731426 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe720158b00 con 0x7fe72015d9c0 2023-10-11T09:41:52.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.657+0000 7fe726b97700 1 -- 172.21.15.78:0/1966731426 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7201582e0 con 0x7fe72015d9c0 2023-10-11T09:41:52.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.657+0000 7fe7157fa700 1 -- 172.21.15.78:0/1966731426 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fe71800ac20 con 0x7fe72015d9c0 2023-10-11T09:41:52.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.657+0000 7fe7157fa700 1 --2- 172.21.15.78:0/1966731426 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe70c065f00 0x7fe70c0683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:52.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.657+0000 7fe7157fa700 1 -- 172.21.15.78:0/1966731426 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe7180834c0 con 0x7fe72015d9c0 2023-10-11T09:41:52.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.657+0000 7fe724933700 1 --2- 172.21.15.78:0/1966731426 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe70c065f00 0x7fe70c0683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:52.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.661+0000 7fe724933700 1 --2- 172.21.15.78:0/1966731426 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe70c065f00 0x7fe70c0683b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe720152f80 tx=0x7fe71c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:52.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.661+0000 7fe7157fa700 1 -- 172.21.15.78:0/1966731426 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fe7201582e0 con 0x7fe72015d9c0 2023-10-11T09:41:52.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.893+0000 7fe726b97700 1 -- 172.21.15.78:0/1966731426 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fe7201582e0 con 0x7fe72015d9c0 2023-10-11T09:41:52.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.905+0000 7fe7157fa700 1 -- 172.21.15.78:0/1966731426 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe7201582e0 con 0x7fe72015d9c0 2023-10-11T09:41:52.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.909+0000 7fe726b97700 1 -- 172.21.15.78:0/1966731426 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe70c065f00 msgr2=0x7fe70c0683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:52.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.909+0000 7fe726b97700 1 --2- 172.21.15.78:0/1966731426 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe70c065f00 0x7fe70c0683b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe720152f80 tx=0x7fe71c008040 comp rx=0 tx=0).stop 2023-10-11T09:41:52.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.909+0000 7fe726b97700 1 -- 172.21.15.78:0/1966731426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe72015d9c0 msgr2=0x7fe72015abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:52.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.909+0000 7fe726b97700 1 --2- 172.21.15.78:0/1966731426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe72015d9c0 0x7fe72015abb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe71800cb50 tx=0x7fe71800ce60 comp rx=0 tx=0).stop 2023-10-11T09:41:52.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.909+0000 7fe726b97700 1 -- 172.21.15.78:0/1966731426 shutdown_connections 2023-10-11T09:41:52.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.909+0000 7fe726b97700 1 --2- 172.21.15.78:0/1966731426 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe70c065f00 0x7fe70c0683b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:52.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.909+0000 7fe726b97700 1 --2- 172.21.15.78:0/1966731426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe72015d9c0 0x7fe72015abb0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:52.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.909+0000 7fe726b97700 1 --2- 172.21.15.78:0/1966731426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe72015d030 0x7fe7201585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:52.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.909+0000 7fe726b97700 1 --2- 172.21.15.78:0/1966731426 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe720159490 0x7fe7201470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:52.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.909+0000 7fe726b97700 1 -- 172.21.15.78:0/1966731426 >> 172.21.15.78:0/1966731426 conn(0x7fe72006aca0 msgr2=0x7fe720147900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:52.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.909+0000 7fe726b97700 1 -- 172.21.15.78:0/1966731426 shutdown_connections 2023-10-11T09:41:52.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:52.909+0000 7fe726b97700 1 -- 172.21.15.78:0/1966731426 wait complete. 2023-10-11T09:41:52.935 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-10-11T09:41:53.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.101+0000 7f20a5770700 1 Processor -- start 2023-10-11T09:41:53.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.101+0000 7f20a5770700 1 -- start start 2023-10-11T09:41:53.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.105+0000 7f20a5770700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f20a015a940 0x7f20a00ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:53.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.105+0000 7f20a5770700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20a015e010 con 0x7f20a00ca780 2023-10-11T09:41:53.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.105+0000 7f20a5770700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20a00cab70 con 0x7f20a015b8e0 2023-10-11T09:41:53.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.105+0000 7f20a5770700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20a015e280 con 0x7f20a015a940 2023-10-11T09:41:53.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.105+0000 7f20a4f6f700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f20a00ca780 0x7f20a013fcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:35724/0 (socket says 172.21.15.78:35724) 2023-10-11T09:41:53.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.105+0000 7f20a4f6f700 1 -- 172.21.15.78:0/1796589283 learned_addr learned my addr 172.21.15.78:0/1796589283 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:53.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.105+0000 7f2097fff700 1 --2- 172.21.15.78:0/1796589283 >> v2:172.21.15.78:3300/0 conn(0x7f20a015a940 0x7f20a00ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:53.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.105+0000 7f209f7fe700 1 -- 172.21.15.78:0/1796589283 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 533469261 0 0) 0x7f20a015e010 con 0x7f20a00ca780 2023-10-11T09:41:53.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.105+0000 7f209f7fe700 1 -- 172.21.15.78:0/1796589283 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20800035d0 con 0x7f20a00ca780 2023-10-11T09:41:53.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.105+0000 7f209f7fe700 1 -- 172.21.15.78:0/1796589283 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 339734230 0 0) 0x7f20a00cab70 con 0x7f20a015b8e0 2023-10-11T09:41:53.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.105+0000 7f209f7fe700 1 -- 172.21.15.78:0/1796589283 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20a015e010 con 0x7f20a015b8e0 2023-10-11T09:41:53.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.105+0000 7f2097fff700 1 -- 172.21.15.78:0/1796589283 >> v1:172.21.15.133:6789/0 conn(0x7f20a015b8e0 legacy=0x7f20a006d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:53.114 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.105+0000 7f2097fff700 1 -- 172.21.15.78:0/1796589283 >> v1:172.21.15.78:6789/0 conn(0x7f20a00ca780 legacy=0x7f20a013fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:53.114 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.105+0000 7f209f7fe700 1 -- 172.21.15.78:0/1796589283 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1981525317 0 0) 0x7f20800035d0 con 0x7f20a00ca780 2023-10-11T09:41:53.114 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.105+0000 7f2097fff700 1 -- 172.21.15.78:0/1796589283 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a015e560 con 0x7f20a015a940 2023-10-11T09:41:53.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.105+0000 7f2097fff700 1 --2- 172.21.15.78:0/1796589283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a015a940 0x7f20a00ca240 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2088009850 tx=0x7f2088009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99427155cd362d9c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:53.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.105+0000 7f209f7fe700 1 -- 172.21.15.78:0/1796589283 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20980019f0 con 0x7f20a015a940 2023-10-11T09:41:53.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.105+0000 7f209f7fe700 1 -- 172.21.15.78:0/1796589283 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2088004030 con 0x7f20a015a940 2023-10-11T09:41:53.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.105+0000 7f209f7fe700 1 -- 172.21.15.78:0/1796589283 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20880043a0 con 0x7f20a015a940 2023-10-11T09:41:53.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.109+0000 7f20a5770700 1 -- 172.21.15.78:0/1796589283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a015a940 msgr2=0x7f20a00ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:53.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.109+0000 7f20a5770700 1 --2- 172.21.15.78:0/1796589283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a015a940 0x7f20a00ca240 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2088009850 tx=0x7f2088009b60 comp rx=0 tx=0).stop 2023-10-11T09:41:53.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.109+0000 7f20a5770700 1 -- 172.21.15.78:0/1796589283 shutdown_connections 2023-10-11T09:41:53.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.109+0000 7f20a5770700 1 --2- 172.21.15.78:0/1796589283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a015a940 0x7f20a00ca240 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:53.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.109+0000 7f20a5770700 1 -- 172.21.15.78:0/1796589283 >> 172.21.15.78:0/1796589283 conn(0x7f20a00cea60 msgr2=0x7f20a00cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:53.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.109+0000 7f20a5770700 1 -- 172.21.15.78:0/1796589283 shutdown_connections 2023-10-11T09:41:53.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.109+0000 7f20a5770700 1 -- 172.21.15.78:0/1796589283 wait complete. 2023-10-11T09:41:53.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.109+0000 7f20a5770700 1 Processor -- start 2023-10-11T09:41:53.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.109+0000 7f20a5770700 1 -- start start 2023-10-11T09:41:53.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.113+0000 7f20a5770700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20a00ca780 0x7f20a00b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:53.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.113+0000 7f20a5770700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a015a940 0x7f20a013dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:53.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.113+0000 7f20a5770700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20a015b8e0 0x7f20a00c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:53.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.113+0000 7f20a5770700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a00a8880 con 0x7f20a015a940 2023-10-11T09:41:53.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.113+0000 7f20a5770700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a01f5280 con 0x7f20a015b8e0 2023-10-11T09:41:53.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.113+0000 7f20a5770700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a01f5560 con 0x7f20a00ca780 2023-10-11T09:41:53.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.113+0000 7f2097fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a015a940 0x7f20a013dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:53.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.113+0000 7f2097fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a015a940 0x7f20a013dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40268/0 (socket says 172.21.15.78:40268) 2023-10-11T09:41:53.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.113+0000 7f2097fff700 1 -- 172.21.15.78:0/3649173658 learned_addr learned my addr 172.21.15.78:0/3649173658 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:53.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.113+0000 7f20a4f6f700 1 --2- 172.21.15.78:0/3649173658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20a015b8e0 0x7f20a00c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:53.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.113+0000 7f209ffff700 1 --2- 172.21.15.78:0/3649173658 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20a00ca780 0x7f20a00b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:53.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.113+0000 7f2097fff700 1 -- 172.21.15.78:0/3649173658 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20a00ca780 msgr2=0x7f20a00b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:53.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.113+0000 7f2097fff700 1 --2- 172.21.15.78:0/3649173658 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20a00ca780 0x7f20a00b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:53.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.113+0000 7f2097fff700 1 -- 172.21.15.78:0/3649173658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20a015b8e0 msgr2=0x7f20a00c7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:53.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.113+0000 7f2097fff700 1 --2- 172.21.15.78:0/3649173658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20a015b8e0 0x7f20a00c7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:53.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.113+0000 7f2097fff700 1 -- 172.21.15.78:0/3649173658 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20880094b0 con 0x7f20a015a940 2023-10-11T09:41:53.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.113+0000 7f2097fff700 1 --2- 172.21.15.78:0/3649173658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a015a940 0x7f20a013dd50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f20880090b0 tx=0x7f2088004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:53.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.113+0000 7f209d7fa700 1 -- 172.21.15.78:0/3649173658 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2088006ea0 con 0x7f20a015a940 2023-10-11T09:41:53.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.113+0000 7f209d7fa700 1 -- 172.21.15.78:0/3649173658 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f208801f070 con 0x7f20a015a940 2023-10-11T09:41:53.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.113+0000 7f20a5770700 1 -- 172.21.15.78:0/3649173658 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20a01f5840 con 0x7f20a015a940 2023-10-11T09:41:53.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.113+0000 7f209d7fa700 1 -- 172.21.15.78:0/3649173658 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2088011630 con 0x7f20a015a940 2023-10-11T09:41:53.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.113+0000 7f20a5770700 1 -- 172.21.15.78:0/3649173658 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a00b4830 con 0x7f20a015a940 2023-10-11T09:41:53.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.117+0000 7f20a5770700 1 -- 172.21.15.78:0/3649173658 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20a0067f50 con 0x7f20a015a940 2023-10-11T09:41:53.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.117+0000 7f209d7fa700 1 -- 172.21.15.78:0/3649173658 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f2088021030 con 0x7f20a015a940 2023-10-11T09:41:53.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.117+0000 7f209d7fa700 1 --2- 172.21.15.78:0/3649173658 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2084065ed0 0x7f2084068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:53.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.121+0000 7f209d7fa700 1 -- 172.21.15.78:0/3649173658 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f20880847d0 con 0x7f20a015a940 2023-10-11T09:41:53.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.121+0000 7f209ffff700 1 --2- 172.21.15.78:0/3649173658 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2084065ed0 0x7f2084068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:53.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.121+0000 7f209ffff700 1 --2- 172.21.15.78:0/3649173658 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2084065ed0 0x7f2084068380 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2090000f60 tx=0x7f2090009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:53.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.121+0000 7f209d7fa700 1 -- 172.21.15.78:0/3649173658 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f2088055c90 con 0x7f20a015a940 2023-10-11T09:41:53.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.353+0000 7f20a5770700 1 -- 172.21.15.78:0/3649173658 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f20a0067f50 con 0x7f20a015a940 2023-10-11T09:41:53.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.361+0000 7f209d7fa700 1 -- 172.21.15.78:0/3649173658 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f2088006ce0 con 0x7f20a015a940 2023-10-11T09:41:53.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.361+0000 7f20a5770700 1 -- 172.21.15.78:0/3649173658 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2084065ed0 msgr2=0x7f2084068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:53.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.361+0000 7f20a5770700 1 --2- 172.21.15.78:0/3649173658 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2084065ed0 0x7f2084068380 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2090000f60 tx=0x7f2090009040 comp rx=0 tx=0).stop 2023-10-11T09:41:53.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.361+0000 7f20a5770700 1 -- 172.21.15.78:0/3649173658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a015a940 msgr2=0x7f20a013dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:53.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.361+0000 7f20a5770700 1 --2- 172.21.15.78:0/3649173658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a015a940 0x7f20a013dd50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f20880090b0 tx=0x7f2088004770 comp rx=0 tx=0).stop 2023-10-11T09:41:53.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.361+0000 7f20a5770700 1 -- 172.21.15.78:0/3649173658 shutdown_connections 2023-10-11T09:41:53.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.361+0000 7f20a5770700 1 --2- 172.21.15.78:0/3649173658 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2084065ed0 0x7f2084068380 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:53.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.361+0000 7f20a5770700 1 --2- 172.21.15.78:0/3649173658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20a015b8e0 0x7f20a00c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:53.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.361+0000 7f20a5770700 1 --2- 172.21.15.78:0/3649173658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a015a940 0x7f20a013dd50 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:53.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.361+0000 7f20a5770700 1 --2- 172.21.15.78:0/3649173658 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20a00ca780 0x7f20a00b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:53.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.361+0000 7f20a5770700 1 -- 172.21.15.78:0/3649173658 >> 172.21.15.78:0/3649173658 conn(0x7f20a00cea60 msgr2=0x7f20a013f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:53.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.361+0000 7f20a5770700 1 -- 172.21.15.78:0/3649173658 shutdown_connections 2023-10-11T09:41:53.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.361+0000 7f20a5770700 1 -- 172.21.15.78:0/3649173658 wait complete. 2023-10-11T09:41:53.391 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-10-11T09:41:53.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.565+0000 7f8d0d70e700 1 Processor -- start 2023-10-11T09:41:53.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.565+0000 7f8d0d70e700 1 -- start start 2023-10-11T09:41:53.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.565+0000 7f8d0d70e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d080ca950 0x7f8d081650f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:53.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.565+0000 7f8d0d70e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d0815e010 con 0x7f8d0815a940 2023-10-11T09:41:53.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.565+0000 7f8d0d70e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d0815e280 con 0x7f8d0815b8e0 2023-10-11T09:41:53.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.565+0000 7f8d0d70e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0815e560 con 0x7f8d080ca950 2023-10-11T09:41:53.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.565+0000 7f8cff7fe700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f8d0815a940 0x7f8d080ca240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:35726/0 (socket says 172.21.15.78:35726) 2023-10-11T09:41:53.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.565+0000 7f8cff7fe700 1 -- 172.21.15.78:0/3165151274 learned_addr learned my addr 172.21.15.78:0/3165151274 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:53.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.569+0000 7f8d0cf0d700 1 --2- 172.21.15.78:0/3165151274 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d080ca950 0x7f8d081650f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:53.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.569+0000 7f8d077fe700 1 -- 172.21.15.78:0/3165151274 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2285061889 0 0) 0x7f8d0815e010 con 0x7f8d0815a940 2023-10-11T09:41:53.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.569+0000 7f8d077fe700 1 -- 172.21.15.78:0/3165151274 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cf40035d0 con 0x7f8d0815a940 2023-10-11T09:41:53.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.569+0000 7f8d077fe700 1 -- 172.21.15.78:0/3165151274 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2630924353 0 0) 0x7f8d0815e280 con 0x7f8d0815b8e0 2023-10-11T09:41:53.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.569+0000 7f8d077fe700 1 -- 172.21.15.78:0/3165151274 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d0815e010 con 0x7f8d0815b8e0 2023-10-11T09:41:53.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.569+0000 7f8d077fe700 1 -- 172.21.15.78:0/3165151274 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2884613495 0 0) 0x7f8cf40035d0 con 0x7f8d0815a940 2023-10-11T09:41:53.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.569+0000 7f8d077fe700 1 -- 172.21.15.78:0/3165151274 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d080ca950 msgr2=0x7f8d081650f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:53.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.569+0000 7f8d077fe700 1 --2- 172.21.15.78:0/3165151274 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d080ca950 0x7f8d081650f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:53.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.569+0000 7f8d077fe700 1 -- 172.21.15.78:0/3165151274 >> v1:172.21.15.133:6789/0 conn(0x7f8d0815b8e0 legacy=0x7f8d0806d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:53.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.569+0000 7f8d077fe700 1 -- 172.21.15.78:0/3165151274 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0815e840 con 0x7f8d0815a940 2023-10-11T09:41:53.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.569+0000 7f8d077fe700 1 -- 172.21.15.78:0/3165151274 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f8cf0001670 con 0x7f8d0815a940 2023-10-11T09:41:53.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.569+0000 7f8d077fe700 1 -- 172.21.15.78:0/3165151274 >> v1:172.21.15.78:6789/0 conn(0x7f8d0815a940 legacy=0x7f8d080ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:53.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.569+0000 7f8d077fe700 1 --2- 172.21.15.78:0/3165151274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cf4003be0 0x7f8cf4003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:53.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.569+0000 7f8d077fe700 1 --2- 172.21.15.78:0/3165151274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8cf4004870 0x7f8cf4004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:53.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.569+0000 7f8d077fe700 1 --2- 172.21.15.78:0/3165151274 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d080ca950 0x7f8cf4005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:53.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.569+0000 7f8d077fe700 1 -- 172.21.15.78:0/3165151274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0815e560 con 0x7f8cf4003be0 2023-10-11T09:41:53.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.569+0000 7f8d077fe700 1 -- 172.21.15.78:0/3165151274 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf400bae0 con 0x7f8cf4004870 2023-10-11T09:41:53.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.569+0000 7f8d077fe700 1 -- 172.21.15.78:0/3165151274 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf400bc50 con 0x7f8d080ca950 2023-10-11T09:41:53.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.569+0000 7f8d07fff700 1 --2- 172.21.15.78:0/3165151274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cf4003be0 0x7f8cf4003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:53.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.569+0000 7f8cff7fe700 1 --2- 172.21.15.78:0/3165151274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8cf4004870 0x7f8cf4004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:53.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.569+0000 7f8d0cf0d700 1 --2- 172.21.15.78:0/3165151274 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d080ca950 0x7f8cf4005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:53.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.569+0000 7f8d07fff700 1 -- 172.21.15.78:0/3165151274 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d080ca950 msgr2=0x7f8cf4005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:53.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.569+0000 7f8d07fff700 1 --2- 172.21.15.78:0/3165151274 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d080ca950 0x7f8cf4005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:53.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.569+0000 7f8d07fff700 1 -- 172.21.15.78:0/3165151274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8cf4004870 msgr2=0x7f8cf4004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:53.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.569+0000 7f8d07fff700 1 --2- 172.21.15.78:0/3165151274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8cf4004870 0x7f8cf4004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:53.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.569+0000 7f8d07fff700 1 -- 172.21.15.78:0/3165151274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf40035d0 con 0x7f8cf4003be0 2023-10-11T09:41:53.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.569+0000 7f8d0cf0d700 1 --2- 172.21.15.78:0/3165151274 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d080ca950 0x7f8cf4005220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:41:53.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.569+0000 7f8cff7fe700 1 --2- 172.21.15.78:0/3165151274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8cf4004870 0x7f8cf4004ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:41:53.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.569+0000 7f8d07fff700 1 --2- 172.21.15.78:0/3165151274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cf4003be0 0x7f8cf4003fd0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8001960 tx=0x7f8cf40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b864bcc559ba6bec server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:53.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.569+0000 7f8d077fe700 1 -- 172.21.15.78:0/3165151274 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf8001450 con 0x7f8cf4003be0 2023-10-11T09:41:53.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.569+0000 7f8d077fe700 1 -- 172.21.15.78:0/3165151274 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf80052f0 con 0x7f8cf4003be0 2023-10-11T09:41:53.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.569+0000 7f8d077fe700 1 -- 172.21.15.78:0/3165151274 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf80036a0 con 0x7f8cf4003be0 2023-10-11T09:41:53.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.569+0000 7f8d0d70e700 1 -- 172.21.15.78:0/3165151274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cf4003be0 msgr2=0x7f8cf4003fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:53.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.569+0000 7f8d0d70e700 1 --2- 172.21.15.78:0/3165151274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cf4003be0 0x7f8cf4003fd0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8001960 tx=0x7f8cf40058a0 comp rx=0 tx=0).stop 2023-10-11T09:41:53.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.569+0000 7f8d0cf0d700 1 -- 172.21.15.78:0/3165151274 reap_dead start 2023-10-11T09:41:53.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.569+0000 7f8d0d70e700 1 -- 172.21.15.78:0/3165151274 shutdown_connections 2023-10-11T09:41:53.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.573+0000 7f8d0d70e700 1 -- 172.21.15.78:0/3165151274 >> 172.21.15.78:0/3165151274 conn(0x7f8d080cea60 msgr2=0x7f8d080cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:53.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.573+0000 7f8d0d70e700 1 -- 172.21.15.78:0/3165151274 shutdown_connections 2023-10-11T09:41:53.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.573+0000 7f8d0d70e700 1 -- 172.21.15.78:0/3165151274 wait complete. 2023-10-11T09:41:53.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.573+0000 7f8d0d70e700 1 Processor -- start 2023-10-11T09:41:53.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.573+0000 7f8d0d70e700 1 -- start start 2023-10-11T09:41:53.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.577+0000 7f8d0d70e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8cf4003be0 0x7f8d080b38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:53.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.577+0000 7f8d0d70e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d08159300 0x7f8d0813d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:53.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.577+0000 7f8d0d70e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d080cccf0 0x7f8d080cd190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:53.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.577+0000 7f8d0d70e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d080cd6d0 con 0x7f8d08159300 2023-10-11T09:41:53.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.577+0000 7f8d0d70e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d08159d60 con 0x7f8cf4003be0 2023-10-11T09:41:53.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.577+0000 7f8d0d70e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d080b2800 con 0x7f8d080cccf0 2023-10-11T09:41:53.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.577+0000 7f8cff7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d08159300 0x7f8d0813d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:53.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.577+0000 7f8cff7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d08159300 0x7f8d0813d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40292/0 (socket says 172.21.15.78:40292) 2023-10-11T09:41:53.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.577+0000 7f8cff7fe700 1 -- 172.21.15.78:0/2404163855 learned_addr learned my addr 172.21.15.78:0/2404163855 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:53.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.577+0000 7f8d07fff700 1 --2- 172.21.15.78:0/2404163855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8cf4003be0 0x7f8d080b38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:53.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.577+0000 7f8d0cf0d700 1 --2- 172.21.15.78:0/2404163855 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d080cccf0 0x7f8d080cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:53.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.577+0000 7f8cff7fe700 1 -- 172.21.15.78:0/2404163855 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d080cccf0 msgr2=0x7f8d080cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:53.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.577+0000 7f8cff7fe700 1 --2- 172.21.15.78:0/2404163855 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d080cccf0 0x7f8d080cd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:53.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.577+0000 7f8cff7fe700 1 -- 172.21.15.78:0/2404163855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8cf4003be0 msgr2=0x7f8d080b38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:53.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.577+0000 7f8cff7fe700 1 --2- 172.21.15.78:0/2404163855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8cf4003be0 0x7f8d080b38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:53.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.577+0000 7f8cff7fe700 1 -- 172.21.15.78:0/2404163855 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d080b2ae0 con 0x7f8d08159300 2023-10-11T09:41:53.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.577+0000 7f8cff7fe700 1 --2- 172.21.15.78:0/2404163855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d08159300 0x7f8d0813d550 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8cf000db10 tx=0x7f8cf000de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:53.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.577+0000 7f8d057fa700 1 -- 172.21.15.78:0/2404163855 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0004ca0 con 0x7f8d08159300 2023-10-11T09:41:53.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.577+0000 7f8d057fa700 1 -- 172.21.15.78:0/2404163855 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf0004e00 con 0x7f8d08159300 2023-10-11T09:41:53.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.577+0000 7f8d0d70e700 1 -- 172.21.15.78:0/2404163855 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d080b2dc0 con 0x7f8d08159300 2023-10-11T09:41:53.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.577+0000 7f8d057fa700 1 -- 172.21.15.78:0/2404163855 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf00072d0 con 0x7f8d08159300 2023-10-11T09:41:53.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.577+0000 7f8d0d70e700 1 -- 172.21.15.78:0/2404163855 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d080b3380 con 0x7f8d08159300 2023-10-11T09:41:53.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.581+0000 7f8d0d70e700 1 -- 172.21.15.78:0/2404163855 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d08065d40 con 0x7f8d08159300 2023-10-11T09:41:53.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.581+0000 7f8d057fa700 1 -- 172.21.15.78:0/2404163855 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f8cf0007430 con 0x7f8d08159300 2023-10-11T09:41:53.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.585+0000 7f8d057fa700 1 --2- 172.21.15.78:0/2404163855 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8ce8065ec0 0x7f8ce8068370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:53.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.585+0000 7f8d057fa700 1 -- 172.21.15.78:0/2404163855 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8cf0082ca0 con 0x7f8d08159300 2023-10-11T09:41:53.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.585+0000 7f8d07fff700 1 --2- 172.21.15.78:0/2404163855 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8ce8065ec0 0x7f8ce8068370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:53.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.585+0000 7f8d07fff700 1 --2- 172.21.15.78:0/2404163855 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8ce8065ec0 0x7f8ce8068370 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8015040 tx=0x7f8cf800a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:53.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.589+0000 7f8d057fa700 1 -- 172.21.15.78:0/2404163855 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f8cf0050510 con 0x7f8d08159300 2023-10-11T09:41:53.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.821+0000 7f8d0d70e700 1 -- 172.21.15.78:0/2404163855 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f8d08143d20 con 0x7f8d08159300 2023-10-11T09:41:53.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.829+0000 7f8d057fa700 1 -- 172.21.15.78:0/2404163855 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f8cf0019070 con 0x7f8d08159300 2023-10-11T09:41:53.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.833+0000 7f8d0d70e700 1 -- 172.21.15.78:0/2404163855 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8ce8065ec0 msgr2=0x7f8ce8068370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:53.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.833+0000 7f8d0d70e700 1 --2- 172.21.15.78:0/2404163855 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8ce8065ec0 0x7f8ce8068370 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8015040 tx=0x7f8cf800a000 comp rx=0 tx=0).stop 2023-10-11T09:41:53.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.833+0000 7f8d0d70e700 1 -- 172.21.15.78:0/2404163855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d08159300 msgr2=0x7f8d0813d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:53.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.833+0000 7f8d0d70e700 1 --2- 172.21.15.78:0/2404163855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d08159300 0x7f8d0813d550 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8cf000db10 tx=0x7f8cf000de20 comp rx=0 tx=0).stop 2023-10-11T09:41:53.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.833+0000 7f8d0d70e700 1 -- 172.21.15.78:0/2404163855 shutdown_connections 2023-10-11T09:41:53.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.833+0000 7f8d0d70e700 1 --2- 172.21.15.78:0/2404163855 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8ce8065ec0 0x7f8ce8068370 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:53.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.833+0000 7f8d0d70e700 1 --2- 172.21.15.78:0/2404163855 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d080cccf0 0x7f8d080cd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:53.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.833+0000 7f8d0d70e700 1 --2- 172.21.15.78:0/2404163855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d08159300 0x7f8d0813d550 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:53.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.833+0000 7f8d0d70e700 1 --2- 172.21.15.78:0/2404163855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8cf4003be0 0x7f8d080b38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:53.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.833+0000 7f8d0d70e700 1 -- 172.21.15.78:0/2404163855 >> 172.21.15.78:0/2404163855 conn(0x7f8d080cea60 msgr2=0x7f8d0806d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:53.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.833+0000 7f8d0d70e700 1 -- 172.21.15.78:0/2404163855 shutdown_connections 2023-10-11T09:41:53.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:53.833+0000 7f8d0d70e700 1 -- 172.21.15.78:0/2404163855 wait complete. 2023-10-11T09:41:53.859 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-10-11T09:41:54.030 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.025+0000 7f2ba54c7700 1 Processor -- start 2023-10-11T09:41:54.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.025+0000 7f2ba54c7700 1 -- start start 2023-10-11T09:41:54.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.029+0000 7f2ba54c7700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f2ba00b0db0 0x7f2ba0160c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:54.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.029+0000 7f2ba54c7700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ba0160210 con 0x7f2ba00cc820 2023-10-11T09:41:54.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.029+0000 7f2ba54c7700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ba01611a0 con 0x7f2ba015d050 2023-10-11T09:41:54.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.029+0000 7f2ba54c7700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ba00ccfc0 con 0x7f2ba00b0db0 2023-10-11T09:41:54.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.029+0000 7f2ba4cc6700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f2ba00b0db0 0x7f2ba0160c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:54.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.029+0000 7f2b97fff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f2ba00cc820 0x7f2ba00b06a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:35732/0 (socket says 172.21.15.78:35732) 2023-10-11T09:41:54.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.029+0000 7f2b97fff700 1 -- 172.21.15.78:0/570479044 learned_addr learned my addr 172.21.15.78:0/570479044 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:54.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.029+0000 7f2ba4cc6700 1 -- 172.21.15.78:0/570479044 >> v1:172.21.15.133:6789/0 conn(0x7f2ba015d050 legacy=0x7f2ba015d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:54.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.029+0000 7f2ba4cc6700 1 -- 172.21.15.78:0/570479044 >> v1:172.21.15.78:6789/0 conn(0x7f2ba00cc820 legacy=0x7f2ba00b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:54.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.029+0000 7f2ba4cc6700 1 -- 172.21.15.78:0/570479044 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba00cd2a0 con 0x7f2ba00b0db0 2023-10-11T09:41:54.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.029+0000 7f2ba4cc6700 1 --2- 172.21.15.78:0/570479044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ba00b0db0 0x7f2ba0160c60 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2b98009d90 tx=0x7f2b9800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2966d569974751 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:54.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.029+0000 7f2b9f7fe700 1 -- 172.21.15.78:0/570479044 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9800e070 con 0x7f2ba00b0db0 2023-10-11T09:41:54.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.029+0000 7f2b9f7fe700 1 -- 172.21.15.78:0/570479044 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b9800bbd0 con 0x7f2ba00b0db0 2023-10-11T09:41:54.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.029+0000 7f2b9f7fe700 1 -- 172.21.15.78:0/570479044 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b98012430 con 0x7f2ba00b0db0 2023-10-11T09:41:54.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.033+0000 7f2ba54c7700 1 -- 172.21.15.78:0/570479044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ba00b0db0 msgr2=0x7f2ba0160c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:54.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.033+0000 7f2ba54c7700 1 --2- 172.21.15.78:0/570479044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ba00b0db0 0x7f2ba0160c60 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2b98009d90 tx=0x7f2b9800b330 comp rx=0 tx=0).stop 2023-10-11T09:41:54.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.033+0000 7f2ba54c7700 1 -- 172.21.15.78:0/570479044 shutdown_connections 2023-10-11T09:41:54.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.033+0000 7f2ba54c7700 1 --2- 172.21.15.78:0/570479044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ba00b0db0 0x7f2ba0160c60 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:54.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.033+0000 7f2ba54c7700 1 -- 172.21.15.78:0/570479044 >> 172.21.15.78:0/570479044 conn(0x7f2ba00cea60 msgr2=0x7f2ba00cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:54.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.033+0000 7f2ba54c7700 1 -- 172.21.15.78:0/570479044 shutdown_connections 2023-10-11T09:41:54.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.033+0000 7f2ba54c7700 1 -- 172.21.15.78:0/570479044 wait complete. 2023-10-11T09:41:54.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.033+0000 7f2ba54c7700 1 Processor -- start 2023-10-11T09:41:54.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.033+0000 7f2ba54c7700 1 -- start start 2023-10-11T09:41:54.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.037+0000 7f2ba54c7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ba00b0db0 0x7f2ba013d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:54.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.037+0000 7f2ba54c7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ba00cc820 0x7f2ba01f2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:54.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.037+0000 7f2ba54c7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ba015d050 0x7f2ba00b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:54.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.037+0000 7f2ba54c7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba01f60c0 con 0x7f2ba00b0db0 2023-10-11T09:41:54.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.037+0000 7f2ba54c7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba01f31f0 con 0x7f2ba015d050 2023-10-11T09:41:54.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.037+0000 7f2ba54c7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba015d5c0 con 0x7f2ba00cc820 2023-10-11T09:41:54.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.037+0000 7f2b9ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ba00b0db0 0x7f2ba013d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:54.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.037+0000 7f2b9ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ba00b0db0 0x7f2ba013d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40302/0 (socket says 172.21.15.78:40302) 2023-10-11T09:41:54.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.037+0000 7f2b9ffff700 1 -- 172.21.15.78:0/164952322 learned_addr learned my addr 172.21.15.78:0/164952322 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:54.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.037+0000 7f2b97fff700 1 --2- 172.21.15.78:0/164952322 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ba00cc820 0x7f2ba01f2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:54.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.037+0000 7f2ba4cc6700 1 --2- 172.21.15.78:0/164952322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ba015d050 0x7f2ba00b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:54.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.037+0000 7f2b9ffff700 1 -- 172.21.15.78:0/164952322 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ba00cc820 msgr2=0x7f2ba01f2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:54.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.037+0000 7f2b9ffff700 1 --2- 172.21.15.78:0/164952322 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ba00cc820 0x7f2ba01f2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:54.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.037+0000 7f2b9ffff700 1 -- 172.21.15.78:0/164952322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ba015d050 msgr2=0x7f2ba00b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:54.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.037+0000 7f2b9ffff700 1 --2- 172.21.15.78:0/164952322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ba015d050 0x7f2ba00b3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:54.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.037+0000 7f2b9ffff700 1 -- 172.21.15.78:0/164952322 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b98009a70 con 0x7f2ba00b0db0 2023-10-11T09:41:54.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.037+0000 7f2b9ffff700 1 --2- 172.21.15.78:0/164952322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ba00b0db0 0x7f2ba013d570 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2b9000c500 tx=0x7f2b9000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:54.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.037+0000 7f2b9d7fa700 1 -- 172.21.15.78:0/164952322 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b900052d0 con 0x7f2ba00b0db0 2023-10-11T09:41:54.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.037+0000 7f2b9d7fa700 1 -- 172.21.15.78:0/164952322 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b90005430 con 0x7f2ba00b0db0 2023-10-11T09:41:54.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.037+0000 7f2b9d7fa700 1 -- 172.21.15.78:0/164952322 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b900055a0 con 0x7f2ba00b0db0 2023-10-11T09:41:54.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.037+0000 7f2ba54c7700 1 -- 172.21.15.78:0/164952322 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ba015d8a0 con 0x7f2ba00b0db0 2023-10-11T09:41:54.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.037+0000 7f2ba54c7700 1 -- 172.21.15.78:0/164952322 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ba015de60 con 0x7f2ba00b0db0 2023-10-11T09:41:54.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.041+0000 7f2ba54c7700 1 -- 172.21.15.78:0/164952322 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ba0065d40 con 0x7f2ba00b0db0 2023-10-11T09:41:54.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.041+0000 7f2b9d7fa700 1 -- 172.21.15.78:0/164952322 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f2b900038e0 con 0x7f2ba00b0db0 2023-10-11T09:41:54.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.045+0000 7f2b9d7fa700 1 --2- 172.21.15.78:0/164952322 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2b80065ed0 0x7f2b80068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:54.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.045+0000 7f2b9d7fa700 1 -- 172.21.15.78:0/164952322 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2b900828d0 con 0x7f2ba00b0db0 2023-10-11T09:41:54.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.045+0000 7f2b97fff700 1 --2- 172.21.15.78:0/164952322 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2b80065ed0 0x7f2b80068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:54.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.045+0000 7f2b97fff700 1 --2- 172.21.15.78:0/164952322 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2b80065ed0 0x7f2b80068380 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2ba006d350 tx=0x7f2b88009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:54.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.049+0000 7f2b9d7fa700 1 -- 172.21.15.78:0/164952322 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f2b900501d0 con 0x7f2ba00b0db0 2023-10-11T09:41:54.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.281+0000 7f2ba54c7700 1 -- 172.21.15.78:0/164952322 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f2ba00b1310 con 0x7f2ba00b0db0 2023-10-11T09:41:54.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.289+0000 7f2b9d7fa700 1 -- 172.21.15.78:0/164952322 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f2b900170a0 con 0x7f2ba00b0db0 2023-10-11T09:41:54.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.293+0000 7f2ba54c7700 1 -- 172.21.15.78:0/164952322 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2b80065ed0 msgr2=0x7f2b80068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:54.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.293+0000 7f2ba54c7700 1 --2- 172.21.15.78:0/164952322 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2b80065ed0 0x7f2b80068380 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2ba006d350 tx=0x7f2b88009040 comp rx=0 tx=0).stop 2023-10-11T09:41:54.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.293+0000 7f2ba54c7700 1 -- 172.21.15.78:0/164952322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ba00b0db0 msgr2=0x7f2ba013d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:54.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.293+0000 7f2ba54c7700 1 --2- 172.21.15.78:0/164952322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ba00b0db0 0x7f2ba013d570 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2b9000c500 tx=0x7f2b9000c810 comp rx=0 tx=0).stop 2023-10-11T09:41:54.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.293+0000 7f2ba54c7700 1 -- 172.21.15.78:0/164952322 shutdown_connections 2023-10-11T09:41:54.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.293+0000 7f2ba54c7700 1 --2- 172.21.15.78:0/164952322 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2b80065ed0 0x7f2b80068380 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:54.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.293+0000 7f2ba54c7700 1 --2- 172.21.15.78:0/164952322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ba015d050 0x7f2ba00b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:54.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.293+0000 7f2ba54c7700 1 --2- 172.21.15.78:0/164952322 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ba00cc820 0x7f2ba01f2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:54.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.293+0000 7f2ba54c7700 1 --2- 172.21.15.78:0/164952322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ba00b0db0 0x7f2ba013d570 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:54.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.293+0000 7f2ba54c7700 1 -- 172.21.15.78:0/164952322 >> 172.21.15.78:0/164952322 conn(0x7f2ba00cea60 msgr2=0x7f2ba0070ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.293+0000 7f2ba54c7700 1 -- 172.21.15.78:0/164952322 shutdown_connections 2023-10-11T09:41:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.293+0000 7f2ba54c7700 1 -- 172.21.15.78:0/164952322 wait complete. 2023-10-11T09:41:54.322 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-10-11T09:41:54.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.489+0000 7fc899819700 1 Processor -- start 2023-10-11T09:41:54.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.489+0000 7fc899819700 1 -- start start 2023-10-11T09:41:54.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.493+0000 7fc899819700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fc8940b23e0 0x7fc8941641a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:54.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.493+0000 7fc899819700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8940b2c00 con 0x7fc89415a720 2023-10-11T09:41:54.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.493+0000 7fc899819700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8940b40a0 con 0x7fc894150580 2023-10-11T09:41:54.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.493+0000 7fc899819700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc89406e490 con 0x7fc8940b23e0 2023-10-11T09:41:54.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.493+0000 7fc8927fc700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fc89415a720 0x7fc8940b1cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:35742/0 (socket says 172.21.15.78:35742) 2023-10-11T09:41:54.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.493+0000 7fc8927fc700 1 -- 172.21.15.78:0/323013884 learned_addr learned my addr 172.21.15.78:0/323013884 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:54.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.493+0000 7fc8937fe700 1 --2- 172.21.15.78:0/323013884 >> v2:172.21.15.78:3300/0 conn(0x7fc8940b23e0 0x7fc8941641a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:54.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.493+0000 7fc8937fe700 1 -- 172.21.15.78:0/323013884 >> v1:172.21.15.133:6789/0 conn(0x7fc894150580 legacy=0x7fc89406dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:54.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.493+0000 7fc8937fe700 1 -- 172.21.15.78:0/323013884 >> v1:172.21.15.78:6789/0 conn(0x7fc89415a720 legacy=0x7fc8940b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:54.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.493+0000 7fc8937fe700 1 -- 172.21.15.78:0/323013884 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc894150950 con 0x7fc8940b23e0 2023-10-11T09:41:54.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.493+0000 7fc8937fe700 1 --2- 172.21.15.78:0/323013884 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8940b23e0 0x7fc8941641a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc88c009d90 tx=0x7fc88c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd6fa68984aec3f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:54.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.493+0000 7fc891ffb700 1 -- 172.21.15.78:0/323013884 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c00e070 con 0x7fc8940b23e0 2023-10-11T09:41:54.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.497+0000 7fc891ffb700 1 -- 172.21.15.78:0/323013884 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc88c00bbd0 con 0x7fc8940b23e0 2023-10-11T09:41:54.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.497+0000 7fc891ffb700 1 -- 172.21.15.78:0/323013884 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c012430 con 0x7fc8940b23e0 2023-10-11T09:41:54.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.497+0000 7fc899819700 1 -- 172.21.15.78:0/323013884 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8940b23e0 msgr2=0x7fc8941641a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:54.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.497+0000 7fc899819700 1 --2- 172.21.15.78:0/323013884 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8940b23e0 0x7fc8941641a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc88c009d90 tx=0x7fc88c00b330 comp rx=0 tx=0).stop 2023-10-11T09:41:54.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.497+0000 7fc899819700 1 -- 172.21.15.78:0/323013884 shutdown_connections 2023-10-11T09:41:54.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.497+0000 7fc899819700 1 --2- 172.21.15.78:0/323013884 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8940b23e0 0x7fc8941641a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:54.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.497+0000 7fc899819700 1 -- 172.21.15.78:0/323013884 >> 172.21.15.78:0/323013884 conn(0x7fc8940730c0 msgr2=0x7fc894073490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:54.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.497+0000 7fc899819700 1 -- 172.21.15.78:0/323013884 shutdown_connections 2023-10-11T09:41:54.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.497+0000 7fc899819700 1 -- 172.21.15.78:0/323013884 wait complete. 2023-10-11T09:41:54.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.497+0000 7fc899819700 1 Processor -- start 2023-10-11T09:41:54.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.501+0000 7fc899819700 1 -- start start 2023-10-11T09:41:54.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.501+0000 7fc899819700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8940b23e0 0x7fc8941efa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:54.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.501+0000 7fc899819700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc894150580 0x7fc8941520d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:54.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.501+0000 7fc899819700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc89415a720 0x7fc8941497d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:54.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.501+0000 7fc899819700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc894153530 con 0x7fc8940b23e0 2023-10-11T09:41:54.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.501+0000 7fc899819700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc894154340 con 0x7fc894150580 2023-10-11T09:41:54.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.501+0000 7fc899819700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc894152610 con 0x7fc89415a720 2023-10-11T09:41:54.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.501+0000 7fc892ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8940b23e0 0x7fc8941efa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:54.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.501+0000 7fc892ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8940b23e0 0x7fc8941efa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40320/0 (socket says 172.21.15.78:40320) 2023-10-11T09:41:54.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.501+0000 7fc892ffd700 1 -- 172.21.15.78:0/1200272459 learned_addr learned my addr 172.21.15.78:0/1200272459 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:54.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.501+0000 7fc8937fe700 1 --2- 172.21.15.78:0/1200272459 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc89415a720 0x7fc8941497d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:54.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.501+0000 7fc8927fc700 1 --2- 172.21.15.78:0/1200272459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc894150580 0x7fc8941520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:54.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.501+0000 7fc892ffd700 1 -- 172.21.15.78:0/1200272459 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc89415a720 msgr2=0x7fc8941497d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:54.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.501+0000 7fc892ffd700 1 --2- 172.21.15.78:0/1200272459 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc89415a720 0x7fc8941497d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:54.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.501+0000 7fc892ffd700 1 -- 172.21.15.78:0/1200272459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc894150580 msgr2=0x7fc8941520d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:54.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.501+0000 7fc892ffd700 1 --2- 172.21.15.78:0/1200272459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc894150580 0x7fc8941520d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:54.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.501+0000 7fc892ffd700 1 -- 172.21.15.78:0/1200272459 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc88c009a70 con 0x7fc8940b23e0 2023-10-11T09:41:54.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.501+0000 7fc892ffd700 1 --2- 172.21.15.78:0/1200272459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8940b23e0 0x7fc8941efa30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc88400c500 tx=0x7fc88400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:54.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.501+0000 7fc87bfff700 1 -- 172.21.15.78:0/1200272459 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8840052d0 con 0x7fc8940b23e0 2023-10-11T09:41:54.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.501+0000 7fc87bfff700 1 -- 172.21.15.78:0/1200272459 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc884005430 con 0x7fc8940b23e0 2023-10-11T09:41:54.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.501+0000 7fc87bfff700 1 -- 172.21.15.78:0/1200272459 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8840055a0 con 0x7fc8940b23e0 2023-10-11T09:41:54.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.501+0000 7fc899819700 1 -- 172.21.15.78:0/1200272459 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc894149d10 con 0x7fc8940b23e0 2023-10-11T09:41:54.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.505+0000 7fc899819700 1 -- 172.21.15.78:0/1200272459 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc894152b90 con 0x7fc8940b23e0 2023-10-11T09:41:54.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.505+0000 7fc899819700 1 -- 172.21.15.78:0/1200272459 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc894065d40 con 0x7fc8940b23e0 2023-10-11T09:41:54.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.505+0000 7fc87bfff700 1 -- 172.21.15.78:0/1200272459 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fc8840038e0 con 0x7fc8940b23e0 2023-10-11T09:41:54.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.509+0000 7fc87bfff700 1 --2- 172.21.15.78:0/1200272459 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc880065ed0 0x7fc880068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:54.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.509+0000 7fc87bfff700 1 -- 172.21.15.78:0/1200272459 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc884082990 con 0x7fc8940b23e0 2023-10-11T09:41:54.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.509+0000 7fc8927fc700 1 --2- 172.21.15.78:0/1200272459 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc880065ed0 0x7fc880068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:54.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.509+0000 7fc8927fc700 1 --2- 172.21.15.78:0/1200272459 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc880065ed0 0x7fc880068380 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc8940ced10 tx=0x7fc888009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:54.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.509+0000 7fc87bfff700 1 -- 172.21.15.78:0/1200272459 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fc884050320 con 0x7fc8940b23e0 2023-10-11T09:41:54.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.745+0000 7fc899819700 1 -- 172.21.15.78:0/1200272459 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fc8940cbdf0 con 0x7fc8940b23e0 2023-10-11T09:41:54.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.757+0000 7fc87bfff700 1 -- 172.21.15.78:0/1200272459 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc8840170a0 con 0x7fc8940b23e0 2023-10-11T09:41:54.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.757+0000 7fc899819700 1 -- 172.21.15.78:0/1200272459 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc880065ed0 msgr2=0x7fc880068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:54.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.757+0000 7fc899819700 1 --2- 172.21.15.78:0/1200272459 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc880065ed0 0x7fc880068380 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc8940ced10 tx=0x7fc888009040 comp rx=0 tx=0).stop 2023-10-11T09:41:54.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.757+0000 7fc899819700 1 -- 172.21.15.78:0/1200272459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8940b23e0 msgr2=0x7fc8941efa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:54.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.757+0000 7fc899819700 1 --2- 172.21.15.78:0/1200272459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8940b23e0 0x7fc8941efa30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc88400c500 tx=0x7fc88400c8c0 comp rx=0 tx=0).stop 2023-10-11T09:41:54.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.757+0000 7fc899819700 1 -- 172.21.15.78:0/1200272459 shutdown_connections 2023-10-11T09:41:54.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.757+0000 7fc899819700 1 --2- 172.21.15.78:0/1200272459 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc880065ed0 0x7fc880068380 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:54.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.757+0000 7fc899819700 1 --2- 172.21.15.78:0/1200272459 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc89415a720 0x7fc8941497d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:54.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.757+0000 7fc899819700 1 --2- 172.21.15.78:0/1200272459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc894150580 0x7fc8941520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:54.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.757+0000 7fc899819700 1 --2- 172.21.15.78:0/1200272459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8940b23e0 0x7fc8941efa30 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:54.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.757+0000 7fc899819700 1 -- 172.21.15.78:0/1200272459 >> 172.21.15.78:0/1200272459 conn(0x7fc8940730c0 msgr2=0x7fc89415cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:54.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.757+0000 7fc899819700 1 -- 172.21.15.78:0/1200272459 shutdown_connections 2023-10-11T09:41:54.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.757+0000 7fc899819700 1 -- 172.21.15.78:0/1200272459 wait complete. 2023-10-11T09:41:54.785 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-10-11T09:41:54.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.949+0000 7fe30359e700 1 Processor -- start 2023-10-11T09:41:54.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.949+0000 7fe30359e700 1 -- start start 2023-10-11T09:41:54.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.949+0000 7fe30359e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe30415b8e0 0x7fe30406d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:54.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.949+0000 7fe30359e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe30406d610 0x7fe3040ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:54.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.949+0000 7fe30359e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe30415a940 0x7fe3040ca780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:54.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.949+0000 7fe30359e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe30415e010 con 0x7fe30415b8e0 2023-10-11T09:41:54.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.949+0000 7fe30359e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe30415e280 con 0x7fe30415a940 2023-10-11T09:41:54.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.949+0000 7fe30359e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe30415e560 con 0x7fe30406d610 2023-10-11T09:41:54.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.953+0000 7fe30259c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe30415b8e0 0x7fe30406d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:54.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.953+0000 7fe30259c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe30415b8e0 0x7fe30406d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57046/0 (socket says 172.21.15.78:57046) 2023-10-11T09:41:54.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.953+0000 7fe30259c700 1 -- 172.21.15.78:0/1007261715 learned_addr learned my addr 172.21.15.78:0/1007261715 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:54.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.953+0000 7fe302d9d700 1 --2- 172.21.15.78:0/1007261715 >> v2:172.21.15.133:3300/0 conn(0x7fe30415a940 0x7fe3040ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:54.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.953+0000 7fe2fbfff700 1 --2- 172.21.15.78:0/1007261715 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe30406d610 0x7fe3040ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:54.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.953+0000 7fe30259c700 1 -- 172.21.15.78:0/1007261715 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe30406d610 msgr2=0x7fe3040ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:54.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.953+0000 7fe30259c700 1 --2- 172.21.15.78:0/1007261715 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe30406d610 0x7fe3040ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:54.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.953+0000 7fe30259c700 1 -- 172.21.15.78:0/1007261715 >> v2:172.21.15.133:3300/0 conn(0x7fe30415a940 msgr2=0x7fe3040ca780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:54.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.953+0000 7fe30259c700 1 --2- 172.21.15.78:0/1007261715 >> v2:172.21.15.133:3300/0 conn(0x7fe30415a940 0x7fe3040ca780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:54.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.953+0000 7fe30259c700 1 -- 172.21.15.78:0/1007261715 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe30415e840 con 0x7fe30415b8e0 2023-10-11T09:41:54.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.953+0000 7fe30259c700 1 --2- 172.21.15.78:0/1007261715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe30415b8e0 0x7fe30406d0d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe3040cf800 tx=0x7fe2f4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a45dd0c385162d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:54.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.953+0000 7fe301d9b700 1 -- 172.21.15.78:0/1007261715 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f4011830 con 0x7fe30415b8e0 2023-10-11T09:41:54.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.953+0000 7fe301d9b700 1 -- 172.21.15.78:0/1007261715 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f4011990 con 0x7fe30415b8e0 2023-10-11T09:41:54.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.953+0000 7fe301d9b700 1 -- 172.21.15.78:0/1007261715 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f4011d20 con 0x7fe30415b8e0 2023-10-11T09:41:54.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.953+0000 7fe30359e700 1 -- 172.21.15.78:0/1007261715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe30415b8e0 msgr2=0x7fe30406d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:54.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.953+0000 7fe30359e700 1 --2- 172.21.15.78:0/1007261715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe30415b8e0 0x7fe30406d0d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe3040cf800 tx=0x7fe2f4009bf0 comp rx=0 tx=0).stop 2023-10-11T09:41:54.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.953+0000 7fe30359e700 1 -- 172.21.15.78:0/1007261715 shutdown_connections 2023-10-11T09:41:54.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.953+0000 7fe30359e700 1 --2- 172.21.15.78:0/1007261715 >> v2:172.21.15.133:3300/0 conn(0x7fe30415a940 0x7fe3040ca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:54.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.953+0000 7fe30359e700 1 --2- 172.21.15.78:0/1007261715 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe30406d610 0x7fe3040ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:54.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.953+0000 7fe30359e700 1 --2- 172.21.15.78:0/1007261715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe30415b8e0 0x7fe30406d0d0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:54.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.953+0000 7fe30359e700 1 -- 172.21.15.78:0/1007261715 >> 172.21.15.78:0/1007261715 conn(0x7fe3040cea60 msgr2=0x7fe3040cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:54.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.953+0000 7fe30359e700 1 -- 172.21.15.78:0/1007261715 shutdown_connections 2023-10-11T09:41:54.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.953+0000 7fe30359e700 1 -- 172.21.15.78:0/1007261715 wait complete. 2023-10-11T09:41:54.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.957+0000 7fe30359e700 1 Processor -- start 2023-10-11T09:41:54.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.957+0000 7fe30359e700 1 -- start start 2023-10-11T09:41:54.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.957+0000 7fe30359e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe30406d610 0x7fe30413f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:54.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.957+0000 7fe30359e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe30415a940 0x7fe3040c9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:54.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.957+0000 7fe30359e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe30415b8e0 0x7fe3040af910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:54.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.957+0000 7fe30359e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3041f1b10 con 0x7fe30415b8e0 2023-10-11T09:41:54.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.957+0000 7fe30359e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30413fd40 con 0x7fe30406d610 2023-10-11T09:41:54.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.957+0000 7fe30359e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe304140020 con 0x7fe30415a940 2023-10-11T09:41:54.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.961+0000 7fe302d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe30415b8e0 0x7fe3040af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:54.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.961+0000 7fe302d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe30415b8e0 0x7fe3040af910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57060/0 (socket says 172.21.15.78:57060) 2023-10-11T09:41:54.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.961+0000 7fe302d9d700 1 -- 172.21.15.78:0/2547100493 learned_addr learned my addr 172.21.15.78:0/2547100493 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:54.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.961+0000 7fe30259c700 1 --2- 172.21.15.78:0/2547100493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe30406d610 0x7fe30413f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:54.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.961+0000 7fe2fbfff700 1 --2- 172.21.15.78:0/2547100493 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe30415a940 0x7fe3040c9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:54.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.961+0000 7fe302d9d700 1 -- 172.21.15.78:0/2547100493 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe30415a940 msgr2=0x7fe3040c9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:54.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.961+0000 7fe302d9d700 1 --2- 172.21.15.78:0/2547100493 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe30415a940 0x7fe3040c9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:54.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.961+0000 7fe302d9d700 1 -- 172.21.15.78:0/2547100493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe30406d610 msgr2=0x7fe30413f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:54.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.961+0000 7fe302d9d700 1 --2- 172.21.15.78:0/2547100493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe30406d610 0x7fe30413f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:54.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.961+0000 7fe302d9d700 1 -- 172.21.15.78:0/2547100493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f4009570 con 0x7fe30415b8e0 2023-10-11T09:41:54.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.961+0000 7fe302d9d700 1 --2- 172.21.15.78:0/2547100493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe30415b8e0 0x7fe3040af910 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc007e40 tx=0x7fe2fc00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:54.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.961+0000 7fe2faffd700 1 -- 172.21.15.78:0/2547100493 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc00eea0 con 0x7fe30415b8e0 2023-10-11T09:41:54.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.961+0000 7fe2faffd700 1 -- 172.21.15.78:0/2547100493 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2fc014420 con 0x7fe30415b8e0 2023-10-11T09:41:54.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.961+0000 7fe30359e700 1 -- 172.21.15.78:0/2547100493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe304140300 con 0x7fe30415b8e0 2023-10-11T09:41:54.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.961+0000 7fe2faffd700 1 -- 172.21.15.78:0/2547100493 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc0147b0 con 0x7fe30415b8e0 2023-10-11T09:41:54.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.961+0000 7fe30359e700 1 -- 172.21.15.78:0/2547100493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe30413f950 con 0x7fe30415b8e0 2023-10-11T09:41:54.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.961+0000 7fe30359e700 1 -- 172.21.15.78:0/2547100493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe30413fb10 con 0x7fe30415b8e0 2023-10-11T09:41:54.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.961+0000 7fe2faffd700 1 -- 172.21.15.78:0/2547100493 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fe2fc014910 con 0x7fe30415b8e0 2023-10-11T09:41:54.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.965+0000 7fe2faffd700 1 --2- 172.21.15.78:0/2547100493 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe2e4065f00 0x7fe2e40683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:54.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.965+0000 7fe2faffd700 1 -- 172.21.15.78:0/2547100493 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe2fc083bb0 con 0x7fe30415b8e0 2023-10-11T09:41:54.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.965+0000 7fe30259c700 1 --2- 172.21.15.78:0/2547100493 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe2e4065f00 0x7fe2e40683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:54.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.965+0000 7fe30259c700 1 --2- 172.21.15.78:0/2547100493 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe2e4065f00 0x7fe2e40683b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4000c00 tx=0x7fe2f4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:54.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:54.969+0000 7fe2faffd700 1 -- 172.21.15.78:0/2547100493 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fe2fc051550 con 0x7fe30415b8e0 2023-10-11T09:41:55.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.205+0000 7fe30359e700 1 -- 172.21.15.78:0/2547100493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fe304065d40 con 0x7fe30415b8e0 2023-10-11T09:41:55.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.213+0000 7fe2faffd700 1 -- 172.21.15.78:0/2547100493 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe2fc054b70 con 0x7fe30415b8e0 2023-10-11T09:41:55.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.217+0000 7fe30359e700 1 -- 172.21.15.78:0/2547100493 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe2e4065f00 msgr2=0x7fe2e40683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:55.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.217+0000 7fe30359e700 1 --2- 172.21.15.78:0/2547100493 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe2e4065f00 0x7fe2e40683b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4000c00 tx=0x7fe2f4008030 comp rx=0 tx=0).stop 2023-10-11T09:41:55.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.217+0000 7fe30359e700 1 -- 172.21.15.78:0/2547100493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe30415b8e0 msgr2=0x7fe3040af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:55.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.217+0000 7fe30359e700 1 --2- 172.21.15.78:0/2547100493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe30415b8e0 0x7fe3040af910 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc007e40 tx=0x7fe2fc00e3c0 comp rx=0 tx=0).stop 2023-10-11T09:41:55.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.217+0000 7fe30359e700 1 -- 172.21.15.78:0/2547100493 shutdown_connections 2023-10-11T09:41:55.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.217+0000 7fe30359e700 1 --2- 172.21.15.78:0/2547100493 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe2e4065f00 0x7fe2e40683b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:55.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.217+0000 7fe30359e700 1 --2- 172.21.15.78:0/2547100493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe30415b8e0 0x7fe3040af910 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:55.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.217+0000 7fe30359e700 1 --2- 172.21.15.78:0/2547100493 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe30415a940 0x7fe3040c9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:55.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.217+0000 7fe30359e700 1 --2- 172.21.15.78:0/2547100493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe30406d610 0x7fe30413f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:55.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.217+0000 7fe30359e700 1 -- 172.21.15.78:0/2547100493 >> 172.21.15.78:0/2547100493 conn(0x7fe3040cea60 msgr2=0x7fe3040b0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:55.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.217+0000 7fe30359e700 1 -- 172.21.15.78:0/2547100493 shutdown_connections 2023-10-11T09:41:55.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.217+0000 7fe30359e700 1 -- 172.21.15.78:0/2547100493 wait complete. 2023-10-11T09:41:55.245 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:41:55.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.409+0000 7f10fc819700 1 Processor -- start 2023-10-11T09:41:55.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.409+0000 7f10fc819700 1 -- start start 2023-10-11T09:41:55.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.413+0000 7f10fc819700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10f80ca950 0x7f10f81650f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:55.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.413+0000 7f10fc819700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10f815e010 con 0x7f10f815a940 2023-10-11T09:41:55.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.413+0000 7f10fc819700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10f815e280 con 0x7f10f815b8e0 2023-10-11T09:41:55.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.413+0000 7f10fc819700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f815e560 con 0x7f10f80ca950 2023-10-11T09:41:55.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.413+0000 7f10f6ffd700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f10f815a940 0x7f10f80ca240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:58424/0 (socket says 172.21.15.78:58424) 2023-10-11T09:41:55.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.413+0000 7f10f6ffd700 1 -- 172.21.15.78:0/1091043139 learned_addr learned my addr 172.21.15.78:0/1091043139 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:55.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.413+0000 7f10f7fff700 1 --2- 172.21.15.78:0/1091043139 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10f80ca950 0x7f10f81650f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:55.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.413+0000 7f10f67fc700 1 -- 172.21.15.78:0/1091043139 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2904794858 0 0) 0x7f10f815e010 con 0x7f10f815a940 2023-10-11T09:41:55.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.413+0000 7f10f67fc700 1 -- 172.21.15.78:0/1091043139 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10e00035d0 con 0x7f10f815a940 2023-10-11T09:41:55.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.413+0000 7f10f67fc700 1 -- 172.21.15.78:0/1091043139 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2252086905 0 0) 0x7f10e00035d0 con 0x7f10f815a940 2023-10-11T09:41:55.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.413+0000 7f10f67fc700 1 -- 172.21.15.78:0/1091043139 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10f80ca950 msgr2=0x7f10f81650f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:55.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.413+0000 7f10f67fc700 1 --2- 172.21.15.78:0/1091043139 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10f80ca950 0x7f10f81650f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:55.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.413+0000 7f10f67fc700 1 -- 172.21.15.78:0/1091043139 >> v1:172.21.15.133:6789/0 conn(0x7f10f815b8e0 legacy=0x7f10f806d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:55.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.413+0000 7f10f67fc700 1 -- 172.21.15.78:0/1091043139 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f815e840 con 0x7f10f815a940 2023-10-11T09:41:55.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.413+0000 7f10f67fc700 1 -- 172.21.15.78:0/1091043139 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f10ec001670 con 0x7f10f815a940 2023-10-11T09:41:55.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.413+0000 7f10f67fc700 1 -- 172.21.15.78:0/1091043139 >> v1:172.21.15.78:6789/0 conn(0x7f10f815a940 legacy=0x7f10f80ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:55.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.413+0000 7f10f67fc700 1 --2- 172.21.15.78:0/1091043139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10e0003a50 0x7f10e0003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:55.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.413+0000 7f10f67fc700 1 --2- 172.21.15.78:0/1091043139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10e0004870 0x7f10e0004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:55.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.413+0000 7f10f67fc700 1 --2- 172.21.15.78:0/1091043139 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10f80ca950 0x7f10e0005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:55.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.413+0000 7f10f67fc700 1 -- 172.21.15.78:0/1091043139 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f815e560 con 0x7f10e0004870 2023-10-11T09:41:55.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.413+0000 7f10f67fc700 1 -- 172.21.15.78:0/1091043139 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e000bb10 con 0x7f10e0003a50 2023-10-11T09:41:55.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.413+0000 7f10f67fc700 1 -- 172.21.15.78:0/1091043139 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10e000bcd0 con 0x7f10f80ca950 2023-10-11T09:41:55.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.413+0000 7f10f6ffd700 1 --2- 172.21.15.78:0/1091043139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10e0004870 0x7f10e0004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:55.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.413+0000 7f10f77fe700 1 --2- 172.21.15.78:0/1091043139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10e0003a50 0x7f10e0003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:55.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.413+0000 7f10f7fff700 1 --2- 172.21.15.78:0/1091043139 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10f80ca950 0x7f10e0005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:55.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.413+0000 7f10f6ffd700 1 -- 172.21.15.78:0/1091043139 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10f80ca950 msgr2=0x7f10e0005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:55.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.413+0000 7f10f6ffd700 1 --2- 172.21.15.78:0/1091043139 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10f80ca950 0x7f10e0005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:55.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.413+0000 7f10f6ffd700 1 -- 172.21.15.78:0/1091043139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10e0003a50 msgr2=0x7f10e0003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:55.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.413+0000 7f10f6ffd700 1 --2- 172.21.15.78:0/1091043139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10e0003a50 0x7f10e0003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:55.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.413+0000 7f10f6ffd700 1 -- 172.21.15.78:0/1091043139 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e00035d0 con 0x7f10e0004870 2023-10-11T09:41:55.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.413+0000 7f10f6ffd700 1 --2- 172.21.15.78:0/1091043139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10e0004870 0x7f10e0004ce0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f10ec00c210 tx=0x7f10e00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=612f2353b6b59c91 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:55.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.417+0000 7f10f67fc700 1 -- 172.21.15.78:0/1091043139 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ec007b90 con 0x7f10e0004870 2023-10-11T09:41:55.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.417+0000 7f10f67fc700 1 -- 172.21.15.78:0/1091043139 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10ec00cd70 con 0x7f10e0004870 2023-10-11T09:41:55.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.417+0000 7f10f67fc700 1 -- 172.21.15.78:0/1091043139 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ec005bc0 con 0x7f10e0004870 2023-10-11T09:41:55.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.417+0000 7f10fc819700 1 -- 172.21.15.78:0/1091043139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10e0004870 msgr2=0x7f10e0004ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:55.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.417+0000 7f10fc819700 1 --2- 172.21.15.78:0/1091043139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10e0004870 0x7f10e0004ce0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f10ec00c210 tx=0x7f10e00058a0 comp rx=0 tx=0).stop 2023-10-11T09:41:55.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.417+0000 7f10f7fff700 1 -- 172.21.15.78:0/1091043139 reap_dead start 2023-10-11T09:41:55.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.417+0000 7f10fc819700 1 -- 172.21.15.78:0/1091043139 shutdown_connections 2023-10-11T09:41:55.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.417+0000 7f10fc819700 1 -- 172.21.15.78:0/1091043139 >> 172.21.15.78:0/1091043139 conn(0x7f10f80cea60 msgr2=0x7f10f80cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:55.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.417+0000 7f10fc819700 1 -- 172.21.15.78:0/1091043139 shutdown_connections 2023-10-11T09:41:55.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.417+0000 7f10fc819700 1 -- 172.21.15.78:0/1091043139 wait complete. 2023-10-11T09:41:55.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.421+0000 7f10fc819700 1 Processor -- start 2023-10-11T09:41:55.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.421+0000 7f10fc819700 1 -- start start 2023-10-11T09:41:55.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.421+0000 7f10fc819700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10e0004870 0x7f10f80b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:55.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.421+0000 7f10fc819700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10f80ce0b0 0x7f10f8159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:55.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.421+0000 7f10fc819700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10f81f5530 0x7f10f81f59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:55.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.421+0000 7f10fc819700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f81f5ef0 con 0x7f10e0004870 2023-10-11T09:41:55.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.421+0000 7f10fc819700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f80b43c0 con 0x7f10f80ce0b0 2023-10-11T09:41:55.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.421+0000 7f10fc819700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f80b3320 con 0x7f10f81f5530 2023-10-11T09:41:55.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.421+0000 7f10f77fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10e0004870 0x7f10f80b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:55.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.421+0000 7f10f77fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10e0004870 0x7f10f80b2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57064/0 (socket says 172.21.15.78:57064) 2023-10-11T09:41:55.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.421+0000 7f10f77fe700 1 -- 172.21.15.78:0/1658560731 learned_addr learned my addr 172.21.15.78:0/1658560731 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:55.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.421+0000 7f10f6ffd700 1 --2- 172.21.15.78:0/1658560731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10f80ce0b0 0x7f10f8159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:55.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.421+0000 7f10f7fff700 1 --2- 172.21.15.78:0/1658560731 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10f81f5530 0x7f10f81f59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:55.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.421+0000 7f10f77fe700 1 -- 172.21.15.78:0/1658560731 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10f81f5530 msgr2=0x7f10f81f59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:55.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.421+0000 7f10f77fe700 1 --2- 172.21.15.78:0/1658560731 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10f81f5530 0x7f10f81f59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:55.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.421+0000 7f10f77fe700 1 -- 172.21.15.78:0/1658560731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10f80ce0b0 msgr2=0x7f10f8159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:55.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.421+0000 7f10f77fe700 1 --2- 172.21.15.78:0/1658560731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10f80ce0b0 0x7f10f8159cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:55.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.421+0000 7f10f77fe700 1 -- 172.21.15.78:0/1658560731 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f80b3600 con 0x7f10e0004870 2023-10-11T09:41:55.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.421+0000 7f10f77fe700 1 --2- 172.21.15.78:0/1658560731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10e0004870 0x7f10f80b2a60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f10e800e4b0 tx=0x7f10e800e7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:55.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.421+0000 7f10dffff700 1 -- 172.21.15.78:0/1658560731 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e80052e0 con 0x7f10e0004870 2023-10-11T09:41:55.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.425+0000 7f10dffff700 1 -- 172.21.15.78:0/1658560731 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e8005440 con 0x7f10e0004870 2023-10-11T09:41:55.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.425+0000 7f10dffff700 1 -- 172.21.15.78:0/1658560731 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e80038a0 con 0x7f10e0004870 2023-10-11T09:41:55.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.425+0000 7f10fc819700 1 -- 172.21.15.78:0/1658560731 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f80b38e0 con 0x7f10e0004870 2023-10-11T09:41:55.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.425+0000 7f10fc819700 1 -- 172.21.15.78:0/1658560731 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f80b3ea0 con 0x7f10e0004870 2023-10-11T09:41:55.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.425+0000 7f10fc819700 1 -- 172.21.15.78:0/1658560731 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f80b4030 con 0x7f10e0004870 2023-10-11T09:41:55.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.425+0000 7f10dffff700 1 -- 172.21.15.78:0/1658560731 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f10e8003a00 con 0x7f10e0004870 2023-10-11T09:41:55.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.429+0000 7f10dffff700 1 --2- 172.21.15.78:0/1658560731 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f10e4065f00 0x7f10e40683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:55.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.429+0000 7f10dffff700 1 -- 172.21.15.78:0/1658560731 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f10e8082a40 con 0x7f10e0004870 2023-10-11T09:41:55.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.429+0000 7f10f6ffd700 1 --2- 172.21.15.78:0/1658560731 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f10e4065f00 0x7f10e40683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:55.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.429+0000 7f10f6ffd700 1 --2- 172.21.15.78:0/1658560731 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f10e4065f00 0x7f10e40683b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f10ec015040 tx=0x7f10ec007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:55.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.433+0000 7f10dffff700 1 -- 172.21.15.78:0/1658560731 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f10e8084e60 con 0x7f10e0004870 2023-10-11T09:41:55.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.609+0000 7f10fc819700 1 -- 172.21.15.78:0/1658560731 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f10f8143d20 con 0x7f10e0004870 2023-10-11T09:41:55.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.609+0000 7f10dffff700 1 -- 172.21.15.78:0/1658560731 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+14171 (secure 0 0 0) 0x7f10e8050410 con 0x7f10e0004870 2023-10-11T09:41:55.617 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:41:55.617 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":33,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:41:51.949249+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:41:55.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.617+0000 7f10fc819700 1 -- 172.21.15.78:0/1658560731 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f10e4065f00 msgr2=0x7f10e40683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:55.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.617+0000 7f10fc819700 1 --2- 172.21.15.78:0/1658560731 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f10e4065f00 0x7f10e40683b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f10ec015040 tx=0x7f10ec007000 comp rx=0 tx=0).stop 2023-10-11T09:41:55.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.617+0000 7f10fc819700 1 -- 172.21.15.78:0/1658560731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10e0004870 msgr2=0x7f10f80b2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:55.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.617+0000 7f10fc819700 1 --2- 172.21.15.78:0/1658560731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10e0004870 0x7f10f80b2a60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f10e800e4b0 tx=0x7f10e800e7c0 comp rx=0 tx=0).stop 2023-10-11T09:41:55.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.617+0000 7f10fc819700 1 -- 172.21.15.78:0/1658560731 shutdown_connections 2023-10-11T09:41:55.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.617+0000 7f10fc819700 1 --2- 172.21.15.78:0/1658560731 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f10e4065f00 0x7f10e40683b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:55.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.617+0000 7f10fc819700 1 --2- 172.21.15.78:0/1658560731 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10f81f5530 0x7f10f81f59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:55.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.617+0000 7f10fc819700 1 --2- 172.21.15.78:0/1658560731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10f80ce0b0 0x7f10f8159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:55.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.617+0000 7f10fc819700 1 --2- 172.21.15.78:0/1658560731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10e0004870 0x7f10f80b2a60 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:55.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.617+0000 7f10fc819700 1 -- 172.21.15.78:0/1658560731 >> 172.21.15.78:0/1658560731 conn(0x7f10f80cea60 msgr2=0x7f10f8142d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:55.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.617+0000 7f10fc819700 1 -- 172.21.15.78:0/1658560731 shutdown_connections 2023-10-11T09:41:55.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.617+0000 7f10fc819700 1 -- 172.21.15.78:0/1658560731 wait complete. 2023-10-11T09:41:55.641 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:41:55.642 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:41:55.819 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.813+0000 7f2e3f59e700 1 Processor -- start 2023-10-11T09:41:55.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.813+0000 7f2e3f59e700 1 -- start start 2023-10-11T09:41:55.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.817+0000 7f2e3f59e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2e4015a6b0 0x7f2e400ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:55.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.817+0000 7f2e3f59e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f2e4006d730 0x7f2e4006bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:55.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.817+0000 7f2e3f59e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e400b2d30 con 0x7f2e400b3a60 2023-10-11T09:41:55.824 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.817+0000 7f2e3f59e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e400cf360 con 0x7f2e4006d730 2023-10-11T09:41:55.824 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.817+0000 7f2e3f59e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e400b2f80 con 0x7f2e4015a6b0 2023-10-11T09:41:55.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.817+0000 7f2e35d9b700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f2e400b3a60 0x7f2e4006d020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:58428/0 (socket says 172.21.15.78:58428) 2023-10-11T09:41:55.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.817+0000 7f2e35d9b700 1 -- 172.21.15.78:0/3726597927 learned_addr learned my addr 172.21.15.78:0/3726597927 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:55.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.817+0000 7f2e3ed9d700 1 --2- 172.21.15.78:0/3726597927 >> v2:172.21.15.78:3300/0 conn(0x7f2e4006d730 0x7f2e4006bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:55.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.817+0000 7f2e3e59c700 1 --2- 172.21.15.78:0/3726597927 >> v2:172.21.15.133:3300/0 conn(0x7f2e4015a6b0 0x7f2e400ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:55.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.817+0000 7f2e3dd9b700 1 -- 172.21.15.78:0/3726597927 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4134871691 0 0) 0x7f2e400b2d30 con 0x7f2e400b3a60 2023-10-11T09:41:55.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.817+0000 7f2e3dd9b700 1 -- 172.21.15.78:0/3726597927 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e200035d0 con 0x7f2e400b3a60 2023-10-11T09:41:55.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.821+0000 7f2e3ed9d700 1 -- 172.21.15.78:0/3726597927 >> v2:172.21.15.133:3300/0 conn(0x7f2e4015a6b0 msgr2=0x7f2e400ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:55.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.821+0000 7f2e3ed9d700 1 --2- 172.21.15.78:0/3726597927 >> v2:172.21.15.133:3300/0 conn(0x7f2e4015a6b0 0x7f2e400ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:55.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.821+0000 7f2e3ed9d700 1 -- 172.21.15.78:0/3726597927 >> v1:172.21.15.78:6789/0 conn(0x7f2e400b3a60 legacy=0x7f2e4006d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:55.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.821+0000 7f2e3ed9d700 1 -- 172.21.15.78:0/3726597927 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e400b3260 con 0x7f2e4006d730 2023-10-11T09:41:55.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.821+0000 7f2e3ed9d700 1 --2- 172.21.15.78:0/3726597927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e4006d730 0x7f2e4006bbe0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2e40072550 tx=0x7f2e3800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6edbf9979c506640 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:55.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.821+0000 7f2e3dd9b700 1 -- 172.21.15.78:0/3726597927 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3800e070 con 0x7f2e4006d730 2023-10-11T09:41:55.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.821+0000 7f2e3dd9b700 1 -- 172.21.15.78:0/3726597927 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e3800bb20 con 0x7f2e4006d730 2023-10-11T09:41:55.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.821+0000 7f2e3dd9b700 1 -- 172.21.15.78:0/3726597927 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e38012430 con 0x7f2e4006d730 2023-10-11T09:41:55.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.821+0000 7f2e3f59e700 1 -- 172.21.15.78:0/3726597927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e4006d730 msgr2=0x7f2e4006bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:55.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.821+0000 7f2e3f59e700 1 --2- 172.21.15.78:0/3726597927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e4006d730 0x7f2e4006bbe0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2e40072550 tx=0x7f2e3800b330 comp rx=0 tx=0).stop 2023-10-11T09:41:55.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.821+0000 7f2e3f59e700 1 -- 172.21.15.78:0/3726597927 shutdown_connections 2023-10-11T09:41:55.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.821+0000 7f2e3f59e700 1 --2- 172.21.15.78:0/3726597927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e4006d730 0x7f2e4006bbe0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:55.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.821+0000 7f2e3f59e700 1 --2- 172.21.15.78:0/3726597927 >> v2:172.21.15.133:3300/0 conn(0x7f2e4015a6b0 0x7f2e400ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:55.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.821+0000 7f2e3f59e700 1 -- 172.21.15.78:0/3726597927 >> 172.21.15.78:0/3726597927 conn(0x7f2e4006a060 msgr2=0x7f2e4006a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:55.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.821+0000 7f2e3f59e700 1 -- 172.21.15.78:0/3726597927 shutdown_connections 2023-10-11T09:41:55.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.821+0000 7f2e3f59e700 1 -- 172.21.15.78:0/3726597927 wait complete. 2023-10-11T09:41:55.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.825+0000 7f2e3f59e700 1 Processor -- start 2023-10-11T09:41:55.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.825+0000 7f2e3f59e700 1 -- start start 2023-10-11T09:41:55.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.825+0000 7f2e3f59e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2e4006d730 0x7f2e400cac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:55.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.825+0000 7f2e3f59e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e400b3a60 0x7f2e4014c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:55.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.825+0000 7f2e3f59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e4015a6b0 0x7f2e400af770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:55.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.825+0000 7f2e3f59e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4014cdd0 con 0x7f2e4015a6b0 2023-10-11T09:41:55.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.825+0000 7f2e3f59e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4015d260 con 0x7f2e400b3a60 2023-10-11T09:41:55.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.825+0000 7f2e3f59e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4015d540 con 0x7f2e4006d730 2023-10-11T09:41:55.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.829+0000 7f2e3ed9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e4015a6b0 0x7f2e400af770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:55.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.829+0000 7f2e3ed9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e4015a6b0 0x7f2e400af770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57078/0 (socket says 172.21.15.78:57078) 2023-10-11T09:41:55.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.829+0000 7f2e3ed9d700 1 -- 172.21.15.78:0/3831832396 learned_addr learned my addr 172.21.15.78:0/3831832396 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:55.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.829+0000 7f2e3e59c700 1 --2- 172.21.15.78:0/3831832396 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2e4006d730 0x7f2e400cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:55.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.829+0000 7f2e35d9b700 1 --2- 172.21.15.78:0/3831832396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e400b3a60 0x7f2e4014c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:55.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.829+0000 7f2e3ed9d700 1 -- 172.21.15.78:0/3831832396 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2e4006d730 msgr2=0x7f2e400cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:55.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.829+0000 7f2e3ed9d700 1 --2- 172.21.15.78:0/3831832396 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2e4006d730 0x7f2e400cac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:55.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.829+0000 7f2e3ed9d700 1 -- 172.21.15.78:0/3831832396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e400b3a60 msgr2=0x7f2e4014c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:55.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.829+0000 7f2e3ed9d700 1 --2- 172.21.15.78:0/3831832396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e400b3a60 0x7f2e4014c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:55.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.829+0000 7f2e3ed9d700 1 -- 172.21.15.78:0/3831832396 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e38009a70 con 0x7f2e4015a6b0 2023-10-11T09:41:55.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.829+0000 7f2e3ed9d700 1 --2- 172.21.15.78:0/3831832396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e4015a6b0 0x7f2e400af770 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2e38014040 tx=0x7f2e38004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:55.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.829+0000 7f2e377fe700 1 -- 172.21.15.78:0/3831832396 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e38012430 con 0x7f2e4015a6b0 2023-10-11T09:41:55.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.829+0000 7f2e377fe700 1 -- 172.21.15.78:0/3831832396 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e38020070 con 0x7f2e4015a6b0 2023-10-11T09:41:55.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.829+0000 7f2e377fe700 1 -- 172.21.15.78:0/3831832396 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e380098f0 con 0x7f2e4015a6b0 2023-10-11T09:41:55.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.829+0000 7f2e3f59e700 1 -- 172.21.15.78:0/3831832396 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e4015d820 con 0x7f2e4015a6b0 2023-10-11T09:41:55.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.829+0000 7f2e3f59e700 1 -- 172.21.15.78:0/3831832396 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e4014e240 con 0x7f2e4015a6b0 2023-10-11T09:41:55.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.829+0000 7f2e3f59e700 1 -- 172.21.15.78:0/3831832396 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e4015dd40 con 0x7f2e4015a6b0 2023-10-11T09:41:55.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.833+0000 7f2e377fe700 1 -- 172.21.15.78:0/3831832396 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f2e38004040 con 0x7f2e4015a6b0 2023-10-11T09:41:55.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.833+0000 7f2e377fe700 1 --2- 172.21.15.78:0/3831832396 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2e24065f00 0x7f2e240683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:55.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.833+0000 7f2e377fe700 1 -- 172.21.15.78:0/3831832396 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2e38022070 con 0x7f2e4015a6b0 2023-10-11T09:41:55.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.833+0000 7f2e3e59c700 1 --2- 172.21.15.78:0/3831832396 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2e24065f00 0x7f2e240683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:55.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.837+0000 7f2e3e59c700 1 --2- 172.21.15.78:0/3831832396 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2e24065f00 0x7f2e240683b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2e30009b50 tx=0x7f2e30009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:55.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:55.841+0000 7f2e377fe700 1 -- 172.21.15.78:0/3831832396 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f2e38056380 con 0x7f2e4015a6b0 2023-10-11T09:41:56.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.013+0000 7f2e3f59e700 1 -- 172.21.15.78:0/3831832396 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2e40065d40 con 0x7f2e4015a6b0 2023-10-11T09:41:56.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.017+0000 7f2e377fe700 1 -- 172.21.15.78:0/3831832396 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2e3802c020 con 0x7f2e4015a6b0 2023-10-11T09:41:56.021 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:41:56.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.017+0000 7f2e3f59e700 1 -- 172.21.15.78:0/3831832396 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2e24065f00 msgr2=0x7f2e240683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:56.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.017+0000 7f2e3f59e700 1 --2- 172.21.15.78:0/3831832396 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2e24065f00 0x7f2e240683b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2e30009b50 tx=0x7f2e30009240 comp rx=0 tx=0).stop 2023-10-11T09:41:56.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.017+0000 7f2e3f59e700 1 -- 172.21.15.78:0/3831832396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e4015a6b0 msgr2=0x7f2e400af770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:56.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.017+0000 7f2e3f59e700 1 --2- 172.21.15.78:0/3831832396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e4015a6b0 0x7f2e400af770 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2e38014040 tx=0x7f2e38004af0 comp rx=0 tx=0).stop 2023-10-11T09:41:56.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.017+0000 7f2e3f59e700 1 -- 172.21.15.78:0/3831832396 shutdown_connections 2023-10-11T09:41:56.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.017+0000 7f2e3f59e700 1 --2- 172.21.15.78:0/3831832396 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2e24065f00 0x7f2e240683b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:56.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.017+0000 7f2e3f59e700 1 --2- 172.21.15.78:0/3831832396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e4015a6b0 0x7f2e400af770 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:56.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.017+0000 7f2e3f59e700 1 --2- 172.21.15.78:0/3831832396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e400b3a60 0x7f2e4014c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:56.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.017+0000 7f2e3f59e700 1 --2- 172.21.15.78:0/3831832396 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2e4006d730 0x7f2e400cac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:56.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.017+0000 7f2e3f59e700 1 -- 172.21.15.78:0/3831832396 >> 172.21.15.78:0/3831832396 conn(0x7f2e4006a060 msgr2=0x7f2e400cc620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:56.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.017+0000 7f2e3f59e700 1 -- 172.21.15.78:0/3831832396 shutdown_connections 2023-10-11T09:41:56.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.017+0000 7f2e3f59e700 1 -- 172.21.15.78:0/3831832396 wait complete. 2023-10-11T09:41:56.045 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:41:56.218 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.213+0000 7f18fd57f700 1 Processor -- start 2023-10-11T09:41:56.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.213+0000 7f18fd57f700 1 -- start start 2023-10-11T09:41:56.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.217+0000 7f18fd57f700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f18f815c900 0x7f18f80b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:56.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.217+0000 7f18fd57f700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f18f80ad290 0x7f18f8071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:56.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.217+0000 7f18fd57f700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18f806fde0 con 0x7f18f80ce7f0 2023-10-11T09:41:56.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.217+0000 7f18fd57f700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18f80705f0 con 0x7f18f815c900 2023-10-11T09:41:56.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.217+0000 7f18fd57f700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f8071fc0 con 0x7f18f80ad290 2023-10-11T09:41:56.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.217+0000 7f18f67fc700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f18f80ad290 0x7f18f8071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:56.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.217+0000 7f18f6ffd700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f18f815c900 0x7f18f80b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:56.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.217+0000 7f18f67fc700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f18f80ad290 0x7f18f8071a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:33066/0 (socket says 172.21.15.78:33066) 2023-10-11T09:41:56.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.217+0000 7f18f67fc700 1 -- 172.21.15.78:0/1773613758 learned_addr learned my addr 172.21.15.78:0/1773613758 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:56.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.217+0000 7f18f67fc700 1 -- 172.21.15.78:0/1773613758 >> v2:172.21.15.133:3300/0 conn(0x7f18f815c900 msgr2=0x7f18f80b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:56.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.217+0000 7f18f67fc700 1 --2- 172.21.15.78:0/1773613758 >> v2:172.21.15.133:3300/0 conn(0x7f18f815c900 0x7f18f80b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:56.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.217+0000 7f18f67fc700 1 -- 172.21.15.78:0/1773613758 >> v1:172.21.15.133:6789/0 conn(0x7f18f80ce7f0 legacy=0x7f18f806bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:56.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.217+0000 7f18f67fc700 1 -- 172.21.15.78:0/1773613758 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f80ae000 con 0x7f18f80ad290 2023-10-11T09:41:56.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.217+0000 7f18f6ffd700 1 --2- 172.21.15.78:0/1773613758 >> v2:172.21.15.133:3300/0 conn(0x7f18f815c900 0x7f18f80b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:41:56.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.217+0000 7f18f5ffb700 1 -- 172.21.15.78:0/1773613758 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3449896992 0 0) 0x7f18f806fde0 con 0x7f18f80ce7f0 2023-10-11T09:41:56.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.217+0000 7f18f67fc700 1 --2- 172.21.15.78:0/1773613758 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f18f80ad290 0x7f18f8071a80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f18ec009940 tx=0x7f18ec009c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8de9b3751c651bee server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:56.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.221+0000 7f18f5ffb700 1 -- 172.21.15.78:0/1773613758 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ec011830 con 0x7f18f80ad290 2023-10-11T09:41:56.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.221+0000 7f18f5ffb700 1 -- 172.21.15.78:0/1773613758 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18ec011990 con 0x7f18f80ad290 2023-10-11T09:41:56.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.221+0000 7f18f5ffb700 1 -- 172.21.15.78:0/1773613758 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ec011d20 con 0x7f18f80ad290 2023-10-11T09:41:56.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.221+0000 7f18fd57f700 1 -- 172.21.15.78:0/1773613758 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f18f80ad290 msgr2=0x7f18f8071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:56.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.221+0000 7f18fd57f700 1 --2- 172.21.15.78:0/1773613758 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f18f80ad290 0x7f18f8071a80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f18ec009940 tx=0x7f18ec009c50 comp rx=0 tx=0).stop 2023-10-11T09:41:56.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.221+0000 7f18fd57f700 1 -- 172.21.15.78:0/1773613758 shutdown_connections 2023-10-11T09:41:56.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.221+0000 7f18fd57f700 1 --2- 172.21.15.78:0/1773613758 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f18f80ad290 0x7f18f8071a80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:56.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.221+0000 7f18fd57f700 1 --2- 172.21.15.78:0/1773613758 >> v2:172.21.15.133:3300/0 conn(0x7f18f815c900 0x7f18f80b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:56.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.221+0000 7f18fd57f700 1 -- 172.21.15.78:0/1773613758 >> 172.21.15.78:0/1773613758 conn(0x7f18f806a060 msgr2=0x7f18f806a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:56.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.221+0000 7f18fd57f700 1 -- 172.21.15.78:0/1773613758 shutdown_connections 2023-10-11T09:41:56.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.221+0000 7f18fd57f700 1 -- 172.21.15.78:0/1773613758 wait complete. 2023-10-11T09:41:56.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.225+0000 7f18fd57f700 1 Processor -- start 2023-10-11T09:41:56.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.225+0000 7f18fd57f700 1 -- start start 2023-10-11T09:41:56.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.225+0000 7f18fd57f700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f18f80ad290 0x7f18f80b1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:56.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.225+0000 7f18fd57f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18f80ce7f0 0x7f18f80cc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:56.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.225+0000 7f18fd57f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18f815c900 0x7f18f80cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:56.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.225+0000 7f18fd57f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f81f95a0 con 0x7f18f815c900 2023-10-11T09:41:56.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.225+0000 7f18fd57f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f81f9e00 con 0x7f18f80ce7f0 2023-10-11T09:41:56.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.225+0000 7f18fd57f700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f81f6d30 con 0x7f18f80ad290 2023-10-11T09:41:56.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.225+0000 7f18f77fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18f815c900 0x7f18f80cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:56.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.225+0000 7f18f6ffd700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f18f80ad290 0x7f18f80b1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:56.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.225+0000 7f18f77fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18f815c900 0x7f18f80cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57080/0 (socket says 172.21.15.78:57080) 2023-10-11T09:41:56.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.225+0000 7f18f77fe700 1 -- 172.21.15.78:0/1565631241 learned_addr learned my addr 172.21.15.78:0/1565631241 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:56.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.225+0000 7f18f67fc700 1 --2- 172.21.15.78:0/1565631241 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18f80ce7f0 0x7f18f80cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:56.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.229+0000 7f18f6ffd700 1 -- 172.21.15.78:0/1565631241 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18f80ce7f0 msgr2=0x7f18f80cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:56.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.229+0000 7f18f6ffd700 1 --2- 172.21.15.78:0/1565631241 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18f80ce7f0 0x7f18f80cc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:56.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.229+0000 7f18f6ffd700 1 -- 172.21.15.78:0/1565631241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18f815c900 msgr2=0x7f18f80cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:56.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.229+0000 7f18f6ffd700 1 --2- 172.21.15.78:0/1565631241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18f815c900 0x7f18f80cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:56.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.229+0000 7f18f6ffd700 1 -- 172.21.15.78:0/1565631241 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e80094b0 con 0x7f18f80ad290 2023-10-11T09:41:56.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.229+0000 7f18f67fc700 1 --2- 172.21.15.78:0/1565631241 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18f80ce7f0 0x7f18f80cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:41:56.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.229+0000 7f18f6ffd700 1 --2- 172.21.15.78:0/1565631241 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f18f80ad290 0x7f18f80b1c80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f18e800e9b0 tx=0x7f18e800ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:56.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.229+0000 7f18df7fe700 1 -- 172.21.15.78:0/1565631241 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e800ee50 con 0x7f18f80ad290 2023-10-11T09:41:56.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.229+0000 7f18df7fe700 1 -- 172.21.15.78:0/1565631241 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e800c850 con 0x7f18f80ad290 2023-10-11T09:41:56.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.229+0000 7f18fd57f700 1 -- 172.21.15.78:0/1565631241 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18ec009570 con 0x7f18f80ad290 2023-10-11T09:41:56.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.229+0000 7f18df7fe700 1 -- 172.21.15.78:0/1565631241 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e800cc00 con 0x7f18f80ad290 2023-10-11T09:41:56.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.229+0000 7f18fd57f700 1 -- 172.21.15.78:0/1565631241 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18f81f9a90 con 0x7f18f80ad290 2023-10-11T09:41:56.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.229+0000 7f18fd57f700 1 -- 172.21.15.78:0/1565631241 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18f8065d40 con 0x7f18f80ad290 2023-10-11T09:41:56.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.233+0000 7f18df7fe700 1 -- 172.21.15.78:0/1565631241 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f18e8013470 con 0x7f18f80ad290 2023-10-11T09:41:56.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.233+0000 7f18df7fe700 1 --2- 172.21.15.78:0/1565631241 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f18e4065f20 0x7f18e40683d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:56.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.233+0000 7f18df7fe700 1 -- 172.21.15.78:0/1565631241 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f18e8083490 con 0x7f18f80ad290 2023-10-11T09:41:56.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.233+0000 7f18f67fc700 1 --2- 172.21.15.78:0/1565631241 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f18e4065f20 0x7f18e40683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:56.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.233+0000 7f18f67fc700 1 --2- 172.21.15.78:0/1565631241 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f18e4065f20 0x7f18e40683d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f18ec014040 tx=0x7f18ec008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:56.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.241+0000 7f18df7fe700 1 -- 172.21.15.78:0/1565631241 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f18e8050e10 con 0x7f18f80ad290 2023-10-11T09:41:56.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.413+0000 7f18fd57f700 1 -- 172.21.15.78:0/1565631241 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f18f80cf5b0 con 0x7f18f80ad290 2023-10-11T09:41:56.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.417+0000 7f18df7fe700 1 -- 172.21.15.78:0/1565631241 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v33) v1 ==== 92+0+10 (secure 0 0 0) 0x7f18e8054430 con 0x7f18f80ad290 2023-10-11T09:41:56.422 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:41:56.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.417+0000 7f18fd57f700 1 -- 172.21.15.78:0/1565631241 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f18e4065f20 msgr2=0x7f18e40683d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:56.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.417+0000 7f18fd57f700 1 --2- 172.21.15.78:0/1565631241 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f18e4065f20 0x7f18e40683d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f18ec014040 tx=0x7f18ec008030 comp rx=0 tx=0).stop 2023-10-11T09:41:56.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.417+0000 7f18fd57f700 1 -- 172.21.15.78:0/1565631241 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f18f80ad290 msgr2=0x7f18f80b1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:56.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.417+0000 7f18fd57f700 1 --2- 172.21.15.78:0/1565631241 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f18f80ad290 0x7f18f80b1c80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f18e800e9b0 tx=0x7f18e800ecc0 comp rx=0 tx=0).stop 2023-10-11T09:41:56.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.421+0000 7f18fd57f700 1 -- 172.21.15.78:0/1565631241 shutdown_connections 2023-10-11T09:41:56.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.421+0000 7f18fd57f700 1 --2- 172.21.15.78:0/1565631241 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f18e4065f20 0x7f18e40683d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:56.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.421+0000 7f18fd57f700 1 --2- 172.21.15.78:0/1565631241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18f815c900 0x7f18f80cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:56.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.421+0000 7f18fd57f700 1 --2- 172.21.15.78:0/1565631241 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18f80ce7f0 0x7f18f80cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:56.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.421+0000 7f18fd57f700 1 --2- 172.21.15.78:0/1565631241 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f18f80ad290 0x7f18f80b1c80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:56.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.421+0000 7f18fd57f700 1 -- 172.21.15.78:0/1565631241 >> 172.21.15.78:0/1565631241 conn(0x7f18f806a060 msgr2=0x7f18f8157800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:56.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.421+0000 7f18fd57f700 1 -- 172.21.15.78:0/1565631241 shutdown_connections 2023-10-11T09:41:56.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.421+0000 7f18fd57f700 1 -- 172.21.15.78:0/1565631241 wait complete. 2023-10-11T09:41:56.447 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-11T09:41:56.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.613+0000 7f9ae46c6700 1 Processor -- start 2023-10-11T09:41:56.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.613+0000 7f9ae46c6700 1 -- start start 2023-10-11T09:41:56.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.617+0000 7f9ae46c6700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9adc06fb90 0x7f9adc0cfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:56.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.617+0000 7f9ae46c6700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9adc160dc0 0x7f9adc15cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:56.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.617+0000 7f9ae46c6700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9adc0d0620 con 0x7f9adc06e1c0 2023-10-11T09:41:56.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.617+0000 7f9ae46c6700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9adc06f140 con 0x7f9adc06fb90 2023-10-11T09:41:56.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.617+0000 7f9ae46c6700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc160690 con 0x7f9adc160dc0 2023-10-11T09:41:56.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.617+0000 7f9ae2462700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f9adc06e1c0 0x7f9adc06e590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:58432/0 (socket says 172.21.15.78:58432) 2023-10-11T09:41:56.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.617+0000 7f9ae2462700 1 -- 172.21.15.78:0/1731995925 learned_addr learned my addr 172.21.15.78:0/1731995925 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:56.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.617+0000 7f9ae2c63700 1 --2- 172.21.15.78:0/1731995925 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9adc160dc0 0x7f9adc15cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:56.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.617+0000 7f9ae1c61700 1 --2- 172.21.15.78:0/1731995925 >> v2:172.21.15.133:3300/0 conn(0x7f9adc06fb90 0x7f9adc0cfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:56.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.617+0000 7f9ae1460700 1 -- 172.21.15.78:0/1731995925 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1801864009 0 0) 0x7f9adc0d0620 con 0x7f9adc06e1c0 2023-10-11T09:41:56.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.617+0000 7f9ae1460700 1 -- 172.21.15.78:0/1731995925 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ac80035d0 con 0x7f9adc06e1c0 2023-10-11T09:41:56.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.617+0000 7f9ae2c63700 1 -- 172.21.15.78:0/1731995925 >> v2:172.21.15.133:3300/0 conn(0x7f9adc06fb90 msgr2=0x7f9adc0cfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:56.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.617+0000 7f9ae2c63700 1 --2- 172.21.15.78:0/1731995925 >> v2:172.21.15.133:3300/0 conn(0x7f9adc06fb90 0x7f9adc0cfdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:56.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.617+0000 7f9ae2c63700 1 -- 172.21.15.78:0/1731995925 >> v1:172.21.15.78:6789/0 conn(0x7f9adc06e1c0 legacy=0x7f9adc06e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:56.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.617+0000 7f9ae2c63700 1 -- 172.21.15.78:0/1731995925 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9adc0d0900 con 0x7f9adc160dc0 2023-10-11T09:41:56.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.621+0000 7f9ae2c63700 1 --2- 172.21.15.78:0/1731995925 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9adc160dc0 0x7f9adc15cfa0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4009d90 tx=0x7f9ad400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28a3922af314ee6a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:56.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.621+0000 7f9ae1460700 1 -- 172.21.15.78:0/1731995925 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad400e070 con 0x7f9adc160dc0 2023-10-11T09:41:56.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.621+0000 7f9ae1460700 1 -- 172.21.15.78:0/1731995925 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad400bb20 con 0x7f9adc160dc0 2023-10-11T09:41:56.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.621+0000 7f9ae1460700 1 -- 172.21.15.78:0/1731995925 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4012430 con 0x7f9adc160dc0 2023-10-11T09:41:56.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.621+0000 7f9ae46c6700 1 -- 172.21.15.78:0/1731995925 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9adc160dc0 msgr2=0x7f9adc15cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:56.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.621+0000 7f9ae46c6700 1 --2- 172.21.15.78:0/1731995925 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9adc160dc0 0x7f9adc15cfa0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4009d90 tx=0x7f9ad400b330 comp rx=0 tx=0).stop 2023-10-11T09:41:56.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.621+0000 7f9ae46c6700 1 -- 172.21.15.78:0/1731995925 shutdown_connections 2023-10-11T09:41:56.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.621+0000 7f9ae46c6700 1 --2- 172.21.15.78:0/1731995925 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9adc160dc0 0x7f9adc15cfa0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:56.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.621+0000 7f9ae46c6700 1 --2- 172.21.15.78:0/1731995925 >> v2:172.21.15.133:3300/0 conn(0x7f9adc06fb90 0x7f9adc0cfdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:56.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.621+0000 7f9ae46c6700 1 -- 172.21.15.78:0/1731995925 >> 172.21.15.78:0/1731995925 conn(0x7f9adc0d0f70 msgr2=0x7f9adc0d1370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:56.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.621+0000 7f9ae46c6700 1 -- 172.21.15.78:0/1731995925 shutdown_connections 2023-10-11T09:41:56.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.621+0000 7f9ae46c6700 1 -- 172.21.15.78:0/1731995925 wait complete. 2023-10-11T09:41:56.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.625+0000 7f9ae46c6700 1 Processor -- start 2023-10-11T09:41:56.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.625+0000 7f9ae46c6700 1 -- start start 2023-10-11T09:41:56.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.625+0000 7f9ae46c6700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9adc06e1c0 0x7f9adc0af680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:56.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.625+0000 7f9ae46c6700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9adc06fb90 0x7f9adc1ef4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:56.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.625+0000 7f9ae46c6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9adc160dc0 0x7f9adc0ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:56.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.625+0000 7f9ae46c6700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc1f2ef0 con 0x7f9adc160dc0 2023-10-11T09:41:56.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.625+0000 7f9ae46c6700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc0ad7a0 con 0x7f9adc06e1c0 2023-10-11T09:41:56.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.625+0000 7f9ae46c6700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc1efa10 con 0x7f9adc06fb90 2023-10-11T09:41:56.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.625+0000 7f9ae2c63700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9adc160dc0 0x7f9adc0ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:56.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.625+0000 7f9ae2c63700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9adc160dc0 0x7f9adc0ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57086/0 (socket says 172.21.15.78:57086) 2023-10-11T09:41:56.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.625+0000 7f9ae2c63700 1 -- 172.21.15.78:0/2993713296 learned_addr learned my addr 172.21.15.78:0/2993713296 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:56.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.625+0000 7f9ae1c61700 1 --2- 172.21.15.78:0/2993713296 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9adc06fb90 0x7f9adc1ef4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:56.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.625+0000 7f9ae2c63700 1 -- 172.21.15.78:0/2993713296 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9adc06fb90 msgr2=0x7f9adc1ef4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:56.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.625+0000 7f9ae2462700 1 --2- 172.21.15.78:0/2993713296 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9adc06e1c0 0x7f9adc0af680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:56.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.625+0000 7f9ae2c63700 1 --2- 172.21.15.78:0/2993713296 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9adc06fb90 0x7f9adc1ef4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:56.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.625+0000 7f9ae2c63700 1 -- 172.21.15.78:0/2993713296 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9adc06e1c0 msgr2=0x7f9adc0af680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:56.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.625+0000 7f9ae2c63700 1 --2- 172.21.15.78:0/2993713296 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9adc06e1c0 0x7f9adc0af680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:56.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.625+0000 7f9ae2c63700 1 -- 172.21.15.78:0/2993713296 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad4009a70 con 0x7f9adc160dc0 2023-10-11T09:41:56.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.625+0000 7f9ae1c61700 1 --2- 172.21.15.78:0/2993713296 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9adc06fb90 0x7f9adc1ef4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:41:56.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.625+0000 7f9ae2462700 1 --2- 172.21.15.78:0/2993713296 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9adc06e1c0 0x7f9adc0af680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:41:56.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.625+0000 7f9ae2c63700 1 --2- 172.21.15.78:0/2993713296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9adc160dc0 0x7f9adc0ac8e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4014040 tx=0x7f9ad4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:56.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.625+0000 7f9ad2ffd700 1 -- 172.21.15.78:0/2993713296 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4012430 con 0x7f9adc160dc0 2023-10-11T09:41:56.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.625+0000 7f9ae46c6700 1 -- 172.21.15.78:0/2993713296 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9adc1f2270 con 0x7f9adc160dc0 2023-10-11T09:41:56.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.625+0000 7f9ad2ffd700 1 -- 172.21.15.78:0/2993713296 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad4020070 con 0x7f9adc160dc0 2023-10-11T09:41:56.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.625+0000 7f9ad2ffd700 1 -- 172.21.15.78:0/2993713296 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4004280 con 0x7f9adc160dc0 2023-10-11T09:41:56.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.625+0000 7f9ae46c6700 1 -- 172.21.15.78:0/2993713296 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9adc1f2830 con 0x7f9adc160dc0 2023-10-11T09:41:56.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.629+0000 7f9ae46c6700 1 -- 172.21.15.78:0/2993713296 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9adc04e530 con 0x7f9adc160dc0 2023-10-11T09:41:56.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.629+0000 7f9ad2ffd700 1 -- 172.21.15.78:0/2993713296 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f9ad40043e0 con 0x7f9adc160dc0 2023-10-11T09:41:56.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.633+0000 7f9ad2ffd700 1 --2- 172.21.15.78:0/2993713296 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9ac0065ed0 0x7f9ac0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:56.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.633+0000 7f9ad2ffd700 1 -- 172.21.15.78:0/2993713296 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9ad4022070 con 0x7f9adc160dc0 2023-10-11T09:41:56.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.633+0000 7f9ae2462700 1 --2- 172.21.15.78:0/2993713296 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9ac0065ed0 0x7f9ac0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:56.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.633+0000 7f9ae2462700 1 --2- 172.21.15.78:0/2993713296 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9ac0065ed0 0x7f9ac0068380 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8005930 tx=0x7f9ad8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:56.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.637+0000 7f9ad2ffd700 1 -- 172.21.15.78:0/2993713296 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f9ad40b5fa0 con 0x7f9adc160dc0 2023-10-11T09:41:56.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.817+0000 7f9ae46c6700 1 -- 172.21.15.78:0/2993713296 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9adc071c40 con 0x7f9adc160dc0 2023-10-11T09:41:56.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.817+0000 7f9ad2ffd700 1 -- 172.21.15.78:0/2993713296 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v33) v1 ==== 84+0+126 (secure 0 0 0) 0x7f9ad402c090 con 0x7f9adc160dc0 2023-10-11T09:41:56.823 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:41:56.823 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2023-10-11T09:41:56.824 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:41:56.824 INFO:teuthology.orchestra.run.smithi078.stdout: "poolnum": 1, 2023-10-11T09:41:56.824 INFO:teuthology.orchestra.run.smithi078.stdout: "poolname": ".mgr" 2023-10-11T09:41:56.824 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:41:56.824 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:41:56.824 INFO:teuthology.orchestra.run.smithi078.stdout: "poolnum": 2, 2023-10-11T09:41:56.825 INFO:teuthology.orchestra.run.smithi078.stdout: "poolname": "rbd" 2023-10-11T09:41:56.825 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:41:56.825 INFO:teuthology.orchestra.run.smithi078.stdout:] 2023-10-11T09:41:56.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.821+0000 7f9ae46c6700 1 -- 172.21.15.78:0/2993713296 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9ac0065ed0 msgr2=0x7f9ac0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:56.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.821+0000 7f9ae46c6700 1 --2- 172.21.15.78:0/2993713296 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9ac0065ed0 0x7f9ac0068380 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8005930 tx=0x7f9ad8009040 comp rx=0 tx=0).stop 2023-10-11T09:41:56.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.821+0000 7f9ae46c6700 1 -- 172.21.15.78:0/2993713296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9adc160dc0 msgr2=0x7f9adc0ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:56.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.821+0000 7f9ae46c6700 1 --2- 172.21.15.78:0/2993713296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9adc160dc0 0x7f9adc0ac8e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4014040 tx=0x7f9ad4004af0 comp rx=0 tx=0).stop 2023-10-11T09:41:56.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.821+0000 7f9ae46c6700 1 -- 172.21.15.78:0/2993713296 shutdown_connections 2023-10-11T09:41:56.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.821+0000 7f9ae46c6700 1 --2- 172.21.15.78:0/2993713296 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9ac0065ed0 0x7f9ac0068380 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:56.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.821+0000 7f9ae46c6700 1 --2- 172.21.15.78:0/2993713296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9adc160dc0 0x7f9adc0ac8e0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:56.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.821+0000 7f9ae46c6700 1 --2- 172.21.15.78:0/2993713296 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9adc06fb90 0x7f9adc1ef4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:56.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.821+0000 7f9ae46c6700 1 --2- 172.21.15.78:0/2993713296 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9adc06e1c0 0x7f9adc0af680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:56.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.821+0000 7f9ae46c6700 1 -- 172.21.15.78:0/2993713296 >> 172.21.15.78:0/2993713296 conn(0x7f9adc0d0f70 msgr2=0x7f9adc0d1340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:56.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.821+0000 7f9ae46c6700 1 -- 172.21.15.78:0/2993713296 shutdown_connections 2023-10-11T09:41:56.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:56.821+0000 7f9ae46c6700 1 -- 172.21.15.78:0/2993713296 wait complete. 2023-10-11T09:41:56.844 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-11T09:41:56.845 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-11T09:41:57.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.013+0000 7f8633fff700 1 Processor -- start 2023-10-11T09:41:57.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.013+0000 7f8633fff700 1 -- start start 2023-10-11T09:41:57.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.017+0000 7f8633fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f8634159950 0x7f8634159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:57.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.017+0000 7f8633fff700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86340ccde0 con 0x7f86340c9170 2023-10-11T09:41:57.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.017+0000 7f8633fff700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f863415d540 con 0x7f863415a300 2023-10-11T09:41:57.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.017+0000 7f8633fff700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f863415a8f0 con 0x7f8634159950 2023-10-11T09:41:57.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.017+0000 7f86327fc700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f8634159950 0x7f8634159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:57.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.017+0000 7f8632ffd700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f86340c9170 0x7f86340c9540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:58446/0 (socket says 172.21.15.78:58446) 2023-10-11T09:41:57.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.017+0000 7f8632ffd700 1 -- 172.21.15.78:0/1500329967 learned_addr learned my addr 172.21.15.78:0/1500329967 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:57.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.017+0000 7f86327fc700 1 -- 172.21.15.78:0/1500329967 >> v1:172.21.15.133:6789/0 conn(0x7f863415a300 legacy=0x7f86340cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:57.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.017+0000 7f86327fc700 1 -- 172.21.15.78:0/1500329967 >> v1:172.21.15.78:6789/0 conn(0x7f86340c9170 legacy=0x7f86340c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:57.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.017+0000 7f86327fc700 1 -- 172.21.15.78:0/1500329967 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86340cc790 con 0x7f8634159950 2023-10-11T09:41:57.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.017+0000 7f86327fc700 1 --2- 172.21.15.78:0/1500329967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8634159950 0x7f8634159dc0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8628009850 tx=0x7f8628009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e79f5dfd8493ff server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:57.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.021+0000 7f8631ffb700 1 -- 172.21.15.78:0/1500329967 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8628011830 con 0x7f8634159950 2023-10-11T09:41:57.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.021+0000 7f8631ffb700 1 -- 172.21.15.78:0/1500329967 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8628011990 con 0x7f8634159950 2023-10-11T09:41:57.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.021+0000 7f8631ffb700 1 -- 172.21.15.78:0/1500329967 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8628011d20 con 0x7f8634159950 2023-10-11T09:41:57.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.021+0000 7f8633fff700 1 -- 172.21.15.78:0/1500329967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8634159950 msgr2=0x7f8634159dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:57.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.021+0000 7f8633fff700 1 --2- 172.21.15.78:0/1500329967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8634159950 0x7f8634159dc0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8628009850 tx=0x7f8628009b60 comp rx=0 tx=0).stop 2023-10-11T09:41:57.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.021+0000 7f8633fff700 1 -- 172.21.15.78:0/1500329967 shutdown_connections 2023-10-11T09:41:57.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.021+0000 7f8633fff700 1 --2- 172.21.15.78:0/1500329967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8634159950 0x7f8634159dc0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:57.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.021+0000 7f8633fff700 1 -- 172.21.15.78:0/1500329967 >> 172.21.15.78:0/1500329967 conn(0x7f863406acf0 msgr2=0x7f86340b0710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:57.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.021+0000 7f8633fff700 1 -- 172.21.15.78:0/1500329967 shutdown_connections 2023-10-11T09:41:57.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.021+0000 7f8633fff700 1 -- 172.21.15.78:0/1500329967 wait complete. 2023-10-11T09:41:57.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.021+0000 7f8633fff700 1 Processor -- start 2023-10-11T09:41:57.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.021+0000 7f8633fff700 1 -- start start 2023-10-11T09:41:57.030 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.025+0000 7f8633fff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f86340c9170 0x7f8634150ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:57.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.025+0000 7f8633fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8634159950 0x7f86340b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:57.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.025+0000 7f8633fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f863415a300 0x7f86340b1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:57.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.025+0000 7f8633fff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8634151020 con 0x7f8634159950 2023-10-11T09:41:57.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.025+0000 7f8633fff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8634151a80 con 0x7f863415a300 2023-10-11T09:41:57.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.025+0000 7f8633fff700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86340b2580 con 0x7f86340c9170 2023-10-11T09:41:57.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.025+0000 7f86327fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8634159950 0x7f86340b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:57.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.025+0000 7f86327fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8634159950 0x7f86340b1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57104/0 (socket says 172.21.15.78:57104) 2023-10-11T09:41:57.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.025+0000 7f86327fc700 1 -- 172.21.15.78:0/3918430337 learned_addr learned my addr 172.21.15.78:0/3918430337 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:57.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.025+0000 7f8632ffd700 1 --2- 172.21.15.78:0/3918430337 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f86340c9170 0x7f8634150ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:57.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.025+0000 7f86337fe700 1 --2- 172.21.15.78:0/3918430337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f863415a300 0x7f86340b1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:57.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.025+0000 7f86327fc700 1 -- 172.21.15.78:0/3918430337 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f86340c9170 msgr2=0x7f8634150ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:57.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.025+0000 7f86327fc700 1 --2- 172.21.15.78:0/3918430337 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f86340c9170 0x7f8634150ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:57.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.025+0000 7f86327fc700 1 -- 172.21.15.78:0/3918430337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f863415a300 msgr2=0x7f86340b1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:57.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.025+0000 7f86327fc700 1 --2- 172.21.15.78:0/3918430337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f863415a300 0x7f86340b1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:57.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.025+0000 7f86327fc700 1 -- 172.21.15.78:0/3918430337 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86280094b0 con 0x7f8634159950 2023-10-11T09:41:57.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.025+0000 7f86327fc700 1 --2- 172.21.15.78:0/3918430337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8634159950 0x7f86340b1a10 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8628013000 tx=0x7f8628004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:57.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.025+0000 7f861b7fe700 1 -- 172.21.15.78:0/3918430337 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86280036c0 con 0x7f8634159950 2023-10-11T09:41:57.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.025+0000 7f861b7fe700 1 -- 172.21.15.78:0/3918430337 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8628003820 con 0x7f8634159950 2023-10-11T09:41:57.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.025+0000 7f8633fff700 1 -- 172.21.15.78:0/3918430337 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86340b2860 con 0x7f8634159950 2023-10-11T09:41:57.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.025+0000 7f861b7fe700 1 -- 172.21.15.78:0/3918430337 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8628003bd0 con 0x7f8634159950 2023-10-11T09:41:57.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.025+0000 7f8633fff700 1 -- 172.21.15.78:0/3918430337 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86340b2e20 con 0x7f8634159950 2023-10-11T09:41:57.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.029+0000 7f8633fff700 1 -- 172.21.15.78:0/3918430337 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f863406eca0 con 0x7f8634159950 2023-10-11T09:41:57.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.029+0000 7f861b7fe700 1 -- 172.21.15.78:0/3918430337 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f8628003d30 con 0x7f8634159950 2023-10-11T09:41:57.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.033+0000 7f861b7fe700 1 --2- 172.21.15.78:0/3918430337 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8620065f20 0x7f86200683d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:57.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.033+0000 7f861b7fe700 1 -- 172.21.15.78:0/3918430337 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8628021030 con 0x7f8634159950 2023-10-11T09:41:57.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.033+0000 7f8632ffd700 1 --2- 172.21.15.78:0/3918430337 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8620065f20 0x7f86200683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:57.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.033+0000 7f8632ffd700 1 --2- 172.21.15.78:0/3918430337 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8620065f20 0x7f86200683d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8624005930 tx=0x7f8624009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:57.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.037+0000 7f861b7fe700 1 -- 172.21.15.78:0/3918430337 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f86280525d0 con 0x7f8634159950 2023-10-11T09:41:57.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.209+0000 7f8633fff700 1 -- 172.21.15.78:0/3918430337 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8634065db0 con 0x7f8634159950 2023-10-11T09:41:57.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.733+0000 7f861b7fe700 1 -- 172.21.15.78:0/3918430337 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v34) v1 ==== 138+0+0 (secure 0 0 0) 0x7f86280150b0 con 0x7f8634159950 2023-10-11T09:41:57.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.741+0000 7f8633fff700 1 -- 172.21.15.78:0/3918430337 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8620065f20 msgr2=0x7f86200683d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:57.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.741+0000 7f8633fff700 1 --2- 172.21.15.78:0/3918430337 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8620065f20 0x7f86200683d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8624005930 tx=0x7f8624009040 comp rx=0 tx=0).stop 2023-10-11T09:41:57.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.741+0000 7f8633fff700 1 -- 172.21.15.78:0/3918430337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8634159950 msgr2=0x7f86340b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:57.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.741+0000 7f8633fff700 1 --2- 172.21.15.78:0/3918430337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8634159950 0x7f86340b1a10 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8628013000 tx=0x7f8628004770 comp rx=0 tx=0).stop 2023-10-11T09:41:57.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.741+0000 7f8633fff700 1 -- 172.21.15.78:0/3918430337 shutdown_connections 2023-10-11T09:41:57.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.741+0000 7f8633fff700 1 --2- 172.21.15.78:0/3918430337 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8620065f20 0x7f86200683d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:57.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.741+0000 7f8633fff700 1 --2- 172.21.15.78:0/3918430337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f863415a300 0x7f86340b1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:57.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.741+0000 7f8633fff700 1 --2- 172.21.15.78:0/3918430337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8634159950 0x7f86340b1a10 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:57.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.741+0000 7f8633fff700 1 --2- 172.21.15.78:0/3918430337 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f86340c9170 0x7f8634150ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:57.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.741+0000 7f8633fff700 1 -- 172.21.15.78:0/3918430337 >> 172.21.15.78:0/3918430337 conn(0x7f863406acf0 msgr2=0x7f8634071000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:57.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.745+0000 7f8633fff700 1 -- 172.21.15.78:0/3918430337 shutdown_connections 2023-10-11T09:41:57.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.745+0000 7f8633fff700 1 -- 172.21.15.78:0/3918430337 wait complete. 2023-10-11T09:41:57.753 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' created 2023-10-11T09:41:57.772 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-11T09:41:57.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.953+0000 7f027b039700 1 Processor -- start 2023-10-11T09:41:57.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.953+0000 7f027b039700 1 -- start start 2023-10-11T09:41:57.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.953+0000 7f027b039700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02740cf8d0 0x7f02740b4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:57.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.953+0000 7f027b039700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02740b4fd0 con 0x7f0274160870 2023-10-11T09:41:57.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.953+0000 7f027b039700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02740b52b0 con 0x7f02740cfca0 2023-10-11T09:41:57.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.953+0000 7f027b039700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02740b5870 con 0x7f02740cf8d0 2023-10-11T09:41:57.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.953+0000 7f027a838700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f0274160870 0x7f027415cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:58452/0 (socket says 172.21.15.78:58452) 2023-10-11T09:41:57.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.953+0000 7f027a838700 1 -- 172.21.15.78:0/3286115931 learned_addr learned my addr 172.21.15.78:0/3286115931 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:57.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.953+0000 7f027a037700 1 --2- 172.21.15.78:0/3286115931 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02740cf8d0 0x7f02740b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:57.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.957+0000 7f0279035700 1 -- 172.21.15.78:0/3286115931 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3479492793 0 0) 0x7f02740b4fd0 con 0x7f0274160870 2023-10-11T09:41:57.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.957+0000 7f0279035700 1 -- 172.21.15.78:0/3286115931 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02600035d0 con 0x7f0274160870 2023-10-11T09:41:57.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.957+0000 7f027a037700 1 -- 172.21.15.78:0/3286115931 >> v1:172.21.15.133:6789/0 conn(0x7f02740cfca0 legacy=0x7f02740cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:57.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.957+0000 7f027a037700 1 -- 172.21.15.78:0/3286115931 >> v1:172.21.15.78:6789/0 conn(0x7f0274160870 legacy=0x7f027415cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:57.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.957+0000 7f027a037700 1 -- 172.21.15.78:0/3286115931 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02740b5590 con 0x7f02740cf8d0 2023-10-11T09:41:57.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.957+0000 7f027a037700 1 --2- 172.21.15.78:0/3286115931 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02740cf8d0 0x7f02740b4940 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0270009880 tx=0x7f0270009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0487e946dbe0af9 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:57.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.957+0000 7f0279035700 1 -- 172.21.15.78:0/3286115931 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0270011830 con 0x7f02740cf8d0 2023-10-11T09:41:57.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.957+0000 7f0279035700 1 -- 172.21.15.78:0/3286115931 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0270011990 con 0x7f02740cf8d0 2023-10-11T09:41:57.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.957+0000 7f0279035700 1 -- 172.21.15.78:0/3286115931 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0270011d20 con 0x7f02740cf8d0 2023-10-11T09:41:57.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.957+0000 7f027b039700 1 -- 172.21.15.78:0/3286115931 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02740cf8d0 msgr2=0x7f02740b4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:57.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.957+0000 7f027b039700 1 --2- 172.21.15.78:0/3286115931 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02740cf8d0 0x7f02740b4940 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0270009880 tx=0x7f0270009b90 comp rx=0 tx=0).stop 2023-10-11T09:41:57.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.957+0000 7f027b039700 1 -- 172.21.15.78:0/3286115931 shutdown_connections 2023-10-11T09:41:57.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.957+0000 7f027b039700 1 --2- 172.21.15.78:0/3286115931 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02740cf8d0 0x7f02740b4940 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:57.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.957+0000 7f027b039700 1 -- 172.21.15.78:0/3286115931 >> 172.21.15.78:0/3286115931 conn(0x7f027406b010 msgr2=0x7f027406b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:57.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.957+0000 7f027b039700 1 -- 172.21.15.78:0/3286115931 shutdown_connections 2023-10-11T09:41:57.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.957+0000 7f027b039700 1 -- 172.21.15.78:0/3286115931 wait complete. 2023-10-11T09:41:57.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.961+0000 7f027b039700 1 Processor -- start 2023-10-11T09:41:57.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.961+0000 7f027b039700 1 -- start start 2023-10-11T09:41:57.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.965+0000 7f027b039700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02740cf8d0 0x7f02740c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:57.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.965+0000 7f027b039700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02740cfca0 0x7f02740c69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:57.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.965+0000 7f027b039700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0274160870 0x7f02740cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:57.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.965+0000 7f027b039700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02740c6f30 con 0x7f0274160870 2023-10-11T09:41:57.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.965+0000 7f027b039700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02741f21d0 con 0x7f02740cfca0 2023-10-11T09:41:57.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.965+0000 7f027b039700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02741f24b0 con 0x7f02740cf8d0 2023-10-11T09:41:57.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.965+0000 7f027a838700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0274160870 0x7f02740cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:57.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.965+0000 7f027a838700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0274160870 0x7f02740cb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57116/0 (socket says 172.21.15.78:57116) 2023-10-11T09:41:57.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.965+0000 7f027a838700 1 -- 172.21.15.78:0/1173966207 learned_addr learned my addr 172.21.15.78:0/1173966207 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:57.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.965+0000 7f027a037700 1 --2- 172.21.15.78:0/1173966207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02740cf8d0 0x7f02740c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:57.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.965+0000 7f0279836700 1 --2- 172.21.15.78:0/1173966207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02740cfca0 0x7f02740c69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:57.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.965+0000 7f027a838700 1 -- 172.21.15.78:0/1173966207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02740cf8d0 msgr2=0x7f02740c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:57.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.965+0000 7f027a838700 1 --2- 172.21.15.78:0/1173966207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02740cf8d0 0x7f02740c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:57.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.965+0000 7f027a838700 1 -- 172.21.15.78:0/1173966207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02740cfca0 msgr2=0x7f02740c69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:57.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.965+0000 7f027a838700 1 --2- 172.21.15.78:0/1173966207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02740cfca0 0x7f02740c69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:57.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.965+0000 7f027a838700 1 -- 172.21.15.78:0/1173966207 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02700094b0 con 0x7f0274160870 2023-10-11T09:41:57.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.965+0000 7f027a838700 1 --2- 172.21.15.78:0/1173966207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0274160870 0x7f02740cb6e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f026c00cb40 tx=0x7f026c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:57.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.965+0000 7f026affd700 1 -- 172.21.15.78:0/1173966207 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c007500 con 0x7f0274160870 2023-10-11T09:41:57.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.965+0000 7f026affd700 1 -- 172.21.15.78:0/1173966207 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f026c007660 con 0x7f0274160870 2023-10-11T09:41:57.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.965+0000 7f027b039700 1 -- 172.21.15.78:0/1173966207 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02741f2790 con 0x7f0274160870 2023-10-11T09:41:57.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.965+0000 7f026affd700 1 -- 172.21.15.78:0/1173966207 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c0079f0 con 0x7f0274160870 2023-10-11T09:41:57.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.965+0000 7f027b039700 1 -- 172.21.15.78:0/1173966207 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02741f2d50 con 0x7f0274160870 2023-10-11T09:41:57.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.965+0000 7f027b039700 1 -- 172.21.15.78:0/1173966207 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02741f2f70 con 0x7f0274160870 2023-10-11T09:41:57.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.969+0000 7f026affd700 1 -- 172.21.15.78:0/1173966207 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f026c01b3b0 con 0x7f0274160870 2023-10-11T09:41:57.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.973+0000 7f026affd700 1 --2- 172.21.15.78:0/1173966207 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0258065ed0 0x7f0258068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:57.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.973+0000 7f026affd700 1 -- 172.21.15.78:0/1173966207 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f026c083480 con 0x7f0274160870 2023-10-11T09:41:57.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.973+0000 7f027a037700 1 --2- 172.21.15.78:0/1173966207 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0258065ed0 0x7f0258068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:57.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.973+0000 7f027a037700 1 --2- 172.21.15.78:0/1173966207 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0258065ed0 0x7f0258068380 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f02700090b0 tx=0x7f0270010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:57.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:57.977+0000 7f026affd700 1 -- 172.21.15.78:0/1173966207 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f026c050ca0 con 0x7f0274160870 2023-10-11T09:41:58.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.153+0000 7f027b039700 1 -- 172.21.15.78:0/1173966207 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0274065d40 con 0x7f0274160870 2023-10-11T09:41:58.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.749+0000 7f026affd700 1 -- 172.21.15.78:0/1173966207 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v35) v1 ==== 170+0+0 (secure 0 0 0) 0x7f026c0542c0 con 0x7f0274160870 2023-10-11T09:41:58.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.765+0000 7f027b039700 1 -- 172.21.15.78:0/1173966207 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0258065ed0 msgr2=0x7f0258068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:58.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.765+0000 7f027b039700 1 --2- 172.21.15.78:0/1173966207 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0258065ed0 0x7f0258068380 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f02700090b0 tx=0x7f0270010000 comp rx=0 tx=0).stop 2023-10-11T09:41:58.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.765+0000 7f027b039700 1 -- 172.21.15.78:0/1173966207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0274160870 msgr2=0x7f02740cb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:58.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.765+0000 7f027b039700 1 --2- 172.21.15.78:0/1173966207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0274160870 0x7f02740cb6e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f026c00cb40 tx=0x7f026c00cb70 comp rx=0 tx=0).stop 2023-10-11T09:41:58.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.765+0000 7f027b039700 1 -- 172.21.15.78:0/1173966207 shutdown_connections 2023-10-11T09:41:58.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.765+0000 7f027b039700 1 --2- 172.21.15.78:0/1173966207 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0258065ed0 0x7f0258068380 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:58.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.765+0000 7f027b039700 1 --2- 172.21.15.78:0/1173966207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0274160870 0x7f02740cb6e0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:58.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.765+0000 7f027b039700 1 --2- 172.21.15.78:0/1173966207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02740cfca0 0x7f02740c69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:58.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.765+0000 7f027b039700 1 --2- 172.21.15.78:0/1173966207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02740cf8d0 0x7f02740c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:58.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.765+0000 7f027b039700 1 -- 172.21.15.78:0/1173966207 >> 172.21.15.78:0/1173966207 conn(0x7f027406b010 msgr2=0x7f02740cd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:58.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.765+0000 7f027b039700 1 -- 172.21.15.78:0/1173966207 shutdown_connections 2023-10-11T09:41:58.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.769+0000 7f027b039700 1 -- 172.21.15.78:0/1173966207 wait complete. 2023-10-11T09:41:58.783 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' created 2023-10-11T09:41:58.801 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-11T09:41:58.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.981+0000 7f77f0e0a700 1 Processor -- start 2023-10-11T09:41:58.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.981+0000 7f77f0e0a700 1 -- start start 2023-10-11T09:41:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.981+0000 7f77f0e0a700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77ec158430 0x7f77ec0710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:58.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.981+0000 7f77f0e0a700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f77ec06d340 0x7f77ec0ca880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:58.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.981+0000 7f77f0e0a700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77ec071610 con 0x7f77ec0cadc0 2023-10-11T09:41:58.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.981+0000 7f77f0e0a700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77ec070c10 con 0x7f77ec06d340 2023-10-11T09:41:58.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.981+0000 7f77f0e0a700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77ec0cb240 con 0x7f77ec158430 2023-10-11T09:41:58.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.981+0000 7f77e9d9b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f77ec06d340 0x7f77ec0ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:58.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.981+0000 7f77ead9d700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f77ec0cadc0 0x7f77ec15afe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:58468/0 (socket says 172.21.15.78:58468) 2023-10-11T09:41:58.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.981+0000 7f77ead9d700 1 -- 172.21.15.78:0/882277368 learned_addr learned my addr 172.21.15.78:0/882277368 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:58.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.981+0000 7f77ea59c700 1 --2- 172.21.15.78:0/882277368 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77ec158430 0x7f77ec0710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:58.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.985+0000 7f77e9d9b700 1 -- 172.21.15.78:0/882277368 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77ec158430 msgr2=0x7f77ec0710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:58.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.985+0000 7f77e9d9b700 1 --2- 172.21.15.78:0/882277368 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77ec158430 0x7f77ec0710d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:58.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.985+0000 7f77e9d9b700 1 -- 172.21.15.78:0/882277368 >> v1:172.21.15.78:6789/0 conn(0x7f77ec0cadc0 legacy=0x7f77ec15afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:58.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.985+0000 7f77e9d9b700 1 -- 172.21.15.78:0/882277368 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77ec15e280 con 0x7f77ec06d340 2023-10-11T09:41:58.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.985+0000 7f77e9d9b700 1 --2- 172.21.15.78:0/882277368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77ec06d340 0x7f77ec0ca880 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f77e0009850 tx=0x7f77e0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2606b0096a04698d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:58.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.985+0000 7f77e959a700 1 -- 172.21.15.78:0/882277368 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e0011830 con 0x7f77ec06d340 2023-10-11T09:41:58.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.985+0000 7f77e959a700 1 -- 172.21.15.78:0/882277368 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e0011990 con 0x7f77ec06d340 2023-10-11T09:41:58.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.985+0000 7f77e959a700 1 -- 172.21.15.78:0/882277368 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e0011d40 con 0x7f77ec06d340 2023-10-11T09:41:58.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.985+0000 7f77f0e0a700 1 -- 172.21.15.78:0/882277368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77ec06d340 msgr2=0x7f77ec0ca880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:58.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.985+0000 7f77f0e0a700 1 --2- 172.21.15.78:0/882277368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77ec06d340 0x7f77ec0ca880 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f77e0009850 tx=0x7f77e0009b60 comp rx=0 tx=0).stop 2023-10-11T09:41:58.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.985+0000 7f77f0e0a700 1 -- 172.21.15.78:0/882277368 shutdown_connections 2023-10-11T09:41:58.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.985+0000 7f77f0e0a700 1 --2- 172.21.15.78:0/882277368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77ec06d340 0x7f77ec0ca880 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:58.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.985+0000 7f77f0e0a700 1 --2- 172.21.15.78:0/882277368 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77ec158430 0x7f77ec0710d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:58.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.985+0000 7f77f0e0a700 1 -- 172.21.15.78:0/882277368 >> 172.21.15.78:0/882277368 conn(0x7f77ec0cecc0 msgr2=0x7f77ec0cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:58.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.985+0000 7f77f0e0a700 1 -- 172.21.15.78:0/882277368 shutdown_connections 2023-10-11T09:41:58.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.985+0000 7f77f0e0a700 1 -- 172.21.15.78:0/882277368 wait complete. 2023-10-11T09:41:58.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.989+0000 7f77f0e0a700 1 Processor -- start 2023-10-11T09:41:58.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.989+0000 7f77f0e0a700 1 -- start start 2023-10-11T09:41:58.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.993+0000 7f77f0e0a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77ec06d340 0x7f77ec153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:58.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.993+0000 7f77f0e0a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77ec0cadc0 0x7f77ec0a84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:58.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.993+0000 7f77f0e0a700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77ec158430 0x7f77ec0aba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:58.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.993+0000 7f77f0e0a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77ec0abfa0 con 0x7f77ec06d340 2023-10-11T09:41:58.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.993+0000 7f77f0e0a700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77ec0a8bc0 con 0x7f77ec0cadc0 2023-10-11T09:41:58.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.993+0000 7f77f0e0a700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77ec0a8ea0 con 0x7f77ec158430 2023-10-11T09:41:59.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.993+0000 7f77ea59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77ec06d340 0x7f77ec153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:59.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.993+0000 7f77ea59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77ec06d340 0x7f77ec153bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57142/0 (socket says 172.21.15.78:57142) 2023-10-11T09:41:59.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.993+0000 7f77ea59c700 1 -- 172.21.15.78:0/718443637 learned_addr learned my addr 172.21.15.78:0/718443637 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:59.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.993+0000 7f77ead9d700 1 --2- 172.21.15.78:0/718443637 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77ec158430 0x7f77ec0aba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:59.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.993+0000 7f77e9d9b700 1 --2- 172.21.15.78:0/718443637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77ec0cadc0 0x7f77ec0a84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:59.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.993+0000 7f77ea59c700 1 -- 172.21.15.78:0/718443637 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77ec158430 msgr2=0x7f77ec0aba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:59.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.993+0000 7f77ea59c700 1 --2- 172.21.15.78:0/718443637 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77ec158430 0x7f77ec0aba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:59.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.993+0000 7f77ea59c700 1 -- 172.21.15.78:0/718443637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77ec0cadc0 msgr2=0x7f77ec0a84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:59.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.993+0000 7f77ea59c700 1 --2- 172.21.15.78:0/718443637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77ec0cadc0 0x7f77ec0a84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:59.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.993+0000 7f77ea59c700 1 -- 172.21.15.78:0/718443637 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e00094b0 con 0x7f77ec06d340 2023-10-11T09:41:59.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.993+0000 7f77ead9d700 1 --2- 172.21.15.78:0/718443637 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77ec158430 0x7f77ec0aba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:41:59.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.993+0000 7f77e9d9b700 1 --2- 172.21.15.78:0/718443637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77ec0cadc0 0x7f77ec0a84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:41:59.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.993+0000 7f77ea59c700 1 --2- 172.21.15.78:0/718443637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77ec06d340 0x7f77ec153bd0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00e840 tx=0x7f77dc00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:59.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.993+0000 7f77daffd700 1 -- 172.21.15.78:0/718443637 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77dc0095a0 con 0x7f77ec06d340 2023-10-11T09:41:59.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.993+0000 7f77daffd700 1 -- 172.21.15.78:0/718443637 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77dc00c690 con 0x7f77ec06d340 2023-10-11T09:41:59.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.993+0000 7f77f0e0a700 1 -- 172.21.15.78:0/718443637 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77ec0a9180 con 0x7f77ec06d340 2023-10-11T09:41:59.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.993+0000 7f77daffd700 1 -- 172.21.15.78:0/718443637 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77dc00ca00 con 0x7f77ec06d340 2023-10-11T09:41:59.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.993+0000 7f77f0e0a700 1 -- 172.21.15.78:0/718443637 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77ec0a9740 con 0x7f77ec06d340 2023-10-11T09:41:59.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.997+0000 7f77f0e0a700 1 -- 172.21.15.78:0/718443637 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77ec06eb80 con 0x7f77ec06d340 2023-10-11T09:41:59.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.997+0000 7f77daffd700 1 -- 172.21.15.78:0/718443637 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f77dc00cb60 con 0x7f77ec06d340 2023-10-11T09:41:59.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.997+0000 7f77daffd700 1 --2- 172.21.15.78:0/718443637 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f77cc065ed0 0x7f77cc068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:59.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:58.997+0000 7f77daffd700 1 -- 172.21.15.78:0/718443637 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f77dc083720 con 0x7f77ec06d340 2023-10-11T09:41:59.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.001+0000 7f77e9d9b700 1 --2- 172.21.15.78:0/718443637 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f77cc065ed0 0x7f77cc068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:59.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.001+0000 7f77e9d9b700 1 --2- 172.21.15.78:0/718443637 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f77cc065ed0 0x7f77cc068380 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f77ec15d8e0 tx=0x7f77e0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:59.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.005+0000 7f77daffd700 1 -- 172.21.15.78:0/718443637 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f77dc050e60 con 0x7f77ec06d340 2023-10-11T09:41:59.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.217+0000 7f77f0e0a700 1 -- 172.21.15.78:0/718443637 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f77ec0c9a40 con 0x7f77ec06d340 2023-10-11T09:41:59.245 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:41:59.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.233+0000 7f77daffd700 1 -- 172.21.15.78:0/718443637 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7f77dc054480 con 0x7f77ec06d340 2023-10-11T09:41:59.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.241+0000 7f77f0e0a700 1 -- 172.21.15.78:0/718443637 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f77cc065ed0 msgr2=0x7f77cc068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:59.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.241+0000 7f77f0e0a700 1 --2- 172.21.15.78:0/718443637 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f77cc065ed0 0x7f77cc068380 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f77ec15d8e0 tx=0x7f77e0008030 comp rx=0 tx=0).stop 2023-10-11T09:41:59.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.241+0000 7f77f0e0a700 1 -- 172.21.15.78:0/718443637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77ec06d340 msgr2=0x7f77ec153bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:59.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.241+0000 7f77f0e0a700 1 --2- 172.21.15.78:0/718443637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77ec06d340 0x7f77ec153bd0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00e840 tx=0x7f77dc00ec00 comp rx=0 tx=0).stop 2023-10-11T09:41:59.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.241+0000 7f77f0e0a700 1 -- 172.21.15.78:0/718443637 shutdown_connections 2023-10-11T09:41:59.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.241+0000 7f77f0e0a700 1 --2- 172.21.15.78:0/718443637 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f77cc065ed0 0x7f77cc068380 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:59.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.241+0000 7f77f0e0a700 1 --2- 172.21.15.78:0/718443637 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77ec158430 0x7f77ec0aba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:59.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.241+0000 7f77f0e0a700 1 --2- 172.21.15.78:0/718443637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77ec0cadc0 0x7f77ec0a84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:59.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.241+0000 7f77f0e0a700 1 --2- 172.21.15.78:0/718443637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77ec06d340 0x7f77ec153bd0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:59.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.241+0000 7f77f0e0a700 1 -- 172.21.15.78:0/718443637 >> 172.21.15.78:0/718443637 conn(0x7f77ec0cecc0 msgr2=0x7f77ec0ae900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:59.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.241+0000 7f77f0e0a700 1 -- 172.21.15.78:0/718443637 shutdown_connections 2023-10-11T09:41:59.257 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:41:59.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.245+0000 7f77f0e0a700 1 -- 172.21.15.78:0/718443637 wait complete. 2023-10-11T09:41:59.258 INFO:teuthology.orchestra.run.smithi078.stderr:new fs with metadata pool 5 and data pool 6 2023-10-11T09:41:59.284 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:41:59.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.517+0000 7f1d0d937700 1 Processor -- start 2023-10-11T09:41:59.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.517+0000 7f1d0d937700 1 -- start start 2023-10-11T09:41:59.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.521+0000 7f1d0d937700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1d08154e00 0x7f1d0815b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:59.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.521+0000 7f1d0d937700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d08071040 0x7f1d080cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:59.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.521+0000 7f1d0d937700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d080720f0 con 0x7f1d080cba00 2023-10-11T09:41:59.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.521+0000 7f1d0d937700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d0815bb90 con 0x7f1d08154e00 2023-10-11T09:41:59.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.521+0000 7f1d0d937700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d080714e0 con 0x7f1d08071040 2023-10-11T09:41:59.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.521+0000 7f1d077fe700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f1d080cba00 0x7f1d0815e1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:58472/0 (socket says 172.21.15.78:58472) 2023-10-11T09:41:59.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.521+0000 7f1d077fe700 1 -- 172.21.15.78:0/1368183719 learned_addr learned my addr 172.21.15.78:0/1368183719 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:41:59.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.521+0000 7f1d06ffd700 1 --2- 172.21.15.78:0/1368183719 >> v2:172.21.15.133:3300/0 conn(0x7f1d08154e00 0x7f1d0815b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:59.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.521+0000 7f1cfe7fc700 1 --2- 172.21.15.78:0/1368183719 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d08071040 0x7f1d080cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:59.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.521+0000 7f1d067fc700 1 -- 172.21.15.78:0/1368183719 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 761376622 0 0) 0x7f1d080720f0 con 0x7f1d080cba00 2023-10-11T09:41:59.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.521+0000 7f1d067fc700 1 -- 172.21.15.78:0/1368183719 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cf40035d0 con 0x7f1d080cba00 2023-10-11T09:41:59.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.521+0000 7f1d06ffd700 1 -- 172.21.15.78:0/1368183719 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d08071040 msgr2=0x7f1d080cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:59.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.521+0000 7f1d06ffd700 1 --2- 172.21.15.78:0/1368183719 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d08071040 0x7f1d080cb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:59.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.521+0000 7f1d06ffd700 1 -- 172.21.15.78:0/1368183719 >> v1:172.21.15.78:6789/0 conn(0x7f1d080cba00 legacy=0x7f1d0815e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:59.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.521+0000 7f1d06ffd700 1 -- 172.21.15.78:0/1368183719 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d080717c0 con 0x7f1d08154e00 2023-10-11T09:41:59.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.521+0000 7f1d06ffd700 1 --2- 172.21.15.78:0/1368183719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d08154e00 0x7f1d0815b650 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8009850 tx=0x7f1cf8009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9d4b7c8d18c6b08 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:59.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.525+0000 7f1d067fc700 1 -- 172.21.15.78:0/1368183719 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf8011830 con 0x7f1d08154e00 2023-10-11T09:41:59.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.525+0000 7f1d067fc700 1 -- 172.21.15.78:0/1368183719 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf8011990 con 0x7f1d08154e00 2023-10-11T09:41:59.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.525+0000 7f1d067fc700 1 -- 172.21.15.78:0/1368183719 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf8011d40 con 0x7f1d08154e00 2023-10-11T09:41:59.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.525+0000 7f1d0d937700 1 -- 172.21.15.78:0/1368183719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d08154e00 msgr2=0x7f1d0815b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:59.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.525+0000 7f1d0d937700 1 --2- 172.21.15.78:0/1368183719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d08154e00 0x7f1d0815b650 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8009850 tx=0x7f1cf8009b60 comp rx=0 tx=0).stop 2023-10-11T09:41:59.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.525+0000 7f1d0d937700 1 -- 172.21.15.78:0/1368183719 shutdown_connections 2023-10-11T09:41:59.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.525+0000 7f1d0d937700 1 --2- 172.21.15.78:0/1368183719 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d08071040 0x7f1d080cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:59.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.525+0000 7f1d0d937700 1 --2- 172.21.15.78:0/1368183719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d08154e00 0x7f1d0815b650 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:59.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.525+0000 7f1d0d937700 1 -- 172.21.15.78:0/1368183719 >> 172.21.15.78:0/1368183719 conn(0x7f1d0806aca0 msgr2=0x7f1d080b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:59.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.525+0000 7f1d0d937700 1 -- 172.21.15.78:0/1368183719 shutdown_connections 2023-10-11T09:41:59.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.525+0000 7f1d0d937700 1 -- 172.21.15.78:0/1368183719 wait complete. 2023-10-11T09:41:59.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.525+0000 7f1d0d937700 1 Processor -- start 2023-10-11T09:41:59.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.525+0000 7f1d0d937700 1 -- start start 2023-10-11T09:41:59.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.529+0000 7f1d0d937700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d08071040 0x7f1d0814c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:59.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.529+0000 7f1d0d937700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d080cba00 0x7f1d080b20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:59.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.529+0000 7f1d0d937700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d08154e00 0x7f1d080c8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:59.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.529+0000 7f1d0d937700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d081f6430 con 0x7f1d08071040 2023-10-11T09:41:59.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.529+0000 7f1d0d937700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d080cdbf0 con 0x7f1d08154e00 2023-10-11T09:41:59.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.529+0000 7f1d0d937700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d081f5840 con 0x7f1d080cba00 2023-10-11T09:41:59.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.529+0000 7f1d06ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d08071040 0x7f1d0814c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:59.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.529+0000 7f1d06ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d08071040 0x7f1d0814c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57154/0 (socket says 172.21.15.78:57154) 2023-10-11T09:41:59.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.529+0000 7f1d06ffd700 1 -- 172.21.15.78:0/570358962 learned_addr learned my addr 172.21.15.78:0/570358962 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:59.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.529+0000 7f1d077fe700 1 --2- 172.21.15.78:0/570358962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d08154e00 0x7f1d080c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:59.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.529+0000 7f1cfe7fc700 1 --2- 172.21.15.78:0/570358962 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d080cba00 0x7f1d080b20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:59.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.529+0000 7f1d06ffd700 1 -- 172.21.15.78:0/570358962 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d080cba00 msgr2=0x7f1d080b20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:59.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.529+0000 7f1d06ffd700 1 --2- 172.21.15.78:0/570358962 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d080cba00 0x7f1d080b20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:59.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.529+0000 7f1d06ffd700 1 -- 172.21.15.78:0/570358962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d08154e00 msgr2=0x7f1d080c8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:59.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.529+0000 7f1d06ffd700 1 --2- 172.21.15.78:0/570358962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d08154e00 0x7f1d080c8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:59.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.529+0000 7f1d06ffd700 1 -- 172.21.15.78:0/570358962 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf80094b0 con 0x7f1d08071040 2023-10-11T09:41:59.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.529+0000 7f1d077fe700 1 --2- 172.21.15.78:0/570358962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d08154e00 0x7f1d080c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:41:59.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.529+0000 7f1cfe7fc700 1 --2- 172.21.15.78:0/570358962 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d080cba00 0x7f1d080b20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:41:59.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.529+0000 7f1d06ffd700 1 --2- 172.21.15.78:0/570358962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d08071040 0x7f1d0814c9c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8014040 tx=0x7f1cf8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:59.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.529+0000 7f1d04ff9700 1 -- 172.21.15.78:0/570358962 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf80036c0 con 0x7f1d08071040 2023-10-11T09:41:59.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.529+0000 7f1d0d937700 1 -- 172.21.15.78:0/570358962 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d081f5a90 con 0x7f1d08071040 2023-10-11T09:41:59.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.529+0000 7f1d0d937700 1 -- 172.21.15.78:0/570358962 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d081f6050 con 0x7f1d08071040 2023-10-11T09:41:59.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.529+0000 7f1d04ff9700 1 -- 172.21.15.78:0/570358962 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf801e070 con 0x7f1d08071040 2023-10-11T09:41:59.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.529+0000 7f1d04ff9700 1 -- 172.21.15.78:0/570358962 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf8003a20 con 0x7f1d08071040 2023-10-11T09:41:59.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.533+0000 7f1d0d937700 1 -- 172.21.15.78:0/570358962 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d0804e530 con 0x7f1d08071040 2023-10-11T09:41:59.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.533+0000 7f1d04ff9700 1 -- 172.21.15.78:0/570358962 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f1cf8003b80 con 0x7f1d08071040 2023-10-11T09:41:59.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.533+0000 7f1d04ff9700 1 --2- 172.21.15.78:0/570358962 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1ce8065ed0 0x7f1ce8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:59.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.537+0000 7f1d04ff9700 1 -- 172.21.15.78:0/570358962 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1cf8020070 con 0x7f1d08071040 2023-10-11T09:41:59.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.537+0000 7f1cfe7fc700 1 --2- 172.21.15.78:0/570358962 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1ce8065ed0 0x7f1ce8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:59.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.537+0000 7f1cfe7fc700 1 --2- 172.21.15.78:0/570358962 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1ce8065ed0 0x7f1ce8068380 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1d080b2060 tx=0x7f1cf000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:59.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.541+0000 7f1d04ff9700 1 -- 172.21.15.78:0/570358962 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f1cf80522a0 con 0x7f1d08071040 2023-10-11T09:41:59.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.717+0000 7f1d0d937700 1 -- 172.21.15.78:0/570358962 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d0806e1e0 con 0x7f1d08071040 2023-10-11T09:41:59.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.721+0000 7f1d04ff9700 1 -- 172.21.15.78:0/570358962 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+17896 (secure 0 0 0) 0x7f1cf8015090 con 0x7f1d08071040 2023-10-11T09:41:59.728 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:41:59.728 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":36,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:41:59.224526+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:41:57.280079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-11T09:41:58.218860+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:41:59.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.725+0000 7f1d0d937700 1 -- 172.21.15.78:0/570358962 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1ce8065ed0 msgr2=0x7f1ce8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:59.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.725+0000 7f1d0d937700 1 --2- 172.21.15.78:0/570358962 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1ce8065ed0 0x7f1ce8068380 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1d080b2060 tx=0x7f1cf000b040 comp rx=0 tx=0).stop 2023-10-11T09:41:59.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.725+0000 7f1d0d937700 1 -- 172.21.15.78:0/570358962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d08071040 msgr2=0x7f1d0814c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:59.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.725+0000 7f1d0d937700 1 --2- 172.21.15.78:0/570358962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d08071040 0x7f1d0814c9c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8014040 tx=0x7f1cf8004770 comp rx=0 tx=0).stop 2023-10-11T09:41:59.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.725+0000 7f1d0d937700 1 -- 172.21.15.78:0/570358962 shutdown_connections 2023-10-11T09:41:59.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.725+0000 7f1d0d937700 1 --2- 172.21.15.78:0/570358962 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1ce8065ed0 0x7f1ce8068380 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:59.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.725+0000 7f1d0d937700 1 --2- 172.21.15.78:0/570358962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d08154e00 0x7f1d080c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:59.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.725+0000 7f1d0d937700 1 --2- 172.21.15.78:0/570358962 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d080cba00 0x7f1d080b20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:59.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.725+0000 7f1d0d937700 1 --2- 172.21.15.78:0/570358962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d08071040 0x7f1d0814c9c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:59.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.725+0000 7f1d0d937700 1 -- 172.21.15.78:0/570358962 >> 172.21.15.78:0/570358962 conn(0x7f1d0806aca0 msgr2=0x7f1d0815e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:59.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.725+0000 7f1d0d937700 1 -- 172.21.15.78:0/570358962 shutdown_connections 2023-10-11T09:41:59.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.725+0000 7f1d0d937700 1 -- 172.21.15.78:0/570358962 wait complete. 2023-10-11T09:41:59.755 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:41:59.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.941+0000 7f6bf988c700 1 Processor -- start 2023-10-11T09:41:59.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.941+0000 7f6bf988c700 1 -- start start 2023-10-11T09:41:59.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.941+0000 7f6bf988c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f6bf40cda10 0x7f6bf40caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:59.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.941+0000 7f6bf988c700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6bf40cdde0 0x7f6bf4164610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:59.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.941+0000 7f6bf988c700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bf4165390 con 0x7f6bf4164b50 2023-10-11T09:41:59.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.941+0000 7f6bf988c700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bf40ca440 con 0x7f6bf40cda10 2023-10-11T09:41:59.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.941+0000 7f6bf988c700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bf40ca720 con 0x7f6bf40cdde0 2023-10-11T09:41:59.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.945+0000 7f6bf888a700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f6bf40cda10 0x7f6bf40caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:59.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.945+0000 7f6bebfff700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6bf40cdde0 0x7f6bf4164610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:59.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.945+0000 7f6bf888a700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f6bf40cda10 0x7f6bf40caf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57158/0 (socket says 172.21.15.78:57158) 2023-10-11T09:41:59.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.945+0000 7f6bf888a700 1 -- 172.21.15.78:0/860614266 learned_addr learned my addr 172.21.15.78:0/860614266 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:59.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.945+0000 7f6bf888a700 1 -- 172.21.15.78:0/860614266 >> v2:172.21.15.133:3300/0 conn(0x7f6bf40cdde0 msgr2=0x7f6bf4164610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:41:59.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.945+0000 7f6bf888a700 1 --2- 172.21.15.78:0/860614266 >> v2:172.21.15.133:3300/0 conn(0x7f6bf40cdde0 0x7f6bf4164610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:59.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.945+0000 7f6bf888a700 1 -- 172.21.15.78:0/860614266 >> v1:172.21.15.133:6789/0 conn(0x7f6bf4164b50 legacy=0x7f6bf406bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:59.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.945+0000 7f6bf888a700 1 -- 172.21.15.78:0/860614266 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf40caa00 con 0x7f6bf40cda10 2023-10-11T09:41:59.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.945+0000 7f6bebfff700 1 --2- 172.21.15.78:0/860614266 >> v2:172.21.15.133:3300/0 conn(0x7f6bf40cdde0 0x7f6bf4164610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:41:59.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.945+0000 7f6beb7fe700 1 -- 172.21.15.78:0/860614266 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 707623368 0 0) 0x7f6bf4165390 con 0x7f6bf4164b50 2023-10-11T09:41:59.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.945+0000 7f6bf888a700 1 --2- 172.21.15.78:0/860614266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bf40cda10 0x7f6bf40caf20 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6bf40cea10 tx=0x7f6bf0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a284bcb2e90bf server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:59.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.945+0000 7f6beb7fe700 1 -- 172.21.15.78:0/860614266 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf0011830 con 0x7f6bf40cda10 2023-10-11T09:41:59.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.945+0000 7f6beb7fe700 1 -- 172.21.15.78:0/860614266 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf0011990 con 0x7f6bf40cda10 2023-10-11T09:41:59.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.945+0000 7f6beb7fe700 1 -- 172.21.15.78:0/860614266 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf0011d20 con 0x7f6bf40cda10 2023-10-11T09:41:59.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.945+0000 7f6bf988c700 1 -- 172.21.15.78:0/860614266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bf40cda10 msgr2=0x7f6bf40caf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:59.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.945+0000 7f6bf988c700 1 --2- 172.21.15.78:0/860614266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bf40cda10 0x7f6bf40caf20 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6bf40cea10 tx=0x7f6bf0009bf0 comp rx=0 tx=0).stop 2023-10-11T09:41:59.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.945+0000 7f6bf988c700 1 -- 172.21.15.78:0/860614266 shutdown_connections 2023-10-11T09:41:59.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.945+0000 7f6bf988c700 1 --2- 172.21.15.78:0/860614266 >> v2:172.21.15.133:3300/0 conn(0x7f6bf40cdde0 0x7f6bf4164610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:59.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.945+0000 7f6bf988c700 1 --2- 172.21.15.78:0/860614266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bf40cda10 0x7f6bf40caf20 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:59.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.945+0000 7f6bf988c700 1 -- 172.21.15.78:0/860614266 >> 172.21.15.78:0/860614266 conn(0x7f6bf40cea60 msgr2=0x7f6bf40cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:41:59.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.949+0000 7f6bf988c700 1 -- 172.21.15.78:0/860614266 shutdown_connections 2023-10-11T09:41:59.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.949+0000 7f6bf988c700 1 -- 172.21.15.78:0/860614266 wait complete. 2023-10-11T09:41:59.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.949+0000 7f6bf988c700 1 Processor -- start 2023-10-11T09:41:59.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.949+0000 7f6bf988c700 1 -- start start 2023-10-11T09:41:59.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.953+0000 7f6bf988c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bf40cda10 0x7f6bf4161980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:59.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.953+0000 7f6bf988c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6bf40cdde0 0x7f6bf40b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:59.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.953+0000 7f6bf988c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6bf4164b50 0x7f6bf41497b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:59.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.953+0000 7f6bf988c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf4162240 con 0x7f6bf40cda10 2023-10-11T09:41:59.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.953+0000 7f6bf988c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf4162520 con 0x7f6bf4164b50 2023-10-11T09:41:59.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.953+0000 7f6bf988c700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf4148fb0 con 0x7f6bf40cdde0 2023-10-11T09:41:59.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.953+0000 7f6bf888a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bf40cda10 0x7f6bf4161980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:59.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.953+0000 7f6bf888a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bf40cda10 0x7f6bf4161980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57160/0 (socket says 172.21.15.78:57160) 2023-10-11T09:41:59.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.953+0000 7f6bf888a700 1 -- 172.21.15.78:0/2556619677 learned_addr learned my addr 172.21.15.78:0/2556619677 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:41:59.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.953+0000 7f6bebfff700 1 --2- 172.21.15.78:0/2556619677 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6bf40cdde0 0x7f6bf40b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:59.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.953+0000 7f6bf908b700 1 --2- 172.21.15.78:0/2556619677 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6bf4164b50 0x7f6bf41497b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:59.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.953+0000 7f6bf888a700 1 -- 172.21.15.78:0/2556619677 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6bf40cdde0 msgr2=0x7f6bf40b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:59.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.953+0000 7f6bf888a700 1 --2- 172.21.15.78:0/2556619677 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6bf40cdde0 0x7f6bf40b3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:59.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.953+0000 7f6bf888a700 1 -- 172.21.15.78:0/2556619677 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6bf4164b50 msgr2=0x7f6bf41497b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:41:59.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.953+0000 7f6bf888a700 1 --2- 172.21.15.78:0/2556619677 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6bf4164b50 0x7f6bf41497b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:41:59.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.953+0000 7f6bf888a700 1 -- 172.21.15.78:0/2556619677 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be40094b0 con 0x7f6bf40cda10 2023-10-11T09:41:59.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.953+0000 7f6bf888a700 1 --2- 172.21.15.78:0/2556619677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bf40cda10 0x7f6bf4161980 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4072690 tx=0x7f6bf0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:59.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.953+0000 7f6be97fa700 1 -- 172.21.15.78:0/2556619677 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf00045d0 con 0x7f6bf40cda10 2023-10-11T09:41:59.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.953+0000 7f6be97fa700 1 -- 172.21.15.78:0/2556619677 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf001f070 con 0x7f6bf40cda10 2023-10-11T09:41:59.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.953+0000 7f6be97fa700 1 -- 172.21.15.78:0/2556619677 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf00038e0 con 0x7f6bf40cda10 2023-10-11T09:41:59.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.953+0000 7f6bf988c700 1 -- 172.21.15.78:0/2556619677 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bf0009570 con 0x7f6bf40cda10 2023-10-11T09:41:59.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.953+0000 7f6bf988c700 1 -- 172.21.15.78:0/2556619677 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bf4161ec0 con 0x7f6bf40cda10 2023-10-11T09:41:59.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.953+0000 7f6bf988c700 1 -- 172.21.15.78:0/2556619677 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bf41494d0 con 0x7f6bf40cda10 2023-10-11T09:41:59.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.957+0000 7f6be97fa700 1 -- 172.21.15.78:0/2556619677 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f6bf0003a40 con 0x7f6bf40cda10 2023-10-11T09:41:59.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.957+0000 7f6be97fa700 1 --2- 172.21.15.78:0/2556619677 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6be0065ed0 0x7f6be0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:41:59.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.957+0000 7f6be97fa700 1 -- 172.21.15.78:0/2556619677 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6bf0021070 con 0x7f6bf40cda10 2023-10-11T09:41:59.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.957+0000 7f6bebfff700 1 --2- 172.21.15.78:0/2556619677 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6be0065ed0 0x7f6be0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:41:59.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.957+0000 7f6bebfff700 1 --2- 172.21.15.78:0/2556619677 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6be0065ed0 0x7f6be0068380 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4160140 tx=0x7f6be4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:41:59.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:41:59.961+0000 7f6be97fa700 1 -- 172.21.15.78:0/2556619677 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f6bf41494d0 con 0x7f6bf40cda10 2023-10-11T09:42:00.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.141+0000 7f6bf988c700 1 -- 172.21.15.78:0/2556619677 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6bf41494d0 con 0x7f6bf40cda10 2023-10-11T09:42:00.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.141+0000 7f6be97fa700 1 -- 172.21.15.78:0/2556619677 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+17896 (secure 0 0 0) 0x7f6bf41494d0 con 0x7f6bf40cda10 2023-10-11T09:42:00.154 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:00.155 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":36,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:41:59.224526+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:41:57.280079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-11T09:41:58.218860+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:42:00.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.145+0000 7f6bf988c700 1 -- 172.21.15.78:0/2556619677 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6be0065ed0 msgr2=0x7f6be0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:00.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.145+0000 7f6bf988c700 1 --2- 172.21.15.78:0/2556619677 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6be0065ed0 0x7f6be0068380 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4160140 tx=0x7f6be4008040 comp rx=0 tx=0).stop 2023-10-11T09:42:00.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.145+0000 7f6bf988c700 1 -- 172.21.15.78:0/2556619677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bf40cda10 msgr2=0x7f6bf4161980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:00.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.145+0000 7f6bf988c700 1 --2- 172.21.15.78:0/2556619677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bf40cda10 0x7f6bf4161980 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4072690 tx=0x7f6bf0004770 comp rx=0 tx=0).stop 2023-10-11T09:42:00.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.145+0000 7f6bf988c700 1 -- 172.21.15.78:0/2556619677 shutdown_connections 2023-10-11T09:42:00.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.145+0000 7f6bf988c700 1 --2- 172.21.15.78:0/2556619677 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6be0065ed0 0x7f6be0068380 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:00.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.145+0000 7f6bf988c700 1 --2- 172.21.15.78:0/2556619677 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6bf4164b50 0x7f6bf41497b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:00.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.145+0000 7f6bf988c700 1 --2- 172.21.15.78:0/2556619677 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6bf40cdde0 0x7f6bf40b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:00.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.145+0000 7f6bf988c700 1 --2- 172.21.15.78:0/2556619677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bf40cda10 0x7f6bf4161980 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:00.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.145+0000 7f6bf988c700 1 -- 172.21.15.78:0/2556619677 >> 172.21.15.78:0/2556619677 conn(0x7f6bf40cea60 msgr2=0x7f6bf4070940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:00.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.145+0000 7f6bf988c700 1 -- 172.21.15.78:0/2556619677 shutdown_connections 2023-10-11T09:42:00.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.145+0000 7f6bf988c700 1 -- 172.21.15.78:0/2556619677 wait complete. 2023-10-11T09:42:00.173 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-11T09:42:00.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.345+0000 7fe33de9f700 1 Processor -- start 2023-10-11T09:42:00.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.345+0000 7fe33de9f700 1 -- start start 2023-10-11T09:42:00.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.349+0000 7fe33de9f700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3380d0590 0x7fe338071950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:00.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.349+0000 7fe33de9f700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe3380b1820 0x7fe3380b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:00.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.349+0000 7fe33de9f700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3380cbe00 con 0x7fe33815a520 2023-10-11T09:42:00.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.349+0000 7fe33de9f700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3380cafe0 con 0x7fe3380b1820 2023-10-11T09:42:00.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.349+0000 7fe33de9f700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3380b21e0 con 0x7fe3380d0590 2023-10-11T09:42:00.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.349+0000 7fe33ce9d700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fe33815a520 0x7fe3380b2b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:58478/0 (socket says 172.21.15.78:58478) 2023-10-11T09:42:00.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.349+0000 7fe33ce9d700 1 -- 172.21.15.78:0/3629373663 learned_addr learned my addr 172.21.15.78:0/3629373663 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:42:00.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.349+0000 7fe33d69e700 1 --2- 172.21.15.78:0/3629373663 >> v2:172.21.15.78:3300/0 conn(0x7fe3380b1820 0x7fe3380b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:00.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.349+0000 7fe32ffff700 1 --2- 172.21.15.78:0/3629373663 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3380d0590 0x7fe338071950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:00.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.349+0000 7fe32f7fe700 1 -- 172.21.15.78:0/3629373663 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3630380321 0 0) 0x7fe3380cbe00 con 0x7fe33815a520 2023-10-11T09:42:00.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.349+0000 7fe33d69e700 1 -- 172.21.15.78:0/3629373663 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3380d0590 msgr2=0x7fe338071950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:00.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.349+0000 7fe33d69e700 1 --2- 172.21.15.78:0/3629373663 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3380d0590 0x7fe338071950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:00.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.349+0000 7fe33d69e700 1 -- 172.21.15.78:0/3629373663 >> v1:172.21.15.78:6789/0 conn(0x7fe33815a520 legacy=0x7fe3380b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:00.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.349+0000 7fe33d69e700 1 -- 172.21.15.78:0/3629373663 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3380cb1f0 con 0x7fe3380b1820 2023-10-11T09:42:00.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.349+0000 7fe33d69e700 1 --2- 172.21.15.78:0/3629373663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3380b1820 0x7fe3380b1ca0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe330009850 tx=0x7fe330009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab5baab870de7fcb server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:00.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.349+0000 7fe32f7fe700 1 -- 172.21.15.78:0/3629373663 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe330011830 con 0x7fe3380b1820 2023-10-11T09:42:00.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.349+0000 7fe32f7fe700 1 -- 172.21.15.78:0/3629373663 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe330011990 con 0x7fe3380b1820 2023-10-11T09:42:00.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.349+0000 7fe32f7fe700 1 -- 172.21.15.78:0/3629373663 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe330011d20 con 0x7fe3380b1820 2023-10-11T09:42:00.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.349+0000 7fe33de9f700 1 -- 172.21.15.78:0/3629373663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3380b1820 msgr2=0x7fe3380b1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:00.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.349+0000 7fe33de9f700 1 --2- 172.21.15.78:0/3629373663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3380b1820 0x7fe3380b1ca0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe330009850 tx=0x7fe330009b60 comp rx=0 tx=0).stop 2023-10-11T09:42:00.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.349+0000 7fe33de9f700 1 -- 172.21.15.78:0/3629373663 shutdown_connections 2023-10-11T09:42:00.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.349+0000 7fe33de9f700 1 --2- 172.21.15.78:0/3629373663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3380b1820 0x7fe3380b1ca0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:00.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.349+0000 7fe33de9f700 1 --2- 172.21.15.78:0/3629373663 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3380d0590 0x7fe338071950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:00.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.349+0000 7fe33de9f700 1 -- 172.21.15.78:0/3629373663 >> 172.21.15.78:0/3629373663 conn(0x7fe33806dd80 msgr2=0x7fe33806e180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:00.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.349+0000 7fe33de9f700 1 -- 172.21.15.78:0/3629373663 shutdown_connections 2023-10-11T09:42:00.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.349+0000 7fe33de9f700 1 -- 172.21.15.78:0/3629373663 wait complete. 2023-10-11T09:42:00.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.353+0000 7fe33de9f700 1 Processor -- start 2023-10-11T09:42:00.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.353+0000 7fe33de9f700 1 -- start start 2023-10-11T09:42:00.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.353+0000 7fe33de9f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3380b1820 0x7fe33815d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:00.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.353+0000 7fe33de9f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3380d0590 0x7fe33814da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:00.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.353+0000 7fe33de9f700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe33815a520 0x7fe3381403d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:00.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.353+0000 7fe33de9f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33814df40 con 0x7fe3380b1820 2023-10-11T09:42:00.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.353+0000 7fe33de9f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33815b9e0 con 0x7fe3380d0590 2023-10-11T09:42:00.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.353+0000 7fe33de9f700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe33814e130 con 0x7fe33815a520 2023-10-11T09:42:00.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.353+0000 7fe33ce9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3380b1820 0x7fe33815d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:00.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.357+0000 7fe32ffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3380d0590 0x7fe33814da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:00.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.357+0000 7fe33ce9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3380b1820 0x7fe33815d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57176/0 (socket says 172.21.15.78:57176) 2023-10-11T09:42:00.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.357+0000 7fe33ce9d700 1 -- 172.21.15.78:0/3674038184 learned_addr learned my addr 172.21.15.78:0/3674038184 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:00.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.357+0000 7fe33d69e700 1 --2- 172.21.15.78:0/3674038184 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe33815a520 0x7fe3381403d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:00.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.357+0000 7fe33ce9d700 1 -- 172.21.15.78:0/3674038184 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe33815a520 msgr2=0x7fe3381403d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:00.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.357+0000 7fe33ce9d700 1 --2- 172.21.15.78:0/3674038184 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe33815a520 0x7fe3381403d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:00.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.357+0000 7fe33ce9d700 1 -- 172.21.15.78:0/3674038184 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3380d0590 msgr2=0x7fe33814da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:00.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.357+0000 7fe33ce9d700 1 --2- 172.21.15.78:0/3674038184 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3380d0590 0x7fe33814da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:00.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.357+0000 7fe33ce9d700 1 -- 172.21.15.78:0/3674038184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3300094b0 con 0x7fe3380b1820 2023-10-11T09:42:00.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.357+0000 7fe33ce9d700 1 --2- 172.21.15.78:0/3674038184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3380b1820 0x7fe33815d120 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe334001f80 tx=0x7fe33400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:00.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.357+0000 7fe32d7fa700 1 -- 172.21.15.78:0/3674038184 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33400acd0 con 0x7fe3380b1820 2023-10-11T09:42:00.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.357+0000 7fe32d7fa700 1 -- 172.21.15.78:0/3674038184 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe33400ae50 con 0x7fe3380b1820 2023-10-11T09:42:00.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.357+0000 7fe33de9f700 1 -- 172.21.15.78:0/3674038184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe33814e410 con 0x7fe3380b1820 2023-10-11T09:42:00.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.357+0000 7fe32d7fa700 1 -- 172.21.15.78:0/3674038184 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe334007310 con 0x7fe3380b1820 2023-10-11T09:42:00.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.357+0000 7fe33de9f700 1 -- 172.21.15.78:0/3674038184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe33814e9d0 con 0x7fe3380b1820 2023-10-11T09:42:00.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.357+0000 7fe33de9f700 1 -- 172.21.15.78:0/3674038184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe33815bbe0 con 0x7fe3380b1820 2023-10-11T09:42:00.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.361+0000 7fe32d7fa700 1 -- 172.21.15.78:0/3674038184 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fe334007470 con 0x7fe3380b1820 2023-10-11T09:42:00.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.361+0000 7fe32d7fa700 1 --2- 172.21.15.78:0/3674038184 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe324065ed0 0x7fe324068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:00.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.361+0000 7fe32d7fa700 1 -- 172.21.15.78:0/3674038184 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe334083240 con 0x7fe3380b1820 2023-10-11T09:42:00.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.361+0000 7fe32ffff700 1 --2- 172.21.15.78:0/3674038184 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe324065ed0 0x7fe324068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:00.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.361+0000 7fe32ffff700 1 --2- 172.21.15.78:0/3674038184 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe324065ed0 0x7fe324068380 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe338140030 tx=0x7fe328008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:00.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.365+0000 7fe32d7fa700 1 -- 172.21.15.78:0/3674038184 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fe33815bbe0 con 0x7fe3380b1820 2023-10-11T09:42:00.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:00.577+0000 7fe33de9f700 1 -- 172.21.15.78:0/3674038184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe338065d40 con 0x7fe3380b1820 2023-10-11T09:42:01.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.249+0000 7fe32d7fa700 1 -- 172.21.15.78:0/3674038184 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe334050880 con 0x7fe3380b1820 2023-10-11T09:42:01.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.253+0000 7fe33de9f700 1 -- 172.21.15.78:0/3674038184 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe324065ed0 msgr2=0x7fe324068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:01.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.253+0000 7fe33de9f700 1 --2- 172.21.15.78:0/3674038184 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe324065ed0 0x7fe324068380 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe338140030 tx=0x7fe328008040 comp rx=0 tx=0).stop 2023-10-11T09:42:01.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.253+0000 7fe33de9f700 1 -- 172.21.15.78:0/3674038184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3380b1820 msgr2=0x7fe33815d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:01.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.253+0000 7fe33de9f700 1 --2- 172.21.15.78:0/3674038184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3380b1820 0x7fe33815d120 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe334001f80 tx=0x7fe33400a340 comp rx=0 tx=0).stop 2023-10-11T09:42:01.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.253+0000 7fe33de9f700 1 -- 172.21.15.78:0/3674038184 shutdown_connections 2023-10-11T09:42:01.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.253+0000 7fe33de9f700 1 --2- 172.21.15.78:0/3674038184 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe324065ed0 0x7fe324068380 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:01.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.253+0000 7fe33de9f700 1 --2- 172.21.15.78:0/3674038184 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe33815a520 0x7fe3381403d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:01.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.253+0000 7fe33de9f700 1 --2- 172.21.15.78:0/3674038184 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3380d0590 0x7fe33814da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:01.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.253+0000 7fe33de9f700 1 --2- 172.21.15.78:0/3674038184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3380b1820 0x7fe33815d120 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:01.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.253+0000 7fe33de9f700 1 -- 172.21.15.78:0/3674038184 >> 172.21.15.78:0/3674038184 conn(0x7fe33806dd80 msgr2=0x7fe33806ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:01.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.253+0000 7fe33de9f700 1 -- 172.21.15.78:0/3674038184 shutdown_connections 2023-10-11T09:42:01.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.253+0000 7fe33de9f700 1 -- 172.21.15.78:0/3674038184 wait complete. 2023-10-11T09:42:01.285 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-11T09:42:01.285 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-11T09:42:01.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.465+0000 7f33ab53e700 1 Processor -- start 2023-10-11T09:42:01.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.465+0000 7f33ab53e700 1 -- start start 2023-10-11T09:42:01.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.465+0000 7f33ab53e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f33a40b3610 0x7f33a40d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:01.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.465+0000 7f33ab53e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f33a40b3a90 0x7f33a40b85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:01.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.465+0000 7f33ab53e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a40b47f0 con 0x7f33a40b3240 2023-10-11T09:42:01.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.465+0000 7f33ab53e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33a40b8130 con 0x7f33a40b3a90 2023-10-11T09:42:01.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.465+0000 7f33ab53e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33a40b41e0 con 0x7f33a40b3610 2023-10-11T09:42:01.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.465+0000 7f33aa53c700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f33a40b3240 0x7f33a415d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:58488/0 (socket says 172.21.15.78:58488) 2023-10-11T09:42:01.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.465+0000 7f33aa53c700 1 -- 172.21.15.78:0/485958608 learned_addr learned my addr 172.21.15.78:0/485958608 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:42:01.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.465+0000 7f33aad3d700 1 --2- 172.21.15.78:0/485958608 >> v2:172.21.15.78:3300/0 conn(0x7f33a40b3a90 0x7f33a40b85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:01.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.465+0000 7f33a9d3b700 1 --2- 172.21.15.78:0/485958608 >> v2:172.21.15.133:3300/0 conn(0x7f33a40b3610 0x7f33a40d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:01.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.465+0000 7f33aad3d700 1 -- 172.21.15.78:0/485958608 >> v2:172.21.15.133:3300/0 conn(0x7f33a40b3610 msgr2=0x7f33a40d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:01.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.465+0000 7f33aad3d700 1 --2- 172.21.15.78:0/485958608 >> v2:172.21.15.133:3300/0 conn(0x7f33a40b3610 0x7f33a40d29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:01.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.465+0000 7f33aad3d700 1 -- 172.21.15.78:0/485958608 >> v1:172.21.15.78:6789/0 conn(0x7f33a40b3240 legacy=0x7f33a415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:01.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.465+0000 7f33aad3d700 1 -- 172.21.15.78:0/485958608 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a40b7800 con 0x7f33a40b3a90 2023-10-11T09:42:01.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.469+0000 7f33aad3d700 1 --2- 172.21.15.78:0/485958608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33a40b3a90 0x7f33a40b85c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f33a40d0a90 tx=0x7f339c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a5156729ae3446 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:01.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.469+0000 7f33a953a700 1 -- 172.21.15.78:0/485958608 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339c00e070 con 0x7f33a40b3a90 2023-10-11T09:42:01.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.469+0000 7f33a953a700 1 -- 172.21.15.78:0/485958608 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f339c00bb20 con 0x7f33a40b3a90 2023-10-11T09:42:01.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.469+0000 7f33a953a700 1 -- 172.21.15.78:0/485958608 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339c012430 con 0x7f33a40b3a90 2023-10-11T09:42:01.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.469+0000 7f33ab53e700 1 -- 172.21.15.78:0/485958608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33a40b3a90 msgr2=0x7f33a40b85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:01.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.469+0000 7f33ab53e700 1 --2- 172.21.15.78:0/485958608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33a40b3a90 0x7f33a40b85c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f33a40d0a90 tx=0x7f339c00b330 comp rx=0 tx=0).stop 2023-10-11T09:42:01.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.469+0000 7f33ab53e700 1 -- 172.21.15.78:0/485958608 shutdown_connections 2023-10-11T09:42:01.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.469+0000 7f33ab53e700 1 --2- 172.21.15.78:0/485958608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33a40b3a90 0x7f33a40b85c0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:01.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.469+0000 7f33ab53e700 1 --2- 172.21.15.78:0/485958608 >> v2:172.21.15.133:3300/0 conn(0x7f33a40b3610 0x7f33a40d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:01.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.469+0000 7f33ab53e700 1 -- 172.21.15.78:0/485958608 >> 172.21.15.78:0/485958608 conn(0x7f33a409c260 msgr2=0x7f33a40d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:01.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.469+0000 7f33ab53e700 1 -- 172.21.15.78:0/485958608 shutdown_connections 2023-10-11T09:42:01.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.469+0000 7f33ab53e700 1 -- 172.21.15.78:0/485958608 wait complete. 2023-10-11T09:42:01.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.469+0000 7f33ab53e700 1 Processor -- start 2023-10-11T09:42:01.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.469+0000 7f33ab53e700 1 -- start start 2023-10-11T09:42:01.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.473+0000 7f33ab53e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f33a40b3240 0x7f33a415f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:01.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.473+0000 7f33ab53e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33a40b3610 0x7f33a414b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:01.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.473+0000 7f33ab53e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33a40b3a90 0x7f33a415df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:01.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.473+0000 7f33ab53e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a415f470 con 0x7f33a40b3610 2023-10-11T09:42:01.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.473+0000 7f33ab53e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a415e7f0 con 0x7f33a40b3a90 2023-10-11T09:42:01.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.473+0000 7f33ab53e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a415ead0 con 0x7f33a40b3240 2023-10-11T09:42:01.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.473+0000 7f33a9d3b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33a40b3610 0x7f33a414b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:01.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.473+0000 7f33a9d3b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33a40b3610 0x7f33a414b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57198/0 (socket says 172.21.15.78:57198) 2023-10-11T09:42:01.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.473+0000 7f33a9d3b700 1 -- 172.21.15.78:0/2180204067 learned_addr learned my addr 172.21.15.78:0/2180204067 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:01.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.473+0000 7f33aad3d700 1 --2- 172.21.15.78:0/2180204067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33a40b3a90 0x7f33a415df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:01.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.473+0000 7f33aa53c700 1 --2- 172.21.15.78:0/2180204067 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f33a40b3240 0x7f33a415f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:01.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.473+0000 7f33a9d3b700 1 -- 172.21.15.78:0/2180204067 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f33a40b3240 msgr2=0x7f33a415f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:01.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.473+0000 7f33a9d3b700 1 --2- 172.21.15.78:0/2180204067 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f33a40b3240 0x7f33a415f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:01.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.473+0000 7f33a9d3b700 1 -- 172.21.15.78:0/2180204067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33a40b3a90 msgr2=0x7f33a415df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:01.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.473+0000 7f33a9d3b700 1 --2- 172.21.15.78:0/2180204067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33a40b3a90 0x7f33a415df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:01.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.473+0000 7f33a9d3b700 1 -- 172.21.15.78:0/2180204067 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f339c009a70 con 0x7f33a40b3610 2023-10-11T09:42:01.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.473+0000 7f33a9d3b700 1 --2- 172.21.15.78:0/2180204067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33a40b3610 0x7f33a414b540 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f339400e840 tx=0x7f339400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:01.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.473+0000 7f339affd700 1 -- 172.21.15.78:0/2180204067 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33940095a0 con 0x7f33a40b3610 2023-10-11T09:42:01.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.473+0000 7f339affd700 1 -- 172.21.15.78:0/2180204067 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f339400c690 con 0x7f33a40b3610 2023-10-11T09:42:01.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.473+0000 7f33ab53e700 1 -- 172.21.15.78:0/2180204067 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a415edb0 con 0x7f33a40b3610 2023-10-11T09:42:01.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.473+0000 7f339affd700 1 -- 172.21.15.78:0/2180204067 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339400ca00 con 0x7f33a40b3610 2023-10-11T09:42:01.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.473+0000 7f33ab53e700 1 -- 172.21.15.78:0/2180204067 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a415e470 con 0x7f33a40b3610 2023-10-11T09:42:01.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.477+0000 7f33ab53e700 1 -- 172.21.15.78:0/2180204067 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a415f5b0 con 0x7f33a40b3610 2023-10-11T09:42:01.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.477+0000 7f339affd700 1 -- 172.21.15.78:0/2180204067 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f339400cb60 con 0x7f33a40b3610 2023-10-11T09:42:01.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.477+0000 7f339affd700 1 --2- 172.21.15.78:0/2180204067 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3388065ed0 0x7f3388068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:01.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.477+0000 7f339affd700 1 -- 172.21.15.78:0/2180204067 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f33940837f0 con 0x7f33a40b3610 2023-10-11T09:42:01.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.477+0000 7f33aa53c700 1 --2- 172.21.15.78:0/2180204067 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3388065ed0 0x7f3388068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:01.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.477+0000 7f33aa53c700 1 --2- 172.21.15.78:0/2180204067 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3388065ed0 0x7f3388068380 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f33a0005930 tx=0x7f33a0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:01.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.485+0000 7f339affd700 1 -- 172.21.15.78:0/2180204067 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f33a415f5b0 con 0x7f33a40b3610 2023-10-11T09:42:01.706 INFO:tasks.ceph.mds.b.smithi133.stderr:2023-10-11T09:42:01.702+0000 7feebb877700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-11T09:42:01.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:01.701+0000 7f33ab53e700 1 -- 172.21.15.78:0/2180204067 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f33a415f5b0 con 0x7f33a40b3610 2023-10-11T09:42:02.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.257+0000 7f339affd700 1 -- 172.21.15.78:0/2180204067 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f33a415f5b0 con 0x7f33a40b3610 2023-10-11T09:42:02.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.261+0000 7f33ab53e700 1 -- 172.21.15.78:0/2180204067 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3388065ed0 msgr2=0x7f3388068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:02.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.261+0000 7f33ab53e700 1 --2- 172.21.15.78:0/2180204067 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3388065ed0 0x7f3388068380 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f33a0005930 tx=0x7f33a0009040 comp rx=0 tx=0).stop 2023-10-11T09:42:02.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.261+0000 7f33ab53e700 1 -- 172.21.15.78:0/2180204067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33a40b3610 msgr2=0x7f33a414b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:02.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.261+0000 7f33ab53e700 1 --2- 172.21.15.78:0/2180204067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33a40b3610 0x7f33a414b540 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f339400e840 tx=0x7f339400ec00 comp rx=0 tx=0).stop 2023-10-11T09:42:02.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.261+0000 7f33ab53e700 1 -- 172.21.15.78:0/2180204067 shutdown_connections 2023-10-11T09:42:02.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.261+0000 7f33ab53e700 1 --2- 172.21.15.78:0/2180204067 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3388065ed0 0x7f3388068380 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:02.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.261+0000 7f33ab53e700 1 --2- 172.21.15.78:0/2180204067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33a40b3a90 0x7f33a415df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:02.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.261+0000 7f33ab53e700 1 --2- 172.21.15.78:0/2180204067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33a40b3610 0x7f33a414b540 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:02.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.261+0000 7f33ab53e700 1 --2- 172.21.15.78:0/2180204067 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f33a40b3240 0x7f33a415f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:02.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.261+0000 7f33ab53e700 1 -- 172.21.15.78:0/2180204067 >> 172.21.15.78:0/2180204067 conn(0x7f33a409c260 msgr2=0x7f33a415c440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:02.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.261+0000 7f33ab53e700 1 -- 172.21.15.78:0/2180204067 shutdown_connections 2023-10-11T09:42:02.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.261+0000 7f33ab53e700 1 -- 172.21.15.78:0/2180204067 wait complete. 2023-10-11T09:42:02.293 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-11T09:42:02.294 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:42:02.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.465+0000 7eff6da7e700 1 Processor -- start 2023-10-11T09:42:02.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.465+0000 7eff6da7e700 1 -- start start 2023-10-11T09:42:02.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.465+0000 7eff6da7e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff680cda10 0x7eff680caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:02.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.465+0000 7eff6da7e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7eff68164d20 0x7eff6806bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:02.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.465+0000 7eff6da7e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff68165390 con 0x7eff680cdde0 2023-10-11T09:42:02.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.465+0000 7eff6da7e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff681651a0 con 0x7eff68164d20 2023-10-11T09:42:02.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.465+0000 7eff6da7e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff680ca440 con 0x7eff680cda10 2023-10-11T09:42:02.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.465+0000 7eff6d27d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7eff68164d20 0x7eff6806bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:02.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.469+0000 7eff6d27d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7eff68164d20 0x7eff6806bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57208/0 (socket says 172.21.15.78:57208) 2023-10-11T09:42:02.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.469+0000 7eff6d27d700 1 -- 172.21.15.78:0/550945656 learned_addr learned my addr 172.21.15.78:0/550945656 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:02.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.469+0000 7eff6ca7c700 1 --2- 172.21.15.78:0/550945656 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff680cda10 0x7eff680caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:02.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.469+0000 7eff6d27d700 1 -- 172.21.15.78:0/550945656 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff680cda10 msgr2=0x7eff680caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:02.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.469+0000 7eff6d27d700 1 --2- 172.21.15.78:0/550945656 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff680cda10 0x7eff680caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:02.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.469+0000 7eff6d27d700 1 -- 172.21.15.78:0/550945656 >> v1:172.21.15.133:6789/0 conn(0x7eff680cdde0 legacy=0x7eff68164610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:02.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.469+0000 7eff6d27d700 1 -- 172.21.15.78:0/550945656 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff680ca720 con 0x7eff68164d20 2023-10-11T09:42:02.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.469+0000 7eff6d27d700 1 --2- 172.21.15.78:0/550945656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff68164d20 0x7eff6806bbf0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7eff60009910 tx=0x7eff60009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a973c9343b708009 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:02.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.469+0000 7eff677fe700 1 -- 172.21.15.78:0/550945656 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6000ba60 con 0x7eff68164d20 2023-10-11T09:42:02.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.469+0000 7eff677fe700 1 -- 172.21.15.78:0/550945656 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff6000bbc0 con 0x7eff68164d20 2023-10-11T09:42:02.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.469+0000 7eff677fe700 1 -- 172.21.15.78:0/550945656 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff60009e60 con 0x7eff68164d20 2023-10-11T09:42:02.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.469+0000 7eff6da7e700 1 -- 172.21.15.78:0/550945656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff68164d20 msgr2=0x7eff6806bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:02.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.469+0000 7eff6da7e700 1 --2- 172.21.15.78:0/550945656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff68164d20 0x7eff6806bbf0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7eff60009910 tx=0x7eff60009c20 comp rx=0 tx=0).stop 2023-10-11T09:42:02.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.469+0000 7eff6da7e700 1 -- 172.21.15.78:0/550945656 shutdown_connections 2023-10-11T09:42:02.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.469+0000 7eff6da7e700 1 --2- 172.21.15.78:0/550945656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff68164d20 0x7eff6806bbf0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:02.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.469+0000 7eff6da7e700 1 --2- 172.21.15.78:0/550945656 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff680cda10 0x7eff680caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:02.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.469+0000 7eff6da7e700 1 -- 172.21.15.78:0/550945656 >> 172.21.15.78:0/550945656 conn(0x7eff680cea60 msgr2=0x7eff680cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:02.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.469+0000 7eff6da7e700 1 -- 172.21.15.78:0/550945656 shutdown_connections 2023-10-11T09:42:02.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.469+0000 7eff6da7e700 1 -- 172.21.15.78:0/550945656 wait complete. 2023-10-11T09:42:02.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.473+0000 7eff6da7e700 1 Processor -- start 2023-10-11T09:42:02.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.473+0000 7eff6da7e700 1 -- start start 2023-10-11T09:42:02.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.473+0000 7eff6da7e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff680cda10 0x7eff68146cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:02.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.473+0000 7eff6da7e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff680cdde0 0x7eff68163980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:02.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.473+0000 7eff6da7e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff68164d20 0x7eff6815ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:02.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.473+0000 7eff6da7e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff68149740 con 0x7eff680cda10 2023-10-11T09:42:02.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.473+0000 7eff6da7e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff68161010 con 0x7eff680cdde0 2023-10-11T09:42:02.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.473+0000 7eff6da7e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff68161b50 con 0x7eff68164d20 2023-10-11T09:42:02.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.473+0000 7eff6ca7c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff680cda10 0x7eff68146cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:02.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.473+0000 7eff6ca7c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff680cda10 0x7eff68146cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57210/0 (socket says 172.21.15.78:57210) 2023-10-11T09:42:02.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.473+0000 7eff6d27d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff68164d20 0x7eff6815ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:02.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.473+0000 7eff6ca7c700 1 -- 172.21.15.78:0/3392496116 learned_addr learned my addr 172.21.15.78:0/3392496116 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:02.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.473+0000 7eff67fff700 1 --2- 172.21.15.78:0/3392496116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff680cdde0 0x7eff68163980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:02.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.473+0000 7eff6ca7c700 1 -- 172.21.15.78:0/3392496116 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff68164d20 msgr2=0x7eff6815ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:02.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.473+0000 7eff6ca7c700 1 --2- 172.21.15.78:0/3392496116 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff68164d20 0x7eff6815ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:02.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.473+0000 7eff6ca7c700 1 -- 172.21.15.78:0/3392496116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff680cdde0 msgr2=0x7eff68163980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:02.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.473+0000 7eff6ca7c700 1 --2- 172.21.15.78:0/3392496116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff680cdde0 0x7eff68163980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:02.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.473+0000 7eff6ca7c700 1 -- 172.21.15.78:0/3392496116 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff60009570 con 0x7eff680cda10 2023-10-11T09:42:02.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.477+0000 7eff6ca7c700 1 --2- 172.21.15.78:0/3392496116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff680cda10 0x7eff68146cb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7eff5c007ec0 tx=0x7eff5c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:02.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.477+0000 7eff657fa700 1 -- 172.21.15.78:0/3392496116 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c00eea0 con 0x7eff680cda10 2023-10-11T09:42:02.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.477+0000 7eff657fa700 1 -- 172.21.15.78:0/3392496116 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5c014420 con 0x7eff680cda10 2023-10-11T09:42:02.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.477+0000 7eff6da7e700 1 -- 172.21.15.78:0/3392496116 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff680ce3b0 con 0x7eff680cda10 2023-10-11T09:42:02.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.477+0000 7eff657fa700 1 -- 172.21.15.78:0/3392496116 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c0147b0 con 0x7eff680cda10 2023-10-11T09:42:02.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.477+0000 7eff6da7e700 1 -- 172.21.15.78:0/3392496116 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff68161500 con 0x7eff680cda10 2023-10-11T09:42:02.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.477+0000 7eff6da7e700 1 -- 172.21.15.78:0/3392496116 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff68065d40 con 0x7eff680cda10 2023-10-11T09:42:02.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.481+0000 7eff657fa700 1 -- 172.21.15.78:0/3392496116 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7eff5c014910 con 0x7eff680cda10 2023-10-11T09:42:02.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.481+0000 7eff657fa700 1 --2- 172.21.15.78:0/3392496116 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7eff4c065ed0 0x7eff4c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:02.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.481+0000 7eff657fa700 1 -- 172.21.15.78:0/3392496116 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7eff5c0840a0 con 0x7eff680cda10 2023-10-11T09:42:02.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.481+0000 7eff67fff700 1 --2- 172.21.15.78:0/3392496116 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7eff4c065ed0 0x7eff4c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:02.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.481+0000 7eff67fff700 1 --2- 172.21.15.78:0/3392496116 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7eff4c065ed0 0x7eff4c068380 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7eff680ae890 tx=0x7eff54009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:02.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.489+0000 7eff657fa700 1 -- 172.21.15.78:0/3392496116 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7eff5c051650 con 0x7eff680cda10 2023-10-11T09:42:02.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.693+0000 7eff6da7e700 1 -- 172.21.15.78:0/3392496116 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff680b1480 con 0x7eff680cda10 2023-10-11T09:42:02.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.693+0000 7eff657fa700 1 -- 172.21.15.78:0/3392496116 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4648 (secure 0 0 0) 0x7eff5c054c70 con 0x7eff680cda10 2023-10-11T09:42:02.700 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:02.700 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5308,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/655413753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":655413753},{"type":"v1","addr":"172.21.15.133:6835","nonce":655413753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5352,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2966622437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2966622437},{"type":"v1","addr":"172.21.15.78:6835","nonce":2966622437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5355,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/700246445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":700246445},{"type":"v1","addr":"172.21.15.78:6837","nonce":700246445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:41:59.225318+0000","modified":"2023-10-11T09:42:02.252787+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5364},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5364":{"gid":5364,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.133:6833/1862542010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1862542010},{"type":"v1","addr":"172.21.15.133:6833","nonce":1862542010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-11T09:42:02.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.697+0000 7eff6da7e700 1 -- 172.21.15.78:0/3392496116 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7eff4c065ed0 msgr2=0x7eff4c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:02.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.697+0000 7eff6da7e700 1 --2- 172.21.15.78:0/3392496116 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7eff4c065ed0 0x7eff4c068380 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7eff680ae890 tx=0x7eff54009040 comp rx=0 tx=0).stop 2023-10-11T09:42:02.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.697+0000 7eff6da7e700 1 -- 172.21.15.78:0/3392496116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff680cda10 msgr2=0x7eff68146cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:02.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.697+0000 7eff6da7e700 1 --2- 172.21.15.78:0/3392496116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff680cda10 0x7eff68146cb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7eff5c007ec0 tx=0x7eff5c00e3c0 comp rx=0 tx=0).stop 2023-10-11T09:42:02.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.697+0000 7eff6da7e700 1 -- 172.21.15.78:0/3392496116 shutdown_connections 2023-10-11T09:42:02.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.697+0000 7eff6da7e700 1 --2- 172.21.15.78:0/3392496116 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7eff4c065ed0 0x7eff4c068380 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:02.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.697+0000 7eff6da7e700 1 --2- 172.21.15.78:0/3392496116 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff68164d20 0x7eff6815ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:02.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.697+0000 7eff6da7e700 1 --2- 172.21.15.78:0/3392496116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff680cdde0 0x7eff68163980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:02.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.697+0000 7eff6da7e700 1 --2- 172.21.15.78:0/3392496116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff680cda10 0x7eff68146cb0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:02.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.697+0000 7eff6da7e700 1 -- 172.21.15.78:0/3392496116 >> 172.21.15.78:0/3392496116 conn(0x7eff680cea60 msgr2=0x7eff6806fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:02.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.697+0000 7eff6da7e700 1 -- 172.21.15.78:0/3392496116 shutdown_connections 2023-10-11T09:42:02.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.697+0000 7eff6da7e700 1 -- 172.21.15.78:0/3392496116 wait complete. 2023-10-11T09:42:02.707 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 16 2023-10-11T09:42:02.724 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:42:02.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.901+0000 7f3824192700 1 Processor -- start 2023-10-11T09:42:02.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.905+0000 7f3824192700 1 -- start start 2023-10-11T09:42:02.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.905+0000 7f3824192700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f381c160d40 0x7f381c0cd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:02.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.905+0000 7f3824192700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f381c071040 0x7f381c0cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:02.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.905+0000 7f3824192700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f381c06ffa0 con 0x7f381c0cd7c0 2023-10-11T09:42:02.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.905+0000 7f3824192700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f381c070c20 con 0x7f381c160d40 2023-10-11T09:42:02.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.905+0000 7f3824192700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f381c161110 con 0x7f381c071040 2023-10-11T09:42:02.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.905+0000 7f3821f2e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f381c160d40 0x7f381c0cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:02.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.905+0000 7f3821f2e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f381c160d40 0x7f381c0cd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57218/0 (socket says 172.21.15.78:57218) 2023-10-11T09:42:02.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.905+0000 7f3821f2e700 1 -- 172.21.15.78:0/1144510612 learned_addr learned my addr 172.21.15.78:0/1144510612 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:02.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.905+0000 7f3821f2e700 1 -- 172.21.15.78:0/1144510612 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f381c071040 msgr2=0x7f381c0cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:02.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.905+0000 7f3821f2e700 1 --2- 172.21.15.78:0/1144510612 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f381c071040 0x7f381c0cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:02.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.905+0000 7f3821f2e700 1 -- 172.21.15.78:0/1144510612 >> v1:172.21.15.133:6789/0 conn(0x7f381c0cd7c0 legacy=0x7f381c1647f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:02.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.905+0000 7f3821f2e700 1 -- 172.21.15.78:0/1144510612 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f381c070280 con 0x7f381c160d40 2023-10-11T09:42:02.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.905+0000 7f3821f2e700 1 --2- 172.21.15.78:0/1144510612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f381c160d40 0x7f381c0cd280 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3818009e50 tx=0x7f381800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4c164369644cd2 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:02.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.905+0000 7f3820f2c700 1 -- 172.21.15.78:0/1144510612 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381800e070 con 0x7f381c160d40 2023-10-11T09:42:02.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.905+0000 7f3820f2c700 1 -- 172.21.15.78:0/1144510612 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3818003a40 con 0x7f381c160d40 2023-10-11T09:42:02.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.905+0000 7f3820f2c700 1 -- 172.21.15.78:0/1144510612 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3818003df0 con 0x7f381c160d40 2023-10-11T09:42:02.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.909+0000 7f3824192700 1 -- 172.21.15.78:0/1144510612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f381c160d40 msgr2=0x7f381c0cd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:02.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.909+0000 7f3824192700 1 --2- 172.21.15.78:0/1144510612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f381c160d40 0x7f381c0cd280 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3818009e50 tx=0x7f381800b330 comp rx=0 tx=0).stop 2023-10-11T09:42:02.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.909+0000 7f3824192700 1 -- 172.21.15.78:0/1144510612 shutdown_connections 2023-10-11T09:42:02.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.909+0000 7f3824192700 1 --2- 172.21.15.78:0/1144510612 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f381c071040 0x7f381c0cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:02.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.909+0000 7f3824192700 1 --2- 172.21.15.78:0/1144510612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f381c160d40 0x7f381c0cd280 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:02.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.909+0000 7f3824192700 1 -- 172.21.15.78:0/1144510612 >> 172.21.15.78:0/1144510612 conn(0x7f381c06aca0 msgr2=0x7f381c0b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:02.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.909+0000 7f3824192700 1 -- 172.21.15.78:0/1144510612 shutdown_connections 2023-10-11T09:42:02.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.909+0000 7f3824192700 1 -- 172.21.15.78:0/1144510612 wait complete. 2023-10-11T09:42:02.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.913+0000 7f3824192700 1 Processor -- start 2023-10-11T09:42:02.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.913+0000 7f3824192700 1 -- start start 2023-10-11T09:42:02.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.913+0000 7f3824192700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f381c071040 0x7f381c15bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:02.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.913+0000 7f3824192700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f381c0cd7c0 0x7f381c1f6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:02.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.913+0000 7f3824192700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f381c160d40 0x7f381c1ea7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:02.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.913+0000 7f3824192700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381c0b1200 con 0x7f381c0cd7c0 2023-10-11T09:42:02.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.913+0000 7f3824192700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381c1ec130 con 0x7f381c071040 2023-10-11T09:42:02.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.913+0000 7f3824192700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f381c15c110 con 0x7f381c160d40 2023-10-11T09:42:02.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.913+0000 7f382172d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f381c0cd7c0 0x7f381c1f6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:02.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.913+0000 7f382172d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f381c0cd7c0 0x7f381c1f6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57232/0 (socket says 172.21.15.78:57232) 2023-10-11T09:42:02.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.913+0000 7f382172d700 1 -- 172.21.15.78:0/2105155337 learned_addr learned my addr 172.21.15.78:0/2105155337 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:02.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.913+0000 7f382272f700 1 --2- 172.21.15.78:0/2105155337 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f381c160d40 0x7f381c1ea7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:02.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.913+0000 7f3821f2e700 1 --2- 172.21.15.78:0/2105155337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f381c071040 0x7f381c15bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:02.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.913+0000 7f382172d700 1 -- 172.21.15.78:0/2105155337 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f381c160d40 msgr2=0x7f381c1ea7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:02.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.913+0000 7f382172d700 1 --2- 172.21.15.78:0/2105155337 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f381c160d40 0x7f381c1ea7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:02.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.913+0000 7f382172d700 1 -- 172.21.15.78:0/2105155337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f381c071040 msgr2=0x7f381c15bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:02.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.913+0000 7f382172d700 1 --2- 172.21.15.78:0/2105155337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f381c071040 0x7f381c15bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:02.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.913+0000 7f382172d700 1 -- 172.21.15.78:0/2105155337 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3818009b30 con 0x7f381c0cd7c0 2023-10-11T09:42:02.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.917+0000 7f382172d700 1 --2- 172.21.15.78:0/2105155337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f381c0cd7c0 0x7f381c1f6ae0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f380c00b550 tx=0x7f380c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:02.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.917+0000 7f38127fc700 1 -- 172.21.15.78:0/2105155337 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c011830 con 0x7f381c0cd7c0 2023-10-11T09:42:02.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.917+0000 7f38127fc700 1 -- 172.21.15.78:0/2105155337 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380c011990 con 0x7f381c0cd7c0 2023-10-11T09:42:02.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.917+0000 7f3824192700 1 -- 172.21.15.78:0/2105155337 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f381c1eb520 con 0x7f381c0cd7c0 2023-10-11T09:42:02.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.917+0000 7f38127fc700 1 -- 172.21.15.78:0/2105155337 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c011d40 con 0x7f381c0cd7c0 2023-10-11T09:42:02.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.917+0000 7f3824192700 1 -- 172.21.15.78:0/2105155337 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f381c1ebae0 con 0x7f381c0cd7c0 2023-10-11T09:42:02.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.917+0000 7f3824192700 1 -- 172.21.15.78:0/2105155337 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f381c072250 con 0x7f381c0cd7c0 2023-10-11T09:42:02.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.921+0000 7f38127fc700 1 -- 172.21.15.78:0/2105155337 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f380c01a400 con 0x7f381c0cd7c0 2023-10-11T09:42:02.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.921+0000 7f38127fc700 1 --2- 172.21.15.78:0/2105155337 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3800065ed0 0x7f3800068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:02.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.921+0000 7f38127fc700 1 -- 172.21.15.78:0/2105155337 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f380c0838b0 con 0x7f381c0cd7c0 2023-10-11T09:42:02.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.921+0000 7f3821f2e700 1 --2- 172.21.15.78:0/2105155337 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3800065ed0 0x7f3800068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:02.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.921+0000 7f3821f2e700 1 --2- 172.21.15.78:0/2105155337 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3800065ed0 0x7f3800068380 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3818015040 tx=0x7f3818009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:02.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:02.925+0000 7f38127fc700 1 -- 172.21.15.78:0/2105155337 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f380c050ef0 con 0x7f381c0cd7c0 2023-10-11T09:42:03.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.101+0000 7f3824192700 1 -- 172.21.15.78:0/2105155337 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f381c1ec2e0 con 0x7f381c0cd7c0 2023-10-11T09:42:03.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.105+0000 7f38127fc700 1 -- 172.21.15.78:0/2105155337 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+17883 (secure 0 0 0) 0x7f381c1ec2e0 con 0x7f381c0cd7c0 2023-10-11T09:42:03.111 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:03.111 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":37,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:42:00.230976+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:41:57.280079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-11T09:41:58.218860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:42:03.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.109+0000 7f3824192700 1 -- 172.21.15.78:0/2105155337 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3800065ed0 msgr2=0x7f3800068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:03.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.109+0000 7f3824192700 1 --2- 172.21.15.78:0/2105155337 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3800065ed0 0x7f3800068380 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3818015040 tx=0x7f3818009000 comp rx=0 tx=0).stop 2023-10-11T09:42:03.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.109+0000 7f3824192700 1 -- 172.21.15.78:0/2105155337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f381c0cd7c0 msgr2=0x7f381c1f6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:03.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.109+0000 7f3824192700 1 --2- 172.21.15.78:0/2105155337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f381c0cd7c0 0x7f381c1f6ae0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f380c00b550 tx=0x7f380c00b910 comp rx=0 tx=0).stop 2023-10-11T09:42:03.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.109+0000 7f3824192700 1 -- 172.21.15.78:0/2105155337 shutdown_connections 2023-10-11T09:42:03.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.109+0000 7f3824192700 1 --2- 172.21.15.78:0/2105155337 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3800065ed0 0x7f3800068380 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:03.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.109+0000 7f3824192700 1 --2- 172.21.15.78:0/2105155337 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f381c160d40 0x7f381c1ea7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:03.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.109+0000 7f3824192700 1 --2- 172.21.15.78:0/2105155337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f381c0cd7c0 0x7f381c1f6ae0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:03.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.109+0000 7f3824192700 1 --2- 172.21.15.78:0/2105155337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f381c071040 0x7f381c15bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:03.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.109+0000 7f3824192700 1 -- 172.21.15.78:0/2105155337 >> 172.21.15.78:0/2105155337 conn(0x7f381c06aca0 msgr2=0x7f381c15f910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:03.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.109+0000 7f3824192700 1 -- 172.21.15.78:0/2105155337 shutdown_connections 2023-10-11T09:42:03.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.109+0000 7f3824192700 1 -- 172.21.15.78:0/2105155337 wait complete. 2023-10-11T09:42:03.140 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-11T09:42:03.140 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-11T09:42:03.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.309+0000 7f8b9f17d700 1 Processor -- start 2023-10-11T09:42:03.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.309+0000 7f8b9f17d700 1 -- start start 2023-10-11T09:42:03.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.313+0000 7f8b9f17d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8b980caf20 0x7f8b98164610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:03.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.313+0000 7f8b9f17d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f8b98164b50 0x7f8b9806bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:03.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.313+0000 7f8b9f17d700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b98165390 con 0x7f8b980cda10 2023-10-11T09:42:03.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.313+0000 7f8b9f17d700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b980cb370 con 0x7f8b98164b50 2023-10-11T09:42:03.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.313+0000 7f8b9f17d700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b980ca440 con 0x7f8b980caf20 2023-10-11T09:42:03.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.313+0000 7f8b9e97c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f8b98164b50 0x7f8b9806bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:03.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.313+0000 7f8b9e97c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f8b98164b50 0x7f8b9806bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57238/0 (socket says 172.21.15.78:57238) 2023-10-11T09:42:03.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.313+0000 7f8b9e97c700 1 -- 172.21.15.78:0/1784966389 learned_addr learned my addr 172.21.15.78:0/1784966389 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:03.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.313+0000 7f8b9d97a700 1 --2- 172.21.15.78:0/1784966389 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8b980caf20 0x7f8b98164610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:03.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.313+0000 7f8b9e97c700 1 -- 172.21.15.78:0/1784966389 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8b980caf20 msgr2=0x7f8b98164610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:03.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.313+0000 7f8b9e97c700 1 --2- 172.21.15.78:0/1784966389 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8b980caf20 0x7f8b98164610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:03.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.313+0000 7f8b9e97c700 1 -- 172.21.15.78:0/1784966389 >> v1:172.21.15.133:6789/0 conn(0x7f8b980cda10 legacy=0x7f8b980cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:03.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.313+0000 7f8b9e97c700 1 -- 172.21.15.78:0/1784966389 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b980ca720 con 0x7f8b98164b50 2023-10-11T09:42:03.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.313+0000 7f8b9e97c700 1 --2- 172.21.15.78:0/1784966389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b98164b50 0x7f8b9806bbf0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f8b90009e50 tx=0x7f8b9000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d06d512c30636a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:03.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.313+0000 7f8b9d179700 1 -- 172.21.15.78:0/1784966389 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9000e070 con 0x7f8b98164b50 2023-10-11T09:42:03.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.313+0000 7f8b9d179700 1 -- 172.21.15.78:0/1784966389 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b9000b9c0 con 0x7f8b98164b50 2023-10-11T09:42:03.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.313+0000 7f8b9d179700 1 -- 172.21.15.78:0/1784966389 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9000bd70 con 0x7f8b98164b50 2023-10-11T09:42:03.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.313+0000 7f8b9f17d700 1 -- 172.21.15.78:0/1784966389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b98164b50 msgr2=0x7f8b9806bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:03.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.313+0000 7f8b9f17d700 1 --2- 172.21.15.78:0/1784966389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b98164b50 0x7f8b9806bbf0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f8b90009e50 tx=0x7f8b9000b330 comp rx=0 tx=0).stop 2023-10-11T09:42:03.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.317+0000 7f8b9f17d700 1 -- 172.21.15.78:0/1784966389 shutdown_connections 2023-10-11T09:42:03.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.317+0000 7f8b9f17d700 1 --2- 172.21.15.78:0/1784966389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b98164b50 0x7f8b9806bbf0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:03.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.317+0000 7f8b9f17d700 1 --2- 172.21.15.78:0/1784966389 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8b980caf20 0x7f8b98164610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:03.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.317+0000 7f8b9f17d700 1 -- 172.21.15.78:0/1784966389 >> 172.21.15.78:0/1784966389 conn(0x7f8b980cea60 msgr2=0x7f8b980cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:03.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.317+0000 7f8b9f17d700 1 -- 172.21.15.78:0/1784966389 shutdown_connections 2023-10-11T09:42:03.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.317+0000 7f8b9f17d700 1 -- 172.21.15.78:0/1784966389 wait complete. 2023-10-11T09:42:03.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.317+0000 7f8b9f17d700 1 Processor -- start 2023-10-11T09:42:03.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.317+0000 7f8b9f17d700 1 -- start start 2023-10-11T09:42:03.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.317+0000 7f8b9f17d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b980caf20 0x7f8b980b3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:03.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.317+0000 7f8b9f17d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b980cda10 0x7f8b98146d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:03.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.317+0000 7f8b9f17d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8b98164b50 0x7f8b981f9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:03.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.317+0000 7f8b9f17d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b980b1de0 con 0x7f8b980cda10 2023-10-11T09:42:03.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.317+0000 7f8b9f17d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b980b2bd0 con 0x7f8b980caf20 2023-10-11T09:42:03.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.317+0000 7f8b9f17d700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b980b35a0 con 0x7f8b98164b50 2023-10-11T09:42:03.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.321+0000 7f8b9e17b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b980caf20 0x7f8b980b3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:03.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.321+0000 7f8b9d97a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b980cda10 0x7f8b98146d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:03.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.321+0000 7f8b9e17b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b980caf20 0x7f8b980b3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:41120/0 (socket says 172.21.15.78:41120) 2023-10-11T09:42:03.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.321+0000 7f8b9e17b700 1 -- 172.21.15.78:0/2053937911 learned_addr learned my addr 172.21.15.78:0/2053937911 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:03.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.321+0000 7f8b9e97c700 1 --2- 172.21.15.78:0/2053937911 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8b98164b50 0x7f8b981f9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:03.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.321+0000 7f8b9d97a700 1 -- 172.21.15.78:0/2053937911 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8b98164b50 msgr2=0x7f8b981f9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:03.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.321+0000 7f8b9d97a700 1 --2- 172.21.15.78:0/2053937911 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8b98164b50 0x7f8b981f9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:03.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.321+0000 7f8b9d97a700 1 -- 172.21.15.78:0/2053937911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b980caf20 msgr2=0x7f8b980b3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:03.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.321+0000 7f8b9d97a700 1 --2- 172.21.15.78:0/2053937911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b980caf20 0x7f8b980b3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:03.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.321+0000 7f8b9d97a700 1 -- 172.21.15.78:0/2053937911 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b90009b30 con 0x7f8b980cda10 2023-10-11T09:42:03.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.321+0000 7f8b9e17b700 1 --2- 172.21.15.78:0/2053937911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b980caf20 0x7f8b980b3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:42:03.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.321+0000 7f8b9d97a700 1 --2- 172.21.15.78:0/2053937911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b980cda10 0x7f8b98146d40 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8b8800e7e0 tx=0x7f8b8800eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:03.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.321+0000 7f8b8effd700 1 -- 172.21.15.78:0/2053937911 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8800c070 con 0x7f8b980cda10 2023-10-11T09:42:03.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.321+0000 7f8b8effd700 1 -- 172.21.15.78:0/2053937911 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b88013610 con 0x7f8b980cda10 2023-10-11T09:42:03.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.321+0000 7f8b8effd700 1 -- 172.21.15.78:0/2053937911 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b880139c0 con 0x7f8b980cda10 2023-10-11T09:42:03.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.321+0000 7f8b9f17d700 1 -- 172.21.15.78:0/2053937911 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b980b1fc0 con 0x7f8b980cda10 2023-10-11T09:42:03.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.321+0000 7f8b9f17d700 1 -- 172.21.15.78:0/2053937911 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b980b2580 con 0x7f8b980cda10 2023-10-11T09:42:03.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.321+0000 7f8b9f17d700 1 -- 172.21.15.78:0/2053937911 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b980b2d40 con 0x7f8b980cda10 2023-10-11T09:42:03.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.325+0000 7f8b8effd700 1 -- 172.21.15.78:0/2053937911 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f8b88013b20 con 0x7f8b980cda10 2023-10-11T09:42:03.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.325+0000 7f8b8effd700 1 --2- 172.21.15.78:0/2053937911 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8b7c065ed0 0x7f8b7c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:03.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.325+0000 7f8b8effd700 1 -- 172.21.15.78:0/2053937911 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8b88083550 con 0x7f8b980cda10 2023-10-11T09:42:03.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.325+0000 7f8b9e17b700 1 --2- 172.21.15.78:0/2053937911 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8b7c065ed0 0x7f8b7c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:03.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.325+0000 7f8b9e17b700 1 --2- 172.21.15.78:0/2053937911 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8b7c065ed0 0x7f8b7c068380 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8b94001d30 tx=0x7f8b94009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:03.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.333+0000 7f8b8effd700 1 -- 172.21.15.78:0/2053937911 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f8b88050b10 con 0x7f8b980cda10 2023-10-11T09:42:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.509+0000 7f8b9f17d700 1 -- 172.21.15.78:0/2053937911 --> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8b980caa00 con 0x7f8b7c065ed0 2023-10-11T09:42:03.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.525+0000 7f8b8effd700 1 -- 172.21.15.78:0/2053937911 <== mgr.4101 v2:172.21.15.78:6832/35606 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+297931 (secure 0 0 0) 0x7f8b9400fa50 con 0x7f8b7c065ed0 2023-10-11T09:42:03.536 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:03.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.529+0000 7f8b9f17d700 1 -- 172.21.15.78:0/2053937911 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8b7c065ed0 msgr2=0x7f8b7c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:03.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.529+0000 7f8b9f17d700 1 --2- 172.21.15.78:0/2053937911 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8b7c065ed0 0x7f8b7c068380 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8b94001d30 tx=0x7f8b94009040 comp rx=0 tx=0).stop 2023-10-11T09:42:03.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.529+0000 7f8b9f17d700 1 -- 172.21.15.78:0/2053937911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b980cda10 msgr2=0x7f8b98146d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:03.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.529+0000 7f8b9f17d700 1 --2- 172.21.15.78:0/2053937911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b980cda10 0x7f8b98146d40 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8b8800e7e0 tx=0x7f8b8800eaf0 comp rx=0 tx=0).stop 2023-10-11T09:42:03.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.533+0000 7f8b9f17d700 1 -- 172.21.15.78:0/2053937911 shutdown_connections 2023-10-11T09:42:03.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.533+0000 7f8b9f17d700 1 --2- 172.21.15.78:0/2053937911 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8b7c065ed0 0x7f8b7c068380 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:03.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.533+0000 7f8b9f17d700 1 --2- 172.21.15.78:0/2053937911 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8b98164b50 0x7f8b981f9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:03.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.533+0000 7f8b9f17d700 1 --2- 172.21.15.78:0/2053937911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b980cda10 0x7f8b98146d40 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:03.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.533+0000 7f8b9f17d700 1 --2- 172.21.15.78:0/2053937911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b980caf20 0x7f8b980b3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:03.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.533+0000 7f8b9f17d700 1 -- 172.21.15.78:0/2053937911 >> 172.21.15.78:0/2053937911 conn(0x7f8b980cea60 msgr2=0x7f8b980ae540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:03.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.533+0000 7f8b9f17d700 1 -- 172.21.15.78:0/2053937911 shutdown_connections 2023-10-11T09:42:03.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.533+0000 7f8b9f17d700 1 -- 172.21.15.78:0/2053937911 wait complete. 2023-10-11T09:42:03.541 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2023-10-11T09:42:03.558 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2023-10-11T09:42:02.346527+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":27,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219772,"kb_used_data":4156,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754948,"statfs":{"total":773094113280,"available":772869066752,"internally_reserved":0,"allocated":4255744,"data_stored":2414398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.607345"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.748611+0000","last_change":"2023-10-11T09:41:57.748611+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.748611+0000","last_undegraded":"2023-10-11T09:41:57.748611+0000","last_fullsized":"2023-10-11T09:41:57.748611+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:34:50.579527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.747892+0000","last_change":"2023-10-11T09:41:57.747892+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.747892+0000","last_undegraded":"2023-10-11T09:41:57.747892+0000","last_fullsized":"2023-10-11T09:41:57.747892+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:00:41.605405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.747589+0000","last_change":"2023-10-11T09:41:57.747589+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.747589+0000","last_undegraded":"2023-10-11T09:41:57.747589+0000","last_fullsized":"2023-10-11T09:41:57.747589+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:16:23.663632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.747803+0000","last_change":"2023-10-11T09:41:57.747803+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.747803+0000","last_undegraded":"2023-10-11T09:41:57.747803+0000","last_fullsized":"2023-10-11T09:41:57.747803+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:30:07.413635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.743430+0000","last_change":"2023-10-11T09:41:57.743430+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.743430+0000","last_undegraded":"2023-10-11T09:41:57.743430+0000","last_fullsized":"2023-10-11T09:41:57.743430+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:52:20.712655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.743800+0000","last_change":"2023-10-11T09:41:57.743800+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.743800+0000","last_undegraded":"2023-10-11T09:41:57.743800+0000","last_fullsized":"2023-10-11T09:41:57.743800+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:50:34.455895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.747388+0000","last_change":"2023-10-11T09:41:57.747388+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.747388+0000","last_undegraded":"2023-10-11T09:41:57.747388+0000","last_fullsized":"2023-10-11T09:41:57.747388+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:25:01.903933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.747795+0000","last_change":"2023-10-11T09:41:57.747795+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.747795+0000","last_undegraded":"2023-10-11T09:41:57.747795+0000","last_fullsized":"2023-10-11T09:41:57.747795+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:00:41.605405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.742844+0000","last_change":"2023-10-11T09:41:57.742844+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.742844+0000","last_undegraded":"2023-10-11T09:41:57.742844+0000","last_fullsized":"2023-10-11T09:41:57.742844+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:48:02.478605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.743207+0000","last_change":"2023-10-11T09:41:57.743207+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.743207+0000","last_undegraded":"2023-10-11T09:41:57.743207+0000","last_fullsized":"2023-10-11T09:41:57.743207+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:24:11.694544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.743022+0000","last_change":"2023-10-11T09:41:57.743022+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.743022+0000","last_undegraded":"2023-10-11T09:41:57.743022+0000","last_fullsized":"2023-10-11T09:41:57.743022+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:55:29.823667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.742745+0000","last_change":"2023-10-11T09:41:57.742745+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.742745+0000","last_undegraded":"2023-10-11T09:41:57.742745+0000","last_fullsized":"2023-10-11T09:41:57.742745+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:23:59.470658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.742711+0000","last_change":"2023-10-11T09:41:57.742711+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.742711+0000","last_undegraded":"2023-10-11T09:41:57.742711+0000","last_fullsized":"2023-10-11T09:41:57.742711+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:24:11.694544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.747798+0000","last_change":"2023-10-11T09:41:57.747798+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.747798+0000","last_undegraded":"2023-10-11T09:41:57.747798+0000","last_fullsized":"2023-10-11T09:41:57.747798+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:18:56.139478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.748556+0000","last_change":"2023-10-11T09:41:57.748556+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.748556+0000","last_undegraded":"2023-10-11T09:41:57.748556+0000","last_fullsized":"2023-10-11T09:41:57.748556+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:02:09.585130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.744176+0000","last_change":"2023-10-11T09:41:57.744176+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.744176+0000","last_undegraded":"2023-10-11T09:41:57.744176+0000","last_fullsized":"2023-10-11T09:41:57.744176+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:05:43.880930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.744324+0000","last_change":"2023-10-11T09:41:57.744324+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.744324+0000","last_undegraded":"2023-10-11T09:41:57.744324+0000","last_fullsized":"2023-10-11T09:41:57.744324+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:11:34.532326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.748087+0000","last_change":"2023-10-11T09:41:57.748087+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.748087+0000","last_undegraded":"2023-10-11T09:41:57.748087+0000","last_fullsized":"2023-10-11T09:41:57.748087+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:42:13.962137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.749372+0000","last_change":"2023-10-11T09:41:57.749372+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.749372+0000","last_undegraded":"2023-10-11T09:41:57.749372+0000","last_fullsized":"2023-10-11T09:41:57.749372+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:49:37.959487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.747393+0000","last_change":"2023-10-11T09:41:57.747393+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.747393+0000","last_undegraded":"2023-10-11T09:41:57.747393+0000","last_fullsized":"2023-10-11T09:41:57.747393+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:35:43.267132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.748695+0000","last_change":"2023-10-11T09:41:57.748695+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.748695+0000","last_undegraded":"2023-10-11T09:41:57.748695+0000","last_fullsized":"2023-10-11T09:41:57.748695+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:55:46.983239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.748522+0000","last_change":"2023-10-11T09:41:57.748522+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.748522+0000","last_undegraded":"2023-10-11T09:41:57.748522+0000","last_fullsized":"2023-10-11T09:41:57.748522+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:44:25.949842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.743871+0000","last_change":"2023-10-11T09:41:57.743871+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.743871+0000","last_undegraded":"2023-10-11T09:41:57.743871+0000","last_fullsized":"2023-10-11T09:41:57.743871+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:09:10.546323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.749170+0000","last_change":"2023-10-11T09:41:57.749170+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.749170+0000","last_undegraded":"2023-10-11T09:41:57.749170+0000","last_fullsized":"2023-10-11T09:41:57.749170+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:40:16.473783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.744185+0000","last_change":"2023-10-11T09:41:57.744185+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.744185+0000","last_undegraded":"2023-10-11T09:41:57.744185+0000","last_fullsized":"2023-10-11T09:41:57.744185+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:23:59.470658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-11T09:41:51.960771+0000","last_change":"2023-10-11T09:40:29.874372+0000","last_active":"2023-10-11T09:41:51.960771+0000","last_peered":"2023-10-11T09:41:51.960771+0000","last_clean":"2023-10-11T09:41:51.960771+0000","last_became_active":"2023-10-11T09:40:29.873621+0000","last_became_peered":"2023-10-11T09:40:29.873621+0000","last_unstale":"2023-10-11T09:41:51.960771+0000","last_undegraded":"2023-10-11T09:41:51.960771+0000","last_fullsized":"2023-10-11T09:41:51.960771+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_clean_scrub_stamp":"2023-10-11T09:40:28.742276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:22:07.985311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-11T09:41:52.378754+0000","last_change":"2023-10-11T09:40:36.148773+0000","last_active":"2023-10-11T09:41:52.378754+0000","last_peered":"2023-10-11T09:41:52.378754+0000","last_clean":"2023-10-11T09:41:52.378754+0000","last_became_active":"2023-10-11T09:40:36.148489+0000","last_became_peered":"2023-10-11T09:40:36.148489+0000","last_unstale":"2023-10-11T09:41:52.378754+0000","last_undegraded":"2023-10-11T09:41:52.378754+0000","last_fullsized":"2023-10-11T09:41:52.378754+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.743190+0000","last_change":"2023-10-11T09:41:57.743190+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.743190+0000","last_undegraded":"2023-10-11T09:41:57.743190+0000","last_fullsized":"2023-10-11T09:41:57.743190+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:58:21.613408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-11T09:41:51.960732+0000","last_change":"2023-10-11T09:40:36.149739+0000","last_active":"2023-10-11T09:41:51.960732+0000","last_peered":"2023-10-11T09:41:51.960732+0000","last_clean":"2023-10-11T09:41:51.960732+0000","last_became_active":"2023-10-11T09:40:36.149419+0000","last_became_peered":"2023-10-11T09:40:36.149419+0000","last_unstale":"2023-10-11T09:41:51.960732+0000","last_undegraded":"2023-10-11T09:41:51.960732+0000","last_fullsized":"2023-10-11T09:41:51.960732+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:54:50.640362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.748743+0000","last_change":"2023-10-11T09:41:57.748743+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.748743+0000","last_undegraded":"2023-10-11T09:41:57.748743+0000","last_fullsized":"2023-10-11T09:41:57.748743+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:41:27.673781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-11T09:41:52.377445+0000","last_change":"2023-10-11T09:40:36.216248+0000","last_active":"2023-10-11T09:41:52.377445+0000","last_peered":"2023-10-11T09:41:52.377445+0000","last_clean":"2023-10-11T09:41:52.377445+0000","last_became_active":"2023-10-11T09:40:36.215890+0000","last_became_peered":"2023-10-11T09:40:36.215890+0000","last_unstale":"2023-10-11T09:41:52.377445+0000","last_undegraded":"2023-10-11T09:41:52.377445+0000","last_fullsized":"2023-10-11T09:41:52.377445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.747308+0000","last_change":"2023-10-11T09:41:57.747308+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.747308+0000","last_undegraded":"2023-10-11T09:41:57.747308+0000","last_fullsized":"2023-10-11T09:41:57.747308+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:55:29.823667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-11T09:41:52.378754+0000","last_change":"2023-10-11T09:40:36.151710+0000","last_active":"2023-10-11T09:41:52.378754+0000","last_peered":"2023-10-11T09:41:52.378754+0000","last_clean":"2023-10-11T09:41:52.378754+0000","last_became_active":"2023-10-11T09:40:36.151219+0000","last_became_peered":"2023-10-11T09:40:36.151219+0000","last_unstale":"2023-10-11T09:41:52.378754+0000","last_undegraded":"2023-10-11T09:41:52.378754+0000","last_fullsized":"2023-10-11T09:41:52.378754+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.747247+0000","last_change":"2023-10-11T09:41:57.747247+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.747247+0000","last_undegraded":"2023-10-11T09:41:57.747247+0000","last_fullsized":"2023-10-11T09:41:57.747247+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:58:21.613408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-11T09:41:52.259702+0000","last_change":"2023-10-11T09:40:36.160620+0000","last_active":"2023-10-11T09:41:52.259702+0000","last_peered":"2023-10-11T09:41:52.259702+0000","last_clean":"2023-10-11T09:41:52.259702+0000","last_became_active":"2023-10-11T09:40:36.160171+0000","last_became_peered":"2023-10-11T09:40:36.160171+0000","last_unstale":"2023-10-11T09:41:52.259702+0000","last_undegraded":"2023-10-11T09:41:52.259702+0000","last_fullsized":"2023-10-11T09:41:52.259702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.748389+0000","last_change":"2023-10-11T09:41:57.748389+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.748389+0000","last_undegraded":"2023-10-11T09:41:57.748389+0000","last_fullsized":"2023-10-11T09:41:57.748389+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:28:16.256141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-11T09:41:51.960723+0000","last_change":"2023-10-11T09:40:36.162174+0000","last_active":"2023-10-11T09:41:51.960723+0000","last_peered":"2023-10-11T09:41:51.960723+0000","last_clean":"2023-10-11T09:41:51.960723+0000","last_became_active":"2023-10-11T09:40:36.161710+0000","last_became_peered":"2023-10-11T09:40:36.161710+0000","last_unstale":"2023-10-11T09:41:51.960723+0000","last_undegraded":"2023-10-11T09:41:51.960723+0000","last_fullsized":"2023-10-11T09:41:51.960723+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:57:03.630878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.747858+0000","last_change":"2023-10-11T09:41:57.747858+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.747858+0000","last_undegraded":"2023-10-11T09:41:57.747858+0000","last_fullsized":"2023-10-11T09:41:57.747858+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:07:42.669679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-11T09:41:52.259702+0000","last_change":"2023-10-11T09:40:36.150224+0000","last_active":"2023-10-11T09:41:52.259702+0000","last_peered":"2023-10-11T09:41:52.259702+0000","last_clean":"2023-10-11T09:41:52.259702+0000","last_became_active":"2023-10-11T09:40:36.149919+0000","last_became_peered":"2023-10-11T09:40:36.149919+0000","last_unstale":"2023-10-11T09:41:52.259702+0000","last_undegraded":"2023-10-11T09:41:52.259702+0000","last_fullsized":"2023-10-11T09:41:52.259702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.742402+0000","last_change":"2023-10-11T09:41:57.742402+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.742402+0000","last_undegraded":"2023-10-11T09:41:57.742402+0000","last_fullsized":"2023-10-11T09:41:57.742402+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:05:43.880930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-11T09:41:52.378813+0000","last_change":"2023-10-11T09:40:36.151839+0000","last_active":"2023-10-11T09:41:52.378813+0000","last_peered":"2023-10-11T09:41:52.378813+0000","last_clean":"2023-10-11T09:41:52.378813+0000","last_became_active":"2023-10-11T09:40:36.151179+0000","last_became_peered":"2023-10-11T09:40:36.151179+0000","last_unstale":"2023-10-11T09:41:52.378813+0000","last_undegraded":"2023-10-11T09:41:52.378813+0000","last_fullsized":"2023-10-11T09:41:52.378813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27908,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343932,"statfs":{"total":96636764160,"available":96608186368,"internally_reserved":0,"allocated":978944,"data_stored":745347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.21,"15min":1.21},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.5229999999999999}]},{"osd":1,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":2.25,"5min":2.25,"15min":2.25},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.0649999999999999}]},{"osd":2,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":1.431}]},{"osd":3,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.286,"15min":1.286},"min":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.306},{"interface":"front","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":2.1579999999999999,"5min":2.1579999999999999,"15min":2.1579999999999999},"last":1.601}]},{"osd":4,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":1.206},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":1.099}]},{"osd":5,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.681},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":3.0750000000000002,"5min":3.0750000000000002,"15min":3.0750000000000002},"last":1.147}]},{"osd":6,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.49,"15min":1.49},"min":{"1min":1.153,"5min":1.153,"15min":1.153},"max":{"1min":3.137,"5min":3.137,"15min":3.137},"last":1.28},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"min":{"1min":1.099,"5min":1.099,"15min":1.099},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":1.248}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":385024,"data_stored":154979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.071},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.65100000000000002}]},{"osd":1,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.202,"15min":1.202},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":1.8260000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.1479999999999999}]},{"osd":2,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":1.167},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":2.605,"5min":2.605,"15min":2.605},"last":1.2090000000000001}]},{"osd":3,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":0.872},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":3.0819999999999999,"5min":3.0819999999999999,"15min":3.0819999999999999},"last":1.024}]},{"osd":4,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.337,"15min":1.337},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.978,"5min":1.978,"15min":1.978},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.6920000000000002,"5min":2.6920000000000002,"15min":2.6920000000000002},"last":1.054}]},{"osd":5,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":2.7290000000000001,"5min":2.7290000000000001,"15min":2.7290000000000001},"last":1.268}]},{"osd":7,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":2.1899999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.3050000000000002,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.236}]}]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":385024,"data_stored":154979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":1.23}]},{"osd":1,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":1.1559999999999999}]},{"osd":2,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.831,"5min":1.831,"15min":1.831},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.284}]},{"osd":3,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.319},{"interface":"front","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.419}]},{"osd":4,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":1.369},{"interface":"front","average":{"1min":1.228,"5min":1.228,"15min":1.228},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":1.0840000000000001}]},{"osd":6,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.4529999999999998,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":1.4610000000000001}]},{"osd":7,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":3.6120000000000001,"5min":3.6120000000000001,"15min":3.6120000000000001},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"min":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":3.2109999999999999,"5min":3.2109999999999999,"15min":3.2109999999999999},"last":1.55}]}]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":385024,"data_stored":154979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.1160000000000001,"5min":2.1160000000000001,"15min":2.1160000000000001},"last":0.54700000000000004}]},{"osd":1,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.83399999999999996}]},{"osd":2,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":0.71899999999999997},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.73799999999999999}]},{"osd":3,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.367,"5min":1.367,"15min":1.367},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.3340000000000001,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":0.85199999999999998}]},{"osd":5,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.236,"15min":1.236},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.871},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.69999999999999996}]},{"osd":6,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.8,"5min":1.8,"15min":1.8},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.357,"5min":1.357,"15min":1.357},"min":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":0.90500000000000003}]},{"osd":7,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":1.516,"5min":1.516,"15min":1.516},"min":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"max":{"1min":1.9990000000000001,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":0.75900000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":372736,"data_stored":148809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.61299999999999999}]},{"osd":1,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":1.099}]},{"osd":2,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.5870000000000002,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":0.79700000000000004}]},{"osd":4,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.395,"5min":2.395,"15min":2.395},"last":1.012},{"interface":"front","average":{"1min":1.367,"5min":1.367,"15min":1.367},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":3.1579999999999999,"5min":3.1579999999999999,"15min":3.1579999999999999},"last":1.0609999999999999}]},{"osd":5,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.393,"15min":1.393},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":3.0009999999999999,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":0.998},{"interface":"front","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":3.8050000000000002,"5min":3.8050000000000002,"15min":3.8050000000000002},"last":1.2190000000000001}]},{"osd":6,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":2.1949999999999998,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":3.2549999999999999,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":1.133}]},{"osd":7,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":3.7490000000000001,"5min":3.7490000000000001,"15min":3.7490000000000001},"last":1.177},{"interface":"front","average":{"1min":1.448,"5min":1.448,"15min":1.448},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":1.244}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":385024,"data_stored":154979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":2.0600000000000001,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":2.8919999999999999,"5min":2.8919999999999999,"15min":2.8919999999999999},"last":0.65800000000000003}]},{"osd":1,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":3.7650000000000001,"5min":3.7650000000000001,"15min":3.7650000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.5339999999999998,"5min":2.5339999999999998,"15min":2.5339999999999998},"last":0.97999999999999998}]},{"osd":3,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.403,"5min":2.403,"15min":2.403},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.117,"5min":1.117,"15min":1.117},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":2.7589999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.0509999999999999}]},{"osd":4,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":1.5509999999999999},{"interface":"front","average":{"1min":1.476,"5min":1.476,"15min":1.476},"min":{"1min":1.085,"5min":1.085,"15min":1.085},"max":{"1min":2.6859999999999999,"5min":2.6859999999999999,"15min":2.6859999999999999},"last":1.52}]},{"osd":5,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.409,"15min":1.409},"min":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"max":{"1min":3.044,"5min":3.044,"15min":3.044},"last":1.585},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":3.5790000000000002,"5min":3.5790000000000002,"15min":3.5790000000000002},"last":1.385}]},{"osd":6,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"min":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"max":{"1min":3.1600000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.397,"5min":1.397,"15min":1.397},"min":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"max":{"1min":2.839,"5min":2.839,"15min":2.839},"last":1.6679999999999999}]},{"osd":7,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.49,"15min":1.49},"min":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":3.1230000000000002,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":1.702},{"interface":"front","average":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"min":{"1min":1.161,"5min":1.161,"15min":1.161},"max":{"1min":2.988,"5min":2.988,"15min":2.988},"last":1.728}]}]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":385024,"data_stored":154979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:41:28 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":1.198}]},{"osd":2,"last update":"Wed Oct 11 09:41:28 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.70499999999999996},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.81899999999999995}]},{"osd":3,"last update":"Wed Oct 11 09:41:28 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":1.137}]},{"osd":4,"last update":"Wed Oct 11 09:41:28 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":2.016,"5min":2.016,"15min":2.016},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":2.323,"5min":2.323,"15min":2.323},"last":1.0820000000000001}]},{"osd":5,"last update":"Wed Oct 11 09:41:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":1.04,"5min":1.04,"15min":1.04},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.1830000000000001}]},{"osd":6,"last update":"Wed Oct 11 09:41:28 2023","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"min":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"max":{"1min":2.8940000000000001,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":1.022,"5min":1.022,"15min":1.022},"max":{"1min":3.3719999999999999,"5min":3.3719999999999999,"15min":3.3719999999999999},"last":1.417}]},{"osd":7,"last update":"Wed Oct 11 09:41:28 2023","interfaces":[{"interface":"back","average":{"1min":1.468,"5min":1.468,"15min":1.468},"min":{"1min":1.171,"5min":1.171,"15min":1.171},"max":{"1min":2.194,"5min":2.194,"15min":2.194},"last":1.375},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"min":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"max":{"1min":1.968,"5min":1.968,"15min":1.968},"last":1.397}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27908,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343932,"statfs":{"total":96636764160,"available":96608186368,"internally_reserved":0,"allocated":978944,"data_stored":745347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 11 09:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.3100000000000001,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":0.91600000000000004}]},{"osd":2,"last update":"Wed Oct 11 09:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.02}]},{"osd":3,"last update":"Wed Oct 11 09:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.7639999999999998,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":0.746}]},{"osd":4,"last update":"Wed Oct 11 09:41:27 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.137},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.6629999999999998,"5min":2.6629999999999998,"15min":2.6629999999999998},"last":0.98499999999999999}]},{"osd":5,"last update":"Wed Oct 11 09:41:27 2023","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"min":{"1min":0.998,"5min":0.998,"15min":0.998},"max":{"1min":3.0150000000000001,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":1.389},{"interface":"front","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":3.601,"5min":3.601,"15min":3.601},"last":1.232}]},{"osd":6,"last update":"Wed Oct 11 09:41:27 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.825,"5min":1.825,"15min":1.825},"last":1.081},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"min":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":1.26}]},{"osd":7,"last update":"Wed Oct 11 09:41:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"min":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":1.353},{"interface":"front","average":{"1min":1.417,"5min":1.417,"15min":1.417},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":2.5419999999999998,"5min":2.5419999999999998,"15min":2.5419999999999998},"last":1.3029999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-11T09:42:03.566 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-11T09:42:03.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.781+0000 7fd42daaa700 1 Processor -- start 2023-10-11T09:42:03.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.781+0000 7fd42daaa700 1 -- start start 2023-10-11T09:42:03.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.785+0000 7fd42daaa700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd42813f740 0x7fd4280ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:03.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.785+0000 7fd42daaa700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fd42806cd00 0x7fd42806d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:03.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.785+0000 7fd42daaa700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd428070c80 con 0x7fd4280b06a0 2023-10-11T09:42:03.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.785+0000 7fd42daaa700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd42806d6c0 con 0x7fd42806cd00 2023-10-11T09:42:03.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.785+0000 7fd42daaa700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4280ce4c0 con 0x7fd42813f740 2023-10-11T09:42:03.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.785+0000 7fd427fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fd42806cd00 0x7fd42806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:03.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.785+0000 7fd42d2a9700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fd4280b06a0 0x7fd4280b0b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:58500/0 (socket says 172.21.15.78:58500) 2023-10-11T09:42:03.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.785+0000 7fd42d2a9700 1 -- 172.21.15.78:0/1826334796 learned_addr learned my addr 172.21.15.78:0/1826334796 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:42:03.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.785+0000 7fd42caa8700 1 --2- 172.21.15.78:0/1826334796 >> v2:172.21.15.133:3300/0 conn(0x7fd42813f740 0x7fd4280ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:03.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.785+0000 7fd427fff700 1 -- 172.21.15.78:0/1826334796 >> v2:172.21.15.133:3300/0 conn(0x7fd42813f740 msgr2=0x7fd4280ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:03.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.785+0000 7fd427fff700 1 --2- 172.21.15.78:0/1826334796 >> v2:172.21.15.133:3300/0 conn(0x7fd42813f740 0x7fd4280ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:03.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.785+0000 7fd4277fe700 1 -- 172.21.15.78:0/1826334796 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2564868334 0 0) 0x7fd428070c80 con 0x7fd4280b06a0 2023-10-11T09:42:03.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.785+0000 7fd427fff700 1 -- 172.21.15.78:0/1826334796 >> v1:172.21.15.78:6789/0 conn(0x7fd4280b06a0 legacy=0x7fd4280b0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:03.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.785+0000 7fd427fff700 1 -- 172.21.15.78:0/1826334796 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4280cd420 con 0x7fd42806cd00 2023-10-11T09:42:03.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.785+0000 7fd427fff700 1 --2- 172.21.15.78:0/1826334796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd42806cd00 0x7fd42806d180 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd42806fb70 tx=0x7fd414009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe0f20d32e6047a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:03.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.785+0000 7fd4277fe700 1 -- 172.21.15.78:0/1826334796 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd414011830 con 0x7fd42806cd00 2023-10-11T09:42:03.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.785+0000 7fd4277fe700 1 -- 172.21.15.78:0/1826334796 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd414011990 con 0x7fd42806cd00 2023-10-11T09:42:03.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.785+0000 7fd4277fe700 1 -- 172.21.15.78:0/1826334796 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd414011d20 con 0x7fd42806cd00 2023-10-11T09:42:03.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.789+0000 7fd42daaa700 1 -- 172.21.15.78:0/1826334796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd42806cd00 msgr2=0x7fd42806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:03.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.789+0000 7fd42daaa700 1 --2- 172.21.15.78:0/1826334796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd42806cd00 0x7fd42806d180 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd42806fb70 tx=0x7fd414009b30 comp rx=0 tx=0).stop 2023-10-11T09:42:03.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.789+0000 7fd42daaa700 1 -- 172.21.15.78:0/1826334796 shutdown_connections 2023-10-11T09:42:03.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.789+0000 7fd42daaa700 1 --2- 172.21.15.78:0/1826334796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd42806cd00 0x7fd42806d180 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:03.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.789+0000 7fd42daaa700 1 --2- 172.21.15.78:0/1826334796 >> v2:172.21.15.133:3300/0 conn(0x7fd42813f740 0x7fd4280ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:03.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.789+0000 7fd42daaa700 1 -- 172.21.15.78:0/1826334796 >> 172.21.15.78:0/1826334796 conn(0x7fd4280cea60 msgr2=0x7fd4280cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:03.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.789+0000 7fd42daaa700 1 -- 172.21.15.78:0/1826334796 shutdown_connections 2023-10-11T09:42:03.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.789+0000 7fd42daaa700 1 -- 172.21.15.78:0/1826334796 wait complete. 2023-10-11T09:42:03.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.793+0000 7fd42daaa700 1 Processor -- start 2023-10-11T09:42:03.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.793+0000 7fd42daaa700 1 -- start start 2023-10-11T09:42:03.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.793+0000 7fd42daaa700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd42806cd00 0x7fd4281537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:03.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.793+0000 7fd42daaa700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4280b06a0 0x7fd4281f20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:03.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.793+0000 7fd42daaa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd42813f740 0x7fd428153df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:03.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.793+0000 7fd42daaa700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4280b1a10 con 0x7fd42813f740 2023-10-11T09:42:03.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.793+0000 7fd42daaa700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4281f2630 con 0x7fd42806cd00 2023-10-11T09:42:03.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.793+0000 7fd42daaa700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd428154330 con 0x7fd4280b06a0 2023-10-11T09:42:03.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.793+0000 7fd42d2a9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd42813f740 0x7fd428153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:03.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.793+0000 7fd42d2a9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd42813f740 0x7fd428153df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57278/0 (socket says 172.21.15.78:57278) 2023-10-11T09:42:03.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.793+0000 7fd42d2a9700 1 -- 172.21.15.78:0/4001039240 learned_addr learned my addr 172.21.15.78:0/4001039240 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:03.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.793+0000 7fd42caa8700 1 --2- 172.21.15.78:0/4001039240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd42806cd00 0x7fd4281537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:03.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.797+0000 7fd427fff700 1 --2- 172.21.15.78:0/4001039240 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4280b06a0 0x7fd4281f20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:03.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.797+0000 7fd42d2a9700 1 -- 172.21.15.78:0/4001039240 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4280b06a0 msgr2=0x7fd4281f20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:03.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.797+0000 7fd42d2a9700 1 --2- 172.21.15.78:0/4001039240 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4280b06a0 0x7fd4281f20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:03.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.797+0000 7fd42d2a9700 1 -- 172.21.15.78:0/4001039240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd42806cd00 msgr2=0x7fd4281537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:03.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.797+0000 7fd42d2a9700 1 --2- 172.21.15.78:0/4001039240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd42806cd00 0x7fd4281537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:03.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.797+0000 7fd42d2a9700 1 -- 172.21.15.78:0/4001039240 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4140094b0 con 0x7fd42813f740 2023-10-11T09:42:03.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.797+0000 7fd42caa8700 1 --2- 172.21.15.78:0/4001039240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd42806cd00 0x7fd4281537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:42:03.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.797+0000 7fd42d2a9700 1 --2- 172.21.15.78:0/4001039240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd42813f740 0x7fd428153df0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd420002230 tx=0x7fd42000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:03.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.797+0000 7fd4257fa700 1 -- 172.21.15.78:0/4001039240 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42000ac20 con 0x7fd42813f740 2023-10-11T09:42:03.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.797+0000 7fd4257fa700 1 -- 172.21.15.78:0/4001039240 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd42000ada0 con 0x7fd42813f740 2023-10-11T09:42:03.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.797+0000 7fd4257fa700 1 -- 172.21.15.78:0/4001039240 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd420007310 con 0x7fd42813f740 2023-10-11T09:42:03.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.797+0000 7fd42daaa700 1 -- 172.21.15.78:0/4001039240 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4281f2860 con 0x7fd42813f740 2023-10-11T09:42:03.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.797+0000 7fd42daaa700 1 -- 172.21.15.78:0/4001039240 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4281f2e20 con 0x7fd42813f740 2023-10-11T09:42:03.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.797+0000 7fd42daaa700 1 -- 172.21.15.78:0/4001039240 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4280b1bc0 con 0x7fd42813f740 2023-10-11T09:42:03.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.801+0000 7fd4257fa700 1 -- 172.21.15.78:0/4001039240 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fd420007470 con 0x7fd42813f740 2023-10-11T09:42:03.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.801+0000 7fd4257fa700 1 --2- 172.21.15.78:0/4001039240 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd40c065ed0 0x7fd40c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:03.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.801+0000 7fd4257fa700 1 -- 172.21.15.78:0/4001039240 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd420083410 con 0x7fd42813f740 2023-10-11T09:42:03.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.801+0000 7fd42caa8700 1 --2- 172.21.15.78:0/4001039240 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd40c065ed0 0x7fd40c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:03.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.805+0000 7fd42caa8700 1 --2- 172.21.15.78:0/4001039240 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd40c065ed0 0x7fd40c068380 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd41c009c40 tx=0x7fd41c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:03.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.809+0000 7fd4257fa700 1 -- 172.21.15.78:0/4001039240 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fd4280b1bc0 con 0x7fd42813f740 2023-10-11T09:42:03.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.977+0000 7fd42daaa700 1 -- 172.21.15.78:0/4001039240 --> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd4280cd9e0 con 0x7fd40c065ed0 2023-10-11T09:42:03.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.993+0000 7fd4257fa700 1 -- 172.21.15.78:0/4001039240 <== mgr.4101 v2:172.21.15.78:6832/35606 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+297931 (secure 0 0 0) 0x7fd41c004030 con 0x7fd40c065ed0 2023-10-11T09:42:04.003 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:04.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.997+0000 7fd42daaa700 1 -- 172.21.15.78:0/4001039240 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd40c065ed0 msgr2=0x7fd40c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:04.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.997+0000 7fd42daaa700 1 --2- 172.21.15.78:0/4001039240 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd40c065ed0 0x7fd40c068380 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd41c009c40 tx=0x7fd41c008040 comp rx=0 tx=0).stop 2023-10-11T09:42:04.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.997+0000 7fd42daaa700 1 -- 172.21.15.78:0/4001039240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd42813f740 msgr2=0x7fd428153df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:04.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:03.997+0000 7fd42daaa700 1 --2- 172.21.15.78:0/4001039240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd42813f740 0x7fd428153df0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd420002230 tx=0x7fd42000a060 comp rx=0 tx=0).stop 2023-10-11T09:42:04.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.001+0000 7fd42daaa700 1 -- 172.21.15.78:0/4001039240 shutdown_connections 2023-10-11T09:42:04.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.001+0000 7fd42daaa700 1 --2- 172.21.15.78:0/4001039240 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd40c065ed0 0x7fd40c068380 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:04.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.001+0000 7fd42daaa700 1 --2- 172.21.15.78:0/4001039240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd42813f740 0x7fd428153df0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:04.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.001+0000 7fd42daaa700 1 --2- 172.21.15.78:0/4001039240 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4280b06a0 0x7fd4281f20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:04.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.001+0000 7fd42daaa700 1 --2- 172.21.15.78:0/4001039240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd42806cd00 0x7fd4281537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:04.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.001+0000 7fd42daaa700 1 -- 172.21.15.78:0/4001039240 >> 172.21.15.78:0/4001039240 conn(0x7fd4280cea60 msgr2=0x7fd4280cee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:04.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.001+0000 7fd42daaa700 1 -- 172.21.15.78:0/4001039240 shutdown_connections 2023-10-11T09:42:04.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.001+0000 7fd42daaa700 1 -- 172.21.15.78:0/4001039240 wait complete. 2023-10-11T09:42:04.008 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2023-10-11T09:42:04.025 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2023-10-11T09:42:02.346527+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":27,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219772,"kb_used_data":4156,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754948,"statfs":{"total":773094113280,"available":772869066752,"internally_reserved":0,"allocated":4255744,"data_stored":2414398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.607345"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.748611+0000","last_change":"2023-10-11T09:41:57.748611+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.748611+0000","last_undegraded":"2023-10-11T09:41:57.748611+0000","last_fullsized":"2023-10-11T09:41:57.748611+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:34:50.579527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.747892+0000","last_change":"2023-10-11T09:41:57.747892+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.747892+0000","last_undegraded":"2023-10-11T09:41:57.747892+0000","last_fullsized":"2023-10-11T09:41:57.747892+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:00:41.605405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.747589+0000","last_change":"2023-10-11T09:41:57.747589+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.747589+0000","last_undegraded":"2023-10-11T09:41:57.747589+0000","last_fullsized":"2023-10-11T09:41:57.747589+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:16:23.663632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.747803+0000","last_change":"2023-10-11T09:41:57.747803+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.747803+0000","last_undegraded":"2023-10-11T09:41:57.747803+0000","last_fullsized":"2023-10-11T09:41:57.747803+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:30:07.413635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.743430+0000","last_change":"2023-10-11T09:41:57.743430+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.743430+0000","last_undegraded":"2023-10-11T09:41:57.743430+0000","last_fullsized":"2023-10-11T09:41:57.743430+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:52:20.712655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.743800+0000","last_change":"2023-10-11T09:41:57.743800+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.743800+0000","last_undegraded":"2023-10-11T09:41:57.743800+0000","last_fullsized":"2023-10-11T09:41:57.743800+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:50:34.455895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.747388+0000","last_change":"2023-10-11T09:41:57.747388+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.747388+0000","last_undegraded":"2023-10-11T09:41:57.747388+0000","last_fullsized":"2023-10-11T09:41:57.747388+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:25:01.903933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.747795+0000","last_change":"2023-10-11T09:41:57.747795+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.747795+0000","last_undegraded":"2023-10-11T09:41:57.747795+0000","last_fullsized":"2023-10-11T09:41:57.747795+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:00:41.605405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.742844+0000","last_change":"2023-10-11T09:41:57.742844+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.742844+0000","last_undegraded":"2023-10-11T09:41:57.742844+0000","last_fullsized":"2023-10-11T09:41:57.742844+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:48:02.478605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.743207+0000","last_change":"2023-10-11T09:41:57.743207+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.743207+0000","last_undegraded":"2023-10-11T09:41:57.743207+0000","last_fullsized":"2023-10-11T09:41:57.743207+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:24:11.694544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.743022+0000","last_change":"2023-10-11T09:41:57.743022+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.743022+0000","last_undegraded":"2023-10-11T09:41:57.743022+0000","last_fullsized":"2023-10-11T09:41:57.743022+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:55:29.823667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.742745+0000","last_change":"2023-10-11T09:41:57.742745+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.742745+0000","last_undegraded":"2023-10-11T09:41:57.742745+0000","last_fullsized":"2023-10-11T09:41:57.742745+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:23:59.470658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.742711+0000","last_change":"2023-10-11T09:41:57.742711+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.742711+0000","last_undegraded":"2023-10-11T09:41:57.742711+0000","last_fullsized":"2023-10-11T09:41:57.742711+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:24:11.694544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.747798+0000","last_change":"2023-10-11T09:41:57.747798+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.747798+0000","last_undegraded":"2023-10-11T09:41:57.747798+0000","last_fullsized":"2023-10-11T09:41:57.747798+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:18:56.139478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.748556+0000","last_change":"2023-10-11T09:41:57.748556+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.748556+0000","last_undegraded":"2023-10-11T09:41:57.748556+0000","last_fullsized":"2023-10-11T09:41:57.748556+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:02:09.585130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.744176+0000","last_change":"2023-10-11T09:41:57.744176+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.744176+0000","last_undegraded":"2023-10-11T09:41:57.744176+0000","last_fullsized":"2023-10-11T09:41:57.744176+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:05:43.880930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.744324+0000","last_change":"2023-10-11T09:41:57.744324+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.744324+0000","last_undegraded":"2023-10-11T09:41:57.744324+0000","last_fullsized":"2023-10-11T09:41:57.744324+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:11:34.532326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.748087+0000","last_change":"2023-10-11T09:41:57.748087+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.748087+0000","last_undegraded":"2023-10-11T09:41:57.748087+0000","last_fullsized":"2023-10-11T09:41:57.748087+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:42:13.962137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.749372+0000","last_change":"2023-10-11T09:41:57.749372+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.749372+0000","last_undegraded":"2023-10-11T09:41:57.749372+0000","last_fullsized":"2023-10-11T09:41:57.749372+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:49:37.959487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.747393+0000","last_change":"2023-10-11T09:41:57.747393+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.747393+0000","last_undegraded":"2023-10-11T09:41:57.747393+0000","last_fullsized":"2023-10-11T09:41:57.747393+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:35:43.267132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.748695+0000","last_change":"2023-10-11T09:41:57.748695+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.748695+0000","last_undegraded":"2023-10-11T09:41:57.748695+0000","last_fullsized":"2023-10-11T09:41:57.748695+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:55:46.983239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.748522+0000","last_change":"2023-10-11T09:41:57.748522+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.748522+0000","last_undegraded":"2023-10-11T09:41:57.748522+0000","last_fullsized":"2023-10-11T09:41:57.748522+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:44:25.949842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.743871+0000","last_change":"2023-10-11T09:41:57.743871+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.743871+0000","last_undegraded":"2023-10-11T09:41:57.743871+0000","last_fullsized":"2023-10-11T09:41:57.743871+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:09:10.546323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.749170+0000","last_change":"2023-10-11T09:41:57.749170+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.749170+0000","last_undegraded":"2023-10-11T09:41:57.749170+0000","last_fullsized":"2023-10-11T09:41:57.749170+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:40:16.473783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.744185+0000","last_change":"2023-10-11T09:41:57.744185+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.744185+0000","last_undegraded":"2023-10-11T09:41:57.744185+0000","last_fullsized":"2023-10-11T09:41:57.744185+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:23:59.470658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-11T09:41:51.960771+0000","last_change":"2023-10-11T09:40:29.874372+0000","last_active":"2023-10-11T09:41:51.960771+0000","last_peered":"2023-10-11T09:41:51.960771+0000","last_clean":"2023-10-11T09:41:51.960771+0000","last_became_active":"2023-10-11T09:40:29.873621+0000","last_became_peered":"2023-10-11T09:40:29.873621+0000","last_unstale":"2023-10-11T09:41:51.960771+0000","last_undegraded":"2023-10-11T09:41:51.960771+0000","last_fullsized":"2023-10-11T09:41:51.960771+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_clean_scrub_stamp":"2023-10-11T09:40:28.742276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:22:07.985311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-11T09:41:52.378754+0000","last_change":"2023-10-11T09:40:36.148773+0000","last_active":"2023-10-11T09:41:52.378754+0000","last_peered":"2023-10-11T09:41:52.378754+0000","last_clean":"2023-10-11T09:41:52.378754+0000","last_became_active":"2023-10-11T09:40:36.148489+0000","last_became_peered":"2023-10-11T09:40:36.148489+0000","last_unstale":"2023-10-11T09:41:52.378754+0000","last_undegraded":"2023-10-11T09:41:52.378754+0000","last_fullsized":"2023-10-11T09:41:52.378754+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.743190+0000","last_change":"2023-10-11T09:41:57.743190+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.743190+0000","last_undegraded":"2023-10-11T09:41:57.743190+0000","last_fullsized":"2023-10-11T09:41:57.743190+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:58:21.613408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-11T09:41:51.960732+0000","last_change":"2023-10-11T09:40:36.149739+0000","last_active":"2023-10-11T09:41:51.960732+0000","last_peered":"2023-10-11T09:41:51.960732+0000","last_clean":"2023-10-11T09:41:51.960732+0000","last_became_active":"2023-10-11T09:40:36.149419+0000","last_became_peered":"2023-10-11T09:40:36.149419+0000","last_unstale":"2023-10-11T09:41:51.960732+0000","last_undegraded":"2023-10-11T09:41:51.960732+0000","last_fullsized":"2023-10-11T09:41:51.960732+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:54:50.640362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.748743+0000","last_change":"2023-10-11T09:41:57.748743+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.748743+0000","last_undegraded":"2023-10-11T09:41:57.748743+0000","last_fullsized":"2023-10-11T09:41:57.748743+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:41:27.673781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-11T09:41:52.377445+0000","last_change":"2023-10-11T09:40:36.216248+0000","last_active":"2023-10-11T09:41:52.377445+0000","last_peered":"2023-10-11T09:41:52.377445+0000","last_clean":"2023-10-11T09:41:52.377445+0000","last_became_active":"2023-10-11T09:40:36.215890+0000","last_became_peered":"2023-10-11T09:40:36.215890+0000","last_unstale":"2023-10-11T09:41:52.377445+0000","last_undegraded":"2023-10-11T09:41:52.377445+0000","last_fullsized":"2023-10-11T09:41:52.377445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.747308+0000","last_change":"2023-10-11T09:41:57.747308+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.747308+0000","last_undegraded":"2023-10-11T09:41:57.747308+0000","last_fullsized":"2023-10-11T09:41:57.747308+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:55:29.823667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-11T09:41:52.378754+0000","last_change":"2023-10-11T09:40:36.151710+0000","last_active":"2023-10-11T09:41:52.378754+0000","last_peered":"2023-10-11T09:41:52.378754+0000","last_clean":"2023-10-11T09:41:52.378754+0000","last_became_active":"2023-10-11T09:40:36.151219+0000","last_became_peered":"2023-10-11T09:40:36.151219+0000","last_unstale":"2023-10-11T09:41:52.378754+0000","last_undegraded":"2023-10-11T09:41:52.378754+0000","last_fullsized":"2023-10-11T09:41:52.378754+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.747247+0000","last_change":"2023-10-11T09:41:57.747247+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.747247+0000","last_undegraded":"2023-10-11T09:41:57.747247+0000","last_fullsized":"2023-10-11T09:41:57.747247+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:58:21.613408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-11T09:41:52.259702+0000","last_change":"2023-10-11T09:40:36.160620+0000","last_active":"2023-10-11T09:41:52.259702+0000","last_peered":"2023-10-11T09:41:52.259702+0000","last_clean":"2023-10-11T09:41:52.259702+0000","last_became_active":"2023-10-11T09:40:36.160171+0000","last_became_peered":"2023-10-11T09:40:36.160171+0000","last_unstale":"2023-10-11T09:41:52.259702+0000","last_undegraded":"2023-10-11T09:41:52.259702+0000","last_fullsized":"2023-10-11T09:41:52.259702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.748389+0000","last_change":"2023-10-11T09:41:57.748389+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.748389+0000","last_undegraded":"2023-10-11T09:41:57.748389+0000","last_fullsized":"2023-10-11T09:41:57.748389+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:28:16.256141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-11T09:41:51.960723+0000","last_change":"2023-10-11T09:40:36.162174+0000","last_active":"2023-10-11T09:41:51.960723+0000","last_peered":"2023-10-11T09:41:51.960723+0000","last_clean":"2023-10-11T09:41:51.960723+0000","last_became_active":"2023-10-11T09:40:36.161710+0000","last_became_peered":"2023-10-11T09:40:36.161710+0000","last_unstale":"2023-10-11T09:41:51.960723+0000","last_undegraded":"2023-10-11T09:41:51.960723+0000","last_fullsized":"2023-10-11T09:41:51.960723+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:57:03.630878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.747858+0000","last_change":"2023-10-11T09:41:57.747858+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.747858+0000","last_undegraded":"2023-10-11T09:41:57.747858+0000","last_fullsized":"2023-10-11T09:41:57.747858+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:07:42.669679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-11T09:41:52.259702+0000","last_change":"2023-10-11T09:40:36.150224+0000","last_active":"2023-10-11T09:41:52.259702+0000","last_peered":"2023-10-11T09:41:52.259702+0000","last_clean":"2023-10-11T09:41:52.259702+0000","last_became_active":"2023-10-11T09:40:36.149919+0000","last_became_peered":"2023-10-11T09:40:36.149919+0000","last_unstale":"2023-10-11T09:41:52.259702+0000","last_undegraded":"2023-10-11T09:41:52.259702+0000","last_fullsized":"2023-10-11T09:41:52.259702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:41:58.738403+0000","last_change":"2023-10-11T09:41:58.738403+0000","last_active":"2023-10-11T09:41:58.738403+0000","last_peered":"2023-10-11T09:41:58.738403+0000","last_clean":"2023-10-11T09:41:58.738403+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:58.738403+0000","last_undegraded":"2023-10-11T09:41:58.738403+0000","last_fullsized":"2023-10-11T09:41:58.738403+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-10-11T09:41:57.742402+0000","last_change":"2023-10-11T09:41:57.742402+0000","last_active":"2023-10-11T09:41:57.723271+0000","last_peered":"2023-10-11T09:41:57.723271+0000","last_clean":"2023-10-11T09:41:57.723271+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:41:57.742402+0000","last_undegraded":"2023-10-11T09:41:57.742402+0000","last_fullsized":"2023-10-11T09:41:57.742402+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:05:43.880930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-11T09:41:52.378813+0000","last_change":"2023-10-11T09:40:36.151839+0000","last_active":"2023-10-11T09:41:52.378813+0000","last_peered":"2023-10-11T09:41:52.378813+0000","last_clean":"2023-10-11T09:41:52.378813+0000","last_became_active":"2023-10-11T09:40:36.151179+0000","last_became_peered":"2023-10-11T09:40:36.151179+0000","last_unstale":"2023-10-11T09:41:52.378813+0000","last_undegraded":"2023-10-11T09:41:52.378813+0000","last_fullsized":"2023-10-11T09:41:52.378813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27908,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343932,"statfs":{"total":96636764160,"available":96608186368,"internally_reserved":0,"allocated":978944,"data_stored":745347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.21,"15min":1.21},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.5229999999999999}]},{"osd":1,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":2.25,"5min":2.25,"15min":2.25},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.0649999999999999}]},{"osd":2,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":1.431}]},{"osd":3,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.286,"15min":1.286},"min":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.306},{"interface":"front","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":2.1579999999999999,"5min":2.1579999999999999,"15min":2.1579999999999999},"last":1.601}]},{"osd":4,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":1.206},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":1.099}]},{"osd":5,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.681},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":3.0750000000000002,"5min":3.0750000000000002,"15min":3.0750000000000002},"last":1.147}]},{"osd":6,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.49,"15min":1.49},"min":{"1min":1.153,"5min":1.153,"15min":1.153},"max":{"1min":3.137,"5min":3.137,"15min":3.137},"last":1.28},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"min":{"1min":1.099,"5min":1.099,"15min":1.099},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":1.248}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":385024,"data_stored":154979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.071},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.65100000000000002}]},{"osd":1,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.202,"15min":1.202},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":1.8260000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.1479999999999999}]},{"osd":2,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":1.167},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":2.605,"5min":2.605,"15min":2.605},"last":1.2090000000000001}]},{"osd":3,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":0.872},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":3.0819999999999999,"5min":3.0819999999999999,"15min":3.0819999999999999},"last":1.024}]},{"osd":4,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.337,"15min":1.337},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.978,"5min":1.978,"15min":1.978},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.6920000000000002,"5min":2.6920000000000002,"15min":2.6920000000000002},"last":1.054}]},{"osd":5,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":2.7290000000000001,"5min":2.7290000000000001,"15min":2.7290000000000001},"last":1.268}]},{"osd":7,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":2.1899999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.3050000000000002,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.236}]}]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":385024,"data_stored":154979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":1.23}]},{"osd":1,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":1.1559999999999999}]},{"osd":2,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.831,"5min":1.831,"15min":1.831},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.284}]},{"osd":3,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.319},{"interface":"front","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.419}]},{"osd":4,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":1.369},{"interface":"front","average":{"1min":1.228,"5min":1.228,"15min":1.228},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":1.0840000000000001}]},{"osd":6,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.4529999999999998,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":1.4610000000000001}]},{"osd":7,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":3.6120000000000001,"5min":3.6120000000000001,"15min":3.6120000000000001},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"min":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":3.2109999999999999,"5min":3.2109999999999999,"15min":3.2109999999999999},"last":1.55}]}]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":385024,"data_stored":154979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.1160000000000001,"5min":2.1160000000000001,"15min":2.1160000000000001},"last":0.54700000000000004}]},{"osd":1,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.83399999999999996}]},{"osd":2,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":0.71899999999999997},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.73799999999999999}]},{"osd":3,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.367,"5min":1.367,"15min":1.367},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.3340000000000001,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":0.85199999999999998}]},{"osd":5,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.236,"15min":1.236},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.871},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.69999999999999996}]},{"osd":6,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.8,"5min":1.8,"15min":1.8},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.357,"5min":1.357,"15min":1.357},"min":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":0.90500000000000003}]},{"osd":7,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":1.516,"5min":1.516,"15min":1.516},"min":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"max":{"1min":1.9990000000000001,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":0.75900000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":372736,"data_stored":148809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.61299999999999999}]},{"osd":1,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":1.099}]},{"osd":2,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.5870000000000002,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":0.79700000000000004}]},{"osd":4,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.395,"5min":2.395,"15min":2.395},"last":1.012},{"interface":"front","average":{"1min":1.367,"5min":1.367,"15min":1.367},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":3.1579999999999999,"5min":3.1579999999999999,"15min":3.1579999999999999},"last":1.0609999999999999}]},{"osd":5,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.393,"15min":1.393},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":3.0009999999999999,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":0.998},{"interface":"front","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":3.8050000000000002,"5min":3.8050000000000002,"15min":3.8050000000000002},"last":1.2190000000000001}]},{"osd":6,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":2.1949999999999998,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":3.2549999999999999,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":1.133}]},{"osd":7,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":3.7490000000000001,"5min":3.7490000000000001,"15min":3.7490000000000001},"last":1.177},{"interface":"front","average":{"1min":1.448,"5min":1.448,"15min":1.448},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":1.244}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":385024,"data_stored":154979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":2.0600000000000001,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":2.8919999999999999,"5min":2.8919999999999999,"15min":2.8919999999999999},"last":0.65800000000000003}]},{"osd":1,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":3.7650000000000001,"5min":3.7650000000000001,"15min":3.7650000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.5339999999999998,"5min":2.5339999999999998,"15min":2.5339999999999998},"last":0.97999999999999998}]},{"osd":3,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.403,"5min":2.403,"15min":2.403},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.117,"5min":1.117,"15min":1.117},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":2.7589999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.0509999999999999}]},{"osd":4,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":1.5509999999999999},{"interface":"front","average":{"1min":1.476,"5min":1.476,"15min":1.476},"min":{"1min":1.085,"5min":1.085,"15min":1.085},"max":{"1min":2.6859999999999999,"5min":2.6859999999999999,"15min":2.6859999999999999},"last":1.52}]},{"osd":5,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.409,"15min":1.409},"min":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"max":{"1min":3.044,"5min":3.044,"15min":3.044},"last":1.585},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":3.5790000000000002,"5min":3.5790000000000002,"15min":3.5790000000000002},"last":1.385}]},{"osd":6,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"min":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"max":{"1min":3.1600000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.397,"5min":1.397,"15min":1.397},"min":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"max":{"1min":2.839,"5min":2.839,"15min":2.839},"last":1.6679999999999999}]},{"osd":7,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.49,"15min":1.49},"min":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":3.1230000000000002,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":1.702},{"interface":"front","average":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"min":{"1min":1.161,"5min":1.161,"15min":1.161},"max":{"1min":2.988,"5min":2.988,"15min":2.988},"last":1.728}]}]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":385024,"data_stored":154979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:41:28 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":1.198}]},{"osd":2,"last update":"Wed Oct 11 09:41:28 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.70499999999999996},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.81899999999999995}]},{"osd":3,"last update":"Wed Oct 11 09:41:28 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":1.137}]},{"osd":4,"last update":"Wed Oct 11 09:41:28 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":2.016,"5min":2.016,"15min":2.016},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":2.323,"5min":2.323,"15min":2.323},"last":1.0820000000000001}]},{"osd":5,"last update":"Wed Oct 11 09:41:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":1.04,"5min":1.04,"15min":1.04},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.1830000000000001}]},{"osd":6,"last update":"Wed Oct 11 09:41:28 2023","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"min":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"max":{"1min":2.8940000000000001,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":1.022,"5min":1.022,"15min":1.022},"max":{"1min":3.3719999999999999,"5min":3.3719999999999999,"15min":3.3719999999999999},"last":1.417}]},{"osd":7,"last update":"Wed Oct 11 09:41:28 2023","interfaces":[{"interface":"back","average":{"1min":1.468,"5min":1.468,"15min":1.468},"min":{"1min":1.171,"5min":1.171,"15min":1.171},"max":{"1min":2.194,"5min":2.194,"15min":2.194},"last":1.375},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"min":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"max":{"1min":1.968,"5min":1.968,"15min":1.968},"last":1.397}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27908,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343932,"statfs":{"total":96636764160,"available":96608186368,"internally_reserved":0,"allocated":978944,"data_stored":745347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 11 09:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.3100000000000001,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":0.91600000000000004}]},{"osd":2,"last update":"Wed Oct 11 09:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.02}]},{"osd":3,"last update":"Wed Oct 11 09:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.7639999999999998,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":0.746}]},{"osd":4,"last update":"Wed Oct 11 09:41:27 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.137},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.6629999999999998,"5min":2.6629999999999998,"15min":2.6629999999999998},"last":0.98499999999999999}]},{"osd":5,"last update":"Wed Oct 11 09:41:27 2023","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"min":{"1min":0.998,"5min":0.998,"15min":0.998},"max":{"1min":3.0150000000000001,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":1.389},{"interface":"front","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":3.601,"5min":3.601,"15min":3.601},"last":1.232}]},{"osd":6,"last update":"Wed Oct 11 09:41:27 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.825,"5min":1.825,"15min":1.825},"last":1.081},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"min":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":1.26}]},{"osd":7,"last update":"Wed Oct 11 09:41:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"min":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":1.353},{"interface":"front","average":{"1min":1.417,"5min":1.417,"15min":1.417},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":2.5419999999999998,"5min":2.5419999999999998,"15min":2.5419999999999998},"last":1.3029999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-11T09:42:04.032 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-11T09:42:04.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.233+0000 7fa3392c8700 1 Processor -- start 2023-10-11T09:42:04.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.233+0000 7fa3392c8700 1 -- start start 2023-10-11T09:42:04.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.237+0000 7fa3392c8700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa3340b2310 0x7fa334161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:04.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.237+0000 7fa3392c8700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3340d11e0 con 0x7fa3340b5f30 2023-10-11T09:42:04.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.237+0000 7fa3392c8700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3340b5370 con 0x7fa3340b4b60 2023-10-11T09:42:04.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.237+0000 7fa3392c8700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3340d14a0 con 0x7fa3340b2310 2023-10-11T09:42:04.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.237+0000 7fa3337fe700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fa3340b5f30 0x7fa3340cd730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:58508/0 (socket says 172.21.15.78:58508) 2023-10-11T09:42:04.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.237+0000 7fa332ffd700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa3340b2310 0x7fa334161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:04.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.237+0000 7fa3337fe700 1 -- 172.21.15.78:0/1128115201 learned_addr learned my addr 172.21.15.78:0/1128115201 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:42:04.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.237+0000 7fa3327fc700 1 -- 172.21.15.78:0/1128115201 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1269649541 0 0) 0x7fa3340d11e0 con 0x7fa3340b5f30 2023-10-11T09:42:04.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.237+0000 7fa3327fc700 1 -- 172.21.15.78:0/1128115201 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3200035d0 con 0x7fa3340b5f30 2023-10-11T09:42:04.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.237+0000 7fa3327fc700 1 -- 172.21.15.78:0/1128115201 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2230339003 0 0) 0x7fa3340b5370 con 0x7fa3340b4b60 2023-10-11T09:42:04.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.237+0000 7fa332ffd700 1 -- 172.21.15.78:0/1128115201 >> v1:172.21.15.133:6789/0 conn(0x7fa3340b4b60 legacy=0x7fa3340d1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:04.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.237+0000 7fa332ffd700 1 -- 172.21.15.78:0/1128115201 >> v1:172.21.15.78:6789/0 conn(0x7fa3340b5f30 legacy=0x7fa3340cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:04.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.237+0000 7fa332ffd700 1 -- 172.21.15.78:0/1128115201 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3340d1780 con 0x7fa3340b2310 2023-10-11T09:42:04.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.241+0000 7fa332ffd700 1 --2- 172.21.15.78:0/1128115201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3340b2310 0x7fa334161150 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa324009880 tx=0x7fa324009b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d66d5c8bf211deb server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:04.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.241+0000 7fa3327fc700 1 -- 172.21.15.78:0/1128115201 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa324011830 con 0x7fa3340b2310 2023-10-11T09:42:04.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.241+0000 7fa3327fc700 1 -- 172.21.15.78:0/1128115201 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa324011990 con 0x7fa3340b2310 2023-10-11T09:42:04.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.241+0000 7fa3327fc700 1 -- 172.21.15.78:0/1128115201 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa324011d20 con 0x7fa3340b2310 2023-10-11T09:42:04.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.241+0000 7fa3392c8700 1 -- 172.21.15.78:0/1128115201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3340b2310 msgr2=0x7fa334161150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:04.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.241+0000 7fa3392c8700 1 --2- 172.21.15.78:0/1128115201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3340b2310 0x7fa334161150 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa324009880 tx=0x7fa324009b90 comp rx=0 tx=0).stop 2023-10-11T09:42:04.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.241+0000 7fa3392c8700 1 -- 172.21.15.78:0/1128115201 shutdown_connections 2023-10-11T09:42:04.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.241+0000 7fa3392c8700 1 --2- 172.21.15.78:0/1128115201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3340b2310 0x7fa334161150 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:04.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.241+0000 7fa3392c8700 1 -- 172.21.15.78:0/1128115201 >> 172.21.15.78:0/1128115201 conn(0x7fa3340cfb60 msgr2=0x7fa3340cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:04.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.241+0000 7fa3392c8700 1 -- 172.21.15.78:0/1128115201 shutdown_connections 2023-10-11T09:42:04.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.241+0000 7fa3392c8700 1 -- 172.21.15.78:0/1128115201 wait complete. 2023-10-11T09:42:04.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.245+0000 7fa3392c8700 1 Processor -- start 2023-10-11T09:42:04.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.245+0000 7fa3392c8700 1 -- start start 2023-10-11T09:42:04.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.245+0000 7fa3392c8700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3340b2310 0x7fa3341f1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:04.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.245+0000 7fa3392c8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3340b4b60 0x7fa33415a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:04.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.245+0000 7fa3392c8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3340b5f30 0x7fa3340c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:04.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.245+0000 7fa3392c8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3340ce0f0 con 0x7fa3340b4b60 2023-10-11T09:42:04.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.245+0000 7fa3392c8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3341f54b0 con 0x7fa3340b5f30 2023-10-11T09:42:04.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.245+0000 7fa3392c8700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3341f2420 con 0x7fa3340b2310 2023-10-11T09:42:04.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.245+0000 7fa32a7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3340b4b60 0x7fa33415a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:04.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.245+0000 7fa332ffd700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3340b2310 0x7fa3341f1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:04.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.245+0000 7fa3337fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3340b5f30 0x7fa3340c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:04.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.245+0000 7fa332ffd700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3340b2310 0x7fa3341f1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:33248/0 (socket says 172.21.15.78:33248) 2023-10-11T09:42:04.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.245+0000 7fa332ffd700 1 -- 172.21.15.78:0/3415873050 learned_addr learned my addr 172.21.15.78:0/3415873050 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:04.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.249+0000 7fa32a7fc700 1 -- 172.21.15.78:0/3415873050 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3340b2310 msgr2=0x7fa3341f1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:04.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.249+0000 7fa32a7fc700 1 --2- 172.21.15.78:0/3415873050 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3340b2310 0x7fa3341f1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:04.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.249+0000 7fa32a7fc700 1 -- 172.21.15.78:0/3415873050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3340b5f30 msgr2=0x7fa3340c89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:04.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.249+0000 7fa32a7fc700 1 --2- 172.21.15.78:0/3415873050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3340b5f30 0x7fa3340c89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:04.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.249+0000 7fa32a7fc700 1 -- 172.21.15.78:0/3415873050 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3240094b0 con 0x7fa3340b4b60 2023-10-11T09:42:04.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.249+0000 7fa3337fe700 1 --2- 172.21.15.78:0/3415873050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3340b5f30 0x7fa3340c89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:42:04.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.249+0000 7fa32a7fc700 1 --2- 172.21.15.78:0/3415873050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3340b4b60 0x7fa33415a1a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa31c00cc90 tx=0x7fa31c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:04.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.249+0000 7fa32bfff700 1 -- 172.21.15.78:0/3415873050 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31c003e50 con 0x7fa3340b4b60 2023-10-11T09:42:04.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.249+0000 7fa3392c8700 1 -- 172.21.15.78:0/3415873050 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3340b5140 con 0x7fa3340b4b60 2023-10-11T09:42:04.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.249+0000 7fa32bfff700 1 -- 172.21.15.78:0/3415873050 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa31c00a6a0 con 0x7fa3340b4b60 2023-10-11T09:42:04.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.249+0000 7fa32bfff700 1 -- 172.21.15.78:0/3415873050 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31c00aa50 con 0x7fa3340b4b60 2023-10-11T09:42:04.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.249+0000 7fa3392c8700 1 -- 172.21.15.78:0/3415873050 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3341f29e0 con 0x7fa3340b4b60 2023-10-11T09:42:04.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.249+0000 7fa3392c8700 1 -- 172.21.15.78:0/3415873050 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3341f2d20 con 0x7fa3340b4b60 2023-10-11T09:42:04.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.253+0000 7fa32bfff700 1 -- 172.21.15.78:0/3415873050 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fa31c00abb0 con 0x7fa3340b4b60 2023-10-11T09:42:04.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.253+0000 7fa32bfff700 1 --2- 172.21.15.78:0/3415873050 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa314065f00 0x7fa3140683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:04.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.253+0000 7fa32bfff700 1 -- 172.21.15.78:0/3415873050 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa31c083230 con 0x7fa3340b4b60 2023-10-11T09:42:04.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.253+0000 7fa332ffd700 1 --2- 172.21.15.78:0/3415873050 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa314065f00 0x7fa3140683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:04.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.253+0000 7fa332ffd700 1 --2- 172.21.15.78:0/3415873050 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa314065f00 0x7fa3140683b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa324009fd0 tx=0x7fa324008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:04.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.261+0000 7fa32bfff700 1 -- 172.21.15.78:0/3415873050 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fa31c0507f0 con 0x7fa3340b4b60 2023-10-11T09:42:04.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.433+0000 7fa3392c8700 1 -- 172.21.15.78:0/3415873050 --> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa3340b4fc0 con 0x7fa314065f00 2023-10-11T09:42:04.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.441+0000 7fa32bfff700 1 -- 172.21.15.78:0/3415873050 <== mgr.4101 v2:172.21.15.78:6832/35606 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308742 (secure 0 0 0) 0x7fa324011cb0 con 0x7fa314065f00 2023-10-11T09:42:04.452 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:04.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.449+0000 7fa3392c8700 1 -- 172.21.15.78:0/3415873050 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa314065f00 msgr2=0x7fa3140683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:04.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.449+0000 7fa3392c8700 1 --2- 172.21.15.78:0/3415873050 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa314065f00 0x7fa3140683b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa324009fd0 tx=0x7fa324008030 comp rx=0 tx=0).stop 2023-10-11T09:42:04.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.449+0000 7fa3392c8700 1 -- 172.21.15.78:0/3415873050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3340b4b60 msgr2=0x7fa33415a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:04.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.449+0000 7fa3392c8700 1 --2- 172.21.15.78:0/3415873050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3340b4b60 0x7fa33415a1a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa31c00cc90 tx=0x7fa31c00a060 comp rx=0 tx=0).stop 2023-10-11T09:42:04.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.449+0000 7fa3392c8700 1 -- 172.21.15.78:0/3415873050 shutdown_connections 2023-10-11T09:42:04.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.449+0000 7fa3392c8700 1 --2- 172.21.15.78:0/3415873050 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa314065f00 0x7fa3140683b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:04.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.449+0000 7fa3392c8700 1 --2- 172.21.15.78:0/3415873050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3340b5f30 0x7fa3340c89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:04.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.449+0000 7fa3392c8700 1 --2- 172.21.15.78:0/3415873050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3340b4b60 0x7fa33415a1a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:04.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.449+0000 7fa3392c8700 1 --2- 172.21.15.78:0/3415873050 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3340b2310 0x7fa3341f1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:04.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.449+0000 7fa3392c8700 1 -- 172.21.15.78:0/3415873050 >> 172.21.15.78:0/3415873050 conn(0x7fa3340cfb60 msgr2=0x7fa33415b580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:04.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.449+0000 7fa3392c8700 1 -- 172.21.15.78:0/3415873050 shutdown_connections 2023-10-11T09:42:04.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.449+0000 7fa3392c8700 1 -- 172.21.15.78:0/3415873050 wait complete. 2023-10-11T09:42:04.458 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2023-10-11T09:42:04.477 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2023-10-11T09:42:04.347112+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":184,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220136,"kb_used_data":4520,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754584,"statfs":{"total":773094113280,"available":772868694016,"internally_reserved":0,"allocated":4628480,"data_stored":2588564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":54,"apply_latency_ms":54,"commit_latency_ns":54000000,"apply_latency_ns":54000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001529"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.134185+0000","last_change":"2023-10-11T09:41:59.251004+0000","last_active":"2023-10-11T09:42:02.134185+0000","last_peered":"2023-10-11T09:42:02.134185+0000","last_clean":"2023-10-11T09:42:02.134185+0000","last_became_active":"2023-10-11T09:41:59.250745+0000","last_became_peered":"2023-10-11T09:41:59.250745+0000","last_unstale":"2023-10-11T09:42:02.134185+0000","last_undegraded":"2023-10-11T09:42:02.134185+0000","last_fullsized":"2023-10-11T09:42:02.134185+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:32:44.448136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.135281+0000","last_change":"2023-10-11T09:41:59.258617+0000","last_active":"2023-10-11T09:42:02.135281+0000","last_peered":"2023-10-11T09:42:02.135281+0000","last_clean":"2023-10-11T09:42:02.135281+0000","last_became_active":"2023-10-11T09:41:59.258374+0000","last_became_peered":"2023-10-11T09:41:59.258374+0000","last_unstale":"2023-10-11T09:42:02.135281+0000","last_undegraded":"2023-10-11T09:42:02.135281+0000","last_fullsized":"2023-10-11T09:42:02.135281+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:46:05.674097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133528+0000","last_change":"2023-10-11T09:41:59.270985+0000","last_active":"2023-10-11T09:42:02.133528+0000","last_peered":"2023-10-11T09:42:02.133528+0000","last_clean":"2023-10-11T09:42:02.133528+0000","last_became_active":"2023-10-11T09:41:59.270588+0000","last_became_peered":"2023-10-11T09:41:59.270588+0000","last_unstale":"2023-10-11T09:42:02.133528+0000","last_undegraded":"2023-10-11T09:42:02.133528+0000","last_fullsized":"2023-10-11T09:42:02.133528+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:51:26.096609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.244926+0000","last_change":"2023-10-11T09:41:59.254605+0000","last_active":"2023-10-11T09:42:00.244926+0000","last_peered":"2023-10-11T09:42:00.244926+0000","last_clean":"2023-10-11T09:42:00.244926+0000","last_became_active":"2023-10-11T09:41:59.254130+0000","last_became_peered":"2023-10-11T09:41:59.254130+0000","last_unstale":"2023-10-11T09:42:00.244926+0000","last_undegraded":"2023-10-11T09:42:00.244926+0000","last_fullsized":"2023-10-11T09:42:00.244926+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:09:11.561455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.244764+0000","last_change":"2023-10-11T09:41:59.251753+0000","last_active":"2023-10-11T09:42:00.244764+0000","last_peered":"2023-10-11T09:42:00.244764+0000","last_clean":"2023-10-11T09:42:00.244764+0000","last_became_active":"2023-10-11T09:41:59.251460+0000","last_became_peered":"2023-10-11T09:41:59.251460+0000","last_unstale":"2023-10-11T09:42:00.244764+0000","last_undegraded":"2023-10-11T09:42:00.244764+0000","last_fullsized":"2023-10-11T09:42:00.244764+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:16:24.678764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.134098+0000","last_change":"2023-10-11T09:41:59.250567+0000","last_active":"2023-10-11T09:42:02.134098+0000","last_peered":"2023-10-11T09:42:02.134098+0000","last_clean":"2023-10-11T09:42:02.134098+0000","last_became_active":"2023-10-11T09:41:59.250202+0000","last_became_peered":"2023-10-11T09:41:59.250202+0000","last_unstale":"2023-10-11T09:42:02.134098+0000","last_undegraded":"2023-10-11T09:42:02.134098+0000","last_fullsized":"2023-10-11T09:42:02.134098+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:52:21.727787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.245752+0000","last_change":"2023-10-11T09:41:59.261239+0000","last_active":"2023-10-11T09:42:00.245752+0000","last_peered":"2023-10-11T09:42:00.245752+0000","last_clean":"2023-10-11T09:42:00.245752+0000","last_became_active":"2023-10-11T09:41:59.260991+0000","last_became_peered":"2023-10-11T09:41:59.260991+0000","last_unstale":"2023-10-11T09:42:00.245752+0000","last_undegraded":"2023-10-11T09:42:00.245752+0000","last_fullsized":"2023-10-11T09:42:00.245752+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:14:50.524508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.132362+0000","last_change":"2023-10-11T09:41:59.267670+0000","last_active":"2023-10-11T09:42:02.132362+0000","last_peered":"2023-10-11T09:42:02.132362+0000","last_clean":"2023-10-11T09:42:02.132362+0000","last_became_active":"2023-10-11T09:41:59.267410+0000","last_became_peered":"2023-10-11T09:41:59.267410+0000","last_unstale":"2023-10-11T09:42:02.132362+0000","last_undegraded":"2023-10-11T09:42:02.132362+0000","last_fullsized":"2023-10-11T09:42:02.132362+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:14:53.871518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133935+0000","last_change":"2023-10-11T09:41:59.271233+0000","last_active":"2023-10-11T09:42:02.133935+0000","last_peered":"2023-10-11T09:42:02.133935+0000","last_clean":"2023-10-11T09:42:02.133935+0000","last_became_active":"2023-10-11T09:41:59.270942+0000","last_became_peered":"2023-10-11T09:41:59.270942+0000","last_unstale":"2023-10-11T09:42:02.133935+0000","last_undegraded":"2023-10-11T09:42:02.133935+0000","last_fullsized":"2023-10-11T09:42:02.133935+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:46:05.674097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.247294+0000","last_change":"2023-10-11T09:41:59.276669+0000","last_active":"2023-10-11T09:42:00.247294+0000","last_peered":"2023-10-11T09:42:00.247294+0000","last_clean":"2023-10-11T09:42:00.247294+0000","last_became_active":"2023-10-11T09:41:59.276458+0000","last_became_peered":"2023-10-11T09:41:59.276458+0000","last_unstale":"2023-10-11T09:42:00.247294+0000","last_undegraded":"2023-10-11T09:42:00.247294+0000","last_fullsized":"2023-10-11T09:42:00.247294+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:29:15.375697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.247187+0000","last_change":"2023-10-11T09:41:59.270643+0000","last_active":"2023-10-11T09:42:00.247187+0000","last_peered":"2023-10-11T09:42:00.247187+0000","last_clean":"2023-10-11T09:42:00.247187+0000","last_became_active":"2023-10-11T09:41:59.270327+0000","last_became_peered":"2023-10-11T09:41:59.270327+0000","last_unstale":"2023-10-11T09:42:00.247187+0000","last_undegraded":"2023-10-11T09:42:00.247187+0000","last_fullsized":"2023-10-11T09:42:00.247187+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:46:05.674097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.134764+0000","last_change":"2023-10-11T09:41:59.261251+0000","last_active":"2023-10-11T09:42:02.134764+0000","last_peered":"2023-10-11T09:42:02.134764+0000","last_clean":"2023-10-11T09:42:02.134764+0000","last_became_active":"2023-10-11T09:41:59.260849+0000","last_became_peered":"2023-10-11T09:41:59.260849+0000","last_unstale":"2023-10-11T09:42:02.134764+0000","last_undegraded":"2023-10-11T09:42:02.134764+0000","last_fullsized":"2023-10-11T09:42:02.134764+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:29:09.854758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.246833+0000","last_change":"2023-10-11T09:41:59.276141+0000","last_active":"2023-10-11T09:42:00.246833+0000","last_peered":"2023-10-11T09:42:00.246833+0000","last_clean":"2023-10-11T09:42:00.246833+0000","last_became_active":"2023-10-11T09:41:59.275873+0000","last_became_peered":"2023-10-11T09:41:59.275873+0000","last_unstale":"2023-10-11T09:42:00.246833+0000","last_undegraded":"2023-10-11T09:42:00.246833+0000","last_fullsized":"2023-10-11T09:42:00.246833+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:06:13.791884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:01.632123+0000","last_change":"2023-10-11T09:41:59.258250+0000","last_active":"2023-10-11T09:42:01.632123+0000","last_peered":"2023-10-11T09:42:01.632123+0000","last_clean":"2023-10-11T09:42:01.632123+0000","last_became_active":"2023-10-11T09:41:59.257976+0000","last_became_peered":"2023-10-11T09:41:59.257976+0000","last_unstale":"2023-10-11T09:42:01.632123+0000","last_undegraded":"2023-10-11T09:42:01.632123+0000","last_fullsized":"2023-10-11T09:42:01.632123+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:46:05.674097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.245867+0000","last_change":"2023-10-11T09:41:59.268687+0000","last_active":"2023-10-11T09:42:00.245867+0000","last_peered":"2023-10-11T09:42:00.245867+0000","last_clean":"2023-10-11T09:42:00.245867+0000","last_became_active":"2023-10-11T09:41:59.268386+0000","last_became_peered":"2023-10-11T09:41:59.268386+0000","last_unstale":"2023-10-11T09:42:00.245867+0000","last_undegraded":"2023-10-11T09:42:00.245867+0000","last_fullsized":"2023-10-11T09:42:00.245867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:29:09.854758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.245820+0000","last_change":"2023-10-11T09:41:59.276367+0000","last_active":"2023-10-11T09:42:00.245820+0000","last_peered":"2023-10-11T09:42:00.245820+0000","last_clean":"2023-10-11T09:42:00.245820+0000","last_became_active":"2023-10-11T09:41:59.275721+0000","last_became_peered":"2023-10-11T09:41:59.275721+0000","last_unstale":"2023-10-11T09:42:00.245820+0000","last_undegraded":"2023-10-11T09:42:00.245820+0000","last_fullsized":"2023-10-11T09:42:00.245820+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:32:44.448136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.246578+0000","last_change":"2023-10-11T09:41:59.272244+0000","last_active":"2023-10-11T09:42:00.246578+0000","last_peered":"2023-10-11T09:42:00.246578+0000","last_clean":"2023-10-11T09:42:00.246578+0000","last_became_active":"2023-10-11T09:41:59.271864+0000","last_became_peered":"2023-10-11T09:41:59.271864+0000","last_unstale":"2023-10-11T09:42:00.246578+0000","last_undegraded":"2023-10-11T09:42:00.246578+0000","last_fullsized":"2023-10-11T09:42:00.246578+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:51:26.096609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133551+0000","last_change":"2023-10-11T09:41:59.257454+0000","last_active":"2023-10-11T09:42:02.133551+0000","last_peered":"2023-10-11T09:42:02.133551+0000","last_clean":"2023-10-11T09:42:02.133551+0000","last_became_active":"2023-10-11T09:41:59.257169+0000","last_became_peered":"2023-10-11T09:41:59.257169+0000","last_unstale":"2023-10-11T09:42:02.133551+0000","last_undegraded":"2023-10-11T09:42:02.133551+0000","last_fullsized":"2023-10-11T09:42:02.133551+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:50:35.471027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.245210+0000","last_change":"2023-10-11T09:41:59.247417+0000","last_active":"2023-10-11T09:42:00.245210+0000","last_peered":"2023-10-11T09:42:00.245210+0000","last_clean":"2023-10-11T09:42:00.245210+0000","last_became_active":"2023-10-11T09:41:59.247155+0000","last_became_peered":"2023-10-11T09:41:59.247155+0000","last_unstale":"2023-10-11T09:42:00.245210+0000","last_undegraded":"2023-10-11T09:42:00.245210+0000","last_fullsized":"2023-10-11T09:42:00.245210+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:18:57.154610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133410+0000","last_change":"2023-10-11T09:41:59.258025+0000","last_active":"2023-10-11T09:42:02.133410+0000","last_peered":"2023-10-11T09:42:02.133410+0000","last_clean":"2023-10-11T09:42:02.133410+0000","last_became_active":"2023-10-11T09:41:59.257530+0000","last_became_peered":"2023-10-11T09:41:59.257530+0000","last_unstale":"2023-10-11T09:42:02.133410+0000","last_undegraded":"2023-10-11T09:42:02.133410+0000","last_fullsized":"2023-10-11T09:42:02.133410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:09:11.561455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.246676+0000","last_change":"2023-10-11T09:41:59.271200+0000","last_active":"2023-10-11T09:42:00.246676+0000","last_peered":"2023-10-11T09:42:00.246676+0000","last_clean":"2023-10-11T09:42:00.246676+0000","last_became_active":"2023-10-11T09:41:59.270934+0000","last_became_peered":"2023-10-11T09:41:59.270934+0000","last_unstale":"2023-10-11T09:42:00.246676+0000","last_undegraded":"2023-10-11T09:42:00.246676+0000","last_fullsized":"2023-10-11T09:42:00.246676+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:03:35.795099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:01.632676+0000","last_change":"2023-10-11T09:41:59.253265+0000","last_active":"2023-10-11T09:42:01.632676+0000","last_peered":"2023-10-11T09:42:01.632676+0000","last_clean":"2023-10-11T09:42:01.632676+0000","last_became_active":"2023-10-11T09:41:59.252970+0000","last_became_peered":"2023-10-11T09:41:59.252970+0000","last_unstale":"2023-10-11T09:42:01.632676+0000","last_undegraded":"2023-10-11T09:42:01.632676+0000","last_fullsized":"2023-10-11T09:42:01.632676+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:06:13.791884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.246079+0000","last_change":"2023-10-11T09:41:59.275092+0000","last_active":"2023-10-11T09:42:00.246079+0000","last_peered":"2023-10-11T09:42:00.246079+0000","last_clean":"2023-10-11T09:42:00.246079+0000","last_became_active":"2023-10-11T09:41:59.274296+0000","last_became_peered":"2023-10-11T09:41:59.274296+0000","last_unstale":"2023-10-11T09:42:00.246079+0000","last_undegraded":"2023-10-11T09:42:00.246079+0000","last_fullsized":"2023-10-11T09:42:00.246079+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:07:46.727754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.244267+0000","last_change":"2023-10-11T09:41:59.266457+0000","last_active":"2023-10-11T09:42:00.244267+0000","last_peered":"2023-10-11T09:42:00.244267+0000","last_clean":"2023-10-11T09:42:00.244267+0000","last_became_active":"2023-10-11T09:41:59.266177+0000","last_became_peered":"2023-10-11T09:41:59.266177+0000","last_unstale":"2023-10-11T09:42:00.244267+0000","last_undegraded":"2023-10-11T09:42:00.244267+0000","last_fullsized":"2023-10-11T09:42:00.244267+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:24:35.472967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133671+0000","last_change":"2023-10-11T09:41:59.256922+0000","last_active":"2023-10-11T09:42:02.133671+0000","last_peered":"2023-10-11T09:42:02.133671+0000","last_clean":"2023-10-11T09:42:02.133671+0000","last_became_active":"2023-10-11T09:41:59.256620+0000","last_became_peered":"2023-10-11T09:41:59.256620+0000","last_unstale":"2023-10-11T09:42:02.133671+0000","last_undegraded":"2023-10-11T09:42:02.133671+0000","last_fullsized":"2023-10-11T09:42:02.133671+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:48:03.493737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.134430+0000","last_change":"2023-10-11T09:41:59.267830+0000","last_active":"2023-10-11T09:42:02.134430+0000","last_peered":"2023-10-11T09:42:02.134430+0000","last_clean":"2023-10-11T09:42:02.134430+0000","last_became_active":"2023-10-11T09:41:59.267543+0000","last_became_peered":"2023-10-11T09:41:59.267543+0000","last_unstale":"2023-10-11T09:42:02.134430+0000","last_undegraded":"2023-10-11T09:42:02.134430+0000","last_fullsized":"2023-10-11T09:42:02.134430+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:00:42.620537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133519+0000","last_change":"2023-10-11T09:41:59.259928+0000","last_active":"2023-10-11T09:42:02.133519+0000","last_peered":"2023-10-11T09:42:02.133519+0000","last_clean":"2023-10-11T09:42:02.133519+0000","last_became_active":"2023-10-11T09:41:59.259642+0000","last_became_peered":"2023-10-11T09:41:59.259642+0000","last_unstale":"2023-10-11T09:42:02.133519+0000","last_undegraded":"2023-10-11T09:42:02.133519+0000","last_fullsized":"2023-10-11T09:42:02.133519+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:05:44.896062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.247105+0000","last_change":"2023-10-11T09:41:59.262115+0000","last_active":"2023-10-11T09:42:00.247105+0000","last_peered":"2023-10-11T09:42:00.247105+0000","last_clean":"2023-10-11T09:42:00.247105+0000","last_became_active":"2023-10-11T09:41:59.261752+0000","last_became_peered":"2023-10-11T09:41:59.261752+0000","last_unstale":"2023-10-11T09:42:00.247105+0000","last_undegraded":"2023-10-11T09:42:00.247105+0000","last_fullsized":"2023-10-11T09:42:00.247105+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:55:01.198068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133191+0000","last_change":"2023-10-11T09:41:59.261554+0000","last_active":"2023-10-11T09:42:02.133191+0000","last_peered":"2023-10-11T09:42:02.133191+0000","last_clean":"2023-10-11T09:42:02.133191+0000","last_became_active":"2023-10-11T09:41:59.261064+0000","last_became_peered":"2023-10-11T09:41:59.261064+0000","last_unstale":"2023-10-11T09:42:02.133191+0000","last_undegraded":"2023-10-11T09:42:02.133191+0000","last_fullsized":"2023-10-11T09:42:02.133191+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:24:00.485790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133455+0000","last_change":"2023-10-11T09:41:59.256304+0000","last_active":"2023-10-11T09:42:02.133455+0000","last_peered":"2023-10-11T09:42:02.133455+0000","last_clean":"2023-10-11T09:42:02.133455+0000","last_became_active":"2023-10-11T09:41:59.255941+0000","last_became_peered":"2023-10-11T09:41:59.255941+0000","last_unstale":"2023-10-11T09:42:02.133455+0000","last_undegraded":"2023-10-11T09:42:02.133455+0000","last_fullsized":"2023-10-11T09:42:02.133455+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:16:24.678764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.246243+0000","last_change":"2023-10-11T09:41:59.275647+0000","last_active":"2023-10-11T09:42:00.246243+0000","last_peered":"2023-10-11T09:42:00.246243+0000","last_clean":"2023-10-11T09:42:00.246243+0000","last_became_active":"2023-10-11T09:41:59.275328+0000","last_became_peered":"2023-10-11T09:41:59.275328+0000","last_unstale":"2023-10-11T09:42:00.246243+0000","last_undegraded":"2023-10-11T09:42:00.246243+0000","last_fullsized":"2023-10-11T09:42:00.246243+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:49:29.335655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133147+0000","last_change":"2023-10-11T09:41:59.261148+0000","last_active":"2023-10-11T09:42:02.133147+0000","last_peered":"2023-10-11T09:42:02.133147+0000","last_clean":"2023-10-11T09:42:02.133147+0000","last_became_active":"2023-10-11T09:41:59.260863+0000","last_became_peered":"2023-10-11T09:41:59.260863+0000","last_unstale":"2023-10-11T09:42:02.133147+0000","last_undegraded":"2023-10-11T09:42:02.133147+0000","last_fullsized":"2023-10-11T09:42:02.133147+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:58:22.628540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.246187+0000","last_change":"2023-10-11T09:41:59.268534+0000","last_active":"2023-10-11T09:42:00.246187+0000","last_peered":"2023-10-11T09:42:00.246187+0000","last_clean":"2023-10-11T09:42:00.246187+0000","last_became_active":"2023-10-11T09:41:59.268260+0000","last_became_peered":"2023-10-11T09:41:59.268260+0000","last_unstale":"2023-10-11T09:42:00.246187+0000","last_undegraded":"2023-10-11T09:42:00.246187+0000","last_fullsized":"2023-10-11T09:42:00.246187+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:46:05.674097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"36'4","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.278577+0000","last_change":"2023-10-11T09:41:58.781253+0000","last_active":"2023-10-11T09:41:59.278577+0000","last_peered":"2023-10-11T09:41:59.278577+0000","last_clean":"2023-10-11T09:41:59.278577+0000","last_became_active":"2023-10-11T09:41:58.780932+0000","last_became_peered":"2023-10-11T09:41:58.780932+0000","last_unstale":"2023-10-11T09:41:59.278577+0000","last_undegraded":"2023-10-11T09:41:59.278577+0000","last_fullsized":"2023-10-11T09:41:59.278577+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:34:50.579527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.135085+0000","last_change":"2023-10-11T09:41:59.270949+0000","last_active":"2023-10-11T09:42:02.135085+0000","last_peered":"2023-10-11T09:42:02.135085+0000","last_clean":"2023-10-11T09:42:02.135085+0000","last_became_active":"2023-10-11T09:41:59.270558+0000","last_became_peered":"2023-10-11T09:41:59.270558+0000","last_unstale":"2023-10-11T09:42:02.135085+0000","last_undegraded":"2023-10-11T09:42:02.135085+0000","last_fullsized":"2023-10-11T09:42:02.135085+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:07:43.684811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"36'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.278298+0000","last_change":"2023-10-11T09:41:58.773997+0000","last_active":"2023-10-11T09:41:59.278298+0000","last_peered":"2023-10-11T09:41:59.278298+0000","last_clean":"2023-10-11T09:41:59.278298+0000","last_became_active":"2023-10-11T09:41:58.773582+0000","last_became_peered":"2023-10-11T09:41:58.773582+0000","last_unstale":"2023-10-11T09:41:59.278298+0000","last_undegraded":"2023-10-11T09:41:59.278298+0000","last_fullsized":"2023-10-11T09:41:59.278298+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:00:41.605405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:01.632879+0000","last_change":"2023-10-11T09:41:59.264983+0000","last_active":"2023-10-11T09:42:01.632879+0000","last_peered":"2023-10-11T09:42:01.632879+0000","last_clean":"2023-10-11T09:42:01.632879+0000","last_became_active":"2023-10-11T09:41:59.264662+0000","last_became_peered":"2023-10-11T09:41:59.264662+0000","last_unstale":"2023-10-11T09:42:01.632879+0000","last_undegraded":"2023-10-11T09:42:01.632879+0000","last_fullsized":"2023-10-11T09:42:01.632879+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:51:26.096609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"36'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.272941+0000","last_change":"2023-10-11T09:41:58.775695+0000","last_active":"2023-10-11T09:41:59.272941+0000","last_peered":"2023-10-11T09:41:59.272941+0000","last_clean":"2023-10-11T09:41:59.272941+0000","last_became_active":"2023-10-11T09:41:58.775435+0000","last_became_peered":"2023-10-11T09:41:58.775435+0000","last_unstale":"2023-10-11T09:41:59.272941+0000","last_undegraded":"2023-10-11T09:41:59.272941+0000","last_fullsized":"2023-10-11T09:41:59.272941+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:16:23.663632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.134236+0000","last_change":"2023-10-11T09:41:59.259129+0000","last_active":"2023-10-11T09:42:02.134236+0000","last_peered":"2023-10-11T09:42:02.134236+0000","last_clean":"2023-10-11T09:42:02.134236+0000","last_became_active":"2023-10-11T09:41:59.258907+0000","last_became_peered":"2023-10-11T09:41:59.258907+0000","last_unstale":"2023-10-11T09:42:02.134236+0000","last_undegraded":"2023-10-11T09:42:02.134236+0000","last_fullsized":"2023-10-11T09:42:02.134236+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:07:46.727754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"36'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.273479+0000","last_change":"2023-10-11T09:41:58.777078+0000","last_active":"2023-10-11T09:41:59.273479+0000","last_peered":"2023-10-11T09:41:59.273479+0000","last_clean":"2023-10-11T09:41:59.273479+0000","last_became_active":"2023-10-11T09:41:58.776760+0000","last_became_peered":"2023-10-11T09:41:58.776760+0000","last_unstale":"2023-10-11T09:41:59.273479+0000","last_undegraded":"2023-10-11T09:41:59.273479+0000","last_fullsized":"2023-10-11T09:41:59.273479+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:30:07.413635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133334+0000","last_change":"2023-10-11T09:41:59.249575+0000","last_active":"2023-10-11T09:42:02.133334+0000","last_peered":"2023-10-11T09:42:02.133334+0000","last_clean":"2023-10-11T09:42:02.133334+0000","last_became_active":"2023-10-11T09:41:59.249320+0000","last_became_peered":"2023-10-11T09:41:59.249320+0000","last_unstale":"2023-10-11T09:42:02.133334+0000","last_undegraded":"2023-10-11T09:42:02.133334+0000","last_fullsized":"2023-10-11T09:42:02.133334+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:29:15.375697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"36'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.273485+0000","last_change":"2023-10-11T09:41:58.780188+0000","last_active":"2023-10-11T09:41:59.273485+0000","last_peered":"2023-10-11T09:41:59.273485+0000","last_clean":"2023-10-11T09:41:59.273485+0000","last_became_active":"2023-10-11T09:41:58.779846+0000","last_became_peered":"2023-10-11T09:41:58.779846+0000","last_unstale":"2023-10-11T09:41:59.273485+0000","last_undegraded":"2023-10-11T09:41:59.273485+0000","last_fullsized":"2023-10-11T09:41:59.273485+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:52:20.712655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.245928+0000","last_change":"2023-10-11T09:41:59.258246+0000","last_active":"2023-10-11T09:42:00.245928+0000","last_peered":"2023-10-11T09:42:00.245928+0000","last_clean":"2023-10-11T09:42:00.245928+0000","last_became_active":"2023-10-11T09:41:59.257935+0000","last_became_peered":"2023-10-11T09:41:59.257935+0000","last_unstale":"2023-10-11T09:42:00.245928+0000","last_undegraded":"2023-10-11T09:42:00.245928+0000","last_fullsized":"2023-10-11T09:42:00.245928+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:27:48.375035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"36'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.275029+0000","last_change":"2023-10-11T09:41:58.781289+0000","last_active":"2023-10-11T09:41:59.275029+0000","last_peered":"2023-10-11T09:41:59.275029+0000","last_clean":"2023-10-11T09:41:59.275029+0000","last_became_active":"2023-10-11T09:41:58.781013+0000","last_became_peered":"2023-10-11T09:41:58.781013+0000","last_unstale":"2023-10-11T09:41:59.275029+0000","last_undegraded":"2023-10-11T09:41:59.275029+0000","last_fullsized":"2023-10-11T09:41:59.275029+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:50:34.455895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:01.632527+0000","last_change":"2023-10-11T09:41:59.262808+0000","last_active":"2023-10-11T09:42:01.632527+0000","last_peered":"2023-10-11T09:42:01.632527+0000","last_clean":"2023-10-11T09:42:01.632527+0000","last_became_active":"2023-10-11T09:41:59.262515+0000","last_became_peered":"2023-10-11T09:41:59.262515+0000","last_unstale":"2023-10-11T09:42:01.632527+0000","last_undegraded":"2023-10-11T09:42:01.632527+0000","last_fullsized":"2023-10-11T09:42:01.632527+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:29:15.375697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.242698+0000","last_change":"2023-10-11T09:41:58.777202+0000","last_active":"2023-10-11T09:41:59.242698+0000","last_peered":"2023-10-11T09:41:59.242698+0000","last_clean":"2023-10-11T09:41:59.242698+0000","last_became_active":"2023-10-11T09:41:58.776863+0000","last_became_peered":"2023-10-11T09:41:58.776863+0000","last_unstale":"2023-10-11T09:41:59.242698+0000","last_undegraded":"2023-10-11T09:41:59.242698+0000","last_fullsized":"2023-10-11T09:41:59.242698+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:25:01.903933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133295+0000","last_change":"2023-10-11T09:41:59.275572+0000","last_active":"2023-10-11T09:42:02.133295+0000","last_peered":"2023-10-11T09:42:02.133295+0000","last_clean":"2023-10-11T09:42:02.133295+0000","last_became_active":"2023-10-11T09:41:59.275159+0000","last_became_peered":"2023-10-11T09:41:59.275159+0000","last_unstale":"2023-10-11T09:42:02.133295+0000","last_undegraded":"2023-10-11T09:42:02.133295+0000","last_fullsized":"2023-10-11T09:42:02.133295+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:29:09.854758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.242721+0000","last_change":"2023-10-11T09:41:58.774053+0000","last_active":"2023-10-11T09:41:59.242721+0000","last_peered":"2023-10-11T09:41:59.242721+0000","last_clean":"2023-10-11T09:41:59.242721+0000","last_became_active":"2023-10-11T09:41:58.773761+0000","last_became_peered":"2023-10-11T09:41:58.773761+0000","last_unstale":"2023-10-11T09:41:59.242721+0000","last_undegraded":"2023-10-11T09:41:59.242721+0000","last_fullsized":"2023-10-11T09:41:59.242721+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:00:41.605405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.245053+0000","last_change":"2023-10-11T09:41:59.254445+0000","last_active":"2023-10-11T09:42:00.245053+0000","last_peered":"2023-10-11T09:42:00.245053+0000","last_clean":"2023-10-11T09:42:00.245053+0000","last_became_active":"2023-10-11T09:41:59.253764+0000","last_became_peered":"2023-10-11T09:41:59.253764+0000","last_unstale":"2023-10-11T09:42:00.245053+0000","last_undegraded":"2023-10-11T09:42:00.245053+0000","last_fullsized":"2023-10-11T09:42:00.245053+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:50:35.471027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.245062+0000","last_change":"2023-10-11T09:41:58.775649+0000","last_active":"2023-10-11T09:41:59.245062+0000","last_peered":"2023-10-11T09:41:59.245062+0000","last_clean":"2023-10-11T09:41:59.245062+0000","last_became_active":"2023-10-11T09:41:58.773119+0000","last_became_peered":"2023-10-11T09:41:58.773119+0000","last_unstale":"2023-10-11T09:41:59.245062+0000","last_undegraded":"2023-10-11T09:41:59.245062+0000","last_fullsized":"2023-10-11T09:41:59.245062+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:48:02.478605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.246299+0000","last_change":"2023-10-11T09:41:59.266483+0000","last_active":"2023-10-11T09:42:00.246299+0000","last_peered":"2023-10-11T09:42:00.246299+0000","last_clean":"2023-10-11T09:42:00.246299+0000","last_became_active":"2023-10-11T09:41:59.266228+0000","last_became_peered":"2023-10-11T09:41:59.266228+0000","last_unstale":"2023-10-11T09:42:00.246299+0000","last_undegraded":"2023-10-11T09:42:00.246299+0000","last_fullsized":"2023-10-11T09:42:00.246299+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:14:53.871518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.256436+0000","last_change":"2023-10-11T09:41:58.780367+0000","last_active":"2023-10-11T09:41:59.256436+0000","last_peered":"2023-10-11T09:41:59.256436+0000","last_clean":"2023-10-11T09:41:59.256436+0000","last_became_active":"2023-10-11T09:41:58.779355+0000","last_became_peered":"2023-10-11T09:41:58.779355+0000","last_unstale":"2023-10-11T09:41:59.256436+0000","last_undegraded":"2023-10-11T09:41:59.256436+0000","last_fullsized":"2023-10-11T09:41:59.256436+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:24:11.694544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.134680+0000","last_change":"2023-10-11T09:41:59.274318+0000","last_active":"2023-10-11T09:42:02.134680+0000","last_peered":"2023-10-11T09:42:02.134680+0000","last_clean":"2023-10-11T09:42:02.134680+0000","last_became_active":"2023-10-11T09:41:59.274070+0000","last_became_peered":"2023-10-11T09:41:59.274070+0000","last_unstale":"2023-10-11T09:42:02.134680+0000","last_undegraded":"2023-10-11T09:42:02.134680+0000","last_fullsized":"2023-10-11T09:42:02.134680+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:32:44.448136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"36'3","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.273507+0000","last_change":"2023-10-11T09:41:58.776724+0000","last_active":"2023-10-11T09:41:59.273507+0000","last_peered":"2023-10-11T09:41:59.273507+0000","last_clean":"2023-10-11T09:41:59.273507+0000","last_became_active":"2023-10-11T09:41:58.776359+0000","last_became_peered":"2023-10-11T09:41:58.776359+0000","last_unstale":"2023-10-11T09:41:59.273507+0000","last_undegraded":"2023-10-11T09:41:59.273507+0000","last_fullsized":"2023-10-11T09:41:59.273507+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:55:29.823667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:01.632819+0000","last_change":"2023-10-11T09:41:59.252336+0000","last_active":"2023-10-11T09:42:01.632819+0000","last_peered":"2023-10-11T09:42:01.632819+0000","last_clean":"2023-10-11T09:42:01.632819+0000","last_became_active":"2023-10-11T09:41:59.252084+0000","last_became_peered":"2023-10-11T09:41:59.252084+0000","last_unstale":"2023-10-11T09:42:01.632819+0000","last_undegraded":"2023-10-11T09:42:01.632819+0000","last_fullsized":"2023-10-11T09:42:01.632819+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:29:09.854758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"36'6","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.281790+0000","last_change":"2023-10-11T09:41:58.775314+0000","last_active":"2023-10-11T09:41:59.281790+0000","last_peered":"2023-10-11T09:41:59.281790+0000","last_clean":"2023-10-11T09:41:59.281790+0000","last_became_active":"2023-10-11T09:41:58.775000+0000","last_became_peered":"2023-10-11T09:41:58.775000+0000","last_unstale":"2023-10-11T09:41:59.281790+0000","last_undegraded":"2023-10-11T09:41:59.281790+0000","last_fullsized":"2023-10-11T09:41:59.281790+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:23:59.470658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133578+0000","last_change":"2023-10-11T09:41:59.267987+0000","last_active":"2023-10-11T09:42:02.133578+0000","last_peered":"2023-10-11T09:42:02.133578+0000","last_clean":"2023-10-11T09:42:02.133578+0000","last_became_active":"2023-10-11T09:41:59.267705+0000","last_became_peered":"2023-10-11T09:41:59.267705+0000","last_unstale":"2023-10-11T09:42:02.133578+0000","last_undegraded":"2023-10-11T09:42:02.133578+0000","last_fullsized":"2023-10-11T09:42:02.133578+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:34:02.831174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"36'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.256670+0000","last_change":"2023-10-11T09:41:58.776379+0000","last_active":"2023-10-11T09:41:59.256670+0000","last_peered":"2023-10-11T09:41:59.256670+0000","last_clean":"2023-10-11T09:41:59.256670+0000","last_became_active":"2023-10-11T09:41:58.776071+0000","last_became_peered":"2023-10-11T09:41:58.776071+0000","last_unstale":"2023-10-11T09:41:59.256670+0000","last_undegraded":"2023-10-11T09:41:59.256670+0000","last_fullsized":"2023-10-11T09:41:59.256670+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:24:11.694544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.132753+0000","last_change":"2023-10-11T09:41:59.250221+0000","last_active":"2023-10-11T09:42:02.132753+0000","last_peered":"2023-10-11T09:42:02.132753+0000","last_clean":"2023-10-11T09:42:02.132753+0000","last_became_active":"2023-10-11T09:41:59.249523+0000","last_became_peered":"2023-10-11T09:41:59.249523+0000","last_unstale":"2023-10-11T09:42:02.132753+0000","last_undegraded":"2023-10-11T09:42:02.132753+0000","last_fullsized":"2023-10-11T09:42:02.132753+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:49:29.335655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.242895+0000","last_change":"2023-10-11T09:41:58.776351+0000","last_active":"2023-10-11T09:41:59.242895+0000","last_peered":"2023-10-11T09:41:59.242895+0000","last_clean":"2023-10-11T09:41:59.242895+0000","last_became_active":"2023-10-11T09:41:58.776076+0000","last_became_peered":"2023-10-11T09:41:58.776076+0000","last_unstale":"2023-10-11T09:41:59.242895+0000","last_undegraded":"2023-10-11T09:41:59.242895+0000","last_fullsized":"2023-10-11T09:41:59.242895+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:18:56.139478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.246103+0000","last_change":"2023-10-11T09:41:59.266545+0000","last_active":"2023-10-11T09:42:00.246103+0000","last_peered":"2023-10-11T09:42:00.246103+0000","last_clean":"2023-10-11T09:42:00.246103+0000","last_became_active":"2023-10-11T09:41:59.266181+0000","last_became_peered":"2023-10-11T09:41:59.266181+0000","last_unstale":"2023-10-11T09:42:00.246103+0000","last_undegraded":"2023-10-11T09:42:00.246103+0000","last_fullsized":"2023-10-11T09:42:00.246103+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:46:25.915286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.255044+0000","last_change":"2023-10-11T09:41:58.781510+0000","last_active":"2023-10-11T09:41:59.255044+0000","last_peered":"2023-10-11T09:41:59.255044+0000","last_clean":"2023-10-11T09:41:59.255044+0000","last_became_active":"2023-10-11T09:41:58.781083+0000","last_became_peered":"2023-10-11T09:41:58.781083+0000","last_unstale":"2023-10-11T09:41:59.255044+0000","last_undegraded":"2023-10-11T09:41:59.255044+0000","last_fullsized":"2023-10-11T09:41:59.255044+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:02:09.585130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133419+0000","last_change":"2023-10-11T09:41:59.250127+0000","last_active":"2023-10-11T09:42:02.133419+0000","last_peered":"2023-10-11T09:42:02.133419+0000","last_clean":"2023-10-11T09:42:02.133419+0000","last_became_active":"2023-10-11T09:41:59.249850+0000","last_became_peered":"2023-10-11T09:41:59.249850+0000","last_unstale":"2023-10-11T09:42:02.133419+0000","last_undegraded":"2023-10-11T09:42:02.133419+0000","last_fullsized":"2023-10-11T09:42:02.133419+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:29:15.375697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"36'8","reported_seq":19,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.276220+0000","last_change":"2023-10-11T09:41:58.782484+0000","last_active":"2023-10-11T09:41:59.276220+0000","last_peered":"2023-10-11T09:41:59.276220+0000","last_clean":"2023-10-11T09:41:59.276220+0000","last_became_active":"2023-10-11T09:41:58.781980+0000","last_became_peered":"2023-10-11T09:41:58.781980+0000","last_unstale":"2023-10-11T09:41:59.276220+0000","last_undegraded":"2023-10-11T09:41:59.276220+0000","last_fullsized":"2023-10-11T09:41:59.276220+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:05:43.880930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.134151+0000","last_change":"2023-10-11T09:41:59.257871+0000","last_active":"2023-10-11T09:42:02.134151+0000","last_peered":"2023-10-11T09:42:02.134151+0000","last_clean":"2023-10-11T09:42:02.134151+0000","last_became_active":"2023-10-11T09:41:59.257622+0000","last_became_peered":"2023-10-11T09:41:59.257622+0000","last_unstale":"2023-10-11T09:42:02.134151+0000","last_undegraded":"2023-10-11T09:42:02.134151+0000","last_fullsized":"2023-10-11T09:42:02.134151+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:34:02.831174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"36'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.278482+0000","last_change":"2023-10-11T09:41:58.782590+0000","last_active":"2023-10-11T09:41:59.278482+0000","last_peered":"2023-10-11T09:41:59.278482+0000","last_clean":"2023-10-11T09:41:59.278482+0000","last_became_active":"2023-10-11T09:41:58.782326+0000","last_became_peered":"2023-10-11T09:41:58.782326+0000","last_unstale":"2023-10-11T09:41:59.278482+0000","last_undegraded":"2023-10-11T09:41:59.278482+0000","last_fullsized":"2023-10-11T09:41:59.278482+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:11:34.532326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.247230+0000","last_change":"2023-10-11T09:41:59.270677+0000","last_active":"2023-10-11T09:42:00.247230+0000","last_peered":"2023-10-11T09:42:00.247230+0000","last_clean":"2023-10-11T09:42:00.247230+0000","last_became_active":"2023-10-11T09:41:59.270348+0000","last_became_peered":"2023-10-11T09:41:59.270348+0000","last_unstale":"2023-10-11T09:42:00.247230+0000","last_undegraded":"2023-10-11T09:42:00.247230+0000","last_fullsized":"2023-10-11T09:42:00.247230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:34:02.831174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"36'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.273284+0000","last_change":"2023-10-11T09:41:58.778144+0000","last_active":"2023-10-11T09:41:59.273284+0000","last_peered":"2023-10-11T09:41:59.273284+0000","last_clean":"2023-10-11T09:41:59.273284+0000","last_became_active":"2023-10-11T09:41:58.777788+0000","last_became_peered":"2023-10-11T09:41:58.777788+0000","last_unstale":"2023-10-11T09:41:59.273284+0000","last_undegraded":"2023-10-11T09:41:59.273284+0000","last_fullsized":"2023-10-11T09:41:59.273284+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:42:13.962137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:01.632636+0000","last_change":"2023-10-11T09:41:59.251031+0000","last_active":"2023-10-11T09:42:01.632636+0000","last_peered":"2023-10-11T09:42:01.632636+0000","last_clean":"2023-10-11T09:42:01.632636+0000","last_became_active":"2023-10-11T09:41:59.250626+0000","last_became_peered":"2023-10-11T09:41:59.250626+0000","last_unstale":"2023-10-11T09:42:01.632636+0000","last_undegraded":"2023-10-11T09:42:01.632636+0000","last_fullsized":"2023-10-11T09:42:01.632636+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:32:44.448136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"36'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.274594+0000","last_change":"2023-10-11T09:41:58.773947+0000","last_active":"2023-10-11T09:41:59.274594+0000","last_peered":"2023-10-11T09:41:59.274594+0000","last_clean":"2023-10-11T09:41:59.274594+0000","last_became_active":"2023-10-11T09:41:58.773642+0000","last_became_peered":"2023-10-11T09:41:58.773642+0000","last_unstale":"2023-10-11T09:41:59.274594+0000","last_undegraded":"2023-10-11T09:41:59.274594+0000","last_fullsized":"2023-10-11T09:41:59.274594+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:49:37.959487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.245415+0000","last_change":"2023-10-11T09:41:59.252857+0000","last_active":"2023-10-11T09:42:00.245415+0000","last_peered":"2023-10-11T09:42:00.245415+0000","last_clean":"2023-10-11T09:42:00.245415+0000","last_became_active":"2023-10-11T09:41:59.252603+0000","last_became_peered":"2023-10-11T09:41:59.252603+0000","last_unstale":"2023-10-11T09:42:00.245415+0000","last_undegraded":"2023-10-11T09:42:00.245415+0000","last_fullsized":"2023-10-11T09:42:00.245415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:35:44.282265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.242457+0000","last_change":"2023-10-11T09:41:58.775051+0000","last_active":"2023-10-11T09:41:59.242457+0000","last_peered":"2023-10-11T09:41:59.242457+0000","last_clean":"2023-10-11T09:41:59.242457+0000","last_became_active":"2023-10-11T09:41:58.774781+0000","last_became_peered":"2023-10-11T09:41:58.774781+0000","last_unstale":"2023-10-11T09:41:59.242457+0000","last_undegraded":"2023-10-11T09:41:59.242457+0000","last_fullsized":"2023-10-11T09:41:59.242457+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:35:43.267132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.245933+0000","last_change":"2023-10-11T09:41:59.276846+0000","last_active":"2023-10-11T09:42:00.245933+0000","last_peered":"2023-10-11T09:42:00.245933+0000","last_clean":"2023-10-11T09:42:00.245933+0000","last_became_active":"2023-10-11T09:41:59.276042+0000","last_became_peered":"2023-10-11T09:41:59.276042+0000","last_unstale":"2023-10-11T09:42:00.245933+0000","last_undegraded":"2023-10-11T09:42:00.245933+0000","last_fullsized":"2023-10-11T09:42:00.245933+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:07:13.635301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"36'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.266966+0000","last_change":"2023-10-11T09:41:58.777266+0000","last_active":"2023-10-11T09:41:59.266966+0000","last_peered":"2023-10-11T09:41:59.266966+0000","last_clean":"2023-10-11T09:41:59.266966+0000","last_became_active":"2023-10-11T09:41:58.776620+0000","last_became_peered":"2023-10-11T09:41:58.776620+0000","last_unstale":"2023-10-11T09:41:59.266966+0000","last_undegraded":"2023-10-11T09:41:59.266966+0000","last_fullsized":"2023-10-11T09:41:59.266966+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:55:46.983239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.132993+0000","last_change":"2023-10-11T09:41:59.276325+0000","last_active":"2023-10-11T09:42:02.132993+0000","last_peered":"2023-10-11T09:42:02.132993+0000","last_clean":"2023-10-11T09:42:02.132993+0000","last_became_active":"2023-10-11T09:41:59.275987+0000","last_became_peered":"2023-10-11T09:41:59.275987+0000","last_unstale":"2023-10-11T09:42:02.132993+0000","last_undegraded":"2023-10-11T09:42:02.132993+0000","last_fullsized":"2023-10-11T09:42:02.132993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:14:53.871518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.234885+0000","last_change":"2023-10-11T09:41:58.779536+0000","last_active":"2023-10-11T09:41:59.234885+0000","last_peered":"2023-10-11T09:41:59.234885+0000","last_clean":"2023-10-11T09:41:59.234885+0000","last_became_active":"2023-10-11T09:41:58.779262+0000","last_became_peered":"2023-10-11T09:41:58.779262+0000","last_unstale":"2023-10-11T09:41:59.234885+0000","last_undegraded":"2023-10-11T09:41:59.234885+0000","last_fullsized":"2023-10-11T09:41:59.234885+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:44:25.949842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.246258+0000","last_change":"2023-10-11T09:41:59.265507+0000","last_active":"2023-10-11T09:42:00.246258+0000","last_peered":"2023-10-11T09:42:00.246258+0000","last_clean":"2023-10-11T09:42:00.246258+0000","last_became_active":"2023-10-11T09:41:59.265264+0000","last_became_peered":"2023-10-11T09:41:59.265264+0000","last_unstale":"2023-10-11T09:42:00.246258+0000","last_undegraded":"2023-10-11T09:42:00.246258+0000","last_fullsized":"2023-10-11T09:42:00.246258+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:51:26.096609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.241218+0000","last_change":"2023-10-11T09:41:58.773543+0000","last_active":"2023-10-11T09:41:59.241218+0000","last_peered":"2023-10-11T09:41:59.241218+0000","last_clean":"2023-10-11T09:41:59.241218+0000","last_became_active":"2023-10-11T09:41:58.773285+0000","last_became_peered":"2023-10-11T09:41:58.773285+0000","last_unstale":"2023-10-11T09:41:59.241218+0000","last_undegraded":"2023-10-11T09:41:59.241218+0000","last_fullsized":"2023-10-11T09:41:59.241218+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:09:10.546323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133525+0000","last_change":"2023-10-11T09:41:59.275401+0000","last_active":"2023-10-11T09:42:02.133525+0000","last_peered":"2023-10-11T09:42:02.133525+0000","last_clean":"2023-10-11T09:42:02.133525+0000","last_became_active":"2023-10-11T09:41:59.274821+0000","last_became_peered":"2023-10-11T09:41:59.274821+0000","last_unstale":"2023-10-11T09:42:02.133525+0000","last_undegraded":"2023-10-11T09:42:02.133525+0000","last_fullsized":"2023-10-11T09:42:02.133525+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:51:26.096609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.234879+0000","last_change":"2023-10-11T09:41:58.778916+0000","last_active":"2023-10-11T09:41:59.234879+0000","last_peered":"2023-10-11T09:41:59.234879+0000","last_clean":"2023-10-11T09:41:59.234879+0000","last_became_active":"2023-10-11T09:41:58.778610+0000","last_became_peered":"2023-10-11T09:41:58.778610+0000","last_unstale":"2023-10-11T09:41:59.234879+0000","last_undegraded":"2023-10-11T09:41:59.234879+0000","last_fullsized":"2023-10-11T09:41:59.234879+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:40:16.473783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.247174+0000","last_change":"2023-10-11T09:41:59.270447+0000","last_active":"2023-10-11T09:42:00.247174+0000","last_peered":"2023-10-11T09:42:00.247174+0000","last_clean":"2023-10-11T09:42:00.247174+0000","last_became_active":"2023-10-11T09:41:59.270124+0000","last_became_peered":"2023-10-11T09:41:59.270124+0000","last_unstale":"2023-10-11T09:42:00.247174+0000","last_undegraded":"2023-10-11T09:42:00.247174+0000","last_fullsized":"2023-10-11T09:42:00.247174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:45:37.542487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.241255+0000","last_change":"2023-10-11T09:41:58.777578+0000","last_active":"2023-10-11T09:41:59.241255+0000","last_peered":"2023-10-11T09:41:59.241255+0000","last_clean":"2023-10-11T09:41:59.241255+0000","last_became_active":"2023-10-11T09:41:58.777251+0000","last_became_peered":"2023-10-11T09:41:58.777251+0000","last_unstale":"2023-10-11T09:41:59.241255+0000","last_undegraded":"2023-10-11T09:41:59.241255+0000","last_fullsized":"2023-10-11T09:41:59.241255+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:23:59.470658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":181,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.240598+0000","last_change":"2023-10-11T09:40:29.874372+0000","last_active":"2023-10-11T09:41:59.240598+0000","last_peered":"2023-10-11T09:41:59.240598+0000","last_clean":"2023-10-11T09:41:59.240598+0000","last_became_active":"2023-10-11T09:40:29.873621+0000","last_became_peered":"2023-10-11T09:40:29.873621+0000","last_unstale":"2023-10-11T09:41:59.240598+0000","last_undegraded":"2023-10-11T09:41:59.240598+0000","last_fullsized":"2023-10-11T09:41:59.240598+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_clean_scrub_stamp":"2023-10-11T09:40:28.742276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:22:07.985311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.251308+0000","last_change":"2023-10-11T09:40:36.148773+0000","last_active":"2023-10-11T09:41:59.251308+0000","last_peered":"2023-10-11T09:41:59.251308+0000","last_clean":"2023-10-11T09:41:59.251308+0000","last_became_active":"2023-10-11T09:40:36.148489+0000","last_became_peered":"2023-10-11T09:40:36.148489+0000","last_unstale":"2023-10-11T09:41:59.251308+0000","last_undegraded":"2023-10-11T09:41:59.251308+0000","last_fullsized":"2023-10-11T09:41:59.251308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.132837+0000","last_change":"2023-10-11T09:41:59.268241+0000","last_active":"2023-10-11T09:42:02.132837+0000","last_peered":"2023-10-11T09:42:02.132837+0000","last_clean":"2023-10-11T09:42:02.132837+0000","last_became_active":"2023-10-11T09:41:59.267982+0000","last_became_peered":"2023-10-11T09:41:59.267982+0000","last_unstale":"2023-10-11T09:42:02.132837+0000","last_undegraded":"2023-10-11T09:42:02.132837+0000","last_fullsized":"2023-10-11T09:42:02.132837+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:42:14.977269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"36'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.273771+0000","last_change":"2023-10-11T09:41:58.777702+0000","last_active":"2023-10-11T09:41:59.273771+0000","last_peered":"2023-10-11T09:41:59.273771+0000","last_clean":"2023-10-11T09:41:59.273771+0000","last_became_active":"2023-10-11T09:41:58.777413+0000","last_became_peered":"2023-10-11T09:41:58.777413+0000","last_unstale":"2023-10-11T09:41:59.273771+0000","last_undegraded":"2023-10-11T09:41:59.273771+0000","last_fullsized":"2023-10-11T09:41:59.273771+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:58:21.613408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":28,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.240155+0000","last_change":"2023-10-11T09:40:36.149739+0000","last_active":"2023-10-11T09:41:59.240155+0000","last_peered":"2023-10-11T09:41:59.240155+0000","last_clean":"2023-10-11T09:41:59.240155+0000","last_became_active":"2023-10-11T09:40:36.149419+0000","last_became_peered":"2023-10-11T09:40:36.149419+0000","last_unstale":"2023-10-11T09:41:59.240155+0000","last_undegraded":"2023-10-11T09:41:59.240155+0000","last_fullsized":"2023-10-11T09:41:59.240155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:54:50.640362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.246403+0000","last_change":"2023-10-11T09:41:59.274654+0000","last_active":"2023-10-11T09:42:00.246403+0000","last_peered":"2023-10-11T09:42:00.246403+0000","last_clean":"2023-10-11T09:42:00.246403+0000","last_became_active":"2023-10-11T09:41:59.274296+0000","last_became_peered":"2023-10-11T09:41:59.274296+0000","last_unstale":"2023-10-11T09:42:00.246403+0000","last_undegraded":"2023-10-11T09:42:00.246403+0000","last_fullsized":"2023-10-11T09:42:00.246403+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:29:17.473816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"37'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.245245+0000","last_change":"2023-10-11T09:41:58.778782+0000","last_active":"2023-10-11T09:42:00.245245+0000","last_peered":"2023-10-11T09:42:00.245245+0000","last_clean":"2023-10-11T09:42:00.245245+0000","last_became_active":"2023-10-11T09:41:58.778514+0000","last_became_peered":"2023-10-11T09:41:58.778514+0000","last_unstale":"2023-10-11T09:42:00.245245+0000","last_undegraded":"2023-10-11T09:42:00.245245+0000","last_fullsized":"2023-10-11T09:42:00.245245+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:41:27.673781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":25,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.234843+0000","last_change":"2023-10-11T09:40:36.216248+0000","last_active":"2023-10-11T09:41:59.234843+0000","last_peered":"2023-10-11T09:41:59.234843+0000","last_clean":"2023-10-11T09:41:59.234843+0000","last_became_active":"2023-10-11T09:40:36.215890+0000","last_became_peered":"2023-10-11T09:40:36.215890+0000","last_unstale":"2023-10-11T09:41:59.234843+0000","last_undegraded":"2023-10-11T09:41:59.234843+0000","last_fullsized":"2023-10-11T09:41:59.234843+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.135487+0000","last_change":"2023-10-11T09:41:59.253002+0000","last_active":"2023-10-11T09:42:02.135487+0000","last_peered":"2023-10-11T09:42:02.135487+0000","last_clean":"2023-10-11T09:42:02.135487+0000","last_became_active":"2023-10-11T09:41:59.252742+0000","last_became_peered":"2023-10-11T09:41:59.252742+0000","last_unstale":"2023-10-11T09:42:02.135487+0000","last_undegraded":"2023-10-11T09:42:02.135487+0000","last_fullsized":"2023-10-11T09:42:02.135487+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:46:25.915286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.242326+0000","last_change":"2023-10-11T09:41:58.778606+0000","last_active":"2023-10-11T09:41:59.242326+0000","last_peered":"2023-10-11T09:41:59.242326+0000","last_clean":"2023-10-11T09:41:59.242326+0000","last_became_active":"2023-10-11T09:41:58.778305+0000","last_became_peered":"2023-10-11T09:41:58.778305+0000","last_unstale":"2023-10-11T09:41:59.242326+0000","last_undegraded":"2023-10-11T09:41:59.242326+0000","last_fullsized":"2023-10-11T09:41:59.242326+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:55:29.823667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.246884+0000","last_change":"2023-10-11T09:40:36.151710+0000","last_active":"2023-10-11T09:41:59.246884+0000","last_peered":"2023-10-11T09:41:59.246884+0000","last_clean":"2023-10-11T09:41:59.246884+0000","last_became_active":"2023-10-11T09:40:36.151219+0000","last_became_peered":"2023-10-11T09:40:36.151219+0000","last_unstale":"2023-10-11T09:41:59.246884+0000","last_undegraded":"2023-10-11T09:41:59.246884+0000","last_fullsized":"2023-10-11T09:41:59.246884+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133803+0000","last_change":"2023-10-11T09:41:59.252548+0000","last_active":"2023-10-11T09:42:02.133803+0000","last_peered":"2023-10-11T09:42:02.133803+0000","last_clean":"2023-10-11T09:42:02.133803+0000","last_became_active":"2023-10-11T09:41:59.252259+0000","last_became_peered":"2023-10-11T09:41:59.252259+0000","last_unstale":"2023-10-11T09:42:02.133803+0000","last_undegraded":"2023-10-11T09:42:02.133803+0000","last_fullsized":"2023-10-11T09:42:02.133803+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:07:46.727754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.240925+0000","last_change":"2023-10-11T09:41:58.772968+0000","last_active":"2023-10-11T09:41:59.240925+0000","last_peered":"2023-10-11T09:41:59.240925+0000","last_clean":"2023-10-11T09:41:59.240925+0000","last_became_active":"2023-10-11T09:41:58.772686+0000","last_became_peered":"2023-10-11T09:41:58.772686+0000","last_unstale":"2023-10-11T09:41:59.240925+0000","last_undegraded":"2023-10-11T09:41:59.240925+0000","last_fullsized":"2023-10-11T09:41:59.240925+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:58:21.613408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.242524+0000","last_change":"2023-10-11T09:40:36.160620+0000","last_active":"2023-10-11T09:41:59.242524+0000","last_peered":"2023-10-11T09:41:59.242524+0000","last_clean":"2023-10-11T09:41:59.242524+0000","last_became_active":"2023-10-11T09:40:36.160171+0000","last_became_peered":"2023-10-11T09:40:36.160171+0000","last_unstale":"2023-10-11T09:41:59.242524+0000","last_undegraded":"2023-10-11T09:41:59.242524+0000","last_fullsized":"2023-10-11T09:41:59.242524+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133691+0000","last_change":"2023-10-11T09:41:59.260111+0000","last_active":"2023-10-11T09:42:02.133691+0000","last_peered":"2023-10-11T09:42:02.133691+0000","last_clean":"2023-10-11T09:42:02.133691+0000","last_became_active":"2023-10-11T09:41:59.259796+0000","last_became_peered":"2023-10-11T09:41:59.259796+0000","last_unstale":"2023-10-11T09:42:02.133691+0000","last_undegraded":"2023-10-11T09:42:02.133691+0000","last_fullsized":"2023-10-11T09:42:02.133691+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:55:30.838799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.255282+0000","last_change":"2023-10-11T09:41:58.773403+0000","last_active":"2023-10-11T09:41:59.255282+0000","last_peered":"2023-10-11T09:41:59.255282+0000","last_clean":"2023-10-11T09:41:59.255282+0000","last_became_active":"2023-10-11T09:41:58.773132+0000","last_became_peered":"2023-10-11T09:41:58.773132+0000","last_unstale":"2023-10-11T09:41:59.255282+0000","last_undegraded":"2023-10-11T09:41:59.255282+0000","last_fullsized":"2023-10-11T09:41:59.255282+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:28:16.256141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":28,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.241892+0000","last_change":"2023-10-11T09:40:36.162174+0000","last_active":"2023-10-11T09:41:59.241892+0000","last_peered":"2023-10-11T09:41:59.241892+0000","last_clean":"2023-10-11T09:41:59.241892+0000","last_became_active":"2023-10-11T09:40:36.161710+0000","last_became_peered":"2023-10-11T09:40:36.161710+0000","last_unstale":"2023-10-11T09:41:59.241892+0000","last_undegraded":"2023-10-11T09:41:59.241892+0000","last_fullsized":"2023-10-11T09:41:59.241892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:57:03.630878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:01.632131+0000","last_change":"2023-10-11T09:41:59.251157+0000","last_active":"2023-10-11T09:42:01.632131+0000","last_peered":"2023-10-11T09:42:01.632131+0000","last_clean":"2023-10-11T09:42:01.632131+0000","last_became_active":"2023-10-11T09:41:59.250831+0000","last_became_peered":"2023-10-11T09:41:59.250831+0000","last_unstale":"2023-10-11T09:42:01.632131+0000","last_undegraded":"2023-10-11T09:42:01.632131+0000","last_fullsized":"2023-10-11T09:42:01.632131+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:34:02.831174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.242946+0000","last_change":"2023-10-11T09:41:58.779092+0000","last_active":"2023-10-11T09:41:59.242946+0000","last_peered":"2023-10-11T09:41:59.242946+0000","last_clean":"2023-10-11T09:41:59.242946+0000","last_became_active":"2023-10-11T09:41:58.778781+0000","last_became_peered":"2023-10-11T09:41:58.778781+0000","last_unstale":"2023-10-11T09:41:59.242946+0000","last_undegraded":"2023-10-11T09:41:59.242946+0000","last_fullsized":"2023-10-11T09:41:59.242946+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:07:42.669679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":24,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.244192+0000","last_change":"2023-10-11T09:40:36.150224+0000","last_active":"2023-10-11T09:41:59.244192+0000","last_peered":"2023-10-11T09:41:59.244192+0000","last_clean":"2023-10-11T09:41:59.244192+0000","last_became_active":"2023-10-11T09:40:36.149919+0000","last_became_peered":"2023-10-11T09:40:36.149919+0000","last_unstale":"2023-10-11T09:41:59.244192+0000","last_undegraded":"2023-10-11T09:41:59.244192+0000","last_fullsized":"2023-10-11T09:41:59.244192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.246688+0000","last_change":"2023-10-11T09:41:59.275142+0000","last_active":"2023-10-11T09:42:00.246688+0000","last_peered":"2023-10-11T09:42:00.246688+0000","last_clean":"2023-10-11T09:42:00.246688+0000","last_became_active":"2023-10-11T09:41:59.274690+0000","last_became_peered":"2023-10-11T09:41:59.274690+0000","last_unstale":"2023-10-11T09:42:00.246688+0000","last_undegraded":"2023-10-11T09:42:00.246688+0000","last_fullsized":"2023-10-11T09:42:00.246688+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:32:36.655338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.241558+0000","last_change":"2023-10-11T09:41:58.773081+0000","last_active":"2023-10-11T09:41:59.241558+0000","last_peered":"2023-10-11T09:41:59.241558+0000","last_clean":"2023-10-11T09:41:59.241558+0000","last_became_active":"2023-10-11T09:41:58.772778+0000","last_became_peered":"2023-10-11T09:41:58.772778+0000","last_unstale":"2023-10-11T09:41:59.241558+0000","last_undegraded":"2023-10-11T09:41:59.241558+0000","last_fullsized":"2023-10-11T09:41:59.241558+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:05:43.880930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":24,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.255713+0000","last_change":"2023-10-11T09:40:36.151839+0000","last_active":"2023-10-11T09:41:59.255713+0000","last_peered":"2023-10-11T09:41:59.255713+0000","last_clean":"2023-10-11T09:41:59.255713+0000","last_became_active":"2023-10-11T09:40:36.151179+0000","last_became_peered":"2023-10-11T09:40:36.151179+0000","last_unstale":"2023-10-11T09:41:59.255713+0000","last_undegraded":"2023-10-11T09:41:59.255713+0000","last_fullsized":"2023-10-11T09:41:59.255713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":1032192,"data_stored":767390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.21,"15min":1.21},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.3440000000000001}]},{"osd":1,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":2.25,"5min":2.25,"15min":2.25},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.359}]},{"osd":2,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.206},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":1.2390000000000001}]},{"osd":3,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.286,"15min":1.286},"min":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":2.1579999999999999,"5min":2.1579999999999999,"15min":2.1579999999999999},"last":1.073}]},{"osd":4,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":1.036}]},{"osd":5,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.292},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":3.0750000000000002,"5min":3.0750000000000002,"15min":3.0750000000000002},"last":1.2689999999999999}]},{"osd":6,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.49,"15min":1.49},"min":{"1min":1.153,"5min":1.153,"15min":1.153},"max":{"1min":3.137,"5min":3.137,"15min":3.137},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"min":{"1min":1.099,"5min":1.099,"15min":1.099},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":1.3200000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":425984,"data_stored":175970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.90600000000000003}]},{"osd":1,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.202,"15min":1.202},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":1.8260000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.2250000000000001}]},{"osd":2,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":1.079},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":2.605,"5min":2.605,"15min":2.605},"last":0.98999999999999999}]},{"osd":3,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":3.0819999999999999,"5min":3.0819999999999999,"15min":3.0819999999999999},"last":0.82599999999999996}]},{"osd":4,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.337,"15min":1.337},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.978,"5min":1.978,"15min":1.978},"last":1.111},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.6920000000000002,"5min":2.6920000000000002,"15min":2.6920000000000002},"last":1.302}]},{"osd":5,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.264},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":2.7290000000000001,"5min":2.7290000000000001,"15min":2.7290000000000001},"last":1.347}]},{"osd":7,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":2.1899999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.3050000000000002,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.175}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":430080,"data_stored":175500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":1.23}]},{"osd":1,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":1.1559999999999999}]},{"osd":2,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.831,"5min":1.831,"15min":1.831},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.284}]},{"osd":3,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.319},{"interface":"front","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.419}]},{"osd":4,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":1.369},{"interface":"front","average":{"1min":1.228,"5min":1.228,"15min":1.228},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":1.0840000000000001}]},{"osd":6,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.4529999999999998,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":1.4610000000000001}]},{"osd":7,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":3.6120000000000001,"5min":3.6120000000000001,"15min":3.6120000000000001},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"min":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":3.2109999999999999,"5min":3.2109999999999999,"15min":3.2109999999999999},"last":1.55}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":430080,"data_stored":175992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":1.286},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.1160000000000001,"5min":2.1160000000000001,"15min":2.1160000000000001},"last":1.381}]},{"osd":1,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.98399999999999999}]},{"osd":2,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":1.345},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":1.7130000000000001}]},{"osd":3,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.367,"5min":1.367,"15min":1.367},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.3340000000000001,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":1.6679999999999999}]},{"osd":5,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.236,"15min":1.236},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.93999999999999995}]},{"osd":6,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.8,"5min":1.8,"15min":1.8},"last":1.121},{"interface":"front","average":{"1min":1.357,"5min":1.357,"15min":1.357},"min":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.5549999999999999}]},{"osd":7,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.482},{"interface":"front","average":{"1min":1.516,"5min":1.516,"15min":1.516},"min":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"max":{"1min":1.9990000000000001,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":1.786}]}]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":438272,"data_stored":176986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.747},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.48699999999999999}]},{"osd":1,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.60199999999999998}]},{"osd":2,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.5870000000000002,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":0.98399999999999999}]},{"osd":4,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.395,"5min":2.395,"15min":2.395},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.367,"5min":1.367,"15min":1.367},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":3.1579999999999999,"5min":3.1579999999999999,"15min":3.1579999999999999},"last":1.1870000000000001}]},{"osd":5,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.393,"15min":1.393},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":3.0009999999999999,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":1.03},{"interface":"front","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":3.8050000000000002,"5min":3.8050000000000002,"15min":3.8050000000000002},"last":1.224}]},{"osd":6,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":2.1949999999999998,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":3.2549999999999999,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":1.266}]},{"osd":7,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":3.7490000000000001,"5min":3.7490000000000001,"15min":3.7490000000000001},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.448,"5min":1.448,"15min":1.448},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":1.121}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":425984,"data_stored":175458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":2.0600000000000001,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":2.8919999999999999,"5min":2.8919999999999999,"15min":2.8919999999999999},"last":0.68000000000000005}]},{"osd":1,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":3.7650000000000001,"5min":3.7650000000000001,"15min":3.7650000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.5339999999999998,"5min":2.5339999999999998,"15min":2.5339999999999998},"last":0.71499999999999997}]},{"osd":3,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.403,"5min":2.403,"15min":2.403},"last":0.65500000000000003},{"interface":"front","average":{"1min":1.117,"5min":1.117,"15min":1.117},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":2.7589999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":0.63600000000000001}]},{"osd":4,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.476,"5min":1.476,"15min":1.476},"min":{"1min":1.085,"5min":1.085,"15min":1.085},"max":{"1min":2.6859999999999999,"5min":2.6859999999999999,"15min":2.6859999999999999},"last":1.111}]},{"osd":5,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.409,"15min":1.409},"min":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"max":{"1min":3.044,"5min":3.044,"15min":3.044},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":3.5790000000000002,"5min":3.5790000000000002,"15min":3.5790000000000002},"last":0.89500000000000002}]},{"osd":6,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"min":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"max":{"1min":3.1600000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.044},{"interface":"front","average":{"1min":1.397,"5min":1.397,"15min":1.397},"min":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"max":{"1min":2.839,"5min":2.839,"15min":2.839},"last":0.85399999999999998}]},{"osd":7,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.49,"15min":1.49},"min":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":3.1230000000000002,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"min":{"1min":1.161,"5min":1.161,"15min":1.161},"max":{"1min":2.988,"5min":2.988,"15min":2.988},"last":1.0780000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":425984,"data_stored":175478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:41:28 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":1.159}]},{"osd":2,"last update":"Wed Oct 11 09:41:28 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.85599999999999998}]},{"osd":3,"last update":"Wed Oct 11 09:41:28 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":0.60699999999999998}]},{"osd":4,"last update":"Wed Oct 11 09:41:28 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":2.016,"5min":2.016,"15min":2.016},"last":1.147},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":2.323,"5min":2.323,"15min":2.323},"last":1.1240000000000001}]},{"osd":5,"last update":"Wed Oct 11 09:41:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":1.04,"5min":1.04,"15min":1.04},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.113}]},{"osd":6,"last update":"Wed Oct 11 09:41:28 2023","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"min":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"max":{"1min":2.8940000000000001,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":1.022,"5min":1.022,"15min":1.022},"max":{"1min":3.3719999999999999,"5min":3.3719999999999999,"15min":3.3719999999999999},"last":1.173}]},{"osd":7,"last update":"Wed Oct 11 09:41:28 2023","interfaces":[{"interface":"back","average":{"1min":1.468,"5min":1.468,"15min":1.468},"min":{"1min":1.171,"5min":1.171,"15min":1.171},"max":{"1min":2.194,"5min":2.194,"15min":2.194},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"min":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"max":{"1min":1.968,"5min":1.968,"15min":1.968},"last":0.99399999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":1019904,"data_stored":765790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 11 09:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.3100000000000001,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":0.73799999999999999}]},{"osd":2,"last update":"Wed Oct 11 09:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.63900000000000001}]},{"osd":3,"last update":"Wed Oct 11 09:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.7639999999999998,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":0.57699999999999996},{"interface":"front","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":0.93100000000000005}]},{"osd":4,"last update":"Wed Oct 11 09:41:27 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.242},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.6629999999999998,"5min":2.6629999999999998,"15min":2.6629999999999998},"last":1.1539999999999999}]},{"osd":5,"last update":"Wed Oct 11 09:41:27 2023","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"min":{"1min":0.998,"5min":0.998,"15min":0.998},"max":{"1min":3.0150000000000001,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":1.276},{"interface":"front","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":3.601,"5min":3.601,"15min":3.601},"last":1.331}]},{"osd":6,"last update":"Wed Oct 11 09:41:27 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.825,"5min":1.825,"15min":1.825},"last":0.997},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"min":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":1.1200000000000001}]},{"osd":7,"last update":"Wed Oct 11 09:41:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"min":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":1.381},{"interface":"front","average":{"1min":1.417,"5min":1.417,"15min":1.417},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":2.5419999999999998,"5min":2.5419999999999998,"15min":2.5419999999999998},"last":1.3069999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-11T09:42:04.484 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-11T09:42:04.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.681+0000 7f383f8e7700 1 Processor -- start 2023-10-11T09:42:04.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.681+0000 7f383f8e7700 1 -- start start 2023-10-11T09:42:04.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.681+0000 7f383f8e7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f383815e210 0x7f3838071040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:04.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.681+0000 7f383f8e7700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f38380ca760 0x7f38380cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:04.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.681+0000 7f383f8e7700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38380cb0f0 con 0x7f383815a750 2023-10-11T09:42:04.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.681+0000 7f383f8e7700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f383815bb30 con 0x7f38380ca760 2023-10-11T09:42:04.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.681+0000 7f383f8e7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f383815e7b0 con 0x7f383815e210 2023-10-11T09:42:04.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.685+0000 7f383d683700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f383815a750 0x7f38380cc0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:35968/0 (socket says 172.21.15.78:35968) 2023-10-11T09:42:04.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.685+0000 7f383d683700 1 -- 172.21.15.78:0/505740452 learned_addr learned my addr 172.21.15.78:0/505740452 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:42:04.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.685+0000 7f383de84700 1 --2- 172.21.15.78:0/505740452 >> v2:172.21.15.133:3300/0 conn(0x7f38380ca760 0x7f38380cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:04.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.685+0000 7f383ce82700 1 --2- 172.21.15.78:0/505740452 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f383815e210 0x7f3838071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:04.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.685+0000 7f382ffff700 1 -- 172.21.15.78:0/505740452 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1877772932 0 0) 0x7f3834001110 con 0x7f383815a750 2023-10-11T09:42:04.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.685+0000 7f382ffff700 1 -- 172.21.15.78:0/505740452 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38240035d0 con 0x7f383815a750 2023-10-11T09:42:04.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.685+0000 7f383de84700 1 -- 172.21.15.78:0/505740452 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f383815e210 msgr2=0x7f3838071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:04.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.685+0000 7f382ffff700 1 -- 172.21.15.78:0/505740452 <== mon.1 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2327204974 0 0) 0x7f38240035d0 con 0x7f383815a750 2023-10-11T09:42:04.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.685+0000 7f383de84700 1 --2- 172.21.15.78:0/505740452 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f383815e210 0x7f3838071040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:04.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.685+0000 7f383de84700 1 -- 172.21.15.78:0/505740452 >> v1:172.21.15.133:6789/0 conn(0x7f383815a750 legacy=0x7f38380cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:04.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.685+0000 7f383de84700 1 -- 172.21.15.78:0/505740452 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f383815ea90 con 0x7f38380ca760 2023-10-11T09:42:04.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.685+0000 7f383de84700 1 --2- 172.21.15.78:0/505740452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38380ca760 0x7f38380cabb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3830009d90 tx=0x7f383000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a89e3cac36cc47de server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:04.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.685+0000 7f382ffff700 1 -- 172.21.15.78:0/505740452 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383000e070 con 0x7f38380ca760 2023-10-11T09:42:04.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.685+0000 7f382ffff700 1 -- 172.21.15.78:0/505740452 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3830003a40 con 0x7f38380ca760 2023-10-11T09:42:04.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.685+0000 7f382ffff700 1 -- 172.21.15.78:0/505740452 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3834001800 con 0x7f38380ca760 2023-10-11T09:42:04.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.689+0000 7f383f8e7700 1 -- 172.21.15.78:0/505740452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38380ca760 msgr2=0x7f38380cabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:04.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.689+0000 7f383f8e7700 1 --2- 172.21.15.78:0/505740452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38380ca760 0x7f38380cabb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3830009d90 tx=0x7f383000b330 comp rx=0 tx=0).stop 2023-10-11T09:42:04.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.689+0000 7f383f8e7700 1 -- 172.21.15.78:0/505740452 shutdown_connections 2023-10-11T09:42:04.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.689+0000 7f383f8e7700 1 --2- 172.21.15.78:0/505740452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38380ca760 0x7f38380cabb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:04.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.689+0000 7f383f8e7700 1 --2- 172.21.15.78:0/505740452 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f383815e210 0x7f3838071040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:04.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.689+0000 7f383f8e7700 1 -- 172.21.15.78:0/505740452 >> 172.21.15.78:0/505740452 conn(0x7f38380b0990 msgr2=0x7f38380b0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:04.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.689+0000 7f383f8e7700 1 -- 172.21.15.78:0/505740452 shutdown_connections 2023-10-11T09:42:04.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.689+0000 7f383f8e7700 1 -- 172.21.15.78:0/505740452 wait complete. 2023-10-11T09:42:04.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.689+0000 7f383f8e7700 1 Processor -- start 2023-10-11T09:42:04.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.689+0000 7f383f8e7700 1 -- start start 2023-10-11T09:42:04.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.693+0000 7f383f8e7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38380ca760 0x7f38381f2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:04.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.693+0000 7f383f8e7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f383815a750 0x7f38380ce090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:04.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.693+0000 7f383f8e7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f383815e210 0x7f383806c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:04.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.693+0000 7f383f8e7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38380ce720 con 0x7f38380ca760 2023-10-11T09:42:04.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.693+0000 7f383f8e7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38381f2260 con 0x7f383815e210 2023-10-11T09:42:04.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.693+0000 7f383f8e7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38381f2540 con 0x7f383815a750 2023-10-11T09:42:04.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.693+0000 7f383d683700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38380ca760 0x7f38381f2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:04.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.693+0000 7f383d683700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38380ca760 0x7f38381f2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60594/0 (socket says 172.21.15.78:60594) 2023-10-11T09:42:04.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.693+0000 7f383d683700 1 -- 172.21.15.78:0/2963686011 learned_addr learned my addr 172.21.15.78:0/2963686011 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:04.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.693+0000 7f383ce82700 1 --2- 172.21.15.78:0/2963686011 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f383815a750 0x7f38380ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:04.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.693+0000 7f383de84700 1 --2- 172.21.15.78:0/2963686011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f383815e210 0x7f383806c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:04.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.693+0000 7f383d683700 1 -- 172.21.15.78:0/2963686011 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f383815a750 msgr2=0x7f38380ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:04.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.693+0000 7f383d683700 1 --2- 172.21.15.78:0/2963686011 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f383815a750 0x7f38380ce090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:04.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.693+0000 7f383d683700 1 -- 172.21.15.78:0/2963686011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f383815e210 msgr2=0x7f383806c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:04.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.693+0000 7f383d683700 1 --2- 172.21.15.78:0/2963686011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f383815e210 0x7f383806c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:04.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.693+0000 7f383d683700 1 -- 172.21.15.78:0/2963686011 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3830009a70 con 0x7f38380ca760 2023-10-11T09:42:04.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.693+0000 7f383de84700 1 --2- 172.21.15.78:0/2963686011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f383815e210 0x7f383806c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:42:04.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.693+0000 7f383d683700 1 --2- 172.21.15.78:0/2963686011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38380ca760 0x7f38381f2d40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f383400c500 tx=0x7f383400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:04.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.693+0000 7f382dffb700 1 -- 172.21.15.78:0/2963686011 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3834004ca0 con 0x7f38380ca760 2023-10-11T09:42:04.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.693+0000 7f383f8e7700 1 -- 172.21.15.78:0/2963686011 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38381f2820 con 0x7f38380ca760 2023-10-11T09:42:04.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.693+0000 7f382dffb700 1 -- 172.21.15.78:0/2963686011 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3834004e20 con 0x7f38380ca760 2023-10-11T09:42:04.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.693+0000 7f382dffb700 1 -- 172.21.15.78:0/2963686011 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3834007310 con 0x7f38380ca760 2023-10-11T09:42:04.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.693+0000 7f383f8e7700 1 -- 172.21.15.78:0/2963686011 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38381f1ec0 con 0x7f38380ca760 2023-10-11T09:42:04.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.697+0000 7f383f8e7700 1 -- 172.21.15.78:0/2963686011 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38381f2a40 con 0x7f38380ca760 2023-10-11T09:42:04.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.697+0000 7f382dffb700 1 -- 172.21.15.78:0/2963686011 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f3834007470 con 0x7f38380ca760 2023-10-11T09:42:04.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.701+0000 7f382dffb700 1 --2- 172.21.15.78:0/2963686011 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f381c065ed0 0x7f381c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:04.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.701+0000 7f382dffb700 1 -- 172.21.15.78:0/2963686011 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f38340834e0 con 0x7f38380ca760 2023-10-11T09:42:04.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.701+0000 7f383ce82700 1 --2- 172.21.15.78:0/2963686011 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f381c065ed0 0x7f381c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:04.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.701+0000 7f383ce82700 1 --2- 172.21.15.78:0/2963686011 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f381c065ed0 0x7f381c068380 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f38380cf3d0 tx=0x7f3828008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:04.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.705+0000 7f382dffb700 1 -- 172.21.15.78:0/2963686011 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f3834050a10 con 0x7f38380ca760 2023-10-11T09:42:04.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.937+0000 7f383f8e7700 1 -- 172.21.15.78:0/2963686011 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f38380cb0f0 con 0x7f38380ca760 2023-10-11T09:42:04.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.937+0000 7f382dffb700 1 -- 172.21.15.78:0/2963686011 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1270 (secure 0 0 0) 0x7f38380cb0f0 con 0x7f38380ca760 2023-10-11T09:42:04.945 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:04.945 INFO:teuthology.orchestra.run.smithi078.stdout:{"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":110,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":37,"num_osds":8,"num_up_osds":8,"osd_up_since":1697017227,"num_in_osds":8,"osd_in_since":1697017218,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":9}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":225046528,"bytes_avail":772869066752,"bytes_total":773094113280,"unknown_pgs_ratio":0.60952383279800415,"inactive_pgs_ratio":0.30476191639900208},"fsmap":{"epoch":16,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"b","status":"up:active","gid":5364}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2023-10-11T09:41:38.339642+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-11T09:42:04.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.941+0000 7f383f8e7700 1 -- 172.21.15.78:0/2963686011 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f381c065ed0 msgr2=0x7f381c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:04.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.941+0000 7f383f8e7700 1 --2- 172.21.15.78:0/2963686011 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f381c065ed0 0x7f381c068380 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f38380cf3d0 tx=0x7f3828008040 comp rx=0 tx=0).stop 2023-10-11T09:42:04.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.941+0000 7f383f8e7700 1 -- 172.21.15.78:0/2963686011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38380ca760 msgr2=0x7f38381f2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:04.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.941+0000 7f383f8e7700 1 --2- 172.21.15.78:0/2963686011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38380ca760 0x7f38381f2d40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f383400c500 tx=0x7f383400c8c0 comp rx=0 tx=0).stop 2023-10-11T09:42:04.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.941+0000 7f383f8e7700 1 -- 172.21.15.78:0/2963686011 shutdown_connections 2023-10-11T09:42:04.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.941+0000 7f383f8e7700 1 --2- 172.21.15.78:0/2963686011 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f381c065ed0 0x7f381c068380 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:04.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.941+0000 7f383f8e7700 1 --2- 172.21.15.78:0/2963686011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f383815e210 0x7f383806c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:04.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.941+0000 7f383f8e7700 1 --2- 172.21.15.78:0/2963686011 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f383815a750 0x7f38380ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:04.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.941+0000 7f383f8e7700 1 --2- 172.21.15.78:0/2963686011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38380ca760 0x7f38381f2d40 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:04.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.941+0000 7f383f8e7700 1 -- 172.21.15.78:0/2963686011 >> 172.21.15.78:0/2963686011 conn(0x7f38380b0990 msgr2=0x7f38380c96b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:04.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.941+0000 7f383f8e7700 1 -- 172.21.15.78:0/2963686011 shutdown_connections 2023-10-11T09:42:04.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:04.941+0000 7f383f8e7700 1 -- 172.21.15.78:0/2963686011 wait complete. 2023-10-11T09:42:04.970 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-11T09:42:04.971 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-11T09:42:05.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.129+0000 7ff709076700 1 Processor -- start 2023-10-11T09:42:05.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.129+0000 7ff709076700 1 -- start start 2023-10-11T09:42:05.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.133+0000 7ff709076700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff70415b8e0 0x7ff70406d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:05.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.133+0000 7ff709076700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff70406d610 0x7ff7040ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:05.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.133+0000 7ff709076700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff70415e010 con 0x7ff70415a940 2023-10-11T09:42:05.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.133+0000 7ff709076700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff70415e280 con 0x7ff70406d610 2023-10-11T09:42:05.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.133+0000 7ff709076700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff70415e560 con 0x7ff70415b8e0 2023-10-11T09:42:05.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.133+0000 7ff6fb7fe700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff70406d610 0x7ff7040ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:05.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.133+0000 7ff703fff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff70415b8e0 0x7ff70406d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:05.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.133+0000 7ff708875700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff70415a940 0x7ff7040c9020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:35984/0 (socket says 172.21.15.78:35984) 2023-10-11T09:42:05.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.133+0000 7ff708875700 1 -- 172.21.15.78:0/3787645108 learned_addr learned my addr 172.21.15.78:0/3787645108 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:42:05.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.133+0000 7ff703fff700 1 -- 172.21.15.78:0/3787645108 >> v2:172.21.15.133:3300/0 conn(0x7ff70406d610 msgr2=0x7ff7040ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:05.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.133+0000 7ff703fff700 1 --2- 172.21.15.78:0/3787645108 >> v2:172.21.15.133:3300/0 conn(0x7ff70406d610 0x7ff7040ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:05.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.133+0000 7ff703fff700 1 -- 172.21.15.78:0/3787645108 >> v1:172.21.15.133:6789/0 conn(0x7ff70415a940 legacy=0x7ff7040c9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:05.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.133+0000 7ff703fff700 1 -- 172.21.15.78:0/3787645108 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff70415e840 con 0x7ff70415b8e0 2023-10-11T09:42:05.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.133+0000 7ff7037fe700 1 -- 172.21.15.78:0/3787645108 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3392153963 0 0) 0x7ff70415e010 con 0x7ff70415a940 2023-10-11T09:42:05.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.133+0000 7ff703fff700 1 --2- 172.21.15.78:0/3787645108 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff70415b8e0 0x7ff70406d0d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4009850 tx=0x7ff6f4009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84c19146ed4da73e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:05.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.133+0000 7ff7037fe700 1 -- 172.21.15.78:0/3787645108 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4011830 con 0x7ff70415b8e0 2023-10-11T09:42:05.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.133+0000 7ff7037fe700 1 -- 172.21.15.78:0/3787645108 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f4011990 con 0x7ff70415b8e0 2023-10-11T09:42:05.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.133+0000 7ff7037fe700 1 -- 172.21.15.78:0/3787645108 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4011d40 con 0x7ff70415b8e0 2023-10-11T09:42:05.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.137+0000 7ff709076700 1 -- 172.21.15.78:0/3787645108 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff70415b8e0 msgr2=0x7ff70406d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:05.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.137+0000 7ff709076700 1 --2- 172.21.15.78:0/3787645108 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff70415b8e0 0x7ff70406d0d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4009850 tx=0x7ff6f4009b60 comp rx=0 tx=0).stop 2023-10-11T09:42:05.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.137+0000 7ff709076700 1 -- 172.21.15.78:0/3787645108 shutdown_connections 2023-10-11T09:42:05.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.137+0000 7ff709076700 1 --2- 172.21.15.78:0/3787645108 >> v2:172.21.15.133:3300/0 conn(0x7ff70406d610 0x7ff7040ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:05.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.137+0000 7ff709076700 1 --2- 172.21.15.78:0/3787645108 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff70415b8e0 0x7ff70406d0d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:05.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.137+0000 7ff709076700 1 -- 172.21.15.78:0/3787645108 >> 172.21.15.78:0/3787645108 conn(0x7ff7040cea60 msgr2=0x7ff7040cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:05.145 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.137+0000 7ff709076700 1 -- 172.21.15.78:0/3787645108 shutdown_connections 2023-10-11T09:42:05.145 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.137+0000 7ff709076700 1 -- 172.21.15.78:0/3787645108 wait complete. 2023-10-11T09:42:05.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.137+0000 7ff709076700 1 Processor -- start 2023-10-11T09:42:05.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.137+0000 7ff709076700 1 -- start start 2023-10-11T09:42:05.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.141+0000 7ff709076700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff70406d610 0x7ff704159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:05.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.141+0000 7ff709076700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff70415a940 0x7ff704142640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:05.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.141+0000 7ff709076700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff70415b8e0 0x7ff7040cb340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:05.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.141+0000 7ff709076700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7040b1a10 con 0x7ff70415b8e0 2023-10-11T09:42:05.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.141+0000 7ff709076700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7040b1cf0 con 0x7ff70406d610 2023-10-11T09:42:05.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.141+0000 7ff709076700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7040b1fd0 con 0x7ff70415a940 2023-10-11T09:42:05.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.141+0000 7ff708875700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff70415b8e0 0x7ff7040cb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:05.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.141+0000 7ff703fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff70406d610 0x7ff704159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:05.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.141+0000 7ff6fb7fe700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff70415a940 0x7ff704142640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:05.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.141+0000 7ff708875700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff70415b8e0 0x7ff7040cb340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60604/0 (socket says 172.21.15.78:60604) 2023-10-11T09:42:05.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.141+0000 7ff708875700 1 -- 172.21.15.78:0/217102596 learned_addr learned my addr 172.21.15.78:0/217102596 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:05.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.141+0000 7ff708875700 1 -- 172.21.15.78:0/217102596 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff70415a940 msgr2=0x7ff704142640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:05.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.141+0000 7ff708875700 1 --2- 172.21.15.78:0/217102596 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff70415a940 0x7ff704142640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:05.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.141+0000 7ff708875700 1 -- 172.21.15.78:0/217102596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff70406d610 msgr2=0x7ff704159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:05.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.141+0000 7ff708875700 1 --2- 172.21.15.78:0/217102596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff70406d610 0x7ff704159d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:05.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.141+0000 7ff708875700 1 -- 172.21.15.78:0/217102596 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f40094b0 con 0x7ff70415b8e0 2023-10-11T09:42:05.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.141+0000 7ff703fff700 1 --2- 172.21.15.78:0/217102596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff70406d610 0x7ff704159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:42:05.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.141+0000 7ff708875700 1 --2- 172.21.15.78:0/217102596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff70415b8e0 0x7ff7040cb340 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc002230 tx=0x7ff6fc00cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:05.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.141+0000 7ff7017fa700 1 -- 172.21.15.78:0/217102596 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc00aac0 con 0x7ff70415b8e0 2023-10-11T09:42:05.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.141+0000 7ff7017fa700 1 -- 172.21.15.78:0/217102596 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6fc00ac40 con 0x7ff70415b8e0 2023-10-11T09:42:05.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.141+0000 7ff7017fa700 1 -- 172.21.15.78:0/217102596 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc0070d0 con 0x7ff70415b8e0 2023-10-11T09:42:05.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.141+0000 7ff709076700 1 -- 172.21.15.78:0/217102596 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7040b22b0 con 0x7ff70415b8e0 2023-10-11T09:42:05.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.141+0000 7ff709076700 1 -- 172.21.15.78:0/217102596 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff70413b1b0 con 0x7ff70415b8e0 2023-10-11T09:42:05.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.145+0000 7ff709076700 1 -- 172.21.15.78:0/217102596 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff704065d40 con 0x7ff70415b8e0 2023-10-11T09:42:05.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.145+0000 7ff7017fa700 1 -- 172.21.15.78:0/217102596 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7ff6fc007250 con 0x7ff70415b8e0 2023-10-11T09:42:05.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.149+0000 7ff7017fa700 1 --2- 172.21.15.78:0/217102596 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff6e8065f00 0x7ff6e80683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:05.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.149+0000 7ff7017fa700 1 -- 172.21.15.78:0/217102596 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff6fc0831d0 con 0x7ff70415b8e0 2023-10-11T09:42:05.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.149+0000 7ff703fff700 1 --2- 172.21.15.78:0/217102596 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff6e8065f00 0x7ff6e80683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:05.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.149+0000 7ff703fff700 1 --2- 172.21.15.78:0/217102596 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff6e8065f00 0x7ff6e80683b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff6f40090b0 tx=0x7ff6f401c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:05.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.153+0000 7ff7017fa700 1 -- 172.21.15.78:0/217102596 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7ff6fc050810 con 0x7ff70415b8e0 2023-10-11T09:42:05.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.321+0000 7ff709076700 1 -- 172.21.15.78:0/217102596 --> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff70415eb20 con 0x7ff6e8065f00 2023-10-11T09:42:05.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.333+0000 7ff7017fa700 1 -- 172.21.15.78:0/217102596 <== mgr.4101 v2:172.21.15.78:6832/35606 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308742 (secure 0 0 0) 0x7ff6f4011c00 con 0x7ff6e8065f00 2023-10-11T09:42:05.341 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:05.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.341+0000 7ff709076700 1 -- 172.21.15.78:0/217102596 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff6e8065f00 msgr2=0x7ff6e80683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:05.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.341+0000 7ff709076700 1 --2- 172.21.15.78:0/217102596 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff6e8065f00 0x7ff6e80683b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff6f40090b0 tx=0x7ff6f401c040 comp rx=0 tx=0).stop 2023-10-11T09:42:05.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.341+0000 7ff709076700 1 -- 172.21.15.78:0/217102596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff70415b8e0 msgr2=0x7ff7040cb340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:05.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.341+0000 7ff709076700 1 --2- 172.21.15.78:0/217102596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff70415b8e0 0x7ff7040cb340 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc002230 tx=0x7ff6fc00cf50 comp rx=0 tx=0).stop 2023-10-11T09:42:05.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.341+0000 7ff709076700 1 -- 172.21.15.78:0/217102596 shutdown_connections 2023-10-11T09:42:05.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.341+0000 7ff709076700 1 --2- 172.21.15.78:0/217102596 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff6e8065f00 0x7ff6e80683b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:05.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.341+0000 7ff709076700 1 --2- 172.21.15.78:0/217102596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff70415b8e0 0x7ff7040cb340 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:05.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.341+0000 7ff709076700 1 --2- 172.21.15.78:0/217102596 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff70415a940 0x7ff704142640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:05.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.341+0000 7ff709076700 1 --2- 172.21.15.78:0/217102596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff70406d610 0x7ff704159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:05.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.341+0000 7ff709076700 1 -- 172.21.15.78:0/217102596 >> 172.21.15.78:0/217102596 conn(0x7ff7040cea60 msgr2=0x7ff7040cee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:05.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.341+0000 7ff709076700 1 -- 172.21.15.78:0/217102596 shutdown_connections 2023-10-11T09:42:05.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:05.341+0000 7ff709076700 1 -- 172.21.15.78:0/217102596 wait complete. 2023-10-11T09:42:05.349 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2023-10-11T09:42:05.368 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2023-10-11T09:42:04.347112+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":184,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220136,"kb_used_data":4520,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754584,"statfs":{"total":773094113280,"available":772868694016,"internally_reserved":0,"allocated":4628480,"data_stored":2588564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":54,"apply_latency_ms":54,"commit_latency_ns":54000000,"apply_latency_ns":54000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001529"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.134185+0000","last_change":"2023-10-11T09:41:59.251004+0000","last_active":"2023-10-11T09:42:02.134185+0000","last_peered":"2023-10-11T09:42:02.134185+0000","last_clean":"2023-10-11T09:42:02.134185+0000","last_became_active":"2023-10-11T09:41:59.250745+0000","last_became_peered":"2023-10-11T09:41:59.250745+0000","last_unstale":"2023-10-11T09:42:02.134185+0000","last_undegraded":"2023-10-11T09:42:02.134185+0000","last_fullsized":"2023-10-11T09:42:02.134185+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:32:44.448136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.135281+0000","last_change":"2023-10-11T09:41:59.258617+0000","last_active":"2023-10-11T09:42:02.135281+0000","last_peered":"2023-10-11T09:42:02.135281+0000","last_clean":"2023-10-11T09:42:02.135281+0000","last_became_active":"2023-10-11T09:41:59.258374+0000","last_became_peered":"2023-10-11T09:41:59.258374+0000","last_unstale":"2023-10-11T09:42:02.135281+0000","last_undegraded":"2023-10-11T09:42:02.135281+0000","last_fullsized":"2023-10-11T09:42:02.135281+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:46:05.674097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133528+0000","last_change":"2023-10-11T09:41:59.270985+0000","last_active":"2023-10-11T09:42:02.133528+0000","last_peered":"2023-10-11T09:42:02.133528+0000","last_clean":"2023-10-11T09:42:02.133528+0000","last_became_active":"2023-10-11T09:41:59.270588+0000","last_became_peered":"2023-10-11T09:41:59.270588+0000","last_unstale":"2023-10-11T09:42:02.133528+0000","last_undegraded":"2023-10-11T09:42:02.133528+0000","last_fullsized":"2023-10-11T09:42:02.133528+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:51:26.096609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.244926+0000","last_change":"2023-10-11T09:41:59.254605+0000","last_active":"2023-10-11T09:42:00.244926+0000","last_peered":"2023-10-11T09:42:00.244926+0000","last_clean":"2023-10-11T09:42:00.244926+0000","last_became_active":"2023-10-11T09:41:59.254130+0000","last_became_peered":"2023-10-11T09:41:59.254130+0000","last_unstale":"2023-10-11T09:42:00.244926+0000","last_undegraded":"2023-10-11T09:42:00.244926+0000","last_fullsized":"2023-10-11T09:42:00.244926+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:09:11.561455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.244764+0000","last_change":"2023-10-11T09:41:59.251753+0000","last_active":"2023-10-11T09:42:00.244764+0000","last_peered":"2023-10-11T09:42:00.244764+0000","last_clean":"2023-10-11T09:42:00.244764+0000","last_became_active":"2023-10-11T09:41:59.251460+0000","last_became_peered":"2023-10-11T09:41:59.251460+0000","last_unstale":"2023-10-11T09:42:00.244764+0000","last_undegraded":"2023-10-11T09:42:00.244764+0000","last_fullsized":"2023-10-11T09:42:00.244764+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:16:24.678764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.134098+0000","last_change":"2023-10-11T09:41:59.250567+0000","last_active":"2023-10-11T09:42:02.134098+0000","last_peered":"2023-10-11T09:42:02.134098+0000","last_clean":"2023-10-11T09:42:02.134098+0000","last_became_active":"2023-10-11T09:41:59.250202+0000","last_became_peered":"2023-10-11T09:41:59.250202+0000","last_unstale":"2023-10-11T09:42:02.134098+0000","last_undegraded":"2023-10-11T09:42:02.134098+0000","last_fullsized":"2023-10-11T09:42:02.134098+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:52:21.727787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.245752+0000","last_change":"2023-10-11T09:41:59.261239+0000","last_active":"2023-10-11T09:42:00.245752+0000","last_peered":"2023-10-11T09:42:00.245752+0000","last_clean":"2023-10-11T09:42:00.245752+0000","last_became_active":"2023-10-11T09:41:59.260991+0000","last_became_peered":"2023-10-11T09:41:59.260991+0000","last_unstale":"2023-10-11T09:42:00.245752+0000","last_undegraded":"2023-10-11T09:42:00.245752+0000","last_fullsized":"2023-10-11T09:42:00.245752+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:14:50.524508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.132362+0000","last_change":"2023-10-11T09:41:59.267670+0000","last_active":"2023-10-11T09:42:02.132362+0000","last_peered":"2023-10-11T09:42:02.132362+0000","last_clean":"2023-10-11T09:42:02.132362+0000","last_became_active":"2023-10-11T09:41:59.267410+0000","last_became_peered":"2023-10-11T09:41:59.267410+0000","last_unstale":"2023-10-11T09:42:02.132362+0000","last_undegraded":"2023-10-11T09:42:02.132362+0000","last_fullsized":"2023-10-11T09:42:02.132362+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:14:53.871518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133935+0000","last_change":"2023-10-11T09:41:59.271233+0000","last_active":"2023-10-11T09:42:02.133935+0000","last_peered":"2023-10-11T09:42:02.133935+0000","last_clean":"2023-10-11T09:42:02.133935+0000","last_became_active":"2023-10-11T09:41:59.270942+0000","last_became_peered":"2023-10-11T09:41:59.270942+0000","last_unstale":"2023-10-11T09:42:02.133935+0000","last_undegraded":"2023-10-11T09:42:02.133935+0000","last_fullsized":"2023-10-11T09:42:02.133935+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:46:05.674097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.247294+0000","last_change":"2023-10-11T09:41:59.276669+0000","last_active":"2023-10-11T09:42:00.247294+0000","last_peered":"2023-10-11T09:42:00.247294+0000","last_clean":"2023-10-11T09:42:00.247294+0000","last_became_active":"2023-10-11T09:41:59.276458+0000","last_became_peered":"2023-10-11T09:41:59.276458+0000","last_unstale":"2023-10-11T09:42:00.247294+0000","last_undegraded":"2023-10-11T09:42:00.247294+0000","last_fullsized":"2023-10-11T09:42:00.247294+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:29:15.375697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.247187+0000","last_change":"2023-10-11T09:41:59.270643+0000","last_active":"2023-10-11T09:42:00.247187+0000","last_peered":"2023-10-11T09:42:00.247187+0000","last_clean":"2023-10-11T09:42:00.247187+0000","last_became_active":"2023-10-11T09:41:59.270327+0000","last_became_peered":"2023-10-11T09:41:59.270327+0000","last_unstale":"2023-10-11T09:42:00.247187+0000","last_undegraded":"2023-10-11T09:42:00.247187+0000","last_fullsized":"2023-10-11T09:42:00.247187+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:46:05.674097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.134764+0000","last_change":"2023-10-11T09:41:59.261251+0000","last_active":"2023-10-11T09:42:02.134764+0000","last_peered":"2023-10-11T09:42:02.134764+0000","last_clean":"2023-10-11T09:42:02.134764+0000","last_became_active":"2023-10-11T09:41:59.260849+0000","last_became_peered":"2023-10-11T09:41:59.260849+0000","last_unstale":"2023-10-11T09:42:02.134764+0000","last_undegraded":"2023-10-11T09:42:02.134764+0000","last_fullsized":"2023-10-11T09:42:02.134764+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:29:09.854758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.246833+0000","last_change":"2023-10-11T09:41:59.276141+0000","last_active":"2023-10-11T09:42:00.246833+0000","last_peered":"2023-10-11T09:42:00.246833+0000","last_clean":"2023-10-11T09:42:00.246833+0000","last_became_active":"2023-10-11T09:41:59.275873+0000","last_became_peered":"2023-10-11T09:41:59.275873+0000","last_unstale":"2023-10-11T09:42:00.246833+0000","last_undegraded":"2023-10-11T09:42:00.246833+0000","last_fullsized":"2023-10-11T09:42:00.246833+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:06:13.791884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:01.632123+0000","last_change":"2023-10-11T09:41:59.258250+0000","last_active":"2023-10-11T09:42:01.632123+0000","last_peered":"2023-10-11T09:42:01.632123+0000","last_clean":"2023-10-11T09:42:01.632123+0000","last_became_active":"2023-10-11T09:41:59.257976+0000","last_became_peered":"2023-10-11T09:41:59.257976+0000","last_unstale":"2023-10-11T09:42:01.632123+0000","last_undegraded":"2023-10-11T09:42:01.632123+0000","last_fullsized":"2023-10-11T09:42:01.632123+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:46:05.674097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.245867+0000","last_change":"2023-10-11T09:41:59.268687+0000","last_active":"2023-10-11T09:42:00.245867+0000","last_peered":"2023-10-11T09:42:00.245867+0000","last_clean":"2023-10-11T09:42:00.245867+0000","last_became_active":"2023-10-11T09:41:59.268386+0000","last_became_peered":"2023-10-11T09:41:59.268386+0000","last_unstale":"2023-10-11T09:42:00.245867+0000","last_undegraded":"2023-10-11T09:42:00.245867+0000","last_fullsized":"2023-10-11T09:42:00.245867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:29:09.854758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.245820+0000","last_change":"2023-10-11T09:41:59.276367+0000","last_active":"2023-10-11T09:42:00.245820+0000","last_peered":"2023-10-11T09:42:00.245820+0000","last_clean":"2023-10-11T09:42:00.245820+0000","last_became_active":"2023-10-11T09:41:59.275721+0000","last_became_peered":"2023-10-11T09:41:59.275721+0000","last_unstale":"2023-10-11T09:42:00.245820+0000","last_undegraded":"2023-10-11T09:42:00.245820+0000","last_fullsized":"2023-10-11T09:42:00.245820+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:32:44.448136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.246578+0000","last_change":"2023-10-11T09:41:59.272244+0000","last_active":"2023-10-11T09:42:00.246578+0000","last_peered":"2023-10-11T09:42:00.246578+0000","last_clean":"2023-10-11T09:42:00.246578+0000","last_became_active":"2023-10-11T09:41:59.271864+0000","last_became_peered":"2023-10-11T09:41:59.271864+0000","last_unstale":"2023-10-11T09:42:00.246578+0000","last_undegraded":"2023-10-11T09:42:00.246578+0000","last_fullsized":"2023-10-11T09:42:00.246578+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:51:26.096609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133551+0000","last_change":"2023-10-11T09:41:59.257454+0000","last_active":"2023-10-11T09:42:02.133551+0000","last_peered":"2023-10-11T09:42:02.133551+0000","last_clean":"2023-10-11T09:42:02.133551+0000","last_became_active":"2023-10-11T09:41:59.257169+0000","last_became_peered":"2023-10-11T09:41:59.257169+0000","last_unstale":"2023-10-11T09:42:02.133551+0000","last_undegraded":"2023-10-11T09:42:02.133551+0000","last_fullsized":"2023-10-11T09:42:02.133551+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:50:35.471027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.245210+0000","last_change":"2023-10-11T09:41:59.247417+0000","last_active":"2023-10-11T09:42:00.245210+0000","last_peered":"2023-10-11T09:42:00.245210+0000","last_clean":"2023-10-11T09:42:00.245210+0000","last_became_active":"2023-10-11T09:41:59.247155+0000","last_became_peered":"2023-10-11T09:41:59.247155+0000","last_unstale":"2023-10-11T09:42:00.245210+0000","last_undegraded":"2023-10-11T09:42:00.245210+0000","last_fullsized":"2023-10-11T09:42:00.245210+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:18:57.154610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133410+0000","last_change":"2023-10-11T09:41:59.258025+0000","last_active":"2023-10-11T09:42:02.133410+0000","last_peered":"2023-10-11T09:42:02.133410+0000","last_clean":"2023-10-11T09:42:02.133410+0000","last_became_active":"2023-10-11T09:41:59.257530+0000","last_became_peered":"2023-10-11T09:41:59.257530+0000","last_unstale":"2023-10-11T09:42:02.133410+0000","last_undegraded":"2023-10-11T09:42:02.133410+0000","last_fullsized":"2023-10-11T09:42:02.133410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:09:11.561455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.246676+0000","last_change":"2023-10-11T09:41:59.271200+0000","last_active":"2023-10-11T09:42:00.246676+0000","last_peered":"2023-10-11T09:42:00.246676+0000","last_clean":"2023-10-11T09:42:00.246676+0000","last_became_active":"2023-10-11T09:41:59.270934+0000","last_became_peered":"2023-10-11T09:41:59.270934+0000","last_unstale":"2023-10-11T09:42:00.246676+0000","last_undegraded":"2023-10-11T09:42:00.246676+0000","last_fullsized":"2023-10-11T09:42:00.246676+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:03:35.795099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:01.632676+0000","last_change":"2023-10-11T09:41:59.253265+0000","last_active":"2023-10-11T09:42:01.632676+0000","last_peered":"2023-10-11T09:42:01.632676+0000","last_clean":"2023-10-11T09:42:01.632676+0000","last_became_active":"2023-10-11T09:41:59.252970+0000","last_became_peered":"2023-10-11T09:41:59.252970+0000","last_unstale":"2023-10-11T09:42:01.632676+0000","last_undegraded":"2023-10-11T09:42:01.632676+0000","last_fullsized":"2023-10-11T09:42:01.632676+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:06:13.791884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.246079+0000","last_change":"2023-10-11T09:41:59.275092+0000","last_active":"2023-10-11T09:42:00.246079+0000","last_peered":"2023-10-11T09:42:00.246079+0000","last_clean":"2023-10-11T09:42:00.246079+0000","last_became_active":"2023-10-11T09:41:59.274296+0000","last_became_peered":"2023-10-11T09:41:59.274296+0000","last_unstale":"2023-10-11T09:42:00.246079+0000","last_undegraded":"2023-10-11T09:42:00.246079+0000","last_fullsized":"2023-10-11T09:42:00.246079+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:07:46.727754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.244267+0000","last_change":"2023-10-11T09:41:59.266457+0000","last_active":"2023-10-11T09:42:00.244267+0000","last_peered":"2023-10-11T09:42:00.244267+0000","last_clean":"2023-10-11T09:42:00.244267+0000","last_became_active":"2023-10-11T09:41:59.266177+0000","last_became_peered":"2023-10-11T09:41:59.266177+0000","last_unstale":"2023-10-11T09:42:00.244267+0000","last_undegraded":"2023-10-11T09:42:00.244267+0000","last_fullsized":"2023-10-11T09:42:00.244267+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:24:35.472967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133671+0000","last_change":"2023-10-11T09:41:59.256922+0000","last_active":"2023-10-11T09:42:02.133671+0000","last_peered":"2023-10-11T09:42:02.133671+0000","last_clean":"2023-10-11T09:42:02.133671+0000","last_became_active":"2023-10-11T09:41:59.256620+0000","last_became_peered":"2023-10-11T09:41:59.256620+0000","last_unstale":"2023-10-11T09:42:02.133671+0000","last_undegraded":"2023-10-11T09:42:02.133671+0000","last_fullsized":"2023-10-11T09:42:02.133671+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:48:03.493737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.134430+0000","last_change":"2023-10-11T09:41:59.267830+0000","last_active":"2023-10-11T09:42:02.134430+0000","last_peered":"2023-10-11T09:42:02.134430+0000","last_clean":"2023-10-11T09:42:02.134430+0000","last_became_active":"2023-10-11T09:41:59.267543+0000","last_became_peered":"2023-10-11T09:41:59.267543+0000","last_unstale":"2023-10-11T09:42:02.134430+0000","last_undegraded":"2023-10-11T09:42:02.134430+0000","last_fullsized":"2023-10-11T09:42:02.134430+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:00:42.620537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133519+0000","last_change":"2023-10-11T09:41:59.259928+0000","last_active":"2023-10-11T09:42:02.133519+0000","last_peered":"2023-10-11T09:42:02.133519+0000","last_clean":"2023-10-11T09:42:02.133519+0000","last_became_active":"2023-10-11T09:41:59.259642+0000","last_became_peered":"2023-10-11T09:41:59.259642+0000","last_unstale":"2023-10-11T09:42:02.133519+0000","last_undegraded":"2023-10-11T09:42:02.133519+0000","last_fullsized":"2023-10-11T09:42:02.133519+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:05:44.896062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.247105+0000","last_change":"2023-10-11T09:41:59.262115+0000","last_active":"2023-10-11T09:42:00.247105+0000","last_peered":"2023-10-11T09:42:00.247105+0000","last_clean":"2023-10-11T09:42:00.247105+0000","last_became_active":"2023-10-11T09:41:59.261752+0000","last_became_peered":"2023-10-11T09:41:59.261752+0000","last_unstale":"2023-10-11T09:42:00.247105+0000","last_undegraded":"2023-10-11T09:42:00.247105+0000","last_fullsized":"2023-10-11T09:42:00.247105+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:55:01.198068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133191+0000","last_change":"2023-10-11T09:41:59.261554+0000","last_active":"2023-10-11T09:42:02.133191+0000","last_peered":"2023-10-11T09:42:02.133191+0000","last_clean":"2023-10-11T09:42:02.133191+0000","last_became_active":"2023-10-11T09:41:59.261064+0000","last_became_peered":"2023-10-11T09:41:59.261064+0000","last_unstale":"2023-10-11T09:42:02.133191+0000","last_undegraded":"2023-10-11T09:42:02.133191+0000","last_fullsized":"2023-10-11T09:42:02.133191+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:24:00.485790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133455+0000","last_change":"2023-10-11T09:41:59.256304+0000","last_active":"2023-10-11T09:42:02.133455+0000","last_peered":"2023-10-11T09:42:02.133455+0000","last_clean":"2023-10-11T09:42:02.133455+0000","last_became_active":"2023-10-11T09:41:59.255941+0000","last_became_peered":"2023-10-11T09:41:59.255941+0000","last_unstale":"2023-10-11T09:42:02.133455+0000","last_undegraded":"2023-10-11T09:42:02.133455+0000","last_fullsized":"2023-10-11T09:42:02.133455+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:16:24.678764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.246243+0000","last_change":"2023-10-11T09:41:59.275647+0000","last_active":"2023-10-11T09:42:00.246243+0000","last_peered":"2023-10-11T09:42:00.246243+0000","last_clean":"2023-10-11T09:42:00.246243+0000","last_became_active":"2023-10-11T09:41:59.275328+0000","last_became_peered":"2023-10-11T09:41:59.275328+0000","last_unstale":"2023-10-11T09:42:00.246243+0000","last_undegraded":"2023-10-11T09:42:00.246243+0000","last_fullsized":"2023-10-11T09:42:00.246243+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:49:29.335655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133147+0000","last_change":"2023-10-11T09:41:59.261148+0000","last_active":"2023-10-11T09:42:02.133147+0000","last_peered":"2023-10-11T09:42:02.133147+0000","last_clean":"2023-10-11T09:42:02.133147+0000","last_became_active":"2023-10-11T09:41:59.260863+0000","last_became_peered":"2023-10-11T09:41:59.260863+0000","last_unstale":"2023-10-11T09:42:02.133147+0000","last_undegraded":"2023-10-11T09:42:02.133147+0000","last_fullsized":"2023-10-11T09:42:02.133147+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:58:22.628540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.246187+0000","last_change":"2023-10-11T09:41:59.268534+0000","last_active":"2023-10-11T09:42:00.246187+0000","last_peered":"2023-10-11T09:42:00.246187+0000","last_clean":"2023-10-11T09:42:00.246187+0000","last_became_active":"2023-10-11T09:41:59.268260+0000","last_became_peered":"2023-10-11T09:41:59.268260+0000","last_unstale":"2023-10-11T09:42:00.246187+0000","last_undegraded":"2023-10-11T09:42:00.246187+0000","last_fullsized":"2023-10-11T09:42:00.246187+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:46:05.674097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"36'4","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.278577+0000","last_change":"2023-10-11T09:41:58.781253+0000","last_active":"2023-10-11T09:41:59.278577+0000","last_peered":"2023-10-11T09:41:59.278577+0000","last_clean":"2023-10-11T09:41:59.278577+0000","last_became_active":"2023-10-11T09:41:58.780932+0000","last_became_peered":"2023-10-11T09:41:58.780932+0000","last_unstale":"2023-10-11T09:41:59.278577+0000","last_undegraded":"2023-10-11T09:41:59.278577+0000","last_fullsized":"2023-10-11T09:41:59.278577+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:34:50.579527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.135085+0000","last_change":"2023-10-11T09:41:59.270949+0000","last_active":"2023-10-11T09:42:02.135085+0000","last_peered":"2023-10-11T09:42:02.135085+0000","last_clean":"2023-10-11T09:42:02.135085+0000","last_became_active":"2023-10-11T09:41:59.270558+0000","last_became_peered":"2023-10-11T09:41:59.270558+0000","last_unstale":"2023-10-11T09:42:02.135085+0000","last_undegraded":"2023-10-11T09:42:02.135085+0000","last_fullsized":"2023-10-11T09:42:02.135085+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:07:43.684811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"36'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.278298+0000","last_change":"2023-10-11T09:41:58.773997+0000","last_active":"2023-10-11T09:41:59.278298+0000","last_peered":"2023-10-11T09:41:59.278298+0000","last_clean":"2023-10-11T09:41:59.278298+0000","last_became_active":"2023-10-11T09:41:58.773582+0000","last_became_peered":"2023-10-11T09:41:58.773582+0000","last_unstale":"2023-10-11T09:41:59.278298+0000","last_undegraded":"2023-10-11T09:41:59.278298+0000","last_fullsized":"2023-10-11T09:41:59.278298+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:00:41.605405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:01.632879+0000","last_change":"2023-10-11T09:41:59.264983+0000","last_active":"2023-10-11T09:42:01.632879+0000","last_peered":"2023-10-11T09:42:01.632879+0000","last_clean":"2023-10-11T09:42:01.632879+0000","last_became_active":"2023-10-11T09:41:59.264662+0000","last_became_peered":"2023-10-11T09:41:59.264662+0000","last_unstale":"2023-10-11T09:42:01.632879+0000","last_undegraded":"2023-10-11T09:42:01.632879+0000","last_fullsized":"2023-10-11T09:42:01.632879+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:51:26.096609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"36'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.272941+0000","last_change":"2023-10-11T09:41:58.775695+0000","last_active":"2023-10-11T09:41:59.272941+0000","last_peered":"2023-10-11T09:41:59.272941+0000","last_clean":"2023-10-11T09:41:59.272941+0000","last_became_active":"2023-10-11T09:41:58.775435+0000","last_became_peered":"2023-10-11T09:41:58.775435+0000","last_unstale":"2023-10-11T09:41:59.272941+0000","last_undegraded":"2023-10-11T09:41:59.272941+0000","last_fullsized":"2023-10-11T09:41:59.272941+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:16:23.663632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.134236+0000","last_change":"2023-10-11T09:41:59.259129+0000","last_active":"2023-10-11T09:42:02.134236+0000","last_peered":"2023-10-11T09:42:02.134236+0000","last_clean":"2023-10-11T09:42:02.134236+0000","last_became_active":"2023-10-11T09:41:59.258907+0000","last_became_peered":"2023-10-11T09:41:59.258907+0000","last_unstale":"2023-10-11T09:42:02.134236+0000","last_undegraded":"2023-10-11T09:42:02.134236+0000","last_fullsized":"2023-10-11T09:42:02.134236+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:07:46.727754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"36'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.273479+0000","last_change":"2023-10-11T09:41:58.777078+0000","last_active":"2023-10-11T09:41:59.273479+0000","last_peered":"2023-10-11T09:41:59.273479+0000","last_clean":"2023-10-11T09:41:59.273479+0000","last_became_active":"2023-10-11T09:41:58.776760+0000","last_became_peered":"2023-10-11T09:41:58.776760+0000","last_unstale":"2023-10-11T09:41:59.273479+0000","last_undegraded":"2023-10-11T09:41:59.273479+0000","last_fullsized":"2023-10-11T09:41:59.273479+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:30:07.413635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133334+0000","last_change":"2023-10-11T09:41:59.249575+0000","last_active":"2023-10-11T09:42:02.133334+0000","last_peered":"2023-10-11T09:42:02.133334+0000","last_clean":"2023-10-11T09:42:02.133334+0000","last_became_active":"2023-10-11T09:41:59.249320+0000","last_became_peered":"2023-10-11T09:41:59.249320+0000","last_unstale":"2023-10-11T09:42:02.133334+0000","last_undegraded":"2023-10-11T09:42:02.133334+0000","last_fullsized":"2023-10-11T09:42:02.133334+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:29:15.375697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"36'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.273485+0000","last_change":"2023-10-11T09:41:58.780188+0000","last_active":"2023-10-11T09:41:59.273485+0000","last_peered":"2023-10-11T09:41:59.273485+0000","last_clean":"2023-10-11T09:41:59.273485+0000","last_became_active":"2023-10-11T09:41:58.779846+0000","last_became_peered":"2023-10-11T09:41:58.779846+0000","last_unstale":"2023-10-11T09:41:59.273485+0000","last_undegraded":"2023-10-11T09:41:59.273485+0000","last_fullsized":"2023-10-11T09:41:59.273485+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:52:20.712655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.245928+0000","last_change":"2023-10-11T09:41:59.258246+0000","last_active":"2023-10-11T09:42:00.245928+0000","last_peered":"2023-10-11T09:42:00.245928+0000","last_clean":"2023-10-11T09:42:00.245928+0000","last_became_active":"2023-10-11T09:41:59.257935+0000","last_became_peered":"2023-10-11T09:41:59.257935+0000","last_unstale":"2023-10-11T09:42:00.245928+0000","last_undegraded":"2023-10-11T09:42:00.245928+0000","last_fullsized":"2023-10-11T09:42:00.245928+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:27:48.375035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"36'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.275029+0000","last_change":"2023-10-11T09:41:58.781289+0000","last_active":"2023-10-11T09:41:59.275029+0000","last_peered":"2023-10-11T09:41:59.275029+0000","last_clean":"2023-10-11T09:41:59.275029+0000","last_became_active":"2023-10-11T09:41:58.781013+0000","last_became_peered":"2023-10-11T09:41:58.781013+0000","last_unstale":"2023-10-11T09:41:59.275029+0000","last_undegraded":"2023-10-11T09:41:59.275029+0000","last_fullsized":"2023-10-11T09:41:59.275029+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:50:34.455895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:01.632527+0000","last_change":"2023-10-11T09:41:59.262808+0000","last_active":"2023-10-11T09:42:01.632527+0000","last_peered":"2023-10-11T09:42:01.632527+0000","last_clean":"2023-10-11T09:42:01.632527+0000","last_became_active":"2023-10-11T09:41:59.262515+0000","last_became_peered":"2023-10-11T09:41:59.262515+0000","last_unstale":"2023-10-11T09:42:01.632527+0000","last_undegraded":"2023-10-11T09:42:01.632527+0000","last_fullsized":"2023-10-11T09:42:01.632527+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:29:15.375697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.242698+0000","last_change":"2023-10-11T09:41:58.777202+0000","last_active":"2023-10-11T09:41:59.242698+0000","last_peered":"2023-10-11T09:41:59.242698+0000","last_clean":"2023-10-11T09:41:59.242698+0000","last_became_active":"2023-10-11T09:41:58.776863+0000","last_became_peered":"2023-10-11T09:41:58.776863+0000","last_unstale":"2023-10-11T09:41:59.242698+0000","last_undegraded":"2023-10-11T09:41:59.242698+0000","last_fullsized":"2023-10-11T09:41:59.242698+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:25:01.903933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133295+0000","last_change":"2023-10-11T09:41:59.275572+0000","last_active":"2023-10-11T09:42:02.133295+0000","last_peered":"2023-10-11T09:42:02.133295+0000","last_clean":"2023-10-11T09:42:02.133295+0000","last_became_active":"2023-10-11T09:41:59.275159+0000","last_became_peered":"2023-10-11T09:41:59.275159+0000","last_unstale":"2023-10-11T09:42:02.133295+0000","last_undegraded":"2023-10-11T09:42:02.133295+0000","last_fullsized":"2023-10-11T09:42:02.133295+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:29:09.854758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.242721+0000","last_change":"2023-10-11T09:41:58.774053+0000","last_active":"2023-10-11T09:41:59.242721+0000","last_peered":"2023-10-11T09:41:59.242721+0000","last_clean":"2023-10-11T09:41:59.242721+0000","last_became_active":"2023-10-11T09:41:58.773761+0000","last_became_peered":"2023-10-11T09:41:58.773761+0000","last_unstale":"2023-10-11T09:41:59.242721+0000","last_undegraded":"2023-10-11T09:41:59.242721+0000","last_fullsized":"2023-10-11T09:41:59.242721+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:00:41.605405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.245053+0000","last_change":"2023-10-11T09:41:59.254445+0000","last_active":"2023-10-11T09:42:00.245053+0000","last_peered":"2023-10-11T09:42:00.245053+0000","last_clean":"2023-10-11T09:42:00.245053+0000","last_became_active":"2023-10-11T09:41:59.253764+0000","last_became_peered":"2023-10-11T09:41:59.253764+0000","last_unstale":"2023-10-11T09:42:00.245053+0000","last_undegraded":"2023-10-11T09:42:00.245053+0000","last_fullsized":"2023-10-11T09:42:00.245053+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:50:35.471027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.245062+0000","last_change":"2023-10-11T09:41:58.775649+0000","last_active":"2023-10-11T09:41:59.245062+0000","last_peered":"2023-10-11T09:41:59.245062+0000","last_clean":"2023-10-11T09:41:59.245062+0000","last_became_active":"2023-10-11T09:41:58.773119+0000","last_became_peered":"2023-10-11T09:41:58.773119+0000","last_unstale":"2023-10-11T09:41:59.245062+0000","last_undegraded":"2023-10-11T09:41:59.245062+0000","last_fullsized":"2023-10-11T09:41:59.245062+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:48:02.478605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.246299+0000","last_change":"2023-10-11T09:41:59.266483+0000","last_active":"2023-10-11T09:42:00.246299+0000","last_peered":"2023-10-11T09:42:00.246299+0000","last_clean":"2023-10-11T09:42:00.246299+0000","last_became_active":"2023-10-11T09:41:59.266228+0000","last_became_peered":"2023-10-11T09:41:59.266228+0000","last_unstale":"2023-10-11T09:42:00.246299+0000","last_undegraded":"2023-10-11T09:42:00.246299+0000","last_fullsized":"2023-10-11T09:42:00.246299+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:14:53.871518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.256436+0000","last_change":"2023-10-11T09:41:58.780367+0000","last_active":"2023-10-11T09:41:59.256436+0000","last_peered":"2023-10-11T09:41:59.256436+0000","last_clean":"2023-10-11T09:41:59.256436+0000","last_became_active":"2023-10-11T09:41:58.779355+0000","last_became_peered":"2023-10-11T09:41:58.779355+0000","last_unstale":"2023-10-11T09:41:59.256436+0000","last_undegraded":"2023-10-11T09:41:59.256436+0000","last_fullsized":"2023-10-11T09:41:59.256436+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:24:11.694544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.134680+0000","last_change":"2023-10-11T09:41:59.274318+0000","last_active":"2023-10-11T09:42:02.134680+0000","last_peered":"2023-10-11T09:42:02.134680+0000","last_clean":"2023-10-11T09:42:02.134680+0000","last_became_active":"2023-10-11T09:41:59.274070+0000","last_became_peered":"2023-10-11T09:41:59.274070+0000","last_unstale":"2023-10-11T09:42:02.134680+0000","last_undegraded":"2023-10-11T09:42:02.134680+0000","last_fullsized":"2023-10-11T09:42:02.134680+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:32:44.448136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"36'3","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.273507+0000","last_change":"2023-10-11T09:41:58.776724+0000","last_active":"2023-10-11T09:41:59.273507+0000","last_peered":"2023-10-11T09:41:59.273507+0000","last_clean":"2023-10-11T09:41:59.273507+0000","last_became_active":"2023-10-11T09:41:58.776359+0000","last_became_peered":"2023-10-11T09:41:58.776359+0000","last_unstale":"2023-10-11T09:41:59.273507+0000","last_undegraded":"2023-10-11T09:41:59.273507+0000","last_fullsized":"2023-10-11T09:41:59.273507+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:55:29.823667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:01.632819+0000","last_change":"2023-10-11T09:41:59.252336+0000","last_active":"2023-10-11T09:42:01.632819+0000","last_peered":"2023-10-11T09:42:01.632819+0000","last_clean":"2023-10-11T09:42:01.632819+0000","last_became_active":"2023-10-11T09:41:59.252084+0000","last_became_peered":"2023-10-11T09:41:59.252084+0000","last_unstale":"2023-10-11T09:42:01.632819+0000","last_undegraded":"2023-10-11T09:42:01.632819+0000","last_fullsized":"2023-10-11T09:42:01.632819+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:29:09.854758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"36'6","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.281790+0000","last_change":"2023-10-11T09:41:58.775314+0000","last_active":"2023-10-11T09:41:59.281790+0000","last_peered":"2023-10-11T09:41:59.281790+0000","last_clean":"2023-10-11T09:41:59.281790+0000","last_became_active":"2023-10-11T09:41:58.775000+0000","last_became_peered":"2023-10-11T09:41:58.775000+0000","last_unstale":"2023-10-11T09:41:59.281790+0000","last_undegraded":"2023-10-11T09:41:59.281790+0000","last_fullsized":"2023-10-11T09:41:59.281790+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:23:59.470658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133578+0000","last_change":"2023-10-11T09:41:59.267987+0000","last_active":"2023-10-11T09:42:02.133578+0000","last_peered":"2023-10-11T09:42:02.133578+0000","last_clean":"2023-10-11T09:42:02.133578+0000","last_became_active":"2023-10-11T09:41:59.267705+0000","last_became_peered":"2023-10-11T09:41:59.267705+0000","last_unstale":"2023-10-11T09:42:02.133578+0000","last_undegraded":"2023-10-11T09:42:02.133578+0000","last_fullsized":"2023-10-11T09:42:02.133578+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:34:02.831174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"36'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.256670+0000","last_change":"2023-10-11T09:41:58.776379+0000","last_active":"2023-10-11T09:41:59.256670+0000","last_peered":"2023-10-11T09:41:59.256670+0000","last_clean":"2023-10-11T09:41:59.256670+0000","last_became_active":"2023-10-11T09:41:58.776071+0000","last_became_peered":"2023-10-11T09:41:58.776071+0000","last_unstale":"2023-10-11T09:41:59.256670+0000","last_undegraded":"2023-10-11T09:41:59.256670+0000","last_fullsized":"2023-10-11T09:41:59.256670+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:24:11.694544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.132753+0000","last_change":"2023-10-11T09:41:59.250221+0000","last_active":"2023-10-11T09:42:02.132753+0000","last_peered":"2023-10-11T09:42:02.132753+0000","last_clean":"2023-10-11T09:42:02.132753+0000","last_became_active":"2023-10-11T09:41:59.249523+0000","last_became_peered":"2023-10-11T09:41:59.249523+0000","last_unstale":"2023-10-11T09:42:02.132753+0000","last_undegraded":"2023-10-11T09:42:02.132753+0000","last_fullsized":"2023-10-11T09:42:02.132753+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:49:29.335655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.242895+0000","last_change":"2023-10-11T09:41:58.776351+0000","last_active":"2023-10-11T09:41:59.242895+0000","last_peered":"2023-10-11T09:41:59.242895+0000","last_clean":"2023-10-11T09:41:59.242895+0000","last_became_active":"2023-10-11T09:41:58.776076+0000","last_became_peered":"2023-10-11T09:41:58.776076+0000","last_unstale":"2023-10-11T09:41:59.242895+0000","last_undegraded":"2023-10-11T09:41:59.242895+0000","last_fullsized":"2023-10-11T09:41:59.242895+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:18:56.139478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.246103+0000","last_change":"2023-10-11T09:41:59.266545+0000","last_active":"2023-10-11T09:42:00.246103+0000","last_peered":"2023-10-11T09:42:00.246103+0000","last_clean":"2023-10-11T09:42:00.246103+0000","last_became_active":"2023-10-11T09:41:59.266181+0000","last_became_peered":"2023-10-11T09:41:59.266181+0000","last_unstale":"2023-10-11T09:42:00.246103+0000","last_undegraded":"2023-10-11T09:42:00.246103+0000","last_fullsized":"2023-10-11T09:42:00.246103+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:46:25.915286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.255044+0000","last_change":"2023-10-11T09:41:58.781510+0000","last_active":"2023-10-11T09:41:59.255044+0000","last_peered":"2023-10-11T09:41:59.255044+0000","last_clean":"2023-10-11T09:41:59.255044+0000","last_became_active":"2023-10-11T09:41:58.781083+0000","last_became_peered":"2023-10-11T09:41:58.781083+0000","last_unstale":"2023-10-11T09:41:59.255044+0000","last_undegraded":"2023-10-11T09:41:59.255044+0000","last_fullsized":"2023-10-11T09:41:59.255044+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:02:09.585130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133419+0000","last_change":"2023-10-11T09:41:59.250127+0000","last_active":"2023-10-11T09:42:02.133419+0000","last_peered":"2023-10-11T09:42:02.133419+0000","last_clean":"2023-10-11T09:42:02.133419+0000","last_became_active":"2023-10-11T09:41:59.249850+0000","last_became_peered":"2023-10-11T09:41:59.249850+0000","last_unstale":"2023-10-11T09:42:02.133419+0000","last_undegraded":"2023-10-11T09:42:02.133419+0000","last_fullsized":"2023-10-11T09:42:02.133419+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:29:15.375697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"36'8","reported_seq":19,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.276220+0000","last_change":"2023-10-11T09:41:58.782484+0000","last_active":"2023-10-11T09:41:59.276220+0000","last_peered":"2023-10-11T09:41:59.276220+0000","last_clean":"2023-10-11T09:41:59.276220+0000","last_became_active":"2023-10-11T09:41:58.781980+0000","last_became_peered":"2023-10-11T09:41:58.781980+0000","last_unstale":"2023-10-11T09:41:59.276220+0000","last_undegraded":"2023-10-11T09:41:59.276220+0000","last_fullsized":"2023-10-11T09:41:59.276220+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:05:43.880930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.134151+0000","last_change":"2023-10-11T09:41:59.257871+0000","last_active":"2023-10-11T09:42:02.134151+0000","last_peered":"2023-10-11T09:42:02.134151+0000","last_clean":"2023-10-11T09:42:02.134151+0000","last_became_active":"2023-10-11T09:41:59.257622+0000","last_became_peered":"2023-10-11T09:41:59.257622+0000","last_unstale":"2023-10-11T09:42:02.134151+0000","last_undegraded":"2023-10-11T09:42:02.134151+0000","last_fullsized":"2023-10-11T09:42:02.134151+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:34:02.831174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"36'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.278482+0000","last_change":"2023-10-11T09:41:58.782590+0000","last_active":"2023-10-11T09:41:59.278482+0000","last_peered":"2023-10-11T09:41:59.278482+0000","last_clean":"2023-10-11T09:41:59.278482+0000","last_became_active":"2023-10-11T09:41:58.782326+0000","last_became_peered":"2023-10-11T09:41:58.782326+0000","last_unstale":"2023-10-11T09:41:59.278482+0000","last_undegraded":"2023-10-11T09:41:59.278482+0000","last_fullsized":"2023-10-11T09:41:59.278482+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:11:34.532326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.247230+0000","last_change":"2023-10-11T09:41:59.270677+0000","last_active":"2023-10-11T09:42:00.247230+0000","last_peered":"2023-10-11T09:42:00.247230+0000","last_clean":"2023-10-11T09:42:00.247230+0000","last_became_active":"2023-10-11T09:41:59.270348+0000","last_became_peered":"2023-10-11T09:41:59.270348+0000","last_unstale":"2023-10-11T09:42:00.247230+0000","last_undegraded":"2023-10-11T09:42:00.247230+0000","last_fullsized":"2023-10-11T09:42:00.247230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:34:02.831174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"36'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.273284+0000","last_change":"2023-10-11T09:41:58.778144+0000","last_active":"2023-10-11T09:41:59.273284+0000","last_peered":"2023-10-11T09:41:59.273284+0000","last_clean":"2023-10-11T09:41:59.273284+0000","last_became_active":"2023-10-11T09:41:58.777788+0000","last_became_peered":"2023-10-11T09:41:58.777788+0000","last_unstale":"2023-10-11T09:41:59.273284+0000","last_undegraded":"2023-10-11T09:41:59.273284+0000","last_fullsized":"2023-10-11T09:41:59.273284+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:42:13.962137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:01.632636+0000","last_change":"2023-10-11T09:41:59.251031+0000","last_active":"2023-10-11T09:42:01.632636+0000","last_peered":"2023-10-11T09:42:01.632636+0000","last_clean":"2023-10-11T09:42:01.632636+0000","last_became_active":"2023-10-11T09:41:59.250626+0000","last_became_peered":"2023-10-11T09:41:59.250626+0000","last_unstale":"2023-10-11T09:42:01.632636+0000","last_undegraded":"2023-10-11T09:42:01.632636+0000","last_fullsized":"2023-10-11T09:42:01.632636+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:32:44.448136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"36'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.274594+0000","last_change":"2023-10-11T09:41:58.773947+0000","last_active":"2023-10-11T09:41:59.274594+0000","last_peered":"2023-10-11T09:41:59.274594+0000","last_clean":"2023-10-11T09:41:59.274594+0000","last_became_active":"2023-10-11T09:41:58.773642+0000","last_became_peered":"2023-10-11T09:41:58.773642+0000","last_unstale":"2023-10-11T09:41:59.274594+0000","last_undegraded":"2023-10-11T09:41:59.274594+0000","last_fullsized":"2023-10-11T09:41:59.274594+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:49:37.959487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.245415+0000","last_change":"2023-10-11T09:41:59.252857+0000","last_active":"2023-10-11T09:42:00.245415+0000","last_peered":"2023-10-11T09:42:00.245415+0000","last_clean":"2023-10-11T09:42:00.245415+0000","last_became_active":"2023-10-11T09:41:59.252603+0000","last_became_peered":"2023-10-11T09:41:59.252603+0000","last_unstale":"2023-10-11T09:42:00.245415+0000","last_undegraded":"2023-10-11T09:42:00.245415+0000","last_fullsized":"2023-10-11T09:42:00.245415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:35:44.282265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.242457+0000","last_change":"2023-10-11T09:41:58.775051+0000","last_active":"2023-10-11T09:41:59.242457+0000","last_peered":"2023-10-11T09:41:59.242457+0000","last_clean":"2023-10-11T09:41:59.242457+0000","last_became_active":"2023-10-11T09:41:58.774781+0000","last_became_peered":"2023-10-11T09:41:58.774781+0000","last_unstale":"2023-10-11T09:41:59.242457+0000","last_undegraded":"2023-10-11T09:41:59.242457+0000","last_fullsized":"2023-10-11T09:41:59.242457+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:35:43.267132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.245933+0000","last_change":"2023-10-11T09:41:59.276846+0000","last_active":"2023-10-11T09:42:00.245933+0000","last_peered":"2023-10-11T09:42:00.245933+0000","last_clean":"2023-10-11T09:42:00.245933+0000","last_became_active":"2023-10-11T09:41:59.276042+0000","last_became_peered":"2023-10-11T09:41:59.276042+0000","last_unstale":"2023-10-11T09:42:00.245933+0000","last_undegraded":"2023-10-11T09:42:00.245933+0000","last_fullsized":"2023-10-11T09:42:00.245933+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:07:13.635301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"36'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.266966+0000","last_change":"2023-10-11T09:41:58.777266+0000","last_active":"2023-10-11T09:41:59.266966+0000","last_peered":"2023-10-11T09:41:59.266966+0000","last_clean":"2023-10-11T09:41:59.266966+0000","last_became_active":"2023-10-11T09:41:58.776620+0000","last_became_peered":"2023-10-11T09:41:58.776620+0000","last_unstale":"2023-10-11T09:41:59.266966+0000","last_undegraded":"2023-10-11T09:41:59.266966+0000","last_fullsized":"2023-10-11T09:41:59.266966+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:55:46.983239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.132993+0000","last_change":"2023-10-11T09:41:59.276325+0000","last_active":"2023-10-11T09:42:02.132993+0000","last_peered":"2023-10-11T09:42:02.132993+0000","last_clean":"2023-10-11T09:42:02.132993+0000","last_became_active":"2023-10-11T09:41:59.275987+0000","last_became_peered":"2023-10-11T09:41:59.275987+0000","last_unstale":"2023-10-11T09:42:02.132993+0000","last_undegraded":"2023-10-11T09:42:02.132993+0000","last_fullsized":"2023-10-11T09:42:02.132993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:14:53.871518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.234885+0000","last_change":"2023-10-11T09:41:58.779536+0000","last_active":"2023-10-11T09:41:59.234885+0000","last_peered":"2023-10-11T09:41:59.234885+0000","last_clean":"2023-10-11T09:41:59.234885+0000","last_became_active":"2023-10-11T09:41:58.779262+0000","last_became_peered":"2023-10-11T09:41:58.779262+0000","last_unstale":"2023-10-11T09:41:59.234885+0000","last_undegraded":"2023-10-11T09:41:59.234885+0000","last_fullsized":"2023-10-11T09:41:59.234885+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:44:25.949842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.246258+0000","last_change":"2023-10-11T09:41:59.265507+0000","last_active":"2023-10-11T09:42:00.246258+0000","last_peered":"2023-10-11T09:42:00.246258+0000","last_clean":"2023-10-11T09:42:00.246258+0000","last_became_active":"2023-10-11T09:41:59.265264+0000","last_became_peered":"2023-10-11T09:41:59.265264+0000","last_unstale":"2023-10-11T09:42:00.246258+0000","last_undegraded":"2023-10-11T09:42:00.246258+0000","last_fullsized":"2023-10-11T09:42:00.246258+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:51:26.096609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.241218+0000","last_change":"2023-10-11T09:41:58.773543+0000","last_active":"2023-10-11T09:41:59.241218+0000","last_peered":"2023-10-11T09:41:59.241218+0000","last_clean":"2023-10-11T09:41:59.241218+0000","last_became_active":"2023-10-11T09:41:58.773285+0000","last_became_peered":"2023-10-11T09:41:58.773285+0000","last_unstale":"2023-10-11T09:41:59.241218+0000","last_undegraded":"2023-10-11T09:41:59.241218+0000","last_fullsized":"2023-10-11T09:41:59.241218+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:09:10.546323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133525+0000","last_change":"2023-10-11T09:41:59.275401+0000","last_active":"2023-10-11T09:42:02.133525+0000","last_peered":"2023-10-11T09:42:02.133525+0000","last_clean":"2023-10-11T09:42:02.133525+0000","last_became_active":"2023-10-11T09:41:59.274821+0000","last_became_peered":"2023-10-11T09:41:59.274821+0000","last_unstale":"2023-10-11T09:42:02.133525+0000","last_undegraded":"2023-10-11T09:42:02.133525+0000","last_fullsized":"2023-10-11T09:42:02.133525+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:51:26.096609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.234879+0000","last_change":"2023-10-11T09:41:58.778916+0000","last_active":"2023-10-11T09:41:59.234879+0000","last_peered":"2023-10-11T09:41:59.234879+0000","last_clean":"2023-10-11T09:41:59.234879+0000","last_became_active":"2023-10-11T09:41:58.778610+0000","last_became_peered":"2023-10-11T09:41:58.778610+0000","last_unstale":"2023-10-11T09:41:59.234879+0000","last_undegraded":"2023-10-11T09:41:59.234879+0000","last_fullsized":"2023-10-11T09:41:59.234879+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:40:16.473783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.247174+0000","last_change":"2023-10-11T09:41:59.270447+0000","last_active":"2023-10-11T09:42:00.247174+0000","last_peered":"2023-10-11T09:42:00.247174+0000","last_clean":"2023-10-11T09:42:00.247174+0000","last_became_active":"2023-10-11T09:41:59.270124+0000","last_became_peered":"2023-10-11T09:41:59.270124+0000","last_unstale":"2023-10-11T09:42:00.247174+0000","last_undegraded":"2023-10-11T09:42:00.247174+0000","last_fullsized":"2023-10-11T09:42:00.247174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:45:37.542487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.241255+0000","last_change":"2023-10-11T09:41:58.777578+0000","last_active":"2023-10-11T09:41:59.241255+0000","last_peered":"2023-10-11T09:41:59.241255+0000","last_clean":"2023-10-11T09:41:59.241255+0000","last_became_active":"2023-10-11T09:41:58.777251+0000","last_became_peered":"2023-10-11T09:41:58.777251+0000","last_unstale":"2023-10-11T09:41:59.241255+0000","last_undegraded":"2023-10-11T09:41:59.241255+0000","last_fullsized":"2023-10-11T09:41:59.241255+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:23:59.470658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":181,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.240598+0000","last_change":"2023-10-11T09:40:29.874372+0000","last_active":"2023-10-11T09:41:59.240598+0000","last_peered":"2023-10-11T09:41:59.240598+0000","last_clean":"2023-10-11T09:41:59.240598+0000","last_became_active":"2023-10-11T09:40:29.873621+0000","last_became_peered":"2023-10-11T09:40:29.873621+0000","last_unstale":"2023-10-11T09:41:59.240598+0000","last_undegraded":"2023-10-11T09:41:59.240598+0000","last_fullsized":"2023-10-11T09:41:59.240598+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_clean_scrub_stamp":"2023-10-11T09:40:28.742276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:22:07.985311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.251308+0000","last_change":"2023-10-11T09:40:36.148773+0000","last_active":"2023-10-11T09:41:59.251308+0000","last_peered":"2023-10-11T09:41:59.251308+0000","last_clean":"2023-10-11T09:41:59.251308+0000","last_became_active":"2023-10-11T09:40:36.148489+0000","last_became_peered":"2023-10-11T09:40:36.148489+0000","last_unstale":"2023-10-11T09:41:59.251308+0000","last_undegraded":"2023-10-11T09:41:59.251308+0000","last_fullsized":"2023-10-11T09:41:59.251308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.132837+0000","last_change":"2023-10-11T09:41:59.268241+0000","last_active":"2023-10-11T09:42:02.132837+0000","last_peered":"2023-10-11T09:42:02.132837+0000","last_clean":"2023-10-11T09:42:02.132837+0000","last_became_active":"2023-10-11T09:41:59.267982+0000","last_became_peered":"2023-10-11T09:41:59.267982+0000","last_unstale":"2023-10-11T09:42:02.132837+0000","last_undegraded":"2023-10-11T09:42:02.132837+0000","last_fullsized":"2023-10-11T09:42:02.132837+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:42:14.977269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"36'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.273771+0000","last_change":"2023-10-11T09:41:58.777702+0000","last_active":"2023-10-11T09:41:59.273771+0000","last_peered":"2023-10-11T09:41:59.273771+0000","last_clean":"2023-10-11T09:41:59.273771+0000","last_became_active":"2023-10-11T09:41:58.777413+0000","last_became_peered":"2023-10-11T09:41:58.777413+0000","last_unstale":"2023-10-11T09:41:59.273771+0000","last_undegraded":"2023-10-11T09:41:59.273771+0000","last_fullsized":"2023-10-11T09:41:59.273771+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:58:21.613408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":28,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.240155+0000","last_change":"2023-10-11T09:40:36.149739+0000","last_active":"2023-10-11T09:41:59.240155+0000","last_peered":"2023-10-11T09:41:59.240155+0000","last_clean":"2023-10-11T09:41:59.240155+0000","last_became_active":"2023-10-11T09:40:36.149419+0000","last_became_peered":"2023-10-11T09:40:36.149419+0000","last_unstale":"2023-10-11T09:41:59.240155+0000","last_undegraded":"2023-10-11T09:41:59.240155+0000","last_fullsized":"2023-10-11T09:41:59.240155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:54:50.640362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.246403+0000","last_change":"2023-10-11T09:41:59.274654+0000","last_active":"2023-10-11T09:42:00.246403+0000","last_peered":"2023-10-11T09:42:00.246403+0000","last_clean":"2023-10-11T09:42:00.246403+0000","last_became_active":"2023-10-11T09:41:59.274296+0000","last_became_peered":"2023-10-11T09:41:59.274296+0000","last_unstale":"2023-10-11T09:42:00.246403+0000","last_undegraded":"2023-10-11T09:42:00.246403+0000","last_fullsized":"2023-10-11T09:42:00.246403+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:29:17.473816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"37'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.245245+0000","last_change":"2023-10-11T09:41:58.778782+0000","last_active":"2023-10-11T09:42:00.245245+0000","last_peered":"2023-10-11T09:42:00.245245+0000","last_clean":"2023-10-11T09:42:00.245245+0000","last_became_active":"2023-10-11T09:41:58.778514+0000","last_became_peered":"2023-10-11T09:41:58.778514+0000","last_unstale":"2023-10-11T09:42:00.245245+0000","last_undegraded":"2023-10-11T09:42:00.245245+0000","last_fullsized":"2023-10-11T09:42:00.245245+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:41:27.673781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":25,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.234843+0000","last_change":"2023-10-11T09:40:36.216248+0000","last_active":"2023-10-11T09:41:59.234843+0000","last_peered":"2023-10-11T09:41:59.234843+0000","last_clean":"2023-10-11T09:41:59.234843+0000","last_became_active":"2023-10-11T09:40:36.215890+0000","last_became_peered":"2023-10-11T09:40:36.215890+0000","last_unstale":"2023-10-11T09:41:59.234843+0000","last_undegraded":"2023-10-11T09:41:59.234843+0000","last_fullsized":"2023-10-11T09:41:59.234843+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.135487+0000","last_change":"2023-10-11T09:41:59.253002+0000","last_active":"2023-10-11T09:42:02.135487+0000","last_peered":"2023-10-11T09:42:02.135487+0000","last_clean":"2023-10-11T09:42:02.135487+0000","last_became_active":"2023-10-11T09:41:59.252742+0000","last_became_peered":"2023-10-11T09:41:59.252742+0000","last_unstale":"2023-10-11T09:42:02.135487+0000","last_undegraded":"2023-10-11T09:42:02.135487+0000","last_fullsized":"2023-10-11T09:42:02.135487+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:46:25.915286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.242326+0000","last_change":"2023-10-11T09:41:58.778606+0000","last_active":"2023-10-11T09:41:59.242326+0000","last_peered":"2023-10-11T09:41:59.242326+0000","last_clean":"2023-10-11T09:41:59.242326+0000","last_became_active":"2023-10-11T09:41:58.778305+0000","last_became_peered":"2023-10-11T09:41:58.778305+0000","last_unstale":"2023-10-11T09:41:59.242326+0000","last_undegraded":"2023-10-11T09:41:59.242326+0000","last_fullsized":"2023-10-11T09:41:59.242326+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:55:29.823667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.246884+0000","last_change":"2023-10-11T09:40:36.151710+0000","last_active":"2023-10-11T09:41:59.246884+0000","last_peered":"2023-10-11T09:41:59.246884+0000","last_clean":"2023-10-11T09:41:59.246884+0000","last_became_active":"2023-10-11T09:40:36.151219+0000","last_became_peered":"2023-10-11T09:40:36.151219+0000","last_unstale":"2023-10-11T09:41:59.246884+0000","last_undegraded":"2023-10-11T09:41:59.246884+0000","last_fullsized":"2023-10-11T09:41:59.246884+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133803+0000","last_change":"2023-10-11T09:41:59.252548+0000","last_active":"2023-10-11T09:42:02.133803+0000","last_peered":"2023-10-11T09:42:02.133803+0000","last_clean":"2023-10-11T09:42:02.133803+0000","last_became_active":"2023-10-11T09:41:59.252259+0000","last_became_peered":"2023-10-11T09:41:59.252259+0000","last_unstale":"2023-10-11T09:42:02.133803+0000","last_undegraded":"2023-10-11T09:42:02.133803+0000","last_fullsized":"2023-10-11T09:42:02.133803+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:07:46.727754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.240925+0000","last_change":"2023-10-11T09:41:58.772968+0000","last_active":"2023-10-11T09:41:59.240925+0000","last_peered":"2023-10-11T09:41:59.240925+0000","last_clean":"2023-10-11T09:41:59.240925+0000","last_became_active":"2023-10-11T09:41:58.772686+0000","last_became_peered":"2023-10-11T09:41:58.772686+0000","last_unstale":"2023-10-11T09:41:59.240925+0000","last_undegraded":"2023-10-11T09:41:59.240925+0000","last_fullsized":"2023-10-11T09:41:59.240925+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:58:21.613408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.242524+0000","last_change":"2023-10-11T09:40:36.160620+0000","last_active":"2023-10-11T09:41:59.242524+0000","last_peered":"2023-10-11T09:41:59.242524+0000","last_clean":"2023-10-11T09:41:59.242524+0000","last_became_active":"2023-10-11T09:40:36.160171+0000","last_became_peered":"2023-10-11T09:40:36.160171+0000","last_unstale":"2023-10-11T09:41:59.242524+0000","last_undegraded":"2023-10-11T09:41:59.242524+0000","last_fullsized":"2023-10-11T09:41:59.242524+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133691+0000","last_change":"2023-10-11T09:41:59.260111+0000","last_active":"2023-10-11T09:42:02.133691+0000","last_peered":"2023-10-11T09:42:02.133691+0000","last_clean":"2023-10-11T09:42:02.133691+0000","last_became_active":"2023-10-11T09:41:59.259796+0000","last_became_peered":"2023-10-11T09:41:59.259796+0000","last_unstale":"2023-10-11T09:42:02.133691+0000","last_undegraded":"2023-10-11T09:42:02.133691+0000","last_fullsized":"2023-10-11T09:42:02.133691+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:55:30.838799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.255282+0000","last_change":"2023-10-11T09:41:58.773403+0000","last_active":"2023-10-11T09:41:59.255282+0000","last_peered":"2023-10-11T09:41:59.255282+0000","last_clean":"2023-10-11T09:41:59.255282+0000","last_became_active":"2023-10-11T09:41:58.773132+0000","last_became_peered":"2023-10-11T09:41:58.773132+0000","last_unstale":"2023-10-11T09:41:59.255282+0000","last_undegraded":"2023-10-11T09:41:59.255282+0000","last_fullsized":"2023-10-11T09:41:59.255282+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:28:16.256141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":28,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.241892+0000","last_change":"2023-10-11T09:40:36.162174+0000","last_active":"2023-10-11T09:41:59.241892+0000","last_peered":"2023-10-11T09:41:59.241892+0000","last_clean":"2023-10-11T09:41:59.241892+0000","last_became_active":"2023-10-11T09:40:36.161710+0000","last_became_peered":"2023-10-11T09:40:36.161710+0000","last_unstale":"2023-10-11T09:41:59.241892+0000","last_undegraded":"2023-10-11T09:41:59.241892+0000","last_fullsized":"2023-10-11T09:41:59.241892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:57:03.630878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:01.632131+0000","last_change":"2023-10-11T09:41:59.251157+0000","last_active":"2023-10-11T09:42:01.632131+0000","last_peered":"2023-10-11T09:42:01.632131+0000","last_clean":"2023-10-11T09:42:01.632131+0000","last_became_active":"2023-10-11T09:41:59.250831+0000","last_became_peered":"2023-10-11T09:41:59.250831+0000","last_unstale":"2023-10-11T09:42:01.632131+0000","last_undegraded":"2023-10-11T09:42:01.632131+0000","last_fullsized":"2023-10-11T09:42:01.632131+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:34:02.831174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.242946+0000","last_change":"2023-10-11T09:41:58.779092+0000","last_active":"2023-10-11T09:41:59.242946+0000","last_peered":"2023-10-11T09:41:59.242946+0000","last_clean":"2023-10-11T09:41:59.242946+0000","last_became_active":"2023-10-11T09:41:58.778781+0000","last_became_peered":"2023-10-11T09:41:58.778781+0000","last_unstale":"2023-10-11T09:41:59.242946+0000","last_undegraded":"2023-10-11T09:41:59.242946+0000","last_fullsized":"2023-10-11T09:41:59.242946+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:07:42.669679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":24,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.244192+0000","last_change":"2023-10-11T09:40:36.150224+0000","last_active":"2023-10-11T09:41:59.244192+0000","last_peered":"2023-10-11T09:41:59.244192+0000","last_clean":"2023-10-11T09:41:59.244192+0000","last_became_active":"2023-10-11T09:40:36.149919+0000","last_became_peered":"2023-10-11T09:40:36.149919+0000","last_unstale":"2023-10-11T09:41:59.244192+0000","last_undegraded":"2023-10-11T09:41:59.244192+0000","last_fullsized":"2023-10-11T09:41:59.244192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.246688+0000","last_change":"2023-10-11T09:41:59.275142+0000","last_active":"2023-10-11T09:42:00.246688+0000","last_peered":"2023-10-11T09:42:00.246688+0000","last_clean":"2023-10-11T09:42:00.246688+0000","last_became_active":"2023-10-11T09:41:59.274690+0000","last_became_peered":"2023-10-11T09:41:59.274690+0000","last_unstale":"2023-10-11T09:42:00.246688+0000","last_undegraded":"2023-10-11T09:42:00.246688+0000","last_fullsized":"2023-10-11T09:42:00.246688+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:32:36.655338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.241558+0000","last_change":"2023-10-11T09:41:58.773081+0000","last_active":"2023-10-11T09:41:59.241558+0000","last_peered":"2023-10-11T09:41:59.241558+0000","last_clean":"2023-10-11T09:41:59.241558+0000","last_became_active":"2023-10-11T09:41:58.772778+0000","last_became_peered":"2023-10-11T09:41:58.772778+0000","last_unstale":"2023-10-11T09:41:59.241558+0000","last_undegraded":"2023-10-11T09:41:59.241558+0000","last_fullsized":"2023-10-11T09:41:59.241558+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:05:43.880930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":24,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.255713+0000","last_change":"2023-10-11T09:40:36.151839+0000","last_active":"2023-10-11T09:41:59.255713+0000","last_peered":"2023-10-11T09:41:59.255713+0000","last_clean":"2023-10-11T09:41:59.255713+0000","last_became_active":"2023-10-11T09:40:36.151179+0000","last_became_peered":"2023-10-11T09:40:36.151179+0000","last_unstale":"2023-10-11T09:41:59.255713+0000","last_undegraded":"2023-10-11T09:41:59.255713+0000","last_fullsized":"2023-10-11T09:41:59.255713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":1032192,"data_stored":767390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.21,"15min":1.21},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.3440000000000001}]},{"osd":1,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":2.25,"5min":2.25,"15min":2.25},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.359}]},{"osd":2,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.206},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":1.2390000000000001}]},{"osd":3,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.286,"15min":1.286},"min":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":2.1579999999999999,"5min":2.1579999999999999,"15min":2.1579999999999999},"last":1.073}]},{"osd":4,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":1.036}]},{"osd":5,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.292},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":3.0750000000000002,"5min":3.0750000000000002,"15min":3.0750000000000002},"last":1.2689999999999999}]},{"osd":6,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.49,"15min":1.49},"min":{"1min":1.153,"5min":1.153,"15min":1.153},"max":{"1min":3.137,"5min":3.137,"15min":3.137},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"min":{"1min":1.099,"5min":1.099,"15min":1.099},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":1.3200000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":425984,"data_stored":175970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.90600000000000003}]},{"osd":1,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.202,"15min":1.202},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":1.8260000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.2250000000000001}]},{"osd":2,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":1.079},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":2.605,"5min":2.605,"15min":2.605},"last":0.98999999999999999}]},{"osd":3,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":3.0819999999999999,"5min":3.0819999999999999,"15min":3.0819999999999999},"last":0.82599999999999996}]},{"osd":4,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.337,"15min":1.337},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.978,"5min":1.978,"15min":1.978},"last":1.111},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.6920000000000002,"5min":2.6920000000000002,"15min":2.6920000000000002},"last":1.302}]},{"osd":5,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.264},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":2.7290000000000001,"5min":2.7290000000000001,"15min":2.7290000000000001},"last":1.347}]},{"osd":7,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":2.1899999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.3050000000000002,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.175}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":430080,"data_stored":175500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":1.23}]},{"osd":1,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":1.1559999999999999}]},{"osd":2,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.831,"5min":1.831,"15min":1.831},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.284}]},{"osd":3,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.319},{"interface":"front","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.419}]},{"osd":4,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":1.369},{"interface":"front","average":{"1min":1.228,"5min":1.228,"15min":1.228},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":1.0840000000000001}]},{"osd":6,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.4529999999999998,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":1.4610000000000001}]},{"osd":7,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":3.6120000000000001,"5min":3.6120000000000001,"15min":3.6120000000000001},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"min":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":3.2109999999999999,"5min":3.2109999999999999,"15min":3.2109999999999999},"last":1.55}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":430080,"data_stored":175992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":1.286},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.1160000000000001,"5min":2.1160000000000001,"15min":2.1160000000000001},"last":1.381}]},{"osd":1,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.98399999999999999}]},{"osd":2,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":1.345},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":1.7130000000000001}]},{"osd":3,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.367,"5min":1.367,"15min":1.367},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.3340000000000001,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":1.6679999999999999}]},{"osd":5,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.236,"15min":1.236},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.93999999999999995}]},{"osd":6,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.8,"5min":1.8,"15min":1.8},"last":1.121},{"interface":"front","average":{"1min":1.357,"5min":1.357,"15min":1.357},"min":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.5549999999999999}]},{"osd":7,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.482},{"interface":"front","average":{"1min":1.516,"5min":1.516,"15min":1.516},"min":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"max":{"1min":1.9990000000000001,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":1.786}]}]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":438272,"data_stored":176986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.747},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.48699999999999999}]},{"osd":1,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.60199999999999998}]},{"osd":2,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.5870000000000002,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":0.98399999999999999}]},{"osd":4,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.395,"5min":2.395,"15min":2.395},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.367,"5min":1.367,"15min":1.367},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":3.1579999999999999,"5min":3.1579999999999999,"15min":3.1579999999999999},"last":1.1870000000000001}]},{"osd":5,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.393,"15min":1.393},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":3.0009999999999999,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":1.03},{"interface":"front","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":3.8050000000000002,"5min":3.8050000000000002,"15min":3.8050000000000002},"last":1.224}]},{"osd":6,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":2.1949999999999998,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":3.2549999999999999,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":1.266}]},{"osd":7,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":3.7490000000000001,"5min":3.7490000000000001,"15min":3.7490000000000001},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.448,"5min":1.448,"15min":1.448},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":1.121}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":425984,"data_stored":175458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":2.0600000000000001,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":2.8919999999999999,"5min":2.8919999999999999,"15min":2.8919999999999999},"last":0.68000000000000005}]},{"osd":1,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":3.7650000000000001,"5min":3.7650000000000001,"15min":3.7650000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.5339999999999998,"5min":2.5339999999999998,"15min":2.5339999999999998},"last":0.71499999999999997}]},{"osd":3,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.403,"5min":2.403,"15min":2.403},"last":0.65500000000000003},{"interface":"front","average":{"1min":1.117,"5min":1.117,"15min":1.117},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":2.7589999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":0.63600000000000001}]},{"osd":4,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.476,"5min":1.476,"15min":1.476},"min":{"1min":1.085,"5min":1.085,"15min":1.085},"max":{"1min":2.6859999999999999,"5min":2.6859999999999999,"15min":2.6859999999999999},"last":1.111}]},{"osd":5,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.409,"15min":1.409},"min":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"max":{"1min":3.044,"5min":3.044,"15min":3.044},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":3.5790000000000002,"5min":3.5790000000000002,"15min":3.5790000000000002},"last":0.89500000000000002}]},{"osd":6,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"min":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"max":{"1min":3.1600000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.044},{"interface":"front","average":{"1min":1.397,"5min":1.397,"15min":1.397},"min":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"max":{"1min":2.839,"5min":2.839,"15min":2.839},"last":0.85399999999999998}]},{"osd":7,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.49,"15min":1.49},"min":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":3.1230000000000002,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"min":{"1min":1.161,"5min":1.161,"15min":1.161},"max":{"1min":2.988,"5min":2.988,"15min":2.988},"last":1.0780000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":425984,"data_stored":175478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:41:28 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":1.159}]},{"osd":2,"last update":"Wed Oct 11 09:41:28 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.85599999999999998}]},{"osd":3,"last update":"Wed Oct 11 09:41:28 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":0.60699999999999998}]},{"osd":4,"last update":"Wed Oct 11 09:41:28 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":2.016,"5min":2.016,"15min":2.016},"last":1.147},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":2.323,"5min":2.323,"15min":2.323},"last":1.1240000000000001}]},{"osd":5,"last update":"Wed Oct 11 09:41:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":1.04,"5min":1.04,"15min":1.04},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.113}]},{"osd":6,"last update":"Wed Oct 11 09:41:28 2023","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"min":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"max":{"1min":2.8940000000000001,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":1.022,"5min":1.022,"15min":1.022},"max":{"1min":3.3719999999999999,"5min":3.3719999999999999,"15min":3.3719999999999999},"last":1.173}]},{"osd":7,"last update":"Wed Oct 11 09:41:28 2023","interfaces":[{"interface":"back","average":{"1min":1.468,"5min":1.468,"15min":1.468},"min":{"1min":1.171,"5min":1.171,"15min":1.171},"max":{"1min":2.194,"5min":2.194,"15min":2.194},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"min":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"max":{"1min":1.968,"5min":1.968,"15min":1.968},"last":0.99399999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":1019904,"data_stored":765790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 11 09:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.3100000000000001,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":0.73799999999999999}]},{"osd":2,"last update":"Wed Oct 11 09:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.63900000000000001}]},{"osd":3,"last update":"Wed Oct 11 09:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.7639999999999998,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":0.57699999999999996},{"interface":"front","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":0.93100000000000005}]},{"osd":4,"last update":"Wed Oct 11 09:41:27 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.242},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.6629999999999998,"5min":2.6629999999999998,"15min":2.6629999999999998},"last":1.1539999999999999}]},{"osd":5,"last update":"Wed Oct 11 09:41:27 2023","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"min":{"1min":0.998,"5min":0.998,"15min":0.998},"max":{"1min":3.0150000000000001,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":1.276},{"interface":"front","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":3.601,"5min":3.601,"15min":3.601},"last":1.331}]},{"osd":6,"last update":"Wed Oct 11 09:41:27 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.825,"5min":1.825,"15min":1.825},"last":0.997},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"min":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":1.1200000000000001}]},{"osd":7,"last update":"Wed Oct 11 09:41:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"min":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":1.381},{"interface":"front","average":{"1min":1.417,"5min":1.417,"15min":1.417},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":2.5419999999999998,"5min":2.5419999999999998,"15min":2.5419999999999998},"last":1.3069999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-11T09:42:08.376 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-11T09:42:08.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.553+0000 7f8f69e20700 1 Processor -- start 2023-10-11T09:42:08.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.553+0000 7f8f69e20700 1 -- start start 2023-10-11T09:42:08.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.557+0000 7f8f69e20700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f8f640cc820 0x7f8f64160c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:08.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.557+0000 7f8f69e20700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f64160210 con 0x7f8f6415d050 2023-10-11T09:42:08.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.557+0000 7f8f69e20700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f641611a0 con 0x7f8f640b06a0 2023-10-11T09:42:08.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.557+0000 7f8f69e20700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f640ccfc0 con 0x7f8f640cc820 2023-10-11T09:42:08.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.557+0000 7f8f63fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f8f640cc820 0x7f8f64160c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:08.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.557+0000 7f8f68e1e700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f8f6415d050 0x7f8f6415d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:51818/0 (socket says 172.21.15.78:51818) 2023-10-11T09:42:08.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.557+0000 7f8f68e1e700 1 -- 172.21.15.78:0/1867317212 learned_addr learned my addr 172.21.15.78:0/1867317212 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:42:08.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.557+0000 7f8f63fff700 1 -- 172.21.15.78:0/1867317212 >> v1:172.21.15.133:6789/0 conn(0x7f8f640b06a0 legacy=0x7f8f640b0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:08.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.557+0000 7f8f63fff700 1 -- 172.21.15.78:0/1867317212 >> v1:172.21.15.78:6789/0 conn(0x7f8f6415d050 legacy=0x7f8f6415d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:08.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.557+0000 7f8f63fff700 1 -- 172.21.15.78:0/1867317212 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f640cd2a0 con 0x7f8f640cc820 2023-10-11T09:42:08.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.557+0000 7f8f637fe700 1 -- 172.21.15.78:0/1867317212 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2569830995 0 0) 0x7f8f64160210 con 0x7f8f6415d050 2023-10-11T09:42:08.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.557+0000 7f8f63fff700 1 --2- 172.21.15.78:0/1867317212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f640cc820 0x7f8f64160c60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8f50009850 tx=0x7f8f50009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40576e6056c25058 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:08.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.557+0000 7f8f637fe700 1 -- 172.21.15.78:0/1867317212 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f50011830 con 0x7f8f640cc820 2023-10-11T09:42:08.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.557+0000 7f8f637fe700 1 -- 172.21.15.78:0/1867317212 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f50011990 con 0x7f8f640cc820 2023-10-11T09:42:08.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.557+0000 7f8f637fe700 1 -- 172.21.15.78:0/1867317212 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f50011d20 con 0x7f8f640cc820 2023-10-11T09:42:08.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.557+0000 7f8f69e20700 1 -- 172.21.15.78:0/1867317212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f640cc820 msgr2=0x7f8f64160c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:08.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.557+0000 7f8f69e20700 1 --2- 172.21.15.78:0/1867317212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f640cc820 0x7f8f64160c60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8f50009850 tx=0x7f8f50009b60 comp rx=0 tx=0).stop 2023-10-11T09:42:08.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.557+0000 7f8f69e20700 1 -- 172.21.15.78:0/1867317212 shutdown_connections 2023-10-11T09:42:08.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.557+0000 7f8f69e20700 1 --2- 172.21.15.78:0/1867317212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f640cc820 0x7f8f64160c60 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:08.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.557+0000 7f8f69e20700 1 -- 172.21.15.78:0/1867317212 >> 172.21.15.78:0/1867317212 conn(0x7f8f640cea60 msgr2=0x7f8f640cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:08.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.557+0000 7f8f69e20700 1 -- 172.21.15.78:0/1867317212 shutdown_connections 2023-10-11T09:42:08.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.557+0000 7f8f69e20700 1 -- 172.21.15.78:0/1867317212 wait complete. 2023-10-11T09:42:08.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.561+0000 7f8f69e20700 1 Processor -- start 2023-10-11T09:42:08.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.561+0000 7f8f69e20700 1 -- start start 2023-10-11T09:42:08.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.561+0000 7f8f69e20700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f640b06a0 0x7f8f64151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:08.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.561+0000 7f8f69e20700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8f640cc820 0x7f8f640713c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:08.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.561+0000 7f8f69e20700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f6415d050 0x7f8f64071900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:08.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.561+0000 7f8f69e20700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f641e6b30 con 0x7f8f640b06a0 2023-10-11T09:42:08.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.561+0000 7f8f69e20700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f640b2800 con 0x7f8f6415d050 2023-10-11T09:42:08.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.561+0000 7f8f69e20700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f640b1f60 con 0x7f8f640cc820 2023-10-11T09:42:08.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.565+0000 7f8f68e1e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f640b06a0 0x7f8f64151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:08.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.565+0000 7f8f68e1e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f640b06a0 0x7f8f64151b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60626/0 (socket says 172.21.15.78:60626) 2023-10-11T09:42:08.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.565+0000 7f8f68e1e700 1 -- 172.21.15.78:0/2222896105 learned_addr learned my addr 172.21.15.78:0/2222896105 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:08.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.565+0000 7f8f6961f700 1 --2- 172.21.15.78:0/2222896105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f6415d050 0x7f8f64071900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:08.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.565+0000 7f8f63fff700 1 --2- 172.21.15.78:0/2222896105 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8f640cc820 0x7f8f640713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:08.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.565+0000 7f8f68e1e700 1 -- 172.21.15.78:0/2222896105 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8f640cc820 msgr2=0x7f8f640713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:08.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.565+0000 7f8f68e1e700 1 --2- 172.21.15.78:0/2222896105 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8f640cc820 0x7f8f640713c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:08.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.565+0000 7f8f68e1e700 1 -- 172.21.15.78:0/2222896105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f6415d050 msgr2=0x7f8f64071900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:08.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.565+0000 7f8f68e1e700 1 --2- 172.21.15.78:0/2222896105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f6415d050 0x7f8f64071900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:08.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.565+0000 7f8f68e1e700 1 -- 172.21.15.78:0/2222896105 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f500094b0 con 0x7f8f640b06a0 2023-10-11T09:42:08.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.565+0000 7f8f6961f700 1 --2- 172.21.15.78:0/2222896105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f6415d050 0x7f8f64071900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:42:08.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.565+0000 7f8f68e1e700 1 --2- 172.21.15.78:0/2222896105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f640b06a0 0x7f8f64151b50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8f5800cc80 tx=0x7f8f5800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:08.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.565+0000 7f8f617fa700 1 -- 172.21.15.78:0/2222896105 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5800abb0 con 0x7f8f640b06a0 2023-10-11T09:42:08.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.565+0000 7f8f617fa700 1 -- 172.21.15.78:0/2222896105 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f5800ad10 con 0x7f8f640b06a0 2023-10-11T09:42:08.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.565+0000 7f8f617fa700 1 -- 172.21.15.78:0/2222896105 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f58007310 con 0x7f8f640b06a0 2023-10-11T09:42:08.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.565+0000 7f8f69e20700 1 -- 172.21.15.78:0/2222896105 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f640b2240 con 0x7f8f640b06a0 2023-10-11T09:42:08.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.565+0000 7f8f69e20700 1 -- 172.21.15.78:0/2222896105 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f6415c490 con 0x7f8f640b06a0 2023-10-11T09:42:08.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.565+0000 7f8f69e20700 1 -- 172.21.15.78:0/2222896105 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f641e6990 con 0x7f8f640b06a0 2023-10-11T09:42:08.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.569+0000 7f8f617fa700 1 -- 172.21.15.78:0/2222896105 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f8f58007470 con 0x7f8f640b06a0 2023-10-11T09:42:08.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.569+0000 7f8f617fa700 1 --2- 172.21.15.78:0/2222896105 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8f48065f00 0x7f8f480683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:08.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.569+0000 7f8f617fa700 1 -- 172.21.15.78:0/2222896105 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8f58083370 con 0x7f8f640b06a0 2023-10-11T09:42:08.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.569+0000 7f8f63fff700 1 --2- 172.21.15.78:0/2222896105 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8f48065f00 0x7f8f480683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:08.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.573+0000 7f8f63fff700 1 --2- 172.21.15.78:0/2222896105 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8f48065f00 0x7f8f480683b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8f500090b0 tx=0x7f8f5001c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:08.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.577+0000 7f8f617fa700 1 -- 172.21.15.78:0/2222896105 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f8f641e6990 con 0x7f8f640b06a0 2023-10-11T09:42:08.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.745+0000 7f8f69e20700 1 -- 172.21.15.78:0/2222896105 --> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8f64071e40 con 0x7f8f48065f00 2023-10-11T09:42:08.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.753+0000 7f8f617fa700 1 -- 172.21.15.78:0/2222896105 <== mgr.4101 v2:172.21.15.78:6832/35606 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308742 (secure 0 0 0) 0x7f8f64071e40 con 0x7f8f48065f00 2023-10-11T09:42:08.762 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:08.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.761+0000 7f8f69e20700 1 -- 172.21.15.78:0/2222896105 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8f48065f00 msgr2=0x7f8f480683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:08.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.761+0000 7f8f69e20700 1 --2- 172.21.15.78:0/2222896105 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8f48065f00 0x7f8f480683b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8f500090b0 tx=0x7f8f5001c040 comp rx=0 tx=0).stop 2023-10-11T09:42:08.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.761+0000 7f8f69e20700 1 -- 172.21.15.78:0/2222896105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f640b06a0 msgr2=0x7f8f64151b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:08.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.761+0000 7f8f69e20700 1 --2- 172.21.15.78:0/2222896105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f640b06a0 0x7f8f64151b50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8f5800cc80 tx=0x7f8f5800cf90 comp rx=0 tx=0).stop 2023-10-11T09:42:08.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.761+0000 7f8f69e20700 1 -- 172.21.15.78:0/2222896105 shutdown_connections 2023-10-11T09:42:08.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.761+0000 7f8f69e20700 1 --2- 172.21.15.78:0/2222896105 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8f48065f00 0x7f8f480683b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:08.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.761+0000 7f8f69e20700 1 --2- 172.21.15.78:0/2222896105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f6415d050 0x7f8f64071900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:08.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.761+0000 7f8f69e20700 1 --2- 172.21.15.78:0/2222896105 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8f640cc820 0x7f8f640713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:08.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.761+0000 7f8f69e20700 1 --2- 172.21.15.78:0/2222896105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f640b06a0 0x7f8f64151b50 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:08.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.761+0000 7f8f69e20700 1 -- 172.21.15.78:0/2222896105 >> 172.21.15.78:0/2222896105 conn(0x7f8f640cea60 msgr2=0x7f8f640c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:08.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.761+0000 7f8f69e20700 1 -- 172.21.15.78:0/2222896105 shutdown_connections 2023-10-11T09:42:08.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:08.761+0000 7f8f69e20700 1 -- 172.21.15.78:0/2222896105 wait complete. 2023-10-11T09:42:08.772 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2023-10-11T09:42:08.790 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2023-10-11T09:42:08.348784+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":184,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220136,"kb_used_data":4520,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754584,"statfs":{"total":773094113280,"available":772868694016,"internally_reserved":0,"allocated":4628480,"data_stored":2588564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":54,"apply_latency_ms":54,"commit_latency_ns":54000000,"apply_latency_ns":54000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.114637"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.134185+0000","last_change":"2023-10-11T09:41:59.251004+0000","last_active":"2023-10-11T09:42:02.134185+0000","last_peered":"2023-10-11T09:42:02.134185+0000","last_clean":"2023-10-11T09:42:02.134185+0000","last_became_active":"2023-10-11T09:41:59.250745+0000","last_became_peered":"2023-10-11T09:41:59.250745+0000","last_unstale":"2023-10-11T09:42:02.134185+0000","last_undegraded":"2023-10-11T09:42:02.134185+0000","last_fullsized":"2023-10-11T09:42:02.134185+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:32:44.448136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.135281+0000","last_change":"2023-10-11T09:41:59.258617+0000","last_active":"2023-10-11T09:42:02.135281+0000","last_peered":"2023-10-11T09:42:02.135281+0000","last_clean":"2023-10-11T09:42:02.135281+0000","last_became_active":"2023-10-11T09:41:59.258374+0000","last_became_peered":"2023-10-11T09:41:59.258374+0000","last_unstale":"2023-10-11T09:42:02.135281+0000","last_undegraded":"2023-10-11T09:42:02.135281+0000","last_fullsized":"2023-10-11T09:42:02.135281+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:46:05.674097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133528+0000","last_change":"2023-10-11T09:41:59.270985+0000","last_active":"2023-10-11T09:42:02.133528+0000","last_peered":"2023-10-11T09:42:02.133528+0000","last_clean":"2023-10-11T09:42:02.133528+0000","last_became_active":"2023-10-11T09:41:59.270588+0000","last_became_peered":"2023-10-11T09:41:59.270588+0000","last_unstale":"2023-10-11T09:42:02.133528+0000","last_undegraded":"2023-10-11T09:42:02.133528+0000","last_fullsized":"2023-10-11T09:42:02.133528+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:51:26.096609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.244926+0000","last_change":"2023-10-11T09:41:59.254605+0000","last_active":"2023-10-11T09:42:00.244926+0000","last_peered":"2023-10-11T09:42:00.244926+0000","last_clean":"2023-10-11T09:42:00.244926+0000","last_became_active":"2023-10-11T09:41:59.254130+0000","last_became_peered":"2023-10-11T09:41:59.254130+0000","last_unstale":"2023-10-11T09:42:00.244926+0000","last_undegraded":"2023-10-11T09:42:00.244926+0000","last_fullsized":"2023-10-11T09:42:00.244926+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:09:11.561455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.244764+0000","last_change":"2023-10-11T09:41:59.251753+0000","last_active":"2023-10-11T09:42:00.244764+0000","last_peered":"2023-10-11T09:42:00.244764+0000","last_clean":"2023-10-11T09:42:00.244764+0000","last_became_active":"2023-10-11T09:41:59.251460+0000","last_became_peered":"2023-10-11T09:41:59.251460+0000","last_unstale":"2023-10-11T09:42:00.244764+0000","last_undegraded":"2023-10-11T09:42:00.244764+0000","last_fullsized":"2023-10-11T09:42:00.244764+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:16:24.678764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.134098+0000","last_change":"2023-10-11T09:41:59.250567+0000","last_active":"2023-10-11T09:42:02.134098+0000","last_peered":"2023-10-11T09:42:02.134098+0000","last_clean":"2023-10-11T09:42:02.134098+0000","last_became_active":"2023-10-11T09:41:59.250202+0000","last_became_peered":"2023-10-11T09:41:59.250202+0000","last_unstale":"2023-10-11T09:42:02.134098+0000","last_undegraded":"2023-10-11T09:42:02.134098+0000","last_fullsized":"2023-10-11T09:42:02.134098+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:52:21.727787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.245752+0000","last_change":"2023-10-11T09:41:59.261239+0000","last_active":"2023-10-11T09:42:00.245752+0000","last_peered":"2023-10-11T09:42:00.245752+0000","last_clean":"2023-10-11T09:42:00.245752+0000","last_became_active":"2023-10-11T09:41:59.260991+0000","last_became_peered":"2023-10-11T09:41:59.260991+0000","last_unstale":"2023-10-11T09:42:00.245752+0000","last_undegraded":"2023-10-11T09:42:00.245752+0000","last_fullsized":"2023-10-11T09:42:00.245752+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:14:50.524508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.132362+0000","last_change":"2023-10-11T09:41:59.267670+0000","last_active":"2023-10-11T09:42:02.132362+0000","last_peered":"2023-10-11T09:42:02.132362+0000","last_clean":"2023-10-11T09:42:02.132362+0000","last_became_active":"2023-10-11T09:41:59.267410+0000","last_became_peered":"2023-10-11T09:41:59.267410+0000","last_unstale":"2023-10-11T09:42:02.132362+0000","last_undegraded":"2023-10-11T09:42:02.132362+0000","last_fullsized":"2023-10-11T09:42:02.132362+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:14:53.871518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133935+0000","last_change":"2023-10-11T09:41:59.271233+0000","last_active":"2023-10-11T09:42:02.133935+0000","last_peered":"2023-10-11T09:42:02.133935+0000","last_clean":"2023-10-11T09:42:02.133935+0000","last_became_active":"2023-10-11T09:41:59.270942+0000","last_became_peered":"2023-10-11T09:41:59.270942+0000","last_unstale":"2023-10-11T09:42:02.133935+0000","last_undegraded":"2023-10-11T09:42:02.133935+0000","last_fullsized":"2023-10-11T09:42:02.133935+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:46:05.674097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.247294+0000","last_change":"2023-10-11T09:41:59.276669+0000","last_active":"2023-10-11T09:42:00.247294+0000","last_peered":"2023-10-11T09:42:00.247294+0000","last_clean":"2023-10-11T09:42:00.247294+0000","last_became_active":"2023-10-11T09:41:59.276458+0000","last_became_peered":"2023-10-11T09:41:59.276458+0000","last_unstale":"2023-10-11T09:42:00.247294+0000","last_undegraded":"2023-10-11T09:42:00.247294+0000","last_fullsized":"2023-10-11T09:42:00.247294+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:29:15.375697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.247187+0000","last_change":"2023-10-11T09:41:59.270643+0000","last_active":"2023-10-11T09:42:00.247187+0000","last_peered":"2023-10-11T09:42:00.247187+0000","last_clean":"2023-10-11T09:42:00.247187+0000","last_became_active":"2023-10-11T09:41:59.270327+0000","last_became_peered":"2023-10-11T09:41:59.270327+0000","last_unstale":"2023-10-11T09:42:00.247187+0000","last_undegraded":"2023-10-11T09:42:00.247187+0000","last_fullsized":"2023-10-11T09:42:00.247187+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:46:05.674097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.134764+0000","last_change":"2023-10-11T09:41:59.261251+0000","last_active":"2023-10-11T09:42:02.134764+0000","last_peered":"2023-10-11T09:42:02.134764+0000","last_clean":"2023-10-11T09:42:02.134764+0000","last_became_active":"2023-10-11T09:41:59.260849+0000","last_became_peered":"2023-10-11T09:41:59.260849+0000","last_unstale":"2023-10-11T09:42:02.134764+0000","last_undegraded":"2023-10-11T09:42:02.134764+0000","last_fullsized":"2023-10-11T09:42:02.134764+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:29:09.854758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.246833+0000","last_change":"2023-10-11T09:41:59.276141+0000","last_active":"2023-10-11T09:42:00.246833+0000","last_peered":"2023-10-11T09:42:00.246833+0000","last_clean":"2023-10-11T09:42:00.246833+0000","last_became_active":"2023-10-11T09:41:59.275873+0000","last_became_peered":"2023-10-11T09:41:59.275873+0000","last_unstale":"2023-10-11T09:42:00.246833+0000","last_undegraded":"2023-10-11T09:42:00.246833+0000","last_fullsized":"2023-10-11T09:42:00.246833+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:06:13.791884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:01.632123+0000","last_change":"2023-10-11T09:41:59.258250+0000","last_active":"2023-10-11T09:42:01.632123+0000","last_peered":"2023-10-11T09:42:01.632123+0000","last_clean":"2023-10-11T09:42:01.632123+0000","last_became_active":"2023-10-11T09:41:59.257976+0000","last_became_peered":"2023-10-11T09:41:59.257976+0000","last_unstale":"2023-10-11T09:42:01.632123+0000","last_undegraded":"2023-10-11T09:42:01.632123+0000","last_fullsized":"2023-10-11T09:42:01.632123+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:46:05.674097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.245867+0000","last_change":"2023-10-11T09:41:59.268687+0000","last_active":"2023-10-11T09:42:00.245867+0000","last_peered":"2023-10-11T09:42:00.245867+0000","last_clean":"2023-10-11T09:42:00.245867+0000","last_became_active":"2023-10-11T09:41:59.268386+0000","last_became_peered":"2023-10-11T09:41:59.268386+0000","last_unstale":"2023-10-11T09:42:00.245867+0000","last_undegraded":"2023-10-11T09:42:00.245867+0000","last_fullsized":"2023-10-11T09:42:00.245867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:29:09.854758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.245820+0000","last_change":"2023-10-11T09:41:59.276367+0000","last_active":"2023-10-11T09:42:00.245820+0000","last_peered":"2023-10-11T09:42:00.245820+0000","last_clean":"2023-10-11T09:42:00.245820+0000","last_became_active":"2023-10-11T09:41:59.275721+0000","last_became_peered":"2023-10-11T09:41:59.275721+0000","last_unstale":"2023-10-11T09:42:00.245820+0000","last_undegraded":"2023-10-11T09:42:00.245820+0000","last_fullsized":"2023-10-11T09:42:00.245820+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:32:44.448136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.246578+0000","last_change":"2023-10-11T09:41:59.272244+0000","last_active":"2023-10-11T09:42:00.246578+0000","last_peered":"2023-10-11T09:42:00.246578+0000","last_clean":"2023-10-11T09:42:00.246578+0000","last_became_active":"2023-10-11T09:41:59.271864+0000","last_became_peered":"2023-10-11T09:41:59.271864+0000","last_unstale":"2023-10-11T09:42:00.246578+0000","last_undegraded":"2023-10-11T09:42:00.246578+0000","last_fullsized":"2023-10-11T09:42:00.246578+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:51:26.096609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133551+0000","last_change":"2023-10-11T09:41:59.257454+0000","last_active":"2023-10-11T09:42:02.133551+0000","last_peered":"2023-10-11T09:42:02.133551+0000","last_clean":"2023-10-11T09:42:02.133551+0000","last_became_active":"2023-10-11T09:41:59.257169+0000","last_became_peered":"2023-10-11T09:41:59.257169+0000","last_unstale":"2023-10-11T09:42:02.133551+0000","last_undegraded":"2023-10-11T09:42:02.133551+0000","last_fullsized":"2023-10-11T09:42:02.133551+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:50:35.471027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.245210+0000","last_change":"2023-10-11T09:41:59.247417+0000","last_active":"2023-10-11T09:42:00.245210+0000","last_peered":"2023-10-11T09:42:00.245210+0000","last_clean":"2023-10-11T09:42:00.245210+0000","last_became_active":"2023-10-11T09:41:59.247155+0000","last_became_peered":"2023-10-11T09:41:59.247155+0000","last_unstale":"2023-10-11T09:42:00.245210+0000","last_undegraded":"2023-10-11T09:42:00.245210+0000","last_fullsized":"2023-10-11T09:42:00.245210+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:18:57.154610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133410+0000","last_change":"2023-10-11T09:41:59.258025+0000","last_active":"2023-10-11T09:42:02.133410+0000","last_peered":"2023-10-11T09:42:02.133410+0000","last_clean":"2023-10-11T09:42:02.133410+0000","last_became_active":"2023-10-11T09:41:59.257530+0000","last_became_peered":"2023-10-11T09:41:59.257530+0000","last_unstale":"2023-10-11T09:42:02.133410+0000","last_undegraded":"2023-10-11T09:42:02.133410+0000","last_fullsized":"2023-10-11T09:42:02.133410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:09:11.561455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.246676+0000","last_change":"2023-10-11T09:41:59.271200+0000","last_active":"2023-10-11T09:42:00.246676+0000","last_peered":"2023-10-11T09:42:00.246676+0000","last_clean":"2023-10-11T09:42:00.246676+0000","last_became_active":"2023-10-11T09:41:59.270934+0000","last_became_peered":"2023-10-11T09:41:59.270934+0000","last_unstale":"2023-10-11T09:42:00.246676+0000","last_undegraded":"2023-10-11T09:42:00.246676+0000","last_fullsized":"2023-10-11T09:42:00.246676+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:03:35.795099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:01.632676+0000","last_change":"2023-10-11T09:41:59.253265+0000","last_active":"2023-10-11T09:42:01.632676+0000","last_peered":"2023-10-11T09:42:01.632676+0000","last_clean":"2023-10-11T09:42:01.632676+0000","last_became_active":"2023-10-11T09:41:59.252970+0000","last_became_peered":"2023-10-11T09:41:59.252970+0000","last_unstale":"2023-10-11T09:42:01.632676+0000","last_undegraded":"2023-10-11T09:42:01.632676+0000","last_fullsized":"2023-10-11T09:42:01.632676+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:06:13.791884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.246079+0000","last_change":"2023-10-11T09:41:59.275092+0000","last_active":"2023-10-11T09:42:00.246079+0000","last_peered":"2023-10-11T09:42:00.246079+0000","last_clean":"2023-10-11T09:42:00.246079+0000","last_became_active":"2023-10-11T09:41:59.274296+0000","last_became_peered":"2023-10-11T09:41:59.274296+0000","last_unstale":"2023-10-11T09:42:00.246079+0000","last_undegraded":"2023-10-11T09:42:00.246079+0000","last_fullsized":"2023-10-11T09:42:00.246079+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:07:46.727754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.244267+0000","last_change":"2023-10-11T09:41:59.266457+0000","last_active":"2023-10-11T09:42:00.244267+0000","last_peered":"2023-10-11T09:42:00.244267+0000","last_clean":"2023-10-11T09:42:00.244267+0000","last_became_active":"2023-10-11T09:41:59.266177+0000","last_became_peered":"2023-10-11T09:41:59.266177+0000","last_unstale":"2023-10-11T09:42:00.244267+0000","last_undegraded":"2023-10-11T09:42:00.244267+0000","last_fullsized":"2023-10-11T09:42:00.244267+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:24:35.472967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133671+0000","last_change":"2023-10-11T09:41:59.256922+0000","last_active":"2023-10-11T09:42:02.133671+0000","last_peered":"2023-10-11T09:42:02.133671+0000","last_clean":"2023-10-11T09:42:02.133671+0000","last_became_active":"2023-10-11T09:41:59.256620+0000","last_became_peered":"2023-10-11T09:41:59.256620+0000","last_unstale":"2023-10-11T09:42:02.133671+0000","last_undegraded":"2023-10-11T09:42:02.133671+0000","last_fullsized":"2023-10-11T09:42:02.133671+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:48:03.493737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.134430+0000","last_change":"2023-10-11T09:41:59.267830+0000","last_active":"2023-10-11T09:42:02.134430+0000","last_peered":"2023-10-11T09:42:02.134430+0000","last_clean":"2023-10-11T09:42:02.134430+0000","last_became_active":"2023-10-11T09:41:59.267543+0000","last_became_peered":"2023-10-11T09:41:59.267543+0000","last_unstale":"2023-10-11T09:42:02.134430+0000","last_undegraded":"2023-10-11T09:42:02.134430+0000","last_fullsized":"2023-10-11T09:42:02.134430+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:00:42.620537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133519+0000","last_change":"2023-10-11T09:41:59.259928+0000","last_active":"2023-10-11T09:42:02.133519+0000","last_peered":"2023-10-11T09:42:02.133519+0000","last_clean":"2023-10-11T09:42:02.133519+0000","last_became_active":"2023-10-11T09:41:59.259642+0000","last_became_peered":"2023-10-11T09:41:59.259642+0000","last_unstale":"2023-10-11T09:42:02.133519+0000","last_undegraded":"2023-10-11T09:42:02.133519+0000","last_fullsized":"2023-10-11T09:42:02.133519+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:05:44.896062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.247105+0000","last_change":"2023-10-11T09:41:59.262115+0000","last_active":"2023-10-11T09:42:00.247105+0000","last_peered":"2023-10-11T09:42:00.247105+0000","last_clean":"2023-10-11T09:42:00.247105+0000","last_became_active":"2023-10-11T09:41:59.261752+0000","last_became_peered":"2023-10-11T09:41:59.261752+0000","last_unstale":"2023-10-11T09:42:00.247105+0000","last_undegraded":"2023-10-11T09:42:00.247105+0000","last_fullsized":"2023-10-11T09:42:00.247105+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:55:01.198068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133191+0000","last_change":"2023-10-11T09:41:59.261554+0000","last_active":"2023-10-11T09:42:02.133191+0000","last_peered":"2023-10-11T09:42:02.133191+0000","last_clean":"2023-10-11T09:42:02.133191+0000","last_became_active":"2023-10-11T09:41:59.261064+0000","last_became_peered":"2023-10-11T09:41:59.261064+0000","last_unstale":"2023-10-11T09:42:02.133191+0000","last_undegraded":"2023-10-11T09:42:02.133191+0000","last_fullsized":"2023-10-11T09:42:02.133191+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:24:00.485790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133455+0000","last_change":"2023-10-11T09:41:59.256304+0000","last_active":"2023-10-11T09:42:02.133455+0000","last_peered":"2023-10-11T09:42:02.133455+0000","last_clean":"2023-10-11T09:42:02.133455+0000","last_became_active":"2023-10-11T09:41:59.255941+0000","last_became_peered":"2023-10-11T09:41:59.255941+0000","last_unstale":"2023-10-11T09:42:02.133455+0000","last_undegraded":"2023-10-11T09:42:02.133455+0000","last_fullsized":"2023-10-11T09:42:02.133455+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:16:24.678764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.246243+0000","last_change":"2023-10-11T09:41:59.275647+0000","last_active":"2023-10-11T09:42:00.246243+0000","last_peered":"2023-10-11T09:42:00.246243+0000","last_clean":"2023-10-11T09:42:00.246243+0000","last_became_active":"2023-10-11T09:41:59.275328+0000","last_became_peered":"2023-10-11T09:41:59.275328+0000","last_unstale":"2023-10-11T09:42:00.246243+0000","last_undegraded":"2023-10-11T09:42:00.246243+0000","last_fullsized":"2023-10-11T09:42:00.246243+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:49:29.335655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133147+0000","last_change":"2023-10-11T09:41:59.261148+0000","last_active":"2023-10-11T09:42:02.133147+0000","last_peered":"2023-10-11T09:42:02.133147+0000","last_clean":"2023-10-11T09:42:02.133147+0000","last_became_active":"2023-10-11T09:41:59.260863+0000","last_became_peered":"2023-10-11T09:41:59.260863+0000","last_unstale":"2023-10-11T09:42:02.133147+0000","last_undegraded":"2023-10-11T09:42:02.133147+0000","last_fullsized":"2023-10-11T09:42:02.133147+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:58:22.628540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.246187+0000","last_change":"2023-10-11T09:41:59.268534+0000","last_active":"2023-10-11T09:42:00.246187+0000","last_peered":"2023-10-11T09:42:00.246187+0000","last_clean":"2023-10-11T09:42:00.246187+0000","last_became_active":"2023-10-11T09:41:59.268260+0000","last_became_peered":"2023-10-11T09:41:59.268260+0000","last_unstale":"2023-10-11T09:42:00.246187+0000","last_undegraded":"2023-10-11T09:42:00.246187+0000","last_fullsized":"2023-10-11T09:42:00.246187+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:46:05.674097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"36'4","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.278577+0000","last_change":"2023-10-11T09:41:58.781253+0000","last_active":"2023-10-11T09:41:59.278577+0000","last_peered":"2023-10-11T09:41:59.278577+0000","last_clean":"2023-10-11T09:41:59.278577+0000","last_became_active":"2023-10-11T09:41:58.780932+0000","last_became_peered":"2023-10-11T09:41:58.780932+0000","last_unstale":"2023-10-11T09:41:59.278577+0000","last_undegraded":"2023-10-11T09:41:59.278577+0000","last_fullsized":"2023-10-11T09:41:59.278577+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:34:50.579527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.135085+0000","last_change":"2023-10-11T09:41:59.270949+0000","last_active":"2023-10-11T09:42:02.135085+0000","last_peered":"2023-10-11T09:42:02.135085+0000","last_clean":"2023-10-11T09:42:02.135085+0000","last_became_active":"2023-10-11T09:41:59.270558+0000","last_became_peered":"2023-10-11T09:41:59.270558+0000","last_unstale":"2023-10-11T09:42:02.135085+0000","last_undegraded":"2023-10-11T09:42:02.135085+0000","last_fullsized":"2023-10-11T09:42:02.135085+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:07:43.684811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"36'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.278298+0000","last_change":"2023-10-11T09:41:58.773997+0000","last_active":"2023-10-11T09:41:59.278298+0000","last_peered":"2023-10-11T09:41:59.278298+0000","last_clean":"2023-10-11T09:41:59.278298+0000","last_became_active":"2023-10-11T09:41:58.773582+0000","last_became_peered":"2023-10-11T09:41:58.773582+0000","last_unstale":"2023-10-11T09:41:59.278298+0000","last_undegraded":"2023-10-11T09:41:59.278298+0000","last_fullsized":"2023-10-11T09:41:59.278298+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:00:41.605405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:01.632879+0000","last_change":"2023-10-11T09:41:59.264983+0000","last_active":"2023-10-11T09:42:01.632879+0000","last_peered":"2023-10-11T09:42:01.632879+0000","last_clean":"2023-10-11T09:42:01.632879+0000","last_became_active":"2023-10-11T09:41:59.264662+0000","last_became_peered":"2023-10-11T09:41:59.264662+0000","last_unstale":"2023-10-11T09:42:01.632879+0000","last_undegraded":"2023-10-11T09:42:01.632879+0000","last_fullsized":"2023-10-11T09:42:01.632879+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:51:26.096609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"36'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.272941+0000","last_change":"2023-10-11T09:41:58.775695+0000","last_active":"2023-10-11T09:41:59.272941+0000","last_peered":"2023-10-11T09:41:59.272941+0000","last_clean":"2023-10-11T09:41:59.272941+0000","last_became_active":"2023-10-11T09:41:58.775435+0000","last_became_peered":"2023-10-11T09:41:58.775435+0000","last_unstale":"2023-10-11T09:41:59.272941+0000","last_undegraded":"2023-10-11T09:41:59.272941+0000","last_fullsized":"2023-10-11T09:41:59.272941+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:16:23.663632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.134236+0000","last_change":"2023-10-11T09:41:59.259129+0000","last_active":"2023-10-11T09:42:02.134236+0000","last_peered":"2023-10-11T09:42:02.134236+0000","last_clean":"2023-10-11T09:42:02.134236+0000","last_became_active":"2023-10-11T09:41:59.258907+0000","last_became_peered":"2023-10-11T09:41:59.258907+0000","last_unstale":"2023-10-11T09:42:02.134236+0000","last_undegraded":"2023-10-11T09:42:02.134236+0000","last_fullsized":"2023-10-11T09:42:02.134236+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:07:46.727754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"36'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.273479+0000","last_change":"2023-10-11T09:41:58.777078+0000","last_active":"2023-10-11T09:41:59.273479+0000","last_peered":"2023-10-11T09:41:59.273479+0000","last_clean":"2023-10-11T09:41:59.273479+0000","last_became_active":"2023-10-11T09:41:58.776760+0000","last_became_peered":"2023-10-11T09:41:58.776760+0000","last_unstale":"2023-10-11T09:41:59.273479+0000","last_undegraded":"2023-10-11T09:41:59.273479+0000","last_fullsized":"2023-10-11T09:41:59.273479+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:30:07.413635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133334+0000","last_change":"2023-10-11T09:41:59.249575+0000","last_active":"2023-10-11T09:42:02.133334+0000","last_peered":"2023-10-11T09:42:02.133334+0000","last_clean":"2023-10-11T09:42:02.133334+0000","last_became_active":"2023-10-11T09:41:59.249320+0000","last_became_peered":"2023-10-11T09:41:59.249320+0000","last_unstale":"2023-10-11T09:42:02.133334+0000","last_undegraded":"2023-10-11T09:42:02.133334+0000","last_fullsized":"2023-10-11T09:42:02.133334+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:29:15.375697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"36'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.273485+0000","last_change":"2023-10-11T09:41:58.780188+0000","last_active":"2023-10-11T09:41:59.273485+0000","last_peered":"2023-10-11T09:41:59.273485+0000","last_clean":"2023-10-11T09:41:59.273485+0000","last_became_active":"2023-10-11T09:41:58.779846+0000","last_became_peered":"2023-10-11T09:41:58.779846+0000","last_unstale":"2023-10-11T09:41:59.273485+0000","last_undegraded":"2023-10-11T09:41:59.273485+0000","last_fullsized":"2023-10-11T09:41:59.273485+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:52:20.712655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.245928+0000","last_change":"2023-10-11T09:41:59.258246+0000","last_active":"2023-10-11T09:42:00.245928+0000","last_peered":"2023-10-11T09:42:00.245928+0000","last_clean":"2023-10-11T09:42:00.245928+0000","last_became_active":"2023-10-11T09:41:59.257935+0000","last_became_peered":"2023-10-11T09:41:59.257935+0000","last_unstale":"2023-10-11T09:42:00.245928+0000","last_undegraded":"2023-10-11T09:42:00.245928+0000","last_fullsized":"2023-10-11T09:42:00.245928+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:27:48.375035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"36'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.275029+0000","last_change":"2023-10-11T09:41:58.781289+0000","last_active":"2023-10-11T09:41:59.275029+0000","last_peered":"2023-10-11T09:41:59.275029+0000","last_clean":"2023-10-11T09:41:59.275029+0000","last_became_active":"2023-10-11T09:41:58.781013+0000","last_became_peered":"2023-10-11T09:41:58.781013+0000","last_unstale":"2023-10-11T09:41:59.275029+0000","last_undegraded":"2023-10-11T09:41:59.275029+0000","last_fullsized":"2023-10-11T09:41:59.275029+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:50:34.455895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:01.632527+0000","last_change":"2023-10-11T09:41:59.262808+0000","last_active":"2023-10-11T09:42:01.632527+0000","last_peered":"2023-10-11T09:42:01.632527+0000","last_clean":"2023-10-11T09:42:01.632527+0000","last_became_active":"2023-10-11T09:41:59.262515+0000","last_became_peered":"2023-10-11T09:41:59.262515+0000","last_unstale":"2023-10-11T09:42:01.632527+0000","last_undegraded":"2023-10-11T09:42:01.632527+0000","last_fullsized":"2023-10-11T09:42:01.632527+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:29:15.375697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.242698+0000","last_change":"2023-10-11T09:41:58.777202+0000","last_active":"2023-10-11T09:41:59.242698+0000","last_peered":"2023-10-11T09:41:59.242698+0000","last_clean":"2023-10-11T09:41:59.242698+0000","last_became_active":"2023-10-11T09:41:58.776863+0000","last_became_peered":"2023-10-11T09:41:58.776863+0000","last_unstale":"2023-10-11T09:41:59.242698+0000","last_undegraded":"2023-10-11T09:41:59.242698+0000","last_fullsized":"2023-10-11T09:41:59.242698+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:25:01.903933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133295+0000","last_change":"2023-10-11T09:41:59.275572+0000","last_active":"2023-10-11T09:42:02.133295+0000","last_peered":"2023-10-11T09:42:02.133295+0000","last_clean":"2023-10-11T09:42:02.133295+0000","last_became_active":"2023-10-11T09:41:59.275159+0000","last_became_peered":"2023-10-11T09:41:59.275159+0000","last_unstale":"2023-10-11T09:42:02.133295+0000","last_undegraded":"2023-10-11T09:42:02.133295+0000","last_fullsized":"2023-10-11T09:42:02.133295+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:29:09.854758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.242721+0000","last_change":"2023-10-11T09:41:58.774053+0000","last_active":"2023-10-11T09:41:59.242721+0000","last_peered":"2023-10-11T09:41:59.242721+0000","last_clean":"2023-10-11T09:41:59.242721+0000","last_became_active":"2023-10-11T09:41:58.773761+0000","last_became_peered":"2023-10-11T09:41:58.773761+0000","last_unstale":"2023-10-11T09:41:59.242721+0000","last_undegraded":"2023-10-11T09:41:59.242721+0000","last_fullsized":"2023-10-11T09:41:59.242721+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:00:41.605405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.245053+0000","last_change":"2023-10-11T09:41:59.254445+0000","last_active":"2023-10-11T09:42:00.245053+0000","last_peered":"2023-10-11T09:42:00.245053+0000","last_clean":"2023-10-11T09:42:00.245053+0000","last_became_active":"2023-10-11T09:41:59.253764+0000","last_became_peered":"2023-10-11T09:41:59.253764+0000","last_unstale":"2023-10-11T09:42:00.245053+0000","last_undegraded":"2023-10-11T09:42:00.245053+0000","last_fullsized":"2023-10-11T09:42:00.245053+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:50:35.471027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.245062+0000","last_change":"2023-10-11T09:41:58.775649+0000","last_active":"2023-10-11T09:41:59.245062+0000","last_peered":"2023-10-11T09:41:59.245062+0000","last_clean":"2023-10-11T09:41:59.245062+0000","last_became_active":"2023-10-11T09:41:58.773119+0000","last_became_peered":"2023-10-11T09:41:58.773119+0000","last_unstale":"2023-10-11T09:41:59.245062+0000","last_undegraded":"2023-10-11T09:41:59.245062+0000","last_fullsized":"2023-10-11T09:41:59.245062+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:48:02.478605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.246299+0000","last_change":"2023-10-11T09:41:59.266483+0000","last_active":"2023-10-11T09:42:00.246299+0000","last_peered":"2023-10-11T09:42:00.246299+0000","last_clean":"2023-10-11T09:42:00.246299+0000","last_became_active":"2023-10-11T09:41:59.266228+0000","last_became_peered":"2023-10-11T09:41:59.266228+0000","last_unstale":"2023-10-11T09:42:00.246299+0000","last_undegraded":"2023-10-11T09:42:00.246299+0000","last_fullsized":"2023-10-11T09:42:00.246299+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:14:53.871518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.256436+0000","last_change":"2023-10-11T09:41:58.780367+0000","last_active":"2023-10-11T09:41:59.256436+0000","last_peered":"2023-10-11T09:41:59.256436+0000","last_clean":"2023-10-11T09:41:59.256436+0000","last_became_active":"2023-10-11T09:41:58.779355+0000","last_became_peered":"2023-10-11T09:41:58.779355+0000","last_unstale":"2023-10-11T09:41:59.256436+0000","last_undegraded":"2023-10-11T09:41:59.256436+0000","last_fullsized":"2023-10-11T09:41:59.256436+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:24:11.694544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.134680+0000","last_change":"2023-10-11T09:41:59.274318+0000","last_active":"2023-10-11T09:42:02.134680+0000","last_peered":"2023-10-11T09:42:02.134680+0000","last_clean":"2023-10-11T09:42:02.134680+0000","last_became_active":"2023-10-11T09:41:59.274070+0000","last_became_peered":"2023-10-11T09:41:59.274070+0000","last_unstale":"2023-10-11T09:42:02.134680+0000","last_undegraded":"2023-10-11T09:42:02.134680+0000","last_fullsized":"2023-10-11T09:42:02.134680+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:32:44.448136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"36'3","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.273507+0000","last_change":"2023-10-11T09:41:58.776724+0000","last_active":"2023-10-11T09:41:59.273507+0000","last_peered":"2023-10-11T09:41:59.273507+0000","last_clean":"2023-10-11T09:41:59.273507+0000","last_became_active":"2023-10-11T09:41:58.776359+0000","last_became_peered":"2023-10-11T09:41:58.776359+0000","last_unstale":"2023-10-11T09:41:59.273507+0000","last_undegraded":"2023-10-11T09:41:59.273507+0000","last_fullsized":"2023-10-11T09:41:59.273507+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:55:29.823667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:01.632819+0000","last_change":"2023-10-11T09:41:59.252336+0000","last_active":"2023-10-11T09:42:01.632819+0000","last_peered":"2023-10-11T09:42:01.632819+0000","last_clean":"2023-10-11T09:42:01.632819+0000","last_became_active":"2023-10-11T09:41:59.252084+0000","last_became_peered":"2023-10-11T09:41:59.252084+0000","last_unstale":"2023-10-11T09:42:01.632819+0000","last_undegraded":"2023-10-11T09:42:01.632819+0000","last_fullsized":"2023-10-11T09:42:01.632819+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:29:09.854758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"36'6","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.281790+0000","last_change":"2023-10-11T09:41:58.775314+0000","last_active":"2023-10-11T09:41:59.281790+0000","last_peered":"2023-10-11T09:41:59.281790+0000","last_clean":"2023-10-11T09:41:59.281790+0000","last_became_active":"2023-10-11T09:41:58.775000+0000","last_became_peered":"2023-10-11T09:41:58.775000+0000","last_unstale":"2023-10-11T09:41:59.281790+0000","last_undegraded":"2023-10-11T09:41:59.281790+0000","last_fullsized":"2023-10-11T09:41:59.281790+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:23:59.470658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133578+0000","last_change":"2023-10-11T09:41:59.267987+0000","last_active":"2023-10-11T09:42:02.133578+0000","last_peered":"2023-10-11T09:42:02.133578+0000","last_clean":"2023-10-11T09:42:02.133578+0000","last_became_active":"2023-10-11T09:41:59.267705+0000","last_became_peered":"2023-10-11T09:41:59.267705+0000","last_unstale":"2023-10-11T09:42:02.133578+0000","last_undegraded":"2023-10-11T09:42:02.133578+0000","last_fullsized":"2023-10-11T09:42:02.133578+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:34:02.831174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"36'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.256670+0000","last_change":"2023-10-11T09:41:58.776379+0000","last_active":"2023-10-11T09:41:59.256670+0000","last_peered":"2023-10-11T09:41:59.256670+0000","last_clean":"2023-10-11T09:41:59.256670+0000","last_became_active":"2023-10-11T09:41:58.776071+0000","last_became_peered":"2023-10-11T09:41:58.776071+0000","last_unstale":"2023-10-11T09:41:59.256670+0000","last_undegraded":"2023-10-11T09:41:59.256670+0000","last_fullsized":"2023-10-11T09:41:59.256670+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:24:11.694544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.132753+0000","last_change":"2023-10-11T09:41:59.250221+0000","last_active":"2023-10-11T09:42:02.132753+0000","last_peered":"2023-10-11T09:42:02.132753+0000","last_clean":"2023-10-11T09:42:02.132753+0000","last_became_active":"2023-10-11T09:41:59.249523+0000","last_became_peered":"2023-10-11T09:41:59.249523+0000","last_unstale":"2023-10-11T09:42:02.132753+0000","last_undegraded":"2023-10-11T09:42:02.132753+0000","last_fullsized":"2023-10-11T09:42:02.132753+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:49:29.335655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.242895+0000","last_change":"2023-10-11T09:41:58.776351+0000","last_active":"2023-10-11T09:41:59.242895+0000","last_peered":"2023-10-11T09:41:59.242895+0000","last_clean":"2023-10-11T09:41:59.242895+0000","last_became_active":"2023-10-11T09:41:58.776076+0000","last_became_peered":"2023-10-11T09:41:58.776076+0000","last_unstale":"2023-10-11T09:41:59.242895+0000","last_undegraded":"2023-10-11T09:41:59.242895+0000","last_fullsized":"2023-10-11T09:41:59.242895+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:18:56.139478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.246103+0000","last_change":"2023-10-11T09:41:59.266545+0000","last_active":"2023-10-11T09:42:00.246103+0000","last_peered":"2023-10-11T09:42:00.246103+0000","last_clean":"2023-10-11T09:42:00.246103+0000","last_became_active":"2023-10-11T09:41:59.266181+0000","last_became_peered":"2023-10-11T09:41:59.266181+0000","last_unstale":"2023-10-11T09:42:00.246103+0000","last_undegraded":"2023-10-11T09:42:00.246103+0000","last_fullsized":"2023-10-11T09:42:00.246103+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:46:25.915286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.255044+0000","last_change":"2023-10-11T09:41:58.781510+0000","last_active":"2023-10-11T09:41:59.255044+0000","last_peered":"2023-10-11T09:41:59.255044+0000","last_clean":"2023-10-11T09:41:59.255044+0000","last_became_active":"2023-10-11T09:41:58.781083+0000","last_became_peered":"2023-10-11T09:41:58.781083+0000","last_unstale":"2023-10-11T09:41:59.255044+0000","last_undegraded":"2023-10-11T09:41:59.255044+0000","last_fullsized":"2023-10-11T09:41:59.255044+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:02:09.585130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133419+0000","last_change":"2023-10-11T09:41:59.250127+0000","last_active":"2023-10-11T09:42:02.133419+0000","last_peered":"2023-10-11T09:42:02.133419+0000","last_clean":"2023-10-11T09:42:02.133419+0000","last_became_active":"2023-10-11T09:41:59.249850+0000","last_became_peered":"2023-10-11T09:41:59.249850+0000","last_unstale":"2023-10-11T09:42:02.133419+0000","last_undegraded":"2023-10-11T09:42:02.133419+0000","last_fullsized":"2023-10-11T09:42:02.133419+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:29:15.375697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"36'8","reported_seq":19,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.276220+0000","last_change":"2023-10-11T09:41:58.782484+0000","last_active":"2023-10-11T09:41:59.276220+0000","last_peered":"2023-10-11T09:41:59.276220+0000","last_clean":"2023-10-11T09:41:59.276220+0000","last_became_active":"2023-10-11T09:41:58.781980+0000","last_became_peered":"2023-10-11T09:41:58.781980+0000","last_unstale":"2023-10-11T09:41:59.276220+0000","last_undegraded":"2023-10-11T09:41:59.276220+0000","last_fullsized":"2023-10-11T09:41:59.276220+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:05:43.880930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.134151+0000","last_change":"2023-10-11T09:41:59.257871+0000","last_active":"2023-10-11T09:42:02.134151+0000","last_peered":"2023-10-11T09:42:02.134151+0000","last_clean":"2023-10-11T09:42:02.134151+0000","last_became_active":"2023-10-11T09:41:59.257622+0000","last_became_peered":"2023-10-11T09:41:59.257622+0000","last_unstale":"2023-10-11T09:42:02.134151+0000","last_undegraded":"2023-10-11T09:42:02.134151+0000","last_fullsized":"2023-10-11T09:42:02.134151+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:34:02.831174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"36'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.278482+0000","last_change":"2023-10-11T09:41:58.782590+0000","last_active":"2023-10-11T09:41:59.278482+0000","last_peered":"2023-10-11T09:41:59.278482+0000","last_clean":"2023-10-11T09:41:59.278482+0000","last_became_active":"2023-10-11T09:41:58.782326+0000","last_became_peered":"2023-10-11T09:41:58.782326+0000","last_unstale":"2023-10-11T09:41:59.278482+0000","last_undegraded":"2023-10-11T09:41:59.278482+0000","last_fullsized":"2023-10-11T09:41:59.278482+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:11:34.532326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.247230+0000","last_change":"2023-10-11T09:41:59.270677+0000","last_active":"2023-10-11T09:42:00.247230+0000","last_peered":"2023-10-11T09:42:00.247230+0000","last_clean":"2023-10-11T09:42:00.247230+0000","last_became_active":"2023-10-11T09:41:59.270348+0000","last_became_peered":"2023-10-11T09:41:59.270348+0000","last_unstale":"2023-10-11T09:42:00.247230+0000","last_undegraded":"2023-10-11T09:42:00.247230+0000","last_fullsized":"2023-10-11T09:42:00.247230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:34:02.831174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"36'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.273284+0000","last_change":"2023-10-11T09:41:58.778144+0000","last_active":"2023-10-11T09:41:59.273284+0000","last_peered":"2023-10-11T09:41:59.273284+0000","last_clean":"2023-10-11T09:41:59.273284+0000","last_became_active":"2023-10-11T09:41:58.777788+0000","last_became_peered":"2023-10-11T09:41:58.777788+0000","last_unstale":"2023-10-11T09:41:59.273284+0000","last_undegraded":"2023-10-11T09:41:59.273284+0000","last_fullsized":"2023-10-11T09:41:59.273284+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:42:13.962137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:01.632636+0000","last_change":"2023-10-11T09:41:59.251031+0000","last_active":"2023-10-11T09:42:01.632636+0000","last_peered":"2023-10-11T09:42:01.632636+0000","last_clean":"2023-10-11T09:42:01.632636+0000","last_became_active":"2023-10-11T09:41:59.250626+0000","last_became_peered":"2023-10-11T09:41:59.250626+0000","last_unstale":"2023-10-11T09:42:01.632636+0000","last_undegraded":"2023-10-11T09:42:01.632636+0000","last_fullsized":"2023-10-11T09:42:01.632636+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:32:44.448136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"36'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.274594+0000","last_change":"2023-10-11T09:41:58.773947+0000","last_active":"2023-10-11T09:41:59.274594+0000","last_peered":"2023-10-11T09:41:59.274594+0000","last_clean":"2023-10-11T09:41:59.274594+0000","last_became_active":"2023-10-11T09:41:58.773642+0000","last_became_peered":"2023-10-11T09:41:58.773642+0000","last_unstale":"2023-10-11T09:41:59.274594+0000","last_undegraded":"2023-10-11T09:41:59.274594+0000","last_fullsized":"2023-10-11T09:41:59.274594+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:49:37.959487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.245415+0000","last_change":"2023-10-11T09:41:59.252857+0000","last_active":"2023-10-11T09:42:00.245415+0000","last_peered":"2023-10-11T09:42:00.245415+0000","last_clean":"2023-10-11T09:42:00.245415+0000","last_became_active":"2023-10-11T09:41:59.252603+0000","last_became_peered":"2023-10-11T09:41:59.252603+0000","last_unstale":"2023-10-11T09:42:00.245415+0000","last_undegraded":"2023-10-11T09:42:00.245415+0000","last_fullsized":"2023-10-11T09:42:00.245415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:35:44.282265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.242457+0000","last_change":"2023-10-11T09:41:58.775051+0000","last_active":"2023-10-11T09:41:59.242457+0000","last_peered":"2023-10-11T09:41:59.242457+0000","last_clean":"2023-10-11T09:41:59.242457+0000","last_became_active":"2023-10-11T09:41:58.774781+0000","last_became_peered":"2023-10-11T09:41:58.774781+0000","last_unstale":"2023-10-11T09:41:59.242457+0000","last_undegraded":"2023-10-11T09:41:59.242457+0000","last_fullsized":"2023-10-11T09:41:59.242457+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:35:43.267132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.245933+0000","last_change":"2023-10-11T09:41:59.276846+0000","last_active":"2023-10-11T09:42:00.245933+0000","last_peered":"2023-10-11T09:42:00.245933+0000","last_clean":"2023-10-11T09:42:00.245933+0000","last_became_active":"2023-10-11T09:41:59.276042+0000","last_became_peered":"2023-10-11T09:41:59.276042+0000","last_unstale":"2023-10-11T09:42:00.245933+0000","last_undegraded":"2023-10-11T09:42:00.245933+0000","last_fullsized":"2023-10-11T09:42:00.245933+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:07:13.635301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"36'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.266966+0000","last_change":"2023-10-11T09:41:58.777266+0000","last_active":"2023-10-11T09:41:59.266966+0000","last_peered":"2023-10-11T09:41:59.266966+0000","last_clean":"2023-10-11T09:41:59.266966+0000","last_became_active":"2023-10-11T09:41:58.776620+0000","last_became_peered":"2023-10-11T09:41:58.776620+0000","last_unstale":"2023-10-11T09:41:59.266966+0000","last_undegraded":"2023-10-11T09:41:59.266966+0000","last_fullsized":"2023-10-11T09:41:59.266966+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:55:46.983239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.132993+0000","last_change":"2023-10-11T09:41:59.276325+0000","last_active":"2023-10-11T09:42:02.132993+0000","last_peered":"2023-10-11T09:42:02.132993+0000","last_clean":"2023-10-11T09:42:02.132993+0000","last_became_active":"2023-10-11T09:41:59.275987+0000","last_became_peered":"2023-10-11T09:41:59.275987+0000","last_unstale":"2023-10-11T09:42:02.132993+0000","last_undegraded":"2023-10-11T09:42:02.132993+0000","last_fullsized":"2023-10-11T09:42:02.132993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:14:53.871518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.234885+0000","last_change":"2023-10-11T09:41:58.779536+0000","last_active":"2023-10-11T09:41:59.234885+0000","last_peered":"2023-10-11T09:41:59.234885+0000","last_clean":"2023-10-11T09:41:59.234885+0000","last_became_active":"2023-10-11T09:41:58.779262+0000","last_became_peered":"2023-10-11T09:41:58.779262+0000","last_unstale":"2023-10-11T09:41:59.234885+0000","last_undegraded":"2023-10-11T09:41:59.234885+0000","last_fullsized":"2023-10-11T09:41:59.234885+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:44:25.949842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.246258+0000","last_change":"2023-10-11T09:41:59.265507+0000","last_active":"2023-10-11T09:42:00.246258+0000","last_peered":"2023-10-11T09:42:00.246258+0000","last_clean":"2023-10-11T09:42:00.246258+0000","last_became_active":"2023-10-11T09:41:59.265264+0000","last_became_peered":"2023-10-11T09:41:59.265264+0000","last_unstale":"2023-10-11T09:42:00.246258+0000","last_undegraded":"2023-10-11T09:42:00.246258+0000","last_fullsized":"2023-10-11T09:42:00.246258+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:51:26.096609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.241218+0000","last_change":"2023-10-11T09:41:58.773543+0000","last_active":"2023-10-11T09:41:59.241218+0000","last_peered":"2023-10-11T09:41:59.241218+0000","last_clean":"2023-10-11T09:41:59.241218+0000","last_became_active":"2023-10-11T09:41:58.773285+0000","last_became_peered":"2023-10-11T09:41:58.773285+0000","last_unstale":"2023-10-11T09:41:59.241218+0000","last_undegraded":"2023-10-11T09:41:59.241218+0000","last_fullsized":"2023-10-11T09:41:59.241218+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:09:10.546323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133525+0000","last_change":"2023-10-11T09:41:59.275401+0000","last_active":"2023-10-11T09:42:02.133525+0000","last_peered":"2023-10-11T09:42:02.133525+0000","last_clean":"2023-10-11T09:42:02.133525+0000","last_became_active":"2023-10-11T09:41:59.274821+0000","last_became_peered":"2023-10-11T09:41:59.274821+0000","last_unstale":"2023-10-11T09:42:02.133525+0000","last_undegraded":"2023-10-11T09:42:02.133525+0000","last_fullsized":"2023-10-11T09:42:02.133525+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:51:26.096609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.234879+0000","last_change":"2023-10-11T09:41:58.778916+0000","last_active":"2023-10-11T09:41:59.234879+0000","last_peered":"2023-10-11T09:41:59.234879+0000","last_clean":"2023-10-11T09:41:59.234879+0000","last_became_active":"2023-10-11T09:41:58.778610+0000","last_became_peered":"2023-10-11T09:41:58.778610+0000","last_unstale":"2023-10-11T09:41:59.234879+0000","last_undegraded":"2023-10-11T09:41:59.234879+0000","last_fullsized":"2023-10-11T09:41:59.234879+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:40:16.473783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.247174+0000","last_change":"2023-10-11T09:41:59.270447+0000","last_active":"2023-10-11T09:42:00.247174+0000","last_peered":"2023-10-11T09:42:00.247174+0000","last_clean":"2023-10-11T09:42:00.247174+0000","last_became_active":"2023-10-11T09:41:59.270124+0000","last_became_peered":"2023-10-11T09:41:59.270124+0000","last_unstale":"2023-10-11T09:42:00.247174+0000","last_undegraded":"2023-10-11T09:42:00.247174+0000","last_fullsized":"2023-10-11T09:42:00.247174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:45:37.542487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.241255+0000","last_change":"2023-10-11T09:41:58.777578+0000","last_active":"2023-10-11T09:41:59.241255+0000","last_peered":"2023-10-11T09:41:59.241255+0000","last_clean":"2023-10-11T09:41:59.241255+0000","last_became_active":"2023-10-11T09:41:58.777251+0000","last_became_peered":"2023-10-11T09:41:58.777251+0000","last_unstale":"2023-10-11T09:41:59.241255+0000","last_undegraded":"2023-10-11T09:41:59.241255+0000","last_fullsized":"2023-10-11T09:41:59.241255+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:23:59.470658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":181,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.240598+0000","last_change":"2023-10-11T09:40:29.874372+0000","last_active":"2023-10-11T09:41:59.240598+0000","last_peered":"2023-10-11T09:41:59.240598+0000","last_clean":"2023-10-11T09:41:59.240598+0000","last_became_active":"2023-10-11T09:40:29.873621+0000","last_became_peered":"2023-10-11T09:40:29.873621+0000","last_unstale":"2023-10-11T09:41:59.240598+0000","last_undegraded":"2023-10-11T09:41:59.240598+0000","last_fullsized":"2023-10-11T09:41:59.240598+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_clean_scrub_stamp":"2023-10-11T09:40:28.742276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:22:07.985311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.251308+0000","last_change":"2023-10-11T09:40:36.148773+0000","last_active":"2023-10-11T09:41:59.251308+0000","last_peered":"2023-10-11T09:41:59.251308+0000","last_clean":"2023-10-11T09:41:59.251308+0000","last_became_active":"2023-10-11T09:40:36.148489+0000","last_became_peered":"2023-10-11T09:40:36.148489+0000","last_unstale":"2023-10-11T09:41:59.251308+0000","last_undegraded":"2023-10-11T09:41:59.251308+0000","last_fullsized":"2023-10-11T09:41:59.251308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.132837+0000","last_change":"2023-10-11T09:41:59.268241+0000","last_active":"2023-10-11T09:42:02.132837+0000","last_peered":"2023-10-11T09:42:02.132837+0000","last_clean":"2023-10-11T09:42:02.132837+0000","last_became_active":"2023-10-11T09:41:59.267982+0000","last_became_peered":"2023-10-11T09:41:59.267982+0000","last_unstale":"2023-10-11T09:42:02.132837+0000","last_undegraded":"2023-10-11T09:42:02.132837+0000","last_fullsized":"2023-10-11T09:42:02.132837+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:42:14.977269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"36'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.273771+0000","last_change":"2023-10-11T09:41:58.777702+0000","last_active":"2023-10-11T09:41:59.273771+0000","last_peered":"2023-10-11T09:41:59.273771+0000","last_clean":"2023-10-11T09:41:59.273771+0000","last_became_active":"2023-10-11T09:41:58.777413+0000","last_became_peered":"2023-10-11T09:41:58.777413+0000","last_unstale":"2023-10-11T09:41:59.273771+0000","last_undegraded":"2023-10-11T09:41:59.273771+0000","last_fullsized":"2023-10-11T09:41:59.273771+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:58:21.613408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":28,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.240155+0000","last_change":"2023-10-11T09:40:36.149739+0000","last_active":"2023-10-11T09:41:59.240155+0000","last_peered":"2023-10-11T09:41:59.240155+0000","last_clean":"2023-10-11T09:41:59.240155+0000","last_became_active":"2023-10-11T09:40:36.149419+0000","last_became_peered":"2023-10-11T09:40:36.149419+0000","last_unstale":"2023-10-11T09:41:59.240155+0000","last_undegraded":"2023-10-11T09:41:59.240155+0000","last_fullsized":"2023-10-11T09:41:59.240155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:54:50.640362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.246403+0000","last_change":"2023-10-11T09:41:59.274654+0000","last_active":"2023-10-11T09:42:00.246403+0000","last_peered":"2023-10-11T09:42:00.246403+0000","last_clean":"2023-10-11T09:42:00.246403+0000","last_became_active":"2023-10-11T09:41:59.274296+0000","last_became_peered":"2023-10-11T09:41:59.274296+0000","last_unstale":"2023-10-11T09:42:00.246403+0000","last_undegraded":"2023-10-11T09:42:00.246403+0000","last_fullsized":"2023-10-11T09:42:00.246403+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:29:17.473816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"37'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.245245+0000","last_change":"2023-10-11T09:41:58.778782+0000","last_active":"2023-10-11T09:42:00.245245+0000","last_peered":"2023-10-11T09:42:00.245245+0000","last_clean":"2023-10-11T09:42:00.245245+0000","last_became_active":"2023-10-11T09:41:58.778514+0000","last_became_peered":"2023-10-11T09:41:58.778514+0000","last_unstale":"2023-10-11T09:42:00.245245+0000","last_undegraded":"2023-10-11T09:42:00.245245+0000","last_fullsized":"2023-10-11T09:42:00.245245+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:41:27.673781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":25,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.234843+0000","last_change":"2023-10-11T09:40:36.216248+0000","last_active":"2023-10-11T09:41:59.234843+0000","last_peered":"2023-10-11T09:41:59.234843+0000","last_clean":"2023-10-11T09:41:59.234843+0000","last_became_active":"2023-10-11T09:40:36.215890+0000","last_became_peered":"2023-10-11T09:40:36.215890+0000","last_unstale":"2023-10-11T09:41:59.234843+0000","last_undegraded":"2023-10-11T09:41:59.234843+0000","last_fullsized":"2023-10-11T09:41:59.234843+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.135487+0000","last_change":"2023-10-11T09:41:59.253002+0000","last_active":"2023-10-11T09:42:02.135487+0000","last_peered":"2023-10-11T09:42:02.135487+0000","last_clean":"2023-10-11T09:42:02.135487+0000","last_became_active":"2023-10-11T09:41:59.252742+0000","last_became_peered":"2023-10-11T09:41:59.252742+0000","last_unstale":"2023-10-11T09:42:02.135487+0000","last_undegraded":"2023-10-11T09:42:02.135487+0000","last_fullsized":"2023-10-11T09:42:02.135487+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:46:25.915286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.242326+0000","last_change":"2023-10-11T09:41:58.778606+0000","last_active":"2023-10-11T09:41:59.242326+0000","last_peered":"2023-10-11T09:41:59.242326+0000","last_clean":"2023-10-11T09:41:59.242326+0000","last_became_active":"2023-10-11T09:41:58.778305+0000","last_became_peered":"2023-10-11T09:41:58.778305+0000","last_unstale":"2023-10-11T09:41:59.242326+0000","last_undegraded":"2023-10-11T09:41:59.242326+0000","last_fullsized":"2023-10-11T09:41:59.242326+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:55:29.823667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.246884+0000","last_change":"2023-10-11T09:40:36.151710+0000","last_active":"2023-10-11T09:41:59.246884+0000","last_peered":"2023-10-11T09:41:59.246884+0000","last_clean":"2023-10-11T09:41:59.246884+0000","last_became_active":"2023-10-11T09:40:36.151219+0000","last_became_peered":"2023-10-11T09:40:36.151219+0000","last_unstale":"2023-10-11T09:41:59.246884+0000","last_undegraded":"2023-10-11T09:41:59.246884+0000","last_fullsized":"2023-10-11T09:41:59.246884+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133803+0000","last_change":"2023-10-11T09:41:59.252548+0000","last_active":"2023-10-11T09:42:02.133803+0000","last_peered":"2023-10-11T09:42:02.133803+0000","last_clean":"2023-10-11T09:42:02.133803+0000","last_became_active":"2023-10-11T09:41:59.252259+0000","last_became_peered":"2023-10-11T09:41:59.252259+0000","last_unstale":"2023-10-11T09:42:02.133803+0000","last_undegraded":"2023-10-11T09:42:02.133803+0000","last_fullsized":"2023-10-11T09:42:02.133803+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:07:46.727754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.240925+0000","last_change":"2023-10-11T09:41:58.772968+0000","last_active":"2023-10-11T09:41:59.240925+0000","last_peered":"2023-10-11T09:41:59.240925+0000","last_clean":"2023-10-11T09:41:59.240925+0000","last_became_active":"2023-10-11T09:41:58.772686+0000","last_became_peered":"2023-10-11T09:41:58.772686+0000","last_unstale":"2023-10-11T09:41:59.240925+0000","last_undegraded":"2023-10-11T09:41:59.240925+0000","last_fullsized":"2023-10-11T09:41:59.240925+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:58:21.613408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.242524+0000","last_change":"2023-10-11T09:40:36.160620+0000","last_active":"2023-10-11T09:41:59.242524+0000","last_peered":"2023-10-11T09:41:59.242524+0000","last_clean":"2023-10-11T09:41:59.242524+0000","last_became_active":"2023-10-11T09:40:36.160171+0000","last_became_peered":"2023-10-11T09:40:36.160171+0000","last_unstale":"2023-10-11T09:41:59.242524+0000","last_undegraded":"2023-10-11T09:41:59.242524+0000","last_fullsized":"2023-10-11T09:41:59.242524+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:02.133691+0000","last_change":"2023-10-11T09:41:59.260111+0000","last_active":"2023-10-11T09:42:02.133691+0000","last_peered":"2023-10-11T09:42:02.133691+0000","last_clean":"2023-10-11T09:42:02.133691+0000","last_became_active":"2023-10-11T09:41:59.259796+0000","last_became_peered":"2023-10-11T09:41:59.259796+0000","last_unstale":"2023-10-11T09:42:02.133691+0000","last_undegraded":"2023-10-11T09:42:02.133691+0000","last_fullsized":"2023-10-11T09:42:02.133691+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:55:30.838799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.255282+0000","last_change":"2023-10-11T09:41:58.773403+0000","last_active":"2023-10-11T09:41:59.255282+0000","last_peered":"2023-10-11T09:41:59.255282+0000","last_clean":"2023-10-11T09:41:59.255282+0000","last_became_active":"2023-10-11T09:41:58.773132+0000","last_became_peered":"2023-10-11T09:41:58.773132+0000","last_unstale":"2023-10-11T09:41:59.255282+0000","last_undegraded":"2023-10-11T09:41:59.255282+0000","last_fullsized":"2023-10-11T09:41:59.255282+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:28:16.256141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":28,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.241892+0000","last_change":"2023-10-11T09:40:36.162174+0000","last_active":"2023-10-11T09:41:59.241892+0000","last_peered":"2023-10-11T09:41:59.241892+0000","last_clean":"2023-10-11T09:41:59.241892+0000","last_became_active":"2023-10-11T09:40:36.161710+0000","last_became_peered":"2023-10-11T09:40:36.161710+0000","last_unstale":"2023-10-11T09:41:59.241892+0000","last_undegraded":"2023-10-11T09:41:59.241892+0000","last_fullsized":"2023-10-11T09:41:59.241892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:57:03.630878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:01.632131+0000","last_change":"2023-10-11T09:41:59.251157+0000","last_active":"2023-10-11T09:42:01.632131+0000","last_peered":"2023-10-11T09:42:01.632131+0000","last_clean":"2023-10-11T09:42:01.632131+0000","last_became_active":"2023-10-11T09:41:59.250831+0000","last_became_peered":"2023-10-11T09:41:59.250831+0000","last_unstale":"2023-10-11T09:42:01.632131+0000","last_undegraded":"2023-10-11T09:42:01.632131+0000","last_fullsized":"2023-10-11T09:42:01.632131+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:34:02.831174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.242946+0000","last_change":"2023-10-11T09:41:58.779092+0000","last_active":"2023-10-11T09:41:59.242946+0000","last_peered":"2023-10-11T09:41:59.242946+0000","last_clean":"2023-10-11T09:41:59.242946+0000","last_became_active":"2023-10-11T09:41:58.778781+0000","last_became_peered":"2023-10-11T09:41:58.778781+0000","last_unstale":"2023-10-11T09:41:59.242946+0000","last_undegraded":"2023-10-11T09:41:59.242946+0000","last_fullsized":"2023-10-11T09:41:59.242946+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:07:42.669679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":24,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.244192+0000","last_change":"2023-10-11T09:40:36.150224+0000","last_active":"2023-10-11T09:41:59.244192+0000","last_peered":"2023-10-11T09:41:59.244192+0000","last_clean":"2023-10-11T09:41:59.244192+0000","last_became_active":"2023-10-11T09:40:36.149919+0000","last_became_peered":"2023-10-11T09:40:36.149919+0000","last_unstale":"2023-10-11T09:41:59.244192+0000","last_undegraded":"2023-10-11T09:41:59.244192+0000","last_fullsized":"2023-10-11T09:41:59.244192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T09:42:00.246688+0000","last_change":"2023-10-11T09:41:59.275142+0000","last_active":"2023-10-11T09:42:00.246688+0000","last_peered":"2023-10-11T09:42:00.246688+0000","last_clean":"2023-10-11T09:42:00.246688+0000","last_became_active":"2023-10-11T09:41:59.274690+0000","last_became_peered":"2023-10-11T09:41:59.274690+0000","last_unstale":"2023-10-11T09:42:00.246688+0000","last_undegraded":"2023-10-11T09:42:00.246688+0000","last_fullsized":"2023-10-11T09:42:00.246688+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:58.738403+0000","last_clean_scrub_stamp":"2023-10-11T09:41:58.738403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:32:36.655338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.241558+0000","last_change":"2023-10-11T09:41:58.773081+0000","last_active":"2023-10-11T09:41:59.241558+0000","last_peered":"2023-10-11T09:41:59.241558+0000","last_clean":"2023-10-11T09:41:59.241558+0000","last_became_active":"2023-10-11T09:41:58.772778+0000","last_became_peered":"2023-10-11T09:41:58.772778+0000","last_unstale":"2023-10-11T09:41:59.241558+0000","last_undegraded":"2023-10-11T09:41:59.241558+0000","last_fullsized":"2023-10-11T09:41:59.241558+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:41:57.723271+0000","last_clean_scrub_stamp":"2023-10-11T09:41:57.723271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:05:43.880930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":24,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T09:41:59.255713+0000","last_change":"2023-10-11T09:40:36.151839+0000","last_active":"2023-10-11T09:41:59.255713+0000","last_peered":"2023-10-11T09:41:59.255713+0000","last_clean":"2023-10-11T09:41:59.255713+0000","last_became_active":"2023-10-11T09:40:36.151179+0000","last_became_peered":"2023-10-11T09:40:36.151179+0000","last_unstale":"2023-10-11T09:41:59.255713+0000","last_undegraded":"2023-10-11T09:41:59.255713+0000","last_fullsized":"2023-10-11T09:41:59.255713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":1032192,"data_stored":767390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.21,"15min":1.21},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.3440000000000001}]},{"osd":1,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":2.25,"5min":2.25,"15min":2.25},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.359}]},{"osd":2,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.206},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":1.2390000000000001}]},{"osd":3,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.286,"15min":1.286},"min":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":2.1579999999999999,"5min":2.1579999999999999,"15min":2.1579999999999999},"last":1.073}]},{"osd":4,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":1.036}]},{"osd":5,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.292},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":3.0750000000000002,"5min":3.0750000000000002,"15min":3.0750000000000002},"last":1.2689999999999999}]},{"osd":6,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.49,"15min":1.49},"min":{"1min":1.153,"5min":1.153,"15min":1.153},"max":{"1min":3.137,"5min":3.137,"15min":3.137},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"min":{"1min":1.099,"5min":1.099,"15min":1.099},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":1.3200000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":425984,"data_stored":175970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.90600000000000003}]},{"osd":1,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.202,"15min":1.202},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":1.8260000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.2250000000000001}]},{"osd":2,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":1.079},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":2.605,"5min":2.605,"15min":2.605},"last":0.98999999999999999}]},{"osd":3,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":3.0819999999999999,"5min":3.0819999999999999,"15min":3.0819999999999999},"last":0.82599999999999996}]},{"osd":4,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.337,"15min":1.337},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.978,"5min":1.978,"15min":1.978},"last":1.111},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.6920000000000002,"5min":2.6920000000000002,"15min":2.6920000000000002},"last":1.302}]},{"osd":5,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.264},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":2.7290000000000001,"5min":2.7290000000000001,"15min":2.7290000000000001},"last":1.347}]},{"osd":7,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":2.1899999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.3050000000000002,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.175}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":430080,"data_stored":175500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":1.23}]},{"osd":1,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":1.1559999999999999}]},{"osd":2,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.831,"5min":1.831,"15min":1.831},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.284}]},{"osd":3,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.319},{"interface":"front","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.419}]},{"osd":4,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":1.369},{"interface":"front","average":{"1min":1.228,"5min":1.228,"15min":1.228},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":1.0840000000000001}]},{"osd":6,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.4529999999999998,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":1.4610000000000001}]},{"osd":7,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":3.6120000000000001,"5min":3.6120000000000001,"15min":3.6120000000000001},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"min":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":3.2109999999999999,"5min":3.2109999999999999,"15min":3.2109999999999999},"last":1.55}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":430080,"data_stored":175992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":1.286},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.1160000000000001,"5min":2.1160000000000001,"15min":2.1160000000000001},"last":1.381}]},{"osd":1,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.98399999999999999}]},{"osd":2,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":1.345},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":1.7130000000000001}]},{"osd":3,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.367,"5min":1.367,"15min":1.367},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.3340000000000001,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":1.6679999999999999}]},{"osd":5,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.236,"15min":1.236},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.93999999999999995}]},{"osd":6,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.8,"5min":1.8,"15min":1.8},"last":1.121},{"interface":"front","average":{"1min":1.357,"5min":1.357,"15min":1.357},"min":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.5549999999999999}]},{"osd":7,"last update":"Wed Oct 11 09:41:33 2023","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.482},{"interface":"front","average":{"1min":1.516,"5min":1.516,"15min":1.516},"min":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"max":{"1min":1.9990000000000001,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":1.786}]}]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":438272,"data_stored":176986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.747},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.48699999999999999}]},{"osd":1,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.60199999999999998}]},{"osd":2,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.5870000000000002,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":0.98399999999999999}]},{"osd":4,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.395,"5min":2.395,"15min":2.395},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.367,"5min":1.367,"15min":1.367},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":3.1579999999999999,"5min":3.1579999999999999,"15min":3.1579999999999999},"last":1.1870000000000001}]},{"osd":5,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.393,"15min":1.393},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":3.0009999999999999,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":1.03},{"interface":"front","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":3.8050000000000002,"5min":3.8050000000000002,"15min":3.8050000000000002},"last":1.224}]},{"osd":6,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":2.1949999999999998,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":3.2549999999999999,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":1.266}]},{"osd":7,"last update":"Wed Oct 11 09:41:29 2023","interfaces":[{"interface":"back","average":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":3.7490000000000001,"5min":3.7490000000000001,"15min":3.7490000000000001},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.448,"5min":1.448,"15min":1.448},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":1.121}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":425984,"data_stored":175458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":2.0600000000000001,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":2.8919999999999999,"5min":2.8919999999999999,"15min":2.8919999999999999},"last":0.68000000000000005}]},{"osd":1,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":3.7650000000000001,"5min":3.7650000000000001,"15min":3.7650000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.5339999999999998,"5min":2.5339999999999998,"15min":2.5339999999999998},"last":0.71499999999999997}]},{"osd":3,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.403,"5min":2.403,"15min":2.403},"last":0.65500000000000003},{"interface":"front","average":{"1min":1.117,"5min":1.117,"15min":1.117},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":2.7589999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":0.63600000000000001}]},{"osd":4,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.476,"5min":1.476,"15min":1.476},"min":{"1min":1.085,"5min":1.085,"15min":1.085},"max":{"1min":2.6859999999999999,"5min":2.6859999999999999,"15min":2.6859999999999999},"last":1.111}]},{"osd":5,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.409,"15min":1.409},"min":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"max":{"1min":3.044,"5min":3.044,"15min":3.044},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":3.5790000000000002,"5min":3.5790000000000002,"15min":3.5790000000000002},"last":0.89500000000000002}]},{"osd":6,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"min":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"max":{"1min":3.1600000000000001,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.044},{"interface":"front","average":{"1min":1.397,"5min":1.397,"15min":1.397},"min":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"max":{"1min":2.839,"5min":2.839,"15min":2.839},"last":0.85399999999999998}]},{"osd":7,"last update":"Wed Oct 11 09:41:30 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.49,"15min":1.49},"min":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":3.1230000000000002,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"min":{"1min":1.161,"5min":1.161,"15min":1.161},"max":{"1min":2.988,"5min":2.988,"15min":2.988},"last":1.0780000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":425984,"data_stored":175478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:41:28 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":1.159}]},{"osd":2,"last update":"Wed Oct 11 09:41:28 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.85599999999999998}]},{"osd":3,"last update":"Wed Oct 11 09:41:28 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":0.60699999999999998}]},{"osd":4,"last update":"Wed Oct 11 09:41:28 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":2.016,"5min":2.016,"15min":2.016},"last":1.147},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":2.323,"5min":2.323,"15min":2.323},"last":1.1240000000000001}]},{"osd":5,"last update":"Wed Oct 11 09:41:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":1.04,"5min":1.04,"15min":1.04},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.113}]},{"osd":6,"last update":"Wed Oct 11 09:41:28 2023","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"min":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"max":{"1min":2.8940000000000001,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":1.022,"5min":1.022,"15min":1.022},"max":{"1min":3.3719999999999999,"5min":3.3719999999999999,"15min":3.3719999999999999},"last":1.173}]},{"osd":7,"last update":"Wed Oct 11 09:41:28 2023","interfaces":[{"interface":"back","average":{"1min":1.468,"5min":1.468,"15min":1.468},"min":{"1min":1.171,"5min":1.171,"15min":1.171},"max":{"1min":2.194,"5min":2.194,"15min":2.194},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"min":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"max":{"1min":1.968,"5min":1.968,"15min":1.968},"last":0.99399999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":1019904,"data_stored":765790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 11 09:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.3100000000000001,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":0.73799999999999999}]},{"osd":2,"last update":"Wed Oct 11 09:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.63900000000000001}]},{"osd":3,"last update":"Wed Oct 11 09:41:27 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.7639999999999998,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":0.57699999999999996},{"interface":"front","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":0.93100000000000005}]},{"osd":4,"last update":"Wed Oct 11 09:41:27 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.242},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.6629999999999998,"5min":2.6629999999999998,"15min":2.6629999999999998},"last":1.1539999999999999}]},{"osd":5,"last update":"Wed Oct 11 09:41:27 2023","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"min":{"1min":0.998,"5min":0.998,"15min":0.998},"max":{"1min":3.0150000000000001,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":1.276},{"interface":"front","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":3.601,"5min":3.601,"15min":3.601},"last":1.331}]},{"osd":6,"last update":"Wed Oct 11 09:41:27 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.825,"5min":1.825,"15min":1.825},"last":0.997},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"min":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":1.1200000000000001}]},{"osd":7,"last update":"Wed Oct 11 09:41:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"min":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":1.381},{"interface":"front","average":{"1min":1.417,"5min":1.417,"15min":1.417},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":2.5419999999999998,"5min":2.5419999999999998,"15min":2.5419999999999998},"last":1.3069999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-11T09:42:08.800 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-11T09:42:08.801 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-11T09:42:09.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.016+0000 7fa6a9482700 1 Processor -- start 2023-10-11T09:42:09.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.016+0000 7fa6a9482700 1 -- start start 2023-10-11T09:42:09.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.016+0000 7fa6a9482700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa6a40cf8d0 0x7fa6a40b4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:09.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.020+0000 7fa6a9482700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6a40b4fd0 con 0x7fa6a40cfca0 2023-10-11T09:42:09.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.020+0000 7fa6a9482700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6a40b52b0 con 0x7fa6a4160870 2023-10-11T09:42:09.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.020+0000 7fa6a9482700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a40b5870 con 0x7fa6a40cf8d0 2023-10-11T09:42:09.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.020+0000 7fa6a27fc700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fa6a40cfca0 0x7fa6a40cd5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:51820/0 (socket says 172.21.15.78:51820) 2023-10-11T09:42:09.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.020+0000 7fa6a27fc700 1 -- 172.21.15.78:0/1346090775 learned_addr learned my addr 172.21.15.78:0/1346090775 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:42:09.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.020+0000 7fa6a2ffd700 1 --2- 172.21.15.78:0/1346090775 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa6a40cf8d0 0x7fa6a40b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:09.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.020+0000 7fa6a1ffb700 1 -- 172.21.15.78:0/1346090775 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 390736954 0 0) 0x7fa6a40b4fd0 con 0x7fa6a40cfca0 2023-10-11T09:42:09.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.020+0000 7fa6a1ffb700 1 -- 172.21.15.78:0/1346090775 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa68c0035d0 con 0x7fa6a40cfca0 2023-10-11T09:42:09.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.020+0000 7fa6a1ffb700 1 -- 172.21.15.78:0/1346090775 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1163628584 0 0) 0x7fa6a40b52b0 con 0x7fa6a4160870 2023-10-11T09:42:09.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.020+0000 7fa6a1ffb700 1 -- 172.21.15.78:0/1346090775 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6a40b4fd0 con 0x7fa6a4160870 2023-10-11T09:42:09.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.020+0000 7fa6a2ffd700 1 -- 172.21.15.78:0/1346090775 >> v1:172.21.15.133:6789/0 conn(0x7fa6a4160870 legacy=0x7fa6a415cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:09.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.020+0000 7fa6a2ffd700 1 -- 172.21.15.78:0/1346090775 >> v1:172.21.15.78:6789/0 conn(0x7fa6a40cfca0 legacy=0x7fa6a40cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:09.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.020+0000 7fa6a2ffd700 1 -- 172.21.15.78:0/1346090775 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a40b5590 con 0x7fa6a40cf8d0 2023-10-11T09:42:09.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.020+0000 7fa6a2ffd700 1 --2- 172.21.15.78:0/1346090775 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa6a40cf8d0 0x7fa6a40b4940 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa694009880 tx=0x7fa694009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=364f72e4dac32034 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:09.030 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.020+0000 7fa6a1ffb700 1 -- 172.21.15.78:0/1346090775 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa694011830 con 0x7fa6a40cf8d0 2023-10-11T09:42:09.030 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.020+0000 7fa6a1ffb700 1 -- 172.21.15.78:0/1346090775 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa694011990 con 0x7fa6a40cf8d0 2023-10-11T09:42:09.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.020+0000 7fa6a1ffb700 1 -- 172.21.15.78:0/1346090775 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa694011d20 con 0x7fa6a40cf8d0 2023-10-11T09:42:09.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.024+0000 7fa6a9482700 1 -- 172.21.15.78:0/1346090775 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa6a40cf8d0 msgr2=0x7fa6a40b4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:09.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.024+0000 7fa6a9482700 1 --2- 172.21.15.78:0/1346090775 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa6a40cf8d0 0x7fa6a40b4940 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa694009880 tx=0x7fa694009b90 comp rx=0 tx=0).stop 2023-10-11T09:42:09.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.024+0000 7fa6a9482700 1 -- 172.21.15.78:0/1346090775 shutdown_connections 2023-10-11T09:42:09.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.024+0000 7fa6a9482700 1 --2- 172.21.15.78:0/1346090775 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa6a40cf8d0 0x7fa6a40b4940 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:09.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.024+0000 7fa6a9482700 1 -- 172.21.15.78:0/1346090775 >> 172.21.15.78:0/1346090775 conn(0x7fa6a406b010 msgr2=0x7fa6a406b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:09.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.024+0000 7fa6a9482700 1 -- 172.21.15.78:0/1346090775 shutdown_connections 2023-10-11T09:42:09.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.024+0000 7fa6a9482700 1 -- 172.21.15.78:0/1346090775 wait complete. 2023-10-11T09:42:09.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.024+0000 7fa6a9482700 1 Processor -- start 2023-10-11T09:42:09.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.028+0000 7fa6a9482700 1 -- start start 2023-10-11T09:42:09.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.028+0000 7fa6a9482700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6a40cf8d0 0x7fa6a40c69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:09.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.028+0000 7fa6a9482700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6a40cfca0 0x7fa6a40b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:09.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.028+0000 7fa6a9482700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa6a4160870 0x7fa6a40cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:09.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.028+0000 7fa6a9482700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a40b3a20 con 0x7fa6a40cf8d0 2023-10-11T09:42:09.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.028+0000 7fa6a9482700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a40c6f30 con 0x7fa6a40cfca0 2023-10-11T09:42:09.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.028+0000 7fa6a9482700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a40b3140 con 0x7fa6a4160870 2023-10-11T09:42:09.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.028+0000 7fa6a2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6a40cf8d0 0x7fa6a40c69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:09.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.028+0000 7fa6a2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6a40cf8d0 0x7fa6a40c69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60640/0 (socket says 172.21.15.78:60640) 2023-10-11T09:42:09.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.028+0000 7fa6a2ffd700 1 -- 172.21.15.78:0/2607055991 learned_addr learned my addr 172.21.15.78:0/2607055991 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:09.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.028+0000 7fa6a37fe700 1 --2- 172.21.15.78:0/2607055991 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa6a4160870 0x7fa6a40cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:09.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.028+0000 7fa6a27fc700 1 --2- 172.21.15.78:0/2607055991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6a40cfca0 0x7fa6a40b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:09.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.028+0000 7fa6a2ffd700 1 -- 172.21.15.78:0/2607055991 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa6a4160870 msgr2=0x7fa6a40cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:09.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.028+0000 7fa6a2ffd700 1 --2- 172.21.15.78:0/2607055991 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa6a4160870 0x7fa6a40cbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:09.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.028+0000 7fa6a2ffd700 1 -- 172.21.15.78:0/2607055991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6a40cfca0 msgr2=0x7fa6a40b20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:09.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.028+0000 7fa6a2ffd700 1 --2- 172.21.15.78:0/2607055991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6a40cfca0 0x7fa6a40b20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:09.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.028+0000 7fa6a2ffd700 1 -- 172.21.15.78:0/2607055991 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6940094b0 con 0x7fa6a40cf8d0 2023-10-11T09:42:09.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.028+0000 7fa6a37fe700 1 --2- 172.21.15.78:0/2607055991 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa6a4160870 0x7fa6a40cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:42:09.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.028+0000 7fa6a2ffd700 1 --2- 172.21.15.78:0/2607055991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6a40cf8d0 0x7fa6a40c69f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa694013040 tx=0x7fa694004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:09.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.032+0000 7fa68b7fe700 1 -- 172.21.15.78:0/2607055991 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6940198d0 con 0x7fa6a40cf8d0 2023-10-11T09:42:09.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.032+0000 7fa68b7fe700 1 -- 172.21.15.78:0/2607055991 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69401e070 con 0x7fa6a40cf8d0 2023-10-11T09:42:09.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.032+0000 7fa6a9482700 1 -- 172.21.15.78:0/2607055991 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6a40b2b80 con 0x7fa6a40cf8d0 2023-10-11T09:42:09.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.032+0000 7fa68b7fe700 1 -- 172.21.15.78:0/2607055991 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa694019c50 con 0x7fa6a40cf8d0 2023-10-11T09:42:09.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.032+0000 7fa6a9482700 1 -- 172.21.15.78:0/2607055991 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a40b3640 con 0x7fa6a40cf8d0 2023-10-11T09:42:09.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.032+0000 7fa6a9482700 1 -- 172.21.15.78:0/2607055991 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6a4065d40 con 0x7fa6a40cf8d0 2023-10-11T09:42:09.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.032+0000 7fa68b7fe700 1 -- 172.21.15.78:0/2607055991 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fa69402d450 con 0x7fa6a40cf8d0 2023-10-11T09:42:09.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.036+0000 7fa68b7fe700 1 --2- 172.21.15.78:0/2607055991 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa690065f20 0x7fa6900683d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:09.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.036+0000 7fa68b7fe700 1 -- 172.21.15.78:0/2607055991 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa6940847c0 con 0x7fa6a40cf8d0 2023-10-11T09:42:09.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.036+0000 7fa6a27fc700 1 --2- 172.21.15.78:0/2607055991 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa690065f20 0x7fa6900683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:09.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.036+0000 7fa6a27fc700 1 --2- 172.21.15.78:0/2607055991 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa690065f20 0x7fa6900683d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa6a414fa80 tx=0x7fa698009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:09.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.040+0000 7fa68b7fe700 1 -- 172.21.15.78:0/2607055991 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fa694051f30 con 0x7fa6a40cf8d0 2023-10-11T09:42:09.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.276+0000 7fa6a9482700 1 -- 172.21.15.78:0/2607055991 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa6a406d350 con 0x7fa6a40cf8d0 2023-10-11T09:42:09.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.284+0000 7fa68b7fe700 1 -- 172.21.15.78:0/2607055991 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+0 (secure 0 0 0) 0x7fa694019db0 con 0x7fa6a40cf8d0 2023-10-11T09:42:09.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.288+0000 7fa6a9482700 1 -- 172.21.15.78:0/2607055991 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa690065f20 msgr2=0x7fa6900683d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:09.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.288+0000 7fa6a9482700 1 --2- 172.21.15.78:0/2607055991 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa690065f20 0x7fa6900683d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa6a414fa80 tx=0x7fa698009040 comp rx=0 tx=0).stop 2023-10-11T09:42:09.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.288+0000 7fa6a9482700 1 -- 172.21.15.78:0/2607055991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6a40cf8d0 msgr2=0x7fa6a40c69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:09.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.288+0000 7fa6a9482700 1 --2- 172.21.15.78:0/2607055991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6a40cf8d0 0x7fa6a40c69f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa694013040 tx=0x7fa694004770 comp rx=0 tx=0).stop 2023-10-11T09:42:09.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.288+0000 7fa6a9482700 1 -- 172.21.15.78:0/2607055991 shutdown_connections 2023-10-11T09:42:09.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.288+0000 7fa6a9482700 1 --2- 172.21.15.78:0/2607055991 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa690065f20 0x7fa6900683d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:09.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.288+0000 7fa6a9482700 1 --2- 172.21.15.78:0/2607055991 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa6a4160870 0x7fa6a40cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:09.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.288+0000 7fa6a9482700 1 --2- 172.21.15.78:0/2607055991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6a40cfca0 0x7fa6a40b20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:09.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.288+0000 7fa6a9482700 1 --2- 172.21.15.78:0/2607055991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6a40cf8d0 0x7fa6a40c69f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:09.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.288+0000 7fa6a9482700 1 -- 172.21.15.78:0/2607055991 >> 172.21.15.78:0/2607055991 conn(0x7fa6a406b010 msgr2=0x7fa6a40b7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:09.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.288+0000 7fa6a9482700 1 -- 172.21.15.78:0/2607055991 shutdown_connections 2023-10-11T09:42:09.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.288+0000 7fa6a9482700 1 -- 172.21.15.78:0/2607055991 wait complete. 2023-10-11T09:42:09.299 INFO:teuthology.orchestra.run.smithi078.stderr:updated caps for client.0 2023-10-11T09:42:09.317 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:42:09.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.480+0000 7f99a94e2700 1 Processor -- start 2023-10-11T09:42:09.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.480+0000 7f99a94e2700 1 -- start start 2023-10-11T09:42:09.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.480+0000 7f99a94e2700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f99a4156ce0 0x7f99a4160b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:09.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.480+0000 7f99a94e2700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f99a40d1660 0x7f99a40d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:09.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.480+0000 7f99a94e2700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a415dc10 con 0x7f99a40d28e0 2023-10-11T09:42:09.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.480+0000 7f99a94e2700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99a4160380 con 0x7f99a40d1660 2023-10-11T09:42:09.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.480+0000 7f99a94e2700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a415d930 con 0x7f99a4156ce0 2023-10-11T09:42:09.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.480+0000 7f99a8ce1700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f99a40d28e0 0x7f99a40cefc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:51826/0 (socket says 172.21.15.78:51826) 2023-10-11T09:42:09.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.480+0000 7f99a8ce1700 1 -- 172.21.15.78:0/2417728665 learned_addr learned my addr 172.21.15.78:0/2417728665 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:42:09.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.480+0000 7f99a3fff700 1 --2- 172.21.15.78:0/2417728665 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f99a4156ce0 0x7f99a4160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:09.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.480+0000 7f999b7fe700 1 --2- 172.21.15.78:0/2417728665 >> v2:172.21.15.133:3300/0 conn(0x7f99a40d1660 0x7f99a40d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:09.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.484+0000 7f99a37fe700 1 -- 172.21.15.78:0/2417728665 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3960796462 0 0) 0x7f99a415dc10 con 0x7f99a40d28e0 2023-10-11T09:42:09.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.484+0000 7f99a37fe700 1 -- 172.21.15.78:0/2417728665 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99900035d0 con 0x7f99a40d28e0 2023-10-11T09:42:09.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.484+0000 7f99a3fff700 1 -- 172.21.15.78:0/2417728665 >> v2:172.21.15.133:3300/0 conn(0x7f99a40d1660 msgr2=0x7f99a40d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:09.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.484+0000 7f99a3fff700 1 --2- 172.21.15.78:0/2417728665 >> v2:172.21.15.133:3300/0 conn(0x7f99a40d1660 0x7f99a40d0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:09.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.484+0000 7f99a3fff700 1 -- 172.21.15.78:0/2417728665 >> v1:172.21.15.78:6789/0 conn(0x7f99a40d28e0 legacy=0x7f99a40cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:09.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.484+0000 7f99a3fff700 1 -- 172.21.15.78:0/2417728665 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a4160620 con 0x7f99a4156ce0 2023-10-11T09:42:09.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.484+0000 7f99a3fff700 1 --2- 172.21.15.78:0/2417728665 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f99a4156ce0 0x7f99a4160b40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9994009d90 tx=0x7f999400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11a99a66650968e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:09.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.484+0000 7f99a37fe700 1 -- 172.21.15.78:0/2417728665 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999400e070 con 0x7f99a4156ce0 2023-10-11T09:42:09.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.484+0000 7f99a37fe700 1 -- 172.21.15.78:0/2417728665 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f999400bb20 con 0x7f99a4156ce0 2023-10-11T09:42:09.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.484+0000 7f99a37fe700 1 -- 172.21.15.78:0/2417728665 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9994012430 con 0x7f99a4156ce0 2023-10-11T09:42:09.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.484+0000 7f99a94e2700 1 -- 172.21.15.78:0/2417728665 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f99a4156ce0 msgr2=0x7f99a4160b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:09.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.484+0000 7f99a94e2700 1 --2- 172.21.15.78:0/2417728665 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f99a4156ce0 0x7f99a4160b40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9994009d90 tx=0x7f999400b330 comp rx=0 tx=0).stop 2023-10-11T09:42:09.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.484+0000 7f99a94e2700 1 -- 172.21.15.78:0/2417728665 shutdown_connections 2023-10-11T09:42:09.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.484+0000 7f99a94e2700 1 --2- 172.21.15.78:0/2417728665 >> v2:172.21.15.133:3300/0 conn(0x7f99a40d1660 0x7f99a40d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:09.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.484+0000 7f99a94e2700 1 --2- 172.21.15.78:0/2417728665 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f99a4156ce0 0x7f99a4160b40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:09.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.484+0000 7f99a94e2700 1 -- 172.21.15.78:0/2417728665 >> 172.21.15.78:0/2417728665 conn(0x7f99a406afb0 msgr2=0x7f99a406b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:09.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.484+0000 7f99a94e2700 1 -- 172.21.15.78:0/2417728665 shutdown_connections 2023-10-11T09:42:09.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.484+0000 7f99a94e2700 1 -- 172.21.15.78:0/2417728665 wait complete. 2023-10-11T09:42:09.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.488+0000 7f99a94e2700 1 Processor -- start 2023-10-11T09:42:09.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.488+0000 7f99a94e2700 1 -- start start 2023-10-11T09:42:09.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.488+0000 7f99a94e2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99a40d1660 0x7f99a4157310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:09.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.488+0000 7f99a94e2700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f99a40d28e0 0x7f99a41f30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:09.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.488+0000 7f99a94e2700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99a4156ce0 0x7f99a41f3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:09.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.488+0000 7f99a94e2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a40cbe90 con 0x7f99a40d1660 2023-10-11T09:42:09.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.488+0000 7f99a94e2700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a4157850 con 0x7f99a4156ce0 2023-10-11T09:42:09.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.488+0000 7f99a94e2700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a4158170 con 0x7f99a40d28e0 2023-10-11T09:42:09.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.488+0000 7f99a3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99a40d1660 0x7f99a4157310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:09.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.488+0000 7f99a3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99a40d1660 0x7f99a4157310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60644/0 (socket says 172.21.15.78:60644) 2023-10-11T09:42:09.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.488+0000 7f99a3fff700 1 -- 172.21.15.78:0/1734753185 learned_addr learned my addr 172.21.15.78:0/1734753185 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:09.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.492+0000 7f999b7fe700 1 --2- 172.21.15.78:0/1734753185 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f99a40d28e0 0x7f99a41f30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:09.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.492+0000 7f99a8ce1700 1 --2- 172.21.15.78:0/1734753185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99a4156ce0 0x7f99a41f3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:09.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.492+0000 7f99a3fff700 1 -- 172.21.15.78:0/1734753185 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f99a40d28e0 msgr2=0x7f99a41f30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:09.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.492+0000 7f99a3fff700 1 --2- 172.21.15.78:0/1734753185 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f99a40d28e0 0x7f99a41f30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:09.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.492+0000 7f99a3fff700 1 -- 172.21.15.78:0/1734753185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99a4156ce0 msgr2=0x7f99a41f3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:09.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.492+0000 7f99a3fff700 1 --2- 172.21.15.78:0/1734753185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99a4156ce0 0x7f99a41f3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:09.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.492+0000 7f99a3fff700 1 -- 172.21.15.78:0/1734753185 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9994009a70 con 0x7f99a40d1660 2023-10-11T09:42:09.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.492+0000 7f99a8ce1700 1 --2- 172.21.15.78:0/1734753185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99a4156ce0 0x7f99a41f3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:42:09.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.492+0000 7f99a3fff700 1 --2- 172.21.15.78:0/1734753185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99a40d1660 0x7f99a4157310 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9994014040 tx=0x7f9994004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:09.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.492+0000 7f99a17fa700 1 -- 172.21.15.78:0/1734753185 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9994012430 con 0x7f99a40d1660 2023-10-11T09:42:09.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.492+0000 7f99a17fa700 1 -- 172.21.15.78:0/1734753185 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9994020070 con 0x7f99a40d1660 2023-10-11T09:42:09.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.492+0000 7f99a94e2700 1 -- 172.21.15.78:0/1734753185 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a40cc110 con 0x7f99a40d1660 2023-10-11T09:42:09.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.492+0000 7f99a17fa700 1 -- 172.21.15.78:0/1734753185 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9994004280 con 0x7f99a40d1660 2023-10-11T09:42:09.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.492+0000 7f99a94e2700 1 -- 172.21.15.78:0/1734753185 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a40cc6d0 con 0x7f99a40d1660 2023-10-11T09:42:09.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.492+0000 7f99a94e2700 1 -- 172.21.15.78:0/1734753185 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99a4065d40 con 0x7f99a40d1660 2023-10-11T09:42:09.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.496+0000 7f99a17fa700 1 -- 172.21.15.78:0/1734753185 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f99940043e0 con 0x7f99a40d1660 2023-10-11T09:42:09.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.496+0000 7f99a17fa700 1 --2- 172.21.15.78:0/1734753185 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9984065ed0 0x7f9984068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:09.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.496+0000 7f99a17fa700 1 -- 172.21.15.78:0/1734753185 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9994022070 con 0x7f99a40d1660 2023-10-11T09:42:09.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.496+0000 7f999b7fe700 1 --2- 172.21.15.78:0/1734753185 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9984065ed0 0x7f9984068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:09.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.500+0000 7f999b7fe700 1 --2- 172.21.15.78:0/1734753185 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9984065ed0 0x7f9984068380 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f99a415c250 tx=0x7f998c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:09.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.500+0000 7f99a17fa700 1 -- 172.21.15.78:0/1734753185 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f9994052cf0 con 0x7f99a40d1660 2023-10-11T09:42:09.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.704+0000 7f99a94e2700 1 -- 172.21.15.78:0/1734753185 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99a40ce5a0 con 0x7f99a40d1660 2023-10-11T09:42:09.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.708+0000 7f99a17fa700 1 -- 172.21.15.78:0/1734753185 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4648 (secure 0 0 0) 0x7f9994056310 con 0x7f99a40d1660 2023-10-11T09:42:09.715 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:09.715 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5308,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/655413753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":655413753},{"type":"v1","addr":"172.21.15.133:6835","nonce":655413753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5352,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2966622437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2966622437},{"type":"v1","addr":"172.21.15.78:6835","nonce":2966622437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5355,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/700246445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":700246445},{"type":"v1","addr":"172.21.15.78:6837","nonce":700246445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:41:59.225318+0000","modified":"2023-10-11T09:42:02.252787+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5364},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5364":{"gid":5364,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.133:6833/1862542010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1862542010},{"type":"v1","addr":"172.21.15.133:6833","nonce":1862542010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-11T09:42:09.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.708+0000 7f99a94e2700 1 -- 172.21.15.78:0/1734753185 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9984065ed0 msgr2=0x7f9984068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:09.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.708+0000 7f99a94e2700 1 --2- 172.21.15.78:0/1734753185 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9984065ed0 0x7f9984068380 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f99a415c250 tx=0x7f998c008040 comp rx=0 tx=0).stop 2023-10-11T09:42:09.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.708+0000 7f99a94e2700 1 -- 172.21.15.78:0/1734753185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99a40d1660 msgr2=0x7f99a4157310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:09.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.708+0000 7f99a94e2700 1 --2- 172.21.15.78:0/1734753185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99a40d1660 0x7f99a4157310 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9994014040 tx=0x7f9994004af0 comp rx=0 tx=0).stop 2023-10-11T09:42:09.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.712+0000 7f99a94e2700 1 -- 172.21.15.78:0/1734753185 shutdown_connections 2023-10-11T09:42:09.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.712+0000 7f99a94e2700 1 --2- 172.21.15.78:0/1734753185 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9984065ed0 0x7f9984068380 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:09.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.712+0000 7f99a94e2700 1 --2- 172.21.15.78:0/1734753185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99a4156ce0 0x7f99a41f3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:09.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.712+0000 7f99a94e2700 1 --2- 172.21.15.78:0/1734753185 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f99a40d28e0 0x7f99a41f30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:09.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.712+0000 7f99a94e2700 1 --2- 172.21.15.78:0/1734753185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99a40d1660 0x7f99a4157310 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:09.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.712+0000 7f99a94e2700 1 -- 172.21.15.78:0/1734753185 >> 172.21.15.78:0/1734753185 conn(0x7f99a406afb0 msgr2=0x7f99a415ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:09.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.712+0000 7f99a94e2700 1 -- 172.21.15.78:0/1734753185 shutdown_connections 2023-10-11T09:42:09.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:09.712+0000 7f99a94e2700 1 -- 172.21.15.78:0/1734753185 wait complete. 2023-10-11T09:42:09.720 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 16 2023-10-11T09:42:10.143 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-11T09:41:59.225318+0000', 'modified': '2023-10-11T09:42:02.252787+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5364}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5364': {'gid': 5364, 'name': 'b', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.133:6833/1862542010', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6832', 'nonce': 1862542010}, {'type': 'v1', 'addr': '172.21.15.133:6833', 'nonce': 1862542010}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-11T09:42:10.144 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-11T09:42:10.145 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-11T09:42:10.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.316+0000 7f64b8d37700 1 Processor -- start 2023-10-11T09:42:10.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.316+0000 7f64b8d37700 1 -- start start 2023-10-11T09:42:10.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.320+0000 7f64b8d37700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f64b40cda10 0x7f64b40caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:10.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.320+0000 7f64b8d37700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64b40cdde0 0x7f64b4164610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:10.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.320+0000 7f64b8d37700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64b4165390 con 0x7f64b4164b50 2023-10-11T09:42:10.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.320+0000 7f64b8d37700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64b40ca440 con 0x7f64b40cda10 2023-10-11T09:42:10.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.320+0000 7f64b8d37700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64b40ca720 con 0x7f64b40cdde0 2023-10-11T09:42:10.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.320+0000 7f64aaffd700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64b40cdde0 0x7f64b4164610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:10.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.320+0000 7f64b37fe700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f64b40cda10 0x7f64b40caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:10.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.320+0000 7f64aaffd700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64b40cdde0 0x7f64b4164610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:42726/0 (socket says 172.21.15.78:42726) 2023-10-11T09:42:10.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.320+0000 7f64aaffd700 1 -- 172.21.15.78:0/482007356 learned_addr learned my addr 172.21.15.78:0/482007356 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:10.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.320+0000 7f64aaffd700 1 -- 172.21.15.78:0/482007356 >> v2:172.21.15.133:3300/0 conn(0x7f64b40cda10 msgr2=0x7f64b40caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:10.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.320+0000 7f64aaffd700 1 --2- 172.21.15.78:0/482007356 >> v2:172.21.15.133:3300/0 conn(0x7f64b40cda10 0x7f64b40caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.320+0000 7f64aaffd700 1 -- 172.21.15.78:0/482007356 >> v1:172.21.15.133:6789/0 conn(0x7f64b4164b50 legacy=0x7f64b406bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:10.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.320+0000 7f64aaffd700 1 -- 172.21.15.78:0/482007356 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b40caa00 con 0x7f64b40cdde0 2023-10-11T09:42:10.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.320+0000 7f64aaffd700 1 --2- 172.21.15.78:0/482007356 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64b40cdde0 0x7f64b4164610 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f649c009e50 tx=0x7f649c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf1b9ce87f49b9a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:10.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.324+0000 7f64b2ffd700 1 -- 172.21.15.78:0/482007356 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649c00e070 con 0x7f64b40cdde0 2023-10-11T09:42:10.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.324+0000 7f64b2ffd700 1 -- 172.21.15.78:0/482007356 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f649c00b910 con 0x7f64b40cdde0 2023-10-11T09:42:10.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.324+0000 7f64b2ffd700 1 -- 172.21.15.78:0/482007356 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649c00bcc0 con 0x7f64b40cdde0 2023-10-11T09:42:10.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.324+0000 7f64b8d37700 1 -- 172.21.15.78:0/482007356 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64b40cdde0 msgr2=0x7f64b4164610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:10.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.324+0000 7f64b8d37700 1 --2- 172.21.15.78:0/482007356 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64b40cdde0 0x7f64b4164610 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f649c009e50 tx=0x7f649c00b330 comp rx=0 tx=0).stop 2023-10-11T09:42:10.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.324+0000 7f64b8d37700 1 -- 172.21.15.78:0/482007356 shutdown_connections 2023-10-11T09:42:10.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.324+0000 7f64b8d37700 1 --2- 172.21.15.78:0/482007356 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64b40cdde0 0x7f64b4164610 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.324+0000 7f64b8d37700 1 --2- 172.21.15.78:0/482007356 >> v2:172.21.15.133:3300/0 conn(0x7f64b40cda10 0x7f64b40caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.324+0000 7f64b8d37700 1 -- 172.21.15.78:0/482007356 >> 172.21.15.78:0/482007356 conn(0x7f64b40cea60 msgr2=0x7f64b40cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:10.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.324+0000 7f64b8d37700 1 -- 172.21.15.78:0/482007356 shutdown_connections 2023-10-11T09:42:10.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.324+0000 7f64b8d37700 1 -- 172.21.15.78:0/482007356 wait complete. 2023-10-11T09:42:10.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.324+0000 7f64b8d37700 1 Processor -- start 2023-10-11T09:42:10.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.328+0000 7f64b8d37700 1 -- start start 2023-10-11T09:42:10.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.328+0000 7f64b8d37700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64b40cda10 0x7f64b40b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:10.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.328+0000 7f64b8d37700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64b40cdde0 0x7f64b40b4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:10.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.328+0000 7f64b8d37700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64b4164b50 0x7f64b4144e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:10.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.328+0000 7f64b8d37700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b40b4850 con 0x7f64b40cda10 2023-10-11T09:42:10.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.328+0000 7f64b8d37700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b4152990 con 0x7f64b4164b50 2023-10-11T09:42:10.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.328+0000 7f64b8d37700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64b4152c70 con 0x7f64b40cdde0 2023-10-11T09:42:10.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.328+0000 7f64b37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64b40cda10 0x7f64b40b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:10.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.328+0000 7f64b37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64b40cda10 0x7f64b40b1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60660/0 (socket says 172.21.15.78:60660) 2023-10-11T09:42:10.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.328+0000 7f64b37fe700 1 -- 172.21.15.78:0/1664623583 learned_addr learned my addr 172.21.15.78:0/1664623583 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:10.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.328+0000 7f64aaffd700 1 --2- 172.21.15.78:0/1664623583 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64b40cdde0 0x7f64b40b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:10.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.328+0000 7f64b3fff700 1 --2- 172.21.15.78:0/1664623583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64b4164b50 0x7f64b4144e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:10.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.328+0000 7f64b37fe700 1 -- 172.21.15.78:0/1664623583 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64b40cdde0 msgr2=0x7f64b40b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:10.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.328+0000 7f64b37fe700 1 --2- 172.21.15.78:0/1664623583 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64b40cdde0 0x7f64b40b4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.328+0000 7f64b37fe700 1 -- 172.21.15.78:0/1664623583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64b4164b50 msgr2=0x7f64b4144e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:10.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.328+0000 7f64b37fe700 1 --2- 172.21.15.78:0/1664623583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64b4164b50 0x7f64b4144e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.328+0000 7f64b37fe700 1 -- 172.21.15.78:0/1664623583 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f649c009b30 con 0x7f64b40cda10 2023-10-11T09:42:10.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.328+0000 7f64b37fe700 1 --2- 172.21.15.78:0/1664623583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64b40cda10 0x7f64b40b1a10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f64a400e840 tx=0x7f64a400eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:10.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.328+0000 7f64b0ff9700 1 -- 172.21.15.78:0/1664623583 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a40095a0 con 0x7f64b40cda10 2023-10-11T09:42:10.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.328+0000 7f64b0ff9700 1 -- 172.21.15.78:0/1664623583 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a4013680 con 0x7f64b40cda10 2023-10-11T09:42:10.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.328+0000 7f64b8d37700 1 -- 172.21.15.78:0/1664623583 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64b40ce3b0 con 0x7f64b40cda10 2023-10-11T09:42:10.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.328+0000 7f64b0ff9700 1 -- 172.21.15.78:0/1664623583 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a40139f0 con 0x7f64b40cda10 2023-10-11T09:42:10.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.328+0000 7f64b8d37700 1 -- 172.21.15.78:0/1664623583 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64b4153230 con 0x7f64b40cda10 2023-10-11T09:42:10.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.332+0000 7f64b0ff9700 1 -- 172.21.15.78:0/1664623583 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f64a4013b50 con 0x7f64b40cda10 2023-10-11T09:42:10.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.336+0000 7f64b0ff9700 1 --2- 172.21.15.78:0/1664623583 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6494065ed0 0x7f6494068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:10.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.336+0000 7f64b0ff9700 1 -- 172.21.15.78:0/1664623583 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f64a4083720 con 0x7f64b40cda10 2023-10-11T09:42:10.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.336+0000 7f64aaffd700 1 --2- 172.21.15.78:0/1664623583 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6494065ed0 0x7f6494068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:10.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.336+0000 7f64bd222740 1 Processor -- start 2023-10-11T09:42:10.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.336+0000 7f64aaffd700 1 --2- 172.21.15.78:0/1664623583 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6494065ed0 0x7f6494068380 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f649c00b370 tx=0x7f649c009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:10.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.336+0000 7f64bd222740 1 -- start start 2023-10-11T09:42:10.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.340+0000 7f64bd222740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1f47eb0 0x2200020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:10.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.340+0000 7f64bd222740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1f47ae0 0x2200560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:10.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.340+0000 7f64bd222740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1f47710 0x21c7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:10.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.340+0000 7f64bd222740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x21274a0 con 0x1f47eb0 2023-10-11T09:42:10.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.340+0000 7f64bd222740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x2124c30 con 0x1f47ae0 2023-10-11T09:42:10.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.340+0000 7f64bd222740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x2120520 con 0x1f47710 2023-10-11T09:42:10.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.340+0000 7f64b37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1f47eb0 0x2200020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:10.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.340+0000 7f64b37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1f47eb0 0x2200020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60674/0 (socket says 172.21.15.78:60674) 2023-10-11T09:42:10.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.340+0000 7f64b37fe700 1 -- 172.21.15.78:0/1940923920 learned_addr learned my addr 172.21.15.78:0/1940923920 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:10.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.340+0000 7f64aaffd700 1 --2- 172.21.15.78:0/1940923920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1f47ae0 0x2200560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:10.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.340+0000 7f64b3fff700 1 --2- 172.21.15.78:0/1940923920 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1f47710 0x21c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:10.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.340+0000 7f64b37fe700 1 -- 172.21.15.78:0/1940923920 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1f47710 msgr2=0x21c7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:10.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.340+0000 7f64b37fe700 1 --2- 172.21.15.78:0/1940923920 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1f47710 0x21c7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.340+0000 7f64b37fe700 1 -- 172.21.15.78:0/1940923920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1f47ae0 msgr2=0x2200560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:10.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.340+0000 7f64b37fe700 1 --2- 172.21.15.78:0/1940923920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1f47ae0 0x2200560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.340+0000 7f64b37fe700 1 -- 172.21.15.78:0/1940923920 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1f939e0 con 0x1f47eb0 2023-10-11T09:42:10.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.340+0000 7f64aaffd700 1 --2- 172.21.15.78:0/1940923920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1f47ae0 0x2200560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:42:10.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.340+0000 7f64b3fff700 1 --2- 172.21.15.78:0/1940923920 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1f47710 0x21c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:42:10.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.340+0000 7f64b37fe700 1 --2- 172.21.15.78:0/1940923920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1f47eb0 0x2200020 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f64a4081a90 tx=0x7f64a40544c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d95254aff55fca server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:10.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.340+0000 7f64a97fa700 1 -- 172.21.15.78:0/1940923920 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a4052b70 con 0x1f47eb0 2023-10-11T09:42:10.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.340+0000 7f64a97fa700 1 -- 172.21.15.78:0/1940923920 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a4052dd0 con 0x1f47eb0 2023-10-11T09:42:10.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.340+0000 7f64a97fa700 1 -- 172.21.15.78:0/1940923920 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a4054eb0 con 0x1f47eb0 2023-10-11T09:42:10.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.340+0000 7f64bd222740 1 -- 172.21.15.78:0/1940923920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1f47eb0 msgr2=0x2200020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:10.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.340+0000 7f64bd222740 1 --2- 172.21.15.78:0/1940923920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1f47eb0 0x2200020 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f64a4081a90 tx=0x7f64a40544c0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.340+0000 7f64bd222740 1 -- 172.21.15.78:0/1940923920 shutdown_connections 2023-10-11T09:42:10.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.340+0000 7f64bd222740 1 --2- 172.21.15.78:0/1940923920 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1f47710 0x21c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.340+0000 7f64bd222740 1 --2- 172.21.15.78:0/1940923920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1f47ae0 0x2200560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.340+0000 7f64bd222740 1 --2- 172.21.15.78:0/1940923920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1f47eb0 0x2200020 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.340+0000 7f64bd222740 1 -- 172.21.15.78:0/1940923920 >> 172.21.15.78:0/1940923920 conn(0x21affa0 msgr2=0x21e1420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:10.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.340+0000 7f64bd222740 1 -- 172.21.15.78:0/1940923920 shutdown_connections 2023-10-11T09:42:10.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.340+0000 7f64bd222740 1 -- 172.21.15.78:0/1940923920 wait complete. 2023-10-11T09:42:10.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.344+0000 7f64bd222740 1 -- start start 2023-10-11T09:42:10.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.344+0000 7f64bd222740 1 Processor -- start 2023-10-11T09:42:10.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.344+0000 7f64bd222740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1f47710 0x21d9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:10.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.344+0000 7f64bd222740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1f47ae0 0x21f59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:10.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.344+0000 7f64bd222740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1f47eb0 0x21f5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:10.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.344+0000 7f64bd222740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x20e3b00 con 0x1f47710 2023-10-11T09:42:10.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.344+0000 7f64bd222740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x20e37a0 con 0x1f47ae0 2023-10-11T09:42:10.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.344+0000 7f64bd222740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x1fccb20 con 0x1f47eb0 2023-10-11T09:42:10.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.344+0000 7f64bd222740 10 client.0 _ll_register_callbacks cb 0x2119e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:42:10.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.344+0000 7f64b37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1f47710 0x21d9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:10.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.344+0000 7f64b37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1f47710 0x21d9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60688/0 (socket says 172.21.15.78:60688) 2023-10-11T09:42:10.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.344+0000 7f64b37fe700 1 -- 172.21.15.78:0/1479604890 learned_addr learned my addr 172.21.15.78:0/1479604890 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:10.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.344+0000 7f64aaffd700 1 --2- 172.21.15.78:0/1479604890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1f47ae0 0x21f59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:10.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.344+0000 7f64b37fe700 1 -- 172.21.15.78:0/1479604890 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1f47eb0 msgr2=0x21f5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:10.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.344+0000 7f64b37fe700 1 --2- 172.21.15.78:0/1479604890 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1f47eb0 0x21f5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.344+0000 7f64b37fe700 1 -- 172.21.15.78:0/1479604890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1f47ae0 msgr2=0x21f59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:10.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.344+0000 7f64b37fe700 1 --2- 172.21.15.78:0/1479604890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1f47ae0 0x21f59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.344+0000 7f64b37fe700 1 -- 172.21.15.78:0/1479604890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64b4153230 con 0x1f47710 2023-10-11T09:42:10.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.344+0000 7f64aaffd700 1 --2- 172.21.15.78:0/1479604890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1f47ae0 0x21f59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:42:10.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.344+0000 7f64b37fe700 1 --2- 172.21.15.78:0/1479604890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1f47710 0x21d9f10 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f64a4083010 tx=0x7f64a4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c819f77d216d93 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:10.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.344+0000 7f64927fc700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:42:10.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.344+0000 7f64927fc700 1 -- 172.21.15.78:0/1479604890 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a4013e00 con 0x1f47710 2023-10-11T09:42:10.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.344+0000 7f64bd222740 1 -- 172.21.15.78:0/1479604890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x21195a0 con 0x1f47710 2023-10-11T09:42:10.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.344+0000 7f64927fc700 1 -- 172.21.15.78:0/1479604890 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f64a4054ea0 con 0x1f47710 2023-10-11T09:42:10.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.344+0000 7f64927fc700 1 -- 172.21.15.78:0/1479604890 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64a405c580 con 0x1f47710 2023-10-11T09:42:10.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.344+0000 7f64bd222740 10 client.5574 fetch_fsmap learned FSMap version 16 2023-10-11T09:42:10.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.344+0000 7f64bd222740 1 -- 172.21.15.78:0/1479604890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x20e6750 con 0x1f47710 2023-10-11T09:42:10.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.344+0000 7f64927fc700 1 -- 172.21.15.78:0/1479604890 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x21195a0 con 0x1f47710 2023-10-11T09:42:10.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.348+0000 7f64927fc700 1 -- 172.21.15.78:0/1479604890 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 16) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f64a4072af0 con 0x1f47710 2023-10-11T09:42:10.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.348+0000 7f64bd222740 10 client.5574 fetch_fsmap finished waiting for FSMap version 16 2023-10-11T09:42:10.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.348+0000 7f64bd222740 10 client.5574 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-11T09:42:10.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.348+0000 7f64bd222740 20 client.5574 populate_metadata read hostname 'smithi078' 2023-10-11T09:42:10.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.348+0000 7f64bd222740 1 --2- 172.21.15.78:0/1479604890 >> [v2:172.21.15.133:6832/1862542010,v1:172.21.15.133:6833/1862542010] conn(0x1f92790 0x21dcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:10.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.348+0000 7f64bd222740 4 client.5574 mds_command: new command op to 5364 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-11T09:42:10.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.348+0000 7f64bd222740 1 -- 172.21.15.78:0/1479604890 --> [v2:172.21.15.133:6832/1862542010,v1:172.21.15.133:6833/1862542010] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x21192c0 con 0x1f92790 2023-10-11T09:42:10.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.348+0000 7f64aaffd700 1 --2- 172.21.15.78:0/1479604890 >> [v2:172.21.15.133:6832/1862542010,v1:172.21.15.133:6833/1862542010] conn(0x1f92790 0x21dcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:10.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.348+0000 7f64aaffd700 1 --2- 172.21.15.78:0/1479604890 >> [v2:172.21.15.133:6832/1862542010,v1:172.21.15.133:6833/1862542010] conn(0x1f92790 0x21dcfc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69fcf7a915dcc6d0 server_cookie=bceb2821f185cecd in_seq=0 out_seq=0 2023-10-11T09:42:10.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.348+0000 7f64927fc700 10 client.5574 ms_handle_connect on v2:172.21.15.133:6832/1862542010 2023-10-11T09:42:10.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.348+0000 7f64927fc700 1 -- 172.21.15.78:0/1479604890 <== mds.0 v2:172.21.15.133:6832/1862542010 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x21192c0 con 0x1f92790 2023-10-11T09:42:10.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.348+0000 7f64927fc700 10 client.5574 handle_command_reply: tid=0 2023-10-11T09:42:10.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.348+0000 7f64bd222740 1 client.5574 shutdown 2023-10-11T09:42:10.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.352+0000 7f64bd222740 1 -- 172.21.15.78:0/1479604890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1f47710 msgr2=0x21d9f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:10.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.352+0000 7f64bd222740 1 --2- 172.21.15.78:0/1479604890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1f47710 0x21d9f10 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f64a4083010 tx=0x7f64a4000bc0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.352+0000 7f64bd222740 1 -- 172.21.15.78:0/1479604890 shutdown_connections 2023-10-11T09:42:10.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.352+0000 7f64bd222740 1 --2- 172.21.15.78:0/1479604890 >> [v2:172.21.15.133:6832/1862542010,v1:172.21.15.133:6833/1862542010] conn(0x1f92790 0x21dcfc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.352+0000 7f64bd222740 1 --2- 172.21.15.78:0/1479604890 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1f47eb0 0x21f5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.352+0000 7f64bd222740 1 --2- 172.21.15.78:0/1479604890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1f47ae0 0x21f59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.352+0000 7f64bd222740 1 --2- 172.21.15.78:0/1479604890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1f47710 0x21d9f10 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.352+0000 7f64927fc700 0 client.5574 ms_handle_reset on v2:172.21.15.133:6832/1862542010 2023-10-11T09:42:10.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.352+0000 7f64bd222740 1 -- 172.21.15.78:0/1479604890 >> 172.21.15.78:0/1479604890 conn(0x21affa0 msgr2=0x21d4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:10.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.352+0000 7f64bd222740 1 -- 172.21.15.78:0/1479604890 shutdown_connections 2023-10-11T09:42:10.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.352+0000 7f64bd222740 1 -- 172.21.15.78:0/1479604890 wait complete. 2023-10-11T09:42:10.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.352+0000 7f64bd222740 20 client.5574 trim_cache size 0 max 16384 2023-10-11T09:42:10.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.372+0000 7f64bd222740 1 Processor -- start 2023-10-11T09:42:10.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.372+0000 7f64bd222740 1 -- start start 2023-10-11T09:42:10.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.372+0000 7f64bd222740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1f47710 0x21f2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:10.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.372+0000 7f64bd222740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1f47ae0 0x21f3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:10.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.372+0000 7f64bd222740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1f47eb0 0x21d51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:10.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.372+0000 7f64bd222740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x20da760 con 0x1f47eb0 2023-10-11T09:42:10.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.372+0000 7f64bd222740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x1fccb20 con 0x1f47710 2023-10-11T09:42:10.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.372+0000 7f64bd222740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x20e55c0 con 0x1f47ae0 2023-10-11T09:42:10.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.372+0000 7f64b3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1f47eb0 0x21d51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:10.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.372+0000 7f64b3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1f47eb0 0x21d51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60694/0 (socket says 172.21.15.78:60694) 2023-10-11T09:42:10.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.372+0000 7f64b3fff700 1 -- 172.21.15.78:0/731345846 learned_addr learned my addr 172.21.15.78:0/731345846 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:10.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.372+0000 7f64b37fe700 1 --2- 172.21.15.78:0/731345846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1f47710 0x21f2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:10.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.372+0000 7f64aaffd700 1 --2- 172.21.15.78:0/731345846 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1f47ae0 0x21f3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:10.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.372+0000 7f64b3fff700 1 -- 172.21.15.78:0/731345846 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1f47ae0 msgr2=0x21f3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:10.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.372+0000 7f64b3fff700 1 --2- 172.21.15.78:0/731345846 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1f47ae0 0x21f3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.372+0000 7f64b3fff700 1 -- 172.21.15.78:0/731345846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1f47710 msgr2=0x21f2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:10.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.372+0000 7f64b3fff700 1 --2- 172.21.15.78:0/731345846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1f47710 0x21f2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.372+0000 7f64b3fff700 1 -- 172.21.15.78:0/731345846 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b4145a90 con 0x1f47eb0 2023-10-11T09:42:10.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.372+0000 7f64b3fff700 1 --2- 172.21.15.78:0/731345846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1f47eb0 0x21d51f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f64ac0014d0 tx=0x7f64ac0017e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd7b8f147ff1a8fa server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:10.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.376+0000 7f64a9ffb700 1 -- 172.21.15.78:0/731345846 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ac005320 con 0x1f47eb0 2023-10-11T09:42:10.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.376+0000 7f64a9ffb700 1 -- 172.21.15.78:0/731345846 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64ac00d070 con 0x1f47eb0 2023-10-11T09:42:10.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.376+0000 7f64a9ffb700 1 -- 172.21.15.78:0/731345846 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ac0036a0 con 0x1f47eb0 2023-10-11T09:42:10.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.376+0000 7f64bd222740 1 -- 172.21.15.78:0/731345846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1f47eb0 msgr2=0x21d51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:10.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.376+0000 7f64bd222740 1 --2- 172.21.15.78:0/731345846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1f47eb0 0x21d51f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f64ac0014d0 tx=0x7f64ac0017e0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.376+0000 7f64bd222740 1 -- 172.21.15.78:0/731345846 shutdown_connections 2023-10-11T09:42:10.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.376+0000 7f64bd222740 1 --2- 172.21.15.78:0/731345846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1f47eb0 0x21d51f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.376+0000 7f64bd222740 1 --2- 172.21.15.78:0/731345846 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1f47ae0 0x21f3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.376+0000 7f64bd222740 1 --2- 172.21.15.78:0/731345846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1f47710 0x21f2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.376+0000 7f64bd222740 1 -- 172.21.15.78:0/731345846 >> 172.21.15.78:0/731345846 conn(0x21affa0 msgr2=0x21f26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:10.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.376+0000 7f64bd222740 1 -- 172.21.15.78:0/731345846 shutdown_connections 2023-10-11T09:42:10.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.376+0000 7f64bd222740 1 -- 172.21.15.78:0/731345846 wait complete. 2023-10-11T09:42:10.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.376+0000 7f64bd222740 1 -- start start 2023-10-11T09:42:10.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.376+0000 7f64bd222740 1 Processor -- start 2023-10-11T09:42:10.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.380+0000 7f64bd222740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1f47710 0x225bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:10.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.380+0000 7f64bd222740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1f47ae0 0x2215fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:10.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.380+0000 7f64bd222740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1f47eb0 0x2216510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:10.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.380+0000 7f64bd222740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x20efcd0 con 0x1f47eb0 2023-10-11T09:42:10.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.380+0000 7f64bd222740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x20e37a0 con 0x1f47ae0 2023-10-11T09:42:10.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.380+0000 7f64bd222740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x2121ab0 con 0x1f47710 2023-10-11T09:42:10.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.380+0000 7f64bd222740 10 client.0 _ll_register_callbacks cb 0x211ec80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:42:10.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.380+0000 7f64b3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1f47eb0 0x2216510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:10.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.380+0000 7f64b3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1f47eb0 0x2216510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60706/0 (socket says 172.21.15.78:60706) 2023-10-11T09:42:10.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.380+0000 7f64b3fff700 1 -- 172.21.15.78:0/2093318074 learned_addr learned my addr 172.21.15.78:0/2093318074 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:10.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.380+0000 7f64aaffd700 1 --2- 172.21.15.78:0/2093318074 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1f47ae0 0x2215fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:10.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.380+0000 7f64b37fe700 1 --2- 172.21.15.78:0/2093318074 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1f47710 0x225bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:10.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.380+0000 7f64b3fff700 1 -- 172.21.15.78:0/2093318074 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1f47710 msgr2=0x225bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:10.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.380+0000 7f64b3fff700 1 --2- 172.21.15.78:0/2093318074 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1f47710 0x225bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.380+0000 7f64b3fff700 1 -- 172.21.15.78:0/2093318074 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1f47ae0 msgr2=0x2215fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:10.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.380+0000 7f64b3fff700 1 --2- 172.21.15.78:0/2093318074 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1f47ae0 0x2215fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.380+0000 7f64b3fff700 1 -- 172.21.15.78:0/2093318074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64b4165390 con 0x1f47eb0 2023-10-11T09:42:10.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.380+0000 7f64b3fff700 1 --2- 172.21.15.78:0/2093318074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1f47eb0 0x2216510 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f64ac0038d0 tx=0x7f64ac009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7272d517583e5cef server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:10.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.380+0000 7f64927fc700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:42:10.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.380+0000 7f64927fc700 1 -- 172.21.15.78:0/2093318074 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ac003e40 con 0x1f47eb0 2023-10-11T09:42:10.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.380+0000 7f64927fc700 1 -- 172.21.15.78:0/2093318074 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f64ac007be0 con 0x1f47eb0 2023-10-11T09:42:10.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.380+0000 7f64bd222740 1 -- 172.21.15.78:0/2093318074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f64a4052dd0 con 0x1f47eb0 2023-10-11T09:42:10.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.380+0000 7f64927fc700 1 -- 172.21.15.78:0/2093318074 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64ac014450 con 0x1f47eb0 2023-10-11T09:42:10.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.380+0000 7f64bd222740 10 client.5580 fetch_fsmap learned FSMap version 16 2023-10-11T09:42:10.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.380+0000 7f64bd222740 1 -- 172.21.15.78:0/2093318074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x20cc980 con 0x1f47eb0 2023-10-11T09:42:10.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.380+0000 7f64927fc700 1 -- 172.21.15.78:0/2093318074 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a4052dd0 con 0x1f47eb0 2023-10-11T09:42:10.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.380+0000 7f64927fc700 1 -- 172.21.15.78:0/2093318074 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 16) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f64ac012070 con 0x1f47eb0 2023-10-11T09:42:10.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.380+0000 7f64bd222740 10 client.5580 fetch_fsmap finished waiting for FSMap version 16 2023-10-11T09:42:10.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.380+0000 7f64bd222740 10 client.5580 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-11T09:42:10.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.380+0000 7f64bd222740 20 client.5580 populate_metadata read hostname 'smithi078' 2023-10-11T09:42:10.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.380+0000 7f64bd222740 1 --2- 172.21.15.78:0/2093318074 >> [v2:172.21.15.133:6832/1862542010,v1:172.21.15.133:6833/1862542010] conn(0x2212040 0x2212410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:10.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.380+0000 7f64bd222740 4 client.5580 mds_command: new command op to 5364 tid=0[{"prefix": "status"}] 2023-10-11T09:42:10.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.380+0000 7f64bd222740 1 -- 172.21.15.78:0/2093318074 --> [v2:172.21.15.133:6832/1862542010,v1:172.21.15.133:6833/1862542010] -- command(tid 0: {"prefix": "status"}) v1 -- 0x1f9f4e0 con 0x2212040 2023-10-11T09:42:10.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.384+0000 7f64b37fe700 1 --2- 172.21.15.78:0/2093318074 >> [v2:172.21.15.133:6832/1862542010,v1:172.21.15.133:6833/1862542010] conn(0x2212040 0x2212410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:10.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.384+0000 7f64b37fe700 1 --2- 172.21.15.78:0/2093318074 >> [v2:172.21.15.133:6832/1862542010,v1:172.21.15.133:6833/1862542010] conn(0x2212040 0x2212410 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=731cf49e178d4dfe server_cookie=e438059c23142f53 in_seq=0 out_seq=0 2023-10-11T09:42:10.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.384+0000 7f64927fc700 10 client.5580 ms_handle_connect on v2:172.21.15.133:6832/1862542010 2023-10-11T09:42:10.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.384+0000 7f64927fc700 1 -- 172.21.15.78:0/2093318074 <== mds.0 v2:172.21.15.133:6832/1862542010 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x1f9f4e0 con 0x2212040 2023-10-11T09:42:10.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.384+0000 7f64927fc700 10 client.5580 handle_command_reply: tid=0 2023-10-11T09:42:10.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.384+0000 7f64bd222740 1 client.5580 shutdown 2023-10-11T09:42:10.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.384+0000 7f64bd222740 1 -- 172.21.15.78:0/2093318074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1f47eb0 msgr2=0x2216510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:10.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.384+0000 7f64bd222740 1 --2- 172.21.15.78:0/2093318074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1f47eb0 0x2216510 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f64ac0038d0 tx=0x7f64ac009040 comp rx=0 tx=0).stop 2023-10-11T09:42:10.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.384+0000 7f64bd222740 1 -- 172.21.15.78:0/2093318074 shutdown_connections 2023-10-11T09:42:10.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.384+0000 7f64bd222740 1 --2- 172.21.15.78:0/2093318074 >> [v2:172.21.15.133:6832/1862542010,v1:172.21.15.133:6833/1862542010] conn(0x2212040 0x2212410 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.384+0000 7f64bd222740 1 --2- 172.21.15.78:0/2093318074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1f47eb0 0x2216510 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.384+0000 7f64bd222740 1 --2- 172.21.15.78:0/2093318074 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1f47ae0 0x2215fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.384+0000 7f64bd222740 1 --2- 172.21.15.78:0/2093318074 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1f47710 0x225bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.384+0000 7f64927fc700 0 client.5580 ms_handle_reset on v2:172.21.15.133:6832/1862542010 2023-10-11T09:42:10.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.384+0000 7f64bd222740 1 -- 172.21.15.78:0/2093318074 >> 172.21.15.78:0/2093318074 conn(0x21affa0 msgr2=0x21ba000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:10.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.384+0000 7f64bd222740 1 -- 172.21.15.78:0/2093318074 shutdown_connections 2023-10-11T09:42:10.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.384+0000 7f64bd222740 1 -- 172.21.15.78:0/2093318074 wait complete. 2023-10-11T09:42:10.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.384+0000 7f64bd222740 20 client.5580 trim_cache size 0 max 16384 2023-10-11T09:42:10.400 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:42:10.401 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "dee99db9-7a72-406f-a1e2-917943f2b13d", 2023-10-11T09:42:10.401 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2023-10-11T09:42:10.401 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 5364, 2023-10-11T09:42:10.401 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2023-10-11T09:42:10.401 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2023-10-11T09:42:10.402 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2023-10-11T09:42:10.402 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 11.158260896, 2023-10-11T09:42:10.402 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 16, 2023-10-11T09:42:10.402 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 37, 2023-10-11T09:42:10.402 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 37, 2023-10-11T09:42:10.403 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 19.47230253 2023-10-11T09:42:10.403 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:42:10.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.384+0000 7f64b8d37700 1 -- 172.21.15.78:0/1664623583 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6494065ed0 msgr2=0x7f6494068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:10.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.384+0000 7f64b8d37700 1 --2- 172.21.15.78:0/1664623583 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6494065ed0 0x7f6494068380 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f649c00b370 tx=0x7f649c009000 comp rx=0 tx=0).stop 2023-10-11T09:42:10.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.384+0000 7f64b8d37700 1 -- 172.21.15.78:0/1664623583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64b40cda10 msgr2=0x7f64b40b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:10.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.384+0000 7f64b8d37700 1 --2- 172.21.15.78:0/1664623583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64b40cda10 0x7f64b40b1a10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f64a400e840 tx=0x7f64a400eb50 comp rx=0 tx=0).stop 2023-10-11T09:42:10.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.384+0000 7f64b8d37700 1 -- 172.21.15.78:0/1664623583 shutdown_connections 2023-10-11T09:42:10.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.384+0000 7f64b8d37700 1 --2- 172.21.15.78:0/1664623583 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6494065ed0 0x7f6494068380 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.384+0000 7f64b8d37700 1 --2- 172.21.15.78:0/1664623583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64b4164b50 0x7f64b4144e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.388+0000 7f64b8d37700 1 --2- 172.21.15.78:0/1664623583 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64b40cdde0 0x7f64b40b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.388+0000 7f64b8d37700 1 --2- 172.21.15.78:0/1664623583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64b40cda10 0x7f64b40b1a10 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.388+0000 7f64b8d37700 1 -- 172.21.15.78:0/1664623583 >> 172.21.15.78:0/1664623583 conn(0x7f64b40cea60 msgr2=0x7f64b4146dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:10.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.388+0000 7f64b8d37700 1 -- 172.21.15.78:0/1664623583 shutdown_connections 2023-10-11T09:42:10.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.388+0000 7f64b8d37700 1 -- 172.21.15.78:0/1664623583 wait complete. 2023-10-11T09:42:10.416 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-11T09:42:10.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.576+0000 7fc026298700 1 Processor -- start 2023-10-11T09:42:10.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.580+0000 7fc026298700 1 -- start start 2023-10-11T09:42:10.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.580+0000 7fc026298700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc02013f740 0x7fc0200ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:10.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.580+0000 7fc026298700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc02006cd00 0x7fc02006d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:10.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.580+0000 7fc026298700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fc0200b06a0 0x7fc0200b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:10.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.580+0000 7fc026298700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc02006d690 con 0x7fc0200b06a0 2023-10-11T09:42:10.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.580+0000 7fc026298700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0200ce4c0 con 0x7fc02013f740 2023-10-11T09:42:10.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.580+0000 7fc026298700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0200cd420 con 0x7fc02006cd00 2023-10-11T09:42:10.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.580+0000 7fc025a97700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fc0200b06a0 0x7fc0200b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:10.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.580+0000 7fc025a97700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fc0200b06a0 0x7fc0200b0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60718/0 (socket says 172.21.15.78:60718) 2023-10-11T09:42:10.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.580+0000 7fc025a97700 1 -- 172.21.15.78:0/3033964506 learned_addr learned my addr 172.21.15.78:0/3033964506 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:10.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.580+0000 7fc024a95700 1 --2- 172.21.15.78:0/3033964506 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc02006cd00 0x7fc02006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:10.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.580+0000 7fc025296700 1 --2- 172.21.15.78:0/3033964506 >> v2:172.21.15.133:3300/0 conn(0x7fc02013f740 0x7fc0200ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:10.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.580+0000 7fc025a97700 1 -- 172.21.15.78:0/3033964506 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc02006cd00 msgr2=0x7fc02006d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:10.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.580+0000 7fc025a97700 1 --2- 172.21.15.78:0/3033964506 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc02006cd00 0x7fc02006d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.580+0000 7fc025a97700 1 -- 172.21.15.78:0/3033964506 >> v2:172.21.15.133:3300/0 conn(0x7fc02013f740 msgr2=0x7fc0200ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:10.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.580+0000 7fc025a97700 1 --2- 172.21.15.78:0/3033964506 >> v2:172.21.15.133:3300/0 conn(0x7fc02013f740 0x7fc0200ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.580+0000 7fc025a97700 1 -- 172.21.15.78:0/3033964506 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0200cd700 con 0x7fc0200b06a0 2023-10-11T09:42:10.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.580+0000 7fc025a97700 1 --2- 172.21.15.78:0/3033964506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0200b06a0 0x7fc0200b0b20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc02006fb70 tx=0x7fc01800d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2f6cf848790af9 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:10.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.584+0000 7fc017fff700 1 -- 172.21.15.78:0/3033964506 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc018010070 con 0x7fc0200b06a0 2023-10-11T09:42:10.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.584+0000 7fc017fff700 1 -- 172.21.15.78:0/3033964506 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc01800dd90 con 0x7fc0200b06a0 2023-10-11T09:42:10.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.584+0000 7fc017fff700 1 -- 172.21.15.78:0/3033964506 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01800b310 con 0x7fc0200b06a0 2023-10-11T09:42:10.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.584+0000 7fc026298700 1 -- 172.21.15.78:0/3033964506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0200b06a0 msgr2=0x7fc0200b0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:10.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.584+0000 7fc026298700 1 --2- 172.21.15.78:0/3033964506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0200b06a0 0x7fc0200b0b20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc02006fb70 tx=0x7fc01800d430 comp rx=0 tx=0).stop 2023-10-11T09:42:10.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.584+0000 7fc026298700 1 -- 172.21.15.78:0/3033964506 shutdown_connections 2023-10-11T09:42:10.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.584+0000 7fc026298700 1 --2- 172.21.15.78:0/3033964506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0200b06a0 0x7fc0200b0b20 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.584+0000 7fc026298700 1 --2- 172.21.15.78:0/3033964506 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc02006cd00 0x7fc02006d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.584+0000 7fc026298700 1 --2- 172.21.15.78:0/3033964506 >> v2:172.21.15.133:3300/0 conn(0x7fc02013f740 0x7fc0200ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.584+0000 7fc026298700 1 -- 172.21.15.78:0/3033964506 >> 172.21.15.78:0/3033964506 conn(0x7fc0200cea60 msgr2=0x7fc0200cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:10.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.584+0000 7fc026298700 1 -- 172.21.15.78:0/3033964506 shutdown_connections 2023-10-11T09:42:10.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.584+0000 7fc026298700 1 -- 172.21.15.78:0/3033964506 wait complete. 2023-10-11T09:42:10.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.588+0000 7fc026298700 1 Processor -- start 2023-10-11T09:42:10.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.588+0000 7fc026298700 1 -- start start 2023-10-11T09:42:10.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.588+0000 7fc026298700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc02006cd00 0x7fc0201f5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:10.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.588+0000 7fc026298700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc0200b06a0 0x7fc02015af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:10.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.588+0000 7fc026298700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc02013f740 0x7fc0200b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:10.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.588+0000 7fc026298700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0201f6440 con 0x7fc02013f740 2023-10-11T09:42:10.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.588+0000 7fc026298700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc020139e70 con 0x7fc02006cd00 2023-10-11T09:42:10.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.588+0000 7fc026298700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc02013a710 con 0x7fc0200b06a0 2023-10-11T09:42:10.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.588+0000 7fc025296700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc02006cd00 0x7fc0201f5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:10.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.588+0000 7fc025a97700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc02013f740 0x7fc0200b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:10.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.588+0000 7fc024a95700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc0200b06a0 0x7fc02015af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:10.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.588+0000 7fc025a97700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc02013f740 0x7fc0200b3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60724/0 (socket says 172.21.15.78:60724) 2023-10-11T09:42:10.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.588+0000 7fc025296700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc02006cd00 0x7fc0201f5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:58438/0 (socket says 172.21.15.78:58438) 2023-10-11T09:42:10.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.588+0000 7fc025a97700 1 -- 172.21.15.78:0/1659502948 learned_addr learned my addr 172.21.15.78:0/1659502948 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:10.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.588+0000 7fc025a97700 1 -- 172.21.15.78:0/1659502948 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc0200b06a0 msgr2=0x7fc02015af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:10.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.588+0000 7fc025a97700 1 --2- 172.21.15.78:0/1659502948 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc0200b06a0 0x7fc02015af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.588+0000 7fc025a97700 1 -- 172.21.15.78:0/1659502948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc02006cd00 msgr2=0x7fc0201f5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:10.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.588+0000 7fc025a97700 1 --2- 172.21.15.78:0/1659502948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc02006cd00 0x7fc0201f5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.588+0000 7fc025a97700 1 -- 172.21.15.78:0/1659502948 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc01800bd40 con 0x7fc02013f740 2023-10-11T09:42:10.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.588+0000 7fc025a97700 1 --2- 172.21.15.78:0/1659502948 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc02013f740 0x7fc0200b3fc0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc01800d830 tx=0x7fc018004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:10.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.592+0000 7fc015ffb700 1 -- 172.21.15.78:0/1659502948 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc018009a10 con 0x7fc02013f740 2023-10-11T09:42:10.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.592+0000 7fc015ffb700 1 -- 172.21.15.78:0/1659502948 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc018009b90 con 0x7fc02013f740 2023-10-11T09:42:10.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.592+0000 7fc026298700 1 -- 172.21.15.78:0/1659502948 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0200b3dc0 con 0x7fc02013f740 2023-10-11T09:42:10.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.592+0000 7fc015ffb700 1 -- 172.21.15.78:0/1659502948 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01800b4c0 con 0x7fc02013f740 2023-10-11T09:42:10.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.592+0000 7fc026298700 1 -- 172.21.15.78:0/1659502948 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc02013a0c0 con 0x7fc02013f740 2023-10-11T09:42:10.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.592+0000 7fc026298700 1 -- 172.21.15.78:0/1659502948 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc020065d40 con 0x7fc02013f740 2023-10-11T09:42:10.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.592+0000 7fc015ffb700 1 -- 172.21.15.78:0/1659502948 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fc018004040 con 0x7fc02013f740 2023-10-11T09:42:10.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.596+0000 7fc015ffb700 1 --2- 172.21.15.78:0/1659502948 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc004065ed0 0x7fc004068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:10.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.596+0000 7fc015ffb700 1 -- 172.21.15.78:0/1659502948 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc018084ca0 con 0x7fc02013f740 2023-10-11T09:42:10.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.596+0000 7fc025296700 1 --2- 172.21.15.78:0/1659502948 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc004065ed0 0x7fc004068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:10.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.596+0000 7fc025296700 1 --2- 172.21.15.78:0/1659502948 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc004065ed0 0x7fc004068380 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc01c009980 tx=0x7fc01c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:10.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.600+0000 7fc015ffb700 1 -- 172.21.15.78:0/1659502948 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fc018052390 con 0x7fc02013f740 2023-10-11T09:42:10.818 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.812+0000 7fc026298700 1 -- 172.21.15.78:0/1659502948 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc02013a850 con 0x7fc02013f740 2023-10-11T09:42:10.819 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.812+0000 7fc015ffb700 1 -- 172.21.15.78:0/1659502948 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc018021060 con 0x7fc02013f740 2023-10-11T09:42:10.819 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-11T09:42:10.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.816+0000 7fc026298700 1 -- 172.21.15.78:0/1659502948 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc004065ed0 msgr2=0x7fc004068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:10.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.816+0000 7fc026298700 1 --2- 172.21.15.78:0/1659502948 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc004065ed0 0x7fc004068380 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc01c009980 tx=0x7fc01c008040 comp rx=0 tx=0).stop 2023-10-11T09:42:10.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.816+0000 7fc026298700 1 -- 172.21.15.78:0/1659502948 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc02013f740 msgr2=0x7fc0200b3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:10.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.816+0000 7fc026298700 1 --2- 172.21.15.78:0/1659502948 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc02013f740 0x7fc0200b3fc0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc01800d830 tx=0x7fc018004af0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.816+0000 7fc026298700 1 -- 172.21.15.78:0/1659502948 shutdown_connections 2023-10-11T09:42:10.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.816+0000 7fc026298700 1 --2- 172.21.15.78:0/1659502948 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc004065ed0 0x7fc004068380 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.824 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.816+0000 7fc026298700 1 --2- 172.21.15.78:0/1659502948 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc02013f740 0x7fc0200b3fc0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.824 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.816+0000 7fc026298700 1 --2- 172.21.15.78:0/1659502948 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc0200b06a0 0x7fc02015af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.824 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.816+0000 7fc026298700 1 --2- 172.21.15.78:0/1659502948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc02006cd00 0x7fc0201f5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:10.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.816+0000 7fc026298700 1 -- 172.21.15.78:0/1659502948 >> 172.21.15.78:0/1659502948 conn(0x7fc0200cea60 msgr2=0x7fc02013c930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:10.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.816+0000 7fc026298700 1 -- 172.21.15.78:0/1659502948 shutdown_connections 2023-10-11T09:42:10.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:10.816+0000 7fc026298700 1 -- 172.21.15.78:0/1659502948 wait complete. 2023-10-11T09:42:10.842 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-11T09:42:10.843 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-11T09:42:10.843 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi078.front.sepia.ceph.com 2023-10-11T09:42:10.843 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-11T09:42:10.844 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-11T09:42:10.844 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-11T09:42:10.844 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-11T09:42:10.845 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-11T09:42:10.845 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-11T09:42:10.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:42:10.846 DEBUG:teuthology.orchestra.run.smithi078:> ip addr 2023-10-11T09:42:10.865 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-11T09:42:10.865 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-11T09:42:10.865 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2023-10-11T09:42:10.865 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:42:10.866 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2023-10-11T09:42:10.866 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:42:10.866 INFO:teuthology.orchestra.run.smithi078.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:42:10.866 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:22 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:42:10.866 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f0 2023-10-11T09:42:10.867 INFO:teuthology.orchestra.run.smithi078.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:42:10.867 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:23 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:42:10.867 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f1 2023-10-11T09:42:10.867 INFO:teuthology.orchestra.run.smithi078.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:42:10.867 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dc brd ff:ff:ff:ff:ff:ff 2023-10-11T09:42:10.867 INFO:teuthology.orchestra.run.smithi078.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-11T09:42:10.868 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dd brd ff:ff:ff:ff:ff:ff 2023-10-11T09:42:10.868 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.21.15.78/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-11T09:42:10.868 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft 42333sec preferred_lft 42333sec 2023-10-11T09:42:10.868 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::ec4:7aff:fe8f:d4dd/64 scope link 2023-10-11T09:42:10.868 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:42:10.869 INFO:teuthology.orchestra.run.smithi078.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-11T09:42:10.869 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 02:42:b6:e4:f8:0d brd ff:ff:ff:ff:ff:ff 2023-10-11T09:42:10.869 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-11T09:42:10.869 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:42:10.869 INFO:teuthology.orchestra.run.smithi078.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-11T09:42:10.870 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 7a:89:8a:a9:e1:23 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:42:10.870 INFO:teuthology.orchestra.run.smithi078.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-11T09:42:10.870 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:42:10.870 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::8ca:ebff:fea9:13f1/64 scope link 2023-10-11T09:42:10.870 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:42:10.871 INFO:teuthology.orchestra.run.smithi078.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-11T09:42:10.871 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 7a:89:8a:a9:e1:23 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-11T09:42:10.871 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::7889:8aff:fea9:e123/64 scope link 2023-10-11T09:42:10.871 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:42:10.871 INFO:teuthology.orchestra.run.smithi078.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-11T09:42:10.871 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether ca:ac:e8:27:51:0d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-11T09:42:10.872 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::c8ac:e8ff:fe27:510d/64 scope link 2023-10-11T09:42:10.872 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:42:10.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:42:10.873 DEBUG:teuthology.orchestra.run.smithi078:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-11T09:42:10.957 INFO:teuthology.orchestra.run.smithi078.stdout:1 2023-10-11T09:42:10.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:42:10.960 DEBUG:teuthology.orchestra.run.smithi078:> route 2023-10-11T09:42:11.022 INFO:teuthology.orchestra.run.smithi078.stdout:Kernel IP routing table 2023-10-11T09:42:11.022 INFO:teuthology.orchestra.run.smithi078.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-11T09:42:11.022 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-11T09:42:11.022 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-11T09:42:11.023 INFO:teuthology.orchestra.run.smithi078.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-11T09:42:11.023 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-10-11T09:42:11.023 INFO:teuthology.orchestra.run.smithi078.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-11T09:42:11.023 INFO:teuthology.orchestra.run.smithi078.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-11T09:42:11.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:42:11.025 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec bash -c ' 2023-10-11T09:42:11.025 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2023-10-11T09:42:11.025 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-11T09:42:11.026 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-11T09:42:11.026 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-11T09:42:11.027 DEBUG:teuthology.orchestra.run.smithi078:> ') 2023-10-11T09:42:11.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:42:11.159 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2023-10-11T09:42:11.178 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-11T09:42:11.178 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-11T09:42:11.179 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-11T09:42:11.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:42:11.180 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set brx.0 up 2023-10-11T09:42:11.261 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-11T09:42:11.262 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T09:42:11.262 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-11T09:42:11.322 INFO:teuthology.orchestra.run.smithi078.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-11T09:42:11.322 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T09:42:11.323 DEBUG:teuthology.orchestra.run.smithi078:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-11T09:42:11.386 DEBUG:teuthology.orchestra.run.smithi078:> sudo modprobe fuse 2023-10-11T09:42:11.463 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T09:42:11.464 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T09:42:11.538 INFO:teuthology.orchestra.run.smithi078.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T09:42:11.540 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T09:42:11.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:42:11.541 DEBUG:teuthology.orchestra.run.smithi078:> ls /sys/fs/fuse/connections 2023-10-11T09:42:11.606 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-11T09:42:11.607 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-11T09:42:11.655 DEBUG:teuthology.orchestra.run.smithi078:> sudo modprobe fuse 2023-10-11T09:42:11.695 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T09:42:11.695 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T09:42:11.770 INFO:teuthology.orchestra.run.smithi078.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T09:42:11.774 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T09:42:11.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:42:11.776 DEBUG:teuthology.orchestra.run.smithi078:> ls /sys/fs/fuse/connections 2023-10-11T09:42:11.796 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:2023-10-11T09:42:11.788+0000 7ffbd6a23640 -1 init, newargv = 0x557c5f21d870 newargc=15 2023-10-11T09:42:11.797 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:ceph-fuse[46411]: starting ceph client 2023-10-11T09:42:11.809 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:ceph-fuse[46411]: starting fuse 2023-10-11T09:42:11.810 INFO:teuthology.orchestra.run.smithi078.stdout:49 2023-10-11T09:42:11.811 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2023-10-11T09:42:11.811 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:42:11.812 DEBUG:teuthology.orchestra.run.smithi078:> import glob 2023-10-11T09:42:11.812 DEBUG:teuthology.orchestra.run.smithi078:> import re 2023-10-11T09:42:11.813 DEBUG:teuthology.orchestra.run.smithi078:> import os 2023-10-11T09:42:11.813 DEBUG:teuthology.orchestra.run.smithi078:> import subprocess 2023-10-11T09:42:11.814 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:42:11.814 DEBUG:teuthology.orchestra.run.smithi078:> def _find_admin_socket(client_name): 2023-10-11T09:42:11.815 DEBUG:teuthology.orchestra.run.smithi078:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-11T09:42:11.815 DEBUG:teuthology.orchestra.run.smithi078:> files = glob.glob(asok_path) 2023-10-11T09:42:11.817 DEBUG:teuthology.orchestra.run.smithi078:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-11T09:42:11.817 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:42:11.818 DEBUG:teuthology.orchestra.run.smithi078:> # Given a non-glob path, it better be there 2023-10-11T09:42:11.818 DEBUG:teuthology.orchestra.run.smithi078:> if "*" not in asok_path: 2023-10-11T09:42:11.818 DEBUG:teuthology.orchestra.run.smithi078:> assert(len(files) == 1) 2023-10-11T09:42:11.819 DEBUG:teuthology.orchestra.run.smithi078:> return files[0] 2023-10-11T09:42:11.819 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:42:11.819 DEBUG:teuthology.orchestra.run.smithi078:> for f in files: 2023-10-11T09:42:11.820 DEBUG:teuthology.orchestra.run.smithi078:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-11T09:42:11.820 DEBUG:teuthology.orchestra.run.smithi078:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-11T09:42:11.820 DEBUG:teuthology.orchestra.run.smithi078:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-11T09:42:11.821 DEBUG:teuthology.orchestra.run.smithi078:> contents = proc_f.read() 2023-10-11T09:42:11.821 DEBUG:teuthology.orchestra.run.smithi078:> if mountpoint in contents: 2023-10-11T09:42:11.822 DEBUG:teuthology.orchestra.run.smithi078:> return f 2023-10-11T09:42:11.822 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-11T09:42:11.822 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:42:11.823 DEBUG:teuthology.orchestra.run.smithi078:> print(_find_admin_socket("client.0")) 2023-10-11T09:42:11.823 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:42:11.971 INFO:teuthology.orchestra.run.smithi078.stdout:/var/run/ceph/ceph-client.0.46411.asok 2023-10-11T09:42:12.157 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.46411.asok 2023-10-11T09:42:12.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:42:12.158 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.46411.asok status 2023-10-11T09:42:12.355 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:42:12.355 INFO:teuthology.orchestra.run.smithi078.stdout: "metadata": { 2023-10-11T09:42:12.356 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_sha1": "9a774796f0ecdd11a4b771de396e627aa1a187af", 2023-10-11T09:42:12.356 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_version": "ceph version 18.2.0-662-g9a774796 (9a774796f0ecdd11a4b771de396e627aa1a187af) reef (stable)", 2023-10-11T09:42:12.356 INFO:teuthology.orchestra.run.smithi078.stdout: "entity_id": "0", 2023-10-11T09:42:12.356 INFO:teuthology.orchestra.run.smithi078.stdout: "hostname": "smithi078", 2023-10-11T09:42:12.356 INFO:teuthology.orchestra.run.smithi078.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-11T09:42:12.356 INFO:teuthology.orchestra.run.smithi078.stdout: "pid": "46411", 2023-10-11T09:42:12.357 INFO:teuthology.orchestra.run.smithi078.stdout: "root": "/" 2023-10-11T09:42:12.357 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:42:12.357 INFO:teuthology.orchestra.run.smithi078.stdout: "dentry_count": 0, 2023-10-11T09:42:12.357 INFO:teuthology.orchestra.run.smithi078.stdout: "dentry_pinned_count": 0, 2023-10-11T09:42:12.357 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 5595, 2023-10-11T09:42:12.358 INFO:teuthology.orchestra.run.smithi078.stdout: "inst": { 2023-10-11T09:42:12.358 INFO:teuthology.orchestra.run.smithi078.stdout: "name": { 2023-10-11T09:42:12.358 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "client", 2023-10-11T09:42:12.358 INFO:teuthology.orchestra.run.smithi078.stdout: "num": 5595 2023-10-11T09:42:12.358 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:42:12.359 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": { 2023-10-11T09:42:12.359 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2023-10-11T09:42:12.359 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "192.168.0.1:0", 2023-10-11T09:42:12.359 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 2365001196 2023-10-11T09:42:12.359 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:42:12.359 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:42:12.360 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": { 2023-10-11T09:42:12.360 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2023-10-11T09:42:12.360 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "192.168.0.1:0", 2023-10-11T09:42:12.360 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 2365001196 2023-10-11T09:42:12.360 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:42:12.361 INFO:teuthology.orchestra.run.smithi078.stdout: "inst_str": "client.5595 192.168.0.1:0/2365001196", 2023-10-11T09:42:12.361 INFO:teuthology.orchestra.run.smithi078.stdout: "addr_str": "192.168.0.1:0/2365001196", 2023-10-11T09:42:12.361 INFO:teuthology.orchestra.run.smithi078.stdout: "inode_count": 1, 2023-10-11T09:42:12.361 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_epoch": 16, 2023-10-11T09:42:12.361 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_epoch": 37, 2023-10-11T09:42:12.362 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_epoch_barrier": 0, 2023-10-11T09:42:12.362 INFO:teuthology.orchestra.run.smithi078.stdout: "blocklisted": false, 2023-10-11T09:42:12.362 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs" 2023-10-11T09:42:12.362 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:42:12.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:42:12.370 DEBUG:teuthology.orchestra.run.smithi078:> stat --file-system '--printf=%T 2023-10-11T09:42:12.370 DEBUG:teuthology.orchestra.run.smithi078:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-11T09:42:12.387 INFO:teuthology.orchestra.run.smithi078.stdout:fuseblk 2023-10-11T09:42:12.387 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-11T09:42:12.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:42:12.389 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-11T09:42:12.467 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_extra --pg_num_min 64 2023-10-11T09:42:12.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.688+0000 7f1f6af16700 1 Processor -- start 2023-10-11T09:42:12.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.688+0000 7f1f6af16700 1 -- start start 2023-10-11T09:42:12.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.688+0000 7f1f6af16700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1f640cdbd0 0x7f1f6406cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:12.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.688+0000 7f1f6af16700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f1f641655d0 0x7f1f640cdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:12.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.688+0000 7f1f6af16700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f64165140 con 0x7f1f640b0eb0 2023-10-11T09:42:12.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.688+0000 7f1f6af16700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f641648e0 con 0x7f1f641655d0 2023-10-11T09:42:12.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.688+0000 7f1f6af16700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f640cb2c0 con 0x7f1f640cdbd0 2023-10-11T09:42:12.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.688+0000 7f1f69713700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f1f641655d0 0x7f1f640cdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:12.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.688+0000 7f1f69713700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f1f641655d0 0x7f1f640cdfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60734/0 (socket says 172.21.15.78:60734) 2023-10-11T09:42:12.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.688+0000 7f1f69713700 1 -- 172.21.15.78:0/916961514 learned_addr learned my addr 172.21.15.78:0/916961514 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:12.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.688+0000 7f1f69f14700 1 --2- 172.21.15.78:0/916961514 >> v2:172.21.15.133:3300/0 conn(0x7f1f640cdbd0 0x7f1f6406cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:12.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.688+0000 7f1f69713700 1 -- 172.21.15.78:0/916961514 >> v2:172.21.15.133:3300/0 conn(0x7f1f640cdbd0 msgr2=0x7f1f6406cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:12.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.688+0000 7f1f69713700 1 --2- 172.21.15.78:0/916961514 >> v2:172.21.15.133:3300/0 conn(0x7f1f640cdbd0 0x7f1f6406cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:12.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.688+0000 7f1f69713700 1 -- 172.21.15.78:0/916961514 >> v1:172.21.15.78:6789/0 conn(0x7f1f640b0eb0 legacy=0x7f1f6406bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:12.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.688+0000 7f1f69713700 1 -- 172.21.15.78:0/916961514 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f640ce4e0 con 0x7f1f641655d0 2023-10-11T09:42:12.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.692+0000 7f1f69713700 1 --2- 172.21.15.78:0/916961514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f641655d0 0x7f1f640cdfa0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f1f64068380 tx=0x7f1f54009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=586c2627e6cae0dc server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:12.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.692+0000 7f1f68f12700 1 -- 172.21.15.78:0/916961514 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f54011830 con 0x7f1f641655d0 2023-10-11T09:42:12.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.692+0000 7f1f68f12700 1 -- 172.21.15.78:0/916961514 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f54011990 con 0x7f1f641655d0 2023-10-11T09:42:12.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.692+0000 7f1f68f12700 1 -- 172.21.15.78:0/916961514 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f54011d20 con 0x7f1f641655d0 2023-10-11T09:42:12.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.692+0000 7f1f6af16700 1 -- 172.21.15.78:0/916961514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f641655d0 msgr2=0x7f1f640cdfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:12.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.692+0000 7f1f6af16700 1 --2- 172.21.15.78:0/916961514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f641655d0 0x7f1f640cdfa0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f1f64068380 tx=0x7f1f54009bf0 comp rx=0 tx=0).stop 2023-10-11T09:42:12.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.692+0000 7f1f6af16700 1 -- 172.21.15.78:0/916961514 shutdown_connections 2023-10-11T09:42:12.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.692+0000 7f1f6af16700 1 --2- 172.21.15.78:0/916961514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f641655d0 0x7f1f640cdfa0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:12.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.692+0000 7f1f6af16700 1 --2- 172.21.15.78:0/916961514 >> v2:172.21.15.133:3300/0 conn(0x7f1f640cdbd0 0x7f1f6406cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:12.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.692+0000 7f1f6af16700 1 -- 172.21.15.78:0/916961514 >> 172.21.15.78:0/916961514 conn(0x7f1f6406acf0 msgr2=0x7f1f640b0710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:12.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.692+0000 7f1f6af16700 1 -- 172.21.15.78:0/916961514 shutdown_connections 2023-10-11T09:42:12.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.692+0000 7f1f6af16700 1 -- 172.21.15.78:0/916961514 wait complete. 2023-10-11T09:42:12.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.696+0000 7f1f6af16700 1 Processor -- start 2023-10-11T09:42:12.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.696+0000 7f1f6af16700 1 -- start start 2023-10-11T09:42:12.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.696+0000 7f1f6af16700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f640b0eb0 0x7f1f641600b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:12.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.696+0000 7f1f6af16700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f640cdbd0 0x7f1f640c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:12.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.696+0000 7f1f6af16700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1f641655d0 0x7f1f6415f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:12.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.696+0000 7f1f6af16700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f640c40e0 con 0x7f1f640b0eb0 2023-10-11T09:42:12.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.696+0000 7f1f6af16700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f640c2820 con 0x7f1f640cdbd0 2023-10-11T09:42:12.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.696+0000 7f1f6af16700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f640c2b00 con 0x7f1f641655d0 2023-10-11T09:42:12.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.696+0000 7f1f69f14700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f640b0eb0 0x7f1f641600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:12.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.696+0000 7f1f69f14700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f640b0eb0 0x7f1f641600b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60744/0 (socket says 172.21.15.78:60744) 2023-10-11T09:42:12.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.696+0000 7f1f69f14700 1 -- 172.21.15.78:0/2060724246 learned_addr learned my addr 172.21.15.78:0/2060724246 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:12.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.696+0000 7f1f6a715700 1 --2- 172.21.15.78:0/2060724246 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1f641655d0 0x7f1f6415f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:12.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.696+0000 7f1f69713700 1 --2- 172.21.15.78:0/2060724246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f640cdbd0 0x7f1f640c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:12.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.696+0000 7f1f69f14700 1 -- 172.21.15.78:0/2060724246 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1f641655d0 msgr2=0x7f1f6415f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:12.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.696+0000 7f1f69f14700 1 --2- 172.21.15.78:0/2060724246 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1f641655d0 0x7f1f6415f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:12.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.696+0000 7f1f69f14700 1 -- 172.21.15.78:0/2060724246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f640cdbd0 msgr2=0x7f1f640c3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:12.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.696+0000 7f1f69f14700 1 --2- 172.21.15.78:0/2060724246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f640cdbd0 0x7f1f640c3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:12.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.696+0000 7f1f69f14700 1 -- 172.21.15.78:0/2060724246 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f54009570 con 0x7f1f640b0eb0 2023-10-11T09:42:12.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.700+0000 7f1f69f14700 1 --2- 172.21.15.78:0/2060724246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f640b0eb0 0x7f1f641600b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1f6000e840 tx=0x7f1f6000eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:12.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.700+0000 7f1f5a7fc700 1 -- 172.21.15.78:0/2060724246 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f600095a0 con 0x7f1f640b0eb0 2023-10-11T09:42:12.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.700+0000 7f1f5a7fc700 1 -- 172.21.15.78:0/2060724246 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f60013680 con 0x7f1f640b0eb0 2023-10-11T09:42:12.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.700+0000 7f1f6af16700 1 -- 172.21.15.78:0/2060724246 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f640c2de0 con 0x7f1f640b0eb0 2023-10-11T09:42:12.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.700+0000 7f1f5a7fc700 1 -- 172.21.15.78:0/2060724246 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f600139f0 con 0x7f1f640b0eb0 2023-10-11T09:42:12.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.700+0000 7f1f6af16700 1 -- 172.21.15.78:0/2060724246 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f640c33a0 con 0x7f1f640b0eb0 2023-10-11T09:42:12.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.700+0000 7f1f6af16700 1 -- 172.21.15.78:0/2060724246 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f6406eca0 con 0x7f1f640b0eb0 2023-10-11T09:42:12.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.700+0000 7f1f5a7fc700 1 -- 172.21.15.78:0/2060724246 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f1f60013b50 con 0x7f1f640b0eb0 2023-10-11T09:42:12.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.704+0000 7f1f5a7fc700 1 --2- 172.21.15.78:0/2060724246 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1f48065ed0 0x7f1f48068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:12.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.704+0000 7f1f5a7fc700 1 -- 172.21.15.78:0/2060724246 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1f60083820 con 0x7f1f640b0eb0 2023-10-11T09:42:12.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.704+0000 7f1f69713700 1 --2- 172.21.15.78:0/2060724246 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1f48065ed0 0x7f1f48068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:12.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.704+0000 7f1f69713700 1 --2- 172.21.15.78:0/2060724246 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1f48065ed0 0x7f1f48068380 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1f641659e0 tx=0x7f1f54008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:12.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.712+0000 7f1f5a7fc700 1 -- 172.21.15.78:0/2060724246 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f1f60050dd0 con 0x7f1f640b0eb0 2023-10-11T09:42:12.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:12.888+0000 7f1f6af16700 1 -- 172.21.15.78:0/2060724246 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_extra", "pg_num_min": 64} v 0) v1 -- 0x7f1f641640f0 con 0x7f1f640b0eb0 2023-10-11T09:42:13.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.440+0000 7f1f5a7fc700 1 -- 172.21.15.78:0/2060724246 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_extra", "pg_num_min": 64}]=0 pool 'cephfs_data_extra' created v38) v1 ==== 142+0+0 (secure 0 0 0) 0x7f1f600543f0 con 0x7f1f640b0eb0 2023-10-11T09:42:13.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.444+0000 7f1f6af16700 1 -- 172.21.15.78:0/2060724246 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1f48065ed0 msgr2=0x7f1f48068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:13.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.444+0000 7f1f6af16700 1 --2- 172.21.15.78:0/2060724246 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1f48065ed0 0x7f1f48068380 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1f641659e0 tx=0x7f1f54008030 comp rx=0 tx=0).stop 2023-10-11T09:42:13.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.444+0000 7f1f6af16700 1 -- 172.21.15.78:0/2060724246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f640b0eb0 msgr2=0x7f1f641600b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:13.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.444+0000 7f1f6af16700 1 --2- 172.21.15.78:0/2060724246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f640b0eb0 0x7f1f641600b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1f6000e840 tx=0x7f1f6000eb50 comp rx=0 tx=0).stop 2023-10-11T09:42:13.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.444+0000 7f1f6af16700 1 -- 172.21.15.78:0/2060724246 shutdown_connections 2023-10-11T09:42:13.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.448+0000 7f1f6af16700 1 --2- 172.21.15.78:0/2060724246 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1f48065ed0 0x7f1f48068380 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:13.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.448+0000 7f1f6af16700 1 --2- 172.21.15.78:0/2060724246 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1f641655d0 0x7f1f6415f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:13.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.448+0000 7f1f6af16700 1 --2- 172.21.15.78:0/2060724246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f640cdbd0 0x7f1f640c3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:13.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.448+0000 7f1f6af16700 1 --2- 172.21.15.78:0/2060724246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f640b0eb0 0x7f1f641600b0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:13.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.448+0000 7f1f6af16700 1 -- 172.21.15.78:0/2060724246 >> 172.21.15.78:0/2060724246 conn(0x7f1f6406acf0 msgr2=0x7f1f6406fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:13.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.448+0000 7f1f6af16700 1 -- 172.21.15.78:0/2060724246 shutdown_connections 2023-10-11T09:42:13.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.448+0000 7f1f6af16700 1 -- 172.21.15.78:0/2060724246 wait complete. 2023-10-11T09:42:13.460 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data_extra' created 2023-10-11T09:42:13.479 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_extra 2023-10-11T09:42:13.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.648+0000 7efd0aeae700 1 Processor -- start 2023-10-11T09:42:13.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.648+0000 7efd0aeae700 1 -- start start 2023-10-11T09:42:13.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.652+0000 7efd0aeae700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd040b1030 0x7efd040cb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:13.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.652+0000 7efd0aeae700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efd0406d080 0x7efd0406ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:13.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.652+0000 7efd0aeae700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd04165570 con 0x7efd04160cb0 2023-10-11T09:42:13.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.652+0000 7efd0aeae700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd04161080 con 0x7efd0406d080 2023-10-11T09:42:13.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.652+0000 7efd0aeae700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd04164750 con 0x7efd040b1030 2023-10-11T09:42:13.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.652+0000 7efd08c4a700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7efd04160cb0 0x7efd040b0920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:51852/0 (socket says 172.21.15.78:51852) 2023-10-11T09:42:13.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.652+0000 7efd08c4a700 1 -- 172.21.15.78:0/1660866755 learned_addr learned my addr 172.21.15.78:0/1660866755 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:42:13.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.652+0000 7efd03fff700 1 --2- 172.21.15.78:0/1660866755 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd040b1030 0x7efd040cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:13.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.652+0000 7efd0944b700 1 --2- 172.21.15.78:0/1660866755 >> v2:172.21.15.133:3300/0 conn(0x7efd0406d080 0x7efd0406ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:13.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.652+0000 7efd037fe700 1 -- 172.21.15.78:0/1660866755 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2049964531 0 0) 0x7efd04165570 con 0x7efd04160cb0 2023-10-11T09:42:13.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.652+0000 7efd037fe700 1 -- 172.21.15.78:0/1660866755 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcf40035d0 con 0x7efd04160cb0 2023-10-11T09:42:13.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.652+0000 7efd03fff700 1 -- 172.21.15.78:0/1660866755 >> v2:172.21.15.133:3300/0 conn(0x7efd0406d080 msgr2=0x7efd0406ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:13.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.652+0000 7efd03fff700 1 --2- 172.21.15.78:0/1660866755 >> v2:172.21.15.133:3300/0 conn(0x7efd0406d080 0x7efd0406ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:13.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.652+0000 7efd03fff700 1 -- 172.21.15.78:0/1660866755 >> v1:172.21.15.78:6789/0 conn(0x7efd04160cb0 legacy=0x7efd040b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:13.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.652+0000 7efd037fe700 1 -- 172.21.15.78:0/1660866755 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 568633395 0 0) 0x7efcf40035d0 con 0x7efd04160cb0 2023-10-11T09:42:13.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.652+0000 7efd03fff700 1 -- 172.21.15.78:0/1660866755 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd04164960 con 0x7efd040b1030 2023-10-11T09:42:13.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.652+0000 7efd03fff700 1 --2- 172.21.15.78:0/1660866755 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd040b1030 0x7efd040cb100 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efcf0009850 tx=0x7efcf0009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8843df001ef886d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:13.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.656+0000 7efd037fe700 1 -- 172.21.15.78:0/1660866755 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf8001670 con 0x7efd040b1030 2023-10-11T09:42:13.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.656+0000 7efd037fe700 1 -- 172.21.15.78:0/1660866755 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcf0004030 con 0x7efd040b1030 2023-10-11T09:42:13.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.656+0000 7efd037fe700 1 -- 172.21.15.78:0/1660866755 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf00043e0 con 0x7efd040b1030 2023-10-11T09:42:13.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.656+0000 7efd0aeae700 1 -- 172.21.15.78:0/1660866755 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd040b1030 msgr2=0x7efd040cb100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:13.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.656+0000 7efd0aeae700 1 --2- 172.21.15.78:0/1660866755 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd040b1030 0x7efd040cb100 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efcf0009850 tx=0x7efcf0009b60 comp rx=0 tx=0).stop 2023-10-11T09:42:13.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.656+0000 7efd0aeae700 1 -- 172.21.15.78:0/1660866755 shutdown_connections 2023-10-11T09:42:13.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.656+0000 7efd0aeae700 1 --2- 172.21.15.78:0/1660866755 >> v2:172.21.15.133:3300/0 conn(0x7efd0406d080 0x7efd0406ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:13.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.656+0000 7efd0aeae700 1 --2- 172.21.15.78:0/1660866755 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd040b1030 0x7efd040cb100 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:13.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.656+0000 7efd0aeae700 1 -- 172.21.15.78:0/1660866755 >> 172.21.15.78:0/1660866755 conn(0x7efd040cecc0 msgr2=0x7efd040cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:13.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.656+0000 7efd0aeae700 1 -- 172.21.15.78:0/1660866755 shutdown_connections 2023-10-11T09:42:13.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.656+0000 7efd0aeae700 1 -- 172.21.15.78:0/1660866755 wait complete. 2023-10-11T09:42:13.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.656+0000 7efd0aeae700 1 Processor -- start 2023-10-11T09:42:13.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.660+0000 7efd0aeae700 1 -- start start 2023-10-11T09:42:13.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.660+0000 7efd0aeae700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd0406d080 0x7efd04147dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:13.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.660+0000 7efd0aeae700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd040b1030 0x7efd04158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:13.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.660+0000 7efd0aeae700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd04160cb0 0x7efd040b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:13.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.660+0000 7efd0aeae700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd04159d40 con 0x7efd04160cb0 2023-10-11T09:42:13.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.660+0000 7efd0aeae700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd04159130 con 0x7efd040b1030 2023-10-11T09:42:13.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.660+0000 7efd0aeae700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd04159410 con 0x7efd0406d080 2023-10-11T09:42:13.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.660+0000 7efd0944b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd04160cb0 0x7efd040b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:13.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.660+0000 7efd0944b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd04160cb0 0x7efd040b41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60756/0 (socket says 172.21.15.78:60756) 2023-10-11T09:42:13.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.660+0000 7efd0944b700 1 -- 172.21.15.78:0/1528847573 learned_addr learned my addr 172.21.15.78:0/1528847573 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:13.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.660+0000 7efd08c4a700 1 --2- 172.21.15.78:0/1528847573 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd0406d080 0x7efd04147dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:13.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.660+0000 7efd03fff700 1 --2- 172.21.15.78:0/1528847573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd040b1030 0x7efd04158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:13.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.660+0000 7efd0944b700 1 -- 172.21.15.78:0/1528847573 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd0406d080 msgr2=0x7efd04147dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:13.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.660+0000 7efd0944b700 1 --2- 172.21.15.78:0/1528847573 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd0406d080 0x7efd04147dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:13.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.660+0000 7efd0944b700 1 -- 172.21.15.78:0/1528847573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd040b1030 msgr2=0x7efd04158730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:13.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.660+0000 7efd0944b700 1 --2- 172.21.15.78:0/1528847573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd040b1030 0x7efd04158730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:13.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.660+0000 7efd0944b700 1 -- 172.21.15.78:0/1528847573 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf00094b0 con 0x7efd04160cb0 2023-10-11T09:42:13.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.660+0000 7efd08c4a700 1 --2- 172.21.15.78:0/1528847573 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd0406d080 0x7efd04147dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:42:13.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.660+0000 7efd03fff700 1 --2- 172.21.15.78:0/1528847573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd040b1030 0x7efd04158730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:42:13.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.660+0000 7efd0944b700 1 --2- 172.21.15.78:0/1528847573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd04160cb0 0x7efd040b41a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7efcfc00efc0 tx=0x7efcfc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:13.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.660+0000 7efd017fa700 1 -- 172.21.15.78:0/1528847573 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcfc009ba0 con 0x7efd04160cb0 2023-10-11T09:42:13.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.660+0000 7efd017fa700 1 -- 172.21.15.78:0/1528847573 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcfc00f040 con 0x7efd04160cb0 2023-10-11T09:42:13.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.660+0000 7efd017fa700 1 -- 172.21.15.78:0/1528847573 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcfc014480 con 0x7efd04160cb0 2023-10-11T09:42:13.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.660+0000 7efd0aeae700 1 -- 172.21.15.78:0/1528847573 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd041596f0 con 0x7efd04160cb0 2023-10-11T09:42:13.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.660+0000 7efd0aeae700 1 -- 172.21.15.78:0/1528847573 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd04148310 con 0x7efd04160cb0 2023-10-11T09:42:13.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.664+0000 7efd0aeae700 1 -- 172.21.15.78:0/1528847573 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd04148500 con 0x7efd04160cb0 2023-10-11T09:42:13.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.664+0000 7efd017fa700 1 -- 172.21.15.78:0/1528847573 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7efcfc0145e0 con 0x7efd04160cb0 2023-10-11T09:42:13.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.668+0000 7efd017fa700 1 --2- 172.21.15.78:0/1528847573 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efce8065ed0 0x7efce8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:13.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.668+0000 7efd017fa700 1 -- 172.21.15.78:0/1528847573 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6434+0+0 (secure 0 0 0) 0x7efcfc0841f0 con 0x7efd04160cb0 2023-10-11T09:42:13.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.668+0000 7efd08c4a700 1 --2- 172.21.15.78:0/1528847573 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efce8065ed0 0x7efce8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:13.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.668+0000 7efd08c4a700 1 --2- 172.21.15.78:0/1528847573 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efce8065ed0 0x7efce8068380 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efcf8007fd0 tx=0x7efcf8007b20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:13.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.672+0000 7efd017fa700 1 -- 172.21.15.78:0/1528847573 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7efcfc0516c0 con 0x7efd04160cb0 2023-10-11T09:42:13.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.888+0000 7efd0aeae700 1 -- 172.21.15.78:0/1528847573 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_extra"} v 0) v1 -- 0x7efd04159e80 con 0x7efd04160cb0 2023-10-11T09:42:13.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.900+0000 7efd017fa700 1 -- 172.21.15.78:0/1528847573 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_extra"}]=0 added data pool 7 to fsmap v17) v1 ==== 140+0+0 (secure 0 0 0) 0x7efd04159e80 con 0x7efd04160cb0 2023-10-11T09:42:13.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.908+0000 7efd0aeae700 1 -- 172.21.15.78:0/1528847573 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efce8065ed0 msgr2=0x7efce8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:13.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.908+0000 7efd0aeae700 1 --2- 172.21.15.78:0/1528847573 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efce8065ed0 0x7efce8068380 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efcf8007fd0 tx=0x7efcf8007b20 comp rx=0 tx=0).stop 2023-10-11T09:42:13.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.908+0000 7efd0aeae700 1 -- 172.21.15.78:0/1528847573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd04160cb0 msgr2=0x7efd040b41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:13.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.908+0000 7efd0aeae700 1 --2- 172.21.15.78:0/1528847573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd04160cb0 0x7efd040b41a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7efcfc00efc0 tx=0x7efcfc00c3a0 comp rx=0 tx=0).stop 2023-10-11T09:42:13.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.908+0000 7efd0aeae700 1 -- 172.21.15.78:0/1528847573 shutdown_connections 2023-10-11T09:42:13.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.908+0000 7efd0aeae700 1 --2- 172.21.15.78:0/1528847573 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efce8065ed0 0x7efce8068380 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:13.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.908+0000 7efd0aeae700 1 --2- 172.21.15.78:0/1528847573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd04160cb0 0x7efd040b41a0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:13.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.908+0000 7efd0aeae700 1 --2- 172.21.15.78:0/1528847573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd040b1030 0x7efd04158730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:13.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.908+0000 7efd0aeae700 1 --2- 172.21.15.78:0/1528847573 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd0406d080 0x7efd04147dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:13.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.908+0000 7efd0aeae700 1 -- 172.21.15.78:0/1528847573 >> 172.21.15.78:0/1528847573 conn(0x7efd040cecc0 msgr2=0x7efd040cd910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:13.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.912+0000 7efd0aeae700 1 -- 172.21.15.78:0/1528847573 shutdown_connections 2023-10-11T09:42:13.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:13.912+0000 7efd0aeae700 1 -- 172.21.15.78:0/1528847573 wait complete. 2023-10-11T09:42:13.924 INFO:teuthology.orchestra.run.smithi078.stderr:added data pool 7 to fsmap 2023-10-11T09:42:13.941 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:42:14.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.096+0000 7fd85149b700 1 Processor -- start 2023-10-11T09:42:14.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.096+0000 7fd85149b700 1 -- start start 2023-10-11T09:42:14.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.100+0000 7fd85149b700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd84c154e00 0x7fd84c15b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:14.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.100+0000 7fd85149b700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd84c071040 0x7fd84c0cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:14.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.100+0000 7fd85149b700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd84c0720f0 con 0x7fd84c0cba00 2023-10-11T09:42:14.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.100+0000 7fd85149b700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd84c15bb90 con 0x7fd84c154e00 2023-10-11T09:42:14.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.100+0000 7fd85149b700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd84c0714e0 con 0x7fd84c071040 2023-10-11T09:42:14.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.100+0000 7fd84b7fe700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fd84c0cba00 0x7fd84c15e1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:51858/0 (socket says 172.21.15.78:51858) 2023-10-11T09:42:14.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.100+0000 7fd84b7fe700 1 -- 172.21.15.78:0/1406281585 learned_addr learned my addr 172.21.15.78:0/1406281585 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:42:14.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.100+0000 7fd84affd700 1 --2- 172.21.15.78:0/1406281585 >> v2:172.21.15.133:3300/0 conn(0x7fd84c154e00 0x7fd84c15b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:14.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.100+0000 7fd84a7fc700 1 --2- 172.21.15.78:0/1406281585 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd84c071040 0x7fd84c0cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:14.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.100+0000 7fd84affd700 1 -- 172.21.15.78:0/1406281585 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd84c071040 msgr2=0x7fd84c0cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:14.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.100+0000 7fd849ffb700 1 -- 172.21.15.78:0/1406281585 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 960483043 0 0) 0x7fd84c0720f0 con 0x7fd84c0cba00 2023-10-11T09:42:14.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.100+0000 7fd84affd700 1 --2- 172.21.15.78:0/1406281585 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd84c071040 0x7fd84c0cb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:14.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.100+0000 7fd84affd700 1 -- 172.21.15.78:0/1406281585 >> v1:172.21.15.78:6789/0 conn(0x7fd84c0cba00 legacy=0x7fd84c15e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:14.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.100+0000 7fd84affd700 1 -- 172.21.15.78:0/1406281585 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd84c0717c0 con 0x7fd84c154e00 2023-10-11T09:42:14.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.100+0000 7fd84affd700 1 --2- 172.21.15.78:0/1406281585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd84c154e00 0x7fd84c15b650 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd83c009d90 tx=0x7fd83c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7eeb01c9f1daacd2 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:14.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.104+0000 7fd849ffb700 1 -- 172.21.15.78:0/1406281585 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c00e070 con 0x7fd84c154e00 2023-10-11T09:42:14.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.104+0000 7fd849ffb700 1 -- 172.21.15.78:0/1406281585 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd83c00bb20 con 0x7fd84c154e00 2023-10-11T09:42:14.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.104+0000 7fd849ffb700 1 -- 172.21.15.78:0/1406281585 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c012430 con 0x7fd84c154e00 2023-10-11T09:42:14.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.104+0000 7fd85149b700 1 -- 172.21.15.78:0/1406281585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd84c154e00 msgr2=0x7fd84c15b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:14.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.104+0000 7fd85149b700 1 --2- 172.21.15.78:0/1406281585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd84c154e00 0x7fd84c15b650 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd83c009d90 tx=0x7fd83c00b330 comp rx=0 tx=0).stop 2023-10-11T09:42:14.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.104+0000 7fd85149b700 1 -- 172.21.15.78:0/1406281585 shutdown_connections 2023-10-11T09:42:14.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.104+0000 7fd85149b700 1 --2- 172.21.15.78:0/1406281585 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd84c071040 0x7fd84c0cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:14.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.104+0000 7fd85149b700 1 --2- 172.21.15.78:0/1406281585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd84c154e00 0x7fd84c15b650 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:14.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.104+0000 7fd85149b700 1 -- 172.21.15.78:0/1406281585 >> 172.21.15.78:0/1406281585 conn(0x7fd84c06aca0 msgr2=0x7fd84c0b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:14.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.104+0000 7fd85149b700 1 -- 172.21.15.78:0/1406281585 shutdown_connections 2023-10-11T09:42:14.114 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.104+0000 7fd85149b700 1 -- 172.21.15.78:0/1406281585 wait complete. 2023-10-11T09:42:14.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.108+0000 7fd85149b700 1 Processor -- start 2023-10-11T09:42:14.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.108+0000 7fd85149b700 1 -- start start 2023-10-11T09:42:14.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.108+0000 7fd85149b700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd84c071040 0x7fd84c1f5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:14.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.108+0000 7fd85149b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd84c0cba00 0x7fd84c0cc870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:14.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.108+0000 7fd85149b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd84c154e00 0x7fd84c14f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:14.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.108+0000 7fd85149b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84c1f1e50 con 0x7fd84c0cba00 2023-10-11T09:42:14.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.108+0000 7fd85149b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84c1f6360 con 0x7fd84c154e00 2023-10-11T09:42:14.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.108+0000 7fd85149b700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd84c0b0420 con 0x7fd84c071040 2023-10-11T09:42:14.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.108+0000 7fd84a7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd84c0cba00 0x7fd84c0cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:14.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.108+0000 7fd84a7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd84c0cba00 0x7fd84c0cc870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60770/0 (socket says 172.21.15.78:60770) 2023-10-11T09:42:14.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.108+0000 7fd84a7fc700 1 -- 172.21.15.78:0/3478185558 learned_addr learned my addr 172.21.15.78:0/3478185558 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:14.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.108+0000 7fd84affd700 1 --2- 172.21.15.78:0/3478185558 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd84c071040 0x7fd84c1f5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:14.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.108+0000 7fd84b7fe700 1 --2- 172.21.15.78:0/3478185558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd84c154e00 0x7fd84c14f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:14.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.108+0000 7fd84a7fc700 1 -- 172.21.15.78:0/3478185558 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd84c071040 msgr2=0x7fd84c1f5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:14.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.108+0000 7fd84a7fc700 1 --2- 172.21.15.78:0/3478185558 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd84c071040 0x7fd84c1f5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:14.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.108+0000 7fd84a7fc700 1 -- 172.21.15.78:0/3478185558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd84c154e00 msgr2=0x7fd84c14f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:14.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.108+0000 7fd84a7fc700 1 --2- 172.21.15.78:0/3478185558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd84c154e00 0x7fd84c14f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:14.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.108+0000 7fd84a7fc700 1 -- 172.21.15.78:0/3478185558 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd83c009a70 con 0x7fd84c0cba00 2023-10-11T09:42:14.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.108+0000 7fd84affd700 1 --2- 172.21.15.78:0/3478185558 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd84c071040 0x7fd84c1f5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:42:14.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.108+0000 7fd84a7fc700 1 --2- 172.21.15.78:0/3478185558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd84c0cba00 0x7fd84c0cc870 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd84000e9a0 tx=0x7fd84000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:14.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.108+0000 7fd8337fe700 1 -- 172.21.15.78:0/3478185558 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd840009720 con 0x7fd84c0cba00 2023-10-11T09:42:14.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.108+0000 7fd8337fe700 1 -- 172.21.15.78:0/3478185558 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd84000c850 con 0x7fd84c0cba00 2023-10-11T09:42:14.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.108+0000 7fd85149b700 1 -- 172.21.15.78:0/3478185558 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd84c0b0700 con 0x7fd84c0cba00 2023-10-11T09:42:14.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.108+0000 7fd8337fe700 1 -- 172.21.15.78:0/3478185558 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84000cc00 con 0x7fd84c0cba00 2023-10-11T09:42:14.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.108+0000 7fd85149b700 1 -- 172.21.15.78:0/3478185558 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd84c0b0cc0 con 0x7fd84c0cba00 2023-10-11T09:42:14.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.112+0000 7fd85149b700 1 -- 172.21.15.78:0/3478185558 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd84c071a80 con 0x7fd84c0cba00 2023-10-11T09:42:14.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.112+0000 7fd8337fe700 1 -- 172.21.15.78:0/3478185558 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fd840013470 con 0x7fd84c0cba00 2023-10-11T09:42:14.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.116+0000 7fd8337fe700 1 --2- 172.21.15.78:0/3478185558 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd838065ed0 0x7fd838068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:14.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.116+0000 7fd8337fe700 1 -- 172.21.15.78:0/3478185558 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6466+0+0 (secure 0 0 0) 0x7fd840083b30 con 0x7fd84c0cba00 2023-10-11T09:42:14.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.116+0000 7fd84affd700 1 --2- 172.21.15.78:0/3478185558 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd838065ed0 0x7fd838068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:14.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.116+0000 7fd84affd700 1 --2- 172.21.15.78:0/3478185558 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd838065ed0 0x7fd838068380 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd83c014040 tx=0x7fd83c011000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:14.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.124+0000 7fd8337fe700 1 -- 172.21.15.78:0/3478185558 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fd84c071a80 con 0x7fd84c0cba00 2023-10-11T09:42:14.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.332+0000 7fd85149b700 1 -- 172.21.15.78:0/3478185558 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd84c071a80 con 0x7fd84c0cba00 2023-10-11T09:42:14.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.332+0000 7fd8337fe700 1 -- 172.21.15.78:0/3478185558 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4650 (secure 0 0 0) 0x7fd84c071a80 con 0x7fd84c0cba00 2023-10-11T09:42:14.339 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:14.339 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5308,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/655413753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":655413753},{"type":"v1","addr":"172.21.15.133:6835","nonce":655413753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5352,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2966622437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2966622437},{"type":"v1","addr":"172.21.15.78:6835","nonce":2966622437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5355,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/700246445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":700246445},{"type":"v1","addr":"172.21.15.78:6837","nonce":700246445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:41:59.225318+0000","modified":"2023-10-11T09:42:13.894931+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5364},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5364":{"gid":5364,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.133:6833/1862542010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1862542010},{"type":"v1","addr":"172.21.15.133:6833","nonce":1862542010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-11T09:42:14.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.336+0000 7fd85149b700 1 -- 172.21.15.78:0/3478185558 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd838065ed0 msgr2=0x7fd838068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:14.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.336+0000 7fd85149b700 1 --2- 172.21.15.78:0/3478185558 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd838065ed0 0x7fd838068380 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd83c014040 tx=0x7fd83c011000 comp rx=0 tx=0).stop 2023-10-11T09:42:14.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.336+0000 7fd85149b700 1 -- 172.21.15.78:0/3478185558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd84c0cba00 msgr2=0x7fd84c0cc870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:14.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.336+0000 7fd85149b700 1 --2- 172.21.15.78:0/3478185558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd84c0cba00 0x7fd84c0cc870 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd84000e9a0 tx=0x7fd84000ed60 comp rx=0 tx=0).stop 2023-10-11T09:42:14.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.336+0000 7fd85149b700 1 -- 172.21.15.78:0/3478185558 shutdown_connections 2023-10-11T09:42:14.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.336+0000 7fd85149b700 1 --2- 172.21.15.78:0/3478185558 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd838065ed0 0x7fd838068380 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:14.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.336+0000 7fd85149b700 1 --2- 172.21.15.78:0/3478185558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd84c154e00 0x7fd84c14f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:14.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.336+0000 7fd85149b700 1 --2- 172.21.15.78:0/3478185558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd84c0cba00 0x7fd84c0cc870 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:14.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.336+0000 7fd85149b700 1 --2- 172.21.15.78:0/3478185558 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd84c071040 0x7fd84c1f5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:14.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.336+0000 7fd85149b700 1 -- 172.21.15.78:0/3478185558 >> 172.21.15.78:0/3478185558 conn(0x7fd84c06aca0 msgr2=0x7fd84c15c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:14.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.336+0000 7fd85149b700 1 -- 172.21.15.78:0/3478185558 shutdown_connections 2023-10-11T09:42:14.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.336+0000 7fd85149b700 1 -- 172.21.15.78:0/3478185558 wait complete. 2023-10-11T09:42:14.344 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 17 2023-10-11T09:42:14.365 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:42:14.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.528+0000 7fb63f6d8700 1 Processor -- start 2023-10-11T09:42:14.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.528+0000 7fb63f6d8700 1 -- start start 2023-10-11T09:42:14.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.528+0000 7fb63f6d8700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb638160d40 0x7fb6380cd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:14.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.528+0000 7fb63f6d8700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb638071040 0x7fb6380cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:14.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.528+0000 7fb63f6d8700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb63806ffa0 con 0x7fb6380cd7c0 2023-10-11T09:42:14.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.528+0000 7fb63f6d8700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb638070c20 con 0x7fb638071040 2023-10-11T09:42:14.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.528+0000 7fb63f6d8700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb638161110 con 0x7fb638160d40 2023-10-11T09:42:14.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.528+0000 7fb63dc75700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fb6380cd7c0 0x7fb6381647f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:51872/0 (socket says 172.21.15.78:51872) 2023-10-11T09:42:14.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.528+0000 7fb63dc75700 1 -- 172.21.15.78:0/1706240401 learned_addr learned my addr 172.21.15.78:0/1706240401 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:42:14.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.532+0000 7fb63d474700 1 --2- 172.21.15.78:0/1706240401 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb638160d40 0x7fb6380cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:14.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.532+0000 7fb63cc73700 1 --2- 172.21.15.78:0/1706240401 >> v2:172.21.15.133:3300/0 conn(0x7fb638071040 0x7fb6380cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:14.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.532+0000 7fb62ffff700 1 -- 172.21.15.78:0/1706240401 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1842679534 0 0) 0x7fb63806ffa0 con 0x7fb6380cd7c0 2023-10-11T09:42:14.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.532+0000 7fb62ffff700 1 -- 172.21.15.78:0/1706240401 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6240035d0 con 0x7fb6380cd7c0 2023-10-11T09:42:14.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.532+0000 7fb63d474700 1 -- 172.21.15.78:0/1706240401 >> v2:172.21.15.133:3300/0 conn(0x7fb638071040 msgr2=0x7fb6380cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:14.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.532+0000 7fb63d474700 1 --2- 172.21.15.78:0/1706240401 >> v2:172.21.15.133:3300/0 conn(0x7fb638071040 0x7fb6380cddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:14.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.532+0000 7fb63d474700 1 -- 172.21.15.78:0/1706240401 >> v1:172.21.15.78:6789/0 conn(0x7fb6380cd7c0 legacy=0x7fb6381647f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:14.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.532+0000 7fb63d474700 1 -- 172.21.15.78:0/1706240401 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb638070280 con 0x7fb638160d40 2023-10-11T09:42:14.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.532+0000 7fb63d474700 1 --2- 172.21.15.78:0/1706240401 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb638160d40 0x7fb6380cd280 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb634009850 tx=0x7fb634009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c48bc53c56578e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:14.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.532+0000 7fb62ffff700 1 -- 172.21.15.78:0/1706240401 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb634011830 con 0x7fb638160d40 2023-10-11T09:42:14.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.532+0000 7fb62ffff700 1 -- 172.21.15.78:0/1706240401 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb634011990 con 0x7fb638160d40 2023-10-11T09:42:14.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.532+0000 7fb62ffff700 1 -- 172.21.15.78:0/1706240401 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb634011d40 con 0x7fb638160d40 2023-10-11T09:42:14.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.532+0000 7fb63f6d8700 1 -- 172.21.15.78:0/1706240401 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb638160d40 msgr2=0x7fb6380cd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:14.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.532+0000 7fb63f6d8700 1 --2- 172.21.15.78:0/1706240401 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb638160d40 0x7fb6380cd280 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb634009850 tx=0x7fb634009b60 comp rx=0 tx=0).stop 2023-10-11T09:42:14.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.532+0000 7fb63f6d8700 1 -- 172.21.15.78:0/1706240401 shutdown_connections 2023-10-11T09:42:14.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.532+0000 7fb63f6d8700 1 --2- 172.21.15.78:0/1706240401 >> v2:172.21.15.133:3300/0 conn(0x7fb638071040 0x7fb6380cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:14.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.532+0000 7fb63f6d8700 1 --2- 172.21.15.78:0/1706240401 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb638160d40 0x7fb6380cd280 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:14.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.532+0000 7fb63f6d8700 1 -- 172.21.15.78:0/1706240401 >> 172.21.15.78:0/1706240401 conn(0x7fb63806aca0 msgr2=0x7fb6380b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:14.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.532+0000 7fb63f6d8700 1 -- 172.21.15.78:0/1706240401 shutdown_connections 2023-10-11T09:42:14.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.532+0000 7fb63f6d8700 1 -- 172.21.15.78:0/1706240401 wait complete. 2023-10-11T09:42:14.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.536+0000 7fb63f6d8700 1 Processor -- start 2023-10-11T09:42:14.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.536+0000 7fb63f6d8700 1 -- start start 2023-10-11T09:42:14.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.536+0000 7fb63f6d8700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb638071040 0x7fb63814d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:14.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.536+0000 7fb63f6d8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6380cd7c0 0x7fb6380cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:14.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.536+0000 7fb63f6d8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb638160d40 0x7fb6380ce170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:14.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.536+0000 7fb63f6d8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63814ff30 con 0x7fb6380cd7c0 2023-10-11T09:42:14.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.536+0000 7fb63f6d8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb638151830 con 0x7fb638160d40 2023-10-11T09:42:14.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.536+0000 7fb63f6d8700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb63815e4a0 con 0x7fb638071040 2023-10-11T09:42:14.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.536+0000 7fb63cc73700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6380cd7c0 0x7fb6380cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:14.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.536+0000 7fb63cc73700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6380cd7c0 0x7fb6380cdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60780/0 (socket says 172.21.15.78:60780) 2023-10-11T09:42:14.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.536+0000 7fb63cc73700 1 -- 172.21.15.78:0/1300747741 learned_addr learned my addr 172.21.15.78:0/1300747741 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:14.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.536+0000 7fb63dc75700 1 --2- 172.21.15.78:0/1300747741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb638160d40 0x7fb6380ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:14.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.536+0000 7fb63d474700 1 --2- 172.21.15.78:0/1300747741 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb638071040 0x7fb63814d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:14.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.540+0000 7fb63cc73700 1 -- 172.21.15.78:0/1300747741 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb638071040 msgr2=0x7fb63814d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:14.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.540+0000 7fb63cc73700 1 --2- 172.21.15.78:0/1300747741 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb638071040 0x7fb63814d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:14.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.540+0000 7fb63cc73700 1 -- 172.21.15.78:0/1300747741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb638160d40 msgr2=0x7fb6380ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:14.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.540+0000 7fb63cc73700 1 --2- 172.21.15.78:0/1300747741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb638160d40 0x7fb6380ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:14.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.540+0000 7fb63cc73700 1 -- 172.21.15.78:0/1300747741 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6340094b0 con 0x7fb6380cd7c0 2023-10-11T09:42:14.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.540+0000 7fb63cc73700 1 --2- 172.21.15.78:0/1300747741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6380cd7c0 0x7fb6380cdc30 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb628009fd0 tx=0x7fb62800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:14.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.540+0000 7fb62dffb700 1 -- 172.21.15.78:0/1300747741 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6280095a0 con 0x7fb6380cd7c0 2023-10-11T09:42:14.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.540+0000 7fb62dffb700 1 -- 172.21.15.78:0/1300747741 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb628013680 con 0x7fb6380cd7c0 2023-10-11T09:42:14.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.540+0000 7fb62dffb700 1 -- 172.21.15.78:0/1300747741 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb628013a30 con 0x7fb6380cd7c0 2023-10-11T09:42:14.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.540+0000 7fb63f6d8700 1 -- 172.21.15.78:0/1300747741 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb63814de00 con 0x7fb6380cd7c0 2023-10-11T09:42:14.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.540+0000 7fb63f6d8700 1 -- 172.21.15.78:0/1300747741 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb63814f970 con 0x7fb6380cd7c0 2023-10-11T09:42:14.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.540+0000 7fb63f6d8700 1 -- 172.21.15.78:0/1300747741 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb638072250 con 0x7fb6380cd7c0 2023-10-11T09:42:14.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.544+0000 7fb62dffb700 1 -- 172.21.15.78:0/1300747741 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fb628013b90 con 0x7fb6380cd7c0 2023-10-11T09:42:14.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.544+0000 7fb62dffb700 1 --2- 172.21.15.78:0/1300747741 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb61c065f00 0x7fb61c0683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:14.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.544+0000 7fb62dffb700 1 -- 172.21.15.78:0/1300747741 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6466+0+0 (secure 0 0 0) 0x7fb628083600 con 0x7fb6380cd7c0 2023-10-11T09:42:14.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.544+0000 7fb63d474700 1 --2- 172.21.15.78:0/1300747741 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb61c065f00 0x7fb61c0683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:14.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.544+0000 7fb63d474700 1 --2- 172.21.15.78:0/1300747741 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb61c065f00 0x7fb61c0683b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb6340090b0 tx=0x7fb634008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:14.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.552+0000 7fb62dffb700 1 -- 172.21.15.78:0/1300747741 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fb628050ab0 con 0x7fb6380cd7c0 2023-10-11T09:42:14.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.728+0000 7fb63f6d8700 1 -- 172.21.15.78:0/1300747741 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb638151590 con 0x7fb6380cd7c0 2023-10-11T09:42:14.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.732+0000 7fb62dffb700 1 -- 172.21.15.78:0/1300747741 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+19684 (secure 0 0 0) 0x7fb638151590 con 0x7fb6380cd7c0 2023-10-11T09:42:14.739 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:14.739 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":39,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:42:13.894328+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:41:57.280079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-11T09:41:58.218860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data_extra","create_time":"2023-10-11T09:42:12.954795+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:42:14.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.736+0000 7fb63f6d8700 1 -- 172.21.15.78:0/1300747741 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb61c065f00 msgr2=0x7fb61c0683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:14.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.736+0000 7fb63f6d8700 1 --2- 172.21.15.78:0/1300747741 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb61c065f00 0x7fb61c0683b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb6340090b0 tx=0x7fb634008030 comp rx=0 tx=0).stop 2023-10-11T09:42:14.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.736+0000 7fb63f6d8700 1 -- 172.21.15.78:0/1300747741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6380cd7c0 msgr2=0x7fb6380cdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:14.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.736+0000 7fb63f6d8700 1 --2- 172.21.15.78:0/1300747741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6380cd7c0 0x7fb6380cdc30 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb628009fd0 tx=0x7fb62800eb20 comp rx=0 tx=0).stop 2023-10-11T09:42:14.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.736+0000 7fb63f6d8700 1 -- 172.21.15.78:0/1300747741 shutdown_connections 2023-10-11T09:42:14.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.736+0000 7fb63f6d8700 1 --2- 172.21.15.78:0/1300747741 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb61c065f00 0x7fb61c0683b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:14.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.736+0000 7fb63f6d8700 1 --2- 172.21.15.78:0/1300747741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb638160d40 0x7fb6380ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:14.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.736+0000 7fb63f6d8700 1 --2- 172.21.15.78:0/1300747741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6380cd7c0 0x7fb6380cdc30 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:14.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.736+0000 7fb63f6d8700 1 --2- 172.21.15.78:0/1300747741 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb638071040 0x7fb63814d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:14.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.736+0000 7fb63f6d8700 1 -- 172.21.15.78:0/1300747741 >> 172.21.15.78:0/1300747741 conn(0x7fb63806aca0 msgr2=0x7fb6380cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:14.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.736+0000 7fb63f6d8700 1 -- 172.21.15.78:0/1300747741 shutdown_connections 2023-10-11T09:42:14.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:14.736+0000 7fb63f6d8700 1 -- 172.21.15.78:0/1300747741 wait complete. 2023-10-11T09:42:14.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:42:14.763 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v cephfs_data_extra .) 2023-10-11T09:42:14.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:42:14.802 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2023-10-11T09:42:14.877 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/0 2023-10-11T09:42:14.877 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:42:14.878 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:42:14.878 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/0" 2023-10-11T09:42:14.879 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'w'"'"') as f: 2023-10-11T09:42:14.879 DEBUG:teuthology.orchestra.run.smithi078:> for i in range(0, 6291456): 2023-10-11T09:42:14.879 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:42:14.880 DEBUG:teuthology.orchestra.run.smithi078:> f.write(chr(val)) 2023-10-11T09:42:14.880 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:42:18.209 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/1 2023-10-11T09:42:18.209 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:42:18.210 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:42:18.210 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/1" 2023-10-11T09:42:18.211 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'w'"'"') as f: 2023-10-11T09:42:18.211 DEBUG:teuthology.orchestra.run.smithi078:> for i in range(0, 6291456): 2023-10-11T09:42:18.211 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:42:18.212 DEBUG:teuthology.orchestra.run.smithi078:> f.write(chr(val)) 2023-10-11T09:42:18.212 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:42:21.536 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/2 2023-10-11T09:42:21.536 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:42:21.537 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:42:21.537 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/2" 2023-10-11T09:42:21.537 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'w'"'"') as f: 2023-10-11T09:42:21.538 DEBUG:teuthology.orchestra.run.smithi078:> for i in range(0, 6291456): 2023-10-11T09:42:21.538 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:42:21.538 DEBUG:teuthology.orchestra.run.smithi078:> f.write(chr(val)) 2023-10-11T09:42:21.541 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:42:24.861 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/3 2023-10-11T09:42:24.862 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:42:24.862 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:42:24.862 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/3" 2023-10-11T09:42:24.863 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'w'"'"') as f: 2023-10-11T09:42:24.863 DEBUG:teuthology.orchestra.run.smithi078:> for i in range(0, 6291456): 2023-10-11T09:42:24.864 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:42:24.864 DEBUG:teuthology.orchestra.run.smithi078:> f.write(chr(val)) 2023-10-11T09:42:24.864 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:42:28.189 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/4 2023-10-11T09:42:28.190 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:42:28.190 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:42:28.190 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/4" 2023-10-11T09:42:28.191 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'w'"'"') as f: 2023-10-11T09:42:28.191 DEBUG:teuthology.orchestra.run.smithi078:> for i in range(0, 6291456): 2023-10-11T09:42:28.192 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:42:28.192 DEBUG:teuthology.orchestra.run.smithi078:> f.write(chr(val)) 2023-10-11T09:42:28.192 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:42:29.330 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:42:29.339 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:42:31.517 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:42:31.518 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:42:31.536 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:42:31.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.748+0000 7f06d8f2c700 1 Processor -- start 2023-10-11T09:42:31.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.748+0000 7f06d8f2c700 1 -- start start 2023-10-11T09:42:31.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.752+0000 7f06d8f2c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f06d4071040 0x7f06d40cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:31.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.752+0000 7f06d8f2c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f06d40ce330 0x7f06d406be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:31.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.752+0000 7f06d8f2c700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06d406ffa0 con 0x7f06d4160d40 2023-10-11T09:42:31.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.752+0000 7f06d8f2c700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06d4070c20 con 0x7f06d40ce330 2023-10-11T09:42:31.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.752+0000 7f06d8f2c700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d4161110 con 0x7f06d4071040 2023-10-11T09:42:31.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.752+0000 7f06d2d9d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f06d40ce330 0x7f06d406be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:31.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.752+0000 7f06d2d9d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f06d40ce330 0x7f06d406be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37254/0 (socket says 172.21.15.78:37254) 2023-10-11T09:42:31.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.752+0000 7f06d2d9d700 1 -- 172.21.15.78:0/2564975519 learned_addr learned my addr 172.21.15.78:0/2564975519 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:31.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.752+0000 7f06d1d9b700 1 --2- 172.21.15.78:0/2564975519 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f06d4071040 0x7f06d40cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:31.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.752+0000 7f06d2d9d700 1 -- 172.21.15.78:0/2564975519 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f06d4071040 msgr2=0x7f06d40cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:31.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.752+0000 7f06d2d9d700 1 --2- 172.21.15.78:0/2564975519 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f06d4071040 0x7f06d40cddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:31.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.752+0000 7f06d2d9d700 1 -- 172.21.15.78:0/2564975519 >> v1:172.21.15.133:6789/0 conn(0x7f06d4160d40 legacy=0x7f06d40cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:31.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.752+0000 7f06d2d9d700 1 -- 172.21.15.78:0/2564975519 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d4070280 con 0x7f06d40ce330 2023-10-11T09:42:31.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.752+0000 7f06d1d9b700 1 --2- 172.21.15.78:0/2564975519 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f06d4071040 0x7f06d40cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:42:31.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.756+0000 7f06d2d9d700 1 --2- 172.21.15.78:0/2564975519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06d40ce330 0x7f06d406be60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f06cc009e50 tx=0x7f06cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=559c88d7d94a88b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:31.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.756+0000 7f06d159a700 1 -- 172.21.15.78:0/2564975519 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06cc00e070 con 0x7f06d40ce330 2023-10-11T09:42:31.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.756+0000 7f06d159a700 1 -- 172.21.15.78:0/2564975519 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06cc00b9c0 con 0x7f06d40ce330 2023-10-11T09:42:31.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.756+0000 7f06d159a700 1 -- 172.21.15.78:0/2564975519 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06cc00bd70 con 0x7f06d40ce330 2023-10-11T09:42:31.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.756+0000 7f06d8f2c700 1 -- 172.21.15.78:0/2564975519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06d40ce330 msgr2=0x7f06d406be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:31.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.756+0000 7f06d8f2c700 1 --2- 172.21.15.78:0/2564975519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06d40ce330 0x7f06d406be60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f06cc009e50 tx=0x7f06cc00b330 comp rx=0 tx=0).stop 2023-10-11T09:42:31.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.756+0000 7f06d8f2c700 1 -- 172.21.15.78:0/2564975519 shutdown_connections 2023-10-11T09:42:31.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.756+0000 7f06d8f2c700 1 --2- 172.21.15.78:0/2564975519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06d40ce330 0x7f06d406be60 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:31.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.756+0000 7f06d8f2c700 1 --2- 172.21.15.78:0/2564975519 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f06d4071040 0x7f06d40cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:31.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.756+0000 7f06d8f2c700 1 -- 172.21.15.78:0/2564975519 >> 172.21.15.78:0/2564975519 conn(0x7f06d406aca0 msgr2=0x7f06d40b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:31.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.756+0000 7f06d8f2c700 1 -- 172.21.15.78:0/2564975519 shutdown_connections 2023-10-11T09:42:31.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.756+0000 7f06d8f2c700 1 -- 172.21.15.78:0/2564975519 wait complete. 2023-10-11T09:42:31.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.760+0000 7f06d8f2c700 1 Processor -- start 2023-10-11T09:42:31.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.760+0000 7f06d8f2c700 1 -- start start 2023-10-11T09:42:31.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.760+0000 7f06d8f2c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f06d4071040 0x7f06d40c8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:31.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.760+0000 7f06d8f2c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06d40ce330 0x7f06d406be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:31.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.760+0000 7f06d8f2c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06d4160d40 0x7f06d406c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:31.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.760+0000 7f06d8f2c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d414cf60 con 0x7f06d4160d40 2023-10-11T09:42:31.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.760+0000 7f06d8f2c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d414d240 con 0x7f06d40ce330 2023-10-11T09:42:31.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.760+0000 7f06d8f2c700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d414d520 con 0x7f06d4071040 2023-10-11T09:42:31.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.760+0000 7f06d2d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06d4160d40 0x7f06d406c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:31.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.764+0000 7f06d2d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06d4160d40 0x7f06d406c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37260/0 (socket says 172.21.15.78:37260) 2023-10-11T09:42:31.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.764+0000 7f06d2d9d700 1 -- 172.21.15.78:0/4080893997 learned_addr learned my addr 172.21.15.78:0/4080893997 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:31.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.764+0000 7f06d1d9b700 1 --2- 172.21.15.78:0/4080893997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06d40ce330 0x7f06d406be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:31.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.764+0000 7f06d259c700 1 --2- 172.21.15.78:0/4080893997 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f06d4071040 0x7f06d40c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:31.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.764+0000 7f06d2d9d700 1 -- 172.21.15.78:0/4080893997 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f06d4071040 msgr2=0x7f06d40c8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:31.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.764+0000 7f06d2d9d700 1 --2- 172.21.15.78:0/4080893997 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f06d4071040 0x7f06d40c8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:31.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.764+0000 7f06d2d9d700 1 -- 172.21.15.78:0/4080893997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06d40ce330 msgr2=0x7f06d406be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:31.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.764+0000 7f06d2d9d700 1 --2- 172.21.15.78:0/4080893997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06d40ce330 0x7f06d406be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:31.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.764+0000 7f06d2d9d700 1 -- 172.21.15.78:0/4080893997 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06cc009b30 con 0x7f06d4160d40 2023-10-11T09:42:31.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.764+0000 7f06d2d9d700 1 --2- 172.21.15.78:0/4080893997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06d4160d40 0x7f06d406c3a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f06d406fbc0 tx=0x7f06cc004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:31.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.764+0000 7f06c2ffd700 1 -- 172.21.15.78:0/4080893997 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06cc004800 con 0x7f06d4160d40 2023-10-11T09:42:31.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.764+0000 7f06c2ffd700 1 -- 172.21.15.78:0/4080893997 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06cc0099b0 con 0x7f06d4160d40 2023-10-11T09:42:31.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.764+0000 7f06c2ffd700 1 -- 172.21.15.78:0/4080893997 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06cc01c680 con 0x7f06d4160d40 2023-10-11T09:42:31.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.764+0000 7f06d8f2c700 1 -- 172.21.15.78:0/4080893997 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06d414d800 con 0x7f06d4160d40 2023-10-11T09:42:31.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.764+0000 7f06d8f2c700 1 -- 172.21.15.78:0/4080893997 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06d40c3f40 con 0x7f06d4160d40 2023-10-11T09:42:31.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.764+0000 7f06d8f2c700 1 -- 172.21.15.78:0/4080893997 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06d4072250 con 0x7f06d4160d40 2023-10-11T09:42:31.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.768+0000 7f06c2ffd700 1 -- 172.21.15.78:0/4080893997 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f06cc01c7e0 con 0x7f06d4160d40 2023-10-11T09:42:31.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.768+0000 7f06c2ffd700 1 --2- 172.21.15.78:0/4080893997 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f06b4065ed0 0x7f06b4068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:31.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.768+0000 7f06c2ffd700 1 -- 172.21.15.78:0/4080893997 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6495+0+0 (secure 0 0 0) 0x7f06cc085540 con 0x7f06d4160d40 2023-10-11T09:42:31.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.768+0000 7f06d259c700 1 --2- 172.21.15.78:0/4080893997 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f06b4065ed0 0x7f06b4068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:31.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.772+0000 7f06d259c700 1 --2- 172.21.15.78:0/4080893997 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f06b4065ed0 0x7f06b4068380 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f06c4000f60 tx=0x7f06c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:31.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.776+0000 7f06c2ffd700 1 -- 172.21.15.78:0/4080893997 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f06cc0529d0 con 0x7f06d4160d40 2023-10-11T09:42:31.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.948+0000 7f06d8f2c700 1 -- 172.21.15.78:0/4080893997 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06d414dd20 con 0x7f06d4160d40 2023-10-11T09:42:31.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.948+0000 7f06c2ffd700 1 -- 172.21.15.78:0/4080893997 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+19720 (secure 0 0 0) 0x7f06d414dd20 con 0x7f06d4160d40 2023-10-11T09:42:31.957 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:31.957 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":42,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:42:22.478227+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:41:57.280079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-11T09:41:58.218860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data_extra","create_time":"2023-10-11T09:42:12.954795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.10","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:42:31.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.956+0000 7f06d8f2c700 1 -- 172.21.15.78:0/4080893997 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f06b4065ed0 msgr2=0x7f06b4068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:31.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.956+0000 7f06d8f2c700 1 --2- 172.21.15.78:0/4080893997 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f06b4065ed0 0x7f06b4068380 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f06c4000f60 tx=0x7f06c4009040 comp rx=0 tx=0).stop 2023-10-11T09:42:31.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.956+0000 7f06d8f2c700 1 -- 172.21.15.78:0/4080893997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06d4160d40 msgr2=0x7f06d406c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:31.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.956+0000 7f06d8f2c700 1 --2- 172.21.15.78:0/4080893997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06d4160d40 0x7f06d406c3a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f06d406fbc0 tx=0x7f06cc004af0 comp rx=0 tx=0).stop 2023-10-11T09:42:31.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.956+0000 7f06d8f2c700 1 -- 172.21.15.78:0/4080893997 shutdown_connections 2023-10-11T09:42:31.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.956+0000 7f06d8f2c700 1 --2- 172.21.15.78:0/4080893997 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f06b4065ed0 0x7f06b4068380 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:31.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.956+0000 7f06d8f2c700 1 --2- 172.21.15.78:0/4080893997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06d4160d40 0x7f06d406c3a0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:31.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.956+0000 7f06d8f2c700 1 --2- 172.21.15.78:0/4080893997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06d40ce330 0x7f06d406be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:31.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.956+0000 7f06d8f2c700 1 --2- 172.21.15.78:0/4080893997 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f06d4071040 0x7f06d40c8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:31.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.956+0000 7f06d8f2c700 1 -- 172.21.15.78:0/4080893997 >> 172.21.15.78:0/4080893997 conn(0x7f06d406aca0 msgr2=0x7f06d4161340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:31.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.956+0000 7f06d8f2c700 1 -- 172.21.15.78:0/4080893997 shutdown_connections 2023-10-11T09:42:31.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:31.956+0000 7f06d8f2c700 1 -- 172.21.15.78:0/4080893997 wait complete. 2023-10-11T09:42:31.982 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T09:41:57.280079+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T09:41:58.218860+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-10-11T09:42:12.954795+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:42:31.983 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:42:32.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.156+0000 7efdfac51700 1 Processor -- start 2023-10-11T09:42:32.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.156+0000 7efdfac51700 1 -- start start 2023-10-11T09:42:32.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.156+0000 7efdfac51700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efdf40b84c0 0x7efdf40cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:32.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.156+0000 7efdfac51700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdf40cd690 con 0x7efdf4160a80 2023-10-11T09:42:32.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.156+0000 7efdfac51700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdf40d02e0 con 0x7efdf40b8890 2023-10-11T09:42:32.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.156+0000 7efdfac51700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdf40cdea0 con 0x7efdf40b84c0 2023-10-11T09:42:32.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.160+0000 7efdf91ee700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7efdf4160a80 0x7efdf40d1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:46344/0 (socket says 172.21.15.78:46344) 2023-10-11T09:42:32.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.160+0000 7efdf91ee700 1 -- 172.21.15.78:0/1974221914 learned_addr learned my addr 172.21.15.78:0/1974221914 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:42:32.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.160+0000 7efdf89ed700 1 --2- 172.21.15.78:0/1974221914 >> v2:172.21.15.133:3300/0 conn(0x7efdf40b84c0 0x7efdf40cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:32.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.160+0000 7efdf37fe700 1 -- 172.21.15.78:0/1974221914 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 404222427 0 0) 0x7efdf40cd690 con 0x7efdf4160a80 2023-10-11T09:42:32.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.160+0000 7efdf37fe700 1 -- 172.21.15.78:0/1974221914 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efde40035d0 con 0x7efdf4160a80 2023-10-11T09:42:32.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.160+0000 7efdf89ed700 1 -- 172.21.15.78:0/1974221914 >> v1:172.21.15.133:6789/0 conn(0x7efdf40b8890 legacy=0x7efdf4160370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:32.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.160+0000 7efdf37fe700 1 -- 172.21.15.78:0/1974221914 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1231232858 0 0) 0x7efdf40d02e0 con 0x7efdf40b8890 2023-10-11T09:42:32.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.160+0000 7efdf89ed700 1 -- 172.21.15.78:0/1974221914 >> v1:172.21.15.78:6789/0 conn(0x7efdf4160a80 legacy=0x7efdf40d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:32.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.160+0000 7efdf89ed700 1 -- 172.21.15.78:0/1974221914 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf40cd8e0 con 0x7efdf40b84c0 2023-10-11T09:42:32.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.160+0000 7efdf89ed700 1 --2- 172.21.15.78:0/1974221914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdf40b84c0 0x7efdf40cfda0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efde8009850 tx=0x7efde8009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fad1fe4ff5da468a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:32.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.160+0000 7efdf37fe700 1 -- 172.21.15.78:0/1974221914 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde8011830 con 0x7efdf40b84c0 2023-10-11T09:42:32.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.160+0000 7efdf37fe700 1 -- 172.21.15.78:0/1974221914 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde8011990 con 0x7efdf40b84c0 2023-10-11T09:42:32.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.160+0000 7efdf37fe700 1 -- 172.21.15.78:0/1974221914 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde8011d40 con 0x7efdf40b84c0 2023-10-11T09:42:32.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.160+0000 7efdfac51700 1 -- 172.21.15.78:0/1974221914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdf40b84c0 msgr2=0x7efdf40cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:32.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.160+0000 7efdfac51700 1 --2- 172.21.15.78:0/1974221914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdf40b84c0 0x7efdf40cfda0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efde8009850 tx=0x7efde8009b60 comp rx=0 tx=0).stop 2023-10-11T09:42:32.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.160+0000 7efdfac51700 1 -- 172.21.15.78:0/1974221914 shutdown_connections 2023-10-11T09:42:32.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.160+0000 7efdfac51700 1 --2- 172.21.15.78:0/1974221914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdf40b84c0 0x7efdf40cfda0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:32.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.160+0000 7efdfac51700 1 -- 172.21.15.78:0/1974221914 >> 172.21.15.78:0/1974221914 conn(0x7efdf406cb50 msgr2=0x7efdf406cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:32.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.164+0000 7efdfac51700 1 -- 172.21.15.78:0/1974221914 shutdown_connections 2023-10-11T09:42:32.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.164+0000 7efdfac51700 1 -- 172.21.15.78:0/1974221914 wait complete. 2023-10-11T09:42:32.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.164+0000 7efdfac51700 1 Processor -- start 2023-10-11T09:42:32.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.164+0000 7efdfac51700 1 -- start start 2023-10-11T09:42:32.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.168+0000 7efdfac51700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdf40b84c0 0x7efdf40c6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:32.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.168+0000 7efdfac51700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efdf40b8890 0x7efdf41f1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:32.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.168+0000 7efdfac51700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdf4160a80 0x7efdf40cc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:32.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.168+0000 7efdfac51700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf40c7b50 con 0x7efdf4160a80 2023-10-11T09:42:32.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.168+0000 7efdfac51700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf40cbd30 con 0x7efdf40b84c0 2023-10-11T09:42:32.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.168+0000 7efdfac51700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdf40c7130 con 0x7efdf40b8890 2023-10-11T09:42:32.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.168+0000 7efdf91ee700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdf4160a80 0x7efdf40cc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:32.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.168+0000 7efdf91ee700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdf4160a80 0x7efdf40cc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37270/0 (socket says 172.21.15.78:37270) 2023-10-11T09:42:32.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.168+0000 7efdf91ee700 1 -- 172.21.15.78:0/4034763674 learned_addr learned my addr 172.21.15.78:0/4034763674 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:32.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.168+0000 7efdf89ed700 1 --2- 172.21.15.78:0/4034763674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdf40b84c0 0x7efdf40c6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:32.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.168+0000 7efdf3fff700 1 --2- 172.21.15.78:0/4034763674 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efdf40b8890 0x7efdf41f1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:32.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.168+0000 7efdf91ee700 1 -- 172.21.15.78:0/4034763674 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efdf40b8890 msgr2=0x7efdf41f1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:32.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.168+0000 7efdf91ee700 1 --2- 172.21.15.78:0/4034763674 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efdf40b8890 0x7efdf41f1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:32.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.168+0000 7efdf91ee700 1 -- 172.21.15.78:0/4034763674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdf40b84c0 msgr2=0x7efdf40c6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:32.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.168+0000 7efdf91ee700 1 --2- 172.21.15.78:0/4034763674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdf40b84c0 0x7efdf40c6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:32.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.168+0000 7efdf91ee700 1 -- 172.21.15.78:0/4034763674 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde80094b0 con 0x7efdf4160a80 2023-10-11T09:42:32.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.168+0000 7efdf91ee700 1 --2- 172.21.15.78:0/4034763674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdf4160a80 0x7efdf40cc1c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7efdec00caa0 tx=0x7efdec00cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:32.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.172+0000 7efdf17fa700 1 -- 172.21.15.78:0/4034763674 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec007480 con 0x7efdf4160a80 2023-10-11T09:42:32.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.172+0000 7efdf17fa700 1 -- 172.21.15.78:0/4034763674 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdec0075e0 con 0x7efdf4160a80 2023-10-11T09:42:32.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.172+0000 7efdfac51700 1 -- 172.21.15.78:0/4034763674 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdf40c7590 con 0x7efdf4160a80 2023-10-11T09:42:32.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.172+0000 7efdf17fa700 1 -- 172.21.15.78:0/4034763674 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec007970 con 0x7efdf4160a80 2023-10-11T09:42:32.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.172+0000 7efdfac51700 1 -- 172.21.15.78:0/4034763674 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdf415fb40 con 0x7efdf4160a80 2023-10-11T09:42:32.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.172+0000 7efdfac51700 1 -- 172.21.15.78:0/4034763674 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdf40cbee0 con 0x7efdf4160a80 2023-10-11T09:42:32.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.172+0000 7efdf17fa700 1 -- 172.21.15.78:0/4034763674 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7efdec01b3b0 con 0x7efdf4160a80 2023-10-11T09:42:32.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.176+0000 7efdf17fa700 1 --2- 172.21.15.78:0/4034763674 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efdd8065ed0 0x7efdd8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:32.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.176+0000 7efdf17fa700 1 -- 172.21.15.78:0/4034763674 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6495+0+0 (secure 0 0 0) 0x7efdec083280 con 0x7efdf4160a80 2023-10-11T09:42:32.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.176+0000 7efdf89ed700 1 --2- 172.21.15.78:0/4034763674 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efdd8065ed0 0x7efdd8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:32.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.176+0000 7efdf89ed700 1 --2- 172.21.15.78:0/4034763674 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efdd8065ed0 0x7efdd8068380 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7efde80090b0 tx=0x7efde8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:32.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.180+0000 7efdf17fa700 1 -- 172.21.15.78:0/4034763674 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7efdf40cbee0 con 0x7efdf4160a80 2023-10-11T09:42:32.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.356+0000 7efdfac51700 1 -- 172.21.15.78:0/4034763674 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efdf40cbea0 con 0x7efdf4160a80 2023-10-11T09:42:32.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.356+0000 7efdf17fa700 1 -- 172.21.15.78:0/4034763674 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7efdec050710 con 0x7efdf4160a80 2023-10-11T09:42:32.363 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:42:32.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.360+0000 7efdfac51700 1 -- 172.21.15.78:0/4034763674 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efdd8065ed0 msgr2=0x7efdd8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:32.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.360+0000 7efdfac51700 1 --2- 172.21.15.78:0/4034763674 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efdd8065ed0 0x7efdd8068380 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7efde80090b0 tx=0x7efde8008030 comp rx=0 tx=0).stop 2023-10-11T09:42:32.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.360+0000 7efdfac51700 1 -- 172.21.15.78:0/4034763674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdf4160a80 msgr2=0x7efdf40cc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:32.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.360+0000 7efdfac51700 1 --2- 172.21.15.78:0/4034763674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdf4160a80 0x7efdf40cc1c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7efdec00caa0 tx=0x7efdec00cad0 comp rx=0 tx=0).stop 2023-10-11T09:42:32.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.360+0000 7efdfac51700 1 -- 172.21.15.78:0/4034763674 shutdown_connections 2023-10-11T09:42:32.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.360+0000 7efdfac51700 1 --2- 172.21.15.78:0/4034763674 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efdd8065ed0 0x7efdd8068380 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:32.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.360+0000 7efdfac51700 1 --2- 172.21.15.78:0/4034763674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdf4160a80 0x7efdf40cc1c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:32.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.360+0000 7efdfac51700 1 --2- 172.21.15.78:0/4034763674 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efdf40b8890 0x7efdf41f1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:32.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.360+0000 7efdfac51700 1 --2- 172.21.15.78:0/4034763674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdf40b84c0 0x7efdf40c6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:32.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.360+0000 7efdfac51700 1 -- 172.21.15.78:0/4034763674 >> 172.21.15.78:0/4034763674 conn(0x7efdf406cb50 msgr2=0x7efdf4066110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:32.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.360+0000 7efdfac51700 1 -- 172.21.15.78:0/4034763674 shutdown_connections 2023-10-11T09:42:32.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.360+0000 7efdfac51700 1 -- 172.21.15.78:0/4034763674 wait complete. 2023-10-11T09:42:32.389 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:42:32.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.552+0000 7f3be9b3e700 1 Processor -- start 2023-10-11T09:42:32.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.552+0000 7f3be9b3e700 1 -- start start 2023-10-11T09:42:32.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.556+0000 7f3be9b3e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f3be40b3610 0x7f3be40d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:32.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.556+0000 7f3be9b3e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3be40b3a90 0x7f3be40b85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:32.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.556+0000 7f3be9b3e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3be40b47f0 con 0x7f3be40b3240 2023-10-11T09:42:32.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.556+0000 7f3be9b3e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3be40b8130 con 0x7f3be40b3610 2023-10-11T09:42:32.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.556+0000 7f3be9b3e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3be40b41e0 con 0x7f3be40b3a90 2023-10-11T09:42:32.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.556+0000 7f3be3fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f3be40b3610 0x7f3be40d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:32.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.556+0000 7f3be8b3c700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f3be40b3240 0x7f3be415d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:46352/0 (socket says 172.21.15.78:46352) 2023-10-11T09:42:32.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.556+0000 7f3be8b3c700 1 -- 172.21.15.78:0/2835337784 learned_addr learned my addr 172.21.15.78:0/2835337784 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:42:32.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.556+0000 7f3be933d700 1 --2- 172.21.15.78:0/2835337784 >> v2:172.21.15.133:3300/0 conn(0x7f3be40b3a90 0x7f3be40b85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:32.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.556+0000 7f3be3fff700 1 -- 172.21.15.78:0/2835337784 >> v2:172.21.15.133:3300/0 conn(0x7f3be40b3a90 msgr2=0x7f3be40b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:32.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.556+0000 7f3be3fff700 1 --2- 172.21.15.78:0/2835337784 >> v2:172.21.15.133:3300/0 conn(0x7f3be40b3a90 0x7f3be40b85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:32.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.556+0000 7f3be3fff700 1 -- 172.21.15.78:0/2835337784 >> v1:172.21.15.78:6789/0 conn(0x7f3be40b3240 legacy=0x7f3be415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:32.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.556+0000 7f3be37fe700 1 -- 172.21.15.78:0/2835337784 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1873246229 0 0) 0x7f3bd8001110 con 0x7f3be40b3240 2023-10-11T09:42:32.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.556+0000 7f3be3fff700 1 -- 172.21.15.78:0/2835337784 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be40b7800 con 0x7f3be40b3610 2023-10-11T09:42:32.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.560+0000 7f3be3fff700 1 --2- 172.21.15.78:0/2835337784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3be40b3610 0x7f3be40d29d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3be4072b40 tx=0x7f3bd0009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ee061c9bf2139f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:32.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.560+0000 7f3be37fe700 1 -- 172.21.15.78:0/2835337784 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0011830 con 0x7f3be40b3610 2023-10-11T09:42:32.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.560+0000 7f3be37fe700 1 -- 172.21.15.78:0/2835337784 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bd0011990 con 0x7f3be40b3610 2023-10-11T09:42:32.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.560+0000 7f3be37fe700 1 -- 172.21.15.78:0/2835337784 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0011d20 con 0x7f3be40b3610 2023-10-11T09:42:32.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.560+0000 7f3be9b3e700 1 -- 172.21.15.78:0/2835337784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3be40b3610 msgr2=0x7f3be40d29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:32.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.560+0000 7f3be9b3e700 1 --2- 172.21.15.78:0/2835337784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3be40b3610 0x7f3be40d29d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3be4072b40 tx=0x7f3bd0009b30 comp rx=0 tx=0).stop 2023-10-11T09:42:32.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.560+0000 7f3be9b3e700 1 -- 172.21.15.78:0/2835337784 shutdown_connections 2023-10-11T09:42:32.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.560+0000 7f3be9b3e700 1 --2- 172.21.15.78:0/2835337784 >> v2:172.21.15.133:3300/0 conn(0x7f3be40b3a90 0x7f3be40b85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:32.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.560+0000 7f3be9b3e700 1 --2- 172.21.15.78:0/2835337784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3be40b3610 0x7f3be40d29d0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:32.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.560+0000 7f3be9b3e700 1 -- 172.21.15.78:0/2835337784 >> 172.21.15.78:0/2835337784 conn(0x7f3be409c260 msgr2=0x7f3be40d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:32.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.560+0000 7f3be9b3e700 1 -- 172.21.15.78:0/2835337784 shutdown_connections 2023-10-11T09:42:32.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.560+0000 7f3be9b3e700 1 -- 172.21.15.78:0/2835337784 wait complete. 2023-10-11T09:42:32.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.564+0000 7f3be9b3e700 1 Processor -- start 2023-10-11T09:42:32.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.564+0000 7f3be9b3e700 1 -- start start 2023-10-11T09:42:32.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.564+0000 7f3be9b3e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3be40b3240 0x7f3be4158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:32.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.564+0000 7f3be9b3e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3be40b3610 0x7f3be409d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:32.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.564+0000 7f3be9b3e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3be40b3a90 0x7f3be415ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:32.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.564+0000 7f3be9b3e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be409e2c0 con 0x7f3be40b3a90 2023-10-11T09:42:32.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.564+0000 7f3be9b3e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be409e5a0 con 0x7f3be40b3240 2023-10-11T09:42:32.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.564+0000 7f3be9b3e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3be409e880 con 0x7f3be40b3610 2023-10-11T09:42:32.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.564+0000 7f3be933d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3be40b3a90 0x7f3be415ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:32.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.564+0000 7f3be933d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3be40b3a90 0x7f3be415ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37286/0 (socket says 172.21.15.78:37286) 2023-10-11T09:42:32.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.564+0000 7f3be933d700 1 -- 172.21.15.78:0/643946355 learned_addr learned my addr 172.21.15.78:0/643946355 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:32.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.564+0000 7f3be8b3c700 1 --2- 172.21.15.78:0/643946355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3be40b3240 0x7f3be4158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:32.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.568+0000 7f3be3fff700 1 --2- 172.21.15.78:0/643946355 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3be40b3610 0x7f3be409d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:32.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.568+0000 7f3be933d700 1 -- 172.21.15.78:0/643946355 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3be40b3610 msgr2=0x7f3be409d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:32.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.568+0000 7f3be933d700 1 --2- 172.21.15.78:0/643946355 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3be40b3610 0x7f3be409d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:32.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.568+0000 7f3be933d700 1 -- 172.21.15.78:0/643946355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3be40b3240 msgr2=0x7f3be4158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:32.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.568+0000 7f3be933d700 1 --2- 172.21.15.78:0/643946355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3be40b3240 0x7f3be4158f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:32.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.568+0000 7f3be933d700 1 -- 172.21.15.78:0/643946355 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd00094b0 con 0x7f3be40b3a90 2023-10-11T09:42:32.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.568+0000 7f3be933d700 1 --2- 172.21.15.78:0/643946355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3be40b3a90 0x7f3be415ce00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00efc0 tx=0x7f3bdc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:32.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.568+0000 7f3be17fa700 1 -- 172.21.15.78:0/643946355 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc009ba0 con 0x7f3be40b3a90 2023-10-11T09:42:32.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.568+0000 7f3be17fa700 1 -- 172.21.15.78:0/643946355 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bdc00f040 con 0x7f3be40b3a90 2023-10-11T09:42:32.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.568+0000 7f3be17fa700 1 -- 172.21.15.78:0/643946355 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc014480 con 0x7f3be40b3a90 2023-10-11T09:42:32.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.568+0000 7f3be9b3e700 1 -- 172.21.15.78:0/643946355 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3be409eb60 con 0x7f3be40b3a90 2023-10-11T09:42:32.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.568+0000 7f3be9b3e700 1 -- 172.21.15.78:0/643946355 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3be41594d0 con 0x7f3be40b3a90 2023-10-11T09:42:32.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.568+0000 7f3be9b3e700 1 -- 172.21.15.78:0/643946355 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3be409f080 con 0x7f3be40b3a90 2023-10-11T09:42:32.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.568+0000 7f3be17fa700 1 -- 172.21.15.78:0/643946355 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f3bdc0145e0 con 0x7f3be40b3a90 2023-10-11T09:42:32.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.572+0000 7f3be17fa700 1 --2- 172.21.15.78:0/643946355 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3bc8065ed0 0x7f3bc8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:32.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.572+0000 7f3be17fa700 1 -- 172.21.15.78:0/643946355 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6495+0+0 (secure 0 0 0) 0x7f3bdc084300 con 0x7f3be40b3a90 2023-10-11T09:42:32.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.572+0000 7f3be8b3c700 1 --2- 172.21.15.78:0/643946355 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3bc8065ed0 0x7f3bc8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:32.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.572+0000 7f3be8b3c700 1 --2- 172.21.15.78:0/643946355 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3bc8065ed0 0x7f3bc8068380 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8000f60 tx=0x7f3bd8007cb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:32.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.576+0000 7f3be17fa700 1 -- 172.21.15.78:0/643946355 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f3be409f080 con 0x7f3be40b3a90 2023-10-11T09:42:32.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.744+0000 7f3be9b3e700 1 -- 172.21.15.78:0/643946355 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3be409f080 con 0x7f3be40b3a90 2023-10-11T09:42:32.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.748+0000 7f3be17fa700 1 -- 172.21.15.78:0/643946355 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v42) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3be409f080 con 0x7f3be40b3a90 2023-10-11T09:42:32.754 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:42:32.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.752+0000 7f3be9b3e700 1 -- 172.21.15.78:0/643946355 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3bc8065ed0 msgr2=0x7f3bc8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:32.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.752+0000 7f3be9b3e700 1 --2- 172.21.15.78:0/643946355 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3bc8065ed0 0x7f3bc8068380 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8000f60 tx=0x7f3bd8007cb0 comp rx=0 tx=0).stop 2023-10-11T09:42:32.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.752+0000 7f3be9b3e700 1 -- 172.21.15.78:0/643946355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3be40b3a90 msgr2=0x7f3be415ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:32.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.752+0000 7f3be9b3e700 1 --2- 172.21.15.78:0/643946355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3be40b3a90 0x7f3be415ce00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00efc0 tx=0x7f3bdc00c3a0 comp rx=0 tx=0).stop 2023-10-11T09:42:32.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.752+0000 7f3be9b3e700 1 -- 172.21.15.78:0/643946355 shutdown_connections 2023-10-11T09:42:32.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.752+0000 7f3be9b3e700 1 --2- 172.21.15.78:0/643946355 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3bc8065ed0 0x7f3bc8068380 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:32.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.752+0000 7f3be9b3e700 1 --2- 172.21.15.78:0/643946355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3be40b3a90 0x7f3be415ce00 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:32.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.752+0000 7f3be9b3e700 1 --2- 172.21.15.78:0/643946355 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3be40b3610 0x7f3be409d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:32.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.752+0000 7f3be9b3e700 1 --2- 172.21.15.78:0/643946355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3be40b3240 0x7f3be4158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:32.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.752+0000 7f3be9b3e700 1 -- 172.21.15.78:0/643946355 >> 172.21.15.78:0/643946355 conn(0x7f3be409c260 msgr2=0x7f3be40d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:32.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.752+0000 7f3be9b3e700 1 -- 172.21.15.78:0/643946355 shutdown_connections 2023-10-11T09:42:32.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.752+0000 7f3be9b3e700 1 -- 172.21.15.78:0/643946355 wait complete. 2023-10-11T09:42:32.780 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T09:42:32.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.952+0000 7f63fc1d9700 1 Processor -- start 2023-10-11T09:42:32.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.952+0000 7f63fc1d9700 1 -- start start 2023-10-11T09:42:32.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.952+0000 7f63fc1d9700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f63f4158430 0x7f63f40710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:32.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.952+0000 7f63fc1d9700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f63f406d340 0x7f63f40ca880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:32.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.952+0000 7f63fc1d9700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63f40cadc0 0x7f63f415afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:32.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.952+0000 7f63fc1d9700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63f4070c10 con 0x7f63f406d340 2023-10-11T09:42:32.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.952+0000 7f63fc1d9700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63f40cb240 con 0x7f63f4158430 2023-10-11T09:42:32.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.952+0000 7f63fc1d9700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f4071610 con 0x7f63f40cadc0 2023-10-11T09:42:32.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.952+0000 7f63f9774700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f63f406d340 0x7f63f40ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:32.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.952+0000 7f63f9774700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f63f406d340 0x7f63f40ca880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37294/0 (socket says 172.21.15.78:37294) 2023-10-11T09:42:32.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.952+0000 7f63f9774700 1 -- 172.21.15.78:0/2002352368 learned_addr learned my addr 172.21.15.78:0/2002352368 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:32.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.952+0000 7f63fa776700 1 --2- 172.21.15.78:0/2002352368 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63f40cadc0 0x7f63f415afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:32.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.952+0000 7f63f9f75700 1 --2- 172.21.15.78:0/2002352368 >> v2:172.21.15.133:3300/0 conn(0x7f63f4158430 0x7f63f40710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:32.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.952+0000 7f63f9774700 1 -- 172.21.15.78:0/2002352368 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63f40cadc0 msgr2=0x7f63f415afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:32.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.952+0000 7f63f9774700 1 --2- 172.21.15.78:0/2002352368 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63f40cadc0 0x7f63f415afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:32.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.956+0000 7f63f9774700 1 -- 172.21.15.78:0/2002352368 >> v2:172.21.15.133:3300/0 conn(0x7f63f4158430 msgr2=0x7f63f40710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:32.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.956+0000 7f63f9774700 1 --2- 172.21.15.78:0/2002352368 >> v2:172.21.15.133:3300/0 conn(0x7f63f4158430 0x7f63f40710d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:32.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.956+0000 7f63f9774700 1 -- 172.21.15.78:0/2002352368 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f415e280 con 0x7f63f406d340 2023-10-11T09:42:32.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.956+0000 7f63f9f75700 1 --2- 172.21.15.78:0/2002352368 >> v2:172.21.15.133:3300/0 conn(0x7f63f4158430 0x7f63f40710d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:42:32.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.956+0000 7f63f9774700 1 --2- 172.21.15.78:0/2002352368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63f406d340 0x7f63f40ca880 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f63f406fd00 tx=0x7f63e4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f30ab0963977da71 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:32.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.956+0000 7f63f8f73700 1 -- 172.21.15.78:0/2002352368 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e4011830 con 0x7f63f406d340 2023-10-11T09:42:32.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.956+0000 7f63f8f73700 1 -- 172.21.15.78:0/2002352368 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e4011990 con 0x7f63f406d340 2023-10-11T09:42:32.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.956+0000 7f63f8f73700 1 -- 172.21.15.78:0/2002352368 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e4011d20 con 0x7f63f406d340 2023-10-11T09:42:32.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.956+0000 7f63fc1d9700 1 -- 172.21.15.78:0/2002352368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63f406d340 msgr2=0x7f63f40ca880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:32.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.956+0000 7f63fc1d9700 1 --2- 172.21.15.78:0/2002352368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63f406d340 0x7f63f40ca880 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f63f406fd00 tx=0x7f63e4009bf0 comp rx=0 tx=0).stop 2023-10-11T09:42:32.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.956+0000 7f63fc1d9700 1 -- 172.21.15.78:0/2002352368 shutdown_connections 2023-10-11T09:42:32.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.956+0000 7f63fc1d9700 1 --2- 172.21.15.78:0/2002352368 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63f40cadc0 0x7f63f415afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:32.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.956+0000 7f63fc1d9700 1 --2- 172.21.15.78:0/2002352368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63f406d340 0x7f63f40ca880 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:32.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.956+0000 7f63fc1d9700 1 --2- 172.21.15.78:0/2002352368 >> v2:172.21.15.133:3300/0 conn(0x7f63f4158430 0x7f63f40710d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:32.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.956+0000 7f63fc1d9700 1 -- 172.21.15.78:0/2002352368 >> 172.21.15.78:0/2002352368 conn(0x7f63f40cecc0 msgr2=0x7f63f40cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:32.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.956+0000 7f63fc1d9700 1 -- 172.21.15.78:0/2002352368 shutdown_connections 2023-10-11T09:42:32.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.956+0000 7f63fc1d9700 1 -- 172.21.15.78:0/2002352368 wait complete. 2023-10-11T09:42:32.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.960+0000 7f63fc1d9700 1 Processor -- start 2023-10-11T09:42:32.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.960+0000 7f63fc1d9700 1 -- start start 2023-10-11T09:42:32.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.960+0000 7f63fc1d9700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63f406d340 0x7f63f414a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:32.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.960+0000 7f63fc1d9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63f40cadc0 0x7f63f40ce100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:32.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.960+0000 7f63fc1d9700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63f4158430 0x7f63f41f5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:32.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.960+0000 7f63fc1d9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f40c8730 con 0x7f63f40cadc0 2023-10-11T09:42:32.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.960+0000 7f63fc1d9700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f40c8a10 con 0x7f63f406d340 2023-10-11T09:42:32.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.960+0000 7f63fc1d9700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f40c8cf0 con 0x7f63f4158430 2023-10-11T09:42:32.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.960+0000 7f63f9774700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63f40cadc0 0x7f63f40ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:32.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.960+0000 7f63f9774700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63f40cadc0 0x7f63f40ce100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37304/0 (socket says 172.21.15.78:37304) 2023-10-11T09:42:32.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.960+0000 7f63f9774700 1 -- 172.21.15.78:0/3563571080 learned_addr learned my addr 172.21.15.78:0/3563571080 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:32.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.960+0000 7f63fa776700 1 --2- 172.21.15.78:0/3563571080 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63f4158430 0x7f63f41f5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:32.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.960+0000 7f63f9f75700 1 --2- 172.21.15.78:0/3563571080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63f406d340 0x7f63f414a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:32.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.960+0000 7f63f9774700 1 -- 172.21.15.78:0/3563571080 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63f4158430 msgr2=0x7f63f41f5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:32.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.960+0000 7f63f9774700 1 --2- 172.21.15.78:0/3563571080 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63f4158430 0x7f63f41f5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:32.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.960+0000 7f63f9774700 1 -- 172.21.15.78:0/3563571080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63f406d340 msgr2=0x7f63f414a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:32.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.960+0000 7f63f9774700 1 --2- 172.21.15.78:0/3563571080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63f406d340 0x7f63f414a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:32.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.960+0000 7f63f9774700 1 -- 172.21.15.78:0/3563571080 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e4009570 con 0x7f63f40cadc0 2023-10-11T09:42:32.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.960+0000 7f63f9f75700 1 --2- 172.21.15.78:0/3563571080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63f406d340 0x7f63f414a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:42:32.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.960+0000 7f63f9774700 1 --2- 172.21.15.78:0/3563571080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63f40cadc0 0x7f63f40ce100 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f63f406fbc0 tx=0x7f63e4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:32.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.960+0000 7f63ea7fc700 1 -- 172.21.15.78:0/3563571080 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e40044d0 con 0x7f63f40cadc0 2023-10-11T09:42:32.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.964+0000 7f63ea7fc700 1 -- 172.21.15.78:0/3563571080 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e401f070 con 0x7f63f40cadc0 2023-10-11T09:42:32.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.964+0000 7f63ea7fc700 1 -- 172.21.15.78:0/3563571080 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e4003900 con 0x7f63f40cadc0 2023-10-11T09:42:32.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.964+0000 7f63fc1d9700 1 -- 172.21.15.78:0/3563571080 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63f40c8fd0 con 0x7f63f40cadc0 2023-10-11T09:42:32.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.964+0000 7f63fc1d9700 1 -- 172.21.15.78:0/3563571080 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f40ce640 con 0x7f63f40cadc0 2023-10-11T09:42:32.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.964+0000 7f63fc1d9700 1 -- 172.21.15.78:0/3563571080 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63f406ae30 con 0x7f63f40cadc0 2023-10-11T09:42:32.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.964+0000 7f63ea7fc700 1 -- 172.21.15.78:0/3563571080 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f63e4003a60 con 0x7f63f40cadc0 2023-10-11T09:42:32.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.968+0000 7f63ea7fc700 1 --2- 172.21.15.78:0/3563571080 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f63d8065ed0 0x7f63d8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:32.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.968+0000 7f63ea7fc700 1 -- 172.21.15.78:0/3563571080 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6495+0+0 (secure 0 0 0) 0x7f63e4021070 con 0x7f63f40cadc0 2023-10-11T09:42:32.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.968+0000 7f63f9f75700 1 --2- 172.21.15.78:0/3563571080 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f63d8065ed0 0x7f63d8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:32.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.968+0000 7f63f9f75700 1 --2- 172.21.15.78:0/3563571080 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f63d8065ed0 0x7f63d8068380 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f63f40b41c0 tx=0x7f63f0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:32.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:32.972+0000 7f63ea7fc700 1 -- 172.21.15.78:0/3563571080 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f63e4052290 con 0x7f63f40cadc0 2023-10-11T09:42:33.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.148+0000 7f63fc1d9700 1 -- 172.21.15.78:0/3563571080 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f63f4065db0 con 0x7f63f40cadc0 2023-10-11T09:42:33.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.148+0000 7f63ea7fc700 1 -- 172.21.15.78:0/3563571080 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+11 (secure 0 0 0) 0x7f63e40150a0 con 0x7f63f40cadc0 2023-10-11T09:42:33.157 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:42:33.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.152+0000 7f63fc1d9700 1 -- 172.21.15.78:0/3563571080 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f63d8065ed0 msgr2=0x7f63d8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:33.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.152+0000 7f63fc1d9700 1 --2- 172.21.15.78:0/3563571080 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f63d8065ed0 0x7f63d8068380 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f63f40b41c0 tx=0x7f63f0008040 comp rx=0 tx=0).stop 2023-10-11T09:42:33.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.152+0000 7f63fc1d9700 1 -- 172.21.15.78:0/3563571080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63f40cadc0 msgr2=0x7f63f40ce100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:33.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.152+0000 7f63fc1d9700 1 --2- 172.21.15.78:0/3563571080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63f40cadc0 0x7f63f40ce100 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f63f406fbc0 tx=0x7f63e4004770 comp rx=0 tx=0).stop 2023-10-11T09:42:33.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.156+0000 7f63fc1d9700 1 -- 172.21.15.78:0/3563571080 shutdown_connections 2023-10-11T09:42:33.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.156+0000 7f63fc1d9700 1 --2- 172.21.15.78:0/3563571080 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f63d8065ed0 0x7f63d8068380 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:33.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.156+0000 7f63fc1d9700 1 --2- 172.21.15.78:0/3563571080 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63f4158430 0x7f63f41f5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:33.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.156+0000 7f63fc1d9700 1 --2- 172.21.15.78:0/3563571080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63f40cadc0 0x7f63f40ce100 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:33.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.156+0000 7f63fc1d9700 1 --2- 172.21.15.78:0/3563571080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63f406d340 0x7f63f414a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:33.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.156+0000 7f63fc1d9700 1 -- 172.21.15.78:0/3563571080 >> 172.21.15.78:0/3563571080 conn(0x7f63f40cecc0 msgr2=0x7f63f4156d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:33.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.156+0000 7f63fc1d9700 1 -- 172.21.15.78:0/3563571080 shutdown_connections 2023-10-11T09:42:33.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.156+0000 7f63fc1d9700 1 -- 172.21.15.78:0/3563571080 wait complete. 2023-10-11T09:42:33.180 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T09:42:33.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.344+0000 7f66b0cfa700 1 Processor -- start 2023-10-11T09:42:33.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.344+0000 7f66b0cfa700 1 -- start start 2023-10-11T09:42:33.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.348+0000 7f66b0cfa700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f66ac0b8890 0x7f66ac0d2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:33.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.348+0000 7f66b0cfa700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f66ac0d3510 0x7f66ac0d3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:33.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.348+0000 7f66b0cfa700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66ac0cd690 con 0x7f66ac0b84c0 2023-10-11T09:42:33.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.348+0000 7f66b0cfa700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66ac0d3eb0 con 0x7f66ac0d3510 2023-10-11T09:42:33.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.348+0000 7f66b0cfa700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66ac0cdea0 con 0x7f66ac0b8890 2023-10-11T09:42:33.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.348+0000 7f66aa59c700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f66ac0b84c0 0x7f66ac0cfda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:46360/0 (socket says 172.21.15.78:46360) 2023-10-11T09:42:33.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.348+0000 7f66aa59c700 1 -- 172.21.15.78:0/1058530574 learned_addr learned my addr 172.21.15.78:0/1058530574 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:42:33.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.348+0000 7f66aad9d700 1 --2- 172.21.15.78:0/1058530574 >> v2:172.21.15.78:3300/0 conn(0x7f66ac0d3510 0x7f66ac0d3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:33.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.348+0000 7f66a9d9b700 1 --2- 172.21.15.78:0/1058530574 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f66ac0b8890 0x7f66ac0d2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:33.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.348+0000 7f66a959a700 1 -- 172.21.15.78:0/1058530574 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 606702105 0 0) 0x7f66ac0cd690 con 0x7f66ac0b84c0 2023-10-11T09:42:33.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.348+0000 7f66a959a700 1 -- 172.21.15.78:0/1058530574 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66940035d0 con 0x7f66ac0b84c0 2023-10-11T09:42:33.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.348+0000 7f66aad9d700 1 -- 172.21.15.78:0/1058530574 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f66ac0b8890 msgr2=0x7f66ac0d2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:33.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.348+0000 7f66aad9d700 1 --2- 172.21.15.78:0/1058530574 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f66ac0b8890 0x7f66ac0d2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:33.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.348+0000 7f66aad9d700 1 -- 172.21.15.78:0/1058530574 >> v1:172.21.15.78:6789/0 conn(0x7f66ac0b84c0 legacy=0x7f66ac0cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:33.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.348+0000 7f66aad9d700 1 -- 172.21.15.78:0/1058530574 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66ac0cd8e0 con 0x7f66ac0d3510 2023-10-11T09:42:33.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.352+0000 7f66aad9d700 1 --2- 172.21.15.78:0/1058530574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66ac0d3510 0x7f66ac0d3970 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f66a4009d90 tx=0x7f66a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce52bee3fbb34bd server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:33.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.352+0000 7f66a959a700 1 -- 172.21.15.78:0/1058530574 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a400e070 con 0x7f66ac0d3510 2023-10-11T09:42:33.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.352+0000 7f66a959a700 1 -- 172.21.15.78:0/1058530574 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a400bb20 con 0x7f66ac0d3510 2023-10-11T09:42:33.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.352+0000 7f66a959a700 1 -- 172.21.15.78:0/1058530574 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a400be90 con 0x7f66ac0d3510 2023-10-11T09:42:33.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.352+0000 7f66b0cfa700 1 -- 172.21.15.78:0/1058530574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66ac0d3510 msgr2=0x7f66ac0d3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:33.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.352+0000 7f66b0cfa700 1 --2- 172.21.15.78:0/1058530574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66ac0d3510 0x7f66ac0d3970 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f66a4009d90 tx=0x7f66a400b330 comp rx=0 tx=0).stop 2023-10-11T09:42:33.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.352+0000 7f66b0cfa700 1 -- 172.21.15.78:0/1058530574 shutdown_connections 2023-10-11T09:42:33.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.352+0000 7f66b0cfa700 1 --2- 172.21.15.78:0/1058530574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66ac0d3510 0x7f66ac0d3970 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:33.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.352+0000 7f66b0cfa700 1 --2- 172.21.15.78:0/1058530574 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f66ac0b8890 0x7f66ac0d2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:33.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.352+0000 7f66b0cfa700 1 -- 172.21.15.78:0/1058530574 >> 172.21.15.78:0/1058530574 conn(0x7f66ac06cb50 msgr2=0x7f66ac06cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:33.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.352+0000 7f66b0cfa700 1 -- 172.21.15.78:0/1058530574 shutdown_connections 2023-10-11T09:42:33.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.352+0000 7f66b0cfa700 1 -- 172.21.15.78:0/1058530574 wait complete. 2023-10-11T09:42:33.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.356+0000 7f66b0cfa700 1 Processor -- start 2023-10-11T09:42:33.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.356+0000 7f66b0cfa700 1 -- start start 2023-10-11T09:42:33.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.356+0000 7f66b0cfa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66ac0b84c0 0x7f66ac0c6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:33.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.356+0000 7f66b0cfa700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66ac0b8890 0x7f66ac145470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:33.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.356+0000 7f66b0cfa700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f66ac0d3510 0x7f66ac0b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:33.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.356+0000 7f66b0cfa700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66ac1e56c0 con 0x7f66ac0b84c0 2023-10-11T09:42:33.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.356+0000 7f66b0cfa700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66ac1459b0 con 0x7f66ac0b8890 2023-10-11T09:42:33.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.356+0000 7f66b0cfa700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66ac0c9250 con 0x7f66ac0d3510 2023-10-11T09:42:33.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.356+0000 7f66aa59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66ac0b84c0 0x7f66ac0c6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:33.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.356+0000 7f66aa59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66ac0b84c0 0x7f66ac0c6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37318/0 (socket says 172.21.15.78:37318) 2023-10-11T09:42:33.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.356+0000 7f66aa59c700 1 -- 172.21.15.78:0/4100303963 learned_addr learned my addr 172.21.15.78:0/4100303963 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:33.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.356+0000 7f66a9d9b700 1 --2- 172.21.15.78:0/4100303963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66ac0b8890 0x7f66ac145470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:33.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.356+0000 7f66aad9d700 1 --2- 172.21.15.78:0/4100303963 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f66ac0d3510 0x7f66ac0b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:33.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.356+0000 7f66aa59c700 1 -- 172.21.15.78:0/4100303963 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f66ac0d3510 msgr2=0x7f66ac0b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:33.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.356+0000 7f66aa59c700 1 --2- 172.21.15.78:0/4100303963 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f66ac0d3510 0x7f66ac0b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:33.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.356+0000 7f66aa59c700 1 -- 172.21.15.78:0/4100303963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66ac0b8890 msgr2=0x7f66ac145470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:33.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.356+0000 7f66aa59c700 1 --2- 172.21.15.78:0/4100303963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66ac0b8890 0x7f66ac145470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:33.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.356+0000 7f66aa59c700 1 -- 172.21.15.78:0/4100303963 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a4009a70 con 0x7f66ac0b84c0 2023-10-11T09:42:33.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.356+0000 7f66aa59c700 1 --2- 172.21.15.78:0/4100303963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66ac0b84c0 0x7f66ac0c6b50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f669c00c700 tx=0x7f669c00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:33.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.360+0000 7f669affd700 1 -- 172.21.15.78:0/4100303963 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669c005d90 con 0x7f66ac0b84c0 2023-10-11T09:42:33.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.360+0000 7f669affd700 1 -- 172.21.15.78:0/4100303963 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f669c0070d0 con 0x7f66ac0b84c0 2023-10-11T09:42:33.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.360+0000 7f66b0cfa700 1 -- 172.21.15.78:0/4100303963 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66ac1428e0 con 0x7f66ac0b84c0 2023-10-11T09:42:33.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.360+0000 7f669affd700 1 -- 172.21.15.78:0/4100303963 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669c007480 con 0x7f66ac0b84c0 2023-10-11T09:42:33.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.360+0000 7f66b0cfa700 1 -- 172.21.15.78:0/4100303963 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66ac0c97b0 con 0x7f66ac0b84c0 2023-10-11T09:42:33.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.360+0000 7f66b0cfa700 1 -- 172.21.15.78:0/4100303963 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66ac0c6550 con 0x7f66ac0b84c0 2023-10-11T09:42:33.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.360+0000 7f669affd700 1 -- 172.21.15.78:0/4100303963 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f669c0075e0 con 0x7f66ac0b84c0 2023-10-11T09:42:33.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.364+0000 7f669affd700 1 --2- 172.21.15.78:0/4100303963 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f668c065ed0 0x7f668c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:33.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.364+0000 7f669affd700 1 -- 172.21.15.78:0/4100303963 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6495+0+0 (secure 0 0 0) 0x7f669c082ed0 con 0x7f66ac0b84c0 2023-10-11T09:42:33.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.364+0000 7f66a9d9b700 1 --2- 172.21.15.78:0/4100303963 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f668c065ed0 0x7f668c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:33.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.364+0000 7f66a9d9b700 1 --2- 172.21.15.78:0/4100303963 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f668c065ed0 0x7f668c068380 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f66ac0b81a0 tx=0x7f66a0009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:33.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.368+0000 7f669affd700 1 -- 172.21.15.78:0/4100303963 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f66ac0c6550 con 0x7f66ac0b84c0 2023-10-11T09:42:33.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.544+0000 7f66b0cfa700 1 -- 172.21.15.78:0/4100303963 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f66ac0c6550 con 0x7f66ac0b84c0 2023-10-11T09:42:33.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.544+0000 7f669affd700 1 -- 172.21.15.78:0/4100303963 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+11 (secure 0 0 0) 0x7f66ac0c6550 con 0x7f66ac0b84c0 2023-10-11T09:42:33.552 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2023-10-11T09:42:33.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.548+0000 7f66b0cfa700 1 -- 172.21.15.78:0/4100303963 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f668c065ed0 msgr2=0x7f668c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:33.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.548+0000 7f66b0cfa700 1 --2- 172.21.15.78:0/4100303963 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f668c065ed0 0x7f668c068380 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f66ac0b81a0 tx=0x7f66a0009240 comp rx=0 tx=0).stop 2023-10-11T09:42:33.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.548+0000 7f66b0cfa700 1 -- 172.21.15.78:0/4100303963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66ac0b84c0 msgr2=0x7f66ac0c6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:33.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.548+0000 7f66b0cfa700 1 --2- 172.21.15.78:0/4100303963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66ac0b84c0 0x7f66ac0c6b50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f669c00c700 tx=0x7f669c00ca10 comp rx=0 tx=0).stop 2023-10-11T09:42:33.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.548+0000 7f66b0cfa700 1 -- 172.21.15.78:0/4100303963 shutdown_connections 2023-10-11T09:42:33.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.548+0000 7f66b0cfa700 1 --2- 172.21.15.78:0/4100303963 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f668c065ed0 0x7f668c068380 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:33.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.548+0000 7f66b0cfa700 1 --2- 172.21.15.78:0/4100303963 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f66ac0d3510 0x7f66ac0b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:33.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.548+0000 7f66b0cfa700 1 --2- 172.21.15.78:0/4100303963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66ac0b8890 0x7f66ac145470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:33.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.548+0000 7f66b0cfa700 1 --2- 172.21.15.78:0/4100303963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66ac0b84c0 0x7f66ac0c6b50 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:33.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.548+0000 7f66b0cfa700 1 -- 172.21.15.78:0/4100303963 >> 172.21.15.78:0/4100303963 conn(0x7f66ac06cb50 msgr2=0x7f66ac15c590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:33.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.548+0000 7f66b0cfa700 1 -- 172.21.15.78:0/4100303963 shutdown_connections 2023-10-11T09:42:33.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.548+0000 7f66b0cfa700 1 -- 172.21.15.78:0/4100303963 wait complete. 2023-10-11T09:42:33.579 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-10-11T09:42:33.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.744+0000 7fbc51a4a700 1 Processor -- start 2023-10-11T09:42:33.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.744+0000 7fbc51a4a700 1 -- start start 2023-10-11T09:42:33.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.748+0000 7fbc51a4a700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbc4c1596b0 0x7fbc4c0c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:33.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.748+0000 7fbc51a4a700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fbc4c06cf70 0x7fbc4c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:33.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.748+0000 7fbc51a4a700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc4c0c9280 con 0x7fbc4c15db30 2023-10-11T09:42:33.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.748+0000 7fbc51a4a700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc4c06d930 con 0x7fbc4c06cf70 2023-10-11T09:42:33.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.748+0000 7fbc51a4a700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc4c0cd430 con 0x7fbc4c1596b0 2023-10-11T09:42:33.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.748+0000 7fbc43fff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fbc4c15db30 0x7fbc4c0ccd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:46370/0 (socket says 172.21.15.78:46370) 2023-10-11T09:42:33.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.748+0000 7fbc43fff700 1 -- 172.21.15.78:0/1734059526 learned_addr learned my addr 172.21.15.78:0/1734059526 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:42:33.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.748+0000 7fbc4b7fe700 1 --2- 172.21.15.78:0/1734059526 >> v2:172.21.15.78:3300/0 conn(0x7fbc4c06cf70 0x7fbc4c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:33.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.748+0000 7fbc4affd700 1 --2- 172.21.15.78:0/1734059526 >> v2:172.21.15.133:3300/0 conn(0x7fbc4c1596b0 0x7fbc4c0c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:33.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.748+0000 7fbc4a7fc700 1 -- 172.21.15.78:0/1734059526 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 481176359 0 0) 0x7fbc4c0c9280 con 0x7fbc4c15db30 2023-10-11T09:42:33.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.748+0000 7fbc4a7fc700 1 -- 172.21.15.78:0/1734059526 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc380035d0 con 0x7fbc4c15db30 2023-10-11T09:42:33.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.748+0000 7fbc4b7fe700 1 -- 172.21.15.78:0/1734059526 >> v2:172.21.15.133:3300/0 conn(0x7fbc4c1596b0 msgr2=0x7fbc4c0c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:33.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.748+0000 7fbc4b7fe700 1 --2- 172.21.15.78:0/1734059526 >> v2:172.21.15.133:3300/0 conn(0x7fbc4c1596b0 0x7fbc4c0c9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:33.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.748+0000 7fbc4b7fe700 1 -- 172.21.15.78:0/1734059526 >> v1:172.21.15.78:6789/0 conn(0x7fbc4c15db30 legacy=0x7fbc4c0ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:33.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.748+0000 7fbc4b7fe700 1 -- 172.21.15.78:0/1734059526 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc4c15df80 con 0x7fbc4c06cf70 2023-10-11T09:42:33.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.748+0000 7fbc4b7fe700 1 --2- 172.21.15.78:0/1734059526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc4c06cf70 0x7fbc4c06d3f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c06f910 tx=0x7fbc4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=699dd04149312a5e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:33.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.748+0000 7fbc4a7fc700 1 -- 172.21.15.78:0/1734059526 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4400e070 con 0x7fbc4c06cf70 2023-10-11T09:42:33.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.748+0000 7fbc4a7fc700 1 -- 172.21.15.78:0/1734059526 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4400bb20 con 0x7fbc4c06cf70 2023-10-11T09:42:33.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.748+0000 7fbc4a7fc700 1 -- 172.21.15.78:0/1734059526 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc44012430 con 0x7fbc4c06cf70 2023-10-11T09:42:33.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.748+0000 7fbc51a4a700 1 -- 172.21.15.78:0/1734059526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc4c06cf70 msgr2=0x7fbc4c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:33.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.748+0000 7fbc51a4a700 1 --2- 172.21.15.78:0/1734059526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc4c06cf70 0x7fbc4c06d3f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c06f910 tx=0x7fbc4400b330 comp rx=0 tx=0).stop 2023-10-11T09:42:33.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.752+0000 7fbc51a4a700 1 -- 172.21.15.78:0/1734059526 shutdown_connections 2023-10-11T09:42:33.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.752+0000 7fbc51a4a700 1 --2- 172.21.15.78:0/1734059526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc4c06cf70 0x7fbc4c06d3f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:33.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.752+0000 7fbc51a4a700 1 --2- 172.21.15.78:0/1734059526 >> v2:172.21.15.133:3300/0 conn(0x7fbc4c1596b0 0x7fbc4c0c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:33.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.752+0000 7fbc51a4a700 1 -- 172.21.15.78:0/1734059526 >> 172.21.15.78:0/1734059526 conn(0x7fbc4c0cecc0 msgr2=0x7fbc4c0cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:33.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.752+0000 7fbc51a4a700 1 -- 172.21.15.78:0/1734059526 shutdown_connections 2023-10-11T09:42:33.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.752+0000 7fbc51a4a700 1 -- 172.21.15.78:0/1734059526 wait complete. 2023-10-11T09:42:33.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.752+0000 7fbc51a4a700 1 Processor -- start 2023-10-11T09:42:33.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.752+0000 7fbc51a4a700 1 -- start start 2023-10-11T09:42:33.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.756+0000 7fbc51a4a700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbc4c06cf70 0x7fbc4c140e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:33.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.756+0000 7fbc51a4a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc4c1596b0 0x7fbc4c142020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:33.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.756+0000 7fbc51a4a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc4c15db30 0x7fbc4c1f5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:33.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.756+0000 7fbc51a4a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4c0af0b0 con 0x7fbc4c15db30 2023-10-11T09:42:33.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.756+0000 7fbc51a4a700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4c0b45b0 con 0x7fbc4c1596b0 2023-10-11T09:42:33.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.756+0000 7fbc51a4a700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4c0aea60 con 0x7fbc4c06cf70 2023-10-11T09:42:33.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.756+0000 7fbc4b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc4c15db30 0x7fbc4c1f5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:33.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.756+0000 7fbc4b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc4c15db30 0x7fbc4c1f5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37338/0 (socket says 172.21.15.78:37338) 2023-10-11T09:42:33.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.756+0000 7fbc4b7fe700 1 -- 172.21.15.78:0/2775715765 learned_addr learned my addr 172.21.15.78:0/2775715765 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:33.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.756+0000 7fbc43fff700 1 --2- 172.21.15.78:0/2775715765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc4c1596b0 0x7fbc4c142020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:33.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.756+0000 7fbc4affd700 1 --2- 172.21.15.78:0/2775715765 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbc4c06cf70 0x7fbc4c140e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:33.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.756+0000 7fbc4b7fe700 1 -- 172.21.15.78:0/2775715765 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbc4c06cf70 msgr2=0x7fbc4c140e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:33.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.756+0000 7fbc4b7fe700 1 --2- 172.21.15.78:0/2775715765 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbc4c06cf70 0x7fbc4c140e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:33.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.756+0000 7fbc4b7fe700 1 -- 172.21.15.78:0/2775715765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc4c1596b0 msgr2=0x7fbc4c142020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:33.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.756+0000 7fbc4b7fe700 1 --2- 172.21.15.78:0/2775715765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc4c1596b0 0x7fbc4c142020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:33.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.756+0000 7fbc4b7fe700 1 -- 172.21.15.78:0/2775715765 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc44009a70 con 0x7fbc4c15db30 2023-10-11T09:42:33.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.756+0000 7fbc4b7fe700 1 --2- 172.21.15.78:0/2775715765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc4c15db30 0x7fbc4c1f5790 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fbc44014040 tx=0x7fbc44004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:33.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.756+0000 7fbc48ff9700 1 -- 172.21.15.78:0/2775715765 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc44012430 con 0x7fbc4c15db30 2023-10-11T09:42:33.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.756+0000 7fbc48ff9700 1 -- 172.21.15.78:0/2775715765 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc44020070 con 0x7fbc4c15db30 2023-10-11T09:42:33.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.756+0000 7fbc51a4a700 1 -- 172.21.15.78:0/2775715765 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc4c0aed40 con 0x7fbc4c15db30 2023-10-11T09:42:33.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.756+0000 7fbc48ff9700 1 -- 172.21.15.78:0/2775715765 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc44003e10 con 0x7fbc4c15db30 2023-10-11T09:42:33.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.756+0000 7fbc51a4a700 1 -- 172.21.15.78:0/2775715765 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc4c142560 con 0x7fbc4c15db30 2023-10-11T09:42:33.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.760+0000 7fbc51a4a700 1 -- 172.21.15.78:0/2775715765 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc4c1427b0 con 0x7fbc4c15db30 2023-10-11T09:42:33.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.760+0000 7fbc48ff9700 1 -- 172.21.15.78:0/2775715765 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fbc44004040 con 0x7fbc4c15db30 2023-10-11T09:42:33.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.764+0000 7fbc48ff9700 1 --2- 172.21.15.78:0/2775715765 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbc2c065ed0 0x7fbc2c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:33.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.764+0000 7fbc48ff9700 1 -- 172.21.15.78:0/2775715765 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6495+0+0 (secure 0 0 0) 0x7fbc44022070 con 0x7fbc4c15db30 2023-10-11T09:42:33.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.764+0000 7fbc4affd700 1 --2- 172.21.15.78:0/2775715765 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbc2c065ed0 0x7fbc2c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:33.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.764+0000 7fbc4affd700 1 --2- 172.21.15.78:0/2775715765 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbc2c065ed0 0x7fbc2c068380 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c009bf0 tx=0x7fbc3c009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:33.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.768+0000 7fbc48ff9700 1 -- 172.21.15.78:0/2775715765 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fbc44052a40 con 0x7fbc4c15db30 2023-10-11T09:42:33.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.936+0000 7fbc51a4a700 1 -- 172.21.15.78:0/2775715765 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7fbc4c0af260 con 0x7fbc4c15db30 2023-10-11T09:42:33.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.936+0000 7fbc48ff9700 1 -- 172.21.15.78:0/2775715765 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v42) v1 ==== 106+0+11 (secure 0 0 0) 0x7fbc4402c020 con 0x7fbc4c15db30 2023-10-11T09:42:33.945 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:42:33.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.940+0000 7fbc51a4a700 1 -- 172.21.15.78:0/2775715765 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbc2c065ed0 msgr2=0x7fbc2c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:33.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.940+0000 7fbc51a4a700 1 --2- 172.21.15.78:0/2775715765 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbc2c065ed0 0x7fbc2c068380 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c009bf0 tx=0x7fbc3c009240 comp rx=0 tx=0).stop 2023-10-11T09:42:33.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.940+0000 7fbc51a4a700 1 -- 172.21.15.78:0/2775715765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc4c15db30 msgr2=0x7fbc4c1f5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:33.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.940+0000 7fbc51a4a700 1 --2- 172.21.15.78:0/2775715765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc4c15db30 0x7fbc4c1f5790 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fbc44014040 tx=0x7fbc44004af0 comp rx=0 tx=0).stop 2023-10-11T09:42:33.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.940+0000 7fbc51a4a700 1 -- 172.21.15.78:0/2775715765 shutdown_connections 2023-10-11T09:42:33.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.940+0000 7fbc51a4a700 1 --2- 172.21.15.78:0/2775715765 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbc2c065ed0 0x7fbc2c068380 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:33.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.940+0000 7fbc51a4a700 1 --2- 172.21.15.78:0/2775715765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc4c15db30 0x7fbc4c1f5790 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:33.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.940+0000 7fbc51a4a700 1 --2- 172.21.15.78:0/2775715765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc4c1596b0 0x7fbc4c142020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:33.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.940+0000 7fbc51a4a700 1 --2- 172.21.15.78:0/2775715765 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbc4c06cf70 0x7fbc4c140e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:33.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.940+0000 7fbc51a4a700 1 -- 172.21.15.78:0/2775715765 >> 172.21.15.78:0/2775715765 conn(0x7fbc4c0cecc0 msgr2=0x7fbc4c14cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:33.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.940+0000 7fbc51a4a700 1 -- 172.21.15.78:0/2775715765 shutdown_connections 2023-10-11T09:42:33.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:33.940+0000 7fbc51a4a700 1 -- 172.21.15.78:0/2775715765 wait complete. 2023-10-11T09:42:33.969 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T09:42:34.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.144+0000 7fd43c72c700 1 Processor -- start 2023-10-11T09:42:34.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.144+0000 7fd43c72c700 1 -- start start 2023-10-11T09:42:34.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.148+0000 7fd43c72c700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd4340b6c40 0x7fd4340b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:34.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.148+0000 7fd43c72c700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4340d0030 con 0x7fd43406e1c0 2023-10-11T09:42:34.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.148+0000 7fd43c72c700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4340d02d0 con 0x7fd43415a850 2023-10-11T09:42:34.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.148+0000 7fd43c72c700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4340d05b0 con 0x7fd4340b6c40 2023-10-11T09:42:34.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.148+0000 7fd43acc9700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fd43406e1c0 0x7fd4340ccb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:46382/0 (socket says 172.21.15.78:46382) 2023-10-11T09:42:34.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.148+0000 7fd43acc9700 1 -- 172.21.15.78:0/1957702273 learned_addr learned my addr 172.21.15.78:0/1957702273 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:42:34.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.148+0000 7fd439cc7700 1 --2- 172.21.15.78:0/1957702273 >> v2:172.21.15.133:3300/0 conn(0x7fd4340b6c40 0x7fd4340b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:34.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.148+0000 7fd4394c6700 1 -- 172.21.15.78:0/1957702273 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1614683633 0 0) 0x7fd4340d0030 con 0x7fd43406e1c0 2023-10-11T09:42:34.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.148+0000 7fd4394c6700 1 -- 172.21.15.78:0/1957702273 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4200035d0 con 0x7fd43406e1c0 2023-10-11T09:42:34.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.148+0000 7fd439cc7700 1 -- 172.21.15.78:0/1957702273 >> v1:172.21.15.133:6789/0 conn(0x7fd43415a850 legacy=0x7fd4340cf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:34.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.148+0000 7fd439cc7700 1 -- 172.21.15.78:0/1957702273 >> v1:172.21.15.78:6789/0 conn(0x7fd43406e1c0 legacy=0x7fd4340ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:34.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.148+0000 7fd439cc7700 1 -- 172.21.15.78:0/1957702273 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4340d0890 con 0x7fd4340b6c40 2023-10-11T09:42:34.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.152+0000 7fd439cc7700 1 --2- 172.21.15.78:0/1957702273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4340b6c40 0x7fd4340b3da0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd424009850 tx=0x7fd424009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e68c7d6488d9cf server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:34.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.152+0000 7fd4394c6700 1 -- 172.21.15.78:0/1957702273 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd424011830 con 0x7fd4340b6c40 2023-10-11T09:42:34.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.152+0000 7fd4394c6700 1 -- 172.21.15.78:0/1957702273 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd424011990 con 0x7fd4340b6c40 2023-10-11T09:42:34.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.152+0000 7fd4394c6700 1 -- 172.21.15.78:0/1957702273 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd424011d40 con 0x7fd4340b6c40 2023-10-11T09:42:34.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.152+0000 7fd43c72c700 1 -- 172.21.15.78:0/1957702273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4340b6c40 msgr2=0x7fd4340b3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:34.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.152+0000 7fd43c72c700 1 --2- 172.21.15.78:0/1957702273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4340b6c40 0x7fd4340b3da0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd424009850 tx=0x7fd424009b60 comp rx=0 tx=0).stop 2023-10-11T09:42:34.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.152+0000 7fd43c72c700 1 -- 172.21.15.78:0/1957702273 shutdown_connections 2023-10-11T09:42:34.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.152+0000 7fd43c72c700 1 --2- 172.21.15.78:0/1957702273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4340b6c40 0x7fd4340b3da0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:34.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.152+0000 7fd43c72c700 1 -- 172.21.15.78:0/1957702273 >> 172.21.15.78:0/1957702273 conn(0x7fd4340d0f70 msgr2=0x7fd4340d1370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:34.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.152+0000 7fd43c72c700 1 -- 172.21.15.78:0/1957702273 shutdown_connections 2023-10-11T09:42:34.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.152+0000 7fd43c72c700 1 -- 172.21.15.78:0/1957702273 wait complete. 2023-10-11T09:42:34.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.156+0000 7fd43c72c700 1 Processor -- start 2023-10-11T09:42:34.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.156+0000 7fd43c72c700 1 -- start start 2023-10-11T09:42:34.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.156+0000 7fd43c72c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd43406e1c0 0x7fd43414d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:34.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.156+0000 7fd43c72c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4340b6c40 0x7fd43406c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:34.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.156+0000 7fd43c72c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd43415a850 0x7fd43406c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:34.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.156+0000 7fd43c72c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43415ada0 con 0x7fd43406e1c0 2023-10-11T09:42:34.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.156+0000 7fd43c72c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4340b2e30 con 0x7fd43415a850 2023-10-11T09:42:34.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.156+0000 7fd43c72c700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4340b3110 con 0x7fd4340b6c40 2023-10-11T09:42:34.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.156+0000 7fd43a4c8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd43406e1c0 0x7fd43414d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:34.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.156+0000 7fd43a4c8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd43406e1c0 0x7fd43414d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37350/0 (socket says 172.21.15.78:37350) 2023-10-11T09:42:34.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.156+0000 7fd43a4c8700 1 -- 172.21.15.78:0/2616670101 learned_addr learned my addr 172.21.15.78:0/2616670101 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:34.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.156+0000 7fd439cc7700 1 --2- 172.21.15.78:0/2616670101 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4340b6c40 0x7fd43406c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:34.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.156+0000 7fd43acc9700 1 --2- 172.21.15.78:0/2616670101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd43415a850 0x7fd43406c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:34.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.156+0000 7fd43a4c8700 1 -- 172.21.15.78:0/2616670101 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4340b6c40 msgr2=0x7fd43406c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:34.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.156+0000 7fd43a4c8700 1 --2- 172.21.15.78:0/2616670101 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4340b6c40 0x7fd43406c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:34.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.156+0000 7fd43a4c8700 1 -- 172.21.15.78:0/2616670101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd43415a850 msgr2=0x7fd43406c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:34.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.156+0000 7fd43a4c8700 1 --2- 172.21.15.78:0/2616670101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd43415a850 0x7fd43406c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:34.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.156+0000 7fd43a4c8700 1 -- 172.21.15.78:0/2616670101 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4240094b0 con 0x7fd43406e1c0 2023-10-11T09:42:34.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.156+0000 7fd43acc9700 1 --2- 172.21.15.78:0/2616670101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd43415a850 0x7fd43406c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:42:34.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.156+0000 7fd43a4c8700 1 --2- 172.21.15.78:0/2616670101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd43406e1c0 0x7fd43414d6f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd43000c500 tx=0x7fd43000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:34.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.160+0000 7fd42affd700 1 -- 172.21.15.78:0/2616670101 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4300052d0 con 0x7fd43406e1c0 2023-10-11T09:42:34.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.160+0000 7fd42affd700 1 -- 172.21.15.78:0/2616670101 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd430005430 con 0x7fd43406e1c0 2023-10-11T09:42:34.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.160+0000 7fd43c72c700 1 -- 172.21.15.78:0/2616670101 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4340b33f0 con 0x7fd43406e1c0 2023-10-11T09:42:34.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.160+0000 7fd42affd700 1 -- 172.21.15.78:0/2616670101 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4300055a0 con 0x7fd43406e1c0 2023-10-11T09:42:34.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.160+0000 7fd43c72c700 1 -- 172.21.15.78:0/2616670101 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4340b2100 con 0x7fd43406e1c0 2023-10-11T09:42:34.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.160+0000 7fd43c72c700 1 -- 172.21.15.78:0/2616670101 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd43404e530 con 0x7fd43406e1c0 2023-10-11T09:42:34.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.164+0000 7fd42affd700 1 -- 172.21.15.78:0/2616670101 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fd4300038e0 con 0x7fd43406e1c0 2023-10-11T09:42:34.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.164+0000 7fd42affd700 1 --2- 172.21.15.78:0/2616670101 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd418065f00 0x7fd4180683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:34.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.164+0000 7fd42affd700 1 -- 172.21.15.78:0/2616670101 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6495+0+0 (secure 0 0 0) 0x7fd430082d80 con 0x7fd43406e1c0 2023-10-11T09:42:34.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.164+0000 7fd439cc7700 1 --2- 172.21.15.78:0/2616670101 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd418065f00 0x7fd4180683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:34.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.168+0000 7fd439cc7700 1 --2- 172.21.15.78:0/2616670101 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd418065f00 0x7fd4180683b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd4240090b0 tx=0x7fd424008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:34.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.168+0000 7fd42affd700 1 -- 172.21.15.78:0/2616670101 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fd430050210 con 0x7fd43406e1c0 2023-10-11T09:42:34.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.356+0000 7fd43c72c700 1 -- 172.21.15.78:0/2616670101 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd4340ce820 con 0x7fd43406e1c0 2023-10-11T09:42:34.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.356+0000 7fd42affd700 1 -- 172.21.15.78:0/2616670101 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v42) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd430017080 con 0x7fd43406e1c0 2023-10-11T09:42:34.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.360+0000 7fd43c72c700 1 -- 172.21.15.78:0/2616670101 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd418065f00 msgr2=0x7fd4180683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:34.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.360+0000 7fd43c72c700 1 --2- 172.21.15.78:0/2616670101 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd418065f00 0x7fd4180683b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd4240090b0 tx=0x7fd424008030 comp rx=0 tx=0).stop 2023-10-11T09:42:34.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.360+0000 7fd43c72c700 1 -- 172.21.15.78:0/2616670101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd43406e1c0 msgr2=0x7fd43414d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:34.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.360+0000 7fd43c72c700 1 --2- 172.21.15.78:0/2616670101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd43406e1c0 0x7fd43414d6f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd43000c500 tx=0x7fd43000c810 comp rx=0 tx=0).stop 2023-10-11T09:42:34.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.360+0000 7fd43c72c700 1 -- 172.21.15.78:0/2616670101 shutdown_connections 2023-10-11T09:42:34.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.360+0000 7fd43c72c700 1 --2- 172.21.15.78:0/2616670101 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd418065f00 0x7fd4180683b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:34.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.360+0000 7fd43c72c700 1 --2- 172.21.15.78:0/2616670101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd43415a850 0x7fd43406c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:34.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.360+0000 7fd43c72c700 1 --2- 172.21.15.78:0/2616670101 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4340b6c40 0x7fd43406c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:34.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.360+0000 7fd43c72c700 1 --2- 172.21.15.78:0/2616670101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd43406e1c0 0x7fd43414d6f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:34.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.360+0000 7fd43c72c700 1 -- 172.21.15.78:0/2616670101 >> 172.21.15.78:0/2616670101 conn(0x7fd4340d0f70 msgr2=0x7fd4340b4080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:34.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.360+0000 7fd43c72c700 1 -- 172.21.15.78:0/2616670101 shutdown_connections 2023-10-11T09:42:34.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.360+0000 7fd43c72c700 1 -- 172.21.15.78:0/2616670101 wait complete. 2023-10-11T09:42:34.370 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2023-10-11T09:42:34.388 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:42:34.388 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:42:34.407 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:42:34.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.620+0000 7f1e12a98700 1 Processor -- start 2023-10-11T09:42:34.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.620+0000 7f1e12a98700 1 -- start start 2023-10-11T09:42:34.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.620+0000 7f1e12a98700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e0c0cda10 0x7f1e0c0caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:34.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.620+0000 7f1e12a98700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f1e0c164d20 0x7f1e0c06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:34.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.620+0000 7f1e12a98700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e0c165390 con 0x7f1e0c0cdde0 2023-10-11T09:42:34.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.620+0000 7f1e12a98700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e0c1651a0 con 0x7f1e0c164d20 2023-10-11T09:42:34.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.620+0000 7f1e12a98700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c0ca440 con 0x7f1e0c0cda10 2023-10-11T09:42:34.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.620+0000 7f1e12297700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f1e0c164d20 0x7f1e0c06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:34.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.620+0000 7f1e12297700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f1e0c164d20 0x7f1e0c06bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35004/0 (socket says 172.21.15.78:35004) 2023-10-11T09:42:34.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.620+0000 7f1e12297700 1 -- 172.21.15.78:0/646177995 learned_addr learned my addr 172.21.15.78:0/646177995 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:34.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.624+0000 7f1e11a96700 1 --2- 172.21.15.78:0/646177995 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e0c0cda10 0x7f1e0c0caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:34.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.624+0000 7f1e12297700 1 -- 172.21.15.78:0/646177995 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e0c0cda10 msgr2=0x7f1e0c0caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:34.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.624+0000 7f1e12297700 1 --2- 172.21.15.78:0/646177995 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e0c0cda10 0x7f1e0c0caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:34.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.624+0000 7f1e12297700 1 -- 172.21.15.78:0/646177995 >> v1:172.21.15.133:6789/0 conn(0x7f1e0c0cdde0 legacy=0x7f1e0c164610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:34.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.624+0000 7f1e12297700 1 -- 172.21.15.78:0/646177995 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0c0ca720 con 0x7f1e0c164d20 2023-10-11T09:42:34.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.624+0000 7f1e11a96700 1 --2- 172.21.15.78:0/646177995 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e0c0cda10 0x7f1e0c0caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:42:34.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.624+0000 7f1e12297700 1 --2- 172.21.15.78:0/646177995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e0c164d20 0x7f1e0c06bbf0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1e04009910 tx=0x7f1e04009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e351d67f966efaaa server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:34.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.624+0000 7f1e10a94700 1 -- 172.21.15.78:0/646177995 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e04004230 con 0x7f1e0c164d20 2023-10-11T09:42:34.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.624+0000 7f1e10a94700 1 -- 172.21.15.78:0/646177995 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e04004390 con 0x7f1e0c164d20 2023-10-11T09:42:34.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.624+0000 7f1e10a94700 1 -- 172.21.15.78:0/646177995 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e04009e60 con 0x7f1e0c164d20 2023-10-11T09:42:34.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.624+0000 7f1e12a98700 1 -- 172.21.15.78:0/646177995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e0c164d20 msgr2=0x7f1e0c06bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:34.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.624+0000 7f1e12a98700 1 --2- 172.21.15.78:0/646177995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e0c164d20 0x7f1e0c06bbf0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1e04009910 tx=0x7f1e04009c20 comp rx=0 tx=0).stop 2023-10-11T09:42:34.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.624+0000 7f1e12a98700 1 -- 172.21.15.78:0/646177995 shutdown_connections 2023-10-11T09:42:34.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.624+0000 7f1e12a98700 1 --2- 172.21.15.78:0/646177995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e0c164d20 0x7f1e0c06bbf0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:34.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.624+0000 7f1e12a98700 1 --2- 172.21.15.78:0/646177995 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e0c0cda10 0x7f1e0c0caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:34.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.624+0000 7f1e12a98700 1 -- 172.21.15.78:0/646177995 >> 172.21.15.78:0/646177995 conn(0x7f1e0c0cea60 msgr2=0x7f1e0c0cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:34.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.624+0000 7f1e12a98700 1 -- 172.21.15.78:0/646177995 shutdown_connections 2023-10-11T09:42:34.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.624+0000 7f1e12a98700 1 -- 172.21.15.78:0/646177995 wait complete. 2023-10-11T09:42:34.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.628+0000 7f1e12a98700 1 Processor -- start 2023-10-11T09:42:34.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.628+0000 7f1e12a98700 1 -- start start 2023-10-11T09:42:34.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.632+0000 7f1e12a98700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e0c0cda10 0x7f1e0c144c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:34.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.632+0000 7f1e12a98700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e0c0cdde0 0x7f1e0c0afa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:34.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.632+0000 7f1e12a98700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e0c164d20 0x7f1e0c147e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:34.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.632+0000 7f1e12a98700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c0b26c0 con 0x7f1e0c0cda10 2023-10-11T09:42:34.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.632+0000 7f1e12a98700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c1451b0 con 0x7f1e0c164d20 2023-10-11T09:42:34.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.632+0000 7f1e12a98700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c0b1ab0 con 0x7f1e0c0cdde0 2023-10-11T09:42:34.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.632+0000 7f1e11a96700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e0c0cda10 0x7f1e0c144c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:34.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.632+0000 7f1e11a96700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e0c0cda10 0x7f1e0c144c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35010/0 (socket says 172.21.15.78:35010) 2023-10-11T09:42:34.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.632+0000 7f1e11a96700 1 -- 172.21.15.78:0/3331310536 learned_addr learned my addr 172.21.15.78:0/3331310536 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:34.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.632+0000 7f1e12297700 1 --2- 172.21.15.78:0/3331310536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e0c164d20 0x7f1e0c147e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:34.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.632+0000 7f1e11295700 1 --2- 172.21.15.78:0/3331310536 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e0c0cdde0 0x7f1e0c0afa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:34.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.632+0000 7f1e11a96700 1 -- 172.21.15.78:0/3331310536 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e0c0cdde0 msgr2=0x7f1e0c0afa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:34.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.632+0000 7f1e11a96700 1 --2- 172.21.15.78:0/3331310536 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e0c0cdde0 0x7f1e0c0afa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:34.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.632+0000 7f1e11a96700 1 -- 172.21.15.78:0/3331310536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e0c164d20 msgr2=0x7f1e0c147e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:34.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.632+0000 7f1e11a96700 1 --2- 172.21.15.78:0/3331310536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e0c164d20 0x7f1e0c147e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:34.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.632+0000 7f1e11a96700 1 -- 172.21.15.78:0/3331310536 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e04009570 con 0x7f1e0c0cda10 2023-10-11T09:42:34.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.632+0000 7f1e12297700 1 --2- 172.21.15.78:0/3331310536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e0c164d20 0x7f1e0c147e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:42:34.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.632+0000 7f1e11a96700 1 --2- 172.21.15.78:0/3331310536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e0c0cda10 0x7f1e0c144c70 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800ce00 tx=0x7f1e0800a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:34.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.632+0000 7f1e027fc700 1 -- 172.21.15.78:0/3331310536 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0800f480 con 0x7f1e0c0cda10 2023-10-11T09:42:34.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.632+0000 7f1e027fc700 1 -- 172.21.15.78:0/3331310536 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0800f5e0 con 0x7f1e0c0cda10 2023-10-11T09:42:34.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.632+0000 7f1e12a98700 1 -- 172.21.15.78:0/3331310536 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e0c0ce3b0 con 0x7f1e0c0cda10 2023-10-11T09:42:34.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.632+0000 7f1e027fc700 1 -- 172.21.15.78:0/3331310536 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0800f970 con 0x7f1e0c0cda10 2023-10-11T09:42:34.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.632+0000 7f1e12a98700 1 -- 172.21.15.78:0/3331310536 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e0c0b2070 con 0x7f1e0c0cda10 2023-10-11T09:42:34.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.632+0000 7f1e12a98700 1 -- 172.21.15.78:0/3331310536 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e0c0ce210 con 0x7f1e0c0cda10 2023-10-11T09:42:34.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.636+0000 7f1e027fc700 1 -- 172.21.15.78:0/3331310536 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f1e0800b040 con 0x7f1e0c0cda10 2023-10-11T09:42:34.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.636+0000 7f1e027fc700 1 --2- 172.21.15.78:0/3331310536 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1df0065ed0 0x7f1df0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:34.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.636+0000 7f1e027fc700 1 -- 172.21.15.78:0/3331310536 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6495+0+0 (secure 0 0 0) 0x7f1e08084310 con 0x7f1e0c0cda10 2023-10-11T09:42:34.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.636+0000 7f1e11295700 1 --2- 172.21.15.78:0/3331310536 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1df0065ed0 0x7f1df0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:34.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.640+0000 7f1e11295700 1 --2- 172.21.15.78:0/3331310536 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1df0065ed0 0x7f1df0068380 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc007fd0 tx=0x7f1dfc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:34.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.644+0000 7f1e027fc700 1 -- 172.21.15.78:0/3331310536 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f1e0c0ce210 con 0x7f1e0c0cda10 2023-10-11T09:42:34.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.820+0000 7f1e12a98700 1 -- 172.21.15.78:0/3331310536 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e0c0ce210 con 0x7f1e0c0cda10 2023-10-11T09:42:34.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.824+0000 7f1e027fc700 1 -- 172.21.15.78:0/3331310536 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+19720 (secure 0 0 0) 0x7f1e0c0ce210 con 0x7f1e0c0cda10 2023-10-11T09:42:34.831 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:34.831 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":42,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:42:22.478227+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:41:57.280079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-11T09:41:58.218860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data_extra","create_time":"2023-10-11T09:42:12.954795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.10","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:42:34.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.828+0000 7f1e12a98700 1 -- 172.21.15.78:0/3331310536 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1df0065ed0 msgr2=0x7f1df0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:34.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.828+0000 7f1e12a98700 1 --2- 172.21.15.78:0/3331310536 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1df0065ed0 0x7f1df0068380 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc007fd0 tx=0x7f1dfc009040 comp rx=0 tx=0).stop 2023-10-11T09:42:34.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.828+0000 7f1e12a98700 1 -- 172.21.15.78:0/3331310536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e0c0cda10 msgr2=0x7f1e0c144c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:34.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.828+0000 7f1e12a98700 1 --2- 172.21.15.78:0/3331310536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e0c0cda10 0x7f1e0c144c70 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800ce00 tx=0x7f1e0800a560 comp rx=0 tx=0).stop 2023-10-11T09:42:34.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.828+0000 7f1e12a98700 1 -- 172.21.15.78:0/3331310536 shutdown_connections 2023-10-11T09:42:34.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.828+0000 7f1e12a98700 1 --2- 172.21.15.78:0/3331310536 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1df0065ed0 0x7f1df0068380 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:34.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.828+0000 7f1e12a98700 1 --2- 172.21.15.78:0/3331310536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e0c164d20 0x7f1e0c147e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:34.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.828+0000 7f1e12a98700 1 --2- 172.21.15.78:0/3331310536 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e0c0cdde0 0x7f1e0c0afa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:34.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.828+0000 7f1e12a98700 1 --2- 172.21.15.78:0/3331310536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e0c0cda10 0x7f1e0c144c70 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:34.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.828+0000 7f1e12a98700 1 -- 172.21.15.78:0/3331310536 >> 172.21.15.78:0/3331310536 conn(0x7f1e0c0cea60 msgr2=0x7f1e0c06fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:34.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.828+0000 7f1e12a98700 1 -- 172.21.15.78:0/3331310536 shutdown_connections 2023-10-11T09:42:34.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:34.828+0000 7f1e12a98700 1 -- 172.21.15.78:0/3331310536 wait complete. 2023-10-11T09:42:34.854 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T09:41:57.280079+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T09:41:58.218860+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-10-11T09:42:12.954795+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:42:34.856 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:42:35.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.024+0000 7f2c81d2c700 1 Processor -- start 2023-10-11T09:42:35.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.024+0000 7f2c81d2c700 1 -- start start 2023-10-11T09:42:35.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.028+0000 7f2c81d2c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f2c7c158430 0x7f2c7c0710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:35.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.028+0000 7f2c81d2c700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2c7c06d340 0x7f2c7c0ca880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:35.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.028+0000 7f2c81d2c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2c7c0cadc0 0x7f2c7c15afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:35.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.028+0000 7f2c81d2c700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c7c070c10 con 0x7f2c7c158430 2023-10-11T09:42:35.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.028+0000 7f2c81d2c700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c7c0cb240 con 0x7f2c7c06d340 2023-10-11T09:42:35.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.028+0000 7f2c81d2c700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7c071610 con 0x7f2c7c0cadc0 2023-10-11T09:42:35.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.028+0000 7f2c7b7fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f2c7c158430 0x7f2c7c0710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:35.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.028+0000 7f2c7b7fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f2c7c158430 0x7f2c7c0710d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35022/0 (socket says 172.21.15.78:35022) 2023-10-11T09:42:35.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.028+0000 7f2c7b7fe700 1 -- 172.21.15.78:0/2426252887 learned_addr learned my addr 172.21.15.78:0/2426252887 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:35.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.028+0000 7f2c7bfff700 1 --2- 172.21.15.78:0/2426252887 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2c7c0cadc0 0x7f2c7c15afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:35.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.028+0000 7f2c72ffd700 1 --2- 172.21.15.78:0/2426252887 >> v2:172.21.15.133:3300/0 conn(0x7f2c7c06d340 0x7f2c7c0ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:35.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.028+0000 7f2c7b7fe700 1 -- 172.21.15.78:0/2426252887 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2c7c0cadc0 msgr2=0x7f2c7c15afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:35.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.028+0000 7f2c7b7fe700 1 --2- 172.21.15.78:0/2426252887 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2c7c0cadc0 0x7f2c7c15afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:35.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.028+0000 7f2c7b7fe700 1 -- 172.21.15.78:0/2426252887 >> v2:172.21.15.133:3300/0 conn(0x7f2c7c06d340 msgr2=0x7f2c7c0ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:35.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.028+0000 7f2c7b7fe700 1 --2- 172.21.15.78:0/2426252887 >> v2:172.21.15.133:3300/0 conn(0x7f2c7c06d340 0x7f2c7c0ca880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:35.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.028+0000 7f2c7b7fe700 1 -- 172.21.15.78:0/2426252887 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c7c15e280 con 0x7f2c7c158430 2023-10-11T09:42:35.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.032+0000 7f2c7b7fe700 1 --2- 172.21.15.78:0/2426252887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c7c158430 0x7f2c7c0710d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c06fbf0 tx=0x7f2c6c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3151ae240db9784b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:35.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.032+0000 7f2c7affd700 1 -- 172.21.15.78:0/2426252887 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c00b8a0 con 0x7f2c7c158430 2023-10-11T09:42:35.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.032+0000 7f2c7affd700 1 -- 172.21.15.78:0/2426252887 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c6c00ba00 con 0x7f2c7c158430 2023-10-11T09:42:35.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.032+0000 7f2c7affd700 1 -- 172.21.15.78:0/2426252887 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c00bd90 con 0x7f2c7c158430 2023-10-11T09:42:35.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.032+0000 7f2c81d2c700 1 -- 172.21.15.78:0/2426252887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c7c158430 msgr2=0x7f2c7c0710d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:35.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.032+0000 7f2c81d2c700 1 --2- 172.21.15.78:0/2426252887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c7c158430 0x7f2c7c0710d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c06fbf0 tx=0x7f2c6c009bf0 comp rx=0 tx=0).stop 2023-10-11T09:42:35.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.032+0000 7f2c81d2c700 1 -- 172.21.15.78:0/2426252887 shutdown_connections 2023-10-11T09:42:35.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.032+0000 7f2c81d2c700 1 --2- 172.21.15.78:0/2426252887 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2c7c0cadc0 0x7f2c7c15afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:35.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.032+0000 7f2c81d2c700 1 --2- 172.21.15.78:0/2426252887 >> v2:172.21.15.133:3300/0 conn(0x7f2c7c06d340 0x7f2c7c0ca880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:35.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.032+0000 7f2c81d2c700 1 --2- 172.21.15.78:0/2426252887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c7c158430 0x7f2c7c0710d0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:35.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.032+0000 7f2c81d2c700 1 -- 172.21.15.78:0/2426252887 >> 172.21.15.78:0/2426252887 conn(0x7f2c7c0cecc0 msgr2=0x7f2c7c0cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:35.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.032+0000 7f2c81d2c700 1 -- 172.21.15.78:0/2426252887 shutdown_connections 2023-10-11T09:42:35.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.032+0000 7f2c81d2c700 1 -- 172.21.15.78:0/2426252887 wait complete. 2023-10-11T09:42:35.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.036+0000 7f2c81d2c700 1 Processor -- start 2023-10-11T09:42:35.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.036+0000 7f2c81d2c700 1 -- start start 2023-10-11T09:42:35.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.036+0000 7f2c81d2c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c7c06d340 0x7f2c7c0aff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:35.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.036+0000 7f2c81d2c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c7c0cadc0 0x7f2c7c0cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:35.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.036+0000 7f2c81d2c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2c7c158430 0x7f2c7c0cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:35.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.036+0000 7f2c81d2c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7c0cd7d0 con 0x7f2c7c0cadc0 2023-10-11T09:42:35.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.036+0000 7f2c81d2c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7c1f6360 con 0x7f2c7c06d340 2023-10-11T09:42:35.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.036+0000 7f2c81d2c700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7c0ccb50 con 0x7f2c7c158430 2023-10-11T09:42:35.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.036+0000 7f2c72ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c7c0cadc0 0x7f2c7c0cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:35.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.036+0000 7f2c72ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c7c0cadc0 0x7f2c7c0cb9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35038/0 (socket says 172.21.15.78:35038) 2023-10-11T09:42:35.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.036+0000 7f2c72ffd700 1 -- 172.21.15.78:0/1830109882 learned_addr learned my addr 172.21.15.78:0/1830109882 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:35.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.036+0000 7f2c7b7fe700 1 --2- 172.21.15.78:0/1830109882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c7c06d340 0x7f2c7c0aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:35.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.036+0000 7f2c7bfff700 1 --2- 172.21.15.78:0/1830109882 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2c7c158430 0x7f2c7c0cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:35.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.036+0000 7f2c72ffd700 1 -- 172.21.15.78:0/1830109882 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2c7c158430 msgr2=0x7f2c7c0cbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:35.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.036+0000 7f2c72ffd700 1 --2- 172.21.15.78:0/1830109882 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2c7c158430 0x7f2c7c0cbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:35.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.036+0000 7f2c72ffd700 1 -- 172.21.15.78:0/1830109882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c7c06d340 msgr2=0x7f2c7c0aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:35.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.036+0000 7f2c72ffd700 1 --2- 172.21.15.78:0/1830109882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c7c06d340 0x7f2c7c0aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:35.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.036+0000 7f2c72ffd700 1 -- 172.21.15.78:0/1830109882 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c6c009570 con 0x7f2c7c0cadc0 2023-10-11T09:42:35.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.036+0000 7f2c7bfff700 1 --2- 172.21.15.78:0/1830109882 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2c7c158430 0x7f2c7c0cbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:42:35.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.036+0000 7f2c72ffd700 1 --2- 172.21.15.78:0/1830109882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c7c0cadc0 0x7f2c7c0cb9d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2c6400d800 tx=0x7f2c6400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:35.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.036+0000 7f2c78ff9700 1 -- 172.21.15.78:0/1830109882 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c64014620 con 0x7f2c7c0cadc0 2023-10-11T09:42:35.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.036+0000 7f2c78ff9700 1 -- 172.21.15.78:0/1830109882 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c64014780 con 0x7f2c7c0cadc0 2023-10-11T09:42:35.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.036+0000 7f2c78ff9700 1 -- 172.21.15.78:0/1830109882 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c64014b10 con 0x7f2c7c0cadc0 2023-10-11T09:42:35.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.036+0000 7f2c81d2c700 1 -- 172.21.15.78:0/1830109882 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c7c0cce30 con 0x7f2c7c0cadc0 2023-10-11T09:42:35.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.040+0000 7f2c81d2c700 1 -- 172.21.15.78:0/1830109882 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c7c0cd3f0 con 0x7f2c7c0cadc0 2023-10-11T09:42:35.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.040+0000 7f2c81d2c700 1 -- 172.21.15.78:0/1830109882 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c7c06eb80 con 0x7f2c7c0cadc0 2023-10-11T09:42:35.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.040+0000 7f2c78ff9700 1 -- 172.21.15.78:0/1830109882 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f2c64014c70 con 0x7f2c7c0cadc0 2023-10-11T09:42:35.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.040+0000 7f2c78ff9700 1 --2- 172.21.15.78:0/1830109882 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2c5c065ed0 0x7f2c5c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:35.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.040+0000 7f2c78ff9700 1 -- 172.21.15.78:0/1830109882 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6495+0+0 (secure 0 0 0) 0x7f2c64083e90 con 0x7f2c7c0cadc0 2023-10-11T09:42:35.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.044+0000 7f2c7b7fe700 1 --2- 172.21.15.78:0/1830109882 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2c5c065ed0 0x7f2c5c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:35.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.044+0000 7f2c7b7fe700 1 --2- 172.21.15.78:0/1830109882 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2c5c065ed0 0x7f2c5c068380 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c000c00 tx=0x7f2c6c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:35.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.048+0000 7f2c78ff9700 1 -- 172.21.15.78:0/1830109882 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f2c640513c0 con 0x7f2c7c0cadc0 2023-10-11T09:42:35.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.220+0000 7f2c81d2c700 1 -- 172.21.15.78:0/1830109882 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2c7c065db0 con 0x7f2c7c0cadc0 2023-10-11T09:42:35.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.224+0000 7f2c78ff9700 1 -- 172.21.15.78:0/1830109882 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2c640549e0 con 0x7f2c7c0cadc0 2023-10-11T09:42:35.230 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:42:35.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.224+0000 7f2c81d2c700 1 -- 172.21.15.78:0/1830109882 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2c5c065ed0 msgr2=0x7f2c5c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:35.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.224+0000 7f2c81d2c700 1 --2- 172.21.15.78:0/1830109882 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2c5c065ed0 0x7f2c5c068380 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c000c00 tx=0x7f2c6c008030 comp rx=0 tx=0).stop 2023-10-11T09:42:35.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.224+0000 7f2c81d2c700 1 -- 172.21.15.78:0/1830109882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c7c0cadc0 msgr2=0x7f2c7c0cb9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:35.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.224+0000 7f2c81d2c700 1 --2- 172.21.15.78:0/1830109882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c7c0cadc0 0x7f2c7c0cb9d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2c6400d800 tx=0x7f2c6400dbc0 comp rx=0 tx=0).stop 2023-10-11T09:42:35.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.224+0000 7f2c81d2c700 1 -- 172.21.15.78:0/1830109882 shutdown_connections 2023-10-11T09:42:35.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.224+0000 7f2c81d2c700 1 --2- 172.21.15.78:0/1830109882 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2c5c065ed0 0x7f2c5c068380 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:35.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.224+0000 7f2c81d2c700 1 --2- 172.21.15.78:0/1830109882 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2c7c158430 0x7f2c7c0cbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:35.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.224+0000 7f2c81d2c700 1 --2- 172.21.15.78:0/1830109882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c7c0cadc0 0x7f2c7c0cb9d0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:35.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.224+0000 7f2c81d2c700 1 --2- 172.21.15.78:0/1830109882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c7c06d340 0x7f2c7c0aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:35.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.224+0000 7f2c81d2c700 1 -- 172.21.15.78:0/1830109882 >> 172.21.15.78:0/1830109882 conn(0x7f2c7c0cecc0 msgr2=0x7f2c7c071810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:35.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.224+0000 7f2c81d2c700 1 -- 172.21.15.78:0/1830109882 shutdown_connections 2023-10-11T09:42:35.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.224+0000 7f2c81d2c700 1 -- 172.21.15.78:0/1830109882 wait complete. 2023-10-11T09:42:35.253 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:42:35.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.424+0000 7fa919de8700 1 Processor -- start 2023-10-11T09:42:35.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.424+0000 7fa919de8700 1 -- start start 2023-10-11T09:42:35.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.428+0000 7fa919de8700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa9140d29d0 0x7fa9140b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:35.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.428+0000 7fa919de8700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9140b47f0 con 0x7fa9140b3240 2023-10-11T09:42:35.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.428+0000 7fa919de8700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9140b8130 con 0x7fa9140b3610 2023-10-11T09:42:35.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.428+0000 7fa919de8700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9140b3fd0 con 0x7fa9140d29d0 2023-10-11T09:42:35.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.428+0000 7fa918de6700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fa9140b3240 0x7fa91415d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:45068/0 (socket says 172.21.15.78:45068) 2023-10-11T09:42:35.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.428+0000 7fa918de6700 1 -- 172.21.15.78:0/1533627705 learned_addr learned my addr 172.21.15.78:0/1533627705 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:42:35.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.428+0000 7fa9195e7700 1 --2- 172.21.15.78:0/1533627705 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa9140d29d0 0x7fa9140b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:35.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.428+0000 7fa9137fe700 1 -- 172.21.15.78:0/1533627705 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 14158682 0 0) 0x7fa908001110 con 0x7fa9140b3240 2023-10-11T09:42:35.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.428+0000 7fa9137fe700 1 -- 172.21.15.78:0/1533627705 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9040035d0 con 0x7fa9140b3240 2023-10-11T09:42:35.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.428+0000 7fa9137fe700 1 -- 172.21.15.78:0/1533627705 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3069305142 0 0) 0x7fa9140b8130 con 0x7fa9140b3610 2023-10-11T09:42:35.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.428+0000 7fa9137fe700 1 -- 172.21.15.78:0/1533627705 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa908001110 con 0x7fa9140b3610 2023-10-11T09:42:35.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.428+0000 7fa9137fe700 1 -- 172.21.15.78:0/1533627705 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1771068707 0 0) 0x7fa9040035d0 con 0x7fa9140b3240 2023-10-11T09:42:35.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.428+0000 7fa9137fe700 1 -- 172.21.15.78:0/1533627705 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa9140d29d0 msgr2=0x7fa9140b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:35.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.428+0000 7fa9137fe700 1 --2- 172.21.15.78:0/1533627705 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa9140d29d0 0x7fa9140b3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:35.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.428+0000 7fa9137fe700 1 -- 172.21.15.78:0/1533627705 >> v1:172.21.15.133:6789/0 conn(0x7fa9140b3610 legacy=0x7fa9140b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:35.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.428+0000 7fa9137fe700 1 -- 172.21.15.78:0/1533627705 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9140b41e0 con 0x7fa9140b3240 2023-10-11T09:42:35.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.428+0000 7fa9137fe700 1 -- 172.21.15.78:0/1533627705 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7fa908001800 con 0x7fa9140b3240 2023-10-11T09:42:35.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.428+0000 7fa9137fe700 1 -- 172.21.15.78:0/1533627705 >> v1:172.21.15.78:6789/0 conn(0x7fa9140b3240 legacy=0x7fa91415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:35.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.428+0000 7fa9137fe700 1 --2- 172.21.15.78:0/1533627705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa904003be0 0x7fa904003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:35.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.428+0000 7fa9137fe700 1 --2- 172.21.15.78:0/1533627705 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa9140d29d0 0x7fa904004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:35.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.428+0000 7fa9137fe700 1 --2- 172.21.15.78:0/1533627705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa904004db0 0x7fa904005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:35.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.428+0000 7fa9137fe700 1 -- 172.21.15.78:0/1533627705 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9140b3fd0 con 0x7fa904004db0 2023-10-11T09:42:35.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.428+0000 7fa9137fe700 1 -- 172.21.15.78:0/1533627705 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90400bb30 con 0x7fa904003be0 2023-10-11T09:42:35.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.428+0000 7fa9137fe700 1 -- 172.21.15.78:0/1533627705 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa90400bcf0 con 0x7fa9140d29d0 2023-10-11T09:42:35.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.428+0000 7fa9195e7700 1 --2- 172.21.15.78:0/1533627705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa904004db0 0x7fa904005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:35.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.428+0000 7fa918de6700 1 --2- 172.21.15.78:0/1533627705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa904003be0 0x7fa904003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:35.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.428+0000 7fa913fff700 1 --2- 172.21.15.78:0/1533627705 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa9140d29d0 0x7fa904004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:35.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.432+0000 7fa9195e7700 1 -- 172.21.15.78:0/1533627705 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa9140d29d0 msgr2=0x7fa904004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:35.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.432+0000 7fa9195e7700 1 --2- 172.21.15.78:0/1533627705 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa9140d29d0 0x7fa904004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:35.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.432+0000 7fa9195e7700 1 -- 172.21.15.78:0/1533627705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa904003be0 msgr2=0x7fa904003fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:35.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.432+0000 7fa9195e7700 1 --2- 172.21.15.78:0/1533627705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa904003be0 0x7fa904003fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:35.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.432+0000 7fa9195e7700 1 -- 172.21.15.78:0/1533627705 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9140b41e0 con 0x7fa904004db0 2023-10-11T09:42:35.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.432+0000 7fa9195e7700 1 --2- 172.21.15.78:0/1533627705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa904004db0 0x7fa904005220 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa90c009fc0 tx=0x7fa9040058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67be0e8fbe6556a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:35.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.432+0000 7fa9137fe700 1 -- 172.21.15.78:0/1533627705 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c00ec30 con 0x7fa904004db0 2023-10-11T09:42:35.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.432+0000 7fa9137fe700 1 -- 172.21.15.78:0/1533627705 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90c00f040 con 0x7fa904004db0 2023-10-11T09:42:35.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.432+0000 7fa9137fe700 1 -- 172.21.15.78:0/1533627705 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c014480 con 0x7fa904004db0 2023-10-11T09:42:35.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.432+0000 7fa919de8700 1 -- 172.21.15.78:0/1533627705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa904004db0 msgr2=0x7fa904005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:35.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.432+0000 7fa919de8700 1 --2- 172.21.15.78:0/1533627705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa904004db0 0x7fa904005220 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa90c009fc0 tx=0x7fa9040058a0 comp rx=0 tx=0).stop 2023-10-11T09:42:35.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.432+0000 7fa9195e7700 1 -- 172.21.15.78:0/1533627705 reap_dead start 2023-10-11T09:42:35.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.432+0000 7fa919de8700 1 -- 172.21.15.78:0/1533627705 shutdown_connections 2023-10-11T09:42:35.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.432+0000 7fa919de8700 1 -- 172.21.15.78:0/1533627705 >> 172.21.15.78:0/1533627705 conn(0x7fa91409c260 msgr2=0x7fa9140d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:35.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.432+0000 7fa919de8700 1 -- 172.21.15.78:0/1533627705 shutdown_connections 2023-10-11T09:42:35.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.432+0000 7fa919de8700 1 -- 172.21.15.78:0/1533627705 wait complete. 2023-10-11T09:42:35.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.436+0000 7fa919de8700 1 Processor -- start 2023-10-11T09:42:35.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.436+0000 7fa919de8700 1 -- start start 2023-10-11T09:42:35.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.436+0000 7fa919de8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa904004db0 0x7fa9141f3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:35.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.436+0000 7fa919de8700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa9140c7b10 0x7fa9141f2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:35.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.436+0000 7fa919de8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa91415f690 0x7fa9141e6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:35.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.436+0000 7fa919de8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9140c8b00 con 0x7fa904004db0 2023-10-11T09:42:35.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.436+0000 7fa919de8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9140c9940 con 0x7fa91415f690 2023-10-11T09:42:35.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.436+0000 7fa919de8700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9140c7f80 con 0x7fa9140c7b10 2023-10-11T09:42:35.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.436+0000 7fa918de6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa904004db0 0x7fa9141f3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:35.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.436+0000 7fa918de6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa904004db0 0x7fa9141f3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35054/0 (socket says 172.21.15.78:35054) 2023-10-11T09:42:35.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.436+0000 7fa918de6700 1 -- 172.21.15.78:0/3479152569 learned_addr learned my addr 172.21.15.78:0/3479152569 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:35.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.436+0000 7fa9195e7700 1 --2- 172.21.15.78:0/3479152569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa91415f690 0x7fa9141e6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:35.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.436+0000 7fa913fff700 1 --2- 172.21.15.78:0/3479152569 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa9140c7b10 0x7fa9141f2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:35.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.440+0000 7fa918de6700 1 -- 172.21.15.78:0/3479152569 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa9140c7b10 msgr2=0x7fa9141f2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:35.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.440+0000 7fa918de6700 1 --2- 172.21.15.78:0/3479152569 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa9140c7b10 0x7fa9141f2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:35.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.440+0000 7fa918de6700 1 -- 172.21.15.78:0/3479152569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa91415f690 msgr2=0x7fa9141e6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:35.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.440+0000 7fa918de6700 1 --2- 172.21.15.78:0/3479152569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa91415f690 0x7fa9141e6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:35.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.440+0000 7fa918de6700 1 -- 172.21.15.78:0/3479152569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9140c8260 con 0x7fa904004db0 2023-10-11T09:42:35.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.440+0000 7fa9195e7700 1 --2- 172.21.15.78:0/3479152569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa91415f690 0x7fa9141e6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:42:35.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.440+0000 7fa918de6700 1 --2- 172.21.15.78:0/3479152569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa904004db0 0x7fa9141f3660 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa908007420 tx=0x7fa908007730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:35.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.440+0000 7fa9117fa700 1 -- 172.21.15.78:0/3479152569 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90800f8d0 con 0x7fa904004db0 2023-10-11T09:42:35.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.440+0000 7fa9117fa700 1 -- 172.21.15.78:0/3479152569 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90800fa30 con 0x7fa904004db0 2023-10-11T09:42:35.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.440+0000 7fa919de8700 1 -- 172.21.15.78:0/3479152569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9140c8540 con 0x7fa904004db0 2023-10-11T09:42:35.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.440+0000 7fa9117fa700 1 -- 172.21.15.78:0/3479152569 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90800fde0 con 0x7fa904004db0 2023-10-11T09:42:35.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.440+0000 7fa919de8700 1 -- 172.21.15.78:0/3479152569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9141f3390 con 0x7fa904004db0 2023-10-11T09:42:35.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.440+0000 7fa919de8700 1 -- 172.21.15.78:0/3479152569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa914065d40 con 0x7fa904004db0 2023-10-11T09:42:35.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.444+0000 7fa9117fa700 1 -- 172.21.15.78:0/3479152569 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fa90801d360 con 0x7fa904004db0 2023-10-11T09:42:35.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.444+0000 7fa9117fa700 1 --2- 172.21.15.78:0/3479152569 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa8f8065ed0 0x7fa8f8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:35.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.444+0000 7fa9117fa700 1 -- 172.21.15.78:0/3479152569 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6495+0+0 (secure 0 0 0) 0x7fa9080830e0 con 0x7fa904004db0 2023-10-11T09:42:35.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.444+0000 7fa913fff700 1 --2- 172.21.15.78:0/3479152569 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa8f8065ed0 0x7fa8f8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:35.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.444+0000 7fa913fff700 1 --2- 172.21.15.78:0/3479152569 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa8f8065ed0 0x7fa8f8068380 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa9141f6b50 tx=0x7fa900009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:35.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.448+0000 7fa9117fa700 1 -- 172.21.15.78:0/3479152569 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fa908050570 con 0x7fa904004db0 2023-10-11T09:42:35.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.624+0000 7fa919de8700 1 -- 172.21.15.78:0/3479152569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa9140b47f0 con 0x7fa904004db0 2023-10-11T09:42:35.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.624+0000 7fa9117fa700 1 -- 172.21.15.78:0/3479152569 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v42) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa908053b90 con 0x7fa904004db0 2023-10-11T09:42:35.632 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:42:35.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.628+0000 7fa919de8700 1 -- 172.21.15.78:0/3479152569 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa8f8065ed0 msgr2=0x7fa8f8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:35.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.628+0000 7fa919de8700 1 --2- 172.21.15.78:0/3479152569 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa8f8065ed0 0x7fa8f8068380 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa9141f6b50 tx=0x7fa900009040 comp rx=0 tx=0).stop 2023-10-11T09:42:35.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.628+0000 7fa919de8700 1 -- 172.21.15.78:0/3479152569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa904004db0 msgr2=0x7fa9141f3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:35.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.628+0000 7fa919de8700 1 --2- 172.21.15.78:0/3479152569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa904004db0 0x7fa9141f3660 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa908007420 tx=0x7fa908007730 comp rx=0 tx=0).stop 2023-10-11T09:42:35.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.628+0000 7fa919de8700 1 -- 172.21.15.78:0/3479152569 shutdown_connections 2023-10-11T09:42:35.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.628+0000 7fa919de8700 1 --2- 172.21.15.78:0/3479152569 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa8f8065ed0 0x7fa8f8068380 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:35.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.628+0000 7fa919de8700 1 --2- 172.21.15.78:0/3479152569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa91415f690 0x7fa9141e6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:35.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.628+0000 7fa919de8700 1 --2- 172.21.15.78:0/3479152569 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa9140c7b10 0x7fa9141f2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:35.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.628+0000 7fa919de8700 1 --2- 172.21.15.78:0/3479152569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa904004db0 0x7fa9141f3660 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:35.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.628+0000 7fa919de8700 1 -- 172.21.15.78:0/3479152569 >> 172.21.15.78:0/3479152569 conn(0x7fa91409c260 msgr2=0x7fa9140b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:35.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.628+0000 7fa919de8700 1 -- 172.21.15.78:0/3479152569 shutdown_connections 2023-10-11T09:42:35.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.628+0000 7fa919de8700 1 -- 172.21.15.78:0/3479152569 wait complete. 2023-10-11T09:42:35.659 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T09:42:35.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.832+0000 7f530a645700 1 Processor -- start 2023-10-11T09:42:35.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.832+0000 7f530a645700 1 -- start start 2023-10-11T09:42:35.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.836+0000 7f530a645700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f530415a6b0 0x7f53040ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:35.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.836+0000 7f530a645700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f530406d730 0x7f530406bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:35.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.836+0000 7f530a645700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53040b2d30 con 0x7f53040b3a60 2023-10-11T09:42:35.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.836+0000 7f530a645700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53040cf360 con 0x7f530406d730 2023-10-11T09:42:35.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.836+0000 7f530a645700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53040b2f80 con 0x7f530415a6b0 2023-10-11T09:42:35.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.836+0000 7f5309e44700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f530406d730 0x7f530406bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:35.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.836+0000 7f5309e44700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f530406d730 0x7f530406bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35068/0 (socket says 172.21.15.78:35068) 2023-10-11T09:42:35.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.836+0000 7f5309e44700 1 -- 172.21.15.78:0/2587603306 learned_addr learned my addr 172.21.15.78:0/2587603306 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:35.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.836+0000 7f5309643700 1 --2- 172.21.15.78:0/2587603306 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f530415a6b0 0x7f53040ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:35.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.836+0000 7f5309e44700 1 -- 172.21.15.78:0/2587603306 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f530415a6b0 msgr2=0x7f53040ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:35.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.836+0000 7f5309e44700 1 --2- 172.21.15.78:0/2587603306 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f530415a6b0 0x7f53040ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:35.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.836+0000 7f5309e44700 1 -- 172.21.15.78:0/2587603306 >> v1:172.21.15.78:6789/0 conn(0x7f53040b3a60 legacy=0x7f530406d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:35.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.836+0000 7f5309e44700 1 -- 172.21.15.78:0/2587603306 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53040b3260 con 0x7f530406d730 2023-10-11T09:42:35.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.836+0000 7f5309e44700 1 --2- 172.21.15.78:0/2587603306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f530406d730 0x7f530406bbe0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f52fc009e50 tx=0x7f52fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=223ad6c300e883c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:35.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.836+0000 7f52fbfff700 1 -- 172.21.15.78:0/2587603306 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52fc00e070 con 0x7f530406d730 2023-10-11T09:42:35.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.836+0000 7f52fbfff700 1 -- 172.21.15.78:0/2587603306 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52fc00b9c0 con 0x7f530406d730 2023-10-11T09:42:35.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.836+0000 7f52fbfff700 1 -- 172.21.15.78:0/2587603306 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52fc00bd70 con 0x7f530406d730 2023-10-11T09:42:35.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.836+0000 7f530a645700 1 -- 172.21.15.78:0/2587603306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f530406d730 msgr2=0x7f530406bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:35.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.836+0000 7f530a645700 1 --2- 172.21.15.78:0/2587603306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f530406d730 0x7f530406bbe0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f52fc009e50 tx=0x7f52fc00b330 comp rx=0 tx=0).stop 2023-10-11T09:42:35.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.836+0000 7f530a645700 1 -- 172.21.15.78:0/2587603306 shutdown_connections 2023-10-11T09:42:35.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.836+0000 7f530a645700 1 --2- 172.21.15.78:0/2587603306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f530406d730 0x7f530406bbe0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:35.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.836+0000 7f530a645700 1 --2- 172.21.15.78:0/2587603306 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f530415a6b0 0x7f53040ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:35.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.836+0000 7f530a645700 1 -- 172.21.15.78:0/2587603306 >> 172.21.15.78:0/2587603306 conn(0x7f530406a060 msgr2=0x7f530406a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:35.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.836+0000 7f530a645700 1 -- 172.21.15.78:0/2587603306 shutdown_connections 2023-10-11T09:42:35.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.836+0000 7f530a645700 1 -- 172.21.15.78:0/2587603306 wait complete. 2023-10-11T09:42:35.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.840+0000 7f530a645700 1 Processor -- start 2023-10-11T09:42:35.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.840+0000 7f530a645700 1 -- start start 2023-10-11T09:42:35.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.844+0000 7f530a645700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f530406d730 0x7f530414bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:35.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.844+0000 7f530a645700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53040b3a60 0x7f53040711f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:35.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.844+0000 7f530a645700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f530415a6b0 0x7f530413f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:35.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.844+0000 7f530a645700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530413cff0 con 0x7f530406d730 2023-10-11T09:42:35.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.844+0000 7f530a645700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530414c2e0 con 0x7f53040b3a60 2023-10-11T09:42:35.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.844+0000 7f530a645700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5304070150 con 0x7f530415a6b0 2023-10-11T09:42:35.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.844+0000 7f5309643700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f530406d730 0x7f530414bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:35.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.844+0000 7f5309643700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f530406d730 0x7f530414bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35084/0 (socket says 172.21.15.78:35084) 2023-10-11T09:42:35.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.844+0000 7f5309643700 1 -- 172.21.15.78:0/2500935073 learned_addr learned my addr 172.21.15.78:0/2500935073 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:35.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.844+0000 7f5308e42700 1 --2- 172.21.15.78:0/2500935073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53040b3a60 0x7f53040711f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:35.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.844+0000 7f5309e44700 1 --2- 172.21.15.78:0/2500935073 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f530415a6b0 0x7f530413f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:35.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.844+0000 7f5309643700 1 -- 172.21.15.78:0/2500935073 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f530415a6b0 msgr2=0x7f530413f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:35.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.844+0000 7f5309643700 1 --2- 172.21.15.78:0/2500935073 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f530415a6b0 0x7f530413f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:35.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.844+0000 7f5309643700 1 -- 172.21.15.78:0/2500935073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53040b3a60 msgr2=0x7f53040711f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:35.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.844+0000 7f5309643700 1 --2- 172.21.15.78:0/2500935073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53040b3a60 0x7f53040711f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:35.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.844+0000 7f5309643700 1 -- 172.21.15.78:0/2500935073 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52fc009b30 con 0x7f530406d730 2023-10-11T09:42:35.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.844+0000 7f5309643700 1 --2- 172.21.15.78:0/2500935073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f530406d730 0x7f530414bda0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f530000d800 tx=0x7f530000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:35.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.844+0000 7f52f9ffb700 1 -- 172.21.15.78:0/2500935073 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53000125f0 con 0x7f530406d730 2023-10-11T09:42:35.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.844+0000 7f52f9ffb700 1 -- 172.21.15.78:0/2500935073 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5300012750 con 0x7f530406d730 2023-10-11T09:42:35.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.844+0000 7f530a645700 1 -- 172.21.15.78:0/2500935073 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5304070430 con 0x7f530406d730 2023-10-11T09:42:35.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.844+0000 7f52f9ffb700 1 -- 172.21.15.78:0/2500935073 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5300012ac0 con 0x7f530406d730 2023-10-11T09:42:35.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.844+0000 7f530a645700 1 -- 172.21.15.78:0/2500935073 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53040709f0 con 0x7f530406d730 2023-10-11T09:42:35.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.848+0000 7f530a645700 1 -- 172.21.15.78:0/2500935073 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5304065d40 con 0x7f530406d730 2023-10-11T09:42:35.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.848+0000 7f52f9ffb700 1 -- 172.21.15.78:0/2500935073 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f5300012c20 con 0x7f530406d730 2023-10-11T09:42:35.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.848+0000 7f52f9ffb700 1 --2- 172.21.15.78:0/2500935073 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f52e8065ed0 0x7f52e8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:35.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.852+0000 7f52f9ffb700 1 -- 172.21.15.78:0/2500935073 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6495+0+0 (secure 0 0 0) 0x7f53000839e0 con 0x7f530406d730 2023-10-11T09:42:35.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.852+0000 7f5308e42700 1 --2- 172.21.15.78:0/2500935073 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f52e8065ed0 0x7f52e8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:35.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.852+0000 7f5308e42700 1 --2- 172.21.15.78:0/2500935073 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f52e8065ed0 0x7f52e8068380 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5304157480 tx=0x7f52f4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:35.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:35.856+0000 7f52f9ffb700 1 -- 172.21.15.78:0/2500935073 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f5300050e80 con 0x7f530406d730 2023-10-11T09:42:36.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.036+0000 7f530a645700 1 -- 172.21.15.78:0/2500935073 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f53040cb6a0 con 0x7f530406d730 2023-10-11T09:42:36.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.036+0000 7f52f9ffb700 1 -- 172.21.15.78:0/2500935073 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+11 (secure 0 0 0) 0x7f53000544a0 con 0x7f530406d730 2023-10-11T09:42:36.044 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:42:36.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.040+0000 7f530a645700 1 -- 172.21.15.78:0/2500935073 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f52e8065ed0 msgr2=0x7f52e8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:36.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.040+0000 7f530a645700 1 --2- 172.21.15.78:0/2500935073 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f52e8065ed0 0x7f52e8068380 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5304157480 tx=0x7f52f4009040 comp rx=0 tx=0).stop 2023-10-11T09:42:36.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.040+0000 7f530a645700 1 -- 172.21.15.78:0/2500935073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f530406d730 msgr2=0x7f530414bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:36.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.040+0000 7f530a645700 1 --2- 172.21.15.78:0/2500935073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f530406d730 0x7f530414bda0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f530000d800 tx=0x7f530000db10 comp rx=0 tx=0).stop 2023-10-11T09:42:36.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.040+0000 7f530a645700 1 -- 172.21.15.78:0/2500935073 shutdown_connections 2023-10-11T09:42:36.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.040+0000 7f530a645700 1 --2- 172.21.15.78:0/2500935073 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f52e8065ed0 0x7f52e8068380 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:36.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.040+0000 7f530a645700 1 --2- 172.21.15.78:0/2500935073 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f530415a6b0 0x7f530413f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:36.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.040+0000 7f530a645700 1 --2- 172.21.15.78:0/2500935073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53040b3a60 0x7f53040711f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:36.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.040+0000 7f530a645700 1 --2- 172.21.15.78:0/2500935073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f530406d730 0x7f530414bda0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:36.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.040+0000 7f530a645700 1 -- 172.21.15.78:0/2500935073 >> 172.21.15.78:0/2500935073 conn(0x7f530406a060 msgr2=0x7f53040cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:36.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.040+0000 7f530a645700 1 -- 172.21.15.78:0/2500935073 shutdown_connections 2023-10-11T09:42:36.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.040+0000 7f530a645700 1 -- 172.21.15.78:0/2500935073 wait complete. 2023-10-11T09:42:36.069 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T09:42:36.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.240+0000 7fb2161a8700 1 Processor -- start 2023-10-11T09:42:36.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.240+0000 7fb2161a8700 1 -- start start 2023-10-11T09:42:36.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.244+0000 7fb2161a8700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb2100b3240 0x7fb2100d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:36.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.244+0000 7fb2161a8700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2100b3610 0x7fb2100b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:36.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.244+0000 7fb2161a8700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2100b47f0 con 0x7fb21015d040 2023-10-11T09:42:36.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.244+0000 7fb2161a8700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2100b3fd0 con 0x7fb2100b3240 2023-10-11T09:42:36.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.244+0000 7fb2161a8700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2100b8130 con 0x7fb2100b3610 2023-10-11T09:42:36.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.244+0000 7fb2151a6700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb2100b3240 0x7fb2100d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:36.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.244+0000 7fb2149a5700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2100b3610 0x7fb2100b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:36.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.244+0000 7fb2149a5700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2100b3610 0x7fb2100b3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:56592/0 (socket says 172.21.15.78:56592) 2023-10-11T09:42:36.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.244+0000 7fb2149a5700 1 -- 172.21.15.78:0/2899762321 learned_addr learned my addr 172.21.15.78:0/2899762321 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:36.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.244+0000 7fb2159a7700 1 --1- 172.21.15.78:0/2899762321 >> v1:172.21.15.133:6789/0 conn(0x7fb21015d040 0x7fb2100b85c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:48842/0 (socket says 172.21.15.78:48842) 2023-10-11T09:42:36.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.244+0000 7fb2149a5700 1 -- 172.21.15.78:0/2899762321 >> v2:172.21.15.133:3300/0 conn(0x7fb2100b3240 msgr2=0x7fb2100d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:36.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.244+0000 7fb2149a5700 1 --2- 172.21.15.78:0/2899762321 >> v2:172.21.15.133:3300/0 conn(0x7fb2100b3240 0x7fb2100d29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:36.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.244+0000 7fb2149a5700 1 -- 172.21.15.78:0/2899762321 >> v1:172.21.15.133:6789/0 conn(0x7fb21015d040 legacy=0x7fb2100b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:36.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.244+0000 7fb2149a5700 1 -- 172.21.15.78:0/2899762321 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2100b41e0 con 0x7fb2100b3610 2023-10-11T09:42:36.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.244+0000 7fb2149a5700 1 --2- 172.21.15.78:0/2899762321 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2100b3610 0x7fb2100b3a90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb200009910 tx=0x7fb200009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=982f9b885a5c369b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:36.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.244+0000 7fb207fff700 1 -- 172.21.15.78:0/2899762321 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb200011830 con 0x7fb2100b3610 2023-10-11T09:42:36.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.244+0000 7fb207fff700 1 -- 172.21.15.78:0/2899762321 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb200011990 con 0x7fb2100b3610 2023-10-11T09:42:36.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.244+0000 7fb207fff700 1 -- 172.21.15.78:0/2899762321 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb200011d40 con 0x7fb2100b3610 2023-10-11T09:42:36.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.248+0000 7fb2161a8700 1 -- 172.21.15.78:0/2899762321 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2100b3610 msgr2=0x7fb2100b3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:36.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.248+0000 7fb2161a8700 1 --2- 172.21.15.78:0/2899762321 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2100b3610 0x7fb2100b3a90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb200009910 tx=0x7fb200009c20 comp rx=0 tx=0).stop 2023-10-11T09:42:36.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.248+0000 7fb2161a8700 1 -- 172.21.15.78:0/2899762321 shutdown_connections 2023-10-11T09:42:36.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.248+0000 7fb2161a8700 1 --2- 172.21.15.78:0/2899762321 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2100b3610 0x7fb2100b3a90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:36.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.248+0000 7fb2161a8700 1 --2- 172.21.15.78:0/2899762321 >> v2:172.21.15.133:3300/0 conn(0x7fb2100b3240 0x7fb2100d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:36.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.248+0000 7fb2161a8700 1 -- 172.21.15.78:0/2899762321 >> 172.21.15.78:0/2899762321 conn(0x7fb21009c260 msgr2=0x7fb2100d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:36.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.248+0000 7fb2161a8700 1 -- 172.21.15.78:0/2899762321 shutdown_connections 2023-10-11T09:42:36.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.248+0000 7fb2161a8700 1 -- 172.21.15.78:0/2899762321 wait complete. 2023-10-11T09:42:36.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.252+0000 7fb2161a8700 1 Processor -- start 2023-10-11T09:42:36.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.252+0000 7fb2161a8700 1 -- start start 2023-10-11T09:42:36.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.252+0000 7fb2161a8700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2100b3240 0x7fb210159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:36.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.252+0000 7fb2161a8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2100b3610 0x7fb210158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:36.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.252+0000 7fb2161a8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb21015d040 0x7fb2100b1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:36.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.252+0000 7fb2161a8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2101f2520 con 0x7fb2100b3610 2023-10-11T09:42:36.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.252+0000 7fb2161a8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2101f27e0 con 0x7fb21015d040 2023-10-11T09:42:36.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.252+0000 7fb2161a8700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2101f2ef0 con 0x7fb2100b3240 2023-10-11T09:42:36.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.252+0000 7fb2149a5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2100b3610 0x7fb210158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:36.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.252+0000 7fb2149a5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2100b3610 0x7fb210158fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35088/0 (socket says 172.21.15.78:35088) 2023-10-11T09:42:36.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.252+0000 7fb2149a5700 1 -- 172.21.15.78:0/1773394310 learned_addr learned my addr 172.21.15.78:0/1773394310 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:36.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.252+0000 7fb2151a6700 1 --2- 172.21.15.78:0/1773394310 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2100b3240 0x7fb210159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:36.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.252+0000 7fb2159a7700 1 --2- 172.21.15.78:0/1773394310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb21015d040 0x7fb2100b1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:36.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.252+0000 7fb2149a5700 1 -- 172.21.15.78:0/1773394310 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2100b3240 msgr2=0x7fb210159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:36.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.252+0000 7fb2149a5700 1 --2- 172.21.15.78:0/1773394310 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2100b3240 0x7fb210159640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:36.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.252+0000 7fb2149a5700 1 -- 172.21.15.78:0/1773394310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb21015d040 msgr2=0x7fb2100b1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:36.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.252+0000 7fb2149a5700 1 --2- 172.21.15.78:0/1773394310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb21015d040 0x7fb2100b1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:36.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.252+0000 7fb2149a5700 1 -- 172.21.15.78:0/1773394310 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb200009570 con 0x7fb2100b3610 2023-10-11T09:42:36.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.256+0000 7fb2149a5700 1 --2- 172.21.15.78:0/1773394310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2100b3610 0x7fb210158fb0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb200013000 tx=0x7fb200004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:36.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.256+0000 7fb205ffb700 1 -- 172.21.15.78:0/1773394310 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb200004510 con 0x7fb2100b3610 2023-10-11T09:42:36.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.256+0000 7fb205ffb700 1 -- 172.21.15.78:0/1773394310 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb20001e070 con 0x7fb2100b3610 2023-10-11T09:42:36.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.256+0000 7fb2161a8700 1 -- 172.21.15.78:0/1773394310 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2101f31d0 con 0x7fb2100b3610 2023-10-11T09:42:36.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.256+0000 7fb205ffb700 1 -- 172.21.15.78:0/1773394310 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb200003900 con 0x7fb2100b3610 2023-10-11T09:42:36.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.256+0000 7fb2161a8700 1 -- 172.21.15.78:0/1773394310 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2100d3830 con 0x7fb2100b3610 2023-10-11T09:42:36.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.256+0000 7fb2161a8700 1 -- 172.21.15.78:0/1773394310 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2100d09b0 con 0x7fb2100b3610 2023-10-11T09:42:36.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.260+0000 7fb205ffb700 1 -- 172.21.15.78:0/1773394310 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fb200003a60 con 0x7fb2100b3610 2023-10-11T09:42:36.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.260+0000 7fb205ffb700 1 --2- 172.21.15.78:0/1773394310 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb1f4065ed0 0x7fb1f4068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:36.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.260+0000 7fb205ffb700 1 -- 172.21.15.78:0/1773394310 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6495+0+0 (secure 0 0 0) 0x7fb2000844a0 con 0x7fb2100b3610 2023-10-11T09:42:36.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.260+0000 7fb2151a6700 1 --2- 172.21.15.78:0/1773394310 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb1f4065ed0 0x7fb1f4068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:36.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.260+0000 7fb2151a6700 1 --2- 172.21.15.78:0/1773394310 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb1f4065ed0 0x7fb1f4068380 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb20c009b50 tx=0x7fb20c009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:36.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.268+0000 7fb205ffb700 1 -- 172.21.15.78:0/1773394310 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fb2000519d0 con 0x7fb2100b3610 2023-10-11T09:42:36.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.448+0000 7fb2161a8700 1 -- 172.21.15.78:0/1773394310 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb2100b7d90 con 0x7fb2100b3610 2023-10-11T09:42:36.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.448+0000 7fb205ffb700 1 -- 172.21.15.78:0/1773394310 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb200020020 con 0x7fb2100b3610 2023-10-11T09:42:36.455 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2023-10-11T09:42:36.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.452+0000 7fb2161a8700 1 -- 172.21.15.78:0/1773394310 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb1f4065ed0 msgr2=0x7fb1f4068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:36.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.452+0000 7fb2161a8700 1 --2- 172.21.15.78:0/1773394310 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb1f4065ed0 0x7fb1f4068380 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb20c009b50 tx=0x7fb20c009240 comp rx=0 tx=0).stop 2023-10-11T09:42:36.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.452+0000 7fb2161a8700 1 -- 172.21.15.78:0/1773394310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2100b3610 msgr2=0x7fb210158fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:36.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.452+0000 7fb2161a8700 1 --2- 172.21.15.78:0/1773394310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2100b3610 0x7fb210158fb0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb200013000 tx=0x7fb200004770 comp rx=0 tx=0).stop 2023-10-11T09:42:36.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.452+0000 7fb2161a8700 1 -- 172.21.15.78:0/1773394310 shutdown_connections 2023-10-11T09:42:36.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.452+0000 7fb2161a8700 1 --2- 172.21.15.78:0/1773394310 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb1f4065ed0 0x7fb1f4068380 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:36.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.452+0000 7fb2161a8700 1 --2- 172.21.15.78:0/1773394310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb21015d040 0x7fb2100b1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:36.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.452+0000 7fb2161a8700 1 --2- 172.21.15.78:0/1773394310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2100b3610 0x7fb210158fb0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:36.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.452+0000 7fb2161a8700 1 --2- 172.21.15.78:0/1773394310 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2100b3240 0x7fb210159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:36.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.452+0000 7fb2161a8700 1 -- 172.21.15.78:0/1773394310 >> 172.21.15.78:0/1773394310 conn(0x7fb21009c260 msgr2=0x7fb21015faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:36.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.452+0000 7fb2161a8700 1 -- 172.21.15.78:0/1773394310 shutdown_connections 2023-10-11T09:42:36.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.452+0000 7fb2161a8700 1 -- 172.21.15.78:0/1773394310 wait complete. 2023-10-11T09:42:36.480 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-10-11T09:42:36.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.652+0000 7f6ede866700 1 Processor -- start 2023-10-11T09:42:36.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.652+0000 7f6ede866700 1 -- start start 2023-10-11T09:42:36.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.652+0000 7f6ede866700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6ed8161d90 0x7f6ed80b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:36.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.652+0000 7f6ede866700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6ed80b3520 0x7f6ed81629e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:36.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.652+0000 7f6ede866700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ed806ccb0 con 0x7f6ed8146fe0 2023-10-11T09:42:36.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.652+0000 7f6ede866700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ed806c410 con 0x7f6ed8161d90 2023-10-11T09:42:36.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.652+0000 7f6ede866700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed806c6f0 con 0x7f6ed80b3520 2023-10-11T09:42:36.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.656+0000 7f6edd864700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f6ed8146fe0 0x7f6ed806bbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:45086/0 (socket says 172.21.15.78:45086) 2023-10-11T09:42:36.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.656+0000 7f6edd864700 1 -- 172.21.15.78:0/1060131095 learned_addr learned my addr 172.21.15.78:0/1060131095 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:42:36.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.656+0000 7f6ede065700 1 --2- 172.21.15.78:0/1060131095 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6ed80b3520 0x7f6ed81629e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:36.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.656+0000 7f6edd063700 1 --2- 172.21.15.78:0/1060131095 >> v2:172.21.15.133:3300/0 conn(0x7f6ed8161d90 0x7f6ed80b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:36.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.656+0000 7f6edc862700 1 -- 172.21.15.78:0/1060131095 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2522745288 0 0) 0x7f6ed4001110 con 0x7f6ed8146fe0 2023-10-11T09:42:36.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.656+0000 7f6edc862700 1 -- 172.21.15.78:0/1060131095 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ec40035d0 con 0x7f6ed8146fe0 2023-10-11T09:42:36.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.656+0000 7f6ede065700 1 -- 172.21.15.78:0/1060131095 >> v2:172.21.15.133:3300/0 conn(0x7f6ed8161d90 msgr2=0x7f6ed80b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:36.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.656+0000 7f6ede065700 1 --2- 172.21.15.78:0/1060131095 >> v2:172.21.15.133:3300/0 conn(0x7f6ed8161d90 0x7f6ed80b2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:36.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.656+0000 7f6ede065700 1 -- 172.21.15.78:0/1060131095 >> v1:172.21.15.78:6789/0 conn(0x7f6ed8146fe0 legacy=0x7f6ed806bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:36.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.656+0000 7f6ede065700 1 -- 172.21.15.78:0/1060131095 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed806c9d0 con 0x7f6ed80b3520 2023-10-11T09:42:36.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.656+0000 7f6edd063700 1 --2- 172.21.15.78:0/1060131095 >> v2:172.21.15.133:3300/0 conn(0x7f6ed8161d90 0x7f6ed80b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:42:36.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.656+0000 7f6ede065700 1 --2- 172.21.15.78:0/1060131095 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6ed80b3520 0x7f6ed81629e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6ed0009d90 tx=0x7f6ed000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84c7968983bc33fd server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:36.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.656+0000 7f6edc862700 1 -- 172.21.15.78:0/1060131095 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed000e070 con 0x7f6ed80b3520 2023-10-11T09:42:36.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.656+0000 7f6edc862700 1 -- 172.21.15.78:0/1060131095 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed000bb20 con 0x7f6ed80b3520 2023-10-11T09:42:36.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.656+0000 7f6edc862700 1 -- 172.21.15.78:0/1060131095 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed0012430 con 0x7f6ed80b3520 2023-10-11T09:42:36.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.660+0000 7f6ede866700 1 -- 172.21.15.78:0/1060131095 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6ed80b3520 msgr2=0x7f6ed81629e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:36.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.660+0000 7f6ede866700 1 --2- 172.21.15.78:0/1060131095 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6ed80b3520 0x7f6ed81629e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6ed0009d90 tx=0x7f6ed000b330 comp rx=0 tx=0).stop 2023-10-11T09:42:36.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.660+0000 7f6ede866700 1 -- 172.21.15.78:0/1060131095 shutdown_connections 2023-10-11T09:42:36.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.660+0000 7f6ede866700 1 --2- 172.21.15.78:0/1060131095 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6ed80b3520 0x7f6ed81629e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:36.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.660+0000 7f6ede866700 1 --2- 172.21.15.78:0/1060131095 >> v2:172.21.15.133:3300/0 conn(0x7f6ed8161d90 0x7f6ed80b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:36.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.660+0000 7f6ede866700 1 -- 172.21.15.78:0/1060131095 >> 172.21.15.78:0/1060131095 conn(0x7f6ed806a060 msgr2=0x7f6ed806a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:36.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.660+0000 7f6ede866700 1 -- 172.21.15.78:0/1060131095 shutdown_connections 2023-10-11T09:42:36.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.660+0000 7f6ede866700 1 -- 172.21.15.78:0/1060131095 wait complete. 2023-10-11T09:42:36.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.660+0000 7f6ede866700 1 Processor -- start 2023-10-11T09:42:36.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.660+0000 7f6ede866700 1 -- start start 2023-10-11T09:42:36.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.664+0000 7f6ede866700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ed80b3520 0x7f6ed8145fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:36.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.664+0000 7f6ede866700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ed8146fe0 0x7f6ed80cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:36.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.664+0000 7f6ede866700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6ed8161d90 0x7f6ed80cd730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:36.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.664+0000 7f6ede866700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed80ce970 con 0x7f6ed80b3520 2023-10-11T09:42:36.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.664+0000 7f6ede866700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed80cdd60 con 0x7f6ed8146fe0 2023-10-11T09:42:36.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.664+0000 7f6ede866700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed80ce040 con 0x7f6ed8161d90 2023-10-11T09:42:36.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.664+0000 7f6edd864700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ed80b3520 0x7f6ed8145fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:36.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.664+0000 7f6edd864700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ed80b3520 0x7f6ed8145fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35098/0 (socket says 172.21.15.78:35098) 2023-10-11T09:42:36.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.664+0000 7f6edd864700 1 -- 172.21.15.78:0/108222789 learned_addr learned my addr 172.21.15.78:0/108222789 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:36.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.664+0000 7f6ede065700 1 --2- 172.21.15.78:0/108222789 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6ed8161d90 0x7f6ed80cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:36.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.664+0000 7f6edd063700 1 --2- 172.21.15.78:0/108222789 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ed8146fe0 0x7f6ed80cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:36.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.664+0000 7f6edd864700 1 -- 172.21.15.78:0/108222789 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6ed8161d90 msgr2=0x7f6ed80cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:36.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.664+0000 7f6edd864700 1 --2- 172.21.15.78:0/108222789 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6ed8161d90 0x7f6ed80cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:36.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.664+0000 7f6edd864700 1 -- 172.21.15.78:0/108222789 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ed8146fe0 msgr2=0x7f6ed80cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:36.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.664+0000 7f6edd864700 1 --2- 172.21.15.78:0/108222789 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ed8146fe0 0x7f6ed80cd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:36.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.664+0000 7f6edd864700 1 -- 172.21.15.78:0/108222789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec80094b0 con 0x7f6ed80b3520 2023-10-11T09:42:36.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.664+0000 7f6edd864700 1 --2- 172.21.15.78:0/108222789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ed80b3520 0x7f6ed8145fd0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4001270 tx=0x7f6ed400c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:36.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.664+0000 7f6ece7fc700 1 -- 172.21.15.78:0/108222789 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed4005d90 con 0x7f6ed80b3520 2023-10-11T09:42:36.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.664+0000 7f6ece7fc700 1 -- 172.21.15.78:0/108222789 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed40070d0 con 0x7f6ed80b3520 2023-10-11T09:42:36.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.664+0000 7f6ede866700 1 -- 172.21.15.78:0/108222789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ed0009a70 con 0x7f6ed80b3520 2023-10-11T09:42:36.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.664+0000 7f6ece7fc700 1 -- 172.21.15.78:0/108222789 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed4007480 con 0x7f6ed80b3520 2023-10-11T09:42:36.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.664+0000 7f6ede866700 1 -- 172.21.15.78:0/108222789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ed80ce600 con 0x7f6ed80b3520 2023-10-11T09:42:36.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.668+0000 7f6ede866700 1 -- 172.21.15.78:0/108222789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ed8065d40 con 0x7f6ed80b3520 2023-10-11T09:42:36.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.668+0000 7f6ece7fc700 1 -- 172.21.15.78:0/108222789 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f6ed40075e0 con 0x7f6ed80b3520 2023-10-11T09:42:36.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.668+0000 7f6ece7fc700 1 --2- 172.21.15.78:0/108222789 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6ebc065ed0 0x7f6ebc068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:36.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.672+0000 7f6ece7fc700 1 -- 172.21.15.78:0/108222789 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6495+0+0 (secure 0 0 0) 0x7f6ed4082cd0 con 0x7f6ed80b3520 2023-10-11T09:42:36.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.672+0000 7f6edd063700 1 --2- 172.21.15.78:0/108222789 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6ebc065ed0 0x7f6ebc068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:36.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.672+0000 7f6edd063700 1 --2- 172.21.15.78:0/108222789 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6ebc065ed0 0x7f6ebc068380 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6ed81602e0 tx=0x7f6ec8009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:36.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.676+0000 7f6ece7fc700 1 -- 172.21.15.78:0/108222789 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f6ed40500d0 con 0x7f6ed80b3520 2023-10-11T09:42:36.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.848+0000 7f6ede866700 1 -- 172.21.15.78:0/108222789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7f6ed806b570 con 0x7f6ed80b3520 2023-10-11T09:42:36.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.848+0000 7f6ece7fc700 1 -- 172.21.15.78:0/108222789 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v42) v1 ==== 106+0+11 (secure 0 0 0) 0x7f6ed40170b0 con 0x7f6ed80b3520 2023-10-11T09:42:36.857 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:42:36.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.852+0000 7f6ede866700 1 -- 172.21.15.78:0/108222789 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6ebc065ed0 msgr2=0x7f6ebc068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:36.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.852+0000 7f6ede866700 1 --2- 172.21.15.78:0/108222789 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6ebc065ed0 0x7f6ebc068380 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6ed81602e0 tx=0x7f6ec8009240 comp rx=0 tx=0).stop 2023-10-11T09:42:36.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.852+0000 7f6ede866700 1 -- 172.21.15.78:0/108222789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ed80b3520 msgr2=0x7f6ed8145fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:36.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.852+0000 7f6ede866700 1 --2- 172.21.15.78:0/108222789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ed80b3520 0x7f6ed8145fd0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4001270 tx=0x7f6ed400c7d0 comp rx=0 tx=0).stop 2023-10-11T09:42:36.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.852+0000 7f6ede866700 1 -- 172.21.15.78:0/108222789 shutdown_connections 2023-10-11T09:42:36.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.852+0000 7f6ede866700 1 --2- 172.21.15.78:0/108222789 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6ebc065ed0 0x7f6ebc068380 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:36.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.852+0000 7f6ede866700 1 --2- 172.21.15.78:0/108222789 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6ed8161d90 0x7f6ed80cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:36.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.852+0000 7f6ede866700 1 --2- 172.21.15.78:0/108222789 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ed8146fe0 0x7f6ed80cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:36.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.852+0000 7f6ede866700 1 --2- 172.21.15.78:0/108222789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ed80b3520 0x7f6ed8145fd0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:36.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.852+0000 7f6ede866700 1 -- 172.21.15.78:0/108222789 >> 172.21.15.78:0/108222789 conn(0x7f6ed806a060 msgr2=0x7f6ed80ae660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:36.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.852+0000 7f6ede866700 1 -- 172.21.15.78:0/108222789 shutdown_connections 2023-10-11T09:42:36.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:36.852+0000 7f6ede866700 1 -- 172.21.15.78:0/108222789 wait complete. 2023-10-11T09:42:36.882 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T09:42:37.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.044+0000 7fe8dd9b4700 1 Processor -- start 2023-10-11T09:42:37.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.044+0000 7fe8dd9b4700 1 -- start start 2023-10-11T09:42:37.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.048+0000 7fe8dd9b4700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe8d815b8e0 0x7fe8d806d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:37.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.048+0000 7fe8dd9b4700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe8d806d610 0x7fe8d80ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:37.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.048+0000 7fe8dd9b4700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8d815e010 con 0x7fe8d815a940 2023-10-11T09:42:37.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.048+0000 7fe8dd9b4700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8d815e280 con 0x7fe8d815b8e0 2023-10-11T09:42:37.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.048+0000 7fe8dd9b4700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d815e560 con 0x7fe8d806d610 2023-10-11T09:42:37.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.048+0000 7fe8dc9b2700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe8d815b8e0 0x7fe8d806d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:37.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.048+0000 7fe8dc9b2700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe8d815b8e0 0x7fe8d806d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:47626/0 (socket says 172.21.15.78:47626) 2023-10-11T09:42:37.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.048+0000 7fe8dc9b2700 1 -- 172.21.15.78:0/3286414777 learned_addr learned my addr 172.21.15.78:0/3286414777 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:37.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.048+0000 7fe8cffff700 1 --2- 172.21.15.78:0/3286414777 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe8d806d610 0x7fe8d80ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:37.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.048+0000 7fe8dc9b2700 1 -- 172.21.15.78:0/3286414777 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe8d806d610 msgr2=0x7fe8d80ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:37.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.048+0000 7fe8dc9b2700 1 --2- 172.21.15.78:0/3286414777 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe8d806d610 0x7fe8d80ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:37.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.048+0000 7fe8dc9b2700 1 -- 172.21.15.78:0/3286414777 >> v1:172.21.15.133:6789/0 conn(0x7fe8d815a940 legacy=0x7fe8d80c9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:37.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.048+0000 7fe8dc9b2700 1 -- 172.21.15.78:0/3286414777 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d815e840 con 0x7fe8d815b8e0 2023-10-11T09:42:37.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.048+0000 7fe8dc9b2700 1 --2- 172.21.15.78:0/3286414777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8d815b8e0 0x7fe8d806d0d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4009910 tx=0x7fe8d4009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2372f53812b731d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:37.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.052+0000 7fe8cf7fe700 1 -- 172.21.15.78:0/3286414777 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4011830 con 0x7fe8d815b8e0 2023-10-11T09:42:37.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.052+0000 7fe8cf7fe700 1 -- 172.21.15.78:0/3286414777 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d4011990 con 0x7fe8d815b8e0 2023-10-11T09:42:37.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.052+0000 7fe8cf7fe700 1 -- 172.21.15.78:0/3286414777 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4011d40 con 0x7fe8d815b8e0 2023-10-11T09:42:37.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.052+0000 7fe8dd9b4700 1 -- 172.21.15.78:0/3286414777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8d815b8e0 msgr2=0x7fe8d806d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:37.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.052+0000 7fe8dd9b4700 1 --2- 172.21.15.78:0/3286414777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8d815b8e0 0x7fe8d806d0d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4009910 tx=0x7fe8d4009c20 comp rx=0 tx=0).stop 2023-10-11T09:42:37.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.052+0000 7fe8dd9b4700 1 -- 172.21.15.78:0/3286414777 shutdown_connections 2023-10-11T09:42:37.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.052+0000 7fe8dd9b4700 1 --2- 172.21.15.78:0/3286414777 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe8d806d610 0x7fe8d80ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:37.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.052+0000 7fe8dd9b4700 1 --2- 172.21.15.78:0/3286414777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8d815b8e0 0x7fe8d806d0d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:37.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.052+0000 7fe8dd9b4700 1 -- 172.21.15.78:0/3286414777 >> 172.21.15.78:0/3286414777 conn(0x7fe8d80cea60 msgr2=0x7fe8d80cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:37.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.052+0000 7fe8dd9b4700 1 -- 172.21.15.78:0/3286414777 shutdown_connections 2023-10-11T09:42:37.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.052+0000 7fe8dd9b4700 1 -- 172.21.15.78:0/3286414777 wait complete. 2023-10-11T09:42:37.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.056+0000 7fe8dd9b4700 1 Processor -- start 2023-10-11T09:42:37.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.056+0000 7fe8dd9b4700 1 -- start start 2023-10-11T09:42:37.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.056+0000 7fe8dd9b4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8d806d610 0x7fe8d80ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:37.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.056+0000 7fe8dd9b4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8d815a940 0x7fe8d813d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:37.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.056+0000 7fe8dd9b4700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe8d815b8e0 0x7fe8d80cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:37.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.056+0000 7fe8dd9b4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d80cdd20 con 0x7fe8d815a940 2023-10-11T09:42:37.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.056+0000 7fe8dd9b4700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d81412e0 con 0x7fe8d806d610 2023-10-11T09:42:37.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.056+0000 7fe8dd9b4700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d80cd110 con 0x7fe8d815b8e0 2023-10-11T09:42:37.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.060+0000 7fe8cffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8d815a940 0x7fe8d813d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:37.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.060+0000 7fe8cffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8d815a940 0x7fe8d813d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35102/0 (socket says 172.21.15.78:35102) 2023-10-11T09:42:37.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.060+0000 7fe8dc9b2700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8d806d610 0x7fe8d80ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:37.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.060+0000 7fe8cffff700 1 -- 172.21.15.78:0/375352890 learned_addr learned my addr 172.21.15.78:0/375352890 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:37.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.060+0000 7fe8dd1b3700 1 --2- 172.21.15.78:0/375352890 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe8d815b8e0 0x7fe8d80cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:37.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.060+0000 7fe8cffff700 1 -- 172.21.15.78:0/375352890 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe8d815b8e0 msgr2=0x7fe8d80cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:37.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.060+0000 7fe8cffff700 1 --2- 172.21.15.78:0/375352890 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe8d815b8e0 0x7fe8d80cc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:37.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.060+0000 7fe8cffff700 1 -- 172.21.15.78:0/375352890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8d806d610 msgr2=0x7fe8d80ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:37.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.060+0000 7fe8cffff700 1 --2- 172.21.15.78:0/375352890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8d806d610 0x7fe8d80ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:37.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.060+0000 7fe8cffff700 1 -- 172.21.15.78:0/375352890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d4009570 con 0x7fe8d815a940 2023-10-11T09:42:37.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.060+0000 7fe8dc9b2700 1 --2- 172.21.15.78:0/375352890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8d806d610 0x7fe8d80ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:42:37.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.060+0000 7fe8cffff700 1 --2- 172.21.15.78:0/375352890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8d815a940 0x7fe8d813d550 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe8c800e840 tx=0x7fe8c800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:37.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.060+0000 7fe8cd7fa700 1 -- 172.21.15.78:0/375352890 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c80095a0 con 0x7fe8d815a940 2023-10-11T09:42:37.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.060+0000 7fe8dd9b4700 1 -- 172.21.15.78:0/375352890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d80cd3f0 con 0x7fe8d815a940 2023-10-11T09:42:37.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.060+0000 7fe8cd7fa700 1 -- 172.21.15.78:0/375352890 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c8013680 con 0x7fe8d815a940 2023-10-11T09:42:37.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.060+0000 7fe8cd7fa700 1 -- 172.21.15.78:0/375352890 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c80139f0 con 0x7fe8d815a940 2023-10-11T09:42:37.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.060+0000 7fe8dd9b4700 1 -- 172.21.15.78:0/375352890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d80cd9b0 con 0x7fe8d815a940 2023-10-11T09:42:37.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.064+0000 7fe8dd9b4700 1 -- 172.21.15.78:0/375352890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d80cded0 con 0x7fe8d815a940 2023-10-11T09:42:37.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.064+0000 7fe8cd7fa700 1 -- 172.21.15.78:0/375352890 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fe8c8013b50 con 0x7fe8d815a940 2023-10-11T09:42:37.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.064+0000 7fe8cd7fa700 1 --2- 172.21.15.78:0/375352890 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe8c4065ed0 0x7fe8c4068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:37.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.064+0000 7fe8cd7fa700 1 -- 172.21.15.78:0/375352890 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6495+0+0 (secure 0 0 0) 0x7fe8c8083820 con 0x7fe8d815a940 2023-10-11T09:42:37.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.064+0000 7fe8dc9b2700 1 --2- 172.21.15.78:0/375352890 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe8c4065ed0 0x7fe8c4068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:37.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.068+0000 7fe8dc9b2700 1 --2- 172.21.15.78:0/375352890 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe8c4065ed0 0x7fe8c4068380 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4000c00 tx=0x7fe8d4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:37.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.072+0000 7fe8cd7fa700 1 -- 172.21.15.78:0/375352890 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fe8d80cded0 con 0x7fe8d815a940 2023-10-11T09:42:37.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.252+0000 7fe8dd9b4700 1 -- 172.21.15.78:0/375352890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe8d8065d40 con 0x7fe8d815a940 2023-10-11T09:42:37.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.252+0000 7fe8cd7fa700 1 -- 172.21.15.78:0/375352890 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v42) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe8c80547c0 con 0x7fe8d815a940 2023-10-11T09:42:37.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.256+0000 7fe8dd9b4700 1 -- 172.21.15.78:0/375352890 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe8c4065ed0 msgr2=0x7fe8c4068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:37.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.256+0000 7fe8dd9b4700 1 --2- 172.21.15.78:0/375352890 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe8c4065ed0 0x7fe8c4068380 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4000c00 tx=0x7fe8d4008030 comp rx=0 tx=0).stop 2023-10-11T09:42:37.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.256+0000 7fe8dd9b4700 1 -- 172.21.15.78:0/375352890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8d815a940 msgr2=0x7fe8d813d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:37.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.256+0000 7fe8dd9b4700 1 --2- 172.21.15.78:0/375352890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8d815a940 0x7fe8d813d550 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe8c800e840 tx=0x7fe8c800eb50 comp rx=0 tx=0).stop 2023-10-11T09:42:37.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.260+0000 7fe8dd9b4700 1 -- 172.21.15.78:0/375352890 shutdown_connections 2023-10-11T09:42:37.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.260+0000 7fe8dd9b4700 1 --2- 172.21.15.78:0/375352890 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe8c4065ed0 0x7fe8c4068380 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:37.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.260+0000 7fe8dd9b4700 1 --2- 172.21.15.78:0/375352890 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe8d815b8e0 0x7fe8d80cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:37.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.260+0000 7fe8dd9b4700 1 --2- 172.21.15.78:0/375352890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8d815a940 0x7fe8d813d550 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:37.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.260+0000 7fe8dd9b4700 1 --2- 172.21.15.78:0/375352890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8d806d610 0x7fe8d80ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:37.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.260+0000 7fe8dd9b4700 1 -- 172.21.15.78:0/375352890 >> 172.21.15.78:0/375352890 conn(0x7fe8d80cea60 msgr2=0x7fe8d8070c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:37.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.260+0000 7fe8dd9b4700 1 -- 172.21.15.78:0/375352890 shutdown_connections 2023-10-11T09:42:37.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:37.260+0000 7fe8dd9b4700 1 -- 172.21.15.78:0/375352890 wait complete. 2023-10-11T09:42:37.271 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2023-10-11T09:42:37.288 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi078.front.sepia.ceph.com... 2023-10-11T09:42:37.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:42:37.289 DEBUG:teuthology.orchestra.run.smithi078:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-11T09:42:37.318 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:ceph-fuse[46411]: fuse finished with error 0 and tester_r 0 2023-10-11T09:42:37.330 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:42:43.432 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2023-10-11T09:42:43.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:42:43.433 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-11T09:42:43.455 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:42:43.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.660+0000 7f158fde6700 1 Processor -- start 2023-10-11T09:42:43.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.660+0000 7f158fde6700 1 -- start start 2023-10-11T09:42:43.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.664+0000 7f158fde6700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f1588160d40 0x7f15880cd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:43.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.664+0000 7f158fde6700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1588071040 0x7f15880cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:43.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.664+0000 7f158fde6700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f158806ffa0 con 0x7f15880cd7c0 2023-10-11T09:42:43.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.664+0000 7f158fde6700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1588070c20 con 0x7f1588160d40 2023-10-11T09:42:43.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.664+0000 7f158fde6700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1588161110 con 0x7f1588071040 2023-10-11T09:42:43.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.664+0000 7f158db82700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f1588160d40 0x7f15880cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:43.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.664+0000 7f158db82700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f1588160d40 0x7f15880cd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35104/0 (socket says 172.21.15.78:35104) 2023-10-11T09:42:43.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.664+0000 7f158db82700 1 -- 172.21.15.78:0/110835219 learned_addr learned my addr 172.21.15.78:0/110835219 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:43.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.664+0000 7f158d381700 1 --2- 172.21.15.78:0/110835219 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1588071040 0x7f15880cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:43.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.664+0000 7f158db82700 1 -- 172.21.15.78:0/110835219 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1588071040 msgr2=0x7f15880cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:43.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.664+0000 7f158db82700 1 --2- 172.21.15.78:0/110835219 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1588071040 0x7f15880cddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:43.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.664+0000 7f158db82700 1 -- 172.21.15.78:0/110835219 >> v1:172.21.15.133:6789/0 conn(0x7f15880cd7c0 legacy=0x7f15881647f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:43.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.664+0000 7f158db82700 1 -- 172.21.15.78:0/110835219 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1588070280 con 0x7f1588160d40 2023-10-11T09:42:43.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.664+0000 7f158db82700 1 --2- 172.21.15.78:0/110835219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1588160d40 0x7f15880cd280 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f1584009910 tx=0x7f1584009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef90f2e86c36f894 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:43.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.664+0000 7f158cb80700 1 -- 172.21.15.78:0/110835219 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1584011830 con 0x7f1588160d40 2023-10-11T09:42:43.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.664+0000 7f158cb80700 1 -- 172.21.15.78:0/110835219 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1584011990 con 0x7f1588160d40 2023-10-11T09:42:43.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.664+0000 7f158cb80700 1 -- 172.21.15.78:0/110835219 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1584011d40 con 0x7f1588160d40 2023-10-11T09:42:43.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.668+0000 7f158fde6700 1 -- 172.21.15.78:0/110835219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1588160d40 msgr2=0x7f15880cd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:43.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.668+0000 7f158fde6700 1 --2- 172.21.15.78:0/110835219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1588160d40 0x7f15880cd280 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f1584009910 tx=0x7f1584009c20 comp rx=0 tx=0).stop 2023-10-11T09:42:43.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.668+0000 7f158fde6700 1 -- 172.21.15.78:0/110835219 shutdown_connections 2023-10-11T09:42:43.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.668+0000 7f158fde6700 1 --2- 172.21.15.78:0/110835219 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1588071040 0x7f15880cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:43.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.668+0000 7f158fde6700 1 --2- 172.21.15.78:0/110835219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1588160d40 0x7f15880cd280 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:43.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.668+0000 7f158fde6700 1 -- 172.21.15.78:0/110835219 >> 172.21.15.78:0/110835219 conn(0x7f158806aca0 msgr2=0x7f15880b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:43.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.668+0000 7f158fde6700 1 -- 172.21.15.78:0/110835219 shutdown_connections 2023-10-11T09:42:43.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.668+0000 7f158fde6700 1 -- 172.21.15.78:0/110835219 wait complete. 2023-10-11T09:42:43.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.672+0000 7f158fde6700 1 Processor -- start 2023-10-11T09:42:43.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.672+0000 7f158fde6700 1 -- start start 2023-10-11T09:42:43.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.672+0000 7f158fde6700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1588071040 0x7f15880b35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:43.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.672+0000 7f158fde6700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15880cd7c0 0x7f1588148300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:43.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.672+0000 7f158fde6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1588160d40 0x7f15880b1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:43.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.672+0000 7f158fde6700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15880b2550 con 0x7f1588160d40 2023-10-11T09:42:43.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.672+0000 7f158fde6700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15880b2830 con 0x7f15880cd7c0 2023-10-11T09:42:43.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.672+0000 7f158fde6700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15880b2b10 con 0x7f1588071040 2023-10-11T09:42:43.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.672+0000 7f158e383700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1588160d40 0x7f15880b1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:43.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.672+0000 7f158e383700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1588160d40 0x7f15880b1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35110/0 (socket says 172.21.15.78:35110) 2023-10-11T09:42:43.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.672+0000 7f158e383700 1 -- 172.21.15.78:0/2354149112 learned_addr learned my addr 172.21.15.78:0/2354149112 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:43.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.672+0000 7f158d381700 1 --2- 172.21.15.78:0/2354149112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15880cd7c0 0x7f1588148300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:43.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.672+0000 7f158db82700 1 --2- 172.21.15.78:0/2354149112 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1588071040 0x7f15880b35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:43.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.676+0000 7f158e383700 1 -- 172.21.15.78:0/2354149112 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1588071040 msgr2=0x7f15880b35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:43.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.676+0000 7f158e383700 1 --2- 172.21.15.78:0/2354149112 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1588071040 0x7f15880b35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:43.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.676+0000 7f158e383700 1 -- 172.21.15.78:0/2354149112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15880cd7c0 msgr2=0x7f1588148300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:43.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.676+0000 7f158e383700 1 --2- 172.21.15.78:0/2354149112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15880cd7c0 0x7f1588148300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:43.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.676+0000 7f158e383700 1 -- 172.21.15.78:0/2354149112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1584009570 con 0x7f1588160d40 2023-10-11T09:42:43.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.676+0000 7f158e383700 1 --2- 172.21.15.78:0/2354149112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1588160d40 0x7f15880b1c90 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f158000cbc0 tx=0x7f158000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:43.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.676+0000 7f157e7fc700 1 -- 172.21.15.78:0/2354149112 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1580003c20 con 0x7f1588160d40 2023-10-11T09:42:43.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.676+0000 7f157e7fc700 1 -- 172.21.15.78:0/2354149112 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1580003d80 con 0x7f1588160d40 2023-10-11T09:42:43.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.676+0000 7f158fde6700 1 -- 172.21.15.78:0/2354149112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15880b2df0 con 0x7f1588160d40 2023-10-11T09:42:43.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.676+0000 7f157e7fc700 1 -- 172.21.15.78:0/2354149112 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158000aae0 con 0x7f1588160d40 2023-10-11T09:42:43.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.676+0000 7f158fde6700 1 -- 172.21.15.78:0/2354149112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1588148840 con 0x7f1588160d40 2023-10-11T09:42:43.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.676+0000 7f158fde6700 1 -- 172.21.15.78:0/2354149112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1588072250 con 0x7f1588160d40 2023-10-11T09:42:43.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.680+0000 7f157e7fc700 1 -- 172.21.15.78:0/2354149112 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f158000ac40 con 0x7f1588160d40 2023-10-11T09:42:43.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.680+0000 7f157e7fc700 1 --2- 172.21.15.78:0/2354149112 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f156c065ed0 0x7f156c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:43.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.680+0000 7f157e7fc700 1 -- 172.21.15.78:0/2354149112 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6495+0+0 (secure 0 0 0) 0x7f1580083980 con 0x7f1588160d40 2023-10-11T09:42:43.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.680+0000 7f158db82700 1 --2- 172.21.15.78:0/2354149112 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f156c065ed0 0x7f156c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:43.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.680+0000 7f158db82700 1 --2- 172.21.15.78:0/2354149112 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f156c065ed0 0x7f156c068380 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1584000c00 tx=0x7f1584008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:43.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.688+0000 7f157e7fc700 1 -- 172.21.15.78:0/2354149112 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f1580050e10 con 0x7f1588160d40 2023-10-11T09:42:43.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.896+0000 7f158fde6700 1 -- 172.21.15.78:0/2354149112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15880b3310 con 0x7f1588160d40 2023-10-11T09:42:43.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.900+0000 7f157e7fc700 1 -- 172.21.15.78:0/2354149112 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4650 (secure 0 0 0) 0x7f15880b3310 con 0x7f1588160d40 2023-10-11T09:42:43.903 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:43.903 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5308,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/655413753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":655413753},{"type":"v1","addr":"172.21.15.133:6835","nonce":655413753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5352,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2966622437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2966622437},{"type":"v1","addr":"172.21.15.78:6835","nonce":2966622437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5355,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/700246445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":700246445},{"type":"v1","addr":"172.21.15.78:6837","nonce":700246445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:41:59.225318+0000","modified":"2023-10-11T09:42:13.894931+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5364},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5364":{"gid":5364,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.133:6833/1862542010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1862542010},{"type":"v1","addr":"172.21.15.133:6833","nonce":1862542010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-11T09:42:43.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.900+0000 7f158fde6700 1 -- 172.21.15.78:0/2354149112 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f156c065ed0 msgr2=0x7f156c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:43.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.900+0000 7f158fde6700 1 --2- 172.21.15.78:0/2354149112 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f156c065ed0 0x7f156c068380 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1584000c00 tx=0x7f1584008030 comp rx=0 tx=0).stop 2023-10-11T09:42:43.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.900+0000 7f158fde6700 1 -- 172.21.15.78:0/2354149112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1588160d40 msgr2=0x7f15880b1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:43.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.900+0000 7f158fde6700 1 --2- 172.21.15.78:0/2354149112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1588160d40 0x7f15880b1c90 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f158000cbc0 tx=0x7f158000ced0 comp rx=0 tx=0).stop 2023-10-11T09:42:43.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.904+0000 7f158fde6700 1 -- 172.21.15.78:0/2354149112 shutdown_connections 2023-10-11T09:42:43.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.904+0000 7f158fde6700 1 --2- 172.21.15.78:0/2354149112 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f156c065ed0 0x7f156c068380 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:43.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.904+0000 7f158fde6700 1 --2- 172.21.15.78:0/2354149112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1588160d40 0x7f15880b1c90 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:43.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.904+0000 7f158fde6700 1 --2- 172.21.15.78:0/2354149112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15880cd7c0 0x7f1588148300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:43.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.904+0000 7f158fde6700 1 --2- 172.21.15.78:0/2354149112 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1588071040 0x7f15880b35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:43.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.904+0000 7f158fde6700 1 -- 172.21.15.78:0/2354149112 >> 172.21.15.78:0/2354149112 conn(0x7f158806aca0 msgr2=0x7f1588161250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:43.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.904+0000 7f158fde6700 1 -- 172.21.15.78:0/2354149112 shutdown_connections 2023-10-11T09:42:43.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:43.904+0000 7f158fde6700 1 -- 172.21.15.78:0/2354149112 wait complete. 2023-10-11T09:42:43.908 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 17 2023-10-11T09:42:43.927 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2023-10-11T09:42:44.134 INFO:teuthology.orchestra.run.smithi133.stdout:{"message":"","return_code":0} 2023-10-11T09:42:44.135 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-10-11T09:42:44.135 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-11T09:42:44.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.316+0000 7f7cd607e700 1 Processor -- start 2023-10-11T09:42:44.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.316+0000 7f7cd607e700 1 -- start start 2023-10-11T09:42:44.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.320+0000 7f7cd607e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f7cd00ce8c0 0x7f7cd00cb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:44.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.320+0000 7f7cd607e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7cd00b2800 0x7f7cd00cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:44.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.320+0000 7f7cd607e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cd015ad70 con 0x7f7cd00cf1d0 2023-10-11T09:42:44.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.320+0000 7f7cd607e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cd00cf620 con 0x7f7cd00ce8c0 2023-10-11T09:42:44.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.320+0000 7f7cd607e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd00d1c40 con 0x7f7cd00b2800 2023-10-11T09:42:44.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.320+0000 7f7cd507c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f7cd00ce8c0 0x7f7cd00cb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:44.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.320+0000 7f7cd507c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f7cd00ce8c0 0x7f7cd00cb340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35116/0 (socket says 172.21.15.78:35116) 2023-10-11T09:42:44.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.320+0000 7f7cd507c700 1 -- 172.21.15.78:0/2722012462 learned_addr learned my addr 172.21.15.78:0/2722012462 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:44.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.320+0000 7f7cd487b700 1 --2- 172.21.15.78:0/2722012462 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7cd00b2800 0x7f7cd00cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:44.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.320+0000 7f7cd507c700 1 -- 172.21.15.78:0/2722012462 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7cd00b2800 msgr2=0x7f7cd00cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:44.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.320+0000 7f7cd507c700 1 --2- 172.21.15.78:0/2722012462 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7cd00b2800 0x7f7cd00cec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:44.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.320+0000 7f7cd507c700 1 -- 172.21.15.78:0/2722012462 >> v1:172.21.15.133:6789/0 conn(0x7f7cd00cf1d0 legacy=0x7f7cd006f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:44.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.320+0000 7f7cd507c700 1 -- 172.21.15.78:0/2722012462 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd006e0f0 con 0x7f7cd00ce8c0 2023-10-11T09:42:44.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.320+0000 7f7cd507c700 1 --2- 172.21.15.78:0/2722012462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cd00ce8c0 0x7f7cd00cb340 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc009910 tx=0x7f7ccc009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9498c79926e8c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:44.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.320+0000 7f7cc7fff700 1 -- 172.21.15.78:0/2722012462 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc005e50 con 0x7f7cd00ce8c0 2023-10-11T09:42:44.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.324+0000 7f7cc7fff700 1 -- 172.21.15.78:0/2722012462 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ccc006030 con 0x7f7cd00ce8c0 2023-10-11T09:42:44.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.324+0000 7f7cc7fff700 1 -- 172.21.15.78:0/2722012462 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc0063e0 con 0x7f7cd00ce8c0 2023-10-11T09:42:44.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.324+0000 7f7cd607e700 1 -- 172.21.15.78:0/2722012462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cd00ce8c0 msgr2=0x7f7cd00cb340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:44.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.324+0000 7f7cd607e700 1 --2- 172.21.15.78:0/2722012462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cd00ce8c0 0x7f7cd00cb340 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc009910 tx=0x7f7ccc009c20 comp rx=0 tx=0).stop 2023-10-11T09:42:44.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.324+0000 7f7cd607e700 1 -- 172.21.15.78:0/2722012462 shutdown_connections 2023-10-11T09:42:44.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.324+0000 7f7cd607e700 1 --2- 172.21.15.78:0/2722012462 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7cd00b2800 0x7f7cd00cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:44.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.324+0000 7f7cd607e700 1 --2- 172.21.15.78:0/2722012462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cd00ce8c0 0x7f7cd00cb340 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:44.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.324+0000 7f7cd607e700 1 -- 172.21.15.78:0/2722012462 >> 172.21.15.78:0/2722012462 conn(0x7f7cd009a130 msgr2=0x7f7cd00d0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:44.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.324+0000 7f7cd607e700 1 -- 172.21.15.78:0/2722012462 shutdown_connections 2023-10-11T09:42:44.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.324+0000 7f7cd607e700 1 -- 172.21.15.78:0/2722012462 wait complete. 2023-10-11T09:42:44.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.328+0000 7f7cd607e700 1 Processor -- start 2023-10-11T09:42:44.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.328+0000 7f7cd607e700 1 -- start start 2023-10-11T09:42:44.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.328+0000 7f7cd607e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7cd00b2800 0x7f7cd00c67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:44.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.328+0000 7f7cd607e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cd00ce8c0 0x7f7cd01f0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:44.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.328+0000 7f7cd607e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cd00cf1d0 0x7f7cd01ef150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:44.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.328+0000 7f7cd607e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd00b59c0 con 0x7f7cd00cf1d0 2023-10-11T09:42:44.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.328+0000 7f7cd607e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd00b4990 con 0x7f7cd00ce8c0 2023-10-11T09:42:44.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.328+0000 7f7cd607e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd00b4dd0 con 0x7f7cd00b2800 2023-10-11T09:42:44.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.328+0000 7f7cd587d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cd00cf1d0 0x7f7cd01ef150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:44.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.328+0000 7f7cd587d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cd00cf1d0 0x7f7cd01ef150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35126/0 (socket says 172.21.15.78:35126) 2023-10-11T09:42:44.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.328+0000 7f7cd587d700 1 -- 172.21.15.78:0/3940146541 learned_addr learned my addr 172.21.15.78:0/3940146541 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:44.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.328+0000 7f7cd507c700 1 --2- 172.21.15.78:0/3940146541 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7cd00b2800 0x7f7cd00c67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:44.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.332+0000 7f7cd487b700 1 --2- 172.21.15.78:0/3940146541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cd00ce8c0 0x7f7cd01f0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:44.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.332+0000 7f7cd587d700 1 -- 172.21.15.78:0/3940146541 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7cd00b2800 msgr2=0x7f7cd00c67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:44.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.332+0000 7f7cd587d700 1 --2- 172.21.15.78:0/3940146541 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7cd00b2800 0x7f7cd00c67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:44.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.332+0000 7f7cd587d700 1 -- 172.21.15.78:0/3940146541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cd00ce8c0 msgr2=0x7f7cd01f0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:44.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.332+0000 7f7cd587d700 1 --2- 172.21.15.78:0/3940146541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cd00ce8c0 0x7f7cd01f0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:44.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.332+0000 7f7cd587d700 1 -- 172.21.15.78:0/3940146541 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ccc009520 con 0x7f7cd00cf1d0 2023-10-11T09:42:44.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.332+0000 7f7cd587d700 1 --2- 172.21.15.78:0/3940146541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cd00cf1d0 0x7f7cd01ef150 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7cc800cc80 tx=0x7f7cc800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:44.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.332+0000 7f7cc5ffb700 1 -- 172.21.15.78:0/3940146541 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8003c20 con 0x7f7cd00cf1d0 2023-10-11T09:42:44.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.332+0000 7f7cc5ffb700 1 -- 172.21.15.78:0/3940146541 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cc8003d80 con 0x7f7cd00cf1d0 2023-10-11T09:42:44.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.332+0000 7f7cd607e700 1 -- 172.21.15.78:0/3940146541 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cd00b5090 con 0x7f7cd00cf1d0 2023-10-11T09:42:44.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.332+0000 7f7cc5ffb700 1 -- 172.21.15.78:0/3940146541 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc800aaa0 con 0x7f7cd00cf1d0 2023-10-11T09:42:44.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.332+0000 7f7cd607e700 1 -- 172.21.15.78:0/3940146541 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cd00b5650 con 0x7f7cd00cf1d0 2023-10-11T09:42:44.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.336+0000 7f7cd607e700 1 -- 172.21.15.78:0/3940146541 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cd00b5b70 con 0x7f7cd00cf1d0 2023-10-11T09:42:44.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.336+0000 7f7cc5ffb700 1 -- 172.21.15.78:0/3940146541 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f7cc800ac00 con 0x7f7cd00cf1d0 2023-10-11T09:42:44.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.336+0000 7f7cc5ffb700 1 --2- 172.21.15.78:0/3940146541 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7cb4065ed0 0x7f7cb4068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:44.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.336+0000 7f7cc5ffb700 1 -- 172.21.15.78:0/3940146541 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6495+0+0 (secure 0 0 0) 0x7f7cc8083770 con 0x7f7cd00cf1d0 2023-10-11T09:42:44.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.336+0000 7f7cd507c700 1 --2- 172.21.15.78:0/3940146541 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7cb4065ed0 0x7f7cb4068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:44.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.340+0000 7f7cd507c700 1 --2- 172.21.15.78:0/3940146541 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7cb4065ed0 0x7f7cb4068380 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc006000 tx=0x7f7ccc008070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:44.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.344+0000 7f7cc5ffb700 1 -- 172.21.15.78:0/3940146541 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f7cd00b5b70 con 0x7f7cd00cf1d0 2023-10-11T09:42:44.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.560+0000 7f7cd607e700 1 -- 172.21.15.78:0/3940146541 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7cd015ad70 con 0x7f7cd00cf1d0 2023-10-11T09:42:44.566 INFO:tasks.ceph.mon.a.smithi078.stderr:2023-10-11T09:42:44.564+0000 7f90131f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-11T09:42:44.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.576+0000 7f7cc5ffb700 1 -- 172.21.15.78:0/3940146541 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7cd015ad70 con 0x7f7cd00cf1d0 2023-10-11T09:42:44.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.580+0000 7f7cd607e700 1 -- 172.21.15.78:0/3940146541 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7cb4065ed0 msgr2=0x7f7cb4068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:44.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.580+0000 7f7cd607e700 1 --2- 172.21.15.78:0/3940146541 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7cb4065ed0 0x7f7cb4068380 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc006000 tx=0x7f7ccc008070 comp rx=0 tx=0).stop 2023-10-11T09:42:44.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.580+0000 7f7cd607e700 1 -- 172.21.15.78:0/3940146541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cd00cf1d0 msgr2=0x7f7cd01ef150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:44.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.580+0000 7f7cd607e700 1 --2- 172.21.15.78:0/3940146541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cd00cf1d0 0x7f7cd01ef150 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7cc800cc80 tx=0x7f7cc800cf90 comp rx=0 tx=0).stop 2023-10-11T09:42:44.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.580+0000 7f7cd607e700 1 -- 172.21.15.78:0/3940146541 shutdown_connections 2023-10-11T09:42:44.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.580+0000 7f7cd607e700 1 --2- 172.21.15.78:0/3940146541 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7cb4065ed0 0x7f7cb4068380 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:44.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.580+0000 7f7cd607e700 1 --2- 172.21.15.78:0/3940146541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cd00cf1d0 0x7f7cd01ef150 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:44.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.580+0000 7f7cd607e700 1 --2- 172.21.15.78:0/3940146541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cd00ce8c0 0x7f7cd01f0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:44.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.580+0000 7f7cd607e700 1 --2- 172.21.15.78:0/3940146541 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7cd00b2800 0x7f7cd00c67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:44.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.580+0000 7f7cd607e700 1 -- 172.21.15.78:0/3940146541 >> 172.21.15.78:0/3940146541 conn(0x7f7cd009a130 msgr2=0x7f7cd00afab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:44.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.580+0000 7f7cd607e700 1 -- 172.21.15.78:0/3940146541 shutdown_connections 2023-10-11T09:42:44.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.580+0000 7f7cd607e700 1 -- 172.21.15.78:0/3940146541 wait complete. 2023-10-11T09:42:44.587 INFO:teuthology.orchestra.run.smithi078.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-11T09:42:44.594 INFO:tasks.ceph.mds.b.smithi133.stderr: -7267> 2023-10-11T09:42:01.702+0000 7feebb877700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-11T09:42:44.607 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-11T09:42:44.614 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-11T09:42:44.656 INFO:tasks.ceph.mds.b.smithi133.stdout:starting mds.b at 2023-10-11T09:42:44.682 INFO:teuthology.orchestra.run.smithi078.stdout:[global] 2023-10-11T09:42:44.682 INFO:teuthology.orchestra.run.smithi078.stdout: chdir = "" 2023-10-11T09:42:44.682 INFO:teuthology.orchestra.run.smithi078.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-11T09:42:44.683 INFO:teuthology.orchestra.run.smithi078.stdout: auth supported = cephx 2023-10-11T09:42:44.683 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.683 INFO:teuthology.orchestra.run.smithi078.stdout: filestore xattr use omap = true 2023-10-11T09:42:44.683 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.683 INFO:teuthology.orchestra.run.smithi078.stdout: mon clock drift allowed = 1.000 2023-10-11T09:42:44.684 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.684 INFO:teuthology.orchestra.run.smithi078.stdout: osd crush chooseleaf type = 0 2023-10-11T09:42:44.684 INFO:teuthology.orchestra.run.smithi078.stdout: auth debug = true 2023-10-11T09:42:44.684 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.684 INFO:teuthology.orchestra.run.smithi078.stdout: ms die on old message = true 2023-10-11T09:42:44.685 INFO:teuthology.orchestra.run.smithi078.stdout: ms die on bug = true 2023-10-11T09:42:44.685 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.685 INFO:teuthology.orchestra.run.smithi078.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-11T09:42:44.685 INFO:teuthology.orchestra.run.smithi078.stdout: mon pg warn max object skew = 0 2023-10-11T09:42:44.685 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.686 INFO:teuthology.orchestra.run.smithi078.stdout: # disable pg_autoscaler by default for new pools 2023-10-11T09:42:44.686 INFO:teuthology.orchestra.run.smithi078.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-11T09:42:44.686 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.686 INFO:teuthology.orchestra.run.smithi078.stdout: osd pool default size = 2 2023-10-11T09:42:44.686 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.687 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd allow primary affinity = true 2023-10-11T09:42:44.687 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd allow pg remap = true 2023-10-11T09:42:44.687 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on legacy crush tunables = false 2023-10-11T09:42:44.687 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on crush straw calc version zero = false 2023-10-11T09:42:44.687 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on no sortbitwise = false 2023-10-11T09:42:44.688 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on osd down out interval zero = false 2023-10-11T09:42:44.688 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on too few osds = false 2023-10-11T09:42:44.688 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-11T09:42:44.688 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-11T09:42:44.688 INFO:teuthology.orchestra.run.smithi078.stdout: mon_allow_pool_size_one = true 2023-10-11T09:42:44.689 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.689 INFO:teuthology.orchestra.run.smithi078.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-11T09:42:44.689 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.689 INFO:teuthology.orchestra.run.smithi078.stdout: osd default data pool replay window = 5 2023-10-11T09:42:44.689 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.690 INFO:teuthology.orchestra.run.smithi078.stdout: mon allow pool delete = true 2023-10-11T09:42:44.690 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.690 INFO:teuthology.orchestra.run.smithi078.stdout: mon cluster log file level = debug 2023-10-11T09:42:44.690 INFO:teuthology.orchestra.run.smithi078.stdout: debug asserts on shutdown = true 2023-10-11T09:42:44.690 INFO:teuthology.orchestra.run.smithi078.stdout: mon health detail to clog = false 2023-10-11T09:42:44.691 INFO:teuthology.orchestra.run.smithi078.stdout: mon host = "172.21.15.78,172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790]" 2023-10-11T09:42:44.691 INFO:teuthology.orchestra.run.smithi078.stdout: fsid = dee99db9-7a72-406f-a1e2-917943f2b13d 2023-10-11T09:42:44.691 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.691 INFO:teuthology.orchestra.run.smithi078.stdout:[osd] 2023-10-11T09:42:44.691 INFO:teuthology.orchestra.run.smithi078.stdout: osd journal size = 100 2023-10-11T09:42:44.692 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.692 INFO:teuthology.orchestra.run.smithi078.stdout: osd scrub load threshold = 5.0 2023-10-11T09:42:44.692 INFO:teuthology.orchestra.run.smithi078.stdout: osd scrub max interval = 600 2023-10-11T09:42:44.692 INFO:teuthology.orchestra.run.smithi078.stdout: osd mclock profile = high_recovery_ops 2023-10-11T09:42:44.692 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.692 INFO:teuthology.orchestra.run.smithi078.stdout: osd recover clone overlap = true 2023-10-11T09:42:44.693 INFO:teuthology.orchestra.run.smithi078.stdout: osd recovery max chunk = 1048576 2023-10-11T09:42:44.693 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.693 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug shutdown = true 2023-10-11T09:42:44.693 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug op order = true 2023-10-11T09:42:44.693 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug verify stray on activate = true 2023-10-11T09:42:44.694 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.694 INFO:teuthology.orchestra.run.smithi078.stdout: osd open classes on start = true 2023-10-11T09:42:44.694 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug pg log writeout = true 2023-10-11T09:42:44.694 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.694 INFO:teuthology.orchestra.run.smithi078.stdout: osd deep scrub update digest min age = 30 2023-10-11T09:42:44.695 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.695 INFO:teuthology.orchestra.run.smithi078.stdout: osd map max advance = 10 2023-10-11T09:42:44.695 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.695 INFO:teuthology.orchestra.run.smithi078.stdout: journal zero on create = true 2023-10-11T09:42:44.695 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.695 INFO:teuthology.orchestra.run.smithi078.stdout: filestore ondisk finisher threads = 3 2023-10-11T09:42:44.696 INFO:teuthology.orchestra.run.smithi078.stdout: filestore apply finisher threads = 3 2023-10-11T09:42:44.696 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.696 INFO:teuthology.orchestra.run.smithi078.stdout: bdev debug aio = true 2023-10-11T09:42:44.696 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug misdirected ops = true 2023-10-11T09:42:44.696 INFO:teuthology.orchestra.run.smithi078.stdout: bdev async discard = True 2023-10-11T09:42:44.697 INFO:teuthology.orchestra.run.smithi078.stdout: bdev enable discard = True 2023-10-11T09:42:44.697 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore allocator = bitmap 2023-10-11T09:42:44.697 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore block size = 96636764160 2023-10-11T09:42:44.697 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore fsck on mount = True 2023-10-11T09:42:44.697 INFO:teuthology.orchestra.run.smithi078.stdout: debug bluefs = 1/20 2023-10-11T09:42:44.698 INFO:teuthology.orchestra.run.smithi078.stdout: debug bluestore = 1/20 2023-10-11T09:42:44.698 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:42:44.698 INFO:teuthology.orchestra.run.smithi078.stdout: debug osd = 20 2023-10-11T09:42:44.698 INFO:teuthology.orchestra.run.smithi078.stdout: debug rocksdb = 4/10 2023-10-11T09:42:44.698 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd backfillfull_ratio = 0.85 2023-10-11T09:42:44.698 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd full ratio = 0.9 2023-10-11T09:42:44.699 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd nearfull ratio = 0.8 2023-10-11T09:42:44.699 INFO:teuthology.orchestra.run.smithi078.stdout: osd failsafe full ratio = 0.95 2023-10-11T09:42:44.699 INFO:teuthology.orchestra.run.smithi078.stdout: osd objectstore = bluestore 2023-10-11T09:42:44.699 INFO:teuthology.orchestra.run.smithi078.stdout: osd op complaint time = 180 2023-10-11T09:42:44.699 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.700 INFO:teuthology.orchestra.run.smithi078.stdout:[mgr] 2023-10-11T09:42:44.700 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:42:44.700 INFO:teuthology.orchestra.run.smithi078.stdout: debug mgr = 20 2023-10-11T09:42:44.700 INFO:teuthology.orchestra.run.smithi078.stdout: debug mon = 20 2023-10-11T09:42:44.700 INFO:teuthology.orchestra.run.smithi078.stdout: debug auth = 20 2023-10-11T09:42:44.701 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:42:44.701 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:42:44.701 INFO:teuthology.orchestra.run.smithi078.stdout: mgr/telemetry/nag = false 2023-10-11T09:42:44.701 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.702 INFO:teuthology.orchestra.run.smithi078.stdout:[mon] 2023-10-11T09:42:44.702 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:42:44.702 INFO:teuthology.orchestra.run.smithi078.stdout: debug mon = 20 2023-10-11T09:42:44.702 INFO:teuthology.orchestra.run.smithi078.stdout: debug paxos = 20 2023-10-11T09:42:44.702 INFO:teuthology.orchestra.run.smithi078.stdout: debug auth = 20 2023-10-11T09:42:44.703 INFO:teuthology.orchestra.run.smithi078.stdout: mon data avail warn = 5 2023-10-11T09:42:44.703 INFO:teuthology.orchestra.run.smithi078.stdout: mon mgr mkfs grace = 240 2023-10-11T09:42:44.703 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:42:44.703 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd reporter subtree level = osd 2023-10-11T09:42:44.704 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd prime pg temp = true 2023-10-11T09:42:44.704 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:42:44.704 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.704 INFO:teuthology.orchestra.run.smithi078.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-11T09:42:44.704 INFO:teuthology.orchestra.run.smithi078.stdout: auth mon ticket ttl = 660 # 11m 2023-10-11T09:42:44.705 INFO:teuthology.orchestra.run.smithi078.stdout: auth service ticket ttl = 240 # 4m 2023-10-11T09:42:44.705 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.705 INFO:teuthology.orchestra.run.smithi078.stdout: # don't complain about insecure global_id in the test suite 2023-10-11T09:42:44.705 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-11T09:42:44.706 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-11T09:42:44.706 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.706 INFO:teuthology.orchestra.run.smithi078.stdout: # 1m isn't quite enough 2023-10-11T09:42:44.706 INFO:teuthology.orchestra.run.smithi078.stdout: mon_down_mkfs_grace = 2m 2023-10-11T09:42:44.706 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.707 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_filestore_osds = false 2023-10-11T09:42:44.707 INFO:teuthology.orchestra.run.smithi078.stdout: mon op complaint time = 120 2023-10-11T09:42:44.707 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.707 INFO:teuthology.orchestra.run.smithi078.stdout:[client] 2023-10-11T09:42:44.708 INFO:teuthology.orchestra.run.smithi078.stdout: rgw cache enabled = true 2023-10-11T09:42:44.708 INFO:teuthology.orchestra.run.smithi078.stdout: rgw enable ops log = true 2023-10-11T09:42:44.708 INFO:teuthology.orchestra.run.smithi078.stdout: rgw enable usage log = true 2023-10-11T09:42:44.708 INFO:teuthology.orchestra.run.smithi078.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-11T09:42:44.709 INFO:teuthology.orchestra.run.smithi078.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-11T09:42:44.709 INFO:teuthology.orchestra.run.smithi078.stdout: client mount timeout = 600 2023-10-11T09:42:44.709 INFO:teuthology.orchestra.run.smithi078.stdout: debug client = 20 2023-10-11T09:42:44.709 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:42:44.710 INFO:teuthology.orchestra.run.smithi078.stdout: pid file = "" 2023-10-11T09:42:44.710 INFO:teuthology.orchestra.run.smithi078.stdout: rados mon op timeout = 900 2023-10-11T09:42:44.710 INFO:teuthology.orchestra.run.smithi078.stdout: rados osd op timeout = 900 2023-10-11T09:42:44.710 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.a] 2023-10-11T09:42:44.711 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.b] 2023-10-11T09:42:44.711 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.c] 2023-10-11T09:42:44.711 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.a] 2023-10-11T09:42:44.711 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.c] 2023-10-11T09:42:44.711 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.b] 2023-10-11T09:42:44.712 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.d] 2023-10-11T09:42:44.712 INFO:teuthology.orchestra.run.smithi078.stdout:[mds] 2023-10-11T09:42:44.712 INFO:teuthology.orchestra.run.smithi078.stdout: debug mds = 20 2023-10-11T09:42:44.712 INFO:teuthology.orchestra.run.smithi078.stdout: debug mds balancer = 20 2023-10-11T09:42:44.712 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:42:44.713 INFO:teuthology.orchestra.run.smithi078.stdout: mds debug frag = True 2023-10-11T09:42:44.713 INFO:teuthology.orchestra.run.smithi078.stdout: mds debug scatterstat = True 2023-10-11T09:42:44.713 INFO:teuthology.orchestra.run.smithi078.stdout: mds op complaint time = 180 2023-10-11T09:42:44.713 INFO:teuthology.orchestra.run.smithi078.stdout: mds verify scatter = False 2023-10-11T09:42:44.713 INFO:teuthology.orchestra.run.smithi078.stdout: osd op complaint time = 180 2023-10-11T09:42:44.713 INFO:teuthology.orchestra.run.smithi078.stdout: rados mon op timeout = 900 2023-10-11T09:42:44.714 INFO:teuthology.orchestra.run.smithi078.stdout: rados osd op timeout = 900 2023-10-11T09:42:44.715 INFO:teuthology.orchestra.run.smithi133.stdout:[global] 2023-10-11T09:42:44.716 INFO:teuthology.orchestra.run.smithi133.stdout: chdir = "" 2023-10-11T09:42:44.716 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-11T09:42:44.716 INFO:teuthology.orchestra.run.smithi133.stdout: auth supported = cephx 2023-10-11T09:42:44.716 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.716 INFO:teuthology.orchestra.run.smithi133.stdout: filestore xattr use omap = true 2023-10-11T09:42:44.717 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.717 INFO:teuthology.orchestra.run.smithi133.stdout: mon clock drift allowed = 1.000 2023-10-11T09:42:44.717 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.717 INFO:teuthology.orchestra.run.smithi133.stdout: osd crush chooseleaf type = 0 2023-10-11T09:42:44.717 INFO:teuthology.orchestra.run.smithi133.stdout: auth debug = true 2023-10-11T09:42:44.717 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.718 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on old message = true 2023-10-11T09:42:44.718 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on bug = true 2023-10-11T09:42:44.718 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.718 INFO:teuthology.orchestra.run.smithi133.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-11T09:42:44.718 INFO:teuthology.orchestra.run.smithi133.stdout: mon pg warn max object skew = 0 2023-10-11T09:42:44.719 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.719 INFO:teuthology.orchestra.run.smithi133.stdout: # disable pg_autoscaler by default for new pools 2023-10-11T09:42:44.719 INFO:teuthology.orchestra.run.smithi133.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-11T09:42:44.719 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.719 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default size = 2 2023-10-11T09:42:44.720 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.720 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow primary affinity = true 2023-10-11T09:42:44.720 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow pg remap = true 2023-10-11T09:42:44.720 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on legacy crush tunables = false 2023-10-11T09:42:44.720 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on crush straw calc version zero = false 2023-10-11T09:42:44.721 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on no sortbitwise = false 2023-10-11T09:42:44.721 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on osd down out interval zero = false 2023-10-11T09:42:44.721 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on too few osds = false 2023-10-11T09:42:44.721 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-11T09:42:44.721 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-11T09:42:44.722 INFO:teuthology.orchestra.run.smithi133.stdout: mon_allow_pool_size_one = true 2023-10-11T09:42:44.722 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.722 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-11T09:42:44.722 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.722 INFO:teuthology.orchestra.run.smithi133.stdout: osd default data pool replay window = 5 2023-10-11T09:42:44.723 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.723 INFO:teuthology.orchestra.run.smithi133.stdout: mon allow pool delete = true 2023-10-11T09:42:44.723 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.723 INFO:teuthology.orchestra.run.smithi133.stdout: mon cluster log file level = debug 2023-10-11T09:42:44.723 INFO:teuthology.orchestra.run.smithi133.stdout: debug asserts on shutdown = true 2023-10-11T09:42:44.723 INFO:teuthology.orchestra.run.smithi133.stdout: mon health detail to clog = false 2023-10-11T09:42:44.724 INFO:teuthology.orchestra.run.smithi133.stdout: mon host = "172.21.15.78,172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790]" 2023-10-11T09:42:44.724 INFO:teuthology.orchestra.run.smithi133.stdout: fsid = dee99db9-7a72-406f-a1e2-917943f2b13d 2023-10-11T09:42:44.724 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.724 INFO:teuthology.orchestra.run.smithi133.stdout:[osd] 2023-10-11T09:42:44.724 INFO:teuthology.orchestra.run.smithi133.stdout: osd journal size = 100 2023-10-11T09:42:44.725 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.725 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub load threshold = 5.0 2023-10-11T09:42:44.725 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub max interval = 600 2023-10-11T09:42:44.725 INFO:teuthology.orchestra.run.smithi133.stdout: osd mclock profile = high_recovery_ops 2023-10-11T09:42:44.725 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.725 INFO:teuthology.orchestra.run.smithi133.stdout: osd recover clone overlap = true 2023-10-11T09:42:44.726 INFO:teuthology.orchestra.run.smithi133.stdout: osd recovery max chunk = 1048576 2023-10-11T09:42:44.726 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.726 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug shutdown = true 2023-10-11T09:42:44.726 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug op order = true 2023-10-11T09:42:44.726 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug verify stray on activate = true 2023-10-11T09:42:44.727 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.727 INFO:teuthology.orchestra.run.smithi133.stdout: osd open classes on start = true 2023-10-11T09:42:44.727 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug pg log writeout = true 2023-10-11T09:42:44.727 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.727 INFO:teuthology.orchestra.run.smithi133.stdout: osd deep scrub update digest min age = 30 2023-10-11T09:42:44.727 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.728 INFO:teuthology.orchestra.run.smithi133.stdout: osd map max advance = 10 2023-10-11T09:42:44.728 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.728 INFO:teuthology.orchestra.run.smithi133.stdout: journal zero on create = true 2023-10-11T09:42:44.728 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.728 INFO:teuthology.orchestra.run.smithi133.stdout: filestore ondisk finisher threads = 3 2023-10-11T09:42:44.729 INFO:teuthology.orchestra.run.smithi133.stdout: filestore apply finisher threads = 3 2023-10-11T09:42:44.729 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.729 INFO:teuthology.orchestra.run.smithi133.stdout: bdev debug aio = true 2023-10-11T09:42:44.729 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug misdirected ops = true 2023-10-11T09:42:44.729 INFO:teuthology.orchestra.run.smithi133.stdout: bdev async discard = True 2023-10-11T09:42:44.730 INFO:teuthology.orchestra.run.smithi133.stdout: bdev enable discard = True 2023-10-11T09:42:44.730 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore allocator = bitmap 2023-10-11T09:42:44.730 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore block size = 96636764160 2023-10-11T09:42:44.730 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore fsck on mount = True 2023-10-11T09:42:44.730 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluefs = 1/20 2023-10-11T09:42:44.731 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluestore = 1/20 2023-10-11T09:42:44.731 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:42:44.731 INFO:teuthology.orchestra.run.smithi133.stdout: debug osd = 20 2023-10-11T09:42:44.731 INFO:teuthology.orchestra.run.smithi133.stdout: debug rocksdb = 4/10 2023-10-11T09:42:44.731 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd backfillfull_ratio = 0.85 2023-10-11T09:42:44.731 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd full ratio = 0.9 2023-10-11T09:42:44.732 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd nearfull ratio = 0.8 2023-10-11T09:42:44.732 INFO:teuthology.orchestra.run.smithi133.stdout: osd failsafe full ratio = 0.95 2023-10-11T09:42:44.732 INFO:teuthology.orchestra.run.smithi133.stdout: osd objectstore = bluestore 2023-10-11T09:42:44.732 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2023-10-11T09:42:44.732 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.733 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr] 2023-10-11T09:42:44.733 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:42:44.733 INFO:teuthology.orchestra.run.smithi133.stdout: debug mgr = 20 2023-10-11T09:42:44.733 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2023-10-11T09:42:44.733 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2023-10-11T09:42:44.734 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:42:44.734 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:42:44.734 INFO:teuthology.orchestra.run.smithi133.stdout: mgr/telemetry/nag = false 2023-10-11T09:42:44.734 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.734 INFO:teuthology.orchestra.run.smithi133.stdout:[mon] 2023-10-11T09:42:44.734 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:42:44.735 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2023-10-11T09:42:44.735 INFO:teuthology.orchestra.run.smithi133.stdout: debug paxos = 20 2023-10-11T09:42:44.735 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2023-10-11T09:42:44.735 INFO:teuthology.orchestra.run.smithi133.stdout: mon data avail warn = 5 2023-10-11T09:42:44.735 INFO:teuthology.orchestra.run.smithi133.stdout: mon mgr mkfs grace = 240 2023-10-11T09:42:44.736 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:42:44.736 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd reporter subtree level = osd 2023-10-11T09:42:44.736 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd prime pg temp = true 2023-10-11T09:42:44.736 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:42:44.736 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.737 INFO:teuthology.orchestra.run.smithi133.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-11T09:42:44.737 INFO:teuthology.orchestra.run.smithi133.stdout: auth mon ticket ttl = 660 # 11m 2023-10-11T09:42:44.737 INFO:teuthology.orchestra.run.smithi133.stdout: auth service ticket ttl = 240 # 4m 2023-10-11T09:42:44.737 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.737 INFO:teuthology.orchestra.run.smithi133.stdout: # don't complain about insecure global_id in the test suite 2023-10-11T09:42:44.737 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-11T09:42:44.738 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-11T09:42:44.738 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.738 INFO:teuthology.orchestra.run.smithi133.stdout: # 1m isn't quite enough 2023-10-11T09:42:44.738 INFO:teuthology.orchestra.run.smithi133.stdout: mon_down_mkfs_grace = 2m 2023-10-11T09:42:44.738 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.739 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_filestore_osds = false 2023-10-11T09:42:44.739 INFO:teuthology.orchestra.run.smithi133.stdout: mon op complaint time = 120 2023-10-11T09:42:44.739 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.739 INFO:teuthology.orchestra.run.smithi133.stdout:[client] 2023-10-11T09:42:44.739 INFO:teuthology.orchestra.run.smithi133.stdout: rgw cache enabled = true 2023-10-11T09:42:44.740 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable ops log = true 2023-10-11T09:42:44.740 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable usage log = true 2023-10-11T09:42:44.740 INFO:teuthology.orchestra.run.smithi133.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-11T09:42:44.740 INFO:teuthology.orchestra.run.smithi133.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-11T09:42:44.740 INFO:teuthology.orchestra.run.smithi133.stdout: client mount timeout = 600 2023-10-11T09:42:44.741 INFO:teuthology.orchestra.run.smithi133.stdout: debug client = 20 2023-10-11T09:42:44.741 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:42:44.741 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = "" 2023-10-11T09:42:44.741 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2023-10-11T09:42:44.741 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2023-10-11T09:42:44.741 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.a] 2023-10-11T09:42:44.742 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.b] 2023-10-11T09:42:44.742 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.c] 2023-10-11T09:42:44.742 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.a] 2023-10-11T09:42:44.742 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.c] 2023-10-11T09:42:44.742 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.b] 2023-10-11T09:42:44.743 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.d] 2023-10-11T09:42:44.743 INFO:teuthology.orchestra.run.smithi133.stdout:[mds] 2023-10-11T09:42:44.743 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds = 20 2023-10-11T09:42:44.743 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds balancer = 20 2023-10-11T09:42:44.743 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:42:44.744 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug frag = True 2023-10-11T09:42:44.744 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug scatterstat = True 2023-10-11T09:42:44.744 INFO:teuthology.orchestra.run.smithi133.stdout: mds op complaint time = 180 2023-10-11T09:42:44.744 INFO:teuthology.orchestra.run.smithi133.stdout: mds verify scatter = False 2023-10-11T09:42:44.744 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2023-10-11T09:42:44.744 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2023-10-11T09:42:44.745 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2023-10-11T09:42:44.746 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-11T09:42:44.753 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-11T09:42:44.827 INFO:teuthology.orchestra.run.smithi078.stdout:[global] 2023-10-11T09:42:44.828 INFO:teuthology.orchestra.run.smithi078.stdout: chdir = "" 2023-10-11T09:42:44.828 INFO:teuthology.orchestra.run.smithi078.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-11T09:42:44.828 INFO:teuthology.orchestra.run.smithi078.stdout: auth supported = cephx 2023-10-11T09:42:44.828 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.829 INFO:teuthology.orchestra.run.smithi078.stdout: filestore xattr use omap = true 2023-10-11T09:42:44.829 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.829 INFO:teuthology.orchestra.run.smithi078.stdout: mon clock drift allowed = 1.000 2023-10-11T09:42:44.829 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.829 INFO:teuthology.orchestra.run.smithi078.stdout: osd crush chooseleaf type = 0 2023-10-11T09:42:44.829 INFO:teuthology.orchestra.run.smithi078.stdout: auth debug = true 2023-10-11T09:42:44.830 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.830 INFO:teuthology.orchestra.run.smithi078.stdout: ms die on old message = true 2023-10-11T09:42:44.830 INFO:teuthology.orchestra.run.smithi078.stdout: ms die on bug = true 2023-10-11T09:42:44.830 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.830 INFO:teuthology.orchestra.run.smithi078.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-11T09:42:44.831 INFO:teuthology.orchestra.run.smithi078.stdout: mon pg warn max object skew = 0 2023-10-11T09:42:44.831 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.831 INFO:teuthology.orchestra.run.smithi078.stdout: # disable pg_autoscaler by default for new pools 2023-10-11T09:42:44.831 INFO:teuthology.orchestra.run.smithi078.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-11T09:42:44.831 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.832 INFO:teuthology.orchestra.run.smithi078.stdout: osd pool default size = 2 2023-10-11T09:42:44.832 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.832 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd allow primary affinity = true 2023-10-11T09:42:44.832 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd allow pg remap = true 2023-10-11T09:42:44.832 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on legacy crush tunables = false 2023-10-11T09:42:44.833 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on crush straw calc version zero = false 2023-10-11T09:42:44.833 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on no sortbitwise = false 2023-10-11T09:42:44.833 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on osd down out interval zero = false 2023-10-11T09:42:44.833 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on too few osds = false 2023-10-11T09:42:44.833 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-11T09:42:44.834 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-11T09:42:44.834 INFO:teuthology.orchestra.run.smithi078.stdout: mon_allow_pool_size_one = true 2023-10-11T09:42:44.834 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.834 INFO:teuthology.orchestra.run.smithi078.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-11T09:42:44.834 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.835 INFO:teuthology.orchestra.run.smithi078.stdout: osd default data pool replay window = 5 2023-10-11T09:42:44.835 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.835 INFO:teuthology.orchestra.run.smithi078.stdout: mon allow pool delete = true 2023-10-11T09:42:44.835 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.835 INFO:teuthology.orchestra.run.smithi078.stdout: mon cluster log file level = debug 2023-10-11T09:42:44.836 INFO:teuthology.orchestra.run.smithi078.stdout: debug asserts on shutdown = true 2023-10-11T09:42:44.836 INFO:teuthology.orchestra.run.smithi078.stdout: mon health detail to clog = false 2023-10-11T09:42:44.836 INFO:teuthology.orchestra.run.smithi078.stdout: mon host = "172.21.15.78,172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790]" 2023-10-11T09:42:44.836 INFO:teuthology.orchestra.run.smithi078.stdout: fsid = dee99db9-7a72-406f-a1e2-917943f2b13d 2023-10-11T09:42:44.836 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.837 INFO:teuthology.orchestra.run.smithi078.stdout:[osd] 2023-10-11T09:42:44.837 INFO:teuthology.orchestra.run.smithi078.stdout: osd journal size = 100 2023-10-11T09:42:44.837 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.837 INFO:teuthology.orchestra.run.smithi078.stdout: osd scrub load threshold = 5.0 2023-10-11T09:42:44.837 INFO:teuthology.orchestra.run.smithi078.stdout: osd scrub max interval = 600 2023-10-11T09:42:44.838 INFO:teuthology.orchestra.run.smithi078.stdout: osd mclock profile = high_recovery_ops 2023-10-11T09:42:44.838 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.838 INFO:teuthology.orchestra.run.smithi078.stdout: osd recover clone overlap = true 2023-10-11T09:42:44.838 INFO:teuthology.orchestra.run.smithi078.stdout: osd recovery max chunk = 1048576 2023-10-11T09:42:44.838 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.839 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug shutdown = true 2023-10-11T09:42:44.839 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug op order = true 2023-10-11T09:42:44.839 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug verify stray on activate = true 2023-10-11T09:42:44.839 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.839 INFO:teuthology.orchestra.run.smithi078.stdout: osd open classes on start = true 2023-10-11T09:42:44.840 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug pg log writeout = true 2023-10-11T09:42:44.840 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.840 INFO:teuthology.orchestra.run.smithi078.stdout: osd deep scrub update digest min age = 30 2023-10-11T09:42:44.840 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.840 INFO:teuthology.orchestra.run.smithi078.stdout: osd map max advance = 10 2023-10-11T09:42:44.841 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.841 INFO:teuthology.orchestra.run.smithi078.stdout: journal zero on create = true 2023-10-11T09:42:44.841 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.841 INFO:teuthology.orchestra.run.smithi078.stdout: filestore ondisk finisher threads = 3 2023-10-11T09:42:44.841 INFO:teuthology.orchestra.run.smithi078.stdout: filestore apply finisher threads = 3 2023-10-11T09:42:44.842 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.842 INFO:teuthology.orchestra.run.smithi078.stdout: bdev debug aio = true 2023-10-11T09:42:44.842 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug misdirected ops = true 2023-10-11T09:42:44.842 INFO:teuthology.orchestra.run.smithi078.stdout: bdev async discard = True 2023-10-11T09:42:44.842 INFO:teuthology.orchestra.run.smithi078.stdout: bdev enable discard = True 2023-10-11T09:42:44.843 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore allocator = bitmap 2023-10-11T09:42:44.843 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore block size = 96636764160 2023-10-11T09:42:44.843 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore fsck on mount = True 2023-10-11T09:42:44.843 INFO:teuthology.orchestra.run.smithi078.stdout: debug bluefs = 1/20 2023-10-11T09:42:44.843 INFO:teuthology.orchestra.run.smithi078.stdout: debug bluestore = 1/20 2023-10-11T09:42:44.844 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:42:44.844 INFO:teuthology.orchestra.run.smithi078.stdout: debug osd = 20 2023-10-11T09:42:44.844 INFO:teuthology.orchestra.run.smithi078.stdout: debug rocksdb = 4/10 2023-10-11T09:42:44.844 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd backfillfull_ratio = 0.85 2023-10-11T09:42:44.844 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd full ratio = 0.9 2023-10-11T09:42:44.844 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd nearfull ratio = 0.8 2023-10-11T09:42:44.845 INFO:teuthology.orchestra.run.smithi078.stdout: osd failsafe full ratio = 0.95 2023-10-11T09:42:44.845 INFO:teuthology.orchestra.run.smithi078.stdout: osd objectstore = bluestore 2023-10-11T09:42:44.845 INFO:teuthology.orchestra.run.smithi078.stdout: osd op complaint time = 180 2023-10-11T09:42:44.845 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.845 INFO:teuthology.orchestra.run.smithi078.stdout:[mgr] 2023-10-11T09:42:44.846 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:42:44.846 INFO:teuthology.orchestra.run.smithi078.stdout: debug mgr = 20 2023-10-11T09:42:44.846 INFO:teuthology.orchestra.run.smithi078.stdout: debug mon = 20 2023-10-11T09:42:44.846 INFO:teuthology.orchestra.run.smithi078.stdout: debug auth = 20 2023-10-11T09:42:44.846 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:42:44.847 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:42:44.847 INFO:teuthology.orchestra.run.smithi078.stdout: mgr/telemetry/nag = false 2023-10-11T09:42:44.847 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.847 INFO:teuthology.orchestra.run.smithi078.stdout:[mon] 2023-10-11T09:42:44.847 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:42:44.848 INFO:teuthology.orchestra.run.smithi078.stdout: debug mon = 20 2023-10-11T09:42:44.848 INFO:teuthology.orchestra.run.smithi078.stdout: debug paxos = 20 2023-10-11T09:42:44.848 INFO:teuthology.orchestra.run.smithi078.stdout: debug auth = 20 2023-10-11T09:42:44.848 INFO:teuthology.orchestra.run.smithi078.stdout: mon data avail warn = 5 2023-10-11T09:42:44.848 INFO:teuthology.orchestra.run.smithi078.stdout: mon mgr mkfs grace = 240 2023-10-11T09:42:44.848 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:42:44.849 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd reporter subtree level = osd 2023-10-11T09:42:44.849 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd prime pg temp = true 2023-10-11T09:42:44.849 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:42:44.849 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.849 INFO:teuthology.orchestra.run.smithi078.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-11T09:42:44.850 INFO:teuthology.orchestra.run.smithi078.stdout: auth mon ticket ttl = 660 # 11m 2023-10-11T09:42:44.850 INFO:teuthology.orchestra.run.smithi078.stdout: auth service ticket ttl = 240 # 4m 2023-10-11T09:42:44.850 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.850 INFO:teuthology.orchestra.run.smithi078.stdout: # don't complain about insecure global_id in the test suite 2023-10-11T09:42:44.850 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-11T09:42:44.851 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-11T09:42:44.851 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.851 INFO:teuthology.orchestra.run.smithi078.stdout: # 1m isn't quite enough 2023-10-11T09:42:44.851 INFO:teuthology.orchestra.run.smithi078.stdout: mon_down_mkfs_grace = 2m 2023-10-11T09:42:44.851 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.852 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_filestore_osds = false 2023-10-11T09:42:44.852 INFO:teuthology.orchestra.run.smithi078.stdout: mon op complaint time = 120 2023-10-11T09:42:44.852 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:44.852 INFO:teuthology.orchestra.run.smithi078.stdout:[client] 2023-10-11T09:42:44.852 INFO:teuthology.orchestra.run.smithi078.stdout: rgw cache enabled = true 2023-10-11T09:42:44.853 INFO:teuthology.orchestra.run.smithi078.stdout: rgw enable ops log = true 2023-10-11T09:42:44.853 INFO:teuthology.orchestra.run.smithi078.stdout: rgw enable usage log = true 2023-10-11T09:42:44.853 INFO:teuthology.orchestra.run.smithi078.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-11T09:42:44.853 INFO:teuthology.orchestra.run.smithi078.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-11T09:42:44.853 INFO:teuthology.orchestra.run.smithi078.stdout: client mount timeout = 600 2023-10-11T09:42:44.853 INFO:teuthology.orchestra.run.smithi078.stdout: debug client = 20 2023-10-11T09:42:44.854 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:42:44.854 INFO:teuthology.orchestra.run.smithi078.stdout: pid file = "" 2023-10-11T09:42:44.854 INFO:teuthology.orchestra.run.smithi078.stdout: rados mon op timeout = 900 2023-10-11T09:42:44.854 INFO:teuthology.orchestra.run.smithi078.stdout: rados osd op timeout = 900 2023-10-11T09:42:44.854 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.a] 2023-10-11T09:42:44.855 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.b] 2023-10-11T09:42:44.855 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.c] 2023-10-11T09:42:44.855 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.a] 2023-10-11T09:42:44.855 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.c] 2023-10-11T09:42:44.855 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.b] 2023-10-11T09:42:44.856 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.d] 2023-10-11T09:42:44.856 INFO:teuthology.orchestra.run.smithi078.stdout:[mds] 2023-10-11T09:42:44.856 INFO:teuthology.orchestra.run.smithi078.stdout: debug mds = 20 2023-10-11T09:42:44.856 INFO:teuthology.orchestra.run.smithi078.stdout: debug mds balancer = 20 2023-10-11T09:42:44.856 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:42:44.857 INFO:teuthology.orchestra.run.smithi078.stdout: mds debug frag = True 2023-10-11T09:42:44.857 INFO:teuthology.orchestra.run.smithi078.stdout: mds debug scatterstat = False 2023-10-11T09:42:44.857 INFO:teuthology.orchestra.run.smithi078.stdout: mds op complaint time = 180 2023-10-11T09:42:44.857 INFO:teuthology.orchestra.run.smithi078.stdout: mds verify scatter = False 2023-10-11T09:42:44.857 INFO:teuthology.orchestra.run.smithi078.stdout: osd op complaint time = 180 2023-10-11T09:42:44.857 INFO:teuthology.orchestra.run.smithi078.stdout: rados mon op timeout = 900 2023-10-11T09:42:44.858 INFO:teuthology.orchestra.run.smithi078.stdout: rados osd op timeout = 900 2023-10-11T09:42:44.859 INFO:teuthology.orchestra.run.smithi133.stdout:[global] 2023-10-11T09:42:44.859 INFO:teuthology.orchestra.run.smithi133.stdout: chdir = "" 2023-10-11T09:42:44.860 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-11T09:42:44.860 INFO:teuthology.orchestra.run.smithi133.stdout: auth supported = cephx 2023-10-11T09:42:44.860 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.860 INFO:teuthology.orchestra.run.smithi133.stdout: filestore xattr use omap = true 2023-10-11T09:42:44.861 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.861 INFO:teuthology.orchestra.run.smithi133.stdout: mon clock drift allowed = 1.000 2023-10-11T09:42:44.861 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.861 INFO:teuthology.orchestra.run.smithi133.stdout: osd crush chooseleaf type = 0 2023-10-11T09:42:44.861 INFO:teuthology.orchestra.run.smithi133.stdout: auth debug = true 2023-10-11T09:42:44.862 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.862 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on old message = true 2023-10-11T09:42:44.862 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on bug = true 2023-10-11T09:42:44.862 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.862 INFO:teuthology.orchestra.run.smithi133.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-11T09:42:44.863 INFO:teuthology.orchestra.run.smithi133.stdout: mon pg warn max object skew = 0 2023-10-11T09:42:44.863 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.863 INFO:teuthology.orchestra.run.smithi133.stdout: # disable pg_autoscaler by default for new pools 2023-10-11T09:42:44.863 INFO:teuthology.orchestra.run.smithi133.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-11T09:42:44.863 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.864 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default size = 2 2023-10-11T09:42:44.864 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.864 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow primary affinity = true 2023-10-11T09:42:44.864 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow pg remap = true 2023-10-11T09:42:44.864 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on legacy crush tunables = false 2023-10-11T09:42:44.864 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on crush straw calc version zero = false 2023-10-11T09:42:44.865 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on no sortbitwise = false 2023-10-11T09:42:44.865 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on osd down out interval zero = false 2023-10-11T09:42:44.865 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on too few osds = false 2023-10-11T09:42:44.865 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-11T09:42:44.865 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-11T09:42:44.866 INFO:teuthology.orchestra.run.smithi133.stdout: mon_allow_pool_size_one = true 2023-10-11T09:42:44.866 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.866 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-11T09:42:44.866 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.866 INFO:teuthology.orchestra.run.smithi133.stdout: osd default data pool replay window = 5 2023-10-11T09:42:44.867 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.867 INFO:teuthology.orchestra.run.smithi133.stdout: mon allow pool delete = true 2023-10-11T09:42:44.867 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.867 INFO:teuthology.orchestra.run.smithi133.stdout: mon cluster log file level = debug 2023-10-11T09:42:44.867 INFO:teuthology.orchestra.run.smithi133.stdout: debug asserts on shutdown = true 2023-10-11T09:42:44.868 INFO:teuthology.orchestra.run.smithi133.stdout: mon health detail to clog = false 2023-10-11T09:42:44.868 INFO:teuthology.orchestra.run.smithi133.stdout: mon host = "172.21.15.78,172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790]" 2023-10-11T09:42:44.868 INFO:teuthology.orchestra.run.smithi133.stdout: fsid = dee99db9-7a72-406f-a1e2-917943f2b13d 2023-10-11T09:42:44.868 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.868 INFO:teuthology.orchestra.run.smithi133.stdout:[osd] 2023-10-11T09:42:44.868 INFO:teuthology.orchestra.run.smithi133.stdout: osd journal size = 100 2023-10-11T09:42:44.869 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.869 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub load threshold = 5.0 2023-10-11T09:42:44.869 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub max interval = 600 2023-10-11T09:42:44.869 INFO:teuthology.orchestra.run.smithi133.stdout: osd mclock profile = high_recovery_ops 2023-10-11T09:42:44.869 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.870 INFO:teuthology.orchestra.run.smithi133.stdout: osd recover clone overlap = true 2023-10-11T09:42:44.870 INFO:teuthology.orchestra.run.smithi133.stdout: osd recovery max chunk = 1048576 2023-10-11T09:42:44.870 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.870 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug shutdown = true 2023-10-11T09:42:44.870 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug op order = true 2023-10-11T09:42:44.871 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug verify stray on activate = true 2023-10-11T09:42:44.871 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.871 INFO:teuthology.orchestra.run.smithi133.stdout: osd open classes on start = true 2023-10-11T09:42:44.871 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug pg log writeout = true 2023-10-11T09:42:44.871 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.872 INFO:teuthology.orchestra.run.smithi133.stdout: osd deep scrub update digest min age = 30 2023-10-11T09:42:44.872 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.872 INFO:teuthology.orchestra.run.smithi133.stdout: osd map max advance = 10 2023-10-11T09:42:44.872 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.872 INFO:teuthology.orchestra.run.smithi133.stdout: journal zero on create = true 2023-10-11T09:42:44.873 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.873 INFO:teuthology.orchestra.run.smithi133.stdout: filestore ondisk finisher threads = 3 2023-10-11T09:42:44.873 INFO:teuthology.orchestra.run.smithi133.stdout: filestore apply finisher threads = 3 2023-10-11T09:42:44.873 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.873 INFO:teuthology.orchestra.run.smithi133.stdout: bdev debug aio = true 2023-10-11T09:42:44.873 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug misdirected ops = true 2023-10-11T09:42:44.874 INFO:teuthology.orchestra.run.smithi133.stdout: bdev async discard = True 2023-10-11T09:42:44.874 INFO:teuthology.orchestra.run.smithi133.stdout: bdev enable discard = True 2023-10-11T09:42:44.874 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore allocator = bitmap 2023-10-11T09:42:44.874 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore block size = 96636764160 2023-10-11T09:42:44.874 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore fsck on mount = True 2023-10-11T09:42:44.875 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluefs = 1/20 2023-10-11T09:42:44.875 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluestore = 1/20 2023-10-11T09:42:44.875 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:42:44.875 INFO:teuthology.orchestra.run.smithi133.stdout: debug osd = 20 2023-10-11T09:42:44.875 INFO:teuthology.orchestra.run.smithi133.stdout: debug rocksdb = 4/10 2023-10-11T09:42:44.876 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd backfillfull_ratio = 0.85 2023-10-11T09:42:44.876 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd full ratio = 0.9 2023-10-11T09:42:44.876 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd nearfull ratio = 0.8 2023-10-11T09:42:44.876 INFO:teuthology.orchestra.run.smithi133.stdout: osd failsafe full ratio = 0.95 2023-10-11T09:42:44.876 INFO:teuthology.orchestra.run.smithi133.stdout: osd objectstore = bluestore 2023-10-11T09:42:44.876 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2023-10-11T09:42:44.877 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.877 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr] 2023-10-11T09:42:44.877 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:42:44.877 INFO:teuthology.orchestra.run.smithi133.stdout: debug mgr = 20 2023-10-11T09:42:44.877 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2023-10-11T09:42:44.878 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2023-10-11T09:42:44.878 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:42:44.878 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:42:44.879 INFO:teuthology.orchestra.run.smithi133.stdout: mgr/telemetry/nag = false 2023-10-11T09:42:44.879 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.879 INFO:teuthology.orchestra.run.smithi133.stdout:[mon] 2023-10-11T09:42:44.879 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:42:44.879 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2023-10-11T09:42:44.879 INFO:teuthology.orchestra.run.smithi133.stdout: debug paxos = 20 2023-10-11T09:42:44.880 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2023-10-11T09:42:44.880 INFO:teuthology.orchestra.run.smithi133.stdout: mon data avail warn = 5 2023-10-11T09:42:44.880 INFO:teuthology.orchestra.run.smithi133.stdout: mon mgr mkfs grace = 240 2023-10-11T09:42:44.880 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:42:44.881 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd reporter subtree level = osd 2023-10-11T09:42:44.881 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd prime pg temp = true 2023-10-11T09:42:44.881 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:42:44.881 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.882 INFO:teuthology.orchestra.run.smithi133.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-11T09:42:44.882 INFO:teuthology.orchestra.run.smithi133.stdout: auth mon ticket ttl = 660 # 11m 2023-10-11T09:42:44.882 INFO:teuthology.orchestra.run.smithi133.stdout: auth service ticket ttl = 240 # 4m 2023-10-11T09:42:44.882 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.883 INFO:teuthology.orchestra.run.smithi133.stdout: # don't complain about insecure global_id in the test suite 2023-10-11T09:42:44.883 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-11T09:42:44.883 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-11T09:42:44.883 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.884 INFO:teuthology.orchestra.run.smithi133.stdout: # 1m isn't quite enough 2023-10-11T09:42:44.884 INFO:teuthology.orchestra.run.smithi133.stdout: mon_down_mkfs_grace = 2m 2023-10-11T09:42:44.884 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.884 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_filestore_osds = false 2023-10-11T09:42:44.884 INFO:teuthology.orchestra.run.smithi133.stdout: mon op complaint time = 120 2023-10-11T09:42:44.885 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:42:44.885 INFO:teuthology.orchestra.run.smithi133.stdout:[client] 2023-10-11T09:42:44.885 INFO:teuthology.orchestra.run.smithi133.stdout: rgw cache enabled = true 2023-10-11T09:42:44.885 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable ops log = true 2023-10-11T09:42:44.885 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable usage log = true 2023-10-11T09:42:44.886 INFO:teuthology.orchestra.run.smithi133.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-11T09:42:44.886 INFO:teuthology.orchestra.run.smithi133.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-11T09:42:44.886 INFO:teuthology.orchestra.run.smithi133.stdout: client mount timeout = 600 2023-10-11T09:42:44.886 INFO:teuthology.orchestra.run.smithi133.stdout: debug client = 20 2023-10-11T09:42:44.886 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:42:44.887 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = "" 2023-10-11T09:42:44.887 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2023-10-11T09:42:44.887 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2023-10-11T09:42:44.887 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.a] 2023-10-11T09:42:44.888 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.b] 2023-10-11T09:42:44.888 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.c] 2023-10-11T09:42:44.888 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.a] 2023-10-11T09:42:44.889 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.c] 2023-10-11T09:42:44.889 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.b] 2023-10-11T09:42:44.889 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.d] 2023-10-11T09:42:44.889 INFO:teuthology.orchestra.run.smithi133.stdout:[mds] 2023-10-11T09:42:44.889 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds = 20 2023-10-11T09:42:44.890 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds balancer = 20 2023-10-11T09:42:44.890 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:42:44.890 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug frag = True 2023-10-11T09:42:44.891 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug scatterstat = False 2023-10-11T09:42:44.891 INFO:teuthology.orchestra.run.smithi133.stdout: mds op complaint time = 180 2023-10-11T09:42:44.891 INFO:teuthology.orchestra.run.smithi133.stdout: mds verify scatter = False 2023-10-11T09:42:44.892 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2023-10-11T09:42:44.892 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2023-10-11T09:42:44.892 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2023-10-11T09:42:44.892 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph purge cephfs_metadata --yes-i-really-really-mean-it 2023-10-11T09:42:44.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.960+0000 7fc5c3acf140 1 Processor -- start 2023-10-11T09:42:44.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.960+0000 7fc5c3acf140 1 -- start start 2023-10-11T09:42:44.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.960+0000 7fc5c3acf140 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f43efb66b0 0x55f43f08b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:44.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.960+0000 7fc5c3acf140 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f43efb6a80 0x55f43efe5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:44.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.960+0000 7fc5c3acf140 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f43efead00 0x55f43efe61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:44.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.960+0000 7fc5c3acf140 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f43ef07250 con 0x55f43efb6a80 2023-10-11T09:42:44.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.960+0000 7fc5c3acf140 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f43ef06ef0 con 0x55f43efb66b0 2023-10-11T09:42:44.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.960+0000 7fc5c3acf140 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f43eedd240 con 0x55f43efead00 2023-10-11T09:42:44.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.960+0000 7fc5c0e03700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f43efb6a80 0x55f43efe5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:44.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.960+0000 7fc5c0e03700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f43efb6a80 0x55f43efe5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57782/0 (socket says 172.21.15.78:57782) 2023-10-11T09:42:44.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.960+0000 7fc5c0e03700 1 -- 172.21.15.78:0/2530384696 learned_addr learned my addr 172.21.15.78:0/2530384696 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:44.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.960+0000 7fc5c1604700 1 --2- 172.21.15.78:0/2530384696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f43efb66b0 0x55f43f08b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:44.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.960+0000 7fc5c1e05700 1 --2- 172.21.15.78:0/2530384696 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f43efead00 0x55f43efe61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:44.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.960+0000 7fc5c0e03700 1 -- 172.21.15.78:0/2530384696 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f43efead00 msgr2=0x55f43efe61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:44.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.960+0000 7fc5c0e03700 1 --2- 172.21.15.78:0/2530384696 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f43efead00 0x55f43efe61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:44.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.960+0000 7fc5c0e03700 1 -- 172.21.15.78:0/2530384696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f43efb66b0 msgr2=0x55f43f08b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:44.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.960+0000 7fc5c0e03700 1 --2- 172.21.15.78:0/2530384696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f43efb66b0 0x55f43f08b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:44.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.960+0000 7fc5c0e03700 1 -- 172.21.15.78:0/2530384696 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ac00a100 con 0x55f43efb6a80 2023-10-11T09:42:44.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.960+0000 7fc5c1604700 1 --2- 172.21.15.78:0/2530384696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f43efb66b0 0x55f43f08b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:42:44.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.960+0000 7fc5c0e03700 1 --2- 172.21.15.78:0/2530384696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f43efb6a80 0x55f43efe5cb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac005f10 tx=0x7fc5ac00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afbceace78c64ca5 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:44.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.964+0000 7fc5bb7fe700 1 -- 172.21.15.78:0/2530384696 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac00f590 con 0x55f43efb6a80 2023-10-11T09:42:44.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.964+0000 7fc5bb7fe700 1 -- 172.21.15.78:0/2530384696 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5ac00f710 con 0x55f43efb6a80 2023-10-11T09:42:44.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.964+0000 7fc5bb7fe700 1 -- 172.21.15.78:0/2530384696 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac00fca0 con 0x55f43efb6a80 2023-10-11T09:42:44.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.964+0000 7fc5c3acf140 1 -- 172.21.15.78:0/2530384696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f43efb6a80 msgr2=0x55f43efe5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:44.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.964+0000 7fc5c3acf140 1 --2- 172.21.15.78:0/2530384696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f43efb6a80 0x55f43efe5cb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac005f10 tx=0x7fc5ac00f000 comp rx=0 tx=0).stop 2023-10-11T09:42:44.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.964+0000 7fc5c3acf140 1 -- 172.21.15.78:0/2530384696 shutdown_connections 2023-10-11T09:42:44.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.964+0000 7fc5c3acf140 1 --2- 172.21.15.78:0/2530384696 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f43efead00 0x55f43efe61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:44.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.964+0000 7fc5c3acf140 1 --2- 172.21.15.78:0/2530384696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f43efb6a80 0x55f43efe5cb0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:44.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.964+0000 7fc5c3acf140 1 --2- 172.21.15.78:0/2530384696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f43efb66b0 0x55f43f08b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:44.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.964+0000 7fc5c3acf140 1 -- 172.21.15.78:0/2530384696 >> 172.21.15.78:0/2530384696 conn(0x55f43efbcbe0 msgr2=0x55f43f08e160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:44.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.964+0000 7fc5c3acf140 1 -- 172.21.15.78:0/2530384696 shutdown_connections 2023-10-11T09:42:44.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.964+0000 7fc5c3acf140 1 -- 172.21.15.78:0/2530384696 wait complete. 2023-10-11T09:42:44.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.964+0000 7fc5c3acf140 1 Processor -- start 2023-10-11T09:42:44.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.964+0000 7fc5c3acf140 1 -- start start 2023-10-11T09:42:44.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.964+0000 7fc5c3acf140 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f43efb66b0 0x55f43f089360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:44.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.964+0000 7fc5c3acf140 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f43efb6a80 0x55f43effcec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:44.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.964+0000 7fc5c3acf140 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f43efead00 0x55f43f087030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:44.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.964+0000 7fc5c3acf140 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f43f089990 con 0x55f43efead00 2023-10-11T09:42:44.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.964+0000 7fc5c3acf140 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f43f089c30 con 0x55f43efb66b0 2023-10-11T09:42:44.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.964+0000 7fc5c3acf140 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f43f089ed0 con 0x55f43efb6a80 2023-10-11T09:42:44.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.964+0000 7fc5c1e05700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f43efead00 0x55f43f087030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:44.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.964+0000 7fc5c1e05700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f43efead00 0x55f43f087030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57796/0 (socket says 172.21.15.78:57796) 2023-10-11T09:42:44.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.964+0000 7fc5c1e05700 1 -- 172.21.15.78:0/401155673 learned_addr learned my addr 172.21.15.78:0/401155673 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:44.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.964+0000 7fc5c1604700 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f43efb66b0 0x55f43f089360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:44.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.964+0000 7fc5c0e03700 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f43efb6a80 0x55f43effcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:44.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.968+0000 7fc5c1e05700 1 -- 172.21.15.78:0/401155673 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f43efb6a80 msgr2=0x55f43effcec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:44.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.968+0000 7fc5c1e05700 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f43efb6a80 0x55f43effcec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:44.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.968+0000 7fc5c1e05700 1 -- 172.21.15.78:0/401155673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f43efb66b0 msgr2=0x55f43f089360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:44.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.968+0000 7fc5c1e05700 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f43efb66b0 0x55f43f089360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:44.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.968+0000 7fc5c1e05700 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f43ef0f610 con 0x55f43efead00 2023-10-11T09:42:44.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.968+0000 7fc5c0e03700 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f43efb6a80 0x55f43effcec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:42:44.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.968+0000 7fc5c1604700 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f43efb66b0 0x55f43f089360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:42:44.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.968+0000 7fc5c1e05700 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f43efead00 0x55f43f087030 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc00bd90 tx=0x7fc5bc0076d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:44.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.968+0000 7fc5ba7fc700 1 -- 172.21.15.78:0/401155673 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc010070 con 0x55f43efead00 2023-10-11T09:42:44.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.968+0000 7fc5ba7fc700 1 -- 172.21.15.78:0/401155673 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5bc00be70 con 0x55f43efead00 2023-10-11T09:42:44.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.968+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f43eef5470 con 0x55f43efead00 2023-10-11T09:42:44.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.968+0000 7fc5ba7fc700 1 -- 172.21.15.78:0/401155673 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc015420 con 0x55f43efead00 2023-10-11T09:42:44.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.968+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f43eee88b0 con 0x55f43efead00 2023-10-11T09:42:44.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.972+0000 7fc5ba7fc700 1 -- 172.21.15.78:0/401155673 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fc5bc01e440 con 0x55f43efead00 2023-10-11T09:42:44.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.972+0000 7fc5ba7fc700 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc5a8064800 0x7fc5a8066cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:44.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.972+0000 7fc5ba7fc700 1 -- 172.21.15.78:0/401155673 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6581+0+0 (secure 0 0 0) 0x7fc5bc081d80 con 0x55f43efead00 2023-10-11T09:42:44.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.972+0000 7fc5c1604700 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc5a8064800 0x7fc5a8066cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:44.980 INFO:teuthology.orchestra.run.smithi078.stdout:Warning: using slow linear search 2023-10-11T09:42:44.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.972+0000 7fc5c3acf140 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55f43f08a8b0 0x55f43f120a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:44.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.972+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:1 5.0 5:00000000:%01:::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55f43f120f60 con 0x55f43f08a8b0 2023-10-11T09:42:44.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.972+0000 7fc5c0e03700 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55f43f08a8b0 0x55f43f120a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:44.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.972+0000 7fc5c1604700 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc5a8064800 0x7fc5a8066cb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4000c00 tx=0x7fc5b4001740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:44.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.972+0000 7fc5c0e03700 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55f43f08a8b0 0x55f43f120a20 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:44.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.976+0000 7fc5c0e03700 1 -- 172.21.15.78:0/401155673 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(1 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc5ac00fbd0 con 0x55f43f08a8b0 2023-10-11T09:42:44.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.976+0000 7fc5c0e03700 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x7fc5ac010420 0x7fc5ac010850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:44.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.976+0000 7fc5c0e03700 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:2 5.10 5:08000000:%01:::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55f43f120f60 con 0x7fc5ac010420 2023-10-11T09:42:44.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.976+0000 7fc5c1604700 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x7fc5ac010420 0x7fc5ac010850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:44.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.976+0000 7fc5c1604700 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x7fc5ac010420 0x7fc5ac010850 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:44.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.976+0000 7fc5c1604700 1 -- 172.21.15.78:0/401155673 <== osd.7 v2:172.21.15.133:6824/35901 1 ==== osd_op_reply(2 [pgnls start_epoch 43 out=177b] v42'10 uv10 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x7fc5b4011760 con 0x7fc5ac010420 2023-10-11T09:42:44.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.976+0000 7fc5c1604700 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x7fc5b400f370 0x7fc5b400f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:44.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.976+0000 7fc5c1604700 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:3 5.8 5:10000000:%01:::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55f43f120f60 con 0x7fc5b400f370 2023-10-11T09:42:44.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.980+0000 7fc5c0e03700 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x7fc5b400f370 0x7fc5b400f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:44.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.980+0000 7fc5c0e03700 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x7fc5b400f370 0x7fc5b400f7a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:44.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.980+0000 7fc5c0e03700 1 -- 172.21.15.78:0/401155673 <== osd.2 v2:172.21.15.78:6808/35732 1 ==== osd_op_reply(3 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc5ac00fbd0 con 0x7fc5b400f370 2023-10-11T09:42:44.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.980+0000 7fc5c0e03700 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x7fc5ac0244a0 0x7fc5ac0223c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:44.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.980+0000 7fc5c0e03700 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:4 5.18 5:18000000:%01:::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55f43f120f60 con 0x7fc5ac0244a0 2023-10-11T09:42:44.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.980+0000 7fc5c1e05700 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x7fc5ac0244a0 0x7fc5ac0223c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:44.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.980+0000 7fc5c1e05700 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x7fc5ac0244a0 0x7fc5ac0223c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:44.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.984+0000 7fc5c1e05700 1 -- 172.21.15.78:0/401155673 <== osd.4 v2:172.21.15.133:6808/35898 1 ==== osd_op_reply(4 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc5bc051210 con 0x7fc5ac0244a0 2023-10-11T09:42:44.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.984+0000 7fc5c1e05700 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:5 5.4 5:20000000:%01:::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55f43f120f60 con 0x7fc5ac010420 2023-10-11T09:42:44.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.984+0000 7fc5c1604700 1 -- 172.21.15.78:0/401155673 <== osd.7 v2:172.21.15.133:6824/35901 2 ==== osd_op_reply(5 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc5b4011760 con 0x7fc5ac010420 2023-10-11T09:42:44.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.984+0000 7fc5c1604700 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:6 5.14 5:28000000:%01:::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55f43f120f60 con 0x55f43f08a8b0 2023-10-11T09:42:44.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.984+0000 7fc5c0e03700 1 -- 172.21.15.78:0/401155673 <== osd.3 v2:172.21.15.78:6824/35738 2 ==== osd_op_reply(6 [pgnls start_epoch 43 out=149b] v42'33 uv33 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x7fc5ac00fbd0 con 0x55f43f08a8b0 2023-10-11T09:42:44.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.984+0000 7fc5c0e03700 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x7fc5ac035040 0x7fc5ac035410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:44.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.984+0000 7fc5c0e03700 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:7 5.c 5:30000000:%01:::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55f43f120f60 con 0x7fc5ac035040 2023-10-11T09:42:44.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.984+0000 7fc5c1604700 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x7fc5ac035040 0x7fc5ac035410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:44.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.988+0000 7fc5c1604700 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x7fc5ac035040 0x7fc5ac035410 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:44.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.992+0000 7fc5c1604700 1 -- 172.21.15.78:0/401155673 <== osd.1 v2:172.21.15.78:6809/35734 1 ==== osd_op_reply(7 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc5b4011760 con 0x7fc5ac035040 2023-10-11T09:42:44.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.992+0000 7fc5c1604700 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:8 5.1c 5:38000000:%01:::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55f43f120f60 con 0x7fc5ac0244a0 2023-10-11T09:42:44.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.992+0000 7fc5c1e05700 1 -- 172.21.15.78:0/401155673 <== osd.4 v2:172.21.15.133:6808/35898 2 ==== osd_op_reply(8 [pgnls start_epoch 43 out=73b] v36'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fc5bc051210 con 0x7fc5ac0244a0 2023-10-11T09:42:44.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.992+0000 7fc5c1e05700 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x7fc5bc01e070 0x7fc5bc0579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:44.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.992+0000 7fc5c1e05700 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:9 5.2 5:40000000:%01:::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55f43f120f60 con 0x7fc5bc01e070 2023-10-11T09:42:44.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.992+0000 7fc5c0e03700 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x7fc5bc01e070 0x7fc5bc0579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:44.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:44.996+0000 7fc5c0e03700 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x7fc5bc01e070 0x7fc5bc0579e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:45.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.004+0000 7fc5c0e03700 1 -- 172.21.15.78:0/401155673 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== osd_op_reply(9 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc5ac00fbd0 con 0x7fc5bc01e070 2023-10-11T09:42:45.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.004+0000 7fc5c0e03700 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:10 5.12 5:48000000:%01:::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x7fc5ac042040 con 0x7fc5ac035040 2023-10-11T09:42:45.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.004+0000 7fc5c1604700 1 -- 172.21.15.78:0/401155673 <== osd.1 v2:172.21.15.78:6809/35734 2 ==== osd_op_reply(10 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc5b4011760 con 0x7fc5ac035040 2023-10-11T09:42:45.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.004+0000 7fc5c1604700 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:11 5.a 5:50000000:%01:::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x7fc5ac042040 con 0x7fc5b400f370 2023-10-11T09:42:45.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.004+0000 7fc5c0e03700 1 -- 172.21.15.78:0/401155673 <== osd.2 v2:172.21.15.78:6808/35732 2 ==== osd_op_reply(11 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc5ac00fbd0 con 0x7fc5b400f370 2023-10-11T09:42:45.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.004+0000 7fc5c0e03700 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:12 5.1a 5:58000000:%01:::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x7fc5ac042040 con 0x7fc5ac010420 2023-10-11T09:42:45.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.004+0000 7fc5c1604700 1 -- 172.21.15.78:0/401155673 <== osd.7 v2:172.21.15.133:6824/35901 3 ==== osd_op_reply(12 [pgnls start_epoch 43 out=49b] v36'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc5b4011760 con 0x7fc5ac010420 2023-10-11T09:42:45.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.004+0000 7fc5c1604700 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:13 5.6 5:60000000:%01:::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x7fc5ac042040 con 0x7fc5b400f370 2023-10-11T09:42:45.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.008+0000 7fc5c0e03700 1 -- 172.21.15.78:0/401155673 <== osd.2 v2:172.21.15.78:6808/35732 3 ==== osd_op_reply(13 [pgnls start_epoch 43 out=77b] v42'8 uv8 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x7fc5ac00fbd0 con 0x7fc5b400f370 2023-10-11T09:42:45.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.008+0000 7fc5c0e03700 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x7fc5ac0343a0 0x7fc5ac0347f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:45.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.008+0000 7fc5c0e03700 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:14 5.16 5:68000000:%01:::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x7fc5ac042040 con 0x7fc5ac0343a0 2023-10-11T09:42:45.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.008+0000 7fc5c1e05700 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x7fc5ac0343a0 0x7fc5ac0347f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:45.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.008+0000 7fc5c1e05700 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x7fc5ac0343a0 0x7fc5ac0347f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:45.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.012+0000 7fc5c1e05700 1 -- 172.21.15.78:0/401155673 <== osd.5 v2:172.21.15.133:6816/35899 1 ==== osd_op_reply(14 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc5bc051210 con 0x7fc5ac0343a0 2023-10-11T09:42:45.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.012+0000 7fc5c1e05700 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:15 5.e 5:70000000:%01:::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x7fc5ac042040 con 0x7fc5ac0244a0 2023-10-11T09:42:45.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.016+0000 7fc5c1e05700 1 -- 172.21.15.78:0/401155673 <== osd.4 v2:172.21.15.133:6808/35898 3 ==== osd_op_reply(15 [pgnls start_epoch 43 out=73b] v36'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fc5bc051210 con 0x7fc5ac0244a0 2023-10-11T09:42:45.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.016+0000 7fc5c1e05700 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x7fc5bc009040 0x7fc5bc0668a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:45.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.016+0000 7fc5c1e05700 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:16 5.1e 5:78000000:%01:::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x7fc5ac042040 con 0x7fc5bc009040 2023-10-11T09:42:45.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.016+0000 7fc5c1604700 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x7fc5bc009040 0x7fc5bc0668a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:45.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.016+0000 7fc5c1604700 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x7fc5bc009040 0x7fc5bc0668a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:45.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.024+0000 7fc5c1604700 1 -- 172.21.15.78:0/401155673 <== osd.0 v2:172.21.15.78:6800/35735 1 ==== osd_op_reply(16 [pgnls start_epoch 43 out=49b] v36'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc5b4011760 con 0x7fc5bc009040 2023-10-11T09:42:45.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.024+0000 7fc5c1604700 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:17 5.1 5:80000000:%01:::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x7fc5ac042040 con 0x7fc5ac0244a0 2023-10-11T09:42:45.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.024+0000 7fc5c1e05700 1 -- 172.21.15.78:0/401155673 <== osd.4 v2:172.21.15.133:6808/35898 4 ==== osd_op_reply(17 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc5bc070040 con 0x7fc5ac0244a0 2023-10-11T09:42:45.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.024+0000 7fc5c1e05700 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:18 5.11 5:88000000:%01:::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x7fc5ac042040 con 0x7fc5bc01e070 2023-10-11T09:42:45.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.024+0000 7fc5c0e03700 1 -- 172.21.15.78:0/401155673 <== osd.6 v2:172.21.15.133:6800/35900 2 ==== osd_op_reply(18 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc5ac00fbd0 con 0x7fc5bc01e070 2023-10-11T09:42:45.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.024+0000 7fc5c0e03700 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:19 5.9 5:90000000:%01:::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x7fc5ac042040 con 0x7fc5ac010420 2023-10-11T09:42:45.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.028+0000 7fc5c1604700 1 -- 172.21.15.78:0/401155673 <== osd.7 v2:172.21.15.133:6824/35901 4 ==== osd_op_reply(19 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc5b4011760 con 0x7fc5ac010420 2023-10-11T09:42:45.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.028+0000 7fc5c1604700 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:20 5.19 5:98000000:%01:::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x7fc5ac042040 con 0x7fc5ac035040 2023-10-11T09:42:45.030 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.028+0000 7fc5c1604700 1 -- 172.21.15.78:0/401155673 <== osd.1 v2:172.21.15.78:6809/35734 3 ==== osd_op_reply(20 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc5b4011760 con 0x7fc5ac035040 2023-10-11T09:42:45.030 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.028+0000 7fc5c1604700 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:21 5.5 5:a0000000:%01:::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x7fc5ac042040 con 0x7fc5bc009040 2023-10-11T09:42:45.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.028+0000 7fc5c1604700 1 -- 172.21.15.78:0/401155673 <== osd.0 v2:172.21.15.78:6800/35735 2 ==== osd_op_reply(21 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc5b4011760 con 0x7fc5bc009040 2023-10-11T09:42:45.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.028+0000 7fc5c1604700 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:22 5.15 5:a8000000:%01:::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x7fc5ac042040 con 0x7fc5ac0343a0 2023-10-11T09:42:45.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.028+0000 7fc5c1e05700 1 -- 172.21.15.78:0/401155673 <== osd.5 v2:172.21.15.133:6816/35899 2 ==== osd_op_reply(22 [pgnls start_epoch 43 out=97b] v36'3 uv3 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x7fc5bc070040 con 0x7fc5ac0343a0 2023-10-11T09:42:45.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.028+0000 7fc5c1e05700 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:23 5.d 5:b0000000:%01:::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x7fc5ac042040 con 0x7fc5b400f370 2023-10-11T09:42:45.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.028+0000 7fc5c0e03700 1 -- 172.21.15.78:0/401155673 <== osd.2 v2:172.21.15.78:6808/35732 4 ==== osd_op_reply(23 [pgnls start_epoch 43 out=74b] v36'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fc5ac00fbd0 con 0x7fc5b400f370 2023-10-11T09:42:45.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.028+0000 7fc5c0e03700 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:24 5.1d 5:b8000000:%01:::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x7fc5ac042040 con 0x7fc5ac035040 2023-10-11T09:42:45.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.032+0000 7fc5c1604700 1 -- 172.21.15.78:0/401155673 <== osd.1 v2:172.21.15.78:6809/35734 4 ==== osd_op_reply(24 [pgnls start_epoch 43 out=73b] v36'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fc5b4011760 con 0x7fc5ac035040 2023-10-11T09:42:45.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.032+0000 7fc5c1604700 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:25 5.3 5:c0000000:%01:::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x7fc5ac042040 con 0x7fc5bc009040 2023-10-11T09:42:45.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.032+0000 7fc5c1604700 1 -- 172.21.15.78:0/401155673 <== osd.0 v2:172.21.15.78:6800/35735 3 ==== osd_op_reply(25 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc5b4011760 con 0x7fc5bc009040 2023-10-11T09:42:45.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.032+0000 7fc5c1604700 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:26 5.13 5:c8000000:%01:::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x7fc5ac042040 con 0x55f43f08a8b0 2023-10-11T09:42:45.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.032+0000 7fc5c0e03700 1 -- 172.21.15.78:0/401155673 <== osd.3 v2:172.21.15.78:6824/35738 3 ==== osd_op_reply(26 [pgnls start_epoch 43 out=98b] v42'3 uv3 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x7fc5ac00fbd0 con 0x55f43f08a8b0 2023-10-11T09:42:45.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.032+0000 7fc5c0e03700 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:27 5.b 5:d0000000:%01:::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x7fc5ac042040 con 0x7fc5b400f370 2023-10-11T09:42:45.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.032+0000 7fc5c0e03700 1 -- 172.21.15.78:0/401155673 <== osd.2 v2:172.21.15.78:6808/35732 5 ==== osd_op_reply(27 [pgnls start_epoch 43 out=76b] v42'3 uv3 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x7fc5ac00fbd0 con 0x7fc5b400f370 2023-10-11T09:42:45.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.032+0000 7fc5c0e03700 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:28 5.1b 5:d8000000:%01:::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x7fc5ac042040 con 0x7fc5ac0343a0 2023-10-11T09:42:45.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.036+0000 7fc5c1e05700 1 -- 172.21.15.78:0/401155673 <== osd.5 v2:172.21.15.133:6816/35899 3 ==== osd_op_reply(28 [pgnls start_epoch 43 out=49b] v36'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc5bc070040 con 0x7fc5ac0343a0 2023-10-11T09:42:45.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.036+0000 7fc5c1e05700 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:29 5.7 5:e0000000:%01:::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x7fc5ac042040 con 0x7fc5ac0343a0 2023-10-11T09:42:45.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.036+0000 7fc5c1e05700 1 -- 172.21.15.78:0/401155673 <== osd.5 v2:172.21.15.133:6816/35899 4 ==== osd_op_reply(29 [pgnls start_epoch 43 out=73b] v36'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fc5bc070040 con 0x7fc5ac0343a0 2023-10-11T09:42:45.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.036+0000 7fc5c1e05700 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:30 5.17 5:e8000000:%01:::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x7fc5ac042040 con 0x55f43f08a8b0 2023-10-11T09:42:45.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.036+0000 7fc5c0e03700 1 -- 172.21.15.78:0/401155673 <== osd.3 v2:172.21.15.78:6824/35738 4 ==== osd_op_reply(30 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc5ac00fbd0 con 0x55f43f08a8b0 2023-10-11T09:42:45.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.036+0000 7fc5c0e03700 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:31 5.f 5:f0000000:%01:::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x7fc5ac042040 con 0x7fc5ac0343a0 2023-10-11T09:42:45.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.036+0000 7fc5c1e05700 1 -- 172.21.15.78:0/401155673 <== osd.5 v2:172.21.15.133:6816/35899 5 ==== osd_op_reply(31 [pgnls start_epoch 43 out=73b] v36'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fc5bc070040 con 0x7fc5ac0343a0 2023-10-11T09:42:45.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.036+0000 7fc5c1e05700 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:32 5.1f 5:f8000000:%01:::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x7fc5ac042040 con 0x7fc5bc01e070 2023-10-11T09:42:45.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.036+0000 7fc5c0e03700 1 -- 172.21.15.78:0/401155673 <== osd.6 v2:172.21.15.133:6800/35900 3 ==== osd_op_reply(32 [pgnls start_epoch 43 out=101b] v42'6 uv6 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x7fc5ac00fbd0 con 0x7fc5bc01e070 2023-10-11T09:42:45.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.036+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:33 5.10 5:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e43) v8 -- 0x55f43f1213d0 con 0x7fc5ac010420 2023-10-11T09:42:45.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.036+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:34 5.10 5:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e43) v8 -- 0x55f43f12e700 con 0x7fc5ac010420 2023-10-11T09:42:45.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.036+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:35 5.10 5:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e43) v8 -- 0x55f43f131230 con 0x7fc5ac010420 2023-10-11T09:42:45.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.036+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:36 5.10 5:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e43) v8 -- 0x55f43f131ad0 con 0x7fc5ac010420 2023-10-11T09:42:45.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.036+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:37 5.10 5:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e43) v8 -- 0x55f43f132370 con 0x7fc5ac010420 2023-10-11T09:42:45.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.036+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:38 5.14 5:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e43) v8 -- 0x55f43f132c10 con 0x55f43f08a8b0 2023-10-11T09:42:45.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.040+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:39 5.14 5:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e43) v8 -- 0x55f43f1334b0 con 0x55f43f08a8b0 2023-10-11T09:42:45.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.040+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:40 5.14 5:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e43) v8 -- 0x55f43f133d50 con 0x55f43f08a8b0 2023-10-11T09:42:45.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.040+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:41 5.14 5:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e43) v8 -- 0x55f43f1345f0 con 0x55f43f08a8b0 2023-10-11T09:42:45.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.040+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:42 5.1c 5:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e43) v8 -- 0x55f43f134e90 con 0x7fc5ac0244a0 2023-10-11T09:42:45.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.040+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:43 5.6 5:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e43) v8 -- 0x55f43f135910 con 0x7fc5b400f370 2023-10-11T09:42:45.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.040+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:44 5.e 5:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e43) v8 -- 0x55f43f136450 con 0x7fc5ac0244a0 2023-10-11T09:42:45.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.040+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:45 5.15 5:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e43) v8 -- 0x55f43f136fb0 con 0x7fc5ac0343a0 2023-10-11T09:42:45.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.040+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:46 5.15 5:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e43) v8 -- 0x55f43f137b40 con 0x7fc5ac0343a0 2023-10-11T09:42:45.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.040+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:47 5.d 5:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e43) v8 -- 0x55f43f1386d0 con 0x7fc5b400f370 2023-10-11T09:42:45.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.040+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:48 5.1d 5:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e43) v8 -- 0x55f43f1391e0 con 0x7fc5ac035040 2023-10-11T09:42:45.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.044+0000 7fc5c1e05700 1 -- 172.21.15.78:0/401155673 <== osd.4 v2:172.21.15.133:6808/35898 5 ==== osd_op_reply(42 607.00000000 [delete] v43'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fc5bc070040 con 0x7fc5ac0244a0 2023-10-11T09:42:45.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.044+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:49 5.13 5:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e43) v8 -- 0x55f43f1386d0 con 0x55f43f08a8b0 2023-10-11T09:42:45.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.044+0000 7fc5c0e03700 1 -- 172.21.15.78:0/401155673 <== osd.3 v2:172.21.15.78:6824/35738 5 ==== osd_op_reply(38 1.00000000.inode [delete] v43'34 uv34 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fc5ac00fbd0 con 0x55f43f08a8b0 2023-10-11T09:42:45.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.044+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:50 5.13 5:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e43) v8 -- 0x55f43f1386d0 con 0x55f43f08a8b0 2023-10-11T09:42:45.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.044+0000 7fc5c1e05700 1 -- 172.21.15.78:0/401155673 <== osd.5 v2:172.21.15.133:6816/35899 6 ==== osd_op_reply(45 604.00000000 [delete] v43'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fc5bc070040 con 0x7fc5ac0343a0 2023-10-11T09:42:45.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.044+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:51 5.b 5:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e43) v8 -- 0x55f43f1386d0 con 0x7fc5b400f370 2023-10-11T09:42:45.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.044+0000 7fc5c1604700 1 -- 172.21.15.78:0/401155673 <== osd.1 v2:172.21.15.78:6809/35734 5 ==== osd_op_reply(48 605.00000000 [delete] v43'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fc5b4011760 con 0x7fc5ac035040 2023-10-11T09:42:45.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.044+0000 7fc5c1604700 1 -- 172.21.15.78:0/401155673 <== osd.7 v2:172.21.15.133:6824/35901 5 ==== osd_op_reply(33 601.00000000 [delete] v43'11 uv11 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fc5b4011760 con 0x7fc5ac010420 2023-10-11T09:42:45.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.044+0000 7fc5c1604700 1 -- 172.21.15.78:0/401155673 <== osd.7 v2:172.21.15.133:6824/35901 6 ==== osd_op_reply(34 602.00000000 [delete] v43'12 uv12 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fc5b4011760 con 0x7fc5ac010420 2023-10-11T09:42:45.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.044+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:52 5.7 5:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e43) v8 -- 0x55f43f1386d0 con 0x7fc5ac0343a0 2023-10-11T09:42:45.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.044+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:53 5.f 5:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e43) v8 -- 0x55f43f11fb70 con 0x7fc5ac0343a0 2023-10-11T09:42:45.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.044+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:54 5.1f 5:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e43) v8 -- 0x55f43f139b20 con 0x7fc5bc01e070 2023-10-11T09:42:45.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.044+0000 7fc5c1e05700 1 -- 172.21.15.78:0/401155673 <== osd.5 v2:172.21.15.133:6816/35899 7 ==== osd_op_reply(46 500.00000000 [delete] v43'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fc5bc070040 con 0x7fc5ac0343a0 2023-10-11T09:42:45.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.048+0000 7fc5c0e03700 1 -- 172.21.15.78:0/401155673 <== osd.3 v2:172.21.15.78:6824/35738 6 ==== osd_op_reply(49 mds0_inotable [delete] v43'4 uv4 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc5ac00fbd0 con 0x55f43f08a8b0 2023-10-11T09:42:45.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.048+0000 7fc5c1e05700 1 -- 172.21.15.78:0/401155673 <== osd.4 v2:172.21.15.133:6808/35898 6 ==== osd_op_reply(44 608.00000000 [delete] v43'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fc5bc070040 con 0x7fc5ac0244a0 2023-10-11T09:42:45.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.048+0000 7fc5c0e03700 1 -- 172.21.15.78:0/401155673 <== osd.2 v2:172.21.15.78:6808/35732 6 ==== osd_op_reply(43 mds0_openfiles.0 [delete] v43'9 uv9 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fc5ac00fbd0 con 0x7fc5b400f370 2023-10-11T09:42:45.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.048+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:55 5.1f 5:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e43) v8 -- 0x55f43f139b20 con 0x7fc5bc01e070 2023-10-11T09:42:45.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.048+0000 7fc5c1604700 1 -- 172.21.15.78:0/401155673 <== osd.7 v2:172.21.15.133:6824/35901 7 ==== osd_op_reply(35 10000000000.00000000 [delete] v43'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fc5b4011760 con 0x7fc5ac010420 2023-10-11T09:42:45.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.048+0000 7fc5c0e03700 1 -- 172.21.15.78:0/401155673 <== osd.2 v2:172.21.15.78:6808/35732 7 ==== osd_op_reply(47 mds_snaptable [delete] v43'2 uv2 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc5ac00fbd0 con 0x7fc5b400f370 2023-10-11T09:42:45.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.048+0000 7fc5c0e03700 1 -- 172.21.15.78:0/401155673 <== osd.3 v2:172.21.15.78:6824/35738 7 ==== osd_op_reply(39 200.00000000 [delete] v43'35 uv35 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fc5ac00fbd0 con 0x55f43f08a8b0 2023-10-11T09:42:45.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.048+0000 7fc5c0e03700 1 -- 172.21.15.78:0/401155673 <== osd.3 v2:172.21.15.78:6824/35738 8 ==== osd_op_reply(40 200.00000001 [delete] v43'36 uv36 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fc5ac00fbd0 con 0x55f43f08a8b0 2023-10-11T09:42:45.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.048+0000 7fc5c1604700 1 -- 172.21.15.78:0/401155673 <== osd.7 v2:172.21.15.133:6824/35901 8 ==== osd_op_reply(36 600.00000000 [delete] v43'14 uv14 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fc5b4011760 con 0x7fc5ac010420 2023-10-11T09:42:45.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.048+0000 7fc5c0e03700 1 -- 172.21.15.78:0/401155673 <== osd.3 v2:172.21.15.78:6824/35738 9 ==== osd_op_reply(41 606.00000000 [delete] v43'37 uv37 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fc5ac00fbd0 con 0x55f43f08a8b0 2023-10-11T09:42:45.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.048+0000 7fc5c0e03700 1 -- 172.21.15.78:0/401155673 <== osd.3 v2:172.21.15.78:6824/35738 10 ==== osd_op_reply(50 100.00000000 [delete] v43'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fc5ac00fbd0 con 0x55f43f08a8b0 2023-10-11T09:42:45.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.048+0000 7fc5c0e03700 1 -- 172.21.15.78:0/401155673 <== osd.2 v2:172.21.15.78:6808/35732 8 ==== osd_op_reply(51 mds0_sessionmap [delete] v43'4 uv4 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fc5ac00fbd0 con 0x7fc5b400f370 2023-10-11T09:42:45.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.048+0000 7fc5c1e05700 1 -- 172.21.15.78:0/401155673 <== osd.5 v2:172.21.15.133:6816/35899 8 ==== osd_op_reply(52 609.00000000 [delete] v43'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fc5bc070040 con 0x7fc5ac0343a0 2023-10-11T09:42:45.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.048+0000 7fc5c1604700 1 -- 172.21.15.78:0/401155673 <== osd.7 v2:172.21.15.133:6824/35901 9 ==== osd_op_reply(37 603.00000000 [delete] v43'15 uv15 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fc5b4011760 con 0x7fc5ac010420 2023-10-11T09:42:45.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.048+0000 7fc5c1e05700 1 -- 172.21.15.78:0/401155673 <== osd.5 v2:172.21.15.133:6816/35899 9 ==== osd_op_reply(53 400.00000000 [delete] v43'2 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fc5bc070040 con 0x7fc5ac0343a0 2023-10-11T09:42:45.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.048+0000 7fc5c0e03700 1 -- 172.21.15.78:0/401155673 <== osd.6 v2:172.21.15.133:6800/35900 4 ==== osd_op_reply(54 100.00000000.inode [delete] v43'7 uv7 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fc5ac00fbd0 con 0x7fc5bc01e070 2023-10-11T09:42:45.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.048+0000 7fc5c0e03700 1 -- 172.21.15.78:0/401155673 <== osd.6 v2:172.21.15.133:6800/35900 5 ==== osd_op_reply(55 1.00000000 [delete] v43'8 uv8 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fc5ac00fbd0 con 0x7fc5bc01e070 2023-10-11T09:42:45.055 INFO:teuthology.orchestra.run.smithi078.stdout:Removed 23 objects 2023-10-11T09:42:45.055 INFO:teuthology.orchestra.run.smithi078.stdout:successfully purged pool cephfs_metadata 2023-10-11T09:42:45.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.048+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x7fc5bc009040 msgr2=0x7fc5bc0668a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:45.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.048+0000 7fc5c3acf140 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x7fc5bc009040 0x7fc5bc0668a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:45.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.048+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x7fc5ac035040 msgr2=0x7fc5ac035410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:45.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.048+0000 7fc5c3acf140 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x7fc5ac035040 0x7fc5ac035410 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:45.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.048+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x7fc5b400f370 msgr2=0x7fc5b400f7a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:45.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.048+0000 7fc5c3acf140 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x7fc5b400f370 0x7fc5b400f7a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:45.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.048+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55f43f08a8b0 msgr2=0x55f43f120a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:45.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.048+0000 7fc5c3acf140 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55f43f08a8b0 0x55f43f120a20 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:45.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.048+0000 7fc5c1e05700 1 -- 172.21.15.78:0/401155673 reap_dead start 2023-10-11T09:42:45.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.048+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x7fc5ac0244a0 msgr2=0x7fc5ac0223c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:45.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.048+0000 7fc5c3acf140 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x7fc5ac0244a0 0x7fc5ac0223c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:45.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.048+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x7fc5ac0343a0 msgr2=0x7fc5ac0347f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:45.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.048+0000 7fc5c3acf140 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x7fc5ac0343a0 0x7fc5ac0347f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:45.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.048+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x7fc5bc01e070 msgr2=0x7fc5bc0579e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:45.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.048+0000 7fc5c3acf140 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x7fc5bc01e070 0x7fc5bc0579e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:45.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.048+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x7fc5ac010420 msgr2=0x7fc5ac010850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:45.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.048+0000 7fc5c3acf140 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x7fc5ac010420 0x7fc5ac010850 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:45.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.048+0000 7fc5c1e05700 1 -- 172.21.15.78:0/401155673 reap_dead start 2023-10-11T09:42:45.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.048+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc5a8064800 msgr2=0x7fc5a8066cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:45.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.052+0000 7fc5c3acf140 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc5a8064800 0x7fc5a8066cb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4000c00 tx=0x7fc5b4001740 comp rx=0 tx=0).stop 2023-10-11T09:42:45.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.052+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f43efead00 msgr2=0x55f43f087030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:45.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.052+0000 7fc5c3acf140 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f43efead00 0x55f43f087030 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc00bd90 tx=0x7fc5bc0076d0 comp rx=0 tx=0).stop 2023-10-11T09:42:45.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.052+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 shutdown_connections 2023-10-11T09:42:45.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.052+0000 7fc5c3acf140 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc5a8064800 0x7fc5a8066cb0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:45.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.052+0000 7fc5c3acf140 1 --2- 172.21.15.78:0/401155673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f43efead00 0x55f43f087030 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:45.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.052+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 >> 172.21.15.78:0/401155673 conn(0x55f43efbcbe0 msgr2=0x55f43f08fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:45.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.052+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 shutdown_connections 2023-10-11T09:42:45.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.052+0000 7fc5c3acf140 1 -- 172.21.15.78:0/401155673 wait complete. 2023-10-11T09:42:45.063 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs reset cephfs --yes-i-really-mean-it 2023-10-11T09:42:45.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.244+0000 7f4fb085d700 1 Processor -- start 2023-10-11T09:42:45.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.244+0000 7f4fb085d700 1 -- start start 2023-10-11T09:42:45.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.248+0000 7f4fb085d700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4fac0b3240 0x7f4fac0d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:45.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.248+0000 7f4fb085d700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fac0b47f0 con 0x7f4fac0b3a90 2023-10-11T09:42:45.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.248+0000 7f4fb085d700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fac0b8130 con 0x7f4fac0b3610 2023-10-11T09:42:45.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.248+0000 7f4fb085d700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fac0b41e0 con 0x7f4fac0b3240 2023-10-11T09:42:45.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.248+0000 7f4fab59e700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f4fac0b3a90 0x7f4fac0b85c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:59318/0 (socket says 172.21.15.78:59318) 2023-10-11T09:42:45.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.248+0000 7f4fab59e700 1 -- 172.21.15.78:0/1628703304 learned_addr learned my addr 172.21.15.78:0/1628703304 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:42:45.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.248+0000 7f4faad9d700 1 --2- 172.21.15.78:0/1628703304 >> v2:172.21.15.133:3300/0 conn(0x7f4fac0b3240 0x7f4fac0d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:45.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.248+0000 7f4fa9d9b700 1 -- 172.21.15.78:0/1628703304 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2625961309 0 0) 0x7f4fa4001490 con 0x7f4fac0b3a90 2023-10-11T09:42:45.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.248+0000 7f4fa9d9b700 1 -- 172.21.15.78:0/1628703304 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f940035d0 con 0x7f4fac0b3a90 2023-10-11T09:42:45.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.248+0000 7f4faad9d700 1 -- 172.21.15.78:0/1628703304 >> v1:172.21.15.133:6789/0 conn(0x7f4fac0b3610 legacy=0x7f4fac15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:45.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.248+0000 7f4faad9d700 1 -- 172.21.15.78:0/1628703304 >> v1:172.21.15.78:6789/0 conn(0x7f4fac0b3a90 legacy=0x7f4fac0b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:45.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.248+0000 7f4faad9d700 1 -- 172.21.15.78:0/1628703304 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fac0b7800 con 0x7f4fac0b3240 2023-10-11T09:42:45.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.248+0000 7f4faad9d700 1 --2- 172.21.15.78:0/1628703304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fac0b3240 0x7f4fac0d29d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c009850 tx=0x7f4f9c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db8ee76e97c024ca server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:45.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.252+0000 7f4fa9d9b700 1 -- 172.21.15.78:0/1628703304 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c011830 con 0x7f4fac0b3240 2023-10-11T09:42:45.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.252+0000 7f4fa9d9b700 1 -- 172.21.15.78:0/1628703304 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f9c011990 con 0x7f4fac0b3240 2023-10-11T09:42:45.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.252+0000 7f4fa9d9b700 1 -- 172.21.15.78:0/1628703304 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c011d40 con 0x7f4fac0b3240 2023-10-11T09:42:45.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.252+0000 7f4fb085d700 1 -- 172.21.15.78:0/1628703304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fac0b3240 msgr2=0x7f4fac0d29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:45.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.252+0000 7f4fb085d700 1 --2- 172.21.15.78:0/1628703304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fac0b3240 0x7f4fac0d29d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c009850 tx=0x7f4f9c009b60 comp rx=0 tx=0).stop 2023-10-11T09:42:45.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.252+0000 7f4fb085d700 1 -- 172.21.15.78:0/1628703304 shutdown_connections 2023-10-11T09:42:45.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.252+0000 7f4fb085d700 1 --2- 172.21.15.78:0/1628703304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fac0b3240 0x7f4fac0d29d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:45.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.252+0000 7f4fb085d700 1 -- 172.21.15.78:0/1628703304 >> 172.21.15.78:0/1628703304 conn(0x7f4fac09c260 msgr2=0x7f4fac0d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:45.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.252+0000 7f4fb085d700 1 -- 172.21.15.78:0/1628703304 shutdown_connections 2023-10-11T09:42:45.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.252+0000 7f4fb085d700 1 -- 172.21.15.78:0/1628703304 wait complete. 2023-10-11T09:42:45.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.256+0000 7f4fb085d700 1 Processor -- start 2023-10-11T09:42:45.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.256+0000 7f4fb085d700 1 -- start start 2023-10-11T09:42:45.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.256+0000 7f4fb085d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fac0b3240 0x7f4fac0b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:45.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.256+0000 7f4fb085d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fac0b3610 0x7f4fac15ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:45.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.256+0000 7f4fb085d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fac0b3a90 0x7f4fac15f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:45.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.256+0000 7f4fb085d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac159b90 con 0x7f4fac0b3a90 2023-10-11T09:42:45.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.256+0000 7f4fb085d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac0b1750 con 0x7f4fac0b3240 2023-10-11T09:42:45.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.256+0000 7f4fb085d700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac0b0eb0 con 0x7f4fac0b3610 2023-10-11T09:42:45.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.256+0000 7f4fab59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fac0b3a90 0x7f4fac15f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:45.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.256+0000 7f4fab59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fac0b3a90 0x7f4fac15f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57808/0 (socket says 172.21.15.78:57808) 2023-10-11T09:42:45.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.256+0000 7f4fab59e700 1 -- 172.21.15.78:0/3961342524 learned_addr learned my addr 172.21.15.78:0/3961342524 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:45.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.256+0000 7f4faad9d700 1 --2- 172.21.15.78:0/3961342524 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fac0b3240 0x7f4fac0b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:45.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.256+0000 7f4faa59c700 1 --2- 172.21.15.78:0/3961342524 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fac0b3610 0x7f4fac15ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:45.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.256+0000 7f4fab59e700 1 -- 172.21.15.78:0/3961342524 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fac0b3610 msgr2=0x7f4fac15ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:45.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.256+0000 7f4fab59e700 1 --2- 172.21.15.78:0/3961342524 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fac0b3610 0x7f4fac15ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:45.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.256+0000 7f4fab59e700 1 -- 172.21.15.78:0/3961342524 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fac0b3240 msgr2=0x7f4fac0b07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:45.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.256+0000 7f4fab59e700 1 --2- 172.21.15.78:0/3961342524 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fac0b3240 0x7f4fac0b07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:45.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.256+0000 7f4fab59e700 1 -- 172.21.15.78:0/3961342524 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f9c0094b0 con 0x7f4fac0b3a90 2023-10-11T09:42:45.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.256+0000 7f4fab59e700 1 --2- 172.21.15.78:0/3961342524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fac0b3a90 0x7f4fac15f350 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4fa400cae0 tx=0x7f4fa400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:45.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.256+0000 7f4f937fe700 1 -- 172.21.15.78:0/3961342524 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa400a7e0 con 0x7f4fac0b3a90 2023-10-11T09:42:45.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.256+0000 7f4f937fe700 1 -- 172.21.15.78:0/3961342524 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa400a940 con 0x7f4fac0b3a90 2023-10-11T09:42:45.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.256+0000 7f4fb085d700 1 -- 172.21.15.78:0/3961342524 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fac0b1190 con 0x7f4fac0b3a90 2023-10-11T09:42:45.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.256+0000 7f4f937fe700 1 -- 172.21.15.78:0/3961342524 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa400acd0 con 0x7f4fac0b3a90 2023-10-11T09:42:45.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.256+0000 7f4fb085d700 1 -- 172.21.15.78:0/3961342524 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fac0c6b60 con 0x7f4fac0b3a90 2023-10-11T09:42:45.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.260+0000 7f4fb085d700 1 -- 172.21.15.78:0/3961342524 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fac0d09b0 con 0x7f4fac0b3a90 2023-10-11T09:42:45.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.260+0000 7f4f937fe700 1 -- 172.21.15.78:0/3961342524 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f4fa401b3b0 con 0x7f4fac0b3a90 2023-10-11T09:42:45.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.264+0000 7f4f937fe700 1 --2- 172.21.15.78:0/3961342524 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4f98065ed0 0x7f4f98068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:45.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.264+0000 7f4f937fe700 1 -- 172.21.15.78:0/3961342524 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6581+0+0 (secure 0 0 0) 0x7f4fa4083330 con 0x7f4fac0b3a90 2023-10-11T09:42:45.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.264+0000 7f4faad9d700 1 --2- 172.21.15.78:0/3961342524 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4f98065ed0 0x7f4f98068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:45.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.264+0000 7f4faad9d700 1 --2- 172.21.15.78:0/3961342524 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4f98065ed0 0x7f4f98068380 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c0090b0 tx=0x7f4f9c01c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:45.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.272+0000 7f4f937fe700 1 -- 172.21.15.78:0/3961342524 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f4fa40506d0 con 0x7f4fac0b3a90 2023-10-11T09:42:45.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.488+0000 7f4fb085d700 1 -- 172.21.15.78:0/3961342524 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs reset", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4fac0c6d80 con 0x7f4fac0b3a90 2023-10-11T09:42:45.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.584+0000 7f4f937fe700 1 -- 172.21.15.78:0/3961342524 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs reset", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 107+0+0 (secure 0 0 0) 0x7f4fa4001b70 con 0x7f4fac0b3a90 2023-10-11T09:42:45.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.588+0000 7f4fb085d700 1 -- 172.21.15.78:0/3961342524 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4f98065ed0 msgr2=0x7f4f98068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:45.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.588+0000 7f4fb085d700 1 --2- 172.21.15.78:0/3961342524 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4f98065ed0 0x7f4f98068380 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c0090b0 tx=0x7f4f9c01c040 comp rx=0 tx=0).stop 2023-10-11T09:42:45.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.588+0000 7f4fb085d700 1 -- 172.21.15.78:0/3961342524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fac0b3a90 msgr2=0x7f4fac15f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:45.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.588+0000 7f4fb085d700 1 --2- 172.21.15.78:0/3961342524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fac0b3a90 0x7f4fac15f350 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4fa400cae0 tx=0x7f4fa400cbc0 comp rx=0 tx=0).stop 2023-10-11T09:42:45.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.588+0000 7f4fb085d700 1 -- 172.21.15.78:0/3961342524 shutdown_connections 2023-10-11T09:42:45.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.588+0000 7f4fb085d700 1 --2- 172.21.15.78:0/3961342524 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4f98065ed0 0x7f4f98068380 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:45.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.588+0000 7f4fb085d700 1 --2- 172.21.15.78:0/3961342524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fac0b3a90 0x7f4fac15f350 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:45.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.588+0000 7f4fb085d700 1 --2- 172.21.15.78:0/3961342524 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fac0b3610 0x7f4fac15ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:45.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.588+0000 7f4fb085d700 1 --2- 172.21.15.78:0/3961342524 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fac0b3240 0x7f4fac0b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:45.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.588+0000 7f4fb085d700 1 -- 172.21.15.78:0/3961342524 >> 172.21.15.78:0/3961342524 conn(0x7f4fac09c260 msgr2=0x7f4fac0afb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:45.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.588+0000 7f4fb085d700 1 -- 172.21.15.78:0/3961342524 shutdown_connections 2023-10-11T09:42:45.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.588+0000 7f4fb085d700 1 -- 172.21.15.78:0/3961342524 wait complete. 2023-10-11T09:42:45.600 INFO:tasks.ceph.mds.b.smithi133.stderr:2023-10-11T09:42:45.593+0000 7fa631928700 -1 mds.0.inotable: load_2 could not read table: -2 2023-10-11T09:42:45.600 INFO:tasks.ceph.mds.b.smithi133.stderr:2023-10-11T09:42:45.593+0000 7fa631928700 -1 log_channel(cluster) log [ERR] : error reading table object 'mds0_inotable' -2 ((2) No such file or directory) 2023-10-11T09:42:45.601 INFO:tasks.ceph.mon.a.smithi078.stderr:2023-10-11T09:42:45.600+0000 7f90131f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-11T09:42:45.602 INFO:tasks.ceph.mon.a.smithi078.stderr:2023-10-11T09:42:45.600+0000 7f90131f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-10-11T09:42:45.617 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-11T09:42:45.618 INFO:tasks.ceph.mds.b.smithi133.stderr: -20> 2023-10-11T09:42:45.593+0000 7fa631928700 -1 mds.0.inotable: load_2 could not read table: -2 2023-10-11T09:42:45.618 INFO:tasks.ceph.mds.b.smithi133.stderr: -19> 2023-10-11T09:42:45.593+0000 7fa631928700 -1 log_channel(cluster) log [ERR] : error reading table object 'mds0_inotable' -2 ((2) No such file or directory) 2023-10-11T09:42:45.669 INFO:tasks.ceph.mds.b.smithi133.stdout:starting mds.b at 2023-10-11T09:42:45.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.788+0000 7f168f59e700 1 Processor -- start 2023-10-11T09:42:45.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.792+0000 7f168f59e700 1 -- start start 2023-10-11T09:42:45.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.792+0000 7f168f59e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f169015a6b0 0x7f16900ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:45.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.792+0000 7f168f59e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f16900b3a60 0x7f169006d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:45.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.796+0000 7f168f59e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16900b2d30 con 0x7f16900cede0 2023-10-11T09:42:45.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.796+0000 7f168f59e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16900cf360 con 0x7f169015a6b0 2023-10-11T09:42:45.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.796+0000 7f168f59e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16900b2f80 con 0x7f16900b3a60 2023-10-11T09:42:45.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.796+0000 7f168e59c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f169015a6b0 0x7f16900ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:45.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.796+0000 7f168e59c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f169015a6b0 0x7f16900ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57816/0 (socket says 172.21.15.78:57816) 2023-10-11T09:42:45.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.796+0000 7f168ed9d700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f16900cede0 0x7f16900b1a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:59330/0 (socket says 172.21.15.78:59330) 2023-10-11T09:42:45.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.796+0000 7f168e59c700 1 -- 172.21.15.78:0/3613156865 learned_addr learned my addr 172.21.15.78:0/3613156865 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:45.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.796+0000 7f168dd9b700 1 --2- 172.21.15.78:0/3613156865 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f16900b3a60 0x7f169006d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:45.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.796+0000 7f168e59c700 1 -- 172.21.15.78:0/3613156865 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f16900b3a60 msgr2=0x7f169006d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:45.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.796+0000 7f168e59c700 1 --2- 172.21.15.78:0/3613156865 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f16900b3a60 0x7f169006d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:45.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.796+0000 7f168e59c700 1 -- 172.21.15.78:0/3613156865 >> v1:172.21.15.78:6789/0 conn(0x7f16900cede0 legacy=0x7f16900b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:45.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.796+0000 7f168e59c700 1 -- 172.21.15.78:0/3613156865 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16900b3260 con 0x7f169015a6b0 2023-10-11T09:42:45.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.796+0000 7f168e59c700 1 --2- 172.21.15.78:0/3613156865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f169015a6b0 0x7f16900ce8a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1680009910 tx=0x7f1680009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c81c723a66c61e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:45.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.796+0000 7f168d59a700 1 -- 172.21.15.78:0/3613156865 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168000ba60 con 0x7f169015a6b0 2023-10-11T09:42:45.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.796+0000 7f168d59a700 1 -- 172.21.15.78:0/3613156865 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f168000bbc0 con 0x7f169015a6b0 2023-10-11T09:42:45.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.796+0000 7f168d59a700 1 -- 172.21.15.78:0/3613156865 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1680009e60 con 0x7f169015a6b0 2023-10-11T09:42:45.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.796+0000 7f168f59e700 1 -- 172.21.15.78:0/3613156865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f169015a6b0 msgr2=0x7f16900ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:45.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.796+0000 7f168f59e700 1 --2- 172.21.15.78:0/3613156865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f169015a6b0 0x7f16900ce8a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1680009910 tx=0x7f1680009c20 comp rx=0 tx=0).stop 2023-10-11T09:42:45.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.796+0000 7f168f59e700 1 -- 172.21.15.78:0/3613156865 shutdown_connections 2023-10-11T09:42:45.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.796+0000 7f168f59e700 1 --2- 172.21.15.78:0/3613156865 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f16900b3a60 0x7f169006d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:45.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.796+0000 7f168f59e700 1 --2- 172.21.15.78:0/3613156865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f169015a6b0 0x7f16900ce8a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:45.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.796+0000 7f168f59e700 1 -- 172.21.15.78:0/3613156865 >> 172.21.15.78:0/3613156865 conn(0x7f169006a060 msgr2=0x7f169006a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:45.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.796+0000 7f168f59e700 1 -- 172.21.15.78:0/3613156865 shutdown_connections 2023-10-11T09:42:45.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.796+0000 7f168f59e700 1 -- 172.21.15.78:0/3613156865 wait complete. 2023-10-11T09:42:45.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.800+0000 7f168f59e700 1 Processor -- start 2023-10-11T09:42:45.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.800+0000 7f168f59e700 1 -- start start 2023-10-11T09:42:45.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.804+0000 7f168f59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16900b3a60 0x7f169006fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:45.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.804+0000 7f168f59e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16900cede0 0x7f1690071920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:45.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.804+0000 7f168f59e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f169015a6b0 0x7f169006a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:45.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.804+0000 7f168f59e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16901568d0 con 0x7f16900b3a60 2023-10-11T09:42:45.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.804+0000 7f168f59e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1690157fd0 con 0x7f16900cede0 2023-10-11T09:42:45.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.804+0000 7f168f59e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16900703c0 con 0x7f169015a6b0 2023-10-11T09:42:45.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.804+0000 7f168e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16900b3a60 0x7f169006fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:45.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.804+0000 7f168e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16900b3a60 0x7f169006fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57828/0 (socket says 172.21.15.78:57828) 2023-10-11T09:42:45.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.804+0000 7f168e59c700 1 -- 172.21.15.78:0/1231567694 learned_addr learned my addr 172.21.15.78:0/1231567694 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:45.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.804+0000 7f168e59c700 1 -- 172.21.15.78:0/1231567694 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f169015a6b0 msgr2=0x7f169006a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:45.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.804+0000 7f168ed9d700 1 --2- 172.21.15.78:0/1231567694 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f169015a6b0 0x7f169006a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:45.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.804+0000 7f168dd9b700 1 --2- 172.21.15.78:0/1231567694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16900cede0 0x7f1690071920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:45.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.804+0000 7f168e59c700 1 --2- 172.21.15.78:0/1231567694 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f169015a6b0 0x7f169006a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:45.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.804+0000 7f168e59c700 1 -- 172.21.15.78:0/1231567694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16900cede0 msgr2=0x7f1690071920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:45.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.804+0000 7f168e59c700 1 --2- 172.21.15.78:0/1231567694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16900cede0 0x7f1690071920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:45.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.804+0000 7f168e59c700 1 -- 172.21.15.78:0/1231567694 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1680009570 con 0x7f16900b3a60 2023-10-11T09:42:45.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.804+0000 7f168ed9d700 1 --2- 172.21.15.78:0/1231567694 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f169015a6b0 0x7f169006a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:42:45.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.804+0000 7f168dd9b700 1 --2- 172.21.15.78:0/1231567694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16900cede0 0x7f1690071920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:42:45.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.804+0000 7f168e59c700 1 --2- 172.21.15.78:0/1231567694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16900b3a60 0x7f169006fde0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1680000c00 tx=0x7f1680004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:45.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.804+0000 7f167effd700 1 -- 172.21.15.78:0/1231567694 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1680004610 con 0x7f16900b3a60 2023-10-11T09:42:45.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.804+0000 7f167effd700 1 -- 172.21.15.78:0/1231567694 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f168001e070 con 0x7f16900b3a60 2023-10-11T09:42:45.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.804+0000 7f168f59e700 1 -- 172.21.15.78:0/1231567694 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16900706a0 con 0x7f16900b3a60 2023-10-11T09:42:45.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.804+0000 7f167effd700 1 -- 172.21.15.78:0/1231567694 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1680003920 con 0x7f16900b3a60 2023-10-11T09:42:45.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.804+0000 7f168f59e700 1 -- 172.21.15.78:0/1231567694 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1690070c60 con 0x7f16900b3a60 2023-10-11T09:42:45.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.808+0000 7f168f59e700 1 -- 172.21.15.78:0/1231567694 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1690065d40 con 0x7f16900b3a60 2023-10-11T09:42:45.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.808+0000 7f167effd700 1 -- 172.21.15.78:0/1231567694 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f1680003a80 con 0x7f16900b3a60 2023-10-11T09:42:45.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.808+0000 7f167effd700 1 --2- 172.21.15.78:0/1231567694 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1670065ed0 0x7f1670068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:45.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.812+0000 7f167effd700 1 -- 172.21.15.78:0/1231567694 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6667+0+0 (secure 0 0 0) 0x7f1680020070 con 0x7f16900b3a60 2023-10-11T09:42:45.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.812+0000 7f168dd9b700 1 --2- 172.21.15.78:0/1231567694 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1670065ed0 0x7f1670068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:45.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.812+0000 7f168dd9b700 1 --2- 172.21.15.78:0/1231567694 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1670065ed0 0x7f1670068380 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f169014eef0 tx=0x7f1684008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:45.819 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:45.816+0000 7f167effd700 1 -- 172.21.15.78:0/1231567694 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f1680052380 con 0x7f16900b3a60 2023-10-11T09:42:46.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.032+0000 7f168f59e700 1 -- 172.21.15.78:0/1231567694 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f16900cb6a0 con 0x7f16900b3a60 2023-10-11T09:42:46.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.616+0000 7f167effd700 1 -- 172.21.15.78:0/1231567694 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v22) v1 ==== 162+0+0 (secure 0 0 0) 0x7f16800150b0 con 0x7f16900b3a60 2023-10-11T09:42:46.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.624+0000 7f168f59e700 1 -- 172.21.15.78:0/1231567694 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1670065ed0 msgr2=0x7f1670068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:46.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.624+0000 7f168f59e700 1 --2- 172.21.15.78:0/1231567694 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1670065ed0 0x7f1670068380 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f169014eef0 tx=0x7f1684008040 comp rx=0 tx=0).stop 2023-10-11T09:42:46.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.624+0000 7f168f59e700 1 -- 172.21.15.78:0/1231567694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16900b3a60 msgr2=0x7f169006fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:46.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.624+0000 7f168f59e700 1 --2- 172.21.15.78:0/1231567694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16900b3a60 0x7f169006fde0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1680000c00 tx=0x7f1680004770 comp rx=0 tx=0).stop 2023-10-11T09:42:46.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.624+0000 7f168f59e700 1 -- 172.21.15.78:0/1231567694 shutdown_connections 2023-10-11T09:42:46.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.624+0000 7f168f59e700 1 --2- 172.21.15.78:0/1231567694 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1670065ed0 0x7f1670068380 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:46.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.624+0000 7f168f59e700 1 --2- 172.21.15.78:0/1231567694 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f169015a6b0 0x7f169006a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:46.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.624+0000 7f168f59e700 1 --2- 172.21.15.78:0/1231567694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16900cede0 0x7f1690071920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:46.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.624+0000 7f168f59e700 1 --2- 172.21.15.78:0/1231567694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16900b3a60 0x7f169006fde0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:46.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.624+0000 7f168f59e700 1 -- 172.21.15.78:0/1231567694 >> 172.21.15.78:0/1231567694 conn(0x7f169006a060 msgr2=0x7f16901508f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:46.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.624+0000 7f168f59e700 1 -- 172.21.15.78:0/1231567694 shutdown_connections 2023-10-11T09:42:46.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.624+0000 7f168f59e700 1 -- 172.21.15.78:0/1231567694 wait complete. 2023-10-11T09:42:46.629 INFO:teuthology.orchestra.run.smithi078.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-11T09:42:46.651 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:42:46.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.816+0000 7f8bcaceb700 1 Processor -- start 2023-10-11T09:42:46.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.816+0000 7f8bcaceb700 1 -- start start 2023-10-11T09:42:46.824 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.820+0000 7f8bcaceb700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f8bc40cda10 0x7f8bc40caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:46.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.820+0000 7f8bcaceb700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8bc40cdde0 0x7f8bc4164610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:46.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.820+0000 7f8bcaceb700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8bc4164b50 0x7f8bc406bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:46.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.820+0000 7f8bcaceb700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bc4165390 con 0x7f8bc40cda10 2023-10-11T09:42:46.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.820+0000 7f8bcaceb700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bc40ca440 con 0x7f8bc4164b50 2023-10-11T09:42:46.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.820+0000 7f8bcaceb700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc40ca720 con 0x7f8bc40cdde0 2023-10-11T09:42:46.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.820+0000 7f8bc9ce9700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f8bc40cda10 0x7f8bc40caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:46.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.820+0000 7f8bc9ce9700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f8bc40cda10 0x7f8bc40caf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57844/0 (socket says 172.21.15.78:57844) 2023-10-11T09:42:46.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.820+0000 7f8bc9ce9700 1 -- 172.21.15.78:0/125480282 learned_addr learned my addr 172.21.15.78:0/125480282 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:46.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.820+0000 7f8bc94e8700 1 --2- 172.21.15.78:0/125480282 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8bc40cdde0 0x7f8bc4164610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:46.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.820+0000 7f8bca4ea700 1 --2- 172.21.15.78:0/125480282 >> v2:172.21.15.133:3300/0 conn(0x7f8bc4164b50 0x7f8bc406bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:46.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.824+0000 7f8bc9ce9700 1 -- 172.21.15.78:0/125480282 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8bc40cdde0 msgr2=0x7f8bc4164610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:46.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.824+0000 7f8bc9ce9700 1 --2- 172.21.15.78:0/125480282 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8bc40cdde0 0x7f8bc4164610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:46.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.824+0000 7f8bc9ce9700 1 -- 172.21.15.78:0/125480282 >> v2:172.21.15.133:3300/0 conn(0x7f8bc4164b50 msgr2=0x7f8bc406bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:46.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.824+0000 7f8bc9ce9700 1 --2- 172.21.15.78:0/125480282 >> v2:172.21.15.133:3300/0 conn(0x7f8bc4164b50 0x7f8bc406bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:46.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.824+0000 7f8bc9ce9700 1 -- 172.21.15.78:0/125480282 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc40caa00 con 0x7f8bc40cda10 2023-10-11T09:42:46.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.824+0000 7f8bc9ce9700 1 --2- 172.21.15.78:0/125480282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8bc40cda10 0x7f8bc40caf20 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8bc40b1730 tx=0x7f8bc0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e2ab60ff3842767 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:46.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.824+0000 7f8bc8ce7700 1 -- 172.21.15.78:0/125480282 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc000b8a0 con 0x7f8bc40cda10 2023-10-11T09:42:46.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.824+0000 7f8bc8ce7700 1 -- 172.21.15.78:0/125480282 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc000ba00 con 0x7f8bc40cda10 2023-10-11T09:42:46.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.824+0000 7f8bc8ce7700 1 -- 172.21.15.78:0/125480282 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc000bd90 con 0x7f8bc40cda10 2023-10-11T09:42:46.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.824+0000 7f8bcaceb700 1 -- 172.21.15.78:0/125480282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8bc40cda10 msgr2=0x7f8bc40caf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:46.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.824+0000 7f8bcaceb700 1 --2- 172.21.15.78:0/125480282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8bc40cda10 0x7f8bc40caf20 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8bc40b1730 tx=0x7f8bc0009bf0 comp rx=0 tx=0).stop 2023-10-11T09:42:46.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.824+0000 7f8bcaceb700 1 -- 172.21.15.78:0/125480282 shutdown_connections 2023-10-11T09:42:46.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.824+0000 7f8bcaceb700 1 --2- 172.21.15.78:0/125480282 >> v2:172.21.15.133:3300/0 conn(0x7f8bc4164b50 0x7f8bc406bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:46.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.824+0000 7f8bcaceb700 1 --2- 172.21.15.78:0/125480282 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8bc40cdde0 0x7f8bc4164610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:46.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.824+0000 7f8bcaceb700 1 --2- 172.21.15.78:0/125480282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8bc40cda10 0x7f8bc40caf20 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:46.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.824+0000 7f8bcaceb700 1 -- 172.21.15.78:0/125480282 >> 172.21.15.78:0/125480282 conn(0x7f8bc40cea60 msgr2=0x7f8bc40cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:46.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.824+0000 7f8bcaceb700 1 -- 172.21.15.78:0/125480282 shutdown_connections 2023-10-11T09:42:46.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.824+0000 7f8bcaceb700 1 -- 172.21.15.78:0/125480282 wait complete. 2023-10-11T09:42:46.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.828+0000 7f8bcaceb700 1 Processor -- start 2023-10-11T09:42:46.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.828+0000 7f8bcaceb700 1 -- start start 2023-10-11T09:42:46.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.828+0000 7f8bcaceb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8bc40cda10 0x7f8bc41632d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:46.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.828+0000 7f8bcaceb700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bc40cdde0 0x7f8bc4144760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:46.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.828+0000 7f8bcaceb700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8bc4164b50 0x7f8bc4144ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:46.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.828+0000 7f8bcaceb700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc41451e0 con 0x7f8bc40cda10 2023-10-11T09:42:46.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.828+0000 7f8bcaceb700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc41f9890 con 0x7f8bc40cdde0 2023-10-11T09:42:46.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.828+0000 7f8bcaceb700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc41f9b30 con 0x7f8bc4164b50 2023-10-11T09:42:46.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.828+0000 7f8bc9ce9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8bc40cda10 0x7f8bc41632d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:46.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.828+0000 7f8bc9ce9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8bc40cda10 0x7f8bc41632d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57846/0 (socket says 172.21.15.78:57846) 2023-10-11T09:42:46.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.828+0000 7f8bc9ce9700 1 -- 172.21.15.78:0/1572169821 learned_addr learned my addr 172.21.15.78:0/1572169821 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:46.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.828+0000 7f8bca4ea700 1 --2- 172.21.15.78:0/1572169821 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8bc4164b50 0x7f8bc4144ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:46.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.828+0000 7f8bc94e8700 1 --2- 172.21.15.78:0/1572169821 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bc40cdde0 0x7f8bc4144760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:46.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.828+0000 7f8bc9ce9700 1 -- 172.21.15.78:0/1572169821 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8bc4164b50 msgr2=0x7f8bc4144ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:46.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.828+0000 7f8bc9ce9700 1 --2- 172.21.15.78:0/1572169821 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8bc4164b50 0x7f8bc4144ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:46.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.828+0000 7f8bc9ce9700 1 -- 172.21.15.78:0/1572169821 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bc40cdde0 msgr2=0x7f8bc4144760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:46.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.828+0000 7f8bc9ce9700 1 --2- 172.21.15.78:0/1572169821 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bc40cdde0 0x7f8bc4144760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:46.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.828+0000 7f8bc9ce9700 1 -- 172.21.15.78:0/1572169821 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc0009570 con 0x7f8bc40cda10 2023-10-11T09:42:46.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.832+0000 7f8bc9ce9700 1 --2- 172.21.15.78:0/1572169821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8bc40cda10 0x7f8bc41632d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0000c00 tx=0x7f8bc0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:46.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.832+0000 7f8bba7fc700 1 -- 172.21.15.78:0/1572169821 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc00038c0 con 0x7f8bc40cda10 2023-10-11T09:42:46.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.832+0000 7f8bba7fc700 1 -- 172.21.15.78:0/1572169821 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc001e070 con 0x7f8bc40cda10 2023-10-11T09:42:46.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.832+0000 7f8bba7fc700 1 -- 172.21.15.78:0/1572169821 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0003c80 con 0x7f8bc40cda10 2023-10-11T09:42:46.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.832+0000 7f8bcaceb700 1 -- 172.21.15.78:0/1572169821 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc40ce3b0 con 0x7f8bc40cda10 2023-10-11T09:42:46.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.832+0000 7f8bcaceb700 1 -- 172.21.15.78:0/1572169821 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc41fa0f0 con 0x7f8bc40cda10 2023-10-11T09:42:46.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.832+0000 7f8bcaceb700 1 -- 172.21.15.78:0/1572169821 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bc4065d40 con 0x7f8bc40cda10 2023-10-11T09:42:46.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.836+0000 7f8bba7fc700 1 -- 172.21.15.78:0/1572169821 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f8bc0011420 con 0x7f8bc40cda10 2023-10-11T09:42:46.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.836+0000 7f8bba7fc700 1 --2- 172.21.15.78:0/1572169821 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8ba8065ed0 0x7f8ba8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:46.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.836+0000 7f8bba7fc700 1 -- 172.21.15.78:0/1572169821 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6667+0+0 (secure 0 0 0) 0x7f8bc0020070 con 0x7f8bc40cda10 2023-10-11T09:42:46.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.836+0000 7f8bc94e8700 1 --2- 172.21.15.78:0/1572169821 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8ba8065ed0 0x7f8ba8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:46.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.836+0000 7f8bc94e8700 1 --2- 172.21.15.78:0/1572169821 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8ba8065ed0 0x7f8ba8068380 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8bc40c8c20 tx=0x7f8bb4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:46.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:46.844+0000 7f8bba7fc700 1 -- 172.21.15.78:0/1572169821 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f8bc0052510 con 0x7f8bc40cda10 2023-10-11T09:42:47.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.052+0000 7f8bcaceb700 1 -- 172.21.15.78:0/1572169821 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8bc40b1480 con 0x7f8bc40cda10 2023-10-11T09:42:47.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.052+0000 7f8bba7fc700 1 -- 172.21.15.78:0/1572169821 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4642 (secure 0 0 0) 0x7f8bc0055b30 con 0x7f8bc40cda10 2023-10-11T09:42:47.057 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:47.057 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5308,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/655413753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":655413753},{"type":"v1","addr":"172.21.15.133:6835","nonce":655413753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5352,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2966622437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2966622437},{"type":"v1","addr":"172.21.15.78:6835","nonce":2966622437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5355,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/700246445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":700246445},{"type":"v1","addr":"172.21.15.78:6837","nonce":700246445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5641,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/1601241351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1601241351},{"type":"v1","addr":"172.21.15.133:6833","nonce":1601241351}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:42:45.491515+0000","modified":"2023-10-11T09:42:46.610488+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-11T09:42:47.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.056+0000 7f8bcaceb700 1 -- 172.21.15.78:0/1572169821 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8ba8065ed0 msgr2=0x7f8ba8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:47.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.056+0000 7f8bcaceb700 1 --2- 172.21.15.78:0/1572169821 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8ba8065ed0 0x7f8ba8068380 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8bc40c8c20 tx=0x7f8bb4008040 comp rx=0 tx=0).stop 2023-10-11T09:42:47.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.056+0000 7f8bcaceb700 1 -- 172.21.15.78:0/1572169821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8bc40cda10 msgr2=0x7f8bc41632d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:47.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.056+0000 7f8bcaceb700 1 --2- 172.21.15.78:0/1572169821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8bc40cda10 0x7f8bc41632d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0000c00 tx=0x7f8bc0004770 comp rx=0 tx=0).stop 2023-10-11T09:42:47.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.056+0000 7f8bcaceb700 1 -- 172.21.15.78:0/1572169821 shutdown_connections 2023-10-11T09:42:47.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.056+0000 7f8bcaceb700 1 --2- 172.21.15.78:0/1572169821 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8ba8065ed0 0x7f8ba8068380 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:47.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.056+0000 7f8bcaceb700 1 --2- 172.21.15.78:0/1572169821 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8bc4164b50 0x7f8bc4144ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:47.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.056+0000 7f8bcaceb700 1 --2- 172.21.15.78:0/1572169821 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bc40cdde0 0x7f8bc4144760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:47.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.056+0000 7f8bcaceb700 1 --2- 172.21.15.78:0/1572169821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8bc40cda10 0x7f8bc41632d0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:47.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.056+0000 7f8bcaceb700 1 -- 172.21.15.78:0/1572169821 >> 172.21.15.78:0/1572169821 conn(0x7f8bc40cea60 msgr2=0x7f8bc4071310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:47.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.056+0000 7f8bcaceb700 1 -- 172.21.15.78:0/1572169821 shutdown_connections 2023-10-11T09:42:47.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.056+0000 7f8bcaceb700 1 -- 172.21.15.78:0/1572169821 wait complete. 2023-10-11T09:42:47.063 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 22 2023-10-11T09:42:47.079 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-11T09:42:47.080 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:42:47.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.244+0000 7f3135ed5700 1 Processor -- start 2023-10-11T09:42:47.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.244+0000 7f3135ed5700 1 -- start start 2023-10-11T09:42:47.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.248+0000 7f3135ed5700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f31300cc820 0x7f3130160c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:47.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.248+0000 7f3135ed5700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f31300b06a0 0x7f31300b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:47.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.248+0000 7f3135ed5700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3130160210 con 0x7f313015d050 2023-10-11T09:42:47.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.248+0000 7f3135ed5700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31300b1020 con 0x7f31300b06a0 2023-10-11T09:42:47.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.248+0000 7f3135ed5700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31301611a0 con 0x7f31300cc820 2023-10-11T09:42:47.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.248+0000 7f31356d4700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f31300b06a0 0x7f31300b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:47.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.248+0000 7f31356d4700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f31300b06a0 0x7f31300b0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57850/0 (socket says 172.21.15.78:57850) 2023-10-11T09:42:47.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.248+0000 7f31356d4700 1 -- 172.21.15.78:0/4258386321 learned_addr learned my addr 172.21.15.78:0/4258386321 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:47.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.248+0000 7f3127fff700 1 --2- 172.21.15.78:0/4258386321 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f31300cc820 0x7f3130160c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:47.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.248+0000 7f31356d4700 1 -- 172.21.15.78:0/4258386321 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f31300cc820 msgr2=0x7f3130160c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:47.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.248+0000 7f31356d4700 1 --2- 172.21.15.78:0/4258386321 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f31300cc820 0x7f3130160c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:47.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.248+0000 7f31356d4700 1 -- 172.21.15.78:0/4258386321 >> v1:172.21.15.133:6789/0 conn(0x7f313015d050 legacy=0x7f313015d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:47.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.248+0000 7f31356d4700 1 -- 172.21.15.78:0/4258386321 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31300ccfc0 con 0x7f31300b06a0 2023-10-11T09:42:47.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.248+0000 7f31356d4700 1 --2- 172.21.15.78:0/4258386321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31300b06a0 0x7f31300b0ae0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3128009e50 tx=0x7f312800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7664e6615b103e5f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:47.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.252+0000 7f31277fe700 1 -- 172.21.15.78:0/4258386321 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312800e070 con 0x7f31300b06a0 2023-10-11T09:42:47.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.252+0000 7f31277fe700 1 -- 172.21.15.78:0/4258386321 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3128003a40 con 0x7f31300b06a0 2023-10-11T09:42:47.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.252+0000 7f31277fe700 1 -- 172.21.15.78:0/4258386321 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3128003df0 con 0x7f31300b06a0 2023-10-11T09:42:47.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.252+0000 7f3135ed5700 1 -- 172.21.15.78:0/4258386321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31300b06a0 msgr2=0x7f31300b0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:47.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.252+0000 7f3135ed5700 1 --2- 172.21.15.78:0/4258386321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31300b06a0 0x7f31300b0ae0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3128009e50 tx=0x7f312800b330 comp rx=0 tx=0).stop 2023-10-11T09:42:47.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.252+0000 7f3135ed5700 1 -- 172.21.15.78:0/4258386321 shutdown_connections 2023-10-11T09:42:47.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.252+0000 7f3135ed5700 1 --2- 172.21.15.78:0/4258386321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31300b06a0 0x7f31300b0ae0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:47.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.252+0000 7f3135ed5700 1 --2- 172.21.15.78:0/4258386321 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f31300cc820 0x7f3130160c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:47.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.252+0000 7f3135ed5700 1 -- 172.21.15.78:0/4258386321 >> 172.21.15.78:0/4258386321 conn(0x7f31300cea60 msgr2=0x7f31300cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:47.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.252+0000 7f3135ed5700 1 -- 172.21.15.78:0/4258386321 shutdown_connections 2023-10-11T09:42:47.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.252+0000 7f3135ed5700 1 -- 172.21.15.78:0/4258386321 wait complete. 2023-10-11T09:42:47.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.256+0000 7f3135ed5700 1 Processor -- start 2023-10-11T09:42:47.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.256+0000 7f3135ed5700 1 -- start start 2023-10-11T09:42:47.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.256+0000 7f3135ed5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31300b06a0 0x7f31301f5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:47.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.256+0000 7f3135ed5700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f31300cc820 0x7f3130145c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:47.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.256+0000 7f3135ed5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f313015d050 0x7f3130151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:47.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.256+0000 7f3135ed5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31301e6190 con 0x7f31300b06a0 2023-10-11T09:42:47.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.256+0000 7f3135ed5700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31301f1d30 con 0x7f313015d050 2023-10-11T09:42:47.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.256+0000 7f3135ed5700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31301f6370 con 0x7f31300cc820 2023-10-11T09:42:47.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.256+0000 7f3134ed3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31300b06a0 0x7f31301f5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:47.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.256+0000 7f3134ed3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31300b06a0 0x7f31301f5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57862/0 (socket says 172.21.15.78:57862) 2023-10-11T09:42:47.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.256+0000 7f3127fff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f31300cc820 0x7f3130145c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:47.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.256+0000 7f3134ed3700 1 -- 172.21.15.78:0/3958480712 learned_addr learned my addr 172.21.15.78:0/3958480712 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:47.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.256+0000 7f31356d4700 1 --2- 172.21.15.78:0/3958480712 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f313015d050 0x7f3130151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:47.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.256+0000 7f3134ed3700 1 -- 172.21.15.78:0/3958480712 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f31300cc820 msgr2=0x7f3130145c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:47.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.256+0000 7f3134ed3700 1 --2- 172.21.15.78:0/3958480712 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f31300cc820 0x7f3130145c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:47.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.260+0000 7f3134ed3700 1 -- 172.21.15.78:0/3958480712 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f313015d050 msgr2=0x7f3130151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:47.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.260+0000 7f3134ed3700 1 --2- 172.21.15.78:0/3958480712 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f313015d050 0x7f3130151ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:47.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.260+0000 7f3134ed3700 1 -- 172.21.15.78:0/3958480712 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3128009b30 con 0x7f31300b06a0 2023-10-11T09:42:47.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.260+0000 7f3134ed3700 1 --2- 172.21.15.78:0/3958480712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31300b06a0 0x7f31301f5e00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f312c00c4e0 tx=0x7f312c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:47.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.260+0000 7f31257fa700 1 -- 172.21.15.78:0/3958480712 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312c011790 con 0x7f31300b06a0 2023-10-11T09:42:47.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.260+0000 7f31257fa700 1 -- 172.21.15.78:0/3958480712 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f312c00ce90 con 0x7f31300b06a0 2023-10-11T09:42:47.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.260+0000 7f31257fa700 1 -- 172.21.15.78:0/3958480712 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312c011b50 con 0x7f31300b06a0 2023-10-11T09:42:47.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.260+0000 7f3135ed5700 1 -- 172.21.15.78:0/3958480712 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3130150d00 con 0x7f31300b06a0 2023-10-11T09:42:47.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.260+0000 7f3135ed5700 1 -- 172.21.15.78:0/3958480712 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31301512c0 con 0x7f31300b06a0 2023-10-11T09:42:47.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.260+0000 7f3135ed5700 1 -- 172.21.15.78:0/3958480712 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3130065d40 con 0x7f31300b06a0 2023-10-11T09:42:47.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.264+0000 7f31257fa700 1 -- 172.21.15.78:0/3958480712 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f312c011cb0 con 0x7f31300b06a0 2023-10-11T09:42:47.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.264+0000 7f31257fa700 1 --2- 172.21.15.78:0/3958480712 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f311c065ed0 0x7f311c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:47.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.264+0000 7f31257fa700 1 -- 172.21.15.78:0/3958480712 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6667+0+0 (secure 0 0 0) 0x7f312c083700 con 0x7f31300b06a0 2023-10-11T09:42:47.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.264+0000 7f3127fff700 1 --2- 172.21.15.78:0/3958480712 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f311c065ed0 0x7f311c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:47.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.264+0000 7f3127fff700 1 --2- 172.21.15.78:0/3958480712 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f311c065ed0 0x7f311c068380 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f31300ae5f0 tx=0x7f3120008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:47.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.268+0000 7f31257fa700 1 -- 172.21.15.78:0/3958480712 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f312c050af0 con 0x7f31300b06a0 2023-10-11T09:42:47.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.476+0000 7f3135ed5700 1 -- 172.21.15.78:0/3958480712 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f313015c760 con 0x7f31300b06a0 2023-10-11T09:42:47.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.476+0000 7f31257fa700 1 -- 172.21.15.78:0/3958480712 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4642 (secure 0 0 0) 0x7f312c054110 con 0x7f31300b06a0 2023-10-11T09:42:47.479 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:47.479 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5308,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/655413753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":655413753},{"type":"v1","addr":"172.21.15.133:6835","nonce":655413753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5352,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2966622437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2966622437},{"type":"v1","addr":"172.21.15.78:6835","nonce":2966622437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5355,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/700246445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":700246445},{"type":"v1","addr":"172.21.15.78:6837","nonce":700246445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5641,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/1601241351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1601241351},{"type":"v1","addr":"172.21.15.133:6833","nonce":1601241351}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:42:45.491515+0000","modified":"2023-10-11T09:42:46.610488+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-11T09:42:47.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.476+0000 7f3135ed5700 1 -- 172.21.15.78:0/3958480712 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f311c065ed0 msgr2=0x7f311c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:47.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.476+0000 7f3135ed5700 1 --2- 172.21.15.78:0/3958480712 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f311c065ed0 0x7f311c068380 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f31300ae5f0 tx=0x7f3120008040 comp rx=0 tx=0).stop 2023-10-11T09:42:47.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.476+0000 7f3135ed5700 1 -- 172.21.15.78:0/3958480712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31300b06a0 msgr2=0x7f31301f5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:47.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.476+0000 7f3135ed5700 1 --2- 172.21.15.78:0/3958480712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31300b06a0 0x7f31301f5e00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f312c00c4e0 tx=0x7f312c00c7f0 comp rx=0 tx=0).stop 2023-10-11T09:42:47.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.480+0000 7f3135ed5700 1 -- 172.21.15.78:0/3958480712 shutdown_connections 2023-10-11T09:42:47.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.480+0000 7f3135ed5700 1 --2- 172.21.15.78:0/3958480712 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f311c065ed0 0x7f311c068380 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:47.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.480+0000 7f3135ed5700 1 --2- 172.21.15.78:0/3958480712 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f313015d050 0x7f3130151ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:47.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.480+0000 7f3135ed5700 1 --2- 172.21.15.78:0/3958480712 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f31300cc820 0x7f3130145c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:47.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.480+0000 7f3135ed5700 1 --2- 172.21.15.78:0/3958480712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31300b06a0 0x7f31301f5e00 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:47.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.480+0000 7f3135ed5700 1 -- 172.21.15.78:0/3958480712 >> 172.21.15.78:0/3958480712 conn(0x7f31300cea60 msgr2=0x7f313006c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:47.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.480+0000 7f3135ed5700 1 -- 172.21.15.78:0/3958480712 shutdown_connections 2023-10-11T09:42:47.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.480+0000 7f3135ed5700 1 -- 172.21.15.78:0/3958480712 wait complete. 2023-10-11T09:42:47.484 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 22 2023-10-11T09:42:47.502 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:42:47.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.672+0000 7f75b6184700 1 Processor -- start 2023-10-11T09:42:47.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.672+0000 7f75b6184700 1 -- start start 2023-10-11T09:42:47.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.676+0000 7f75b6184700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f75b015a940 0x7f75b00ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:47.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.676+0000 7f75b6184700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f75b00ca780 0x7f75b013fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:47.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.676+0000 7f75b6184700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b015e010 con 0x7f75b015b8e0 2023-10-11T09:42:47.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.676+0000 7f75b6184700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75b00cab70 con 0x7f75b00ca780 2023-10-11T09:42:47.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.676+0000 7f75b6184700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75b015e280 con 0x7f75b015a940 2023-10-11T09:42:47.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.676+0000 7f75b5983700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f75b00ca780 0x7f75b013fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:47.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.676+0000 7f75b4981700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f75b015a940 0x7f75b00ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:47.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.676+0000 7f75b4981700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f75b015a940 0x7f75b00ca240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:33544/0 (socket says 172.21.15.78:33544) 2023-10-11T09:42:47.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.676+0000 7f75b4981700 1 -- 172.21.15.78:0/1425264694 learned_addr learned my addr 172.21.15.78:0/1425264694 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:47.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.676+0000 7f75b5182700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f75b015b8e0 0x7f75b006d0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:46046/0 (socket says 172.21.15.78:46046) 2023-10-11T09:42:47.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.676+0000 7f75b4981700 1 -- 172.21.15.78:0/1425264694 >> v2:172.21.15.78:3300/0 conn(0x7f75b00ca780 msgr2=0x7f75b013fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:47.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.676+0000 7f75b4981700 1 --2- 172.21.15.78:0/1425264694 >> v2:172.21.15.78:3300/0 conn(0x7f75b00ca780 0x7f75b013fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:47.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.676+0000 7f75b4981700 1 -- 172.21.15.78:0/1425264694 >> v1:172.21.15.133:6789/0 conn(0x7f75b015b8e0 legacy=0x7f75b006d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:47.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.676+0000 7f75b4981700 1 -- 172.21.15.78:0/1425264694 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b015e560 con 0x7f75b015a940 2023-10-11T09:42:47.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.676+0000 7f75b5983700 1 --2- 172.21.15.78:0/1425264694 >> v2:172.21.15.78:3300/0 conn(0x7f75b00ca780 0x7f75b013fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:42:47.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.680+0000 7f75b4981700 1 --2- 172.21.15.78:0/1425264694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75b015a940 0x7f75b00ca240 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f75b006f800 tx=0x7f75a0009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db80ab6ecaeeccbb server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:47.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.680+0000 7f75a7fff700 1 -- 172.21.15.78:0/1425264694 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a0011830 con 0x7f75b015a940 2023-10-11T09:42:47.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.680+0000 7f75a7fff700 1 -- 172.21.15.78:0/1425264694 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a0011990 con 0x7f75b015a940 2023-10-11T09:42:47.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.680+0000 7f75a7fff700 1 -- 172.21.15.78:0/1425264694 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a0011d20 con 0x7f75b015a940 2023-10-11T09:42:47.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.680+0000 7f75b6184700 1 -- 172.21.15.78:0/1425264694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75b015a940 msgr2=0x7f75b00ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:47.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.680+0000 7f75b6184700 1 --2- 172.21.15.78:0/1425264694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75b015a940 0x7f75b00ca240 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f75b006f800 tx=0x7f75a0009bf0 comp rx=0 tx=0).stop 2023-10-11T09:42:47.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.680+0000 7f75b6184700 1 -- 172.21.15.78:0/1425264694 shutdown_connections 2023-10-11T09:42:47.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.680+0000 7f75b6184700 1 --2- 172.21.15.78:0/1425264694 >> v2:172.21.15.78:3300/0 conn(0x7f75b00ca780 0x7f75b013fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:47.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.680+0000 7f75b6184700 1 --2- 172.21.15.78:0/1425264694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75b015a940 0x7f75b00ca240 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:47.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.680+0000 7f75b6184700 1 -- 172.21.15.78:0/1425264694 >> 172.21.15.78:0/1425264694 conn(0x7f75b00cea60 msgr2=0x7f75b00cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:47.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.680+0000 7f75b6184700 1 -- 172.21.15.78:0/1425264694 shutdown_connections 2023-10-11T09:42:47.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.680+0000 7f75b6184700 1 -- 172.21.15.78:0/1425264694 wait complete. 2023-10-11T09:42:47.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.684+0000 7f75b6184700 1 Processor -- start 2023-10-11T09:42:47.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.684+0000 7f75b6184700 1 -- start start 2023-10-11T09:42:47.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.684+0000 7f75b6184700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f75b00ca780 0x7f75b01362e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:47.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.684+0000 7f75b6184700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75b015a940 0x7f75b00b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:47.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.684+0000 7f75b6184700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75b015b8e0 0x7f75b0155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:47.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.684+0000 7f75b6184700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b00b3eb0 con 0x7f75b015b8e0 2023-10-11T09:42:47.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.684+0000 7f75b6184700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b00b3330 con 0x7f75b015a940 2023-10-11T09:42:47.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.684+0000 7f75b6184700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b00b3610 con 0x7f75b00ca780 2023-10-11T09:42:47.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.684+0000 7f75b5983700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75b015b8e0 0x7f75b0155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:47.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.684+0000 7f75b5983700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75b015b8e0 0x7f75b0155430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57884/0 (socket says 172.21.15.78:57884) 2023-10-11T09:42:47.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.684+0000 7f75b5983700 1 -- 172.21.15.78:0/3269817752 learned_addr learned my addr 172.21.15.78:0/3269817752 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:47.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.684+0000 7f75b4981700 1 --2- 172.21.15.78:0/3269817752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75b015a940 0x7f75b00b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:47.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.684+0000 7f75b5182700 1 --2- 172.21.15.78:0/3269817752 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f75b00ca780 0x7f75b01362e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:47.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.684+0000 7f75b5983700 1 -- 172.21.15.78:0/3269817752 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f75b00ca780 msgr2=0x7f75b01362e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:47.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.684+0000 7f75b5983700 1 --2- 172.21.15.78:0/3269817752 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f75b00ca780 0x7f75b01362e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:47.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.684+0000 7f75b5983700 1 -- 172.21.15.78:0/3269817752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75b015a940 msgr2=0x7f75b00b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:47.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.684+0000 7f75b5983700 1 --2- 172.21.15.78:0/3269817752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75b015a940 0x7f75b00b2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:47.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.684+0000 7f75b5983700 1 -- 172.21.15.78:0/3269817752 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a8009a70 con 0x7f75b015b8e0 2023-10-11T09:42:47.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.684+0000 7f75b5182700 1 --2- 172.21.15.78:0/3269817752 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f75b00ca780 0x7f75b01362e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:42:47.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.688+0000 7f75b5983700 1 --2- 172.21.15.78:0/3269817752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75b015b8e0 0x7f75b0155430 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f75a800efc0 tx=0x7f75a800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:47.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.688+0000 7f75a5ffb700 1 -- 172.21.15.78:0/3269817752 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a800cb40 con 0x7f75b015b8e0 2023-10-11T09:42:47.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.688+0000 7f75a5ffb700 1 -- 172.21.15.78:0/3269817752 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a800f040 con 0x7f75b015b8e0 2023-10-11T09:42:47.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.688+0000 7f75a5ffb700 1 -- 172.21.15.78:0/3269817752 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a8003a40 con 0x7f75b015b8e0 2023-10-11T09:42:47.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.688+0000 7f75b6184700 1 -- 172.21.15.78:0/3269817752 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75a0009570 con 0x7f75b015b8e0 2023-10-11T09:42:47.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.688+0000 7f75b6184700 1 -- 172.21.15.78:0/3269817752 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75b00b3bd0 con 0x7f75b015b8e0 2023-10-11T09:42:47.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.688+0000 7f75b6184700 1 -- 172.21.15.78:0/3269817752 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75b0136cd0 con 0x7f75b015b8e0 2023-10-11T09:42:47.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.688+0000 7f75a5ffb700 1 -- 172.21.15.78:0/3269817752 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f75a8014480 con 0x7f75b015b8e0 2023-10-11T09:42:47.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.692+0000 7f75a5ffb700 1 --2- 172.21.15.78:0/3269817752 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7594065ed0 0x7f7594068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:47.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.692+0000 7f75a5ffb700 1 -- 172.21.15.78:0/3269817752 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6667+0+0 (secure 0 0 0) 0x7f75a8084480 con 0x7f75b015b8e0 2023-10-11T09:42:47.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.692+0000 7f75b5182700 1 --2- 172.21.15.78:0/3269817752 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7594065ed0 0x7f7594068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:47.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.692+0000 7f75b5182700 1 --2- 172.21.15.78:0/3269817752 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7594065ed0 0x7f7594068380 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f75ac000f60 tx=0x7f75ac007df0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:47.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.696+0000 7f75a5ffb700 1 -- 172.21.15.78:0/3269817752 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f75b0136cd0 con 0x7f75b015b8e0 2023-10-11T09:42:47.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.908+0000 7f75b6184700 1 -- 172.21.15.78:0/3269817752 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75b0136cd0 con 0x7f75b015b8e0 2023-10-11T09:42:47.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.908+0000 7f75a5ffb700 1 -- 172.21.15.78:0/3269817752 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4642 (secure 0 0 0) 0x7f75b0136cd0 con 0x7f75b015b8e0 2023-10-11T09:42:47.911 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:47.911 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5308,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/655413753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":655413753},{"type":"v1","addr":"172.21.15.133:6835","nonce":655413753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5352,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2966622437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2966622437},{"type":"v1","addr":"172.21.15.78:6835","nonce":2966622437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5355,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/700246445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":700246445},{"type":"v1","addr":"172.21.15.78:6837","nonce":700246445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5641,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/1601241351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1601241351},{"type":"v1","addr":"172.21.15.133:6833","nonce":1601241351}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:42:45.491515+0000","modified":"2023-10-11T09:42:46.610488+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-11T09:42:47.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.912+0000 7f75b6184700 1 -- 172.21.15.78:0/3269817752 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7594065ed0 msgr2=0x7f7594068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:47.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.912+0000 7f75b6184700 1 --2- 172.21.15.78:0/3269817752 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7594065ed0 0x7f7594068380 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f75ac000f60 tx=0x7f75ac007df0 comp rx=0 tx=0).stop 2023-10-11T09:42:47.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.912+0000 7f75b6184700 1 -- 172.21.15.78:0/3269817752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75b015b8e0 msgr2=0x7f75b0155430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:47.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.912+0000 7f75b6184700 1 --2- 172.21.15.78:0/3269817752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75b015b8e0 0x7f75b0155430 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f75a800efc0 tx=0x7f75a800c3a0 comp rx=0 tx=0).stop 2023-10-11T09:42:47.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.912+0000 7f75b6184700 1 -- 172.21.15.78:0/3269817752 shutdown_connections 2023-10-11T09:42:47.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.912+0000 7f75b6184700 1 --2- 172.21.15.78:0/3269817752 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7594065ed0 0x7f7594068380 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:47.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.912+0000 7f75b6184700 1 --2- 172.21.15.78:0/3269817752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75b015b8e0 0x7f75b0155430 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:47.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.912+0000 7f75b6184700 1 --2- 172.21.15.78:0/3269817752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75b015a940 0x7f75b00b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:47.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.912+0000 7f75b6184700 1 --2- 172.21.15.78:0/3269817752 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f75b00ca780 0x7f75b01362e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:47.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.912+0000 7f75b6184700 1 -- 172.21.15.78:0/3269817752 >> 172.21.15.78:0/3269817752 conn(0x7f75b00cea60 msgr2=0x7f75b00c9da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:47.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.912+0000 7f75b6184700 1 -- 172.21.15.78:0/3269817752 shutdown_connections 2023-10-11T09:42:47.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:47.912+0000 7f75b6184700 1 -- 172.21.15.78:0/3269817752 wait complete. 2023-10-11T09:42:47.917 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 22 2023-10-11T09:42:47.934 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:42:48.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.096+0000 7f83a1cae700 1 Processor -- start 2023-10-11T09:42:48.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.096+0000 7f83a1cae700 1 -- start start 2023-10-11T09:42:48.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.100+0000 7f83a1cae700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f839c15c880 0x7f839c06be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:48.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.100+0000 7f83a1cae700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f839c0cd210 0x7f839c0cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:48.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.100+0000 7f83a1cae700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f839c160e60 0x7f839c0cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:48.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.100+0000 7f83a1cae700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f839c161420 con 0x7f839c0cd210 2023-10-11T09:42:48.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.100+0000 7f83a1cae700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f839c0c93a0 con 0x7f839c15c880 2023-10-11T09:42:48.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.100+0000 7f83a1cae700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f839c0cca10 con 0x7f839c160e60 2023-10-11T09:42:48.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.100+0000 7f8392ffd700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f839c0cd210 0x7f839c0cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:48.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.100+0000 7f8392ffd700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f839c0cd210 0x7f839c0cb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57892/0 (socket says 172.21.15.78:57892) 2023-10-11T09:42:48.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.100+0000 7f8392ffd700 1 -- 172.21.15.78:0/3236326836 learned_addr learned my addr 172.21.15.78:0/3236326836 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:48.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.100+0000 7f839b7fe700 1 --2- 172.21.15.78:0/3236326836 >> v2:172.21.15.133:3300/0 conn(0x7f839c15c880 0x7f839c06be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:48.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.100+0000 7f839bfff700 1 --2- 172.21.15.78:0/3236326836 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f839c160e60 0x7f839c0cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:48.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.100+0000 7f8392ffd700 1 -- 172.21.15.78:0/3236326836 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f839c160e60 msgr2=0x7f839c0cbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:48.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.100+0000 7f8392ffd700 1 --2- 172.21.15.78:0/3236326836 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f839c160e60 0x7f839c0cbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.100+0000 7f8392ffd700 1 -- 172.21.15.78:0/3236326836 >> v2:172.21.15.133:3300/0 conn(0x7f839c15c880 msgr2=0x7f839c06be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:48.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.100+0000 7f8392ffd700 1 --2- 172.21.15.78:0/3236326836 >> v2:172.21.15.133:3300/0 conn(0x7f839c15c880 0x7f839c06be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.100+0000 7f8392ffd700 1 -- 172.21.15.78:0/3236326836 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f839c0cccb0 con 0x7f839c0cd210 2023-10-11T09:42:48.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.100+0000 7f839b7fe700 1 --2- 172.21.15.78:0/3236326836 >> v2:172.21.15.133:3300/0 conn(0x7f839c15c880 0x7f839c06be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:42:48.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.100+0000 7f8392ffd700 1 --2- 172.21.15.78:0/3236326836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f839c0cd210 0x7f839c0cb620 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f839c06d8e0 tx=0x7f838400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9dee732aac16e20 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:48.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.100+0000 7f839affd700 1 -- 172.21.15.78:0/3236326836 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838400e070 con 0x7f839c0cd210 2023-10-11T09:42:48.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.100+0000 7f839affd700 1 -- 172.21.15.78:0/3236326836 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f838400b9c0 con 0x7f839c0cd210 2023-10-11T09:42:48.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.100+0000 7f839affd700 1 -- 172.21.15.78:0/3236326836 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838400bd70 con 0x7f839c0cd210 2023-10-11T09:42:48.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.100+0000 7f83a1cae700 1 -- 172.21.15.78:0/3236326836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f839c0cd210 msgr2=0x7f839c0cb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:48.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.100+0000 7f83a1cae700 1 --2- 172.21.15.78:0/3236326836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f839c0cd210 0x7f839c0cb620 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f839c06d8e0 tx=0x7f838400b330 comp rx=0 tx=0).stop 2023-10-11T09:42:48.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.104+0000 7f83a1cae700 1 -- 172.21.15.78:0/3236326836 shutdown_connections 2023-10-11T09:42:48.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.104+0000 7f83a1cae700 1 --2- 172.21.15.78:0/3236326836 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f839c160e60 0x7f839c0cbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.104+0000 7f83a1cae700 1 --2- 172.21.15.78:0/3236326836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f839c0cd210 0x7f839c0cb620 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.104+0000 7f83a1cae700 1 --2- 172.21.15.78:0/3236326836 >> v2:172.21.15.133:3300/0 conn(0x7f839c15c880 0x7f839c06be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.104+0000 7f83a1cae700 1 -- 172.21.15.78:0/3236326836 >> 172.21.15.78:0/3236326836 conn(0x7f839c06aca0 msgr2=0x7f839c0b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:48.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.104+0000 7f83a1cae700 1 -- 172.21.15.78:0/3236326836 shutdown_connections 2023-10-11T09:42:48.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.104+0000 7f83a1cae700 1 -- 172.21.15.78:0/3236326836 wait complete. 2023-10-11T09:42:48.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.104+0000 7f83a1cae700 1 Processor -- start 2023-10-11T09:42:48.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.104+0000 7f83a1cae700 1 -- start start 2023-10-11T09:42:48.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.108+0000 7f83a1cae700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f839c0cd210 0x7f839c1478e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:48.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.108+0000 7f83a1cae700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f839c15c880 0x7f839c0afb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:48.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.108+0000 7f83a1cae700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f839c160e60 0x7f839c0b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:48.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.108+0000 7f83a1cae700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839c15ad10 con 0x7f839c160e60 2023-10-11T09:42:48.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.108+0000 7f83a1cae700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839c1f5b40 con 0x7f839c15c880 2023-10-11T09:42:48.114 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.108+0000 7f83a1cae700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f839c1f6680 con 0x7f839c0cd210 2023-10-11T09:42:48.114 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.108+0000 7f839bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f839c160e60 0x7f839c0b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:48.114 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.108+0000 7f839bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f839c160e60 0x7f839c0b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57896/0 (socket says 172.21.15.78:57896) 2023-10-11T09:42:48.114 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.108+0000 7f839bfff700 1 -- 172.21.15.78:0/810452805 learned_addr learned my addr 172.21.15.78:0/810452805 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:48.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.108+0000 7f8392ffd700 1 --2- 172.21.15.78:0/810452805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f839c15c880 0x7f839c0afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:48.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.108+0000 7f839b7fe700 1 --2- 172.21.15.78:0/810452805 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f839c0cd210 0x7f839c1478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:48.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.108+0000 7f839bfff700 1 -- 172.21.15.78:0/810452805 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f839c0cd210 msgr2=0x7f839c1478e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:48.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.108+0000 7f839bfff700 1 --2- 172.21.15.78:0/810452805 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f839c0cd210 0x7f839c1478e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.108+0000 7f839bfff700 1 -- 172.21.15.78:0/810452805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f839c15c880 msgr2=0x7f839c0afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:48.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.108+0000 7f839bfff700 1 --2- 172.21.15.78:0/810452805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f839c15c880 0x7f839c0afb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.108+0000 7f839bfff700 1 -- 172.21.15.78:0/810452805 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8384009b30 con 0x7f839c160e60 2023-10-11T09:42:48.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.108+0000 7f839bfff700 1 --2- 172.21.15.78:0/810452805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f839c160e60 0x7f839c0b00c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f839400e810 tx=0x7f839400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:48.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.108+0000 7f8398ff9700 1 -- 172.21.15.78:0/810452805 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8394013610 con 0x7f839c160e60 2023-10-11T09:42:48.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.108+0000 7f8398ff9700 1 -- 172.21.15.78:0/810452805 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8394013770 con 0x7f839c160e60 2023-10-11T09:42:48.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.108+0000 7f8398ff9700 1 -- 172.21.15.78:0/810452805 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8394013b20 con 0x7f839c160e60 2023-10-11T09:42:48.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.108+0000 7f83a1cae700 1 -- 172.21.15.78:0/810452805 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f839c147e20 con 0x7f839c160e60 2023-10-11T09:42:48.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.108+0000 7f83a1cae700 1 -- 172.21.15.78:0/810452805 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f839c1f6030 con 0x7f839c160e60 2023-10-11T09:42:48.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.112+0000 7f83a1cae700 1 -- 172.21.15.78:0/810452805 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f839c1605c0 con 0x7f839c160e60 2023-10-11T09:42:48.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.112+0000 7f8398ff9700 1 -- 172.21.15.78:0/810452805 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f8394013c80 con 0x7f839c160e60 2023-10-11T09:42:48.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.112+0000 7f8398ff9700 1 --2- 172.21.15.78:0/810452805 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f837c065ed0 0x7f837c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:48.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.116+0000 7f8398ff9700 1 -- 172.21.15.78:0/810452805 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6667+0+0 (secure 0 0 0) 0x7f8394083660 con 0x7f839c160e60 2023-10-11T09:42:48.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.116+0000 7f839b7fe700 1 --2- 172.21.15.78:0/810452805 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f837c065ed0 0x7f837c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:48.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.116+0000 7f839b7fe700 1 --2- 172.21.15.78:0/810452805 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f837c065ed0 0x7f837c068380 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f838c003910 tx=0x7f838c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:48.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.120+0000 7f8398ff9700 1 -- 172.21.15.78:0/810452805 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f8394050a60 con 0x7f839c160e60 2023-10-11T09:42:48.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.328+0000 7f83a1cae700 1 -- 172.21.15.78:0/810452805 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f839c04e530 con 0x7f839c160e60 2023-10-11T09:42:48.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.328+0000 7f8398ff9700 1 -- 172.21.15.78:0/810452805 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4642 (secure 0 0 0) 0x7f8394054080 con 0x7f839c160e60 2023-10-11T09:42:48.332 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:48.332 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5308,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/655413753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":655413753},{"type":"v1","addr":"172.21.15.133:6835","nonce":655413753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5352,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2966622437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2966622437},{"type":"v1","addr":"172.21.15.78:6835","nonce":2966622437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5355,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/700246445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":700246445},{"type":"v1","addr":"172.21.15.78:6837","nonce":700246445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5641,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/1601241351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1601241351},{"type":"v1","addr":"172.21.15.133:6833","nonce":1601241351}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:42:45.491515+0000","modified":"2023-10-11T09:42:46.610488+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-11T09:42:48.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.332+0000 7f83a1cae700 1 -- 172.21.15.78:0/810452805 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f837c065ed0 msgr2=0x7f837c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:48.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.332+0000 7f83a1cae700 1 --2- 172.21.15.78:0/810452805 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f837c065ed0 0x7f837c068380 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f838c003910 tx=0x7f838c008040 comp rx=0 tx=0).stop 2023-10-11T09:42:48.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.332+0000 7f83a1cae700 1 -- 172.21.15.78:0/810452805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f839c160e60 msgr2=0x7f839c0b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:48.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.332+0000 7f83a1cae700 1 --2- 172.21.15.78:0/810452805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f839c160e60 0x7f839c0b00c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f839400e810 tx=0x7f839400eb20 comp rx=0 tx=0).stop 2023-10-11T09:42:48.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.332+0000 7f83a1cae700 1 -- 172.21.15.78:0/810452805 shutdown_connections 2023-10-11T09:42:48.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.332+0000 7f83a1cae700 1 --2- 172.21.15.78:0/810452805 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f837c065ed0 0x7f837c068380 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.332+0000 7f83a1cae700 1 --2- 172.21.15.78:0/810452805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f839c160e60 0x7f839c0b00c0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.332+0000 7f83a1cae700 1 --2- 172.21.15.78:0/810452805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f839c15c880 0x7f839c0afb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.332+0000 7f83a1cae700 1 --2- 172.21.15.78:0/810452805 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f839c0cd210 0x7f839c1478e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.332+0000 7f83a1cae700 1 -- 172.21.15.78:0/810452805 >> 172.21.15.78:0/810452805 conn(0x7f839c06aca0 msgr2=0x7f839c151b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:48.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.332+0000 7f83a1cae700 1 -- 172.21.15.78:0/810452805 shutdown_connections 2023-10-11T09:42:48.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.332+0000 7f83a1cae700 1 -- 172.21.15.78:0/810452805 wait complete. 2023-10-11T09:42:48.337 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 22 2023-10-11T09:42:48.358 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:42:48.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.524+0000 7fee09071700 1 Processor -- start 2023-10-11T09:42:48.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.524+0000 7fee09071700 1 -- start start 2023-10-11T09:42:48.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.524+0000 7fee09071700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fee040b5f30 0x7fee040b63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:48.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.524+0000 7fee09071700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee040d11e0 con 0x7fee040b2310 2023-10-11T09:42:48.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.524+0000 7fee09071700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee040b5370 con 0x7fee04161150 2023-10-11T09:42:48.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.524+0000 7fee09071700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee041615b0 con 0x7fee040b5f30 2023-10-11T09:42:48.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.528+0000 7fee0359e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fee040b5f30 0x7fee040b63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:48.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.528+0000 7fee02d9d700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fee040b2310 0x7fee040b4b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:59342/0 (socket says 172.21.15.78:59342) 2023-10-11T09:42:48.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.528+0000 7fee02d9d700 1 -- 172.21.15.78:0/3779388991 learned_addr learned my addr 172.21.15.78:0/3779388991 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:42:48.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.528+0000 7fee0359e700 1 -- 172.21.15.78:0/3779388991 >> v1:172.21.15.133:6789/0 conn(0x7fee04161150 legacy=0x7fee040d1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:48.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.528+0000 7fee0359e700 1 -- 172.21.15.78:0/3779388991 >> v1:172.21.15.78:6789/0 conn(0x7fee040b2310 legacy=0x7fee040b4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:48.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.528+0000 7fee0359e700 1 -- 172.21.15.78:0/3779388991 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee040d14a0 con 0x7fee040b5f30 2023-10-11T09:42:48.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.528+0000 7fee0359e700 1 --2- 172.21.15.78:0/3779388991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee040b5f30 0x7fee040b63b0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fedfc009d90 tx=0x7fedfc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e817327c758e0db1 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:48.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.528+0000 7fee01d9b700 1 -- 172.21.15.78:0/3779388991 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedfc00e070 con 0x7fee040b5f30 2023-10-11T09:42:48.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.528+0000 7fee01d9b700 1 -- 172.21.15.78:0/3779388991 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedfc00bb90 con 0x7fee040b5f30 2023-10-11T09:42:48.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.528+0000 7fee01d9b700 1 -- 172.21.15.78:0/3779388991 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedfc003a40 con 0x7fee040b5f30 2023-10-11T09:42:48.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.528+0000 7fee09071700 1 -- 172.21.15.78:0/3779388991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee040b5f30 msgr2=0x7fee040b63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:48.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.528+0000 7fee09071700 1 --2- 172.21.15.78:0/3779388991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee040b5f30 0x7fee040b63b0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fedfc009d90 tx=0x7fedfc00b330 comp rx=0 tx=0).stop 2023-10-11T09:42:48.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.528+0000 7fee09071700 1 -- 172.21.15.78:0/3779388991 shutdown_connections 2023-10-11T09:42:48.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.528+0000 7fee09071700 1 --2- 172.21.15.78:0/3779388991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee040b5f30 0x7fee040b63b0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.528+0000 7fee09071700 1 -- 172.21.15.78:0/3779388991 >> 172.21.15.78:0/3779388991 conn(0x7fee040cfb60 msgr2=0x7fee040cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:48.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.528+0000 7fee09071700 1 -- 172.21.15.78:0/3779388991 shutdown_connections 2023-10-11T09:42:48.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.528+0000 7fee09071700 1 -- 172.21.15.78:0/3779388991 wait complete. 2023-10-11T09:42:48.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.532+0000 7fee09071700 1 Processor -- start 2023-10-11T09:42:48.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.532+0000 7fee09071700 1 -- start start 2023-10-11T09:42:48.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.532+0000 7fee09071700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee040b2310 0x7fee041f66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:48.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.532+0000 7fee09071700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fee040b5f30 0x7fee040b7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:48.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.532+0000 7fee09071700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee04161150 0x7fee040b8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:48.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.532+0000 7fee09071700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee041f7d60 con 0x7fee040b2310 2023-10-11T09:42:48.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.532+0000 7fee09071700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee041f7a10 con 0x7fee04161150 2023-10-11T09:42:48.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.532+0000 7fee09071700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee041f7190 con 0x7fee040b5f30 2023-10-11T09:42:48.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.536+0000 7fee02d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee040b2310 0x7fee041f66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:48.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.536+0000 7fee02d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee040b2310 0x7fee041f66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57902/0 (socket says 172.21.15.78:57902) 2023-10-11T09:42:48.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.536+0000 7fee02d9d700 1 -- 172.21.15.78:0/4151639928 learned_addr learned my addr 172.21.15.78:0/4151639928 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:48.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.536+0000 7fee02d9d700 1 -- 172.21.15.78:0/4151639928 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fee040b5f30 msgr2=0x7fee040b7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:48.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.536+0000 7fee02d9d700 1 --2- 172.21.15.78:0/4151639928 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fee040b5f30 0x7fee040b7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.536+0000 7fee02d9d700 1 -- 172.21.15.78:0/4151639928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee04161150 msgr2=0x7fee040b8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:48.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.536+0000 7fee02d9d700 1 --2- 172.21.15.78:0/4151639928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee04161150 0x7fee040b8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.536+0000 7fee02d9d700 1 -- 172.21.15.78:0/4151639928 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedfc009a70 con 0x7fee040b2310 2023-10-11T09:42:48.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.536+0000 7fee02d9d700 1 --2- 172.21.15.78:0/4151639928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee040b2310 0x7fee041f66f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fedf400c500 tx=0x7fedf400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:48.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.536+0000 7fedeb7fe700 1 -- 172.21.15.78:0/4151639928 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf4005d90 con 0x7fee040b2310 2023-10-11T09:42:48.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.536+0000 7fedeb7fe700 1 -- 172.21.15.78:0/4151639928 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedf40070d0 con 0x7fee040b2310 2023-10-11T09:42:48.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.536+0000 7fee09071700 1 -- 172.21.15.78:0/4151639928 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee041f7470 con 0x7fee040b2310 2023-10-11T09:42:48.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.536+0000 7fedeb7fe700 1 -- 172.21.15.78:0/4151639928 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf4007440 con 0x7fee040b2310 2023-10-11T09:42:48.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.536+0000 7fee09071700 1 -- 172.21.15.78:0/4151639928 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee041f6c30 con 0x7fee040b2310 2023-10-11T09:42:48.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.536+0000 7fee09071700 1 -- 172.21.15.78:0/4151639928 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee041f76b0 con 0x7fee040b2310 2023-10-11T09:42:48.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.540+0000 7fedeb7fe700 1 -- 172.21.15.78:0/4151639928 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fedf40075a0 con 0x7fee040b2310 2023-10-11T09:42:48.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.540+0000 7fedeb7fe700 1 --2- 172.21.15.78:0/4151639928 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fedf0065fa0 0x7fedf0068450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:48.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.540+0000 7fedeb7fe700 1 -- 172.21.15.78:0/4151639928 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6667+0+0 (secure 0 0 0) 0x7fedf4082e80 con 0x7fee040b2310 2023-10-11T09:42:48.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.540+0000 7fee0259c700 1 --2- 172.21.15.78:0/4151639928 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fedf0065fa0 0x7fedf0068450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:48.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.540+0000 7fee0259c700 1 --2- 172.21.15.78:0/4151639928 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fedf0065fa0 0x7fedf0068450 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fee040b2d90 tx=0x7fedf8006c70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:48.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.544+0000 7fedeb7fe700 1 -- 172.21.15.78:0/4151639928 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fedf4050300 con 0x7fee040b2310 2023-10-11T09:42:48.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.748+0000 7fee09071700 1 -- 172.21.15.78:0/4151639928 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee0404e530 con 0x7fee040b2310 2023-10-11T09:42:48.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.752+0000 7fedeb7fe700 1 -- 172.21.15.78:0/4151639928 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4642 (secure 0 0 0) 0x7fedf4053920 con 0x7fee040b2310 2023-10-11T09:42:48.754 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:48.754 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5308,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/655413753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":655413753},{"type":"v1","addr":"172.21.15.133:6835","nonce":655413753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5352,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2966622437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2966622437},{"type":"v1","addr":"172.21.15.78:6835","nonce":2966622437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5355,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/700246445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":700246445},{"type":"v1","addr":"172.21.15.78:6837","nonce":700246445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5641,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/1601241351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1601241351},{"type":"v1","addr":"172.21.15.133:6833","nonce":1601241351}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:42:45.491515+0000","modified":"2023-10-11T09:42:46.610488+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-11T09:42:48.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.752+0000 7fee09071700 1 -- 172.21.15.78:0/4151639928 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fedf0065fa0 msgr2=0x7fedf0068450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:48.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.752+0000 7fee09071700 1 --2- 172.21.15.78:0/4151639928 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fedf0065fa0 0x7fedf0068450 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fee040b2d90 tx=0x7fedf8006c70 comp rx=0 tx=0).stop 2023-10-11T09:42:48.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.752+0000 7fee09071700 1 -- 172.21.15.78:0/4151639928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee040b2310 msgr2=0x7fee041f66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:48.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.752+0000 7fee09071700 1 --2- 172.21.15.78:0/4151639928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee040b2310 0x7fee041f66f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fedf400c500 tx=0x7fedf400c8c0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.752+0000 7fee09071700 1 -- 172.21.15.78:0/4151639928 shutdown_connections 2023-10-11T09:42:48.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.752+0000 7fee09071700 1 --2- 172.21.15.78:0/4151639928 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fedf0065fa0 0x7fedf0068450 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.752+0000 7fee09071700 1 --2- 172.21.15.78:0/4151639928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee04161150 0x7fee040b8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.752+0000 7fee09071700 1 --2- 172.21.15.78:0/4151639928 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fee040b5f30 0x7fee040b7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.752+0000 7fee09071700 1 --2- 172.21.15.78:0/4151639928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee040b2310 0x7fee041f66f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.752+0000 7fee09071700 1 -- 172.21.15.78:0/4151639928 >> 172.21.15.78:0/4151639928 conn(0x7fee040cfb60 msgr2=0x7fee040d2530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:48.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.752+0000 7fee09071700 1 -- 172.21.15.78:0/4151639928 shutdown_connections 2023-10-11T09:42:48.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.752+0000 7fee09071700 1 -- 172.21.15.78:0/4151639928 wait complete. 2023-10-11T09:42:48.759 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 22 2023-10-11T09:42:48.779 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset session 2023-10-11T09:42:48.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.836+0000 7f97c4b6b380 1 -- start start 2023-10-11T09:42:48.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.836+0000 7f97c4b6b380 1 Processor -- start 2023-10-11T09:42:48.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.836+0000 7f97c4b6b380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562169cad800 0x562169d52b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:48.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.836+0000 7f97c4b6b380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562169cad400 0x562169d52580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:48.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.836+0000 7f97c4b6b380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562169cad000 0x562169d53080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:48.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.836+0000 7f97c4b6b380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x562169ce3e00 con 0x562169cad400 2023-10-11T09:42:48.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.836+0000 7f97c4b6b380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562169d2ca00 con 0x562169cad800 2023-10-11T09:42:48.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.836+0000 7f97c4b6b380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x562169d2c640 con 0x562169cad000 2023-10-11T09:42:48.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.836+0000 7f97c2b24700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562169cad400 0x562169d52580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:48.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.836+0000 7f97c2b24700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562169cad400 0x562169d52580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57904/0 (socket says 172.21.15.78:57904) 2023-10-11T09:42:48.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.836+0000 7f97c2b24700 1 -- 172.21.15.78:0/78431393 learned_addr learned my addr 172.21.15.78:0/78431393 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:48.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.836+0000 7f97c3325700 1 --2- 172.21.15.78:0/78431393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562169cad800 0x562169d52b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:48.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.836+0000 7f97c3b26700 1 --2- 172.21.15.78:0/78431393 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562169cad000 0x562169d53080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:48.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.840+0000 7f97c2b24700 1 -- 172.21.15.78:0/78431393 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562169cad000 msgr2=0x562169d53080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:48.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.840+0000 7f97c2b24700 1 --2- 172.21.15.78:0/78431393 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562169cad000 0x562169d53080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.840+0000 7f97c2b24700 1 -- 172.21.15.78:0/78431393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562169cad800 msgr2=0x562169d52b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:48.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.840+0000 7f97c2b24700 1 --2- 172.21.15.78:0/78431393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562169cad800 0x562169d52b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.840+0000 7f97c2b24700 1 -- 172.21.15.78:0/78431393 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562169cceb60 con 0x562169cad400 2023-10-11T09:42:48.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.840+0000 7f97c3325700 1 --2- 172.21.15.78:0/78431393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562169cad800 0x562169d52b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:42:48.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.840+0000 7f97c2b24700 1 --2- 172.21.15.78:0/78431393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562169cad400 0x562169d52580 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x56216aa7ec30 tx=0x56216aa88000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab24e8b85bc46b93 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:48.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.840+0000 7f97c0b20700 1 -- 172.21.15.78:0/78431393 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562169cdd600 con 0x562169cad400 2023-10-11T09:42:48.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.840+0000 7f97c0b20700 1 -- 172.21.15.78:0/78431393 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56216aa8a180 con 0x562169cad400 2023-10-11T09:42:48.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.840+0000 7f97c0b20700 1 -- 172.21.15.78:0/78431393 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562169cdd1e0 con 0x562169cad400 2023-10-11T09:42:48.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.840+0000 7f97c4b6b380 1 -- 172.21.15.78:0/78431393 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562169ccf1e0 con 0x562169cad400 2023-10-11T09:42:48.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.840+0000 7f97c4b6b380 1 -- 172.21.15.78:0/78431393 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x562169cceb60 con 0x562169cad400 2023-10-11T09:42:48.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.840+0000 7f97c4b6b380 4 waiting for MDS map... 2023-10-11T09:42:48.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.840+0000 7f97c0b20700 1 -- 172.21.15.78:0/78431393 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(44..44 src has 1..44) v4 ==== 6667+0+0 (secure 0 0 0) 0x562169ce6c40 con 0x562169cad400 2023-10-11T09:42:48.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.840+0000 7f97c0b20700 1 -- 172.21.15.78:0/78431393 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) v1 ==== 3118+0+0 (secure 0 0 0) 0x562169ceef00 con 0x562169cad400 2023-10-11T09:42:48.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.840+0000 7f97c4b6b380 4 Got MDS map 22 2023-10-11T09:42:48.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.840+0000 7f97c4b6b380 10 main: main 2023-10-11T09:42:48.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.840+0000 7f97c4b6b380 4 main: connecting to RADOS... 2023-10-11T09:42:48.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.840+0000 7f97c4b6b380 1 Processor -- start 2023-10-11T09:42:48.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.840+0000 7f97c4b6b380 1 -- start start 2023-10-11T09:42:48.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.844+0000 7f97c4b6b380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562169fbe800 0x562169d53600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:48.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.844+0000 7f97c4b6b380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562169fbec00 0x562169d53b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:48.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.844+0000 7f97c4b6b380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562169fbf000 0x562169d54100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:48.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.844+0000 7f97c4b6b380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x562169d2c500 con 0x562169fbf000 2023-10-11T09:42:48.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.844+0000 7f97c4b6b380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562169d2c3c0 con 0x562169fbec00 2023-10-11T09:42:48.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.844+0000 7f97c4b6b380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x562169d2c640 con 0x562169fbe800 2023-10-11T09:42:48.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.844+0000 7f97c2b24700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562169fbf000 0x562169d54100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:48.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.844+0000 7f97c2b24700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562169fbf000 0x562169d54100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57906/0 (socket says 172.21.15.78:57906) 2023-10-11T09:42:48.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.844+0000 7f97c2b24700 1 -- 172.21.15.78:0/477525100 learned_addr learned my addr 172.21.15.78:0/477525100 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:48.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.844+0000 7f97c3325700 1 --2- 172.21.15.78:0/477525100 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562169fbec00 0x562169d53b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:48.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.844+0000 7f97c3b26700 1 --2- 172.21.15.78:0/477525100 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562169fbe800 0x562169d53600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:48.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.844+0000 7f97c2b24700 1 -- 172.21.15.78:0/477525100 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562169fbe800 msgr2=0x562169d53600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:48.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.844+0000 7f97c2b24700 1 --2- 172.21.15.78:0/477525100 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562169fbe800 0x562169d53600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.844+0000 7f97c2b24700 1 -- 172.21.15.78:0/477525100 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562169fbec00 msgr2=0x562169d53b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:48.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.844+0000 7f97c2b24700 1 --2- 172.21.15.78:0/477525100 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562169fbec00 0x562169d53b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.844+0000 7f97c2b24700 1 -- 172.21.15.78:0/477525100 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562169cce000 con 0x562169fbf000 2023-10-11T09:42:48.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.844+0000 7f97c3b26700 1 --2- 172.21.15.78:0/477525100 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562169fbe800 0x562169d53600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:42:48.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.844+0000 7f97c3325700 1 --2- 172.21.15.78:0/477525100 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562169fbec00 0x562169d53b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:42:48.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.844+0000 7f97c2b24700 1 --2- 172.21.15.78:0/477525100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562169fbf000 0x562169d54100 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x56216aa7c180 tx=0x56216aa88060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aea956dd246d90f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:48.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.844+0000 7f97beb1c700 1 -- 172.21.15.78:0/477525100 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562169cdd600 con 0x562169fbf000 2023-10-11T09:42:48.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.844+0000 7f97beb1c700 1 -- 172.21.15.78:0/477525100 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56216aa8ac00 con 0x562169fbf000 2023-10-11T09:42:48.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.844+0000 7f97beb1c700 1 -- 172.21.15.78:0/477525100 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562169cdd1e0 con 0x562169fbf000 2023-10-11T09:42:48.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.844+0000 7f97c4b6b380 1 -- 172.21.15.78:0/477525100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562169fbf000 msgr2=0x562169d54100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:48.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.844+0000 7f97c4b6b380 1 --2- 172.21.15.78:0/477525100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562169fbf000 0x562169d54100 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x56216aa7c180 tx=0x56216aa88060 comp rx=0 tx=0).stop 2023-10-11T09:42:48.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.844+0000 7f97c4b6b380 1 -- 172.21.15.78:0/477525100 shutdown_connections 2023-10-11T09:42:48.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.844+0000 7f97c4b6b380 1 --2- 172.21.15.78:0/477525100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562169fbf000 0x562169d54100 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.844+0000 7f97c4b6b380 1 --2- 172.21.15.78:0/477525100 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562169fbec00 0x562169d53b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.844+0000 7f97c4b6b380 1 --2- 172.21.15.78:0/477525100 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562169fbe800 0x562169d53600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.844+0000 7f97c4b6b380 1 -- 172.21.15.78:0/477525100 >> 172.21.15.78:0/477525100 conn(0x562169cacc00 msgr2=0x562169dc1800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:48.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.844+0000 7f97c4b6b380 1 -- 172.21.15.78:0/477525100 shutdown_connections 2023-10-11T09:42:48.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.844+0000 7f97c4b6b380 1 -- 172.21.15.78:0/477525100 wait complete. 2023-10-11T09:42:48.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.848+0000 7f97c4b6b380 1 Processor -- start 2023-10-11T09:42:48.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.848+0000 7f97c4b6b380 1 -- start start 2023-10-11T09:42:48.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.848+0000 7f97c4b6b380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562169fbe800 0x562169d53600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:48.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.848+0000 7f97c4b6b380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562169fbec00 0x562169d53b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:48.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.848+0000 7f97c4b6b380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562169fbf000 0x562169d54100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:48.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.848+0000 7f97c4b6b380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x562169d2c280 con 0x562169fbe800 2023-10-11T09:42:48.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.848+0000 7f97c4b6b380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562169d2c140 con 0x562169fbec00 2023-10-11T09:42:48.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.848+0000 7f97c4b6b380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x562169d2c000 con 0x562169fbf000 2023-10-11T09:42:48.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.848+0000 7f97c3b26700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562169fbe800 0x562169d53600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:48.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.848+0000 7f97c3b26700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562169fbe800 0x562169d53600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57918/0 (socket says 172.21.15.78:57918) 2023-10-11T09:42:48.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.848+0000 7f97c3b26700 1 -- 172.21.15.78:0/1664084002 learned_addr learned my addr 172.21.15.78:0/1664084002 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:48.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.848+0000 7f97c3325700 1 --2- 172.21.15.78:0/1664084002 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562169fbec00 0x562169d53b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:48.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.848+0000 7f97c2b24700 1 --2- 172.21.15.78:0/1664084002 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562169fbf000 0x562169d54100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:48.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.848+0000 7f97c3b26700 1 -- 172.21.15.78:0/1664084002 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562169fbf000 msgr2=0x562169d54100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:48.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.848+0000 7f97c3b26700 1 --2- 172.21.15.78:0/1664084002 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562169fbf000 0x562169d54100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.848+0000 7f97c3b26700 1 -- 172.21.15.78:0/1664084002 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562169fbec00 msgr2=0x562169d53b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:48.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.848+0000 7f97c3b26700 1 --2- 172.21.15.78:0/1664084002 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562169fbec00 0x562169d53b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.848+0000 7f97c3b26700 1 -- 172.21.15.78:0/1664084002 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562169ccf1e0 con 0x562169fbe800 2023-10-11T09:42:48.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.848+0000 7f97c3325700 1 --2- 172.21.15.78:0/1664084002 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562169fbec00 0x562169d53b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:42:48.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.848+0000 7f97c2b24700 1 --2- 172.21.15.78:0/1664084002 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562169fbf000 0x562169d54100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:42:48.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.848+0000 7f97c3b26700 1 --2- 172.21.15.78:0/1664084002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562169fbe800 0x562169d53600 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x56216aa7cf30 tx=0x56216aa96e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:48.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.848+0000 7f97bd319700 1 -- 172.21.15.78:0/1664084002 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562169cdd600 con 0x562169fbe800 2023-10-11T09:42:48.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.848+0000 7f97bd319700 1 -- 172.21.15.78:0/1664084002 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56216aa8a480 con 0x562169fbe800 2023-10-11T09:42:48.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.848+0000 7f97c4b6b380 1 -- 172.21.15.78:0/1664084002 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562169cceb60 con 0x562169fbe800 2023-10-11T09:42:48.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.848+0000 7f97bd319700 1 -- 172.21.15.78:0/1664084002 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562169cdd1e0 con 0x562169fbe800 2023-10-11T09:42:48.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.848+0000 7f97c4b6b380 1 -- 172.21.15.78:0/1664084002 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562169cce000 con 0x562169fbe800 2023-10-11T09:42:48.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.848+0000 7f97c4b6b380 4 main: resolving pool 5 2023-10-11T09:42:48.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.852+0000 7f97bd319700 1 -- 172.21.15.78:0/1664084002 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x562169d2b600 con 0x562169fbe800 2023-10-11T09:42:48.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.852+0000 7f97bd319700 1 --2- 172.21.15.78:0/1664084002 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x56216a01a400 0x562169d55180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:48.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.852+0000 7f97bd319700 1 -- 172.21.15.78:0/1664084002 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6667+0+0 (secure 0 0 0) 0x562169ce6a80 con 0x562169fbe800 2023-10-11T09:42:48.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.852+0000 7f97c4b6b380 4 main: creating IoCtx.. 2023-10-11T09:42:48.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.852+0000 7f97c4b6b380 1 --2- 172.21.15.78:0/1664084002 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x562169fbf400 0x562169d54680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:48.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.852+0000 7f97c3325700 1 --2- 172.21.15.78:0/1664084002 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x56216a01a400 0x562169d55180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:48.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.852+0000 7f97c4b6b380 1 -- 172.21.15.78:0/1664084002 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:1 5.b 5:d0630e4c:::mds0_sessionmap:head [omap-clear,omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x562169fbf800 con 0x562169fbf400 2023-10-11T09:42:48.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.852+0000 7f97c2b24700 1 --2- 172.21.15.78:0/1664084002 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x562169fbf400 0x562169d54680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:48.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.852+0000 7f97c2b24700 1 --2- 172.21.15.78:0/1664084002 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x562169fbf400 0x562169d54680 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:48.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.852+0000 7f97c3325700 1 --2- 172.21.15.78:0/1664084002 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x56216a01a400 0x562169d55180 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x56216aa7e150 tx=0x56216aa972c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:48.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.856+0000 7f97c2b24700 1 -- 172.21.15.78:0/1664084002 <== osd.2 v2:172.21.15.78:6808/35732 1 ==== osd_op_reply(1 mds0_sessionmap [omap-clear,omap-set-header] v44'5 uv5 ondisk = 0) v8 ==== 201+0+0 (crc 0 0 0) 0x56216ab26fc0 con 0x562169fbf400 2023-10-11T09:42:48.882 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:42:48.882 INFO:teuthology.orchestra.run.smithi078.stdout: "0": { 2023-10-11T09:42:48.882 INFO:teuthology.orchestra.run.smithi078.stdout: "data": {}, 2023-10-11T09:42:48.882 INFO:teuthology.orchestra.run.smithi078.stdout: "result": 0 2023-10-11T09:42:48.882 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:42:48.883 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:42:48.883 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:48.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.856+0000 7f97c4b6b380 1 -- 172.21.15.78:0/1664084002 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x562169fbf400 msgr2=0x562169d54680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:48.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.856+0000 7f97c4b6b380 1 --2- 172.21.15.78:0/1664084002 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x562169fbf400 0x562169d54680 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.856+0000 7f97c4b6b380 1 -- 172.21.15.78:0/1664084002 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x56216a01a400 msgr2=0x562169d55180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:48.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.856+0000 7f97c4b6b380 1 --2- 172.21.15.78:0/1664084002 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x56216a01a400 0x562169d55180 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x56216aa7e150 tx=0x56216aa972c0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.856+0000 7f97c4b6b380 1 -- 172.21.15.78:0/1664084002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562169fbe800 msgr2=0x562169d53600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:48.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.856+0000 7f97c4b6b380 1 --2- 172.21.15.78:0/1664084002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562169fbe800 0x562169d53600 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x56216aa7cf30 tx=0x56216aa96e40 comp rx=0 tx=0).stop 2023-10-11T09:42:48.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.856+0000 7f97c3325700 1 -- 172.21.15.78:0/1664084002 reap_dead start 2023-10-11T09:42:48.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.856+0000 7f97c4b6b380 1 -- 172.21.15.78:0/1664084002 shutdown_connections 2023-10-11T09:42:48.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.856+0000 7f97c4b6b380 1 -- 172.21.15.78:0/1664084002 >> 172.21.15.78:0/1664084002 conn(0x562169cacc00 msgr2=0x562169dc1800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:48.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.856+0000 7f97c4b6b380 1 -- 172.21.15.78:0/1664084002 shutdown_connections 2023-10-11T09:42:48.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.856+0000 7f97c4b6b380 1 -- 172.21.15.78:0/1664084002 wait complete. 2023-10-11T09:42:48.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.856+0000 7f97c4b6b380 1 -- 172.21.15.78:0/78431393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562169cad400 msgr2=0x562169d52580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:48.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.856+0000 7f97c4b6b380 1 --2- 172.21.15.78:0/78431393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562169cad400 0x562169d52580 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x56216aa7ec30 tx=0x56216aa88000 comp rx=0 tx=0).stop 2023-10-11T09:42:48.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.856+0000 7f97c4b6b380 1 -- 172.21.15.78:0/78431393 shutdown_connections 2023-10-11T09:42:48.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.856+0000 7f97c4b6b380 1 --2- 172.21.15.78:0/78431393 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562169cad000 0x562169d53080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.856+0000 7f97c4b6b380 1 --2- 172.21.15.78:0/78431393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562169cad400 0x562169d52580 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.856+0000 7f97c4b6b380 1 --2- 172.21.15.78:0/78431393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562169cad800 0x562169d52b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.856+0000 7f97c4b6b380 1 -- 172.21.15.78:0/78431393 >> 172.21.15.78:0/78431393 conn(0x562169cadc00 msgr2=0x562169dc1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:48.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.860+0000 7f97c4b6b380 1 -- 172.21.15.78:0/78431393 shutdown_connections 2023-10-11T09:42:48.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.860+0000 7f97c4b6b380 1 -- 172.21.15.78:0/78431393 wait complete. 2023-10-11T09:42:48.889 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'session'] in time 0:00:00.109640, result: { "0": { "data": {}, "result": 0 } } 2023-10-11T09:42:48.889 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset snap 2023-10-11T09:42:48.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.952+0000 7f6dc0f1e380 1 -- start start 2023-10-11T09:42:48.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.952+0000 7f6dc0f1e380 1 Processor -- start 2023-10-11T09:42:48.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.956+0000 7f6dc0f1e380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc4a420400 0x55dc496eb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:48.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.956+0000 7f6dc0f1e380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55dc49645c00 0x55dc496eab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:48.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.956+0000 7f6dc0f1e380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc4a420800 0x55dc496ebb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:48.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.956+0000 7f6dc0f1e380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc4967be00 con 0x55dc4a420400 2023-10-11T09:42:48.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.956+0000 7f6dc0f1e380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc496c4a00 con 0x55dc4a420800 2023-10-11T09:42:48.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.956+0000 7f6dc0f1e380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dc496c4280 con 0x55dc49645c00 2023-10-11T09:42:48.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.956+0000 7f6dbf6d8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc4a420400 0x55dc496eb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:48.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.956+0000 7f6dbf6d8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc4a420400 0x55dc496eb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57938/0 (socket says 172.21.15.78:57938) 2023-10-11T09:42:48.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.956+0000 7f6dbf6d8700 1 -- 172.21.15.78:0/3590285259 learned_addr learned my addr 172.21.15.78:0/3590285259 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:48.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.956+0000 7f6dbf6d8700 1 -- 172.21.15.78:0/3590285259 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55dc49645c00 msgr2=0x55dc496eab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:48.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.956+0000 7f6dbf6d8700 1 --2- 172.21.15.78:0/3590285259 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55dc49645c00 0x55dc496eab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.956+0000 7f6dbf6d8700 1 -- 172.21.15.78:0/3590285259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc4a420800 msgr2=0x55dc496ebb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:48.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.956+0000 7f6dbf6d8700 1 --2- 172.21.15.78:0/3590285259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc4a420800 0x55dc496ebb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.956+0000 7f6dbf6d8700 1 -- 172.21.15.78:0/3590285259 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc49666000 con 0x55dc4a420400 2023-10-11T09:42:48.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.956+0000 7f6dbf6d8700 1 --2- 172.21.15.78:0/3590285259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc4a420400 0x55dc496eb600 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x55dc4a41ebd0 tx=0x55dc4a424f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a2dc7897e7c95d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:48.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.956+0000 7f6dbced3700 1 -- 172.21.15.78:0/3590285259 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55dc49675080 con 0x55dc4a420400 2023-10-11T09:42:48.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.956+0000 7f6dbced3700 1 -- 172.21.15.78:0/3590285259 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55dc4992e000 con 0x55dc4a420400 2023-10-11T09:42:48.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.956+0000 7f6dbced3700 1 -- 172.21.15.78:0/3590285259 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55dc49675600 con 0x55dc4a420400 2023-10-11T09:42:48.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.956+0000 7f6dc0f1e380 1 -- 172.21.15.78:0/3590285259 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc49666ea0 con 0x55dc4a420400 2023-10-11T09:42:48.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.956+0000 7f6dc0f1e380 1 -- 172.21.15.78:0/3590285259 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55dc49667520 con 0x55dc4a420400 2023-10-11T09:42:48.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.956+0000 7f6dc0f1e380 4 waiting for MDS map... 2023-10-11T09:42:48.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.956+0000 7f6dbced3700 1 -- 172.21.15.78:0/3590285259 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(44..44 src has 1..44) v4 ==== 6667+0+0 (secure 0 0 0) 0x55dc4967ec40 con 0x55dc4a420400 2023-10-11T09:42:48.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.956+0000 7f6dbced3700 1 -- 172.21.15.78:0/3590285259 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) v1 ==== 3118+0+0 (secure 0 0 0) 0x55dc49930300 con 0x55dc4a420400 2023-10-11T09:42:48.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.956+0000 7f6dc0f1e380 4 Got MDS map 22 2023-10-11T09:42:48.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.956+0000 7f6dc0f1e380 10 main: main 2023-10-11T09:42:48.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.956+0000 7f6dc0f1e380 4 main: connecting to RADOS... 2023-10-11T09:42:48.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.956+0000 7f6dc0f1e380 1 Processor -- start 2023-10-11T09:42:48.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.956+0000 7f6dc0f1e380 1 -- start start 2023-10-11T09:42:48.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.960+0000 7f6dc0f1e380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc4a420c00 0x55dc496eb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:48.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.960+0000 7f6dc0f1e380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc49645800 0x55dc496ea580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:48.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.960+0000 7f6dc0f1e380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55dc4a421400 0x55dc496ec100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:48.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.960+0000 7f6dc0f1e380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc496c43c0 con 0x55dc49645800 2023-10-11T09:42:48.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.960+0000 7f6dc0f1e380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc496c4500 con 0x55dc4a420c00 2023-10-11T09:42:48.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.960+0000 7f6dc0f1e380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dc496c4a00 con 0x55dc4a421400 2023-10-11T09:42:48.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.960+0000 7f6dbf6d8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc49645800 0x55dc496ea580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:48.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.960+0000 7f6dbf6d8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc49645800 0x55dc496ea580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57944/0 (socket says 172.21.15.78:57944) 2023-10-11T09:42:48.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.960+0000 7f6dbf6d8700 1 -- 172.21.15.78:0/1212670039 learned_addr learned my addr 172.21.15.78:0/1212670039 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:48.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.960+0000 7f6dbfed9700 1 --2- 172.21.15.78:0/1212670039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc4a420c00 0x55dc496eb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:48.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.960+0000 7f6dbeed7700 1 --2- 172.21.15.78:0/1212670039 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55dc4a421400 0x55dc496ec100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:48.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.960+0000 7f6dbf6d8700 1 -- 172.21.15.78:0/1212670039 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55dc4a421400 msgr2=0x55dc496ec100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:48.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.960+0000 7f6dbf6d8700 1 --2- 172.21.15.78:0/1212670039 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55dc4a421400 0x55dc496ec100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.960+0000 7f6dbf6d8700 1 -- 172.21.15.78:0/1212670039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc4a420c00 msgr2=0x55dc496eb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:48.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.960+0000 7f6dbf6d8700 1 --2- 172.21.15.78:0/1212670039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc4a420c00 0x55dc496eb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.960+0000 7f6dbf6d8700 1 -- 172.21.15.78:0/1212670039 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc496676c0 con 0x55dc49645800 2023-10-11T09:42:48.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.960+0000 7f6dbfed9700 1 --2- 172.21.15.78:0/1212670039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc4a420c00 0x55dc496eb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:42:48.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.960+0000 7f6dbeed7700 1 --2- 172.21.15.78:0/1212670039 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55dc4a421400 0x55dc496ec100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:42:48.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.960+0000 7f6dbf6d8700 1 --2- 172.21.15.78:0/1212670039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc49645800 0x55dc496ea580 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x55dc4999a000 tx=0x55dc4a4240c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2a97c7b5fb3f2a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:48.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.960+0000 7f6dbaecf700 1 -- 172.21.15.78:0/1212670039 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55dc496751e0 con 0x55dc49645800 2023-10-11T09:42:48.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.960+0000 7f6dbaecf700 1 -- 172.21.15.78:0/1212670039 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55dc4992e480 con 0x55dc49645800 2023-10-11T09:42:48.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.960+0000 7f6dbaecf700 1 -- 172.21.15.78:0/1212670039 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55dc49675080 con 0x55dc49645800 2023-10-11T09:42:48.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.960+0000 7f6dc0f1e380 1 -- 172.21.15.78:0/1212670039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc49645800 msgr2=0x55dc496ea580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:48.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.960+0000 7f6dc0f1e380 1 --2- 172.21.15.78:0/1212670039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc49645800 0x55dc496ea580 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x55dc4999a000 tx=0x55dc4a4240c0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.960+0000 7f6dc0f1e380 1 -- 172.21.15.78:0/1212670039 shutdown_connections 2023-10-11T09:42:48.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.960+0000 7f6dc0f1e380 1 --2- 172.21.15.78:0/1212670039 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55dc4a421400 0x55dc496ec100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.960+0000 7f6dc0f1e380 1 --2- 172.21.15.78:0/1212670039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc49645800 0x55dc496ea580 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.960+0000 7f6dc0f1e380 1 --2- 172.21.15.78:0/1212670039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc4a420c00 0x55dc496eb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.960+0000 7f6dc0f1e380 1 -- 172.21.15.78:0/1212670039 >> 172.21.15.78:0/1212670039 conn(0x55dc4a420000 msgr2=0x55dc49701000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:48.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.960+0000 7f6dc0f1e380 1 -- 172.21.15.78:0/1212670039 shutdown_connections 2023-10-11T09:42:48.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.960+0000 7f6dc0f1e380 1 -- 172.21.15.78:0/1212670039 wait complete. 2023-10-11T09:42:48.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.960+0000 7f6dc0f1e380 1 Processor -- start 2023-10-11T09:42:48.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.960+0000 7f6dc0f1e380 1 -- start start 2023-10-11T09:42:48.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.964+0000 7f6dc0f1e380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc49645800 0x55dc496ed180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:48.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.964+0000 7f6dc0f1e380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc4a420c00 0x55dc496ecc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:48.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.964+0000 7f6dc0f1e380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55dc4a421400 0x55dc496ec680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:48.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.964+0000 7f6dc0f1e380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc496c4640 con 0x55dc49645800 2023-10-11T09:42:48.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.964+0000 7f6dc0f1e380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc496c4140 con 0x55dc4a420c00 2023-10-11T09:42:48.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.964+0000 7f6dc0f1e380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dc496c4000 con 0x55dc4a421400 2023-10-11T09:42:48.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.964+0000 7f6dbfed9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc49645800 0x55dc496ed180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:48.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.964+0000 7f6dbfed9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc49645800 0x55dc496ed180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57954/0 (socket says 172.21.15.78:57954) 2023-10-11T09:42:48.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.964+0000 7f6dbfed9700 1 -- 172.21.15.78:0/1528985424 learned_addr learned my addr 172.21.15.78:0/1528985424 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:48.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.964+0000 7f6dbeed7700 1 --2- 172.21.15.78:0/1528985424 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55dc4a421400 0x55dc496ec680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:48.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.964+0000 7f6dbf6d8700 1 --2- 172.21.15.78:0/1528985424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc4a420c00 0x55dc496ecc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:48.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.964+0000 7f6dbfed9700 1 -- 172.21.15.78:0/1528985424 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55dc4a421400 msgr2=0x55dc496ec680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:48.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.964+0000 7f6dbfed9700 1 --2- 172.21.15.78:0/1528985424 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55dc4a421400 0x55dc496ec680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.964+0000 7f6dbfed9700 1 -- 172.21.15.78:0/1528985424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc4a420c00 msgr2=0x55dc496ecc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:48.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.964+0000 7f6dbfed9700 1 --2- 172.21.15.78:0/1528985424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc4a420c00 0x55dc496ecc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.964+0000 7f6dbfed9700 1 -- 172.21.15.78:0/1528985424 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc49666000 con 0x55dc49645800 2023-10-11T09:42:48.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.964+0000 7f6dbeed7700 1 --2- 172.21.15.78:0/1528985424 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55dc4a421400 0x55dc496ec680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:42:48.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.964+0000 7f6dbf6d8700 1 --2- 172.21.15.78:0/1528985424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc4a420c00 0x55dc496ecc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:42:48.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.964+0000 7f6dbfed9700 1 --2- 172.21.15.78:0/1528985424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc49645800 0x55dc496ed180 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x55dc4999b410 tx=0x55dc49944f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:48.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.964+0000 7f6db96cc700 1 -- 172.21.15.78:0/1528985424 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55dc496751e0 con 0x55dc49645800 2023-10-11T09:42:48.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.964+0000 7f6db96cc700 1 -- 172.21.15.78:0/1528985424 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55dc49690480 con 0x55dc49645800 2023-10-11T09:42:48.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.964+0000 7f6db96cc700 1 -- 172.21.15.78:0/1528985424 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55dc49675080 con 0x55dc49645800 2023-10-11T09:42:48.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.964+0000 7f6dc0f1e380 1 -- 172.21.15.78:0/1528985424 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55dc49666ea0 con 0x55dc49645800 2023-10-11T09:42:48.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.964+0000 7f6dc0f1e380 1 -- 172.21.15.78:0/1528985424 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc49667860 con 0x55dc49645800 2023-10-11T09:42:48.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.964+0000 7f6dc0f1e380 4 main: resolving pool 5 2023-10-11T09:42:48.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.968+0000 7f6db96cc700 1 -- 172.21.15.78:0/1528985424 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x55dc496c2840 con 0x55dc49645800 2023-10-11T09:42:48.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.968+0000 7f6db96cc700 1 --2- 172.21.15.78:0/1528985424 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55dc49a8e000 0x55dc496ed700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:48.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.968+0000 7f6db96cc700 1 -- 172.21.15.78:0/1528985424 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6667+0+0 (secure 0 0 0) 0x55dc4967ea80 con 0x55dc49645800 2023-10-11T09:42:48.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.968+0000 7f6dc0f1e380 4 main: creating IoCtx.. 2023-10-11T09:42:48.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.968+0000 7f6dbf6d8700 1 --2- 172.21.15.78:0/1528985424 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55dc49a8e000 0x55dc496ed700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:48.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.968+0000 7f6dc0f1e380 1 --2- 172.21.15.78:0/1528985424 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55dc4a421800 0x55dc496ec100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:48.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.968+0000 7f6dc0f1e380 1 -- 172.21.15.78:0/1528985424 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:1 5.d 5:b50e409b:::mds_snaptable:head [writefull 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x55dc4a421c00 con 0x55dc4a421800 2023-10-11T09:42:48.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.968+0000 7f6dbeed7700 1 --2- 172.21.15.78:0/1528985424 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55dc4a421800 0x55dc496ec100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:48.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.968+0000 7f6dbf6d8700 1 --2- 172.21.15.78:0/1528985424 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55dc49a8e000 0x55dc496ed700 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55dc4999a000 tx=0x55dc49944c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:48.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.972+0000 7f6dbeed7700 1 --2- 172.21.15.78:0/1528985424 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55dc4a421800 0x55dc496ec100 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:48.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.972+0000 7f6dbeed7700 1 -- 172.21.15.78:0/1528985424 <== osd.2 v2:172.21.15.78:6808/35732 1 ==== osd_op_reply(1 mds_snaptable [writefull 0~70] v44'3 uv3 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x55dc4a3aafc0 con 0x55dc4a421800 2023-10-11T09:42:48.992 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:42:48.992 INFO:teuthology.orchestra.run.smithi078.stdout: "result": 0 2023-10-11T09:42:48.992 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:42:48.992 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:48.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.972+0000 7f6dc0f1e380 1 -- 172.21.15.78:0/1528985424 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55dc4a421800 msgr2=0x55dc496ec100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:48.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.972+0000 7f6dc0f1e380 1 --2- 172.21.15.78:0/1528985424 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55dc4a421800 0x55dc496ec100 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.972+0000 7f6dc0f1e380 1 -- 172.21.15.78:0/1528985424 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55dc49a8e000 msgr2=0x55dc496ed700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:48.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.972+0000 7f6dc0f1e380 1 --2- 172.21.15.78:0/1528985424 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55dc49a8e000 0x55dc496ed700 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55dc4999a000 tx=0x55dc49944c00 comp rx=0 tx=0).stop 2023-10-11T09:42:48.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.972+0000 7f6dc0f1e380 1 -- 172.21.15.78:0/1528985424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc49645800 msgr2=0x55dc496ed180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:48.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.972+0000 7f6dc0f1e380 1 --2- 172.21.15.78:0/1528985424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc49645800 0x55dc496ed180 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x55dc4999b410 tx=0x55dc49944f00 comp rx=0 tx=0).stop 2023-10-11T09:42:48.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.972+0000 7f6dbeed7700 1 -- 172.21.15.78:0/1528985424 reap_dead start 2023-10-11T09:42:48.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.976+0000 7f6dc0f1e380 1 -- 172.21.15.78:0/1528985424 shutdown_connections 2023-10-11T09:42:48.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.976+0000 7f6dc0f1e380 1 -- 172.21.15.78:0/1528985424 >> 172.21.15.78:0/1528985424 conn(0x55dc4a420000 msgr2=0x55dc49701000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:48.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.976+0000 7f6dc0f1e380 1 -- 172.21.15.78:0/1528985424 shutdown_connections 2023-10-11T09:42:48.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.976+0000 7f6dc0f1e380 1 -- 172.21.15.78:0/1528985424 wait complete. 2023-10-11T09:42:48.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.976+0000 7f6dc0f1e380 1 -- 172.21.15.78:0/3590285259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc4a420400 msgr2=0x55dc496eb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:48.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.976+0000 7f6dc0f1e380 1 --2- 172.21.15.78:0/3590285259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc4a420400 0x55dc496eb600 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x55dc4a41ebd0 tx=0x55dc4a424f00 comp rx=0 tx=0).stop 2023-10-11T09:42:48.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.976+0000 7f6dc0f1e380 1 -- 172.21.15.78:0/3590285259 shutdown_connections 2023-10-11T09:42:48.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.976+0000 7f6dc0f1e380 1 --2- 172.21.15.78:0/3590285259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc4a420800 0x55dc496ebb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.976+0000 7f6dc0f1e380 1 --2- 172.21.15.78:0/3590285259 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55dc49645c00 0x55dc496eab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.976+0000 7f6dc0f1e380 1 --2- 172.21.15.78:0/3590285259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc4a420400 0x55dc496eb600 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:48.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.976+0000 7f6dc0f1e380 1 -- 172.21.15.78:0/3590285259 >> 172.21.15.78:0/3590285259 conn(0x55dc49644c00 msgr2=0x55dc49702000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:48.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.976+0000 7f6dc0f1e380 1 -- 172.21.15.78:0/3590285259 shutdown_connections 2023-10-11T09:42:48.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:48.976+0000 7f6dc0f1e380 1 -- 172.21.15.78:0/3590285259 wait complete. 2023-10-11T09:42:48.998 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'snap'] in time 0:00:00.108966, result: { "result": 0 } 2023-10-11T09:42:48.999 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset inode 2023-10-11T09:42:49.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.056+0000 7fc876a15380 1 -- start start 2023-10-11T09:42:49.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.056+0000 7fc876a15380 1 Processor -- start 2023-10-11T09:42:49.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.060+0000 7fc876a15380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c0e345800 0x564c0e3ea580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.060+0000 7fc876a15380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564c0e345400 0x564c0e3eab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.060+0000 7fc876a15380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c0e345000 0x564c0e3eb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.060+0000 7fc876a15380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0e3c4a00 con 0x564c0e345000 2023-10-11T09:42:49.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.060+0000 7fc876a15380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0e3c4780 con 0x564c0e345800 2023-10-11T09:42:49.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.060+0000 7fc876a15380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c0e3c4640 con 0x564c0e345400 2023-10-11T09:42:49.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.060+0000 7fc8759d0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c0e345000 0x564c0e3eb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.060+0000 7fc8749ce700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564c0e345400 0x564c0e3eab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.060+0000 7fc8759d0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c0e345000 0x564c0e3eb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57964/0 (socket says 172.21.15.78:57964) 2023-10-11T09:42:49.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.060+0000 7fc8749ce700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564c0e345400 0x564c0e3eab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:56504/0 (socket says 172.21.15.78:56504) 2023-10-11T09:42:49.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.060+0000 7fc8759d0700 1 -- 172.21.15.78:0/3987526030 learned_addr learned my addr 172.21.15.78:0/3987526030 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:49.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.060+0000 7fc8751cf700 1 --2- 172.21.15.78:0/3987526030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c0e345800 0x564c0e3ea580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.060+0000 7fc8759d0700 1 -- 172.21.15.78:0/3987526030 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564c0e345400 msgr2=0x564c0e3eab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:49.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.060+0000 7fc8759d0700 1 --2- 172.21.15.78:0/3987526030 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564c0e345400 0x564c0e3eab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.060+0000 7fc8759d0700 1 -- 172.21.15.78:0/3987526030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c0e345800 msgr2=0x564c0e3ea580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:49.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.060+0000 7fc8759d0700 1 --2- 172.21.15.78:0/3987526030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c0e345800 0x564c0e3ea580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.060+0000 7fc8759d0700 1 -- 172.21.15.78:0/3987526030 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c0e366b60 con 0x564c0e345000 2023-10-11T09:42:49.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.060+0000 7fc8759d0700 1 --2- 172.21.15.78:0/3987526030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c0e345000 0x564c0e3eb080 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x564c0f11eb70 tx=0x564c0f12cfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc029e00efa6cfa server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.060+0000 7fc8729ca700 1 -- 172.21.15.78:0/3987526030 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564c0e375600 con 0x564c0e345000 2023-10-11T09:42:49.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.060+0000 7fc8729ca700 1 -- 172.21.15.78:0/3987526030 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564c0e390480 con 0x564c0e345000 2023-10-11T09:42:49.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.060+0000 7fc8729ca700 1 -- 172.21.15.78:0/3987526030 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564c0e3751e0 con 0x564c0e345000 2023-10-11T09:42:49.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.060+0000 7fc876a15380 1 -- 172.21.15.78:0/3987526030 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c0e366340 con 0x564c0e345000 2023-10-11T09:42:49.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.060+0000 7fc876a15380 1 -- 172.21.15.78:0/3987526030 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x564c0e366000 con 0x564c0e345000 2023-10-11T09:42:49.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.060+0000 7fc876a15380 4 waiting for MDS map... 2023-10-11T09:42:49.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.060+0000 7fc8729ca700 1 -- 172.21.15.78:0/3987526030 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(44..44 src has 1..44) v4 ==== 6667+0+0 (secure 0 0 0) 0x564c0e37ec40 con 0x564c0e345000 2023-10-11T09:42:49.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.060+0000 7fc8729ca700 1 -- 172.21.15.78:0/3987526030 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) v1 ==== 3118+0+0 (secure 0 0 0) 0x564c0f138300 con 0x564c0e345000 2023-10-11T09:42:49.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.060+0000 7fc876a15380 4 Got MDS map 22 2023-10-11T09:42:49.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.060+0000 7fc876a15380 10 main: main 2023-10-11T09:42:49.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.060+0000 7fc876a15380 4 main: connecting to RADOS... 2023-10-11T09:42:49.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.064+0000 7fc876a15380 1 Processor -- start 2023-10-11T09:42:49.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.064+0000 7fc876a15380 1 -- start start 2023-10-11T09:42:49.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.064+0000 7fc876a15380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c0f1dc400 0x564c0e3eb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.064+0000 7fc876a15380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c0f1dc800 0x564c0e3ebb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.064+0000 7fc876a15380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564c0f1dcc00 0x564c0e3ec100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.064+0000 7fc876a15380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0e37be00 con 0x564c0f1dc400 2023-10-11T09:42:49.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.064+0000 7fc876a15380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0e3c4780 con 0x564c0f1dc800 2023-10-11T09:42:49.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.064+0000 7fc876a15380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c0e3c4500 con 0x564c0f1dcc00 2023-10-11T09:42:49.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.064+0000 7fc8749ce700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c0f1dc400 0x564c0e3eb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.064+0000 7fc8749ce700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c0f1dc400 0x564c0e3eb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57980/0 (socket says 172.21.15.78:57980) 2023-10-11T09:42:49.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.064+0000 7fc8749ce700 1 -- 172.21.15.78:0/794557093 learned_addr learned my addr 172.21.15.78:0/794557093 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:49.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.064+0000 7fc8751cf700 1 --2- 172.21.15.78:0/794557093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c0f1dc800 0x564c0e3ebb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.064+0000 7fc8749ce700 1 --2- 172.21.15.78:0/794557093 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564c0f1dcc00 0x564c0e3ec100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.064+0000 7fc8749ce700 1 -- 172.21.15.78:0/794557093 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564c0f1dcc00 msgr2=0x564c0e3ec100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:49.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.064+0000 7fc8749ce700 1 --2- 172.21.15.78:0/794557093 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564c0f1dcc00 0x564c0e3ec100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.064+0000 7fc8749ce700 1 -- 172.21.15.78:0/794557093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c0f1dc800 msgr2=0x564c0e3ebb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:49.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.064+0000 7fc8749ce700 1 --2- 172.21.15.78:0/794557093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c0f1dc800 0x564c0e3ebb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.064+0000 7fc8749ce700 1 -- 172.21.15.78:0/794557093 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c0e366ea0 con 0x564c0f1dc400 2023-10-11T09:42:49.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.064+0000 7fc8749ce700 1 --2- 172.21.15.78:0/794557093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c0f1dc400 0x564c0e3eb600 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x564c0f120090 tx=0x564c0f1dec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b07293d9a540e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.064+0000 7fc8709c6700 1 -- 172.21.15.78:0/794557093 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564c0e375600 con 0x564c0f1dc400 2023-10-11T09:42:49.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.064+0000 7fc8709c6700 1 -- 172.21.15.78:0/794557093 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564c0f13cd80 con 0x564c0f1dc400 2023-10-11T09:42:49.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.064+0000 7fc8709c6700 1 -- 172.21.15.78:0/794557093 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564c0e3751e0 con 0x564c0f1dc400 2023-10-11T09:42:49.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.068+0000 7fc876a15380 1 -- 172.21.15.78:0/794557093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c0f1dc400 msgr2=0x564c0e3eb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.068+0000 7fc876a15380 1 --2- 172.21.15.78:0/794557093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c0f1dc400 0x564c0e3eb600 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x564c0f120090 tx=0x564c0f1dec00 comp rx=0 tx=0).stop 2023-10-11T09:42:49.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.068+0000 7fc876a15380 1 -- 172.21.15.78:0/794557093 shutdown_connections 2023-10-11T09:42:49.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.068+0000 7fc876a15380 1 --2- 172.21.15.78:0/794557093 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564c0f1dcc00 0x564c0e3ec100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.068+0000 7fc876a15380 1 --2- 172.21.15.78:0/794557093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c0f1dc800 0x564c0e3ebb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.068+0000 7fc876a15380 1 --2- 172.21.15.78:0/794557093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c0f1dc400 0x564c0e3eb600 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.068+0000 7fc876a15380 1 -- 172.21.15.78:0/794557093 >> 172.21.15.78:0/794557093 conn(0x564c0e344c00 msgr2=0x564c0e431800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:49.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.068+0000 7fc876a15380 1 -- 172.21.15.78:0/794557093 shutdown_connections 2023-10-11T09:42:49.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.068+0000 7fc876a15380 1 -- 172.21.15.78:0/794557093 wait complete. 2023-10-11T09:42:49.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.068+0000 7fc876a15380 1 Processor -- start 2023-10-11T09:42:49.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.068+0000 7fc876a15380 1 -- start start 2023-10-11T09:42:49.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.068+0000 7fc876a15380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564c0f1dc400 0x564c0e3ed180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.068+0000 7fc876a15380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c0f1dc800 0x564c0e3ecc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.068+0000 7fc876a15380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c0f1dcc00 0x564c0e3ec680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.068+0000 7fc876a15380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0e3c43c0 con 0x564c0f1dcc00 2023-10-11T09:42:49.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.068+0000 7fc876a15380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0e3c4280 con 0x564c0f1dc800 2023-10-11T09:42:49.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.068+0000 7fc876a15380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c0e3c4140 con 0x564c0f1dc400 2023-10-11T09:42:49.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.068+0000 7fc8749ce700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c0f1dcc00 0x564c0e3ec680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.068+0000 7fc8749ce700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c0f1dcc00 0x564c0e3ec680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57986/0 (socket says 172.21.15.78:57986) 2023-10-11T09:42:49.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.068+0000 7fc8749ce700 1 -- 172.21.15.78:0/3131387828 learned_addr learned my addr 172.21.15.78:0/3131387828 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:49.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.068+0000 7fc8751cf700 1 --2- 172.21.15.78:0/3131387828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c0f1dc800 0x564c0e3ecc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.068+0000 7fc8749ce700 1 --2- 172.21.15.78:0/3131387828 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564c0f1dc400 0x564c0e3ed180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.072+0000 7fc8751cf700 1 -- 172.21.15.78:0/3131387828 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564c0f1dc400 msgr2=0x564c0e3ed180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.072+0000 7fc8751cf700 1 --2- 172.21.15.78:0/3131387828 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564c0f1dc400 0x564c0e3ed180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.072+0000 7fc8751cf700 1 -- 172.21.15.78:0/3131387828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c0f1dcc00 msgr2=0x564c0e3ec680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.072+0000 7fc8751cf700 1 --2- 172.21.15.78:0/3131387828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c0f1dcc00 0x564c0e3ec680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.072+0000 7fc8751cf700 1 -- 172.21.15.78:0/3131387828 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c0e366b60 con 0x564c0f1dc800 2023-10-11T09:42:49.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.072+0000 7fc8751cf700 1 --2- 172.21.15.78:0/3131387828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c0f1dc800 0x564c0e3ecc00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x564c0f120ed0 tx=0x564c0f12c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.072+0000 7fc86f1c3700 1 -- 172.21.15.78:0/3131387828 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564c0e375080 con 0x564c0f1dc800 2023-10-11T09:42:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.072+0000 7fc86f1c3700 1 -- 172.21.15.78:0/3131387828 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564c0e390180 con 0x564c0f1dc800 2023-10-11T09:42:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.072+0000 7fc86f1c3700 1 -- 172.21.15.78:0/3131387828 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564c0e375600 con 0x564c0f1dc800 2023-10-11T09:42:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.072+0000 7fc876a15380 1 -- 172.21.15.78:0/3131387828 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564c0e366340 con 0x564c0f1dc800 2023-10-11T09:42:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.072+0000 7fc876a15380 1 -- 172.21.15.78:0/3131387828 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c0e366000 con 0x564c0f1dc800 2023-10-11T09:42:49.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.072+0000 7fc876a15380 4 main: resolving pool 5 2023-10-11T09:42:49.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.072+0000 7fc86f1c3700 1 -- 172.21.15.78:0/3131387828 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x564c0e3c2840 con 0x564c0f1dc800 2023-10-11T09:42:49.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.076+0000 7fc86f1c3700 1 --2- 172.21.15.78:0/3131387828 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x564c0e692000 0x564c0e3ed700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.076+0000 7fc86f1c3700 1 -- 172.21.15.78:0/3131387828 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6667+0+0 (secure 0 0 0) 0x564c0e37ea80 con 0x564c0f1dc800 2023-10-11T09:42:49.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.076+0000 7fc876a15380 4 main: creating IoCtx.. 2023-10-11T09:42:49.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.076+0000 7fc8749ce700 1 --2- 172.21.15.78:0/3131387828 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x564c0e692000 0x564c0e3ed700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.076+0000 7fc876a15380 1 --2- 172.21.15.78:0/3131387828 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x564c0f1dd000 0x564c0e3ec100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.076+0000 7fc876a15380 1 -- 172.21.15.78:0/3131387828 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:1 5.13 5:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x564c0f1dd400 con 0x564c0f1dd000 2023-10-11T09:42:49.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.076+0000 7fc8749ce700 1 --2- 172.21.15.78:0/3131387828 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x564c0f1dd000 0x564c0e3ec100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.076+0000 7fc8749ce700 1 --2- 172.21.15.78:0/3131387828 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x564c0e692000 0x564c0e3ed700 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x564c0f1220c0 tx=0x564c0f12c1e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.076+0000 7fc8749ce700 1 --2- 172.21.15.78:0/3131387828 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x564c0f1dd000 0x564c0e3ec100 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.080+0000 7fc8749ce700 1 -- 172.21.15.78:0/3131387828 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v44'6 uv6 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x564c0f1d2fc0 con 0x564c0f1dd000 2023-10-11T09:42:49.087 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:42:49.088 INFO:teuthology.orchestra.run.smithi078.stdout: "0": { 2023-10-11T09:42:49.088 INFO:teuthology.orchestra.run.smithi078.stdout: "data": {}, 2023-10-11T09:42:49.088 INFO:teuthology.orchestra.run.smithi078.stdout: "result": 0 2023-10-11T09:42:49.088 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:42:49.089 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:42:49.089 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:49.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.080+0000 7fc876a15380 1 -- 172.21.15.78:0/3131387828 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x564c0f1dd000 msgr2=0x564c0e3ec100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.080+0000 7fc876a15380 1 --2- 172.21.15.78:0/3131387828 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x564c0f1dd000 0x564c0e3ec100 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.080+0000 7fc876a15380 1 -- 172.21.15.78:0/3131387828 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x564c0e692000 msgr2=0x564c0e3ed700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.080+0000 7fc876a15380 1 --2- 172.21.15.78:0/3131387828 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x564c0e692000 0x564c0e3ed700 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x564c0f1220c0 tx=0x564c0f12c1e0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.080+0000 7fc876a15380 1 -- 172.21.15.78:0/3131387828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c0f1dc800 msgr2=0x564c0e3ecc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.080+0000 7fc876a15380 1 --2- 172.21.15.78:0/3131387828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c0f1dc800 0x564c0e3ecc00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x564c0f120ed0 tx=0x564c0f12c000 comp rx=0 tx=0).stop 2023-10-11T09:42:49.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.080+0000 7fc8751cf700 1 -- 172.21.15.78:0/3131387828 reap_dead start 2023-10-11T09:42:49.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.080+0000 7fc876a15380 1 -- 172.21.15.78:0/3131387828 shutdown_connections 2023-10-11T09:42:49.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.080+0000 7fc876a15380 1 -- 172.21.15.78:0/3131387828 >> 172.21.15.78:0/3131387828 conn(0x564c0e344c00 msgr2=0x564c0e431800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:49.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.080+0000 7fc876a15380 1 -- 172.21.15.78:0/3131387828 shutdown_connections 2023-10-11T09:42:49.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.080+0000 7fc876a15380 1 -- 172.21.15.78:0/3131387828 wait complete. 2023-10-11T09:42:49.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.080+0000 7fc876a15380 1 -- 172.21.15.78:0/3987526030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c0e345000 msgr2=0x564c0e3eb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.080+0000 7fc876a15380 1 --2- 172.21.15.78:0/3987526030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c0e345000 0x564c0e3eb080 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x564c0f11eb70 tx=0x564c0f12cfc0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.080+0000 7fc876a15380 1 -- 172.21.15.78:0/3987526030 shutdown_connections 2023-10-11T09:42:49.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.080+0000 7fc876a15380 1 --2- 172.21.15.78:0/3987526030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c0e345000 0x564c0e3eb080 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.080+0000 7fc876a15380 1 --2- 172.21.15.78:0/3987526030 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564c0e345400 0x564c0e3eab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.080+0000 7fc876a15380 1 --2- 172.21.15.78:0/3987526030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c0e345800 0x564c0e3ea580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.080+0000 7fc876a15380 1 -- 172.21.15.78:0/3987526030 >> 172.21.15.78:0/3987526030 conn(0x564c0e345c00 msgr2=0x564c0e431000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:49.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.080+0000 7fc876a15380 1 -- 172.21.15.78:0/3987526030 shutdown_connections 2023-10-11T09:42:49.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.080+0000 7fc876a15380 1 -- 172.21.15.78:0/3987526030 wait complete. 2023-10-11T09:42:49.095 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'inode'] in time 0:00:00.096141, result: { "0": { "data": {}, "result": 0 } } 2023-10-11T09:42:49.095 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2023-10-11T09:42:49.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.152+0000 7fd1cd3a9380 1 -- start start 2023-10-11T09:42:49.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.152+0000 7fd1cd3a9380 1 Processor -- start 2023-10-11T09:42:49.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.156+0000 7fd1cd3a9380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b2951800 0x5595b29f6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.156+0000 7fd1cd3a9380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5595b2951400 0x5595b29f6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.156+0000 7fd1cd3a9380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595b2951000 0x5595b29f7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.156+0000 7fd1cd3a9380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b2987e00 con 0x5595b2951000 2023-10-11T09:42:49.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.156+0000 7fd1cd3a9380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b29d0a00 con 0x5595b2951800 2023-10-11T09:42:49.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.156+0000 7fd1cd3a9380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x5595b29d0640 con 0x5595b2951400 2023-10-11T09:42:49.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.156+0000 7fd1cbb63700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b2951800 0x5595b29f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.156+0000 7fd1cc364700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595b2951000 0x5595b29f7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.156+0000 7fd1cbb63700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b2951800 0x5595b29f6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:33688/0 (socket says 172.21.15.78:33688) 2023-10-11T09:42:49.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.156+0000 7fd1cbb63700 1 -- 172.21.15.78:0/524787977 learned_addr learned my addr 172.21.15.78:0/524787977 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:49.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.156+0000 7fd1cb362700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5595b2951400 0x5595b29f6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.156+0000 7fd1cc364700 1 -- 172.21.15.78:0/524787977 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5595b2951400 msgr2=0x5595b29f6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:49.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.156+0000 7fd1cc364700 1 --2- 172.21.15.78:0/524787977 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5595b2951400 0x5595b29f6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.156+0000 7fd1cc364700 1 -- 172.21.15.78:0/524787977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b2951800 msgr2=0x5595b29f6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:49.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.156+0000 7fd1cc364700 1 --2- 172.21.15.78:0/524787977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b2951800 0x5595b29f6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.156+0000 7fd1cc364700 1 -- 172.21.15.78:0/524787977 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595b2972b60 con 0x5595b2951000 2023-10-11T09:42:49.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.156+0000 7fd1cc364700 1 --2- 172.21.15.78:0/524787977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595b2951000 0x5595b29f7080 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x5595b29a69f0 tx=0x5595b3732d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=455f12b3340e4e7f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.156+0000 7fd1c935e700 1 -- 172.21.15.78:0/524787977 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5595b2981080 con 0x5595b2951000 2023-10-11T09:42:49.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.156+0000 7fd1c935e700 1 -- 172.21.15.78:0/524787977 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5595b372c300 con 0x5595b2951000 2023-10-11T09:42:49.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.156+0000 7fd1c935e700 1 -- 172.21.15.78:0/524787977 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5595b2981600 con 0x5595b2951000 2023-10-11T09:42:49.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.156+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/524787977 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5595b29731e0 con 0x5595b2951000 2023-10-11T09:42:49.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.156+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/524787977 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5595b2972340 con 0x5595b2951000 2023-10-11T09:42:49.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.156+0000 7fd1cd3a9380 4 waiting for MDS map... 2023-10-11T09:42:49.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.156+0000 7fd1c935e700 1 -- 172.21.15.78:0/524787977 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(44..44 src has 1..44) v4 ==== 6667+0+0 (secure 0 0 0) 0x5595b298ac40 con 0x5595b2951000 2023-10-11T09:42:49.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.156+0000 7fd1c935e700 1 -- 172.21.15.78:0/524787977 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) v1 ==== 3118+0+0 (secure 0 0 0) 0x5595b2992f00 con 0x5595b2951000 2023-10-11T09:42:49.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.156+0000 7fd1cd3a9380 4 Got MDS map 22 2023-10-11T09:42:49.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.156+0000 7fd1cd3a9380 10 main: JournalTool::main 2023-10-11T09:42:49.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.156+0000 7fd1cd3a9380 4 main: JournalTool: connecting to RADOS... 2023-10-11T09:42:49.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.160+0000 7fd1cd3a9380 1 Processor -- start 2023-10-11T09:42:49.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.160+0000 7fd1cd3a9380 1 -- start start 2023-10-11T09:42:49.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.160+0000 7fd1cd3a9380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595b2c52400 0x5595b29f7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.160+0000 7fd1cd3a9380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5595b2c52800 0x5595b29f7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.160+0000 7fd1cd3a9380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b2c52c00 0x5595b29f8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.160+0000 7fd1cd3a9380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b29d0500 con 0x5595b2c52400 2023-10-11T09:42:49.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.160+0000 7fd1cd3a9380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b29d03c0 con 0x5595b2c52c00 2023-10-11T09:42:49.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.160+0000 7fd1cd3a9380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x5595b29d0640 con 0x5595b2c52800 2023-10-11T09:42:49.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.160+0000 7fd1cb362700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595b2c52400 0x5595b29f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.160+0000 7fd1cb362700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595b2c52400 0x5595b29f7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58002/0 (socket says 172.21.15.78:58002) 2023-10-11T09:42:49.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.160+0000 7fd1cb362700 1 -- 172.21.15.78:0/4056835994 learned_addr learned my addr 172.21.15.78:0/4056835994 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:49.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.160+0000 7fd1cbb63700 1 --2- 172.21.15.78:0/4056835994 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5595b2c52800 0x5595b29f7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.160+0000 7fd1cb362700 1 --2- 172.21.15.78:0/4056835994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b2c52c00 0x5595b29f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.160+0000 7fd1cb362700 1 -- 172.21.15.78:0/4056835994 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5595b2c52800 msgr2=0x5595b29f7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:49.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.160+0000 7fd1cb362700 1 --2- 172.21.15.78:0/4056835994 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5595b2c52800 0x5595b29f7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.160+0000 7fd1cb362700 1 -- 172.21.15.78:0/4056835994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b2c52c00 msgr2=0x5595b29f8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:49.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.160+0000 7fd1cb362700 1 --2- 172.21.15.78:0/4056835994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b2c52c00 0x5595b29f8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.160+0000 7fd1cb362700 1 -- 172.21.15.78:0/4056835994 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595b2972000 con 0x5595b2c52400 2023-10-11T09:42:49.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.160+0000 7fd1cb362700 1 --2- 172.21.15.78:0/4056835994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595b2c52400 0x5595b29f7600 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x5595b2c9e120 tx=0x5595b2c56c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=696699c07857a781 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.160+0000 7fd1c735a700 1 -- 172.21.15.78:0/4056835994 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5595b2981080 con 0x5595b2c52400 2023-10-11T09:42:49.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.164+0000 7fd1c735a700 1 -- 172.21.15.78:0/4056835994 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5595b299c900 con 0x5595b2c52400 2023-10-11T09:42:49.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.164+0000 7fd1c735a700 1 -- 172.21.15.78:0/4056835994 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5595b2981600 con 0x5595b2c52400 2023-10-11T09:42:49.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.164+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/4056835994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595b2c52400 msgr2=0x5595b29f7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.164+0000 7fd1cd3a9380 1 --2- 172.21.15.78:0/4056835994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595b2c52400 0x5595b29f7600 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x5595b2c9e120 tx=0x5595b2c56c60 comp rx=0 tx=0).stop 2023-10-11T09:42:49.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.164+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/4056835994 shutdown_connections 2023-10-11T09:42:49.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.164+0000 7fd1cd3a9380 1 --2- 172.21.15.78:0/4056835994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b2c52c00 0x5595b29f8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.164+0000 7fd1cd3a9380 1 --2- 172.21.15.78:0/4056835994 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5595b2c52800 0x5595b29f7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.164+0000 7fd1cd3a9380 1 --2- 172.21.15.78:0/4056835994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595b2c52400 0x5595b29f7600 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.164+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/4056835994 >> 172.21.15.78:0/4056835994 conn(0x5595b2950c00 msgr2=0x5595b2a3d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:49.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.164+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/4056835994 shutdown_connections 2023-10-11T09:42:49.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.164+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/4056835994 wait complete. 2023-10-11T09:42:49.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.164+0000 7fd1cd3a9380 1 Processor -- start 2023-10-11T09:42:49.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.164+0000 7fd1cd3a9380 1 -- start start 2023-10-11T09:42:49.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.164+0000 7fd1cd3a9380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b2c52400 0x5595b29f9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.164+0000 7fd1cd3a9380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595b2c52800 0x5595b29f8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.164+0000 7fd1cd3a9380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5595b2c52c00 0x5595b29f8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.164+0000 7fd1cd3a9380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b29d0280 con 0x5595b2c52800 2023-10-11T09:42:49.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.164+0000 7fd1cd3a9380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b29d0140 con 0x5595b2c52400 2023-10-11T09:42:49.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.164+0000 7fd1cd3a9380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x5595b29d0000 con 0x5595b2c52c00 2023-10-11T09:42:49.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.168+0000 7fd1cbb63700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595b2c52800 0x5595b29f8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.168+0000 7fd1cbb63700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595b2c52800 0x5595b29f8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58014/0 (socket says 172.21.15.78:58014) 2023-10-11T09:42:49.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.168+0000 7fd1cbb63700 1 -- 172.21.15.78:0/2682406212 learned_addr learned my addr 172.21.15.78:0/2682406212 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:49.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.168+0000 7fd1cb362700 1 --2- 172.21.15.78:0/2682406212 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5595b2c52c00 0x5595b29f8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.168+0000 7fd1cb362700 1 --2- 172.21.15.78:0/2682406212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b2c52400 0x5595b29f9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.168+0000 7fd1cbb63700 1 -- 172.21.15.78:0/2682406212 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5595b2c52c00 msgr2=0x5595b29f8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.168+0000 7fd1cbb63700 1 --2- 172.21.15.78:0/2682406212 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5595b2c52c00 0x5595b29f8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.168+0000 7fd1cbb63700 1 -- 172.21.15.78:0/2682406212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b2c52400 msgr2=0x5595b29f9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.168+0000 7fd1cbb63700 1 --2- 172.21.15.78:0/2682406212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b2c52400 0x5595b29f9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.168+0000 7fd1cbb63700 1 -- 172.21.15.78:0/2682406212 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595b2972b60 con 0x5595b2c52800 2023-10-11T09:42:49.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.168+0000 7fd1cb362700 1 --2- 172.21.15.78:0/2682406212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b2c52400 0x5595b29f9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:42:49.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.168+0000 7fd1cbb63700 1 --2- 172.21.15.78:0/2682406212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595b2c52800 0x5595b29f8c00 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x5595b2c9f500 tx=0x5595b2c56d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.172+0000 7fd1c5b57700 1 -- 172.21.15.78:0/2682406212 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5595b2981080 con 0x5595b2c52800 2023-10-11T09:42:49.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.172+0000 7fd1c5b57700 1 -- 172.21.15.78:0/2682406212 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5595b299c480 con 0x5595b2c52800 2023-10-11T09:42:49.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.172+0000 7fd1c5b57700 1 -- 172.21.15.78:0/2682406212 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5595b2981600 con 0x5595b2c52800 2023-10-11T09:42:49.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.172+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/2682406212 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5595b29731e0 con 0x5595b2c52800 2023-10-11T09:42:49.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.172+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/2682406212 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5595b2972340 con 0x5595b2c52800 2023-10-11T09:42:49.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.172+0000 7fd1cd3a9380 4 main: JournalTool: resolving pool 5 2023-10-11T09:42:49.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.172+0000 7fd1c5b57700 1 -- 172.21.15.78:0/2682406212 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x5595b29ce840 con 0x5595b2c52800 2023-10-11T09:42:49.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.172+0000 7fd1c5b57700 1 --2- 172.21.15.78:0/2682406212 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5595b2d94000 0x5595b29f9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.172+0000 7fd1c5b57700 1 -- 172.21.15.78:0/2682406212 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6667+0+0 (secure 0 0 0) 0x5595b298aa80 con 0x5595b2c52800 2023-10-11T09:42:49.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.172+0000 7fd1cd3a9380 4 main: JournalTool: creating IoCtx.. 2023-10-11T09:42:49.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.172+0000 7fd1cd3a9380 4 main: Executing for rank 0 2023-10-11T09:42:49.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.176+0000 7fd1cb362700 1 --2- 172.21.15.78:0/2682406212 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5595b2d94000 0x5595b29f9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.176+0000 7fd1cb362700 1 --2- 172.21.15.78:0/2682406212 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5595b2d94000 0x5595b29f9700 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x5595b2c9e120 tx=0x5595b2c56cc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.176+0000 7fd1cd3a9380 1 -- start start 2023-10-11T09:42:49.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.176+0000 7fd1cd3a9380 1 Processor -- start 2023-10-11T09:42:49.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.176+0000 7fd1cd3a9380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595b2c53400 0x5595b29f8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.176+0000 7fd1cd3a9380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b2c53800 0x5595b29f7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.176+0000 7fd1cd3a9380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5595b2d94400 0x5595b29f7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.176+0000 7fd1cd3a9380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b29d1040 con 0x5595b2c53400 2023-10-11T09:42:49.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.176+0000 7fd1cd3a9380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b29d0140 con 0x5595b2c53800 2023-10-11T09:42:49.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.176+0000 7fd1cd3a9380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x5595b29d0640 con 0x5595b2d94400 2023-10-11T09:42:49.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.176+0000 7fd1cc364700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595b2c53400 0x5595b29f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.176+0000 7fd1cc364700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595b2c53400 0x5595b29f8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58024/0 (socket says 172.21.15.78:58024) 2023-10-11T09:42:49.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.176+0000 7fd1cc364700 1 -- 172.21.15.78:0/1738178942 learned_addr learned my addr 172.21.15.78:0/1738178942 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:49.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.176+0000 7fd1cbb63700 1 --2- 172.21.15.78:0/1738178942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b2c53800 0x5595b29f7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.176+0000 7fd1cb362700 1 --2- 172.21.15.78:0/1738178942 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5595b2d94400 0x5595b29f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.180+0000 7fd1cc364700 1 -- 172.21.15.78:0/1738178942 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5595b2d94400 msgr2=0x5595b29f7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:49.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.180+0000 7fd1cc364700 1 --2- 172.21.15.78:0/1738178942 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5595b2d94400 0x5595b29f7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.180+0000 7fd1cc364700 1 -- 172.21.15.78:0/1738178942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b2c53800 msgr2=0x5595b29f7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:49.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.180+0000 7fd1cc364700 1 --2- 172.21.15.78:0/1738178942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b2c53800 0x5595b29f7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.180+0000 7fd1cc364700 1 -- 172.21.15.78:0/1738178942 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595b2ce6d00 con 0x5595b2c53400 2023-10-11T09:42:49.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.180+0000 7fd1cbb63700 1 --2- 172.21.15.78:0/1738178942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b2c53800 0x5595b29f7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:42:49.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.180+0000 7fd1cc364700 1 --2- 172.21.15.78:0/1738178942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595b2c53400 0x5595b29f8100 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x5595b2d650b0 tx=0x5595b3732f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4fdeea4a287df7b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.180+0000 7fd1c2b51700 1 -- 172.21.15.78:0/1738178942 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5595b29811e0 con 0x5595b2c53400 2023-10-11T09:42:49.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.180+0000 7fd1c2b51700 1 -- 172.21.15.78:0/1738178942 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5595b372cc00 con 0x5595b2c53400 2023-10-11T09:42:49.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.180+0000 7fd1c2b51700 1 -- 172.21.15.78:0/1738178942 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5595b2980f20 con 0x5595b2c53400 2023-10-11T09:42:49.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.180+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/1738178942 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5595b2ce6b60 con 0x5595b2c53400 2023-10-11T09:42:49.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.180+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/1738178942 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5595b2ce69c0 con 0x5595b2c53400 2023-10-11T09:42:49.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.180+0000 7fd1cd3a9380 4 waiting for MDS map... 2023-10-11T09:42:49.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.180+0000 7fd1c2b51700 1 -- 172.21.15.78:0/1738178942 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(44..44 src has 1..44) v4 ==== 6667+0+0 (secure 0 0 0) 0x5595b298a8c0 con 0x5595b2c53400 2023-10-11T09:42:49.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.180+0000 7fd1c2b51700 1 -- 172.21.15.78:0/1738178942 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) v1 ==== 3118+0+0 (secure 0 0 0) 0x5595b2ded500 con 0x5595b2c53400 2023-10-11T09:42:49.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.180+0000 7fd1cd3a9380 4 Got MDS map 22 2023-10-11T09:42:49.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.180+0000 7fd1cd3a9380 1 --2- 172.21.15.78:0/1738178942 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x5595b2d94800 0x5595b2e1a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.180+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/1738178942 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:1 5.f 5:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x5595b2d94c00 con 0x5595b2d94800 2023-10-11T09:42:49.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.180+0000 7fd1cbb63700 1 --2- 172.21.15.78:0/1738178942 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x5595b2d94800 0x5595b2e1a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.180+0000 7fd1cbb63700 1 --2- 172.21.15.78:0/1738178942 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x5595b2d94800 0x5595b2e1a000 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=73c80266d10d25c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.184+0000 7fd1cbb63700 1 -- 172.21.15.78:0/1738178942 <== osd.5 v2:172.21.15.133:6816/35899 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v44'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x5595b37c5200 con 0x5595b2d94800 2023-10-11T09:42:49.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.184+0000 7fd1cd3a9380 1 --2- 172.21.15.78:0/1738178942 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5595b2d95000 0x5595b2e1a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.184+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/1738178942 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:2 5.14 5:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x5595b2d95400 con 0x5595b2d95000 2023-10-11T09:42:49.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.184+0000 7fd1cb362700 1 --2- 172.21.15.78:0/1738178942 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5595b2d95000 0x5595b2e1a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.184+0000 7fd1cb362700 1 --2- 172.21.15.78:0/1738178942 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5595b2d95000 0x5595b2e1a580 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=7264f5eabb081484 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.188+0000 7fd1cb362700 1 -- 172.21.15.78:0/1738178942 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v44'38 uv38 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x5595b37c5440 con 0x5595b2d95000 2023-10-11T09:42:49.195 INFO:teuthology.orchestra.run.smithi078.stdout:writing EResetJournal entry 2023-10-11T09:42:49.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.188+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/1738178942 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:3 5.14 5:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x5595b2d95800 con 0x5595b2d95000 2023-10-11T09:42:49.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.188+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/1738178942 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:4 5.1b 5:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x5595b2d95c00 con 0x5595b2d94800 2023-10-11T09:42:49.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.188+0000 7fd1cd3a9380 1 --2- 172.21.15.78:0/1738178942 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x5595b2dfc800 0x5595b2e1ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.188+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/1738178942 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:5 5.b 5:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x5595b2dfcc00 con 0x5595b2dfc800 2023-10-11T09:42:49.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.188+0000 7fd1cd3a9380 1 --2- 172.21.15.78:0/1738178942 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x5595b2dfd000 0x5595b2e1b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.188+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/1738178942 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:6 5.1f 5:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x5595b2dfd400 con 0x5595b2dfd000 2023-10-11T09:42:49.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.188+0000 7fd1cd3a9380 1 --2- 172.21.15.78:0/1738178942 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x5595b2dfd800 0x5595b2e1b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.188+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/1738178942 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:7 5.1a 5:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x5595b2dfdc00 con 0x5595b2dfd800 2023-10-11T09:42:49.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.188+0000 7fd1cd3a9380 1 --2- 172.21.15.78:0/1738178942 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x5595b2d95400 0x5595b2e1bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.188+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/1738178942 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:8 5.1e 5:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x5595b2d95800 con 0x5595b2d95400 2023-10-11T09:42:49.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.192+0000 7fd1cc364700 1 --2- 172.21.15.78:0/1738178942 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x5595b2dfc800 0x5595b2e1ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.192+0000 7fd1cc364700 1 --2- 172.21.15.78:0/1738178942 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x5595b2d95400 0x5595b2e1bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.192+0000 7fd1cbb63700 1 --2- 172.21.15.78:0/1738178942 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x5595b2dfd000 0x5595b2e1b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.192+0000 7fd1cb362700 1 --2- 172.21.15.78:0/1738178942 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x5595b2dfd800 0x5595b2e1b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.192+0000 7fd1cc364700 1 --2- 172.21.15.78:0/1738178942 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x5595b2dfc800 0x5595b2e1ab00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=d6b52194408e1f27 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.192+0000 7fd1cbb63700 1 --2- 172.21.15.78:0/1738178942 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x5595b2dfd000 0x5595b2e1b080 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=ff2b5de27397e2d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.192+0000 7fd1cb362700 1 --2- 172.21.15.78:0/1738178942 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x5595b2dfd800 0x5595b2e1b600 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=6679548db71bf server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.192+0000 7fd1cb362700 1 -- 172.21.15.78:0/1738178942 <== osd.3 v2:172.21.15.78:6824/35738 2 ==== osd_op_reply(3 200.00000001 [delete] v44'39 uv38 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5595b37c5440 con 0x5595b2d95000 2023-10-11T09:42:49.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.192+0000 7fd1cc364700 1 --2- 172.21.15.78:0/1738178942 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x5595b2d95400 0x5595b2e1bb80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=a9c4e970566285b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.192+0000 7fd1cbb63700 1 -- 172.21.15.78:0/1738178942 <== osd.5 v2:172.21.15.133:6816/35899 2 ==== osd_op_reply(4 200.00000002 [delete] v44'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5595b37c5200 con 0x5595b2d94800 2023-10-11T09:42:49.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.192+0000 7fd1c134e700 1 -- 172.21.15.78:0/1738178942 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:9 5.14 5:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x5595b2d94c00 con 0x5595b2d95000 2023-10-11T09:42:49.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cbb63700 1 -- 172.21.15.78:0/1738178942 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== osd_op_reply(6 200.00000004 [delete] v44'9 uv8 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5595b37c5200 con 0x5595b2dfd000 2023-10-11T09:42:49.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cc364700 1 -- 172.21.15.78:0/1738178942 <== osd.2 v2:172.21.15.78:6808/35732 1 ==== osd_op_reply(5 200.00000003 [delete] v44'6 uv5 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5595b2e3c000 con 0x5595b2dfc800 2023-10-11T09:42:49.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cb362700 1 -- 172.21.15.78:0/1738178942 <== osd.7 v2:172.21.15.133:6824/35901 1 ==== osd_op_reply(7 200.00000005 [delete] v44'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5595b37c5440 con 0x5595b2dfd800 2023-10-11T09:42:49.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cb362700 1 -- 172.21.15.78:0/1738178942 <== osd.3 v2:172.21.15.78:6824/35738 3 ==== osd_op_reply(9 200.00000001 [write 0~48] v44'40 uv40 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x5595b37c5440 con 0x5595b2d95000 2023-10-11T09:42:49.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cc364700 1 -- 172.21.15.78:0/1738178942 <== osd.0 v2:172.21.15.78:6800/35735 1 ==== osd_op_reply(8 200.00000006 [delete] v44'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5595b2e3c000 con 0x5595b2d95400 2023-10-11T09:42:49.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cd3a9380 4 Successfully wrote new journal pointer and header for rank 2:0 2023-10-11T09:42:49.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/1738178942 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x5595b2d95400 msgr2=0x5595b2e1bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cd3a9380 1 --2- 172.21.15.78:0/1738178942 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x5595b2d95400 0x5595b2e1bb80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/1738178942 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x5595b2dfc800 msgr2=0x5595b2e1ab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cd3a9380 1 --2- 172.21.15.78:0/1738178942 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x5595b2dfc800 0x5595b2e1ab00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/1738178942 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5595b2d95000 msgr2=0x5595b2e1a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cd3a9380 1 --2- 172.21.15.78:0/1738178942 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5595b2d95000 0x5595b2e1a580 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/1738178942 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x5595b2d94800 msgr2=0x5595b2e1a000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cd3a9380 1 --2- 172.21.15.78:0/1738178942 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x5595b2d94800 0x5595b2e1a000 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/1738178942 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x5595b2dfd000 msgr2=0x5595b2e1b080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cd3a9380 1 --2- 172.21.15.78:0/1738178942 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x5595b2dfd000 0x5595b2e1b080 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cb362700 1 -- 172.21.15.78:0/1738178942 reap_dead start 2023-10-11T09:42:49.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/1738178942 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x5595b2dfd800 msgr2=0x5595b2e1b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cd3a9380 1 --2- 172.21.15.78:0/1738178942 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x5595b2dfd800 0x5595b2e1b600 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/1738178942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595b2c53400 msgr2=0x5595b29f8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cd3a9380 1 --2- 172.21.15.78:0/1738178942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595b2c53400 0x5595b29f8100 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x5595b2d650b0 tx=0x5595b3732f60 comp rx=0 tx=0).stop 2023-10-11T09:42:49.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cb362700 1 -- 172.21.15.78:0/1738178942 reap_dead start 2023-10-11T09:42:49.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cb362700 1 -- 172.21.15.78:0/1738178942 reap_dead start 2023-10-11T09:42:49.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/1738178942 shutdown_connections 2023-10-11T09:42:49.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/1738178942 >> 172.21.15.78:0/1738178942 conn(0x5595b2c53000 msgr2=0x5595b2a3e000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:49.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/1738178942 shutdown_connections 2023-10-11T09:42:49.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/1738178942 wait complete. 2023-10-11T09:42:49.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/2682406212 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5595b2d94000 msgr2=0x5595b29f9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cd3a9380 1 --2- 172.21.15.78:0/2682406212 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5595b2d94000 0x5595b29f9700 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x5595b2c9e120 tx=0x5595b2c56cc0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/2682406212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595b2c52800 msgr2=0x5595b29f8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cd3a9380 1 --2- 172.21.15.78:0/2682406212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595b2c52800 0x5595b29f8c00 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x5595b2c9f500 tx=0x5595b2c56d20 comp rx=0 tx=0).stop 2023-10-11T09:42:49.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/2682406212 shutdown_connections 2023-10-11T09:42:49.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cd3a9380 1 --2- 172.21.15.78:0/2682406212 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5595b2d94000 0x5595b29f9700 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cd3a9380 1 --2- 172.21.15.78:0/2682406212 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5595b2c52c00 0x5595b29f8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cd3a9380 1 --2- 172.21.15.78:0/2682406212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595b2c52800 0x5595b29f8c00 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cd3a9380 1 --2- 172.21.15.78:0/2682406212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b2c52400 0x5595b29f9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/2682406212 >> 172.21.15.78:0/2682406212 conn(0x5595b2950c00 msgr2=0x5595b2a3d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:49.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/2682406212 shutdown_connections 2023-10-11T09:42:49.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.196+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/2682406212 wait complete. 2023-10-11T09:42:49.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.200+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/524787977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595b2951000 msgr2=0x5595b29f7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.200+0000 7fd1cd3a9380 1 --2- 172.21.15.78:0/524787977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595b2951000 0x5595b29f7080 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x5595b29a69f0 tx=0x5595b3732d80 comp rx=0 tx=0).stop 2023-10-11T09:42:49.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.200+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/524787977 shutdown_connections 2023-10-11T09:42:49.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.200+0000 7fd1cd3a9380 1 --2- 172.21.15.78:0/524787977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595b2951000 0x5595b29f7080 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.200+0000 7fd1cd3a9380 1 --2- 172.21.15.78:0/524787977 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5595b2951400 0x5595b29f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.200+0000 7fd1cd3a9380 1 --2- 172.21.15.78:0/524787977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595b2951800 0x5595b29f6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.200+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/524787977 >> 172.21.15.78:0/524787977 conn(0x5595b2951c00 msgr2=0x5595b2a3d000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:49.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.200+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/524787977 shutdown_connections 2023-10-11T09:42:49.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.200+0000 7fd1cd3a9380 1 -- 172.21.15.78:0/524787977 wait complete. 2023-10-11T09:42:49.215 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.120194, result: writing EResetJournal entry 2023-10-11T09:42:49.216 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 init 2023-10-11T09:42:49.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.276+0000 7fe2ccbb4840 1 -- start start 2023-10-11T09:42:49.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.276+0000 7fe2ccbb4840 1 Processor -- start 2023-10-11T09:42:49.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.276+0000 7fe2ccbb4840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f65e2ed800 0x55f65e392580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.276+0000 7fe2ccbb4840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f65e2ed400 0x55f65e392b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.276+0000 7fe2ccbb4840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f65e2ed000 0x55f65e393080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.276+0000 7fe2ccbb4840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f65e36ca00 con 0x55f65e2ed800 2023-10-11T09:42:49.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.276+0000 7fe2ccbb4840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f65e36c780 con 0x55f65e2ed400 2023-10-11T09:42:49.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.276+0000 7fe2ccbb4840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f65e36c640 con 0x55f65e2ed000 2023-10-11T09:42:49.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.276+0000 7fe2cb370700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f65e2ed800 0x55f65e392580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.276+0000 7fe2cb370700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f65e2ed800 0x55f65e392580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58046/0 (socket says 172.21.15.78:58046) 2023-10-11T09:42:49.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.276+0000 7fe2cb370700 1 -- 172.21.15.78:0/1099509247 learned_addr learned my addr 172.21.15.78:0/1099509247 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:49.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.276+0000 7fe2cab6f700 1 --2- 172.21.15.78:0/1099509247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f65e2ed400 0x55f65e392b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.276+0000 7fe2cbb71700 1 --2- 172.21.15.78:0/1099509247 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f65e2ed000 0x55f65e393080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.276+0000 7fe2cb370700 1 -- 172.21.15.78:0/1099509247 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f65e2ed000 msgr2=0x55f65e393080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:49.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.276+0000 7fe2cb370700 1 --2- 172.21.15.78:0/1099509247 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f65e2ed000 0x55f65e393080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.276+0000 7fe2cb370700 1 -- 172.21.15.78:0/1099509247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f65e2ed400 msgr2=0x55f65e392b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:49.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.276+0000 7fe2cb370700 1 --2- 172.21.15.78:0/1099509247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f65e2ed400 0x55f65e392b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.276+0000 7fe2cb370700 1 -- 172.21.15.78:0/1099509247 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f65e30eb60 con 0x55f65e2ed800 2023-10-11T09:42:49.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.276+0000 7fe2cab6f700 1 --2- 172.21.15.78:0/1099509247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f65e2ed400 0x55f65e392b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:42:49.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.276+0000 7fe2cbb71700 1 --2- 172.21.15.78:0/1099509247 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f65e2ed000 0x55f65e393080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:42:49.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.276+0000 7fe2cb370700 1 --2- 172.21.15.78:0/1099509247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f65e2ed800 0x55f65e392580 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x55f65f055410 tx=0x55f65f0dae40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a61c31a0721764 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.280+0000 7fe2c8b6b700 1 -- 172.21.15.78:0/1099509247 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f65e31d600 con 0x55f65e2ed800 2023-10-11T09:42:49.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.280+0000 7fe2c8b6b700 1 -- 172.21.15.78:0/1099509247 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f65e338180 con 0x55f65e2ed800 2023-10-11T09:42:49.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.280+0000 7fe2c8b6b700 1 -- 172.21.15.78:0/1099509247 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f65e31d1e0 con 0x55f65e2ed800 2023-10-11T09:42:49.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.280+0000 7fe2ccbb4840 1 -- 172.21.15.78:0/1099509247 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f65e30e340 con 0x55f65e2ed800 2023-10-11T09:42:49.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.280+0000 7fe2ccbb4840 1 -- 172.21.15.78:0/1099509247 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55f65e30e000 con 0x55f65e2ed800 2023-10-11T09:42:49.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.280+0000 7fe2ccbb4840 4 waiting for MDS map... 2023-10-11T09:42:49.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.280+0000 7fe2c8b6b700 1 -- 172.21.15.78:0/1099509247 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(44..44 src has 1..44) v4 ==== 6667+0+0 (secure 0 0 0) 0x55f65e326c40 con 0x55f65e2ed800 2023-10-11T09:42:49.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.280+0000 7fe2c8b6b700 1 -- 172.21.15.78:0/1099509247 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) v1 ==== 3118+0+0 (secure 0 0 0) 0x55f65f174000 con 0x55f65e2ed800 2023-10-11T09:42:49.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.280+0000 7fe2ccbb4840 4 Got MDS map 22 2023-10-11T09:42:49.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.280+0000 7fe2ccbb4840 4 datascan.main: Using metadata pool output 2023-10-11T09:42:49.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.280+0000 7fe2ccbb4840 4 datascan.main: connecting to RADOS... 2023-10-11T09:42:49.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.280+0000 7fe2ccbb4840 1 Processor -- start 2023-10-11T09:42:49.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.280+0000 7fe2ccbb4840 1 -- start start 2023-10-11T09:42:49.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.280+0000 7fe2ccbb4840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f65f170400 0x55f65e393600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.280+0000 7fe2ccbb4840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f65f170800 0x55f65e393b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.280+0000 7fe2ccbb4840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f65f170c00 0x55f65e394100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.280+0000 7fe2ccbb4840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f65e323e00 con 0x55f65f170c00 2023-10-11T09:42:49.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.280+0000 7fe2ccbb4840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f65e36c780 con 0x55f65f170400 2023-10-11T09:42:49.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.280+0000 7fe2ccbb4840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f65e36c500 con 0x55f65f170800 2023-10-11T09:42:49.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.280+0000 7fe2cab6f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f65f170c00 0x55f65e394100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.280+0000 7fe2cab6f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f65f170c00 0x55f65e394100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58058/0 (socket says 172.21.15.78:58058) 2023-10-11T09:42:49.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.280+0000 7fe2cab6f700 1 -- 172.21.15.78:0/2748349903 learned_addr learned my addr 172.21.15.78:0/2748349903 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:49.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.280+0000 7fe2cbb71700 1 --2- 172.21.15.78:0/2748349903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f65f170400 0x55f65e393600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.280+0000 7fe2cb370700 1 --2- 172.21.15.78:0/2748349903 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f65f170800 0x55f65e393b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.284+0000 7fe2cab6f700 1 -- 172.21.15.78:0/2748349903 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f65f170800 msgr2=0x55f65e393b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:49.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.284+0000 7fe2cab6f700 1 --2- 172.21.15.78:0/2748349903 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f65f170800 0x55f65e393b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.284+0000 7fe2cab6f700 1 -- 172.21.15.78:0/2748349903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f65f170400 msgr2=0x55f65e393600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:49.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.284+0000 7fe2cab6f700 1 --2- 172.21.15.78:0/2748349903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f65f170400 0x55f65e393600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.284+0000 7fe2cab6f700 1 -- 172.21.15.78:0/2748349903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f65e30eea0 con 0x55f65f170c00 2023-10-11T09:42:49.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.284+0000 7fe2cbb71700 1 --2- 172.21.15.78:0/2748349903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f65f170400 0x55f65e393600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:42:49.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.284+0000 7fe2cb370700 1 --2- 172.21.15.78:0/2748349903 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f65f170800 0x55f65e393b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:42:49.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.284+0000 7fe2cab6f700 1 --2- 172.21.15.78:0/2748349903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f65f170c00 0x55f65e394100 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x55f65e5e0750 tx=0x55f65e3c33e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d090c68e5f77b71e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.284+0000 7fe2c6b67700 1 -- 172.21.15.78:0/2748349903 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f65e31d080 con 0x55f65f170c00 2023-10-11T09:42:49.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.284+0000 7fe2c6b67700 1 -- 172.21.15.78:0/2748349903 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f65e3e2a80 con 0x55f65f170c00 2023-10-11T09:42:49.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.284+0000 7fe2c6b67700 1 -- 172.21.15.78:0/2748349903 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f65e31d600 con 0x55f65f170c00 2023-10-11T09:42:49.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.284+0000 7fe2ccbb4840 1 -- 172.21.15.78:0/2748349903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f65f170c00 msgr2=0x55f65e394100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.284+0000 7fe2ccbb4840 1 --2- 172.21.15.78:0/2748349903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f65f170c00 0x55f65e394100 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x55f65e5e0750 tx=0x55f65e3c33e0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.284+0000 7fe2ccbb4840 1 -- 172.21.15.78:0/2748349903 shutdown_connections 2023-10-11T09:42:49.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.284+0000 7fe2ccbb4840 1 --2- 172.21.15.78:0/2748349903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f65f170c00 0x55f65e394100 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.284+0000 7fe2ccbb4840 1 --2- 172.21.15.78:0/2748349903 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f65f170800 0x55f65e393b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.284+0000 7fe2ccbb4840 1 --2- 172.21.15.78:0/2748349903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f65f170400 0x55f65e393600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.284+0000 7fe2ccbb4840 1 -- 172.21.15.78:0/2748349903 >> 172.21.15.78:0/2748349903 conn(0x55f65e2ecc00 msgr2=0x55f65e3bb800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:49.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.284+0000 7fe2ccbb4840 1 -- 172.21.15.78:0/2748349903 shutdown_connections 2023-10-11T09:42:49.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.284+0000 7fe2ccbb4840 1 -- 172.21.15.78:0/2748349903 wait complete. 2023-10-11T09:42:49.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.284+0000 7fe2ccbb4840 1 Processor -- start 2023-10-11T09:42:49.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.284+0000 7fe2ccbb4840 1 -- start start 2023-10-11T09:42:49.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.288+0000 7fe2ccbb4840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f65f170400 0x55f65e395180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.288+0000 7fe2ccbb4840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f65f170800 0x55f65e394c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.288+0000 7fe2ccbb4840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f65f170c00 0x55f65e394680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.288+0000 7fe2ccbb4840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f65e36c3c0 con 0x55f65f170800 2023-10-11T09:42:49.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.288+0000 7fe2ccbb4840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f65e36c280 con 0x55f65f170400 2023-10-11T09:42:49.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.288+0000 7fe2ccbb4840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f65e36c140 con 0x55f65f170c00 2023-10-11T09:42:49.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.288+0000 7fe2cb370700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f65f170800 0x55f65e394c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.288+0000 7fe2cb370700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f65f170800 0x55f65e394c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58066/0 (socket says 172.21.15.78:58066) 2023-10-11T09:42:49.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.288+0000 7fe2cb370700 1 -- 172.21.15.78:0/4066774506 learned_addr learned my addr 172.21.15.78:0/4066774506 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:49.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.288+0000 7fe2cab6f700 1 --2- 172.21.15.78:0/4066774506 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f65f170c00 0x55f65e394680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.288+0000 7fe2cbb71700 1 --2- 172.21.15.78:0/4066774506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f65f170400 0x55f65e395180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.288+0000 7fe2cb370700 1 -- 172.21.15.78:0/4066774506 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f65f170c00 msgr2=0x55f65e394680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.288+0000 7fe2cb370700 1 --2- 172.21.15.78:0/4066774506 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f65f170c00 0x55f65e394680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.288+0000 7fe2cb370700 1 -- 172.21.15.78:0/4066774506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f65f170400 msgr2=0x55f65e395180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.288+0000 7fe2cb370700 1 --2- 172.21.15.78:0/4066774506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f65f170400 0x55f65e395180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.288+0000 7fe2cb370700 1 -- 172.21.15.78:0/4066774506 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f65e30f1e0 con 0x55f65f170800 2023-10-11T09:42:49.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.288+0000 7fe2cb370700 1 --2- 172.21.15.78:0/4066774506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f65f170800 0x55f65e394c00 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x55f65e5e06f0 tx=0x55f65f0dad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.288+0000 7fe2c5364700 1 -- 172.21.15.78:0/4066774506 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f65e31cf20 con 0x55f65f170800 2023-10-11T09:42:49.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.288+0000 7fe2c5364700 1 -- 172.21.15.78:0/4066774506 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f65e3e2c00 con 0x55f65f170800 2023-10-11T09:42:49.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.288+0000 7fe2c5364700 1 -- 172.21.15.78:0/4066774506 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f65e31d080 con 0x55f65f170800 2023-10-11T09:42:49.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.288+0000 7fe2ccbb4840 1 -- 172.21.15.78:0/4066774506 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f65e30eea0 con 0x55f65f170800 2023-10-11T09:42:49.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.288+0000 7fe2ccbb4840 1 -- 172.21.15.78:0/4066774506 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f65e30eb60 con 0x55f65f170800 2023-10-11T09:42:49.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.288+0000 7fe2ccbb4840 4 datascan.init: resolving metadata pool 5 2023-10-11T09:42:49.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.292+0000 7fe2c5364700 1 -- 172.21.15.78:0/4066774506 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x55f65e34a840 con 0x55f65f170800 2023-10-11T09:42:49.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.292+0000 7fe2c5364700 1 --2- 172.21.15.78:0/4066774506 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55f65e72a000 0x55f65e395700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.292+0000 7fe2c5364700 1 -- 172.21.15.78:0/4066774506 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6667+0+0 (secure 0 0 0) 0x55f65e326a80 con 0x55f65f170800 2023-10-11T09:42:49.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.292+0000 7fe2ccbb4840 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-10-11T09:42:49.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.292+0000 7fe2ccbb4840 1 --2- 172.21.15.78:0/4066774506 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55f65f171000 0x55f65e394100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.292+0000 7fe2cbb71700 1 --2- 172.21.15.78:0/4066774506 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55f65e72a000 0x55f65e395700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.292+0000 7fe2ccbb4840 1 -- 172.21.15.78:0/4066774506 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:1 5.14 5:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55f65f171400 con 0x55f65f171000 2023-10-11T09:42:49.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.292+0000 7fe2cab6f700 1 --2- 172.21.15.78:0/4066774506 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55f65f171000 0x55f65e394100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.296+0000 7fe2cbb71700 1 --2- 172.21.15.78:0/4066774506 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55f65e72a000 0x55f65e395700 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55f65f0546f0 tx=0x55f65e3c3260 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.296+0000 7fe2cab6f700 1 --2- 172.21.15.78:0/4066774506 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55f65f171000 0x55f65e394100 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.296+0000 7fe2cab6f700 1 -- 172.21.15.78:0/4066774506 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(1 1.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 160+0+0 (crc 0 0 0) 0x55f65f0ccfc0 con 0x55f65f171000 2023-10-11T09:42:49.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.296+0000 7fe2ccbb4840 1 -- 172.21.15.78:0/4066774506 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:2 5.14 5:287034c4:::1.00000000.inode:head [writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x55f65f171800 con 0x55f65f171000 2023-10-11T09:42:49.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.300+0000 7fe2cab6f700 1 -- 172.21.15.78:0/4066774506 <== osd.3 v2:172.21.15.78:6824/35738 2 ==== osd_op_reply(2 1.00000000.inode [writefull 0~582] v44'41 uv41 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x55f65f0ccfc0 con 0x55f65f171000 2023-10-11T09:42:49.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.300+0000 7fe2ccbb4840 1 --2- 172.21.15.78:0/4066774506 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55f65e72a400 0x55f65e393b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.300+0000 7fe2ccbb4840 1 -- 172.21.15.78:0/4066774506 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:3 5.1f 5:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55f65e72a800 con 0x55f65e72a400 2023-10-11T09:42:49.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.300+0000 7fe2cbb71700 1 --2- 172.21.15.78:0/4066774506 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55f65e72a400 0x55f65e393b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.300+0000 7fe2cbb71700 1 --2- 172.21.15.78:0/4066774506 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55f65e72a400 0x55f65e393b80 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.300+0000 7fe2cbb71700 1 -- 172.21.15.78:0/4066774506 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== osd_op_reply(3 100.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 162+0+0 (crc 0 0 0) 0x55f65f0cd680 con 0x55f65e72a400 2023-10-11T09:42:49.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.300+0000 7fe2ccbb4840 1 -- 172.21.15.78:0/4066774506 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:4 5.1f 5:fe07bba1:::100.00000000.inode:head [writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x55f65e72ac00 con 0x55f65e72a400 2023-10-11T09:42:49.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.304+0000 7fe2cbb71700 1 -- 172.21.15.78:0/4066774506 <== osd.6 v2:172.21.15.133:6800/35900 2 ==== osd_op_reply(4 100.00000000.inode [writefull 0~582] v44'10 uv10 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x55f65f0cd680 con 0x55f65e72a400 2023-10-11T09:42:49.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.304+0000 7fe2ccbb4840 1 -- 172.21.15.78:0/4066774506 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:5 5.13 5:cd5a64a3:::100.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55f65e72b000 con 0x55f65f171000 2023-10-11T09:42:49.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.304+0000 7fe2cab6f700 1 -- 172.21.15.78:0/4066774506 <== osd.3 v2:172.21.15.78:6824/35738 3 ==== osd_op_reply(5 100.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55f65f0ccfc0 con 0x55f65f171000 2023-10-11T09:42:49.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.304+0000 7fe2ccbb4840 10 datascan.find_or_create_dirfrag: read_version = 0 2023-10-11T09:42:49.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.304+0000 7fe2ccbb4840 1 -- 172.21.15.78:0/4066774506 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:6 5.13 5:cd5a64a3:::100.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x55f65e72b400 con 0x55f65f171000 2023-10-11T09:42:49.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.308+0000 7fe2cab6f700 1 -- 172.21.15.78:0/4066774506 <== osd.3 v2:172.21.15.78:6824/35738 4 ==== osd_op_reply(6 100.00000000 [create,omap-set-header] v44'7 uv7 ondisk = 0) v8 ==== 198+0+0 (crc 0 0 0) 0x55f65f0ccfc0 con 0x55f65f171000 2023-10-11T09:42:49.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.308+0000 7fe2ccbb4840 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x100 2023-10-11T09:42:49.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.308+0000 7fe2ccbb4840 1 -- 172.21.15.78:0/4066774506 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55f65f171000 msgr2=0x55f65e394100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.308+0000 7fe2ccbb4840 1 --2- 172.21.15.78:0/4066774506 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55f65f171000 0x55f65e394100 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.308+0000 7fe2ccbb4840 1 -- 172.21.15.78:0/4066774506 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55f65e72a400 msgr2=0x55f65e393b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.308+0000 7fe2ccbb4840 1 --2- 172.21.15.78:0/4066774506 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55f65e72a400 0x55f65e393b80 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.308+0000 7fe2ccbb4840 1 -- 172.21.15.78:0/4066774506 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55f65e72a000 msgr2=0x55f65e395700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.308+0000 7fe2ccbb4840 1 --2- 172.21.15.78:0/4066774506 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55f65e72a000 0x55f65e395700 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55f65f0546f0 tx=0x55f65e3c3260 comp rx=0 tx=0).stop 2023-10-11T09:42:49.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.308+0000 7fe2ccbb4840 1 -- 172.21.15.78:0/4066774506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f65f170800 msgr2=0x55f65e394c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.308+0000 7fe2cab6f700 1 -- 172.21.15.78:0/4066774506 reap_dead start 2023-10-11T09:42:49.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.308+0000 7fe2ccbb4840 1 --2- 172.21.15.78:0/4066774506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f65f170800 0x55f65e394c00 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x55f65e5e06f0 tx=0x55f65f0dad80 comp rx=0 tx=0).stop 2023-10-11T09:42:49.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.308+0000 7fe2ccbb4840 1 -- 172.21.15.78:0/4066774506 shutdown_connections 2023-10-11T09:42:49.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.308+0000 7fe2ccbb4840 1 --2- 172.21.15.78:0/4066774506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f65f170800 0x55f65e394c00 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.308+0000 7fe2ccbb4840 1 -- 172.21.15.78:0/4066774506 >> 172.21.15.78:0/4066774506 conn(0x55f65e2ecc00 msgr2=0x55f65e3bb800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:49.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.308+0000 7fe2ccbb4840 1 -- 172.21.15.78:0/4066774506 shutdown_connections 2023-10-11T09:42:49.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.308+0000 7fe2ccbb4840 1 -- 172.21.15.78:0/4066774506 wait complete. 2023-10-11T09:42:49.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.308+0000 7fe2ccbb4840 1 -- 172.21.15.78:0/1099509247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f65e2ed800 msgr2=0x55f65e392580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.308+0000 7fe2ccbb4840 1 --2- 172.21.15.78:0/1099509247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f65e2ed800 0x55f65e392580 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x55f65f055410 tx=0x55f65f0dae40 comp rx=0 tx=0).stop 2023-10-11T09:42:49.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.308+0000 7fe2ccbb4840 1 -- 172.21.15.78:0/1099509247 shutdown_connections 2023-10-11T09:42:49.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.308+0000 7fe2ccbb4840 1 --2- 172.21.15.78:0/1099509247 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f65e2ed000 0x55f65e393080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.308+0000 7fe2ccbb4840 1 --2- 172.21.15.78:0/1099509247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f65e2ed400 0x55f65e392b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.308+0000 7fe2ccbb4840 1 --2- 172.21.15.78:0/1099509247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f65e2ed800 0x55f65e392580 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.308+0000 7fe2ccbb4840 1 -- 172.21.15.78:0/1099509247 >> 172.21.15.78:0/1099509247 conn(0x55f65e2edc00 msgr2=0x55f65e3bb000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:49.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.308+0000 7fe2ccbb4840 1 -- 172.21.15.78:0/1099509247 shutdown_connections 2023-10-11T09:42:49.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.308+0000 7fe2ccbb4840 1 -- 172.21.15.78:0/1099509247 wait complete. 2023-10-11T09:42:49.320 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'init'] in time 0:00:00.103954, result: 2023-10-11T09:42:49.320 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents 2023-10-11T09:42:49.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.376+0000 7f13d4bc4840 1 -- start start 2023-10-11T09:42:49.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.376+0000 7f13d4bc4840 1 Processor -- start 2023-10-11T09:42:49.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.376+0000 7f13d4bc4840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5650f6e03800 0x5650f6ea8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.376+0000 7f13d4bc4840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5650f6e03400 0x5650f6ea8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.376+0000 7f13d4bc4840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650f6e03000 0x5650f6ea9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.376+0000 7f13d4bc4840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f6e82a00 con 0x5650f6e03000 2023-10-11T09:42:49.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.376+0000 7f13d4bc4840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f6e82780 con 0x5650f6e03800 2023-10-11T09:42:49.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.376+0000 7f13d4bc4840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x5650f6e82640 con 0x5650f6e03400 2023-10-11T09:42:49.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.376+0000 7f13d3b81700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650f6e03000 0x5650f6ea9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.376+0000 7f13d3b81700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650f6e03000 0x5650f6ea9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58072/0 (socket says 172.21.15.78:58072) 2023-10-11T09:42:49.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.376+0000 7f13d3b81700 1 -- 172.21.15.78:0/2040202056 learned_addr learned my addr 172.21.15.78:0/2040202056 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:49.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.376+0000 7f13d3380700 1 --2- 172.21.15.78:0/2040202056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5650f6e03800 0x5650f6ea8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.376+0000 7f13d2b7f700 1 --2- 172.21.15.78:0/2040202056 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5650f6e03400 0x5650f6ea8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.376+0000 7f13d3b81700 1 -- 172.21.15.78:0/2040202056 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5650f6e03400 msgr2=0x5650f6ea8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:49.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.376+0000 7f13d3b81700 1 --2- 172.21.15.78:0/2040202056 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5650f6e03400 0x5650f6ea8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.376+0000 7f13d3b81700 1 -- 172.21.15.78:0/2040202056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5650f6e03800 msgr2=0x5650f6ea8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:49.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.376+0000 7f13d3b81700 1 --2- 172.21.15.78:0/2040202056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5650f6e03800 0x5650f6ea8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.376+0000 7f13d3b81700 1 -- 172.21.15.78:0/2040202056 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650f6e24b60 con 0x5650f6e03000 2023-10-11T09:42:49.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.376+0000 7f13d2b7f700 1 --2- 172.21.15.78:0/2040202056 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5650f6e03400 0x5650f6ea8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:42:49.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.376+0000 7f13d3b81700 1 --2- 172.21.15.78:0/2040202056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650f6e03000 0x5650f6ea9080 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x5650f7b6b380 tx=0x5650f6ef8d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42fbde91bfb415dd server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.380+0000 7f13d0b7b700 1 -- 172.21.15.78:0/2040202056 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5650f6e33600 con 0x5650f6e03000 2023-10-11T09:42:49.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.380+0000 7f13d4bc4840 1 -- 172.21.15.78:0/2040202056 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5650f6e24340 con 0x5650f6e03000 2023-10-11T09:42:49.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.380+0000 7f13d0b7b700 1 -- 172.21.15.78:0/2040202056 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5650f6e4e480 con 0x5650f6e03000 2023-10-11T09:42:49.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.380+0000 7f13d4bc4840 1 -- 172.21.15.78:0/2040202056 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5650f6e24000 con 0x5650f6e03000 2023-10-11T09:42:49.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.380+0000 7f13d4bc4840 4 waiting for MDS map... 2023-10-11T09:42:49.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.380+0000 7f13d0b7b700 1 -- 172.21.15.78:0/2040202056 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5650f6e331e0 con 0x5650f6e03000 2023-10-11T09:42:49.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.380+0000 7f13d0b7b700 1 -- 172.21.15.78:0/2040202056 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(44..44 src has 1..44) v4 ==== 6667+0+0 (secure 0 0 0) 0x5650f6e3cc40 con 0x5650f6e03000 2023-10-11T09:42:49.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.380+0000 7f13d0b7b700 1 -- 172.21.15.78:0/2040202056 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) v1 ==== 3118+0+0 (secure 0 0 0) 0x5650f7bee300 con 0x5650f6e03000 2023-10-11T09:42:49.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.380+0000 7f13d4bc4840 4 Got MDS map 22 2023-10-11T09:42:49.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.380+0000 7f13d4bc4840 4 datascan.main: Using metadata pool output 2023-10-11T09:42:49.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.380+0000 7f13d4bc4840 4 datascan.main: connecting to RADOS... 2023-10-11T09:42:49.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.380+0000 7f13d4bc4840 1 Processor -- start 2023-10-11T09:42:49.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.380+0000 7f13d4bc4840 1 -- start start 2023-10-11T09:42:49.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.380+0000 7f13d4bc4840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650f70f2400 0x5650f6ea9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.380+0000 7f13d4bc4840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5650f70f2800 0x5650f6ea9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.380+0000 7f13d4bc4840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5650f70f2c00 0x5650f6eaa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.380+0000 7f13d4bc4840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f6e39e00 con 0x5650f70f2400 2023-10-11T09:42:49.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.380+0000 7f13d4bc4840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f6e82780 con 0x5650f70f2800 2023-10-11T09:42:49.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.380+0000 7f13d4bc4840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x5650f6e82500 con 0x5650f70f2c00 2023-10-11T09:42:49.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.380+0000 7f13d2b7f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650f70f2400 0x5650f6ea9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.384+0000 7f13d2b7f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650f70f2400 0x5650f6ea9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58074/0 (socket says 172.21.15.78:58074) 2023-10-11T09:42:49.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.384+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1192262914 learned_addr learned my addr 172.21.15.78:0/1192262914 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:49.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.384+0000 7f13d3380700 1 --2- 172.21.15.78:0/1192262914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5650f70f2800 0x5650f6ea9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.384+0000 7f13d2b7f700 1 --2- 172.21.15.78:0/1192262914 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5650f70f2c00 0x5650f6eaa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.384+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1192262914 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5650f70f2c00 msgr2=0x5650f6eaa100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:49.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.384+0000 7f13d2b7f700 1 --2- 172.21.15.78:0/1192262914 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5650f70f2c00 0x5650f6eaa100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.384+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1192262914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5650f70f2800 msgr2=0x5650f6ea9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:49.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.384+0000 7f13d2b7f700 1 --2- 172.21.15.78:0/1192262914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5650f70f2800 0x5650f6ea9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.384+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1192262914 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650f6e24ea0 con 0x5650f70f2400 2023-10-11T09:42:49.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.384+0000 7f13d2b7f700 1 --2- 172.21.15.78:0/1192262914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650f70f2400 0x5650f6ea9600 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x5650f70fc8d0 tx=0x5650f6ed9500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a5a23da9e687f86 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.384+0000 7f13ceb77700 1 -- 172.21.15.78:0/1192262914 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5650f6e33080 con 0x5650f70f2400 2023-10-11T09:42:49.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.384+0000 7f13ceb77700 1 -- 172.21.15.78:0/1192262914 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5650f7bec780 con 0x5650f70f2400 2023-10-11T09:42:49.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.384+0000 7f13ceb77700 1 -- 172.21.15.78:0/1192262914 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5650f6e33600 con 0x5650f70f2400 2023-10-11T09:42:49.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.384+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1192262914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650f70f2400 msgr2=0x5650f6ea9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.384+0000 7f13d4bc4840 1 --2- 172.21.15.78:0/1192262914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650f70f2400 0x5650f6ea9600 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x5650f70fc8d0 tx=0x5650f6ed9500 comp rx=0 tx=0).stop 2023-10-11T09:42:49.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.384+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1192262914 shutdown_connections 2023-10-11T09:42:49.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.384+0000 7f13d4bc4840 1 --2- 172.21.15.78:0/1192262914 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5650f70f2c00 0x5650f6eaa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.384+0000 7f13d4bc4840 1 --2- 172.21.15.78:0/1192262914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5650f70f2800 0x5650f6ea9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.384+0000 7f13d4bc4840 1 --2- 172.21.15.78:0/1192262914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650f70f2400 0x5650f6ea9600 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.384+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1192262914 >> 172.21.15.78:0/1192262914 conn(0x5650f6e02c00 msgr2=0x5650f6ed1800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:49.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.384+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1192262914 shutdown_connections 2023-10-11T09:42:49.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.384+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1192262914 wait complete. 2023-10-11T09:42:49.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.384+0000 7f13d4bc4840 1 Processor -- start 2023-10-11T09:42:49.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.384+0000 7f13d4bc4840 1 -- start start 2023-10-11T09:42:49.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.388+0000 7f13d4bc4840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5650f70f2400 0x5650f6eab180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.388+0000 7f13d4bc4840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5650f70f2800 0x5650f6eaac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.388+0000 7f13d4bc4840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650f70f2c00 0x5650f6eaa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.388+0000 7f13d4bc4840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f6e823c0 con 0x5650f70f2c00 2023-10-11T09:42:49.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.388+0000 7f13d4bc4840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f6e82280 con 0x5650f70f2800 2023-10-11T09:42:49.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.388+0000 7f13d4bc4840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x5650f6e82140 con 0x5650f70f2400 2023-10-11T09:42:49.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.388+0000 7f13d2b7f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650f70f2c00 0x5650f6eaa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.388+0000 7f13d2b7f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650f70f2c00 0x5650f6eaa680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58076/0 (socket says 172.21.15.78:58076) 2023-10-11T09:42:49.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.388+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 learned_addr learned my addr 172.21.15.78:0/1542120834 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:49.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.388+0000 7f13d2b7f700 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5650f70f2400 0x5650f6eab180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.388+0000 7f13d3380700 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5650f70f2800 0x5650f6eaac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.388+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5650f70f2400 msgr2=0x5650f6eab180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.388+0000 7f13d2b7f700 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5650f70f2400 0x5650f6eab180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.388+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5650f70f2800 msgr2=0x5650f6eaac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.388+0000 7f13d2b7f700 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5650f70f2800 0x5650f6eaac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.388+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650f6e251e0 con 0x5650f70f2c00 2023-10-11T09:42:49.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.388+0000 7f13d3380700 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5650f70f2800 0x5650f6eaac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:42:49.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.388+0000 7f13d2b7f700 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650f70f2c00 0x5650f6eaa680 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x5650f70fcb40 tx=0x5650f6ed9380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.388+0000 7f13cd374700 1 -- 172.21.15.78:0/1542120834 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5650f6e331e0 con 0x5650f70f2c00 2023-10-11T09:42:49.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.388+0000 7f13cd374700 1 -- 172.21.15.78:0/1542120834 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5650f7beca80 con 0x5650f70f2c00 2023-10-11T09:42:49.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.388+0000 7f13cd374700 1 -- 172.21.15.78:0/1542120834 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5650f6e32dc0 con 0x5650f70f2c00 2023-10-11T09:42:49.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.388+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5650f6e24ea0 con 0x5650f70f2c00 2023-10-11T09:42:49.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.388+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5650f6e24b60 con 0x5650f70f2c00 2023-10-11T09:42:49.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.388+0000 7f13d4bc4840 4 datascan.init: resolving metadata pool 5 2023-10-11T09:42:49.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.392+0000 7f13cd374700 1 -- 172.21.15.78:0/1542120834 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x5650f6e60840 con 0x5650f70f2c00 2023-10-11T09:42:49.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.392+0000 7f13cd374700 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5650f7248000 0x5650f6eab700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.392+0000 7f13cd374700 1 -- 172.21.15.78:0/1542120834 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6667+0+0 (secure 0 0 0) 0x5650f6e3ca80 con 0x5650f70f2c00 2023-10-11T09:42:49.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.392+0000 7f13d4bc4840 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-10-11T09:42:49.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.392+0000 7f13d4bc4840 4 datascan.main: data pool 'cephfs_data' has ID 6 2023-10-11T09:42:49.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.392+0000 7f13d4bc4840 4 datascan.main: opening data pool 'cephfs_data' 2023-10-11T09:42:49.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.392+0000 7f13d4bc4840 4 datascan.main: data pool 'cephfs_data_extra' has ID 7 2023-10-11T09:42:49.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.392+0000 7f13d4bc4840 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-10-11T09:42:49.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.392+0000 7f13d4bc4840 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x5650f70f3000 0x5650f6eaa100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.392+0000 7f13d3380700 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5650f7248000 0x5650f6eab700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.392+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:1 6.0 6:00000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f70f3400 con 0x5650f70f3000 2023-10-11T09:42:49.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.392+0000 7f13d2b7f700 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x5650f70f3000 0x5650f6eaa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.392+0000 7f13d3380700 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5650f7248000 0x5650f6eab700 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x5650f7b6a8a0 tx=0x5650f7107c80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.392+0000 7f13d2b7f700 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x5650f70f3000 0x5650f6eaa100 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.392+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.0 v2:172.21.15.78:6800/35735 1 ==== osd_op_reply(1 [pgnls start_epoch 44 out=49b] v42'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f70f3000 2023-10-11T09:42:49.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.392+0000 7f13ce376700 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x5650f7248400 0x5650f7266000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.396+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:2 6.20 6:04000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7248800 con 0x5650f7248400 2023-10-11T09:42:49.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.396+0000 7f13d3b81700 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x5650f7248400 0x5650f7266000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.396+0000 7f13d3b81700 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x5650f7248400 0x5650f7266000 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.396+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.1 v2:172.21.15.78:6809/35734 1 ==== osd_op_reply(2 [pgnls start_epoch 44 out=49b] v42'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f7248400 2023-10-11T09:42:49.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.396+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:3 6.10 6:08000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7249000 con 0x5650f70f3000 2023-10-11T09:42:49.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.396+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.0 v2:172.21.15.78:6800/35735 2 ==== osd_op_reply(3 [pgnls start_epoch 44 out=49b] v42'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f70f3000 2023-10-11T09:42:49.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.396+0000 7f13ce376700 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x5650f7248c00 0x5650f7266580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.396+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:4 6.30 6:0c000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7249400 con 0x5650f7248c00 2023-10-11T09:42:49.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.396+0000 7f13d3380700 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x5650f7248c00 0x5650f7266580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.400+0000 7f13d3380700 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x5650f7248c00 0x5650f7266580 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.400+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.7 v2:172.21.15.133:6824/35901 1 ==== osd_op_reply(4 [pgnls start_epoch 44 out=49b] v42'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7248c00 2023-10-11T09:42:49.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.400+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:5 6.8 6:10000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f727a000 con 0x5650f7248c00 2023-10-11T09:42:49.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.400+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.7 v2:172.21.15.133:6824/35901 2 ==== osd_op_reply(5 [pgnls start_epoch 44 out=49b] v42'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7248c00 2023-10-11T09:42:49.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.400+0000 7f13ce376700 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x5650f7249800 0x5650f7266b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.400+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:6 6.28 6:14000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7249c00 con 0x5650f7249800 2023-10-11T09:42:49.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.400+0000 7f13d2b7f700 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x5650f7249800 0x5650f7266b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.400+0000 7f13d2b7f700 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x5650f7249800 0x5650f7266b00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.404+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.2 v2:172.21.15.78:6808/35732 1 ==== osd_op_reply(6 [pgnls start_epoch 44 out=49b] v42'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f7249800 2023-10-11T09:42:49.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.404+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:7 6.18 6:18000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f727a400 con 0x5650f70f3000 2023-10-11T09:42:49.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.404+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.0 v2:172.21.15.78:6800/35735 3 ==== osd_op_reply(7 [pgnls start_epoch 44 out=49b] v42'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f70f3000 2023-10-11T09:42:49.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.404+0000 7f13ce376700 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5650f7249000 0x5650f7267080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.404+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:8 6.38 6:1c000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7249c00 con 0x5650f7249000 2023-10-11T09:42:49.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.404+0000 7f13d3b81700 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5650f7249000 0x5650f7267080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.404+0000 7f13d3b81700 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5650f7249000 0x5650f7267080 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.404+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(8 [pgnls start_epoch 44 out=49b] v42'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f7249000 2023-10-11T09:42:49.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.404+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:9 6.4 6:20000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f727a000 con 0x5650f7248400 2023-10-11T09:42:49.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.404+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.1 v2:172.21.15.78:6809/35734 2 ==== osd_op_reply(9 [pgnls start_epoch 44 out=49b] v42'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f7248400 2023-10-11T09:42:49.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.404+0000 7f13ce376700 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x5650f7249400 0x5650f7267600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.404+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:10 6.24 6:24000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f70f3400 con 0x5650f7249400 2023-10-11T09:42:49.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.408+0000 7f13d3380700 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x5650f7249400 0x5650f7267600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.408+0000 7f13d3380700 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x5650f7249400 0x5650f7267600 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.408+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.5 v2:172.21.15.133:6816/35899 1 ==== osd_op_reply(10 [pgnls start_epoch 44 out=49b] v42'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7249400 2023-10-11T09:42:49.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.408+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:11 6.14 6:28000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f727a800 con 0x5650f7249800 2023-10-11T09:42:49.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.408+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.2 v2:172.21.15.78:6808/35732 2 ==== osd_op_reply(11 [pgnls start_epoch 44 out=49b] v42'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f7249800 2023-10-11T09:42:49.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.408+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:12 6.34 6:2c000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f727a000 con 0x5650f70f3000 2023-10-11T09:42:49.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.408+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.0 v2:172.21.15.78:6800/35735 4 ==== osd_op_reply(12 [pgnls start_epoch 44 out=49b] v42'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f70f3000 2023-10-11T09:42:49.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.408+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:13 6.c 6:30000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f727ac00 con 0x5650f7249000 2023-10-11T09:42:49.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.412+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.3 v2:172.21.15.78:6824/35738 2 ==== osd_op_reply(13 [pgnls start_epoch 44 out=49b] v42'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f7249000 2023-10-11T09:42:49.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.412+0000 7f13ce376700 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x5650f727b000 0x5650f7267b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.412+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:14 6.2c 6:34000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f727b400 con 0x5650f727b000 2023-10-11T09:42:49.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.412+0000 7f13d2b7f700 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x5650f727b000 0x5650f7267b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.412+0000 7f13d2b7f700 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x5650f727b000 0x5650f7267b80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.412+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== osd_op_reply(14 [pgnls start_epoch 44 out=49b] v42'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f727b000 2023-10-11T09:42:49.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.412+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:15 6.1c 6:38000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f727a400 con 0x5650f7248c00 2023-10-11T09:42:49.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.412+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.7 v2:172.21.15.133:6824/35901 3 ==== osd_op_reply(15 [pgnls start_epoch 44 out=49b] v42'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7248c00 2023-10-11T09:42:49.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.416+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:16 6.3c 6:3c000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7248800 con 0x5650f7249800 2023-10-11T09:42:49.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.416+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.2 v2:172.21.15.78:6808/35732 3 ==== osd_op_reply(16 [pgnls start_epoch 44 out=49b] v42'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f7249800 2023-10-11T09:42:49.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.416+0000 7f13ceb77700 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x5650f727a800 0x5650f7268100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.416+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:17 6.2 6:40000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f727a000 con 0x5650f727a800 2023-10-11T09:42:49.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.416+0000 7f13d3b81700 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x5650f727a800 0x5650f7268100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.416+0000 7f13d3b81700 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x5650f727a800 0x5650f7268100 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.416+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.4 v2:172.21.15.133:6808/35898 1 ==== osd_op_reply(17 [pgnls start_epoch 44 out=49b] v42'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f727a800 2023-10-11T09:42:49.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.416+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:18 6.22 6:44000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7249c00 con 0x5650f727b000 2023-10-11T09:42:49.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.420+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.6 v2:172.21.15.133:6800/35900 2 ==== osd_op_reply(18 [pgnls start_epoch 44 out=49b] v42'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f727b000 2023-10-11T09:42:49.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.420+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:19 6.12 6:48000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f727b400 con 0x5650f7248c00 2023-10-11T09:42:49.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.420+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.7 v2:172.21.15.133:6824/35901 4 ==== osd_op_reply(19 [pgnls start_epoch 44 out=49b] v42'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7248c00 2023-10-11T09:42:49.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.420+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:20 6.32 6:4c000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f727a400 con 0x5650f727a800 2023-10-11T09:42:49.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.420+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.4 v2:172.21.15.133:6808/35898 2 ==== osd_op_reply(20 [pgnls start_epoch 44 out=49b] v42'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f727a800 2023-10-11T09:42:49.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.420+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:21 6.a 6:50000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7290000 con 0x5650f7249400 2023-10-11T09:42:49.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.420+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.5 v2:172.21.15.133:6816/35899 2 ==== osd_op_reply(21 [pgnls start_epoch 44 out=49b] v42'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7249400 2023-10-11T09:42:49.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.420+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:22 6.2a 6:54000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f727b400 con 0x5650f727a800 2023-10-11T09:42:49.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.424+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.4 v2:172.21.15.133:6808/35898 3 ==== osd_op_reply(22 [pgnls start_epoch 44 out=81b] v42'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f727a800 2023-10-11T09:42:49.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.424+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:23 6.2a 6:57fe7535:::100000001f6.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f70f3800 con 0x5650f727a800 2023-10-11T09:42:49.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.424+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.4 v2:172.21.15.133:6808/35898 4 ==== osd_op_reply(23 100000001f6.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f727a800 2023-10-11T09:42:49.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.424+0000 7f13d4bc4840 10 datascan.operator(): handling object 1099511628278.0 2023-10-11T09:42:49.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.424+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:24 6.2a 6:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7291400 con 0x5650f727a800 2023-10-11T09:42:49.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.428+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.4 v2:172.21.15.133:6808/35898 5 ==== osd_op_reply(24 100000001f6.00000000 [call] v44'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f727a800 2023-10-11T09:42:49.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.428+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:25 6.1a 6:58000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7291800 con 0x5650f727a800 2023-10-11T09:42:49.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.428+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.4 v2:172.21.15.133:6808/35898 6 ==== osd_op_reply(25 [pgnls start_epoch 44 out=49b] v42'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f727a800 2023-10-11T09:42:49.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.428+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:26 6.3a 6:5c000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f727b800 con 0x5650f7248400 2023-10-11T09:42:49.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.428+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.1 v2:172.21.15.78:6809/35734 3 ==== osd_op_reply(26 [pgnls start_epoch 44 out=49b] v42'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f7248400 2023-10-11T09:42:49.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.428+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:27 6.6 6:60000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7290400 con 0x5650f7249000 2023-10-11T09:42:49.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.432+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.3 v2:172.21.15.78:6824/35738 3 ==== osd_op_reply(27 [pgnls start_epoch 44 out=49b] v42'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f7249000 2023-10-11T09:42:49.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.432+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:28 6.26 6:64000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f727bc00 con 0x5650f7248400 2023-10-11T09:42:49.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.432+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.1 v2:172.21.15.78:6809/35734 4 ==== osd_op_reply(28 [pgnls start_epoch 44 out=49b] v42'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f7248400 2023-10-11T09:42:49.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.432+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:29 6.16 6:68000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7290800 con 0x5650f70f3000 2023-10-11T09:42:49.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.432+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.0 v2:172.21.15.78:6800/35735 5 ==== osd_op_reply(29 [pgnls start_epoch 44 out=49b] v42'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f70f3000 2023-10-11T09:42:49.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.432+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:30 6.36 6:6c000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f70f3400 con 0x5650f7249400 2023-10-11T09:42:49.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.432+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.5 v2:172.21.15.133:6816/35899 3 ==== osd_op_reply(30 [pgnls start_epoch 44 out=101b] v42'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7249400 2023-10-11T09:42:49.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.432+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:31 6.36 6:6ee5e2c4:::100000001f9.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7291c00 con 0x5650f7249400 2023-10-11T09:42:49.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.436+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.5 v2:172.21.15.133:6816/35899 4 ==== osd_op_reply(31 100000001f9.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7249400 2023-10-11T09:42:49.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.436+0000 7f13d4bc4840 10 datascan.operator(): handling object 1099511628281.0 2023-10-11T09:42:49.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.436+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:32 6.36 6:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f727ac00 con 0x5650f7249400 2023-10-11T09:42:49.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.436+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.5 v2:172.21.15.133:6816/35899 5 ==== osd_op_reply(32 100000001f9.00000000 [call] v44'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7249400 2023-10-11T09:42:49.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.436+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:33 6.36 6:6fe38dc2::::head [pgnls start_epoch 44 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f727a000 con 0x5650f7249400 2023-10-11T09:42:49.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.440+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.5 v2:172.21.15.133:6816/35899 6 ==== osd_op_reply(33 [pgnls start_epoch 44 out=81b] v44'18 uv18 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7249400 2023-10-11T09:42:49.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.440+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:34 6.36 6:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f727a400 con 0x5650f7249400 2023-10-11T09:42:49.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.440+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.5 v2:172.21.15.133:6816/35899 7 ==== osd_op_reply(34 100000001f7.00000000 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7249400 2023-10-11T09:42:49.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.440+0000 7f13d4bc4840 10 datascan.operator(): handling object 1099511628279.0 2023-10-11T09:42:49.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.440+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:35 6.36 6:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f727b400 con 0x5650f7249400 2023-10-11T09:42:49.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.444+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.5 v2:172.21.15.133:6816/35899 8 ==== osd_op_reply(35 100000001f7.00000000 [call] v44'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7249400 2023-10-11T09:42:49.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.444+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:36 6.e 6:70000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7290000 con 0x5650f727a800 2023-10-11T09:42:49.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.444+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.4 v2:172.21.15.133:6808/35898 7 ==== osd_op_reply(36 [pgnls start_epoch 44 out=49b] v42'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f727a800 2023-10-11T09:42:49.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.444+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:37 6.2e 6:74000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7290c00 con 0x5650f727b000 2023-10-11T09:42:49.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.444+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.6 v2:172.21.15.133:6800/35900 3 ==== osd_op_reply(37 [pgnls start_epoch 44 out=49b] v42'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f727b000 2023-10-11T09:42:49.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.444+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:38 6.1e 6:78000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7290400 con 0x5650f727a800 2023-10-11T09:42:49.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.444+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.4 v2:172.21.15.133:6808/35898 8 ==== osd_op_reply(38 [pgnls start_epoch 44 out=49b] v42'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f727a800 2023-10-11T09:42:49.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.444+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:39 6.3e 6:7c000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7291000 con 0x5650f7249000 2023-10-11T09:42:49.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.448+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.3 v2:172.21.15.78:6824/35738 4 ==== osd_op_reply(39 [pgnls start_epoch 44 out=49b] v42'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f7249000 2023-10-11T09:42:49.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.448+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:40 6.1 6:80000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72a0000 con 0x5650f7248400 2023-10-11T09:42:49.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.448+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.1 v2:172.21.15.78:6809/35734 5 ==== osd_op_reply(40 [pgnls start_epoch 44 out=49b] v42'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f7248400 2023-10-11T09:42:49.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.448+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:41 6.21 6:84000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72a2000 con 0x5650f7248c00 2023-10-11T09:42:49.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.448+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.7 v2:172.21.15.133:6824/35901 5 ==== osd_op_reply(41 [pgnls start_epoch 44 out=49b] v42'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7248c00 2023-10-11T09:42:49.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.448+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:42 6.11 6:88000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72a0400 con 0x5650f7249000 2023-10-11T09:42:49.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.452+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.3 v2:172.21.15.78:6824/35738 5 ==== osd_op_reply(42 [pgnls start_epoch 44 out=49b] v42'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f7249000 2023-10-11T09:42:49.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.452+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:43 6.31 6:8c000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72a2400 con 0x5650f7249400 2023-10-11T09:42:49.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.452+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.5 v2:172.21.15.133:6816/35899 9 ==== osd_op_reply(43 [pgnls start_epoch 44 out=49b] v42'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7249400 2023-10-11T09:42:49.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.452+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:44 6.9 6:90000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72a0800 con 0x5650f70f3000 2023-10-11T09:42:49.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.452+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.0 v2:172.21.15.78:6800/35735 6 ==== osd_op_reply(44 [pgnls start_epoch 44 out=49b] v42'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f70f3000 2023-10-11T09:42:49.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.452+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:45 6.29 6:94000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72a2800 con 0x5650f7248c00 2023-10-11T09:42:49.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.452+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.7 v2:172.21.15.133:6824/35901 6 ==== osd_op_reply(45 [pgnls start_epoch 44 out=49b] v42'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7248c00 2023-10-11T09:42:49.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.452+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:46 6.19 6:98000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72a0c00 con 0x5650f7249400 2023-10-11T09:42:49.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.456+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.5 v2:172.21.15.133:6816/35899 10 ==== osd_op_reply(46 [pgnls start_epoch 44 out=81b] v42'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7249400 2023-10-11T09:42:49.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.456+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:47 6.19 6:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7291c00 con 0x5650f7249400 2023-10-11T09:42:49.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.456+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.5 v2:172.21.15.133:6816/35899 11 ==== osd_op_reply(47 100000001f8.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7249400 2023-10-11T09:42:49.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.456+0000 7f13d4bc4840 10 datascan.operator(): handling object 1099511628280.0 2023-10-11T09:42:49.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.456+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:48 6.19 6:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7291400 con 0x5650f7249400 2023-10-11T09:42:49.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.460+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.5 v2:172.21.15.133:6816/35899 12 ==== osd_op_reply(48 100000001f8.00000000 [call] v44'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7249400 2023-10-11T09:42:49.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.460+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:49 6.39 6:9c000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7291800 con 0x5650f7249400 2023-10-11T09:42:49.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.460+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.5 v2:172.21.15.133:6816/35899 13 ==== osd_op_reply(49 [pgnls start_epoch 44 out=49b] v42'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7249400 2023-10-11T09:42:49.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.460+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:50 6.5 6:a0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72a1000 con 0x5650f7248c00 2023-10-11T09:42:49.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.460+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.7 v2:172.21.15.133:6824/35901 7 ==== osd_op_reply(50 [pgnls start_epoch 44 out=49b] v42'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7248c00 2023-10-11T09:42:49.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.460+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:51 6.25 6:a4000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72a2c00 con 0x5650f727b000 2023-10-11T09:42:49.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.464+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.6 v2:172.21.15.133:6800/35900 4 ==== osd_op_reply(51 [pgnls start_epoch 44 out=49b] v42'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f727b000 2023-10-11T09:42:49.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.464+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:52 6.15 6:a8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72a1400 con 0x5650f7248c00 2023-10-11T09:42:49.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.464+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.7 v2:172.21.15.133:6824/35901 8 ==== osd_op_reply(52 [pgnls start_epoch 44 out=49b] v42'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7248c00 2023-10-11T09:42:49.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.464+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:53 6.35 6:ac000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72a3000 con 0x5650f7249400 2023-10-11T09:42:49.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.464+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.5 v2:172.21.15.133:6816/35899 14 ==== osd_op_reply(53 [pgnls start_epoch 44 out=49b] v42'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7249400 2023-10-11T09:42:49.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.464+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:54 6.d 6:b0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72a1800 con 0x5650f7249400 2023-10-11T09:42:49.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.464+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.5 v2:172.21.15.133:6816/35899 15 ==== osd_op_reply(54 [pgnls start_epoch 44 out=49b] v42'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7249400 2023-10-11T09:42:49.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.468+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:55 6.2d 6:b4000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72a3400 con 0x5650f7249800 2023-10-11T09:42:49.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.468+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.2 v2:172.21.15.78:6808/35732 4 ==== osd_op_reply(55 [pgnls start_epoch 44 out=49b] v42'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f7249800 2023-10-11T09:42:49.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.468+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:56 6.1d 6:b8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72a1c00 con 0x5650f7248400 2023-10-11T09:42:49.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.468+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.1 v2:172.21.15.78:6809/35734 6 ==== osd_op_reply(56 [pgnls start_epoch 44 out=49b] v42'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f7248400 2023-10-11T09:42:49.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.468+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:57 6.3d 6:bc000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72a0c00 con 0x5650f70f3000 2023-10-11T09:42:49.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.468+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.0 v2:172.21.15.78:6800/35735 7 ==== osd_op_reply(57 [pgnls start_epoch 44 out=49b] v42'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f70f3000 2023-10-11T09:42:49.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.468+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:58 6.3 6:c0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72a2800 con 0x5650f7248c00 2023-10-11T09:42:49.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.468+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.7 v2:172.21.15.133:6824/35901 9 ==== osd_op_reply(58 [pgnls start_epoch 44 out=49b] v42'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7248c00 2023-10-11T09:42:49.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.468+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:59 6.23 6:c4000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f727a000 con 0x5650f727b000 2023-10-11T09:42:49.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.472+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.6 v2:172.21.15.133:6800/35900 5 ==== osd_op_reply(59 [pgnls start_epoch 44 out=49b] v42'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f727b000 2023-10-11T09:42:49.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.472+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:60 6.13 6:c8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72a2000 con 0x5650f7249000 2023-10-11T09:42:49.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.472+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.3 v2:172.21.15.78:6824/35738 6 ==== osd_op_reply(60 [pgnls start_epoch 44 out=49b] v42'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f7249000 2023-10-11T09:42:49.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.472+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:61 6.33 6:cc000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f727a400 con 0x5650f7249400 2023-10-11T09:42:49.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.472+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.5 v2:172.21.15.133:6816/35899 16 ==== osd_op_reply(61 [pgnls start_epoch 44 out=49b] v42'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7249400 2023-10-11T09:42:49.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.472+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:62 6.b 6:d0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72a2400 con 0x5650f7249000 2023-10-11T09:42:49.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.472+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.3 v2:172.21.15.78:6824/35738 7 ==== osd_op_reply(62 [pgnls start_epoch 44 out=49b] v42'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f7249000 2023-10-11T09:42:49.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.472+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:63 6.2b 6:d4000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f727b400 con 0x5650f7249400 2023-10-11T09:42:49.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.476+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.5 v2:172.21.15.133:6816/35899 17 ==== osd_op_reply(63 [pgnls start_epoch 44 out=49b] v42'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7249400 2023-10-11T09:42:49.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.476+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:64 6.1b 6:d8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72a3800 con 0x5650f7249000 2023-10-11T09:42:49.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.476+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.3 v2:172.21.15.78:6824/35738 8 ==== osd_op_reply(64 [pgnls start_epoch 44 out=49b] v42'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f7249000 2023-10-11T09:42:49.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.476+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:65 6.3b 6:dc000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f727bc00 con 0x5650f7249800 2023-10-11T09:42:49.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.476+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.2 v2:172.21.15.78:6808/35732 5 ==== osd_op_reply(65 [pgnls start_epoch 44 out=49b] v42'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f7249800 2023-10-11T09:42:49.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.476+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:66 6.7 6:e0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72a3c00 con 0x5650f7249400 2023-10-11T09:42:49.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.480+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.5 v2:172.21.15.133:6816/35899 18 ==== osd_op_reply(66 [pgnls start_epoch 44 out=81b] v42'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7249400 2023-10-11T09:42:49.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.480+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:67 6.7 6:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f727ac00 con 0x5650f7249400 2023-10-11T09:42:49.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.480+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.5 v2:172.21.15.133:6816/35899 19 ==== osd_op_reply(67 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7249400 2023-10-11T09:42:49.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.480+0000 7f13d4bc4840 10 datascan.operator(): handling object 1099511627777.0 2023-10-11T09:42:49.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.480+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:68 6.7 6:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f727b800 con 0x5650f7249400 2023-10-11T09:42:49.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.484+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.5 v2:172.21.15.133:6816/35899 20 ==== osd_op_reply(68 10000000001.00000000 [call] v44'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7249400 2023-10-11T09:42:49.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.484+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:69 6.27 6:e4000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f70f3400 con 0x5650f727b000 2023-10-11T09:42:49.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.484+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.6 v2:172.21.15.133:6800/35900 6 ==== osd_op_reply(69 [pgnls start_epoch 44 out=49b] v42'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f727b000 2023-10-11T09:42:49.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.484+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:70 6.17 6:e8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72a1000 con 0x5650f727a800 2023-10-11T09:42:49.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.484+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.4 v2:172.21.15.133:6808/35898 9 ==== osd_op_reply(70 [pgnls start_epoch 44 out=49b] v42'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f727a800 2023-10-11T09:42:49.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.484+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:71 6.37 6:ec000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7248800 con 0x5650f70f3000 2023-10-11T09:42:49.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.484+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.0 v2:172.21.15.78:6800/35735 8 ==== osd_op_reply(71 [pgnls start_epoch 44 out=49b] v42'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f70f3000 2023-10-11T09:42:49.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.484+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:72 6.f 6:f0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72a0800 con 0x5650f7249800 2023-10-11T09:42:49.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.488+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.2 v2:172.21.15.78:6808/35732 6 ==== osd_op_reply(72 [pgnls start_epoch 44 out=49b] v42'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f7249800 2023-10-11T09:42:49.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.488+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:73 6.2f 6:f4000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7249c00 con 0x5650f7248c00 2023-10-11T09:42:49.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.488+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.7 v2:172.21.15.133:6824/35901 10 ==== osd_op_reply(73 [pgnls start_epoch 44 out=49b] v42'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7248c00 2023-10-11T09:42:49.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.488+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:74 6.1f 6:f8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72a0000 con 0x5650f7249000 2023-10-11T09:42:49.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.488+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.3 v2:172.21.15.78:6824/35738 9 ==== osd_op_reply(74 [pgnls start_epoch 44 out=49b] v42'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f7249000 2023-10-11T09:42:49.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.488+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:75 6.3f 6:fc000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f727a000 con 0x5650f7249000 2023-10-11T09:42:49.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.488+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.3 v2:172.21.15.78:6824/35738 10 ==== osd_op_reply(75 [pgnls start_epoch 44 out=49b] v42'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f7249000 2023-10-11T09:42:49.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.488+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:76 7.0 7:00000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f70f3800 con 0x5650f7249400 2023-10-11T09:42:49.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.492+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.5 v2:172.21.15.133:6816/35899 21 ==== osd_op_reply(76 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7249400 2023-10-11T09:42:49.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.492+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:77 7.10 7:08000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f727bc00 con 0x5650f7249000 2023-10-11T09:42:49.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.492+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.3 v2:172.21.15.78:6824/35738 11 ==== osd_op_reply(77 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f7249000 2023-10-11T09:42:49.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.492+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:78 7.8 7:10000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72a0400 con 0x5650f7249800 2023-10-11T09:42:49.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.492+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.2 v2:172.21.15.78:6808/35732 7 ==== osd_op_reply(78 [pgnls start_epoch 44 out=81b] v42'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f7249800 2023-10-11T09:42:49.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.492+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:79 7.8 7:1247e94b:::100000001f9.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f727a000 con 0x5650f7249800 2023-10-11T09:42:49.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.496+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.2 v2:172.21.15.78:6808/35732 8 ==== osd_op_reply(79 100000001f9.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f7249800 2023-10-11T09:42:49.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.496+0000 7f13d4bc4840 10 datascan.operator(): handling object 1099511628281.1 2023-10-11T09:42:49.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.496+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:80 6.36 6:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72a1c00 con 0x5650f7249400 2023-10-11T09:42:49.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.496+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.5 v2:172.21.15.133:6816/35899 22 ==== osd_op_reply(80 100000001f9.00000000 [call,setxattr (8)] v44'20 uv20 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7249400 2023-10-11T09:42:49.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.496+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:81 7.18 7:18000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72a1000 con 0x5650f70f3000 2023-10-11T09:42:49.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.500+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.0 v2:172.21.15.78:6800/35735 9 ==== osd_op_reply(81 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f70f3000 2023-10-11T09:42:49.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.500+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:82 7.4 7:20000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7290000 con 0x5650f70f3000 2023-10-11T09:42:49.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.500+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.0 v2:172.21.15.78:6800/35735 10 ==== osd_op_reply(82 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f70f3000 2023-10-11T09:42:49.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.500+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:83 7.14 7:28000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f727a400 con 0x5650f7249000 2023-10-11T09:42:49.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.500+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.3 v2:172.21.15.78:6824/35738 12 ==== osd_op_reply(83 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f7249000 2023-10-11T09:42:49.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.500+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:84 7.c 7:30000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7290400 con 0x5650f727b000 2023-10-11T09:42:49.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.500+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.6 v2:172.21.15.133:6800/35900 7 ==== osd_op_reply(84 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f727b000 2023-10-11T09:42:49.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.500+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:85 7.1c 7:38000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f727b400 con 0x5650f7249400 2023-10-11T09:42:49.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.504+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.5 v2:172.21.15.133:6816/35899 23 ==== osd_op_reply(85 [pgnls start_epoch 44 out=81b] v40'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7249400 2023-10-11T09:42:49.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.504+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:86 7.1c 7:3d6db04b:::100000001f6.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72a0000 con 0x5650f7249400 2023-10-11T09:42:49.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.504+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.5 v2:172.21.15.133:6816/35899 24 ==== osd_op_reply(86 100000001f6.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7249400 2023-10-11T09:42:49.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.504+0000 7f13d4bc4840 10 datascan.operator(): handling object 1099511628278.1 2023-10-11T09:42:49.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.504+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:87 6.2a 6:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7290000 con 0x5650f727a800 2023-10-11T09:42:49.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.508+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.4 v2:172.21.15.133:6808/35898 10 ==== osd_op_reply(87 100000001f6.00000000 [call,setxattr (8)] v44'12 uv12 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f727a800 2023-10-11T09:42:49.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.508+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:88 7.2 7:40000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7290400 con 0x5650f7249000 2023-10-11T09:42:49.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.508+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.3 v2:172.21.15.78:6824/35738 13 ==== osd_op_reply(88 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f7249000 2023-10-11T09:42:49.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.508+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:89 7.12 7:48000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72a0c00 con 0x5650f727b000 2023-10-11T09:42:49.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.508+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.6 v2:172.21.15.133:6800/35900 8 ==== osd_op_reply(89 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f727b000 2023-10-11T09:42:49.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.508+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:90 7.a 7:50000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7290800 con 0x5650f7249000 2023-10-11T09:42:49.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.512+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.3 v2:172.21.15.78:6824/35738 14 ==== osd_op_reply(90 [pgnls start_epoch 44 out=81b] v42'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f7249000 2023-10-11T09:42:49.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.512+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:91 7.a 7:57fe7535:::100000001f6.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72a0800 con 0x5650f7249000 2023-10-11T09:42:49.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.512+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.3 v2:172.21.15.78:6824/35738 15 ==== osd_op_reply(91 100000001f6.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f7249000 2023-10-11T09:42:49.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.512+0000 7f13d4bc4840 10 datascan.operator(): handling object 1099511628278.0 2023-10-11T09:42:49.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.512+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:92 6.2a 6:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72a0400 con 0x5650f727a800 2023-10-11T09:42:49.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.512+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.4 v2:172.21.15.133:6808/35898 11 ==== osd_op_reply(92 100000001f6.00000000 [call,setxattr (8)] v44'13 uv13 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f727a800 2023-10-11T09:42:49.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.512+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:93 7.1a 7:58000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72a1000 con 0x5650f727a800 2023-10-11T09:42:49.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.516+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.4 v2:172.21.15.133:6808/35898 12 ==== osd_op_reply(93 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f727a800 2023-10-11T09:42:49.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.516+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:94 7.6 7:60000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7290c00 con 0x5650f70f3000 2023-10-11T09:42:49.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.516+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.0 v2:172.21.15.78:6800/35735 11 ==== osd_op_reply(94 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f70f3000 2023-10-11T09:42:49.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.516+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:95 7.16 7:68000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72a1400 con 0x5650f727a800 2023-10-11T09:42:49.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.516+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.4 v2:172.21.15.133:6808/35898 13 ==== osd_op_reply(95 [pgnls start_epoch 44 out=101b] v42'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f727a800 2023-10-11T09:42:49.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.516+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:96 7.16 7:6ee5e2c4:::100000001f9.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f727a000 con 0x5650f727a800 2023-10-11T09:42:49.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.516+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.4 v2:172.21.15.133:6808/35898 14 ==== osd_op_reply(96 100000001f9.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f727a800 2023-10-11T09:42:49.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.516+0000 7f13d4bc4840 10 datascan.operator(): handling object 1099511628281.0 2023-10-11T09:42:49.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.516+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:97 6.36 6:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7248800 con 0x5650f7249400 2023-10-11T09:42:49.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.520+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.5 v2:172.21.15.133:6816/35899 25 ==== osd_op_reply(97 100000001f9.00000000 [call,setxattr (8)] v44'21 uv21 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7249400 2023-10-11T09:42:49.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.520+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:98 7.16 7:6fe38dc2::::head [pgnls start_epoch 44 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7290000 con 0x5650f727a800 2023-10-11T09:42:49.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.520+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.4 v2:172.21.15.133:6808/35898 15 ==== osd_op_reply(98 [pgnls start_epoch 44 out=81b] v42'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f727a800 2023-10-11T09:42:49.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.520+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:99 7.16 7:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7290400 con 0x5650f727a800 2023-10-11T09:42:49.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.520+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.4 v2:172.21.15.133:6808/35898 16 ==== osd_op_reply(99 100000001f7.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f727a800 2023-10-11T09:42:49.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.520+0000 7f13d4bc4840 10 datascan.operator(): handling object 1099511628279.0 2023-10-11T09:42:49.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.520+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:100 6.36 6:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7290800 con 0x5650f7249400 2023-10-11T09:42:49.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.524+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.5 v2:172.21.15.133:6816/35899 26 ==== osd_op_reply(100 100000001f7.00000000 [call,setxattr (8)] v44'22 uv22 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7249400 2023-10-11T09:42:49.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.524+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:101 7.e 7:70000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f727ac00 con 0x5650f7249800 2023-10-11T09:42:49.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.524+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.2 v2:172.21.15.78:6808/35732 9 ==== osd_op_reply(101 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f7249800 2023-10-11T09:42:49.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.524+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:102 7.1e 7:78000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7291000 con 0x5650f7248400 2023-10-11T09:42:49.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.524+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.1 v2:172.21.15.78:6809/35734 7 ==== osd_op_reply(102 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f7248400 2023-10-11T09:42:49.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.528+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:103 7.1 7:80000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72a1800 con 0x5650f727a800 2023-10-11T09:42:49.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.528+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.4 v2:172.21.15.133:6808/35898 17 ==== osd_op_reply(103 [pgnls start_epoch 44 out=81b] v42'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f727a800 2023-10-11T09:42:49.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.528+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:104 7.1 7:848ebb0b:::100000001f7.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f727b800 con 0x5650f727a800 2023-10-11T09:42:49.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.528+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.4 v2:172.21.15.133:6808/35898 18 ==== osd_op_reply(104 100000001f7.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f727a800 2023-10-11T09:42:49.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.528+0000 7f13d4bc4840 10 datascan.operator(): handling object 1099511628279.1 2023-10-11T09:42:49.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.528+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:105 6.36 6:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f727b400 con 0x5650f7249400 2023-10-11T09:42:49.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.532+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.5 v2:172.21.15.133:6816/35899 27 ==== osd_op_reply(105 100000001f7.00000000 [call,setxattr (8)] v44'23 uv23 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7249400 2023-10-11T09:42:49.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.532+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:106 7.11 7:88000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f727bc00 con 0x5650f7248c00 2023-10-11T09:42:49.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.532+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.7 v2:172.21.15.133:6824/35901 11 ==== osd_op_reply(106 [pgnls start_epoch 44 out=81b] v42'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7248c00 2023-10-11T09:42:49.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.532+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:107 7.11 7:893be98d:::100000001f8.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f727a400 con 0x5650f7248c00 2023-10-11T09:42:49.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.532+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.7 v2:172.21.15.133:6824/35901 12 ==== osd_op_reply(107 100000001f8.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7248c00 2023-10-11T09:42:49.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.532+0000 7f13d4bc4840 10 datascan.operator(): handling object 1099511628280.1 2023-10-11T09:42:49.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.532+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:108 6.19 6:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7249c00 con 0x5650f7249400 2023-10-11T09:42:49.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.536+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.5 v2:172.21.15.133:6816/35899 28 ==== osd_op_reply(108 100000001f8.00000000 [call,setxattr (8)] v44'13 uv13 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7249400 2023-10-11T09:42:49.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.536+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:109 7.9 7:90000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7248800 con 0x5650f70f3000 2023-10-11T09:42:49.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.536+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.0 v2:172.21.15.78:6800/35735 12 ==== osd_op_reply(109 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f70f3000 2023-10-11T09:42:49.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.536+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:110 7.19 7:98000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7291400 con 0x5650f727b000 2023-10-11T09:42:49.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.540+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.6 v2:172.21.15.133:6800/35900 9 ==== osd_op_reply(110 [pgnls start_epoch 44 out=81b] v42'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f727b000 2023-10-11T09:42:49.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.540+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:111 7.19 7:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f70f3800 con 0x5650f727b000 2023-10-11T09:42:49.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.540+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.6 v2:172.21.15.133:6800/35900 10 ==== osd_op_reply(111 100000001f8.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f727b000 2023-10-11T09:42:49.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.540+0000 7f13d4bc4840 10 datascan.operator(): handling object 1099511628280.0 2023-10-11T09:42:49.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.540+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:112 6.19 6:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72a1c00 con 0x5650f7249400 2023-10-11T09:42:49.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.544+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.5 v2:172.21.15.133:6816/35899 29 ==== osd_op_reply(112 100000001f8.00000000 [call,setxattr (8)] v44'14 uv14 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7249400 2023-10-11T09:42:49.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.544+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:113 7.5 7:a0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72a0000 con 0x5650f7248c00 2023-10-11T09:42:49.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.544+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.7 v2:172.21.15.133:6824/35901 13 ==== osd_op_reply(113 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7248c00 2023-10-11T09:42:49.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.544+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:114 7.15 7:a8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7291800 con 0x5650f7248c00 2023-10-11T09:42:49.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.544+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.7 v2:172.21.15.133:6824/35901 14 ==== osd_op_reply(114 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7248c00 2023-10-11T09:42:49.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.544+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:115 7.d 7:b0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f70f3400 con 0x5650f727b000 2023-10-11T09:42:49.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.544+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.6 v2:172.21.15.133:6800/35900 11 ==== osd_op_reply(115 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f727b000 2023-10-11T09:42:49.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.544+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:116 7.1d 7:b8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7291c00 con 0x5650f7248400 2023-10-11T09:42:49.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.548+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.1 v2:172.21.15.78:6809/35734 8 ==== osd_op_reply(116 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f7248400 2023-10-11T09:42:49.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.548+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:117 7.3 7:c0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72b4000 con 0x5650f7249000 2023-10-11T09:42:49.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.548+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.3 v2:172.21.15.78:6824/35738 16 ==== osd_op_reply(117 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f7249000 2023-10-11T09:42:49.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.548+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:118 7.13 7:c8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72b4000 con 0x5650f70f3000 2023-10-11T09:42:49.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.548+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.0 v2:172.21.15.78:6800/35735 13 ==== osd_op_reply(118 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f70f3000 2023-10-11T09:42:49.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.548+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:119 7.b 7:d0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72b4400 con 0x5650f7248400 2023-10-11T09:42:49.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.548+0000 7f13d3b81700 1 -- 172.21.15.78:0/1542120834 <== osd.1 v2:172.21.15.78:6809/35734 9 ==== osd_op_reply(119 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d200 con 0x5650f7248400 2023-10-11T09:42:49.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.548+0000 7f13ce376700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:120 7.1b 7:d8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7290000 con 0x5650f70f3000 2023-10-11T09:42:49.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.548+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.0 v2:172.21.15.78:6800/35735 14 ==== osd_op_reply(120 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f70f3000 2023-10-11T09:42:49.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.548+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:121 7.7 7:e0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72b4800 con 0x5650f727b000 2023-10-11T09:42:49.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.552+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.6 v2:172.21.15.133:6800/35900 12 ==== osd_op_reply(121 [pgnls start_epoch 44 out=81b] v42'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f727b000 2023-10-11T09:42:49.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.552+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:122 7.7 7:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72a0800 con 0x5650f727b000 2023-10-11T09:42:49.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.552+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.6 v2:172.21.15.133:6800/35900 13 ==== osd_op_reply(122 10000000001.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f727b000 2023-10-11T09:42:49.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.552+0000 7f13d4bc4840 10 datascan.operator(): handling object 1099511627777.0 2023-10-11T09:42:49.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.552+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:123 6.7 6:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72a0400 con 0x5650f7249400 2023-10-11T09:42:49.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.556+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.5 v2:172.21.15.133:6816/35899 30 ==== osd_op_reply(123 10000000001.00000000 [call,setxattr (8)] v44'8 uv8 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7249400 2023-10-11T09:42:49.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.556+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:124 7.17 7:e8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72a1000 con 0x5650f7248c00 2023-10-11T09:42:49.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.556+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.7 v2:172.21.15.133:6824/35901 15 ==== osd_op_reply(124 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7248c00 2023-10-11T09:42:49.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.556+0000 7f13ceb77700 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:125 7.f 7:f0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f72b4c00 con 0x5650f70f3000 2023-10-11T09:42:49.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.556+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.0 v2:172.21.15.78:6800/35735 15 ==== osd_op_reply(125 [pgnls start_epoch 44 out=81b] v40'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f70f3000 2023-10-11T09:42:49.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.556+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:126 7.f 7:f526278d:::10000000001.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7290800 con 0x5650f70f3000 2023-10-11T09:42:49.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.556+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.0 v2:172.21.15.78:6800/35735 16 ==== osd_op_reply(126 10000000001.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f70f3000 2023-10-11T09:42:49.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.556+0000 7f13d4bc4840 10 datascan.operator(): handling object 1099511627777.1 2023-10-11T09:42:49.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.556+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:127 6.7 6:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7291800 con 0x5650f7249400 2023-10-11T09:42:49.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.560+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 <== osd.5 v2:172.21.15.133:6816/35899 31 ==== osd_op_reply(127 10000000001.00000000 [call,setxattr (8)] v44'9 uv9 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5650f7c8d440 con 0x5650f7249400 2023-10-11T09:42:49.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.560+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:128 7.1f 7:f8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x5650f7290c00 con 0x5650f727b000 2023-10-11T09:42:49.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.560+0000 7f13d2b7f700 1 -- 172.21.15.78:0/1542120834 <== osd.6 v2:172.21.15.133:6800/35900 14 ==== osd_op_reply(128 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650f7c8cfc0 con 0x5650f727b000 2023-10-11T09:42:49.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.560+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x5650f70f3000 msgr2=0x5650f6eaa100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.560+0000 7f13d4bc4840 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x5650f70f3000 0x5650f6eaa100 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.560+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x5650f7248400 msgr2=0x5650f7266000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.560+0000 7f13d4bc4840 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x5650f7248400 0x5650f7266000 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.560+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x5650f7249800 msgr2=0x5650f7266b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.560+0000 7f13d4bc4840 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x5650f7249800 0x5650f7266b00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.560+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5650f7249000 msgr2=0x5650f7267080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.560+0000 7f13d4bc4840 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5650f7249000 0x5650f7267080 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.560+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 reap_dead start 2023-10-11T09:42:49.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.560+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x5650f727a800 msgr2=0x5650f7268100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.560+0000 7f13d4bc4840 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x5650f727a800 0x5650f7268100 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.560+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x5650f7249400 msgr2=0x5650f7267600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.560+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 reap_dead start 2023-10-11T09:42:49.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.560+0000 7f13d4bc4840 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x5650f7249400 0x5650f7267600 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.560+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x5650f727b000 msgr2=0x5650f7267b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.560+0000 7f13d4bc4840 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x5650f727b000 0x5650f7267b80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.560+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x5650f7248c00 msgr2=0x5650f7266580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.560+0000 7f13d4bc4840 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x5650f7248c00 0x5650f7266580 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.564+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5650f7248000 msgr2=0x5650f6eab700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.564+0000 7f13d4bc4840 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5650f7248000 0x5650f6eab700 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x5650f7b6a8a0 tx=0x5650f7107c80 comp rx=0 tx=0).stop 2023-10-11T09:42:49.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.564+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650f70f2c00 msgr2=0x5650f6eaa680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.564+0000 7f13d4bc4840 1 --2- 172.21.15.78:0/1542120834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650f70f2c00 0x5650f6eaa680 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x5650f70fcb40 tx=0x5650f6ed9380 comp rx=0 tx=0).stop 2023-10-11T09:42:49.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.564+0000 7f13d3380700 1 -- 172.21.15.78:0/1542120834 reap_dead start 2023-10-11T09:42:49.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.564+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 shutdown_connections 2023-10-11T09:42:49.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.564+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 >> 172.21.15.78:0/1542120834 conn(0x5650f6e02c00 msgr2=0x5650f6ed1800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:49.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.564+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 shutdown_connections 2023-10-11T09:42:49.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.564+0000 7f13d4bc4840 1 -- 172.21.15.78:0/1542120834 wait complete. 2023-10-11T09:42:49.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.564+0000 7f13d4bc4840 1 -- 172.21.15.78:0/2040202056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650f6e03000 msgr2=0x5650f6ea9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.564+0000 7f13d4bc4840 1 --2- 172.21.15.78:0/2040202056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650f6e03000 0x5650f6ea9080 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x5650f7b6b380 tx=0x5650f6ef8d80 comp rx=0 tx=0).stop 2023-10-11T09:42:49.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.564+0000 7f13d4bc4840 1 -- 172.21.15.78:0/2040202056 shutdown_connections 2023-10-11T09:42:49.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.564+0000 7f13d4bc4840 1 --2- 172.21.15.78:0/2040202056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650f6e03000 0x5650f6ea9080 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.564+0000 7f13d4bc4840 1 --2- 172.21.15.78:0/2040202056 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5650f6e03400 0x5650f6ea8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.564+0000 7f13d4bc4840 1 --2- 172.21.15.78:0/2040202056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5650f6e03800 0x5650f6ea8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.564+0000 7f13d4bc4840 1 -- 172.21.15.78:0/2040202056 >> 172.21.15.78:0/2040202056 conn(0x5650f6e03c00 msgr2=0x5650f6ed1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:49.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.564+0000 7f13d4bc4840 1 -- 172.21.15.78:0/2040202056 shutdown_connections 2023-10-11T09:42:49.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.564+0000 7f13d4bc4840 1 -- 172.21.15.78:0/2040202056 wait complete. 2023-10-11T09:42:49.576 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents'] in time 0:00:00.255474, result: 2023-10-11T09:42:49.576 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes 2023-10-11T09:42:49.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.636+0000 7f9bd08e5840 1 -- start start 2023-10-11T09:42:49.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.636+0000 7f9bd08e5840 1 Processor -- start 2023-10-11T09:42:49.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.636+0000 7f9bd08e5840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e0fe81b800 0x55e0fe8c0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.636+0000 7f9bd08e5840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0fe81b400 0x55e0fe8c0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.636+0000 7f9bd08e5840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0fe81b000 0x55e0fe8c1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.636+0000 7f9bd08e5840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0fe851e00 con 0x55e0fe81b400 2023-10-11T09:42:49.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.636+0000 7f9bd08e5840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0fe89aa00 con 0x55e0fe81b000 2023-10-11T09:42:49.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.636+0000 7f9bd08e5840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e0fe89a640 con 0x55e0fe81b800 2023-10-11T09:42:49.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.636+0000 7f9bce8a0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0fe81b400 0x55e0fe8c0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.636+0000 7f9bce8a0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0fe81b400 0x55e0fe8c0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58078/0 (socket says 172.21.15.78:58078) 2023-10-11T09:42:49.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.636+0000 7f9bce8a0700 1 -- 172.21.15.78:0/2301909666 learned_addr learned my addr 172.21.15.78:0/2301909666 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:49.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.636+0000 7f9bcf8a2700 1 --2- 172.21.15.78:0/2301909666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0fe81b000 0x55e0fe8c1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.636+0000 7f9bcf0a1700 1 --2- 172.21.15.78:0/2301909666 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e0fe81b800 0x55e0fe8c0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.636+0000 7f9bce8a0700 1 -- 172.21.15.78:0/2301909666 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e0fe81b800 msgr2=0x55e0fe8c0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:49.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.636+0000 7f9bce8a0700 1 --2- 172.21.15.78:0/2301909666 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e0fe81b800 0x55e0fe8c0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.636+0000 7f9bce8a0700 1 -- 172.21.15.78:0/2301909666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0fe81b000 msgr2=0x55e0fe8c1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:49.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.636+0000 7f9bce8a0700 1 --2- 172.21.15.78:0/2301909666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0fe81b000 0x55e0fe8c1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.636+0000 7f9bce8a0700 1 -- 172.21.15.78:0/2301909666 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0fe83cb60 con 0x55e0fe81b400 2023-10-11T09:42:49.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.636+0000 7f9bce8a0700 1 --2- 172.21.15.78:0/2301909666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0fe81b400 0x55e0fe8c0b00 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x55e0fe977380 tx=0x55e0ff595140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7010152c9f7d9f8e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.640+0000 7f9bcc89c700 1 -- 172.21.15.78:0/2301909666 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e0fe84b600 con 0x55e0fe81b400 2023-10-11T09:42:49.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.640+0000 7f9bd08e5840 1 -- 172.21.15.78:0/2301909666 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e0fe83c340 con 0x55e0fe81b400 2023-10-11T09:42:49.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.640+0000 7f9bd08e5840 1 -- 172.21.15.78:0/2301909666 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55e0fe83c000 con 0x55e0fe81b400 2023-10-11T09:42:49.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.640+0000 7f9bd08e5840 4 waiting for MDS map... 2023-10-11T09:42:49.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.640+0000 7f9bcc89c700 1 -- 172.21.15.78:0/2301909666 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e0fe866900 con 0x55e0fe81b400 2023-10-11T09:42:49.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.640+0000 7f9bcc89c700 1 -- 172.21.15.78:0/2301909666 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e0fe84b1e0 con 0x55e0fe81b400 2023-10-11T09:42:49.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.640+0000 7f9bcc89c700 1 -- 172.21.15.78:0/2301909666 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(44..44 src has 1..44) v4 ==== 6667+0+0 (secure 0 0 0) 0x55e0fe854c40 con 0x55e0fe81b400 2023-10-11T09:42:49.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.640+0000 7f9bcc89c700 1 -- 172.21.15.78:0/2301909666 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) v1 ==== 3118+0+0 (secure 0 0 0) 0x55e0feb14000 con 0x55e0fe81b400 2023-10-11T09:42:49.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.640+0000 7f9bd08e5840 4 Got MDS map 22 2023-10-11T09:42:49.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.640+0000 7f9bd08e5840 4 datascan.main: Using metadata pool output 2023-10-11T09:42:49.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.640+0000 7f9bd08e5840 4 datascan.main: connecting to RADOS... 2023-10-11T09:42:49.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.640+0000 7f9bd08e5840 1 Processor -- start 2023-10-11T09:42:49.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.640+0000 7f9bd08e5840 1 -- start start 2023-10-11T09:42:49.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.640+0000 7f9bd08e5840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e0feb1c400 0x55e0fe8c1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.640+0000 7f9bd08e5840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0feb1c800 0x55e0fe8c1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.640+0000 7f9bd08e5840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0feb1cc00 0x55e0fe8c2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.640+0000 7f9bd08e5840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0fe89a500 con 0x55e0feb1cc00 2023-10-11T09:42:49.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.640+0000 7f9bd08e5840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0fe89a3c0 con 0x55e0feb1c800 2023-10-11T09:42:49.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.640+0000 7f9bd08e5840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e0fe89aa00 con 0x55e0feb1c400 2023-10-11T09:42:49.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.644+0000 7f9bce8a0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0feb1cc00 0x55e0fe8c2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.644+0000 7f9bce8a0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0feb1cc00 0x55e0fe8c2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58088/0 (socket says 172.21.15.78:58088) 2023-10-11T09:42:49.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.644+0000 7f9bce8a0700 1 -- 172.21.15.78:0/1510318195 learned_addr learned my addr 172.21.15.78:0/1510318195 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:49.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.644+0000 7f9bcf0a1700 1 --2- 172.21.15.78:0/1510318195 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0feb1c800 0x55e0fe8c1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.644+0000 7f9bcf8a2700 1 --2- 172.21.15.78:0/1510318195 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e0feb1c400 0x55e0fe8c1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.644+0000 7f9bce8a0700 1 -- 172.21.15.78:0/1510318195 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e0feb1c400 msgr2=0x55e0fe8c1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:49.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.644+0000 7f9bce8a0700 1 --2- 172.21.15.78:0/1510318195 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e0feb1c400 0x55e0fe8c1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.644+0000 7f9bce8a0700 1 -- 172.21.15.78:0/1510318195 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0feb1c800 msgr2=0x55e0fe8c1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:49.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.644+0000 7f9bce8a0700 1 --2- 172.21.15.78:0/1510318195 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0feb1c800 0x55e0fe8c1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.644+0000 7f9bce8a0700 1 -- 172.21.15.78:0/1510318195 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0fe83cea0 con 0x55e0feb1cc00 2023-10-11T09:42:49.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.644+0000 7f9bce8a0700 1 --2- 172.21.15.78:0/1510318195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0feb1cc00 0x55e0fe8c2100 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x55e0feb0e690 tx=0x55e0fe8f1260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3e179e1c0cc224 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.644+0000 7f9bca898700 1 -- 172.21.15.78:0/1510318195 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e0fe84af20 con 0x55e0feb1cc00 2023-10-11T09:42:49.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.644+0000 7f9bca898700 1 -- 172.21.15.78:0/1510318195 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e0ff608c00 con 0x55e0feb1cc00 2023-10-11T09:42:49.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.644+0000 7f9bca898700 1 -- 172.21.15.78:0/1510318195 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e0fe84b600 con 0x55e0feb1cc00 2023-10-11T09:42:49.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.644+0000 7f9bd08e5840 1 -- 172.21.15.78:0/1510318195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0feb1cc00 msgr2=0x55e0fe8c2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.644+0000 7f9bd08e5840 1 --2- 172.21.15.78:0/1510318195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0feb1cc00 0x55e0fe8c2100 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x55e0feb0e690 tx=0x55e0fe8f1260 comp rx=0 tx=0).stop 2023-10-11T09:42:49.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.644+0000 7f9bd08e5840 1 -- 172.21.15.78:0/1510318195 shutdown_connections 2023-10-11T09:42:49.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.644+0000 7f9bd08e5840 1 --2- 172.21.15.78:0/1510318195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0feb1cc00 0x55e0fe8c2100 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.644+0000 7f9bd08e5840 1 --2- 172.21.15.78:0/1510318195 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0feb1c800 0x55e0fe8c1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.644+0000 7f9bd08e5840 1 --2- 172.21.15.78:0/1510318195 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e0feb1c400 0x55e0fe8c1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.644+0000 7f9bd08e5840 1 -- 172.21.15.78:0/1510318195 >> 172.21.15.78:0/1510318195 conn(0x55e0fe81ac00 msgr2=0x55e0fe8e9800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:49.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.644+0000 7f9bd08e5840 1 -- 172.21.15.78:0/1510318195 shutdown_connections 2023-10-11T09:42:49.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.644+0000 7f9bd08e5840 1 -- 172.21.15.78:0/1510318195 wait complete. 2023-10-11T09:42:49.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.644+0000 7f9bd08e5840 1 Processor -- start 2023-10-11T09:42:49.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.648+0000 7f9bd08e5840 1 -- start start 2023-10-11T09:42:49.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.648+0000 7f9bd08e5840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0feb1c400 0x55e0fe8c3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.648+0000 7f9bd08e5840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0feb1c800 0x55e0fe8c2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.648+0000 7f9bd08e5840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e0feb1cc00 0x55e0fe8c2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.648+0000 7f9bd08e5840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0fe89a280 con 0x55e0feb1c800 2023-10-11T09:42:49.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.648+0000 7f9bd08e5840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0fe89a140 con 0x55e0feb1c400 2023-10-11T09:42:49.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.648+0000 7f9bd08e5840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e0fe89a000 con 0x55e0feb1cc00 2023-10-11T09:42:49.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.648+0000 7f9bcf0a1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0feb1c800 0x55e0fe8c2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.648+0000 7f9bcf0a1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0feb1c800 0x55e0fe8c2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58092/0 (socket says 172.21.15.78:58092) 2023-10-11T09:42:49.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.648+0000 7f9bcf0a1700 1 -- 172.21.15.78:0/82329120 learned_addr learned my addr 172.21.15.78:0/82329120 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:49.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.648+0000 7f9bce8a0700 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e0feb1cc00 0x55e0fe8c2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.648+0000 7f9bcf8a2700 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0feb1c400 0x55e0fe8c3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.648+0000 7f9bcf0a1700 1 -- 172.21.15.78:0/82329120 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e0feb1cc00 msgr2=0x55e0fe8c2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.648+0000 7f9bcf0a1700 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e0feb1cc00 0x55e0fe8c2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.648+0000 7f9bcf0a1700 1 -- 172.21.15.78:0/82329120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0feb1c400 msgr2=0x55e0fe8c3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.648+0000 7f9bcf0a1700 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0feb1c400 0x55e0fe8c3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.648+0000 7f9bcf0a1700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0fe83cb60 con 0x55e0feb1c800 2023-10-11T09:42:49.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.648+0000 7f9bcf0a1700 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0feb1c800 0x55e0fe8c2c00 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x55e0feb0f410 tx=0x55e0ff69b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.648+0000 7f9bc9095700 1 -- 172.21.15.78:0/82329120 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e0fe84af20 con 0x55e0feb1c800 2023-10-11T09:42:49.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.648+0000 7f9bc9095700 1 -- 172.21.15.78:0/82329120 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e0fe866480 con 0x55e0feb1c800 2023-10-11T09:42:49.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.648+0000 7f9bc9095700 1 -- 172.21.15.78:0/82329120 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e0fe84b600 con 0x55e0feb1c800 2023-10-11T09:42:49.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.648+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e0fe83c340 con 0x55e0feb1c800 2023-10-11T09:42:49.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.648+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e0fe83c000 con 0x55e0feb1c800 2023-10-11T09:42:49.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.648+0000 7f9bd08e5840 4 datascan.init: resolving metadata pool 5 2023-10-11T09:42:49.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.652+0000 7f9bc9095700 1 -- 172.21.15.78:0/82329120 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x55e0fe878840 con 0x55e0feb1c800 2023-10-11T09:42:49.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.652+0000 7f9bc9095700 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55e0fec72000 0x55e0fe8c3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.652+0000 7f9bc9095700 1 -- 172.21.15.78:0/82329120 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6667+0+0 (secure 0 0 0) 0x55e0fe854a80 con 0x55e0feb1c800 2023-10-11T09:42:49.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.652+0000 7f9bd08e5840 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-10-11T09:42:49.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.652+0000 7f9bd08e5840 4 datascan.main: data pool 'cephfs_data' has ID 6 2023-10-11T09:42:49.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.652+0000 7f9bd08e5840 4 datascan.main: opening data pool 'cephfs_data' 2023-10-11T09:42:49.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.652+0000 7f9bcf8a2700 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55e0fec72000 0x55e0fe8c3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.652+0000 7f9bd08e5840 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55e0feb1d000 0x55e0fe8c2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.652+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:1 5.14 5:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0feb1d400 con 0x55e0feb1d000 2023-10-11T09:42:49.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.656+0000 7f9bce8a0700 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55e0feb1d000 0x55e0fe8c2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.656+0000 7f9bcf8a2700 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55e0fec72000 0x55e0fe8c3700 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55e0fe871c20 tx=0x55e0ff69ad20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.656+0000 7f9bce8a0700 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55e0feb1d000 0x55e0fe8c2100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.656+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv41 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0feb1d000 2023-10-11T09:42:49.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.656+0000 7f9bd08e5840 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55e0feb1d800 0x55e0fe8c1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.656+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:2 5.1f 5:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec72400 con 0x55e0feb1d800 2023-10-11T09:42:49.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.656+0000 7f9bcf8a2700 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55e0feb1d800 0x55e0fe8c1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.656+0000 7f9bcf8a2700 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55e0feb1d800 0x55e0fe8c1b80 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.656+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x55e0ff593200 con 0x55e0feb1d800 2023-10-11T09:42:49.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.656+0000 7f9bd08e5840 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55e0fec72800 0x55e0fe8c1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.656+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:3 6.0 6:00000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec72c00 con 0x55e0fec72800 2023-10-11T09:42:49.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.660+0000 7f9bcf0a1700 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55e0fec72800 0x55e0fe8c1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.660+0000 7f9bcf0a1700 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55e0fec72800 0x55e0fe8c1600 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.660+0000 7f9bcf0a1700 1 -- 172.21.15.78:0/82329120 <== osd.0 v2:172.21.15.78:6800/35735 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 44 out=49b] v42'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593440 con 0x55e0fec72800 2023-10-11T09:42:49.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.660+0000 7f9bca097700 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55e0fec92000 0x55e0fec98000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.660+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:4 6.20 6:04000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec92400 con 0x55e0fec92000 2023-10-11T09:42:49.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.660+0000 7f9bce8a0700 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55e0fec92000 0x55e0fec98000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.660+0000 7f9bce8a0700 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55e0fec92000 0x55e0fec98000 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.660+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.1 v2:172.21.15.78:6809/35734 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 44 out=49b] v42'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0fec92000 2023-10-11T09:42:49.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.660+0000 7f9bca898700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:5 6.10 6:08000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec92c00 con 0x55e0fec72800 2023-10-11T09:42:49.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.664+0000 7f9bcf0a1700 1 -- 172.21.15.78:0/82329120 <== osd.0 v2:172.21.15.78:6800/35735 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 44 out=49b] v42'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593440 con 0x55e0fec72800 2023-10-11T09:42:49.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.664+0000 7f9bca097700 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55e0fec92800 0x55e0fec98580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.664+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:6 6.30 6:0c000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec93000 con 0x55e0fec92800 2023-10-11T09:42:49.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.664+0000 7f9bcf8a2700 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55e0fec92800 0x55e0fec98580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.664+0000 7f9bcf8a2700 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55e0fec92800 0x55e0fec98580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.664+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.7 v2:172.21.15.133:6824/35901 1 ==== osd_op_reply(6 [pgnls-filter start_epoch 44 out=49b] v42'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fec92800 2023-10-11T09:42:49.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.664+0000 7f9bca898700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:7 6.8 6:10000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec93000 con 0x55e0fec92800 2023-10-11T09:42:49.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.668+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.7 v2:172.21.15.133:6824/35901 2 ==== osd_op_reply(7 [pgnls-filter start_epoch 44 out=49b] v42'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fec92800 2023-10-11T09:42:49.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.668+0000 7f9bca097700 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55e0fec93400 0x55e0fec98b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.668+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:8 6.28 6:14000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec93800 con 0x55e0fec93400 2023-10-11T09:42:49.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.668+0000 7f9bcf0a1700 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55e0fec93400 0x55e0fec98b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.668+0000 7f9bcf0a1700 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55e0fec93400 0x55e0fec98b00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.668+0000 7f9bcf0a1700 1 -- 172.21.15.78:0/82329120 <== osd.2 v2:172.21.15.78:6808/35732 1 ==== osd_op_reply(8 [pgnls-filter start_epoch 44 out=49b] v42'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593440 con 0x55e0fec93400 2023-10-11T09:42:49.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.668+0000 7f9bca898700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:9 6.18 6:18000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec72400 con 0x55e0fec72800 2023-10-11T09:42:49.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.668+0000 7f9bcf0a1700 1 -- 172.21.15.78:0/82329120 <== osd.0 v2:172.21.15.78:6800/35735 3 ==== osd_op_reply(9 [pgnls-filter start_epoch 44 out=49b] v42'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593440 con 0x55e0fec72800 2023-10-11T09:42:49.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.668+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:10 6.38 6:1c000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec92c00 con 0x55e0feb1d000 2023-10-11T09:42:49.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.668+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.3 v2:172.21.15.78:6824/35738 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 44 out=49b] v42'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0feb1d000 2023-10-11T09:42:49.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.668+0000 7f9bca898700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:11 6.4 6:20000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec92400 con 0x55e0fec92000 2023-10-11T09:42:49.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.668+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.1 v2:172.21.15.78:6809/35734 2 ==== osd_op_reply(11 [pgnls-filter start_epoch 44 out=49b] v42'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0fec92000 2023-10-11T09:42:49.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.668+0000 7f9bca097700 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55e0fec93800 0x55e0fec99080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.668+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:12 6.24 6:24000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec93c00 con 0x55e0fec93800 2023-10-11T09:42:49.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.672+0000 7f9bce8a0700 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55e0fec93800 0x55e0fec99080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.672+0000 7f9bce8a0700 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55e0fec93800 0x55e0fec99080 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.672+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.5 v2:172.21.15.133:6816/35899 1 ==== osd_op_reply(12 [pgnls-filter start_epoch 44 out=49b] v42'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0fec93800 2023-10-11T09:42:49.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.672+0000 7f9bca898700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:13 6.14 6:28000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec92c00 con 0x55e0fec93400 2023-10-11T09:42:49.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.672+0000 7f9bcf0a1700 1 -- 172.21.15.78:0/82329120 <== osd.2 v2:172.21.15.78:6808/35732 2 ==== osd_op_reply(13 [pgnls-filter start_epoch 44 out=49b] v42'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593440 con 0x55e0fec93400 2023-10-11T09:42:49.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.672+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:14 6.34 6:2c000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec72c00 con 0x55e0fec72800 2023-10-11T09:42:49.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.672+0000 7f9bcf0a1700 1 -- 172.21.15.78:0/82329120 <== osd.0 v2:172.21.15.78:6800/35735 4 ==== osd_op_reply(14 [pgnls-filter start_epoch 44 out=49b] v42'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593440 con 0x55e0fec72800 2023-10-11T09:42:49.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.672+0000 7f9bca898700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:15 6.c 6:30000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0feb1d400 con 0x55e0feb1d000 2023-10-11T09:42:49.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.676+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.3 v2:172.21.15.78:6824/35738 3 ==== osd_op_reply(15 [pgnls-filter start_epoch 44 out=49b] v42'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0feb1d000 2023-10-11T09:42:49.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.676+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:16 6.2c 6:34000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecac000 con 0x55e0feb1d800 2023-10-11T09:42:49.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.676+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.6 v2:172.21.15.133:6800/35900 2 ==== osd_op_reply(16 [pgnls-filter start_epoch 44 out=49b] v42'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593200 con 0x55e0feb1d800 2023-10-11T09:42:49.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.676+0000 7f9bca898700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:17 6.1c 6:38000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecad400 con 0x55e0fec92800 2023-10-11T09:42:49.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.676+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.7 v2:172.21.15.133:6824/35901 3 ==== osd_op_reply(17 [pgnls-filter start_epoch 44 out=49b] v42'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fec92800 2023-10-11T09:42:49.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.676+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:18 6.3c 6:3c000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecac400 con 0x55e0fec93400 2023-10-11T09:42:49.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.676+0000 7f9bcf0a1700 1 -- 172.21.15.78:0/82329120 <== osd.2 v2:172.21.15.78:6808/35732 3 ==== osd_op_reply(18 [pgnls-filter start_epoch 44 out=49b] v42'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593440 con 0x55e0fec93400 2023-10-11T09:42:49.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.676+0000 7f9bca898700 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55e0fecad800 0x55e0fec9a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.676+0000 7f9bca898700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:19 6.2 6:40000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecadc00 con 0x55e0fecad800 2023-10-11T09:42:49.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.680+0000 7f9bcf8a2700 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55e0fecad800 0x55e0fec9a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.680+0000 7f9bcf8a2700 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55e0fecad800 0x55e0fec9a100 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.680+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.4 v2:172.21.15.133:6808/35898 1 ==== osd_op_reply(19 [pgnls-filter start_epoch 44 out=49b] v42'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fecad800 2023-10-11T09:42:49.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.680+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:20 6.22 6:44000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecac800 con 0x55e0feb1d800 2023-10-11T09:42:49.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.680+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.6 v2:172.21.15.133:6800/35900 3 ==== osd_op_reply(20 [pgnls-filter start_epoch 44 out=49b] v42'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593200 con 0x55e0feb1d800 2023-10-11T09:42:49.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.680+0000 7f9bca898700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:21 6.12 6:48000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecac400 con 0x55e0fec92800 2023-10-11T09:42:49.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.684+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.7 v2:172.21.15.133:6824/35901 4 ==== osd_op_reply(21 [pgnls-filter start_epoch 44 out=49b] v42'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fec92800 2023-10-11T09:42:49.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.684+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:22 6.32 6:4c000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecacc00 con 0x55e0fecad800 2023-10-11T09:42:49.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.684+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.4 v2:172.21.15.133:6808/35898 2 ==== osd_op_reply(22 [pgnls-filter start_epoch 44 out=49b] v42'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fecad800 2023-10-11T09:42:49.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.684+0000 7f9bca898700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:23 6.a 6:50000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec92c00 con 0x55e0fec93800 2023-10-11T09:42:49.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.684+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.5 v2:172.21.15.133:6816/35899 2 ==== osd_op_reply(23 [pgnls-filter start_epoch 44 out=49b] v42'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0fec93800 2023-10-11T09:42:49.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.684+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:24 6.2a 6:54000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecad000 con 0x55e0fecad800 2023-10-11T09:42:49.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.684+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.4 v2:172.21.15.133:6808/35898 3 ==== osd_op_reply(24 [pgnls-filter start_epoch 44 out=81b] v44'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fecad800 2023-10-11T09:42:49.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.684+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:25 6.0 6:00000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec73000 con 0x55e0fec72800 2023-10-11T09:42:49.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.688+0000 7f9bcf0a1700 1 -- 172.21.15.78:0/82329120 <== osd.0 v2:172.21.15.78:6800/35735 5 ==== osd_op_reply(25 [pgnls-filter start_epoch 44 out=49b] v42'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593440 con 0x55e0fec72800 2023-10-11T09:42:49.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.688+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:26 6.20 6:04000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecadc00 con 0x55e0fec92000 2023-10-11T09:42:49.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.688+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.1 v2:172.21.15.78:6809/35734 3 ==== osd_op_reply(26 [pgnls-filter start_epoch 44 out=49b] v42'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0fec92000 2023-10-11T09:42:49.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.688+0000 7f9bca898700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:27 6.10 6:08000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec93000 con 0x55e0fec72800 2023-10-11T09:42:49.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.688+0000 7f9bcf0a1700 1 -- 172.21.15.78:0/82329120 <== osd.0 v2:172.21.15.78:6800/35735 6 ==== osd_op_reply(27 [pgnls-filter start_epoch 44 out=49b] v42'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593440 con 0x55e0fec72800 2023-10-11T09:42:49.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.688+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:28 6.30 6:0c000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecac800 con 0x55e0fec92800 2023-10-11T09:42:49.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.688+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.7 v2:172.21.15.133:6824/35901 5 ==== osd_op_reply(28 [pgnls-filter start_epoch 44 out=49b] v42'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fec92800 2023-10-11T09:42:49.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.688+0000 7f9bca898700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:29 6.8 6:10000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec72400 con 0x55e0fec92800 2023-10-11T09:42:49.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.692+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.7 v2:172.21.15.133:6824/35901 6 ==== osd_op_reply(29 [pgnls-filter start_epoch 44 out=49b] v42'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fec92800 2023-10-11T09:42:49.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.692+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:30 6.28 6:14000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec92400 con 0x55e0fec93400 2023-10-11T09:42:49.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.692+0000 7f9bcf0a1700 1 -- 172.21.15.78:0/82329120 <== osd.2 v2:172.21.15.78:6808/35732 4 ==== osd_op_reply(30 [pgnls-filter start_epoch 44 out=49b] v42'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593440 con 0x55e0fec93400 2023-10-11T09:42:49.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.692+0000 7f9bca898700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:31 6.18 6:18000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec72c00 con 0x55e0fec72800 2023-10-11T09:42:49.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.692+0000 7f9bcf0a1700 1 -- 172.21.15.78:0/82329120 <== osd.0 v2:172.21.15.78:6800/35735 7 ==== osd_op_reply(31 [pgnls-filter start_epoch 44 out=49b] v42'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593440 con 0x55e0fec72800 2023-10-11T09:42:49.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.692+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:32 6.38 6:1c000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec93c00 con 0x55e0feb1d000 2023-10-11T09:42:49.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.692+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.3 v2:172.21.15.78:6824/35738 4 ==== osd_op_reply(32 [pgnls-filter start_epoch 44 out=49b] v42'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0feb1d000 2023-10-11T09:42:49.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.692+0000 7f9bca898700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:33 6.4 6:20000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecac400 con 0x55e0fec92000 2023-10-11T09:42:49.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.692+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.1 v2:172.21.15.78:6809/35734 4 ==== osd_op_reply(33 [pgnls-filter start_epoch 44 out=49b] v42'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0fec92000 2023-10-11T09:42:49.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.692+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:34 6.24 6:24000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec92c00 con 0x55e0fec93800 2023-10-11T09:42:49.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.696+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.5 v2:172.21.15.133:6816/35899 3 ==== osd_op_reply(34 [pgnls-filter start_epoch 44 out=49b] v42'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0fec93800 2023-10-11T09:42:49.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.696+0000 7f9bca898700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:35 6.14 6:28000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecacc00 con 0x55e0fec93400 2023-10-11T09:42:49.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.696+0000 7f9bcf0a1700 1 -- 172.21.15.78:0/82329120 <== osd.2 v2:172.21.15.78:6808/35732 5 ==== osd_op_reply(35 [pgnls-filter start_epoch 44 out=49b] v42'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593440 con 0x55e0fec93400 2023-10-11T09:42:49.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.696+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:36 6.34 6:2c000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0feb1d400 con 0x55e0fec72800 2023-10-11T09:42:49.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.696+0000 7f9bcf0a1700 1 -- 172.21.15.78:0/82329120 <== osd.0 v2:172.21.15.78:6800/35735 8 ==== osd_op_reply(36 [pgnls-filter start_epoch 44 out=49b] v42'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593440 con 0x55e0fec72800 2023-10-11T09:42:49.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.696+0000 7f9bca898700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:37 6.c 6:30000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecac000 con 0x55e0feb1d000 2023-10-11T09:42:49.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.696+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.3 v2:172.21.15.78:6824/35738 5 ==== osd_op_reply(37 [pgnls-filter start_epoch 44 out=49b] v42'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0feb1d000 2023-10-11T09:42:49.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.696+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:38 6.2c 6:34000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecadc00 con 0x55e0feb1d800 2023-10-11T09:42:49.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.696+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.6 v2:172.21.15.133:6800/35900 4 ==== osd_op_reply(38 [pgnls-filter start_epoch 44 out=49b] v42'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593200 con 0x55e0feb1d800 2023-10-11T09:42:49.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.696+0000 7f9bca898700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:39 6.1c 6:38000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecad400 con 0x55e0fec92800 2023-10-11T09:42:49.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.700+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.7 v2:172.21.15.133:6824/35901 7 ==== osd_op_reply(39 [pgnls-filter start_epoch 44 out=49b] v42'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fec92800 2023-10-11T09:42:49.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.700+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:40 6.3c 6:3c000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec73000 con 0x55e0fec93400 2023-10-11T09:42:49.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.700+0000 7f9bcf0a1700 1 -- 172.21.15.78:0/82329120 <== osd.2 v2:172.21.15.78:6808/35732 6 ==== osd_op_reply(40 [pgnls-filter start_epoch 44 out=49b] v42'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593440 con 0x55e0fec93400 2023-10-11T09:42:49.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.700+0000 7f9bca898700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:41 6.2 6:40000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecad000 con 0x55e0fecad800 2023-10-11T09:42:49.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.700+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.4 v2:172.21.15.133:6808/35898 4 ==== osd_op_reply(41 [pgnls-filter start_epoch 44 out=49b] v42'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fecad800 2023-10-11T09:42:49.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.700+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:42 6.22 6:44000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec72c00 con 0x55e0feb1d800 2023-10-11T09:42:49.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.700+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.6 v2:172.21.15.133:6800/35900 5 ==== osd_op_reply(42 [pgnls-filter start_epoch 44 out=49b] v42'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593200 con 0x55e0feb1d800 2023-10-11T09:42:49.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.700+0000 7f9bca898700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:43 6.12 6:48000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecac800 con 0x55e0fec92800 2023-10-11T09:42:49.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.704+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.7 v2:172.21.15.133:6824/35901 8 ==== osd_op_reply(43 [pgnls-filter start_epoch 44 out=49b] v42'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fec92800 2023-10-11T09:42:49.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.704+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:44 6.32 6:4c000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec93c00 con 0x55e0fecad800 2023-10-11T09:42:49.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.704+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.4 v2:172.21.15.133:6808/35898 5 ==== osd_op_reply(44 [pgnls-filter start_epoch 44 out=49b] v42'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fecad800 2023-10-11T09:42:49.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.704+0000 7f9bca898700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:45 6.a 6:50000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecadc00 con 0x55e0fec93800 2023-10-11T09:42:49.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.704+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.5 v2:172.21.15.133:6816/35899 4 ==== osd_op_reply(45 [pgnls-filter start_epoch 44 out=49b] v42'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0fec93800 2023-10-11T09:42:49.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.704+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:46 6.2a 6:54000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec92c00 con 0x55e0fecad800 2023-10-11T09:42:49.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.704+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.4 v2:172.21.15.133:6808/35898 6 ==== osd_op_reply(46 [pgnls-filter start_epoch 44 out=81b] v44'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fecad800 2023-10-11T09:42:49.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.704+0000 7f9bd08e5840 20 datascan.forall_objects: OSD matched oid 100000001f6.00000000 2023-10-11T09:42:49.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.704+0000 7f9bd08e5840 10 datascan.operator(): handling object 100000001f6.0 2023-10-11T09:42:49.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.708+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:47 6.2a 6:57fe7535:::100000001f6.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec73400 con 0x55e0fecad800 2023-10-11T09:42:49.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.708+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.4 v2:172.21.15.133:6808/35898 7 ==== osd_op_reply(47 100000001f6.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fecad800 2023-10-11T09:42:49.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.708+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:48 7.a 7:57fe7535:::100000001f6.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec73800 con 0x55e0feb1d000 2023-10-11T09:42:49.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.708+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.3 v2:172.21.15.78:6824/35738 6 ==== osd_op_reply(48 100000001f6.00000000 [getxattr (30) out=30b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0feb1d000 2023-10-11T09:42:49.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.708+0000 7f9bd08e5840 20 datascan.operator(): loaded layout from xattr: pi: 7 os: 4194304 sc: 1 su: 4194304 2023-10-11T09:42:49.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.708+0000 7f9bd08e5840 10 datascan.inject_with_backtrace: inode: 0x0x100000001f6 2023-10-11T09:42:49.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.708+0000 7f9bd08e5840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/1 2023-10-11T09:42:49.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.708+0000 7f9bd08e5840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=1 2023-10-11T09:42:49.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.708+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:49 5.10 5:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec73c00 con 0x55e0fec92800 2023-10-11T09:42:49.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.708+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.7 v2:172.21.15.133:6824/35901 9 ==== osd_op_reply(49 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fec92800 2023-10-11T09:42:49.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.708+0000 7f9bd08e5840 4 datascan.get_frag_of: Unexpected error on '10000000000.00000000': (2) No such file or directory 2023-10-11T09:42:49.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.708+0000 7f9bd08e5840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/1, will insert to root 2023-10-11T09:42:49.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.708+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:50 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecad000 con 0x55e0fec92800 2023-10-11T09:42:49.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.708+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.7 v2:172.21.15.133:6824/35901 10 ==== osd_op_reply(50 10000000000.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fec92800 2023-10-11T09:42:49.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.708+0000 7f9bd08e5840 10 datascan.find_or_create_dirfrag: read_version = 0 2023-10-11T09:42:49.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.708+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:51 5.10 5:0cf6ad0f:::10000000000.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecac800 con 0x55e0fec92800 2023-10-11T09:42:49.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.712+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.7 v2:172.21.15.133:6824/35901 11 ==== osd_op_reply(51 10000000000.00000000 [create,omap-set-header] v44'16 uv16 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fec92800 2023-10-11T09:42:49.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.712+0000 7f9bd08e5840 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x10000000000 2023-10-11T09:42:49.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.712+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:52 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecacc00 con 0x55e0fec92800 2023-10-11T09:42:49.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.716+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.7 v2:172.21.15.133:6824/35901 12 ==== osd_op_reply(52 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fec92800 2023-10-11T09:42:49.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.716+0000 7f9bd08e5840 20 datascan.read_dentry: oid=10000000000.00000000 dname=1 frag=*, r=0 2023-10-11T09:42:49.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.716+0000 7f9bd08e5840 20 datascan.read_dentry: 1_head not found in result 2023-10-11T09:42:49.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.716+0000 7f9bd08e5840 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f6 at 0x0x10000000000/1 with size=6291456 bytes 2023-10-11T09:42:49.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.716+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:53 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec72c00 con 0x55e0fec92800 2023-10-11T09:42:49.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.716+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.7 v2:172.21.15.133:6824/35901 13 ==== osd_op_reply(53 10000000000.00000000 [omap-set-vals] v44'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fec92800 2023-10-11T09:42:49.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.716+0000 7f9bd08e5840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/1 pointing to 0x0x100000001f6 2023-10-11T09:42:49.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.716+0000 7f9bd08e5840 10 datascan.inject_with_backtrace: backptr: 0x0x1/subdir 2023-10-11T09:42:49.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.716+0000 7f9bd08e5840 20 datascan.get_frag_of: dirino=0x1 target_dname=subdir 2023-10-11T09:42:49.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.716+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:54 5.1f 5:ff5b34d6:::1.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec73400 con 0x55e0feb1d800 2023-10-11T09:42:49.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.716+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.6 v2:172.21.15.133:6800/35900 6 ==== osd_op_reply(54 1.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x55e0ff593200 con 0x55e0feb1d800 2023-10-11T09:42:49.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.716+0000 7f9bd08e5840 4 datascan.get_frag_of: Unexpected error on '1.00000000': (2) No such file or directory 2023-10-11T09:42:49.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.716+0000 7f9bd08e5840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x1/subdir, will insert to root 2023-10-11T09:42:49.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.716+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:55 5.1f 5:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec93c00 con 0x55e0feb1d800 2023-10-11T09:42:49.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.720+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.6 v2:172.21.15.133:6800/35900 7 ==== osd_op_reply(55 1.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x55e0ff593200 con 0x55e0feb1d800 2023-10-11T09:42:49.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.720+0000 7f9bd08e5840 10 datascan.find_or_create_dirfrag: read_version = 0 2023-10-11T09:42:49.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.720+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:56 5.1f 5:ff5b34d6:::1.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec92c00 con 0x55e0feb1d800 2023-10-11T09:42:49.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.720+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.6 v2:172.21.15.133:6800/35900 8 ==== osd_op_reply(56 1.00000000 [create,omap-set-header] v44'11 uv11 ondisk = 0) v8 ==== 196+0+0 (crc 0 0 0) 0x55e0ff593200 con 0x55e0feb1d800 2023-10-11T09:42:49.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.720+0000 7f9bd08e5840 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x1 2023-10-11T09:42:49.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.720+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:57 5.1f 5:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec93c00 con 0x55e0feb1d800 2023-10-11T09:42:49.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.724+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.6 v2:172.21.15.133:6800/35900 9 ==== osd_op_reply(57 1.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv11 ondisk = 0) v8 ==== 154+0+4 (crc 0 0 0) 0x55e0ff593200 con 0x55e0feb1d800 2023-10-11T09:42:49.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.724+0000 7f9bd08e5840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:42:49.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.724+0000 7f9bd08e5840 20 datascan.read_dentry: subdir_head not found in result 2023-10-11T09:42:49.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.724+0000 7f9bd08e5840 10 datascan.inject_with_backtrace: Linking ancestor directory of inode 0x0x10000000000 at 0x0x1:subdir 2023-10-11T09:42:49.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.724+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:58 5.1f 5:ff5b34d6:::1.00000000:head [omap-set-vals in=507b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec73c00 con 0x55e0feb1d800 2023-10-11T09:42:49.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.724+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.6 v2:172.21.15.133:6800/35900 10 ==== osd_op_reply(58 1.00000000 [omap-set-vals] v44'12 uv12 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x55e0ff593200 con 0x55e0feb1d800 2023-10-11T09:42:49.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.724+0000 7f9bd08e5840 20 datascan.inject_linkage: Injected dentry 0x0x1/subdir pointing to 0x0x10000000000 2023-10-11T09:42:49.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.724+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:59 6.1a 6:58000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec72c00 con 0x55e0fecad800 2023-10-11T09:42:49.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.728+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.4 v2:172.21.15.133:6808/35898 8 ==== osd_op_reply(59 [pgnls-filter start_epoch 44 out=49b] v42'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fecad800 2023-10-11T09:42:49.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.728+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:60 6.3a 6:5c000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec93000 con 0x55e0fec92000 2023-10-11T09:42:49.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.728+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.1 v2:172.21.15.78:6809/35734 5 ==== osd_op_reply(60 [pgnls-filter start_epoch 44 out=49b] v42'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0fec92000 2023-10-11T09:42:49.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.728+0000 7f9bca898700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:61 6.6 6:60000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecad400 con 0x55e0feb1d000 2023-10-11T09:42:49.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.728+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.3 v2:172.21.15.78:6824/35738 7 ==== osd_op_reply(61 [pgnls-filter start_epoch 44 out=49b] v42'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0feb1d000 2023-10-11T09:42:49.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.728+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:62 6.26 6:64000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec92400 con 0x55e0fec92000 2023-10-11T09:42:49.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.728+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.1 v2:172.21.15.78:6809/35734 6 ==== osd_op_reply(62 [pgnls-filter start_epoch 44 out=49b] v42'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0fec92000 2023-10-11T09:42:49.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.728+0000 7f9bca898700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:63 6.16 6:68000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecac400 con 0x55e0fec72800 2023-10-11T09:42:49.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.728+0000 7f9bcf0a1700 1 -- 172.21.15.78:0/82329120 <== osd.0 v2:172.21.15.78:6800/35735 9 ==== osd_op_reply(63 [pgnls-filter start_epoch 44 out=49b] v42'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593440 con 0x55e0fec72800 2023-10-11T09:42:49.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.732+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:64 6.36 6:6c000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecc6000 con 0x55e0fec93800 2023-10-11T09:42:49.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.732+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.5 v2:172.21.15.133:6816/35899 5 ==== osd_op_reply(64 [pgnls-filter start_epoch 44 out=101b] v44'23 uv23 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0fec93800 2023-10-11T09:42:49.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.732+0000 7f9bd08e5840 20 datascan.forall_objects: OSD matched oid 100000001f9.00000000 2023-10-11T09:42:49.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.732+0000 7f9bd08e5840 10 datascan.operator(): handling object 100000001f9.0 2023-10-11T09:42:49.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.732+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:65 6.36 6:6ee5e2c4:::100000001f9.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec73400 con 0x55e0fec93800 2023-10-11T09:42:49.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.732+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.5 v2:172.21.15.133:6816/35899 6 ==== osd_op_reply(65 100000001f9.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv21 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0fec93800 2023-10-11T09:42:49.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.732+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:66 7.16 7:6ee5e2c4:::100000001f9.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecad000 con 0x55e0fecad800 2023-10-11T09:42:49.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.732+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.4 v2:172.21.15.133:6808/35898 9 ==== osd_op_reply(66 100000001f9.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fecad800 2023-10-11T09:42:49.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.732+0000 7f9bd08e5840 20 datascan.operator(): loaded layout from xattr: pi: 7 os: 4194304 sc: 1 su: 4194304 2023-10-11T09:42:49.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.732+0000 7f9bd08e5840 10 datascan.inject_with_backtrace: inode: 0x0x100000001f9 2023-10-11T09:42:49.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.732+0000 7f9bd08e5840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/4 2023-10-11T09:42:49.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.732+0000 7f9bd08e5840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=4 2023-10-11T09:42:49.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.732+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:67 5.10 5:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecac800 con 0x55e0fec92800 2023-10-11T09:42:49.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.732+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.7 v2:172.21.15.133:6824/35901 14 ==== osd_op_reply(67 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fec92800 2023-10-11T09:42:49.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.732+0000 7f9bd08e5840 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-10-11T09:42:49.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.732+0000 7f9bd08e5840 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-10-11T09:42:49.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.732+0000 7f9bd08e5840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/4, will insert to root 2023-10-11T09:42:49.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.736+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:68 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecacc00 con 0x55e0fec92800 2023-10-11T09:42:49.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.736+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.7 v2:172.21.15.133:6824/35901 15 ==== osd_op_reply(68 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fec92800 2023-10-11T09:42:49.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.736+0000 7f9bd08e5840 10 datascan.find_or_create_dirfrag: read_version = 17 2023-10-11T09:42:49.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.736+0000 7f9bd08e5840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-10-11T09:42:49.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.736+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:69 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecc6000 con 0x55e0fec92800 2023-10-11T09:42:49.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.736+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.7 v2:172.21.15.133:6824/35901 16 ==== osd_op_reply(69 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fec92800 2023-10-11T09:42:49.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.736+0000 7f9bd08e5840 20 datascan.read_dentry: oid=10000000000.00000000 dname=4 frag=*, r=0 2023-10-11T09:42:49.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.736+0000 7f9bd08e5840 20 datascan.read_dentry: 4_head not found in result 2023-10-11T09:42:49.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.736+0000 7f9bd08e5840 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f9 at 0x0x10000000000/4 with size=6291456 bytes 2023-10-11T09:42:49.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.736+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:70 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec93000 con 0x55e0fec92800 2023-10-11T09:42:49.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.740+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.7 v2:172.21.15.133:6824/35901 17 ==== osd_op_reply(70 10000000000.00000000 [omap-set-vals] v44'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fec92800 2023-10-11T09:42:49.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.740+0000 7f9bd08e5840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/4 pointing to 0x0x100000001f9 2023-10-11T09:42:49.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.740+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:71 6.36 6:6fe38dc2::::head [pgnls-filter start_epoch 44 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec92400 con 0x55e0fec93800 2023-10-11T09:42:49.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.740+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.5 v2:172.21.15.133:6816/35899 7 ==== osd_op_reply(71 [pgnls-filter start_epoch 44 out=81b] v44'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0fec93800 2023-10-11T09:42:49.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.740+0000 7f9bd08e5840 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2023-10-11T09:42:49.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.740+0000 7f9bd08e5840 10 datascan.operator(): handling object 100000001f7.0 2023-10-11T09:42:49.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.740+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:72 6.36 6:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecadc00 con 0x55e0fec93800 2023-10-11T09:42:49.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.740+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.5 v2:172.21.15.133:6816/35899 8 ==== osd_op_reply(72 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv23 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0fec93800 2023-10-11T09:42:49.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.740+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:73 7.16 7:6fe38dc2:::100000001f7.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecad400 con 0x55e0fecad800 2023-10-11T09:42:49.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.744+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.4 v2:172.21.15.133:6808/35898 10 ==== osd_op_reply(73 100000001f7.00000000 [getxattr (30) out=30b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fecad800 2023-10-11T09:42:49.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.744+0000 7f9bd08e5840 20 datascan.operator(): loaded layout from xattr: pi: 7 os: 4194304 sc: 1 su: 4194304 2023-10-11T09:42:49.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.744+0000 7f9bd08e5840 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2023-10-11T09:42:49.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.744+0000 7f9bd08e5840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/2 2023-10-11T09:42:49.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.744+0000 7f9bd08e5840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=2 2023-10-11T09:42:49.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.744+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:74 5.10 5:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec73800 con 0x55e0fec92800 2023-10-11T09:42:49.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.744+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.7 v2:172.21.15.133:6824/35901 18 ==== osd_op_reply(74 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fec92800 2023-10-11T09:42:49.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.744+0000 7f9bd08e5840 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-10-11T09:42:49.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.744+0000 7f9bd08e5840 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-10-11T09:42:49.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.744+0000 7f9bd08e5840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/2, will insert to root 2023-10-11T09:42:49.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.744+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:75 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecc6000 con 0x55e0fec92800 2023-10-11T09:42:49.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.744+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.7 v2:172.21.15.133:6824/35901 19 ==== osd_op_reply(75 10000000000.00000000 [omap-get-header out=274b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fec92800 2023-10-11T09:42:49.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.744+0000 7f9bd08e5840 10 datascan.find_or_create_dirfrag: read_version = 18 2023-10-11T09:42:49.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.744+0000 7f9bd08e5840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-10-11T09:42:49.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.744+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:76 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecad000 con 0x55e0fec92800 2023-10-11T09:42:49.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.744+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.7 v2:172.21.15.133:6824/35901 20 ==== osd_op_reply(76 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fec92800 2023-10-11T09:42:49.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.744+0000 7f9bd08e5840 20 datascan.read_dentry: oid=10000000000.00000000 dname=2 frag=*, r=0 2023-10-11T09:42:49.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.744+0000 7f9bd08e5840 20 datascan.read_dentry: 2_head not found in result 2023-10-11T09:42:49.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.744+0000 7f9bd08e5840 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000000/2 with size=6291456 bytes 2023-10-11T09:42:49.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.744+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:77 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecac800 con 0x55e0fec92800 2023-10-11T09:42:49.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.748+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.7 v2:172.21.15.133:6824/35901 21 ==== osd_op_reply(77 10000000000.00000000 [omap-set-vals] v44'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fec92800 2023-10-11T09:42:49.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.748+0000 7f9bd08e5840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/2 pointing to 0x0x100000001f7 2023-10-11T09:42:49.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.748+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:78 6.e 6:70000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecacc00 con 0x55e0fecad800 2023-10-11T09:42:49.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.748+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.4 v2:172.21.15.133:6808/35898 11 ==== osd_op_reply(78 [pgnls-filter start_epoch 44 out=49b] v42'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fecad800 2023-10-11T09:42:49.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.748+0000 7f9bca898700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:79 6.2e 6:74000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecac000 con 0x55e0feb1d800 2023-10-11T09:42:49.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.748+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.6 v2:172.21.15.133:6800/35900 11 ==== osd_op_reply(79 [pgnls-filter start_epoch 44 out=49b] v42'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593200 con 0x55e0feb1d800 2023-10-11T09:42:49.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.748+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:80 6.1e 6:78000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecc6400 con 0x55e0fecad800 2023-10-11T09:42:49.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.752+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.4 v2:172.21.15.133:6808/35898 12 ==== osd_op_reply(80 [pgnls-filter start_epoch 44 out=49b] v42'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fecad800 2023-10-11T09:42:49.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.752+0000 7f9bca898700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:81 6.3e 6:7c000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec72400 con 0x55e0feb1d000 2023-10-11T09:42:49.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.752+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.3 v2:172.21.15.78:6824/35738 8 ==== osd_op_reply(81 [pgnls-filter start_epoch 44 out=49b] v42'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0feb1d000 2023-10-11T09:42:49.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.752+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:82 6.1 6:80000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecc6800 con 0x55e0fec92000 2023-10-11T09:42:49.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.752+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.1 v2:172.21.15.78:6809/35734 7 ==== osd_op_reply(82 [pgnls-filter start_epoch 44 out=49b] v42'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0fec92000 2023-10-11T09:42:49.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.752+0000 7f9bca898700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:83 6.21 6:84000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecc6000 con 0x55e0fec92800 2023-10-11T09:42:49.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.752+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.7 v2:172.21.15.133:6824/35901 22 ==== osd_op_reply(83 [pgnls-filter start_epoch 44 out=49b] v42'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fec92800 2023-10-11T09:42:49.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.752+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:84 6.11 6:88000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecc6c00 con 0x55e0feb1d000 2023-10-11T09:42:49.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.756+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.3 v2:172.21.15.78:6824/35738 9 ==== osd_op_reply(84 [pgnls-filter start_epoch 44 out=49b] v42'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0feb1d000 2023-10-11T09:42:49.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.756+0000 7f9bca898700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:85 6.31 6:8c000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecad400 con 0x55e0fec93800 2023-10-11T09:42:49.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.756+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.5 v2:172.21.15.133:6816/35899 9 ==== osd_op_reply(85 [pgnls-filter start_epoch 44 out=49b] v42'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0fec93800 2023-10-11T09:42:49.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.756+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:86 6.9 6:90000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecc7000 con 0x55e0fec72800 2023-10-11T09:42:49.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.756+0000 7f9bcf0a1700 1 -- 172.21.15.78:0/82329120 <== osd.0 v2:172.21.15.78:6800/35735 10 ==== osd_op_reply(86 [pgnls-filter start_epoch 44 out=49b] v42'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593440 con 0x55e0fec72800 2023-10-11T09:42:49.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.756+0000 7f9bca898700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:87 6.29 6:94000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecad000 con 0x55e0fec92800 2023-10-11T09:42:49.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.756+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.7 v2:172.21.15.133:6824/35901 23 ==== osd_op_reply(87 [pgnls-filter start_epoch 44 out=49b] v42'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fec92800 2023-10-11T09:42:49.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.756+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:88 6.19 6:98000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecc7400 con 0x55e0fec93800 2023-10-11T09:42:49.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.760+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.5 v2:172.21.15.133:6816/35899 10 ==== osd_op_reply(88 [pgnls-filter start_epoch 44 out=81b] v44'14 uv14 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0fec93800 2023-10-11T09:42:49.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.760+0000 7f9bd08e5840 20 datascan.forall_objects: OSD matched oid 100000001f8.00000000 2023-10-11T09:42:49.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.760+0000 7f9bd08e5840 10 datascan.operator(): handling object 100000001f8.0 2023-10-11T09:42:49.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.760+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:89 6.19 6:997b7129:::100000001f8.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec73c00 con 0x55e0fec93800 2023-10-11T09:42:49.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.760+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.5 v2:172.21.15.133:6816/35899 11 ==== osd_op_reply(89 100000001f8.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv14 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0fec93800 2023-10-11T09:42:49.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.760+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:90 7.19 7:997b7129:::100000001f8.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec72c00 con 0x55e0feb1d800 2023-10-11T09:42:49.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.760+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.6 v2:172.21.15.133:6800/35900 12 ==== osd_op_reply(90 100000001f8.00000000 [getxattr (30) out=30b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55e0ff593200 con 0x55e0feb1d800 2023-10-11T09:42:49.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.760+0000 7f9bd08e5840 20 datascan.operator(): loaded layout from xattr: pi: 7 os: 4194304 sc: 1 su: 4194304 2023-10-11T09:42:49.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.760+0000 7f9bd08e5840 10 datascan.inject_with_backtrace: inode: 0x0x100000001f8 2023-10-11T09:42:49.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.760+0000 7f9bd08e5840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/3 2023-10-11T09:42:49.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.760+0000 7f9bd08e5840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=3 2023-10-11T09:42:49.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.760+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:91 5.10 5:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec92c00 con 0x55e0fec92800 2023-10-11T09:42:49.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.764+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.7 v2:172.21.15.133:6824/35901 24 ==== osd_op_reply(91 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fec92800 2023-10-11T09:42:49.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.764+0000 7f9bd08e5840 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-10-11T09:42:49.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.764+0000 7f9bd08e5840 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-10-11T09:42:49.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.764+0000 7f9bd08e5840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/3, will insert to root 2023-10-11T09:42:49.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.764+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:92 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec93c00 con 0x55e0fec92800 2023-10-11T09:42:49.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.764+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.7 v2:172.21.15.133:6824/35901 25 ==== osd_op_reply(92 10000000000.00000000 [omap-get-header out=274b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fec92800 2023-10-11T09:42:49.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.764+0000 7f9bd08e5840 10 datascan.find_or_create_dirfrag: read_version = 19 2023-10-11T09:42:49.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.764+0000 7f9bd08e5840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-10-11T09:42:49.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.764+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:93 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecc6800 con 0x55e0fec92800 2023-10-11T09:42:49.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.764+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.7 v2:172.21.15.133:6824/35901 26 ==== osd_op_reply(93 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fec92800 2023-10-11T09:42:49.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.764+0000 7f9bd08e5840 20 datascan.read_dentry: oid=10000000000.00000000 dname=3 frag=*, r=0 2023-10-11T09:42:49.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.764+0000 7f9bd08e5840 20 datascan.read_dentry: 3_head not found in result 2023-10-11T09:42:49.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.764+0000 7f9bd08e5840 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f8 at 0x0x10000000000/3 with size=6291456 bytes 2023-10-11T09:42:49.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.764+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:94 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecc6c00 con 0x55e0fec92800 2023-10-11T09:42:49.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.768+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.7 v2:172.21.15.133:6824/35901 27 ==== osd_op_reply(94 10000000000.00000000 [omap-set-vals] v44'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fec92800 2023-10-11T09:42:49.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.768+0000 7f9bd08e5840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/3 pointing to 0x0x100000001f8 2023-10-11T09:42:49.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.768+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:95 6.39 6:9c000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecc6400 con 0x55e0fec93800 2023-10-11T09:42:49.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.768+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.5 v2:172.21.15.133:6816/35899 12 ==== osd_op_reply(95 [pgnls-filter start_epoch 44 out=49b] v42'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0fec93800 2023-10-11T09:42:49.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.768+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:96 6.5 6:a0000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecc7800 con 0x55e0fec92800 2023-10-11T09:42:49.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.772+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.7 v2:172.21.15.133:6824/35901 28 ==== osd_op_reply(96 [pgnls-filter start_epoch 44 out=49b] v42'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fec92800 2023-10-11T09:42:49.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.772+0000 7f9bca898700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:97 6.25 6:a4000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecac800 con 0x55e0feb1d800 2023-10-11T09:42:49.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.772+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.6 v2:172.21.15.133:6800/35900 13 ==== osd_op_reply(97 [pgnls-filter start_epoch 44 out=49b] v42'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593200 con 0x55e0feb1d800 2023-10-11T09:42:49.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.772+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:98 6.15 6:a8000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecc7c00 con 0x55e0fec92800 2023-10-11T09:42:49.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.772+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.7 v2:172.21.15.133:6824/35901 29 ==== osd_op_reply(98 [pgnls-filter start_epoch 44 out=49b] v42'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fec92800 2023-10-11T09:42:49.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.772+0000 7f9bca898700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:99 6.35 6:ac000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecacc00 con 0x55e0fec93800 2023-10-11T09:42:49.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.772+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.5 v2:172.21.15.133:6816/35899 13 ==== osd_op_reply(99 [pgnls-filter start_epoch 44 out=49b] v42'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0fec93800 2023-10-11T09:42:49.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.772+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:100 6.d 6:b0000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecc6000 con 0x55e0fec93800 2023-10-11T09:42:49.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.776+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.5 v2:172.21.15.133:6816/35899 14 ==== osd_op_reply(100 [pgnls-filter start_epoch 44 out=49b] v42'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0fec93800 2023-10-11T09:42:49.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.776+0000 7f9bca898700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:101 6.2d 6:b4000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecac000 con 0x55e0fec93400 2023-10-11T09:42:49.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.776+0000 7f9bcf0a1700 1 -- 172.21.15.78:0/82329120 <== osd.2 v2:172.21.15.78:6808/35732 7 ==== osd_op_reply(101 [pgnls-filter start_epoch 44 out=49b] v42'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593440 con 0x55e0fec93400 2023-10-11T09:42:49.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.776+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:102 6.1d 6:b8000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecc6800 con 0x55e0fec92000 2023-10-11T09:42:49.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.776+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.1 v2:172.21.15.78:6809/35734 8 ==== osd_op_reply(102 [pgnls-filter start_epoch 44 out=49b] v42'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0fec92000 2023-10-11T09:42:49.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.776+0000 7f9bca898700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:103 6.3d 6:bc000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec73800 con 0x55e0fec72800 2023-10-11T09:42:49.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.776+0000 7f9bcf0a1700 1 -- 172.21.15.78:0/82329120 <== osd.0 v2:172.21.15.78:6800/35735 11 ==== osd_op_reply(103 [pgnls-filter start_epoch 44 out=49b] v42'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593440 con 0x55e0fec72800 2023-10-11T09:42:49.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.776+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:104 6.3 6:c0000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecc6c00 con 0x55e0fec92800 2023-10-11T09:42:49.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.780+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.7 v2:172.21.15.133:6824/35901 30 ==== osd_op_reply(104 [pgnls-filter start_epoch 44 out=49b] v42'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fec92800 2023-10-11T09:42:49.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.780+0000 7f9bca898700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:105 6.23 6:c4000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec93000 con 0x55e0feb1d800 2023-10-11T09:42:49.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.780+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.6 v2:172.21.15.133:6800/35900 14 ==== osd_op_reply(105 [pgnls-filter start_epoch 44 out=49b] v42'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593200 con 0x55e0feb1d800 2023-10-11T09:42:49.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.780+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:106 6.13 6:c8000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecc7800 con 0x55e0feb1d000 2023-10-11T09:42:49.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.780+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.3 v2:172.21.15.78:6824/35738 10 ==== osd_op_reply(106 [pgnls-filter start_epoch 44 out=49b] v42'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0feb1d000 2023-10-11T09:42:49.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.780+0000 7f9bca898700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:107 6.33 6:cc000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecc7000 con 0x55e0fec93800 2023-10-11T09:42:49.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.784+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.5 v2:172.21.15.133:6816/35899 15 ==== osd_op_reply(107 [pgnls-filter start_epoch 44 out=49b] v42'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0fec93800 2023-10-11T09:42:49.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.784+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:108 6.b 6:d0000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecad000 con 0x55e0feb1d000 2023-10-11T09:42:49.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.784+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.3 v2:172.21.15.78:6824/35738 11 ==== osd_op_reply(108 [pgnls-filter start_epoch 44 out=49b] v42'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0feb1d000 2023-10-11T09:42:49.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.784+0000 7f9bca898700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:109 6.2b 6:d4000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecc7c00 con 0x55e0fec93800 2023-10-11T09:42:49.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.784+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.5 v2:172.21.15.133:6816/35899 16 ==== osd_op_reply(109 [pgnls-filter start_epoch 44 out=49b] v42'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0fec93800 2023-10-11T09:42:49.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.784+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:110 6.1b 6:d8000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecac800 con 0x55e0feb1d000 2023-10-11T09:42:49.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.784+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.3 v2:172.21.15.78:6824/35738 12 ==== osd_op_reply(110 [pgnls-filter start_epoch 44 out=49b] v42'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0feb1d000 2023-10-11T09:42:49.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.784+0000 7f9bca898700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:111 6.3b 6:dc000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec73000 con 0x55e0fec93400 2023-10-11T09:42:49.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.784+0000 7f9bcf0a1700 1 -- 172.21.15.78:0/82329120 <== osd.2 v2:172.21.15.78:6808/35732 8 ==== osd_op_reply(111 [pgnls-filter start_epoch 44 out=49b] v42'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593440 con 0x55e0fec93400 2023-10-11T09:42:49.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.784+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:112 6.7 6:e0000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec92c00 con 0x55e0fec93800 2023-10-11T09:42:49.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.788+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.5 v2:172.21.15.133:6816/35899 17 ==== osd_op_reply(112 [pgnls-filter start_epoch 44 out=81b] v44'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0fec93800 2023-10-11T09:42:49.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.788+0000 7f9bd08e5840 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2023-10-11T09:42:49.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.788+0000 7f9bd08e5840 10 datascan.operator(): handling object 10000000001.0 2023-10-11T09:42:49.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.788+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:113 6.7 6:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecadc00 con 0x55e0fec93800 2023-10-11T09:42:49.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.788+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.5 v2:172.21.15.133:6816/35899 18 ==== osd_op_reply(113 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv9 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0fec93800 2023-10-11T09:42:49.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.788+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:114 7.7 7:e2e07930:::10000000001.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecad400 con 0x55e0feb1d800 2023-10-11T09:42:49.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.788+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.6 v2:172.21.15.133:6800/35900 15 ==== osd_op_reply(114 10000000001.00000000 [getxattr (30) out=30b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55e0ff593200 con 0x55e0feb1d800 2023-10-11T09:42:49.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.788+0000 7f9bd08e5840 20 datascan.operator(): loaded layout from xattr: pi: 7 os: 4194304 sc: 1 su: 4194304 2023-10-11T09:42:49.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.788+0000 7f9bd08e5840 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2023-10-11T09:42:49.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.788+0000 7f9bd08e5840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/0 2023-10-11T09:42:49.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.788+0000 7f9bd08e5840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=0 2023-10-11T09:42:49.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.788+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:115 5.10 5:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec73400 con 0x55e0fec92800 2023-10-11T09:42:49.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.792+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.7 v2:172.21.15.133:6824/35901 31 ==== osd_op_reply(115 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fec92800 2023-10-11T09:42:49.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.792+0000 7f9bd08e5840 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-10-11T09:42:49.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.792+0000 7f9bd08e5840 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-10-11T09:42:49.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.792+0000 7f9bd08e5840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/0, will insert to root 2023-10-11T09:42:49.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.792+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:116 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec72400 con 0x55e0fec92800 2023-10-11T09:42:49.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.792+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.7 v2:172.21.15.133:6824/35901 32 ==== osd_op_reply(116 10000000000.00000000 [omap-get-header out=274b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fec92800 2023-10-11T09:42:49.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.792+0000 7f9bd08e5840 10 datascan.find_or_create_dirfrag: read_version = 20 2023-10-11T09:42:49.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.792+0000 7f9bd08e5840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-10-11T09:42:49.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.792+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:117 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec92400 con 0x55e0fec92800 2023-10-11T09:42:49.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.792+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.7 v2:172.21.15.133:6824/35901 33 ==== osd_op_reply(117 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fec92800 2023-10-11T09:42:49.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.792+0000 7f9bd08e5840 20 datascan.read_dentry: oid=10000000000.00000000 dname=0 frag=*, r=0 2023-10-11T09:42:49.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.792+0000 7f9bd08e5840 20 datascan.read_dentry: 0_head not found in result 2023-10-11T09:42:49.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.792+0000 7f9bd08e5840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/0 with size=6291456 bytes 2023-10-11T09:42:49.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.792+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:118 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecc7400 con 0x55e0fec92800 2023-10-11T09:42:49.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.796+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.7 v2:172.21.15.133:6824/35901 34 ==== osd_op_reply(118 10000000000.00000000 [omap-set-vals] v44'21 uv21 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fec92800 2023-10-11T09:42:49.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.796+0000 7f9bd08e5840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/0 pointing to 0x0x10000000001 2023-10-11T09:42:49.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.796+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:119 6.27 6:e4000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecc6400 con 0x55e0feb1d800 2023-10-11T09:42:49.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.796+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.6 v2:172.21.15.133:6800/35900 16 ==== osd_op_reply(119 [pgnls-filter start_epoch 44 out=49b] v42'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593200 con 0x55e0feb1d800 2023-10-11T09:42:49.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.796+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:120 6.17 6:e8000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec93c00 con 0x55e0fecad800 2023-10-11T09:42:49.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.796+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.4 v2:172.21.15.133:6808/35898 13 ==== osd_op_reply(120 [pgnls-filter start_epoch 44 out=49b] v42'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fecad800 2023-10-11T09:42:49.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.796+0000 7f9bca898700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:121 6.37 6:ec000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec73800 con 0x55e0fec72800 2023-10-11T09:42:49.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.800+0000 7f9bcf0a1700 1 -- 172.21.15.78:0/82329120 <== osd.0 v2:172.21.15.78:6800/35735 12 ==== osd_op_reply(121 [pgnls-filter start_epoch 44 out=49b] v42'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593440 con 0x55e0fec72800 2023-10-11T09:42:49.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.800+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:122 6.f 6:f0000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fec72c00 con 0x55e0fec93400 2023-10-11T09:42:49.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.800+0000 7f9bcf0a1700 1 -- 172.21.15.78:0/82329120 <== osd.2 v2:172.21.15.78:6808/35732 9 ==== osd_op_reply(122 [pgnls-filter start_epoch 44 out=49b] v42'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593440 con 0x55e0fec93400 2023-10-11T09:42:49.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.800+0000 7f9bca898700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:123 6.2f 6:f4000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecac400 con 0x55e0fec92800 2023-10-11T09:42:49.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.800+0000 7f9bcf8a2700 1 -- 172.21.15.78:0/82329120 <== osd.7 v2:172.21.15.133:6824/35901 35 ==== osd_op_reply(123 [pgnls-filter start_epoch 44 out=49b] v42'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff593200 con 0x55e0fec92800 2023-10-11T09:42:49.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.800+0000 7f9bca097700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:124 6.1f 6:f8000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecda000 con 0x55e0feb1d000 2023-10-11T09:42:49.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.800+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.3 v2:172.21.15.78:6824/35738 13 ==== osd_op_reply(124 [pgnls-filter start_epoch 44 out=49b] v42'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0feb1d000 2023-10-11T09:42:49.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.800+0000 7f9bca898700 1 -- 172.21.15.78:0/82329120 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:125 6.3f 6:fc000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55e0fecac000 con 0x55e0feb1d000 2023-10-11T09:42:49.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.804+0000 7f9bce8a0700 1 -- 172.21.15.78:0/82329120 <== osd.3 v2:172.21.15.78:6824/35738 14 ==== osd_op_reply(125 [pgnls-filter start_epoch 44 out=49b] v42'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e0ff592fc0 con 0x55e0feb1d000 2023-10-11T09:42:49.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.804+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55e0fec72800 msgr2=0x55e0fe8c1600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.804+0000 7f9bd08e5840 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55e0fec72800 0x55e0fe8c1600 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.804+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55e0fec92000 msgr2=0x55e0fec98000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.804+0000 7f9bd08e5840 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55e0fec92000 0x55e0fec98000 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.804+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55e0fec93400 msgr2=0x55e0fec98b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.804+0000 7f9bd08e5840 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55e0fec93400 0x55e0fec98b00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.804+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55e0feb1d000 msgr2=0x55e0fe8c2100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.804+0000 7f9bd08e5840 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55e0feb1d000 0x55e0fe8c2100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.804+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55e0fecad800 msgr2=0x55e0fec9a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.804+0000 7f9bd08e5840 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55e0fecad800 0x55e0fec9a100 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.804+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55e0fec93800 msgr2=0x55e0fec99080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.804+0000 7f9bd08e5840 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55e0fec93800 0x55e0fec99080 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.804+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55e0feb1d800 msgr2=0x55e0fe8c1b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.804+0000 7f9bd08e5840 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55e0feb1d800 0x55e0fe8c1b80 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.804+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55e0fec92800 msgr2=0x55e0fec98580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.804+0000 7f9bd08e5840 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55e0fec92800 0x55e0fec98580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.804+0000 7f9bcf0a1700 1 -- 172.21.15.78:0/82329120 reap_dead start 2023-10-11T09:42:49.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.804+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55e0fec72000 msgr2=0x55e0fe8c3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.804+0000 7f9bd08e5840 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55e0fec72000 0x55e0fe8c3700 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55e0fe871c20 tx=0x55e0ff69ad20 comp rx=0 tx=0).stop 2023-10-11T09:42:49.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.804+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0feb1c800 msgr2=0x55e0fe8c2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.804+0000 7f9bd08e5840 1 --2- 172.21.15.78:0/82329120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0feb1c800 0x55e0fe8c2c00 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x55e0feb0f410 tx=0x55e0ff69b260 comp rx=0 tx=0).stop 2023-10-11T09:42:49.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.804+0000 7f9bcf0a1700 1 -- 172.21.15.78:0/82329120 reap_dead start 2023-10-11T09:42:49.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.804+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 shutdown_connections 2023-10-11T09:42:49.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.804+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 >> 172.21.15.78:0/82329120 conn(0x55e0fe81ac00 msgr2=0x55e0fe8e9800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:49.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.804+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 shutdown_connections 2023-10-11T09:42:49.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.804+0000 7f9bd08e5840 1 -- 172.21.15.78:0/82329120 wait complete. 2023-10-11T09:42:49.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.804+0000 7f9bd08e5840 1 -- 172.21.15.78:0/2301909666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0fe81b400 msgr2=0x55e0fe8c0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.804+0000 7f9bd08e5840 1 --2- 172.21.15.78:0/2301909666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0fe81b400 0x55e0fe8c0b00 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x55e0fe977380 tx=0x55e0ff595140 comp rx=0 tx=0).stop 2023-10-11T09:42:49.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.804+0000 7f9bd08e5840 1 -- 172.21.15.78:0/2301909666 shutdown_connections 2023-10-11T09:42:49.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.804+0000 7f9bd08e5840 1 --2- 172.21.15.78:0/2301909666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0fe81b000 0x55e0fe8c1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.804+0000 7f9bd08e5840 1 --2- 172.21.15.78:0/2301909666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0fe81b400 0x55e0fe8c0b00 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.804+0000 7f9bd08e5840 1 --2- 172.21.15.78:0/2301909666 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e0fe81b800 0x55e0fe8c0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.804+0000 7f9bd08e5840 1 -- 172.21.15.78:0/2301909666 >> 172.21.15.78:0/2301909666 conn(0x55e0fe81bc00 msgr2=0x55e0fe8e9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:49.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.804+0000 7f9bd08e5840 1 -- 172.21.15.78:0/2301909666 shutdown_connections 2023-10-11T09:42:49.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.804+0000 7f9bd08e5840 1 -- 172.21.15.78:0/2301909666 wait complete. 2023-10-11T09:42:49.817 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes'] in time 0:00:00.240761, result: 2023-10-11T09:42:49.818 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2023-10-11T09:42:49.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.876+0000 7efe3f050840 1 -- start start 2023-10-11T09:42:49.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.876+0000 7efe3f050840 1 Processor -- start 2023-10-11T09:42:49.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.876+0000 7efe3f050840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5565393eb800 0x556539490580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.876+0000 7efe3f050840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565393eb400 0x556539490b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.876+0000 7efe3f050840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565393eb000 0x556539491080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.876+0000 7efe3f050840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x556539421e00 con 0x5565393eb000 2023-10-11T09:42:49.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.876+0000 7efe3f050840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55653946aa00 con 0x5565393eb400 2023-10-11T09:42:49.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.876+0000 7efe3f050840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55653946a640 con 0x5565393eb800 2023-10-11T09:42:49.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.880+0000 7efe3e00d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565393eb000 0x556539491080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.880+0000 7efe3e00d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565393eb000 0x556539491080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58098/0 (socket says 172.21.15.78:58098) 2023-10-11T09:42:49.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.880+0000 7efe3e00d700 1 -- 172.21.15.78:0/3145686363 learned_addr learned my addr 172.21.15.78:0/3145686363 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:49.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.880+0000 7efe3d80c700 1 --2- 172.21.15.78:0/3145686363 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5565393eb800 0x556539490580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.880+0000 7efe3d00b700 1 --2- 172.21.15.78:0/3145686363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565393eb400 0x556539490b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.880+0000 7efe3e00d700 1 -- 172.21.15.78:0/3145686363 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5565393eb800 msgr2=0x556539490580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:49.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.880+0000 7efe3e00d700 1 --2- 172.21.15.78:0/3145686363 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5565393eb800 0x556539490580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.880+0000 7efe3e00d700 1 -- 172.21.15.78:0/3145686363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565393eb400 msgr2=0x556539490b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:49.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.880+0000 7efe3e00d700 1 --2- 172.21.15.78:0/3145686363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565393eb400 0x556539490b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.880+0000 7efe3e00d700 1 -- 172.21.15.78:0/3145686363 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55653940cb60 con 0x5565393eb000 2023-10-11T09:42:49.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.880+0000 7efe3e00d700 1 --2- 172.21.15.78:0/3145686363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565393eb000 0x556539491080 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x55653a1532f0 tx=0x55653a256c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb03109b2076195 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.880+0000 7efe3b007700 1 -- 172.21.15.78:0/3145686363 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55653941b080 con 0x5565393eb000 2023-10-11T09:42:49.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.880+0000 7efe3f050840 1 -- 172.21.15.78:0/3145686363 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55653940d1e0 con 0x5565393eb000 2023-10-11T09:42:49.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.880+0000 7efe3f050840 1 -- 172.21.15.78:0/3145686363 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55653940cb60 con 0x5565393eb000 2023-10-11T09:42:49.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.880+0000 7efe3f050840 4 waiting for MDS map... 2023-10-11T09:42:49.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.880+0000 7efe3b007700 1 -- 172.21.15.78:0/3145686363 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5565394e0180 con 0x5565393eb000 2023-10-11T09:42:49.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.880+0000 7efe3b007700 1 -- 172.21.15.78:0/3145686363 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55653941b600 con 0x5565393eb000 2023-10-11T09:42:49.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.880+0000 7efe3b007700 1 -- 172.21.15.78:0/3145686363 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(44..44 src has 1..44) v4 ==== 6667+0+0 (secure 0 0 0) 0x556539424c40 con 0x5565393eb000 2023-10-11T09:42:49.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.880+0000 7efe3b007700 1 -- 172.21.15.78:0/3145686363 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) v1 ==== 3118+0+0 (secure 0 0 0) 0x55653942cf00 con 0x5565393eb000 2023-10-11T09:42:49.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.880+0000 7efe3f050840 4 Got MDS map 22 2023-10-11T09:42:49.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.880+0000 7efe3f050840 4 datascan.main: Using metadata pool output 2023-10-11T09:42:49.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.880+0000 7efe3f050840 4 datascan.main: connecting to RADOS... 2023-10-11T09:42:49.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.880+0000 7efe3f050840 1 Processor -- start 2023-10-11T09:42:49.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.884+0000 7efe3f050840 1 -- start start 2023-10-11T09:42:49.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.884+0000 7efe3f050840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55653a274400 0x556539491600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.884+0000 7efe3f050840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55653a274800 0x556539491b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.884+0000 7efe3f050840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55653a274c00 0x556539492100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.884+0000 7efe3f050840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55653946a500 con 0x55653a274800 2023-10-11T09:42:49.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.884+0000 7efe3f050840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55653946a3c0 con 0x55653a274c00 2023-10-11T09:42:49.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.884+0000 7efe3f050840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55653946a640 con 0x55653a274400 2023-10-11T09:42:49.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.884+0000 7efe3d80c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55653a274800 0x556539491b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.884+0000 7efe3d80c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55653a274800 0x556539491b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58104/0 (socket says 172.21.15.78:58104) 2023-10-11T09:42:49.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.884+0000 7efe3d80c700 1 -- 172.21.15.78:0/1229050601 learned_addr learned my addr 172.21.15.78:0/1229050601 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:49.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.884+0000 7efe3d00b700 1 --2- 172.21.15.78:0/1229050601 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55653a274400 0x556539491600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.884+0000 7efe3d00b700 1 --2- 172.21.15.78:0/1229050601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55653a274c00 0x556539492100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.884+0000 7efe3d80c700 1 -- 172.21.15.78:0/1229050601 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55653a274400 msgr2=0x556539491600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:49.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.884+0000 7efe3d80c700 1 --2- 172.21.15.78:0/1229050601 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55653a274400 0x556539491600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.884+0000 7efe3d80c700 1 -- 172.21.15.78:0/1229050601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55653a274c00 msgr2=0x556539492100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:49.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.884+0000 7efe3d80c700 1 --2- 172.21.15.78:0/1229050601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55653a274c00 0x556539492100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.884+0000 7efe3d80c700 1 -- 172.21.15.78:0/1229050601 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55653940c340 con 0x55653a274800 2023-10-11T09:42:49.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.884+0000 7efe3d00b700 1 --2- 172.21.15.78:0/1229050601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55653a274c00 0x556539492100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:42:49.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.884+0000 7efe3d80c700 1 --2- 172.21.15.78:0/1229050601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55653a274800 0x556539491b80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x5565396d8720 tx=0x5565396e4d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc14a3896daa76f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.884+0000 7efe39003700 1 -- 172.21.15.78:0/1229050601 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55653941b080 con 0x55653a274800 2023-10-11T09:42:49.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.884+0000 7efe39003700 1 -- 172.21.15.78:0/1229050601 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556539436900 con 0x55653a274800 2023-10-11T09:42:49.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.884+0000 7efe39003700 1 -- 172.21.15.78:0/1229050601 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55653941b600 con 0x55653a274800 2023-10-11T09:42:49.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.888+0000 7efe3f050840 1 -- 172.21.15.78:0/1229050601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55653a274800 msgr2=0x556539491b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.888+0000 7efe3f050840 1 --2- 172.21.15.78:0/1229050601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55653a274800 0x556539491b80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x5565396d8720 tx=0x5565396e4d20 comp rx=0 tx=0).stop 2023-10-11T09:42:49.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.888+0000 7efe3f050840 1 -- 172.21.15.78:0/1229050601 shutdown_connections 2023-10-11T09:42:49.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.888+0000 7efe3f050840 1 --2- 172.21.15.78:0/1229050601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55653a274c00 0x556539492100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.888+0000 7efe3f050840 1 --2- 172.21.15.78:0/1229050601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55653a274800 0x556539491b80 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.888+0000 7efe3f050840 1 --2- 172.21.15.78:0/1229050601 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55653a274400 0x556539491600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.888+0000 7efe3f050840 1 -- 172.21.15.78:0/1229050601 >> 172.21.15.78:0/1229050601 conn(0x5565393eac00 msgr2=0x5565394b9800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:49.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.888+0000 7efe3f050840 1 -- 172.21.15.78:0/1229050601 shutdown_connections 2023-10-11T09:42:49.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.888+0000 7efe3f050840 1 -- 172.21.15.78:0/1229050601 wait complete. 2023-10-11T09:42:49.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.888+0000 7efe3f050840 1 Processor -- start 2023-10-11T09:42:49.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.888+0000 7efe3f050840 1 -- start start 2023-10-11T09:42:49.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.888+0000 7efe3f050840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55653a274400 0x556539493180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.888+0000 7efe3f050840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55653a274800 0x556539492c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.888+0000 7efe3f050840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55653a274c00 0x556539492680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.888+0000 7efe3f050840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55653946a280 con 0x55653a274400 2023-10-11T09:42:49.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.888+0000 7efe3f050840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55653946a140 con 0x55653a274c00 2023-10-11T09:42:49.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.888+0000 7efe3f050840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55653946a000 con 0x55653a274800 2023-10-11T09:42:49.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.892+0000 7efe3d00b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55653a274400 0x556539493180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.892+0000 7efe3d00b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55653a274400 0x556539493180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58114/0 (socket says 172.21.15.78:58114) 2023-10-11T09:42:49.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.892+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 learned_addr learned my addr 172.21.15.78:0/1780204447 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:49.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.892+0000 7efe3d80c700 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55653a274800 0x556539492c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.892+0000 7efe3d00b700 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55653a274c00 0x556539492680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.892+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55653a274800 msgr2=0x556539492c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.892+0000 7efe3d00b700 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55653a274800 0x556539492c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.892+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55653a274c00 msgr2=0x556539492680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:49.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.892+0000 7efe3d00b700 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55653a274c00 0x556539492680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:49.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.892+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55653940c000 con 0x55653a274400 2023-10-11T09:42:49.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.892+0000 7efe3d00b700 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55653a274400 0x556539493180 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x5565396d93b0 tx=0x5565396e4c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.892+0000 7efe37800700 1 -- 172.21.15.78:0/1780204447 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55653941b080 con 0x55653a274400 2023-10-11T09:42:49.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.892+0000 7efe37800700 1 -- 172.21.15.78:0/1780204447 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556539436480 con 0x55653a274400 2023-10-11T09:42:49.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.892+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55653940cea0 con 0x55653a274400 2023-10-11T09:42:49.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.892+0000 7efe37800700 1 -- 172.21.15.78:0/1780204447 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55653941b600 con 0x55653a274400 2023-10-11T09:42:49.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.892+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55653940d520 con 0x55653a274400 2023-10-11T09:42:49.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.892+0000 7efe3f050840 4 datascan.init: resolving metadata pool 5 2023-10-11T09:42:49.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.896+0000 7efe37800700 1 -- 172.21.15.78:0/1780204447 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x556539448840 con 0x55653a274400 2023-10-11T09:42:49.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.896+0000 7efe37800700 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55653983e000 0x556539493700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.896+0000 7efe37800700 1 -- 172.21.15.78:0/1780204447 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6667+0+0 (secure 0 0 0) 0x556539424a80 con 0x55653a274400 2023-10-11T09:42:49.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.896+0000 7efe3f050840 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-10-11T09:42:49.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.896+0000 7efe3f050840 4 datascan.main: resolving metadata pool 5 2023-10-11T09:42:49.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.896+0000 7efe3f050840 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55653a275000 0x556539492100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.896+0000 7efe3d80c700 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55653983e000 0x556539493700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.896+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:1 5.0 5:00000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55653a275400 con 0x55653a275000 2023-10-11T09:42:49.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.896+0000 7efe3d00b700 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55653a275000 0x556539492100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.896+0000 7efe3d80c700 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55653983e000 0x556539493700 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x5565396d8720 tx=0x55653a257080 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.896+0000 7efe3d00b700 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55653a275000 0x556539492100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.900+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(1 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c6fc0 con 0x55653a275000 2023-10-11T09:42:49.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.900+0000 7efe3d00b700 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55653a275400 0x55653984e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.900+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:2 5.10 5:08000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55653983e400 con 0x55653a275400 2023-10-11T09:42:49.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.900+0000 7efe3e00d700 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55653a275400 0x55653984e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.900+0000 7efe3e00d700 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55653a275400 0x55653984e000 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.900+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 <== osd.7 v2:172.21.15.133:6824/35901 1 ==== osd_op_reply(2 [pgnls start_epoch 44 out=81b] v44'21 uv21 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55653a1c7200 con 0x55653a275400 2023-10-11T09:42:49.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.900+0000 7efe3e00d700 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55653983e400 0x55653984e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.900+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:3 5.8 5:10000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55653983ec00 con 0x55653983e400 2023-10-11T09:42:49.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.900+0000 7efe3d80c700 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55653983e400 0x55653984e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.904+0000 7efe3d80c700 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55653983e400 0x55653984e580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.904+0000 7efe3d80c700 1 -- 172.21.15.78:0/1780204447 <== osd.2 v2:172.21.15.78:6808/35732 1 ==== osd_op_reply(3 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c7440 con 0x55653983e400 2023-10-11T09:42:49.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.904+0000 7efe3d80c700 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55653983ec00 0x55653984eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.904+0000 7efe3d80c700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:4 5.18 5:18000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55653983f400 con 0x55653983ec00 2023-10-11T09:42:49.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.904+0000 7efe3d00b700 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55653983ec00 0x55653984eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.904+0000 7efe3d00b700 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55653983ec00 0x55653984eb00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.904+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 <== osd.4 v2:172.21.15.133:6808/35898 1 ==== osd_op_reply(4 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c6fc0 con 0x55653983ec00 2023-10-11T09:42:49.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.904+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:5 5.4 5:20000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55653983f400 con 0x55653a275400 2023-10-11T09:42:49.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.908+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 <== osd.7 v2:172.21.15.133:6824/35901 2 ==== osd_op_reply(5 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c7200 con 0x55653a275400 2023-10-11T09:42:49.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.908+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:6 5.14 5:28000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55653983f400 con 0x55653a275000 2023-10-11T09:42:49.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.908+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 <== osd.3 v2:172.21.15.78:6824/35738 2 ==== osd_op_reply(6 [pgnls start_epoch 44 out=125b] v44'41 uv41 ondisk = 1) v8 ==== 144+0+125 (crc 0 0 0) 0x55653a1c6fc0 con 0x55653a275000 2023-10-11T09:42:49.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.908+0000 7efe3d00b700 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55653983f400 0x55653984f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.908+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:7 5.c 5:30000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55653983e800 con 0x55653983f400 2023-10-11T09:42:49.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.908+0000 7efe3e00d700 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55653983f400 0x55653984f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.908+0000 7efe3e00d700 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55653983f400 0x55653984f080 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.908+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 <== osd.1 v2:172.21.15.78:6809/35734 1 ==== osd_op_reply(7 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c7200 con 0x55653983f400 2023-10-11T09:42:49.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.908+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:8 5.1c 5:38000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55653983e800 con 0x55653983ec00 2023-10-11T09:42:49.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.912+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 <== osd.4 v2:172.21.15.133:6808/35898 2 ==== osd_op_reply(8 [pgnls start_epoch 44 out=49b] v43'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c6fc0 con 0x55653983ec00 2023-10-11T09:42:49.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.912+0000 7efe3d00b700 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55653983e800 0x55653984f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.912+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:9 5.2 5:40000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55653983f800 con 0x55653983e800 2023-10-11T09:42:49.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.912+0000 7efe3d80c700 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55653983e800 0x55653984f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.912+0000 7efe3d80c700 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55653983e800 0x55653984f600 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.912+0000 7efe3d80c700 1 -- 172.21.15.78:0/1780204447 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== osd_op_reply(9 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c7440 con 0x55653983e800 2023-10-11T09:42:49.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.912+0000 7efe3d80c700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:10 5.12 5:48000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55653983f800 con 0x55653983f400 2023-10-11T09:42:49.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.912+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 <== osd.1 v2:172.21.15.78:6809/35734 2 ==== osd_op_reply(10 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c7200 con 0x55653983f400 2023-10-11T09:42:49.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.912+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:11 5.a 5:50000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55653983f800 con 0x55653983e400 2023-10-11T09:42:49.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.916+0000 7efe3d80c700 1 -- 172.21.15.78:0/1780204447 <== osd.2 v2:172.21.15.78:6808/35732 2 ==== osd_op_reply(11 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c7440 con 0x55653983e400 2023-10-11T09:42:49.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.916+0000 7efe3d80c700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:12 5.1a 5:58000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55653983f800 con 0x55653a275400 2023-10-11T09:42:49.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.916+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 <== osd.7 v2:172.21.15.133:6824/35901 3 ==== osd_op_reply(12 [pgnls start_epoch 44 out=49b] v44'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c7200 con 0x55653a275400 2023-10-11T09:42:49.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.916+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:13 5.6 5:60000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55653983f800 con 0x55653983e400 2023-10-11T09:42:49.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.916+0000 7efe3d80c700 1 -- 172.21.15.78:0/1780204447 <== osd.2 v2:172.21.15.78:6808/35732 3 ==== osd_op_reply(13 [pgnls start_epoch 44 out=49b] v43'9 uv9 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c7440 con 0x55653983e400 2023-10-11T09:42:49.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.916+0000 7efe3d80c700 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55653983f800 0x55653984fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.916+0000 7efe3d80c700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:14 5.16 5:68000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539864000 con 0x55653983f800 2023-10-11T09:42:49.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.916+0000 7efe3d00b700 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55653983f800 0x55653984fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.920+0000 7efe3d00b700 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55653983f800 0x55653984fb80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.920+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 <== osd.5 v2:172.21.15.133:6816/35899 1 ==== osd_op_reply(14 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c6fc0 con 0x55653983f800 2023-10-11T09:42:49.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.920+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:15 5.e 5:70000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539864000 con 0x55653983ec00 2023-10-11T09:42:49.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.920+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 <== osd.4 v2:172.21.15.133:6808/35898 3 ==== osd_op_reply(15 [pgnls start_epoch 44 out=49b] v43'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c6fc0 con 0x55653983ec00 2023-10-11T09:42:49.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.920+0000 7efe3d00b700 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x556539864000 0x556539850680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:49.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.920+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:16 5.1e 5:78000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55653983fc00 con 0x556539864000 2023-10-11T09:42:49.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.920+0000 7efe3e00d700 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x556539864000 0x556539850680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:49.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.920+0000 7efe3e00d700 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x556539864000 0x556539850680 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:49.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.920+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 <== osd.0 v2:172.21.15.78:6800/35735 1 ==== osd_op_reply(16 [pgnls start_epoch 44 out=49b] v44'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c7200 con 0x556539864000 2023-10-11T09:42:49.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.924+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:17 5.1 5:80000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55653983fc00 con 0x55653983ec00 2023-10-11T09:42:49.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.924+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 <== osd.4 v2:172.21.15.133:6808/35898 4 ==== osd_op_reply(17 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c6fc0 con 0x55653983ec00 2023-10-11T09:42:49.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.924+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:18 5.11 5:88000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55653983fc00 con 0x55653983e800 2023-10-11T09:42:49.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.924+0000 7efe3d80c700 1 -- 172.21.15.78:0/1780204447 <== osd.6 v2:172.21.15.133:6800/35900 2 ==== osd_op_reply(18 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c7440 con 0x55653983e800 2023-10-11T09:42:49.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.924+0000 7efe3d80c700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:19 5.9 5:90000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55653983fc00 con 0x55653a275400 2023-10-11T09:42:49.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.924+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 <== osd.7 v2:172.21.15.133:6824/35901 4 ==== osd_op_reply(19 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c7200 con 0x55653a275400 2023-10-11T09:42:49.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.924+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:20 5.19 5:98000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55653983fc00 con 0x55653983f400 2023-10-11T09:42:49.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.924+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 <== osd.1 v2:172.21.15.78:6809/35734 3 ==== osd_op_reply(20 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c7200 con 0x55653983f400 2023-10-11T09:42:49.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.924+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:21 5.5 5:a0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55653983fc00 con 0x556539864000 2023-10-11T09:42:49.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.928+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 <== osd.0 v2:172.21.15.78:6800/35735 2 ==== osd_op_reply(21 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c7200 con 0x556539864000 2023-10-11T09:42:49.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.928+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:22 5.15 5:a8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55653983fc00 con 0x55653983f800 2023-10-11T09:42:49.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.928+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 <== osd.5 v2:172.21.15.133:6816/35899 2 ==== osd_op_reply(22 [pgnls start_epoch 44 out=73b] v43'6 uv6 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55653a1c6fc0 con 0x55653983f800 2023-10-11T09:42:49.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.928+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:23 5.d 5:b0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55653983fc00 con 0x55653983e400 2023-10-11T09:42:49.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.928+0000 7efe3d80c700 1 -- 172.21.15.78:0/1780204447 <== osd.2 v2:172.21.15.78:6808/35732 4 ==== osd_op_reply(23 [pgnls start_epoch 44 out=74b] v44'3 uv3 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55653a1c7440 con 0x55653983e400 2023-10-11T09:42:49.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.928+0000 7efe3d80c700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:24 5.1d 5:b8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55653983fc00 con 0x55653983f400 2023-10-11T09:42:49.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.928+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 <== osd.1 v2:172.21.15.78:6809/35734 4 ==== osd_op_reply(24 [pgnls start_epoch 44 out=49b] v43'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c7200 con 0x55653983f400 2023-10-11T09:42:49.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.928+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:25 5.3 5:c0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55653983fc00 con 0x556539864000 2023-10-11T09:42:49.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.928+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 <== osd.0 v2:172.21.15.78:6800/35735 3 ==== osd_op_reply(25 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c7200 con 0x556539864000 2023-10-11T09:42:49.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.928+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:26 5.13 5:c8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55653983fc00 con 0x55653a275000 2023-10-11T09:42:49.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.932+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 <== osd.3 v2:172.21.15.78:6824/35738 3 ==== osd_op_reply(26 [pgnls start_epoch 44 out=98b] v44'7 uv7 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55653a1c6fc0 con 0x55653a275000 2023-10-11T09:42:49.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.932+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:27 5.b 5:d0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55653983fc00 con 0x55653983e400 2023-10-11T09:42:49.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.932+0000 7efe3d80c700 1 -- 172.21.15.78:0/1780204447 <== osd.2 v2:172.21.15.78:6808/35732 5 ==== osd_op_reply(27 [pgnls start_epoch 44 out=76b] v44'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55653a1c7440 con 0x55653983e400 2023-10-11T09:42:49.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.932+0000 7efe3d80c700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:28 5.1b 5:d8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55653983fc00 con 0x55653983f800 2023-10-11T09:42:49.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.932+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 <== osd.5 v2:172.21.15.133:6816/35899 3 ==== osd_op_reply(28 [pgnls start_epoch 44 out=49b] v44'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c6fc0 con 0x55653983f800 2023-10-11T09:42:49.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.932+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:29 5.7 5:e0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55653983fc00 con 0x55653983f800 2023-10-11T09:42:49.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.932+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 <== osd.5 v2:172.21.15.133:6816/35899 4 ==== osd_op_reply(29 [pgnls start_epoch 44 out=49b] v43'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c6fc0 con 0x55653983f800 2023-10-11T09:42:49.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.932+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:30 5.17 5:e8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55653983fc00 con 0x55653a275000 2023-10-11T09:42:49.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.932+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 <== osd.3 v2:172.21.15.78:6824/35738 4 ==== osd_op_reply(30 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c6fc0 con 0x55653a275000 2023-10-11T09:42:49.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.932+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:31 5.f 5:f0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55653983fc00 con 0x55653983f800 2023-10-11T09:42:49.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.936+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 <== osd.5 v2:172.21.15.133:6816/35899 5 ==== osd_op_reply(31 [pgnls start_epoch 44 out=73b] v44'4 uv4 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55653a1c6fc0 con 0x55653983f800 2023-10-11T09:42:49.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.936+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:32 5.1f 5:f8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x55653983fc00 con 0x55653983e800 2023-10-11T09:42:49.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.936+0000 7efe3d80c700 1 -- 172.21.15.78:0/1780204447 <== osd.6 v2:172.21.15.133:6800/35900 3 ==== osd_op_reply(32 [pgnls start_epoch 44 out=101b] v44'12 uv12 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55653a1c7440 con 0x55653983e800 2023-10-11T09:42:49.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.936+0000 7efe3f050840 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2023-10-11T09:42:49.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.936+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:33 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55653a275800 con 0x55653a275400 2023-10-11T09:42:49.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.936+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 <== osd.7 v2:172.21.15.133:6824/35901 5 ==== osd_op_reply(33 10000000000.00000000 [omap-get-vals out=2495b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+2495 (crc 0 0 0) 0x55653a1c7200 con 0x55653a275400 2023-10-11T09:42:49.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.936+0000 7efe3f050840 20 datascan.scan_links: injected ino detected 2023-10-11T09:42:49.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.936+0000 7efe3f050840 20 datascan.scan_links: injected ino detected 2023-10-11T09:42:49.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.936+0000 7efe3f050840 20 datascan.scan_links: injected ino detected 2023-10-11T09:42:49.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.936+0000 7efe3f050840 20 datascan.scan_links: injected ino detected 2023-10-11T09:42:49.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.936+0000 7efe3f050840 20 datascan.scan_links: injected ino detected 2023-10-11T09:42:49.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.936+0000 7efe3f050840 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2023-10-11T09:42:49.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.936+0000 7efe3f050840 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-10-11T09:42:49.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.936+0000 7efe3f050840 10 datascan.scan_links: step 1: handling object 200.00000000 2023-10-11T09:42:49.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.936+0000 7efe3f050840 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-10-11T09:42:49.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.936+0000 7efe3f050840 10 datascan.scan_links: step 1: handling object 200.00000001 2023-10-11T09:42:49.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.936+0000 7efe3f050840 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-10-11T09:42:49.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.936+0000 7efe3f050840 10 datascan.scan_links: step 1: handling object 500.00000000 2023-10-11T09:42:49.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.936+0000 7efe3f050840 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-10-11T09:42:49.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.936+0000 7efe3f050840 10 datascan.scan_links: step 1: handling object mds_snaptable 2023-10-11T09:42:49.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.936+0000 7efe3f050840 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-10-11T09:42:49.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.936+0000 7efe3f050840 10 datascan.scan_links: step 1: handling object mds0_inotable 2023-10-11T09:42:49.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.936+0000 7efe3f050840 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-10-11T09:42:49.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.936+0000 7efe3f050840 10 datascan.scan_links: step 1: handling object 100.00000000 2023-10-11T09:42:49.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.936+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:34 5.13 5:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539864800 con 0x55653a275000 2023-10-11T09:42:49.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.936+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 <== osd.3 v2:172.21.15.78:6824/35738 5 ==== osd_op_reply(34 100.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55653a1c6fc0 con 0x55653a275000 2023-10-11T09:42:49.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.936+0000 7efe3f050840 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2023-10-11T09:42:49.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.936+0000 7efe3f050840 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-10-11T09:42:49.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.936+0000 7efe3f050840 10 datascan.scan_links: step 1: handling object 400.00000000 2023-10-11T09:42:49.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.936+0000 7efe3f050840 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-10-11T09:42:49.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.936+0000 7efe3f050840 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2023-10-11T09:42:49.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.936+0000 7efe3f050840 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-10-11T09:42:49.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.936+0000 7efe3f050840 10 datascan.scan_links: step 1: handling object 1.00000000 2023-10-11T09:42:49.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.936+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:35 5.1f 5:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539864c00 con 0x55653983e800 2023-10-11T09:42:49.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.940+0000 7efe3d80c700 1 -- 172.21.15.78:0/1780204447 <== osd.6 v2:172.21.15.133:6800/35900 4 ==== osd_op_reply(35 1.00000000 [omap-get-vals out=508b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+508 (crc 0 0 0) 0x55653a1c7440 con 0x55653983e800 2023-10-11T09:42:49.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.940+0000 7efe3f050840 20 datascan.scan_links: injected ino detected 2023-10-11T09:42:49.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.940+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:36 5.0 5:00000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539865000 con 0x55653a275000 2023-10-11T09:42:49.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.940+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 <== osd.3 v2:172.21.15.78:6824/35738 6 ==== osd_op_reply(36 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c6fc0 con 0x55653a275000 2023-10-11T09:42:49.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.940+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:37 5.10 5:08000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539865000 con 0x55653a275400 2023-10-11T09:42:49.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.940+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 <== osd.7 v2:172.21.15.133:6824/35901 6 ==== osd_op_reply(37 [pgnls start_epoch 44 out=81b] v44'21 uv21 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55653a1c7200 con 0x55653a275400 2023-10-11T09:42:49.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.940+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:38 5.8 5:10000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539865000 con 0x55653983e400 2023-10-11T09:42:49.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.940+0000 7efe3d80c700 1 -- 172.21.15.78:0/1780204447 <== osd.2 v2:172.21.15.78:6808/35732 6 ==== osd_op_reply(38 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c7440 con 0x55653983e400 2023-10-11T09:42:49.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.940+0000 7efe3d80c700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:39 5.18 5:18000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539864400 con 0x55653983ec00 2023-10-11T09:42:49.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.944+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 <== osd.4 v2:172.21.15.133:6808/35898 5 ==== osd_op_reply(39 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c6fc0 con 0x55653983ec00 2023-10-11T09:42:49.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.944+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:40 5.4 5:20000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539864400 con 0x55653a275400 2023-10-11T09:42:49.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.944+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 <== osd.7 v2:172.21.15.133:6824/35901 7 ==== osd_op_reply(40 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c7200 con 0x55653a275400 2023-10-11T09:42:49.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.944+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:41 5.14 5:28000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539864400 con 0x55653a275000 2023-10-11T09:42:49.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.944+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 <== osd.3 v2:172.21.15.78:6824/35738 7 ==== osd_op_reply(41 [pgnls start_epoch 44 out=125b] v44'41 uv41 ondisk = 1) v8 ==== 144+0+125 (crc 0 0 0) 0x55653a1c6fc0 con 0x55653a275000 2023-10-11T09:42:49.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.944+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:42 5.c 5:30000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539864400 con 0x55653983f400 2023-10-11T09:42:49.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.944+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 <== osd.1 v2:172.21.15.78:6809/35734 5 ==== osd_op_reply(42 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c7200 con 0x55653983f400 2023-10-11T09:42:49.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.944+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:43 5.1c 5:38000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539864400 con 0x55653983ec00 2023-10-11T09:42:49.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.948+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 <== osd.4 v2:172.21.15.133:6808/35898 6 ==== osd_op_reply(43 [pgnls start_epoch 44 out=49b] v43'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c6fc0 con 0x55653983ec00 2023-10-11T09:42:49.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.948+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:44 5.2 5:40000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539864400 con 0x55653983e800 2023-10-11T09:42:49.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.948+0000 7efe3d80c700 1 -- 172.21.15.78:0/1780204447 <== osd.6 v2:172.21.15.133:6800/35900 5 ==== osd_op_reply(44 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c7440 con 0x55653983e800 2023-10-11T09:42:49.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.948+0000 7efe3d80c700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:45 5.12 5:48000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539864400 con 0x55653983f400 2023-10-11T09:42:49.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.948+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 <== osd.1 v2:172.21.15.78:6809/35734 6 ==== osd_op_reply(45 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c7200 con 0x55653983f400 2023-10-11T09:42:49.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.948+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:46 5.a 5:50000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539864400 con 0x55653983e400 2023-10-11T09:42:49.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.948+0000 7efe3d80c700 1 -- 172.21.15.78:0/1780204447 <== osd.2 v2:172.21.15.78:6808/35732 7 ==== osd_op_reply(46 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c7440 con 0x55653983e400 2023-10-11T09:42:49.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.948+0000 7efe3d80c700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:47 5.1a 5:58000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539864400 con 0x55653a275400 2023-10-11T09:42:49.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.952+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 <== osd.7 v2:172.21.15.133:6824/35901 8 ==== osd_op_reply(47 [pgnls start_epoch 44 out=49b] v44'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c7200 con 0x55653a275400 2023-10-11T09:42:49.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.952+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:48 5.6 5:60000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539864400 con 0x55653983e400 2023-10-11T09:42:49.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.952+0000 7efe3d80c700 1 -- 172.21.15.78:0/1780204447 <== osd.2 v2:172.21.15.78:6808/35732 8 ==== osd_op_reply(48 [pgnls start_epoch 44 out=49b] v43'9 uv9 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c7440 con 0x55653983e400 2023-10-11T09:42:49.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.952+0000 7efe3d80c700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:49 5.16 5:68000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539864400 con 0x55653983f800 2023-10-11T09:42:49.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.952+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 <== osd.5 v2:172.21.15.133:6816/35899 6 ==== osd_op_reply(49 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c6fc0 con 0x55653983f800 2023-10-11T09:42:49.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.952+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:50 5.e 5:70000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539864400 con 0x55653983ec00 2023-10-11T09:42:49.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.952+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 <== osd.4 v2:172.21.15.133:6808/35898 7 ==== osd_op_reply(50 [pgnls start_epoch 44 out=49b] v43'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c6fc0 con 0x55653983ec00 2023-10-11T09:42:49.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.952+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:51 5.1e 5:78000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539864400 con 0x556539864000 2023-10-11T09:42:49.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.956+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 <== osd.0 v2:172.21.15.78:6800/35735 4 ==== osd_op_reply(51 [pgnls start_epoch 44 out=49b] v44'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c7200 con 0x556539864000 2023-10-11T09:42:49.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.956+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:52 5.1 5:80000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539864400 con 0x55653983ec00 2023-10-11T09:42:49.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.956+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 <== osd.4 v2:172.21.15.133:6808/35898 8 ==== osd_op_reply(52 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c6fc0 con 0x55653983ec00 2023-10-11T09:42:49.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.956+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:53 5.11 5:88000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539864400 con 0x55653983e800 2023-10-11T09:42:49.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.956+0000 7efe3d80c700 1 -- 172.21.15.78:0/1780204447 <== osd.6 v2:172.21.15.133:6800/35900 6 ==== osd_op_reply(53 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c7440 con 0x55653983e800 2023-10-11T09:42:49.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.956+0000 7efe3d80c700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:54 5.9 5:90000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539864400 con 0x55653a275400 2023-10-11T09:42:49.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.956+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 <== osd.7 v2:172.21.15.133:6824/35901 9 ==== osd_op_reply(54 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c7200 con 0x55653a275400 2023-10-11T09:42:49.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.956+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:55 5.19 5:98000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539864400 con 0x55653983f400 2023-10-11T09:42:49.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.960+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 <== osd.1 v2:172.21.15.78:6809/35734 7 ==== osd_op_reply(55 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c7200 con 0x55653983f400 2023-10-11T09:42:49.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.960+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:56 5.5 5:a0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539864400 con 0x556539864000 2023-10-11T09:42:49.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.960+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 <== osd.0 v2:172.21.15.78:6800/35735 5 ==== osd_op_reply(56 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c7200 con 0x556539864000 2023-10-11T09:42:49.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.960+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:57 5.15 5:a8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539864400 con 0x55653983f800 2023-10-11T09:42:49.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.960+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 <== osd.5 v2:172.21.15.133:6816/35899 7 ==== osd_op_reply(57 [pgnls start_epoch 44 out=73b] v43'6 uv6 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55653a1c6fc0 con 0x55653983f800 2023-10-11T09:42:49.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.960+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:58 5.d 5:b0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539864400 con 0x55653983e400 2023-10-11T09:42:49.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.960+0000 7efe3d80c700 1 -- 172.21.15.78:0/1780204447 <== osd.2 v2:172.21.15.78:6808/35732 9 ==== osd_op_reply(58 [pgnls start_epoch 44 out=74b] v44'3 uv3 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55653a1c7440 con 0x55653983e400 2023-10-11T09:42:49.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.960+0000 7efe3d80c700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:59 5.1d 5:b8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539864400 con 0x55653983f400 2023-10-11T09:42:49.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.960+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 <== osd.1 v2:172.21.15.78:6809/35734 8 ==== osd_op_reply(59 [pgnls start_epoch 44 out=49b] v43'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c7200 con 0x55653983f400 2023-10-11T09:42:49.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.960+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:60 5.3 5:c0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539864400 con 0x556539864000 2023-10-11T09:42:49.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.964+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 <== osd.0 v2:172.21.15.78:6800/35735 6 ==== osd_op_reply(60 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c7200 con 0x556539864000 2023-10-11T09:42:49.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.964+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:61 5.13 5:c8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539864400 con 0x55653a275000 2023-10-11T09:42:49.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.964+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 <== osd.3 v2:172.21.15.78:6824/35738 8 ==== osd_op_reply(61 [pgnls start_epoch 44 out=98b] v44'7 uv7 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55653a1c6fc0 con 0x55653a275000 2023-10-11T09:42:49.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.964+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:62 5.b 5:d0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539864400 con 0x55653983e400 2023-10-11T09:42:49.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.964+0000 7efe3d80c700 1 -- 172.21.15.78:0/1780204447 <== osd.2 v2:172.21.15.78:6808/35732 10 ==== osd_op_reply(62 [pgnls start_epoch 44 out=76b] v44'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55653a1c7440 con 0x55653983e400 2023-10-11T09:42:49.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.964+0000 7efe3d80c700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:63 5.1b 5:d8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539864400 con 0x55653983f800 2023-10-11T09:42:49.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.964+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 <== osd.5 v2:172.21.15.133:6816/35899 8 ==== osd_op_reply(63 [pgnls start_epoch 44 out=49b] v44'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c6fc0 con 0x55653983f800 2023-10-11T09:42:49.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.964+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:64 5.7 5:e0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539864400 con 0x55653983f800 2023-10-11T09:42:49.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.968+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 <== osd.5 v2:172.21.15.133:6816/35899 9 ==== osd_op_reply(64 [pgnls start_epoch 44 out=49b] v43'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c6fc0 con 0x55653983f800 2023-10-11T09:42:49.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.968+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:65 5.17 5:e8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539864400 con 0x55653a275000 2023-10-11T09:42:49.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.968+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 <== osd.3 v2:172.21.15.78:6824/35738 9 ==== osd_op_reply(65 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653a1c6fc0 con 0x55653a275000 2023-10-11T09:42:49.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.968+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:66 5.f 5:f0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539864400 con 0x55653983f800 2023-10-11T09:42:49.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.968+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 <== osd.5 v2:172.21.15.133:6816/35899 10 ==== osd_op_reply(66 [pgnls start_epoch 44 out=73b] v44'4 uv4 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55653a1c6fc0 con 0x55653983f800 2023-10-11T09:42:49.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.968+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:67 5.1f 5:f8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539864400 con 0x55653983e800 2023-10-11T09:42:49.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.968+0000 7efe3d80c700 1 -- 172.21.15.78:0/1780204447 <== osd.6 v2:172.21.15.133:6800/35900 7 ==== osd_op_reply(67 [pgnls start_epoch 44 out=101b] v44'12 uv12 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55653a1c7440 con 0x55653983e800 2023-10-11T09:42:49.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.968+0000 7efe3f050840 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2023-10-11T09:42:49.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.968+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:68 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539865400 con 0x55653a275400 2023-10-11T09:42:49.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 <== osd.7 v2:172.21.15.133:6824/35901 10 ==== osd_op_reply(68 10000000000.00000000 [omap-get-vals out=2495b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+2495 (crc 0 0 0) 0x55653a1c7200 con 0x55653a275400 2023-10-11T09:42:49.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 20 datascan.scan_links: injected ino detected 2023-10-11T09:42:49.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2023-10-11T09:42:49.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 20 datascan.scan_links: injected ino detected 2023-10-11T09:42:49.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 20 datascan.scan_links: adding 0x100000001f6 for future processing to fix dnfirst 2023-10-11T09:42:49.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 20 datascan.scan_links: injected ino detected 2023-10-11T09:42:49.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2023-10-11T09:42:49.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 20 datascan.scan_links: injected ino detected 2023-10-11T09:42:49.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 20 datascan.scan_links: adding 0x100000001f8 for future processing to fix dnfirst 2023-10-11T09:42:49.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 20 datascan.scan_links: injected ino detected 2023-10-11T09:42:49.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 20 datascan.scan_links: adding 0x100000001f9 for future processing to fix dnfirst 2023-10-11T09:42:49.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2023-10-11T09:42:49.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-10-11T09:42:49.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 10 datascan.scan_links: step 2: handling object 200.00000000 2023-10-11T09:42:49.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-10-11T09:42:49.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 10 datascan.scan_links: step 2: handling object 200.00000001 2023-10-11T09:42:49.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-10-11T09:42:49.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 10 datascan.scan_links: step 2: handling object 500.00000000 2023-10-11T09:42:49.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-10-11T09:42:49.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 10 datascan.scan_links: step 2: handling object mds_snaptable 2023-10-11T09:42:49.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-10-11T09:42:49.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 10 datascan.scan_links: step 2: handling object mds0_inotable 2023-10-11T09:42:49.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-10-11T09:42:49.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 10 datascan.scan_links: step 2: handling object 100.00000000 2023-10-11T09:42:49.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:69 5.13 5:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539865800 con 0x55653a275000 2023-10-11T09:42:49.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 <== osd.3 v2:172.21.15.78:6824/35738 10 ==== osd_op_reply(69 100.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55653a1c6fc0 con 0x55653a275000 2023-10-11T09:42:49.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2023-10-11T09:42:49.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-10-11T09:42:49.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 10 datascan.scan_links: step 2: handling object 400.00000000 2023-10-11T09:42:49.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-10-11T09:42:49.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2023-10-11T09:42:49.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-10-11T09:42:49.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 10 datascan.scan_links: step 2: handling object 1.00000000 2023-10-11T09:42:49.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:70 5.1f 5:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539865c00 con 0x55653983e800 2023-10-11T09:42:49.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3d80c700 1 -- 172.21.15.78:0/1780204447 <== osd.6 v2:172.21.15.133:6800/35900 8 ==== osd_op_reply(70 1.00000000 [omap-get-vals out=508b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+508 (crc 0 0 0) 0x55653a1c7440 con 0x55653983e800 2023-10-11T09:42:49.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 20 datascan.scan_links: injected ino detected 2023-10-11T09:42:49.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 20 datascan.scan_links: adding 0x10000000000 for future processing to fix dnfirst 2023-10-11T09:42:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2023-10-11T09:42:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 10 datascan.scan_links: removing dup dentries from 0 objects 2023-10-11T09:42:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 10 datascan.scan_links: processing 0 bad_nlink_inos 2023-10-11T09:42:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 10 datascan.scan_links: processing 6 injected_inos 2023-10-11T09:42:49.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 10 datascan.scan_links: handling injected_ino 0x10000000000 2023-10-11T09:42:49.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x1:subdir 2023-10-11T09:42:49.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:71 5.1f 5:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539882000 con 0x55653983e800 2023-10-11T09:42:49.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3d80c700 1 -- 172.21.15.78:0/1780204447 <== osd.6 v2:172.21.15.133:6800/35900 9 ==== osd_op_reply(71 1.00000000 [omap-get-vals-by-keys out=507b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+507 (crc 0 0 0) 0x55653a1c7440 con 0x55653983e800 2023-10-11T09:42:49.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:42:49.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 20 datascan.scan_links: first is now 2 2023-10-11T09:42:49.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.972+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:72 5.1f 5:ff5b34d6:::1.00000000:head [omap-set-vals in=507b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539882400 con 0x55653983e800 2023-10-11T09:42:49.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.976+0000 7efe3d80c700 1 -- 172.21.15.78:0/1780204447 <== osd.6 v2:172.21.15.133:6800/35900 10 ==== osd_op_reply(72 1.00000000 [omap-set-vals] v44'13 uv13 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x55653a1c7440 con 0x55653983e800 2023-10-11T09:42:49.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.976+0000 7efe3f050840 20 datascan.inject_linkage: Injected dentry 0x0x1/subdir pointing to 0x0x10000000000 2023-10-11T09:42:49.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.976+0000 7efe3f050840 10 datascan.scan_links: handling injected_ino 0x10000000001 2023-10-11T09:42:49.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.976+0000 7efe3f050840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:0 2023-10-11T09:42:49.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.976+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:73 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539882800 con 0x55653a275400 2023-10-11T09:42:49.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.980+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 <== osd.7 v2:172.21.15.133:6824/35901 11 ==== osd_op_reply(73 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55653a1c7200 con 0x55653a275400 2023-10-11T09:42:49.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.980+0000 7efe3f050840 20 datascan.read_dentry: oid=10000000000.00000000 dname=0 frag=*, r=0 2023-10-11T09:42:49.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.980+0000 7efe3f050840 20 datascan.scan_links: first is now 2 2023-10-11T09:42:49.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.980+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:74 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539882c00 con 0x55653a275400 2023-10-11T09:42:49.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.980+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 <== osd.7 v2:172.21.15.133:6824/35901 12 ==== osd_op_reply(74 10000000000.00000000 [omap-set-vals] v44'22 uv22 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55653a1c7200 con 0x55653a275400 2023-10-11T09:42:49.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.980+0000 7efe3f050840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/0 pointing to 0x0x10000000001 2023-10-11T09:42:49.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.980+0000 7efe3f050840 10 datascan.scan_links: handling injected_ino 0x100000001f6 2023-10-11T09:42:49.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.980+0000 7efe3f050840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:1 2023-10-11T09:42:49.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.980+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:75 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539883000 con 0x55653a275400 2023-10-11T09:42:49.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.980+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 <== osd.7 v2:172.21.15.133:6824/35901 13 ==== osd_op_reply(75 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55653a1c7200 con 0x55653a275400 2023-10-11T09:42:49.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.980+0000 7efe3f050840 20 datascan.read_dentry: oid=10000000000.00000000 dname=1 frag=*, r=0 2023-10-11T09:42:49.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.980+0000 7efe3f050840 20 datascan.scan_links: first is now 2 2023-10-11T09:42:49.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.980+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:76 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539883400 con 0x55653a275400 2023-10-11T09:42:49.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.984+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 <== osd.7 v2:172.21.15.133:6824/35901 14 ==== osd_op_reply(76 10000000000.00000000 [omap-set-vals] v44'23 uv23 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55653a1c7200 con 0x55653a275400 2023-10-11T09:42:49.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.984+0000 7efe3f050840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/1 pointing to 0x0x100000001f6 2023-10-11T09:42:49.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.984+0000 7efe3f050840 10 datascan.scan_links: handling injected_ino 0x100000001f7 2023-10-11T09:42:49.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.984+0000 7efe3f050840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:2 2023-10-11T09:42:49.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.984+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:77 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539883800 con 0x55653a275400 2023-10-11T09:42:49.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.984+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 <== osd.7 v2:172.21.15.133:6824/35901 15 ==== osd_op_reply(77 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55653a1c7200 con 0x55653a275400 2023-10-11T09:42:49.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.984+0000 7efe3f050840 20 datascan.read_dentry: oid=10000000000.00000000 dname=2 frag=*, r=0 2023-10-11T09:42:49.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.984+0000 7efe3f050840 20 datascan.scan_links: first is now 2 2023-10-11T09:42:49.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.984+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:78 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539883c00 con 0x55653a275400 2023-10-11T09:42:49.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.988+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 <== osd.7 v2:172.21.15.133:6824/35901 16 ==== osd_op_reply(78 10000000000.00000000 [omap-set-vals] v44'24 uv24 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55653a1c7200 con 0x55653a275400 2023-10-11T09:42:49.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.988+0000 7efe3f050840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/2 pointing to 0x0x100000001f7 2023-10-11T09:42:49.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.988+0000 7efe3f050840 10 datascan.scan_links: handling injected_ino 0x100000001f8 2023-10-11T09:42:49.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.988+0000 7efe3f050840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:3 2023-10-11T09:42:49.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.988+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:79 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539882c00 con 0x55653a275400 2023-10-11T09:42:49.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.988+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 <== osd.7 v2:172.21.15.133:6824/35901 17 ==== osd_op_reply(79 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv24 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55653a1c7200 con 0x55653a275400 2023-10-11T09:42:49.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.988+0000 7efe3f050840 20 datascan.read_dentry: oid=10000000000.00000000 dname=3 frag=*, r=0 2023-10-11T09:42:49.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.988+0000 7efe3f050840 20 datascan.scan_links: first is now 2 2023-10-11T09:42:49.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.988+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:80 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539865400 con 0x55653a275400 2023-10-11T09:42:49.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.992+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 <== osd.7 v2:172.21.15.133:6824/35901 18 ==== osd_op_reply(80 10000000000.00000000 [omap-set-vals] v44'25 uv25 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55653a1c7200 con 0x55653a275400 2023-10-11T09:42:49.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.992+0000 7efe3f050840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/3 pointing to 0x0x100000001f8 2023-10-11T09:42:49.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.992+0000 7efe3f050840 10 datascan.scan_links: handling injected_ino 0x100000001f9 2023-10-11T09:42:49.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.992+0000 7efe3f050840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:4 2023-10-11T09:42:49.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.992+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:81 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539865000 con 0x55653a275400 2023-10-11T09:42:49.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.992+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 <== osd.7 v2:172.21.15.133:6824/35901 19 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv25 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55653a1c7200 con 0x55653a275400 2023-10-11T09:42:49.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.992+0000 7efe3f050840 20 datascan.read_dentry: oid=10000000000.00000000 dname=4 frag=*, r=0 2023-10-11T09:42:49.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.992+0000 7efe3f050840 20 datascan.scan_links: first is now 2 2023-10-11T09:42:49.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.992+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:82 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539864c00 con 0x55653a275400 2023-10-11T09:42:49.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.996+0000 7efe3e00d700 1 -- 172.21.15.78:0/1780204447 <== osd.7 v2:172.21.15.133:6824/35901 20 ==== osd_op_reply(82 10000000000.00000000 [omap-set-vals] v44'26 uv26 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55653a1c7200 con 0x55653a275400 2023-10-11T09:42:49.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.996+0000 7efe3f050840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/4 pointing to 0x0x100000001f9 2023-10-11T09:42:50.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.996+0000 7efe3f050840 10 datascan.scan_links: updating inotable 2023-10-11T09:42:50.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.996+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:83 5.13 5:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539864400 con 0x55653a275000 2023-10-11T09:42:50.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.996+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 <== osd.3 v2:172.21.15.78:6824/35738 11 ==== osd_op_reply(83 mds0_inotable [read 0~34 out=34b] v0'0 uv6 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x55653a1c6fc0 con 0x55653a275000 2023-10-11T09:42:50.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.996+0000 7efe3f050840 10 mds.0.inotable: skip_inos was [0x10000000000~0x10000000000] 2023-10-11T09:42:50.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.996+0000 7efe3f050840 10 mds.0.inotable: skip_inos now [0x100000001fa~0xfffffffe06] 2023-10-11T09:42:50.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:49.996+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:84 5.13 5:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v8 -- 0x556539865c00 con 0x55653a275000 2023-10-11T09:42:50.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.000+0000 7efe3d00b700 1 -- 172.21.15.78:0/1780204447 <== osd.3 v2:172.21.15.78:6824/35738 12 ==== osd_op_reply(84 mds0_inotable [writefull 0~34] v44'8 uv8 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x55653a1c6fc0 con 0x55653a275000 2023-10-11T09:42:50.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.000+0000 7efe3f050840 10 datascan.scan_links: updating snaptable 2023-10-11T09:42:50.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.000+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:85 5.d 5:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55653983f000 con 0x55653983e400 2023-10-11T09:42:50.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3d80c700 1 -- 172.21.15.78:0/1780204447 <== osd.2 v2:172.21.15.78:6808/35732 11 ==== osd_op_reply(85 mds_snaptable [read 0~70 out=70b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x55653a1c7440 con 0x55653983e400 2023-10-11T09:42:50.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x556539864000 msgr2=0x556539850680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:50.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3f050840 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x556539864000 0x556539850680 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:50.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55653983f400 msgr2=0x55653984f080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:50.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3f050840 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55653983f400 0x55653984f080 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:50.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55653983e400 msgr2=0x55653984e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:50.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3f050840 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55653983e400 0x55653984e580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:50.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55653a275000 msgr2=0x556539492100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:50.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3f050840 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55653a275000 0x556539492100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:50.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55653983ec00 msgr2=0x55653984eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:50.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3f050840 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55653983ec00 0x55653984eb00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:50.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3d80c700 1 -- 172.21.15.78:0/1780204447 reap_dead start 2023-10-11T09:42:50.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55653983f800 msgr2=0x55653984fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:50.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3f050840 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55653983f800 0x55653984fb80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:50.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55653983e800 msgr2=0x55653984f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:50.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3f050840 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55653983e800 0x55653984f600 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:50.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55653a275400 msgr2=0x55653984e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:50.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3f050840 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55653a275400 0x55653984e000 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:50.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3d80c700 1 -- 172.21.15.78:0/1780204447 reap_dead start 2023-10-11T09:42:50.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3d80c700 1 -- 172.21.15.78:0/1780204447 reap_dead start 2023-10-11T09:42:50.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55653983e000 msgr2=0x556539493700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:50.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3f050840 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55653983e000 0x556539493700 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x5565396d8720 tx=0x55653a257080 comp rx=0 tx=0).stop 2023-10-11T09:42:50.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55653a274400 msgr2=0x556539493180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:50.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3f050840 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55653a274400 0x556539493180 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x5565396d93b0 tx=0x5565396e4c60 comp rx=0 tx=0).stop 2023-10-11T09:42:50.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 shutdown_connections 2023-10-11T09:42:50.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3f050840 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55653983e000 0x556539493700 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:50.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3f050840 1 --2- 172.21.15.78:0/1780204447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55653a274400 0x556539493180 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:50.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 >> 172.21.15.78:0/1780204447 conn(0x5565393eac00 msgr2=0x5565394b9800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:50.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 shutdown_connections 2023-10-11T09:42:50.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3f050840 1 -- 172.21.15.78:0/1780204447 wait complete. 2023-10-11T09:42:50.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3f050840 1 -- 172.21.15.78:0/3145686363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565393eb000 msgr2=0x556539491080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:50.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3f050840 1 --2- 172.21.15.78:0/3145686363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565393eb000 0x556539491080 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x55653a1532f0 tx=0x55653a256c60 comp rx=0 tx=0).stop 2023-10-11T09:42:50.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3f050840 1 -- 172.21.15.78:0/3145686363 shutdown_connections 2023-10-11T09:42:50.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3f050840 1 --2- 172.21.15.78:0/3145686363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565393eb000 0x556539491080 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:50.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3f050840 1 --2- 172.21.15.78:0/3145686363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5565393eb400 0x556539490b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:50.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3f050840 1 --2- 172.21.15.78:0/3145686363 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5565393eb800 0x556539490580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:50.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3f050840 1 -- 172.21.15.78:0/3145686363 >> 172.21.15.78:0/3145686363 conn(0x5565393ebc00 msgr2=0x5565394b9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:50.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3f050840 1 -- 172.21.15.78:0/3145686363 shutdown_connections 2023-10-11T09:42:50.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.004+0000 7efe3f050840 1 -- 172.21.15.78:0/3145686363 wait complete. 2023-10-11T09:42:50.016 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.198794, result: 2023-10-11T09:42:50.017 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-11T09:42:50.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.196+0000 7f149d937700 1 Processor -- start 2023-10-11T09:42:50.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.196+0000 7f149d937700 1 -- start start 2023-10-11T09:42:50.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.200+0000 7f149d937700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f14980d1660 0x7f14980d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:50.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.200+0000 7f149d937700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f149815dc10 con 0x7f1498156ce0 2023-10-11T09:42:50.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.200+0000 7f149d937700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1498160380 con 0x7f14980d28e0 2023-10-11T09:42:50.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.200+0000 7f149d937700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f149815d930 con 0x7f14980d1660 2023-10-11T09:42:50.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.200+0000 7f1497fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f14980d1660 0x7f14980d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:50.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.200+0000 7f1497fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f14980d1660 0x7f14980d0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58120/0 (socket says 172.21.15.78:58120) 2023-10-11T09:42:50.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.200+0000 7f1497fff700 1 -- 172.21.15.78:0/765260742 learned_addr learned my addr 172.21.15.78:0/765260742 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:50.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.200+0000 7f1497fff700 1 -- 172.21.15.78:0/765260742 >> v1:172.21.15.133:6789/0 conn(0x7f14980d28e0 legacy=0x7f14980cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:50.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.200+0000 7f1497fff700 1 -- 172.21.15.78:0/765260742 >> v1:172.21.15.78:6789/0 conn(0x7f1498156ce0 legacy=0x7f1498160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:50.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.200+0000 7f1497fff700 1 -- 172.21.15.78:0/765260742 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1498160620 con 0x7f14980d1660 2023-10-11T09:42:50.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.200+0000 7f1497fff700 1 --2- 172.21.15.78:0/765260742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14980d1660 0x7f14980d0a50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f1484009910 tx=0x7f1484009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f6cab2751006a75 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:50.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.204+0000 7f14977fe700 1 -- 172.21.15.78:0/765260742 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1484004230 con 0x7f14980d1660 2023-10-11T09:42:50.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.204+0000 7f14977fe700 1 -- 172.21.15.78:0/765260742 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1484004390 con 0x7f14980d1660 2023-10-11T09:42:50.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.204+0000 7f14977fe700 1 -- 172.21.15.78:0/765260742 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1484009e60 con 0x7f14980d1660 2023-10-11T09:42:50.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.204+0000 7f149d937700 1 -- 172.21.15.78:0/765260742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14980d1660 msgr2=0x7f14980d0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:50.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.204+0000 7f149d937700 1 --2- 172.21.15.78:0/765260742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14980d1660 0x7f14980d0a50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f1484009910 tx=0x7f1484009c20 comp rx=0 tx=0).stop 2023-10-11T09:42:50.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.204+0000 7f149d937700 1 -- 172.21.15.78:0/765260742 shutdown_connections 2023-10-11T09:42:50.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.204+0000 7f149d937700 1 --2- 172.21.15.78:0/765260742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14980d1660 0x7f14980d0a50 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:50.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.204+0000 7f149d937700 1 -- 172.21.15.78:0/765260742 >> 172.21.15.78:0/765260742 conn(0x7f149806afb0 msgr2=0x7f149806b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:50.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.204+0000 7f149d937700 1 -- 172.21.15.78:0/765260742 shutdown_connections 2023-10-11T09:42:50.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.204+0000 7f149d937700 1 -- 172.21.15.78:0/765260742 wait complete. 2023-10-11T09:42:50.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.204+0000 7f149d937700 1 Processor -- start 2023-10-11T09:42:50.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.204+0000 7f149d937700 1 -- start start 2023-10-11T09:42:50.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.208+0000 7f149d937700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14980d1660 0x7f14981e8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:50.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.208+0000 7f149d937700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f14980d28e0 0x7f1498153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:50.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.208+0000 7f149d937700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1498156ce0 0x7f14980cb010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:50.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.208+0000 7f149d937700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14981e8d70 con 0x7f1498156ce0 2023-10-11T09:42:50.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.208+0000 7f149d937700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1498154470 con 0x7f14980d1660 2023-10-11T09:42:50.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.208+0000 7f149d937700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14981e8f80 con 0x7f14980d28e0 2023-10-11T09:42:50.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.208+0000 7f149d136700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1498156ce0 0x7f14980cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:50.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.208+0000 7f149d136700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1498156ce0 0x7f14980cb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58122/0 (socket says 172.21.15.78:58122) 2023-10-11T09:42:50.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.208+0000 7f149d136700 1 -- 172.21.15.78:0/4270246034 learned_addr learned my addr 172.21.15.78:0/4270246034 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:50.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.208+0000 7f149c935700 1 --2- 172.21.15.78:0/4270246034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14980d1660 0x7f14981e8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:50.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.208+0000 7f1497fff700 1 --2- 172.21.15.78:0/4270246034 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f14980d28e0 0x7f1498153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:50.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.208+0000 7f149d136700 1 -- 172.21.15.78:0/4270246034 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f14980d28e0 msgr2=0x7f1498153f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:50.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.208+0000 7f149d136700 1 --2- 172.21.15.78:0/4270246034 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f14980d28e0 0x7f1498153f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:50.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.208+0000 7f149d136700 1 -- 172.21.15.78:0/4270246034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14980d1660 msgr2=0x7f14981e8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:50.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.208+0000 7f149d136700 1 --2- 172.21.15.78:0/4270246034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14980d1660 0x7f14981e8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:50.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.208+0000 7f149d136700 1 -- 172.21.15.78:0/4270246034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1484009570 con 0x7f1498156ce0 2023-10-11T09:42:50.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.208+0000 7f149d136700 1 --2- 172.21.15.78:0/4270246034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1498156ce0 0x7f14980cb010 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f149000cc10 tx=0x7f149000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:50.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.208+0000 7f14957fa700 1 -- 172.21.15.78:0/4270246034 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149000aa60 con 0x7f1498156ce0 2023-10-11T09:42:50.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.208+0000 7f14957fa700 1 -- 172.21.15.78:0/4270246034 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f149000abc0 con 0x7f1498156ce0 2023-10-11T09:42:50.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.208+0000 7f149d937700 1 -- 172.21.15.78:0/4270246034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14981e9260 con 0x7f1498156ce0 2023-10-11T09:42:50.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.208+0000 7f14957fa700 1 -- 172.21.15.78:0/4270246034 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1490012420 con 0x7f1498156ce0 2023-10-11T09:42:50.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.212+0000 7f149d937700 1 -- 172.21.15.78:0/4270246034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14981e9820 con 0x7f1498156ce0 2023-10-11T09:42:50.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.212+0000 7f149d937700 1 -- 172.21.15.78:0/4270246034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1498065d40 con 0x7f1498156ce0 2023-10-11T09:42:50.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.212+0000 7f14957fa700 1 -- 172.21.15.78:0/4270246034 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f1490012580 con 0x7f1498156ce0 2023-10-11T09:42:50.218 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.216+0000 7f14957fa700 1 --2- 172.21.15.78:0/4270246034 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f147c065ed0 0x7f147c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:50.218 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.216+0000 7f14957fa700 1 -- 172.21.15.78:0/4270246034 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6667+0+0 (secure 0 0 0) 0x7f1490083c80 con 0x7f1498156ce0 2023-10-11T09:42:50.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.216+0000 7f149c935700 1 --2- 172.21.15.78:0/4270246034 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f147c065ed0 0x7f147c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:50.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.216+0000 7f149c935700 1 --2- 172.21.15.78:0/4270246034 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f147c065ed0 0x7f147c068380 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f148c001800 tx=0x7f148c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:50.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.220+0000 7f14957fa700 1 -- 172.21.15.78:0/4270246034 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f1490051100 con 0x7f1498156ce0 2023-10-11T09:42:50.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:50.468+0000 7f149d937700 1 -- 172.21.15.78:0/4270246034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f14980c66b0 con 0x7f1498156ce0 2023-10-11T09:42:51.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.428+0000 7f14957fa700 1 -- 172.21.15.78:0/4270246034 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 2:0 v23) v1 ==== 101+0+0 (secure 0 0 0) 0x7f1490054720 con 0x7f1498156ce0 2023-10-11T09:42:51.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.432+0000 7f149d937700 1 -- 172.21.15.78:0/4270246034 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f147c065ed0 msgr2=0x7f147c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:51.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.432+0000 7f149d937700 1 --2- 172.21.15.78:0/4270246034 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f147c065ed0 0x7f147c068380 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f148c001800 tx=0x7f148c009040 comp rx=0 tx=0).stop 2023-10-11T09:42:51.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.432+0000 7f149d937700 1 -- 172.21.15.78:0/4270246034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1498156ce0 msgr2=0x7f14980cb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:51.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.432+0000 7f149d937700 1 --2- 172.21.15.78:0/4270246034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1498156ce0 0x7f14980cb010 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f149000cc10 tx=0x7f149000cf20 comp rx=0 tx=0).stop 2023-10-11T09:42:51.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.432+0000 7f149d937700 1 -- 172.21.15.78:0/4270246034 shutdown_connections 2023-10-11T09:42:51.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.432+0000 7f149d937700 1 --2- 172.21.15.78:0/4270246034 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f147c065ed0 0x7f147c068380 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:51.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.432+0000 7f149d937700 1 --2- 172.21.15.78:0/4270246034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1498156ce0 0x7f14980cb010 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:51.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.432+0000 7f149d937700 1 --2- 172.21.15.78:0/4270246034 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f14980d28e0 0x7f1498153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:51.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.432+0000 7f149d937700 1 --2- 172.21.15.78:0/4270246034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14980d1660 0x7f14981e8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:51.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.432+0000 7f149d937700 1 -- 172.21.15.78:0/4270246034 >> 172.21.15.78:0/4270246034 conn(0x7f149806afb0 msgr2=0x7f14980b8280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:51.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.432+0000 7f149d937700 1 -- 172.21.15.78:0/4270246034 shutdown_connections 2023-10-11T09:42:51.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.432+0000 7f149d937700 1 -- 172.21.15.78:0/4270246034 wait complete. 2023-10-11T09:42:51.438 INFO:teuthology.orchestra.run.smithi078.stderr:repaired: restoring rank 2:0 2023-10-11T09:42:51.443 INFO:tasks.ceph.mds.b.smithi133.stderr:2023-10-11T09:42:51.437+0000 7f7834858700 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-10-11T09:42:51.459 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-11T09:42:51.459 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-11T09:42:51.460 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-11T09:42:51.460 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:42:51.460 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-11T09:42:51.460 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-11T09:42:51.461 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-11T09:42:51.461 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:42:51.461 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-11T09:42:51.462 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-11T09:42:51.462 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-11T09:42:51.462 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:42:51.463 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-11T09:42:51.463 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-11T09:42:51.463 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-11T09:42:51.463 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:42:51.482 INFO:tasks.ceph.mds.a:Stopped 2023-10-11T09:42:51.482 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-11T09:42:51.483 INFO:tasks.ceph.mds.c:Stopped 2023-10-11T09:42:51.484 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-11T09:42:51.488 INFO:tasks.ceph.mds.b:Stopped 2023-10-11T09:42:51.488 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-11T09:42:51.489 INFO:tasks.ceph.mds.d:Stopped 2023-10-11T09:42:51.490 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-11T09:42:51.531 INFO:tasks.ceph.mds.a:Started 2023-10-11T09:42:51.532 DEBUG:teuthology.parallel:result is None 2023-10-11T09:42:51.537 INFO:tasks.ceph.mds.c:Started 2023-10-11T09:42:51.538 DEBUG:teuthology.parallel:result is None 2023-10-11T09:42:51.539 INFO:tasks.ceph.mds.b:Started 2023-10-11T09:42:51.539 DEBUG:teuthology.parallel:result is None 2023-10-11T09:42:51.544 INFO:tasks.ceph.mds.d:Started 2023-10-11T09:42:51.545 DEBUG:teuthology.parallel:result is None 2023-10-11T09:42:51.545 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:42:51.649 INFO:tasks.ceph.mds.a.smithi078.stdout:starting mds.a at 2023-10-11T09:42:51.649 INFO:tasks.ceph.mds.c.smithi078.stdout:starting mds.c at 2023-10-11T09:42:51.659 INFO:tasks.ceph.mds.b.smithi133.stdout:starting mds.b at 2023-10-11T09:42:51.660 INFO:tasks.ceph.mds.d.smithi133.stdout:starting mds.d at 2023-10-11T09:42:51.688 INFO:tasks.ceph.mon.a.smithi078.stderr:2023-10-11T09:42:51.684+0000 7f90131f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-11T09:42:51.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.712+0000 7fc632c64700 1 Processor -- start 2023-10-11T09:42:51.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.716+0000 7fc632c64700 1 -- start start 2023-10-11T09:42:51.719 INFO:tasks.ceph.mds.c.smithi078.stderr:2023-10-11T09:42:51.716+0000 7f6906d7f700 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-10-11T09:42:51.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.716+0000 7fc632c64700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc62c0d1660 0x7fc62c0d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:51.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.716+0000 7fc632c64700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fc62c0d28e0 0x7fc62c0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:51.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.716+0000 7fc632c64700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc62c15dc10 con 0x7fc62c156ce0 2023-10-11T09:42:51.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.716+0000 7fc632c64700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc62c160380 con 0x7fc62c0d28e0 2023-10-11T09:42:51.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.716+0000 7fc632c64700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c15d930 con 0x7fc62c0d1660 2023-10-11T09:42:51.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.716+0000 7fc631c62700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fc62c156ce0 0x7fc62c160b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:46130/0 (socket says 172.21.15.78:46130) 2023-10-11T09:42:51.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.716+0000 7fc631c62700 1 -- 172.21.15.78:0/1064556191 learned_addr learned my addr 172.21.15.78:0/1064556191 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:42:51.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.716+0000 7fc631461700 1 --2- 172.21.15.78:0/1064556191 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc62c0d1660 0x7fc62c0d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:51.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.716+0000 7fc632463700 1 --2- 172.21.15.78:0/1064556191 >> v2:172.21.15.78:3300/0 conn(0x7fc62c0d28e0 0x7fc62c0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:51.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.716+0000 7fc630c60700 1 -- 172.21.15.78:0/1064556191 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4261935209 0 0) 0x7fc62c15dc10 con 0x7fc62c156ce0 2023-10-11T09:42:51.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.716+0000 7fc630c60700 1 -- 172.21.15.78:0/1064556191 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6180035d0 con 0x7fc62c156ce0 2023-10-11T09:42:51.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.716+0000 7fc631461700 1 -- 172.21.15.78:0/1064556191 >> v2:172.21.15.78:3300/0 conn(0x7fc62c0d28e0 msgr2=0x7fc62c0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:51.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.716+0000 7fc631461700 1 --2- 172.21.15.78:0/1064556191 >> v2:172.21.15.78:3300/0 conn(0x7fc62c0d28e0 0x7fc62c0d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:51.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.716+0000 7fc631461700 1 -- 172.21.15.78:0/1064556191 >> v1:172.21.15.133:6789/0 conn(0x7fc62c156ce0 legacy=0x7fc62c160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:51.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.716+0000 7fc631461700 1 -- 172.21.15.78:0/1064556191 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc62c160620 con 0x7fc62c0d1660 2023-10-11T09:42:51.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.720+0000 7fc632463700 1 --2- 172.21.15.78:0/1064556191 >> v2:172.21.15.78:3300/0 conn(0x7fc62c0d28e0 0x7fc62c0d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:42:51.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.720+0000 7fc631461700 1 --2- 172.21.15.78:0/1064556191 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc62c0d1660 0x7fc62c0d0a50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc61c009850 tx=0x7fc61c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfba10af425606cc server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:51.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.720+0000 7fc630c60700 1 -- 172.21.15.78:0/1064556191 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61c011830 con 0x7fc62c0d1660 2023-10-11T09:42:51.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.720+0000 7fc630c60700 1 -- 172.21.15.78:0/1064556191 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc61c011990 con 0x7fc62c0d1660 2023-10-11T09:42:51.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.720+0000 7fc630c60700 1 -- 172.21.15.78:0/1064556191 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61c011d40 con 0x7fc62c0d1660 2023-10-11T09:42:51.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.720+0000 7fc632c64700 1 -- 172.21.15.78:0/1064556191 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc62c0d1660 msgr2=0x7fc62c0d0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:51.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.720+0000 7fc632c64700 1 --2- 172.21.15.78:0/1064556191 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc62c0d1660 0x7fc62c0d0a50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc61c009850 tx=0x7fc61c009b60 comp rx=0 tx=0).stop 2023-10-11T09:42:51.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.720+0000 7fc632c64700 1 -- 172.21.15.78:0/1064556191 shutdown_connections 2023-10-11T09:42:51.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.720+0000 7fc632c64700 1 --2- 172.21.15.78:0/1064556191 >> v2:172.21.15.78:3300/0 conn(0x7fc62c0d28e0 0x7fc62c0d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:51.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.720+0000 7fc632c64700 1 --2- 172.21.15.78:0/1064556191 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc62c0d1660 0x7fc62c0d0a50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:51.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.720+0000 7fc632c64700 1 -- 172.21.15.78:0/1064556191 >> 172.21.15.78:0/1064556191 conn(0x7fc62c06afb0 msgr2=0x7fc62c06b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:51.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.720+0000 7fc632c64700 1 -- 172.21.15.78:0/1064556191 shutdown_connections 2023-10-11T09:42:51.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.720+0000 7fc632c64700 1 -- 172.21.15.78:0/1064556191 wait complete. 2023-10-11T09:42:51.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.720+0000 7fc632c64700 1 Processor -- start 2023-10-11T09:42:51.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.724+0000 7fc632c64700 1 -- start start 2023-10-11T09:42:51.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.724+0000 7fc632c64700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc62c0d1660 0x7fc62c1f1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:51.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.724+0000 7fc632c64700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc62c0d28e0 0x7fc62c0c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:51.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.724+0000 7fc632c64700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc62c156ce0 0x7fc62c0c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:51.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.724+0000 7fc632c64700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c154cd0 con 0x7fc62c0d28e0 2023-10-11T09:42:51.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.724+0000 7fc632c64700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c0cb660 con 0x7fc62c0d1660 2023-10-11T09:42:51.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.724+0000 7fc632c64700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c0cb940 con 0x7fc62c156ce0 2023-10-11T09:42:51.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.724+0000 7fc631461700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc62c0d28e0 0x7fc62c0c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:51.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.724+0000 7fc631461700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc62c0d28e0 0x7fc62c0c8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58142/0 (socket says 172.21.15.78:58142) 2023-10-11T09:42:51.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.724+0000 7fc631461700 1 -- 172.21.15.78:0/2760064908 learned_addr learned my addr 172.21.15.78:0/2760064908 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:51.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.724+0000 7fc631c62700 1 --2- 172.21.15.78:0/2760064908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc62c0d1660 0x7fc62c1f1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:51.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.724+0000 7fc631461700 1 -- 172.21.15.78:0/2760064908 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc62c156ce0 msgr2=0x7fc62c0c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:51.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.724+0000 7fc631461700 1 --2- 172.21.15.78:0/2760064908 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc62c156ce0 0x7fc62c0c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:51.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.724+0000 7fc631461700 1 -- 172.21.15.78:0/2760064908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc62c0d1660 msgr2=0x7fc62c1f1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:51.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.724+0000 7fc631461700 1 --2- 172.21.15.78:0/2760064908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc62c0d1660 0x7fc62c1f1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:51.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.724+0000 7fc631461700 1 -- 172.21.15.78:0/2760064908 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc624009a70 con 0x7fc62c0d28e0 2023-10-11T09:42:51.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.724+0000 7fc631c62700 1 --2- 172.21.15.78:0/2760064908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc62c0d1660 0x7fc62c1f1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:42:51.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.724+0000 7fc631461700 1 --2- 172.21.15.78:0/2760064908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc62c0d28e0 0x7fc62c0c8b80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc61c0090b0 tx=0x7fc61c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:51.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.724+0000 7fc6227fc700 1 -- 172.21.15.78:0/2760064908 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61c004400 con 0x7fc62c0d28e0 2023-10-11T09:42:51.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.724+0000 7fc632c64700 1 -- 172.21.15.78:0/2760064908 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc61c0094b0 con 0x7fc62c0d28e0 2023-10-11T09:42:51.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.724+0000 7fc6227fc700 1 -- 172.21.15.78:0/2760064908 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc61c01f070 con 0x7fc62c0d28e0 2023-10-11T09:42:51.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.724+0000 7fc6227fc700 1 -- 172.21.15.78:0/2760064908 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61c0036c0 con 0x7fc62c0d28e0 2023-10-11T09:42:51.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.724+0000 7fc632c64700 1 -- 172.21.15.78:0/2760064908 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc62c0cbf00 con 0x7fc62c0d28e0 2023-10-11T09:42:51.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.728+0000 7fc632c64700 1 -- 172.21.15.78:0/2760064908 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc62c0cd320 con 0x7fc62c0d28e0 2023-10-11T09:42:51.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.728+0000 7fc6227fc700 1 -- 172.21.15.78:0/2760064908 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fc61c021030 con 0x7fc62c0d28e0 2023-10-11T09:42:51.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.732+0000 7fc6227fc700 1 --2- 172.21.15.78:0/2760064908 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc610065ed0 0x7fc610068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:51.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.732+0000 7fc6227fc700 1 -- 172.21.15.78:0/2760064908 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6753+0+0 (secure 0 0 0) 0x7fc61c084e00 con 0x7fc62c0d28e0 2023-10-11T09:42:51.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.732+0000 7fc631c62700 1 --2- 172.21.15.78:0/2760064908 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc610065ed0 0x7fc610068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:51.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.732+0000 7fc631c62700 1 --2- 172.21.15.78:0/2760064908 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc610065ed0 0x7fc610068380 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc6280011e0 tx=0x7fc628009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:51.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.736+0000 7fc6227fc700 1 -- 172.21.15.78:0/2760064908 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fc61c0522c0 con 0x7fc62c0d28e0 2023-10-11T09:42:51.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.940+0000 7fc632c64700 1 -- 172.21.15.78:0/2760064908 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc62c065d40 con 0x7fc62c0d28e0 2023-10-11T09:42:51.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.944+0000 7fc6227fc700 1 -- 172.21.15.78:0/2760064908 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4654 (secure 0 0 0) 0x7fc61c015090 con 0x7fc62c0d28e0 2023-10-11T09:42:51.946 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:51.947 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":26,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/588936111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":588936111},{"type":"v1","addr":"172.21.15.78:6836","nonce":588936111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5357,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/2574909888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2574909888},{"type":"v1","addr":"172.21.15.133:6834","nonce":2574909888}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5791,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2948223101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":2948223101},{"type":"v1","addr":"172.21.15.133:6835","nonce":2948223101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:42:45.491515+0000","modified":"2023-10-11T09:42:51.699947+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5922":{"gid":5922,"name":"c","rank":0,"incarnation":26,"state":"up:replay","state_seq":1,"addr":"172.21.15.78:6837/3137573764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":3137573764},{"type":"v1","addr":"172.21.15.78:6837","nonce":3137573764}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-11T09:42:51.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.944+0000 7fc632c64700 1 -- 172.21.15.78:0/2760064908 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc610065ed0 msgr2=0x7fc610068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:51.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.944+0000 7fc632c64700 1 --2- 172.21.15.78:0/2760064908 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc610065ed0 0x7fc610068380 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc6280011e0 tx=0x7fc628009040 comp rx=0 tx=0).stop 2023-10-11T09:42:51.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.944+0000 7fc632c64700 1 -- 172.21.15.78:0/2760064908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc62c0d28e0 msgr2=0x7fc62c0c8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:51.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.944+0000 7fc632c64700 1 --2- 172.21.15.78:0/2760064908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc62c0d28e0 0x7fc62c0c8b80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc61c0090b0 tx=0x7fc61c004770 comp rx=0 tx=0).stop 2023-10-11T09:42:51.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.944+0000 7fc632c64700 1 -- 172.21.15.78:0/2760064908 shutdown_connections 2023-10-11T09:42:51.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.944+0000 7fc632c64700 1 --2- 172.21.15.78:0/2760064908 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc610065ed0 0x7fc610068380 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:51.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.944+0000 7fc632c64700 1 --2- 172.21.15.78:0/2760064908 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc62c156ce0 0x7fc62c0c90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:51.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.944+0000 7fc632c64700 1 --2- 172.21.15.78:0/2760064908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc62c0d28e0 0x7fc62c0c8b80 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:51.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.944+0000 7fc632c64700 1 --2- 172.21.15.78:0/2760064908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc62c0d1660 0x7fc62c1f1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:51.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.944+0000 7fc632c64700 1 -- 172.21.15.78:0/2760064908 >> 172.21.15.78:0/2760064908 conn(0x7fc62c06afb0 msgr2=0x7fc62c15ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:51.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.948+0000 7fc632c64700 1 -- 172.21.15.78:0/2760064908 shutdown_connections 2023-10-11T09:42:51.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:51.948+0000 7fc632c64700 1 -- 172.21.15.78:0/2760064908 wait complete. 2023-10-11T09:42:51.952 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 26 2023-10-11T09:42:52.371 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-11T09:42:45.491515+0000', 'modified': '2023-10-11T09:42:51.699947+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 48, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5922}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5922': {'gid': 5922, 'name': 'c', 'rank': 0, 'incarnation': 26, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.78:6837/3137573764', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6835', 'nonce': 3137573764}, {'type': 'v1', 'addr': '172.21.15.78:6837', 'nonce': 3137573764}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6, 7], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-11T09:42:52.372 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5922:up:replay 2023-10-11T09:42:53.374 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:42:53.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.552+0000 7f65fd7f2700 1 Processor -- start 2023-10-11T09:42:53.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.552+0000 7f65fd7f2700 1 -- start start 2023-10-11T09:42:53.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.552+0000 7f65fd7f2700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f65f80c5e00 0x7f65f80c6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:53.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.552+0000 7f65fd7f2700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65f80c6a30 con 0x7f65f80b2050 2023-10-11T09:42:53.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.552+0000 7f65fd7f2700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65f80727a0 con 0x7f65f8072330 2023-10-11T09:42:53.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.552+0000 7f65fd7f2700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65f80cac50 con 0x7f65f80c5e00 2023-10-11T09:42:53.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.552+0000 7f65f6ffd700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f65f80b2050 0x7f65f80b2420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:59454/0 (socket says 172.21.15.78:59454) 2023-10-11T09:42:53.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.552+0000 7f65f6ffd700 1 -- 172.21.15.78:0/3743286 learned_addr learned my addr 172.21.15.78:0/3743286 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:42:53.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.556+0000 7f65f7fff700 1 --2- 172.21.15.78:0/3743286 >> v2:172.21.15.133:3300/0 conn(0x7f65f80c5e00 0x7f65f80c6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:53.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.556+0000 7f65f67fc700 1 -- 172.21.15.78:0/3743286 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1741695439 0 0) 0x7f65f80c6a30 con 0x7f65f80b2050 2023-10-11T09:42:53.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.556+0000 7f65f67fc700 1 -- 172.21.15.78:0/3743286 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65e40035d0 con 0x7f65f80b2050 2023-10-11T09:42:53.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.556+0000 7f65f67fc700 1 -- 172.21.15.78:0/3743286 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1504693690 0 0) 0x7f65e40035d0 con 0x7f65f80b2050 2023-10-11T09:42:53.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.556+0000 7f65f67fc700 1 -- 172.21.15.78:0/3743286 >> v2:172.21.15.133:3300/0 conn(0x7f65f80c5e00 msgr2=0x7f65f80c6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:53.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.556+0000 7f65f67fc700 1 --2- 172.21.15.78:0/3743286 >> v2:172.21.15.133:3300/0 conn(0x7f65f80c5e00 0x7f65f80c6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:53.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.556+0000 7f65f67fc700 1 -- 172.21.15.78:0/3743286 >> v1:172.21.15.133:6789/0 conn(0x7f65f8072330 legacy=0x7f65f80c56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:53.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.556+0000 7f65f67fc700 1 -- 172.21.15.78:0/3743286 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f815b8a0 con 0x7f65f80b2050 2023-10-11T09:42:53.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.556+0000 7f65f67fc700 1 -- 172.21.15.78:0/3743286 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f65e8001670 con 0x7f65f80b2050 2023-10-11T09:42:53.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.556+0000 7f65f67fc700 1 -- 172.21.15.78:0/3743286 >> v1:172.21.15.78:6789/0 conn(0x7f65f80b2050 legacy=0x7f65f80b2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:53.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.556+0000 7f65f67fc700 1 --2- 172.21.15.78:0/3743286 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65e4003a20 0x7f65e4003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:53.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.556+0000 7f65f67fc700 1 --2- 172.21.15.78:0/3743286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65e4004870 0x7f65e4004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:53.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.556+0000 7f65f67fc700 1 --2- 172.21.15.78:0/3743286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65e4005220 0x7f65e400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:53.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.556+0000 7f65f67fc700 1 -- 172.21.15.78:0/3743286 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f80cac50 con 0x7f65e4005220 2023-10-11T09:42:53.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.556+0000 7f65f67fc700 1 -- 172.21.15.78:0/3743286 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e400c160 con 0x7f65e4004870 2023-10-11T09:42:53.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.556+0000 7f65f67fc700 1 -- 172.21.15.78:0/3743286 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e400c320 con 0x7f65e4003a20 2023-10-11T09:42:53.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.556+0000 7f65f7fff700 1 --2- 172.21.15.78:0/3743286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65e4005220 0x7f65e400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:53.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.556+0000 7f65f6ffd700 1 --2- 172.21.15.78:0/3743286 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65e4003a20 0x7f65e4003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:53.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.556+0000 7f65effff700 1 --2- 172.21.15.78:0/3743286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65e4004870 0x7f65e4004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:53.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.556+0000 7f65f7fff700 1 -- 172.21.15.78:0/3743286 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65e4003a20 msgr2=0x7f65e4003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:53.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.556+0000 7f65f7fff700 1 --2- 172.21.15.78:0/3743286 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65e4003a20 0x7f65e4003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:53.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.556+0000 7f65f7fff700 1 -- 172.21.15.78:0/3743286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65e4004870 msgr2=0x7f65e4004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:53.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.556+0000 7f65f7fff700 1 --2- 172.21.15.78:0/3743286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65e4004870 0x7f65e4004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:53.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.556+0000 7f65f7fff700 1 -- 172.21.15.78:0/3743286 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e40035d0 con 0x7f65e4005220 2023-10-11T09:42:53.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.556+0000 7f65effff700 1 --2- 172.21.15.78:0/3743286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65e4004870 0x7f65e4004ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:42:53.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.556+0000 7f65f7fff700 1 -- 172.21.15.78:0/3743286 reap_dead start 2023-10-11T09:42:53.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.556+0000 7f65f7fff700 1 --2- 172.21.15.78:0/3743286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65e4005220 0x7f65e400ba30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f65f0007330 tx=0x7f65e40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9ef0d59b3043e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:53.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.556+0000 7f65f67fc700 1 -- 172.21.15.78:0/3743286 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f0007e50 con 0x7f65e4005220 2023-10-11T09:42:53.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.556+0000 7f65f67fc700 1 -- 172.21.15.78:0/3743286 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f0003a40 con 0x7f65e4005220 2023-10-11T09:42:53.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.556+0000 7f65f67fc700 1 -- 172.21.15.78:0/3743286 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f0003df0 con 0x7f65e4005220 2023-10-11T09:42:53.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.556+0000 7f65fd7f2700 1 -- 172.21.15.78:0/3743286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65e4005220 msgr2=0x7f65e400ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:53.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.556+0000 7f65fd7f2700 1 --2- 172.21.15.78:0/3743286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65e4005220 0x7f65e400ba30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f65f0007330 tx=0x7f65e40058a0 comp rx=0 tx=0).stop 2023-10-11T09:42:53.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.556+0000 7f65fd7f2700 1 -- 172.21.15.78:0/3743286 shutdown_connections 2023-10-11T09:42:53.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.556+0000 7f65fd7f2700 1 --2- 172.21.15.78:0/3743286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65e4005220 0x7f65e400ba30 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:53.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.556+0000 7f65fd7f2700 1 -- 172.21.15.78:0/3743286 >> 172.21.15.78:0/3743286 conn(0x7f65f806dd80 msgr2=0x7f65f806e150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:53.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.556+0000 7f65fd7f2700 1 -- 172.21.15.78:0/3743286 shutdown_connections 2023-10-11T09:42:53.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.556+0000 7f65fd7f2700 1 -- 172.21.15.78:0/3743286 wait complete. 2023-10-11T09:42:53.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.560+0000 7f65fd7f2700 1 Processor -- start 2023-10-11T09:42:53.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.560+0000 7f65fd7f2700 1 -- start start 2023-10-11T09:42:53.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.564+0000 7f65fd7f2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65e4005220 0x7f65f81598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:53.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.564+0000 7f65fd7f2700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65f8144f50 0x7f65f80c7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:53.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.564+0000 7f65fd7f2700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65f80cc840 0x7f65f80cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:53.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.564+0000 7f65fd7f2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f80c67c0 con 0x7f65e4005220 2023-10-11T09:42:53.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.564+0000 7f65fd7f2700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f80ca3f0 con 0x7f65f8144f50 2023-10-11T09:42:53.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.564+0000 7f65fd7f2700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65f80c7a80 con 0x7f65f80cc840 2023-10-11T09:42:53.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.564+0000 7f65f6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65e4005220 0x7f65f81598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:53.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.564+0000 7f65f6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65e4005220 0x7f65f81598d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58158/0 (socket says 172.21.15.78:58158) 2023-10-11T09:42:53.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.564+0000 7f65f6ffd700 1 -- 172.21.15.78:0/2674956427 learned_addr learned my addr 172.21.15.78:0/2674956427 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:53.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.564+0000 7f65effff700 1 --2- 172.21.15.78:0/2674956427 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65f8144f50 0x7f65f80c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:53.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.564+0000 7f65f6ffd700 1 -- 172.21.15.78:0/2674956427 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65f80cc840 msgr2=0x7f65f80cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:53.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.564+0000 7f65f7fff700 1 --2- 172.21.15.78:0/2674956427 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65f80cc840 0x7f65f80cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:53.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.564+0000 7f65f6ffd700 1 --2- 172.21.15.78:0/2674956427 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65f80cc840 0x7f65f80cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:53.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.564+0000 7f65f6ffd700 1 -- 172.21.15.78:0/2674956427 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65f8144f50 msgr2=0x7f65f80c7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:53.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.564+0000 7f65f6ffd700 1 --2- 172.21.15.78:0/2674956427 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65f8144f50 0x7f65f80c7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:53.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.564+0000 7f65f6ffd700 1 -- 172.21.15.78:0/2674956427 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f80cd1f0 con 0x7f65e4005220 2023-10-11T09:42:53.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.564+0000 7f65effff700 1 --2- 172.21.15.78:0/2674956427 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65f8144f50 0x7f65f80c7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:42:53.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.564+0000 7f65f7fff700 1 --2- 172.21.15.78:0/2674956427 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65f80cc840 0x7f65f80cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:42:53.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.564+0000 7f65f6ffd700 1 --2- 172.21.15.78:0/2674956427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65e4005220 0x7f65f81598d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f65e800e7b0 tx=0x7f65e800eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:53.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.564+0000 7f65ef7fe700 1 -- 172.21.15.78:0/2674956427 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e8012590 con 0x7f65e4005220 2023-10-11T09:42:53.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.564+0000 7f65ef7fe700 1 -- 172.21.15.78:0/2674956427 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e80126f0 con 0x7f65e4005220 2023-10-11T09:42:53.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.564+0000 7f65ef7fe700 1 -- 172.21.15.78:0/2674956427 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e8012aa0 con 0x7f65e4005220 2023-10-11T09:42:53.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.564+0000 7f65fd7f2700 1 -- 172.21.15.78:0/2674956427 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65f8159f60 con 0x7f65e4005220 2023-10-11T09:42:53.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.564+0000 7f65fd7f2700 1 -- 172.21.15.78:0/2674956427 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65f80c9e30 con 0x7f65e4005220 2023-10-11T09:42:53.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.568+0000 7f65fd7f2700 1 -- 172.21.15.78:0/2674956427 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65f804e530 con 0x7f65e4005220 2023-10-11T09:42:53.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.568+0000 7f65ef7fe700 1 -- 172.21.15.78:0/2674956427 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f65e8012c00 con 0x7f65e4005220 2023-10-11T09:42:53.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.568+0000 7f65ef7fe700 1 --2- 172.21.15.78:0/2674956427 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f65d8065ed0 0x7f65d8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:53.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.568+0000 7f65ef7fe700 1 -- 172.21.15.78:0/2674956427 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6753+0+0 (secure 0 0 0) 0x7f65e80837d0 con 0x7f65e4005220 2023-10-11T09:42:53.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.568+0000 7f65effff700 1 --2- 172.21.15.78:0/2674956427 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f65d8065ed0 0x7f65d8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:53.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.572+0000 7f65effff700 1 --2- 172.21.15.78:0/2674956427 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f65d8065ed0 0x7f65d8068380 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f65e0000f60 tx=0x7f65e0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:53.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.576+0000 7f65ef7fe700 1 -- 172.21.15.78:0/2674956427 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f65e8050b60 con 0x7f65e4005220 2023-10-11T09:42:53.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.788+0000 7f65fd7f2700 1 -- 172.21.15.78:0/2674956427 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65f8156580 con 0x7f65e4005220 2023-10-11T09:42:53.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.788+0000 7f65ef7fe700 1 -- 172.21.15.78:0/2674956427 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4654 (secure 0 0 0) 0x7f65e8054180 con 0x7f65e4005220 2023-10-11T09:42:53.792 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:53.792 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":28,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/588936111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":588936111},{"type":"v1","addr":"172.21.15.78:6836","nonce":588936111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5357,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/2574909888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2574909888},{"type":"v1","addr":"172.21.15.133:6834","nonce":2574909888}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5791,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2948223101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":2948223101},{"type":"v1","addr":"172.21.15.133:6835","nonce":2948223101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:42:45.491515+0000","modified":"2023-10-11T09:42:53.719340+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5922":{"gid":5922,"name":"c","rank":0,"incarnation":26,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.78:6837/3137573764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":3137573764},{"type":"v1","addr":"172.21.15.78:6837","nonce":3137573764}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-11T09:42:53.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.792+0000 7f65fd7f2700 1 -- 172.21.15.78:0/2674956427 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f65d8065ed0 msgr2=0x7f65d8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:53.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.792+0000 7f65fd7f2700 1 --2- 172.21.15.78:0/2674956427 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f65d8065ed0 0x7f65d8068380 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f65e0000f60 tx=0x7f65e0009040 comp rx=0 tx=0).stop 2023-10-11T09:42:53.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.792+0000 7f65fd7f2700 1 -- 172.21.15.78:0/2674956427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65e4005220 msgr2=0x7f65f81598d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:53.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.792+0000 7f65fd7f2700 1 --2- 172.21.15.78:0/2674956427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65e4005220 0x7f65f81598d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f65e800e7b0 tx=0x7f65e800eb70 comp rx=0 tx=0).stop 2023-10-11T09:42:53.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.792+0000 7f65fd7f2700 1 -- 172.21.15.78:0/2674956427 shutdown_connections 2023-10-11T09:42:53.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.792+0000 7f65fd7f2700 1 --2- 172.21.15.78:0/2674956427 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f65d8065ed0 0x7f65d8068380 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:53.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.792+0000 7f65fd7f2700 1 --2- 172.21.15.78:0/2674956427 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65f80cc840 0x7f65f80cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:53.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.792+0000 7f65fd7f2700 1 --2- 172.21.15.78:0/2674956427 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65f8144f50 0x7f65f80c7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:53.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.792+0000 7f65fd7f2700 1 --2- 172.21.15.78:0/2674956427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65e4005220 0x7f65f81598d0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:53.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.792+0000 7f65fd7f2700 1 -- 172.21.15.78:0/2674956427 >> 172.21.15.78:0/2674956427 conn(0x7f65f806dd80 msgr2=0x7f65f80cbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:53.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.792+0000 7f65fd7f2700 1 -- 172.21.15.78:0/2674956427 shutdown_connections 2023-10-11T09:42:53.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:53.792+0000 7f65fd7f2700 1 -- 172.21.15.78:0/2674956427 wait complete. 2023-10-11T09:42:53.797 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 28 2023-10-11T09:42:54.218 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-11T09:42:45.491515+0000', 'modified': '2023-10-11T09:42:53.719340+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 48, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5922}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5922': {'gid': 5922, 'name': 'c', 'rank': 0, 'incarnation': 26, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.78:6837/3137573764', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6835', 'nonce': 3137573764}, {'type': 'v1', 'addr': '172.21.15.78:6837', 'nonce': 3137573764}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6, 7], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-11T09:42:54.218 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5922:up:rejoin 2023-10-11T09:42:55.220 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:42:55.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.395+0000 7f540c5b3700 1 Processor -- start 2023-10-11T09:42:55.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.395+0000 7f540c5b3700 1 -- start start 2023-10-11T09:42:55.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.399+0000 7f540c5b3700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f5404160d40 0x7f54040cd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:55.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.399+0000 7f540c5b3700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f54040ce500 0x7f540406be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:55.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.399+0000 7f540c5b3700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f540406ffa0 con 0x7f5404071040 2023-10-11T09:42:55.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.399+0000 7f540c5b3700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5404070c20 con 0x7f5404160d40 2023-10-11T09:42:55.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.399+0000 7f540c5b3700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5404161110 con 0x7f54040ce500 2023-10-11T09:42:55.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.399+0000 7f540a34f700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f5404160d40 0x7f54040cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:55.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.399+0000 7f540a34f700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f5404160d40 0x7f54040cd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38976/0 (socket says 172.21.15.78:38976) 2023-10-11T09:42:55.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.399+0000 7f540a34f700 1 -- 172.21.15.78:0/2774818207 learned_addr learned my addr 172.21.15.78:0/2774818207 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:55.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.399+0000 7f540ab50700 1 --2- 172.21.15.78:0/2774818207 >> v2:172.21.15.133:3300/0 conn(0x7f54040ce500 0x7f540406be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:55.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.403+0000 7f540a34f700 1 -- 172.21.15.78:0/2774818207 >> v2:172.21.15.133:3300/0 conn(0x7f54040ce500 msgr2=0x7f540406be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:55.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.403+0000 7f540a34f700 1 --2- 172.21.15.78:0/2774818207 >> v2:172.21.15.133:3300/0 conn(0x7f54040ce500 0x7f540406be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:55.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.403+0000 7f540a34f700 1 -- 172.21.15.78:0/2774818207 >> v1:172.21.15.133:6789/0 conn(0x7f5404071040 legacy=0x7f54040cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:55.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.403+0000 7f540a34f700 1 -- 172.21.15.78:0/2774818207 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5404070280 con 0x7f5404160d40 2023-10-11T09:42:55.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.403+0000 7f540ab50700 1 --2- 172.21.15.78:0/2774818207 >> v2:172.21.15.133:3300/0 conn(0x7f54040ce500 0x7f540406be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:42:55.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.403+0000 7f540a34f700 1 --2- 172.21.15.78:0/2774818207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5404160d40 0x7f54040cd280 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f540406f890 tx=0x7f5400009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d0dd87fdbe5d47 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:55.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.403+0000 7f540934d700 1 -- 172.21.15.78:0/2774818207 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5400011830 con 0x7f5404160d40 2023-10-11T09:42:55.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.403+0000 7f540934d700 1 -- 172.21.15.78:0/2774818207 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5400011990 con 0x7f5404160d40 2023-10-11T09:42:55.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.403+0000 7f540934d700 1 -- 172.21.15.78:0/2774818207 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5400011d20 con 0x7f5404160d40 2023-10-11T09:42:55.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.403+0000 7f540c5b3700 1 -- 172.21.15.78:0/2774818207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5404160d40 msgr2=0x7f54040cd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:55.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.403+0000 7f540c5b3700 1 --2- 172.21.15.78:0/2774818207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5404160d40 0x7f54040cd280 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f540406f890 tx=0x7f5400009bf0 comp rx=0 tx=0).stop 2023-10-11T09:42:55.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.403+0000 7f540c5b3700 1 -- 172.21.15.78:0/2774818207 shutdown_connections 2023-10-11T09:42:55.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.403+0000 7f540c5b3700 1 --2- 172.21.15.78:0/2774818207 >> v2:172.21.15.133:3300/0 conn(0x7f54040ce500 0x7f540406be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:55.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.403+0000 7f540c5b3700 1 --2- 172.21.15.78:0/2774818207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5404160d40 0x7f54040cd280 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:55.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.403+0000 7f540c5b3700 1 -- 172.21.15.78:0/2774818207 >> 172.21.15.78:0/2774818207 conn(0x7f540406aca0 msgr2=0x7f54040b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:55.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.403+0000 7f540c5b3700 1 -- 172.21.15.78:0/2774818207 shutdown_connections 2023-10-11T09:42:55.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.403+0000 7f540c5b3700 1 -- 172.21.15.78:0/2774818207 wait complete. 2023-10-11T09:42:55.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.407+0000 7f540c5b3700 1 Processor -- start 2023-10-11T09:42:55.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.407+0000 7f540c5b3700 1 -- start start 2023-10-11T09:42:55.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.407+0000 7f540c5b3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5404071040 0x7f54040b4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:55.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.407+0000 7f540c5b3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54040ce500 0x7f54040c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:55.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.407+0000 7f540c5b3700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5404160d40 0x7f54041f9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:55.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.407+0000 7f540c5b3700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54040c5450 con 0x7f54040ce500 2023-10-11T09:42:55.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.407+0000 7f540c5b3700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54041fa010 con 0x7f5404071040 2023-10-11T09:42:55.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.407+0000 7f540c5b3700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54040c5680 con 0x7f5404160d40 2023-10-11T09:42:55.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.407+0000 7f5409b4e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54040ce500 0x7f54040c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:55.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.407+0000 7f5409b4e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54040ce500 0x7f54040c4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38982/0 (socket says 172.21.15.78:38982) 2023-10-11T09:42:55.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.407+0000 7f5409b4e700 1 -- 172.21.15.78:0/1716370879 learned_addr learned my addr 172.21.15.78:0/1716370879 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:55.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.407+0000 7f540a34f700 1 --2- 172.21.15.78:0/1716370879 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5404071040 0x7f54040b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:55.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.407+0000 7f540ab50700 1 --2- 172.21.15.78:0/1716370879 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5404160d40 0x7f54041f9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:55.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.407+0000 7f5409b4e700 1 -- 172.21.15.78:0/1716370879 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5404160d40 msgr2=0x7f54041f9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:55.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.407+0000 7f5409b4e700 1 --2- 172.21.15.78:0/1716370879 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5404160d40 0x7f54041f9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:55.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.407+0000 7f5409b4e700 1 -- 172.21.15.78:0/1716370879 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5404071040 msgr2=0x7f54040b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:55.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.407+0000 7f5409b4e700 1 --2- 172.21.15.78:0/1716370879 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5404071040 0x7f54040b4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:55.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.407+0000 7f5409b4e700 1 -- 172.21.15.78:0/1716370879 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53fc009a70 con 0x7f54040ce500 2023-10-11T09:42:55.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.407+0000 7f5409b4e700 1 --2- 172.21.15.78:0/1716370879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54040ce500 0x7f54040c4f10 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f53f400c4d0 tx=0x7f53f400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:55.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.411+0000 7f53faffd700 1 -- 172.21.15.78:0/1716370879 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f40052d0 con 0x7f54040ce500 2023-10-11T09:42:55.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.411+0000 7f53faffd700 1 -- 172.21.15.78:0/1716370879 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53f4005430 con 0x7f54040ce500 2023-10-11T09:42:55.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.411+0000 7f540c5b3700 1 -- 172.21.15.78:0/1716370879 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5400009570 con 0x7f54040ce500 2023-10-11T09:42:55.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.411+0000 7f53faffd700 1 -- 172.21.15.78:0/1716370879 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f40055a0 con 0x7f54040ce500 2023-10-11T09:42:55.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.411+0000 7f540c5b3700 1 -- 172.21.15.78:0/1716370879 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54040c5c40 con 0x7f54040ce500 2023-10-11T09:42:55.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.411+0000 7f540c5b3700 1 -- 172.21.15.78:0/1716370879 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54040b4c90 con 0x7f54040ce500 2023-10-11T09:42:55.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.411+0000 7f53faffd700 1 -- 172.21.15.78:0/1716370879 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f53f40038c0 con 0x7f54040ce500 2023-10-11T09:42:55.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.415+0000 7f53faffd700 1 --2- 172.21.15.78:0/1716370879 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f53e8065ed0 0x7f53e8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:55.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.415+0000 7f53faffd700 1 -- 172.21.15.78:0/1716370879 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6753+0+0 (secure 0 0 0) 0x7f53f4082ef0 con 0x7f54040ce500 2023-10-11T09:42:55.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.415+0000 7f540a34f700 1 --2- 172.21.15.78:0/1716370879 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f53e8065ed0 0x7f53e8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:55.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.415+0000 7f540a34f700 1 --2- 172.21.15.78:0/1716370879 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f53e8065ed0 0x7f53e8068380 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5400000c00 tx=0x7f5400008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:55.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.419+0000 7f53faffd700 1 -- 172.21.15.78:0/1716370879 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f53f4050170 con 0x7f54040ce500 2023-10-11T09:42:55.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.627+0000 7f540c5b3700 1 -- 172.21.15.78:0/1716370879 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5404072250 con 0x7f54040ce500 2023-10-11T09:42:55.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.627+0000 7f53faffd700 1 -- 172.21.15.78:0/1716370879 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4654 (secure 0 0 0) 0x7f53f4017090 con 0x7f54040ce500 2023-10-11T09:42:55.630 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:55.631 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":29,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/588936111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":588936111},{"type":"v1","addr":"172.21.15.78:6836","nonce":588936111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5357,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/2574909888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2574909888},{"type":"v1","addr":"172.21.15.133:6834","nonce":2574909888}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5791,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2948223101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":2948223101},{"type":"v1","addr":"172.21.15.133:6835","nonce":2948223101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:42:45.491515+0000","modified":"2023-10-11T09:42:54.734536+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5922":{"gid":5922,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":4,"addr":"172.21.15.78:6837/3137573764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":3137573764},{"type":"v1","addr":"172.21.15.78:6837","nonce":3137573764}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-11T09:42:55.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.627+0000 7f540c5b3700 1 -- 172.21.15.78:0/1716370879 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f53e8065ed0 msgr2=0x7f53e8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:55.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.627+0000 7f540c5b3700 1 --2- 172.21.15.78:0/1716370879 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f53e8065ed0 0x7f53e8068380 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5400000c00 tx=0x7f5400008030 comp rx=0 tx=0).stop 2023-10-11T09:42:55.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.627+0000 7f540c5b3700 1 -- 172.21.15.78:0/1716370879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54040ce500 msgr2=0x7f54040c4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:55.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.627+0000 7f540c5b3700 1 --2- 172.21.15.78:0/1716370879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54040ce500 0x7f54040c4f10 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f53f400c4d0 tx=0x7f53f400c7e0 comp rx=0 tx=0).stop 2023-10-11T09:42:55.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.627+0000 7f540c5b3700 1 -- 172.21.15.78:0/1716370879 shutdown_connections 2023-10-11T09:42:55.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.627+0000 7f540c5b3700 1 --2- 172.21.15.78:0/1716370879 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f53e8065ed0 0x7f53e8068380 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:55.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.627+0000 7f540c5b3700 1 --2- 172.21.15.78:0/1716370879 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5404160d40 0x7f54041f9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:55.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.627+0000 7f540c5b3700 1 --2- 172.21.15.78:0/1716370879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54040ce500 0x7f54040c4f10 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:55.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.627+0000 7f540c5b3700 1 --2- 172.21.15.78:0/1716370879 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5404071040 0x7f54040b4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:55.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.627+0000 7f540c5b3700 1 -- 172.21.15.78:0/1716370879 >> 172.21.15.78:0/1716370879 conn(0x7f540406aca0 msgr2=0x7f54040cb7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:55.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.627+0000 7f540c5b3700 1 -- 172.21.15.78:0/1716370879 shutdown_connections 2023-10-11T09:42:55.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:55.627+0000 7f540c5b3700 1 -- 172.21.15.78:0/1716370879 wait complete. 2023-10-11T09:42:55.635 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 29 2023-10-11T09:42:56.052 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-11T09:42:45.491515+0000', 'modified': '2023-10-11T09:42:54.734536+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 48, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5922}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5922': {'gid': 5922, 'name': 'c', 'rank': 0, 'incarnation': 26, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.78:6837/3137573764', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6835', 'nonce': 3137573764}, {'type': 'v1', 'addr': '172.21.15.78:6837', 'nonce': 3137573764}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6, 7], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-11T09:42:56.053 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-11T09:42:56.053 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-11T09:42:56.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.227+0000 7f517187e700 1 Processor -- start 2023-10-11T09:42:56.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.227+0000 7f517187e700 1 -- start start 2023-10-11T09:42:56.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.231+0000 7f517187e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f516c15b8e0 0x7f516c06d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:56.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.231+0000 7f517187e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f516c06d610 0x7f516c0ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:56.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.231+0000 7f517187e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f516c15a940 0x7f516c0ca780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:56.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.231+0000 7f517187e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f516c15e010 con 0x7f516c15b8e0 2023-10-11T09:42:56.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.231+0000 7f517187e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f516c15e280 con 0x7f516c06d610 2023-10-11T09:42:56.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.231+0000 7f517187e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f516c15e560 con 0x7f516c15a940 2023-10-11T09:42:56.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.231+0000 7f517087c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f516c15b8e0 0x7f516c06d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:56.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.231+0000 7f517087c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f516c15b8e0 0x7f516c06d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38994/0 (socket says 172.21.15.78:38994) 2023-10-11T09:42:56.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.231+0000 7f517087c700 1 -- 172.21.15.78:0/3651114894 learned_addr learned my addr 172.21.15.78:0/3651114894 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:56.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.231+0000 7f5163fff700 1 --2- 172.21.15.78:0/3651114894 >> v2:172.21.15.133:3300/0 conn(0x7f516c06d610 0x7f516c0ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:56.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.231+0000 7f517107d700 1 --2- 172.21.15.78:0/3651114894 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f516c15a940 0x7f516c0ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:56.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.235+0000 7f517087c700 1 -- 172.21.15.78:0/3651114894 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f516c15a940 msgr2=0x7f516c0ca780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:56.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.235+0000 7f517087c700 1 --2- 172.21.15.78:0/3651114894 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f516c15a940 0x7f516c0ca780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.235+0000 7f517087c700 1 -- 172.21.15.78:0/3651114894 >> v2:172.21.15.133:3300/0 conn(0x7f516c06d610 msgr2=0x7f516c0ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:56.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.235+0000 7f517087c700 1 --2- 172.21.15.78:0/3651114894 >> v2:172.21.15.133:3300/0 conn(0x7f516c06d610 0x7f516c0ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.235+0000 7f517087c700 1 -- 172.21.15.78:0/3651114894 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f516c15e840 con 0x7f516c15b8e0 2023-10-11T09:42:56.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.235+0000 7f517087c700 1 --2- 172.21.15.78:0/3651114894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f516c15b8e0 0x7f516c06d0d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f516c06f800 tx=0x7f515c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4996fac887fa698a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:56.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.235+0000 7f51637fe700 1 -- 172.21.15.78:0/3651114894 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c011830 con 0x7f516c15b8e0 2023-10-11T09:42:56.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.235+0000 7f51637fe700 1 -- 172.21.15.78:0/3651114894 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515c011990 con 0x7f516c15b8e0 2023-10-11T09:42:56.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.235+0000 7f51637fe700 1 -- 172.21.15.78:0/3651114894 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c011d20 con 0x7f516c15b8e0 2023-10-11T09:42:56.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.235+0000 7f517187e700 1 -- 172.21.15.78:0/3651114894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f516c15b8e0 msgr2=0x7f516c06d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:56.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.235+0000 7f517187e700 1 --2- 172.21.15.78:0/3651114894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f516c15b8e0 0x7f516c06d0d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f516c06f800 tx=0x7f515c009bf0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.235+0000 7f517187e700 1 -- 172.21.15.78:0/3651114894 shutdown_connections 2023-10-11T09:42:56.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.235+0000 7f517187e700 1 --2- 172.21.15.78:0/3651114894 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f516c15a940 0x7f516c0ca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.235+0000 7f517187e700 1 --2- 172.21.15.78:0/3651114894 >> v2:172.21.15.133:3300/0 conn(0x7f516c06d610 0x7f516c0ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.235+0000 7f517187e700 1 --2- 172.21.15.78:0/3651114894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f516c15b8e0 0x7f516c06d0d0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.235+0000 7f517187e700 1 -- 172.21.15.78:0/3651114894 >> 172.21.15.78:0/3651114894 conn(0x7f516c0cea60 msgr2=0x7f516c0cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:56.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.235+0000 7f517187e700 1 -- 172.21.15.78:0/3651114894 shutdown_connections 2023-10-11T09:42:56.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.235+0000 7f517187e700 1 -- 172.21.15.78:0/3651114894 wait complete. 2023-10-11T09:42:56.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.239+0000 7f517187e700 1 Processor -- start 2023-10-11T09:42:56.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.239+0000 7f517187e700 1 -- start start 2023-10-11T09:42:56.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.243+0000 7f517187e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f516c06d610 0x7f516c13f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:56.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.243+0000 7f517187e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f516c15a940 0x7f516c151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:56.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.243+0000 7f517187e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f516c15b8e0 0x7f516c0b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:56.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.243+0000 7f517187e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516c13fd90 con 0x7f516c06d610 2023-10-11T09:42:56.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.243+0000 7f517187e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516c13ef90 con 0x7f516c15b8e0 2023-10-11T09:42:56.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.243+0000 7f517187e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f516c13ea10 con 0x7f516c15a940 2023-10-11T09:42:56.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.243+0000 7f517087c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f516c06d610 0x7f516c13f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:56.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.243+0000 7f517087c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f516c06d610 0x7f516c13f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38996/0 (socket says 172.21.15.78:38996) 2023-10-11T09:42:56.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.243+0000 7f517087c700 1 -- 172.21.15.78:0/1781409210 learned_addr learned my addr 172.21.15.78:0/1781409210 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:56.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.243+0000 7f5163fff700 1 --2- 172.21.15.78:0/1781409210 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f516c15a940 0x7f516c151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:56.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.243+0000 7f517107d700 1 --2- 172.21.15.78:0/1781409210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f516c15b8e0 0x7f516c0b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:56.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.243+0000 7f517087c700 1 -- 172.21.15.78:0/1781409210 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f516c15a940 msgr2=0x7f516c151ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:56.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.243+0000 7f517087c700 1 --2- 172.21.15.78:0/1781409210 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f516c15a940 0x7f516c151ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.243+0000 7f517087c700 1 -- 172.21.15.78:0/1781409210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f516c15b8e0 msgr2=0x7f516c0b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:56.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.243+0000 7f517087c700 1 --2- 172.21.15.78:0/1781409210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f516c15b8e0 0x7f516c0b37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.243+0000 7f517087c700 1 -- 172.21.15.78:0/1781409210 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f515c009570 con 0x7f516c06d610 2023-10-11T09:42:56.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.243+0000 7f517107d700 1 --2- 172.21.15.78:0/1781409210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f516c15b8e0 0x7f516c0b37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:42:56.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.243+0000 7f517087c700 1 --2- 172.21.15.78:0/1781409210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f516c06d610 0x7f516c13f850 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f516c072460 tx=0x7f515c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:56.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.243+0000 7f51617fa700 1 -- 172.21.15.78:0/1781409210 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c004240 con 0x7f516c06d610 2023-10-11T09:42:56.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.243+0000 7f51617fa700 1 -- 172.21.15.78:0/1781409210 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515c01f070 con 0x7f516c06d610 2023-10-11T09:42:56.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.243+0000 7f517187e700 1 -- 172.21.15.78:0/1781409210 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f516c152010 con 0x7f516c06d610 2023-10-11T09:42:56.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.243+0000 7f51617fa700 1 -- 172.21.15.78:0/1781409210 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c0045e0 con 0x7f516c06d610 2023-10-11T09:42:56.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.243+0000 7f517187e700 1 -- 172.21.15.78:0/1781409210 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f516c13bfe0 con 0x7f516c06d610 2023-10-11T09:42:56.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.251+0000 7f51617fa700 1 -- 172.21.15.78:0/1781409210 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f515c021030 con 0x7f516c06d610 2023-10-11T09:42:56.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.251+0000 7f51617fa700 1 --2- 172.21.15.78:0/1781409210 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5158065ed0 0x7f5158068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:56.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.251+0000 7f51617fa700 1 -- 172.21.15.78:0/1781409210 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6753+0+0 (secure 0 0 0) 0x7f515c084a40 con 0x7f516c06d610 2023-10-11T09:42:56.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.251+0000 7f5163fff700 1 --2- 172.21.15.78:0/1781409210 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5158065ed0 0x7f5158068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:56.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.251+0000 7f5175d6e740 1 Processor -- start 2023-10-11T09:42:56.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.251+0000 7f5175d6e740 1 -- start start 2023-10-11T09:42:56.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.251+0000 7f5163fff700 1 --2- 172.21.15.78:0/1781409210 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5158065ed0 0x7f5158068380 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f516c0b0b10 tx=0x7f5168009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:56.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.255+0000 7f5175d6e740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1033eb0 0x12ec020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:56.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.255+0000 7f5175d6e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1033ae0 0x12ec560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:56.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.255+0000 7f5175d6e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1033710 0x12b3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:56.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.255+0000 7f5175d6e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x12134a0 con 0x1033ae0 2023-10-11T09:42:56.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.255+0000 7f5175d6e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x1210c30 con 0x1033710 2023-10-11T09:42:56.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.255+0000 7f5175d6e740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x120c520 con 0x1033eb0 2023-10-11T09:42:56.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.255+0000 7f5163fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1033ae0 0x12ec560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:56.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.255+0000 7f5163fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1033ae0 0x12ec560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39002/0 (socket says 172.21.15.78:39002) 2023-10-11T09:42:56.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.255+0000 7f5163fff700 1 -- 172.21.15.78:0/1693117557 learned_addr learned my addr 172.21.15.78:0/1693117557 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:56.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.255+0000 7f517087c700 1 --2- 172.21.15.78:0/1693117557 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1033eb0 0x12ec020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:56.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.255+0000 7f517107d700 1 --2- 172.21.15.78:0/1693117557 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1033710 0x12b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:56.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.255+0000 7f5163fff700 1 -- 172.21.15.78:0/1693117557 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1033eb0 msgr2=0x12ec020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:56.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.255+0000 7f5163fff700 1 --2- 172.21.15.78:0/1693117557 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1033eb0 0x12ec020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.255+0000 7f5163fff700 1 -- 172.21.15.78:0/1693117557 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1033710 msgr2=0x12b3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:56.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.255+0000 7f5163fff700 1 --2- 172.21.15.78:0/1693117557 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1033710 0x12b3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.255+0000 7f5163fff700 1 -- 172.21.15.78:0/1693117557 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x107f9e0 con 0x1033ae0 2023-10-11T09:42:56.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.255+0000 7f5163fff700 1 --2- 172.21.15.78:0/1693117557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1033ae0 0x12ec560 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f51680044d0 tx=0x7f5168009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c82d946c3023890a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:56.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.255+0000 7f51527fc700 1 -- 172.21.15.78:0/1693117557 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516801c040 con 0x1033ae0 2023-10-11T09:42:56.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.255+0000 7f51527fc700 1 -- 172.21.15.78:0/1693117557 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51680036c0 con 0x1033ae0 2023-10-11T09:42:56.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.255+0000 7f51527fc700 1 -- 172.21.15.78:0/1693117557 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5168003a30 con 0x1033ae0 2023-10-11T09:42:56.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.255+0000 7f5175d6e740 1 -- 172.21.15.78:0/1693117557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1033ae0 msgr2=0x12ec560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:56.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.255+0000 7f5175d6e740 1 --2- 172.21.15.78:0/1693117557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1033ae0 0x12ec560 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f51680044d0 tx=0x7f5168009f90 comp rx=0 tx=0).stop 2023-10-11T09:42:56.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.255+0000 7f5175d6e740 1 -- 172.21.15.78:0/1693117557 shutdown_connections 2023-10-11T09:42:56.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.255+0000 7f5175d6e740 1 --2- 172.21.15.78:0/1693117557 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1033710 0x12b3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.255+0000 7f5175d6e740 1 --2- 172.21.15.78:0/1693117557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1033ae0 0x12ec560 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.255+0000 7f5175d6e740 1 --2- 172.21.15.78:0/1693117557 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1033eb0 0x12ec020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.255+0000 7f5175d6e740 1 -- 172.21.15.78:0/1693117557 >> 172.21.15.78:0/1693117557 conn(0x129bfa0 msgr2=0x12cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:56.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.255+0000 7f5175d6e740 1 -- 172.21.15.78:0/1693117557 shutdown_connections 2023-10-11T09:42:56.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.255+0000 7f5175d6e740 1 -- 172.21.15.78:0/1693117557 wait complete. 2023-10-11T09:42:56.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.259+0000 7f5175d6e740 1 -- start start 2023-10-11T09:42:56.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.259+0000 7f5175d6e740 1 Processor -- start 2023-10-11T09:42:56.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.259+0000 7f5175d6e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1033710 0x12a0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:56.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.259+0000 7f5175d6e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1033ae0 0x12f35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:56.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.259+0000 7f5175d6e740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1033eb0 0x12f3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:56.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.259+0000 7f5175d6e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x11cfb00 con 0x1033710 2023-10-11T09:42:56.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.259+0000 7f5175d6e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x11cf7a0 con 0x1033ae0 2023-10-11T09:42:56.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.259+0000 7f5175d6e740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x10b8b20 con 0x1033eb0 2023-10-11T09:42:56.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.259+0000 7f5175d6e740 10 client.0 _ll_register_callbacks cb 0x1205e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:42:56.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.259+0000 7f517087c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1033710 0x12a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:56.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.259+0000 7f517087c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1033710 0x12a0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39010/0 (socket says 172.21.15.78:39010) 2023-10-11T09:42:56.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.259+0000 7f517087c700 1 -- 172.21.15.78:0/2988001185 learned_addr learned my addr 172.21.15.78:0/2988001185 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:56.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.259+0000 7f5163fff700 1 --2- 172.21.15.78:0/2988001185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1033ae0 0x12f35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:56.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.259+0000 7f517107d700 1 --2- 172.21.15.78:0/2988001185 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1033eb0 0x12f3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:56.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.259+0000 7f517087c700 1 -- 172.21.15.78:0/2988001185 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1033eb0 msgr2=0x12f3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:56.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.259+0000 7f517087c700 1 --2- 172.21.15.78:0/2988001185 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1033eb0 0x12f3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.259+0000 7f517087c700 1 -- 172.21.15.78:0/2988001185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1033ae0 msgr2=0x12f35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:56.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.259+0000 7f517087c700 1 --2- 172.21.15.78:0/2988001185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1033ae0 0x12f35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.259+0000 7f517087c700 1 -- 172.21.15.78:0/2988001185 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51680041b0 con 0x1033710 2023-10-11T09:42:56.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.259+0000 7f517087c700 1 --2- 172.21.15.78:0/2988001185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1033710 0x12a0150 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f515c009f40 tx=0x7f515c0562c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdea6dc37bf228b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:56.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.259+0000 7f514b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:42:56.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.259+0000 7f514b7fe700 1 -- 172.21.15.78:0/2988001185 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c055fa0 con 0x1033710 2023-10-11T09:42:56.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.259+0000 7f514b7fe700 1 -- 172.21.15.78:0/2988001185 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 6753+0+0 (secure 0 0 0) 0x7f515c059970 con 0x1033710 2023-10-11T09:42:56.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.259+0000 7f5175d6e740 1 -- 172.21.15.78:0/2988001185 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x12055a0 con 0x1033710 2023-10-11T09:42:56.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.259+0000 7f514b7fe700 1 -- 172.21.15.78:0/2988001185 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f515c05db20 con 0x1033710 2023-10-11T09:42:56.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.259+0000 7f5175d6e740 10 client.5958 fetch_fsmap learned FSMap version 29 2023-10-11T09:42:56.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.259+0000 7f5175d6e740 1 -- 172.21.15.78:0/2988001185 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x11d2750 con 0x1033710 2023-10-11T09:42:56.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.263+0000 7f514b7fe700 1 -- 172.21.15.78:0/2988001185 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x12055a0 con 0x1033710 2023-10-11T09:42:56.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.263+0000 7f514b7fe700 1 -- 172.21.15.78:0/2988001185 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f515c055030 con 0x1033710 2023-10-11T09:42:56.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.263+0000 7f5175d6e740 10 client.5958 fetch_fsmap finished waiting for FSMap version 29 2023-10-11T09:42:56.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.263+0000 7f5175d6e740 10 client.5958 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-11T09:42:56.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.263+0000 7f5175d6e740 20 client.5958 populate_metadata read hostname 'smithi078' 2023-10-11T09:42:56.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.263+0000 7f5175d6e740 1 --2- 172.21.15.78:0/2988001185 >> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] conn(0x107e790 0x12f22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:56.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.263+0000 7f5175d6e740 4 client.5958 mds_command: new command op to 5922 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-11T09:42:56.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.263+0000 7f5175d6e740 1 -- 172.21.15.78:0/2988001185 --> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x12052c0 con 0x107e790 2023-10-11T09:42:56.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.263+0000 7f5163fff700 1 --2- 172.21.15.78:0/2988001185 >> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] conn(0x107e790 0x12f22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:56.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.263+0000 7f5163fff700 1 --2- 172.21.15.78:0/2988001185 >> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] conn(0x107e790 0x12f22f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=767ee306aedbf6d1 server_cookie=815b9646c261d41c in_seq=0 out_seq=0 2023-10-11T09:42:56.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.263+0000 7f514b7fe700 10 client.5958 ms_handle_connect on v2:172.21.15.78:6835/3137573764 2023-10-11T09:42:56.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.263+0000 7f514b7fe700 1 -- 172.21.15.78:0/2988001185 <== mds.0 v2:172.21.15.78:6835/3137573764 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x12052c0 con 0x107e790 2023-10-11T09:42:56.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.263+0000 7f514b7fe700 10 client.5958 handle_command_reply: tid=0 2023-10-11T09:42:56.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.263+0000 7f5175d6e740 1 client.5958 shutdown 2023-10-11T09:42:56.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.263+0000 7f5175d6e740 1 -- 172.21.15.78:0/2988001185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1033710 msgr2=0x12a0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:56.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.263+0000 7f5175d6e740 1 --2- 172.21.15.78:0/2988001185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1033710 0x12a0150 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f515c009f40 tx=0x7f515c0562c0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.263+0000 7f5175d6e740 1 -- 172.21.15.78:0/2988001185 shutdown_connections 2023-10-11T09:42:56.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.263+0000 7f5175d6e740 1 --2- 172.21.15.78:0/2988001185 >> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] conn(0x107e790 0x12f22f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.263+0000 7f5175d6e740 1 --2- 172.21.15.78:0/2988001185 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1033eb0 0x12f3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.263+0000 7f5175d6e740 1 --2- 172.21.15.78:0/2988001185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1033ae0 0x12f35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.263+0000 7f514b7fe700 0 client.5958 ms_handle_reset on v2:172.21.15.78:6835/3137573764 2023-10-11T09:42:56.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.263+0000 7f5175d6e740 1 --2- 172.21.15.78:0/2988001185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1033710 0x12a0150 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.263+0000 7f5175d6e740 1 -- 172.21.15.78:0/2988001185 >> 172.21.15.78:0/2988001185 conn(0x129bfa0 msgr2=0x12cba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:56.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.263+0000 7f5175d6e740 1 -- 172.21.15.78:0/2988001185 shutdown_connections 2023-10-11T09:42:56.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.263+0000 7f5175d6e740 1 -- 172.21.15.78:0/2988001185 wait complete. 2023-10-11T09:42:56.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.263+0000 7f5175d6e740 20 client.5958 trim_cache size 0 max 16384 2023-10-11T09:42:56.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.283+0000 7f5175d6e740 1 Processor -- start 2023-10-11T09:42:56.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.287+0000 7f5175d6e740 1 -- start start 2023-10-11T09:42:56.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.287+0000 7f5175d6e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1033710 0x12c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:56.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.287+0000 7f5175d6e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1033ae0 0x12c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:56.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.287+0000 7f5175d6e740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1033eb0 0x12f32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:56.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.287+0000 7f5175d6e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x11d7c10 con 0x1033710 2023-10-11T09:42:56.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.287+0000 7f5175d6e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x11cfb00 con 0x1033ae0 2023-10-11T09:42:56.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.287+0000 7f5175d6e740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x11c6760 con 0x1033eb0 2023-10-11T09:42:56.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.287+0000 7f517087c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1033710 0x12c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:56.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.287+0000 7f517087c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1033710 0x12c9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39016/0 (socket says 172.21.15.78:39016) 2023-10-11T09:42:56.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.287+0000 7f517087c700 1 -- 172.21.15.78:0/3435663400 learned_addr learned my addr 172.21.15.78:0/3435663400 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:56.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.287+0000 7f517107d700 1 --2- 172.21.15.78:0/3435663400 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1033eb0 0x12f32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:56.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.287+0000 7f517087c700 1 -- 172.21.15.78:0/3435663400 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1033eb0 msgr2=0x12f32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:56.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.287+0000 7f5163fff700 1 --2- 172.21.15.78:0/3435663400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1033ae0 0x12c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:56.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.287+0000 7f517087c700 1 --2- 172.21.15.78:0/3435663400 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1033eb0 0x12f32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.287+0000 7f517087c700 1 -- 172.21.15.78:0/3435663400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1033ae0 msgr2=0x12c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:56.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.287+0000 7f517087c700 1 --2- 172.21.15.78:0/3435663400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1033ae0 0x12c9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.287+0000 7f517087c700 1 -- 172.21.15.78:0/3435663400 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f516c0cb410 con 0x1033710 2023-10-11T09:42:56.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.287+0000 7f5163fff700 1 --2- 172.21.15.78:0/3435663400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1033ae0 0x12c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:42:56.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.287+0000 7f517107d700 1 --2- 172.21.15.78:0/3435663400 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1033eb0 0x12f32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:42:56.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.287+0000 7f517087c700 1 --2- 172.21.15.78:0/3435663400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1033710 0x12c9a10 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f515c05d5f0 tx=0x7f515c059000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6bd09716e94a28 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:56.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.287+0000 7f5152ffd700 1 -- 172.21.15.78:0/3435663400 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c05db20 con 0x1033710 2023-10-11T09:42:56.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.287+0000 7f5152ffd700 1 -- 172.21.15.78:0/3435663400 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515c059970 con 0x1033710 2023-10-11T09:42:56.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.287+0000 7f5152ffd700 1 -- 172.21.15.78:0/3435663400 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c055da0 con 0x1033710 2023-10-11T09:42:56.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.287+0000 7f5175d6e740 1 -- 172.21.15.78:0/3435663400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1033710 msgr2=0x12c9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:56.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.287+0000 7f5175d6e740 1 --2- 172.21.15.78:0/3435663400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1033710 0x12c9a10 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f515c05d5f0 tx=0x7f515c059000 comp rx=0 tx=0).stop 2023-10-11T09:42:56.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.287+0000 7f5175d6e740 1 -- 172.21.15.78:0/3435663400 shutdown_connections 2023-10-11T09:42:56.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.287+0000 7f5175d6e740 1 --2- 172.21.15.78:0/3435663400 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1033eb0 0x12f32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.287+0000 7f5175d6e740 1 --2- 172.21.15.78:0/3435663400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1033ae0 0x12c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.287+0000 7f5175d6e740 1 --2- 172.21.15.78:0/3435663400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1033710 0x12c9a10 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.287+0000 7f5175d6e740 1 -- 172.21.15.78:0/3435663400 >> 172.21.15.78:0/3435663400 conn(0x129bfa0 msgr2=0x12c3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:56.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.287+0000 7f5175d6e740 1 -- 172.21.15.78:0/3435663400 shutdown_connections 2023-10-11T09:42:56.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.287+0000 7f5175d6e740 1 -- 172.21.15.78:0/3435663400 wait complete. 2023-10-11T09:42:56.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.291+0000 7f5175d6e740 1 -- start start 2023-10-11T09:42:56.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.291+0000 7f5175d6e740 1 Processor -- start 2023-10-11T09:42:56.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.291+0000 7f5175d6e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1033710 0x12c2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:56.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.291+0000 7f5175d6e740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1033ae0 0x134fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:56.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.291+0000 7f5175d6e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1033eb0 0x134fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:56.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.291+0000 7f5175d6e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x11d15c0 con 0x1033710 2023-10-11T09:42:56.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.291+0000 7f5175d6e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x12f15a0 con 0x1033eb0 2023-10-11T09:42:56.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.291+0000 7f5175d6e740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x12a49c0 con 0x1033ae0 2023-10-11T09:42:56.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.291+0000 7f5175d6e740 10 client.0 _ll_register_callbacks cb 0x120ac80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:42:56.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.295+0000 7f517087c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1033710 0x12c2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:56.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.295+0000 7f517087c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1033710 0x12c2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39022/0 (socket says 172.21.15.78:39022) 2023-10-11T09:42:56.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.295+0000 7f517087c700 1 -- 172.21.15.78:0/3816188800 learned_addr learned my addr 172.21.15.78:0/3816188800 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:56.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.295+0000 7f5163fff700 1 --2- 172.21.15.78:0/3816188800 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1033ae0 0x134fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:56.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.295+0000 7f517107d700 1 --2- 172.21.15.78:0/3816188800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1033eb0 0x134fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:56.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.295+0000 7f5163fff700 1 -- 172.21.15.78:0/3816188800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1033eb0 msgr2=0x134fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:56.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.295+0000 7f5163fff700 1 --2- 172.21.15.78:0/3816188800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1033eb0 0x134fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.295+0000 7f5163fff700 1 -- 172.21.15.78:0/3816188800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1033710 msgr2=0x12c2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:56.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.295+0000 7f5163fff700 1 --2- 172.21.15.78:0/3816188800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1033710 0x12c2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.295+0000 7f5163fff700 1 -- 172.21.15.78:0/3816188800 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x11d2750 con 0x1033ae0 2023-10-11T09:42:56.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.295+0000 7f517087c700 1 --2- 172.21.15.78:0/3816188800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1033710 0x12c2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:42:56.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.295+0000 7f5163fff700 1 --2- 172.21.15.78:0/3816188800 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1033ae0 0x134fab0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5168009930 tx=0x7f51680047e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20d161b921888f70 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:56.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.295+0000 7f514b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.133:3301/0 2023-10-11T09:42:56.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.295+0000 7f514b7fe700 1 -- 172.21.15.78:0/3816188800 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5168032040 con 0x1033ae0 2023-10-11T09:42:56.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.295+0000 7f514b7fe700 1 -- 172.21.15.78:0/3816188800 <== mon.2 v2:172.21.15.133:3301/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 6753+0+0 (secure 0 0 0) 0x7f5168003d60 con 0x1033ae0 2023-10-11T09:42:56.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.295+0000 7f5175d6e740 1 -- 172.21.15.78:0/3816188800 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51680036c0 con 0x1033ae0 2023-10-11T09:42:56.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.295+0000 7f514b7fe700 1 -- 172.21.15.78:0/3816188800 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5168038420 con 0x1033ae0 2023-10-11T09:42:56.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.295+0000 7f5175d6e740 10 client.5381 fetch_fsmap learned FSMap version 29 2023-10-11T09:42:56.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.295+0000 7f5175d6e740 1 -- 172.21.15.78:0/3816188800 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x11b8980 con 0x1033ae0 2023-10-11T09:42:56.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.295+0000 7f514b7fe700 1 -- 172.21.15.78:0/3816188800 <== mon.2 v2:172.21.15.133:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51680036c0 con 0x1033ae0 2023-10-11T09:42:56.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.295+0000 7f514b7fe700 1 -- 172.21.15.78:0/3816188800 <== mon.2 v2:172.21.15.133:3301/0 5 ==== fsmap(e 29) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f51680387f0 con 0x1033ae0 2023-10-11T09:42:56.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.295+0000 7f5175d6e740 10 client.5381 fetch_fsmap finished waiting for FSMap version 29 2023-10-11T09:42:56.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.295+0000 7f5175d6e740 10 client.5381 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-11T09:42:56.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.295+0000 7f5175d6e740 20 client.5381 populate_metadata read hostname 'smithi078' 2023-10-11T09:42:56.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.295+0000 7f5175d6e740 1 --2- 172.21.15.78:0/3816188800 >> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] conn(0x12a2bd0 0x12a2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:56.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.295+0000 7f5175d6e740 4 client.5381 mds_command: new command op to 5922 tid=0[{"prefix": "status"}] 2023-10-11T09:42:56.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.295+0000 7f5175d6e740 1 -- 172.21.15.78:0/3816188800 --> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] -- command(tid 0: {"prefix": "status"}) v1 -- 0x108b4e0 con 0x12a2bd0 2023-10-11T09:42:56.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.295+0000 7f517087c700 1 --2- 172.21.15.78:0/3816188800 >> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] conn(0x12a2bd0 0x12a2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:56.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.299+0000 7f517087c700 1 --2- 172.21.15.78:0/3816188800 >> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] conn(0x12a2bd0 0x12a2fa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59a42e54ed2c461e server_cookie=80406d0d0a6f7bdb in_seq=0 out_seq=0 2023-10-11T09:42:56.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.299+0000 7f514b7fe700 10 client.5381 ms_handle_connect on v2:172.21.15.78:6835/3137573764 2023-10-11T09:42:56.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.299+0000 7f514b7fe700 1 -- 172.21.15.78:0/3816188800 <== mds.0 v2:172.21.15.78:6835/3137573764 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x108b4e0 con 0x12a2bd0 2023-10-11T09:42:56.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.299+0000 7f514b7fe700 10 client.5381 handle_command_reply: tid=0 2023-10-11T09:42:56.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.299+0000 7f5175d6e740 1 client.5381 shutdown 2023-10-11T09:42:56.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.299+0000 7f5175d6e740 1 -- 172.21.15.78:0/3816188800 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1033ae0 msgr2=0x134fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:56.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.299+0000 7f5175d6e740 1 --2- 172.21.15.78:0/3816188800 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1033ae0 0x134fab0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5168009930 tx=0x7f51680047e0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.299+0000 7f5175d6e740 1 -- 172.21.15.78:0/3816188800 shutdown_connections 2023-10-11T09:42:56.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.299+0000 7f5175d6e740 1 --2- 172.21.15.78:0/3816188800 >> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] conn(0x12a2bd0 0x12a2fa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.299+0000 7f5175d6e740 1 --2- 172.21.15.78:0/3816188800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1033eb0 0x134fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.299+0000 7f5175d6e740 1 --2- 172.21.15.78:0/3816188800 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1033ae0 0x134fab0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.299+0000 7f514b7fe700 0 client.5381 ms_handle_reset on v2:172.21.15.78:6835/3137573764 2023-10-11T09:42:56.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.299+0000 7f5175d6e740 1 --2- 172.21.15.78:0/3816188800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1033710 0x12c2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.299+0000 7f5175d6e740 1 -- 172.21.15.78:0/3816188800 >> 172.21.15.78:0/3816188800 conn(0x129bfa0 msgr2=0x12e30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:56.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.299+0000 7f5175d6e740 1 -- 172.21.15.78:0/3816188800 shutdown_connections 2023-10-11T09:42:56.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.299+0000 7f5175d6e740 1 -- 172.21.15.78:0/3816188800 wait complete. 2023-10-11T09:42:56.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.299+0000 7f5175d6e740 20 client.5381 trim_cache size 0 max 16384 2023-10-11T09:42:56.312 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:42:56.312 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "dee99db9-7a72-406f-a1e2-917943f2b13d", 2023-10-11T09:42:56.313 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2023-10-11T09:42:56.313 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 5922, 2023-10-11T09:42:56.313 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2023-10-11T09:42:56.313 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2023-10-11T09:42:56.313 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2023-10-11T09:42:56.314 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 4.5937035420000001, 2023-10-11T09:42:56.314 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 29, 2023-10-11T09:42:56.314 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 48, 2023-10-11T09:42:56.314 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 48, 2023-10-11T09:42:56.314 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 4.6466176749999999 2023-10-11T09:42:56.315 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:42:56.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.299+0000 7f517187e700 1 -- 172.21.15.78:0/1781409210 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5158065ed0 msgr2=0x7f5158068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:56.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.299+0000 7f517187e700 1 --2- 172.21.15.78:0/1781409210 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5158065ed0 0x7f5158068380 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f516c0b0b10 tx=0x7f5168009240 comp rx=0 tx=0).stop 2023-10-11T09:42:56.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.299+0000 7f517187e700 1 -- 172.21.15.78:0/1781409210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f516c06d610 msgr2=0x7f516c13f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:56.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.299+0000 7f517187e700 1 --2- 172.21.15.78:0/1781409210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f516c06d610 0x7f516c13f850 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f516c072460 tx=0x7f515c004770 comp rx=0 tx=0).stop 2023-10-11T09:42:56.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.299+0000 7f517187e700 1 -- 172.21.15.78:0/1781409210 shutdown_connections 2023-10-11T09:42:56.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.299+0000 7f517187e700 1 --2- 172.21.15.78:0/1781409210 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5158065ed0 0x7f5158068380 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.299+0000 7f517187e700 1 --2- 172.21.15.78:0/1781409210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f516c15b8e0 0x7f516c0b37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.299+0000 7f517187e700 1 --2- 172.21.15.78:0/1781409210 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f516c15a940 0x7f516c151ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.299+0000 7f517187e700 1 --2- 172.21.15.78:0/1781409210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f516c06d610 0x7f516c13f850 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.299+0000 7f517187e700 1 -- 172.21.15.78:0/1781409210 >> 172.21.15.78:0/1781409210 conn(0x7f516c0cea60 msgr2=0x7f516c15d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:56.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.299+0000 7f517187e700 1 -- 172.21.15.78:0/1781409210 shutdown_connections 2023-10-11T09:42:56.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.299+0000 7f517187e700 1 -- 172.21.15.78:0/1781409210 wait complete. 2023-10-11T09:42:56.326 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:42:56.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.495+0000 7f69f2fd2700 1 Processor -- start 2023-10-11T09:42:56.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.495+0000 7f69f2fd2700 1 -- start start 2023-10-11T09:42:56.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.499+0000 7f69f2fd2700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f69ec160d40 0x7f69ec0cd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:56.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.499+0000 7f69f2fd2700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69ec06ffa0 con 0x7f69ec0ce500 2023-10-11T09:42:56.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.499+0000 7f69f2fd2700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69ec070280 con 0x7f69ec071040 2023-10-11T09:42:56.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.499+0000 7f69f2fd2700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69ec070c20 con 0x7f69ec160d40 2023-10-11T09:42:56.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.499+0000 7f69f156f700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f69ec0ce500 0x7f69ec06be60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:56384/0 (socket says 172.21.15.78:56384) 2023-10-11T09:42:56.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.499+0000 7f69f156f700 1 -- 172.21.15.78:0/3828003470 learned_addr learned my addr 172.21.15.78:0/3828003470 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:42:56.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.499+0000 7f69f0d6e700 1 --2- 172.21.15.78:0/3828003470 >> v2:172.21.15.133:3300/0 conn(0x7f69ec160d40 0x7f69ec0cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:56.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.499+0000 7f69e37fe700 1 -- 172.21.15.78:0/3828003470 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4048986128 0 0) 0x7f69ec06ffa0 con 0x7f69ec0ce500 2023-10-11T09:42:56.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.499+0000 7f69e37fe700 1 -- 172.21.15.78:0/3828003470 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69d40035d0 con 0x7f69ec0ce500 2023-10-11T09:42:56.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.499+0000 7f69e37fe700 1 -- 172.21.15.78:0/3828003470 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1011750354 0 0) 0x7f69ec070280 con 0x7f69ec071040 2023-10-11T09:42:56.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.499+0000 7f69e37fe700 1 -- 172.21.15.78:0/3828003470 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69ec06ffa0 con 0x7f69ec071040 2023-10-11T09:42:56.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.499+0000 7f69f0d6e700 1 -- 172.21.15.78:0/3828003470 >> v1:172.21.15.133:6789/0 conn(0x7f69ec071040 legacy=0x7f69ec0cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:56.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.499+0000 7f69f0d6e700 1 -- 172.21.15.78:0/3828003470 >> v1:172.21.15.78:6789/0 conn(0x7f69ec0ce500 legacy=0x7f69ec06be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:56.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.499+0000 7f69f0d6e700 1 -- 172.21.15.78:0/3828003470 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69ec070560 con 0x7f69ec160d40 2023-10-11T09:42:56.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.503+0000 7f69f0d6e700 1 --2- 172.21.15.78:0/3828003470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69ec160d40 0x7f69ec0cd280 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f69e8009850 tx=0x7f69e8009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93400fdbd8a48c2e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:56.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.503+0000 7f69e37fe700 1 -- 172.21.15.78:0/3828003470 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e8011830 con 0x7f69ec160d40 2023-10-11T09:42:56.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.503+0000 7f69e37fe700 1 -- 172.21.15.78:0/3828003470 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e8011990 con 0x7f69ec160d40 2023-10-11T09:42:56.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.503+0000 7f69e37fe700 1 -- 172.21.15.78:0/3828003470 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e8011d40 con 0x7f69ec160d40 2023-10-11T09:42:56.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.503+0000 7f69f2fd2700 1 -- 172.21.15.78:0/3828003470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69ec160d40 msgr2=0x7f69ec0cd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:56.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.503+0000 7f69f2fd2700 1 --2- 172.21.15.78:0/3828003470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69ec160d40 0x7f69ec0cd280 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f69e8009850 tx=0x7f69e8009b60 comp rx=0 tx=0).stop 2023-10-11T09:42:56.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.503+0000 7f69f2fd2700 1 -- 172.21.15.78:0/3828003470 shutdown_connections 2023-10-11T09:42:56.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.503+0000 7f69f2fd2700 1 --2- 172.21.15.78:0/3828003470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69ec160d40 0x7f69ec0cd280 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.503+0000 7f69f2fd2700 1 -- 172.21.15.78:0/3828003470 >> 172.21.15.78:0/3828003470 conn(0x7f69ec06aca0 msgr2=0x7f69ec0b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:56.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.503+0000 7f69f2fd2700 1 -- 172.21.15.78:0/3828003470 shutdown_connections 2023-10-11T09:42:56.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.503+0000 7f69f2fd2700 1 -- 172.21.15.78:0/3828003470 wait complete. 2023-10-11T09:42:56.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.503+0000 7f69f2fd2700 1 Processor -- start 2023-10-11T09:42:56.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.507+0000 7f69f2fd2700 1 -- start start 2023-10-11T09:42:56.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.507+0000 7f69f2fd2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69ec071040 0x7f69ec1fa800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:56.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.507+0000 7f69f2fd2700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69ec0ce500 0x7f69ec0b4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:56.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.507+0000 7f69f2fd2700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f69ec160d40 0x7f69ec0b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:56.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.507+0000 7f69f2fd2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec0b4100 con 0x7f69ec071040 2023-10-11T09:42:56.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.507+0000 7f69f2fd2700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec0b3220 con 0x7f69ec0ce500 2023-10-11T09:42:56.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.507+0000 7f69f2fd2700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec0b34f0 con 0x7f69ec160d40 2023-10-11T09:42:56.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.507+0000 7f69f0d6e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69ec071040 0x7f69ec1fa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:56.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.507+0000 7f69f0d6e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69ec071040 0x7f69ec1fa800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39034/0 (socket says 172.21.15.78:39034) 2023-10-11T09:42:56.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.507+0000 7f69f0d6e700 1 -- 172.21.15.78:0/4249817269 learned_addr learned my addr 172.21.15.78:0/4249817269 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:56.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.507+0000 7f69f156f700 1 --2- 172.21.15.78:0/4249817269 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f69ec160d40 0x7f69ec0b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:56.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.507+0000 7f69e3fff700 1 --2- 172.21.15.78:0/4249817269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69ec0ce500 0x7f69ec0b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:56.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.507+0000 7f69f0d6e700 1 -- 172.21.15.78:0/4249817269 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f69ec160d40 msgr2=0x7f69ec0b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:56.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.507+0000 7f69f0d6e700 1 --2- 172.21.15.78:0/4249817269 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f69ec160d40 0x7f69ec0b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.507+0000 7f69f0d6e700 1 -- 172.21.15.78:0/4249817269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69ec0ce500 msgr2=0x7f69ec0b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:56.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.507+0000 7f69f0d6e700 1 --2- 172.21.15.78:0/4249817269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69ec0ce500 0x7f69ec0b4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.507+0000 7f69f0d6e700 1 -- 172.21.15.78:0/4249817269 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e80094b0 con 0x7f69ec071040 2023-10-11T09:42:56.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.507+0000 7f69f0d6e700 1 --2- 172.21.15.78:0/4249817269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69ec071040 0x7f69ec1fa800 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f69e8014040 tx=0x7f69e8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:56.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.507+0000 7f69e17fa700 1 -- 172.21.15.78:0/4249817269 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e8004590 con 0x7f69ec071040 2023-10-11T09:42:56.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.507+0000 7f69e17fa700 1 -- 172.21.15.78:0/4249817269 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e801e070 con 0x7f69ec071040 2023-10-11T09:42:56.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.507+0000 7f69f2fd2700 1 -- 172.21.15.78:0/4249817269 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69ec0b37d0 con 0x7f69ec071040 2023-10-11T09:42:56.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.507+0000 7f69e17fa700 1 -- 172.21.15.78:0/4249817269 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e8003900 con 0x7f69ec071040 2023-10-11T09:42:56.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.507+0000 7f69f2fd2700 1 -- 172.21.15.78:0/4249817269 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69ec0b3d90 con 0x7f69ec071040 2023-10-11T09:42:56.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.511+0000 7f69f2fd2700 1 -- 172.21.15.78:0/4249817269 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69ec072250 con 0x7f69ec071040 2023-10-11T09:42:56.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.511+0000 7f69e17fa700 1 -- 172.21.15.78:0/4249817269 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f69e8003a60 con 0x7f69ec071040 2023-10-11T09:42:56.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.515+0000 7f69e17fa700 1 --2- 172.21.15.78:0/4249817269 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f69d8065ed0 0x7f69d8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:56.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.515+0000 7f69e17fa700 1 -- 172.21.15.78:0/4249817269 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6753+0+0 (secure 0 0 0) 0x7f69e8084940 con 0x7f69ec071040 2023-10-11T09:42:56.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.515+0000 7f69e3fff700 1 --2- 172.21.15.78:0/4249817269 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f69d8065ed0 0x7f69d8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:56.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.515+0000 7f69e3fff700 1 --2- 172.21.15.78:0/4249817269 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f69d8065ed0 0x7f69d8068380 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f69ec150ad0 tx=0x7f69dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:56.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.519+0000 7f69e17fa700 1 -- 172.21.15.78:0/4249817269 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f69e8051cc0 con 0x7f69ec071040 2023-10-11T09:42:56.711 INFO:tasks.ceph.mds.c.smithi078.stderr:2023-10-11T09:42:56.707+0000 7f690a586700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-11T09:42:56.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.727+0000 7f69f2fd2700 1 -- 172.21.15.78:0/4249817269 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69ec04e530 con 0x7f69ec071040 2023-10-11T09:42:56.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.727+0000 7f69e17fa700 1 -- 172.21.15.78:0/4249817269 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4654 (secure 0 0 0) 0x7f69e80552e0 con 0x7f69ec071040 2023-10-11T09:42:56.733 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:42:56.734 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":29,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/588936111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":588936111},{"type":"v1","addr":"172.21.15.78:6836","nonce":588936111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5357,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/2574909888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2574909888},{"type":"v1","addr":"172.21.15.133:6834","nonce":2574909888}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5791,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2948223101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":2948223101},{"type":"v1","addr":"172.21.15.133:6835","nonce":2948223101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:42:45.491515+0000","modified":"2023-10-11T09:42:54.734536+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5922":{"gid":5922,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":4,"addr":"172.21.15.78:6837/3137573764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":3137573764},{"type":"v1","addr":"172.21.15.78:6837","nonce":3137573764}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-11T09:42:56.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.735+0000 7f69f2fd2700 1 -- 172.21.15.78:0/4249817269 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f69d8065ed0 msgr2=0x7f69d8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:56.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.735+0000 7f69f2fd2700 1 --2- 172.21.15.78:0/4249817269 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f69d8065ed0 0x7f69d8068380 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f69ec150ad0 tx=0x7f69dc009040 comp rx=0 tx=0).stop 2023-10-11T09:42:56.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.735+0000 7f69f2fd2700 1 -- 172.21.15.78:0/4249817269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69ec071040 msgr2=0x7f69ec1fa800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:56.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.735+0000 7f69f2fd2700 1 --2- 172.21.15.78:0/4249817269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69ec071040 0x7f69ec1fa800 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f69e8014040 tx=0x7f69e8004770 comp rx=0 tx=0).stop 2023-10-11T09:42:56.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.735+0000 7f69f2fd2700 1 -- 172.21.15.78:0/4249817269 shutdown_connections 2023-10-11T09:42:56.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.735+0000 7f69f2fd2700 1 --2- 172.21.15.78:0/4249817269 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f69d8065ed0 0x7f69d8068380 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.735+0000 7f69f2fd2700 1 --2- 172.21.15.78:0/4249817269 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f69ec160d40 0x7f69ec0b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.735+0000 7f69f2fd2700 1 --2- 172.21.15.78:0/4249817269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69ec0ce500 0x7f69ec0b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.735+0000 7f69f2fd2700 1 --2- 172.21.15.78:0/4249817269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69ec071040 0x7f69ec1fa800 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.735+0000 7f69f2fd2700 1 -- 172.21.15.78:0/4249817269 >> 172.21.15.78:0/4249817269 conn(0x7f69ec06aca0 msgr2=0x7f69ec161110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:56.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.735+0000 7f69f2fd2700 1 -- 172.21.15.78:0/4249817269 shutdown_connections 2023-10-11T09:42:56.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.735+0000 7f69f2fd2700 1 -- 172.21.15.78:0/4249817269 wait complete. 2023-10-11T09:42:56.741 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 29 2023-10-11T09:42:56.761 INFO:tasks.cephfs.test_data_scan:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 2, "epoch": 29, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-11T09:42:45.491515+0000", "damaged": [], "data_pools": [ 6, 7 ], "enabled": true, "epoch": 29, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5922": { "addr": "172.21.15.78:6837/3137573764", "addrs": { "addrvec": [ { "addr": "172.21.15.78:6835", "nonce": 3137573764, "type": "v2" }, { "addr": "172.21.15.78:6837", "nonce": 3137573764, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5922, "incarnation": 26, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 4 } }, "last_failure": 0, "last_failure_osd_epoch": 48, "max_file_size": 1099511627776, "max_mds": 1, "metadata_pool": 5, "modified": "2023-10-11T09:42:54.734536+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5922 } } } ], "standbys": [ { "addr": "172.21.15.78:6836/588936111", "addrs": { "addrvec": [ { "addr": "172.21.15.78:6834", "nonce": 588936111, "type": "v2" }, { "addr": "172.21.15.78:6836", "nonce": 588936111, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 25, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5348, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.133:6834/2574909888", "addrs": { "addrvec": [ { "addr": "172.21.15.133:6832", "nonce": 2574909888, "type": "v2" }, { "addr": "172.21.15.133:6834", "nonce": 2574909888, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 25, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5357, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.133:6835/2948223101", "addrs": { "addrvec": [ { "addr": "172.21.15.133:6833", "nonce": 2948223101, "type": "v2" }, { "addr": "172.21.15.133:6835", "nonce": 2948223101, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 25, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5791, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-11T09:42:56.762 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-11T09:42:56.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.927+0000 7f18ba271700 1 Processor -- start 2023-10-11T09:42:56.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.931+0000 7f18ba271700 1 -- start start 2023-10-11T09:42:56.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.931+0000 7f18ba271700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f18b40b2310 0x7f18b4161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:56.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.931+0000 7f18ba271700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f18b40b4b60 0x7f18b40d1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:56.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.931+0000 7f18ba271700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18b40d11e0 con 0x7f18b40d21e0 2023-10-11T09:42:56.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.931+0000 7f18ba271700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18b40b5370 con 0x7f18b40b4b60 2023-10-11T09:42:56.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.931+0000 7f18ba271700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18b40d14a0 con 0x7f18b40b2310 2023-10-11T09:42:56.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.931+0000 7f18abfff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f18b40b4b60 0x7f18b40d1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:56.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.931+0000 7f18abfff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f18b40b4b60 0x7f18b40d1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39048/0 (socket says 172.21.15.78:39048) 2023-10-11T09:42:56.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.931+0000 7f18abfff700 1 -- 172.21.15.78:0/1382667283 learned_addr learned my addr 172.21.15.78:0/1382667283 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:56.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.931+0000 7f18b3fff700 1 --2- 172.21.15.78:0/1382667283 >> v2:172.21.15.133:3300/0 conn(0x7f18b40b2310 0x7f18b4161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:56.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.931+0000 7f18abfff700 1 -- 172.21.15.78:0/1382667283 >> v2:172.21.15.133:3300/0 conn(0x7f18b40b2310 msgr2=0x7f18b4161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:56.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.931+0000 7f18abfff700 1 --2- 172.21.15.78:0/1382667283 >> v2:172.21.15.133:3300/0 conn(0x7f18b40b2310 0x7f18b4161150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.931+0000 7f18abfff700 1 -- 172.21.15.78:0/1382667283 >> v1:172.21.15.78:6789/0 conn(0x7f18b40d21e0 legacy=0x7f18b40b5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:56.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.931+0000 7f18abfff700 1 -- 172.21.15.78:0/1382667283 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b40d1780 con 0x7f18b40b4b60 2023-10-11T09:42:56.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.931+0000 7f18abfff700 1 --2- 172.21.15.78:0/1382667283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18b40b4b60 0x7f18b40d1ca0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f189c009910 tx=0x7f189c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66358466ee75656 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:56.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.935+0000 7f18b37fe700 1 -- 172.21.15.78:0/1382667283 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189c00ba20 con 0x7f18b40b4b60 2023-10-11T09:42:56.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.935+0000 7f18b37fe700 1 -- 172.21.15.78:0/1382667283 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f189c00bb80 con 0x7f18b40b4b60 2023-10-11T09:42:56.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.935+0000 7f18b37fe700 1 -- 172.21.15.78:0/1382667283 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189c009e60 con 0x7f18b40b4b60 2023-10-11T09:42:56.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.935+0000 7f18ba271700 1 -- 172.21.15.78:0/1382667283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18b40b4b60 msgr2=0x7f18b40d1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:56.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.935+0000 7f18ba271700 1 --2- 172.21.15.78:0/1382667283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18b40b4b60 0x7f18b40d1ca0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f189c009910 tx=0x7f189c009c20 comp rx=0 tx=0).stop 2023-10-11T09:42:56.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.935+0000 7f18ba271700 1 -- 172.21.15.78:0/1382667283 shutdown_connections 2023-10-11T09:42:56.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.935+0000 7f18ba271700 1 --2- 172.21.15.78:0/1382667283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18b40b4b60 0x7f18b40d1ca0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.935+0000 7f18ba271700 1 --2- 172.21.15.78:0/1382667283 >> v2:172.21.15.133:3300/0 conn(0x7f18b40b2310 0x7f18b4161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.935+0000 7f18ba271700 1 -- 172.21.15.78:0/1382667283 >> 172.21.15.78:0/1382667283 conn(0x7f18b40cfb60 msgr2=0x7f18b40cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:56.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.935+0000 7f18ba271700 1 -- 172.21.15.78:0/1382667283 shutdown_connections 2023-10-11T09:42:56.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.935+0000 7f18ba271700 1 -- 172.21.15.78:0/1382667283 wait complete. 2023-10-11T09:42:56.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.939+0000 7f18ba271700 1 Processor -- start 2023-10-11T09:42:56.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.939+0000 7f18ba271700 1 -- start start 2023-10-11T09:42:56.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.939+0000 7f18ba271700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f18b40b2310 0x7f18b41574d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:56.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.939+0000 7f18ba271700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18b40b4b60 0x7f18b4157a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:56.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.939+0000 7f18ba271700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18b40d21e0 0x7f18b415d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:56.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.939+0000 7f18ba271700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b41f59c0 con 0x7f18b40b4b60 2023-10-11T09:42:56.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.939+0000 7f18ba271700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b4156560 con 0x7f18b40d21e0 2023-10-11T09:42:56.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.939+0000 7f18ba271700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b41f5c10 con 0x7f18b40b2310 2023-10-11T09:42:56.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.939+0000 7f18abfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18b40b4b60 0x7f18b4157a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:56.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.939+0000 7f18abfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18b40b4b60 0x7f18b4157a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39058/0 (socket says 172.21.15.78:39058) 2023-10-11T09:42:56.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.939+0000 7f18abfff700 1 -- 172.21.15.78:0/2274825766 learned_addr learned my addr 172.21.15.78:0/2274825766 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:56.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.939+0000 7f18b3fff700 1 --2- 172.21.15.78:0/2274825766 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f18b40b2310 0x7f18b41574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:56.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.939+0000 7f18b880e700 1 --2- 172.21.15.78:0/2274825766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18b40d21e0 0x7f18b415d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:56.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.939+0000 7f18abfff700 1 -- 172.21.15.78:0/2274825766 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f18b40b2310 msgr2=0x7f18b41574d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:56.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.939+0000 7f18abfff700 1 --2- 172.21.15.78:0/2274825766 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f18b40b2310 0x7f18b41574d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.939+0000 7f18abfff700 1 -- 172.21.15.78:0/2274825766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18b40d21e0 msgr2=0x7f18b415d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:56.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.939+0000 7f18abfff700 1 --2- 172.21.15.78:0/2274825766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18b40d21e0 0x7f18b415d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:56.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.939+0000 7f18abfff700 1 -- 172.21.15.78:0/2274825766 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f189c009570 con 0x7f18b40b4b60 2023-10-11T09:42:56.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.939+0000 7f18abfff700 1 --2- 172.21.15.78:0/2274825766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18b40b4b60 0x7f18b4157a10 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f18b4067480 tx=0x7f189c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:56.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.943+0000 7f18b17fa700 1 -- 172.21.15.78:0/2274825766 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189c004310 con 0x7f18b40b4b60 2023-10-11T09:42:56.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.943+0000 7f18b17fa700 1 -- 172.21.15.78:0/2274825766 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f189c01f070 con 0x7f18b40b4b60 2023-10-11T09:42:56.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.943+0000 7f18b17fa700 1 -- 172.21.15.78:0/2274825766 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189c0036c0 con 0x7f18b40b4b60 2023-10-11T09:42:56.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.943+0000 7f18ba271700 1 -- 172.21.15.78:0/2274825766 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b40b5140 con 0x7f18b40b4b60 2023-10-11T09:42:56.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.943+0000 7f18ba271700 1 -- 172.21.15.78:0/2274825766 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b41f61d0 con 0x7f18b40b4b60 2023-10-11T09:42:56.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.943+0000 7f18ba271700 1 -- 172.21.15.78:0/2274825766 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18b404e530 con 0x7f18b40b4b60 2023-10-11T09:42:56.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.943+0000 7f18b17fa700 1 -- 172.21.15.78:0/2274825766 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f189c021030 con 0x7f18b40b4b60 2023-10-11T09:42:56.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.947+0000 7f18b17fa700 1 --2- 172.21.15.78:0/2274825766 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1894065f00 0x7f18940683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:56.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.947+0000 7f18b17fa700 1 -- 172.21.15.78:0/2274825766 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6753+0+0 (secure 0 0 0) 0x7f189c084e00 con 0x7f18b40b4b60 2023-10-11T09:42:56.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.947+0000 7f18b3fff700 1 --2- 172.21.15.78:0/2274825766 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1894065f00 0x7f18940683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:56.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.947+0000 7f18b3fff700 1 --2- 172.21.15.78:0/2274825766 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1894065f00 0x7f18940683b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f18a4009950 tx=0x7f18a4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:56.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:56.951+0000 7f18b17fa700 1 -- 172.21.15.78:0/2274825766 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f189c052180 con 0x7f18b40b4b60 2023-10-11T09:42:57.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:57.167+0000 7f18ba271700 1 -- 172.21.15.78:0/2274825766 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f18b40b4910 con 0x7f18b40b4b60 2023-10-11T09:42:57.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:57.167+0000 7f18b17fa700 1 -- 172.21.15.78:0/2274825766 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) v1 ==== 53+0+91 (secure 0 0 0) 0x7f189c015070 con 0x7f18b40b4b60 2023-10-11T09:42:57.172 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_extra ] 2023-10-11T09:42:57.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:57.171+0000 7f18ba271700 1 -- 172.21.15.78:0/2274825766 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1894065f00 msgr2=0x7f18940683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:57.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:57.171+0000 7f18ba271700 1 --2- 172.21.15.78:0/2274825766 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1894065f00 0x7f18940683b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f18a4009950 tx=0x7f18a4008040 comp rx=0 tx=0).stop 2023-10-11T09:42:57.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:57.171+0000 7f18ba271700 1 -- 172.21.15.78:0/2274825766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18b40b4b60 msgr2=0x7f18b4157a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:57.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:57.171+0000 7f18ba271700 1 --2- 172.21.15.78:0/2274825766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18b40b4b60 0x7f18b4157a10 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f18b4067480 tx=0x7f189c004770 comp rx=0 tx=0).stop 2023-10-11T09:42:57.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:57.171+0000 7f18ba271700 1 -- 172.21.15.78:0/2274825766 shutdown_connections 2023-10-11T09:42:57.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:57.171+0000 7f18ba271700 1 --2- 172.21.15.78:0/2274825766 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1894065f00 0x7f18940683b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:57.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:57.171+0000 7f18ba271700 1 --2- 172.21.15.78:0/2274825766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18b40d21e0 0x7f18b415d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:57.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:57.171+0000 7f18ba271700 1 --2- 172.21.15.78:0/2274825766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18b40b4b60 0x7f18b4157a10 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:57.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:57.171+0000 7f18ba271700 1 --2- 172.21.15.78:0/2274825766 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f18b40b2310 0x7f18b41574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:57.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:57.171+0000 7f18ba271700 1 -- 172.21.15.78:0/2274825766 >> 172.21.15.78:0/2274825766 conn(0x7f18b40cfb60 msgr2=0x7f18b40d2850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:57.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:57.171+0000 7f18ba271700 1 -- 172.21.15.78:0/2274825766 shutdown_connections 2023-10-11T09:42:57.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:57.171+0000 7f18ba271700 1 -- 172.21.15.78:0/2274825766 wait complete. 2023-10-11T09:42:57.194 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-11T09:42:57.195 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-11T09:42:57.195 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi078.front.sepia.ceph.com 2023-10-11T09:42:57.196 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-11T09:42:57.196 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-11T09:42:57.196 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-11T09:42:57.197 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-11T09:42:57.197 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-11T09:42:57.198 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-11T09:42:57.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:42:57.199 DEBUG:teuthology.orchestra.run.smithi078:> ip addr 2023-10-11T09:42:57.216 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-11T09:42:57.216 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-11T09:42:57.217 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2023-10-11T09:42:57.217 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:42:57.217 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2023-10-11T09:42:57.217 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:42:57.217 INFO:teuthology.orchestra.run.smithi078.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:42:57.218 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:22 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:42:57.218 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f0 2023-10-11T09:42:57.218 INFO:teuthology.orchestra.run.smithi078.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:42:57.218 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:23 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:42:57.219 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f1 2023-10-11T09:42:57.219 INFO:teuthology.orchestra.run.smithi078.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:42:57.219 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dc brd ff:ff:ff:ff:ff:ff 2023-10-11T09:42:57.219 INFO:teuthology.orchestra.run.smithi078.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-11T09:42:57.219 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dd brd ff:ff:ff:ff:ff:ff 2023-10-11T09:42:57.220 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.21.15.78/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-11T09:42:57.220 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft 42287sec preferred_lft 42287sec 2023-10-11T09:42:57.220 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::ec4:7aff:fe8f:d4dd/64 scope link 2023-10-11T09:42:57.220 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:42:57.220 INFO:teuthology.orchestra.run.smithi078.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-11T09:42:57.221 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 02:42:b6:e4:f8:0d brd ff:ff:ff:ff:ff:ff 2023-10-11T09:42:57.221 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-11T09:42:57.221 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:42:57.221 INFO:teuthology.orchestra.run.smithi078.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-11T09:42:57.221 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 7a:89:8a:a9:e1:23 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:42:57.222 INFO:teuthology.orchestra.run.smithi078.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-11T09:42:57.222 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:42:57.222 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::8ca:ebff:fea9:13f1/64 scope link 2023-10-11T09:42:57.222 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:42:57.222 INFO:teuthology.orchestra.run.smithi078.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-11T09:42:57.222 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 7a:89:8a:a9:e1:23 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-11T09:42:57.223 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::7889:8aff:fea9:e123/64 scope link 2023-10-11T09:42:57.223 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:42:57.223 INFO:teuthology.orchestra.run.smithi078.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-11T09:42:57.223 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether ca:ac:e8:27:51:0d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-11T09:42:57.223 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::c8ac:e8ff:fe27:510d/64 scope link 2023-10-11T09:42:57.224 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:42:57.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:42:57.225 DEBUG:teuthology.orchestra.run.smithi078:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-11T09:42:57.307 INFO:teuthology.orchestra.run.smithi078.stdout:1 2023-10-11T09:42:57.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:42:57.310 DEBUG:teuthology.orchestra.run.smithi078:> route 2023-10-11T09:42:57.373 INFO:teuthology.orchestra.run.smithi078.stdout:Kernel IP routing table 2023-10-11T09:42:57.373 INFO:teuthology.orchestra.run.smithi078.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-11T09:42:57.373 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-11T09:42:57.373 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-11T09:42:57.374 INFO:teuthology.orchestra.run.smithi078.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-11T09:42:57.374 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-10-11T09:42:57.374 INFO:teuthology.orchestra.run.smithi078.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-11T09:42:57.374 INFO:teuthology.orchestra.run.smithi078.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-11T09:42:57.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:42:57.375 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec bash -c ' 2023-10-11T09:42:57.375 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2023-10-11T09:42:57.376 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-11T09:42:57.376 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-11T09:42:57.377 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-11T09:42:57.377 DEBUG:teuthology.orchestra.run.smithi078:> ') 2023-10-11T09:42:57.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:42:57.513 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2023-10-11T09:42:57.530 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-11T09:42:57.530 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-11T09:42:57.531 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-11T09:42:57.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:42:57.532 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set brx.0 up 2023-10-11T09:42:57.608 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-11T09:42:57.608 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T09:42:57.609 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-11T09:42:57.666 INFO:teuthology.orchestra.run.smithi078.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-11T09:42:57.667 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T09:42:57.667 DEBUG:teuthology.orchestra.run.smithi078:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-11T09:42:57.730 DEBUG:teuthology.orchestra.run.smithi078:> sudo modprobe fuse 2023-10-11T09:42:57.815 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T09:42:57.815 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T09:42:57.894 INFO:teuthology.orchestra.run.smithi078.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T09:42:57.895 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T09:42:57.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:42:57.897 DEBUG:teuthology.orchestra.run.smithi078:> ls /sys/fs/fuse/connections 2023-10-11T09:42:57.954 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-11T09:42:57.954 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-11T09:42:58.003 DEBUG:teuthology.orchestra.run.smithi078:> sudo modprobe fuse 2023-10-11T09:42:58.040 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T09:42:58.041 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T09:42:58.114 INFO:teuthology.orchestra.run.smithi078.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T09:42:58.115 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T09:42:58.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:42:58.117 DEBUG:teuthology.orchestra.run.smithi078:> ls /sys/fs/fuse/connections 2023-10-11T09:42:58.141 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:2023-10-11T09:42:58.135+0000 7f05eae7e640 -1 init, newargv = 0x55ccaad296c0 newargc=15 2023-10-11T09:42:58.141 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:ceph-fuse[47784]: starting ceph client 2023-10-11T09:42:58.151 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:ceph-fuse[47784]: starting fuse 2023-10-11T09:42:58.155 INFO:teuthology.orchestra.run.smithi078.stdout:49 2023-10-11T09:42:58.156 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2023-10-11T09:42:58.156 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:42:58.157 DEBUG:teuthology.orchestra.run.smithi078:> import glob 2023-10-11T09:42:58.157 DEBUG:teuthology.orchestra.run.smithi078:> import re 2023-10-11T09:42:58.157 DEBUG:teuthology.orchestra.run.smithi078:> import os 2023-10-11T09:42:58.158 DEBUG:teuthology.orchestra.run.smithi078:> import subprocess 2023-10-11T09:42:58.158 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:42:58.159 DEBUG:teuthology.orchestra.run.smithi078:> def _find_admin_socket(client_name): 2023-10-11T09:42:58.159 DEBUG:teuthology.orchestra.run.smithi078:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-11T09:42:58.159 DEBUG:teuthology.orchestra.run.smithi078:> files = glob.glob(asok_path) 2023-10-11T09:42:58.160 DEBUG:teuthology.orchestra.run.smithi078:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-11T09:42:58.160 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:42:58.160 DEBUG:teuthology.orchestra.run.smithi078:> # Given a non-glob path, it better be there 2023-10-11T09:42:58.161 DEBUG:teuthology.orchestra.run.smithi078:> if "*" not in asok_path: 2023-10-11T09:42:58.161 DEBUG:teuthology.orchestra.run.smithi078:> assert(len(files) == 1) 2023-10-11T09:42:58.162 DEBUG:teuthology.orchestra.run.smithi078:> return files[0] 2023-10-11T09:42:58.162 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:42:58.162 DEBUG:teuthology.orchestra.run.smithi078:> for f in files: 2023-10-11T09:42:58.163 DEBUG:teuthology.orchestra.run.smithi078:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-11T09:42:58.163 DEBUG:teuthology.orchestra.run.smithi078:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-11T09:42:58.163 DEBUG:teuthology.orchestra.run.smithi078:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-11T09:42:58.164 DEBUG:teuthology.orchestra.run.smithi078:> contents = proc_f.read() 2023-10-11T09:42:58.164 DEBUG:teuthology.orchestra.run.smithi078:> if mountpoint in contents: 2023-10-11T09:42:58.165 DEBUG:teuthology.orchestra.run.smithi078:> return f 2023-10-11T09:42:58.165 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-11T09:42:58.165 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:42:58.166 DEBUG:teuthology.orchestra.run.smithi078:> print(_find_admin_socket("client.0")) 2023-10-11T09:42:58.167 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:42:58.313 INFO:teuthology.orchestra.run.smithi078.stdout:/var/run/ceph/ceph-client.0.47784.asok 2023-10-11T09:42:58.496 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.47784.asok 2023-10-11T09:42:58.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:42:58.497 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.47784.asok status 2023-10-11T09:42:58.695 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:42:58.695 INFO:teuthology.orchestra.run.smithi078.stdout: "metadata": { 2023-10-11T09:42:58.695 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_sha1": "9a774796f0ecdd11a4b771de396e627aa1a187af", 2023-10-11T09:42:58.696 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_version": "ceph version 18.2.0-662-g9a774796 (9a774796f0ecdd11a4b771de396e627aa1a187af) reef (stable)", 2023-10-11T09:42:58.696 INFO:teuthology.orchestra.run.smithi078.stdout: "entity_id": "0", 2023-10-11T09:42:58.696 INFO:teuthology.orchestra.run.smithi078.stdout: "hostname": "smithi078", 2023-10-11T09:42:58.696 INFO:teuthology.orchestra.run.smithi078.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-11T09:42:58.696 INFO:teuthology.orchestra.run.smithi078.stdout: "pid": "47784", 2023-10-11T09:42:58.697 INFO:teuthology.orchestra.run.smithi078.stdout: "root": "/" 2023-10-11T09:42:58.697 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:42:58.697 INFO:teuthology.orchestra.run.smithi078.stdout: "dentry_count": 0, 2023-10-11T09:42:58.697 INFO:teuthology.orchestra.run.smithi078.stdout: "dentry_pinned_count": 0, 2023-10-11T09:42:58.697 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 5985, 2023-10-11T09:42:58.698 INFO:teuthology.orchestra.run.smithi078.stdout: "inst": { 2023-10-11T09:42:58.698 INFO:teuthology.orchestra.run.smithi078.stdout: "name": { 2023-10-11T09:42:58.698 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "client", 2023-10-11T09:42:58.698 INFO:teuthology.orchestra.run.smithi078.stdout: "num": 5985 2023-10-11T09:42:58.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:42:58.699 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": { 2023-10-11T09:42:58.699 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2023-10-11T09:42:58.699 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "192.168.0.1:0", 2023-10-11T09:42:58.699 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 952901233 2023-10-11T09:42:58.699 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:42:58.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:42:58.700 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": { 2023-10-11T09:42:58.700 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2023-10-11T09:42:58.700 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "192.168.0.1:0", 2023-10-11T09:42:58.700 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 952901233 2023-10-11T09:42:58.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:42:58.701 INFO:teuthology.orchestra.run.smithi078.stdout: "inst_str": "client.5985 192.168.0.1:0/952901233", 2023-10-11T09:42:58.701 INFO:teuthology.orchestra.run.smithi078.stdout: "addr_str": "192.168.0.1:0/952901233", 2023-10-11T09:42:58.701 INFO:teuthology.orchestra.run.smithi078.stdout: "inode_count": 1, 2023-10-11T09:42:58.701 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_epoch": 29, 2023-10-11T09:42:58.701 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_epoch": 48, 2023-10-11T09:42:58.702 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_epoch_barrier": 0, 2023-10-11T09:42:58.702 INFO:teuthology.orchestra.run.smithi078.stdout: "blocklisted": false, 2023-10-11T09:42:58.702 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs" 2023-10-11T09:42:58.702 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:42:58.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:42:58.710 DEBUG:teuthology.orchestra.run.smithi078:> stat --file-system '--printf=%T 2023-10-11T09:42:58.710 DEBUG:teuthology.orchestra.run.smithi078:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-11T09:42:58.729 INFO:teuthology.orchestra.run.smithi078.stdout:fuseblk 2023-10-11T09:42:58.729 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-11T09:42:58.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:42:58.730 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-11T09:42:58.811 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 scrub start / repair,recursive 2023-10-11T09:42:59.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.031+0000 7fe4f359e700 1 Processor -- start 2023-10-11T09:42:59.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.031+0000 7fe4f359e700 1 -- start start 2023-10-11T09:42:59.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.035+0000 7fe4f359e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4f415a0f0 0x7fe4f40cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.035+0000 7fe4f359e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe4f415cbf0 0x7fe4f415d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.035+0000 7fe4f359e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4f40d2bd0 con 0x7fe4f40b52e0 2023-10-11T09:42:59.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.035+0000 7fe4f359e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4f415d580 con 0x7fe4f415cbf0 2023-10-11T09:42:59.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.035+0000 7fe4f359e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f40cf630 con 0x7fe4f415a0f0 2023-10-11T09:42:59.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.035+0000 7fe4f2d9d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe4f415cbf0 0x7fe4f415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:59.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.035+0000 7fe4f2d9d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe4f415cbf0 0x7fe4f415d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39062/0 (socket says 172.21.15.78:39062) 2023-10-11T09:42:59.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.035+0000 7fe4f2d9d700 1 -- 172.21.15.78:0/1947169430 learned_addr learned my addr 172.21.15.78:0/1947169430 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:59.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.035+0000 7fe4f259c700 1 --2- 172.21.15.78:0/1947169430 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4f415a0f0 0x7fe4f40cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:59.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.035+0000 7fe4f2d9d700 1 -- 172.21.15.78:0/1947169430 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4f415a0f0 msgr2=0x7fe4f40cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:59.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.035+0000 7fe4f2d9d700 1 --2- 172.21.15.78:0/1947169430 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4f415a0f0 0x7fe4f40cefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.035+0000 7fe4f2d9d700 1 -- 172.21.15.78:0/1947169430 >> v1:172.21.15.133:6789/0 conn(0x7fe4f40b52e0 legacy=0x7fe4f41608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:59.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.035+0000 7fe4f2d9d700 1 -- 172.21.15.78:0/1947169430 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f4160410 con 0x7fe4f415cbf0 2023-10-11T09:42:59.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.035+0000 7fe4f2d9d700 1 --2- 172.21.15.78:0/1947169430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4f415cbf0 0x7fe4f415d040 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009e50 tx=0x7fe4ec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceedf7cd169aed01 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:59.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.035+0000 7fe4f159a700 1 -- 172.21.15.78:0/1947169430 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec00e070 con 0x7fe4f415cbf0 2023-10-11T09:42:59.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.039+0000 7fe4f159a700 1 -- 172.21.15.78:0/1947169430 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4ec00b9c0 con 0x7fe4f415cbf0 2023-10-11T09:42:59.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.039+0000 7fe4f159a700 1 -- 172.21.15.78:0/1947169430 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec00bd70 con 0x7fe4f415cbf0 2023-10-11T09:42:59.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.039+0000 7fe4f359e700 1 -- 172.21.15.78:0/1947169430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4f415cbf0 msgr2=0x7fe4f415d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:59.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.039+0000 7fe4f359e700 1 --2- 172.21.15.78:0/1947169430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4f415cbf0 0x7fe4f415d040 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009e50 tx=0x7fe4ec00b330 comp rx=0 tx=0).stop 2023-10-11T09:42:59.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.039+0000 7fe4f359e700 1 -- 172.21.15.78:0/1947169430 shutdown_connections 2023-10-11T09:42:59.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.039+0000 7fe4f359e700 1 --2- 172.21.15.78:0/1947169430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4f415cbf0 0x7fe4f415d040 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.039+0000 7fe4f359e700 1 --2- 172.21.15.78:0/1947169430 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4f415a0f0 0x7fe4f40cefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.039+0000 7fe4f359e700 1 -- 172.21.15.78:0/1947169430 >> 172.21.15.78:0/1947169430 conn(0x7fe4f406afd0 msgr2=0x7fe4f406b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:59.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.039+0000 7fe4f359e700 1 -- 172.21.15.78:0/1947169430 shutdown_connections 2023-10-11T09:42:59.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.039+0000 7fe4f359e700 1 -- 172.21.15.78:0/1947169430 wait complete. 2023-10-11T09:42:59.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.043+0000 7fe4f359e700 1 Processor -- start 2023-10-11T09:42:59.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.043+0000 7fe4f359e700 1 -- start start 2023-10-11T09:42:59.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.043+0000 7fe4f359e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4f40b52e0 0x7fe4f40b7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.043+0000 7fe4f359e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4f415a0f0 0x7fe4f415eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.043+0000 7fe4f359e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4f415cbf0 0x7fe4f415f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.043+0000 7fe4f359e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f4157b60 con 0x7fe4f415cbf0 2023-10-11T09:42:59.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.043+0000 7fe4f359e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f40b7f40 con 0x7fe4f415a0f0 2023-10-11T09:42:59.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.043+0000 7fe4f359e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f40b6220 con 0x7fe4f40b52e0 2023-10-11T09:42:59.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.043+0000 7fe4f2d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4f415cbf0 0x7fe4f415f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:59.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.043+0000 7fe4f2d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4f415cbf0 0x7fe4f415f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39074/0 (socket says 172.21.15.78:39074) 2023-10-11T09:42:59.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.043+0000 7fe4f2d9d700 1 -- 172.21.15.78:0/1600825767 learned_addr learned my addr 172.21.15.78:0/1600825767 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:59.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.043+0000 7fe4f259c700 1 --2- 172.21.15.78:0/1600825767 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4f40b52e0 0x7fe4f40b7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:59.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.043+0000 7fe4f1d9b700 1 --2- 172.21.15.78:0/1600825767 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4f415a0f0 0x7fe4f415eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:59.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.043+0000 7fe4f2d9d700 1 -- 172.21.15.78:0/1600825767 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4f40b52e0 msgr2=0x7fe4f40b7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:59.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.043+0000 7fe4f2d9d700 1 --2- 172.21.15.78:0/1600825767 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4f40b52e0 0x7fe4f40b7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.043+0000 7fe4f2d9d700 1 -- 172.21.15.78:0/1600825767 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4f415a0f0 msgr2=0x7fe4f415eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:59.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.043+0000 7fe4f2d9d700 1 --2- 172.21.15.78:0/1600825767 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4f415a0f0 0x7fe4f415eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.043+0000 7fe4f2d9d700 1 -- 172.21.15.78:0/1600825767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4ec009b30 con 0x7fe4f415cbf0 2023-10-11T09:42:59.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.043+0000 7fe4f1d9b700 1 --2- 172.21.15.78:0/1600825767 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4f415a0f0 0x7fe4f415eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:42:59.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.043+0000 7fe4f259c700 1 --2- 172.21.15.78:0/1600825767 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4f40b52e0 0x7fe4f40b7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:42:59.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.047+0000 7fe4f2d9d700 1 --2- 172.21.15.78:0/1600825767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4f415cbf0 0x7fe4f415f0a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec00b730 tx=0x7fe4ec004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:59.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.047+0000 7fe4e2ffd700 1 -- 172.21.15.78:0/1600825767 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec004990 con 0x7fe4f415cbf0 2023-10-11T09:42:59.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.047+0000 7fe4e2ffd700 1 -- 172.21.15.78:0/1600825767 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4ec0099b0 con 0x7fe4f415cbf0 2023-10-11T09:42:59.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.047+0000 7fe4f359e700 1 -- 172.21.15.78:0/1600825767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f40b6500 con 0x7fe4f415cbf0 2023-10-11T09:42:59.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.047+0000 7fe4e2ffd700 1 -- 172.21.15.78:0/1600825767 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec01c680 con 0x7fe4f415cbf0 2023-10-11T09:42:59.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.047+0000 7fe4f359e700 1 -- 172.21.15.78:0/1600825767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f40b6ac0 con 0x7fe4f415cbf0 2023-10-11T09:42:59.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.051+0000 7fe4e2ffd700 1 -- 172.21.15.78:0/1600825767 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fe4ec01c7e0 con 0x7fe4f415cbf0 2023-10-11T09:42:59.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.051+0000 7fe4e2ffd700 1 --2- 172.21.15.78:0/1600825767 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe4d4065ed0 0x7fe4d4068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.051+0000 7fe4e2ffd700 1 -- 172.21.15.78:0/1600825767 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6753+0+0 (secure 0 0 0) 0x7fe4ec085710 con 0x7fe4f415cbf0 2023-10-11T09:42:59.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.055+0000 7fe4f259c700 1 --2- 172.21.15.78:0/1600825767 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe4d4065ed0 0x7fe4d4068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:59.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.055+0000 7fe4fbf47740 1 Processor -- start 2023-10-11T09:42:59.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.055+0000 7fe4f259c700 1 --2- 172.21.15.78:0/1600825767 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe4d4065ed0 0x7fe4d4068380 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4009a10 tx=0x7fe4e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:59.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.055+0000 7fe4fbf47740 1 -- start start 2023-10-11T09:42:59.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.055+0000 7fe4fbf47740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1727d80 0x19dd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.055+0000 7fe4fbf47740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x17279b0 0x19dd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.055+0000 7fe4fbf47740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1772a30 0x19ddd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.055+0000 7fe4fbf47740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x18ba2f0 con 0x1727d80 2023-10-11T09:42:59.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.055+0000 7fe4fbf47740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x1907610 con 0x1772a30 2023-10-11T09:42:59.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.055+0000 7fe4fbf47740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x1904da0 con 0x17279b0 2023-10-11T09:42:59.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.055+0000 7fe4f259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1727d80 0x19dd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:59.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.055+0000 7fe4f259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1727d80 0x19dd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39086/0 (socket says 172.21.15.78:39086) 2023-10-11T09:42:59.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.055+0000 7fe4f259c700 1 -- 172.21.15.78:0/4107974110 learned_addr learned my addr 172.21.15.78:0/4107974110 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:59.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.055+0000 7fe4f2d9d700 1 --2- 172.21.15.78:0/4107974110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1772a30 0x19ddd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:59.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.055+0000 7fe4f1d9b700 1 --2- 172.21.15.78:0/4107974110 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x17279b0 0x19dd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:59.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.055+0000 7fe4f2d9d700 1 -- 172.21.15.78:0/4107974110 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x17279b0 msgr2=0x19dd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:59.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.055+0000 7fe4f2d9d700 1 --2- 172.21.15.78:0/4107974110 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x17279b0 0x19dd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.055+0000 7fe4f2d9d700 1 -- 172.21.15.78:0/4107974110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1727d80 msgr2=0x19dd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:59.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.055+0000 7fe4f2d9d700 1 --2- 172.21.15.78:0/4107974110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1727d80 0x19dd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.055+0000 7fe4f2d9d700 1 -- 172.21.15.78:0/4107974110 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1927b50 con 0x1772a30 2023-10-11T09:42:59.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.055+0000 7fe4f259c700 1 --2- 172.21.15.78:0/4107974110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1727d80 0x19dd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:42:59.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.059+0000 7fe4f2d9d700 1 --2- 172.21.15.78:0/4107974110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1772a30 0x19ddd20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec083810 tx=0x7fe4ec0560b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75d77522513a4498 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:59.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.059+0000 7fe4d3fff700 1 -- 172.21.15.78:0/4107974110 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec05a580 con 0x1772a30 2023-10-11T09:42:59.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.059+0000 7fe4d3fff700 1 -- 172.21.15.78:0/4107974110 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4ec05e730 con 0x1772a30 2023-10-11T09:42:59.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.059+0000 7fe4d3fff700 1 -- 172.21.15.78:0/4107974110 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec0092e0 con 0x1772a30 2023-10-11T09:42:59.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.059+0000 7fe4fbf47740 1 -- 172.21.15.78:0/4107974110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1772a30 msgr2=0x19ddd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:59.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.059+0000 7fe4fbf47740 1 --2- 172.21.15.78:0/4107974110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1772a30 0x19ddd20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec083810 tx=0x7fe4ec0560b0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.059+0000 7fe4fbf47740 1 -- 172.21.15.78:0/4107974110 shutdown_connections 2023-10-11T09:42:59.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.059+0000 7fe4fbf47740 1 --2- 172.21.15.78:0/4107974110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1772a30 0x19ddd20 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.059+0000 7fe4fbf47740 1 --2- 172.21.15.78:0/4107974110 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x17279b0 0x19dd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.059+0000 7fe4fbf47740 1 --2- 172.21.15.78:0/4107974110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1727d80 0x19dd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.059+0000 7fe4fbf47740 1 -- 172.21.15.78:0/4107974110 >> 172.21.15.78:0/4107974110 conn(0x1728150 msgr2=0x199c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:59.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.059+0000 7fe4fbf47740 1 -- 172.21.15.78:0/4107974110 shutdown_connections 2023-10-11T09:42:59.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.059+0000 7fe4fbf47740 1 -- 172.21.15.78:0/4107974110 wait complete. 2023-10-11T09:42:59.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.059+0000 7fe4fbf47740 1 -- start start 2023-10-11T09:42:59.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.059+0000 7fe4fbf47740 1 Processor -- start 2023-10-11T09:42:59.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.063+0000 7fe4fbf47740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x17279b0 0x19dd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.063+0000 7fe4fbf47740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1727d80 0x19bb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.063+0000 7fe4fbf47740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1772a30 0x199a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.063+0000 7fe4fbf47740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x1900690 con 0x17279b0 2023-10-11T09:42:59.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.063+0000 7fe4fbf47740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x18c3c50 con 0x1772a30 2023-10-11T09:42:59.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.063+0000 7fe4fbf47740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x18c38f0 con 0x1727d80 2023-10-11T09:42:59.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.063+0000 7fe4fbf47740 10 client.0 _ll_register_callbacks cb 0x1953a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:42:59.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.063+0000 7fe4f259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x17279b0 0x19dd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:59.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.063+0000 7fe4f259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x17279b0 0x19dd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39088/0 (socket says 172.21.15.78:39088) 2023-10-11T09:42:59.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.063+0000 7fe4f259c700 1 -- 172.21.15.78:0/1334285366 learned_addr learned my addr 172.21.15.78:0/1334285366 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:59.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.063+0000 7fe4f1d9b700 1 --2- 172.21.15.78:0/1334285366 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1727d80 0x19bb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:59.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.063+0000 7fe4f2d9d700 1 --2- 172.21.15.78:0/1334285366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1772a30 0x199a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:59.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.063+0000 7fe4f259c700 1 -- 172.21.15.78:0/1334285366 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1727d80 msgr2=0x19bb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:59.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.063+0000 7fe4f259c700 1 --2- 172.21.15.78:0/1334285366 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1727d80 0x19bb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.063+0000 7fe4f259c700 1 -- 172.21.15.78:0/1334285366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1772a30 msgr2=0x199a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:59.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.063+0000 7fe4f259c700 1 --2- 172.21.15.78:0/1334285366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1772a30 0x199a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.063+0000 7fe4f259c700 1 -- 172.21.15.78:0/1334285366 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e400e050 con 0x17279b0 2023-10-11T09:42:59.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.063+0000 7fe4f259c700 1 --2- 172.21.15.78:0/1334285366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x17279b0 0x19dd4b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4015780 tx=0x7fe4e4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c94080a830bb6b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:59.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.063+0000 7fe4d17fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:42:59.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.063+0000 7fe4d17fa700 1 -- 172.21.15.78:0/1334285366 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4016830 con 0x17279b0 2023-10-11T09:42:59.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.063+0000 7fe4d17fa700 1 -- 172.21.15.78:0/1334285366 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 6753+0+0 (secure 0 0 0) 0x7fe4e4016990 con 0x17279b0 2023-10-11T09:42:59.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.063+0000 7fe4fbf47740 1 -- 172.21.15.78:0/1334285366 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x18f9710 con 0x17279b0 2023-10-11T09:42:59.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.063+0000 7fe4d17fa700 1 -- 172.21.15.78:0/1334285366 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4e4016b50 con 0x17279b0 2023-10-11T09:42:59.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.063+0000 7fe4fbf47740 10 client.5997 fetch_fsmap learned FSMap version 29 2023-10-11T09:42:59.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.063+0000 7fe4fbf47740 1 -- 172.21.15.78:0/1334285366 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x7fe4f40b6ac0 con 0x17279b0 2023-10-11T09:42:59.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.063+0000 7fe4d17fa700 1 -- 172.21.15.78:0/1334285366 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x18f9710 con 0x17279b0 2023-10-11T09:42:59.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.063+0000 7fe4d17fa700 1 -- 172.21.15.78:0/1334285366 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fe4e4016cb0 con 0x17279b0 2023-10-11T09:42:59.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.063+0000 7fe4fbf47740 10 client.5997 fetch_fsmap finished waiting for FSMap version 29 2023-10-11T09:42:59.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.063+0000 7fe4fbf47740 10 client.5997 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-10-11T09:42:59.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.063+0000 7fe4fbf47740 20 client.5997 populate_metadata read hostname 'smithi078' 2023-10-11T09:42:59.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.063+0000 7fe4fbf47740 1 --2- 172.21.15.78:0/1334285366 >> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] conn(0x199bee0 0x19987c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.063+0000 7fe4fbf47740 4 client.5997 mds_command: new command op to 5922 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-11T09:42:59.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.063+0000 7fe4fbf47740 1 -- 172.21.15.78:0/1334285366 --> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x18f9430 con 0x199bee0 2023-10-11T09:42:59.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.067+0000 7fe4f1d9b700 1 --2- 172.21.15.78:0/1334285366 >> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] conn(0x199bee0 0x19987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:59.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.067+0000 7fe4f1d9b700 1 --2- 172.21.15.78:0/1334285366 >> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] conn(0x199bee0 0x19987c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bce56916ff19f499 server_cookie=c5d5c6cc94897808 in_seq=0 out_seq=0 2023-10-11T09:42:59.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.067+0000 7fe4d17fa700 10 client.5997 ms_handle_connect on v2:172.21.15.78:6835/3137573764 2023-10-11T09:42:59.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.067+0000 7fe4d17fa700 1 -- 172.21.15.78:0/1334285366 <== mds.0 v2:172.21.15.78:6835/3137573764 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x18f9430 con 0x199bee0 2023-10-11T09:42:59.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.067+0000 7fe4d17fa700 10 client.5997 handle_command_reply: tid=0 2023-10-11T09:42:59.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.067+0000 7fe4fbf47740 1 client.5997 shutdown 2023-10-11T09:42:59.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.067+0000 7fe4fbf47740 1 -- 172.21.15.78:0/1334285366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x17279b0 msgr2=0x19dd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:59.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.067+0000 7fe4fbf47740 1 --2- 172.21.15.78:0/1334285366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x17279b0 0x19dd4b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4015780 tx=0x7fe4e4004770 comp rx=0 tx=0).stop 2023-10-11T09:42:59.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.067+0000 7fe4fbf47740 1 -- 172.21.15.78:0/1334285366 shutdown_connections 2023-10-11T09:42:59.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.067+0000 7fe4fbf47740 1 --2- 172.21.15.78:0/1334285366 >> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] conn(0x199bee0 0x19987c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.067+0000 7fe4fbf47740 1 --2- 172.21.15.78:0/1334285366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1772a30 0x199a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.067+0000 7fe4fbf47740 1 --2- 172.21.15.78:0/1334285366 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1727d80 0x19bb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.067+0000 7fe4fbf47740 1 --2- 172.21.15.78:0/1334285366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x17279b0 0x19dd4b0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.067+0000 7fe4d17fa700 0 client.5997 ms_handle_reset on v2:172.21.15.78:6835/3137573764 2023-10-11T09:42:59.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.067+0000 7fe4fbf47740 1 -- 172.21.15.78:0/1334285366 >> 172.21.15.78:0/1334285366 conn(0x1728150 msgr2=0x19be480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:59.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.067+0000 7fe4fbf47740 1 -- 172.21.15.78:0/1334285366 shutdown_connections 2023-10-11T09:42:59.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.067+0000 7fe4fbf47740 1 -- 172.21.15.78:0/1334285366 wait complete. 2023-10-11T09:42:59.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.067+0000 7fe4fbf47740 20 client.5997 trim_cache size 0 max 16384 2023-10-11T09:42:59.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.087+0000 7fe4fbf47740 1 Processor -- start 2023-10-11T09:42:59.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.087+0000 7fe4fbf47740 1 -- start start 2023-10-11T09:42:59.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.087+0000 7fe4fbf47740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x17279b0 0x19c2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.087+0000 7fe4fbf47740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1727d80 0x19c3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.091+0000 7fe4fbf47740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1772a30 0x19b7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.091+0000 7fe4fbf47740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x17acdc0 con 0x1727d80 2023-10-11T09:42:59.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.091+0000 7fe4fbf47740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x1890bc0 con 0x17279b0 2023-10-11T09:42:59.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.091+0000 7fe4fbf47740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x1959700 con 0x1772a30 2023-10-11T09:42:59.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.091+0000 7fe4f1d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1727d80 0x19c3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:59.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.091+0000 7fe4f1d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1727d80 0x19c3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39092/0 (socket says 172.21.15.78:39092) 2023-10-11T09:42:59.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.091+0000 7fe4f1d9b700 1 -- 172.21.15.78:0/640107591 learned_addr learned my addr 172.21.15.78:0/640107591 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:59.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.091+0000 7fe4f2d9d700 1 --2- 172.21.15.78:0/640107591 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1772a30 0x19b7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:59.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.091+0000 7fe4f259c700 1 --2- 172.21.15.78:0/640107591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x17279b0 0x19c2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:59.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.091+0000 7fe4f1d9b700 1 -- 172.21.15.78:0/640107591 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1772a30 msgr2=0x19b7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:59.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.091+0000 7fe4f1d9b700 1 --2- 172.21.15.78:0/640107591 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1772a30 0x19b7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.091+0000 7fe4f1d9b700 1 -- 172.21.15.78:0/640107591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x17279b0 msgr2=0x19c2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:59.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.091+0000 7fe4f1d9b700 1 --2- 172.21.15.78:0/640107591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x17279b0 0x19c2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.091+0000 7fe4f1d9b700 1 -- 172.21.15.78:0/640107591 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f40b6fb0 con 0x1727d80 2023-10-11T09:42:59.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.091+0000 7fe4f2d9d700 1 --2- 172.21.15.78:0/640107591 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1772a30 0x19b7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:42:59.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.091+0000 7fe4f1d9b700 1 --2- 172.21.15.78:0/640107591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1727d80 0x19c3030 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8007f70 tx=0x7fe4e8001d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b967b2a9a99c23 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:59.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.091+0000 7fe4e0ff9700 1 -- 172.21.15.78:0/640107591 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8006d00 con 0x1727d80 2023-10-11T09:42:59.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.091+0000 7fe4e0ff9700 1 -- 172.21.15.78:0/640107591 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e801c040 con 0x1727d80 2023-10-11T09:42:59.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.091+0000 7fe4e0ff9700 1 -- 172.21.15.78:0/640107591 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8005310 con 0x1727d80 2023-10-11T09:42:59.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.091+0000 7fe4fbf47740 1 -- 172.21.15.78:0/640107591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1727d80 msgr2=0x19c3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:59.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.091+0000 7fe4fbf47740 1 --2- 172.21.15.78:0/640107591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1727d80 0x19c3030 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8007f70 tx=0x7fe4e8001d30 comp rx=0 tx=0).stop 2023-10-11T09:42:59.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.091+0000 7fe4fbf47740 1 -- 172.21.15.78:0/640107591 shutdown_connections 2023-10-11T09:42:59.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.091+0000 7fe4fbf47740 1 --2- 172.21.15.78:0/640107591 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1772a30 0x19b7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.091+0000 7fe4fbf47740 1 --2- 172.21.15.78:0/640107591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1727d80 0x19c3030 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.091+0000 7fe4fbf47740 1 --2- 172.21.15.78:0/640107591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x17279b0 0x19c2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.091+0000 7fe4fbf47740 1 -- 172.21.15.78:0/640107591 >> 172.21.15.78:0/640107591 conn(0x1728150 msgr2=0x19c23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:59.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.091+0000 7fe4fbf47740 1 -- 172.21.15.78:0/640107591 shutdown_connections 2023-10-11T09:42:59.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.091+0000 7fe4fbf47740 1 -- 172.21.15.78:0/640107591 wait complete. 2023-10-11T09:42:59.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.095+0000 7fe4fbf47740 1 -- start start 2023-10-11T09:42:59.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.095+0000 7fe4fbf47740 1 Processor -- start 2023-10-11T09:42:59.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.095+0000 7fe4fbf47740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x17279b0 0x19ba360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.095+0000 7fe4fbf47740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1727d80 0x1a47d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.095+0000 7fe4fbf47740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1772a30 0x1a482b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.095+0000 7fe4fbf47740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x19bad30 con 0x1727d80 2023-10-11T09:42:59.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.095+0000 7fe4fbf47740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x19f3f10 con 0x17279b0 2023-10-11T09:42:59.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.095+0000 7fe4fbf47740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x19f4e30 con 0x1772a30 2023-10-11T09:42:59.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.095+0000 7fe4fbf47740 10 client.0 _ll_register_callbacks cb 0x18b8d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:42:59.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.095+0000 7fe4f1d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1727d80 0x1a47d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:59.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.095+0000 7fe4f1d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1727d80 0x1a47d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39098/0 (socket says 172.21.15.78:39098) 2023-10-11T09:42:59.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.095+0000 7fe4f1d9b700 1 -- 172.21.15.78:0/2154805757 learned_addr learned my addr 172.21.15.78:0/2154805757 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:59.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.095+0000 7fe4f259c700 1 --2- 172.21.15.78:0/2154805757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x17279b0 0x19ba360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:59.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.095+0000 7fe4f2d9d700 1 --2- 172.21.15.78:0/2154805757 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1772a30 0x1a482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:59.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.095+0000 7fe4f1d9b700 1 -- 172.21.15.78:0/2154805757 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1772a30 msgr2=0x1a482b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:59.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.095+0000 7fe4f1d9b700 1 --2- 172.21.15.78:0/2154805757 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1772a30 0x1a482b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.095+0000 7fe4f1d9b700 1 -- 172.21.15.78:0/2154805757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x17279b0 msgr2=0x19ba360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:59.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.095+0000 7fe4f1d9b700 1 --2- 172.21.15.78:0/2154805757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x17279b0 0x19ba360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.095+0000 7fe4f1d9b700 1 -- 172.21.15.78:0/2154805757 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e800f040 con 0x1727d80 2023-10-11T09:42:59.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.095+0000 7fe4f1d9b700 1 --2- 172.21.15.78:0/2154805757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1727d80 0x1a47d70 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8001160 tx=0x7fe4e8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdffdb6ad93ffe03 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:59.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.095+0000 7fe4d17fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:42:59.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.099+0000 7fe4d17fa700 1 -- 172.21.15.78:0/2154805757 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8005560 con 0x1727d80 2023-10-11T09:42:59.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.099+0000 7fe4fbf47740 1 -- 172.21.15.78:0/2154805757 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe4ec05e730 con 0x1727d80 2023-10-11T09:42:59.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.099+0000 7fe4d17fa700 1 -- 172.21.15.78:0/2154805757 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 6753+0+0 (secure 0 0 0) 0x7fe4e8007040 con 0x1727d80 2023-10-11T09:42:59.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.099+0000 7fe4d17fa700 1 -- 172.21.15.78:0/2154805757 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4e8007200 con 0x1727d80 2023-10-11T09:42:59.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.099+0000 7fe4fbf47740 10 client.6003 fetch_fsmap learned FSMap version 29 2023-10-11T09:42:59.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.099+0000 7fe4fbf47740 1 -- 172.21.15.78:0/2154805757 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x1773c80 con 0x1727d80 2023-10-11T09:42:59.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.099+0000 7fe4d17fa700 1 -- 172.21.15.78:0/2154805757 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4ec05e730 con 0x1727d80 2023-10-11T09:42:59.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.099+0000 7fe4d17fa700 1 -- 172.21.15.78:0/2154805757 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fe4e801f070 con 0x1727d80 2023-10-11T09:42:59.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.099+0000 7fe4fbf47740 10 client.6003 fetch_fsmap finished waiting for FSMap version 29 2023-10-11T09:42:59.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.099+0000 7fe4fbf47740 10 client.6003 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-10-11T09:42:59.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.099+0000 7fe4fbf47740 20 client.6003 populate_metadata read hostname 'smithi078' 2023-10-11T09:42:59.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.099+0000 7fe4fbf47740 1 --2- 172.21.15.78:0/2154805757 >> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] conn(0x19ba8a0 0x19f2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.099+0000 7fe4fbf47740 4 client.6003 mds_command: new command op to 5922 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2023-10-11T09:42:59.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.099+0000 7fe4fbf47740 1 -- 172.21.15.78:0/2154805757 --> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x177f780 con 0x19ba8a0 2023-10-11T09:42:59.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.099+0000 7fe4f259c700 1 --2- 172.21.15.78:0/2154805757 >> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] conn(0x19ba8a0 0x19f2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:59.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.099+0000 7fe4f259c700 1 --2- 172.21.15.78:0/2154805757 >> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] conn(0x19ba8a0 0x19f2d80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9d942cae5fdfb81 server_cookie=59090179d85d3d31 in_seq=0 out_seq=0 2023-10-11T09:42:59.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.099+0000 7fe4d17fa700 10 client.6003 ms_handle_connect on v2:172.21.15.78:6835/3137573764 2023-10-11T09:42:59.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.099+0000 7fe4d17fa700 1 -- 172.21.15.78:0/2154805757 <== mds.0 v2:172.21.15.78:6835/3137573764 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x177f780 con 0x19ba8a0 2023-10-11T09:42:59.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.099+0000 7fe4d17fa700 10 client.6003 handle_command_reply: tid=0 2023-10-11T09:42:59.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.099+0000 7fe4fbf47740 1 client.6003 shutdown 2023-10-11T09:42:59.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.103+0000 7fe4fbf47740 1 -- 172.21.15.78:0/2154805757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1727d80 msgr2=0x1a47d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:59.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.103+0000 7fe4fbf47740 1 --2- 172.21.15.78:0/2154805757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1727d80 0x1a47d70 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8001160 tx=0x7fe4e8000ed0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.103+0000 7fe4fbf47740 1 -- 172.21.15.78:0/2154805757 shutdown_connections 2023-10-11T09:42:59.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.103+0000 7fe4fbf47740 1 --2- 172.21.15.78:0/2154805757 >> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] conn(0x19ba8a0 0x19f2d80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.103+0000 7fe4fbf47740 1 --2- 172.21.15.78:0/2154805757 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1772a30 0x1a482b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.114 INFO:tasks.ceph.mds.c.smithi078.stderr:2023-10-11T09:42:59.103+0000 7f6906d7f700 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2023-10-11T09:42:59.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.103+0000 7fe4fbf47740 1 --2- 172.21.15.78:0/2154805757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1727d80 0x1a47d70 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.103+0000 7fe4d17fa700 0 client.6003 ms_handle_reset on v2:172.21.15.78:6835/3137573764 2023-10-11T09:42:59.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.103+0000 7fe4fbf47740 1 --2- 172.21.15.78:0/2154805757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x17279b0 0x19ba360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.103+0000 7fe4fbf47740 1 -- 172.21.15.78:0/2154805757 >> 172.21.15.78:0/2154805757 conn(0x1728150 msgr2=0x19dc260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:59.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.103+0000 7fe4fbf47740 1 -- 172.21.15.78:0/2154805757 shutdown_connections 2023-10-11T09:42:59.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.103+0000 7fe4fbf47740 1 -- 172.21.15.78:0/2154805757 wait complete. 2023-10-11T09:42:59.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.103+0000 7fe4fbf47740 20 client.6003 trim_cache size 0 max 16384 2023-10-11T09:42:59.116 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:42:59.116 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2023-10-11T09:42:59.117 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "adf0ee7c-b37b-4787-9377-54135a236381", 2023-10-11T09:42:59.117 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2023-10-11T09:42:59.117 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:42:59.117 INFO:tasks.ceph.mds.c.smithi078.stderr:2023-10-11T09:42:59.103+0000 7f6906d7f700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /subdir/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x564d6ceb5180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(5)0x10000000000:[<0x1/subdir v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-10-11T09:42:59.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.103+0000 7fe4f359e700 1 -- 172.21.15.78:0/1600825767 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe4d4065ed0 msgr2=0x7fe4d4068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:59.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.103+0000 7fe4f359e700 1 --2- 172.21.15.78:0/1600825767 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe4d4065ed0 0x7fe4d4068380 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4009a10 tx=0x7fe4e4008040 comp rx=0 tx=0).stop 2023-10-11T09:42:59.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.103+0000 7fe4f359e700 1 -- 172.21.15.78:0/1600825767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4f415cbf0 msgr2=0x7fe4f415f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:59.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.103+0000 7fe4f359e700 1 --2- 172.21.15.78:0/1600825767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4f415cbf0 0x7fe4f415f0a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec00b730 tx=0x7fe4ec004af0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.103+0000 7fe4f359e700 1 -- 172.21.15.78:0/1600825767 shutdown_connections 2023-10-11T09:42:59.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.103+0000 7fe4f359e700 1 --2- 172.21.15.78:0/1600825767 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe4d4065ed0 0x7fe4d4068380 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.103+0000 7fe4f359e700 1 --2- 172.21.15.78:0/1600825767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4f415cbf0 0x7fe4f415f0a0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.103+0000 7fe4f359e700 1 --2- 172.21.15.78:0/1600825767 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4f415a0f0 0x7fe4f415eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.103+0000 7fe4f359e700 1 --2- 172.21.15.78:0/1600825767 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4f40b52e0 0x7fe4f40b7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.103+0000 7fe4f359e700 1 -- 172.21.15.78:0/1600825767 >> 172.21.15.78:0/1600825767 conn(0x7fe4f406afd0 msgr2=0x7fe4f41e6fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:59.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.103+0000 7fe4f359e700 1 -- 172.21.15.78:0/1600825767 shutdown_connections 2023-10-11T09:42:59.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.103+0000 7fe4f359e700 1 -- 172.21.15.78:0/1600825767 wait complete. 2023-10-11T09:42:59.121 INFO:tasks.ceph.mds.c.smithi078.stderr:2023-10-11T09:42:59.107+0000 7f6906d7f700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f6 [2,head] /subdir/1 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x564d6db89600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(7)0x100000001f6:[<0x10000000000/1 v24>,<0x1/subdir v46>]//[6]","memoryvalue":"(7)0x100000001f6:[<0x10000000000/1 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:42:59.122 INFO:tasks.ceph.mds.c.smithi078.stderr:2023-10-11T09:42:59.107+0000 7f6906d7f700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f8 [2,head] /subdir/3 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x564d6ddbcc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(7)0x100000001f8:[<0x10000000000/3 v40>,<0x1/subdir v46>]//[6]","memoryvalue":"(7)0x100000001f8:[<0x10000000000/3 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:42:59.122 INFO:tasks.ceph.mds.c.smithi078.stderr:2023-10-11T09:42:59.107+0000 7f6906d7f700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f7 [2,head] /subdir/2 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x564d6db89080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(7)0x100000001f7:[<0x10000000000/2 v32>,<0x1/subdir v46>]//[6]","memoryvalue":"(7)0x100000001f7:[<0x10000000000/2 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:42:59.122 INFO:tasks.ceph.mds.c.smithi078.stderr:2023-10-11T09:42:59.107+0000 7f6906d7f700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f9 [2,head] /subdir/4 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x564d6ddbc680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(7)0x100000001f9:[<0x10000000000/4 v42>,<0x1/subdir v46>]//[6]","memoryvalue":"(7)0x100000001f9:[<0x10000000000/4 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:42:59.123 INFO:tasks.ceph.mds.c.smithi078.stderr:2023-10-11T09:42:59.107+0000 7f6906d7f700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/0 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x564d6db89b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(7)0x10000000001:[<0x10000000000/0 v20>,<0x1/subdir v46>]//[6]","memoryvalue":"(7)0x10000000001:[<0x10000000000/0 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:42:59.123 INFO:tasks.ceph.mds.c.smithi078.stderr:2023-10-11T09:42:59.111+0000 7f6906d7f700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v5 ap=5 snaprealm=0x564d6dd44b40 DIRTYPARENT f(v1 1=0+1) n(v0 rc2023-10-11T09:42:58.806279+0000 1=0+1)/n() (idft sync r=1) (inest sync r=1) (ifile sync->mix r=1) (iversion lock) caps={5985=pLsXs/-@3} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x564d6dd47600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(5)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=0+1)","ondisk_value.rstat":"n(v0 2=0+2)","memory_value.dirstat":"f(v1 1=0+1)","memory_value.rstat":"n(v0 rc2023-10-11T09:42:58.806279+0000 1=0+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-10-11T09:42:59.130 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 scrub status 2023-10-11T09:42:59.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.303+0000 7f6a6645d700 1 Processor -- start 2023-10-11T09:42:59.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.303+0000 7f6a6645d700 1 -- start start 2023-10-11T09:42:59.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.303+0000 7f6a6645d700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6a600b84c0 0x7f6a600cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.303+0000 7f6a6645d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6a60160a80 0x7f6a600d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.303+0000 7f6a6645d700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a600cd690 con 0x7f6a600b8890 2023-10-11T09:42:59.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.303+0000 7f6a6645d700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a600cdea0 con 0x7f6a600b84c0 2023-10-11T09:42:59.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.303+0000 7f6a6645d700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a600d0c10 con 0x7f6a60160a80 2023-10-11T09:42:59.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.307+0000 7f6a5f7fe700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f6a600b8890 0x7f6a60160370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:56396/0 (socket says 172.21.15.78:56396) 2023-10-11T09:42:59.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.307+0000 7f6a5f7fe700 1 -- 172.21.15.78:0/205437409 learned_addr learned my addr 172.21.15.78:0/205437409 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:42:59.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.307+0000 7f6a5ffff700 1 --2- 172.21.15.78:0/205437409 >> v2:172.21.15.133:3300/0 conn(0x7f6a600b84c0 0x7f6a600cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:59.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.307+0000 7f6a649fa700 1 --2- 172.21.15.78:0/205437409 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6a60160a80 0x7f6a600d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:59.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.307+0000 7f6a5effd700 1 -- 172.21.15.78:0/205437409 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1695667232 0 0) 0x7f6a600cd690 con 0x7f6a600b8890 2023-10-11T09:42:59.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.307+0000 7f6a5effd700 1 -- 172.21.15.78:0/205437409 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a480035d0 con 0x7f6a600b8890 2023-10-11T09:42:59.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.307+0000 7f6a5ffff700 1 -- 172.21.15.78:0/205437409 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6a60160a80 msgr2=0x7f6a600d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:59.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.307+0000 7f6a5ffff700 1 --2- 172.21.15.78:0/205437409 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6a60160a80 0x7f6a600d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.307+0000 7f6a5ffff700 1 -- 172.21.15.78:0/205437409 >> v1:172.21.15.78:6789/0 conn(0x7f6a600b8890 legacy=0x7f6a60160370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:59.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.307+0000 7f6a5effd700 1 -- 172.21.15.78:0/205437409 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 584572452 0 0) 0x7f6a480035d0 con 0x7f6a600b8890 2023-10-11T09:42:59.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.307+0000 7f6a5ffff700 1 -- 172.21.15.78:0/205437409 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a600d02e0 con 0x7f6a600b84c0 2023-10-11T09:42:59.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.307+0000 7f6a5ffff700 1 --2- 172.21.15.78:0/205437409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a600b84c0 0x7f6a600cfda0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6a600cdfe0 tx=0x7f6a50009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5e222f24b43fff3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:59.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.307+0000 7f6a5effd700 1 -- 172.21.15.78:0/205437409 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a54001670 con 0x7f6a600b84c0 2023-10-11T09:42:59.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.307+0000 7f6a5effd700 1 -- 172.21.15.78:0/205437409 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a50004030 con 0x7f6a600b84c0 2023-10-11T09:42:59.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.307+0000 7f6a5effd700 1 -- 172.21.15.78:0/205437409 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a500043c0 con 0x7f6a600b84c0 2023-10-11T09:42:59.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.307+0000 7f6a6645d700 1 -- 172.21.15.78:0/205437409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a600b84c0 msgr2=0x7f6a600cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:59.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.307+0000 7f6a6645d700 1 --2- 172.21.15.78:0/205437409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a600b84c0 0x7f6a600cfda0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6a600cdfe0 tx=0x7f6a50009b30 comp rx=0 tx=0).stop 2023-10-11T09:42:59.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.307+0000 7f6a6645d700 1 -- 172.21.15.78:0/205437409 shutdown_connections 2023-10-11T09:42:59.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.307+0000 7f6a6645d700 1 --2- 172.21.15.78:0/205437409 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6a60160a80 0x7f6a600d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.307+0000 7f6a6645d700 1 --2- 172.21.15.78:0/205437409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a600b84c0 0x7f6a600cfda0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.307+0000 7f6a6645d700 1 -- 172.21.15.78:0/205437409 >> 172.21.15.78:0/205437409 conn(0x7f6a6006cb50 msgr2=0x7f6a6006cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:59.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.307+0000 7f6a6645d700 1 -- 172.21.15.78:0/205437409 shutdown_connections 2023-10-11T09:42:59.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.307+0000 7f6a6645d700 1 -- 172.21.15.78:0/205437409 wait complete. 2023-10-11T09:42:59.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.311+0000 7f6a6645d700 1 Processor -- start 2023-10-11T09:42:59.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.311+0000 7f6a6645d700 1 -- start start 2023-10-11T09:42:59.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.311+0000 7f6a6645d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6a600b84c0 0x7f6a60145f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.311+0000 7f6a6645d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a600b8890 0x7f6a600c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.311+0000 7f6a6645d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a60160a80 0x7f6a600cb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.311+0000 7f6a6645d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6014a320 con 0x7f6a600b8890 2023-10-11T09:42:59.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.311+0000 7f6a6645d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a600cc680 con 0x7f6a60160a80 2023-10-11T09:42:59.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.311+0000 7f6a6645d700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a600cc960 con 0x7f6a600b84c0 2023-10-11T09:42:59.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.311+0000 7f6a5f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a600b8890 0x7f6a600c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:59.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.311+0000 7f6a5f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a600b8890 0x7f6a600c80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39104/0 (socket says 172.21.15.78:39104) 2023-10-11T09:42:59.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.311+0000 7f6a5f7fe700 1 -- 172.21.15.78:0/1756448190 learned_addr learned my addr 172.21.15.78:0/1756448190 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:59.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.311+0000 7f6a649fa700 1 --2- 172.21.15.78:0/1756448190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a60160a80 0x7f6a600cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:59.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.311+0000 7f6a5ffff700 1 --2- 172.21.15.78:0/1756448190 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6a600b84c0 0x7f6a60145f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:59.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.311+0000 7f6a5f7fe700 1 -- 172.21.15.78:0/1756448190 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6a600b84c0 msgr2=0x7f6a60145f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:59.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.311+0000 7f6a5f7fe700 1 --2- 172.21.15.78:0/1756448190 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6a600b84c0 0x7f6a60145f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.311+0000 7f6a5f7fe700 1 -- 172.21.15.78:0/1756448190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a60160a80 msgr2=0x7f6a600cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:59.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.311+0000 7f6a5f7fe700 1 --2- 172.21.15.78:0/1756448190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a60160a80 0x7f6a600cb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.311+0000 7f6a5f7fe700 1 -- 172.21.15.78:0/1756448190 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a500094b0 con 0x7f6a600b8890 2023-10-11T09:42:59.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.315+0000 7f6a5f7fe700 1 --2- 172.21.15.78:0/1756448190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a600b8890 0x7f6a600c80c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6a540011e0 tx=0x7f6a5400ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:59.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.315+0000 7f6a5cff9700 1 -- 172.21.15.78:0/1756448190 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a54004ca0 con 0x7f6a600b8890 2023-10-11T09:42:59.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.315+0000 7f6a5cff9700 1 -- 172.21.15.78:0/1756448190 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a54004e20 con 0x7f6a600b8890 2023-10-11T09:42:59.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.315+0000 7f6a5cff9700 1 -- 172.21.15.78:0/1756448190 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a54007310 con 0x7f6a600b8890 2023-10-11T09:42:59.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.315+0000 7f6a6645d700 1 -- 172.21.15.78:0/1756448190 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a600ccc40 con 0x7f6a600b8890 2023-10-11T09:42:59.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.315+0000 7f6a6645d700 1 -- 172.21.15.78:0/1756448190 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a60146490 con 0x7f6a600b8890 2023-10-11T09:42:59.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.319+0000 7f6a5cff9700 1 -- 172.21.15.78:0/1756448190 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f6a54007470 con 0x7f6a600b8890 2023-10-11T09:42:59.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.319+0000 7f6a5cff9700 1 --2- 172.21.15.78:0/1756448190 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6a4c065ed0 0x7f6a4c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.319+0000 7f6a5cff9700 1 -- 172.21.15.78:0/1756448190 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6753+0+0 (secure 0 0 0) 0x7f6a54082d20 con 0x7f6a600b8890 2023-10-11T09:42:59.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.319+0000 7f6a5ffff700 1 --2- 172.21.15.78:0/1756448190 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6a4c065ed0 0x7f6a4c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:59.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.319+0000 7f6a5ffff700 1 --2- 172.21.15.78:0/1756448190 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6a4c065ed0 0x7f6a4c068380 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6a50000c00 tx=0x7f6a50013040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:59.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.323+0000 7f6a696e9740 1 Processor -- start 2023-10-11T09:42:59.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.323+0000 7f6a696e9740 1 -- start start 2023-10-11T09:42:59.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.323+0000 7f6a696e9740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x16a7c10 0x193d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.323+0000 7f6a696e9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16a7840 0x195ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.323+0000 7f6a696e9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x16f28c0 0x195f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.323+0000 7f6a696e9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x1887630 con 0x16a7840 2023-10-11T09:42:59.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.323+0000 7f6a696e9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x1884dc0 con 0x16f28c0 2023-10-11T09:42:59.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.323+0000 7f6a696e9740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x18806b0 con 0x16a7c10 2023-10-11T09:42:59.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.323+0000 7f6a5f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16a7840 0x195ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:59.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.323+0000 7f6a5f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16a7840 0x195ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39114/0 (socket says 172.21.15.78:39114) 2023-10-11T09:42:59.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.323+0000 7f6a5f7fe700 1 -- 172.21.15.78:0/1018301470 learned_addr learned my addr 172.21.15.78:0/1018301470 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:59.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.323+0000 7f6a649fa700 1 --2- 172.21.15.78:0/1018301470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x16f28c0 0x195f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:59.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.323+0000 7f6a5ffff700 1 --2- 172.21.15.78:0/1018301470 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x16a7c10 0x193d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:59.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.323+0000 7f6a5f7fe700 1 -- 172.21.15.78:0/1018301470 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x16a7c10 msgr2=0x193d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:59.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.323+0000 7f6a5f7fe700 1 --2- 172.21.15.78:0/1018301470 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x16a7c10 0x193d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.323+0000 7f6a5f7fe700 1 -- 172.21.15.78:0/1018301470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x16f28c0 msgr2=0x195f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:59.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.323+0000 7f6a5f7fe700 1 --2- 172.21.15.78:0/1018301470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x16f28c0 0x195f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.323+0000 7f6a5f7fe700 1 -- 172.21.15.78:0/1018301470 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x18a65b0 con 0x16a7840 2023-10-11T09:42:59.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.323+0000 7f6a5f7fe700 1 --2- 172.21.15.78:0/1018301470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16a7840 0x195ebb0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6a54050150 tx=0x7f6a54007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15fcb1f8aa1e352f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:59.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.323+0000 7f6a45ffb700 1 -- 172.21.15.78:0/1018301470 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a54053c50 con 0x16a7840 2023-10-11T09:42:59.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.323+0000 7f6a45ffb700 1 -- 172.21.15.78:0/1018301470 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a54057a80 con 0x16a7840 2023-10-11T09:42:59.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.323+0000 7f6a45ffb700 1 -- 172.21.15.78:0/1018301470 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5405bc30 con 0x16a7840 2023-10-11T09:42:59.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.323+0000 7f6a696e9740 1 -- 172.21.15.78:0/1018301470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16a7840 msgr2=0x195ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:59.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.323+0000 7f6a696e9740 1 --2- 172.21.15.78:0/1018301470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16a7840 0x195ebb0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6a54050150 tx=0x7f6a54007720 comp rx=0 tx=0).stop 2023-10-11T09:42:59.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.323+0000 7f6a696e9740 1 -- 172.21.15.78:0/1018301470 shutdown_connections 2023-10-11T09:42:59.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.323+0000 7f6a696e9740 1 --2- 172.21.15.78:0/1018301470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x16f28c0 0x195f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.323+0000 7f6a696e9740 1 --2- 172.21.15.78:0/1018301470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16a7840 0x195ebb0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.323+0000 7f6a696e9740 1 --2- 172.21.15.78:0/1018301470 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x16a7c10 0x193d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.323+0000 7f6a696e9740 1 -- 172.21.15.78:0/1018301470 >> 172.21.15.78:0/1018301470 conn(0x16a7fe0 msgr2=0x1940350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:59.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.323+0000 7f6a696e9740 1 -- 172.21.15.78:0/1018301470 shutdown_connections 2023-10-11T09:42:59.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.327+0000 7f6a696e9740 1 -- 172.21.15.78:0/1018301470 wait complete. 2023-10-11T09:42:59.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.327+0000 7f6a696e9740 1 -- start start 2023-10-11T09:42:59.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.327+0000 7f6a696e9740 1 Processor -- start 2023-10-11T09:42:59.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.327+0000 7f6a696e9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x16a7840 0x193bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.327+0000 7f6a696e9740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x16a7c10 0x1914230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.327+0000 7f6a696e9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16f28c0 0x1912810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.327+0000 7f6a696e9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x1843c10 con 0x16f28c0 2023-10-11T09:42:59.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.327+0000 7f6a696e9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x18438b0 con 0x16a7840 2023-10-11T09:42:59.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.327+0000 7f6a696e9740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x172cc50 con 0x16a7c10 2023-10-11T09:42:59.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.327+0000 7f6a696e9740 10 client.0 _ll_register_callbacks cb 0x1879ff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:42:59.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.327+0000 7f6a649fa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16f28c0 0x1912810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:59.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.327+0000 7f6a649fa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16f28c0 0x1912810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39120/0 (socket says 172.21.15.78:39120) 2023-10-11T09:42:59.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.327+0000 7f6a649fa700 1 -- 172.21.15.78:0/1616091945 learned_addr learned my addr 172.21.15.78:0/1616091945 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:59.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.327+0000 7f6a5ffff700 1 --2- 172.21.15.78:0/1616091945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x16a7840 0x193bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:59.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.327+0000 7f6a5f7fe700 1 --2- 172.21.15.78:0/1616091945 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x16a7c10 0x1914230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:59.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.327+0000 7f6a649fa700 1 -- 172.21.15.78:0/1616091945 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x16a7c10 msgr2=0x1914230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:59.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.327+0000 7f6a649fa700 1 --2- 172.21.15.78:0/1616091945 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x16a7c10 0x1914230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.327+0000 7f6a649fa700 1 -- 172.21.15.78:0/1616091945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x16a7840 msgr2=0x193bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:59.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.327+0000 7f6a649fa700 1 --2- 172.21.15.78:0/1616091945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x16a7840 0x193bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.327+0000 7f6a649fa700 1 -- 172.21.15.78:0/1616091945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a60146490 con 0x16f28c0 2023-10-11T09:42:59.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.327+0000 7f6a649fa700 1 --2- 172.21.15.78:0/1616091945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16f28c0 0x1912810 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6a58007da0 tx=0x7f6a58012520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df885cf9650e0fe server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:59.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.327+0000 7f6a3effd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:42:59.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.327+0000 7f6a3effd700 1 -- 172.21.15.78:0/1616091945 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5800e040 con 0x16f28c0 2023-10-11T09:42:59.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.327+0000 7f6a696e9740 1 -- 172.21.15.78:0/1616091945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1879730 con 0x16f28c0 2023-10-11T09:42:59.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.327+0000 7f6a3effd700 1 -- 172.21.15.78:0/1616091945 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 6753+0+0 (secure 0 0 0) 0x7f6a58012c50 con 0x16f28c0 2023-10-11T09:42:59.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.331+0000 7f6a3effd700 1 -- 172.21.15.78:0/1616091945 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a58012e60 con 0x16f28c0 2023-10-11T09:42:59.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.331+0000 7f6a696e9740 10 client.6015 fetch_fsmap learned FSMap version 29 2023-10-11T09:42:59.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.331+0000 7f6a696e9740 1 -- 172.21.15.78:0/1616091945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x16f3b10 con 0x16f28c0 2023-10-11T09:42:59.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.331+0000 7f6a3effd700 1 -- 172.21.15.78:0/1616091945 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1879730 con 0x16f28c0 2023-10-11T09:42:59.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.331+0000 7f6a3effd700 1 -- 172.21.15.78:0/1616091945 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f6a58014040 con 0x16f28c0 2023-10-11T09:42:59.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.331+0000 7f6a696e9740 10 client.6015 fetch_fsmap finished waiting for FSMap version 29 2023-10-11T09:42:59.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.331+0000 7f6a696e9740 10 client.6015 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-10-11T09:42:59.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.331+0000 7f6a696e9740 20 client.6015 populate_metadata read hostname 'smithi078' 2023-10-11T09:42:59.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.331+0000 7f6a696e9740 1 --2- 172.21.15.78:0/1616091945 >> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] conn(0x1913240 0x1967c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.331+0000 7f6a696e9740 4 client.6015 mds_command: new command op to 5922 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-11T09:42:59.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.331+0000 7f6a696e9740 1 -- 172.21.15.78:0/1616091945 --> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1879450 con 0x1913240 2023-10-11T09:42:59.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.331+0000 7f6a5ffff700 1 --2- 172.21.15.78:0/1616091945 >> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] conn(0x1913240 0x1967c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:59.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.331+0000 7f6a5ffff700 1 --2- 172.21.15.78:0/1616091945 >> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] conn(0x1913240 0x1967c40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5540b3c740b46c37 server_cookie=988c164fc7ebcb25 in_seq=0 out_seq=0 2023-10-11T09:42:59.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.331+0000 7f6a3effd700 10 client.6015 ms_handle_connect on v2:172.21.15.78:6835/3137573764 2023-10-11T09:42:59.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.331+0000 7f6a3effd700 1 -- 172.21.15.78:0/1616091945 <== mds.0 v2:172.21.15.78:6835/3137573764 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x1879450 con 0x1913240 2023-10-11T09:42:59.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.331+0000 7f6a3effd700 10 client.6015 handle_command_reply: tid=0 2023-10-11T09:42:59.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.331+0000 7f6a696e9740 1 client.6015 shutdown 2023-10-11T09:42:59.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.331+0000 7f6a696e9740 1 -- 172.21.15.78:0/1616091945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16f28c0 msgr2=0x1912810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:59.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.331+0000 7f6a696e9740 1 --2- 172.21.15.78:0/1616091945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16f28c0 0x1912810 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6a58007da0 tx=0x7f6a58012520 comp rx=0 tx=0).stop 2023-10-11T09:42:59.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.331+0000 7f6a696e9740 1 -- 172.21.15.78:0/1616091945 shutdown_connections 2023-10-11T09:42:59.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.331+0000 7f6a696e9740 1 --2- 172.21.15.78:0/1616091945 >> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] conn(0x1913240 0x1967c40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.331+0000 7f6a696e9740 1 --2- 172.21.15.78:0/1616091945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16f28c0 0x1912810 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.331+0000 7f6a696e9740 1 --2- 172.21.15.78:0/1616091945 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x16a7c10 0x1914230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.331+0000 7f6a3effd700 0 client.6015 ms_handle_reset on v2:172.21.15.78:6835/3137573764 2023-10-11T09:42:59.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.331+0000 7f6a696e9740 1 --2- 172.21.15.78:0/1616091945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x16a7840 0x193bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.331+0000 7f6a696e9740 1 -- 172.21.15.78:0/1616091945 >> 172.21.15.78:0/1616091945 conn(0x16a7fe0 msgr2=0x1940920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:59.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.335+0000 7f6a696e9740 1 -- 172.21.15.78:0/1616091945 shutdown_connections 2023-10-11T09:42:59.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.335+0000 7f6a696e9740 1 -- 172.21.15.78:0/1616091945 wait complete. 2023-10-11T09:42:59.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.335+0000 7f6a696e9740 20 client.6015 trim_cache size 0 max 16384 2023-10-11T09:42:59.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.351+0000 7f6a696e9740 1 Processor -- start 2023-10-11T09:42:59.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.351+0000 7f6a696e9740 1 -- start start 2023-10-11T09:42:59.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.355+0000 7f6a696e9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16a7840 0x1935d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.355+0000 7f6a696e9740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x16a7c10 0x19afb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.355+0000 7f6a696e9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x16f28c0 0x19b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.355+0000 7f6a696e9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x187f3d0 con 0x16a7840 2023-10-11T09:42:59.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.355+0000 7f6a696e9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x187f540 con 0x16f28c0 2023-10-11T09:42:59.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.355+0000 7f6a696e9740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x193c210 con 0x16a7c10 2023-10-11T09:42:59.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.355+0000 7f6a5ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16a7840 0x1935d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:59.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.355+0000 7f6a5ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16a7840 0x1935d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39136/0 (socket says 172.21.15.78:39136) 2023-10-11T09:42:59.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.355+0000 7f6a5ffff700 1 -- 172.21.15.78:0/3466117968 learned_addr learned my addr 172.21.15.78:0/3466117968 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:59.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.355+0000 7f6a649fa700 1 --2- 172.21.15.78:0/3466117968 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x16f28c0 0x19b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:59.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.355+0000 7f6a5f7fe700 1 --2- 172.21.15.78:0/3466117968 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x16a7c10 0x19afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:59.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.355+0000 7f6a5ffff700 1 -- 172.21.15.78:0/3466117968 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x16a7c10 msgr2=0x19afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:59.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.355+0000 7f6a5ffff700 1 --2- 172.21.15.78:0/3466117968 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x16a7c10 0x19afb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.355+0000 7f6a5ffff700 1 -- 172.21.15.78:0/3466117968 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x16f28c0 msgr2=0x19b00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:59.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.355+0000 7f6a5ffff700 1 --2- 172.21.15.78:0/3466117968 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x16f28c0 0x19b00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.355+0000 7f6a5ffff700 1 -- 172.21.15.78:0/3466117968 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a600cc220 con 0x16a7840 2023-10-11T09:42:59.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.355+0000 7f6a649fa700 1 --2- 172.21.15.78:0/3466117968 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x16f28c0 0x19b00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:42:59.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.355+0000 7f6a5f7fe700 1 --2- 172.21.15.78:0/3466117968 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x16a7c10 0x19afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:42:59.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.355+0000 7f6a5ffff700 1 --2- 172.21.15.78:0/3466117968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16a7840 0x1935d70 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f6a600c6ae0 tx=0x7f6a50010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c709c47988605446 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:59.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.355+0000 7f6a467fc700 1 -- 172.21.15.78:0/3466117968 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5002d0b0 con 0x16a7840 2023-10-11T09:42:59.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.355+0000 7f6a467fc700 1 -- 172.21.15.78:0/3466117968 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5002d210 con 0x16a7840 2023-10-11T09:42:59.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.355+0000 7f6a467fc700 1 -- 172.21.15.78:0/3466117968 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5002d9a0 con 0x16a7840 2023-10-11T09:42:59.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.355+0000 7f6a696e9740 1 -- 172.21.15.78:0/3466117968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16a7840 msgr2=0x1935d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:59.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.355+0000 7f6a696e9740 1 --2- 172.21.15.78:0/3466117968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16a7840 0x1935d70 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f6a600c6ae0 tx=0x7f6a50010000 comp rx=0 tx=0).stop 2023-10-11T09:42:59.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.355+0000 7f6a696e9740 1 -- 172.21.15.78:0/3466117968 shutdown_connections 2023-10-11T09:42:59.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.355+0000 7f6a696e9740 1 --2- 172.21.15.78:0/3466117968 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x16f28c0 0x19b00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.355+0000 7f6a696e9740 1 --2- 172.21.15.78:0/3466117968 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x16a7c10 0x19afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.355+0000 7f6a696e9740 1 --2- 172.21.15.78:0/3466117968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16a7840 0x1935d70 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.355+0000 7f6a696e9740 1 -- 172.21.15.78:0/3466117968 >> 172.21.15.78:0/3466117968 conn(0x16a7fe0 msgr2=0x193d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:59.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.355+0000 7f6a696e9740 1 -- 172.21.15.78:0/3466117968 shutdown_connections 2023-10-11T09:42:59.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.355+0000 7f6a696e9740 1 -- 172.21.15.78:0/3466117968 wait complete. 2023-10-11T09:42:59.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.359+0000 7f6a696e9740 1 -- start start 2023-10-11T09:42:59.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.359+0000 7f6a696e9740 1 Processor -- start 2023-10-11T09:42:59.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.359+0000 7f6a696e9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16a7840 0x19af160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.359+0000 7f6a696e9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x16a7c10 0x19b2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.359+0000 7f6a696e9740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x16f28c0 0x19b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.359+0000 7f6a696e9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x18888c0 con 0x16a7840 2023-10-11T09:42:59.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.359+0000 7f6a696e9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x187dab0 con 0x16a7c10 2023-10-11T09:42:59.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.359+0000 7f6a696e9740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x19af6a0 con 0x16f28c0 2023-10-11T09:42:59.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.359+0000 7f6a696e9740 10 client.0 _ll_register_callbacks cb 0x187ee10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:42:59.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.359+0000 7f6a5ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16a7840 0x19af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:59.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.359+0000 7f6a5ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16a7840 0x19af160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39142/0 (socket says 172.21.15.78:39142) 2023-10-11T09:42:59.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.359+0000 7f6a5ffff700 1 -- 172.21.15.78:0/4052191509 learned_addr learned my addr 172.21.15.78:0/4052191509 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:42:59.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.359+0000 7f6a5ffff700 1 -- 172.21.15.78:0/4052191509 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x16f28c0 msgr2=0x19b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:59.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.359+0000 7f6a5ffff700 1 --2- 172.21.15.78:0/4052191509 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x16f28c0 0x19b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.359+0000 7f6a5ffff700 1 -- 172.21.15.78:0/4052191509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x16a7c10 msgr2=0x19b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:42:59.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.359+0000 7f6a5ffff700 1 --2- 172.21.15.78:0/4052191509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x16a7c10 0x19b2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.359+0000 7f6a5ffff700 1 -- 172.21.15.78:0/4052191509 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a600d02e0 con 0x16a7840 2023-10-11T09:42:59.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.363+0000 7f6a5ffff700 1 --2- 172.21.15.78:0/4052191509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16a7840 0x19af160 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f6a5000a010 tx=0x7f6a5000e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed810771ae1f0110 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:42:59.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.363+0000 7f6a3effd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:42:59.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.363+0000 7f6a3effd700 1 -- 172.21.15.78:0/4052191509 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5002dba0 con 0x16a7840 2023-10-11T09:42:59.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.363+0000 7f6a3effd700 1 -- 172.21.15.78:0/4052191509 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 6753+0+0 (secure 0 0 0) 0x7f6a5002dd00 con 0x16a7840 2023-10-11T09:42:59.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.363+0000 7f6a696e9740 1 -- 172.21.15.78:0/4052191509 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a54057a80 con 0x16a7840 2023-10-11T09:42:59.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.363+0000 7f6a3effd700 1 -- 172.21.15.78:0/4052191509 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a50011630 con 0x16a7840 2023-10-11T09:42:59.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.363+0000 7f6a696e9740 10 client.6021 fetch_fsmap learned FSMap version 29 2023-10-11T09:42:59.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.363+0000 7f6a696e9740 1 -- 172.21.15.78:0/4052191509 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x1846820 con 0x16a7840 2023-10-11T09:42:59.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.363+0000 7f6a3effd700 1 -- 172.21.15.78:0/4052191509 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a54057a80 con 0x16a7840 2023-10-11T09:42:59.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.363+0000 7f6a3effd700 1 -- 172.21.15.78:0/4052191509 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f6a50033020 con 0x16a7840 2023-10-11T09:42:59.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.363+0000 7f6a696e9740 10 client.6021 fetch_fsmap finished waiting for FSMap version 29 2023-10-11T09:42:59.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.363+0000 7f6a696e9740 10 client.6021 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-10-11T09:42:59.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.363+0000 7f6a696e9740 20 client.6021 populate_metadata read hostname 'smithi078' 2023-10-11T09:42:59.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.363+0000 7f6a696e9740 1 --2- 172.21.15.78:0/4052191509 >> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] conn(0x193a830 0x1938c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:42:59.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.363+0000 7f6a696e9740 4 client.6021 mds_command: new command op to 5922 tid=0[{"prefix": "scrub status"}] 2023-10-11T09:42:59.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.363+0000 7f6a696e9740 1 -- 172.21.15.78:0/4052191509 --> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x16ff610 con 0x193a830 2023-10-11T09:42:59.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.363+0000 7f6a5f7fe700 1 --2- 172.21.15.78:0/4052191509 >> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] conn(0x193a830 0x1938c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:42:59.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.363+0000 7f6a3effd700 10 client.6021 ms_handle_connect on v2:172.21.15.78:6835/3137573764 2023-10-11T09:42:59.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.363+0000 7f6a5f7fe700 1 --2- 172.21.15.78:0/4052191509 >> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] conn(0x193a830 0x1938c40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44be10eda19bba43 server_cookie=66b37dca8942ada0 in_seq=0 out_seq=0 2023-10-11T09:42:59.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.363+0000 7f6a3effd700 1 -- 172.21.15.78:0/4052191509 <== mds.0 v2:172.21.15.78:6835/3137573764 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x16ff610 con 0x193a830 2023-10-11T09:42:59.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.363+0000 7f6a3effd700 10 client.6021 handle_command_reply: tid=0 2023-10-11T09:42:59.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.363+0000 7f6a696e9740 1 client.6021 shutdown 2023-10-11T09:42:59.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.363+0000 7f6a696e9740 1 -- 172.21.15.78:0/4052191509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16a7840 msgr2=0x19af160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:59.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.363+0000 7f6a696e9740 1 --2- 172.21.15.78:0/4052191509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16a7840 0x19af160 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f6a5000a010 tx=0x7f6a5000e000 comp rx=0 tx=0).stop 2023-10-11T09:42:59.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.363+0000 7f6a696e9740 1 -- 172.21.15.78:0/4052191509 shutdown_connections 2023-10-11T09:42:59.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.363+0000 7f6a696e9740 1 --2- 172.21.15.78:0/4052191509 >> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] conn(0x193a830 0x1938c40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.363+0000 7f6a696e9740 1 --2- 172.21.15.78:0/4052191509 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x16f28c0 0x19b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.363+0000 7f6a696e9740 1 --2- 172.21.15.78:0/4052191509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x16a7c10 0x19b2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.363+0000 7f6a696e9740 1 --2- 172.21.15.78:0/4052191509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16a7840 0x19af160 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.363+0000 7f6a3effd700 0 client.6021 ms_handle_reset on v2:172.21.15.78:6835/3137573764 2023-10-11T09:42:59.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.363+0000 7f6a696e9740 1 -- 172.21.15.78:0/4052191509 >> 172.21.15.78:0/4052191509 conn(0x16a7fe0 msgr2=0x193cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:59.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.363+0000 7f6a696e9740 1 -- 172.21.15.78:0/4052191509 shutdown_connections 2023-10-11T09:42:59.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.363+0000 7f6a696e9740 1 -- 172.21.15.78:0/4052191509 wait complete. 2023-10-11T09:42:59.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.367+0000 7f6a696e9740 20 client.6021 trim_cache size 0 max 16384 2023-10-11T09:42:59.378 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:42:59.379 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-11T09:42:59.379 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2023-10-11T09:42:59.379 INFO:teuthology.orchestra.run.smithi078.stdout: "adf0ee7c-b37b-4787-9377-54135a236381": { 2023-10-11T09:42:59.379 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2023-10-11T09:42:59.379 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "adf0ee7c-b37b-4787-9377-54135a236381", 2023-10-11T09:42:59.380 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,repair" 2023-10-11T09:42:59.380 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:42:59.380 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:42:59.380 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:42:59.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.367+0000 7f6a6645d700 1 -- 172.21.15.78:0/1756448190 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6a4c065ed0 msgr2=0x7f6a4c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:59.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.367+0000 7f6a6645d700 1 --2- 172.21.15.78:0/1756448190 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6a4c065ed0 0x7f6a4c068380 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6a50000c00 tx=0x7f6a50013040 comp rx=0 tx=0).stop 2023-10-11T09:42:59.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.367+0000 7f6a6645d700 1 -- 172.21.15.78:0/1756448190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a600b8890 msgr2=0x7f6a600c80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:42:59.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.367+0000 7f6a6645d700 1 --2- 172.21.15.78:0/1756448190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a600b8890 0x7f6a600c80c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6a540011e0 tx=0x7f6a5400ca90 comp rx=0 tx=0).stop 2023-10-11T09:42:59.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.367+0000 7f6a6645d700 1 -- 172.21.15.78:0/1756448190 shutdown_connections 2023-10-11T09:42:59.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.367+0000 7f6a6645d700 1 --2- 172.21.15.78:0/1756448190 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6a4c065ed0 0x7f6a4c068380 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.367+0000 7f6a6645d700 1 --2- 172.21.15.78:0/1756448190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a60160a80 0x7f6a600cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.367+0000 7f6a6645d700 1 --2- 172.21.15.78:0/1756448190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a600b8890 0x7f6a600c80c0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.367+0000 7f6a6645d700 1 --2- 172.21.15.78:0/1756448190 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6a600b84c0 0x7f6a60145f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:42:59.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.367+0000 7f6a6645d700 1 -- 172.21.15.78:0/1756448190 >> 172.21.15.78:0/1756448190 conn(0x7f6a6006cb50 msgr2=0x7f6a600b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:42:59.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.367+0000 7f6a6645d700 1 -- 172.21.15.78:0/1756448190 shutdown_connections 2023-10-11T09:42:59.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:42:59.367+0000 7f6a6645d700 1 -- 172.21.15.78:0/1756448190 wait complete. 2023-10-11T09:42:59.391 INFO:tasks.cephfs.filesystem:scrub status for tag:adf0ee7c-b37b-4787-9377-54135a236381 - {'path': '/', 'tag': 'adf0ee7c-b37b-4787-9377-54135a236381', 'options': 'recursive,repair'} 2023-10-11T09:42:59.446 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:42:59.454 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:43:29.393 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 scrub status 2023-10-11T09:43:29.525 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:43:29.533 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:43:29.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.567+0000 7f47d6fd2700 1 Processor -- start 2023-10-11T09:43:29.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.567+0000 7f47d6fd2700 1 -- start start 2023-10-11T09:43:29.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.567+0000 7f47d6fd2700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f47d00d29d0 0x7f47d00b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:29.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.567+0000 7f47d6fd2700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47d00b47f0 con 0x7f47d00b3610 2023-10-11T09:43:29.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.567+0000 7f47d6fd2700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47d00b8130 con 0x7f47d00b3240 2023-10-11T09:43:29.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.567+0000 7f47d6fd2700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47d00d2e30 con 0x7f47d00d29d0 2023-10-11T09:43:29.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.567+0000 7f47d57cf700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f47d00b3610 0x7f47d00b85c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:41638/0 (socket says 172.21.15.78:41638) 2023-10-11T09:43:29.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.567+0000 7f47d57cf700 1 -- 172.21.15.78:0/3268137470 learned_addr learned my addr 172.21.15.78:0/3268137470 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:43:29.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.567+0000 7f47d4fce700 1 -- 172.21.15.78:0/3268137470 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 598538824 0 0) 0x7f47c0001110 con 0x7f47d00b3610 2023-10-11T09:43:29.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.571+0000 7f47d67d1700 1 --2- 172.21.15.78:0/3268137470 >> v2:172.21.15.133:3300/0 conn(0x7f47d00d29d0 0x7f47d00b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:29.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.571+0000 7f47d4fce700 1 -- 172.21.15.78:0/3268137470 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47bc0035d0 con 0x7f47d00b3610 2023-10-11T09:43:29.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.571+0000 7f47d4fce700 1 -- 172.21.15.78:0/3268137470 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2184778011 0 0) 0x7f47bc0035d0 con 0x7f47d00b3610 2023-10-11T09:43:29.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.571+0000 7f47d4fce700 1 -- 172.21.15.78:0/3268137470 >> v2:172.21.15.133:3300/0 conn(0x7f47d00d29d0 msgr2=0x7f47d00b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:29.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.571+0000 7f47d4fce700 1 --2- 172.21.15.78:0/3268137470 >> v2:172.21.15.133:3300/0 conn(0x7f47d00d29d0 0x7f47d00b3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:29.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.571+0000 7f47d4fce700 1 -- 172.21.15.78:0/3268137470 >> v1:172.21.15.133:6789/0 conn(0x7f47d00b3240 legacy=0x7f47d015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:29.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.571+0000 7f47d4fce700 1 -- 172.21.15.78:0/3268137470 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d00b41e0 con 0x7f47d00b3610 2023-10-11T09:43:29.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.571+0000 7f47d4fce700 1 -- 172.21.15.78:0/3268137470 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f47c0001800 con 0x7f47d00b3610 2023-10-11T09:43:29.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.571+0000 7f47d67d1700 1 --2- 172.21.15.78:0/3268137470 >> v2:172.21.15.133:3300/0 conn(0x7f47d00d29d0 0x7f47d00b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:43:29.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.571+0000 7f47d4fce700 1 -- 172.21.15.78:0/3268137470 >> v1:172.21.15.78:6789/0 conn(0x7f47d00b3610 legacy=0x7f47d00b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:29.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.571+0000 7f47d4fce700 1 --2- 172.21.15.78:0/3268137470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47bc003a20 0x7f47bc003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:29.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.571+0000 7f47d4fce700 1 --2- 172.21.15.78:0/3268137470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47bc004870 0x7f47bc004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:29.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.571+0000 7f47d4fce700 1 --2- 172.21.15.78:0/3268137470 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f47bc005340 0x7f47bc00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:29.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.571+0000 7f47d4fce700 1 -- 172.21.15.78:0/3268137470 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d00d2e30 con 0x7f47bc003a20 2023-10-11T09:43:29.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.571+0000 7f47d4fce700 1 -- 172.21.15.78:0/3268137470 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc00c200 con 0x7f47bc004870 2023-10-11T09:43:29.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.571+0000 7f47d4fce700 1 -- 172.21.15.78:0/3268137470 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc00c3c0 con 0x7f47bc005340 2023-10-11T09:43:29.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.571+0000 7f47d5fd0700 1 --2- 172.21.15.78:0/3268137470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47bc003a20 0x7f47bc003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:29.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.571+0000 7f47d5fd0700 1 -- 172.21.15.78:0/3268137470 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f47bc005340 msgr2=0x7f47bc00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:29.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.571+0000 7f47d5fd0700 1 --2- 172.21.15.78:0/3268137470 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f47bc005340 0x7f47bc00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:29.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.571+0000 7f47d5fd0700 1 -- 172.21.15.78:0/3268137470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47bc004870 msgr2=0x7f47bc004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:29.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.571+0000 7f47d5fd0700 1 --2- 172.21.15.78:0/3268137470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47bc004870 0x7f47bc004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:29.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.571+0000 7f47d5fd0700 1 -- 172.21.15.78:0/3268137470 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47bc0035d0 con 0x7f47bc003a20 2023-10-11T09:43:29.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.571+0000 7f47d67d1700 1 -- 172.21.15.78:0/3268137470 reap_dead start 2023-10-11T09:43:29.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.571+0000 7f47d5fd0700 1 --2- 172.21.15.78:0/3268137470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47bc003a20 0x7f47bc003e30 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f47cc0017e0 tx=0x7f47bc0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c81c1f01bc0218af server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:29.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.571+0000 7f47d4fce700 1 -- 172.21.15.78:0/3268137470 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc0052d0 con 0x7f47bc003a20 2023-10-11T09:43:29.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.571+0000 7f47d4fce700 1 -- 172.21.15.78:0/3268137470 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47cc005430 con 0x7f47bc003a20 2023-10-11T09:43:29.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.571+0000 7f47d4fce700 1 -- 172.21.15.78:0/3268137470 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc0055a0 con 0x7f47bc003a20 2023-10-11T09:43:29.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.571+0000 7f47d6fd2700 1 -- 172.21.15.78:0/3268137470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47bc003a20 msgr2=0x7f47bc003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:29.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.571+0000 7f47d6fd2700 1 --2- 172.21.15.78:0/3268137470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47bc003a20 0x7f47bc003e30 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f47cc0017e0 tx=0x7f47bc0058a0 comp rx=0 tx=0).stop 2023-10-11T09:43:29.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.571+0000 7f47d6fd2700 1 -- 172.21.15.78:0/3268137470 shutdown_connections 2023-10-11T09:43:29.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.571+0000 7f47d6fd2700 1 --2- 172.21.15.78:0/3268137470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47bc003a20 0x7f47bc003e30 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:29.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.571+0000 7f47d6fd2700 1 -- 172.21.15.78:0/3268137470 >> 172.21.15.78:0/3268137470 conn(0x7f47d009c260 msgr2=0x7f47d00d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:29.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.571+0000 7f47d6fd2700 1 -- 172.21.15.78:0/3268137470 shutdown_connections 2023-10-11T09:43:29.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.571+0000 7f47d6fd2700 1 -- 172.21.15.78:0/3268137470 wait complete. 2023-10-11T09:43:29.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.575+0000 7f47d6fd2700 1 Processor -- start 2023-10-11T09:43:29.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.575+0000 7f47d6fd2700 1 -- start start 2023-10-11T09:43:29.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.575+0000 7f47d6fd2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47bc003a20 0x7f47d00b6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:29.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.575+0000 7f47d6fd2700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47d00b68a0 0x7f47d009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:29.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.575+0000 7f47d6fd2700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f47d009f240 0x7f47d009f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:29.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.575+0000 7f47d6fd2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d00a09b0 con 0x7f47bc003a20 2023-10-11T09:43:29.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.575+0000 7f47d6fd2700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d00b6d10 con 0x7f47d00b68a0 2023-10-11T09:43:29.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.575+0000 7f47d6fd2700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d009fd30 con 0x7f47d009f240 2023-10-11T09:43:29.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.575+0000 7f47d5fd0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47bc003a20 0x7f47d00b6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:29.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.575+0000 7f47d5fd0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47bc003a20 0x7f47d00b6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49180/0 (socket says 172.21.15.78:49180) 2023-10-11T09:43:29.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.575+0000 7f47d5fd0700 1 -- 172.21.15.78:0/1471782634 learned_addr learned my addr 172.21.15.78:0/1471782634 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:29.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.575+0000 7f47d67d1700 1 --2- 172.21.15.78:0/1471782634 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f47d009f240 0x7f47d009f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:29.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.575+0000 7f47d5fd0700 1 -- 172.21.15.78:0/1471782634 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f47d009f240 msgr2=0x7f47d009f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:29.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.575+0000 7f47d57cf700 1 --2- 172.21.15.78:0/1471782634 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47d00b68a0 0x7f47d009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:29.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.575+0000 7f47d5fd0700 1 --2- 172.21.15.78:0/1471782634 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f47d009f240 0x7f47d009f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:29.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.575+0000 7f47d5fd0700 1 -- 172.21.15.78:0/1471782634 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47d00b68a0 msgr2=0x7f47d009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:29.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.575+0000 7f47d5fd0700 1 --2- 172.21.15.78:0/1471782634 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47d00b68a0 0x7f47d009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:29.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.575+0000 7f47d5fd0700 1 -- 172.21.15.78:0/1471782634 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d00a0010 con 0x7f47bc003a20 2023-10-11T09:43:29.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.575+0000 7f47d57cf700 1 --2- 172.21.15.78:0/1471782634 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47d00b68a0 0x7f47d009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:43:29.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.575+0000 7f47d67d1700 1 --2- 172.21.15.78:0/1471782634 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f47d009f240 0x7f47d009f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:43:29.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.575+0000 7f47d5fd0700 1 --2- 172.21.15.78:0/1471782634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47bc003a20 0x7f47d00b6360 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00c1d0 tx=0x7f47cc001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:29.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.575+0000 7f47c67fc700 1 -- 172.21.15.78:0/1471782634 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc0055a0 con 0x7f47bc003a20 2023-10-11T09:43:29.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.575+0000 7f47c67fc700 1 -- 172.21.15.78:0/1471782634 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47cc001290 con 0x7f47bc003a20 2023-10-11T09:43:29.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.575+0000 7f47c67fc700 1 -- 172.21.15.78:0/1471782634 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc01d6c0 con 0x7f47bc003a20 2023-10-11T09:43:29.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.575+0000 7f47d6fd2700 1 -- 172.21.15.78:0/1471782634 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47d00a02f0 con 0x7f47bc003a20 2023-10-11T09:43:29.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.579+0000 7f47d6fd2700 1 -- 172.21.15.78:0/1471782634 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47d015de50 con 0x7f47bc003a20 2023-10-11T09:43:29.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.579+0000 7f47c67fc700 1 -- 172.21.15.78:0/1471782634 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f47cc019040 con 0x7f47bc003a20 2023-10-11T09:43:29.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.583+0000 7f47c67fc700 1 --2- 172.21.15.78:0/1471782634 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f47b4065f00 0x7f47b40683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:29.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.583+0000 7f47c67fc700 1 -- 172.21.15.78:0/1471782634 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6753+0+0 (secure 0 0 0) 0x7f47cc084750 con 0x7f47bc003a20 2023-10-11T09:43:29.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.583+0000 7f47d57cf700 1 --2- 172.21.15.78:0/1471782634 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f47b4065f00 0x7f47b40683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:29.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.583+0000 7f47db4bf740 1 Processor -- start 2023-10-11T09:43:29.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.583+0000 7f47db4bf740 1 -- start start 2023-10-11T09:43:29.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.583+0000 7f47d57cf700 1 --2- 172.21.15.78:0/1471782634 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f47b4065f00 0x7f47b40683b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f47d00b7030 tx=0x7f47c0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:29.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.583+0000 7f47db4bf740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1285c10 0x151b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:29.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.583+0000 7f47db4bf740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1285840 0x153cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:29.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.583+0000 7f47db4bf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x12d08c0 0x153d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:29.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.583+0000 7f47db4bf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x1465630 con 0x12d08c0 2023-10-11T09:43:29.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.583+0000 7f47db4bf740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x1462dc0 con 0x1285840 2023-10-11T09:43:29.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.583+0000 7f47db4bf740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x145e6b0 con 0x1285c10 2023-10-11T09:43:29.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.583+0000 7f47d67d1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x12d08c0 0x153d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:29.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.583+0000 7f47d5fd0700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1285c10 0x151b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:29.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.583+0000 7f47d67d1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x12d08c0 0x153d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49186/0 (socket says 172.21.15.78:49186) 2023-10-11T09:43:29.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.583+0000 7f47d67d1700 1 -- 172.21.15.78:0/3165937527 learned_addr learned my addr 172.21.15.78:0/3165937527 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:29.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.583+0000 7f47d57cf700 1 --2- 172.21.15.78:0/3165937527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1285840 0x153cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:29.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.583+0000 7f47d67d1700 1 -- 172.21.15.78:0/3165937527 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1285c10 msgr2=0x151b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:29.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.583+0000 7f47d67d1700 1 --2- 172.21.15.78:0/3165937527 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1285c10 0x151b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:29.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.583+0000 7f47d67d1700 1 -- 172.21.15.78:0/3165937527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1285840 msgr2=0x153cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:29.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.583+0000 7f47d67d1700 1 --2- 172.21.15.78:0/3165937527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1285840 0x153cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:29.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.583+0000 7f47d67d1700 1 -- 172.21.15.78:0/3165937527 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x14845b0 con 0x12d08c0 2023-10-11T09:43:29.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.587+0000 7f47d67d1700 1 --2- 172.21.15.78:0/3165937527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x12d08c0 0x153d2d0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f47d00672d0 tx=0x7f47c8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd207dc0cbc1f292 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:29.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.587+0000 7f47b37fe700 1 -- 172.21.15.78:0/3165937527 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c800e050 con 0x12d08c0 2023-10-11T09:43:29.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.587+0000 7f47b37fe700 1 -- 172.21.15.78:0/3165937527 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c800f040 con 0x12d08c0 2023-10-11T09:43:29.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.587+0000 7f47b37fe700 1 -- 172.21.15.78:0/3165937527 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c8014460 con 0x12d08c0 2023-10-11T09:43:29.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.587+0000 7f47db4bf740 1 -- 172.21.15.78:0/3165937527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x12d08c0 msgr2=0x153d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:29.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.587+0000 7f47db4bf740 1 --2- 172.21.15.78:0/3165937527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x12d08c0 0x153d2d0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f47d00672d0 tx=0x7f47c8004af0 comp rx=0 tx=0).stop 2023-10-11T09:43:29.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.587+0000 7f47db4bf740 1 -- 172.21.15.78:0/3165937527 shutdown_connections 2023-10-11T09:43:29.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.587+0000 7f47db4bf740 1 --2- 172.21.15.78:0/3165937527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x12d08c0 0x153d2d0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:29.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.587+0000 7f47db4bf740 1 --2- 172.21.15.78:0/3165937527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1285840 0x153cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:29.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.587+0000 7f47db4bf740 1 --2- 172.21.15.78:0/3165937527 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1285c10 0x151b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:29.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.587+0000 7f47db4bf740 1 -- 172.21.15.78:0/3165937527 >> 172.21.15.78:0/3165937527 conn(0x1285fe0 msgr2=0x151e350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:29.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.587+0000 7f47db4bf740 1 -- 172.21.15.78:0/3165937527 shutdown_connections 2023-10-11T09:43:29.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.587+0000 7f47db4bf740 1 -- 172.21.15.78:0/3165937527 wait complete. 2023-10-11T09:43:29.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.587+0000 7f47db4bf740 1 -- start start 2023-10-11T09:43:29.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.587+0000 7f47db4bf740 1 Processor -- start 2023-10-11T09:43:29.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.587+0000 7f47db4bf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1285840 0x1517400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:29.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.587+0000 7f47db4bf740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1285c10 0x151a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:29.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.587+0000 7f47db4bf740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x12d08c0 0x151ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:29.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.587+0000 7f47db4bf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x1421c10 con 0x1285840 2023-10-11T09:43:29.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.587+0000 7f47db4bf740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x14218b0 con 0x1285c10 2023-10-11T09:43:29.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.587+0000 7f47db4bf740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x130ac50 con 0x12d08c0 2023-10-11T09:43:29.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.587+0000 7f47db4bf740 10 client.0 _ll_register_callbacks cb 0x1457ff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:43:29.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.591+0000 7f47d5fd0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1285840 0x1517400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:29.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.591+0000 7f47d5fd0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1285840 0x1517400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49200/0 (socket says 172.21.15.78:49200) 2023-10-11T09:43:29.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.591+0000 7f47d5fd0700 1 -- 172.21.15.78:0/248593611 learned_addr learned my addr 172.21.15.78:0/248593611 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:29.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.591+0000 7f47d57cf700 1 --2- 172.21.15.78:0/248593611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1285c10 0x151a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:29.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.591+0000 7f47d67d1700 1 --2- 172.21.15.78:0/248593611 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x12d08c0 0x151ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:29.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.591+0000 7f47d5fd0700 1 -- 172.21.15.78:0/248593611 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x12d08c0 msgr2=0x151ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:29.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.591+0000 7f47d5fd0700 1 --2- 172.21.15.78:0/248593611 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x12d08c0 0x151ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:29.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.591+0000 7f47d5fd0700 1 -- 172.21.15.78:0/248593611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1285c10 msgr2=0x151a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:29.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.591+0000 7f47d5fd0700 1 --2- 172.21.15.78:0/248593611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1285c10 0x151a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:29.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.591+0000 7f47d5fd0700 1 -- 172.21.15.78:0/248593611 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c8009bc0 con 0x1285840 2023-10-11T09:43:29.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.591+0000 7f47d67d1700 1 --2- 172.21.15.78:0/248593611 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x12d08c0 0x151ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:43:29.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.591+0000 7f47d5fd0700 1 --2- 172.21.15.78:0/248593611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1285840 0x1517400 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f47cc055490 tx=0x7f47cc0151d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39af1302c356c656 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:29.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.591+0000 7f47b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:43:29.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.591+0000 7f47b0ff9700 1 -- 172.21.15.78:0/248593611 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc055790 con 0x1285840 2023-10-11T09:43:29.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.591+0000 7f47b0ff9700 1 -- 172.21.15.78:0/248593611 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 6753+0+0 (secure 0 0 0) 0x7f47cc0595c0 con 0x1285840 2023-10-11T09:43:29.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.591+0000 7f47db4bf740 1 -- 172.21.15.78:0/248593611 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1457730 con 0x1285840 2023-10-11T09:43:29.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.591+0000 7f47b0ff9700 1 -- 172.21.15.78:0/248593611 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47cc05d770 con 0x1285840 2023-10-11T09:43:29.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.591+0000 7f47db4bf740 10 client.6033 fetch_fsmap learned FSMap version 29 2023-10-11T09:43:29.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.591+0000 7f47db4bf740 1 -- 172.21.15.78:0/248593611 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x12d1b10 con 0x1285840 2023-10-11T09:43:29.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.591+0000 7f47b0ff9700 1 -- 172.21.15.78:0/248593611 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47cc070080 con 0x1285840 2023-10-11T09:43:29.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.591+0000 7f47b0ff9700 1 -- 172.21.15.78:0/248593611 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f47cc059170 con 0x1285840 2023-10-11T09:43:29.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.591+0000 7f47db4bf740 10 client.6033 fetch_fsmap finished waiting for FSMap version 29 2023-10-11T09:43:29.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.591+0000 7f47db4bf740 10 client.6033 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-10-11T09:43:29.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.591+0000 7f47db4bf740 20 client.6033 populate_metadata read hostname 'smithi078' 2023-10-11T09:43:29.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.591+0000 7f47db4bf740 1 --2- 172.21.15.78:0/248593611 >> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] conn(0x14fb810 0x154fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:29.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.591+0000 7f47db4bf740 4 client.6033 mds_command: new command op to 5922 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-11T09:43:29.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.591+0000 7f47db4bf740 1 -- 172.21.15.78:0/248593611 --> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1457450 con 0x14fb810 2023-10-11T09:43:29.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.591+0000 7f47d57cf700 1 --2- 172.21.15.78:0/248593611 >> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] conn(0x14fb810 0x154fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:29.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.591+0000 7f47d57cf700 1 --2- 172.21.15.78:0/248593611 >> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] conn(0x14fb810 0x154fd70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edab831277aa20c6 server_cookie=81d05c6825863ec3 in_seq=0 out_seq=0 2023-10-11T09:43:29.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.591+0000 7f47b0ff9700 10 client.6033 ms_handle_connect on v2:172.21.15.78:6835/3137573764 2023-10-11T09:43:29.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.595+0000 7f47b0ff9700 1 -- 172.21.15.78:0/248593611 <== mds.0 v2:172.21.15.78:6835/3137573764 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x1457450 con 0x14fb810 2023-10-11T09:43:29.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.595+0000 7f47b0ff9700 10 client.6033 handle_command_reply: tid=0 2023-10-11T09:43:29.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.595+0000 7f47db4bf740 1 client.6033 shutdown 2023-10-11T09:43:29.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.595+0000 7f47db4bf740 1 -- 172.21.15.78:0/248593611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1285840 msgr2=0x1517400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:29.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.595+0000 7f47db4bf740 1 --2- 172.21.15.78:0/248593611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1285840 0x1517400 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f47cc055490 tx=0x7f47cc0151d0 comp rx=0 tx=0).stop 2023-10-11T09:43:29.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.595+0000 7f47db4bf740 1 -- 172.21.15.78:0/248593611 shutdown_connections 2023-10-11T09:43:29.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.595+0000 7f47db4bf740 1 --2- 172.21.15.78:0/248593611 >> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] conn(0x14fb810 0x154fd70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:29.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.595+0000 7f47db4bf740 1 --2- 172.21.15.78:0/248593611 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x12d08c0 0x151ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:29.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.595+0000 7f47db4bf740 1 --2- 172.21.15.78:0/248593611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1285c10 0x151a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:29.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.595+0000 7f47db4bf740 1 --2- 172.21.15.78:0/248593611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1285840 0x1517400 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:29.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.595+0000 7f47db4bf740 1 -- 172.21.15.78:0/248593611 >> 172.21.15.78:0/248593611 conn(0x1285fe0 msgr2=0x151e750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:29.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.595+0000 7f47b0ff9700 0 client.6033 ms_handle_reset on v2:172.21.15.78:6835/3137573764 2023-10-11T09:43:29.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.595+0000 7f47db4bf740 1 -- 172.21.15.78:0/248593611 shutdown_connections 2023-10-11T09:43:29.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.595+0000 7f47db4bf740 1 -- 172.21.15.78:0/248593611 wait complete. 2023-10-11T09:43:29.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.595+0000 7f47db4bf740 20 client.6033 trim_cache size 0 max 16384 2023-10-11T09:43:29.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.615+0000 7f47db4bf740 1 Processor -- start 2023-10-11T09:43:29.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.615+0000 7f47db4bf740 1 -- start start 2023-10-11T09:43:29.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.615+0000 7f47db4bf740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1285840 0x151e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:29.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.615+0000 7f47db4bf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1285c10 0x14f6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:29.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.615+0000 7f47db4bf740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x12d08c0 0x14f6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:29.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.615+0000 7f47db4bf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x145bab0 con 0x1285c10 2023-10-11T09:43:29.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.615+0000 7f47db4bf740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x1421c10 con 0x1285840 2023-10-11T09:43:29.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.615+0000 7f47db4bf740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x1406480 con 0x12d08c0 2023-10-11T09:43:29.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.615+0000 7f47d57cf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1285c10 0x14f6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:29.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.615+0000 7f47d5fd0700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1285840 0x151e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:29.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.615+0000 7f47d57cf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1285c10 0x14f6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49204/0 (socket says 172.21.15.78:49204) 2023-10-11T09:43:29.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.615+0000 7f47d57cf700 1 -- 172.21.15.78:0/3542981 learned_addr learned my addr 172.21.15.78:0/3542981 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:29.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.615+0000 7f47d67d1700 1 --2- 172.21.15.78:0/3542981 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x12d08c0 0x14f6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:29.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.615+0000 7f47d57cf700 1 -- 172.21.15.78:0/3542981 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x12d08c0 msgr2=0x14f6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:29.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.615+0000 7f47d57cf700 1 --2- 172.21.15.78:0/3542981 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x12d08c0 0x14f6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:29.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.615+0000 7f47d57cf700 1 -- 172.21.15.78:0/3542981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1285840 msgr2=0x151e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:29.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.615+0000 7f47d57cf700 1 --2- 172.21.15.78:0/3542981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1285840 0x151e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:29.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.615+0000 7f47d57cf700 1 -- 172.21.15.78:0/3542981 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d00afeb0 con 0x1285c10 2023-10-11T09:43:29.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.615+0000 7f47d5fd0700 1 --2- 172.21.15.78:0/3542981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1285840 0x151e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:43:29.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.615+0000 7f47d57cf700 1 --2- 172.21.15.78:0/3542981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1285c10 0x14f6840 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f47c0001370 tx=0x7f47c000b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=223b19b2233c29dd server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:29.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.619+0000 7f47b3fff700 1 -- 172.21.15.78:0/3542981 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c0007480 con 0x1285c10 2023-10-11T09:43:29.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.619+0000 7f47b3fff700 1 -- 172.21.15.78:0/3542981 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c0028070 con 0x1285c10 2023-10-11T09:43:29.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.619+0000 7f47b3fff700 1 -- 172.21.15.78:0/3542981 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c00077e0 con 0x1285c10 2023-10-11T09:43:29.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.619+0000 7f47db4bf740 1 -- 172.21.15.78:0/3542981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1285c10 msgr2=0x14f6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:29.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.619+0000 7f47db4bf740 1 --2- 172.21.15.78:0/3542981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1285c10 0x14f6840 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f47c0001370 tx=0x7f47c000b000 comp rx=0 tx=0).stop 2023-10-11T09:43:29.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.619+0000 7f47db4bf740 1 -- 172.21.15.78:0/3542981 shutdown_connections 2023-10-11T09:43:29.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.619+0000 7f47db4bf740 1 --2- 172.21.15.78:0/3542981 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x12d08c0 0x14f6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:29.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.619+0000 7f47db4bf740 1 --2- 172.21.15.78:0/3542981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1285c10 0x14f6840 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:29.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.619+0000 7f47db4bf740 1 --2- 172.21.15.78:0/3542981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1285840 0x151e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:29.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.619+0000 7f47db4bf740 1 -- 172.21.15.78:0/3542981 >> 172.21.15.78:0/3542981 conn(0x1285fe0 msgr2=0x1544f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:29.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.619+0000 7f47db4bf740 1 -- 172.21.15.78:0/3542981 shutdown_connections 2023-10-11T09:43:29.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.619+0000 7f47db4bf740 1 -- 172.21.15.78:0/3542981 wait complete. 2023-10-11T09:43:29.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.619+0000 7f47db4bf740 1 -- start start 2023-10-11T09:43:29.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.619+0000 7f47db4bf740 1 Processor -- start 2023-10-11T09:43:29.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.623+0000 7f47db4bf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1285840 0x15a1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:29.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.623+0000 7f47db4bf740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1285c10 0x154c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:29.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.623+0000 7f47db4bf740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x12d08c0 0x154c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:29.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.623+0000 7f47db4bf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x1423360 con 0x1285840 2023-10-11T09:43:29.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.623+0000 7f47db4bf740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x141eb60 con 0x12d08c0 2023-10-11T09:43:29.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.623+0000 7f47db4bf740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x140b6c0 con 0x1285c10 2023-10-11T09:43:29.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.623+0000 7f47db4bf740 10 client.0 _ll_register_callbacks cb 0x145ce10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:43:29.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.623+0000 7f47d5fd0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1285840 0x15a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:29.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.623+0000 7f47d5fd0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1285840 0x15a1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49208/0 (socket says 172.21.15.78:49208) 2023-10-11T09:43:29.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.623+0000 7f47d5fd0700 1 -- 172.21.15.78:0/3172191331 learned_addr learned my addr 172.21.15.78:0/3172191331 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:29.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.623+0000 7f47d57cf700 1 --2- 172.21.15.78:0/3172191331 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1285c10 0x154c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:29.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.623+0000 7f47d67d1700 1 --2- 172.21.15.78:0/3172191331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x12d08c0 0x154c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:29.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.623+0000 7f47d5fd0700 1 -- 172.21.15.78:0/3172191331 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1285c10 msgr2=0x154c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:29.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.623+0000 7f47d5fd0700 1 --2- 172.21.15.78:0/3172191331 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1285c10 0x154c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:29.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.623+0000 7f47d5fd0700 1 -- 172.21.15.78:0/3172191331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x12d08c0 msgr2=0x154c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:29.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.623+0000 7f47d5fd0700 1 --2- 172.21.15.78:0/3172191331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x12d08c0 0x154c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:29.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.623+0000 7f47d5fd0700 1 -- 172.21.15.78:0/3172191331 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x12d1b10 con 0x1285840 2023-10-11T09:43:29.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.623+0000 7f47d67d1700 1 --2- 172.21.15.78:0/3172191331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x12d08c0 0x154c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:43:29.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.623+0000 7f47d5fd0700 1 --2- 172.21.15.78:0/3172191331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1285840 0x15a1410 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f47cc001dd0 tx=0x7f47cc084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894aee7907b33259 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:29.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.623+0000 7f47b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:43:29.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.623+0000 7f47b0ff9700 1 -- 172.21.15.78:0/3172191331 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc055790 con 0x1285840 2023-10-11T09:43:29.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.623+0000 7f47b0ff9700 1 -- 172.21.15.78:0/3172191331 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 6753+0+0 (secure 0 0 0) 0x7f47cc0595c0 con 0x1285840 2023-10-11T09:43:29.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.623+0000 7f47db4bf740 1 -- 172.21.15.78:0/3172191331 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f47c800f040 con 0x1285840 2023-10-11T09:43:29.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.623+0000 7f47b0ff9700 1 -- 172.21.15.78:0/3172191331 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47cc055560 con 0x1285840 2023-10-11T09:43:29.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.623+0000 7f47db4bf740 10 client.6039 fetch_fsmap learned FSMap version 29 2023-10-11T09:43:29.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.623+0000 7f47db4bf740 1 -- 172.21.15.78:0/3172191331 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x7f47c001c040 con 0x1285840 2023-10-11T09:43:29.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.623+0000 7f47b0ff9700 1 -- 172.21.15.78:0/3172191331 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c800f040 con 0x1285840 2023-10-11T09:43:29.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.623+0000 7f47b0ff9700 1 -- 172.21.15.78:0/3172191331 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f47cc054c80 con 0x1285840 2023-10-11T09:43:29.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.623+0000 7f47db4bf740 10 client.6039 fetch_fsmap finished waiting for FSMap version 29 2023-10-11T09:43:29.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.623+0000 7f47db4bf740 10 client.6039 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-10-11T09:43:29.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.623+0000 7f47db4bf740 20 client.6039 populate_metadata read hostname 'smithi078' 2023-10-11T09:43:29.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.623+0000 7f47db4bf740 1 --2- 172.21.15.78:0/3172191331 >> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] conn(0x151d460 0x151d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:29.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.623+0000 7f47db4bf740 4 client.6039 mds_command: new command op to 5922 tid=0[{"prefix": "scrub status"}] 2023-10-11T09:43:29.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.623+0000 7f47db4bf740 1 -- 172.21.15.78:0/3172191331 --> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x12dd610 con 0x151d460 2023-10-11T09:43:29.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.623+0000 7f47d57cf700 1 --2- 172.21.15.78:0/3172191331 >> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] conn(0x151d460 0x151d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:29.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.627+0000 7f47d57cf700 1 --2- 172.21.15.78:0/3172191331 >> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] conn(0x151d460 0x151d860 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7bf44b81d776291 server_cookie=b446830531707633 in_seq=0 out_seq=0 2023-10-11T09:43:29.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.627+0000 7f47b0ff9700 10 client.6039 ms_handle_connect on v2:172.21.15.78:6835/3137573764 2023-10-11T09:43:29.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.627+0000 7f47b0ff9700 1 -- 172.21.15.78:0/3172191331 <== mds.0 v2:172.21.15.78:6835/3137573764 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x12dd610 con 0x151d460 2023-10-11T09:43:29.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.627+0000 7f47b0ff9700 10 client.6039 handle_command_reply: tid=0 2023-10-11T09:43:29.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.627+0000 7f47db4bf740 1 client.6039 shutdown 2023-10-11T09:43:29.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.627+0000 7f47db4bf740 1 -- 172.21.15.78:0/3172191331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1285840 msgr2=0x15a1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:29.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.627+0000 7f47db4bf740 1 --2- 172.21.15.78:0/3172191331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1285840 0x15a1410 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f47cc001dd0 tx=0x7f47cc084040 comp rx=0 tx=0).stop 2023-10-11T09:43:29.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.627+0000 7f47db4bf740 1 -- 172.21.15.78:0/3172191331 shutdown_connections 2023-10-11T09:43:29.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.627+0000 7f47db4bf740 1 --2- 172.21.15.78:0/3172191331 >> [v2:172.21.15.78:6835/3137573764,v1:172.21.15.78:6837/3137573764] conn(0x151d460 0x151d860 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:29.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.627+0000 7f47db4bf740 1 --2- 172.21.15.78:0/3172191331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x12d08c0 0x154c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:29.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.627+0000 7f47db4bf740 1 --2- 172.21.15.78:0/3172191331 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1285c10 0x154c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:29.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.627+0000 7f47b0ff9700 0 client.6039 ms_handle_reset on v2:172.21.15.78:6835/3137573764 2023-10-11T09:43:29.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.627+0000 7f47db4bf740 1 --2- 172.21.15.78:0/3172191331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1285840 0x15a1410 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:29.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.627+0000 7f47db4bf740 1 -- 172.21.15.78:0/3172191331 >> 172.21.15.78:0/3172191331 conn(0x1285fe0 msgr2=0x15446b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:29.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.627+0000 7f47db4bf740 1 -- 172.21.15.78:0/3172191331 shutdown_connections 2023-10-11T09:43:29.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.627+0000 7f47db4bf740 1 -- 172.21.15.78:0/3172191331 wait complete. 2023-10-11T09:43:29.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.627+0000 7f47db4bf740 20 client.6039 trim_cache size 0 max 16384 2023-10-11T09:43:29.641 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:43:29.641 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2023-10-11T09:43:29.642 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2023-10-11T09:43:29.642 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:43:29.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.627+0000 7f47d6fd2700 1 -- 172.21.15.78:0/1471782634 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f47b4065f00 msgr2=0x7f47b40683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:29.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.627+0000 7f47d6fd2700 1 --2- 172.21.15.78:0/1471782634 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f47b4065f00 0x7f47b40683b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f47d00b7030 tx=0x7f47c0009040 comp rx=0 tx=0).stop 2023-10-11T09:43:29.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.627+0000 7f47d6fd2700 1 -- 172.21.15.78:0/1471782634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47bc003a20 msgr2=0x7f47d00b6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:29.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.627+0000 7f47d6fd2700 1 --2- 172.21.15.78:0/1471782634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47bc003a20 0x7f47d00b6360 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00c1d0 tx=0x7f47cc001120 comp rx=0 tx=0).stop 2023-10-11T09:43:29.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.627+0000 7f47d6fd2700 1 -- 172.21.15.78:0/1471782634 shutdown_connections 2023-10-11T09:43:29.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.627+0000 7f47d6fd2700 1 --2- 172.21.15.78:0/1471782634 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f47b4065f00 0x7f47b40683b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:29.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.627+0000 7f47d6fd2700 1 --2- 172.21.15.78:0/1471782634 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f47d009f240 0x7f47d009f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:29.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.627+0000 7f47d6fd2700 1 --2- 172.21.15.78:0/1471782634 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47d00b68a0 0x7f47d009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:29.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.627+0000 7f47d6fd2700 1 --2- 172.21.15.78:0/1471782634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47bc003a20 0x7f47d00b6360 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:29.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.627+0000 7f47d6fd2700 1 -- 172.21.15.78:0/1471782634 >> 172.21.15.78:0/1471782634 conn(0x7f47d009c260 msgr2=0x7f47d006b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:29.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.627+0000 7f47d6fd2700 1 -- 172.21.15.78:0/1471782634 shutdown_connections 2023-10-11T09:43:29.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:29.627+0000 7f47d6fd2700 1 -- 172.21.15.78:0/1471782634 wait complete. 2023-10-11T09:43:29.654 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-11T09:43:29.654 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/0 2023-10-11T09:43:29.655 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:43:29.655 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:43:29.656 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/0" 2023-10-11T09:43:29.656 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:43:29.656 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:43:29.657 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 6291456: 2023-10-11T09:43:29.657 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:43:29.657 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 6291456 2023-10-11T09:43:29.658 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:43:29.658 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:43:29.658 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:43:29.659 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:43:29.659 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:43:29.660 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:43:32.798 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/1 2023-10-11T09:43:32.799 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:43:32.799 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:43:32.800 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/1" 2023-10-11T09:43:32.800 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:43:32.801 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:43:32.801 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 6291456: 2023-10-11T09:43:32.801 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:43:32.802 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 6291456 2023-10-11T09:43:32.802 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:43:32.802 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:43:32.803 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:43:32.803 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:43:32.803 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:43:32.804 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:43:35.944 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/2 2023-10-11T09:43:35.944 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:43:35.945 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:43:35.945 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/2" 2023-10-11T09:43:35.946 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:43:35.946 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:43:35.946 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 6291456: 2023-10-11T09:43:35.947 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:43:35.947 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 6291456 2023-10-11T09:43:35.947 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:43:35.948 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:43:35.948 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:43:35.948 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:43:35.949 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:43:35.949 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:43:39.083 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/3 2023-10-11T09:43:39.084 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:43:39.084 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:43:39.084 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/3" 2023-10-11T09:43:39.085 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:43:39.085 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:43:39.085 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 6291456: 2023-10-11T09:43:39.086 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:43:39.086 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 6291456 2023-10-11T09:43:39.086 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:43:39.087 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:43:39.087 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:43:39.088 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:43:39.088 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:43:39.088 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:43:42.224 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/4 2023-10-11T09:43:42.224 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:43:42.225 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:43:42.225 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/4" 2023-10-11T09:43:42.225 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:43:42.226 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:43:42.226 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 6291456: 2023-10-11T09:43:42.226 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:43:42.227 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 6291456 2023-10-11T09:43:42.227 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:43:42.228 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:43:42.228 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:43:42.228 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:43:42.229 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:43:42.229 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:43:45.363 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-11T09:43:45.364 DEBUG:teuthology.orchestra.run.smithi078:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-11T09:43:45.450 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-11T09:43:45.495 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-11T09:43:45.496 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:43:45.496 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:43:45.516 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:43:45.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.726+0000 7ff322d92700 1 Processor -- start 2023-10-11T09:43:45.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.726+0000 7ff322d92700 1 -- start start 2023-10-11T09:43:45.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.730+0000 7ff322d92700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff31c0ce500 0x7ff31c16bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:45.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.730+0000 7ff322d92700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff31c06ffa0 con 0x7ff31c071040 2023-10-11T09:43:45.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.730+0000 7ff322d92700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff31c070280 con 0x7ff31c160d40 2023-10-11T09:43:45.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.730+0000 7ff322d92700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff31c070c20 con 0x7ff31c0ce500 2023-10-11T09:43:45.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.730+0000 7ff31bfff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7ff31c071040 0x7ff31c0cddf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:42134/0 (socket says 172.21.15.78:42134) 2023-10-11T09:43:45.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.730+0000 7ff31bfff700 1 -- 172.21.15.78:0/458777752 learned_addr learned my addr 172.21.15.78:0/458777752 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:43:45.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.730+0000 7ff32132f700 1 --2- 172.21.15.78:0/458777752 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff31c0ce500 0x7ff31c16bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:45.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.734+0000 7ff31b7fe700 1 -- 172.21.15.78:0/458777752 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1539129240 0 0) 0x7ff31c06ffa0 con 0x7ff31c071040 2023-10-11T09:43:45.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.734+0000 7ff32132f700 1 -- 172.21.15.78:0/458777752 >> v1:172.21.15.133:6789/0 conn(0x7ff31c160d40 legacy=0x7ff31c0cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:45.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.734+0000 7ff32132f700 1 -- 172.21.15.78:0/458777752 >> v1:172.21.15.78:6789/0 conn(0x7ff31c071040 legacy=0x7ff31c0cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:45.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.734+0000 7ff32132f700 1 -- 172.21.15.78:0/458777752 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff31c070560 con 0x7ff31c0ce500 2023-10-11T09:43:45.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.734+0000 7ff32132f700 1 --2- 172.21.15.78:0/458777752 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff31c0ce500 0x7ff31c16bb80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff314009dc0 tx=0x7ff31400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd575cfa8fd87307 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:45.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.734+0000 7ff31b7fe700 1 -- 172.21.15.78:0/458777752 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31400e070 con 0x7ff31c0ce500 2023-10-11T09:43:45.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.734+0000 7ff31b7fe700 1 -- 172.21.15.78:0/458777752 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31400bad0 con 0x7ff31c0ce500 2023-10-11T09:43:45.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.734+0000 7ff31b7fe700 1 -- 172.21.15.78:0/458777752 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31400be80 con 0x7ff31c0ce500 2023-10-11T09:43:45.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.734+0000 7ff322d92700 1 -- 172.21.15.78:0/458777752 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff31c0ce500 msgr2=0x7ff31c16bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:45.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.734+0000 7ff322d92700 1 --2- 172.21.15.78:0/458777752 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff31c0ce500 0x7ff31c16bb80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff314009dc0 tx=0x7ff31400b330 comp rx=0 tx=0).stop 2023-10-11T09:43:45.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.734+0000 7ff322d92700 1 -- 172.21.15.78:0/458777752 shutdown_connections 2023-10-11T09:43:45.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.734+0000 7ff322d92700 1 --2- 172.21.15.78:0/458777752 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff31c0ce500 0x7ff31c16bb80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:45.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.734+0000 7ff322d92700 1 -- 172.21.15.78:0/458777752 >> 172.21.15.78:0/458777752 conn(0x7ff31c06aca0 msgr2=0x7ff31c0b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:45.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.734+0000 7ff322d92700 1 -- 172.21.15.78:0/458777752 shutdown_connections 2023-10-11T09:43:45.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.734+0000 7ff322d92700 1 -- 172.21.15.78:0/458777752 wait complete. 2023-10-11T09:43:45.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.738+0000 7ff322d92700 1 Processor -- start 2023-10-11T09:43:45.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.738+0000 7ff322d92700 1 -- start start 2023-10-11T09:43:45.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.738+0000 7ff322d92700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff31c071040 0x7ff31c151e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:45.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.738+0000 7ff322d92700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff31c0ce500 0x7ff31c0c8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:45.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.742+0000 7ff322d92700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff31c160d40 0x7ff31c0c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:45.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.742+0000 7ff322d92700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31c152fc0 con 0x7ff31c071040 2023-10-11T09:43:45.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.742+0000 7ff322d92700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31c1529d0 con 0x7ff31c0ce500 2023-10-11T09:43:45.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.742+0000 7ff322d92700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff31c155840 con 0x7ff31c160d40 2023-10-11T09:43:45.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.742+0000 7ff320b2e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff31c071040 0x7ff31c151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:45.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.742+0000 7ff320b2e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff31c071040 0x7ff31c151e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50088/0 (socket says 172.21.15.78:50088) 2023-10-11T09:43:45.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.742+0000 7ff320b2e700 1 -- 172.21.15.78:0/2776704649 learned_addr learned my addr 172.21.15.78:0/2776704649 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:45.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.742+0000 7ff32132f700 1 --2- 172.21.15.78:0/2776704649 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff31c160d40 0x7ff31c0c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:45.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.742+0000 7ff31bfff700 1 --2- 172.21.15.78:0/2776704649 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff31c0ce500 0x7ff31c0c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:45.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.742+0000 7ff320b2e700 1 -- 172.21.15.78:0/2776704649 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff31c160d40 msgr2=0x7ff31c0c88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:45.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.742+0000 7ff320b2e700 1 --2- 172.21.15.78:0/2776704649 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff31c160d40 0x7ff31c0c88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:45.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.742+0000 7ff320b2e700 1 -- 172.21.15.78:0/2776704649 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff31c0ce500 msgr2=0x7ff31c0c8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:45.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.742+0000 7ff320b2e700 1 --2- 172.21.15.78:0/2776704649 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff31c0ce500 0x7ff31c0c8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:45.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.742+0000 7ff320b2e700 1 -- 172.21.15.78:0/2776704649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff314009a70 con 0x7ff31c071040 2023-10-11T09:43:45.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.742+0000 7ff31bfff700 1 --2- 172.21.15.78:0/2776704649 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff31c0ce500 0x7ff31c0c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:43:45.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.742+0000 7ff320b2e700 1 --2- 172.21.15.78:0/2776704649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff31c071040 0x7ff31c151e50 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff31000c780 tx=0x7ff31000ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:45.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.742+0000 7ff3197fa700 1 -- 172.21.15.78:0/2776704649 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3100054e0 con 0x7ff31c071040 2023-10-11T09:43:45.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.742+0000 7ff322d92700 1 -- 172.21.15.78:0/2776704649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff31c0c5220 con 0x7ff31c071040 2023-10-11T09:43:45.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.742+0000 7ff3197fa700 1 -- 172.21.15.78:0/2776704649 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3100036a0 con 0x7ff31c071040 2023-10-11T09:43:45.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.742+0000 7ff3197fa700 1 -- 172.21.15.78:0/2776704649 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff310003a50 con 0x7ff31c071040 2023-10-11T09:43:45.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.742+0000 7ff322d92700 1 -- 172.21.15.78:0/2776704649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff31c152be0 con 0x7ff31c071040 2023-10-11T09:43:45.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.742+0000 7ff322d92700 1 -- 172.21.15.78:0/2776704649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff31c072250 con 0x7ff31c071040 2023-10-11T09:43:45.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.746+0000 7ff3197fa700 1 -- 172.21.15.78:0/2776704649 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7ff310003bb0 con 0x7ff31c071040 2023-10-11T09:43:45.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.746+0000 7ff3197fa700 1 --2- 172.21.15.78:0/2776704649 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff300065ed0 0x7ff300068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:45.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.746+0000 7ff3197fa700 1 -- 172.21.15.78:0/2776704649 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6753+0+0 (secure 0 0 0) 0x7ff310082c70 con 0x7ff31c071040 2023-10-11T09:43:45.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.746+0000 7ff31bfff700 1 --2- 172.21.15.78:0/2776704649 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff300065ed0 0x7ff300068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:45.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.750+0000 7ff31bfff700 1 --2- 172.21.15.78:0/2776704649 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff300065ed0 0x7ff300068380 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff31c1554a0 tx=0x7ff308007e70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:45.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.754+0000 7ff3197fa700 1 -- 172.21.15.78:0/2776704649 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7ff310050010 con 0x7ff31c071040 2023-10-11T09:43:45.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.926+0000 7ff322d92700 1 -- 172.21.15.78:0/2776704649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff31c04e530 con 0x7ff31c071040 2023-10-11T09:43:45.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.930+0000 7ff3197fa700 1 -- 172.21.15.78:0/2776704649 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+20115 (secure 0 0 0) 0x7ff310003e60 con 0x7ff31c071040 2023-10-11T09:43:45.937 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:43:45.938 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":48,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:42:51.686927+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:41:57.280079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-11T09:41:58.218860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data_extra","create_time":"2023-10-11T09:42:12.954795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.10","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6833/1601241351":"2023-10-12T09:42:51.686883+0000","172.21.15.133:6832/1601241351":"2023-10-12T09:42:51.686883+0000","172.21.15.133:6833/4103940204":"2023-10-12T09:42:45.599736+0000","172.21.15.133:6832/4103940204":"2023-10-12T09:42:45.599736+0000","172.21.15.133:6833/1862542010":"2023-10-12T09:42:44.563306+0000","172.21.15.133:6832/1862542010":"2023-10-12T09:42:44.563306+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:43:45.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.934+0000 7ff322d92700 1 -- 172.21.15.78:0/2776704649 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff300065ed0 msgr2=0x7ff300068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:45.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.934+0000 7ff322d92700 1 --2- 172.21.15.78:0/2776704649 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff300065ed0 0x7ff300068380 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff31c1554a0 tx=0x7ff308007e70 comp rx=0 tx=0).stop 2023-10-11T09:43:45.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.934+0000 7ff322d92700 1 -- 172.21.15.78:0/2776704649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff31c071040 msgr2=0x7ff31c151e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:45.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.934+0000 7ff322d92700 1 --2- 172.21.15.78:0/2776704649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff31c071040 0x7ff31c151e50 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff31000c780 tx=0x7ff31000ca90 comp rx=0 tx=0).stop 2023-10-11T09:43:45.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.938+0000 7ff322d92700 1 -- 172.21.15.78:0/2776704649 shutdown_connections 2023-10-11T09:43:45.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.938+0000 7ff322d92700 1 --2- 172.21.15.78:0/2776704649 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff300065ed0 0x7ff300068380 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:45.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.938+0000 7ff322d92700 1 --2- 172.21.15.78:0/2776704649 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff31c160d40 0x7ff31c0c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:45.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.938+0000 7ff322d92700 1 --2- 172.21.15.78:0/2776704649 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff31c0ce500 0x7ff31c0c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:45.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.938+0000 7ff322d92700 1 --2- 172.21.15.78:0/2776704649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff31c071040 0x7ff31c151e50 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:45.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.938+0000 7ff322d92700 1 -- 172.21.15.78:0/2776704649 >> 172.21.15.78:0/2776704649 conn(0x7ff31c06aca0 msgr2=0x7ff31c0cd280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:45.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.938+0000 7ff322d92700 1 -- 172.21.15.78:0/2776704649 shutdown_connections 2023-10-11T09:43:45.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:45.938+0000 7ff322d92700 1 -- 172.21.15.78:0/2776704649 wait complete. 2023-10-11T09:43:45.965 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T09:41:57.280079+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T09:41:58.218860+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-10-11T09:42:12.954795+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:43:45.966 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:43:46.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.142+0000 7f2c51239700 1 Processor -- start 2023-10-11T09:43:46.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.142+0000 7f2c51239700 1 -- start start 2023-10-11T09:43:46.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.146+0000 7f2c51239700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f2c4c0b3240 0x7f2c4c0d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:46.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.146+0000 7f2c51239700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c4c0b47f0 con 0x7f2c4c0b3610 2023-10-11T09:43:46.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.146+0000 7f2c51239700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c4c0b8130 con 0x7f2c4c0b3a90 2023-10-11T09:43:46.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.146+0000 7f2c51239700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c4c0b41e0 con 0x7f2c4c0b3240 2023-10-11T09:43:46.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.146+0000 7f2c4bfff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f2c4c0b3240 0x7f2c4c0d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:46.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.146+0000 7f2c4bfff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f2c4c0b3240 0x7f2c4c0d29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50094/0 (socket says 172.21.15.78:50094) 2023-10-11T09:43:46.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.146+0000 7f2c4bfff700 1 -- 172.21.15.78:0/1291191558 learned_addr learned my addr 172.21.15.78:0/1291191558 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:46.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.146+0000 7f2c4b7fe700 1 --1- 172.21.15.78:0/1291191558 >> v1:172.21.15.78:6789/0 conn(0x7f2c4c0b3610 0x7f2c4c15d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:42140/0 (socket says 172.21.15.78:42140) 2023-10-11T09:43:46.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.146+0000 7f2c4bfff700 1 -- 172.21.15.78:0/1291191558 >> v1:172.21.15.133:6789/0 conn(0x7f2c4c0b3a90 legacy=0x7f2c4c0b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:46.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.146+0000 7f2c4affd700 1 -- 172.21.15.78:0/1291191558 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2609196599 0 0) 0x7f2c4c0b8130 con 0x7f2c4c0b3a90 2023-10-11T09:43:46.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.146+0000 7f2c4bfff700 1 -- 172.21.15.78:0/1291191558 >> v1:172.21.15.78:6789/0 conn(0x7f2c4c0b3610 legacy=0x7f2c4c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:46.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.146+0000 7f2c4bfff700 1 -- 172.21.15.78:0/1291191558 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c4c0b7800 con 0x7f2c4c0b3240 2023-10-11T09:43:46.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.146+0000 7f2c4bfff700 1 --2- 172.21.15.78:0/1291191558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c4c0b3240 0x7f2c4c0d29d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c009910 tx=0x7f2c3c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=183df5deb6ba89fb server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:46.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.146+0000 7f2c4affd700 1 -- 172.21.15.78:0/1291191558 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c00ba60 con 0x7f2c4c0b3240 2023-10-11T09:43:46.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.146+0000 7f2c4affd700 1 -- 172.21.15.78:0/1291191558 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c3c00bbc0 con 0x7f2c4c0b3240 2023-10-11T09:43:46.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.146+0000 7f2c4affd700 1 -- 172.21.15.78:0/1291191558 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c009e60 con 0x7f2c4c0b3240 2023-10-11T09:43:46.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.150+0000 7f2c51239700 1 -- 172.21.15.78:0/1291191558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c4c0b3240 msgr2=0x7f2c4c0d29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:46.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.150+0000 7f2c51239700 1 --2- 172.21.15.78:0/1291191558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c4c0b3240 0x7f2c4c0d29d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c009910 tx=0x7f2c3c009c20 comp rx=0 tx=0).stop 2023-10-11T09:43:46.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.150+0000 7f2c51239700 1 -- 172.21.15.78:0/1291191558 shutdown_connections 2023-10-11T09:43:46.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.150+0000 7f2c51239700 1 --2- 172.21.15.78:0/1291191558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c4c0b3240 0x7f2c4c0d29d0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:46.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.150+0000 7f2c51239700 1 -- 172.21.15.78:0/1291191558 >> 172.21.15.78:0/1291191558 conn(0x7f2c4c09c260 msgr2=0x7f2c4c0d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:46.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.150+0000 7f2c51239700 1 -- 172.21.15.78:0/1291191558 shutdown_connections 2023-10-11T09:43:46.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.150+0000 7f2c51239700 1 -- 172.21.15.78:0/1291191558 wait complete. 2023-10-11T09:43:46.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.150+0000 7f2c51239700 1 Processor -- start 2023-10-11T09:43:46.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.154+0000 7f2c51239700 1 -- start start 2023-10-11T09:43:46.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.154+0000 7f2c51239700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2c4c0b3240 0x7f2c4c0c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:46.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.154+0000 7f2c51239700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c4c0b3610 0x7f2c4c0c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:46.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.154+0000 7f2c51239700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c4c0b3a90 0x7f2c4c06bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:46.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.154+0000 7f2c51239700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c15de10 con 0x7f2c4c0b3610 2023-10-11T09:43:46.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.154+0000 7f2c51239700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c15ce00 con 0x7f2c4c0b3a90 2023-10-11T09:43:46.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.154+0000 7f2c51239700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c15d050 con 0x7f2c4c0b3240 2023-10-11T09:43:46.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.154+0000 7f2c4b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c4c0b3610 0x7f2c4c0c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:46.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.154+0000 7f2c4b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c4c0b3610 0x7f2c4c0c95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50102/0 (socket says 172.21.15.78:50102) 2023-10-11T09:43:46.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.154+0000 7f2c4b7fe700 1 -- 172.21.15.78:0/1426927731 learned_addr learned my addr 172.21.15.78:0/1426927731 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:46.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.154+0000 7f2c4bfff700 1 --2- 172.21.15.78:0/1426927731 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2c4c0b3240 0x7f2c4c0c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:46.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.154+0000 7f2c50a38700 1 --2- 172.21.15.78:0/1426927731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c4c0b3a90 0x7f2c4c06bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:46.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.154+0000 7f2c4b7fe700 1 -- 172.21.15.78:0/1426927731 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2c4c0b3240 msgr2=0x7f2c4c0c7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:46.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.154+0000 7f2c4b7fe700 1 --2- 172.21.15.78:0/1426927731 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2c4c0b3240 0x7f2c4c0c7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:46.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.154+0000 7f2c4b7fe700 1 -- 172.21.15.78:0/1426927731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c4c0b3a90 msgr2=0x7f2c4c06bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:46.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.154+0000 7f2c4b7fe700 1 --2- 172.21.15.78:0/1426927731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c4c0b3a90 0x7f2c4c06bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:46.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.154+0000 7f2c4b7fe700 1 -- 172.21.15.78:0/1426927731 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c3c009570 con 0x7f2c4c0b3610 2023-10-11T09:43:46.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.154+0000 7f2c4b7fe700 1 --2- 172.21.15.78:0/1426927731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c4c0b3610 0x7f2c4c0c95a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f2c4000c4d0 tx=0x7f2c4000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:46.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.154+0000 7f2c48ff9700 1 -- 172.21.15.78:0/1426927731 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c40005cd0 con 0x7f2c4c0b3610 2023-10-11T09:43:46.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.154+0000 7f2c48ff9700 1 -- 172.21.15.78:0/1426927731 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c40005e30 con 0x7f2c4c0b3610 2023-10-11T09:43:46.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.154+0000 7f2c48ff9700 1 -- 172.21.15.78:0/1426927731 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c400072d0 con 0x7f2c4c0b3610 2023-10-11T09:43:46.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.154+0000 7f2c51239700 1 -- 172.21.15.78:0/1426927731 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c4c15d330 con 0x7f2c4c0b3610 2023-10-11T09:43:46.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.158+0000 7f2c51239700 1 -- 172.21.15.78:0/1426927731 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c4c15d8f0 con 0x7f2c4c0b3610 2023-10-11T09:43:46.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.158+0000 7f2c51239700 1 -- 172.21.15.78:0/1426927731 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c4c15db10 con 0x7f2c4c0b3610 2023-10-11T09:43:46.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.158+0000 7f2c48ff9700 1 -- 172.21.15.78:0/1426927731 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f2c40007430 con 0x7f2c4c0b3610 2023-10-11T09:43:46.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.162+0000 7f2c48ff9700 1 --2- 172.21.15.78:0/1426927731 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2c38065ed0 0x7f2c38068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:46.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.162+0000 7f2c48ff9700 1 -- 172.21.15.78:0/1426927731 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6753+0+0 (secure 0 0 0) 0x7f2c40082c30 con 0x7f2c4c0b3610 2023-10-11T09:43:46.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.162+0000 7f2c4bfff700 1 --2- 172.21.15.78:0/1426927731 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2c38065ed0 0x7f2c38068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:46.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.162+0000 7f2c4bfff700 1 --2- 172.21.15.78:0/1426927731 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2c38065ed0 0x7f2c38068380 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c014040 tx=0x7f2c3c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:46.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.166+0000 7f2c48ff9700 1 -- 172.21.15.78:0/1426927731 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f2c40050080 con 0x7f2c4c0b3610 2023-10-11T09:43:46.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.338+0000 7f2c51239700 1 -- 172.21.15.78:0/1426927731 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2c4c065d40 con 0x7f2c4c0b3610 2023-10-11T09:43:46.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.342+0000 7f2c48ff9700 1 -- 172.21.15.78:0/1426927731 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2c400170a0 con 0x7f2c4c0b3610 2023-10-11T09:43:46.346 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:43:46.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.342+0000 7f2c51239700 1 -- 172.21.15.78:0/1426927731 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2c38065ed0 msgr2=0x7f2c38068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:46.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.342+0000 7f2c51239700 1 --2- 172.21.15.78:0/1426927731 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2c38065ed0 0x7f2c38068380 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c014040 tx=0x7f2c3c008030 comp rx=0 tx=0).stop 2023-10-11T09:43:46.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.342+0000 7f2c51239700 1 -- 172.21.15.78:0/1426927731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c4c0b3610 msgr2=0x7f2c4c0c95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:46.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.342+0000 7f2c51239700 1 --2- 172.21.15.78:0/1426927731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c4c0b3610 0x7f2c4c0c95a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f2c4000c4d0 tx=0x7f2c4000c890 comp rx=0 tx=0).stop 2023-10-11T09:43:46.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.342+0000 7f2c51239700 1 -- 172.21.15.78:0/1426927731 shutdown_connections 2023-10-11T09:43:46.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.342+0000 7f2c51239700 1 --2- 172.21.15.78:0/1426927731 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2c38065ed0 0x7f2c38068380 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:46.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.342+0000 7f2c51239700 1 --2- 172.21.15.78:0/1426927731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c4c0b3a90 0x7f2c4c06bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:46.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.342+0000 7f2c51239700 1 --2- 172.21.15.78:0/1426927731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c4c0b3610 0x7f2c4c0c95a0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:46.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.342+0000 7f2c51239700 1 --2- 172.21.15.78:0/1426927731 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2c4c0b3240 0x7f2c4c0c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:46.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.342+0000 7f2c51239700 1 -- 172.21.15.78:0/1426927731 >> 172.21.15.78:0/1426927731 conn(0x7f2c4c09c260 msgr2=0x7f2c4c0afb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:46.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.342+0000 7f2c51239700 1 -- 172.21.15.78:0/1426927731 shutdown_connections 2023-10-11T09:43:46.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.342+0000 7f2c51239700 1 -- 172.21.15.78:0/1426927731 wait complete. 2023-10-11T09:43:46.373 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:43:46.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.546+0000 7f1861e34700 1 Processor -- start 2023-10-11T09:43:46.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.546+0000 7f1861e34700 1 -- start start 2023-10-11T09:43:46.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.550+0000 7f1861e34700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f185c160cb0 0x7f185c0cb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:46.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.550+0000 7f1861e34700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f185c06d080 0x7f185c0b0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:46.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.550+0000 7f1861e34700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f185c0b0e60 0x7f185c06be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:46.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.550+0000 7f1861e34700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f185c161080 con 0x7f185c160cb0 2023-10-11T09:43:46.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.550+0000 7f1861e34700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f185c164750 con 0x7f185c06d080 2023-10-11T09:43:46.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.550+0000 7f1861e34700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f185c0b12e0 con 0x7f185c0b0e60 2023-10-11T09:43:46.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.550+0000 7f185b7fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f185c160cb0 0x7f185c0cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:46.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.550+0000 7f185b7fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f185c160cb0 0x7f185c0cb100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50106/0 (socket says 172.21.15.78:50106) 2023-10-11T09:43:46.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.550+0000 7f185b7fe700 1 -- 172.21.15.78:0/4152039775 learned_addr learned my addr 172.21.15.78:0/4152039775 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:46.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.550+0000 7f185bfff700 1 --2- 172.21.15.78:0/4152039775 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f185c0b0e60 0x7f185c06be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:46.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.550+0000 7f185affd700 1 --2- 172.21.15.78:0/4152039775 >> v2:172.21.15.133:3300/0 conn(0x7f185c06d080 0x7f185c0b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:46.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.550+0000 7f185b7fe700 1 -- 172.21.15.78:0/4152039775 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f185c0b0e60 msgr2=0x7f185c06be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:46.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.550+0000 7f185b7fe700 1 --2- 172.21.15.78:0/4152039775 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f185c0b0e60 0x7f185c06be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:46.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.550+0000 7f185b7fe700 1 -- 172.21.15.78:0/4152039775 >> v2:172.21.15.133:3300/0 conn(0x7f185c06d080 msgr2=0x7f185c0b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:46.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.550+0000 7f185b7fe700 1 --2- 172.21.15.78:0/4152039775 >> v2:172.21.15.133:3300/0 conn(0x7f185c06d080 0x7f185c0b0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:46.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.550+0000 7f185b7fe700 1 -- 172.21.15.78:0/4152039775 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f185c165570 con 0x7f185c160cb0 2023-10-11T09:43:46.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.550+0000 7f185bfff700 1 --2- 172.21.15.78:0/4152039775 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f185c0b0e60 0x7f185c06be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:43:46.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.550+0000 7f185b7fe700 1 --2- 172.21.15.78:0/4152039775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f185c160cb0 0x7f185c0cb100 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f185c0c9ac0 tx=0x7f184c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=199f83168830c3f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:46.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.550+0000 7f185a7fc700 1 -- 172.21.15.78:0/4152039775 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184c011830 con 0x7f185c160cb0 2023-10-11T09:43:46.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.550+0000 7f185a7fc700 1 -- 172.21.15.78:0/4152039775 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f184c011990 con 0x7f185c160cb0 2023-10-11T09:43:46.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.550+0000 7f185a7fc700 1 -- 172.21.15.78:0/4152039775 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184c011d20 con 0x7f185c160cb0 2023-10-11T09:43:46.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.554+0000 7f1861e34700 1 -- 172.21.15.78:0/4152039775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f185c160cb0 msgr2=0x7f185c0cb100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:46.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.554+0000 7f1861e34700 1 --2- 172.21.15.78:0/4152039775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f185c160cb0 0x7f185c0cb100 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f185c0c9ac0 tx=0x7f184c009bf0 comp rx=0 tx=0).stop 2023-10-11T09:43:46.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.554+0000 7f1861e34700 1 -- 172.21.15.78:0/4152039775 shutdown_connections 2023-10-11T09:43:46.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.554+0000 7f1861e34700 1 --2- 172.21.15.78:0/4152039775 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f185c0b0e60 0x7f185c06be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:46.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.554+0000 7f1861e34700 1 --2- 172.21.15.78:0/4152039775 >> v2:172.21.15.133:3300/0 conn(0x7f185c06d080 0x7f185c0b0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:46.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.554+0000 7f1861e34700 1 --2- 172.21.15.78:0/4152039775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f185c160cb0 0x7f185c0cb100 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:46.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.554+0000 7f1861e34700 1 -- 172.21.15.78:0/4152039775 >> 172.21.15.78:0/4152039775 conn(0x7f185c0cecc0 msgr2=0x7f185c0cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:46.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.554+0000 7f1861e34700 1 -- 172.21.15.78:0/4152039775 shutdown_connections 2023-10-11T09:43:46.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.554+0000 7f1861e34700 1 -- 172.21.15.78:0/4152039775 wait complete. 2023-10-11T09:43:46.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.554+0000 7f1861e34700 1 Processor -- start 2023-10-11T09:43:46.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.554+0000 7f1861e34700 1 -- start start 2023-10-11T09:43:46.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.558+0000 7f1861e34700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f185c06d080 0x7f185c147be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:46.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.558+0000 7f1861e34700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f185c0b0e60 0x7f185c148120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:46.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.558+0000 7f1861e34700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f185c160cb0 0x7f185c1499d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:46.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.558+0000 7f1861e34700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185c148950 con 0x7f185c0b0e60 2023-10-11T09:43:46.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.558+0000 7f1861e34700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185c148c30 con 0x7f185c06d080 2023-10-11T09:43:46.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.558+0000 7f1861e34700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f185c148f10 con 0x7f185c160cb0 2023-10-11T09:43:46.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.558+0000 7f185affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f185c0b0e60 0x7f185c148120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:46.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.558+0000 7f185affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f185c0b0e60 0x7f185c148120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50108/0 (socket says 172.21.15.78:50108) 2023-10-11T09:43:46.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.558+0000 7f185affd700 1 -- 172.21.15.78:0/1430720205 learned_addr learned my addr 172.21.15.78:0/1430720205 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:46.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.558+0000 7f185b7fe700 1 --2- 172.21.15.78:0/1430720205 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f185c06d080 0x7f185c147be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:46.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.558+0000 7f185bfff700 1 --2- 172.21.15.78:0/1430720205 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f185c160cb0 0x7f185c1499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:46.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.558+0000 7f185affd700 1 -- 172.21.15.78:0/1430720205 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f185c160cb0 msgr2=0x7f185c1499d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:46.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.558+0000 7f185affd700 1 --2- 172.21.15.78:0/1430720205 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f185c160cb0 0x7f185c1499d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:46.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.558+0000 7f185affd700 1 -- 172.21.15.78:0/1430720205 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f185c06d080 msgr2=0x7f185c147be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:46.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.558+0000 7f185affd700 1 --2- 172.21.15.78:0/1430720205 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f185c06d080 0x7f185c147be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:46.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.558+0000 7f185affd700 1 -- 172.21.15.78:0/1430720205 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f184c009570 con 0x7f185c0b0e60 2023-10-11T09:43:46.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.558+0000 7f185affd700 1 --2- 172.21.15.78:0/1430720205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f185c0b0e60 0x7f185c148120 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f185000e7c0 tx=0x7f185000ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:46.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.558+0000 7f1843fff700 1 -- 172.21.15.78:0/1430720205 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1850013680 con 0x7f185c0b0e60 2023-10-11T09:43:46.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.558+0000 7f1843fff700 1 -- 172.21.15.78:0/1430720205 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18500137e0 con 0x7f185c0b0e60 2023-10-11T09:43:46.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.558+0000 7f1843fff700 1 -- 172.21.15.78:0/1430720205 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1850013b90 con 0x7f185c0b0e60 2023-10-11T09:43:46.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.558+0000 7f1861e34700 1 -- 172.21.15.78:0/1430720205 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f185c1491f0 con 0x7f185c0b0e60 2023-10-11T09:43:46.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.558+0000 7f1861e34700 1 -- 172.21.15.78:0/1430720205 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f185c148660 con 0x7f185c0b0e60 2023-10-11T09:43:46.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.562+0000 7f1861e34700 1 -- 172.21.15.78:0/1430720205 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f185c06eb80 con 0x7f185c0b0e60 2023-10-11T09:43:46.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.562+0000 7f1843fff700 1 -- 172.21.15.78:0/1430720205 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f1850013cf0 con 0x7f185c0b0e60 2023-10-11T09:43:46.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.566+0000 7f1843fff700 1 --2- 172.21.15.78:0/1430720205 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1848065ed0 0x7f1848068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:46.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.566+0000 7f1843fff700 1 -- 172.21.15.78:0/1430720205 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6753+0+0 (secure 0 0 0) 0x7f1850083a90 con 0x7f185c0b0e60 2023-10-11T09:43:46.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.566+0000 7f185b7fe700 1 --2- 172.21.15.78:0/1430720205 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1848065ed0 0x7f1848068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:46.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.566+0000 7f185b7fe700 1 --2- 172.21.15.78:0/1430720205 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1848065ed0 0x7f1848068380 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f184c000c00 tx=0x7f184c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:46.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.570+0000 7f1843fff700 1 -- 172.21.15.78:0/1430720205 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f1850050d00 con 0x7f185c0b0e60 2023-10-11T09:43:46.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.742+0000 7f1861e34700 1 -- 172.21.15.78:0/1430720205 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f185c065db0 con 0x7f185c0b0e60 2023-10-11T09:43:46.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.746+0000 7f1843fff700 1 -- 172.21.15.78:0/1430720205 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v48) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1850054320 con 0x7f185c0b0e60 2023-10-11T09:43:46.750 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:43:46.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.746+0000 7f1861e34700 1 -- 172.21.15.78:0/1430720205 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1848065ed0 msgr2=0x7f1848068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:46.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.746+0000 7f1861e34700 1 --2- 172.21.15.78:0/1430720205 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1848065ed0 0x7f1848068380 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f184c000c00 tx=0x7f184c008030 comp rx=0 tx=0).stop 2023-10-11T09:43:46.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.746+0000 7f1861e34700 1 -- 172.21.15.78:0/1430720205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f185c0b0e60 msgr2=0x7f185c148120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:46.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.746+0000 7f1861e34700 1 --2- 172.21.15.78:0/1430720205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f185c0b0e60 0x7f185c148120 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f185000e7c0 tx=0x7f185000ead0 comp rx=0 tx=0).stop 2023-10-11T09:43:46.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.746+0000 7f1861e34700 1 -- 172.21.15.78:0/1430720205 shutdown_connections 2023-10-11T09:43:46.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.746+0000 7f1861e34700 1 --2- 172.21.15.78:0/1430720205 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1848065ed0 0x7f1848068380 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:46.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.746+0000 7f1861e34700 1 --2- 172.21.15.78:0/1430720205 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f185c160cb0 0x7f185c1499d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:46.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.746+0000 7f1861e34700 1 --2- 172.21.15.78:0/1430720205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f185c0b0e60 0x7f185c148120 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:46.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.746+0000 7f1861e34700 1 --2- 172.21.15.78:0/1430720205 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f185c06d080 0x7f185c147be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:46.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.746+0000 7f1861e34700 1 -- 172.21.15.78:0/1430720205 >> 172.21.15.78:0/1430720205 conn(0x7f185c0cecc0 msgr2=0x7f185c0cc800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:46.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.746+0000 7f1861e34700 1 -- 172.21.15.78:0/1430720205 shutdown_connections 2023-10-11T09:43:46.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.746+0000 7f1861e34700 1 -- 172.21.15.78:0/1430720205 wait complete. 2023-10-11T09:43:46.773 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T09:43:46.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.946+0000 7f5936158700 1 Processor -- start 2023-10-11T09:43:46.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.946+0000 7f5936158700 1 -- start start 2023-10-11T09:43:46.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.946+0000 7f5936158700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f593015a6b0 0x7f59300ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:46.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.946+0000 7f5936158700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59300b2d30 con 0x7f593006d730 2023-10-11T09:43:46.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.946+0000 7f5936158700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59300cf360 con 0x7f59300b3a60 2023-10-11T09:43:46.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.946+0000 7f5936158700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59300b2f80 con 0x7f593015a6b0 2023-10-11T09:43:46.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.946+0000 7f5935957700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f593006d730 0x7f593006bbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:42144/0 (socket says 172.21.15.78:42144) 2023-10-11T09:43:46.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.946+0000 7f5935957700 1 -- 172.21.15.78:0/1562073701 learned_addr learned my addr 172.21.15.78:0/1562073701 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:43:46.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.946+0000 7f5935156700 1 --2- 172.21.15.78:0/1562073701 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f593015a6b0 0x7f59300ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:46.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.950+0000 7f5927fff700 1 -- 172.21.15.78:0/1562073701 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1646898026 0 0) 0x7f59300b2d30 con 0x7f593006d730 2023-10-11T09:43:46.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.950+0000 7f5927fff700 1 -- 172.21.15.78:0/1562073701 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f591c0035d0 con 0x7f593006d730 2023-10-11T09:43:46.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.950+0000 7f5927fff700 1 -- 172.21.15.78:0/1562073701 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3576395947 0 0) 0x7f59300cf360 con 0x7f59300b3a60 2023-10-11T09:43:46.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.950+0000 7f5927fff700 1 -- 172.21.15.78:0/1562073701 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59300b2d30 con 0x7f59300b3a60 2023-10-11T09:43:46.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.950+0000 7f5927fff700 1 -- 172.21.15.78:0/1562073701 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3483002767 0 0) 0x7f591c0035d0 con 0x7f593006d730 2023-10-11T09:43:46.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.950+0000 7f5935156700 1 -- 172.21.15.78:0/1562073701 >> v1:172.21.15.133:6789/0 conn(0x7f59300b3a60 legacy=0x7f593006d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:46.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.950+0000 7f5935156700 1 -- 172.21.15.78:0/1562073701 >> v1:172.21.15.78:6789/0 conn(0x7f593006d730 legacy=0x7f593006bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:46.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.950+0000 7f5935156700 1 -- 172.21.15.78:0/1562073701 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59300b3260 con 0x7f593015a6b0 2023-10-11T09:43:46.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.950+0000 7f5935156700 1 --2- 172.21.15.78:0/1562073701 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f593015a6b0 0x7f59300ce8a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f592c009880 tx=0x7f592c009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f24157c276cfb05a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:46.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.950+0000 7f5927fff700 1 -- 172.21.15.78:0/1562073701 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59280019f0 con 0x7f593015a6b0 2023-10-11T09:43:46.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.950+0000 7f5927fff700 1 -- 172.21.15.78:0/1562073701 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f592c004030 con 0x7f593015a6b0 2023-10-11T09:43:46.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.950+0000 7f5927fff700 1 -- 172.21.15.78:0/1562073701 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592c0043a0 con 0x7f593015a6b0 2023-10-11T09:43:46.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.950+0000 7f5936158700 1 -- 172.21.15.78:0/1562073701 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f593015a6b0 msgr2=0x7f59300ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:46.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.950+0000 7f5936158700 1 --2- 172.21.15.78:0/1562073701 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f593015a6b0 0x7f59300ce8a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f592c009880 tx=0x7f592c009b90 comp rx=0 tx=0).stop 2023-10-11T09:43:46.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.950+0000 7f5936158700 1 -- 172.21.15.78:0/1562073701 shutdown_connections 2023-10-11T09:43:46.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.950+0000 7f5936158700 1 --2- 172.21.15.78:0/1562073701 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f593015a6b0 0x7f59300ce8a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:46.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.950+0000 7f5936158700 1 -- 172.21.15.78:0/1562073701 >> 172.21.15.78:0/1562073701 conn(0x7f593006a060 msgr2=0x7f593006a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:46.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.950+0000 7f5936158700 1 -- 172.21.15.78:0/1562073701 shutdown_connections 2023-10-11T09:43:46.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.950+0000 7f5936158700 1 -- 172.21.15.78:0/1562073701 wait complete. 2023-10-11T09:43:46.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.954+0000 7f5936158700 1 Processor -- start 2023-10-11T09:43:46.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.954+0000 7f5936158700 1 -- start start 2023-10-11T09:43:46.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.958+0000 7f5936158700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f593006d730 0x7f59301446a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:46.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.958+0000 7f5936158700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59300b3a60 0x7f59300b0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:46.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.958+0000 7f5936158700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f593015a6b0 0x7f59300b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:46.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.958+0000 7f5936158700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f593015e7c0 con 0x7f59300b3a60 2023-10-11T09:43:46.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.958+0000 7f5936158700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5930157130 con 0x7f593015a6b0 2023-10-11T09:43:46.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.958+0000 7f5936158700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5930157410 con 0x7f593006d730 2023-10-11T09:43:46.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.958+0000 7f5934955700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59300b3a60 0x7f59300b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:46.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.958+0000 7f5934955700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59300b3a60 0x7f59300b0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50112/0 (socket says 172.21.15.78:50112) 2023-10-11T09:43:46.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.958+0000 7f5934955700 1 -- 172.21.15.78:0/3966951501 learned_addr learned my addr 172.21.15.78:0/3966951501 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:46.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.958+0000 7f5935156700 1 --2- 172.21.15.78:0/3966951501 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f593006d730 0x7f59301446a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:46.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.958+0000 7f5935957700 1 --2- 172.21.15.78:0/3966951501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f593015a6b0 0x7f59300b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:46.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.958+0000 7f5934955700 1 -- 172.21.15.78:0/3966951501 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f593006d730 msgr2=0x7f59301446a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:46.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.958+0000 7f5934955700 1 --2- 172.21.15.78:0/3966951501 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f593006d730 0x7f59301446a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:46.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.958+0000 7f5934955700 1 -- 172.21.15.78:0/3966951501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f593015a6b0 msgr2=0x7f59300b07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:46.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.958+0000 7f5934955700 1 --2- 172.21.15.78:0/3966951501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f593015a6b0 0x7f59300b07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:46.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.958+0000 7f5934955700 1 -- 172.21.15.78:0/3966951501 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592c0094b0 con 0x7f59300b3a60 2023-10-11T09:43:46.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.958+0000 7f5935156700 1 --2- 172.21.15.78:0/3966951501 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f593006d730 0x7f59301446a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:43:46.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.958+0000 7f5935957700 1 --2- 172.21.15.78:0/3966951501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f593015a6b0 0x7f59300b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:43:46.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.958+0000 7f5934955700 1 --2- 172.21.15.78:0/3966951501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59300b3a60 0x7f59300b0290 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f592000c730 tx=0x7f592000caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:46.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.958+0000 7f5925ffb700 1 -- 172.21.15.78:0/3966951501 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59200052d0 con 0x7f59300b3a60 2023-10-11T09:43:46.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.958+0000 7f5925ffb700 1 -- 172.21.15.78:0/3966951501 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5920005430 con 0x7f59300b3a60 2023-10-11T09:43:46.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.958+0000 7f5925ffb700 1 -- 172.21.15.78:0/3966951501 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59200055a0 con 0x7f59300b3a60 2023-10-11T09:43:46.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.958+0000 7f5936158700 1 -- 172.21.15.78:0/3966951501 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59301576f0 con 0x7f59300b3a60 2023-10-11T09:43:46.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.958+0000 7f5936158700 1 -- 172.21.15.78:0/3966951501 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f593015da00 con 0x7f59300b3a60 2023-10-11T09:43:46.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.962+0000 7f5936158700 1 -- 172.21.15.78:0/3966951501 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5930157910 con 0x7f59300b3a60 2023-10-11T09:43:46.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.962+0000 7f5925ffb700 1 -- 172.21.15.78:0/3966951501 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f59200038e0 con 0x7f59300b3a60 2023-10-11T09:43:46.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.962+0000 7f5925ffb700 1 --2- 172.21.15.78:0/3966951501 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5914065ed0 0x7f5914068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:46.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.966+0000 7f5925ffb700 1 -- 172.21.15.78:0/3966951501 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6753+0+0 (secure 0 0 0) 0x7f59200830e0 con 0x7f59300b3a60 2023-10-11T09:43:46.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.966+0000 7f5935156700 1 --2- 172.21.15.78:0/3966951501 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5914065ed0 0x7f5914068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:46.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.966+0000 7f5935156700 1 --2- 172.21.15.78:0/3966951501 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5914065ed0 0x7f5914068380 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f592c000c00 tx=0x7f592c010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:46.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:46.970+0000 7f5925ffb700 1 -- 172.21.15.78:0/3966951501 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f59200503d0 con 0x7f59300b3a60 2023-10-11T09:43:47.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.142+0000 7f5936158700 1 -- 172.21.15.78:0/3966951501 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5930065d40 con 0x7f59300b3a60 2023-10-11T09:43:47.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.142+0000 7f5925ffb700 1 -- 172.21.15.78:0/3966951501 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v48) v1 ==== 104+0+11 (secure 0 0 0) 0x7f59200539f0 con 0x7f59300b3a60 2023-10-11T09:43:47.148 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:43:47.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.146+0000 7f5936158700 1 -- 172.21.15.78:0/3966951501 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5914065ed0 msgr2=0x7f5914068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:47.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.146+0000 7f5936158700 1 --2- 172.21.15.78:0/3966951501 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5914065ed0 0x7f5914068380 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f592c000c00 tx=0x7f592c010000 comp rx=0 tx=0).stop 2023-10-11T09:43:47.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.146+0000 7f5936158700 1 -- 172.21.15.78:0/3966951501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59300b3a60 msgr2=0x7f59300b0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:47.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.146+0000 7f5936158700 1 --2- 172.21.15.78:0/3966951501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59300b3a60 0x7f59300b0290 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f592000c730 tx=0x7f592000caf0 comp rx=0 tx=0).stop 2023-10-11T09:43:47.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.146+0000 7f5936158700 1 -- 172.21.15.78:0/3966951501 shutdown_connections 2023-10-11T09:43:47.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.146+0000 7f5936158700 1 --2- 172.21.15.78:0/3966951501 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5914065ed0 0x7f5914068380 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:47.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.146+0000 7f5936158700 1 --2- 172.21.15.78:0/3966951501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f593015a6b0 0x7f59300b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:47.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.146+0000 7f5936158700 1 --2- 172.21.15.78:0/3966951501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59300b3a60 0x7f59300b0290 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:47.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.146+0000 7f5936158700 1 --2- 172.21.15.78:0/3966951501 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f593006d730 0x7f59301446a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:47.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.146+0000 7f5936158700 1 -- 172.21.15.78:0/3966951501 >> 172.21.15.78:0/3966951501 conn(0x7f593006a060 msgr2=0x7f59300ae680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:47.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.146+0000 7f5936158700 1 -- 172.21.15.78:0/3966951501 shutdown_connections 2023-10-11T09:43:47.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.146+0000 7f5936158700 1 -- 172.21.15.78:0/3966951501 wait complete. 2023-10-11T09:43:47.173 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T09:43:47.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.350+0000 7f4a1d913700 1 Processor -- start 2023-10-11T09:43:47.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.350+0000 7f4a1d913700 1 -- start start 2023-10-11T09:43:47.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.350+0000 7f4a1d913700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4a180d2350 0x7f4a180d36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:47.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.350+0000 7f4a1d913700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a180cd690 con 0x7f4a180b84c0 2023-10-11T09:43:47.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.350+0000 7f4a1d913700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a180cd8e0 con 0x7f4a180b8890 2023-10-11T09:43:47.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.350+0000 7f4a1d913700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a180cdea0 con 0x7f4a180d2350 2023-10-11T09:43:47.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.350+0000 7f4a16ffd700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f4a180b84c0 0x7f4a180cfda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:42150/0 (socket says 172.21.15.78:42150) 2023-10-11T09:43:47.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.350+0000 7f4a16ffd700 1 -- 172.21.15.78:0/1579895840 learned_addr learned my addr 172.21.15.78:0/1579895840 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:43:47.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.350+0000 7f4a177fe700 1 --2- 172.21.15.78:0/1579895840 >> v2:172.21.15.133:3300/0 conn(0x7f4a180d2350 0x7f4a180d36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:47.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.350+0000 7f4a15ffb700 1 -- 172.21.15.78:0/1579895840 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 429570888 0 0) 0x7f4a180cd690 con 0x7f4a180b84c0 2023-10-11T09:43:47.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.350+0000 7f4a15ffb700 1 -- 172.21.15.78:0/1579895840 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a000035d0 con 0x7f4a180b84c0 2023-10-11T09:43:47.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.354+0000 7f4a15ffb700 1 -- 172.21.15.78:0/1579895840 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4073899353 0 0) 0x7f4a000035d0 con 0x7f4a180b84c0 2023-10-11T09:43:47.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.354+0000 7f4a15ffb700 1 -- 172.21.15.78:0/1579895840 >> v2:172.21.15.133:3300/0 conn(0x7f4a180d2350 msgr2=0x7f4a180d36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:47.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.354+0000 7f4a15ffb700 1 --2- 172.21.15.78:0/1579895840 >> v2:172.21.15.133:3300/0 conn(0x7f4a180d2350 0x7f4a180d36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:47.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.354+0000 7f4a15ffb700 1 -- 172.21.15.78:0/1579895840 >> v1:172.21.15.133:6789/0 conn(0x7f4a180b8890 legacy=0x7f4a180d2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:47.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.354+0000 7f4a15ffb700 1 -- 172.21.15.78:0/1579895840 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a180cdbc0 con 0x7f4a180b84c0 2023-10-11T09:43:47.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.354+0000 7f4a15ffb700 1 -- 172.21.15.78:0/1579895840 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f4a08001670 con 0x7f4a180b84c0 2023-10-11T09:43:47.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.354+0000 7f4a15ffb700 1 -- 172.21.15.78:0/1579895840 >> v1:172.21.15.78:6789/0 conn(0x7f4a180b84c0 legacy=0x7f4a180cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:47.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.354+0000 7f4a15ffb700 1 --2- 172.21.15.78:0/1579895840 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4a00003a20 0x7f4a00003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:47.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.354+0000 7f4a15ffb700 1 --2- 172.21.15.78:0/1579895840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a00004870 0x7f4a00004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:47.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.354+0000 7f4a15ffb700 1 --2- 172.21.15.78:0/1579895840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a00005220 0x7f4a0000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:47.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.354+0000 7f4a15ffb700 1 -- 172.21.15.78:0/1579895840 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a180cdea0 con 0x7f4a00005220 2023-10-11T09:43:47.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.354+0000 7f4a15ffb700 1 -- 172.21.15.78:0/1579895840 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0000c100 con 0x7f4a00004870 2023-10-11T09:43:47.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.354+0000 7f4a15ffb700 1 -- 172.21.15.78:0/1579895840 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0000c2c0 con 0x7f4a00003a20 2023-10-11T09:43:47.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.354+0000 7f4a177fe700 1 --2- 172.21.15.78:0/1579895840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a00005220 0x7f4a0000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:47.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.354+0000 7f4a167fc700 1 --2- 172.21.15.78:0/1579895840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a00004870 0x7f4a00004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:47.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.354+0000 7f4a16ffd700 1 --2- 172.21.15.78:0/1579895840 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4a00003a20 0x7f4a00003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:47.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.354+0000 7f4a177fe700 1 -- 172.21.15.78:0/1579895840 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4a00003a20 msgr2=0x7f4a00003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:47.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.354+0000 7f4a177fe700 1 --2- 172.21.15.78:0/1579895840 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4a00003a20 0x7f4a00003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:47.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.354+0000 7f4a177fe700 1 -- 172.21.15.78:0/1579895840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a00004870 msgr2=0x7f4a00004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:47.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.354+0000 7f4a177fe700 1 --2- 172.21.15.78:0/1579895840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a00004870 0x7f4a00004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:47.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.354+0000 7f4a177fe700 1 -- 172.21.15.78:0/1579895840 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a000035d0 con 0x7f4a00005220 2023-10-11T09:43:47.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.354+0000 7f4a177fe700 1 -- 172.21.15.78:0/1579895840 reap_dead start 2023-10-11T09:43:47.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.354+0000 7f4a177fe700 1 --2- 172.21.15.78:0/1579895840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a00005220 0x7f4a0000ba30 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f4a10009f90 tx=0x7f4a000058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e15fed933facfa server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:47.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.354+0000 7f4a15ffb700 1 -- 172.21.15.78:0/1579895840 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a10003a40 con 0x7f4a00005220 2023-10-11T09:43:47.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.354+0000 7f4a15ffb700 1 -- 172.21.15.78:0/1579895840 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1000f040 con 0x7f4a00005220 2023-10-11T09:43:47.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.354+0000 7f4a15ffb700 1 -- 172.21.15.78:0/1579895840 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a10003de0 con 0x7f4a00005220 2023-10-11T09:43:47.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.354+0000 7f4a1d913700 1 -- 172.21.15.78:0/1579895840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a00005220 msgr2=0x7f4a0000ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:47.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.354+0000 7f4a1d913700 1 --2- 172.21.15.78:0/1579895840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a00005220 0x7f4a0000ba30 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f4a10009f90 tx=0x7f4a000058a0 comp rx=0 tx=0).stop 2023-10-11T09:43:47.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.358+0000 7f4a1d913700 1 -- 172.21.15.78:0/1579895840 shutdown_connections 2023-10-11T09:43:47.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.358+0000 7f4a1d913700 1 --2- 172.21.15.78:0/1579895840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a00005220 0x7f4a0000ba30 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:47.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.358+0000 7f4a1d913700 1 -- 172.21.15.78:0/1579895840 >> 172.21.15.78:0/1579895840 conn(0x7f4a1806cb50 msgr2=0x7f4a1806cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:47.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.358+0000 7f4a1d913700 1 -- 172.21.15.78:0/1579895840 shutdown_connections 2023-10-11T09:43:47.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.358+0000 7f4a1d913700 1 -- 172.21.15.78:0/1579895840 wait complete. 2023-10-11T09:43:47.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.358+0000 7f4a1d913700 1 Processor -- start 2023-10-11T09:43:47.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.362+0000 7f4a1d913700 1 -- start start 2023-10-11T09:43:47.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.362+0000 7f4a1d913700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a00005220 0x7f4a180c8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:47.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.362+0000 7f4a1d913700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4a180cd030 0x7f4a180b4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:47.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.362+0000 7f4a1d913700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a180b47c0 0x7f4a181e5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:47.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.362+0000 7f4a1d913700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a180b4c30 con 0x7f4a00005220 2023-10-11T09:43:47.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.362+0000 7f4a1d913700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a181595f0 con 0x7f4a180b47c0 2023-10-11T09:43:47.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.362+0000 7f4a1d913700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a181f54c0 con 0x7f4a180cd030 2023-10-11T09:43:47.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.362+0000 7f4a16ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a00005220 0x7f4a180c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:47.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.362+0000 7f4a16ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a00005220 0x7f4a180c8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50124/0 (socket says 172.21.15.78:50124) 2023-10-11T09:43:47.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.362+0000 7f4a16ffd700 1 -- 172.21.15.78:0/896241066 learned_addr learned my addr 172.21.15.78:0/896241066 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:47.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.362+0000 7f4a177fe700 1 --2- 172.21.15.78:0/896241066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a180b47c0 0x7f4a181e5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:47.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.362+0000 7f4a167fc700 1 --2- 172.21.15.78:0/896241066 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4a180cd030 0x7f4a180b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:47.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.362+0000 7f4a16ffd700 1 -- 172.21.15.78:0/896241066 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4a180cd030 msgr2=0x7f4a180b4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:47.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.362+0000 7f4a16ffd700 1 --2- 172.21.15.78:0/896241066 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4a180cd030 0x7f4a180b4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:47.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.362+0000 7f4a16ffd700 1 -- 172.21.15.78:0/896241066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a180b47c0 msgr2=0x7f4a181e5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:47.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.362+0000 7f4a16ffd700 1 --2- 172.21.15.78:0/896241066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a180b47c0 0x7f4a181e5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:47.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.362+0000 7f4a16ffd700 1 -- 172.21.15.78:0/896241066 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a180c88b0 con 0x7f4a00005220 2023-10-11T09:43:47.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.362+0000 7f4a16ffd700 1 --2- 172.21.15.78:0/896241066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a00005220 0x7f4a180c8370 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f4a0800e7b0 tx=0x7f4a0800eac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:47.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.362+0000 7f49fffff700 1 -- 172.21.15.78:0/896241066 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a08010510 con 0x7f4a00005220 2023-10-11T09:43:47.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.362+0000 7f49fffff700 1 -- 172.21.15.78:0/896241066 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a08010670 con 0x7f4a00005220 2023-10-11T09:43:47.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.362+0000 7f49fffff700 1 -- 172.21.15.78:0/896241066 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a08010a20 con 0x7f4a00005220 2023-10-11T09:43:47.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.362+0000 7f4a1d913700 1 -- 172.21.15.78:0/896241066 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a181f5710 con 0x7f4a00005220 2023-10-11T09:43:47.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.362+0000 7f4a1d913700 1 -- 172.21.15.78:0/896241066 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a181f5cd0 con 0x7f4a00005220 2023-10-11T09:43:47.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.366+0000 7f4a1d913700 1 -- 172.21.15.78:0/896241066 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a1808bff0 con 0x7f4a00005220 2023-10-11T09:43:47.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.366+0000 7f49fffff700 1 -- 172.21.15.78:0/896241066 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f4a08010b80 con 0x7f4a00005220 2023-10-11T09:43:47.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.366+0000 7f49fffff700 1 --2- 172.21.15.78:0/896241066 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4a04065ed0 0x7f4a04068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:47.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.366+0000 7f49fffff700 1 -- 172.21.15.78:0/896241066 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6753+0+0 (secure 0 0 0) 0x7f4a08083330 con 0x7f4a00005220 2023-10-11T09:43:47.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.366+0000 7f4a167fc700 1 --2- 172.21.15.78:0/896241066 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4a04065ed0 0x7f4a04068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:47.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.370+0000 7f4a167fc700 1 --2- 172.21.15.78:0/896241066 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4a04065ed0 0x7f4a04068380 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4a180b1bb0 tx=0x7f4a0c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:47.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.374+0000 7f49fffff700 1 -- 172.21.15.78:0/896241066 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f4a080506d0 con 0x7f4a00005220 2023-10-11T09:43:47.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.542+0000 7f4a1d913700 1 -- 172.21.15.78:0/896241066 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4a18065db0 con 0x7f4a00005220 2023-10-11T09:43:47.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.542+0000 7f49fffff700 1 -- 172.21.15.78:0/896241066 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v48) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4a08053cf0 con 0x7f4a00005220 2023-10-11T09:43:47.549 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2023-10-11T09:43:47.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.550+0000 7f4a1d913700 1 -- 172.21.15.78:0/896241066 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4a04065ed0 msgr2=0x7f4a04068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:47.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.550+0000 7f4a1d913700 1 --2- 172.21.15.78:0/896241066 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4a04065ed0 0x7f4a04068380 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4a180b1bb0 tx=0x7f4a0c009040 comp rx=0 tx=0).stop 2023-10-11T09:43:47.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.550+0000 7f4a1d913700 1 -- 172.21.15.78:0/896241066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a00005220 msgr2=0x7f4a180c8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:47.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.550+0000 7f4a1d913700 1 --2- 172.21.15.78:0/896241066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a00005220 0x7f4a180c8370 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f4a0800e7b0 tx=0x7f4a0800eac0 comp rx=0 tx=0).stop 2023-10-11T09:43:47.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.550+0000 7f4a1d913700 1 -- 172.21.15.78:0/896241066 shutdown_connections 2023-10-11T09:43:47.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.550+0000 7f4a1d913700 1 --2- 172.21.15.78:0/896241066 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4a04065ed0 0x7f4a04068380 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:47.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.550+0000 7f4a1d913700 1 --2- 172.21.15.78:0/896241066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a180b47c0 0x7f4a181e5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:47.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.550+0000 7f4a1d913700 1 --2- 172.21.15.78:0/896241066 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4a180cd030 0x7f4a180b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:47.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.550+0000 7f4a1d913700 1 --2- 172.21.15.78:0/896241066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a00005220 0x7f4a180c8370 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:47.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.550+0000 7f4a1d913700 1 -- 172.21.15.78:0/896241066 >> 172.21.15.78:0/896241066 conn(0x7f4a1806cb50 msgr2=0x7f4a1815c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:47.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.550+0000 7f4a1d913700 1 -- 172.21.15.78:0/896241066 shutdown_connections 2023-10-11T09:43:47.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.550+0000 7f4a1d913700 1 -- 172.21.15.78:0/896241066 wait complete. 2023-10-11T09:43:47.574 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-10-11T09:43:47.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.746+0000 7f3c9f4e7700 1 Processor -- start 2023-10-11T09:43:47.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.746+0000 7f3c9f4e7700 1 -- start start 2023-10-11T09:43:47.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.746+0000 7f3c9f4e7700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f3c980cbd20 0x7f3c980ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:47.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.746+0000 7f3c9f4e7700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c98071fd0 con 0x7f3c980c2b40 2023-10-11T09:43:47.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.746+0000 7f3c9f4e7700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c98161360 con 0x7f3c9806a060 2023-10-11T09:43:47.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.746+0000 7f3c9f4e7700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c980cf0d0 con 0x7f3c980cbd20 2023-10-11T09:43:47.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.746+0000 7f3c9da84700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f3c980c2b40 0x7f3c9806de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:42160/0 (socket says 172.21.15.78:42160) 2023-10-11T09:43:47.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.746+0000 7f3c9da84700 1 -- 172.21.15.78:0/4216282351 learned_addr learned my addr 172.21.15.78:0/4216282351 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:43:47.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.746+0000 7f3c9ca82700 1 --2- 172.21.15.78:0/4216282351 >> v2:172.21.15.78:3300/0 conn(0x7f3c980cbd20 0x7f3c980ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:47.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.750+0000 7f3c8ffff700 1 -- 172.21.15.78:0/4216282351 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 92488415 0 0) 0x7f3c98071fd0 con 0x7f3c980c2b40 2023-10-11T09:43:47.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.750+0000 7f3c8ffff700 1 -- 172.21.15.78:0/4216282351 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c840035d0 con 0x7f3c980c2b40 2023-10-11T09:43:47.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.750+0000 7f3c8ffff700 1 -- 172.21.15.78:0/4216282351 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1370696690 0 0) 0x7f3c840035d0 con 0x7f3c980c2b40 2023-10-11T09:43:47.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.750+0000 7f3c8ffff700 1 -- 172.21.15.78:0/4216282351 >> v2:172.21.15.78:3300/0 conn(0x7f3c980cbd20 msgr2=0x7f3c980ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:47.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.750+0000 7f3c8ffff700 1 --2- 172.21.15.78:0/4216282351 >> v2:172.21.15.78:3300/0 conn(0x7f3c980cbd20 0x7f3c980ceb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:47.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.750+0000 7f3c8ffff700 1 -- 172.21.15.78:0/4216282351 >> v1:172.21.15.133:6789/0 conn(0x7f3c9806a060 legacy=0x7f3c9806a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:47.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.750+0000 7f3c8ffff700 1 -- 172.21.15.78:0/4216282351 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c980cc290 con 0x7f3c980c2b40 2023-10-11T09:43:47.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.750+0000 7f3c8ffff700 1 -- 172.21.15.78:0/4216282351 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f3c900019f0 con 0x7f3c980c2b40 2023-10-11T09:43:47.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.750+0000 7f3c8ffff700 1 -- 172.21.15.78:0/4216282351 >> v1:172.21.15.78:6789/0 conn(0x7f3c980c2b40 legacy=0x7f3c9806de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:47.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.750+0000 7f3c8ffff700 1 --2- 172.21.15.78:0/4216282351 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c84003a20 0x7f3c84003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:47.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.750+0000 7f3c8ffff700 1 --2- 172.21.15.78:0/4216282351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c84004870 0x7f3c84004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:47.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.750+0000 7f3c8ffff700 1 --2- 172.21.15.78:0/4216282351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c84005220 0x7f3c8400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:47.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.750+0000 7f3c8ffff700 1 -- 172.21.15.78:0/4216282351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c980cf0d0 con 0x7f3c84005220 2023-10-11T09:43:47.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.750+0000 7f3c8ffff700 1 -- 172.21.15.78:0/4216282351 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8400c100 con 0x7f3c84004870 2023-10-11T09:43:47.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.750+0000 7f3c8ffff700 1 -- 172.21.15.78:0/4216282351 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8400c2c0 con 0x7f3c84003a20 2023-10-11T09:43:47.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.750+0000 7f3c9da84700 1 --2- 172.21.15.78:0/4216282351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c84005220 0x7f3c8400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:47.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.750+0000 7f3c9ca82700 1 --2- 172.21.15.78:0/4216282351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c84004870 0x7f3c84004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:47.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.750+0000 7f3c9d283700 1 --2- 172.21.15.78:0/4216282351 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c84003a20 0x7f3c84003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:47.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.750+0000 7f3c9da84700 1 -- 172.21.15.78:0/4216282351 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c84003a20 msgr2=0x7f3c84003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:47.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.750+0000 7f3c9da84700 1 --2- 172.21.15.78:0/4216282351 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c84003a20 0x7f3c84003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:47.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.750+0000 7f3c9da84700 1 -- 172.21.15.78:0/4216282351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c84004870 msgr2=0x7f3c84004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:47.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.750+0000 7f3c9da84700 1 --2- 172.21.15.78:0/4216282351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c84004870 0x7f3c84004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:47.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.750+0000 7f3c9da84700 1 -- 172.21.15.78:0/4216282351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c840035d0 con 0x7f3c84005220 2023-10-11T09:43:47.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.750+0000 7f3c9da84700 1 -- 172.21.15.78:0/4216282351 reap_dead start 2023-10-11T09:43:47.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.750+0000 7f3c9da84700 1 --2- 172.21.15.78:0/4216282351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c84005220 0x7f3c8400ba30 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3c9000a7e0 tx=0x7f3c840058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943debf989a94639 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:47.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.750+0000 7f3c8ffff700 1 -- 172.21.15.78:0/4216282351 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9000a250 con 0x7f3c84005220 2023-10-11T09:43:47.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.750+0000 7f3c8ffff700 1 -- 172.21.15.78:0/4216282351 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c900072d0 con 0x7f3c84005220 2023-10-11T09:43:47.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.750+0000 7f3c8ffff700 1 -- 172.21.15.78:0/4216282351 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c90007680 con 0x7f3c84005220 2023-10-11T09:43:47.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.750+0000 7f3c9f4e7700 1 -- 172.21.15.78:0/4216282351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c84005220 msgr2=0x7f3c8400ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:47.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.750+0000 7f3c9f4e7700 1 --2- 172.21.15.78:0/4216282351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c84005220 0x7f3c8400ba30 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3c9000a7e0 tx=0x7f3c840058a0 comp rx=0 tx=0).stop 2023-10-11T09:43:47.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.754+0000 7f3c9f4e7700 1 -- 172.21.15.78:0/4216282351 shutdown_connections 2023-10-11T09:43:47.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.754+0000 7f3c9f4e7700 1 --2- 172.21.15.78:0/4216282351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c84005220 0x7f3c8400ba30 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:47.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.754+0000 7f3c9f4e7700 1 -- 172.21.15.78:0/4216282351 >> 172.21.15.78:0/4216282351 conn(0x7f3c980b24c0 msgr2=0x7f3c980cc7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:47.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.754+0000 7f3c9f4e7700 1 -- 172.21.15.78:0/4216282351 shutdown_connections 2023-10-11T09:43:47.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.754+0000 7f3c9f4e7700 1 -- 172.21.15.78:0/4216282351 wait complete. 2023-10-11T09:43:47.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.754+0000 7f3c9f4e7700 1 Processor -- start 2023-10-11T09:43:47.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.754+0000 7f3c9f4e7700 1 -- start start 2023-10-11T09:43:47.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.758+0000 7f3c9f4e7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c84005220 0x7f3c98165450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:47.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.758+0000 7f3c9f4e7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c98153180 0x7f3c98163910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:47.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.758+0000 7f3c9f4e7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c98163e50 0x7f3c980b0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:47.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.758+0000 7f3c9f4e7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c98164fc0 con 0x7f3c84005220 2023-10-11T09:43:47.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.758+0000 7f3c9f4e7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c981643b0 con 0x7f3c98163e50 2023-10-11T09:43:47.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.758+0000 7f3c9f4e7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c98164690 con 0x7f3c98153180 2023-10-11T09:43:47.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.758+0000 7f3c9d283700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c84005220 0x7f3c98165450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:47.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.758+0000 7f3c9d283700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c84005220 0x7f3c98165450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50152/0 (socket says 172.21.15.78:50152) 2023-10-11T09:43:47.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.758+0000 7f3c9d283700 1 -- 172.21.15.78:0/2631361771 learned_addr learned my addr 172.21.15.78:0/2631361771 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:47.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.758+0000 7f3c9ca82700 1 --2- 172.21.15.78:0/2631361771 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c98153180 0x7f3c98163910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:47.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.758+0000 7f3c9da84700 1 --2- 172.21.15.78:0/2631361771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c98163e50 0x7f3c980b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:47.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.758+0000 7f3c9d283700 1 -- 172.21.15.78:0/2631361771 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c98153180 msgr2=0x7f3c98163910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:47.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.758+0000 7f3c9d283700 1 --2- 172.21.15.78:0/2631361771 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c98153180 0x7f3c98163910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:47.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.758+0000 7f3c9d283700 1 -- 172.21.15.78:0/2631361771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c98163e50 msgr2=0x7f3c980b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:47.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.758+0000 7f3c9d283700 1 --2- 172.21.15.78:0/2631361771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c98163e50 0x7f3c980b0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:47.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.758+0000 7f3c9d283700 1 -- 172.21.15.78:0/2631361771 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c98164970 con 0x7f3c84005220 2023-10-11T09:43:47.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.758+0000 7f3c9da84700 1 --2- 172.21.15.78:0/2631361771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c98163e50 0x7f3c980b0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:43:47.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.758+0000 7f3c9d283700 1 --2- 172.21.15.78:0/2631361771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c84005220 0x7f3c98165450 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3c9400f7a0 tx=0x7f3c9400fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:47.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.758+0000 7f3c8dffb700 1 -- 172.21.15.78:0/2631361771 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c94017070 con 0x7f3c84005220 2023-10-11T09:43:47.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.758+0000 7f3c8dffb700 1 -- 172.21.15.78:0/2631361771 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c940052e0 con 0x7f3c84005220 2023-10-11T09:43:47.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.758+0000 7f3c8dffb700 1 -- 172.21.15.78:0/2631361771 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c94005470 con 0x7f3c84005220 2023-10-11T09:43:47.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.758+0000 7f3c9f4e7700 1 -- 172.21.15.78:0/2631361771 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c98164c50 con 0x7f3c84005220 2023-10-11T09:43:47.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.758+0000 7f3c9f4e7700 1 -- 172.21.15.78:0/2631361771 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c980c4490 con 0x7f3c84005220 2023-10-11T09:43:47.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.762+0000 7f3c9f4e7700 1 -- 172.21.15.78:0/2631361771 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c98165990 con 0x7f3c84005220 2023-10-11T09:43:47.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.762+0000 7f3c8dffb700 1 -- 172.21.15.78:0/2631361771 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f3c94006880 con 0x7f3c84005220 2023-10-11T09:43:47.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.762+0000 7f3c8dffb700 1 --2- 172.21.15.78:0/2631361771 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3c7c065ed0 0x7f3c7c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:47.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.762+0000 7f3c8dffb700 1 -- 172.21.15.78:0/2631361771 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6753+0+0 (secure 0 0 0) 0x7f3c94083360 con 0x7f3c84005220 2023-10-11T09:43:47.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.762+0000 7f3c9ca82700 1 --2- 172.21.15.78:0/2631361771 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3c7c065ed0 0x7f3c7c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:47.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.766+0000 7f3c9ca82700 1 --2- 172.21.15.78:0/2631361771 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3c7c065ed0 0x7f3c7c068380 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3c980b15c0 tx=0x7f3c88008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:47.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.766+0000 7f3c8dffb700 1 -- 172.21.15.78:0/2631361771 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f3c94050660 con 0x7f3c84005220 2023-10-11T09:43:47.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.942+0000 7f3c9f4e7700 1 -- 172.21.15.78:0/2631361771 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7f3c980ce540 con 0x7f3c84005220 2023-10-11T09:43:47.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.946+0000 7f3c8dffb700 1 -- 172.21.15.78:0/2631361771 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v48) v1 ==== 106+0+11 (secure 0 0 0) 0x7f3c94053c80 con 0x7f3c84005220 2023-10-11T09:43:47.950 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:43:47.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.946+0000 7f3c9f4e7700 1 -- 172.21.15.78:0/2631361771 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3c7c065ed0 msgr2=0x7f3c7c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:47.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.946+0000 7f3c9f4e7700 1 --2- 172.21.15.78:0/2631361771 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3c7c065ed0 0x7f3c7c068380 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3c980b15c0 tx=0x7f3c88008040 comp rx=0 tx=0).stop 2023-10-11T09:43:47.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.946+0000 7f3c9f4e7700 1 -- 172.21.15.78:0/2631361771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c84005220 msgr2=0x7f3c98165450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:47.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.946+0000 7f3c9f4e7700 1 --2- 172.21.15.78:0/2631361771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c84005220 0x7f3c98165450 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3c9400f7a0 tx=0x7f3c9400fb60 comp rx=0 tx=0).stop 2023-10-11T09:43:47.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.946+0000 7f3c9f4e7700 1 -- 172.21.15.78:0/2631361771 shutdown_connections 2023-10-11T09:43:47.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.946+0000 7f3c9f4e7700 1 --2- 172.21.15.78:0/2631361771 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3c7c065ed0 0x7f3c7c068380 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:47.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.946+0000 7f3c9f4e7700 1 --2- 172.21.15.78:0/2631361771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c98163e50 0x7f3c980b0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:47.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.946+0000 7f3c9f4e7700 1 --2- 172.21.15.78:0/2631361771 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c98153180 0x7f3c98163910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:47.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.946+0000 7f3c9f4e7700 1 --2- 172.21.15.78:0/2631361771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c84005220 0x7f3c98165450 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:47.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.946+0000 7f3c9f4e7700 1 -- 172.21.15.78:0/2631361771 >> 172.21.15.78:0/2631361771 conn(0x7f3c980b24c0 msgr2=0x7f3c980cc780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:47.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.946+0000 7f3c9f4e7700 1 -- 172.21.15.78:0/2631361771 shutdown_connections 2023-10-11T09:43:47.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:47.946+0000 7f3c9f4e7700 1 -- 172.21.15.78:0/2631361771 wait complete. 2023-10-11T09:43:47.973 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T09:43:48.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.138+0000 7f615b6ee700 1 Processor -- start 2023-10-11T09:43:48.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.138+0000 7f615b6ee700 1 -- start start 2023-10-11T09:43:48.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.142+0000 7f615b6ee700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61540cbd30 0x7f615406be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:48.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.142+0000 7f615b6ee700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f61540cd210 0x7f61540c9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:48.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.142+0000 7f615b6ee700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61540c93a0 con 0x7f615415c880 2023-10-11T09:43:48.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.142+0000 7f615b6ee700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6154161420 con 0x7f61540cd210 2023-10-11T09:43:48.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.142+0000 7f615b6ee700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61540cca10 con 0x7f61540cbd30 2023-10-11T09:43:48.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.142+0000 7f6159c8b700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f61540cd210 0x7f61540c9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:48.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.142+0000 7f6158c89700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61540cbd30 0x7f615406be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:48.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.142+0000 7f6158c89700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61540cbd30 0x7f615406be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:55444/0 (socket says 172.21.15.78:55444) 2023-10-11T09:43:48.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.142+0000 7f6158c89700 1 -- 172.21.15.78:0/3510507954 learned_addr learned my addr 172.21.15.78:0/3510507954 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:48.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.142+0000 7f6158c89700 1 -- 172.21.15.78:0/3510507954 >> v2:172.21.15.133:3300/0 conn(0x7f61540cd210 msgr2=0x7f61540c9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:48.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.142+0000 7f6158c89700 1 --2- 172.21.15.78:0/3510507954 >> v2:172.21.15.133:3300/0 conn(0x7f61540cd210 0x7f61540c9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:48.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.142+0000 7f6158c89700 1 -- 172.21.15.78:0/3510507954 >> v1:172.21.15.133:6789/0 conn(0x7f615415c880 legacy=0x7f61540cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:48.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.142+0000 7f6158c89700 1 -- 172.21.15.78:0/3510507954 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61540cccb0 con 0x7f61540cbd30 2023-10-11T09:43:48.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.142+0000 7f6158c89700 1 --2- 172.21.15.78:0/3510507954 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61540cbd30 0x7f615406be60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6144009910 tx=0x7f6144009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82fe4253f9727a31 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:48.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.142+0000 7f614bfff700 1 -- 172.21.15.78:0/3510507954 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6144011830 con 0x7f61540cbd30 2023-10-11T09:43:48.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.142+0000 7f614bfff700 1 -- 172.21.15.78:0/3510507954 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6144011990 con 0x7f61540cbd30 2023-10-11T09:43:48.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.142+0000 7f614bfff700 1 -- 172.21.15.78:0/3510507954 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6144011d40 con 0x7f61540cbd30 2023-10-11T09:43:48.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.146+0000 7f615b6ee700 1 -- 172.21.15.78:0/3510507954 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61540cbd30 msgr2=0x7f615406be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:48.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.146+0000 7f615b6ee700 1 --2- 172.21.15.78:0/3510507954 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61540cbd30 0x7f615406be60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6144009910 tx=0x7f6144009c20 comp rx=0 tx=0).stop 2023-10-11T09:43:48.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.146+0000 7f615b6ee700 1 -- 172.21.15.78:0/3510507954 shutdown_connections 2023-10-11T09:43:48.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.146+0000 7f615b6ee700 1 --2- 172.21.15.78:0/3510507954 >> v2:172.21.15.133:3300/0 conn(0x7f61540cd210 0x7f61540c9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:48.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.146+0000 7f615b6ee700 1 --2- 172.21.15.78:0/3510507954 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61540cbd30 0x7f615406be60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:48.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.146+0000 7f615b6ee700 1 -- 172.21.15.78:0/3510507954 >> 172.21.15.78:0/3510507954 conn(0x7f615406aca0 msgr2=0x7f61540b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:48.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.146+0000 7f615b6ee700 1 -- 172.21.15.78:0/3510507954 shutdown_connections 2023-10-11T09:43:48.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.146+0000 7f615b6ee700 1 -- 172.21.15.78:0/3510507954 wait complete. 2023-10-11T09:43:48.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.146+0000 7f615b6ee700 1 Processor -- start 2023-10-11T09:43:48.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.150+0000 7f615b6ee700 1 -- start start 2023-10-11T09:43:48.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.150+0000 7f615b6ee700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61540cbd30 0x7f615415f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:48.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.150+0000 7f615b6ee700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61540cd210 0x7f61541f9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:48.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.150+0000 7f615b6ee700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f615415c880 0x7f61541f9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:48.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.150+0000 7f615b6ee700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6154159e70 con 0x7f61540cbd30 2023-10-11T09:43:48.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.150+0000 7f615b6ee700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61540aeec0 con 0x7f61540cd210 2023-10-11T09:43:48.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.150+0000 7f615b6ee700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f615415d000 con 0x7f615415c880 2023-10-11T09:43:48.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.150+0000 7f615948a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61540cbd30 0x7f615415f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:48.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.150+0000 7f615948a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61540cbd30 0x7f615415f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50158/0 (socket says 172.21.15.78:50158) 2023-10-11T09:43:48.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.150+0000 7f615948a700 1 -- 172.21.15.78:0/3399752280 learned_addr learned my addr 172.21.15.78:0/3399752280 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:48.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.150+0000 7f6158c89700 1 --2- 172.21.15.78:0/3399752280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61540cd210 0x7f61541f9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:48.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.150+0000 7f6159c8b700 1 --2- 172.21.15.78:0/3399752280 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f615415c880 0x7f61541f9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:48.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.150+0000 7f615948a700 1 -- 172.21.15.78:0/3399752280 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f615415c880 msgr2=0x7f61541f9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:48.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.150+0000 7f615948a700 1 --2- 172.21.15.78:0/3399752280 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f615415c880 0x7f61541f9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:48.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.150+0000 7f615948a700 1 -- 172.21.15.78:0/3399752280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61540cd210 msgr2=0x7f61541f9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:48.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.150+0000 7f615948a700 1 --2- 172.21.15.78:0/3399752280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61540cd210 0x7f61541f9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:48.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.150+0000 7f615948a700 1 -- 172.21.15.78:0/3399752280 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6144009570 con 0x7f61540cbd30 2023-10-11T09:43:48.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.154+0000 7f615948a700 1 --2- 172.21.15.78:0/3399752280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61540cbd30 0x7f615415f670 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f615000c500 tx=0x7f615000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:48.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.154+0000 7f6149ffb700 1 -- 172.21.15.78:0/3399752280 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61500052d0 con 0x7f61540cbd30 2023-10-11T09:43:48.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.154+0000 7f6149ffb700 1 -- 172.21.15.78:0/3399752280 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6150005430 con 0x7f61540cbd30 2023-10-11T09:43:48.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.154+0000 7f6149ffb700 1 -- 172.21.15.78:0/3399752280 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61500055a0 con 0x7f61540cbd30 2023-10-11T09:43:48.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.154+0000 7f615b6ee700 1 -- 172.21.15.78:0/3399752280 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f615415d2e0 con 0x7f61540cbd30 2023-10-11T09:43:48.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.154+0000 7f615b6ee700 1 -- 172.21.15.78:0/3399752280 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f615415fbb0 con 0x7f61540cbd30 2023-10-11T09:43:48.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.154+0000 7f615b6ee700 1 -- 172.21.15.78:0/3399752280 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f615404e530 con 0x7f61540cbd30 2023-10-11T09:43:48.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.158+0000 7f6149ffb700 1 -- 172.21.15.78:0/3399752280 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f61500038e0 con 0x7f61540cbd30 2023-10-11T09:43:48.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.158+0000 7f6149ffb700 1 --2- 172.21.15.78:0/3399752280 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6138065ed0 0x7f6138068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:48.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.158+0000 7f6149ffb700 1 -- 172.21.15.78:0/3399752280 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6753+0+0 (secure 0 0 0) 0x7f6150082de0 con 0x7f61540cbd30 2023-10-11T09:43:48.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.158+0000 7f6158c89700 1 --2- 172.21.15.78:0/3399752280 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6138065ed0 0x7f6138068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:48.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.158+0000 7f6158c89700 1 --2- 172.21.15.78:0/3399752280 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6138065ed0 0x7f6138068380 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f61540cc120 tx=0x7f6144008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:48.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.166+0000 7f6149ffb700 1 -- 172.21.15.78:0/3399752280 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f6150050160 con 0x7f61540cbd30 2023-10-11T09:43:48.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.354+0000 7f615b6ee700 1 -- 172.21.15.78:0/3399752280 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f61540b1170 con 0x7f61540cbd30 2023-10-11T09:43:48.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.358+0000 7f6149ffb700 1 -- 172.21.15.78:0/3399752280 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v48) v1 ==== 80+0+372 (secure 0 0 0) 0x7f6150017080 con 0x7f61540cbd30 2023-10-11T09:43:48.362 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.133:6833/1601241351 2023-10-12T09:42:51.686883+0000 2023-10-11T09:43:48.362 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.133:6832/1601241351 2023-10-12T09:42:51.686883+0000 2023-10-11T09:43:48.362 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.133:6833/4103940204 2023-10-12T09:42:45.599736+0000 2023-10-11T09:43:48.363 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.133:6832/4103940204 2023-10-12T09:42:45.599736+0000 2023-10-11T09:43:48.363 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.133:6833/1862542010 2023-10-12T09:42:44.563306+0000 2023-10-11T09:43:48.363 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.133:6832/1862542010 2023-10-12T09:42:44.563306+0000 2023-10-11T09:43:48.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.358+0000 7f615b6ee700 1 -- 172.21.15.78:0/3399752280 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6138065ed0 msgr2=0x7f6138068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:48.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.358+0000 7f615b6ee700 1 --2- 172.21.15.78:0/3399752280 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6138065ed0 0x7f6138068380 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f61540cc120 tx=0x7f6144008030 comp rx=0 tx=0).stop 2023-10-11T09:43:48.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.358+0000 7f615b6ee700 1 -- 172.21.15.78:0/3399752280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61540cbd30 msgr2=0x7f615415f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:48.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.358+0000 7f615b6ee700 1 --2- 172.21.15.78:0/3399752280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61540cbd30 0x7f615415f670 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f615000c500 tx=0x7f615000c810 comp rx=0 tx=0).stop 2023-10-11T09:43:48.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.358+0000 7f615b6ee700 1 -- 172.21.15.78:0/3399752280 shutdown_connections 2023-10-11T09:43:48.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.358+0000 7f615b6ee700 1 --2- 172.21.15.78:0/3399752280 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6138065ed0 0x7f6138068380 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:48.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.358+0000 7f615b6ee700 1 --2- 172.21.15.78:0/3399752280 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f615415c880 0x7f61541f9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:48.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.358+0000 7f615b6ee700 1 --2- 172.21.15.78:0/3399752280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61540cd210 0x7f61541f9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:48.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.358+0000 7f615b6ee700 1 --2- 172.21.15.78:0/3399752280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61540cbd30 0x7f615415f670 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:48.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.358+0000 7f615b6ee700 1 -- 172.21.15.78:0/3399752280 >> 172.21.15.78:0/3399752280 conn(0x7f615406aca0 msgr2=0x7f615406cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:48.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.358+0000 7f615b6ee700 1 -- 172.21.15.78:0/3399752280 shutdown_connections 2023-10-11T09:43:48.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:48.358+0000 7f615b6ee700 1 -- 172.21.15.78:0/3399752280 wait complete. 2023-10-11T09:43:48.368 INFO:teuthology.orchestra.run.smithi078.stderr:listed 6 entries 2023-10-11T09:43:48.387 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi078.front.sepia.ceph.com... 2023-10-11T09:43:48.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:43:48.388 DEBUG:teuthology.orchestra.run.smithi078:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-11T09:43:48.414 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:ceph-fuse[47784]: fuse finished with error 0 and tester_r 0 2023-10-11T09:43:48.430 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2023-10-11T09:43:48.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:43:48.431 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-11T09:43:48.599 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:43:48.600 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:43:48.619 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352292k,nr_inodes=4088073,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1430 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275376k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-11T09:43:48.620 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2023-10-11T09:43:48.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:43:48.621 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-11T09:43:48.678 INFO:teuthology.orchestra.run.smithi133.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-10-11T09:43:48.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T09:43:48.680 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:43:48.680 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:43:48.697 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352272k,nr_inodes=4088068,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17487 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-11T09:43:48.697 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2023-10-11T09:43:48.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:43:48.698 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-11T09:43:48.761 INFO:teuthology.orchestra.run.smithi078.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-11T09:43:48.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T09:43:48.762 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:43:48.763 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:43:48.782 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352292k,nr_inodes=4088073,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1430 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275376k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-11T09:43:48.783 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2023-10-11T09:43:48.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:43:48.784 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-11T09:43:48.848 INFO:teuthology.orchestra.run.smithi133.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-11T09:43:48.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T09:43:48.850 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:43:49.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.022+0000 7fa28359e700 1 Processor -- start 2023-10-11T09:43:49.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.022+0000 7fa28359e700 1 -- start start 2023-10-11T09:43:49.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.026+0000 7fa28359e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fa2840b23e0 0x7fa2841641a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:49.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.026+0000 7fa28359e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2840b2c00 con 0x7fa28415a720 2023-10-11T09:43:49.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.026+0000 7fa28359e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2840b40a0 con 0x7fa284150580 2023-10-11T09:43:49.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.026+0000 7fa28359e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa28406e490 con 0x7fa2840b23e0 2023-10-11T09:43:49.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.026+0000 7fa281d9b700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fa28415a720 0x7fa2840b1cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:42166/0 (socket says 172.21.15.78:42166) 2023-10-11T09:43:49.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.026+0000 7fa281d9b700 1 -- 172.21.15.78:0/1451794161 learned_addr learned my addr 172.21.15.78:0/1451794161 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:43:49.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.026+0000 7fa282d9d700 1 --2- 172.21.15.78:0/1451794161 >> v2:172.21.15.78:3300/0 conn(0x7fa2840b23e0 0x7fa2841641a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:49.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.026+0000 7fa28159a700 1 -- 172.21.15.78:0/1451794161 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 531156970 0 0) 0x7fa2840b2c00 con 0x7fa28415a720 2023-10-11T09:43:49.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.026+0000 7fa28159a700 1 -- 172.21.15.78:0/1451794161 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa26c0035d0 con 0x7fa28415a720 2023-10-11T09:43:49.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.026+0000 7fa28159a700 1 -- 172.21.15.78:0/1451794161 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3128297020 0 0) 0x7fa2840b40a0 con 0x7fa284150580 2023-10-11T09:43:49.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.026+0000 7fa28159a700 1 -- 172.21.15.78:0/1451794161 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2840b2c00 con 0x7fa284150580 2023-10-11T09:43:49.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.026+0000 7fa282d9d700 1 -- 172.21.15.78:0/1451794161 >> v1:172.21.15.133:6789/0 conn(0x7fa284150580 legacy=0x7fa28406dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:49.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.026+0000 7fa28159a700 1 -- 172.21.15.78:0/1451794161 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 403558626 0 0) 0x7fa26c0035d0 con 0x7fa28415a720 2023-10-11T09:43:49.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.026+0000 7fa282d9d700 1 -- 172.21.15.78:0/1451794161 >> v1:172.21.15.78:6789/0 conn(0x7fa28415a720 legacy=0x7fa2840b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:49.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.026+0000 7fa282d9d700 1 -- 172.21.15.78:0/1451794161 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa284150950 con 0x7fa2840b23e0 2023-10-11T09:43:49.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.026+0000 7fa282d9d700 1 --2- 172.21.15.78:0/1451794161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2840b23e0 0x7fa2841641a0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fa27c009d90 tx=0x7fa27c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2417654ae7990f5a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:49.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.030+0000 7fa28159a700 1 -- 172.21.15.78:0/1451794161 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27c00e070 con 0x7fa2840b23e0 2023-10-11T09:43:49.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.030+0000 7fa28159a700 1 -- 172.21.15.78:0/1451794161 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa27c003a40 con 0x7fa2840b23e0 2023-10-11T09:43:49.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.030+0000 7fa28159a700 1 -- 172.21.15.78:0/1451794161 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa278001670 con 0x7fa2840b23e0 2023-10-11T09:43:49.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.030+0000 7fa28359e700 1 -- 172.21.15.78:0/1451794161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2840b23e0 msgr2=0x7fa2841641a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:49.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.030+0000 7fa28359e700 1 --2- 172.21.15.78:0/1451794161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2840b23e0 0x7fa2841641a0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fa27c009d90 tx=0x7fa27c00b330 comp rx=0 tx=0).stop 2023-10-11T09:43:49.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.030+0000 7fa28359e700 1 -- 172.21.15.78:0/1451794161 shutdown_connections 2023-10-11T09:43:49.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.030+0000 7fa28359e700 1 --2- 172.21.15.78:0/1451794161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2840b23e0 0x7fa2841641a0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:49.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.030+0000 7fa28359e700 1 -- 172.21.15.78:0/1451794161 >> 172.21.15.78:0/1451794161 conn(0x7fa2840730c0 msgr2=0x7fa284073490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:49.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.030+0000 7fa28359e700 1 -- 172.21.15.78:0/1451794161 shutdown_connections 2023-10-11T09:43:49.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.030+0000 7fa28359e700 1 -- 172.21.15.78:0/1451794161 wait complete. 2023-10-11T09:43:49.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.030+0000 7fa28359e700 1 Processor -- start 2023-10-11T09:43:49.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.030+0000 7fa28359e700 1 -- start start 2023-10-11T09:43:49.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.034+0000 7fa28359e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2840b23e0 0x7fa2840cb330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:49.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.034+0000 7fa28359e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa284150580 0x7fa284141120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:49.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.034+0000 7fa28359e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa28415a720 0x7fa284153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:49.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.034+0000 7fa28359e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2840cff00 con 0x7fa28415a720 2023-10-11T09:43:49.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.034+0000 7fa28359e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28414d410 con 0x7fa2840b23e0 2023-10-11T09:43:49.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.034+0000 7fa28359e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa28414d0e0 con 0x7fa284150580 2023-10-11T09:43:49.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.034+0000 7fa282d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa28415a720 0x7fa284153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:49.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.034+0000 7fa282d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa28415a720 0x7fa284153280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50172/0 (socket says 172.21.15.78:50172) 2023-10-11T09:43:49.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.034+0000 7fa282d9d700 1 -- 172.21.15.78:0/536099986 learned_addr learned my addr 172.21.15.78:0/536099986 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:49.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.034+0000 7fa28259c700 1 --2- 172.21.15.78:0/536099986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2840b23e0 0x7fa2840cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:49.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.034+0000 7fa281d9b700 1 --2- 172.21.15.78:0/536099986 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa284150580 0x7fa284141120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:49.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.034+0000 7fa282d9d700 1 -- 172.21.15.78:0/536099986 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa284150580 msgr2=0x7fa284141120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:49.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.034+0000 7fa282d9d700 1 --2- 172.21.15.78:0/536099986 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa284150580 0x7fa284141120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:49.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.034+0000 7fa282d9d700 1 -- 172.21.15.78:0/536099986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2840b23e0 msgr2=0x7fa2840cb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:49.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.034+0000 7fa282d9d700 1 --2- 172.21.15.78:0/536099986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2840b23e0 0x7fa2840cb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:49.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.034+0000 7fa282d9d700 1 -- 172.21.15.78:0/536099986 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa27c009a70 con 0x7fa28415a720 2023-10-11T09:43:49.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.034+0000 7fa28259c700 1 --2- 172.21.15.78:0/536099986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2840b23e0 0x7fa2840cb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:43:49.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.034+0000 7fa282d9d700 1 --2- 172.21.15.78:0/536099986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa28415a720 0x7fa284153280 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fa27c004000 tx=0x7fa27c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:49.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.034+0000 7fa272ffd700 1 -- 172.21.15.78:0/536099986 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27c0045c0 con 0x7fa28415a720 2023-10-11T09:43:49.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.034+0000 7fa272ffd700 1 -- 172.21.15.78:0/536099986 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa27c004720 con 0x7fa28415a720 2023-10-11T09:43:49.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.034+0000 7fa28359e700 1 -- 172.21.15.78:0/536099986 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2840cf940 con 0x7fa28415a720 2023-10-11T09:43:49.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.034+0000 7fa272ffd700 1 -- 172.21.15.78:0/536099986 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27c01d470 con 0x7fa28415a720 2023-10-11T09:43:49.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.034+0000 7fa28359e700 1 -- 172.21.15.78:0/536099986 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2840cf5a0 con 0x7fa28415a720 2023-10-11T09:43:49.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.038+0000 7fa28359e700 1 -- 172.21.15.78:0/536099986 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa28414d5a0 con 0x7fa28415a720 2023-10-11T09:43:49.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.038+0000 7fa272ffd700 1 -- 172.21.15.78:0/536099986 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fa27c01d5d0 con 0x7fa28415a720 2023-10-11T09:43:49.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.038+0000 7fa272ffd700 1 --2- 172.21.15.78:0/536099986 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa264065ed0 0x7fa264068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:49.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.038+0000 7fa272ffd700 1 -- 172.21.15.78:0/536099986 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6753+0+0 (secure 0 0 0) 0x7fa27c085370 con 0x7fa28415a720 2023-10-11T09:43:49.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.038+0000 7fa28259c700 1 --2- 172.21.15.78:0/536099986 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa264065ed0 0x7fa264068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:49.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.042+0000 7fa28259c700 1 --2- 172.21.15.78:0/536099986 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa264065ed0 0x7fa264068380 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa2740011e0 tx=0x7fa274009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:49.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.046+0000 7fa272ffd700 1 -- 172.21.15.78:0/536099986 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fa27c0562a0 con 0x7fa28415a720 2023-10-11T09:43:49.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.254+0000 7fa28359e700 1 -- 172.21.15.78:0/536099986 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa284065d40 con 0x7fa28415a720 2023-10-11T09:43:49.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.254+0000 7fa272ffd700 1 -- 172.21.15.78:0/536099986 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4654 (secure 0 0 0) 0x7fa27c055e30 con 0x7fa28415a720 2023-10-11T09:43:49.259 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:43:49.260 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":29,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/588936111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":588936111},{"type":"v1","addr":"172.21.15.78:6836","nonce":588936111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5357,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/2574909888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2574909888},{"type":"v1","addr":"172.21.15.133:6834","nonce":2574909888}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5791,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2948223101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":2948223101},{"type":"v1","addr":"172.21.15.133:6835","nonce":2948223101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:42:45.491515+0000","modified":"2023-10-11T09:42:54.734536+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5922":{"gid":5922,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":4,"addr":"172.21.15.78:6837/3137573764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":3137573764},{"type":"v1","addr":"172.21.15.78:6837","nonce":3137573764}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-11T09:43:49.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.258+0000 7fa28359e700 1 -- 172.21.15.78:0/536099986 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa264065ed0 msgr2=0x7fa264068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:49.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.258+0000 7fa28359e700 1 --2- 172.21.15.78:0/536099986 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa264065ed0 0x7fa264068380 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa2740011e0 tx=0x7fa274009040 comp rx=0 tx=0).stop 2023-10-11T09:43:49.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.258+0000 7fa28359e700 1 -- 172.21.15.78:0/536099986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa28415a720 msgr2=0x7fa284153280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:49.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.258+0000 7fa28359e700 1 --2- 172.21.15.78:0/536099986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa28415a720 0x7fa284153280 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fa27c004000 tx=0x7fa27c004af0 comp rx=0 tx=0).stop 2023-10-11T09:43:49.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.258+0000 7fa28359e700 1 -- 172.21.15.78:0/536099986 shutdown_connections 2023-10-11T09:43:49.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.258+0000 7fa28359e700 1 --2- 172.21.15.78:0/536099986 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa264065ed0 0x7fa264068380 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:49.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.258+0000 7fa28359e700 1 --2- 172.21.15.78:0/536099986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa28415a720 0x7fa284153280 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:49.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.258+0000 7fa28359e700 1 --2- 172.21.15.78:0/536099986 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa284150580 0x7fa284141120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:49.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.258+0000 7fa28359e700 1 --2- 172.21.15.78:0/536099986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2840b23e0 0x7fa2840cb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:49.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.258+0000 7fa28359e700 1 -- 172.21.15.78:0/536099986 >> 172.21.15.78:0/536099986 conn(0x7fa2840730c0 msgr2=0x7fa2840b2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:49.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.258+0000 7fa28359e700 1 -- 172.21.15.78:0/536099986 shutdown_connections 2023-10-11T09:43:49.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.258+0000 7fa28359e700 1 -- 172.21.15.78:0/536099986 wait complete. 2023-10-11T09:43:49.265 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 29 2023-10-11T09:43:49.283 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:43:49.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.446+0000 7f0e1b59e700 1 Processor -- start 2023-10-11T09:43:49.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.450+0000 7f0e1b59e700 1 -- start start 2023-10-11T09:43:49.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.450+0000 7f0e1b59e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0e1c164d20 0x7f0e1c06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:49.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.450+0000 7f0e1b59e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e1c165390 con 0x7f0e1c0caf20 2023-10-11T09:43:49.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.450+0000 7f0e1b59e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e1c0ca440 con 0x7f0e1c0cda10 2023-10-11T09:43:49.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.450+0000 7f0e1b59e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e1c0cb3a0 con 0x7f0e1c164d20 2023-10-11T09:43:49.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.454+0000 7f0e11d9b700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f0e1c0caf20 0x7f0e1c164610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:42182/0 (socket says 172.21.15.78:42182) 2023-10-11T09:43:49.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.454+0000 7f0e11d9b700 1 -- 172.21.15.78:0/3435219679 learned_addr learned my addr 172.21.15.78:0/3435219679 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:43:49.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.454+0000 7f0e1ad9d700 1 --2- 172.21.15.78:0/3435219679 >> v2:172.21.15.78:3300/0 conn(0x7f0e1c164d20 0x7f0e1c06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:49.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.454+0000 7f0e19d9b700 1 -- 172.21.15.78:0/3435219679 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2653452736 0 0) 0x7f0e1c165390 con 0x7f0e1c0caf20 2023-10-11T09:43:49.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.454+0000 7f0e19d9b700 1 -- 172.21.15.78:0/3435219679 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e080035d0 con 0x7f0e1c0caf20 2023-10-11T09:43:49.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.454+0000 7f0e1ad9d700 1 -- 172.21.15.78:0/3435219679 >> v1:172.21.15.133:6789/0 conn(0x7f0e1c0cda10 legacy=0x7f0e1c0cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:49.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.454+0000 7f0e1ad9d700 1 -- 172.21.15.78:0/3435219679 >> v1:172.21.15.78:6789/0 conn(0x7f0e1c0caf20 legacy=0x7f0e1c164610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:49.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.454+0000 7f0e1ad9d700 1 -- 172.21.15.78:0/3435219679 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e1c0ca720 con 0x7f0e1c164d20 2023-10-11T09:43:49.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.454+0000 7f0e1ad9d700 1 --2- 172.21.15.78:0/3435219679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e1c164d20 0x7f0e1c06bbf0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0e14009d90 tx=0x7f0e1400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee500be7d9610af server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:49.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.454+0000 7f0e19d9b700 1 -- 172.21.15.78:0/3435219679 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1400e070 con 0x7f0e1c164d20 2023-10-11T09:43:49.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.454+0000 7f0e19d9b700 1 -- 172.21.15.78:0/3435219679 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1400bb20 con 0x7f0e1c164d20 2023-10-11T09:43:49.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.454+0000 7f0e19d9b700 1 -- 172.21.15.78:0/3435219679 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e14012430 con 0x7f0e1c164d20 2023-10-11T09:43:49.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.454+0000 7f0e1b59e700 1 -- 172.21.15.78:0/3435219679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e1c164d20 msgr2=0x7f0e1c06bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:49.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.454+0000 7f0e1b59e700 1 --2- 172.21.15.78:0/3435219679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e1c164d20 0x7f0e1c06bbf0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0e14009d90 tx=0x7f0e1400b330 comp rx=0 tx=0).stop 2023-10-11T09:43:49.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.454+0000 7f0e1b59e700 1 -- 172.21.15.78:0/3435219679 shutdown_connections 2023-10-11T09:43:49.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.454+0000 7f0e1b59e700 1 --2- 172.21.15.78:0/3435219679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e1c164d20 0x7f0e1c06bbf0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:49.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.454+0000 7f0e1b59e700 1 -- 172.21.15.78:0/3435219679 >> 172.21.15.78:0/3435219679 conn(0x7f0e1c0cea60 msgr2=0x7f0e1c0cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:49.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.454+0000 7f0e1b59e700 1 -- 172.21.15.78:0/3435219679 shutdown_connections 2023-10-11T09:43:49.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.454+0000 7f0e1b59e700 1 -- 172.21.15.78:0/3435219679 wait complete. 2023-10-11T09:43:49.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.458+0000 7f0e1b59e700 1 Processor -- start 2023-10-11T09:43:49.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.458+0000 7f0e1b59e700 1 -- start start 2023-10-11T09:43:49.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.458+0000 7f0e1b59e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e1c0caf20 0x7f0e1c1449d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:49.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.458+0000 7f0e1b59e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e1c0cda10 0x7f0e1c06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:49.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.458+0000 7f0e1b59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e1c164d20 0x7f0e1c06c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:49.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.458+0000 7f0e1b59e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c147ff0 con 0x7f0e1c164d20 2023-10-11T09:43:49.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.458+0000 7f0e1b59e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c0b1f20 con 0x7f0e1c0cda10 2023-10-11T09:43:49.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.458+0000 7f0e1b59e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c147a30 con 0x7f0e1c0caf20 2023-10-11T09:43:49.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.458+0000 7f0e1ad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e1c164d20 0x7f0e1c06c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:49.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.458+0000 7f0e1ad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e1c164d20 0x7f0e1c06c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50192/0 (socket says 172.21.15.78:50192) 2023-10-11T09:43:49.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.458+0000 7f0e1ad9d700 1 -- 172.21.15.78:0/2731230833 learned_addr learned my addr 172.21.15.78:0/2731230833 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:49.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.458+0000 7f0e11d9b700 1 --2- 172.21.15.78:0/2731230833 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e1c0cda10 0x7f0e1c06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:49.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.458+0000 7f0e1a59c700 1 --2- 172.21.15.78:0/2731230833 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e1c0caf20 0x7f0e1c1449d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:49.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.458+0000 7f0e1ad9d700 1 -- 172.21.15.78:0/2731230833 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e1c0caf20 msgr2=0x7f0e1c1449d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:49.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.458+0000 7f0e1ad9d700 1 --2- 172.21.15.78:0/2731230833 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e1c0caf20 0x7f0e1c1449d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:49.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.458+0000 7f0e1ad9d700 1 -- 172.21.15.78:0/2731230833 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e1c0cda10 msgr2=0x7f0e1c06bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:49.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.458+0000 7f0e1ad9d700 1 --2- 172.21.15.78:0/2731230833 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e1c0cda10 0x7f0e1c06bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:49.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.458+0000 7f0e1ad9d700 1 -- 172.21.15.78:0/2731230833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e14009a70 con 0x7f0e1c164d20 2023-10-11T09:43:49.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.462+0000 7f0e1ad9d700 1 --2- 172.21.15.78:0/2731230833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e1c164d20 0x7f0e1c06c130 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c072690 tx=0x7f0e14004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:49.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.462+0000 7f0e137fe700 1 -- 172.21.15.78:0/2731230833 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e14012430 con 0x7f0e1c164d20 2023-10-11T09:43:49.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.462+0000 7f0e1b59e700 1 -- 172.21.15.78:0/2731230833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e1c147d10 con 0x7f0e1c164d20 2023-10-11T09:43:49.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.462+0000 7f0e137fe700 1 -- 172.21.15.78:0/2731230833 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e14020070 con 0x7f0e1c164d20 2023-10-11T09:43:49.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.462+0000 7f0e137fe700 1 -- 172.21.15.78:0/2731230833 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e14004280 con 0x7f0e1c164d20 2023-10-11T09:43:49.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.462+0000 7f0e1b59e700 1 -- 172.21.15.78:0/2731230833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e1c144f10 con 0x7f0e1c164d20 2023-10-11T09:43:49.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.462+0000 7f0e1b59e700 1 -- 172.21.15.78:0/2731230833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e1c0b1a10 con 0x7f0e1c164d20 2023-10-11T09:43:49.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.462+0000 7f0e137fe700 1 -- 172.21.15.78:0/2731230833 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f0e140043e0 con 0x7f0e1c164d20 2023-10-11T09:43:49.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.466+0000 7f0e137fe700 1 --2- 172.21.15.78:0/2731230833 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0dfc065ed0 0x7f0dfc068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:49.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.466+0000 7f0e137fe700 1 -- 172.21.15.78:0/2731230833 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6753+0+0 (secure 0 0 0) 0x7f0e14022070 con 0x7f0e1c164d20 2023-10-11T09:43:49.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.466+0000 7f0e1a59c700 1 --2- 172.21.15.78:0/2731230833 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0dfc065ed0 0x7f0dfc068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:49.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.466+0000 7f0e1a59c700 1 --2- 172.21.15.78:0/2731230833 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0dfc065ed0 0x7f0dfc068380 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c000f60 tx=0x7f0e0c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:49.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.470+0000 7f0e137fe700 1 -- 172.21.15.78:0/2731230833 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f0e1c0b1a10 con 0x7f0e1c164d20 2023-10-11T09:43:49.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.646+0000 7f0e1b59e700 1 -- 172.21.15.78:0/2731230833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e1c0b1a10 con 0x7f0e1c164d20 2023-10-11T09:43:49.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.646+0000 7f0e137fe700 1 -- 172.21.15.78:0/2731230833 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+20115 (secure 0 0 0) 0x7f0e1c0b1a10 con 0x7f0e1c164d20 2023-10-11T09:43:49.653 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:43:49.654 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":48,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:42:51.686927+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:41:57.280079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-11T09:41:58.218860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data_extra","create_time":"2023-10-11T09:42:12.954795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.10","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6833/1601241351":"2023-10-12T09:42:51.686883+0000","172.21.15.133:6832/1601241351":"2023-10-12T09:42:51.686883+0000","172.21.15.133:6833/4103940204":"2023-10-12T09:42:45.599736+0000","172.21.15.133:6832/4103940204":"2023-10-12T09:42:45.599736+0000","172.21.15.133:6833/1862542010":"2023-10-12T09:42:44.563306+0000","172.21.15.133:6832/1862542010":"2023-10-12T09:42:44.563306+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:43:49.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.650+0000 7f0e1b59e700 1 -- 172.21.15.78:0/2731230833 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0dfc065ed0 msgr2=0x7f0dfc068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:49.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.650+0000 7f0e1b59e700 1 --2- 172.21.15.78:0/2731230833 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0dfc065ed0 0x7f0dfc068380 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c000f60 tx=0x7f0e0c009040 comp rx=0 tx=0).stop 2023-10-11T09:43:49.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.650+0000 7f0e1b59e700 1 -- 172.21.15.78:0/2731230833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e1c164d20 msgr2=0x7f0e1c06c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:49.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.650+0000 7f0e1b59e700 1 --2- 172.21.15.78:0/2731230833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e1c164d20 0x7f0e1c06c130 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c072690 tx=0x7f0e14004af0 comp rx=0 tx=0).stop 2023-10-11T09:43:49.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.650+0000 7f0e1b59e700 1 -- 172.21.15.78:0/2731230833 shutdown_connections 2023-10-11T09:43:49.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.650+0000 7f0e1b59e700 1 --2- 172.21.15.78:0/2731230833 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0dfc065ed0 0x7f0dfc068380 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:49.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.650+0000 7f0e1b59e700 1 --2- 172.21.15.78:0/2731230833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e1c164d20 0x7f0e1c06c130 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:49.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.650+0000 7f0e1b59e700 1 --2- 172.21.15.78:0/2731230833 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e1c0cda10 0x7f0e1c06bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:49.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.650+0000 7f0e1b59e700 1 --2- 172.21.15.78:0/2731230833 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e1c0caf20 0x7f0e1c1449d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:49.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.650+0000 7f0e1b59e700 1 -- 172.21.15.78:0/2731230833 >> 172.21.15.78:0/2731230833 conn(0x7f0e1c0cea60 msgr2=0x7f0e1c1461d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:49.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.650+0000 7f0e1b59e700 1 -- 172.21.15.78:0/2731230833 shutdown_connections 2023-10-11T09:43:49.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.650+0000 7f0e1b59e700 1 -- 172.21.15.78:0/2731230833 wait complete. 2023-10-11T09:43:49.679 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T09:41:57.280079+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T09:41:58.218860+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-10-11T09:42:12.954795+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:43:49.680 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:43:49.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.854+0000 7fc2d5551700 1 Processor -- start 2023-10-11T09:43:49.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.854+0000 7fc2d5551700 1 -- start start 2023-10-11T09:43:49.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.858+0000 7fc2d5551700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2d013f970 0x7fc2d0071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:49.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.858+0000 7fc2d5551700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2d0071870 con 0x7fc2d00ca900 2023-10-11T09:43:49.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.858+0000 7fc2d5551700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2d015dc90 con 0x7fc2d0072240 2023-10-11T09:43:49.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.858+0000 7fc2d5551700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d0070f40 con 0x7fc2d013f970 2023-10-11T09:43:49.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.858+0000 7fc2d4d50700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fc2d00ca900 0x7fc2d00b1820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:42186/0 (socket says 172.21.15.78:42186) 2023-10-11T09:43:49.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.858+0000 7fc2d4d50700 1 -- 172.21.15.78:0/3226450944 learned_addr learned my addr 172.21.15.78:0/3226450944 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:43:49.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.858+0000 7fc2cffff700 1 --2- 172.21.15.78:0/3226450944 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2d013f970 0x7fc2d0071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:49.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.858+0000 7fc2ceffd700 1 -- 172.21.15.78:0/3226450944 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2367296920 0 0) 0x7fc2d0071870 con 0x7fc2d00ca900 2023-10-11T09:43:49.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.858+0000 7fc2ceffd700 1 -- 172.21.15.78:0/3226450944 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2b80035d0 con 0x7fc2d00ca900 2023-10-11T09:43:49.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.862+0000 7fc2ceffd700 1 -- 172.21.15.78:0/3226450944 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 271033432 0 0) 0x7fc2b80035d0 con 0x7fc2d00ca900 2023-10-11T09:43:49.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.862+0000 7fc2ceffd700 1 -- 172.21.15.78:0/3226450944 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2d013f970 msgr2=0x7fc2d0071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:49.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.862+0000 7fc2ceffd700 1 --2- 172.21.15.78:0/3226450944 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2d013f970 0x7fc2d0071d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:49.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.862+0000 7fc2ceffd700 1 -- 172.21.15.78:0/3226450944 >> v1:172.21.15.133:6789/0 conn(0x7fc2d0072240 legacy=0x7fc2d00ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:49.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.862+0000 7fc2ceffd700 1 -- 172.21.15.78:0/3226450944 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d0071220 con 0x7fc2d00ca900 2023-10-11T09:43:49.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.862+0000 7fc2ceffd700 1 -- 172.21.15.78:0/3226450944 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7fc2c80019f0 con 0x7fc2d00ca900 2023-10-11T09:43:49.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.862+0000 7fc2ceffd700 1 -- 172.21.15.78:0/3226450944 >> v1:172.21.15.78:6789/0 conn(0x7fc2d00ca900 legacy=0x7fc2d00b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:49.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.862+0000 7fc2ceffd700 1 --2- 172.21.15.78:0/3226450944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2b8003a50 0x7fc2b8003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:49.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.862+0000 7fc2ceffd700 1 --2- 172.21.15.78:0/3226450944 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2b8004870 0x7fc2b8004ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:49.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.862+0000 7fc2ceffd700 1 --2- 172.21.15.78:0/3226450944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b80051e0 0x7fc2b800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:49.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.862+0000 7fc2ceffd700 1 -- 172.21.15.78:0/3226450944 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d0070f40 con 0x7fc2b80051e0 2023-10-11T09:43:49.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.862+0000 7fc2ceffd700 1 -- 172.21.15.78:0/3226450944 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b800bf70 con 0x7fc2b8003a50 2023-10-11T09:43:49.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.862+0000 7fc2ceffd700 1 -- 172.21.15.78:0/3226450944 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b800c0b0 con 0x7fc2b8004870 2023-10-11T09:43:49.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.862+0000 7fc2d4d50700 1 --2- 172.21.15.78:0/3226450944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b80051e0 0x7fc2b800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:49.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.862+0000 7fc2d4d50700 1 -- 172.21.15.78:0/3226450944 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2b8004870 msgr2=0x7fc2b8004ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:49.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.862+0000 7fc2d4d50700 1 --2- 172.21.15.78:0/3226450944 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2b8004870 0x7fc2b8004ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:49.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.862+0000 7fc2d4d50700 1 -- 172.21.15.78:0/3226450944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2b8003a50 msgr2=0x7fc2b8003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:49.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.862+0000 7fc2d4d50700 1 --2- 172.21.15.78:0/3226450944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2b8003a50 0x7fc2b8003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:49.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.862+0000 7fc2d4d50700 1 -- 172.21.15.78:0/3226450944 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b80035d0 con 0x7fc2b80051e0 2023-10-11T09:43:49.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.862+0000 7fc2d4d50700 1 --2- 172.21.15.78:0/3226450944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b80051e0 0x7fc2b800ba30 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc2c800a760 tx=0x7fc2b80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c725731e3abefd6f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:49.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.862+0000 7fc2ceffd700 1 -- 172.21.15.78:0/3226450944 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c800a250 con 0x7fc2b80051e0 2023-10-11T09:43:49.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.862+0000 7fc2ceffd700 1 -- 172.21.15.78:0/3226450944 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c80070d0 con 0x7fc2b80051e0 2023-10-11T09:43:49.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.862+0000 7fc2ceffd700 1 -- 172.21.15.78:0/3226450944 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8007480 con 0x7fc2b80051e0 2023-10-11T09:43:49.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.862+0000 7fc2d5551700 1 -- 172.21.15.78:0/3226450944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b80051e0 msgr2=0x7fc2b800ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:49.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.862+0000 7fc2d5551700 1 --2- 172.21.15.78:0/3226450944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b80051e0 0x7fc2b800ba30 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc2c800a760 tx=0x7fc2b80058a0 comp rx=0 tx=0).stop 2023-10-11T09:43:49.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.862+0000 7fc2d4d50700 1 -- 172.21.15.78:0/3226450944 reap_dead start 2023-10-11T09:43:49.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.862+0000 7fc2d5551700 1 -- 172.21.15.78:0/3226450944 shutdown_connections 2023-10-11T09:43:49.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.862+0000 7fc2d5551700 1 -- 172.21.15.78:0/3226450944 >> 172.21.15.78:0/3226450944 conn(0x7fc2d006dd80 msgr2=0x7fc2d006e180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:49.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.862+0000 7fc2d5551700 1 -- 172.21.15.78:0/3226450944 shutdown_connections 2023-10-11T09:43:49.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.862+0000 7fc2d5551700 1 -- 172.21.15.78:0/3226450944 wait complete. 2023-10-11T09:43:49.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.866+0000 7fc2d5551700 1 Processor -- start 2023-10-11T09:43:49.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.866+0000 7fc2d5551700 1 -- start start 2023-10-11T09:43:49.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.866+0000 7fc2d5551700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b80051e0 0x7fc2d013ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:49.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.866+0000 7fc2d5551700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2d013ec00 0x7fc2d013a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:49.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.866+0000 7fc2d5551700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2d00b5590 0x7fc2d00cb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:49.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.866+0000 7fc2d5551700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d013ac40 con 0x7fc2b80051e0 2023-10-11T09:43:49.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.866+0000 7fc2d5551700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d00b4810 con 0x7fc2d013ec00 2023-10-11T09:43:49.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.866+0000 7fc2d5551700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d013bb00 con 0x7fc2d00b5590 2023-10-11T09:43:49.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.866+0000 7fc2cffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b80051e0 0x7fc2d013ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:49.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.866+0000 7fc2cffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b80051e0 0x7fc2d013ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50214/0 (socket says 172.21.15.78:50214) 2023-10-11T09:43:49.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.866+0000 7fc2cffff700 1 -- 172.21.15.78:0/1714447562 learned_addr learned my addr 172.21.15.78:0/1714447562 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:49.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.866+0000 7fc2cf7fe700 1 --2- 172.21.15.78:0/1714447562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2d013ec00 0x7fc2d013a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:49.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.866+0000 7fc2d4d50700 1 --2- 172.21.15.78:0/1714447562 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2d00b5590 0x7fc2d00cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:49.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.866+0000 7fc2cffff700 1 -- 172.21.15.78:0/1714447562 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2d00b5590 msgr2=0x7fc2d00cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:49.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.866+0000 7fc2cffff700 1 --2- 172.21.15.78:0/1714447562 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2d00b5590 0x7fc2d00cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:49.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.866+0000 7fc2cffff700 1 -- 172.21.15.78:0/1714447562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2d013ec00 msgr2=0x7fc2d013a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:49.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.866+0000 7fc2cffff700 1 --2- 172.21.15.78:0/1714447562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2d013ec00 0x7fc2d013a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:49.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.866+0000 7fc2cffff700 1 -- 172.21.15.78:0/1714447562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d00b4ab0 con 0x7fc2b80051e0 2023-10-11T09:43:49.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.866+0000 7fc2cffff700 1 --2- 172.21.15.78:0/1714447562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b80051e0 0x7fc2d013ae40 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000e830 tx=0x7fc2c000eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:49.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.866+0000 7fc2ccff9700 1 -- 172.21.15.78:0/1714447562 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0013680 con 0x7fc2b80051e0 2023-10-11T09:43:49.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.866+0000 7fc2ccff9700 1 -- 172.21.15.78:0/1714447562 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c00137e0 con 0x7fc2b80051e0 2023-10-11T09:43:49.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.866+0000 7fc2d5551700 1 -- 172.21.15.78:0/1714447562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2d00b4d90 con 0x7fc2b80051e0 2023-10-11T09:43:49.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.866+0000 7fc2ccff9700 1 -- 172.21.15.78:0/1714447562 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0013b90 con 0x7fc2b80051e0 2023-10-11T09:43:49.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.866+0000 7fc2d5551700 1 -- 172.21.15.78:0/1714447562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2d013b4d0 con 0x7fc2b80051e0 2023-10-11T09:43:49.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.870+0000 7fc2d5551700 1 -- 172.21.15.78:0/1714447562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2d0065d40 con 0x7fc2b80051e0 2023-10-11T09:43:49.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.870+0000 7fc2ccff9700 1 -- 172.21.15.78:0/1714447562 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fc2c0013cf0 con 0x7fc2b80051e0 2023-10-11T09:43:49.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.870+0000 7fc2ccff9700 1 --2- 172.21.15.78:0/1714447562 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc2bc065ed0 0x7fc2bc068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:49.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.874+0000 7fc2ccff9700 1 -- 172.21.15.78:0/1714447562 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6753+0+0 (secure 0 0 0) 0x7fc2c0083a80 con 0x7fc2b80051e0 2023-10-11T09:43:49.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.874+0000 7fc2cf7fe700 1 --2- 172.21.15.78:0/1714447562 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc2bc065ed0 0x7fc2bc068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:49.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.874+0000 7fc2cf7fe700 1 --2- 172.21.15.78:0/1714447562 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc2bc065ed0 0x7fc2bc068380 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc2d013eb90 tx=0x7fc2c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:49.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:49.878+0000 7fc2ccff9700 1 -- 172.21.15.78:0/1714447562 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fc2c0050d80 con 0x7fc2b80051e0 2023-10-11T09:43:50.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.046+0000 7fc2d5551700 1 -- 172.21.15.78:0/1714447562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc2d00cc5a0 con 0x7fc2b80051e0 2023-10-11T09:43:50.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.046+0000 7fc2ccff9700 1 -- 172.21.15.78:0/1714447562 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc2c00543a0 con 0x7fc2b80051e0 2023-10-11T09:43:50.054 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:43:50.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.050+0000 7fc2d5551700 1 -- 172.21.15.78:0/1714447562 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc2bc065ed0 msgr2=0x7fc2bc068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:50.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.050+0000 7fc2d5551700 1 --2- 172.21.15.78:0/1714447562 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc2bc065ed0 0x7fc2bc068380 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc2d013eb90 tx=0x7fc2c4009040 comp rx=0 tx=0).stop 2023-10-11T09:43:50.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.050+0000 7fc2d5551700 1 -- 172.21.15.78:0/1714447562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b80051e0 msgr2=0x7fc2d013ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:50.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.050+0000 7fc2d5551700 1 --2- 172.21.15.78:0/1714447562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b80051e0 0x7fc2d013ae40 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000e830 tx=0x7fc2c000eb40 comp rx=0 tx=0).stop 2023-10-11T09:43:50.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.050+0000 7fc2d5551700 1 -- 172.21.15.78:0/1714447562 shutdown_connections 2023-10-11T09:43:50.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.050+0000 7fc2d5551700 1 --2- 172.21.15.78:0/1714447562 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc2bc065ed0 0x7fc2bc068380 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:50.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.050+0000 7fc2d5551700 1 --2- 172.21.15.78:0/1714447562 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2d00b5590 0x7fc2d00cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:50.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.050+0000 7fc2d5551700 1 --2- 172.21.15.78:0/1714447562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2d013ec00 0x7fc2d013a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:50.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.050+0000 7fc2d5551700 1 --2- 172.21.15.78:0/1714447562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b80051e0 0x7fc2d013ae40 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:50.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.050+0000 7fc2d5551700 1 -- 172.21.15.78:0/1714447562 >> 172.21.15.78:0/1714447562 conn(0x7fc2d006dd80 msgr2=0x7fc2d006a060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:50.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.050+0000 7fc2d5551700 1 -- 172.21.15.78:0/1714447562 shutdown_connections 2023-10-11T09:43:50.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.050+0000 7fc2d5551700 1 -- 172.21.15.78:0/1714447562 wait complete. 2023-10-11T09:43:50.078 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:43:50.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.242+0000 7f1faf59e700 1 Processor -- start 2023-10-11T09:43:50.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.242+0000 7f1faf59e700 1 -- start start 2023-10-11T09:43:50.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.246+0000 7f1faf59e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1fb00b3610 0x7f1fb00d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:50.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.246+0000 7f1faf59e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f1fb00b3a90 0x7f1fb00b85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:50.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.246+0000 7f1faf59e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fb00b47f0 con 0x7f1fb00b3240 2023-10-11T09:43:50.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.246+0000 7f1faf59e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fb00b8130 con 0x7f1fb00b3a90 2023-10-11T09:43:50.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.246+0000 7f1faf59e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fb00b41e0 con 0x7f1fb00b3610 2023-10-11T09:43:50.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.246+0000 7f1fae59c700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f1fb00b3240 0x7f1fb015d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:42188/0 (socket says 172.21.15.78:42188) 2023-10-11T09:43:50.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.246+0000 7f1fae59c700 1 -- 172.21.15.78:0/2523247572 learned_addr learned my addr 172.21.15.78:0/2523247572 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:43:50.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.246+0000 7f1faed9d700 1 --2- 172.21.15.78:0/2523247572 >> v2:172.21.15.78:3300/0 conn(0x7f1fb00b3a90 0x7f1fb00b85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:50.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.246+0000 7f1fa5d9b700 1 --2- 172.21.15.78:0/2523247572 >> v2:172.21.15.133:3300/0 conn(0x7f1fb00b3610 0x7f1fb00d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:50.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.246+0000 7f1fadd9b700 1 -- 172.21.15.78:0/2523247572 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3907349381 0 0) 0x7f1fa0001110 con 0x7f1fb00b3240 2023-10-11T09:43:50.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.246+0000 7f1fadd9b700 1 -- 172.21.15.78:0/2523247572 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f900035d0 con 0x7f1fb00b3240 2023-10-11T09:43:50.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.246+0000 7f1faed9d700 1 -- 172.21.15.78:0/2523247572 >> v2:172.21.15.133:3300/0 conn(0x7f1fb00b3610 msgr2=0x7f1fb00d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:50.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.246+0000 7f1faed9d700 1 --2- 172.21.15.78:0/2523247572 >> v2:172.21.15.133:3300/0 conn(0x7f1fb00b3610 0x7f1fb00d29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:50.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.246+0000 7f1faed9d700 1 -- 172.21.15.78:0/2523247572 >> v1:172.21.15.78:6789/0 conn(0x7f1fb00b3240 legacy=0x7f1fb015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:50.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.246+0000 7f1faed9d700 1 -- 172.21.15.78:0/2523247572 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb00b7800 con 0x7f1fb00b3a90 2023-10-11T09:43:50.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.246+0000 7f1faed9d700 1 --2- 172.21.15.78:0/2523247572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fb00b3a90 0x7f1fb00b85c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1fb00d0a90 tx=0x7f1fa800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54cf18717d6082e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:50.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.250+0000 7f1fadd9b700 1 -- 172.21.15.78:0/2523247572 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa800e070 con 0x7f1fb00b3a90 2023-10-11T09:43:50.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.250+0000 7f1fadd9b700 1 -- 172.21.15.78:0/2523247572 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa800bb20 con 0x7f1fb00b3a90 2023-10-11T09:43:50.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.250+0000 7f1fadd9b700 1 -- 172.21.15.78:0/2523247572 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa800be90 con 0x7f1fb00b3a90 2023-10-11T09:43:50.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.250+0000 7f1faf59e700 1 -- 172.21.15.78:0/2523247572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fb00b3a90 msgr2=0x7f1fb00b85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:50.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.250+0000 7f1faf59e700 1 --2- 172.21.15.78:0/2523247572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fb00b3a90 0x7f1fb00b85c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1fb00d0a90 tx=0x7f1fa800b330 comp rx=0 tx=0).stop 2023-10-11T09:43:50.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.250+0000 7f1faf59e700 1 -- 172.21.15.78:0/2523247572 shutdown_connections 2023-10-11T09:43:50.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.250+0000 7f1faf59e700 1 --2- 172.21.15.78:0/2523247572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fb00b3a90 0x7f1fb00b85c0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:50.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.250+0000 7f1faf59e700 1 --2- 172.21.15.78:0/2523247572 >> v2:172.21.15.133:3300/0 conn(0x7f1fb00b3610 0x7f1fb00d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:50.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.250+0000 7f1faf59e700 1 -- 172.21.15.78:0/2523247572 >> 172.21.15.78:0/2523247572 conn(0x7f1fb009c260 msgr2=0x7f1fb00d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:50.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.250+0000 7f1faf59e700 1 -- 172.21.15.78:0/2523247572 shutdown_connections 2023-10-11T09:43:50.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.250+0000 7f1faf59e700 1 -- 172.21.15.78:0/2523247572 wait complete. 2023-10-11T09:43:50.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.254+0000 7f1faf59e700 1 Processor -- start 2023-10-11T09:43:50.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.254+0000 7f1faf59e700 1 -- start start 2023-10-11T09:43:50.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.254+0000 7f1faf59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fb00b3240 0x7f1fb01f2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:50.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.254+0000 7f1faf59e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fb00b3610 0x7f1fb00c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:50.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.254+0000 7f1faf59e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1fb00b3a90 0x7f1fb015e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:50.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.254+0000 7f1faf59e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb015ce70 con 0x7f1fb00b3240 2023-10-11T09:43:50.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.254+0000 7f1faf59e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb01f2700 con 0x7f1fb00b3610 2023-10-11T09:43:50.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.254+0000 7f1faf59e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb01f29e0 con 0x7f1fb00b3a90 2023-10-11T09:43:50.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.254+0000 7f1fae59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fb00b3240 0x7f1fb01f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:50.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.254+0000 7f1fae59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fb00b3240 0x7f1fb01f2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50234/0 (socket says 172.21.15.78:50234) 2023-10-11T09:43:50.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.254+0000 7f1fae59c700 1 -- 172.21.15.78:0/611636570 learned_addr learned my addr 172.21.15.78:0/611636570 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:50.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.254+0000 7f1fa5d9b700 1 --2- 172.21.15.78:0/611636570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fb00b3610 0x7f1fb00c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:50.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.254+0000 7f1faed9d700 1 --2- 172.21.15.78:0/611636570 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1fb00b3a90 0x7f1fb015e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:50.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.254+0000 7f1fae59c700 1 -- 172.21.15.78:0/611636570 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1fb00b3a90 msgr2=0x7f1fb015e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:50.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.254+0000 7f1fae59c700 1 --2- 172.21.15.78:0/611636570 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1fb00b3a90 0x7f1fb015e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:50.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.254+0000 7f1fae59c700 1 -- 172.21.15.78:0/611636570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fb00b3610 msgr2=0x7f1fb00c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:50.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.254+0000 7f1fae59c700 1 --2- 172.21.15.78:0/611636570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fb00b3610 0x7f1fb00c7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:50.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.254+0000 7f1fae59c700 1 -- 172.21.15.78:0/611636570 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa8009a70 con 0x7f1fb00b3240 2023-10-11T09:43:50.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.254+0000 7f1fae59c700 1 --2- 172.21.15.78:0/611636570 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fb00b3240 0x7f1fb01f2100 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000c4d0 tx=0x7f1fa000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:50.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.258+0000 7f1fa77fe700 1 -- 172.21.15.78:0/611636570 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0005d90 con 0x7f1fb00b3240 2023-10-11T09:43:50.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.258+0000 7f1fa77fe700 1 -- 172.21.15.78:0/611636570 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa000ce80 con 0x7f1fb00b3240 2023-10-11T09:43:50.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.258+0000 7f1faf59e700 1 -- 172.21.15.78:0/611636570 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fb01f2cc0 con 0x7f1fb00b3240 2023-10-11T09:43:50.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.258+0000 7f1fa77fe700 1 -- 172.21.15.78:0/611636570 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0007310 con 0x7f1fb00b3240 2023-10-11T09:43:50.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.258+0000 7f1faf59e700 1 -- 172.21.15.78:0/611636570 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fb00c75c0 con 0x7f1fb00b3240 2023-10-11T09:43:50.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.258+0000 7f1faf59e700 1 -- 172.21.15.78:0/611636570 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fb015cfb0 con 0x7f1fb00b3240 2023-10-11T09:43:50.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.258+0000 7f1fa77fe700 1 -- 172.21.15.78:0/611636570 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f1fa0007470 con 0x7f1fb00b3240 2023-10-11T09:43:50.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.262+0000 7f1fa77fe700 1 --2- 172.21.15.78:0/611636570 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1f94065ed0 0x7f1f94068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:50.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.262+0000 7f1fa77fe700 1 -- 172.21.15.78:0/611636570 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6753+0+0 (secure 0 0 0) 0x7f1fa0082d40 con 0x7f1fb00b3240 2023-10-11T09:43:50.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.262+0000 7f1fa5d9b700 1 --2- 172.21.15.78:0/611636570 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1f94065ed0 0x7f1f94068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:50.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.262+0000 7f1fa5d9b700 1 --2- 172.21.15.78:0/611636570 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1f94065ed0 0x7f1f94068380 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1fb015fe90 tx=0x7f1f98009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:50.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.266+0000 7f1fa77fe700 1 -- 172.21.15.78:0/611636570 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f1fb015cfb0 con 0x7f1fb00b3240 2023-10-11T09:43:50.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.442+0000 7f1faf59e700 1 -- 172.21.15.78:0/611636570 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1fb015cfb0 con 0x7f1fb00b3240 2023-10-11T09:43:50.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.442+0000 7f1fa77fe700 1 -- 172.21.15.78:0/611636570 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v48) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1fb015cfb0 con 0x7f1fb00b3240 2023-10-11T09:43:50.447 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:43:50.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.446+0000 7f1faf59e700 1 -- 172.21.15.78:0/611636570 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1f94065ed0 msgr2=0x7f1f94068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:50.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.446+0000 7f1faf59e700 1 --2- 172.21.15.78:0/611636570 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1f94065ed0 0x7f1f94068380 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1fb015fe90 tx=0x7f1f98009240 comp rx=0 tx=0).stop 2023-10-11T09:43:50.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.446+0000 7f1faf59e700 1 -- 172.21.15.78:0/611636570 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fb00b3240 msgr2=0x7f1fb01f2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:50.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.446+0000 7f1faf59e700 1 --2- 172.21.15.78:0/611636570 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fb00b3240 0x7f1fb01f2100 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000c4d0 tx=0x7f1fa000c7e0 comp rx=0 tx=0).stop 2023-10-11T09:43:50.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.446+0000 7f1faf59e700 1 -- 172.21.15.78:0/611636570 shutdown_connections 2023-10-11T09:43:50.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.446+0000 7f1faf59e700 1 --2- 172.21.15.78:0/611636570 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1f94065ed0 0x7f1f94068380 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:50.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.446+0000 7f1faf59e700 1 --2- 172.21.15.78:0/611636570 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1fb00b3a90 0x7f1fb015e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:50.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.446+0000 7f1faf59e700 1 --2- 172.21.15.78:0/611636570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fb00b3610 0x7f1fb00c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:50.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.446+0000 7f1faf59e700 1 --2- 172.21.15.78:0/611636570 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fb00b3240 0x7f1fb01f2100 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:50.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.446+0000 7f1faf59e700 1 -- 172.21.15.78:0/611636570 >> 172.21.15.78:0/611636570 conn(0x7f1fb009c260 msgr2=0x7f1fb00afb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:50.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.446+0000 7f1faf59e700 1 -- 172.21.15.78:0/611636570 shutdown_connections 2023-10-11T09:43:50.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.446+0000 7f1faf59e700 1 -- 172.21.15.78:0/611636570 wait complete. 2023-10-11T09:43:50.471 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T09:43:50.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.642+0000 7fcaf13b7700 1 Processor -- start 2023-10-11T09:43:50.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.642+0000 7fcaf13b7700 1 -- start start 2023-10-11T09:43:50.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.642+0000 7fcaf13b7700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcaec0b3240 0x7fcaec0d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:50.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.642+0000 7fcaf13b7700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fcaec0b3a90 0x7fcaec0b85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:50.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.642+0000 7fcaf13b7700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaec0b47f0 con 0x7fcaec0b3610 2023-10-11T09:43:50.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.642+0000 7fcaf13b7700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaec0b8130 con 0x7fcaec0b3a90 2023-10-11T09:43:50.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.642+0000 7fcaf13b7700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaec0b41e0 con 0x7fcaec0b3240 2023-10-11T09:43:50.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.646+0000 7fcaf0bb6700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fcaec0b3a90 0x7fcaec0b85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:50.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.646+0000 7fcae37fe700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fcaec0b3610 0x7fcaec15d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:42200/0 (socket says 172.21.15.78:42200) 2023-10-11T09:43:50.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.646+0000 7fcae37fe700 1 -- 172.21.15.78:0/927025605 learned_addr learned my addr 172.21.15.78:0/927025605 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:43:50.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.646+0000 7fcaf0bb6700 1 --2- 172.21.15.78:0/927025605 >> v2:172.21.15.78:3300/0 conn(0x7fcaec0b3a90 0x7fcaec0b85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50242/0 (socket says 172.21.15.78:50242) 2023-10-11T09:43:50.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.646+0000 7fcaebfff700 1 --2- 172.21.15.78:0/927025605 >> v2:172.21.15.133:3300/0 conn(0x7fcaec0b3240 0x7fcaec0d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:50.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.646+0000 7fcaf0bb6700 1 -- 172.21.15.78:0/927025605 >> v2:172.21.15.133:3300/0 conn(0x7fcaec0b3240 msgr2=0x7fcaec0d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:50.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.646+0000 7fcaf0bb6700 1 --2- 172.21.15.78:0/927025605 >> v2:172.21.15.133:3300/0 conn(0x7fcaec0b3240 0x7fcaec0d29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:50.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.646+0000 7fcaf0bb6700 1 -- 172.21.15.78:0/927025605 >> v1:172.21.15.78:6789/0 conn(0x7fcaec0b3610 legacy=0x7fcaec15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:50.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.646+0000 7fcaf0bb6700 1 -- 172.21.15.78:0/927025605 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaec0b7800 con 0x7fcaec0b3a90 2023-10-11T09:43:50.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.646+0000 7fcaf0bb6700 1 --2- 172.21.15.78:0/927025605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaec0b3a90 0x7fcaec0b85c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fcaec06aa40 tx=0x7fcae400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe81b57af1603dc0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:50.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.646+0000 7fcaeb7fe700 1 -- 172.21.15.78:0/927025605 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae400e070 con 0x7fcaec0b3a90 2023-10-11T09:43:50.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.646+0000 7fcaeb7fe700 1 -- 172.21.15.78:0/927025605 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae400bb20 con 0x7fcaec0b3a90 2023-10-11T09:43:50.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.646+0000 7fcaeb7fe700 1 -- 172.21.15.78:0/927025605 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae4012430 con 0x7fcaec0b3a90 2023-10-11T09:43:50.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.646+0000 7fcaf13b7700 1 -- 172.21.15.78:0/927025605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaec0b3a90 msgr2=0x7fcaec0b85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:50.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.646+0000 7fcaf13b7700 1 --2- 172.21.15.78:0/927025605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaec0b3a90 0x7fcaec0b85c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fcaec06aa40 tx=0x7fcae400b330 comp rx=0 tx=0).stop 2023-10-11T09:43:50.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.646+0000 7fcaf13b7700 1 -- 172.21.15.78:0/927025605 shutdown_connections 2023-10-11T09:43:50.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.646+0000 7fcaf13b7700 1 --2- 172.21.15.78:0/927025605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaec0b3a90 0x7fcaec0b85c0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:50.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.646+0000 7fcaf13b7700 1 --2- 172.21.15.78:0/927025605 >> v2:172.21.15.133:3300/0 conn(0x7fcaec0b3240 0x7fcaec0d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:50.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.646+0000 7fcaf13b7700 1 -- 172.21.15.78:0/927025605 >> 172.21.15.78:0/927025605 conn(0x7fcaec09c260 msgr2=0x7fcaec0d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:50.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.646+0000 7fcaf13b7700 1 -- 172.21.15.78:0/927025605 shutdown_connections 2023-10-11T09:43:50.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.646+0000 7fcaf13b7700 1 -- 172.21.15.78:0/927025605 wait complete. 2023-10-11T09:43:50.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.650+0000 7fcaf13b7700 1 Processor -- start 2023-10-11T09:43:50.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.650+0000 7fcaf13b7700 1 -- start start 2023-10-11T09:43:50.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.650+0000 7fcaf13b7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaec0b3240 0x7fcaec15f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:50.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.650+0000 7fcaf13b7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcaec0b3610 0x7fcaec15ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:50.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.650+0000 7fcaf13b7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcaec0b3a90 0x7fcaec14b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:50.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.650+0000 7fcaf13b7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaec15fda0 con 0x7fcaec0b3240 2023-10-11T09:43:50.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.650+0000 7fcaf13b7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaec14bbd0 con 0x7fcaec0b3a90 2023-10-11T09:43:50.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.650+0000 7fcaf13b7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaec14beb0 con 0x7fcaec0b3610 2023-10-11T09:43:50.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.650+0000 7fcaebfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaec0b3240 0x7fcaec15f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:50.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.654+0000 7fcaebfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaec0b3240 0x7fcaec15f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50244/0 (socket says 172.21.15.78:50244) 2023-10-11T09:43:50.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.654+0000 7fcaebfff700 1 -- 172.21.15.78:0/220240700 learned_addr learned my addr 172.21.15.78:0/220240700 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:50.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.654+0000 7fcaf0bb6700 1 --2- 172.21.15.78:0/220240700 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcaec0b3a90 0x7fcaec14b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:50.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.654+0000 7fcae37fe700 1 --2- 172.21.15.78:0/220240700 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcaec0b3610 0x7fcaec15ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:50.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.654+0000 7fcaebfff700 1 -- 172.21.15.78:0/220240700 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcaec0b3610 msgr2=0x7fcaec15ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:50.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.654+0000 7fcaebfff700 1 --2- 172.21.15.78:0/220240700 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcaec0b3610 0x7fcaec15ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:50.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.654+0000 7fcaebfff700 1 -- 172.21.15.78:0/220240700 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcaec0b3a90 msgr2=0x7fcaec14b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:50.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.654+0000 7fcaebfff700 1 --2- 172.21.15.78:0/220240700 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcaec0b3a90 0x7fcaec14b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:50.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.654+0000 7fcaebfff700 1 -- 172.21.15.78:0/220240700 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae4009a70 con 0x7fcaec0b3240 2023-10-11T09:43:50.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.654+0000 7fcaebfff700 1 --2- 172.21.15.78:0/220240700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaec0b3240 0x7fcaec15f7c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fcadc00e840 tx=0x7fcadc00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:50.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.654+0000 7fcae97fa700 1 -- 172.21.15.78:0/220240700 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcadc0095a0 con 0x7fcaec0b3240 2023-10-11T09:43:50.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.654+0000 7fcae97fa700 1 -- 172.21.15.78:0/220240700 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcadc013680 con 0x7fcaec0b3240 2023-10-11T09:43:50.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.654+0000 7fcaf13b7700 1 -- 172.21.15.78:0/220240700 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaec14c190 con 0x7fcaec0b3240 2023-10-11T09:43:50.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.654+0000 7fcae97fa700 1 -- 172.21.15.78:0/220240700 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcadc0139f0 con 0x7fcaec0b3240 2023-10-11T09:43:50.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.654+0000 7fcaf13b7700 1 -- 172.21.15.78:0/220240700 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaec14b7c0 con 0x7fcaec0b3240 2023-10-11T09:43:50.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.654+0000 7fcaf13b7700 1 -- 172.21.15.78:0/220240700 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaec065d40 con 0x7fcaec0b3240 2023-10-11T09:43:50.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.658+0000 7fcae97fa700 1 -- 172.21.15.78:0/220240700 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fcadc013b50 con 0x7fcaec0b3240 2023-10-11T09:43:50.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.658+0000 7fcae97fa700 1 --2- 172.21.15.78:0/220240700 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fcacc065ed0 0x7fcacc068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:50.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.658+0000 7fcae97fa700 1 -- 172.21.15.78:0/220240700 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6753+0+0 (secure 0 0 0) 0x7fcadc083990 con 0x7fcaec0b3240 2023-10-11T09:43:50.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.658+0000 7fcae37fe700 1 --2- 172.21.15.78:0/220240700 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fcacc065ed0 0x7fcacc068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:50.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.662+0000 7fcae37fe700 1 --2- 172.21.15.78:0/220240700 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fcacc065ed0 0x7fcacc068380 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0a0ea0 tx=0x7fcad4007f30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:50.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.666+0000 7fcae97fa700 1 -- 172.21.15.78:0/220240700 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fcadc050c80 con 0x7fcaec0b3240 2023-10-11T09:43:50.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.838+0000 7fcaf13b7700 1 -- 172.21.15.78:0/220240700 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcaec0d1d10 con 0x7fcaec0b3240 2023-10-11T09:43:50.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.838+0000 7fcae97fa700 1 -- 172.21.15.78:0/220240700 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v48) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcadc0542a0 con 0x7fcaec0b3240 2023-10-11T09:43:50.843 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:43:50.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.842+0000 7fcaf13b7700 1 -- 172.21.15.78:0/220240700 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fcacc065ed0 msgr2=0x7fcacc068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:50.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.842+0000 7fcaf13b7700 1 --2- 172.21.15.78:0/220240700 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fcacc065ed0 0x7fcacc068380 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0a0ea0 tx=0x7fcad4007f30 comp rx=0 tx=0).stop 2023-10-11T09:43:50.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.842+0000 7fcaf13b7700 1 -- 172.21.15.78:0/220240700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaec0b3240 msgr2=0x7fcaec15f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:50.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.842+0000 7fcaf13b7700 1 --2- 172.21.15.78:0/220240700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaec0b3240 0x7fcaec15f7c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fcadc00e840 tx=0x7fcadc00eb50 comp rx=0 tx=0).stop 2023-10-11T09:43:50.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.842+0000 7fcaf13b7700 1 -- 172.21.15.78:0/220240700 shutdown_connections 2023-10-11T09:43:50.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.842+0000 7fcaf13b7700 1 --2- 172.21.15.78:0/220240700 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fcacc065ed0 0x7fcacc068380 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:50.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.842+0000 7fcaf13b7700 1 --2- 172.21.15.78:0/220240700 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcaec0b3a90 0x7fcaec14b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:50.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.842+0000 7fcaf13b7700 1 --2- 172.21.15.78:0/220240700 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcaec0b3610 0x7fcaec15ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:50.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.842+0000 7fcaf13b7700 1 --2- 172.21.15.78:0/220240700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaec0b3240 0x7fcaec15f7c0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:50.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.842+0000 7fcaf13b7700 1 -- 172.21.15.78:0/220240700 >> 172.21.15.78:0/220240700 conn(0x7fcaec09c260 msgr2=0x7fcaec15c340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:50.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.842+0000 7fcaf13b7700 1 -- 172.21.15.78:0/220240700 shutdown_connections 2023-10-11T09:43:50.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:50.842+0000 7fcaf13b7700 1 -- 172.21.15.78:0/220240700 wait complete. 2023-10-11T09:43:50.865 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T09:43:51.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.038+0000 7fd7703a7700 1 Processor -- start 2023-10-11T09:43:51.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.038+0000 7fd7703a7700 1 -- start start 2023-10-11T09:43:51.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.042+0000 7fd7703a7700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fd76806d340 0x7fd7680ca880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:51.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.042+0000 7fd7703a7700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd7680cadc0 0x7fd76815afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:51.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.042+0000 7fd7703a7700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd76815e280 con 0x7fd768158430 2023-10-11T09:43:51.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.042+0000 7fd7703a7700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd768070c10 con 0x7fd76806d340 2023-10-11T09:43:51.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.042+0000 7fd7703a7700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7680cb240 con 0x7fd7680cadc0 2023-10-11T09:43:51.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.042+0000 7fd76d942700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fd76806d340 0x7fd7680ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:51.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.042+0000 7fd76d942700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fd76806d340 0x7fd7680ca880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50254/0 (socket says 172.21.15.78:50254) 2023-10-11T09:43:51.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.042+0000 7fd76d942700 1 -- 172.21.15.78:0/3576144612 learned_addr learned my addr 172.21.15.78:0/3576144612 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:51.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.042+0000 7fd76e944700 1 --2- 172.21.15.78:0/3576144612 >> v2:172.21.15.133:3300/0 conn(0x7fd7680cadc0 0x7fd76815afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:51.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.042+0000 7fd76d942700 1 -- 172.21.15.78:0/3576144612 >> v2:172.21.15.133:3300/0 conn(0x7fd7680cadc0 msgr2=0x7fd76815afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:51.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.042+0000 7fd76d942700 1 --2- 172.21.15.78:0/3576144612 >> v2:172.21.15.133:3300/0 conn(0x7fd7680cadc0 0x7fd76815afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:51.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.042+0000 7fd76d942700 1 -- 172.21.15.78:0/3576144612 >> v1:172.21.15.133:6789/0 conn(0x7fd768158430 legacy=0x7fd7680710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:51.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.042+0000 7fd76d942700 1 -- 172.21.15.78:0/3576144612 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd76815e560 con 0x7fd76806d340 2023-10-11T09:43:51.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.042+0000 7fd76d942700 1 --2- 172.21.15.78:0/3576144612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd76806d340 0x7fd7680ca880 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd76815acb0 tx=0x7fd758009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c10aa2be403b19 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:51.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.042+0000 7fd76d141700 1 -- 172.21.15.78:0/3576144612 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd758011830 con 0x7fd76806d340 2023-10-11T09:43:51.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.042+0000 7fd76d141700 1 -- 172.21.15.78:0/3576144612 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd758011990 con 0x7fd76806d340 2023-10-11T09:43:51.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.042+0000 7fd76d141700 1 -- 172.21.15.78:0/3576144612 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd758011d20 con 0x7fd76806d340 2023-10-11T09:43:51.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.042+0000 7fd7703a7700 1 -- 172.21.15.78:0/3576144612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd76806d340 msgr2=0x7fd7680ca880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:51.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.042+0000 7fd7703a7700 1 --2- 172.21.15.78:0/3576144612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd76806d340 0x7fd7680ca880 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd76815acb0 tx=0x7fd758009bf0 comp rx=0 tx=0).stop 2023-10-11T09:43:51.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.046+0000 7fd7703a7700 1 -- 172.21.15.78:0/3576144612 shutdown_connections 2023-10-11T09:43:51.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.046+0000 7fd7703a7700 1 --2- 172.21.15.78:0/3576144612 >> v2:172.21.15.133:3300/0 conn(0x7fd7680cadc0 0x7fd76815afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:51.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.046+0000 7fd7703a7700 1 --2- 172.21.15.78:0/3576144612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd76806d340 0x7fd7680ca880 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:51.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.046+0000 7fd7703a7700 1 -- 172.21.15.78:0/3576144612 >> 172.21.15.78:0/3576144612 conn(0x7fd7680cecc0 msgr2=0x7fd7680cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:51.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.046+0000 7fd7703a7700 1 -- 172.21.15.78:0/3576144612 shutdown_connections 2023-10-11T09:43:51.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.046+0000 7fd7703a7700 1 -- 172.21.15.78:0/3576144612 wait complete. 2023-10-11T09:43:51.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.046+0000 7fd7703a7700 1 Processor -- start 2023-10-11T09:43:51.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.046+0000 7fd7703a7700 1 -- start start 2023-10-11T09:43:51.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.046+0000 7fd7703a7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd76806d340 0x7fd7680b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:51.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.046+0000 7fd7703a7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7680cadc0 0x7fd7680b1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:51.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.046+0000 7fd7703a7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd768158430 0x7fd7681569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:51.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.046+0000 7fd7703a7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7680b21d0 con 0x7fd768158430 2023-10-11T09:43:51.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.046+0000 7fd7703a7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76815afe0 con 0x7fd7680cadc0 2023-10-11T09:43:51.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.046+0000 7fd7703a7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7680b2400 con 0x7fd76806d340 2023-10-11T09:43:51.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.050+0000 7fd76e944700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd768158430 0x7fd7681569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:51.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.050+0000 7fd76e944700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd768158430 0x7fd7681569d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50266/0 (socket says 172.21.15.78:50266) 2023-10-11T09:43:51.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.050+0000 7fd76e944700 1 -- 172.21.15.78:0/1453002327 learned_addr learned my addr 172.21.15.78:0/1453002327 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:51.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.050+0000 7fd76d942700 1 --2- 172.21.15.78:0/1453002327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7680cadc0 0x7fd7680b1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:51.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.050+0000 7fd76e143700 1 --2- 172.21.15.78:0/1453002327 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd76806d340 0x7fd7680b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:51.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.050+0000 7fd76e944700 1 -- 172.21.15.78:0/1453002327 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd76806d340 msgr2=0x7fd7680b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:51.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.050+0000 7fd76e944700 1 --2- 172.21.15.78:0/1453002327 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd76806d340 0x7fd7680b3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:51.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.050+0000 7fd76e944700 1 -- 172.21.15.78:0/1453002327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7680cadc0 msgr2=0x7fd7680b1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:51.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.050+0000 7fd76e944700 1 --2- 172.21.15.78:0/1453002327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7680cadc0 0x7fd7680b1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:51.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.050+0000 7fd76e944700 1 -- 172.21.15.78:0/1453002327 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd758009570 con 0x7fd768158430 2023-10-11T09:43:51.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.050+0000 7fd76e143700 1 --2- 172.21.15.78:0/1453002327 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd76806d340 0x7fd7680b3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:43:51.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.050+0000 7fd76e944700 1 --2- 172.21.15.78:0/1453002327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd768158430 0x7fd7681569d0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd76000efc0 tx=0x7fd76000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:51.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.050+0000 7fd75effd700 1 -- 172.21.15.78:0/1453002327 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76000cd40 con 0x7fd768158430 2023-10-11T09:43:51.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.050+0000 7fd75effd700 1 -- 172.21.15.78:0/1453002327 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd76000f040 con 0x7fd768158430 2023-10-11T09:43:51.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.050+0000 7fd7703a7700 1 -- 172.21.15.78:0/1453002327 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7680b26e0 con 0x7fd768158430 2023-10-11T09:43:51.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.050+0000 7fd75effd700 1 -- 172.21.15.78:0/1453002327 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7600146c0 con 0x7fd768158430 2023-10-11T09:43:51.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.050+0000 7fd7703a7700 1 -- 172.21.15.78:0/1453002327 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd76815b230 con 0x7fd768158430 2023-10-11T09:43:51.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.050+0000 7fd7703a7700 1 -- 172.21.15.78:0/1453002327 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd76806eb80 con 0x7fd768158430 2023-10-11T09:43:51.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.054+0000 7fd75effd700 1 -- 172.21.15.78:0/1453002327 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fd760014820 con 0x7fd768158430 2023-10-11T09:43:51.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.054+0000 7fd75effd700 1 --2- 172.21.15.78:0/1453002327 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd74c065ed0 0x7fd74c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:51.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.054+0000 7fd75effd700 1 -- 172.21.15.78:0/1453002327 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6753+0+0 (secure 0 0 0) 0x7fd7600843e0 con 0x7fd768158430 2023-10-11T09:43:51.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.054+0000 7fd76e143700 1 --2- 172.21.15.78:0/1453002327 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd74c065ed0 0x7fd74c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:51.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.054+0000 7fd76e143700 1 --2- 172.21.15.78:0/1453002327 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd74c065ed0 0x7fd74c068380 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd764000f60 tx=0x7fd764009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:51.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.062+0000 7fd75effd700 1 -- 172.21.15.78:0/1453002327 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fd760051760 con 0x7fd768158430 2023-10-11T09:43:51.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.234+0000 7fd7703a7700 1 -- 172.21.15.78:0/1453002327 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd768065db0 con 0x7fd768158430 2023-10-11T09:43:51.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.238+0000 7fd75effd700 1 -- 172.21.15.78:0/1453002327 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v48) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd76000cea0 con 0x7fd768158430 2023-10-11T09:43:51.244 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2023-10-11T09:43:51.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.242+0000 7fd7703a7700 1 -- 172.21.15.78:0/1453002327 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd74c065ed0 msgr2=0x7fd74c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:51.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.242+0000 7fd7703a7700 1 --2- 172.21.15.78:0/1453002327 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd74c065ed0 0x7fd74c068380 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd764000f60 tx=0x7fd764009040 comp rx=0 tx=0).stop 2023-10-11T09:43:51.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.242+0000 7fd7703a7700 1 -- 172.21.15.78:0/1453002327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd768158430 msgr2=0x7fd7681569d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:51.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.242+0000 7fd7703a7700 1 --2- 172.21.15.78:0/1453002327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd768158430 0x7fd7681569d0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd76000efc0 tx=0x7fd76000c3a0 comp rx=0 tx=0).stop 2023-10-11T09:43:51.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.242+0000 7fd7703a7700 1 -- 172.21.15.78:0/1453002327 shutdown_connections 2023-10-11T09:43:51.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.242+0000 7fd7703a7700 1 --2- 172.21.15.78:0/1453002327 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd74c065ed0 0x7fd74c068380 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:51.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.242+0000 7fd7703a7700 1 --2- 172.21.15.78:0/1453002327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd768158430 0x7fd7681569d0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:51.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.242+0000 7fd7703a7700 1 --2- 172.21.15.78:0/1453002327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7680cadc0 0x7fd7680b1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:51.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.242+0000 7fd7703a7700 1 --2- 172.21.15.78:0/1453002327 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd76806d340 0x7fd7680b3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:51.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.242+0000 7fd7703a7700 1 -- 172.21.15.78:0/1453002327 >> 172.21.15.78:0/1453002327 conn(0x7fd7680cecc0 msgr2=0x7fd768066160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:51.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.242+0000 7fd7703a7700 1 -- 172.21.15.78:0/1453002327 shutdown_connections 2023-10-11T09:43:51.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.242+0000 7fd7703a7700 1 -- 172.21.15.78:0/1453002327 wait complete. 2023-10-11T09:43:51.267 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-10-11T09:43:51.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.434+0000 7f2358d01700 1 Processor -- start 2023-10-11T09:43:51.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.434+0000 7f2358d01700 1 -- start start 2023-10-11T09:43:51.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.434+0000 7f2358d01700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23540caf90 0x7f235415afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:51.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.434+0000 7f2358d01700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f235415e280 con 0x7f2354158430 2023-10-11T09:43:51.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.434+0000 7f2358d01700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f235415e560 con 0x7f235406d340 2023-10-11T09:43:51.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.434+0000 7f2358d01700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2354070c10 con 0x7f23540caf90 2023-10-11T09:43:51.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.434+0000 7f235259c700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f2354158430 0x7f23540710d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:42216/0 (socket says 172.21.15.78:42216) 2023-10-11T09:43:51.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.434+0000 7f235259c700 1 -- 172.21.15.78:0/4048677277 learned_addr learned my addr 172.21.15.78:0/4048677277 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:43:51.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.434+0000 7f2352d9d700 1 --2- 172.21.15.78:0/4048677277 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23540caf90 0x7f235415afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:51.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.434+0000 7f2351d9b700 1 -- 172.21.15.78:0/4048677277 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 799534329 0 0) 0x7f235415e280 con 0x7f2354158430 2023-10-11T09:43:51.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.438+0000 7f2351d9b700 1 -- 172.21.15.78:0/4048677277 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23400035d0 con 0x7f2354158430 2023-10-11T09:43:51.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.438+0000 7f2351d9b700 1 -- 172.21.15.78:0/4048677277 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3351193719 0 0) 0x7f235415e560 con 0x7f235406d340 2023-10-11T09:43:51.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.438+0000 7f2351d9b700 1 -- 172.21.15.78:0/4048677277 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f235415e280 con 0x7f235406d340 2023-10-11T09:43:51.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.438+0000 7f2351d9b700 1 -- 172.21.15.78:0/4048677277 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2737474088 0 0) 0x7f23400035d0 con 0x7f2354158430 2023-10-11T09:43:51.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.438+0000 7f2351d9b700 1 -- 172.21.15.78:0/4048677277 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23540caf90 msgr2=0x7f235415afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:51.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.438+0000 7f2351d9b700 1 --2- 172.21.15.78:0/4048677277 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23540caf90 0x7f235415afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:51.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.438+0000 7f2351d9b700 1 -- 172.21.15.78:0/4048677277 >> v1:172.21.15.133:6789/0 conn(0x7f235406d340 legacy=0x7f23540ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:51.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.438+0000 7f2351d9b700 1 -- 172.21.15.78:0/4048677277 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235415e840 con 0x7f2354158430 2023-10-11T09:43:51.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.438+0000 7f2351d9b700 1 -- 172.21.15.78:0/4048677277 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f2344001670 con 0x7f2354158430 2023-10-11T09:43:51.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.438+0000 7f2351d9b700 1 -- 172.21.15.78:0/4048677277 >> v1:172.21.15.78:6789/0 conn(0x7f2354158430 legacy=0x7f23540710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:51.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.438+0000 7f2351d9b700 1 --2- 172.21.15.78:0/4048677277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2340003be0 0x7f2340003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:51.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.438+0000 7f2351d9b700 1 --2- 172.21.15.78:0/4048677277 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23540caf90 0x7f2340004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:51.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.438+0000 7f2351d9b700 1 --2- 172.21.15.78:0/4048677277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2340004db0 0x7f2340005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:51.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.438+0000 7f2351d9b700 1 -- 172.21.15.78:0/4048677277 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234000bae0 con 0x7f2340004db0 2023-10-11T09:43:51.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.438+0000 7f2351d9b700 1 -- 172.21.15.78:0/4048677277 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234000bc50 con 0x7f2340003be0 2023-10-11T09:43:51.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.438+0000 7f2351d9b700 1 -- 172.21.15.78:0/4048677277 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f234000be10 con 0x7f23540caf90 2023-10-11T09:43:51.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.438+0000 7f2352d9d700 1 --2- 172.21.15.78:0/4048677277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2340004db0 0x7f2340005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:51.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.438+0000 7f235259c700 1 --2- 172.21.15.78:0/4048677277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2340003be0 0x7f2340003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:51.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.438+0000 7f2349d9b700 1 --2- 172.21.15.78:0/4048677277 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23540caf90 0x7f2340004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:51.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.438+0000 7f2352d9d700 1 -- 172.21.15.78:0/4048677277 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23540caf90 msgr2=0x7f2340004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:51.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.438+0000 7f2352d9d700 1 --2- 172.21.15.78:0/4048677277 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23540caf90 0x7f2340004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:51.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.438+0000 7f2352d9d700 1 -- 172.21.15.78:0/4048677277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2340003be0 msgr2=0x7f2340003fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:51.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.438+0000 7f2352d9d700 1 --2- 172.21.15.78:0/4048677277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2340003be0 0x7f2340003fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:51.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.438+0000 7f2352d9d700 1 -- 172.21.15.78:0/4048677277 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23400035d0 con 0x7f2340004db0 2023-10-11T09:43:51.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.438+0000 7f2352d9d700 1 --2- 172.21.15.78:0/4048677277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2340004db0 0x7f2340005220 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f234c009f90 tx=0x7f23400058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab9c341809d21428 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:51.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.438+0000 7f2351d9b700 1 -- 172.21.15.78:0/4048677277 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234c00ec30 con 0x7f2340004db0 2023-10-11T09:43:51.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.438+0000 7f2351d9b700 1 -- 172.21.15.78:0/4048677277 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f234c00f040 con 0x7f2340004db0 2023-10-11T09:43:51.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.438+0000 7f2351d9b700 1 -- 172.21.15.78:0/4048677277 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234c014480 con 0x7f2340004db0 2023-10-11T09:43:51.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.438+0000 7f2358d01700 1 -- 172.21.15.78:0/4048677277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2340004db0 msgr2=0x7f2340005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:51.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.438+0000 7f2358d01700 1 --2- 172.21.15.78:0/4048677277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2340004db0 0x7f2340005220 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f234c009f90 tx=0x7f23400058a0 comp rx=0 tx=0).stop 2023-10-11T09:43:51.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.438+0000 7f2352d9d700 1 -- 172.21.15.78:0/4048677277 reap_dead start 2023-10-11T09:43:51.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.442+0000 7f2358d01700 1 -- 172.21.15.78:0/4048677277 shutdown_connections 2023-10-11T09:43:51.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.442+0000 7f2358d01700 1 -- 172.21.15.78:0/4048677277 >> 172.21.15.78:0/4048677277 conn(0x7f23540cecc0 msgr2=0x7f23540cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:51.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.442+0000 7f2358d01700 1 -- 172.21.15.78:0/4048677277 shutdown_connections 2023-10-11T09:43:51.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.442+0000 7f2358d01700 1 -- 172.21.15.78:0/4048677277 wait complete. 2023-10-11T09:43:51.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.442+0000 7f2358d01700 1 Processor -- start 2023-10-11T09:43:51.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.442+0000 7f2358d01700 1 -- start start 2023-10-11T09:43:51.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.446+0000 7f2358d01700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2340004db0 0x7f2354149700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:51.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.446+0000 7f2358d01700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23540b1d40 0x7f23540b3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:51.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.446+0000 7f2358d01700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23540b3cc0 0x7f2354158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:51.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.446+0000 7f2358d01700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235415b080 con 0x7f23540b3cc0 2023-10-11T09:43:51.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.446+0000 7f2358d01700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235415b2f0 con 0x7f23540b1d40 2023-10-11T09:43:51.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.446+0000 7f2358d01700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f235415b5d0 con 0x7f2340004db0 2023-10-11T09:43:51.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.446+0000 7f2352d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23540b3cc0 0x7f2354158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:51.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.446+0000 7f2352d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23540b3cc0 0x7f2354158920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50286/0 (socket says 172.21.15.78:50286) 2023-10-11T09:43:51.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.446+0000 7f2352d9d700 1 -- 172.21.15.78:0/2280751193 learned_addr learned my addr 172.21.15.78:0/2280751193 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:51.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.446+0000 7f235259c700 1 --2- 172.21.15.78:0/2280751193 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2340004db0 0x7f2354149700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:51.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.446+0000 7f2349d9b700 1 --2- 172.21.15.78:0/2280751193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23540b1d40 0x7f23540b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:51.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.446+0000 7f2352d9d700 1 -- 172.21.15.78:0/2280751193 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2340004db0 msgr2=0x7f2354149700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:51.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.446+0000 7f2352d9d700 1 --2- 172.21.15.78:0/2280751193 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2340004db0 0x7f2354149700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:51.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.446+0000 7f2352d9d700 1 -- 172.21.15.78:0/2280751193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23540b1d40 msgr2=0x7f23540b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:51.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.446+0000 7f2352d9d700 1 --2- 172.21.15.78:0/2280751193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23540b1d40 0x7f23540b3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:51.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.446+0000 7f2352d9d700 1 -- 172.21.15.78:0/2280751193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235415b8b0 con 0x7f23540b3cc0 2023-10-11T09:43:51.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.446+0000 7f2352d9d700 1 --2- 172.21.15.78:0/2280751193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23540b3cc0 0x7f2354158920 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f234c016000 tx=0x7f234c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:51.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.446+0000 7f234b7fe700 1 -- 172.21.15.78:0/2280751193 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234c009410 con 0x7f23540b3cc0 2023-10-11T09:43:51.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.446+0000 7f234b7fe700 1 -- 172.21.15.78:0/2280751193 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f234c00ee40 con 0x7f23540b3cc0 2023-10-11T09:43:51.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.446+0000 7f2358d01700 1 -- 172.21.15.78:0/2280751193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f235415bb90 con 0x7f23540b3cc0 2023-10-11T09:43:51.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.446+0000 7f234b7fe700 1 -- 172.21.15.78:0/2280751193 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234c014e50 con 0x7f23540b3cc0 2023-10-11T09:43:51.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.446+0000 7f2358d01700 1 -- 172.21.15.78:0/2280751193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f235415c0b0 con 0x7f23540b3cc0 2023-10-11T09:43:51.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.450+0000 7f2358d01700 1 -- 172.21.15.78:0/2280751193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f235406eb80 con 0x7f23540b3cc0 2023-10-11T09:43:51.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.450+0000 7f234b7fe700 1 -- 172.21.15.78:0/2280751193 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f234c023020 con 0x7f23540b3cc0 2023-10-11T09:43:51.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.454+0000 7f234b7fe700 1 --2- 172.21.15.78:0/2280751193 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2334065f00 0x7f23340683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:51.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.454+0000 7f234b7fe700 1 -- 172.21.15.78:0/2280751193 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6753+0+0 (secure 0 0 0) 0x7f234c084be0 con 0x7f23540b3cc0 2023-10-11T09:43:51.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.454+0000 7f235259c700 1 --2- 172.21.15.78:0/2280751193 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2334065f00 0x7f23340683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:51.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.454+0000 7f235259c700 1 --2- 172.21.15.78:0/2280751193 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2334065f00 0x7f23340683b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2344001050 tx=0x7f2344000f80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:51.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.458+0000 7f234b7fe700 1 -- 172.21.15.78:0/2280751193 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f234c052000 con 0x7f23540b3cc0 2023-10-11T09:43:51.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.634+0000 7f2358d01700 1 -- 172.21.15.78:0/2280751193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7f2354065db0 con 0x7f23540b3cc0 2023-10-11T09:43:51.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.634+0000 7f234b7fe700 1 -- 172.21.15.78:0/2280751193 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v48) v1 ==== 106+0+11 (secure 0 0 0) 0x7f234c02b020 con 0x7f23540b3cc0 2023-10-11T09:43:51.641 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:43:51.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.638+0000 7f2358d01700 1 -- 172.21.15.78:0/2280751193 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2334065f00 msgr2=0x7f23340683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:51.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.638+0000 7f2358d01700 1 --2- 172.21.15.78:0/2280751193 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2334065f00 0x7f23340683b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2344001050 tx=0x7f2344000f80 comp rx=0 tx=0).stop 2023-10-11T09:43:51.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.638+0000 7f2358d01700 1 -- 172.21.15.78:0/2280751193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23540b3cc0 msgr2=0x7f2354158920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:51.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.638+0000 7f2358d01700 1 --2- 172.21.15.78:0/2280751193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23540b3cc0 0x7f2354158920 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f234c016000 tx=0x7f234c009750 comp rx=0 tx=0).stop 2023-10-11T09:43:51.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.638+0000 7f2358d01700 1 -- 172.21.15.78:0/2280751193 shutdown_connections 2023-10-11T09:43:51.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.638+0000 7f2358d01700 1 --2- 172.21.15.78:0/2280751193 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2334065f00 0x7f23340683b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:51.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.638+0000 7f2358d01700 1 --2- 172.21.15.78:0/2280751193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23540b3cc0 0x7f2354158920 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:51.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.638+0000 7f2358d01700 1 --2- 172.21.15.78:0/2280751193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23540b1d40 0x7f23540b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:51.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.638+0000 7f2358d01700 1 --2- 172.21.15.78:0/2280751193 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2340004db0 0x7f2354149700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:51.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.638+0000 7f2358d01700 1 -- 172.21.15.78:0/2280751193 >> 172.21.15.78:0/2280751193 conn(0x7f23540cecc0 msgr2=0x7f2354156000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:51.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.638+0000 7f2358d01700 1 -- 172.21.15.78:0/2280751193 shutdown_connections 2023-10-11T09:43:51.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.638+0000 7f2358d01700 1 -- 172.21.15.78:0/2280751193 wait complete. 2023-10-11T09:43:51.665 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:43:51.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.822+0000 7fe10c991700 1 Processor -- start 2023-10-11T09:43:51.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.822+0000 7fe10c991700 1 -- start start 2023-10-11T09:43:51.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.826+0000 7fe10c991700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe108164d20 0x7fe10806bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:51.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.826+0000 7fe10c991700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe108165390 con 0x7fe1080caf20 2023-10-11T09:43:51.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.826+0000 7fe10c991700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1080ca440 con 0x7fe1080cda10 2023-10-11T09:43:51.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.826+0000 7fe10c991700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1080cb3a0 con 0x7fe108164d20 2023-10-11T09:43:51.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.826+0000 7fe10659c700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fe1080caf20 0x7fe108164610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:42224/0 (socket says 172.21.15.78:42224) 2023-10-11T09:43:51.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.826+0000 7fe10659c700 1 -- 172.21.15.78:0/2230534156 learned_addr learned my addr 172.21.15.78:0/2230534156 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:43:51.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.826+0000 7fe10759e700 1 --2- 172.21.15.78:0/2230534156 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe108164d20 0x7fe10806bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:51.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.826+0000 7fe105d9b700 1 -- 172.21.15.78:0/2230534156 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2591898360 0 0) 0x7fe108165390 con 0x7fe1080caf20 2023-10-11T09:43:51.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.826+0000 7fe105d9b700 1 -- 172.21.15.78:0/2230534156 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0f00035d0 con 0x7fe1080caf20 2023-10-11T09:43:51.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.826+0000 7fe10759e700 1 -- 172.21.15.78:0/2230534156 >> v1:172.21.15.133:6789/0 conn(0x7fe1080cda10 legacy=0x7fe1080cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:51.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.826+0000 7fe105d9b700 1 -- 172.21.15.78:0/2230534156 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3974767333 0 0) 0x7fe1080ca440 con 0x7fe1080cda10 2023-10-11T09:43:51.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.826+0000 7fe10759e700 1 -- 172.21.15.78:0/2230534156 >> v1:172.21.15.78:6789/0 conn(0x7fe1080caf20 legacy=0x7fe108164610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:51.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.826+0000 7fe10759e700 1 -- 172.21.15.78:0/2230534156 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1080ca720 con 0x7fe108164d20 2023-10-11T09:43:51.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.826+0000 7fe10759e700 1 --2- 172.21.15.78:0/2230534156 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe108164d20 0x7fe10806bbf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe100009dc0 tx=0x7fe10000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2e8ab2546d974a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:51.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.826+0000 7fe105d9b700 1 -- 172.21.15.78:0/2230534156 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10000e070 con 0x7fe108164d20 2023-10-11T09:43:51.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.830+0000 7fe105d9b700 1 -- 172.21.15.78:0/2230534156 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe10000bb20 con 0x7fe108164d20 2023-10-11T09:43:51.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.830+0000 7fe105d9b700 1 -- 172.21.15.78:0/2230534156 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10000bed0 con 0x7fe108164d20 2023-10-11T09:43:51.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.830+0000 7fe10c991700 1 -- 172.21.15.78:0/2230534156 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe108164d20 msgr2=0x7fe10806bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:51.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.830+0000 7fe10c991700 1 --2- 172.21.15.78:0/2230534156 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe108164d20 0x7fe10806bbf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe100009dc0 tx=0x7fe10000b330 comp rx=0 tx=0).stop 2023-10-11T09:43:51.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.830+0000 7fe10c991700 1 -- 172.21.15.78:0/2230534156 shutdown_connections 2023-10-11T09:43:51.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.830+0000 7fe10c991700 1 --2- 172.21.15.78:0/2230534156 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe108164d20 0x7fe10806bbf0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:51.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.830+0000 7fe10c991700 1 -- 172.21.15.78:0/2230534156 >> 172.21.15.78:0/2230534156 conn(0x7fe1080cea60 msgr2=0x7fe1080cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:51.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.830+0000 7fe10c991700 1 -- 172.21.15.78:0/2230534156 shutdown_connections 2023-10-11T09:43:51.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.830+0000 7fe10c991700 1 -- 172.21.15.78:0/2230534156 wait complete. 2023-10-11T09:43:51.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.830+0000 7fe10c991700 1 Processor -- start 2023-10-11T09:43:51.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.830+0000 7fe10c991700 1 -- start start 2023-10-11T09:43:51.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.834+0000 7fe10c991700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1080caf20 0x7fe1080cbe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:51.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.834+0000 7fe10c991700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1080cda10 0x7fe1081620a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:51.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.834+0000 7fe10c991700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe108164d20 0x7fe10815ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:51.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.834+0000 7fe10c991700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1080c3b90 con 0x7fe108164d20 2023-10-11T09:43:51.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.834+0000 7fe10c991700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1081625e0 con 0x7fe1080caf20 2023-10-11T09:43:51.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.834+0000 7fe10c991700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe108160450 con 0x7fe1080cda10 2023-10-11T09:43:51.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.834+0000 7fe10759e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe108164d20 0x7fe10815ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:51.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.834+0000 7fe10759e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe108164d20 0x7fe10815ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50292/0 (socket says 172.21.15.78:50292) 2023-10-11T09:43:51.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.834+0000 7fe10759e700 1 -- 172.21.15.78:0/2266353923 learned_addr learned my addr 172.21.15.78:0/2266353923 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:51.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.834+0000 7fe10659c700 1 --2- 172.21.15.78:0/2266353923 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1080cda10 0x7fe1081620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:51.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.834+0000 7fe106d9d700 1 --2- 172.21.15.78:0/2266353923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1080caf20 0x7fe1080cbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:51.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.834+0000 7fe10759e700 1 -- 172.21.15.78:0/2266353923 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1080cda10 msgr2=0x7fe1081620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:51.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.834+0000 7fe10759e700 1 --2- 172.21.15.78:0/2266353923 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1080cda10 0x7fe1081620a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:51.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.834+0000 7fe10759e700 1 -- 172.21.15.78:0/2266353923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1080caf20 msgr2=0x7fe1080cbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:51.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.834+0000 7fe10759e700 1 --2- 172.21.15.78:0/2266353923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1080caf20 0x7fe1080cbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:51.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.834+0000 7fe10759e700 1 -- 172.21.15.78:0/2266353923 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe100009a70 con 0x7fe108164d20 2023-10-11T09:43:51.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.834+0000 7fe10759e700 1 --2- 172.21.15.78:0/2266353923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe108164d20 0x7fe10815ff10 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe100009f30 tx=0x7fe100004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:51.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.834+0000 7fe0ef7fe700 1 -- 172.21.15.78:0/2266353923 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10001b810 con 0x7fe108164d20 2023-10-11T09:43:51.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.834+0000 7fe0ef7fe700 1 -- 172.21.15.78:0/2266353923 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe100020070 con 0x7fe108164d20 2023-10-11T09:43:51.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.834+0000 7fe10c991700 1 -- 172.21.15.78:0/2266353923 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe108162880 con 0x7fe108164d20 2023-10-11T09:43:51.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.834+0000 7fe0ef7fe700 1 -- 172.21.15.78:0/2266353923 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10001bbf0 con 0x7fe108164d20 2023-10-11T09:43:51.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.834+0000 7fe10c991700 1 -- 172.21.15.78:0/2266353923 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe108162e40 con 0x7fe108164d20 2023-10-11T09:43:51.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.838+0000 7fe10c991700 1 -- 172.21.15.78:0/2266353923 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe108065d40 con 0x7fe108164d20 2023-10-11T09:43:51.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.838+0000 7fe0ef7fe700 1 -- 172.21.15.78:0/2266353923 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fe10001bd50 con 0x7fe108164d20 2023-10-11T09:43:51.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.838+0000 7fe0ef7fe700 1 --2- 172.21.15.78:0/2266353923 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe0f4065ed0 0x7fe0f4068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:51.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.838+0000 7fe0ef7fe700 1 -- 172.21.15.78:0/2266353923 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6753+0+0 (secure 0 0 0) 0x7fe100022070 con 0x7fe108164d20 2023-10-11T09:43:51.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.838+0000 7fe106d9d700 1 --2- 172.21.15.78:0/2266353923 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe0f4065ed0 0x7fe0f4068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:51.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.842+0000 7fe106d9d700 1 --2- 172.21.15.78:0/2266353923 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe0f4065ed0 0x7fe0f4068380 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8000f60 tx=0x7fe0f8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:51.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:51.846+0000 7fe0ef7fe700 1 -- 172.21.15.78:0/2266353923 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fe100053210 con 0x7fe108164d20 2023-10-11T09:43:52.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.050+0000 7fe10c991700 1 -- 172.21.15.78:0/2266353923 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1080b1480 con 0x7fe108164d20 2023-10-11T09:43:52.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.054+0000 7fe0ef7fe700 1 -- 172.21.15.78:0/2266353923 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4654 (secure 0 0 0) 0x7fe10002c090 con 0x7fe108164d20 2023-10-11T09:43:52.062 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:43:52.062 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":29,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/588936111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":588936111},{"type":"v1","addr":"172.21.15.78:6836","nonce":588936111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5357,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/2574909888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2574909888},{"type":"v1","addr":"172.21.15.133:6834","nonce":2574909888}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5791,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2948223101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":2948223101},{"type":"v1","addr":"172.21.15.133:6835","nonce":2948223101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:42:45.491515+0000","modified":"2023-10-11T09:42:54.734536+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5922":{"gid":5922,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":4,"addr":"172.21.15.78:6837/3137573764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":3137573764},{"type":"v1","addr":"172.21.15.78:6837","nonce":3137573764}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-11T09:43:52.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.058+0000 7fe10c991700 1 -- 172.21.15.78:0/2266353923 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe0f4065ed0 msgr2=0x7fe0f4068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:52.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.058+0000 7fe10c991700 1 --2- 172.21.15.78:0/2266353923 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe0f4065ed0 0x7fe0f4068380 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8000f60 tx=0x7fe0f8009040 comp rx=0 tx=0).stop 2023-10-11T09:43:52.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.058+0000 7fe10c991700 1 -- 172.21.15.78:0/2266353923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe108164d20 msgr2=0x7fe10815ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:52.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.058+0000 7fe10c991700 1 --2- 172.21.15.78:0/2266353923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe108164d20 0x7fe10815ff10 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe100009f30 tx=0x7fe100004af0 comp rx=0 tx=0).stop 2023-10-11T09:43:52.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.058+0000 7fe10c991700 1 -- 172.21.15.78:0/2266353923 shutdown_connections 2023-10-11T09:43:52.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.058+0000 7fe10c991700 1 --2- 172.21.15.78:0/2266353923 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe0f4065ed0 0x7fe0f4068380 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:52.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.058+0000 7fe10c991700 1 --2- 172.21.15.78:0/2266353923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe108164d20 0x7fe10815ff10 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:52.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.058+0000 7fe10c991700 1 --2- 172.21.15.78:0/2266353923 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1080cda10 0x7fe1081620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:52.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.058+0000 7fe10c991700 1 --2- 172.21.15.78:0/2266353923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1080caf20 0x7fe1080cbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:52.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.058+0000 7fe10c991700 1 -- 172.21.15.78:0/2266353923 >> 172.21.15.78:0/2266353923 conn(0x7fe1080cea60 msgr2=0x7fe108070620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:52.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.058+0000 7fe10c991700 1 -- 172.21.15.78:0/2266353923 shutdown_connections 2023-10-11T09:43:52.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.058+0000 7fe10c991700 1 -- 172.21.15.78:0/2266353923 wait complete. 2023-10-11T09:43:52.067 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 29 2023-10-11T09:43:52.087 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:43:52.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.254+0000 7f1f95759700 1 Processor -- start 2023-10-11T09:43:52.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.254+0000 7f1f95759700 1 -- start start 2023-10-11T09:43:52.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.258+0000 7f1f95759700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1f90154e00 0x7f1f9015b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:52.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.258+0000 7f1f95759700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1f90071040 0x7f1f900cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:52.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.258+0000 7f1f95759700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f900720f0 con 0x7f1f900cba00 2023-10-11T09:43:52.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.258+0000 7f1f95759700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f9015bb90 con 0x7f1f90154e00 2023-10-11T09:43:52.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.258+0000 7f1f95759700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f900714e0 con 0x7f1f90071040 2023-10-11T09:43:52.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.258+0000 7f1f8effd700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1f90154e00 0x7f1f9015b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:52.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.258+0000 7f1f8effd700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1f90154e00 0x7f1f9015b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:32892/0 (socket says 172.21.15.78:32892) 2023-10-11T09:43:52.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.258+0000 7f1f8effd700 1 -- 172.21.15.78:0/2263902201 learned_addr learned my addr 172.21.15.78:0/2263902201 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:52.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.258+0000 7f1f8e7fc700 1 --2- 172.21.15.78:0/2263902201 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1f90071040 0x7f1f900cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:52.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.258+0000 7f1f8effd700 1 -- 172.21.15.78:0/2263902201 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1f90071040 msgr2=0x7f1f900cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:52.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.258+0000 7f1f8effd700 1 --2- 172.21.15.78:0/2263902201 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1f90071040 0x7f1f900cb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:52.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.258+0000 7f1f8effd700 1 -- 172.21.15.78:0/2263902201 >> v1:172.21.15.133:6789/0 conn(0x7f1f900cba00 legacy=0x7f1f9015e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:52.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.258+0000 7f1f8effd700 1 -- 172.21.15.78:0/2263902201 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f900717c0 con 0x7f1f90154e00 2023-10-11T09:43:52.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.258+0000 7f1f8dffb700 1 -- 172.21.15.78:0/2263902201 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2481329395 0 0) 0x7f1f900720f0 con 0x7f1f900cba00 2023-10-11T09:43:52.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.258+0000 7f1f8effd700 1 --2- 172.21.15.78:0/2263902201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f90154e00 0x7f1f9015b650 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1f80009910 tx=0x7f1f80009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22aefd139216c903 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:52.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.258+0000 7f1f8dffb700 1 -- 172.21.15.78:0/2263902201 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f80011830 con 0x7f1f90154e00 2023-10-11T09:43:52.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.258+0000 7f1f8dffb700 1 -- 172.21.15.78:0/2263902201 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f80011990 con 0x7f1f90154e00 2023-10-11T09:43:52.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.258+0000 7f1f8dffb700 1 -- 172.21.15.78:0/2263902201 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f80011d40 con 0x7f1f90154e00 2023-10-11T09:43:52.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.262+0000 7f1f95759700 1 -- 172.21.15.78:0/2263902201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f90154e00 msgr2=0x7f1f9015b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:52.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.262+0000 7f1f95759700 1 --2- 172.21.15.78:0/2263902201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f90154e00 0x7f1f9015b650 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1f80009910 tx=0x7f1f80009c20 comp rx=0 tx=0).stop 2023-10-11T09:43:52.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.262+0000 7f1f95759700 1 -- 172.21.15.78:0/2263902201 shutdown_connections 2023-10-11T09:43:52.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.262+0000 7f1f95759700 1 --2- 172.21.15.78:0/2263902201 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1f90071040 0x7f1f900cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:52.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.262+0000 7f1f95759700 1 --2- 172.21.15.78:0/2263902201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f90154e00 0x7f1f9015b650 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:52.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.262+0000 7f1f95759700 1 -- 172.21.15.78:0/2263902201 >> 172.21.15.78:0/2263902201 conn(0x7f1f9006aca0 msgr2=0x7f1f900b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:52.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.262+0000 7f1f95759700 1 -- 172.21.15.78:0/2263902201 shutdown_connections 2023-10-11T09:43:52.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.262+0000 7f1f95759700 1 -- 172.21.15.78:0/2263902201 wait complete. 2023-10-11T09:43:52.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.262+0000 7f1f95759700 1 Processor -- start 2023-10-11T09:43:52.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.262+0000 7f1f95759700 1 -- start start 2023-10-11T09:43:52.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.266+0000 7f1f95759700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f90071040 0x7f1f901f18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:52.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.266+0000 7f1f95759700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f900cba00 0x7f1f900c8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:52.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.266+0000 7f1f95759700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1f90154e00 0x7f1f900c8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:52.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.266+0000 7f1f95759700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f900ce7c0 con 0x7f1f90071040 2023-10-11T09:43:52.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.266+0000 7f1f95759700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f900b0370 con 0x7f1f900cba00 2023-10-11T09:43:52.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.266+0000 7f1f95759700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f900b0650 con 0x7f1f90154e00 2023-10-11T09:43:52.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.266+0000 7f1f8effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f90071040 0x7f1f901f18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:52.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.266+0000 7f1f8effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f90071040 0x7f1f901f18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50306/0 (socket says 172.21.15.78:50306) 2023-10-11T09:43:52.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.266+0000 7f1f8effd700 1 -- 172.21.15.78:0/3865036206 learned_addr learned my addr 172.21.15.78:0/3865036206 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:52.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.266+0000 7f1f8e7fc700 1 --2- 172.21.15.78:0/3865036206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f900cba00 0x7f1f900c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:52.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.266+0000 7f1f8f7fe700 1 --2- 172.21.15.78:0/3865036206 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1f90154e00 0x7f1f900c8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:52.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.266+0000 7f1f8effd700 1 -- 172.21.15.78:0/3865036206 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1f90154e00 msgr2=0x7f1f900c8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:52.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.266+0000 7f1f8effd700 1 --2- 172.21.15.78:0/3865036206 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1f90154e00 0x7f1f900c8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:52.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.266+0000 7f1f8effd700 1 -- 172.21.15.78:0/3865036206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f900cba00 msgr2=0x7f1f900c8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:52.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.266+0000 7f1f8effd700 1 --2- 172.21.15.78:0/3865036206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f900cba00 0x7f1f900c8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:52.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.266+0000 7f1f8effd700 1 -- 172.21.15.78:0/3865036206 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f80009570 con 0x7f1f90071040 2023-10-11T09:43:52.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.266+0000 7f1f8effd700 1 --2- 172.21.15.78:0/3865036206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f90071040 0x7f1f901f18a0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f1f80014040 tx=0x7f1f80004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:52.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.266+0000 7f1f777fe700 1 -- 172.21.15.78:0/3865036206 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8001a8f0 con 0x7f1f90071040 2023-10-11T09:43:52.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.266+0000 7f1f777fe700 1 -- 172.21.15.78:0/3865036206 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f8001e070 con 0x7f1f90071040 2023-10-11T09:43:52.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.266+0000 7f1f95759700 1 -- 172.21.15.78:0/3865036206 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f900b0930 con 0x7f1f90071040 2023-10-11T09:43:52.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.266+0000 7f1f777fe700 1 -- 172.21.15.78:0/3865036206 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8001acb0 con 0x7f1f90071040 2023-10-11T09:43:52.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.266+0000 7f1f95759700 1 -- 172.21.15.78:0/3865036206 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f901f1de0 con 0x7f1f90071040 2023-10-11T09:43:52.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.270+0000 7f1f95759700 1 -- 172.21.15.78:0/3865036206 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f900b0e50 con 0x7f1f90071040 2023-10-11T09:43:52.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.270+0000 7f1f777fe700 1 -- 172.21.15.78:0/3865036206 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f1f8002d450 con 0x7f1f90071040 2023-10-11T09:43:52.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.274+0000 7f1f777fe700 1 --2- 172.21.15.78:0/3865036206 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1f7c065ed0 0x7f1f7c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:52.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.274+0000 7f1f777fe700 1 -- 172.21.15.78:0/3865036206 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6753+0+0 (secure 0 0 0) 0x7f1f800849d0 con 0x7f1f90071040 2023-10-11T09:43:52.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.274+0000 7f1f8e7fc700 1 --2- 172.21.15.78:0/3865036206 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1f7c065ed0 0x7f1f7c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:52.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.274+0000 7f1f8e7fc700 1 --2- 172.21.15.78:0/3865036206 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1f7c065ed0 0x7f1f7c068380 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1f901578b0 tx=0x7f1f84009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:52.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.278+0000 7f1f777fe700 1 -- 172.21.15.78:0/3865036206 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f1f900b0e50 con 0x7f1f90071040 2023-10-11T09:43:52.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.450+0000 7f1f95759700 1 -- 172.21.15.78:0/3865036206 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f900b0e50 con 0x7f1f90071040 2023-10-11T09:43:52.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.454+0000 7f1f777fe700 1 -- 172.21.15.78:0/3865036206 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+20115 (secure 0 0 0) 0x7f1f900b0e50 con 0x7f1f90071040 2023-10-11T09:43:52.458 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:43:52.458 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":48,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:42:51.686927+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:41:57.280079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-11T09:41:58.218860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data_extra","create_time":"2023-10-11T09:42:12.954795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.10","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6833/1601241351":"2023-10-12T09:42:51.686883+0000","172.21.15.133:6832/1601241351":"2023-10-12T09:42:51.686883+0000","172.21.15.133:6833/4103940204":"2023-10-12T09:42:45.599736+0000","172.21.15.133:6832/4103940204":"2023-10-12T09:42:45.599736+0000","172.21.15.133:6833/1862542010":"2023-10-12T09:42:44.563306+0000","172.21.15.133:6832/1862542010":"2023-10-12T09:42:44.563306+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:43:52.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.454+0000 7f1f95759700 1 -- 172.21.15.78:0/3865036206 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1f7c065ed0 msgr2=0x7f1f7c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:52.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.454+0000 7f1f95759700 1 --2- 172.21.15.78:0/3865036206 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1f7c065ed0 0x7f1f7c068380 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1f901578b0 tx=0x7f1f84009240 comp rx=0 tx=0).stop 2023-10-11T09:43:52.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.454+0000 7f1f95759700 1 -- 172.21.15.78:0/3865036206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f90071040 msgr2=0x7f1f901f18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:52.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.454+0000 7f1f95759700 1 --2- 172.21.15.78:0/3865036206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f90071040 0x7f1f901f18a0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f1f80014040 tx=0x7f1f80004770 comp rx=0 tx=0).stop 2023-10-11T09:43:52.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.454+0000 7f1f95759700 1 -- 172.21.15.78:0/3865036206 shutdown_connections 2023-10-11T09:43:52.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.454+0000 7f1f95759700 1 --2- 172.21.15.78:0/3865036206 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1f7c065ed0 0x7f1f7c068380 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:52.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.454+0000 7f1f95759700 1 --2- 172.21.15.78:0/3865036206 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1f90154e00 0x7f1f900c8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:52.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.454+0000 7f1f95759700 1 --2- 172.21.15.78:0/3865036206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f900cba00 0x7f1f900c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:52.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.454+0000 7f1f95759700 1 --2- 172.21.15.78:0/3865036206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f90071040 0x7f1f901f18a0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:52.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.454+0000 7f1f95759700 1 -- 172.21.15.78:0/3865036206 >> 172.21.15.78:0/3865036206 conn(0x7f1f9006aca0 msgr2=0x7f1f9015d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:52.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.454+0000 7f1f95759700 1 -- 172.21.15.78:0/3865036206 shutdown_connections 2023-10-11T09:43:52.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.454+0000 7f1f95759700 1 -- 172.21.15.78:0/3865036206 wait complete. 2023-10-11T09:43:52.484 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-11T09:43:52.485 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:43:52.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.650+0000 7f32ba5f9700 1 Processor -- start 2023-10-11T09:43:52.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.650+0000 7f32ba5f9700 1 -- start start 2023-10-11T09:43:52.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.654+0000 7f32ba5f9700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f32b4160d40 0x7f32b40cd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:52.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.654+0000 7f32ba5f9700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f32b40ce500 0x7f32b406be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:52.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.654+0000 7f32ba5f9700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b406ffa0 con 0x7f32b4071040 2023-10-11T09:43:52.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.654+0000 7f32ba5f9700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32b4070c20 con 0x7f32b40ce500 2023-10-11T09:43:52.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.654+0000 7f32ba5f9700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32b4161110 con 0x7f32b4160d40 2023-10-11T09:43:52.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.654+0000 7f32b8b96700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f32b40ce500 0x7f32b406be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:52.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.654+0000 7f32b8b96700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f32b40ce500 0x7f32b406be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50308/0 (socket says 172.21.15.78:50308) 2023-10-11T09:43:52.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.654+0000 7f32b8b96700 1 -- 172.21.15.78:0/1022519329 learned_addr learned my addr 172.21.15.78:0/1022519329 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:52.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.654+0000 7f32b3fff700 1 --2- 172.21.15.78:0/1022519329 >> v2:172.21.15.133:3300/0 conn(0x7f32b4160d40 0x7f32b40cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:52.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.654+0000 7f32b8b96700 1 -- 172.21.15.78:0/1022519329 >> v2:172.21.15.133:3300/0 conn(0x7f32b4160d40 msgr2=0x7f32b40cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:52.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.654+0000 7f32b8b96700 1 --2- 172.21.15.78:0/1022519329 >> v2:172.21.15.133:3300/0 conn(0x7f32b4160d40 0x7f32b40cd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:52.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.654+0000 7f32b8b96700 1 -- 172.21.15.78:0/1022519329 >> v1:172.21.15.133:6789/0 conn(0x7f32b4071040 legacy=0x7f32b40cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:52.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.654+0000 7f32b8b96700 1 -- 172.21.15.78:0/1022519329 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b4070280 con 0x7f32b40ce500 2023-10-11T09:43:52.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.654+0000 7f32b8b96700 1 --2- 172.21.15.78:0/1022519329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32b40ce500 0x7f32b406be60 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f32b40b1a50 tx=0x7f32ac009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=252c1ad5507b69ff server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:52.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.658+0000 7f32b37fe700 1 -- 172.21.15.78:0/1022519329 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac011830 con 0x7f32b40ce500 2023-10-11T09:43:52.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.658+0000 7f32b37fe700 1 -- 172.21.15.78:0/1022519329 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ac011990 con 0x7f32b40ce500 2023-10-11T09:43:52.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.658+0000 7f32b37fe700 1 -- 172.21.15.78:0/1022519329 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac011d20 con 0x7f32b40ce500 2023-10-11T09:43:52.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.658+0000 7f32ba5f9700 1 -- 172.21.15.78:0/1022519329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32b40ce500 msgr2=0x7f32b406be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:52.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.658+0000 7f32ba5f9700 1 --2- 172.21.15.78:0/1022519329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32b40ce500 0x7f32b406be60 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f32b40b1a50 tx=0x7f32ac009bf0 comp rx=0 tx=0).stop 2023-10-11T09:43:52.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.658+0000 7f32ba5f9700 1 -- 172.21.15.78:0/1022519329 shutdown_connections 2023-10-11T09:43:52.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.658+0000 7f32ba5f9700 1 --2- 172.21.15.78:0/1022519329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32b40ce500 0x7f32b406be60 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:52.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.658+0000 7f32ba5f9700 1 --2- 172.21.15.78:0/1022519329 >> v2:172.21.15.133:3300/0 conn(0x7f32b4160d40 0x7f32b40cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:52.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.658+0000 7f32ba5f9700 1 -- 172.21.15.78:0/1022519329 >> 172.21.15.78:0/1022519329 conn(0x7f32b406aca0 msgr2=0x7f32b40b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:52.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.658+0000 7f32ba5f9700 1 -- 172.21.15.78:0/1022519329 shutdown_connections 2023-10-11T09:43:52.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.658+0000 7f32ba5f9700 1 -- 172.21.15.78:0/1022519329 wait complete. 2023-10-11T09:43:52.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.658+0000 7f32ba5f9700 1 Processor -- start 2023-10-11T09:43:52.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.662+0000 7f32ba5f9700 1 -- start start 2023-10-11T09:43:52.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.662+0000 7f32ba5f9700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32b4071040 0x7f32b415c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:52.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.662+0000 7f32ba5f9700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32b40ce500 0x7f32b40c73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:52.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.662+0000 7f32ba5f9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32b4160d40 0x7f32b40c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:52.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.662+0000 7f32ba5f9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b40c8b70 con 0x7f32b4160d40 2023-10-11T09:43:52.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.662+0000 7f32ba5f9700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b40c7f60 con 0x7f32b40ce500 2023-10-11T09:43:52.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.662+0000 7f32ba5f9700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b40c8240 con 0x7f32b4071040 2023-10-11T09:43:52.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.662+0000 7f32b8b96700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32b4160d40 0x7f32b40c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:52.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.662+0000 7f32ab7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32b40ce500 0x7f32b40c73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:52.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.662+0000 7f32b8b96700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32b4160d40 0x7f32b40c78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50316/0 (socket says 172.21.15.78:50316) 2023-10-11T09:43:52.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.662+0000 7f32b8b96700 1 -- 172.21.15.78:0/3807226988 learned_addr learned my addr 172.21.15.78:0/3807226988 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:52.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.662+0000 7f32b3fff700 1 --2- 172.21.15.78:0/3807226988 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32b4071040 0x7f32b415c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:52.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.662+0000 7f32ab7fe700 1 -- 172.21.15.78:0/3807226988 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32b4071040 msgr2=0x7f32b415c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:52.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.662+0000 7f32ab7fe700 1 --2- 172.21.15.78:0/3807226988 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32b4071040 0x7f32b415c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:52.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.662+0000 7f32ab7fe700 1 -- 172.21.15.78:0/3807226988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32b4160d40 msgr2=0x7f32b40c78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:52.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.662+0000 7f32ab7fe700 1 --2- 172.21.15.78:0/3807226988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32b4160d40 0x7f32b40c78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:52.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.662+0000 7f32ab7fe700 1 -- 172.21.15.78:0/3807226988 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ac009570 con 0x7f32b40ce500 2023-10-11T09:43:52.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.662+0000 7f32b8b96700 1 --2- 172.21.15.78:0/3807226988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32b4160d40 0x7f32b40c78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:43:52.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.662+0000 7f32ab7fe700 1 --2- 172.21.15.78:0/3807226988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32b40ce500 0x7f32b40c73a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f32b4071430 tx=0x7f329c00c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:52.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.662+0000 7f32b17fa700 1 -- 172.21.15.78:0/3807226988 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c0052d0 con 0x7f32b40ce500 2023-10-11T09:43:52.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.662+0000 7f32b17fa700 1 -- 172.21.15.78:0/3807226988 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329c005430 con 0x7f32b40ce500 2023-10-11T09:43:52.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.662+0000 7f32ba5f9700 1 -- 172.21.15.78:0/3807226988 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b40c8520 con 0x7f32b40ce500 2023-10-11T09:43:52.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.662+0000 7f32b17fa700 1 -- 172.21.15.78:0/3807226988 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c0055a0 con 0x7f32b40ce500 2023-10-11T09:43:52.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.662+0000 7f32ba5f9700 1 -- 172.21.15.78:0/3807226988 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b415c560 con 0x7f32b40ce500 2023-10-11T09:43:52.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.666+0000 7f32ba5f9700 1 -- 172.21.15.78:0/3807226988 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b4072250 con 0x7f32b40ce500 2023-10-11T09:43:52.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.666+0000 7f32b17fa700 1 -- 172.21.15.78:0/3807226988 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f329c0038e0 con 0x7f32b40ce500 2023-10-11T09:43:52.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.670+0000 7f32b17fa700 1 --2- 172.21.15.78:0/3807226988 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3294065ed0 0x7f3294068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:52.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.670+0000 7f32b17fa700 1 -- 172.21.15.78:0/3807226988 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6753+0+0 (secure 0 0 0) 0x7f329c082d40 con 0x7f32b40ce500 2023-10-11T09:43:52.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.670+0000 7f32b3fff700 1 --2- 172.21.15.78:0/3807226988 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3294065ed0 0x7f3294068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:52.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.670+0000 7f32b3fff700 1 --2- 172.21.15.78:0/3807226988 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3294065ed0 0x7f3294068380 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f32b40b0a20 tx=0x7f32a400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:52.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.678+0000 7f32b17fa700 1 -- 172.21.15.78:0/3807226988 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f329c050140 con 0x7f32b40ce500 2023-10-11T09:43:52.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.886+0000 7f32ba5f9700 1 -- 172.21.15.78:0/3807226988 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32b404e530 con 0x7f32b40ce500 2023-10-11T09:43:52.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.886+0000 7f32b17fa700 1 -- 172.21.15.78:0/3807226988 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4654 (secure 0 0 0) 0x7f329c053760 con 0x7f32b40ce500 2023-10-11T09:43:52.892 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:43:52.893 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":29,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/588936111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":588936111},{"type":"v1","addr":"172.21.15.78:6836","nonce":588936111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5357,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/2574909888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2574909888},{"type":"v1","addr":"172.21.15.133:6834","nonce":2574909888}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5791,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2948223101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":2948223101},{"type":"v1","addr":"172.21.15.133:6835","nonce":2948223101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:42:45.491515+0000","modified":"2023-10-11T09:42:54.734536+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5922":{"gid":5922,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":4,"addr":"172.21.15.78:6837/3137573764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":3137573764},{"type":"v1","addr":"172.21.15.78:6837","nonce":3137573764}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-11T09:43:52.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.890+0000 7f32ba5f9700 1 -- 172.21.15.78:0/3807226988 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3294065ed0 msgr2=0x7f3294068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:52.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.890+0000 7f32ba5f9700 1 --2- 172.21.15.78:0/3807226988 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3294065ed0 0x7f3294068380 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f32b40b0a20 tx=0x7f32a400b040 comp rx=0 tx=0).stop 2023-10-11T09:43:52.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.890+0000 7f32ba5f9700 1 -- 172.21.15.78:0/3807226988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32b40ce500 msgr2=0x7f32b40c73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:52.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.890+0000 7f32ba5f9700 1 --2- 172.21.15.78:0/3807226988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32b40ce500 0x7f32b40c73a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f32b4071430 tx=0x7f329c00c760 comp rx=0 tx=0).stop 2023-10-11T09:43:52.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.890+0000 7f32ba5f9700 1 -- 172.21.15.78:0/3807226988 shutdown_connections 2023-10-11T09:43:52.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.890+0000 7f32ba5f9700 1 --2- 172.21.15.78:0/3807226988 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3294065ed0 0x7f3294068380 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:52.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.890+0000 7f32ba5f9700 1 --2- 172.21.15.78:0/3807226988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32b4160d40 0x7f32b40c78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:52.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.890+0000 7f32ba5f9700 1 --2- 172.21.15.78:0/3807226988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32b40ce500 0x7f32b40c73a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:52.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.890+0000 7f32ba5f9700 1 --2- 172.21.15.78:0/3807226988 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32b4071040 0x7f32b415c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:52.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.890+0000 7f32ba5f9700 1 -- 172.21.15.78:0/3807226988 >> 172.21.15.78:0/3807226988 conn(0x7f32b406aca0 msgr2=0x7f32b415ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:52.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.890+0000 7f32ba5f9700 1 -- 172.21.15.78:0/3807226988 shutdown_connections 2023-10-11T09:43:52.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:52.890+0000 7f32ba5f9700 1 -- 172.21.15.78:0/3807226988 wait complete. 2023-10-11T09:43:52.898 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 29 2023-10-11T09:43:52.917 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:43:53.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.078+0000 7fc4a1876700 1 Processor -- start 2023-10-11T09:43:53.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.078+0000 7fc4a1876700 1 -- start start 2023-10-11T09:43:53.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.082+0000 7fc4a1876700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fc49c13f740 0x7fc49c0ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:53.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.082+0000 7fc4a1876700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc49c06d180 0x7fc49c0b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:53.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.082+0000 7fc4a1876700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc49c070c80 con 0x7fc49c06cd00 2023-10-11T09:43:53.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.082+0000 7fc4a1876700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc49c0ce4c0 con 0x7fc49c13f740 2023-10-11T09:43:53.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.082+0000 7fc4a1876700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c06d600 con 0x7fc49c06d180 2023-10-11T09:43:53.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.082+0000 7fc4a0874700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fc49c13f740 0x7fc49c0ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:53.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.082+0000 7fc493fff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fc49c06cd00 0x7fc49c0b06a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:42230/0 (socket says 172.21.15.78:42230) 2023-10-11T09:43:53.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.082+0000 7fc493fff700 1 -- 172.21.15.78:0/281328689 learned_addr learned my addr 172.21.15.78:0/281328689 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:43:53.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.082+0000 7fc4a1075700 1 --2- 172.21.15.78:0/281328689 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc49c06d180 0x7fc49c0b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:53.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.086+0000 7fc4937fe700 1 -- 172.21.15.78:0/281328689 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 907965514 0 0) 0x7fc49c070c80 con 0x7fc49c06cd00 2023-10-11T09:43:53.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.086+0000 7fc4937fe700 1 -- 172.21.15.78:0/281328689 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4840035d0 con 0x7fc49c06cd00 2023-10-11T09:43:53.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.086+0000 7fc4a0874700 1 -- 172.21.15.78:0/281328689 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc49c06d180 msgr2=0x7fc49c0b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:53.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.086+0000 7fc4a0874700 1 --2- 172.21.15.78:0/281328689 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc49c06d180 0x7fc49c0b0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:53.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.086+0000 7fc4a0874700 1 -- 172.21.15.78:0/281328689 >> v1:172.21.15.78:6789/0 conn(0x7fc49c06cd00 legacy=0x7fc49c0b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:53.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.086+0000 7fc4a0874700 1 -- 172.21.15.78:0/281328689 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49c0cd420 con 0x7fc49c13f740 2023-10-11T09:43:53.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.086+0000 7fc4a1075700 1 --2- 172.21.15.78:0/281328689 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc49c06d180 0x7fc49c0b0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:43:53.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.086+0000 7fc4a0874700 1 --2- 172.21.15.78:0/281328689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc49c13f740 0x7fc49c0ccab0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc498009850 tx=0x7fc498009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de05e97bb4336613 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:53.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.086+0000 7fc4937fe700 1 -- 172.21.15.78:0/281328689 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc498011830 con 0x7fc49c13f740 2023-10-11T09:43:53.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.086+0000 7fc4937fe700 1 -- 172.21.15.78:0/281328689 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc498011990 con 0x7fc49c13f740 2023-10-11T09:43:53.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.086+0000 7fc4937fe700 1 -- 172.21.15.78:0/281328689 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc498011d40 con 0x7fc49c13f740 2023-10-11T09:43:53.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.086+0000 7fc4a1876700 1 -- 172.21.15.78:0/281328689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc49c13f740 msgr2=0x7fc49c0ccab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:53.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.086+0000 7fc4a1876700 1 --2- 172.21.15.78:0/281328689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc49c13f740 0x7fc49c0ccab0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc498009850 tx=0x7fc498009b60 comp rx=0 tx=0).stop 2023-10-11T09:43:53.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.086+0000 7fc4a1876700 1 -- 172.21.15.78:0/281328689 shutdown_connections 2023-10-11T09:43:53.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.086+0000 7fc4a1876700 1 --2- 172.21.15.78:0/281328689 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc49c06d180 0x7fc49c0b0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:53.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.086+0000 7fc4a1876700 1 --2- 172.21.15.78:0/281328689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc49c13f740 0x7fc49c0ccab0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:53.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.086+0000 7fc4a1876700 1 -- 172.21.15.78:0/281328689 >> 172.21.15.78:0/281328689 conn(0x7fc49c0cea60 msgr2=0x7fc49c0cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:53.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.086+0000 7fc4a1876700 1 -- 172.21.15.78:0/281328689 shutdown_connections 2023-10-11T09:43:53.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.086+0000 7fc4a1876700 1 -- 172.21.15.78:0/281328689 wait complete. 2023-10-11T09:43:53.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.090+0000 7fc4a1876700 1 Processor -- start 2023-10-11T09:43:53.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.090+0000 7fc4a1876700 1 -- start start 2023-10-11T09:43:53.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.090+0000 7fc4a1876700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc49c06cd00 0x7fc49c131140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:53.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.090+0000 7fc4a1876700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc49c06d180 0x7fc49c0b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:53.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.090+0000 7fc4a1876700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc49c13f740 0x7fc49c156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:53.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.090+0000 7fc4a1876700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c0c70a0 con 0x7fc49c06cd00 2023-10-11T09:43:53.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.090+0000 7fc4a1876700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c0c7380 con 0x7fc49c13f740 2023-10-11T09:43:53.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.090+0000 7fc4a1876700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c0c7660 con 0x7fc49c06d180 2023-10-11T09:43:53.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.090+0000 7fc4a0874700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc49c06cd00 0x7fc49c131140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:53.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.090+0000 7fc4a0874700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc49c06cd00 0x7fc49c131140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50338/0 (socket says 172.21.15.78:50338) 2023-10-11T09:43:53.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.090+0000 7fc4a0874700 1 -- 172.21.15.78:0/4067556086 learned_addr learned my addr 172.21.15.78:0/4067556086 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:53.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.090+0000 7fc493fff700 1 --2- 172.21.15.78:0/4067556086 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc49c06d180 0x7fc49c0b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:53.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.090+0000 7fc4a1075700 1 --2- 172.21.15.78:0/4067556086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc49c13f740 0x7fc49c156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:53.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.090+0000 7fc4a0874700 1 -- 172.21.15.78:0/4067556086 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc49c06d180 msgr2=0x7fc49c0b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:53.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.090+0000 7fc4a0874700 1 --2- 172.21.15.78:0/4067556086 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc49c06d180 0x7fc49c0b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:53.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.090+0000 7fc4a0874700 1 -- 172.21.15.78:0/4067556086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc49c13f740 msgr2=0x7fc49c156b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:53.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.090+0000 7fc4a0874700 1 --2- 172.21.15.78:0/4067556086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc49c13f740 0x7fc49c156b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:53.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.090+0000 7fc4a0874700 1 -- 172.21.15.78:0/4067556086 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4980094b0 con 0x7fc49c06cd00 2023-10-11T09:43:53.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.090+0000 7fc493fff700 1 --2- 172.21.15.78:0/4067556086 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc49c06d180 0x7fc49c0b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:43:53.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.094+0000 7fc4a0874700 1 --2- 172.21.15.78:0/4067556086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc49c06cd00 0x7fc49c131140 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fc4980090b0 tx=0x7fc498004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:53.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.094+0000 7fc4917fa700 1 -- 172.21.15.78:0/4067556086 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc498004570 con 0x7fc49c06cd00 2023-10-11T09:43:53.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.094+0000 7fc4917fa700 1 -- 172.21.15.78:0/4067556086 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc49801f070 con 0x7fc49c06cd00 2023-10-11T09:43:53.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.094+0000 7fc4917fa700 1 -- 172.21.15.78:0/4067556086 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc498003900 con 0x7fc49c06cd00 2023-10-11T09:43:53.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.094+0000 7fc4a1876700 1 -- 172.21.15.78:0/4067556086 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc49c0c7940 con 0x7fc49c06cd00 2023-10-11T09:43:53.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.094+0000 7fc4a1876700 1 -- 172.21.15.78:0/4067556086 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc49c131680 con 0x7fc49c06cd00 2023-10-11T09:43:53.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.094+0000 7fc4a1876700 1 -- 172.21.15.78:0/4067556086 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc49c065d40 con 0x7fc49c06cd00 2023-10-11T09:43:53.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.094+0000 7fc4917fa700 1 -- 172.21.15.78:0/4067556086 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fc498021030 con 0x7fc49c06cd00 2023-10-11T09:43:53.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.098+0000 7fc4917fa700 1 --2- 172.21.15.78:0/4067556086 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc488065ed0 0x7fc488068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:53.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.098+0000 7fc4917fa700 1 -- 172.21.15.78:0/4067556086 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6753+0+0 (secure 0 0 0) 0x7fc498084fc0 con 0x7fc49c06cd00 2023-10-11T09:43:53.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.098+0000 7fc493fff700 1 --2- 172.21.15.78:0/4067556086 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc488065ed0 0x7fc488068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:53.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.098+0000 7fc493fff700 1 --2- 172.21.15.78:0/4067556086 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc488065ed0 0x7fc488068380 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc49c070d00 tx=0x7fc48c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:53.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.102+0000 7fc4917fa700 1 -- 172.21.15.78:0/4067556086 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fc4980522c0 con 0x7fc49c06cd00 2023-10-11T09:43:53.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.310+0000 7fc4a1876700 1 -- 172.21.15.78:0/4067556086 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc49c0c71e0 con 0x7fc49c06cd00 2023-10-11T09:43:53.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.310+0000 7fc4917fa700 1 -- 172.21.15.78:0/4067556086 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4654 (secure 0 0 0) 0x7fc49c0c71e0 con 0x7fc49c06cd00 2023-10-11T09:43:53.317 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:43:53.317 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":29,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/588936111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":588936111},{"type":"v1","addr":"172.21.15.78:6836","nonce":588936111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5357,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/2574909888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2574909888},{"type":"v1","addr":"172.21.15.133:6834","nonce":2574909888}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5791,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2948223101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":2948223101},{"type":"v1","addr":"172.21.15.133:6835","nonce":2948223101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:42:45.491515+0000","modified":"2023-10-11T09:42:54.734536+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5922":{"gid":5922,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":4,"addr":"172.21.15.78:6837/3137573764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":3137573764},{"type":"v1","addr":"172.21.15.78:6837","nonce":3137573764}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-11T09:43:53.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.314+0000 7fc4a1876700 1 -- 172.21.15.78:0/4067556086 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc488065ed0 msgr2=0x7fc488068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:53.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.314+0000 7fc4a1876700 1 --2- 172.21.15.78:0/4067556086 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc488065ed0 0x7fc488068380 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc49c070d00 tx=0x7fc48c009040 comp rx=0 tx=0).stop 2023-10-11T09:43:53.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.314+0000 7fc4a1876700 1 -- 172.21.15.78:0/4067556086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc49c06cd00 msgr2=0x7fc49c131140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:53.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.314+0000 7fc4a1876700 1 --2- 172.21.15.78:0/4067556086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc49c06cd00 0x7fc49c131140 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fc4980090b0 tx=0x7fc498004770 comp rx=0 tx=0).stop 2023-10-11T09:43:53.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.314+0000 7fc4a1876700 1 -- 172.21.15.78:0/4067556086 shutdown_connections 2023-10-11T09:43:53.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.314+0000 7fc4a1876700 1 --2- 172.21.15.78:0/4067556086 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc488065ed0 0x7fc488068380 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:53.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.314+0000 7fc4a1876700 1 --2- 172.21.15.78:0/4067556086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc49c13f740 0x7fc49c156b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:53.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.314+0000 7fc4a1876700 1 --2- 172.21.15.78:0/4067556086 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc49c06d180 0x7fc49c0b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:53.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.314+0000 7fc4a1876700 1 --2- 172.21.15.78:0/4067556086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc49c06cd00 0x7fc49c131140 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:53.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.314+0000 7fc4a1876700 1 -- 172.21.15.78:0/4067556086 >> 172.21.15.78:0/4067556086 conn(0x7fc49c0cea60 msgr2=0x7fc49c158d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:53.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.314+0000 7fc4a1876700 1 -- 172.21.15.78:0/4067556086 shutdown_connections 2023-10-11T09:43:53.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.314+0000 7fc4a1876700 1 -- 172.21.15.78:0/4067556086 wait complete. 2023-10-11T09:43:53.323 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 29 2023-10-11T09:43:53.342 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:43:53.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.510+0000 7f9ee0a71700 1 Processor -- start 2023-10-11T09:43:53.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.510+0000 7f9ee0a71700 1 -- start start 2023-10-11T09:43:53.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.514+0000 7f9ee0a71700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9edc0b2310 0x7f9edc161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:53.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.514+0000 7f9ee0a71700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f9edc0b4b60 0x7f9edc0d1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:53.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.514+0000 7f9ee0a71700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9edc0d11e0 con 0x7f9edc0d21e0 2023-10-11T09:43:53.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.514+0000 7f9ee0a71700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9edc0b5370 con 0x7f9edc0b4b60 2023-10-11T09:43:53.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.514+0000 7f9ee0a71700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9edc0d14a0 con 0x7f9edc0b2310 2023-10-11T09:43:53.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.514+0000 7f9ed9d9b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f9edc0b4b60 0x7f9edc0d1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:53.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.514+0000 7f9eda59c700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9edc0b2310 0x7f9edc161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:53.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.514+0000 7f9ed9d9b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f9edc0b4b60 0x7f9edc0d1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50346/0 (socket says 172.21.15.78:50346) 2023-10-11T09:43:53.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.514+0000 7f9ed9d9b700 1 -- 172.21.15.78:0/2452144024 learned_addr learned my addr 172.21.15.78:0/2452144024 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:53.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.514+0000 7f9ed9d9b700 1 -- 172.21.15.78:0/2452144024 >> v2:172.21.15.133:3300/0 conn(0x7f9edc0b2310 msgr2=0x7f9edc161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:53.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.514+0000 7f9ed9d9b700 1 --2- 172.21.15.78:0/2452144024 >> v2:172.21.15.133:3300/0 conn(0x7f9edc0b2310 0x7f9edc161150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:53.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.514+0000 7f9ed9d9b700 1 -- 172.21.15.78:0/2452144024 >> v1:172.21.15.133:6789/0 conn(0x7f9edc0d21e0 legacy=0x7f9edc0b5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:53.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.514+0000 7f9ed9d9b700 1 -- 172.21.15.78:0/2452144024 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9edc0d1780 con 0x7f9edc0b4b60 2023-10-11T09:43:53.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.514+0000 7f9eda59c700 1 --2- 172.21.15.78:0/2452144024 >> v2:172.21.15.133:3300/0 conn(0x7f9edc0b2310 0x7f9edc161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:43:53.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.518+0000 7f9ed9d9b700 1 --2- 172.21.15.78:0/2452144024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9edc0b4b60 0x7f9edc0d1ca0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0009910 tx=0x7f9ed0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a7eccbc4b6c634a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:53.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.518+0000 7f9ed959a700 1 -- 172.21.15.78:0/2452144024 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0011830 con 0x7f9edc0b4b60 2023-10-11T09:43:53.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.518+0000 7f9ed959a700 1 -- 172.21.15.78:0/2452144024 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed0011990 con 0x7f9edc0b4b60 2023-10-11T09:43:53.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.518+0000 7f9ed959a700 1 -- 172.21.15.78:0/2452144024 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0011d40 con 0x7f9edc0b4b60 2023-10-11T09:43:53.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.518+0000 7f9ee0a71700 1 -- 172.21.15.78:0/2452144024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9edc0b4b60 msgr2=0x7f9edc0d1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:53.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.518+0000 7f9ee0a71700 1 --2- 172.21.15.78:0/2452144024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9edc0b4b60 0x7f9edc0d1ca0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0009910 tx=0x7f9ed0009c20 comp rx=0 tx=0).stop 2023-10-11T09:43:53.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.518+0000 7f9ee0a71700 1 -- 172.21.15.78:0/2452144024 shutdown_connections 2023-10-11T09:43:53.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.518+0000 7f9ee0a71700 1 --2- 172.21.15.78:0/2452144024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9edc0b4b60 0x7f9edc0d1ca0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:53.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.518+0000 7f9ee0a71700 1 --2- 172.21.15.78:0/2452144024 >> v2:172.21.15.133:3300/0 conn(0x7f9edc0b2310 0x7f9edc161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:53.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.518+0000 7f9ee0a71700 1 -- 172.21.15.78:0/2452144024 >> 172.21.15.78:0/2452144024 conn(0x7f9edc0cfb60 msgr2=0x7f9edc0cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:53.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.518+0000 7f9ee0a71700 1 -- 172.21.15.78:0/2452144024 shutdown_connections 2023-10-11T09:43:53.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.518+0000 7f9ee0a71700 1 -- 172.21.15.78:0/2452144024 wait complete. 2023-10-11T09:43:53.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.522+0000 7f9ee0a71700 1 Processor -- start 2023-10-11T09:43:53.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.522+0000 7f9ee0a71700 1 -- start start 2023-10-11T09:43:53.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.522+0000 7f9ee0a71700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9edc0b2310 0x7f9edc1f5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:53.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.522+0000 7f9ee0a71700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9edc0b4b60 0x7f9edc0c8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:53.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.522+0000 7f9ee0a71700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9edc0d21e0 0x7f9edc151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:53.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.522+0000 7f9ee0a71700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc0c7c60 con 0x7f9edc0b2310 2023-10-11T09:43:53.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.522+0000 7f9ee0a71700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc1f6280 con 0x7f9edc0d21e0 2023-10-11T09:43:53.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.522+0000 7f9ee0a71700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc1f6560 con 0x7f9edc0b4b60 2023-10-11T09:43:53.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.522+0000 7f9eda59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9edc0b2310 0x7f9edc1f5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:53.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.522+0000 7f9eda59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9edc0b2310 0x7f9edc1f5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50360/0 (socket says 172.21.15.78:50360) 2023-10-11T09:43:53.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.522+0000 7f9eda59c700 1 -- 172.21.15.78:0/120966308 learned_addr learned my addr 172.21.15.78:0/120966308 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:53.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.522+0000 7f9ed9d9b700 1 --2- 172.21.15.78:0/120966308 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9edc0b4b60 0x7f9edc0c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:53.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.522+0000 7f9edad9d700 1 --2- 172.21.15.78:0/120966308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9edc0d21e0 0x7f9edc151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:53.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.522+0000 7f9eda59c700 1 -- 172.21.15.78:0/120966308 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9edc0b4b60 msgr2=0x7f9edc0c8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:53.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.522+0000 7f9eda59c700 1 --2- 172.21.15.78:0/120966308 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9edc0b4b60 0x7f9edc0c8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:53.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.522+0000 7f9eda59c700 1 -- 172.21.15.78:0/120966308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9edc0d21e0 msgr2=0x7f9edc151da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:53.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.522+0000 7f9eda59c700 1 --2- 172.21.15.78:0/120966308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9edc0d21e0 0x7f9edc151da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:53.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.522+0000 7f9eda59c700 1 -- 172.21.15.78:0/120966308 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ecc0094b0 con 0x7f9edc0b2310 2023-10-11T09:43:53.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.522+0000 7f9ed9d9b700 1 --2- 172.21.15.78:0/120966308 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9edc0b4b60 0x7f9edc0c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:43:53.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.522+0000 7f9eda59c700 1 --2- 172.21.15.78:0/120966308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9edc0b2310 0x7f9edc1f5a30 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc00ebe0 tx=0x7f9ecc009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:53.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.522+0000 7f9ecaffd700 1 -- 172.21.15.78:0/120966308 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc00c910 con 0x7f9edc0b2310 2023-10-11T09:43:53.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.522+0000 7f9ecaffd700 1 -- 172.21.15.78:0/120966308 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ecc00ca70 con 0x7f9edc0b2310 2023-10-11T09:43:53.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.522+0000 7f9ecaffd700 1 -- 172.21.15.78:0/120966308 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc00ce00 con 0x7f9edc0b2310 2023-10-11T09:43:53.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.522+0000 7f9ee0a71700 1 -- 172.21.15.78:0/120966308 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed0009570 con 0x7f9edc0b2310 2023-10-11T09:43:53.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.522+0000 7f9ee0a71700 1 -- 172.21.15.78:0/120966308 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9edc0b5140 con 0x7f9edc0b2310 2023-10-11T09:43:53.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.526+0000 7f9ee0a71700 1 -- 172.21.15.78:0/120966308 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9edc04e530 con 0x7f9edc0b2310 2023-10-11T09:43:53.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.526+0000 7f9ecaffd700 1 -- 172.21.15.78:0/120966308 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f9ecc013470 con 0x7f9edc0b2310 2023-10-11T09:43:53.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.530+0000 7f9ecaffd700 1 --2- 172.21.15.78:0/120966308 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9ebc065ed0 0x7f9ebc068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:53.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.530+0000 7f9ecaffd700 1 -- 172.21.15.78:0/120966308 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6753+0+0 (secure 0 0 0) 0x7f9ecc083ca0 con 0x7f9edc0b2310 2023-10-11T09:43:53.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.530+0000 7f9ed9d9b700 1 --2- 172.21.15.78:0/120966308 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9ebc065ed0 0x7f9ebc068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:53.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.530+0000 7f9ed9d9b700 1 --2- 172.21.15.78:0/120966308 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9ebc065ed0 0x7f9ebc068380 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0000c00 tx=0x7f9ed0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:53.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.534+0000 7f9ecaffd700 1 -- 172.21.15.78:0/120966308 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f9ecc051030 con 0x7f9edc0b2310 2023-10-11T09:43:53.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.706+0000 7f9ee0a71700 1 -- 172.21.15.78:0/120966308 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9edc0cf1e0 con 0x7f9edc0b2310 2023-10-11T09:43:53.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.710+0000 7f9ecaffd700 1 -- 172.21.15.78:0/120966308 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+20115 (secure 0 0 0) 0x7f9ecc017090 con 0x7f9edc0b2310 2023-10-11T09:43:53.716 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:43:53.716 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":48,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:42:51.686927+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:41:57.280079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-11T09:41:58.218860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data_extra","create_time":"2023-10-11T09:42:12.954795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.10","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6833/1601241351":"2023-10-12T09:42:51.686883+0000","172.21.15.133:6832/1601241351":"2023-10-12T09:42:51.686883+0000","172.21.15.133:6833/4103940204":"2023-10-12T09:42:45.599736+0000","172.21.15.133:6832/4103940204":"2023-10-12T09:42:45.599736+0000","172.21.15.133:6833/1862542010":"2023-10-12T09:42:44.563306+0000","172.21.15.133:6832/1862542010":"2023-10-12T09:42:44.563306+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:43:53.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.714+0000 7f9ee0a71700 1 -- 172.21.15.78:0/120966308 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9ebc065ed0 msgr2=0x7f9ebc068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:53.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.714+0000 7f9ee0a71700 1 --2- 172.21.15.78:0/120966308 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9ebc065ed0 0x7f9ebc068380 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0000c00 tx=0x7f9ed0008030 comp rx=0 tx=0).stop 2023-10-11T09:43:53.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.714+0000 7f9ee0a71700 1 -- 172.21.15.78:0/120966308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9edc0b2310 msgr2=0x7f9edc1f5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:53.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.714+0000 7f9ee0a71700 1 --2- 172.21.15.78:0/120966308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9edc0b2310 0x7f9edc1f5a30 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc00ebe0 tx=0x7f9ecc009ed0 comp rx=0 tx=0).stop 2023-10-11T09:43:53.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.714+0000 7f9ee0a71700 1 -- 172.21.15.78:0/120966308 shutdown_connections 2023-10-11T09:43:53.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.714+0000 7f9ee0a71700 1 --2- 172.21.15.78:0/120966308 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9ebc065ed0 0x7f9ebc068380 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:53.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.714+0000 7f9ee0a71700 1 --2- 172.21.15.78:0/120966308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9edc0d21e0 0x7f9edc151da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:53.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.714+0000 7f9ee0a71700 1 --2- 172.21.15.78:0/120966308 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9edc0b4b60 0x7f9edc0c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:53.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.714+0000 7f9ee0a71700 1 --2- 172.21.15.78:0/120966308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9edc0b2310 0x7f9edc1f5a30 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:53.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.714+0000 7f9ee0a71700 1 -- 172.21.15.78:0/120966308 >> 172.21.15.78:0/120966308 conn(0x7f9edc0cfb60 msgr2=0x7f9edc06af50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:53.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.714+0000 7f9ee0a71700 1 -- 172.21.15.78:0/120966308 shutdown_connections 2023-10-11T09:43:53.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.714+0000 7f9ee0a71700 1 -- 172.21.15.78:0/120966308 wait complete. 2023-10-11T09:43:53.740 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-11T09:43:53.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.910+0000 7f23dcd16700 1 Processor -- start 2023-10-11T09:43:53.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.910+0000 7f23dcd16700 1 -- start start 2023-10-11T09:43:53.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.914+0000 7f23dcd16700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f23d8164d20 0x7f23d806bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:53.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.914+0000 7f23dcd16700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23d8165390 con 0x7f23d80cda10 2023-10-11T09:43:53.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.914+0000 7f23dcd16700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23d80ca440 con 0x7f23d80caf20 2023-10-11T09:43:53.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.914+0000 7f23dcd16700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23d80cb3a0 con 0x7f23d8164d20 2023-10-11T09:43:53.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.914+0000 7f23d77fe700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f23d80cda10 0x7f23d80cdde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:42242/0 (socket says 172.21.15.78:42242) 2023-10-11T09:43:53.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.914+0000 7f23d77fe700 1 -- 172.21.15.78:0/4113303450 learned_addr learned my addr 172.21.15.78:0/4113303450 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:43:53.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.914+0000 7f23d7fff700 1 --2- 172.21.15.78:0/4113303450 >> v2:172.21.15.133:3300/0 conn(0x7f23d8164d20 0x7f23d806bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:53.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.914+0000 7f23d67fc700 1 -- 172.21.15.78:0/4113303450 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3603309042 0 0) 0x7f23d8165390 con 0x7f23d80cda10 2023-10-11T09:43:53.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.914+0000 7f23d67fc700 1 -- 172.21.15.78:0/4113303450 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23c00035d0 con 0x7f23d80cda10 2023-10-11T09:43:53.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.914+0000 7f23d7fff700 1 -- 172.21.15.78:0/4113303450 >> v1:172.21.15.133:6789/0 conn(0x7f23d80caf20 legacy=0x7f23d8164610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:53.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.914+0000 7f23d7fff700 1 -- 172.21.15.78:0/4113303450 >> v1:172.21.15.78:6789/0 conn(0x7f23d80cda10 legacy=0x7f23d80cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:53.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.914+0000 7f23d7fff700 1 -- 172.21.15.78:0/4113303450 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d80ca720 con 0x7f23d8164d20 2023-10-11T09:43:53.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.914+0000 7f23d7fff700 1 --2- 172.21.15.78:0/4113303450 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23d8164d20 0x7f23d806bbf0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f23d0009d90 tx=0x7f23d000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6100cb65728deca server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:53.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.914+0000 7f23d67fc700 1 -- 172.21.15.78:0/4113303450 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d000e070 con 0x7f23d8164d20 2023-10-11T09:43:53.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.914+0000 7f23d67fc700 1 -- 172.21.15.78:0/4113303450 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d000bb20 con 0x7f23d8164d20 2023-10-11T09:43:53.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.918+0000 7f23d67fc700 1 -- 172.21.15.78:0/4113303450 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d0012430 con 0x7f23d8164d20 2023-10-11T09:43:53.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.918+0000 7f23dcd16700 1 -- 172.21.15.78:0/4113303450 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23d8164d20 msgr2=0x7f23d806bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:53.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.918+0000 7f23dcd16700 1 --2- 172.21.15.78:0/4113303450 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23d8164d20 0x7f23d806bbf0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f23d0009d90 tx=0x7f23d000b330 comp rx=0 tx=0).stop 2023-10-11T09:43:53.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.918+0000 7f23dcd16700 1 -- 172.21.15.78:0/4113303450 shutdown_connections 2023-10-11T09:43:53.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.918+0000 7f23dcd16700 1 --2- 172.21.15.78:0/4113303450 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23d8164d20 0x7f23d806bbf0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:53.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.918+0000 7f23dcd16700 1 -- 172.21.15.78:0/4113303450 >> 172.21.15.78:0/4113303450 conn(0x7f23d80cea60 msgr2=0x7f23d80cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:53.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.918+0000 7f23dcd16700 1 -- 172.21.15.78:0/4113303450 shutdown_connections 2023-10-11T09:43:53.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.918+0000 7f23dcd16700 1 -- 172.21.15.78:0/4113303450 wait complete. 2023-10-11T09:43:53.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.922+0000 7f23dcd16700 1 Processor -- start 2023-10-11T09:43:53.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.922+0000 7f23dcd16700 1 -- start start 2023-10-11T09:43:53.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.922+0000 7f23dcd16700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23d80caf20 0x7f23d8146bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:53.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.922+0000 7f23dcd16700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23d80cda10 0x7f23d80b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:53.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.922+0000 7f23dcd16700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23d8164d20 0x7f23d81f5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:53.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.922+0000 7f23dcd16700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d80b4260 con 0x7f23d8164d20 2023-10-11T09:43:53.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.922+0000 7f23dcd16700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d81472d0 con 0x7f23d80cda10 2023-10-11T09:43:53.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.922+0000 7f23dcd16700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d81475b0 con 0x7f23d80caf20 2023-10-11T09:43:53.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.922+0000 7f23d7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23d8164d20 0x7f23d81f5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:53.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.922+0000 7f23d7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23d8164d20 0x7f23d81f5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50362/0 (socket says 172.21.15.78:50362) 2023-10-11T09:43:53.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.922+0000 7f23d7fff700 1 -- 172.21.15.78:0/3341329693 learned_addr learned my addr 172.21.15.78:0/3341329693 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:53.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.922+0000 7f23d6ffd700 1 --2- 172.21.15.78:0/3341329693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23d80cda10 0x7f23d80b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:53.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.922+0000 7f23d77fe700 1 --2- 172.21.15.78:0/3341329693 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23d80caf20 0x7f23d8146bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:53.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.922+0000 7f23d7fff700 1 -- 172.21.15.78:0/3341329693 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23d80caf20 msgr2=0x7f23d8146bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:53.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.926+0000 7f23d7fff700 1 --2- 172.21.15.78:0/3341329693 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23d80caf20 0x7f23d8146bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:53.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.926+0000 7f23d7fff700 1 -- 172.21.15.78:0/3341329693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23d80cda10 msgr2=0x7f23d80b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:53.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.926+0000 7f23d7fff700 1 --2- 172.21.15.78:0/3341329693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23d80cda10 0x7f23d80b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:53.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.926+0000 7f23d7fff700 1 -- 172.21.15.78:0/3341329693 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d0009a70 con 0x7f23d8164d20 2023-10-11T09:43:53.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.926+0000 7f23d7fff700 1 --2- 172.21.15.78:0/3341329693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23d8164d20 0x7f23d81f5a50 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f23d8072690 tx=0x7f23d0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:53.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.926+0000 7f23bffff700 1 -- 172.21.15.78:0/3341329693 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d0012430 con 0x7f23d8164d20 2023-10-11T09:43:53.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.926+0000 7f23bffff700 1 -- 172.21.15.78:0/3341329693 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d0020070 con 0x7f23d8164d20 2023-10-11T09:43:53.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.926+0000 7f23bffff700 1 -- 172.21.15.78:0/3341329693 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d0004280 con 0x7f23d8164d20 2023-10-11T09:43:53.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.926+0000 7f23dcd16700 1 -- 172.21.15.78:0/3341329693 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d8147890 con 0x7f23d8164d20 2023-10-11T09:43:53.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.926+0000 7f23dcd16700 1 -- 172.21.15.78:0/3341329693 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d8147e50 con 0x7f23d8164d20 2023-10-11T09:43:53.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.926+0000 7f23dcd16700 1 -- 172.21.15.78:0/3341329693 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23d8065d40 con 0x7f23d8164d20 2023-10-11T09:43:53.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.930+0000 7f23bffff700 1 -- 172.21.15.78:0/3341329693 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f23d00043e0 con 0x7f23d8164d20 2023-10-11T09:43:53.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.930+0000 7f23bffff700 1 --2- 172.21.15.78:0/3341329693 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f23c4065ed0 0x7f23c4068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:53.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.930+0000 7f23bffff700 1 -- 172.21.15.78:0/3341329693 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6753+0+0 (secure 0 0 0) 0x7f23d0022070 con 0x7f23d8164d20 2023-10-11T09:43:53.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.930+0000 7f23d77fe700 1 --2- 172.21.15.78:0/3341329693 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f23c4065ed0 0x7f23c4068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:53.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.930+0000 7f23d77fe700 1 --2- 172.21.15.78:0/3341329693 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f23c4065ed0 0x7f23c4068380 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f23c8007fd0 tx=0x7f23c8007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:53.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:53.938+0000 7f23bffff700 1 -- 172.21.15.78:0/3341329693 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f23d00529e0 con 0x7f23d8164d20 2023-10-11T09:43:54.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.146+0000 7f23dcd16700 1 -- 172.21.15.78:0/3341329693 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f23d80b1480 con 0x7f23d8164d20 2023-10-11T09:43:54.155 INFO:tasks.ceph.mon.a.smithi078.stderr:2023-10-11T09:43:54.150+0000 7f90131f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-11T09:43:54.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.166+0000 7f23bffff700 1 -- 172.21.15.78:0/3341329693 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v30) v1 ==== 161+0+0 (secure 0 0 0) 0x7f23d0056000 con 0x7f23d8164d20 2023-10-11T09:43:54.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.170+0000 7f23dcd16700 1 -- 172.21.15.78:0/3341329693 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f23c4065ed0 msgr2=0x7f23c4068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:54.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.170+0000 7f23dcd16700 1 --2- 172.21.15.78:0/3341329693 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f23c4065ed0 0x7f23c4068380 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f23c8007fd0 tx=0x7f23c8007de0 comp rx=0 tx=0).stop 2023-10-11T09:43:54.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.170+0000 7f23dcd16700 1 -- 172.21.15.78:0/3341329693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23d8164d20 msgr2=0x7f23d81f5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:54.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.170+0000 7f23dcd16700 1 --2- 172.21.15.78:0/3341329693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23d8164d20 0x7f23d81f5a50 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f23d8072690 tx=0x7f23d0004af0 comp rx=0 tx=0).stop 2023-10-11T09:43:54.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.170+0000 7f23dcd16700 1 -- 172.21.15.78:0/3341329693 shutdown_connections 2023-10-11T09:43:54.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.170+0000 7f23dcd16700 1 --2- 172.21.15.78:0/3341329693 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f23c4065ed0 0x7f23c4068380 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:54.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.170+0000 7f23dcd16700 1 --2- 172.21.15.78:0/3341329693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23d8164d20 0x7f23d81f5a50 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:54.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.170+0000 7f23dcd16700 1 --2- 172.21.15.78:0/3341329693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23d80cda10 0x7f23d80b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:54.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.170+0000 7f23dcd16700 1 --2- 172.21.15.78:0/3341329693 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23d80caf20 0x7f23d8146bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:54.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.170+0000 7f23dcd16700 1 -- 172.21.15.78:0/3341329693 >> 172.21.15.78:0/3341329693 conn(0x7f23d80cea60 msgr2=0x7f23d8070500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:54.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.170+0000 7f23dcd16700 1 -- 172.21.15.78:0/3341329693 shutdown_connections 2023-10-11T09:43:54.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.170+0000 7f23dcd16700 1 -- 172.21.15.78:0/3341329693 wait complete. 2023-10-11T09:43:54.177 INFO:teuthology.orchestra.run.smithi078.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-11T09:43:54.182 INFO:tasks.ceph.mds.c.smithi078.stderr: -5799> 2023-10-11T09:42:51.716+0000 7f6906d7f700 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-10-11T09:43:54.186 INFO:tasks.ceph.mds.c.smithi078.stderr: -4343> 2023-10-11T09:42:56.707+0000 7f690a586700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-11T09:43:54.187 INFO:tasks.ceph.mds.c.smithi078.stderr: -3955> 2023-10-11T09:42:59.103+0000 7f6906d7f700 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2023-10-11T09:43:54.188 INFO:tasks.ceph.mds.c.smithi078.stderr: -3786> 2023-10-11T09:42:59.103+0000 7f6906d7f700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /subdir/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x564d6ceb5180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(5)0x10000000000:[<0x1/subdir v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-10-11T09:43:54.188 INFO:tasks.ceph.mds.c.smithi078.stderr: -3649> 2023-10-11T09:42:59.107+0000 7f6906d7f700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f6 [2,head] /subdir/1 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x564d6db89600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(7)0x100000001f6:[<0x10000000000/1 v24>,<0x1/subdir v46>]//[6]","memoryvalue":"(7)0x100000001f6:[<0x10000000000/1 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:43:54.189 INFO:tasks.ceph.mds.c.smithi078.stderr: -3626> 2023-10-11T09:42:59.107+0000 7f6906d7f700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f8 [2,head] /subdir/3 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x564d6ddbcc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(7)0x100000001f8:[<0x10000000000/3 v40>,<0x1/subdir v46>]//[6]","memoryvalue":"(7)0x100000001f8:[<0x10000000000/3 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:43:54.189 INFO:tasks.ceph.mds.c.smithi078.stderr: -3610> 2023-10-11T09:42:59.107+0000 7f6906d7f700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f7 [2,head] /subdir/2 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x564d6db89080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(7)0x100000001f7:[<0x10000000000/2 v32>,<0x1/subdir v46>]//[6]","memoryvalue":"(7)0x100000001f7:[<0x10000000000/2 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:43:54.189 INFO:tasks.ceph.mds.c.smithi078.stderr: -3593> 2023-10-11T09:42:59.107+0000 7f6906d7f700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f9 [2,head] /subdir/4 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x564d6ddbc680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(7)0x100000001f9:[<0x10000000000/4 v42>,<0x1/subdir v46>]//[6]","memoryvalue":"(7)0x100000001f9:[<0x10000000000/4 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:43:54.189 INFO:tasks.ceph.mds.c.smithi078.stderr: -3575> 2023-10-11T09:42:59.107+0000 7f6906d7f700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/0 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x564d6db89b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(7)0x10000000001:[<0x10000000000/0 v20>,<0x1/subdir v46>]//[6]","memoryvalue":"(7)0x10000000001:[<0x10000000000/0 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:43:54.190 INFO:tasks.ceph.mds.c.smithi078.stderr: -3220> 2023-10-11T09:42:59.111+0000 7f6906d7f700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v5 ap=5 snaprealm=0x564d6dd44b40 DIRTYPARENT f(v1 1=0+1) n(v0 rc2023-10-11T09:42:58.806279+0000 1=0+1)/n() (idft sync r=1) (inest sync r=1) (ifile sync->mix r=1) (iversion lock) caps={5985=pLsXs/-@3} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x564d6dd47600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(5)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=0+1)","ondisk_value.rstat":"n(v0 2=0+2)","memory_value.dirstat":"f(v1 1=0+1)","memory_value.rstat":"n(v0 rc2023-10-11T09:42:58.806279+0000 1=0+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-10-11T09:43:54.196 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-11T09:43:54.246 INFO:tasks.ceph.mds.c.smithi078.stdout:starting mds.c at 2023-10-11T09:43:54.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.362+0000 7f039f129700 1 Processor -- start 2023-10-11T09:43:54.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.362+0000 7f039f129700 1 -- start start 2023-10-11T09:43:54.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.362+0000 7f039f129700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0398161d90 0x7f03980b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:54.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.362+0000 7f039f129700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f03980b3520 0x7f03981629e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:54.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.362+0000 7f039f129700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f039806ccb0 con 0x7f0398146fe0 2023-10-11T09:43:54.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.362+0000 7f039f129700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f039806c410 con 0x7f0398161d90 2023-10-11T09:43:54.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.362+0000 7f039f129700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f039806c6f0 con 0x7f03980b3520 2023-10-11T09:43:54.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.362+0000 7f039d926700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0398161d90 0x7f03980b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:54.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.362+0000 7f039d926700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0398161d90 0x7f03980b2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50392/0 (socket says 172.21.15.78:50392) 2023-10-11T09:43:54.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.362+0000 7f039d926700 1 -- 172.21.15.78:0/2296469535 learned_addr learned my addr 172.21.15.78:0/2296469535 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:54.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.366+0000 7f039d926700 1 -- 172.21.15.78:0/2296469535 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f03980b3520 msgr2=0x7f03981629e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:54.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.366+0000 7f039d926700 1 --2- 172.21.15.78:0/2296469535 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f03980b3520 0x7f03981629e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:54.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.366+0000 7f039d926700 1 -- 172.21.15.78:0/2296469535 >> v1:172.21.15.78:6789/0 conn(0x7f0398146fe0 legacy=0x7f039806bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:54.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.366+0000 7f039d926700 1 -- 172.21.15.78:0/2296469535 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039806c9d0 con 0x7f0398161d90 2023-10-11T09:43:54.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.366+0000 7f039d926700 1 --2- 172.21.15.78:0/2296469535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0398161d90 0x7f03980b2fe0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f0388009910 tx=0x7f0388009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b856c673b2034d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:54.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.366+0000 7f039d125700 1 -- 172.21.15.78:0/2296469535 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038800ba60 con 0x7f0398161d90 2023-10-11T09:43:54.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.366+0000 7f039d125700 1 -- 172.21.15.78:0/2296469535 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f038800bbc0 con 0x7f0398161d90 2023-10-11T09:43:54.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.366+0000 7f039d125700 1 -- 172.21.15.78:0/2296469535 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0388009e60 con 0x7f0398161d90 2023-10-11T09:43:54.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.366+0000 7f039f129700 1 -- 172.21.15.78:0/2296469535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0398161d90 msgr2=0x7f03980b2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:54.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.366+0000 7f039f129700 1 --2- 172.21.15.78:0/2296469535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0398161d90 0x7f03980b2fe0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f0388009910 tx=0x7f0388009c20 comp rx=0 tx=0).stop 2023-10-11T09:43:54.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.366+0000 7f039f129700 1 -- 172.21.15.78:0/2296469535 shutdown_connections 2023-10-11T09:43:54.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.366+0000 7f039f129700 1 --2- 172.21.15.78:0/2296469535 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f03980b3520 0x7f03981629e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:54.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.366+0000 7f039f129700 1 --2- 172.21.15.78:0/2296469535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0398161d90 0x7f03980b2fe0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:54.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.366+0000 7f039f129700 1 -- 172.21.15.78:0/2296469535 >> 172.21.15.78:0/2296469535 conn(0x7f039806a060 msgr2=0x7f039806a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:54.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.366+0000 7f039f129700 1 -- 172.21.15.78:0/2296469535 shutdown_connections 2023-10-11T09:43:54.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.366+0000 7f039f129700 1 -- 172.21.15.78:0/2296469535 wait complete. 2023-10-11T09:43:54.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.370+0000 7f039f129700 1 Processor -- start 2023-10-11T09:43:54.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.370+0000 7f039f129700 1 -- start start 2023-10-11T09:43:54.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.370+0000 7f039f129700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f03980b3520 0x7f0398145fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:54.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.370+0000 7f039f129700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0398146fe0 0x7f03980af6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:54.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.370+0000 7f039f129700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0398161d90 0x7f03980afc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:54.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.370+0000 7f039f129700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03981f6340 con 0x7f0398146fe0 2023-10-11T09:43:54.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.370+0000 7f039f129700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03981464f0 con 0x7f0398161d90 2023-10-11T09:43:54.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.370+0000 7f039f129700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f039815cf00 con 0x7f03980b3520 2023-10-11T09:43:54.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.374+0000 7f039d926700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0398146fe0 0x7f03980af6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:54.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.374+0000 7f039d926700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0398146fe0 0x7f03980af6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50396/0 (socket says 172.21.15.78:50396) 2023-10-11T09:43:54.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.374+0000 7f039d926700 1 -- 172.21.15.78:0/845126075 learned_addr learned my addr 172.21.15.78:0/845126075 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:54.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.374+0000 7f039e127700 1 --2- 172.21.15.78:0/845126075 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f03980b3520 0x7f0398145fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:54.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.374+0000 7f039e928700 1 --2- 172.21.15.78:0/845126075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0398161d90 0x7f03980afc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:54.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.374+0000 7f039d926700 1 -- 172.21.15.78:0/845126075 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f03980b3520 msgr2=0x7f0398145fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:54.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.374+0000 7f039d926700 1 --2- 172.21.15.78:0/845126075 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f03980b3520 0x7f0398145fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:54.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.374+0000 7f039d926700 1 -- 172.21.15.78:0/845126075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0398161d90 msgr2=0x7f03980afc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:54.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.374+0000 7f039d926700 1 --2- 172.21.15.78:0/845126075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0398161d90 0x7f03980afc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:54.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.374+0000 7f039d926700 1 -- 172.21.15.78:0/845126075 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0388009570 con 0x7f0398146fe0 2023-10-11T09:43:54.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.374+0000 7f039e928700 1 --2- 172.21.15.78:0/845126075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0398161d90 0x7f03980afc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:43:54.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.374+0000 7f039d926700 1 --2- 172.21.15.78:0/845126075 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0398146fe0 0x7f03980af6d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f0388009c60 tx=0x7f0388004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:54.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.374+0000 7f038effd700 1 -- 172.21.15.78:0/845126075 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0388004480 con 0x7f0398146fe0 2023-10-11T09:43:54.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.374+0000 7f038effd700 1 -- 172.21.15.78:0/845126075 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f038801f070 con 0x7f0398146fe0 2023-10-11T09:43:54.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.374+0000 7f038effd700 1 -- 172.21.15.78:0/845126075 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03880038e0 con 0x7f0398146fe0 2023-10-11T09:43:54.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.374+0000 7f039f129700 1 -- 172.21.15.78:0/845126075 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f039815d1e0 con 0x7f0398146fe0 2023-10-11T09:43:54.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.374+0000 7f039f129700 1 -- 172.21.15.78:0/845126075 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f039815d7a0 con 0x7f0398146fe0 2023-10-11T09:43:54.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.374+0000 7f039f129700 1 -- 172.21.15.78:0/845126075 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03981f64b0 con 0x7f0398146fe0 2023-10-11T09:43:54.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.378+0000 7f038effd700 1 -- 172.21.15.78:0/845126075 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f0388021030 con 0x7f0398146fe0 2023-10-11T09:43:54.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.378+0000 7f038effd700 1 --2- 172.21.15.78:0/845126075 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f037c065f00 0x7f037c0683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:54.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.378+0000 7f038effd700 1 -- 172.21.15.78:0/845126075 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6839+0+0 (secure 0 0 0) 0x7f0388084ff0 con 0x7f0398146fe0 2023-10-11T09:43:54.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.378+0000 7f039e127700 1 --2- 172.21.15.78:0/845126075 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f037c065f00 0x7f037c0683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:54.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.382+0000 7f039e127700 1 --2- 172.21.15.78:0/845126075 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f037c065f00 0x7f037c0683b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0394000f60 tx=0x7f0394009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:54.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.386+0000 7f038effd700 1 -- 172.21.15.78:0/845126075 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f0388052320 con 0x7f0398146fe0 2023-10-11T09:43:54.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:54.606+0000 7f039f129700 1 -- 172.21.15.78:0/845126075 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0398065d40 con 0x7f0398146fe0 2023-10-11T09:43:55.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.202+0000 7f038effd700 1 -- 172.21.15.78:0/845126075 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v31) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0388055940 con 0x7f0398146fe0 2023-10-11T09:43:55.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.206+0000 7f039f129700 1 -- 172.21.15.78:0/845126075 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f037c065f00 msgr2=0x7f037c0683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:55.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.206+0000 7f039f129700 1 --2- 172.21.15.78:0/845126075 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f037c065f00 0x7f037c0683b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0394000f60 tx=0x7f0394009040 comp rx=0 tx=0).stop 2023-10-11T09:43:55.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.206+0000 7f039f129700 1 -- 172.21.15.78:0/845126075 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0398146fe0 msgr2=0x7f03980af6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:55.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.206+0000 7f039f129700 1 --2- 172.21.15.78:0/845126075 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0398146fe0 0x7f03980af6d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f0388009c60 tx=0x7f0388004770 comp rx=0 tx=0).stop 2023-10-11T09:43:55.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.206+0000 7f039f129700 1 -- 172.21.15.78:0/845126075 shutdown_connections 2023-10-11T09:43:55.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.206+0000 7f039f129700 1 --2- 172.21.15.78:0/845126075 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f037c065f00 0x7f037c0683b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:55.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.206+0000 7f039f129700 1 --2- 172.21.15.78:0/845126075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0398161d90 0x7f03980afc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:55.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.206+0000 7f039f129700 1 --2- 172.21.15.78:0/845126075 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0398146fe0 0x7f03980af6d0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:55.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.206+0000 7f039f129700 1 --2- 172.21.15.78:0/845126075 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f03980b3520 0x7f0398145fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:55.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.206+0000 7f039f129700 1 -- 172.21.15.78:0/845126075 >> 172.21.15.78:0/845126075 conn(0x7f039806a060 msgr2=0x7f0398070e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:55.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.206+0000 7f039f129700 1 -- 172.21.15.78:0/845126075 shutdown_connections 2023-10-11T09:43:55.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.206+0000 7f039f129700 1 -- 172.21.15.78:0/845126075 wait complete. 2023-10-11T09:43:55.234 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-11T09:43:55.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.406+0000 7f83a7bc6700 1 Processor -- start 2023-10-11T09:43:55.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.406+0000 7f83a7bc6700 1 -- start start 2023-10-11T09:43:55.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.410+0000 7f83a7bc6700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f83a01649c0 0x7f83a0164e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:55.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.410+0000 7f83a7bc6700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83a00cd3e0 con 0x7f83a006c630 2023-10-11T09:43:55.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.410+0000 7f83a7bc6700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83a00710f0 con 0x7f83a0160d90 2023-10-11T09:43:55.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.410+0000 7f83a7bc6700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83a00cb8f0 con 0x7f83a01649c0 2023-10-11T09:43:55.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.410+0000 7f83a5161700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f83a006c630 0x7f83a00c2180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:46326/0 (socket says 172.21.15.78:46326) 2023-10-11T09:43:55.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.410+0000 7f83a5161700 1 -- 172.21.15.78:0/2129083721 learned_addr learned my addr 172.21.15.78:0/2129083721 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:43:55.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.410+0000 7f83a6163700 1 --2- 172.21.15.78:0/2129083721 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f83a01649c0 0x7f83a0164e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:55.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.410+0000 7f83a4960700 1 -- 172.21.15.78:0/2129083721 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2445111758 0 0) 0x7f83a00cd3e0 con 0x7f83a006c630 2023-10-11T09:43:55.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.410+0000 7f83a4960700 1 -- 172.21.15.78:0/2129083721 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f838c0035d0 con 0x7f83a006c630 2023-10-11T09:43:55.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.410+0000 7f83a4960700 1 -- 172.21.15.78:0/2129083721 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1534272919 0 0) 0x7f838c0035d0 con 0x7f83a006c630 2023-10-11T09:43:55.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.410+0000 7f83a4960700 1 -- 172.21.15.78:0/2129083721 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f83a01649c0 msgr2=0x7f83a0164e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:55.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.410+0000 7f83a4960700 1 --2- 172.21.15.78:0/2129083721 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f83a01649c0 0x7f83a0164e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:55.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.410+0000 7f83a4960700 1 -- 172.21.15.78:0/2129083721 >> v1:172.21.15.133:6789/0 conn(0x7f83a0160d90 legacy=0x7f83a00c1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:55.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.410+0000 7f83a4960700 1 -- 172.21.15.78:0/2129083721 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a0072300 con 0x7f83a006c630 2023-10-11T09:43:55.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.410+0000 7f83a4960700 1 -- 172.21.15.78:0/2129083721 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f8390001810 con 0x7f83a006c630 2023-10-11T09:43:55.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.410+0000 7f83a4960700 1 -- 172.21.15.78:0/2129083721 >> v1:172.21.15.78:6789/0 conn(0x7f83a006c630 legacy=0x7f83a00c2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:55.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.410+0000 7f83a4960700 1 --2- 172.21.15.78:0/2129083721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f838c003a50 0x7f838c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:55.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.410+0000 7f83a4960700 1 --2- 172.21.15.78:0/2129083721 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f83a01649c0 0x7f838c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:55.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.410+0000 7f83a4960700 1 --2- 172.21.15.78:0/2129083721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f838c004db0 0x7f838c005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:55.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.410+0000 7f83a4960700 1 -- 172.21.15.78:0/2129083721 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a00cb8f0 con 0x7f838c004db0 2023-10-11T09:43:55.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.410+0000 7f83a4960700 1 -- 172.21.15.78:0/2129083721 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838c00bb10 con 0x7f838c003a50 2023-10-11T09:43:55.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.410+0000 7f83a4960700 1 -- 172.21.15.78:0/2129083721 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f838c00bcd0 con 0x7f83a01649c0 2023-10-11T09:43:55.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.410+0000 7f83a5161700 1 --2- 172.21.15.78:0/2129083721 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f83a01649c0 0x7f838c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:55.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.410+0000 7f83a6163700 1 --2- 172.21.15.78:0/2129083721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f838c004db0 0x7f838c005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:55.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.410+0000 7f83a5962700 1 --2- 172.21.15.78:0/2129083721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f838c003a50 0x7f838c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:55.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.414+0000 7f83a5161700 1 -- 172.21.15.78:0/2129083721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f838c003a50 msgr2=0x7f838c003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:55.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.414+0000 7f83a5161700 1 --2- 172.21.15.78:0/2129083721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f838c003a50 0x7f838c003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:55.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.414+0000 7f83a5161700 1 -- 172.21.15.78:0/2129083721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f838c004db0 msgr2=0x7f838c005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:55.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.414+0000 7f83a5161700 1 --2- 172.21.15.78:0/2129083721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f838c004db0 0x7f838c005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:55.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.414+0000 7f83a5161700 1 -- 172.21.15.78:0/2129083721 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f838c0035d0 con 0x7f83a01649c0 2023-10-11T09:43:55.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.414+0000 7f83a5161700 1 --2- 172.21.15.78:0/2129083721 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f83a01649c0 0x7f838c004870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f839000c160 tx=0x7f838c0058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e533b37435c9f8c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:55.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.414+0000 7f83a4960700 1 -- 172.21.15.78:0/2129083721 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839000cd00 con 0x7f83a01649c0 2023-10-11T09:43:55.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.414+0000 7f83a4960700 1 -- 172.21.15.78:0/2129083721 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839000ce60 con 0x7f83a01649c0 2023-10-11T09:43:55.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.414+0000 7f83a4960700 1 -- 172.21.15.78:0/2129083721 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8390005a70 con 0x7f83a01649c0 2023-10-11T09:43:55.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.414+0000 7f83a7bc6700 1 -- 172.21.15.78:0/2129083721 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f83a01649c0 msgr2=0x7f838c004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:55.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.414+0000 7f83a7bc6700 1 --2- 172.21.15.78:0/2129083721 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f83a01649c0 0x7f838c004870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f839000c160 tx=0x7f838c0058a0 comp rx=0 tx=0).stop 2023-10-11T09:43:55.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.414+0000 7f83a6163700 1 -- 172.21.15.78:0/2129083721 reap_dead start 2023-10-11T09:43:55.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.414+0000 7f83a7bc6700 1 -- 172.21.15.78:0/2129083721 shutdown_connections 2023-10-11T09:43:55.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.414+0000 7f83a7bc6700 1 -- 172.21.15.78:0/2129083721 >> 172.21.15.78:0/2129083721 conn(0x7f83a006a060 msgr2=0x7f83a006a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:55.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.414+0000 7f83a7bc6700 1 -- 172.21.15.78:0/2129083721 shutdown_connections 2023-10-11T09:43:55.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.414+0000 7f83a7bc6700 1 -- 172.21.15.78:0/2129083721 wait complete. 2023-10-11T09:43:55.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.418+0000 7f83a7bc6700 1 Processor -- start 2023-10-11T09:43:55.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.418+0000 7f83a7bc6700 1 -- start start 2023-10-11T09:43:55.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.418+0000 7f83a7bc6700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83a01649c0 0x7f83a00c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:55.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.418+0000 7f83a7bc6700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f83a01f6930 0x7f83a0162840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:55.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.418+0000 7f83a7bc6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83a01501e0 0x7f83a00ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:55.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.418+0000 7f83a7bc6700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a00c93d0 con 0x7f83a01501e0 2023-10-11T09:43:55.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.418+0000 7f83a7bc6700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a01f6ed0 con 0x7f83a01649c0 2023-10-11T09:43:55.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.418+0000 7f83a7bc6700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83a01f71b0 con 0x7f83a01f6930 2023-10-11T09:43:55.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.418+0000 7f83a6163700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83a01501e0 0x7f83a00ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:55.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.418+0000 7f83a6163700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83a01501e0 0x7f83a00ceb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35390/0 (socket says 172.21.15.78:35390) 2023-10-11T09:43:55.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.418+0000 7f83a6163700 1 -- 172.21.15.78:0/3596491125 learned_addr learned my addr 172.21.15.78:0/3596491125 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:55.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.422+0000 7f83a5962700 1 --2- 172.21.15.78:0/3596491125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83a01649c0 0x7f83a00c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:55.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.422+0000 7f83a5161700 1 --2- 172.21.15.78:0/3596491125 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f83a01f6930 0x7f83a0162840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:55.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.422+0000 7f83a6163700 1 -- 172.21.15.78:0/3596491125 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f83a01f6930 msgr2=0x7f83a0162840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:55.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.422+0000 7f83a6163700 1 --2- 172.21.15.78:0/3596491125 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f83a01f6930 0x7f83a0162840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:55.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.422+0000 7f83a6163700 1 -- 172.21.15.78:0/3596491125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83a01649c0 msgr2=0x7f83a00c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:55.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.422+0000 7f83a6163700 1 --2- 172.21.15.78:0/3596491125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83a01649c0 0x7f83a00c8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:55.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.422+0000 7f83a6163700 1 -- 172.21.15.78:0/3596491125 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a01f7490 con 0x7f83a01501e0 2023-10-11T09:43:55.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.422+0000 7f83a6163700 1 --2- 172.21.15.78:0/3596491125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83a01501e0 0x7f83a00ceb70 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f83a0164920 tx=0x7f8398011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:55.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.422+0000 7f83967fc700 1 -- 172.21.15.78:0/3596491125 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839800e040 con 0x7f83a01501e0 2023-10-11T09:43:55.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.422+0000 7f83967fc700 1 -- 172.21.15.78:0/3596491125 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839800f040 con 0x7f83a01501e0 2023-10-11T09:43:55.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.422+0000 7f83967fc700 1 -- 172.21.15.78:0/3596491125 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839800f8f0 con 0x7f83a01501e0 2023-10-11T09:43:55.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.422+0000 7f83a7bc6700 1 -- 172.21.15.78:0/3596491125 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83a01f7770 con 0x7f83a01501e0 2023-10-11T09:43:55.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.422+0000 7f83a7bc6700 1 -- 172.21.15.78:0/3596491125 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83a0162d80 con 0x7f83a01501e0 2023-10-11T09:43:55.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.422+0000 7f83a7bc6700 1 -- 172.21.15.78:0/3596491125 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83a01613a0 con 0x7f83a01501e0 2023-10-11T09:43:55.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.426+0000 7f83967fc700 1 -- 172.21.15.78:0/3596491125 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f839800fa50 con 0x7f83a01501e0 2023-10-11T09:43:55.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.426+0000 7f83967fc700 1 --2- 172.21.15.78:0/3596491125 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8384065ed0 0x7f8384068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:55.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.430+0000 7f83967fc700 1 -- 172.21.15.78:0/3596491125 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6839+0+0 (secure 0 0 0) 0x7f83980841d0 con 0x7f83a01501e0 2023-10-11T09:43:55.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.430+0000 7f83a5962700 1 --2- 172.21.15.78:0/3596491125 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8384065ed0 0x7f8384068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:55.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.430+0000 7f83a5962700 1 --2- 172.21.15.78:0/3596491125 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8384065ed0 0x7f8384068380 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f839c007fd0 tx=0x7f839c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:55.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.434+0000 7f83967fc700 1 -- 172.21.15.78:0/3596491125 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f83a01613a0 con 0x7f83a01501e0 2023-10-11T09:43:55.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:55.602+0000 7f83a7bc6700 1 -- 172.21.15.78:0/3596491125 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f83a01613a0 con 0x7f83a01501e0 2023-10-11T09:43:56.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.290+0000 7f83967fc700 1 -- 172.21.15.78:0/3596491125 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v50) v1 ==== 181+0+0 (secure 0 0 0) 0x7f83a01613a0 con 0x7f83a01501e0 2023-10-11T09:43:56.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.294+0000 7f83a7bc6700 1 -- 172.21.15.78:0/3596491125 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8384065ed0 msgr2=0x7f8384068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:56.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.294+0000 7f83a7bc6700 1 --2- 172.21.15.78:0/3596491125 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8384065ed0 0x7f8384068380 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f839c007fd0 tx=0x7f839c009040 comp rx=0 tx=0).stop 2023-10-11T09:43:56.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.294+0000 7f83a7bc6700 1 -- 172.21.15.78:0/3596491125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83a01501e0 msgr2=0x7f83a00ceb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:56.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.294+0000 7f83a7bc6700 1 --2- 172.21.15.78:0/3596491125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83a01501e0 0x7f83a00ceb70 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f83a0164920 tx=0x7f8398011f40 comp rx=0 tx=0).stop 2023-10-11T09:43:56.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.294+0000 7f83a7bc6700 1 -- 172.21.15.78:0/3596491125 shutdown_connections 2023-10-11T09:43:56.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.294+0000 7f83a7bc6700 1 --2- 172.21.15.78:0/3596491125 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8384065ed0 0x7f8384068380 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:56.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.294+0000 7f83a7bc6700 1 --2- 172.21.15.78:0/3596491125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83a01501e0 0x7f83a00ceb70 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:56.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.294+0000 7f83a7bc6700 1 --2- 172.21.15.78:0/3596491125 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f83a01f6930 0x7f83a0162840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:56.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.294+0000 7f83a7bc6700 1 --2- 172.21.15.78:0/3596491125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83a01649c0 0x7f83a00c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:56.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.294+0000 7f83a7bc6700 1 -- 172.21.15.78:0/3596491125 >> 172.21.15.78:0/3596491125 conn(0x7f83a006a060 msgr2=0x7f83a006a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:56.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.294+0000 7f83a7bc6700 1 -- 172.21.15.78:0/3596491125 shutdown_connections 2023-10-11T09:43:56.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.298+0000 7f83a7bc6700 1 -- 172.21.15.78:0/3596491125 wait complete. 2023-10-11T09:43:56.304 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' removed 2023-10-11T09:43:56.322 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-11T09:43:56.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.494+0000 7f3351e5e700 1 Processor -- start 2023-10-11T09:43:56.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.494+0000 7f3351e5e700 1 -- start start 2023-10-11T09:43:56.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.494+0000 7f3351e5e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f334c160d90 0x7f334c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:56.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.498+0000 7f3351e5e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f334c1649c0 0x7f334c164e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:56.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.498+0000 7f3351e5e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f334c0cd3e0 con 0x7f334c0c1a70 2023-10-11T09:43:56.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.498+0000 7f3351e5e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f334c0710f0 con 0x7f334c1649c0 2023-10-11T09:43:56.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.498+0000 7f3351e5e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f334c0cb8f0 con 0x7f334c160d90 2023-10-11T09:43:56.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.498+0000 7f3342ffd700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f334c0c1a70 0x7f334c0c1ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:46332/0 (socket says 172.21.15.78:46332) 2023-10-11T09:43:56.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.498+0000 7f3342ffd700 1 -- 172.21.15.78:0/3849956960 learned_addr learned my addr 172.21.15.78:0/3849956960 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:43:56.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.498+0000 7f334b7fe700 1 --2- 172.21.15.78:0/3849956960 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f334c160d90 0x7f334c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:56.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.498+0000 7f334bfff700 1 --2- 172.21.15.78:0/3849956960 >> v2:172.21.15.133:3300/0 conn(0x7f334c1649c0 0x7f334c164e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:56.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.498+0000 7f334affd700 1 -- 172.21.15.78:0/3849956960 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3875527959 0 0) 0x7f334c0cd3e0 con 0x7f334c0c1a70 2023-10-11T09:43:56.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.498+0000 7f334affd700 1 -- 172.21.15.78:0/3849956960 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33380035d0 con 0x7f334c0c1a70 2023-10-11T09:43:56.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.498+0000 7f334b7fe700 1 -- 172.21.15.78:0/3849956960 >> v2:172.21.15.133:3300/0 conn(0x7f334c1649c0 msgr2=0x7f334c164e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:56.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.498+0000 7f334b7fe700 1 --2- 172.21.15.78:0/3849956960 >> v2:172.21.15.133:3300/0 conn(0x7f334c1649c0 0x7f334c164e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:56.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.498+0000 7f334b7fe700 1 -- 172.21.15.78:0/3849956960 >> v1:172.21.15.78:6789/0 conn(0x7f334c0c1a70 legacy=0x7f334c0c1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:56.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.498+0000 7f334b7fe700 1 -- 172.21.15.78:0/3849956960 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f334c072300 con 0x7f334c160d90 2023-10-11T09:43:56.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.498+0000 7f334affd700 1 -- 172.21.15.78:0/3849956960 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2442933357 0 0) 0x7f33380035d0 con 0x7f334c0c1a70 2023-10-11T09:43:56.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.498+0000 7f334b7fe700 1 --2- 172.21.15.78:0/3849956960 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f334c160d90 0x7f334c06c630 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f333c009d90 tx=0x7f333c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed170c0753d69ff server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:56.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.498+0000 7f334affd700 1 -- 172.21.15.78:0/3849956960 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c00e070 con 0x7f334c160d90 2023-10-11T09:43:56.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.498+0000 7f334affd700 1 -- 172.21.15.78:0/3849956960 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f333c003a40 con 0x7f334c160d90 2023-10-11T09:43:56.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.498+0000 7f334affd700 1 -- 172.21.15.78:0/3849956960 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33340017f0 con 0x7f334c160d90 2023-10-11T09:43:56.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.498+0000 7f3351e5e700 1 -- 172.21.15.78:0/3849956960 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f334c160d90 msgr2=0x7f334c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:56.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.498+0000 7f3351e5e700 1 --2- 172.21.15.78:0/3849956960 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f334c160d90 0x7f334c06c630 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f333c009d90 tx=0x7f333c00b330 comp rx=0 tx=0).stop 2023-10-11T09:43:56.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.498+0000 7f3351e5e700 1 -- 172.21.15.78:0/3849956960 shutdown_connections 2023-10-11T09:43:56.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.498+0000 7f3351e5e700 1 --2- 172.21.15.78:0/3849956960 >> v2:172.21.15.133:3300/0 conn(0x7f334c1649c0 0x7f334c164e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:56.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.498+0000 7f3351e5e700 1 --2- 172.21.15.78:0/3849956960 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f334c160d90 0x7f334c06c630 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:56.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.498+0000 7f3351e5e700 1 -- 172.21.15.78:0/3849956960 >> 172.21.15.78:0/3849956960 conn(0x7f334c06a060 msgr2=0x7f334c06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:56.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.498+0000 7f3351e5e700 1 -- 172.21.15.78:0/3849956960 shutdown_connections 2023-10-11T09:43:56.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.498+0000 7f3351e5e700 1 -- 172.21.15.78:0/3849956960 wait complete. 2023-10-11T09:43:56.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.502+0000 7f3351e5e700 1 Processor -- start 2023-10-11T09:43:56.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.502+0000 7f3351e5e700 1 -- start start 2023-10-11T09:43:56.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.502+0000 7f3351e5e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f334c0c1a70 0x7f334c0c1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:56.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.502+0000 7f3351e5e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f334c160d90 0x7f334c0c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:56.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.502+0000 7f3351e5e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f334c1649c0 0x7f334c0c8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:56.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.502+0000 7f3351e5e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c0b0760 con 0x7f334c160d90 2023-10-11T09:43:56.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.502+0000 7f3351e5e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c0af880 con 0x7f334c0c1a70 2023-10-11T09:43:56.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.502+0000 7f3351e5e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f334c0afdc0 con 0x7f334c1649c0 2023-10-11T09:43:56.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.502+0000 7f3342ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f334c160d90 0x7f334c0c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:56.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.502+0000 7f3342ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f334c160d90 0x7f334c0c80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35398/0 (socket says 172.21.15.78:35398) 2023-10-11T09:43:56.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.502+0000 7f3342ffd700 1 -- 172.21.15.78:0/2881378533 learned_addr learned my addr 172.21.15.78:0/2881378533 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:56.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.502+0000 7f334b7fe700 1 --2- 172.21.15.78:0/2881378533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f334c0c1a70 0x7f334c0c1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:56.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.502+0000 7f334bfff700 1 --2- 172.21.15.78:0/2881378533 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f334c1649c0 0x7f334c0c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:56.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.506+0000 7f3342ffd700 1 -- 172.21.15.78:0/2881378533 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f334c1649c0 msgr2=0x7f334c0c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:56.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.506+0000 7f3342ffd700 1 --2- 172.21.15.78:0/2881378533 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f334c1649c0 0x7f334c0c8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:56.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.506+0000 7f3342ffd700 1 -- 172.21.15.78:0/2881378533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f334c0c1a70 msgr2=0x7f334c0c1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:56.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.506+0000 7f3342ffd700 1 --2- 172.21.15.78:0/2881378533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f334c0c1a70 0x7f334c0c1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:56.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.506+0000 7f3342ffd700 1 -- 172.21.15.78:0/2881378533 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333c009a70 con 0x7f334c160d90 2023-10-11T09:43:56.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.506+0000 7f334bfff700 1 --2- 172.21.15.78:0/2881378533 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f334c1649c0 0x7f334c0c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:43:56.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.506+0000 7f3342ffd700 1 --2- 172.21.15.78:0/2881378533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f334c160d90 0x7f334c0c80f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f333400c780 tx=0x7f333400ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:56.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.506+0000 7f3348ff9700 1 -- 172.21.15.78:0/2881378533 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33340074e0 con 0x7f334c160d90 2023-10-11T09:43:56.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.506+0000 7f3348ff9700 1 -- 172.21.15.78:0/2881378533 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3334007640 con 0x7f334c160d90 2023-10-11T09:43:56.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.506+0000 7f3348ff9700 1 -- 172.21.15.78:0/2881378533 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3334005a90 con 0x7f334c160d90 2023-10-11T09:43:56.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.506+0000 7f3351e5e700 1 -- 172.21.15.78:0/2881378533 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f334c0b00a0 con 0x7f334c160d90 2023-10-11T09:43:56.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.506+0000 7f3351e5e700 1 -- 172.21.15.78:0/2881378533 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f334c0c2420 con 0x7f334c160d90 2023-10-11T09:43:56.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.506+0000 7f3351e5e700 1 -- 172.21.15.78:0/2881378533 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f334c0b08a0 con 0x7f334c160d90 2023-10-11T09:43:56.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.506+0000 7f3348ff9700 1 -- 172.21.15.78:0/2881378533 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f3334005bf0 con 0x7f334c160d90 2023-10-11T09:43:56.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.510+0000 7f3348ff9700 1 --2- 172.21.15.78:0/2881378533 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f332c065f00 0x7f332c0683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:56.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.510+0000 7f3348ff9700 1 -- 172.21.15.78:0/2881378533 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6401+0+0 (secure 0 0 0) 0x7f3334082f50 con 0x7f334c160d90 2023-10-11T09:43:56.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.510+0000 7f334b7fe700 1 --2- 172.21.15.78:0/2881378533 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f332c065f00 0x7f332c0683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:56.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.510+0000 7f334b7fe700 1 --2- 172.21.15.78:0/2881378533 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f332c065f00 0x7f332c0683b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f333c00b370 tx=0x7f333c011000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:56.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.518+0000 7f3348ff9700 1 -- 172.21.15.78:0/2881378533 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f334c0b08a0 con 0x7f334c160d90 2023-10-11T09:43:56.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:56.690+0000 7f3351e5e700 1 -- 172.21.15.78:0/2881378533 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f334c0b08a0 con 0x7f334c160d90 2023-10-11T09:43:57.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.294+0000 7f3348ff9700 1 -- 172.21.15.78:0/2881378533 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v51) v1 ==== 169+0+0 (secure 0 0 0) 0x7f334c0b08a0 con 0x7f334c160d90 2023-10-11T09:43:57.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.302+0000 7f3351e5e700 1 -- 172.21.15.78:0/2881378533 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f332c065f00 msgr2=0x7f332c0683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:57.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.302+0000 7f3351e5e700 1 --2- 172.21.15.78:0/2881378533 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f332c065f00 0x7f332c0683b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f333c00b370 tx=0x7f333c011000 comp rx=0 tx=0).stop 2023-10-11T09:43:57.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.302+0000 7f3351e5e700 1 -- 172.21.15.78:0/2881378533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f334c160d90 msgr2=0x7f334c0c80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:57.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.302+0000 7f3351e5e700 1 --2- 172.21.15.78:0/2881378533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f334c160d90 0x7f334c0c80f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f333400c780 tx=0x7f333400ca90 comp rx=0 tx=0).stop 2023-10-11T09:43:57.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.302+0000 7f3351e5e700 1 -- 172.21.15.78:0/2881378533 shutdown_connections 2023-10-11T09:43:57.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.302+0000 7f3351e5e700 1 --2- 172.21.15.78:0/2881378533 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f332c065f00 0x7f332c0683b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:57.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.302+0000 7f3351e5e700 1 --2- 172.21.15.78:0/2881378533 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f334c1649c0 0x7f334c0c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:57.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.302+0000 7f3351e5e700 1 --2- 172.21.15.78:0/2881378533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f334c160d90 0x7f334c0c80f0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:57.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.302+0000 7f3351e5e700 1 --2- 172.21.15.78:0/2881378533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f334c0c1a70 0x7f334c0c1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:57.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.302+0000 7f3351e5e700 1 -- 172.21.15.78:0/2881378533 >> 172.21.15.78:0/2881378533 conn(0x7f334c06a060 msgr2=0x7f334c0c5710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:57.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.302+0000 7f3351e5e700 1 -- 172.21.15.78:0/2881378533 shutdown_connections 2023-10-11T09:43:57.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.302+0000 7f3351e5e700 1 -- 172.21.15.78:0/2881378533 wait complete. 2023-10-11T09:43:57.311 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' removed 2023-10-11T09:43:57.332 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_extra cephfs_data_extra --yes-i-really-really-mean-it 2023-10-11T09:43:57.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.514+0000 7fba18a43700 1 Processor -- start 2023-10-11T09:43:57.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.514+0000 7fba18a43700 1 -- start start 2023-10-11T09:43:57.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.518+0000 7fba18a43700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fba140cea30 0x7fba140cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:57.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.518+0000 7fba18a43700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fba140b04e0 0x7fba140cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:57.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.518+0000 7fba18a43700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fba1406dd80 0x7fba140cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:57.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.518+0000 7fba18a43700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba1406ee70 con 0x7fba140b04e0 2023-10-11T09:43:57.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.518+0000 7fba18a43700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba140cf340 con 0x7fba1406dd80 2023-10-11T09:43:57.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.518+0000 7fba18a43700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba1406e260 con 0x7fba140cea30 2023-10-11T09:43:57.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.518+0000 7fba11d9b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fba140b04e0 0x7fba140cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:57.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.518+0000 7fba11d9b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fba140b04e0 0x7fba140cb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35406/0 (socket says 172.21.15.78:35406) 2023-10-11T09:43:57.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.518+0000 7fba11d9b700 1 -- 172.21.15.78:0/3279405965 learned_addr learned my addr 172.21.15.78:0/3279405965 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:57.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.518+0000 7fba1259c700 1 --2- 172.21.15.78:0/3279405965 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fba140cea30 0x7fba140cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:57.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.518+0000 7fba12d9d700 1 --2- 172.21.15.78:0/3279405965 >> v2:172.21.15.133:3300/0 conn(0x7fba1406dd80 0x7fba140cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:57.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.518+0000 7fba11d9b700 1 -- 172.21.15.78:0/3279405965 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fba140cea30 msgr2=0x7fba140cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:57.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.518+0000 7fba11d9b700 1 --2- 172.21.15.78:0/3279405965 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fba140cea30 0x7fba140cee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:57.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.518+0000 7fba11d9b700 1 -- 172.21.15.78:0/3279405965 >> v2:172.21.15.133:3300/0 conn(0x7fba1406dd80 msgr2=0x7fba140cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:57.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.518+0000 7fba11d9b700 1 --2- 172.21.15.78:0/3279405965 >> v2:172.21.15.133:3300/0 conn(0x7fba1406dd80 0x7fba140cb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:57.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.518+0000 7fba11d9b700 1 -- 172.21.15.78:0/3279405965 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba1406e540 con 0x7fba140b04e0 2023-10-11T09:43:57.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.518+0000 7fba11d9b700 1 --2- 172.21.15.78:0/3279405965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba140b04e0 0x7fba140cb4b0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fba140cb440 tx=0x7fba08009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f347e29c1bc120c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:57.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.518+0000 7fba1159a700 1 -- 172.21.15.78:0/3279405965 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0800b8a0 con 0x7fba140b04e0 2023-10-11T09:43:57.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.518+0000 7fba1159a700 1 -- 172.21.15.78:0/3279405965 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba0800ba00 con 0x7fba140b04e0 2023-10-11T09:43:57.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.518+0000 7fba1159a700 1 -- 172.21.15.78:0/3279405965 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0800bd90 con 0x7fba140b04e0 2023-10-11T09:43:57.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.522+0000 7fba18a43700 1 -- 172.21.15.78:0/3279405965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba140b04e0 msgr2=0x7fba140cb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:57.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.522+0000 7fba18a43700 1 --2- 172.21.15.78:0/3279405965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba140b04e0 0x7fba140cb4b0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fba140cb440 tx=0x7fba08009bf0 comp rx=0 tx=0).stop 2023-10-11T09:43:57.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.522+0000 7fba18a43700 1 -- 172.21.15.78:0/3279405965 shutdown_connections 2023-10-11T09:43:57.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.522+0000 7fba18a43700 1 --2- 172.21.15.78:0/3279405965 >> v2:172.21.15.133:3300/0 conn(0x7fba1406dd80 0x7fba140cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:57.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.522+0000 7fba18a43700 1 --2- 172.21.15.78:0/3279405965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba140b04e0 0x7fba140cb4b0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:57.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.522+0000 7fba18a43700 1 --2- 172.21.15.78:0/3279405965 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fba140cea30 0x7fba140cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:57.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.522+0000 7fba18a43700 1 -- 172.21.15.78:0/3279405965 >> 172.21.15.78:0/3279405965 conn(0x7fba1409a1d0 msgr2=0x7fba140d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:57.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.522+0000 7fba18a43700 1 -- 172.21.15.78:0/3279405965 shutdown_connections 2023-10-11T09:43:57.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.522+0000 7fba18a43700 1 -- 172.21.15.78:0/3279405965 wait complete. 2023-10-11T09:43:57.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.526+0000 7fba18a43700 1 Processor -- start 2023-10-11T09:43:57.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.526+0000 7fba18a43700 1 -- start start 2023-10-11T09:43:57.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.526+0000 7fba18a43700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba1406dd80 0x7fba140ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:57.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.526+0000 7fba18a43700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fba140cea30 0x7fba1406c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:57.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.526+0000 7fba18a43700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba140b6960 0x7fba141f5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:57.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.526+0000 7fba18a43700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1406b170 con 0x7fba1406dd80 2023-10-11T09:43:57.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.526+0000 7fba18a43700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1406b410 con 0x7fba140b6960 2023-10-11T09:43:57.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.526+0000 7fba18a43700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba1406b6f0 con 0x7fba140cea30 2023-10-11T09:43:57.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.526+0000 7fba1259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba1406dd80 0x7fba140ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:57.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.526+0000 7fba1259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba1406dd80 0x7fba140ca6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35414/0 (socket says 172.21.15.78:35414) 2023-10-11T09:43:57.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.526+0000 7fba12d9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba140b6960 0x7fba141f5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:57.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.526+0000 7fba1259c700 1 -- 172.21.15.78:0/983674281 learned_addr learned my addr 172.21.15.78:0/983674281 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:57.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.526+0000 7fba11d9b700 1 --2- 172.21.15.78:0/983674281 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fba140cea30 0x7fba1406c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:57.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.526+0000 7fba1259c700 1 -- 172.21.15.78:0/983674281 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fba140cea30 msgr2=0x7fba1406c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:57.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.526+0000 7fba1259c700 1 --2- 172.21.15.78:0/983674281 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fba140cea30 0x7fba1406c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:57.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.526+0000 7fba1259c700 1 -- 172.21.15.78:0/983674281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba140b6960 msgr2=0x7fba141f5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:57.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.526+0000 7fba1259c700 1 --2- 172.21.15.78:0/983674281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba140b6960 0x7fba141f5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:57.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.526+0000 7fba1259c700 1 -- 172.21.15.78:0/983674281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba08009570 con 0x7fba1406dd80 2023-10-11T09:43:57.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.526+0000 7fba1259c700 1 --2- 172.21.15.78:0/983674281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba1406dd80 0x7fba140ca6b0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fba0400e810 tx=0x7fba0400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:57.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.526+0000 7fba02ffd700 1 -- 172.21.15.78:0/983674281 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0400c070 con 0x7fba1406dd80 2023-10-11T09:43:57.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.526+0000 7fba02ffd700 1 -- 172.21.15.78:0/983674281 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba04013610 con 0x7fba1406dd80 2023-10-11T09:43:57.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.526+0000 7fba02ffd700 1 -- 172.21.15.78:0/983674281 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba040139c0 con 0x7fba1406dd80 2023-10-11T09:43:57.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.526+0000 7fba18a43700 1 -- 172.21.15.78:0/983674281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba1406b9d0 con 0x7fba1406dd80 2023-10-11T09:43:57.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.530+0000 7fba18a43700 1 -- 172.21.15.78:0/983674281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba1406bf90 con 0x7fba1406dd80 2023-10-11T09:43:57.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.530+0000 7fba18a43700 1 -- 172.21.15.78:0/983674281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba1406c180 con 0x7fba1406dd80 2023-10-11T09:43:57.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.530+0000 7fba02ffd700 1 -- 172.21.15.78:0/983674281 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fba04013b20 con 0x7fba1406dd80 2023-10-11T09:43:57.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.534+0000 7fba02ffd700 1 --2- 172.21.15.78:0/983674281 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb9f4065ed0 0x7fb9f4068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:57.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.534+0000 7fba02ffd700 1 -- 172.21.15.78:0/983674281 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5958+0+0 (secure 0 0 0) 0x7fba04083610 con 0x7fba1406dd80 2023-10-11T09:43:57.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.534+0000 7fba11d9b700 1 --2- 172.21.15.78:0/983674281 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb9f4065ed0 0x7fb9f4068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:57.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.534+0000 7fba11d9b700 1 --2- 172.21.15.78:0/983674281 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb9f4065ed0 0x7fb9f4068380 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fba08000c00 tx=0x7fba08008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:57.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.538+0000 7fba02ffd700 1 -- 172.21.15.78:0/983674281 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fba04050c30 con 0x7fba1406dd80 2023-10-11T09:43:57.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:57.710+0000 7fba18a43700 1 -- 172.21.15.78:0/983674281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_extra", "pool2": "cephfs_data_extra", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fba1404e530 con 0x7fba1406dd80 2023-10-11T09:43:58.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.318+0000 7fba02ffd700 1 -- 172.21.15.78:0/983674281 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_extra", "pool2": "cephfs_data_extra", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_extra' removed v52) v1 ==== 187+0+0 (secure 0 0 0) 0x7fba04054250 con 0x7fba1406dd80 2023-10-11T09:43:58.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.322+0000 7fba18a43700 1 -- 172.21.15.78:0/983674281 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb9f4065ed0 msgr2=0x7fb9f4068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:58.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.322+0000 7fba18a43700 1 --2- 172.21.15.78:0/983674281 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb9f4065ed0 0x7fb9f4068380 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fba08000c00 tx=0x7fba08008030 comp rx=0 tx=0).stop 2023-10-11T09:43:58.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.322+0000 7fba18a43700 1 -- 172.21.15.78:0/983674281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba1406dd80 msgr2=0x7fba140ca6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:58.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.322+0000 7fba18a43700 1 --2- 172.21.15.78:0/983674281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba1406dd80 0x7fba140ca6b0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fba0400e810 tx=0x7fba0400eb20 comp rx=0 tx=0).stop 2023-10-11T09:43:58.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.326+0000 7fba18a43700 1 -- 172.21.15.78:0/983674281 shutdown_connections 2023-10-11T09:43:58.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.326+0000 7fba18a43700 1 --2- 172.21.15.78:0/983674281 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb9f4065ed0 0x7fb9f4068380 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:58.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.326+0000 7fba18a43700 1 --2- 172.21.15.78:0/983674281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba140b6960 0x7fba141f5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:58.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.326+0000 7fba18a43700 1 --2- 172.21.15.78:0/983674281 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fba140cea30 0x7fba1406c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:58.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.326+0000 7fba18a43700 1 --2- 172.21.15.78:0/983674281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba1406dd80 0x7fba140ca6b0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:58.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.326+0000 7fba18a43700 1 -- 172.21.15.78:0/983674281 >> 172.21.15.78:0/983674281 conn(0x7fba1409a1d0 msgr2=0x7fba1406f300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:58.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.326+0000 7fba18a43700 1 -- 172.21.15.78:0/983674281 shutdown_connections 2023-10-11T09:43:58.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.326+0000 7fba18a43700 1 -- 172.21.15.78:0/983674281 wait complete. 2023-10-11T09:43:58.333 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data_extra' removed 2023-10-11T09:43:58.352 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-11T09:43:58.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.522+0000 7f2f35d45700 1 Processor -- start 2023-10-11T09:43:58.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.522+0000 7f2f35d45700 1 -- start start 2023-10-11T09:43:58.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.522+0000 7f2f35d45700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f300b3240 0x7f2f300d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:58.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.522+0000 7f2f35d45700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f2f300b3610 0x7f2f300b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:58.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.522+0000 7f2f35d45700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f300b47f0 con 0x7f2f3015d040 2023-10-11T09:43:58.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.522+0000 7f2f35d45700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f300b3fd0 con 0x7f2f300b3610 2023-10-11T09:43:58.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.522+0000 7f2f35d45700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f300b8130 con 0x7f2f300b3240 2023-10-11T09:43:58.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.522+0000 7f2f2ffff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f2f300b3610 0x7f2f300b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:58.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.526+0000 7f2f2ffff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f2f300b3610 0x7f2f300b3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35418/0 (socket says 172.21.15.78:35418) 2023-10-11T09:43:58.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.526+0000 7f2f2ffff700 1 -- 172.21.15.78:0/152244897 learned_addr learned my addr 172.21.15.78:0/152244897 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:58.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.526+0000 7f2f34d43700 1 --2- 172.21.15.78:0/152244897 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f300b3240 0x7f2f300d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:58.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.526+0000 7f2f2ffff700 1 -- 172.21.15.78:0/152244897 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f300b3240 msgr2=0x7f2f300d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:58.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.526+0000 7f2f2ffff700 1 --2- 172.21.15.78:0/152244897 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f300b3240 0x7f2f300d29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:58.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.526+0000 7f2f2ffff700 1 -- 172.21.15.78:0/152244897 >> v1:172.21.15.133:6789/0 conn(0x7f2f3015d040 legacy=0x7f2f300b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:58.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.526+0000 7f2f2ffff700 1 -- 172.21.15.78:0/152244897 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f300b41e0 con 0x7f2f300b3610 2023-10-11T09:43:58.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.526+0000 7f2f2ffff700 1 --2- 172.21.15.78:0/152244897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f300b3610 0x7f2f300b3a90 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c009910 tx=0x7f2f1c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e5d316ef75285b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:58.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.526+0000 7f2f2f7fe700 1 -- 172.21.15.78:0/152244897 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c00ba60 con 0x7f2f300b3610 2023-10-11T09:43:58.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.526+0000 7f2f2f7fe700 1 -- 172.21.15.78:0/152244897 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f1c00bbc0 con 0x7f2f300b3610 2023-10-11T09:43:58.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.526+0000 7f2f2f7fe700 1 -- 172.21.15.78:0/152244897 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c009e60 con 0x7f2f300b3610 2023-10-11T09:43:58.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.526+0000 7f2f35d45700 1 -- 172.21.15.78:0/152244897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f300b3610 msgr2=0x7f2f300b3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:58.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.526+0000 7f2f35d45700 1 --2- 172.21.15.78:0/152244897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f300b3610 0x7f2f300b3a90 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c009910 tx=0x7f2f1c009c20 comp rx=0 tx=0).stop 2023-10-11T09:43:58.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.526+0000 7f2f35d45700 1 -- 172.21.15.78:0/152244897 shutdown_connections 2023-10-11T09:43:58.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.526+0000 7f2f35d45700 1 --2- 172.21.15.78:0/152244897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f300b3610 0x7f2f300b3a90 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:58.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.526+0000 7f2f35d45700 1 --2- 172.21.15.78:0/152244897 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f300b3240 0x7f2f300d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:58.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.526+0000 7f2f35d45700 1 -- 172.21.15.78:0/152244897 >> 172.21.15.78:0/152244897 conn(0x7f2f3009c260 msgr2=0x7f2f300d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:58.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.526+0000 7f2f35d45700 1 -- 172.21.15.78:0/152244897 shutdown_connections 2023-10-11T09:43:58.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.526+0000 7f2f35d45700 1 -- 172.21.15.78:0/152244897 wait complete. 2023-10-11T09:43:58.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.530+0000 7f2f35d45700 1 Processor -- start 2023-10-11T09:43:58.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.530+0000 7f2f35d45700 1 -- start start 2023-10-11T09:43:58.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.534+0000 7f2f35d45700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f300b3240 0x7f2f3009dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:58.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.534+0000 7f2f35d45700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f300b3610 0x7f2f3009fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:58.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.534+0000 7f2f35d45700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f3015d040 0x7f2f3015aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:58.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.534+0000 7f2f35d45700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f300c8190 con 0x7f2f3015d040 2023-10-11T09:43:58.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.534+0000 7f2f35d45700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f300a0530 con 0x7f2f300b3240 2023-10-11T09:43:58.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.534+0000 7f2f35d45700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f300a07d0 con 0x7f2f300b3610 2023-10-11T09:43:58.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.534+0000 7f2f35544700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f3015d040 0x7f2f3015aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:58.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.534+0000 7f2f35544700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f3015d040 0x7f2f3015aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35430/0 (socket says 172.21.15.78:35430) 2023-10-11T09:43:58.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.534+0000 7f2f35544700 1 -- 172.21.15.78:0/4115872990 learned_addr learned my addr 172.21.15.78:0/4115872990 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:58.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.534+0000 7f2f34d43700 1 --2- 172.21.15.78:0/4115872990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f300b3240 0x7f2f3009dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:58.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.534+0000 7f2f2ffff700 1 --2- 172.21.15.78:0/4115872990 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f300b3610 0x7f2f3009fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:58.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.534+0000 7f2f35544700 1 -- 172.21.15.78:0/4115872990 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f300b3610 msgr2=0x7f2f3009fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:58.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.534+0000 7f2f35544700 1 --2- 172.21.15.78:0/4115872990 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f300b3610 0x7f2f3009fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:58.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.534+0000 7f2f35544700 1 -- 172.21.15.78:0/4115872990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f300b3240 msgr2=0x7f2f3009dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:58.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.534+0000 7f2f35544700 1 --2- 172.21.15.78:0/4115872990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f300b3240 0x7f2f3009dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:58.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.534+0000 7f2f35544700 1 -- 172.21.15.78:0/4115872990 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f1c009570 con 0x7f2f3015d040 2023-10-11T09:43:58.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.534+0000 7f2f35544700 1 --2- 172.21.15.78:0/4115872990 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f3015d040 0x7f2f3015aaf0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2f2800cc50 tx=0x7f2f2800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:58.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.534+0000 7f2f2d7fa700 1 -- 172.21.15.78:0/4115872990 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2800ad80 con 0x7f2f3015d040 2023-10-11T09:43:58.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.534+0000 7f2f2d7fa700 1 -- 172.21.15.78:0/4115872990 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f2800ce50 con 0x7f2f3015d040 2023-10-11T09:43:58.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.534+0000 7f2f35d45700 1 -- 172.21.15.78:0/4115872990 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f300a0ab0 con 0x7f2f3015d040 2023-10-11T09:43:58.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.534+0000 7f2f2d7fa700 1 -- 172.21.15.78:0/4115872990 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f28003c60 con 0x7f2f3015d040 2023-10-11T09:43:58.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.534+0000 7f2f35d45700 1 -- 172.21.15.78:0/4115872990 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f3015b030 con 0x7f2f3015d040 2023-10-11T09:43:58.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.538+0000 7f2f35d45700 1 -- 172.21.15.78:0/4115872990 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f300c8300 con 0x7f2f3015d040 2023-10-11T09:43:58.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.538+0000 7f2f2d7fa700 1 -- 172.21.15.78:0/4115872990 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f2f280050d0 con 0x7f2f3015d040 2023-10-11T09:43:58.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.538+0000 7f2f2d7fa700 1 --2- 172.21.15.78:0/4115872990 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2f14065f00 0x7f2f140683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:58.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.542+0000 7f2f2d7fa700 1 -- 172.21.15.78:0/4115872990 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f2f28083600 con 0x7f2f3015d040 2023-10-11T09:43:58.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.542+0000 7f2f34d43700 1 --2- 172.21.15.78:0/4115872990 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2f14065f00 0x7f2f140683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:58.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.542+0000 7f2f34d43700 1 --2- 172.21.15.78:0/4115872990 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2f14065f00 0x7f2f140683b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2f2400b9d0 tx=0x7f2f24008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:58.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.546+0000 7f2f2d7fa700 1 -- 172.21.15.78:0/4115872990 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f2f28050e50 con 0x7f2f3015d040 2023-10-11T09:43:58.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.718+0000 7f2f35d45700 1 -- 172.21.15.78:0/4115872990 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f2f30065d40 con 0x7f2f3015d040 2023-10-11T09:43:58.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.730+0000 7f2f2d7fa700 1 -- 172.21.15.78:0/4115872990 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2f28054470 con 0x7f2f3015d040 2023-10-11T09:43:58.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.734+0000 7f2f35d45700 1 -- 172.21.15.78:0/4115872990 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2f14065f00 msgr2=0x7f2f140683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:58.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.734+0000 7f2f35d45700 1 --2- 172.21.15.78:0/4115872990 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2f14065f00 0x7f2f140683b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2f2400b9d0 tx=0x7f2f24008040 comp rx=0 tx=0).stop 2023-10-11T09:43:58.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.734+0000 7f2f35d45700 1 -- 172.21.15.78:0/4115872990 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f3015d040 msgr2=0x7f2f3015aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:58.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.734+0000 7f2f35d45700 1 --2- 172.21.15.78:0/4115872990 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f3015d040 0x7f2f3015aaf0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2f2800cc50 tx=0x7f2f2800a340 comp rx=0 tx=0).stop 2023-10-11T09:43:58.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.734+0000 7f2f35d45700 1 -- 172.21.15.78:0/4115872990 shutdown_connections 2023-10-11T09:43:58.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.734+0000 7f2f35d45700 1 --2- 172.21.15.78:0/4115872990 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2f14065f00 0x7f2f140683b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:58.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.734+0000 7f2f35d45700 1 --2- 172.21.15.78:0/4115872990 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f3015d040 0x7f2f3015aaf0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:58.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.734+0000 7f2f35d45700 1 --2- 172.21.15.78:0/4115872990 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f300b3610 0x7f2f3009fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:58.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.734+0000 7f2f35d45700 1 --2- 172.21.15.78:0/4115872990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f300b3240 0x7f2f3009dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:58.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.734+0000 7f2f35d45700 1 -- 172.21.15.78:0/4115872990 >> 172.21.15.78:0/4115872990 conn(0x7f2f3009c260 msgr2=0x7f2f3015faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:58.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.734+0000 7f2f35d45700 1 -- 172.21.15.78:0/4115872990 shutdown_connections 2023-10-11T09:43:58.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.734+0000 7f2f35d45700 1 -- 172.21.15.78:0/4115872990 wait complete. 2023-10-11T09:43:58.764 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_few_files' 2023-10-11T09:43:58.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.934+0000 7f093b59e700 1 Processor -- start 2023-10-11T09:43:58.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.934+0000 7f093b59e700 1 -- start start 2023-10-11T09:43:58.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.934+0000 7f093b59e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f093c15b8e0 0x7f093c06d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:58.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.934+0000 7f093b59e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f093c06d610 0x7f093c0ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:58.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.934+0000 7f093b59e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f093c15a940 0x7f093c0ca780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:58.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.934+0000 7f093b59e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f093c15e010 con 0x7f093c15b8e0 2023-10-11T09:43:58.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.934+0000 7f093b59e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f093c15e280 con 0x7f093c15a940 2023-10-11T09:43:58.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.934+0000 7f093b59e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093c15e560 con 0x7f093c06d610 2023-10-11T09:43:58.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.938+0000 7f0939d9b700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f093c06d610 0x7f093c0ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:58.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.938+0000 7f093a59c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f093c15b8e0 0x7f093c06d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:58.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.938+0000 7f0939d9b700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f093c06d610 0x7f093c0ca240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:39916/0 (socket says 172.21.15.78:39916) 2023-10-11T09:43:58.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.938+0000 7f0939d9b700 1 -- 172.21.15.78:0/2595215380 learned_addr learned my addr 172.21.15.78:0/2595215380 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:58.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.938+0000 7f093ad9d700 1 --2- 172.21.15.78:0/2595215380 >> v2:172.21.15.133:3300/0 conn(0x7f093c15a940 0x7f093c0ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:58.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.938+0000 7f0939d9b700 1 -- 172.21.15.78:0/2595215380 >> v2:172.21.15.133:3300/0 conn(0x7f093c15a940 msgr2=0x7f093c0ca780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:58.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.938+0000 7f0939d9b700 1 --2- 172.21.15.78:0/2595215380 >> v2:172.21.15.133:3300/0 conn(0x7f093c15a940 0x7f093c0ca780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:58.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.938+0000 7f0939d9b700 1 -- 172.21.15.78:0/2595215380 >> v2:172.21.15.78:3300/0 conn(0x7f093c15b8e0 msgr2=0x7f093c06d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:58.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.938+0000 7f0939d9b700 1 --2- 172.21.15.78:0/2595215380 >> v2:172.21.15.78:3300/0 conn(0x7f093c15b8e0 0x7f093c06d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:58.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.938+0000 7f0939d9b700 1 -- 172.21.15.78:0/2595215380 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f093c15e840 con 0x7f093c06d610 2023-10-11T09:43:58.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.938+0000 7f093a59c700 1 --2- 172.21.15.78:0/2595215380 >> v2:172.21.15.78:3300/0 conn(0x7f093c15b8e0 0x7f093c06d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:43:58.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.938+0000 7f0939d9b700 1 --2- 172.21.15.78:0/2595215380 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f093c06d610 0x7f093c0ca240 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f093c0cf800 tx=0x7f0930009bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7be0d48ddc0025c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:58.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.938+0000 7f093959a700 1 -- 172.21.15.78:0/2595215380 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0930011830 con 0x7f093c06d610 2023-10-11T09:43:58.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.938+0000 7f093959a700 1 -- 172.21.15.78:0/2595215380 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0930011990 con 0x7f093c06d610 2023-10-11T09:43:58.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.938+0000 7f093959a700 1 -- 172.21.15.78:0/2595215380 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0930011d20 con 0x7f093c06d610 2023-10-11T09:43:58.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.938+0000 7f093b59e700 1 -- 172.21.15.78:0/2595215380 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f093c06d610 msgr2=0x7f093c0ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:58.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.938+0000 7f093b59e700 1 --2- 172.21.15.78:0/2595215380 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f093c06d610 0x7f093c0ca240 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f093c0cf800 tx=0x7f0930009bf0 comp rx=0 tx=0).stop 2023-10-11T09:43:58.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.938+0000 7f093b59e700 1 -- 172.21.15.78:0/2595215380 shutdown_connections 2023-10-11T09:43:58.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.938+0000 7f093b59e700 1 --2- 172.21.15.78:0/2595215380 >> v2:172.21.15.133:3300/0 conn(0x7f093c15a940 0x7f093c0ca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:58.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.938+0000 7f093b59e700 1 --2- 172.21.15.78:0/2595215380 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f093c06d610 0x7f093c0ca240 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:58.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.938+0000 7f093b59e700 1 --2- 172.21.15.78:0/2595215380 >> v2:172.21.15.78:3300/0 conn(0x7f093c15b8e0 0x7f093c06d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:58.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.938+0000 7f093b59e700 1 -- 172.21.15.78:0/2595215380 >> 172.21.15.78:0/2595215380 conn(0x7f093c0cea60 msgr2=0x7f093c0cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:58.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.938+0000 7f093b59e700 1 -- 172.21.15.78:0/2595215380 shutdown_connections 2023-10-11T09:43:58.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.938+0000 7f093b59e700 1 -- 172.21.15.78:0/2595215380 wait complete. 2023-10-11T09:43:58.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.942+0000 7f093b59e700 1 Processor -- start 2023-10-11T09:43:58.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.942+0000 7f093b59e700 1 -- start start 2023-10-11T09:43:58.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.942+0000 7f093b59e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f093c06d610 0x7f093c142240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:58.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.942+0000 7f093b59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f093c15a940 0x7f093c13e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:58.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.942+0000 7f093b59e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f093c15b8e0 0x7f093c0c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:58.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.942+0000 7f093b59e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093c13e860 con 0x7f093c15a940 2023-10-11T09:43:58.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.942+0000 7f093b59e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093c1f1970 con 0x7f093c15b8e0 2023-10-11T09:43:58.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.942+0000 7f093b59e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093c0cca30 con 0x7f093c06d610 2023-10-11T09:43:58.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.942+0000 7f0939d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f093c15a940 0x7f093c13e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:58.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.942+0000 7f0939d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f093c15a940 0x7f093c13e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35446/0 (socket says 172.21.15.78:35446) 2023-10-11T09:43:58.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.942+0000 7f0939d9b700 1 -- 172.21.15.78:0/3992590751 learned_addr learned my addr 172.21.15.78:0/3992590751 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:58.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.942+0000 7f093ad9d700 1 --2- 172.21.15.78:0/3992590751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f093c15b8e0 0x7f093c0c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:58.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.942+0000 7f093a59c700 1 --2- 172.21.15.78:0/3992590751 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f093c06d610 0x7f093c142240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:58.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.946+0000 7f0939d9b700 1 -- 172.21.15.78:0/3992590751 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f093c06d610 msgr2=0x7f093c142240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:58.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.946+0000 7f0939d9b700 1 --2- 172.21.15.78:0/3992590751 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f093c06d610 0x7f093c142240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:58.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.946+0000 7f0939d9b700 1 -- 172.21.15.78:0/3992590751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f093c15b8e0 msgr2=0x7f093c0c7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:58.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.946+0000 7f0939d9b700 1 --2- 172.21.15.78:0/3992590751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f093c15b8e0 0x7f093c0c7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:58.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.946+0000 7f0939d9b700 1 -- 172.21.15.78:0/3992590751 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092c0094b0 con 0x7f093c15a940 2023-10-11T09:43:58.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.946+0000 7f093ad9d700 1 --2- 172.21.15.78:0/3992590751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f093c15b8e0 0x7f093c0c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:43:58.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.946+0000 7f0939d9b700 1 --2- 172.21.15.78:0/3992590751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f093c15a940 0x7f093c13e320 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f0930000c00 tx=0x7f0930004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:58.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.946+0000 7f092affd700 1 -- 172.21.15.78:0/3992590751 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0930004590 con 0x7f093c15a940 2023-10-11T09:43:58.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.946+0000 7f092affd700 1 -- 172.21.15.78:0/3992590751 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f093001e070 con 0x7f093c15a940 2023-10-11T09:43:58.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.946+0000 7f092affd700 1 -- 172.21.15.78:0/3992590751 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0930003900 con 0x7f093c15a940 2023-10-11T09:43:58.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.946+0000 7f093b59e700 1 -- 172.21.15.78:0/3992590751 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0930009570 con 0x7f093c15a940 2023-10-11T09:43:58.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.946+0000 7f093b59e700 1 -- 172.21.15.78:0/3992590751 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f093c0ccff0 con 0x7f093c15a940 2023-10-11T09:43:58.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.946+0000 7f093b59e700 1 -- 172.21.15.78:0/3992590751 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f093c1f1b30 con 0x7f093c15a940 2023-10-11T09:43:58.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.950+0000 7f092affd700 1 -- 172.21.15.78:0/3992590751 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f0930003a60 con 0x7f093c15a940 2023-10-11T09:43:58.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.950+0000 7f092affd700 1 --2- 172.21.15.78:0/3992590751 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f091c065ed0 0x7f091c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:58.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.950+0000 7f092affd700 1 -- 172.21.15.78:0/3992590751 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f0930084720 con 0x7f093c15a940 2023-10-11T09:43:58.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.950+0000 7f093a59c700 1 --2- 172.21.15.78:0/3992590751 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f091c065ed0 0x7f091c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:58.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.950+0000 7f093a59c700 1 --2- 172.21.15.78:0/3992590751 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f091c065ed0 0x7f091c068380 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f092c009c90 tx=0x7f092c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:58.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:58.958+0000 7f092affd700 1 -- 172.21.15.78:0/3992590751 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f0930051f60 con 0x7f093c15a940 2023-10-11T09:43:59.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.202+0000 7f093b59e700 1 -- 172.21.15.78:0/3992590751 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_few_files"]} v 0) v1 -- 0x7f093c065d40 con 0x7f093c15a940 2023-10-11T09:43:59.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.330+0000 7f092affd700 1 -- 172.21.15.78:0/3992590751 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_few_files"]}]=0 v184) v1 ==== 158+0+0 (secure 0 0 0) 0x7f09300200e0 con 0x7f093c15a940 2023-10-11T09:43:59.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.334+0000 7f093b59e700 1 -- 172.21.15.78:0/3992590751 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f091c065ed0 msgr2=0x7f091c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:59.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.334+0000 7f093b59e700 1 --2- 172.21.15.78:0/3992590751 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f091c065ed0 0x7f091c068380 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f092c009c90 tx=0x7f092c008040 comp rx=0 tx=0).stop 2023-10-11T09:43:59.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.334+0000 7f093b59e700 1 -- 172.21.15.78:0/3992590751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f093c15a940 msgr2=0x7f093c13e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:59.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.334+0000 7f093b59e700 1 --2- 172.21.15.78:0/3992590751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f093c15a940 0x7f093c13e320 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f0930000c00 tx=0x7f0930004770 comp rx=0 tx=0).stop 2023-10-11T09:43:59.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.334+0000 7f093b59e700 1 -- 172.21.15.78:0/3992590751 shutdown_connections 2023-10-11T09:43:59.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.334+0000 7f093b59e700 1 --2- 172.21.15.78:0/3992590751 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f091c065ed0 0x7f091c068380 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:59.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.334+0000 7f093b59e700 1 --2- 172.21.15.78:0/3992590751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f093c15b8e0 0x7f093c0c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:59.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.334+0000 7f093b59e700 1 --2- 172.21.15.78:0/3992590751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f093c15a940 0x7f093c13e320 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:59.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.334+0000 7f093b59e700 1 --2- 172.21.15.78:0/3992590751 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f093c06d610 0x7f093c142240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:59.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.334+0000 7f093b59e700 1 -- 172.21.15.78:0/3992590751 >> 172.21.15.78:0/3992590751 conn(0x7f093c0cea60 msgr2=0x7f093c0ca2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:59.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.334+0000 7f093b59e700 1 -- 172.21.15.78:0/3992590751 shutdown_connections 2023-10-11T09:43:59.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.334+0000 7f093b59e700 1 -- 172.21.15.78:0/3992590751 wait complete. 2023-10-11T09:43:59.363 INFO:tasks.cephfs_test_runner:test_extra_data_pool_rebuild_few_files (tasks.cephfs.test_data_scan.TestDataScan) ... ok 2023-10-11T09:43:59.364 INFO:tasks.cephfs_test_runner:Starting test: test_extra_data_pool_rebuild_many_files_many_workers (tasks.cephfs.test_data_scan.TestDataScan) 2023-10-11T09:43:59.364 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_many_files_many_workers' 2023-10-11T09:43:59.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.526+0000 7ff1fac48700 1 Processor -- start 2023-10-11T09:43:59.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.526+0000 7ff1fac48700 1 -- start start 2023-10-11T09:43:59.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.530+0000 7ff1fac48700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ff1f413f740 0x7ff1f40ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:59.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.530+0000 7ff1fac48700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1f406d180 0x7ff1f40b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:59.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.530+0000 7ff1fac48700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1f4070c80 con 0x7ff1f406cd00 2023-10-11T09:43:59.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.530+0000 7ff1fac48700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1f40ce4c0 con 0x7ff1f413f740 2023-10-11T09:43:59.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.530+0000 7ff1fac48700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f406d600 con 0x7ff1f406d180 2023-10-11T09:43:59.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.530+0000 7ff1f9c46700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ff1f413f740 0x7ff1f40ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:59.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.530+0000 7ff1f9c46700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ff1f413f740 0x7ff1f40ccab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35458/0 (socket says 172.21.15.78:35458) 2023-10-11T09:43:59.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.530+0000 7ff1f9c46700 1 -- 172.21.15.78:0/212343070 learned_addr learned my addr 172.21.15.78:0/212343070 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:59.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.530+0000 7ff1fa447700 1 --2- 172.21.15.78:0/212343070 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1f406d180 0x7ff1f40b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:59.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.530+0000 7ff1f9c46700 1 -- 172.21.15.78:0/212343070 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1f406d180 msgr2=0x7ff1f40b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:43:59.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.530+0000 7ff1f9c46700 1 --2- 172.21.15.78:0/212343070 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1f406d180 0x7ff1f40b0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:59.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.530+0000 7ff1f9c46700 1 -- 172.21.15.78:0/212343070 >> v1:172.21.15.78:6789/0 conn(0x7ff1f406cd00 legacy=0x7ff1f40b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:59.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.530+0000 7ff1f9c46700 1 -- 172.21.15.78:0/212343070 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f40cd420 con 0x7ff1f413f740 2023-10-11T09:43:59.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.530+0000 7ff1f8c44700 1 -- 172.21.15.78:0/212343070 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1187241406 0 0) 0x7ff1f4070c80 con 0x7ff1f406cd00 2023-10-11T09:43:59.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.530+0000 7ff1f9c46700 1 --2- 172.21.15.78:0/212343070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1f413f740 0x7ff1f40ccab0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0009910 tx=0x7ff1f0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c82f17557690dd0b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:59.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.530+0000 7ff1f8c44700 1 -- 172.21.15.78:0/212343070 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f000ba60 con 0x7ff1f413f740 2023-10-11T09:43:59.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.530+0000 7ff1f8c44700 1 -- 172.21.15.78:0/212343070 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f000bbc0 con 0x7ff1f413f740 2023-10-11T09:43:59.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.530+0000 7ff1f8c44700 1 -- 172.21.15.78:0/212343070 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0009e60 con 0x7ff1f413f740 2023-10-11T09:43:59.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.534+0000 7ff1fac48700 1 -- 172.21.15.78:0/212343070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1f413f740 msgr2=0x7ff1f40ccab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:59.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.534+0000 7ff1fac48700 1 --2- 172.21.15.78:0/212343070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1f413f740 0x7ff1f40ccab0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0009910 tx=0x7ff1f0009c20 comp rx=0 tx=0).stop 2023-10-11T09:43:59.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.534+0000 7ff1fac48700 1 -- 172.21.15.78:0/212343070 shutdown_connections 2023-10-11T09:43:59.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.534+0000 7ff1fac48700 1 --2- 172.21.15.78:0/212343070 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1f406d180 0x7ff1f40b0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:59.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.534+0000 7ff1fac48700 1 --2- 172.21.15.78:0/212343070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1f413f740 0x7ff1f40ccab0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:59.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.534+0000 7ff1fac48700 1 -- 172.21.15.78:0/212343070 >> 172.21.15.78:0/212343070 conn(0x7ff1f40cea60 msgr2=0x7ff1f40cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:43:59.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.534+0000 7ff1fac48700 1 -- 172.21.15.78:0/212343070 shutdown_connections 2023-10-11T09:43:59.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.534+0000 7ff1fac48700 1 -- 172.21.15.78:0/212343070 wait complete. 2023-10-11T09:43:59.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.538+0000 7ff1fac48700 1 Processor -- start 2023-10-11T09:43:59.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.538+0000 7ff1fac48700 1 -- start start 2023-10-11T09:43:59.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.538+0000 7ff1fac48700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1f406cd00 0x7ff1f4159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:59.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.538+0000 7ff1fac48700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1f406d180 0x7ff1f413da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:59.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.538+0000 7ff1fac48700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1f413f740 0x7ff1f413df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:59.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.538+0000 7ff1fac48700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f41f22e0 con 0x7ff1f406cd00 2023-10-11T09:43:59.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.538+0000 7ff1fac48700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f41f5d10 con 0x7ff1f406d180 2023-10-11T09:43:59.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.538+0000 7ff1fac48700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f41f1c90 con 0x7ff1f413f740 2023-10-11T09:43:59.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.538+0000 7ff1f9c46700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1f406cd00 0x7ff1f4159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:59.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.542+0000 7ff1f9c46700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1f406cd00 0x7ff1f4159c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35466/0 (socket says 172.21.15.78:35466) 2023-10-11T09:43:59.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.542+0000 7ff1f9c46700 1 -- 172.21.15.78:0/2292711904 learned_addr learned my addr 172.21.15.78:0/2292711904 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:43:59.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.542+0000 7ff1f9445700 1 --2- 172.21.15.78:0/2292711904 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1f406d180 0x7ff1f413da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:59.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.542+0000 7ff1fa447700 1 --2- 172.21.15.78:0/2292711904 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1f413f740 0x7ff1f413df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:59.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.542+0000 7ff1f9c46700 1 -- 172.21.15.78:0/2292711904 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1f413f740 msgr2=0x7ff1f413df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:59.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.542+0000 7ff1f9c46700 1 --2- 172.21.15.78:0/2292711904 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1f413f740 0x7ff1f413df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:59.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.542+0000 7ff1f9c46700 1 -- 172.21.15.78:0/2292711904 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1f406d180 msgr2=0x7ff1f413da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:43:59.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.542+0000 7ff1f9c46700 1 --2- 172.21.15.78:0/2292711904 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1f406d180 0x7ff1f413da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:43:59.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.542+0000 7ff1f9c46700 1 -- 172.21.15.78:0/2292711904 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f0009570 con 0x7ff1f406cd00 2023-10-11T09:43:59.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.542+0000 7ff1f9c46700 1 --2- 172.21.15.78:0/2292711904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1f406cd00 0x7ff1f4159c70 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0000c00 tx=0x7ff1f0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:59.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.542+0000 7ff1ea7fc700 1 -- 172.21.15.78:0/2292711904 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f001a840 con 0x7ff1f406cd00 2023-10-11T09:43:59.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.542+0000 7ff1ea7fc700 1 -- 172.21.15.78:0/2292711904 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f001f070 con 0x7ff1f406cd00 2023-10-11T09:43:59.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.542+0000 7ff1fac48700 1 -- 172.21.15.78:0/2292711904 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1f41f1f70 con 0x7ff1f406cd00 2023-10-11T09:43:59.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.542+0000 7ff1ea7fc700 1 -- 172.21.15.78:0/2292711904 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f001ac00 con 0x7ff1f406cd00 2023-10-11T09:43:59.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.542+0000 7ff1fac48700 1 -- 172.21.15.78:0/2292711904 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1f41f1920 con 0x7ff1f406cd00 2023-10-11T09:43:59.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.542+0000 7ff1fac48700 1 -- 172.21.15.78:0/2292711904 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1f4065d40 con 0x7ff1f406cd00 2023-10-11T09:43:59.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.546+0000 7ff1ea7fc700 1 -- 172.21.15.78:0/2292711904 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7ff1f001b430 con 0x7ff1f406cd00 2023-10-11T09:43:59.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.546+0000 7ff1ea7fc700 1 --2- 172.21.15.78:0/2292711904 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff1d8065ed0 0x7ff1d8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:43:59.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.546+0000 7ff1ea7fc700 1 -- 172.21.15.78:0/2292711904 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5554+0+0 (secure 0 0 0) 0x7ff1f0021070 con 0x7ff1f406cd00 2023-10-11T09:43:59.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.546+0000 7ff1f9445700 1 --2- 172.21.15.78:0/2292711904 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff1d8065ed0 0x7ff1d8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:43:59.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.550+0000 7ff1f9445700 1 --2- 172.21.15.78:0/2292711904 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff1d8065ed0 0x7ff1d8068380 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff1f41361f0 tx=0x7ff1e4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:43:59.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.554+0000 7ff1ea7fc700 1 -- 172.21.15.78:0/2292711904 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7ff1f0052760 con 0x7ff1f406cd00 2023-10-11T09:43:59.613 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:43:59.619 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:43:59.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:43:59.798+0000 7ff1fac48700 1 -- 172.21.15.78:0/2292711904 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_many_files_many_workers"]} v 0) v1 -- 0x7ff1f413ecf0 con 0x7ff1f406cd00 2023-10-11T09:44:00.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.334+0000 7ff1ea7fc700 1 -- 172.21.15.78:0/2292711904 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_many_files_many_workers"]}]=0 v185) v1 ==== 175+0+0 (secure 0 0 0) 0x7ff1f0055d80 con 0x7ff1f406cd00 2023-10-11T09:44:00.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.338+0000 7ff1fac48700 1 -- 172.21.15.78:0/2292711904 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff1d8065ed0 msgr2=0x7ff1d8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:00.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.338+0000 7ff1fac48700 1 --2- 172.21.15.78:0/2292711904 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff1d8065ed0 0x7ff1d8068380 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff1f41361f0 tx=0x7ff1e4009040 comp rx=0 tx=0).stop 2023-10-11T09:44:00.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.338+0000 7ff1fac48700 1 -- 172.21.15.78:0/2292711904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1f406cd00 msgr2=0x7ff1f4159c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:00.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.338+0000 7ff1fac48700 1 --2- 172.21.15.78:0/2292711904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1f406cd00 0x7ff1f4159c70 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0000c00 tx=0x7ff1f0004770 comp rx=0 tx=0).stop 2023-10-11T09:44:00.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.338+0000 7ff1fac48700 1 -- 172.21.15.78:0/2292711904 shutdown_connections 2023-10-11T09:44:00.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.338+0000 7ff1fac48700 1 --2- 172.21.15.78:0/2292711904 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff1d8065ed0 0x7ff1d8068380 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:00.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.338+0000 7ff1fac48700 1 --2- 172.21.15.78:0/2292711904 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1f413f740 0x7ff1f413df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:00.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.338+0000 7ff1fac48700 1 --2- 172.21.15.78:0/2292711904 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1f406d180 0x7ff1f413da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:00.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.338+0000 7ff1fac48700 1 --2- 172.21.15.78:0/2292711904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1f406cd00 0x7ff1f4159c70 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:00.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.338+0000 7ff1fac48700 1 -- 172.21.15.78:0/2292711904 >> 172.21.15.78:0/2292711904 conn(0x7ff1f40cea60 msgr2=0x7ff1f4070180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:00.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.338+0000 7ff1fac48700 1 -- 172.21.15.78:0/2292711904 shutdown_connections 2023-10-11T09:44:00.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.338+0000 7ff1fac48700 1 -- 172.21.15.78:0/2292711904 wait complete. 2023-10-11T09:44:00.364 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-11T09:44:00.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.530+0000 7f925e759700 1 Processor -- start 2023-10-11T09:44:00.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.530+0000 7f925e759700 1 -- start start 2023-10-11T09:44:00.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.534+0000 7f925e759700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f92580d2350 0x7f92580d36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:00.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.534+0000 7f925e759700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92580cd690 con 0x7f92580b84c0 2023-10-11T09:44:00.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.534+0000 7f925e759700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92580cd8e0 con 0x7f92580b8890 2023-10-11T09:44:00.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.534+0000 7f925e759700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92580cdea0 con 0x7f92580d2350 2023-10-11T09:44:00.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.534+0000 7f9257fff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f92580b84c0 0x7f92580cfda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:46354/0 (socket says 172.21.15.78:46354) 2023-10-11T09:44:00.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.534+0000 7f9257fff700 1 -- 172.21.15.78:0/1266661989 learned_addr learned my addr 172.21.15.78:0/1266661989 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:44:00.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.534+0000 7f925ccf6700 1 --2- 172.21.15.78:0/1266661989 >> v2:172.21.15.133:3300/0 conn(0x7f92580d2350 0x7f92580d36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:00.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.534+0000 7f92577fe700 1 -- 172.21.15.78:0/1266661989 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 400418522 0 0) 0x7f92580cd690 con 0x7f92580b84c0 2023-10-11T09:44:00.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.534+0000 7f92577fe700 1 -- 172.21.15.78:0/1266661989 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92440035d0 con 0x7f92580b84c0 2023-10-11T09:44:00.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.534+0000 7f92577fe700 1 -- 172.21.15.78:0/1266661989 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3757521143 0 0) 0x7f92580cd8e0 con 0x7f92580b8890 2023-10-11T09:44:00.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.534+0000 7f92577fe700 1 -- 172.21.15.78:0/1266661989 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92580cd690 con 0x7f92580b8890 2023-10-11T09:44:00.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.534+0000 7f92577fe700 1 -- 172.21.15.78:0/1266661989 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2993205079 0 0) 0x7f92440035d0 con 0x7f92580b84c0 2023-10-11T09:44:00.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.534+0000 7f92577fe700 1 -- 172.21.15.78:0/1266661989 >> v2:172.21.15.133:3300/0 conn(0x7f92580d2350 msgr2=0x7f92580d36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:44:00.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.534+0000 7f92577fe700 1 --2- 172.21.15.78:0/1266661989 >> v2:172.21.15.133:3300/0 conn(0x7f92580d2350 0x7f92580d36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.534+0000 7f92577fe700 1 -- 172.21.15.78:0/1266661989 >> v1:172.21.15.133:6789/0 conn(0x7f92580b8890 legacy=0x7f92580d2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.534+0000 7f92577fe700 1 -- 172.21.15.78:0/1266661989 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92580cdbc0 con 0x7f92580b84c0 2023-10-11T09:44:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.534+0000 7f92577fe700 1 -- 172.21.15.78:0/1266661989 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f9248001670 con 0x7f92580b84c0 2023-10-11T09:44:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.534+0000 7f925ccf6700 1 --2- 172.21.15.78:0/1266661989 >> v2:172.21.15.133:3300/0 conn(0x7f92580d2350 0x7f92580d36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:44:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.534+0000 7f92577fe700 1 -- 172.21.15.78:0/1266661989 >> v1:172.21.15.78:6789/0 conn(0x7f92580b84c0 legacy=0x7f92580cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.534+0000 7f92577fe700 1 --2- 172.21.15.78:0/1266661989 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9244003c00 0x7f9244003ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:00.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.534+0000 7f92577fe700 1 --2- 172.21.15.78:0/1266661989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9244004870 0x7f9244004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:00.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.534+0000 7f92577fe700 1 --2- 172.21.15.78:0/1266661989 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9244005270 0x7f924400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:00.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.534+0000 7f92577fe700 1 -- 172.21.15.78:0/1266661989 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92580cdea0 con 0x7f9244004870 2023-10-11T09:44:00.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.534+0000 7f92577fe700 1 -- 172.21.15.78:0/1266661989 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924400c180 con 0x7f9244005270 2023-10-11T09:44:00.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.534+0000 7f92577fe700 1 -- 172.21.15.78:0/1266661989 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f924400c340 con 0x7f9244003c00 2023-10-11T09:44:00.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.538+0000 7f924ffff700 1 --2- 172.21.15.78:0/1266661989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9244004870 0x7f9244004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:00.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.538+0000 7f925ccf6700 1 --2- 172.21.15.78:0/1266661989 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9244005270 0x7f924400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:00.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.538+0000 7f924ffff700 1 -- 172.21.15.78:0/1266661989 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9244003c00 msgr2=0x7f9244003ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:44:00.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.538+0000 7f924ffff700 1 --2- 172.21.15.78:0/1266661989 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9244003c00 0x7f9244003ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:00.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.538+0000 7f924ffff700 1 -- 172.21.15.78:0/1266661989 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9244005270 msgr2=0x7f924400ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:44:00.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.538+0000 7f924ffff700 1 --2- 172.21.15.78:0/1266661989 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9244005270 0x7f924400ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:00.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.538+0000 7f925ccf6700 1 -- 172.21.15.78:0/1266661989 reap_dead start 2023-10-11T09:44:00.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.538+0000 7f924ffff700 1 -- 172.21.15.78:0/1266661989 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92440035d0 con 0x7f9244004870 2023-10-11T09:44:00.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.538+0000 7f924ffff700 1 --2- 172.21.15.78:0/1266661989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9244004870 0x7f9244004ce0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f9240007fd0 tx=0x7f92440058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8de6f4fa3459c7b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:00.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.538+0000 7f92577fe700 1 -- 172.21.15.78:0/1266661989 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9240001450 con 0x7f9244004870 2023-10-11T09:44:00.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.538+0000 7f92577fe700 1 -- 172.21.15.78:0/1266661989 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9240006aa0 con 0x7f9244004870 2023-10-11T09:44:00.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.538+0000 7f92577fe700 1 -- 172.21.15.78:0/1266661989 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9240006e50 con 0x7f9244004870 2023-10-11T09:44:00.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.538+0000 7f925e759700 1 -- 172.21.15.78:0/1266661989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9244004870 msgr2=0x7f9244004ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:00.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.538+0000 7f925e759700 1 --2- 172.21.15.78:0/1266661989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9244004870 0x7f9244004ce0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f9240007fd0 tx=0x7f92440058a0 comp rx=0 tx=0).stop 2023-10-11T09:44:00.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.538+0000 7f925e759700 1 -- 172.21.15.78:0/1266661989 shutdown_connections 2023-10-11T09:44:00.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.538+0000 7f925e759700 1 --2- 172.21.15.78:0/1266661989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9244004870 0x7f9244004ce0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:00.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.538+0000 7f925e759700 1 -- 172.21.15.78:0/1266661989 >> 172.21.15.78:0/1266661989 conn(0x7f925806cb50 msgr2=0x7f925806cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:00.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.538+0000 7f925e759700 1 -- 172.21.15.78:0/1266661989 shutdown_connections 2023-10-11T09:44:00.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.538+0000 7f925e759700 1 -- 172.21.15.78:0/1266661989 wait complete. 2023-10-11T09:44:00.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.542+0000 7f925e759700 1 Processor -- start 2023-10-11T09:44:00.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.542+0000 7f925e759700 1 -- start start 2023-10-11T09:44:00.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.542+0000 7f925e759700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9244004870 0x7f92581f22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:00.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.542+0000 7f925e759700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92581492d0 0x7f92580cc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:00.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.542+0000 7f925e759700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9258149740 0x7f92580cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:00.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.542+0000 7f925e759700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92580c7d50 con 0x7f9258149740 2023-10-11T09:44:00.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.542+0000 7f925e759700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92580c7fa0 con 0x7f9244004870 2023-10-11T09:44:00.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.542+0000 7f925e759700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92580c8280 con 0x7f92581492d0 2023-10-11T09:44:00.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.542+0000 7f925ccf6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9258149740 0x7f92580cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:00.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.542+0000 7f925ccf6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9258149740 0x7f92580cc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35476/0 (socket says 172.21.15.78:35476) 2023-10-11T09:44:00.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.542+0000 7f925ccf6700 1 -- 172.21.15.78:0/3194762992 learned_addr learned my addr 172.21.15.78:0/3194762992 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:00.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.542+0000 7f924ffff700 1 --2- 172.21.15.78:0/3194762992 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92581492d0 0x7f92580cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:00.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.542+0000 7f9257fff700 1 --2- 172.21.15.78:0/3194762992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9244004870 0x7f92581f22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:00.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.542+0000 7f925ccf6700 1 -- 172.21.15.78:0/3194762992 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92581492d0 msgr2=0x7f92580cc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:00.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.542+0000 7f925ccf6700 1 --2- 172.21.15.78:0/3194762992 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92581492d0 0x7f92580cc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:00.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.542+0000 7f925ccf6700 1 -- 172.21.15.78:0/3194762992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9244004870 msgr2=0x7f92581f22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:00.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.542+0000 7f925ccf6700 1 --2- 172.21.15.78:0/3194762992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9244004870 0x7f92581f22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:00.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.542+0000 7f925ccf6700 1 -- 172.21.15.78:0/3194762992 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92580c8560 con 0x7f9258149740 2023-10-11T09:44:00.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.542+0000 7f9257fff700 1 --2- 172.21.15.78:0/3194762992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9244004870 0x7f92581f22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:44:00.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.542+0000 7f925ccf6700 1 --2- 172.21.15.78:0/3194762992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9258149740 0x7f92580cc660 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f92580d0580 tx=0x7f925000f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:00.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.542+0000 7f92557fa700 1 -- 172.21.15.78:0/3194762992 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9250009ea0 con 0x7f9258149740 2023-10-11T09:44:00.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.542+0000 7f92557fa700 1 -- 172.21.15.78:0/3194762992 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9250003c60 con 0x7f9258149740 2023-10-11T09:44:00.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.542+0000 7f92557fa700 1 -- 172.21.15.78:0/3194762992 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925000fe80 con 0x7f9258149740 2023-10-11T09:44:00.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.542+0000 7f925e759700 1 -- 172.21.15.78:0/3194762992 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92580c8840 con 0x7f9258149740 2023-10-11T09:44:00.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.542+0000 7f925e759700 1 -- 172.21.15.78:0/3194762992 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92581578d0 con 0x7f9258149740 2023-10-11T09:44:00.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.546+0000 7f925e759700 1 -- 172.21.15.78:0/3194762992 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92580c8a30 con 0x7f9258149740 2023-10-11T09:44:00.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.546+0000 7f92557fa700 1 -- 172.21.15.78:0/3194762992 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f9250004030 con 0x7f9258149740 2023-10-11T09:44:00.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.550+0000 7f92557fa700 1 --2- 172.21.15.78:0/3194762992 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9238065ed0 0x7f9238068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:00.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.550+0000 7f92557fa700 1 -- 172.21.15.78:0/3194762992 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f9250083fe0 con 0x7f9258149740 2023-10-11T09:44:00.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.550+0000 7f9257fff700 1 --2- 172.21.15.78:0/3194762992 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9238065ed0 0x7f9238068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:00.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.550+0000 7f9257fff700 1 --2- 172.21.15.78:0/3194762992 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9238065ed0 0x7f9238068380 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f92480011e0 tx=0x7f9248009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:00.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.554+0000 7f92557fa700 1 -- 172.21.15.78:0/3194762992 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f9250051810 con 0x7f9258149740 2023-10-11T09:44:00.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.726+0000 7f925e759700 1 -- 172.21.15.78:0/3194762992 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f925808bff0 con 0x7f9258149740 2023-10-11T09:44:00.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.734+0000 7f92557fa700 1 -- 172.21.15.78:0/3194762992 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f925001c0c0 con 0x7f9258149740 2023-10-11T09:44:00.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.738+0000 7f925e759700 1 -- 172.21.15.78:0/3194762992 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9238065ed0 msgr2=0x7f9238068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:00.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.738+0000 7f925e759700 1 --2- 172.21.15.78:0/3194762992 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9238065ed0 0x7f9238068380 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f92480011e0 tx=0x7f9248009040 comp rx=0 tx=0).stop 2023-10-11T09:44:00.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.738+0000 7f925e759700 1 -- 172.21.15.78:0/3194762992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9258149740 msgr2=0x7f92580cc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:00.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.738+0000 7f925e759700 1 --2- 172.21.15.78:0/3194762992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9258149740 0x7f92580cc660 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f92580d0580 tx=0x7f925000f710 comp rx=0 tx=0).stop 2023-10-11T09:44:00.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.738+0000 7f925e759700 1 -- 172.21.15.78:0/3194762992 shutdown_connections 2023-10-11T09:44:00.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.738+0000 7f925e759700 1 --2- 172.21.15.78:0/3194762992 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9238065ed0 0x7f9238068380 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:00.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.738+0000 7f925e759700 1 --2- 172.21.15.78:0/3194762992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9258149740 0x7f92580cc660 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:00.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.738+0000 7f925e759700 1 --2- 172.21.15.78:0/3194762992 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92581492d0 0x7f92580cc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:00.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.738+0000 7f925e759700 1 --2- 172.21.15.78:0/3194762992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9244004870 0x7f92581f22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:00.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.738+0000 7f925e759700 1 -- 172.21.15.78:0/3194762992 >> 172.21.15.78:0/3194762992 conn(0x7f925806cb50 msgr2=0x7f92580b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:00.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.738+0000 7f925e759700 1 -- 172.21.15.78:0/3194762992 shutdown_connections 2023-10-11T09:44:00.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:00.738+0000 7f925e759700 1 -- 172.21.15.78:0/3194762992 wait complete. 2023-10-11T09:44:00.763 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-11T09:44:00.764 DEBUG:teuthology.orchestra.run.smithi078:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-11T09:44:00.804 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-11T09:44:00.846 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-11T09:44:00.847 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:44:00.847 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:44:00.867 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352272k,nr_inodes=4088068,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17487 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-11T09:44:00.867 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:44:00.868 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:44:00.906 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352292k,nr_inodes=4088073,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1430 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275376k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-11T09:44:00.907 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:44:00.907 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:44:00.932 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352272k,nr_inodes=4088068,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17487 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-11T09:44:00.933 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:44:00.934 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:44:00.966 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352292k,nr_inodes=4088073,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1430 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275376k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-11T09:44:00.966 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-11T09:44:00.967 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:44:01.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.142+0000 7fab4c882700 1 Processor -- start 2023-10-11T09:44:01.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.146+0000 7fab4c882700 1 -- start start 2023-10-11T09:44:01.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.146+0000 7fab4c882700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fab48154e00 0x7fab4815b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:01.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.146+0000 7fab4c882700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab480cbbd0 0x7fab4806c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:01.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.146+0000 7fab4c882700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab480720f0 con 0x7fab48071040 2023-10-11T09:44:01.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.146+0000 7fab4c882700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab4815bb90 con 0x7fab48154e00 2023-10-11T09:44:01.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.146+0000 7fab4c882700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab480714e0 con 0x7fab480cbbd0 2023-10-11T09:44:01.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.146+0000 7fab4659c700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fab48154e00 0x7fab4815b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:01.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.146+0000 7fab46d9d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab480cbbd0 0x7fab4806c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:01.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.146+0000 7fab4659c700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fab48154e00 0x7fab4815b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:48766/0 (socket says 172.21.15.78:48766) 2023-10-11T09:44:01.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.146+0000 7fab4659c700 1 -- 172.21.15.78:0/2426823670 learned_addr learned my addr 172.21.15.78:0/2426823670 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:01.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.146+0000 7fab3ffff700 1 --1- 172.21.15.78:0/2426823670 >> v1:172.21.15.133:6789/0 conn(0x7fab48071040 0x7fab480cb4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:37940/0 (socket says 172.21.15.78:37940) 2023-10-11T09:44:01.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.150+0000 7fab4659c700 1 -- 172.21.15.78:0/2426823670 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab480cbbd0 msgr2=0x7fab4806c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:44:01.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.150+0000 7fab4659c700 1 --2- 172.21.15.78:0/2426823670 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab480cbbd0 0x7fab4806c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:01.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.150+0000 7fab4659c700 1 -- 172.21.15.78:0/2426823670 >> v1:172.21.15.133:6789/0 conn(0x7fab48071040 legacy=0x7fab480cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:01.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.150+0000 7fab4659c700 1 -- 172.21.15.78:0/2426823670 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab480717c0 con 0x7fab48154e00 2023-10-11T09:44:01.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.150+0000 7fab46d9d700 1 --2- 172.21.15.78:0/2426823670 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab480cbbd0 0x7fab4806c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:44:01.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.150+0000 7fab4659c700 1 --2- 172.21.15.78:0/2426823670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab48154e00 0x7fab4815b650 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fab38009910 tx=0x7fab38009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e410d78e8b52d70 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:01.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.150+0000 7fab45d9b700 1 -- 172.21.15.78:0/2426823670 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab38004230 con 0x7fab48154e00 2023-10-11T09:44:01.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.150+0000 7fab45d9b700 1 -- 172.21.15.78:0/2426823670 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab38004390 con 0x7fab48154e00 2023-10-11T09:44:01.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.150+0000 7fab45d9b700 1 -- 172.21.15.78:0/2426823670 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab380036c0 con 0x7fab48154e00 2023-10-11T09:44:01.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.150+0000 7fab4c882700 1 -- 172.21.15.78:0/2426823670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab48154e00 msgr2=0x7fab4815b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:01.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.150+0000 7fab4c882700 1 --2- 172.21.15.78:0/2426823670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab48154e00 0x7fab4815b650 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fab38009910 tx=0x7fab38009c20 comp rx=0 tx=0).stop 2023-10-11T09:44:01.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.150+0000 7fab4c882700 1 -- 172.21.15.78:0/2426823670 shutdown_connections 2023-10-11T09:44:01.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.150+0000 7fab4c882700 1 --2- 172.21.15.78:0/2426823670 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab480cbbd0 0x7fab4806c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:01.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.150+0000 7fab4c882700 1 --2- 172.21.15.78:0/2426823670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab48154e00 0x7fab4815b650 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:01.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.150+0000 7fab4c882700 1 -- 172.21.15.78:0/2426823670 >> 172.21.15.78:0/2426823670 conn(0x7fab4806aca0 msgr2=0x7fab480b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:01.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.150+0000 7fab4c882700 1 -- 172.21.15.78:0/2426823670 shutdown_connections 2023-10-11T09:44:01.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.150+0000 7fab4c882700 1 -- 172.21.15.78:0/2426823670 wait complete. 2023-10-11T09:44:01.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.154+0000 7fab4c882700 1 Processor -- start 2023-10-11T09:44:01.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.154+0000 7fab4c882700 1 -- start start 2023-10-11T09:44:01.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.154+0000 7fab4c882700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab48071040 0x7fab4814c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:01.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.154+0000 7fab4c882700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab480cbbd0 0x7fab4814ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:01.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.154+0000 7fab4c882700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab48154e00 0x7fab48159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:01.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.154+0000 7fab4c882700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4814cee0 con 0x7fab48071040 2023-10-11T09:44:01.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.154+0000 7fab4c882700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab480cc870 con 0x7fab48154e00 2023-10-11T09:44:01.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.154+0000 7fab4c882700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab481f5c00 con 0x7fab480cbbd0 2023-10-11T09:44:01.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.154+0000 7fab4659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab48071040 0x7fab4814c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:01.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.154+0000 7fab4659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab48071040 0x7fab4814c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35482/0 (socket says 172.21.15.78:35482) 2023-10-11T09:44:01.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.154+0000 7fab4659c700 1 -- 172.21.15.78:0/1603747573 learned_addr learned my addr 172.21.15.78:0/1603747573 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:01.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.154+0000 7fab3ffff700 1 --2- 172.21.15.78:0/1603747573 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab480cbbd0 0x7fab4814ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:01.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.154+0000 7fab46d9d700 1 --2- 172.21.15.78:0/1603747573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab48154e00 0x7fab48159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:01.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.154+0000 7fab4659c700 1 -- 172.21.15.78:0/1603747573 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab480cbbd0 msgr2=0x7fab4814ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:01.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.154+0000 7fab4659c700 1 --2- 172.21.15.78:0/1603747573 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab480cbbd0 0x7fab4814ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:01.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.154+0000 7fab4659c700 1 -- 172.21.15.78:0/1603747573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab48154e00 msgr2=0x7fab48159910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:01.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.154+0000 7fab4659c700 1 --2- 172.21.15.78:0/1603747573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab48154e00 0x7fab48159910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:01.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.154+0000 7fab4659c700 1 -- 172.21.15.78:0/1603747573 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab40009a70 con 0x7fab48071040 2023-10-11T09:44:01.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.158+0000 7fab4659c700 1 --2- 172.21.15.78:0/1603747573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab48071040 0x7fab4814c9a0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fab38004000 tx=0x7fab38004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:01.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.158+0000 7fab3effd700 1 -- 172.21.15.78:0/1603747573 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab380036e0 con 0x7fab48071040 2023-10-11T09:44:01.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.158+0000 7fab3effd700 1 -- 172.21.15.78:0/1603747573 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab38011630 con 0x7fab48071040 2023-10-11T09:44:01.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.158+0000 7fab3effd700 1 -- 172.21.15.78:0/1603747573 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab380119e0 con 0x7fab48071040 2023-10-11T09:44:01.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.158+0000 7fab4c882700 1 -- 172.21.15.78:0/1603747573 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab38009570 con 0x7fab48071040 2023-10-11T09:44:01.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.158+0000 7fab4c882700 1 -- 172.21.15.78:0/1603747573 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab481f61c0 con 0x7fab48071040 2023-10-11T09:44:01.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.158+0000 7fab4c882700 1 -- 172.21.15.78:0/1603747573 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab4804e530 con 0x7fab48071040 2023-10-11T09:44:01.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.162+0000 7fab3effd700 1 -- 172.21.15.78:0/1603747573 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fab38022070 con 0x7fab48071040 2023-10-11T09:44:01.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.162+0000 7fab3effd700 1 --2- 172.21.15.78:0/1603747573 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fab28065ed0 0x7fab28068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:01.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.162+0000 7fab3effd700 1 -- 172.21.15.78:0/1603747573 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fab38084f70 con 0x7fab48071040 2023-10-11T09:44:01.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.162+0000 7fab3ffff700 1 --2- 172.21.15.78:0/1603747573 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fab28065ed0 0x7fab28068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:01.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.162+0000 7fab3ffff700 1 --2- 172.21.15.78:0/1603747573 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fab28065ed0 0x7fab28068380 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fab4814e460 tx=0x7fab30004a70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:01.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.170+0000 7fab3effd700 1 -- 172.21.15.78:0/1603747573 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fab380527a0 con 0x7fab48071040 2023-10-11T09:44:01.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.378+0000 7fab4c882700 1 -- 172.21.15.78:0/1603747573 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab4806e1e0 con 0x7fab48071040 2023-10-11T09:44:01.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.378+0000 7fab3effd700 1 -- 172.21.15.78:0/1603747573 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+3529 (secure 0 0 0) 0x7fab38055dc0 con 0x7fab48071040 2023-10-11T09:44:01.383 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:44:01.383 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":31,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/588936111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":588936111},{"type":"v1","addr":"172.21.15.78:6836","nonce":588936111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5357,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/2574909888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2574909888},{"type":"v1","addr":"172.21.15.133:6834","nonce":2574909888}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5791,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2948223101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":2948223101},{"type":"v1","addr":"172.21.15.133:6835","nonce":2948223101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":6180,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/969267916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":969267916},{"type":"v1","addr":"172.21.15.78:6837","nonce":969267916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31}],"filesystems":[]} 2023-10-11T09:44:01.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.378+0000 7fab4c882700 1 -- 172.21.15.78:0/1603747573 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fab28065ed0 msgr2=0x7fab28068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:01.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.378+0000 7fab4c882700 1 --2- 172.21.15.78:0/1603747573 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fab28065ed0 0x7fab28068380 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fab4814e460 tx=0x7fab30004a70 comp rx=0 tx=0).stop 2023-10-11T09:44:01.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.378+0000 7fab4c882700 1 -- 172.21.15.78:0/1603747573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab48071040 msgr2=0x7fab4814c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:01.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.378+0000 7fab4c882700 1 --2- 172.21.15.78:0/1603747573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab48071040 0x7fab4814c9a0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fab38004000 tx=0x7fab38004770 comp rx=0 tx=0).stop 2023-10-11T09:44:01.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.378+0000 7fab4c882700 1 -- 172.21.15.78:0/1603747573 shutdown_connections 2023-10-11T09:44:01.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.378+0000 7fab4c882700 1 --2- 172.21.15.78:0/1603747573 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fab28065ed0 0x7fab28068380 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:01.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.378+0000 7fab4c882700 1 --2- 172.21.15.78:0/1603747573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab48154e00 0x7fab48159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:01.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.378+0000 7fab4c882700 1 --2- 172.21.15.78:0/1603747573 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab480cbbd0 0x7fab4814ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:01.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.378+0000 7fab4c882700 1 --2- 172.21.15.78:0/1603747573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab48071040 0x7fab4814c9a0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:01.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.378+0000 7fab4c882700 1 -- 172.21.15.78:0/1603747573 >> 172.21.15.78:0/1603747573 conn(0x7fab4806aca0 msgr2=0x7fab481587f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:01.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.378+0000 7fab4c882700 1 -- 172.21.15.78:0/1603747573 shutdown_connections 2023-10-11T09:44:01.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.378+0000 7fab4c882700 1 -- 172.21.15.78:0/1603747573 wait complete. 2023-10-11T09:44:01.388 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 31 2023-10-11T09:44:01.406 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-11T09:44:01.406 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-11T09:44:01.407 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-11T09:44:01.407 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:44:01.408 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-11T09:44:01.408 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-11T09:44:01.408 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-11T09:44:01.408 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:44:01.409 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-11T09:44:01.409 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-11T09:44:01.409 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-11T09:44:01.409 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:44:01.410 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-11T09:44:01.410 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-11T09:44:01.410 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-11T09:44:01.411 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:44:01.426 INFO:tasks.ceph.mds.a:Stopped 2023-10-11T09:44:01.426 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-11T09:44:01.428 INFO:tasks.ceph.mds.c:Stopped 2023-10-11T09:44:01.428 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-11T09:44:01.431 INFO:tasks.ceph.mds.b:Stopped 2023-10-11T09:44:01.431 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-11T09:44:01.433 INFO:tasks.ceph.mds.d:Stopped 2023-10-11T09:44:01.433 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-11T09:44:01.479 INFO:tasks.ceph.mds.a:Started 2023-10-11T09:44:01.479 DEBUG:teuthology.parallel:result is None 2023-10-11T09:44:01.483 INFO:tasks.ceph.mds.c:Started 2023-10-11T09:44:01.483 DEBUG:teuthology.parallel:result is None 2023-10-11T09:44:01.486 INFO:tasks.ceph.mds.b:Started 2023-10-11T09:44:01.486 DEBUG:teuthology.parallel:result is None 2023-10-11T09:44:01.490 INFO:tasks.ceph.mds.d:Started 2023-10-11T09:44:01.491 DEBUG:teuthology.parallel:result is None 2023-10-11T09:44:01.491 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-11T09:44:01.590 INFO:tasks.ceph.mds.a.smithi078.stdout:starting mds.a at 2023-10-11T09:44:01.591 INFO:tasks.ceph.mds.c.smithi078.stdout:starting mds.c at 2023-10-11T09:44:01.608 INFO:tasks.ceph.mds.d.smithi133.stdout:starting mds.d at 2023-10-11T09:44:01.609 INFO:tasks.ceph.mds.b.smithi133.stdout:starting mds.b at 2023-10-11T09:44:01.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.658+0000 7f7a3759e700 1 Processor -- start 2023-10-11T09:44:01.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.658+0000 7f7a3759e700 1 -- start start 2023-10-11T09:44:01.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.658+0000 7f7a3759e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7a380d1660 0x7f7a380d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:01.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.658+0000 7f7a3759e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f7a380d28e0 0x7f7a380d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:01.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.658+0000 7f7a3759e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a3815dc10 con 0x7f7a38156ce0 2023-10-11T09:44:01.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.658+0000 7f7a3759e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a38160380 con 0x7f7a380d28e0 2023-10-11T09:44:01.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.658+0000 7f7a3759e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a3815d930 con 0x7f7a380d1660 2023-10-11T09:44:01.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.658+0000 7f7a36d9d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f7a380d28e0 0x7f7a380d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:01.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.658+0000 7f7a35d9b700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7a380d1660 0x7f7a380d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:01.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.658+0000 7f7a3659c700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f7a38156ce0 0x7f7a38160b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:46382/0 (socket says 172.21.15.78:46382) 2023-10-11T09:44:01.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.658+0000 7f7a36d9d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f7a380d28e0 0x7f7a380d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35528/0 (socket says 172.21.15.78:35528) 2023-10-11T09:44:01.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.658+0000 7f7a3659c700 1 -- 172.21.15.78:0/3959155564 learned_addr learned my addr 172.21.15.78:0/3959155564 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:44:01.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.662+0000 7f7a36d9d700 1 -- 172.21.15.78:0/3959155564 >> v2:172.21.15.133:3300/0 conn(0x7f7a380d1660 msgr2=0x7f7a380d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:44:01.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.662+0000 7f7a36d9d700 1 --2- 172.21.15.78:0/3959155564 >> v2:172.21.15.133:3300/0 conn(0x7f7a380d1660 0x7f7a380d0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:01.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.662+0000 7f7a36d9d700 1 -- 172.21.15.78:0/3959155564 >> v1:172.21.15.78:6789/0 conn(0x7f7a38156ce0 legacy=0x7f7a38160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:01.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.662+0000 7f7a36d9d700 1 -- 172.21.15.78:0/3959155564 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a38160620 con 0x7f7a380d28e0 2023-10-11T09:44:01.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.662+0000 7f7a35d9b700 1 --2- 172.21.15.78:0/3959155564 >> v2:172.21.15.133:3300/0 conn(0x7f7a380d1660 0x7f7a380d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:44:01.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.662+0000 7f7a36d9d700 1 --2- 172.21.15.78:0/3959155564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a380d28e0 0x7f7a380d0f90 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f7a3806d2e0 tx=0x7f7a3000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=118347e73dbe9309 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:01.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.662+0000 7f7a3559a700 1 -- 172.21.15.78:0/3959155564 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3000e070 con 0x7f7a380d28e0 2023-10-11T09:44:01.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.662+0000 7f7a3559a700 1 -- 172.21.15.78:0/3959155564 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3000bb20 con 0x7f7a380d28e0 2023-10-11T09:44:01.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.662+0000 7f7a3559a700 1 -- 172.21.15.78:0/3959155564 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a30012430 con 0x7f7a380d28e0 2023-10-11T09:44:01.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.662+0000 7f7a3759e700 1 -- 172.21.15.78:0/3959155564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a380d28e0 msgr2=0x7f7a380d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:01.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.662+0000 7f7a3759e700 1 --2- 172.21.15.78:0/3959155564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a380d28e0 0x7f7a380d0f90 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f7a3806d2e0 tx=0x7f7a3000b330 comp rx=0 tx=0).stop 2023-10-11T09:44:01.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.662+0000 7f7a3759e700 1 -- 172.21.15.78:0/3959155564 shutdown_connections 2023-10-11T09:44:01.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.662+0000 7f7a3759e700 1 --2- 172.21.15.78:0/3959155564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a380d28e0 0x7f7a380d0f90 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:01.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.662+0000 7f7a3759e700 1 --2- 172.21.15.78:0/3959155564 >> v2:172.21.15.133:3300/0 conn(0x7f7a380d1660 0x7f7a380d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:01.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.662+0000 7f7a3759e700 1 -- 172.21.15.78:0/3959155564 >> 172.21.15.78:0/3959155564 conn(0x7f7a3806afb0 msgr2=0x7f7a3806b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:01.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.662+0000 7f7a3759e700 1 -- 172.21.15.78:0/3959155564 shutdown_connections 2023-10-11T09:44:01.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.662+0000 7f7a3759e700 1 -- 172.21.15.78:0/3959155564 wait complete. 2023-10-11T09:44:01.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.666+0000 7f7a3759e700 1 Processor -- start 2023-10-11T09:44:01.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.666+0000 7f7a3759e700 1 -- start start 2023-10-11T09:44:01.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.666+0000 7f7a3759e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a380d1660 0x7f7a380b72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:01.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.666+0000 7f7a3759e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a380d28e0 0x7f7a380cab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:01.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.666+0000 7f7a3759e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a38156ce0 0x7f7a381f1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:01.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.666+0000 7f7a3759e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a380b8c60 con 0x7f7a38156ce0 2023-10-11T09:44:01.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.666+0000 7f7a3759e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a380b7bc0 con 0x7f7a380d1660 2023-10-11T09:44:01.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.666+0000 7f7a3759e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a380b7ea0 con 0x7f7a380d28e0 2023-10-11T09:44:01.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.666+0000 7f7a36d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a38156ce0 0x7f7a381f1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:01.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.666+0000 7f7a36d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a38156ce0 0x7f7a381f1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35536/0 (socket says 172.21.15.78:35536) 2023-10-11T09:44:01.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.666+0000 7f7a36d9d700 1 -- 172.21.15.78:0/399925423 learned_addr learned my addr 172.21.15.78:0/399925423 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:01.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.666+0000 7f7a3659c700 1 --2- 172.21.15.78:0/399925423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a380d1660 0x7f7a380b72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:01.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.666+0000 7f7a35d9b700 1 --2- 172.21.15.78:0/399925423 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a380d28e0 0x7f7a380cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:01.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.666+0000 7f7a36d9d700 1 -- 172.21.15.78:0/399925423 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a380d28e0 msgr2=0x7f7a380cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:01.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.666+0000 7f7a36d9d700 1 --2- 172.21.15.78:0/399925423 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a380d28e0 0x7f7a380cab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:01.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.666+0000 7f7a36d9d700 1 -- 172.21.15.78:0/399925423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a380d1660 msgr2=0x7f7a380b72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:01.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.666+0000 7f7a36d9d700 1 --2- 172.21.15.78:0/399925423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a380d1660 0x7f7a380b72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:01.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.666+0000 7f7a36d9d700 1 -- 172.21.15.78:0/399925423 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a2c0094b0 con 0x7f7a38156ce0 2023-10-11T09:44:01.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.670+0000 7f7a36d9d700 1 --2- 172.21.15.78:0/399925423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a38156ce0 0x7f7a381f1520 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f7a30014040 tx=0x7f7a30004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:01.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.670+0000 7f7a26ffd700 1 -- 172.21.15.78:0/399925423 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a30012430 con 0x7f7a38156ce0 2023-10-11T09:44:01.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.670+0000 7f7a26ffd700 1 -- 172.21.15.78:0/399925423 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a30020070 con 0x7f7a38156ce0 2023-10-11T09:44:01.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.670+0000 7f7a3759e700 1 -- 172.21.15.78:0/399925423 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a30009a70 con 0x7f7a38156ce0 2023-10-11T09:44:01.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.670+0000 7f7a26ffd700 1 -- 172.21.15.78:0/399925423 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a300098f0 con 0x7f7a38156ce0 2023-10-11T09:44:01.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.670+0000 7f7a3759e700 1 -- 172.21.15.78:0/399925423 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a380b8460 con 0x7f7a38156ce0 2023-10-11T09:44:01.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.670+0000 7f7a3759e700 1 -- 172.21.15.78:0/399925423 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a380c6820 con 0x7f7a38156ce0 2023-10-11T09:44:01.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.674+0000 7f7a26ffd700 1 -- 172.21.15.78:0/399925423 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f7a30004040 con 0x7f7a38156ce0 2023-10-11T09:44:01.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.674+0000 7f7a26ffd700 1 --2- 172.21.15.78:0/399925423 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7a18065ed0 0x7f7a18068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:01.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.674+0000 7f7a26ffd700 1 -- 172.21.15.78:0/399925423 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f7a30022070 con 0x7f7a38156ce0 2023-10-11T09:44:01.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.674+0000 7f7a3659c700 1 --2- 172.21.15.78:0/399925423 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7a18065ed0 0x7f7a18068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:01.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.674+0000 7f7a3659c700 1 --2- 172.21.15.78:0/399925423 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7a18065ed0 0x7f7a18068380 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7a28005930 tx=0x7f7a28009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:01.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.678+0000 7f7a26ffd700 1 -- 172.21.15.78:0/399925423 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f7a30052e40 con 0x7f7a38156ce0 2023-10-11T09:44:01.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:01.862+0000 7f7a3759e700 1 -- 172.21.15.78:0/399925423 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f7a38065d40 con 0x7f7a38156ce0 2023-10-11T09:44:02.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.642+0000 7f7a26ffd700 1 -- 172.21.15.78:0/399925423 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v57) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7a30056460 con 0x7f7a38156ce0 2023-10-11T09:44:02.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.646+0000 7f7a3759e700 1 -- 172.21.15.78:0/399925423 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7a18065ed0 msgr2=0x7f7a18068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:02.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.646+0000 7f7a3759e700 1 --2- 172.21.15.78:0/399925423 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7a18065ed0 0x7f7a18068380 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7a28005930 tx=0x7f7a28009040 comp rx=0 tx=0).stop 2023-10-11T09:44:02.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.646+0000 7f7a3759e700 1 -- 172.21.15.78:0/399925423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a38156ce0 msgr2=0x7f7a381f1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:02.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.646+0000 7f7a3759e700 1 --2- 172.21.15.78:0/399925423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a38156ce0 0x7f7a381f1520 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f7a30014040 tx=0x7f7a30004af0 comp rx=0 tx=0).stop 2023-10-11T09:44:02.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.646+0000 7f7a3759e700 1 -- 172.21.15.78:0/399925423 shutdown_connections 2023-10-11T09:44:02.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.646+0000 7f7a3759e700 1 --2- 172.21.15.78:0/399925423 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7a18065ed0 0x7f7a18068380 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:02.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.646+0000 7f7a3759e700 1 --2- 172.21.15.78:0/399925423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a38156ce0 0x7f7a381f1520 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:02.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.646+0000 7f7a3759e700 1 --2- 172.21.15.78:0/399925423 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a380d28e0 0x7f7a380cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:02.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.646+0000 7f7a3759e700 1 --2- 172.21.15.78:0/399925423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a380d1660 0x7f7a380b72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:02.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.646+0000 7f7a3759e700 1 -- 172.21.15.78:0/399925423 >> 172.21.15.78:0/399925423 conn(0x7f7a3806afb0 msgr2=0x7f7a380d0780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:02.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.646+0000 7f7a3759e700 1 -- 172.21.15.78:0/399925423 shutdown_connections 2023-10-11T09:44:02.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.646+0000 7f7a3759e700 1 -- 172.21.15.78:0/399925423 wait complete. 2023-10-11T09:44:02.655 INFO:teuthology.orchestra.run.smithi078.stderr: removed all blocklist entries 2023-10-11T09:44:02.675 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-11T09:44:02.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.850+0000 7efe77037700 1 Processor -- start 2023-10-11T09:44:02.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.850+0000 7efe77037700 1 -- start start 2023-10-11T09:44:02.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.850+0000 7efe77037700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7efe70154e00 0x7efe7015b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:02.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.850+0000 7efe77037700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe70071040 0x7efe700cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:02.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.850+0000 7efe77037700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe700720f0 con 0x7efe700cba00 2023-10-11T09:44:02.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.850+0000 7efe77037700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe7015bb90 con 0x7efe70154e00 2023-10-11T09:44:02.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.850+0000 7efe77037700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe700714e0 con 0x7efe70071040 2023-10-11T09:44:02.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.850+0000 7efe755d4700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7efe700cba00 0x7efe7015e1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:46386/0 (socket says 172.21.15.78:46386) 2023-10-11T09:44:02.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.850+0000 7efe755d4700 1 -- 172.21.15.78:0/2509660764 learned_addr learned my addr 172.21.15.78:0/2509660764 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:44:02.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.850+0000 7efe74dd3700 1 --2- 172.21.15.78:0/2509660764 >> v2:172.21.15.78:3300/0 conn(0x7efe70154e00 0x7efe7015b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:02.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.854+0000 7efe6ffff700 1 --2- 172.21.15.78:0/2509660764 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe70071040 0x7efe700cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:02.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.854+0000 7efe74dd3700 1 -- 172.21.15.78:0/2509660764 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe70071040 msgr2=0x7efe700cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:44:02.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.854+0000 7efe74dd3700 1 --2- 172.21.15.78:0/2509660764 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe70071040 0x7efe700cb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:02.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.854+0000 7efe6f7fe700 1 -- 172.21.15.78:0/2509660764 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1433713238 0 0) 0x7efe700720f0 con 0x7efe700cba00 2023-10-11T09:44:02.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.854+0000 7efe74dd3700 1 -- 172.21.15.78:0/2509660764 >> v1:172.21.15.78:6789/0 conn(0x7efe700cba00 legacy=0x7efe7015e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:02.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.854+0000 7efe74dd3700 1 -- 172.21.15.78:0/2509660764 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe700717c0 con 0x7efe70154e00 2023-10-11T09:44:02.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.854+0000 7efe74dd3700 1 --2- 172.21.15.78:0/2509660764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe70154e00 0x7efe7015b650 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7efe64009850 tx=0x7efe64009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec3675b82606637 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:02.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.854+0000 7efe6f7fe700 1 -- 172.21.15.78:0/2509660764 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe64011830 con 0x7efe70154e00 2023-10-11T09:44:02.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.854+0000 7efe6f7fe700 1 -- 172.21.15.78:0/2509660764 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe64011990 con 0x7efe70154e00 2023-10-11T09:44:02.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.854+0000 7efe6f7fe700 1 -- 172.21.15.78:0/2509660764 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe64011d20 con 0x7efe70154e00 2023-10-11T09:44:02.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.854+0000 7efe77037700 1 -- 172.21.15.78:0/2509660764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe70154e00 msgr2=0x7efe7015b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:02.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.854+0000 7efe77037700 1 --2- 172.21.15.78:0/2509660764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe70154e00 0x7efe7015b650 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7efe64009850 tx=0x7efe64009b60 comp rx=0 tx=0).stop 2023-10-11T09:44:02.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.854+0000 7efe77037700 1 -- 172.21.15.78:0/2509660764 shutdown_connections 2023-10-11T09:44:02.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.854+0000 7efe77037700 1 --2- 172.21.15.78:0/2509660764 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe70071040 0x7efe700cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:02.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.854+0000 7efe77037700 1 --2- 172.21.15.78:0/2509660764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe70154e00 0x7efe7015b650 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:02.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.854+0000 7efe77037700 1 -- 172.21.15.78:0/2509660764 >> 172.21.15.78:0/2509660764 conn(0x7efe7006aca0 msgr2=0x7efe700b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:02.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.854+0000 7efe77037700 1 -- 172.21.15.78:0/2509660764 shutdown_connections 2023-10-11T09:44:02.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.854+0000 7efe77037700 1 -- 172.21.15.78:0/2509660764 wait complete. 2023-10-11T09:44:02.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.858+0000 7efe77037700 1 Processor -- start 2023-10-11T09:44:02.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.858+0000 7efe77037700 1 -- start start 2023-10-11T09:44:02.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.862+0000 7efe77037700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe70071040 0x7efe700c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:02.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.862+0000 7efe77037700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe700cba00 0x7efe70157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:02.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.862+0000 7efe77037700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe70154e00 0x7efe701ef720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:02.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.862+0000 7efe77037700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe700b20b0 con 0x7efe700cba00 2023-10-11T09:44:02.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.862+0000 7efe77037700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe700ca3a0 con 0x7efe70071040 2023-10-11T09:44:02.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.862+0000 7efe77037700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe700ca680 con 0x7efe70154e00 2023-10-11T09:44:02.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.862+0000 7efe6ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe700cba00 0x7efe70157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:02.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.862+0000 7efe6ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe700cba00 0x7efe70157f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35552/0 (socket says 172.21.15.78:35552) 2023-10-11T09:44:02.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.862+0000 7efe6ffff700 1 -- 172.21.15.78:0/1870017392 learned_addr learned my addr 172.21.15.78:0/1870017392 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:02.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.862+0000 7efe74dd3700 1 --2- 172.21.15.78:0/1870017392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe70071040 0x7efe700c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:02.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.862+0000 7efe755d4700 1 --2- 172.21.15.78:0/1870017392 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe70154e00 0x7efe701ef720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:02.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.862+0000 7efe6ffff700 1 -- 172.21.15.78:0/1870017392 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe70154e00 msgr2=0x7efe701ef720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:02.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.862+0000 7efe6ffff700 1 --2- 172.21.15.78:0/1870017392 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe70154e00 0x7efe701ef720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:02.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.862+0000 7efe6ffff700 1 -- 172.21.15.78:0/1870017392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe70071040 msgr2=0x7efe700c9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:02.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.862+0000 7efe6ffff700 1 --2- 172.21.15.78:0/1870017392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe70071040 0x7efe700c9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:02.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.862+0000 7efe6ffff700 1 -- 172.21.15.78:0/1870017392 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe640094b0 con 0x7efe700cba00 2023-10-11T09:44:02.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.862+0000 7efe6ffff700 1 --2- 172.21.15.78:0/1870017392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe700cba00 0x7efe70157f70 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7efe5c00e730 tx=0x7efe5c00ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:02.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.862+0000 7efe6d7fa700 1 -- 172.21.15.78:0/1870017392 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5c00c070 con 0x7efe700cba00 2023-10-11T09:44:02.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.862+0000 7efe6d7fa700 1 -- 172.21.15.78:0/1870017392 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe5c013610 con 0x7efe700cba00 2023-10-11T09:44:02.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.862+0000 7efe6d7fa700 1 -- 172.21.15.78:0/1870017392 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5c0139c0 con 0x7efe700cba00 2023-10-11T09:44:02.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.862+0000 7efe77037700 1 -- 172.21.15.78:0/1870017392 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe700ca960 con 0x7efe700cba00 2023-10-11T09:44:02.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.862+0000 7efe77037700 1 -- 172.21.15.78:0/1870017392 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe701efc60 con 0x7efe700cba00 2023-10-11T09:44:02.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.866+0000 7efe77037700 1 -- 172.21.15.78:0/1870017392 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe7004e530 con 0x7efe700cba00 2023-10-11T09:44:02.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.866+0000 7efe6d7fa700 1 -- 172.21.15.78:0/1870017392 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7efe5c013b20 con 0x7efe700cba00 2023-10-11T09:44:02.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.870+0000 7efe6d7fa700 1 --2- 172.21.15.78:0/1870017392 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efe54065ed0 0x7efe54068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:02.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.870+0000 7efe6d7fa700 1 -- 172.21.15.78:0/1870017392 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5210+0+0 (secure 0 0 0) 0x7efe5c083200 con 0x7efe700cba00 2023-10-11T09:44:02.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.870+0000 7efe74dd3700 1 --2- 172.21.15.78:0/1870017392 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efe54065ed0 0x7efe54068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:02.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.870+0000 7efe74dd3700 1 --2- 172.21.15.78:0/1870017392 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efe54065ed0 0x7efe54068380 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7efe64014040 tx=0x7efe64008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:02.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:02.874+0000 7efe6d7fa700 1 -- 172.21.15.78:0/1870017392 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7efe5c050b90 con 0x7efe700cba00 2023-10-11T09:44:03.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.106+0000 7efe77037700 1 -- 172.21.15.78:0/1870017392 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7efe700b21f0 con 0x7efe700cba00 2023-10-11T09:44:03.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.106+0000 7efe6d7fa700 1 -- 172.21.15.78:0/1870017392 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7efe5c0541b0 con 0x7efe700cba00 2023-10-11T09:44:03.112 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:44:03.112 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:44:03.112 INFO:teuthology.orchestra.run.smithi078.stdout: "auth_dump": [ 2023-10-11T09:44:03.113 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:44:03.113 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.a", 2023-10-11T09:44:03.113 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZlL58cAxAALe+WtkV/aLopy1w9i6OVRg==", 2023-10-11T09:44:03.113 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:44:03.113 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:44:03.114 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2023-10-11T09:44:03.114 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2023-10-11T09:44:03.114 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:44:03.114 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:44:03.114 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:44:03.115 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:44:03.115 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.b", 2023-10-11T09:44:03.115 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZliPLmFhAA+6wegaD6yTUsttFzo9OrIg==", 2023-10-11T09:44:03.115 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:44:03.115 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:44:03.115 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2023-10-11T09:44:03.116 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2023-10-11T09:44:03.116 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:44:03.116 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:44:03.116 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:44:03.116 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:44:03.117 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.c", 2023-10-11T09:44:03.117 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZl9jJBDhAAjos7/02OqF0WldFm70/fVA==", 2023-10-11T09:44:03.117 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:44:03.117 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:44:03.117 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2023-10-11T09:44:03.117 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2023-10-11T09:44:03.118 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:44:03.118 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:44:03.118 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:44:03.118 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:44:03.118 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.d", 2023-10-11T09:44:03.119 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZlTb/4IRAAeXtcbWVB5TnSoYch+uMZWw==", 2023-10-11T09:44:03.119 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:44:03.119 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:44:03.119 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2023-10-11T09:44:03.119 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2023-10-11T09:44:03.120 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:44:03.120 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:44:03.120 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:44:03.120 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:44:03.120 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.0", 2023-10-11T09:44:03.120 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBYbSZldbrVFxAA6ogUpCcFO8LhHn4zdkudNQ==", 2023-10-11T09:44:03.121 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:44:03.121 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:44:03.121 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:44:03.121 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:44:03.121 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:44:03.122 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:44:03.122 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:44:03.122 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.1", 2023-10-11T09:44:03.122 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBbbSZl+mkPJxAA4Rrt0j5lzXapYjXhdulhNQ==", 2023-10-11T09:44:03.122 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:44:03.122 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:44:03.123 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:44:03.123 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:44:03.123 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:44:03.123 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:44:03.123 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:44:03.124 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.2", 2023-10-11T09:44:03.124 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBebSZlZ6lSNRAA3ltw4ZTkFNpudCPc2zTYzQ==", 2023-10-11T09:44:03.124 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:44:03.124 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:44:03.124 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:44:03.125 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:44:03.125 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:44:03.125 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:44:03.125 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:44:03.125 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.3", 2023-10-11T09:44:03.126 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBibSZlsE+jBxAA2Ix2JAFTUxljpHAwPpJeUA==", 2023-10-11T09:44:03.126 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:44:03.126 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:44:03.126 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:44:03.126 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:44:03.127 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:44:03.127 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:44:03.127 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:44:03.127 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.4", 2023-10-11T09:44:03.127 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBobSZlII4OGxAA0Z5wGuu4wOs/14tQRnHB1Q==", 2023-10-11T09:44:03.127 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:44:03.128 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:44:03.128 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:44:03.128 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:44:03.128 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:44:03.128 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:44:03.128 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:44:03.129 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.5", 2023-10-11T09:44:03.129 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBrbSZlG6aSKhAAZq179SfpkelcJJLaUcLDCg==", 2023-10-11T09:44:03.129 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:44:03.129 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:44:03.129 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:44:03.130 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:44:03.130 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:44:03.130 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:44:03.130 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:44:03.130 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.6", 2023-10-11T09:44:03.131 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBubSZl3W89NxAAgUrd0/yMOWlrhAOU1BpM9A==", 2023-10-11T09:44:03.131 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:44:03.131 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:44:03.131 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:44:03.131 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:44:03.131 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:44:03.132 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:44:03.132 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:44:03.132 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.7", 2023-10-11T09:44:03.132 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBybSZloIjyCRAAjmvyK9zMSEsV8bVebwzPxg==", 2023-10-11T09:44:03.132 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:44:03.132 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:44:03.133 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:44:03.133 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:44:03.133 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:44:03.133 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:44:03.133 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:44:03.134 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.0", 2023-10-11T09:44:03.134 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZlmGmCKBAAOI8EvyRwg8HZt+q/vUy5Ow==", 2023-10-11T09:44:03.134 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:44:03.134 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:44:03.134 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow r", 2023-10-11T09:44:03.135 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-11T09:44:03.135 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:44:03.135 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:44:03.135 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:44:03.135 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.1", 2023-10-11T09:44:03.135 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZlYvfHNBAAReum0efx4MmqpLMxFnTROQ==", 2023-10-11T09:44:03.136 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:44:03.136 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:44:03.136 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2023-10-11T09:44:03.136 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2023-10-11T09:44:03.136 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2023-10-11T09:44:03.137 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:44:03.137 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:44:03.137 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:44:03.137 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.2", 2023-10-11T09:44:03.137 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZlbd9lLhAA9LAjreTcxwAjhiNlKndl+Q==", 2023-10-11T09:44:03.137 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:44:03.138 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:44:03.138 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2023-10-11T09:44:03.138 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2023-10-11T09:44:03.138 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2023-10-11T09:44:03.138 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:44:03.139 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:44:03.139 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:44:03.139 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.3", 2023-10-11T09:44:03.139 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZlHUjeOhAASPKTRvsillob4gjdW0akhQ==", 2023-10-11T09:44:03.139 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:44:03.141 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:44:03.141 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2023-10-11T09:44:03.141 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2023-10-11T09:44:03.141 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2023-10-11T09:44:03.142 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:44:03.142 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:44:03.142 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:44:03.142 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.admin", 2023-10-11T09:44:03.142 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBTbSZlhqcOFRAA/lkGYL03++trOuIqNLS0kQ==", 2023-10-11T09:44:03.142 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:44:03.143 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2023-10-11T09:44:03.143 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2023-10-11T09:44:03.143 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2023-10-11T09:44:03.143 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:44:03.143 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:44:03.143 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:44:03.144 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:44:03.144 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mgr.x", 2023-10-11T09:44:03.144 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBTbSZlS/TjNxAAMkq9xRMwgsbLlT4FdobATw==", 2023-10-11T09:44:03.144 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:44:03.144 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2023-10-11T09:44:03.145 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile mgr", 2023-10-11T09:44:03.145 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:44:03.145 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:44:03.145 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:44:03.145 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:44:03.145 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mgr.y", 2023-10-11T09:44:03.146 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBTbSZlnv4yMRAAQh8w18k76KRjdCvSDJ4QbA==", 2023-10-11T09:44:03.146 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:44:03.146 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2023-10-11T09:44:03.146 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile mgr", 2023-10-11T09:44:03.146 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:44:03.147 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:44:03.147 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:44:03.147 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2023-10-11T09:44:03.147 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:44:03.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.110+0000 7efe77037700 1 -- 172.21.15.78:0/1870017392 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efe54065ed0 msgr2=0x7efe54068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:03.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.110+0000 7efe77037700 1 --2- 172.21.15.78:0/1870017392 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efe54065ed0 0x7efe54068380 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7efe64014040 tx=0x7efe64008030 comp rx=0 tx=0).stop 2023-10-11T09:44:03.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.110+0000 7efe77037700 1 -- 172.21.15.78:0/1870017392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe700cba00 msgr2=0x7efe70157f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:03.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.110+0000 7efe77037700 1 --2- 172.21.15.78:0/1870017392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe700cba00 0x7efe70157f70 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7efe5c00e730 tx=0x7efe5c00ea40 comp rx=0 tx=0).stop 2023-10-11T09:44:03.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.110+0000 7efe77037700 1 -- 172.21.15.78:0/1870017392 shutdown_connections 2023-10-11T09:44:03.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.110+0000 7efe77037700 1 --2- 172.21.15.78:0/1870017392 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efe54065ed0 0x7efe54068380 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:03.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.110+0000 7efe77037700 1 --2- 172.21.15.78:0/1870017392 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe70154e00 0x7efe701ef720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:03.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.110+0000 7efe77037700 1 --2- 172.21.15.78:0/1870017392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe700cba00 0x7efe70157f70 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:03.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.110+0000 7efe77037700 1 --2- 172.21.15.78:0/1870017392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe70071040 0x7efe700c9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:03.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.110+0000 7efe77037700 1 -- 172.21.15.78:0/1870017392 >> 172.21.15.78:0/1870017392 conn(0x7efe7006aca0 msgr2=0x7efe7015d670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:03.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.110+0000 7efe77037700 1 -- 172.21.15.78:0/1870017392 shutdown_connections 2023-10-11T09:44:03.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.110+0000 7efe77037700 1 -- 172.21.15.78:0/1870017392 wait complete. 2023-10-11T09:44:03.151 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:44:03.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.318+0000 7fe55b3f9700 1 Processor -- start 2023-10-11T09:44:03.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.322+0000 7fe55b3f9700 1 -- start start 2023-10-11T09:44:03.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.322+0000 7fe55b3f9700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe554159490 0x7fe5540ce500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:03.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.322+0000 7fe55b3f9700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe55415db90 0x7fe554070100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:03.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.322+0000 7fe55b3f9700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe554159c50 con 0x7fe55415d030 2023-10-11T09:44:03.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.322+0000 7fe55b3f9700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5540cea40 con 0x7fe55415db90 2023-10-11T09:44:03.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.322+0000 7fe55b3f9700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe554159ef0 con 0x7fe554159490 2023-10-11T09:44:03.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.322+0000 7fe559996700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe55415db90 0x7fe554070100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:03.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.322+0000 7fe559195700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe554159490 0x7fe5540ce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:03.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.322+0000 7fe558994700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe55415d030 0x7fe55415d480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:37962/0 (socket says 172.21.15.78:37962) 2023-10-11T09:44:03.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.322+0000 7fe559996700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe55415db90 0x7fe554070100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:48868/0 (socket says 172.21.15.78:48868) 2023-10-11T09:44:03.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.322+0000 7fe558994700 1 -- 172.21.15.78:0/478605438 learned_addr learned my addr 172.21.15.78:0/478605438 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:44:03.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.322+0000 7fe559195700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe554159490 0x7fe5540ce500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:40046/0 (socket says 172.21.15.78:40046) 2023-10-11T09:44:03.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.322+0000 7fe54bfff700 1 -- 172.21.15.78:0/478605438 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 884004336 0 0) 0x7fe554159c50 con 0x7fe55415d030 2023-10-11T09:44:03.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.322+0000 7fe54bfff700 1 -- 172.21.15.78:0/478605438 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5400035d0 con 0x7fe55415d030 2023-10-11T09:44:03.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.322+0000 7fe559996700 1 -- 172.21.15.78:0/478605438 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe554159490 msgr2=0x7fe5540ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:44:03.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.322+0000 7fe559996700 1 --2- 172.21.15.78:0/478605438 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe554159490 0x7fe5540ce500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:03.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.322+0000 7fe559996700 1 -- 172.21.15.78:0/478605438 >> v1:172.21.15.133:6789/0 conn(0x7fe55415d030 legacy=0x7fe55415d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:03.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.322+0000 7fe559996700 1 -- 172.21.15.78:0/478605438 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe55415a1d0 con 0x7fe55415db90 2023-10-11T09:44:03.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.326+0000 7fe559996700 1 --2- 172.21.15.78:0/478605438 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe55415db90 0x7fe554070100 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe54c009d90 tx=0x7fe54c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdd39645c5126434 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:03.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.326+0000 7fe54bfff700 1 -- 172.21.15.78:0/478605438 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c00e070 con 0x7fe55415db90 2023-10-11T09:44:03.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.326+0000 7fe54bfff700 1 -- 172.21.15.78:0/478605438 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe54c00bb20 con 0x7fe55415db90 2023-10-11T09:44:03.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.326+0000 7fe54bfff700 1 -- 172.21.15.78:0/478605438 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c012430 con 0x7fe55415db90 2023-10-11T09:44:03.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.326+0000 7fe55b3f9700 1 -- 172.21.15.78:0/478605438 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe55415db90 msgr2=0x7fe554070100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:03.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.326+0000 7fe55b3f9700 1 --2- 172.21.15.78:0/478605438 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe55415db90 0x7fe554070100 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe54c009d90 tx=0x7fe54c00b330 comp rx=0 tx=0).stop 2023-10-11T09:44:03.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.326+0000 7fe55b3f9700 1 -- 172.21.15.78:0/478605438 shutdown_connections 2023-10-11T09:44:03.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.326+0000 7fe55b3f9700 1 --2- 172.21.15.78:0/478605438 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe55415db90 0x7fe554070100 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:03.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.326+0000 7fe55b3f9700 1 --2- 172.21.15.78:0/478605438 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe554159490 0x7fe5540ce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:03.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.326+0000 7fe55b3f9700 1 -- 172.21.15.78:0/478605438 >> 172.21.15.78:0/478605438 conn(0x7fe55406aca0 msgr2=0x7fe5540b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:03.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.326+0000 7fe55b3f9700 1 -- 172.21.15.78:0/478605438 shutdown_connections 2023-10-11T09:44:03.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.326+0000 7fe55b3f9700 1 -- 172.21.15.78:0/478605438 wait complete. 2023-10-11T09:44:03.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.330+0000 7fe55b3f9700 1 Processor -- start 2023-10-11T09:44:03.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.330+0000 7fe55b3f9700 1 -- start start 2023-10-11T09:44:03.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.330+0000 7fe55b3f9700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe554159490 0x7fe55414eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:03.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.330+0000 7fe55b3f9700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe55415d030 0x7fe5540b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:03.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.330+0000 7fe55b3f9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe55415db90 0x7fe554156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:03.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.330+0000 7fe55b3f9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe554157400 con 0x7fe55415db90 2023-10-11T09:44:03.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.330+0000 7fe55b3f9700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe554158240 con 0x7fe55415d030 2023-10-11T09:44:03.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.330+0000 7fe55b3f9700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe554157630 con 0x7fe554159490 2023-10-11T09:44:03.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.330+0000 7fe559996700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe55415db90 0x7fe554156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:03.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.330+0000 7fe559996700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe55415db90 0x7fe554156ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35556/0 (socket says 172.21.15.78:35556) 2023-10-11T09:44:03.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.330+0000 7fe559996700 1 -- 172.21.15.78:0/2971767552 learned_addr learned my addr 172.21.15.78:0/2971767552 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:03.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.330+0000 7fe559195700 1 --2- 172.21.15.78:0/2971767552 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe554159490 0x7fe55414eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:03.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.330+0000 7fe558994700 1 --2- 172.21.15.78:0/2971767552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe55415d030 0x7fe5540b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:03.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.330+0000 7fe559195700 1 -- 172.21.15.78:0/2971767552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe55415d030 msgr2=0x7fe5540b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:03.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.330+0000 7fe559195700 1 --2- 172.21.15.78:0/2971767552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe55415d030 0x7fe5540b2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:03.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.330+0000 7fe559195700 1 -- 172.21.15.78:0/2971767552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe55415db90 msgr2=0x7fe554156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:03.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.330+0000 7fe559195700 1 --2- 172.21.15.78:0/2971767552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe55415db90 0x7fe554156ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:03.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.330+0000 7fe559195700 1 -- 172.21.15.78:0/2971767552 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54c009a70 con 0x7fe554159490 2023-10-11T09:44:03.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.330+0000 7fe559996700 1 --2- 172.21.15.78:0/2971767552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe55415db90 0x7fe554156ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:44:03.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.334+0000 7fe559195700 1 --2- 172.21.15.78:0/2971767552 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe554159490 0x7fe55414eae0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe55000eb40 tx=0x7fe55000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:03.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.334+0000 7fe549ffb700 1 -- 172.21.15.78:0/2971767552 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55000dc50 con 0x7fe554159490 2023-10-11T09:44:03.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.334+0000 7fe549ffb700 1 -- 172.21.15.78:0/2971767552 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55000ddb0 con 0x7fe554159490 2023-10-11T09:44:03.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.334+0000 7fe549ffb700 1 -- 172.21.15.78:0/2971767552 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe550010640 con 0x7fe554159490 2023-10-11T09:44:03.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.334+0000 7fe55b3f9700 1 -- 172.21.15.78:0/2971767552 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe554157910 con 0x7fe554159490 2023-10-11T09:44:03.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.334+0000 7fe55b3f9700 1 -- 172.21.15.78:0/2971767552 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe554157ed0 con 0x7fe554159490 2023-10-11T09:44:03.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.334+0000 7fe55b3f9700 1 -- 172.21.15.78:0/2971767552 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe55404e530 con 0x7fe554159490 2023-10-11T09:44:03.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.338+0000 7fe549ffb700 1 -- 172.21.15.78:0/2971767552 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fe5500107a0 con 0x7fe554159490 2023-10-11T09:44:03.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.338+0000 7fe549ffb700 1 --2- 172.21.15.78:0/2971767552 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe538065ed0 0x7fe538068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:03.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.338+0000 7fe549ffb700 1 -- 172.21.15.78:0/2971767552 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe550083820 con 0x7fe554159490 2023-10-11T09:44:03.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.338+0000 7fe558994700 1 --2- 172.21.15.78:0/2971767552 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe538065ed0 0x7fe538068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:03.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.338+0000 7fe558994700 1 --2- 172.21.15.78:0/2971767552 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe538065ed0 0x7fe538068380 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe5541f51b0 tx=0x7fe544007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:03.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.346+0000 7fe549ffb700 1 -- 172.21.15.78:0/2971767552 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fe550051120 con 0x7fe554159490 2023-10-11T09:44:03.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.526+0000 7fe55b3f9700 1 -- 172.21.15.78:0/2971767552 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe5540cdd00 con 0x7fe554159490 2023-10-11T09:44:03.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.526+0000 7fe549ffb700 1 -- 172.21.15.78:0/2971767552 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+14171 (secure 0 0 0) 0x7fe550054740 con 0x7fe554159490 2023-10-11T09:44:03.534 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:44:03.534 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":57,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:44:02.635679+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:44:03.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.534+0000 7fe55b3f9700 1 -- 172.21.15.78:0/2971767552 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe538065ed0 msgr2=0x7fe538068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:03.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.534+0000 7fe55b3f9700 1 --2- 172.21.15.78:0/2971767552 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe538065ed0 0x7fe538068380 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe5541f51b0 tx=0x7fe544007de0 comp rx=0 tx=0).stop 2023-10-11T09:44:03.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.534+0000 7fe55b3f9700 1 -- 172.21.15.78:0/2971767552 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe554159490 msgr2=0x7fe55414eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:03.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.534+0000 7fe55b3f9700 1 --2- 172.21.15.78:0/2971767552 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe554159490 0x7fe55414eae0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe55000eb40 tx=0x7fe55000ee50 comp rx=0 tx=0).stop 2023-10-11T09:44:03.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.534+0000 7fe55b3f9700 1 -- 172.21.15.78:0/2971767552 shutdown_connections 2023-10-11T09:44:03.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.534+0000 7fe55b3f9700 1 --2- 172.21.15.78:0/2971767552 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe538065ed0 0x7fe538068380 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:03.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.534+0000 7fe55b3f9700 1 --2- 172.21.15.78:0/2971767552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe55415db90 0x7fe554156ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:03.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.534+0000 7fe55b3f9700 1 --2- 172.21.15.78:0/2971767552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe55415d030 0x7fe5540b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:03.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.534+0000 7fe55b3f9700 1 --2- 172.21.15.78:0/2971767552 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe554159490 0x7fe55414eae0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:03.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.534+0000 7fe55b3f9700 1 -- 172.21.15.78:0/2971767552 >> 172.21.15.78:0/2971767552 conn(0x7fe55406aca0 msgr2=0x7fe5540c9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:03.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.534+0000 7fe55b3f9700 1 -- 172.21.15.78:0/2971767552 shutdown_connections 2023-10-11T09:44:03.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.534+0000 7fe55b3f9700 1 -- 172.21.15.78:0/2971767552 wait complete. 2023-10-11T09:44:03.562 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:44:03.563 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:44:03.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.730+0000 7fec6d7df700 1 Processor -- start 2023-10-11T09:44:03.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.730+0000 7fec6d7df700 1 -- start start 2023-10-11T09:44:03.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.730+0000 7fec6d7df700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fec680b8890 0x7fec680d2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:03.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.730+0000 7fec6d7df700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fec680d3510 0x7fec680d3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:03.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.730+0000 7fec6d7df700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec680cd690 con 0x7fec680b84c0 2023-10-11T09:44:03.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.730+0000 7fec6d7df700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec680d3eb0 con 0x7fec680d3510 2023-10-11T09:44:03.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.730+0000 7fec6d7df700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec680cdea0 con 0x7fec680b8890 2023-10-11T09:44:03.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.734+0000 7fec677fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fec680d3510 0x7fec680d3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:03.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.734+0000 7fec677fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fec680d3510 0x7fec680d3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35562/0 (socket says 172.21.15.78:35562) 2023-10-11T09:44:03.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.734+0000 7fec677fe700 1 -- 172.21.15.78:0/238708114 learned_addr learned my addr 172.21.15.78:0/238708114 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:03.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.734+0000 7fec667fc700 1 --2- 172.21.15.78:0/238708114 >> v2:172.21.15.133:3300/0 conn(0x7fec680b8890 0x7fec680d2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:03.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.734+0000 7fec677fe700 1 -- 172.21.15.78:0/238708114 >> v2:172.21.15.133:3300/0 conn(0x7fec680b8890 msgr2=0x7fec680d2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:44:03.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.734+0000 7fec677fe700 1 --2- 172.21.15.78:0/238708114 >> v2:172.21.15.133:3300/0 conn(0x7fec680b8890 0x7fec680d2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:03.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.734+0000 7fec677fe700 1 -- 172.21.15.78:0/238708114 >> v1:172.21.15.133:6789/0 conn(0x7fec680b84c0 legacy=0x7fec680cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:44:03.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.734+0000 7fec677fe700 1 -- 172.21.15.78:0/238708114 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec680cd8e0 con 0x7fec680d3510 2023-10-11T09:44:03.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.734+0000 7fec677fe700 1 --2- 172.21.15.78:0/238708114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec680d3510 0x7fec680d3970 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fec6808c380 tx=0x7fec6000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b873a03afc6d4c72 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:03.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.734+0000 7fec65ffb700 1 -- 172.21.15.78:0/238708114 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6000e070 con 0x7fec680d3510 2023-10-11T09:44:03.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.734+0000 7fec65ffb700 1 -- 172.21.15.78:0/238708114 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec60003a40 con 0x7fec680d3510 2023-10-11T09:44:03.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.734+0000 7fec65ffb700 1 -- 172.21.15.78:0/238708114 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec60003df0 con 0x7fec680d3510 2023-10-11T09:44:03.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.734+0000 7fec6d7df700 1 -- 172.21.15.78:0/238708114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec680d3510 msgr2=0x7fec680d3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:03.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.734+0000 7fec6d7df700 1 --2- 172.21.15.78:0/238708114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec680d3510 0x7fec680d3970 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fec6808c380 tx=0x7fec6000b330 comp rx=0 tx=0).stop 2023-10-11T09:44:03.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.734+0000 7fec6d7df700 1 -- 172.21.15.78:0/238708114 shutdown_connections 2023-10-11T09:44:03.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.734+0000 7fec6d7df700 1 --2- 172.21.15.78:0/238708114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec680d3510 0x7fec680d3970 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:03.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.734+0000 7fec6d7df700 1 --2- 172.21.15.78:0/238708114 >> v2:172.21.15.133:3300/0 conn(0x7fec680b8890 0x7fec680d2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:03.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.734+0000 7fec6d7df700 1 -- 172.21.15.78:0/238708114 >> 172.21.15.78:0/238708114 conn(0x7fec6806cb50 msgr2=0x7fec6806cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:03.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.734+0000 7fec6d7df700 1 -- 172.21.15.78:0/238708114 shutdown_connections 2023-10-11T09:44:03.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.734+0000 7fec6d7df700 1 -- 172.21.15.78:0/238708114 wait complete. 2023-10-11T09:44:03.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.738+0000 7fec6d7df700 1 Processor -- start 2023-10-11T09:44:03.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.738+0000 7fec6d7df700 1 -- start start 2023-10-11T09:44:03.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.742+0000 7fec6d7df700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fec680b84c0 0x7fec6814c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:03.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.742+0000 7fec6d7df700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec680b8890 0x7fec6814c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:03.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.742+0000 7fec6d7df700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec680d3510 0x7fec681f57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:03.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.742+0000 7fec6d7df700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec681e2650 con 0x7fec680d3510 2023-10-11T09:44:03.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.742+0000 7fec6d7df700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec681e2930 con 0x7fec680b8890 2023-10-11T09:44:03.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.742+0000 7fec6d7df700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec680c7850 con 0x7fec680b84c0 2023-10-11T09:44:03.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.742+0000 7fec677fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec680d3510 0x7fec681f57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:03.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.742+0000 7fec677fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec680d3510 0x7fec681f57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35574/0 (socket says 172.21.15.78:35574) 2023-10-11T09:44:03.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.742+0000 7fec677fe700 1 -- 172.21.15.78:0/3540650833 learned_addr learned my addr 172.21.15.78:0/3540650833 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:03.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.742+0000 7fec667fc700 1 --2- 172.21.15.78:0/3540650833 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec680b8890 0x7fec6814c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:03.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.742+0000 7fec66ffd700 1 --2- 172.21.15.78:0/3540650833 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fec680b84c0 0x7fec6814c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:03.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.742+0000 7fec677fe700 1 -- 172.21.15.78:0/3540650833 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fec680b84c0 msgr2=0x7fec6814c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:03.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.742+0000 7fec677fe700 1 --2- 172.21.15.78:0/3540650833 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fec680b84c0 0x7fec6814c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:03.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.742+0000 7fec677fe700 1 -- 172.21.15.78:0/3540650833 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec680b8890 msgr2=0x7fec6814c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:03.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.742+0000 7fec677fe700 1 --2- 172.21.15.78:0/3540650833 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec680b8890 0x7fec6814c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:03.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.742+0000 7fec677fe700 1 -- 172.21.15.78:0/3540650833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec60009b30 con 0x7fec680d3510 2023-10-11T09:44:03.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.742+0000 7fec677fe700 1 --2- 172.21.15.78:0/3540650833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec680d3510 0x7fec681f57b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fec6000b370 tx=0x7fec60004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:03.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.742+0000 7fec4f7fe700 1 -- 172.21.15.78:0/3540650833 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec60004440 con 0x7fec680d3510 2023-10-11T09:44:03.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.742+0000 7fec4f7fe700 1 -- 172.21.15.78:0/3540650833 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec600045a0 con 0x7fec680d3510 2023-10-11T09:44:03.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.742+0000 7fec6d7df700 1 -- 172.21.15.78:0/3540650833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec680c7b30 con 0x7fec680d3510 2023-10-11T09:44:03.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.742+0000 7fec4f7fe700 1 -- 172.21.15.78:0/3540650833 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec60004950 con 0x7fec680d3510 2023-10-11T09:44:03.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.742+0000 7fec6d7df700 1 -- 172.21.15.78:0/3540650833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec681e2270 con 0x7fec680d3510 2023-10-11T09:44:03.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.746+0000 7fec6d7df700 1 -- 172.21.15.78:0/3540650833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec680c8050 con 0x7fec680d3510 2023-10-11T09:44:03.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.746+0000 7fec4f7fe700 1 -- 172.21.15.78:0/3540650833 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fec60012430 con 0x7fec680d3510 2023-10-11T09:44:03.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.746+0000 7fec4f7fe700 1 --2- 172.21.15.78:0/3540650833 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fec54065ed0 0x7fec54068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:03.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.750+0000 7fec4f7fe700 1 -- 172.21.15.78:0/3540650833 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fec60084ac0 con 0x7fec680d3510 2023-10-11T09:44:03.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.750+0000 7fec66ffd700 1 --2- 172.21.15.78:0/3540650833 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fec54065ed0 0x7fec54068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:03.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.750+0000 7fec66ffd700 1 --2- 172.21.15.78:0/3540650833 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fec54065ed0 0x7fec54068380 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fec68159e70 tx=0x7fec58009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:03.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.754+0000 7fec4f7fe700 1 -- 172.21.15.78:0/3540650833 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fec680c8050 con 0x7fec680d3510 2023-10-11T09:44:03.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.926+0000 7fec6d7df700 1 -- 172.21.15.78:0/3540650833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fec680c8050 con 0x7fec680d3510 2023-10-11T09:44:03.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.926+0000 7fec4f7fe700 1 -- 172.21.15.78:0/3540650833 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) v1 ==== 93+0+10 (secure 0 0 0) 0x7fec680c8050 con 0x7fec680d3510 2023-10-11T09:44:03.933 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:44:03.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.930+0000 7fec6d7df700 1 -- 172.21.15.78:0/3540650833 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fec54065ed0 msgr2=0x7fec54068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:03.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.930+0000 7fec6d7df700 1 --2- 172.21.15.78:0/3540650833 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fec54065ed0 0x7fec54068380 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fec68159e70 tx=0x7fec58009040 comp rx=0 tx=0).stop 2023-10-11T09:44:03.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.930+0000 7fec6d7df700 1 -- 172.21.15.78:0/3540650833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec680d3510 msgr2=0x7fec681f57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:03.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.930+0000 7fec6d7df700 1 --2- 172.21.15.78:0/3540650833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec680d3510 0x7fec681f57b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fec6000b370 tx=0x7fec60004af0 comp rx=0 tx=0).stop 2023-10-11T09:44:03.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.930+0000 7fec6d7df700 1 -- 172.21.15.78:0/3540650833 shutdown_connections 2023-10-11T09:44:03.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.930+0000 7fec6d7df700 1 --2- 172.21.15.78:0/3540650833 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fec54065ed0 0x7fec54068380 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:03.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.930+0000 7fec6d7df700 1 --2- 172.21.15.78:0/3540650833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec680d3510 0x7fec681f57b0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:03.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.930+0000 7fec6d7df700 1 --2- 172.21.15.78:0/3540650833 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec680b8890 0x7fec6814c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:03.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.930+0000 7fec6d7df700 1 --2- 172.21.15.78:0/3540650833 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fec680b84c0 0x7fec6814c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:03.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.930+0000 7fec6d7df700 1 -- 172.21.15.78:0/3540650833 >> 172.21.15.78:0/3540650833 conn(0x7fec6806cb50 msgr2=0x7fec680b6880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:03.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.934+0000 7fec6d7df700 1 -- 172.21.15.78:0/3540650833 shutdown_connections 2023-10-11T09:44:03.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:03.934+0000 7fec6d7df700 1 -- 172.21.15.78:0/3540650833 wait complete. 2023-10-11T09:44:03.959 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:44:04.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.138+0000 7f8e469a0700 1 Processor -- start 2023-10-11T09:44:04.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.138+0000 7f8e469a0700 1 -- start start 2023-10-11T09:44:04.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.142+0000 7f8e469a0700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8e400b1030 0x7f8e400cb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:04.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.142+0000 7f8e469a0700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f8e4006d080 0x7f8e4006ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:04.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.142+0000 7f8e469a0700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e40165570 con 0x7f8e40160cb0 2023-10-11T09:44:04.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.142+0000 7f8e469a0700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e40161080 con 0x7f8e4006d080 2023-10-11T09:44:04.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.142+0000 7f8e469a0700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e40164750 con 0x7f8e400b1030 2023-10-11T09:44:04.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.142+0000 7f8e44f3d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f8e4006d080 0x7f8e4006ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:04.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.142+0000 7f8e3ffff700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f8e40160cb0 0x7f8e400b0920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:37974/0 (socket says 172.21.15.78:37974) 2023-10-11T09:44:04.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.142+0000 7f8e3ffff700 1 -- 172.21.15.78:0/2923873167 learned_addr learned my addr 172.21.15.78:0/2923873167 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:44:04.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.142+0000 7f8e377fe700 1 --2- 172.21.15.78:0/2923873167 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8e400b1030 0x7f8e400cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:04.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.142+0000 7f8e3f7fe700 1 -- 172.21.15.78:0/2923873167 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1936443947 0 0) 0x7f8e40165570 con 0x7f8e40160cb0 2023-10-11T09:44:04.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.142+0000 7f8e44f3d700 1 -- 172.21.15.78:0/2923873167 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8e400b1030 msgr2=0x7f8e400cb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:44:04.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.142+0000 7f8e44f3d700 1 --2- 172.21.15.78:0/2923873167 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8e400b1030 0x7f8e400cb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:04.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.142+0000 7f8e44f3d700 1 -- 172.21.15.78:0/2923873167 >> v1:172.21.15.133:6789/0 conn(0x7f8e40160cb0 legacy=0x7f8e400b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:04.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.142+0000 7f8e44f3d700 1 -- 172.21.15.78:0/2923873167 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e40164960 con 0x7f8e4006d080 2023-10-11T09:44:04.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.142+0000 7f8e44f3d700 1 --2- 172.21.15.78:0/2923873167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e4006d080 0x7f8e4006ffb0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f8e38009850 tx=0x7f8e38009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9307f7f7943646f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:04.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.142+0000 7f8e3f7fe700 1 -- 172.21.15.78:0/2923873167 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e38011830 con 0x7f8e4006d080 2023-10-11T09:44:04.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.142+0000 7f8e3f7fe700 1 -- 172.21.15.78:0/2923873167 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e38011990 con 0x7f8e4006d080 2023-10-11T09:44:04.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.142+0000 7f8e3f7fe700 1 -- 172.21.15.78:0/2923873167 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e38011d40 con 0x7f8e4006d080 2023-10-11T09:44:04.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.146+0000 7f8e469a0700 1 -- 172.21.15.78:0/2923873167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e4006d080 msgr2=0x7f8e4006ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:04.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.146+0000 7f8e469a0700 1 --2- 172.21.15.78:0/2923873167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e4006d080 0x7f8e4006ffb0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f8e38009850 tx=0x7f8e38009b60 comp rx=0 tx=0).stop 2023-10-11T09:44:04.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.146+0000 7f8e469a0700 1 -- 172.21.15.78:0/2923873167 shutdown_connections 2023-10-11T09:44:04.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.146+0000 7f8e469a0700 1 --2- 172.21.15.78:0/2923873167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e4006d080 0x7f8e4006ffb0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:04.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.146+0000 7f8e469a0700 1 --2- 172.21.15.78:0/2923873167 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8e400b1030 0x7f8e400cb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:04.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.146+0000 7f8e469a0700 1 -- 172.21.15.78:0/2923873167 >> 172.21.15.78:0/2923873167 conn(0x7f8e400cecc0 msgr2=0x7f8e400cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:04.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.146+0000 7f8e469a0700 1 -- 172.21.15.78:0/2923873167 shutdown_connections 2023-10-11T09:44:04.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.146+0000 7f8e469a0700 1 -- 172.21.15.78:0/2923873167 wait complete. 2023-10-11T09:44:04.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.150+0000 7f8e469a0700 1 Processor -- start 2023-10-11T09:44:04.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.150+0000 7f8e469a0700 1 -- start start 2023-10-11T09:44:04.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.150+0000 7f8e469a0700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e4006d080 0x7f8e401639e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:04.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.150+0000 7f8e469a0700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8e400b1030 0x7f8e401f7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:04.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.150+0000 7f8e469a0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e40160cb0 0x7f8e401f3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:04.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.150+0000 7f8e469a0700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e401f76d0 con 0x7f8e40160cb0 2023-10-11T09:44:04.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.150+0000 7f8e469a0700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e400b1d30 con 0x7f8e4006d080 2023-10-11T09:44:04.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.150+0000 7f8e469a0700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e400b2010 con 0x7f8e400b1030 2023-10-11T09:44:04.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.150+0000 7f8e44f3d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e40160cb0 0x7f8e401f3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:04.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.150+0000 7f8e44f3d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e40160cb0 0x7f8e401f3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35602/0 (socket says 172.21.15.78:35602) 2023-10-11T09:44:04.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.150+0000 7f8e44f3d700 1 -- 172.21.15.78:0/1830502129 learned_addr learned my addr 172.21.15.78:0/1830502129 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:04.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.150+0000 7f8e3ffff700 1 --2- 172.21.15.78:0/1830502129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e4006d080 0x7f8e401639e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:04.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.150+0000 7f8e377fe700 1 --2- 172.21.15.78:0/1830502129 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8e400b1030 0x7f8e401f7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:04.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.150+0000 7f8e44f3d700 1 -- 172.21.15.78:0/1830502129 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8e400b1030 msgr2=0x7f8e401f7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:04.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.150+0000 7f8e44f3d700 1 --2- 172.21.15.78:0/1830502129 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8e400b1030 0x7f8e401f7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:04.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.150+0000 7f8e44f3d700 1 -- 172.21.15.78:0/1830502129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e4006d080 msgr2=0x7f8e401639e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:04.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.150+0000 7f8e44f3d700 1 --2- 172.21.15.78:0/1830502129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e4006d080 0x7f8e401639e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:04.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.150+0000 7f8e44f3d700 1 -- 172.21.15.78:0/1830502129 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e380094b0 con 0x7f8e40160cb0 2023-10-11T09:44:04.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.154+0000 7f8e44f3d700 1 --2- 172.21.15.78:0/1830502129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e40160cb0 0x7f8e401f3bb0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f8e38009ba0 tx=0x7f8e38004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:04.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.154+0000 7f8e3d7fa700 1 -- 172.21.15.78:0/1830502129 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e38019980 con 0x7f8e40160cb0 2023-10-11T09:44:04.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.154+0000 7f8e3d7fa700 1 -- 172.21.15.78:0/1830502129 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e3801e070 con 0x7f8e40160cb0 2023-10-11T09:44:04.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.154+0000 7f8e3d7fa700 1 -- 172.21.15.78:0/1830502129 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e38019d40 con 0x7f8e40160cb0 2023-10-11T09:44:04.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.154+0000 7f8e469a0700 1 -- 172.21.15.78:0/1830502129 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e400b22f0 con 0x7f8e40160cb0 2023-10-11T09:44:04.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.154+0000 7f8e469a0700 1 -- 172.21.15.78:0/1830502129 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e400b28b0 con 0x7f8e40160cb0 2023-10-11T09:44:04.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.154+0000 7f8e469a0700 1 -- 172.21.15.78:0/1830502129 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e400b2aa0 con 0x7f8e40160cb0 2023-10-11T09:44:04.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.158+0000 7f8e3d7fa700 1 -- 172.21.15.78:0/1830502129 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f8e3801a420 con 0x7f8e40160cb0 2023-10-11T09:44:04.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.158+0000 7f8e3d7fa700 1 --2- 172.21.15.78:0/1830502129 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8e20065f50 0x7f8e20068400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:04.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.158+0000 7f8e3d7fa700 1 -- 172.21.15.78:0/1830502129 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8e38084480 con 0x7f8e40160cb0 2023-10-11T09:44:04.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.158+0000 7f8e3ffff700 1 --2- 172.21.15.78:0/1830502129 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8e20065f50 0x7f8e20068400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:04.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.158+0000 7f8e3ffff700 1 --2- 172.21.15.78:0/1830502129 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8e20065f50 0x7f8e20068400 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8e300012e0 tx=0x7f8e30001aa0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:04.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.166+0000 7f8e3d7fa700 1 -- 172.21.15.78:0/1830502129 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f8e38051e90 con 0x7f8e40160cb0 2023-10-11T09:44:04.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.338+0000 7f8e469a0700 1 -- 172.21.15.78:0/1830502129 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8e40065db0 con 0x7f8e40160cb0 2023-10-11T09:44:04.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.338+0000 7f8e3d7fa700 1 -- 172.21.15.78:0/1830502129 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v57) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8e38006ce0 con 0x7f8e40160cb0 2023-10-11T09:44:04.343 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:44:04.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.338+0000 7f8e469a0700 1 -- 172.21.15.78:0/1830502129 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8e20065f50 msgr2=0x7f8e20068400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:04.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.338+0000 7f8e469a0700 1 --2- 172.21.15.78:0/1830502129 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8e20065f50 0x7f8e20068400 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8e300012e0 tx=0x7f8e30001aa0 comp rx=0 tx=0).stop 2023-10-11T09:44:04.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.338+0000 7f8e469a0700 1 -- 172.21.15.78:0/1830502129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e40160cb0 msgr2=0x7f8e401f3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:04.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.338+0000 7f8e469a0700 1 --2- 172.21.15.78:0/1830502129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e40160cb0 0x7f8e401f3bb0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f8e38009ba0 tx=0x7f8e38004770 comp rx=0 tx=0).stop 2023-10-11T09:44:04.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.342+0000 7f8e469a0700 1 -- 172.21.15.78:0/1830502129 shutdown_connections 2023-10-11T09:44:04.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.342+0000 7f8e469a0700 1 --2- 172.21.15.78:0/1830502129 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8e20065f50 0x7f8e20068400 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:04.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.342+0000 7f8e469a0700 1 --2- 172.21.15.78:0/1830502129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e40160cb0 0x7f8e401f3bb0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:04.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.342+0000 7f8e469a0700 1 --2- 172.21.15.78:0/1830502129 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8e400b1030 0x7f8e401f7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:04.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.342+0000 7f8e469a0700 1 --2- 172.21.15.78:0/1830502129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e4006d080 0x7f8e401639e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:04.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.342+0000 7f8e469a0700 1 -- 172.21.15.78:0/1830502129 >> 172.21.15.78:0/1830502129 conn(0x7f8e400cecc0 msgr2=0x7f8e400ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:04.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.342+0000 7f8e469a0700 1 -- 172.21.15.78:0/1830502129 shutdown_connections 2023-10-11T09:44:04.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.342+0000 7f8e469a0700 1 -- 172.21.15.78:0/1830502129 wait complete. 2023-10-11T09:44:04.368 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-11T09:44:04.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.530+0000 7f8fc22d8700 1 Processor -- start 2023-10-11T09:44:04.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.530+0000 7f8fc22d8700 1 -- start start 2023-10-11T09:44:04.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.530+0000 7f8fc22d8700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8fbc15c880 0x7f8fbc06be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:04.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.530+0000 7f8fc22d8700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8fbc0cbd30 0x7f8fbc0c9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:04.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.530+0000 7f8fc22d8700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fbc0c93a0 con 0x7f8fbc0cd210 2023-10-11T09:44:04.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.530+0000 7f8fc22d8700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fbc161420 con 0x7f8fbc0cbd30 2023-10-11T09:44:04.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.530+0000 7f8fc22d8700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc0cca10 con 0x7f8fbc15c880 2023-10-11T09:44:04.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.534+0000 7f8fc0875700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8fbc0cbd30 0x7f8fbc0c9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:04.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.534+0000 7f8fb3fff700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f8fbc0cd210 0x7f8fbc0cb620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:37984/0 (socket says 172.21.15.78:37984) 2023-10-11T09:44:04.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.534+0000 7f8fb3fff700 1 -- 172.21.15.78:0/2375735692 learned_addr learned my addr 172.21.15.78:0/2375735692 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:44:04.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.534+0000 7f8fc0875700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8fbc0cbd30 0x7f8fbc0c9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:48920/0 (socket says 172.21.15.78:48920) 2023-10-11T09:44:04.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.534+0000 7f8fbbfff700 1 --2- 172.21.15.78:0/2375735692 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8fbc15c880 0x7f8fbc06be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:04.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.534+0000 7f8fbb7fe700 1 -- 172.21.15.78:0/2375735692 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1061795515 0 0) 0x7f8fbc0c93a0 con 0x7f8fbc0cd210 2023-10-11T09:44:04.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.534+0000 7f8fbb7fe700 1 -- 172.21.15.78:0/2375735692 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fa80035d0 con 0x7f8fbc0cd210 2023-10-11T09:44:04.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.534+0000 7f8fc0875700 1 -- 172.21.15.78:0/2375735692 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8fbc15c880 msgr2=0x7f8fbc06be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:44:04.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.534+0000 7f8fc0875700 1 --2- 172.21.15.78:0/2375735692 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8fbc15c880 0x7f8fbc06be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:04.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.534+0000 7f8fc0875700 1 -- 172.21.15.78:0/2375735692 >> v1:172.21.15.133:6789/0 conn(0x7f8fbc0cd210 legacy=0x7f8fbc0cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:04.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.534+0000 7f8fc0875700 1 -- 172.21.15.78:0/2375735692 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fbc0cccb0 con 0x7f8fbc0cbd30 2023-10-11T09:44:04.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.534+0000 7f8fbbfff700 1 --2- 172.21.15.78:0/2375735692 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8fbc15c880 0x7f8fbc06be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:44:04.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.534+0000 7f8fc0875700 1 --2- 172.21.15.78:0/2375735692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fbc0cbd30 0x7f8fbc0c9b40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009d90 tx=0x7f8fb400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a24ef693530eff0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:04.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.534+0000 7f8fbb7fe700 1 -- 172.21.15.78:0/2375735692 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb400e070 con 0x7f8fbc0cbd30 2023-10-11T09:44:04.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.534+0000 7f8fbb7fe700 1 -- 172.21.15.78:0/2375735692 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb400bb20 con 0x7f8fbc0cbd30 2023-10-11T09:44:04.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.534+0000 7f8fbb7fe700 1 -- 172.21.15.78:0/2375735692 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4012430 con 0x7f8fbc0cbd30 2023-10-11T09:44:04.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.534+0000 7f8fc22d8700 1 -- 172.21.15.78:0/2375735692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fbc0cbd30 msgr2=0x7f8fbc0c9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:04.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.534+0000 7f8fc22d8700 1 --2- 172.21.15.78:0/2375735692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fbc0cbd30 0x7f8fbc0c9b40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009d90 tx=0x7f8fb400b330 comp rx=0 tx=0).stop 2023-10-11T09:44:04.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.538+0000 7f8fc22d8700 1 -- 172.21.15.78:0/2375735692 shutdown_connections 2023-10-11T09:44:04.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.538+0000 7f8fc22d8700 1 --2- 172.21.15.78:0/2375735692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fbc0cbd30 0x7f8fbc0c9b40 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:04.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.538+0000 7f8fc22d8700 1 --2- 172.21.15.78:0/2375735692 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8fbc15c880 0x7f8fbc06be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:04.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.538+0000 7f8fc22d8700 1 -- 172.21.15.78:0/2375735692 >> 172.21.15.78:0/2375735692 conn(0x7f8fbc06aca0 msgr2=0x7f8fbc0b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:04.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.538+0000 7f8fc22d8700 1 -- 172.21.15.78:0/2375735692 shutdown_connections 2023-10-11T09:44:04.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.538+0000 7f8fc22d8700 1 -- 172.21.15.78:0/2375735692 wait complete. 2023-10-11T09:44:04.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.538+0000 7f8fc22d8700 1 Processor -- start 2023-10-11T09:44:04.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.538+0000 7f8fc22d8700 1 -- start start 2023-10-11T09:44:04.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.538+0000 7f8fc22d8700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8fbc0cbd30 0x7f8fbc14d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:04.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.538+0000 7f8fc22d8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8fbc0cd210 0x7f8fbc155b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:04.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.538+0000 7f8fc22d8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fbc15c880 0x7f8fbc0afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:04.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.538+0000 7f8fc22d8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc150340 con 0x7f8fbc0cd210 2023-10-11T09:44:04.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.538+0000 7f8fc22d8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc1f5440 con 0x7f8fbc15c880 2023-10-11T09:44:04.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.538+0000 7f8fc22d8700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc14d670 con 0x7f8fbc0cbd30 2023-10-11T09:44:04.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.538+0000 7f8fb3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8fbc0cd210 0x7f8fbc155b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:04.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.538+0000 7f8fb3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8fbc0cd210 0x7f8fbc155b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35606/0 (socket says 172.21.15.78:35606) 2023-10-11T09:44:04.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.538+0000 7f8fb3fff700 1 -- 172.21.15.78:0/913594775 learned_addr learned my addr 172.21.15.78:0/913594775 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:04.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.542+0000 7f8fbbfff700 1 --2- 172.21.15.78:0/913594775 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8fbc0cbd30 0x7f8fbc14d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:04.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.542+0000 7f8fc0875700 1 --2- 172.21.15.78:0/913594775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fbc15c880 0x7f8fbc0afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:04.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.542+0000 7f8fb3fff700 1 -- 172.21.15.78:0/913594775 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8fbc0cbd30 msgr2=0x7f8fbc14d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:04.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.542+0000 7f8fb3fff700 1 --2- 172.21.15.78:0/913594775 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8fbc0cbd30 0x7f8fbc14d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:04.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.542+0000 7f8fb3fff700 1 -- 172.21.15.78:0/913594775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fbc15c880 msgr2=0x7f8fbc0afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:04.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.542+0000 7f8fb3fff700 1 --2- 172.21.15.78:0/913594775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fbc15c880 0x7f8fbc0afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:04.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.542+0000 7f8fb3fff700 1 -- 172.21.15.78:0/913594775 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fac0094b0 con 0x7f8fbc0cd210 2023-10-11T09:44:04.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.542+0000 7f8fb3fff700 1 --2- 172.21.15.78:0/913594775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8fbc0cd210 0x7f8fbc155b80 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f8fa400c730 tx=0x7f8fa400ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:04.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.542+0000 7f8fb97fa700 1 -- 172.21.15.78:0/913594775 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa4005d90 con 0x7f8fbc0cd210 2023-10-11T09:44:04.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.542+0000 7f8fb97fa700 1 -- 172.21.15.78:0/913594775 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa40070d0 con 0x7f8fbc0cd210 2023-10-11T09:44:04.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.542+0000 7f8fb97fa700 1 -- 172.21.15.78:0/913594775 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa4007460 con 0x7f8fbc0cd210 2023-10-11T09:44:04.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.542+0000 7f8fc22d8700 1 -- 172.21.15.78:0/913594775 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fb4009a70 con 0x7f8fbc0cd210 2023-10-11T09:44:04.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.542+0000 7f8fc22d8700 1 -- 172.21.15.78:0/913594775 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fbc15b700 con 0x7f8fbc0cd210 2023-10-11T09:44:04.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.542+0000 7f8fc22d8700 1 -- 172.21.15.78:0/913594775 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fbc1f55f0 con 0x7f8fbc0cd210 2023-10-11T09:44:04.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.546+0000 7f8fb97fa700 1 -- 172.21.15.78:0/913594775 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f8fa40075c0 con 0x7f8fbc0cd210 2023-10-11T09:44:04.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.546+0000 7f8fb97fa700 1 --2- 172.21.15.78:0/913594775 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8f9c065ed0 0x7f8f9c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:04.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.546+0000 7f8fb97fa700 1 -- 172.21.15.78:0/913594775 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8fa40829c0 con 0x7f8fbc0cd210 2023-10-11T09:44:04.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.546+0000 7f8fbbfff700 1 --2- 172.21.15.78:0/913594775 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8f9c065ed0 0x7f8f9c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:04.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.550+0000 7f8fbbfff700 1 --2- 172.21.15.78:0/913594775 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8f9c065ed0 0x7f8f9c068380 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8fac009ce0 tx=0x7f8fac009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:04.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.550+0000 7f8fb97fa700 1 -- 172.21.15.78:0/913594775 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f8fbc1f55f0 con 0x7f8fbc0cd210 2023-10-11T09:44:04.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.730+0000 7f8fc22d8700 1 -- 172.21.15.78:0/913594775 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8fbc1f55f0 con 0x7f8fbc0cd210 2023-10-11T09:44:04.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.730+0000 7f8fb97fa700 1 -- 172.21.15.78:0/913594775 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v57) v1 ==== 84+0+126 (secure 0 0 0) 0x7f8fbc1f55f0 con 0x7f8fbc0cd210 2023-10-11T09:44:04.736 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:44:04.737 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2023-10-11T09:44:04.737 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:44:04.737 INFO:teuthology.orchestra.run.smithi078.stdout: "poolnum": 1, 2023-10-11T09:44:04.737 INFO:teuthology.orchestra.run.smithi078.stdout: "poolname": ".mgr" 2023-10-11T09:44:04.737 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:44:04.738 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:44:04.738 INFO:teuthology.orchestra.run.smithi078.stdout: "poolnum": 2, 2023-10-11T09:44:04.738 INFO:teuthology.orchestra.run.smithi078.stdout: "poolname": "rbd" 2023-10-11T09:44:04.738 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:44:04.738 INFO:teuthology.orchestra.run.smithi078.stdout:] 2023-10-11T09:44:04.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.734+0000 7f8fc22d8700 1 -- 172.21.15.78:0/913594775 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8f9c065ed0 msgr2=0x7f8f9c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:04.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.734+0000 7f8fc22d8700 1 --2- 172.21.15.78:0/913594775 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8f9c065ed0 0x7f8f9c068380 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8fac009ce0 tx=0x7f8fac009240 comp rx=0 tx=0).stop 2023-10-11T09:44:04.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.734+0000 7f8fc22d8700 1 -- 172.21.15.78:0/913594775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8fbc0cd210 msgr2=0x7f8fbc155b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:04.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.734+0000 7f8fc22d8700 1 --2- 172.21.15.78:0/913594775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8fbc0cd210 0x7f8fbc155b80 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f8fa400c730 tx=0x7f8fa400ca40 comp rx=0 tx=0).stop 2023-10-11T09:44:04.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.734+0000 7f8fc22d8700 1 -- 172.21.15.78:0/913594775 shutdown_connections 2023-10-11T09:44:04.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.734+0000 7f8fc22d8700 1 --2- 172.21.15.78:0/913594775 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8f9c065ed0 0x7f8f9c068380 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:04.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.734+0000 7f8fc22d8700 1 --2- 172.21.15.78:0/913594775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fbc15c880 0x7f8fbc0afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:04.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.734+0000 7f8fc22d8700 1 --2- 172.21.15.78:0/913594775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8fbc0cd210 0x7f8fbc155b80 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:04.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.734+0000 7f8fc22d8700 1 --2- 172.21.15.78:0/913594775 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8fbc0cbd30 0x7f8fbc14d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:04.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.734+0000 7f8fc22d8700 1 -- 172.21.15.78:0/913594775 >> 172.21.15.78:0/913594775 conn(0x7f8fbc06aca0 msgr2=0x7f8fbc06cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:04.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.734+0000 7f8fc22d8700 1 -- 172.21.15.78:0/913594775 shutdown_connections 2023-10-11T09:44:04.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.734+0000 7f8fc22d8700 1 -- 172.21.15.78:0/913594775 wait complete. 2023-10-11T09:44:04.760 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-11T09:44:04.761 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-11T09:44:04.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.930+0000 7f147ffff700 1 Processor -- start 2023-10-11T09:44:04.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.930+0000 7f147ffff700 1 -- start start 2023-10-11T09:44:04.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.934+0000 7f147ffff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f14801655d0 0x7f148006cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:04.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.934+0000 7f147ffff700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1480165140 con 0x7f14800cdbd0 2023-10-11T09:44:04.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.934+0000 7f147ffff700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14800cd470 con 0x7f14800b0eb0 2023-10-11T09:44:04.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.934+0000 7f147ffff700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14801648e0 con 0x7f14801655d0 2023-10-11T09:44:04.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.934+0000 7f147effd700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f14800cdbd0 0x7f14800cdfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33758/0 (socket says 172.21.15.78:33758) 2023-10-11T09:44:04.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.934+0000 7f147effd700 1 -- 172.21.15.78:0/1406888758 learned_addr learned my addr 172.21.15.78:0/1406888758 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:44:04.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.934+0000 7f1477fff700 1 --2- 172.21.15.78:0/1406888758 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f14801655d0 0x7f148006cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:04.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.934+0000 7f147e7fc700 1 -- 172.21.15.78:0/1406888758 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 449173084 0 0) 0x7f1480165140 con 0x7f14800cdbd0 2023-10-11T09:44:04.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.934+0000 7f147e7fc700 1 -- 172.21.15.78:0/1406888758 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f146c0035d0 con 0x7f14800cdbd0 2023-10-11T09:44:04.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.934+0000 7f1477fff700 1 -- 172.21.15.78:0/1406888758 >> v1:172.21.15.133:6789/0 conn(0x7f14800b0eb0 legacy=0x7f14800ca2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:04.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.934+0000 7f1477fff700 1 -- 172.21.15.78:0/1406888758 >> v1:172.21.15.78:6789/0 conn(0x7f14800cdbd0 legacy=0x7f14800cdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:04.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.934+0000 7f1477fff700 1 -- 172.21.15.78:0/1406888758 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1480164af0 con 0x7f14801655d0 2023-10-11T09:44:04.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.934+0000 7f1477fff700 1 --2- 172.21.15.78:0/1406888758 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f14801655d0 0x7f148006cfa0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1468009880 tx=0x7f1468009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ee591de68026dc1 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:04.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.938+0000 7f147e7fc700 1 -- 172.21.15.78:0/1406888758 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1468011830 con 0x7f14801655d0 2023-10-11T09:44:04.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.938+0000 7f147e7fc700 1 -- 172.21.15.78:0/1406888758 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1468011990 con 0x7f14801655d0 2023-10-11T09:44:04.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.938+0000 7f147e7fc700 1 -- 172.21.15.78:0/1406888758 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1468011d20 con 0x7f14801655d0 2023-10-11T09:44:04.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.938+0000 7f147ffff700 1 -- 172.21.15.78:0/1406888758 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f14801655d0 msgr2=0x7f148006cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:04.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.938+0000 7f147ffff700 1 --2- 172.21.15.78:0/1406888758 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f14801655d0 0x7f148006cfa0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1468009880 tx=0x7f1468009b90 comp rx=0 tx=0).stop 2023-10-11T09:44:04.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.938+0000 7f147ffff700 1 -- 172.21.15.78:0/1406888758 shutdown_connections 2023-10-11T09:44:04.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.938+0000 7f147ffff700 1 --2- 172.21.15.78:0/1406888758 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f14801655d0 0x7f148006cfa0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:04.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.938+0000 7f147ffff700 1 -- 172.21.15.78:0/1406888758 >> 172.21.15.78:0/1406888758 conn(0x7f148006acf0 msgr2=0x7f14800b0710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:04.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.938+0000 7f147ffff700 1 -- 172.21.15.78:0/1406888758 shutdown_connections 2023-10-11T09:44:04.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.938+0000 7f147ffff700 1 -- 172.21.15.78:0/1406888758 wait complete. 2023-10-11T09:44:04.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.942+0000 7f147ffff700 1 Processor -- start 2023-10-11T09:44:04.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.942+0000 7f147ffff700 1 -- start start 2023-10-11T09:44:04.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.942+0000 7f147ffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14800b0eb0 0x7f148015a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:04.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.942+0000 7f147ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14800cdbd0 0x7f1480161e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:04.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.942+0000 7f147ffff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f14801655d0 0x7f14800b3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:04.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.942+0000 7f147ffff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14800c3da0 con 0x7f14800cdbd0 2023-10-11T09:44:04.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.942+0000 7f147ffff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14800c4080 con 0x7f14800b0eb0 2023-10-11T09:44:04.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.942+0000 7f147ffff700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14800c4360 con 0x7f14801655d0 2023-10-11T09:44:04.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.942+0000 7f1477fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14800cdbd0 0x7f1480161e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:04.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.942+0000 7f1477fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14800cdbd0 0x7f1480161e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34872/0 (socket says 172.21.15.78:34872) 2023-10-11T09:44:04.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.942+0000 7f1477fff700 1 -- 172.21.15.78:0/1128620359 learned_addr learned my addr 172.21.15.78:0/1128620359 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:04.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.942+0000 7f147f7fe700 1 --2- 172.21.15.78:0/1128620359 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f14801655d0 0x7f14800b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:04.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.942+0000 7f147effd700 1 --2- 172.21.15.78:0/1128620359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14800b0eb0 0x7f148015a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:04.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.942+0000 7f1477fff700 1 -- 172.21.15.78:0/1128620359 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f14801655d0 msgr2=0x7f14800b3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:04.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.942+0000 7f1477fff700 1 --2- 172.21.15.78:0/1128620359 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f14801655d0 0x7f14800b3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:04.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.942+0000 7f1477fff700 1 -- 172.21.15.78:0/1128620359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14800b0eb0 msgr2=0x7f148015a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:04.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.942+0000 7f1477fff700 1 --2- 172.21.15.78:0/1128620359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14800b0eb0 0x7f148015a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:04.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.942+0000 7f1477fff700 1 -- 172.21.15.78:0/1128620359 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14680094b0 con 0x7f14800cdbd0 2023-10-11T09:44:04.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.942+0000 7f1477fff700 1 --2- 172.21.15.78:0/1128620359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14800cdbd0 0x7f1480161e20 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f1468013040 tx=0x7f1468004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:04.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.942+0000 7f14777fe700 1 -- 172.21.15.78:0/1128620359 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1468004610 con 0x7f14800cdbd0 2023-10-11T09:44:04.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.942+0000 7f14777fe700 1 -- 172.21.15.78:0/1128620359 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14680036c0 con 0x7f14800cdbd0 2023-10-11T09:44:04.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.942+0000 7f147ffff700 1 -- 172.21.15.78:0/1128620359 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14800c4640 con 0x7f14800cdbd0 2023-10-11T09:44:04.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.942+0000 7f14777fe700 1 -- 172.21.15.78:0/1128620359 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1468003a70 con 0x7f14800cdbd0 2023-10-11T09:44:04.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.942+0000 7f147ffff700 1 -- 172.21.15.78:0/1128620359 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f148015a670 con 0x7f14800cdbd0 2023-10-11T09:44:04.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.946+0000 7f147ffff700 1 -- 172.21.15.78:0/1128620359 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14800c4b60 con 0x7f14800cdbd0 2023-10-11T09:44:04.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.946+0000 7f14777fe700 1 -- 172.21.15.78:0/1128620359 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f1468003bd0 con 0x7f14800cdbd0 2023-10-11T09:44:04.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.950+0000 7f14777fe700 1 --2- 172.21.15.78:0/1128620359 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1460065ed0 0x7f1460068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:04.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.950+0000 7f14777fe700 1 -- 172.21.15.78:0/1128620359 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1468084950 con 0x7f14800cdbd0 2023-10-11T09:44:04.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.950+0000 7f147effd700 1 --2- 172.21.15.78:0/1128620359 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1460065ed0 0x7f1460068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:04.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.950+0000 7f147effd700 1 --2- 172.21.15.78:0/1128620359 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1460065ed0 0x7f1460068380 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f14700011e0 tx=0x7f1470009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:04.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:04.954+0000 7f14777fe700 1 -- 172.21.15.78:0/1128620359 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f14800c4b60 con 0x7f14800cdbd0 2023-10-11T09:44:05.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.130+0000 7f147ffff700 1 -- 172.21.15.78:0/1128620359 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f14800c4b60 con 0x7f14800cdbd0 2023-10-11T09:44:05.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.386+0000 7f14777fe700 1 -- 172.21.15.78:0/1128620359 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v58) v1 ==== 138+0+0 (secure 0 0 0) 0x7f14800c4b60 con 0x7f14800cdbd0 2023-10-11T09:44:05.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.390+0000 7f147ffff700 1 -- 172.21.15.78:0/1128620359 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1460065ed0 msgr2=0x7f1460068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:05.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.390+0000 7f147ffff700 1 --2- 172.21.15.78:0/1128620359 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1460065ed0 0x7f1460068380 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f14700011e0 tx=0x7f1470009040 comp rx=0 tx=0).stop 2023-10-11T09:44:05.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.390+0000 7f147ffff700 1 -- 172.21.15.78:0/1128620359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14800cdbd0 msgr2=0x7f1480161e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:05.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.390+0000 7f147ffff700 1 --2- 172.21.15.78:0/1128620359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14800cdbd0 0x7f1480161e20 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f1468013040 tx=0x7f1468004770 comp rx=0 tx=0).stop 2023-10-11T09:44:05.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.390+0000 7f147ffff700 1 -- 172.21.15.78:0/1128620359 shutdown_connections 2023-10-11T09:44:05.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.390+0000 7f147ffff700 1 --2- 172.21.15.78:0/1128620359 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1460065ed0 0x7f1460068380 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:05.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.390+0000 7f147ffff700 1 --2- 172.21.15.78:0/1128620359 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f14801655d0 0x7f14800b3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:05.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.390+0000 7f147ffff700 1 --2- 172.21.15.78:0/1128620359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14800cdbd0 0x7f1480161e20 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:05.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.390+0000 7f147ffff700 1 --2- 172.21.15.78:0/1128620359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14800b0eb0 0x7f148015a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:05.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.390+0000 7f147ffff700 1 -- 172.21.15.78:0/1128620359 >> 172.21.15.78:0/1128620359 conn(0x7f148006acf0 msgr2=0x7f14800cdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:05.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.390+0000 7f147ffff700 1 -- 172.21.15.78:0/1128620359 shutdown_connections 2023-10-11T09:44:05.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.390+0000 7f147ffff700 1 -- 172.21.15.78:0/1128620359 wait complete. 2023-10-11T09:44:05.401 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' created 2023-10-11T09:44:05.423 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-11T09:44:05.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.602+0000 7f01da7ef700 1 Processor -- start 2023-10-11T09:44:05.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.602+0000 7f01da7ef700 1 -- start start 2023-10-11T09:44:05.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.602+0000 7f01da7ef700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f01d4146f50 0x7f01d40afbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:05.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.602+0000 7f01da7ef700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01d4164930 con 0x7f01d40ce150 2023-10-11T09:44:05.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.602+0000 7f01da7ef700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01d4164c10 con 0x7f01d41477b0 2023-10-11T09:44:05.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.602+0000 7f01da7ef700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01d4147320 con 0x7f01d4146f50 2023-10-11T09:44:05.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.606+0000 7f01d97ed700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f01d4146f50 0x7f01d40afbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:05.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.606+0000 7f01d9fee700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f01d40ce150 0x7f01d4071190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33760/0 (socket says 172.21.15.78:33760) 2023-10-11T09:44:05.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.606+0000 7f01d9fee700 1 -- 172.21.15.78:0/3448277975 learned_addr learned my addr 172.21.15.78:0/3448277975 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:44:05.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.606+0000 7f01d97ed700 1 -- 172.21.15.78:0/3448277975 >> v1:172.21.15.133:6789/0 conn(0x7f01d41477b0 legacy=0x7f01d40cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:05.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.606+0000 7f01d97ed700 1 -- 172.21.15.78:0/3448277975 >> v1:172.21.15.78:6789/0 conn(0x7f01d40ce150 legacy=0x7f01d4071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:05.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.606+0000 7f01cbfff700 1 -- 172.21.15.78:0/3448277975 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2353622577 0 0) 0x7f01d4164930 con 0x7f01d40ce150 2023-10-11T09:44:05.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.606+0000 7f01d97ed700 1 -- 172.21.15.78:0/3448277975 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d4164ef0 con 0x7f01d4146f50 2023-10-11T09:44:05.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.606+0000 7f01d97ed700 1 --2- 172.21.15.78:0/3448277975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01d4146f50 0x7f01d40afbd0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f01d0009850 tx=0x7f01d0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76b68925b91a91f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:05.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.606+0000 7f01cbfff700 1 -- 172.21.15.78:0/3448277975 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d0011830 con 0x7f01d4146f50 2023-10-11T09:44:05.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.606+0000 7f01cbfff700 1 -- 172.21.15.78:0/3448277975 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d0011990 con 0x7f01d4146f50 2023-10-11T09:44:05.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.606+0000 7f01cbfff700 1 -- 172.21.15.78:0/3448277975 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d0011d20 con 0x7f01d4146f50 2023-10-11T09:44:05.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.606+0000 7f01da7ef700 1 -- 172.21.15.78:0/3448277975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01d4146f50 msgr2=0x7f01d40afbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:05.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.606+0000 7f01da7ef700 1 --2- 172.21.15.78:0/3448277975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01d4146f50 0x7f01d40afbd0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f01d0009850 tx=0x7f01d0009b60 comp rx=0 tx=0).stop 2023-10-11T09:44:05.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.606+0000 7f01da7ef700 1 -- 172.21.15.78:0/3448277975 shutdown_connections 2023-10-11T09:44:05.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.606+0000 7f01da7ef700 1 --2- 172.21.15.78:0/3448277975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01d4146f50 0x7f01d40afbd0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:05.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.606+0000 7f01da7ef700 1 -- 172.21.15.78:0/3448277975 >> 172.21.15.78:0/3448277975 conn(0x7f01d40b0780 msgr2=0x7f01d40b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:05.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.606+0000 7f01da7ef700 1 -- 172.21.15.78:0/3448277975 shutdown_connections 2023-10-11T09:44:05.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.606+0000 7f01da7ef700 1 -- 172.21.15.78:0/3448277975 wait complete. 2023-10-11T09:44:05.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.610+0000 7f01da7ef700 1 Processor -- start 2023-10-11T09:44:05.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.610+0000 7f01da7ef700 1 -- start start 2023-10-11T09:44:05.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.610+0000 7f01da7ef700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f01d40ce150 0x7f01d415dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:05.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.610+0000 7f01da7ef700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01d4146f50 0x7f01d41f65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:05.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.610+0000 7f01da7ef700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01d41477b0 0x7f01d4145d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:05.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.610+0000 7f01da7ef700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d40b3a30 con 0x7f01d41477b0 2023-10-11T09:44:05.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.610+0000 7f01da7ef700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d40b4890 con 0x7f01d4146f50 2023-10-11T09:44:05.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.610+0000 7f01da7ef700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d4146260 con 0x7f01d40ce150 2023-10-11T09:44:05.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.614+0000 7f01d9fee700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01d41477b0 0x7f01d4145d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:05.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.614+0000 7f01d9fee700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01d41477b0 0x7f01d4145d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34900/0 (socket says 172.21.15.78:34900) 2023-10-11T09:44:05.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.614+0000 7f01d9fee700 1 -- 172.21.15.78:0/13428627 learned_addr learned my addr 172.21.15.78:0/13428627 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:05.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.614+0000 7f01d97ed700 1 --2- 172.21.15.78:0/13428627 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f01d40ce150 0x7f01d415dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:05.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.614+0000 7f01d8fec700 1 --2- 172.21.15.78:0/13428627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01d4146f50 0x7f01d41f65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:05.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.614+0000 7f01d9fee700 1 -- 172.21.15.78:0/13428627 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f01d40ce150 msgr2=0x7f01d415dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:05.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.614+0000 7f01d9fee700 1 --2- 172.21.15.78:0/13428627 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f01d40ce150 0x7f01d415dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:05.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.614+0000 7f01d9fee700 1 -- 172.21.15.78:0/13428627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01d4146f50 msgr2=0x7f01d41f65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:05.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.614+0000 7f01d9fee700 1 --2- 172.21.15.78:0/13428627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01d4146f50 0x7f01d41f65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:05.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.614+0000 7f01d9fee700 1 -- 172.21.15.78:0/13428627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d00094b0 con 0x7f01d41477b0 2023-10-11T09:44:05.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.614+0000 7f01d9fee700 1 --2- 172.21.15.78:0/13428627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01d41477b0 0x7f01d4145d20 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f01cc002230 tx=0x7f01cc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:05.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.614+0000 7f01c9ffb700 1 -- 172.21.15.78:0/13428627 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01cc00ac20 con 0x7f01d41477b0 2023-10-11T09:44:05.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.614+0000 7f01c9ffb700 1 -- 172.21.15.78:0/13428627 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01cc00ada0 con 0x7f01d41477b0 2023-10-11T09:44:05.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.614+0000 7f01da7ef700 1 -- 172.21.15.78:0/13428627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d40b3c80 con 0x7f01d41477b0 2023-10-11T09:44:05.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.614+0000 7f01c9ffb700 1 -- 172.21.15.78:0/13428627 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01cc007310 con 0x7f01d41477b0 2023-10-11T09:44:05.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.614+0000 7f01da7ef700 1 -- 172.21.15.78:0/13428627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d40b4240 con 0x7f01d41477b0 2023-10-11T09:44:05.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.614+0000 7f01da7ef700 1 -- 172.21.15.78:0/13428627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01d404ed80 con 0x7f01d41477b0 2023-10-11T09:44:05.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.618+0000 7f01c9ffb700 1 -- 172.21.15.78:0/13428627 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f01cc007470 con 0x7f01d41477b0 2023-10-11T09:44:05.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.618+0000 7f01c9ffb700 1 --2- 172.21.15.78:0/13428627 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f01b8065ed0 0x7f01b8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:05.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.618+0000 7f01c9ffb700 1 -- 172.21.15.78:0/13428627 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f01cc0831f0 con 0x7f01d41477b0 2023-10-11T09:44:05.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.618+0000 7f01d97ed700 1 --2- 172.21.15.78:0/13428627 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f01b8065ed0 0x7f01b8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:05.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.618+0000 7f01d97ed700 1 --2- 172.21.15.78:0/13428627 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f01b8065ed0 0x7f01b8068380 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f01d00090b0 tx=0x7f01d0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:05.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.626+0000 7f01c9ffb700 1 -- 172.21.15.78:0/13428627 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f01cc0509a0 con 0x7f01d41477b0 2023-10-11T09:44:05.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:05.798+0000 7f01da7ef700 1 -- 172.21.15.78:0/13428627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f01d40b14c0 con 0x7f01d41477b0 2023-10-11T09:44:06.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.398+0000 7f01c9ffb700 1 -- 172.21.15.78:0/13428627 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v59) v1 ==== 170+0+0 (secure 0 0 0) 0x7f01cc053fc0 con 0x7f01d41477b0 2023-10-11T09:44:06.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.410+0000 7f01da7ef700 1 -- 172.21.15.78:0/13428627 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f01b8065ed0 msgr2=0x7f01b8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:06.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.410+0000 7f01da7ef700 1 --2- 172.21.15.78:0/13428627 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f01b8065ed0 0x7f01b8068380 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f01d00090b0 tx=0x7f01d0008030 comp rx=0 tx=0).stop 2023-10-11T09:44:06.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.410+0000 7f01da7ef700 1 -- 172.21.15.78:0/13428627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01d41477b0 msgr2=0x7f01d4145d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:06.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.410+0000 7f01da7ef700 1 --2- 172.21.15.78:0/13428627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01d41477b0 0x7f01d4145d20 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f01cc002230 tx=0x7f01cc00a060 comp rx=0 tx=0).stop 2023-10-11T09:44:06.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.414+0000 7f01da7ef700 1 -- 172.21.15.78:0/13428627 shutdown_connections 2023-10-11T09:44:06.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.414+0000 7f01da7ef700 1 --2- 172.21.15.78:0/13428627 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f01b8065ed0 0x7f01b8068380 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:06.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.414+0000 7f01da7ef700 1 --2- 172.21.15.78:0/13428627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01d41477b0 0x7f01d4145d20 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:06.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.414+0000 7f01da7ef700 1 --2- 172.21.15.78:0/13428627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01d4146f50 0x7f01d41f65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:06.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.414+0000 7f01da7ef700 1 --2- 172.21.15.78:0/13428627 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f01d40ce150 0x7f01d415dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:06.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.414+0000 7f01da7ef700 1 -- 172.21.15.78:0/13428627 >> 172.21.15.78:0/13428627 conn(0x7f01d40b0780 msgr2=0x7f01d40afbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:06.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.414+0000 7f01da7ef700 1 -- 172.21.15.78:0/13428627 shutdown_connections 2023-10-11T09:44:06.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.418+0000 7f01da7ef700 1 -- 172.21.15.78:0/13428627 wait complete. 2023-10-11T09:44:06.429 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' created 2023-10-11T09:44:06.446 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-11T09:44:06.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.614+0000 7f8ffa066700 1 Processor -- start 2023-10-11T09:44:06.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.614+0000 7f8ffa066700 1 -- start start 2023-10-11T09:44:06.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.614+0000 7f8ffa066700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f8ff4146fe0 0x7f8ff406bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:06.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.614+0000 7f8ffa066700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8ff4161d90 0x7f8ff40b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:06.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.614+0000 7f8ffa066700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ff406ccb0 con 0x7f8ff40b3520 2023-10-11T09:44:06.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.614+0000 7f8ffa066700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ff406c410 con 0x7f8ff4146fe0 2023-10-11T09:44:06.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.614+0000 7f8ffa066700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ff406c6f0 con 0x7f8ff4161d90 2023-10-11T09:44:06.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.614+0000 7f8ff9865700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f8ff40b3520 0x7f8ff41647e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33768/0 (socket says 172.21.15.78:33768) 2023-10-11T09:44:06.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.614+0000 7f8ff9865700 1 -- 172.21.15.78:0/1727825967 learned_addr learned my addr 172.21.15.78:0/1727825967 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:44:06.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.614+0000 7f8ff9064700 1 --2- 172.21.15.78:0/1727825967 >> v2:172.21.15.78:3300/0 conn(0x7f8ff4146fe0 0x7f8ff406bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:06.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.618+0000 7f8ff8863700 1 --2- 172.21.15.78:0/1727825967 >> v2:172.21.15.133:3300/0 conn(0x7f8ff4161d90 0x7f8ff40b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:06.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.618+0000 7f8febfff700 1 -- 172.21.15.78:0/1727825967 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 60127868 0 0) 0x7f8fec001490 con 0x7f8ff40b3520 2023-10-11T09:44:06.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.618+0000 7f8febfff700 1 -- 172.21.15.78:0/1727825967 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fe00035d0 con 0x7f8ff40b3520 2023-10-11T09:44:06.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.618+0000 7f8ff9064700 1 -- 172.21.15.78:0/1727825967 >> v2:172.21.15.133:3300/0 conn(0x7f8ff4161d90 msgr2=0x7f8ff40b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:44:06.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.618+0000 7f8ff9064700 1 --2- 172.21.15.78:0/1727825967 >> v2:172.21.15.133:3300/0 conn(0x7f8ff4161d90 0x7f8ff40b2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:06.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.618+0000 7f8ff9064700 1 -- 172.21.15.78:0/1727825967 >> v1:172.21.15.78:6789/0 conn(0x7f8ff40b3520 legacy=0x7f8ff41647e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:06.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.618+0000 7f8ff9064700 1 -- 172.21.15.78:0/1727825967 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff406c9d0 con 0x7f8ff4146fe0 2023-10-11T09:44:06.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.618+0000 7f8ff9064700 1 --2- 172.21.15.78:0/1727825967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ff4146fe0 0x7f8ff406bbe0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f8ff406b920 tx=0x7f8ff0009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb9905fb887a980 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:06.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.618+0000 7f8febfff700 1 -- 172.21.15.78:0/1727825967 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0011830 con 0x7f8ff4146fe0 2023-10-11T09:44:06.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.618+0000 7f8febfff700 1 -- 172.21.15.78:0/1727825967 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ff0011990 con 0x7f8ff4146fe0 2023-10-11T09:44:06.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.618+0000 7f8febfff700 1 -- 172.21.15.78:0/1727825967 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0011d20 con 0x7f8ff4146fe0 2023-10-11T09:44:06.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.618+0000 7f8ffa066700 1 -- 172.21.15.78:0/1727825967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ff4146fe0 msgr2=0x7f8ff406bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:06.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.618+0000 7f8ffa066700 1 --2- 172.21.15.78:0/1727825967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ff4146fe0 0x7f8ff406bbe0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f8ff406b920 tx=0x7f8ff0009b30 comp rx=0 tx=0).stop 2023-10-11T09:44:06.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.618+0000 7f8ffa066700 1 -- 172.21.15.78:0/1727825967 shutdown_connections 2023-10-11T09:44:06.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.618+0000 7f8ffa066700 1 --2- 172.21.15.78:0/1727825967 >> v2:172.21.15.133:3300/0 conn(0x7f8ff4161d90 0x7f8ff40b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:06.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.618+0000 7f8ffa066700 1 --2- 172.21.15.78:0/1727825967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ff4146fe0 0x7f8ff406bbe0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:06.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.618+0000 7f8ffa066700 1 -- 172.21.15.78:0/1727825967 >> 172.21.15.78:0/1727825967 conn(0x7f8ff406a060 msgr2=0x7f8ff406a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:06.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.618+0000 7f8ffa066700 1 -- 172.21.15.78:0/1727825967 shutdown_connections 2023-10-11T09:44:06.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.618+0000 7f8ffa066700 1 -- 172.21.15.78:0/1727825967 wait complete. 2023-10-11T09:44:06.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.622+0000 7f8ffa066700 1 Processor -- start 2023-10-11T09:44:06.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.622+0000 7f8ffa066700 1 -- start start 2023-10-11T09:44:06.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.622+0000 7f8ffa066700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ff40b3520 0x7f8ff41f71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:06.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.622+0000 7f8ffa066700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ff4146fe0 0x7f8ff4158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:06.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.622+0000 7f8ffa066700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8ff4161d90 0x7f8ff41f77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:06.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.622+0000 7f8ffa066700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff4158b90 con 0x7f8ff40b3520 2023-10-11T09:44:06.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.622+0000 7f8ffa066700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff4159690 con 0x7f8ff4146fe0 2023-10-11T09:44:06.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.622+0000 7f8ffa066700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff4158df0 con 0x7f8ff4161d90 2023-10-11T09:44:06.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.622+0000 7f8ff9064700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ff40b3520 0x7f8ff41f71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:06.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.622+0000 7f8ff9064700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ff40b3520 0x7f8ff41f71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34916/0 (socket says 172.21.15.78:34916) 2023-10-11T09:44:06.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.622+0000 7f8ff9064700 1 -- 172.21.15.78:0/4205520134 learned_addr learned my addr 172.21.15.78:0/4205520134 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:06.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.622+0000 7f8ff8863700 1 --2- 172.21.15.78:0/4205520134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ff4146fe0 0x7f8ff4158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:06.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.622+0000 7f8ff9865700 1 --2- 172.21.15.78:0/4205520134 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8ff4161d90 0x7f8ff41f77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:06.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.622+0000 7f8ff9064700 1 -- 172.21.15.78:0/4205520134 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8ff4161d90 msgr2=0x7f8ff41f77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:06.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.622+0000 7f8ff9064700 1 --2- 172.21.15.78:0/4205520134 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8ff4161d90 0x7f8ff41f77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:06.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.622+0000 7f8ff9064700 1 -- 172.21.15.78:0/4205520134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ff4146fe0 msgr2=0x7f8ff4158650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:06.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.622+0000 7f8ff9064700 1 --2- 172.21.15.78:0/4205520134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ff4146fe0 0x7f8ff4158650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:06.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.626+0000 7f8ff9064700 1 -- 172.21.15.78:0/4205520134 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff00094b0 con 0x7f8ff40b3520 2023-10-11T09:44:06.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.626+0000 7f8ff9064700 1 --2- 172.21.15.78:0/4205520134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ff40b3520 0x7f8ff41f71d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0009b70 tx=0x7f8ff0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:06.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.626+0000 7f8fe9ffb700 1 -- 172.21.15.78:0/4205520134 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0004400 con 0x7f8ff40b3520 2023-10-11T09:44:06.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.626+0000 7f8fe9ffb700 1 -- 172.21.15.78:0/4205520134 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ff001f070 con 0x7f8ff40b3520 2023-10-11T09:44:06.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.626+0000 7f8ffa066700 1 -- 172.21.15.78:0/4205520134 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ff41590d0 con 0x7f8ff40b3520 2023-10-11T09:44:06.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.626+0000 7f8fe9ffb700 1 -- 172.21.15.78:0/4205520134 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff00036c0 con 0x7f8ff40b3520 2023-10-11T09:44:06.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.626+0000 7f8ffa066700 1 -- 172.21.15.78:0/4205520134 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff415b6a0 con 0x7f8ff40b3520 2023-10-11T09:44:06.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.626+0000 7f8ffa066700 1 -- 172.21.15.78:0/4205520134 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ff4165690 con 0x7f8ff40b3520 2023-10-11T09:44:06.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.630+0000 7f8fe9ffb700 1 -- 172.21.15.78:0/4205520134 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f8ff0021030 con 0x7f8ff40b3520 2023-10-11T09:44:06.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.630+0000 7f8fe9ffb700 1 --2- 172.21.15.78:0/4205520134 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8fd8065ed0 0x7f8fd8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:06.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.630+0000 7f8fe9ffb700 1 -- 172.21.15.78:0/4205520134 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f8ff0084b00 con 0x7f8ff40b3520 2023-10-11T09:44:06.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.630+0000 7f8ff8863700 1 --2- 172.21.15.78:0/4205520134 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8fd8065ed0 0x7f8fd8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:06.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.630+0000 7f8ff8863700 1 --2- 172.21.15.78:0/4205520134 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8fd8065ed0 0x7f8fd8068380 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8ff41f9b80 tx=0x7f8fe4009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:06.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.634+0000 7f8fe9ffb700 1 -- 172.21.15.78:0/4205520134 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f8ff0086e60 con 0x7f8ff40b3520 2023-10-11T09:44:06.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.846+0000 7f8ffa066700 1 -- 172.21.15.78:0/4205520134 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f8ff415b890 con 0x7f8ff40b3520 2023-10-11T09:44:06.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.866+0000 7f8fe9ffb700 1 -- 172.21.15.78:0/4205520134 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v33) v1 ==== 172+0+0 (secure 0 0 0) 0x7f8ff0015020 con 0x7f8ff40b3520 2023-10-11T09:44:06.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.874+0000 7f8ffa066700 1 -- 172.21.15.78:0/4205520134 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8fd8065ed0 msgr2=0x7f8fd8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:06.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.874+0000 7f8ffa066700 1 --2- 172.21.15.78:0/4205520134 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8fd8065ed0 0x7f8fd8068380 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8ff41f9b80 tx=0x7f8fe4009240 comp rx=0 tx=0).stop 2023-10-11T09:44:06.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.874+0000 7f8ffa066700 1 -- 172.21.15.78:0/4205520134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ff40b3520 msgr2=0x7f8ff41f71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:06.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.874+0000 7f8ffa066700 1 --2- 172.21.15.78:0/4205520134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ff40b3520 0x7f8ff41f71d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0009b70 tx=0x7f8ff0004770 comp rx=0 tx=0).stop 2023-10-11T09:44:06.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.874+0000 7f8ffa066700 1 -- 172.21.15.78:0/4205520134 shutdown_connections 2023-10-11T09:44:06.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.874+0000 7f8ffa066700 1 --2- 172.21.15.78:0/4205520134 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8fd8065ed0 0x7f8fd8068380 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:06.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.874+0000 7f8ffa066700 1 --2- 172.21.15.78:0/4205520134 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8ff4161d90 0x7f8ff41f77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:06.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.874+0000 7f8ffa066700 1 --2- 172.21.15.78:0/4205520134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ff4146fe0 0x7f8ff4158650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:06.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.874+0000 7f8ffa066700 1 --2- 172.21.15.78:0/4205520134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ff40b3520 0x7f8ff41f71d0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:06.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.874+0000 7f8ffa066700 1 -- 172.21.15.78:0/4205520134 >> 172.21.15.78:0/4205520134 conn(0x7f8ff406a060 msgr2=0x7f8ff40ae660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:06.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.882+0000 7f8ffa066700 1 -- 172.21.15.78:0/4205520134 shutdown_connections 2023-10-11T09:44:06.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:06.886+0000 7f8ffa066700 1 -- 172.21.15.78:0/4205520134 wait complete. 2023-10-11T09:44:06.898 INFO:teuthology.orchestra.run.smithi078.stderr:new fs with metadata pool 8 and data pool 9 2023-10-11T09:44:06.915 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:44:07.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.078+0000 7f32d8c17700 1 Processor -- start 2023-10-11T09:44:07.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.078+0000 7f32d8c17700 1 -- start start 2023-10-11T09:44:07.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.078+0000 7f32d8c17700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f32d4071040 0x7f32d40cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:07.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.078+0000 7f32d8c17700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32d40ce330 0x7f32d406be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:07.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.078+0000 7f32d8c17700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d406ffa0 con 0x7f32d4160d40 2023-10-11T09:44:07.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.078+0000 7f32d8c17700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32d4070c20 con 0x7f32d4071040 2023-10-11T09:44:07.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.078+0000 7f32d8c17700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d4161110 con 0x7f32d40ce330 2023-10-11T09:44:07.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.082+0000 7f32d259c700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f32d4160d40 0x7f32d40cd280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33784/0 (socket says 172.21.15.78:33784) 2023-10-11T09:44:07.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.082+0000 7f32d259c700 1 -- 172.21.15.78:0/3217043506 learned_addr learned my addr 172.21.15.78:0/3217043506 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:44:07.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.082+0000 7f32cbfff700 1 --2- 172.21.15.78:0/3217043506 >> v2:172.21.15.78:3300/0 conn(0x7f32d4071040 0x7f32d40cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:07.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.082+0000 7f32d2d9d700 1 --2- 172.21.15.78:0/3217043506 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32d40ce330 0x7f32d406be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:07.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.082+0000 7f32d1d9b700 1 -- 172.21.15.78:0/3217043506 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 317432923 0 0) 0x7f32d406ffa0 con 0x7f32d4160d40 2023-10-11T09:44:07.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.082+0000 7f32d1d9b700 1 -- 172.21.15.78:0/3217043506 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32b40035d0 con 0x7f32d4160d40 2023-10-11T09:44:07.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.082+0000 7f32cbfff700 1 -- 172.21.15.78:0/3217043506 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32d40ce330 msgr2=0x7f32d406be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:44:07.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.082+0000 7f32cbfff700 1 --2- 172.21.15.78:0/3217043506 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32d40ce330 0x7f32d406be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:07.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.082+0000 7f32cbfff700 1 -- 172.21.15.78:0/3217043506 >> v1:172.21.15.78:6789/0 conn(0x7f32d4160d40 legacy=0x7f32d40cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:07.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.082+0000 7f32cbfff700 1 -- 172.21.15.78:0/3217043506 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d4070280 con 0x7f32d4071040 2023-10-11T09:44:07.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.082+0000 7f32cbfff700 1 --2- 172.21.15.78:0/3217043506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32d4071040 0x7f32d40cddf0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009850 tx=0x7f32bc009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78aa54538a64dea7 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:07.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.082+0000 7f32d1d9b700 1 -- 172.21.15.78:0/3217043506 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32bc011830 con 0x7f32d4071040 2023-10-11T09:44:07.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.082+0000 7f32d1d9b700 1 -- 172.21.15.78:0/3217043506 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32bc011990 con 0x7f32d4071040 2023-10-11T09:44:07.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.082+0000 7f32d1d9b700 1 -- 172.21.15.78:0/3217043506 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32bc011d00 con 0x7f32d4071040 2023-10-11T09:44:07.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.082+0000 7f32d8c17700 1 -- 172.21.15.78:0/3217043506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32d4071040 msgr2=0x7f32d40cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:07.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.082+0000 7f32d8c17700 1 --2- 172.21.15.78:0/3217043506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32d4071040 0x7f32d40cddf0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009850 tx=0x7f32bc009b60 comp rx=0 tx=0).stop 2023-10-11T09:44:07.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.082+0000 7f32d8c17700 1 -- 172.21.15.78:0/3217043506 shutdown_connections 2023-10-11T09:44:07.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.082+0000 7f32d8c17700 1 --2- 172.21.15.78:0/3217043506 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32d40ce330 0x7f32d406be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:07.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.082+0000 7f32d8c17700 1 --2- 172.21.15.78:0/3217043506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32d4071040 0x7f32d40cddf0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:07.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.082+0000 7f32d8c17700 1 -- 172.21.15.78:0/3217043506 >> 172.21.15.78:0/3217043506 conn(0x7f32d406aca0 msgr2=0x7f32d40b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:07.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.082+0000 7f32d8c17700 1 -- 172.21.15.78:0/3217043506 shutdown_connections 2023-10-11T09:44:07.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.082+0000 7f32d8c17700 1 -- 172.21.15.78:0/3217043506 wait complete. 2023-10-11T09:44:07.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.086+0000 7f32d8c17700 1 Processor -- start 2023-10-11T09:44:07.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.086+0000 7f32d8c17700 1 -- start start 2023-10-11T09:44:07.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.086+0000 7f32d8c17700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32d4071040 0x7f32d415a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:07.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.086+0000 7f32d8c17700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32d40ce330 0x7f32d40cb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:07.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.086+0000 7f32d8c17700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32d4160d40 0x7f32d41ec680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:07.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.086+0000 7f32d8c17700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d41f6180 con 0x7f32d4071040 2023-10-11T09:44:07.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.090+0000 7f32d8c17700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d41f6440 con 0x7f32d4160d40 2023-10-11T09:44:07.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.090+0000 7f32d8c17700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d41f6720 con 0x7f32d40ce330 2023-10-11T09:44:07.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.090+0000 7f32d259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32d4071040 0x7f32d415a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:07.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.090+0000 7f32d259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32d4071040 0x7f32d415a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34930/0 (socket says 172.21.15.78:34930) 2023-10-11T09:44:07.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.090+0000 7f32d259c700 1 -- 172.21.15.78:0/714977572 learned_addr learned my addr 172.21.15.78:0/714977572 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:07.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.090+0000 7f32cbfff700 1 --2- 172.21.15.78:0/714977572 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32d40ce330 0x7f32d40cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:07.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.090+0000 7f32d2d9d700 1 --2- 172.21.15.78:0/714977572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32d4160d40 0x7f32d41ec680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:07.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.090+0000 7f32d259c700 1 -- 172.21.15.78:0/714977572 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32d40ce330 msgr2=0x7f32d40cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:07.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.090+0000 7f32d259c700 1 --2- 172.21.15.78:0/714977572 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32d40ce330 0x7f32d40cb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:07.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.090+0000 7f32d259c700 1 -- 172.21.15.78:0/714977572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32d4160d40 msgr2=0x7f32d41ec680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:07.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.090+0000 7f32d259c700 1 --2- 172.21.15.78:0/714977572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32d4160d40 0x7f32d41ec680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:07.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.090+0000 7f32d259c700 1 -- 172.21.15.78:0/714977572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32bc0094b0 con 0x7f32d4071040 2023-10-11T09:44:07.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.090+0000 7f32cbfff700 1 --2- 172.21.15.78:0/714977572 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32d40ce330 0x7f32d40cb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:44:07.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.090+0000 7f32d259c700 1 --2- 172.21.15.78:0/714977572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32d4071040 0x7f32d415a0b0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f32c400c4d0 tx=0x7f32c400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:07.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.090+0000 7f32caffd700 1 -- 172.21.15.78:0/714977572 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c40011d0 con 0x7f32d4071040 2023-10-11T09:44:07.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.090+0000 7f32caffd700 1 -- 172.21.15.78:0/714977572 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c4005d90 con 0x7f32d4071040 2023-10-11T09:44:07.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.090+0000 7f32d8c17700 1 -- 172.21.15.78:0/714977572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32d41f6a00 con 0x7f32d4071040 2023-10-11T09:44:07.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.090+0000 7f32caffd700 1 -- 172.21.15.78:0/714977572 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c40072d0 con 0x7f32d4071040 2023-10-11T09:44:07.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.090+0000 7f32d8c17700 1 -- 172.21.15.78:0/714977572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d41f6fc0 con 0x7f32d4071040 2023-10-11T09:44:07.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.090+0000 7f32d8c17700 1 -- 172.21.15.78:0/714977572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32d41f7150 con 0x7f32d4071040 2023-10-11T09:44:07.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.094+0000 7f32caffd700 1 -- 172.21.15.78:0/714977572 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f32c4007430 con 0x7f32d4071040 2023-10-11T09:44:07.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.094+0000 7f32caffd700 1 --2- 172.21.15.78:0/714977572 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f32b8065ed0 0x7f32b8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:07.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.094+0000 7f32caffd700 1 -- 172.21.15.78:0/714977572 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f32c4082c00 con 0x7f32d4071040 2023-10-11T09:44:07.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.094+0000 7f32cbfff700 1 --2- 172.21.15.78:0/714977572 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f32b8065ed0 0x7f32b8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:07.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.094+0000 7f32cbfff700 1 --2- 172.21.15.78:0/714977572 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f32b8065ed0 0x7f32b8068380 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f32d4071430 tx=0x7f32bc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:07.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.098+0000 7f32caffd700 1 -- 172.21.15.78:0/714977572 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f32c4089080 con 0x7f32d4071040 2023-10-11T09:44:07.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.274+0000 7f32d8c17700 1 -- 172.21.15.78:0/714977572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32d4072250 con 0x7f32d4071040 2023-10-11T09:44:07.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.278+0000 7f32caffd700 1 -- 172.21.15.78:0/714977572 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+17892 (secure 0 0 0) 0x7f32c4017090 con 0x7f32d4071040 2023-10-11T09:44:07.285 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:44:07.286 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":60,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:44:06.849730+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:44:05.198742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-11T09:44:05.868171+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:44:07.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.282+0000 7f32d8c17700 1 -- 172.21.15.78:0/714977572 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f32b8065ed0 msgr2=0x7f32b8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:07.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.282+0000 7f32d8c17700 1 --2- 172.21.15.78:0/714977572 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f32b8065ed0 0x7f32b8068380 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f32d4071430 tx=0x7f32bc008030 comp rx=0 tx=0).stop 2023-10-11T09:44:07.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.282+0000 7f32d8c17700 1 -- 172.21.15.78:0/714977572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32d4071040 msgr2=0x7f32d415a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:07.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.282+0000 7f32d8c17700 1 --2- 172.21.15.78:0/714977572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32d4071040 0x7f32d415a0b0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f32c400c4d0 tx=0x7f32c400c890 comp rx=0 tx=0).stop 2023-10-11T09:44:07.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.282+0000 7f32d8c17700 1 -- 172.21.15.78:0/714977572 shutdown_connections 2023-10-11T09:44:07.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.282+0000 7f32d8c17700 1 --2- 172.21.15.78:0/714977572 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f32b8065ed0 0x7f32b8068380 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:07.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.282+0000 7f32d8c17700 1 --2- 172.21.15.78:0/714977572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32d4160d40 0x7f32d41ec680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:07.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.282+0000 7f32d8c17700 1 --2- 172.21.15.78:0/714977572 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32d40ce330 0x7f32d40cb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:07.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.282+0000 7f32d8c17700 1 --2- 172.21.15.78:0/714977572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32d4071040 0x7f32d415a0b0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:07.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.282+0000 7f32d8c17700 1 -- 172.21.15.78:0/714977572 >> 172.21.15.78:0/714977572 conn(0x7f32d406aca0 msgr2=0x7f32d4161250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:07.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.282+0000 7f32d8c17700 1 -- 172.21.15.78:0/714977572 shutdown_connections 2023-10-11T09:44:07.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.282+0000 7f32d8c17700 1 -- 172.21.15.78:0/714977572 wait complete. 2023-10-11T09:44:07.310 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:44:07.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.474+0000 7f3b02ec8700 1 Processor -- start 2023-10-11T09:44:07.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.474+0000 7f3b02ec8700 1 -- start start 2023-10-11T09:44:07.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.478+0000 7f3b02ec8700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f3afc160d40 0x7f3afc0cd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:07.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.478+0000 7f3b02ec8700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3afc0ce500 0x7f3afc06be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:07.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.478+0000 7f3b02ec8700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3afc06ffa0 con 0x7f3afc071040 2023-10-11T09:44:07.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.478+0000 7f3b02ec8700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3afc070c20 con 0x7f3afc160d40 2023-10-11T09:44:07.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.478+0000 7f3b02ec8700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3afc161110 con 0x7f3afc0ce500 2023-10-11T09:44:07.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.478+0000 7f3b00c64700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f3afc160d40 0x7f3afc0cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:07.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.478+0000 7f3b00c64700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f3afc160d40 0x7f3afc0cd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34932/0 (socket says 172.21.15.78:34932) 2023-10-11T09:44:07.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.478+0000 7f3b00c64700 1 -- 172.21.15.78:0/1165810158 learned_addr learned my addr 172.21.15.78:0/1165810158 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:07.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.478+0000 7f3b01465700 1 --2- 172.21.15.78:0/1165810158 >> v2:172.21.15.133:3300/0 conn(0x7f3afc0ce500 0x7f3afc06be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:07.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.478+0000 7f3b00c64700 1 -- 172.21.15.78:0/1165810158 >> v2:172.21.15.133:3300/0 conn(0x7f3afc0ce500 msgr2=0x7f3afc06be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:44:07.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.478+0000 7f3b00c64700 1 --2- 172.21.15.78:0/1165810158 >> v2:172.21.15.133:3300/0 conn(0x7f3afc0ce500 0x7f3afc06be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:07.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.478+0000 7f3b00c64700 1 -- 172.21.15.78:0/1165810158 >> v1:172.21.15.133:6789/0 conn(0x7f3afc071040 legacy=0x7f3afc0cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:44:07.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.478+0000 7f3b00c64700 1 -- 172.21.15.78:0/1165810158 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3afc070280 con 0x7f3afc160d40 2023-10-11T09:44:07.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.478+0000 7f3b01465700 1 --2- 172.21.15.78:0/1165810158 >> v2:172.21.15.133:3300/0 conn(0x7f3afc0ce500 0x7f3afc06be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:44:07.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.478+0000 7f3b00c64700 1 --2- 172.21.15.78:0/1165810158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3afc160d40 0x7f3afc0cd280 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f3afc06f890 tx=0x7f3af800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1874d240da405e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:07.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.478+0000 7f3af37fe700 1 -- 172.21.15.78:0/1165810158 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af800e070 con 0x7f3afc160d40 2023-10-11T09:44:07.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.478+0000 7f3af37fe700 1 -- 172.21.15.78:0/1165810158 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af8003a40 con 0x7f3afc160d40 2023-10-11T09:44:07.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.478+0000 7f3af37fe700 1 -- 172.21.15.78:0/1165810158 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af8003df0 con 0x7f3afc160d40 2023-10-11T09:44:07.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.478+0000 7f3b02ec8700 1 -- 172.21.15.78:0/1165810158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3afc160d40 msgr2=0x7f3afc0cd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:07.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.478+0000 7f3b02ec8700 1 --2- 172.21.15.78:0/1165810158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3afc160d40 0x7f3afc0cd280 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f3afc06f890 tx=0x7f3af800b330 comp rx=0 tx=0).stop 2023-10-11T09:44:07.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.478+0000 7f3b02ec8700 1 -- 172.21.15.78:0/1165810158 shutdown_connections 2023-10-11T09:44:07.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.478+0000 7f3b02ec8700 1 --2- 172.21.15.78:0/1165810158 >> v2:172.21.15.133:3300/0 conn(0x7f3afc0ce500 0x7f3afc06be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:07.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.482+0000 7f3b02ec8700 1 --2- 172.21.15.78:0/1165810158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3afc160d40 0x7f3afc0cd280 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:07.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.482+0000 7f3b02ec8700 1 -- 172.21.15.78:0/1165810158 >> 172.21.15.78:0/1165810158 conn(0x7f3afc06aca0 msgr2=0x7f3afc0b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:07.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.482+0000 7f3b02ec8700 1 -- 172.21.15.78:0/1165810158 shutdown_connections 2023-10-11T09:44:07.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.482+0000 7f3b02ec8700 1 -- 172.21.15.78:0/1165810158 wait complete. 2023-10-11T09:44:07.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.482+0000 7f3b02ec8700 1 Processor -- start 2023-10-11T09:44:07.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.482+0000 7f3b02ec8700 1 -- start start 2023-10-11T09:44:07.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.486+0000 7f3b02ec8700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3afc071040 0x7f3afc146ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:07.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.486+0000 7f3b02ec8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3afc0ce500 0x7f3afc15c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:07.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.486+0000 7f3b02ec8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3afc160d40 0x7f3afc15f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:07.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.486+0000 7f3b02ec8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3afc147410 con 0x7f3afc0ce500 2023-10-11T09:44:07.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.486+0000 7f3b02ec8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3afc15ce60 con 0x7f3afc160d40 2023-10-11T09:44:07.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.486+0000 7f3b02ec8700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3afc15d140 con 0x7f3afc071040 2023-10-11T09:44:07.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.486+0000 7f3af3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3afc0ce500 0x7f3afc15c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:07.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.486+0000 7f3af3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3afc0ce500 0x7f3afc15c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34948/0 (socket says 172.21.15.78:34948) 2023-10-11T09:44:07.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.486+0000 7f3af3fff700 1 -- 172.21.15.78:0/2929736566 learned_addr learned my addr 172.21.15.78:0/2929736566 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:07.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.486+0000 7f3b00c64700 1 --2- 172.21.15.78:0/2929736566 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3afc071040 0x7f3afc146ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:07.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.486+0000 7f3b01465700 1 --2- 172.21.15.78:0/2929736566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3afc160d40 0x7f3afc15f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:07.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.486+0000 7f3af3fff700 1 -- 172.21.15.78:0/2929736566 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3afc071040 msgr2=0x7f3afc146ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:07.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.486+0000 7f3af3fff700 1 --2- 172.21.15.78:0/2929736566 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3afc071040 0x7f3afc146ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:07.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.486+0000 7f3af3fff700 1 -- 172.21.15.78:0/2929736566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3afc160d40 msgr2=0x7f3afc15f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:07.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.486+0000 7f3af3fff700 1 --2- 172.21.15.78:0/2929736566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3afc160d40 0x7f3afc15f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:07.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.486+0000 7f3af3fff700 1 -- 172.21.15.78:0/2929736566 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af8009b30 con 0x7f3afc0ce500 2023-10-11T09:44:07.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.486+0000 7f3b01465700 1 --2- 172.21.15.78:0/2929736566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3afc160d40 0x7f3afc15f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:44:07.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.486+0000 7f3af3fff700 1 --2- 172.21.15.78:0/2929736566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3afc0ce500 0x7f3afc15c410 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f3aec00c550 tx=0x7f3aec00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:07.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.486+0000 7f3af17fa700 1 -- 172.21.15.78:0/2929736566 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec011790 con 0x7f3afc0ce500 2023-10-11T09:44:07.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.486+0000 7f3af17fa700 1 -- 172.21.15.78:0/2929736566 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aec0118f0 con 0x7f3afc0ce500 2023-10-11T09:44:07.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.486+0000 7f3b02ec8700 1 -- 172.21.15.78:0/2929736566 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3afc1eb100 con 0x7f3afc0ce500 2023-10-11T09:44:07.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.486+0000 7f3af17fa700 1 -- 172.21.15.78:0/2929736566 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec011ca0 con 0x7f3afc0ce500 2023-10-11T09:44:07.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.486+0000 7f3b02ec8700 1 -- 172.21.15.78:0/2929736566 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3afc1eb6c0 con 0x7f3afc0ce500 2023-10-11T09:44:07.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.490+0000 7f3b02ec8700 1 -- 172.21.15.78:0/2929736566 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3afc072250 con 0x7f3afc0ce500 2023-10-11T09:44:07.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.490+0000 7f3af17fa700 1 -- 172.21.15.78:0/2929736566 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f3aec01c460 con 0x7f3afc0ce500 2023-10-11T09:44:07.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.494+0000 7f3af17fa700 1 --2- 172.21.15.78:0/2929736566 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3ae8065ed0 0x7f3ae8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:07.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.494+0000 7f3af17fa700 1 -- 172.21.15.78:0/2929736566 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3aec083540 con 0x7f3afc0ce500 2023-10-11T09:44:07.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.494+0000 7f3b00c64700 1 --2- 172.21.15.78:0/2929736566 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3ae8065ed0 0x7f3ae8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:07.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.494+0000 7f3b00c64700 1 --2- 172.21.15.78:0/2929736566 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3ae8065ed0 0x7f3ae8068380 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3af8015040 tx=0x7f3af8009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:07.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.498+0000 7f3af17fa700 1 -- 172.21.15.78:0/2929736566 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f3aec050ba0 con 0x7f3afc0ce500 2023-10-11T09:44:07.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.670+0000 7f3b02ec8700 1 -- 172.21.15.78:0/2929736566 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3afc0c9110 con 0x7f3afc0ce500 2023-10-11T09:44:07.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.670+0000 7f3af17fa700 1 -- 172.21.15.78:0/2929736566 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+17892 (secure 0 0 0) 0x7f3aec0541c0 con 0x7f3afc0ce500 2023-10-11T09:44:07.678 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:44:07.678 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":60,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:44:06.849730+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:44:05.198742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-11T09:44:05.868171+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:44:07.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.674+0000 7f3b02ec8700 1 -- 172.21.15.78:0/2929736566 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3ae8065ed0 msgr2=0x7f3ae8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:07.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.674+0000 7f3b02ec8700 1 --2- 172.21.15.78:0/2929736566 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3ae8065ed0 0x7f3ae8068380 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3af8015040 tx=0x7f3af8009000 comp rx=0 tx=0).stop 2023-10-11T09:44:07.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.674+0000 7f3b02ec8700 1 -- 172.21.15.78:0/2929736566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3afc0ce500 msgr2=0x7f3afc15c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:07.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.674+0000 7f3b02ec8700 1 --2- 172.21.15.78:0/2929736566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3afc0ce500 0x7f3afc15c410 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f3aec00c550 tx=0x7f3aec00c860 comp rx=0 tx=0).stop 2023-10-11T09:44:07.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.674+0000 7f3b02ec8700 1 -- 172.21.15.78:0/2929736566 shutdown_connections 2023-10-11T09:44:07.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.674+0000 7f3b02ec8700 1 --2- 172.21.15.78:0/2929736566 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3ae8065ed0 0x7f3ae8068380 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:07.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.674+0000 7f3b02ec8700 1 --2- 172.21.15.78:0/2929736566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3afc160d40 0x7f3afc15f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:07.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.674+0000 7f3b02ec8700 1 --2- 172.21.15.78:0/2929736566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3afc0ce500 0x7f3afc15c410 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:07.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.674+0000 7f3b02ec8700 1 --2- 172.21.15.78:0/2929736566 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3afc071040 0x7f3afc146ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:07.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.674+0000 7f3b02ec8700 1 -- 172.21.15.78:0/2929736566 >> 172.21.15.78:0/2929736566 conn(0x7f3afc06aca0 msgr2=0x7f3afc161250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:07.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.674+0000 7f3b02ec8700 1 -- 172.21.15.78:0/2929736566 shutdown_connections 2023-10-11T09:44:07.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.674+0000 7f3b02ec8700 1 -- 172.21.15.78:0/2929736566 wait complete. 2023-10-11T09:44:07.702 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-11T09:44:07.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.866+0000 7fb522726700 1 Processor -- start 2023-10-11T09:44:07.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.870+0000 7fb522726700 1 -- start start 2023-10-11T09:44:07.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.870+0000 7fb522726700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fb51c0727d0 0x7fb51c06b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:07.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.870+0000 7fb522726700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb51c0cb320 con 0x7fb51c15ce80 2023-10-11T09:44:07.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.870+0000 7fb522726700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb51c0cb600 con 0x7fb51c06c6a0 2023-10-11T09:44:07.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.870+0000 7fb522726700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb51c160500 con 0x7fb51c0727d0 2023-10-11T09:44:07.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.870+0000 7fb520cc3700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fb51c15ce80 0x7fb51c160920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33796/0 (socket says 172.21.15.78:33796) 2023-10-11T09:44:07.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.870+0000 7fb520cc3700 1 -- 172.21.15.78:0/409423936 learned_addr learned my addr 172.21.15.78:0/409423936 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:44:07.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.874+0000 7fb51b7fe700 1 --2- 172.21.15.78:0/409423936 >> v2:172.21.15.78:3300/0 conn(0x7fb51c0727d0 0x7fb51c06b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:07.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.874+0000 7fb51affd700 1 -- 172.21.15.78:0/409423936 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 153490088 0 0) 0x7fb51c0cb320 con 0x7fb51c15ce80 2023-10-11T09:44:07.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.874+0000 7fb51affd700 1 -- 172.21.15.78:0/409423936 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5040035d0 con 0x7fb51c15ce80 2023-10-11T09:44:07.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.874+0000 7fb51b7fe700 1 -- 172.21.15.78:0/409423936 >> v1:172.21.15.133:6789/0 conn(0x7fb51c06c6a0 legacy=0x7fb51c06ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:07.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.874+0000 7fb51affd700 1 -- 172.21.15.78:0/409423936 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3575753322 0 0) 0x7fb51c0cb600 con 0x7fb51c06c6a0 2023-10-11T09:44:07.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.874+0000 7fb51b7fe700 1 -- 172.21.15.78:0/409423936 >> v1:172.21.15.78:6789/0 conn(0x7fb51c15ce80 legacy=0x7fb51c160920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:07.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.874+0000 7fb51b7fe700 1 -- 172.21.15.78:0/409423936 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb51c0cb8e0 con 0x7fb51c0727d0 2023-10-11T09:44:07.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.874+0000 7fb51b7fe700 1 --2- 172.21.15.78:0/409423936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb51c0727d0 0x7fb51c06b540 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fb510009850 tx=0x7fb510009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d61137f530a03989 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:07.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.874+0000 7fb51affd700 1 -- 172.21.15.78:0/409423936 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb510011830 con 0x7fb51c0727d0 2023-10-11T09:44:07.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.874+0000 7fb51affd700 1 -- 172.21.15.78:0/409423936 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb510011990 con 0x7fb51c0727d0 2023-10-11T09:44:07.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.874+0000 7fb51affd700 1 -- 172.21.15.78:0/409423936 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb510011d40 con 0x7fb51c0727d0 2023-10-11T09:44:07.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.874+0000 7fb522726700 1 -- 172.21.15.78:0/409423936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb51c0727d0 msgr2=0x7fb51c06b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:07.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.874+0000 7fb522726700 1 --2- 172.21.15.78:0/409423936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb51c0727d0 0x7fb51c06b540 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fb510009850 tx=0x7fb510009b60 comp rx=0 tx=0).stop 2023-10-11T09:44:07.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.874+0000 7fb522726700 1 -- 172.21.15.78:0/409423936 shutdown_connections 2023-10-11T09:44:07.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.874+0000 7fb522726700 1 --2- 172.21.15.78:0/409423936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb51c0727d0 0x7fb51c06b540 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:07.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.874+0000 7fb522726700 1 -- 172.21.15.78:0/409423936 >> 172.21.15.78:0/409423936 conn(0x7fb51c0b1d60 msgr2=0x7fb51c0cf5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:07.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.874+0000 7fb522726700 1 -- 172.21.15.78:0/409423936 shutdown_connections 2023-10-11T09:44:07.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.874+0000 7fb522726700 1 -- 172.21.15.78:0/409423936 wait complete. 2023-10-11T09:44:07.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.878+0000 7fb522726700 1 Processor -- start 2023-10-11T09:44:07.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.878+0000 7fb522726700 1 -- start start 2023-10-11T09:44:07.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.878+0000 7fb522726700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb51c06c6a0 0x7fb51c149900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:07.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.878+0000 7fb522726700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb51c0727d0 0x7fb51c15f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:07.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.878+0000 7fb522726700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb51c15ce80 0x7fb51c15aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:07.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.878+0000 7fb522726700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c158ca0 con 0x7fb51c15ce80 2023-10-11T09:44:07.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.878+0000 7fb522726700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c158f80 con 0x7fb51c06c6a0 2023-10-11T09:44:07.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.878+0000 7fb522726700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c159260 con 0x7fb51c0727d0 2023-10-11T09:44:07.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.878+0000 7fb520cc3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb51c15ce80 0x7fb51c15aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:07.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.878+0000 7fb520cc3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb51c15ce80 0x7fb51c15aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34960/0 (socket says 172.21.15.78:34960) 2023-10-11T09:44:07.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.878+0000 7fb520cc3700 1 -- 172.21.15.78:0/3260377627 learned_addr learned my addr 172.21.15.78:0/3260377627 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:07.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.878+0000 7fb51bfff700 1 --2- 172.21.15.78:0/3260377627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb51c06c6a0 0x7fb51c149900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:07.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.878+0000 7fb51b7fe700 1 --2- 172.21.15.78:0/3260377627 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb51c0727d0 0x7fb51c15f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:07.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.878+0000 7fb520cc3700 1 -- 172.21.15.78:0/3260377627 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb51c0727d0 msgr2=0x7fb51c15f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:07.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.878+0000 7fb520cc3700 1 --2- 172.21.15.78:0/3260377627 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb51c0727d0 0x7fb51c15f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:07.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.878+0000 7fb520cc3700 1 -- 172.21.15.78:0/3260377627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb51c06c6a0 msgr2=0x7fb51c149900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:07.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.878+0000 7fb520cc3700 1 --2- 172.21.15.78:0/3260377627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb51c06c6a0 0x7fb51c149900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:07.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.878+0000 7fb520cc3700 1 -- 172.21.15.78:0/3260377627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5100094b0 con 0x7fb51c15ce80 2023-10-11T09:44:07.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.878+0000 7fb520cc3700 1 --2- 172.21.15.78:0/3260377627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb51c15ce80 0x7fb51c15aea0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fb51400cb10 tx=0x7fb51400cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:07.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.882+0000 7fb518ff9700 1 -- 172.21.15.78:0/3260377627 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb514007480 con 0x7fb51c15ce80 2023-10-11T09:44:07.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.882+0000 7fb518ff9700 1 -- 172.21.15.78:0/3260377627 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5140075e0 con 0x7fb51c15ce80 2023-10-11T09:44:07.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.882+0000 7fb518ff9700 1 -- 172.21.15.78:0/3260377627 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb514007970 con 0x7fb51c15ce80 2023-10-11T09:44:07.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.882+0000 7fb522726700 1 -- 172.21.15.78:0/3260377627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb51c159540 con 0x7fb51c15ce80 2023-10-11T09:44:07.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.882+0000 7fb522726700 1 -- 172.21.15.78:0/3260377627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb51c149e40 con 0x7fb51c15ce80 2023-10-11T09:44:07.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.882+0000 7fb522726700 1 -- 172.21.15.78:0/3260377627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb51c159a60 con 0x7fb51c15ce80 2023-10-11T09:44:07.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.882+0000 7fb518ff9700 1 -- 172.21.15.78:0/3260377627 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fb51401b3b0 con 0x7fb51c15ce80 2023-10-11T09:44:07.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.886+0000 7fb518ff9700 1 --2- 172.21.15.78:0/3260377627 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb508065ed0 0x7fb508068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:07.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.886+0000 7fb518ff9700 1 -- 172.21.15.78:0/3260377627 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb514083190 con 0x7fb51c15ce80 2023-10-11T09:44:07.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.886+0000 7fb51bfff700 1 --2- 172.21.15.78:0/3260377627 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb508065ed0 0x7fb508068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:07.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.886+0000 7fb51bfff700 1 --2- 172.21.15.78:0/3260377627 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb508065ed0 0x7fb508068380 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb50c000f60 tx=0x7fb50c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:07.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:07.890+0000 7fb518ff9700 1 -- 172.21.15.78:0/3260377627 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fb51c159a60 con 0x7fb51c15ce80 2023-10-11T09:44:08.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:08.106+0000 7fb522726700 1 -- 172.21.15.78:0/3260377627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb51c159a60 con 0x7fb51c15ce80 2023-10-11T09:44:08.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:08.882+0000 7fb518ff9700 1 -- 172.21.15.78:0/3260377627 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v36) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb51c159a60 con 0x7fb51c15ce80 2023-10-11T09:44:08.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:08.886+0000 7fb522726700 1 -- 172.21.15.78:0/3260377627 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb508065ed0 msgr2=0x7fb508068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:08.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:08.886+0000 7fb522726700 1 --2- 172.21.15.78:0/3260377627 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb508065ed0 0x7fb508068380 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb50c000f60 tx=0x7fb50c009040 comp rx=0 tx=0).stop 2023-10-11T09:44:08.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:08.886+0000 7fb522726700 1 -- 172.21.15.78:0/3260377627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb51c15ce80 msgr2=0x7fb51c15aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:08.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:08.886+0000 7fb522726700 1 --2- 172.21.15.78:0/3260377627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb51c15ce80 0x7fb51c15aea0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fb51400cb10 tx=0x7fb51400cb40 comp rx=0 tx=0).stop 2023-10-11T09:44:08.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:08.886+0000 7fb522726700 1 -- 172.21.15.78:0/3260377627 shutdown_connections 2023-10-11T09:44:08.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:08.886+0000 7fb522726700 1 --2- 172.21.15.78:0/3260377627 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb508065ed0 0x7fb508068380 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:08.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:08.886+0000 7fb522726700 1 --2- 172.21.15.78:0/3260377627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb51c15ce80 0x7fb51c15aea0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:08.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:08.886+0000 7fb522726700 1 --2- 172.21.15.78:0/3260377627 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb51c0727d0 0x7fb51c15f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:08.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:08.886+0000 7fb522726700 1 --2- 172.21.15.78:0/3260377627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb51c06c6a0 0x7fb51c149900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:08.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:08.886+0000 7fb522726700 1 -- 172.21.15.78:0/3260377627 >> 172.21.15.78:0/3260377627 conn(0x7fb51c0b1d60 msgr2=0x7fb51c0ac8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:08.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:08.886+0000 7fb522726700 1 -- 172.21.15.78:0/3260377627 shutdown_connections 2023-10-11T09:44:08.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:08.886+0000 7fb522726700 1 -- 172.21.15.78:0/3260377627 wait complete. 2023-10-11T09:44:08.914 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-11T09:44:08.915 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-11T09:44:09.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.082+0000 7f07c1071700 1 Processor -- start 2023-10-11T09:44:09.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.082+0000 7f07c1071700 1 -- start start 2023-10-11T09:44:09.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.082+0000 7f07c1071700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f07bc158430 0x7f07bc0710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:09.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.082+0000 7f07c1071700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f07bc06d340 0x7f07bc0ca880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:09.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.082+0000 7f07c1071700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07bc0cadc0 0x7f07bc15afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:09.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.082+0000 7f07c1071700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07bc070c10 con 0x7f07bc06d340 2023-10-11T09:44:09.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.082+0000 7f07c1071700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07bc0cb240 con 0x7f07bc158430 2023-10-11T09:44:09.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.082+0000 7f07c1071700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07bc071610 con 0x7f07bc0cadc0 2023-10-11T09:44:09.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.086+0000 7f07b259c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f07bc06d340 0x7f07bc0ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:09.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.086+0000 7f07b259c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f07bc06d340 0x7f07bc0ca880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34962/0 (socket says 172.21.15.78:34962) 2023-10-11T09:44:09.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.086+0000 7f07b259c700 1 -- 172.21.15.78:0/2018516632 learned_addr learned my addr 172.21.15.78:0/2018516632 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:09.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.086+0000 7f07bad9d700 1 --2- 172.21.15.78:0/2018516632 >> v2:172.21.15.133:3300/0 conn(0x7f07bc158430 0x7f07bc0710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:09.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.086+0000 7f07bb59e700 1 --2- 172.21.15.78:0/2018516632 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07bc0cadc0 0x7f07bc15afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:09.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.086+0000 7f07b259c700 1 -- 172.21.15.78:0/2018516632 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07bc0cadc0 msgr2=0x7f07bc15afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:44:09.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.086+0000 7f07b259c700 1 --2- 172.21.15.78:0/2018516632 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07bc0cadc0 0x7f07bc15afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:09.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.086+0000 7f07b259c700 1 -- 172.21.15.78:0/2018516632 >> v2:172.21.15.133:3300/0 conn(0x7f07bc158430 msgr2=0x7f07bc0710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:44:09.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.086+0000 7f07b259c700 1 --2- 172.21.15.78:0/2018516632 >> v2:172.21.15.133:3300/0 conn(0x7f07bc158430 0x7f07bc0710d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:09.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.086+0000 7f07b259c700 1 -- 172.21.15.78:0/2018516632 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07bc15e280 con 0x7f07bc06d340 2023-10-11T09:44:09.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.086+0000 7f07bb59e700 1 --2- 172.21.15.78:0/2018516632 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07bc0cadc0 0x7f07bc15afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:44:09.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.086+0000 7f07bad9d700 1 --2- 172.21.15.78:0/2018516632 >> v2:172.21.15.133:3300/0 conn(0x7f07bc158430 0x7f07bc0710d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:44:09.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.086+0000 7f07b259c700 1 --2- 172.21.15.78:0/2018516632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07bc06d340 0x7f07bc0ca880 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f07bc06fd00 tx=0x7f07a4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e3cbc99124d10d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:09.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.086+0000 7f07ba59c700 1 -- 172.21.15.78:0/2018516632 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a4011830 con 0x7f07bc06d340 2023-10-11T09:44:09.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.086+0000 7f07ba59c700 1 -- 172.21.15.78:0/2018516632 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a4011990 con 0x7f07bc06d340 2023-10-11T09:44:09.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.086+0000 7f07ba59c700 1 -- 172.21.15.78:0/2018516632 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a4011d20 con 0x7f07bc06d340 2023-10-11T09:44:09.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.086+0000 7f07c1071700 1 -- 172.21.15.78:0/2018516632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07bc06d340 msgr2=0x7f07bc0ca880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:09.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.086+0000 7f07c1071700 1 --2- 172.21.15.78:0/2018516632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07bc06d340 0x7f07bc0ca880 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f07bc06fd00 tx=0x7f07a4009bf0 comp rx=0 tx=0).stop 2023-10-11T09:44:09.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.086+0000 7f07c1071700 1 -- 172.21.15.78:0/2018516632 shutdown_connections 2023-10-11T09:44:09.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.086+0000 7f07c1071700 1 --2- 172.21.15.78:0/2018516632 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07bc0cadc0 0x7f07bc15afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:09.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.086+0000 7f07c1071700 1 --2- 172.21.15.78:0/2018516632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07bc06d340 0x7f07bc0ca880 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:09.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.086+0000 7f07c1071700 1 --2- 172.21.15.78:0/2018516632 >> v2:172.21.15.133:3300/0 conn(0x7f07bc158430 0x7f07bc0710d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:09.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.086+0000 7f07c1071700 1 -- 172.21.15.78:0/2018516632 >> 172.21.15.78:0/2018516632 conn(0x7f07bc0cecc0 msgr2=0x7f07bc0cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:09.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.086+0000 7f07c1071700 1 -- 172.21.15.78:0/2018516632 shutdown_connections 2023-10-11T09:44:09.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.086+0000 7f07c1071700 1 -- 172.21.15.78:0/2018516632 wait complete. 2023-10-11T09:44:09.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.090+0000 7f07c1071700 1 Processor -- start 2023-10-11T09:44:09.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.090+0000 7f07c1071700 1 -- start start 2023-10-11T09:44:09.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.094+0000 7f07c1071700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07bc06d340 0x7f07bc1efa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:09.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.094+0000 7f07c1071700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07bc0cadc0 0x7f07bc1f27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:09.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.094+0000 7f07c1071700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07bc158430 0x7f07bc15bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:09.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.094+0000 7f07c1071700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07bc1548d0 con 0x7f07bc06d340 2023-10-11T09:44:09.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.094+0000 7f07c1071700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07bc15b310 con 0x7f07bc158430 2023-10-11T09:44:09.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.094+0000 7f07c1071700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07bc0b0250 con 0x7f07bc0cadc0 2023-10-11T09:44:09.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.094+0000 7f07bad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07bc06d340 0x7f07bc1efa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:09.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.094+0000 7f07bad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07bc06d340 0x7f07bc1efa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34972/0 (socket says 172.21.15.78:34972) 2023-10-11T09:44:09.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.094+0000 7f07bad9d700 1 -- 172.21.15.78:0/3842183387 learned_addr learned my addr 172.21.15.78:0/3842183387 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:09.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.094+0000 7f07b259c700 1 --2- 172.21.15.78:0/3842183387 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07bc0cadc0 0x7f07bc1f27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:09.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.094+0000 7f07bb59e700 1 --2- 172.21.15.78:0/3842183387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07bc158430 0x7f07bc15bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:09.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.094+0000 7f07bad9d700 1 -- 172.21.15.78:0/3842183387 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07bc0cadc0 msgr2=0x7f07bc1f27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:09.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.094+0000 7f07bad9d700 1 --2- 172.21.15.78:0/3842183387 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07bc0cadc0 0x7f07bc1f27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:09.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.094+0000 7f07bad9d700 1 -- 172.21.15.78:0/3842183387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07bc158430 msgr2=0x7f07bc15bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:09.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.094+0000 7f07bad9d700 1 --2- 172.21.15.78:0/3842183387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07bc158430 0x7f07bc15bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:09.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.094+0000 7f07bad9d700 1 -- 172.21.15.78:0/3842183387 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b4009a70 con 0x7f07bc06d340 2023-10-11T09:44:09.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.094+0000 7f07bad9d700 1 --2- 172.21.15.78:0/3842183387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07bc06d340 0x7f07bc1efa60 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f07ac00ec10 tx=0x7f07ac009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:09.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.094+0000 7f07b3fff700 1 -- 172.21.15.78:0/3842183387 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ac00c940 con 0x7f07bc06d340 2023-10-11T09:44:09.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.094+0000 7f07b3fff700 1 -- 172.21.15.78:0/3842183387 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07ac00caa0 con 0x7f07bc06d340 2023-10-11T09:44:09.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.094+0000 7f07b3fff700 1 -- 172.21.15.78:0/3842183387 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ac00ce30 con 0x7f07bc06d340 2023-10-11T09:44:09.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.094+0000 7f07c1071700 1 -- 172.21.15.78:0/3842183387 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07ac0094b0 con 0x7f07bc06d340 2023-10-11T09:44:09.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.094+0000 7f07c1071700 1 -- 172.21.15.78:0/3842183387 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a4009570 con 0x7f07bc06d340 2023-10-11T09:44:09.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.098+0000 7f07c1071700 1 -- 172.21.15.78:0/3842183387 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07bc06eb80 con 0x7f07bc06d340 2023-10-11T09:44:09.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.098+0000 7f07b3fff700 1 -- 172.21.15.78:0/3842183387 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f07ac013470 con 0x7f07bc06d340 2023-10-11T09:44:09.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.098+0000 7f07b3fff700 1 --2- 172.21.15.78:0/3842183387 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f079c065ed0 0x7f079c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:09.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.098+0000 7f07b3fff700 1 -- 172.21.15.78:0/3842183387 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f07ac083e40 con 0x7f07bc06d340 2023-10-11T09:44:09.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.098+0000 7f07b259c700 1 --2- 172.21.15.78:0/3842183387 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f079c065ed0 0x7f079c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:09.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.102+0000 7f07b259c700 1 --2- 172.21.15.78:0/3842183387 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f079c065ed0 0x7f079c068380 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f07a4000c00 tx=0x7f07a4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:09.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.106+0000 7f07b3fff700 1 -- 172.21.15.78:0/3842183387 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f07ac0513f0 con 0x7f07bc06d340 2023-10-11T09:44:09.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:09.322+0000 7f07c1071700 1 -- 172.21.15.78:0/3842183387 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f07bc065db0 con 0x7f07bc06d340 2023-10-11T09:44:10.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.206+0000 7f07b3fff700 1 -- 172.21.15.78:0/3842183387 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v37) v1 ==== 122+0+0 (secure 0 0 0) 0x7f07ac054a10 con 0x7f07bc06d340 2023-10-11T09:44:10.218 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.214+0000 7f07c1071700 1 -- 172.21.15.78:0/3842183387 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f079c065ed0 msgr2=0x7f079c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:10.218 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.214+0000 7f07c1071700 1 --2- 172.21.15.78:0/3842183387 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f079c065ed0 0x7f079c068380 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f07a4000c00 tx=0x7f07a4008030 comp rx=0 tx=0).stop 2023-10-11T09:44:10.218 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.214+0000 7f07c1071700 1 -- 172.21.15.78:0/3842183387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07bc06d340 msgr2=0x7f07bc1efa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:10.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.214+0000 7f07c1071700 1 --2- 172.21.15.78:0/3842183387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07bc06d340 0x7f07bc1efa60 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f07ac00ec10 tx=0x7f07ac009ed0 comp rx=0 tx=0).stop 2023-10-11T09:44:10.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.214+0000 7f07c1071700 1 -- 172.21.15.78:0/3842183387 shutdown_connections 2023-10-11T09:44:10.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.214+0000 7f07c1071700 1 --2- 172.21.15.78:0/3842183387 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f079c065ed0 0x7f079c068380 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:10.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.214+0000 7f07c1071700 1 --2- 172.21.15.78:0/3842183387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07bc158430 0x7f07bc15bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:10.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.214+0000 7f07c1071700 1 --2- 172.21.15.78:0/3842183387 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07bc0cadc0 0x7f07bc1f27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:10.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.214+0000 7f07c1071700 1 --2- 172.21.15.78:0/3842183387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07bc06d340 0x7f07bc1efa60 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:10.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.214+0000 7f07c1071700 1 -- 172.21.15.78:0/3842183387 >> 172.21.15.78:0/3842183387 conn(0x7f07bc0cecc0 msgr2=0x7f07bc0cf090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:10.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.214+0000 7f07c1071700 1 -- 172.21.15.78:0/3842183387 shutdown_connections 2023-10-11T09:44:10.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.214+0000 7f07c1071700 1 -- 172.21.15.78:0/3842183387 wait complete. 2023-10-11T09:44:10.244 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-11T09:44:10.244 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:44:10.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.410+0000 7ff2e2164700 1 Processor -- start 2023-10-11T09:44:10.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.410+0000 7ff2e2164700 1 -- start start 2023-10-11T09:44:10.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.414+0000 7ff2e2164700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff2dc158be0 0x7ff2dc0ce260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:10.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.414+0000 7ff2e2164700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2dc0ce840 con 0x7ff2dc15c630 2023-10-11T09:44:10.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.414+0000 7ff2e2164700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2dc0c9450 con 0x7ff2dc0c9020 2023-10-11T09:44:10.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.414+0000 7ff2e2164700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc158fb0 con 0x7ff2dc158be0 2023-10-11T09:44:10.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.414+0000 7ff2e1963700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7ff2dc15c630 0x7ff2dc06ca50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33812/0 (socket says 172.21.15.78:33812) 2023-10-11T09:44:10.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.414+0000 7ff2e1963700 1 -- 172.21.15.78:0/3688319816 learned_addr learned my addr 172.21.15.78:0/3688319816 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:44:10.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.414+0000 7ff2e1162700 1 --2- 172.21.15.78:0/3688319816 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff2dc158be0 0x7ff2dc0ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:10.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.414+0000 7ff2d3fff700 1 -- 172.21.15.78:0/3688319816 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2289745566 0 0) 0x7ff2dc0ce840 con 0x7ff2dc15c630 2023-10-11T09:44:10.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.414+0000 7ff2d3fff700 1 -- 172.21.15.78:0/3688319816 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2c80035d0 con 0x7ff2dc15c630 2023-10-11T09:44:10.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.414+0000 7ff2d3fff700 1 -- 172.21.15.78:0/3688319816 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3633947606 0 0) 0x7ff2dc0c9450 con 0x7ff2dc0c9020 2023-10-11T09:44:10.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.414+0000 7ff2d3fff700 1 -- 172.21.15.78:0/3688319816 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2dc0ce840 con 0x7ff2dc0c9020 2023-10-11T09:44:10.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.414+0000 7ff2e1162700 1 -- 172.21.15.78:0/3688319816 >> v1:172.21.15.133:6789/0 conn(0x7ff2dc0c9020 legacy=0x7ff2dc06fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:10.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.414+0000 7ff2e1162700 1 -- 172.21.15.78:0/3688319816 >> v1:172.21.15.78:6789/0 conn(0x7ff2dc15c630 legacy=0x7ff2dc06ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:10.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.414+0000 7ff2e1162700 1 -- 172.21.15.78:0/3688319816 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2dc06bc90 con 0x7ff2dc158be0 2023-10-11T09:44:10.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.418+0000 7ff2e1162700 1 --2- 172.21.15.78:0/3688319816 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff2dc158be0 0x7ff2dc0ce260 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8009880 tx=0x7ff2d8009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87b0de8abb038a43 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:10.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.418+0000 7ff2d3fff700 1 -- 172.21.15.78:0/3688319816 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d800c8a0 con 0x7ff2dc158be0 2023-10-11T09:44:10.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.418+0000 7ff2d3fff700 1 -- 172.21.15.78:0/3688319816 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d800c050 con 0x7ff2dc158be0 2023-10-11T09:44:10.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.418+0000 7ff2d3fff700 1 -- 172.21.15.78:0/3688319816 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d800cc20 con 0x7ff2dc158be0 2023-10-11T09:44:10.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.418+0000 7ff2e2164700 1 -- 172.21.15.78:0/3688319816 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff2dc158be0 msgr2=0x7ff2dc0ce260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:10.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.418+0000 7ff2e2164700 1 --2- 172.21.15.78:0/3688319816 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff2dc158be0 0x7ff2dc0ce260 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8009880 tx=0x7ff2d8009b90 comp rx=0 tx=0).stop 2023-10-11T09:44:10.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.418+0000 7ff2e2164700 1 -- 172.21.15.78:0/3688319816 shutdown_connections 2023-10-11T09:44:10.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.418+0000 7ff2e2164700 1 --2- 172.21.15.78:0/3688319816 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff2dc158be0 0x7ff2dc0ce260 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:10.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.418+0000 7ff2e2164700 1 -- 172.21.15.78:0/3688319816 >> 172.21.15.78:0/3688319816 conn(0x7ff2dc0cea60 msgr2=0x7ff2dc0cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:10.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.418+0000 7ff2e2164700 1 -- 172.21.15.78:0/3688319816 shutdown_connections 2023-10-11T09:44:10.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.418+0000 7ff2e2164700 1 -- 172.21.15.78:0/3688319816 wait complete. 2023-10-11T09:44:10.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.422+0000 7ff2e2164700 1 Processor -- start 2023-10-11T09:44:10.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.422+0000 7ff2e2164700 1 -- start start 2023-10-11T09:44:10.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.422+0000 7ff2e2164700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2dc0c9020 0x7ff2dc1ebc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:10.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.422+0000 7ff2e2164700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff2dc158be0 0x7ff2dc1f5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:10.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.422+0000 7ff2e2164700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2dc15c630 0x7ff2dc1f2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:10.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.422+0000 7ff2e2164700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc1f5670 con 0x7ff2dc15c630 2023-10-11T09:44:10.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.422+0000 7ff2e2164700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc1f2b70 con 0x7ff2dc0c9020 2023-10-11T09:44:10.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.422+0000 7ff2e2164700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc1f58a0 con 0x7ff2dc158be0 2023-10-11T09:44:10.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.422+0000 7ff2e1963700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2dc15c630 0x7ff2dc1f2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:10.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.422+0000 7ff2e1963700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2dc15c630 0x7ff2dc1f2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34986/0 (socket says 172.21.15.78:34986) 2023-10-11T09:44:10.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.422+0000 7ff2e1963700 1 -- 172.21.15.78:0/1417746596 learned_addr learned my addr 172.21.15.78:0/1417746596 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:10.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.426+0000 7ff2e1162700 1 --2- 172.21.15.78:0/1417746596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2dc0c9020 0x7ff2dc1ebc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:10.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.426+0000 7ff2e0961700 1 --2- 172.21.15.78:0/1417746596 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff2dc158be0 0x7ff2dc1f5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:10.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.426+0000 7ff2e1963700 1 -- 172.21.15.78:0/1417746596 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff2dc158be0 msgr2=0x7ff2dc1f5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:10.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.426+0000 7ff2e1963700 1 --2- 172.21.15.78:0/1417746596 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff2dc158be0 0x7ff2dc1f5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:10.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.426+0000 7ff2e1963700 1 -- 172.21.15.78:0/1417746596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2dc0c9020 msgr2=0x7ff2dc1ebc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:10.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.426+0000 7ff2e1963700 1 --2- 172.21.15.78:0/1417746596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2dc0c9020 0x7ff2dc1ebc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:10.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.426+0000 7ff2e1963700 1 -- 172.21.15.78:0/1417746596 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d80094b0 con 0x7ff2dc15c630 2023-10-11T09:44:10.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.426+0000 7ff2e1162700 1 --2- 172.21.15.78:0/1417746596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2dc0c9020 0x7ff2dc1ebc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:44:10.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.426+0000 7ff2e1963700 1 --2- 172.21.15.78:0/1417746596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2dc15c630 0x7ff2dc1f2630 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff2d400caf0 tx=0x7ff2d400cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:10.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.426+0000 7ff2d1ffb700 1 -- 172.21.15.78:0/1417746596 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d400a770 con 0x7ff2dc15c630 2023-10-11T09:44:10.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.426+0000 7ff2d1ffb700 1 -- 172.21.15.78:0/1417746596 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d400a8d0 con 0x7ff2dc15c630 2023-10-11T09:44:10.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.426+0000 7ff2e2164700 1 -- 172.21.15.78:0/1417746596 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2dc1f5b80 con 0x7ff2dc15c630 2023-10-11T09:44:10.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.426+0000 7ff2d1ffb700 1 -- 172.21.15.78:0/1417746596 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d400ac60 con 0x7ff2dc15c630 2023-10-11T09:44:10.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.426+0000 7ff2e2164700 1 -- 172.21.15.78:0/1417746596 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2dc15a5f0 con 0x7ff2dc15c630 2023-10-11T09:44:10.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.430+0000 7ff2e2164700 1 -- 172.21.15.78:0/1417746596 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2dc15a810 con 0x7ff2dc15c630 2023-10-11T09:44:10.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.430+0000 7ff2d1ffb700 1 -- 172.21.15.78:0/1417746596 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7ff2d401b3b0 con 0x7ff2dc15c630 2023-10-11T09:44:10.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.430+0000 7ff2d1ffb700 1 --2- 172.21.15.78:0/1417746596 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff2c0065ed0 0x7ff2c0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:10.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.430+0000 7ff2d1ffb700 1 -- 172.21.15.78:0/1417746596 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff2d4083230 con 0x7ff2dc15c630 2023-10-11T09:44:10.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.430+0000 7ff2e1162700 1 --2- 172.21.15.78:0/1417746596 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff2c0065ed0 0x7ff2c0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:10.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.430+0000 7ff2e1162700 1 --2- 172.21.15.78:0/1417746596 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff2c0065ed0 0x7ff2c0068380 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff2d80097d0 tx=0x7ff2d800f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:10.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.438+0000 7ff2d1ffb700 1 -- 172.21.15.78:0/1417746596 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7ff2d40507e0 con 0x7ff2dc15c630 2023-10-11T09:44:10.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.642+0000 7ff2e2164700 1 -- 172.21.15.78:0/1417746596 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2dc065d40 con 0x7ff2dc15c630 2023-10-11T09:44:10.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.646+0000 7ff2d1ffb700 1 -- 172.21.15.78:0/1417746596 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4651 (secure 0 0 0) 0x7ff2d4053e00 con 0x7ff2dc15c630 2023-10-11T09:44:10.650 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:44:10.650 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":37,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5579,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3954348479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":3954348479},{"type":"v1","addr":"172.21.15.133:6835","nonce":3954348479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6100,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/4029676589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4029676589},{"type":"v1","addr":"172.21.15.133:6834","nonce":4029676589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6249,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/1819646091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":1819646091},{"type":"v1","addr":"172.21.15.78:6836","nonce":1819646091}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:44:06.850489+0000","modified":"2023-10-11T09:44:10.203237+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6252},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6252":{"gid":6252,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6837/852594750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":852594750},{"type":"v1","addr":"172.21.15.78:6837","nonce":852594750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-11T09:44:10.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.646+0000 7ff2e2164700 1 -- 172.21.15.78:0/1417746596 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff2c0065ed0 msgr2=0x7ff2c0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:10.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.646+0000 7ff2e2164700 1 --2- 172.21.15.78:0/1417746596 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff2c0065ed0 0x7ff2c0068380 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff2d80097d0 tx=0x7ff2d800f000 comp rx=0 tx=0).stop 2023-10-11T09:44:10.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.646+0000 7ff2e2164700 1 -- 172.21.15.78:0/1417746596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2dc15c630 msgr2=0x7ff2dc1f2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:10.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.646+0000 7ff2e2164700 1 --2- 172.21.15.78:0/1417746596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2dc15c630 0x7ff2dc1f2630 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff2d400caf0 tx=0x7ff2d400cb20 comp rx=0 tx=0).stop 2023-10-11T09:44:10.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.646+0000 7ff2e2164700 1 -- 172.21.15.78:0/1417746596 shutdown_connections 2023-10-11T09:44:10.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.646+0000 7ff2e2164700 1 --2- 172.21.15.78:0/1417746596 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff2c0065ed0 0x7ff2c0068380 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:10.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.646+0000 7ff2e2164700 1 --2- 172.21.15.78:0/1417746596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2dc15c630 0x7ff2dc1f2630 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:10.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.646+0000 7ff2e2164700 1 --2- 172.21.15.78:0/1417746596 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff2dc158be0 0x7ff2dc1f5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:10.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.646+0000 7ff2e2164700 1 --2- 172.21.15.78:0/1417746596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2dc0c9020 0x7ff2dc1ebc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:10.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.646+0000 7ff2e2164700 1 -- 172.21.15.78:0/1417746596 >> 172.21.15.78:0/1417746596 conn(0x7ff2dc0cea60 msgr2=0x7ff2dc15cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:10.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.646+0000 7ff2e2164700 1 -- 172.21.15.78:0/1417746596 shutdown_connections 2023-10-11T09:44:10.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.646+0000 7ff2e2164700 1 -- 172.21.15.78:0/1417746596 wait complete. 2023-10-11T09:44:10.656 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 37 2023-10-11T09:44:10.674 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:44:10.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.838+0000 7fdd59737700 1 Processor -- start 2023-10-11T09:44:10.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.838+0000 7fdd59737700 1 -- start start 2023-10-11T09:44:10.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.842+0000 7fdd59737700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fdd5415b8e0 0x7fdd5406d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:10.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.842+0000 7fdd59737700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd5406d610 0x7fdd540ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:10.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.842+0000 7fdd59737700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdd5415a940 0x7fdd540ca780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:10.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.842+0000 7fdd59737700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd5415e010 con 0x7fdd5415b8e0 2023-10-11T09:44:10.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.842+0000 7fdd59737700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd5415e280 con 0x7fdd5415a940 2023-10-11T09:44:10.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.842+0000 7fdd59737700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5415e560 con 0x7fdd5406d610 2023-10-11T09:44:10.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.842+0000 7fdd52ffd700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fdd5415b8e0 0x7fdd5406d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:10.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.842+0000 7fdd52ffd700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fdd5415b8e0 0x7fdd5406d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34988/0 (socket says 172.21.15.78:34988) 2023-10-11T09:44:10.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.842+0000 7fdd52ffd700 1 -- 172.21.15.78:0/3245053881 learned_addr learned my addr 172.21.15.78:0/3245053881 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:10.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.842+0000 7fdd4bfff700 1 --2- 172.21.15.78:0/3245053881 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd5406d610 0x7fdd540ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:10.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.842+0000 7fdd537fe700 1 --2- 172.21.15.78:0/3245053881 >> v2:172.21.15.133:3300/0 conn(0x7fdd5415a940 0x7fdd540ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:10.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.842+0000 7fdd52ffd700 1 -- 172.21.15.78:0/3245053881 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd5406d610 msgr2=0x7fdd540ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:44:10.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.842+0000 7fdd52ffd700 1 --2- 172.21.15.78:0/3245053881 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd5406d610 0x7fdd540ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:10.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.842+0000 7fdd52ffd700 1 -- 172.21.15.78:0/3245053881 >> v2:172.21.15.133:3300/0 conn(0x7fdd5415a940 msgr2=0x7fdd540ca780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:44:10.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.842+0000 7fdd52ffd700 1 --2- 172.21.15.78:0/3245053881 >> v2:172.21.15.133:3300/0 conn(0x7fdd5415a940 0x7fdd540ca780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:10.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.842+0000 7fdd52ffd700 1 -- 172.21.15.78:0/3245053881 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5415e840 con 0x7fdd5415b8e0 2023-10-11T09:44:10.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.842+0000 7fdd52ffd700 1 --2- 172.21.15.78:0/3245053881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd5415b8e0 0x7fdd5406d0d0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fdd540cf800 tx=0x7fdd44009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec867ef1b1da0b0f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:10.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.842+0000 7fdd527fc700 1 -- 172.21.15.78:0/3245053881 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4400b8a0 con 0x7fdd5415b8e0 2023-10-11T09:44:10.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.842+0000 7fdd527fc700 1 -- 172.21.15.78:0/3245053881 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4400ba00 con 0x7fdd5415b8e0 2023-10-11T09:44:10.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.842+0000 7fdd527fc700 1 -- 172.21.15.78:0/3245053881 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4400bd90 con 0x7fdd5415b8e0 2023-10-11T09:44:10.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.846+0000 7fdd59737700 1 -- 172.21.15.78:0/3245053881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd5415b8e0 msgr2=0x7fdd5406d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:10.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.846+0000 7fdd59737700 1 --2- 172.21.15.78:0/3245053881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd5415b8e0 0x7fdd5406d0d0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fdd540cf800 tx=0x7fdd44009bf0 comp rx=0 tx=0).stop 2023-10-11T09:44:10.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.846+0000 7fdd59737700 1 -- 172.21.15.78:0/3245053881 shutdown_connections 2023-10-11T09:44:10.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.846+0000 7fdd59737700 1 --2- 172.21.15.78:0/3245053881 >> v2:172.21.15.133:3300/0 conn(0x7fdd5415a940 0x7fdd540ca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:10.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.846+0000 7fdd59737700 1 --2- 172.21.15.78:0/3245053881 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd5406d610 0x7fdd540ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:10.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.846+0000 7fdd59737700 1 --2- 172.21.15.78:0/3245053881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd5415b8e0 0x7fdd5406d0d0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:10.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.846+0000 7fdd59737700 1 -- 172.21.15.78:0/3245053881 >> 172.21.15.78:0/3245053881 conn(0x7fdd540cea60 msgr2=0x7fdd540cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:10.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.846+0000 7fdd59737700 1 -- 172.21.15.78:0/3245053881 shutdown_connections 2023-10-11T09:44:10.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.846+0000 7fdd59737700 1 -- 172.21.15.78:0/3245053881 wait complete. 2023-10-11T09:44:10.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.850+0000 7fdd59737700 1 Processor -- start 2023-10-11T09:44:10.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.850+0000 7fdd59737700 1 -- start start 2023-10-11T09:44:10.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.850+0000 7fdd59737700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd5406d610 0x7fdd540c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:10.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.850+0000 7fdd59737700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd5415a940 0x7fdd540cd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:10.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.850+0000 7fdd59737700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd5415b8e0 0x7fdd54159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:10.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.850+0000 7fdd59737700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd54140320 con 0x7fdd5415a940 2023-10-11T09:44:10.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.850+0000 7fdd59737700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd54140600 con 0x7fdd5406d610 2023-10-11T09:44:10.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.850+0000 7fdd59737700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd541408e0 con 0x7fdd5415b8e0 2023-10-11T09:44:10.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.850+0000 7fdd4bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd5415a940 0x7fdd540cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:10.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.850+0000 7fdd4bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd5415a940 0x7fdd540cd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34990/0 (socket says 172.21.15.78:34990) 2023-10-11T09:44:10.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.850+0000 7fdd4bfff700 1 -- 172.21.15.78:0/3778746764 learned_addr learned my addr 172.21.15.78:0/3778746764 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:10.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.850+0000 7fdd52ffd700 1 --2- 172.21.15.78:0/3778746764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd5406d610 0x7fdd540c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:10.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.850+0000 7fdd537fe700 1 --2- 172.21.15.78:0/3778746764 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd5415b8e0 0x7fdd54159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:10.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.850+0000 7fdd4bfff700 1 -- 172.21.15.78:0/3778746764 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd5415b8e0 msgr2=0x7fdd54159360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:10.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.850+0000 7fdd4bfff700 1 --2- 172.21.15.78:0/3778746764 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd5415b8e0 0x7fdd54159360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:10.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.850+0000 7fdd4bfff700 1 -- 172.21.15.78:0/3778746764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd5406d610 msgr2=0x7fdd540c9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:10.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.850+0000 7fdd4bfff700 1 --2- 172.21.15.78:0/3778746764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd5406d610 0x7fdd540c9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:10.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.850+0000 7fdd4bfff700 1 -- 172.21.15.78:0/3778746764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd44009570 con 0x7fdd5415a940 2023-10-11T09:44:10.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.850+0000 7fdd4bfff700 1 --2- 172.21.15.78:0/3778746764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd5415a940 0x7fdd540cd990 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c00d800 tx=0x7fdd3c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:10.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.850+0000 7fdd4b7fe700 1 -- 172.21.15.78:0/3778746764 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c0125f0 con 0x7fdd5415a940 2023-10-11T09:44:10.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.850+0000 7fdd4b7fe700 1 -- 172.21.15.78:0/3778746764 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd3c012750 con 0x7fdd5415a940 2023-10-11T09:44:10.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.850+0000 7fdd59737700 1 -- 172.21.15.78:0/3778746764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd54140bc0 con 0x7fdd5415a940 2023-10-11T09:44:10.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.850+0000 7fdd4b7fe700 1 -- 172.21.15.78:0/3778746764 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c012ac0 con 0x7fdd5415a940 2023-10-11T09:44:10.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.850+0000 7fdd59737700 1 -- 172.21.15.78:0/3778746764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd54134ee0 con 0x7fdd5415a940 2023-10-11T09:44:10.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.854+0000 7fdd59737700 1 -- 172.21.15.78:0/3778746764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd54065d40 con 0x7fdd5415a940 2023-10-11T09:44:10.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.854+0000 7fdd4b7fe700 1 -- 172.21.15.78:0/3778746764 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fdd3c012c20 con 0x7fdd5415a940 2023-10-11T09:44:10.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.858+0000 7fdd4b7fe700 1 --2- 172.21.15.78:0/3778746764 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdd38065ed0 0x7fdd38068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:10.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.858+0000 7fdd4b7fe700 1 -- 172.21.15.78:0/3778746764 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdd3c083b20 con 0x7fdd5415a940 2023-10-11T09:44:10.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.858+0000 7fdd52ffd700 1 --2- 172.21.15.78:0/3778746764 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdd38065ed0 0x7fdd38068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:10.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.858+0000 7fdd52ffd700 1 --2- 172.21.15.78:0/3778746764 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdd38065ed0 0x7fdd38068380 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fdd44000c00 tx=0x7fdd44008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:10.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:10.862+0000 7fdd4b7fe700 1 -- 172.21.15.78:0/3778746764 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fdd3c051160 con 0x7fdd5415a940 2023-10-11T09:44:11.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.034+0000 7fdd59737700 1 -- 172.21.15.78:0/3778746764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd5415b090 con 0x7fdd5415a940 2023-10-11T09:44:11.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.038+0000 7fdd4b7fe700 1 -- 172.21.15.78:0/3778746764 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+17879 (secure 0 0 0) 0x7fdd3c012ed0 con 0x7fdd5415a940 2023-10-11T09:44:11.045 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:44:11.045 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":61,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:44:07.856961+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:44:05.198742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-11T09:44:05.868171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:44:11.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.042+0000 7fdd59737700 1 -- 172.21.15.78:0/3778746764 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdd38065ed0 msgr2=0x7fdd38068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:11.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.042+0000 7fdd59737700 1 --2- 172.21.15.78:0/3778746764 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdd38065ed0 0x7fdd38068380 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fdd44000c00 tx=0x7fdd44008030 comp rx=0 tx=0).stop 2023-10-11T09:44:11.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.042+0000 7fdd59737700 1 -- 172.21.15.78:0/3778746764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd5415a940 msgr2=0x7fdd540cd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:11.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.042+0000 7fdd59737700 1 --2- 172.21.15.78:0/3778746764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd5415a940 0x7fdd540cd990 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c00d800 tx=0x7fdd3c00dbc0 comp rx=0 tx=0).stop 2023-10-11T09:44:11.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.042+0000 7fdd59737700 1 -- 172.21.15.78:0/3778746764 shutdown_connections 2023-10-11T09:44:11.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.042+0000 7fdd59737700 1 --2- 172.21.15.78:0/3778746764 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdd38065ed0 0x7fdd38068380 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:11.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.042+0000 7fdd59737700 1 --2- 172.21.15.78:0/3778746764 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd5415b8e0 0x7fdd54159360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:11.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.042+0000 7fdd59737700 1 --2- 172.21.15.78:0/3778746764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd5415a940 0x7fdd540cd990 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:11.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.042+0000 7fdd59737700 1 --2- 172.21.15.78:0/3778746764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd5406d610 0x7fdd540c9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:11.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.042+0000 7fdd59737700 1 -- 172.21.15.78:0/3778746764 >> 172.21.15.78:0/3778746764 conn(0x7fdd540cea60 msgr2=0x7fdd540c7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:11.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.042+0000 7fdd59737700 1 -- 172.21.15.78:0/3778746764 shutdown_connections 2023-10-11T09:44:11.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.042+0000 7fdd59737700 1 -- 172.21.15.78:0/3778746764 wait complete. 2023-10-11T09:44:11.069 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-11T09:44:11.070 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-11T09:44:11.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.226+0000 7fd48730c700 1 Processor -- start 2023-10-11T09:44:11.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.230+0000 7fd48730c700 1 -- start start 2023-10-11T09:44:11.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.230+0000 7fd48730c700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd480071040 0x7fd4800cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:11.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.230+0000 7fd48730c700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4800720f0 con 0x7fd4800cba00 2023-10-11T09:44:11.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.230+0000 7fd48730c700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4800714e0 con 0x7fd480154e00 2023-10-11T09:44:11.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.230+0000 7fd48730c700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4800717c0 con 0x7fd480071040 2023-10-11T09:44:11.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.230+0000 7fd4858a9700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fd4800cba00 0x7fd48006c010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33822/0 (socket says 172.21.15.78:33822) 2023-10-11T09:44:11.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.230+0000 7fd4858a9700 1 -- 172.21.15.78:0/1638255089 learned_addr learned my addr 172.21.15.78:0/1638255089 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:44:11.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.230+0000 7fd4848a7700 1 --2- 172.21.15.78:0/1638255089 >> v2:172.21.15.133:3300/0 conn(0x7fd480071040 0x7fd4800cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:11.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.230+0000 7fd477fff700 1 -- 172.21.15.78:0/1638255089 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3692329931 0 0) 0x7fd4800720f0 con 0x7fd4800cba00 2023-10-11T09:44:11.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.230+0000 7fd477fff700 1 -- 172.21.15.78:0/1638255089 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd46c0035d0 con 0x7fd4800cba00 2023-10-11T09:44:11.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.230+0000 7fd4848a7700 1 -- 172.21.15.78:0/1638255089 >> v1:172.21.15.133:6789/0 conn(0x7fd480154e00 legacy=0x7fd48015b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:11.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.230+0000 7fd477fff700 1 -- 172.21.15.78:0/1638255089 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3281082155 0 0) 0x7fd4800714e0 con 0x7fd480154e00 2023-10-11T09:44:11.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.230+0000 7fd4848a7700 1 -- 172.21.15.78:0/1638255089 >> v1:172.21.15.78:6789/0 conn(0x7fd4800cba00 legacy=0x7fd48006c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:11.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.230+0000 7fd4848a7700 1 -- 172.21.15.78:0/1638255089 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd480071aa0 con 0x7fd480071040 2023-10-11T09:44:11.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.234+0000 7fd4848a7700 1 --2- 172.21.15.78:0/1638255089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd480071040 0x7fd4800cb4c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd470009850 tx=0x7fd470009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c127704683ddde7f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:11.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.234+0000 7fd477fff700 1 -- 172.21.15.78:0/1638255089 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd470011830 con 0x7fd480071040 2023-10-11T09:44:11.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.234+0000 7fd477fff700 1 -- 172.21.15.78:0/1638255089 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd470011990 con 0x7fd480071040 2023-10-11T09:44:11.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.234+0000 7fd477fff700 1 -- 172.21.15.78:0/1638255089 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd470011d40 con 0x7fd480071040 2023-10-11T09:44:11.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.234+0000 7fd48730c700 1 -- 172.21.15.78:0/1638255089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd480071040 msgr2=0x7fd4800cb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:11.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.234+0000 7fd48730c700 1 --2- 172.21.15.78:0/1638255089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd480071040 0x7fd4800cb4c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd470009850 tx=0x7fd470009b60 comp rx=0 tx=0).stop 2023-10-11T09:44:11.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.234+0000 7fd48730c700 1 -- 172.21.15.78:0/1638255089 shutdown_connections 2023-10-11T09:44:11.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.234+0000 7fd48730c700 1 --2- 172.21.15.78:0/1638255089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd480071040 0x7fd4800cb4c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:11.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.234+0000 7fd48730c700 1 -- 172.21.15.78:0/1638255089 >> 172.21.15.78:0/1638255089 conn(0x7fd48006aca0 msgr2=0x7fd4800b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:11.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.234+0000 7fd48730c700 1 -- 172.21.15.78:0/1638255089 shutdown_connections 2023-10-11T09:44:11.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.234+0000 7fd48730c700 1 -- 172.21.15.78:0/1638255089 wait complete. 2023-10-11T09:44:11.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.238+0000 7fd48730c700 1 Processor -- start 2023-10-11T09:44:11.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.238+0000 7fd48730c700 1 -- start start 2023-10-11T09:44:11.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.238+0000 7fd48730c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd480071040 0x7fd4800ce510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:11.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.238+0000 7fd48730c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4800cba00 0x7fd4800cc870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:11.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.238+0000 7fd48730c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd480154e00 0x7fd4800af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:11.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.238+0000 7fd48730c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4800ccdb0 con 0x7fd480071040 2023-10-11T09:44:11.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.238+0000 7fd48730c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd480155260 con 0x7fd4800cba00 2023-10-11T09:44:11.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.238+0000 7fd48730c700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4800ccfa0 con 0x7fd480154e00 2023-10-11T09:44:11.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.238+0000 7fd4850a8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd480071040 0x7fd4800ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:11.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.238+0000 7fd4850a8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd480071040 0x7fd4800ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34992/0 (socket says 172.21.15.78:34992) 2023-10-11T09:44:11.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.238+0000 7fd4850a8700 1 -- 172.21.15.78:0/1321314572 learned_addr learned my addr 172.21.15.78:0/1321314572 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:11.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.238+0000 7fd4858a9700 1 --2- 172.21.15.78:0/1321314572 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd480154e00 0x7fd4800af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:11.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.238+0000 7fd4848a7700 1 --2- 172.21.15.78:0/1321314572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4800cba00 0x7fd4800cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:11.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.238+0000 7fd4850a8700 1 -- 172.21.15.78:0/1321314572 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd480154e00 msgr2=0x7fd4800af7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:11.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.238+0000 7fd4850a8700 1 --2- 172.21.15.78:0/1321314572 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd480154e00 0x7fd4800af7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:11.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.238+0000 7fd4850a8700 1 -- 172.21.15.78:0/1321314572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4800cba00 msgr2=0x7fd4800cc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:11.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.238+0000 7fd4850a8700 1 --2- 172.21.15.78:0/1321314572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4800cba00 0x7fd4800cc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:11.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.238+0000 7fd4850a8700 1 -- 172.21.15.78:0/1321314572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4700094b0 con 0x7fd480071040 2023-10-11T09:44:11.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.238+0000 7fd4850a8700 1 --2- 172.21.15.78:0/1321314572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd480071040 0x7fd4800ce510 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd47c00c520 tx=0x7fd47c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:11.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.242+0000 7fd475ffb700 1 -- 172.21.15.78:0/1321314572 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47c0054e0 con 0x7fd480071040 2023-10-11T09:44:11.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.242+0000 7fd475ffb700 1 -- 172.21.15.78:0/1321314572 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd47c0036a0 con 0x7fd480071040 2023-10-11T09:44:11.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.242+0000 7fd48730c700 1 -- 172.21.15.78:0/1321314572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4800cd280 con 0x7fd480071040 2023-10-11T09:44:11.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.242+0000 7fd475ffb700 1 -- 172.21.15.78:0/1321314572 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47c003a50 con 0x7fd480071040 2023-10-11T09:44:11.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.242+0000 7fd48730c700 1 -- 172.21.15.78:0/1321314572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd480155470 con 0x7fd480071040 2023-10-11T09:44:11.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.242+0000 7fd48730c700 1 -- 172.21.15.78:0/1321314572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd48004e530 con 0x7fd480071040 2023-10-11T09:44:11.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.242+0000 7fd475ffb700 1 -- 172.21.15.78:0/1321314572 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fd47c003bb0 con 0x7fd480071040 2023-10-11T09:44:11.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.246+0000 7fd475ffb700 1 --2- 172.21.15.78:0/1321314572 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd464065ed0 0x7fd464068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:11.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.246+0000 7fd475ffb700 1 -- 172.21.15.78:0/1321314572 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd47c082b80 con 0x7fd480071040 2023-10-11T09:44:11.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.246+0000 7fd4848a7700 1 --2- 172.21.15.78:0/1321314572 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd464065ed0 0x7fd464068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:11.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.246+0000 7fd4848a7700 1 --2- 172.21.15.78:0/1321314572 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd464065ed0 0x7fd464068380 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd480071410 tx=0x7fd470008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:11.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.250+0000 7fd475ffb700 1 -- 172.21.15.78:0/1321314572 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fd47c0501d0 con 0x7fd480071040 2023-10-11T09:44:11.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.418+0000 7fd48730c700 1 -- 172.21.15.78:0/1321314572 --> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd48014fa50 con 0x7fd464065ed0 2023-10-11T09:44:11.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.434+0000 7fd475ffb700 1 -- 172.21.15.78:0/1321314572 <== mgr.4101 v2:172.21.15.78:6832/35606 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308629 (secure 0 0 0) 0x7fd48014fa50 con 0x7fd464065ed0 2023-10-11T09:44:11.445 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:44:11.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.442+0000 7fd48730c700 1 -- 172.21.15.78:0/1321314572 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd464065ed0 msgr2=0x7fd464068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:11.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.442+0000 7fd48730c700 1 --2- 172.21.15.78:0/1321314572 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd464065ed0 0x7fd464068380 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd480071410 tx=0x7fd470008030 comp rx=0 tx=0).stop 2023-10-11T09:44:11.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.442+0000 7fd48730c700 1 -- 172.21.15.78:0/1321314572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd480071040 msgr2=0x7fd4800ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:11.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.442+0000 7fd48730c700 1 --2- 172.21.15.78:0/1321314572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd480071040 0x7fd4800ce510 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd47c00c520 tx=0x7fd47c00c830 comp rx=0 tx=0).stop 2023-10-11T09:44:11.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.442+0000 7fd48730c700 1 -- 172.21.15.78:0/1321314572 shutdown_connections 2023-10-11T09:44:11.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.442+0000 7fd48730c700 1 --2- 172.21.15.78:0/1321314572 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd464065ed0 0x7fd464068380 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:11.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.442+0000 7fd48730c700 1 --2- 172.21.15.78:0/1321314572 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd480154e00 0x7fd4800af7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:11.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.442+0000 7fd48730c700 1 --2- 172.21.15.78:0/1321314572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4800cba00 0x7fd4800cc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:11.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.442+0000 7fd48730c700 1 --2- 172.21.15.78:0/1321314572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd480071040 0x7fd4800ce510 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:11.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.442+0000 7fd48730c700 1 -- 172.21.15.78:0/1321314572 >> 172.21.15.78:0/1321314572 conn(0x7fd48006aca0 msgr2=0x7fd4801587d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:11.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.442+0000 7fd48730c700 1 -- 172.21.15.78:0/1321314572 shutdown_connections 2023-10-11T09:44:11.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.442+0000 7fd48730c700 1 -- 172.21.15.78:0/1321314572 wait complete. 2023-10-11T09:44:11.450 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2023-10-11T09:44:11.468 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2023-10-11T09:44:10.393955+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":93,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230656,"kb_used_data":6848,"kb_used_omap":0,"kb_used_meta":223744,"kb_avail":754744064,"statfs":{"total":773094113280,"available":772857921536,"internally_reserved":0,"allocated":7012352,"data_stored":3816902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":229113856},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.003965"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.874112+0000","last_change":"2023-10-11T09:44:06.885569+0000","last_active":"2023-10-11T09:44:07.874112+0000","last_peered":"2023-10-11T09:44:07.874112+0000","last_clean":"2023-10-11T09:44:07.874112+0000","last_became_active":"2023-10-11T09:44:06.885327+0000","last_became_peered":"2023-10-11T09:44:06.885327+0000","last_unstale":"2023-10-11T09:44:07.874112+0000","last_undegraded":"2023-10-11T09:44:07.874112+0000","last_fullsized":"2023-10-11T09:44:07.874112+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:29:14.313842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.895846+0000","last_change":"2023-10-11T09:44:06.887209+0000","last_active":"2023-10-11T09:44:06.895846+0000","last_peered":"2023-10-11T09:44:06.895846+0000","last_clean":"2023-10-11T09:44:06.895846+0000","last_became_active":"2023-10-11T09:44:06.886687+0000","last_became_peered":"2023-10-11T09:44:06.886687+0000","last_unstale":"2023-10-11T09:44:06.895846+0000","last_undegraded":"2023-10-11T09:44:06.895846+0000","last_fullsized":"2023-10-11T09:44:06.895846+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:12:30.534498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.876274+0000","last_change":"2023-10-11T09:44:06.870609+0000","last_active":"2023-10-11T09:44:07.876274+0000","last_peered":"2023-10-11T09:44:07.876274+0000","last_clean":"2023-10-11T09:44:07.876274+0000","last_became_active":"2023-10-11T09:44:06.870314+0000","last_became_peered":"2023-10-11T09:44:06.870314+0000","last_unstale":"2023-10-11T09:44:07.876274+0000","last_undegraded":"2023-10-11T09:44:07.876274+0000","last_fullsized":"2023-10-11T09:44:07.876274+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:14:19.388175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.896989+0000","last_change":"2023-10-11T09:44:06.883407+0000","last_active":"2023-10-11T09:44:06.896989+0000","last_peered":"2023-10-11T09:44:06.896989+0000","last_clean":"2023-10-11T09:44:06.896989+0000","last_became_active":"2023-10-11T09:44:06.883141+0000","last_became_peered":"2023-10-11T09:44:06.883141+0000","last_unstale":"2023-10-11T09:44:06.896989+0000","last_undegraded":"2023-10-11T09:44:06.896989+0000","last_fullsized":"2023-10-11T09:44:06.896989+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:38:14.424051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.873935+0000","last_change":"2023-10-11T09:44:06.896547+0000","last_active":"2023-10-11T09:44:07.873935+0000","last_peered":"2023-10-11T09:44:07.873935+0000","last_clean":"2023-10-11T09:44:07.873935+0000","last_became_active":"2023-10-11T09:44:06.896275+0000","last_became_peered":"2023-10-11T09:44:06.896275+0000","last_unstale":"2023-10-11T09:44:07.873935+0000","last_undegraded":"2023-10-11T09:44:07.873935+0000","last_fullsized":"2023-10-11T09:44:07.873935+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:17:54.521775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:08.410616+0000","last_change":"2023-10-11T09:44:06.883258+0000","last_active":"2023-10-11T09:44:08.410616+0000","last_peered":"2023-10-11T09:44:08.410616+0000","last_clean":"2023-10-11T09:44:08.410616+0000","last_became_active":"2023-10-11T09:44:06.883006+0000","last_became_peered":"2023-10-11T09:44:06.883006+0000","last_unstale":"2023-10-11T09:44:08.410616+0000","last_undegraded":"2023-10-11T09:44:08.410616+0000","last_fullsized":"2023-10-11T09:44:08.410616+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:06:50.772284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.874166+0000","last_change":"2023-10-11T09:44:06.884457+0000","last_active":"2023-10-11T09:44:07.874166+0000","last_peered":"2023-10-11T09:44:07.874166+0000","last_clean":"2023-10-11T09:44:07.874166+0000","last_became_active":"2023-10-11T09:44:06.884188+0000","last_became_peered":"2023-10-11T09:44:06.884188+0000","last_unstale":"2023-10-11T09:44:07.874166+0000","last_undegraded":"2023-10-11T09:44:07.874166+0000","last_fullsized":"2023-10-11T09:44:07.874166+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:14:19.388175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.885023+0000","last_change":"2023-10-11T09:44:06.879891+0000","last_active":"2023-10-11T09:44:06.885023+0000","last_peered":"2023-10-11T09:44:06.885023+0000","last_clean":"2023-10-11T09:44:06.885023+0000","last_became_active":"2023-10-11T09:44:06.879625+0000","last_became_peered":"2023-10-11T09:44:06.879625+0000","last_unstale":"2023-10-11T09:44:06.885023+0000","last_undegraded":"2023-10-11T09:44:06.885023+0000","last_fullsized":"2023-10-11T09:44:06.885023+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:59:54.587753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.874411+0000","last_change":"2023-10-11T09:44:06.894942+0000","last_active":"2023-10-11T09:44:07.874411+0000","last_peered":"2023-10-11T09:44:07.874411+0000","last_clean":"2023-10-11T09:44:07.874411+0000","last_became_active":"2023-10-11T09:44:06.894554+0000","last_became_peered":"2023-10-11T09:44:06.894554+0000","last_unstale":"2023-10-11T09:44:07.874411+0000","last_undegraded":"2023-10-11T09:44:07.874411+0000","last_fullsized":"2023-10-11T09:44:07.874411+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:58:17.041296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.892979+0000","last_change":"2023-10-11T09:44:06.884597+0000","last_active":"2023-10-11T09:44:06.892979+0000","last_peered":"2023-10-11T09:44:06.892979+0000","last_clean":"2023-10-11T09:44:06.892979+0000","last_became_active":"2023-10-11T09:44:06.884327+0000","last_became_peered":"2023-10-11T09:44:06.884327+0000","last_unstale":"2023-10-11T09:44:06.892979+0000","last_undegraded":"2023-10-11T09:44:06.892979+0000","last_fullsized":"2023-10-11T09:44:06.892979+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:35:07.327877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.896575+0000","last_change":"2023-10-11T09:44:06.891071+0000","last_active":"2023-10-11T09:44:06.896575+0000","last_peered":"2023-10-11T09:44:06.896575+0000","last_clean":"2023-10-11T09:44:06.896575+0000","last_became_active":"2023-10-11T09:44:06.890807+0000","last_became_peered":"2023-10-11T09:44:06.890807+0000","last_unstale":"2023-10-11T09:44:06.896575+0000","last_undegraded":"2023-10-11T09:44:06.896575+0000","last_fullsized":"2023-10-11T09:44:06.896575+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:07:11.630325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.901803+0000","last_change":"2023-10-11T09:44:06.890407+0000","last_active":"2023-10-11T09:44:06.901803+0000","last_peered":"2023-10-11T09:44:06.901803+0000","last_clean":"2023-10-11T09:44:06.901803+0000","last_became_active":"2023-10-11T09:44:06.890110+0000","last_became_peered":"2023-10-11T09:44:06.890110+0000","last_unstale":"2023-10-11T09:44:06.901803+0000","last_undegraded":"2023-10-11T09:44:06.901803+0000","last_fullsized":"2023-10-11T09:44:06.901803+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:57:01.167316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.896615+0000","last_change":"2023-10-11T09:44:06.891217+0000","last_active":"2023-10-11T09:44:06.896615+0000","last_peered":"2023-10-11T09:44:06.896615+0000","last_clean":"2023-10-11T09:44:06.896615+0000","last_became_active":"2023-10-11T09:44:06.890919+0000","last_became_peered":"2023-10-11T09:44:06.890919+0000","last_unstale":"2023-10-11T09:44:06.896615+0000","last_undegraded":"2023-10-11T09:44:06.896615+0000","last_fullsized":"2023-10-11T09:44:06.896615+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:15:47.307014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.874633+0000","last_change":"2023-10-11T09:44:06.880741+0000","last_active":"2023-10-11T09:44:07.874633+0000","last_peered":"2023-10-11T09:44:07.874633+0000","last_clean":"2023-10-11T09:44:07.874633+0000","last_became_active":"2023-10-11T09:44:06.880471+0000","last_became_peered":"2023-10-11T09:44:06.880471+0000","last_unstale":"2023-10-11T09:44:07.874633+0000","last_undegraded":"2023-10-11T09:44:07.874633+0000","last_fullsized":"2023-10-11T09:44:07.874633+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:42:38.468268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.874184+0000","last_change":"2023-10-11T09:44:06.886081+0000","last_active":"2023-10-11T09:44:07.874184+0000","last_peered":"2023-10-11T09:44:07.874184+0000","last_clean":"2023-10-11T09:44:07.874184+0000","last_became_active":"2023-10-11T09:44:06.885781+0000","last_became_peered":"2023-10-11T09:44:06.885781+0000","last_unstale":"2023-10-11T09:44:07.874184+0000","last_undegraded":"2023-10-11T09:44:07.874184+0000","last_fullsized":"2023-10-11T09:44:07.874184+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T21:17:07.726161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:08.410596+0000","last_change":"2023-10-11T09:44:06.884835+0000","last_active":"2023-10-11T09:44:08.410596+0000","last_peered":"2023-10-11T09:44:08.410596+0000","last_clean":"2023-10-11T09:44:08.410596+0000","last_became_active":"2023-10-11T09:44:06.884569+0000","last_became_peered":"2023-10-11T09:44:06.884569+0000","last_unstale":"2023-10-11T09:44:08.410596+0000","last_undegraded":"2023-10-11T09:44:08.410596+0000","last_fullsized":"2023-10-11T09:44:08.410596+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:48:36.453678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.885866+0000","last_change":"2023-10-11T09:44:06.881997+0000","last_active":"2023-10-11T09:44:06.885866+0000","last_peered":"2023-10-11T09:44:06.885866+0000","last_clean":"2023-10-11T09:44:06.885866+0000","last_became_active":"2023-10-11T09:44:06.881709+0000","last_became_peered":"2023-10-11T09:44:06.881709+0000","last_unstale":"2023-10-11T09:44:06.885866+0000","last_undegraded":"2023-10-11T09:44:06.885866+0000","last_fullsized":"2023-10-11T09:44:06.885866+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:37:20.675117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.876497+0000","last_change":"2023-10-11T09:44:06.887605+0000","last_active":"2023-10-11T09:44:07.876497+0000","last_peered":"2023-10-11T09:44:07.876497+0000","last_clean":"2023-10-11T09:44:07.876497+0000","last_became_active":"2023-10-11T09:44:06.887305+0000","last_became_peered":"2023-10-11T09:44:06.887305+0000","last_unstale":"2023-10-11T09:44:07.876497+0000","last_undegraded":"2023-10-11T09:44:07.876497+0000","last_fullsized":"2023-10-11T09:44:07.876497+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:26:59.491028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.874187+0000","last_change":"2023-10-11T09:44:06.885721+0000","last_active":"2023-10-11T09:44:07.874187+0000","last_peered":"2023-10-11T09:44:07.874187+0000","last_clean":"2023-10-11T09:44:07.874187+0000","last_became_active":"2023-10-11T09:44:06.885307+0000","last_became_peered":"2023-10-11T09:44:06.885307+0000","last_unstale":"2023-10-11T09:44:07.874187+0000","last_undegraded":"2023-10-11T09:44:07.874187+0000","last_fullsized":"2023-10-11T09:44:07.874187+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:06:00.273203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.889942+0000","last_change":"2023-10-11T09:44:06.882987+0000","last_active":"2023-10-11T09:44:06.889942+0000","last_peered":"2023-10-11T09:44:06.889942+0000","last_clean":"2023-10-11T09:44:06.889942+0000","last_became_active":"2023-10-11T09:44:06.882659+0000","last_became_peered":"2023-10-11T09:44:06.882659+0000","last_unstale":"2023-10-11T09:44:06.889942+0000","last_undegraded":"2023-10-11T09:44:06.889942+0000","last_fullsized":"2023-10-11T09:44:06.889942+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:15:01.902476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.876809+0000","last_change":"2023-10-11T09:44:06.881149+0000","last_active":"2023-10-11T09:44:07.876809+0000","last_peered":"2023-10-11T09:44:07.876809+0000","last_clean":"2023-10-11T09:44:07.876809+0000","last_became_active":"2023-10-11T09:44:06.879758+0000","last_became_peered":"2023-10-11T09:44:06.879758+0000","last_unstale":"2023-10-11T09:44:07.876809+0000","last_undegraded":"2023-10-11T09:44:07.876809+0000","last_fullsized":"2023-10-11T09:44:07.876809+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T21:14:29.347600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.890240+0000","last_change":"2023-10-11T09:44:06.884612+0000","last_active":"2023-10-11T09:44:06.890240+0000","last_peered":"2023-10-11T09:44:06.890240+0000","last_clean":"2023-10-11T09:44:06.890240+0000","last_became_active":"2023-10-11T09:44:06.884214+0000","last_became_peered":"2023-10-11T09:44:06.884214+0000","last_unstale":"2023-10-11T09:44:06.890240+0000","last_undegraded":"2023-10-11T09:44:06.890240+0000","last_fullsized":"2023-10-11T09:44:06.890240+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:38:14.424051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.876503+0000","last_change":"2023-10-11T09:44:06.886857+0000","last_active":"2023-10-11T09:44:07.876503+0000","last_peered":"2023-10-11T09:44:07.876503+0000","last_clean":"2023-10-11T09:44:07.876503+0000","last_became_active":"2023-10-11T09:44:06.886349+0000","last_became_peered":"2023-10-11T09:44:06.886349+0000","last_unstale":"2023-10-11T09:44:07.876503+0000","last_undegraded":"2023-10-11T09:44:07.876503+0000","last_fullsized":"2023-10-11T09:44:07.876503+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:21:29.593748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.897787+0000","last_change":"2023-10-11T09:44:06.892090+0000","last_active":"2023-10-11T09:44:06.897787+0000","last_peered":"2023-10-11T09:44:06.897787+0000","last_clean":"2023-10-11T09:44:06.897787+0000","last_became_active":"2023-10-11T09:44:06.891764+0000","last_became_peered":"2023-10-11T09:44:06.891764+0000","last_unstale":"2023-10-11T09:44:06.897787+0000","last_undegraded":"2023-10-11T09:44:06.897787+0000","last_fullsized":"2023-10-11T09:44:06.897787+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:25:15.702261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.874222+0000","last_change":"2023-10-11T09:44:06.888272+0000","last_active":"2023-10-11T09:44:07.874222+0000","last_peered":"2023-10-11T09:44:07.874222+0000","last_clean":"2023-10-11T09:44:07.874222+0000","last_became_active":"2023-10-11T09:44:06.888015+0000","last_became_peered":"2023-10-11T09:44:06.888015+0000","last_unstale":"2023-10-11T09:44:07.874222+0000","last_undegraded":"2023-10-11T09:44:07.874222+0000","last_fullsized":"2023-10-11T09:44:07.874222+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:04:16.290001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.896843+0000","last_change":"2023-10-11T09:44:06.882392+0000","last_active":"2023-10-11T09:44:06.896843+0000","last_peered":"2023-10-11T09:44:06.896843+0000","last_clean":"2023-10-11T09:44:06.896843+0000","last_became_active":"2023-10-11T09:44:06.882059+0000","last_became_peered":"2023-10-11T09:44:06.882059+0000","last_unstale":"2023-10-11T09:44:06.896843+0000","last_undegraded":"2023-10-11T09:44:06.896843+0000","last_fullsized":"2023-10-11T09:44:06.896843+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T21:14:29.347600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:08.410855+0000","last_change":"2023-10-11T09:44:06.883337+0000","last_active":"2023-10-11T09:44:08.410855+0000","last_peered":"2023-10-11T09:44:08.410855+0000","last_clean":"2023-10-11T09:44:08.410855+0000","last_became_active":"2023-10-11T09:44:06.882735+0000","last_became_peered":"2023-10-11T09:44:06.882735+0000","last_unstale":"2023-10-11T09:44:08.410855+0000","last_undegraded":"2023-10-11T09:44:08.410855+0000","last_fullsized":"2023-10-11T09:44:08.410855+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T21:04:02.806218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.874330+0000","last_change":"2023-10-11T09:44:06.890036+0000","last_active":"2023-10-11T09:44:07.874330+0000","last_peered":"2023-10-11T09:44:07.874330+0000","last_clean":"2023-10-11T09:44:07.874330+0000","last_became_active":"2023-10-11T09:44:06.889804+0000","last_became_peered":"2023-10-11T09:44:06.889804+0000","last_unstale":"2023-10-11T09:44:07.874330+0000","last_undegraded":"2023-10-11T09:44:07.874330+0000","last_fullsized":"2023-10-11T09:44:07.874330+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:13:41.647635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.882948+0000","last_change":"2023-10-11T09:44:06.879165+0000","last_active":"2023-10-11T09:44:06.882948+0000","last_peered":"2023-10-11T09:44:06.882948+0000","last_clean":"2023-10-11T09:44:06.882948+0000","last_became_active":"2023-10-11T09:44:06.878777+0000","last_became_peered":"2023-10-11T09:44:06.878777+0000","last_unstale":"2023-10-11T09:44:06.882948+0000","last_undegraded":"2023-10-11T09:44:06.882948+0000","last_fullsized":"2023-10-11T09:44:06.882948+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:28:43.416105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.876192+0000","last_change":"2023-10-11T09:44:06.877795+0000","last_active":"2023-10-11T09:44:07.876192+0000","last_peered":"2023-10-11T09:44:07.876192+0000","last_clean":"2023-10-11T09:44:07.876192+0000","last_became_active":"2023-10-11T09:44:06.877446+0000","last_became_peered":"2023-10-11T09:44:06.877446+0000","last_unstale":"2023-10-11T09:44:07.876192+0000","last_undegraded":"2023-10-11T09:44:07.876192+0000","last_fullsized":"2023-10-11T09:44:07.876192+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T21:17:07.726161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.874315+0000","last_change":"2023-10-11T09:44:06.889009+0000","last_active":"2023-10-11T09:44:07.874315+0000","last_peered":"2023-10-11T09:44:07.874315+0000","last_clean":"2023-10-11T09:44:07.874315+0000","last_became_active":"2023-10-11T09:44:06.888775+0000","last_became_peered":"2023-10-11T09:44:06.888775+0000","last_unstale":"2023-10-11T09:44:07.874315+0000","last_undegraded":"2023-10-11T09:44:07.874315+0000","last_fullsized":"2023-10-11T09:44:07.874315+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:17:36.489468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:08.410760+0000","last_change":"2023-10-11T09:44:06.892952+0000","last_active":"2023-10-11T09:44:08.410760+0000","last_peered":"2023-10-11T09:44:08.410760+0000","last_clean":"2023-10-11T09:44:08.410760+0000","last_became_active":"2023-10-11T09:44:06.892709+0000","last_became_peered":"2023-10-11T09:44:06.892709+0000","last_unstale":"2023-10-11T09:44:08.410760+0000","last_undegraded":"2023-10-11T09:44:08.410760+0000","last_fullsized":"2023-10-11T09:44:08.410760+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:00:03.686268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:08.411222+0000","last_change":"2023-10-11T09:44:06.891485+0000","last_active":"2023-10-11T09:44:08.411222+0000","last_peered":"2023-10-11T09:44:08.411222+0000","last_clean":"2023-10-11T09:44:08.411222+0000","last_became_active":"2023-10-11T09:44:06.885845+0000","last_became_peered":"2023-10-11T09:44:06.885845+0000","last_unstale":"2023-10-11T09:44:08.411222+0000","last_undegraded":"2023-10-11T09:44:08.411222+0000","last_fullsized":"2023-10-11T09:44:08.411222+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:59:54.587753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"60'4","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:08.411211+0000","last_change":"2023-10-11T09:44:06.420454+0000","last_active":"2023-10-11T09:44:08.411211+0000","last_peered":"2023-10-11T09:44:08.411211+0000","last_clean":"2023-10-11T09:44:08.411211+0000","last_became_active":"2023-10-11T09:44:06.420196+0000","last_became_peered":"2023-10-11T09:44:06.420196+0000","last_unstale":"2023-10-11T09:44:08.411211+0000","last_undegraded":"2023-10-11T09:44:08.411211+0000","last_fullsized":"2023-10-11T09:44:08.411211+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:14:18.374502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.901231+0000","last_change":"2023-10-11T09:44:06.893045+0000","last_active":"2023-10-11T09:44:06.901231+0000","last_peered":"2023-10-11T09:44:06.901231+0000","last_clean":"2023-10-11T09:44:06.901231+0000","last_became_active":"2023-10-11T09:44:06.892786+0000","last_became_peered":"2023-10-11T09:44:06.892786+0000","last_unstale":"2023-10-11T09:44:06.901231+0000","last_undegraded":"2023-10-11T09:44:06.901231+0000","last_fullsized":"2023-10-11T09:44:06.901231+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:16:00.990878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"60'1","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.887050+0000","last_change":"2023-10-11T09:44:06.426515+0000","last_active":"2023-10-11T09:44:06.887050+0000","last_peered":"2023-10-11T09:44:06.887050+0000","last_clean":"2023-10-11T09:44:06.887050+0000","last_became_active":"2023-10-11T09:44:06.426254+0000","last_became_peered":"2023-10-11T09:44:06.426254+0000","last_unstale":"2023-10-11T09:44:06.887050+0000","last_undegraded":"2023-10-11T09:44:06.887050+0000","last_fullsized":"2023-10-11T09:44:06.887050+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:23:23.869338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:08.411274+0000","last_change":"2023-10-11T09:44:06.895073+0000","last_active":"2023-10-11T09:44:08.411274+0000","last_peered":"2023-10-11T09:44:08.411274+0000","last_clean":"2023-10-11T09:44:08.411274+0000","last_became_active":"2023-10-11T09:44:06.893974+0000","last_became_peered":"2023-10-11T09:44:06.893974+0000","last_unstale":"2023-10-11T09:44:08.411274+0000","last_undegraded":"2023-10-11T09:44:08.411274+0000","last_fullsized":"2023-10-11T09:44:08.411274+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:20:36.948577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"60'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:08.411101+0000","last_change":"2023-10-11T09:44:06.418911+0000","last_active":"2023-10-11T09:44:08.411101+0000","last_peered":"2023-10-11T09:44:08.411101+0000","last_clean":"2023-10-11T09:44:08.411101+0000","last_became_active":"2023-10-11T09:44:06.418644+0000","last_became_peered":"2023-10-11T09:44:06.418644+0000","last_unstale":"2023-10-11T09:44:08.411101+0000","last_undegraded":"2023-10-11T09:44:08.411101+0000","last_fullsized":"2023-10-11T09:44:08.411101+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:58:16.027623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:08.411384+0000","last_change":"2023-10-11T09:44:06.882997+0000","last_active":"2023-10-11T09:44:08.411384+0000","last_peered":"2023-10-11T09:44:08.411384+0000","last_clean":"2023-10-11T09:44:08.411384+0000","last_became_active":"2023-10-11T09:44:06.882074+0000","last_became_peered":"2023-10-11T09:44:06.882074+0000","last_unstale":"2023-10-11T09:44:08.411384+0000","last_undegraded":"2023-10-11T09:44:08.411384+0000","last_fullsized":"2023-10-11T09:44:08.411384+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:15:01.902476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"60'2","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.902131+0000","last_change":"2023-10-11T09:44:06.424342+0000","last_active":"2023-10-11T09:44:06.902131+0000","last_peered":"2023-10-11T09:44:06.902131+0000","last_clean":"2023-10-11T09:44:06.902131+0000","last_became_active":"2023-10-11T09:44:06.424058+0000","last_became_peered":"2023-10-11T09:44:06.424058+0000","last_unstale":"2023-10-11T09:44:06.902131+0000","last_undegraded":"2023-10-11T09:44:06.902131+0000","last_fullsized":"2023-10-11T09:44:06.902131+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:47:17.742443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.890223+0000","last_change":"2023-10-11T09:44:06.884459+0000","last_active":"2023-10-11T09:44:06.890223+0000","last_peered":"2023-10-11T09:44:06.890223+0000","last_clean":"2023-10-11T09:44:06.890223+0000","last_became_active":"2023-10-11T09:44:06.883887+0000","last_became_peered":"2023-10-11T09:44:06.883887+0000","last_unstale":"2023-10-11T09:44:06.890223+0000","last_undegraded":"2023-10-11T09:44:06.890223+0000","last_fullsized":"2023-10-11T09:44:06.890223+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:00:38.019876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"60'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.874385+0000","last_change":"2023-10-11T09:44:06.421809+0000","last_active":"2023-10-11T09:44:07.874385+0000","last_peered":"2023-10-11T09:44:07.874385+0000","last_clean":"2023-10-11T09:44:07.874385+0000","last_became_active":"2023-10-11T09:44:06.421252+0000","last_became_peered":"2023-10-11T09:44:06.421252+0000","last_unstale":"2023-10-11T09:44:07.874385+0000","last_undegraded":"2023-10-11T09:44:07.874385+0000","last_fullsized":"2023-10-11T09:44:07.874385+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:33:38.705914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.901364+0000","last_change":"2023-10-11T09:44:06.892848+0000","last_active":"2023-10-11T09:44:06.901364+0000","last_peered":"2023-10-11T09:44:06.901364+0000","last_clean":"2023-10-11T09:44:06.901364+0000","last_became_active":"2023-10-11T09:44:06.892582+0000","last_became_peered":"2023-10-11T09:44:06.892582+0000","last_unstale":"2023-10-11T09:44:06.901364+0000","last_undegraded":"2023-10-11T09:44:06.901364+0000","last_fullsized":"2023-10-11T09:44:06.901364+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:01:02.059081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"60'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:08.410812+0000","last_change":"2023-10-11T09:44:06.423425+0000","last_active":"2023-10-11T09:44:08.410812+0000","last_peered":"2023-10-11T09:44:08.410812+0000","last_clean":"2023-10-11T09:44:08.410812+0000","last_became_active":"2023-10-11T09:44:06.423124+0000","last_became_peered":"2023-10-11T09:44:06.423124+0000","last_unstale":"2023-10-11T09:44:08.410812+0000","last_undegraded":"2023-10-11T09:44:08.410812+0000","last_fullsized":"2023-10-11T09:44:08.410812+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:50:52.322578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.896553+0000","last_change":"2023-10-11T09:44:06.889441+0000","last_active":"2023-10-11T09:44:06.896553+0000","last_peered":"2023-10-11T09:44:06.896553+0000","last_clean":"2023-10-11T09:44:06.896553+0000","last_became_active":"2023-10-11T09:44:06.888835+0000","last_became_peered":"2023-10-11T09:44:06.888835+0000","last_unstale":"2023-10-11T09:44:06.896553+0000","last_undegraded":"2023-10-11T09:44:06.896553+0000","last_fullsized":"2023-10-11T09:44:06.896553+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:23:24.163168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.878098+0000","last_change":"2023-10-11T09:44:06.422145+0000","last_active":"2023-10-11T09:44:06.878098+0000","last_peered":"2023-10-11T09:44:06.878098+0000","last_clean":"2023-10-11T09:44:06.878098+0000","last_became_active":"2023-10-11T09:44:06.421418+0000","last_became_peered":"2023-10-11T09:44:06.421418+0000","last_unstale":"2023-10-11T09:44:06.878098+0000","last_undegraded":"2023-10-11T09:44:06.878098+0000","last_fullsized":"2023-10-11T09:44:06.878098+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:41:17.407368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.885941+0000","last_change":"2023-10-11T09:44:06.880771+0000","last_active":"2023-10-11T09:44:06.885941+0000","last_peered":"2023-10-11T09:44:06.885941+0000","last_clean":"2023-10-11T09:44:06.885941+0000","last_became_active":"2023-10-11T09:44:06.880162+0000","last_became_peered":"2023-10-11T09:44:06.880162+0000","last_unstale":"2023-10-11T09:44:06.885941+0000","last_undegraded":"2023-10-11T09:44:06.885941+0000","last_fullsized":"2023-10-11T09:44:06.885941+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:21:29.593748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.874279+0000","last_change":"2023-10-11T09:44:06.419933+0000","last_active":"2023-10-11T09:44:07.874279+0000","last_peered":"2023-10-11T09:44:07.874279+0000","last_clean":"2023-10-11T09:44:07.874279+0000","last_became_active":"2023-10-11T09:44:06.419631+0000","last_became_peered":"2023-10-11T09:44:06.419631+0000","last_unstale":"2023-10-11T09:44:07.874279+0000","last_undegraded":"2023-10-11T09:44:07.874279+0000","last_fullsized":"2023-10-11T09:44:07.874279+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:53:48.608829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.874442+0000","last_change":"2023-10-11T09:44:06.885643+0000","last_active":"2023-10-11T09:44:07.874442+0000","last_peered":"2023-10-11T09:44:07.874442+0000","last_clean":"2023-10-11T09:44:07.874442+0000","last_became_active":"2023-10-11T09:44:06.885406+0000","last_became_peered":"2023-10-11T09:44:06.885406+0000","last_unstale":"2023-10-11T09:44:07.874442+0000","last_undegraded":"2023-10-11T09:44:07.874442+0000","last_fullsized":"2023-10-11T09:44:07.874442+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:26:36.532157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.876810+0000","last_change":"2023-10-11T09:44:06.427959+0000","last_active":"2023-10-11T09:44:07.876810+0000","last_peered":"2023-10-11T09:44:07.876810+0000","last_clean":"2023-10-11T09:44:07.876810+0000","last_became_active":"2023-10-11T09:44:06.427566+0000","last_became_peered":"2023-10-11T09:44:06.427566+0000","last_unstale":"2023-10-11T09:44:07.876810+0000","last_undegraded":"2023-10-11T09:44:07.876810+0000","last_fullsized":"2023-10-11T09:44:07.876810+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:44:59.122368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.876683+0000","last_change":"2023-10-11T09:44:06.889415+0000","last_active":"2023-10-11T09:44:07.876683+0000","last_peered":"2023-10-11T09:44:07.876683+0000","last_clean":"2023-10-11T09:44:07.876683+0000","last_became_active":"2023-10-11T09:44:06.889126+0000","last_became_peered":"2023-10-11T09:44:06.889126+0000","last_unstale":"2023-10-11T09:44:07.876683+0000","last_undegraded":"2023-10-11T09:44:07.876683+0000","last_fullsized":"2023-10-11T09:44:07.876683+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:43:50.221576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.863543+0000","last_change":"2023-10-11T09:44:06.420327+0000","last_active":"2023-10-11T09:44:06.863543+0000","last_peered":"2023-10-11T09:44:06.863543+0000","last_clean":"2023-10-11T09:44:06.863543+0000","last_became_active":"2023-10-11T09:44:06.420014+0000","last_became_peered":"2023-10-11T09:44:06.420014+0000","last_unstale":"2023-10-11T09:44:06.863543+0000","last_undegraded":"2023-10-11T09:44:06.863543+0000","last_fullsized":"2023-10-11T09:44:06.863543+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:35:54.243262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.896260+0000","last_change":"2023-10-11T09:44:06.888192+0000","last_active":"2023-10-11T09:44:06.896260+0000","last_peered":"2023-10-11T09:44:06.896260+0000","last_clean":"2023-10-11T09:44:06.896260+0000","last_became_active":"2023-10-11T09:44:06.887861+0000","last_became_peered":"2023-10-11T09:44:06.887861+0000","last_unstale":"2023-10-11T09:44:06.896260+0000","last_undegraded":"2023-10-11T09:44:06.896260+0000","last_fullsized":"2023-10-11T09:44:06.896260+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:28:37.019905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"60'3","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.876556+0000","last_change":"2023-10-11T09:44:06.417241+0000","last_active":"2023-10-11T09:44:07.876556+0000","last_peered":"2023-10-11T09:44:07.876556+0000","last_clean":"2023-10-11T09:44:07.876556+0000","last_became_active":"2023-10-11T09:44:06.416977+0000","last_became_peered":"2023-10-11T09:44:06.416977+0000","last_unstale":"2023-10-11T09:44:07.876556+0000","last_undegraded":"2023-10-11T09:44:07.876556+0000","last_fullsized":"2023-10-11T09:44:07.876556+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:15:59.977205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.876403+0000","last_change":"2023-10-11T09:44:06.888984+0000","last_active":"2023-10-11T09:44:07.876403+0000","last_peered":"2023-10-11T09:44:07.876403+0000","last_clean":"2023-10-11T09:44:07.876403+0000","last_became_active":"2023-10-11T09:44:06.888598+0000","last_became_peered":"2023-10-11T09:44:06.888598+0000","last_unstale":"2023-10-11T09:44:07.876403+0000","last_undegraded":"2023-10-11T09:44:07.876403+0000","last_fullsized":"2023-10-11T09:44:07.876403+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:58:17.041296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"60'6","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.874803+0000","last_change":"2023-10-11T09:44:06.424833+0000","last_active":"2023-10-11T09:44:07.874803+0000","last_peered":"2023-10-11T09:44:07.874803+0000","last_clean":"2023-10-11T09:44:07.874803+0000","last_became_active":"2023-10-11T09:44:06.424549+0000","last_became_peered":"2023-10-11T09:44:06.424549+0000","last_unstale":"2023-10-11T09:44:07.874803+0000","last_undegraded":"2023-10-11T09:44:07.874803+0000","last_fullsized":"2023-10-11T09:44:07.874803+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:02:06.108614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.901763+0000","last_change":"2023-10-11T09:44:06.889524+0000","last_active":"2023-10-11T09:44:06.901763+0000","last_peered":"2023-10-11T09:44:06.901763+0000","last_clean":"2023-10-11T09:44:06.901763+0000","last_became_active":"2023-10-11T09:44:06.889184+0000","last_became_peered":"2023-10-11T09:44:06.889184+0000","last_unstale":"2023-10-11T09:44:06.901763+0000","last_undegraded":"2023-10-11T09:44:06.901763+0000","last_fullsized":"2023-10-11T09:44:06.901763+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:00:38.019876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"60'2","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.887925+0000","last_change":"2023-10-11T09:44:06.420736+0000","last_active":"2023-10-11T09:44:06.887925+0000","last_peered":"2023-10-11T09:44:06.887925+0000","last_clean":"2023-10-11T09:44:06.887925+0000","last_became_active":"2023-10-11T09:44:06.420497+0000","last_became_peered":"2023-10-11T09:44:06.420497+0000","last_unstale":"2023-10-11T09:44:06.887925+0000","last_undegraded":"2023-10-11T09:44:06.887925+0000","last_fullsized":"2023-10-11T09:44:06.887925+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:53:48.608829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.876389+0000","last_change":"2023-10-11T09:44:06.880910+0000","last_active":"2023-10-11T09:44:07.876389+0000","last_peered":"2023-10-11T09:44:07.876389+0000","last_clean":"2023-10-11T09:44:07.876389+0000","last_became_active":"2023-10-11T09:44:06.879820+0000","last_became_peered":"2023-10-11T09:44:06.879820+0000","last_unstale":"2023-10-11T09:44:07.876389+0000","last_undegraded":"2023-10-11T09:44:07.876389+0000","last_fullsized":"2023-10-11T09:44:07.876389+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:50:53.336251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.872418+0000","last_change":"2023-10-11T09:44:06.415138+0000","last_active":"2023-10-11T09:44:06.872418+0000","last_peered":"2023-10-11T09:44:06.872418+0000","last_clean":"2023-10-11T09:44:06.872418+0000","last_became_active":"2023-10-11T09:44:06.414820+0000","last_became_peered":"2023-10-11T09:44:06.414820+0000","last_unstale":"2023-10-11T09:44:06.872418+0000","last_undegraded":"2023-10-11T09:44:06.872418+0000","last_fullsized":"2023-10-11T09:44:06.872418+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:15:48.644460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.874170+0000","last_change":"2023-10-11T09:44:06.873793+0000","last_active":"2023-10-11T09:44:07.874170+0000","last_peered":"2023-10-11T09:44:07.874170+0000","last_clean":"2023-10-11T09:44:07.874170+0000","last_became_active":"2023-10-11T09:44:06.873511+0000","last_became_peered":"2023-10-11T09:44:06.873511+0000","last_unstale":"2023-10-11T09:44:07.874170+0000","last_undegraded":"2023-10-11T09:44:07.874170+0000","last_fullsized":"2023-10-11T09:44:07.874170+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:16:06.882025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.878196+0000","last_change":"2023-10-11T09:44:06.418154+0000","last_active":"2023-10-11T09:44:06.878196+0000","last_peered":"2023-10-11T09:44:06.878196+0000","last_clean":"2023-10-11T09:44:06.878196+0000","last_became_active":"2023-10-11T09:44:06.416980+0000","last_became_peered":"2023-10-11T09:44:06.416980+0000","last_unstale":"2023-10-11T09:44:06.878196+0000","last_undegraded":"2023-10-11T09:44:06.878196+0000","last_fullsized":"2023-10-11T09:44:06.878196+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:23:44.206245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.876715+0000","last_change":"2023-10-11T09:44:06.887020+0000","last_active":"2023-10-11T09:44:07.876715+0000","last_peered":"2023-10-11T09:44:07.876715+0000","last_clean":"2023-10-11T09:44:07.876715+0000","last_became_active":"2023-10-11T09:44:06.886747+0000","last_became_peered":"2023-10-11T09:44:06.886747+0000","last_unstale":"2023-10-11T09:44:07.876715+0000","last_undegraded":"2023-10-11T09:44:07.876715+0000","last_fullsized":"2023-10-11T09:44:07.876715+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:05:38.172887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"60'8","reported_seq":19,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.876069+0000","last_change":"2023-10-11T09:44:06.424419+0000","last_active":"2023-10-11T09:44:07.876069+0000","last_peered":"2023-10-11T09:44:07.876069+0000","last_clean":"2023-10-11T09:44:07.876069+0000","last_became_active":"2023-10-11T09:44:06.424154+0000","last_became_peered":"2023-10-11T09:44:06.424154+0000","last_unstale":"2023-10-11T09:44:07.876069+0000","last_undegraded":"2023-10-11T09:44:07.876069+0000","last_fullsized":"2023-10-11T09:44:07.876069+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:35:06.314203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.901793+0000","last_change":"2023-10-11T09:44:06.889856+0000","last_active":"2023-10-11T09:44:06.901793+0000","last_peered":"2023-10-11T09:44:06.901793+0000","last_clean":"2023-10-11T09:44:06.901793+0000","last_became_active":"2023-10-11T09:44:06.889547+0000","last_became_peered":"2023-10-11T09:44:06.889547+0000","last_unstale":"2023-10-11T09:44:06.901793+0000","last_undegraded":"2023-10-11T09:44:06.901793+0000","last_fullsized":"2023-10-11T09:44:06.901793+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:28:43.416105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"60'2","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.897104+0000","last_change":"2023-10-11T09:44:06.416326+0000","last_active":"2023-10-11T09:44:06.897104+0000","last_peered":"2023-10-11T09:44:06.897104+0000","last_clean":"2023-10-11T09:44:06.897104+0000","last_became_active":"2023-10-11T09:44:06.415780+0000","last_became_peered":"2023-10-11T09:44:06.415780+0000","last_unstale":"2023-10-11T09:44:06.897104+0000","last_undegraded":"2023-10-11T09:44:06.897104+0000","last_fullsized":"2023-10-11T09:44:06.897104+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:25:14.688588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.901863+0000","last_change":"2023-10-11T09:44:06.891082+0000","last_active":"2023-10-11T09:44:06.901863+0000","last_peered":"2023-10-11T09:44:06.901863+0000","last_clean":"2023-10-11T09:44:06.901863+0000","last_became_active":"2023-10-11T09:44:06.890806+0000","last_became_peered":"2023-10-11T09:44:06.890806+0000","last_unstale":"2023-10-11T09:44:06.901863+0000","last_undegraded":"2023-10-11T09:44:06.901863+0000","last_fullsized":"2023-10-11T09:44:06.901863+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:21:29.593748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:08.411496+0000","last_change":"2023-10-11T09:44:06.423554+0000","last_active":"2023-10-11T09:44:08.411496+0000","last_peered":"2023-10-11T09:44:08.411496+0000","last_clean":"2023-10-11T09:44:08.411496+0000","last_became_active":"2023-10-11T09:44:06.423239+0000","last_became_peered":"2023-10-11T09:44:06.423239+0000","last_unstale":"2023-10-11T09:44:08.411496+0000","last_undegraded":"2023-10-11T09:44:08.411496+0000","last_fullsized":"2023-10-11T09:44:08.411496+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:26:35.518484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.896239+0000","last_change":"2023-10-11T09:44:06.887802+0000","last_active":"2023-10-11T09:44:06.896239+0000","last_peered":"2023-10-11T09:44:06.896239+0000","last_clean":"2023-10-11T09:44:06.896239+0000","last_became_active":"2023-10-11T09:44:06.887540+0000","last_became_peered":"2023-10-11T09:44:06.887540+0000","last_unstale":"2023-10-11T09:44:06.896239+0000","last_undegraded":"2023-10-11T09:44:06.896239+0000","last_fullsized":"2023-10-11T09:44:06.896239+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:45:00.136041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.872379+0000","last_change":"2023-10-11T09:44:06.416939+0000","last_active":"2023-10-11T09:44:06.872379+0000","last_peered":"2023-10-11T09:44:06.872379+0000","last_clean":"2023-10-11T09:44:06.872379+0000","last_became_active":"2023-10-11T09:44:06.416680+0000","last_became_peered":"2023-10-11T09:44:06.416680+0000","last_unstale":"2023-10-11T09:44:06.872379+0000","last_undegraded":"2023-10-11T09:44:06.872379+0000","last_fullsized":"2023-10-11T09:44:06.872379+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:25:37.480182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:08.411336+0000","last_change":"2023-10-11T09:44:06.883171+0000","last_active":"2023-10-11T09:44:08.411336+0000","last_peered":"2023-10-11T09:44:08.411336+0000","last_clean":"2023-10-11T09:44:08.411336+0000","last_became_active":"2023-10-11T09:44:06.882388+0000","last_became_peered":"2023-10-11T09:44:06.882388+0000","last_unstale":"2023-10-11T09:44:08.411336+0000","last_undegraded":"2023-10-11T09:44:08.411336+0000","last_fullsized":"2023-10-11T09:44:08.411336+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:01:02.059081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.877036+0000","last_change":"2023-10-11T09:44:06.422820+0000","last_active":"2023-10-11T09:44:06.877036+0000","last_peered":"2023-10-11T09:44:06.877036+0000","last_clean":"2023-10-11T09:44:06.877036+0000","last_became_active":"2023-10-11T09:44:06.422574+0000","last_became_peered":"2023-10-11T09:44:06.422574+0000","last_unstale":"2023-10-11T09:44:06.877036+0000","last_undegraded":"2023-10-11T09:44:06.877036+0000","last_fullsized":"2023-10-11T09:44:06.877036+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:04:55.976065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.895896+0000","last_change":"2023-10-11T09:44:06.887573+0000","last_active":"2023-10-11T09:44:06.895896+0000","last_peered":"2023-10-11T09:44:06.895896+0000","last_clean":"2023-10-11T09:44:06.895896+0000","last_became_active":"2023-10-11T09:44:06.887284+0000","last_became_peered":"2023-10-11T09:44:06.887284+0000","last_unstale":"2023-10-11T09:44:06.895896+0000","last_undegraded":"2023-10-11T09:44:06.895896+0000","last_fullsized":"2023-10-11T09:44:06.895896+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:47:18.756116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.883564+0000","last_change":"2023-10-11T09:44:06.422220+0000","last_active":"2023-10-11T09:44:06.883564+0000","last_peered":"2023-10-11T09:44:06.883564+0000","last_clean":"2023-10-11T09:44:06.883564+0000","last_became_active":"2023-10-11T09:44:06.421940+0000","last_became_peered":"2023-10-11T09:44:06.421940+0000","last_unstale":"2023-10-11T09:44:06.883564+0000","last_undegraded":"2023-10-11T09:44:06.883564+0000","last_fullsized":"2023-10-11T09:44:06.883564+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:45:06.217550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.876566+0000","last_change":"2023-10-11T09:44:06.887452+0000","last_active":"2023-10-11T09:44:07.876566+0000","last_peered":"2023-10-11T09:44:07.876566+0000","last_clean":"2023-10-11T09:44:07.876566+0000","last_became_active":"2023-10-11T09:44:06.886790+0000","last_became_peered":"2023-10-11T09:44:06.886790+0000","last_unstale":"2023-10-11T09:44:07.876566+0000","last_undegraded":"2023-10-11T09:44:07.876566+0000","last_fullsized":"2023-10-11T09:44:07.876566+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:54:24.389470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.874671+0000","last_change":"2023-10-11T09:44:06.419796+0000","last_active":"2023-10-11T09:44:07.874671+0000","last_peered":"2023-10-11T09:44:07.874671+0000","last_clean":"2023-10-11T09:44:07.874671+0000","last_became_active":"2023-10-11T09:44:06.419509+0000","last_became_peered":"2023-10-11T09:44:06.419509+0000","last_unstale":"2023-10-11T09:44:07.874671+0000","last_undegraded":"2023-10-11T09:44:07.874671+0000","last_fullsized":"2023-10-11T09:44:07.874671+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:06:53.008484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.893150+0000","last_change":"2023-10-11T09:44:06.885086+0000","last_active":"2023-10-11T09:44:06.893150+0000","last_peered":"2023-10-11T09:44:06.893150+0000","last_clean":"2023-10-11T09:44:06.893150+0000","last_became_active":"2023-10-11T09:44:06.884852+0000","last_became_peered":"2023-10-11T09:44:06.884852+0000","last_unstale":"2023-10-11T09:44:06.893150+0000","last_undegraded":"2023-10-11T09:44:06.893150+0000","last_fullsized":"2023-10-11T09:44:06.893150+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:27:02.672754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.883669+0000","last_change":"2023-10-11T09:44:06.428888+0000","last_active":"2023-10-11T09:44:06.883669+0000","last_peered":"2023-10-11T09:44:06.883669+0000","last_clean":"2023-10-11T09:44:06.883669+0000","last_became_active":"2023-10-11T09:44:06.428556+0000","last_became_peered":"2023-10-11T09:44:06.428556+0000","last_unstale":"2023-10-11T09:44:06.883669+0000","last_undegraded":"2023-10-11T09:44:06.883669+0000","last_fullsized":"2023-10-11T09:44:06.883669+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:15:46.293341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.885924+0000","last_change":"2023-10-11T09:44:06.880433+0000","last_active":"2023-10-11T09:44:06.885924+0000","last_peered":"2023-10-11T09:44:06.885924+0000","last_clean":"2023-10-11T09:44:06.885924+0000","last_became_active":"2023-10-11T09:44:06.879802+0000","last_became_peered":"2023-10-11T09:44:06.879802+0000","last_unstale":"2023-10-11T09:44:06.885924+0000","last_undegraded":"2023-10-11T09:44:06.885924+0000","last_fullsized":"2023-10-11T09:44:06.885924+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T21:14:29.347600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.862599+0000","last_change":"2023-10-11T09:44:06.425553+0000","last_active":"2023-10-11T09:44:06.862599+0000","last_peered":"2023-10-11T09:44:06.862599+0000","last_clean":"2023-10-11T09:44:06.862599+0000","last_became_active":"2023-10-11T09:44:06.425289+0000","last_became_peered":"2023-10-11T09:44:06.425289+0000","last_unstale":"2023-10-11T09:44:06.862599+0000","last_undegraded":"2023-10-11T09:44:06.862599+0000","last_fullsized":"2023-10-11T09:44:06.862599+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:23:44.206245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.882384+0000","last_change":"2023-10-11T09:44:06.871913+0000","last_active":"2023-10-11T09:44:06.882384+0000","last_peered":"2023-10-11T09:44:06.882384+0000","last_clean":"2023-10-11T09:44:06.882384+0000","last_became_active":"2023-10-11T09:44:06.871627+0000","last_became_peered":"2023-10-11T09:44:06.871627+0000","last_unstale":"2023-10-11T09:44:06.882384+0000","last_undegraded":"2023-10-11T09:44:06.882384+0000","last_fullsized":"2023-10-11T09:44:06.882384+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:50:11.352357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"60'1","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.902027+0000","last_change":"2023-10-11T09:44:06.422188+0000","last_active":"2023-10-11T09:44:06.902027+0000","last_peered":"2023-10-11T09:44:06.902027+0000","last_clean":"2023-10-11T09:44:06.902027+0000","last_became_active":"2023-10-11T09:44:06.421885+0000","last_became_peered":"2023-10-11T09:44:06.421885+0000","last_unstale":"2023-10-11T09:44:06.902027+0000","last_undegraded":"2023-10-11T09:44:06.902027+0000","last_fullsized":"2023-10-11T09:44:06.902027+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:16:05.868352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":41,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.410771+0000","last_change":"2023-10-11T09:40:36.151839+0000","last_active":"2023-10-11T09:44:06.410771+0000","last_peered":"2023-10-11T09:44:06.410771+0000","last_clean":"2023-10-11T09:44:06.410771+0000","last_became_active":"2023-10-11T09:40:36.151179+0000","last_became_peered":"2023-10-11T09:40:36.151179+0000","last_unstale":"2023-10-11T09:44:06.410771+0000","last_undegraded":"2023-10-11T09:44:06.410771+0000","last_fullsized":"2023-10-11T09:44:06.410771+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.876303+0000","last_change":"2023-10-11T09:44:06.889607+0000","last_active":"2023-10-11T09:44:07.876303+0000","last_peered":"2023-10-11T09:44:07.876303+0000","last_clean":"2023-10-11T09:44:07.876303+0000","last_became_active":"2023-10-11T09:44:06.888974+0000","last_became_peered":"2023-10-11T09:44:06.888974+0000","last_unstale":"2023-10-11T09:44:07.876303+0000","last_undegraded":"2023-10-11T09:44:07.876303+0000","last_fullsized":"2023-10-11T09:44:07.876303+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:25:40.535782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:08.411240+0000","last_change":"2023-10-11T09:44:06.419836+0000","last_active":"2023-10-11T09:44:08.411240+0000","last_peered":"2023-10-11T09:44:08.411240+0000","last_clean":"2023-10-11T09:44:08.411240+0000","last_became_active":"2023-10-11T09:44:06.419574+0000","last_became_peered":"2023-10-11T09:44:06.419574+0000","last_unstale":"2023-10-11T09:44:08.411240+0000","last_undegraded":"2023-10-11T09:44:08.411240+0000","last_fullsized":"2023-10-11T09:44:08.411240+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:43:49.207902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":43,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.410079+0000","last_change":"2023-10-11T09:40:36.150224+0000","last_active":"2023-10-11T09:44:06.410079+0000","last_peered":"2023-10-11T09:44:06.410079+0000","last_clean":"2023-10-11T09:44:06.410079+0000","last_became_active":"2023-10-11T09:40:36.149919+0000","last_became_peered":"2023-10-11T09:40:36.149919+0000","last_unstale":"2023-10-11T09:44:06.410079+0000","last_undegraded":"2023-10-11T09:44:06.410079+0000","last_fullsized":"2023-10-11T09:44:06.410079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.876642+0000","last_change":"2023-10-11T09:44:06.878178+0000","last_active":"2023-10-11T09:44:07.876642+0000","last_peered":"2023-10-11T09:44:07.876642+0000","last_clean":"2023-10-11T09:44:07.876642+0000","last_became_active":"2023-10-11T09:44:06.877875+0000","last_became_peered":"2023-10-11T09:44:06.877875+0000","last_unstale":"2023-10-11T09:44:07.876642+0000","last_undegraded":"2023-10-11T09:44:07.876642+0000","last_fullsized":"2023-10-11T09:44:07.876642+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:26:36.532157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"60'1","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.888757+0000","last_change":"2023-10-11T09:44:06.420099+0000","last_active":"2023-10-11T09:44:06.888757+0000","last_peered":"2023-10-11T09:44:06.888757+0000","last_clean":"2023-10-11T09:44:06.888757+0000","last_became_active":"2023-10-11T09:44:06.419770+0000","last_became_peered":"2023-10-11T09:44:06.419770+0000","last_unstale":"2023-10-11T09:44:06.888757+0000","last_undegraded":"2023-10-11T09:44:06.888757+0000","last_fullsized":"2023-10-11T09:44:06.888757+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:52:00.935219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":53,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.866112+0000","last_change":"2023-10-11T09:40:36.162174+0000","last_active":"2023-10-11T09:44:06.866112+0000","last_peered":"2023-10-11T09:44:06.866112+0000","last_clean":"2023-10-11T09:44:06.866112+0000","last_became_active":"2023-10-11T09:40:36.161710+0000","last_became_peered":"2023-10-11T09:40:36.161710+0000","last_unstale":"2023-10-11T09:44:06.866112+0000","last_undegraded":"2023-10-11T09:44:06.866112+0000","last_fullsized":"2023-10-11T09:44:06.866112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:57:03.630878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.901839+0000","last_change":"2023-10-11T09:44:06.889681+0000","last_active":"2023-10-11T09:44:06.901839+0000","last_peered":"2023-10-11T09:44:06.901839+0000","last_clean":"2023-10-11T09:44:06.901839+0000","last_became_active":"2023-10-11T09:44:06.889173+0000","last_became_peered":"2023-10-11T09:44:06.889173+0000","last_unstale":"2023-10-11T09:44:06.901839+0000","last_undegraded":"2023-10-11T09:44:06.901839+0000","last_fullsized":"2023-10-11T09:44:06.901839+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:37:20.675117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"60'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:08.411403+0000","last_change":"2023-10-11T09:44:06.421620+0000","last_active":"2023-10-11T09:44:08.411403+0000","last_peered":"2023-10-11T09:44:08.411403+0000","last_clean":"2023-10-11T09:44:08.411403+0000","last_became_active":"2023-10-11T09:44:06.421294+0000","last_became_peered":"2023-10-11T09:44:06.421294+0000","last_unstale":"2023-10-11T09:44:08.411403+0000","last_undegraded":"2023-10-11T09:44:08.411403+0000","last_fullsized":"2023-10-11T09:44:08.411403+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:25:39.522109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.407336+0000","last_change":"2023-10-11T09:40:36.160620+0000","last_active":"2023-10-11T09:44:06.407336+0000","last_peered":"2023-10-11T09:44:06.407336+0000","last_clean":"2023-10-11T09:44:06.407336+0000","last_became_active":"2023-10-11T09:40:36.160171+0000","last_became_peered":"2023-10-11T09:40:36.160171+0000","last_unstale":"2023-10-11T09:44:06.407336+0000","last_undegraded":"2023-10-11T09:44:06.407336+0000","last_fullsized":"2023-10-11T09:44:06.407336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:08.410625+0000","last_change":"2023-10-11T09:44:06.881819+0000","last_active":"2023-10-11T09:44:08.410625+0000","last_peered":"2023-10-11T09:44:08.410625+0000","last_clean":"2023-10-11T09:44:08.410625+0000","last_became_active":"2023-10-11T09:44:06.881540+0000","last_became_peered":"2023-10-11T09:44:06.881540+0000","last_unstale":"2023-10-11T09:44:08.410625+0000","last_undegraded":"2023-10-11T09:44:08.410625+0000","last_fullsized":"2023-10-11T09:44:08.410625+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:08:57.620867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.876750+0000","last_change":"2023-10-11T09:44:06.420099+0000","last_active":"2023-10-11T09:44:06.876750+0000","last_peered":"2023-10-11T09:44:06.876750+0000","last_clean":"2023-10-11T09:44:06.876750+0000","last_became_active":"2023-10-11T09:44:06.419821+0000","last_became_peered":"2023-10-11T09:44:06.419821+0000","last_unstale":"2023-10-11T09:44:06.876750+0000","last_undegraded":"2023-10-11T09:44:06.876750+0000","last_fullsized":"2023-10-11T09:44:06.876750+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:42:37.454595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":43,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.418395+0000","last_change":"2023-10-11T09:40:36.151710+0000","last_active":"2023-10-11T09:44:06.418395+0000","last_peered":"2023-10-11T09:44:06.418395+0000","last_clean":"2023-10-11T09:44:06.418395+0000","last_became_active":"2023-10-11T09:40:36.151219+0000","last_became_peered":"2023-10-11T09:40:36.151219+0000","last_unstale":"2023-10-11T09:44:06.418395+0000","last_undegraded":"2023-10-11T09:44:06.418395+0000","last_fullsized":"2023-10-11T09:44:06.418395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.901825+0000","last_change":"2023-10-11T09:44:06.890156+0000","last_active":"2023-10-11T09:44:06.901825+0000","last_peered":"2023-10-11T09:44:06.901825+0000","last_clean":"2023-10-11T09:44:06.901825+0000","last_became_active":"2023-10-11T09:44:06.889898+0000","last_became_peered":"2023-10-11T09:44:06.889898+0000","last_unstale":"2023-10-11T09:44:06.901825+0000","last_undegraded":"2023-10-11T09:44:06.901825+0000","last_fullsized":"2023-10-11T09:44:06.901825+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:12:30.534498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"60'2","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.889245+0000","last_change":"2023-10-11T09:44:06.424956+0000","last_active":"2023-10-11T09:44:06.889245+0000","last_peered":"2023-10-11T09:44:06.889245+0000","last_clean":"2023-10-11T09:44:06.889245+0000","last_became_active":"2023-10-11T09:44:06.424673+0000","last_became_peered":"2023-10-11T09:44:06.424673+0000","last_unstale":"2023-10-11T09:44:06.889245+0000","last_undegraded":"2023-10-11T09:44:06.889245+0000","last_fullsized":"2023-10-11T09:44:06.889245+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:02:06.108614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":44,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.870195+0000","last_change":"2023-10-11T09:40:36.216248+0000","last_active":"2023-10-11T09:44:06.870195+0000","last_peered":"2023-10-11T09:44:06.870195+0000","last_clean":"2023-10-11T09:44:06.870195+0000","last_became_active":"2023-10-11T09:40:36.215890+0000","last_became_peered":"2023-10-11T09:40:36.215890+0000","last_unstale":"2023-10-11T09:44:06.870195+0000","last_undegraded":"2023-10-11T09:44:06.870195+0000","last_fullsized":"2023-10-11T09:44:06.870195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:08.411271+0000","last_change":"2023-10-11T09:44:06.885598+0000","last_active":"2023-10-11T09:44:08.411271+0000","last_peered":"2023-10-11T09:44:08.411271+0000","last_clean":"2023-10-11T09:44:08.411271+0000","last_became_active":"2023-10-11T09:44:06.885294+0000","last_became_peered":"2023-10-11T09:44:06.885294+0000","last_unstale":"2023-10-11T09:44:08.411271+0000","last_undegraded":"2023-10-11T09:44:08.411271+0000","last_fullsized":"2023-10-11T09:44:08.411271+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:39:13.083637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.884838+0000","last_change":"2023-10-11T09:44:06.420620+0000","last_active":"2023-10-11T09:44:06.884838+0000","last_peered":"2023-10-11T09:44:06.884838+0000","last_clean":"2023-10-11T09:44:06.884838+0000","last_became_active":"2023-10-11T09:44:06.420243+0000","last_became_peered":"2023-10-11T09:44:06.420243+0000","last_unstale":"2023-10-11T09:44:06.884838+0000","last_undegraded":"2023-10-11T09:44:06.884838+0000","last_fullsized":"2023-10-11T09:44:06.884838+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:23:23.149495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":53,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.864353+0000","last_change":"2023-10-11T09:40:36.149739+0000","last_active":"2023-10-11T09:44:06.864353+0000","last_peered":"2023-10-11T09:44:06.864353+0000","last_clean":"2023-10-11T09:44:06.864353+0000","last_became_active":"2023-10-11T09:40:36.149419+0000","last_became_peered":"2023-10-11T09:40:36.149419+0000","last_unstale":"2023-10-11T09:44:06.864353+0000","last_undegraded":"2023-10-11T09:44:06.864353+0000","last_fullsized":"2023-10-11T09:44:06.864353+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:54:50.640362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.889950+0000","last_change":"2023-10-11T09:44:06.882947+0000","last_active":"2023-10-11T09:44:06.889950+0000","last_peered":"2023-10-11T09:44:06.889950+0000","last_clean":"2023-10-11T09:44:06.889950+0000","last_became_active":"2023-10-11T09:44:06.882705+0000","last_became_peered":"2023-10-11T09:44:06.882705+0000","last_unstale":"2023-10-11T09:44:06.889950+0000","last_undegraded":"2023-10-11T09:44:06.889950+0000","last_fullsized":"2023-10-11T09:44:06.889950+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:57:01.167316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.874816+0000","last_change":"2023-10-11T09:44:06.423372+0000","last_active":"2023-10-11T09:44:06.874816+0000","last_peered":"2023-10-11T09:44:06.874816+0000","last_clean":"2023-10-11T09:44:06.874816+0000","last_became_active":"2023-10-11T09:44:06.423093+0000","last_became_peered":"2023-10-11T09:44:06.423093+0000","last_unstale":"2023-10-11T09:44:06.874816+0000","last_undegraded":"2023-10-11T09:44:06.874816+0000","last_fullsized":"2023-10-11T09:44:06.874816+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:49:28.537540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":43,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.418441+0000","last_change":"2023-10-11T09:40:36.148773+0000","last_active":"2023-10-11T09:44:06.418441+0000","last_peered":"2023-10-11T09:44:06.418441+0000","last_clean":"2023-10-11T09:44:06.418441+0000","last_became_active":"2023-10-11T09:40:36.148489+0000","last_became_peered":"2023-10-11T09:40:36.148489+0000","last_unstale":"2023-10-11T09:44:06.418441+0000","last_undegraded":"2023-10-11T09:44:06.418441+0000","last_fullsized":"2023-10-11T09:44:06.418441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":206,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.864461+0000","last_change":"2023-10-11T09:40:29.874372+0000","last_active":"2023-10-11T09:44:06.864461+0000","last_peered":"2023-10-11T09:44:06.864461+0000","last_clean":"2023-10-11T09:44:06.864461+0000","last_became_active":"2023-10-11T09:40:29.873621+0000","last_became_peered":"2023-10-11T09:40:29.873621+0000","last_unstale":"2023-10-11T09:44:06.864461+0000","last_undegraded":"2023-10-11T09:44:06.864461+0000","last_fullsized":"2023-10-11T09:44:06.864461+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_clean_scrub_stamp":"2023-10-11T09:40:28.742276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:22:07.985311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574896,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1288,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1318912,"data_stored":917795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:43:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.135,"15min":1.1850000000000001},"min":{"1min":0.61899999999999999,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.367,"5min":1.903,"15min":1.903},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.167,"5min":1.149,"15min":1.1579999999999999},"min":{"1min":0.84399999999999997,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.5329999999999999,"5min":1.696,"15min":1.696},"last":0.95199999999999996}]},{"osd":1,"last update":"Wed Oct 11 09:43:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.1279999999999999,"15min":1.143},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.484,"5min":2.25,"15min":2.25},"last":1.097},{"interface":"front","average":{"1min":1.105,"5min":1.139,"15min":1.147},"min":{"1min":0.83599999999999997,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.3320000000000001,"5min":1.696,"15min":1.696},"last":0.872}]},{"osd":2,"last update":"Wed Oct 11 09:43:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.1020000000000001,"15min":1.1279999999999999},"min":{"1min":0.67100000000000004,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.4610000000000001,"5min":1.873,"15min":1.873},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.139,"15min":1.153},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.548,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.2230000000000001}]},{"osd":3,"last update":"Wed Oct 11 09:43:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.244,"15min":1.272},"min":{"1min":0.92400000000000004,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.504,"5min":1.724,"15min":1.724},"last":1.123},{"interface":"front","average":{"1min":1.296,"5min":1.2929999999999999,"15min":1.3109999999999999},"min":{"1min":1.093,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.516,"5min":2.1579999999999999,"15min":2.1579999999999999},"last":1.3260000000000001}]},{"osd":4,"last update":"Wed Oct 11 09:43:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.161,"15min":1.198},"min":{"1min":0.74199999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.329,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.101,"5min":1.228,"15min":1.2669999999999999},"min":{"1min":0.81999999999999995,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.393,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":1.29}]},{"osd":5,"last update":"Wed Oct 11 09:43:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.268,"15min":1.2929999999999999},"min":{"1min":0.85999999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.3540000000000001,"5min":1.744,"15min":1.744},"last":1.425},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.331,"15min":1.385},"min":{"1min":0.93100000000000005,"5min":0.872,"15min":0.872},"max":{"1min":1.4319999999999999,"5min":3.0750000000000002,"15min":3.0750000000000002},"last":1.1819999999999999}]},{"osd":6,"last update":"Wed Oct 11 09:43:37 2023","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.427,"15min":1.4690000000000001},"min":{"1min":1.173,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":1.52,"5min":3.137,"15min":3.137},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.329,"5min":1.4019999999999999,"15min":1.4279999999999999},"min":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":1.6879999999999999,"5min":1.839,"15min":1.839},"last":1.3899999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574895,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":700,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":716800,"data_stored":327267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.0069999999999999,"15min":0.998},"min":{"1min":0.77800000000000002,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.466,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.034,"15min":1.052},"min":{"1min":0.68000000000000005,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4330000000000001,"5min":1.524,"15min":1.524},"last":1.028}]},{"osd":1,"last update":"Wed Oct 11 09:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.1890000000000001,"15min":1.1970000000000001},"min":{"1min":0.94499999999999995,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.484,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.077},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.2410000000000001,"15min":1.264},"min":{"1min":1.0429999999999999,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.4630000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.2669999999999999}]},{"osd":2,"last update":"Wed Oct 11 09:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.1399999999999999,"15min":1.1599999999999999},"min":{"1min":0.76900000000000002,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.419,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.123,"5min":1.22,"15min":1.2629999999999999},"min":{"1min":0.85499999999999998,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.6419999999999999,"5min":2.605,"15min":2.605},"last":1.107}]},{"osd":3,"last update":"Wed Oct 11 09:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.1679999999999999,"15min":1.1919999999999999},"min":{"1min":0.89800000000000002,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.5589999999999999,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.2729999999999999,"15min":1.302},"min":{"1min":0.84599999999999997,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.7509999999999999,"5min":3.0819999999999999,"15min":3.0819999999999999},"last":1.292}]},{"osd":4,"last update":"Wed Oct 11 09:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.2809999999999999,"15min":1.3180000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.514,"5min":1.978,"15min":1.978},"last":1.198},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3580000000000001,"15min":1.387},"min":{"1min":0.72399999999999998,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.141,"5min":2.6920000000000002,"15min":2.6920000000000002},"last":1.1779999999999999}]},{"osd":5,"last update":"Wed Oct 11 09:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.2709999999999999,"15min":1.2849999999999999},"min":{"1min":0.76800000000000002,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.6859999999999999,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3999999999999999,"15min":1.4239999999999999},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.823,"5min":2.7290000000000001,"15min":2.7290000000000001},"last":1.329}]},{"osd":7,"last update":"Wed Oct 11 09:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.4159999999999999,"15min":1.4199999999999999},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.895,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.474,"5min":1.476,"15min":1.4830000000000001},"min":{"1min":1.0860000000000001,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.859,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.232}]}]},{"osd":5,"up_from":13,"seq":55834574895,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":724992,"data_stored":327919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:43:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.026,"15min":0.98899999999999999},"min":{"1min":0.86399999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.075,"5min":0.99299999999999999,"15min":0.97099999999999997},"min":{"1min":0.80500000000000005,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.327,"5min":1.637,"15min":1.637},"last":1.22}]},{"osd":1,"last update":"Wed Oct 11 09:43:34 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":0.97799999999999998,"15min":0.94699999999999995},"min":{"1min":0.78700000000000003,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.363,"5min":1.444,"15min":1.444},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.0629999999999999,"15min":1.0589999999999999},"min":{"1min":0.86099999999999999,"5min":0.437,"15min":0.437},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":1.2929999999999999}]},{"osd":2,"last update":"Wed Oct 11 09:43:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.204,"15min":1.2030000000000001},"min":{"1min":0.79600000000000004,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.4890000000000001,"5min":1.831,"15min":1.831},"last":1.173},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.2490000000000001,"15min":1.2450000000000001},"min":{"1min":0.95799999999999996,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.5329999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.248}]},{"osd":3,"last update":"Wed Oct 11 09:43:34 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.1339999999999999,"15min":1.143},"min":{"1min":0.69199999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.506,"5min":1.921,"15min":1.921},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.206,"5min":1.163,"15min":1.165},"min":{"1min":0.93400000000000005,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.4390000000000001,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.0369999999999999}]},{"osd":4,"last update":"Wed Oct 11 09:43:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1699999999999999,"15min":1.1850000000000001},"min":{"1min":0.82399999999999995,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.603,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.1830000000000001,"15min":1.2130000000000001},"min":{"1min":0.73399999999999999,"5min":0.62,"15min":0.62},"max":{"1min":1.391,"5min":1.988,"15min":1.988},"last":1.091}]},{"osd":6,"last update":"Wed Oct 11 09:43:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.254,"15min":1.2849999999999999},"min":{"1min":0.86699999999999999,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.54,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"min":{"1min":1,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.6890000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":1.2050000000000001}]},{"osd":7,"last update":"Wed Oct 11 09:43:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.3740000000000001,"15min":1.425},"min":{"1min":0.90000000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.635,"5min":3.6120000000000001,"15min":3.6120000000000001},"last":1.429},{"interface":"front","average":{"1min":1.397,"5min":1.448,"15min":1.484},"min":{"1min":0.99399999999999999,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.6419999999999999,"5min":3.2109999999999999,"15min":3.2109999999999999},"last":1.528}]}]},{"osd":4,"up_from":13,"seq":55834574895,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":724,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":741376,"data_stored":335484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:43:36 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.0800000000000001,"15min":1.0700000000000001},"min":{"1min":0.70199999999999996,"5min":0.629,"15min":0.629},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1020000000000001,"15min":1.1060000000000001},"min":{"1min":0.70199999999999996,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.466,"5min":2.1160000000000001,"15min":2.1160000000000001},"last":0.65100000000000002}]},{"osd":1,"last update":"Wed Oct 11 09:43:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.0960000000000001,"15min":1.1020000000000001},"min":{"1min":0.73399999999999999,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.238,"5min":1.173,"15min":1.1799999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.635,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.1879999999999999}]},{"osd":2,"last update":"Wed Oct 11 09:43:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.2110000000000001,"15min":1.2},"min":{"1min":0.77700000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.512,"5min":1.919,"15min":1.919},"last":0.876},{"interface":"front","average":{"1min":1.242,"5min":1.2390000000000001,"15min":1.2350000000000001},"min":{"1min":0.86099999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.5409999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.77800000000000002}]},{"osd":3,"last update":"Wed Oct 11 09:43:36 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.153,"15min":1.161},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.3919999999999999,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.296},{"interface":"front","average":{"1min":1.157,"5min":1.284,"15min":1.339},"min":{"1min":0.81399999999999995,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.4419999999999999,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":1.159}]},{"osd":5,"last update":"Wed Oct 11 09:43:36 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.2070000000000001,"15min":1.226},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.577,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1910000000000001,"15min":1.2270000000000001},"min":{"1min":0.78800000000000003,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.5269999999999999,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":1.391}]},{"osd":6,"last update":"Wed Oct 11 09:43:36 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.2709999999999999,"15min":1.2969999999999999},"min":{"1min":0.85999999999999999,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.5109999999999999,"5min":1.8,"15min":1.8},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.3,"5min":1.337,"15min":1.3500000000000001},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.667,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.5209999999999999}]},{"osd":7,"last update":"Wed Oct 11 09:43:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.3320000000000001,"15min":1.365},"min":{"1min":0.93100000000000005,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.4690000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.4570000000000001,"15min":1.496},"min":{"1min":0.995,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.5900000000000001,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":1.2709999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":724992,"data_stored":327323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.83799999999999997,"15min":0.88500000000000001},"min":{"1min":0.43099999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.262,"5min":1.462,"15min":1.462},"last":0.75},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86299999999999999,"15min":0.88600000000000001},"min":{"1min":0.502,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.256,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.82399999999999995}]},{"osd":1,"last update":"Wed Oct 11 09:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.82399999999999995,"15min":0.82799999999999996},"min":{"1min":0.53600000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":1.0700000000000001},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.95299999999999996,"15min":0.96199999999999997},"min":{"1min":0.57599999999999996,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.294,"5min":1.341,"15min":1.341},"last":1.121}]},{"osd":2,"last update":"Wed Oct 11 09:43:36 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":0.98299999999999998,"15min":0.996},"min":{"1min":0.70199999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.383,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.171},{"interface":"front","average":{"1min":1.036,"5min":1.1120000000000001,"15min":1.145},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.3140000000000001,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":0.90800000000000003}]},{"osd":4,"last update":"Wed Oct 11 09:43:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.2050000000000001,"15min":1.2470000000000001},"min":{"1min":0.79000000000000004,"5min":0.754,"15min":0.754},"max":{"1min":1.472,"5min":2.395,"15min":2.395},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.198,"5min":1.2909999999999999,"15min":1.341},"min":{"1min":0.98599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.577,"5min":3.1579999999999999,"15min":3.1579999999999999},"last":1.411}]},{"osd":5,"last update":"Wed Oct 11 09:43:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.3180000000000001,"15min":1.3680000000000001},"min":{"1min":0.83099999999999996,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.7430000000000001,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":1.278},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.3999999999999999,"15min":1.478},"min":{"1min":0.92300000000000004,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.395,"5min":3.8050000000000002,"15min":3.8050000000000002},"last":1.2130000000000001}]},{"osd":6,"last update":"Wed Oct 11 09:43:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.286,"15min":1.3260000000000001},"min":{"1min":0.91300000000000003,"5min":0.75,"15min":0.75},"max":{"1min":1.4370000000000001,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.663},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.3839999999999999,"15min":1.4359999999999999},"min":{"1min":1.0109999999999999,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":1.663,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":1.349}]},{"osd":7,"last update":"Wed Oct 11 09:43:36 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.4470000000000001,"15min":1.5089999999999999},"min":{"1min":0.89600000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.478,"5min":3.7490000000000001,"15min":3.7490000000000001},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.377,"15min":1.4239999999999999},"min":{"1min":0.97399999999999998,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.5349999999999999,"5min":2.625,"15min":2.625},"last":1.4570000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27672,"kb_used_data":720,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":737280,"data_stored":333976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.82099999999999995,"15min":0.83499999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.2370000000000001,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.93700000000000006,"15min":0.98999999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1779999999999999,"5min":2.8919999999999999,"15min":2.8919999999999999},"last":0.82099999999999995}]},{"osd":1,"last update":"Wed Oct 11 09:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":1.0169999999999999,"15min":1.075},"min":{"1min":0.57299999999999995,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.2649999999999999,"5min":3.7650000000000001,"15min":3.7650000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.97899999999999998,"15min":1.0029999999999999},"min":{"1min":0.48999999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.413,"5min":2.5339999999999998,"15min":2.5339999999999998},"last":1.302}]},{"osd":3,"last update":"Wed Oct 11 09:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.083,"15min":1.139},"min":{"1min":0.63300000000000001,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.381,"5min":2.403,"15min":2.403},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.0680000000000001,"15min":1.1000000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.3660000000000001,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.216}]},{"osd":4,"last update":"Wed Oct 11 09:43:33 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.22,"15min":1.234},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.502,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.395,"15min":1.4490000000000001},"min":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":1.651,"5min":2.6859999999999999,"15min":2.6859999999999999},"last":1.0880000000000001}]},{"osd":5,"last update":"Wed Oct 11 09:43:33 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.3400000000000001,"15min":1.3859999999999999},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.6799999999999999,"5min":3.044,"15min":3.044},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.202,"5min":1.306,"15min":1.355},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.5649999999999999,"5min":3.5790000000000002,"15min":3.5790000000000002},"last":1.512}]},{"osd":6,"last update":"Wed Oct 11 09:43:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.3879999999999999,"15min":1.462},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":1.8049999999999999,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.022},{"interface":"front","average":{"1min":1.262,"5min":1.353,"15min":1.3819999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.742,"5min":2.839,"15min":2.839},"last":1.484}]},{"osd":7,"last update":"Wed Oct 11 09:43:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.399,"15min":1.4590000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.9079999999999999,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.49,"15min":1.552},"min":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":1.9570000000000001,"5min":2.988,"15min":2.988},"last":1.363}]}]},{"osd":1,"up_from":13,"seq":55834574895,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":720896,"data_stored":327357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:43:34 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.79700000000000004,"15min":0.80800000000000005},"min":{"1min":0.53900000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.236,"5min":1.27,"15min":1.27},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.84999999999999998,"15min":0.85099999999999998},"min":{"1min":0.54300000000000004,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.105,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.995}]},{"osd":2,"last update":"Wed Oct 11 09:43:34 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.86799999999999999,"15min":0.88500000000000001},"min":{"1min":0.629,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.127,"5min":1.292,"15min":1.292},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.98399999999999999,"15min":1.022},"min":{"1min":0.60799999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.403,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.89100000000000001}]},{"osd":3,"last update":"Wed Oct 11 09:43:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0269999999999999,"15min":1.038},"min":{"1min":0.59999999999999998,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.002,"15min":1.002},"min":{"1min":0.60099999999999998,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.5289999999999999,"5min":1.589,"15min":1.589},"last":0.55300000000000005}]},{"osd":4,"last update":"Wed Oct 11 09:43:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.169,"15min":1.2},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.3879999999999999,"5min":2.016,"15min":2.016},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.3009999999999999,"15min":1.337},"min":{"1min":0.84899999999999998,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.5569999999999999,"5min":2.323,"15min":2.323},"last":1.107}]},{"osd":5,"last update":"Wed Oct 11 09:43:34 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.319,"15min":1.349},"min":{"1min":0.95599999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.4590000000000001,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.2649999999999999,"15min":1.292},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.397,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.1339999999999999}]},{"osd":6,"last update":"Wed Oct 11 09:43:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.357,"15min":1.4119999999999999},"min":{"1min":0.97099999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.5249999999999999,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":1.165},{"interface":"front","average":{"1min":1.202,"5min":1.355,"15min":1.403},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.4930000000000001,"5min":3.3719999999999999,"15min":3.3719999999999999},"last":1.0449999999999999}]},{"osd":7,"last update":"Wed Oct 11 09:43:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.405,"15min":1.4470000000000001},"min":{"1min":0.92700000000000005,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.915,"5min":2.194,"15min":2.194},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.335,"5min":1.429,"15min":1.462},"min":{"1min":0.99399999999999999,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":1.5980000000000001,"5min":1.968,"15min":1.968},"last":1.3120000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574894,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32344,"kb_used_data":1296,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339496,"statfs":{"total":96636764160,"available":96603643904,"internally_reserved":0,"allocated":1327104,"data_stored":919781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 11 09:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.80500000000000005,"15min":0.82099999999999995},"min":{"1min":0.56000000000000005,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.135,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.92600000000000005,"15min":0.95299999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.254,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":0.91400000000000003}]},{"osd":2,"last update":"Wed Oct 11 09:43:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.96099999999999997,"15min":0.96899999999999997},"min":{"1min":0.63600000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.365,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.343},{"interface":"front","average":{"1min":1.046,"5min":0.98899999999999999,"15min":0.98399999999999999},"min":{"1min":0.69999999999999996,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.2250000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.244}]},{"osd":3,"last update":"Wed Oct 11 09:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.88200000000000001,"15min":0.91200000000000003},"min":{"1min":0.629,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.079,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.0469999999999999,"15min":1.087},"min":{"1min":0.71699999999999997,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.1899999999999999,"5min":1.996,"15min":1.996},"last":0.99099999999999999}]},{"osd":4,"last update":"Wed Oct 11 09:43:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.1990000000000001,"15min":1.214},"min":{"1min":1.018,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.5649999999999999,"5min":1.744,"15min":1.744},"last":1.5840000000000001},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.22,"15min":1.2509999999999999},"min":{"1min":0.84599999999999997,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.3999999999999999,"5min":2.6629999999999998,"15min":2.6629999999999998},"last":1.216}]},{"osd":5,"last update":"Wed Oct 11 09:43:29 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.3540000000000001,"15min":1.4099999999999999},"min":{"1min":0.96799999999999997,"5min":0.88,"15min":0.88},"max":{"1min":1.448,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":1.6659999999999999},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.3340000000000001,"15min":1.3700000000000001},"min":{"1min":0.99299999999999999,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":1.4930000000000001,"5min":3.601,"15min":3.601},"last":1.3919999999999999}]},{"osd":6,"last update":"Wed Oct 11 09:43:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.24,"15min":1.266},"min":{"1min":0.98299999999999998,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.476,"5min":1.825,"15min":1.825},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.341,"15min":1.377},"min":{"1min":1.105,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":1.5149999999999999,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":1.425}]},{"osd":7,"last update":"Wed Oct 11 09:43:29 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.3260000000000001,"15min":1.3580000000000001},"min":{"1min":0.95099999999999996,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":1.585,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.339,"15min":1.391},"min":{"1min":0.95099999999999996,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.532,"5min":2.5419999999999998,"15min":2.5419999999999998},"last":1.171}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-11T09:44:11.475 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-11T09:44:11.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.678+0000 7f36167dc700 1 Processor -- start 2023-10-11T09:44:11.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.678+0000 7f36167dc700 1 -- start start 2023-10-11T09:44:11.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.682+0000 7f36167dc700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f361015b8e0 0x7f361006d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:11.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.682+0000 7f36167dc700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f361006d610 0x7f36100ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:11.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.682+0000 7f36167dc700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f361015a940 0x7f36100ca780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:11.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.682+0000 7f36167dc700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f361015e010 con 0x7f361015a940 2023-10-11T09:44:11.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.682+0000 7f36167dc700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f361015e280 con 0x7f361006d610 2023-10-11T09:44:11.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.682+0000 7f36167dc700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f361015e560 con 0x7f361015b8e0 2023-10-11T09:44:11.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.682+0000 7f3615fdb700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f361015a940 0x7f36100ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:11.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.682+0000 7f3615fdb700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f361015a940 0x7f36100ca780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35006/0 (socket says 172.21.15.78:35006) 2023-10-11T09:44:11.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.682+0000 7f3615fdb700 1 -- 172.21.15.78:0/1589345611 learned_addr learned my addr 172.21.15.78:0/1589345611 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:11.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.682+0000 7f36157da700 1 --2- 172.21.15.78:0/1589345611 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f361015b8e0 0x7f361006d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:11.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.682+0000 7f3614fd9700 1 --2- 172.21.15.78:0/1589345611 >> v2:172.21.15.133:3300/0 conn(0x7f361006d610 0x7f36100ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:11.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.682+0000 7f3615fdb700 1 -- 172.21.15.78:0/1589345611 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f361015b8e0 msgr2=0x7f361006d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:44:11.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.682+0000 7f3615fdb700 1 --2- 172.21.15.78:0/1589345611 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f361015b8e0 0x7f361006d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:11.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.682+0000 7f3615fdb700 1 -- 172.21.15.78:0/1589345611 >> v2:172.21.15.133:3300/0 conn(0x7f361006d610 msgr2=0x7f36100ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:44:11.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.682+0000 7f3615fdb700 1 --2- 172.21.15.78:0/1589345611 >> v2:172.21.15.133:3300/0 conn(0x7f361006d610 0x7f36100ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:11.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.682+0000 7f3615fdb700 1 -- 172.21.15.78:0/1589345611 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361015e840 con 0x7f361015a940 2023-10-11T09:44:11.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.682+0000 7f3614fd9700 1 --2- 172.21.15.78:0/1589345611 >> v2:172.21.15.133:3300/0 conn(0x7f361006d610 0x7f36100ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:44:11.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.682+0000 7f3615fdb700 1 --2- 172.21.15.78:0/1589345611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f361015a940 0x7f36100ca780 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f361006f960 tx=0x7f360800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36333761e025e339 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:11.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.682+0000 7f3607fff700 1 -- 172.21.15.78:0/1589345611 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360800e070 con 0x7f361015a940 2023-10-11T09:44:11.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.682+0000 7f3607fff700 1 -- 172.21.15.78:0/1589345611 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360800b910 con 0x7f361015a940 2023-10-11T09:44:11.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.682+0000 7f3607fff700 1 -- 172.21.15.78:0/1589345611 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360800bcc0 con 0x7f361015a940 2023-10-11T09:44:11.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.682+0000 7f36167dc700 1 -- 172.21.15.78:0/1589345611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f361015a940 msgr2=0x7f36100ca780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:11.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.682+0000 7f36167dc700 1 --2- 172.21.15.78:0/1589345611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f361015a940 0x7f36100ca780 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f361006f960 tx=0x7f360800b330 comp rx=0 tx=0).stop 2023-10-11T09:44:11.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.686+0000 7f36167dc700 1 -- 172.21.15.78:0/1589345611 shutdown_connections 2023-10-11T09:44:11.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.686+0000 7f36167dc700 1 --2- 172.21.15.78:0/1589345611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f361015a940 0x7f36100ca780 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:11.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.686+0000 7f36167dc700 1 --2- 172.21.15.78:0/1589345611 >> v2:172.21.15.133:3300/0 conn(0x7f361006d610 0x7f36100ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:11.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.686+0000 7f36167dc700 1 --2- 172.21.15.78:0/1589345611 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f361015b8e0 0x7f361006d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:11.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.686+0000 7f36167dc700 1 -- 172.21.15.78:0/1589345611 >> 172.21.15.78:0/1589345611 conn(0x7f36100cea60 msgr2=0x7f36100cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:11.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.686+0000 7f36167dc700 1 -- 172.21.15.78:0/1589345611 shutdown_connections 2023-10-11T09:44:11.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.686+0000 7f36167dc700 1 -- 172.21.15.78:0/1589345611 wait complete. 2023-10-11T09:44:11.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.690+0000 7f36167dc700 1 Processor -- start 2023-10-11T09:44:11.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.690+0000 7f36167dc700 1 -- start start 2023-10-11T09:44:11.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.690+0000 7f36167dc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f361006d610 0x7f3610159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:11.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.690+0000 7f36167dc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f361015a940 0x7f36101f2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:11.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.690+0000 7f36167dc700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f361015b8e0 0x7f361013abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:11.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.690+0000 7f36167dc700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361015a190 con 0x7f361015a940 2023-10-11T09:44:11.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.690+0000 7f36167dc700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361013d340 con 0x7f361006d610 2023-10-11T09:44:11.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.690+0000 7f36167dc700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36101f29d0 con 0x7f361015b8e0 2023-10-11T09:44:11.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.690+0000 7f3614fd9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f361015a940 0x7f36101f2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:11.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.690+0000 7f3614fd9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f361015a940 0x7f36101f2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35018/0 (socket says 172.21.15.78:35018) 2023-10-11T09:44:11.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.690+0000 7f3614fd9700 1 -- 172.21.15.78:0/2078133467 learned_addr learned my addr 172.21.15.78:0/2078133467 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:11.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.690+0000 7f3615fdb700 1 --2- 172.21.15.78:0/2078133467 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f361015b8e0 0x7f361013abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:11.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.690+0000 7f36157da700 1 --2- 172.21.15.78:0/2078133467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f361006d610 0x7f3610159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:11.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.690+0000 7f3614fd9700 1 -- 172.21.15.78:0/2078133467 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f361015b8e0 msgr2=0x7f361013abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:11.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.690+0000 7f3614fd9700 1 --2- 172.21.15.78:0/2078133467 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f361015b8e0 0x7f361013abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:11.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.690+0000 7f3614fd9700 1 -- 172.21.15.78:0/2078133467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f361006d610 msgr2=0x7f3610159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:11.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.690+0000 7f3614fd9700 1 --2- 172.21.15.78:0/2078133467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f361006d610 0x7f3610159c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:11.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.690+0000 7f3614fd9700 1 -- 172.21.15.78:0/2078133467 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3608009b30 con 0x7f361015a940 2023-10-11T09:44:11.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.690+0000 7f3615fdb700 1 --2- 172.21.15.78:0/2078133467 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f361015b8e0 0x7f361013abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:44:11.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.690+0000 7f3614fd9700 1 --2- 172.21.15.78:0/2078133467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f361015a940 0x7f36101f2490 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f360000e810 tx=0x7f360000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:11.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.690+0000 7f3605ffb700 1 -- 172.21.15.78:0/2078133467 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3600013610 con 0x7f361015a940 2023-10-11T09:44:11.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.690+0000 7f3605ffb700 1 -- 172.21.15.78:0/2078133467 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3600013770 con 0x7f361015a940 2023-10-11T09:44:11.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.690+0000 7f3605ffb700 1 -- 172.21.15.78:0/2078133467 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3600013b00 con 0x7f361015a940 2023-10-11T09:44:11.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.690+0000 7f36167dc700 1 -- 172.21.15.78:0/2078133467 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f361013c730 con 0x7f361015a940 2023-10-11T09:44:11.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.694+0000 7f36167dc700 1 -- 172.21.15.78:0/2078133467 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f361013ccf0 con 0x7f361015a940 2023-10-11T09:44:11.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.694+0000 7f36167dc700 1 -- 172.21.15.78:0/2078133467 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3610065d40 con 0x7f361015a940 2023-10-11T09:44:11.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.694+0000 7f3605ffb700 1 -- 172.21.15.78:0/2078133467 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f3600013c60 con 0x7f361015a940 2023-10-11T09:44:11.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.698+0000 7f3605ffb700 1 --2- 172.21.15.78:0/2078133467 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f35f4065ed0 0x7f35f4068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:11.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.698+0000 7f3605ffb700 1 -- 172.21.15.78:0/2078133467 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f36000834f0 con 0x7f361015a940 2023-10-11T09:44:11.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.698+0000 7f36157da700 1 --2- 172.21.15.78:0/2078133467 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f35f4065ed0 0x7f35f4068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:11.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.698+0000 7f36157da700 1 --2- 172.21.15.78:0/2078133467 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f35f4065ed0 0x7f35f4068380 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3610137420 tx=0x7f360c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:11.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.702+0000 7f3605ffb700 1 -- 172.21.15.78:0/2078133467 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f3600050ab0 con 0x7f361015a940 2023-10-11T09:44:11.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.874+0000 7f36167dc700 1 -- 172.21.15.78:0/2078133467 --> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f361015eb20 con 0x7f35f4065ed0 2023-10-11T09:44:11.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.882+0000 7f3605ffb700 1 -- 172.21.15.78:0/2078133467 <== mgr.4101 v2:172.21.15.78:6832/35606 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308629 (secure 0 0 0) 0x7f360c011750 con 0x7f35f4065ed0 2023-10-11T09:44:11.893 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:44:11.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.890+0000 7f36167dc700 1 -- 172.21.15.78:0/2078133467 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f35f4065ed0 msgr2=0x7f35f4068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:11.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.890+0000 7f36167dc700 1 --2- 172.21.15.78:0/2078133467 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f35f4065ed0 0x7f35f4068380 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3610137420 tx=0x7f360c008040 comp rx=0 tx=0).stop 2023-10-11T09:44:11.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.890+0000 7f36167dc700 1 -- 172.21.15.78:0/2078133467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f361015a940 msgr2=0x7f36101f2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:11.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.890+0000 7f36167dc700 1 --2- 172.21.15.78:0/2078133467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f361015a940 0x7f36101f2490 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f360000e810 tx=0x7f360000ebd0 comp rx=0 tx=0).stop 2023-10-11T09:44:11.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.890+0000 7f36167dc700 1 -- 172.21.15.78:0/2078133467 shutdown_connections 2023-10-11T09:44:11.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.890+0000 7f36167dc700 1 --2- 172.21.15.78:0/2078133467 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f35f4065ed0 0x7f35f4068380 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:11.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.890+0000 7f36167dc700 1 --2- 172.21.15.78:0/2078133467 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f361015b8e0 0x7f361013abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:11.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.890+0000 7f36167dc700 1 --2- 172.21.15.78:0/2078133467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f361015a940 0x7f36101f2490 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:11.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.890+0000 7f36167dc700 1 --2- 172.21.15.78:0/2078133467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f361006d610 0x7f3610159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:11.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.890+0000 7f36167dc700 1 -- 172.21.15.78:0/2078133467 >> 172.21.15.78:0/2078133467 conn(0x7f36100cea60 msgr2=0x7f3610143640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:11.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.890+0000 7f36167dc700 1 -- 172.21.15.78:0/2078133467 shutdown_connections 2023-10-11T09:44:11.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:11.890+0000 7f36167dc700 1 -- 172.21.15.78:0/2078133467 wait complete. 2023-10-11T09:44:11.899 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2023-10-11T09:44:11.918 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2023-10-11T09:44:10.393955+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":93,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230656,"kb_used_data":6848,"kb_used_omap":0,"kb_used_meta":223744,"kb_avail":754744064,"statfs":{"total":773094113280,"available":772857921536,"internally_reserved":0,"allocated":7012352,"data_stored":3816902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":229113856},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.003965"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.874112+0000","last_change":"2023-10-11T09:44:06.885569+0000","last_active":"2023-10-11T09:44:07.874112+0000","last_peered":"2023-10-11T09:44:07.874112+0000","last_clean":"2023-10-11T09:44:07.874112+0000","last_became_active":"2023-10-11T09:44:06.885327+0000","last_became_peered":"2023-10-11T09:44:06.885327+0000","last_unstale":"2023-10-11T09:44:07.874112+0000","last_undegraded":"2023-10-11T09:44:07.874112+0000","last_fullsized":"2023-10-11T09:44:07.874112+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:29:14.313842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.895846+0000","last_change":"2023-10-11T09:44:06.887209+0000","last_active":"2023-10-11T09:44:06.895846+0000","last_peered":"2023-10-11T09:44:06.895846+0000","last_clean":"2023-10-11T09:44:06.895846+0000","last_became_active":"2023-10-11T09:44:06.886687+0000","last_became_peered":"2023-10-11T09:44:06.886687+0000","last_unstale":"2023-10-11T09:44:06.895846+0000","last_undegraded":"2023-10-11T09:44:06.895846+0000","last_fullsized":"2023-10-11T09:44:06.895846+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:12:30.534498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.876274+0000","last_change":"2023-10-11T09:44:06.870609+0000","last_active":"2023-10-11T09:44:07.876274+0000","last_peered":"2023-10-11T09:44:07.876274+0000","last_clean":"2023-10-11T09:44:07.876274+0000","last_became_active":"2023-10-11T09:44:06.870314+0000","last_became_peered":"2023-10-11T09:44:06.870314+0000","last_unstale":"2023-10-11T09:44:07.876274+0000","last_undegraded":"2023-10-11T09:44:07.876274+0000","last_fullsized":"2023-10-11T09:44:07.876274+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:14:19.388175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.896989+0000","last_change":"2023-10-11T09:44:06.883407+0000","last_active":"2023-10-11T09:44:06.896989+0000","last_peered":"2023-10-11T09:44:06.896989+0000","last_clean":"2023-10-11T09:44:06.896989+0000","last_became_active":"2023-10-11T09:44:06.883141+0000","last_became_peered":"2023-10-11T09:44:06.883141+0000","last_unstale":"2023-10-11T09:44:06.896989+0000","last_undegraded":"2023-10-11T09:44:06.896989+0000","last_fullsized":"2023-10-11T09:44:06.896989+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:38:14.424051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.873935+0000","last_change":"2023-10-11T09:44:06.896547+0000","last_active":"2023-10-11T09:44:07.873935+0000","last_peered":"2023-10-11T09:44:07.873935+0000","last_clean":"2023-10-11T09:44:07.873935+0000","last_became_active":"2023-10-11T09:44:06.896275+0000","last_became_peered":"2023-10-11T09:44:06.896275+0000","last_unstale":"2023-10-11T09:44:07.873935+0000","last_undegraded":"2023-10-11T09:44:07.873935+0000","last_fullsized":"2023-10-11T09:44:07.873935+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:17:54.521775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:08.410616+0000","last_change":"2023-10-11T09:44:06.883258+0000","last_active":"2023-10-11T09:44:08.410616+0000","last_peered":"2023-10-11T09:44:08.410616+0000","last_clean":"2023-10-11T09:44:08.410616+0000","last_became_active":"2023-10-11T09:44:06.883006+0000","last_became_peered":"2023-10-11T09:44:06.883006+0000","last_unstale":"2023-10-11T09:44:08.410616+0000","last_undegraded":"2023-10-11T09:44:08.410616+0000","last_fullsized":"2023-10-11T09:44:08.410616+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:06:50.772284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.874166+0000","last_change":"2023-10-11T09:44:06.884457+0000","last_active":"2023-10-11T09:44:07.874166+0000","last_peered":"2023-10-11T09:44:07.874166+0000","last_clean":"2023-10-11T09:44:07.874166+0000","last_became_active":"2023-10-11T09:44:06.884188+0000","last_became_peered":"2023-10-11T09:44:06.884188+0000","last_unstale":"2023-10-11T09:44:07.874166+0000","last_undegraded":"2023-10-11T09:44:07.874166+0000","last_fullsized":"2023-10-11T09:44:07.874166+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:14:19.388175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.885023+0000","last_change":"2023-10-11T09:44:06.879891+0000","last_active":"2023-10-11T09:44:06.885023+0000","last_peered":"2023-10-11T09:44:06.885023+0000","last_clean":"2023-10-11T09:44:06.885023+0000","last_became_active":"2023-10-11T09:44:06.879625+0000","last_became_peered":"2023-10-11T09:44:06.879625+0000","last_unstale":"2023-10-11T09:44:06.885023+0000","last_undegraded":"2023-10-11T09:44:06.885023+0000","last_fullsized":"2023-10-11T09:44:06.885023+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:59:54.587753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.874411+0000","last_change":"2023-10-11T09:44:06.894942+0000","last_active":"2023-10-11T09:44:07.874411+0000","last_peered":"2023-10-11T09:44:07.874411+0000","last_clean":"2023-10-11T09:44:07.874411+0000","last_became_active":"2023-10-11T09:44:06.894554+0000","last_became_peered":"2023-10-11T09:44:06.894554+0000","last_unstale":"2023-10-11T09:44:07.874411+0000","last_undegraded":"2023-10-11T09:44:07.874411+0000","last_fullsized":"2023-10-11T09:44:07.874411+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:58:17.041296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.892979+0000","last_change":"2023-10-11T09:44:06.884597+0000","last_active":"2023-10-11T09:44:06.892979+0000","last_peered":"2023-10-11T09:44:06.892979+0000","last_clean":"2023-10-11T09:44:06.892979+0000","last_became_active":"2023-10-11T09:44:06.884327+0000","last_became_peered":"2023-10-11T09:44:06.884327+0000","last_unstale":"2023-10-11T09:44:06.892979+0000","last_undegraded":"2023-10-11T09:44:06.892979+0000","last_fullsized":"2023-10-11T09:44:06.892979+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:35:07.327877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.896575+0000","last_change":"2023-10-11T09:44:06.891071+0000","last_active":"2023-10-11T09:44:06.896575+0000","last_peered":"2023-10-11T09:44:06.896575+0000","last_clean":"2023-10-11T09:44:06.896575+0000","last_became_active":"2023-10-11T09:44:06.890807+0000","last_became_peered":"2023-10-11T09:44:06.890807+0000","last_unstale":"2023-10-11T09:44:06.896575+0000","last_undegraded":"2023-10-11T09:44:06.896575+0000","last_fullsized":"2023-10-11T09:44:06.896575+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:07:11.630325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.901803+0000","last_change":"2023-10-11T09:44:06.890407+0000","last_active":"2023-10-11T09:44:06.901803+0000","last_peered":"2023-10-11T09:44:06.901803+0000","last_clean":"2023-10-11T09:44:06.901803+0000","last_became_active":"2023-10-11T09:44:06.890110+0000","last_became_peered":"2023-10-11T09:44:06.890110+0000","last_unstale":"2023-10-11T09:44:06.901803+0000","last_undegraded":"2023-10-11T09:44:06.901803+0000","last_fullsized":"2023-10-11T09:44:06.901803+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:57:01.167316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.896615+0000","last_change":"2023-10-11T09:44:06.891217+0000","last_active":"2023-10-11T09:44:06.896615+0000","last_peered":"2023-10-11T09:44:06.896615+0000","last_clean":"2023-10-11T09:44:06.896615+0000","last_became_active":"2023-10-11T09:44:06.890919+0000","last_became_peered":"2023-10-11T09:44:06.890919+0000","last_unstale":"2023-10-11T09:44:06.896615+0000","last_undegraded":"2023-10-11T09:44:06.896615+0000","last_fullsized":"2023-10-11T09:44:06.896615+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:15:47.307014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.874633+0000","last_change":"2023-10-11T09:44:06.880741+0000","last_active":"2023-10-11T09:44:07.874633+0000","last_peered":"2023-10-11T09:44:07.874633+0000","last_clean":"2023-10-11T09:44:07.874633+0000","last_became_active":"2023-10-11T09:44:06.880471+0000","last_became_peered":"2023-10-11T09:44:06.880471+0000","last_unstale":"2023-10-11T09:44:07.874633+0000","last_undegraded":"2023-10-11T09:44:07.874633+0000","last_fullsized":"2023-10-11T09:44:07.874633+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:42:38.468268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.874184+0000","last_change":"2023-10-11T09:44:06.886081+0000","last_active":"2023-10-11T09:44:07.874184+0000","last_peered":"2023-10-11T09:44:07.874184+0000","last_clean":"2023-10-11T09:44:07.874184+0000","last_became_active":"2023-10-11T09:44:06.885781+0000","last_became_peered":"2023-10-11T09:44:06.885781+0000","last_unstale":"2023-10-11T09:44:07.874184+0000","last_undegraded":"2023-10-11T09:44:07.874184+0000","last_fullsized":"2023-10-11T09:44:07.874184+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T21:17:07.726161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:08.410596+0000","last_change":"2023-10-11T09:44:06.884835+0000","last_active":"2023-10-11T09:44:08.410596+0000","last_peered":"2023-10-11T09:44:08.410596+0000","last_clean":"2023-10-11T09:44:08.410596+0000","last_became_active":"2023-10-11T09:44:06.884569+0000","last_became_peered":"2023-10-11T09:44:06.884569+0000","last_unstale":"2023-10-11T09:44:08.410596+0000","last_undegraded":"2023-10-11T09:44:08.410596+0000","last_fullsized":"2023-10-11T09:44:08.410596+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:48:36.453678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.885866+0000","last_change":"2023-10-11T09:44:06.881997+0000","last_active":"2023-10-11T09:44:06.885866+0000","last_peered":"2023-10-11T09:44:06.885866+0000","last_clean":"2023-10-11T09:44:06.885866+0000","last_became_active":"2023-10-11T09:44:06.881709+0000","last_became_peered":"2023-10-11T09:44:06.881709+0000","last_unstale":"2023-10-11T09:44:06.885866+0000","last_undegraded":"2023-10-11T09:44:06.885866+0000","last_fullsized":"2023-10-11T09:44:06.885866+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:37:20.675117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.876497+0000","last_change":"2023-10-11T09:44:06.887605+0000","last_active":"2023-10-11T09:44:07.876497+0000","last_peered":"2023-10-11T09:44:07.876497+0000","last_clean":"2023-10-11T09:44:07.876497+0000","last_became_active":"2023-10-11T09:44:06.887305+0000","last_became_peered":"2023-10-11T09:44:06.887305+0000","last_unstale":"2023-10-11T09:44:07.876497+0000","last_undegraded":"2023-10-11T09:44:07.876497+0000","last_fullsized":"2023-10-11T09:44:07.876497+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:26:59.491028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.874187+0000","last_change":"2023-10-11T09:44:06.885721+0000","last_active":"2023-10-11T09:44:07.874187+0000","last_peered":"2023-10-11T09:44:07.874187+0000","last_clean":"2023-10-11T09:44:07.874187+0000","last_became_active":"2023-10-11T09:44:06.885307+0000","last_became_peered":"2023-10-11T09:44:06.885307+0000","last_unstale":"2023-10-11T09:44:07.874187+0000","last_undegraded":"2023-10-11T09:44:07.874187+0000","last_fullsized":"2023-10-11T09:44:07.874187+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:06:00.273203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.889942+0000","last_change":"2023-10-11T09:44:06.882987+0000","last_active":"2023-10-11T09:44:06.889942+0000","last_peered":"2023-10-11T09:44:06.889942+0000","last_clean":"2023-10-11T09:44:06.889942+0000","last_became_active":"2023-10-11T09:44:06.882659+0000","last_became_peered":"2023-10-11T09:44:06.882659+0000","last_unstale":"2023-10-11T09:44:06.889942+0000","last_undegraded":"2023-10-11T09:44:06.889942+0000","last_fullsized":"2023-10-11T09:44:06.889942+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:15:01.902476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.876809+0000","last_change":"2023-10-11T09:44:06.881149+0000","last_active":"2023-10-11T09:44:07.876809+0000","last_peered":"2023-10-11T09:44:07.876809+0000","last_clean":"2023-10-11T09:44:07.876809+0000","last_became_active":"2023-10-11T09:44:06.879758+0000","last_became_peered":"2023-10-11T09:44:06.879758+0000","last_unstale":"2023-10-11T09:44:07.876809+0000","last_undegraded":"2023-10-11T09:44:07.876809+0000","last_fullsized":"2023-10-11T09:44:07.876809+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T21:14:29.347600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.890240+0000","last_change":"2023-10-11T09:44:06.884612+0000","last_active":"2023-10-11T09:44:06.890240+0000","last_peered":"2023-10-11T09:44:06.890240+0000","last_clean":"2023-10-11T09:44:06.890240+0000","last_became_active":"2023-10-11T09:44:06.884214+0000","last_became_peered":"2023-10-11T09:44:06.884214+0000","last_unstale":"2023-10-11T09:44:06.890240+0000","last_undegraded":"2023-10-11T09:44:06.890240+0000","last_fullsized":"2023-10-11T09:44:06.890240+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:38:14.424051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.876503+0000","last_change":"2023-10-11T09:44:06.886857+0000","last_active":"2023-10-11T09:44:07.876503+0000","last_peered":"2023-10-11T09:44:07.876503+0000","last_clean":"2023-10-11T09:44:07.876503+0000","last_became_active":"2023-10-11T09:44:06.886349+0000","last_became_peered":"2023-10-11T09:44:06.886349+0000","last_unstale":"2023-10-11T09:44:07.876503+0000","last_undegraded":"2023-10-11T09:44:07.876503+0000","last_fullsized":"2023-10-11T09:44:07.876503+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:21:29.593748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.897787+0000","last_change":"2023-10-11T09:44:06.892090+0000","last_active":"2023-10-11T09:44:06.897787+0000","last_peered":"2023-10-11T09:44:06.897787+0000","last_clean":"2023-10-11T09:44:06.897787+0000","last_became_active":"2023-10-11T09:44:06.891764+0000","last_became_peered":"2023-10-11T09:44:06.891764+0000","last_unstale":"2023-10-11T09:44:06.897787+0000","last_undegraded":"2023-10-11T09:44:06.897787+0000","last_fullsized":"2023-10-11T09:44:06.897787+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:25:15.702261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.874222+0000","last_change":"2023-10-11T09:44:06.888272+0000","last_active":"2023-10-11T09:44:07.874222+0000","last_peered":"2023-10-11T09:44:07.874222+0000","last_clean":"2023-10-11T09:44:07.874222+0000","last_became_active":"2023-10-11T09:44:06.888015+0000","last_became_peered":"2023-10-11T09:44:06.888015+0000","last_unstale":"2023-10-11T09:44:07.874222+0000","last_undegraded":"2023-10-11T09:44:07.874222+0000","last_fullsized":"2023-10-11T09:44:07.874222+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:04:16.290001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.896843+0000","last_change":"2023-10-11T09:44:06.882392+0000","last_active":"2023-10-11T09:44:06.896843+0000","last_peered":"2023-10-11T09:44:06.896843+0000","last_clean":"2023-10-11T09:44:06.896843+0000","last_became_active":"2023-10-11T09:44:06.882059+0000","last_became_peered":"2023-10-11T09:44:06.882059+0000","last_unstale":"2023-10-11T09:44:06.896843+0000","last_undegraded":"2023-10-11T09:44:06.896843+0000","last_fullsized":"2023-10-11T09:44:06.896843+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T21:14:29.347600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:08.410855+0000","last_change":"2023-10-11T09:44:06.883337+0000","last_active":"2023-10-11T09:44:08.410855+0000","last_peered":"2023-10-11T09:44:08.410855+0000","last_clean":"2023-10-11T09:44:08.410855+0000","last_became_active":"2023-10-11T09:44:06.882735+0000","last_became_peered":"2023-10-11T09:44:06.882735+0000","last_unstale":"2023-10-11T09:44:08.410855+0000","last_undegraded":"2023-10-11T09:44:08.410855+0000","last_fullsized":"2023-10-11T09:44:08.410855+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T21:04:02.806218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.874330+0000","last_change":"2023-10-11T09:44:06.890036+0000","last_active":"2023-10-11T09:44:07.874330+0000","last_peered":"2023-10-11T09:44:07.874330+0000","last_clean":"2023-10-11T09:44:07.874330+0000","last_became_active":"2023-10-11T09:44:06.889804+0000","last_became_peered":"2023-10-11T09:44:06.889804+0000","last_unstale":"2023-10-11T09:44:07.874330+0000","last_undegraded":"2023-10-11T09:44:07.874330+0000","last_fullsized":"2023-10-11T09:44:07.874330+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:13:41.647635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.882948+0000","last_change":"2023-10-11T09:44:06.879165+0000","last_active":"2023-10-11T09:44:06.882948+0000","last_peered":"2023-10-11T09:44:06.882948+0000","last_clean":"2023-10-11T09:44:06.882948+0000","last_became_active":"2023-10-11T09:44:06.878777+0000","last_became_peered":"2023-10-11T09:44:06.878777+0000","last_unstale":"2023-10-11T09:44:06.882948+0000","last_undegraded":"2023-10-11T09:44:06.882948+0000","last_fullsized":"2023-10-11T09:44:06.882948+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:28:43.416105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.876192+0000","last_change":"2023-10-11T09:44:06.877795+0000","last_active":"2023-10-11T09:44:07.876192+0000","last_peered":"2023-10-11T09:44:07.876192+0000","last_clean":"2023-10-11T09:44:07.876192+0000","last_became_active":"2023-10-11T09:44:06.877446+0000","last_became_peered":"2023-10-11T09:44:06.877446+0000","last_unstale":"2023-10-11T09:44:07.876192+0000","last_undegraded":"2023-10-11T09:44:07.876192+0000","last_fullsized":"2023-10-11T09:44:07.876192+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T21:17:07.726161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.874315+0000","last_change":"2023-10-11T09:44:06.889009+0000","last_active":"2023-10-11T09:44:07.874315+0000","last_peered":"2023-10-11T09:44:07.874315+0000","last_clean":"2023-10-11T09:44:07.874315+0000","last_became_active":"2023-10-11T09:44:06.888775+0000","last_became_peered":"2023-10-11T09:44:06.888775+0000","last_unstale":"2023-10-11T09:44:07.874315+0000","last_undegraded":"2023-10-11T09:44:07.874315+0000","last_fullsized":"2023-10-11T09:44:07.874315+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:17:36.489468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:08.410760+0000","last_change":"2023-10-11T09:44:06.892952+0000","last_active":"2023-10-11T09:44:08.410760+0000","last_peered":"2023-10-11T09:44:08.410760+0000","last_clean":"2023-10-11T09:44:08.410760+0000","last_became_active":"2023-10-11T09:44:06.892709+0000","last_became_peered":"2023-10-11T09:44:06.892709+0000","last_unstale":"2023-10-11T09:44:08.410760+0000","last_undegraded":"2023-10-11T09:44:08.410760+0000","last_fullsized":"2023-10-11T09:44:08.410760+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:00:03.686268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:08.411222+0000","last_change":"2023-10-11T09:44:06.891485+0000","last_active":"2023-10-11T09:44:08.411222+0000","last_peered":"2023-10-11T09:44:08.411222+0000","last_clean":"2023-10-11T09:44:08.411222+0000","last_became_active":"2023-10-11T09:44:06.885845+0000","last_became_peered":"2023-10-11T09:44:06.885845+0000","last_unstale":"2023-10-11T09:44:08.411222+0000","last_undegraded":"2023-10-11T09:44:08.411222+0000","last_fullsized":"2023-10-11T09:44:08.411222+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:59:54.587753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"60'4","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:08.411211+0000","last_change":"2023-10-11T09:44:06.420454+0000","last_active":"2023-10-11T09:44:08.411211+0000","last_peered":"2023-10-11T09:44:08.411211+0000","last_clean":"2023-10-11T09:44:08.411211+0000","last_became_active":"2023-10-11T09:44:06.420196+0000","last_became_peered":"2023-10-11T09:44:06.420196+0000","last_unstale":"2023-10-11T09:44:08.411211+0000","last_undegraded":"2023-10-11T09:44:08.411211+0000","last_fullsized":"2023-10-11T09:44:08.411211+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:14:18.374502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.901231+0000","last_change":"2023-10-11T09:44:06.893045+0000","last_active":"2023-10-11T09:44:06.901231+0000","last_peered":"2023-10-11T09:44:06.901231+0000","last_clean":"2023-10-11T09:44:06.901231+0000","last_became_active":"2023-10-11T09:44:06.892786+0000","last_became_peered":"2023-10-11T09:44:06.892786+0000","last_unstale":"2023-10-11T09:44:06.901231+0000","last_undegraded":"2023-10-11T09:44:06.901231+0000","last_fullsized":"2023-10-11T09:44:06.901231+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:16:00.990878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"60'1","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.887050+0000","last_change":"2023-10-11T09:44:06.426515+0000","last_active":"2023-10-11T09:44:06.887050+0000","last_peered":"2023-10-11T09:44:06.887050+0000","last_clean":"2023-10-11T09:44:06.887050+0000","last_became_active":"2023-10-11T09:44:06.426254+0000","last_became_peered":"2023-10-11T09:44:06.426254+0000","last_unstale":"2023-10-11T09:44:06.887050+0000","last_undegraded":"2023-10-11T09:44:06.887050+0000","last_fullsized":"2023-10-11T09:44:06.887050+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:23:23.869338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:08.411274+0000","last_change":"2023-10-11T09:44:06.895073+0000","last_active":"2023-10-11T09:44:08.411274+0000","last_peered":"2023-10-11T09:44:08.411274+0000","last_clean":"2023-10-11T09:44:08.411274+0000","last_became_active":"2023-10-11T09:44:06.893974+0000","last_became_peered":"2023-10-11T09:44:06.893974+0000","last_unstale":"2023-10-11T09:44:08.411274+0000","last_undegraded":"2023-10-11T09:44:08.411274+0000","last_fullsized":"2023-10-11T09:44:08.411274+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:20:36.948577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"60'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:08.411101+0000","last_change":"2023-10-11T09:44:06.418911+0000","last_active":"2023-10-11T09:44:08.411101+0000","last_peered":"2023-10-11T09:44:08.411101+0000","last_clean":"2023-10-11T09:44:08.411101+0000","last_became_active":"2023-10-11T09:44:06.418644+0000","last_became_peered":"2023-10-11T09:44:06.418644+0000","last_unstale":"2023-10-11T09:44:08.411101+0000","last_undegraded":"2023-10-11T09:44:08.411101+0000","last_fullsized":"2023-10-11T09:44:08.411101+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:58:16.027623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:08.411384+0000","last_change":"2023-10-11T09:44:06.882997+0000","last_active":"2023-10-11T09:44:08.411384+0000","last_peered":"2023-10-11T09:44:08.411384+0000","last_clean":"2023-10-11T09:44:08.411384+0000","last_became_active":"2023-10-11T09:44:06.882074+0000","last_became_peered":"2023-10-11T09:44:06.882074+0000","last_unstale":"2023-10-11T09:44:08.411384+0000","last_undegraded":"2023-10-11T09:44:08.411384+0000","last_fullsized":"2023-10-11T09:44:08.411384+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:15:01.902476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"60'2","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.902131+0000","last_change":"2023-10-11T09:44:06.424342+0000","last_active":"2023-10-11T09:44:06.902131+0000","last_peered":"2023-10-11T09:44:06.902131+0000","last_clean":"2023-10-11T09:44:06.902131+0000","last_became_active":"2023-10-11T09:44:06.424058+0000","last_became_peered":"2023-10-11T09:44:06.424058+0000","last_unstale":"2023-10-11T09:44:06.902131+0000","last_undegraded":"2023-10-11T09:44:06.902131+0000","last_fullsized":"2023-10-11T09:44:06.902131+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:47:17.742443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.890223+0000","last_change":"2023-10-11T09:44:06.884459+0000","last_active":"2023-10-11T09:44:06.890223+0000","last_peered":"2023-10-11T09:44:06.890223+0000","last_clean":"2023-10-11T09:44:06.890223+0000","last_became_active":"2023-10-11T09:44:06.883887+0000","last_became_peered":"2023-10-11T09:44:06.883887+0000","last_unstale":"2023-10-11T09:44:06.890223+0000","last_undegraded":"2023-10-11T09:44:06.890223+0000","last_fullsized":"2023-10-11T09:44:06.890223+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:00:38.019876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"60'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.874385+0000","last_change":"2023-10-11T09:44:06.421809+0000","last_active":"2023-10-11T09:44:07.874385+0000","last_peered":"2023-10-11T09:44:07.874385+0000","last_clean":"2023-10-11T09:44:07.874385+0000","last_became_active":"2023-10-11T09:44:06.421252+0000","last_became_peered":"2023-10-11T09:44:06.421252+0000","last_unstale":"2023-10-11T09:44:07.874385+0000","last_undegraded":"2023-10-11T09:44:07.874385+0000","last_fullsized":"2023-10-11T09:44:07.874385+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:33:38.705914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.901364+0000","last_change":"2023-10-11T09:44:06.892848+0000","last_active":"2023-10-11T09:44:06.901364+0000","last_peered":"2023-10-11T09:44:06.901364+0000","last_clean":"2023-10-11T09:44:06.901364+0000","last_became_active":"2023-10-11T09:44:06.892582+0000","last_became_peered":"2023-10-11T09:44:06.892582+0000","last_unstale":"2023-10-11T09:44:06.901364+0000","last_undegraded":"2023-10-11T09:44:06.901364+0000","last_fullsized":"2023-10-11T09:44:06.901364+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:01:02.059081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"60'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:08.410812+0000","last_change":"2023-10-11T09:44:06.423425+0000","last_active":"2023-10-11T09:44:08.410812+0000","last_peered":"2023-10-11T09:44:08.410812+0000","last_clean":"2023-10-11T09:44:08.410812+0000","last_became_active":"2023-10-11T09:44:06.423124+0000","last_became_peered":"2023-10-11T09:44:06.423124+0000","last_unstale":"2023-10-11T09:44:08.410812+0000","last_undegraded":"2023-10-11T09:44:08.410812+0000","last_fullsized":"2023-10-11T09:44:08.410812+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:50:52.322578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.896553+0000","last_change":"2023-10-11T09:44:06.889441+0000","last_active":"2023-10-11T09:44:06.896553+0000","last_peered":"2023-10-11T09:44:06.896553+0000","last_clean":"2023-10-11T09:44:06.896553+0000","last_became_active":"2023-10-11T09:44:06.888835+0000","last_became_peered":"2023-10-11T09:44:06.888835+0000","last_unstale":"2023-10-11T09:44:06.896553+0000","last_undegraded":"2023-10-11T09:44:06.896553+0000","last_fullsized":"2023-10-11T09:44:06.896553+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:23:24.163168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.878098+0000","last_change":"2023-10-11T09:44:06.422145+0000","last_active":"2023-10-11T09:44:06.878098+0000","last_peered":"2023-10-11T09:44:06.878098+0000","last_clean":"2023-10-11T09:44:06.878098+0000","last_became_active":"2023-10-11T09:44:06.421418+0000","last_became_peered":"2023-10-11T09:44:06.421418+0000","last_unstale":"2023-10-11T09:44:06.878098+0000","last_undegraded":"2023-10-11T09:44:06.878098+0000","last_fullsized":"2023-10-11T09:44:06.878098+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:41:17.407368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.885941+0000","last_change":"2023-10-11T09:44:06.880771+0000","last_active":"2023-10-11T09:44:06.885941+0000","last_peered":"2023-10-11T09:44:06.885941+0000","last_clean":"2023-10-11T09:44:06.885941+0000","last_became_active":"2023-10-11T09:44:06.880162+0000","last_became_peered":"2023-10-11T09:44:06.880162+0000","last_unstale":"2023-10-11T09:44:06.885941+0000","last_undegraded":"2023-10-11T09:44:06.885941+0000","last_fullsized":"2023-10-11T09:44:06.885941+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:21:29.593748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.874279+0000","last_change":"2023-10-11T09:44:06.419933+0000","last_active":"2023-10-11T09:44:07.874279+0000","last_peered":"2023-10-11T09:44:07.874279+0000","last_clean":"2023-10-11T09:44:07.874279+0000","last_became_active":"2023-10-11T09:44:06.419631+0000","last_became_peered":"2023-10-11T09:44:06.419631+0000","last_unstale":"2023-10-11T09:44:07.874279+0000","last_undegraded":"2023-10-11T09:44:07.874279+0000","last_fullsized":"2023-10-11T09:44:07.874279+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:53:48.608829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.874442+0000","last_change":"2023-10-11T09:44:06.885643+0000","last_active":"2023-10-11T09:44:07.874442+0000","last_peered":"2023-10-11T09:44:07.874442+0000","last_clean":"2023-10-11T09:44:07.874442+0000","last_became_active":"2023-10-11T09:44:06.885406+0000","last_became_peered":"2023-10-11T09:44:06.885406+0000","last_unstale":"2023-10-11T09:44:07.874442+0000","last_undegraded":"2023-10-11T09:44:07.874442+0000","last_fullsized":"2023-10-11T09:44:07.874442+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:26:36.532157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.876810+0000","last_change":"2023-10-11T09:44:06.427959+0000","last_active":"2023-10-11T09:44:07.876810+0000","last_peered":"2023-10-11T09:44:07.876810+0000","last_clean":"2023-10-11T09:44:07.876810+0000","last_became_active":"2023-10-11T09:44:06.427566+0000","last_became_peered":"2023-10-11T09:44:06.427566+0000","last_unstale":"2023-10-11T09:44:07.876810+0000","last_undegraded":"2023-10-11T09:44:07.876810+0000","last_fullsized":"2023-10-11T09:44:07.876810+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:44:59.122368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.876683+0000","last_change":"2023-10-11T09:44:06.889415+0000","last_active":"2023-10-11T09:44:07.876683+0000","last_peered":"2023-10-11T09:44:07.876683+0000","last_clean":"2023-10-11T09:44:07.876683+0000","last_became_active":"2023-10-11T09:44:06.889126+0000","last_became_peered":"2023-10-11T09:44:06.889126+0000","last_unstale":"2023-10-11T09:44:07.876683+0000","last_undegraded":"2023-10-11T09:44:07.876683+0000","last_fullsized":"2023-10-11T09:44:07.876683+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:43:50.221576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.863543+0000","last_change":"2023-10-11T09:44:06.420327+0000","last_active":"2023-10-11T09:44:06.863543+0000","last_peered":"2023-10-11T09:44:06.863543+0000","last_clean":"2023-10-11T09:44:06.863543+0000","last_became_active":"2023-10-11T09:44:06.420014+0000","last_became_peered":"2023-10-11T09:44:06.420014+0000","last_unstale":"2023-10-11T09:44:06.863543+0000","last_undegraded":"2023-10-11T09:44:06.863543+0000","last_fullsized":"2023-10-11T09:44:06.863543+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:35:54.243262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.896260+0000","last_change":"2023-10-11T09:44:06.888192+0000","last_active":"2023-10-11T09:44:06.896260+0000","last_peered":"2023-10-11T09:44:06.896260+0000","last_clean":"2023-10-11T09:44:06.896260+0000","last_became_active":"2023-10-11T09:44:06.887861+0000","last_became_peered":"2023-10-11T09:44:06.887861+0000","last_unstale":"2023-10-11T09:44:06.896260+0000","last_undegraded":"2023-10-11T09:44:06.896260+0000","last_fullsized":"2023-10-11T09:44:06.896260+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:28:37.019905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"60'3","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.876556+0000","last_change":"2023-10-11T09:44:06.417241+0000","last_active":"2023-10-11T09:44:07.876556+0000","last_peered":"2023-10-11T09:44:07.876556+0000","last_clean":"2023-10-11T09:44:07.876556+0000","last_became_active":"2023-10-11T09:44:06.416977+0000","last_became_peered":"2023-10-11T09:44:06.416977+0000","last_unstale":"2023-10-11T09:44:07.876556+0000","last_undegraded":"2023-10-11T09:44:07.876556+0000","last_fullsized":"2023-10-11T09:44:07.876556+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:15:59.977205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.876403+0000","last_change":"2023-10-11T09:44:06.888984+0000","last_active":"2023-10-11T09:44:07.876403+0000","last_peered":"2023-10-11T09:44:07.876403+0000","last_clean":"2023-10-11T09:44:07.876403+0000","last_became_active":"2023-10-11T09:44:06.888598+0000","last_became_peered":"2023-10-11T09:44:06.888598+0000","last_unstale":"2023-10-11T09:44:07.876403+0000","last_undegraded":"2023-10-11T09:44:07.876403+0000","last_fullsized":"2023-10-11T09:44:07.876403+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:58:17.041296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"60'6","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.874803+0000","last_change":"2023-10-11T09:44:06.424833+0000","last_active":"2023-10-11T09:44:07.874803+0000","last_peered":"2023-10-11T09:44:07.874803+0000","last_clean":"2023-10-11T09:44:07.874803+0000","last_became_active":"2023-10-11T09:44:06.424549+0000","last_became_peered":"2023-10-11T09:44:06.424549+0000","last_unstale":"2023-10-11T09:44:07.874803+0000","last_undegraded":"2023-10-11T09:44:07.874803+0000","last_fullsized":"2023-10-11T09:44:07.874803+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:02:06.108614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.901763+0000","last_change":"2023-10-11T09:44:06.889524+0000","last_active":"2023-10-11T09:44:06.901763+0000","last_peered":"2023-10-11T09:44:06.901763+0000","last_clean":"2023-10-11T09:44:06.901763+0000","last_became_active":"2023-10-11T09:44:06.889184+0000","last_became_peered":"2023-10-11T09:44:06.889184+0000","last_unstale":"2023-10-11T09:44:06.901763+0000","last_undegraded":"2023-10-11T09:44:06.901763+0000","last_fullsized":"2023-10-11T09:44:06.901763+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:00:38.019876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"60'2","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.887925+0000","last_change":"2023-10-11T09:44:06.420736+0000","last_active":"2023-10-11T09:44:06.887925+0000","last_peered":"2023-10-11T09:44:06.887925+0000","last_clean":"2023-10-11T09:44:06.887925+0000","last_became_active":"2023-10-11T09:44:06.420497+0000","last_became_peered":"2023-10-11T09:44:06.420497+0000","last_unstale":"2023-10-11T09:44:06.887925+0000","last_undegraded":"2023-10-11T09:44:06.887925+0000","last_fullsized":"2023-10-11T09:44:06.887925+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:53:48.608829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.876389+0000","last_change":"2023-10-11T09:44:06.880910+0000","last_active":"2023-10-11T09:44:07.876389+0000","last_peered":"2023-10-11T09:44:07.876389+0000","last_clean":"2023-10-11T09:44:07.876389+0000","last_became_active":"2023-10-11T09:44:06.879820+0000","last_became_peered":"2023-10-11T09:44:06.879820+0000","last_unstale":"2023-10-11T09:44:07.876389+0000","last_undegraded":"2023-10-11T09:44:07.876389+0000","last_fullsized":"2023-10-11T09:44:07.876389+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:50:53.336251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.872418+0000","last_change":"2023-10-11T09:44:06.415138+0000","last_active":"2023-10-11T09:44:06.872418+0000","last_peered":"2023-10-11T09:44:06.872418+0000","last_clean":"2023-10-11T09:44:06.872418+0000","last_became_active":"2023-10-11T09:44:06.414820+0000","last_became_peered":"2023-10-11T09:44:06.414820+0000","last_unstale":"2023-10-11T09:44:06.872418+0000","last_undegraded":"2023-10-11T09:44:06.872418+0000","last_fullsized":"2023-10-11T09:44:06.872418+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:15:48.644460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.874170+0000","last_change":"2023-10-11T09:44:06.873793+0000","last_active":"2023-10-11T09:44:07.874170+0000","last_peered":"2023-10-11T09:44:07.874170+0000","last_clean":"2023-10-11T09:44:07.874170+0000","last_became_active":"2023-10-11T09:44:06.873511+0000","last_became_peered":"2023-10-11T09:44:06.873511+0000","last_unstale":"2023-10-11T09:44:07.874170+0000","last_undegraded":"2023-10-11T09:44:07.874170+0000","last_fullsized":"2023-10-11T09:44:07.874170+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:16:06.882025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.878196+0000","last_change":"2023-10-11T09:44:06.418154+0000","last_active":"2023-10-11T09:44:06.878196+0000","last_peered":"2023-10-11T09:44:06.878196+0000","last_clean":"2023-10-11T09:44:06.878196+0000","last_became_active":"2023-10-11T09:44:06.416980+0000","last_became_peered":"2023-10-11T09:44:06.416980+0000","last_unstale":"2023-10-11T09:44:06.878196+0000","last_undegraded":"2023-10-11T09:44:06.878196+0000","last_fullsized":"2023-10-11T09:44:06.878196+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:23:44.206245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.876715+0000","last_change":"2023-10-11T09:44:06.887020+0000","last_active":"2023-10-11T09:44:07.876715+0000","last_peered":"2023-10-11T09:44:07.876715+0000","last_clean":"2023-10-11T09:44:07.876715+0000","last_became_active":"2023-10-11T09:44:06.886747+0000","last_became_peered":"2023-10-11T09:44:06.886747+0000","last_unstale":"2023-10-11T09:44:07.876715+0000","last_undegraded":"2023-10-11T09:44:07.876715+0000","last_fullsized":"2023-10-11T09:44:07.876715+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:05:38.172887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"60'8","reported_seq":19,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.876069+0000","last_change":"2023-10-11T09:44:06.424419+0000","last_active":"2023-10-11T09:44:07.876069+0000","last_peered":"2023-10-11T09:44:07.876069+0000","last_clean":"2023-10-11T09:44:07.876069+0000","last_became_active":"2023-10-11T09:44:06.424154+0000","last_became_peered":"2023-10-11T09:44:06.424154+0000","last_unstale":"2023-10-11T09:44:07.876069+0000","last_undegraded":"2023-10-11T09:44:07.876069+0000","last_fullsized":"2023-10-11T09:44:07.876069+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:35:06.314203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.901793+0000","last_change":"2023-10-11T09:44:06.889856+0000","last_active":"2023-10-11T09:44:06.901793+0000","last_peered":"2023-10-11T09:44:06.901793+0000","last_clean":"2023-10-11T09:44:06.901793+0000","last_became_active":"2023-10-11T09:44:06.889547+0000","last_became_peered":"2023-10-11T09:44:06.889547+0000","last_unstale":"2023-10-11T09:44:06.901793+0000","last_undegraded":"2023-10-11T09:44:06.901793+0000","last_fullsized":"2023-10-11T09:44:06.901793+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:28:43.416105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"60'2","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.897104+0000","last_change":"2023-10-11T09:44:06.416326+0000","last_active":"2023-10-11T09:44:06.897104+0000","last_peered":"2023-10-11T09:44:06.897104+0000","last_clean":"2023-10-11T09:44:06.897104+0000","last_became_active":"2023-10-11T09:44:06.415780+0000","last_became_peered":"2023-10-11T09:44:06.415780+0000","last_unstale":"2023-10-11T09:44:06.897104+0000","last_undegraded":"2023-10-11T09:44:06.897104+0000","last_fullsized":"2023-10-11T09:44:06.897104+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:25:14.688588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.901863+0000","last_change":"2023-10-11T09:44:06.891082+0000","last_active":"2023-10-11T09:44:06.901863+0000","last_peered":"2023-10-11T09:44:06.901863+0000","last_clean":"2023-10-11T09:44:06.901863+0000","last_became_active":"2023-10-11T09:44:06.890806+0000","last_became_peered":"2023-10-11T09:44:06.890806+0000","last_unstale":"2023-10-11T09:44:06.901863+0000","last_undegraded":"2023-10-11T09:44:06.901863+0000","last_fullsized":"2023-10-11T09:44:06.901863+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:21:29.593748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:08.411496+0000","last_change":"2023-10-11T09:44:06.423554+0000","last_active":"2023-10-11T09:44:08.411496+0000","last_peered":"2023-10-11T09:44:08.411496+0000","last_clean":"2023-10-11T09:44:08.411496+0000","last_became_active":"2023-10-11T09:44:06.423239+0000","last_became_peered":"2023-10-11T09:44:06.423239+0000","last_unstale":"2023-10-11T09:44:08.411496+0000","last_undegraded":"2023-10-11T09:44:08.411496+0000","last_fullsized":"2023-10-11T09:44:08.411496+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:26:35.518484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.896239+0000","last_change":"2023-10-11T09:44:06.887802+0000","last_active":"2023-10-11T09:44:06.896239+0000","last_peered":"2023-10-11T09:44:06.896239+0000","last_clean":"2023-10-11T09:44:06.896239+0000","last_became_active":"2023-10-11T09:44:06.887540+0000","last_became_peered":"2023-10-11T09:44:06.887540+0000","last_unstale":"2023-10-11T09:44:06.896239+0000","last_undegraded":"2023-10-11T09:44:06.896239+0000","last_fullsized":"2023-10-11T09:44:06.896239+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:45:00.136041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.872379+0000","last_change":"2023-10-11T09:44:06.416939+0000","last_active":"2023-10-11T09:44:06.872379+0000","last_peered":"2023-10-11T09:44:06.872379+0000","last_clean":"2023-10-11T09:44:06.872379+0000","last_became_active":"2023-10-11T09:44:06.416680+0000","last_became_peered":"2023-10-11T09:44:06.416680+0000","last_unstale":"2023-10-11T09:44:06.872379+0000","last_undegraded":"2023-10-11T09:44:06.872379+0000","last_fullsized":"2023-10-11T09:44:06.872379+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:25:37.480182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:08.411336+0000","last_change":"2023-10-11T09:44:06.883171+0000","last_active":"2023-10-11T09:44:08.411336+0000","last_peered":"2023-10-11T09:44:08.411336+0000","last_clean":"2023-10-11T09:44:08.411336+0000","last_became_active":"2023-10-11T09:44:06.882388+0000","last_became_peered":"2023-10-11T09:44:06.882388+0000","last_unstale":"2023-10-11T09:44:08.411336+0000","last_undegraded":"2023-10-11T09:44:08.411336+0000","last_fullsized":"2023-10-11T09:44:08.411336+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:01:02.059081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.877036+0000","last_change":"2023-10-11T09:44:06.422820+0000","last_active":"2023-10-11T09:44:06.877036+0000","last_peered":"2023-10-11T09:44:06.877036+0000","last_clean":"2023-10-11T09:44:06.877036+0000","last_became_active":"2023-10-11T09:44:06.422574+0000","last_became_peered":"2023-10-11T09:44:06.422574+0000","last_unstale":"2023-10-11T09:44:06.877036+0000","last_undegraded":"2023-10-11T09:44:06.877036+0000","last_fullsized":"2023-10-11T09:44:06.877036+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:04:55.976065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.895896+0000","last_change":"2023-10-11T09:44:06.887573+0000","last_active":"2023-10-11T09:44:06.895896+0000","last_peered":"2023-10-11T09:44:06.895896+0000","last_clean":"2023-10-11T09:44:06.895896+0000","last_became_active":"2023-10-11T09:44:06.887284+0000","last_became_peered":"2023-10-11T09:44:06.887284+0000","last_unstale":"2023-10-11T09:44:06.895896+0000","last_undegraded":"2023-10-11T09:44:06.895896+0000","last_fullsized":"2023-10-11T09:44:06.895896+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:47:18.756116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.883564+0000","last_change":"2023-10-11T09:44:06.422220+0000","last_active":"2023-10-11T09:44:06.883564+0000","last_peered":"2023-10-11T09:44:06.883564+0000","last_clean":"2023-10-11T09:44:06.883564+0000","last_became_active":"2023-10-11T09:44:06.421940+0000","last_became_peered":"2023-10-11T09:44:06.421940+0000","last_unstale":"2023-10-11T09:44:06.883564+0000","last_undegraded":"2023-10-11T09:44:06.883564+0000","last_fullsized":"2023-10-11T09:44:06.883564+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:45:06.217550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.876566+0000","last_change":"2023-10-11T09:44:06.887452+0000","last_active":"2023-10-11T09:44:07.876566+0000","last_peered":"2023-10-11T09:44:07.876566+0000","last_clean":"2023-10-11T09:44:07.876566+0000","last_became_active":"2023-10-11T09:44:06.886790+0000","last_became_peered":"2023-10-11T09:44:06.886790+0000","last_unstale":"2023-10-11T09:44:07.876566+0000","last_undegraded":"2023-10-11T09:44:07.876566+0000","last_fullsized":"2023-10-11T09:44:07.876566+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:54:24.389470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.874671+0000","last_change":"2023-10-11T09:44:06.419796+0000","last_active":"2023-10-11T09:44:07.874671+0000","last_peered":"2023-10-11T09:44:07.874671+0000","last_clean":"2023-10-11T09:44:07.874671+0000","last_became_active":"2023-10-11T09:44:06.419509+0000","last_became_peered":"2023-10-11T09:44:06.419509+0000","last_unstale":"2023-10-11T09:44:07.874671+0000","last_undegraded":"2023-10-11T09:44:07.874671+0000","last_fullsized":"2023-10-11T09:44:07.874671+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:06:53.008484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.893150+0000","last_change":"2023-10-11T09:44:06.885086+0000","last_active":"2023-10-11T09:44:06.893150+0000","last_peered":"2023-10-11T09:44:06.893150+0000","last_clean":"2023-10-11T09:44:06.893150+0000","last_became_active":"2023-10-11T09:44:06.884852+0000","last_became_peered":"2023-10-11T09:44:06.884852+0000","last_unstale":"2023-10-11T09:44:06.893150+0000","last_undegraded":"2023-10-11T09:44:06.893150+0000","last_fullsized":"2023-10-11T09:44:06.893150+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:27:02.672754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.883669+0000","last_change":"2023-10-11T09:44:06.428888+0000","last_active":"2023-10-11T09:44:06.883669+0000","last_peered":"2023-10-11T09:44:06.883669+0000","last_clean":"2023-10-11T09:44:06.883669+0000","last_became_active":"2023-10-11T09:44:06.428556+0000","last_became_peered":"2023-10-11T09:44:06.428556+0000","last_unstale":"2023-10-11T09:44:06.883669+0000","last_undegraded":"2023-10-11T09:44:06.883669+0000","last_fullsized":"2023-10-11T09:44:06.883669+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:15:46.293341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.885924+0000","last_change":"2023-10-11T09:44:06.880433+0000","last_active":"2023-10-11T09:44:06.885924+0000","last_peered":"2023-10-11T09:44:06.885924+0000","last_clean":"2023-10-11T09:44:06.885924+0000","last_became_active":"2023-10-11T09:44:06.879802+0000","last_became_peered":"2023-10-11T09:44:06.879802+0000","last_unstale":"2023-10-11T09:44:06.885924+0000","last_undegraded":"2023-10-11T09:44:06.885924+0000","last_fullsized":"2023-10-11T09:44:06.885924+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T21:14:29.347600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.862599+0000","last_change":"2023-10-11T09:44:06.425553+0000","last_active":"2023-10-11T09:44:06.862599+0000","last_peered":"2023-10-11T09:44:06.862599+0000","last_clean":"2023-10-11T09:44:06.862599+0000","last_became_active":"2023-10-11T09:44:06.425289+0000","last_became_peered":"2023-10-11T09:44:06.425289+0000","last_unstale":"2023-10-11T09:44:06.862599+0000","last_undegraded":"2023-10-11T09:44:06.862599+0000","last_fullsized":"2023-10-11T09:44:06.862599+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:23:44.206245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.882384+0000","last_change":"2023-10-11T09:44:06.871913+0000","last_active":"2023-10-11T09:44:06.882384+0000","last_peered":"2023-10-11T09:44:06.882384+0000","last_clean":"2023-10-11T09:44:06.882384+0000","last_became_active":"2023-10-11T09:44:06.871627+0000","last_became_peered":"2023-10-11T09:44:06.871627+0000","last_unstale":"2023-10-11T09:44:06.882384+0000","last_undegraded":"2023-10-11T09:44:06.882384+0000","last_fullsized":"2023-10-11T09:44:06.882384+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:50:11.352357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"60'1","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.902027+0000","last_change":"2023-10-11T09:44:06.422188+0000","last_active":"2023-10-11T09:44:06.902027+0000","last_peered":"2023-10-11T09:44:06.902027+0000","last_clean":"2023-10-11T09:44:06.902027+0000","last_became_active":"2023-10-11T09:44:06.421885+0000","last_became_peered":"2023-10-11T09:44:06.421885+0000","last_unstale":"2023-10-11T09:44:06.902027+0000","last_undegraded":"2023-10-11T09:44:06.902027+0000","last_fullsized":"2023-10-11T09:44:06.902027+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:16:05.868352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":41,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.410771+0000","last_change":"2023-10-11T09:40:36.151839+0000","last_active":"2023-10-11T09:44:06.410771+0000","last_peered":"2023-10-11T09:44:06.410771+0000","last_clean":"2023-10-11T09:44:06.410771+0000","last_became_active":"2023-10-11T09:40:36.151179+0000","last_became_peered":"2023-10-11T09:40:36.151179+0000","last_unstale":"2023-10-11T09:44:06.410771+0000","last_undegraded":"2023-10-11T09:44:06.410771+0000","last_fullsized":"2023-10-11T09:44:06.410771+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.876303+0000","last_change":"2023-10-11T09:44:06.889607+0000","last_active":"2023-10-11T09:44:07.876303+0000","last_peered":"2023-10-11T09:44:07.876303+0000","last_clean":"2023-10-11T09:44:07.876303+0000","last_became_active":"2023-10-11T09:44:06.888974+0000","last_became_peered":"2023-10-11T09:44:06.888974+0000","last_unstale":"2023-10-11T09:44:07.876303+0000","last_undegraded":"2023-10-11T09:44:07.876303+0000","last_fullsized":"2023-10-11T09:44:07.876303+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:25:40.535782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:08.411240+0000","last_change":"2023-10-11T09:44:06.419836+0000","last_active":"2023-10-11T09:44:08.411240+0000","last_peered":"2023-10-11T09:44:08.411240+0000","last_clean":"2023-10-11T09:44:08.411240+0000","last_became_active":"2023-10-11T09:44:06.419574+0000","last_became_peered":"2023-10-11T09:44:06.419574+0000","last_unstale":"2023-10-11T09:44:08.411240+0000","last_undegraded":"2023-10-11T09:44:08.411240+0000","last_fullsized":"2023-10-11T09:44:08.411240+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:43:49.207902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":43,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.410079+0000","last_change":"2023-10-11T09:40:36.150224+0000","last_active":"2023-10-11T09:44:06.410079+0000","last_peered":"2023-10-11T09:44:06.410079+0000","last_clean":"2023-10-11T09:44:06.410079+0000","last_became_active":"2023-10-11T09:40:36.149919+0000","last_became_peered":"2023-10-11T09:40:36.149919+0000","last_unstale":"2023-10-11T09:44:06.410079+0000","last_undegraded":"2023-10-11T09:44:06.410079+0000","last_fullsized":"2023-10-11T09:44:06.410079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:07.876642+0000","last_change":"2023-10-11T09:44:06.878178+0000","last_active":"2023-10-11T09:44:07.876642+0000","last_peered":"2023-10-11T09:44:07.876642+0000","last_clean":"2023-10-11T09:44:07.876642+0000","last_became_active":"2023-10-11T09:44:06.877875+0000","last_became_peered":"2023-10-11T09:44:06.877875+0000","last_unstale":"2023-10-11T09:44:07.876642+0000","last_undegraded":"2023-10-11T09:44:07.876642+0000","last_fullsized":"2023-10-11T09:44:07.876642+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:26:36.532157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"60'1","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.888757+0000","last_change":"2023-10-11T09:44:06.420099+0000","last_active":"2023-10-11T09:44:06.888757+0000","last_peered":"2023-10-11T09:44:06.888757+0000","last_clean":"2023-10-11T09:44:06.888757+0000","last_became_active":"2023-10-11T09:44:06.419770+0000","last_became_peered":"2023-10-11T09:44:06.419770+0000","last_unstale":"2023-10-11T09:44:06.888757+0000","last_undegraded":"2023-10-11T09:44:06.888757+0000","last_fullsized":"2023-10-11T09:44:06.888757+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:52:00.935219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":53,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.866112+0000","last_change":"2023-10-11T09:40:36.162174+0000","last_active":"2023-10-11T09:44:06.866112+0000","last_peered":"2023-10-11T09:44:06.866112+0000","last_clean":"2023-10-11T09:44:06.866112+0000","last_became_active":"2023-10-11T09:40:36.161710+0000","last_became_peered":"2023-10-11T09:40:36.161710+0000","last_unstale":"2023-10-11T09:44:06.866112+0000","last_undegraded":"2023-10-11T09:44:06.866112+0000","last_fullsized":"2023-10-11T09:44:06.866112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:57:03.630878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.901839+0000","last_change":"2023-10-11T09:44:06.889681+0000","last_active":"2023-10-11T09:44:06.901839+0000","last_peered":"2023-10-11T09:44:06.901839+0000","last_clean":"2023-10-11T09:44:06.901839+0000","last_became_active":"2023-10-11T09:44:06.889173+0000","last_became_peered":"2023-10-11T09:44:06.889173+0000","last_unstale":"2023-10-11T09:44:06.901839+0000","last_undegraded":"2023-10-11T09:44:06.901839+0000","last_fullsized":"2023-10-11T09:44:06.901839+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:37:20.675117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"60'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:08.411403+0000","last_change":"2023-10-11T09:44:06.421620+0000","last_active":"2023-10-11T09:44:08.411403+0000","last_peered":"2023-10-11T09:44:08.411403+0000","last_clean":"2023-10-11T09:44:08.411403+0000","last_became_active":"2023-10-11T09:44:06.421294+0000","last_became_peered":"2023-10-11T09:44:06.421294+0000","last_unstale":"2023-10-11T09:44:08.411403+0000","last_undegraded":"2023-10-11T09:44:08.411403+0000","last_fullsized":"2023-10-11T09:44:08.411403+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:25:39.522109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.407336+0000","last_change":"2023-10-11T09:40:36.160620+0000","last_active":"2023-10-11T09:44:06.407336+0000","last_peered":"2023-10-11T09:44:06.407336+0000","last_clean":"2023-10-11T09:44:06.407336+0000","last_became_active":"2023-10-11T09:40:36.160171+0000","last_became_peered":"2023-10-11T09:40:36.160171+0000","last_unstale":"2023-10-11T09:44:06.407336+0000","last_undegraded":"2023-10-11T09:44:06.407336+0000","last_fullsized":"2023-10-11T09:44:06.407336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:08.410625+0000","last_change":"2023-10-11T09:44:06.881819+0000","last_active":"2023-10-11T09:44:08.410625+0000","last_peered":"2023-10-11T09:44:08.410625+0000","last_clean":"2023-10-11T09:44:08.410625+0000","last_became_active":"2023-10-11T09:44:06.881540+0000","last_became_peered":"2023-10-11T09:44:06.881540+0000","last_unstale":"2023-10-11T09:44:08.410625+0000","last_undegraded":"2023-10-11T09:44:08.410625+0000","last_fullsized":"2023-10-11T09:44:08.410625+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:08:57.620867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.876750+0000","last_change":"2023-10-11T09:44:06.420099+0000","last_active":"2023-10-11T09:44:06.876750+0000","last_peered":"2023-10-11T09:44:06.876750+0000","last_clean":"2023-10-11T09:44:06.876750+0000","last_became_active":"2023-10-11T09:44:06.419821+0000","last_became_peered":"2023-10-11T09:44:06.419821+0000","last_unstale":"2023-10-11T09:44:06.876750+0000","last_undegraded":"2023-10-11T09:44:06.876750+0000","last_fullsized":"2023-10-11T09:44:06.876750+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:42:37.454595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":43,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.418395+0000","last_change":"2023-10-11T09:40:36.151710+0000","last_active":"2023-10-11T09:44:06.418395+0000","last_peered":"2023-10-11T09:44:06.418395+0000","last_clean":"2023-10-11T09:44:06.418395+0000","last_became_active":"2023-10-11T09:40:36.151219+0000","last_became_peered":"2023-10-11T09:40:36.151219+0000","last_unstale":"2023-10-11T09:44:06.418395+0000","last_undegraded":"2023-10-11T09:44:06.418395+0000","last_fullsized":"2023-10-11T09:44:06.418395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.901825+0000","last_change":"2023-10-11T09:44:06.890156+0000","last_active":"2023-10-11T09:44:06.901825+0000","last_peered":"2023-10-11T09:44:06.901825+0000","last_clean":"2023-10-11T09:44:06.901825+0000","last_became_active":"2023-10-11T09:44:06.889898+0000","last_became_peered":"2023-10-11T09:44:06.889898+0000","last_unstale":"2023-10-11T09:44:06.901825+0000","last_undegraded":"2023-10-11T09:44:06.901825+0000","last_fullsized":"2023-10-11T09:44:06.901825+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:12:30.534498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"60'2","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.889245+0000","last_change":"2023-10-11T09:44:06.424956+0000","last_active":"2023-10-11T09:44:06.889245+0000","last_peered":"2023-10-11T09:44:06.889245+0000","last_clean":"2023-10-11T09:44:06.889245+0000","last_became_active":"2023-10-11T09:44:06.424673+0000","last_became_peered":"2023-10-11T09:44:06.424673+0000","last_unstale":"2023-10-11T09:44:06.889245+0000","last_undegraded":"2023-10-11T09:44:06.889245+0000","last_fullsized":"2023-10-11T09:44:06.889245+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:02:06.108614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":44,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.870195+0000","last_change":"2023-10-11T09:40:36.216248+0000","last_active":"2023-10-11T09:44:06.870195+0000","last_peered":"2023-10-11T09:44:06.870195+0000","last_clean":"2023-10-11T09:44:06.870195+0000","last_became_active":"2023-10-11T09:40:36.215890+0000","last_became_peered":"2023-10-11T09:40:36.215890+0000","last_unstale":"2023-10-11T09:44:06.870195+0000","last_undegraded":"2023-10-11T09:44:06.870195+0000","last_fullsized":"2023-10-11T09:44:06.870195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-11T09:44:08.411271+0000","last_change":"2023-10-11T09:44:06.885598+0000","last_active":"2023-10-11T09:44:08.411271+0000","last_peered":"2023-10-11T09:44:08.411271+0000","last_clean":"2023-10-11T09:44:08.411271+0000","last_became_active":"2023-10-11T09:44:06.885294+0000","last_became_peered":"2023-10-11T09:44:06.885294+0000","last_unstale":"2023-10-11T09:44:08.411271+0000","last_undegraded":"2023-10-11T09:44:08.411271+0000","last_fullsized":"2023-10-11T09:44:08.411271+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:39:13.083637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.884838+0000","last_change":"2023-10-11T09:44:06.420620+0000","last_active":"2023-10-11T09:44:06.884838+0000","last_peered":"2023-10-11T09:44:06.884838+0000","last_clean":"2023-10-11T09:44:06.884838+0000","last_became_active":"2023-10-11T09:44:06.420243+0000","last_became_peered":"2023-10-11T09:44:06.420243+0000","last_unstale":"2023-10-11T09:44:06.884838+0000","last_undegraded":"2023-10-11T09:44:06.884838+0000","last_fullsized":"2023-10-11T09:44:06.884838+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:23:23.149495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":53,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.864353+0000","last_change":"2023-10-11T09:40:36.149739+0000","last_active":"2023-10-11T09:44:06.864353+0000","last_peered":"2023-10-11T09:44:06.864353+0000","last_clean":"2023-10-11T09:44:06.864353+0000","last_became_active":"2023-10-11T09:40:36.149419+0000","last_became_peered":"2023-10-11T09:40:36.149419+0000","last_unstale":"2023-10-11T09:44:06.864353+0000","last_undegraded":"2023-10-11T09:44:06.864353+0000","last_fullsized":"2023-10-11T09:44:06.864353+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:54:50.640362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.889950+0000","last_change":"2023-10-11T09:44:06.882947+0000","last_active":"2023-10-11T09:44:06.889950+0000","last_peered":"2023-10-11T09:44:06.889950+0000","last_clean":"2023-10-11T09:44:06.889950+0000","last_became_active":"2023-10-11T09:44:06.882705+0000","last_became_peered":"2023-10-11T09:44:06.882705+0000","last_unstale":"2023-10-11T09:44:06.889950+0000","last_undegraded":"2023-10-11T09:44:06.889950+0000","last_fullsized":"2023-10-11T09:44:06.889950+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:06.387107+0000","last_clean_scrub_stamp":"2023-10-11T09:44:06.387107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:57:01.167316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.874816+0000","last_change":"2023-10-11T09:44:06.423372+0000","last_active":"2023-10-11T09:44:06.874816+0000","last_peered":"2023-10-11T09:44:06.874816+0000","last_clean":"2023-10-11T09:44:06.874816+0000","last_became_active":"2023-10-11T09:44:06.423093+0000","last_became_peered":"2023-10-11T09:44:06.423093+0000","last_unstale":"2023-10-11T09:44:06.874816+0000","last_undegraded":"2023-10-11T09:44:06.874816+0000","last_fullsized":"2023-10-11T09:44:06.874816+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:44:05.373434+0000","last_clean_scrub_stamp":"2023-10-11T09:44:05.373434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:49:28.537540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":43,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.418441+0000","last_change":"2023-10-11T09:40:36.148773+0000","last_active":"2023-10-11T09:44:06.418441+0000","last_peered":"2023-10-11T09:44:06.418441+0000","last_clean":"2023-10-11T09:44:06.418441+0000","last_became_active":"2023-10-11T09:40:36.148489+0000","last_became_peered":"2023-10-11T09:40:36.148489+0000","last_unstale":"2023-10-11T09:44:06.418441+0000","last_undegraded":"2023-10-11T09:44:06.418441+0000","last_fullsized":"2023-10-11T09:44:06.418441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":206,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-10-11T09:44:06.864461+0000","last_change":"2023-10-11T09:40:29.874372+0000","last_active":"2023-10-11T09:44:06.864461+0000","last_peered":"2023-10-11T09:44:06.864461+0000","last_clean":"2023-10-11T09:44:06.864461+0000","last_became_active":"2023-10-11T09:40:29.873621+0000","last_became_peered":"2023-10-11T09:40:29.873621+0000","last_unstale":"2023-10-11T09:44:06.864461+0000","last_undegraded":"2023-10-11T09:44:06.864461+0000","last_fullsized":"2023-10-11T09:44:06.864461+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_clean_scrub_stamp":"2023-10-11T09:40:28.742276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:22:07.985311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574896,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1288,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1318912,"data_stored":917795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:43:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.135,"15min":1.1850000000000001},"min":{"1min":0.61899999999999999,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.367,"5min":1.903,"15min":1.903},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.167,"5min":1.149,"15min":1.1579999999999999},"min":{"1min":0.84399999999999997,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.5329999999999999,"5min":1.696,"15min":1.696},"last":0.95199999999999996}]},{"osd":1,"last update":"Wed Oct 11 09:43:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.1279999999999999,"15min":1.143},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.484,"5min":2.25,"15min":2.25},"last":1.097},{"interface":"front","average":{"1min":1.105,"5min":1.139,"15min":1.147},"min":{"1min":0.83599999999999997,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.3320000000000001,"5min":1.696,"15min":1.696},"last":0.872}]},{"osd":2,"last update":"Wed Oct 11 09:43:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.1020000000000001,"15min":1.1279999999999999},"min":{"1min":0.67100000000000004,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.4610000000000001,"5min":1.873,"15min":1.873},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.139,"15min":1.153},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.548,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.2230000000000001}]},{"osd":3,"last update":"Wed Oct 11 09:43:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.244,"15min":1.272},"min":{"1min":0.92400000000000004,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.504,"5min":1.724,"15min":1.724},"last":1.123},{"interface":"front","average":{"1min":1.296,"5min":1.2929999999999999,"15min":1.3109999999999999},"min":{"1min":1.093,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.516,"5min":2.1579999999999999,"15min":2.1579999999999999},"last":1.3260000000000001}]},{"osd":4,"last update":"Wed Oct 11 09:43:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.161,"15min":1.198},"min":{"1min":0.74199999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.329,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.101,"5min":1.228,"15min":1.2669999999999999},"min":{"1min":0.81999999999999995,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.393,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":1.29}]},{"osd":5,"last update":"Wed Oct 11 09:43:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.268,"15min":1.2929999999999999},"min":{"1min":0.85999999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.3540000000000001,"5min":1.744,"15min":1.744},"last":1.425},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.331,"15min":1.385},"min":{"1min":0.93100000000000005,"5min":0.872,"15min":0.872},"max":{"1min":1.4319999999999999,"5min":3.0750000000000002,"15min":3.0750000000000002},"last":1.1819999999999999}]},{"osd":6,"last update":"Wed Oct 11 09:43:37 2023","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.427,"15min":1.4690000000000001},"min":{"1min":1.173,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":1.52,"5min":3.137,"15min":3.137},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.329,"5min":1.4019999999999999,"15min":1.4279999999999999},"min":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":1.6879999999999999,"5min":1.839,"15min":1.839},"last":1.3899999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574895,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":700,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":716800,"data_stored":327267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.0069999999999999,"15min":0.998},"min":{"1min":0.77800000000000002,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.466,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.034,"15min":1.052},"min":{"1min":0.68000000000000005,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4330000000000001,"5min":1.524,"15min":1.524},"last":1.028}]},{"osd":1,"last update":"Wed Oct 11 09:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.1890000000000001,"15min":1.1970000000000001},"min":{"1min":0.94499999999999995,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.484,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.077},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.2410000000000001,"15min":1.264},"min":{"1min":1.0429999999999999,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.4630000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.2669999999999999}]},{"osd":2,"last update":"Wed Oct 11 09:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.1399999999999999,"15min":1.1599999999999999},"min":{"1min":0.76900000000000002,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.419,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.123,"5min":1.22,"15min":1.2629999999999999},"min":{"1min":0.85499999999999998,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.6419999999999999,"5min":2.605,"15min":2.605},"last":1.107}]},{"osd":3,"last update":"Wed Oct 11 09:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.1679999999999999,"15min":1.1919999999999999},"min":{"1min":0.89800000000000002,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.5589999999999999,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.2729999999999999,"15min":1.302},"min":{"1min":0.84599999999999997,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.7509999999999999,"5min":3.0819999999999999,"15min":3.0819999999999999},"last":1.292}]},{"osd":4,"last update":"Wed Oct 11 09:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.2809999999999999,"15min":1.3180000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.514,"5min":1.978,"15min":1.978},"last":1.198},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3580000000000001,"15min":1.387},"min":{"1min":0.72399999999999998,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.141,"5min":2.6920000000000002,"15min":2.6920000000000002},"last":1.1779999999999999}]},{"osd":5,"last update":"Wed Oct 11 09:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.2709999999999999,"15min":1.2849999999999999},"min":{"1min":0.76800000000000002,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.6859999999999999,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3999999999999999,"15min":1.4239999999999999},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.823,"5min":2.7290000000000001,"15min":2.7290000000000001},"last":1.329}]},{"osd":7,"last update":"Wed Oct 11 09:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.4159999999999999,"15min":1.4199999999999999},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.895,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.474,"5min":1.476,"15min":1.4830000000000001},"min":{"1min":1.0860000000000001,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.859,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.232}]}]},{"osd":5,"up_from":13,"seq":55834574895,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":724992,"data_stored":327919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:43:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.026,"15min":0.98899999999999999},"min":{"1min":0.86399999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.075,"5min":0.99299999999999999,"15min":0.97099999999999997},"min":{"1min":0.80500000000000005,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.327,"5min":1.637,"15min":1.637},"last":1.22}]},{"osd":1,"last update":"Wed Oct 11 09:43:34 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":0.97799999999999998,"15min":0.94699999999999995},"min":{"1min":0.78700000000000003,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.363,"5min":1.444,"15min":1.444},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.0629999999999999,"15min":1.0589999999999999},"min":{"1min":0.86099999999999999,"5min":0.437,"15min":0.437},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":1.2929999999999999}]},{"osd":2,"last update":"Wed Oct 11 09:43:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.204,"15min":1.2030000000000001},"min":{"1min":0.79600000000000004,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.4890000000000001,"5min":1.831,"15min":1.831},"last":1.173},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.2490000000000001,"15min":1.2450000000000001},"min":{"1min":0.95799999999999996,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.5329999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.248}]},{"osd":3,"last update":"Wed Oct 11 09:43:34 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.1339999999999999,"15min":1.143},"min":{"1min":0.69199999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.506,"5min":1.921,"15min":1.921},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.206,"5min":1.163,"15min":1.165},"min":{"1min":0.93400000000000005,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.4390000000000001,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.0369999999999999}]},{"osd":4,"last update":"Wed Oct 11 09:43:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1699999999999999,"15min":1.1850000000000001},"min":{"1min":0.82399999999999995,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.603,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.1830000000000001,"15min":1.2130000000000001},"min":{"1min":0.73399999999999999,"5min":0.62,"15min":0.62},"max":{"1min":1.391,"5min":1.988,"15min":1.988},"last":1.091}]},{"osd":6,"last update":"Wed Oct 11 09:43:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.254,"15min":1.2849999999999999},"min":{"1min":0.86699999999999999,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.54,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"min":{"1min":1,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.6890000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":1.2050000000000001}]},{"osd":7,"last update":"Wed Oct 11 09:43:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.3740000000000001,"15min":1.425},"min":{"1min":0.90000000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.635,"5min":3.6120000000000001,"15min":3.6120000000000001},"last":1.429},{"interface":"front","average":{"1min":1.397,"5min":1.448,"15min":1.484},"min":{"1min":0.99399999999999999,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.6419999999999999,"5min":3.2109999999999999,"15min":3.2109999999999999},"last":1.528}]}]},{"osd":4,"up_from":13,"seq":55834574895,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":724,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":741376,"data_stored":335484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:43:36 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.0800000000000001,"15min":1.0700000000000001},"min":{"1min":0.70199999999999996,"5min":0.629,"15min":0.629},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1020000000000001,"15min":1.1060000000000001},"min":{"1min":0.70199999999999996,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.466,"5min":2.1160000000000001,"15min":2.1160000000000001},"last":0.65100000000000002}]},{"osd":1,"last update":"Wed Oct 11 09:43:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.0960000000000001,"15min":1.1020000000000001},"min":{"1min":0.73399999999999999,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.238,"5min":1.173,"15min":1.1799999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.635,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.1879999999999999}]},{"osd":2,"last update":"Wed Oct 11 09:43:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.2110000000000001,"15min":1.2},"min":{"1min":0.77700000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.512,"5min":1.919,"15min":1.919},"last":0.876},{"interface":"front","average":{"1min":1.242,"5min":1.2390000000000001,"15min":1.2350000000000001},"min":{"1min":0.86099999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.5409999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.77800000000000002}]},{"osd":3,"last update":"Wed Oct 11 09:43:36 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.153,"15min":1.161},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.3919999999999999,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.296},{"interface":"front","average":{"1min":1.157,"5min":1.284,"15min":1.339},"min":{"1min":0.81399999999999995,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.4419999999999999,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":1.159}]},{"osd":5,"last update":"Wed Oct 11 09:43:36 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.2070000000000001,"15min":1.226},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.577,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1910000000000001,"15min":1.2270000000000001},"min":{"1min":0.78800000000000003,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.5269999999999999,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":1.391}]},{"osd":6,"last update":"Wed Oct 11 09:43:36 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.2709999999999999,"15min":1.2969999999999999},"min":{"1min":0.85999999999999999,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.5109999999999999,"5min":1.8,"15min":1.8},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.3,"5min":1.337,"15min":1.3500000000000001},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.667,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.5209999999999999}]},{"osd":7,"last update":"Wed Oct 11 09:43:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.3320000000000001,"15min":1.365},"min":{"1min":0.93100000000000005,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.4690000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.4570000000000001,"15min":1.496},"min":{"1min":0.995,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.5900000000000001,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":1.2709999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":724992,"data_stored":327323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.83799999999999997,"15min":0.88500000000000001},"min":{"1min":0.43099999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.262,"5min":1.462,"15min":1.462},"last":0.75},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86299999999999999,"15min":0.88600000000000001},"min":{"1min":0.502,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.256,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.82399999999999995}]},{"osd":1,"last update":"Wed Oct 11 09:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.82399999999999995,"15min":0.82799999999999996},"min":{"1min":0.53600000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":1.0700000000000001},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.95299999999999996,"15min":0.96199999999999997},"min":{"1min":0.57599999999999996,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.294,"5min":1.341,"15min":1.341},"last":1.121}]},{"osd":2,"last update":"Wed Oct 11 09:43:36 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":0.98299999999999998,"15min":0.996},"min":{"1min":0.70199999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.383,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.171},{"interface":"front","average":{"1min":1.036,"5min":1.1120000000000001,"15min":1.145},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.3140000000000001,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":0.90800000000000003}]},{"osd":4,"last update":"Wed Oct 11 09:43:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.2050000000000001,"15min":1.2470000000000001},"min":{"1min":0.79000000000000004,"5min":0.754,"15min":0.754},"max":{"1min":1.472,"5min":2.395,"15min":2.395},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.198,"5min":1.2909999999999999,"15min":1.341},"min":{"1min":0.98599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.577,"5min":3.1579999999999999,"15min":3.1579999999999999},"last":1.411}]},{"osd":5,"last update":"Wed Oct 11 09:43:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.3180000000000001,"15min":1.3680000000000001},"min":{"1min":0.83099999999999996,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.7430000000000001,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":1.278},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.3999999999999999,"15min":1.478},"min":{"1min":0.92300000000000004,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.395,"5min":3.8050000000000002,"15min":3.8050000000000002},"last":1.2130000000000001}]},{"osd":6,"last update":"Wed Oct 11 09:43:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.286,"15min":1.3260000000000001},"min":{"1min":0.91300000000000003,"5min":0.75,"15min":0.75},"max":{"1min":1.4370000000000001,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.663},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.3839999999999999,"15min":1.4359999999999999},"min":{"1min":1.0109999999999999,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":1.663,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":1.349}]},{"osd":7,"last update":"Wed Oct 11 09:43:36 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.4470000000000001,"15min":1.5089999999999999},"min":{"1min":0.89600000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.478,"5min":3.7490000000000001,"15min":3.7490000000000001},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.377,"15min":1.4239999999999999},"min":{"1min":0.97399999999999998,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.5349999999999999,"5min":2.625,"15min":2.625},"last":1.4570000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27672,"kb_used_data":720,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":737280,"data_stored":333976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.82099999999999995,"15min":0.83499999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.2370000000000001,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.93700000000000006,"15min":0.98999999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1779999999999999,"5min":2.8919999999999999,"15min":2.8919999999999999},"last":0.82099999999999995}]},{"osd":1,"last update":"Wed Oct 11 09:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":1.0169999999999999,"15min":1.075},"min":{"1min":0.57299999999999995,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.2649999999999999,"5min":3.7650000000000001,"15min":3.7650000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.97899999999999998,"15min":1.0029999999999999},"min":{"1min":0.48999999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.413,"5min":2.5339999999999998,"15min":2.5339999999999998},"last":1.302}]},{"osd":3,"last update":"Wed Oct 11 09:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.083,"15min":1.139},"min":{"1min":0.63300000000000001,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.381,"5min":2.403,"15min":2.403},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.0680000000000001,"15min":1.1000000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.3660000000000001,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.216}]},{"osd":4,"last update":"Wed Oct 11 09:43:33 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.22,"15min":1.234},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.502,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.395,"15min":1.4490000000000001},"min":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":1.651,"5min":2.6859999999999999,"15min":2.6859999999999999},"last":1.0880000000000001}]},{"osd":5,"last update":"Wed Oct 11 09:43:33 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.3400000000000001,"15min":1.3859999999999999},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.6799999999999999,"5min":3.044,"15min":3.044},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.202,"5min":1.306,"15min":1.355},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.5649999999999999,"5min":3.5790000000000002,"15min":3.5790000000000002},"last":1.512}]},{"osd":6,"last update":"Wed Oct 11 09:43:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.3879999999999999,"15min":1.462},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":1.8049999999999999,"5min":3.1600000000000001,"15min":3.1600000000000001},"last":1.022},{"interface":"front","average":{"1min":1.262,"5min":1.353,"15min":1.3819999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.742,"5min":2.839,"15min":2.839},"last":1.484}]},{"osd":7,"last update":"Wed Oct 11 09:43:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.399,"15min":1.4590000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.9079999999999999,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.49,"15min":1.552},"min":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":1.9570000000000001,"5min":2.988,"15min":2.988},"last":1.363}]}]},{"osd":1,"up_from":13,"seq":55834574895,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":720896,"data_stored":327357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:43:34 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.79700000000000004,"15min":0.80800000000000005},"min":{"1min":0.53900000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.236,"5min":1.27,"15min":1.27},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.84999999999999998,"15min":0.85099999999999998},"min":{"1min":0.54300000000000004,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.105,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.995}]},{"osd":2,"last update":"Wed Oct 11 09:43:34 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.86799999999999999,"15min":0.88500000000000001},"min":{"1min":0.629,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.127,"5min":1.292,"15min":1.292},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.98399999999999999,"15min":1.022},"min":{"1min":0.60799999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.403,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.89100000000000001}]},{"osd":3,"last update":"Wed Oct 11 09:43:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0269999999999999,"15min":1.038},"min":{"1min":0.59999999999999998,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.002,"15min":1.002},"min":{"1min":0.60099999999999998,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.5289999999999999,"5min":1.589,"15min":1.589},"last":0.55300000000000005}]},{"osd":4,"last update":"Wed Oct 11 09:43:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.169,"15min":1.2},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.3879999999999999,"5min":2.016,"15min":2.016},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.3009999999999999,"15min":1.337},"min":{"1min":0.84899999999999998,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.5569999999999999,"5min":2.323,"15min":2.323},"last":1.107}]},{"osd":5,"last update":"Wed Oct 11 09:43:34 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.319,"15min":1.349},"min":{"1min":0.95599999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.4590000000000001,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.2649999999999999,"15min":1.292},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.397,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.1339999999999999}]},{"osd":6,"last update":"Wed Oct 11 09:43:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.357,"15min":1.4119999999999999},"min":{"1min":0.97099999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.5249999999999999,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":1.165},{"interface":"front","average":{"1min":1.202,"5min":1.355,"15min":1.403},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.4930000000000001,"5min":3.3719999999999999,"15min":3.3719999999999999},"last":1.0449999999999999}]},{"osd":7,"last update":"Wed Oct 11 09:43:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.405,"15min":1.4470000000000001},"min":{"1min":0.92700000000000005,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.915,"5min":2.194,"15min":2.194},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.335,"5min":1.429,"15min":1.462},"min":{"1min":0.99399999999999999,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":1.5980000000000001,"5min":1.968,"15min":1.968},"last":1.3120000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574894,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32344,"kb_used_data":1296,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339496,"statfs":{"total":96636764160,"available":96603643904,"internally_reserved":0,"allocated":1327104,"data_stored":919781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 11 09:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.80500000000000005,"15min":0.82099999999999995},"min":{"1min":0.56000000000000005,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.135,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.92600000000000005,"15min":0.95299999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.254,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":0.91400000000000003}]},{"osd":2,"last update":"Wed Oct 11 09:43:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.96099999999999997,"15min":0.96899999999999997},"min":{"1min":0.63600000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.365,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.343},{"interface":"front","average":{"1min":1.046,"5min":0.98899999999999999,"15min":0.98399999999999999},"min":{"1min":0.69999999999999996,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.2250000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.244}]},{"osd":3,"last update":"Wed Oct 11 09:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.88200000000000001,"15min":0.91200000000000003},"min":{"1min":0.629,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.079,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.0469999999999999,"15min":1.087},"min":{"1min":0.71699999999999997,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.1899999999999999,"5min":1.996,"15min":1.996},"last":0.99099999999999999}]},{"osd":4,"last update":"Wed Oct 11 09:43:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.1990000000000001,"15min":1.214},"min":{"1min":1.018,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.5649999999999999,"5min":1.744,"15min":1.744},"last":1.5840000000000001},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.22,"15min":1.2509999999999999},"min":{"1min":0.84599999999999997,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.3999999999999999,"5min":2.6629999999999998,"15min":2.6629999999999998},"last":1.216}]},{"osd":5,"last update":"Wed Oct 11 09:43:29 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.3540000000000001,"15min":1.4099999999999999},"min":{"1min":0.96799999999999997,"5min":0.88,"15min":0.88},"max":{"1min":1.448,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":1.6659999999999999},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.3340000000000001,"15min":1.3700000000000001},"min":{"1min":0.99299999999999999,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":1.4930000000000001,"5min":3.601,"15min":3.601},"last":1.3919999999999999}]},{"osd":6,"last update":"Wed Oct 11 09:43:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.24,"15min":1.266},"min":{"1min":0.98299999999999998,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.476,"5min":1.825,"15min":1.825},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.341,"15min":1.377},"min":{"1min":1.105,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":1.5149999999999999,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":1.425}]},{"osd":7,"last update":"Wed Oct 11 09:43:29 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.3260000000000001,"15min":1.3580000000000001},"min":{"1min":0.95099999999999996,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":1.585,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.339,"15min":1.391},"min":{"1min":0.95099999999999996,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.532,"5min":2.5419999999999998,"15min":2.5419999999999998},"last":1.171}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-11T09:44:11.924 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-11T09:44:11.925 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-11T09:44:12.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.138+0000 7fc354cf7700 1 Processor -- start 2023-10-11T09:44:12.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.138+0000 7fc354cf7700 1 -- start start 2023-10-11T09:44:12.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.142+0000 7fc354cf7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc3500cf8d0 0x7fc3500b4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:12.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.142+0000 7fc354cf7700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc350160870 0x7fc350160c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:12.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.142+0000 7fc354cf7700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3500b4fd0 con 0x7fc3500cfca0 2023-10-11T09:44:12.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.142+0000 7fc354cf7700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3500b5870 con 0x7fc350160870 2023-10-11T09:44:12.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.142+0000 7fc354cf7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3500b52b0 con 0x7fc3500cf8d0 2023-10-11T09:44:12.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.142+0000 7fc34effd700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fc3500cfca0 0x7fc3500cd5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33830/0 (socket says 172.21.15.78:33830) 2023-10-11T09:44:12.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.142+0000 7fc34effd700 1 -- 172.21.15.78:0/2890533583 learned_addr learned my addr 172.21.15.78:0/2890533583 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:44:12.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.142+0000 7fc34f7fe700 1 --2- 172.21.15.78:0/2890533583 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc3500cf8d0 0x7fc3500b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:12.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.142+0000 7fc34ffff700 1 --2- 172.21.15.78:0/2890533583 >> v2:172.21.15.133:3300/0 conn(0x7fc350160870 0x7fc350160c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:12.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.142+0000 7fc34e7fc700 1 -- 172.21.15.78:0/2890533583 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 699393319 0 0) 0x7fc3500b4fd0 con 0x7fc3500cfca0 2023-10-11T09:44:12.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.142+0000 7fc34e7fc700 1 -- 172.21.15.78:0/2890533583 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3380035d0 con 0x7fc3500cfca0 2023-10-11T09:44:12.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.142+0000 7fc34f7fe700 1 -- 172.21.15.78:0/2890533583 >> v2:172.21.15.133:3300/0 conn(0x7fc350160870 msgr2=0x7fc350160c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:44:12.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.142+0000 7fc34f7fe700 1 --2- 172.21.15.78:0/2890533583 >> v2:172.21.15.133:3300/0 conn(0x7fc350160870 0x7fc350160c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:12.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.142+0000 7fc34f7fe700 1 -- 172.21.15.78:0/2890533583 >> v1:172.21.15.78:6789/0 conn(0x7fc3500cfca0 legacy=0x7fc3500cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:12.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.142+0000 7fc34f7fe700 1 -- 172.21.15.78:0/2890533583 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3500b5590 con 0x7fc3500cf8d0 2023-10-11T09:44:12.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.142+0000 7fc34f7fe700 1 --2- 172.21.15.78:0/2890533583 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc3500cf8d0 0x7fc3500b4940 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc340009850 tx=0x7fc340009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3bbb2f0c0d67657 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:12.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.146+0000 7fc34e7fc700 1 -- 172.21.15.78:0/2890533583 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc340011830 con 0x7fc3500cf8d0 2023-10-11T09:44:12.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.146+0000 7fc34e7fc700 1 -- 172.21.15.78:0/2890533583 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc340011990 con 0x7fc3500cf8d0 2023-10-11T09:44:12.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.146+0000 7fc34e7fc700 1 -- 172.21.15.78:0/2890533583 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc340011d40 con 0x7fc3500cf8d0 2023-10-11T09:44:12.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.146+0000 7fc354cf7700 1 -- 172.21.15.78:0/2890533583 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc3500cf8d0 msgr2=0x7fc3500b4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:12.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.146+0000 7fc354cf7700 1 --2- 172.21.15.78:0/2890533583 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc3500cf8d0 0x7fc3500b4940 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc340009850 tx=0x7fc340009b60 comp rx=0 tx=0).stop 2023-10-11T09:44:12.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.146+0000 7fc354cf7700 1 -- 172.21.15.78:0/2890533583 shutdown_connections 2023-10-11T09:44:12.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.146+0000 7fc354cf7700 1 --2- 172.21.15.78:0/2890533583 >> v2:172.21.15.133:3300/0 conn(0x7fc350160870 0x7fc350160c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:12.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.146+0000 7fc354cf7700 1 --2- 172.21.15.78:0/2890533583 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc3500cf8d0 0x7fc3500b4940 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:12.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.146+0000 7fc354cf7700 1 -- 172.21.15.78:0/2890533583 >> 172.21.15.78:0/2890533583 conn(0x7fc35006b010 msgr2=0x7fc35006b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:12.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.146+0000 7fc354cf7700 1 -- 172.21.15.78:0/2890533583 shutdown_connections 2023-10-11T09:44:12.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.146+0000 7fc354cf7700 1 -- 172.21.15.78:0/2890533583 wait complete. 2023-10-11T09:44:12.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.146+0000 7fc354cf7700 1 Processor -- start 2023-10-11T09:44:12.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.150+0000 7fc354cf7700 1 -- start start 2023-10-11T09:44:12.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.150+0000 7fc354cf7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3500cf8d0 0x7fc35015f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:12.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.150+0000 7fc354cf7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3500cfca0 0x7fc3500c73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:12.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.150+0000 7fc354cf7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc350160870 0x7fc3500c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:12.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.150+0000 7fc354cf7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3500c7910 con 0x7fc3500cf8d0 2023-10-11T09:44:12.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.150+0000 7fc354cf7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35015fb70 con 0x7fc3500cfca0 2023-10-11T09:44:12.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.150+0000 7fc354cf7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3500c7d00 con 0x7fc350160870 2023-10-11T09:44:12.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.150+0000 7fc34f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3500cf8d0 0x7fc35015f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:12.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.150+0000 7fc34f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3500cf8d0 0x7fc35015f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35034/0 (socket says 172.21.15.78:35034) 2023-10-11T09:44:12.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.150+0000 7fc34f7fe700 1 -- 172.21.15.78:0/3837699688 learned_addr learned my addr 172.21.15.78:0/3837699688 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:12.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.150+0000 7fc34effd700 1 --2- 172.21.15.78:0/3837699688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3500cfca0 0x7fc3500c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:12.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.150+0000 7fc34f7fe700 1 -- 172.21.15.78:0/3837699688 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc350160870 msgr2=0x7fc3500c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:12.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.150+0000 7fc34f7fe700 1 --2- 172.21.15.78:0/3837699688 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc350160870 0x7fc3500c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:12.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.150+0000 7fc34f7fe700 1 -- 172.21.15.78:0/3837699688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3500cfca0 msgr2=0x7fc3500c73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:12.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.150+0000 7fc34f7fe700 1 --2- 172.21.15.78:0/3837699688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3500cfca0 0x7fc3500c73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:12.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.150+0000 7fc34f7fe700 1 -- 172.21.15.78:0/3837699688 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3400094b0 con 0x7fc3500cf8d0 2023-10-11T09:44:12.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.150+0000 7fc34f7fe700 1 --2- 172.21.15.78:0/3837699688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3500cf8d0 0x7fc35015f490 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fc340014040 tx=0x7fc340004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:12.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.150+0000 7fc337fff700 1 -- 172.21.15.78:0/3837699688 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc340004590 con 0x7fc3500cf8d0 2023-10-11T09:44:12.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.150+0000 7fc337fff700 1 -- 172.21.15.78:0/3837699688 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc34001e070 con 0x7fc3500cf8d0 2023-10-11T09:44:12.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.150+0000 7fc337fff700 1 -- 172.21.15.78:0/3837699688 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3400038e0 con 0x7fc3500cf8d0 2023-10-11T09:44:12.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.150+0000 7fc354cf7700 1 -- 172.21.15.78:0/3837699688 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3500c7fe0 con 0x7fc3500cf8d0 2023-10-11T09:44:12.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.150+0000 7fc354cf7700 1 -- 172.21.15.78:0/3837699688 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3500c85a0 con 0x7fc3500cf8d0 2023-10-11T09:44:12.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.154+0000 7fc354cf7700 1 -- 172.21.15.78:0/3837699688 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc350065d40 con 0x7fc3500cf8d0 2023-10-11T09:44:12.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.154+0000 7fc337fff700 1 -- 172.21.15.78:0/3837699688 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fc340003a40 con 0x7fc3500cf8d0 2023-10-11T09:44:12.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.154+0000 7fc337fff700 1 --2- 172.21.15.78:0/3837699688 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc33c065ed0 0x7fc33c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:12.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.158+0000 7fc337fff700 1 -- 172.21.15.78:0/3837699688 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc340020070 con 0x7fc3500cf8d0 2023-10-11T09:44:12.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.158+0000 7fc34effd700 1 --2- 172.21.15.78:0/3837699688 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc33c065ed0 0x7fc33c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:12.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.158+0000 7fc34effd700 1 --2- 172.21.15.78:0/3837699688 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc33c065ed0 0x7fc33c068380 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc3500b8930 tx=0x7fc344009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:12.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.162+0000 7fc337fff700 1 -- 172.21.15.78:0/3837699688 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fc340052300 con 0x7fc3500cf8d0 2023-10-11T09:44:12.372 INFO:tasks.ceph.mds.c.smithi078.stderr:2023-10-11T09:44:12.366+0000 7f2bfe02d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-11T09:44:12.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.406+0000 7fc354cf7700 1 -- 172.21.15.78:0/3837699688 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc35006d350 con 0x7fc3500cf8d0 2023-10-11T09:44:12.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.414+0000 7fc337fff700 1 -- 172.21.15.78:0/3837699688 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+0 (secure 0 0 0) 0x7fc3400150e0 con 0x7fc3500cf8d0 2023-10-11T09:44:12.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.418+0000 7fc354cf7700 1 -- 172.21.15.78:0/3837699688 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc33c065ed0 msgr2=0x7fc33c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:12.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.418+0000 7fc354cf7700 1 --2- 172.21.15.78:0/3837699688 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc33c065ed0 0x7fc33c068380 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc3500b8930 tx=0x7fc344009040 comp rx=0 tx=0).stop 2023-10-11T09:44:12.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.418+0000 7fc354cf7700 1 -- 172.21.15.78:0/3837699688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3500cf8d0 msgr2=0x7fc35015f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:12.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.418+0000 7fc354cf7700 1 --2- 172.21.15.78:0/3837699688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3500cf8d0 0x7fc35015f490 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fc340014040 tx=0x7fc340004770 comp rx=0 tx=0).stop 2023-10-11T09:44:12.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.418+0000 7fc354cf7700 1 -- 172.21.15.78:0/3837699688 shutdown_connections 2023-10-11T09:44:12.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.418+0000 7fc354cf7700 1 --2- 172.21.15.78:0/3837699688 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc33c065ed0 0x7fc33c068380 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:12.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.418+0000 7fc354cf7700 1 --2- 172.21.15.78:0/3837699688 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc350160870 0x7fc3500c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:12.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.418+0000 7fc354cf7700 1 --2- 172.21.15.78:0/3837699688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3500cfca0 0x7fc3500c73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:12.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.418+0000 7fc354cf7700 1 --2- 172.21.15.78:0/3837699688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3500cf8d0 0x7fc35015f490 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:12.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.418+0000 7fc354cf7700 1 -- 172.21.15.78:0/3837699688 >> 172.21.15.78:0/3837699688 conn(0x7fc35006b010 msgr2=0x7fc350155da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:12.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.418+0000 7fc354cf7700 1 -- 172.21.15.78:0/3837699688 shutdown_connections 2023-10-11T09:44:12.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.418+0000 7fc354cf7700 1 -- 172.21.15.78:0/3837699688 wait complete. 2023-10-11T09:44:12.427 INFO:teuthology.orchestra.run.smithi078.stderr:updated caps for client.0 2023-10-11T09:44:12.446 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:44:12.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.614+0000 7f08956eb700 1 Processor -- start 2023-10-11T09:44:12.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.614+0000 7f08956eb700 1 -- start start 2023-10-11T09:44:12.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.614+0000 7f08956eb700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f089015d050 0x7f08900cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:12.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.614+0000 7f08956eb700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08900b06a0 0x7f08900b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:12.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.614+0000 7f08956eb700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0890160210 con 0x7f0890160c60 2023-10-11T09:44:12.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.614+0000 7f08956eb700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08900ccd60 con 0x7f089015d050 2023-10-11T09:44:12.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.614+0000 7f08956eb700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08900b1000 con 0x7f08900b06a0 2023-10-11T09:44:12.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.614+0000 7f08877fe700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f0890160c60 0x7f089015d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33836/0 (socket says 172.21.15.78:33836) 2023-10-11T09:44:12.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.614+0000 7f08877fe700 1 -- 172.21.15.78:0/351441474 learned_addr learned my addr 172.21.15.78:0/351441474 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:44:12.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.614+0000 7f088ffff700 1 --2- 172.21.15.78:0/351441474 >> v2:172.21.15.78:3300/0 conn(0x7f089015d050 0x7f08900cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:12.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.614+0000 7f0894eea700 1 --2- 172.21.15.78:0/351441474 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08900b06a0 0x7f08900b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:12.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.614+0000 7f088f7fe700 1 -- 172.21.15.78:0/351441474 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2421071092 0 0) 0x7f0890160210 con 0x7f0890160c60 2023-10-11T09:44:12.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.614+0000 7f088f7fe700 1 -- 172.21.15.78:0/351441474 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f087c0035d0 con 0x7f0890160c60 2023-10-11T09:44:12.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.614+0000 7f088ffff700 1 -- 172.21.15.78:0/351441474 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08900b06a0 msgr2=0x7f08900b0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:44:12.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.614+0000 7f088ffff700 1 --2- 172.21.15.78:0/351441474 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08900b06a0 0x7f08900b0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:12.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.614+0000 7f088ffff700 1 -- 172.21.15.78:0/351441474 >> v1:172.21.15.78:6789/0 conn(0x7f0890160c60 legacy=0x7f089015d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:12.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.614+0000 7f088ffff700 1 -- 172.21.15.78:0/351441474 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08900ccfc0 con 0x7f089015d050 2023-10-11T09:44:12.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.618+0000 7f088ffff700 1 --2- 172.21.15.78:0/351441474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f089015d050 0x7f08900cc820 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f0880009850 tx=0x7f0880009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6529b8239993c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:12.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.618+0000 7f088f7fe700 1 -- 172.21.15.78:0/351441474 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0880011830 con 0x7f089015d050 2023-10-11T09:44:12.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.618+0000 7f088f7fe700 1 -- 172.21.15.78:0/351441474 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0880011990 con 0x7f089015d050 2023-10-11T09:44:12.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.618+0000 7f088f7fe700 1 -- 172.21.15.78:0/351441474 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0880011d40 con 0x7f089015d050 2023-10-11T09:44:12.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.618+0000 7f08956eb700 1 -- 172.21.15.78:0/351441474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f089015d050 msgr2=0x7f08900cc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:12.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.618+0000 7f08956eb700 1 --2- 172.21.15.78:0/351441474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f089015d050 0x7f08900cc820 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f0880009850 tx=0x7f0880009b60 comp rx=0 tx=0).stop 2023-10-11T09:44:12.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.618+0000 7f08956eb700 1 -- 172.21.15.78:0/351441474 shutdown_connections 2023-10-11T09:44:12.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.618+0000 7f08956eb700 1 --2- 172.21.15.78:0/351441474 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08900b06a0 0x7f08900b0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:12.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.618+0000 7f08956eb700 1 --2- 172.21.15.78:0/351441474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f089015d050 0x7f08900cc820 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:12.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.618+0000 7f08956eb700 1 -- 172.21.15.78:0/351441474 >> 172.21.15.78:0/351441474 conn(0x7f08900cea60 msgr2=0x7f08900cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:12.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.618+0000 7f08956eb700 1 -- 172.21.15.78:0/351441474 shutdown_connections 2023-10-11T09:44:12.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.618+0000 7f08956eb700 1 -- 172.21.15.78:0/351441474 wait complete. 2023-10-11T09:44:12.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.618+0000 7f08956eb700 1 Processor -- start 2023-10-11T09:44:12.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.622+0000 7f08956eb700 1 -- start start 2023-10-11T09:44:12.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.622+0000 7f08956eb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08900b06a0 0x7f0890155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:12.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.622+0000 7f08956eb700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f089015d050 0x7f089015a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:12.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.622+0000 7f08956eb700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0890160c60 0x7f08900b36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:12.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.622+0000 7f08956eb700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0890155b80 con 0x7f08900b06a0 2023-10-11T09:44:12.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.622+0000 7f08956eb700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089015bbf0 con 0x7f0890160c60 2023-10-11T09:44:12.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.622+0000 7f08956eb700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f089015afe0 con 0x7f089015d050 2023-10-11T09:44:12.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.622+0000 7f088ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08900b06a0 0x7f0890155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:12.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.622+0000 7f088ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08900b06a0 0x7f0890155640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35050/0 (socket says 172.21.15.78:35050) 2023-10-11T09:44:12.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.622+0000 7f088ffff700 1 -- 172.21.15.78:0/3771624193 learned_addr learned my addr 172.21.15.78:0/3771624193 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:12.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.622+0000 7f088ffff700 1 -- 172.21.15.78:0/3771624193 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f089015d050 msgr2=0x7f089015a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:12.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.622+0000 7f08877fe700 1 --2- 172.21.15.78:0/3771624193 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f089015d050 0x7f089015a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:12.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.622+0000 7f0894eea700 1 --2- 172.21.15.78:0/3771624193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0890160c60 0x7f08900b36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:12.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.622+0000 7f088ffff700 1 --2- 172.21.15.78:0/3771624193 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f089015d050 0x7f089015a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:12.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.622+0000 7f088ffff700 1 -- 172.21.15.78:0/3771624193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0890160c60 msgr2=0x7f08900b36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:12.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.622+0000 7f088ffff700 1 --2- 172.21.15.78:0/3771624193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0890160c60 0x7f08900b36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:12.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.622+0000 7f088ffff700 1 -- 172.21.15.78:0/3771624193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08800094b0 con 0x7f08900b06a0 2023-10-11T09:44:12.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.622+0000 7f08877fe700 1 --2- 172.21.15.78:0/3771624193 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f089015d050 0x7f089015a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:44:12.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.622+0000 7f0894eea700 1 --2- 172.21.15.78:0/3771624193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0890160c60 0x7f08900b36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:44:12.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.622+0000 7f088ffff700 1 --2- 172.21.15.78:0/3771624193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08900b06a0 0x7f0890155640 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f08800090b0 tx=0x7f0880004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:12.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.622+0000 7f088d7fa700 1 -- 172.21.15.78:0/3771624193 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0880004590 con 0x7f08900b06a0 2023-10-11T09:44:12.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.622+0000 7f088d7fa700 1 -- 172.21.15.78:0/3771624193 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f088001f070 con 0x7f08900b06a0 2023-10-11T09:44:12.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.622+0000 7f08956eb700 1 -- 172.21.15.78:0/3771624193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f089015b2c0 con 0x7f08900b06a0 2023-10-11T09:44:12.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.622+0000 7f088d7fa700 1 -- 172.21.15.78:0/3771624193 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0880003900 con 0x7f08900b06a0 2023-10-11T09:44:12.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.622+0000 7f08956eb700 1 -- 172.21.15.78:0/3771624193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f089015b880 con 0x7f08900b06a0 2023-10-11T09:44:12.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.626+0000 7f08956eb700 1 -- 172.21.15.78:0/3771624193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0890065d40 con 0x7f08900b06a0 2023-10-11T09:44:12.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.626+0000 7f088d7fa700 1 -- 172.21.15.78:0/3771624193 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f0880003a60 con 0x7f08900b06a0 2023-10-11T09:44:12.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.630+0000 7f088d7fa700 1 --2- 172.21.15.78:0/3771624193 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0870065f00 0x7f08700683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:12.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.630+0000 7f088d7fa700 1 -- 172.21.15.78:0/3771624193 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0880021070 con 0x7f08900b06a0 2023-10-11T09:44:12.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.630+0000 7f08877fe700 1 --2- 172.21.15.78:0/3771624193 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0870065f00 0x7f08700683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:12.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.630+0000 7f08877fe700 1 --2- 172.21.15.78:0/3771624193 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0870065f00 0x7f08700683b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f08901f5770 tx=0x7f0878009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:12.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.634+0000 7f088d7fa700 1 -- 172.21.15.78:0/3771624193 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f0880052600 con 0x7f08900b06a0 2023-10-11T09:44:12.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.834+0000 7f08956eb700 1 -- 172.21.15.78:0/3771624193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08900b1310 con 0x7f08900b06a0 2023-10-11T09:44:12.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.838+0000 7f088d7fa700 1 -- 172.21.15.78:0/3771624193 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4651 (secure 0 0 0) 0x7f0880015090 con 0x7f08900b06a0 2023-10-11T09:44:12.842 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:44:12.843 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":37,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5579,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3954348479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":3954348479},{"type":"v1","addr":"172.21.15.133:6835","nonce":3954348479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6100,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/4029676589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4029676589},{"type":"v1","addr":"172.21.15.133:6834","nonce":4029676589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6249,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/1819646091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":1819646091},{"type":"v1","addr":"172.21.15.78:6836","nonce":1819646091}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:44:06.850489+0000","modified":"2023-10-11T09:44:10.203237+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6252},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6252":{"gid":6252,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6837/852594750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":852594750},{"type":"v1","addr":"172.21.15.78:6837","nonce":852594750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-11T09:44:12.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.842+0000 7f08956eb700 1 -- 172.21.15.78:0/3771624193 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0870065f00 msgr2=0x7f08700683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:12.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.842+0000 7f08956eb700 1 --2- 172.21.15.78:0/3771624193 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0870065f00 0x7f08700683b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f08901f5770 tx=0x7f0878009040 comp rx=0 tx=0).stop 2023-10-11T09:44:12.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.842+0000 7f08956eb700 1 -- 172.21.15.78:0/3771624193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08900b06a0 msgr2=0x7f0890155640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:12.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.842+0000 7f08956eb700 1 --2- 172.21.15.78:0/3771624193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08900b06a0 0x7f0890155640 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f08800090b0 tx=0x7f0880004770 comp rx=0 tx=0).stop 2023-10-11T09:44:12.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.842+0000 7f08956eb700 1 -- 172.21.15.78:0/3771624193 shutdown_connections 2023-10-11T09:44:12.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.842+0000 7f08956eb700 1 --2- 172.21.15.78:0/3771624193 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0870065f00 0x7f08700683b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:12.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.842+0000 7f08956eb700 1 --2- 172.21.15.78:0/3771624193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0890160c60 0x7f08900b36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:12.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.842+0000 7f08956eb700 1 --2- 172.21.15.78:0/3771624193 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f089015d050 0x7f089015a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:12.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.842+0000 7f08956eb700 1 --2- 172.21.15.78:0/3771624193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08900b06a0 0x7f0890155640 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:12.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.842+0000 7f08956eb700 1 -- 172.21.15.78:0/3771624193 >> 172.21.15.78:0/3771624193 conn(0x7f08900cea60 msgr2=0x7f089015f430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:12.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.842+0000 7f08956eb700 1 -- 172.21.15.78:0/3771624193 shutdown_connections 2023-10-11T09:44:12.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:12.842+0000 7f08956eb700 1 -- 172.21.15.78:0/3771624193 wait complete. 2023-10-11T09:44:12.850 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 37 2023-10-11T09:44:13.269 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 37, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-11T09:44:06.850489+0000', 'modified': '2023-10-11T09:44:10.203237+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6252}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6252': {'gid': 6252, 'name': 'c', 'rank': 0, 'incarnation': 33, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.78:6837/852594750', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6835', 'nonce': 852594750}, {'type': 'v1', 'addr': '172.21.15.78:6837', 'nonce': 852594750}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-11T09:44:13.269 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-11T09:44:13.270 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-11T09:44:13.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.442+0000 7ffb27a5e700 1 Processor -- start 2023-10-11T09:44:13.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.442+0000 7ffb27a5e700 1 -- start start 2023-10-11T09:44:13.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.446+0000 7ffb27a5e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ffb20071040 0x7ffb200cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:13.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.446+0000 7ffb27a5e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffb200ce330 0x7ffb2006be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:13.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.446+0000 7ffb27a5e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb2006ffa0 con 0x7ffb20160d40 2023-10-11T09:44:13.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.446+0000 7ffb27a5e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb20070c20 con 0x7ffb20071040 2023-10-11T09:44:13.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.446+0000 7ffb27a5e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb20161110 con 0x7ffb200ce330 2023-10-11T09:44:13.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.446+0000 7ffb24ff9700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ffb20071040 0x7ffb200cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:13.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.446+0000 7ffb24ff9700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ffb20071040 0x7ffb200cddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35056/0 (socket says 172.21.15.78:35056) 2023-10-11T09:44:13.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.446+0000 7ffb24ff9700 1 -- 172.21.15.78:0/2692824048 learned_addr learned my addr 172.21.15.78:0/2692824048 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:13.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.446+0000 7ffb25ffb700 1 --2- 172.21.15.78:0/2692824048 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffb200ce330 0x7ffb2006be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:13.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.446+0000 7ffb24ff9700 1 -- 172.21.15.78:0/2692824048 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffb200ce330 msgr2=0x7ffb2006be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:44:13.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.446+0000 7ffb24ff9700 1 --2- 172.21.15.78:0/2692824048 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffb200ce330 0x7ffb2006be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.446+0000 7ffb24ff9700 1 -- 172.21.15.78:0/2692824048 >> v1:172.21.15.133:6789/0 conn(0x7ffb20160d40 legacy=0x7ffb200cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:13.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.446+0000 7ffb24ff9700 1 -- 172.21.15.78:0/2692824048 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb20070280 con 0x7ffb20071040 2023-10-11T09:44:13.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.446+0000 7ffb24ff9700 1 --2- 172.21.15.78:0/2692824048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb20071040 0x7ffb200cddf0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ffb10009910 tx=0x7ffb10009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9461469cc94f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:13.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.446+0000 7ffb17fff700 1 -- 172.21.15.78:0/2692824048 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1000ba60 con 0x7ffb20071040 2023-10-11T09:44:13.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.446+0000 7ffb17fff700 1 -- 172.21.15.78:0/2692824048 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb1000bbc0 con 0x7ffb20071040 2023-10-11T09:44:13.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.446+0000 7ffb17fff700 1 -- 172.21.15.78:0/2692824048 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb10009e60 con 0x7ffb20071040 2023-10-11T09:44:13.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.450+0000 7ffb27a5e700 1 -- 172.21.15.78:0/2692824048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb20071040 msgr2=0x7ffb200cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:13.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.450+0000 7ffb27a5e700 1 --2- 172.21.15.78:0/2692824048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb20071040 0x7ffb200cddf0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ffb10009910 tx=0x7ffb10009c20 comp rx=0 tx=0).stop 2023-10-11T09:44:13.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.450+0000 7ffb27a5e700 1 -- 172.21.15.78:0/2692824048 shutdown_connections 2023-10-11T09:44:13.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.450+0000 7ffb27a5e700 1 --2- 172.21.15.78:0/2692824048 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffb200ce330 0x7ffb2006be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.450+0000 7ffb27a5e700 1 --2- 172.21.15.78:0/2692824048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb20071040 0x7ffb200cddf0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.450+0000 7ffb27a5e700 1 -- 172.21.15.78:0/2692824048 >> 172.21.15.78:0/2692824048 conn(0x7ffb2006aca0 msgr2=0x7ffb200b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:13.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.450+0000 7ffb27a5e700 1 -- 172.21.15.78:0/2692824048 shutdown_connections 2023-10-11T09:44:13.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.450+0000 7ffb27a5e700 1 -- 172.21.15.78:0/2692824048 wait complete. 2023-10-11T09:44:13.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.450+0000 7ffb27a5e700 1 Processor -- start 2023-10-11T09:44:13.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.450+0000 7ffb27a5e700 1 -- start start 2023-10-11T09:44:13.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.454+0000 7ffb27a5e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffb20071040 0x7ffb201506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:13.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.454+0000 7ffb27a5e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb200ce330 0x7ffb2014c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:13.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.454+0000 7ffb27a5e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb20160d40 0x7ffb200b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:13.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.454+0000 7ffb27a5e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2014d1b0 con 0x7ffb20160d40 2023-10-11T09:44:13.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.454+0000 7ffb27a5e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2014d450 con 0x7ffb200ce330 2023-10-11T09:44:13.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.454+0000 7ffb27a5e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2014d730 con 0x7ffb20071040 2023-10-11T09:44:13.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.454+0000 7ffb25ffb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb20160d40 0x7ffb200b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:13.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.454+0000 7ffb25ffb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb20160d40 0x7ffb200b3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35068/0 (socket says 172.21.15.78:35068) 2023-10-11T09:44:13.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.454+0000 7ffb25ffb700 1 -- 172.21.15.78:0/3919370498 learned_addr learned my addr 172.21.15.78:0/3919370498 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:13.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.454+0000 7ffb257fa700 1 --2- 172.21.15.78:0/3919370498 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffb20071040 0x7ffb201506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:13.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.454+0000 7ffb24ff9700 1 --2- 172.21.15.78:0/3919370498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb200ce330 0x7ffb2014c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:13.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.454+0000 7ffb25ffb700 1 -- 172.21.15.78:0/3919370498 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffb20071040 msgr2=0x7ffb201506a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:13.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.454+0000 7ffb25ffb700 1 --2- 172.21.15.78:0/3919370498 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffb20071040 0x7ffb201506a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.454+0000 7ffb25ffb700 1 -- 172.21.15.78:0/3919370498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb200ce330 msgr2=0x7ffb2014c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:13.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.454+0000 7ffb25ffb700 1 --2- 172.21.15.78:0/3919370498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb200ce330 0x7ffb2014c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.454+0000 7ffb25ffb700 1 -- 172.21.15.78:0/3919370498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb10009570 con 0x7ffb20160d40 2023-10-11T09:44:13.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.454+0000 7ffb257fa700 1 --2- 172.21.15.78:0/3919370498 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffb20071040 0x7ffb201506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:44:13.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.454+0000 7ffb25ffb700 1 --2- 172.21.15.78:0/3919370498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb20160d40 0x7ffb200b3df0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ffb18007e40 tx=0x7ffb1800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:13.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.454+0000 7ffb15ffb700 1 -- 172.21.15.78:0/3919370498 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1800eea0 con 0x7ffb20160d40 2023-10-11T09:44:13.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.454+0000 7ffb15ffb700 1 -- 172.21.15.78:0/3919370498 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb18014420 con 0x7ffb20160d40 2023-10-11T09:44:13.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.454+0000 7ffb15ffb700 1 -- 172.21.15.78:0/3919370498 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb180147b0 con 0x7ffb20160d40 2023-10-11T09:44:13.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.454+0000 7ffb27a5e700 1 -- 172.21.15.78:0/3919370498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb2014da10 con 0x7ffb20160d40 2023-10-11T09:44:13.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.454+0000 7ffb27a5e700 1 -- 172.21.15.78:0/3919370498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb2014dfd0 con 0x7ffb20160d40 2023-10-11T09:44:13.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.458+0000 7ffb15ffb700 1 -- 172.21.15.78:0/3919370498 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7ffb18014910 con 0x7ffb20160d40 2023-10-11T09:44:13.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.458+0000 7ffb15ffb700 1 --2- 172.21.15.78:0/3919370498 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffb04065f00 0x7ffb040683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:13.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.462+0000 7ffb15ffb700 1 -- 172.21.15.78:0/3919370498 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffb18083fe0 con 0x7ffb20160d40 2023-10-11T09:44:13.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.462+0000 7ffb257fa700 1 --2- 172.21.15.78:0/3919370498 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffb04065f00 0x7ffb040683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:13.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.462+0000 7ffb257fa700 1 --2- 172.21.15.78:0/3919370498 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffb04065f00 0x7ffb040683b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ffb20070f60 tx=0x7ffb1c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:13.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.462+0000 7ffb2acf2740 1 Processor -- start 2023-10-11T09:44:13.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.462+0000 7ffb2acf2740 1 -- start start 2023-10-11T09:44:13.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.462+0000 7ffb2acf2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1437eb0 0x16e73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:13.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.462+0000 7ffb2acf2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1437ae0 0x16efd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:13.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.462+0000 7ffb2acf2740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1437710 0x16f0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:13.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.462+0000 7ffb2acf2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x16174a0 con 0x1437ae0 2023-10-11T09:44:13.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.462+0000 7ffb2acf2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x1614c30 con 0x1437eb0 2023-10-11T09:44:13.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.462+0000 7ffb2acf2740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x1610520 con 0x1437710 2023-10-11T09:44:13.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.466+0000 7ffb24ff9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1437ae0 0x16efd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:13.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.466+0000 7ffb24ff9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1437ae0 0x16efd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35080/0 (socket says 172.21.15.78:35080) 2023-10-11T09:44:13.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.466+0000 7ffb24ff9700 1 -- 172.21.15.78:0/2608822353 learned_addr learned my addr 172.21.15.78:0/2608822353 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:13.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.466+0000 7ffb257fa700 1 --2- 172.21.15.78:0/2608822353 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1437eb0 0x16e73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:13.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.466+0000 7ffb25ffb700 1 --2- 172.21.15.78:0/2608822353 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1437710 0x16f0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:13.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.466+0000 7ffb24ff9700 1 -- 172.21.15.78:0/2608822353 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1437710 msgr2=0x16f0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:44:13.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.466+0000 7ffb24ff9700 1 --2- 172.21.15.78:0/2608822353 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1437710 0x16f0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.466+0000 7ffb24ff9700 1 -- 172.21.15.78:0/2608822353 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1437eb0 msgr2=0x16e73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:44:13.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.466+0000 7ffb24ff9700 1 --2- 172.21.15.78:0/2608822353 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1437eb0 0x16e73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.466+0000 7ffb24ff9700 1 -- 172.21.15.78:0/2608822353 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x14839e0 con 0x1437ae0 2023-10-11T09:44:13.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.466+0000 7ffb25ffb700 1 --2- 172.21.15.78:0/2608822353 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1437710 0x16f0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:44:13.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.466+0000 7ffb257fa700 1 --2- 172.21.15.78:0/2608822353 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1437eb0 0x16e73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:44:13.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.466+0000 7ffb24ff9700 1 --2- 172.21.15.78:0/2608822353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1437ae0 0x16efd50 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ffb10013000 tx=0x7ffb10010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5371681b27420b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:13.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.466+0000 7ffb02ffd700 1 -- 172.21.15.78:0/2608822353 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1000b860 con 0x1437ae0 2023-10-11T09:44:13.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.466+0000 7ffb02ffd700 1 -- 172.21.15.78:0/2608822353 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb1000b9c0 con 0x1437ae0 2023-10-11T09:44:13.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.466+0000 7ffb02ffd700 1 -- 172.21.15.78:0/2608822353 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb10004280 con 0x1437ae0 2023-10-11T09:44:13.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.466+0000 7ffb2acf2740 1 -- 172.21.15.78:0/2608822353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1437ae0 msgr2=0x16efd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:13.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.466+0000 7ffb2acf2740 1 --2- 172.21.15.78:0/2608822353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1437ae0 0x16efd50 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ffb10013000 tx=0x7ffb10010000 comp rx=0 tx=0).stop 2023-10-11T09:44:13.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.466+0000 7ffb2acf2740 1 -- 172.21.15.78:0/2608822353 shutdown_connections 2023-10-11T09:44:13.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.466+0000 7ffb2acf2740 1 --2- 172.21.15.78:0/2608822353 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1437710 0x16f0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.466+0000 7ffb2acf2740 1 --2- 172.21.15.78:0/2608822353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1437ae0 0x16efd50 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.466+0000 7ffb2acf2740 1 --2- 172.21.15.78:0/2608822353 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1437eb0 0x16e73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.466+0000 7ffb2acf2740 1 -- 172.21.15.78:0/2608822353 >> 172.21.15.78:0/2608822353 conn(0x169ffa0 msgr2=0x16d1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:13.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.466+0000 7ffb2acf2740 1 -- 172.21.15.78:0/2608822353 shutdown_connections 2023-10-11T09:44:13.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.466+0000 7ffb2acf2740 1 -- 172.21.15.78:0/2608822353 wait complete. 2023-10-11T09:44:13.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.466+0000 7ffb2acf2740 1 -- start start 2023-10-11T09:44:13.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.466+0000 7ffb2acf2740 1 Processor -- start 2023-10-11T09:44:13.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.470+0000 7ffb2acf2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1437710 0x16e5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:13.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.470+0000 7ffb2acf2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1437ae0 0x16a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:13.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.470+0000 7ffb2acf2740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1437eb0 0x16c7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:13.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.470+0000 7ffb2acf2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x15d3b00 con 0x1437710 2023-10-11T09:44:13.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.470+0000 7ffb2acf2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x15d37a0 con 0x1437ae0 2023-10-11T09:44:13.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.470+0000 7ffb2acf2740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x14bcb20 con 0x1437eb0 2023-10-11T09:44:13.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.470+0000 7ffb2acf2740 10 client.0 _ll_register_callbacks cb 0x1609e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:44:13.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.470+0000 7ffb257fa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1437710 0x16e5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:13.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.470+0000 7ffb257fa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1437710 0x16e5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35088/0 (socket says 172.21.15.78:35088) 2023-10-11T09:44:13.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.470+0000 7ffb257fa700 1 -- 172.21.15.78:0/690101916 learned_addr learned my addr 172.21.15.78:0/690101916 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:13.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.470+0000 7ffb25ffb700 1 --2- 172.21.15.78:0/690101916 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1437eb0 0x16c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:13.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.470+0000 7ffb24ff9700 1 --2- 172.21.15.78:0/690101916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1437ae0 0x16a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:13.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.470+0000 7ffb257fa700 1 -- 172.21.15.78:0/690101916 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1437eb0 msgr2=0x16c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:44:13.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.470+0000 7ffb257fa700 1 --2- 172.21.15.78:0/690101916 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1437eb0 0x16c7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.470+0000 7ffb257fa700 1 -- 172.21.15.78:0/690101916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1437ae0 msgr2=0x16a39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:44:13.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.470+0000 7ffb257fa700 1 --2- 172.21.15.78:0/690101916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1437ae0 0x16a39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.470+0000 7ffb257fa700 1 -- 172.21.15.78:0/690101916 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb20070280 con 0x1437710 2023-10-11T09:44:13.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.470+0000 7ffb257fa700 1 --2- 172.21.15.78:0/690101916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1437710 0x16e5f20 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c003a40 tx=0x7ffb1c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65784d6ae465909c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:13.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.470+0000 7ffafbfff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:44:13.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.470+0000 7ffafbfff700 1 -- 172.21.15.78:0/690101916 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c015110 con 0x1437710 2023-10-11T09:44:13.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.470+0000 7ffb2acf2740 1 -- 172.21.15.78:0/690101916 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x16095a0 con 0x1437710 2023-10-11T09:44:13.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.470+0000 7ffafbfff700 1 -- 172.21.15.78:0/690101916 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffb1c015270 con 0x1437710 2023-10-11T09:44:13.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.470+0000 7ffafbfff700 1 -- 172.21.15.78:0/690101916 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb1c0158d0 con 0x1437710 2023-10-11T09:44:13.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.470+0000 7ffb2acf2740 10 client.6405 fetch_fsmap learned FSMap version 37 2023-10-11T09:44:13.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.470+0000 7ffb2acf2740 1 -- 172.21.15.78:0/690101916 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x15d6750 con 0x1437710 2023-10-11T09:44:13.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.474+0000 7ffafbfff700 1 -- 172.21.15.78:0/690101916 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x16095a0 con 0x1437710 2023-10-11T09:44:13.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.474+0000 7ffafbfff700 1 -- 172.21.15.78:0/690101916 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) v1 ==== 3106+0+0 (secure 0 0 0) 0x7ffb1c015a30 con 0x1437710 2023-10-11T09:44:13.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.474+0000 7ffb2acf2740 10 client.6405 fetch_fsmap finished waiting for FSMap version 37 2023-10-11T09:44:13.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.474+0000 7ffb2acf2740 10 client.6405 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-11T09:44:13.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.474+0000 7ffb2acf2740 20 client.6405 populate_metadata read hostname 'smithi078' 2023-10-11T09:44:13.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.474+0000 7ffb2acf2740 1 --2- 172.21.15.78:0/690101916 >> [v2:172.21.15.78:6835/852594750,v1:172.21.15.78:6837/852594750] conn(0x1482790 0x16cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:13.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.474+0000 7ffb2acf2740 4 client.6405 mds_command: new command op to 6252 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-11T09:44:13.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.474+0000 7ffb2acf2740 1 -- 172.21.15.78:0/690101916 --> [v2:172.21.15.78:6835/852594750,v1:172.21.15.78:6837/852594750] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x16092c0 con 0x1482790 2023-10-11T09:44:13.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.474+0000 7ffb24ff9700 1 --2- 172.21.15.78:0/690101916 >> [v2:172.21.15.78:6835/852594750,v1:172.21.15.78:6837/852594750] conn(0x1482790 0x16cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:13.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.474+0000 7ffafbfff700 10 client.6405 ms_handle_connect on v2:172.21.15.78:6835/852594750 2023-10-11T09:44:13.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.474+0000 7ffb24ff9700 1 --2- 172.21.15.78:0/690101916 >> [v2:172.21.15.78:6835/852594750,v1:172.21.15.78:6837/852594750] conn(0x1482790 0x16cbd00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77a57b0643cba65a server_cookie=b2624abbdcaa929 in_seq=0 out_seq=0 2023-10-11T09:44:13.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.474+0000 7ffafbfff700 1 -- 172.21.15.78:0/690101916 <== mds.0 v2:172.21.15.78:6835/852594750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x16092c0 con 0x1482790 2023-10-11T09:44:13.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.474+0000 7ffafbfff700 10 client.6405 handle_command_reply: tid=0 2023-10-11T09:44:13.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.474+0000 7ffb2acf2740 1 client.6405 shutdown 2023-10-11T09:44:13.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.474+0000 7ffb2acf2740 1 -- 172.21.15.78:0/690101916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1437710 msgr2=0x16e5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:13.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.474+0000 7ffb2acf2740 1 --2- 172.21.15.78:0/690101916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1437710 0x16e5f20 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c003a40 tx=0x7ffb1c000ed0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.474+0000 7ffb2acf2740 1 -- 172.21.15.78:0/690101916 shutdown_connections 2023-10-11T09:44:13.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.474+0000 7ffb2acf2740 1 --2- 172.21.15.78:0/690101916 >> [v2:172.21.15.78:6835/852594750,v1:172.21.15.78:6837/852594750] conn(0x1482790 0x16cbd00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.474+0000 7ffb2acf2740 1 --2- 172.21.15.78:0/690101916 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1437eb0 0x16c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.474+0000 7ffb2acf2740 1 --2- 172.21.15.78:0/690101916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1437ae0 0x16a39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.474+0000 7ffb2acf2740 1 --2- 172.21.15.78:0/690101916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1437710 0x16e5f20 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.474+0000 7ffafbfff700 0 client.6405 ms_handle_reset on v2:172.21.15.78:6835/852594750 2023-10-11T09:44:13.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.474+0000 7ffb2acf2740 1 -- 172.21.15.78:0/690101916 >> 172.21.15.78:0/690101916 conn(0x169ffa0 msgr2=0x16ef0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:13.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.478+0000 7ffb2acf2740 1 -- 172.21.15.78:0/690101916 shutdown_connections 2023-10-11T09:44:13.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.478+0000 7ffb2acf2740 1 -- 172.21.15.78:0/690101916 wait complete. 2023-10-11T09:44:13.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.478+0000 7ffb2acf2740 20 client.6405 trim_cache size 0 max 16384 2023-10-11T09:44:13.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.494+0000 7ffb2acf2740 1 Processor -- start 2023-10-11T09:44:13.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.494+0000 7ffb2acf2740 1 -- start start 2023-10-11T09:44:13.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.494+0000 7ffb2acf2740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1437710 0x16c5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:13.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.494+0000 7ffb2acf2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1437ae0 0x16e44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:13.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.498+0000 7ffb2acf2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1437eb0 0x16e4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:13.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.498+0000 7ffb2acf2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x15cdc90 con 0x1437eb0 2023-10-11T09:44:13.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.498+0000 7ffb2acf2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x16ffdc0 con 0x1437ae0 2023-10-11T09:44:13.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.498+0000 7ffb2acf2740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x15dfcd0 con 0x1437710 2023-10-11T09:44:13.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.498+0000 7ffb25ffb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1437eb0 0x16e4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:13.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.498+0000 7ffb24ff9700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1437ae0 0x16e44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:13.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.498+0000 7ffb257fa700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1437710 0x16c5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:13.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.498+0000 7ffb25ffb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1437eb0 0x16e4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35094/0 (socket says 172.21.15.78:35094) 2023-10-11T09:44:13.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.498+0000 7ffb25ffb700 1 -- 172.21.15.78:0/2213060369 learned_addr learned my addr 172.21.15.78:0/2213060369 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:13.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.498+0000 7ffb24ff9700 1 -- 172.21.15.78:0/2213060369 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1437710 msgr2=0x16c5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:44:13.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.498+0000 7ffb24ff9700 1 --2- 172.21.15.78:0/2213060369 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1437710 0x16c5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.498+0000 7ffb24ff9700 1 -- 172.21.15.78:0/2213060369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1437eb0 msgr2=0x16e4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:44:13.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.498+0000 7ffb24ff9700 1 --2- 172.21.15.78:0/2213060369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1437eb0 0x16e4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.498+0000 7ffb24ff9700 1 -- 172.21.15.78:0/2213060369 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb200b4690 con 0x1437ae0 2023-10-11T09:44:13.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.498+0000 7ffb24ff9700 1 --2- 172.21.15.78:0/2213060369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1437ae0 0x16e44d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ffb10000c00 tx=0x7ffb10011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adc9a9f86e480ac9 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:13.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.498+0000 7ffb037fe700 1 -- 172.21.15.78:0/2213060369 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1002a070 con 0x1437ae0 2023-10-11T09:44:13.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.498+0000 7ffb037fe700 1 -- 172.21.15.78:0/2213060369 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb10006ad0 con 0x1437ae0 2023-10-11T09:44:13.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.498+0000 7ffb037fe700 1 -- 172.21.15.78:0/2213060369 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb100040c0 con 0x1437ae0 2023-10-11T09:44:13.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.498+0000 7ffb2acf2740 1 -- 172.21.15.78:0/2213060369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1437ae0 msgr2=0x16e44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:13.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.498+0000 7ffb2acf2740 1 --2- 172.21.15.78:0/2213060369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1437ae0 0x16e44d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ffb10000c00 tx=0x7ffb10011000 comp rx=0 tx=0).stop 2023-10-11T09:44:13.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.498+0000 7ffb2acf2740 1 -- 172.21.15.78:0/2213060369 shutdown_connections 2023-10-11T09:44:13.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.498+0000 7ffb2acf2740 1 --2- 172.21.15.78:0/2213060369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1437eb0 0x16e4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.498+0000 7ffb2acf2740 1 --2- 172.21.15.78:0/2213060369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1437ae0 0x16e44d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.498+0000 7ffb2acf2740 1 --2- 172.21.15.78:0/2213060369 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1437710 0x16c5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.498+0000 7ffb2acf2740 1 -- 172.21.15.78:0/2213060369 >> 172.21.15.78:0/2213060369 conn(0x169ffa0 msgr2=0x16cbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:13.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.498+0000 7ffb2acf2740 1 -- 172.21.15.78:0/2213060369 shutdown_connections 2023-10-11T09:44:13.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.498+0000 7ffb2acf2740 1 -- 172.21.15.78:0/2213060369 wait complete. 2023-10-11T09:44:13.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.502+0000 7ffb2acf2740 1 -- start start 2023-10-11T09:44:13.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.502+0000 7ffb2acf2740 1 Processor -- start 2023-10-11T09:44:13.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.502+0000 7ffb2acf2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1437710 0x16c4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:13.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.502+0000 7ffb2acf2740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1437ae0 0x174b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:13.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.502+0000 7ffb2acf2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1437eb0 0x174b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:13.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.502+0000 7ffb2acf2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x1611ab0 con 0x1437710 2023-10-11T09:44:13.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.502+0000 7ffb2acf2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x15ca760 con 0x1437eb0 2023-10-11T09:44:13.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.502+0000 7ffb2acf2740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x16a8960 con 0x1437ae0 2023-10-11T09:44:13.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.502+0000 7ffb2acf2740 10 client.0 _ll_register_callbacks cb 0x7ffb10009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:44:13.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.502+0000 7ffb257fa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1437710 0x16c4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:13.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.502+0000 7ffb257fa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1437710 0x16c4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35100/0 (socket says 172.21.15.78:35100) 2023-10-11T09:44:13.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.502+0000 7ffb257fa700 1 -- 172.21.15.78:0/3911049394 learned_addr learned my addr 172.21.15.78:0/3911049394 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:13.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.502+0000 7ffb25ffb700 1 --2- 172.21.15.78:0/3911049394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1437eb0 0x174b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:13.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.502+0000 7ffb24ff9700 1 --2- 172.21.15.78:0/3911049394 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1437ae0 0x174b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:13.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.502+0000 7ffb257fa700 1 -- 172.21.15.78:0/3911049394 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1437ae0 msgr2=0x174b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:44:13.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.502+0000 7ffb257fa700 1 --2- 172.21.15.78:0/3911049394 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1437ae0 0x174b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.502+0000 7ffb257fa700 1 -- 172.21.15.78:0/3911049394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1437eb0 msgr2=0x174b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:44:13.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.502+0000 7ffb257fa700 1 --2- 172.21.15.78:0/3911049394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1437eb0 0x174b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.502+0000 7ffb257fa700 1 -- 172.21.15.78:0/3911049394 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x14839e0 con 0x1437710 2023-10-11T09:44:13.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.506+0000 7ffb257fa700 1 --2- 172.21.15.78:0/3911049394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1437710 0x16c4630 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c0159b0 tx=0x7ffb1c005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4617e2647bc22d0c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:13.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.506+0000 7ffafbfff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:44:13.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.506+0000 7ffafbfff700 1 -- 172.21.15.78:0/3911049394 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c015b90 con 0x1437710 2023-10-11T09:44:13.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.506+0000 7ffafbfff700 1 -- 172.21.15.78:0/3911049394 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffb1c024840 con 0x1437710 2023-10-11T09:44:13.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.506+0000 7ffb2acf2740 1 -- 172.21.15.78:0/3911049394 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb1000b9c0 con 0x1437710 2023-10-11T09:44:13.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.506+0000 7ffafbfff700 1 -- 172.21.15.78:0/3911049394 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb1c000ed0 con 0x1437710 2023-10-11T09:44:13.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.506+0000 7ffb2acf2740 10 client.6411 fetch_fsmap learned FSMap version 37 2023-10-11T09:44:13.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.506+0000 7ffb2acf2740 1 -- 172.21.15.78:0/3911049394 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x15bc980 con 0x1437710 2023-10-11T09:44:13.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.506+0000 7ffafbfff700 1 -- 172.21.15.78:0/3911049394 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb1000b9c0 con 0x1437710 2023-10-11T09:44:13.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.506+0000 7ffafbfff700 1 -- 172.21.15.78:0/3911049394 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) v1 ==== 3106+0+0 (secure 0 0 0) 0x7ffb1c024a00 con 0x1437710 2023-10-11T09:44:13.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.506+0000 7ffb2acf2740 10 client.6411 fetch_fsmap finished waiting for FSMap version 37 2023-10-11T09:44:13.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.506+0000 7ffb2acf2740 10 client.6411 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-11T09:44:13.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.506+0000 7ffb2acf2740 20 client.6411 populate_metadata read hostname 'smithi078' 2023-10-11T09:44:13.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.506+0000 7ffb2acf2740 1 --2- 172.21.15.78:0/3911049394 >> [v2:172.21.15.78:6835/852594750,v1:172.21.15.78:6837/852594750] conn(0x16a70c0 0x16a9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:13.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.506+0000 7ffb2acf2740 4 client.6411 mds_command: new command op to 6252 tid=0[{"prefix": "status"}] 2023-10-11T09:44:13.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.506+0000 7ffb2acf2740 1 -- 172.21.15.78:0/3911049394 --> [v2:172.21.15.78:6835/852594750,v1:172.21.15.78:6837/852594750] -- command(tid 0: {"prefix": "status"}) v1 -- 0x148f4e0 con 0x16a70c0 2023-10-11T09:44:13.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.506+0000 7ffb24ff9700 1 --2- 172.21.15.78:0/3911049394 >> [v2:172.21.15.78:6835/852594750,v1:172.21.15.78:6837/852594750] conn(0x16a70c0 0x16a9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:13.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.506+0000 7ffb24ff9700 1 --2- 172.21.15.78:0/3911049394 >> [v2:172.21.15.78:6835/852594750,v1:172.21.15.78:6837/852594750] conn(0x16a70c0 0x16a9110 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80463dee0600ffeb server_cookie=135940e2c401401e in_seq=0 out_seq=0 2023-10-11T09:44:13.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.506+0000 7ffafbfff700 10 client.6411 ms_handle_connect on v2:172.21.15.78:6835/852594750 2023-10-11T09:44:13.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.506+0000 7ffafbfff700 1 -- 172.21.15.78:0/3911049394 <== mds.0 v2:172.21.15.78:6835/852594750 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x148f4e0 con 0x16a70c0 2023-10-11T09:44:13.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.506+0000 7ffafbfff700 10 client.6411 handle_command_reply: tid=0 2023-10-11T09:44:13.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.506+0000 7ffb2acf2740 1 client.6411 shutdown 2023-10-11T09:44:13.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.506+0000 7ffb2acf2740 1 -- 172.21.15.78:0/3911049394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1437710 msgr2=0x16c4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:13.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.506+0000 7ffb2acf2740 1 --2- 172.21.15.78:0/3911049394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1437710 0x16c4630 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c0159b0 tx=0x7ffb1c005000 comp rx=0 tx=0).stop 2023-10-11T09:44:13.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.506+0000 7ffb2acf2740 1 -- 172.21.15.78:0/3911049394 shutdown_connections 2023-10-11T09:44:13.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.506+0000 7ffb2acf2740 1 --2- 172.21.15.78:0/3911049394 >> [v2:172.21.15.78:6835/852594750,v1:172.21.15.78:6837/852594750] conn(0x16a70c0 0x16a9110 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.506+0000 7ffb2acf2740 1 --2- 172.21.15.78:0/3911049394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1437eb0 0x174b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.506+0000 7ffb2acf2740 1 --2- 172.21.15.78:0/3911049394 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1437ae0 0x174b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.506+0000 7ffb2acf2740 1 --2- 172.21.15.78:0/3911049394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1437710 0x16c4630 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.506+0000 7ffafbfff700 0 client.6411 ms_handle_reset on v2:172.21.15.78:6835/852594750 2023-10-11T09:44:13.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.506+0000 7ffb2acf2740 1 -- 172.21.15.78:0/3911049394 >> 172.21.15.78:0/3911049394 conn(0x169ffa0 msgr2=0x16a4280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:13.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.506+0000 7ffb2acf2740 1 -- 172.21.15.78:0/3911049394 shutdown_connections 2023-10-11T09:44:13.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.506+0000 7ffb2acf2740 1 -- 172.21.15.78:0/3911049394 wait complete. 2023-10-11T09:44:13.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.506+0000 7ffb2acf2740 20 client.6411 trim_cache size 0 max 16384 2023-10-11T09:44:13.525 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:44:13.525 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "dee99db9-7a72-406f-a1e2-917943f2b13d", 2023-10-11T09:44:13.525 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2023-10-11T09:44:13.525 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 6252, 2023-10-11T09:44:13.526 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2023-10-11T09:44:13.526 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2023-10-11T09:44:13.526 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2023-10-11T09:44:13.526 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 6.6531727119999999, 2023-10-11T09:44:13.526 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 37, 2023-10-11T09:44:13.527 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 61, 2023-10-11T09:44:13.527 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 61, 2023-10-11T09:44:13.527 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 11.916036463999999 2023-10-11T09:44:13.527 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:44:13.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.510+0000 7ffb27a5e700 1 -- 172.21.15.78:0/3919370498 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffb04065f00 msgr2=0x7ffb040683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:13.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.510+0000 7ffb27a5e700 1 --2- 172.21.15.78:0/3919370498 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffb04065f00 0x7ffb040683b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ffb20070f60 tx=0x7ffb1c009040 comp rx=0 tx=0).stop 2023-10-11T09:44:13.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.510+0000 7ffb27a5e700 1 -- 172.21.15.78:0/3919370498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb20160d40 msgr2=0x7ffb200b3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:13.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.510+0000 7ffb27a5e700 1 --2- 172.21.15.78:0/3919370498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb20160d40 0x7ffb200b3df0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ffb18007e40 tx=0x7ffb1800e3c0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.510+0000 7ffb27a5e700 1 -- 172.21.15.78:0/3919370498 shutdown_connections 2023-10-11T09:44:13.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.510+0000 7ffb27a5e700 1 --2- 172.21.15.78:0/3919370498 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffb04065f00 0x7ffb040683b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.510+0000 7ffb27a5e700 1 --2- 172.21.15.78:0/3919370498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb20160d40 0x7ffb200b3df0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.510+0000 7ffb27a5e700 1 --2- 172.21.15.78:0/3919370498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb200ce330 0x7ffb2014c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.510+0000 7ffb27a5e700 1 --2- 172.21.15.78:0/3919370498 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffb20071040 0x7ffb201506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.510+0000 7ffb27a5e700 1 -- 172.21.15.78:0/3919370498 >> 172.21.15.78:0/3919370498 conn(0x7ffb2006aca0 msgr2=0x7ffb20161350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:13.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.510+0000 7ffb27a5e700 1 -- 172.21.15.78:0/3919370498 shutdown_connections 2023-10-11T09:44:13.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.510+0000 7ffb27a5e700 1 -- 172.21.15.78:0/3919370498 wait complete. 2023-10-11T09:44:13.536 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-11T09:44:13.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.702+0000 7fdb3569f700 1 Processor -- start 2023-10-11T09:44:13.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.702+0000 7fdb3569f700 1 -- start start 2023-10-11T09:44:13.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.702+0000 7fdb3569f700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdb3015d050 0x7fdb300cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:13.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.702+0000 7fdb3569f700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb30160210 con 0x7fdb30160c60 2023-10-11T09:44:13.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.702+0000 7fdb3569f700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb300ccd60 con 0x7fdb300b06a0 2023-10-11T09:44:13.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.702+0000 7fdb3569f700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb300ccfc0 con 0x7fdb3015d050 2023-10-11T09:44:13.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.702+0000 7fdb277fe700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fdb30160c60 0x7fdb3015d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33840/0 (socket says 172.21.15.78:33840) 2023-10-11T09:44:13.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.702+0000 7fdb277fe700 1 -- 172.21.15.78:0/1852395262 learned_addr learned my addr 172.21.15.78:0/1852395262 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:44:13.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.702+0000 7fdb2ffff700 1 --2- 172.21.15.78:0/1852395262 >> v2:172.21.15.133:3300/0 conn(0x7fdb3015d050 0x7fdb300cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:13.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.702+0000 7fdb2f7fe700 1 -- 172.21.15.78:0/1852395262 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2372217085 0 0) 0x7fdb30160210 con 0x7fdb30160c60 2023-10-11T09:44:13.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.702+0000 7fdb2f7fe700 1 -- 172.21.15.78:0/1852395262 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb1c0035d0 con 0x7fdb30160c60 2023-10-11T09:44:13.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.706+0000 7fdb2f7fe700 1 -- 172.21.15.78:0/1852395262 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 482854245 0 0) 0x7fdb300ccd60 con 0x7fdb300b06a0 2023-10-11T09:44:13.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.706+0000 7fdb2f7fe700 1 -- 172.21.15.78:0/1852395262 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb30160210 con 0x7fdb300b06a0 2023-10-11T09:44:13.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.706+0000 7fdb2f7fe700 1 -- 172.21.15.78:0/1852395262 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1044279279 0 0) 0x7fdb1c0035d0 con 0x7fdb30160c60 2023-10-11T09:44:13.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.706+0000 7fdb2f7fe700 1 -- 172.21.15.78:0/1852395262 >> v2:172.21.15.133:3300/0 conn(0x7fdb3015d050 msgr2=0x7fdb300cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:44:13.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.706+0000 7fdb2f7fe700 1 --2- 172.21.15.78:0/1852395262 >> v2:172.21.15.133:3300/0 conn(0x7fdb3015d050 0x7fdb300cc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.706+0000 7fdb2f7fe700 1 -- 172.21.15.78:0/1852395262 >> v1:172.21.15.133:6789/0 conn(0x7fdb300b06a0 legacy=0x7fdb300b0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:13.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.706+0000 7fdb2f7fe700 1 -- 172.21.15.78:0/1852395262 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb300cd2a0 con 0x7fdb30160c60 2023-10-11T09:44:13.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.706+0000 7fdb2f7fe700 1 -- 172.21.15.78:0/1852395262 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7fdb18001670 con 0x7fdb30160c60 2023-10-11T09:44:13.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.706+0000 7fdb2f7fe700 1 -- 172.21.15.78:0/1852395262 >> v1:172.21.15.78:6789/0 conn(0x7fdb30160c60 legacy=0x7fdb3015d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:13.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.706+0000 7fdb2f7fe700 1 --2- 172.21.15.78:0/1852395262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb1c003c00 0x7fdb1c003ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:13.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.706+0000 7fdb2f7fe700 1 --2- 172.21.15.78:0/1852395262 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdb1c004870 0x7fdb1c004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:13.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.706+0000 7fdb2f7fe700 1 --2- 172.21.15.78:0/1852395262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb1c005220 0x7fdb1c00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:13.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.706+0000 7fdb2f7fe700 1 -- 172.21.15.78:0/1852395262 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb300ccfc0 con 0x7fdb1c003c00 2023-10-11T09:44:13.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.706+0000 7fdb2f7fe700 1 -- 172.21.15.78:0/1852395262 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1c00c100 con 0x7fdb1c005220 2023-10-11T09:44:13.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.706+0000 7fdb2f7fe700 1 -- 172.21.15.78:0/1852395262 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1c00c290 con 0x7fdb1c004870 2023-10-11T09:44:13.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.706+0000 7fdb2ffff700 1 --2- 172.21.15.78:0/1852395262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb1c003c00 0x7fdb1c003ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:13.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.706+0000 7fdb34e9e700 1 --2- 172.21.15.78:0/1852395262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb1c005220 0x7fdb1c00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:13.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.706+0000 7fdb277fe700 1 --2- 172.21.15.78:0/1852395262 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdb1c004870 0x7fdb1c004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:13.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.706+0000 7fdb2ffff700 1 -- 172.21.15.78:0/1852395262 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdb1c004870 msgr2=0x7fdb1c004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:44:13.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.706+0000 7fdb2ffff700 1 --2- 172.21.15.78:0/1852395262 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdb1c004870 0x7fdb1c004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.706+0000 7fdb2ffff700 1 -- 172.21.15.78:0/1852395262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb1c005220 msgr2=0x7fdb1c00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:44:13.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.706+0000 7fdb2ffff700 1 --2- 172.21.15.78:0/1852395262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb1c005220 0x7fdb1c00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.706+0000 7fdb34e9e700 1 -- 172.21.15.78:0/1852395262 reap_dead start 2023-10-11T09:44:13.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.706+0000 7fdb2ffff700 1 -- 172.21.15.78:0/1852395262 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb1c0035d0 con 0x7fdb1c003c00 2023-10-11T09:44:13.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.706+0000 7fdb2ffff700 1 --2- 172.21.15.78:0/1852395262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb1c003c00 0x7fdb1c003ff0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fdb20009b40 tx=0x7fdb1c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81d2b7765b02536 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:13.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.706+0000 7fdb2f7fe700 1 -- 172.21.15.78:0/1852395262 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2000e6c0 con 0x7fdb1c003c00 2023-10-11T09:44:13.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.706+0000 7fdb2f7fe700 1 -- 172.21.15.78:0/1852395262 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2000e820 con 0x7fdb1c003c00 2023-10-11T09:44:13.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.706+0000 7fdb2f7fe700 1 -- 172.21.15.78:0/1852395262 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2000eb90 con 0x7fdb1c003c00 2023-10-11T09:44:13.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.710+0000 7fdb3569f700 1 -- 172.21.15.78:0/1852395262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb1c003c00 msgr2=0x7fdb1c003ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:13.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.710+0000 7fdb3569f700 1 --2- 172.21.15.78:0/1852395262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb1c003c00 0x7fdb1c003ff0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fdb20009b40 tx=0x7fdb1c0058a0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.710+0000 7fdb3569f700 1 -- 172.21.15.78:0/1852395262 shutdown_connections 2023-10-11T09:44:13.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.710+0000 7fdb3569f700 1 --2- 172.21.15.78:0/1852395262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb1c003c00 0x7fdb1c003ff0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.710+0000 7fdb3569f700 1 -- 172.21.15.78:0/1852395262 >> 172.21.15.78:0/1852395262 conn(0x7fdb300cea60 msgr2=0x7fdb300cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:13.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.710+0000 7fdb3569f700 1 -- 172.21.15.78:0/1852395262 shutdown_connections 2023-10-11T09:44:13.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.710+0000 7fdb3569f700 1 -- 172.21.15.78:0/1852395262 wait complete. 2023-10-11T09:44:13.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.710+0000 7fdb3569f700 1 Processor -- start 2023-10-11T09:44:13.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.710+0000 7fdb3569f700 1 -- start start 2023-10-11T09:44:13.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.714+0000 7fdb3569f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb1c003c00 0x7fdb3015e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:13.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.714+0000 7fdb3569f700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdb300b2b70 0x7fdb300b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:13.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.714+0000 7fdb3569f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb300c72a0 0x7fdb300c7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:13.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.714+0000 7fdb3569f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb300c7c50 con 0x7fdb1c003c00 2023-10-11T09:44:13.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.714+0000 7fdb3569f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb300c62d0 con 0x7fdb300c72a0 2023-10-11T09:44:13.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.714+0000 7fdb3569f700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb300c6e10 con 0x7fdb300b2b70 2023-10-11T09:44:13.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.714+0000 7fdb2ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb1c003c00 0x7fdb3015e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:13.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.714+0000 7fdb2ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb1c003c00 0x7fdb3015e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35124/0 (socket says 172.21.15.78:35124) 2023-10-11T09:44:13.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.714+0000 7fdb2ffff700 1 -- 172.21.15.78:0/3241083317 learned_addr learned my addr 172.21.15.78:0/3241083317 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:13.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.714+0000 7fdb277fe700 1 --2- 172.21.15.78:0/3241083317 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdb300b2b70 0x7fdb300b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:13.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.714+0000 7fdb34e9e700 1 --2- 172.21.15.78:0/3241083317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb300c72a0 0x7fdb300c7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:13.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.714+0000 7fdb2ffff700 1 -- 172.21.15.78:0/3241083317 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdb300b2b70 msgr2=0x7fdb300b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:13.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.714+0000 7fdb2ffff700 1 --2- 172.21.15.78:0/3241083317 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdb300b2b70 0x7fdb300b2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.714+0000 7fdb2ffff700 1 -- 172.21.15.78:0/3241083317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb300c72a0 msgr2=0x7fdb300c7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:13.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.714+0000 7fdb2ffff700 1 --2- 172.21.15.78:0/3241083317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb300c72a0 0x7fdb300c7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.714+0000 7fdb2ffff700 1 -- 172.21.15.78:0/3241083317 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb301fad30 con 0x7fdb1c003c00 2023-10-11T09:44:13.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.714+0000 7fdb34e9e700 1 --2- 172.21.15.78:0/3241083317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb300c72a0 0x7fdb300c7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:44:13.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.714+0000 7fdb2ffff700 1 --2- 172.21.15.78:0/3241083317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb1c003c00 0x7fdb3015e6e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c0047c0 tx=0x7fdb200047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:13.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.714+0000 7fdb2d7fa700 1 -- 172.21.15.78:0/3241083317 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2000e6c0 con 0x7fdb1c003c00 2023-10-11T09:44:13.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.714+0000 7fdb2d7fa700 1 -- 172.21.15.78:0/3241083317 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb20003af0 con 0x7fdb1c003c00 2023-10-11T09:44:13.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.714+0000 7fdb3569f700 1 -- 172.21.15.78:0/3241083317 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb300c64e0 con 0x7fdb1c003c00 2023-10-11T09:44:13.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.714+0000 7fdb2d7fa700 1 -- 172.21.15.78:0/3241083317 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb20003ea0 con 0x7fdb1c003c00 2023-10-11T09:44:13.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.714+0000 7fdb3569f700 1 -- 172.21.15.78:0/3241083317 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb300c6aa0 con 0x7fdb1c003c00 2023-10-11T09:44:13.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.718+0000 7fdb3569f700 1 -- 172.21.15.78:0/3241083317 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb30065d40 con 0x7fdb1c003c00 2023-10-11T09:44:13.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.718+0000 7fdb2d7fa700 1 -- 172.21.15.78:0/3241083317 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fdb20013470 con 0x7fdb1c003c00 2023-10-11T09:44:13.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.718+0000 7fdb2d7fa700 1 --2- 172.21.15.78:0/3241083317 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdb10065ed0 0x7fdb10068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:13.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.718+0000 7fdb2d7fa700 1 -- 172.21.15.78:0/3241083317 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdb20085650 con 0x7fdb1c003c00 2023-10-11T09:44:13.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.718+0000 7fdb277fe700 1 --2- 172.21.15.78:0/3241083317 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdb10065ed0 0x7fdb10068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:13.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.722+0000 7fdb277fe700 1 --2- 172.21.15.78:0/3241083317 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdb10065ed0 0x7fdb10068380 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdb300b4b10 tx=0x7fdb18009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:13.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.726+0000 7fdb2d7fa700 1 -- 172.21.15.78:0/3241083317 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fdb20052db0 con 0x7fdb1c003c00 2023-10-11T09:44:13.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.942+0000 7fdb3569f700 1 -- 172.21.15.78:0/3241083317 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdb300b1310 con 0x7fdb1c003c00 2023-10-11T09:44:13.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.942+0000 7fdb2d7fa700 1 -- 172.21.15.78:0/3241083317 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v37) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdb200563d0 con 0x7fdb1c003c00 2023-10-11T09:44:13.947 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-11T09:44:13.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.946+0000 7fdb3569f700 1 -- 172.21.15.78:0/3241083317 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdb10065ed0 msgr2=0x7fdb10068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:13.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.946+0000 7fdb3569f700 1 --2- 172.21.15.78:0/3241083317 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdb10065ed0 0x7fdb10068380 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdb300b4b10 tx=0x7fdb18009040 comp rx=0 tx=0).stop 2023-10-11T09:44:13.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.946+0000 7fdb3569f700 1 -- 172.21.15.78:0/3241083317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb1c003c00 msgr2=0x7fdb3015e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:13.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.946+0000 7fdb3569f700 1 --2- 172.21.15.78:0/3241083317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb1c003c00 0x7fdb3015e6e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c0047c0 tx=0x7fdb200047e0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.946+0000 7fdb3569f700 1 -- 172.21.15.78:0/3241083317 shutdown_connections 2023-10-11T09:44:13.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.946+0000 7fdb3569f700 1 --2- 172.21.15.78:0/3241083317 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdb10065ed0 0x7fdb10068380 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.946+0000 7fdb3569f700 1 --2- 172.21.15.78:0/3241083317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb300c72a0 0x7fdb300c7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.946+0000 7fdb3569f700 1 --2- 172.21.15.78:0/3241083317 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdb300b2b70 0x7fdb300b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.946+0000 7fdb3569f700 1 --2- 172.21.15.78:0/3241083317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb1c003c00 0x7fdb3015e6e0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:13.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.946+0000 7fdb3569f700 1 -- 172.21.15.78:0/3241083317 >> 172.21.15.78:0/3241083317 conn(0x7fdb300cea60 msgr2=0x7fdb30155c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:13.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.946+0000 7fdb3569f700 1 -- 172.21.15.78:0/3241083317 shutdown_connections 2023-10-11T09:44:13.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:13.946+0000 7fdb3569f700 1 -- 172.21.15.78:0/3241083317 wait complete. 2023-10-11T09:44:13.974 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-11T09:44:13.975 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-11T09:44:13.975 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi078.front.sepia.ceph.com 2023-10-11T09:44:13.975 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-11T09:44:13.976 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-11T09:44:13.976 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-11T09:44:13.976 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-11T09:44:13.977 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-11T09:44:13.977 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-11T09:44:13.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:44:13.978 DEBUG:teuthology.orchestra.run.smithi078:> ip addr 2023-10-11T09:44:13.992 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-11T09:44:13.992 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-11T09:44:13.992 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2023-10-11T09:44:13.993 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:44:13.993 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2023-10-11T09:44:13.993 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:44:13.993 INFO:teuthology.orchestra.run.smithi078.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:44:13.993 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:22 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:44:13.994 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f0 2023-10-11T09:44:13.994 INFO:teuthology.orchestra.run.smithi078.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:44:13.994 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:23 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:44:13.994 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f1 2023-10-11T09:44:13.994 INFO:teuthology.orchestra.run.smithi078.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:44:13.995 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dc brd ff:ff:ff:ff:ff:ff 2023-10-11T09:44:13.995 INFO:teuthology.orchestra.run.smithi078.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-11T09:44:13.995 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dd brd ff:ff:ff:ff:ff:ff 2023-10-11T09:44:13.995 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.21.15.78/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-11T09:44:13.995 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft 42210sec preferred_lft 42210sec 2023-10-11T09:44:13.996 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::ec4:7aff:fe8f:d4dd/64 scope link 2023-10-11T09:44:13.996 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:44:13.996 INFO:teuthology.orchestra.run.smithi078.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-11T09:44:13.996 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 02:42:b6:e4:f8:0d brd ff:ff:ff:ff:ff:ff 2023-10-11T09:44:13.996 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-11T09:44:13.997 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:44:13.997 INFO:teuthology.orchestra.run.smithi078.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-11T09:44:13.997 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 7a:89:8a:a9:e1:23 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:44:13.997 INFO:teuthology.orchestra.run.smithi078.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-11T09:44:13.997 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:44:13.998 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::8ca:ebff:fea9:13f1/64 scope link 2023-10-11T09:44:13.998 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:44:13.998 INFO:teuthology.orchestra.run.smithi078.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-11T09:44:13.998 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 7a:89:8a:a9:e1:23 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-11T09:44:13.998 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::7889:8aff:fea9:e123/64 scope link 2023-10-11T09:44:13.999 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:44:13.999 INFO:teuthology.orchestra.run.smithi078.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-11T09:44:13.999 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether ca:ac:e8:27:51:0d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-11T09:44:13.999 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::c8ac:e8ff:fe27:510d/64 scope link 2023-10-11T09:44:13.999 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:44:14.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:44:14.001 DEBUG:teuthology.orchestra.run.smithi078:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-11T09:44:14.077 INFO:teuthology.orchestra.run.smithi078.stdout:1 2023-10-11T09:44:14.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:44:14.080 DEBUG:teuthology.orchestra.run.smithi078:> route 2023-10-11T09:44:14.144 INFO:teuthology.orchestra.run.smithi078.stdout:Kernel IP routing table 2023-10-11T09:44:14.145 INFO:teuthology.orchestra.run.smithi078.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-11T09:44:14.145 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-11T09:44:14.145 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-11T09:44:14.146 INFO:teuthology.orchestra.run.smithi078.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-11T09:44:14.146 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-10-11T09:44:14.146 INFO:teuthology.orchestra.run.smithi078.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-11T09:44:14.146 INFO:teuthology.orchestra.run.smithi078.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-11T09:44:14.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:44:14.148 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec bash -c ' 2023-10-11T09:44:14.148 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2023-10-11T09:44:14.148 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-11T09:44:14.149 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-11T09:44:14.149 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-11T09:44:14.149 DEBUG:teuthology.orchestra.run.smithi078:> ') 2023-10-11T09:44:14.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:44:14.284 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2023-10-11T09:44:14.300 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-11T09:44:14.300 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-11T09:44:14.301 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-11T09:44:14.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:44:14.301 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set brx.0 up 2023-10-11T09:44:14.377 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-11T09:44:14.378 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T09:44:14.378 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-11T09:44:14.439 INFO:teuthology.orchestra.run.smithi078.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-11T09:44:14.440 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T09:44:14.440 DEBUG:teuthology.orchestra.run.smithi078:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-11T09:44:14.503 DEBUG:teuthology.orchestra.run.smithi078:> sudo modprobe fuse 2023-10-11T09:44:14.582 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T09:44:14.582 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T09:44:14.659 INFO:teuthology.orchestra.run.smithi078.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T09:44:14.663 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T09:44:14.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:44:14.664 DEBUG:teuthology.orchestra.run.smithi078:> ls /sys/fs/fuse/connections 2023-10-11T09:44:14.722 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-11T09:44:14.723 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-11T09:44:14.772 DEBUG:teuthology.orchestra.run.smithi078:> sudo modprobe fuse 2023-10-11T09:44:14.804 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T09:44:14.804 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T09:44:14.882 INFO:teuthology.orchestra.run.smithi078.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T09:44:14.885 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T09:44:14.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:44:14.886 DEBUG:teuthology.orchestra.run.smithi078:> ls /sys/fs/fuse/connections 2023-10-11T09:44:14.916 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:2023-10-11T09:44:14.910+0000 7f52e2c1b640 -1 init, newargv = 0x55c7033e57c0 newargc=15 2023-10-11T09:44:14.917 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:ceph-fuse[49518]: starting ceph client 2023-10-11T09:44:14.927 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:ceph-fuse[49518]: starting fuse 2023-10-11T09:44:14.929 INFO:teuthology.orchestra.run.smithi078.stdout:49 2023-10-11T09:44:14.929 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2023-10-11T09:44:14.930 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:44:14.930 DEBUG:teuthology.orchestra.run.smithi078:> import glob 2023-10-11T09:44:14.930 DEBUG:teuthology.orchestra.run.smithi078:> import re 2023-10-11T09:44:14.931 DEBUG:teuthology.orchestra.run.smithi078:> import os 2023-10-11T09:44:14.931 DEBUG:teuthology.orchestra.run.smithi078:> import subprocess 2023-10-11T09:44:14.931 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:44:14.932 DEBUG:teuthology.orchestra.run.smithi078:> def _find_admin_socket(client_name): 2023-10-11T09:44:14.932 DEBUG:teuthology.orchestra.run.smithi078:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-11T09:44:14.933 DEBUG:teuthology.orchestra.run.smithi078:> files = glob.glob(asok_path) 2023-10-11T09:44:14.933 DEBUG:teuthology.orchestra.run.smithi078:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-11T09:44:14.933 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:44:14.934 DEBUG:teuthology.orchestra.run.smithi078:> # Given a non-glob path, it better be there 2023-10-11T09:44:14.934 DEBUG:teuthology.orchestra.run.smithi078:> if "*" not in asok_path: 2023-10-11T09:44:14.935 DEBUG:teuthology.orchestra.run.smithi078:> assert(len(files) == 1) 2023-10-11T09:44:14.935 DEBUG:teuthology.orchestra.run.smithi078:> return files[0] 2023-10-11T09:44:14.935 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:44:14.936 DEBUG:teuthology.orchestra.run.smithi078:> for f in files: 2023-10-11T09:44:14.936 DEBUG:teuthology.orchestra.run.smithi078:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-11T09:44:14.937 DEBUG:teuthology.orchestra.run.smithi078:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-11T09:44:14.937 DEBUG:teuthology.orchestra.run.smithi078:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-11T09:44:14.937 DEBUG:teuthology.orchestra.run.smithi078:> contents = proc_f.read() 2023-10-11T09:44:14.938 DEBUG:teuthology.orchestra.run.smithi078:> if mountpoint in contents: 2023-10-11T09:44:14.938 DEBUG:teuthology.orchestra.run.smithi078:> return f 2023-10-11T09:44:14.938 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-11T09:44:14.939 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:44:14.939 DEBUG:teuthology.orchestra.run.smithi078:> print(_find_admin_socket("client.0")) 2023-10-11T09:44:14.940 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:44:15.081 INFO:teuthology.orchestra.run.smithi078.stdout:/var/run/ceph/ceph-client.0.49518.asok 2023-10-11T09:44:15.263 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.49518.asok 2023-10-11T09:44:15.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:44:15.264 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.49518.asok status 2023-10-11T09:44:15.458 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:44:15.459 INFO:teuthology.orchestra.run.smithi078.stdout: "metadata": { 2023-10-11T09:44:15.459 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_sha1": "9a774796f0ecdd11a4b771de396e627aa1a187af", 2023-10-11T09:44:15.459 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_version": "ceph version 18.2.0-662-g9a774796 (9a774796f0ecdd11a4b771de396e627aa1a187af) reef (stable)", 2023-10-11T09:44:15.459 INFO:teuthology.orchestra.run.smithi078.stdout: "entity_id": "0", 2023-10-11T09:44:15.460 INFO:teuthology.orchestra.run.smithi078.stdout: "hostname": "smithi078", 2023-10-11T09:44:15.460 INFO:teuthology.orchestra.run.smithi078.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-11T09:44:15.460 INFO:teuthology.orchestra.run.smithi078.stdout: "pid": "49518", 2023-10-11T09:44:15.460 INFO:teuthology.orchestra.run.smithi078.stdout: "root": "/" 2023-10-11T09:44:15.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:44:15.461 INFO:teuthology.orchestra.run.smithi078.stdout: "dentry_count": 0, 2023-10-11T09:44:15.461 INFO:teuthology.orchestra.run.smithi078.stdout: "dentry_pinned_count": 0, 2023-10-11T09:44:15.461 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 6423, 2023-10-11T09:44:15.461 INFO:teuthology.orchestra.run.smithi078.stdout: "inst": { 2023-10-11T09:44:15.461 INFO:teuthology.orchestra.run.smithi078.stdout: "name": { 2023-10-11T09:44:15.462 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "client", 2023-10-11T09:44:15.462 INFO:teuthology.orchestra.run.smithi078.stdout: "num": 6423 2023-10-11T09:44:15.462 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:44:15.462 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": { 2023-10-11T09:44:15.462 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2023-10-11T09:44:15.463 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "192.168.0.1:0", 2023-10-11T09:44:15.463 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 78584074 2023-10-11T09:44:15.463 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:44:15.463 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:44:15.463 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": { 2023-10-11T09:44:15.464 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2023-10-11T09:44:15.464 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "192.168.0.1:0", 2023-10-11T09:44:15.464 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 78584074 2023-10-11T09:44:15.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:44:15.464 INFO:teuthology.orchestra.run.smithi078.stdout: "inst_str": "client.6423 192.168.0.1:0/78584074", 2023-10-11T09:44:15.465 INFO:teuthology.orchestra.run.smithi078.stdout: "addr_str": "192.168.0.1:0/78584074", 2023-10-11T09:44:15.465 INFO:teuthology.orchestra.run.smithi078.stdout: "inode_count": 1, 2023-10-11T09:44:15.465 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_epoch": 37, 2023-10-11T09:44:15.465 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_epoch": 61, 2023-10-11T09:44:15.465 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_epoch_barrier": 0, 2023-10-11T09:44:15.466 INFO:teuthology.orchestra.run.smithi078.stdout: "blocklisted": false, 2023-10-11T09:44:15.466 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs" 2023-10-11T09:44:15.466 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:44:15.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:44:15.473 DEBUG:teuthology.orchestra.run.smithi078:> stat --file-system '--printf=%T 2023-10-11T09:44:15.474 DEBUG:teuthology.orchestra.run.smithi078:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-11T09:44:15.491 INFO:teuthology.orchestra.run.smithi078.stdout:fuseblk 2023-10-11T09:44:15.491 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-11T09:44:15.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:44:15.492 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-11T09:44:15.566 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_extra --pg_num_min 64 2023-10-11T09:44:15.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.782+0000 7f2e5d6a3700 1 Processor -- start 2023-10-11T09:44:15.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.782+0000 7f2e5d6a3700 1 -- start start 2023-10-11T09:44:15.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.782+0000 7f2e5d6a3700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f2e580cdbd0 0x7f2e5806cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:15.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.782+0000 7f2e5d6a3700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e58165140 con 0x7f2e581655d0 2023-10-11T09:44:15.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.782+0000 7f2e5d6a3700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e580cd470 con 0x7f2e580b0eb0 2023-10-11T09:44:15.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.782+0000 7f2e5d6a3700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e581648e0 con 0x7f2e580cdbd0 2023-10-11T09:44:15.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.786+0000 7f2e567fc700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f2e581655d0 0x7f2e580cdfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33750/0 (socket says 172.21.15.78:33750) 2023-10-11T09:44:15.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.786+0000 7f2e567fc700 1 -- 172.21.15.78:0/926151912 learned_addr learned my addr 172.21.15.78:0/926151912 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:44:15.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.786+0000 7f2e56ffd700 1 --2- 172.21.15.78:0/926151912 >> v2:172.21.15.78:3300/0 conn(0x7f2e580cdbd0 0x7f2e5806cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:15.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.786+0000 7f2e55ffb700 1 -- 172.21.15.78:0/926151912 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1686811244 0 0) 0x7f2e58165140 con 0x7f2e581655d0 2023-10-11T09:44:15.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.786+0000 7f2e55ffb700 1 -- 172.21.15.78:0/926151912 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e400035d0 con 0x7f2e581655d0 2023-10-11T09:44:15.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.786+0000 7f2e55ffb700 1 -- 172.21.15.78:0/926151912 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2687501700 0 0) 0x7f2e580cd470 con 0x7f2e580b0eb0 2023-10-11T09:44:15.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.786+0000 7f2e55ffb700 1 -- 172.21.15.78:0/926151912 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e58165140 con 0x7f2e580b0eb0 2023-10-11T09:44:15.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.786+0000 7f2e56ffd700 1 -- 172.21.15.78:0/926151912 >> v1:172.21.15.133:6789/0 conn(0x7f2e580b0eb0 legacy=0x7f2e580ca2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:15.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.786+0000 7f2e56ffd700 1 -- 172.21.15.78:0/926151912 >> v1:172.21.15.78:6789/0 conn(0x7f2e581655d0 legacy=0x7f2e580cdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:15.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.786+0000 7f2e56ffd700 1 -- 172.21.15.78:0/926151912 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e58164af0 con 0x7f2e580cdbd0 2023-10-11T09:44:15.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.786+0000 7f2e56ffd700 1 --2- 172.21.15.78:0/926151912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e580cdbd0 0x7f2e5806cfa0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f2e48009850 tx=0x7f2e48009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4df743b8115c89a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:15.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.786+0000 7f2e55ffb700 1 -- 172.21.15.78:0/926151912 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e48011830 con 0x7f2e580cdbd0 2023-10-11T09:44:15.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.786+0000 7f2e55ffb700 1 -- 172.21.15.78:0/926151912 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e48011990 con 0x7f2e580cdbd0 2023-10-11T09:44:15.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.786+0000 7f2e55ffb700 1 -- 172.21.15.78:0/926151912 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e48011d40 con 0x7f2e580cdbd0 2023-10-11T09:44:15.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.790+0000 7f2e5d6a3700 1 -- 172.21.15.78:0/926151912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e580cdbd0 msgr2=0x7f2e5806cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:15.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.790+0000 7f2e5d6a3700 1 --2- 172.21.15.78:0/926151912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e580cdbd0 0x7f2e5806cfa0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f2e48009850 tx=0x7f2e48009b60 comp rx=0 tx=0).stop 2023-10-11T09:44:15.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.790+0000 7f2e5d6a3700 1 -- 172.21.15.78:0/926151912 shutdown_connections 2023-10-11T09:44:15.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.790+0000 7f2e5d6a3700 1 --2- 172.21.15.78:0/926151912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e580cdbd0 0x7f2e5806cfa0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:15.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.790+0000 7f2e5d6a3700 1 -- 172.21.15.78:0/926151912 >> 172.21.15.78:0/926151912 conn(0x7f2e5806acf0 msgr2=0x7f2e580b0710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:15.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.790+0000 7f2e5d6a3700 1 -- 172.21.15.78:0/926151912 shutdown_connections 2023-10-11T09:44:15.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.790+0000 7f2e5d6a3700 1 -- 172.21.15.78:0/926151912 wait complete. 2023-10-11T09:44:15.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.790+0000 7f2e5d6a3700 1 Processor -- start 2023-10-11T09:44:15.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.794+0000 7f2e5d6a3700 1 -- start start 2023-10-11T09:44:15.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.794+0000 7f2e5d6a3700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2e580b0eb0 0x7f2e5815dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:15.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.794+0000 7f2e5d6a3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e580cdbd0 0x7f2e58163b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:15.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.794+0000 7f2e5d6a3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e581655d0 0x7f2e580c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:15.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.794+0000 7f2e5d6a3700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e58163740 con 0x7f2e580cdbd0 2023-10-11T09:44:15.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.794+0000 7f2e5d6a3700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5815e400 con 0x7f2e581655d0 2023-10-11T09:44:15.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.794+0000 7f2e5d6a3700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e580b3cb0 con 0x7f2e580b0eb0 2023-10-11T09:44:15.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.794+0000 7f2e567fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e580cdbd0 0x7f2e58163b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:15.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.794+0000 7f2e567fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e580cdbd0 0x7f2e58163b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52924/0 (socket says 172.21.15.78:52924) 2023-10-11T09:44:15.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.794+0000 7f2e567fc700 1 -- 172.21.15.78:0/670557866 learned_addr learned my addr 172.21.15.78:0/670557866 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:15.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.794+0000 7f2e56ffd700 1 --2- 172.21.15.78:0/670557866 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2e580b0eb0 0x7f2e5815dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:15.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.794+0000 7f2e577fe700 1 --2- 172.21.15.78:0/670557866 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e581655d0 0x7f2e580c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:15.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.794+0000 7f2e567fc700 1 -- 172.21.15.78:0/670557866 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2e580b0eb0 msgr2=0x7f2e5815dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:15.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.794+0000 7f2e567fc700 1 --2- 172.21.15.78:0/670557866 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2e580b0eb0 0x7f2e5815dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:15.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.794+0000 7f2e567fc700 1 -- 172.21.15.78:0/670557866 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e581655d0 msgr2=0x7f2e580c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:15.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.794+0000 7f2e567fc700 1 --2- 172.21.15.78:0/670557866 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e581655d0 0x7f2e580c8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:15.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.794+0000 7f2e567fc700 1 -- 172.21.15.78:0/670557866 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e480094b0 con 0x7f2e580cdbd0 2023-10-11T09:44:15.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.794+0000 7f2e567fc700 1 --2- 172.21.15.78:0/670557866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e580cdbd0 0x7f2e58163b60 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c00c560 tx=0x7f2e4c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:15.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.794+0000 7f2e3f7fe700 1 -- 172.21.15.78:0/670557866 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c005d90 con 0x7f2e580cdbd0 2023-10-11T09:44:15.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.794+0000 7f2e3f7fe700 1 -- 172.21.15.78:0/670557866 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e4c0070d0 con 0x7f2e580cdbd0 2023-10-11T09:44:15.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.794+0000 7f2e3f7fe700 1 -- 172.21.15.78:0/670557866 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c007440 con 0x7f2e580cdbd0 2023-10-11T09:44:15.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.794+0000 7f2e5d6a3700 1 -- 172.21.15.78:0/670557866 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e580b3f90 con 0x7f2e580cdbd0 2023-10-11T09:44:15.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.798+0000 7f2e5d6a3700 1 -- 172.21.15.78:0/670557866 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e5815e7b0 con 0x7f2e580cdbd0 2023-10-11T09:44:15.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.798+0000 7f2e5d6a3700 1 -- 172.21.15.78:0/670557866 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e580b44b0 con 0x7f2e580cdbd0 2023-10-11T09:44:15.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.798+0000 7f2e3f7fe700 1 -- 172.21.15.78:0/670557866 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f2e4c0075a0 con 0x7f2e580cdbd0 2023-10-11T09:44:15.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.802+0000 7f2e3f7fe700 1 --2- 172.21.15.78:0/670557866 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2e44065ed0 0x7f2e44068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:15.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.802+0000 7f2e3f7fe700 1 -- 172.21.15.78:0/670557866 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2e4c082b60 con 0x7f2e580cdbd0 2023-10-11T09:44:15.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.802+0000 7f2e56ffd700 1 --2- 172.21.15.78:0/670557866 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2e44065ed0 0x7f2e44068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:15.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.802+0000 7f2e56ffd700 1 --2- 172.21.15.78:0/670557866 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2e44065ed0 0x7f2e44068380 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2e480090b0 tx=0x7f2e4801c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:15.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.806+0000 7f2e3f7fe700 1 -- 172.21.15.78:0/670557866 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f2e580b44b0 con 0x7f2e580cdbd0 2023-10-11T09:44:15.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:15.982+0000 7f2e5d6a3700 1 -- 172.21.15.78:0/670557866 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_extra", "pg_num_min": 64} v 0) v1 -- 0x7f2e58163880 con 0x7f2e580cdbd0 2023-10-11T09:44:16.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.262+0000 7f2e3f7fe700 1 -- 172.21.15.78:0/670557866 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_extra", "pg_num_min": 64}]=0 pool 'cephfs_data_extra' created v62) v1 ==== 142+0+0 (secure 0 0 0) 0x7f2e58163880 con 0x7f2e580cdbd0 2023-10-11T09:44:16.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.266+0000 7f2e5d6a3700 1 -- 172.21.15.78:0/670557866 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2e44065ed0 msgr2=0x7f2e44068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:16.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.266+0000 7f2e5d6a3700 1 --2- 172.21.15.78:0/670557866 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2e44065ed0 0x7f2e44068380 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2e480090b0 tx=0x7f2e4801c040 comp rx=0 tx=0).stop 2023-10-11T09:44:16.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.266+0000 7f2e5d6a3700 1 -- 172.21.15.78:0/670557866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e580cdbd0 msgr2=0x7f2e58163b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:16.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.266+0000 7f2e5d6a3700 1 --2- 172.21.15.78:0/670557866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e580cdbd0 0x7f2e58163b60 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c00c560 tx=0x7f2e4c00c920 comp rx=0 tx=0).stop 2023-10-11T09:44:16.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.270+0000 7f2e5d6a3700 1 -- 172.21.15.78:0/670557866 shutdown_connections 2023-10-11T09:44:16.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.270+0000 7f2e5d6a3700 1 --2- 172.21.15.78:0/670557866 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2e44065ed0 0x7f2e44068380 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:16.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.270+0000 7f2e5d6a3700 1 --2- 172.21.15.78:0/670557866 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e581655d0 0x7f2e580c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:16.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.270+0000 7f2e5d6a3700 1 --2- 172.21.15.78:0/670557866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e580cdbd0 0x7f2e58163b60 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:16.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.270+0000 7f2e5d6a3700 1 --2- 172.21.15.78:0/670557866 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2e580b0eb0 0x7f2e5815dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:16.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.270+0000 7f2e5d6a3700 1 -- 172.21.15.78:0/670557866 >> 172.21.15.78:0/670557866 conn(0x7f2e5806acf0 msgr2=0x7f2e5806bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:16.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.270+0000 7f2e5d6a3700 1 -- 172.21.15.78:0/670557866 shutdown_connections 2023-10-11T09:44:16.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.270+0000 7f2e5d6a3700 1 -- 172.21.15.78:0/670557866 wait complete. 2023-10-11T09:44:16.280 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data_extra' created 2023-10-11T09:44:16.298 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_extra 2023-10-11T09:44:16.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.470+0000 7facc5501700 1 Processor -- start 2023-10-11T09:44:16.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.470+0000 7facc5501700 1 -- start start 2023-10-11T09:44:16.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.474+0000 7facc5501700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7facc0158430 0x7facc00710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:16.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.474+0000 7facc5501700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7facc00caf90 0x7facc015afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:16.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.474+0000 7facc5501700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facc0071610 con 0x7facc006d340 2023-10-11T09:44:16.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.474+0000 7facc5501700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facc0070c10 con 0x7facc00caf90 2023-10-11T09:44:16.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.474+0000 7facc5501700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facc015e280 con 0x7facc0158430 2023-10-11T09:44:16.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.474+0000 7facbf7fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7facc00caf90 0x7facc015afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:16.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.474+0000 7facb7fff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7facc006d340 0x7facc00ca880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33762/0 (socket says 172.21.15.78:33762) 2023-10-11T09:44:16.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.474+0000 7facb7fff700 1 -- 172.21.15.78:0/732284270 learned_addr learned my addr 172.21.15.78:0/732284270 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:44:16.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.474+0000 7facbeffd700 1 --2- 172.21.15.78:0/732284270 >> v2:172.21.15.133:3300/0 conn(0x7facc0158430 0x7facc00710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:16.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.474+0000 7facbf7fe700 1 -- 172.21.15.78:0/732284270 >> v2:172.21.15.133:3300/0 conn(0x7facc0158430 msgr2=0x7facc00710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:44:16.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.474+0000 7facbf7fe700 1 --2- 172.21.15.78:0/732284270 >> v2:172.21.15.133:3300/0 conn(0x7facc0158430 0x7facc00710d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:16.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.474+0000 7facbf7fe700 1 -- 172.21.15.78:0/732284270 >> v1:172.21.15.78:6789/0 conn(0x7facc006d340 legacy=0x7facc00ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:16.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.474+0000 7facbf7fe700 1 -- 172.21.15.78:0/732284270 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc015e560 con 0x7facc00caf90 2023-10-11T09:44:16.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.478+0000 7facbf7fe700 1 --2- 172.21.15.78:0/732284270 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7facc00caf90 0x7facc015afe0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7facc006fd00 tx=0x7facb800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4364f57624f7bd6 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:16.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.478+0000 7facbe7fc700 1 -- 172.21.15.78:0/732284270 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb800e070 con 0x7facc00caf90 2023-10-11T09:44:16.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.478+0000 7facbe7fc700 1 -- 172.21.15.78:0/732284270 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb800bb20 con 0x7facc00caf90 2023-10-11T09:44:16.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.478+0000 7facbe7fc700 1 -- 172.21.15.78:0/732284270 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb8012430 con 0x7facc00caf90 2023-10-11T09:44:16.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.478+0000 7facc5501700 1 -- 172.21.15.78:0/732284270 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7facc00caf90 msgr2=0x7facc015afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:16.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.478+0000 7facc5501700 1 --2- 172.21.15.78:0/732284270 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7facc00caf90 0x7facc015afe0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7facc006fd00 tx=0x7facb800b330 comp rx=0 tx=0).stop 2023-10-11T09:44:16.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.478+0000 7facc5501700 1 -- 172.21.15.78:0/732284270 shutdown_connections 2023-10-11T09:44:16.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.478+0000 7facc5501700 1 --2- 172.21.15.78:0/732284270 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7facc00caf90 0x7facc015afe0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:16.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.478+0000 7facc5501700 1 --2- 172.21.15.78:0/732284270 >> v2:172.21.15.133:3300/0 conn(0x7facc0158430 0x7facc00710d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:16.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.478+0000 7facc5501700 1 -- 172.21.15.78:0/732284270 >> 172.21.15.78:0/732284270 conn(0x7facc00cecc0 msgr2=0x7facc00cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:16.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.478+0000 7facc5501700 1 -- 172.21.15.78:0/732284270 shutdown_connections 2023-10-11T09:44:16.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.478+0000 7facc5501700 1 -- 172.21.15.78:0/732284270 wait complete. 2023-10-11T09:44:16.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.482+0000 7facc5501700 1 Processor -- start 2023-10-11T09:44:16.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.482+0000 7facc5501700 1 -- start start 2023-10-11T09:44:16.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.482+0000 7facc5501700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7facc006d340 0x7facc015d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:16.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.482+0000 7facc5501700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7facc00caf90 0x7facc00c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:16.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.482+0000 7facc5501700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7facc0158430 0x7facc0155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:16.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.482+0000 7facc5501700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc01f2dc0 con 0x7facc0158430 2023-10-11T09:44:16.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.482+0000 7facc5501700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc01f21b0 con 0x7facc00caf90 2023-10-11T09:44:16.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.482+0000 7facc5501700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facc01f2490 con 0x7facc006d340 2023-10-11T09:44:16.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.486+0000 7facbf7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7facc0158430 0x7facc0155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:16.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.486+0000 7facbf7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7facc0158430 0x7facc0155040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52938/0 (socket says 172.21.15.78:52938) 2023-10-11T09:44:16.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.486+0000 7facbf7fe700 1 -- 172.21.15.78:0/4233241755 learned_addr learned my addr 172.21.15.78:0/4233241755 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:16.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.486+0000 7facb7fff700 1 --2- 172.21.15.78:0/4233241755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7facc00caf90 0x7facc00c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:16.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.486+0000 7facbeffd700 1 --2- 172.21.15.78:0/4233241755 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7facc006d340 0x7facc015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:16.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.486+0000 7facbf7fe700 1 -- 172.21.15.78:0/4233241755 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7facc006d340 msgr2=0x7facc015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:16.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.486+0000 7facbf7fe700 1 --2- 172.21.15.78:0/4233241755 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7facc006d340 0x7facc015d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:16.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.486+0000 7facbf7fe700 1 -- 172.21.15.78:0/4233241755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7facc00caf90 msgr2=0x7facc00c9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:16.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.486+0000 7facbf7fe700 1 --2- 172.21.15.78:0/4233241755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7facc00caf90 0x7facc00c9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:16.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.486+0000 7facbf7fe700 1 -- 172.21.15.78:0/4233241755 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb8009a70 con 0x7facc0158430 2023-10-11T09:44:16.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.486+0000 7facbf7fe700 1 --2- 172.21.15.78:0/4233241755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7facc0158430 0x7facc0155040 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7facb8014040 tx=0x7facb8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:16.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.486+0000 7facb77fe700 1 -- 172.21.15.78:0/4233241755 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb8012430 con 0x7facc0158430 2023-10-11T09:44:16.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.486+0000 7facb77fe700 1 -- 172.21.15.78:0/4233241755 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb8020070 con 0x7facc0158430 2023-10-11T09:44:16.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.486+0000 7facb77fe700 1 -- 172.21.15.78:0/4233241755 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb8004040 con 0x7facc0158430 2023-10-11T09:44:16.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.486+0000 7facc5501700 1 -- 172.21.15.78:0/4233241755 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc01f2770 con 0x7facc0158430 2023-10-11T09:44:16.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.486+0000 7facc5501700 1 -- 172.21.15.78:0/4233241755 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc01f1eb0 con 0x7facc0158430 2023-10-11T09:44:16.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.490+0000 7facc5501700 1 -- 172.21.15.78:0/4233241755 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facc01f2f70 con 0x7facc0158430 2023-10-11T09:44:16.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.490+0000 7facb77fe700 1 -- 172.21.15.78:0/4233241755 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7facb80041a0 con 0x7facc0158430 2023-10-11T09:44:16.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.490+0000 7facb77fe700 1 --2- 172.21.15.78:0/4233241755 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7faca0065f00 0x7faca00683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:16.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.490+0000 7facb77fe700 1 -- 172.21.15.78:0/4233241755 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6434+0+0 (secure 0 0 0) 0x7facb8022070 con 0x7facc0158430 2023-10-11T09:44:16.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.494+0000 7facbeffd700 1 --2- 172.21.15.78:0/4233241755 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7faca0065f00 0x7faca00683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:16.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.494+0000 7facbeffd700 1 --2- 172.21.15.78:0/4233241755 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7faca0065f00 0x7faca00683b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7facb0009b50 tx=0x7facb0009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:16.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.498+0000 7facb77fe700 1 -- 172.21.15.78:0/4233241755 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7facc01f2f70 con 0x7facc0158430 2023-10-11T09:44:16.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.710+0000 7facc5501700 1 -- 172.21.15.78:0/4233241755 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_extra"} v 0) v1 -- 0x7facc01f2f00 con 0x7facc0158430 2023-10-11T09:44:16.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.722+0000 7facb77fe700 1 -- 172.21.15.78:0/4233241755 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_extra"}]=0 added data pool 10 to fsmap v38) v1 ==== 141+0+0 (secure 0 0 0) 0x7facb8052a00 con 0x7facc0158430 2023-10-11T09:44:16.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.730+0000 7facc5501700 1 -- 172.21.15.78:0/4233241755 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7faca0065f00 msgr2=0x7faca00683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:16.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.730+0000 7facc5501700 1 --2- 172.21.15.78:0/4233241755 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7faca0065f00 0x7faca00683b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7facb0009b50 tx=0x7facb0009240 comp rx=0 tx=0).stop 2023-10-11T09:44:16.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.730+0000 7facc5501700 1 -- 172.21.15.78:0/4233241755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7facc0158430 msgr2=0x7facc0155040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:16.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.730+0000 7facc5501700 1 --2- 172.21.15.78:0/4233241755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7facc0158430 0x7facc0155040 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7facb8014040 tx=0x7facb8004af0 comp rx=0 tx=0).stop 2023-10-11T09:44:16.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.734+0000 7facc5501700 1 -- 172.21.15.78:0/4233241755 shutdown_connections 2023-10-11T09:44:16.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.734+0000 7facc5501700 1 --2- 172.21.15.78:0/4233241755 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7faca0065f00 0x7faca00683b0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:16.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.734+0000 7facc5501700 1 --2- 172.21.15.78:0/4233241755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7facc0158430 0x7facc0155040 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:16.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.734+0000 7facc5501700 1 --2- 172.21.15.78:0/4233241755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7facc00caf90 0x7facc00c9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:16.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.734+0000 7facc5501700 1 --2- 172.21.15.78:0/4233241755 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7facc006d340 0x7facc015d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:16.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.734+0000 7facc5501700 1 -- 172.21.15.78:0/4233241755 >> 172.21.15.78:0/4233241755 conn(0x7facc00cecc0 msgr2=0x7facc01567f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:16.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.734+0000 7facc5501700 1 -- 172.21.15.78:0/4233241755 shutdown_connections 2023-10-11T09:44:16.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.734+0000 7facc5501700 1 -- 172.21.15.78:0/4233241755 wait complete. 2023-10-11T09:44:16.746 INFO:teuthology.orchestra.run.smithi078.stderr:added data pool 10 to fsmap 2023-10-11T09:44:16.762 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:44:16.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.926+0000 7fec70836700 1 Processor -- start 2023-10-11T09:44:16.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.930+0000 7fec70836700 1 -- start start 2023-10-11T09:44:16.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.930+0000 7fec70836700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fec6c15a940 0x7fec6c0ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:16.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.930+0000 7fec70836700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fec6c0ca780 0x7fec6c13fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:16.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.930+0000 7fec70836700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec6c15e010 con 0x7fec6c15b8e0 2023-10-11T09:44:16.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.930+0000 7fec70836700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec6c0cab70 con 0x7fec6c0ca780 2023-10-11T09:44:16.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.930+0000 7fec70836700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec6c15e280 con 0x7fec6c15a940 2023-10-11T09:44:16.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.930+0000 7fec6ad9d700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fec6c15b8e0 0x7fec6c06d0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33764/0 (socket says 172.21.15.78:33764) 2023-10-11T09:44:16.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.930+0000 7fec6ad9d700 1 -- 172.21.15.78:0/2638909169 learned_addr learned my addr 172.21.15.78:0/2638909169 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:44:16.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.930+0000 7fec6b59e700 1 --2- 172.21.15.78:0/2638909169 >> v2:172.21.15.78:3300/0 conn(0x7fec6c0ca780 0x7fec6c13fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:16.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.930+0000 7fec6a59c700 1 --2- 172.21.15.78:0/2638909169 >> v2:172.21.15.133:3300/0 conn(0x7fec6c15a940 0x7fec6c0ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:16.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.930+0000 7fec6b59e700 1 -- 172.21.15.78:0/2638909169 >> v2:172.21.15.133:3300/0 conn(0x7fec6c15a940 msgr2=0x7fec6c0ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:44:16.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.930+0000 7fec6b59e700 1 --2- 172.21.15.78:0/2638909169 >> v2:172.21.15.133:3300/0 conn(0x7fec6c15a940 0x7fec6c0ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:16.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.930+0000 7fec69d9b700 1 -- 172.21.15.78:0/2638909169 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1207599773 0 0) 0x7fec6c15e010 con 0x7fec6c15b8e0 2023-10-11T09:44:16.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.930+0000 7fec6b59e700 1 -- 172.21.15.78:0/2638909169 >> v1:172.21.15.78:6789/0 conn(0x7fec6c15b8e0 legacy=0x7fec6c06d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:16.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.930+0000 7fec6b59e700 1 -- 172.21.15.78:0/2638909169 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec6c15e560 con 0x7fec6c0ca780 2023-10-11T09:44:16.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.934+0000 7fec6b59e700 1 --2- 172.21.15.78:0/2638909169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec6c0ca780 0x7fec6c13fcb0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fec6c06f800 tx=0x7fec6400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d171da65cf6a5f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:16.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.934+0000 7fec69d9b700 1 -- 172.21.15.78:0/2638909169 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6400e070 con 0x7fec6c0ca780 2023-10-11T09:44:16.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.934+0000 7fec69d9b700 1 -- 172.21.15.78:0/2638909169 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec6400bb20 con 0x7fec6c0ca780 2023-10-11T09:44:16.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.934+0000 7fec69d9b700 1 -- 172.21.15.78:0/2638909169 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec64012430 con 0x7fec6c0ca780 2023-10-11T09:44:16.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.934+0000 7fec70836700 1 -- 172.21.15.78:0/2638909169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec6c0ca780 msgr2=0x7fec6c13fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:16.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.934+0000 7fec70836700 1 --2- 172.21.15.78:0/2638909169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec6c0ca780 0x7fec6c13fcb0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fec6c06f800 tx=0x7fec6400b330 comp rx=0 tx=0).stop 2023-10-11T09:44:16.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.934+0000 7fec70836700 1 -- 172.21.15.78:0/2638909169 shutdown_connections 2023-10-11T09:44:16.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.934+0000 7fec70836700 1 --2- 172.21.15.78:0/2638909169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec6c0ca780 0x7fec6c13fcb0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:16.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.934+0000 7fec70836700 1 --2- 172.21.15.78:0/2638909169 >> v2:172.21.15.133:3300/0 conn(0x7fec6c15a940 0x7fec6c0ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:16.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.934+0000 7fec70836700 1 -- 172.21.15.78:0/2638909169 >> 172.21.15.78:0/2638909169 conn(0x7fec6c0cea60 msgr2=0x7fec6c0cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:16.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.934+0000 7fec70836700 1 -- 172.21.15.78:0/2638909169 shutdown_connections 2023-10-11T09:44:16.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.934+0000 7fec70836700 1 -- 172.21.15.78:0/2638909169 wait complete. 2023-10-11T09:44:16.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.938+0000 7fec70836700 1 Processor -- start 2023-10-11T09:44:16.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.938+0000 7fec70836700 1 -- start start 2023-10-11T09:44:16.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.938+0000 7fec70836700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec6c0ca780 0x7fec6c159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:16.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.938+0000 7fec70836700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fec6c15a940 0x7fec6c0c6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:16.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.938+0000 7fec70836700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec6c15b8e0 0x7fec6c1421d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:16.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.938+0000 7fec70836700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c142e50 con 0x7fec6c15b8e0 2023-10-11T09:44:16.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.938+0000 7fec70836700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c13f120 con 0x7fec6c0ca780 2023-10-11T09:44:16.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.938+0000 7fec70836700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c13f400 con 0x7fec6c15a940 2023-10-11T09:44:16.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.938+0000 7fec6b59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec6c15b8e0 0x7fec6c1421d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:16.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.938+0000 7fec6b59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec6c15b8e0 0x7fec6c1421d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52968/0 (socket says 172.21.15.78:52968) 2023-10-11T09:44:16.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.938+0000 7fec6b59e700 1 -- 172.21.15.78:0/3380169009 learned_addr learned my addr 172.21.15.78:0/3380169009 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:16.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.938+0000 7fec6ad9d700 1 --2- 172.21.15.78:0/3380169009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec6c0ca780 0x7fec6c159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:16.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.938+0000 7fec6a59c700 1 --2- 172.21.15.78:0/3380169009 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fec6c15a940 0x7fec6c0c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:16.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.942+0000 7fec6b59e700 1 -- 172.21.15.78:0/3380169009 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fec6c15a940 msgr2=0x7fec6c0c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:16.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.942+0000 7fec6b59e700 1 --2- 172.21.15.78:0/3380169009 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fec6c15a940 0x7fec6c0c6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:16.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.942+0000 7fec6b59e700 1 -- 172.21.15.78:0/3380169009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec6c0ca780 msgr2=0x7fec6c159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:16.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.942+0000 7fec6b59e700 1 --2- 172.21.15.78:0/3380169009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec6c0ca780 0x7fec6c159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:16.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.942+0000 7fec6b59e700 1 -- 172.21.15.78:0/3380169009 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec64009a70 con 0x7fec6c15b8e0 2023-10-11T09:44:16.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.942+0000 7fec6b59e700 1 --2- 172.21.15.78:0/3380169009 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec6c15b8e0 0x7fec6c1421d0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fec6c072460 tx=0x7fec64004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:16.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.942+0000 7fec537fe700 1 -- 172.21.15.78:0/3380169009 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec64012430 con 0x7fec6c15b8e0 2023-10-11T09:44:16.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.942+0000 7fec537fe700 1 -- 172.21.15.78:0/3380169009 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec64020070 con 0x7fec6c15b8e0 2023-10-11T09:44:16.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.942+0000 7fec537fe700 1 -- 172.21.15.78:0/3380169009 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec64004280 con 0x7fec6c15b8e0 2023-10-11T09:44:16.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.942+0000 7fec70836700 1 -- 172.21.15.78:0/3380169009 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec6c13f6e0 con 0x7fec6c15b8e0 2023-10-11T09:44:16.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.942+0000 7fec70836700 1 -- 172.21.15.78:0/3380169009 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec6c0c6d60 con 0x7fec6c15b8e0 2023-10-11T09:44:16.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.942+0000 7fec70836700 1 -- 172.21.15.78:0/3380169009 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec6c065d40 con 0x7fec6c15b8e0 2023-10-11T09:44:16.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.946+0000 7fec537fe700 1 -- 172.21.15.78:0/3380169009 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fec640043e0 con 0x7fec6c15b8e0 2023-10-11T09:44:16.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.946+0000 7fec537fe700 1 --2- 172.21.15.78:0/3380169009 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fec58065ed0 0x7fec58068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:16.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.946+0000 7fec537fe700 1 -- 172.21.15.78:0/3380169009 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6466+0+0 (secure 0 0 0) 0x7fec64022070 con 0x7fec6c15b8e0 2023-10-11T09:44:16.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.946+0000 7fec6ad9d700 1 --2- 172.21.15.78:0/3380169009 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fec58065ed0 0x7fec58068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:16.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.946+0000 7fec6ad9d700 1 --2- 172.21.15.78:0/3380169009 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fec58065ed0 0x7fec58068380 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fec5c000f60 tx=0x7fec5c007e00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:16.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:16.954+0000 7fec537fe700 1 -- 172.21.15.78:0/3380169009 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fec64052a00 con 0x7fec6c15b8e0 2023-10-11T09:44:17.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.158+0000 7fec70836700 1 -- 172.21.15.78:0/3380169009 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec6c0b0fe0 con 0x7fec6c15b8e0 2023-10-11T09:44:17.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.158+0000 7fec537fe700 1 -- 172.21.15.78:0/3380169009 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4654 (secure 0 0 0) 0x7fec64056020 con 0x7fec6c15b8e0 2023-10-11T09:44:17.165 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:44:17.165 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":38,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5579,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3954348479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":3954348479},{"type":"v1","addr":"172.21.15.133:6835","nonce":3954348479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6100,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/4029676589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4029676589},{"type":"v1","addr":"172.21.15.133:6834","nonce":4029676589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6249,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/1819646091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":1819646091},{"type":"v1","addr":"172.21.15.78:6836","nonce":1819646091}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:44:06.850489+0000","modified":"2023-10-11T09:44:16.715978+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6252},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6252":{"gid":6252,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6837/852594750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":852594750},{"type":"v1","addr":"172.21.15.78:6837","nonce":852594750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9,10],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-11T09:44:17.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.162+0000 7fec70836700 1 -- 172.21.15.78:0/3380169009 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fec58065ed0 msgr2=0x7fec58068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:17.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.162+0000 7fec70836700 1 --2- 172.21.15.78:0/3380169009 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fec58065ed0 0x7fec58068380 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fec5c000f60 tx=0x7fec5c007e00 comp rx=0 tx=0).stop 2023-10-11T09:44:17.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.162+0000 7fec70836700 1 -- 172.21.15.78:0/3380169009 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec6c15b8e0 msgr2=0x7fec6c1421d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:17.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.162+0000 7fec70836700 1 --2- 172.21.15.78:0/3380169009 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec6c15b8e0 0x7fec6c1421d0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fec6c072460 tx=0x7fec64004af0 comp rx=0 tx=0).stop 2023-10-11T09:44:17.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.162+0000 7fec70836700 1 -- 172.21.15.78:0/3380169009 shutdown_connections 2023-10-11T09:44:17.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.162+0000 7fec70836700 1 --2- 172.21.15.78:0/3380169009 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fec58065ed0 0x7fec58068380 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:17.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.162+0000 7fec70836700 1 --2- 172.21.15.78:0/3380169009 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec6c15b8e0 0x7fec6c1421d0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:17.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.162+0000 7fec70836700 1 --2- 172.21.15.78:0/3380169009 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fec6c15a940 0x7fec6c0c6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:17.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.162+0000 7fec70836700 1 --2- 172.21.15.78:0/3380169009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec6c0ca780 0x7fec6c159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:17.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.162+0000 7fec70836700 1 -- 172.21.15.78:0/3380169009 >> 172.21.15.78:0/3380169009 conn(0x7fec6c0cea60 msgr2=0x7fec6c06d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:17.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.162+0000 7fec70836700 1 -- 172.21.15.78:0/3380169009 shutdown_connections 2023-10-11T09:44:17.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.162+0000 7fec70836700 1 -- 172.21.15.78:0/3380169009 wait complete. 2023-10-11T09:44:17.171 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 38 2023-10-11T09:44:17.188 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:44:17.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.362+0000 7efcade7b700 1 Processor -- start 2023-10-11T09:44:17.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.362+0000 7efcade7b700 1 -- start start 2023-10-11T09:44:17.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.366+0000 7efcade7b700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efca815c880 0x7efca806be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:17.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.366+0000 7efcade7b700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efca80cbd30 0x7efca80c9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:17.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.366+0000 7efcade7b700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efca80c93a0 con 0x7efca80cd210 2023-10-11T09:44:17.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.366+0000 7efcade7b700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efca8161420 con 0x7efca80cbd30 2023-10-11T09:44:17.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.366+0000 7efcade7b700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efca80cca10 con 0x7efca815c880 2023-10-11T09:44:17.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.366+0000 7efca7fff700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efca80cbd30 0x7efca80c9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:17.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.366+0000 7efca77fe700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efca815c880 0x7efca806be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:17.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.366+0000 7efca7fff700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efca80cbd30 0x7efca80c9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:37816/0 (socket says 172.21.15.78:37816) 2023-10-11T09:44:17.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.366+0000 7efca7fff700 1 -- 172.21.15.78:0/473169370 learned_addr learned my addr 172.21.15.78:0/473169370 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:17.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.366+0000 7efca7fff700 1 -- 172.21.15.78:0/473169370 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efca815c880 msgr2=0x7efca806be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:44:17.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.366+0000 7efca7fff700 1 --2- 172.21.15.78:0/473169370 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efca815c880 0x7efca806be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:17.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.366+0000 7efca7fff700 1 -- 172.21.15.78:0/473169370 >> v1:172.21.15.133:6789/0 conn(0x7efca80cd210 legacy=0x7efca80cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:17.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.366+0000 7efca7fff700 1 -- 172.21.15.78:0/473169370 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca80cccb0 con 0x7efca80cbd30 2023-10-11T09:44:17.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.366+0000 7efca7fff700 1 --2- 172.21.15.78:0/473169370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efca80cbd30 0x7efca80c9b40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efca0009e50 tx=0x7efca000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79dad2c5331dc2d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:17.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.366+0000 7efca6ffd700 1 -- 172.21.15.78:0/473169370 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca000e070 con 0x7efca80cbd30 2023-10-11T09:44:17.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.370+0000 7efca6ffd700 1 -- 172.21.15.78:0/473169370 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca0003a40 con 0x7efca80cbd30 2023-10-11T09:44:17.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.370+0000 7efca6ffd700 1 -- 172.21.15.78:0/473169370 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca0003df0 con 0x7efca80cbd30 2023-10-11T09:44:17.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.370+0000 7efcade7b700 1 -- 172.21.15.78:0/473169370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efca80cbd30 msgr2=0x7efca80c9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:17.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.370+0000 7efcade7b700 1 --2- 172.21.15.78:0/473169370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efca80cbd30 0x7efca80c9b40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efca0009e50 tx=0x7efca000b330 comp rx=0 tx=0).stop 2023-10-11T09:44:17.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.370+0000 7efcade7b700 1 -- 172.21.15.78:0/473169370 shutdown_connections 2023-10-11T09:44:17.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.370+0000 7efcade7b700 1 --2- 172.21.15.78:0/473169370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efca80cbd30 0x7efca80c9b40 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:17.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.370+0000 7efcade7b700 1 --2- 172.21.15.78:0/473169370 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efca815c880 0x7efca806be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:17.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.370+0000 7efcade7b700 1 -- 172.21.15.78:0/473169370 >> 172.21.15.78:0/473169370 conn(0x7efca806aca0 msgr2=0x7efca80b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:17.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.370+0000 7efcade7b700 1 -- 172.21.15.78:0/473169370 shutdown_connections 2023-10-11T09:44:17.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.370+0000 7efcade7b700 1 -- 172.21.15.78:0/473169370 wait complete. 2023-10-11T09:44:17.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.370+0000 7efcade7b700 1 Processor -- start 2023-10-11T09:44:17.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.370+0000 7efcade7b700 1 -- start start 2023-10-11T09:44:17.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.374+0000 7efcade7b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efca80cbd30 0x7efca8157f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:17.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.374+0000 7efcade7b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efca80cd210 0x7efca81568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:17.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.374+0000 7efcade7b700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efca815c880 0x7efca813feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:17.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.374+0000 7efcade7b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca8156ef0 con 0x7efca80cd210 2023-10-11T09:44:17.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.374+0000 7efcade7b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca81571d0 con 0x7efca80cbd30 2023-10-11T09:44:17.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.374+0000 7efcade7b700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efca81574b0 con 0x7efca815c880 2023-10-11T09:44:17.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.374+0000 7efc9ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efca80cd210 0x7efca81568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:17.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.374+0000 7efc9ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efca80cd210 0x7efca81568f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52970/0 (socket says 172.21.15.78:52970) 2023-10-11T09:44:17.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.374+0000 7efc9ffff700 1 -- 172.21.15.78:0/2109760174 learned_addr learned my addr 172.21.15.78:0/2109760174 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:44:17.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.374+0000 7efca77fe700 1 --2- 172.21.15.78:0/2109760174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efca80cbd30 0x7efca8157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:17.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.374+0000 7efca7fff700 1 --2- 172.21.15.78:0/2109760174 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efca815c880 0x7efca813feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:17.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.374+0000 7efc9ffff700 1 -- 172.21.15.78:0/2109760174 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efca815c880 msgr2=0x7efca813feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:17.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.374+0000 7efc9ffff700 1 --2- 172.21.15.78:0/2109760174 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efca815c880 0x7efca813feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:17.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.374+0000 7efc9ffff700 1 -- 172.21.15.78:0/2109760174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efca80cbd30 msgr2=0x7efca8157f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:17.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.374+0000 7efc9ffff700 1 --2- 172.21.15.78:0/2109760174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efca80cbd30 0x7efca8157f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:17.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.374+0000 7efc9ffff700 1 -- 172.21.15.78:0/2109760174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca0009b30 con 0x7efca80cd210 2023-10-11T09:44:17.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.374+0000 7efc9ffff700 1 --2- 172.21.15.78:0/2109760174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efca80cd210 0x7efca81568f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7efc9000c6b0 tx=0x7efc9000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:17.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.374+0000 7efca4ff9700 1 -- 172.21.15.78:0/2109760174 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc900054e0 con 0x7efca80cd210 2023-10-11T09:44:17.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.374+0000 7efca4ff9700 1 -- 172.21.15.78:0/2109760174 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc900036a0 con 0x7efca80cd210 2023-10-11T09:44:17.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.374+0000 7efcade7b700 1 -- 172.21.15.78:0/2109760174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efca8157790 con 0x7efca80cd210 2023-10-11T09:44:17.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.374+0000 7efca4ff9700 1 -- 172.21.15.78:0/2109760174 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc90003a50 con 0x7efca80cd210 2023-10-11T09:44:17.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.374+0000 7efcade7b700 1 -- 172.21.15.78:0/2109760174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efca8142fd0 con 0x7efca80cd210 2023-10-11T09:44:17.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.378+0000 7efcade7b700 1 -- 172.21.15.78:0/2109760174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efca8157cb0 con 0x7efca80cd210 2023-10-11T09:44:17.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.378+0000 7efca4ff9700 1 -- 172.21.15.78:0/2109760174 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7efc90003bb0 con 0x7efca80cd210 2023-10-11T09:44:17.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.382+0000 7efca4ff9700 1 --2- 172.21.15.78:0/2109760174 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efc88065ed0 0x7efc88068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:44:17.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.382+0000 7efca4ff9700 1 -- 172.21.15.78:0/2109760174 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6466+0+0 (secure 0 0 0) 0x7efc90083070 con 0x7efca80cd210 2023-10-11T09:44:17.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.382+0000 7efca77fe700 1 --2- 172.21.15.78:0/2109760174 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efc88065ed0 0x7efc88068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:44:17.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.382+0000 7efca77fe700 1 --2- 172.21.15.78:0/2109760174 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efc88065ed0 0x7efc88068380 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7efc98009b50 tx=0x7efc98009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:44:17.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.386+0000 7efca4ff9700 1 -- 172.21.15.78:0/2109760174 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7efc900b3fa0 con 0x7efca80cd210 2023-10-11T09:44:17.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.558+0000 7efcade7b700 1 -- 172.21.15.78:0/2109760174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efca806d390 con 0x7efca80cd210 2023-10-11T09:44:17.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.562+0000 7efca4ff9700 1 -- 172.21.15.78:0/2109760174 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+19678 (secure 0 0 0) 0x7efc90017070 con 0x7efca80cd210 2023-10-11T09:44:17.567 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:44:17.567 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":63,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:44:16.715650+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:44:05.198742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-11T09:44:05.868171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data_extra","create_time":"2023-10-11T09:44:16.049941+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:44:17.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.562+0000 7efcade7b700 1 -- 172.21.15.78:0/2109760174 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efc88065ed0 msgr2=0x7efc88068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:17.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.562+0000 7efcade7b700 1 --2- 172.21.15.78:0/2109760174 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efc88065ed0 0x7efc88068380 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7efc98009b50 tx=0x7efc98009240 comp rx=0 tx=0).stop 2023-10-11T09:44:17.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.562+0000 7efcade7b700 1 -- 172.21.15.78:0/2109760174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efca80cd210 msgr2=0x7efca81568f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:44:17.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.562+0000 7efcade7b700 1 --2- 172.21.15.78:0/2109760174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efca80cd210 0x7efca81568f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7efc9000c6b0 tx=0x7efc9000c9c0 comp rx=0 tx=0).stop 2023-10-11T09:44:17.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.562+0000 7efcade7b700 1 -- 172.21.15.78:0/2109760174 shutdown_connections 2023-10-11T09:44:17.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.562+0000 7efcade7b700 1 --2- 172.21.15.78:0/2109760174 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efc88065ed0 0x7efc88068380 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:17.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.562+0000 7efcade7b700 1 --2- 172.21.15.78:0/2109760174 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efca815c880 0x7efca813feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:17.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.562+0000 7efcade7b700 1 --2- 172.21.15.78:0/2109760174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efca80cd210 0x7efca81568f0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:17.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.562+0000 7efcade7b700 1 --2- 172.21.15.78:0/2109760174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efca80cbd30 0x7efca8157f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:44:17.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.562+0000 7efcade7b700 1 -- 172.21.15.78:0/2109760174 >> 172.21.15.78:0/2109760174 conn(0x7efca806aca0 msgr2=0x7efca8151330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:44:17.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.566+0000 7efcade7b700 1 -- 172.21.15.78:0/2109760174 shutdown_connections 2023-10-11T09:44:17.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:44:17.566+0000 7efcade7b700 1 -- 172.21.15.78:0/2109760174 wait complete. 2023-10-11T09:44:17.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:44:17.593 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v cephfs_data_extra .) 2023-10-11T09:44:17.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:44:17.623 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2023-10-11T09:44:17.697 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/0 2023-10-11T09:44:17.698 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:44:17.698 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:44:17.699 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/0" 2023-10-11T09:44:17.699 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'w'"'"') as f: 2023-10-11T09:44:17.699 DEBUG:teuthology.orchestra.run.smithi078:> for i in range(0, 6291456): 2023-10-11T09:44:17.700 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:44:17.700 DEBUG:teuthology.orchestra.run.smithi078:> f.write(chr(val)) 2023-10-11T09:44:17.701 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:44:21.022 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/1 2023-10-11T09:44:21.023 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:44:21.023 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:44:21.023 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/1" 2023-10-11T09:44:21.024 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'w'"'"') as f: 2023-10-11T09:44:21.024 DEBUG:teuthology.orchestra.run.smithi078:> for i in range(0, 6291456): 2023-10-11T09:44:21.024 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:44:21.025 DEBUG:teuthology.orchestra.run.smithi078:> f.write(chr(val)) 2023-10-11T09:44:21.025 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:44:24.357 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/2 2023-10-11T09:44:24.358 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:44:24.358 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:44:24.358 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/2" 2023-10-11T09:44:24.359 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'w'"'"') as f: 2023-10-11T09:44:24.359 DEBUG:teuthology.orchestra.run.smithi078:> for i in range(0, 6291456): 2023-10-11T09:44:24.360 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:44:24.360 DEBUG:teuthology.orchestra.run.smithi078:> f.write(chr(val)) 2023-10-11T09:44:24.361 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:44:27.681 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/3 2023-10-11T09:44:27.682 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:44:27.682 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:44:27.682 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/3" 2023-10-11T09:44:27.683 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'w'"'"') as f: 2023-10-11T09:44:27.683 DEBUG:teuthology.orchestra.run.smithi078:> for i in range(0, 6291456): 2023-10-11T09:44:27.683 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:44:27.684 DEBUG:teuthology.orchestra.run.smithi078:> f.write(chr(val)) 2023-10-11T09:44:27.684 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:44:29.700 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:44:29.708 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:44:31.011 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/4 2023-10-11T09:44:31.011 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:44:31.012 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:44:31.012 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/4" 2023-10-11T09:44:31.012 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'w'"'"') as f: 2023-10-11T09:44:31.013 DEBUG:teuthology.orchestra.run.smithi078:> for i in range(0, 6291456): 2023-10-11T09:44:31.013 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:44:31.013 DEBUG:teuthology.orchestra.run.smithi078:> f.write(chr(val)) 2023-10-11T09:44:31.014 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:44:34.298 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/5 2023-10-11T09:44:34.299 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:44:34.299 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:44:34.300 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/5" 2023-10-11T09:44:34.300 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'w'"'"') as f: 2023-10-11T09:44:34.300 DEBUG:teuthology.orchestra.run.smithi078:> for i in range(0, 6291456): 2023-10-11T09:44:34.301 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:44:34.301 DEBUG:teuthology.orchestra.run.smithi078:> f.write(chr(val)) 2023-10-11T09:44:34.302 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:44:37.631 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/6 2023-10-11T09:44:37.631 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:44:37.632 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:44:37.632 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/6" 2023-10-11T09:44:37.633 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'w'"'"') as f: 2023-10-11T09:44:37.633 DEBUG:teuthology.orchestra.run.smithi078:> for i in range(0, 6291456): 2023-10-11T09:44:37.633 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:44:37.634 DEBUG:teuthology.orchestra.run.smithi078:> f.write(chr(val)) 2023-10-11T09:44:37.634 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:44:40.968 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/7 2023-10-11T09:44:40.968 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:44:40.969 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:44:40.969 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/7" 2023-10-11T09:44:40.970 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'w'"'"') as f: 2023-10-11T09:44:40.970 DEBUG:teuthology.orchestra.run.smithi078:> for i in range(0, 6291456): 2023-10-11T09:44:40.970 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:44:40.971 DEBUG:teuthology.orchestra.run.smithi078:> f.write(chr(val)) 2023-10-11T09:44:40.971 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:44:44.294 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/8 2023-10-11T09:44:44.295 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:44:44.295 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:44:44.296 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/8" 2023-10-11T09:44:44.296 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'w'"'"') as f: 2023-10-11T09:44:44.297 DEBUG:teuthology.orchestra.run.smithi078:> for i in range(0, 6291456): 2023-10-11T09:44:44.297 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:44:44.297 DEBUG:teuthology.orchestra.run.smithi078:> f.write(chr(val)) 2023-10-11T09:44:44.298 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:44:47.629 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/9 2023-10-11T09:44:47.630 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:44:47.630 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:44:47.630 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/9" 2023-10-11T09:44:47.631 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'w'"'"') as f: 2023-10-11T09:44:47.631 DEBUG:teuthology.orchestra.run.smithi078:> for i in range(0, 6291456): 2023-10-11T09:44:47.632 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:44:47.632 DEBUG:teuthology.orchestra.run.smithi078:> f.write(chr(val)) 2023-10-11T09:44:47.632 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:44:50.957 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/10 2023-10-11T09:44:50.958 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:44:50.958 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:44:50.959 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/10" 2023-10-11T09:44:50.959 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'w'"'"') as f: 2023-10-11T09:44:50.959 DEBUG:teuthology.orchestra.run.smithi078:> for i in range(0, 6291456): 2023-10-11T09:44:50.960 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:44:50.960 DEBUG:teuthology.orchestra.run.smithi078:> f.write(chr(val)) 2023-10-11T09:44:50.960 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:44:54.293 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/11 2023-10-11T09:44:54.294 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:44:54.294 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:44:54.295 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/11" 2023-10-11T09:44:54.295 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'w'"'"') as f: 2023-10-11T09:44:54.295 DEBUG:teuthology.orchestra.run.smithi078:> for i in range(0, 6291456): 2023-10-11T09:44:54.296 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:44:54.296 DEBUG:teuthology.orchestra.run.smithi078:> f.write(chr(val)) 2023-10-11T09:44:54.297 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:44:57.421 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/12 2023-10-11T09:44:57.422 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:44:57.422 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:44:57.423 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/12" 2023-10-11T09:44:57.423 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'w'"'"') as f: 2023-10-11T09:44:57.423 DEBUG:teuthology.orchestra.run.smithi078:> for i in range(0, 6291456): 2023-10-11T09:44:57.424 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:44:57.424 DEBUG:teuthology.orchestra.run.smithi078:> f.write(chr(val)) 2023-10-11T09:44:57.424 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:44:59.787 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:44:59.795 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:45:00.748 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/13 2023-10-11T09:45:00.749 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:45:00.749 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:45:00.749 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/13" 2023-10-11T09:45:00.750 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'w'"'"') as f: 2023-10-11T09:45:00.750 DEBUG:teuthology.orchestra.run.smithi078:> for i in range(0, 6291456): 2023-10-11T09:45:00.750 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:45:00.751 DEBUG:teuthology.orchestra.run.smithi078:> f.write(chr(val)) 2023-10-11T09:45:00.751 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:45:03.833 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/14 2023-10-11T09:45:03.834 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:45:03.834 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:45:03.835 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/14" 2023-10-11T09:45:03.835 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'w'"'"') as f: 2023-10-11T09:45:03.835 DEBUG:teuthology.orchestra.run.smithi078:> for i in range(0, 6291456): 2023-10-11T09:45:03.836 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:45:03.836 DEBUG:teuthology.orchestra.run.smithi078:> f.write(chr(val)) 2023-10-11T09:45:03.836 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:45:06.950 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/15 2023-10-11T09:45:06.951 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:45:06.951 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:45:06.951 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/15" 2023-10-11T09:45:06.952 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'w'"'"') as f: 2023-10-11T09:45:06.952 DEBUG:teuthology.orchestra.run.smithi078:> for i in range(0, 6291456): 2023-10-11T09:45:06.953 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:45:06.953 DEBUG:teuthology.orchestra.run.smithi078:> f.write(chr(val)) 2023-10-11T09:45:06.953 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:45:10.282 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/16 2023-10-11T09:45:10.283 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:45:10.284 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:45:10.284 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/16" 2023-10-11T09:45:10.285 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'w'"'"') as f: 2023-10-11T09:45:10.286 DEBUG:teuthology.orchestra.run.smithi078:> for i in range(0, 6291456): 2023-10-11T09:45:10.286 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:45:10.287 DEBUG:teuthology.orchestra.run.smithi078:> f.write(chr(val)) 2023-10-11T09:45:10.287 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:45:13.618 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/17 2023-10-11T09:45:13.621 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:45:13.622 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:45:13.623 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/17" 2023-10-11T09:45:13.623 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'w'"'"') as f: 2023-10-11T09:45:13.623 DEBUG:teuthology.orchestra.run.smithi078:> for i in range(0, 6291456): 2023-10-11T09:45:13.624 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:45:13.624 DEBUG:teuthology.orchestra.run.smithi078:> f.write(chr(val)) 2023-10-11T09:45:13.624 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:45:16.948 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/18 2023-10-11T09:45:16.949 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:45:16.949 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:45:16.950 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/18" 2023-10-11T09:45:16.950 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'w'"'"') as f: 2023-10-11T09:45:16.950 DEBUG:teuthology.orchestra.run.smithi078:> for i in range(0, 6291456): 2023-10-11T09:45:16.951 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:45:16.951 DEBUG:teuthology.orchestra.run.smithi078:> f.write(chr(val)) 2023-10-11T09:45:16.951 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:45:20.281 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/19 2023-10-11T09:45:20.282 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:45:20.282 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:45:20.282 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/19" 2023-10-11T09:45:20.283 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'w'"'"') as f: 2023-10-11T09:45:20.283 DEBUG:teuthology.orchestra.run.smithi078:> for i in range(0, 6291456): 2023-10-11T09:45:20.283 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:45:20.284 DEBUG:teuthology.orchestra.run.smithi078:> f.write(chr(val)) 2023-10-11T09:45:20.284 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:45:23.614 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/20 2023-10-11T09:45:23.615 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:45:23.615 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:45:23.616 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/20" 2023-10-11T09:45:23.616 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'w'"'"') as f: 2023-10-11T09:45:23.616 DEBUG:teuthology.orchestra.run.smithi078:> for i in range(0, 6291456): 2023-10-11T09:45:23.617 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:45:23.617 DEBUG:teuthology.orchestra.run.smithi078:> f.write(chr(val)) 2023-10-11T09:45:23.617 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:45:26.948 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/21 2023-10-11T09:45:26.949 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:45:26.950 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:45:26.950 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/21" 2023-10-11T09:45:26.951 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'w'"'"') as f: 2023-10-11T09:45:26.951 DEBUG:teuthology.orchestra.run.smithi078:> for i in range(0, 6291456): 2023-10-11T09:45:26.951 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:45:26.952 DEBUG:teuthology.orchestra.run.smithi078:> f.write(chr(val)) 2023-10-11T09:45:26.952 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:45:29.868 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:45:29.876 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:45:30.277 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/22 2023-10-11T09:45:30.278 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:45:30.278 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:45:30.279 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/22" 2023-10-11T09:45:30.279 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'w'"'"') as f: 2023-10-11T09:45:30.279 DEBUG:teuthology.orchestra.run.smithi078:> for i in range(0, 6291456): 2023-10-11T09:45:30.280 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:45:30.280 DEBUG:teuthology.orchestra.run.smithi078:> f.write(chr(val)) 2023-10-11T09:45:30.281 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:45:33.566 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/23 2023-10-11T09:45:33.566 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:45:33.567 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:45:33.567 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/23" 2023-10-11T09:45:33.567 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'w'"'"') as f: 2023-10-11T09:45:33.568 DEBUG:teuthology.orchestra.run.smithi078:> for i in range(0, 6291456): 2023-10-11T09:45:33.568 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:45:33.568 DEBUG:teuthology.orchestra.run.smithi078:> f.write(chr(val)) 2023-10-11T09:45:33.569 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:45:36.897 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/24 2023-10-11T09:45:36.898 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:45:36.898 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:45:36.899 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/24" 2023-10-11T09:45:36.899 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'w'"'"') as f: 2023-10-11T09:45:36.899 DEBUG:teuthology.orchestra.run.smithi078:> for i in range(0, 6291456): 2023-10-11T09:45:36.900 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:45:36.900 DEBUG:teuthology.orchestra.run.smithi078:> f.write(chr(val)) 2023-10-11T09:45:36.900 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:45:40.226 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:45:40.226 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:45:40.288 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:45:40.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.500+0000 7f0a3a220700 1 Processor -- start 2023-10-11T09:45:40.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.500+0000 7f0a3a220700 1 -- start start 2023-10-11T09:45:40.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.504+0000 7f0a3a220700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a340d1160 0x7f0a340cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:40.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.504+0000 7f0a3a220700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a3415dc10 con 0x7f0a340d1660 2023-10-11T09:45:40.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.504+0000 7f0a3a220700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a34160380 con 0x7f0a34156ce0 2023-10-11T09:45:40.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.504+0000 7f0a3a220700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3415d930 con 0x7f0a340d1160 2023-10-11T09:45:40.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.504+0000 7f0a38a1d700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f0a340d1660 0x7f0a340d0a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:44522/0 (socket says 172.21.15.78:44522) 2023-10-11T09:45:40.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.504+0000 7f0a38a1d700 1 -- 172.21.15.78:0/3950793920 learned_addr learned my addr 172.21.15.78:0/3950793920 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:45:40.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.504+0000 7f0a39a1f700 1 --2- 172.21.15.78:0/3950793920 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a340d1160 0x7f0a340cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:40.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.504+0000 7f0a39a1f700 1 -- 172.21.15.78:0/3950793920 >> v1:172.21.15.133:6789/0 conn(0x7f0a34156ce0 legacy=0x7f0a34160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:40.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.504+0000 7f0a39a1f700 1 -- 172.21.15.78:0/3950793920 >> v1:172.21.15.78:6789/0 conn(0x7f0a340d1660 legacy=0x7f0a340d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:40.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.504+0000 7f0a39a1f700 1 -- 172.21.15.78:0/3950793920 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a34160620 con 0x7f0a340d1160 2023-10-11T09:45:40.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.504+0000 7f0a39a1f700 1 --2- 172.21.15.78:0/3950793920 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a340d1160 0x7f0a340cefc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c009880 tx=0x7f0a2c009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a516c920619ddea server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:40.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.504+0000 7f0a2bfff700 1 -- 172.21.15.78:0/3950793920 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c011830 con 0x7f0a340d1160 2023-10-11T09:45:40.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.504+0000 7f0a2bfff700 1 -- 172.21.15.78:0/3950793920 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a2c011990 con 0x7f0a340d1160 2023-10-11T09:45:40.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.504+0000 7f0a2bfff700 1 -- 172.21.15.78:0/3950793920 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c011d20 con 0x7f0a340d1160 2023-10-11T09:45:40.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.508+0000 7f0a3a220700 1 -- 172.21.15.78:0/3950793920 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a340d1160 msgr2=0x7f0a340cefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:40.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.508+0000 7f0a3a220700 1 --2- 172.21.15.78:0/3950793920 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a340d1160 0x7f0a340cefc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c009880 tx=0x7f0a2c009b90 comp rx=0 tx=0).stop 2023-10-11T09:45:40.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.508+0000 7f0a3a220700 1 -- 172.21.15.78:0/3950793920 shutdown_connections 2023-10-11T09:45:40.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.508+0000 7f0a3a220700 1 --2- 172.21.15.78:0/3950793920 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a340d1160 0x7f0a340cefc0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:40.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.508+0000 7f0a3a220700 1 -- 172.21.15.78:0/3950793920 >> 172.21.15.78:0/3950793920 conn(0x7f0a3406afb0 msgr2=0x7f0a3406b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:40.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.508+0000 7f0a3a220700 1 -- 172.21.15.78:0/3950793920 shutdown_connections 2023-10-11T09:45:40.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.508+0000 7f0a3a220700 1 -- 172.21.15.78:0/3950793920 wait complete. 2023-10-11T09:45:40.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.508+0000 7f0a3a220700 1 Processor -- start 2023-10-11T09:45:40.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.512+0000 7f0a3a220700 1 -- start start 2023-10-11T09:45:40.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.512+0000 7f0a3a220700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a340d1160 0x7f0a34158be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:40.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.512+0000 7f0a3a220700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a340d1660 0x7f0a341f4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:40.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.512+0000 7f0a3a220700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a34156ce0 0x7f0a340cc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:40.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.512+0000 7f0a3a220700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a340ca2c0 con 0x7f0a340d1160 2023-10-11T09:45:40.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.512+0000 7f0a3a220700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a340ca5a0 con 0x7f0a340d1660 2023-10-11T09:45:40.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.512+0000 7f0a3a220700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a340ca880 con 0x7f0a34156ce0 2023-10-11T09:45:40.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.512+0000 7f0a3921e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a340d1160 0x7f0a34158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:40.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.512+0000 7f0a3921e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a340d1160 0x7f0a34158be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60684/0 (socket says 172.21.15.78:60684) 2023-10-11T09:45:40.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.512+0000 7f0a3921e700 1 -- 172.21.15.78:0/3250642779 learned_addr learned my addr 172.21.15.78:0/3250642779 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:40.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.512+0000 7f0a38a1d700 1 --2- 172.21.15.78:0/3250642779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a340d1660 0x7f0a341f4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:40.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.512+0000 7f0a39a1f700 1 --2- 172.21.15.78:0/3250642779 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a34156ce0 0x7f0a340cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:40.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.512+0000 7f0a3921e700 1 -- 172.21.15.78:0/3250642779 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a34156ce0 msgr2=0x7f0a340cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:40.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.512+0000 7f0a3921e700 1 --2- 172.21.15.78:0/3250642779 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a34156ce0 0x7f0a340cc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:40.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.512+0000 7f0a3921e700 1 -- 172.21.15.78:0/3250642779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a340d1660 msgr2=0x7f0a341f4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:40.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.512+0000 7f0a3921e700 1 --2- 172.21.15.78:0/3250642779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a340d1660 0x7f0a341f4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:40.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.512+0000 7f0a3921e700 1 -- 172.21.15.78:0/3250642779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a2c0094b0 con 0x7f0a340d1160 2023-10-11T09:45:40.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.512+0000 7f0a3921e700 1 --2- 172.21.15.78:0/3250642779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a340d1160 0x7f0a34158be0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f0a3000cbc0 tx=0x7f0a3000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:40.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.512+0000 7f0a29ffb700 1 -- 172.21.15.78:0/3250642779 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a30003c20 con 0x7f0a340d1160 2023-10-11T09:45:40.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.512+0000 7f0a29ffb700 1 -- 172.21.15.78:0/3250642779 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a30003d80 con 0x7f0a340d1160 2023-10-11T09:45:40.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.512+0000 7f0a29ffb700 1 -- 172.21.15.78:0/3250642779 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3000aaa0 con 0x7f0a340d1160 2023-10-11T09:45:40.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.512+0000 7f0a3a220700 1 -- 172.21.15.78:0/3250642779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a340cab60 con 0x7f0a340d1160 2023-10-11T09:45:40.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.516+0000 7f0a3a220700 1 -- 172.21.15.78:0/3250642779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a340c9f50 con 0x7f0a340d1160 2023-10-11T09:45:40.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.516+0000 7f0a3a220700 1 -- 172.21.15.78:0/3250642779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a34065d40 con 0x7f0a340d1160 2023-10-11T09:45:40.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.516+0000 7f0a29ffb700 1 -- 172.21.15.78:0/3250642779 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f0a3000ac00 con 0x7f0a340d1160 2023-10-11T09:45:40.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.520+0000 7f0a29ffb700 1 --2- 172.21.15.78:0/3250642779 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0a18065ed0 0x7f0a18068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:40.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.520+0000 7f0a29ffb700 1 -- 172.21.15.78:0/3250642779 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f0a30083900 con 0x7f0a340d1160 2023-10-11T09:45:40.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.520+0000 7f0a38a1d700 1 --2- 172.21.15.78:0/3250642779 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0a18065ed0 0x7f0a18068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:40.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.520+0000 7f0a38a1d700 1 --2- 172.21.15.78:0/3250642779 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0a18065ed0 0x7f0a18068380 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0a34154130 tx=0x7f0a24009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:40.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.524+0000 7f0a29ffb700 1 -- 172.21.15.78:0/3250642779 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f0a30050da0 con 0x7f0a340d1160 2023-10-11T09:45:40.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.700+0000 7f0a3a220700 1 -- 172.21.15.78:0/3250642779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a341549c0 con 0x7f0a340d1160 2023-10-11T09:45:40.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.704+0000 7f0a29ffb700 1 -- 172.21.15.78:0/3250642779 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+19665 (secure 0 0 0) 0x7f0a300543c0 con 0x7f0a340d1160 2023-10-11T09:45:40.711 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:40.711 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":64,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:44:17.719929+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:44:05.198742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-11T09:44:05.868171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data_extra","create_time":"2023-10-11T09:44:16.049941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:45:40.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.704+0000 7f0a3a220700 1 -- 172.21.15.78:0/3250642779 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0a18065ed0 msgr2=0x7f0a18068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:40.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.704+0000 7f0a3a220700 1 --2- 172.21.15.78:0/3250642779 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0a18065ed0 0x7f0a18068380 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0a34154130 tx=0x7f0a24009040 comp rx=0 tx=0).stop 2023-10-11T09:45:40.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.704+0000 7f0a3a220700 1 -- 172.21.15.78:0/3250642779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a340d1160 msgr2=0x7f0a34158be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:40.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.704+0000 7f0a3a220700 1 --2- 172.21.15.78:0/3250642779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a340d1160 0x7f0a34158be0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f0a3000cbc0 tx=0x7f0a3000ced0 comp rx=0 tx=0).stop 2023-10-11T09:45:40.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.704+0000 7f0a3a220700 1 -- 172.21.15.78:0/3250642779 shutdown_connections 2023-10-11T09:45:40.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.704+0000 7f0a3a220700 1 --2- 172.21.15.78:0/3250642779 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0a18065ed0 0x7f0a18068380 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:40.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.704+0000 7f0a3a220700 1 --2- 172.21.15.78:0/3250642779 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a34156ce0 0x7f0a340cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:40.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.704+0000 7f0a3a220700 1 --2- 172.21.15.78:0/3250642779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a340d1660 0x7f0a341f4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:40.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.704+0000 7f0a3a220700 1 --2- 172.21.15.78:0/3250642779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a340d1160 0x7f0a34158be0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:40.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.704+0000 7f0a3a220700 1 -- 172.21.15.78:0/3250642779 >> 172.21.15.78:0/3250642779 conn(0x7f0a3406afb0 msgr2=0x7f0a341562d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:40.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.704+0000 7f0a3a220700 1 -- 172.21.15.78:0/3250642779 shutdown_connections 2023-10-11T09:45:40.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.708+0000 7f0a3a220700 1 -- 172.21.15.78:0/3250642779 wait complete. 2023-10-11T09:45:40.736 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T09:44:05.198742+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T09:44:05.868171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-10-11T09:44:16.049941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:45:40.766 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:45:40.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.928+0000 7fe1465f9700 1 Processor -- start 2023-10-11T09:45:40.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.932+0000 7fe1465f9700 1 -- start start 2023-10-11T09:45:40.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.932+0000 7fe1465f9700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe140158430 0x7fe1400710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:40.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.932+0000 7fe1465f9700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe14006d340 0x7fe1400ca880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:40.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.932+0000 7fe1465f9700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe140071610 con 0x7fe1400cadc0 2023-10-11T09:45:40.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.932+0000 7fe1465f9700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe140070c10 con 0x7fe14006d340 2023-10-11T09:45:40.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.932+0000 7fe1465f9700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1400cb240 con 0x7fe140158430 2023-10-11T09:45:40.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.932+0000 7fe144b96700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fe1400cadc0 0x7fe14015afe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:44526/0 (socket says 172.21.15.78:44526) 2023-10-11T09:45:40.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.932+0000 7fe144b96700 1 -- 172.21.15.78:0/2897248582 learned_addr learned my addr 172.21.15.78:0/2897248582 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:45:40.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.932+0000 7fe13ffff700 1 --2- 172.21.15.78:0/2897248582 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe140158430 0x7fe1400710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:40.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.932+0000 7fe1377fe700 1 --2- 172.21.15.78:0/2897248582 >> v2:172.21.15.133:3300/0 conn(0x7fe14006d340 0x7fe1400ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:40.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.936+0000 7fe13f7fe700 1 -- 172.21.15.78:0/2897248582 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2485740129 0 0) 0x7fe140071610 con 0x7fe1400cadc0 2023-10-11T09:45:40.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.936+0000 7fe13f7fe700 1 -- 172.21.15.78:0/2897248582 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe12c0035d0 con 0x7fe1400cadc0 2023-10-11T09:45:40.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.936+0000 7fe1377fe700 1 -- 172.21.15.78:0/2897248582 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe140158430 msgr2=0x7fe1400710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:40.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.936+0000 7fe1377fe700 1 --2- 172.21.15.78:0/2897248582 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe140158430 0x7fe1400710d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:40.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.936+0000 7fe1377fe700 1 -- 172.21.15.78:0/2897248582 >> v1:172.21.15.78:6789/0 conn(0x7fe1400cadc0 legacy=0x7fe14015afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:40.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.936+0000 7fe1377fe700 1 -- 172.21.15.78:0/2897248582 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe14015e280 con 0x7fe14006d340 2023-10-11T09:45:40.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.936+0000 7fe1377fe700 1 --2- 172.21.15.78:0/2897248582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe14006d340 0x7fe1400ca880 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe128009850 tx=0x7fe128009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8af6417baeb37a48 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:40.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.936+0000 7fe13f7fe700 1 -- 172.21.15.78:0/2897248582 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe128011830 con 0x7fe14006d340 2023-10-11T09:45:40.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.936+0000 7fe13f7fe700 1 -- 172.21.15.78:0/2897248582 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe128011990 con 0x7fe14006d340 2023-10-11T09:45:40.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.936+0000 7fe13f7fe700 1 -- 172.21.15.78:0/2897248582 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe128011d40 con 0x7fe14006d340 2023-10-11T09:45:40.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.936+0000 7fe1465f9700 1 -- 172.21.15.78:0/2897248582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe14006d340 msgr2=0x7fe1400ca880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:40.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.936+0000 7fe1465f9700 1 --2- 172.21.15.78:0/2897248582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe14006d340 0x7fe1400ca880 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe128009850 tx=0x7fe128009b60 comp rx=0 tx=0).stop 2023-10-11T09:45:40.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.936+0000 7fe1465f9700 1 -- 172.21.15.78:0/2897248582 shutdown_connections 2023-10-11T09:45:40.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.936+0000 7fe1465f9700 1 --2- 172.21.15.78:0/2897248582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe14006d340 0x7fe1400ca880 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:40.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.936+0000 7fe1465f9700 1 --2- 172.21.15.78:0/2897248582 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe140158430 0x7fe1400710d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:40.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.936+0000 7fe1465f9700 1 -- 172.21.15.78:0/2897248582 >> 172.21.15.78:0/2897248582 conn(0x7fe1400cecc0 msgr2=0x7fe1400cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:40.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.936+0000 7fe1465f9700 1 -- 172.21.15.78:0/2897248582 shutdown_connections 2023-10-11T09:45:40.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.936+0000 7fe1465f9700 1 -- 172.21.15.78:0/2897248582 wait complete. 2023-10-11T09:45:40.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.940+0000 7fe1465f9700 1 Processor -- start 2023-10-11T09:45:40.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.940+0000 7fe1465f9700 1 -- start start 2023-10-11T09:45:40.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.940+0000 7fe1465f9700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe14006d340 0x7fe140146b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:40.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.940+0000 7fe1465f9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1400cadc0 0x7fe1400adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:40.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.940+0000 7fe1465f9700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe140158430 0x7fe1400715e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:40.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.940+0000 7fe1465f9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14014a150 con 0x7fe1400cadc0 2023-10-11T09:45:40.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.940+0000 7fe1465f9700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe140147050 con 0x7fe14006d340 2023-10-11T09:45:40.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.940+0000 7fe1465f9700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe140149540 con 0x7fe140158430 2023-10-11T09:45:40.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.940+0000 7fe1377fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1400cadc0 0x7fe1400adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:40.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.940+0000 7fe1377fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1400cadc0 0x7fe1400adaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60700/0 (socket says 172.21.15.78:60700) 2023-10-11T09:45:40.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.940+0000 7fe1377fe700 1 -- 172.21.15.78:0/3656603387 learned_addr learned my addr 172.21.15.78:0/3656603387 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:40.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.940+0000 7fe13ffff700 1 --2- 172.21.15.78:0/3656603387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe14006d340 0x7fe140146b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:40.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.940+0000 7fe1377fe700 1 -- 172.21.15.78:0/3656603387 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe140158430 msgr2=0x7fe1400715e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:40.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.940+0000 7fe1377fe700 1 --2- 172.21.15.78:0/3656603387 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe140158430 0x7fe1400715e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:40.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.940+0000 7fe1377fe700 1 -- 172.21.15.78:0/3656603387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe14006d340 msgr2=0x7fe140146b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:40.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.940+0000 7fe1377fe700 1 --2- 172.21.15.78:0/3656603387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe14006d340 0x7fe140146b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:40.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.940+0000 7fe1377fe700 1 -- 172.21.15.78:0/3656603387 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1280094b0 con 0x7fe1400cadc0 2023-10-11T09:45:40.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.940+0000 7fe13ffff700 1 --2- 172.21.15.78:0/3656603387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe14006d340 0x7fe140146b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:45:40.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.940+0000 7fe1377fe700 1 --2- 172.21.15.78:0/3656603387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1400cadc0 0x7fe1400adaf0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fe1280090b0 tx=0x7fe128004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:40.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.940+0000 7fe13d7fa700 1 -- 172.21.15.78:0/3656603387 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe128004480 con 0x7fe1400cadc0 2023-10-11T09:45:40.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.940+0000 7fe13d7fa700 1 -- 172.21.15.78:0/3656603387 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe12801f070 con 0x7fe1400cadc0 2023-10-11T09:45:40.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.940+0000 7fe13d7fa700 1 -- 172.21.15.78:0/3656603387 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe128003900 con 0x7fe1400cadc0 2023-10-11T09:45:40.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.944+0000 7fe1465f9700 1 -- 172.21.15.78:0/3656603387 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe140149820 con 0x7fe1400cadc0 2023-10-11T09:45:40.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.944+0000 7fe1465f9700 1 -- 172.21.15.78:0/3656603387 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe140149de0 con 0x7fe1400cadc0 2023-10-11T09:45:40.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.944+0000 7fe1465f9700 1 -- 172.21.15.78:0/3656603387 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe14014a300 con 0x7fe1400cadc0 2023-10-11T09:45:40.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.944+0000 7fe13d7fa700 1 -- 172.21.15.78:0/3656603387 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fe128021030 con 0x7fe1400cadc0 2023-10-11T09:45:40.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.948+0000 7fe13d7fa700 1 --2- 172.21.15.78:0/3656603387 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe120065ed0 0x7fe120068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:40.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.948+0000 7fe13d7fa700 1 -- 172.21.15.78:0/3656603387 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6466+0+0 (secure 0 0 0) 0x7fe128084d60 con 0x7fe1400cadc0 2023-10-11T09:45:40.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.948+0000 7fe13ffff700 1 --2- 172.21.15.78:0/3656603387 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe120065ed0 0x7fe120068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:40.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.948+0000 7fe13ffff700 1 --2- 172.21.15.78:0/3656603387 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe120065ed0 0x7fe120068380 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe130009ad0 tx=0x7fe130008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:40.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:40.952+0000 7fe13d7fa700 1 -- 172.21.15.78:0/3656603387 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fe14014a300 con 0x7fe1400cadc0 2023-10-11T09:45:41.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.112+0000 7fe1465f9700 1 -- 172.21.15.78:0/3656603387 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe14014a290 con 0x7fe1400cadc0 2023-10-11T09:45:41.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.116+0000 7fe13d7fa700 1 -- 172.21.15.78:0/3656603387 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v64) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe128052180 con 0x7fe1400cadc0 2023-10-11T09:45:41.123 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:45:41.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.116+0000 7fe1465f9700 1 -- 172.21.15.78:0/3656603387 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe120065ed0 msgr2=0x7fe120068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:41.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.116+0000 7fe1465f9700 1 --2- 172.21.15.78:0/3656603387 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe120065ed0 0x7fe120068380 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe130009ad0 tx=0x7fe130008040 comp rx=0 tx=0).stop 2023-10-11T09:45:41.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.116+0000 7fe1465f9700 1 -- 172.21.15.78:0/3656603387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1400cadc0 msgr2=0x7fe1400adaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:41.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.116+0000 7fe1465f9700 1 --2- 172.21.15.78:0/3656603387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1400cadc0 0x7fe1400adaf0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fe1280090b0 tx=0x7fe128004770 comp rx=0 tx=0).stop 2023-10-11T09:45:41.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.116+0000 7fe1465f9700 1 -- 172.21.15.78:0/3656603387 shutdown_connections 2023-10-11T09:45:41.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.116+0000 7fe1465f9700 1 --2- 172.21.15.78:0/3656603387 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe120065ed0 0x7fe120068380 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:41.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.116+0000 7fe1465f9700 1 --2- 172.21.15.78:0/3656603387 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe140158430 0x7fe1400715e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:41.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.116+0000 7fe1465f9700 1 --2- 172.21.15.78:0/3656603387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1400cadc0 0x7fe1400adaf0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:41.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.116+0000 7fe1465f9700 1 --2- 172.21.15.78:0/3656603387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe14006d340 0x7fe140146b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:41.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.116+0000 7fe1465f9700 1 -- 172.21.15.78:0/3656603387 >> 172.21.15.78:0/3656603387 conn(0x7fe1400cecc0 msgr2=0x7fe1400ae950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:41.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.116+0000 7fe1465f9700 1 -- 172.21.15.78:0/3656603387 shutdown_connections 2023-10-11T09:45:41.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.116+0000 7fe1465f9700 1 -- 172.21.15.78:0/3656603387 wait complete. 2023-10-11T09:45:41.147 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:45:41.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.316+0000 7fd71c9b5700 1 Processor -- start 2023-10-11T09:45:41.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.316+0000 7fd71c9b5700 1 -- start start 2023-10-11T09:45:41.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.320+0000 7fd71c9b5700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7180b3240 0x7fd7180d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:41.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.320+0000 7fd71c9b5700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd7180b3610 0x7fd7180b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:41.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.320+0000 7fd71c9b5700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7180b47f0 con 0x7fd71815d040 2023-10-11T09:45:41.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.320+0000 7fd71c9b5700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7180b3fd0 con 0x7fd7180b3610 2023-10-11T09:45:41.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.320+0000 7fd71c9b5700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7180b8130 con 0x7fd7180b3240 2023-10-11T09:45:41.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.320+0000 7fd717fff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fd71815d040 0x7fd7180b85c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:44538/0 (socket says 172.21.15.78:44538) 2023-10-11T09:45:41.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.320+0000 7fd717fff700 1 -- 172.21.15.78:0/2458101644 learned_addr learned my addr 172.21.15.78:0/2458101644 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:45:41.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.320+0000 7fd7177fe700 1 --2- 172.21.15.78:0/2458101644 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7180b3240 0x7fd7180d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:41.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.320+0000 7fd716ffd700 1 --2- 172.21.15.78:0/2458101644 >> v2:172.21.15.133:3300/0 conn(0x7fd7180b3610 0x7fd7180b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:41.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.320+0000 7fd7167fc700 1 -- 172.21.15.78:0/2458101644 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3193810368 0 0) 0x7fd710001490 con 0x7fd71815d040 2023-10-11T09:45:41.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.320+0000 7fd7167fc700 1 -- 172.21.15.78:0/2458101644 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7000035d0 con 0x7fd71815d040 2023-10-11T09:45:41.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.320+0000 7fd7177fe700 1 -- 172.21.15.78:0/2458101644 >> v2:172.21.15.133:3300/0 conn(0x7fd7180b3610 msgr2=0x7fd7180b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:41.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.320+0000 7fd7177fe700 1 --2- 172.21.15.78:0/2458101644 >> v2:172.21.15.133:3300/0 conn(0x7fd7180b3610 0x7fd7180b3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:41.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.320+0000 7fd7177fe700 1 -- 172.21.15.78:0/2458101644 >> v1:172.21.15.78:6789/0 conn(0x7fd71815d040 legacy=0x7fd7180b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:41.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.320+0000 7fd7177fe700 1 -- 172.21.15.78:0/2458101644 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7180b41e0 con 0x7fd7180b3240 2023-10-11T09:45:41.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.320+0000 7fd7177fe700 1 --2- 172.21.15.78:0/2458101644 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7180b3240 0x7fd7180d29d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd708009850 tx=0x7fd708009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fc17d2630f942db server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:41.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.320+0000 7fd7167fc700 1 -- 172.21.15.78:0/2458101644 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd708011830 con 0x7fd7180b3240 2023-10-11T09:45:41.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.324+0000 7fd7167fc700 1 -- 172.21.15.78:0/2458101644 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd708011990 con 0x7fd7180b3240 2023-10-11T09:45:41.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.324+0000 7fd7167fc700 1 -- 172.21.15.78:0/2458101644 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd708011d00 con 0x7fd7180b3240 2023-10-11T09:45:41.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.324+0000 7fd71c9b5700 1 -- 172.21.15.78:0/2458101644 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7180b3240 msgr2=0x7fd7180d29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:41.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.324+0000 7fd71c9b5700 1 --2- 172.21.15.78:0/2458101644 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7180b3240 0x7fd7180d29d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd708009850 tx=0x7fd708009b60 comp rx=0 tx=0).stop 2023-10-11T09:45:41.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.324+0000 7fd71c9b5700 1 -- 172.21.15.78:0/2458101644 shutdown_connections 2023-10-11T09:45:41.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.324+0000 7fd71c9b5700 1 --2- 172.21.15.78:0/2458101644 >> v2:172.21.15.133:3300/0 conn(0x7fd7180b3610 0x7fd7180b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:41.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.324+0000 7fd71c9b5700 1 --2- 172.21.15.78:0/2458101644 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7180b3240 0x7fd7180d29d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:41.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.324+0000 7fd71c9b5700 1 -- 172.21.15.78:0/2458101644 >> 172.21.15.78:0/2458101644 conn(0x7fd71809c260 msgr2=0x7fd7180d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:41.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.324+0000 7fd71c9b5700 1 -- 172.21.15.78:0/2458101644 shutdown_connections 2023-10-11T09:45:41.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.324+0000 7fd71c9b5700 1 -- 172.21.15.78:0/2458101644 wait complete. 2023-10-11T09:45:41.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.324+0000 7fd71c9b5700 1 Processor -- start 2023-10-11T09:45:41.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.328+0000 7fd71c9b5700 1 -- start start 2023-10-11T09:45:41.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.328+0000 7fd71c9b5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7180b3240 0x7fd71814d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:41.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.328+0000 7fd71c9b5700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7180b3610 0x7fd71815ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:41.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.328+0000 7fd71c9b5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd71815d040 0x7fd71815f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:41.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.328+0000 7fd71c9b5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71814d9f0 con 0x7fd7180b3240 2023-10-11T09:45:41.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.328+0000 7fd71c9b5700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7180aff60 con 0x7fd71815d040 2023-10-11T09:45:41.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.328+0000 7fd71c9b5700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd71809d660 con 0x7fd7180b3610 2023-10-11T09:45:41.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.328+0000 7fd7177fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7180b3240 0x7fd71814d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:41.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.328+0000 7fd7177fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7180b3240 0x7fd71814d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60712/0 (socket says 172.21.15.78:60712) 2023-10-11T09:45:41.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.328+0000 7fd7177fe700 1 -- 172.21.15.78:0/2510685276 learned_addr learned my addr 172.21.15.78:0/2510685276 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:41.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.328+0000 7fd716ffd700 1 --2- 172.21.15.78:0/2510685276 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7180b3610 0x7fd71815ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:41.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.328+0000 7fd717fff700 1 --2- 172.21.15.78:0/2510685276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd71815d040 0x7fd71815f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:41.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.328+0000 7fd7177fe700 1 -- 172.21.15.78:0/2510685276 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7180b3610 msgr2=0x7fd71815ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:41.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.328+0000 7fd7177fe700 1 --2- 172.21.15.78:0/2510685276 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7180b3610 0x7fd71815ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:41.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.328+0000 7fd7177fe700 1 -- 172.21.15.78:0/2510685276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd71815d040 msgr2=0x7fd71815f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:41.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.328+0000 7fd7177fe700 1 --2- 172.21.15.78:0/2510685276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd71815d040 0x7fd71815f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:41.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.328+0000 7fd7177fe700 1 -- 172.21.15.78:0/2510685276 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7080094b0 con 0x7fd7180b3240 2023-10-11T09:45:41.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.328+0000 7fd717fff700 1 --2- 172.21.15.78:0/2510685276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd71815d040 0x7fd71815f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:45:41.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.328+0000 7fd7177fe700 1 --2- 172.21.15.78:0/2510685276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7180b3240 0x7fd71814d4b0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd7080090b0 tx=0x7fd708004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:41.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.328+0000 7fd6fffff700 1 -- 172.21.15.78:0/2510685276 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd708004550 con 0x7fd7180b3240 2023-10-11T09:45:41.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.328+0000 7fd6fffff700 1 -- 172.21.15.78:0/2510685276 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd70801e070 con 0x7fd7180b3240 2023-10-11T09:45:41.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.328+0000 7fd71c9b5700 1 -- 172.21.15.78:0/2510685276 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7180a0910 con 0x7fd7180b3240 2023-10-11T09:45:41.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.328+0000 7fd6fffff700 1 -- 172.21.15.78:0/2510685276 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd708003900 con 0x7fd7180b3240 2023-10-11T09:45:41.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.328+0000 7fd71c9b5700 1 -- 172.21.15.78:0/2510685276 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7180a0630 con 0x7fd7180b3240 2023-10-11T09:45:41.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.332+0000 7fd71c9b5700 1 -- 172.21.15.78:0/2510685276 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd718065d40 con 0x7fd7180b3240 2023-10-11T09:45:41.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.332+0000 7fd6fffff700 1 -- 172.21.15.78:0/2510685276 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fd708003a60 con 0x7fd7180b3240 2023-10-11T09:45:41.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.336+0000 7fd6fffff700 1 --2- 172.21.15.78:0/2510685276 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd704065f00 0x7fd7040683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:41.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.336+0000 7fd6fffff700 1 -- 172.21.15.78:0/2510685276 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6466+0+0 (secure 0 0 0) 0x7fd708084910 con 0x7fd7180b3240 2023-10-11T09:45:41.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.336+0000 7fd716ffd700 1 --2- 172.21.15.78:0/2510685276 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd704065f00 0x7fd7040683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:41.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.336+0000 7fd716ffd700 1 --2- 172.21.15.78:0/2510685276 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd704065f00 0x7fd7040683b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd7181f51b0 tx=0x7fd70c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:41.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.340+0000 7fd6fffff700 1 -- 172.21.15.78:0/2510685276 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fd708051db0 con 0x7fd7180b3240 2023-10-11T09:45:41.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.512+0000 7fd71c9b5700 1 -- 172.21.15.78:0/2510685276 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd7180d2a90 con 0x7fd7180b3240 2023-10-11T09:45:41.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.512+0000 7fd6fffff700 1 -- 172.21.15.78:0/2510685276 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v64) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd7080200a0 con 0x7fd7180b3240 2023-10-11T09:45:41.520 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:45:41.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.512+0000 7fd71c9b5700 1 -- 172.21.15.78:0/2510685276 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd704065f00 msgr2=0x7fd7040683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:41.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.512+0000 7fd71c9b5700 1 --2- 172.21.15.78:0/2510685276 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd704065f00 0x7fd7040683b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd7181f51b0 tx=0x7fd70c008040 comp rx=0 tx=0).stop 2023-10-11T09:45:41.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.512+0000 7fd71c9b5700 1 -- 172.21.15.78:0/2510685276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7180b3240 msgr2=0x7fd71814d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:41.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.512+0000 7fd71c9b5700 1 --2- 172.21.15.78:0/2510685276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7180b3240 0x7fd71814d4b0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd7080090b0 tx=0x7fd708004770 comp rx=0 tx=0).stop 2023-10-11T09:45:41.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.516+0000 7fd71c9b5700 1 -- 172.21.15.78:0/2510685276 shutdown_connections 2023-10-11T09:45:41.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.516+0000 7fd71c9b5700 1 --2- 172.21.15.78:0/2510685276 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd704065f00 0x7fd7040683b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:41.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.516+0000 7fd71c9b5700 1 --2- 172.21.15.78:0/2510685276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd71815d040 0x7fd71815f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:41.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.516+0000 7fd71c9b5700 1 --2- 172.21.15.78:0/2510685276 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7180b3610 0x7fd71815ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:41.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.516+0000 7fd71c9b5700 1 --2- 172.21.15.78:0/2510685276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7180b3240 0x7fd71814d4b0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:41.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.516+0000 7fd71c9b5700 1 -- 172.21.15.78:0/2510685276 >> 172.21.15.78:0/2510685276 conn(0x7fd71809c260 msgr2=0x7fd7180b85c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:41.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.516+0000 7fd71c9b5700 1 -- 172.21.15.78:0/2510685276 shutdown_connections 2023-10-11T09:45:41.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.516+0000 7fd71c9b5700 1 -- 172.21.15.78:0/2510685276 wait complete. 2023-10-11T09:45:41.541 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T09:45:41.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.708+0000 7f42e5730700 1 Processor -- start 2023-10-11T09:45:41.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.708+0000 7f42e5730700 1 -- start start 2023-10-11T09:45:41.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.712+0000 7f42e5730700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f42e006d730 0x7f42e0165240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:41.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.712+0000 7f42e5730700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42e00b2d30 con 0x7f42e00b3a60 2023-10-11T09:45:41.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.712+0000 7f42e5730700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42e00cf360 con 0x7f42e015a6b0 2023-10-11T09:45:41.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.712+0000 7f42e5730700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42e00b2f80 con 0x7f42e006d730 2023-10-11T09:45:41.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.712+0000 7f42e4f2f700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f42e006d730 0x7f42e0165240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:41.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.712+0000 7f42df7fe700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f42e00b3a60 0x7f42e006d020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:44550/0 (socket says 172.21.15.78:44550) 2023-10-11T09:45:41.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.712+0000 7f42df7fe700 1 -- 172.21.15.78:0/3521362725 learned_addr learned my addr 172.21.15.78:0/3521362725 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:45:41.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.712+0000 7f42deffd700 1 -- 172.21.15.78:0/3521362725 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1616049322 0 0) 0x7f42e00b2d30 con 0x7f42e00b3a60 2023-10-11T09:45:41.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.712+0000 7f42e4f2f700 1 -- 172.21.15.78:0/3521362725 >> v1:172.21.15.133:6789/0 conn(0x7f42e015a6b0 legacy=0x7f42e00ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:41.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.712+0000 7f42e4f2f700 1 -- 172.21.15.78:0/3521362725 >> v1:172.21.15.78:6789/0 conn(0x7f42e00b3a60 legacy=0x7f42e006d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:41.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.712+0000 7f42e4f2f700 1 -- 172.21.15.78:0/3521362725 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e00b3260 con 0x7f42e006d730 2023-10-11T09:45:41.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.712+0000 7f42e4f2f700 1 --2- 172.21.15.78:0/3521362725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42e006d730 0x7f42e0165240 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f42d8009d90 tx=0x7f42d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a92927bcf7d40f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:41.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.712+0000 7f42deffd700 1 -- 172.21.15.78:0/3521362725 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d800e070 con 0x7f42e006d730 2023-10-11T09:45:41.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.712+0000 7f42deffd700 1 -- 172.21.15.78:0/3521362725 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d800bbd0 con 0x7f42e006d730 2023-10-11T09:45:41.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.712+0000 7f42deffd700 1 -- 172.21.15.78:0/3521362725 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d8012430 con 0x7f42e006d730 2023-10-11T09:45:41.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.712+0000 7f42e5730700 1 -- 172.21.15.78:0/3521362725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42e006d730 msgr2=0x7f42e0165240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:41.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.712+0000 7f42e5730700 1 --2- 172.21.15.78:0/3521362725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42e006d730 0x7f42e0165240 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f42d8009d90 tx=0x7f42d800b330 comp rx=0 tx=0).stop 2023-10-11T09:45:41.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.712+0000 7f42e5730700 1 -- 172.21.15.78:0/3521362725 shutdown_connections 2023-10-11T09:45:41.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.712+0000 7f42e5730700 1 --2- 172.21.15.78:0/3521362725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42e006d730 0x7f42e0165240 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:41.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.712+0000 7f42e5730700 1 -- 172.21.15.78:0/3521362725 >> 172.21.15.78:0/3521362725 conn(0x7f42e006a060 msgr2=0x7f42e006a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:41.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.712+0000 7f42e5730700 1 -- 172.21.15.78:0/3521362725 shutdown_connections 2023-10-11T09:45:41.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.712+0000 7f42e5730700 1 -- 172.21.15.78:0/3521362725 wait complete. 2023-10-11T09:45:41.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.716+0000 7f42e5730700 1 Processor -- start 2023-10-11T09:45:41.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.716+0000 7f42e5730700 1 -- start start 2023-10-11T09:45:41.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.720+0000 7f42e5730700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42e006d730 0x7f42e0159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:41.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.720+0000 7f42e5730700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42e00b3a60 0x7f42e00afef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:41.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.720+0000 7f42e5730700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42e015a6b0 0x7f42e01475d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:41.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.720+0000 7f42e5730700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e00714c0 con 0x7f42e00b3a60 2023-10-11T09:45:41.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.720+0000 7f42e5730700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e00703a0 con 0x7f42e006d730 2023-10-11T09:45:41.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.720+0000 7f42e5730700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42e0070680 con 0x7f42e015a6b0 2023-10-11T09:45:41.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.720+0000 7f42df7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42e00b3a60 0x7f42e00afef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:41.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.720+0000 7f42df7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42e00b3a60 0x7f42e00afef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60726/0 (socket says 172.21.15.78:60726) 2023-10-11T09:45:41.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.720+0000 7f42df7fe700 1 -- 172.21.15.78:0/691363733 learned_addr learned my addr 172.21.15.78:0/691363733 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:41.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.720+0000 7f42e4f2f700 1 --2- 172.21.15.78:0/691363733 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42e015a6b0 0x7f42e01475d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:41.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.720+0000 7f42dffff700 1 --2- 172.21.15.78:0/691363733 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42e006d730 0x7f42e0159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:41.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.720+0000 7f42df7fe700 1 -- 172.21.15.78:0/691363733 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42e015a6b0 msgr2=0x7f42e01475d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:41.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.720+0000 7f42df7fe700 1 --2- 172.21.15.78:0/691363733 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42e015a6b0 0x7f42e01475d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:41.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.720+0000 7f42df7fe700 1 -- 172.21.15.78:0/691363733 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42e006d730 msgr2=0x7f42e0159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:41.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.720+0000 7f42df7fe700 1 --2- 172.21.15.78:0/691363733 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42e006d730 0x7f42e0159700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:41.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.720+0000 7f42df7fe700 1 -- 172.21.15.78:0/691363733 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d8009a70 con 0x7f42e00b3a60 2023-10-11T09:45:41.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.720+0000 7f42df7fe700 1 --2- 172.21.15.78:0/691363733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42e00b3a60 0x7f42e00afef0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f42d400c450 tx=0x7f42d400c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:41.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.720+0000 7f42dcff9700 1 -- 172.21.15.78:0/691363733 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d4004ca0 con 0x7f42e00b3a60 2023-10-11T09:45:41.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.720+0000 7f42dcff9700 1 -- 172.21.15.78:0/691363733 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d4004e00 con 0x7f42e00b3a60 2023-10-11T09:45:41.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.720+0000 7f42e5730700 1 -- 172.21.15.78:0/691363733 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42e0070960 con 0x7f42e00b3a60 2023-10-11T09:45:41.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.720+0000 7f42dcff9700 1 -- 172.21.15.78:0/691363733 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d4007310 con 0x7f42e00b3a60 2023-10-11T09:45:41.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.720+0000 7f42e5730700 1 -- 172.21.15.78:0/691363733 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42e014c430 con 0x7f42e00b3a60 2023-10-11T09:45:41.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.724+0000 7f42e5730700 1 -- 172.21.15.78:0/691363733 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42e0065d40 con 0x7f42e00b3a60 2023-10-11T09:45:41.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.724+0000 7f42dcff9700 1 -- 172.21.15.78:0/691363733 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f42d4007470 con 0x7f42e00b3a60 2023-10-11T09:45:41.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.724+0000 7f42dcff9700 1 --2- 172.21.15.78:0/691363733 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f42cc065f00 0x7f42cc0683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:41.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.724+0000 7f42dcff9700 1 -- 172.21.15.78:0/691363733 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f42d40833e0 con 0x7f42e00b3a60 2023-10-11T09:45:41.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.724+0000 7f42dffff700 1 --2- 172.21.15.78:0/691363733 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f42cc065f00 0x7f42cc0683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:41.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.728+0000 7f42dffff700 1 --2- 172.21.15.78:0/691363733 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f42cc065f00 0x7f42cc0683b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f42d0000f60 tx=0x7f42d0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:41.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.732+0000 7f42dcff9700 1 -- 172.21.15.78:0/691363733 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f42d40543b0 con 0x7f42e00b3a60 2023-10-11T09:45:41.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.912+0000 7f42e5730700 1 -- 172.21.15.78:0/691363733 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f42e00cb6a0 con 0x7f42e00b3a60 2023-10-11T09:45:41.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.916+0000 7f42dcff9700 1 -- 172.21.15.78:0/691363733 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v64) v1 ==== 104+0+11 (secure 0 0 0) 0x7f42d400ce60 con 0x7f42e00b3a60 2023-10-11T09:45:41.923 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:45:41.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.920+0000 7f42e5730700 1 -- 172.21.15.78:0/691363733 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f42cc065f00 msgr2=0x7f42cc0683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:41.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.920+0000 7f42e5730700 1 --2- 172.21.15.78:0/691363733 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f42cc065f00 0x7f42cc0683b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f42d0000f60 tx=0x7f42d0009040 comp rx=0 tx=0).stop 2023-10-11T09:45:41.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.920+0000 7f42e5730700 1 -- 172.21.15.78:0/691363733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42e00b3a60 msgr2=0x7f42e00afef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:41.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.920+0000 7f42e5730700 1 --2- 172.21.15.78:0/691363733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42e00b3a60 0x7f42e00afef0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f42d400c450 tx=0x7f42d400c760 comp rx=0 tx=0).stop 2023-10-11T09:45:41.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.920+0000 7f42e5730700 1 -- 172.21.15.78:0/691363733 shutdown_connections 2023-10-11T09:45:41.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.920+0000 7f42e5730700 1 --2- 172.21.15.78:0/691363733 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f42cc065f00 0x7f42cc0683b0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:41.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.920+0000 7f42e5730700 1 --2- 172.21.15.78:0/691363733 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42e015a6b0 0x7f42e01475d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:41.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.920+0000 7f42e5730700 1 --2- 172.21.15.78:0/691363733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42e00b3a60 0x7f42e00afef0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:41.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.920+0000 7f42e5730700 1 --2- 172.21.15.78:0/691363733 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42e006d730 0x7f42e0159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:41.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.920+0000 7f42e5730700 1 -- 172.21.15.78:0/691363733 >> 172.21.15.78:0/691363733 conn(0x7f42e006a060 msgr2=0x7f42e00cc620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:41.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.920+0000 7f42e5730700 1 -- 172.21.15.78:0/691363733 shutdown_connections 2023-10-11T09:45:41.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:41.920+0000 7f42e5730700 1 -- 172.21.15.78:0/691363733 wait complete. 2023-10-11T09:45:41.950 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T09:45:42.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.112+0000 7efe7f498700 1 Processor -- start 2023-10-11T09:45:42.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.112+0000 7efe7f498700 1 -- start start 2023-10-11T09:45:42.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.116+0000 7efe7f498700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe780b3610 0x7efe780d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:42.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.116+0000 7efe7f498700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efe780b3a90 0x7efe780b85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:42.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.116+0000 7efe7f498700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe780b47f0 con 0x7efe780b3240 2023-10-11T09:45:42.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.116+0000 7efe7f498700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe780b8130 con 0x7efe780b3a90 2023-10-11T09:45:42.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.116+0000 7efe7f498700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe780b3f10 con 0x7efe780b3610 2023-10-11T09:45:42.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.116+0000 7efe7dc95700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe780b3610 0x7efe780d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:42.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.116+0000 7efe7ec97700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efe780b3a90 0x7efe780b85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:42.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.116+0000 7efe7ec97700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efe780b3a90 0x7efe780b85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:44668/0 (socket says 172.21.15.78:44668) 2023-10-11T09:45:42.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.116+0000 7efe7ec97700 1 -- 172.21.15.78:0/3115229550 learned_addr learned my addr 172.21.15.78:0/3115229550 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:42.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.116+0000 7efe7ec97700 1 -- 172.21.15.78:0/3115229550 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe780b3610 msgr2=0x7efe780d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:42.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.116+0000 7efe7ec97700 1 --2- 172.21.15.78:0/3115229550 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe780b3610 0x7efe780d29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:42.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.116+0000 7efe7ec97700 1 -- 172.21.15.78:0/3115229550 >> v1:172.21.15.133:6789/0 conn(0x7efe780b3240 legacy=0x7efe7815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:42.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.116+0000 7efe7ec97700 1 -- 172.21.15.78:0/3115229550 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe780b41e0 con 0x7efe780b3a90 2023-10-11T09:45:42.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.116+0000 7efe7ec97700 1 --2- 172.21.15.78:0/3115229550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe780b3a90 0x7efe780b85c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7efe70009e50 tx=0x7efe7000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8281b4ae18f595b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:42.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.120+0000 7efe7d494700 1 -- 172.21.15.78:0/3115229550 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7000e070 con 0x7efe780b3a90 2023-10-11T09:45:42.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.120+0000 7efe7d494700 1 -- 172.21.15.78:0/3115229550 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe7000b910 con 0x7efe780b3a90 2023-10-11T09:45:42.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.120+0000 7efe7d494700 1 -- 172.21.15.78:0/3115229550 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7000bcc0 con 0x7efe780b3a90 2023-10-11T09:45:42.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.120+0000 7efe7f498700 1 -- 172.21.15.78:0/3115229550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe780b3a90 msgr2=0x7efe780b85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:42.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.120+0000 7efe7f498700 1 --2- 172.21.15.78:0/3115229550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe780b3a90 0x7efe780b85c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7efe70009e50 tx=0x7efe7000b330 comp rx=0 tx=0).stop 2023-10-11T09:45:42.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.120+0000 7efe7f498700 1 -- 172.21.15.78:0/3115229550 shutdown_connections 2023-10-11T09:45:42.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.120+0000 7efe7f498700 1 --2- 172.21.15.78:0/3115229550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe780b3a90 0x7efe780b85c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:42.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.120+0000 7efe7f498700 1 --2- 172.21.15.78:0/3115229550 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe780b3610 0x7efe780d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:42.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.120+0000 7efe7f498700 1 -- 172.21.15.78:0/3115229550 >> 172.21.15.78:0/3115229550 conn(0x7efe7809c260 msgr2=0x7efe780d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:42.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.120+0000 7efe7f498700 1 -- 172.21.15.78:0/3115229550 shutdown_connections 2023-10-11T09:45:42.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.120+0000 7efe7f498700 1 -- 172.21.15.78:0/3115229550 wait complete. 2023-10-11T09:45:42.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.120+0000 7efe7f498700 1 Processor -- start 2023-10-11T09:45:42.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.124+0000 7efe7f498700 1 -- start start 2023-10-11T09:45:42.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.124+0000 7efe7f498700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe780b3240 0x7efe780c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:42.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.124+0000 7efe7f498700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe780b3610 0x7efe780c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:42.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.124+0000 7efe7f498700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe780b3a90 0x7efe7814c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:42.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.124+0000 7efe7f498700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7809e210 con 0x7efe780b3240 2023-10-11T09:45:42.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.124+0000 7efe7f498700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7815d1e0 con 0x7efe780b3a90 2023-10-11T09:45:42.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.124+0000 7efe7f498700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe7815d4c0 con 0x7efe780b3610 2023-10-11T09:45:42.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.124+0000 7efe7e496700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe780b3240 0x7efe780c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:42.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.124+0000 7efe7e496700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe780b3240 0x7efe780c9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60736/0 (socket says 172.21.15.78:60736) 2023-10-11T09:45:42.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.124+0000 7efe7e496700 1 -- 172.21.15.78:0/1132969063 learned_addr learned my addr 172.21.15.78:0/1132969063 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:42.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.124+0000 7efe7ec97700 1 --2- 172.21.15.78:0/1132969063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe780b3a90 0x7efe7814c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:42.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.124+0000 7efe7dc95700 1 --2- 172.21.15.78:0/1132969063 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe780b3610 0x7efe780c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:42.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.124+0000 7efe7e496700 1 -- 172.21.15.78:0/1132969063 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe780b3610 msgr2=0x7efe780c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:42.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.124+0000 7efe7e496700 1 --2- 172.21.15.78:0/1132969063 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe780b3610 0x7efe780c95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:42.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.124+0000 7efe7e496700 1 -- 172.21.15.78:0/1132969063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe780b3a90 msgr2=0x7efe7814c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:42.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.124+0000 7efe7e496700 1 --2- 172.21.15.78:0/1132969063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe780b3a90 0x7efe7814c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:42.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.124+0000 7efe7e496700 1 -- 172.21.15.78:0/1132969063 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe70009b30 con 0x7efe780b3240 2023-10-11T09:45:42.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.124+0000 7efe7e496700 1 --2- 172.21.15.78:0/1132969063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe780b3240 0x7efe780c9070 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7efe7400c410 tx=0x7efe7400c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:42.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.124+0000 7efe6effd700 1 -- 172.21.15.78:0/1132969063 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe740052d0 con 0x7efe780b3240 2023-10-11T09:45:42.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.124+0000 7efe6effd700 1 -- 172.21.15.78:0/1132969063 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe74005430 con 0x7efe780b3240 2023-10-11T09:45:42.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.124+0000 7efe7f498700 1 -- 172.21.15.78:0/1132969063 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe7815d7a0 con 0x7efe780b3240 2023-10-11T09:45:42.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.124+0000 7efe6effd700 1 -- 172.21.15.78:0/1132969063 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe740055a0 con 0x7efe780b3240 2023-10-11T09:45:42.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.124+0000 7efe7f498700 1 -- 172.21.15.78:0/1132969063 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe7815dd60 con 0x7efe780b3240 2023-10-11T09:45:42.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.128+0000 7efe7f498700 1 -- 172.21.15.78:0/1132969063 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe78065d40 con 0x7efe780b3240 2023-10-11T09:45:42.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.128+0000 7efe6effd700 1 -- 172.21.15.78:0/1132969063 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7efe740036a0 con 0x7efe780b3240 2023-10-11T09:45:42.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.128+0000 7efe6effd700 1 --2- 172.21.15.78:0/1132969063 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efe5c065ed0 0x7efe5c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:42.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.132+0000 7efe6effd700 1 -- 172.21.15.78:0/1132969063 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6466+0+0 (secure 0 0 0) 0x7efe74082960 con 0x7efe780b3240 2023-10-11T09:45:42.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.132+0000 7efe7dc95700 1 --2- 172.21.15.78:0/1132969063 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efe5c065ed0 0x7efe5c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:42.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.132+0000 7efe7dc95700 1 --2- 172.21.15.78:0/1132969063 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efe5c065ed0 0x7efe5c068380 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7efe78072570 tx=0x7efe68008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:42.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.136+0000 7efe6effd700 1 -- 172.21.15.78:0/1132969063 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7efe7404fea0 con 0x7efe780b3240 2023-10-11T09:45:42.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.304+0000 7efe7f498700 1 -- 172.21.15.78:0/1132969063 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efe780b47f0 con 0x7efe780b3240 2023-10-11T09:45:42.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.308+0000 7efe6effd700 1 -- 172.21.15.78:0/1132969063 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v64) v1 ==== 100+0+11 (secure 0 0 0) 0x7efe740534c0 con 0x7efe780b3240 2023-10-11T09:45:42.315 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2023-10-11T09:45:42.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.308+0000 7efe7f498700 1 -- 172.21.15.78:0/1132969063 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efe5c065ed0 msgr2=0x7efe5c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:42.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.308+0000 7efe7f498700 1 --2- 172.21.15.78:0/1132969063 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efe5c065ed0 0x7efe5c068380 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7efe78072570 tx=0x7efe68008040 comp rx=0 tx=0).stop 2023-10-11T09:45:42.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.308+0000 7efe7f498700 1 -- 172.21.15.78:0/1132969063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe780b3240 msgr2=0x7efe780c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:42.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.308+0000 7efe7f498700 1 --2- 172.21.15.78:0/1132969063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe780b3240 0x7efe780c9070 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7efe7400c410 tx=0x7efe7400c720 comp rx=0 tx=0).stop 2023-10-11T09:45:42.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.308+0000 7efe7f498700 1 -- 172.21.15.78:0/1132969063 shutdown_connections 2023-10-11T09:45:42.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.308+0000 7efe7f498700 1 --2- 172.21.15.78:0/1132969063 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efe5c065ed0 0x7efe5c068380 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:42.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.308+0000 7efe7f498700 1 --2- 172.21.15.78:0/1132969063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe780b3a90 0x7efe7814c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:42.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.308+0000 7efe7f498700 1 --2- 172.21.15.78:0/1132969063 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe780b3610 0x7efe780c95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:42.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.308+0000 7efe7f498700 1 --2- 172.21.15.78:0/1132969063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe780b3240 0x7efe780c9070 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:42.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.308+0000 7efe7f498700 1 -- 172.21.15.78:0/1132969063 >> 172.21.15.78:0/1132969063 conn(0x7efe7809c260 msgr2=0x7efe780b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:42.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.308+0000 7efe7f498700 1 -- 172.21.15.78:0/1132969063 shutdown_connections 2023-10-11T09:45:42.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.308+0000 7efe7f498700 1 -- 172.21.15.78:0/1132969063 wait complete. 2023-10-11T09:45:42.337 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-10-11T09:45:42.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.504+0000 7fc2b9700700 1 Processor -- start 2023-10-11T09:45:42.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.504+0000 7fc2b9700700 1 -- start start 2023-10-11T09:45:42.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.504+0000 7fc2b9700700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fc2b40b3240 0x7fc2b40d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:42.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.504+0000 7fc2b9700700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2b40b47f0 con 0x7fc2b40b3610 2023-10-11T09:45:42.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.504+0000 7fc2b9700700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2b40b8130 con 0x7fc2b40b3a90 2023-10-11T09:45:42.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.504+0000 7fc2b9700700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2b40b41e0 con 0x7fc2b40b3240 2023-10-11T09:45:42.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.508+0000 7fc2b3fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fc2b40b3240 0x7fc2b40d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:42.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.508+0000 7fc2b37fe700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fc2b40b3610 0x7fc2b415d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:44564/0 (socket says 172.21.15.78:44564) 2023-10-11T09:45:42.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.508+0000 7fc2b37fe700 1 -- 172.21.15.78:0/4263889391 learned_addr learned my addr 172.21.15.78:0/4263889391 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:45:42.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.508+0000 7fc2b3fff700 1 -- 172.21.15.78:0/4263889391 >> v1:172.21.15.133:6789/0 conn(0x7fc2b40b3a90 legacy=0x7fc2b40b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:42.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.508+0000 7fc2b3fff700 1 -- 172.21.15.78:0/4263889391 >> v1:172.21.15.78:6789/0 conn(0x7fc2b40b3610 legacy=0x7fc2b415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:42.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.508+0000 7fc2b3fff700 1 -- 172.21.15.78:0/4263889391 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b40b7800 con 0x7fc2b40b3240 2023-10-11T09:45:42.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.508+0000 7fc2b3fff700 1 --2- 172.21.15.78:0/4263889391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b40b3240 0x7fc2b40d29d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009850 tx=0x7fc2a4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e38d2176ff07a99 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:42.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.508+0000 7fc2b2ffd700 1 -- 172.21.15.78:0/4263889391 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4011830 con 0x7fc2b40b3240 2023-10-11T09:45:42.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.508+0000 7fc2b2ffd700 1 -- 172.21.15.78:0/4263889391 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a4011990 con 0x7fc2b40b3240 2023-10-11T09:45:42.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.508+0000 7fc2b2ffd700 1 -- 172.21.15.78:0/4263889391 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4011d20 con 0x7fc2b40b3240 2023-10-11T09:45:42.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.508+0000 7fc2b9700700 1 -- 172.21.15.78:0/4263889391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b40b3240 msgr2=0x7fc2b40d29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:42.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.508+0000 7fc2b9700700 1 --2- 172.21.15.78:0/4263889391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b40b3240 0x7fc2b40d29d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009850 tx=0x7fc2a4009b60 comp rx=0 tx=0).stop 2023-10-11T09:45:42.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.508+0000 7fc2b9700700 1 -- 172.21.15.78:0/4263889391 shutdown_connections 2023-10-11T09:45:42.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.508+0000 7fc2b9700700 1 --2- 172.21.15.78:0/4263889391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b40b3240 0x7fc2b40d29d0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:42.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.508+0000 7fc2b9700700 1 -- 172.21.15.78:0/4263889391 >> 172.21.15.78:0/4263889391 conn(0x7fc2b409c260 msgr2=0x7fc2b40d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:42.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.508+0000 7fc2b9700700 1 -- 172.21.15.78:0/4263889391 shutdown_connections 2023-10-11T09:45:42.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.508+0000 7fc2b9700700 1 -- 172.21.15.78:0/4263889391 wait complete. 2023-10-11T09:45:42.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.512+0000 7fc2b9700700 1 Processor -- start 2023-10-11T09:45:42.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.512+0000 7fc2b9700700 1 -- start start 2023-10-11T09:45:42.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.516+0000 7fc2b9700700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2b40b3240 0x7fc2b40c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:42.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.516+0000 7fc2b9700700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2b40b3610 0x7fc2b41f1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:42.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.516+0000 7fc2b9700700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b40b3a90 0x7fc2b40c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:42.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.516+0000 7fc2b9700700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b414c950 con 0x7fc2b40b3a90 2023-10-11T09:45:42.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.516+0000 7fc2b9700700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b40b0930 con 0x7fc2b40b3240 2023-10-11T09:45:42.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.516+0000 7fc2b9700700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b40b0c10 con 0x7fc2b40b3610 2023-10-11T09:45:42.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.516+0000 7fc2b8eff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b40b3a90 0x7fc2b40c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:42.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.516+0000 7fc2b8eff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b40b3a90 0x7fc2b40c6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60750/0 (socket says 172.21.15.78:60750) 2023-10-11T09:45:42.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.516+0000 7fc2b8eff700 1 -- 172.21.15.78:0/2602725990 learned_addr learned my addr 172.21.15.78:0/2602725990 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:42.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.516+0000 7fc2b3fff700 1 --2- 172.21.15.78:0/2602725990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2b40b3240 0x7fc2b40c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:42.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.516+0000 7fc2b37fe700 1 --2- 172.21.15.78:0/2602725990 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2b40b3610 0x7fc2b41f1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:42.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.516+0000 7fc2b8eff700 1 -- 172.21.15.78:0/2602725990 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2b40b3610 msgr2=0x7fc2b41f1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:42.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.516+0000 7fc2b8eff700 1 --2- 172.21.15.78:0/2602725990 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2b40b3610 0x7fc2b41f1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:42.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.516+0000 7fc2b8eff700 1 -- 172.21.15.78:0/2602725990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2b40b3240 msgr2=0x7fc2b40c93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:42.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.516+0000 7fc2b8eff700 1 --2- 172.21.15.78:0/2602725990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2b40b3240 0x7fc2b40c93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:42.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.516+0000 7fc2b8eff700 1 -- 172.21.15.78:0/2602725990 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a40094b0 con 0x7fc2b40b3a90 2023-10-11T09:45:42.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.516+0000 7fc2b8eff700 1 --2- 172.21.15.78:0/2602725990 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b40b3a90 0x7fc2b40c6ac0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac00cb50 tx=0x7fc2ac00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:42.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.516+0000 7fc2b0ff9700 1 -- 172.21.15.78:0/2602725990 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac003c20 con 0x7fc2b40b3a90 2023-10-11T09:45:42.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.516+0000 7fc2b0ff9700 1 -- 172.21.15.78:0/2602725990 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ac003d80 con 0x7fc2b40b3a90 2023-10-11T09:45:42.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.516+0000 7fc2b0ff9700 1 -- 172.21.15.78:0/2602725990 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac00aac0 con 0x7fc2b40b3a90 2023-10-11T09:45:42.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.516+0000 7fc2b9700700 1 -- 172.21.15.78:0/2602725990 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2b40b0ef0 con 0x7fc2b40b3a90 2023-10-11T09:45:42.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.516+0000 7fc2b9700700 1 -- 172.21.15.78:0/2602725990 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b40c9920 con 0x7fc2b40b3a90 2023-10-11T09:45:42.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.520+0000 7fc2b9700700 1 -- 172.21.15.78:0/2602725990 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2b40b1410 con 0x7fc2b40b3a90 2023-10-11T09:45:42.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.520+0000 7fc2b0ff9700 1 -- 172.21.15.78:0/2602725990 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fc2ac00ac20 con 0x7fc2b40b3a90 2023-10-11T09:45:42.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.520+0000 7fc2b0ff9700 1 --2- 172.21.15.78:0/2602725990 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc2a0065ed0 0x7fc2a0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:42.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.520+0000 7fc2b0ff9700 1 -- 172.21.15.78:0/2602725990 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6466+0+0 (secure 0 0 0) 0x7fc2ac0839b0 con 0x7fc2b40b3a90 2023-10-11T09:45:42.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.520+0000 7fc2b3fff700 1 --2- 172.21.15.78:0/2602725990 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc2a0065ed0 0x7fc2a0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:42.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.524+0000 7fc2b3fff700 1 --2- 172.21.15.78:0/2602725990 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc2a0065ed0 0x7fc2a0068380 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4014040 tx=0x7fc2a401c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:42.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.528+0000 7fc2b0ff9700 1 -- 172.21.15.78:0/2602725990 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fc2b40b1410 con 0x7fc2b40b3a90 2023-10-11T09:45:42.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.700+0000 7fc2b9700700 1 -- 172.21.15.78:0/2602725990 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7fc2b40b1410 con 0x7fc2b40b3a90 2023-10-11T09:45:42.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.700+0000 7fc2b0ff9700 1 -- 172.21.15.78:0/2602725990 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v64) v1 ==== 106+0+11 (secure 0 0 0) 0x7fc2b40b1410 con 0x7fc2b40b3a90 2023-10-11T09:45:42.708 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:45:42.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.704+0000 7fc2b9700700 1 -- 172.21.15.78:0/2602725990 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc2a0065ed0 msgr2=0x7fc2a0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:42.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.704+0000 7fc2b9700700 1 --2- 172.21.15.78:0/2602725990 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc2a0065ed0 0x7fc2a0068380 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4014040 tx=0x7fc2a401c040 comp rx=0 tx=0).stop 2023-10-11T09:45:42.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.704+0000 7fc2b9700700 1 -- 172.21.15.78:0/2602725990 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b40b3a90 msgr2=0x7fc2b40c6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:42.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.704+0000 7fc2b9700700 1 --2- 172.21.15.78:0/2602725990 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b40b3a90 0x7fc2b40c6ac0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac00cb50 tx=0x7fc2ac00ce60 comp rx=0 tx=0).stop 2023-10-11T09:45:42.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.704+0000 7fc2b9700700 1 -- 172.21.15.78:0/2602725990 shutdown_connections 2023-10-11T09:45:42.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.704+0000 7fc2b9700700 1 --2- 172.21.15.78:0/2602725990 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc2a0065ed0 0x7fc2a0068380 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:42.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.704+0000 7fc2b9700700 1 --2- 172.21.15.78:0/2602725990 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2b40b3a90 0x7fc2b40c6ac0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:42.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.704+0000 7fc2b9700700 1 --2- 172.21.15.78:0/2602725990 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2b40b3610 0x7fc2b41f1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:42.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.704+0000 7fc2b9700700 1 --2- 172.21.15.78:0/2602725990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2b40b3240 0x7fc2b40c93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:42.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.704+0000 7fc2b9700700 1 -- 172.21.15.78:0/2602725990 >> 172.21.15.78:0/2602725990 conn(0x7fc2b409c260 msgr2=0x7fc2b40afb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:42.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.704+0000 7fc2b9700700 1 -- 172.21.15.78:0/2602725990 shutdown_connections 2023-10-11T09:45:42.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.704+0000 7fc2b9700700 1 -- 172.21.15.78:0/2602725990 wait complete. 2023-10-11T09:45:42.734 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T09:45:42.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.896+0000 7f4cc2384700 1 Processor -- start 2023-10-11T09:45:42.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.896+0000 7f4cc2384700 1 -- start start 2023-10-11T09:45:42.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.896+0000 7f4cc2384700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4cbc156ce0 0x7f4cbc160b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:42.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.896+0000 7f4cc2384700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4cbc0d1660 0x7f4cbc0d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:42.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.896+0000 7f4cc2384700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cbc15dc10 con 0x7f4cbc0d28e0 2023-10-11T09:45:42.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.896+0000 7f4cc2384700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cbc160380 con 0x7f4cbc156ce0 2023-10-11T09:45:42.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.896+0000 7f4cc2384700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cbc15d930 con 0x7f4cbc0d1660 2023-10-11T09:45:42.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.896+0000 7f4cc0b81700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4cbc0d1660 0x7f4cbc0d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:42.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.896+0000 7f4cc1382700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4cbc156ce0 0x7f4cbc160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:42.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.896+0000 7f4cc0b81700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4cbc0d1660 0x7f4cbc0d0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:52120/0 (socket says 172.21.15.78:52120) 2023-10-11T09:45:42.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.896+0000 7f4cc0b81700 1 -- 172.21.15.78:0/848675257 learned_addr learned my addr 172.21.15.78:0/848675257 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:42.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.900+0000 7f4cc1382700 1 -- 172.21.15.78:0/848675257 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4cbc0d1660 msgr2=0x7f4cbc0d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:42.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.900+0000 7f4cc1382700 1 --2- 172.21.15.78:0/848675257 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4cbc0d1660 0x7f4cbc0d0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:42.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.900+0000 7f4cc1382700 1 -- 172.21.15.78:0/848675257 >> v1:172.21.15.133:6789/0 conn(0x7f4cbc0d28e0 legacy=0x7f4cbc0cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:42.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.900+0000 7f4cc1382700 1 -- 172.21.15.78:0/848675257 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cbc160620 con 0x7f4cbc156ce0 2023-10-11T09:45:42.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.900+0000 7f4cc0b81700 1 --2- 172.21.15.78:0/848675257 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4cbc0d1660 0x7f4cbc0d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:45:42.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.900+0000 7f4cc1382700 1 --2- 172.21.15.78:0/848675257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cbc156ce0 0x7f4cbc160b40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8009850 tx=0x7f4cb8009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b809ab2fa567d5d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:42.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.900+0000 7f4cb3fff700 1 -- 172.21.15.78:0/848675257 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb8011830 con 0x7f4cbc156ce0 2023-10-11T09:45:42.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.900+0000 7f4cb3fff700 1 -- 172.21.15.78:0/848675257 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb8011990 con 0x7f4cbc156ce0 2023-10-11T09:45:42.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.900+0000 7f4cb3fff700 1 -- 172.21.15.78:0/848675257 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb8011d40 con 0x7f4cbc156ce0 2023-10-11T09:45:42.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.900+0000 7f4cc2384700 1 -- 172.21.15.78:0/848675257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cbc156ce0 msgr2=0x7f4cbc160b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:42.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.900+0000 7f4cc2384700 1 --2- 172.21.15.78:0/848675257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cbc156ce0 0x7f4cbc160b40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8009850 tx=0x7f4cb8009b60 comp rx=0 tx=0).stop 2023-10-11T09:45:42.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.900+0000 7f4cc2384700 1 -- 172.21.15.78:0/848675257 shutdown_connections 2023-10-11T09:45:42.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.900+0000 7f4cc2384700 1 --2- 172.21.15.78:0/848675257 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4cbc0d1660 0x7f4cbc0d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:42.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.900+0000 7f4cc2384700 1 --2- 172.21.15.78:0/848675257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cbc156ce0 0x7f4cbc160b40 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:42.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.900+0000 7f4cc2384700 1 -- 172.21.15.78:0/848675257 >> 172.21.15.78:0/848675257 conn(0x7f4cbc06afb0 msgr2=0x7f4cbc06b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:42.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.900+0000 7f4cc2384700 1 -- 172.21.15.78:0/848675257 shutdown_connections 2023-10-11T09:45:42.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.900+0000 7f4cc2384700 1 -- 172.21.15.78:0/848675257 wait complete. 2023-10-11T09:45:42.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.904+0000 7f4cc2384700 1 Processor -- start 2023-10-11T09:45:42.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.904+0000 7f4cc2384700 1 -- start start 2023-10-11T09:45:42.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.908+0000 7f4cc2384700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cbc0d1660 0x7f4cbc0cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:42.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.908+0000 7f4cc2384700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4cbc0d28e0 0x7f4cbc0cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:42.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.908+0000 7f4cc2384700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cbc156ce0 0x7f4cbc0c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:42.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.908+0000 7f4cc2384700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cbc0cb930 con 0x7f4cbc156ce0 2023-10-11T09:45:42.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.908+0000 7f4cc2384700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cbc0cbc10 con 0x7f4cbc0d1660 2023-10-11T09:45:42.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.908+0000 7f4cc2384700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cbc0cbef0 con 0x7f4cbc0d28e0 2023-10-11T09:45:42.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.908+0000 7f4cc1b83700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cbc156ce0 0x7f4cbc0c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:42.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.908+0000 7f4cc1382700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cbc0d1660 0x7f4cbc0cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:42.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.908+0000 7f4cc1b83700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cbc156ce0 0x7f4cbc0c7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60754/0 (socket says 172.21.15.78:60754) 2023-10-11T09:45:42.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.908+0000 7f4cc1b83700 1 -- 172.21.15.78:0/1059196896 learned_addr learned my addr 172.21.15.78:0/1059196896 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:42.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.908+0000 7f4cc0b81700 1 --2- 172.21.15.78:0/1059196896 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4cbc0d28e0 0x7f4cbc0cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:42.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.908+0000 7f4cc1b83700 1 -- 172.21.15.78:0/1059196896 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4cbc0d28e0 msgr2=0x7f4cbc0cc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:42.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.908+0000 7f4cc1b83700 1 --2- 172.21.15.78:0/1059196896 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4cbc0d28e0 0x7f4cbc0cc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:42.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.908+0000 7f4cc1b83700 1 -- 172.21.15.78:0/1059196896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cbc0d1660 msgr2=0x7f4cbc0cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:42.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.908+0000 7f4cc1b83700 1 --2- 172.21.15.78:0/1059196896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cbc0d1660 0x7f4cbc0cb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:42.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.908+0000 7f4cc1b83700 1 -- 172.21.15.78:0/1059196896 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cac009570 con 0x7f4cbc156ce0 2023-10-11T09:45:42.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.908+0000 7f4cc1b83700 1 --2- 172.21.15.78:0/1059196896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cbc156ce0 0x7f4cbc0c7cb0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f4cb400cc50 tx=0x7f4cb400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:42.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.908+0000 7f4cb1ffb700 1 -- 172.21.15.78:0/1059196896 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4003c20 con 0x7f4cbc156ce0 2023-10-11T09:45:42.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.908+0000 7f4cb1ffb700 1 -- 172.21.15.78:0/1059196896 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb4003d80 con 0x7f4cbc156ce0 2023-10-11T09:45:42.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.908+0000 7f4cc2384700 1 -- 172.21.15.78:0/1059196896 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cb80094b0 con 0x7f4cbc156ce0 2023-10-11T09:45:42.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.908+0000 7f4cb1ffb700 1 -- 172.21.15.78:0/1059196896 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb400aab0 con 0x7f4cbc156ce0 2023-10-11T09:45:42.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.908+0000 7f4cc2384700 1 -- 172.21.15.78:0/1059196896 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cbc0cc4b0 con 0x7f4cbc156ce0 2023-10-11T09:45:42.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.912+0000 7f4cc2384700 1 -- 172.21.15.78:0/1059196896 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cbc065d40 con 0x7f4cbc156ce0 2023-10-11T09:45:42.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.912+0000 7f4cb1ffb700 1 -- 172.21.15.78:0/1059196896 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f4cb400ac10 con 0x7f4cbc156ce0 2023-10-11T09:45:42.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.912+0000 7f4cb1ffb700 1 --2- 172.21.15.78:0/1059196896 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4ca0065ed0 0x7f4ca0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:42.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.912+0000 7f4cb1ffb700 1 -- 172.21.15.78:0/1059196896 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f4cb4083a50 con 0x7f4cbc156ce0 2023-10-11T09:45:42.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.912+0000 7f4cc1382700 1 --2- 172.21.15.78:0/1059196896 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4ca0065ed0 0x7f4ca0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:42.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.916+0000 7f4cc1382700 1 --2- 172.21.15.78:0/1059196896 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4ca0065ed0 0x7f4ca0068380 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4cb80090b0 tx=0x7f4cb801c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:42.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:42.920+0000 7f4cb1ffb700 1 -- 172.21.15.78:0/1059196896 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f4cb4050dd0 con 0x7f4cbc156ce0 2023-10-11T09:45:43.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.112+0000 7f4cc2384700 1 -- 172.21.15.78:0/1059196896 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4cbc06d310 con 0x7f4cbc156ce0 2023-10-11T09:45:43.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.112+0000 7f4cb1ffb700 1 -- 172.21.15.78:0/1059196896 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v64) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4cb40543f0 con 0x7f4cbc156ce0 2023-10-11T09:45:43.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.116+0000 7f4cc2384700 1 -- 172.21.15.78:0/1059196896 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4ca0065ed0 msgr2=0x7f4ca0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:43.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.116+0000 7f4cc2384700 1 --2- 172.21.15.78:0/1059196896 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4ca0065ed0 0x7f4ca0068380 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4cb80090b0 tx=0x7f4cb801c040 comp rx=0 tx=0).stop 2023-10-11T09:45:43.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.116+0000 7f4cc2384700 1 -- 172.21.15.78:0/1059196896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cbc156ce0 msgr2=0x7f4cbc0c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:43.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.116+0000 7f4cc2384700 1 --2- 172.21.15.78:0/1059196896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cbc156ce0 0x7f4cbc0c7cb0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f4cb400cc50 tx=0x7f4cb400cf60 comp rx=0 tx=0).stop 2023-10-11T09:45:43.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.116+0000 7f4cc2384700 1 -- 172.21.15.78:0/1059196896 shutdown_connections 2023-10-11T09:45:43.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.116+0000 7f4cc2384700 1 --2- 172.21.15.78:0/1059196896 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4ca0065ed0 0x7f4ca0068380 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:43.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.116+0000 7f4cc2384700 1 --2- 172.21.15.78:0/1059196896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cbc156ce0 0x7f4cbc0c7cb0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:43.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.116+0000 7f4cc2384700 1 --2- 172.21.15.78:0/1059196896 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4cbc0d28e0 0x7f4cbc0cc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:43.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.116+0000 7f4cc2384700 1 --2- 172.21.15.78:0/1059196896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cbc0d1660 0x7f4cbc0cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:43.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.116+0000 7f4cc2384700 1 -- 172.21.15.78:0/1059196896 >> 172.21.15.78:0/1059196896 conn(0x7f4cbc06afb0 msgr2=0x7f4cbc0cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:43.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.116+0000 7f4cc2384700 1 -- 172.21.15.78:0/1059196896 shutdown_connections 2023-10-11T09:45:43.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.116+0000 7f4cc2384700 1 -- 172.21.15.78:0/1059196896 wait complete. 2023-10-11T09:45:43.128 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2023-10-11T09:45:43.146 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:45:43.147 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:45:43.164 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:45:43.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.376+0000 7f42df3bf700 1 Processor -- start 2023-10-11T09:45:43.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.376+0000 7f42df3bf700 1 -- start start 2023-10-11T09:45:43.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.380+0000 7f42df3bf700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f42d806d020 0x7f42d8165550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:43.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.380+0000 7f42df3bf700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42d80720f0 con 0x7f42d8154e00 2023-10-11T09:45:43.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.380+0000 7f42df3bf700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42d80714e0 con 0x7f42d8071040 2023-10-11T09:45:43.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.380+0000 7f42df3bf700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d80717c0 con 0x7f42d806d020 2023-10-11T09:45:43.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.380+0000 7f42dd95c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f42d806d020 0x7f42d8165550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:43.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.380+0000 7f42dd15b700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f42d8154e00 0x7f42d815b650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:44568/0 (socket says 172.21.15.78:44568) 2023-10-11T09:45:43.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.380+0000 7f42dd15b700 1 -- 172.21.15.78:0/4242557038 learned_addr learned my addr 172.21.15.78:0/4242557038 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:45:43.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.380+0000 7f42dd95c700 1 -- 172.21.15.78:0/4242557038 >> v1:172.21.15.133:6789/0 conn(0x7f42d8071040 legacy=0x7f42d80cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:43.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.380+0000 7f42dd95c700 1 -- 172.21.15.78:0/4242557038 >> v1:172.21.15.78:6789/0 conn(0x7f42d8154e00 legacy=0x7f42d815b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:43.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.380+0000 7f42dd95c700 1 -- 172.21.15.78:0/4242557038 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d8071aa0 con 0x7f42d806d020 2023-10-11T09:45:43.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.380+0000 7f42dd95c700 1 --2- 172.21.15.78:0/4242557038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42d806d020 0x7f42d8165550 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f42d0009d90 tx=0x7f42d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd76d576b398baff server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:43.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.384+0000 7f42cbfff700 1 -- 172.21.15.78:0/4242557038 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d000e070 con 0x7f42d806d020 2023-10-11T09:45:43.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.384+0000 7f42cbfff700 1 -- 172.21.15.78:0/4242557038 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d000bbd0 con 0x7f42d806d020 2023-10-11T09:45:43.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.384+0000 7f42cbfff700 1 -- 172.21.15.78:0/4242557038 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d0012430 con 0x7f42d806d020 2023-10-11T09:45:43.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.384+0000 7f42df3bf700 1 -- 172.21.15.78:0/4242557038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42d806d020 msgr2=0x7f42d8165550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:43.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.384+0000 7f42df3bf700 1 --2- 172.21.15.78:0/4242557038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42d806d020 0x7f42d8165550 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f42d0009d90 tx=0x7f42d000b330 comp rx=0 tx=0).stop 2023-10-11T09:45:43.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.384+0000 7f42df3bf700 1 -- 172.21.15.78:0/4242557038 shutdown_connections 2023-10-11T09:45:43.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.384+0000 7f42df3bf700 1 --2- 172.21.15.78:0/4242557038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42d806d020 0x7f42d8165550 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:43.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.384+0000 7f42df3bf700 1 -- 172.21.15.78:0/4242557038 >> 172.21.15.78:0/4242557038 conn(0x7f42d806aca0 msgr2=0x7f42d80b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:43.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.384+0000 7f42df3bf700 1 -- 172.21.15.78:0/4242557038 shutdown_connections 2023-10-11T09:45:43.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.384+0000 7f42df3bf700 1 -- 172.21.15.78:0/4242557038 wait complete. 2023-10-11T09:45:43.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.388+0000 7f42df3bf700 1 Processor -- start 2023-10-11T09:45:43.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.388+0000 7f42df3bf700 1 -- start start 2023-10-11T09:45:43.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.388+0000 7f42df3bf700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42d806d020 0x7f42d80c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:43.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.388+0000 7f42df3bf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42d8071040 0x7f42d814c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:43.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.388+0000 7f42df3bf700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42d8154e00 0x7f42d80af930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:43.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.388+0000 7f42df3bf700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d80b2a00 con 0x7f42d8071040 2023-10-11T09:45:43.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.388+0000 7f42df3bf700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d80b2440 con 0x7f42d806d020 2023-10-11T09:45:43.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.388+0000 7f42df3bf700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d80b2720 con 0x7f42d8154e00 2023-10-11T09:45:43.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.388+0000 7f42dc95a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42d8071040 0x7f42d814c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:43.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.388+0000 7f42dc95a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42d8071040 0x7f42d814c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60780/0 (socket says 172.21.15.78:60780) 2023-10-11T09:45:43.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.388+0000 7f42dc95a700 1 -- 172.21.15.78:0/2369289200 learned_addr learned my addr 172.21.15.78:0/2369289200 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:43.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.388+0000 7f42dd15b700 1 --2- 172.21.15.78:0/2369289200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42d806d020 0x7f42d80c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:43.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.388+0000 7f42dd95c700 1 --2- 172.21.15.78:0/2369289200 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42d8154e00 0x7f42d80af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:43.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.388+0000 7f42dc95a700 1 -- 172.21.15.78:0/2369289200 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42d8154e00 msgr2=0x7f42d80af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:43.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.388+0000 7f42dc95a700 1 --2- 172.21.15.78:0/2369289200 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42d8154e00 0x7f42d80af930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:43.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.388+0000 7f42dc95a700 1 -- 172.21.15.78:0/2369289200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42d806d020 msgr2=0x7f42d80c85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:43.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.388+0000 7f42dc95a700 1 --2- 172.21.15.78:0/2369289200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42d806d020 0x7f42d80c85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:43.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.388+0000 7f42dc95a700 1 -- 172.21.15.78:0/2369289200 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d0009a70 con 0x7f42d8071040 2023-10-11T09:45:43.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.388+0000 7f42dc95a700 1 --2- 172.21.15.78:0/2369289200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42d8071040 0x7f42d814c100 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f42d400c4d0 tx=0x7f42d400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:43.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.392+0000 7f42c9ffb700 1 -- 172.21.15.78:0/2369289200 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d40052d0 con 0x7f42d8071040 2023-10-11T09:45:43.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.392+0000 7f42c9ffb700 1 -- 172.21.15.78:0/2369289200 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d4005430 con 0x7f42d8071040 2023-10-11T09:45:43.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.392+0000 7f42df3bf700 1 -- 172.21.15.78:0/2369289200 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d80b2d20 con 0x7f42d8071040 2023-10-11T09:45:43.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.392+0000 7f42c9ffb700 1 -- 172.21.15.78:0/2369289200 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d40055a0 con 0x7f42d8071040 2023-10-11T09:45:43.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.392+0000 7f42df3bf700 1 -- 172.21.15.78:0/2369289200 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d80c8b20 con 0x7f42d8071040 2023-10-11T09:45:43.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.392+0000 7f42df3bf700 1 -- 172.21.15.78:0/2369289200 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42d80b2b80 con 0x7f42d8071040 2023-10-11T09:45:43.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.392+0000 7f42c9ffb700 1 -- 172.21.15.78:0/2369289200 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f42d40038c0 con 0x7f42d8071040 2023-10-11T09:45:43.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.396+0000 7f42c9ffb700 1 --2- 172.21.15.78:0/2369289200 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f42bc065ed0 0x7f42bc068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:43.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.396+0000 7f42c9ffb700 1 -- 172.21.15.78:0/2369289200 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f42d4082ec0 con 0x7f42d8071040 2023-10-11T09:45:43.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.396+0000 7f42dd15b700 1 --2- 172.21.15.78:0/2369289200 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f42bc065ed0 0x7f42bc068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:43.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.396+0000 7f42dd15b700 1 --2- 172.21.15.78:0/2369289200 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f42bc065ed0 0x7f42bc068380 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f42cc005930 tx=0x7f42cc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:43.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.400+0000 7f42c9ffb700 1 -- 172.21.15.78:0/2369289200 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f42d80b2b80 con 0x7f42d8071040 2023-10-11T09:45:43.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.576+0000 7f42df3bf700 1 -- 172.21.15.78:0/2369289200 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42d80b3240 con 0x7f42d8071040 2023-10-11T09:45:43.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.580+0000 7f42c9ffb700 1 -- 172.21.15.78:0/2369289200 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+19665 (secure 0 0 0) 0x7f42d80b3240 con 0x7f42d8071040 2023-10-11T09:45:43.589 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:43.589 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":64,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:44:17.719929+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:44:05.198742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-11T09:44:05.868171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data_extra","create_time":"2023-10-11T09:44:16.049941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:45:43.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.588+0000 7f42df3bf700 1 -- 172.21.15.78:0/2369289200 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f42bc065ed0 msgr2=0x7f42bc068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:43.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.588+0000 7f42df3bf700 1 --2- 172.21.15.78:0/2369289200 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f42bc065ed0 0x7f42bc068380 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f42cc005930 tx=0x7f42cc009040 comp rx=0 tx=0).stop 2023-10-11T09:45:43.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.588+0000 7f42df3bf700 1 -- 172.21.15.78:0/2369289200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42d8071040 msgr2=0x7f42d814c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:43.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.588+0000 7f42df3bf700 1 --2- 172.21.15.78:0/2369289200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42d8071040 0x7f42d814c100 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f42d400c4d0 tx=0x7f42d400c890 comp rx=0 tx=0).stop 2023-10-11T09:45:43.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.588+0000 7f42df3bf700 1 -- 172.21.15.78:0/2369289200 shutdown_connections 2023-10-11T09:45:43.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.588+0000 7f42df3bf700 1 --2- 172.21.15.78:0/2369289200 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f42bc065ed0 0x7f42bc068380 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:43.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.588+0000 7f42df3bf700 1 --2- 172.21.15.78:0/2369289200 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42d8154e00 0x7f42d80af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:43.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.588+0000 7f42df3bf700 1 --2- 172.21.15.78:0/2369289200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42d8071040 0x7f42d814c100 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:43.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.588+0000 7f42df3bf700 1 --2- 172.21.15.78:0/2369289200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42d806d020 0x7f42d80c85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:43.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.588+0000 7f42df3bf700 1 -- 172.21.15.78:0/2369289200 >> 172.21.15.78:0/2369289200 conn(0x7f42d806aca0 msgr2=0x7f42d815e750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:43.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.588+0000 7f42df3bf700 1 -- 172.21.15.78:0/2369289200 shutdown_connections 2023-10-11T09:45:43.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.588+0000 7f42df3bf700 1 -- 172.21.15.78:0/2369289200 wait complete. 2023-10-11T09:45:43.611 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T09:44:05.198742+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T09:44:05.868171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-10-11T09:44:16.049941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:45:43.612 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:45:43.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.780+0000 7f9fff59e700 1 Processor -- start 2023-10-11T09:45:43.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.780+0000 7f9fff59e700 1 -- start start 2023-10-11T09:45:43.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.784+0000 7f9fff59e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa0000b3240 0x7fa0000d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:43.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.784+0000 7f9fff59e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fa0000b3610 0x7fa0000b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:43.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.784+0000 7f9fff59e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa00015d040 0x7fa00015d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:43.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.784+0000 7f9fff59e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0000b3fd0 con 0x7fa0000b3610 2023-10-11T09:45:43.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.784+0000 7f9fff59e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0000b47f0 con 0x7fa0000b3240 2023-10-11T09:45:43.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.784+0000 7f9fff59e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0000b8130 con 0x7fa00015d040 2023-10-11T09:45:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.784+0000 7f9ffdd9b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fa0000b3610 0x7fa0000b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.784+0000 7f9ffdd9b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fa0000b3610 0x7fa0000b3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60784/0 (socket says 172.21.15.78:60784) 2023-10-11T09:45:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.784+0000 7f9ffdd9b700 1 -- 172.21.15.78:0/3586892587 learned_addr learned my addr 172.21.15.78:0/3586892587 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.784+0000 7f9ffe59c700 1 --2- 172.21.15.78:0/3586892587 >> v2:172.21.15.133:3300/0 conn(0x7fa0000b3240 0x7fa0000d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.784+0000 7f9ffed9d700 1 --2- 172.21.15.78:0/3586892587 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa00015d040 0x7fa00015d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:43.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.784+0000 7f9ffdd9b700 1 -- 172.21.15.78:0/3586892587 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa00015d040 msgr2=0x7fa00015d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:43.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.784+0000 7f9ffdd9b700 1 --2- 172.21.15.78:0/3586892587 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa00015d040 0x7fa00015d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:43.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.784+0000 7f9ffdd9b700 1 -- 172.21.15.78:0/3586892587 >> v2:172.21.15.133:3300/0 conn(0x7fa0000b3240 msgr2=0x7fa0000d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:43.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.784+0000 7f9ffdd9b700 1 --2- 172.21.15.78:0/3586892587 >> v2:172.21.15.133:3300/0 conn(0x7fa0000b3240 0x7fa0000d29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:43.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.784+0000 7f9ffdd9b700 1 -- 172.21.15.78:0/3586892587 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0000b41e0 con 0x7fa0000b3610 2023-10-11T09:45:43.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.784+0000 7f9ffdd9b700 1 --2- 172.21.15.78:0/3586892587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0000b3610 0x7fa0000b3a90 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa00006aa40 tx=0x7f9ff4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8dc73239eddcc5 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:43.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.784+0000 7f9ffd59a700 1 -- 172.21.15.78:0/3586892587 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff4011830 con 0x7fa0000b3610 2023-10-11T09:45:43.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.784+0000 7f9ffd59a700 1 -- 172.21.15.78:0/3586892587 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff4011990 con 0x7fa0000b3610 2023-10-11T09:45:43.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.784+0000 7f9ffd59a700 1 -- 172.21.15.78:0/3586892587 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff4011d20 con 0x7fa0000b3610 2023-10-11T09:45:43.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.784+0000 7f9fff59e700 1 -- 172.21.15.78:0/3586892587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0000b3610 msgr2=0x7fa0000b3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:43.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.784+0000 7f9fff59e700 1 --2- 172.21.15.78:0/3586892587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0000b3610 0x7fa0000b3a90 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa00006aa40 tx=0x7f9ff4009bf0 comp rx=0 tx=0).stop 2023-10-11T09:45:43.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.784+0000 7f9fff59e700 1 -- 172.21.15.78:0/3586892587 shutdown_connections 2023-10-11T09:45:43.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.784+0000 7f9fff59e700 1 --2- 172.21.15.78:0/3586892587 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa00015d040 0x7fa00015d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:43.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.784+0000 7f9fff59e700 1 --2- 172.21.15.78:0/3586892587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0000b3610 0x7fa0000b3a90 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:43.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.784+0000 7f9fff59e700 1 --2- 172.21.15.78:0/3586892587 >> v2:172.21.15.133:3300/0 conn(0x7fa0000b3240 0x7fa0000d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:43.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.784+0000 7f9fff59e700 1 -- 172.21.15.78:0/3586892587 >> 172.21.15.78:0/3586892587 conn(0x7fa00009c260 msgr2=0x7fa0000d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:43.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.784+0000 7f9fff59e700 1 -- 172.21.15.78:0/3586892587 shutdown_connections 2023-10-11T09:45:43.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.784+0000 7f9fff59e700 1 -- 172.21.15.78:0/3586892587 wait complete. 2023-10-11T09:45:43.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.788+0000 7f9fff59e700 1 Processor -- start 2023-10-11T09:45:43.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.788+0000 7f9fff59e700 1 -- start start 2023-10-11T09:45:43.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.788+0000 7f9fff59e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0000b3240 0x7fa0000d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:43.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.788+0000 7f9fff59e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0000b3610 0x7fa00015eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:43.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.788+0000 7f9fff59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa00015d040 0x7fa00015a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:43.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.788+0000 7f9fff59e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0000c6d60 con 0x7fa00015d040 2023-10-11T09:45:43.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.788+0000 7f9fff59e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa00009d600 con 0x7fa0000b3240 2023-10-11T09:45:43.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.788+0000 7f9fff59e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa00009d8a0 con 0x7fa0000b3610 2023-10-11T09:45:43.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.788+0000 7f9ffed9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa00015d040 0x7fa00015a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:43.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.788+0000 7f9ffed9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa00015d040 0x7fa00015a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60790/0 (socket says 172.21.15.78:60790) 2023-10-11T09:45:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.788+0000 7f9ffed9d700 1 -- 172.21.15.78:0/1314894121 learned_addr learned my addr 172.21.15.78:0/1314894121 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.788+0000 7f9ffdd9b700 1 --2- 172.21.15.78:0/1314894121 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0000b3610 0x7fa00015eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.788+0000 7f9ffe59c700 1 --2- 172.21.15.78:0/1314894121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0000b3240 0x7fa0000d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:43.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.788+0000 7f9ffed9d700 1 -- 172.21.15.78:0/1314894121 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0000b3610 msgr2=0x7fa00015eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:43.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.788+0000 7f9ffed9d700 1 --2- 172.21.15.78:0/1314894121 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0000b3610 0x7fa00015eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:43.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.788+0000 7f9ffed9d700 1 -- 172.21.15.78:0/1314894121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0000b3240 msgr2=0x7fa0000d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:43.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.788+0000 7f9ffed9d700 1 --2- 172.21.15.78:0/1314894121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0000b3240 0x7fa0000d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:43.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.788+0000 7f9ffed9d700 1 -- 172.21.15.78:0/1314894121 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff4009570 con 0x7fa00015d040 2023-10-11T09:45:43.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.788+0000 7f9ffe59c700 1 --2- 172.21.15.78:0/1314894121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0000b3240 0x7fa0000d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:45:43.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.792+0000 7f9ffed9d700 1 --2- 172.21.15.78:0/1314894121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa00015d040 0x7fa00015a4c0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8007e60 tx=0x7f9ff800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:43.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.792+0000 7f9feeffd700 1 -- 172.21.15.78:0/1314894121 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff800ede0 con 0x7fa00015d040 2023-10-11T09:45:43.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.792+0000 7f9feeffd700 1 -- 172.21.15.78:0/1314894121 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff8014420 con 0x7fa00015d040 2023-10-11T09:45:43.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.792+0000 7f9feeffd700 1 -- 172.21.15.78:0/1314894121 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff80147d0 con 0x7fa00015d040 2023-10-11T09:45:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.792+0000 7f9fff59e700 1 -- 172.21.15.78:0/1314894121 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa00009db80 con 0x7fa00015d040 2023-10-11T09:45:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.792+0000 7f9fff59e700 1 -- 172.21.15.78:0/1314894121 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa00015f3e0 con 0x7fa00015d040 2023-10-11T09:45:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.792+0000 7f9fff59e700 1 -- 172.21.15.78:0/1314894121 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa00015f630 con 0x7fa00015d040 2023-10-11T09:45:43.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.796+0000 7f9feeffd700 1 -- 172.21.15.78:0/1314894121 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f9ff8014930 con 0x7fa00015d040 2023-10-11T09:45:43.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.796+0000 7f9feeffd700 1 --2- 172.21.15.78:0/1314894121 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9fe0065ed0 0x7f9fe0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:43.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.796+0000 7f9feeffd700 1 -- 172.21.15.78:0/1314894121 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f9ff8084080 con 0x7fa00015d040 2023-10-11T09:45:43.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.796+0000 7f9ffe59c700 1 --2- 172.21.15.78:0/1314894121 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9fe0065ed0 0x7f9fe0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.796+0000 7f9ffe59c700 1 --2- 172.21.15.78:0/1314894121 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9fe0065ed0 0x7f9fe0068380 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa00006cbc0 tx=0x7f9ff0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.800+0000 7f9feeffd700 1 -- 172.21.15.78:0/1314894121 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f9ff80b4fa0 con 0x7fa00015d040 2023-10-11T09:45:43.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.976+0000 7f9fff59e700 1 -- 172.21.15.78:0/1314894121 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa0000c6ea0 con 0x7fa00015d040 2023-10-11T09:45:43.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.976+0000 7f9feeffd700 1 -- 172.21.15.78:0/1314894121 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v64) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa0000c6ea0 con 0x7fa00015d040 2023-10-11T09:45:43.985 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:45:43.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.980+0000 7f9fff59e700 1 -- 172.21.15.78:0/1314894121 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9fe0065ed0 msgr2=0x7f9fe0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:43.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.980+0000 7f9fff59e700 1 --2- 172.21.15.78:0/1314894121 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9fe0065ed0 0x7f9fe0068380 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa00006cbc0 tx=0x7f9ff0008040 comp rx=0 tx=0).stop 2023-10-11T09:45:43.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.980+0000 7f9fff59e700 1 -- 172.21.15.78:0/1314894121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa00015d040 msgr2=0x7fa00015a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:43.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.980+0000 7f9fff59e700 1 --2- 172.21.15.78:0/1314894121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa00015d040 0x7fa00015a4c0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8007e60 tx=0x7f9ff800e3c0 comp rx=0 tx=0).stop 2023-10-11T09:45:43.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.980+0000 7f9fff59e700 1 -- 172.21.15.78:0/1314894121 shutdown_connections 2023-10-11T09:45:43.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.980+0000 7f9fff59e700 1 --2- 172.21.15.78:0/1314894121 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9fe0065ed0 0x7f9fe0068380 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:43.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.980+0000 7f9fff59e700 1 --2- 172.21.15.78:0/1314894121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa00015d040 0x7fa00015a4c0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:43.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.980+0000 7f9fff59e700 1 --2- 172.21.15.78:0/1314894121 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0000b3610 0x7fa00015eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:43.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.980+0000 7f9fff59e700 1 --2- 172.21.15.78:0/1314894121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0000b3240 0x7fa0000d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:43.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.980+0000 7f9fff59e700 1 -- 172.21.15.78:0/1314894121 >> 172.21.15.78:0/1314894121 conn(0x7fa00009c260 msgr2=0x7fa00015c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:43.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.980+0000 7f9fff59e700 1 -- 172.21.15.78:0/1314894121 shutdown_connections 2023-10-11T09:45:43.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:43.980+0000 7f9fff59e700 1 -- 172.21.15.78:0/1314894121 wait complete. 2023-10-11T09:45:44.009 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:45:44.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.180+0000 7f602759e700 1 Processor -- start 2023-10-11T09:45:44.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.180+0000 7f602759e700 1 -- start start 2023-10-11T09:45:44.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.180+0000 7f602759e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f60280b3a60 0x7f602806d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:44.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.180+0000 7f602759e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f602806d560 0x7f602806bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:44.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.180+0000 7f602759e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60280b2d30 con 0x7f602815a6b0 2023-10-11T09:45:44.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.180+0000 7f602759e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f602806d960 con 0x7f60280b3a60 2023-10-11T09:45:44.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.180+0000 7f602759e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60280cf360 con 0x7f602806d560 2023-10-11T09:45:44.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.180+0000 7f6026d9d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f602806d560 0x7f602806bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:44.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.180+0000 7f601dd9b700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f60280b3a60 0x7f602806d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:44.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.180+0000 7f602659c700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f602815a6b0 0x7f60280ce8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:47318/0 (socket says 172.21.15.78:47318) 2023-10-11T09:45:44.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.180+0000 7f601dd9b700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f60280b3a60 0x7f602806d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:44722/0 (socket says 172.21.15.78:44722) 2023-10-11T09:45:44.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.180+0000 7f6026d9d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f602806d560 0x7f602806bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:52172/0 (socket says 172.21.15.78:52172) 2023-10-11T09:45:44.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.180+0000 7f602659c700 1 -- 172.21.15.78:0/1002123433 learned_addr learned my addr 172.21.15.78:0/1002123433 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:45:44.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.180+0000 7f6026d9d700 1 -- 172.21.15.78:0/1002123433 >> v2:172.21.15.133:3300/0 conn(0x7f60280b3a60 msgr2=0x7f602806d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:44.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.180+0000 7f6026d9d700 1 --2- 172.21.15.78:0/1002123433 >> v2:172.21.15.133:3300/0 conn(0x7f60280b3a60 0x7f602806d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:44.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.180+0000 7f6026d9d700 1 -- 172.21.15.78:0/1002123433 >> v1:172.21.15.133:6789/0 conn(0x7f602815a6b0 legacy=0x7f60280ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:44.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.180+0000 7f6026d9d700 1 -- 172.21.15.78:0/1002123433 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60280b2f80 con 0x7f602806d560 2023-10-11T09:45:44.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.180+0000 7f6025d9b700 1 -- 172.21.15.78:0/1002123433 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1106998039 0 0) 0x7f60280b2d30 con 0x7f602815a6b0 2023-10-11T09:45:44.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.180+0000 7f6026d9d700 1 --2- 172.21.15.78:0/1002123433 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f602806d560 0x7f602806bbe0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6020009d90 tx=0x7f602000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80b011194ba70bc5 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:44.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.180+0000 7f6025d9b700 1 -- 172.21.15.78:0/1002123433 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602000e070 con 0x7f602806d560 2023-10-11T09:45:44.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.180+0000 7f6025d9b700 1 -- 172.21.15.78:0/1002123433 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602000bb20 con 0x7f602806d560 2023-10-11T09:45:44.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.180+0000 7f6025d9b700 1 -- 172.21.15.78:0/1002123433 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6020012430 con 0x7f602806d560 2023-10-11T09:45:44.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.184+0000 7f602759e700 1 -- 172.21.15.78:0/1002123433 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f602806d560 msgr2=0x7f602806bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:44.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.184+0000 7f602759e700 1 --2- 172.21.15.78:0/1002123433 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f602806d560 0x7f602806bbe0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6020009d90 tx=0x7f602000b330 comp rx=0 tx=0).stop 2023-10-11T09:45:44.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.184+0000 7f602759e700 1 -- 172.21.15.78:0/1002123433 shutdown_connections 2023-10-11T09:45:44.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.184+0000 7f602759e700 1 --2- 172.21.15.78:0/1002123433 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f602806d560 0x7f602806bbe0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:44.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.184+0000 7f602759e700 1 --2- 172.21.15.78:0/1002123433 >> v2:172.21.15.133:3300/0 conn(0x7f60280b3a60 0x7f602806d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:44.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.184+0000 7f602759e700 1 -- 172.21.15.78:0/1002123433 >> 172.21.15.78:0/1002123433 conn(0x7f602806a060 msgr2=0x7f602806a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:44.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.184+0000 7f602759e700 1 -- 172.21.15.78:0/1002123433 shutdown_connections 2023-10-11T09:45:44.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.184+0000 7f602759e700 1 -- 172.21.15.78:0/1002123433 wait complete. 2023-10-11T09:45:44.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.192+0000 7f602759e700 1 Processor -- start 2023-10-11T09:45:44.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.192+0000 7f602759e700 1 -- start start 2023-10-11T09:45:44.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.192+0000 7f602759e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f602806d560 0x7f60281f57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:44.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.192+0000 7f602759e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60280b3a60 0x7f60281ef5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:44.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.192+0000 7f602759e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f602815a6b0 0x7f60281f28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:44.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.192+0000 7f602759e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60281f6770 con 0x7f602815a6b0 2023-10-11T09:45:44.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.192+0000 7f602759e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60281f21e0 con 0x7f60280b3a60 2023-10-11T09:45:44.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.192+0000 7f602759e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6028153940 con 0x7f602806d560 2023-10-11T09:45:44.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.192+0000 7f6026d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f602815a6b0 0x7f60281f28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:44.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.192+0000 7f602659c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f602806d560 0x7f60281f57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:44.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.192+0000 7f6026d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f602815a6b0 0x7f60281f28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60794/0 (socket says 172.21.15.78:60794) 2023-10-11T09:45:44.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.192+0000 7f6026d9d700 1 -- 172.21.15.78:0/1917022765 learned_addr learned my addr 172.21.15.78:0/1917022765 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:44.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.192+0000 7f601dd9b700 1 --2- 172.21.15.78:0/1917022765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60280b3a60 0x7f60281ef5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:44.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.192+0000 7f6026d9d700 1 -- 172.21.15.78:0/1917022765 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f602806d560 msgr2=0x7f60281f57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:44.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.192+0000 7f6026d9d700 1 --2- 172.21.15.78:0/1917022765 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f602806d560 0x7f60281f57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:44.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.192+0000 7f6026d9d700 1 -- 172.21.15.78:0/1917022765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60280b3a60 msgr2=0x7f60281ef5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:44.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.192+0000 7f6026d9d700 1 --2- 172.21.15.78:0/1917022765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60280b3a60 0x7f60281ef5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:44.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.192+0000 7f6026d9d700 1 -- 172.21.15.78:0/1917022765 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6020009a70 con 0x7f602815a6b0 2023-10-11T09:45:44.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.192+0000 7f602659c700 1 --2- 172.21.15.78:0/1917022765 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f602806d560 0x7f60281f57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:45:44.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.192+0000 7f6026d9d700 1 --2- 172.21.15.78:0/1917022765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f602815a6b0 0x7f60281f28f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f60280cf4a0 tx=0x7f6020004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:44.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.192+0000 7f601f7fe700 1 -- 172.21.15.78:0/1917022765 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6020012430 con 0x7f602815a6b0 2023-10-11T09:45:44.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.192+0000 7f601f7fe700 1 -- 172.21.15.78:0/1917022765 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6020020070 con 0x7f602815a6b0 2023-10-11T09:45:44.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.192+0000 7f601f7fe700 1 -- 172.21.15.78:0/1917022765 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6020004280 con 0x7f602815a6b0 2023-10-11T09:45:44.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.192+0000 7f602759e700 1 -- 172.21.15.78:0/1917022765 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60281f5ed0 con 0x7f602815a6b0 2023-10-11T09:45:44.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.192+0000 7f602759e700 1 -- 172.21.15.78:0/1917022765 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60281f6490 con 0x7f602815a6b0 2023-10-11T09:45:44.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.192+0000 7f602759e700 1 -- 172.21.15.78:0/1917022765 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6028065d40 con 0x7f602815a6b0 2023-10-11T09:45:44.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.196+0000 7f601f7fe700 1 -- 172.21.15.78:0/1917022765 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f60200043e0 con 0x7f602815a6b0 2023-10-11T09:45:44.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.196+0000 7f601f7fe700 1 --2- 172.21.15.78:0/1917022765 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f600c065ed0 0x7f600c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:44.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.196+0000 7f601f7fe700 1 -- 172.21.15.78:0/1917022765 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f6020022070 con 0x7f602815a6b0 2023-10-11T09:45:44.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.196+0000 7f602659c700 1 --2- 172.21.15.78:0/1917022765 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f600c065ed0 0x7f600c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:44.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.196+0000 7f602659c700 1 --2- 172.21.15.78:0/1917022765 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f600c065ed0 0x7f600c068380 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6018000f60 tx=0x7f6018007e90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:44.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.204+0000 7f601f7fe700 1 -- 172.21.15.78:0/1917022765 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f6020052a30 con 0x7f602815a6b0 2023-10-11T09:45:44.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.376+0000 7f602759e700 1 -- 172.21.15.78:0/1917022765 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f602814ff80 con 0x7f602815a6b0 2023-10-11T09:45:44.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.376+0000 7f601f7fe700 1 -- 172.21.15.78:0/1917022765 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v64) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6020056050 con 0x7f602815a6b0 2023-10-11T09:45:44.384 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:45:44.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.380+0000 7f602759e700 1 -- 172.21.15.78:0/1917022765 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f600c065ed0 msgr2=0x7f600c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:44.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.380+0000 7f602759e700 1 --2- 172.21.15.78:0/1917022765 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f600c065ed0 0x7f600c068380 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6018000f60 tx=0x7f6018007e90 comp rx=0 tx=0).stop 2023-10-11T09:45:44.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.380+0000 7f602759e700 1 -- 172.21.15.78:0/1917022765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f602815a6b0 msgr2=0x7f60281f28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:44.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.380+0000 7f602759e700 1 --2- 172.21.15.78:0/1917022765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f602815a6b0 0x7f60281f28f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f60280cf4a0 tx=0x7f6020004af0 comp rx=0 tx=0).stop 2023-10-11T09:45:44.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.380+0000 7f602759e700 1 -- 172.21.15.78:0/1917022765 shutdown_connections 2023-10-11T09:45:44.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.380+0000 7f602759e700 1 --2- 172.21.15.78:0/1917022765 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f600c065ed0 0x7f600c068380 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:44.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.380+0000 7f602759e700 1 --2- 172.21.15.78:0/1917022765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f602815a6b0 0x7f60281f28f0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:44.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.380+0000 7f602759e700 1 --2- 172.21.15.78:0/1917022765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60280b3a60 0x7f60281ef5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:44.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.380+0000 7f602759e700 1 --2- 172.21.15.78:0/1917022765 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f602806d560 0x7f60281f57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:44.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.380+0000 7f602759e700 1 -- 172.21.15.78:0/1917022765 >> 172.21.15.78:0/1917022765 conn(0x7f602806a060 msgr2=0x7f60280ce5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:44.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.380+0000 7f602759e700 1 -- 172.21.15.78:0/1917022765 shutdown_connections 2023-10-11T09:45:44.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.380+0000 7f602759e700 1 -- 172.21.15.78:0/1917022765 wait complete. 2023-10-11T09:45:44.410 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T09:45:44.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.572+0000 7f717ce48700 1 Processor -- start 2023-10-11T09:45:44.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.572+0000 7f717ce48700 1 -- start start 2023-10-11T09:45:44.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.572+0000 7f717ce48700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f71780ca1f0 0x7f71780ca640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:44.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.572+0000 7f717ce48700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f71780b1820 0x7f71780b1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:44.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.572+0000 7f717ce48700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71780b21c0 con 0x7f717813f970 2023-10-11T09:45:44.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.572+0000 7f717ce48700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71780cab80 con 0x7f71780b1820 2023-10-11T09:45:44.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.572+0000 7f717ce48700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7178071870 con 0x7f71780ca1f0 2023-10-11T09:45:44.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.576+0000 7f7177fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f71780b1820 0x7f71780b1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:44.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.576+0000 7f7176ffd700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f71780ca1f0 0x7f71780ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:44.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.576+0000 7f7177fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f71780b1820 0x7f71780b1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50566/0 (socket says 172.21.15.78:50566) 2023-10-11T09:45:44.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.576+0000 7f7177fff700 1 -- 172.21.15.78:0/2348113784 learned_addr learned my addr 172.21.15.78:0/2348113784 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:44.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.576+0000 7f7177fff700 1 -- 172.21.15.78:0/2348113784 >> v2:172.21.15.133:3300/0 conn(0x7f71780ca1f0 msgr2=0x7f71780ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:44.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.576+0000 7f7177fff700 1 --2- 172.21.15.78:0/2348113784 >> v2:172.21.15.133:3300/0 conn(0x7f71780ca1f0 0x7f71780ca640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:44.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.576+0000 7f7177fff700 1 -- 172.21.15.78:0/2348113784 >> v1:172.21.15.133:6789/0 conn(0x7f717813f970 legacy=0x7f7178071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:44.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.576+0000 7f7177fff700 1 -- 172.21.15.78:0/2348113784 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f717815dc90 con 0x7f71780b1820 2023-10-11T09:45:44.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.576+0000 7f7177fff700 1 --2- 172.21.15.78:0/2348113784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71780b1820 0x7f71780b1c80 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f71780cd720 tx=0x7f717000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9efaa68137aad48d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:44.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.576+0000 7f71767fc700 1 -- 172.21.15.78:0/2348113784 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717000e070 con 0x7f71780b1820 2023-10-11T09:45:44.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.576+0000 7f71767fc700 1 -- 172.21.15.78:0/2348113784 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f717000b910 con 0x7f71780b1820 2023-10-11T09:45:44.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.576+0000 7f71767fc700 1 -- 172.21.15.78:0/2348113784 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717000bcc0 con 0x7f71780b1820 2023-10-11T09:45:44.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.576+0000 7f717ce48700 1 -- 172.21.15.78:0/2348113784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71780b1820 msgr2=0x7f71780b1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:44.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.576+0000 7f717ce48700 1 --2- 172.21.15.78:0/2348113784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71780b1820 0x7f71780b1c80 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f71780cd720 tx=0x7f717000b330 comp rx=0 tx=0).stop 2023-10-11T09:45:44.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.576+0000 7f717ce48700 1 -- 172.21.15.78:0/2348113784 shutdown_connections 2023-10-11T09:45:44.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.576+0000 7f717ce48700 1 --2- 172.21.15.78:0/2348113784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71780b1820 0x7f71780b1c80 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:44.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.576+0000 7f717ce48700 1 --2- 172.21.15.78:0/2348113784 >> v2:172.21.15.133:3300/0 conn(0x7f71780ca1f0 0x7f71780ca640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:44.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.576+0000 7f717ce48700 1 -- 172.21.15.78:0/2348113784 >> 172.21.15.78:0/2348113784 conn(0x7f717806dd80 msgr2=0x7f717806e180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:44.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.576+0000 7f717ce48700 1 -- 172.21.15.78:0/2348113784 shutdown_connections 2023-10-11T09:45:44.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.576+0000 7f717ce48700 1 -- 172.21.15.78:0/2348113784 wait complete. 2023-10-11T09:45:44.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.580+0000 7f717ce48700 1 Processor -- start 2023-10-11T09:45:44.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.580+0000 7f717ce48700 1 -- start start 2023-10-11T09:45:44.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.584+0000 7f717ce48700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71780b1820 0x7f7178140760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:44.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.584+0000 7f717ce48700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71780ca1f0 0x7f717815c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:44.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.584+0000 7f717ce48700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f71781f5430 0x7f71780b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:44.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.584+0000 7f717ce48700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717815a9f0 con 0x7f71780b1820 2023-10-11T09:45:44.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.584+0000 7f717ce48700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7178140ca0 con 0x7f71780ca1f0 2023-10-11T09:45:44.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.584+0000 7f717ce48700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f717815c8e0 con 0x7f71781f5430 2023-10-11T09:45:44.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.584+0000 7f71777fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71780b1820 0x7f7178140760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:44.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.584+0000 7f71777fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71780b1820 0x7f7178140760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50574/0 (socket says 172.21.15.78:50574) 2023-10-11T09:45:44.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.584+0000 7f71777fe700 1 -- 172.21.15.78:0/4243081575 learned_addr learned my addr 172.21.15.78:0/4243081575 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:44.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.584+0000 7f7177fff700 1 --2- 172.21.15.78:0/4243081575 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f71781f5430 0x7f71780b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:44.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.584+0000 7f7176ffd700 1 --2- 172.21.15.78:0/4243081575 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71780ca1f0 0x7f717815c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:44.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.584+0000 7f71777fe700 1 -- 172.21.15.78:0/4243081575 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f71781f5430 msgr2=0x7f71780b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:44.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.584+0000 7f71777fe700 1 --2- 172.21.15.78:0/4243081575 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f71781f5430 0x7f71780b08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:44.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.584+0000 7f71777fe700 1 -- 172.21.15.78:0/4243081575 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71780ca1f0 msgr2=0x7f717815c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:44.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.584+0000 7f71777fe700 1 --2- 172.21.15.78:0/4243081575 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71780ca1f0 0x7f717815c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:44.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.584+0000 7f71777fe700 1 -- 172.21.15.78:0/4243081575 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7170009b30 con 0x7f71780b1820 2023-10-11T09:45:44.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.584+0000 7f7176ffd700 1 --2- 172.21.15.78:0/4243081575 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71780ca1f0 0x7f717815c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:45:44.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.584+0000 7f71777fe700 1 --2- 172.21.15.78:0/4243081575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71780b1820 0x7f7178140760 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f716800c500 tx=0x7f716800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:44.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.584+0000 7f716ffff700 1 -- 172.21.15.78:0/4243081575 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71680052d0 con 0x7f71780b1820 2023-10-11T09:45:44.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.584+0000 7f716ffff700 1 -- 172.21.15.78:0/4243081575 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7168005430 con 0x7f71780b1820 2023-10-11T09:45:44.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.584+0000 7f716ffff700 1 -- 172.21.15.78:0/4243081575 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71680055a0 con 0x7f71780b1820 2023-10-11T09:45:44.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.584+0000 7f717ce48700 1 -- 172.21.15.78:0/4243081575 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7178159db0 con 0x7f71780b1820 2023-10-11T09:45:44.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.584+0000 7f717ce48700 1 -- 172.21.15.78:0/4243081575 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f717815a330 con 0x7f71780b1820 2023-10-11T09:45:44.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.588+0000 7f717ce48700 1 -- 172.21.15.78:0/4243081575 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f717815ab30 con 0x7f71780b1820 2023-10-11T09:45:44.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.588+0000 7f716ffff700 1 -- 172.21.15.78:0/4243081575 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f71680038e0 con 0x7f71780b1820 2023-10-11T09:45:44.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.592+0000 7f716ffff700 1 --2- 172.21.15.78:0/4243081575 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7158065f00 0x7f71580683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:44.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.592+0000 7f716ffff700 1 -- 172.21.15.78:0/4243081575 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f7168082c40 con 0x7f71780b1820 2023-10-11T09:45:44.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.592+0000 7f7176ffd700 1 --2- 172.21.15.78:0/4243081575 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7158065f00 0x7f71580683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:44.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.592+0000 7f7176ffd700 1 --2- 172.21.15.78:0/4243081575 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7158065f00 0x7f71580683b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f71781f90b0 tx=0x7f7160008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:44.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.596+0000 7f716ffff700 1 -- 172.21.15.78:0/4243081575 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f717815ab30 con 0x7f71780b1820 2023-10-11T09:45:44.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.776+0000 7f717ce48700 1 -- 172.21.15.78:0/4243081575 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f717815ab30 con 0x7f71780b1820 2023-10-11T09:45:44.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.776+0000 7f716ffff700 1 -- 172.21.15.78:0/4243081575 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v64) v1 ==== 104+0+11 (secure 0 0 0) 0x7f717815ab30 con 0x7f71780b1820 2023-10-11T09:45:44.784 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:45:44.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.780+0000 7f717ce48700 1 -- 172.21.15.78:0/4243081575 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7158065f00 msgr2=0x7f71580683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:44.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.780+0000 7f717ce48700 1 --2- 172.21.15.78:0/4243081575 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7158065f00 0x7f71580683b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f71781f90b0 tx=0x7f7160008040 comp rx=0 tx=0).stop 2023-10-11T09:45:44.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.780+0000 7f717ce48700 1 -- 172.21.15.78:0/4243081575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71780b1820 msgr2=0x7f7178140760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:44.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.780+0000 7f717ce48700 1 --2- 172.21.15.78:0/4243081575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71780b1820 0x7f7178140760 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f716800c500 tx=0x7f716800c810 comp rx=0 tx=0).stop 2023-10-11T09:45:44.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.780+0000 7f717ce48700 1 -- 172.21.15.78:0/4243081575 shutdown_connections 2023-10-11T09:45:44.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.780+0000 7f717ce48700 1 --2- 172.21.15.78:0/4243081575 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7158065f00 0x7f71580683b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:44.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.780+0000 7f717ce48700 1 --2- 172.21.15.78:0/4243081575 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f71781f5430 0x7f71780b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:44.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.780+0000 7f717ce48700 1 --2- 172.21.15.78:0/4243081575 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71780ca1f0 0x7f717815c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:44.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.780+0000 7f717ce48700 1 --2- 172.21.15.78:0/4243081575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71780b1820 0x7f7178140760 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:44.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.780+0000 7f717ce48700 1 -- 172.21.15.78:0/4243081575 >> 172.21.15.78:0/4243081575 conn(0x7f717806dd80 msgr2=0x7f71780b0070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:44.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.780+0000 7f717ce48700 1 -- 172.21.15.78:0/4243081575 shutdown_connections 2023-10-11T09:45:44.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.780+0000 7f717ce48700 1 -- 172.21.15.78:0/4243081575 wait complete. 2023-10-11T09:45:44.809 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T09:45:44.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.980+0000 7f91a6cbd700 1 Processor -- start 2023-10-11T09:45:44.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.980+0000 7f91a6cbd700 1 -- start start 2023-10-11T09:45:44.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.984+0000 7f91a6cbd700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f91a00b3240 0x7f91a00d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:44.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.984+0000 7f91a6cbd700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f91a00b3610 0x7f91a00b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:44.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.984+0000 7f91a6cbd700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f91a015d040 0x7f91a015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:44.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.984+0000 7f91a6cbd700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91a00b3fd0 con 0x7f91a00b3610 2023-10-11T09:45:44.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.984+0000 7f91a6cbd700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91a00b47f0 con 0x7f91a015d040 2023-10-11T09:45:44.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.984+0000 7f91a6cbd700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91a00b8130 con 0x7f91a00b3240 2023-10-11T09:45:44.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.984+0000 7f91a54ba700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f91a00b3610 0x7f91a00b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:44.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.984+0000 7f91a54ba700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f91a00b3610 0x7f91a00b3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50584/0 (socket says 172.21.15.78:50584) 2023-10-11T09:45:44.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.984+0000 7f91a54ba700 1 -- 172.21.15.78:0/621791464 learned_addr learned my addr 172.21.15.78:0/621791464 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:44.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.984+0000 7f91a5cbb700 1 --2- 172.21.15.78:0/621791464 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f91a00b3240 0x7f91a00d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:44.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.984+0000 7f91a64bc700 1 --2- 172.21.15.78:0/621791464 >> v2:172.21.15.133:3300/0 conn(0x7f91a015d040 0x7f91a015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:44.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.984+0000 7f91a54ba700 1 -- 172.21.15.78:0/621791464 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f91a00b3240 msgr2=0x7f91a00d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:44.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.984+0000 7f91a54ba700 1 --2- 172.21.15.78:0/621791464 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f91a00b3240 0x7f91a00d29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:44.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.984+0000 7f91a54ba700 1 -- 172.21.15.78:0/621791464 >> v2:172.21.15.133:3300/0 conn(0x7f91a015d040 msgr2=0x7f91a015d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:44.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.984+0000 7f91a54ba700 1 --2- 172.21.15.78:0/621791464 >> v2:172.21.15.133:3300/0 conn(0x7f91a015d040 0x7f91a015d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:44.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.984+0000 7f91a54ba700 1 -- 172.21.15.78:0/621791464 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a00b41e0 con 0x7f91a00b3610 2023-10-11T09:45:44.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.984+0000 7f91a54ba700 1 --2- 172.21.15.78:0/621791464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91a00b3610 0x7f91a00b3a90 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f91a00672a0 tx=0x7f9190009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e3641b15f4e28a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:44.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.984+0000 7f91a4cb9700 1 -- 172.21.15.78:0/621791464 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9190011830 con 0x7f91a00b3610 2023-10-11T09:45:44.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.984+0000 7f91a4cb9700 1 -- 172.21.15.78:0/621791464 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9190011990 con 0x7f91a00b3610 2023-10-11T09:45:44.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.984+0000 7f91a4cb9700 1 -- 172.21.15.78:0/621791464 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9190011d20 con 0x7f91a00b3610 2023-10-11T09:45:44.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.984+0000 7f91a6cbd700 1 -- 172.21.15.78:0/621791464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91a00b3610 msgr2=0x7f91a00b3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:44.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.984+0000 7f91a6cbd700 1 --2- 172.21.15.78:0/621791464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91a00b3610 0x7f91a00b3a90 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f91a00672a0 tx=0x7f9190009bf0 comp rx=0 tx=0).stop 2023-10-11T09:45:44.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.984+0000 7f91a6cbd700 1 -- 172.21.15.78:0/621791464 shutdown_connections 2023-10-11T09:45:44.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.984+0000 7f91a6cbd700 1 --2- 172.21.15.78:0/621791464 >> v2:172.21.15.133:3300/0 conn(0x7f91a015d040 0x7f91a015d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:44.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.984+0000 7f91a6cbd700 1 --2- 172.21.15.78:0/621791464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91a00b3610 0x7f91a00b3a90 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:44.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.984+0000 7f91a6cbd700 1 --2- 172.21.15.78:0/621791464 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f91a00b3240 0x7f91a00d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:44.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.984+0000 7f91a6cbd700 1 -- 172.21.15.78:0/621791464 >> 172.21.15.78:0/621791464 conn(0x7f91a009c260 msgr2=0x7f91a00d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:44.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.984+0000 7f91a6cbd700 1 -- 172.21.15.78:0/621791464 shutdown_connections 2023-10-11T09:45:44.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.984+0000 7f91a6cbd700 1 -- 172.21.15.78:0/621791464 wait complete. 2023-10-11T09:45:44.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.984+0000 7f91a6cbd700 1 Processor -- start 2023-10-11T09:45:44.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.984+0000 7f91a6cbd700 1 -- start start 2023-10-11T09:45:44.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.988+0000 7f91a6cbd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91a00b3240 0x7f91a00b24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:44.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.988+0000 7f91a6cbd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91a00b3610 0x7f91a015a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:44.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.988+0000 7f91a6cbd700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f91a015d040 0x7f91a009d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:44.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.988+0000 7f91a6cbd700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a01f1fe0 con 0x7f91a00b3240 2023-10-11T09:45:44.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.988+0000 7f91a6cbd700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a00c8580 con 0x7f91a00b3610 2023-10-11T09:45:44.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.988+0000 7f91a6cbd700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91a00c7970 con 0x7f91a015d040 2023-10-11T09:45:44.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.988+0000 7f91a5cbb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91a00b3240 0x7f91a00b24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:44.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.988+0000 7f91a5cbb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91a00b3240 0x7f91a00b24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50600/0 (socket says 172.21.15.78:50600) 2023-10-11T09:45:44.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.988+0000 7f91a5cbb700 1 -- 172.21.15.78:0/1489051214 learned_addr learned my addr 172.21.15.78:0/1489051214 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:44.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.988+0000 7f91a64bc700 1 --2- 172.21.15.78:0/1489051214 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f91a015d040 0x7f91a009d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:44.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.988+0000 7f91a54ba700 1 --2- 172.21.15.78:0/1489051214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91a00b3610 0x7f91a015a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:44.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.988+0000 7f91a5cbb700 1 -- 172.21.15.78:0/1489051214 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f91a015d040 msgr2=0x7f91a009d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:44.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.988+0000 7f91a5cbb700 1 --2- 172.21.15.78:0/1489051214 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f91a015d040 0x7f91a009d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:44.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.988+0000 7f91a5cbb700 1 -- 172.21.15.78:0/1489051214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91a00b3610 msgr2=0x7f91a015a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:44.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.988+0000 7f91a5cbb700 1 --2- 172.21.15.78:0/1489051214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91a00b3610 0x7f91a015a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:45.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.988+0000 7f91a5cbb700 1 -- 172.21.15.78:0/1489051214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9190009570 con 0x7f91a00b3240 2023-10-11T09:45:45.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.988+0000 7f91a5cbb700 1 --2- 172.21.15.78:0/1489051214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91a00b3240 0x7f91a00b24c0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f919c00e870 tx=0x7f919c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:45.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.988+0000 7f91967fc700 1 -- 172.21.15.78:0/1489051214 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919c009540 con 0x7f91a00b3240 2023-10-11T09:45:45.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.988+0000 7f91967fc700 1 -- 172.21.15.78:0/1489051214 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f919c00c640 con 0x7f91a00b3240 2023-10-11T09:45:45.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.988+0000 7f91967fc700 1 -- 172.21.15.78:0/1489051214 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919c00c9b0 con 0x7f91a00b3240 2023-10-11T09:45:45.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.988+0000 7f91a6cbd700 1 -- 172.21.15.78:0/1489051214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91a00c7c50 con 0x7f91a00b3240 2023-10-11T09:45:45.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.988+0000 7f91a6cbd700 1 -- 172.21.15.78:0/1489051214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91a00c8210 con 0x7f91a00b3240 2023-10-11T09:45:45.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.996+0000 7f91a6cbd700 1 -- 172.21.15.78:0/1489051214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91a0065d40 con 0x7f91a00b3240 2023-10-11T09:45:45.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:44.996+0000 7f91967fc700 1 -- 172.21.15.78:0/1489051214 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f919c00cb10 con 0x7f91a00b3240 2023-10-11T09:45:45.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.000+0000 7f91967fc700 1 --2- 172.21.15.78:0/1489051214 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9184065f00 0x7f91840683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:45.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.000+0000 7f91967fc700 1 -- 172.21.15.78:0/1489051214 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f919c083970 con 0x7f91a00b3240 2023-10-11T09:45:45.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.000+0000 7f91a54ba700 1 --2- 172.21.15.78:0/1489051214 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9184065f00 0x7f91840683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:45.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.000+0000 7f91a54ba700 1 --2- 172.21.15.78:0/1489051214 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9184065f00 0x7f91840683b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f91a00b3a40 tx=0x7f9190008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:45.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.004+0000 7f91967fc700 1 -- 172.21.15.78:0/1489051214 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f919c050d80 con 0x7f91a00b3240 2023-10-11T09:45:45.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.180+0000 7f91a6cbd700 1 -- 172.21.15.78:0/1489051214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f91a00b8e50 con 0x7f91a00b3240 2023-10-11T09:45:45.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.180+0000 7f91967fc700 1 -- 172.21.15.78:0/1489051214 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v64) v1 ==== 100+0+11 (secure 0 0 0) 0x7f919c0170a0 con 0x7f91a00b3240 2023-10-11T09:45:45.185 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2023-10-11T09:45:45.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.184+0000 7f91a6cbd700 1 -- 172.21.15.78:0/1489051214 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9184065f00 msgr2=0x7f91840683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:45.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.184+0000 7f91a6cbd700 1 --2- 172.21.15.78:0/1489051214 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9184065f00 0x7f91840683b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f91a00b3a40 tx=0x7f9190008030 comp rx=0 tx=0).stop 2023-10-11T09:45:45.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.184+0000 7f91a6cbd700 1 -- 172.21.15.78:0/1489051214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91a00b3240 msgr2=0x7f91a00b24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:45.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.184+0000 7f91a6cbd700 1 --2- 172.21.15.78:0/1489051214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91a00b3240 0x7f91a00b24c0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f919c00e870 tx=0x7f919c00ec30 comp rx=0 tx=0).stop 2023-10-11T09:45:45.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.184+0000 7f91a6cbd700 1 -- 172.21.15.78:0/1489051214 shutdown_connections 2023-10-11T09:45:45.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.184+0000 7f91a6cbd700 1 --2- 172.21.15.78:0/1489051214 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9184065f00 0x7f91840683b0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:45.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.184+0000 7f91a6cbd700 1 --2- 172.21.15.78:0/1489051214 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f91a015d040 0x7f91a009d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:45.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.184+0000 7f91a6cbd700 1 --2- 172.21.15.78:0/1489051214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91a00b3610 0x7f91a015a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:45.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.184+0000 7f91a6cbd700 1 --2- 172.21.15.78:0/1489051214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91a00b3240 0x7f91a00b24c0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:45.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.184+0000 7f91a6cbd700 1 -- 172.21.15.78:0/1489051214 >> 172.21.15.78:0/1489051214 conn(0x7f91a009c260 msgr2=0x7f91a00d0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:45.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.184+0000 7f91a6cbd700 1 -- 172.21.15.78:0/1489051214 shutdown_connections 2023-10-11T09:45:45.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.184+0000 7f91a6cbd700 1 -- 172.21.15.78:0/1489051214 wait complete. 2023-10-11T09:45:45.208 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-10-11T09:45:45.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.384+0000 7f523264c700 1 Processor -- start 2023-10-11T09:45:45.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.384+0000 7f523264c700 1 -- start start 2023-10-11T09:45:45.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.388+0000 7f523264c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f522c06d730 0x7f522c165240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:45.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.388+0000 7f523264c700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f522c0b2d30 con 0x7f522c0b3a60 2023-10-11T09:45:45.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.388+0000 7f523264c700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f522c0cf360 con 0x7f522c15a6b0 2023-10-11T09:45:45.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.388+0000 7f523264c700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f522c0b2f80 con 0x7f522c06d730 2023-10-11T09:45:45.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.388+0000 7f5230e49700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f522c0b3a60 0x7f522c06d020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:41798/0 (socket says 172.21.15.78:41798) 2023-10-11T09:45:45.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.388+0000 7f5230e49700 1 -- 172.21.15.78:0/1708028152 learned_addr learned my addr 172.21.15.78:0/1708028152 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:45:45.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.388+0000 7f5231e4b700 1 --2- 172.21.15.78:0/1708028152 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f522c06d730 0x7f522c165240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:45.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.388+0000 7f5223fff700 1 -- 172.21.15.78:0/1708028152 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 812726039 0 0) 0x7f522c0b2d30 con 0x7f522c0b3a60 2023-10-11T09:45:45.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.388+0000 7f5223fff700 1 -- 172.21.15.78:0/1708028152 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52180035d0 con 0x7f522c0b3a60 2023-10-11T09:45:45.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.388+0000 7f5223fff700 1 -- 172.21.15.78:0/1708028152 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 425486311 0 0) 0x7f522c0cf360 con 0x7f522c15a6b0 2023-10-11T09:45:45.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.388+0000 7f5223fff700 1 -- 172.21.15.78:0/1708028152 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f522c0b2d30 con 0x7f522c15a6b0 2023-10-11T09:45:45.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.388+0000 7f5223fff700 1 -- 172.21.15.78:0/1708028152 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2007108568 0 0) 0x7f52180035d0 con 0x7f522c0b3a60 2023-10-11T09:45:45.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.388+0000 7f5223fff700 1 -- 172.21.15.78:0/1708028152 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f522c06d730 msgr2=0x7f522c165240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:45.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.388+0000 7f5223fff700 1 --2- 172.21.15.78:0/1708028152 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f522c06d730 0x7f522c165240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:45.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.388+0000 7f5223fff700 1 -- 172.21.15.78:0/1708028152 >> v1:172.21.15.133:6789/0 conn(0x7f522c15a6b0 legacy=0x7f522c0ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:45.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.388+0000 7f5223fff700 1 -- 172.21.15.78:0/1708028152 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f522c0b3260 con 0x7f522c0b3a60 2023-10-11T09:45:45.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.388+0000 7f5223fff700 1 -- 172.21.15.78:0/1708028152 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f521c001670 con 0x7f522c0b3a60 2023-10-11T09:45:45.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.388+0000 7f5223fff700 1 -- 172.21.15.78:0/1708028152 >> v1:172.21.15.78:6789/0 conn(0x7f522c0b3a60 legacy=0x7f522c06d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:45.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.388+0000 7f5223fff700 1 --2- 172.21.15.78:0/1708028152 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f522c06d730 0x7f5218003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:45.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.388+0000 7f5223fff700 1 --2- 172.21.15.78:0/1708028152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5218004190 0x7f5218004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:45.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.388+0000 7f5223fff700 1 --2- 172.21.15.78:0/1708028152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5218004db0 0x7f5218005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:45.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.388+0000 7f5223fff700 1 -- 172.21.15.78:0/1708028152 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522c0b2f80 con 0x7f5218004db0 2023-10-11T09:45:45.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.388+0000 7f5223fff700 1 -- 172.21.15.78:0/1708028152 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521800ffd0 con 0x7f5218004190 2023-10-11T09:45:45.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.388+0000 7f5223fff700 1 -- 172.21.15.78:0/1708028152 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5218010190 con 0x7f522c06d730 2023-10-11T09:45:45.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.388+0000 7f5231e4b700 1 --2- 172.21.15.78:0/1708028152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5218004db0 0x7f5218005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:45.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.388+0000 7f523164a700 1 --2- 172.21.15.78:0/1708028152 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f522c06d730 0x7f5218003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:45.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.388+0000 7f5230e49700 1 --2- 172.21.15.78:0/1708028152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5218004190 0x7f5218004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:45.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.392+0000 7f523164a700 1 -- 172.21.15.78:0/1708028152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5218004190 msgr2=0x7f5218004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:45.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.392+0000 7f523164a700 1 --2- 172.21.15.78:0/1708028152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5218004190 0x7f5218004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:45.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.392+0000 7f523164a700 1 -- 172.21.15.78:0/1708028152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5218004db0 msgr2=0x7f5218005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:45.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.392+0000 7f523164a700 1 --2- 172.21.15.78:0/1708028152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5218004db0 0x7f5218005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:45.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.392+0000 7f523164a700 1 -- 172.21.15.78:0/1708028152 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52180035d0 con 0x7f522c06d730 2023-10-11T09:45:45.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.392+0000 7f523164a700 1 --2- 172.21.15.78:0/1708028152 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f522c06d730 0x7f5218003be0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5228001960 tx=0x7f52180058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f870fd3e2244008b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:45.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.392+0000 7f5223fff700 1 -- 172.21.15.78:0/1708028152 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5228001450 con 0x7f522c06d730 2023-10-11T09:45:45.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.392+0000 7f5223fff700 1 -- 172.21.15.78:0/1708028152 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52280052f0 con 0x7f522c06d730 2023-10-11T09:45:45.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.392+0000 7f5223fff700 1 -- 172.21.15.78:0/1708028152 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52280036a0 con 0x7f522c06d730 2023-10-11T09:45:45.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.392+0000 7f523264c700 1 -- 172.21.15.78:0/1708028152 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f522c06d730 msgr2=0x7f5218003be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:45.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.392+0000 7f523264c700 1 --2- 172.21.15.78:0/1708028152 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f522c06d730 0x7f5218003be0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5228001960 tx=0x7f52180058a0 comp rx=0 tx=0).stop 2023-10-11T09:45:45.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.392+0000 7f5231e4b700 1 -- 172.21.15.78:0/1708028152 reap_dead start 2023-10-11T09:45:45.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.392+0000 7f523264c700 1 -- 172.21.15.78:0/1708028152 shutdown_connections 2023-10-11T09:45:45.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.392+0000 7f523264c700 1 -- 172.21.15.78:0/1708028152 >> 172.21.15.78:0/1708028152 conn(0x7f522c06a060 msgr2=0x7f522c06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:45.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.392+0000 7f523264c700 1 -- 172.21.15.78:0/1708028152 shutdown_connections 2023-10-11T09:45:45.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.392+0000 7f523264c700 1 -- 172.21.15.78:0/1708028152 wait complete. 2023-10-11T09:45:45.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.396+0000 7f523264c700 1 Processor -- start 2023-10-11T09:45:45.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.396+0000 7f523264c700 1 -- start start 2023-10-11T09:45:45.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.396+0000 7f523264c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f522c06d730 0x7f522c149df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:45.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.396+0000 7f523264c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f522c1479f0 0x7f522c0b0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:45.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.396+0000 7f523264c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f522c0b0d80 0x7f522c1481a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:45.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.396+0000 7f523264c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522c0b11f0 con 0x7f522c1479f0 2023-10-11T09:45:45.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.396+0000 7f523264c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522c158c80 con 0x7f522c0b0d80 2023-10-11T09:45:45.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.396+0000 7f523264c700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f522c159110 con 0x7f522c06d730 2023-10-11T09:45:45.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.396+0000 7f5230e49700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f522c1479f0 0x7f522c0b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:45.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.396+0000 7f5230e49700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f522c1479f0 0x7f522c0b0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50622/0 (socket says 172.21.15.78:50622) 2023-10-11T09:45:45.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.396+0000 7f5230e49700 1 -- 172.21.15.78:0/3289242040 learned_addr learned my addr 172.21.15.78:0/3289242040 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:45.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.396+0000 7f5231e4b700 1 --2- 172.21.15.78:0/3289242040 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f522c0b0d80 0x7f522c1481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:45.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.396+0000 7f523164a700 1 --2- 172.21.15.78:0/3289242040 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f522c06d730 0x7f522c149df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:45.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.396+0000 7f5230e49700 1 -- 172.21.15.78:0/3289242040 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f522c06d730 msgr2=0x7f522c149df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:45.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.396+0000 7f5230e49700 1 --2- 172.21.15.78:0/3289242040 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f522c06d730 0x7f522c149df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:45.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.396+0000 7f5230e49700 1 -- 172.21.15.78:0/3289242040 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f522c0b0d80 msgr2=0x7f522c1481a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:45.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.396+0000 7f5230e49700 1 --2- 172.21.15.78:0/3289242040 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f522c0b0d80 0x7f522c1481a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:45.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.396+0000 7f5230e49700 1 -- 172.21.15.78:0/3289242040 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f522c158070 con 0x7f522c1479f0 2023-10-11T09:45:45.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.400+0000 7f5230e49700 1 --2- 172.21.15.78:0/3289242040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f522c1479f0 0x7f522c0b0840 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f521c004fd0 tx=0x7f521c007340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:45.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.400+0000 7f5221ffb700 1 -- 172.21.15.78:0/3289242040 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521c012560 con 0x7f522c1479f0 2023-10-11T09:45:45.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.400+0000 7f5221ffb700 1 -- 172.21.15.78:0/3289242040 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f521c0126c0 con 0x7f522c1479f0 2023-10-11T09:45:45.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.400+0000 7f5221ffb700 1 -- 172.21.15.78:0/3289242040 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521c012a70 con 0x7f522c1479f0 2023-10-11T09:45:45.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.400+0000 7f523264c700 1 -- 172.21.15.78:0/3289242040 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f522c158350 con 0x7f522c1479f0 2023-10-11T09:45:45.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.400+0000 7f523264c700 1 -- 172.21.15.78:0/3289242040 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f522c158910 con 0x7f522c1479f0 2023-10-11T09:45:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.400+0000 7f523264c700 1 -- 172.21.15.78:0/3289242040 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f522c158ad0 con 0x7f522c1479f0 2023-10-11T09:45:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.404+0000 7f5221ffb700 1 -- 172.21.15.78:0/3289242040 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f521c012bd0 con 0x7f522c1479f0 2023-10-11T09:45:45.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.404+0000 7f5221ffb700 1 --2- 172.21.15.78:0/3289242040 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5210065f20 0x7f52100683d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:45.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.404+0000 7f5221ffb700 1 -- 172.21.15.78:0/3289242040 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f521c083880 con 0x7f522c1479f0 2023-10-11T09:45:45.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.404+0000 7f523164a700 1 --2- 172.21.15.78:0/3289242040 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5210065f20 0x7f52100683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:45.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.408+0000 7f523164a700 1 --2- 172.21.15.78:0/3289242040 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5210065f20 0x7f52100683d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5228015040 tx=0x7f522800a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:45.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.412+0000 7f5221ffb700 1 -- 172.21.15.78:0/3289242040 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f521c050d10 con 0x7f522c1479f0 2023-10-11T09:45:45.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.584+0000 7f523264c700 1 -- 172.21.15.78:0/3289242040 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7f522c0b3520 con 0x7f522c1479f0 2023-10-11T09:45:45.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.584+0000 7f5221ffb700 1 -- 172.21.15.78:0/3289242040 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v64) v1 ==== 106+0+11 (secure 0 0 0) 0x7f522c0b3520 con 0x7f522c1479f0 2023-10-11T09:45:45.590 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:45:45.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.588+0000 7f523264c700 1 -- 172.21.15.78:0/3289242040 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5210065f20 msgr2=0x7f52100683d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:45.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.588+0000 7f523264c700 1 --2- 172.21.15.78:0/3289242040 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5210065f20 0x7f52100683d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5228015040 tx=0x7f522800a000 comp rx=0 tx=0).stop 2023-10-11T09:45:45.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.588+0000 7f523264c700 1 -- 172.21.15.78:0/3289242040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f522c1479f0 msgr2=0x7f522c0b0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:45.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.588+0000 7f523264c700 1 --2- 172.21.15.78:0/3289242040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f522c1479f0 0x7f522c0b0840 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f521c004fd0 tx=0x7f521c007340 comp rx=0 tx=0).stop 2023-10-11T09:45:45.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.588+0000 7f523264c700 1 -- 172.21.15.78:0/3289242040 shutdown_connections 2023-10-11T09:45:45.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.588+0000 7f523264c700 1 --2- 172.21.15.78:0/3289242040 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5210065f20 0x7f52100683d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:45.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.588+0000 7f523264c700 1 --2- 172.21.15.78:0/3289242040 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f522c0b0d80 0x7f522c1481a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:45.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.588+0000 7f523264c700 1 --2- 172.21.15.78:0/3289242040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f522c1479f0 0x7f522c0b0840 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:45.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.588+0000 7f523264c700 1 --2- 172.21.15.78:0/3289242040 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f522c06d730 0x7f522c149df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:45.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.588+0000 7f523264c700 1 -- 172.21.15.78:0/3289242040 >> 172.21.15.78:0/3289242040 conn(0x7f522c06a060 msgr2=0x7f522c06bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:45.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.588+0000 7f523264c700 1 -- 172.21.15.78:0/3289242040 shutdown_connections 2023-10-11T09:45:45.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.588+0000 7f523264c700 1 -- 172.21.15.78:0/3289242040 wait complete. 2023-10-11T09:45:45.615 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T09:45:45.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.772+0000 7fc49bebf700 1 Processor -- start 2023-10-11T09:45:45.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.772+0000 7fc49bebf700 1 -- start start 2023-10-11T09:45:45.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.772+0000 7fc49bebf700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc494160d40 0x7fc4940cd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:45.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.772+0000 7fc49bebf700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc494071040 0x7fc4940cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:45.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.776+0000 7fc49bebf700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc49406ffa0 con 0x7fc4940cd7c0 2023-10-11T09:45:45.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.776+0000 7fc49bebf700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc494070c20 con 0x7fc494160d40 2023-10-11T09:45:45.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.776+0000 7fc49bebf700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc494161110 con 0x7fc494071040 2023-10-11T09:45:45.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.776+0000 7fc49a45c700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fc4940cd7c0 0x7fc4941647f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:41812/0 (socket says 172.21.15.78:41812) 2023-10-11T09:45:45.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.776+0000 7fc49a45c700 1 -- 172.21.15.78:0/1896263222 learned_addr learned my addr 172.21.15.78:0/1896263222 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:45:45.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.776+0000 7fc49945a700 1 --2- 172.21.15.78:0/1896263222 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc494071040 0x7fc4940cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:45.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.776+0000 7fc499c5b700 1 --2- 172.21.15.78:0/1896263222 >> v2:172.21.15.133:3300/0 conn(0x7fc494160d40 0x7fc4940cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:45.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.776+0000 7fc498c59700 1 -- 172.21.15.78:0/1896263222 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2043408832 0 0) 0x7fc49406ffa0 con 0x7fc4940cd7c0 2023-10-11T09:45:45.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.776+0000 7fc498c59700 1 -- 172.21.15.78:0/1896263222 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4800035d0 con 0x7fc4940cd7c0 2023-10-11T09:45:45.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.776+0000 7fc49945a700 1 -- 172.21.15.78:0/1896263222 >> v2:172.21.15.133:3300/0 conn(0x7fc494160d40 msgr2=0x7fc4940cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:45.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.776+0000 7fc49945a700 1 --2- 172.21.15.78:0/1896263222 >> v2:172.21.15.133:3300/0 conn(0x7fc494160d40 0x7fc4940cd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:45.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.776+0000 7fc49945a700 1 -- 172.21.15.78:0/1896263222 >> v1:172.21.15.78:6789/0 conn(0x7fc4940cd7c0 legacy=0x7fc4941647f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:45.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.776+0000 7fc49945a700 1 -- 172.21.15.78:0/1896263222 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc494070280 con 0x7fc494071040 2023-10-11T09:45:45.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.776+0000 7fc49945a700 1 --2- 172.21.15.78:0/1896263222 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc494071040 0x7fc4940cddf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc484009850 tx=0x7fc484009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e6c62f4c1fa67de server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:45.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.776+0000 7fc498c59700 1 -- 172.21.15.78:0/1896263222 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc484011830 con 0x7fc494071040 2023-10-11T09:45:45.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.776+0000 7fc498c59700 1 -- 172.21.15.78:0/1896263222 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc484011990 con 0x7fc494071040 2023-10-11T09:45:45.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.776+0000 7fc498c59700 1 -- 172.21.15.78:0/1896263222 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc484011d00 con 0x7fc494071040 2023-10-11T09:45:45.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.780+0000 7fc49bebf700 1 -- 172.21.15.78:0/1896263222 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc494071040 msgr2=0x7fc4940cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:45.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.780+0000 7fc49bebf700 1 --2- 172.21.15.78:0/1896263222 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc494071040 0x7fc4940cddf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc484009850 tx=0x7fc484009b60 comp rx=0 tx=0).stop 2023-10-11T09:45:45.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.780+0000 7fc49bebf700 1 -- 172.21.15.78:0/1896263222 shutdown_connections 2023-10-11T09:45:45.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.780+0000 7fc49bebf700 1 --2- 172.21.15.78:0/1896263222 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc494071040 0x7fc4940cddf0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:45.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.780+0000 7fc49bebf700 1 --2- 172.21.15.78:0/1896263222 >> v2:172.21.15.133:3300/0 conn(0x7fc494160d40 0x7fc4940cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:45.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.780+0000 7fc49bebf700 1 -- 172.21.15.78:0/1896263222 >> 172.21.15.78:0/1896263222 conn(0x7fc49406aca0 msgr2=0x7fc4940b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:45.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.780+0000 7fc49bebf700 1 -- 172.21.15.78:0/1896263222 shutdown_connections 2023-10-11T09:45:45.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.780+0000 7fc49bebf700 1 -- 172.21.15.78:0/1896263222 wait complete. 2023-10-11T09:45:45.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.784+0000 7fc49bebf700 1 Processor -- start 2023-10-11T09:45:45.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.784+0000 7fc49bebf700 1 -- start start 2023-10-11T09:45:45.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.784+0000 7fc49bebf700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc494071040 0x7fc494150040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:45.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.784+0000 7fc49bebf700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc4940cd7c0 0x7fc4940b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:45.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.784+0000 7fc49bebf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc494160d40 0x7fc4940c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:45.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.784+0000 7fc49bebf700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4940b41f0 con 0x7fc494160d40 2023-10-11T09:45:45.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.784+0000 7fc49bebf700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc494150580 con 0x7fc494071040 2023-10-11T09:45:45.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.784+0000 7fc49bebf700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4940b35e0 con 0x7fc4940cd7c0 2023-10-11T09:45:45.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.784+0000 7fc49a45c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc494160d40 0x7fc4940c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:45.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.784+0000 7fc49a45c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc494160d40 0x7fc4940c6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50634/0 (socket says 172.21.15.78:50634) 2023-10-11T09:45:45.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.784+0000 7fc49a45c700 1 -- 172.21.15.78:0/2250294890 learned_addr learned my addr 172.21.15.78:0/2250294890 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:45.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.784+0000 7fc499c5b700 1 --2- 172.21.15.78:0/2250294890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc494071040 0x7fc494150040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:45.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.784+0000 7fc49945a700 1 --2- 172.21.15.78:0/2250294890 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc4940cd7c0 0x7fc4940b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:45.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.784+0000 7fc49a45c700 1 -- 172.21.15.78:0/2250294890 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc4940cd7c0 msgr2=0x7fc4940b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:45.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.784+0000 7fc49a45c700 1 --2- 172.21.15.78:0/2250294890 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc4940cd7c0 0x7fc4940b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:45.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.784+0000 7fc49a45c700 1 -- 172.21.15.78:0/2250294890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc494071040 msgr2=0x7fc494150040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:45.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.784+0000 7fc49a45c700 1 --2- 172.21.15.78:0/2250294890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc494071040 0x7fc494150040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:45.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.784+0000 7fc49a45c700 1 -- 172.21.15.78:0/2250294890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4840094b0 con 0x7fc494160d40 2023-10-11T09:45:45.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.788+0000 7fc49a45c700 1 --2- 172.21.15.78:0/2250294890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc494160d40 0x7fc4940c6bc0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fc48c00cae0 tx=0x7fc48c00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:45.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.788+0000 7fc48a7fc700 1 -- 172.21.15.78:0/2250294890 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48c007480 con 0x7fc494160d40 2023-10-11T09:45:45.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.788+0000 7fc48a7fc700 1 -- 172.21.15.78:0/2250294890 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc48c007600 con 0x7fc494160d40 2023-10-11T09:45:45.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.788+0000 7fc48a7fc700 1 -- 172.21.15.78:0/2250294890 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48c0079b0 con 0x7fc494160d40 2023-10-11T09:45:45.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.788+0000 7fc49bebf700 1 -- 172.21.15.78:0/2250294890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4940b38c0 con 0x7fc494160d40 2023-10-11T09:45:45.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.788+0000 7fc49bebf700 1 -- 172.21.15.78:0/2250294890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4940b3e80 con 0x7fc494160d40 2023-10-11T09:45:45.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.788+0000 7fc49bebf700 1 -- 172.21.15.78:0/2250294890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc494072250 con 0x7fc494160d40 2023-10-11T09:45:45.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.792+0000 7fc48a7fc700 1 -- 172.21.15.78:0/2250294890 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fc48c01b3b0 con 0x7fc494160d40 2023-10-11T09:45:45.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.792+0000 7fc48a7fc700 1 --2- 172.21.15.78:0/2250294890 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc478065ed0 0x7fc478068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:45.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.792+0000 7fc48a7fc700 1 -- 172.21.15.78:0/2250294890 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6466+0+0 (secure 0 0 0) 0x7fc48c083320 con 0x7fc494160d40 2023-10-11T09:45:45.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.792+0000 7fc499c5b700 1 --2- 172.21.15.78:0/2250294890 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc478065ed0 0x7fc478068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:45.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.792+0000 7fc499c5b700 1 --2- 172.21.15.78:0/2250294890 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc478065ed0 0x7fc478068380 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc4940b09e0 tx=0x7fc490009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:45.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.800+0000 7fc48a7fc700 1 -- 172.21.15.78:0/2250294890 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fc48c050730 con 0x7fc494160d40 2023-10-11T09:45:45.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.984+0000 7fc49bebf700 1 -- 172.21.15.78:0/2250294890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc49404e530 con 0x7fc494160d40 2023-10-11T09:45:45.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.984+0000 7fc48a7fc700 1 -- 172.21.15.78:0/2250294890 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v64) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc48c053d50 con 0x7fc494160d40 2023-10-11T09:45:45.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.988+0000 7fc49bebf700 1 -- 172.21.15.78:0/2250294890 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc478065ed0 msgr2=0x7fc478068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:45.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.988+0000 7fc49bebf700 1 --2- 172.21.15.78:0/2250294890 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc478065ed0 0x7fc478068380 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc4940b09e0 tx=0x7fc490009240 comp rx=0 tx=0).stop 2023-10-11T09:45:45.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.988+0000 7fc49bebf700 1 -- 172.21.15.78:0/2250294890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc494160d40 msgr2=0x7fc4940c6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:45.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.988+0000 7fc49bebf700 1 --2- 172.21.15.78:0/2250294890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc494160d40 0x7fc4940c6bc0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fc48c00cae0 tx=0x7fc48c00cb10 comp rx=0 tx=0).stop 2023-10-11T09:45:45.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.988+0000 7fc49bebf700 1 -- 172.21.15.78:0/2250294890 shutdown_connections 2023-10-11T09:45:45.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.988+0000 7fc49bebf700 1 --2- 172.21.15.78:0/2250294890 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc478065ed0 0x7fc478068380 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:45.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.988+0000 7fc49bebf700 1 --2- 172.21.15.78:0/2250294890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc494160d40 0x7fc4940c6bc0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:45.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.988+0000 7fc49bebf700 1 --2- 172.21.15.78:0/2250294890 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc4940cd7c0 0x7fc4940b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:45.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.988+0000 7fc49bebf700 1 --2- 172.21.15.78:0/2250294890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc494071040 0x7fc494150040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:45.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.988+0000 7fc49bebf700 1 -- 172.21.15.78:0/2250294890 >> 172.21.15.78:0/2250294890 conn(0x7fc49406aca0 msgr2=0x7fc4940cb220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:45.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.988+0000 7fc49bebf700 1 -- 172.21.15.78:0/2250294890 shutdown_connections 2023-10-11T09:45:45.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:45.988+0000 7fc49bebf700 1 -- 172.21.15.78:0/2250294890 wait complete. 2023-10-11T09:45:45.995 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2023-10-11T09:45:46.014 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi078.front.sepia.ceph.com... 2023-10-11T09:45:46.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:45:46.015 DEBUG:teuthology.orchestra.run.smithi078:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-11T09:45:46.042 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:ceph-fuse[49518]: fuse finished with error 0 and tester_r 0 2023-10-11T09:45:46.076 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:45:52.178 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2023-10-11T09:45:52.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:45:52.180 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-11T09:45:52.202 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:45:52.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.412+0000 7f7e0db51700 1 Processor -- start 2023-10-11T09:45:52.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.412+0000 7f7e0db51700 1 -- start start 2023-10-11T09:45:52.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.416+0000 7f7e0db51700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f7e080ca950 0x7f7e081650f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:52.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.416+0000 7f7e0db51700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e0815e010 con 0x7f7e0815a940 2023-10-11T09:45:52.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.416+0000 7f7e0db51700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e0815e280 con 0x7f7e0815b8e0 2023-10-11T09:45:52.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.416+0000 7f7e0db51700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e0815e560 con 0x7f7e080ca950 2023-10-11T09:45:52.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.416+0000 7f7dfffff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f7e0815a940 0x7f7e080ca240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:41816/0 (socket says 172.21.15.78:41816) 2023-10-11T09:45:52.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.416+0000 7f7dfffff700 1 -- 172.21.15.78:0/2766710713 learned_addr learned my addr 172.21.15.78:0/2766710713 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:45:52.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.416+0000 7f7e0d350700 1 --2- 172.21.15.78:0/2766710713 >> v2:172.21.15.78:3300/0 conn(0x7f7e080ca950 0x7f7e081650f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:52.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.416+0000 7f7dff7fe700 1 -- 172.21.15.78:0/2766710713 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3019430300 0 0) 0x7f7e0815e010 con 0x7f7e0815a940 2023-10-11T09:45:52.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.416+0000 7f7e0d350700 1 -- 172.21.15.78:0/2766710713 >> v1:172.21.15.133:6789/0 conn(0x7f7e0815b8e0 legacy=0x7f7e0806d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:52.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.416+0000 7f7e0d350700 1 -- 172.21.15.78:0/2766710713 >> v1:172.21.15.78:6789/0 conn(0x7f7e0815a940 legacy=0x7f7e080ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:52.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.416+0000 7f7e0d350700 1 -- 172.21.15.78:0/2766710713 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e0815e840 con 0x7f7e080ca950 2023-10-11T09:45:52.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.416+0000 7f7e0d350700 1 --2- 172.21.15.78:0/2766710713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e080ca950 0x7f7e081650f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f7e00009d90 tx=0x7f7e0000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=642d129105893fea server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:52.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.416+0000 7f7dff7fe700 1 -- 172.21.15.78:0/2766710713 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0000e070 con 0x7f7e080ca950 2023-10-11T09:45:52.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.416+0000 7f7dff7fe700 1 -- 172.21.15.78:0/2766710713 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e0000bbd0 con 0x7f7e080ca950 2023-10-11T09:45:52.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.416+0000 7f7dff7fe700 1 -- 172.21.15.78:0/2766710713 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e00012430 con 0x7f7e080ca950 2023-10-11T09:45:52.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.420+0000 7f7e0db51700 1 -- 172.21.15.78:0/2766710713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e080ca950 msgr2=0x7f7e081650f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:52.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.420+0000 7f7e0db51700 1 --2- 172.21.15.78:0/2766710713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e080ca950 0x7f7e081650f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f7e00009d90 tx=0x7f7e0000b330 comp rx=0 tx=0).stop 2023-10-11T09:45:52.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.420+0000 7f7e0db51700 1 -- 172.21.15.78:0/2766710713 shutdown_connections 2023-10-11T09:45:52.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.420+0000 7f7e0db51700 1 --2- 172.21.15.78:0/2766710713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e080ca950 0x7f7e081650f0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:52.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.420+0000 7f7e0db51700 1 -- 172.21.15.78:0/2766710713 >> 172.21.15.78:0/2766710713 conn(0x7f7e080cea60 msgr2=0x7f7e080cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:52.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.420+0000 7f7e0db51700 1 -- 172.21.15.78:0/2766710713 shutdown_connections 2023-10-11T09:45:52.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.420+0000 7f7e0db51700 1 -- 172.21.15.78:0/2766710713 wait complete. 2023-10-11T09:45:52.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.420+0000 7f7e0db51700 1 Processor -- start 2023-10-11T09:45:52.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.420+0000 7f7e0db51700 1 -- start start 2023-10-11T09:45:52.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.424+0000 7f7e0db51700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e080ca950 0x7f7e081f18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:52.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.424+0000 7f7e0db51700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e0815a940 0x7f7e0813f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:52.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.424+0000 7f7e0db51700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7e0815b8e0 0x7f7e080c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:52.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.424+0000 7f7e0db51700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0813bd00 con 0x7f7e080ca950 2023-10-11T09:45:52.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.424+0000 7f7e0db51700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e080b2a60 con 0x7f7e0815a940 2023-10-11T09:45:52.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.424+0000 7f7e0db51700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e080b2cb0 con 0x7f7e0815b8e0 2023-10-11T09:45:52.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.424+0000 7f7e0cb4f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e080ca950 0x7f7e081f18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:52.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.424+0000 7f7e0cb4f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e080ca950 0x7f7e081f18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50650/0 (socket says 172.21.15.78:50650) 2023-10-11T09:45:52.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.424+0000 7f7e0cb4f700 1 -- 172.21.15.78:0/3638683020 learned_addr learned my addr 172.21.15.78:0/3638683020 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:52.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.424+0000 7f7dfffff700 1 --2- 172.21.15.78:0/3638683020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e0815a940 0x7f7e0813f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:52.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.424+0000 7f7e0d350700 1 --2- 172.21.15.78:0/3638683020 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7e0815b8e0 0x7f7e080c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:52.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.424+0000 7f7e0cb4f700 1 -- 172.21.15.78:0/3638683020 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7e0815b8e0 msgr2=0x7f7e080c7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:52.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.424+0000 7f7e0cb4f700 1 --2- 172.21.15.78:0/3638683020 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7e0815b8e0 0x7f7e080c7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:52.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.424+0000 7f7e0cb4f700 1 -- 172.21.15.78:0/3638683020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e0815a940 msgr2=0x7f7e0813f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:52.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.424+0000 7f7e0cb4f700 1 --2- 172.21.15.78:0/3638683020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e0815a940 0x7f7e0813f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:52.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.424+0000 7f7e0cb4f700 1 -- 172.21.15.78:0/3638683020 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e00009a70 con 0x7f7e080ca950 2023-10-11T09:45:52.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.424+0000 7f7e0cb4f700 1 --2- 172.21.15.78:0/3638683020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e080ca950 0x7f7e081f18e0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f7e0400c500 tx=0x7f7e0400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:52.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.424+0000 7f7dfd7fa700 1 -- 172.21.15.78:0/3638683020 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e040052d0 con 0x7f7e080ca950 2023-10-11T09:45:52.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.424+0000 7f7dfd7fa700 1 -- 172.21.15.78:0/3638683020 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e04005430 con 0x7f7e080ca950 2023-10-11T09:45:52.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.424+0000 7f7e0db51700 1 -- 172.21.15.78:0/3638683020 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e080b2f90 con 0x7f7e080ca950 2023-10-11T09:45:52.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.424+0000 7f7dfd7fa700 1 -- 172.21.15.78:0/3638683020 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e040055a0 con 0x7f7e080ca950 2023-10-11T09:45:52.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.424+0000 7f7e0db51700 1 -- 172.21.15.78:0/3638683020 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e080b3550 con 0x7f7e080ca950 2023-10-11T09:45:52.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.428+0000 7f7e0db51700 1 -- 172.21.15.78:0/3638683020 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e08065d40 con 0x7f7e080ca950 2023-10-11T09:45:52.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.428+0000 7f7dfd7fa700 1 -- 172.21.15.78:0/3638683020 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f7e040038e0 con 0x7f7e080ca950 2023-10-11T09:45:52.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.428+0000 7f7dfd7fa700 1 --2- 172.21.15.78:0/3638683020 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7df4065f00 0x7f7df40683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:52.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.428+0000 7f7dfd7fa700 1 -- 172.21.15.78:0/3638683020 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f7e04082e80 con 0x7f7e080ca950 2023-10-11T09:45:52.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.432+0000 7f7dfffff700 1 --2- 172.21.15.78:0/3638683020 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7df4065f00 0x7f7df40683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:52.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.432+0000 7f7dfffff700 1 --2- 172.21.15.78:0/3638683020 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7df4065f00 0x7f7df40683b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7e080cc480 tx=0x7f7df8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:52.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.436+0000 7f7dfd7fa700 1 -- 172.21.15.78:0/3638683020 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f7e04050320 con 0x7f7e080ca950 2023-10-11T09:45:52.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.648+0000 7f7e0db51700 1 -- 172.21.15.78:0/3638683020 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e0815b1e0 con 0x7f7e080ca950 2023-10-11T09:45:52.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.652+0000 7f7dfd7fa700 1 -- 172.21.15.78:0/3638683020 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4654 (secure 0 0 0) 0x7f7e04053940 con 0x7f7e080ca950 2023-10-11T09:45:52.656 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:52.656 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":38,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5579,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3954348479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":3954348479},{"type":"v1","addr":"172.21.15.133:6835","nonce":3954348479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6100,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/4029676589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4029676589},{"type":"v1","addr":"172.21.15.133:6834","nonce":4029676589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6249,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/1819646091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":1819646091},{"type":"v1","addr":"172.21.15.78:6836","nonce":1819646091}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:44:06.850489+0000","modified":"2023-10-11T09:44:16.715978+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6252},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6252":{"gid":6252,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6837/852594750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":852594750},{"type":"v1","addr":"172.21.15.78:6837","nonce":852594750}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9,10],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-11T09:45:52.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.656+0000 7f7e0db51700 1 -- 172.21.15.78:0/3638683020 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7df4065f00 msgr2=0x7f7df40683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:52.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.656+0000 7f7e0db51700 1 --2- 172.21.15.78:0/3638683020 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7df4065f00 0x7f7df40683b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7e080cc480 tx=0x7f7df8009040 comp rx=0 tx=0).stop 2023-10-11T09:45:52.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.656+0000 7f7e0db51700 1 -- 172.21.15.78:0/3638683020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e080ca950 msgr2=0x7f7e081f18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:52.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.656+0000 7f7e0db51700 1 --2- 172.21.15.78:0/3638683020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e080ca950 0x7f7e081f18e0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f7e0400c500 tx=0x7f7e0400c8c0 comp rx=0 tx=0).stop 2023-10-11T09:45:52.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.656+0000 7f7e0db51700 1 -- 172.21.15.78:0/3638683020 shutdown_connections 2023-10-11T09:45:52.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.656+0000 7f7e0db51700 1 --2- 172.21.15.78:0/3638683020 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7df4065f00 0x7f7df40683b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:52.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.656+0000 7f7e0db51700 1 --2- 172.21.15.78:0/3638683020 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7e0815b8e0 0x7f7e080c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:52.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.656+0000 7f7e0db51700 1 --2- 172.21.15.78:0/3638683020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e0815a940 0x7f7e0813f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:52.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.656+0000 7f7e0db51700 1 --2- 172.21.15.78:0/3638683020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e080ca950 0x7f7e081f18e0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:52.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.656+0000 7f7e0db51700 1 -- 172.21.15.78:0/3638683020 >> 172.21.15.78:0/3638683020 conn(0x7f7e080cea60 msgr2=0x7f7e08070c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:52.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.656+0000 7f7e0db51700 1 -- 172.21.15.78:0/3638683020 shutdown_connections 2023-10-11T09:45:52.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:52.656+0000 7f7e0db51700 1 -- 172.21.15.78:0/3638683020 wait complete. 2023-10-11T09:45:52.662 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 38 2023-10-11T09:45:52.680 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2023-10-11T09:45:52.891 INFO:teuthology.orchestra.run.smithi078.stdout:{"message":"","return_code":0} 2023-10-11T09:45:52.892 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-10-11T09:45:52.892 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-11T09:45:53.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.056+0000 7f0e3560f700 1 Processor -- start 2023-10-11T09:45:53.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.056+0000 7f0e3560f700 1 -- start start 2023-10-11T09:45:53.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.060+0000 7f0e3560f700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e300d1660 0x7f0e300d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:53.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.060+0000 7f0e3560f700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0e300d28e0 0x7f0e300d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:53.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.060+0000 7f0e3560f700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e3015dc10 con 0x7f0e30156ce0 2023-10-11T09:45:53.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.060+0000 7f0e3560f700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e30160380 con 0x7f0e300d28e0 2023-10-11T09:45:53.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.060+0000 7f0e3560f700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3015d930 con 0x7f0e300d1660 2023-10-11T09:45:53.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.060+0000 7f0e34e0e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0e300d28e0 0x7f0e300d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:53.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.060+0000 7f0e34e0e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0e300d28e0 0x7f0e300d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50652/0 (socket says 172.21.15.78:50652) 2023-10-11T09:45:53.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.060+0000 7f0e34e0e700 1 -- 172.21.15.78:0/4108313639 learned_addr learned my addr 172.21.15.78:0/4108313639 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:53.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.060+0000 7f0e2f7fe700 1 --2- 172.21.15.78:0/4108313639 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e300d1660 0x7f0e300d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:53.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.060+0000 7f0e34e0e700 1 -- 172.21.15.78:0/4108313639 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e300d1660 msgr2=0x7f0e300d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:53.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.060+0000 7f0e34e0e700 1 --2- 172.21.15.78:0/4108313639 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e300d1660 0x7f0e300d0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:53.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.060+0000 7f0e34e0e700 1 -- 172.21.15.78:0/4108313639 >> v1:172.21.15.78:6789/0 conn(0x7f0e30156ce0 legacy=0x7f0e30160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:53.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.060+0000 7f0e34e0e700 1 -- 172.21.15.78:0/4108313639 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e30160620 con 0x7f0e300d28e0 2023-10-11T09:45:53.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.060+0000 7f0e2f7fe700 1 --2- 172.21.15.78:0/4108313639 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e300d1660 0x7f0e300d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:45:53.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.060+0000 7f0e34e0e700 1 --2- 172.21.15.78:0/4108313639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e300d28e0 0x7f0e300d0f90 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f0e28009e50 tx=0x7f0e2800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41cf2ef40aebb8a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:53.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.060+0000 7f0e2effd700 1 -- 172.21.15.78:0/4108313639 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2800e070 con 0x7f0e300d28e0 2023-10-11T09:45:53.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.060+0000 7f0e2effd700 1 -- 172.21.15.78:0/4108313639 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2800b9c0 con 0x7f0e300d28e0 2023-10-11T09:45:53.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.060+0000 7f0e2effd700 1 -- 172.21.15.78:0/4108313639 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2800bd70 con 0x7f0e300d28e0 2023-10-11T09:45:53.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.060+0000 7f0e3560f700 1 -- 172.21.15.78:0/4108313639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e300d28e0 msgr2=0x7f0e300d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:53.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.060+0000 7f0e3560f700 1 --2- 172.21.15.78:0/4108313639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e300d28e0 0x7f0e300d0f90 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f0e28009e50 tx=0x7f0e2800b330 comp rx=0 tx=0).stop 2023-10-11T09:45:53.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.064+0000 7f0e3560f700 1 -- 172.21.15.78:0/4108313639 shutdown_connections 2023-10-11T09:45:53.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.064+0000 7f0e3560f700 1 --2- 172.21.15.78:0/4108313639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e300d28e0 0x7f0e300d0f90 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:53.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.064+0000 7f0e3560f700 1 --2- 172.21.15.78:0/4108313639 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e300d1660 0x7f0e300d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:53.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.064+0000 7f0e3560f700 1 -- 172.21.15.78:0/4108313639 >> 172.21.15.78:0/4108313639 conn(0x7f0e3006afb0 msgr2=0x7f0e3006b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:53.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.064+0000 7f0e3560f700 1 -- 172.21.15.78:0/4108313639 shutdown_connections 2023-10-11T09:45:53.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.064+0000 7f0e3560f700 1 -- 172.21.15.78:0/4108313639 wait complete. 2023-10-11T09:45:53.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.064+0000 7f0e3560f700 1 Processor -- start 2023-10-11T09:45:53.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.068+0000 7f0e3560f700 1 -- start start 2023-10-11T09:45:53.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.068+0000 7f0e3560f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e300d1660 0x7f0e300ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:53.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.068+0000 7f0e3560f700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e300d28e0 0x7f0e301f47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:53.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.068+0000 7f0e3560f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e30156ce0 0x7f0e3015f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:53.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.068+0000 7f0e3560f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e300cafa0 con 0x7f0e300d1660 2023-10-11T09:45:53.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.068+0000 7f0e3560f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e30143fc0 con 0x7f0e30156ce0 2023-10-11T09:45:53.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.068+0000 7f0e3560f700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e300cd220 con 0x7f0e300d28e0 2023-10-11T09:45:53.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.068+0000 7f0e2ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e300d1660 0x7f0e300ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:53.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.068+0000 7f0e2ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e300d1660 0x7f0e300ca6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50666/0 (socket says 172.21.15.78:50666) 2023-10-11T09:45:53.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.068+0000 7f0e2ffff700 1 -- 172.21.15.78:0/1209432480 learned_addr learned my addr 172.21.15.78:0/1209432480 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:53.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.068+0000 7f0e34e0e700 1 --2- 172.21.15.78:0/1209432480 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e30156ce0 0x7f0e3015f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:53.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.068+0000 7f0e2f7fe700 1 --2- 172.21.15.78:0/1209432480 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e300d28e0 0x7f0e301f47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:53.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.068+0000 7f0e2ffff700 1 -- 172.21.15.78:0/1209432480 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e300d28e0 msgr2=0x7f0e301f47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:53.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.068+0000 7f0e2ffff700 1 --2- 172.21.15.78:0/1209432480 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e300d28e0 0x7f0e301f47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:53.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.068+0000 7f0e2ffff700 1 -- 172.21.15.78:0/1209432480 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e30156ce0 msgr2=0x7f0e3015f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:53.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.068+0000 7f0e2ffff700 1 --2- 172.21.15.78:0/1209432480 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e30156ce0 0x7f0e3015f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:53.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.068+0000 7f0e2ffff700 1 -- 172.21.15.78:0/1209432480 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e28009b30 con 0x7f0e300d1660 2023-10-11T09:45:53.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.068+0000 7f0e2f7fe700 1 --2- 172.21.15.78:0/1209432480 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e300d28e0 0x7f0e301f47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:45:53.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.072+0000 7f0e2ffff700 1 --2- 172.21.15.78:0/1209432480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e300d1660 0x7f0e300ca6b0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f0e2000c500 tx=0x7f0e2000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:53.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.072+0000 7f0e2cff9700 1 -- 172.21.15.78:0/1209432480 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e200052d0 con 0x7f0e300d1660 2023-10-11T09:45:53.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.072+0000 7f0e2cff9700 1 -- 172.21.15.78:0/1209432480 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e20005430 con 0x7f0e300d1660 2023-10-11T09:45:53.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.072+0000 7f0e3560f700 1 -- 172.21.15.78:0/1209432480 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e300ca0f0 con 0x7f0e300d1660 2023-10-11T09:45:53.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.072+0000 7f0e2cff9700 1 -- 172.21.15.78:0/1209432480 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e200055a0 con 0x7f0e300d1660 2023-10-11T09:45:53.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.072+0000 7f0e3560f700 1 -- 172.21.15.78:0/1209432480 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e300cabf0 con 0x7f0e300d1660 2023-10-11T09:45:53.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.072+0000 7f0e3560f700 1 -- 172.21.15.78:0/1209432480 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e30144180 con 0x7f0e300d1660 2023-10-11T09:45:53.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.076+0000 7f0e2cff9700 1 -- 172.21.15.78:0/1209432480 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f0e200038e0 con 0x7f0e300d1660 2023-10-11T09:45:53.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.076+0000 7f0e2cff9700 1 --2- 172.21.15.78:0/1209432480 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0e1c065ed0 0x7f0e1c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:53.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.076+0000 7f0e2cff9700 1 -- 172.21.15.78:0/1209432480 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f0e20082d70 con 0x7f0e300d1660 2023-10-11T09:45:53.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.076+0000 7f0e2f7fe700 1 --2- 172.21.15.78:0/1209432480 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0e1c065ed0 0x7f0e1c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:53.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.076+0000 7f0e2f7fe700 1 --2- 172.21.15.78:0/1209432480 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0e1c065ed0 0x7f0e1c068380 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0e30154260 tx=0x7f0e24008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:53.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.084+0000 7f0e2cff9700 1 -- 172.21.15.78:0/1209432480 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f0e20050210 con 0x7f0e300d1660 2023-10-11T09:45:53.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.296+0000 7f0e3560f700 1 -- 172.21.15.78:0/1209432480 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0e30065d40 con 0x7f0e300d1660 2023-10-11T09:45:53.303 INFO:tasks.ceph.mon.a.smithi078.stderr:2023-10-11T09:45:53.300+0000 7f90131f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-11T09:45:53.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.316+0000 7f0e2cff9700 1 -- 172.21.15.78:0/1209432480 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v39) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0e20053830 con 0x7f0e300d1660 2023-10-11T09:45:53.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.320+0000 7f0e3560f700 1 -- 172.21.15.78:0/1209432480 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0e1c065ed0 msgr2=0x7f0e1c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:53.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.320+0000 7f0e3560f700 1 --2- 172.21.15.78:0/1209432480 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0e1c065ed0 0x7f0e1c068380 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0e30154260 tx=0x7f0e24008040 comp rx=0 tx=0).stop 2023-10-11T09:45:53.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.320+0000 7f0e3560f700 1 -- 172.21.15.78:0/1209432480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e300d1660 msgr2=0x7f0e300ca6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:53.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.320+0000 7f0e3560f700 1 --2- 172.21.15.78:0/1209432480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e300d1660 0x7f0e300ca6b0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f0e2000c500 tx=0x7f0e2000c810 comp rx=0 tx=0).stop 2023-10-11T09:45:53.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.320+0000 7f0e3560f700 1 -- 172.21.15.78:0/1209432480 shutdown_connections 2023-10-11T09:45:53.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.320+0000 7f0e3560f700 1 --2- 172.21.15.78:0/1209432480 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0e1c065ed0 0x7f0e1c068380 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:53.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.320+0000 7f0e3560f700 1 --2- 172.21.15.78:0/1209432480 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e30156ce0 0x7f0e3015f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:53.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.320+0000 7f0e3560f700 1 --2- 172.21.15.78:0/1209432480 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e300d28e0 0x7f0e301f47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:53.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.320+0000 7f0e3560f700 1 --2- 172.21.15.78:0/1209432480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e300d1660 0x7f0e300ca6b0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:53.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.320+0000 7f0e3560f700 1 -- 172.21.15.78:0/1209432480 >> 172.21.15.78:0/1209432480 conn(0x7f0e3006afb0 msgr2=0x7f0e30155670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:53.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.320+0000 7f0e3560f700 1 -- 172.21.15.78:0/1209432480 shutdown_connections 2023-10-11T09:45:53.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.320+0000 7f0e3560f700 1 -- 172.21.15.78:0/1209432480 wait complete. 2023-10-11T09:45:53.327 INFO:teuthology.orchestra.run.smithi078.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-11T09:45:53.346 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-11T09:45:53.351 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-11T09:45:53.394 INFO:tasks.ceph.mds.c.smithi078.stdout:starting mds.c at 2023-10-11T09:45:53.411 INFO:teuthology.orchestra.run.smithi078.stdout:[global] 2023-10-11T09:45:53.411 INFO:teuthology.orchestra.run.smithi078.stdout: chdir = "" 2023-10-11T09:45:53.411 INFO:teuthology.orchestra.run.smithi078.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-11T09:45:53.411 INFO:teuthology.orchestra.run.smithi078.stdout: auth supported = cephx 2023-10-11T09:45:53.412 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.412 INFO:teuthology.orchestra.run.smithi078.stdout: filestore xattr use omap = true 2023-10-11T09:45:53.412 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.412 INFO:teuthology.orchestra.run.smithi078.stdout: mon clock drift allowed = 1.000 2023-10-11T09:45:53.412 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.412 INFO:teuthology.orchestra.run.smithi078.stdout: osd crush chooseleaf type = 0 2023-10-11T09:45:53.413 INFO:teuthology.orchestra.run.smithi078.stdout: auth debug = true 2023-10-11T09:45:53.413 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.413 INFO:teuthology.orchestra.run.smithi078.stdout: ms die on old message = true 2023-10-11T09:45:53.413 INFO:teuthology.orchestra.run.smithi078.stdout: ms die on bug = true 2023-10-11T09:45:53.413 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.414 INFO:teuthology.orchestra.run.smithi078.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-11T09:45:53.414 INFO:teuthology.orchestra.run.smithi078.stdout: mon pg warn max object skew = 0 2023-10-11T09:45:53.414 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.414 INFO:teuthology.orchestra.run.smithi078.stdout: # disable pg_autoscaler by default for new pools 2023-10-11T09:45:53.414 INFO:teuthology.orchestra.run.smithi078.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-11T09:45:53.415 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.415 INFO:teuthology.orchestra.run.smithi078.stdout: osd pool default size = 2 2023-10-11T09:45:53.415 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.415 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd allow primary affinity = true 2023-10-11T09:45:53.415 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd allow pg remap = true 2023-10-11T09:45:53.416 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on legacy crush tunables = false 2023-10-11T09:45:53.416 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on crush straw calc version zero = false 2023-10-11T09:45:53.416 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on no sortbitwise = false 2023-10-11T09:45:53.416 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on osd down out interval zero = false 2023-10-11T09:45:53.416 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on too few osds = false 2023-10-11T09:45:53.417 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-11T09:45:53.417 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-11T09:45:53.417 INFO:teuthology.orchestra.run.smithi078.stdout: mon_allow_pool_size_one = true 2023-10-11T09:45:53.417 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.417 INFO:teuthology.orchestra.run.smithi078.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-11T09:45:53.418 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.418 INFO:teuthology.orchestra.run.smithi078.stdout: osd default data pool replay window = 5 2023-10-11T09:45:53.418 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.419 INFO:teuthology.orchestra.run.smithi078.stdout: mon allow pool delete = true 2023-10-11T09:45:53.419 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.419 INFO:teuthology.orchestra.run.smithi078.stdout: mon cluster log file level = debug 2023-10-11T09:45:53.419 INFO:teuthology.orchestra.run.smithi078.stdout: debug asserts on shutdown = true 2023-10-11T09:45:53.419 INFO:teuthology.orchestra.run.smithi078.stdout: mon health detail to clog = false 2023-10-11T09:45:53.420 INFO:teuthology.orchestra.run.smithi078.stdout: mon host = "172.21.15.78,172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790]" 2023-10-11T09:45:53.420 INFO:teuthology.orchestra.run.smithi078.stdout: fsid = dee99db9-7a72-406f-a1e2-917943f2b13d 2023-10-11T09:45:53.420 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.420 INFO:teuthology.orchestra.run.smithi078.stdout:[osd] 2023-10-11T09:45:53.420 INFO:teuthology.orchestra.run.smithi078.stdout: osd journal size = 100 2023-10-11T09:45:53.421 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.421 INFO:teuthology.orchestra.run.smithi078.stdout: osd scrub load threshold = 5.0 2023-10-11T09:45:53.421 INFO:teuthology.orchestra.run.smithi078.stdout: osd scrub max interval = 600 2023-10-11T09:45:53.421 INFO:teuthology.orchestra.run.smithi078.stdout: osd mclock profile = high_recovery_ops 2023-10-11T09:45:53.421 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.422 INFO:teuthology.orchestra.run.smithi078.stdout: osd recover clone overlap = true 2023-10-11T09:45:53.422 INFO:teuthology.orchestra.run.smithi078.stdout: osd recovery max chunk = 1048576 2023-10-11T09:45:53.422 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.422 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug shutdown = true 2023-10-11T09:45:53.422 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug op order = true 2023-10-11T09:45:53.423 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug verify stray on activate = true 2023-10-11T09:45:53.423 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.423 INFO:teuthology.orchestra.run.smithi078.stdout: osd open classes on start = true 2023-10-11T09:45:53.423 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug pg log writeout = true 2023-10-11T09:45:53.423 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.424 INFO:teuthology.orchestra.run.smithi078.stdout: osd deep scrub update digest min age = 30 2023-10-11T09:45:53.424 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.424 INFO:teuthology.orchestra.run.smithi078.stdout: osd map max advance = 10 2023-10-11T09:45:53.424 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.424 INFO:teuthology.orchestra.run.smithi078.stdout: journal zero on create = true 2023-10-11T09:45:53.424 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.425 INFO:teuthology.orchestra.run.smithi078.stdout: filestore ondisk finisher threads = 3 2023-10-11T09:45:53.425 INFO:teuthology.orchestra.run.smithi078.stdout: filestore apply finisher threads = 3 2023-10-11T09:45:53.425 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.425 INFO:teuthology.orchestra.run.smithi078.stdout: bdev debug aio = true 2023-10-11T09:45:53.426 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug misdirected ops = true 2023-10-11T09:45:53.426 INFO:teuthology.orchestra.run.smithi078.stdout: bdev async discard = True 2023-10-11T09:45:53.426 INFO:teuthology.orchestra.run.smithi078.stdout: bdev enable discard = True 2023-10-11T09:45:53.426 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore allocator = bitmap 2023-10-11T09:45:53.426 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore block size = 96636764160 2023-10-11T09:45:53.427 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore fsck on mount = True 2023-10-11T09:45:53.427 INFO:teuthology.orchestra.run.smithi078.stdout: debug bluefs = 1/20 2023-10-11T09:45:53.427 INFO:teuthology.orchestra.run.smithi078.stdout: debug bluestore = 1/20 2023-10-11T09:45:53.427 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:45:53.427 INFO:teuthology.orchestra.run.smithi078.stdout: debug osd = 20 2023-10-11T09:45:53.428 INFO:teuthology.orchestra.run.smithi078.stdout: debug rocksdb = 4/10 2023-10-11T09:45:53.428 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd backfillfull_ratio = 0.85 2023-10-11T09:45:53.428 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd full ratio = 0.9 2023-10-11T09:45:53.428 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd nearfull ratio = 0.8 2023-10-11T09:45:53.428 INFO:teuthology.orchestra.run.smithi078.stdout: osd failsafe full ratio = 0.95 2023-10-11T09:45:53.429 INFO:teuthology.orchestra.run.smithi078.stdout: osd objectstore = bluestore 2023-10-11T09:45:53.429 INFO:teuthology.orchestra.run.smithi078.stdout: osd op complaint time = 180 2023-10-11T09:45:53.429 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.429 INFO:teuthology.orchestra.run.smithi078.stdout:[mgr] 2023-10-11T09:45:53.429 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:45:53.430 INFO:teuthology.orchestra.run.smithi078.stdout: debug mgr = 20 2023-10-11T09:45:53.430 INFO:teuthology.orchestra.run.smithi078.stdout: debug mon = 20 2023-10-11T09:45:53.430 INFO:teuthology.orchestra.run.smithi078.stdout: debug auth = 20 2023-10-11T09:45:53.430 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:45:53.431 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:45:53.431 INFO:teuthology.orchestra.run.smithi078.stdout: mgr/telemetry/nag = false 2023-10-11T09:45:53.432 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.432 INFO:teuthology.orchestra.run.smithi078.stdout:[mon] 2023-10-11T09:45:53.432 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:45:53.432 INFO:teuthology.orchestra.run.smithi078.stdout: debug mon = 20 2023-10-11T09:45:53.432 INFO:teuthology.orchestra.run.smithi078.stdout: debug paxos = 20 2023-10-11T09:45:53.433 INFO:teuthology.orchestra.run.smithi078.stdout: debug auth = 20 2023-10-11T09:45:53.433 INFO:teuthology.orchestra.run.smithi078.stdout: mon data avail warn = 5 2023-10-11T09:45:53.433 INFO:teuthology.orchestra.run.smithi078.stdout: mon mgr mkfs grace = 240 2023-10-11T09:45:53.433 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:45:53.433 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd reporter subtree level = osd 2023-10-11T09:45:53.433 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd prime pg temp = true 2023-10-11T09:45:53.434 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:45:53.434 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.434 INFO:teuthology.orchestra.run.smithi078.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-11T09:45:53.434 INFO:teuthology.orchestra.run.smithi078.stdout: auth mon ticket ttl = 660 # 11m 2023-10-11T09:45:53.434 INFO:teuthology.orchestra.run.smithi078.stdout: auth service ticket ttl = 240 # 4m 2023-10-11T09:45:53.435 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.435 INFO:teuthology.orchestra.run.smithi078.stdout: # don't complain about insecure global_id in the test suite 2023-10-11T09:45:53.435 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-11T09:45:53.435 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-11T09:45:53.436 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.436 INFO:teuthology.orchestra.run.smithi078.stdout: # 1m isn't quite enough 2023-10-11T09:45:53.436 INFO:teuthology.orchestra.run.smithi078.stdout: mon_down_mkfs_grace = 2m 2023-10-11T09:45:53.436 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.436 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_filestore_osds = false 2023-10-11T09:45:53.437 INFO:teuthology.orchestra.run.smithi078.stdout: mon op complaint time = 120 2023-10-11T09:45:53.437 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.437 INFO:teuthology.orchestra.run.smithi078.stdout:[client] 2023-10-11T09:45:53.437 INFO:teuthology.orchestra.run.smithi078.stdout: rgw cache enabled = true 2023-10-11T09:45:53.437 INFO:teuthology.orchestra.run.smithi078.stdout: rgw enable ops log = true 2023-10-11T09:45:53.438 INFO:teuthology.orchestra.run.smithi078.stdout: rgw enable usage log = true 2023-10-11T09:45:53.438 INFO:teuthology.orchestra.run.smithi078.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-11T09:45:53.438 INFO:teuthology.orchestra.run.smithi078.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-11T09:45:53.438 INFO:teuthology.orchestra.run.smithi078.stdout: client mount timeout = 600 2023-10-11T09:45:53.438 INFO:teuthology.orchestra.run.smithi078.stdout: debug client = 20 2023-10-11T09:45:53.439 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:45:53.439 INFO:teuthology.orchestra.run.smithi078.stdout: pid file = "" 2023-10-11T09:45:53.439 INFO:teuthology.orchestra.run.smithi078.stdout: rados mon op timeout = 900 2023-10-11T09:45:53.439 INFO:teuthology.orchestra.run.smithi078.stdout: rados osd op timeout = 900 2023-10-11T09:45:53.439 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.a] 2023-10-11T09:45:53.440 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.b] 2023-10-11T09:45:53.440 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.c] 2023-10-11T09:45:53.440 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.a] 2023-10-11T09:45:53.440 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.c] 2023-10-11T09:45:53.440 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.b] 2023-10-11T09:45:53.441 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.d] 2023-10-11T09:45:53.441 INFO:teuthology.orchestra.run.smithi078.stdout:[mds] 2023-10-11T09:45:53.441 INFO:teuthology.orchestra.run.smithi078.stdout: debug mds = 20 2023-10-11T09:45:53.441 INFO:teuthology.orchestra.run.smithi078.stdout: debug mds balancer = 20 2023-10-11T09:45:53.441 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:45:53.441 INFO:teuthology.orchestra.run.smithi078.stdout: mds debug frag = True 2023-10-11T09:45:53.442 INFO:teuthology.orchestra.run.smithi078.stdout: mds debug scatterstat = False 2023-10-11T09:45:53.442 INFO:teuthology.orchestra.run.smithi078.stdout: mds op complaint time = 180 2023-10-11T09:45:53.442 INFO:teuthology.orchestra.run.smithi078.stdout: mds verify scatter = False 2023-10-11T09:45:53.442 INFO:teuthology.orchestra.run.smithi078.stdout: osd op complaint time = 180 2023-10-11T09:45:53.442 INFO:teuthology.orchestra.run.smithi078.stdout: rados mon op timeout = 900 2023-10-11T09:45:53.443 INFO:teuthology.orchestra.run.smithi078.stdout: rados osd op timeout = 900 2023-10-11T09:45:53.443 INFO:teuthology.orchestra.run.smithi133.stdout:[global] 2023-10-11T09:45:53.444 INFO:teuthology.orchestra.run.smithi133.stdout: chdir = "" 2023-10-11T09:45:53.444 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-11T09:45:53.444 INFO:teuthology.orchestra.run.smithi133.stdout: auth supported = cephx 2023-10-11T09:45:53.444 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.445 INFO:teuthology.orchestra.run.smithi133.stdout: filestore xattr use omap = true 2023-10-11T09:45:53.445 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.445 INFO:teuthology.orchestra.run.smithi133.stdout: mon clock drift allowed = 1.000 2023-10-11T09:45:53.445 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.445 INFO:teuthology.orchestra.run.smithi133.stdout: osd crush chooseleaf type = 0 2023-10-11T09:45:53.446 INFO:teuthology.orchestra.run.smithi133.stdout: auth debug = true 2023-10-11T09:45:53.446 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.446 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on old message = true 2023-10-11T09:45:53.446 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on bug = true 2023-10-11T09:45:53.446 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.447 INFO:teuthology.orchestra.run.smithi133.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-11T09:45:53.447 INFO:teuthology.orchestra.run.smithi133.stdout: mon pg warn max object skew = 0 2023-10-11T09:45:53.447 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.447 INFO:teuthology.orchestra.run.smithi133.stdout: # disable pg_autoscaler by default for new pools 2023-10-11T09:45:53.447 INFO:teuthology.orchestra.run.smithi133.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-11T09:45:53.448 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.448 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default size = 2 2023-10-11T09:45:53.448 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.448 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow primary affinity = true 2023-10-11T09:45:53.448 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow pg remap = true 2023-10-11T09:45:53.449 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on legacy crush tunables = false 2023-10-11T09:45:53.449 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on crush straw calc version zero = false 2023-10-11T09:45:53.449 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on no sortbitwise = false 2023-10-11T09:45:53.449 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on osd down out interval zero = false 2023-10-11T09:45:53.449 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on too few osds = false 2023-10-11T09:45:53.449 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-11T09:45:53.450 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-11T09:45:53.450 INFO:teuthology.orchestra.run.smithi133.stdout: mon_allow_pool_size_one = true 2023-10-11T09:45:53.450 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.450 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-11T09:45:53.450 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.451 INFO:teuthology.orchestra.run.smithi133.stdout: osd default data pool replay window = 5 2023-10-11T09:45:53.451 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.451 INFO:teuthology.orchestra.run.smithi133.stdout: mon allow pool delete = true 2023-10-11T09:45:53.451 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.451 INFO:teuthology.orchestra.run.smithi133.stdout: mon cluster log file level = debug 2023-10-11T09:45:53.452 INFO:teuthology.orchestra.run.smithi133.stdout: debug asserts on shutdown = true 2023-10-11T09:45:53.452 INFO:teuthology.orchestra.run.smithi133.stdout: mon health detail to clog = false 2023-10-11T09:45:53.452 INFO:teuthology.orchestra.run.smithi133.stdout: mon host = "172.21.15.78,172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790]" 2023-10-11T09:45:53.452 INFO:teuthology.orchestra.run.smithi133.stdout: fsid = dee99db9-7a72-406f-a1e2-917943f2b13d 2023-10-11T09:45:53.452 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.452 INFO:teuthology.orchestra.run.smithi133.stdout:[osd] 2023-10-11T09:45:53.453 INFO:teuthology.orchestra.run.smithi133.stdout: osd journal size = 100 2023-10-11T09:45:53.453 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.453 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub load threshold = 5.0 2023-10-11T09:45:53.453 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub max interval = 600 2023-10-11T09:45:53.453 INFO:teuthology.orchestra.run.smithi133.stdout: osd mclock profile = high_recovery_ops 2023-10-11T09:45:53.453 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.454 INFO:teuthology.orchestra.run.smithi133.stdout: osd recover clone overlap = true 2023-10-11T09:45:53.454 INFO:teuthology.orchestra.run.smithi133.stdout: osd recovery max chunk = 1048576 2023-10-11T09:45:53.454 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.454 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug shutdown = true 2023-10-11T09:45:53.454 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug op order = true 2023-10-11T09:45:53.455 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug verify stray on activate = true 2023-10-11T09:45:53.455 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.455 INFO:teuthology.orchestra.run.smithi133.stdout: osd open classes on start = true 2023-10-11T09:45:53.455 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug pg log writeout = true 2023-10-11T09:45:53.455 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.456 INFO:teuthology.orchestra.run.smithi133.stdout: osd deep scrub update digest min age = 30 2023-10-11T09:45:53.456 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.456 INFO:teuthology.orchestra.run.smithi133.stdout: osd map max advance = 10 2023-10-11T09:45:53.456 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.456 INFO:teuthology.orchestra.run.smithi133.stdout: journal zero on create = true 2023-10-11T09:45:53.456 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.457 INFO:teuthology.orchestra.run.smithi133.stdout: filestore ondisk finisher threads = 3 2023-10-11T09:45:53.457 INFO:teuthology.orchestra.run.smithi133.stdout: filestore apply finisher threads = 3 2023-10-11T09:45:53.457 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.457 INFO:teuthology.orchestra.run.smithi133.stdout: bdev debug aio = true 2023-10-11T09:45:53.458 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug misdirected ops = true 2023-10-11T09:45:53.458 INFO:teuthology.orchestra.run.smithi133.stdout: bdev async discard = True 2023-10-11T09:45:53.458 INFO:teuthology.orchestra.run.smithi133.stdout: bdev enable discard = True 2023-10-11T09:45:53.458 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore allocator = bitmap 2023-10-11T09:45:53.458 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore block size = 96636764160 2023-10-11T09:45:53.459 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore fsck on mount = True 2023-10-11T09:45:53.459 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluefs = 1/20 2023-10-11T09:45:53.459 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluestore = 1/20 2023-10-11T09:45:53.459 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:45:53.459 INFO:teuthology.orchestra.run.smithi133.stdout: debug osd = 20 2023-10-11T09:45:53.459 INFO:teuthology.orchestra.run.smithi133.stdout: debug rocksdb = 4/10 2023-10-11T09:45:53.460 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd backfillfull_ratio = 0.85 2023-10-11T09:45:53.460 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd full ratio = 0.9 2023-10-11T09:45:53.460 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd nearfull ratio = 0.8 2023-10-11T09:45:53.460 INFO:teuthology.orchestra.run.smithi133.stdout: osd failsafe full ratio = 0.95 2023-10-11T09:45:53.460 INFO:teuthology.orchestra.run.smithi133.stdout: osd objectstore = bluestore 2023-10-11T09:45:53.460 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2023-10-11T09:45:53.461 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.461 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr] 2023-10-11T09:45:53.461 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:45:53.461 INFO:teuthology.orchestra.run.smithi133.stdout: debug mgr = 20 2023-10-11T09:45:53.461 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2023-10-11T09:45:53.462 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2023-10-11T09:45:53.462 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:45:53.462 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:45:53.462 INFO:teuthology.orchestra.run.smithi133.stdout: mgr/telemetry/nag = false 2023-10-11T09:45:53.462 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.463 INFO:teuthology.orchestra.run.smithi133.stdout:[mon] 2023-10-11T09:45:53.463 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:45:53.463 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2023-10-11T09:45:53.463 INFO:teuthology.orchestra.run.smithi133.stdout: debug paxos = 20 2023-10-11T09:45:53.463 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2023-10-11T09:45:53.463 INFO:teuthology.orchestra.run.smithi133.stdout: mon data avail warn = 5 2023-10-11T09:45:53.464 INFO:teuthology.orchestra.run.smithi133.stdout: mon mgr mkfs grace = 240 2023-10-11T09:45:53.464 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:45:53.464 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd reporter subtree level = osd 2023-10-11T09:45:53.464 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd prime pg temp = true 2023-10-11T09:45:53.464 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:45:53.464 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.465 INFO:teuthology.orchestra.run.smithi133.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-11T09:45:53.465 INFO:teuthology.orchestra.run.smithi133.stdout: auth mon ticket ttl = 660 # 11m 2023-10-11T09:45:53.465 INFO:teuthology.orchestra.run.smithi133.stdout: auth service ticket ttl = 240 # 4m 2023-10-11T09:45:53.465 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.465 INFO:teuthology.orchestra.run.smithi133.stdout: # don't complain about insecure global_id in the test suite 2023-10-11T09:45:53.466 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-11T09:45:53.466 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-11T09:45:53.466 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.466 INFO:teuthology.orchestra.run.smithi133.stdout: # 1m isn't quite enough 2023-10-11T09:45:53.466 INFO:teuthology.orchestra.run.smithi133.stdout: mon_down_mkfs_grace = 2m 2023-10-11T09:45:53.467 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.467 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_filestore_osds = false 2023-10-11T09:45:53.467 INFO:teuthology.orchestra.run.smithi133.stdout: mon op complaint time = 120 2023-10-11T09:45:53.467 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.467 INFO:teuthology.orchestra.run.smithi133.stdout:[client] 2023-10-11T09:45:53.468 INFO:teuthology.orchestra.run.smithi133.stdout: rgw cache enabled = true 2023-10-11T09:45:53.468 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable ops log = true 2023-10-11T09:45:53.468 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable usage log = true 2023-10-11T09:45:53.469 INFO:teuthology.orchestra.run.smithi133.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-11T09:45:53.469 INFO:teuthology.orchestra.run.smithi133.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-11T09:45:53.469 INFO:teuthology.orchestra.run.smithi133.stdout: client mount timeout = 600 2023-10-11T09:45:53.470 INFO:teuthology.orchestra.run.smithi133.stdout: debug client = 20 2023-10-11T09:45:53.470 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:45:53.470 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = "" 2023-10-11T09:45:53.471 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2023-10-11T09:45:53.471 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2023-10-11T09:45:53.472 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.a] 2023-10-11T09:45:53.472 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.b] 2023-10-11T09:45:53.472 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.c] 2023-10-11T09:45:53.473 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.a] 2023-10-11T09:45:53.473 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.c] 2023-10-11T09:45:53.473 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.b] 2023-10-11T09:45:53.473 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.d] 2023-10-11T09:45:53.474 INFO:teuthology.orchestra.run.smithi133.stdout:[mds] 2023-10-11T09:45:53.474 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds = 20 2023-10-11T09:45:53.474 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds balancer = 20 2023-10-11T09:45:53.475 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:45:53.475 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug frag = True 2023-10-11T09:45:53.475 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug scatterstat = False 2023-10-11T09:45:53.475 INFO:teuthology.orchestra.run.smithi133.stdout: mds op complaint time = 180 2023-10-11T09:45:53.476 INFO:teuthology.orchestra.run.smithi133.stdout: mds verify scatter = False 2023-10-11T09:45:53.476 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2023-10-11T09:45:53.476 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2023-10-11T09:45:53.477 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2023-10-11T09:45:53.479 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-11T09:45:53.532 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-11T09:45:53.602 INFO:teuthology.orchestra.run.smithi078.stdout:[global] 2023-10-11T09:45:53.602 INFO:teuthology.orchestra.run.smithi078.stdout: chdir = "" 2023-10-11T09:45:53.603 INFO:teuthology.orchestra.run.smithi078.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-11T09:45:53.603 INFO:teuthology.orchestra.run.smithi078.stdout: auth supported = cephx 2023-10-11T09:45:53.603 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.603 INFO:teuthology.orchestra.run.smithi078.stdout: filestore xattr use omap = true 2023-10-11T09:45:53.603 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.603 INFO:teuthology.orchestra.run.smithi078.stdout: mon clock drift allowed = 1.000 2023-10-11T09:45:53.604 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.604 INFO:teuthology.orchestra.run.smithi078.stdout: osd crush chooseleaf type = 0 2023-10-11T09:45:53.604 INFO:teuthology.orchestra.run.smithi078.stdout: auth debug = true 2023-10-11T09:45:53.604 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.604 INFO:teuthology.orchestra.run.smithi078.stdout: ms die on old message = true 2023-10-11T09:45:53.605 INFO:teuthology.orchestra.run.smithi078.stdout: ms die on bug = true 2023-10-11T09:45:53.605 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.605 INFO:teuthology.orchestra.run.smithi078.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-11T09:45:53.605 INFO:teuthology.orchestra.run.smithi078.stdout: mon pg warn max object skew = 0 2023-10-11T09:45:53.605 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.606 INFO:teuthology.orchestra.run.smithi078.stdout: # disable pg_autoscaler by default for new pools 2023-10-11T09:45:53.606 INFO:teuthology.orchestra.run.smithi078.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-11T09:45:53.606 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.606 INFO:teuthology.orchestra.run.smithi078.stdout: osd pool default size = 2 2023-10-11T09:45:53.606 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.607 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd allow primary affinity = true 2023-10-11T09:45:53.607 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd allow pg remap = true 2023-10-11T09:45:53.607 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on legacy crush tunables = false 2023-10-11T09:45:53.607 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on crush straw calc version zero = false 2023-10-11T09:45:53.607 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on no sortbitwise = false 2023-10-11T09:45:53.608 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on osd down out interval zero = false 2023-10-11T09:45:53.608 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on too few osds = false 2023-10-11T09:45:53.608 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-11T09:45:53.608 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-11T09:45:53.608 INFO:teuthology.orchestra.run.smithi078.stdout: mon_allow_pool_size_one = true 2023-10-11T09:45:53.609 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.609 INFO:teuthology.orchestra.run.smithi078.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-11T09:45:53.609 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.610 INFO:teuthology.orchestra.run.smithi078.stdout: osd default data pool replay window = 5 2023-10-11T09:45:53.610 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.610 INFO:teuthology.orchestra.run.smithi078.stdout: mon allow pool delete = true 2023-10-11T09:45:53.611 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.611 INFO:teuthology.orchestra.run.smithi078.stdout: mon cluster log file level = debug 2023-10-11T09:45:53.611 INFO:teuthology.orchestra.run.smithi078.stdout: debug asserts on shutdown = true 2023-10-11T09:45:53.612 INFO:teuthology.orchestra.run.smithi078.stdout: mon health detail to clog = false 2023-10-11T09:45:53.612 INFO:teuthology.orchestra.run.smithi078.stdout: mon host = "172.21.15.78,172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790]" 2023-10-11T09:45:53.612 INFO:teuthology.orchestra.run.smithi078.stdout: fsid = dee99db9-7a72-406f-a1e2-917943f2b13d 2023-10-11T09:45:53.613 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.613 INFO:teuthology.orchestra.run.smithi078.stdout:[osd] 2023-10-11T09:45:53.613 INFO:teuthology.orchestra.run.smithi078.stdout: osd journal size = 100 2023-10-11T09:45:53.614 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.614 INFO:teuthology.orchestra.run.smithi078.stdout: osd scrub load threshold = 5.0 2023-10-11T09:45:53.614 INFO:teuthology.orchestra.run.smithi078.stdout: osd scrub max interval = 600 2023-10-11T09:45:53.615 INFO:teuthology.orchestra.run.smithi078.stdout: osd mclock profile = high_recovery_ops 2023-10-11T09:45:53.615 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.615 INFO:teuthology.orchestra.run.smithi078.stdout: osd recover clone overlap = true 2023-10-11T09:45:53.616 INFO:teuthology.orchestra.run.smithi078.stdout: osd recovery max chunk = 1048576 2023-10-11T09:45:53.616 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.616 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug shutdown = true 2023-10-11T09:45:53.617 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug op order = true 2023-10-11T09:45:53.617 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug verify stray on activate = true 2023-10-11T09:45:53.617 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.618 INFO:teuthology.orchestra.run.smithi078.stdout: osd open classes on start = true 2023-10-11T09:45:53.618 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug pg log writeout = true 2023-10-11T09:45:53.618 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.618 INFO:teuthology.orchestra.run.smithi078.stdout: osd deep scrub update digest min age = 30 2023-10-11T09:45:53.619 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.619 INFO:teuthology.orchestra.run.smithi078.stdout: osd map max advance = 10 2023-10-11T09:45:53.619 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.619 INFO:teuthology.orchestra.run.smithi078.stdout: journal zero on create = true 2023-10-11T09:45:53.619 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.619 INFO:teuthology.orchestra.run.smithi078.stdout: filestore ondisk finisher threads = 3 2023-10-11T09:45:53.620 INFO:teuthology.orchestra.run.smithi078.stdout: filestore apply finisher threads = 3 2023-10-11T09:45:53.620 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.620 INFO:teuthology.orchestra.run.smithi078.stdout: bdev debug aio = true 2023-10-11T09:45:53.620 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug misdirected ops = true 2023-10-11T09:45:53.620 INFO:teuthology.orchestra.run.smithi078.stdout: bdev async discard = True 2023-10-11T09:45:53.621 INFO:teuthology.orchestra.run.smithi078.stdout: bdev enable discard = True 2023-10-11T09:45:53.621 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore allocator = bitmap 2023-10-11T09:45:53.621 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore block size = 96636764160 2023-10-11T09:45:53.621 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore fsck on mount = True 2023-10-11T09:45:53.621 INFO:teuthology.orchestra.run.smithi078.stdout: debug bluefs = 1/20 2023-10-11T09:45:53.622 INFO:teuthology.orchestra.run.smithi078.stdout: debug bluestore = 1/20 2023-10-11T09:45:53.622 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:45:53.622 INFO:teuthology.orchestra.run.smithi078.stdout: debug osd = 20 2023-10-11T09:45:53.622 INFO:teuthology.orchestra.run.smithi078.stdout: debug rocksdb = 4/10 2023-10-11T09:45:53.622 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd backfillfull_ratio = 0.85 2023-10-11T09:45:53.623 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd full ratio = 0.9 2023-10-11T09:45:53.623 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd nearfull ratio = 0.8 2023-10-11T09:45:53.623 INFO:teuthology.orchestra.run.smithi078.stdout: osd failsafe full ratio = 0.95 2023-10-11T09:45:53.623 INFO:teuthology.orchestra.run.smithi078.stdout: osd objectstore = bluestore 2023-10-11T09:45:53.624 INFO:teuthology.orchestra.run.smithi078.stdout: osd op complaint time = 180 2023-10-11T09:45:53.624 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.624 INFO:teuthology.orchestra.run.smithi078.stdout:[mgr] 2023-10-11T09:45:53.624 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:45:53.624 INFO:teuthology.orchestra.run.smithi078.stdout: debug mgr = 20 2023-10-11T09:45:53.625 INFO:teuthology.orchestra.run.smithi078.stdout: debug mon = 20 2023-10-11T09:45:53.625 INFO:teuthology.orchestra.run.smithi078.stdout: debug auth = 20 2023-10-11T09:45:53.625 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:45:53.625 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:45:53.625 INFO:teuthology.orchestra.run.smithi078.stdout: mgr/telemetry/nag = false 2023-10-11T09:45:53.625 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.626 INFO:teuthology.orchestra.run.smithi078.stdout:[mon] 2023-10-11T09:45:53.626 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:45:53.626 INFO:teuthology.orchestra.run.smithi078.stdout: debug mon = 20 2023-10-11T09:45:53.626 INFO:teuthology.orchestra.run.smithi078.stdout: debug paxos = 20 2023-10-11T09:45:53.627 INFO:teuthology.orchestra.run.smithi078.stdout: debug auth = 20 2023-10-11T09:45:53.627 INFO:teuthology.orchestra.run.smithi078.stdout: mon data avail warn = 5 2023-10-11T09:45:53.627 INFO:teuthology.orchestra.run.smithi078.stdout: mon mgr mkfs grace = 240 2023-10-11T09:45:53.627 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:45:53.627 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd reporter subtree level = osd 2023-10-11T09:45:53.628 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd prime pg temp = true 2023-10-11T09:45:53.628 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:45:53.628 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.628 INFO:teuthology.orchestra.run.smithi078.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-11T09:45:53.628 INFO:teuthology.orchestra.run.smithi078.stdout: auth mon ticket ttl = 660 # 11m 2023-10-11T09:45:53.628 INFO:teuthology.orchestra.run.smithi078.stdout: auth service ticket ttl = 240 # 4m 2023-10-11T09:45:53.629 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.629 INFO:teuthology.orchestra.run.smithi078.stdout: # don't complain about insecure global_id in the test suite 2023-10-11T09:45:53.629 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-11T09:45:53.629 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-11T09:45:53.629 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.630 INFO:teuthology.orchestra.run.smithi078.stdout: # 1m isn't quite enough 2023-10-11T09:45:53.630 INFO:teuthology.orchestra.run.smithi078.stdout: mon_down_mkfs_grace = 2m 2023-10-11T09:45:53.630 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.630 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_filestore_osds = false 2023-10-11T09:45:53.631 INFO:teuthology.orchestra.run.smithi078.stdout: mon op complaint time = 120 2023-10-11T09:45:53.631 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:53.631 INFO:teuthology.orchestra.run.smithi078.stdout:[client] 2023-10-11T09:45:53.631 INFO:teuthology.orchestra.run.smithi078.stdout: rgw cache enabled = true 2023-10-11T09:45:53.631 INFO:teuthology.orchestra.run.smithi078.stdout: rgw enable ops log = true 2023-10-11T09:45:53.632 INFO:teuthology.orchestra.run.smithi078.stdout: rgw enable usage log = true 2023-10-11T09:45:53.632 INFO:teuthology.orchestra.run.smithi078.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-11T09:45:53.632 INFO:teuthology.orchestra.run.smithi078.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-11T09:45:53.632 INFO:teuthology.orchestra.run.smithi078.stdout: client mount timeout = 600 2023-10-11T09:45:53.632 INFO:teuthology.orchestra.run.smithi078.stdout: debug client = 20 2023-10-11T09:45:53.633 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:45:53.633 INFO:teuthology.orchestra.run.smithi078.stdout: pid file = "" 2023-10-11T09:45:53.633 INFO:teuthology.orchestra.run.smithi078.stdout: rados mon op timeout = 900 2023-10-11T09:45:53.633 INFO:teuthology.orchestra.run.smithi078.stdout: rados osd op timeout = 900 2023-10-11T09:45:53.633 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.a] 2023-10-11T09:45:53.633 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.b] 2023-10-11T09:45:53.634 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.c] 2023-10-11T09:45:53.634 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.a] 2023-10-11T09:45:53.634 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.c] 2023-10-11T09:45:53.634 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.b] 2023-10-11T09:45:53.634 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.d] 2023-10-11T09:45:53.635 INFO:teuthology.orchestra.run.smithi078.stdout:[mds] 2023-10-11T09:45:53.635 INFO:teuthology.orchestra.run.smithi078.stdout: debug mds = 20 2023-10-11T09:45:53.635 INFO:teuthology.orchestra.run.smithi078.stdout: debug mds balancer = 20 2023-10-11T09:45:53.635 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:45:53.635 INFO:teuthology.orchestra.run.smithi078.stdout: mds debug frag = True 2023-10-11T09:45:53.636 INFO:teuthology.orchestra.run.smithi078.stdout: mds debug scatterstat = False 2023-10-11T09:45:53.636 INFO:teuthology.orchestra.run.smithi078.stdout: mds op complaint time = 180 2023-10-11T09:45:53.636 INFO:teuthology.orchestra.run.smithi078.stdout: mds verify scatter = False 2023-10-11T09:45:53.636 INFO:teuthology.orchestra.run.smithi078.stdout: osd op complaint time = 180 2023-10-11T09:45:53.636 INFO:teuthology.orchestra.run.smithi078.stdout: rados mon op timeout = 900 2023-10-11T09:45:53.637 INFO:teuthology.orchestra.run.smithi078.stdout: rados osd op timeout = 900 2023-10-11T09:45:53.638 INFO:teuthology.orchestra.run.smithi133.stdout:[global] 2023-10-11T09:45:53.639 INFO:teuthology.orchestra.run.smithi133.stdout: chdir = "" 2023-10-11T09:45:53.639 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-11T09:45:53.639 INFO:teuthology.orchestra.run.smithi133.stdout: auth supported = cephx 2023-10-11T09:45:53.639 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.639 INFO:teuthology.orchestra.run.smithi133.stdout: filestore xattr use omap = true 2023-10-11T09:45:53.639 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.640 INFO:teuthology.orchestra.run.smithi133.stdout: mon clock drift allowed = 1.000 2023-10-11T09:45:53.640 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.640 INFO:teuthology.orchestra.run.smithi133.stdout: osd crush chooseleaf type = 0 2023-10-11T09:45:53.640 INFO:teuthology.orchestra.run.smithi133.stdout: auth debug = true 2023-10-11T09:45:53.640 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.641 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on old message = true 2023-10-11T09:45:53.641 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on bug = true 2023-10-11T09:45:53.641 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.641 INFO:teuthology.orchestra.run.smithi133.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-11T09:45:53.641 INFO:teuthology.orchestra.run.smithi133.stdout: mon pg warn max object skew = 0 2023-10-11T09:45:53.642 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.642 INFO:teuthology.orchestra.run.smithi133.stdout: # disable pg_autoscaler by default for new pools 2023-10-11T09:45:53.642 INFO:teuthology.orchestra.run.smithi133.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-11T09:45:53.642 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.642 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default size = 2 2023-10-11T09:45:53.643 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.643 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow primary affinity = true 2023-10-11T09:45:53.643 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow pg remap = true 2023-10-11T09:45:53.643 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on legacy crush tunables = false 2023-10-11T09:45:53.643 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on crush straw calc version zero = false 2023-10-11T09:45:53.643 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on no sortbitwise = false 2023-10-11T09:45:53.644 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on osd down out interval zero = false 2023-10-11T09:45:53.644 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on too few osds = false 2023-10-11T09:45:53.644 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-11T09:45:53.644 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-11T09:45:53.644 INFO:teuthology.orchestra.run.smithi133.stdout: mon_allow_pool_size_one = true 2023-10-11T09:45:53.645 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.645 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-11T09:45:53.645 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.645 INFO:teuthology.orchestra.run.smithi133.stdout: osd default data pool replay window = 5 2023-10-11T09:45:53.646 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.646 INFO:teuthology.orchestra.run.smithi133.stdout: mon allow pool delete = true 2023-10-11T09:45:53.646 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.646 INFO:teuthology.orchestra.run.smithi133.stdout: mon cluster log file level = debug 2023-10-11T09:45:53.646 INFO:teuthology.orchestra.run.smithi133.stdout: debug asserts on shutdown = true 2023-10-11T09:45:53.647 INFO:teuthology.orchestra.run.smithi133.stdout: mon health detail to clog = false 2023-10-11T09:45:53.647 INFO:teuthology.orchestra.run.smithi133.stdout: mon host = "172.21.15.78,172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790]" 2023-10-11T09:45:53.647 INFO:teuthology.orchestra.run.smithi133.stdout: fsid = dee99db9-7a72-406f-a1e2-917943f2b13d 2023-10-11T09:45:53.647 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.647 INFO:teuthology.orchestra.run.smithi133.stdout:[osd] 2023-10-11T09:45:53.648 INFO:teuthology.orchestra.run.smithi133.stdout: osd journal size = 100 2023-10-11T09:45:53.648 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.648 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub load threshold = 5.0 2023-10-11T09:45:53.648 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub max interval = 600 2023-10-11T09:45:53.648 INFO:teuthology.orchestra.run.smithi133.stdout: osd mclock profile = high_recovery_ops 2023-10-11T09:45:53.649 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.649 INFO:teuthology.orchestra.run.smithi133.stdout: osd recover clone overlap = true 2023-10-11T09:45:53.649 INFO:teuthology.orchestra.run.smithi133.stdout: osd recovery max chunk = 1048576 2023-10-11T09:45:53.649 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.649 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug shutdown = true 2023-10-11T09:45:53.650 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug op order = true 2023-10-11T09:45:53.650 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug verify stray on activate = true 2023-10-11T09:45:53.650 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.650 INFO:teuthology.orchestra.run.smithi133.stdout: osd open classes on start = true 2023-10-11T09:45:53.650 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug pg log writeout = true 2023-10-11T09:45:53.651 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.651 INFO:teuthology.orchestra.run.smithi133.stdout: osd deep scrub update digest min age = 30 2023-10-11T09:45:53.651 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.651 INFO:teuthology.orchestra.run.smithi133.stdout: osd map max advance = 10 2023-10-11T09:45:53.651 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.652 INFO:teuthology.orchestra.run.smithi133.stdout: journal zero on create = true 2023-10-11T09:45:53.652 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.652 INFO:teuthology.orchestra.run.smithi133.stdout: filestore ondisk finisher threads = 3 2023-10-11T09:45:53.652 INFO:teuthology.orchestra.run.smithi133.stdout: filestore apply finisher threads = 3 2023-10-11T09:45:53.652 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.653 INFO:teuthology.orchestra.run.smithi133.stdout: bdev debug aio = true 2023-10-11T09:45:53.653 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug misdirected ops = true 2023-10-11T09:45:53.653 INFO:teuthology.orchestra.run.smithi133.stdout: bdev async discard = True 2023-10-11T09:45:53.653 INFO:teuthology.orchestra.run.smithi133.stdout: bdev enable discard = True 2023-10-11T09:45:53.654 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore allocator = bitmap 2023-10-11T09:45:53.654 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore block size = 96636764160 2023-10-11T09:45:53.654 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore fsck on mount = True 2023-10-11T09:45:53.654 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluefs = 1/20 2023-10-11T09:45:53.654 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluestore = 1/20 2023-10-11T09:45:53.655 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:45:53.655 INFO:teuthology.orchestra.run.smithi133.stdout: debug osd = 20 2023-10-11T09:45:53.655 INFO:teuthology.orchestra.run.smithi133.stdout: debug rocksdb = 4/10 2023-10-11T09:45:53.655 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd backfillfull_ratio = 0.85 2023-10-11T09:45:53.655 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd full ratio = 0.9 2023-10-11T09:45:53.656 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd nearfull ratio = 0.8 2023-10-11T09:45:53.656 INFO:teuthology.orchestra.run.smithi133.stdout: osd failsafe full ratio = 0.95 2023-10-11T09:45:53.656 INFO:teuthology.orchestra.run.smithi133.stdout: osd objectstore = bluestore 2023-10-11T09:45:53.656 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2023-10-11T09:45:53.656 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.657 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr] 2023-10-11T09:45:53.657 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:45:53.657 INFO:teuthology.orchestra.run.smithi133.stdout: debug mgr = 20 2023-10-11T09:45:53.657 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2023-10-11T09:45:53.657 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2023-10-11T09:45:53.658 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:45:53.658 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:45:53.658 INFO:teuthology.orchestra.run.smithi133.stdout: mgr/telemetry/nag = false 2023-10-11T09:45:53.658 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.659 INFO:teuthology.orchestra.run.smithi133.stdout:[mon] 2023-10-11T09:45:53.659 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:45:53.659 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2023-10-11T09:45:53.659 INFO:teuthology.orchestra.run.smithi133.stdout: debug paxos = 20 2023-10-11T09:45:53.659 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2023-10-11T09:45:53.659 INFO:teuthology.orchestra.run.smithi133.stdout: mon data avail warn = 5 2023-10-11T09:45:53.660 INFO:teuthology.orchestra.run.smithi133.stdout: mon mgr mkfs grace = 240 2023-10-11T09:45:53.660 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:45:53.660 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd reporter subtree level = osd 2023-10-11T09:45:53.660 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd prime pg temp = true 2023-10-11T09:45:53.660 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:45:53.661 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.661 INFO:teuthology.orchestra.run.smithi133.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-11T09:45:53.661 INFO:teuthology.orchestra.run.smithi133.stdout: auth mon ticket ttl = 660 # 11m 2023-10-11T09:45:53.661 INFO:teuthology.orchestra.run.smithi133.stdout: auth service ticket ttl = 240 # 4m 2023-10-11T09:45:53.661 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.662 INFO:teuthology.orchestra.run.smithi133.stdout: # don't complain about insecure global_id in the test suite 2023-10-11T09:45:53.662 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-11T09:45:53.662 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-11T09:45:53.662 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.662 INFO:teuthology.orchestra.run.smithi133.stdout: # 1m isn't quite enough 2023-10-11T09:45:53.663 INFO:teuthology.orchestra.run.smithi133.stdout: mon_down_mkfs_grace = 2m 2023-10-11T09:45:53.663 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.663 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_filestore_osds = false 2023-10-11T09:45:53.663 INFO:teuthology.orchestra.run.smithi133.stdout: mon op complaint time = 120 2023-10-11T09:45:53.663 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:45:53.664 INFO:teuthology.orchestra.run.smithi133.stdout:[client] 2023-10-11T09:45:53.664 INFO:teuthology.orchestra.run.smithi133.stdout: rgw cache enabled = true 2023-10-11T09:45:53.664 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable ops log = true 2023-10-11T09:45:53.664 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable usage log = true 2023-10-11T09:45:53.665 INFO:teuthology.orchestra.run.smithi133.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-11T09:45:53.665 INFO:teuthology.orchestra.run.smithi133.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-11T09:45:53.665 INFO:teuthology.orchestra.run.smithi133.stdout: client mount timeout = 600 2023-10-11T09:45:53.666 INFO:teuthology.orchestra.run.smithi133.stdout: debug client = 20 2023-10-11T09:45:53.666 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:45:53.667 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = "" 2023-10-11T09:45:53.667 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2023-10-11T09:45:53.667 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2023-10-11T09:45:53.667 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.a] 2023-10-11T09:45:53.668 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.b] 2023-10-11T09:45:53.668 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.c] 2023-10-11T09:45:53.668 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.a] 2023-10-11T09:45:53.669 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.c] 2023-10-11T09:45:53.669 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.b] 2023-10-11T09:45:53.669 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.d] 2023-10-11T09:45:53.669 INFO:teuthology.orchestra.run.smithi133.stdout:[mds] 2023-10-11T09:45:53.670 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds = 20 2023-10-11T09:45:53.670 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds balancer = 20 2023-10-11T09:45:53.670 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:45:53.670 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug frag = True 2023-10-11T09:45:53.671 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug scatterstat = False 2023-10-11T09:45:53.671 INFO:teuthology.orchestra.run.smithi133.stdout: mds op complaint time = 180 2023-10-11T09:45:53.671 INFO:teuthology.orchestra.run.smithi133.stdout: mds verify scatter = False 2023-10-11T09:45:53.672 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2023-10-11T09:45:53.672 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2023-10-11T09:45:53.672 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2023-10-11T09:45:53.672 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph purge cephfs_metadata --yes-i-really-really-mean-it 2023-10-11T09:45:53.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.740+0000 7f8de7998140 1 Processor -- start 2023-10-11T09:45:53.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.740+0000 7f8de7998140 1 -- start start 2023-10-11T09:45:53.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.740+0000 7f8de7998140 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dbf9512b0 0x555dbf9655a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:53.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.740+0000 7f8de7998140 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555dbf96ddc0 0x555dbf9f78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:53.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.740+0000 7f8de7998140 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x555dbf9f94e0 0x555dbf9fc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:53.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.740+0000 7f8de7998140 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dbf8b7ce0 con 0x555dbf9512b0 2023-10-11T09:45:53.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.740+0000 7f8de7998140 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dbf874250 con 0x555dbf96ddc0 2023-10-11T09:45:53.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.740+0000 7f8de7998140 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x555dbf873ef0 con 0x555dbf9f94e0 2023-10-11T09:45:53.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.740+0000 7f8de54cd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dbf9512b0 0x555dbf9655a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:53.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.740+0000 7f8de54cd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dbf9512b0 0x555dbf9655a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50694/0 (socket says 172.21.15.78:50694) 2023-10-11T09:45:53.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.740+0000 7f8de54cd700 1 -- 172.21.15.78:0/1235848659 learned_addr learned my addr 172.21.15.78:0/1235848659 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:53.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.740+0000 7f8de4ccc700 1 --2- 172.21.15.78:0/1235848659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555dbf96ddc0 0x555dbf9f78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:53.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.740+0000 7f8de5cce700 1 --2- 172.21.15.78:0/1235848659 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x555dbf9f94e0 0x555dbf9fc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:53.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.740+0000 7f8de54cd700 1 -- 172.21.15.78:0/1235848659 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x555dbf9f94e0 msgr2=0x555dbf9fc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:53.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.740+0000 7f8de54cd700 1 --2- 172.21.15.78:0/1235848659 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x555dbf9f94e0 0x555dbf9fc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:53.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.740+0000 7f8de54cd700 1 -- 172.21.15.78:0/1235848659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555dbf96ddc0 msgr2=0x555dbf9f78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:53.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.740+0000 7f8de54cd700 1 --2- 172.21.15.78:0/1235848659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555dbf96ddc0 0x555dbf9f78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:53.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.740+0000 7f8de54cd700 1 -- 172.21.15.78:0/1235848659 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de000a100 con 0x555dbf9512b0 2023-10-11T09:45:53.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.740+0000 7f8de54cd700 1 --2- 172.21.15.78:0/1235848659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dbf9512b0 0x555dbf9655a0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8001ce0 tx=0x7f8dd800b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f52d7593225d0a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:53.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.744+0000 7f8dd77fe700 1 -- 172.21.15.78:0/1235848659 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8014070 con 0x555dbf9512b0 2023-10-11T09:45:53.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.744+0000 7f8dd77fe700 1 -- 172.21.15.78:0/1235848659 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dd8008030 con 0x555dbf9512b0 2023-10-11T09:45:53.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.744+0000 7f8dd77fe700 1 -- 172.21.15.78:0/1235848659 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd800bce0 con 0x555dbf9512b0 2023-10-11T09:45:53.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.744+0000 7f8de7998140 1 -- 172.21.15.78:0/1235848659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dbf9512b0 msgr2=0x555dbf9655a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:53.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.744+0000 7f8de7998140 1 --2- 172.21.15.78:0/1235848659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dbf9512b0 0x555dbf9655a0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8001ce0 tx=0x7f8dd800b4c0 comp rx=0 tx=0).stop 2023-10-11T09:45:53.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.744+0000 7f8de7998140 1 -- 172.21.15.78:0/1235848659 shutdown_connections 2023-10-11T09:45:53.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.744+0000 7f8de7998140 1 --2- 172.21.15.78:0/1235848659 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x555dbf9f94e0 0x555dbf9fc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:53.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.744+0000 7f8de7998140 1 --2- 172.21.15.78:0/1235848659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555dbf96ddc0 0x555dbf9f78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:53.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.744+0000 7f8de7998140 1 --2- 172.21.15.78:0/1235848659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dbf9512b0 0x555dbf9655a0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:53.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.744+0000 7f8de7998140 1 -- 172.21.15.78:0/1235848659 >> 172.21.15.78:0/1235848659 conn(0x555dbf952330 msgr2=0x555dbf955e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:53.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.744+0000 7f8de7998140 1 -- 172.21.15.78:0/1235848659 shutdown_connections 2023-10-11T09:45:53.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.744+0000 7f8de7998140 1 -- 172.21.15.78:0/1235848659 wait complete. 2023-10-11T09:45:53.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.744+0000 7f8de7998140 1 Processor -- start 2023-10-11T09:45:53.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.744+0000 7f8de7998140 1 -- start start 2023-10-11T09:45:53.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.744+0000 7f8de7998140 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555dbf96ddc0 0x555dbf96ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:53.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.744+0000 7f8de7998140 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x555dbf9f94e0 0x555dbf9f4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:53.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.744+0000 7f8de7998140 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dbf9f4c90 0x555dbfa82010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:53.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.744+0000 7f8de7998140 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dbf84a240 con 0x555dbf9f4c90 2023-10-11T09:45:53.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.744+0000 7f8de7998140 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dbf96c6d0 con 0x555dbf96ddc0 2023-10-11T09:45:53.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.744+0000 7f8de7998140 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x555dbf96d120 con 0x555dbf9f94e0 2023-10-11T09:45:53.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.744+0000 7f8de5cce700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dbf9f4c90 0x555dbfa82010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:53.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.744+0000 7f8de5cce700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dbf9f4c90 0x555dbfa82010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50702/0 (socket says 172.21.15.78:50702) 2023-10-11T09:45:53.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.744+0000 7f8de5cce700 1 -- 172.21.15.78:0/3557916786 learned_addr learned my addr 172.21.15.78:0/3557916786 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:53.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.744+0000 7f8de54cd700 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555dbf96ddc0 0x555dbf96ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:53.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.744+0000 7f8de4ccc700 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x555dbf9f94e0 0x555dbf9f4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:53.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.748+0000 7f8de5cce700 1 -- 172.21.15.78:0/3557916786 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x555dbf9f94e0 msgr2=0x555dbf9f4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:53.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.748+0000 7f8de5cce700 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x555dbf9f94e0 0x555dbf9f4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:53.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.748+0000 7f8de5cce700 1 -- 172.21.15.78:0/3557916786 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555dbf96ddc0 msgr2=0x555dbf96ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:53.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.748+0000 7f8de5cce700 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555dbf96ddc0 0x555dbf96ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:53.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.748+0000 7f8de5cce700 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd800a040 con 0x555dbf9f4c90 2023-10-11T09:45:53.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.748+0000 7f8de4ccc700 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x555dbf9f94e0 0x555dbf9f4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:45:53.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.748+0000 7f8de5cce700 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dbf9f4c90 0x555dbfa82010 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x555dbf9115f0 tx=0x7f8de0003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:53.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.748+0000 7f8dd67fc700 1 -- 172.21.15.78:0/3557916786 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de001b650 con 0x555dbf9f4c90 2023-10-11T09:45:53.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.748+0000 7f8dd67fc700 1 -- 172.21.15.78:0/3557916786 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de001b7b0 con 0x555dbf9f4c90 2023-10-11T09:45:53.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.748+0000 7f8dd67fc700 1 -- 172.21.15.78:0/3557916786 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de001bb20 con 0x555dbf9f4c90 2023-10-11T09:45:53.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.748+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555dbf8558b0 con 0x555dbf9f4c90 2023-10-11T09:45:53.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.748+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555dbf855310 con 0x555dbf9f4c90 2023-10-11T09:45:53.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.748+0000 7f8dd67fc700 1 -- 172.21.15.78:0/3557916786 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f8de0017040 con 0x555dbf9f4c90 2023-10-11T09:45:53.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.752+0000 7f8dd67fc700 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8dc8068c40 0x7f8dc806b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:53.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.752+0000 7f8dd67fc700 1 -- 172.21.15.78:0/3557916786 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6552+0+0 (secure 0 0 0) 0x7f8de0083af0 con 0x555dbf9f4c90 2023-10-11T09:45:53.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.752+0000 7f8de54cd700 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8dc8068c40 0x7f8dc806b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:53.760 INFO:teuthology.orchestra.run.smithi078.stdout:Warning: using slow linear search 2023-10-11T09:45:53.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.752+0000 7f8de7998140 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x555dbf968830 0x555dbf924360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:53.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.752+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:1 8.0 8:00000000:%01:::head [pgnls start_epoch 65 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e65) v8 -- 0x555dbf9fc290 con 0x555dbf968830 2023-10-11T09:45:53.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.752+0000 7f8de54cd700 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8dc8068c40 0x7f8dc806b0f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x555dbf9f6e60 tx=0x7f8dd8009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:53.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.752+0000 7f8de4ccc700 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x555dbf968830 0x555dbf924360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:53.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.752+0000 7f8de4ccc700 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x555dbf968830 0x555dbf924360 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:53.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.760+0000 7f8de4ccc700 1 -- 172.21.15.78:0/3557916786 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(1 [pgnls start_epoch 65 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ddc010800 con 0x555dbf968830 2023-10-11T09:45:53.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.760+0000 7f8de4ccc700 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x7f8ddc00e950 0x7f8ddc00d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:53.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.760+0000 7f8de4ccc700 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:2 8.10 8:08000000:%01:::head [pgnls start_epoch 65 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e65) v8 -- 0x555dbf9fc290 con 0x7f8ddc00e950 2023-10-11T09:45:53.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.760+0000 7f8de54cd700 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x7f8ddc00e950 0x7f8ddc00d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:53.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.760+0000 7f8de54cd700 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x7f8ddc00e950 0x7f8ddc00d3d0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:53.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.760+0000 7f8de54cd700 1 -- 172.21.15.78:0/3557916786 <== osd.7 v2:172.21.15.133:6824/35901 1 ==== osd_op_reply(2 [pgnls start_epoch 65 out=177b] v64'10 uv10 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x7f8dd8016070 con 0x7f8ddc00e950 2023-10-11T09:45:53.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.760+0000 7f8de54cd700 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x555dbf9512b0 0x7f8dd8028930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:53.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.760+0000 7f8de54cd700 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:3 8.8 8:10000000:%01:::head [pgnls start_epoch 65 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e65) v8 -- 0x555dbf9fc290 con 0x555dbf9512b0 2023-10-11T09:45:53.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.764+0000 7f8de4ccc700 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x555dbf9512b0 0x7f8dd8028930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:53.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.764+0000 7f8de4ccc700 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x555dbf9512b0 0x7f8dd8028930 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:53.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.772+0000 7f8de4ccc700 1 -- 172.21.15.78:0/3557916786 <== osd.5 v2:172.21.15.133:6816/35899 1 ==== osd_op_reply(3 [pgnls start_epoch 65 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ddc010800 con 0x555dbf9512b0 2023-10-11T09:45:53.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.772+0000 7f8de4ccc700 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x7f8ddc00b860 0x7f8ddc00c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:53.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.772+0000 7f8de4ccc700 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:4 8.18 8:18000000:%01:::head [pgnls start_epoch 65 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e65) v8 -- 0x555dbf9fc290 con 0x7f8ddc00b860 2023-10-11T09:45:53.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.772+0000 7f8de5cce700 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x7f8ddc00b860 0x7f8ddc00c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:53.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.772+0000 7f8de5cce700 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x7f8ddc00b860 0x7f8ddc00c360 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:53.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.772+0000 7f8de5cce700 1 -- 172.21.15.78:0/3557916786 <== osd.4 v2:172.21.15.133:6808/35898 1 ==== osd_op_reply(4 [pgnls start_epoch 65 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8de0022070 con 0x7f8ddc00b860 2023-10-11T09:45:53.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.772+0000 7f8de5cce700 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:5 8.4 8:20000000:%01:::head [pgnls start_epoch 65 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e65) v8 -- 0x555dbf9fc290 con 0x555dbf9512b0 2023-10-11T09:45:53.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.772+0000 7f8de4ccc700 1 -- 172.21.15.78:0/3557916786 <== osd.5 v2:172.21.15.133:6816/35899 2 ==== osd_op_reply(5 [pgnls start_epoch 65 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ddc010800 con 0x555dbf9512b0 2023-10-11T09:45:53.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.772+0000 7f8de4ccc700 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:6 8.14 8:28000000:%01:::head [pgnls start_epoch 65 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e65) v8 -- 0x555dbf9fc290 con 0x7f8ddc00b860 2023-10-11T09:45:53.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.776+0000 7f8de5cce700 1 -- 172.21.15.78:0/3557916786 <== osd.4 v2:172.21.15.133:6808/35898 2 ==== osd_op_reply(6 [pgnls start_epoch 65 out=149b] v64'89 uv89 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x7f8de0022070 con 0x7f8ddc00b860 2023-10-11T09:45:53.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.776+0000 7f8de5cce700 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x7f8de002f060 0x7f8de00556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:53.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.776+0000 7f8de5cce700 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:7 8.c 8:30000000:%01:::head [pgnls start_epoch 65 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e65) v8 -- 0x555dbf9fc290 con 0x7f8de002f060 2023-10-11T09:45:53.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.776+0000 7f8de54cd700 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x7f8de002f060 0x7f8de00556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:53.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.776+0000 7f8de54cd700 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x7f8de002f060 0x7f8de00556b0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:53.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.776+0000 7f8de54cd700 1 -- 172.21.15.78:0/3557916786 <== osd.0 v2:172.21.15.78:6800/35735 1 ==== osd_op_reply(7 [pgnls start_epoch 65 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8dd8016070 con 0x7f8de002f060 2023-10-11T09:45:53.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.776+0000 7f8de54cd700 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:8 8.1c 8:38000000:%01:::head [pgnls start_epoch 65 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e65) v8 -- 0x555dbf9fc290 con 0x555dbf9512b0 2023-10-11T09:45:53.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.780+0000 7f8de4ccc700 1 -- 172.21.15.78:0/3557916786 <== osd.5 v2:172.21.15.133:6816/35899 3 ==== osd_op_reply(8 [pgnls start_epoch 65 out=73b] v60'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f8ddc010800 con 0x555dbf9512b0 2023-10-11T09:45:53.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.780+0000 7f8de4ccc700 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:9 8.2 8:40000000:%01:::head [pgnls start_epoch 65 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e65) v8 -- 0x555dbf9fc290 con 0x7f8ddc00b860 2023-10-11T09:45:53.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.780+0000 7f8de5cce700 1 -- 172.21.15.78:0/3557916786 <== osd.4 v2:172.21.15.133:6808/35898 3 ==== osd_op_reply(9 [pgnls start_epoch 65 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8de0022070 con 0x7f8ddc00b860 2023-10-11T09:45:53.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.780+0000 7f8de5cce700 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x7f8de0054f80 0x7f8de0068750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:53.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.780+0000 7f8de5cce700 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:10 8.12 8:48000000:%01:::head [pgnls start_epoch 65 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e65) v8 -- 0x555dbf9fc290 con 0x7f8de0054f80 2023-10-11T09:45:53.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.780+0000 7f8de4ccc700 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x7f8de0054f80 0x7f8de0068750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:53.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.780+0000 7f8de4ccc700 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x7f8de0054f80 0x7f8de0068750 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:53.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.780+0000 7f8de4ccc700 1 -- 172.21.15.78:0/3557916786 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== osd_op_reply(10 [pgnls start_epoch 65 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ddc010800 con 0x7f8de0054f80 2023-10-11T09:45:53.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.780+0000 7f8de4ccc700 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x7f8ddc023950 0x7f8ddc022450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:53.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.780+0000 7f8de4ccc700 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:11 8.a 8:50000000:%01:::head [pgnls start_epoch 65 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e65) v8 -- 0x555dbf9fc290 con 0x7f8ddc023950 2023-10-11T09:45:53.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.784+0000 7f8de5cce700 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x7f8ddc023950 0x7f8ddc022450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:53.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.784+0000 7f8de5cce700 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x7f8ddc023950 0x7f8ddc022450 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:53.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.788+0000 7f8de5cce700 1 -- 172.21.15.78:0/3557916786 <== osd.1 v2:172.21.15.78:6809/35734 1 ==== osd_op_reply(11 [pgnls start_epoch 65 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8de0022070 con 0x7f8ddc023950 2023-10-11T09:45:53.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.788+0000 7f8de5cce700 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:12 8.1a 8:58000000:%01:::head [pgnls start_epoch 65 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e65) v8 -- 0x555dbf9fc290 con 0x7f8de002f060 2023-10-11T09:45:53.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.792+0000 7f8de54cd700 1 -- 172.21.15.78:0/3557916786 <== osd.0 v2:172.21.15.78:6800/35735 2 ==== osd_op_reply(12 [pgnls start_epoch 65 out=49b] v60'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8dd8016070 con 0x7f8de002f060 2023-10-11T09:45:53.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.792+0000 7f8de54cd700 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:13 8.6 8:60000000:%01:::head [pgnls start_epoch 65 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e65) v8 -- 0x555dbf9fc290 con 0x7f8de002f060 2023-10-11T09:45:53.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.792+0000 7f8de54cd700 1 -- 172.21.15.78:0/3557916786 <== osd.0 v2:172.21.15.78:6800/35735 3 ==== osd_op_reply(13 [pgnls start_epoch 65 out=77b] v64'21 uv21 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x7f8dd8016070 con 0x7f8de002f060 2023-10-11T09:45:53.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.792+0000 7f8de54cd700 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:14 8.16 8:68000000:%01:::head [pgnls start_epoch 65 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e65) v8 -- 0x555dbf9fc290 con 0x555dbf968830 2023-10-11T09:45:53.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.792+0000 7f8de4ccc700 1 -- 172.21.15.78:0/3557916786 <== osd.3 v2:172.21.15.78:6824/35738 2 ==== osd_op_reply(14 [pgnls start_epoch 65 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ddc010800 con 0x555dbf968830 2023-10-11T09:45:53.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.792+0000 7f8de4ccc700 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:15 8.e 8:70000000:%01:::head [pgnls start_epoch 65 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e65) v8 -- 0x555dbf9fc290 con 0x7f8de002f060 2023-10-11T09:45:53.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.792+0000 7f8de54cd700 1 -- 172.21.15.78:0/3557916786 <== osd.0 v2:172.21.15.78:6800/35735 4 ==== osd_op_reply(15 [pgnls start_epoch 65 out=73b] v60'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f8dd8016070 con 0x7f8de002f060 2023-10-11T09:45:53.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.792+0000 7f8de54cd700 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:16 8.1e 8:78000000:%01:::head [pgnls start_epoch 65 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e65) v8 -- 0x555dbf9fc290 con 0x555dbf968830 2023-10-11T09:45:53.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.792+0000 7f8de4ccc700 1 -- 172.21.15.78:0/3557916786 <== osd.3 v2:172.21.15.78:6824/35738 3 ==== osd_op_reply(16 [pgnls start_epoch 65 out=49b] v60'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ddc010800 con 0x555dbf968830 2023-10-11T09:45:53.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.792+0000 7f8de4ccc700 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:17 8.1 8:80000000:%01:::head [pgnls start_epoch 65 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e65) v8 -- 0x555dbf9fc290 con 0x7f8ddc023950 2023-10-11T09:45:53.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.792+0000 7f8de5cce700 1 -- 172.21.15.78:0/3557916786 <== osd.1 v2:172.21.15.78:6809/35734 2 ==== osd_op_reply(17 [pgnls start_epoch 65 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8de0022070 con 0x7f8ddc023950 2023-10-11T09:45:53.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.792+0000 7f8de5cce700 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:18 8.11 8:88000000:%01:::head [pgnls start_epoch 65 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e65) v8 -- 0x555dbf9fc290 con 0x555dbf9512b0 2023-10-11T09:45:53.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.796+0000 7f8de4ccc700 1 -- 172.21.15.78:0/3557916786 <== osd.5 v2:172.21.15.133:6816/35899 4 ==== osd_op_reply(18 [pgnls start_epoch 65 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ddc010800 con 0x555dbf9512b0 2023-10-11T09:45:53.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.796+0000 7f8de4ccc700 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:19 8.9 8:90000000:%01:::head [pgnls start_epoch 65 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e65) v8 -- 0x555dbf9fc290 con 0x7f8de0054f80 2023-10-11T09:45:53.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.796+0000 7f8de4ccc700 1 -- 172.21.15.78:0/3557916786 <== osd.6 v2:172.21.15.133:6800/35900 2 ==== osd_op_reply(19 [pgnls start_epoch 65 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ddc010800 con 0x7f8de0054f80 2023-10-11T09:45:53.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.796+0000 7f8de4ccc700 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:20 8.19 8:98000000:%01:::head [pgnls start_epoch 65 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e65) v8 -- 0x555dbf9fc290 con 0x555dbf9512b0 2023-10-11T09:45:53.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.796+0000 7f8de4ccc700 1 -- 172.21.15.78:0/3557916786 <== osd.5 v2:172.21.15.133:6816/35899 5 ==== osd_op_reply(20 [pgnls start_epoch 65 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ddc010800 con 0x555dbf9512b0 2023-10-11T09:45:53.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.796+0000 7f8de4ccc700 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:21 8.5 8:a0000000:%01:::head [pgnls start_epoch 65 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e65) v8 -- 0x555dbf9fc290 con 0x7f8de0054f80 2023-10-11T09:45:53.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.800+0000 7f8de4ccc700 1 -- 172.21.15.78:0/3557916786 <== osd.6 v2:172.21.15.133:6800/35900 3 ==== osd_op_reply(21 [pgnls start_epoch 65 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ddc010800 con 0x7f8de0054f80 2023-10-11T09:45:53.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.800+0000 7f8de4ccc700 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:22 8.15 8:a8000000:%01:::head [pgnls start_epoch 65 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e65) v8 -- 0x555dbf9fc290 con 0x7f8ddc00e950 2023-10-11T09:45:53.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.800+0000 7f8de54cd700 1 -- 172.21.15.78:0/3557916786 <== osd.7 v2:172.21.15.133:6824/35901 2 ==== osd_op_reply(22 [pgnls start_epoch 65 out=97b] v60'3 uv3 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x7f8dd8016070 con 0x7f8ddc00e950 2023-10-11T09:45:53.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.800+0000 7f8de54cd700 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:23 8.d 8:b0000000:%01:::head [pgnls start_epoch 65 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e65) v8 -- 0x555dbf9fc290 con 0x555dbf9512b0 2023-10-11T09:45:53.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.800+0000 7f8de4ccc700 1 -- 172.21.15.78:0/3557916786 <== osd.5 v2:172.21.15.133:6816/35899 6 ==== osd_op_reply(23 [pgnls start_epoch 65 out=74b] v60'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f8ddc010800 con 0x555dbf9512b0 2023-10-11T09:45:53.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.800+0000 7f8de4ccc700 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:24 8.1d 8:b8000000:%01:::head [pgnls start_epoch 65 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e65) v8 -- 0x555dbf9fc290 con 0x7f8de002f060 2023-10-11T09:45:53.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.800+0000 7f8de54cd700 1 -- 172.21.15.78:0/3557916786 <== osd.0 v2:172.21.15.78:6800/35735 5 ==== osd_op_reply(24 [pgnls start_epoch 65 out=73b] v60'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f8dd8016070 con 0x7f8de002f060 2023-10-11T09:45:53.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.800+0000 7f8de54cd700 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:25 8.3 8:c0000000:%01:::head [pgnls start_epoch 65 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e65) v8 -- 0x555dbf9fc290 con 0x7f8ddc023950 2023-10-11T09:45:53.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.800+0000 7f8de5cce700 1 -- 172.21.15.78:0/3557916786 <== osd.1 v2:172.21.15.78:6809/35734 3 ==== osd_op_reply(25 [pgnls start_epoch 65 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8de0022070 con 0x7f8ddc023950 2023-10-11T09:45:53.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.800+0000 7f8de5cce700 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:26 8.13 8:c8000000:%01:::head [pgnls start_epoch 65 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e65) v8 -- 0x555dbf9fc290 con 0x555dbf968830 2023-10-11T09:45:53.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.804+0000 7f8de4ccc700 1 -- 172.21.15.78:0/3557916786 <== osd.3 v2:172.21.15.78:6824/35738 4 ==== osd_op_reply(26 [pgnls start_epoch 65 out=98b] v64'3 uv3 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x7f8ddc010800 con 0x555dbf968830 2023-10-11T09:45:53.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.804+0000 7f8de4ccc700 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:27 8.b 8:d0000000:%01:::head [pgnls start_epoch 65 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e65) v8 -- 0x555dbf9fc290 con 0x555dbf968830 2023-10-11T09:45:53.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.804+0000 7f8de4ccc700 1 -- 172.21.15.78:0/3557916786 <== osd.3 v2:172.21.15.78:6824/35738 5 ==== osd_op_reply(27 [pgnls start_epoch 65 out=76b] v64'3 uv3 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x7f8ddc010800 con 0x555dbf968830 2023-10-11T09:45:53.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.804+0000 7f8de4ccc700 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:28 8.1b 8:d8000000:%01:::head [pgnls start_epoch 65 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e65) v8 -- 0x555dbf9fc290 con 0x7f8ddc00b860 2023-10-11T09:45:53.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.804+0000 7f8de5cce700 1 -- 172.21.15.78:0/3557916786 <== osd.4 v2:172.21.15.133:6808/35898 4 ==== osd_op_reply(28 [pgnls start_epoch 65 out=49b] v60'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8de0022070 con 0x7f8ddc00b860 2023-10-11T09:45:53.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.804+0000 7f8de5cce700 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:29 8.7 8:e0000000:%01:::head [pgnls start_epoch 65 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e65) v8 -- 0x555dbf9fc290 con 0x7f8ddc023950 2023-10-11T09:45:53.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.804+0000 7f8de5cce700 1 -- 172.21.15.78:0/3557916786 <== osd.1 v2:172.21.15.78:6809/35734 4 ==== osd_op_reply(29 [pgnls start_epoch 65 out=73b] v60'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f8de0022070 con 0x7f8ddc023950 2023-10-11T09:45:53.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.804+0000 7f8de5cce700 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:30 8.17 8:e8000000:%01:::head [pgnls start_epoch 65 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e65) v8 -- 0x555dbf9fc290 con 0x7f8ddc00e950 2023-10-11T09:45:53.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.808+0000 7f8de54cd700 1 -- 172.21.15.78:0/3557916786 <== osd.7 v2:172.21.15.133:6824/35901 3 ==== osd_op_reply(30 [pgnls start_epoch 65 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8dd8016070 con 0x7f8ddc00e950 2023-10-11T09:45:53.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.808+0000 7f8de54cd700 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:31 8.f 8:f0000000:%01:::head [pgnls start_epoch 65 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e65) v8 -- 0x555dbf9fc290 con 0x555dbf968830 2023-10-11T09:45:53.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.808+0000 7f8de4ccc700 1 -- 172.21.15.78:0/3557916786 <== osd.3 v2:172.21.15.78:6824/35738 6 ==== osd_op_reply(31 [pgnls start_epoch 65 out=73b] v60'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f8ddc010800 con 0x555dbf968830 2023-10-11T09:45:53.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.808+0000 7f8de4ccc700 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:32 8.1f 8:f8000000:%01:::head [pgnls start_epoch 65 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e65) v8 -- 0x555dbf9fc290 con 0x7f8de002f060 2023-10-11T09:45:53.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.808+0000 7f8de54cd700 1 -- 172.21.15.78:0/3557916786 <== osd.0 v2:172.21.15.78:6800/35735 6 ==== osd_op_reply(32 [pgnls start_epoch 65 out=101b] v64'6 uv6 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x7f8dd8016070 con 0x7f8de002f060 2023-10-11T09:45:53.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.808+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:33 8.10 8:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e65) v8 -- 0x555dbf9683d0 con 0x7f8ddc00e950 2023-10-11T09:45:53.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.808+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:34 8.10 8:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e65) v8 -- 0x555dbfa82630 con 0x7f8ddc00e950 2023-10-11T09:45:53.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.808+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:35 8.10 8:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e65) v8 -- 0x555dbf967820 con 0x7f8ddc00e950 2023-10-11T09:45:53.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.808+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:36 8.10 8:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e65) v8 -- 0x555dbf925770 con 0x7f8ddc00e950 2023-10-11T09:45:53.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.808+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:37 8.10 8:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e65) v8 -- 0x555dbf925b90 con 0x7f8ddc00e950 2023-10-11T09:45:53.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.808+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:38 8.14 8:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e65) v8 -- 0x555dbfa9b940 con 0x7f8ddc00b860 2023-10-11T09:45:53.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.808+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:39 8.14 8:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e65) v8 -- 0x555dbfa9c1e0 con 0x7f8ddc00b860 2023-10-11T09:45:53.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.808+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:40 8.14 8:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e65) v8 -- 0x555dbfa9ca80 con 0x7f8ddc00b860 2023-10-11T09:45:53.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.808+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:41 8.14 8:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e65) v8 -- 0x555dbfa9d320 con 0x7f8ddc00b860 2023-10-11T09:45:53.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.808+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:42 8.1c 8:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e65) v8 -- 0x555dbfa9dbc0 con 0x555dbf9512b0 2023-10-11T09:45:53.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.808+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:43 8.6 8:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e65) v8 -- 0x555dbfa9e5d0 con 0x7f8de002f060 2023-10-11T09:45:53.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.808+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:44 8.e 8:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e65) v8 -- 0x555dbfa9f140 con 0x7f8de002f060 2023-10-11T09:45:53.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.808+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:45 8.15 8:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e65) v8 -- 0x555dbfa9fca0 con 0x7f8ddc00e950 2023-10-11T09:45:53.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.808+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:46 8.15 8:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e65) v8 -- 0x555dbfaa0830 con 0x7f8ddc00e950 2023-10-11T09:45:53.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.808+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:47 8.d 8:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e65) v8 -- 0x555dbfaa13c0 con 0x555dbf9512b0 2023-10-11T09:45:53.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.808+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:48 8.1d 8:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e65) v8 -- 0x555dbfaa1f00 con 0x7f8de002f060 2023-10-11T09:45:53.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.812+0000 7f8de54cd700 1 -- 172.21.15.78:0/3557916786 <== osd.0 v2:172.21.15.78:6800/35735 7 ==== osd_op_reply(48 605.00000000 [delete] v65'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8dd8016070 con 0x7f8de002f060 2023-10-11T09:45:53.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.812+0000 7f8de54cd700 1 -- 172.21.15.78:0/3557916786 <== osd.0 v2:172.21.15.78:6800/35735 8 ==== osd_op_reply(44 608.00000000 [delete] v65'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8dd8016070 con 0x7f8de002f060 2023-10-11T09:45:53.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.812+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:49 8.13 8:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e65) v8 -- 0x555dbfa9f140 con 0x555dbf968830 2023-10-11T09:45:53.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.812+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:50 8.13 8:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e65) v8 -- 0x555dbfaa0830 con 0x555dbf968830 2023-10-11T09:45:53.818 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.812+0000 7f8de54cd700 1 -- 172.21.15.78:0/3557916786 <== osd.0 v2:172.21.15.78:6800/35735 9 ==== osd_op_reply(43 mds0_openfiles.0 [delete] v65'22 uv22 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f8dd8016070 con 0x7f8de002f060 2023-10-11T09:45:53.818 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.812+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:51 8.b 8:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e65) v8 -- 0x555dbfa9fca0 con 0x555dbf968830 2023-10-11T09:45:53.819 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.816+0000 7f8de54cd700 1 -- 172.21.15.78:0/3557916786 <== osd.7 v2:172.21.15.133:6824/35901 4 ==== osd_op_reply(33 601.00000000 [delete] v65'11 uv11 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8dd8016070 con 0x7f8ddc00e950 2023-10-11T09:45:53.819 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.816+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:52 8.7 8:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e65) v8 -- 0x555dbfaa1ec0 con 0x7f8ddc023950 2023-10-11T09:45:53.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.816+0000 7f8de4ccc700 1 -- 172.21.15.78:0/3557916786 <== osd.3 v2:172.21.15.78:6824/35738 7 ==== osd_op_reply(49 mds0_inotable [delete] v65'4 uv4 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8ddc010800 con 0x555dbf968830 2023-10-11T09:45:53.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.816+0000 7f8de54cd700 1 -- 172.21.15.78:0/3557916786 <== osd.7 v2:172.21.15.133:6824/35901 5 ==== osd_op_reply(45 604.00000000 [delete] v65'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8dd8016070 con 0x7f8ddc00e950 2023-10-11T09:45:53.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.816+0000 7f8de54cd700 1 -- 172.21.15.78:0/3557916786 <== osd.7 v2:172.21.15.133:6824/35901 6 ==== osd_op_reply(34 602.00000000 [delete] v65'12 uv12 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8dd8016070 con 0x7f8ddc00e950 2023-10-11T09:45:53.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.816+0000 7f8de5cce700 1 -- 172.21.15.78:0/3557916786 <== osd.4 v2:172.21.15.133:6808/35898 5 ==== osd_op_reply(38 1.00000000.inode [delete] v65'90 uv90 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f8de0072040 con 0x7f8ddc00b860 2023-10-11T09:45:53.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.816+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:53 8.f 8:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e65) v8 -- 0x555dbf9248a0 con 0x555dbf968830 2023-10-11T09:45:53.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.816+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:54 8.1f 8:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e65) v8 -- 0x555dbfa9f690 con 0x7f8de002f060 2023-10-11T09:45:53.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.816+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:55 8.1f 8:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e65) v8 -- 0x555dbfaa2820 con 0x7f8de002f060 2023-10-11T09:45:53.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.816+0000 7f8de5cce700 1 -- 172.21.15.78:0/3557916786 <== osd.4 v2:172.21.15.133:6808/35898 6 ==== osd_op_reply(39 200.00000000 [delete] v65'91 uv91 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8de0072040 con 0x7f8ddc00b860 2023-10-11T09:45:53.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.816+0000 7f8de4ccc700 1 -- 172.21.15.78:0/3557916786 <== osd.5 v2:172.21.15.133:6816/35899 7 ==== osd_op_reply(47 mds_snaptable [delete] v65'2 uv2 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8ddc010800 con 0x555dbf9512b0 2023-10-11T09:45:53.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.816+0000 7f8de4ccc700 1 -- 172.21.15.78:0/3557916786 <== osd.5 v2:172.21.15.133:6816/35899 8 ==== osd_op_reply(42 607.00000000 [delete] v65'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8ddc010800 con 0x555dbf9512b0 2023-10-11T09:45:53.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.816+0000 7f8de54cd700 1 -- 172.21.15.78:0/3557916786 <== osd.7 v2:172.21.15.133:6824/35901 7 ==== osd_op_reply(46 500.00000000 [delete] v65'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8dd8016070 con 0x7f8ddc00e950 2023-10-11T09:45:53.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.816+0000 7f8de4ccc700 1 -- 172.21.15.78:0/3557916786 <== osd.3 v2:172.21.15.78:6824/35738 8 ==== osd_op_reply(51 mds0_sessionmap [delete] v65'4 uv4 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f8ddc010800 con 0x555dbf968830 2023-10-11T09:45:53.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.816+0000 7f8de54cd700 1 -- 172.21.15.78:0/3557916786 <== osd.7 v2:172.21.15.133:6824/35901 8 ==== osd_op_reply(35 10000000000.00000000 [delete] v65'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f8dd8016070 con 0x7f8ddc00e950 2023-10-11T09:45:53.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.820+0000 7f8de54cd700 1 -- 172.21.15.78:0/3557916786 <== osd.7 v2:172.21.15.133:6824/35901 9 ==== osd_op_reply(36 600.00000000 [delete] v65'14 uv14 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8dd8016070 con 0x7f8ddc00e950 2023-10-11T09:45:53.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.820+0000 7f8de54cd700 1 -- 172.21.15.78:0/3557916786 <== osd.7 v2:172.21.15.133:6824/35901 10 ==== osd_op_reply(37 603.00000000 [delete] v65'15 uv15 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8dd8016070 con 0x7f8ddc00e950 2023-10-11T09:45:53.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.820+0000 7f8de4ccc700 1 -- 172.21.15.78:0/3557916786 <== osd.3 v2:172.21.15.78:6824/35738 9 ==== osd_op_reply(50 100.00000000 [delete] v65'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8ddc010800 con 0x555dbf968830 2023-10-11T09:45:53.824 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.820+0000 7f8de5cce700 1 -- 172.21.15.78:0/3557916786 <== osd.4 v2:172.21.15.133:6808/35898 7 ==== osd_op_reply(40 200.00000001 [delete] v65'92 uv92 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8de0072040 con 0x7f8ddc00b860 2023-10-11T09:45:53.824 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.820+0000 7f8de5cce700 1 -- 172.21.15.78:0/3557916786 <== osd.4 v2:172.21.15.133:6808/35898 8 ==== osd_op_reply(41 606.00000000 [delete] v65'93 uv93 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8de0072040 con 0x7f8ddc00b860 2023-10-11T09:45:53.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.824+0000 7f8de4ccc700 1 -- 172.21.15.78:0/3557916786 <== osd.3 v2:172.21.15.78:6824/35738 10 ==== osd_op_reply(53 400.00000000 [delete] v65'2 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8ddc010800 con 0x555dbf968830 2023-10-11T09:45:53.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.824+0000 7f8de54cd700 1 -- 172.21.15.78:0/3557916786 <== osd.0 v2:172.21.15.78:6800/35735 10 ==== osd_op_reply(54 100.00000000.inode [delete] v65'7 uv7 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f8dd8016070 con 0x7f8de002f060 2023-10-11T09:45:53.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.824+0000 7f8de5cce700 1 -- 172.21.15.78:0/3557916786 <== osd.1 v2:172.21.15.78:6809/35734 5 ==== osd_op_reply(52 609.00000000 [delete] v65'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8de0072040 con 0x7f8ddc023950 2023-10-11T09:45:53.828 INFO:teuthology.orchestra.run.smithi078.stdout:Removed 23 objects 2023-10-11T09:45:53.828 INFO:teuthology.orchestra.run.smithi078.stdout:successfully purged pool cephfs_metadata 2023-10-11T09:45:53.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.824+0000 7f8de54cd700 1 -- 172.21.15.78:0/3557916786 <== osd.0 v2:172.21.15.78:6800/35735 11 ==== osd_op_reply(55 1.00000000 [delete] v65'8 uv8 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f8dd8016070 con 0x7f8de002f060 2023-10-11T09:45:53.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.824+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x7f8de002f060 msgr2=0x7f8de00556b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:53.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.824+0000 7f8de7998140 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x7f8de002f060 0x7f8de00556b0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:53.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.824+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x7f8ddc023950 msgr2=0x7f8ddc022450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:53.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.824+0000 7f8de7998140 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x7f8ddc023950 0x7f8ddc022450 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:53.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.824+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x555dbf968830 msgr2=0x555dbf924360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:53.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.824+0000 7f8de7998140 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x555dbf968830 0x555dbf924360 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:53.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.824+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x7f8ddc00b860 msgr2=0x7f8ddc00c360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:53.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.824+0000 7f8de7998140 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x7f8ddc00b860 0x7f8ddc00c360 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:53.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.824+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x555dbf9512b0 msgr2=0x7f8dd8028930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:53.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.824+0000 7f8de7998140 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x555dbf9512b0 0x7f8dd8028930 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:53.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.824+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x7f8de0054f80 msgr2=0x7f8de0068750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:53.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.824+0000 7f8de7998140 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x7f8de0054f80 0x7f8de0068750 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:53.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.824+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x7f8ddc00e950 msgr2=0x7f8ddc00d3d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:53.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.824+0000 7f8de7998140 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x7f8ddc00e950 0x7f8ddc00d3d0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:53.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.824+0000 7f8de5cce700 1 -- 172.21.15.78:0/3557916786 reap_dead start 2023-10-11T09:45:53.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.824+0000 7f8de5cce700 1 -- 172.21.15.78:0/3557916786 reap_dead start 2023-10-11T09:45:53.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.824+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8dc8068c40 msgr2=0x7f8dc806b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:53.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.824+0000 7f8de7998140 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8dc8068c40 0x7f8dc806b0f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x555dbf9f6e60 tx=0x7f8dd8009000 comp rx=0 tx=0).stop 2023-10-11T09:45:53.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.824+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dbf9f4c90 msgr2=0x555dbfa82010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:53.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.824+0000 7f8de7998140 1 --2- 172.21.15.78:0/3557916786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dbf9f4c90 0x555dbfa82010 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x555dbf9115f0 tx=0x7f8de0003330 comp rx=0 tx=0).stop 2023-10-11T09:45:53.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.824+0000 7f8de5cce700 1 -- 172.21.15.78:0/3557916786 reap_dead start 2023-10-11T09:45:53.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.824+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 shutdown_connections 2023-10-11T09:45:53.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.824+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 >> 172.21.15.78:0/3557916786 conn(0x555dbf952330 msgr2=0x555dbf9516b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:53.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.824+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 shutdown_connections 2023-10-11T09:45:53.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:53.824+0000 7f8de7998140 1 -- 172.21.15.78:0/3557916786 wait complete. 2023-10-11T09:45:53.835 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs reset cephfs --yes-i-really-mean-it 2023-10-11T09:45:54.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.008+0000 7f397b4fc700 1 Processor -- start 2023-10-11T09:45:54.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.008+0000 7f397b4fc700 1 -- start start 2023-10-11T09:45:54.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.008+0000 7f397b4fc700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f397415c900 0x7f39740b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:54.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.008+0000 7f397b4fc700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f397406fde0 con 0x7f39740ce7f0 2023-10-11T09:45:54.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.008+0000 7f397b4fc700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39740ae000 con 0x7f39740ad290 2023-10-11T09:45:54.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.008+0000 7f397b4fc700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39740705f0 con 0x7f397415c900 2023-10-11T09:45:54.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.012+0000 7f397acfb700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f39740ce7f0 0x7f397406bfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:41852/0 (socket says 172.21.15.78:41852) 2023-10-11T09:45:54.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.012+0000 7f397acfb700 1 -- 172.21.15.78:0/481513286 learned_addr learned my addr 172.21.15.78:0/481513286 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:45:54.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.012+0000 7f397a4fa700 1 --2- 172.21.15.78:0/481513286 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f397415c900 0x7f39740b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:54.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.012+0000 7f39794f8700 1 -- 172.21.15.78:0/481513286 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3609535088 0 0) 0x7f397406fde0 con 0x7f39740ce7f0 2023-10-11T09:45:54.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.012+0000 7f39794f8700 1 -- 172.21.15.78:0/481513286 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39600035d0 con 0x7f39740ce7f0 2023-10-11T09:45:54.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.012+0000 7f39794f8700 1 -- 172.21.15.78:0/481513286 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1979499434 0 0) 0x7f39600035d0 con 0x7f39740ce7f0 2023-10-11T09:45:54.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.012+0000 7f39794f8700 1 -- 172.21.15.78:0/481513286 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f397415c900 msgr2=0x7f39740b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:54.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.012+0000 7f39794f8700 1 --2- 172.21.15.78:0/481513286 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f397415c900 0x7f39740b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:54.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.012+0000 7f39794f8700 1 -- 172.21.15.78:0/481513286 >> v1:172.21.15.133:6789/0 conn(0x7f39740ad290 legacy=0x7f3974071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:54.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.012+0000 7f39794f8700 1 -- 172.21.15.78:0/481513286 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39740ad7e0 con 0x7f39740ce7f0 2023-10-11T09:45:54.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.012+0000 7f39794f8700 1 -- 172.21.15.78:0/481513286 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f396c0019f0 con 0x7f39740ce7f0 2023-10-11T09:45:54.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.012+0000 7f39794f8700 1 -- 172.21.15.78:0/481513286 >> v1:172.21.15.78:6789/0 conn(0x7f39740ce7f0 legacy=0x7f397406bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:54.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.012+0000 7f39794f8700 1 --2- 172.21.15.78:0/481513286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3960003a50 0x7f3960003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:54.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.012+0000 7f39794f8700 1 --2- 172.21.15.78:0/481513286 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f397415c900 0x7f3960004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:54.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.012+0000 7f39794f8700 1 --2- 172.21.15.78:0/481513286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3960004db0 0x7f3960005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:54.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.012+0000 7f39794f8700 1 -- 172.21.15.78:0/481513286 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39740705f0 con 0x7f3960003a50 2023-10-11T09:45:54.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.012+0000 7f39794f8700 1 -- 172.21.15.78:0/481513286 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396000ba30 con 0x7f3960004db0 2023-10-11T09:45:54.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.012+0000 7f39794f8700 1 -- 172.21.15.78:0/481513286 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f396000bbc0 con 0x7f397415c900 2023-10-11T09:45:54.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.012+0000 7f397a4fa700 1 --2- 172.21.15.78:0/481513286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3960003a50 0x7f3960003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:54.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.012+0000 7f397a4fa700 1 -- 172.21.15.78:0/481513286 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f397415c900 msgr2=0x7f3960004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:54.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.012+0000 7f397a4fa700 1 --2- 172.21.15.78:0/481513286 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f397415c900 0x7f3960004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:54.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.012+0000 7f397a4fa700 1 -- 172.21.15.78:0/481513286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3960004db0 msgr2=0x7f3960005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:54.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.012+0000 7f397a4fa700 1 --2- 172.21.15.78:0/481513286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3960004db0 0x7f3960005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:54.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.012+0000 7f397a4fa700 1 -- 172.21.15.78:0/481513286 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39600035d0 con 0x7f3960003a50 2023-10-11T09:45:54.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.012+0000 7f397a4fa700 1 --2- 172.21.15.78:0/481513286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3960003a50 0x7f3960003e20 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f3970009b70 tx=0x7f39600058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb6de3e3c9650de3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:54.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.012+0000 7f39794f8700 1 -- 172.21.15.78:0/481513286 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397000e6c0 con 0x7f3960003a50 2023-10-11T09:45:54.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.012+0000 7f39794f8700 1 -- 172.21.15.78:0/481513286 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397000e820 con 0x7f3960003a50 2023-10-11T09:45:54.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.012+0000 7f39794f8700 1 -- 172.21.15.78:0/481513286 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397000eb90 con 0x7f3960003a50 2023-10-11T09:45:54.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.016+0000 7f397b4fc700 1 -- 172.21.15.78:0/481513286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3960003a50 msgr2=0x7f3960003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:54.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.016+0000 7f397b4fc700 1 --2- 172.21.15.78:0/481513286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3960003a50 0x7f3960003e20 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f3970009b70 tx=0x7f39600058a0 comp rx=0 tx=0).stop 2023-10-11T09:45:54.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.016+0000 7f397acfb700 1 -- 172.21.15.78:0/481513286 reap_dead start 2023-10-11T09:45:54.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.016+0000 7f397b4fc700 1 -- 172.21.15.78:0/481513286 shutdown_connections 2023-10-11T09:45:54.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.016+0000 7f397b4fc700 1 -- 172.21.15.78:0/481513286 >> 172.21.15.78:0/481513286 conn(0x7f397406a060 msgr2=0x7f397406a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:54.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.016+0000 7f397b4fc700 1 -- 172.21.15.78:0/481513286 shutdown_connections 2023-10-11T09:45:54.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.016+0000 7f397b4fc700 1 -- 172.21.15.78:0/481513286 wait complete. 2023-10-11T09:45:54.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.020+0000 7f397b4fc700 1 Processor -- start 2023-10-11T09:45:54.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.020+0000 7f397b4fc700 1 -- start start 2023-10-11T09:45:54.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.020+0000 7f397b4fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3960003a50 0x7f3974154000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:54.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.020+0000 7f397b4fc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39740b3210 0x7f3974154da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:54.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.020+0000 7f397b4fc700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f39741552e0 0x7f3974145310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:54.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.020+0000 7f397b4fc700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3974155760 con 0x7f3960003a50 2023-10-11T09:45:54.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.020+0000 7f397b4fc700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3974155a40 con 0x7f39740b3210 2023-10-11T09:45:54.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.020+0000 7f397b4fc700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3974153520 con 0x7f39741552e0 2023-10-11T09:45:54.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.020+0000 7f397a4fa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3960003a50 0x7f3974154000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:54.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.020+0000 7f397a4fa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3960003a50 0x7f3974154000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50718/0 (socket says 172.21.15.78:50718) 2023-10-11T09:45:54.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.020+0000 7f397a4fa700 1 -- 172.21.15.78:0/114173031 learned_addr learned my addr 172.21.15.78:0/114173031 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:54.030 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.020+0000 7f397acfb700 1 --2- 172.21.15.78:0/114173031 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f39741552e0 0x7f3974145310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:54.030 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.020+0000 7f3979cf9700 1 --2- 172.21.15.78:0/114173031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39740b3210 0x7f3974154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:54.030 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.020+0000 7f397a4fa700 1 -- 172.21.15.78:0/114173031 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f39741552e0 msgr2=0x7f3974145310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:54.030 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.020+0000 7f397a4fa700 1 --2- 172.21.15.78:0/114173031 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f39741552e0 0x7f3974145310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:54.030 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.020+0000 7f397a4fa700 1 -- 172.21.15.78:0/114173031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39740b3210 msgr2=0x7f3974154da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:54.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.020+0000 7f397a4fa700 1 --2- 172.21.15.78:0/114173031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39740b3210 0x7f3974154da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:54.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.020+0000 7f397a4fa700 1 -- 172.21.15.78:0/114173031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3974153800 con 0x7f3960003a50 2023-10-11T09:45:54.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.020+0000 7f3979cf9700 1 --2- 172.21.15.78:0/114173031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39740b3210 0x7f3974154da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:45:54.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.020+0000 7f397a4fa700 1 --2- 172.21.15.78:0/114173031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3960003a50 0x7f3974154000 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f3960005820 tx=0x7f39700047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:54.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.024+0000 7f396affd700 1 -- 172.21.15.78:0/114173031 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397000e6c0 con 0x7f3960003a50 2023-10-11T09:45:54.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.024+0000 7f396affd700 1 -- 172.21.15.78:0/114173031 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3970013e70 con 0x7f3960003a50 2023-10-11T09:45:54.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.024+0000 7f397b4fc700 1 -- 172.21.15.78:0/114173031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3974153ae0 con 0x7f3960003a50 2023-10-11T09:45:54.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.024+0000 7f396affd700 1 -- 172.21.15.78:0/114173031 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3970004250 con 0x7f3960003a50 2023-10-11T09:45:54.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.024+0000 7f397b4fc700 1 -- 172.21.15.78:0/114173031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3974154540 con 0x7f3960003a50 2023-10-11T09:45:54.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.024+0000 7f397b4fc700 1 -- 172.21.15.78:0/114173031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f397406cdc0 con 0x7f3960003a50 2023-10-11T09:45:54.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.024+0000 7f396affd700 1 -- 172.21.15.78:0/114173031 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f39700043b0 con 0x7f3960003a50 2023-10-11T09:45:54.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.024+0000 7f396affd700 1 --2- 172.21.15.78:0/114173031 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3958065ed0 0x7f3958068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:54.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.024+0000 7f396affd700 1 -- 172.21.15.78:0/114173031 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6552+0+0 (secure 0 0 0) 0x7f3970085450 con 0x7f3960003a50 2023-10-11T09:45:54.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.028+0000 7f3979cf9700 1 --2- 172.21.15.78:0/114173031 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3958065ed0 0x7f3958068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:54.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.028+0000 7f3979cf9700 1 --2- 172.21.15.78:0/114173031 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3958065ed0 0x7f3958068380 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3974153290 tx=0x7f3964009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:54.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.032+0000 7f396affd700 1 -- 172.21.15.78:0/114173031 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f39700528c0 con 0x7f3960003a50 2023-10-11T09:45:54.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.248+0000 7f397b4fc700 1 -- 172.21.15.78:0/114173031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs reset", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3974065d40 con 0x7f3960003a50 2023-10-11T09:45:54.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.320+0000 7f396affd700 1 -- 172.21.15.78:0/114173031 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs reset", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v40) v1 ==== 107+0+0 (secure 0 0 0) 0x7f3970055ee0 con 0x7f3960003a50 2023-10-11T09:45:54.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.324+0000 7f397b4fc700 1 -- 172.21.15.78:0/114173031 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3958065ed0 msgr2=0x7f3958068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:54.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.324+0000 7f397b4fc700 1 --2- 172.21.15.78:0/114173031 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3958065ed0 0x7f3958068380 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3974153290 tx=0x7f3964009040 comp rx=0 tx=0).stop 2023-10-11T09:45:54.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.324+0000 7f397b4fc700 1 -- 172.21.15.78:0/114173031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3960003a50 msgr2=0x7f3974154000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:54.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.324+0000 7f397b4fc700 1 --2- 172.21.15.78:0/114173031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3960003a50 0x7f3974154000 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f3960005820 tx=0x7f39700047e0 comp rx=0 tx=0).stop 2023-10-11T09:45:54.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.328+0000 7f397b4fc700 1 -- 172.21.15.78:0/114173031 shutdown_connections 2023-10-11T09:45:54.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.328+0000 7f397b4fc700 1 --2- 172.21.15.78:0/114173031 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3958065ed0 0x7f3958068380 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:54.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.328+0000 7f397b4fc700 1 --2- 172.21.15.78:0/114173031 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f39741552e0 0x7f3974145310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:54.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.328+0000 7f397b4fc700 1 --2- 172.21.15.78:0/114173031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39740b3210 0x7f3974154da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:54.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.328+0000 7f397b4fc700 1 --2- 172.21.15.78:0/114173031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3960003a50 0x7f3974154000 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:54.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.328+0000 7f397b4fc700 1 -- 172.21.15.78:0/114173031 >> 172.21.15.78:0/114173031 conn(0x7f397406a060 msgr2=0x7f397415b010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:54.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.328+0000 7f397b4fc700 1 -- 172.21.15.78:0/114173031 shutdown_connections 2023-10-11T09:45:54.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.328+0000 7f397b4fc700 1 -- 172.21.15.78:0/114173031 wait complete. 2023-10-11T09:45:54.336 INFO:tasks.ceph.mds.a.smithi078.stderr:2023-10-11T09:45:54.332+0000 7f4ef13ee700 -1 mds.0.snaptable: load_2 could not read table: -2 2023-10-11T09:45:54.336 INFO:tasks.ceph.mds.a.smithi078.stderr:2023-10-11T09:45:54.332+0000 7f4ef13ee700 -1 log_channel(cluster) log [ERR] : error reading table object 'mds_snaptable' -2 ((2) No such file or directory) 2023-10-11T09:45:54.337 INFO:tasks.ceph.mon.a.smithi078.stderr:2023-10-11T09:45:54.332+0000 7f90131f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-11T09:45:54.338 INFO:tasks.ceph.mon.a.smithi078.stderr:2023-10-11T09:45:54.332+0000 7f90131f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-10-11T09:45:54.353 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-11T09:45:54.360 INFO:tasks.ceph.mds.a.smithi078.stderr: -30> 2023-10-11T09:45:54.332+0000 7f4ef13ee700 -1 mds.0.snaptable: load_2 could not read table: -2 2023-10-11T09:45:54.361 INFO:tasks.ceph.mds.a.smithi078.stderr: -29> 2023-10-11T09:45:54.332+0000 7f4ef13ee700 -1 log_channel(cluster) log [ERR] : error reading table object 'mds_snaptable' -2 ((2) No such file or directory) 2023-10-11T09:45:54.410 INFO:tasks.ceph.mds.a.smithi078.stdout:starting mds.a at 2023-10-11T09:45:54.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.528+0000 7f9dc1fd2700 1 Processor -- start 2023-10-11T09:45:54.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.528+0000 7f9dc1fd2700 1 -- start start 2023-10-11T09:45:54.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.528+0000 7f9dc1fd2700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9dbc0b3240 0x7f9dbc0d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:54.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.528+0000 7f9dc1fd2700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9dbc0b3a90 0x7f9dbc0b85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:54.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.528+0000 7f9dc1fd2700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dbc0b47f0 con 0x7f9dbc0b3610 2023-10-11T09:45:54.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.528+0000 7f9dc1fd2700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dbc0b8130 con 0x7f9dbc0b3240 2023-10-11T09:45:54.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.528+0000 7f9dc1fd2700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dbc0b3f30 con 0x7f9dbc0b3a90 2023-10-11T09:45:54.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.528+0000 7f9dbbfff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f9dbc0b3610 0x7f9dbc15d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:41876/0 (socket says 172.21.15.78:41876) 2023-10-11T09:45:54.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.528+0000 7f9dbbfff700 1 -- 172.21.15.78:0/3140566848 learned_addr learned my addr 172.21.15.78:0/3140566848 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:45:54.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.528+0000 7f9dc17d1700 1 --2- 172.21.15.78:0/3140566848 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9dbc0b3a90 0x7f9dbc0b85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:54.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.528+0000 7f9dc0fd0700 1 --2- 172.21.15.78:0/3140566848 >> v2:172.21.15.133:3300/0 conn(0x7f9dbc0b3240 0x7f9dbc0d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:54.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.532+0000 7f9dbb7fe700 1 -- 172.21.15.78:0/3140566848 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3992190993 0 0) 0x7f9da8001110 con 0x7f9dbc0b3610 2023-10-11T09:45:54.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.532+0000 7f9dbb7fe700 1 -- 172.21.15.78:0/3140566848 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dac0035d0 con 0x7f9dbc0b3610 2023-10-11T09:45:54.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.532+0000 7f9dc17d1700 1 -- 172.21.15.78:0/3140566848 >> v2:172.21.15.133:3300/0 conn(0x7f9dbc0b3240 msgr2=0x7f9dbc0d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:54.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.532+0000 7f9dc17d1700 1 --2- 172.21.15.78:0/3140566848 >> v2:172.21.15.133:3300/0 conn(0x7f9dbc0b3240 0x7f9dbc0d29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:54.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.532+0000 7f9dc17d1700 1 -- 172.21.15.78:0/3140566848 >> v1:172.21.15.78:6789/0 conn(0x7f9dbc0b3610 legacy=0x7f9dbc15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:54.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.532+0000 7f9dc17d1700 1 -- 172.21.15.78:0/3140566848 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dbc0b41e0 con 0x7f9dbc0b3a90 2023-10-11T09:45:54.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.532+0000 7f9dc17d1700 1 --2- 172.21.15.78:0/3140566848 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9dbc0b3a90 0x7f9dbc0b85c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9db4009d90 tx=0x7f9db400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d1fd61015bf6655 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:54.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.532+0000 7f9dbb7fe700 1 -- 172.21.15.78:0/3140566848 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db400e070 con 0x7f9dbc0b3a90 2023-10-11T09:45:54.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.532+0000 7f9dbb7fe700 1 -- 172.21.15.78:0/3140566848 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db400bb20 con 0x7f9dbc0b3a90 2023-10-11T09:45:54.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.532+0000 7f9dbb7fe700 1 -- 172.21.15.78:0/3140566848 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db4012430 con 0x7f9dbc0b3a90 2023-10-11T09:45:54.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.532+0000 7f9dc1fd2700 1 -- 172.21.15.78:0/3140566848 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9dbc0b3a90 msgr2=0x7f9dbc0b85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:54.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.532+0000 7f9dc1fd2700 1 --2- 172.21.15.78:0/3140566848 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9dbc0b3a90 0x7f9dbc0b85c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9db4009d90 tx=0x7f9db400b330 comp rx=0 tx=0).stop 2023-10-11T09:45:54.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.532+0000 7f9dc1fd2700 1 -- 172.21.15.78:0/3140566848 shutdown_connections 2023-10-11T09:45:54.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.532+0000 7f9dc1fd2700 1 --2- 172.21.15.78:0/3140566848 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9dbc0b3a90 0x7f9dbc0b85c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:54.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.532+0000 7f9dc1fd2700 1 --2- 172.21.15.78:0/3140566848 >> v2:172.21.15.133:3300/0 conn(0x7f9dbc0b3240 0x7f9dbc0d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:54.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.532+0000 7f9dc1fd2700 1 -- 172.21.15.78:0/3140566848 >> 172.21.15.78:0/3140566848 conn(0x7f9dbc09c260 msgr2=0x7f9dbc0d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:54.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.532+0000 7f9dc1fd2700 1 -- 172.21.15.78:0/3140566848 shutdown_connections 2023-10-11T09:45:54.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.532+0000 7f9dc1fd2700 1 -- 172.21.15.78:0/3140566848 wait complete. 2023-10-11T09:45:54.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.536+0000 7f9dc1fd2700 1 Processor -- start 2023-10-11T09:45:54.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.536+0000 7f9dc1fd2700 1 -- start start 2023-10-11T09:45:54.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.536+0000 7f9dc1fd2700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9dbc0b3240 0x7f9dbc158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:54.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.536+0000 7f9dc1fd2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9dbc0b3610 0x7f9dbc159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:54.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.536+0000 7f9dc1fd2700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9dbc0b3a90 0x7f9dbc15ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:54.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.536+0000 7f9dc1fd2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dbc1596c0 con 0x7f9dbc0b3610 2023-10-11T09:45:54.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.536+0000 7f9dc1fd2700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dbc0c6b60 con 0x7f9dbc0b3240 2023-10-11T09:45:54.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.536+0000 7f9dc1fd2700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dbc0c6e40 con 0x7f9dbc0b3a90 2023-10-11T09:45:54.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.536+0000 7f9dbbfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9dbc0b3610 0x7f9dbc159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:54.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.540+0000 7f9dbbfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9dbc0b3610 0x7f9dbc159180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50736/0 (socket says 172.21.15.78:50736) 2023-10-11T09:45:54.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.540+0000 7f9dc0fd0700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9dbc0b3240 0x7f9dbc158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:54.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.540+0000 7f9dbbfff700 1 -- 172.21.15.78:0/2127761333 learned_addr learned my addr 172.21.15.78:0/2127761333 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:54.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.540+0000 7f9dc17d1700 1 --2- 172.21.15.78:0/2127761333 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9dbc0b3a90 0x7f9dbc15ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:54.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.540+0000 7f9dbbfff700 1 -- 172.21.15.78:0/2127761333 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9dbc0b3a90 msgr2=0x7f9dbc15ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:54.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.540+0000 7f9dbbfff700 1 --2- 172.21.15.78:0/2127761333 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9dbc0b3a90 0x7f9dbc15ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:54.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.540+0000 7f9dbbfff700 1 -- 172.21.15.78:0/2127761333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9dbc0b3240 msgr2=0x7f9dbc158c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:54.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.540+0000 7f9dbbfff700 1 --2- 172.21.15.78:0/2127761333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9dbc0b3240 0x7f9dbc158c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:54.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.540+0000 7f9dbbfff700 1 -- 172.21.15.78:0/2127761333 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db4009a70 con 0x7f9dbc0b3610 2023-10-11T09:45:54.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.540+0000 7f9dbbfff700 1 --2- 172.21.15.78:0/2127761333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9dbc0b3610 0x7f9dbc159180 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f9da8001270 tx=0x7f9da800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:54.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.540+0000 7f9db97fa700 1 -- 172.21.15.78:0/2127761333 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da8005d90 con 0x7f9dbc0b3610 2023-10-11T09:45:54.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.540+0000 7f9db97fa700 1 -- 172.21.15.78:0/2127761333 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da80070d0 con 0x7f9dbc0b3610 2023-10-11T09:45:54.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.540+0000 7f9dc1fd2700 1 -- 172.21.15.78:0/2127761333 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dbc0c7120 con 0x7f9dbc0b3610 2023-10-11T09:45:54.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.540+0000 7f9db97fa700 1 -- 172.21.15.78:0/2127761333 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da8007480 con 0x7f9dbc0b3610 2023-10-11T09:45:54.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.540+0000 7f9dc1fd2700 1 -- 172.21.15.78:0/2127761333 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dbc0c76e0 con 0x7f9dbc0b3610 2023-10-11T09:45:54.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.540+0000 7f9dc1fd2700 1 -- 172.21.15.78:0/2127761333 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dbc0c7900 con 0x7f9dbc0b3610 2023-10-11T09:45:54.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.544+0000 7f9db97fa700 1 -- 172.21.15.78:0/2127761333 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f9da80075e0 con 0x7f9dbc0b3610 2023-10-11T09:45:54.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.544+0000 7f9db97fa700 1 --2- 172.21.15.78:0/2127761333 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9da0065ed0 0x7f9da0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:54.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.544+0000 7f9db97fa700 1 -- 172.21.15.78:0/2127761333 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x7f9da8082da0 con 0x7f9dbc0b3610 2023-10-11T09:45:54.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.544+0000 7f9dc0fd0700 1 --2- 172.21.15.78:0/2127761333 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9da0065ed0 0x7f9da0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:54.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.548+0000 7f9dc0fd0700 1 --2- 172.21.15.78:0/2127761333 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9da0065ed0 0x7f9da0068380 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9db0009b50 tx=0x7f9db0009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:54.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.552+0000 7f9db97fa700 1 -- 172.21.15.78:0/2127761333 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f9da8050110 con 0x7f9dbc0b3610 2023-10-11T09:45:54.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:54.760+0000 7f9dc1fd2700 1 -- 172.21.15.78:0/2127761333 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f9dbc065d40 con 0x7f9dbc0b3610 2023-10-11T09:45:55.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.356+0000 7f9db97fa700 1 -- 172.21.15.78:0/2127761333 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v43) v1 ==== 162+0+0 (secure 0 0 0) 0x7f9da80170b0 con 0x7f9dbc0b3610 2023-10-11T09:45:55.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.360+0000 7f9dc1fd2700 1 -- 172.21.15.78:0/2127761333 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9da0065ed0 msgr2=0x7f9da0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:55.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.360+0000 7f9dc1fd2700 1 --2- 172.21.15.78:0/2127761333 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9da0065ed0 0x7f9da0068380 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9db0009b50 tx=0x7f9db0009240 comp rx=0 tx=0).stop 2023-10-11T09:45:55.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.360+0000 7f9dc1fd2700 1 -- 172.21.15.78:0/2127761333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9dbc0b3610 msgr2=0x7f9dbc159180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:55.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.360+0000 7f9dc1fd2700 1 --2- 172.21.15.78:0/2127761333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9dbc0b3610 0x7f9dbc159180 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f9da8001270 tx=0x7f9da800c9e0 comp rx=0 tx=0).stop 2023-10-11T09:45:55.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.364+0000 7f9dc1fd2700 1 -- 172.21.15.78:0/2127761333 shutdown_connections 2023-10-11T09:45:55.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.364+0000 7f9dc1fd2700 1 --2- 172.21.15.78:0/2127761333 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9da0065ed0 0x7f9da0068380 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:55.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.364+0000 7f9dc1fd2700 1 --2- 172.21.15.78:0/2127761333 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9dbc0b3a90 0x7f9dbc15ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:55.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.364+0000 7f9dc1fd2700 1 --2- 172.21.15.78:0/2127761333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9dbc0b3610 0x7f9dbc159180 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:55.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.364+0000 7f9dc1fd2700 1 --2- 172.21.15.78:0/2127761333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9dbc0b3240 0x7f9dbc158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:55.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.364+0000 7f9dc1fd2700 1 -- 172.21.15.78:0/2127761333 >> 172.21.15.78:0/2127761333 conn(0x7f9dbc09c260 msgr2=0x7f9dbc15c500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:55.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.364+0000 7f9dc1fd2700 1 -- 172.21.15.78:0/2127761333 shutdown_connections 2023-10-11T09:45:55.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.364+0000 7f9dc1fd2700 1 -- 172.21.15.78:0/2127761333 wait complete. 2023-10-11T09:45:55.370 INFO:teuthology.orchestra.run.smithi078.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-11T09:45:55.390 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:45:55.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.608+0000 7fbceed7d700 1 Processor -- start 2023-10-11T09:45:55.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.608+0000 7fbceed7d700 1 -- start start 2023-10-11T09:45:55.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.608+0000 7fbceed7d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbce80cda10 0x7fbce80caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:55.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.608+0000 7fbceed7d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fbce80cdde0 0x7fbce8164610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:55.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.608+0000 7fbceed7d700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbce8165390 con 0x7fbce8164b50 2023-10-11T09:45:55.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.608+0000 7fbceed7d700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbce80ca440 con 0x7fbce80cdde0 2023-10-11T09:45:55.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.608+0000 7fbceed7d700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbce80ca720 con 0x7fbce80cda10 2023-10-11T09:45:55.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.608+0000 7fbced57a700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fbce80cdde0 0x7fbce8164610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:55.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.608+0000 7fbcedd7b700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbce80cda10 0x7fbce80caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:55.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.608+0000 7fbced57a700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fbce80cdde0 0x7fbce8164610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45786/0 (socket says 172.21.15.78:45786) 2023-10-11T09:45:55.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.608+0000 7fbced57a700 1 -- 172.21.15.78:0/2488730261 learned_addr learned my addr 172.21.15.78:0/2488730261 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:55.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.608+0000 7fbced57a700 1 -- 172.21.15.78:0/2488730261 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbce80cda10 msgr2=0x7fbce80caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:55.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.608+0000 7fbced57a700 1 --2- 172.21.15.78:0/2488730261 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbce80cda10 0x7fbce80caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:55.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.608+0000 7fbced57a700 1 -- 172.21.15.78:0/2488730261 >> v1:172.21.15.133:6789/0 conn(0x7fbce8164b50 legacy=0x7fbce806bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:55.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.608+0000 7fbced57a700 1 -- 172.21.15.78:0/2488730261 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce80caa00 con 0x7fbce80cdde0 2023-10-11T09:45:55.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.612+0000 7fbced57a700 1 --2- 172.21.15.78:0/2488730261 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbce80cdde0 0x7fbce8164610 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8009910 tx=0x7fbcd8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc016a825229ca1 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:55.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.612+0000 7fbcecd79700 1 -- 172.21.15.78:0/2488730261 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8011830 con 0x7fbce80cdde0 2023-10-11T09:45:55.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.612+0000 7fbcecd79700 1 -- 172.21.15.78:0/2488730261 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd8011990 con 0x7fbce80cdde0 2023-10-11T09:45:55.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.612+0000 7fbcecd79700 1 -- 172.21.15.78:0/2488730261 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8011d40 con 0x7fbce80cdde0 2023-10-11T09:45:55.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.612+0000 7fbceed7d700 1 -- 172.21.15.78:0/2488730261 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbce80cdde0 msgr2=0x7fbce8164610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:55.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.612+0000 7fbceed7d700 1 --2- 172.21.15.78:0/2488730261 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbce80cdde0 0x7fbce8164610 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8009910 tx=0x7fbcd8009c20 comp rx=0 tx=0).stop 2023-10-11T09:45:55.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.612+0000 7fbceed7d700 1 -- 172.21.15.78:0/2488730261 shutdown_connections 2023-10-11T09:45:55.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.612+0000 7fbceed7d700 1 --2- 172.21.15.78:0/2488730261 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbce80cdde0 0x7fbce8164610 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:55.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.612+0000 7fbceed7d700 1 --2- 172.21.15.78:0/2488730261 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbce80cda10 0x7fbce80caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:55.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.612+0000 7fbceed7d700 1 -- 172.21.15.78:0/2488730261 >> 172.21.15.78:0/2488730261 conn(0x7fbce80cea60 msgr2=0x7fbce80cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:55.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.612+0000 7fbceed7d700 1 -- 172.21.15.78:0/2488730261 shutdown_connections 2023-10-11T09:45:55.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.612+0000 7fbceed7d700 1 -- 172.21.15.78:0/2488730261 wait complete. 2023-10-11T09:45:55.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.616+0000 7fbceed7d700 1 Processor -- start 2023-10-11T09:45:55.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.616+0000 7fbceed7d700 1 -- start start 2023-10-11T09:45:55.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.616+0000 7fbceed7d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbce80cda10 0x7fbce815cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:55.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.616+0000 7fbceed7d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbce80cdde0 0x7fbce8162bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:55.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.616+0000 7fbceed7d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbce8164b50 0x7fbce81eadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:55.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.616+0000 7fbceed7d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce8163dc0 con 0x7fbce8164b50 2023-10-11T09:45:55.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.616+0000 7fbceed7d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce8163520 con 0x7fbce80cda10 2023-10-11T09:45:55.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.616+0000 7fbceed7d700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbce8163800 con 0x7fbce80cdde0 2023-10-11T09:45:55.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.616+0000 7fbcee57c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbce8164b50 0x7fbce81eadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:55.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.616+0000 7fbcee57c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbce8164b50 0x7fbce81eadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45792/0 (socket says 172.21.15.78:45792) 2023-10-11T09:45:55.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.616+0000 7fbcee57c700 1 -- 172.21.15.78:0/447156549 learned_addr learned my addr 172.21.15.78:0/447156549 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:55.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.616+0000 7fbced57a700 1 --2- 172.21.15.78:0/447156549 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbce80cdde0 0x7fbce8162bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:55.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.616+0000 7fbcedd7b700 1 --2- 172.21.15.78:0/447156549 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbce80cda10 0x7fbce815cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:55.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.620+0000 7fbcee57c700 1 -- 172.21.15.78:0/447156549 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbce80cdde0 msgr2=0x7fbce8162bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:55.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.620+0000 7fbcee57c700 1 --2- 172.21.15.78:0/447156549 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbce80cdde0 0x7fbce8162bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:55.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.620+0000 7fbcee57c700 1 -- 172.21.15.78:0/447156549 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbce80cda10 msgr2=0x7fbce815cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:55.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.620+0000 7fbcee57c700 1 --2- 172.21.15.78:0/447156549 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbce80cda10 0x7fbce815cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:55.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.620+0000 7fbcee57c700 1 -- 172.21.15.78:0/447156549 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd8009570 con 0x7fbce8164b50 2023-10-11T09:45:55.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.620+0000 7fbcee57c700 1 --2- 172.21.15.78:0/447156549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbce8164b50 0x7fbce81eadf0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fbce000cc50 tx=0x7fbce000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:55.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.620+0000 7fbcde7fc700 1 -- 172.21.15.78:0/447156549 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce0003c20 con 0x7fbce8164b50 2023-10-11T09:45:55.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.620+0000 7fbcde7fc700 1 -- 172.21.15.78:0/447156549 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce0003d80 con 0x7fbce8164b50 2023-10-11T09:45:55.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.620+0000 7fbcde7fc700 1 -- 172.21.15.78:0/447156549 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce000aab0 con 0x7fbce8164b50 2023-10-11T09:45:55.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.620+0000 7fbceed7d700 1 -- 172.21.15.78:0/447156549 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbce80ce3b0 con 0x7fbce8164b50 2023-10-11T09:45:55.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.620+0000 7fbceed7d700 1 -- 172.21.15.78:0/447156549 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbce80b1ab0 con 0x7fbce8164b50 2023-10-11T09:45:55.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.620+0000 7fbceed7d700 1 -- 172.21.15.78:0/447156549 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbce8065d40 con 0x7fbce8164b50 2023-10-11T09:45:55.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.620+0000 7fbcde7fc700 1 -- 172.21.15.78:0/447156549 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fbce000ac10 con 0x7fbce8164b50 2023-10-11T09:45:55.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.624+0000 7fbcde7fc700 1 --2- 172.21.15.78:0/447156549 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbccc065ed0 0x7fbccc068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:55.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.624+0000 7fbcde7fc700 1 -- 172.21.15.78:0/447156549 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x7fbce0083a60 con 0x7fbce8164b50 2023-10-11T09:45:55.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.624+0000 7fbcedd7b700 1 --2- 172.21.15.78:0/447156549 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbccc065ed0 0x7fbccc068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:55.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.624+0000 7fbcedd7b700 1 --2- 172.21.15.78:0/447156549 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbccc065ed0 0x7fbccc068380 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fbce806afb0 tx=0x7fbce4009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:55.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.628+0000 7fbcde7fc700 1 -- 172.21.15.78:0/447156549 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fbce0050dd0 con 0x7fbce8164b50 2023-10-11T09:45:55.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.836+0000 7fbceed7d700 1 -- 172.21.15.78:0/447156549 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbce80b1480 con 0x7fbce8164b50 2023-10-11T09:45:55.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.836+0000 7fbcde7fc700 1 -- 172.21.15.78:0/447156549 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4649 (secure 0 0 0) 0x7fbce00543f0 con 0x7fbce8164b50 2023-10-11T09:45:55.840 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:55.840 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":43,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5579,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3954348479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":3954348479},{"type":"v1","addr":"172.21.15.133:6835","nonce":3954348479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5783,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3328076595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":3328076595},{"type":"v1","addr":"172.21.15.78:6837","nonce":3328076595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":6100,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/4029676589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4029676589},{"type":"v1","addr":"172.21.15.133:6834","nonce":4029676589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6582,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/1666964283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":1666964283},{"type":"v1","addr":"172.21.15.78:6836","nonce":1666964283}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:45:54.252958+0000","modified":"2023-10-11T09:45:55.350786+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[9,10],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-11T09:45:55.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.840+0000 7fbceed7d700 1 -- 172.21.15.78:0/447156549 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbccc065ed0 msgr2=0x7fbccc068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:55.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.840+0000 7fbceed7d700 1 --2- 172.21.15.78:0/447156549 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbccc065ed0 0x7fbccc068380 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fbce806afb0 tx=0x7fbce4009240 comp rx=0 tx=0).stop 2023-10-11T09:45:55.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.840+0000 7fbceed7d700 1 -- 172.21.15.78:0/447156549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbce8164b50 msgr2=0x7fbce81eadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:55.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.840+0000 7fbceed7d700 1 --2- 172.21.15.78:0/447156549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbce8164b50 0x7fbce81eadf0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fbce000cc50 tx=0x7fbce000cf60 comp rx=0 tx=0).stop 2023-10-11T09:45:55.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.840+0000 7fbceed7d700 1 -- 172.21.15.78:0/447156549 shutdown_connections 2023-10-11T09:45:55.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.840+0000 7fbceed7d700 1 --2- 172.21.15.78:0/447156549 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbccc065ed0 0x7fbccc068380 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:55.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.840+0000 7fbceed7d700 1 --2- 172.21.15.78:0/447156549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbce8164b50 0x7fbce81eadf0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:55.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.840+0000 7fbceed7d700 1 --2- 172.21.15.78:0/447156549 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbce80cdde0 0x7fbce8162bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:55.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.840+0000 7fbceed7d700 1 --2- 172.21.15.78:0/447156549 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbce80cda10 0x7fbce815cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:55.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.840+0000 7fbceed7d700 1 -- 172.21.15.78:0/447156549 >> 172.21.15.78:0/447156549 conn(0x7fbce80cea60 msgr2=0x7fbce8149990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:55.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.840+0000 7fbceed7d700 1 -- 172.21.15.78:0/447156549 shutdown_connections 2023-10-11T09:45:55.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:55.840+0000 7fbceed7d700 1 -- 172.21.15.78:0/447156549 wait complete. 2023-10-11T09:45:55.845 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 43 2023-10-11T09:45:55.862 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-11T09:45:55.863 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:45:56.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.028+0000 7f038ca98700 1 Processor -- start 2023-10-11T09:45:56.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.028+0000 7f038ca98700 1 -- start start 2023-10-11T09:45:56.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.032+0000 7f038ca98700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f038815d050 0x7f03880cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:56.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.032+0000 7f038ca98700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0388160210 con 0x7f0388160c60 2023-10-11T09:45:56.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.032+0000 7f038ca98700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03880ccd60 con 0x7f03880b06a0 2023-10-11T09:45:56.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.032+0000 7f038ca98700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03880ccfc0 con 0x7f038815d050 2023-10-11T09:45:56.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.032+0000 7f0386ffd700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f0388160c60 0x7f038815d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:60166/0 (socket says 172.21.15.78:60166) 2023-10-11T09:45:56.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.032+0000 7f0386ffd700 1 -- 172.21.15.78:0/3623597321 learned_addr learned my addr 172.21.15.78:0/3623597321 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:45:56.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.032+0000 7f03877fe700 1 --2- 172.21.15.78:0/3623597321 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f038815d050 0x7f03880cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:56.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.032+0000 7f03867fc700 1 -- 172.21.15.78:0/3623597321 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2791563792 0 0) 0x7f0388160210 con 0x7f0388160c60 2023-10-11T09:45:56.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.032+0000 7f03867fc700 1 -- 172.21.15.78:0/3623597321 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03700035d0 con 0x7f0388160c60 2023-10-11T09:45:56.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.032+0000 7f03867fc700 1 -- 172.21.15.78:0/3623597321 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1336899273 0 0) 0x7f03700035d0 con 0x7f0388160c60 2023-10-11T09:45:56.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.032+0000 7f03867fc700 1 -- 172.21.15.78:0/3623597321 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f038815d050 msgr2=0x7f03880cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:56.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.032+0000 7f03867fc700 1 --2- 172.21.15.78:0/3623597321 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f038815d050 0x7f03880cc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:56.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.032+0000 7f03867fc700 1 -- 172.21.15.78:0/3623597321 >> v1:172.21.15.133:6789/0 conn(0x7f03880b06a0 legacy=0x7f03880c9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:56.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.032+0000 7f03867fc700 1 -- 172.21.15.78:0/3623597321 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03880cd2a0 con 0x7f0388160c60 2023-10-11T09:45:56.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.032+0000 7f03867fc700 1 -- 172.21.15.78:0/3623597321 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f037c001670 con 0x7f0388160c60 2023-10-11T09:45:56.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.032+0000 7f03867fc700 1 -- 172.21.15.78:0/3623597321 >> v1:172.21.15.78:6789/0 conn(0x7f0388160c60 legacy=0x7f038815d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:56.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.032+0000 7f03867fc700 1 --2- 172.21.15.78:0/3623597321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0370003a50 0x7f0370003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:56.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.032+0000 7f03867fc700 1 --2- 172.21.15.78:0/3623597321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0370004870 0x7f0370004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:56.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.032+0000 7f03867fc700 1 --2- 172.21.15.78:0/3623597321 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f038815d050 0x7f0370005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:56.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.032+0000 7f03867fc700 1 -- 172.21.15.78:0/3623597321 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03880ccfc0 con 0x7f0370003a50 2023-10-11T09:45:56.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.032+0000 7f03867fc700 1 -- 172.21.15.78:0/3623597321 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037000bae0 con 0x7f0370004870 2023-10-11T09:45:56.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.032+0000 7f03867fc700 1 -- 172.21.15.78:0/3623597321 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f037000bc70 con 0x7f038815d050 2023-10-11T09:45:56.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.032+0000 7f03877fe700 1 --2- 172.21.15.78:0/3623597321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0370003a50 0x7f0370003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:56.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.032+0000 7f0387fff700 1 --2- 172.21.15.78:0/3623597321 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f038815d050 0x7f0370005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:56.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.032+0000 7f0386ffd700 1 --2- 172.21.15.78:0/3623597321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0370004870 0x7f0370004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:56.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.032+0000 7f03877fe700 1 -- 172.21.15.78:0/3623597321 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f038815d050 msgr2=0x7f0370005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:56.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.032+0000 7f03877fe700 1 --2- 172.21.15.78:0/3623597321 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f038815d050 0x7f0370005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:56.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.032+0000 7f03877fe700 1 -- 172.21.15.78:0/3623597321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0370004870 msgr2=0x7f0370004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:56.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.032+0000 7f03877fe700 1 --2- 172.21.15.78:0/3623597321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0370004870 0x7f0370004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:56.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.032+0000 7f03877fe700 1 -- 172.21.15.78:0/3623597321 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03700035d0 con 0x7f0370003a50 2023-10-11T09:45:56.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.032+0000 7f0386ffd700 1 --2- 172.21.15.78:0/3623597321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0370004870 0x7f0370004ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:45:56.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.036+0000 7f03877fe700 1 --2- 172.21.15.78:0/3623597321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0370003a50 0x7f0370003e60 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0378009b70 tx=0x7f03700058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=532b0a9f10e1cade server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:56.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.036+0000 7f03867fc700 1 -- 172.21.15.78:0/3623597321 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037800e6c0 con 0x7f0370003a50 2023-10-11T09:45:56.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.036+0000 7f03867fc700 1 -- 172.21.15.78:0/3623597321 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f037800e820 con 0x7f0370003a50 2023-10-11T09:45:56.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.036+0000 7f03867fc700 1 -- 172.21.15.78:0/3623597321 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037800eb90 con 0x7f0370003a50 2023-10-11T09:45:56.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.036+0000 7f038ca98700 1 -- 172.21.15.78:0/3623597321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0370003a50 msgr2=0x7f0370003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:56.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.036+0000 7f038ca98700 1 --2- 172.21.15.78:0/3623597321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0370003a50 0x7f0370003e60 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0378009b70 tx=0x7f03700058a0 comp rx=0 tx=0).stop 2023-10-11T09:45:56.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.036+0000 7f0387fff700 1 -- 172.21.15.78:0/3623597321 reap_dead start 2023-10-11T09:45:56.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.036+0000 7f038ca98700 1 -- 172.21.15.78:0/3623597321 shutdown_connections 2023-10-11T09:45:56.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.036+0000 7f038ca98700 1 -- 172.21.15.78:0/3623597321 >> 172.21.15.78:0/3623597321 conn(0x7f03880cea60 msgr2=0x7f03880cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:56.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.036+0000 7f038ca98700 1 -- 172.21.15.78:0/3623597321 shutdown_connections 2023-10-11T09:45:56.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.036+0000 7f038ca98700 1 -- 172.21.15.78:0/3623597321 wait complete. 2023-10-11T09:45:56.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.040+0000 7f038ca98700 1 Processor -- start 2023-10-11T09:45:56.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.040+0000 7f038ca98700 1 -- start start 2023-10-11T09:45:56.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.040+0000 7f038ca98700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0370003a50 0x7f038814c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:56.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.040+0000 7f038ca98700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0388153d20 0x7f0388154190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:56.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.040+0000 7f038ca98700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0388150c30 0x7f03881510a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:56.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.040+0000 7f038ca98700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03881515e0 con 0x7f0388153d20 2023-10-11T09:45:56.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.040+0000 7f038ca98700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0388070640 con 0x7f0370003a50 2023-10-11T09:45:56.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.040+0000 7f038ca98700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f038814c5d0 con 0x7f0388150c30 2023-10-11T09:45:56.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.044+0000 7f0386ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0388153d20 0x7f0388154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:56.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.044+0000 7f0386ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0388153d20 0x7f0388154190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45822/0 (socket says 172.21.15.78:45822) 2023-10-11T09:45:56.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.044+0000 7f0386ffd700 1 -- 172.21.15.78:0/417517346 learned_addr learned my addr 172.21.15.78:0/417517346 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:56.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.044+0000 7f03877fe700 1 --2- 172.21.15.78:0/417517346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0370003a50 0x7f038814c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:56.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.044+0000 7f0387fff700 1 --2- 172.21.15.78:0/417517346 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0388150c30 0x7f03881510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:56.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.044+0000 7f0386ffd700 1 -- 172.21.15.78:0/417517346 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0388150c30 msgr2=0x7f03881510a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:56.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.044+0000 7f0386ffd700 1 --2- 172.21.15.78:0/417517346 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0388150c30 0x7f03881510a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:56.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.044+0000 7f0386ffd700 1 -- 172.21.15.78:0/417517346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0370003a50 msgr2=0x7f038814c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:56.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.044+0000 7f0386ffd700 1 --2- 172.21.15.78:0/417517346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0370003a50 0x7f038814c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:56.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.044+0000 7f0386ffd700 1 -- 172.21.15.78:0/417517346 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03881547c0 con 0x7f0388153d20 2023-10-11T09:45:56.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.044+0000 7f03877fe700 1 --2- 172.21.15.78:0/417517346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0370003a50 0x7f038814c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:45:56.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.044+0000 7f0386ffd700 1 --2- 172.21.15.78:0/417517346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0388153d20 0x7f0388154190 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f037c00eb20 tx=0x7f037c00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:56.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.044+0000 7f036ffff700 1 -- 172.21.15.78:0/417517346 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c00c780 con 0x7f0388153d20 2023-10-11T09:45:56.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.044+0000 7f036ffff700 1 -- 172.21.15.78:0/417517346 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f037c00c8e0 con 0x7f0388153d20 2023-10-11T09:45:56.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.044+0000 7f038ca98700 1 -- 172.21.15.78:0/417517346 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f038806fd10 con 0x7f0388153d20 2023-10-11T09:45:56.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.044+0000 7f036ffff700 1 -- 172.21.15.78:0/417517346 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c00cc50 con 0x7f0388153d20 2023-10-11T09:45:56.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.044+0000 7f038ca98700 1 -- 172.21.15.78:0/417517346 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03880702d0 con 0x7f0388153d20 2023-10-11T09:45:56.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.044+0000 7f038ca98700 1 -- 172.21.15.78:0/417517346 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0388065d40 con 0x7f0388153d20 2023-10-11T09:45:56.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.048+0000 7f036ffff700 1 -- 172.21.15.78:0/417517346 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f037c013420 con 0x7f0388153d20 2023-10-11T09:45:56.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.048+0000 7f036ffff700 1 --2- 172.21.15.78:0/417517346 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0374065ed0 0x7f0374068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:56.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.048+0000 7f036ffff700 1 -- 172.21.15.78:0/417517346 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x7f037c0839a0 con 0x7f0388153d20 2023-10-11T09:45:56.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.048+0000 7f03877fe700 1 --2- 172.21.15.78:0/417517346 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0374065ed0 0x7f0374068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:56.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.052+0000 7f03877fe700 1 --2- 172.21.15.78:0/417517346 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0374065ed0 0x7f0374068380 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f03780090b0 tx=0x7f0378008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:56.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.056+0000 7f036ffff700 1 -- 172.21.15.78:0/417517346 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f037c050d20 con 0x7f0388153d20 2023-10-11T09:45:56.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.268+0000 7f038ca98700 1 -- 172.21.15.78:0/417517346 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0388070780 con 0x7f0388153d20 2023-10-11T09:45:56.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.268+0000 7f036ffff700 1 -- 172.21.15.78:0/417517346 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4649 (secure 0 0 0) 0x7f037c054340 con 0x7f0388153d20 2023-10-11T09:45:56.272 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:56.272 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":43,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5579,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3954348479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":3954348479},{"type":"v1","addr":"172.21.15.133:6835","nonce":3954348479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5783,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3328076595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":3328076595},{"type":"v1","addr":"172.21.15.78:6837","nonce":3328076595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":6100,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/4029676589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4029676589},{"type":"v1","addr":"172.21.15.133:6834","nonce":4029676589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6582,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/1666964283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":1666964283},{"type":"v1","addr":"172.21.15.78:6836","nonce":1666964283}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:45:54.252958+0000","modified":"2023-10-11T09:45:55.350786+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[9,10],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-11T09:45:56.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.272+0000 7f038ca98700 1 -- 172.21.15.78:0/417517346 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0374065ed0 msgr2=0x7f0374068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:56.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.272+0000 7f038ca98700 1 --2- 172.21.15.78:0/417517346 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0374065ed0 0x7f0374068380 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f03780090b0 tx=0x7f0378008030 comp rx=0 tx=0).stop 2023-10-11T09:45:56.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.272+0000 7f038ca98700 1 -- 172.21.15.78:0/417517346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0388153d20 msgr2=0x7f0388154190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:56.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.272+0000 7f038ca98700 1 --2- 172.21.15.78:0/417517346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0388153d20 0x7f0388154190 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f037c00eb20 tx=0x7f037c00eee0 comp rx=0 tx=0).stop 2023-10-11T09:45:56.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.272+0000 7f038ca98700 1 -- 172.21.15.78:0/417517346 shutdown_connections 2023-10-11T09:45:56.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.272+0000 7f038ca98700 1 --2- 172.21.15.78:0/417517346 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0374065ed0 0x7f0374068380 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:56.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.272+0000 7f038ca98700 1 --2- 172.21.15.78:0/417517346 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0388150c30 0x7f03881510a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:56.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.272+0000 7f038ca98700 1 --2- 172.21.15.78:0/417517346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0388153d20 0x7f0388154190 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:56.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.272+0000 7f038ca98700 1 --2- 172.21.15.78:0/417517346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0370003a50 0x7f038814c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:56.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.272+0000 7f038ca98700 1 -- 172.21.15.78:0/417517346 >> 172.21.15.78:0/417517346 conn(0x7f03880cea60 msgr2=0x7f03880c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:56.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.272+0000 7f038ca98700 1 -- 172.21.15.78:0/417517346 shutdown_connections 2023-10-11T09:45:56.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.272+0000 7f038ca98700 1 -- 172.21.15.78:0/417517346 wait complete. 2023-10-11T09:45:56.280 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 43 2023-10-11T09:45:56.298 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:45:56.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.464+0000 7f4943529700 1 Processor -- start 2023-10-11T09:45:56.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.464+0000 7f4943529700 1 -- start start 2023-10-11T09:45:56.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.468+0000 7f4943529700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f493c071040 0x7f493c0cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:56.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.468+0000 7f4943529700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f493c0ce330 0x7f493c06be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:56.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.468+0000 7f4943529700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f493c06ffa0 con 0x7f493c160d40 2023-10-11T09:45:56.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.468+0000 7f4943529700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f493c070c20 con 0x7f493c071040 2023-10-11T09:45:56.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.468+0000 7f4943529700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493c161110 con 0x7f493c0ce330 2023-10-11T09:45:56.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.468+0000 7f49412c5700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f493c160d40 0x7f493c0cd280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:60182/0 (socket says 172.21.15.78:60182) 2023-10-11T09:45:56.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.468+0000 7f49412c5700 1 -- 172.21.15.78:0/2492072608 learned_addr learned my addr 172.21.15.78:0/2492072608 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:45:56.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.468+0000 7f4940ac4700 1 --2- 172.21.15.78:0/2492072608 >> v2:172.21.15.133:3300/0 conn(0x7f493c071040 0x7f493c0cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:56.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.468+0000 7f4941ac6700 1 --2- 172.21.15.78:0/2492072608 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f493c0ce330 0x7f493c06be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:56.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.468+0000 7f4933fff700 1 -- 172.21.15.78:0/2492072608 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1801243998 0 0) 0x7f493c06ffa0 con 0x7f493c160d40 2023-10-11T09:45:56.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.468+0000 7f4933fff700 1 -- 172.21.15.78:0/2492072608 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49280035d0 con 0x7f493c160d40 2023-10-11T09:45:56.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.468+0000 7f4940ac4700 1 -- 172.21.15.78:0/2492072608 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f493c0ce330 msgr2=0x7f493c06be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:56.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.468+0000 7f4940ac4700 1 --2- 172.21.15.78:0/2492072608 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f493c0ce330 0x7f493c06be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:56.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.468+0000 7f4940ac4700 1 -- 172.21.15.78:0/2492072608 >> v1:172.21.15.78:6789/0 conn(0x7f493c160d40 legacy=0x7f493c0cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:56.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.468+0000 7f4940ac4700 1 -- 172.21.15.78:0/2492072608 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f493c070280 con 0x7f493c071040 2023-10-11T09:45:56.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.468+0000 7f4940ac4700 1 --2- 172.21.15.78:0/2492072608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f493c071040 0x7f493c0cddf0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f492c009850 tx=0x7f492c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70240f1c5e29af03 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:56.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.468+0000 7f4933fff700 1 -- 172.21.15.78:0/2492072608 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c011830 con 0x7f493c071040 2023-10-11T09:45:56.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.468+0000 7f4933fff700 1 -- 172.21.15.78:0/2492072608 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492c011990 con 0x7f493c071040 2023-10-11T09:45:56.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.468+0000 7f4933fff700 1 -- 172.21.15.78:0/2492072608 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c011d40 con 0x7f493c071040 2023-10-11T09:45:56.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.468+0000 7f4943529700 1 -- 172.21.15.78:0/2492072608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f493c071040 msgr2=0x7f493c0cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:56.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.468+0000 7f4943529700 1 --2- 172.21.15.78:0/2492072608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f493c071040 0x7f493c0cddf0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f492c009850 tx=0x7f492c009b60 comp rx=0 tx=0).stop 2023-10-11T09:45:56.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.468+0000 7f4943529700 1 -- 172.21.15.78:0/2492072608 shutdown_connections 2023-10-11T09:45:56.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.468+0000 7f4943529700 1 --2- 172.21.15.78:0/2492072608 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f493c0ce330 0x7f493c06be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:56.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.468+0000 7f4943529700 1 --2- 172.21.15.78:0/2492072608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f493c071040 0x7f493c0cddf0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:56.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.468+0000 7f4943529700 1 -- 172.21.15.78:0/2492072608 >> 172.21.15.78:0/2492072608 conn(0x7f493c06aca0 msgr2=0x7f493c0b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:56.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.472+0000 7f4943529700 1 -- 172.21.15.78:0/2492072608 shutdown_connections 2023-10-11T09:45:56.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.472+0000 7f4943529700 1 -- 172.21.15.78:0/2492072608 wait complete. 2023-10-11T09:45:56.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.472+0000 7f4943529700 1 Processor -- start 2023-10-11T09:45:56.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.472+0000 7f4943529700 1 -- start start 2023-10-11T09:45:56.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.476+0000 7f4943529700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f493c071040 0x7f493c0c3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:56.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.476+0000 7f4943529700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f493c0ce330 0x7f493c151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:56.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.476+0000 7f4943529700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f493c160d40 0x7f493c14e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:56.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.476+0000 7f4943529700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493c14eba0 con 0x7f493c160d40 2023-10-11T09:45:56.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.476+0000 7f4943529700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493c151a50 con 0x7f493c0ce330 2023-10-11T09:45:56.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.476+0000 7f4943529700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493c0c42e0 con 0x7f493c071040 2023-10-11T09:45:56.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.476+0000 7f4941ac6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f493c160d40 0x7f493c14e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:56.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.476+0000 7f4941ac6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f493c160d40 0x7f493c14e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45828/0 (socket says 172.21.15.78:45828) 2023-10-11T09:45:56.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.476+0000 7f4941ac6700 1 -- 172.21.15.78:0/1362451194 learned_addr learned my addr 172.21.15.78:0/1362451194 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:56.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.476+0000 7f4940ac4700 1 --2- 172.21.15.78:0/1362451194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f493c0ce330 0x7f493c151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:56.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.476+0000 7f49412c5700 1 --2- 172.21.15.78:0/1362451194 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f493c071040 0x7f493c0c3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:56.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.476+0000 7f4941ac6700 1 -- 172.21.15.78:0/1362451194 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f493c071040 msgr2=0x7f493c0c3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:56.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.476+0000 7f4941ac6700 1 --2- 172.21.15.78:0/1362451194 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f493c071040 0x7f493c0c3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:56.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.476+0000 7f4941ac6700 1 -- 172.21.15.78:0/1362451194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f493c0ce330 msgr2=0x7f493c151510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:56.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.476+0000 7f4941ac6700 1 --2- 172.21.15.78:0/1362451194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f493c0ce330 0x7f493c151510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:56.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.476+0000 7f4941ac6700 1 -- 172.21.15.78:0/1362451194 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492c0094b0 con 0x7f493c160d40 2023-10-11T09:45:56.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.476+0000 7f4941ac6700 1 --2- 172.21.15.78:0/1362451194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f493c160d40 0x7f493c14e160 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f493400efc0 tx=0x7f493400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:56.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.480+0000 7f4931ffb700 1 -- 172.21.15.78:0/1362451194 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4934009ea0 con 0x7f493c160d40 2023-10-11T09:45:56.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.480+0000 7f4931ffb700 1 -- 172.21.15.78:0/1362451194 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493400f040 con 0x7f493c160d40 2023-10-11T09:45:56.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.480+0000 7f4943529700 1 -- 172.21.15.78:0/1362451194 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f493c0c45c0 con 0x7f493c160d40 2023-10-11T09:45:56.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.480+0000 7f4931ffb700 1 -- 172.21.15.78:0/1362451194 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493400ced0 con 0x7f493c160d40 2023-10-11T09:45:56.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.480+0000 7f4943529700 1 -- 172.21.15.78:0/1362451194 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f493c0c4b80 con 0x7f493c160d40 2023-10-11T09:45:56.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.480+0000 7f4943529700 1 -- 172.21.15.78:0/1362451194 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f493c072250 con 0x7f493c160d40 2023-10-11T09:45:56.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.484+0000 7f4931ffb700 1 -- 172.21.15.78:0/1362451194 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f4934014480 con 0x7f493c160d40 2023-10-11T09:45:56.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.484+0000 7f4931ffb700 1 --2- 172.21.15.78:0/1362451194 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4920065ed0 0x7f4920068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:56.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.484+0000 7f4931ffb700 1 -- 172.21.15.78:0/1362451194 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x7f49340842f0 con 0x7f493c160d40 2023-10-11T09:45:56.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.484+0000 7f49412c5700 1 --2- 172.21.15.78:0/1362451194 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4920065ed0 0x7f4920068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:56.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.484+0000 7f49412c5700 1 --2- 172.21.15.78:0/1362451194 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4920065ed0 0x7f4920068380 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4938007fd0 tx=0x7f4938007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:56.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.488+0000 7f4931ffb700 1 -- 172.21.15.78:0/1362451194 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f4934051720 con 0x7f493c160d40 2023-10-11T09:45:56.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.692+0000 7f4943529700 1 -- 172.21.15.78:0/1362451194 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f493c14ece0 con 0x7f493c160d40 2023-10-11T09:45:56.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.692+0000 7f4931ffb700 1 -- 172.21.15.78:0/1362451194 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4649 (secure 0 0 0) 0x7f493c14ece0 con 0x7f493c160d40 2023-10-11T09:45:56.698 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:56.698 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":43,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5579,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3954348479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":3954348479},{"type":"v1","addr":"172.21.15.133:6835","nonce":3954348479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5783,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3328076595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":3328076595},{"type":"v1","addr":"172.21.15.78:6837","nonce":3328076595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":6100,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/4029676589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4029676589},{"type":"v1","addr":"172.21.15.133:6834","nonce":4029676589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6582,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/1666964283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":1666964283},{"type":"v1","addr":"172.21.15.78:6836","nonce":1666964283}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:45:54.252958+0000","modified":"2023-10-11T09:45:55.350786+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[9,10],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-11T09:45:56.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.696+0000 7f4943529700 1 -- 172.21.15.78:0/1362451194 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4920065ed0 msgr2=0x7f4920068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:56.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.696+0000 7f4943529700 1 --2- 172.21.15.78:0/1362451194 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4920065ed0 0x7f4920068380 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4938007fd0 tx=0x7f4938007de0 comp rx=0 tx=0).stop 2023-10-11T09:45:56.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.696+0000 7f4943529700 1 -- 172.21.15.78:0/1362451194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f493c160d40 msgr2=0x7f493c14e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:56.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.696+0000 7f4943529700 1 --2- 172.21.15.78:0/1362451194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f493c160d40 0x7f493c14e160 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f493400efc0 tx=0x7f493400c3a0 comp rx=0 tx=0).stop 2023-10-11T09:45:56.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.696+0000 7f4943529700 1 -- 172.21.15.78:0/1362451194 shutdown_connections 2023-10-11T09:45:56.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.696+0000 7f4943529700 1 --2- 172.21.15.78:0/1362451194 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4920065ed0 0x7f4920068380 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:56.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.696+0000 7f4943529700 1 --2- 172.21.15.78:0/1362451194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f493c160d40 0x7f493c14e160 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:56.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.696+0000 7f4943529700 1 --2- 172.21.15.78:0/1362451194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f493c0ce330 0x7f493c151510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:56.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.696+0000 7f4943529700 1 --2- 172.21.15.78:0/1362451194 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f493c071040 0x7f493c0c3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:56.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.696+0000 7f4943529700 1 -- 172.21.15.78:0/1362451194 >> 172.21.15.78:0/1362451194 conn(0x7f493c06aca0 msgr2=0x7f493c0ca5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:56.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.696+0000 7f4943529700 1 -- 172.21.15.78:0/1362451194 shutdown_connections 2023-10-11T09:45:56.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.696+0000 7f4943529700 1 -- 172.21.15.78:0/1362451194 wait complete. 2023-10-11T09:45:56.704 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 43 2023-10-11T09:45:56.724 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:45:56.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.892+0000 7f2d94234700 1 Processor -- start 2023-10-11T09:45:56.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.892+0000 7f2d94234700 1 -- start start 2023-10-11T09:45:56.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.896+0000 7f2d94234700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2d8c0cd210 0x7f2d8c06be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:56.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.896+0000 7f2d94234700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d8c0c93a0 con 0x7f2d8c0cbd30 2023-10-11T09:45:56.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.896+0000 7f2d94234700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d8c0cca10 con 0x7f2d8c15c880 2023-10-11T09:45:56.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.896+0000 7f2d94234700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d8c161420 con 0x7f2d8c0cd210 2023-10-11T09:45:56.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.896+0000 7f2d917cf700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f2d8c0cbd30 0x7f2d8c0c9b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:60194/0 (socket says 172.21.15.78:60194) 2023-10-11T09:45:56.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.896+0000 7f2d917cf700 1 -- 172.21.15.78:0/445471193 learned_addr learned my addr 172.21.15.78:0/445471193 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:45:56.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.896+0000 7f2d927d1700 1 --2- 172.21.15.78:0/445471193 >> v2:172.21.15.133:3300/0 conn(0x7f2d8c0cd210 0x7f2d8c06be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:56.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.896+0000 7f2d90fce700 1 -- 172.21.15.78:0/445471193 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4111236675 0 0) 0x7f2d8c0c93a0 con 0x7f2d8c0cbd30 2023-10-11T09:45:56.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.896+0000 7f2d90fce700 1 -- 172.21.15.78:0/445471193 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d780035d0 con 0x7f2d8c0cbd30 2023-10-11T09:45:56.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.896+0000 7f2d927d1700 1 -- 172.21.15.78:0/445471193 >> v1:172.21.15.133:6789/0 conn(0x7f2d8c15c880 legacy=0x7f2d8c0cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:56.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.896+0000 7f2d927d1700 1 -- 172.21.15.78:0/445471193 >> v1:172.21.15.78:6789/0 conn(0x7f2d8c0cbd30 legacy=0x7f2d8c0c9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:56.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.896+0000 7f2d927d1700 1 -- 172.21.15.78:0/445471193 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d8c0cccb0 con 0x7f2d8c0cd210 2023-10-11T09:45:56.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.900+0000 7f2d927d1700 1 --2- 172.21.15.78:0/445471193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d8c0cd210 0x7f2d8c06be60 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009850 tx=0x7f2d84009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b4b212015abf504 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:56.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.900+0000 7f2d90fce700 1 -- 172.21.15.78:0/445471193 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d84011830 con 0x7f2d8c0cd210 2023-10-11T09:45:56.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.900+0000 7f2d90fce700 1 -- 172.21.15.78:0/445471193 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d84011990 con 0x7f2d8c0cd210 2023-10-11T09:45:56.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.900+0000 7f2d90fce700 1 -- 172.21.15.78:0/445471193 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d84011d40 con 0x7f2d8c0cd210 2023-10-11T09:45:56.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.900+0000 7f2d94234700 1 -- 172.21.15.78:0/445471193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d8c0cd210 msgr2=0x7f2d8c06be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:56.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.900+0000 7f2d94234700 1 --2- 172.21.15.78:0/445471193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d8c0cd210 0x7f2d8c06be60 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009850 tx=0x7f2d84009b60 comp rx=0 tx=0).stop 2023-10-11T09:45:56.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.900+0000 7f2d94234700 1 -- 172.21.15.78:0/445471193 shutdown_connections 2023-10-11T09:45:56.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.900+0000 7f2d94234700 1 --2- 172.21.15.78:0/445471193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d8c0cd210 0x7f2d8c06be60 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:56.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.900+0000 7f2d94234700 1 -- 172.21.15.78:0/445471193 >> 172.21.15.78:0/445471193 conn(0x7f2d8c06aca0 msgr2=0x7f2d8c0b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:56.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.900+0000 7f2d94234700 1 -- 172.21.15.78:0/445471193 shutdown_connections 2023-10-11T09:45:56.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.900+0000 7f2d94234700 1 -- 172.21.15.78:0/445471193 wait complete. 2023-10-11T09:45:56.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.904+0000 7f2d94234700 1 Processor -- start 2023-10-11T09:45:56.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.904+0000 7f2d94234700 1 -- start start 2023-10-11T09:45:56.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.904+0000 7f2d94234700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d8c0cbd30 0x7f2d8c153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:56.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.904+0000 7f2d94234700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d8c0cd210 0x7f2d8c157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:56.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.904+0000 7f2d94234700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2d8c15c880 0x7f2d8c1518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:56.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.904+0000 7f2d94234700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c0b1dd0 con 0x7f2d8c0cd210 2023-10-11T09:45:56.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.904+0000 7f2d94234700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c157b70 con 0x7f2d8c0cbd30 2023-10-11T09:45:56.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.904+0000 7f2d94234700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c154360 con 0x7f2d8c15c880 2023-10-11T09:45:56.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.904+0000 7f2d917cf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d8c0cd210 0x7f2d8c157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:56.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.908+0000 7f2d917cf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d8c0cd210 0x7f2d8c157630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45836/0 (socket says 172.21.15.78:45836) 2023-10-11T09:45:56.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.908+0000 7f2d917cf700 1 -- 172.21.15.78:0/1202087289 learned_addr learned my addr 172.21.15.78:0/1202087289 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:56.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.908+0000 7f2d91fd0700 1 --2- 172.21.15.78:0/1202087289 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d8c0cbd30 0x7f2d8c153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:56.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.908+0000 7f2d927d1700 1 --2- 172.21.15.78:0/1202087289 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2d8c15c880 0x7f2d8c1518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:56.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.908+0000 7f2d917cf700 1 -- 172.21.15.78:0/1202087289 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2d8c15c880 msgr2=0x7f2d8c1518d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:56.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.908+0000 7f2d917cf700 1 --2- 172.21.15.78:0/1202087289 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2d8c15c880 0x7f2d8c1518d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:56.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.908+0000 7f2d917cf700 1 -- 172.21.15.78:0/1202087289 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d8c0cbd30 msgr2=0x7f2d8c153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:56.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.908+0000 7f2d917cf700 1 --2- 172.21.15.78:0/1202087289 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d8c0cbd30 0x7f2d8c153d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:56.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.908+0000 7f2d917cf700 1 -- 172.21.15.78:0/1202087289 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d840094b0 con 0x7f2d8c0cd210 2023-10-11T09:45:56.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.908+0000 7f2d917cf700 1 --2- 172.21.15.78:0/1202087289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d8c0cd210 0x7f2d8c157630 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c00c4d0 tx=0x7f2d7c00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:56.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.908+0000 7f2d827fc700 1 -- 172.21.15.78:0/1202087289 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c005d90 con 0x7f2d8c0cd210 2023-10-11T09:45:56.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.908+0000 7f2d827fc700 1 -- 172.21.15.78:0/1202087289 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d7c0070d0 con 0x7f2d8c0cd210 2023-10-11T09:45:56.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.908+0000 7f2d94234700 1 -- 172.21.15.78:0/1202087289 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d8c154640 con 0x7f2d8c0cd210 2023-10-11T09:45:56.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.908+0000 7f2d827fc700 1 -- 172.21.15.78:0/1202087289 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c007480 con 0x7f2d8c0cd210 2023-10-11T09:45:56.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.908+0000 7f2d94234700 1 -- 172.21.15.78:0/1202087289 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d8c154c00 con 0x7f2d8c0cd210 2023-10-11T09:45:56.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.908+0000 7f2d94234700 1 -- 172.21.15.78:0/1202087289 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d8c154fa0 con 0x7f2d8c0cd210 2023-10-11T09:45:56.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.912+0000 7f2d827fc700 1 -- 172.21.15.78:0/1202087289 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f2d7c0075e0 con 0x7f2d8c0cd210 2023-10-11T09:45:56.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.912+0000 7f2d827fc700 1 --2- 172.21.15.78:0/1202087289 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2d70065ed0 0x7f2d70068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:56.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.912+0000 7f2d827fc700 1 -- 172.21.15.78:0/1202087289 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x7f2d7c082ed0 con 0x7f2d8c0cd210 2023-10-11T09:45:56.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.912+0000 7f2d91fd0700 1 --2- 172.21.15.78:0/1202087289 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2d70065ed0 0x7f2d70068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:56.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.916+0000 7f2d91fd0700 1 --2- 172.21.15.78:0/1202087289 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2d70065ed0 0x7f2d70068380 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2d880012e0 tx=0x7f2d88009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:56.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:56.916+0000 7f2d827fc700 1 -- 172.21.15.78:0/1202087289 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f2d8c154fa0 con 0x7f2d8c0cd210 2023-10-11T09:45:57.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.124+0000 7f2d94234700 1 -- 172.21.15.78:0/1202087289 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d8c154fa0 con 0x7f2d8c0cd210 2023-10-11T09:45:57.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.124+0000 7f2d827fc700 1 -- 172.21.15.78:0/1202087289 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4649 (secure 0 0 0) 0x7f2d8c154fa0 con 0x7f2d8c0cd210 2023-10-11T09:45:57.128 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:57.129 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":43,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5579,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3954348479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":3954348479},{"type":"v1","addr":"172.21.15.133:6835","nonce":3954348479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5783,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3328076595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":3328076595},{"type":"v1","addr":"172.21.15.78:6837","nonce":3328076595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":6100,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/4029676589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4029676589},{"type":"v1","addr":"172.21.15.133:6834","nonce":4029676589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6582,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/1666964283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":1666964283},{"type":"v1","addr":"172.21.15.78:6836","nonce":1666964283}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:45:54.252958+0000","modified":"2023-10-11T09:45:55.350786+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[9,10],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-11T09:45:57.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.128+0000 7f2d94234700 1 -- 172.21.15.78:0/1202087289 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2d70065ed0 msgr2=0x7f2d70068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.128+0000 7f2d94234700 1 --2- 172.21.15.78:0/1202087289 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2d70065ed0 0x7f2d70068380 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2d880012e0 tx=0x7f2d88009040 comp rx=0 tx=0).stop 2023-10-11T09:45:57.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.128+0000 7f2d94234700 1 -- 172.21.15.78:0/1202087289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d8c0cd210 msgr2=0x7f2d8c157630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.128+0000 7f2d94234700 1 --2- 172.21.15.78:0/1202087289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d8c0cd210 0x7f2d8c157630 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c00c4d0 tx=0x7f2d7c00c7e0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.128+0000 7f2d94234700 1 -- 172.21.15.78:0/1202087289 shutdown_connections 2023-10-11T09:45:57.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.128+0000 7f2d94234700 1 --2- 172.21.15.78:0/1202087289 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2d70065ed0 0x7f2d70068380 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.128+0000 7f2d94234700 1 --2- 172.21.15.78:0/1202087289 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2d8c15c880 0x7f2d8c1518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.128+0000 7f2d94234700 1 --2- 172.21.15.78:0/1202087289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d8c0cd210 0x7f2d8c157630 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.128+0000 7f2d94234700 1 --2- 172.21.15.78:0/1202087289 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d8c0cbd30 0x7f2d8c153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.128+0000 7f2d94234700 1 -- 172.21.15.78:0/1202087289 >> 172.21.15.78:0/1202087289 conn(0x7f2d8c06aca0 msgr2=0x7f2d8c0c9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:57.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.128+0000 7f2d94234700 1 -- 172.21.15.78:0/1202087289 shutdown_connections 2023-10-11T09:45:57.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.128+0000 7f2d94234700 1 -- 172.21.15.78:0/1202087289 wait complete. 2023-10-11T09:45:57.133 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 43 2023-10-11T09:45:57.152 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:45:57.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.316+0000 7fd61f2aa700 1 Processor -- start 2023-10-11T09:45:57.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.316+0000 7fd61f2aa700 1 -- start start 2023-10-11T09:45:57.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.320+0000 7fd61f2aa700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fd61813f740 0x7fd6180ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.320+0000 7fd61f2aa700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6180b14c0 con 0x7fd61806cd00 2023-10-11T09:45:57.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.320+0000 7fd61f2aa700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd618070c80 con 0x7fd61806d180 2023-10-11T09:45:57.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.320+0000 7fd61f2aa700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6180ce4c0 con 0x7fd61813f740 2023-10-11T09:45:57.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.320+0000 7fd61e2a8700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fd61813f740 0x7fd6180ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.320+0000 7fd61e2a8700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fd61813f740 0x7fd6180ccab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45840/0 (socket says 172.21.15.78:45840) 2023-10-11T09:45:57.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.320+0000 7fd61e2a8700 1 -- 172.21.15.78:0/1069129297 learned_addr learned my addr 172.21.15.78:0/1069129297 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:57.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.320+0000 7fd61e2a8700 1 -- 172.21.15.78:0/1069129297 >> v1:172.21.15.133:6789/0 conn(0x7fd61806d180 legacy=0x7fd6180b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:57.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.320+0000 7fd61d2a6700 1 -- 172.21.15.78:0/1069129297 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2075419424 0 0) 0x7fd6180b14c0 con 0x7fd61806cd00 2023-10-11T09:45:57.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.320+0000 7fd61e2a8700 1 -- 172.21.15.78:0/1069129297 >> v1:172.21.15.78:6789/0 conn(0x7fd61806cd00 legacy=0x7fd6180b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.320+0000 7fd61e2a8700 1 -- 172.21.15.78:0/1069129297 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd61806d600 con 0x7fd61813f740 2023-10-11T09:45:57.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.320+0000 7fd61e2a8700 1 --2- 172.21.15.78:0/1069129297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd61813f740 0x7fd6180ccab0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fd614009910 tx=0x7fd614009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a2be87108a36029 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:57.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.320+0000 7fd61d2a6700 1 -- 172.21.15.78:0/1069129297 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61400ba60 con 0x7fd61813f740 2023-10-11T09:45:57.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.324+0000 7fd61d2a6700 1 -- 172.21.15.78:0/1069129297 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd61400bbc0 con 0x7fd61813f740 2023-10-11T09:45:57.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.324+0000 7fd61d2a6700 1 -- 172.21.15.78:0/1069129297 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd614009e60 con 0x7fd61813f740 2023-10-11T09:45:57.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.324+0000 7fd61f2aa700 1 -- 172.21.15.78:0/1069129297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd61813f740 msgr2=0x7fd6180ccab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.324+0000 7fd61f2aa700 1 --2- 172.21.15.78:0/1069129297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd61813f740 0x7fd6180ccab0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fd614009910 tx=0x7fd614009c20 comp rx=0 tx=0).stop 2023-10-11T09:45:57.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.324+0000 7fd61f2aa700 1 -- 172.21.15.78:0/1069129297 shutdown_connections 2023-10-11T09:45:57.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.324+0000 7fd61f2aa700 1 --2- 172.21.15.78:0/1069129297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd61813f740 0x7fd6180ccab0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.324+0000 7fd61f2aa700 1 -- 172.21.15.78:0/1069129297 >> 172.21.15.78:0/1069129297 conn(0x7fd6180cea60 msgr2=0x7fd6180cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:57.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.324+0000 7fd61f2aa700 1 -- 172.21.15.78:0/1069129297 shutdown_connections 2023-10-11T09:45:57.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.324+0000 7fd61f2aa700 1 -- 172.21.15.78:0/1069129297 wait complete. 2023-10-11T09:45:57.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.324+0000 7fd61f2aa700 1 Processor -- start 2023-10-11T09:45:57.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.328+0000 7fd61f2aa700 1 -- start start 2023-10-11T09:45:57.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.328+0000 7fd61f2aa700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd61806cd00 0x7fd6180c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.328+0000 7fd61f2aa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd61806d180 0x7fd618153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.328+0000 7fd61f2aa700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd61813f740 0x7fd61813bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.328+0000 7fd61f2aa700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd618156080 con 0x7fd61806d180 2023-10-11T09:45:57.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.328+0000 7fd61f2aa700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61813c320 con 0x7fd61813f740 2023-10-11T09:45:57.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.328+0000 7fd61f2aa700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6180c6130 con 0x7fd61806cd00 2023-10-11T09:45:57.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.328+0000 7fd61daa7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd61806d180 0x7fd618153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.328+0000 7fd61daa7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd61806d180 0x7fd618153cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45850/0 (socket says 172.21.15.78:45850) 2023-10-11T09:45:57.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.328+0000 7fd61daa7700 1 -- 172.21.15.78:0/3148935471 learned_addr learned my addr 172.21.15.78:0/3148935471 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:57.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.328+0000 7fd61e2a8700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd61806cd00 0x7fd6180c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.328+0000 7fd61eaa9700 1 --2- 172.21.15.78:0/3148935471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd61813f740 0x7fd61813bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.328+0000 7fd61daa7700 1 -- 172.21.15.78:0/3148935471 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd61806cd00 msgr2=0x7fd6180c5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.328+0000 7fd61daa7700 1 --2- 172.21.15.78:0/3148935471 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd61806cd00 0x7fd6180c5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.328+0000 7fd61daa7700 1 -- 172.21.15.78:0/3148935471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd61813f740 msgr2=0x7fd61813bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.328+0000 7fd61daa7700 1 --2- 172.21.15.78:0/3148935471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd61813f740 0x7fd61813bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.328+0000 7fd61daa7700 1 -- 172.21.15.78:0/3148935471 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd614009570 con 0x7fd61806d180 2023-10-11T09:45:57.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.328+0000 7fd61e2a8700 1 --2- 172.21.15.78:0/3148935471 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd61806cd00 0x7fd6180c5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:45:57.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.328+0000 7fd61daa7700 1 --2- 172.21.15.78:0/3148935471 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd61806d180 0x7fd618153cb0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fd60800c480 tx=0x7fd60800c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:57.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.328+0000 7fd60effd700 1 -- 172.21.15.78:0/3148935471 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6080054e0 con 0x7fd61806d180 2023-10-11T09:45:57.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.328+0000 7fd60effd700 1 -- 172.21.15.78:0/3148935471 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd60800ce50 con 0x7fd61806d180 2023-10-11T09:45:57.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.332+0000 7fd60effd700 1 -- 172.21.15.78:0/3148935471 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6080038e0 con 0x7fd61806d180 2023-10-11T09:45:57.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.332+0000 7fd61f2aa700 1 -- 172.21.15.78:0/3148935471 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6180c6410 con 0x7fd61806d180 2023-10-11T09:45:57.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.332+0000 7fd61f2aa700 1 -- 172.21.15.78:0/3148935471 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6180c69d0 con 0x7fd61806d180 2023-10-11T09:45:57.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.332+0000 7fd61f2aa700 1 -- 172.21.15.78:0/3148935471 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd618065d40 con 0x7fd61806d180 2023-10-11T09:45:57.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.332+0000 7fd60effd700 1 -- 172.21.15.78:0/3148935471 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fd608003a40 con 0x7fd61806d180 2023-10-11T09:45:57.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.336+0000 7fd60effd700 1 --2- 172.21.15.78:0/3148935471 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd5fc065ed0 0x7fd5fc068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.336+0000 7fd60effd700 1 -- 172.21.15.78:0/3148935471 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x7fd608082d30 con 0x7fd61806d180 2023-10-11T09:45:57.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.336+0000 7fd61e2a8700 1 --2- 172.21.15.78:0/3148935471 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd5fc065ed0 0x7fd5fc068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.336+0000 7fd61e2a8700 1 --2- 172.21.15.78:0/3148935471 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd5fc065ed0 0x7fd5fc068380 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd614000c00 tx=0x7fd614008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:57.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.340+0000 7fd60effd700 1 -- 172.21.15.78:0/3148935471 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fd608050130 con 0x7fd61806d180 2023-10-11T09:45:57.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.556+0000 7fd61f2aa700 1 -- 172.21.15.78:0/3148935471 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6180cdcb0 con 0x7fd61806d180 2023-10-11T09:45:57.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.556+0000 7fd60effd700 1 -- 172.21.15.78:0/3148935471 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4649 (secure 0 0 0) 0x7fd608017020 con 0x7fd61806d180 2023-10-11T09:45:57.562 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:57.563 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":43,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5579,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3954348479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":3954348479},{"type":"v1","addr":"172.21.15.133:6835","nonce":3954348479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5783,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3328076595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":3328076595},{"type":"v1","addr":"172.21.15.78:6837","nonce":3328076595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":6100,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/4029676589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4029676589},{"type":"v1","addr":"172.21.15.133:6834","nonce":4029676589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":6582,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/1666964283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":1666964283},{"type":"v1","addr":"172.21.15.78:6836","nonce":1666964283}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:45:54.252958+0000","modified":"2023-10-11T09:45:55.350786+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[9,10],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-11T09:45:57.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.560+0000 7fd61f2aa700 1 -- 172.21.15.78:0/3148935471 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd5fc065ed0 msgr2=0x7fd5fc068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.560+0000 7fd61f2aa700 1 --2- 172.21.15.78:0/3148935471 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd5fc065ed0 0x7fd5fc068380 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd614000c00 tx=0x7fd614008030 comp rx=0 tx=0).stop 2023-10-11T09:45:57.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.560+0000 7fd61f2aa700 1 -- 172.21.15.78:0/3148935471 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd61806d180 msgr2=0x7fd618153cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.560+0000 7fd61f2aa700 1 --2- 172.21.15.78:0/3148935471 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd61806d180 0x7fd618153cb0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fd60800c480 tx=0x7fd60800c790 comp rx=0 tx=0).stop 2023-10-11T09:45:57.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.560+0000 7fd61f2aa700 1 -- 172.21.15.78:0/3148935471 shutdown_connections 2023-10-11T09:45:57.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.560+0000 7fd61f2aa700 1 --2- 172.21.15.78:0/3148935471 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd5fc065ed0 0x7fd5fc068380 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.560+0000 7fd61f2aa700 1 --2- 172.21.15.78:0/3148935471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd61813f740 0x7fd61813bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.560+0000 7fd61f2aa700 1 --2- 172.21.15.78:0/3148935471 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd61806d180 0x7fd618153cb0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.560+0000 7fd61f2aa700 1 --2- 172.21.15.78:0/3148935471 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd61806cd00 0x7fd6180c5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.560+0000 7fd61f2aa700 1 -- 172.21.15.78:0/3148935471 >> 172.21.15.78:0/3148935471 conn(0x7fd6180cea60 msgr2=0x7fd618158d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:57.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.560+0000 7fd61f2aa700 1 -- 172.21.15.78:0/3148935471 shutdown_connections 2023-10-11T09:45:57.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.560+0000 7fd61f2aa700 1 -- 172.21.15.78:0/3148935471 wait complete. 2023-10-11T09:45:57.568 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 43 2023-10-11T09:45:57.586 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset session 2023-10-11T09:45:57.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.640+0000 7fbbb365d380 1 -- start start 2023-10-11T09:45:57.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.640+0000 7fbbb365d380 1 Processor -- start 2023-10-11T09:45:57.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.640+0000 7fbbb365d380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ddce8d3400 0x55ddce978b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.640+0000 7fbbb365d380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ddce8d3800 0x55ddce979080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.640+0000 7fbbb365d380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddce8d3c00 0x55ddce979600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.640+0000 7fbbb365d380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddce952a00 con 0x55ddce8d3c00 2023-10-11T09:45:57.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.640+0000 7fbbb365d380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddce952780 con 0x55ddce8d3800 2023-10-11T09:45:57.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.640+0000 7fbbb365d380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ddce952640 con 0x55ddce8d3400 2023-10-11T09:45:57.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.640+0000 7fbbb2618700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddce8d3c00 0x55ddce979600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.640+0000 7fbbb2618700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddce8d3c00 0x55ddce979600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45874/0 (socket says 172.21.15.78:45874) 2023-10-11T09:45:57.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.640+0000 7fbbb2618700 1 -- 172.21.15.78:0/3051805534 learned_addr learned my addr 172.21.15.78:0/3051805534 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:57.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.640+0000 7fbbb1616700 1 --2- 172.21.15.78:0/3051805534 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ddce8d3800 0x55ddce979080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.640+0000 7fbbb1e17700 1 --2- 172.21.15.78:0/3051805534 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ddce8d3400 0x55ddce978b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.644+0000 7fbbb2618700 1 -- 172.21.15.78:0/3051805534 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ddce8d3400 msgr2=0x55ddce978b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:57.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.644+0000 7fbbb2618700 1 --2- 172.21.15.78:0/3051805534 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ddce8d3400 0x55ddce978b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.644+0000 7fbbb2618700 1 -- 172.21.15.78:0/3051805534 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ddce8d3800 msgr2=0x55ddce979080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:57.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.644+0000 7fbbb2618700 1 --2- 172.21.15.78:0/3051805534 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ddce8d3800 0x55ddce979080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.644+0000 7fbbb2618700 1 -- 172.21.15.78:0/3051805534 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddce8f4b60 con 0x55ddce8d3c00 2023-10-11T09:45:57.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.644+0000 7fbbb1e17700 1 --2- 172.21.15.78:0/3051805534 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ddce8d3400 0x55ddce978b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:45:57.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.644+0000 7fbbb2618700 1 --2- 172.21.15.78:0/3051805534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddce8d3c00 0x55ddce979600 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x55ddcf6b0b10 tx=0x55ddcf6bafc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=126ce36c8bda976a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:57.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.644+0000 7fbbaf612700 1 -- 172.21.15.78:0/3051805534 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ddce903600 con 0x55ddce8d3c00 2023-10-11T09:45:57.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.644+0000 7fbbaf612700 1 -- 172.21.15.78:0/3051805534 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ddce91e180 con 0x55ddce8d3c00 2023-10-11T09:45:57.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.644+0000 7fbbaf612700 1 -- 172.21.15.78:0/3051805534 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ddce9031e0 con 0x55ddce8d3c00 2023-10-11T09:45:57.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.644+0000 7fbbb365d380 1 -- 172.21.15.78:0/3051805534 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ddce8f4340 con 0x55ddce8d3c00 2023-10-11T09:45:57.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.644+0000 7fbbb365d380 1 -- 172.21.15.78:0/3051805534 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55ddce8f4000 con 0x55ddce8d3c00 2023-10-11T09:45:57.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.644+0000 7fbbb365d380 4 waiting for MDS map... 2023-10-11T09:45:57.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.644+0000 7fbbaf612700 1 -- 172.21.15.78:0/3051805534 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x55ddce90cc40 con 0x55ddce8d3c00 2023-10-11T09:45:57.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.644+0000 7fbbaf612700 1 -- 172.21.15.78:0/3051805534 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 43) v1 ==== 3118+0+0 (secure 0 0 0) 0x55ddcebce000 con 0x55ddce8d3c00 2023-10-11T09:45:57.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.644+0000 7fbbb365d380 4 Got MDS map 43 2023-10-11T09:45:57.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.644+0000 7fbbb365d380 10 main: main 2023-10-11T09:45:57.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.644+0000 7fbbb365d380 4 main: connecting to RADOS... 2023-10-11T09:45:57.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.644+0000 7fbbb365d380 1 Processor -- start 2023-10-11T09:45:57.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.644+0000 7fbbb365d380 1 -- start start 2023-10-11T09:45:57.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.648+0000 7fbbb365d380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ddcebc2800 0x55ddce979b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.648+0000 7fbbb365d380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ddcebc2c00 0x55ddce97a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.648+0000 7fbbb365d380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcebc3000 0x55ddce97a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.648+0000 7fbbb365d380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddce909e00 con 0x55ddcebc3000 2023-10-11T09:45:57.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.648+0000 7fbbb365d380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddce952780 con 0x55ddcebc2c00 2023-10-11T09:45:57.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.648+0000 7fbbb365d380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ddce952500 con 0x55ddcebc2800 2023-10-11T09:45:57.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.648+0000 7fbbb1616700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcebc3000 0x55ddce97a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.648+0000 7fbbb1616700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcebc3000 0x55ddce97a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45886/0 (socket says 172.21.15.78:45886) 2023-10-11T09:45:57.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.648+0000 7fbbb1616700 1 -- 172.21.15.78:0/2611728651 learned_addr learned my addr 172.21.15.78:0/2611728651 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:57.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.648+0000 7fbbb1e17700 1 --2- 172.21.15.78:0/2611728651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ddcebc2c00 0x55ddce97a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.648+0000 7fbbb1616700 1 --2- 172.21.15.78:0/2611728651 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ddcebc2800 0x55ddce979b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.648+0000 7fbbb1616700 1 -- 172.21.15.78:0/2611728651 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ddcebc2800 msgr2=0x55ddce979b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:57.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.648+0000 7fbbb1616700 1 --2- 172.21.15.78:0/2611728651 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ddcebc2800 0x55ddce979b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.648+0000 7fbbb1616700 1 -- 172.21.15.78:0/2611728651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ddcebc2c00 msgr2=0x55ddce97a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:57.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.648+0000 7fbbb1616700 1 --2- 172.21.15.78:0/2611728651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ddcebc2c00 0x55ddce97a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.648+0000 7fbbb1616700 1 -- 172.21.15.78:0/2611728651 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddce8f4ea0 con 0x55ddcebc3000 2023-10-11T09:45:57.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.648+0000 7fbbb1616700 1 --2- 172.21.15.78:0/2611728651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcebc3000 0x55ddce97a680 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x55ddcf6b1f80 tx=0x55ddcebd4c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=973266b07f206727 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:57.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.648+0000 7fbbad60e700 1 -- 172.21.15.78:0/2611728651 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ddce903600 con 0x55ddcebc3000 2023-10-11T09:45:57.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.648+0000 7fbbad60e700 1 -- 172.21.15.78:0/2611728651 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ddce91e480 con 0x55ddcebc3000 2023-10-11T09:45:57.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.648+0000 7fbbad60e700 1 -- 172.21.15.78:0/2611728651 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ddce9031e0 con 0x55ddcebc3000 2023-10-11T09:45:57.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.648+0000 7fbbb365d380 1 -- 172.21.15.78:0/2611728651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcebc3000 msgr2=0x55ddce97a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.648+0000 7fbbb365d380 1 --2- 172.21.15.78:0/2611728651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcebc3000 0x55ddce97a680 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x55ddcf6b1f80 tx=0x55ddcebd4c00 comp rx=0 tx=0).stop 2023-10-11T09:45:57.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.648+0000 7fbbb365d380 1 -- 172.21.15.78:0/2611728651 shutdown_connections 2023-10-11T09:45:57.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.648+0000 7fbbb365d380 1 --2- 172.21.15.78:0/2611728651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcebc3000 0x55ddce97a680 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.648+0000 7fbbb365d380 1 --2- 172.21.15.78:0/2611728651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ddcebc2c00 0x55ddce97a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.648+0000 7fbbb365d380 1 --2- 172.21.15.78:0/2611728651 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ddcebc2800 0x55ddce979b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.648+0000 7fbbb365d380 1 -- 172.21.15.78:0/2611728651 >> 172.21.15.78:0/2611728651 conn(0x55ddcebc2400 msgr2=0x55ddce9bf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:57.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.648+0000 7fbbb365d380 1 -- 172.21.15.78:0/2611728651 shutdown_connections 2023-10-11T09:45:57.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.648+0000 7fbbb365d380 1 -- 172.21.15.78:0/2611728651 wait complete. 2023-10-11T09:45:57.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.648+0000 7fbbb365d380 1 Processor -- start 2023-10-11T09:45:57.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.648+0000 7fbbb365d380 1 -- start start 2023-10-11T09:45:57.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.652+0000 7fbbb365d380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ddcebc2800 0x55ddce979b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.652+0000 7fbbb365d380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ddcebc2c00 0x55ddce97a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.652+0000 7fbbb365d380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcebc3400 0x55ddce97ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.652+0000 7fbbb365d380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddce9523c0 con 0x55ddcebc3400 2023-10-11T09:45:57.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.652+0000 7fbbb365d380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddce952280 con 0x55ddcebc2c00 2023-10-11T09:45:57.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.652+0000 7fbbb365d380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ddce952140 con 0x55ddcebc2800 2023-10-11T09:45:57.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.652+0000 7fbbb1616700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcebc3400 0x55ddce97ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.652+0000 7fbbb1616700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcebc3400 0x55ddce97ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45890/0 (socket says 172.21.15.78:45890) 2023-10-11T09:45:57.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.652+0000 7fbbb1616700 1 -- 172.21.15.78:0/1053930562 learned_addr learned my addr 172.21.15.78:0/1053930562 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:57.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.652+0000 7fbbb1e17700 1 --2- 172.21.15.78:0/1053930562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ddcebc2c00 0x55ddce97a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.652+0000 7fbbb1616700 1 --2- 172.21.15.78:0/1053930562 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ddcebc2800 0x55ddce979b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.652+0000 7fbbb1616700 1 -- 172.21.15.78:0/1053930562 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ddcebc2800 msgr2=0x55ddce979b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.652+0000 7fbbb1616700 1 --2- 172.21.15.78:0/1053930562 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ddcebc2800 0x55ddce979b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.652+0000 7fbbb1616700 1 -- 172.21.15.78:0/1053930562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ddcebc2c00 msgr2=0x55ddce97a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.652+0000 7fbbb1616700 1 --2- 172.21.15.78:0/1053930562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ddcebc2c00 0x55ddce97a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.652+0000 7fbbb1616700 1 -- 172.21.15.78:0/1053930562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddce8f4b60 con 0x55ddcebc3400 2023-10-11T09:45:57.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.652+0000 7fbbb1616700 1 --2- 172.21.15.78:0/1053930562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcebc3400 0x55ddce97ac00 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x55ddcf6acc00 tx=0x55ddcebd4c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:57.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.652+0000 7fbbabe0b700 1 -- 172.21.15.78:0/1053930562 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ddce902dc0 con 0x55ddcebc3400 2023-10-11T09:45:57.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.652+0000 7fbbabe0b700 1 -- 172.21.15.78:0/1053930562 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ddce91e180 con 0x55ddcebc3400 2023-10-11T09:45:57.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.652+0000 7fbbabe0b700 1 -- 172.21.15.78:0/1053930562 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ddce903600 con 0x55ddcebc3400 2023-10-11T09:45:57.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.652+0000 7fbbb365d380 1 -- 172.21.15.78:0/1053930562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ddce8f4340 con 0x55ddcebc3400 2023-10-11T09:45:57.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.652+0000 7fbbb365d380 1 -- 172.21.15.78:0/1053930562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ddce8f5520 con 0x55ddcebc3400 2023-10-11T09:45:57.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.652+0000 7fbbb365d380 4 main: resolving pool 8 2023-10-11T09:45:57.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.656+0000 7fbbabe0b700 1 -- 172.21.15.78:0/1053930562 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x55ddce950840 con 0x55ddcebc3400 2023-10-11T09:45:57.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.656+0000 7fbbabe0b700 1 --2- 172.21.15.78:0/1053930562 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55ddced22000 0x55ddce97b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.656+0000 7fbbabe0b700 1 -- 172.21.15.78:0/1053930562 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x55ddce90ca80 con 0x55ddcebc3400 2023-10-11T09:45:57.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.656+0000 7fbbb365d380 4 main: creating IoCtx.. 2023-10-11T09:45:57.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.656+0000 7fbbb365d380 1 --2- 172.21.15.78:0/1053930562 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55ddcebc3800 0x55ddce97b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.656+0000 7fbbb365d380 1 -- 172.21.15.78:0/1053930562 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:1 8.b 8:d0630e4c:::mds0_sessionmap:head [omap-clear,omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ddcebc3c00 con 0x55ddcebc3800 2023-10-11T09:45:57.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.656+0000 7fbbb1e17700 1 --2- 172.21.15.78:0/1053930562 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55ddced22000 0x55ddce97b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.656+0000 7fbbb1616700 1 --2- 172.21.15.78:0/1053930562 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55ddcebc3800 0x55ddce97b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.656+0000 7fbbb1616700 1 --2- 172.21.15.78:0/1053930562 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55ddcebc3800 0x55ddce97b180 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:57.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.656+0000 7fbbb1e17700 1 --2- 172.21.15.78:0/1053930562 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55ddced22000 0x55ddce97b700 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x55ddce929800 tx=0x55ddcf6ba000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:57.672 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:45:57.672 INFO:teuthology.orchestra.run.smithi078.stdout: "0": { 2023-10-11T09:45:57.672 INFO:teuthology.orchestra.run.smithi078.stdout: "data": {}, 2023-10-11T09:45:57.672 INFO:teuthology.orchestra.run.smithi078.stdout: "result": 0 2023-10-11T09:45:57.673 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:45:57.673 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:45:57.673 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:57.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.660+0000 7fbbb1616700 1 -- 172.21.15.78:0/1053930562 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(1 mds0_sessionmap [omap-clear,omap-set-header] v66'5 uv5 ondisk = 0) v8 ==== 201+0+0 (crc 0 0 0) 0x55ddcf75afc0 con 0x55ddcebc3800 2023-10-11T09:45:57.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.660+0000 7fbbb365d380 1 -- 172.21.15.78:0/1053930562 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55ddcebc3800 msgr2=0x55ddce97b180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.660+0000 7fbbb365d380 1 --2- 172.21.15.78:0/1053930562 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55ddcebc3800 0x55ddce97b180 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.660+0000 7fbbb365d380 1 -- 172.21.15.78:0/1053930562 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55ddced22000 msgr2=0x55ddce97b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.660+0000 7fbbb365d380 1 --2- 172.21.15.78:0/1053930562 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55ddced22000 0x55ddce97b700 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x55ddce929800 tx=0x55ddcf6ba000 comp rx=0 tx=0).stop 2023-10-11T09:45:57.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.660+0000 7fbbb365d380 1 -- 172.21.15.78:0/1053930562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcebc3400 msgr2=0x55ddce97ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.660+0000 7fbbb365d380 1 --2- 172.21.15.78:0/1053930562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcebc3400 0x55ddce97ac00 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x55ddcf6acc00 tx=0x55ddcebd4c60 comp rx=0 tx=0).stop 2023-10-11T09:45:57.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.660+0000 7fbbb1e17700 1 -- 172.21.15.78:0/1053930562 reap_dead start 2023-10-11T09:45:57.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.660+0000 7fbbb365d380 1 -- 172.21.15.78:0/1053930562 shutdown_connections 2023-10-11T09:45:57.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.660+0000 7fbbb365d380 1 -- 172.21.15.78:0/1053930562 >> 172.21.15.78:0/1053930562 conn(0x55ddcebc2400 msgr2=0x55ddce9bf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:57.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.660+0000 7fbbb365d380 1 -- 172.21.15.78:0/1053930562 shutdown_connections 2023-10-11T09:45:57.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.660+0000 7fbbb365d380 1 -- 172.21.15.78:0/1053930562 wait complete. 2023-10-11T09:45:57.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.660+0000 7fbbb365d380 1 -- 172.21.15.78:0/3051805534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddce8d3c00 msgr2=0x55ddce979600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.660+0000 7fbbb365d380 1 --2- 172.21.15.78:0/3051805534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddce8d3c00 0x55ddce979600 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x55ddcf6b0b10 tx=0x55ddcf6bafc0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.660+0000 7fbbb365d380 1 -- 172.21.15.78:0/3051805534 shutdown_connections 2023-10-11T09:45:57.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.660+0000 7fbbb365d380 1 --2- 172.21.15.78:0/3051805534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddce8d3c00 0x55ddce979600 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.660+0000 7fbbb365d380 1 --2- 172.21.15.78:0/3051805534 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ddce8d3800 0x55ddce979080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.660+0000 7fbbb365d380 1 --2- 172.21.15.78:0/3051805534 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ddce8d3400 0x55ddce978b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.660+0000 7fbbb365d380 1 -- 172.21.15.78:0/3051805534 >> 172.21.15.78:0/3051805534 conn(0x55ddce8d2c00 msgr2=0x55ddce9bf000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:57.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.660+0000 7fbbb365d380 1 -- 172.21.15.78:0/3051805534 shutdown_connections 2023-10-11T09:45:57.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.660+0000 7fbbb365d380 1 -- 172.21.15.78:0/3051805534 wait complete. 2023-10-11T09:45:57.679 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'session'] in time 0:00:00.093416, result: { "0": { "data": {}, "result": 0 } } 2023-10-11T09:45:57.679 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset snap 2023-10-11T09:45:57.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.740+0000 7f8e5ad12380 1 -- start start 2023-10-11T09:45:57.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.740+0000 7f8e5ad12380 1 Processor -- start 2023-10-11T09:45:57.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.744+0000 7f8e5ad12380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55abbe297800 0x55abbe33c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.744+0000 7f8e5ad12380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55abbe297400 0x55abbe33cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.744+0000 7f8e5ad12380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbe297000 0x55abbe33d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.744+0000 7f8e5ad12380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbe2cde00 con 0x55abbe297000 2023-10-11T09:45:57.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.744+0000 7f8e5ad12380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbe316a00 con 0x55abbe297800 2023-10-11T09:45:57.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.744+0000 7f8e5ad12380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55abbe316640 con 0x55abbe297400 2023-10-11T09:45:57.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.744+0000 7f8e59ccd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbe297000 0x55abbe33d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.744+0000 7f8e59ccd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbe297000 0x55abbe33d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45900/0 (socket says 172.21.15.78:45900) 2023-10-11T09:45:57.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.744+0000 7f8e59ccd700 1 -- 172.21.15.78:0/1031146539 learned_addr learned my addr 172.21.15.78:0/1031146539 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:57.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.744+0000 7f8e594cc700 1 --2- 172.21.15.78:0/1031146539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55abbe297800 0x55abbe33c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.744+0000 7f8e58ccb700 1 --2- 172.21.15.78:0/1031146539 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55abbe297400 0x55abbe33cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.744+0000 7f8e59ccd700 1 -- 172.21.15.78:0/1031146539 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55abbe297400 msgr2=0x55abbe33cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:57.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.744+0000 7f8e59ccd700 1 --2- 172.21.15.78:0/1031146539 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55abbe297400 0x55abbe33cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.744+0000 7f8e59ccd700 1 -- 172.21.15.78:0/1031146539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55abbe297800 msgr2=0x55abbe33c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:57.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.744+0000 7f8e59ccd700 1 --2- 172.21.15.78:0/1031146539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55abbe297800 0x55abbe33c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.744+0000 7f8e59ccd700 1 -- 172.21.15.78:0/1031146539 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abbe2b8b60 con 0x55abbe297000 2023-10-11T09:45:57.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.744+0000 7f8e58ccb700 1 --2- 172.21.15.78:0/1031146539 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55abbe297400 0x55abbe33cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:45:57.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.744+0000 7f8e594cc700 1 --2- 172.21.15.78:0/1031146539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55abbe297800 0x55abbe33c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:45:57.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.744+0000 7f8e59ccd700 1 --2- 172.21.15.78:0/1031146539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbe297000 0x55abbe33d080 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x55abbe3f8c30 tx=0x55abbf070fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=944d2f605d90770f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:57.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.744+0000 7f8e56cc7700 1 -- 172.21.15.78:0/1031146539 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55abbe2c7600 con 0x55abbe297000 2023-10-11T09:45:57.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.744+0000 7f8e56cc7700 1 -- 172.21.15.78:0/1031146539 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55abbe2e2480 con 0x55abbe297000 2023-10-11T09:45:57.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.744+0000 7f8e56cc7700 1 -- 172.21.15.78:0/1031146539 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55abbe2c71e0 con 0x55abbe297000 2023-10-11T09:45:57.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.744+0000 7f8e5ad12380 1 -- 172.21.15.78:0/1031146539 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55abbe2b8340 con 0x55abbe297000 2023-10-11T09:45:57.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.744+0000 7f8e5ad12380 1 -- 172.21.15.78:0/1031146539 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55abbe2b8000 con 0x55abbe297000 2023-10-11T09:45:57.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.744+0000 7f8e5ad12380 4 waiting for MDS map... 2023-10-11T09:45:57.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.744+0000 7f8e56cc7700 1 -- 172.21.15.78:0/1031146539 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x55abbe2d0c40 con 0x55abbe297000 2023-10-11T09:45:57.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.744+0000 7f8e56cc7700 1 -- 172.21.15.78:0/1031146539 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 43) v1 ==== 3118+0+0 (secure 0 0 0) 0x55abbf10e000 con 0x55abbe297000 2023-10-11T09:45:57.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.744+0000 7f8e5ad12380 4 Got MDS map 43 2023-10-11T09:45:57.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.744+0000 7f8e5ad12380 10 main: main 2023-10-11T09:45:57.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.744+0000 7f8e5ad12380 4 main: connecting to RADOS... 2023-10-11T09:45:57.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.748+0000 7f8e5ad12380 1 Processor -- start 2023-10-11T09:45:57.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.748+0000 7f8e5ad12380 1 -- start start 2023-10-11T09:45:57.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.748+0000 7f8e5ad12380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbe58c400 0x55abbe33d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.748+0000 7f8e5ad12380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55abbe58c800 0x55abbe33db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.748+0000 7f8e5ad12380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55abbe58cc00 0x55abbe33e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.748+0000 7f8e5ad12380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbe316500 con 0x55abbe58c400 2023-10-11T09:45:57.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.748+0000 7f8e5ad12380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbe3163c0 con 0x55abbe58cc00 2023-10-11T09:45:57.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.748+0000 7f8e5ad12380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55abbe316a00 con 0x55abbe58c800 2023-10-11T09:45:57.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.748+0000 7f8e58ccb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbe58c400 0x55abbe33d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.748+0000 7f8e58ccb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbe58c400 0x55abbe33d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45914/0 (socket says 172.21.15.78:45914) 2023-10-11T09:45:57.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.748+0000 7f8e58ccb700 1 -- 172.21.15.78:0/2621602831 learned_addr learned my addr 172.21.15.78:0/2621602831 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:57.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.748+0000 7f8e58ccb700 1 -- 172.21.15.78:0/2621602831 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55abbe58c800 msgr2=0x55abbe33db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:57.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.748+0000 7f8e594cc700 1 --2- 172.21.15.78:0/2621602831 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55abbe58c800 0x55abbe33db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.748+0000 7f8e58ccb700 1 --2- 172.21.15.78:0/2621602831 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55abbe58c800 0x55abbe33db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.748+0000 7f8e58ccb700 1 -- 172.21.15.78:0/2621602831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55abbe58cc00 msgr2=0x55abbe33e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:57.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.748+0000 7f8e58ccb700 1 --2- 172.21.15.78:0/2621602831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55abbe58cc00 0x55abbe33e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.748+0000 7f8e58ccb700 1 -- 172.21.15.78:0/2621602831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abbe2b8ea0 con 0x55abbe58c400 2023-10-11T09:45:57.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.748+0000 7f8e594cc700 1 --2- 172.21.15.78:0/2621602831 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55abbe58c800 0x55abbe33db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:45:57.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.748+0000 7f8e58ccb700 1 --2- 172.21.15.78:0/2621602831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbe58c400 0x55abbe33d600 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x55abbe5e20c0 tx=0x55abbf070000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61c4ff3211c43d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:57.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.748+0000 7f8e54cc3700 1 -- 172.21.15.78:0/2621602831 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55abbe2c7080 con 0x55abbe58c400 2023-10-11T09:45:57.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.748+0000 7f8e54cc3700 1 -- 172.21.15.78:0/2621602831 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55abbe2e2180 con 0x55abbe58c400 2023-10-11T09:45:57.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.748+0000 7f8e54cc3700 1 -- 172.21.15.78:0/2621602831 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55abbe2c7600 con 0x55abbe58c400 2023-10-11T09:45:57.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.748+0000 7f8e5ad12380 1 -- 172.21.15.78:0/2621602831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbe58c400 msgr2=0x55abbe33d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.748+0000 7f8e5ad12380 1 --2- 172.21.15.78:0/2621602831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbe58c400 0x55abbe33d600 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x55abbe5e20c0 tx=0x55abbf070000 comp rx=0 tx=0).stop 2023-10-11T09:45:57.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.748+0000 7f8e5ad12380 1 -- 172.21.15.78:0/2621602831 shutdown_connections 2023-10-11T09:45:57.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.748+0000 7f8e5ad12380 1 --2- 172.21.15.78:0/2621602831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55abbe58cc00 0x55abbe33e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.748+0000 7f8e5ad12380 1 --2- 172.21.15.78:0/2621602831 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55abbe58c800 0x55abbe33db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.748+0000 7f8e5ad12380 1 --2- 172.21.15.78:0/2621602831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbe58c400 0x55abbe33d600 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.748+0000 7f8e5ad12380 1 -- 172.21.15.78:0/2621602831 >> 172.21.15.78:0/2621602831 conn(0x55abbe296c00 msgr2=0x55abbe383800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:57.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.748+0000 7f8e5ad12380 1 -- 172.21.15.78:0/2621602831 shutdown_connections 2023-10-11T09:45:57.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.752+0000 7f8e5ad12380 1 -- 172.21.15.78:0/2621602831 wait complete. 2023-10-11T09:45:57.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.752+0000 7f8e5ad12380 1 Processor -- start 2023-10-11T09:45:57.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.752+0000 7f8e5ad12380 1 -- start start 2023-10-11T09:45:57.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.752+0000 7f8e5ad12380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbe58c400 0x55abbe33f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.752+0000 7f8e5ad12380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55abbe58c800 0x55abbe33ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.752+0000 7f8e5ad12380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55abbe58cc00 0x55abbe33e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.752+0000 7f8e5ad12380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbe316280 con 0x55abbe58c400 2023-10-11T09:45:57.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.752+0000 7f8e5ad12380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbe316140 con 0x55abbe58c800 2023-10-11T09:45:57.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.752+0000 7f8e5ad12380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55abbe316000 con 0x55abbe58cc00 2023-10-11T09:45:57.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.752+0000 7f8e58ccb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbe58c400 0x55abbe33f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.752+0000 7f8e58ccb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbe58c400 0x55abbe33f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45926/0 (socket says 172.21.15.78:45926) 2023-10-11T09:45:57.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.752+0000 7f8e58ccb700 1 -- 172.21.15.78:0/2436151159 learned_addr learned my addr 172.21.15.78:0/2436151159 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:57.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.752+0000 7f8e594cc700 1 --2- 172.21.15.78:0/2436151159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55abbe58c800 0x55abbe33ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.752+0000 7f8e58ccb700 1 --2- 172.21.15.78:0/2436151159 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55abbe58cc00 0x55abbe33e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.752+0000 7f8e58ccb700 1 -- 172.21.15.78:0/2436151159 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55abbe58cc00 msgr2=0x55abbe33e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.752+0000 7f8e58ccb700 1 --2- 172.21.15.78:0/2436151159 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55abbe58cc00 0x55abbe33e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.752+0000 7f8e58ccb700 1 -- 172.21.15.78:0/2436151159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55abbe58c800 msgr2=0x55abbe33ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.752+0000 7f8e58ccb700 1 --2- 172.21.15.78:0/2436151159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55abbe58c800 0x55abbe33ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.752+0000 7f8e58ccb700 1 -- 172.21.15.78:0/2436151159 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abbe2b91e0 con 0x55abbe58c400 2023-10-11T09:45:57.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.752+0000 7f8e58ccb700 1 --2- 172.21.15.78:0/2436151159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbe58c400 0x55abbe33f180 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x55abbe5e2330 tx=0x55abbf070240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:57.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.752+0000 7f8e534c0700 1 -- 172.21.15.78:0/2436151159 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55abbe2c71e0 con 0x55abbe58c400 2023-10-11T09:45:57.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.752+0000 7f8e534c0700 1 -- 172.21.15.78:0/2436151159 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55abbe2e2900 con 0x55abbe58c400 2023-10-11T09:45:57.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.752+0000 7f8e5ad12380 1 -- 172.21.15.78:0/2436151159 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55abbe2b8ea0 con 0x55abbe58c400 2023-10-11T09:45:57.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.752+0000 7f8e534c0700 1 -- 172.21.15.78:0/2436151159 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55abbe2c6dc0 con 0x55abbe58c400 2023-10-11T09:45:57.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.752+0000 7f8e5ad12380 1 -- 172.21.15.78:0/2436151159 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55abbe2b8b60 con 0x55abbe58c400 2023-10-11T09:45:57.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.752+0000 7f8e5ad12380 4 main: resolving pool 8 2023-10-11T09:45:57.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.756+0000 7f8e534c0700 1 -- 172.21.15.78:0/2436151159 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x55abbe314840 con 0x55abbe58c400 2023-10-11T09:45:57.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.756+0000 7f8e534c0700 1 --2- 172.21.15.78:0/2436151159 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55abbe6d4000 0x55abbe33f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.756+0000 7f8e534c0700 1 -- 172.21.15.78:0/2436151159 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x55abbe2d0a80 con 0x55abbe58c400 2023-10-11T09:45:57.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.756+0000 7f8e5ad12380 4 main: creating IoCtx.. 2023-10-11T09:45:57.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.756+0000 7f8e5ad12380 1 --2- 172.21.15.78:0/2436151159 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55abbe58d000 0x55abbe33e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.756+0000 7f8e594cc700 1 --2- 172.21.15.78:0/2436151159 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55abbe6d4000 0x55abbe33f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.756+0000 7f8e5ad12380 1 -- 172.21.15.78:0/2436151159 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:1 8.d 8:b50e409b:::mds_snaptable:head [writefull 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55abbe58d400 con 0x55abbe58d000 2023-10-11T09:45:57.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.760+0000 7f8e58ccb700 1 --2- 172.21.15.78:0/2436151159 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55abbe58d000 0x55abbe33e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.760+0000 7f8e594cc700 1 --2- 172.21.15.78:0/2436151159 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55abbe6d4000 0x55abbe33f700 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55abbe3f82a0 tx=0x55abbe597c80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:57.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.760+0000 7f8e58ccb700 1 --2- 172.21.15.78:0/2436151159 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55abbe58d000 0x55abbe33e100 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:57.772 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:45:57.773 INFO:teuthology.orchestra.run.smithi078.stdout: "result": 0 2023-10-11T09:45:57.773 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:45:57.773 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:57.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.764+0000 7f8e58ccb700 1 -- 172.21.15.78:0/2436151159 <== osd.5 v2:172.21.15.133:6816/35899 1 ==== osd_op_reply(1 mds_snaptable [writefull 0~70] v66'3 uv3 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x55abbf10afc0 con 0x55abbe58d000 2023-10-11T09:45:57.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.764+0000 7f8e5ad12380 1 -- 172.21.15.78:0/2436151159 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55abbe58d000 msgr2=0x55abbe33e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.764+0000 7f8e5ad12380 1 --2- 172.21.15.78:0/2436151159 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55abbe58d000 0x55abbe33e100 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.764+0000 7f8e5ad12380 1 -- 172.21.15.78:0/2436151159 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55abbe6d4000 msgr2=0x55abbe33f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.764+0000 7f8e5ad12380 1 --2- 172.21.15.78:0/2436151159 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55abbe6d4000 0x55abbe33f700 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55abbe3f82a0 tx=0x55abbe597c80 comp rx=0 tx=0).stop 2023-10-11T09:45:57.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.764+0000 7f8e5ad12380 1 -- 172.21.15.78:0/2436151159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbe58c400 msgr2=0x55abbe33f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.764+0000 7f8e5ad12380 1 --2- 172.21.15.78:0/2436151159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbe58c400 0x55abbe33f180 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x55abbe5e2330 tx=0x55abbf070240 comp rx=0 tx=0).stop 2023-10-11T09:45:57.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.764+0000 7f8e594cc700 1 -- 172.21.15.78:0/2436151159 reap_dead start 2023-10-11T09:45:57.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.764+0000 7f8e5ad12380 1 -- 172.21.15.78:0/2436151159 shutdown_connections 2023-10-11T09:45:57.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.764+0000 7f8e5ad12380 1 -- 172.21.15.78:0/2436151159 >> 172.21.15.78:0/2436151159 conn(0x55abbe296c00 msgr2=0x55abbe383800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:57.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.764+0000 7f8e5ad12380 1 -- 172.21.15.78:0/2436151159 shutdown_connections 2023-10-11T09:45:57.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.764+0000 7f8e5ad12380 1 -- 172.21.15.78:0/2436151159 wait complete. 2023-10-11T09:45:57.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.764+0000 7f8e5ad12380 1 -- 172.21.15.78:0/1031146539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbe297000 msgr2=0x55abbe33d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.764+0000 7f8e5ad12380 1 --2- 172.21.15.78:0/1031146539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbe297000 0x55abbe33d080 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x55abbe3f8c30 tx=0x55abbf070fc0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.764+0000 7f8e5ad12380 1 -- 172.21.15.78:0/1031146539 shutdown_connections 2023-10-11T09:45:57.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.764+0000 7f8e5ad12380 1 --2- 172.21.15.78:0/1031146539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abbe297000 0x55abbe33d080 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.764+0000 7f8e5ad12380 1 --2- 172.21.15.78:0/1031146539 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55abbe297400 0x55abbe33cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.764+0000 7f8e5ad12380 1 --2- 172.21.15.78:0/1031146539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55abbe297800 0x55abbe33c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.764+0000 7f8e5ad12380 1 -- 172.21.15.78:0/1031146539 >> 172.21.15.78:0/1031146539 conn(0x55abbe297c00 msgr2=0x55abbe383000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:57.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.764+0000 7f8e5ad12380 1 -- 172.21.15.78:0/1031146539 shutdown_connections 2023-10-11T09:45:57.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.764+0000 7f8e5ad12380 1 -- 172.21.15.78:0/1031146539 wait complete. 2023-10-11T09:45:57.779 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'snap'] in time 0:00:00.099719, result: { "result": 0 } 2023-10-11T09:45:57.780 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset inode 2023-10-11T09:45:57.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.840+0000 7f10a17b6380 1 -- start start 2023-10-11T09:45:57.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.840+0000 7f10a17b6380 1 Processor -- start 2023-10-11T09:45:57.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.840+0000 7f10a17b6380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5571a8f23800 0x5571a8fc8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.840+0000 7f10a17b6380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571a8f23400 0x5571a8fc8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.840+0000 7f10a17b6380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571a8f23000 0x5571a8fc9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.840+0000 7f10a17b6380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a8fa2a00 con 0x5571a8f23400 2023-10-11T09:45:57.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.840+0000 7f10a17b6380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a8fa2780 con 0x5571a8f23000 2023-10-11T09:45:57.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.840+0000 7f10a17b6380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x5571a8fa2640 con 0x5571a8f23800 2023-10-11T09:45:57.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.844+0000 7f109f76f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571a8f23400 0x5571a8fc8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.844+0000 7f109f76f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571a8f23400 0x5571a8fc8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45950/0 (socket says 172.21.15.78:45950) 2023-10-11T09:45:57.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.844+0000 7f109f76f700 1 -- 172.21.15.78:0/4166448447 learned_addr learned my addr 172.21.15.78:0/4166448447 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:57.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.844+0000 7f109ff70700 1 --2- 172.21.15.78:0/4166448447 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5571a8f23800 0x5571a8fc8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.844+0000 7f10a0771700 1 --2- 172.21.15.78:0/4166448447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571a8f23000 0x5571a8fc9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.844+0000 7f109f76f700 1 -- 172.21.15.78:0/4166448447 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5571a8f23800 msgr2=0x5571a8fc8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:57.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.844+0000 7f109f76f700 1 --2- 172.21.15.78:0/4166448447 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5571a8f23800 0x5571a8fc8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.844+0000 7f109f76f700 1 -- 172.21.15.78:0/4166448447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571a8f23000 msgr2=0x5571a8fc9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:57.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.844+0000 7f109f76f700 1 --2- 172.21.15.78:0/4166448447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571a8f23000 0x5571a8fc9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.844+0000 7f109f76f700 1 -- 172.21.15.78:0/4166448447 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571a8f44b60 con 0x5571a8f23400 2023-10-11T09:45:57.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.844+0000 7f109f76f700 1 --2- 172.21.15.78:0/4166448447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571a8f23400 0x5571a8fc8b00 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x5571a9cfcc90 tx=0x5571a9d0afc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769917b31f6334a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:57.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.844+0000 7f109d76b700 1 -- 172.21.15.78:0/4166448447 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5571a8f53600 con 0x5571a8f23400 2023-10-11T09:45:57.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.844+0000 7f109d76b700 1 -- 172.21.15.78:0/4166448447 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5571a8f6e480 con 0x5571a8f23400 2023-10-11T09:45:57.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.844+0000 7f109d76b700 1 -- 172.21.15.78:0/4166448447 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5571a8f531e0 con 0x5571a8f23400 2023-10-11T09:45:57.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.844+0000 7f10a17b6380 1 -- 172.21.15.78:0/4166448447 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5571a8f44340 con 0x5571a8f23400 2023-10-11T09:45:57.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.844+0000 7f10a17b6380 1 -- 172.21.15.78:0/4166448447 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5571a8f44000 con 0x5571a8f23400 2023-10-11T09:45:57.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.844+0000 7f10a17b6380 4 waiting for MDS map... 2023-10-11T09:45:57.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.844+0000 7f109d76b700 1 -- 172.21.15.78:0/4166448447 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x5571a8f5cc40 con 0x5571a8f23400 2023-10-11T09:45:57.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.844+0000 7f109d76b700 1 -- 172.21.15.78:0/4166448447 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 43) v1 ==== 3118+0+0 (secure 0 0 0) 0x5571a921a000 con 0x5571a8f23400 2023-10-11T09:45:57.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.844+0000 7f10a17b6380 4 Got MDS map 43 2023-10-11T09:45:57.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.844+0000 7f10a17b6380 10 main: main 2023-10-11T09:45:57.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.844+0000 7f10a17b6380 4 main: connecting to RADOS... 2023-10-11T09:45:57.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.844+0000 7f10a17b6380 1 Processor -- start 2023-10-11T09:45:57.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.848+0000 7f10a17b6380 1 -- start start 2023-10-11T09:45:57.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.848+0000 7f10a17b6380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571a9dac400 0x5571a8fc9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.848+0000 7f10a17b6380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5571a9dac800 0x5571a8fc9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.848+0000 7f10a17b6380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571a9dacc00 0x5571a8fca100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.848+0000 7f10a17b6380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a8f59e00 con 0x5571a9dac400 2023-10-11T09:45:57.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.848+0000 7f10a17b6380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a8fa2780 con 0x5571a9dacc00 2023-10-11T09:45:57.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.848+0000 7f10a17b6380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x5571a8fa2500 con 0x5571a9dac800 2023-10-11T09:45:57.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.848+0000 7f10a0771700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571a9dac400 0x5571a8fc9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.848+0000 7f10a0771700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571a9dac400 0x5571a8fc9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45964/0 (socket says 172.21.15.78:45964) 2023-10-11T09:45:57.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.848+0000 7f10a0771700 1 -- 172.21.15.78:0/4016548163 learned_addr learned my addr 172.21.15.78:0/4016548163 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:57.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.848+0000 7f109ff70700 1 --2- 172.21.15.78:0/4016548163 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5571a9dac800 0x5571a8fc9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.848+0000 7f109f76f700 1 --2- 172.21.15.78:0/4016548163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571a9dacc00 0x5571a8fca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.848+0000 7f10a0771700 1 -- 172.21.15.78:0/4016548163 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5571a9dac800 msgr2=0x5571a8fc9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:57.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.848+0000 7f10a0771700 1 --2- 172.21.15.78:0/4016548163 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5571a9dac800 0x5571a8fc9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.848+0000 7f10a0771700 1 -- 172.21.15.78:0/4016548163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571a9dacc00 msgr2=0x5571a8fca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:57.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.848+0000 7f10a0771700 1 --2- 172.21.15.78:0/4016548163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571a9dacc00 0x5571a8fca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.848+0000 7f10a0771700 1 -- 172.21.15.78:0/4016548163 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571a8f44ea0 con 0x5571a9dac400 2023-10-11T09:45:57.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.848+0000 7f10a0771700 1 --2- 172.21.15.78:0/4016548163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571a9dac400 0x5571a8fc9600 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x5571a9270030 tx=0x5571a9d0a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a008f9205a161498 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:57.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.848+0000 7f109b767700 1 -- 172.21.15.78:0/4016548163 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5571a8f53080 con 0x5571a9dac400 2023-10-11T09:45:57.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.848+0000 7f109b767700 1 -- 172.21.15.78:0/4016548163 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5571a9d1d080 con 0x5571a9dac400 2023-10-11T09:45:57.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.848+0000 7f109b767700 1 -- 172.21.15.78:0/4016548163 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5571a8f53600 con 0x5571a9dac400 2023-10-11T09:45:57.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.848+0000 7f10a17b6380 1 -- 172.21.15.78:0/4016548163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571a9dac400 msgr2=0x5571a8fc9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.848+0000 7f10a17b6380 1 --2- 172.21.15.78:0/4016548163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571a9dac400 0x5571a8fc9600 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x5571a9270030 tx=0x5571a9d0a000 comp rx=0 tx=0).stop 2023-10-11T09:45:57.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.852+0000 7f10a17b6380 1 -- 172.21.15.78:0/4016548163 shutdown_connections 2023-10-11T09:45:57.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.852+0000 7f10a17b6380 1 --2- 172.21.15.78:0/4016548163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571a9dacc00 0x5571a8fca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.852+0000 7f10a17b6380 1 --2- 172.21.15.78:0/4016548163 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5571a9dac800 0x5571a8fc9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.852+0000 7f10a17b6380 1 --2- 172.21.15.78:0/4016548163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571a9dac400 0x5571a8fc9600 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.852+0000 7f10a17b6380 1 -- 172.21.15.78:0/4016548163 >> 172.21.15.78:0/4016548163 conn(0x5571a8f22c00 msgr2=0x5571a8fdf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:57.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.852+0000 7f10a17b6380 1 -- 172.21.15.78:0/4016548163 shutdown_connections 2023-10-11T09:45:57.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.852+0000 7f10a17b6380 1 -- 172.21.15.78:0/4016548163 wait complete. 2023-10-11T09:45:57.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.852+0000 7f10a17b6380 1 Processor -- start 2023-10-11T09:45:57.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.852+0000 7f10a17b6380 1 -- start start 2023-10-11T09:45:57.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.852+0000 7f10a17b6380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571a9dac400 0x5571a8fcb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.852+0000 7f10a17b6380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571a9dac800 0x5571a8fcac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.852+0000 7f10a17b6380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5571a9dacc00 0x5571a8fca680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.852+0000 7f10a17b6380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a8fa23c0 con 0x5571a9dac800 2023-10-11T09:45:57.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.852+0000 7f10a17b6380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a8fa2280 con 0x5571a9dac400 2023-10-11T09:45:57.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.852+0000 7f10a17b6380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x5571a8fa2140 con 0x5571a9dacc00 2023-10-11T09:45:57.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.852+0000 7f109ff70700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571a9dac800 0x5571a8fcac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.852+0000 7f109ff70700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571a9dac800 0x5571a8fcac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45974/0 (socket says 172.21.15.78:45974) 2023-10-11T09:45:57.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.852+0000 7f109ff70700 1 -- 172.21.15.78:0/1178255292 learned_addr learned my addr 172.21.15.78:0/1178255292 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:57.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.852+0000 7f109f76f700 1 --2- 172.21.15.78:0/1178255292 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5571a9dacc00 0x5571a8fca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.852+0000 7f10a0771700 1 --2- 172.21.15.78:0/1178255292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571a9dac400 0x5571a8fcb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.852+0000 7f109ff70700 1 -- 172.21.15.78:0/1178255292 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5571a9dacc00 msgr2=0x5571a8fca680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.856+0000 7f109ff70700 1 --2- 172.21.15.78:0/1178255292 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5571a9dacc00 0x5571a8fca680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.856+0000 7f109ff70700 1 -- 172.21.15.78:0/1178255292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571a9dac400 msgr2=0x5571a8fcb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.856+0000 7f109ff70700 1 --2- 172.21.15.78:0/1178255292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571a9dac400 0x5571a8fcb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.856+0000 7f109ff70700 1 -- 172.21.15.78:0/1178255292 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571a8f451e0 con 0x5571a9dac800 2023-10-11T09:45:57.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.856+0000 7f10a0771700 1 --2- 172.21.15.78:0/1178255292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571a9dac400 0x5571a8fcb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:45:57.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.856+0000 7f109f76f700 1 --2- 172.21.15.78:0/1178255292 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5571a9dacc00 0x5571a8fca680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:45:57.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.856+0000 7f109ff70700 1 --2- 172.21.15.78:0/1178255292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571a9dac800 0x5571a8fcac00 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x5571a9270d50 tx=0x5571a9220c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:57.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.856+0000 7f1099f64700 1 -- 172.21.15.78:0/1178255292 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5571a8f53080 con 0x5571a9dac800 2023-10-11T09:45:57.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.856+0000 7f1099f64700 1 -- 172.21.15.78:0/1178255292 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5571a8f6e180 con 0x5571a9dac800 2023-10-11T09:45:57.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.856+0000 7f1099f64700 1 -- 172.21.15.78:0/1178255292 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5571a8f53600 con 0x5571a9dac800 2023-10-11T09:45:57.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.856+0000 7f10a17b6380 1 -- 172.21.15.78:0/1178255292 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5571a8f44ea0 con 0x5571a9dac800 2023-10-11T09:45:57.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.856+0000 7f10a17b6380 1 -- 172.21.15.78:0/1178255292 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5571a8f44b60 con 0x5571a9dac800 2023-10-11T09:45:57.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.856+0000 7f10a17b6380 4 main: resolving pool 8 2023-10-11T09:45:57.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.860+0000 7f1099f64700 1 -- 172.21.15.78:0/1178255292 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x5571a8fa0840 con 0x5571a9dac800 2023-10-11T09:45:57.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.860+0000 7f1099f64700 1 --2- 172.21.15.78:0/1178255292 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5571a9366000 0x5571a8fcb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.860+0000 7f1099f64700 1 -- 172.21.15.78:0/1178255292 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x5571a8f5ca80 con 0x5571a9dac800 2023-10-11T09:45:57.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.860+0000 7f10a17b6380 4 main: creating IoCtx.. 2023-10-11T09:45:57.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.860+0000 7f10a17b6380 1 --2- 172.21.15.78:0/1178255292 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5571a9dad000 0x5571a8fca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.860+0000 7f10a0771700 1 --2- 172.21.15.78:0/1178255292 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5571a9366000 0x5571a8fcb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.860+0000 7f10a17b6380 1 -- 172.21.15.78:0/1178255292 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:1 8.13 8:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5571a9dad400 con 0x5571a9dad000 2023-10-11T09:45:57.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.860+0000 7f109f76f700 1 --2- 172.21.15.78:0/1178255292 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5571a9dad000 0x5571a8fca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.860+0000 7f10a0771700 1 --2- 172.21.15.78:0/1178255292 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5571a9366000 0x5571a8fcb700 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x5571a9270030 tx=0x5571a9d0a0c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:57.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.860+0000 7f109f76f700 1 --2- 172.21.15.78:0/1178255292 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5571a9dad000 0x5571a8fca100 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:57.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.864+0000 7f109f76f700 1 -- 172.21.15.78:0/1178255292 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v66'6 uv6 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x5571a9d1afc0 con 0x5571a9dad000 2023-10-11T09:45:57.875 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:45:57.875 INFO:teuthology.orchestra.run.smithi078.stdout: "0": { 2023-10-11T09:45:57.875 INFO:teuthology.orchestra.run.smithi078.stdout: "data": {}, 2023-10-11T09:45:57.876 INFO:teuthology.orchestra.run.smithi078.stdout: "result": 0 2023-10-11T09:45:57.876 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:45:57.876 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:45:57.876 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:45:57.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.864+0000 7f10a17b6380 1 -- 172.21.15.78:0/1178255292 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5571a9dad000 msgr2=0x5571a8fca100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.864+0000 7f10a17b6380 1 --2- 172.21.15.78:0/1178255292 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5571a9dad000 0x5571a8fca100 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.864+0000 7f10a17b6380 1 -- 172.21.15.78:0/1178255292 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5571a9366000 msgr2=0x5571a8fcb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.864+0000 7f10a17b6380 1 --2- 172.21.15.78:0/1178255292 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5571a9366000 0x5571a8fcb700 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x5571a9270030 tx=0x5571a9d0a0c0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.864+0000 7f10a17b6380 1 -- 172.21.15.78:0/1178255292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571a9dac800 msgr2=0x5571a8fcac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.864+0000 7f10a17b6380 1 --2- 172.21.15.78:0/1178255292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571a9dac800 0x5571a8fcac00 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x5571a9270d50 tx=0x5571a9220c00 comp rx=0 tx=0).stop 2023-10-11T09:45:57.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.864+0000 7f109ff70700 1 -- 172.21.15.78:0/1178255292 reap_dead start 2023-10-11T09:45:57.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.864+0000 7f10a17b6380 1 -- 172.21.15.78:0/1178255292 shutdown_connections 2023-10-11T09:45:57.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.864+0000 7f10a17b6380 1 -- 172.21.15.78:0/1178255292 >> 172.21.15.78:0/1178255292 conn(0x5571a8f22c00 msgr2=0x5571a8fdf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:57.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.864+0000 7f10a17b6380 1 -- 172.21.15.78:0/1178255292 shutdown_connections 2023-10-11T09:45:57.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.864+0000 7f10a17b6380 1 -- 172.21.15.78:0/1178255292 wait complete. 2023-10-11T09:45:57.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.864+0000 7f10a17b6380 1 -- 172.21.15.78:0/4166448447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571a8f23400 msgr2=0x5571a8fc8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.864+0000 7f10a17b6380 1 --2- 172.21.15.78:0/4166448447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571a8f23400 0x5571a8fc8b00 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x5571a9cfcc90 tx=0x5571a9d0afc0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.864+0000 7f10a17b6380 1 -- 172.21.15.78:0/4166448447 shutdown_connections 2023-10-11T09:45:57.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.864+0000 7f10a17b6380 1 --2- 172.21.15.78:0/4166448447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571a8f23000 0x5571a8fc9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.864+0000 7f10a17b6380 1 --2- 172.21.15.78:0/4166448447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571a8f23400 0x5571a8fc8b00 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.864+0000 7f10a17b6380 1 --2- 172.21.15.78:0/4166448447 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5571a8f23800 0x5571a8fc8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.864+0000 7f10a17b6380 1 -- 172.21.15.78:0/4166448447 >> 172.21.15.78:0/4166448447 conn(0x5571a8f23c00 msgr2=0x5571a8fdf000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:57.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.864+0000 7f10a17b6380 1 -- 172.21.15.78:0/4166448447 shutdown_connections 2023-10-11T09:45:57.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.864+0000 7f10a17b6380 1 -- 172.21.15.78:0/4166448447 wait complete. 2023-10-11T09:45:57.882 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'inode'] in time 0:00:00.102416, result: { "0": { "data": {}, "result": 0 } } 2023-10-11T09:45:57.882 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2023-10-11T09:45:57.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.944+0000 7fd322bef380 1 -- start start 2023-10-11T09:45:57.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.944+0000 7fd322bef380 1 Processor -- start 2023-10-11T09:45:57.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.944+0000 7fd322bef380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b6bda1800 0x560b6be46580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.944+0000 7fd322bef380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b6bda1400 0x560b6be46b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.944+0000 7fd322bef380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b6bda1000 0x560b6be47080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.944+0000 7fd322bef380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b6be20a00 con 0x560b6bda1800 2023-10-11T09:45:57.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.944+0000 7fd322bef380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b6be20780 con 0x560b6bda1000 2023-10-11T09:45:57.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.944+0000 7fd322bef380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b6be20640 con 0x560b6bda1400 2023-10-11T09:45:57.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.944+0000 7fd3213a9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b6bda1800 0x560b6be46580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.944+0000 7fd3213a9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b6bda1800 0x560b6be46580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45976/0 (socket says 172.21.15.78:45976) 2023-10-11T09:45:57.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.944+0000 7fd3213a9700 1 -- 172.21.15.78:0/2700103232 learned_addr learned my addr 172.21.15.78:0/2700103232 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:57.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.944+0000 7fd321baa700 1 --2- 172.21.15.78:0/2700103232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b6bda1000 0x560b6be47080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.944+0000 7fd320ba8700 1 --2- 172.21.15.78:0/2700103232 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b6bda1400 0x560b6be46b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.944+0000 7fd3213a9700 1 -- 172.21.15.78:0/2700103232 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b6bda1400 msgr2=0x560b6be46b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:57.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.944+0000 7fd3213a9700 1 --2- 172.21.15.78:0/2700103232 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b6bda1400 0x560b6be46b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.944+0000 7fd3213a9700 1 -- 172.21.15.78:0/2700103232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b6bda1000 msgr2=0x560b6be47080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:57.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.944+0000 7fd3213a9700 1 --2- 172.21.15.78:0/2700103232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b6bda1000 0x560b6be47080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.944+0000 7fd3213a9700 1 -- 172.21.15.78:0/2700103232 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b6bdc2b60 con 0x560b6bda1800 2023-10-11T09:45:57.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.944+0000 7fd3213a9700 1 --2- 172.21.15.78:0/2700103232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b6bda1800 0x560b6be46580 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x560b6cb7cd50 tx=0x560b6be774a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bfe1fce21bd4227 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:57.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.944+0000 7fd31eba4700 1 -- 172.21.15.78:0/2700103232 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560b6bdd1080 con 0x560b6bda1800 2023-10-11T09:45:57.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.944+0000 7fd31eba4700 1 -- 172.21.15.78:0/2700103232 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560b6cb9a300 con 0x560b6bda1800 2023-10-11T09:45:57.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.944+0000 7fd31eba4700 1 -- 172.21.15.78:0/2700103232 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560b6bdd1600 con 0x560b6bda1800 2023-10-11T09:45:57.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.944+0000 7fd322bef380 1 -- 172.21.15.78:0/2700103232 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b6bdc31e0 con 0x560b6bda1800 2023-10-11T09:45:57.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.944+0000 7fd322bef380 1 -- 172.21.15.78:0/2700103232 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x560b6bdc2b60 con 0x560b6bda1800 2023-10-11T09:45:57.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.944+0000 7fd322bef380 4 waiting for MDS map... 2023-10-11T09:45:57.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.944+0000 7fd31eba4700 1 -- 172.21.15.78:0/2700103232 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x560b6bddac40 con 0x560b6bda1800 2023-10-11T09:45:57.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.944+0000 7fd31eba4700 1 -- 172.21.15.78:0/2700103232 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 43) v1 ==== 3118+0+0 (secure 0 0 0) 0x560b6bde2f00 con 0x560b6bda1800 2023-10-11T09:45:57.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.944+0000 7fd322bef380 4 Got MDS map 43 2023-10-11T09:45:57.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.944+0000 7fd322bef380 10 main: JournalTool::main 2023-10-11T09:45:57.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.948+0000 7fd322bef380 4 main: JournalTool: connecting to RADOS... 2023-10-11T09:45:57.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.948+0000 7fd322bef380 1 Processor -- start 2023-10-11T09:45:57.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.948+0000 7fd322bef380 1 -- start start 2023-10-11T09:45:57.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.948+0000 7fd322bef380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b6cc2a400 0x560b6be47600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.948+0000 7fd322bef380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b6cc2a800 0x560b6be47b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.948+0000 7fd322bef380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b6cc2ac00 0x560b6be48100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.948+0000 7fd322bef380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b6bdd7e00 con 0x560b6cc2ac00 2023-10-11T09:45:57.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.948+0000 7fd322bef380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b6be20780 con 0x560b6cc2a400 2023-10-11T09:45:57.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.948+0000 7fd322bef380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b6be20a00 con 0x560b6cc2a800 2023-10-11T09:45:57.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.948+0000 7fd320ba8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b6cc2ac00 0x560b6be48100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.948+0000 7fd320ba8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b6cc2ac00 0x560b6be48100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45992/0 (socket says 172.21.15.78:45992) 2023-10-11T09:45:57.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.948+0000 7fd320ba8700 1 -- 172.21.15.78:0/2665307936 learned_addr learned my addr 172.21.15.78:0/2665307936 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:57.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.948+0000 7fd3213a9700 1 --2- 172.21.15.78:0/2665307936 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b6cc2a800 0x560b6be47b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.948+0000 7fd321baa700 1 --2- 172.21.15.78:0/2665307936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b6cc2a400 0x560b6be47600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.948+0000 7fd320ba8700 1 -- 172.21.15.78:0/2665307936 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b6cc2a800 msgr2=0x560b6be47b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:57.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.948+0000 7fd320ba8700 1 --2- 172.21.15.78:0/2665307936 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b6cc2a800 0x560b6be47b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.948+0000 7fd320ba8700 1 -- 172.21.15.78:0/2665307936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b6cc2a400 msgr2=0x560b6be47600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:57.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.948+0000 7fd320ba8700 1 --2- 172.21.15.78:0/2665307936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b6cc2a400 0x560b6be47600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.948+0000 7fd320ba8700 1 -- 172.21.15.78:0/2665307936 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b6bdc2340 con 0x560b6cc2ac00 2023-10-11T09:45:57.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.948+0000 7fd3213a9700 1 --2- 172.21.15.78:0/2665307936 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b6cc2a800 0x560b6be47b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:45:57.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.948+0000 7fd320ba8700 1 --2- 172.21.15.78:0/2665307936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b6cc2ac00 0x560b6be48100 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x560b6cb7a2a0 tx=0x560b6cb98de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42fc231462d6ab16 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:57.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.948+0000 7fd31cba0700 1 -- 172.21.15.78:0/2665307936 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560b6bdd1080 con 0x560b6cc2ac00 2023-10-11T09:45:57.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.952+0000 7fd31cba0700 1 -- 172.21.15.78:0/2665307936 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560b6bdec480 con 0x560b6cc2ac00 2023-10-11T09:45:57.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.952+0000 7fd31cba0700 1 -- 172.21.15.78:0/2665307936 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560b6bdd1600 con 0x560b6cc2ac00 2023-10-11T09:45:57.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.952+0000 7fd322bef380 1 -- 172.21.15.78:0/2665307936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b6cc2ac00 msgr2=0x560b6be48100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.952+0000 7fd322bef380 1 --2- 172.21.15.78:0/2665307936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b6cc2ac00 0x560b6be48100 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x560b6cb7a2a0 tx=0x560b6cb98de0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.952+0000 7fd322bef380 1 -- 172.21.15.78:0/2665307936 shutdown_connections 2023-10-11T09:45:57.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.952+0000 7fd322bef380 1 --2- 172.21.15.78:0/2665307936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b6cc2ac00 0x560b6be48100 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.952+0000 7fd322bef380 1 --2- 172.21.15.78:0/2665307936 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b6cc2a800 0x560b6be47b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.952+0000 7fd322bef380 1 --2- 172.21.15.78:0/2665307936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b6cc2a400 0x560b6be47600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.952+0000 7fd322bef380 1 -- 172.21.15.78:0/2665307936 >> 172.21.15.78:0/2665307936 conn(0x560b6bda0c00 msgr2=0x560b6be8d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:57.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.952+0000 7fd322bef380 1 -- 172.21.15.78:0/2665307936 shutdown_connections 2023-10-11T09:45:57.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.952+0000 7fd322bef380 1 -- 172.21.15.78:0/2665307936 wait complete. 2023-10-11T09:45:57.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.952+0000 7fd322bef380 1 Processor -- start 2023-10-11T09:45:57.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.952+0000 7fd322bef380 1 -- start start 2023-10-11T09:45:57.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.952+0000 7fd322bef380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b6cc2a400 0x560b6be49180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.952+0000 7fd322bef380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b6cc2a800 0x560b6be48c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.952+0000 7fd322bef380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b6cc2ac00 0x560b6be48680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.952+0000 7fd322bef380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b6be20500 con 0x560b6cc2a400 2023-10-11T09:45:57.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.952+0000 7fd322bef380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b6be203c0 con 0x560b6cc2a800 2023-10-11T09:45:57.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.952+0000 7fd322bef380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b6be20280 con 0x560b6cc2ac00 2023-10-11T09:45:57.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.952+0000 7fd321baa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b6cc2a400 0x560b6be49180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.952+0000 7fd321baa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b6cc2a400 0x560b6be49180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46004/0 (socket says 172.21.15.78:46004) 2023-10-11T09:45:57.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.952+0000 7fd321baa700 1 -- 172.21.15.78:0/1984150694 learned_addr learned my addr 172.21.15.78:0/1984150694 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:57.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.952+0000 7fd3213a9700 1 --2- 172.21.15.78:0/1984150694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b6cc2a800 0x560b6be48c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.952+0000 7fd320ba8700 1 --2- 172.21.15.78:0/1984150694 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b6cc2ac00 0x560b6be48680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.956+0000 7fd321baa700 1 -- 172.21.15.78:0/1984150694 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b6cc2ac00 msgr2=0x560b6be48680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.956+0000 7fd321baa700 1 --2- 172.21.15.78:0/1984150694 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b6cc2ac00 0x560b6be48680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.956+0000 7fd321baa700 1 -- 172.21.15.78:0/1984150694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b6cc2a800 msgr2=0x560b6be48c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.956+0000 7fd321baa700 1 --2- 172.21.15.78:0/1984150694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b6cc2a800 0x560b6be48c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.956+0000 7fd321baa700 1 -- 172.21.15.78:0/1984150694 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b6bdc2000 con 0x560b6cc2a400 2023-10-11T09:45:57.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.956+0000 7fd320ba8700 1 --2- 172.21.15.78:0/1984150694 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b6cc2ac00 0x560b6be48680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:45:57.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.956+0000 7fd321baa700 1 --2- 172.21.15.78:0/1984150694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b6cc2a400 0x560b6be49180 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x560b6cb7b020 tx=0x560b6c098e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:57.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.956+0000 7fd31b39d700 1 -- 172.21.15.78:0/1984150694 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560b6bdd1080 con 0x560b6cc2a400 2023-10-11T09:45:57.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.956+0000 7fd31b39d700 1 -- 172.21.15.78:0/1984150694 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560b6cb9a780 con 0x560b6cc2a400 2023-10-11T09:45:57.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.956+0000 7fd322bef380 1 -- 172.21.15.78:0/1984150694 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560b6bdc2ea0 con 0x560b6cc2a400 2023-10-11T09:45:57.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.956+0000 7fd31b39d700 1 -- 172.21.15.78:0/1984150694 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560b6bdd1600 con 0x560b6cc2a400 2023-10-11T09:45:57.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.956+0000 7fd322bef380 1 -- 172.21.15.78:0/1984150694 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b6bdc3520 con 0x560b6cc2a400 2023-10-11T09:45:57.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.956+0000 7fd322bef380 4 main: JournalTool: resolving pool 8 2023-10-11T09:45:57.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.956+0000 7fd31b39d700 1 -- 172.21.15.78:0/1984150694 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x560b6be1e840 con 0x560b6cc2a400 2023-10-11T09:45:57.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.960+0000 7fd31b39d700 1 --2- 172.21.15.78:0/1984150694 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x560b6c1e2000 0x560b6be49700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.960+0000 7fd31b39d700 1 -- 172.21.15.78:0/1984150694 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x560b6bddaa80 con 0x560b6cc2a400 2023-10-11T09:45:57.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.960+0000 7fd322bef380 4 main: JournalTool: creating IoCtx.. 2023-10-11T09:45:57.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.960+0000 7fd322bef380 4 main: Executing for rank 0 2023-10-11T09:45:57.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.960+0000 7fd3213a9700 1 --2- 172.21.15.78:0/1984150694 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x560b6c1e2000 0x560b6be49700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.960+0000 7fd3213a9700 1 --2- 172.21.15.78:0/1984150694 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x560b6c1e2000 0x560b6be49700 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x560b6cb7d0b0 tx=0x560b6be77500 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:57.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.960+0000 7fd322bef380 1 -- start start 2023-10-11T09:45:57.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.960+0000 7fd322bef380 1 Processor -- start 2023-10-11T09:45:57.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.960+0000 7fd322bef380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b6cc2b400 0x560b6be48100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.960+0000 7fd322bef380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b6cc2b800 0x560b6be47b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.960+0000 7fd322bef380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b6c1e2400 0x560b6be47600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.960+0000 7fd322bef380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b6be203c0 con 0x560b6cc2b800 2023-10-11T09:45:57.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.960+0000 7fd322bef380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b6be20a00 con 0x560b6c1e2400 2023-10-11T09:45:57.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.960+0000 7fd322bef380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b6be20780 con 0x560b6cc2b400 2023-10-11T09:45:57.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.964+0000 7fd3213a9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b6cc2b800 0x560b6be47b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.964+0000 7fd3213a9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b6cc2b800 0x560b6be47b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46018/0 (socket says 172.21.15.78:46018) 2023-10-11T09:45:57.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.964+0000 7fd3213a9700 1 -- 172.21.15.78:0/2483678258 learned_addr learned my addr 172.21.15.78:0/2483678258 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:57.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.964+0000 7fd321baa700 1 --2- 172.21.15.78:0/2483678258 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b6cc2b400 0x560b6be48100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.964+0000 7fd320ba8700 1 --2- 172.21.15.78:0/2483678258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b6c1e2400 0x560b6be47600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.964+0000 7fd3213a9700 1 -- 172.21.15.78:0/2483678258 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b6cc2b400 msgr2=0x560b6be48100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:57.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.964+0000 7fd3213a9700 1 --2- 172.21.15.78:0/2483678258 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b6cc2b400 0x560b6be48100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.964+0000 7fd3213a9700 1 -- 172.21.15.78:0/2483678258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b6c1e2400 msgr2=0x560b6be47600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:57.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.964+0000 7fd3213a9700 1 --2- 172.21.15.78:0/2483678258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b6c1e2400 0x560b6be47600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.964+0000 7fd3213a9700 1 -- 172.21.15.78:0/2483678258 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b6c131040 con 0x560b6cc2b800 2023-10-11T09:45:57.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.964+0000 7fd3213a9700 1 --2- 172.21.15.78:0/2483678258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b6cc2b800 0x560b6be47b80 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x560b6c14e7e0 tx=0x560b6c1f1920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a7ebfe9a4e1e04 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:57.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.964+0000 7fd318397700 1 -- 172.21.15.78:0/2483678258 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560b6bdd11e0 con 0x560b6cc2b800 2023-10-11T09:45:57.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.964+0000 7fd318397700 1 -- 172.21.15.78:0/2483678258 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560b6cb9ac00 con 0x560b6cc2b800 2023-10-11T09:45:57.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.964+0000 7fd318397700 1 -- 172.21.15.78:0/2483678258 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560b6bdd0f20 con 0x560b6cc2b800 2023-10-11T09:45:57.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.964+0000 7fd322bef380 1 -- 172.21.15.78:0/2483678258 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b6c130ea0 con 0x560b6cc2b800 2023-10-11T09:45:57.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.964+0000 7fd322bef380 1 -- 172.21.15.78:0/2483678258 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x560b6c130d00 con 0x560b6cc2b800 2023-10-11T09:45:57.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.964+0000 7fd322bef380 4 waiting for MDS map... 2023-10-11T09:45:57.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.964+0000 7fd318397700 1 -- 172.21.15.78:0/2483678258 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x560b6bdda8c0 con 0x560b6cc2b800 2023-10-11T09:45:57.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.964+0000 7fd318397700 1 -- 172.21.15.78:0/2483678258 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 43) v1 ==== 3118+0+0 (secure 0 0 0) 0x560b6c237200 con 0x560b6cc2b800 2023-10-11T09:45:57.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.964+0000 7fd322bef380 4 Got MDS map 43 2023-10-11T09:45:57.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.964+0000 7fd322bef380 1 --2- 172.21.15.78:0/2483678258 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x560b6c1e2800 0x560b6c26c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.964+0000 7fd322bef380 1 -- 172.21.15.78:0/2483678258 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:1 8.f 8:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b6c1e2c00 con 0x560b6c1e2800 2023-10-11T09:45:57.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.964+0000 7fd321baa700 1 --2- 172.21.15.78:0/2483678258 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x560b6c1e2800 0x560b6c26c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.964+0000 7fd321baa700 1 --2- 172.21.15.78:0/2483678258 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x560b6c1e2800 0x560b6c26c000 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=f8af360a0b508e1d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:57.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.968+0000 7fd321baa700 1 -- 172.21.15.78:0/2483678258 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v66'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x560b6cb97200 con 0x560b6c1e2800 2023-10-11T09:45:57.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.968+0000 7fd322bef380 1 --2- 172.21.15.78:0/2483678258 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x560b6c1e3000 0x560b6c26c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.968+0000 7fd322bef380 1 -- 172.21.15.78:0/2483678258 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:2 8.14 8:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b6c1e3400 con 0x560b6c1e3000 2023-10-11T09:45:57.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.968+0000 7fd320ba8700 1 --2- 172.21.15.78:0/2483678258 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x560b6c1e3000 0x560b6c26c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.972+0000 7fd320ba8700 1 --2- 172.21.15.78:0/2483678258 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x560b6c1e3000 0x560b6c26c580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=478dd9b6456a609a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:57.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.972+0000 7fd320ba8700 1 -- 172.21.15.78:0/2483678258 <== osd.4 v2:172.21.15.133:6808/35898 1 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v66'94 uv94 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x560b6cb97440 con 0x560b6c1e3000 2023-10-11T09:45:57.986 INFO:teuthology.orchestra.run.smithi078.stdout:writing EResetJournal entry 2023-10-11T09:45:57.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.972+0000 7fd322bef380 1 -- 172.21.15.78:0/2483678258 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:3 8.14 8:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b6c1e3800 con 0x560b6c1e3000 2023-10-11T09:45:57.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.972+0000 7fd322bef380 1 -- 172.21.15.78:0/2483678258 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:4 8.1b 8:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b6c1e3c00 con 0x560b6c1e3000 2023-10-11T09:45:57.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.972+0000 7fd322bef380 1 -- 172.21.15.78:0/2483678258 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:5 8.b 8:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b6c254800 con 0x560b6c1e2800 2023-10-11T09:45:57.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.972+0000 7fd322bef380 1 --2- 172.21.15.78:0/2483678258 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x560b6c254c00 0x560b6c26cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:57.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.972+0000 7fd322bef380 1 -- 172.21.15.78:0/2483678258 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:6 8.1f 8:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b6c255000 con 0x560b6c254c00 2023-10-11T09:45:57.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.972+0000 7fd322bef380 1 -- 172.21.15.78:0/2483678258 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:7 8.1a 8:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b6c255400 con 0x560b6c254c00 2023-10-11T09:45:57.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.972+0000 7fd322bef380 1 -- 172.21.15.78:0/2483678258 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:8 8.1e 8:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b6c255800 con 0x560b6c1e2800 2023-10-11T09:45:57.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.976+0000 7fd321baa700 1 --2- 172.21.15.78:0/2483678258 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x560b6c254c00 0x560b6c26cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:57.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.976+0000 7fd321baa700 1 --2- 172.21.15.78:0/2483678258 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x560b6c254c00 0x560b6c26cb00 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=75d3fffd8b4878e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:57.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.976+0000 7fd320ba8700 1 -- 172.21.15.78:0/2483678258 <== osd.4 v2:172.21.15.133:6808/35898 2 ==== osd_op_reply(3 200.00000001 [delete] v66'95 uv94 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x560b6cb97440 con 0x560b6c1e3000 2023-10-11T09:45:57.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.976+0000 7fd321baa700 1 -- 172.21.15.78:0/2483678258 <== osd.3 v2:172.21.15.78:6824/35738 2 ==== osd_op_reply(8 200.00000006 [delete] v66'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x560b6cb97200 con 0x560b6c1e2800 2023-10-11T09:45:57.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.976+0000 7fd320ba8700 1 -- 172.21.15.78:0/2483678258 <== osd.4 v2:172.21.15.133:6808/35898 3 ==== osd_op_reply(4 200.00000002 [delete] v66'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x560b6cb97440 con 0x560b6c1e3000 2023-10-11T09:45:57.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.976+0000 7fd316b94700 1 -- 172.21.15.78:0/2483678258 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:9 8.14 8:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b6c1e2c00 con 0x560b6c1e3000 2023-10-11T09:45:57.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.980+0000 7fd321baa700 1 -- 172.21.15.78:0/2483678258 <== osd.3 v2:172.21.15.78:6824/35738 3 ==== osd_op_reply(5 200.00000003 [delete] v66'6 uv5 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x560b6cb97200 con 0x560b6c1e2800 2023-10-11T09:45:57.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.980+0000 7fd321baa700 1 -- 172.21.15.78:0/2483678258 <== osd.0 v2:172.21.15.78:6800/35735 1 ==== osd_op_reply(6 200.00000004 [delete] v66'9 uv8 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x560b6cb97200 con 0x560b6c254c00 2023-10-11T09:45:57.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.980+0000 7fd321baa700 1 -- 172.21.15.78:0/2483678258 <== osd.0 v2:172.21.15.78:6800/35735 2 ==== osd_op_reply(7 200.00000005 [delete] v66'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x560b6cb97200 con 0x560b6c254c00 2023-10-11T09:45:57.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.980+0000 7fd320ba8700 1 -- 172.21.15.78:0/2483678258 <== osd.4 v2:172.21.15.133:6808/35898 4 ==== osd_op_reply(9 200.00000001 [write 0~48] v66'96 uv96 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x560b6cb97440 con 0x560b6c1e3000 2023-10-11T09:45:57.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.980+0000 7fd322bef380 4 Successfully wrote new journal pointer and header for rank 3:0 2023-10-11T09:45:57.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.980+0000 7fd322bef380 1 -- 172.21.15.78:0/2483678258 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x560b6c254c00 msgr2=0x560b6c26cb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.980+0000 7fd322bef380 1 --2- 172.21.15.78:0/2483678258 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x560b6c254c00 0x560b6c26cb00 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.980+0000 7fd322bef380 1 -- 172.21.15.78:0/2483678258 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x560b6c1e2800 msgr2=0x560b6c26c000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.980+0000 7fd322bef380 1 --2- 172.21.15.78:0/2483678258 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x560b6c1e2800 0x560b6c26c000 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.980+0000 7fd322bef380 1 -- 172.21.15.78:0/2483678258 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x560b6c1e3000 msgr2=0x560b6c26c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.980+0000 7fd322bef380 1 --2- 172.21.15.78:0/2483678258 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x560b6c1e3000 0x560b6c26c580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.980+0000 7fd320ba8700 1 -- 172.21.15.78:0/2483678258 reap_dead start 2023-10-11T09:45:57.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.980+0000 7fd322bef380 1 -- 172.21.15.78:0/2483678258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b6cc2b800 msgr2=0x560b6be47b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.980+0000 7fd322bef380 1 --2- 172.21.15.78:0/2483678258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b6cc2b800 0x560b6be47b80 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x560b6c14e7e0 tx=0x560b6c1f1920 comp rx=0 tx=0).stop 2023-10-11T09:45:57.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.980+0000 7fd322bef380 1 -- 172.21.15.78:0/2483678258 shutdown_connections 2023-10-11T09:45:57.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.980+0000 7fd322bef380 1 --2- 172.21.15.78:0/2483678258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b6cc2b800 0x560b6be47b80 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.980+0000 7fd322bef380 1 -- 172.21.15.78:0/2483678258 >> 172.21.15.78:0/2483678258 conn(0x560b6cc2b000 msgr2=0x560b6be8e000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:57.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.980+0000 7fd322bef380 1 -- 172.21.15.78:0/2483678258 shutdown_connections 2023-10-11T09:45:57.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.980+0000 7fd322bef380 1 -- 172.21.15.78:0/2483678258 wait complete. 2023-10-11T09:45:57.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.980+0000 7fd322bef380 1 -- 172.21.15.78:0/1984150694 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x560b6c1e2000 msgr2=0x560b6be49700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.980+0000 7fd322bef380 1 --2- 172.21.15.78:0/1984150694 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x560b6c1e2000 0x560b6be49700 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x560b6cb7d0b0 tx=0x560b6be77500 comp rx=0 tx=0).stop 2023-10-11T09:45:57.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.980+0000 7fd322bef380 1 -- 172.21.15.78:0/1984150694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b6cc2a400 msgr2=0x560b6be49180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.980+0000 7fd322bef380 1 --2- 172.21.15.78:0/1984150694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b6cc2a400 0x560b6be49180 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x560b6cb7b020 tx=0x560b6c098e40 comp rx=0 tx=0).stop 2023-10-11T09:45:57.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.980+0000 7fd322bef380 1 -- 172.21.15.78:0/1984150694 shutdown_connections 2023-10-11T09:45:57.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.980+0000 7fd322bef380 1 --2- 172.21.15.78:0/1984150694 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x560b6c1e2000 0x560b6be49700 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.980+0000 7fd322bef380 1 --2- 172.21.15.78:0/1984150694 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b6cc2ac00 0x560b6be48680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.980+0000 7fd322bef380 1 --2- 172.21.15.78:0/1984150694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b6cc2a800 0x560b6be48c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.980+0000 7fd322bef380 1 --2- 172.21.15.78:0/1984150694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b6cc2a400 0x560b6be49180 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.980+0000 7fd322bef380 1 -- 172.21.15.78:0/1984150694 >> 172.21.15.78:0/1984150694 conn(0x560b6bda0c00 msgr2=0x560b6be8d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:57.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.980+0000 7fd322bef380 1 -- 172.21.15.78:0/1984150694 shutdown_connections 2023-10-11T09:45:57.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.980+0000 7fd322bef380 1 -- 172.21.15.78:0/1984150694 wait complete. 2023-10-11T09:45:57.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.984+0000 7fd322bef380 1 -- 172.21.15.78:0/2700103232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b6bda1800 msgr2=0x560b6be46580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:57.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.984+0000 7fd322bef380 1 --2- 172.21.15.78:0/2700103232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b6bda1800 0x560b6be46580 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x560b6cb7cd50 tx=0x560b6be774a0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.984+0000 7fd322bef380 1 -- 172.21.15.78:0/2700103232 shutdown_connections 2023-10-11T09:45:57.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.984+0000 7fd322bef380 1 --2- 172.21.15.78:0/2700103232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b6bda1000 0x560b6be47080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.984+0000 7fd322bef380 1 --2- 172.21.15.78:0/2700103232 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b6bda1400 0x560b6be46b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.984+0000 7fd322bef380 1 --2- 172.21.15.78:0/2700103232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b6bda1800 0x560b6be46580 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:57.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.984+0000 7fd322bef380 1 -- 172.21.15.78:0/2700103232 >> 172.21.15.78:0/2700103232 conn(0x560b6bda1c00 msgr2=0x560b6be8d000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:57.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.984+0000 7fd322bef380 1 -- 172.21.15.78:0/2700103232 shutdown_connections 2023-10-11T09:45:58.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:57.984+0000 7fd322bef380 1 -- 172.21.15.78:0/2700103232 wait complete. 2023-10-11T09:45:58.000 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.118039, result: writing EResetJournal entry 2023-10-11T09:45:58.001 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 init 2023-10-11T09:45:58.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.060+0000 7f3bc9109840 1 -- start start 2023-10-11T09:45:58.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.060+0000 7f3bc9109840 1 Processor -- start 2023-10-11T09:45:58.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.060+0000 7f3bc9109840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626b0f97800 0x5626b103c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.060+0000 7f3bc9109840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626b0f97400 0x5626b103cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.060+0000 7f3bc9109840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5626b0f97000 0x5626b103d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.060+0000 7f3bc9109840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626b1016a00 con 0x5626b0f97400 2023-10-11T09:45:58.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.060+0000 7f3bc9109840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626b1016780 con 0x5626b0f97800 2023-10-11T09:45:58.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.060+0000 7f3bc9109840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626b1016640 con 0x5626b0f97000 2023-10-11T09:45:58.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.060+0000 7f3bc70c4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626b0f97400 0x5626b103cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.060+0000 7f3bc70c4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626b0f97400 0x5626b103cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46046/0 (socket says 172.21.15.78:46046) 2023-10-11T09:45:58.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.060+0000 7f3bc70c4700 1 -- 172.21.15.78:0/3603487859 learned_addr learned my addr 172.21.15.78:0/3603487859 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.060+0000 7f3bc80c6700 1 --2- 172.21.15.78:0/3603487859 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5626b0f97000 0x5626b103d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.060+0000 7f3bc78c5700 1 --2- 172.21.15.78:0/3603487859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626b0f97800 0x5626b103c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.060+0000 7f3bc70c4700 1 -- 172.21.15.78:0/3603487859 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5626b0f97000 msgr2=0x5626b103d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.060+0000 7f3bc70c4700 1 --2- 172.21.15.78:0/3603487859 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5626b0f97000 0x5626b103d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.060+0000 7f3bc70c4700 1 -- 172.21.15.78:0/3603487859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626b0f97800 msgr2=0x5626b103c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.060+0000 7f3bc70c4700 1 --2- 172.21.15.78:0/3603487859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626b0f97800 0x5626b103c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.060+0000 7f3bc70c4700 1 -- 172.21.15.78:0/3603487859 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626b0fb8b60 con 0x5626b0f97400 2023-10-11T09:45:58.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.060+0000 7f3bc70c4700 1 --2- 172.21.15.78:0/3603487859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626b0f97400 0x5626b103cb00 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x5626b10f3350 tx=0x5626b106d3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55883139ea904719 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.060+0000 7f3bc50c0700 1 -- 172.21.15.78:0/3603487859 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5626b0fc7080 con 0x5626b0f97400 2023-10-11T09:45:58.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.060+0000 7f3bc50c0700 1 -- 172.21.15.78:0/3603487859 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5626b0fe2480 con 0x5626b0f97400 2023-10-11T09:45:58.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.060+0000 7f3bc50c0700 1 -- 172.21.15.78:0/3603487859 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5626b0fc7600 con 0x5626b0f97400 2023-10-11T09:45:58.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.060+0000 7f3bc9109840 1 -- 172.21.15.78:0/3603487859 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626b0fb91e0 con 0x5626b0f97400 2023-10-11T09:45:58.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.060+0000 7f3bc9109840 1 -- 172.21.15.78:0/3603487859 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5626b0fb8b60 con 0x5626b0f97400 2023-10-11T09:45:58.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.060+0000 7f3bc9109840 4 waiting for MDS map... 2023-10-11T09:45:58.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.064+0000 7f3bc50c0700 1 -- 172.21.15.78:0/3603487859 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x5626b0fd0c40 con 0x5626b0f97400 2023-10-11T09:45:58.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.064+0000 7f3bc50c0700 1 -- 172.21.15.78:0/3603487859 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 43) v1 ==== 3118+0+0 (secure 0 0 0) 0x5626b0fd8f00 con 0x5626b0f97400 2023-10-11T09:45:58.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.064+0000 7f3bc9109840 4 Got MDS map 43 2023-10-11T09:45:58.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.064+0000 7f3bc9109840 4 datascan.main: Using metadata pool output 2023-10-11T09:45:58.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.064+0000 7f3bc9109840 4 datascan.main: connecting to RADOS... 2023-10-11T09:45:58.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.064+0000 7f3bc9109840 1 Processor -- start 2023-10-11T09:45:58.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.064+0000 7f3bc9109840 1 -- start start 2023-10-11T09:45:58.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.064+0000 7f3bc9109840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626b1288400 0x5626b103d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.064+0000 7f3bc9109840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5626b1288800 0x5626b103db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.064+0000 7f3bc9109840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626b1288c00 0x5626b103e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.064+0000 7f3bc9109840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626b0fcde00 con 0x5626b1288c00 2023-10-11T09:45:58.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.064+0000 7f3bc9109840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626b1016780 con 0x5626b1288400 2023-10-11T09:45:58.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.064+0000 7f3bc9109840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626b1016a00 con 0x5626b1288800 2023-10-11T09:45:58.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.064+0000 7f3bc70c4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626b1288c00 0x5626b103e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.064+0000 7f3bc70c4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626b1288c00 0x5626b103e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46048/0 (socket says 172.21.15.78:46048) 2023-10-11T09:45:58.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.064+0000 7f3bc70c4700 1 -- 172.21.15.78:0/1510033582 learned_addr learned my addr 172.21.15.78:0/1510033582 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.064+0000 7f3bc78c5700 1 --2- 172.21.15.78:0/1510033582 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5626b1288800 0x5626b103db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.064+0000 7f3bc80c6700 1 --2- 172.21.15.78:0/1510033582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626b1288400 0x5626b103d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.068+0000 7f3bc70c4700 1 -- 172.21.15.78:0/1510033582 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5626b1288800 msgr2=0x5626b103db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.068+0000 7f3bc70c4700 1 --2- 172.21.15.78:0/1510033582 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5626b1288800 0x5626b103db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.068+0000 7f3bc70c4700 1 -- 172.21.15.78:0/1510033582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626b1288400 msgr2=0x5626b103d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.068+0000 7f3bc70c4700 1 --2- 172.21.15.78:0/1510033582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626b1288400 0x5626b103d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.068+0000 7f3bc70c4700 1 -- 172.21.15.78:0/1510033582 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626b0fb8340 con 0x5626b1288c00 2023-10-11T09:45:58.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.068+0000 7f3bc78c5700 1 --2- 172.21.15.78:0/1510033582 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5626b1288800 0x5626b103db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:45:58.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.068+0000 7f3bc70c4700 1 --2- 172.21.15.78:0/1510033582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626b1288c00 0x5626b103e100 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x5626b1296840 tx=0x5626b106d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5675ee3656fe1e8a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.068+0000 7f3bc30bc700 1 -- 172.21.15.78:0/1510033582 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5626b0fc71e0 con 0x5626b1288c00 2023-10-11T09:45:58.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.068+0000 7f3bc30bc700 1 -- 172.21.15.78:0/1510033582 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5626b1d8c900 con 0x5626b1288c00 2023-10-11T09:45:58.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.068+0000 7f3bc30bc700 1 -- 172.21.15.78:0/1510033582 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5626b0fc6f20 con 0x5626b1288c00 2023-10-11T09:45:58.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.068+0000 7f3bc9109840 1 -- 172.21.15.78:0/1510033582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626b1288c00 msgr2=0x5626b103e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.068+0000 7f3bc9109840 1 --2- 172.21.15.78:0/1510033582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626b1288c00 0x5626b103e100 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x5626b1296840 tx=0x5626b106d4a0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.068+0000 7f3bc9109840 1 -- 172.21.15.78:0/1510033582 shutdown_connections 2023-10-11T09:45:58.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.068+0000 7f3bc9109840 1 --2- 172.21.15.78:0/1510033582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626b1288c00 0x5626b103e100 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.068+0000 7f3bc9109840 1 --2- 172.21.15.78:0/1510033582 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5626b1288800 0x5626b103db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.068+0000 7f3bc9109840 1 --2- 172.21.15.78:0/1510033582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626b1288400 0x5626b103d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.068+0000 7f3bc9109840 1 -- 172.21.15.78:0/1510033582 >> 172.21.15.78:0/1510033582 conn(0x5626b0f96c00 msgr2=0x5626b1065800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:58.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.068+0000 7f3bc9109840 1 -- 172.21.15.78:0/1510033582 shutdown_connections 2023-10-11T09:45:58.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.068+0000 7f3bc9109840 1 -- 172.21.15.78:0/1510033582 wait complete. 2023-10-11T09:45:58.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.068+0000 7f3bc9109840 1 Processor -- start 2023-10-11T09:45:58.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.068+0000 7f3bc9109840 1 -- start start 2023-10-11T09:45:58.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.068+0000 7f3bc9109840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626b1288400 0x5626b103f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.072+0000 7f3bc9109840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626b1288800 0x5626b103ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.072+0000 7f3bc9109840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5626b1288c00 0x5626b103e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.072+0000 7f3bc9109840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626b1016500 con 0x5626b1288800 2023-10-11T09:45:58.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.072+0000 7f3bc9109840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626b10163c0 con 0x5626b1288400 2023-10-11T09:45:58.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.072+0000 7f3bc9109840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626b1016280 con 0x5626b1288c00 2023-10-11T09:45:58.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.072+0000 7f3bc78c5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626b1288800 0x5626b103ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.072+0000 7f3bc78c5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626b1288800 0x5626b103ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46054/0 (socket says 172.21.15.78:46054) 2023-10-11T09:45:58.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.072+0000 7f3bc78c5700 1 -- 172.21.15.78:0/1447308667 learned_addr learned my addr 172.21.15.78:0/1447308667 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.072+0000 7f3bc80c6700 1 --2- 172.21.15.78:0/1447308667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626b1288400 0x5626b103f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.072+0000 7f3bc70c4700 1 --2- 172.21.15.78:0/1447308667 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5626b1288c00 0x5626b103e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.072+0000 7f3bc78c5700 1 -- 172.21.15.78:0/1447308667 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5626b1288c00 msgr2=0x5626b103e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.072+0000 7f3bc78c5700 1 --2- 172.21.15.78:0/1447308667 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5626b1288c00 0x5626b103e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.072+0000 7f3bc78c5700 1 -- 172.21.15.78:0/1447308667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626b1288400 msgr2=0x5626b103f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.072+0000 7f3bc78c5700 1 --2- 172.21.15.78:0/1447308667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626b1288400 0x5626b103f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.072+0000 7f3bc78c5700 1 -- 172.21.15.78:0/1447308667 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626b0fb91e0 con 0x5626b1288800 2023-10-11T09:45:58.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.072+0000 7f3bc78c5700 1 --2- 172.21.15.78:0/1447308667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626b1288800 0x5626b103ec00 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x5626b12976b0 tx=0x5626b1299020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.072+0000 7f3bc18b9700 1 -- 172.21.15.78:0/1447308667 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5626b0fc71e0 con 0x5626b1288800 2023-10-11T09:45:58.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.072+0000 7f3bc18b9700 1 -- 172.21.15.78:0/1447308667 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5626b1d8d500 con 0x5626b1288800 2023-10-11T09:45:58.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.072+0000 7f3bc9109840 1 -- 172.21.15.78:0/1447308667 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5626b0fb8b60 con 0x5626b1288800 2023-10-11T09:45:58.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.072+0000 7f3bc18b9700 1 -- 172.21.15.78:0/1447308667 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5626b0fc6f20 con 0x5626b1288800 2023-10-11T09:45:58.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.072+0000 7f3bc9109840 1 -- 172.21.15.78:0/1447308667 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626b0fb8340 con 0x5626b1288800 2023-10-11T09:45:58.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.072+0000 7f3bc9109840 4 datascan.init: resolving metadata pool 8 2023-10-11T09:45:58.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.076+0000 7f3bc18b9700 1 -- 172.21.15.78:0/1447308667 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x5626b0ff4840 con 0x5626b1288800 2023-10-11T09:45:58.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.076+0000 7f3bc18b9700 1 --2- 172.21.15.78:0/1447308667 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5626b13e6000 0x5626b103f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.076+0000 7f3bc18b9700 1 -- 172.21.15.78:0/1447308667 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x5626b0fd0a80 con 0x5626b1288800 2023-10-11T09:45:58.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.076+0000 7f3bc9109840 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-10-11T09:45:58.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.076+0000 7f3bc80c6700 1 --2- 172.21.15.78:0/1447308667 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5626b13e6000 0x5626b103f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.076+0000 7f3bc9109840 1 --2- 172.21.15.78:0/1447308667 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x5626b1289000 0x5626b103e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.076+0000 7f3bc9109840 1 -- 172.21.15.78:0/1447308667 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:1 8.14 8:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5626b1289400 con 0x5626b1289000 2023-10-11T09:45:58.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.076+0000 7f3bc70c4700 1 --2- 172.21.15.78:0/1447308667 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x5626b1289000 0x5626b103e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.076+0000 7f3bc80c6700 1 --2- 172.21.15.78:0/1447308667 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5626b13e6000 0x5626b103f700 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x5626b10f26f0 tx=0x5626b1298d20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.080+0000 7f3bc70c4700 1 --2- 172.21.15.78:0/1447308667 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x5626b1289000 0x5626b103e100 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.080+0000 7f3bc70c4700 1 -- 172.21.15.78:0/1447308667 <== osd.4 v2:172.21.15.133:6808/35898 1 ==== osd_op_reply(1 1.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 160+0+0 (crc 0 0 0) 0x5626b1e20fc0 con 0x5626b1289000 2023-10-11T09:45:58.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.080+0000 7f3bc9109840 1 -- 172.21.15.78:0/1447308667 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:2 8.14 8:287034c4:::1.00000000.inode:head [writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5626b1289800 con 0x5626b1289000 2023-10-11T09:45:58.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.084+0000 7f3bc70c4700 1 -- 172.21.15.78:0/1447308667 <== osd.4 v2:172.21.15.133:6808/35898 2 ==== osd_op_reply(2 1.00000000.inode [writefull 0~582] v66'97 uv97 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x5626b1e20fc0 con 0x5626b1289000 2023-10-11T09:45:58.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.084+0000 7f3bc9109840 1 --2- 172.21.15.78:0/1447308667 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x5626b13e6400 0x5626b103db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.084+0000 7f3bc9109840 1 -- 172.21.15.78:0/1447308667 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:3 8.1f 8:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5626b13e6800 con 0x5626b13e6400 2023-10-11T09:45:58.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.084+0000 7f3bc80c6700 1 --2- 172.21.15.78:0/1447308667 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x5626b13e6400 0x5626b103db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.084+0000 7f3bc80c6700 1 --2- 172.21.15.78:0/1447308667 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x5626b13e6400 0x5626b103db80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.084+0000 7f3bc80c6700 1 -- 172.21.15.78:0/1447308667 <== osd.0 v2:172.21.15.78:6800/35735 1 ==== osd_op_reply(3 100.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 162+0+0 (crc 0 0 0) 0x5626b1e21680 con 0x5626b13e6400 2023-10-11T09:45:58.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.084+0000 7f3bc9109840 1 -- 172.21.15.78:0/1447308667 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:4 8.1f 8:fe07bba1:::100.00000000.inode:head [writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5626b13e6c00 con 0x5626b13e6400 2023-10-11T09:45:58.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.088+0000 7f3bc80c6700 1 -- 172.21.15.78:0/1447308667 <== osd.0 v2:172.21.15.78:6800/35735 2 ==== osd_op_reply(4 100.00000000.inode [writefull 0~582] v66'10 uv10 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x5626b1e21680 con 0x5626b13e6400 2023-10-11T09:45:58.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.088+0000 7f3bc9109840 1 --2- 172.21.15.78:0/1447308667 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5626b13e7000 0x5626b103d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.088+0000 7f3bc9109840 1 -- 172.21.15.78:0/1447308667 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:5 8.13 8:cd5a64a3:::100.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5626b13e7400 con 0x5626b13e7000 2023-10-11T09:45:58.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.088+0000 7f3bc78c5700 1 --2- 172.21.15.78:0/1447308667 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5626b13e7000 0x5626b103d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.088+0000 7f3bc78c5700 1 --2- 172.21.15.78:0/1447308667 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5626b13e7000 0x5626b103d600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.088+0000 7f3bc78c5700 1 -- 172.21.15.78:0/1447308667 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(5 100.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5626b1e21d40 con 0x5626b13e7000 2023-10-11T09:45:58.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.088+0000 7f3bc9109840 10 datascan.find_or_create_dirfrag: read_version = 0 2023-10-11T09:45:58.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.088+0000 7f3bc9109840 1 -- 172.21.15.78:0/1447308667 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:6 8.13 8:cd5a64a3:::100.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5626b13e7800 con 0x5626b13e7000 2023-10-11T09:45:58.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.092+0000 7f3bc78c5700 1 -- 172.21.15.78:0/1447308667 <== osd.3 v2:172.21.15.78:6824/35738 2 ==== osd_op_reply(6 100.00000000 [create,omap-set-header] v66'7 uv7 ondisk = 0) v8 ==== 198+0+0 (crc 0 0 0) 0x5626b1e21d40 con 0x5626b13e7000 2023-10-11T09:45:58.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.092+0000 7f3bc9109840 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x100 2023-10-11T09:45:58.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.092+0000 7f3bc9109840 1 -- 172.21.15.78:0/1447308667 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x5626b13e6400 msgr2=0x5626b103db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.092+0000 7f3bc9109840 1 --2- 172.21.15.78:0/1447308667 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x5626b13e6400 0x5626b103db80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.092+0000 7f3bc9109840 1 -- 172.21.15.78:0/1447308667 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5626b13e7000 msgr2=0x5626b103d600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.092+0000 7f3bc9109840 1 --2- 172.21.15.78:0/1447308667 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5626b13e7000 0x5626b103d600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.092+0000 7f3bc9109840 1 -- 172.21.15.78:0/1447308667 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x5626b1289000 msgr2=0x5626b103e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.092+0000 7f3bc9109840 1 --2- 172.21.15.78:0/1447308667 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x5626b1289000 0x5626b103e100 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.092+0000 7f3bc78c5700 1 -- 172.21.15.78:0/1447308667 reap_dead start 2023-10-11T09:45:58.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.092+0000 7f3bc9109840 1 -- 172.21.15.78:0/1447308667 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5626b13e6000 msgr2=0x5626b103f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.092+0000 7f3bc9109840 1 --2- 172.21.15.78:0/1447308667 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5626b13e6000 0x5626b103f700 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x5626b10f26f0 tx=0x5626b1298d20 comp rx=0 tx=0).stop 2023-10-11T09:45:58.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.092+0000 7f3bc9109840 1 -- 172.21.15.78:0/1447308667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626b1288800 msgr2=0x5626b103ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.092+0000 7f3bc9109840 1 --2- 172.21.15.78:0/1447308667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626b1288800 0x5626b103ec00 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x5626b12976b0 tx=0x5626b1299020 comp rx=0 tx=0).stop 2023-10-11T09:45:58.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.092+0000 7f3bc9109840 1 -- 172.21.15.78:0/1447308667 shutdown_connections 2023-10-11T09:45:58.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.092+0000 7f3bc9109840 1 --2- 172.21.15.78:0/1447308667 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5626b13e6000 0x5626b103f700 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.092+0000 7f3bc9109840 1 --2- 172.21.15.78:0/1447308667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626b1288800 0x5626b103ec00 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.092+0000 7f3bc9109840 1 -- 172.21.15.78:0/1447308667 >> 172.21.15.78:0/1447308667 conn(0x5626b0f96c00 msgr2=0x5626b1065800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:58.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.092+0000 7f3bc9109840 1 -- 172.21.15.78:0/1447308667 shutdown_connections 2023-10-11T09:45:58.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.092+0000 7f3bc9109840 1 -- 172.21.15.78:0/1447308667 wait complete. 2023-10-11T09:45:58.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.092+0000 7f3bc9109840 1 -- 172.21.15.78:0/3603487859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626b0f97400 msgr2=0x5626b103cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.092+0000 7f3bc9109840 1 --2- 172.21.15.78:0/3603487859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626b0f97400 0x5626b103cb00 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x5626b10f3350 tx=0x5626b106d3e0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.092+0000 7f3bc9109840 1 -- 172.21.15.78:0/3603487859 shutdown_connections 2023-10-11T09:45:58.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.092+0000 7f3bc9109840 1 --2- 172.21.15.78:0/3603487859 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5626b0f97000 0x5626b103d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.092+0000 7f3bc9109840 1 --2- 172.21.15.78:0/3603487859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626b0f97400 0x5626b103cb00 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.092+0000 7f3bc9109840 1 --2- 172.21.15.78:0/3603487859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626b0f97800 0x5626b103c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.092+0000 7f3bc9109840 1 -- 172.21.15.78:0/3603487859 >> 172.21.15.78:0/3603487859 conn(0x5626b0f97c00 msgr2=0x5626b1065000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:58.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.092+0000 7f3bc9109840 1 -- 172.21.15.78:0/3603487859 shutdown_connections 2023-10-11T09:45:58.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.092+0000 7f3bc9109840 1 -- 172.21.15.78:0/3603487859 wait complete. 2023-10-11T09:45:58.110 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'init'] in time 0:00:00.109454, result: 2023-10-11T09:45:58.111 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents --worker_n 0 --worker_m 7 2023-10-11T09:45:58.112 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents --worker_n 1 --worker_m 7 2023-10-11T09:45:58.113 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents --worker_n 2 --worker_m 7 2023-10-11T09:45:58.113 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents --worker_n 3 --worker_m 7 2023-10-11T09:45:58.114 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents --worker_n 4 --worker_m 7 2023-10-11T09:45:58.115 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents --worker_n 5 --worker_m 7 2023-10-11T09:45:58.116 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents --worker_n 6 --worker_m 7 2023-10-11T09:45:58.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.188+0000 7f8f4c2d7840 1 -- start start 2023-10-11T09:45:58.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.188+0000 7f8f4c2d7840 1 Processor -- start 2023-10-11T09:45:58.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.188+0000 7f8f4c2d7840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647eb0e9800 0x5647eb18e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.188+0000 7f8f4c2d7840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647eb0e9400 0x5647eb18eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.188+0000 7f8f4c2d7840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5647eb0e9000 0x5647eb18f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.188+0000 7f8f4a292700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647eb0e9400 0x5647eb18eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.188+0000 7f8f4a292700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647eb0e9400 0x5647eb18eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46068/0 (socket says 172.21.15.78:46068) 2023-10-11T09:45:58.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.188+0000 7f8f4aa93700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647eb0e9800 0x5647eb18e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.188+0000 7f8f4aa93700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647eb0e9800 0x5647eb18e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:38874/0 (socket says 172.21.15.78:38874) 2023-10-11T09:45:58.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.188+0000 7f8f4a292700 1 -- 172.21.15.78:0/1997815101 learned_addr learned my addr 172.21.15.78:0/1997815101 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.188+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/1997815101 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647eb168a00 con 0x5647eb0e9400 2023-10-11T09:45:58.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.188+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/1997815101 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647eb168780 con 0x5647eb0e9800 2023-10-11T09:45:58.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.188+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/1997815101 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x5647eb168640 con 0x5647eb0e9000 2023-10-11T09:45:58.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.188+0000 7f8f4b294700 1 --2- 172.21.15.78:0/1997815101 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5647eb0e9000 0x5647eb18f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.188+0000 7f8f4a292700 1 -- 172.21.15.78:0/1997815101 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5647eb0e9000 msgr2=0x5647eb18f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.188+0000 7f8f4a292700 1 --2- 172.21.15.78:0/1997815101 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5647eb0e9000 0x5647eb18f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.188+0000 7f8f4a292700 1 -- 172.21.15.78:0/1997815101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647eb0e9800 msgr2=0x5647eb18e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.188+0000 7f8f4a292700 1 --2- 172.21.15.78:0/1997815101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647eb0e9800 0x5647eb18e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.188+0000 7f8f4a292700 1 -- 172.21.15.78:0/1997815101 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647eb10ab60 con 0x5647eb0e9400 2023-10-11T09:45:58.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.192+0000 7f8f4a292700 1 --2- 172.21.15.78:0/1997815101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647eb0e9400 0x5647eb18eb00 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x5647eb1b7410 tx=0x5647ebed6ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb0514fca9385529 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.192+0000 7f8f4828e700 1 -- 172.21.15.78:0/1997815101 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5647eb119600 con 0x5647eb0e9400 2023-10-11T09:45:58.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.192+0000 7f8f4828e700 1 -- 172.21.15.78:0/1997815101 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5647eb134480 con 0x5647eb0e9400 2023-10-11T09:45:58.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.192+0000 7f8f4828e700 1 -- 172.21.15.78:0/1997815101 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5647eb1191e0 con 0x5647eb0e9400 2023-10-11T09:45:58.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.192+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/1997815101 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5647eb10a340 con 0x5647eb0e9400 2023-10-11T09:45:58.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.192+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/1997815101 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5647eb10a000 con 0x5647eb0e9400 2023-10-11T09:45:58.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.192+0000 7f8f4c2d7840 4 waiting for MDS map... 2023-10-11T09:45:58.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.192+0000 7f8f4828e700 1 -- 172.21.15.78:0/1997815101 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x5647eb122c40 con 0x5647eb0e9400 2023-10-11T09:45:58.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.192+0000 7f8f4828e700 1 -- 172.21.15.78:0/1997815101 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 43) v1 ==== 3118+0+0 (secure 0 0 0) 0x5647ebecc000 con 0x5647eb0e9400 2023-10-11T09:45:58.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.192+0000 7f8f4c2d7840 4 Got MDS map 43 2023-10-11T09:45:58.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.192+0000 7f8f4c2d7840 4 datascan.main: Using metadata pool output 2023-10-11T09:45:58.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.192+0000 7f8f4c2d7840 4 datascan.main: connecting to RADOS... 2023-10-11T09:45:58.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.192+0000 7f8f4c2d7840 1 Processor -- start 2023-10-11T09:45:58.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.192+0000 7f8f4c2d7840 1 -- start start 2023-10-11T09:45:58.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.192+0000 7f8f4c2d7840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647eb3dc400 0x5647eb18f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.192+0000 7f8f4c2d7840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5647eb3dc800 0x5647eb18fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.192+0000 7f8f4c2d7840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647eb3dcc00 0x5647eb190100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.192+0000 7f8f4c2d7840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647eb11fe00 con 0x5647eb3dcc00 2023-10-11T09:45:58.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.192+0000 7f8f4c2d7840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647eb168780 con 0x5647eb3dc400 2023-10-11T09:45:58.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.192+0000 7f8f4c2d7840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x5647eb168500 con 0x5647eb3dc800 2023-10-11T09:45:58.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.196+0000 7f8f4b294700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647eb3dc400 0x5647eb18f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.196+0000 7f8f4a292700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647eb3dcc00 0x5647eb190100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.196+0000 7f8f4b294700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647eb3dc400 0x5647eb18f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:38880/0 (socket says 172.21.15.78:38880) 2023-10-11T09:45:58.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.196+0000 7f8f4b294700 1 -- 172.21.15.78:0/4063922658 learned_addr learned my addr 172.21.15.78:0/4063922658 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.196+0000 7f8f4a292700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647eb3dcc00 0x5647eb190100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46074/0 (socket says 172.21.15.78:46074) 2023-10-11T09:45:58.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.196+0000 7f8f4aa93700 1 --2- 172.21.15.78:0/4063922658 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5647eb3dc800 0x5647eb18fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.196+0000 7f8f4a292700 1 -- 172.21.15.78:0/4063922658 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5647eb3dc800 msgr2=0x5647eb18fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.196+0000 7f8f4a292700 1 --2- 172.21.15.78:0/4063922658 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5647eb3dc800 0x5647eb18fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.196+0000 7f8f4a292700 1 -- 172.21.15.78:0/4063922658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647eb3dc400 msgr2=0x5647eb18f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.196+0000 7f8f4a292700 1 --2- 172.21.15.78:0/4063922658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647eb3dc400 0x5647eb18f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.196+0000 7f8f4a292700 1 -- 172.21.15.78:0/4063922658 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647eb10aea0 con 0x5647eb3dcc00 2023-10-11T09:45:58.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.196+0000 7f8f4a292700 1 --2- 172.21.15.78:0/4063922658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647eb3dcc00 0x5647eb190100 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x5647eb1b77a0 tx=0x5647ebed7020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32d46344e031b862 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.196+0000 7f8f4628a700 1 -- 172.21.15.78:0/4063922658 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5647eb118f20 con 0x5647eb3dcc00 2023-10-11T09:45:58.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.196+0000 7f8f4628a700 1 -- 172.21.15.78:0/4063922658 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5647eb24ed80 con 0x5647eb3dcc00 2023-10-11T09:45:58.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.196+0000 7f8f4628a700 1 -- 172.21.15.78:0/4063922658 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5647eb119600 con 0x5647eb3dcc00 2023-10-11T09:45:58.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.196+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/4063922658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647eb3dcc00 msgr2=0x5647eb190100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.196+0000 7f8f4c2d7840 1 --2- 172.21.15.78:0/4063922658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647eb3dcc00 0x5647eb190100 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x5647eb1b77a0 tx=0x5647ebed7020 comp rx=0 tx=0).stop 2023-10-11T09:45:58.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.196+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/4063922658 shutdown_connections 2023-10-11T09:45:58.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.196+0000 7f8f4c2d7840 1 --2- 172.21.15.78:0/4063922658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647eb3dcc00 0x5647eb190100 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.196+0000 7f8f4c2d7840 1 --2- 172.21.15.78:0/4063922658 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5647eb3dc800 0x5647eb18fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.196+0000 7f8f4c2d7840 1 --2- 172.21.15.78:0/4063922658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647eb3dc400 0x5647eb18f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.196+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/4063922658 >> 172.21.15.78:0/4063922658 conn(0x5647eb0e8c00 msgr2=0x5647eb1dd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:58.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.196+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/4063922658 shutdown_connections 2023-10-11T09:45:58.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.196+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/4063922658 wait complete. 2023-10-11T09:45:58.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.200+0000 7f8f4c2d7840 1 Processor -- start 2023-10-11T09:45:58.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.200+0000 7f8f4c2d7840 1 -- start start 2023-10-11T09:45:58.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.200+0000 7f8f4c2d7840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647eb3dc400 0x5647eb191180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.200+0000 7f8f4c2d7840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647eb3dc800 0x5647eb190c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.200+0000 7f8f4c2d7840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5647eb3dcc00 0x5647eb190680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.200+0000 7f8f4c2d7840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647eb1683c0 con 0x5647eb3dc400 2023-10-11T09:45:58.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.200+0000 7f8f4c2d7840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647eb168280 con 0x5647eb3dc800 2023-10-11T09:45:58.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.200+0000 7f8f4c2d7840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x5647eb168140 con 0x5647eb3dcc00 2023-10-11T09:45:58.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.200+0000 7f8f4b294700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647eb3dc400 0x5647eb191180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.200+0000 7f8f4b294700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647eb3dc400 0x5647eb191180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46090/0 (socket says 172.21.15.78:46090) 2023-10-11T09:45:58.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.200+0000 7f8f4b294700 1 -- 172.21.15.78:0/229716028 learned_addr learned my addr 172.21.15.78:0/229716028 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.200+0000 7f8f4aa93700 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647eb3dc800 0x5647eb190c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.200+0000 7f8f4a292700 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5647eb3dcc00 0x5647eb190680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.204+0000 7f8f4b294700 1 -- 172.21.15.78:0/229716028 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5647eb3dcc00 msgr2=0x5647eb190680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.204+0000 7f8f4b294700 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5647eb3dcc00 0x5647eb190680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.204+0000 7f8f4b294700 1 -- 172.21.15.78:0/229716028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647eb3dc800 msgr2=0x5647eb190c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.204+0000 7f8f4b294700 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647eb3dc800 0x5647eb190c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.204+0000 7f8f4b294700 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647eb10ab60 con 0x5647eb3dc400 2023-10-11T09:45:58.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.204+0000 7f8f4aa93700 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647eb3dc800 0x5647eb190c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:45:58.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.204+0000 7f8f4b294700 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647eb3dc400 0x5647eb191180 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x5647ebed14d0 tx=0x5647eb3e2c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.204+0000 7f8f44a87700 1 -- 172.21.15.78:0/229716028 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5647eb118f20 con 0x5647eb3dc400 2023-10-11T09:45:58.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.204+0000 7f8f44a87700 1 -- 172.21.15.78:0/229716028 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5647eb24f680 con 0x5647eb3dc400 2023-10-11T09:45:58.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.204+0000 7f8f44a87700 1 -- 172.21.15.78:0/229716028 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5647eb119600 con 0x5647eb3dc400 2023-10-11T09:45:58.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.204+0000 7effacf1a840 1 -- start start 2023-10-11T09:45:58.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.204+0000 7effacf1a840 1 Processor -- start 2023-10-11T09:45:58.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.204+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5647eb10a340 con 0x5647eb3dc400 2023-10-11T09:45:58.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.204+0000 7effacf1a840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597a16a7800 0x5597a174c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.204+0000 7fae1b9bb840 1 -- start start 2023-10-11T09:45:58.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.204+0000 7fae1b9bb840 1 Processor -- start 2023-10-11T09:45:58.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.204+0000 7f8f44a87700 1 -- 172.21.15.78:0/229716028 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x5647eb146840 con 0x5647eb3dc400 2023-10-11T09:45:58.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.204+0000 7effab6d6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597a16a7800 0x5597a174c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.204+0000 7effab6d6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597a16a7800 0x5597a174c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46100/0 (socket says 172.21.15.78:46100) 2023-10-11T09:45:58.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7effacf1a840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5597a16a7400 0x5597a174cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7effacf1a840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5597a16a7000 0x5597a174d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.218 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7f8f44a87700 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5647eb542000 0x5647eb191700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.218 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7fae1b9bb840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613b9e17800 0x5613b9ebc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.218 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7fae1b9bb840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5613b9e17400 0x5613b9ebcb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5647eb10b520 con 0x5647eb3dc400 2023-10-11T09:45:58.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7f8f4c2d7840 4 datascan.init: resolving metadata pool 8 2023-10-11T09:45:58.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7f8f4aa93700 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5647eb542000 0x5647eb191700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7f8f44a87700 1 -- 172.21.15.78:0/229716028 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x5647eb122a80 con 0x5647eb3dc400 2023-10-11T09:45:58.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7effacf1a840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597a1726780 con 0x5597a16a7800 2023-10-11T09:45:58.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7effacf1a840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597a1726a00 con 0x5597a16a7400 2023-10-11T09:45:58.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7effacf1a840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x5597a1726640 con 0x5597a16a7000 2023-10-11T09:45:58.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7effab6d6700 1 -- 172.21.15.78:0/2311838660 learned_addr learned my addr 172.21.15.78:0/2311838660 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7fae1a177700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613b9e17800 0x5613b9ebc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7effaaed5700 1 --2- 172.21.15.78:0/2311838660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5597a16a7400 0x5597a174cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7effabed7700 1 --2- 172.21.15.78:0/2311838660 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5597a16a7000 0x5597a174d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7fae1a177700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613b9e17800 0x5613b9ebc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46110/0 (socket says 172.21.15.78:46110) 2023-10-11T09:45:58.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7effab6d6700 1 -- 172.21.15.78:0/2311838660 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5597a16a7000 msgr2=0x5597a174d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7effab6d6700 1 --2- 172.21.15.78:0/2311838660 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5597a16a7000 0x5597a174d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7effab6d6700 1 -- 172.21.15.78:0/2311838660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5597a16a7400 msgr2=0x5597a174cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7effab6d6700 1 --2- 172.21.15.78:0/2311838660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5597a16a7400 0x5597a174cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7effab6d6700 1 -- 172.21.15.78:0/2311838660 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597a16c8b60 con 0x5597a16a7800 2023-10-11T09:45:58.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7fae1a177700 1 -- 172.21.15.78:0/3608809413 learned_addr learned my addr 172.21.15.78:0/3608809413 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7fae1b9bb840 1 --2- 172.21.15.78:0/3608809413 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5613b9e17000 0x5613b9ebd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7f8f4c2d7840 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-10-11T09:45:58.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7f8f4c2d7840 4 datascan.main: data pool 'cephfs_data' has ID 9 2023-10-11T09:45:58.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7f8f4c2d7840 4 datascan.main: opening data pool 'cephfs_data' 2023-10-11T09:45:58.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7f8f4c2d7840 4 datascan.main: data pool 'cephfs_data_extra' has ID 10 2023-10-11T09:45:58.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7f8f4c2d7840 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-10-11T09:45:58.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7f8f4c2d7840 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x5647eb3dd000 0x5647eb190100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:1 9.36 9:6db6db6d::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb3dd400 con 0x5647eb3dd000 2023-10-11T09:45:58.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7fae19976700 1 --2- 172.21.15.78:0/3608809413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5613b9e17400 0x5613b9ebcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7fae1a978700 1 --2- 172.21.15.78:0/3608809413 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5613b9e17000 0x5613b9ebd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7fae1b9bb840 1 -- 172.21.15.78:0/3608809413 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613b9e4de00 con 0x5613b9e17800 2023-10-11T09:45:58.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7fae1b9bb840 1 -- 172.21.15.78:0/3608809413 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613b9e96a00 con 0x5613b9e17400 2023-10-11T09:45:58.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7fae1b9bb840 1 -- 172.21.15.78:0/3608809413 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x5613b9e96640 con 0x5613b9e17000 2023-10-11T09:45:58.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7f8f4a292700 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x5647eb3dd000 0x5647eb190100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7effab6d6700 1 --2- 172.21.15.78:0/2311838660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597a16a7800 0x5597a174c580 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x5597a17af350 tx=0x5597a2488000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d6adf07710610f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7f8f4aa93700 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5647eb542000 0x5647eb191700 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x5647eb13ffb0 tx=0x5647eb1bf380 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7fae1a978700 1 -- 172.21.15.78:0/3608809413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5613b9e17400 msgr2=0x5613b9ebcb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7fae1a978700 1 --2- 172.21.15.78:0/3608809413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5613b9e17400 0x5613b9ebcb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7fae1a978700 1 -- 172.21.15.78:0/3608809413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613b9e17800 msgr2=0x5613b9ebc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7fae1a978700 1 --2- 172.21.15.78:0/3608809413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613b9e17800 0x5613b9ebc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7fae1a978700 1 -- 172.21.15.78:0/3608809413 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613b9e38b60 con 0x5613b9e17000 2023-10-11T09:45:58.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7f8f4a292700 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x5647eb3dd000 0x5647eb190100 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.212+0000 7f8f4a292700 1 -- 172.21.15.78:0/229716028 <== osd.1 v2:172.21.15.78:6809/35734 1 ==== osd_op_reply(1 [pgnls start_epoch 66 out=101b] v64'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5647ebf68fc0 con 0x5647eb3dd000 2023-10-11T09:45:58.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.212+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:2 9.36 9:6ee5e2c4:::100000001f9.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb3dd800 con 0x5647eb3dd000 2023-10-11T09:45:58.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7effa8ed1700 1 -- 172.21.15.78:0/2311838660 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5597a16d7080 con 0x5597a16a7800 2023-10-11T09:45:58.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7effa8ed1700 1 -- 172.21.15.78:0/2311838660 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5597a2492300 con 0x5597a16a7800 2023-10-11T09:45:58.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7effa8ed1700 1 -- 172.21.15.78:0/2311838660 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5597a16d7600 con 0x5597a16a7800 2023-10-11T09:45:58.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7effacf1a840 1 -- 172.21.15.78:0/2311838660 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5597a16c91e0 con 0x5597a16a7800 2023-10-11T09:45:58.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7effacf1a840 1 -- 172.21.15.78:0/2311838660 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5597a16c8b60 con 0x5597a16a7800 2023-10-11T09:45:58.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.208+0000 7effacf1a840 4 waiting for MDS map... 2023-10-11T09:45:58.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.212+0000 7fae1a978700 1 --2- 172.21.15.78:0/3608809413 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5613b9e17000 0x5613b9ebd080 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x5613b9f2f380 tx=0x5613babf6f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f1dd16ac0c9b218 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.212+0000 7f8f4a292700 1 -- 172.21.15.78:0/229716028 <== osd.1 v2:172.21.15.78:6809/35734 2 ==== osd_op_reply(2 100000001f9.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5647ebf68fc0 con 0x5647eb3dd000 2023-10-11T09:45:58.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.212+0000 7f8f4c2d7840 10 datascan.operator(): handling object 1099511628281.0 2023-10-11T09:45:58.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.212+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:3 9.36 9:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb542400 con 0x5647eb3dd000 2023-10-11T09:45:58.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.212+0000 7fae17972700 1 -- 172.21.15.78:0/3608809413 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5613b9e47600 con 0x5613b9e17000 2023-10-11T09:45:58.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.212+0000 7fae17972700 1 -- 172.21.15.78:0/3608809413 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5613b9e62480 con 0x5613b9e17000 2023-10-11T09:45:58.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.212+0000 7fae17972700 1 -- 172.21.15.78:0/3608809413 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5613b9e471e0 con 0x5613b9e17000 2023-10-11T09:45:58.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.212+0000 7fae1b9bb840 1 -- 172.21.15.78:0/3608809413 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5613b9e38340 con 0x5613b9e17000 2023-10-11T09:45:58.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.212+0000 7fae1b9bb840 1 -- 172.21.15.78:0/3608809413 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5613b9e38000 con 0x5613b9e17000 2023-10-11T09:45:58.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.212+0000 7fae1b9bb840 4 waiting for MDS map... 2023-10-11T09:45:58.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.212+0000 7fae17972700 1 -- 172.21.15.78:0/3608809413 <== mon.2 v2:172.21.15.133:3301/0 4 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x5613b9e50c40 con 0x5613b9e17000 2023-10-11T09:45:58.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.212+0000 7fae17972700 1 -- 172.21.15.78:0/3608809413 <== mon.2 v2:172.21.15.133:3301/0 5 ==== fsmap(e 43) v1 ==== 3118+0+0 (secure 0 0 0) 0x5613ba10c000 con 0x5613b9e17000 2023-10-11T09:45:58.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.212+0000 7effa8ed1700 1 -- 172.21.15.78:0/2311838660 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x5597a16e0c40 con 0x5597a16a7800 2023-10-11T09:45:58.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.212+0000 7effa8ed1700 1 -- 172.21.15.78:0/2311838660 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 43) v1 ==== 3118+0+0 (secure 0 0 0) 0x5597a16e8f00 con 0x5597a16a7800 2023-10-11T09:45:58.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.212+0000 7effacf1a840 4 Got MDS map 43 2023-10-11T09:45:58.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.212+0000 7effacf1a840 4 datascan.main: Using metadata pool output 2023-10-11T09:45:58.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.212+0000 7effacf1a840 4 datascan.main: connecting to RADOS... 2023-10-11T09:45:58.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.212+0000 7fae1b9bb840 4 Got MDS map 43 2023-10-11T09:45:58.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.212+0000 7effacf1a840 1 Processor -- start 2023-10-11T09:45:58.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.212+0000 7fae1b9bb840 4 datascan.main: Using metadata pool output 2023-10-11T09:45:58.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.212+0000 7fae1b9bb840 4 datascan.main: connecting to RADOS... 2023-10-11T09:45:58.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.212+0000 7effacf1a840 1 -- start start 2023-10-11T09:45:58.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.212+0000 7fae1b9bb840 1 Processor -- start 2023-10-11T09:45:58.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.212+0000 7fae1b9bb840 1 -- start start 2023-10-11T09:45:58.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7f4698f2d840 1 -- start start 2023-10-11T09:45:58.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7f4698f2d840 1 Processor -- start 2023-10-11T09:45:58.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7effacf1a840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5597a1996400 0x5597a174d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7effacf1a840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5597a1996800 0x5597a174db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7effacf1a840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597a1996c00 0x5597a174e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7effacf1a840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597a16dde00 con 0x5597a1996c00 2023-10-11T09:45:58.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7effacf1a840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597a1726a00 con 0x5597a1996400 2023-10-11T09:45:58.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7effacf1a840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x5597a1726780 con 0x5597a1996800 2023-10-11T09:45:58.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7fae1b9bb840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613ba10e400 0x5613b9ebd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7fb86dabf840 1 -- start start 2023-10-11T09:45:58.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7fb86dabf840 1 Processor -- start 2023-10-11T09:45:58.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7fae1b9bb840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5613ba10e800 0x5613b9ebdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7effaaed5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597a1996c00 0x5597a174e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7effaaed5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597a1996c00 0x5597a174e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46128/0 (socket says 172.21.15.78:46128) 2023-10-11T09:45:58.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7effaaed5700 1 -- 172.21.15.78:0/3505873246 learned_addr learned my addr 172.21.15.78:0/3505873246 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7effabed7700 1 --2- 172.21.15.78:0/3505873246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5597a1996400 0x5597a174d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7fae1b9bb840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5613ba10ec00 0x5613b9ebe100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7fae1b9bb840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613b9e96500 con 0x5613ba10e400 2023-10-11T09:45:58.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7effabed7700 1 -- 172.21.15.78:0/3505873246 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5597a1996800 msgr2=0x5597a174db80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-11T09:45:58.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7effabed7700 1 --2- 172.21.15.78:0/3505873246 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5597a1996800 0x5597a174db80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7effabed7700 1 -- 172.21.15.78:0/3505873246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597a1996c00 msgr2=0x5597a174e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7effabed7700 1 --2- 172.21.15.78:0/3505873246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597a1996c00 0x5597a174e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7f4698f2d840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bf0935000 0x557bf09da580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7f4698f2d840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x557bf0935400 0x557bf09dab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7fae1b9bb840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613b9e963c0 con 0x5613ba10ec00 2023-10-11T09:45:58.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7fae1b9bb840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x5613b9e96a00 con 0x5613ba10e800 2023-10-11T09:45:58.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7fae1a177700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5613ba10e800 0x5613b9ebdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7fae1a177700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5613ba10e800 0x5613b9ebdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:53508/0 (socket says 172.21.15.78:53508) 2023-10-11T09:45:58.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7effabed7700 1 -- 172.21.15.78:0/3505873246 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597a16c8340 con 0x5597a1996400 2023-10-11T09:45:58.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7fae1a177700 1 -- 172.21.15.78:0/2063938977 learned_addr learned my addr 172.21.15.78:0/2063938977 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7f4698f2d840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bf0935800 0x557bf09db080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7f8f4a292700 1 -- 172.21.15.78:0/229716028 <== osd.1 v2:172.21.15.78:6809/35734 3 ==== osd_op_reply(3 100000001f9.00000000 [call] v66'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5647ebf68fc0 con 0x5647eb3dd000 2023-10-11T09:45:58.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:4 9.36 9:6fe38dc2::::head [pgnls start_epoch 66 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb542800 con 0x5647eb3dd000 2023-10-11T09:45:58.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7f8f4a292700 1 -- 172.21.15.78:0/229716028 <== osd.1 v2:172.21.15.78:6809/35734 4 ==== osd_op_reply(4 [pgnls start_epoch 66 out=81b] v66'18 uv18 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5647ebf68fc0 con 0x5647eb3dd000 2023-10-11T09:45:58.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:5 9.36 9:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb542c00 con 0x5647eb3dd000 2023-10-11T09:45:58.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7f8f4a292700 1 -- 172.21.15.78:0/229716028 <== osd.1 v2:172.21.15.78:6809/35734 5 ==== osd_op_reply(5 100000001f7.00000000 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5647ebf68fc0 con 0x5647eb3dd000 2023-10-11T09:45:58.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7f8f4c2d7840 10 datascan.operator(): handling object 1099511628279.0 2023-10-11T09:45:58.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:6 9.36 9:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb543000 con 0x5647eb3dd000 2023-10-11T09:45:58.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7effabed7700 1 --2- 172.21.15.78:0/3505873246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5597a1996400 0x5597a174d600 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x5597a19b8690 tx=0x5597a2486e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7badfb45c7e13b58 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7f4698f2d840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf096be00 con 0x557bf0935800 2023-10-11T09:45:58.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7f4698f2d840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf09b4780 con 0x557bf0935000 2023-10-11T09:45:58.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7f4698f2d840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x557bf09b4a00 con 0x557bf0935400 2023-10-11T09:45:58.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7f46976e9700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bf0935000 0x557bf09da580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7f46976e9700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bf0935000 0x557bf09da580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:38944/0 (socket says 172.21.15.78:38944) 2023-10-11T09:45:58.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7f46976e9700 1 -- 172.21.15.78:0/1193264917 learned_addr learned my addr 172.21.15.78:0/1193264917 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7f4696ee8700 1 --2- 172.21.15.78:0/1193264917 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x557bf0935400 0x557bf09dab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7fae1a177700 1 -- 172.21.15.78:0/2063938977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5613ba10ec00 msgr2=0x5613b9ebe100 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-11T09:45:58.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7fae1a177700 1 --2- 172.21.15.78:0/2063938977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5613ba10ec00 0x5613b9ebe100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7fae1a177700 1 -- 172.21.15.78:0/2063938977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613ba10e400 msgr2=0x5613b9ebd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7fae1a177700 1 --2- 172.21.15.78:0/2063938977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613ba10e400 0x5613b9ebd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7fae1a177700 1 -- 172.21.15.78:0/2063938977 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613b9e38ea0 con 0x5613ba10e800 2023-10-11T09:45:58.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7fb86dabf840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ab9731800 0x562ab97d6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7fb86dabf840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ab9731400 0x562ab97d6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7fb86dabf840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562ab9731000 0x562ab97d7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7fb86dabf840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ab97b0780 con 0x562ab9731800 2023-10-11T09:45:58.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7fb86dabf840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ab97b0a00 con 0x562ab9731400 2023-10-11T09:45:58.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7fb86dabf840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ab97b0640 con 0x562ab9731000 2023-10-11T09:45:58.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7effa6ecd700 1 -- 172.21.15.78:0/3505873246 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5597a16d7080 con 0x5597a1996400 2023-10-11T09:45:58.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7effa6ecd700 1 -- 172.21.15.78:0/3505873246 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5597a2493200 con 0x5597a1996400 2023-10-11T09:45:58.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7effa6ecd700 1 -- 172.21.15.78:0/3505873246 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5597a16d7600 con 0x5597a1996400 2023-10-11T09:45:58.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7f4697eea700 1 --2- 172.21.15.78:0/1193264917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bf0935800 0x557bf09db080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7fae1a177700 1 --2- 172.21.15.78:0/2063938977 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5613ba10e800 0x5613b9ebdb80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x5613babf8870 tx=0x5613babf6000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fc98e92d438d1c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7fb86c27b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ab9731800 0x562ab97d6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7fb86c27b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ab9731800 0x562ab97d6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46150/0 (socket says 172.21.15.78:46150) 2023-10-11T09:45:58.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7fb86c27b700 1 -- 172.21.15.78:0/1503568008 learned_addr learned my addr 172.21.15.78:0/1503568008 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7fb86ba7a700 1 --2- 172.21.15.78:0/1503568008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ab9731400 0x562ab97d6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.216+0000 7fb86ca7c700 1 --2- 172.21.15.78:0/1503568008 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562ab9731000 0x562ab97d7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7effacf1a840 1 -- 172.21.15.78:0/3505873246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5597a1996400 msgr2=0x5597a174d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7effacf1a840 1 --2- 172.21.15.78:0/3505873246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5597a1996400 0x5597a174d600 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x5597a19b8690 tx=0x5597a2486e40 comp rx=0 tx=0).stop 2023-10-11T09:45:58.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7effacf1a840 1 -- 172.21.15.78:0/3505873246 shutdown_connections 2023-10-11T09:45:58.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7effacf1a840 1 --2- 172.21.15.78:0/3505873246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597a1996c00 0x5597a174e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7effacf1a840 1 --2- 172.21.15.78:0/3505873246 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5597a1996800 0x5597a174db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7effacf1a840 1 --2- 172.21.15.78:0/3505873246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5597a1996400 0x5597a174d600 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7effacf1a840 1 -- 172.21.15.78:0/3505873246 >> 172.21.15.78:0/3505873246 conn(0x5597a16a6c00 msgr2=0x5597a179b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:58.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7f46976e9700 1 -- 172.21.15.78:0/1193264917 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x557bf0935400 msgr2=0x557bf09dab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7f46976e9700 1 --2- 172.21.15.78:0/1193264917 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x557bf0935400 0x557bf09dab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7f46976e9700 1 -- 172.21.15.78:0/1193264917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bf0935800 msgr2=0x557bf09db080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7f46976e9700 1 --2- 172.21.15.78:0/1193264917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bf0935800 0x557bf09db080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7f46976e9700 1 -- 172.21.15.78:0/1193264917 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bf0956340 con 0x557bf0935000 2023-10-11T09:45:58.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7f46976e9700 1 --2- 172.21.15.78:0/1193264917 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bf0935000 0x557bf09da580 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x557bf0a9b3b0 tx=0x557bf172c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=785d8ca02a028b9b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7fb86c27b700 1 -- 172.21.15.78:0/1503568008 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562ab9731000 msgr2=0x562ab97d7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7fb86c27b700 1 --2- 172.21.15.78:0/1503568008 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562ab9731000 0x562ab97d7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7fb86c27b700 1 -- 172.21.15.78:0/1503568008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ab9731400 msgr2=0x562ab97d6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7fb86c27b700 1 --2- 172.21.15.78:0/1503568008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ab9731400 0x562ab97d6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7fb86c27b700 1 -- 172.21.15.78:0/1503568008 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ab9752b60 con 0x562ab9731800 2023-10-11T09:45:58.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7fae1596e700 1 -- 172.21.15.78:0/2063938977 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5613b9e47080 con 0x5613ba10e800 2023-10-11T09:45:58.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7fae1596e700 1 -- 172.21.15.78:0/2063938977 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5613bac9ec00 con 0x5613ba10e800 2023-10-11T09:45:58.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7fae1596e700 1 -- 172.21.15.78:0/2063938977 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5613b9e47600 con 0x5613ba10e800 2023-10-11T09:45:58.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7fae1b9bb840 1 -- 172.21.15.78:0/2063938977 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5613ba10e800 msgr2=0x5613b9ebdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7fae1b9bb840 1 --2- 172.21.15.78:0/2063938977 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5613ba10e800 0x5613b9ebdb80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x5613babf8870 tx=0x5613babf6000 comp rx=0 tx=0).stop 2023-10-11T09:45:58.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7effacf1a840 1 -- 172.21.15.78:0/3505873246 shutdown_connections 2023-10-11T09:45:58.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7f4694ee4700 1 -- 172.21.15.78:0/1193264917 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557bf0965080 con 0x557bf0935000 2023-10-11T09:45:58.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7f4694ee4700 1 -- 172.21.15.78:0/1193264917 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557bf0980180 con 0x557bf0935000 2023-10-11T09:45:58.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7f4694ee4700 1 -- 172.21.15.78:0/1193264917 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557bf0965600 con 0x557bf0935000 2023-10-11T09:45:58.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7f4698f2d840 1 -- 172.21.15.78:0/1193264917 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557bf09571e0 con 0x557bf0935000 2023-10-11T09:45:58.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7f4698f2d840 1 -- 172.21.15.78:0/1193264917 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x557bf0956340 con 0x557bf0935000 2023-10-11T09:45:58.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7f4698f2d840 4 waiting for MDS map... 2023-10-11T09:45:58.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7fae1b9bb840 1 -- 172.21.15.78:0/2063938977 shutdown_connections 2023-10-11T09:45:58.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7fae1b9bb840 1 --2- 172.21.15.78:0/2063938977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5613ba10ec00 0x5613b9ebe100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7fae1b9bb840 1 --2- 172.21.15.78:0/2063938977 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5613ba10e800 0x5613b9ebdb80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7fae1b9bb840 1 --2- 172.21.15.78:0/2063938977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613ba10e400 0x5613b9ebd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7fae1b9bb840 1 -- 172.21.15.78:0/2063938977 >> 172.21.15.78:0/2063938977 conn(0x5613b9e16c00 msgr2=0x5613b9ed3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:58.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7effacf1a840 1 -- 172.21.15.78:0/3505873246 wait complete. 2023-10-11T09:45:58.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7fb86c27b700 1 --2- 172.21.15.78:0/1503568008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ab9731800 0x562ab97d6580 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x562ab97ff320 tx=0x562ab9896120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0aed4a4495d0b99 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7f4694ee4700 1 -- 172.21.15.78:0/1193264917 <== mon.1 v2:172.21.15.133:3300/0 4 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x557bf096ec40 con 0x557bf0935000 2023-10-11T09:45:58.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7f4694ee4700 1 -- 172.21.15.78:0/1193264917 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 43) v1 ==== 3118+0+0 (secure 0 0 0) 0x557bf0976f00 con 0x557bf0935000 2023-10-11T09:45:58.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7f4698f2d840 4 Got MDS map 43 2023-10-11T09:45:58.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7fae1b9bb840 1 -- 172.21.15.78:0/2063938977 shutdown_connections 2023-10-11T09:45:58.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7fae1b9bb840 1 -- 172.21.15.78:0/2063938977 wait complete. 2023-10-11T09:45:58.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7f8f4a292700 1 -- 172.21.15.78:0/229716028 <== osd.1 v2:172.21.15.78:6809/35734 6 ==== osd_op_reply(6 100000001f7.00000000 [call] v66'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5647ebf68fc0 con 0x5647eb3dd000 2023-10-11T09:45:58.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7f8f4c2d7840 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x5647eb543400 0x5647eb18fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:7 9.e 9:70000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb543800 con 0x5647eb543400 2023-10-11T09:45:58.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7f4698f2d840 4 datascan.main: Using metadata pool output 2023-10-11T09:45:58.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7f4698f2d840 4 datascan.main: connecting to RADOS... 2023-10-11T09:45:58.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7fb869a76700 1 -- 172.21.15.78:0/1503568008 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562ab9761080 con 0x562ab9731800 2023-10-11T09:45:58.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7fb869a76700 1 -- 172.21.15.78:0/1503568008 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562aba51e300 con 0x562ab9731800 2023-10-11T09:45:58.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7fb869a76700 1 -- 172.21.15.78:0/1503568008 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562ab9761600 con 0x562ab9731800 2023-10-11T09:45:58.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7fb86dabf840 1 -- 172.21.15.78:0/1503568008 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ab97531e0 con 0x562ab9731800 2023-10-11T09:45:58.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7fb86dabf840 1 -- 172.21.15.78:0/1503568008 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x562ab9752b60 con 0x562ab9731800 2023-10-11T09:45:58.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7f8f4b294700 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x5647eb543400 0x5647eb18fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7fb86dabf840 4 waiting for MDS map... 2023-10-11T09:45:58.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7fae1b9bb840 1 Processor -- start 2023-10-11T09:45:58.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7fb869a76700 1 -- 172.21.15.78:0/1503568008 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x562ab976ac40 con 0x562ab9731800 2023-10-11T09:45:58.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7f4698f2d840 1 Processor -- start 2023-10-11T09:45:58.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7fb869a76700 1 -- 172.21.15.78:0/1503568008 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 43) v1 ==== 3118+0+0 (secure 0 0 0) 0x562ab9772f00 con 0x562ab9731800 2023-10-11T09:45:58.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.220+0000 7fb86dabf840 4 Got MDS map 43 2023-10-11T09:45:58.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7f8f4b294700 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x5647eb543400 0x5647eb18fb80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7f4698f2d840 1 -- start start 2023-10-11T09:45:58.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7fae1b9bb840 1 -- start start 2023-10-11T09:45:58.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7effacf1a840 1 Processor -- start 2023-10-11T09:45:58.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7f8f4b294700 1 -- 172.21.15.78:0/229716028 <== osd.7 v2:172.21.15.133:6824/35901 1 ==== osd_op_reply(7 [pgnls start_epoch 66 out=49b] v64'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5647ebf69200 con 0x5647eb543400 2023-10-11T09:45:58.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7fb86dabf840 4 datascan.main: Using metadata pool output 2023-10-11T09:45:58.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7fb86dabf840 4 datascan.main: connecting to RADOS... 2023-10-11T09:45:58.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7effacf1a840 1 -- start start 2023-10-11T09:45:58.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7f4698f2d840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x557bf1828400 0x557bf09db600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7f4698f2d840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bf1828800 0x557bf09dbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7f4698f2d840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bf1828c00 0x557bf09dc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7f4698f2d840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf09b4640 con 0x557bf1828c00 2023-10-11T09:45:58.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7f4698f2d840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf09b4500 con 0x557bf1828800 2023-10-11T09:45:58.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7f4698f2d840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x557bf09b4a00 con 0x557bf1828400 2023-10-11T09:45:58.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7fae1b9bb840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5613ba10e400 0x5613b9ebf180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7fae1b9bb840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5613ba10e800 0x5613b9ebec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7fae1b9bb840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613ba10ec00 0x5613b9ebe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7fae1b9bb840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613b9e96280 con 0x5613ba10ec00 2023-10-11T09:45:58.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7fae1b9bb840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613b9e96140 con 0x5613ba10e400 2023-10-11T09:45:58.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7fae1b9bb840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x5613b9e96000 con 0x5613ba10e800 2023-10-11T09:45:58.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7f8f45a89700 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:8 9.2e 9:74000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb3dd400 con 0x5647eb543400 2023-10-11T09:45:58.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7f4696ee8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bf1828c00 0x557bf09dc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7f4696ee8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bf1828c00 0x557bf09dc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46168/0 (socket says 172.21.15.78:46168) 2023-10-11T09:45:58.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7f4696ee8700 1 -- 172.21.15.78:0/3355666413 learned_addr learned my addr 172.21.15.78:0/3355666413 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7fae1a177700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5613ba10e800 0x5613b9ebec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7fae1a177700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5613ba10e800 0x5613b9ebec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:53548/0 (socket says 172.21.15.78:53548) 2023-10-11T09:45:58.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7fae1a177700 1 -- 172.21.15.78:0/1365619850 learned_addr learned my addr 172.21.15.78:0/1365619850 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7fa8b4d81840 1 -- start start 2023-10-11T09:45:58.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7fa8b4d81840 1 Processor -- start 2023-10-11T09:45:58.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7fae19976700 1 --2- 172.21.15.78:0/1365619850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613ba10ec00 0x5613b9ebe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7fae19976700 1 --2- 172.21.15.78:0/1365619850 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5613ba10e400 0x5613b9ebf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7f4697eea700 1 --2- 172.21.15.78:0/3355666413 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x557bf1828400 0x557bf09db600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7f8f4b294700 1 -- 172.21.15.78:0/229716028 <== osd.7 v2:172.21.15.133:6824/35901 2 ==== osd_op_reply(8 [pgnls start_epoch 66 out=49b] v64'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5647ebf69200 con 0x5647eb543400 2023-10-11T09:45:58.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7f8f4628a700 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x5647eb3dd800 0x5647eb566000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7f8f4628a700 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:9 9.1e 9:78000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb542400 con 0x5647eb3dd800 2023-10-11T09:45:58.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7fb86dabf840 1 Processor -- start 2023-10-11T09:45:58.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7f46976e9700 1 --2- 172.21.15.78:0/3355666413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bf1828800 0x557bf09dbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7fae1a177700 1 -- 172.21.15.78:0/1365619850 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5613ba10e400 msgr2=0x5613b9ebf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7fae1a177700 1 --2- 172.21.15.78:0/1365619850 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5613ba10e400 0x5613b9ebf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7fae1a177700 1 -- 172.21.15.78:0/1365619850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613ba10ec00 msgr2=0x5613b9ebe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7fae1a177700 1 --2- 172.21.15.78:0/1365619850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613ba10ec00 0x5613b9ebe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7fae1a177700 1 -- 172.21.15.78:0/1365619850 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613b9e391e0 con 0x5613ba10e800 2023-10-11T09:45:58.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7f8f4aa93700 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x5647eb3dd800 0x5647eb566000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7fb86dabf840 1 -- start start 2023-10-11T09:45:58.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7fae1a177700 1 --2- 172.21.15.78:0/1365619850 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5613ba10e800 0x5613b9ebec00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x5613babf9620 tx=0x5613b9ef73e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7f4696ee8700 1 -- 172.21.15.78:0/3355666413 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x557bf1828400 msgr2=0x557bf09db600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7f4696ee8700 1 --2- 172.21.15.78:0/3355666413 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x557bf1828400 0x557bf09db600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7f4696ee8700 1 -- 172.21.15.78:0/3355666413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bf1828800 msgr2=0x557bf09dbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7f4696ee8700 1 --2- 172.21.15.78:0/3355666413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bf1828800 0x557bf09dbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7f4696ee8700 1 -- 172.21.15.78:0/3355666413 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bf0956000 con 0x557bf1828c00 2023-10-11T09:45:58.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7effacf1a840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597a1996400 0x5597a174f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7fae1416b700 1 -- 172.21.15.78:0/1365619850 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5613b9e471e0 con 0x5613ba10e800 2023-10-11T09:45:58.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7fae1416b700 1 -- 172.21.15.78:0/1365619850 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5613bac9f500 con 0x5613ba10e800 2023-10-11T09:45:58.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7fae1416b700 1 -- 172.21.15.78:0/1365619850 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5613b9e46dc0 con 0x5613ba10e800 2023-10-11T09:45:58.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7fb86dabf840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ab9a30400 0x562ab97d7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7fb86dabf840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ab9a30800 0x562ab97d7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7fb86dabf840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562ab9a30c00 0x562ab97d8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7fb86dabf840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ab9767e00 con 0x562ab9a30800 2023-10-11T09:45:58.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7fb86dabf840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ab97b0a00 con 0x562ab9a30400 2023-10-11T09:45:58.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7fb86dabf840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ab97b0780 con 0x562ab9a30c00 2023-10-11T09:45:58.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7effacf1a840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5597a1996800 0x5597a174ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7effacf1a840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5597a1996c00 0x5597a174e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7effabed7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597a1996400 0x5597a174f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7f4696ee8700 1 --2- 172.21.15.78:0/3355666413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bf1828c00 0x557bf09dc100 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x557bf17dc900 tx=0x557bf17e0c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee3220488138781 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7f15a83bb840 1 -- start start 2023-10-11T09:45:58.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5613b9e38ea0 con 0x5613ba10e800 2023-10-11T09:45:58.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5613b9e38b60 con 0x5613ba10e800 2023-10-11T09:45:58.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f15a83bb840 1 Processor -- start 2023-10-11T09:45:58.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.224+0000 7fae1b9bb840 4 datascan.init: resolving metadata pool 8 2023-10-11T09:45:58.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7fb86ba7a700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562ab9a30c00 0x562ab97d8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7fb86ba7a700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562ab9a30c00 0x562ab97d8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:53552/0 (socket says 172.21.15.78:53552) 2023-10-11T09:45:58.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7fb86ba7a700 1 -- 172.21.15.78:0/3373548805 learned_addr learned my addr 172.21.15.78:0/3373548805 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7fb86ca7c700 1 --2- 172.21.15.78:0/3373548805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ab9a30400 0x562ab97d7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7effacf1a840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597a1726500 con 0x5597a1996400 2023-10-11T09:45:58.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7effacf1a840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597a17263c0 con 0x5597a1996c00 2023-10-11T09:45:58.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7effacf1a840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x5597a1726280 con 0x5597a1996800 2023-10-11T09:45:58.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7effab6d6700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5597a1996800 0x5597a174ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7effab6d6700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5597a1996800 0x5597a174ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:53560/0 (socket says 172.21.15.78:53560) 2023-10-11T09:45:58.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7effab6d6700 1 -- 172.21.15.78:0/707681190 learned_addr learned my addr 172.21.15.78:0/707681190 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f8f4aa93700 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x5647eb3dd800 0x5647eb566000 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7effaaed5700 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5597a1996c00 0x5597a174e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7effaaed5700 1 -- 172.21.15.78:0/707681190 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5597a1996800 msgr2=0x5597a174ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7effaaed5700 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5597a1996800 0x5597a174ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7effaaed5700 1 -- 172.21.15.78:0/707681190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597a1996400 msgr2=0x5597a174f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7effaaed5700 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597a1996400 0x5597a174f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7effaaed5700 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597a16c8000 con 0x5597a1996c00 2023-10-11T09:45:58.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f8f4aa93700 1 -- 172.21.15.78:0/229716028 <== osd.0 v2:172.21.15.78:6800/35735 1 ==== osd_op_reply(9 [pgnls start_epoch 66 out=81b] v64'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5647ebf69440 con 0x5647eb3dd800 2023-10-11T09:45:58.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7fb86ca7c700 1 -- 172.21.15.78:0/3373548805 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562ab9a30c00 msgr2=0x562ab97d8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7fb86ca7c700 1 --2- 172.21.15.78:0/3373548805 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562ab9a30c00 0x562ab97d8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7fb86ca7c700 1 -- 172.21.15.78:0/3373548805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ab9a30800 msgr2=0x562ab97d7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7fb86ca7c700 1 --2- 172.21.15.78:0/3373548805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ab9a30800 0x562ab97d7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7fb86ca7c700 1 -- 172.21.15.78:0/3373548805 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ab9752340 con 0x562ab9a30400 2023-10-11T09:45:58.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7fb86ca7c700 1 --2- 172.21.15.78:0/3373548805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ab9a30400 0x562ab97d7600 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x562ab9a20720 tx=0x562ab9a42c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2ca972879ecc381 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:10 9.1e 9:789b801b:::1000000020c.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb543c00 con 0x5647eb3dd800 2023-10-11T09:45:58.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7fa8b4d81840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558cf3b75800 0x558cf3c1a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7effaaed5700 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5597a1996c00 0x5597a174e680 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x5597a19b9500 tx=0x5597a2486f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f4692ee0700 1 -- 172.21.15.78:0/3355666413 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557bf0965080 con 0x557bf1828c00 2023-10-11T09:45:58.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7fb867a72700 1 -- 172.21.15.78:0/3373548805 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562ab9761080 con 0x562ab9a30400 2023-10-11T09:45:58.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7fb867a72700 1 -- 172.21.15.78:0/3373548805 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562aba51e600 con 0x562ab9a30400 2023-10-11T09:45:58.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7fb867a72700 1 -- 172.21.15.78:0/3373548805 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562ab9761600 con 0x562ab9a30400 2023-10-11T09:45:58.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f4692ee0700 1 -- 172.21.15.78:0/3355666413 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557bf0980480 con 0x557bf1828c00 2023-10-11T09:45:58.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f4692ee0700 1 -- 172.21.15.78:0/3355666413 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557bf0965600 con 0x557bf1828c00 2023-10-11T09:45:58.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f4698f2d840 1 -- 172.21.15.78:0/3355666413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bf1828c00 msgr2=0x557bf09dc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f15a83bb840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556cb2477000 0x556cb251c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f15a83bb840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cb2477400 0x556cb251cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f15a83bb840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556cb2477800 0x556cb251d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f15a83bb840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cb24f6a00 con 0x556cb2477400 2023-10-11T09:45:58.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f15a83bb840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cb24f6780 con 0x556cb2477800 2023-10-11T09:45:58.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f15a83bb840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x556cb24f6640 con 0x556cb2477000 2023-10-11T09:45:58.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f4698f2d840 1 --2- 172.21.15.78:0/3355666413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bf1828c00 0x557bf09dc100 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x557bf17dc900 tx=0x557bf17e0c00 comp rx=0 tx=0).stop 2023-10-11T09:45:58.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f4698f2d840 1 -- 172.21.15.78:0/3355666413 shutdown_connections 2023-10-11T09:45:58.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f4698f2d840 1 --2- 172.21.15.78:0/3355666413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bf1828c00 0x557bf09dc100 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f4698f2d840 1 --2- 172.21.15.78:0/3355666413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bf1828800 0x557bf09dbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f4698f2d840 1 --2- 172.21.15.78:0/3355666413 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x557bf1828400 0x557bf09db600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f4698f2d840 1 -- 172.21.15.78:0/3355666413 >> 172.21.15.78:0/3355666413 conn(0x557bf1828000 msgr2=0x557bf0a15800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:58.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f15a6b77700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556cb2477000 0x556cb251c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f15a6b77700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556cb2477000 0x556cb251c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:53572/0 (socket says 172.21.15.78:53572) 2023-10-11T09:45:58.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f15a6b77700 1 -- 172.21.15.78:0/731713680 learned_addr learned my addr 172.21.15.78:0/731713680 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f8f4aa93700 1 -- 172.21.15.78:0/229716028 <== osd.0 v2:172.21.15.78:6800/35735 2 ==== osd_op_reply(10 1000000020c.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5647ebf69440 con 0x5647eb3dd800 2023-10-11T09:45:58.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f8f4c2d7840 10 datascan.operator(): handling object 1099511628300.0 2023-10-11T09:45:58.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:11 9.1e 9:789b801b:::1000000020c.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb542400 con 0x5647eb3dd800 2023-10-11T09:45:58.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7fa8b4d81840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cf3b75400 0x558cf3c1ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7fa8b4d81840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558cf3b75000 0x558cf3c1b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7fa8b4d81840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cf3babe00 con 0x558cf3b75400 2023-10-11T09:45:58.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7fa8b4d81840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cf3bf4a00 con 0x558cf3b75800 2023-10-11T09:45:58.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7fa8b4d81840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x558cf3bf4640 con 0x558cf3b75000 2023-10-11T09:45:58.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f15a6376700 1 --2- 172.21.15.78:0/731713680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cb2477400 0x556cb251cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f15a7378700 1 --2- 172.21.15.78:0/731713680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556cb2477800 0x556cb251d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7fb86dabf840 1 -- 172.21.15.78:0/3373548805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ab9a30400 msgr2=0x562ab97d7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7fb86dabf840 1 --2- 172.21.15.78:0/3373548805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ab9a30400 0x562ab97d7600 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x562ab9a20720 tx=0x562ab9a42c00 comp rx=0 tx=0).stop 2023-10-11T09:45:58.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7fa8b353d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558cf3b75800 0x558cf3c1a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7fa8b3d3e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558cf3b75000 0x558cf3c1b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f15a6376700 1 -- 172.21.15.78:0/731713680 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556cb2477000 msgr2=0x556cb251c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f15a6376700 1 --2- 172.21.15.78:0/731713680 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556cb2477000 0x556cb251c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f15a6376700 1 -- 172.21.15.78:0/731713680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556cb2477800 msgr2=0x556cb251d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f15a6376700 1 --2- 172.21.15.78:0/731713680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556cb2477800 0x556cb251d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f15a6376700 1 -- 172.21.15.78:0/731713680 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556cb2498b60 con 0x556cb2477400 2023-10-11T09:45:58.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f4698f2d840 1 -- 172.21.15.78:0/3355666413 shutdown_connections 2023-10-11T09:45:58.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f4698f2d840 1 -- 172.21.15.78:0/3355666413 wait complete. 2023-10-11T09:45:58.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7fa8b353d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558cf3b75800 0x558cf3c1a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:39018/0 (socket says 172.21.15.78:39018) 2023-10-11T09:45:58.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7fa8b353d700 1 -- 172.21.15.78:0/655546129 learned_addr learned my addr 172.21.15.78:0/655546129 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7effa56ca700 1 -- 172.21.15.78:0/707681190 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5597a16d7080 con 0x5597a1996c00 2023-10-11T09:45:58.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5597a16c8ea0 con 0x5597a1996c00 2023-10-11T09:45:58.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5597a16c9520 con 0x5597a1996c00 2023-10-11T09:45:58.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7effacf1a840 4 datascan.init: resolving metadata pool 8 2023-10-11T09:45:58.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7effa56ca700 1 -- 172.21.15.78:0/707681190 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5597a16f2180 con 0x5597a1996c00 2023-10-11T09:45:58.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f15a7378700 1 --2- 172.21.15.78:0/731713680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556cb2477800 0x556cb251d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:45:58.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7fae1416b700 1 -- 172.21.15.78:0/1365619850 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x5613b9e74840 con 0x5613ba10e800 2023-10-11T09:45:58.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f4698f2d840 1 Processor -- start 2023-10-11T09:45:58.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7fb86dabf840 1 -- 172.21.15.78:0/3373548805 shutdown_connections 2023-10-11T09:45:58.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7fb86dabf840 1 --2- 172.21.15.78:0/3373548805 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562ab9a30c00 0x562ab97d8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7fb86dabf840 1 --2- 172.21.15.78:0/3373548805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ab9a30800 0x562ab97d7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7fb86dabf840 1 --2- 172.21.15.78:0/3373548805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ab9a30400 0x562ab97d7600 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7fb86dabf840 1 -- 172.21.15.78:0/3373548805 >> 172.21.15.78:0/3373548805 conn(0x562ab9730c00 msgr2=0x562ab9825800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:58.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f15a6376700 1 --2- 172.21.15.78:0/731713680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cb2477400 0x556cb251cb00 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x556cb25d2a80 tx=0x556cb3266060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24cf308329b7b7ec server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7f4698f2d840 1 -- start start 2023-10-11T09:45:58.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7fae1416b700 1 --2- 172.21.15.78:0/1365619850 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5613ba256000 0x5613b9ebf700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7effa56ca700 1 -- 172.21.15.78:0/707681190 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5597a16d7600 con 0x5597a1996c00 2023-10-11T09:45:58.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.228+0000 7fb86dabf840 1 -- 172.21.15.78:0/3373548805 shutdown_connections 2023-10-11T09:45:58.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fb86dabf840 1 -- 172.21.15.78:0/3373548805 wait complete. 2023-10-11T09:45:58.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fae1416b700 1 -- 172.21.15.78:0/1365619850 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x5613b9e50a80 con 0x5613ba10e800 2023-10-11T09:45:58.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fae1b9bb840 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-10-11T09:45:58.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fae1b9bb840 4 datascan.main: data pool 'cephfs_data' has ID 9 2023-10-11T09:45:58.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fae1b9bb840 4 datascan.main: opening data pool 'cephfs_data' 2023-10-11T09:45:58.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fae1b9bb840 4 datascan.main: data pool 'cephfs_data_extra' has ID 10 2023-10-11T09:45:58.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fae1b9bb840 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-10-11T09:45:58.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fae1b9bb840 1 --2- 172.21.15.78:0/1365619850 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x5613ba10f000 0x5613b9ebe100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7f4698f2d840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bf1828400 0x557bf09dd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7f4698f2d840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bf1828800 0x557bf09dcc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7f4698f2d840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x557bf1828c00 0x557bf09dc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7f4698f2d840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf09b4780 con 0x557bf1828800 2023-10-11T09:45:58.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7f4698f2d840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf09b43c0 con 0x557bf1828400 2023-10-11T09:45:58.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7f4698f2d840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x557bf09b4280 con 0x557bf1828c00 2023-10-11T09:45:58.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fae19976700 1 --2- 172.21.15.78:0/1365619850 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5613ba256000 0x5613b9ebf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:1 9.12 9:49249249::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5613ba10f400 con 0x5613ba10f000 2023-10-11T09:45:58.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fa8b2d3c700 1 --2- 172.21.15.78:0/655546129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cf3b75400 0x558cf3c1ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fa8b3d3e700 1 -- 172.21.15.78:0/655546129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558cf3b75800 msgr2=0x558cf3c1a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fa8b3d3e700 1 --2- 172.21.15.78:0/655546129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558cf3b75800 0x558cf3c1a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fa8b3d3e700 1 -- 172.21.15.78:0/655546129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cf3b75400 msgr2=0x558cf3c1ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fa8b3d3e700 1 --2- 172.21.15.78:0/655546129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cf3b75400 0x558cf3c1ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fa8b3d3e700 1 -- 172.21.15.78:0/655546129 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cf3b96b60 con 0x558cf3b75000 2023-10-11T09:45:58.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7f15a4372700 1 -- 172.21.15.78:0/731713680 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556cb24a7080 con 0x556cb2477400 2023-10-11T09:45:58.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7f15a4372700 1 -- 172.21.15.78:0/731713680 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556cb24c2480 con 0x556cb2477400 2023-10-11T09:45:58.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7f15a4372700 1 -- 172.21.15.78:0/731713680 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556cb24a7600 con 0x556cb2477400 2023-10-11T09:45:58.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7f15a83bb840 1 -- 172.21.15.78:0/731713680 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556cb24991e0 con 0x556cb2477400 2023-10-11T09:45:58.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7f15a83bb840 1 -- 172.21.15.78:0/731713680 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x556cb2498b60 con 0x556cb2477400 2023-10-11T09:45:58.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7f15a83bb840 4 waiting for MDS map... 2023-10-11T09:45:58.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7f15a4372700 1 -- 172.21.15.78:0/731713680 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x556cb24b0c40 con 0x556cb2477400 2023-10-11T09:45:58.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7f15a4372700 1 -- 172.21.15.78:0/731713680 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 43) v1 ==== 3118+0+0 (secure 0 0 0) 0x556cb24b8f00 con 0x556cb2477400 2023-10-11T09:45:58.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7f15a83bb840 4 Got MDS map 43 2023-10-11T09:45:58.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fae19976700 1 --2- 172.21.15.78:0/1365619850 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x5613ba10f000 0x5613b9ebe100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7f46976e9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bf1828800 0x557bf09dcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7f46976e9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bf1828800 0x557bf09dcc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46220/0 (socket says 172.21.15.78:46220) 2023-10-11T09:45:58.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7f46976e9700 1 -- 172.21.15.78:0/1439616520 learned_addr learned my addr 172.21.15.78:0/1439616520 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7f15a83bb840 4 datascan.main: Using metadata pool output 2023-10-11T09:45:58.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7f15a83bb840 4 datascan.main: connecting to RADOS... 2023-10-11T09:45:58.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7f8f4aa93700 1 -- 172.21.15.78:0/229716028 <== osd.0 v2:172.21.15.78:6800/35735 3 ==== osd_op_reply(11 1000000020c.00000000 [call] v66'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5647ebf69440 con 0x5647eb3dd800 2023-10-11T09:45:58.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fb86dabf840 1 Processor -- start 2023-10-11T09:45:58.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7f4697eea700 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bf1828400 0x557bf09dd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7f4696ee8700 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x557bf1828c00 0x557bf09dc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7f8f4c2d7840 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5647eb543c00 0x5647eb18f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:12 9.3e 9:7c000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb56c000 con 0x5647eb543c00 2023-10-11T09:45:58.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fa8b3d3e700 1 --2- 172.21.15.78:0/655546129 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558cf3b75000 0x558cf3c1b080 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x558cf3c693b0 tx=0x558cf49fce40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ca78150df9a303c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fa8b0d38700 1 -- 172.21.15.78:0/655546129 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558cf3ba5600 con 0x558cf3b75000 2023-10-11T09:45:58.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fa8b4d81840 1 -- 172.21.15.78:0/655546129 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558cf3b96340 con 0x558cf3b75000 2023-10-11T09:45:58.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fa8b4d81840 1 -- 172.21.15.78:0/655546129 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({fsmap=0}) v3 -- 0x558cf3b96000 con 0x558cf3b75000 2023-10-11T09:45:58.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fa8b4d81840 4 waiting for MDS map... 2023-10-11T09:45:58.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fa8b0d38700 1 -- 172.21.15.78:0/655546129 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558cf3bc0900 con 0x558cf3b75000 2023-10-11T09:45:58.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fa8b0d38700 1 -- 172.21.15.78:0/655546129 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558cf3ba51e0 con 0x558cf3b75000 2023-10-11T09:45:58.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fb86dabf840 1 -- start start 2023-10-11T09:45:58.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7f8f4a292700 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5647eb543c00 0x5647eb18f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7effa56ca700 1 -- 172.21.15.78:0/707681190 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x5597a1704840 con 0x5597a1996c00 2023-10-11T09:45:58.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7f46976e9700 1 -- 172.21.15.78:0/1439616520 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x557bf1828c00 msgr2=0x557bf09dc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7f46976e9700 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x557bf1828c00 0x557bf09dc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7f46976e9700 1 -- 172.21.15.78:0/1439616520 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bf1828400 msgr2=0x557bf09dd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7f46976e9700 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bf1828400 0x557bf09dd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7f46976e9700 1 -- 172.21.15.78:0/1439616520 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bf0956ea0 con 0x557bf1828800 2023-10-11T09:45:58.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fae19976700 1 --2- 172.21.15.78:0/1365619850 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5613ba256000 0x5613b9ebf700 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x5613babf9830 tx=0x5613bac95b00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fae19976700 1 --2- 172.21.15.78:0/1365619850 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x5613ba10f000 0x5613b9ebe100 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7f15a83bb840 1 Processor -- start 2023-10-11T09:45:58.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7f15a83bb840 1 -- start start 2023-10-11T09:45:58.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7effa56ca700 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5597a1b00000 0x5597a174f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7f8f4a292700 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5647eb543c00 0x5647eb18f600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7f8f4a292700 1 -- 172.21.15.78:0/229716028 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(12 [pgnls start_epoch 66 out=49b] v64'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5647ebf68fc0 con 0x5647eb543c00 2023-10-11T09:45:58.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7f8f4628a700 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:13 9.1 9:80000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb542800 con 0x5647eb543c00 2023-10-11T09:45:58.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fa8b0d38700 1 -- 172.21.15.78:0/655546129 <== mon.2 v2:172.21.15.133:3301/0 4 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x558cf3baec40 con 0x558cf3b75000 2023-10-11T09:45:58.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fa8b0d38700 1 -- 172.21.15.78:0/655546129 <== mon.2 v2:172.21.15.133:3301/0 5 ==== fsmap(e 43) v1 ==== 3118+0+0 (secure 0 0 0) 0x558cf3e62000 con 0x558cf3b75000 2023-10-11T09:45:58.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fa8b4d81840 4 Got MDS map 43 2023-10-11T09:45:58.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fa8b4d81840 4 datascan.main: Using metadata pool output 2023-10-11T09:45:58.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fa8b4d81840 4 datascan.main: connecting to RADOS... 2023-10-11T09:45:58.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fb86dabf840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ab9a30400 0x562ab97d9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fb86dabf840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ab9a30800 0x562ab97d8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fb86dabf840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562ab9a30c00 0x562ab97d8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fb86dabf840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ab97b0500 con 0x562ab9a30800 2023-10-11T09:45:58.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fb86dabf840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ab97b03c0 con 0x562ab9a30400 2023-10-11T09:45:58.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7fb86dabf840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ab97b0280 con 0x562ab9a30c00 2023-10-11T09:45:58.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7effa56ca700 1 -- 172.21.15.78:0/707681190 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x5597a16e0a80 con 0x5597a1996c00 2023-10-11T09:45:58.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7effacf1a840 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-10-11T09:45:58.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7effacf1a840 4 datascan.main: data pool 'cephfs_data' has ID 9 2023-10-11T09:45:58.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7effacf1a840 4 datascan.main: opening data pool 'cephfs_data' 2023-10-11T09:45:58.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7effacf1a840 4 datascan.main: data pool 'cephfs_data_extra' has ID 10 2023-10-11T09:45:58.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7effacf1a840 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-10-11T09:45:58.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7effacf1a840 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x5597a1997000 0x5597a174e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:1 9.0 9:00000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1997400 con 0x5597a1997000 2023-10-11T09:45:58.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.232+0000 7f46976e9700 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bf1828800 0x557bf09dcc00 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x557bf17dd5f0 tx=0x557bf172c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7f46916dd700 1 -- 172.21.15.78:0/1439616520 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557bf09651e0 con 0x557bf1828800 2023-10-11T09:45:58.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7f4698f2d840 1 -- 172.21.15.78:0/1439616520 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557bf0957520 con 0x557bf1828800 2023-10-11T09:45:58.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fb86ba7a700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562ab9a30c00 0x562ab97d8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fb86ba7a700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562ab9a30c00 0x562ab97d8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:53600/0 (socket says 172.21.15.78:53600) 2023-10-11T09:45:58.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fb86ba7a700 1 -- 172.21.15.78:0/3073125939 learned_addr learned my addr 172.21.15.78:0/3073125939 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fb86c27b700 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ab9a30800 0x562ab97d8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fb86ca7c700 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ab9a30400 0x562ab97d9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fae19976700 1 -- 172.21.15.78:0/1365619850 <== osd.5 v2:172.21.15.133:6816/35899 1 ==== osd_op_reply(1 [pgnls start_epoch 66 out=49b] v64'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5613bac00fc0 con 0x5613ba10f000 2023-10-11T09:45:58.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fa8b4d81840 1 Processor -- start 2023-10-11T09:45:58.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fb86ba7a700 1 -- 172.21.15.78:0/3073125939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ab9a30400 msgr2=0x562ab97d9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fb86ba7a700 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ab9a30400 0x562ab97d9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7f4698f2d840 1 -- 172.21.15.78:0/1439616520 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557bf09576c0 con 0x557bf1828800 2023-10-11T09:45:58.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7f4698f2d840 4 datascan.init: resolving metadata pool 8 2023-10-11T09:45:58.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fb86ba7a700 1 -- 172.21.15.78:0/3073125939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ab9a30800 msgr2=0x562ab97d8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fb86ba7a700 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ab9a30800 0x562ab97d8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fb86ba7a700 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ab9752000 con 0x562ab9a30c00 2023-10-11T09:45:58.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fb86c27b700 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ab9a30800 0x562ab97d8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:45:58.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fae1516d700 1 --2- 172.21.15.78:0/1365619850 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x5613ba256400 0x5613ba27c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fae1516d700 1 -- 172.21.15.78:0/1365619850 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:2 9.32 9:4c000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5613ba256800 con 0x5613ba256400 2023-10-11T09:45:58.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7f15a83bb840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cb336a400 0x556cb251d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7f15a83bb840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556cb336a800 0x556cb251db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7f15a83bb840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556cb336ac00 0x556cb251e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7f15a83bb840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cb24ade00 con 0x556cb336a400 2023-10-11T09:45:58.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7f15a83bb840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cb24f6780 con 0x556cb336ac00 2023-10-11T09:45:58.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7f15a83bb840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x556cb24f6a00 con 0x556cb336a800 2023-10-11T09:45:58.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7effabed7700 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5597a1b00000 0x5597a174f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7f46916dd700 1 -- 172.21.15.78:0/1439616520 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557bf1744a80 con 0x557bf1828800 2023-10-11T09:45:58.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7f46916dd700 1 -- 172.21.15.78:0/1439616520 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557bf0964f20 con 0x557bf1828800 2023-10-11T09:45:58.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7f8f4a292700 1 -- 172.21.15.78:0/229716028 <== osd.3 v2:172.21.15.78:6824/35738 2 ==== osd_op_reply(13 [pgnls start_epoch 66 out=49b] v64'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5647ebf68fc0 con 0x5647eb543c00 2023-10-11T09:45:58.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7f8f45a89700 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x5647eb56c000 0x5647eb566580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7f8f45a89700 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:14 9.21 9:84000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb576000 con 0x5647eb56c000 2023-10-11T09:45:58.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fa8b4d81840 1 -- start start 2023-10-11T09:45:58.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7f15a7378700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cb336a400 0x556cb251d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7f15a7378700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cb336a400 0x556cb251d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46246/0 (socket says 172.21.15.78:46246) 2023-10-11T09:45:58.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7f15a7378700 1 -- 172.21.15.78:0/2475805629 learned_addr learned my addr 172.21.15.78:0/2475805629 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7f15a6b77700 1 --2- 172.21.15.78:0/2475805629 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556cb336a800 0x556cb251db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7effab6d6700 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x5597a1997000 0x5597a174e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7effabed7700 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5597a1b00000 0x5597a174f700 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x5597a19b8690 tx=0x5597a2486000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fae1a978700 1 --2- 172.21.15.78:0/1365619850 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x5613ba256400 0x5613ba27c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7f8f4b294700 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x5647eb56c000 0x5647eb566580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fa8b4d81840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558cf3e70400 0x558cf3c1b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fa8b4d81840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cf3e70800 0x558cf3c1bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fa8b4d81840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558cf3e70c00 0x558cf3c1c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fa8b4d81840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cf3bf4500 con 0x558cf3e70800 2023-10-11T09:45:58.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fa8b4d81840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cf3bf43c0 con 0x558cf3e70400 2023-10-11T09:45:58.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fa8b4d81840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x558cf3bf4a00 con 0x558cf3e70c00 2023-10-11T09:45:58.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7f15a6376700 1 --2- 172.21.15.78:0/2475805629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556cb336ac00 0x556cb251e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7f15a6376700 1 -- 172.21.15.78:0/2475805629 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556cb336a800 msgr2=0x556cb251db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7f15a6376700 1 --2- 172.21.15.78:0/2475805629 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556cb336a800 0x556cb251db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7f15a6376700 1 -- 172.21.15.78:0/2475805629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cb336a400 msgr2=0x556cb251d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7f15a6376700 1 --2- 172.21.15.78:0/2475805629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cb336a400 0x556cb251d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7f15a6376700 1 -- 172.21.15.78:0/2475805629 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556cb2498340 con 0x556cb336ac00 2023-10-11T09:45:58.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7f15a6376700 1 --2- 172.21.15.78:0/2475805629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556cb336ac00 0x556cb251e100 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x556cb32628a0 tx=0x556cb32660c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2557d91c3f7ebfb2 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7f8f4b294700 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x5647eb56c000 0x5647eb566580 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7effab6d6700 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x5597a1997000 0x5597a174e100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fb86ba7a700 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562ab9a30c00 0x562ab97d8680 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x562ab9a21410 tx=0x562ab9a42d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fae1a978700 1 --2- 172.21.15.78:0/1365619850 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x5613ba256400 0x5613ba27c000 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fa8b2d3c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558cf3e70400 0x558cf3c1b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fa8b2d3c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558cf3e70400 0x558cf3c1b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:39058/0 (socket says 172.21.15.78:39058) 2023-10-11T09:45:58.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fa8b2d3c700 1 -- 172.21.15.78:0/3233125051 learned_addr learned my addr 172.21.15.78:0/3233125051 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fa8b2d3c700 1 --2- 172.21.15.78:0/3233125051 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558cf3e70c00 0x558cf3c1c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7f8f4b294700 1 -- 172.21.15.78:0/229716028 <== osd.2 v2:172.21.15.78:6808/35732 1 ==== osd_op_reply(14 [pgnls start_epoch 66 out=49b] v64'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5647ebf69200 con 0x5647eb56c000 2023-10-11T09:45:58.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fb86626f700 1 -- 172.21.15.78:0/3073125939 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562ab9761080 con 0x562ab9a30c00 2023-10-11T09:45:58.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7f8f4628a700 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:15 9.11 9:88000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb542c00 con 0x5647eb543400 2023-10-11T09:45:58.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fb86626f700 1 -- 172.21.15.78:0/3073125939 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562ab977c180 con 0x562ab9a30c00 2023-10-11T09:45:58.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fb86626f700 1 -- 172.21.15.78:0/3073125939 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562ab9761600 con 0x562ab9a30c00 2023-10-11T09:45:58.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562ab9752ea0 con 0x562ab9a30c00 2023-10-11T09:45:58.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ab9753520 con 0x562ab9a30c00 2023-10-11T09:45:58.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fb86dabf840 4 datascan.init: resolving metadata pool 8 2023-10-11T09:45:58.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7f15a236e700 1 -- 172.21.15.78:0/2475805629 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556cb24a71e0 con 0x556cb336ac00 2023-10-11T09:45:58.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7f15a236e700 1 -- 172.21.15.78:0/2475805629 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556cb3274780 con 0x556cb336ac00 2023-10-11T09:45:58.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7f15a236e700 1 -- 172.21.15.78:0/2475805629 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556cb24a6f20 con 0x556cb336ac00 2023-10-11T09:45:58.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fa8b2d3c700 1 -- 172.21.15.78:0/3233125051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558cf3e70400 msgr2=0x558cf3c1b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fa8b2d3c700 1 --2- 172.21.15.78:0/3233125051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558cf3e70400 0x558cf3c1b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fa8b2d3c700 1 -- 172.21.15.78:0/3233125051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cf3e70800 msgr2=0x558cf3c1bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fa8b2d3c700 1 --2- 172.21.15.78:0/3233125051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cf3e70800 0x558cf3c1bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.236+0000 7fa8b2d3c700 1 -- 172.21.15.78:0/3233125051 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cf3b96ea0 con 0x558cf3e70c00 2023-10-11T09:45:58.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f46916dd700 1 -- 172.21.15.78:0/1439616520 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x557bf0992840 con 0x557bf1828800 2023-10-11T09:45:58.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7fae1a978700 1 -- 172.21.15.78:0/1365619850 <== osd.4 v2:172.21.15.133:6808/35898 1 ==== osd_op_reply(2 [pgnls start_epoch 66 out=49b] v64'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5613bac01200 con 0x5613ba256400 2023-10-11T09:45:58.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7fae1596e700 1 -- 172.21.15.78:0/1365619850 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:3 9.a 9:50000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5613ba257000 con 0x5613ba10f000 2023-10-11T09:45:58.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7effab6d6700 1 -- 172.21.15.78:0/707681190 <== osd.1 v2:172.21.15.78:6809/35734 1 ==== osd_op_reply(1 [pgnls start_epoch 66 out=81b] v64'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5597a2490fc0 con 0x5597a1997000 2023-10-11T09:45:58.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f8f4b294700 1 -- 172.21.15.78:0/229716028 <== osd.7 v2:172.21.15.133:6824/35901 3 ==== osd_op_reply(15 [pgnls start_epoch 66 out=49b] v64'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5647ebf69200 con 0x5647eb543400 2023-10-11T09:45:58.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f8f45a89700 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x5647eb576400 0x5647eb566b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f8f45a89700 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:16 9.31 9:8c000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb576800 con 0x5647eb576400 2023-10-11T09:45:58.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7fa8b2d3c700 1 --2- 172.21.15.78:0/3233125051 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558cf3e70c00 0x558cf3c1c100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x558cf495e8d0 tx=0x558cf3e82cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44ad561f0d2e0f01 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:2 9.0 9:012cdbc7:::10000000206.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1997800 con 0x5597a1997000 2023-10-11T09:45:58.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f15a83bb840 1 -- 172.21.15.78:0/2475805629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556cb336ac00 msgr2=0x556cb251e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f15a83bb840 1 --2- 172.21.15.78:0/2475805629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556cb336ac00 0x556cb251e100 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x556cb32628a0 tx=0x556cb32660c0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f15a83bb840 1 -- 172.21.15.78:0/2475805629 shutdown_connections 2023-10-11T09:45:58.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f15a83bb840 1 --2- 172.21.15.78:0/2475805629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556cb336ac00 0x556cb251e100 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f15a83bb840 1 --2- 172.21.15.78:0/2475805629 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556cb336a800 0x556cb251db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f15a83bb840 1 --2- 172.21.15.78:0/2475805629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cb336a400 0x556cb251d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f15a83bb840 1 -- 172.21.15.78:0/2475805629 >> 172.21.15.78:0/2475805629 conn(0x556cb336a000 msgr2=0x556cb2557800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:58.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7effab6d6700 1 -- 172.21.15.78:0/707681190 <== osd.1 v2:172.21.15.78:6809/35734 2 ==== osd_op_reply(2 10000000206.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5597a2490fc0 con 0x5597a1997000 2023-10-11T09:45:58.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7effacf1a840 10 datascan.operator(): handling object 1099511628294.0 2023-10-11T09:45:58.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f15a83bb840 1 -- 172.21.15.78:0/2475805629 shutdown_connections 2023-10-11T09:45:58.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:3 9.0 9:012cdbc7:::10000000206.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b00400 con 0x5597a1997000 2023-10-11T09:45:58.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7fae19976700 1 -- 172.21.15.78:0/1365619850 <== osd.5 v2:172.21.15.133:6816/35899 2 ==== osd_op_reply(3 [pgnls start_epoch 66 out=49b] v64'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5613bac00fc0 con 0x5613ba10f000 2023-10-11T09:45:58.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f15a83bb840 1 -- 172.21.15.78:0/2475805629 wait complete. 2023-10-11T09:45:58.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f8f4aa93700 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x5647eb576400 0x5647eb566b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7fae1516d700 1 --2- 172.21.15.78:0/1365619850 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5613ba256c00 0x5613ba27c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7fae1516d700 1 -- 172.21.15.78:0/1365619850 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:4 9.2a 9:54000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5613ba257000 con 0x5613ba256c00 2023-10-11T09:45:58.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7fa8aed34700 1 -- 172.21.15.78:0/3233125051 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558cf3ba5080 con 0x558cf3e70c00 2023-10-11T09:45:58.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f46916dd700 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x557bf1829800 0x557bf09dd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f46916dd700 1 -- 172.21.15.78:0/1439616520 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x557bf096ea80 con 0x557bf1828800 2023-10-11T09:45:58.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7fa8aed34700 1 -- 172.21.15.78:0/3233125051 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558cf3e60f00 con 0x558cf3e70c00 2023-10-11T09:45:58.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7fa8aed34700 1 -- 172.21.15.78:0/3233125051 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558cf3ba5600 con 0x558cf3e70c00 2023-10-11T09:45:58.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7fae1a177700 1 --2- 172.21.15.78:0/1365619850 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5613ba256c00 0x5613ba27c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f8f4aa93700 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x5647eb576400 0x5647eb566b00 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f4698f2d840 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-10-11T09:45:58.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f4698f2d840 4 datascan.main: data pool 'cephfs_data' has ID 9 2023-10-11T09:45:58.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f4698f2d840 4 datascan.main: opening data pool 'cephfs_data' 2023-10-11T09:45:58.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f4698f2d840 4 datascan.main: data pool 'cephfs_data_extra' has ID 10 2023-10-11T09:45:58.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f4698f2d840 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-10-11T09:45:58.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f4698f2d840 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x557bf1829c00 0x557bf09dc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f4698f2d840 1 -- 172.21.15.78:0/1439616520 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:1 9.24 9:24924924::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x557bf0ca8000 con 0x557bf1829c00 2023-10-11T09:45:58.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f4697eea700 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x557bf1829800 0x557bf09dd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f4696ee8700 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x557bf1829c00 0x557bf09dc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f15a83bb840 1 Processor -- start 2023-10-11T09:45:58.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7fa8b4d81840 1 -- 172.21.15.78:0/3233125051 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558cf3e70c00 msgr2=0x558cf3c1c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7fa8b4d81840 1 --2- 172.21.15.78:0/3233125051 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558cf3e70c00 0x558cf3c1c100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x558cf495e8d0 tx=0x558cf3e82cc0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7fa8b4d81840 1 -- 172.21.15.78:0/3233125051 shutdown_connections 2023-10-11T09:45:58.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7fa8b4d81840 1 --2- 172.21.15.78:0/3233125051 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558cf3e70c00 0x558cf3c1c100 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7fa8b4d81840 1 --2- 172.21.15.78:0/3233125051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cf3e70800 0x558cf3c1bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7fa8b4d81840 1 --2- 172.21.15.78:0/3233125051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558cf3e70400 0x558cf3c1b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7fae1a177700 1 --2- 172.21.15.78:0/1365619850 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5613ba256c00 0x5613ba27c580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7fa8b4d81840 1 -- 172.21.15.78:0/3233125051 >> 172.21.15.78:0/3233125051 conn(0x558cf3b74c00 msgr2=0x558cf3c31800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:58.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f15a83bb840 1 -- start start 2023-10-11T09:45:58.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7fa8b4d81840 1 -- 172.21.15.78:0/3233125051 shutdown_connections 2023-10-11T09:45:58.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f8f4aa93700 1 -- 172.21.15.78:0/229716028 <== osd.4 v2:172.21.15.133:6808/35898 1 ==== osd_op_reply(16 [pgnls start_epoch 66 out=49b] v64'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5647ebf69440 con 0x5647eb576400 2023-10-11T09:45:58.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7effab6d6700 1 -- 172.21.15.78:0/707681190 <== osd.1 v2:172.21.15.78:6809/35734 3 ==== osd_op_reply(3 10000000206.00000000 [call] v66'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5597a2490fc0 con 0x5597a1997000 2023-10-11T09:45:58.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7effacf1a840 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x5597a1b00800 0x5597a174db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:4 9.20 9:04000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b00c00 con 0x5597a1b00800 2023-10-11T09:45:58.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f4696ee8700 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x557bf1829c00 0x557bf09dc100 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f4697eea700 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x557bf1829800 0x557bf09dd700 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x557bf0a9a6f0 tx=0x557bf17e0e40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f4696ee8700 1 -- 172.21.15.78:0/1439616520 <== osd.2 v2:172.21.15.78:6808/35732 1 ==== osd_op_reply(1 [pgnls start_epoch 66 out=49b] v64'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557bf172afc0 con 0x557bf1829c00 2023-10-11T09:45:58.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f46926df700 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x557bf0ca9800 0x557bf0cb2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7fae1a177700 1 -- 172.21.15.78:0/1365619850 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(4 [pgnls start_epoch 66 out=101b] v64'11 uv11 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5613bac01440 con 0x5613ba256c00 2023-10-11T09:45:58.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:5 9.2a 9:55a6cd83:::1000000020a.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5613ba10f800 con 0x5613ba256c00 2023-10-11T09:45:58.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f8f4628a700 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:17 9.9 9:90000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb543000 con 0x5647eb3dd800 2023-10-11T09:45:58.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7f8f4aa93700 1 -- 172.21.15.78:0/229716028 <== osd.0 v2:172.21.15.78:6800/35735 4 ==== osd_op_reply(17 [pgnls start_epoch 66 out=49b] v64'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5647ebf69440 con 0x5647eb3dd800 2023-10-11T09:45:58.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7f8f4c2d7840 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x5647eb56c400 0x5647eb567600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:18 10.16 10:6db6db6d::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb56c800 con 0x5647eb56c400 2023-10-11T09:45:58.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7fa8b4d81840 1 -- 172.21.15.78:0/3233125051 wait complete. 2023-10-11T09:45:58.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7fa8b4d81840 1 Processor -- start 2023-10-11T09:45:58.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7fb86626f700 1 -- 172.21.15.78:0/3073125939 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x562ab978e840 con 0x562ab9a30c00 2023-10-11T09:45:58.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f15a83bb840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556cb336a400 0x556cb251f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f15a83bb840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556cb336a800 0x556cb251ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f15a83bb840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cb336ac00 0x556cb251e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f15a83bb840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cb24f6500 con 0x556cb336ac00 2023-10-11T09:45:58.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f15a83bb840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cb24f63c0 con 0x556cb336a400 2023-10-11T09:45:58.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f15a83bb840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x556cb24f6280 con 0x556cb336a800 2023-10-11T09:45:58.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7f15a7378700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556cb336a400 0x556cb251f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7f15a7378700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556cb336a400 0x556cb251f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:39070/0 (socket says 172.21.15.78:39070) 2023-10-11T09:45:58.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7f15a7378700 1 -- 172.21.15.78:0/1939471404 learned_addr learned my addr 172.21.15.78:0/1939471404 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.240+0000 7f46926df700 1 -- 172.21.15.78:0/1439616520 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:2 9.14 9:28000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x557bf0ca9c00 con 0x557bf0ca9800 2023-10-11T09:45:58.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fae1a177700 1 -- 172.21.15.78:0/1365619850 <== osd.3 v2:172.21.15.78:6824/35738 2 ==== osd_op_reply(5 1000000020a.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5613bac01440 con 0x5613ba256c00 2023-10-11T09:45:58.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fae1b9bb840 10 datascan.operator(): handling object 1099511628298.0 2023-10-11T09:45:58.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:6 9.2a 9:55a6cd83:::1000000020a.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5613ba257800 con 0x5613ba256c00 2023-10-11T09:45:58.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7effabed7700 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x5597a1b00800 0x5597a174db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7f4697eea700 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x557bf0ca9800 0x557bf0cb2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7f8f4a292700 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x5647eb56c400 0x5647eb567600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7f8f4a292700 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x5647eb56c400 0x5647eb567600 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fa8b4d81840 1 -- start start 2023-10-11T09:45:58.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fa8b4d81840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cf3e70400 0x558cf3c1d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fa8b4d81840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558cf3e70800 0x558cf3c1cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fa8b4d81840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558cf3e70c00 0x558cf3c1c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fa8b4d81840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cf3bf4280 con 0x558cf3e70400 2023-10-11T09:45:58.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fa8b4d81840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cf3bf4140 con 0x558cf3e70c00 2023-10-11T09:45:58.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fa8b2d3c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cf3e70400 0x558cf3c1d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fa8b4d81840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x558cf3bf4000 con 0x558cf3e70800 2023-10-11T09:45:58.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fa8b2d3c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cf3e70400 0x558cf3c1d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46284/0 (socket says 172.21.15.78:46284) 2023-10-11T09:45:58.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fa8b2d3c700 1 -- 172.21.15.78:0/4080171823 learned_addr learned my addr 172.21.15.78:0/4080171823 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fa8b353d700 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558cf3e70800 0x558cf3c1cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fa8b2d3c700 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558cf3e70c00 0x558cf3c1c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fb86626f700 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x562ab9ab0800 0x562ab97d9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fb86626f700 1 -- 172.21.15.78:0/3073125939 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x562ab976aa80 con 0x562ab9a30c00 2023-10-11T09:45:58.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fb86ca7c700 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x562ab9ab0800 0x562ab97d9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fb86ca7c700 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x562ab9ab0800 0x562ab97d9700 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x562ab9a20720 tx=0x562ab9807380 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fb86dabf840 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-10-11T09:45:58.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fb86dabf840 4 datascan.main: data pool 'cephfs_data' has ID 9 2023-10-11T09:45:58.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fb86dabf840 4 datascan.main: opening data pool 'cephfs_data' 2023-10-11T09:45:58.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fb86dabf840 4 datascan.main: data pool 'cephfs_data_extra' has ID 10 2023-10-11T09:45:58.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fb86dabf840 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-10-11T09:45:58.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fb86dabf840 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x562ab9a31000 0x562ab97d8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:1 9.2d 9:b6db6db6::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9a31400 con 0x562ab9a31000 2023-10-11T09:45:58.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7f15a6376700 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cb336ac00 0x556cb251e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7f15a6b77700 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556cb336a800 0x556cb251ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7f15a7378700 1 -- 172.21.15.78:0/1939471404 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556cb336a800 msgr2=0x556cb251ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7f15a7378700 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556cb336a800 0x556cb251ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7f15a7378700 1 -- 172.21.15.78:0/1939471404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cb336ac00 msgr2=0x556cb251e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7f15a7378700 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cb336ac00 0x556cb251e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7f15a7378700 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556cb24991e0 con 0x556cb336a400 2023-10-11T09:45:58.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7f15a6376700 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cb336ac00 0x556cb251e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:45:58.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7f15a7378700 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556cb336a400 0x556cb251f180 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x556cb3263710 tx=0x556cb3322e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7f15a0b6b700 1 -- 172.21.15.78:0/1939471404 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556cb24a71e0 con 0x556cb336a400 2023-10-11T09:45:58.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7f15a0b6b700 1 -- 172.21.15.78:0/1939471404 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556cb3275500 con 0x556cb336a400 2023-10-11T09:45:58.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7effabed7700 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x5597a1b00800 0x5597a174db80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7f4697eea700 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x557bf0ca9800 0x557bf0cb2000 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7effabed7700 1 -- 172.21.15.78:0/707681190 <== osd.0 v2:172.21.15.78:6800/35735 1 ==== osd_op_reply(4 [pgnls start_epoch 66 out=81b] v64'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5597a2491200 con 0x5597a1b00800 2023-10-11T09:45:58.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:5 9.20 9:05552577:::1000000020b.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b01000 con 0x5597a1b00800 2023-10-11T09:45:58.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fa8b2d3c700 1 -- 172.21.15.78:0/4080171823 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558cf3e70800 msgr2=0x558cf3c1cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fa8b2d3c700 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558cf3e70800 0x558cf3c1cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fa8b2d3c700 1 -- 172.21.15.78:0/4080171823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558cf3e70c00 msgr2=0x558cf3c1c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fa8b2d3c700 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558cf3e70c00 0x558cf3c1c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fa8b2d3c700 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cf3b971e0 con 0x558cf3e70400 2023-10-11T09:45:58.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fa8b2d3c700 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cf3e70400 0x558cf3c1d180 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x558cf3bcaf60 tx=0x558cf3e82de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7f8f4a292700 1 -- 172.21.15.78:0/229716028 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== osd_op_reply(18 [pgnls start_epoch 66 out=101b] v64'5 uv5 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5647ebf68fc0 con 0x5647eb56c400 2023-10-11T09:45:58.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:19 10.16 10:6ee5e2c4:::100000001f9.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb56cc00 con 0x5647eb56c400 2023-10-11T09:45:58.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fb86c27b700 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x562ab9a31000 0x562ab97d8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556cb2498b60 con 0x556cb336a400 2023-10-11T09:45:58.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556cb2498340 con 0x556cb336a400 2023-10-11T09:45:58.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7f15a83bb840 4 datascan.init: resolving metadata pool 8 2023-10-11T09:45:58.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7f15a0b6b700 1 -- 172.21.15.78:0/1939471404 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556cb24a6f20 con 0x556cb336a400 2023-10-11T09:45:58.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fa8ad531700 1 -- 172.21.15.78:0/4080171823 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558cf3ba51e0 con 0x558cf3e70400 2023-10-11T09:45:58.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fa8ad531700 1 -- 172.21.15.78:0/4080171823 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558cf3e61800 con 0x558cf3e70400 2023-10-11T09:45:58.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fa8ad531700 1 -- 172.21.15.78:0/4080171823 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558cf3ba4dc0 con 0x558cf3e70400 2023-10-11T09:45:58.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558cf3b96ea0 con 0x558cf3e70400 2023-10-11T09:45:58.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558cf3b96b60 con 0x558cf3e70400 2023-10-11T09:45:58.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7effabed7700 1 -- 172.21.15.78:0/707681190 <== osd.0 v2:172.21.15.78:6800/35735 2 ==== osd_op_reply(5 1000000020b.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5597a2491200 con 0x5597a1b00800 2023-10-11T09:45:58.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7effacf1a840 10 datascan.operator(): handling object 1099511628299.0 2023-10-11T09:45:58.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:6 9.20 9:05552577:::1000000020b.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b01400 con 0x5597a1b00800 2023-10-11T09:45:58.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fae1a177700 1 -- 172.21.15.78:0/1365619850 <== osd.3 v2:172.21.15.78:6824/35738 3 ==== osd_op_reply(6 1000000020a.00000000 [call] v66'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5613bac01440 con 0x5613ba256c00 2023-10-11T09:45:58.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7f4697eea700 1 -- 172.21.15.78:0/1439616520 <== osd.1 v2:172.21.15.78:6809/35734 1 ==== osd_op_reply(2 [pgnls start_epoch 66 out=49b] v64'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557bf172b200 con 0x557bf0ca9800 2023-10-11T09:45:58.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fa8b4d81840 4 datascan.init: resolving metadata pool 8 2023-10-11T09:45:58.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:7 9.2a 9:57fe7535::::head [pgnls start_epoch 66 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5613ba257c00 con 0x5613ba256c00 2023-10-11T09:45:58.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7f8f4a292700 1 -- 172.21.15.78:0/229716028 <== osd.6 v2:172.21.15.133:6800/35900 2 ==== osd_op_reply(19 100000001f9.00000000 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5647ebf68fc0 con 0x5647eb56c400 2023-10-11T09:45:58.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7f8f4c2d7840 10 datascan.operator(): handling object 1099511628281.0 2023-10-11T09:45:58.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:20 9.36 9:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb56d000 con 0x5647eb3dd000 2023-10-11T09:45:58.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7f4692ee0700 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x557bf0cba000 0x557bf0cb2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.244+0000 7f4692ee0700 1 -- 172.21.15.78:0/1439616520 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:3 9.34 9:2c000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x557bf0cba400 con 0x557bf0cba000 2023-10-11T09:45:58.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7f46976e9700 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x557bf0cba000 0x557bf0cb2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7fb86c27b700 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x562ab9a31000 0x562ab97d8100 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7fae1a177700 1 -- 172.21.15.78:0/1365619850 <== osd.3 v2:172.21.15.78:6824/35738 4 ==== osd_op_reply(7 [pgnls start_epoch 66 out=81b] v66'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5613bac01440 con 0x5613ba256c00 2023-10-11T09:45:58.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:8 9.2a 9:57fe7535:::100000001f6.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5613ba257000 con 0x5613ba256c00 2023-10-11T09:45:58.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7fb86c27b700 1 -- 172.21.15.78:0/3073125939 <== osd.2 v2:172.21.15.78:6808/35732 1 ==== osd_op_reply(1 [pgnls start_epoch 66 out=49b] v64'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562aba51cfc0 con 0x562ab9a31000 2023-10-11T09:45:58.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7fb867271700 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x562ab9ab0c00 0x562ab9ba4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7fb867271700 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:2 9.1d 9:b8000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9ab1000 con 0x562ab9ab0c00 2023-10-11T09:45:58.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7f46976e9700 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x557bf0cba000 0x557bf0cb2580 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7fae1a177700 1 -- 172.21.15.78:0/1365619850 <== osd.3 v2:172.21.15.78:6824/35738 5 ==== osd_op_reply(8 100000001f6.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5613bac01440 con 0x5613ba256c00 2023-10-11T09:45:58.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7fae1b9bb840 10 datascan.operator(): handling object 1099511628278.0 2023-10-11T09:45:58.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:9 9.2a 9:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5613ba257800 con 0x5613ba256c00 2023-10-11T09:45:58.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7f46976e9700 1 -- 172.21.15.78:0/1439616520 <== osd.5 v2:172.21.15.133:6816/35899 1 ==== osd_op_reply(3 [pgnls start_epoch 66 out=49b] v64'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557bf172b440 con 0x557bf0cba000 2023-10-11T09:45:58.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7f46926df700 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x557bf0cbac00 0x557bf0cb2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7f46926df700 1 -- 172.21.15.78:0/1439616520 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:4 9.c 9:30000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x557bf0cbb000 con 0x557bf0cbac00 2023-10-11T09:45:58.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7fb86ca7c700 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x562ab9ab0c00 0x562ab9ba4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7f15a0b6b700 1 -- 172.21.15.78:0/1939471404 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x556cb24d4840 con 0x556cb336a400 2023-10-11T09:45:58.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7effabed7700 1 -- 172.21.15.78:0/707681190 <== osd.0 v2:172.21.15.78:6800/35735 3 ==== osd_op_reply(6 1000000020b.00000000 [call] v66'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5597a2491200 con 0x5597a1b00800 2023-10-11T09:45:58.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7effacf1a840 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x5597a1b01800 0x5597a174d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:7 9.10 9:08000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b01c00 con 0x5597a1b01800 2023-10-11T09:45:58.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7effab6d6700 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x5597a1b01800 0x5597a174d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7f8f4a292700 1 -- 172.21.15.78:0/229716028 <== osd.1 v2:172.21.15.78:6809/35734 7 ==== osd_op_reply(20 100000001f9.00000000 [call,setxattr (8)] v66'20 uv20 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5647ebf68fc0 con 0x5647eb3dd000 2023-10-11T09:45:58.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:21 10.16 10:6fe38dc2::::head [pgnls start_epoch 66 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb56d400 con 0x5647eb56c400 2023-10-11T09:45:58.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7fa8ad531700 1 -- 172.21.15.78:0/4080171823 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x558cf3bd2840 con 0x558cf3e70400 2023-10-11T09:45:58.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7fa8ad531700 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x558cf3fbe000 0x558cf3c1d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7fa8ad531700 1 -- 172.21.15.78:0/4080171823 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x558cf3baea80 con 0x558cf3e70400 2023-10-11T09:45:58.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7fa8b4d81840 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-10-11T09:45:58.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7fa8b4d81840 4 datascan.main: data pool 'cephfs_data' has ID 9 2023-10-11T09:45:58.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7fa8b4d81840 4 datascan.main: opening data pool 'cephfs_data' 2023-10-11T09:45:58.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7fa8b4d81840 4 datascan.main: data pool 'cephfs_data_extra' has ID 10 2023-10-11T09:45:58.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7fa8b4d81840 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-10-11T09:45:58.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7fa8b4d81840 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x558cf3e71000 0x558cf3c1c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:1 9.9 9:92492492::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf3e71400 con 0x558cf3e71000 2023-10-11T09:45:58.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7fa8b353d700 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x558cf3fbe000 0x558cf3c1d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7fa8b2d3c700 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x558cf3e71000 0x558cf3c1c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7fb86ca7c700 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x562ab9ab0c00 0x562ab9ba4000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7fb86ca7c700 1 -- 172.21.15.78:0/3073125939 <== osd.0 v2:172.21.15.78:6800/35735 1 ==== osd_op_reply(2 [pgnls start_epoch 66 out=49b] v64'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562aba51d200 con 0x562ab9ab0c00 2023-10-11T09:45:58.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7fb867a72700 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x562ab9ab1800 0x562ab9ba4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7fb867a72700 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:3 9.3d 9:bc000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9ab1c00 con 0x562ab9ab1800 2023-10-11T09:45:58.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7f15a0b6b700 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x556cb336bc00 0x556cb251f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7f15a0b6b700 1 -- 172.21.15.78:0/1939471404 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x556cb24b0a80 con 0x556cb336a400 2023-10-11T09:45:58.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7f15a83bb840 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-10-11T09:45:58.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7f15a83bb840 4 datascan.main: data pool 'cephfs_data' has ID 9 2023-10-11T09:45:58.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7f15a83bb840 4 datascan.main: opening data pool 'cephfs_data' 2023-10-11T09:45:58.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7f15a83bb840 4 datascan.main: data pool 'cephfs_data_extra' has ID 10 2023-10-11T09:45:58.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7f15a83bb840 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-10-11T09:45:58.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7f15a83bb840 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x556cb27ec000 0x556cb251e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:1 9.1b 9:db6db6db::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb27ec400 con 0x556cb27ec000 2023-10-11T09:45:58.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7f15a6b77700 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x556cb336bc00 0x556cb251f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7f15a6376700 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x556cb27ec000 0x556cb251e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.248+0000 7f15a6376700 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x556cb27ec000 0x556cb251e100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7f4696ee8700 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x557bf0cbac00 0x557bf0cb2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7f15a6376700 1 -- 172.21.15.78:0/1939471404 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(1 [pgnls start_epoch 66 out=49b] v64'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556cb330efc0 con 0x556cb27ec000 2023-10-11T09:45:58.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7f15a1b6d700 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x556cb27ed400 0x556cb27f8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7f15a6b77700 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x556cb336bc00 0x556cb251f700 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x556cb25d3590 tx=0x556cb33232c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7f15a1b6d700 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:2 9.3b 9:dc000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb27ed800 con 0x556cb27ed400 2023-10-11T09:45:58.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7fa8b353d700 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x558cf3fbe000 0x558cf3c1d700 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x558cf3c69320 tx=0x558cf3e82ea0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7fa8b2d3c700 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x558cf3e71000 0x558cf3c1c100 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7f8f4a292700 1 -- 172.21.15.78:0/229716028 <== osd.6 v2:172.21.15.133:6800/35900 3 ==== osd_op_reply(21 [pgnls start_epoch 66 out=81b] v64'5 uv5 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5647ebf68fc0 con 0x5647eb56c400 2023-10-11T09:45:58.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:22 10.16 10:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb56d800 con 0x5647eb56c400 2023-10-11T09:45:58.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7fb86c27b700 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x562ab9ab1800 0x562ab9ba4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7fae1a177700 1 -- 172.21.15.78:0/1365619850 <== osd.3 v2:172.21.15.78:6824/35738 6 ==== osd_op_reply(9 100000001f6.00000000 [call] v66'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5613bac01440 con 0x5613ba256c00 2023-10-11T09:45:58.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:10 9.1a 9:58000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5613ba10f800 con 0x5613ba256c00 2023-10-11T09:45:58.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7f15a7378700 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x556cb27ed400 0x556cb27f8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7fa8b2d3c700 1 -- 172.21.15.78:0/4080171823 <== osd.0 v2:172.21.15.78:6800/35735 1 ==== osd_op_reply(1 [pgnls start_epoch 66 out=49b] v64'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558cf4944fc0 con 0x558cf3e71000 2023-10-11T09:45:58.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7f4696ee8700 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x557bf0cbac00 0x557bf0cb2b00 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7fa8ae533700 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x558cf3fbe400 0x558cf3fde000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7fa8ae533700 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:2 9.29 9:94000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf3fbe800 con 0x558cf3fbe400 2023-10-11T09:45:58.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7f8f4a292700 1 -- 172.21.15.78:0/229716028 <== osd.6 v2:172.21.15.133:6800/35900 4 ==== osd_op_reply(22 100000001f7.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5647ebf68fc0 con 0x5647eb56c400 2023-10-11T09:45:58.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7f8f4c2d7840 10 datascan.operator(): handling object 1099511628279.0 2023-10-11T09:45:58.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:23 9.36 9:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb56dc00 con 0x5647eb3dd000 2023-10-11T09:45:58.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7effab6d6700 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x5597a1b01800 0x5597a174d600 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7fae1a177700 1 -- 172.21.15.78:0/1365619850 <== osd.3 v2:172.21.15.78:6824/35738 7 ==== osd_op_reply(10 [pgnls start_epoch 66 out=49b] v64'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5613bac01440 con 0x5613ba256c00 2023-10-11T09:45:58.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7fae1596e700 1 --2- 172.21.15.78:0/1365619850 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x5613ba257c00 0x5613ba27d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7fae1596e700 1 -- 172.21.15.78:0/1365619850 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:11 9.3a 9:5c000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5613ba257000 con 0x5613ba257c00 2023-10-11T09:45:58.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7fb86c27b700 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x562ab9ab1800 0x562ab9ba4580 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7f15a7378700 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x556cb27ed400 0x556cb27f8000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7f4696ee8700 1 -- 172.21.15.78:0/1439616520 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(4 [pgnls start_epoch 66 out=81b] v64'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557bf172afc0 con 0x557bf0cbac00 2023-10-11T09:45:58.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7f4698f2d840 1 -- 172.21.15.78:0/1439616520 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:5 9.c 9:30604137:::100000001fc.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x557bf0ca8400 con 0x557bf0cbac00 2023-10-11T09:45:58.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7fae19976700 1 --2- 172.21.15.78:0/1365619850 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x5613ba257c00 0x5613ba27d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7fa8b3d3e700 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x558cf3fbe400 0x558cf3fde000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7f15a7378700 1 -- 172.21.15.78:0/1939471404 <== osd.2 v2:172.21.15.78:6808/35732 1 ==== osd_op_reply(2 [pgnls start_epoch 66 out=81b] v64'5 uv5 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556cb330f200 con 0x556cb27ed400 2023-10-11T09:45:58.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7fb86c27b700 1 -- 172.21.15.78:0/3073125939 <== osd.7 v2:172.21.15.133:6824/35901 1 ==== osd_op_reply(3 [pgnls start_epoch 66 out=81b] v64'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562aba51cfc0 con 0x562ab9ab1800 2023-10-11T09:45:58.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:3 9.3b 9:deccbd37:::100000001fe.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb27ec800 con 0x556cb27ed400 2023-10-11T09:45:58.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7effab6d6700 1 -- 172.21.15.78:0/707681190 <== osd.4 v2:172.21.15.133:6808/35898 1 ==== osd_op_reply(7 [pgnls start_epoch 66 out=81b] v64'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5597a2490fc0 con 0x5597a1b01800 2023-10-11T09:45:58.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:8 9.10 9:08b99fea:::100000001fd.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b00c00 con 0x5597a1b01800 2023-10-11T09:45:58.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7f4696ee8700 1 -- 172.21.15.78:0/1439616520 <== osd.3 v2:172.21.15.78:6824/35738 2 ==== osd_op_reply(5 100000001fc.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557bf172afc0 con 0x557bf0cbac00 2023-10-11T09:45:58.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7f4698f2d840 10 datascan.operator(): handling object 1099511628284.0 2023-10-11T09:45:58.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7f4698f2d840 1 -- 172.21.15.78:0/1439616520 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:6 9.c 9:30604137:::100000001fc.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x557bf0ca8800 con 0x557bf0cbac00 2023-10-11T09:45:58.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:4 9.3d 9:bfcd3380:::10000000205.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9a31800 con 0x562ab9ab1800 2023-10-11T09:45:58.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7f15a7378700 1 -- 172.21.15.78:0/1939471404 <== osd.2 v2:172.21.15.78:6808/35732 2 ==== osd_op_reply(3 100000001fe.00000000 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556cb330f200 con 0x556cb27ed400 2023-10-11T09:45:58.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7f15a83bb840 10 datascan.operator(): handling object 1099511628286.0 2023-10-11T09:45:58.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:4 9.3b 9:deccbd37:::100000001fe.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb27ecc00 con 0x556cb27ed400 2023-10-11T09:45:58.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7fb86c27b700 1 -- 172.21.15.78:0/3073125939 <== osd.7 v2:172.21.15.133:6824/35901 2 ==== osd_op_reply(4 10000000205.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562aba51cfc0 con 0x562ab9ab1800 2023-10-11T09:45:58.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7fb86dabf840 10 datascan.operator(): handling object 1099511628293.0 2023-10-11T09:45:58.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:5 9.3d 9:bfcd3380:::10000000205.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9a31400 con 0x562ab9ab1800 2023-10-11T09:45:58.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7fae19976700 1 --2- 172.21.15.78:0/1365619850 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x5613ba257c00 0x5613ba27d080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7fa8b3d3e700 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x558cf3fbe400 0x558cf3fde000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7effab6d6700 1 -- 172.21.15.78:0/707681190 <== osd.4 v2:172.21.15.133:6808/35898 2 ==== osd_op_reply(8 100000001fd.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5597a2490fc0 con 0x5597a1b01800 2023-10-11T09:45:58.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7effacf1a840 10 datascan.operator(): handling object 1099511628285.0 2023-10-11T09:45:58.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:9 9.10 9:08b99fea:::100000001fd.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b01000 con 0x5597a1b01800 2023-10-11T09:45:58.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7fae19976700 1 -- 172.21.15.78:0/1365619850 <== osd.0 v2:172.21.15.78:6800/35735 1 ==== osd_op_reply(11 [pgnls start_epoch 66 out=49b] v64'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5613bac00fc0 con 0x5613ba257c00 2023-10-11T09:45:58.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7fae1516d700 1 -- 172.21.15.78:0/1365619850 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:12 9.6 9:60000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5613ba257400 con 0x5613ba10f000 2023-10-11T09:45:58.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7fa8b3d3e700 1 -- 172.21.15.78:0/4080171823 <== osd.7 v2:172.21.15.133:6824/35901 1 ==== osd_op_reply(2 [pgnls start_epoch 66 out=49b] v64'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558cf4945200 con 0x558cf3fbe400 2023-10-11T09:45:58.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7fa8aed34700 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x558cf3fbf000 0x558cf3fde580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.252+0000 7fa8aed34700 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:3 9.19 9:98000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf3fbf400 con 0x558cf3fbf000 2023-10-11T09:45:58.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.256+0000 7fa8b353d700 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x558cf3fbf000 0x558cf3fde580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.256+0000 7f8f4a292700 1 -- 172.21.15.78:0/229716028 <== osd.1 v2:172.21.15.78:6809/35734 8 ==== osd_op_reply(23 100000001f7.00000000 [call,setxattr (8)] v66'21 uv21 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5647ebf68fc0 con 0x5647eb3dd000 2023-10-11T09:45:58.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.256+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:24 10.e 10:70000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb576800 con 0x5647eb576400 2023-10-11T09:45:58.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.256+0000 7fae19976700 1 -- 172.21.15.78:0/1365619850 <== osd.5 v2:172.21.15.133:6816/35899 3 ==== osd_op_reply(12 [pgnls start_epoch 66 out=49b] v64'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5613bac00fc0 con 0x5613ba10f000 2023-10-11T09:45:58.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.256+0000 7fae1596e700 1 -- 172.21.15.78:0/1365619850 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:13 9.26 9:64000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5613ba10f800 con 0x5613ba10f000 2023-10-11T09:45:58.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.256+0000 7fa8b353d700 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x558cf3fbf000 0x558cf3fde580 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.256+0000 7fa8b353d700 1 -- 172.21.15.78:0/4080171823 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(3 [pgnls start_epoch 66 out=101b] v64'12 uv12 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x558cf4945440 con 0x558cf3fbf000 2023-10-11T09:45:58.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.256+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:4 9.19 9:992a6246:::10000000200.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf3e71800 con 0x558cf3fbf000 2023-10-11T09:45:58.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.256+0000 7f8f4aa93700 1 -- 172.21.15.78:0/229716028 <== osd.4 v2:172.21.15.133:6808/35898 2 ==== osd_op_reply(24 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5647ebf69440 con 0x5647eb576400 2023-10-11T09:45:58.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.256+0000 7f8f4628a700 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:25 10.1e 10:78000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb576000 con 0x5647eb3dd800 2023-10-11T09:45:58.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.256+0000 7fa8b353d700 1 -- 172.21.15.78:0/4080171823 <== osd.3 v2:172.21.15.78:6824/35738 2 ==== osd_op_reply(4 10000000200.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x558cf4945440 con 0x558cf3fbf000 2023-10-11T09:45:58.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.256+0000 7fa8b4d81840 10 datascan.operator(): handling object 1099511628288.0 2023-10-11T09:45:58.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.256+0000 7fae19976700 1 -- 172.21.15.78:0/1365619850 <== osd.5 v2:172.21.15.133:6816/35899 4 ==== osd_op_reply(13 [pgnls start_epoch 66 out=81b] v64'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5613bac00fc0 con 0x5613ba10f000 2023-10-11T09:45:58.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.256+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:5 9.19 9:992a6246:::10000000200.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf3fbf400 con 0x558cf3fbf000 2023-10-11T09:45:58.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.256+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:14 9.26 9:658f2455:::100000001ff.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5613ba10f800 con 0x5613ba10f000 2023-10-11T09:45:58.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.256+0000 7f15a7378700 1 -- 172.21.15.78:0/1939471404 <== osd.2 v2:172.21.15.78:6808/35732 3 ==== osd_op_reply(4 100000001fe.00000000 [call] v66'6 uv6 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556cb330f200 con 0x556cb27ed400 2023-10-11T09:45:58.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.256+0000 7f15a83bb840 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x556cb27ed000 0x556cb251db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.256+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:5 9.7 9:e0000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb27ed800 con 0x556cb27ed000 2023-10-11T09:45:58.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.256+0000 7f8f4aa93700 1 -- 172.21.15.78:0/229716028 <== osd.0 v2:172.21.15.78:6800/35735 5 ==== osd_op_reply(25 [pgnls start_epoch 66 out=101b] v64'3 uv3 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5647ebf69440 con 0x5647eb3dd800 2023-10-11T09:45:58.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.256+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:26 10.1e 10:789b801b:::1000000020c.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb3dd400 con 0x5647eb3dd800 2023-10-11T09:45:58.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.256+0000 7f4696ee8700 1 -- 172.21.15.78:0/1439616520 <== osd.3 v2:172.21.15.78:6824/35738 3 ==== osd_op_reply(6 100000001fc.00000000 [call] v66'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x557bf172afc0 con 0x557bf0cbac00 2023-10-11T09:45:58.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.256+0000 7f4698f2d840 1 -- 172.21.15.78:0/1439616520 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:7 9.2c 9:34000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x557bf0ca8c00 con 0x557bf0cbac00 2023-10-11T09:45:58.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.256+0000 7f8f4aa93700 1 -- 172.21.15.78:0/229716028 <== osd.0 v2:172.21.15.78:6800/35735 6 ==== osd_op_reply(26 1000000020c.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5647ebf69440 con 0x5647eb3dd800 2023-10-11T09:45:58.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.256+0000 7f8f4c2d7840 10 datascan.operator(): handling object 1099511628300.0 2023-10-11T09:45:58.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.256+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:27 9.1e 9:789b801b:::1000000020c.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb542800 con 0x5647eb3dd800 2023-10-11T09:45:58.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.256+0000 7f15a6b77700 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x556cb27ed000 0x556cb251db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.256+0000 7f4696ee8700 1 -- 172.21.15.78:0/1439616520 <== osd.3 v2:172.21.15.78:6824/35738 4 ==== osd_op_reply(7 [pgnls start_epoch 66 out=81b] v64'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557bf172afc0 con 0x557bf0cbac00 2023-10-11T09:45:58.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.256+0000 7fae19976700 1 -- 172.21.15.78:0/1365619850 <== osd.5 v2:172.21.15.133:6816/35899 5 ==== osd_op_reply(14 100000001ff.00000000 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5613bac00fc0 con 0x5613ba10f000 2023-10-11T09:45:58.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.256+0000 7fae1b9bb840 10 datascan.operator(): handling object 1099511628287.0 2023-10-11T09:45:58.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.256+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:15 9.26 9:658f2455:::100000001ff.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5613ba257000 con 0x5613ba10f000 2023-10-11T09:45:58.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.256+0000 7f4698f2d840 1 -- 172.21.15.78:0/1439616520 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:8 9.2c 9:37389ade:::10000000209.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x557bf0ca9000 con 0x557bf0cbac00 2023-10-11T09:45:58.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.256+0000 7f4696ee8700 1 -- 172.21.15.78:0/1439616520 <== osd.3 v2:172.21.15.78:6824/35738 5 ==== osd_op_reply(8 10000000209.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557bf172afc0 con 0x557bf0cbac00 2023-10-11T09:45:58.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.256+0000 7f4698f2d840 10 datascan.operator(): handling object 1099511628297.0 2023-10-11T09:45:58.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.256+0000 7f4698f2d840 1 -- 172.21.15.78:0/1439616520 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:9 9.2c 9:37389ade:::10000000209.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x557bf0ca9400 con 0x557bf0cbac00 2023-10-11T09:45:58.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.256+0000 7f15a6b77700 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x556cb27ed000 0x556cb251db80 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7f15a6b77700 1 -- 172.21.15.78:0/1939471404 <== osd.5 v2:172.21.15.133:6816/35899 1 ==== osd_op_reply(5 [pgnls start_epoch 66 out=101b] v64'7 uv7 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x556cb330f440 con 0x556cb27ed000 2023-10-11T09:45:58.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:6 9.7 9:e15a914f:::10000000207.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb27ec800 con 0x556cb27ed000 2023-10-11T09:45:58.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7f4696ee8700 1 -- 172.21.15.78:0/1439616520 <== osd.3 v2:172.21.15.78:6824/35738 6 ==== osd_op_reply(9 10000000209.00000000 [call] v66'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x557bf172afc0 con 0x557bf0cbac00 2023-10-11T09:45:58.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7f4698f2d840 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x557bf0cbb000 0x557bf09dbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7f4698f2d840 1 -- 172.21.15.78:0/1439616520 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:10 9.1c 9:38000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x557bf0cbb800 con 0x557bf0cbb000 2023-10-11T09:45:58.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7f15a6b77700 1 -- 172.21.15.78:0/1939471404 <== osd.5 v2:172.21.15.133:6816/35899 2 ==== osd_op_reply(6 10000000207.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556cb330f440 con 0x556cb27ed000 2023-10-11T09:45:58.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7f15a83bb840 10 datascan.operator(): handling object 1099511628295.0 2023-10-11T09:45:58.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:7 9.7 9:e15a914f:::10000000207.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb27ecc00 con 0x556cb27ed000 2023-10-11T09:45:58.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7f4697eea700 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x557bf0cbb000 0x557bf09dbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7effab6d6700 1 -- 172.21.15.78:0/707681190 <== osd.4 v2:172.21.15.133:6808/35898 3 ==== osd_op_reply(9 100000001fd.00000000 [call] v66'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5597a2490fc0 con 0x5597a1b01800 2023-10-11T09:45:58.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7f4697eea700 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x557bf0cbb000 0x557bf09dbb80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:10 9.30 9:0c000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b00400 con 0x5597a1b00800 2023-10-11T09:45:58.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7fb86c27b700 1 -- 172.21.15.78:0/3073125939 <== osd.7 v2:172.21.15.133:6824/35901 3 ==== osd_op_reply(5 10000000205.00000000 [call] v66'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562aba51cfc0 con 0x562ab9ab1800 2023-10-11T09:45:58.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:6 9.3 9:c0000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9a31800 con 0x562ab9ab1800 2023-10-11T09:45:58.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7fa8b353d700 1 -- 172.21.15.78:0/4080171823 <== osd.3 v2:172.21.15.78:6824/35738 3 ==== osd_op_reply(5 10000000200.00000000 [call] v66'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x558cf4945440 con 0x558cf3fbf000 2023-10-11T09:45:58.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:6 9.19 9:997b7129::::head [pgnls start_epoch 66 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf3fbfc00 con 0x558cf3fbf000 2023-10-11T09:45:58.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7f4697eea700 1 -- 172.21.15.78:0/1439616520 <== osd.0 v2:172.21.15.78:6800/35735 1 ==== osd_op_reply(10 [pgnls start_epoch 66 out=49b] v64'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557bf172b200 con 0x557bf0cbb000 2023-10-11T09:45:58.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7f4692ee0700 1 -- 172.21.15.78:0/1439616520 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:11 9.3c 9:3c000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x557bf0cba800 con 0x557bf0cba000 2023-10-11T09:45:58.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7effabed7700 1 -- 172.21.15.78:0/707681190 <== osd.0 v2:172.21.15.78:6800/35735 4 ==== osd_op_reply(10 [pgnls start_epoch 66 out=49b] v64'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5597a2491200 con 0x5597a1b00800 2023-10-11T09:45:58.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7f8f4aa93700 1 -- 172.21.15.78:0/229716028 <== osd.0 v2:172.21.15.78:6800/35735 7 ==== osd_op_reply(27 1000000020c.00000000 [call,setxattr (8)] v66'8 uv8 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5647ebf69440 con 0x5647eb3dd800 2023-10-11T09:45:58.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7effa6ecd700 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5597a1b1c000 0x5597a1b22000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7effa6ecd700 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:11 9.8 9:10000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b1c400 con 0x5597a1b1c000 2023-10-11T09:45:58.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:28 10.1e 10:7ac44a97::::head [pgnls start_epoch 66 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb542400 con 0x5647eb3dd800 2023-10-11T09:45:58.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7fb86c27b700 1 -- 172.21.15.78:0/3073125939 <== osd.7 v2:172.21.15.133:6824/35901 4 ==== osd_op_reply(6 [pgnls start_epoch 66 out=49b] v64'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562aba51cfc0 con 0x562ab9ab1800 2023-10-11T09:45:58.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7fb867a72700 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x562ab9bb0000 0x562ab9ba4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7fb867a72700 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:7 9.23 9:c4000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9bb0400 con 0x562ab9bb0000 2023-10-11T09:45:58.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7effaaed5700 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5597a1b1c000 0x5597a1b22000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7f46976e9700 1 -- 172.21.15.78:0/1439616520 <== osd.5 v2:172.21.15.133:6816/35899 2 ==== osd_op_reply(11 [pgnls start_epoch 66 out=49b] v64'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557bf172b440 con 0x557bf0cba000 2023-10-11T09:45:58.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7fa8b353d700 1 -- 172.21.15.78:0/4080171823 <== osd.3 v2:172.21.15.78:6824/35738 4 ==== osd_op_reply(6 [pgnls start_epoch 66 out=81b] v66'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x558cf4945440 con 0x558cf3fbf000 2023-10-11T09:45:58.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:7 9.19 9:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf3e71800 con 0x558cf3fbf000 2023-10-11T09:45:58.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7f46926df700 1 -- 172.21.15.78:0/1439616520 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:12 9.2 9:40000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x557bf0cbb400 con 0x557bf0cbac00 2023-10-11T09:45:58.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7f8f4aa93700 1 -- 172.21.15.78:0/229716028 <== osd.0 v2:172.21.15.78:6800/35735 8 ==== osd_op_reply(28 [pgnls start_epoch 66 out=81b] v64'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5647ebf69440 con 0x5647eb3dd800 2023-10-11T09:45:58.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:29 10.1e 10:7ac44a97:::10000000207.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb543000 con 0x5647eb3dd800 2023-10-11T09:45:58.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7fb86ba7a700 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x562ab9bb0000 0x562ab9ba4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7fa8b353d700 1 -- 172.21.15.78:0/4080171823 <== osd.3 v2:172.21.15.78:6824/35738 5 ==== osd_op_reply(7 100000001f8.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x558cf4945440 con 0x558cf3fbf000 2023-10-11T09:45:58.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7fa8b4d81840 10 datascan.operator(): handling object 1099511628280.0 2023-10-11T09:45:58.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:8 9.19 9:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf3e71800 con 0x558cf3fbf000 2023-10-11T09:45:58.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7effaaed5700 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5597a1b1c000 0x5597a1b22000 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7f4696ee8700 1 -- 172.21.15.78:0/1439616520 <== osd.3 v2:172.21.15.78:6824/35738 7 ==== osd_op_reply(12 [pgnls start_epoch 66 out=49b] v64'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557bf172afc0 con 0x557bf0cbac00 2023-10-11T09:45:58.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7f4692ee0700 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x557bf0cbb800 0x557bf0cb3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7f4692ee0700 1 -- 172.21.15.78:0/1439616520 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:13 9.22 9:44000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x557bf0ca9c00 con 0x557bf0cbb800 2023-10-11T09:45:58.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7f8f4aa93700 1 -- 172.21.15.78:0/229716028 <== osd.0 v2:172.21.15.78:6800/35735 9 ==== osd_op_reply(29 10000000207.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5647ebf69440 con 0x5647eb3dd800 2023-10-11T09:45:58.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7f8f4c2d7840 10 datascan.operator(): handling object 1099511628295.1 2023-10-11T09:45:58.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7f8f4c2d7840 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x5647eb543800 0x5647eb567b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.260+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:30 9.7 9:e15a914f:::10000000207.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb56c800 con 0x5647eb543800 2023-10-11T09:45:58.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7fb86ba7a700 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x562ab9bb0000 0x562ab9ba4b00 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7f46976e9700 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x557bf0cbb800 0x557bf0cb3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7f8f4b294700 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x5647eb543800 0x5647eb567b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7effaaed5700 1 -- 172.21.15.78:0/707681190 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(11 [pgnls start_epoch 66 out=49b] v64'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5597a2491440 con 0x5597a1b1c000 2023-10-11T09:45:58.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7effa66cc700 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x5597a1b1cc00 0x5597a1b22580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7effa66cc700 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:12 9.28 9:14000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b1d000 con 0x5597a1b1cc00 2023-10-11T09:45:58.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7fae19976700 1 -- 172.21.15.78:0/1365619850 <== osd.5 v2:172.21.15.133:6816/35899 6 ==== osd_op_reply(15 100000001ff.00000000 [call] v66'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5613bac00fc0 con 0x5613ba10f000 2023-10-11T09:45:58.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:16 9.16 9:68000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5613ba257400 con 0x5613ba256400 2023-10-11T09:45:58.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7fb86ba7a700 1 -- 172.21.15.78:0/3073125939 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(7 [pgnls start_epoch 66 out=49b] v64'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562aba51d440 con 0x562ab9bb0000 2023-10-11T09:45:58.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7fb867271700 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:8 9.13 9:c8000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9ab1400 con 0x562ab9ab1800 2023-10-11T09:45:58.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7effabed7700 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x5597a1b1cc00 0x5597a1b22580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7f46976e9700 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x557bf0cbb800 0x557bf0cb3600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7f8f4b294700 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x5647eb543800 0x5647eb567b80 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7fae1a978700 1 -- 172.21.15.78:0/1365619850 <== osd.4 v2:172.21.15.133:6808/35898 2 ==== osd_op_reply(16 [pgnls start_epoch 66 out=49b] v64'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5613bac01200 con 0x5613ba256400 2023-10-11T09:45:58.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7fae1596e700 1 --2- 172.21.15.78:0/1365619850 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x5613ba257800 0x5613ba27d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7fae1596e700 1 -- 172.21.15.78:0/1365619850 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:17 9.36 9:6c000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5613ba298000 con 0x5613ba257800 2023-10-11T09:45:58.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7f15a6b77700 1 -- 172.21.15.78:0/1939471404 <== osd.5 v2:172.21.15.133:6816/35899 3 ==== osd_op_reply(7 10000000207.00000000 [call] v66'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556cb330f440 con 0x556cb27ed000 2023-10-11T09:45:58.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:8 9.7 9:e2e07930::::head [pgnls start_epoch 66 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb27ed800 con 0x556cb27ed000 2023-10-11T09:45:58.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7f46976e9700 1 -- 172.21.15.78:0/1439616520 <== osd.7 v2:172.21.15.133:6824/35901 1 ==== osd_op_reply(13 [pgnls start_epoch 66 out=49b] v64'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557bf172b440 con 0x557bf0cbb800 2023-10-11T09:45:58.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7fae1a978700 1 --2- 172.21.15.78:0/1365619850 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x5613ba257800 0x5613ba27d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7effabed7700 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x5597a1b1cc00 0x5597a1b22580 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7f46926df700 1 -- 172.21.15.78:0/1439616520 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:14 9.12 9:48000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x557bf0ca9c00 con 0x557bf0cba000 2023-10-11T09:45:58.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7fb86c27b700 1 -- 172.21.15.78:0/3073125939 <== osd.7 v2:172.21.15.133:6824/35901 5 ==== osd_op_reply(8 [pgnls start_epoch 66 out=81b] v64'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562aba51cfc0 con 0x562ab9ab1800 2023-10-11T09:45:58.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7fa8b353d700 1 -- 172.21.15.78:0/4080171823 <== osd.3 v2:172.21.15.78:6824/35738 6 ==== osd_op_reply(8 100000001f8.00000000 [call] v66'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x558cf4945440 con 0x558cf3fbf000 2023-10-11T09:45:58.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7fa8b4d81840 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x558cf3fbf400 0x558cf3c1bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:9 9.13 9:ca8fdbba:::100000001fb.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9ab0400 con 0x562ab9ab1800 2023-10-11T09:45:58.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:9 9.39 9:9c000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf3fbfc00 con 0x558cf3fbf400 2023-10-11T09:45:58.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7f46976e9700 1 -- 172.21.15.78:0/1439616520 <== osd.5 v2:172.21.15.133:6816/35899 3 ==== osd_op_reply(14 [pgnls start_epoch 66 out=49b] v64'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557bf172b440 con 0x557bf0cba000 2023-10-11T09:45:58.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7fae1a978700 1 --2- 172.21.15.78:0/1365619850 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x5613ba257800 0x5613ba27d600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7f4698f2d840 1 -- 172.21.15.78:0/1439616520 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:15 10.4 10:24924924::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x557bf0cbbc00 con 0x557bf0cbb000 2023-10-11T09:45:58.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7f15a6b77700 1 -- 172.21.15.78:0/1939471404 <== osd.5 v2:172.21.15.133:6816/35899 4 ==== osd_op_reply(8 [pgnls start_epoch 66 out=81b] v66'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556cb330f440 con 0x556cb27ed000 2023-10-11T09:45:58.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:9 9.7 9:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb27ec800 con 0x556cb27ed000 2023-10-11T09:45:58.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7fb86c27b700 1 -- 172.21.15.78:0/3073125939 <== osd.7 v2:172.21.15.133:6824/35901 6 ==== osd_op_reply(9 100000001fb.00000000 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562aba51cfc0 con 0x562ab9ab1800 2023-10-11T09:45:58.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7fb86dabf840 10 datascan.operator(): handling object 1099511628283.0 2023-10-11T09:45:58.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7effabed7700 1 -- 172.21.15.78:0/707681190 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== osd_op_reply(12 [pgnls start_epoch 66 out=101b] v64'8 uv8 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5597a2491200 con 0x5597a1b1cc00 2023-10-11T09:45:58.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:10 9.13 9:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9ab1c00 con 0x562ab9ab1800 2023-10-11T09:45:58.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:13 9.28 9:15246602:::10000000202.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1997400 con 0x5597a1b1cc00 2023-10-11T09:45:58.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7fa8b2d3c700 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x558cf3fbf400 0x558cf3c1bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7f4697eea700 1 -- 172.21.15.78:0/1439616520 <== osd.0 v2:172.21.15.78:6800/35735 2 ==== osd_op_reply(15 [pgnls start_epoch 66 out=81b] v64'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557bf172b200 con 0x557bf0cbb000 2023-10-11T09:45:58.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7f15a6b77700 1 -- 172.21.15.78:0/1939471404 <== osd.5 v2:172.21.15.133:6816/35899 5 ==== osd_op_reply(9 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556cb330f440 con 0x556cb27ed000 2023-10-11T09:45:58.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7f15a83bb840 10 datascan.operator(): handling object 1099511627777.0 2023-10-11T09:45:58.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:10 9.7 9:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb27ecc00 con 0x556cb27ed000 2023-10-11T09:45:58.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7f4698f2d840 1 -- 172.21.15.78:0/1439616520 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:16 10.4 10:24ae92ec:::100000001fb.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x557bf0ca8400 con 0x557bf0cbb000 2023-10-11T09:45:58.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7fae1a978700 1 -- 172.21.15.78:0/1365619850 <== osd.1 v2:172.21.15.78:6809/35734 1 ==== osd_op_reply(17 [pgnls start_epoch 66 out=101b] v66'21 uv21 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5613bac01200 con 0x5613ba257800 2023-10-11T09:45:58.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7fae1b9bb840 1 --2- 172.21.15.78:0/1365619850 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x5613ba298000 0x5613b9ebdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7effabed7700 1 -- 172.21.15.78:0/707681190 <== osd.6 v2:172.21.15.133:6800/35900 2 ==== osd_op_reply(13 10000000202.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5597a2491200 con 0x5597a1b1cc00 2023-10-11T09:45:58.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7effacf1a840 10 datascan.operator(): handling object 1099511628290.0 2023-10-11T09:45:58.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:14 9.28 9:15246602:::10000000202.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1997800 con 0x5597a1b1cc00 2023-10-11T09:45:58.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:18 10.12 10:49249249::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5613ba299000 con 0x5613ba298000 2023-10-11T09:45:58.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7f4697eea700 1 -- 172.21.15.78:0/1439616520 <== osd.0 v2:172.21.15.78:6800/35735 3 ==== osd_op_reply(16 100000001fb.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557bf172b200 con 0x557bf0cbb000 2023-10-11T09:45:58.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7f4698f2d840 10 datascan.operator(): handling object 1099511628283.1 2023-10-11T09:45:58.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7f4698f2d840 1 -- 172.21.15.78:0/1439616520 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:17 9.13 9:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x557bf0ca8800 con 0x557bf0cbb800 2023-10-11T09:45:58.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7fae1a177700 1 --2- 172.21.15.78:0/1365619850 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x5613ba298000 0x5613b9ebdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.264+0000 7fa8b2d3c700 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x558cf3fbf400 0x558cf3c1bb80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.268+0000 7f8f4b294700 1 -- 172.21.15.78:0/229716028 <== osd.5 v2:172.21.15.133:6816/35899 1 ==== osd_op_reply(30 10000000207.00000000 [call,setxattr (8)] v66'9 uv9 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5647ebf69200 con 0x5647eb543800 2023-10-11T09:45:58.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.268+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:31 10.1 10:80000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb56cc00 con 0x5647eb576400 2023-10-11T09:45:58.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.268+0000 7fa8b2d3c700 1 -- 172.21.15.78:0/4080171823 <== osd.4 v2:172.21.15.133:6808/35898 1 ==== osd_op_reply(9 [pgnls start_epoch 66 out=49b] v64'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558cf4944fc0 con 0x558cf3fbf400 2023-10-11T09:45:58.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.268+0000 7fae1a177700 1 --2- 172.21.15.78:0/1365619850 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x5613ba298000 0x5613b9ebdb80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.268+0000 7fa8ae533700 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:10 9.5 9:a0000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf3fbec00 con 0x558cf3e71000 2023-10-11T09:45:58.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.268+0000 7fae1a177700 1 -- 172.21.15.78:0/1365619850 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== osd_op_reply(18 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5613bac01440 con 0x5613ba298000 2023-10-11T09:45:58.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.268+0000 7f8f4aa93700 1 -- 172.21.15.78:0/229716028 <== osd.4 v2:172.21.15.133:6808/35898 3 ==== osd_op_reply(31 [pgnls start_epoch 66 out=101b] v64'2 uv2 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5647ebf69440 con 0x5647eb576400 2023-10-11T09:45:58.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.268+0000 7fae1596e700 1 -- 172.21.15.78:0/1365619850 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:19 10.a 10:50000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5613ba298400 con 0x5613ba298000 2023-10-11T09:45:58.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.268+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:32 10.1 10:8410d73e:::10000000201.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb56d000 con 0x5647eb576400 2023-10-11T09:45:58.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.268+0000 7fa8b2d3c700 1 -- 172.21.15.78:0/4080171823 <== osd.0 v2:172.21.15.78:6800/35735 2 ==== osd_op_reply(10 [pgnls start_epoch 66 out=81b] v64'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x558cf4944fc0 con 0x558cf3e71000 2023-10-11T09:45:58.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.268+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:11 9.5 9:a0beda95:::10000000204.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf3fbfc00 con 0x558cf3e71000 2023-10-11T09:45:58.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.268+0000 7fa8b2d3c700 1 -- 172.21.15.78:0/4080171823 <== osd.0 v2:172.21.15.78:6800/35735 3 ==== osd_op_reply(11 10000000204.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x558cf4944fc0 con 0x558cf3e71000 2023-10-11T09:45:58.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.268+0000 7fa8b4d81840 10 datascan.operator(): handling object 1099511628292.0 2023-10-11T09:45:58.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.268+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:12 9.5 9:a0beda95:::10000000204.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf3fbec00 con 0x558cf3e71000 2023-10-11T09:45:58.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.268+0000 7f8f4aa93700 1 -- 172.21.15.78:0/229716028 <== osd.4 v2:172.21.15.133:6808/35898 4 ==== osd_op_reply(32 10000000201.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5647ebf69440 con 0x5647eb576400 2023-10-11T09:45:58.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.268+0000 7f8f4c2d7840 10 datascan.operator(): handling object 1099511628289.1 2023-10-11T09:45:58.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.268+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:33 9.28 9:177a762d:::10000000201.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb56d400 con 0x5647eb56c400 2023-10-11T09:45:58.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.268+0000 7fae1a177700 1 -- 172.21.15.78:0/1365619850 <== osd.6 v2:172.21.15.133:6800/35900 2 ==== osd_op_reply(19 [pgnls start_epoch 66 out=101b] v64'5 uv5 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5613bac01440 con 0x5613ba298000 2023-10-11T09:45:58.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.268+0000 7effabed7700 1 -- 172.21.15.78:0/707681190 <== osd.6 v2:172.21.15.133:6800/35900 3 ==== osd_op_reply(14 10000000202.00000000 [call] v66'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5597a2491200 con 0x5597a1b1cc00 2023-10-11T09:45:58.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.268+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:15 9.28 9:177a762d::::head [pgnls start_epoch 66 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b1d000 con 0x5597a1b1cc00 2023-10-11T09:45:58.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.268+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:20 10.a 10:5097359c:::100000001fd.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5613ba299400 con 0x5613ba298000 2023-10-11T09:45:58.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.268+0000 7fae1a177700 1 -- 172.21.15.78:0/1365619850 <== osd.6 v2:172.21.15.133:6800/35900 3 ==== osd_op_reply(20 100000001fd.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5613bac01440 con 0x5613ba298000 2023-10-11T09:45:58.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.268+0000 7fae1b9bb840 10 datascan.operator(): handling object 1099511628285.1 2023-10-11T09:45:58.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.268+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:21 9.10 9:08b99fea:::100000001fd.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5613ba299800 con 0x5613ba256400 2023-10-11T09:45:58.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.268+0000 7fb86c27b700 1 -- 172.21.15.78:0/3073125939 <== osd.7 v2:172.21.15.133:6824/35901 7 ==== osd_op_reply(10 100000001fb.00000000 [call] v66'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562aba51cfc0 con 0x562ab9ab1800 2023-10-11T09:45:58.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.268+0000 7fb86dabf840 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x562ab9bb0c00 0x562ab97d7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.268+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:11 9.33 9:cc000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9bb1000 con 0x562ab9bb0c00 2023-10-11T09:45:58.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.268+0000 7effabed7700 1 -- 172.21.15.78:0/707681190 <== osd.6 v2:172.21.15.133:6800/35900 4 ==== osd_op_reply(15 [pgnls start_epoch 66 out=81b] v66'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5597a2491200 con 0x5597a1b1cc00 2023-10-11T09:45:58.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.268+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:16 9.28 9:177a762d:::10000000201.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b1d800 con 0x5597a1b1cc00 2023-10-11T09:45:58.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.268+0000 7fb86ca7c700 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x562ab9bb0c00 0x562ab97d7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.272+0000 7fb86ca7c700 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x562ab9bb0c00 0x562ab97d7b80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.272+0000 7f8f4a292700 1 -- 172.21.15.78:0/229716028 <== osd.6 v2:172.21.15.133:6800/35900 5 ==== osd_op_reply(33 10000000201.00000000 [call,setxattr (8)] v66'10 uv10 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5647ebf68fc0 con 0x5647eb56c400 2023-10-11T09:45:58.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.272+0000 7fb86ca7c700 1 -- 172.21.15.78:0/3073125939 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== osd_op_reply(11 [pgnls start_epoch 66 out=49b] v64'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562aba51d200 con 0x562ab9bb0c00 2023-10-11T09:45:58.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.272+0000 7fb867271700 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:12 9.b 9:d0000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9bb1000 con 0x562ab9ab0c00 2023-10-11T09:45:58.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.272+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:34 10.1 10:848ebb0b::::head [pgnls start_epoch 66 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb576800 con 0x5647eb576400 2023-10-11T09:45:58.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.272+0000 7effabed7700 1 -- 172.21.15.78:0/707681190 <== osd.6 v2:172.21.15.133:6800/35900 5 ==== osd_op_reply(16 10000000201.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5597a2491200 con 0x5597a1b1cc00 2023-10-11T09:45:58.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.272+0000 7effacf1a840 10 datascan.operator(): handling object 1099511628289.0 2023-10-11T09:45:58.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.272+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:17 9.28 9:177a762d:::10000000201.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b1dc00 con 0x5597a1b1cc00 2023-10-11T09:45:58.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.272+0000 7f15a6b77700 1 -- 172.21.15.78:0/1939471404 <== osd.5 v2:172.21.15.133:6816/35899 6 ==== osd_op_reply(10 10000000001.00000000 [call] v66'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556cb330f440 con 0x556cb27ed000 2023-10-11T09:45:58.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.272+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:11 9.27 9:e4000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb27ed800 con 0x556cb27ed400 2023-10-11T09:45:58.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.272+0000 7fa8b2d3c700 1 -- 172.21.15.78:0/4080171823 <== osd.0 v2:172.21.15.78:6800/35735 4 ==== osd_op_reply(12 10000000204.00000000 [call] v66'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x558cf4944fc0 con 0x558cf3e71000 2023-10-11T09:45:58.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.272+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:13 9.25 9:a4000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf3e71400 con 0x558cf3e71000 2023-10-11T09:45:58.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.272+0000 7f8f4aa93700 1 -- 172.21.15.78:0/229716028 <== osd.4 v2:172.21.15.133:6808/35898 5 ==== osd_op_reply(34 [pgnls start_epoch 66 out=81b] v64'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5647ebf69440 con 0x5647eb576400 2023-10-11T09:45:58.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.272+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:35 10.1 10:848ebb0b:::100000001f7.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb576000 con 0x5647eb576400 2023-10-11T09:45:58.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.272+0000 7fb86ca7c700 1 -- 172.21.15.78:0/3073125939 <== osd.0 v2:172.21.15.78:6800/35735 2 ==== osd_op_reply(12 [pgnls start_epoch 66 out=49b] v64'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562aba51d200 con 0x562ab9ab0c00 2023-10-11T09:45:58.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.272+0000 7fb867a72700 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:13 9.2b 9:d4000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9bb0800 con 0x562ab9ab1800 2023-10-11T09:45:58.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.272+0000 7f8f4aa93700 1 -- 172.21.15.78:0/229716028 <== osd.4 v2:172.21.15.133:6808/35898 6 ==== osd_op_reply(35 100000001f7.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5647ebf69440 con 0x5647eb576400 2023-10-11T09:45:58.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.272+0000 7f8f4c2d7840 10 datascan.operator(): handling object 1099511628279.1 2023-10-11T09:45:58.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.272+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:36 9.36 9:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb577800 con 0x5647eb3dd000 2023-10-11T09:45:58.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.272+0000 7fa8b2d3c700 1 -- 172.21.15.78:0/4080171823 <== osd.0 v2:172.21.15.78:6800/35735 5 ==== osd_op_reply(13 [pgnls start_epoch 66 out=49b] v64'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558cf4944fc0 con 0x558cf3e71000 2023-10-11T09:45:58.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.272+0000 7f15a7378700 1 -- 172.21.15.78:0/1939471404 <== osd.2 v2:172.21.15.78:6808/35732 4 ==== osd_op_reply(11 [pgnls start_epoch 66 out=81b] v64'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556cb330f200 con 0x556cb27ed400 2023-10-11T09:45:58.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.272+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:12 9.27 9:e5993619:::10000000208.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb27ec800 con 0x556cb27ed400 2023-10-11T09:45:58.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.272+0000 7fa8ae533700 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:14 9.15 9:a8000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf3ff8000 con 0x558cf3fbe400 2023-10-11T09:45:58.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.272+0000 7fb86c27b700 1 -- 172.21.15.78:0/3073125939 <== osd.7 v2:172.21.15.133:6824/35901 8 ==== osd_op_reply(13 [pgnls start_epoch 66 out=81b] v64'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562aba51cfc0 con 0x562ab9ab1800 2023-10-11T09:45:58.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.272+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:14 9.2b 9:d4a58307:::10000000203.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9bb1400 con 0x562ab9ab1800 2023-10-11T09:45:58.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.272+0000 7f15a7378700 1 -- 172.21.15.78:0/1939471404 <== osd.2 v2:172.21.15.78:6808/35732 5 ==== osd_op_reply(12 10000000208.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556cb330f200 con 0x556cb27ed400 2023-10-11T09:45:58.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.272+0000 7f15a83bb840 10 datascan.operator(): handling object 1099511628296.0 2023-10-11T09:45:58.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.272+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:13 9.27 9:e5993619:::10000000208.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb280a000 con 0x556cb27ed400 2023-10-11T09:45:58.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.272+0000 7fa8b3d3e700 1 -- 172.21.15.78:0/4080171823 <== osd.7 v2:172.21.15.133:6824/35901 2 ==== osd_op_reply(14 [pgnls start_epoch 66 out=81b] v64'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x558cf4945200 con 0x558cf3fbe400 2023-10-11T09:45:58.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.272+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:15 9.15 9:a8e69ea8:::1000000020d.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf3ff8c00 con 0x558cf3fbe400 2023-10-11T09:45:58.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.272+0000 7fb86c27b700 1 -- 172.21.15.78:0/3073125939 <== osd.7 v2:172.21.15.133:6824/35901 9 ==== osd_op_reply(14 10000000203.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562aba51cfc0 con 0x562ab9ab1800 2023-10-11T09:45:58.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.272+0000 7fb86dabf840 10 datascan.operator(): handling object 1099511628291.0 2023-10-11T09:45:58.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.272+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:15 9.2b 9:d4a58307:::10000000203.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9ab1400 con 0x562ab9ab1800 2023-10-11T09:45:58.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.272+0000 7fa8b3d3e700 1 -- 172.21.15.78:0/4080171823 <== osd.7 v2:172.21.15.133:6824/35901 3 ==== osd_op_reply(15 1000000020d.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x558cf4945200 con 0x558cf3fbe400 2023-10-11T09:45:58.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.272+0000 7fa8b4d81840 10 datascan.operator(): handling object 1099511628301.0 2023-10-11T09:45:58.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.272+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:16 9.15 9:a8e69ea8:::1000000020d.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf3ff9000 con 0x558cf3fbe400 2023-10-11T09:45:58.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.272+0000 7effabed7700 1 -- 172.21.15.78:0/707681190 <== osd.6 v2:172.21.15.133:6800/35900 6 ==== osd_op_reply(17 10000000201.00000000 [call] v66'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5597a2491200 con 0x5597a1b1cc00 2023-10-11T09:45:58.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.272+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:18 9.18 9:18000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b01400 con 0x5597a1b1cc00 2023-10-11T09:45:58.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7f46976e9700 1 -- 172.21.15.78:0/1439616520 <== osd.7 v2:172.21.15.133:6824/35901 2 ==== osd_op_reply(17 100000001fb.00000000 [call,setxattr (8)] v66'18 uv18 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557bf172b440 con 0x557bf0cbb800 2023-10-11T09:45:58.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7f4698f2d840 1 -- 172.21.15.78:0/1439616520 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:18 10.14 10:28000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x557bf0ca8c00 con 0x557bf0ca9800 2023-10-11T09:45:58.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7fae1a978700 1 -- 172.21.15.78:0/1365619850 <== osd.4 v2:172.21.15.133:6808/35898 3 ==== osd_op_reply(21 100000001fd.00000000 [call,setxattr (8)] v66'9 uv9 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5613bac01200 con 0x5613ba256400 2023-10-11T09:45:58.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7effabed7700 1 -- 172.21.15.78:0/707681190 <== osd.6 v2:172.21.15.133:6800/35900 7 ==== osd_op_reply(18 [pgnls start_epoch 66 out=49b] v64'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5597a2491200 con 0x5597a1b1cc00 2023-10-11T09:45:58.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:22 10.a 10:55a6cd83::::head [pgnls start_epoch 66 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5613ba299c00 con 0x5613ba298000 2023-10-11T09:45:58.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7effa6ecd700 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:19 9.38 9:1c000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b1c800 con 0x5597a1b1c000 2023-10-11T09:45:58.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7f4697eea700 1 -- 172.21.15.78:0/1439616520 <== osd.1 v2:172.21.15.78:6809/35734 2 ==== osd_op_reply(18 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557bf172b200 con 0x557bf0ca9800 2023-10-11T09:45:58.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7f8f4a292700 1 -- 172.21.15.78:0/229716028 <== osd.1 v2:172.21.15.78:6809/35734 9 ==== osd_op_reply(36 100000001f7.00000000 [call,setxattr (8)] v66'22 uv22 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5647ebf68fc0 con 0x5647eb3dd000 2023-10-11T09:45:58.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:37 10.11 10:88000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb577c00 con 0x5647eb543800 2023-10-11T09:45:58.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7f4692ee0700 1 -- 172.21.15.78:0/1439616520 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:19 10.c 10:30000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x557bf0ca8400 con 0x557bf0cbb800 2023-10-11T09:45:58.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7fae1a177700 1 -- 172.21.15.78:0/1365619850 <== osd.6 v2:172.21.15.133:6800/35900 4 ==== osd_op_reply(22 [pgnls start_epoch 66 out=101b] v64'5 uv5 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5613bac01440 con 0x5613ba298000 2023-10-11T09:45:58.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:23 10.a 10:55a6cd83:::1000000020a.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5613ba256800 con 0x5613ba298000 2023-10-11T09:45:58.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7f15a7378700 1 -- 172.21.15.78:0/1939471404 <== osd.2 v2:172.21.15.78:6808/35732 6 ==== osd_op_reply(13 10000000208.00000000 [call] v66'4 uv4 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556cb330f200 con 0x556cb27ed400 2023-10-11T09:45:58.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7f15a83bb840 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x556cb280a400 0x556cb251d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:14 9.17 9:e8000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb280a800 con 0x556cb280a400 2023-10-11T09:45:58.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7effaaed5700 1 -- 172.21.15.78:0/707681190 <== osd.3 v2:172.21.15.78:6824/35738 2 ==== osd_op_reply(19 [pgnls start_epoch 66 out=49b] v64'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5597a2491440 con 0x5597a1b1c000 2023-10-11T09:45:58.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7effa66cc700 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:20 9.4 9:20000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b1d400 con 0x5597a1997000 2023-10-11T09:45:58.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7fae1a177700 1 -- 172.21.15.78:0/1365619850 <== osd.6 v2:172.21.15.133:6800/35900 5 ==== osd_op_reply(23 1000000020a.00000000 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5613bac01440 con 0x5613ba298000 2023-10-11T09:45:58.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7fae1b9bb840 10 datascan.operator(): handling object 1099511628298.0 2023-10-11T09:45:58.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:24 9.2a 9:55a6cd83:::1000000020a.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5613ba257400 con 0x5613ba256c00 2023-10-11T09:45:58.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7f15a6376700 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x556cb280a400 0x556cb251d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7f8f4b294700 1 -- 172.21.15.78:0/229716028 <== osd.5 v2:172.21.15.133:6816/35899 2 ==== osd_op_reply(37 [pgnls start_epoch 66 out=101b] v64'2 uv2 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5647ebf69200 con 0x5647eb543800 2023-10-11T09:45:58.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:38 10.11 10:893be98d:::100000001f8.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb542800 con 0x5647eb543800 2023-10-11T09:45:58.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7effab6d6700 1 -- 172.21.15.78:0/707681190 <== osd.1 v2:172.21.15.78:6809/35734 4 ==== osd_op_reply(20 [pgnls start_epoch 66 out=49b] v64'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5597a2490fc0 con 0x5597a1997000 2023-10-11T09:45:58.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7effa6ecd700 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x5597a1b01c00 0x5597a1b22b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7effa6ecd700 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:21 9.24 9:24000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b00c00 con 0x5597a1b01c00 2023-10-11T09:45:58.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7f46976e9700 1 -- 172.21.15.78:0/1439616520 <== osd.7 v2:172.21.15.133:6824/35901 3 ==== osd_op_reply(19 [pgnls start_epoch 66 out=101b] v64'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x557bf172b440 con 0x557bf0cbb800 2023-10-11T09:45:58.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7f4698f2d840 1 -- 172.21.15.78:0/1439616520 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:20 10.c 10:30604137:::100000001fc.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x557bf0ca8000 con 0x557bf0cbb800 2023-10-11T09:45:58.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7f8f4b294700 1 -- 172.21.15.78:0/229716028 <== osd.5 v2:172.21.15.133:6816/35899 3 ==== osd_op_reply(38 100000001f8.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5647ebf69200 con 0x5647eb543800 2023-10-11T09:45:58.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7f8f4c2d7840 10 datascan.operator(): handling object 1099511628280.1 2023-10-11T09:45:58.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:39 9.19 9:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb3dd400 con 0x5647eb543c00 2023-10-11T09:45:58.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7effab6d6700 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x5597a1b01c00 0x5597a1b22b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7f15a6376700 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x556cb280a400 0x556cb251d600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7f46976e9700 1 -- 172.21.15.78:0/1439616520 <== osd.7 v2:172.21.15.133:6824/35901 4 ==== osd_op_reply(20 100000001fc.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557bf172b440 con 0x557bf0cbb800 2023-10-11T09:45:58.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7f4698f2d840 10 datascan.operator(): handling object 1099511628284.0 2023-10-11T09:45:58.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7fb86c27b700 1 -- 172.21.15.78:0/3073125939 <== osd.7 v2:172.21.15.133:6824/35901 10 ==== osd_op_reply(15 10000000203.00000000 [call] v66'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562aba51cfc0 con 0x562ab9ab1800 2023-10-11T09:45:58.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7f4698f2d840 1 -- 172.21.15.78:0/1439616520 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:21 9.c 9:30604137:::100000001fc.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x557bf0ce0000 con 0x557bf0cbac00 2023-10-11T09:45:58.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:16 9.1b 9:d8000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9ab0400 con 0x562ab9bb0000 2023-10-11T09:45:58.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7f15a6376700 1 -- 172.21.15.78:0/1939471404 <== osd.7 v2:172.21.15.133:6824/35901 1 ==== osd_op_reply(14 [pgnls start_epoch 66 out=49b] v64'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556cb330efc0 con 0x556cb280a400 2023-10-11T09:45:58.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7f15a236e700 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x556cb2810000 0x556cb27f8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7f15a236e700 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:15 9.37 9:ec000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb2810400 con 0x556cb2810000 2023-10-11T09:45:58.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7fb86ba7a700 1 -- 172.21.15.78:0/3073125939 <== osd.3 v2:172.21.15.78:6824/35738 2 ==== osd_op_reply(16 [pgnls start_epoch 66 out=49b] v64'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562aba51d440 con 0x562ab9bb0000 2023-10-11T09:45:58.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.276+0000 7f15a7378700 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x556cb2810000 0x556cb27f8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:17 10.d 10:b6db6db6::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9ab1c00 con 0x562ab9bb0c00 2023-10-11T09:45:58.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7effab6d6700 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x5597a1b01c00 0x5597a1b22b00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7fae1a177700 1 -- 172.21.15.78:0/1365619850 <== osd.3 v2:172.21.15.78:6824/35738 8 ==== osd_op_reply(24 1000000020a.00000000 [call,setxattr (8)] v66'14 uv14 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5613bac01440 con 0x5613ba256c00 2023-10-11T09:45:58.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:25 10.a 10:57fe7535::::head [pgnls start_epoch 66 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5613ba256800 con 0x5613ba298000 2023-10-11T09:45:58.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7fb86ca7c700 1 -- 172.21.15.78:0/3073125939 <== osd.6 v2:172.21.15.133:6800/35900 2 ==== osd_op_reply(17 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562aba51d200 con 0x562ab9bb0c00 2023-10-11T09:45:58.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7fb867271700 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x562ab9bb1800 0x562ab9ba5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7effab6d6700 1 -- 172.21.15.78:0/707681190 <== osd.2 v2:172.21.15.78:6808/35732 1 ==== osd_op_reply(21 [pgnls start_epoch 66 out=49b] v64'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5597a2490fc0 con 0x5597a1b01c00 2023-10-11T09:45:58.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7fb867271700 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:18 10.1d 10:b8000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9ab1000 con 0x562ab9bb1800 2023-10-11T09:45:58.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:22 10.0 10:00000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b00400 con 0x5597a1b1cc00 2023-10-11T09:45:58.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7f15a7378700 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x556cb2810000 0x556cb27f8580 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7fae1a177700 1 -- 172.21.15.78:0/1365619850 <== osd.6 v2:172.21.15.133:6800/35900 6 ==== osd_op_reply(25 [pgnls start_epoch 66 out=81b] v64'5 uv5 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5613bac01440 con 0x5613ba298000 2023-10-11T09:45:58.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:26 10.a 10:57fe7535:::100000001f6.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5613ba299000 con 0x5613ba298000 2023-10-11T09:45:58.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7fb86c27b700 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x562ab9bb1800 0x562ab9ba5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7f15a7378700 1 -- 172.21.15.78:0/1939471404 <== osd.4 v2:172.21.15.133:6808/35898 1 ==== osd_op_reply(15 [pgnls start_epoch 66 out=49b] v64'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556cb330f200 con 0x556cb2810000 2023-10-11T09:45:58.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7f15a1b6d700 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:16 9.f 9:f0000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb27edc00 con 0x556cb27ed000 2023-10-11T09:45:58.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7f8f4a292700 1 -- 172.21.15.78:0/229716028 <== osd.3 v2:172.21.15.78:6824/35738 3 ==== osd_op_reply(39 100000001f8.00000000 [call,setxattr (8)] v66'15 uv15 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5647ebf68fc0 con 0x5647eb543c00 2023-10-11T09:45:58.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:40 10.11 10:8f3fbbd0::::head [pgnls start_epoch 66 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb577c00 con 0x5647eb543800 2023-10-11T09:45:58.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7fae1a177700 1 -- 172.21.15.78:0/1365619850 <== osd.6 v2:172.21.15.133:6800/35900 7 ==== osd_op_reply(26 100000001f6.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5613bac01440 con 0x5613ba298000 2023-10-11T09:45:58.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7fae1b9bb840 10 datascan.operator(): handling object 1099511628278.0 2023-10-11T09:45:58.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:27 9.2a 9:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5613ba298400 con 0x5613ba256c00 2023-10-11T09:45:58.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7f4696ee8700 1 -- 172.21.15.78:0/1439616520 <== osd.3 v2:172.21.15.78:6824/35738 8 ==== osd_op_reply(21 100000001fc.00000000 [call,setxattr (8)] v66'9 uv9 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557bf172afc0 con 0x557bf0cbac00 2023-10-11T09:45:58.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7f4698f2d840 1 -- 172.21.15.78:0/1439616520 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:22 10.c 10:37389ade::::head [pgnls start_epoch 66 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x557bf0ce0400 con 0x557bf0cbb800 2023-10-11T09:45:58.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7fa8b3d3e700 1 -- 172.21.15.78:0/4080171823 <== osd.7 v2:172.21.15.133:6824/35901 4 ==== osd_op_reply(16 1000000020d.00000000 [call] v66'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x558cf4945200 con 0x558cf3fbe400 2023-10-11T09:45:58.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7fa8b4d81840 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x558cf3ff9400 0x558cf3c1b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:17 9.35 9:ac000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf3ff9800 con 0x558cf3ff9400 2023-10-11T09:45:58.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7effabed7700 1 -- 172.21.15.78:0/707681190 <== osd.6 v2:172.21.15.133:6800/35900 8 ==== osd_op_reply(22 [pgnls start_epoch 66 out=101b] v64'5 uv5 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5597a2491200 con 0x5597a1b1cc00 2023-10-11T09:45:58.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7fb86c27b700 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x562ab9bb1800 0x562ab9ba5600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:23 10.0 10:012cdbc7:::10000000206.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b1d400 con 0x5597a1b1cc00 2023-10-11T09:45:58.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7f15a6b77700 1 -- 172.21.15.78:0/1939471404 <== osd.5 v2:172.21.15.133:6816/35899 7 ==== osd_op_reply(16 [pgnls start_epoch 66 out=49b] v64'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556cb330f440 con 0x556cb27ed000 2023-10-11T09:45:58.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7f8f4b294700 1 -- 172.21.15.78:0/229716028 <== osd.5 v2:172.21.15.133:6816/35899 4 ==== osd_op_reply(40 [pgnls start_epoch 66 out=81b] v64'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5647ebf69200 con 0x5647eb543800 2023-10-11T09:45:58.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:41 10.11 10:8f3fbbd0:::1000000020c.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb576800 con 0x5647eb543800 2023-10-11T09:45:58.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7f15a236e700 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:17 9.2f 9:f4000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb2810800 con 0x556cb27ec000 2023-10-11T09:45:58.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7fa8b3d3e700 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x558cf3ff9400 0x558cf3c1b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7effabed7700 1 -- 172.21.15.78:0/707681190 <== osd.6 v2:172.21.15.133:6800/35900 9 ==== osd_op_reply(23 10000000206.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5597a2491200 con 0x5597a1b1cc00 2023-10-11T09:45:58.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7effacf1a840 10 datascan.operator(): handling object 1099511628294.0 2023-10-11T09:45:58.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:24 9.0 9:012cdbc7:::10000000206.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b1d000 con 0x5597a1997000 2023-10-11T09:45:58.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7f46976e9700 1 -- 172.21.15.78:0/1439616520 <== osd.7 v2:172.21.15.133:6824/35901 5 ==== osd_op_reply(22 [pgnls start_epoch 66 out=81b] v64'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557bf172b440 con 0x557bf0cbb800 2023-10-11T09:45:58.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7f4698f2d840 1 -- 172.21.15.78:0/1439616520 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:23 10.c 10:37389ade:::10000000209.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x557bf0ce0800 con 0x557bf0cbb800 2023-10-11T09:45:58.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7fb86c27b700 1 -- 172.21.15.78:0/3073125939 <== osd.4 v2:172.21.15.133:6808/35898 1 ==== osd_op_reply(18 [pgnls start_epoch 66 out=101b] v64'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x562aba51cfc0 con 0x562ab9bb1800 2023-10-11T09:45:58.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7f15a6376700 1 -- 172.21.15.78:0/1939471404 <== osd.3 v2:172.21.15.78:6824/35738 2 ==== osd_op_reply(17 [pgnls start_epoch 66 out=49b] v64'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556cb330efc0 con 0x556cb27ec000 2023-10-11T09:45:58.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7f15a1b6d700 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x556cb27ec400 0x556cb27f8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7f15a1b6d700 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:18 9.1f 9:f8000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb27ed800 con 0x556cb27ec400 2023-10-11T09:45:58.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:19 10.1d 10:b9a363c8:::1000000020d.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9a31400 con 0x562ab9bb1800 2023-10-11T09:45:58.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7f8f4b294700 1 -- 172.21.15.78:0/229716028 <== osd.5 v2:172.21.15.133:6816/35899 5 ==== osd_op_reply(41 1000000020c.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5647ebf69200 con 0x5647eb543800 2023-10-11T09:45:58.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7f8f4c2d7840 10 datascan.operator(): handling object 1099511628300.1 2023-10-11T09:45:58.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:42 9.1e 9:789b801b:::1000000020c.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb56c800 con 0x5647eb3dd800 2023-10-11T09:45:58.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7f46976e9700 1 -- 172.21.15.78:0/1439616520 <== osd.7 v2:172.21.15.133:6824/35901 6 ==== osd_op_reply(23 10000000209.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557bf172b440 con 0x557bf0cbb800 2023-10-11T09:45:58.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7f4698f2d840 10 datascan.operator(): handling object 1099511628297.0 2023-10-11T09:45:58.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7f4698f2d840 1 -- 172.21.15.78:0/1439616520 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:24 9.2c 9:37389ade:::10000000209.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x557bf0ce0c00 con 0x557bf0cbac00 2023-10-11T09:45:58.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7fb86c27b700 1 -- 172.21.15.78:0/3073125939 <== osd.4 v2:172.21.15.133:6808/35898 2 ==== osd_op_reply(19 1000000020d.00000001 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562aba51cfc0 con 0x562ab9bb1800 2023-10-11T09:45:58.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7fb86dabf840 10 datascan.operator(): handling object 1099511628301.1 2023-10-11T09:45:58.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:20 9.15 9:a8e69ea8:::1000000020d.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9a31800 con 0x562ab9ab1800 2023-10-11T09:45:58.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.280+0000 7f15a6b77700 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x556cb27ec400 0x556cb27f8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7fa8b3d3e700 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x558cf3ff9400 0x558cf3c1b600 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7fae1a177700 1 -- 172.21.15.78:0/1365619850 <== osd.3 v2:172.21.15.78:6824/35738 9 ==== osd_op_reply(27 100000001f6.00000000 [call,setxattr (8)] v66'15 uv15 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5613bac01440 con 0x5613ba256c00 2023-10-11T09:45:58.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:28 10.1a 10:58000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5613ba299400 con 0x5613ba298000 2023-10-11T09:45:58.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7f15a6b77700 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x556cb27ec400 0x556cb27f8b00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7fa8b3d3e700 1 -- 172.21.15.78:0/4080171823 <== osd.1 v2:172.21.15.78:6809/35734 1 ==== osd_op_reply(17 [pgnls start_epoch 66 out=49b] v64'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558cf4945200 con 0x558cf3ff9400 2023-10-11T09:45:58.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7fa8ae533700 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:18 9.d 9:b0000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf3ff8400 con 0x558cf3fbe400 2023-10-11T09:45:58.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7f15a6b77700 1 -- 172.21.15.78:0/1939471404 <== osd.1 v2:172.21.15.78:6809/35734 1 ==== osd_op_reply(18 [pgnls start_epoch 66 out=49b] v64'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556cb330f440 con 0x556cb27ec400 2023-10-11T09:45:58.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7f15a236e700 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:19 9.3f 9:fc000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb280a800 con 0x556cb27ed400 2023-10-11T09:45:58.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7fae1a177700 1 -- 172.21.15.78:0/1365619850 <== osd.6 v2:172.21.15.133:6800/35900 8 ==== osd_op_reply(28 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5613bac01440 con 0x5613ba298000 2023-10-11T09:45:58.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7effab6d6700 1 -- 172.21.15.78:0/707681190 <== osd.1 v2:172.21.15.78:6809/35734 5 ==== osd_op_reply(24 10000000206.00000000 [call,setxattr (8)] v66'11 uv11 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5597a2490fc0 con 0x5597a1997000 2023-10-11T09:45:58.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:25 10.0 10:04f22c1d::::head [pgnls start_epoch 66 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b1d800 con 0x5597a1b1cc00 2023-10-11T09:45:58.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7fae1596e700 1 -- 172.21.15.78:0/1365619850 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:29 10.6 10:60000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5613ba298800 con 0x5613ba10f000 2023-10-11T09:45:58.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7fa8b3d3e700 1 -- 172.21.15.78:0/4080171823 <== osd.7 v2:172.21.15.133:6824/35901 5 ==== osd_op_reply(18 [pgnls start_epoch 66 out=49b] v64'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558cf4945200 con 0x558cf3fbe400 2023-10-11T09:45:58.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7fb86c27b700 1 -- 172.21.15.78:0/3073125939 <== osd.7 v2:172.21.15.133:6824/35901 11 ==== osd_op_reply(20 1000000020d.00000000 [call,setxattr (8)] v66'9 uv9 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x562aba51cfc0 con 0x562ab9ab1800 2023-10-11T09:45:58.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7fa8aed34700 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x558cf3fbf800 0x558cf3fdeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7fa8aed34700 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:19 9.2d 9:b4000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf3ff8000 con 0x558cf3fbf800 2023-10-11T09:45:58.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:21 10.1d 10:bbb18289::::head [pgnls start_epoch 66 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9bb0800 con 0x562ab9bb1800 2023-10-11T09:45:58.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7effabed7700 1 -- 172.21.15.78:0/707681190 <== osd.6 v2:172.21.15.133:6800/35900 10 ==== osd_op_reply(25 [pgnls start_epoch 66 out=101b] v64'5 uv5 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5597a2491200 con 0x5597a1b1cc00 2023-10-11T09:45:58.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7fa8b353d700 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x558cf3fbf800 0x558cf3fdeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7f15a7378700 1 -- 172.21.15.78:0/1939471404 <== osd.2 v2:172.21.15.78:6808/35732 7 ==== osd_op_reply(19 [pgnls start_epoch 66 out=81b] v64'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556cb330f200 con 0x556cb27ed400 2023-10-11T09:45:58.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:26 10.0 10:04f22c1d:::1000000020a.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1997400 con 0x5597a1b1cc00 2023-10-11T09:45:58.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7f4696ee8700 1 -- 172.21.15.78:0/1439616520 <== osd.3 v2:172.21.15.78:6824/35738 9 ==== osd_op_reply(24 10000000209.00000000 [call,setxattr (8)] v66'11 uv11 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557bf172afc0 con 0x557bf0cbac00 2023-10-11T09:45:58.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7f4698f2d840 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x557bf0ce1000 0x557bf09db600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7f4698f2d840 1 -- 172.21.15.78:0/1439616520 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:25 10.1c 10:38000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x557bf0ce1400 con 0x557bf0ce1000 2023-10-11T09:45:58.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7fae19976700 1 -- 172.21.15.78:0/1365619850 <== osd.5 v2:172.21.15.133:6816/35899 7 ==== osd_op_reply(29 [pgnls start_epoch 66 out=81b] v64'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5613bac00fc0 con 0x5613ba10f000 2023-10-11T09:45:58.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:20 9.3f 9:feb1c803:::100000001fa.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb280ac00 con 0x556cb27ed400 2023-10-11T09:45:58.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:30 10.6 10:658f2455:::100000001ff.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5613ba299c00 con 0x5613ba10f000 2023-10-11T09:45:58.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7fb86c27b700 1 -- 172.21.15.78:0/3073125939 <== osd.4 v2:172.21.15.133:6808/35898 3 ==== osd_op_reply(21 [pgnls start_epoch 66 out=101b] v64'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x562aba51cfc0 con 0x562ab9bb1800 2023-10-11T09:45:58.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:22 10.1d 10:bbb18289:::10000000204.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9bb1400 con 0x562ab9bb1800 2023-10-11T09:45:58.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7f15a7378700 1 -- 172.21.15.78:0/1939471404 <== osd.2 v2:172.21.15.78:6808/35732 8 ==== osd_op_reply(20 100000001fa.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556cb330f200 con 0x556cb27ed400 2023-10-11T09:45:58.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7effabed7700 1 -- 172.21.15.78:0/707681190 <== osd.6 v2:172.21.15.133:6800/35900 11 ==== osd_op_reply(26 1000000020a.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5597a2491200 con 0x5597a1b1cc00 2023-10-11T09:45:58.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7effacf1a840 10 datascan.operator(): handling object 1099511628298.1 2023-10-11T09:45:58.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7f15a83bb840 10 datascan.operator(): handling object 1099511628282.0 2023-10-11T09:45:58.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:21 9.3f 9:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb280b000 con 0x556cb27ed400 2023-10-11T09:45:58.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7f4696ee8700 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x557bf0ce1000 0x557bf09db600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:27 9.2a 9:55a6cd83:::1000000020a.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1997800 con 0x5597a1b1c000 2023-10-11T09:45:58.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7fae19976700 1 -- 172.21.15.78:0/1365619850 <== osd.5 v2:172.21.15.133:6816/35899 8 ==== osd_op_reply(30 100000001ff.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5613bac00fc0 con 0x5613ba10f000 2023-10-11T09:45:58.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7fae1b9bb840 10 datascan.operator(): handling object 1099511628287.0 2023-10-11T09:45:58.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:31 9.26 9:658f2455:::100000001ff.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5613ba298800 con 0x5613ba10f000 2023-10-11T09:45:58.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7fa8b353d700 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x558cf3fbf800 0x558cf3fdeb00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7fb86c27b700 1 -- 172.21.15.78:0/3073125939 <== osd.4 v2:172.21.15.133:6808/35898 4 ==== osd_op_reply(22 10000000204.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562aba51cfc0 con 0x562ab9bb1800 2023-10-11T09:45:58.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7fb86dabf840 10 datascan.operator(): handling object 1099511628292.1 2023-10-11T09:45:58.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:23 9.5 9:a0beda95:::10000000204.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9bb1c00 con 0x562ab9ab0c00 2023-10-11T09:45:58.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7f4696ee8700 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x557bf0ce1000 0x557bf09db600 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7fa8b353d700 1 -- 172.21.15.78:0/4080171823 <== osd.2 v2:172.21.15.78:6808/35732 1 ==== osd_op_reply(19 [pgnls start_epoch 66 out=49b] v64'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558cf4945440 con 0x558cf3fbf800 2023-10-11T09:45:58.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7fa8b4d81840 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x558cf3ff9c00 0x558cf3fdf600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.284+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:20 10.9 10:92492492::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf4006000 con 0x558cf3ff9c00 2023-10-11T09:45:58.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4aa93700 1 -- 172.21.15.78:0/229716028 <== osd.0 v2:172.21.15.78:6800/35735 10 ==== osd_op_reply(42 1000000020c.00000000 [call,setxattr (8)] v66'9 uv9 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5647ebf69440 con 0x5647eb3dd800 2023-10-11T09:45:58.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:43 10.9 10:90000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5647eb56cc00 con 0x5647eb543800 2023-10-11T09:45:58.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7fa8b2d3c700 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x558cf3ff9c00 0x558cf3fdf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f4696ee8700 1 -- 172.21.15.78:0/1439616520 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== osd_op_reply(25 [pgnls start_epoch 66 out=81b] v64'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557bf172afc0 con 0x557bf0ce1000 2023-10-11T09:45:58.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f4698f2d840 1 -- 172.21.15.78:0/1439616520 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:26 10.1c 10:3d6db04b:::100000001f6.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x557bf0ce1800 con 0x557bf0ce1000 2023-10-11T09:45:58.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4b294700 1 -- 172.21.15.78:0/229716028 <== osd.5 v2:172.21.15.133:6816/35899 6 ==== osd_op_reply(43 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5647ebf69200 con 0x5647eb543800 2023-10-11T09:45:58.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x5647eb3dd800 msgr2=0x5647eb566000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4c2d7840 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x5647eb3dd800 0x5647eb566000 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x5647eb3dd000 msgr2=0x5647eb190100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4c2d7840 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x5647eb3dd000 0x5647eb190100 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x5647eb56c000 msgr2=0x5647eb566580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4c2d7840 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x5647eb56c000 0x5647eb566580 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5647eb543c00 msgr2=0x5647eb18f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4c2d7840 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5647eb543c00 0x5647eb18f600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x5647eb576400 msgr2=0x5647eb566b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4c2d7840 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x5647eb576400 0x5647eb566b00 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x5647eb543800 msgr2=0x5647eb567b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4c2d7840 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x5647eb543800 0x5647eb567b80 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x5647eb56c400 msgr2=0x5647eb567600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4c2d7840 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x5647eb56c400 0x5647eb567600 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x5647eb543400 msgr2=0x5647eb18fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4c2d7840 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x5647eb543400 0x5647eb18fb80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4aa93700 1 -- 172.21.15.78:0/229716028 reap_dead start 2023-10-11T09:45:58.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4aa93700 1 -- 172.21.15.78:0/229716028 reap_dead start 2023-10-11T09:45:58.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4aa93700 1 -- 172.21.15.78:0/229716028 reap_dead start 2023-10-11T09:45:58.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5647eb542000 msgr2=0x5647eb191700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4c2d7840 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5647eb542000 0x5647eb191700 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x5647eb13ffb0 tx=0x5647eb1bf380 comp rx=0 tx=0).stop 2023-10-11T09:45:58.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647eb3dc400 msgr2=0x5647eb191180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4c2d7840 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647eb3dc400 0x5647eb191180 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x5647ebed14d0 tx=0x5647eb3e2c00 comp rx=0 tx=0).stop 2023-10-11T09:45:58.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7effaaed5700 1 -- 172.21.15.78:0/707681190 <== osd.3 v2:172.21.15.78:6824/35738 3 ==== osd_op_reply(27 1000000020a.00000000 [call,setxattr (8)] v66'16 uv16 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5597a2491440 con 0x5597a1b1c000 2023-10-11T09:45:58.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:28 10.0 10:05552577::::head [pgnls start_epoch 66 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b1c400 con 0x5597a1b1cc00 2023-10-11T09:45:58.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7fae19976700 1 -- 172.21.15.78:0/1365619850 <== osd.5 v2:172.21.15.133:6816/35899 9 ==== osd_op_reply(31 100000001ff.00000000 [call,setxattr (8)] v66'18 uv18 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5613bac00fc0 con 0x5613ba10f000 2023-10-11T09:45:58.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:32 10.16 10:68000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5613ba299c00 con 0x5613ba298000 2023-10-11T09:45:58.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7fa8b2d3c700 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x558cf3ff9c00 0x558cf3fdf600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f15a7378700 1 -- 172.21.15.78:0/1939471404 <== osd.2 v2:172.21.15.78:6808/35732 9 ==== osd_op_reply(21 100000001fa.00000000 [call] v66'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556cb330f200 con 0x556cb27ed400 2023-10-11T09:45:58.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:22 10.1b 10:db6db6db::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb280b400 con 0x556cb27ec000 2023-10-11T09:45:58.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f4696ee8700 1 -- 172.21.15.78:0/1439616520 <== osd.6 v2:172.21.15.133:6800/35900 2 ==== osd_op_reply(26 100000001f6.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557bf172afc0 con 0x557bf0ce1000 2023-10-11T09:45:58.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f4698f2d840 10 datascan.operator(): handling object 1099511628278.1 2023-10-11T09:45:58.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f4698f2d840 1 -- 172.21.15.78:0/1439616520 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:27 9.2a 9:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x557bf0ce1c00 con 0x557bf0cbac00 2023-10-11T09:45:58.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7effabed7700 1 -- 172.21.15.78:0/707681190 <== osd.6 v2:172.21.15.133:6800/35900 12 ==== osd_op_reply(28 [pgnls start_epoch 66 out=81b] v64'5 uv5 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5597a2491200 con 0x5597a1b1cc00 2023-10-11T09:45:58.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:29 10.0 10:05552577:::1000000020b.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b1c800 con 0x5597a1b1cc00 2023-10-11T09:45:58.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 shutdown_connections 2023-10-11T09:45:58.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4c2d7840 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5647eb542000 0x5647eb191700 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4c2d7840 1 --2- 172.21.15.78:0/229716028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647eb3dc400 0x5647eb191180 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 >> 172.21.15.78:0/229716028 conn(0x5647eb0e8c00 msgr2=0x5647eb1dd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:58.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7fa8b2d3c700 1 -- 172.21.15.78:0/4080171823 <== osd.5 v2:172.21.15.133:6816/35899 1 ==== osd_op_reply(20 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558cf4944fc0 con 0x558cf3ff9c00 2023-10-11T09:45:58.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 shutdown_connections 2023-10-11T09:45:58.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7fa8aed34700 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:21 10.19 10:98000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf3ff8c00 con 0x558cf3ff9400 2023-10-11T09:45:58.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7fb86ca7c700 1 -- 172.21.15.78:0/3073125939 <== osd.0 v2:172.21.15.78:6800/35735 3 ==== osd_op_reply(23 10000000204.00000000 [call,setxattr (8)] v66'8 uv8 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x562aba51d200 con 0x562ab9ab0c00 2023-10-11T09:45:58.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/229716028 wait complete. 2023-10-11T09:45:58.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f15a6376700 1 -- 172.21.15.78:0/1939471404 <== osd.3 v2:172.21.15.78:6824/35738 3 ==== osd_op_reply(22 [pgnls start_epoch 66 out=81b] v64'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556cb330efc0 con 0x556cb27ec000 2023-10-11T09:45:58.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:24 10.1d 10:bfcd3380::::head [pgnls start_epoch 66 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9a31400 con 0x562ab9bb1800 2023-10-11T09:45:58.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:23 10.1b 10:deccbd37:::100000001fe.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb280b800 con 0x556cb27ec000 2023-10-11T09:45:58.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7effabed7700 1 -- 172.21.15.78:0/707681190 <== osd.6 v2:172.21.15.133:6800/35900 13 ==== osd_op_reply(29 1000000020b.00000000 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5597a2491200 con 0x5597a1b1cc00 2023-10-11T09:45:58.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7effacf1a840 10 datascan.operator(): handling object 1099511628299.0 2023-10-11T09:45:58.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:30 9.20 9:05552577:::1000000020b.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b1dc00 con 0x5597a1b00800 2023-10-11T09:45:58.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7fae1a177700 1 -- 172.21.15.78:0/1365619850 <== osd.6 v2:172.21.15.133:6800/35900 9 ==== osd_op_reply(32 [pgnls start_epoch 66 out=101b] v64'5 uv5 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5613bac01440 con 0x5613ba298000 2023-10-11T09:45:58.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:33 10.16 10:6bb58479:::10000000206.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5613ba257000 con 0x5613ba298000 2023-10-11T09:45:58.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7fae1a177700 1 -- 172.21.15.78:0/1365619850 <== osd.6 v2:172.21.15.133:6800/35900 10 ==== osd_op_reply(33 10000000206.00000001 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5613bac01440 con 0x5613ba298000 2023-10-11T09:45:58.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7fae1b9bb840 10 datascan.operator(): handling object 1099511628294.1 2023-10-11T09:45:58.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:34 9.0 9:012cdbc7:::10000000206.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5613ba10f800 con 0x5613ba257800 2023-10-11T09:45:58.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/1997815101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647eb0e9400 msgr2=0x5647eb18eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4c2d7840 1 --2- 172.21.15.78:0/1997815101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647eb0e9400 0x5647eb18eb00 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x5647eb1b7410 tx=0x5647ebed6ea0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/1997815101 shutdown_connections 2023-10-11T09:45:58.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4c2d7840 1 --2- 172.21.15.78:0/1997815101 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5647eb0e9000 0x5647eb18f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4c2d7840 1 --2- 172.21.15.78:0/1997815101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647eb0e9400 0x5647eb18eb00 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4c2d7840 1 --2- 172.21.15.78:0/1997815101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647eb0e9800 0x5647eb18e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/1997815101 >> 172.21.15.78:0/1997815101 conn(0x5647eb0e9c00 msgr2=0x5647eb1dd000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:58.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/1997815101 shutdown_connections 2023-10-11T09:45:58.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f8f4c2d7840 1 -- 172.21.15.78:0/1997815101 wait complete. 2023-10-11T09:45:58.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7fa8b3d3e700 1 -- 172.21.15.78:0/4080171823 <== osd.1 v2:172.21.15.78:6809/35734 2 ==== osd_op_reply(21 [pgnls start_epoch 66 out=101b] v64'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x558cf4945200 con 0x558cf3ff9400 2023-10-11T09:45:58.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:22 10.19 10:992a6246:::10000000200.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf4006400 con 0x558cf3ff9400 2023-10-11T09:45:58.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7fb86c27b700 1 -- 172.21.15.78:0/3073125939 <== osd.4 v2:172.21.15.133:6808/35898 5 ==== osd_op_reply(24 [pgnls start_epoch 66 out=81b] v64'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562aba51cfc0 con 0x562ab9bb1800 2023-10-11T09:45:58.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:25 10.1d 10:bfcd3380:::10000000205.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9a31800 con 0x562ab9bb1800 2023-10-11T09:45:58.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7fb86c27b700 1 -- 172.21.15.78:0/3073125939 <== osd.4 v2:172.21.15.133:6808/35898 6 ==== osd_op_reply(25 10000000205.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562aba51cfc0 con 0x562ab9bb1800 2023-10-11T09:45:58.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7fb86dabf840 10 datascan.operator(): handling object 1099511628293.0 2023-10-11T09:45:58.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f15a6376700 1 -- 172.21.15.78:0/1939471404 <== osd.3 v2:172.21.15.78:6824/35738 4 ==== osd_op_reply(23 100000001fe.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556cb330efc0 con 0x556cb27ec000 2023-10-11T09:45:58.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f15a83bb840 10 datascan.operator(): handling object 1099511628286.0 2023-10-11T09:45:58.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:24 9.3b 9:deccbd37:::100000001fe.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb280bc00 con 0x556cb27ed400 2023-10-11T09:45:58.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.288+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:26 9.3d 9:bfcd3380:::10000000205.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9ab1400 con 0x562ab9ab1800 2023-10-11T09:45:58.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7fa8b3d3e700 1 -- 172.21.15.78:0/4080171823 <== osd.1 v2:172.21.15.78:6809/35734 3 ==== osd_op_reply(22 10000000200.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x558cf4945200 con 0x558cf3ff9400 2023-10-11T09:45:58.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7fa8b4d81840 10 datascan.operator(): handling object 1099511628288.0 2023-10-11T09:45:58.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:23 9.19 9:992a6246:::10000000200.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf4006800 con 0x558cf3fbf000 2023-10-11T09:45:58.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7effabed7700 1 -- 172.21.15.78:0/707681190 <== osd.0 v2:172.21.15.78:6800/35735 5 ==== osd_op_reply(30 1000000020b.00000000 [call,setxattr (8)] v66'12 uv12 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5597a2491200 con 0x5597a1b00800 2023-10-11T09:45:58.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:31 10.10 10:08000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b1d400 con 0x5597a1b01800 2023-10-11T09:45:58.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7f4696ee8700 1 -- 172.21.15.78:0/1439616520 <== osd.3 v2:172.21.15.78:6824/35738 10 ==== osd_op_reply(27 100000001f6.00000000 [call,setxattr (8)] v66'17 uv17 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557bf172afc0 con 0x557bf0cbac00 2023-10-11T09:45:58.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7f4698f2d840 1 -- 172.21.15.78:0/1439616520 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:28 10.2 10:40000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x557bf0ce0400 con 0x557bf0ca9800 2023-10-11T09:45:58.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7f15a7378700 1 -- 172.21.15.78:0/1939471404 <== osd.2 v2:172.21.15.78:6808/35732 10 ==== osd_op_reply(24 100000001fe.00000000 [call,setxattr (8)] v66'7 uv7 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x556cb330f200 con 0x556cb27ed400 2023-10-11T09:45:58.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7f15a83bb840 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x556cb2810800 0x556cb27f9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:25 10.7 10:e0000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb2810400 con 0x556cb2810800 2023-10-11T09:45:58.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7fb86c27b700 1 -- 172.21.15.78:0/3073125939 <== osd.7 v2:172.21.15.133:6824/35901 12 ==== osd_op_reply(26 10000000205.00000000 [call,setxattr (8)] v66'10 uv10 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x562aba51cfc0 con 0x562ab9ab1800 2023-10-11T09:45:58.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:27 10.3 10:c0000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9ab1000 con 0x562ab9a31000 2023-10-11T09:45:58.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7f4697eea700 1 -- 172.21.15.78:0/1439616520 <== osd.1 v2:172.21.15.78:6809/35734 3 ==== osd_op_reply(28 [pgnls start_epoch 66 out=81b] v64'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557bf172b200 con 0x557bf0ca9800 2023-10-11T09:45:58.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7fae1a978700 1 -- 172.21.15.78:0/1365619850 <== osd.1 v2:172.21.15.78:6809/35734 2 ==== osd_op_reply(34 10000000206.00000000 [call,setxattr (8)] v66'12 uv12 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5613bac01200 con 0x5613ba257800 2023-10-11T09:45:58.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7f4698f2d840 1 -- 172.21.15.78:0/1439616520 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:29 10.2 10:40d56c48:::100000001fc.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x557bf0ce0000 con 0x557bf0ca9800 2023-10-11T09:45:58.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x5613ba257c00 msgr2=0x5613ba27d080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7fae1b9bb840 1 --2- 172.21.15.78:0/1365619850 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x5613ba257c00 0x5613ba27d080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x5613ba257800 msgr2=0x5613ba27d600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7fae1b9bb840 1 --2- 172.21.15.78:0/1365619850 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x5613ba257800 0x5613ba27d600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5613ba256c00 msgr2=0x5613ba27c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7fae1b9bb840 1 --2- 172.21.15.78:0/1365619850 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5613ba256c00 0x5613ba27c580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x5613ba256400 msgr2=0x5613ba27c000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7fae1b9bb840 1 --2- 172.21.15.78:0/1365619850 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x5613ba256400 0x5613ba27c000 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x5613ba10f000 msgr2=0x5613b9ebe100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7fae1b9bb840 1 --2- 172.21.15.78:0/1365619850 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x5613ba10f000 0x5613b9ebe100 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x5613ba298000 msgr2=0x5613b9ebdb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7fae1b9bb840 1 --2- 172.21.15.78:0/1365619850 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x5613ba298000 0x5613b9ebdb80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7fae1a177700 1 -- 172.21.15.78:0/1365619850 reap_dead start 2023-10-11T09:45:58.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7f15a6376700 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x556cb2810800 0x556cb27f9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7fae1a177700 1 -- 172.21.15.78:0/1365619850 reap_dead start 2023-10-11T09:45:58.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7fae1a177700 1 -- 172.21.15.78:0/1365619850 reap_dead start 2023-10-11T09:45:58.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5613ba256000 msgr2=0x5613b9ebf700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7fae1b9bb840 1 --2- 172.21.15.78:0/1365619850 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5613ba256000 0x5613b9ebf700 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x5613babf9830 tx=0x5613bac95b00 comp rx=0 tx=0).stop 2023-10-11T09:45:58.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5613ba10e800 msgr2=0x5613b9ebec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7fae1b9bb840 1 --2- 172.21.15.78:0/1365619850 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5613ba10e800 0x5613b9ebec00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x5613babf9620 tx=0x5613b9ef73e0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 shutdown_connections 2023-10-11T09:45:58.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7fae1b9bb840 1 --2- 172.21.15.78:0/1365619850 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5613ba256000 0x5613b9ebf700 secure :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0x5613babf9830 tx=0x5613bac95b00 comp rx=0 tx=0).stop 2023-10-11T09:45:58.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7fae1b9bb840 1 --2- 172.21.15.78:0/1365619850 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5613ba10e800 0x5613b9ebec00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 >> 172.21.15.78:0/1365619850 conn(0x5613b9e16c00 msgr2=0x5613b9ed3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:58.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7effab6d6700 1 -- 172.21.15.78:0/707681190 <== osd.4 v2:172.21.15.133:6808/35898 4 ==== osd_op_reply(31 [pgnls start_epoch 66 out=81b] v64'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5597a2490fc0 con 0x5597a1b01800 2023-10-11T09:45:58.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7f4697eea700 1 -- 172.21.15.78:0/1439616520 <== osd.1 v2:172.21.15.78:6809/35734 4 ==== osd_op_reply(29 100000001fc.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557bf172b200 con 0x557bf0ca9800 2023-10-11T09:45:58.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7f4698f2d840 10 datascan.operator(): handling object 1099511628284.1 2023-10-11T09:45:58.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:32 10.10 10:08b99fea:::100000001fd.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b1d800 con 0x5597a1b01800 2023-10-11T09:45:58.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7f4698f2d840 1 -- 172.21.15.78:0/1439616520 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:30 9.c 9:30604137:::100000001fc.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x557bf0cbb400 con 0x557bf0cbac00 2023-10-11T09:45:58.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7fb86c27b700 1 -- 172.21.15.78:0/3073125939 <== osd.2 v2:172.21.15.78:6808/35732 2 ==== osd_op_reply(27 [pgnls start_epoch 66 out=81b] v64'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562aba51cfc0 con 0x562ab9a31000 2023-10-11T09:45:58.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:28 10.3 10:c1028715:::10000000202.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9bb1000 con 0x562ab9a31000 2023-10-11T09:45:58.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7f15a6376700 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x556cb2810800 0x556cb27f9600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7fb86c27b700 1 -- 172.21.15.78:0/3073125939 <== osd.2 v2:172.21.15.78:6808/35732 3 ==== osd_op_reply(28 10000000202.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562aba51cfc0 con 0x562ab9a31000 2023-10-11T09:45:58.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7fb86dabf840 10 datascan.operator(): handling object 1099511628290.1 2023-10-11T09:45:58.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:29 9.28 9:15246602:::10000000202.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9bb1c00 con 0x562ab9bb0c00 2023-10-11T09:45:58.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7effab6d6700 1 -- 172.21.15.78:0/707681190 <== osd.4 v2:172.21.15.133:6808/35898 5 ==== osd_op_reply(32 100000001fd.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5597a2490fc0 con 0x5597a1b01800 2023-10-11T09:45:58.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7effacf1a840 10 datascan.operator(): handling object 1099511628285.0 2023-10-11T09:45:58.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:33 9.10 9:08b99fea:::100000001fd.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b01400 con 0x5597a1b01800 2023-10-11T09:45:58.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7fa8b353d700 1 -- 172.21.15.78:0/4080171823 <== osd.3 v2:172.21.15.78:6824/35738 7 ==== osd_op_reply(23 10000000200.00000000 [call,setxattr (8)] v66'16 uv16 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x558cf4945440 con 0x558cf3fbf000 2023-10-11T09:45:58.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.292+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:24 10.19 10:997b7129::::head [pgnls start_epoch 66 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf4006c00 con 0x558cf3ff9400 2023-10-11T09:45:58.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f15a6376700 1 -- 172.21.15.78:0/1939471404 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== osd_op_reply(25 [pgnls start_epoch 66 out=101b] v64'7 uv7 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x556cb330efc0 con 0x556cb2810800 2023-10-11T09:45:58.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:26 10.7 10:e15a914f:::10000000207.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb2810c00 con 0x556cb2810800 2023-10-11T09:45:58.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 shutdown_connections 2023-10-11T09:45:58.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7fa8b3d3e700 1 -- 172.21.15.78:0/4080171823 <== osd.1 v2:172.21.15.78:6809/35734 4 ==== osd_op_reply(24 [pgnls start_epoch 66 out=81b] v64'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x558cf4945200 con 0x558cf3ff9400 2023-10-11T09:45:58.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7fae1b9bb840 1 -- 172.21.15.78:0/1365619850 wait complete. 2023-10-11T09:45:58.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:25 10.19 10:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf4007000 con 0x558cf3ff9400 2023-10-11T09:45:58.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f15a6376700 1 -- 172.21.15.78:0/1939471404 <== osd.6 v2:172.21.15.133:6800/35900 2 ==== osd_op_reply(26 10000000207.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556cb330efc0 con 0x556cb2810800 2023-10-11T09:45:58.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f15a83bb840 10 datascan.operator(): handling object 1099511628295.0 2023-10-11T09:45:58.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7fa8b3d3e700 1 -- 172.21.15.78:0/4080171823 <== osd.1 v2:172.21.15.78:6809/35734 5 ==== osd_op_reply(25 100000001f8.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x558cf4945200 con 0x558cf3ff9400 2023-10-11T09:45:58.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7fa8b4d81840 10 datascan.operator(): handling object 1099511628280.0 2023-10-11T09:45:58.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:26 9.19 9:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf4007400 con 0x558cf3fbf000 2023-10-11T09:45:58.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:27 9.7 9:e15a914f:::10000000207.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb2811000 con 0x556cb27ed000 2023-10-11T09:45:58.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f4696ee8700 1 -- 172.21.15.78:0/1439616520 <== osd.3 v2:172.21.15.78:6824/35738 11 ==== osd_op_reply(30 100000001fc.00000000 [call,setxattr (8)] v66'10 uv10 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557bf172afc0 con 0x557bf0cbac00 2023-10-11T09:45:58.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7fae1b9bb840 1 -- 172.21.15.78:0/3608809413 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5613b9e17000 msgr2=0x5613b9ebd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7fae1b9bb840 1 --2- 172.21.15.78:0/3608809413 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5613b9e17000 0x5613b9ebd080 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x5613b9f2f380 tx=0x5613babf6f60 comp rx=0 tx=0).stop 2023-10-11T09:45:58.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f4698f2d840 1 -- 172.21.15.78:0/1439616520 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:31 10.12 10:48000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x557bf0ca9c00 con 0x557bf0ce1000 2023-10-11T09:45:58.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7fae1b9bb840 1 -- 172.21.15.78:0/3608809413 shutdown_connections 2023-10-11T09:45:58.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7fae1b9bb840 1 --2- 172.21.15.78:0/3608809413 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5613b9e17000 0x5613b9ebd080 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7fae1b9bb840 1 --2- 172.21.15.78:0/3608809413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5613b9e17400 0x5613b9ebcb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7fae1b9bb840 1 --2- 172.21.15.78:0/3608809413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613b9e17800 0x5613b9ebc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7fae1b9bb840 1 -- 172.21.15.78:0/3608809413 >> 172.21.15.78:0/3608809413 conn(0x5613b9e17c00 msgr2=0x5613b9ed3000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:58.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7fae1b9bb840 1 -- 172.21.15.78:0/3608809413 shutdown_connections 2023-10-11T09:45:58.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7fae1b9bb840 1 -- 172.21.15.78:0/3608809413 wait complete. 2023-10-11T09:45:58.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f4696ee8700 1 -- 172.21.15.78:0/1439616520 <== osd.6 v2:172.21.15.133:6800/35900 3 ==== osd_op_reply(31 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557bf172afc0 con 0x557bf0ce1000 2023-10-11T09:45:58.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f4698f2d840 1 -- 172.21.15.78:0/1439616520 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x557bf0cbb000 msgr2=0x557bf09dbb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f4698f2d840 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x557bf0cbb000 0x557bf09dbb80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f4698f2d840 1 -- 172.21.15.78:0/1439616520 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x557bf0ca9800 msgr2=0x557bf0cb2000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f4698f2d840 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x557bf0ca9800 0x557bf0cb2000 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f4698f2d840 1 -- 172.21.15.78:0/1439616520 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x557bf1829c00 msgr2=0x557bf09dc100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f4698f2d840 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x557bf1829c00 0x557bf09dc100 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f4698f2d840 1 -- 172.21.15.78:0/1439616520 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x557bf0cbac00 msgr2=0x557bf0cb2b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f4698f2d840 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x557bf0cbac00 0x557bf0cb2b00 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f4696ee8700 1 -- 172.21.15.78:0/1439616520 reap_dead start 2023-10-11T09:45:58.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f4698f2d840 1 -- 172.21.15.78:0/1439616520 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x557bf0cba000 msgr2=0x557bf0cb2580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f4698f2d840 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x557bf0cba000 0x557bf0cb2580 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f4698f2d840 1 -- 172.21.15.78:0/1439616520 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x557bf0ce1000 msgr2=0x557bf09db600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f4698f2d840 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x557bf0ce1000 0x557bf09db600 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f4698f2d840 1 -- 172.21.15.78:0/1439616520 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x557bf0cbb800 msgr2=0x557bf0cb3600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f4698f2d840 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x557bf0cbb800 0x557bf0cb3600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f4696ee8700 1 -- 172.21.15.78:0/1439616520 reap_dead start 2023-10-11T09:45:58.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f4698f2d840 1 -- 172.21.15.78:0/1439616520 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x557bf1829800 msgr2=0x557bf09dd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f4698f2d840 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x557bf1829800 0x557bf09dd700 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x557bf0a9a6f0 tx=0x557bf17e0e40 comp rx=0 tx=0).stop 2023-10-11T09:45:58.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f4698f2d840 1 -- 172.21.15.78:0/1439616520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bf1828800 msgr2=0x557bf09dcc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f4698f2d840 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bf1828800 0x557bf09dcc00 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x557bf17dd5f0 tx=0x557bf172c0c0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f4698f2d840 1 -- 172.21.15.78:0/1439616520 shutdown_connections 2023-10-11T09:45:58.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f4698f2d840 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x557bf1829800 0x557bf09dd700 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f4698f2d840 1 --2- 172.21.15.78:0/1439616520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bf1828800 0x557bf09dcc00 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f4698f2d840 1 -- 172.21.15.78:0/1439616520 >> 172.21.15.78:0/1439616520 conn(0x557bf1828000 msgr2=0x557bf0a15800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:58.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f4698f2d840 1 -- 172.21.15.78:0/1439616520 shutdown_connections 2023-10-11T09:45:58.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f4698f2d840 1 -- 172.21.15.78:0/1439616520 wait complete. 2023-10-11T09:45:58.619 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', '--worker_n', '3', '--worker_m', '7'] in time 0:00:00.505462, result: 2023-10-11T09:45:58.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f4698f2d840 1 -- 172.21.15.78:0/1193264917 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bf0935000 msgr2=0x557bf09da580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f4698f2d840 1 --2- 172.21.15.78:0/1193264917 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bf0935000 0x557bf09da580 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x557bf0a9b3b0 tx=0x557bf172c000 comp rx=0 tx=0).stop 2023-10-11T09:45:58.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7fb86ca7c700 1 -- 172.21.15.78:0/3073125939 <== osd.6 v2:172.21.15.133:6800/35900 3 ==== osd_op_reply(29 10000000202.00000000 [call,setxattr (8)] v66'12 uv12 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x562aba51d200 con 0x562ab9bb0c00 2023-10-11T09:45:58.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7fb86dabf840 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x562ab9ab1c00 0x562ab97d7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:30 10.13 10:c8000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9bb0800 con 0x562ab9ab1c00 2023-10-11T09:45:58.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f4698f2d840 1 -- 172.21.15.78:0/1193264917 shutdown_connections 2023-10-11T09:45:58.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f4698f2d840 1 --2- 172.21.15.78:0/1193264917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bf0935800 0x557bf09db080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f4698f2d840 1 --2- 172.21.15.78:0/1193264917 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x557bf0935400 0x557bf09dab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f4698f2d840 1 --2- 172.21.15.78:0/1193264917 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bf0935000 0x557bf09da580 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f4698f2d840 1 -- 172.21.15.78:0/1193264917 >> 172.21.15.78:0/1193264917 conn(0x557bf0934c00 msgr2=0x557bf0a15000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:58.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f4698f2d840 1 -- 172.21.15.78:0/1193264917 shutdown_connections 2023-10-11T09:45:58.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7f4698f2d840 1 -- 172.21.15.78:0/1193264917 wait complete. 2023-10-11T09:45:58.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.296+0000 7fb86ba7a700 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x562ab9ab1c00 0x562ab97d7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.300+0000 7fb86ba7a700 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x562ab9ab1c00 0x562ab97d7600 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.300+0000 7fb86ba7a700 1 -- 172.21.15.78:0/3073125939 <== osd.1 v2:172.21.15.78:6809/35734 1 ==== osd_op_reply(30 [pgnls start_epoch 66 out=101b] v64'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x562aba51d440 con 0x562ab9ab1c00 2023-10-11T09:45:58.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.300+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:31 10.13 10:ca8fdbba:::100000001fb.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9bb1400 con 0x562ab9ab1c00 2023-10-11T09:45:58.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.300+0000 7fb86ba7a700 1 -- 172.21.15.78:0/3073125939 <== osd.1 v2:172.21.15.78:6809/35734 2 ==== osd_op_reply(31 100000001fb.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562aba51d440 con 0x562ab9ab1c00 2023-10-11T09:45:58.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.300+0000 7fb86dabf840 10 datascan.operator(): handling object 1099511628283.0 2023-10-11T09:45:58.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.300+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:32 9.13 9:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9bb1000 con 0x562ab9ab1800 2023-10-11T09:45:58.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.300+0000 7effab6d6700 1 -- 172.21.15.78:0/707681190 <== osd.4 v2:172.21.15.133:6808/35898 6 ==== osd_op_reply(33 100000001fd.00000000 [call,setxattr (8)] v66'10 uv10 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5597a2490fc0 con 0x5597a1b01800 2023-10-11T09:45:58.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.300+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:34 10.8 10:10000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b00400 con 0x5597a1b00800 2023-10-11T09:45:58.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.300+0000 7fa8b353d700 1 -- 172.21.15.78:0/4080171823 <== osd.3 v2:172.21.15.78:6824/35738 8 ==== osd_op_reply(26 100000001f8.00000000 [call,setxattr (8)] v66'17 uv17 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x558cf4945440 con 0x558cf3fbf000 2023-10-11T09:45:58.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.300+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:27 10.5 10:a0000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf4007800 con 0x558cf3fbf800 2023-10-11T09:45:58.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.300+0000 7f15a6b77700 1 -- 172.21.15.78:0/1939471404 <== osd.5 v2:172.21.15.133:6816/35899 8 ==== osd_op_reply(27 10000000207.00000000 [call,setxattr (8)] v66'11 uv11 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x556cb330f440 con 0x556cb27ed000 2023-10-11T09:45:58.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.300+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:28 10.7 10:e2e07930::::head [pgnls start_epoch 66 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb2811400 con 0x556cb2810800 2023-10-11T09:45:58.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.300+0000 7effabed7700 1 -- 172.21.15.78:0/707681190 <== osd.0 v2:172.21.15.78:6800/35735 6 ==== osd_op_reply(34 [pgnls start_epoch 66 out=101b] v64'6 uv6 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5597a2491200 con 0x5597a1b00800 2023-10-11T09:45:58.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.300+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:35 10.8 10:101c2845:::100000001fe.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1997800 con 0x5597a1b00800 2023-10-11T09:45:58.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.300+0000 7fa8b353d700 1 -- 172.21.15.78:0/4080171823 <== osd.2 v2:172.21.15.78:6808/35732 2 ==== osd_op_reply(27 [pgnls start_epoch 66 out=81b] v64'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x558cf4945440 con 0x558cf3fbf800 2023-10-11T09:45:58.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.300+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:28 10.5 10:a0beda95:::10000000204.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf4007c00 con 0x558cf3fbf800 2023-10-11T09:45:58.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.300+0000 7effabed7700 1 -- 172.21.15.78:0/707681190 <== osd.0 v2:172.21.15.78:6800/35735 7 ==== osd_op_reply(35 100000001fe.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5597a2491200 con 0x5597a1b00800 2023-10-11T09:45:58.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.300+0000 7effacf1a840 10 datascan.operator(): handling object 1099511628286.1 2023-10-11T09:45:58.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.300+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:36 9.3b 9:deccbd37:::100000001fe.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b00400 con 0x5597a1b01c00 2023-10-11T09:45:58.630 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', '--worker_n', '2', '--worker_m', '7'] in time 0:00:00.517645, result: 2023-10-11T09:45:58.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.300+0000 7f15a6376700 1 -- 172.21.15.78:0/1939471404 <== osd.6 v2:172.21.15.133:6800/35900 3 ==== osd_op_reply(28 [pgnls start_epoch 66 out=101b] v64'7 uv7 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x556cb330efc0 con 0x556cb2810800 2023-10-11T09:45:58.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.300+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:29 10.7 10:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb2811800 con 0x556cb2810800 2023-10-11T09:45:58.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.300+0000 7fa8b353d700 1 -- 172.21.15.78:0/4080171823 <== osd.2 v2:172.21.15.78:6808/35732 3 ==== osd_op_reply(28 10000000204.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x558cf4945440 con 0x558cf3fbf800 2023-10-11T09:45:58.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.300+0000 7fa8b4d81840 10 datascan.operator(): handling object 1099511628292.0 2023-10-11T09:45:58.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.300+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:29 9.5 9:a0beda95:::10000000204.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf4006800 con 0x558cf3e71000 2023-10-11T09:45:58.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.300+0000 7fb86c27b700 1 -- 172.21.15.78:0/3073125939 <== osd.7 v2:172.21.15.133:6824/35901 13 ==== osd_op_reply(32 100000001fb.00000000 [call,setxattr (8)] v66'19 uv19 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x562aba51cfc0 con 0x562ab9ab1800 2023-10-11T09:45:58.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.300+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:33 10.13 10:cb24bee2::::head [pgnls start_epoch 66 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9ab1400 con 0x562ab9ab1c00 2023-10-11T09:45:58.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.304+0000 7f15a6376700 1 -- 172.21.15.78:0/1939471404 <== osd.6 v2:172.21.15.133:6800/35900 4 ==== osd_op_reply(29 10000000001.00000000 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556cb330efc0 con 0x556cb2810800 2023-10-11T09:45:58.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.304+0000 7f15a83bb840 10 datascan.operator(): handling object 1099511627777.0 2023-10-11T09:45:58.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.304+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:30 9.7 9:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb2811c00 con 0x556cb27ed000 2023-10-11T09:45:58.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.304+0000 7fb86ba7a700 1 -- 172.21.15.78:0/3073125939 <== osd.1 v2:172.21.15.78:6809/35734 3 ==== osd_op_reply(33 [pgnls start_epoch 66 out=101b] v64'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x562aba51d440 con 0x562ab9ab1c00 2023-10-11T09:45:58.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.304+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:34 10.13 10:cb24bee2:::100000001fa.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9ab1000 con 0x562ab9ab1c00 2023-10-11T09:45:58.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.304+0000 7fb86ba7a700 1 -- 172.21.15.78:0/3073125939 <== osd.1 v2:172.21.15.78:6809/35734 4 ==== osd_op_reply(34 100000001fa.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562aba51d440 con 0x562ab9ab1c00 2023-10-11T09:45:58.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.304+0000 7fb86dabf840 10 datascan.operator(): handling object 1099511628282.1 2023-10-11T09:45:58.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.304+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:35 9.3f 9:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9a31400 con 0x562ab9a31000 2023-10-11T09:45:58.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.304+0000 7effab6d6700 1 -- 172.21.15.78:0/707681190 <== osd.2 v2:172.21.15.78:6808/35732 2 ==== osd_op_reply(36 100000001fe.00000000 [call,setxattr (8)] v66'8 uv8 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5597a2490fc0 con 0x5597a1b01c00 2023-10-11T09:45:58.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.304+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:37 10.8 10:1247e94b::::head [pgnls start_epoch 66 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b00c00 con 0x5597a1b00800 2023-10-11T09:45:58.636 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', '--worker_n', '1', '--worker_m', '7'] in time 0:00:00.524427, result: 2023-10-11T09:45:58.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.304+0000 7effabed7700 1 -- 172.21.15.78:0/707681190 <== osd.0 v2:172.21.15.78:6800/35735 8 ==== osd_op_reply(37 [pgnls start_epoch 66 out=101b] v64'6 uv6 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5597a2491200 con 0x5597a1b00800 2023-10-11T09:45:58.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.304+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:38 10.8 10:1247e94b:::100000001f9.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b01400 con 0x5597a1b00800 2023-10-11T09:45:58.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.304+0000 7f15a6b77700 1 -- 172.21.15.78:0/1939471404 <== osd.5 v2:172.21.15.133:6816/35899 9 ==== osd_op_reply(30 10000000001.00000000 [call,setxattr (8)] v66'12 uv12 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x556cb330f440 con 0x556cb27ed000 2023-10-11T09:45:58.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.304+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:31 10.7 10:e5993619::::head [pgnls start_epoch 66 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb2810400 con 0x556cb2810800 2023-10-11T09:45:58.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.304+0000 7fa8b2d3c700 1 -- 172.21.15.78:0/4080171823 <== osd.0 v2:172.21.15.78:6800/35735 6 ==== osd_op_reply(29 10000000204.00000000 [call,setxattr (8)] v66'9 uv9 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x558cf4944fc0 con 0x558cf3e71000 2023-10-11T09:45:58.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.304+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:30 10.15 10:a8000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf4007400 con 0x558cf3fbf000 2023-10-11T09:45:58.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.304+0000 7effabed7700 1 -- 172.21.15.78:0/707681190 <== osd.0 v2:172.21.15.78:6800/35735 9 ==== osd_op_reply(38 100000001f9.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5597a2491200 con 0x5597a1b00800 2023-10-11T09:45:58.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.304+0000 7effacf1a840 10 datascan.operator(): handling object 1099511628281.1 2023-10-11T09:45:58.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.304+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:39 9.36 9:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b1c400 con 0x5597a1997000 2023-10-11T09:45:58.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.304+0000 7f15a6376700 1 -- 172.21.15.78:0/1939471404 <== osd.6 v2:172.21.15.133:6800/35900 5 ==== osd_op_reply(31 [pgnls start_epoch 66 out=81b] v64'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556cb330efc0 con 0x556cb2810800 2023-10-11T09:45:58.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.304+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:32 10.7 10:e5993619:::10000000208.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb2810c00 con 0x556cb2810800 2023-10-11T09:45:58.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.304+0000 7fa8b353d700 1 -- 172.21.15.78:0/4080171823 <== osd.3 v2:172.21.15.78:6824/35738 9 ==== osd_op_reply(30 [pgnls start_epoch 66 out=101b] v64'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x558cf4945440 con 0x558cf3fbf000 2023-10-11T09:45:58.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.304+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:31 10.15 10:a8e69ea8:::1000000020d.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf4006000 con 0x558cf3fbf000 2023-10-11T09:45:58.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.304+0000 7f15a6376700 1 -- 172.21.15.78:0/1939471404 <== osd.6 v2:172.21.15.133:6800/35900 6 ==== osd_op_reply(32 10000000208.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556cb330efc0 con 0x556cb2810800 2023-10-11T09:45:58.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.304+0000 7f15a83bb840 10 datascan.operator(): handling object 1099511628296.0 2023-10-11T09:45:58.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.304+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:33 9.27 9:e5993619:::10000000208.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb2811400 con 0x556cb27ed400 2023-10-11T09:45:58.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.304+0000 7fa8b353d700 1 -- 172.21.15.78:0/4080171823 <== osd.3 v2:172.21.15.78:6824/35738 10 ==== osd_op_reply(31 1000000020d.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x558cf4945440 con 0x558cf3fbf000 2023-10-11T09:45:58.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.304+0000 7fa8b4d81840 10 datascan.operator(): handling object 1099511628301.0 2023-10-11T09:45:58.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.304+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:32 9.15 9:a8e69ea8:::1000000020d.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf3e71800 con 0x558cf3fbe400 2023-10-11T09:45:58.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.308+0000 7fb86c27b700 1 -- 172.21.15.78:0/3073125939 <== osd.2 v2:172.21.15.78:6808/35732 4 ==== osd_op_reply(35 100000001fa.00000000 [call,setxattr (8)] v66'11 uv11 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x562aba51cfc0 con 0x562ab9a31000 2023-10-11T09:45:58.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.308+0000 7effab6d6700 1 -- 172.21.15.78:0/707681190 <== osd.1 v2:172.21.15.78:6809/35734 6 ==== osd_op_reply(39 100000001f9.00000000 [call,setxattr (8)] v66'23 uv23 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5597a2490fc0 con 0x5597a1997000 2023-10-11T09:45:58.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.308+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:36 10.13 10:cc6bf2c4::::head [pgnls start_epoch 66 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9a31800 con 0x562ab9ab1c00 2023-10-11T09:45:58.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.308+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:40 10.8 10:15246602::::head [pgnls start_epoch 66 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b1c800 con 0x5597a1b00800 2023-10-11T09:45:58.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.308+0000 7fb86ba7a700 1 -- 172.21.15.78:0/3073125939 <== osd.1 v2:172.21.15.78:6809/35734 5 ==== osd_op_reply(36 [pgnls start_epoch 66 out=81b] v64'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562aba51d440 con 0x562ab9ab1c00 2023-10-11T09:45:58.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.308+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:37 10.13 10:cc6bf2c4:::1000000020b.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9bb1400 con 0x562ab9ab1c00 2023-10-11T09:45:58.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.308+0000 7effabed7700 1 -- 172.21.15.78:0/707681190 <== osd.0 v2:172.21.15.78:6800/35735 10 ==== osd_op_reply(40 [pgnls start_epoch 66 out=101b] v64'6 uv6 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5597a2491200 con 0x5597a1b00800 2023-10-11T09:45:58.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.308+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:41 10.8 10:15246602:::10000000202.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b1dc00 con 0x5597a1b00800 2023-10-11T09:45:58.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.308+0000 7fa8b3d3e700 1 -- 172.21.15.78:0/4080171823 <== osd.7 v2:172.21.15.133:6824/35901 6 ==== osd_op_reply(32 1000000020d.00000000 [call,setxattr (8)] v66'10 uv10 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x558cf4945200 con 0x558cf3fbe400 2023-10-11T09:45:58.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.308+0000 7fb86ba7a700 1 -- 172.21.15.78:0/3073125939 <== osd.1 v2:172.21.15.78:6809/35734 6 ==== osd_op_reply(37 1000000020b.00000001 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562aba51d440 con 0x562ab9ab1c00 2023-10-11T09:45:58.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.308+0000 7fb86dabf840 10 datascan.operator(): handling object 1099511628299.1 2023-10-11T09:45:58.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.308+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:33 10.15 10:ab3d427d::::head [pgnls start_epoch 66 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf3e71400 con 0x558cf3fbf000 2023-10-11T09:45:58.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.308+0000 7f15a7378700 1 -- 172.21.15.78:0/1939471404 <== osd.2 v2:172.21.15.78:6808/35732 11 ==== osd_op_reply(33 10000000208.00000000 [call,setxattr (8)] v66'5 uv5 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x556cb330f200 con 0x556cb27ed400 2023-10-11T09:45:58.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.308+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:34 10.17 10:e8000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb2811000 con 0x556cb2810000 2023-10-11T09:45:58.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.308+0000 7effabed7700 1 -- 172.21.15.78:0/707681190 <== osd.0 v2:172.21.15.78:6800/35735 11 ==== osd_op_reply(41 10000000202.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5597a2491200 con 0x5597a1b00800 2023-10-11T09:45:58.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.308+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:38 9.20 9:05552577:::1000000020b.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9bb0400 con 0x562ab9ab0c00 2023-10-11T09:45:58.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.308+0000 7effacf1a840 10 datascan.operator(): handling object 1099511628290.0 2023-10-11T09:45:58.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.308+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:42 9.28 9:15246602:::10000000202.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b1d000 con 0x5597a1b1cc00 2023-10-11T09:45:58.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.308+0000 7fa8b353d700 1 -- 172.21.15.78:0/4080171823 <== osd.3 v2:172.21.15.78:6824/35738 11 ==== osd_op_reply(33 [pgnls start_epoch 66 out=101b] v64'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x558cf4945440 con 0x558cf3fbf000 2023-10-11T09:45:58.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.308+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:34 10.15 10:ab3d427d:::10000000203.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf3fbfc00 con 0x558cf3fbf000 2023-10-11T09:45:58.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.308+0000 7f15a7378700 1 -- 172.21.15.78:0/1939471404 <== osd.4 v2:172.21.15.133:6808/35898 2 ==== osd_op_reply(34 [pgnls start_epoch 66 out=81b] v64'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556cb330f200 con 0x556cb2810000 2023-10-11T09:45:58.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.308+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:35 10.17 10:e88218fc:::10000000209.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb27ecc00 con 0x556cb2810000 2023-10-11T09:45:58.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.308+0000 7fa8b353d700 1 -- 172.21.15.78:0/4080171823 <== osd.3 v2:172.21.15.78:6824/35738 12 ==== osd_op_reply(34 10000000203.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x558cf4945440 con 0x558cf3fbf000 2023-10-11T09:45:58.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.308+0000 7fa8b4d81840 10 datascan.operator(): handling object 1099511628291.1 2023-10-11T09:45:58.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.308+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:35 9.2b 9:d4a58307:::10000000203.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf3fbec00 con 0x558cf3fbe400 2023-10-11T09:45:58.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.308+0000 7f15a7378700 1 -- 172.21.15.78:0/1939471404 <== osd.4 v2:172.21.15.133:6808/35898 3 ==== osd_op_reply(35 10000000209.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556cb330f200 con 0x556cb2810000 2023-10-11T09:45:58.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.308+0000 7f15a83bb840 10 datascan.operator(): handling object 1099511628297.1 2023-10-11T09:45:58.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.308+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:36 9.2c 9:37389ade:::10000000209.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb27edc00 con 0x556cb27ec000 2023-10-11T09:45:58.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.312+0000 7fb86ca7c700 1 -- 172.21.15.78:0/3073125939 <== osd.0 v2:172.21.15.78:6800/35735 4 ==== osd_op_reply(38 1000000020b.00000000 [call,setxattr (8)] v66'13 uv13 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x562aba51d200 con 0x562ab9ab0c00 2023-10-11T09:45:58.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.312+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:39 10.b 10:d0000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9bb0800 con 0x562ab9bb0c00 2023-10-11T09:45:58.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.312+0000 7effabed7700 1 -- 172.21.15.78:0/707681190 <== osd.6 v2:172.21.15.133:6800/35900 14 ==== osd_op_reply(42 10000000202.00000000 [call,setxattr (8)] v66'13 uv13 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5597a2491200 con 0x5597a1b1cc00 2023-10-11T09:45:58.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.312+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:43 10.8 10:177a762d::::head [pgnls start_epoch 66 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b1d400 con 0x5597a1b00800 2023-10-11T09:45:58.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.312+0000 7effabed7700 1 -- 172.21.15.78:0/707681190 <== osd.0 v2:172.21.15.78:6800/35735 12 ==== osd_op_reply(43 [pgnls start_epoch 66 out=81b] v64'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5597a2491200 con 0x5597a1b00800 2023-10-11T09:45:58.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.312+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:44 10.8 10:177a762d:::10000000201.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b1d800 con 0x5597a1b00800 2023-10-11T09:45:58.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.312+0000 7fb86ca7c700 1 -- 172.21.15.78:0/3073125939 <== osd.6 v2:172.21.15.133:6800/35900 4 ==== osd_op_reply(39 [pgnls start_epoch 66 out=101b] v64'3 uv3 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x562aba51d200 con 0x562ab9bb0c00 2023-10-11T09:45:58.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.312+0000 7f15a6376700 1 -- 172.21.15.78:0/1939471404 <== osd.3 v2:172.21.15.78:6824/35738 5 ==== osd_op_reply(36 10000000209.00000000 [call,setxattr (8)] v66'12 uv12 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x556cb330efc0 con 0x556cb27ec000 2023-10-11T09:45:58.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.312+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:40 10.b 10:d2952c40:::10000000208.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9ab1400 con 0x562ab9bb0c00 2023-10-11T09:45:58.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.312+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:37 10.f 10:f0000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb27ed800 con 0x556cb2810800 2023-10-11T09:45:58.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.312+0000 7effabed7700 1 -- 172.21.15.78:0/707681190 <== osd.0 v2:172.21.15.78:6800/35735 13 ==== osd_op_reply(44 10000000201.00000000 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5597a2491200 con 0x5597a1b00800 2023-10-11T09:45:58.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.312+0000 7effacf1a840 10 datascan.operator(): handling object 1099511628289.0 2023-10-11T09:45:58.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.312+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:45 9.28 9:177a762d:::10000000201.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1997400 con 0x5597a1b1cc00 2023-10-11T09:45:58.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.312+0000 7fa8b3d3e700 1 -- 172.21.15.78:0/4080171823 <== osd.7 v2:172.21.15.133:6824/35901 7 ==== osd_op_reply(35 10000000203.00000000 [call,setxattr (8)] v66'14 uv14 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x558cf4945200 con 0x558cf3fbe400 2023-10-11T09:45:58.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.312+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:36 10.15 10:ab9be7ec::::head [pgnls start_epoch 66 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf3ff8000 con 0x558cf3fbf000 2023-10-11T09:45:58.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.312+0000 7fb86ca7c700 1 -- 172.21.15.78:0/3073125939 <== osd.6 v2:172.21.15.133:6800/35900 5 ==== osd_op_reply(40 10000000208.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562aba51d200 con 0x562ab9bb0c00 2023-10-11T09:45:58.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.312+0000 7fb86dabf840 10 datascan.operator(): handling object 1099511628296.1 2023-10-11T09:45:58.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.312+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:41 9.27 9:e5993619:::10000000208.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9ab1000 con 0x562ab9a31000 2023-10-11T09:45:58.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.312+0000 7fa8b353d700 1 -- 172.21.15.78:0/4080171823 <== osd.3 v2:172.21.15.78:6824/35738 13 ==== osd_op_reply(36 [pgnls start_epoch 66 out=81b] v64'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x558cf4945440 con 0x558cf3fbf000 2023-10-11T09:45:58.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.312+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:37 10.15 10:ab9be7ec:::10000000205.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf3ff9000 con 0x558cf3fbf000 2023-10-11T09:45:58.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.312+0000 7f15a6376700 1 -- 172.21.15.78:0/1939471404 <== osd.6 v2:172.21.15.133:6800/35900 7 ==== osd_op_reply(37 [pgnls start_epoch 66 out=81b] v64'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556cb330efc0 con 0x556cb2810800 2023-10-11T09:45:58.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.312+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:38 10.f 10:f526278d:::10000000001.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb280b400 con 0x556cb2810800 2023-10-11T09:45:58.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.312+0000 7fa8b353d700 1 -- 172.21.15.78:0/4080171823 <== osd.3 v2:172.21.15.78:6824/35738 14 ==== osd_op_reply(37 10000000205.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x558cf4945440 con 0x558cf3fbf000 2023-10-11T09:45:58.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.312+0000 7fa8b4d81840 10 datascan.operator(): handling object 1099511628293.1 2023-10-11T09:45:58.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.312+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:38 9.3d 9:bfcd3380:::10000000205.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf3ff9800 con 0x558cf3fbe400 2023-10-11T09:45:58.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.312+0000 7f15a6376700 1 -- 172.21.15.78:0/1939471404 <== osd.6 v2:172.21.15.133:6800/35900 8 ==== osd_op_reply(38 10000000001.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556cb330efc0 con 0x556cb2810800 2023-10-11T09:45:58.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.312+0000 7f15a83bb840 10 datascan.operator(): handling object 1099511627777.1 2023-10-11T09:45:58.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.312+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:39 9.7 9:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb280b800 con 0x556cb27ed000 2023-10-11T09:45:58.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.312+0000 7fb86c27b700 1 -- 172.21.15.78:0/3073125939 <== osd.2 v2:172.21.15.78:6808/35732 5 ==== osd_op_reply(41 10000000208.00000000 [call,setxattr (8)] v66'6 uv6 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x562aba51cfc0 con 0x562ab9a31000 2023-10-11T09:45:58.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.312+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:42 10.b 10:d4a58307::::head [pgnls start_epoch 66 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9ab0400 con 0x562ab9bb0c00 2023-10-11T09:45:58.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.312+0000 7effabed7700 1 -- 172.21.15.78:0/707681190 <== osd.6 v2:172.21.15.133:6800/35900 15 ==== osd_op_reply(45 10000000201.00000000 [call,setxattr (8)] v66'14 uv14 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5597a2491200 con 0x5597a1b1cc00 2023-10-11T09:45:58.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.312+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:46 10.18 10:18000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1997800 con 0x5597a1b01c00 2023-10-11T09:45:58.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7effab6d6700 1 -- 172.21.15.78:0/707681190 <== osd.2 v2:172.21.15.78:6808/35732 3 ==== osd_op_reply(46 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5597a2490fc0 con 0x5597a1b01c00 2023-10-11T09:45:58.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7effa6ecd700 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:47 10.4 10:20000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b01000 con 0x5597a1b00800 2023-10-11T09:45:58.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7fb86ca7c700 1 -- 172.21.15.78:0/3073125939 <== osd.6 v2:172.21.15.133:6800/35900 6 ==== osd_op_reply(42 [pgnls start_epoch 66 out=81b] v64'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562aba51d200 con 0x562ab9bb0c00 2023-10-11T09:45:58.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:43 10.b 10:d4a58307:::10000000203.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9a31800 con 0x562ab9bb0c00 2023-10-11T09:45:58.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7effabed7700 1 -- 172.21.15.78:0/707681190 <== osd.0 v2:172.21.15.78:6800/35735 14 ==== osd_op_reply(47 [pgnls start_epoch 66 out=101b] v64'3 uv3 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5597a2491200 con 0x5597a1b00800 2023-10-11T09:45:58.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:48 10.4 10:2057972a:::100000001ff.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b1c800 con 0x5597a1b00800 2023-10-11T09:45:58.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7fb86ca7c700 1 -- 172.21.15.78:0/3073125939 <== osd.6 v2:172.21.15.133:6800/35900 7 ==== osd_op_reply(43 10000000203.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562aba51d200 con 0x562ab9bb0c00 2023-10-11T09:45:58.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7fb86dabf840 10 datascan.operator(): handling object 1099511628291.0 2023-10-11T09:45:58.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:44 9.2b 9:d4a58307:::10000000203.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9ab1400 con 0x562ab9ab1800 2023-10-11T09:45:58.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7fa8b3d3e700 1 -- 172.21.15.78:0/4080171823 <== osd.7 v2:172.21.15.133:6824/35901 8 ==== osd_op_reply(38 10000000205.00000000 [call,setxattr (8)] v66'11 uv11 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x558cf4945200 con 0x558cf3fbe400 2023-10-11T09:45:58.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7effabed7700 1 -- 172.21.15.78:0/707681190 <== osd.0 v2:172.21.15.78:6800/35735 15 ==== osd_op_reply(48 100000001ff.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5597a2491200 con 0x5597a1b00800 2023-10-11T09:45:58.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7effacf1a840 10 datascan.operator(): handling object 1099511628287.1 2023-10-11T09:45:58.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7fa8b4d81840 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x558cf3ff8400 0x558cf3fdfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:39 10.d 10:b0000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x558cf3ff8c00 con 0x558cf3ff8400 2023-10-11T09:45:58.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7effacf1a840 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x5597a1b1dc00 0x5597a1b23600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:49 9.26 9:658f2455:::100000001ff.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b1d000 con 0x5597a1b1dc00 2023-10-11T09:45:58.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7fa8b3d3e700 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x558cf3ff8400 0x558cf3fdfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7effaaed5700 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x5597a1b1dc00 0x5597a1b23600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7f15a6b77700 1 -- 172.21.15.78:0/1939471404 <== osd.5 v2:172.21.15.133:6816/35899 10 ==== osd_op_reply(39 10000000001.00000000 [call,setxattr (8)] v66'13 uv13 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x556cb330f440 con 0x556cb27ed000 2023-10-11T09:45:58.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:40 10.1f 10:f8000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb2811800 con 0x556cb280a400 2023-10-11T09:45:58.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7fa8b3d3e700 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x558cf3ff8400 0x558cf3fdfb80 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7effaaed5700 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x5597a1b1dc00 0x5597a1b23600 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7fa8b3d3e700 1 -- 172.21.15.78:0/4080171823 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== osd_op_reply(39 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558cf4945200 con 0x558cf3ff8400 2023-10-11T09:45:58.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7f15a6376700 1 -- 172.21.15.78:0/1939471404 <== osd.7 v2:172.21.15.133:6824/35901 2 ==== osd_op_reply(40 [pgnls start_epoch 66 out=81b] v64'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556cb330efc0 con 0x556cb280a400 2023-10-11T09:45:58.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x558cf3e71000 msgr2=0x558cf3c1c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7fa8b4d81840 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x558cf3e71000 0x558cf3c1c100 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x558cf3ff9400 msgr2=0x558cf3c1b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:41 10.1f 10:feb1c803:::100000001fa.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb2810400 con 0x556cb280a400 2023-10-11T09:45:58.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7fa8b4d81840 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x558cf3ff9400 0x558cf3c1b600 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x558cf3fbf800 msgr2=0x558cf3fdeb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7fa8b4d81840 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x558cf3fbf800 0x558cf3fdeb00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x558cf3fbf000 msgr2=0x558cf3fde580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7fa8b4d81840 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x558cf3fbf000 0x558cf3fde580 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x558cf3fbf400 msgr2=0x558cf3c1bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7fa8b4d81840 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x558cf3fbf400 0x558cf3c1bb80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7fa8b353d700 1 -- 172.21.15.78:0/4080171823 reap_dead start 2023-10-11T09:45:58.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x558cf3ff9c00 msgr2=0x558cf3fdf600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7fa8b4d81840 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x558cf3ff9c00 0x558cf3fdf600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x558cf3ff8400 msgr2=0x558cf3fdfb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7fa8b4d81840 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x558cf3ff8400 0x558cf3fdfb80 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x558cf3fbe400 msgr2=0x558cf3fde000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7fa8b4d81840 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x558cf3fbe400 0x558cf3fde000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7fa8b353d700 1 -- 172.21.15.78:0/4080171823 reap_dead start 2023-10-11T09:45:58.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x558cf3fbe000 msgr2=0x558cf3c1d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7fa8b4d81840 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x558cf3fbe000 0x558cf3c1d700 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x558cf3c69320 tx=0x558cf3e82ea0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cf3e70400 msgr2=0x558cf3c1d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7fa8b4d81840 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cf3e70400 0x558cf3c1d180 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x558cf3bcaf60 tx=0x558cf3e82de0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.316+0000 7fb86c27b700 1 -- 172.21.15.78:0/3073125939 <== osd.7 v2:172.21.15.133:6824/35901 14 ==== osd_op_reply(44 10000000203.00000000 [call,setxattr (8)] v66'15 uv15 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x562aba51cfc0 con 0x562ab9ab1800 2023-10-11T09:45:58.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 shutdown_connections 2023-10-11T09:45:58.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fa8b4d81840 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x558cf3fbe000 0x558cf3c1d700 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fa8b4d81840 1 --2- 172.21.15.78:0/4080171823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cf3e70400 0x558cf3c1d180 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 >> 172.21.15.78:0/4080171823 conn(0x558cf3b74c00 msgr2=0x558cf3c31800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:58.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:45 10.1b 10:d8000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x562ab9bb1c00 con 0x562ab9bb0000 2023-10-11T09:45:58.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 shutdown_connections 2023-10-11T09:45:58.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7f15a6376700 1 -- 172.21.15.78:0/1939471404 <== osd.7 v2:172.21.15.133:6824/35901 3 ==== osd_op_reply(41 100000001fa.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556cb330efc0 con 0x556cb280a400 2023-10-11T09:45:58.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7f15a83bb840 10 datascan.operator(): handling object 1099511628282.0 2023-10-11T09:45:58.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:42 9.3f 9:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x556cb2810c00 con 0x556cb27ed400 2023-10-11T09:45:58.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fa8b4d81840 1 -- 172.21.15.78:0/4080171823 wait complete. 2023-10-11T09:45:58.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fb86ba7a700 1 -- 172.21.15.78:0/3073125939 <== osd.3 v2:172.21.15.78:6824/35738 3 ==== osd_op_reply(45 [pgnls start_epoch 66 out=81b] v64'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562aba51d440 con 0x562ab9bb0000 2023-10-11T09:45:58.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fa8b4d81840 1 -- 172.21.15.78:0/655546129 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558cf3b75000 msgr2=0x558cf3c1b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fa8b4d81840 1 --2- 172.21.15.78:0/655546129 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558cf3b75000 0x558cf3c1b080 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x558cf3c693b0 tx=0x558cf49fce40 comp rx=0 tx=0).stop 2023-10-11T09:45:58.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x562ab9ab0c00 msgr2=0x562ab9ba4000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fb86dabf840 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x562ab9ab0c00 0x562ab9ba4000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x562ab9ab1c00 msgr2=0x562ab97d7600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fb86dabf840 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x562ab9ab1c00 0x562ab97d7600 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x562ab9a31000 msgr2=0x562ab97d8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fb86dabf840 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x562ab9a31000 0x562ab97d8100 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x562ab9bb0000 msgr2=0x562ab9ba4b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fb86dabf840 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x562ab9bb0000 0x562ab9ba4b00 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x562ab9bb1800 msgr2=0x562ab9ba5600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fb86dabf840 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x562ab9bb1800 0x562ab9ba5600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x562ab9bb0c00 msgr2=0x562ab97d7b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fb86dabf840 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x562ab9bb0c00 0x562ab97d7b80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fa8b4d81840 1 -- 172.21.15.78:0/655546129 shutdown_connections 2023-10-11T09:45:58.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fa8b4d81840 1 --2- 172.21.15.78:0/655546129 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558cf3b75000 0x558cf3c1b080 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fa8b4d81840 1 --2- 172.21.15.78:0/655546129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558cf3b75400 0x558cf3c1ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fa8b4d81840 1 --2- 172.21.15.78:0/655546129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558cf3b75800 0x558cf3c1a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fa8b4d81840 1 -- 172.21.15.78:0/655546129 >> 172.21.15.78:0/655546129 conn(0x558cf3b75c00 msgr2=0x558cf3c31000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:58.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x562ab9ab1800 msgr2=0x562ab9ba4580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fb86dabf840 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x562ab9ab1800 0x562ab9ba4580 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7effaaed5700 1 -- 172.21.15.78:0/707681190 <== osd.5 v2:172.21.15.133:6816/35899 1 ==== osd_op_reply(49 100000001ff.00000000 [call,setxattr (8)] v66'19 uv19 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5597a2491440 con 0x5597a1b1dc00 2023-10-11T09:45:58.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:50 10.4 10:218dfa3a::::head [pgnls start_epoch 66 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b1d400 con 0x5597a1b00800 2023-10-11T09:45:58.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fa8b4d81840 1 -- 172.21.15.78:0/655546129 shutdown_connections 2023-10-11T09:45:58.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fa8b4d81840 1 -- 172.21.15.78:0/655546129 wait complete. 2023-10-11T09:45:58.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fb86ba7a700 1 -- 172.21.15.78:0/3073125939 reap_dead start 2023-10-11T09:45:58.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fb86ba7a700 1 -- 172.21.15.78:0/3073125939 reap_dead start 2023-10-11T09:45:58.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fb86ba7a700 1 -- 172.21.15.78:0/3073125939 reap_dead start 2023-10-11T09:45:58.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x562ab9ab0800 msgr2=0x562ab97d9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fb86dabf840 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x562ab9ab0800 0x562ab97d9700 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x562ab9a20720 tx=0x562ab9807380 comp rx=0 tx=0).stop 2023-10-11T09:45:58.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562ab9a30c00 msgr2=0x562ab97d8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fb86dabf840 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562ab9a30c00 0x562ab97d8680 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x562ab9a21410 tx=0x562ab9a42d20 comp rx=0 tx=0).stop 2023-10-11T09:45:58.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 shutdown_connections 2023-10-11T09:45:58.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fb86dabf840 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x562ab9ab0800 0x562ab97d9700 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fb86dabf840 1 --2- 172.21.15.78:0/3073125939 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562ab9a30c00 0x562ab97d8680 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 >> 172.21.15.78:0/3073125939 conn(0x562ab9730c00 msgr2=0x562ab9825800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:58.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 shutdown_connections 2023-10-11T09:45:58.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7effabed7700 1 -- 172.21.15.78:0/707681190 <== osd.0 v2:172.21.15.78:6800/35735 16 ==== osd_op_reply(50 [pgnls start_epoch 66 out=101b] v64'3 uv3 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5597a2491200 con 0x5597a1b00800 2023-10-11T09:45:58.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fb86dabf840 1 -- 172.21.15.78:0/3073125939 wait complete. 2023-10-11T09:45:58.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:51 10.4 10:218dfa3a:::10000000200.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b1d800 con 0x5597a1b00800 2023-10-11T09:45:58.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7f15a7378700 1 -- 172.21.15.78:0/1939471404 <== osd.2 v2:172.21.15.78:6808/35732 12 ==== osd_op_reply(42 100000001fa.00000000 [call,setxattr (8)] v66'12 uv12 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x556cb330f200 con 0x556cb27ed400 2023-10-11T09:45:58.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x556cb27ec400 msgr2=0x556cb27f8b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7f15a83bb840 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x556cb27ec400 0x556cb27f8b00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x556cb27ed400 msgr2=0x556cb27f8000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7f15a83bb840 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x556cb27ed400 0x556cb27f8000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x556cb27ec000 msgr2=0x556cb251e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7f15a83bb840 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x556cb27ec000 0x556cb251e100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x556cb2810000 msgr2=0x556cb27f8580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7f15a83bb840 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x556cb2810000 0x556cb27f8580 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x556cb27ed000 msgr2=0x556cb251db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7f15a83bb840 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x556cb27ed000 0x556cb251db80 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7effabed7700 1 -- 172.21.15.78:0/707681190 <== osd.0 v2:172.21.15.78:6800/35735 17 ==== osd_op_reply(51 10000000200.00000001 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5597a2491200 con 0x5597a1b00800 2023-10-11T09:45:58.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7effacf1a840 10 datascan.operator(): handling object 1099511628288.1 2023-10-11T09:45:58.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fb86dabf840 1 -- 172.21.15.78:0/1503568008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ab9731800 msgr2=0x562ab97d6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fb86dabf840 1 --2- 172.21.15.78:0/1503568008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ab9731800 0x562ab97d6580 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x562ab97ff320 tx=0x562ab9896120 comp rx=0 tx=0).stop 2023-10-11T09:45:58.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x556cb2810800 msgr2=0x556cb27f9600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7f15a83bb840 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x556cb2810800 0x556cb27f9600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x556cb280a400 msgr2=0x556cb251d600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7f15a83bb840 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x556cb280a400 0x556cb251d600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7f15a6b77700 1 -- 172.21.15.78:0/1939471404 reap_dead start 2023-10-11T09:45:58.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fb86dabf840 1 -- 172.21.15.78:0/1503568008 shutdown_connections 2023-10-11T09:45:58.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fb86dabf840 1 --2- 172.21.15.78:0/1503568008 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562ab9731000 0x562ab97d7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fb86dabf840 1 --2- 172.21.15.78:0/1503568008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ab9731400 0x562ab97d6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fb86dabf840 1 --2- 172.21.15.78:0/1503568008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ab9731800 0x562ab97d6580 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fb86dabf840 1 -- 172.21.15.78:0/1503568008 >> 172.21.15.78:0/1503568008 conn(0x562ab9731c00 msgr2=0x562ab9825000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:58.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7f15a6b77700 1 -- 172.21.15.78:0/1939471404 reap_dead start 2023-10-11T09:45:58.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7f15a6b77700 1 -- 172.21.15.78:0/1939471404 reap_dead start 2023-10-11T09:45:58.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7f15a6b77700 1 -- 172.21.15.78:0/1939471404 reap_dead start 2023-10-11T09:45:58.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x556cb336bc00 msgr2=0x556cb251f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7f15a83bb840 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x556cb336bc00 0x556cb251f700 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x556cb25d3590 tx=0x556cb33232c0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:52 9.19 9:992a6246:::10000000200.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5597a1b00c00 con 0x5597a1b1c000 2023-10-11T09:45:58.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556cb336a400 msgr2=0x556cb251f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7f15a83bb840 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556cb336a400 0x556cb251f180 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x556cb3263710 tx=0x556cb3322e40 comp rx=0 tx=0).stop 2023-10-11T09:45:58.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 shutdown_connections 2023-10-11T09:45:58.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7f15a83bb840 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x556cb336bc00 0x556cb251f700 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7f15a83bb840 1 --2- 172.21.15.78:0/1939471404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556cb336a400 0x556cb251f180 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 >> 172.21.15.78:0/1939471404 conn(0x556cb336a000 msgr2=0x556cb2557800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:58.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fb86dabf840 1 -- 172.21.15.78:0/1503568008 shutdown_connections 2023-10-11T09:45:58.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 shutdown_connections 2023-10-11T09:45:58.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7f15a83bb840 1 -- 172.21.15.78:0/1939471404 wait complete. 2023-10-11T09:45:58.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.320+0000 7fb86dabf840 1 -- 172.21.15.78:0/1503568008 wait complete. 2023-10-11T09:45:58.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7f15a83bb840 1 -- 172.21.15.78:0/731713680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cb2477400 msgr2=0x556cb251cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7f15a83bb840 1 --2- 172.21.15.78:0/731713680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cb2477400 0x556cb251cb00 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x556cb25d2a80 tx=0x556cb3266060 comp rx=0 tx=0).stop 2023-10-11T09:45:58.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7f15a83bb840 1 -- 172.21.15.78:0/731713680 shutdown_connections 2023-10-11T09:45:58.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7f15a83bb840 1 --2- 172.21.15.78:0/731713680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556cb2477800 0x556cb251d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7f15a83bb840 1 --2- 172.21.15.78:0/731713680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cb2477400 0x556cb251cb00 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7f15a83bb840 1 --2- 172.21.15.78:0/731713680 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556cb2477000 0x556cb251c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7f15a83bb840 1 -- 172.21.15.78:0/731713680 >> 172.21.15.78:0/731713680 conn(0x556cb2476c00 msgr2=0x556cb2557000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:58.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7f15a83bb840 1 -- 172.21.15.78:0/731713680 shutdown_connections 2023-10-11T09:45:58.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7f15a83bb840 1 -- 172.21.15.78:0/731713680 wait complete. 2023-10-11T09:45:58.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effaaed5700 1 -- 172.21.15.78:0/707681190 <== osd.3 v2:172.21.15.78:6824/35738 4 ==== osd_op_reply(52 10000000200.00000000 [call,setxattr (8)] v66'18 uv18 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5597a2491440 con 0x5597a1b1c000 2023-10-11T09:45:58.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x5597a1b00800 msgr2=0x5597a174db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effacf1a840 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x5597a1b00800 0x5597a174db80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x5597a1997000 msgr2=0x5597a174e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effacf1a840 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x5597a1997000 0x5597a174e100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x5597a1b01c00 msgr2=0x5597a1b22b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effacf1a840 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x5597a1b01c00 0x5597a1b22b00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5597a1b1c000 msgr2=0x5597a1b22000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effacf1a840 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5597a1b1c000 0x5597a1b22000 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x5597a1b01800 msgr2=0x5597a174d600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effacf1a840 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x5597a1b01800 0x5597a174d600 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x5597a1b1dc00 msgr2=0x5597a1b23600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effacf1a840 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x5597a1b1dc00 0x5597a1b23600 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x5597a1b1cc00 msgr2=0x5597a1b22580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effacf1a840 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x5597a1b1cc00 0x5597a1b22580 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effaaed5700 1 -- 172.21.15.78:0/707681190 reap_dead start 2023-10-11T09:45:58.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effaaed5700 1 -- 172.21.15.78:0/707681190 reap_dead start 2023-10-11T09:45:58.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effaaed5700 1 -- 172.21.15.78:0/707681190 reap_dead start 2023-10-11T09:45:58.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effaaed5700 1 -- 172.21.15.78:0/707681190 reap_dead start 2023-10-11T09:45:58.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effaaed5700 1 -- 172.21.15.78:0/707681190 reap_dead start 2023-10-11T09:45:58.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5597a1b00000 msgr2=0x5597a174f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effacf1a840 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5597a1b00000 0x5597a174f700 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x5597a19b8690 tx=0x5597a2486000 comp rx=0 tx=0).stop 2023-10-11T09:45:58.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5597a1996c00 msgr2=0x5597a174e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effacf1a840 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5597a1996c00 0x5597a174e680 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x5597a19b9500 tx=0x5597a2486f60 comp rx=0 tx=0).stop 2023-10-11T09:45:58.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 shutdown_connections 2023-10-11T09:45:58.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effacf1a840 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5597a1b00000 0x5597a174f700 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effacf1a840 1 --2- 172.21.15.78:0/707681190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5597a1996c00 0x5597a174e680 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 >> 172.21.15.78:0/707681190 conn(0x5597a16a6c00 msgr2=0x5597a179b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:58.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 shutdown_connections 2023-10-11T09:45:58.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effacf1a840 1 -- 172.21.15.78:0/707681190 wait complete. 2023-10-11T09:45:58.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effacf1a840 1 -- 172.21.15.78:0/2311838660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597a16a7800 msgr2=0x5597a174c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effacf1a840 1 --2- 172.21.15.78:0/2311838660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597a16a7800 0x5597a174c580 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x5597a17af350 tx=0x5597a2488000 comp rx=0 tx=0).stop 2023-10-11T09:45:58.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effacf1a840 1 -- 172.21.15.78:0/2311838660 shutdown_connections 2023-10-11T09:45:58.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effacf1a840 1 --2- 172.21.15.78:0/2311838660 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5597a16a7000 0x5597a174d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effacf1a840 1 --2- 172.21.15.78:0/2311838660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5597a16a7400 0x5597a174cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effacf1a840 1 --2- 172.21.15.78:0/2311838660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597a16a7800 0x5597a174c580 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effacf1a840 1 -- 172.21.15.78:0/2311838660 >> 172.21.15.78:0/2311838660 conn(0x5597a16a7c00 msgr2=0x5597a179b000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:58.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effacf1a840 1 -- 172.21.15.78:0/2311838660 shutdown_connections 2023-10-11T09:45:58.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.324+0000 7effacf1a840 1 -- 172.21.15.78:0/2311838660 wait complete. 2023-10-11T09:45:58.716 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', '--worker_n', '4', '--worker_m', '7'] in time 0:00:00.601913, result: 2023-10-11T09:45:58.717 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', '--worker_n', '6', '--worker_m', '7'] in time 0:00:00.601380, result: 2023-10-11T09:45:58.717 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', '--worker_n', '5', '--worker_m', '7'] in time 0:00:00.602499, result: 2023-10-11T09:45:58.718 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', '--worker_n', '0', '--worker_m', '7'] in time 0:00:00.607420, result: 2023-10-11T09:45:58.719 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --worker_n 0 --worker_m 7 2023-10-11T09:45:58.719 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --worker_n 1 --worker_m 7 2023-10-11T09:45:58.720 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --worker_n 2 --worker_m 7 2023-10-11T09:45:58.720 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --worker_n 3 --worker_m 7 2023-10-11T09:45:58.721 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --worker_n 4 --worker_m 7 2023-10-11T09:45:58.722 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --worker_n 5 --worker_m 7 2023-10-11T09:45:58.722 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --worker_n 6 --worker_m 7 2023-10-11T09:45:58.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.792+0000 7fabe9f4c840 1 -- start start 2023-10-11T09:45:58.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.792+0000 7fabe9f4c840 1 Processor -- start 2023-10-11T09:45:58.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.792+0000 7fabe9f4c840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d604f29800 0x55d604fce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.792+0000 7fabe9f4c840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d604f29400 0x55d604fceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.796+0000 7fabe8708700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d604f29800 0x55d604fce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.796+0000 7fabe8708700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d604f29800 0x55d604fce580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:53634/0 (socket says 172.21.15.78:53634) 2023-10-11T09:45:58.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.796+0000 7fabe7f07700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d604f29400 0x55d604fceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.796+0000 7fabe7f07700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d604f29400 0x55d604fceb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46298/0 (socket says 172.21.15.78:46298) 2023-10-11T09:45:58.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.796+0000 7fabe8708700 1 -- 172.21.15.78:0/3669947308 learned_addr learned my addr 172.21.15.78:0/3669947308 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.796+0000 7fabe9f4c840 1 --2- 172.21.15.78:0/3669947308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d604f29000 0x55d604fcf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.796+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3669947308 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d604fa8780 con 0x55d604f29400 2023-10-11T09:45:58.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.796+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3669947308 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d604fa8a00 con 0x55d604f29000 2023-10-11T09:45:58.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.796+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3669947308 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d604fa8640 con 0x55d604f29800 2023-10-11T09:45:58.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.796+0000 7fabe8f09700 1 --2- 172.21.15.78:0/3669947308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d604f29000 0x55d604fcf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.796+0000 7fabe7f07700 1 -- 172.21.15.78:0/3669947308 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d604f29800 msgr2=0x55d604fce580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.796+0000 7fabe7f07700 1 --2- 172.21.15.78:0/3669947308 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d604f29800 0x55d604fce580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.796+0000 7fabe7f07700 1 -- 172.21.15.78:0/3669947308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d604f29000 msgr2=0x55d604fcf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.796+0000 7fabe7f07700 1 --2- 172.21.15.78:0/3669947308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d604f29000 0x55d604fcf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.796+0000 7fabe7f07700 1 -- 172.21.15.78:0/3669947308 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d604f4ab60 con 0x55d604f29400 2023-10-11T09:45:58.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.796+0000 7fabe8f09700 1 --2- 172.21.15.78:0/3669947308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d604f29000 0x55d604fcf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:45:58.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.796+0000 7fabe7f07700 1 --2- 172.21.15.78:0/3669947308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d604f29400 0x55d604fceb00 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x55d604ff73b0 tx=0x55d60508eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a65ea3688505674 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.796+0000 7fabe5f03700 1 -- 172.21.15.78:0/3669947308 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d604f59600 con 0x55d604f29400 2023-10-11T09:45:58.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.796+0000 7fabe5f03700 1 -- 172.21.15.78:0/3669947308 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d604f74480 con 0x55d604f29400 2023-10-11T09:45:58.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.796+0000 7fabe5f03700 1 -- 172.21.15.78:0/3669947308 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d604f591e0 con 0x55d604f29400 2023-10-11T09:45:58.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.796+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3669947308 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d604f4a340 con 0x55d604f29400 2023-10-11T09:45:58.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.796+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3669947308 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55d604f4a000 con 0x55d604f29400 2023-10-11T09:45:58.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.800+0000 7fabe9f4c840 4 waiting for MDS map... 2023-10-11T09:45:58.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.800+0000 7fabe5f03700 1 -- 172.21.15.78:0/3669947308 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x55d604f62c40 con 0x55d604f29400 2023-10-11T09:45:58.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.800+0000 7fabe5f03700 1 -- 172.21.15.78:0/3669947308 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 43) v1 ==== 3118+0+0 (secure 0 0 0) 0x55d605cfe000 con 0x55d604f29400 2023-10-11T09:45:58.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.800+0000 7fabe9f4c840 4 Got MDS map 43 2023-10-11T09:45:58.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.800+0000 7fabe9f4c840 4 datascan.main: Using metadata pool output 2023-10-11T09:45:58.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.800+0000 7fabe9f4c840 4 datascan.main: connecting to RADOS... 2023-10-11T09:45:58.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.800+0000 7fabe9f4c840 1 Processor -- start 2023-10-11T09:45:58.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.800+0000 7fabe9f4c840 1 -- start start 2023-10-11T09:45:58.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.800+0000 7fabe9f4c840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d60521a400 0x55d604fcf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.800+0000 7fabe9f4c840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d60521a800 0x55d604fcfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.804+0000 7fabe9f4c840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d60521ac00 0x55d604fd0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.804+0000 7fabe8f09700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d60521a400 0x55d604fcf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.804+0000 7fabe8f09700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d60521a400 0x55d604fcf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:39104/0 (socket says 172.21.15.78:39104) 2023-10-11T09:45:58.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.804+0000 7fabe7f07700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d60521ac00 0x55d604fd0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.804+0000 7fabe7f07700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d60521ac00 0x55d604fd0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46300/0 (socket says 172.21.15.78:46300) 2023-10-11T09:45:58.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.804+0000 7fabe8f09700 1 -- 172.21.15.78:0/118739238 learned_addr learned my addr 172.21.15.78:0/118739238 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.804+0000 7fabe9f4c840 1 -- 172.21.15.78:0/118739238 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d604f5fe00 con 0x55d60521ac00 2023-10-11T09:45:58.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.804+0000 7fabe8708700 1 --2- 172.21.15.78:0/118739238 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d60521a800 0x55d604fcfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.804+0000 7fabe9f4c840 1 -- 172.21.15.78:0/118739238 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d604fa8a00 con 0x55d60521a400 2023-10-11T09:45:58.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.804+0000 7fabe9f4c840 1 -- 172.21.15.78:0/118739238 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d604fa8500 con 0x55d60521a800 2023-10-11T09:45:58.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.804+0000 7fabe7f07700 1 -- 172.21.15.78:0/118739238 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d60521a800 msgr2=0x55d604fcfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.804+0000 7fabe7f07700 1 --2- 172.21.15.78:0/118739238 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d60521a800 0x55d604fcfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.804+0000 7fabe7f07700 1 -- 172.21.15.78:0/118739238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d60521a400 msgr2=0x55d604fcf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.804+0000 7fabe7f07700 1 --2- 172.21.15.78:0/118739238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d60521a400 0x55d604fcf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.804+0000 7fabe7f07700 1 -- 172.21.15.78:0/118739238 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d604f4aea0 con 0x55d60521ac00 2023-10-11T09:45:58.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.804+0000 7fabe8708700 1 --2- 172.21.15.78:0/118739238 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d60521a800 0x55d604fcfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:45:58.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.804+0000 7fabe7f07700 1 --2- 172.21.15.78:0/118739238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d60521ac00 0x55d604fd0100 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x55d605d146c0 tx=0x55d604fff4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6a4cecbe17997a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.804+0000 7fabe3eff700 1 -- 172.21.15.78:0/118739238 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d604f58f20 con 0x55d60521ac00 2023-10-11T09:45:58.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.804+0000 7fabe3eff700 1 -- 172.21.15.78:0/118739238 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d605d12f00 con 0x55d60521ac00 2023-10-11T09:45:58.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.804+0000 7fabe3eff700 1 -- 172.21.15.78:0/118739238 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d604f59600 con 0x55d60521ac00 2023-10-11T09:45:58.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.804+0000 7fabe9f4c840 1 -- 172.21.15.78:0/118739238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d60521ac00 msgr2=0x55d604fd0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.804+0000 7fabe9f4c840 1 --2- 172.21.15.78:0/118739238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d60521ac00 0x55d604fd0100 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x55d605d146c0 tx=0x55d604fff4a0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.808+0000 7fabe9f4c840 1 -- 172.21.15.78:0/118739238 shutdown_connections 2023-10-11T09:45:58.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.808+0000 7fabe9f4c840 1 --2- 172.21.15.78:0/118739238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d60521ac00 0x55d604fd0100 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.808+0000 7fabe9f4c840 1 --2- 172.21.15.78:0/118739238 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d60521a800 0x55d604fcfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.808+0000 7fabe9f4c840 1 --2- 172.21.15.78:0/118739238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d60521a400 0x55d604fcf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.808+0000 7fabe9f4c840 1 -- 172.21.15.78:0/118739238 >> 172.21.15.78:0/118739238 conn(0x55d604f28c00 msgr2=0x55d605015800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:58.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.808+0000 7fabe9f4c840 1 -- 172.21.15.78:0/118739238 shutdown_connections 2023-10-11T09:45:58.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.808+0000 7fabe9f4c840 1 -- 172.21.15.78:0/118739238 wait complete. 2023-10-11T09:45:58.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.808+0000 7fabe9f4c840 1 Processor -- start 2023-10-11T09:45:58.818 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.808+0000 7fabe9f4c840 1 -- start start 2023-10-11T09:45:58.818 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.808+0000 7fabe9f4c840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d60521a400 0x55d604fd1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.818 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.808+0000 7fabe9f4c840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d60521a800 0x55d604fd0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.818 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.808+0000 7fabe9f4c840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d60521ac00 0x55d604fd0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.818 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.808+0000 7fabe9f4c840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d604fa83c0 con 0x55d60521a400 2023-10-11T09:45:58.819 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.808+0000 7fabe9f4c840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d604fa8280 con 0x55d60521ac00 2023-10-11T09:45:58.819 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.808+0000 7fabe9f4c840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d604fa8140 con 0x55d60521a800 2023-10-11T09:45:58.819 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.808+0000 7fabe8f09700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d60521a400 0x55d604fd1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.819 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.808+0000 7fabe8f09700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d60521a400 0x55d604fd1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46310/0 (socket says 172.21.15.78:46310) 2023-10-11T09:45:58.819 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.808+0000 7fabe8f09700 1 -- 172.21.15.78:0/3728357160 learned_addr learned my addr 172.21.15.78:0/3728357160 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.808+0000 7fabe8708700 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d60521a800 0x55d604fd0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.808+0000 7fabe8f09700 1 -- 172.21.15.78:0/3728357160 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d60521a800 msgr2=0x55d604fd0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.808+0000 7fabe8f09700 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d60521a800 0x55d604fd0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.808+0000 7fabe8f09700 1 -- 172.21.15.78:0/3728357160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d60521ac00 msgr2=0x55d604fd0680 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-11T09:45:58.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.808+0000 7fabe8f09700 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d60521ac00 0x55d604fd0680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.808+0000 7fabe8f09700 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d604f4ab60 con 0x55d60521a400 2023-10-11T09:45:58.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.812+0000 7fabe8f09700 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d60521a400 0x55d604fd1180 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x55d605d15470 tx=0x55d60508e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.812+0000 7fabe26fc700 1 -- 172.21.15.78:0/3728357160 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d604f59080 con 0x55d60521a400 2023-10-11T09:45:58.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.812+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d604f4a340 con 0x55d60521a400 2023-10-11T09:45:58.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.812+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d604f4b520 con 0x55d60521a400 2023-10-11T09:45:58.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.812+0000 7fabe9f4c840 4 datascan.init: resolving metadata pool 8 2023-10-11T09:45:58.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.812+0000 7fabe26fc700 1 -- 172.21.15.78:0/3728357160 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d605d12180 con 0x55d60521a400 2023-10-11T09:45:58.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.812+0000 7fabe26fc700 1 -- 172.21.15.78:0/3728357160 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d604f58f20 con 0x55d60521a400 2023-10-11T09:45:58.824 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.812+0000 7f541c581840 1 -- start start 2023-10-11T09:45:58.824 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.812+0000 7f541c581840 1 Processor -- start 2023-10-11T09:45:58.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.812+0000 7f541c581840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf8f51800 0x55caf8ff6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.812+0000 7f541c581840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55caf8f51400 0x55caf8ff6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.812+0000 7f541c581840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55caf8f51000 0x55caf8ff7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.812+0000 7f541c581840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caf8fd0a00 con 0x55caf8f51800 2023-10-11T09:45:58.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.812+0000 7f541c581840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caf8fd0780 con 0x55caf8f51000 2023-10-11T09:45:58.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.812+0000 7f541c581840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55caf8fd0640 con 0x55caf8f51400 2023-10-11T09:45:58.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.812+0000 7f541ad3d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf8f51800 0x55caf8ff6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.812+0000 7f541ad3d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf8f51800 0x55caf8ff6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46328/0 (socket says 172.21.15.78:46328) 2023-10-11T09:45:58.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.812+0000 7f541ad3d700 1 -- 172.21.15.78:0/302911193 learned_addr learned my addr 172.21.15.78:0/302911193 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.812+0000 7f541b53e700 1 --2- 172.21.15.78:0/302911193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55caf8f51000 0x55caf8ff7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.812+0000 7f541ad3d700 1 -- 172.21.15.78:0/302911193 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55caf8f51400 msgr2=0x55caf8ff6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.812+0000 7f541ad3d700 1 --2- 172.21.15.78:0/302911193 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55caf8f51400 0x55caf8ff6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.816+0000 7f541ad3d700 1 -- 172.21.15.78:0/302911193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55caf8f51000 msgr2=0x55caf8ff7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.816+0000 7f541ad3d700 1 --2- 172.21.15.78:0/302911193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55caf8f51000 0x55caf8ff7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.816+0000 7f541ad3d700 1 -- 172.21.15.78:0/302911193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55caf8f72b60 con 0x55caf8f51800 2023-10-11T09:45:58.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.816+0000 7f541ad3d700 1 --2- 172.21.15.78:0/302911193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf8f51800 0x55caf8ff6580 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x55caf9051410 tx=0x55caf9d2afc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef2fe7e5a7874052 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.812+0000 7fabe26fc700 1 -- 172.21.15.78:0/3728357160 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x55d604f86840 con 0x55d60521a400 2023-10-11T09:45:58.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.816+0000 7fabe26fc700 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55d605372000 0x55d604fd1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.816+0000 7f5418538700 1 -- 172.21.15.78:0/302911193 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55caf8f81600 con 0x55caf8f51800 2023-10-11T09:45:58.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.816+0000 7f3a10af2840 1 -- start start 2023-10-11T09:45:58.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.816+0000 7f3a10af2840 1 Processor -- start 2023-10-11T09:45:58.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.816+0000 7fabe26fc700 1 -- 172.21.15.78:0/3728357160 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x55d604f62a80 con 0x55d60521a400 2023-10-11T09:45:58.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.816+0000 7fabe9f4c840 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-10-11T09:45:58.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.816+0000 7fabe9f4c840 4 datascan.main: data pool 'cephfs_data' has ID 9 2023-10-11T09:45:58.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.816+0000 7fabe9f4c840 4 datascan.main: opening data pool 'cephfs_data' 2023-10-11T09:45:58.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.816+0000 7fabe9f4c840 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55d60521b000 0x55d604fd0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.816+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:1 8.14 8:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55d60521b400 con 0x55d60521b000 2023-10-11T09:45:58.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.816+0000 7fabe8708700 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55d605372000 0x55d604fd1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.816+0000 7f541c581840 1 -- 172.21.15.78:0/302911193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55caf8f72340 con 0x55caf8f51800 2023-10-11T09:45:58.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.816+0000 7f541c581840 1 -- 172.21.15.78:0/302911193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55caf8f72000 con 0x55caf8f51800 2023-10-11T09:45:58.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.816+0000 7f541c581840 4 waiting for MDS map... 2023-10-11T09:45:58.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.816+0000 7fabe7f07700 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55d60521b000 0x55d604fd0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.816+0000 7f5418538700 1 -- 172.21.15.78:0/302911193 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55caf8f9c480 con 0x55caf8f51800 2023-10-11T09:45:58.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.816+0000 7f5418538700 1 -- 172.21.15.78:0/302911193 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55caf8f811e0 con 0x55caf8f51800 2023-10-11T09:45:58.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.816+0000 7fabe8708700 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55d605372000 0x55d604fd1700 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x55d604ff7350 tx=0x55d605233ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.816+0000 7f3a10af2840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5595c50d1000 0x5595c5176580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.816+0000 7f3a10af2840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595c50d1400 0x5595c5176b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.816+0000 7f3a10af2840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595c50d1800 0x5595c5177080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.816+0000 7f3a10af2840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595c5150a00 con 0x5595c50d1800 2023-10-11T09:45:58.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.816+0000 7f3a10af2840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595c5150780 con 0x5595c50d1400 2023-10-11T09:45:58.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.816+0000 7f3a10af2840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x5595c5150640 con 0x5595c50d1000 2023-10-11T09:45:58.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.816+0000 7f3a0eaad700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595c50d1400 0x5595c5176b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.816+0000 7f3a0eaad700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595c50d1400 0x5595c5176b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:39150/0 (socket says 172.21.15.78:39150) 2023-10-11T09:45:58.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.816+0000 7f3a0eaad700 1 -- 172.21.15.78:0/2018903296 learned_addr learned my addr 172.21.15.78:0/2018903296 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.816+0000 7f5418538700 1 -- 172.21.15.78:0/302911193 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x55caf8f8ac40 con 0x55caf8f51800 2023-10-11T09:45:58.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.816+0000 7fabe7f07700 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55d60521b000 0x55d604fd0100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.816+0000 7f3a0f2ae700 1 --2- 172.21.15.78:0/2018903296 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5595c50d1000 0x5595c5176580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.816+0000 7f3a0faaf700 1 --2- 172.21.15.78:0/2018903296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595c50d1800 0x5595c5177080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.820+0000 7f5418538700 1 -- 172.21.15.78:0/302911193 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 43) v1 ==== 3118+0+0 (secure 0 0 0) 0x55caf9258000 con 0x55caf8f51800 2023-10-11T09:45:58.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.820+0000 7f3a0eaad700 1 -- 172.21.15.78:0/2018903296 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5595c50d1000 msgr2=0x5595c5176580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.820+0000 7f3a0eaad700 1 --2- 172.21.15.78:0/2018903296 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5595c50d1000 0x5595c5176580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.820+0000 7f3a0eaad700 1 -- 172.21.15.78:0/2018903296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595c50d1800 msgr2=0x5595c5177080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.820+0000 7f3a0eaad700 1 --2- 172.21.15.78:0/2018903296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595c50d1800 0x5595c5177080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.820+0000 7f3a0eaad700 1 -- 172.21.15.78:0/2018903296 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595c50f2b60 con 0x5595c50d1400 2023-10-11T09:45:58.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.820+0000 7fabe7f07700 1 -- 172.21.15.78:0/3728357160 <== osd.4 v2:172.21.15.133:6808/35898 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv97 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x55d605db2fc0 con 0x55d60521b000 2023-10-11T09:45:58.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.820+0000 7fabe9f4c840 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55d60521b800 0x55d604fcfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.820+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:2 8.1f 8:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55d605372400 con 0x55d60521b800 2023-10-11T09:45:58.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.820+0000 7f3a0eaad700 1 --2- 172.21.15.78:0/2018903296 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595c50d1400 0x5595c5176b00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x5595c51d9500 tx=0x5595c5eb2000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ebb2b8aa965674 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.820+0000 7f541c581840 4 Got MDS map 43 2023-10-11T09:45:58.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.820+0000 7f541c581840 4 datascan.main: Using metadata pool output 2023-10-11T09:45:58.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.820+0000 7f541c581840 4 datascan.main: connecting to RADOS... 2023-10-11T09:45:58.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.820+0000 7fabe8f09700 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55d60521b800 0x55d604fcfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.820+0000 7f3a0caa9700 1 -- 172.21.15.78:0/2018903296 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5595c5101080 con 0x5595c50d1400 2023-10-11T09:45:58.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.820+0000 7f3a0caa9700 1 -- 172.21.15.78:0/2018903296 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5595c511c180 con 0x5595c50d1400 2023-10-11T09:45:58.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.820+0000 7f3a0caa9700 1 -- 172.21.15.78:0/2018903296 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5595c5101600 con 0x5595c50d1400 2023-10-11T09:45:58.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.820+0000 7f3a10af2840 1 -- 172.21.15.78:0/2018903296 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5595c50f31e0 con 0x5595c50d1400 2023-10-11T09:45:58.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.820+0000 7f3a10af2840 1 -- 172.21.15.78:0/2018903296 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5595c50f2b60 con 0x5595c50d1400 2023-10-11T09:45:58.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.820+0000 7f3a10af2840 4 waiting for MDS map... 2023-10-11T09:45:58.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.820+0000 7f3a0caa9700 1 -- 172.21.15.78:0/2018903296 <== mon.1 v2:172.21.15.133:3300/0 4 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x5595c510ac40 con 0x5595c50d1400 2023-10-11T09:45:58.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.820+0000 7f3a0caa9700 1 -- 172.21.15.78:0/2018903296 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 43) v1 ==== 3118+0+0 (secure 0 0 0) 0x5595c5112f00 con 0x5595c50d1400 2023-10-11T09:45:58.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.820+0000 7f3a10af2840 4 Got MDS map 43 2023-10-11T09:45:58.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.820+0000 7f3a10af2840 4 datascan.main: Using metadata pool output 2023-10-11T09:45:58.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.820+0000 7f3a10af2840 4 datascan.main: connecting to RADOS... 2023-10-11T09:45:58.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.820+0000 7f541c581840 1 Processor -- start 2023-10-11T09:45:58.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.820+0000 7fabe8f09700 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55d60521b800 0x55d604fcfb80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.820+0000 7f0c16003840 1 -- start start 2023-10-11T09:45:58.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.820+0000 7f0c16003840 1 Processor -- start 2023-10-11T09:45:58.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.820+0000 7fabe8f09700 1 -- 172.21.15.78:0/3728357160 <== osd.0 v2:172.21.15.78:6800/35735 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x55d605db3200 con 0x55d60521b800 2023-10-11T09:45:58.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.820+0000 7fabe9f4c840 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55d605372800 0x55d604fcf600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.820+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:3 9.0 9:00000000::::head [pgnls-filter start_epoch 66 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55d605372c00 con 0x55d605372800 2023-10-11T09:45:58.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.820+0000 7f541c581840 1 -- start start 2023-10-11T09:45:58.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.820+0000 7f3a10af2840 1 Processor -- start 2023-10-11T09:45:58.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.820+0000 7fabe8708700 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55d605372800 0x55d604fcf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.820+0000 7f0c16003840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d58d89000 0x556d58e2eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.820+0000 7f3a10af2840 1 -- start start 2023-10-11T09:45:58.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.820+0000 7f0c16003840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556d58d89800 0x556d58e2f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f0c16003840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d58d89c00 0x556d58e2f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f0c147bf700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d58d89000 0x556d58e2eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f0c147bf700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d58d89000 0x556d58e2eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:39164/0 (socket says 172.21.15.78:39164) 2023-10-11T09:45:58.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f541c581840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55caf925c400 0x55caf8ff7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7fabe8708700 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55d605372800 0x55d604fcf600 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f0c13fbe700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556d58d89800 0x556d58e2f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f0c14fc0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d58d89c00 0x556d58e2f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f0c14fc0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d58d89c00 0x556d58e2f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46350/0 (socket says 172.21.15.78:46350) 2023-10-11T09:45:58.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f541c581840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf925c800 0x55caf8ff7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f541c581840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55caf925cc00 0x55caf8ff8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f541c581840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caf8f87e00 con 0x55caf925c800 2023-10-11T09:45:58.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f541c581840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caf8fd0780 con 0x55caf925c400 2023-10-11T09:45:58.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f541c581840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55caf8fd0500 con 0x55caf925cc00 2023-10-11T09:45:58.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f109f181840 1 -- start start 2023-10-11T09:45:58.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f109f181840 1 Processor -- start 2023-10-11T09:45:58.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f0c13fbe700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556d58d89800 0x556d58e2f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:53724/0 (socket says 172.21.15.78:53724) 2023-10-11T09:45:58.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f0c147bf700 1 -- 172.21.15.78:0/299133903 learned_addr learned my addr 172.21.15.78:0/299133903 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f0c16003840 1 -- 172.21.15.78:0/299133903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d58e08a00 con 0x556d58d89c00 2023-10-11T09:45:58.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f0c16003840 1 -- 172.21.15.78:0/299133903 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d58e08780 con 0x556d58d89000 2023-10-11T09:45:58.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f0c16003840 1 -- 172.21.15.78:0/299133903 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x556d58e08640 con 0x556d58d89800 2023-10-11T09:45:58.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f3a10af2840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595c5416400 0x5595c5177600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f3a10af2840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595c5416800 0x5595c5177b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f3a10af2840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5595c5416c00 0x5595c5178100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f3a10af2840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595c5107e00 con 0x5595c5416800 2023-10-11T09:45:58.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f3a10af2840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595c5150a00 con 0x5595c5416400 2023-10-11T09:45:58.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f3a10af2840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x5595c5150780 con 0x5595c5416c00 2023-10-11T09:45:58.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f541ad3d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf925c800 0x55caf8ff7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f541ad3d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf925c800 0x55caf8ff7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46354/0 (socket says 172.21.15.78:46354) 2023-10-11T09:45:58.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f541ad3d700 1 -- 172.21.15.78:0/3241453290 learned_addr learned my addr 172.21.15.78:0/3241453290 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f541a53c700 1 --2- 172.21.15.78:0/3241453290 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55caf925cc00 0x55caf8ff8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f541b53e700 1 --2- 172.21.15.78:0/3241453290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55caf925c400 0x55caf8ff7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f3a0faaf700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595c5416400 0x5595c5177600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f3a0f2ae700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595c5416800 0x5595c5177b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f3a0f2ae700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595c5416800 0x5595c5177b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46366/0 (socket says 172.21.15.78:46366) 2023-10-11T09:45:58.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f3a0f2ae700 1 -- 172.21.15.78:0/1881668193 learned_addr learned my addr 172.21.15.78:0/1881668193 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f541b53e700 1 -- 172.21.15.78:0/3241453290 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55caf925cc00 msgr2=0x55caf8ff8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f541b53e700 1 --2- 172.21.15.78:0/3241453290 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55caf925cc00 0x55caf8ff8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f541b53e700 1 -- 172.21.15.78:0/3241453290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf925c800 msgr2=0x55caf8ff7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f541b53e700 1 --2- 172.21.15.78:0/3241453290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf925c800 0x55caf8ff7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f541b53e700 1 -- 172.21.15.78:0/3241453290 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55caf8f72ea0 con 0x55caf925c400 2023-10-11T09:45:58.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f109f181840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b63cb9000 0x560b63d5eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f109f181840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b63cb9800 0x560b63d5f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f109f181840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b63cb9c00 0x560b63d5f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f0c147bf700 1 -- 172.21.15.78:0/299133903 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556d58d89800 msgr2=0x556d58e2f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f0c147bf700 1 --2- 172.21.15.78:0/299133903 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556d58d89800 0x556d58e2f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f0c147bf700 1 -- 172.21.15.78:0/299133903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d58d89c00 msgr2=0x556d58e2f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f0c147bf700 1 --2- 172.21.15.78:0/299133903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d58d89c00 0x556d58e2f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f0c147bf700 1 -- 172.21.15.78:0/299133903 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d58daab60 con 0x556d58d89000 2023-10-11T09:45:58.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f0c13fbe700 1 --2- 172.21.15.78:0/299133903 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556d58d89800 0x556d58e2f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:45:58.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f0c147bf700 1 --2- 172.21.15.78:0/299133903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d58d89000 0x556d58e2eb00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x556d58e573b0 tx=0x556d59b80f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55d648d177359fb4 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f109f181840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b63d38a00 con 0x560b63cb9c00 2023-10-11T09:45:58.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f109f181840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b63cefe00 con 0x560b63cb9000 2023-10-11T09:45:58.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f109f181840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b63d38640 con 0x560b63cb9800 2023-10-11T09:45:58.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f3a0f2ae700 1 -- 172.21.15.78:0/1881668193 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5595c5416c00 msgr2=0x5595c5178100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f3a0f2ae700 1 --2- 172.21.15.78:0/1881668193 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5595c5416c00 0x5595c5178100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f3a0f2ae700 1 -- 172.21.15.78:0/1881668193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595c5416400 msgr2=0x5595c5177600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f3a0f2ae700 1 --2- 172.21.15.78:0/1881668193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595c5416400 0x5595c5177600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f3a0f2ae700 1 -- 172.21.15.78:0/1881668193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595c50f2340 con 0x5595c5416800 2023-10-11T09:45:58.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f541b53e700 1 --2- 172.21.15.78:0/3241453290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55caf925c400 0x55caf8ff7600 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x55caf92507b0 tx=0x55caf9266c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10c6dd7f1cd45692 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7fabe8708700 1 -- 172.21.15.78:0/3728357160 <== osd.1 v2:172.21.15.78:6809/35734 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 66 out=81b] v66'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d605db3440 con 0x55d605372800 2023-10-11T09:45:58.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:4 9.36 9:6db6db6d::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55d605373000 con 0x55d605372800 2023-10-11T09:45:58.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7fd2acfc8840 1 -- 172.21.15.78:0/1926931258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec985b5000 msgr2=0x55ec9865a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7fd2acfc8840 1 --2- 172.21.15.78:0/1926931258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec985b5000 0x55ec9865a580 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x55ec9860be00 tx=0x55ec9871a060 comp rx=0 tx=0).stop 2023-10-11T09:45:58.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7fd2acfc8840 1 -- 172.21.15.78:0/1926931258 shutdown_connections 2023-10-11T09:45:58.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7fd2acfc8840 1 --2- 172.21.15.78:0/1926931258 >> v2:172.21.15.133:3300/0 conn(0x55ec985b5800 0x55ec9865b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7fd2acfc8840 1 --2- 172.21.15.78:0/1926931258 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ec985b5400 0x55ec9865ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7fd2acfc8840 1 --2- 172.21.15.78:0/1926931258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec985b5000 0x55ec9865a580 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7fd2acfc8840 1 -- 172.21.15.78:0/1926931258 >> 172.21.15.78:0/1926931258 conn(0x55ec985b4c00 msgr2=0x55ec986a0000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:58.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7fd2acfc8840 1 -- 172.21.15.78:0/1926931258 shutdown_connections 2023-10-11T09:45:58.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7fd2acfc8840 1 -- 172.21.15.78:0/1926931258 wait complete. 2023-10-11T09:45:58.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f0c11fba700 1 -- 172.21.15.78:0/299133903 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556d58db9600 con 0x556d58d89000 2023-10-11T09:45:58.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f0c11fba700 1 -- 172.21.15.78:0/299133903 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556d58dd4480 con 0x556d58d89000 2023-10-11T09:45:58.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f0c11fba700 1 -- 172.21.15.78:0/299133903 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556d58db91e0 con 0x556d58d89000 2023-10-11T09:45:58.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f109d93d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b63cb9000 0x560b63d5eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f109d93d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b63cb9000 0x560b63d5eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:39194/0 (socket says 172.21.15.78:39194) 2023-10-11T09:45:58.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f109d93d700 1 -- 172.21.15.78:0/27808337 learned_addr learned my addr 172.21.15.78:0/27808337 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f5416534700 1 -- 172.21.15.78:0/3241453290 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55caf8f81600 con 0x55caf925c400 2023-10-11T09:45:58.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f3a0f2ae700 1 --2- 172.21.15.78:0/1881668193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595c5416800 0x5595c5177b80 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x5595c5ec6870 tx=0x5595c5eb6c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38808c221ce135a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f109d13c700 1 --2- 172.21.15.78:0/27808337 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b63cb9800 0x560b63d5f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7fabe8708700 1 -- 172.21.15.78:0/3728357160 <== osd.1 v2:172.21.15.78:6809/35734 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 66 out=101b] v66'23 uv23 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d605db3440 con 0x55d605372800 2023-10-11T09:45:58.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7fabe9f4c840 20 datascan.forall_objects: OSD matched oid 100000001f9.00000000 2023-10-11T09:45:58.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7fabe9f4c840 10 datascan.operator(): handling object 100000001f9.0 2023-10-11T09:45:58.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f5416534700 1 -- 172.21.15.78:0/3241453290 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55caf9d38c00 con 0x55caf925c400 2023-10-11T09:45:58.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7f5416534700 1 -- 172.21.15.78:0/3241453290 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55caf8f811e0 con 0x55caf925c400 2023-10-11T09:45:58.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:5 9.36 9:6ee5e2c4:::100000001f9.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55d605373400 con 0x55d605372800 2023-10-11T09:45:58.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7fd2acfc8840 1 -- start start 2023-10-11T09:45:58.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.824+0000 7fd2acfc8840 1 Processor -- start 2023-10-11T09:45:58.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f0c16003840 1 -- 172.21.15.78:0/299133903 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556d58daa340 con 0x556d58d89000 2023-10-11T09:45:58.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f0c16003840 1 -- 172.21.15.78:0/299133903 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x556d58daa000 con 0x556d58d89000 2023-10-11T09:45:58.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f0c16003840 4 waiting for MDS map... 2023-10-11T09:45:58.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f109e13e700 1 --2- 172.21.15.78:0/27808337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b63cb9c00 0x560b63d5f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7fabe8708700 1 -- 172.21.15.78:0/3728357160 <== osd.1 v2:172.21.15.78:6809/35734 3 ==== osd_op_reply(5 100000001f9.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv23 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x55d605db3440 con 0x55d605372800 2023-10-11T09:45:58.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7fabe9f4c840 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55d605373800 0x55d6053a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:6 10.16 10:6ee5e2c4:::100000001f9.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55d605373c00 con 0x55d605373800 2023-10-11T09:45:58.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f541c581840 1 -- 172.21.15.78:0/3241453290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55caf925c400 msgr2=0x55caf8ff7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f541c581840 1 --2- 172.21.15.78:0/3241453290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55caf925c400 0x55caf8ff7600 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x55caf92507b0 tx=0x55caf9266c00 comp rx=0 tx=0).stop 2023-10-11T09:45:58.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f109d13c700 1 -- 172.21.15.78:0/27808337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b63cb9000 msgr2=0x560b63d5eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f109d13c700 1 --2- 172.21.15.78:0/27808337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b63cb9000 0x560b63d5eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f109d13c700 1 -- 172.21.15.78:0/27808337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b63cb9c00 msgr2=0x560b63d5f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f109d13c700 1 --2- 172.21.15.78:0/27808337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b63cb9c00 0x560b63d5f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f3a0aaa5700 1 -- 172.21.15.78:0/1881668193 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5595c5101080 con 0x5595c5416800 2023-10-11T09:45:58.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f109d13c700 1 -- 172.21.15.78:0/27808337 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b63cdab60 con 0x560b63cb9800 2023-10-11T09:45:58.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f3a0aaa5700 1 -- 172.21.15.78:0/1881668193 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5595c511c480 con 0x5595c5416800 2023-10-11T09:45:58.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f3a0aaa5700 1 -- 172.21.15.78:0/1881668193 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5595c5101600 con 0x5595c5416800 2023-10-11T09:45:58.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f541c581840 1 -- 172.21.15.78:0/3241453290 shutdown_connections 2023-10-11T09:45:58.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f541c581840 1 --2- 172.21.15.78:0/3241453290 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55caf925cc00 0x55caf8ff8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f541c581840 1 --2- 172.21.15.78:0/3241453290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf925c800 0x55caf8ff7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f541c581840 1 --2- 172.21.15.78:0/3241453290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55caf925c400 0x55caf8ff7600 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f541c581840 1 -- 172.21.15.78:0/3241453290 >> 172.21.15.78:0/3241453290 conn(0x55caf8f50c00 msgr2=0x55caf9031800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:58.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f0c11fba700 1 -- 172.21.15.78:0/299133903 <== mon.1 v2:172.21.15.133:3300/0 4 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x556d58dc2c40 con 0x556d58d89000 2023-10-11T09:45:58.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f0c11fba700 1 -- 172.21.15.78:0/299133903 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 43) v1 ==== 3118+0+0 (secure 0 0 0) 0x556d59084000 con 0x556d58d89000 2023-10-11T09:45:58.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f109d13c700 1 --2- 172.21.15.78:0/27808337 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b63cb9800 0x560b63d5f080 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x560b63dd1260 tx=0x560b64a9c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfb0dcf2ea33090f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7fabe7f07700 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55d605373800 0x55d6053a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f541c581840 1 -- 172.21.15.78:0/3241453290 shutdown_connections 2023-10-11T09:45:58.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f3a10af2840 1 -- 172.21.15.78:0/1881668193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595c5416800 msgr2=0x5595c5177b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f3a10af2840 1 --2- 172.21.15.78:0/1881668193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595c5416800 0x5595c5177b80 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x5595c5ec6870 tx=0x5595c5eb6c00 comp rx=0 tx=0).stop 2023-10-11T09:45:58.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f541c581840 1 -- 172.21.15.78:0/3241453290 wait complete. 2023-10-11T09:45:58.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f541c581840 1 Processor -- start 2023-10-11T09:45:58.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f541c581840 1 -- start start 2023-10-11T09:45:58.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f109b138700 1 -- 172.21.15.78:0/27808337 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560b63ce9080 con 0x560b63cb9800 2023-10-11T09:45:58.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f109b138700 1 -- 172.21.15.78:0/27808337 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560b64a98180 con 0x560b63cb9800 2023-10-11T09:45:58.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f109b138700 1 -- 172.21.15.78:0/27808337 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560b63ce9600 con 0x560b63cb9800 2023-10-11T09:45:58.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f109f181840 1 -- 172.21.15.78:0/27808337 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b63cdb1e0 con 0x560b63cb9800 2023-10-11T09:45:58.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f109f181840 1 -- 172.21.15.78:0/27808337 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({fsmap=0}) v3 -- 0x560b63cdab60 con 0x560b63cb9800 2023-10-11T09:45:58.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f109f181840 4 waiting for MDS map... 2023-10-11T09:45:58.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f109b138700 1 -- 172.21.15.78:0/27808337 <== mon.2 v2:172.21.15.133:3301/0 4 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x560b63cf2c40 con 0x560b63cb9800 2023-10-11T09:45:58.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f109b138700 1 -- 172.21.15.78:0/27808337 <== mon.2 v2:172.21.15.133:3301/0 5 ==== fsmap(e 43) v1 ==== 3118+0+0 (secure 0 0 0) 0x560b63cfaf00 con 0x560b63cb9800 2023-10-11T09:45:58.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f109f181840 4 Got MDS map 43 2023-10-11T09:45:58.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f109f181840 4 datascan.main: Using metadata pool output 2023-10-11T09:45:58.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f109f181840 4 datascan.main: connecting to RADOS... 2023-10-11T09:45:58.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7fabe7f07700 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55d605373800 0x55d6053a4580 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7fabe7f07700 1 -- 172.21.15.78:0/3728357160 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== osd_op_reply(6 100000001f9.00000000 [getxattr (30) out=30b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55d605db2fc0 con 0x55d605373800 2023-10-11T09:45:58.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7fabe9f4c840 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-10-11T09:45:58.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7fabe9f4c840 10 datascan.inject_with_backtrace: inode: 0x0x100000001f9 2023-10-11T09:45:58.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7fabe9f4c840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/4 2023-10-11T09:45:58.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7fabe9f4c840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=4 2023-10-11T09:45:58.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7fabe9f4c840 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55d605372c00 0x55d6053a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:7 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55d605373000 con 0x55d605372c00 2023-10-11T09:45:58.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7fd2acfc8840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec985b5800 0x55ec9865bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7fd2acfc8840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec985b5400 0x55ec9865b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7fd2acfc8840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ec985b5000 0x55ec9865b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7fd2acfc8840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec98634a00 con 0x55ec985b5800 2023-10-11T09:45:58.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7fd2acfc8840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec98634640 con 0x55ec985b5400 2023-10-11T09:45:58.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7fd2acfc8840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ec98634500 con 0x55ec985b5000 2023-10-11T09:45:58.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7fd2ab784700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec985b5800 0x55ec9865bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7fd2ab784700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec985b5800 0x55ec9865bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46398/0 (socket says 172.21.15.78:46398) 2023-10-11T09:45:58.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7fd2ab784700 1 -- 172.21.15.78:0/3778569310 learned_addr learned my addr 172.21.15.78:0/3778569310 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7fd2aaf83700 1 --2- 172.21.15.78:0/3778569310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec985b5400 0x55ec9865b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7fd2abf85700 1 --2- 172.21.15.78:0/3778569310 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ec985b5000 0x55ec9865b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7fd2ab784700 1 -- 172.21.15.78:0/3778569310 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ec985b5000 msgr2=0x55ec9865b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7fd2ab784700 1 --2- 172.21.15.78:0/3778569310 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ec985b5000 0x55ec9865b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7fd2ab784700 1 -- 172.21.15.78:0/3778569310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec985b5400 msgr2=0x55ec9865b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7fd2ab784700 1 --2- 172.21.15.78:0/3778569310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec985b5400 0x55ec9865b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7fd2ab784700 1 -- 172.21.15.78:0/3778569310 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec985d6b60 con 0x55ec985b5800 2023-10-11T09:45:58.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7fd2ab784700 1 --2- 172.21.15.78:0/3778569310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec985b5800 0x55ec9865bb80 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x55ec98683380 tx=0x55ec9868b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8483af6cb0d78e55 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f0c16003840 4 Got MDS map 43 2023-10-11T09:45:58.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f0c16003840 4 datascan.main: Using metadata pool output 2023-10-11T09:45:58.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f0c16003840 4 datascan.main: connecting to RADOS... 2023-10-11T09:45:58.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f0c16003840 1 Processor -- start 2023-10-11T09:45:58.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f0c16003840 1 -- start start 2023-10-11T09:45:58.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f3a10af2840 1 -- 172.21.15.78:0/1881668193 shutdown_connections 2023-10-11T09:45:58.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7fabe8f09700 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55d605372c00 0x55d6053a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f3a10af2840 1 --2- 172.21.15.78:0/1881668193 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5595c5416c00 0x5595c5178100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f3a10af2840 1 --2- 172.21.15.78:0/1881668193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595c5416800 0x5595c5177b80 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f3a10af2840 1 --2- 172.21.15.78:0/1881668193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595c5416400 0x5595c5177600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f3a10af2840 1 -- 172.21.15.78:0/1881668193 >> 172.21.15.78:0/1881668193 conn(0x5595c5416000 msgr2=0x5595c51c5800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:58.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f3a10af2840 1 -- 172.21.15.78:0/1881668193 shutdown_connections 2023-10-11T09:45:58.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f3a10af2840 1 -- 172.21.15.78:0/1881668193 wait complete. 2023-10-11T09:45:58.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7f109f181840 1 Processor -- start 2023-10-11T09:45:58.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7fd2a8f7f700 1 -- 172.21.15.78:0/3778569310 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ec985e5080 con 0x55ec985b5800 2023-10-11T09:45:58.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7fd2a8f7f700 1 -- 172.21.15.78:0/3778569310 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ec98600900 con 0x55ec985b5800 2023-10-11T09:45:58.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7fd2a8f7f700 1 -- 172.21.15.78:0/3778569310 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ec985e5600 con 0x55ec985b5800 2023-10-11T09:45:58.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7fd2acfc8840 1 -- 172.21.15.78:0/3778569310 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec985d71e0 con 0x55ec985b5800 2023-10-11T09:45:58.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7fd2acfc8840 1 -- 172.21.15.78:0/3778569310 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55ec985d6b60 con 0x55ec985b5800 2023-10-11T09:45:58.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.828+0000 7fd2acfc8840 4 waiting for MDS map... 2023-10-11T09:45:58.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f0c16003840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556d5908c800 0x556d58e2fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f0c16003840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d5908cc00 0x556d58e30100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f0c16003840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d5908d000 0x556d58e30680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f0c16003840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d58dbfe00 con 0x556d5908d000 2023-10-11T09:45:58.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f0c16003840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d58e08a00 con 0x556d5908cc00 2023-10-11T09:45:58.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f0c16003840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x556d58e08500 con 0x556d5908c800 2023-10-11T09:45:58.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7fabe8f09700 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55d605372c00 0x55d6053a4b00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f541c581840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55caf925c400 0x55caf8ff9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f541c581840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55caf925c800 0x55caf8ff8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f541c581840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf925cc00 0x55caf8ff8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f541c581840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caf8fd03c0 con 0x55caf925cc00 2023-10-11T09:45:58.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f541c581840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caf8fd0280 con 0x55caf925c800 2023-10-11T09:45:58.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f541c581840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55caf8fd0140 con 0x55caf925c400 2023-10-11T09:45:58.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f541ad3d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55caf925c800 0x55caf8ff8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f109f181840 1 -- start start 2023-10-11T09:45:58.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7fabe8f09700 1 -- 172.21.15.78:0/3728357160 <== osd.7 v2:172.21.15.133:6824/35901 1 ==== osd_op_reply(7 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x55d605db3200 con 0x55d605372c00 2023-10-11T09:45:58.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7fabe9f4c840 4 datascan.get_frag_of: Unexpected error on '10000000000.00000000': (2) No such file or directory 2023-10-11T09:45:58.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7fabe9f4c840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/4, will insert to root 2023-10-11T09:45:58.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:8 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55d6053a8000 con 0x55d605372c00 2023-10-11T09:45:58.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f541ad3d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55caf925c800 0x55caf8ff8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:39228/0 (socket says 172.21.15.78:39228) 2023-10-11T09:45:58.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f541ad3d700 1 -- 172.21.15.78:0/2226872998 learned_addr learned my addr 172.21.15.78:0/2226872998 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f541a53c700 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf925cc00 0x55caf8ff8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f541b53e700 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55caf925c400 0x55caf8ff9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f3a10af2840 1 Processor -- start 2023-10-11T09:45:58.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7fd2a8f7f700 1 -- 172.21.15.78:0/3778569310 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x55ec985eec40 con 0x55ec985b5800 2023-10-11T09:45:58.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f0c13fbe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d5908d000 0x556d58e30680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f0c13fbe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d5908d000 0x556d58e30680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46410/0 (socket says 172.21.15.78:46410) 2023-10-11T09:45:58.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f0c13fbe700 1 -- 172.21.15.78:0/411577514 learned_addr learned my addr 172.21.15.78:0/411577514 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f0c147bf700 1 --2- 172.21.15.78:0/411577514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d5908cc00 0x556d58e30100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f0c14fc0700 1 --2- 172.21.15.78:0/411577514 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556d5908c800 0x556d58e2fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7fd2a8f7f700 1 -- 172.21.15.78:0/3778569310 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 43) v1 ==== 3118+0+0 (secure 0 0 0) 0x55ec985f6f00 con 0x55ec985b5800 2023-10-11T09:45:58.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f109f181840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b63ff0400 0x560b63d5fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7fd2acfc8840 4 Got MDS map 43 2023-10-11T09:45:58.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7fabe8f09700 1 -- 172.21.15.78:0/3728357160 <== osd.7 v2:172.21.15.133:6824/35901 2 ==== osd_op_reply(8 10000000000.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x55d605db3200 con 0x55d605372c00 2023-10-11T09:45:58.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7fabe9f4c840 10 datascan.find_or_create_dirfrag: read_version = 0 2023-10-11T09:45:58.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:9 8.10 8:0cf6ad0f:::10000000000.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55d6053a8400 con 0x55d605372c00 2023-10-11T09:45:58.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f109f181840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b63ff0800 0x560b63d60100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f109f181840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b63ff0c00 0x560b63d60680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f109f181840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b63d38500 con 0x560b63ff0c00 2023-10-11T09:45:58.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f109f181840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b63d383c0 con 0x560b63ff0800 2023-10-11T09:45:58.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f109f181840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b63d38280 con 0x560b63ff0400 2023-10-11T09:45:58.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7fd2acfc8840 4 datascan.main: Using metadata pool output 2023-10-11T09:45:58.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7fd2acfc8840 4 datascan.main: connecting to RADOS... 2023-10-11T09:45:58.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f109d93d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b63ff0800 0x560b63d60100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f109d93d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b63ff0800 0x560b63d60100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:39248/0 (socket says 172.21.15.78:39248) 2023-10-11T09:45:58.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f0c13fbe700 1 -- 172.21.15.78:0/411577514 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556d5908c800 msgr2=0x556d58e2fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f0c13fbe700 1 --2- 172.21.15.78:0/411577514 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556d5908c800 0x556d58e2fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f0c13fbe700 1 -- 172.21.15.78:0/411577514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d5908cc00 msgr2=0x556d58e30100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f0c13fbe700 1 --2- 172.21.15.78:0/411577514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d5908cc00 0x556d58e30100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f0c13fbe700 1 -- 172.21.15.78:0/411577514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d58daaea0 con 0x556d5908d000 2023-10-11T09:45:58.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f109d93d700 1 -- 172.21.15.78:0/1468692382 learned_addr learned my addr 172.21.15.78:0/1468692382 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f3a10af2840 1 -- start start 2023-10-11T09:45:58.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f541ad3d700 1 -- 172.21.15.78:0/2226872998 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55caf925c400 msgr2=0x55caf8ff9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f541ad3d700 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55caf925c400 0x55caf8ff9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f541ad3d700 1 -- 172.21.15.78:0/2226872998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf925cc00 msgr2=0x55caf8ff8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f541ad3d700 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf925cc00 0x55caf8ff8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f541ad3d700 1 -- 172.21.15.78:0/2226872998 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55caf8f72b60 con 0x55caf925c800 2023-10-11T09:45:58.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f541b53e700 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55caf925c400 0x55caf8ff9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:45:58.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f109e13e700 1 --2- 172.21.15.78:0/1468692382 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b63ff0400 0x560b63d5fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f109d13c700 1 --2- 172.21.15.78:0/1468692382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b63ff0c00 0x560b63d60680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f541ad3d700 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55caf925c800 0x55caf8ff8c00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x55caf92514a0 tx=0x55caf9d2a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f0c13fbe700 1 --2- 172.21.15.78:0/411577514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d5908d000 0x556d58e30680 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x556d58e57290 tx=0x556d58e5f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0f02a69eb402d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f3a10af2840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5595c5416400 0x5595c5179180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f3a10af2840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595c5416800 0x5595c5178c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f3a10af2840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595c5416c00 0x5595c5178680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f3a10af2840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595c5150500 con 0x5595c5416800 2023-10-11T09:45:58.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f109d93d700 1 -- 172.21.15.78:0/1468692382 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b63ff0400 msgr2=0x560b63d5fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f109d93d700 1 --2- 172.21.15.78:0/1468692382 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b63ff0400 0x560b63d5fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f109d93d700 1 -- 172.21.15.78:0/1468692382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b63ff0c00 msgr2=0x560b63d60680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f109d93d700 1 --2- 172.21.15.78:0/1468692382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b63ff0c00 0x560b63d60680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f109d93d700 1 -- 172.21.15.78:0/1468692382 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b63cda340 con 0x560b63ff0800 2023-10-11T09:45:58.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f109e13e700 1 --2- 172.21.15.78:0/1468692382 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b63ff0400 0x560b63d5fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:45:58.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7fd2acfc8840 1 Processor -- start 2023-10-11T09:45:58.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f3a10af2840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595c51503c0 con 0x5595c5416c00 2023-10-11T09:45:58.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f3a10af2840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x5595c5150280 con 0x5595c5416400 2023-10-11T09:45:58.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f109d13c700 1 --2- 172.21.15.78:0/1468692382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b63ff0c00 0x560b63d60680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:45:58.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f5414d31700 1 -- 172.21.15.78:0/2226872998 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55caf8f80f20 con 0x55caf925c800 2023-10-11T09:45:58.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f0c0ffb6700 1 -- 172.21.15.78:0/411577514 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556d58db9080 con 0x556d5908d000 2023-10-11T09:45:58.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f0c0ffb6700 1 -- 172.21.15.78:0/411577514 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556d58eeed80 con 0x556d5908d000 2023-10-11T09:45:58.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f0c0ffb6700 1 -- 172.21.15.78:0/411577514 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556d58db9600 con 0x556d5908d000 2023-10-11T09:45:58.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f5414d31700 1 -- 172.21.15.78:0/2226872998 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55caf9d38a80 con 0x55caf925c800 2023-10-11T09:45:58.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f5414d31700 1 -- 172.21.15.78:0/2226872998 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55caf8f81600 con 0x55caf925c800 2023-10-11T09:45:58.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f541c581840 1 -- 172.21.15.78:0/2226872998 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55caf8f72340 con 0x55caf925c800 2023-10-11T09:45:58.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7fd2acfc8840 1 -- start start 2023-10-11T09:45:58.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f3a0eaad700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595c5416c00 0x5595c5178680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f541c581840 1 -- 172.21.15.78:0/2226872998 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55caf8f73520 con 0x55caf925c800 2023-10-11T09:45:58.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f541c581840 4 datascan.init: resolving metadata pool 8 2023-10-11T09:45:58.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f3a0eaad700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595c5416c00 0x5595c5178680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:39264/0 (socket says 172.21.15.78:39264) 2023-10-11T09:45:58.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f3a0eaad700 1 -- 172.21.15.78:0/4084151273 learned_addr learned my addr 172.21.15.78:0/4084151273 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f109d93d700 1 --2- 172.21.15.78:0/1468692382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b63ff0800 0x560b63d60100 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x560b63fae600 tx=0x560b63fb0c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e99d4bcd8545d3ab server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.832+0000 7f3a0f2ae700 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595c5416800 0x5595c5178c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f0c16003840 1 -- 172.21.15.78:0/411577514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d5908d000 msgr2=0x556d58e30680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f0c16003840 1 --2- 172.21.15.78:0/411577514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d5908d000 0x556d58e30680 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x556d58e57290 tx=0x556d58e5f380 comp rx=0 tx=0).stop 2023-10-11T09:45:58.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f3a0faaf700 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5595c5416400 0x5595c5179180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f3a0eaad700 1 -- 172.21.15.78:0/4084151273 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5595c5416400 msgr2=0x5595c5179180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f3a0eaad700 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5595c5416400 0x5595c5179180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f3a0eaad700 1 -- 172.21.15.78:0/4084151273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595c5416800 msgr2=0x5595c5178c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f3a0eaad700 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595c5416800 0x5595c5178c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f3a0eaad700 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595c50f2000 con 0x5595c5416c00 2023-10-11T09:45:58.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f0c16003840 1 -- 172.21.15.78:0/411577514 shutdown_connections 2023-10-11T09:45:58.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f0c16003840 1 --2- 172.21.15.78:0/411577514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d5908d000 0x556d58e30680 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f0c16003840 1 --2- 172.21.15.78:0/411577514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d5908cc00 0x556d58e30100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f0c16003840 1 --2- 172.21.15.78:0/411577514 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556d5908c800 0x556d58e2fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f0c16003840 1 -- 172.21.15.78:0/411577514 >> 172.21.15.78:0/411577514 conn(0x556d5908c400 msgr2=0x556d58e7d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:58.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f4f03f79840 1 -- start start 2023-10-11T09:45:58.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f4f03f79840 1 Processor -- start 2023-10-11T09:45:58.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f3a0faaf700 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5595c5416400 0x5595c5179180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:45:58.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f0c16003840 1 -- 172.21.15.78:0/411577514 shutdown_connections 2023-10-11T09:45:58.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7fabe8f09700 1 -- 172.21.15.78:0/3728357160 <== osd.7 v2:172.21.15.133:6824/35901 3 ==== osd_op_reply(9 10000000000.00000000 [create,omap-set-header] v66'16 uv16 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55d605db3200 con 0x55d605372c00 2023-10-11T09:45:58.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7fabe9f4c840 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x10000000000 2023-10-11T09:45:58.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:10 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55d6053a8800 con 0x55d605372c00 2023-10-11T09:45:58.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f0c16003840 1 -- 172.21.15.78:0/411577514 wait complete. 2023-10-11T09:45:58.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f3a0eaad700 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595c5416c00 0x5595c5178680 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x5595c5ec7530 tx=0x5595c5eb2060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7fd2acfc8840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec9943e400 0x55ec9865ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7fd2acfc8840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ec9943e800 0x55ec9865a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7fd2acfc8840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec9943ec00 0x55ec9865c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7fd2acfc8840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec985ebe00 con 0x55ec9943ec00 2023-10-11T09:45:58.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7fd2acfc8840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec98634500 con 0x55ec9943e400 2023-10-11T09:45:58.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f3a092a2700 1 -- 172.21.15.78:0/4084151273 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5595c51011e0 con 0x5595c5416c00 2023-10-11T09:45:58.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f3a092a2700 1 -- 172.21.15.78:0/4084151273 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5595c5ec4a80 con 0x5595c5416c00 2023-10-11T09:45:58.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f3a092a2700 1 -- 172.21.15.78:0/4084151273 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5595c5100f20 con 0x5595c5416c00 2023-10-11T09:45:58.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5595c50f2ea0 con 0x5595c5416c00 2023-10-11T09:45:58.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5595c50f3520 con 0x5595c5416c00 2023-10-11T09:45:58.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f3a10af2840 4 datascan.init: resolving metadata pool 8 2023-10-11T09:45:58.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f1099134700 1 -- 172.21.15.78:0/1468692382 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560b63ce9080 con 0x560b63ff0800 2023-10-11T09:45:58.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f1099134700 1 -- 172.21.15.78:0/1468692382 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560b63d04180 con 0x560b63ff0800 2023-10-11T09:45:58.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f1099134700 1 -- 172.21.15.78:0/1468692382 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560b63ce9600 con 0x560b63ff0800 2023-10-11T09:45:58.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7fabe8f09700 1 -- 172.21.15.78:0/3728357160 <== osd.7 v2:172.21.15.133:6824/35901 4 ==== osd_op_reply(10 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55d605db3200 con 0x55d605372c00 2023-10-11T09:45:58.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7fabe9f4c840 20 datascan.read_dentry: oid=10000000000.00000000 dname=4 frag=*, r=0 2023-10-11T09:45:58.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7fabe9f4c840 20 datascan.read_dentry: 4_head not found in result 2023-10-11T09:45:58.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7fabe9f4c840 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f9 at 0x0x10000000000/4 with size=6291456 bytes 2023-10-11T09:45:58.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:11 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55d6053a8c00 con 0x55d605372c00 2023-10-11T09:45:58.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7fd2abf85700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec9943e400 0x55ec9865ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7fd2acfc8840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ec98634640 con 0x55ec9943e800 2023-10-11T09:45:58.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7fd2abf85700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec9943e400 0x55ec9865ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:39274/0 (socket says 172.21.15.78:39274) 2023-10-11T09:45:58.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7fd2abf85700 1 -- 172.21.15.78:0/3928025129 learned_addr learned my addr 172.21.15.78:0/3928025129 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7fd2aaf83700 1 --2- 172.21.15.78:0/3928025129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec9943ec00 0x55ec9865c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f0c16003840 1 Processor -- start 2023-10-11T09:45:58.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f0c16003840 1 -- start start 2023-10-11T09:45:58.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7fd2ab784700 1 --2- 172.21.15.78:0/3928025129 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ec9943e800 0x55ec9865a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f4f03f79840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ab4799a400 0x55ab46c59600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f4f03f79840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab46bb3c00 0x55ab46c58b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f4f03f79840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab4799a800 0x55ab46c59b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f4f03f79840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab46c32a00 con 0x55ab46bb3c00 2023-10-11T09:45:58.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f4f03f79840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab46be9e00 con 0x55ab4799a800 2023-10-11T09:45:58.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f4f03f79840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ab46c32280 con 0x55ab4799a400 2023-10-11T09:45:58.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f109f181840 1 -- 172.21.15.78:0/1468692382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b63ff0800 msgr2=0x560b63d60100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f109f181840 1 --2- 172.21.15.78:0/1468692382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b63ff0800 0x560b63d60100 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x560b63fae600 tx=0x560b63fb0c00 comp rx=0 tx=0).stop 2023-10-11T09:45:58.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f4f01f34700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab46bb3c00 0x55ab46c58b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f4f01f34700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab46bb3c00 0x55ab46c58b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46462/0 (socket says 172.21.15.78:46462) 2023-10-11T09:45:58.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f4f01f34700 1 -- 172.21.15.78:0/2851354210 learned_addr learned my addr 172.21.15.78:0/2851354210 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f109f181840 1 -- 172.21.15.78:0/1468692382 shutdown_connections 2023-10-11T09:45:58.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f109f181840 1 --2- 172.21.15.78:0/1468692382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b63ff0c00 0x560b63d60680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f109f181840 1 --2- 172.21.15.78:0/1468692382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b63ff0800 0x560b63d60100 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f109f181840 1 --2- 172.21.15.78:0/1468692382 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b63ff0400 0x560b63d5fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f109f181840 1 -- 172.21.15.78:0/1468692382 >> 172.21.15.78:0/1468692382 conn(0x560b63ff0000 msgr2=0x560b63d75800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:58.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f109f181840 1 -- 172.21.15.78:0/1468692382 shutdown_connections 2023-10-11T09:45:58.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7fd2abf85700 1 -- 172.21.15.78:0/3928025129 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ec9943e800 msgr2=0x55ec9865a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7fd2abf85700 1 --2- 172.21.15.78:0/3928025129 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ec9943e800 0x55ec9865a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7fd2abf85700 1 -- 172.21.15.78:0/3928025129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec9943ec00 msgr2=0x55ec9865c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7fd2abf85700 1 --2- 172.21.15.78:0/3928025129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec9943ec00 0x55ec9865c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7fd2abf85700 1 -- 172.21.15.78:0/3928025129 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec985d6340 con 0x55ec9943e400 2023-10-11T09:45:58.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f0c16003840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d5908c800 0x556d58e2fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f4f02735700 1 --2- 172.21.15.78:0/2851354210 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ab4799a400 0x55ab46c59600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f4f01f34700 1 -- 172.21.15.78:0/2851354210 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ab4799a400 msgr2=0x55ab46c59600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f4f01f34700 1 --2- 172.21.15.78:0/2851354210 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ab4799a400 0x55ab46c59600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f109f181840 1 -- 172.21.15.78:0/1468692382 wait complete. 2023-10-11T09:45:58.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f4f01f34700 1 -- 172.21.15.78:0/2851354210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab4799a800 msgr2=0x55ab46c59b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:58.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f5414d31700 1 -- 172.21.15.78:0/2226872998 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x55caf8fae840 con 0x55caf925c800 2023-10-11T09:45:58.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f4f01f34700 1 --2- 172.21.15.78:0/2851354210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab4799a800 0x55ab46c59b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f4f01f34700 1 -- 172.21.15.78:0/2851354210 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab46bd4000 con 0x55ab46bb3c00 2023-10-11T09:45:58.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f0c16003840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d5908cc00 0x556d58e30100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7fd2abf85700 1 --2- 172.21.15.78:0/3928025129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec9943e400 0x55ec9865ab00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x55ec994348d0 tx=0x55ec988acc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5038b511350d62cc server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f0c14fc0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d5908c800 0x556d58e2fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f0c16003840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556d5908d000 0x556d58e30680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f0c14fc0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d5908c800 0x556d58e2fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46476/0 (socket says 172.21.15.78:46476) 2023-10-11T09:45:58.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f4f01f34700 1 --2- 172.21.15.78:0/2851354210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab46bb3c00 0x55ab46c58b00 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x55ab46ccb260 tx=0x55ab47990060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e18699cb11a3ee8 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7fabe8f09700 1 -- 172.21.15.78:0/3728357160 <== osd.7 v2:172.21.15.133:6824/35901 5 ==== osd_op_reply(11 10000000000.00000000 [omap-set-vals] v66'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d605db3200 con 0x55d605372c00 2023-10-11T09:45:58.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7fabe9f4c840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/4 pointing to 0x0x100000001f9 2023-10-11T09:45:58.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7fabe9f4c840 10 datascan.inject_with_backtrace: backptr: 0x0x1/subdir 2023-10-11T09:45:58.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7fabe9f4c840 20 datascan.get_frag_of: dirino=0x1 target_dname=subdir 2023-10-11T09:45:58.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:12 8.1f 8:ff5b34d6:::1.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55d6053a9000 con 0x55d60521b800 2023-10-11T09:45:58.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7fd2a6f7b700 1 -- 172.21.15.78:0/3928025129 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ec985e5080 con 0x55ec9943e400 2023-10-11T09:45:58.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7fd2a6f7b700 1 -- 172.21.15.78:0/3928025129 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ec9939cc00 con 0x55ec9943e400 2023-10-11T09:45:58.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7fd2a6f7b700 1 -- 172.21.15.78:0/3928025129 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ec985e5600 con 0x55ec9943e400 2023-10-11T09:45:58.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f5414d31700 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55caf92d2400 0x55caf8ff9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f5414d31700 1 -- 172.21.15.78:0/2226872998 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x55caf8f8aa80 con 0x55caf925c800 2023-10-11T09:45:58.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f109f181840 1 Processor -- start 2023-10-11T09:45:58.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f3a092a2700 1 -- 172.21.15.78:0/4084151273 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x5595c512e840 con 0x5595c5416c00 2023-10-11T09:45:58.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f3a092a2700 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5595c5417800 0x5595c5179700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f3a092a2700 1 -- 172.21.15.78:0/4084151273 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x5595c510aa80 con 0x5595c5416c00 2023-10-11T09:45:58.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f541c581840 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-10-11T09:45:58.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f541c581840 4 datascan.main: data pool 'cephfs_data' has ID 9 2023-10-11T09:45:58.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f541c581840 4 datascan.main: opening data pool 'cephfs_data' 2023-10-11T09:45:58.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f541c581840 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55caf925d000 0x55caf8ff8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f541c581840 1 -- 172.21.15.78:0/2226872998 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:1 8.14 8:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55caf925d400 con 0x55caf925d000 2023-10-11T09:45:58.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f541a53c700 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55caf925d000 0x55caf8ff8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f109f181840 1 -- start start 2023-10-11T09:45:58.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f109f181840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b63ff0400 0x560b63d5fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f109f181840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b63ff0800 0x560b63d60100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f109f181840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b63ff0c00 0x560b63d60680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f109f181840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b63d38280 con 0x560b63ff0c00 2023-10-11T09:45:58.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f109f181840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b63d38500 con 0x560b63ff0400 2023-10-11T09:45:58.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f109f181840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b63d38780 con 0x560b63ff0800 2023-10-11T09:45:58.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7fabe8f09700 1 -- 172.21.15.78:0/3728357160 <== osd.0 v2:172.21.15.78:6800/35735 2 ==== osd_op_reply(12 1.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x55d605db3200 con 0x55d60521b800 2023-10-11T09:45:58.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7fabe9f4c840 4 datascan.get_frag_of: Unexpected error on '1.00000000': (2) No such file or directory 2023-10-11T09:45:58.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7fabe9f4c840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x1/subdir, will insert to root 2023-10-11T09:45:58.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:13 8.1f 8:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55d6053a9400 con 0x55d60521b800 2023-10-11T09:45:58.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7fabe8f09700 1 -- 172.21.15.78:0/3728357160 <== osd.0 v2:172.21.15.78:6800/35735 3 ==== osd_op_reply(13 1.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x55d605db3200 con 0x55d60521b800 2023-10-11T09:45:58.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7fabe9f4c840 10 datascan.find_or_create_dirfrag: read_version = 0 2023-10-11T09:45:58.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:14 8.1f 8:ff5b34d6:::1.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55d6053a9800 con 0x55d60521b800 2023-10-11T09:45:58.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7fd2acfc8840 1 -- 172.21.15.78:0/3928025129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec9943e400 msgr2=0x55ec9865ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7fd2acfc8840 1 --2- 172.21.15.78:0/3928025129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec9943e400 0x55ec9865ab00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x55ec994348d0 tx=0x55ec988acc00 comp rx=0 tx=0).stop 2023-10-11T09:45:58.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7fd2acfc8840 1 -- 172.21.15.78:0/3928025129 shutdown_connections 2023-10-11T09:45:58.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7fd2acfc8840 1 --2- 172.21.15.78:0/3928025129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec9943ec00 0x55ec9865c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7fd2acfc8840 1 --2- 172.21.15.78:0/3928025129 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ec9943e800 0x55ec9865a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7fd2acfc8840 1 --2- 172.21.15.78:0/3928025129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec9943e400 0x55ec9865ab00 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7fd2acfc8840 1 -- 172.21.15.78:0/3928025129 >> 172.21.15.78:0/3928025129 conn(0x55ec985b4c00 msgr2=0x55ec986a1800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:58.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.836+0000 7f0c14fc0700 1 -- 172.21.15.78:0/1160862806 learned_addr learned my addr 172.21.15.78:0/1160862806 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d58e083c0 con 0x556d5908c800 2023-10-11T09:45:58.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d58e08280 con 0x556d5908cc00 2023-10-11T09:45:58.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x556d58e08140 con 0x556d5908d000 2023-10-11T09:45:58.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f0c147bf700 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d5908cc00 0x556d58e30100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f0c13fbe700 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556d5908d000 0x556d58e30680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f0c147bf700 1 -- 172.21.15.78:0/1160862806 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556d5908d000 msgr2=0x556d58e30680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f0c147bf700 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556d5908d000 0x556d58e30680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f0c147bf700 1 -- 172.21.15.78:0/1160862806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d5908c800 msgr2=0x556d58e2fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f0c147bf700 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d5908c800 0x556d58e2fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f0c147bf700 1 -- 172.21.15.78:0/1160862806 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d58dab1e0 con 0x556d5908cc00 2023-10-11T09:45:58.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f4efff30700 1 -- 172.21.15.78:0/2851354210 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ab46be2f20 con 0x55ab46bb3c00 2023-10-11T09:45:58.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f4efff30700 1 -- 172.21.15.78:0/2851354210 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ab46ea4000 con 0x55ab46bb3c00 2023-10-11T09:45:58.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f4efff30700 1 -- 172.21.15.78:0/2851354210 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ab46be3080 con 0x55ab46bb3c00 2023-10-11T09:45:58.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f109d93d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b63ff0800 0x560b63d60100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f109d93d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b63ff0800 0x560b63d60100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:53838/0 (socket says 172.21.15.78:53838) 2023-10-11T09:45:58.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f109d93d700 1 -- 172.21.15.78:0/1612535802 learned_addr learned my addr 172.21.15.78:0/1612535802 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f3a0faaf700 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5595c5417800 0x5595c5179700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f541b53e700 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55caf92d2400 0x55caf8ff9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f541a53c700 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55caf925d000 0x55caf8ff8100 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f4f03f79840 1 -- 172.21.15.78:0/2851354210 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ab46bd4340 con 0x55ab46bb3c00 2023-10-11T09:45:58.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f4f03f79840 1 -- 172.21.15.78:0/2851354210 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55ab46bd51e0 con 0x55ab46bb3c00 2023-10-11T09:45:58.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f4f03f79840 4 waiting for MDS map... 2023-10-11T09:45:58.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f3a10af2840 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-10-11T09:45:58.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f3a10af2840 4 datascan.main: data pool 'cephfs_data' has ID 9 2023-10-11T09:45:58.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f3a10af2840 4 datascan.main: opening data pool 'cephfs_data' 2023-10-11T09:45:58.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f3a10af2840 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x5595c5417c00 0x5595c5178100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:1 8.14 8:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c552e000 con 0x5595c5417c00 2023-10-11T09:45:58.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f109e13e700 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b63ff0400 0x560b63d5fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f109d13c700 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b63ff0c00 0x560b63d60680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f0c147bf700 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d5908cc00 0x556d58e30100 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x556d590a1320 tx=0x556d59b80ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7fd2acfc8840 1 -- 172.21.15.78:0/3928025129 shutdown_connections 2023-10-11T09:45:58.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f3a0f2ae700 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x5595c5417c00 0x5595c5178100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f541a53c700 1 -- 172.21.15.78:0/2226872998 <== osd.4 v2:172.21.15.133:6808/35898 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv97 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x55caf9246fc0 con 0x55caf925d000 2023-10-11T09:45:58.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f541c581840 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55caf925d800 0x55caf8ff7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f541c581840 1 -- 172.21.15.78:0/2226872998 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:2 8.1f 8:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55caf92d2800 con 0x55caf925d800 2023-10-11T09:45:58.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f109d93d700 1 -- 172.21.15.78:0/1612535802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b63ff0400 msgr2=0x560b63d5fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f109d93d700 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b63ff0400 0x560b63d5fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f109d93d700 1 -- 172.21.15.78:0/1612535802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b63ff0c00 msgr2=0x560b63d60680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:58.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f109d93d700 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b63ff0c00 0x560b63d60680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:58.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f109d93d700 1 -- 172.21.15.78:0/1612535802 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b63cda000 con 0x560b63ff0800 2023-10-11T09:45:58.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f4efff30700 1 -- 172.21.15.78:0/2851354210 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x55ab46becc40 con 0x55ab46bb3c00 2023-10-11T09:45:58.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f4efff30700 1 -- 172.21.15.78:0/2851354210 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 43) v1 ==== 3118+0+0 (secure 0 0 0) 0x55ab46bf4f00 con 0x55ab46bb3c00 2023-10-11T09:45:58.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f4f03f79840 4 Got MDS map 43 2023-10-11T09:45:58.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7fd2acfc8840 1 -- 172.21.15.78:0/3928025129 wait complete. 2023-10-11T09:45:58.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f0c0e7b3700 1 -- 172.21.15.78:0/1160862806 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556d58db8f20 con 0x556d5908cc00 2023-10-11T09:45:58.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f0c0e7b3700 1 -- 172.21.15.78:0/1160862806 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556d58eef500 con 0x556d5908cc00 2023-10-11T09:45:58.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f0c0e7b3700 1 -- 172.21.15.78:0/1160862806 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556d58db9080 con 0x556d5908cc00 2023-10-11T09:45:58.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f3a0f2ae700 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x5595c5417c00 0x5595c5178100 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f3a0faaf700 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5595c5417800 0x5595c5179700 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x5595c51d89c0 tx=0x5595c5eb6e40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556d58daaea0 con 0x556d5908cc00 2023-10-11T09:45:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7fabe8f09700 1 -- 172.21.15.78:0/3728357160 <== osd.0 v2:172.21.15.78:6800/35735 4 ==== osd_op_reply(14 1.00000000 [create,omap-set-header] v66'11 uv11 ondisk = 0) v8 ==== 196+0+0 (crc 0 0 0) 0x55d605db3200 con 0x55d60521b800 2023-10-11T09:45:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7fabe9f4c840 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x1 2023-10-11T09:45:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:15 8.1f 8:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55d6053a9c00 con 0x55d60521b800 2023-10-11T09:45:58.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556d58daab60 con 0x556d5908cc00 2023-10-11T09:45:58.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f0c16003840 4 datascan.init: resolving metadata pool 8 2023-10-11T09:45:58.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7fd2acfc8840 1 Processor -- start 2023-10-11T09:45:58.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f109d93d700 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b63ff0800 0x560b63d60100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x560b63fae900 tx=0x560b64a9c2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f3a0f2ae700 1 -- 172.21.15.78:0/4084151273 <== osd.4 v2:172.21.15.133:6808/35898 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv97 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x5595c5ebafc0 con 0x5595c5417c00 2023-10-11T09:45:58.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f541b53e700 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55caf925d800 0x55caf8ff7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7fabe8f09700 1 -- 172.21.15.78:0/3728357160 <== osd.0 v2:172.21.15.78:6800/35735 5 ==== osd_op_reply(15 1.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv11 ondisk = 0) v8 ==== 154+0+4 (crc 0 0 0) 0x55d605db3200 con 0x55d60521b800 2023-10-11T09:45:58.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7fabe9f4c840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:45:58.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7fabe9f4c840 20 datascan.read_dentry: subdir_head not found in result 2023-10-11T09:45:58.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7fabe9f4c840 10 datascan.inject_with_backtrace: Linking ancestor directory of inode 0x0x10000000000 at 0x0x1:subdir 2023-10-11T09:45:58.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:16 8.1f 8:ff5b34d6:::1.00000000:head [omap-set-vals in=507b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55d6053a8400 con 0x55d60521b800 2023-10-11T09:45:58.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f541b53e700 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55caf92d2400 0x55caf8ff9700 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x55caf92507b0 tx=0x55caf9d2af60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7fd2acfc8840 1 -- start start 2023-10-11T09:45:58.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f3a10af2840 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x5595c552e400 0x5595c5177b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:2 8.1f 8:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c552e800 con 0x5595c552e400 2023-10-11T09:45:58.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f3a0faaf700 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x5595c552e400 0x5595c5177b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f4f03f79840 4 datascan.main: Using metadata pool output 2023-10-11T09:45:58.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.840+0000 7f4f03f79840 4 datascan.main: connecting to RADOS... 2023-10-11T09:45:58.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f4f03f79840 1 Processor -- start 2023-10-11T09:45:58.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f4f03f79840 1 -- start start 2023-10-11T09:45:58.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f4f03f79840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ab46bb3400 0x55ab46c59080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f4f03f79840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab46bb3000 0x55ab46c5a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f3a0faaf700 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x5595c552e400 0x5595c5177b80 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:58.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f4f03f79840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab4799ac00 0x55ab46c5a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f4f03f79840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab46c32500 con 0x55ab4799ac00 2023-10-11T09:45:58.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f4f03f79840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab46c32780 con 0x55ab46bb3000 2023-10-11T09:45:58.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f4f03f79840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ab46c32140 con 0x55ab46bb3400 2023-10-11T09:45:58.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f3a0faaf700 1 -- 172.21.15.78:0/4084151273 <== osd.0 v2:172.21.15.78:6800/35735 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x5595c5ebb200 con 0x5595c552e400 2023-10-11T09:45:58.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f3a10af2840 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x5595c552ec00 0x5595c5177600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:3 9.0 9:00000000::::head [pgnls-filter start_epoch 66 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c552f000 con 0x5595c552ec00 2023-10-11T09:45:58.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f4f02735700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab46bb3000 0x55ab46c5a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f4f02735700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab46bb3000 0x55ab46c5a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:39308/0 (socket says 172.21.15.78:39308) 2023-10-11T09:45:58.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f4f02735700 1 -- 172.21.15.78:0/2111322639 learned_addr learned my addr 172.21.15.78:0/2111322639 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:58.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f4f01f34700 1 --2- 172.21.15.78:0/2111322639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab4799ac00 0x55ab46c5a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f4f02f36700 1 --2- 172.21.15.78:0/2111322639 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ab46bb3400 0x55ab46c59080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:58.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7fd2acfc8840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ec9943e400 0x55ec9865ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:58.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7fd2acfc8840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec9943e800 0x55ec9865a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f541b53e700 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55caf925d800 0x55caf8ff7b80 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f541b53e700 1 -- 172.21.15.78:0/2226872998 <== osd.0 v2:172.21.15.78:6800/35735 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x55caf9247200 con 0x55caf925d800 2023-10-11T09:45:59.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f541c581840 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55caf92d2c00 0x55caf8ff7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f541c581840 1 -- 172.21.15.78:0/2226872998 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:3 9.0 9:00000000::::head [pgnls-filter start_epoch 66 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55caf92d3000 con 0x55caf92d2c00 2023-10-11T09:45:59.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f3a0f2ae700 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x5595c552ec00 0x5595c5177600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f4f02f36700 1 -- 172.21.15.78:0/2111322639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab46bb3000 msgr2=0x55ab46c5a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:59.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f4f02f36700 1 --2- 172.21.15.78:0/2111322639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab46bb3000 0x55ab46c5a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f4f02f36700 1 -- 172.21.15.78:0/2111322639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab4799ac00 msgr2=0x55ab46c5a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:59.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f4f02f36700 1 --2- 172.21.15.78:0/2111322639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab4799ac00 0x55ab46c5a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f4f02f36700 1 -- 172.21.15.78:0/2111322639 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab46bd4000 con 0x55ab46bb3400 2023-10-11T09:45:59.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f4f02f36700 1 --2- 172.21.15.78:0/2111322639 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ab46bb3400 0x55ab46c59080 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x55ab46c09bc0 tx=0x55ab46ebac00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c74bc588e1ff4fdf server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f4efdf2c700 1 -- 172.21.15.78:0/2111322639 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ab46be2f20 con 0x55ab46bb3400 2023-10-11T09:45:59.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f4efdf2c700 1 -- 172.21.15.78:0/2111322639 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ab46ea4900 con 0x55ab46bb3400 2023-10-11T09:45:59.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f4efdf2c700 1 -- 172.21.15.78:0/2111322639 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ab46be3080 con 0x55ab46bb3400 2023-10-11T09:45:59.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f0c0e7b3700 1 -- 172.21.15.78:0/1160862806 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x556d58de6840 con 0x556d5908cc00 2023-10-11T09:45:59.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f541ad3d700 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55caf92d2c00 0x55caf8ff7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f1097931700 1 -- 172.21.15.78:0/1612535802 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560b63ce8dc0 con 0x560b63ff0800 2023-10-11T09:45:59.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f1097931700 1 -- 172.21.15.78:0/1612535802 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560b64a98600 con 0x560b63ff0800 2023-10-11T09:45:59.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f1097931700 1 -- 172.21.15.78:0/1612535802 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560b63ce9080 con 0x560b63ff0800 2023-10-11T09:45:59.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560b63cdaea0 con 0x560b63ff0800 2023-10-11T09:45:59.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b63cdb520 con 0x560b63ff0800 2023-10-11T09:45:59.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f109f181840 4 datascan.init: resolving metadata pool 8 2023-10-11T09:45:59.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f0c0e7b3700 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x556d5908dc00 0x556d58e31700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f0c0e7b3700 1 -- 172.21.15.78:0/1160862806 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x556d58dc2a80 con 0x556d5908cc00 2023-10-11T09:45:59.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7fd2abf85700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ec9943e400 0x55ec9865ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7fd2abf85700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ec9943e400 0x55ec9865ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:53854/0 (socket says 172.21.15.78:53854) 2023-10-11T09:45:59.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7fd2acfc8840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec9943ec00 0x55ec9865c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7fd2abf85700 1 -- 172.21.15.78:0/2359111291 learned_addr learned my addr 172.21.15.78:0/2359111291 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:59.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec986343c0 con 0x55ec9943e800 2023-10-11T09:45:59.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec98634280 con 0x55ec9943ec00 2023-10-11T09:45:59.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ec98634140 con 0x55ec9943e400 2023-10-11T09:45:59.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.844+0000 7fd2ab784700 1 --2- 172.21.15.78:0/2359111291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec9943e800 0x55ec9865a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7fd2ab784700 1 -- 172.21.15.78:0/2359111291 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ec9943e400 msgr2=0x55ec9865ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7fd2ab784700 1 --2- 172.21.15.78:0/2359111291 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ec9943e400 0x55ec9865ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7fd2ab784700 1 -- 172.21.15.78:0/2359111291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec9943ec00 msgr2=0x55ec9865c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7fd2ab784700 1 --2- 172.21.15.78:0/2359111291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec9943ec00 0x55ec9865c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7fd2ab784700 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec985d6000 con 0x55ec9943e800 2023-10-11T09:45:59.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f3a0f2ae700 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x5595c552ec00 0x5595c5177600 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f541ad3d700 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55caf92d2c00 0x55caf8ff7600 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7fd2ab784700 1 --2- 172.21.15.78:0/2359111291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec9943e800 0x55ec9865a580 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x55ec98683740 tx=0x55ec9868b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f0c16003840 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-10-11T09:45:59.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f0c16003840 4 datascan.main: data pool 'cephfs_data' has ID 9 2023-10-11T09:45:59.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f0c16003840 4 datascan.main: opening data pool 'cephfs_data' 2023-10-11T09:45:59.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f0c16003840 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x556d591fc000 0x556d58e30c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:1 8.14 8:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x556d591fc400 con 0x556d591fc000 2023-10-11T09:45:59.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f0c13fbe700 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x556d591fc000 0x556d58e30c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f4f03f79840 1 -- 172.21.15.78:0/2111322639 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ab46bb3400 msgr2=0x55ab46c59080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f4f03f79840 1 --2- 172.21.15.78:0/2111322639 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ab46bb3400 0x55ab46c59080 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x55ab46c09bc0 tx=0x55ab46ebac00 comp rx=0 tx=0).stop 2023-10-11T09:45:59.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f541ad3d700 1 -- 172.21.15.78:0/2226872998 <== osd.1 v2:172.21.15.78:6809/35734 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 66 out=81b] v66'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55caf9247440 con 0x55caf92d2c00 2023-10-11T09:45:59.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f4f03f79840 1 -- 172.21.15.78:0/2111322639 shutdown_connections 2023-10-11T09:45:59.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f0c14fc0700 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x556d5908dc00 0x556d58e31700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f4f03f79840 1 --2- 172.21.15.78:0/2111322639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab4799ac00 0x55ab46c5a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7fabe8f09700 1 -- 172.21.15.78:0/3728357160 <== osd.0 v2:172.21.15.78:6800/35735 6 ==== osd_op_reply(16 1.00000000 [omap-set-vals] v66'12 uv12 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x55d605db3200 con 0x55d60521b800 2023-10-11T09:45:59.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f541c581840 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55caf92d3400 0x55caf93e4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f541c581840 1 -- 172.21.15.78:0/2226872998 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:4 9.24 9:24924924::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55caf92d3800 con 0x55caf92d3400 2023-10-11T09:45:59.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7fabe9f4c840 20 datascan.inject_linkage: Injected dentry 0x0x1/subdir pointing to 0x0x10000000000 2023-10-11T09:45:59.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:17 9.36 9:6fe38dc2::::head [pgnls-filter start_epoch 66 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55d6053a8800 con 0x55d605372800 2023-10-11T09:45:59.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f4f03f79840 1 --2- 172.21.15.78:0/2111322639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab46bb3000 0x55ab46c5a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f4f03f79840 1 --2- 172.21.15.78:0/2111322639 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ab46bb3400 0x55ab46c59080 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f0c13fbe700 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x556d591fc000 0x556d58e30c00 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f541a53c700 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55caf92d3400 0x55caf93e4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f3a0f2ae700 1 -- 172.21.15.78:0/4084151273 <== osd.1 v2:172.21.15.78:6809/35734 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 66 out=81b] v66'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5595c5ebafc0 con 0x5595c552ec00 2023-10-11T09:45:59.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f4f03f79840 1 -- 172.21.15.78:0/2111322639 >> 172.21.15.78:0/2111322639 conn(0x55ab46bb3800 msgr2=0x55ab46c6f000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:59.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7fabe8708700 1 -- 172.21.15.78:0/3728357160 <== osd.1 v2:172.21.15.78:6809/35734 4 ==== osd_op_reply(17 [pgnls-filter start_epoch 66 out=81b] v66'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d605db3440 con 0x55d605372800 2023-10-11T09:45:59.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7fabe9f4c840 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2023-10-11T09:45:59.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7fabe9f4c840 10 datascan.operator(): handling object 100000001f7.0 2023-10-11T09:45:59.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:18 9.36 9:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55d6053a8c00 con 0x55d605372800 2023-10-11T09:45:59.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f0c14fc0700 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x556d5908dc00 0x556d58e31700 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x556d590a12c0 tx=0x556d59b80de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f4f03f79840 1 -- 172.21.15.78:0/2111322639 shutdown_connections 2023-10-11T09:45:59.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7fd2a5778700 1 -- 172.21.15.78:0/2359111291 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ec985e51e0 con 0x55ec9943e800 2023-10-11T09:45:59.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7fd2a5778700 1 -- 172.21.15.78:0/2359111291 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ec9939c480 con 0x55ec9943e800 2023-10-11T09:45:59.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7fd2a5778700 1 -- 172.21.15.78:0/2359111291 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ec985e4f20 con 0x55ec9943e800 2023-10-11T09:45:59.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:4 9.0 9:00000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c552f400 con 0x5595c552ec00 2023-10-11T09:45:59.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f0c13fbe700 1 -- 172.21.15.78:0/1160862806 <== osd.4 v2:172.21.15.133:6808/35898 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv97 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x556d59c12fc0 con 0x556d591fc000 2023-10-11T09:45:59.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f0c16003840 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x556d591fc800 0x556d58e31180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:2 8.1f 8:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x556d591fcc00 con 0x556d591fc800 2023-10-11T09:45:59.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ec985d6ea0 con 0x55ec9943e800 2023-10-11T09:45:59.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f541a53c700 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55caf92d3400 0x55caf93e4000 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7fabe8708700 1 -- 172.21.15.78:0/3728357160 <== osd.1 v2:172.21.15.78:6809/35734 5 ==== osd_op_reply(18 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv22 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x55d605db3440 con 0x55d605372800 2023-10-11T09:45:59.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:19 10.16 10:6fe38dc2:::100000001f7.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55d6053a9000 con 0x55d605373800 2023-10-11T09:45:59.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec985d7520 con 0x55ec9943e800 2023-10-11T09:45:59.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7fd2acfc8840 4 datascan.init: resolving metadata pool 8 2023-10-11T09:45:59.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f4f03f79840 1 -- 172.21.15.78:0/2111322639 wait complete. 2023-10-11T09:45:59.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7fabe7f07700 1 -- 172.21.15.78:0/3728357160 <== osd.6 v2:172.21.15.133:6800/35900 2 ==== osd_op_reply(19 100000001f7.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55d605db2fc0 con 0x55d605373800 2023-10-11T09:45:59.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f1097931700 1 -- 172.21.15.78:0/1612535802 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x560b63d16b00 con 0x560b63ff0800 2023-10-11T09:45:59.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7fabe9f4c840 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-10-11T09:45:59.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7fabe9f4c840 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2023-10-11T09:45:59.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7fabe9f4c840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/2 2023-10-11T09:45:59.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7fabe9f4c840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=2 2023-10-11T09:45:59.030 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:20 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55d6053a8000 con 0x55d605372c00 2023-10-11T09:45:59.030 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f0c14fc0700 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x556d591fc800 0x556d58e31180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.030 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f3a0f2ae700 1 -- 172.21.15.78:0/4084151273 <== osd.1 v2:172.21.15.78:6809/35734 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 66 out=81b] v66'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5595c5ebafc0 con 0x5595c552ec00 2023-10-11T09:45:59.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f3a10af2840 20 datascan.forall_objects: OSD matched oid 10000000206.00000000 2023-10-11T09:45:59.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f3a10af2840 10 datascan.operator(): handling object 10000000206.0 2023-10-11T09:45:59.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:5 9.0 9:012cdbc7:::10000000206.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c552e000 con 0x5595c552ec00 2023-10-11T09:45:59.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.848+0000 7f541a53c700 1 -- 172.21.15.78:0/2226872998 <== osd.2 v2:172.21.15.78:6808/35732 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 66 out=49b] v64'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55caf9246fc0 con 0x55caf92d3400 2023-10-11T09:45:59.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f5416534700 1 -- 172.21.15.78:0/2226872998 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:5 9.14 9:28000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55caf925d400 con 0x55caf92d2c00 2023-10-11T09:45:59.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f541ad3d700 1 -- 172.21.15.78:0/2226872998 <== osd.1 v2:172.21.15.78:6809/35734 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 66 out=49b] v64'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55caf9247440 con 0x55caf92d2c00 2023-10-11T09:45:59.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f3a0f2ae700 1 -- 172.21.15.78:0/4084151273 <== osd.1 v2:172.21.15.78:6809/35734 3 ==== osd_op_reply(5 10000000206.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x5595c5ebafc0 con 0x5595c552ec00 2023-10-11T09:45:59.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f3a10af2840 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x5595c552f000 0x5595c554a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:6 10.0 10:012cdbc7:::10000000206.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c552f800 con 0x5595c552f000 2023-10-11T09:45:59.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f5415d33700 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55caf92d3800 0x55caf93e6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f5415d33700 1 -- 172.21.15.78:0/2226872998 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:6 9.34 9:2c000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55caf93f6000 con 0x55caf92d3800 2023-10-11T09:45:59.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f541b53e700 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55caf92d3800 0x55caf93e6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7fabe8f09700 1 -- 172.21.15.78:0/3728357160 <== osd.7 v2:172.21.15.133:6824/35901 6 ==== osd_op_reply(20 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55d605db3200 con 0x55d605372c00 2023-10-11T09:45:59.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7fabe9f4c840 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-10-11T09:45:59.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7fabe9f4c840 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-10-11T09:45:59.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7fabe9f4c840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/2, will insert to root 2023-10-11T09:45:59.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:21 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55d605372400 con 0x55d605372c00 2023-10-11T09:45:59.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f4f03f79840 1 Processor -- start 2023-10-11T09:45:59.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f1097931700 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x560b63ff1400 0x560b63d61700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f1097931700 1 -- 172.21.15.78:0/1612535802 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x560b63cf2a80 con 0x560b63ff0800 2023-10-11T09:45:59.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f109f181840 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-10-11T09:45:59.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f109f181840 4 datascan.main: data pool 'cephfs_data' has ID 9 2023-10-11T09:45:59.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f109f181840 4 datascan.main: opening data pool 'cephfs_data' 2023-10-11T09:45:59.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f109f181840 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x560b63ff1800 0x560b63d60c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:1 8.14 8:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b63ff1c00 con 0x560b63ff1800 2023-10-11T09:45:59.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f3a0eaad700 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x5595c552f000 0x5595c554a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f4f03f79840 1 -- start start 2023-10-11T09:45:59.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f109e13e700 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x560b63ff1400 0x560b63d61700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7fabe8f09700 1 -- 172.21.15.78:0/3728357160 <== osd.7 v2:172.21.15.133:6824/35901 7 ==== osd_op_reply(21 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55d605db3200 con 0x55d605372c00 2023-10-11T09:45:59.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7fabe9f4c840 10 datascan.find_or_create_dirfrag: read_version = 17 2023-10-11T09:45:59.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7fabe9f4c840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-10-11T09:45:59.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:22 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55d605373000 con 0x55d605372c00 2023-10-11T09:45:59.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f541b53e700 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55caf92d3800 0x55caf93e6680 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f109d13c700 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x560b63ff1800 0x560b63d60c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f0c14fc0700 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x556d591fc800 0x556d58e31180 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7fabe8f09700 1 -- 172.21.15.78:0/3728357160 <== osd.7 v2:172.21.15.133:6824/35901 8 ==== osd_op_reply(22 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55d605db3200 con 0x55d605372c00 2023-10-11T09:45:59.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7fabe9f4c840 20 datascan.read_dentry: oid=10000000000.00000000 dname=2 frag=*, r=0 2023-10-11T09:45:59.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7fabe9f4c840 20 datascan.read_dentry: 2_head not found in result 2023-10-11T09:45:59.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7fabe9f4c840 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000000/2 with size=6291456 bytes 2023-10-11T09:45:59.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:23 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55d605373c00 con 0x55d605372c00 2023-10-11T09:45:59.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f109e13e700 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x560b63ff1400 0x560b63d61700 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x560b63dd0870 tx=0x560b63fb0fc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f3a0eaad700 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x5595c552f000 0x5595c554a580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f0c14fc0700 1 -- 172.21.15.78:0/1160862806 <== osd.0 v2:172.21.15.78:6800/35735 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x556d59c13200 con 0x556d591fc800 2023-10-11T09:45:59.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f541b53e700 1 -- 172.21.15.78:0/2226872998 <== osd.5 v2:172.21.15.133:6816/35899 1 ==== osd_op_reply(6 [pgnls-filter start_epoch 66 out=49b] v64'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55caf9247200 con 0x55caf92d3800 2023-10-11T09:45:59.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f5416534700 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55caf93f6800 0x55caf93e6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f5416534700 1 -- 172.21.15.78:0/2226872998 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:7 9.c 9:30000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55caf93f6c00 con 0x55caf93f6800 2023-10-11T09:45:59.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f0c16003840 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x556d591fd000 0x556d59208000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:3 9.0 9:00000000::::head [pgnls-filter start_epoch 66 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x556d591fc400 con 0x556d591fd000 2023-10-11T09:45:59.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f109d13c700 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x560b63ff1800 0x560b63d60c00 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f0c147bf700 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x556d591fd000 0x556d59208000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f3a0eaad700 1 -- 172.21.15.78:0/4084151273 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== osd_op_reply(6 10000000206.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x5595c5ebb440 con 0x5595c552f000 2023-10-11T09:45:59.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f3a10af2840 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-10-11T09:45:59.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f3a10af2840 10 datascan.inject_with_backtrace: inode: 0x0x10000000206 2023-10-11T09:45:59.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f3a10af2840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/17 2023-10-11T09:45:59.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f3a10af2840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=17 2023-10-11T09:45:59.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f3a10af2840 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x5595c552fc00 0x5595c554ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:7 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c5560000 con 0x5595c552fc00 2023-10-11T09:45:59.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7f541ad3d700 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55caf93f6800 0x55caf93e6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f0c147bf700 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x556d591fd000 0x556d59208000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f3a0faaf700 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x5595c552fc00 0x5595c554ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f541ad3d700 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55caf93f6800 0x55caf93e6c00 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.852+0000 7fd2a5778700 1 -- 172.21.15.78:0/2359111291 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x55ec98612840 con 0x55ec9943e800 2023-10-11T09:45:59.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7fd2a5778700 1 --2- 172.21.15.78:0/2359111291 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55ec989fa000 0x55ec98a00000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7fd2a5778700 1 -- 172.21.15.78:0/2359111291 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x55ec985eea80 con 0x55ec9943e800 2023-10-11T09:45:59.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f109d13c700 1 -- 172.21.15.78:0/1612535802 <== osd.4 v2:172.21.15.133:6808/35898 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv97 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x560b64b24fc0 con 0x560b63ff1800 2023-10-11T09:45:59.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7fd2acfc8840 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-10-11T09:45:59.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f109f181840 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x560b64118000 0x560b63d61180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:2 8.1f 8:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b64118400 con 0x560b64118000 2023-10-11T09:45:59.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7fd2acfc8840 4 datascan.main: data pool 'cephfs_data' has ID 9 2023-10-11T09:45:59.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f541ad3d700 1 -- 172.21.15.78:0/2226872998 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(7 [pgnls-filter start_epoch 66 out=81b] v66'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55caf9247440 con 0x55caf93f6800 2023-10-11T09:45:59.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f541c581840 20 datascan.forall_objects: OSD matched oid 100000001fc.00000000 2023-10-11T09:45:59.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f541c581840 10 datascan.operator(): handling object 100000001fc.0 2023-10-11T09:45:59.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7fd2acfc8840 4 datascan.main: opening data pool 'cephfs_data' 2023-10-11T09:45:59.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f541c581840 1 -- 172.21.15.78:0/2226872998 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:8 9.c 9:30604137:::100000001fc.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55caf92d3c00 con 0x55caf93f6800 2023-10-11T09:45:59.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7fd2acfc8840 1 --2- 172.21.15.78:0/2359111291 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55ec9943f000 0x55ec9865c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7fd2abf85700 1 --2- 172.21.15.78:0/2359111291 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55ec989fa000 0x55ec98a00000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f3a0faaf700 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x5595c552fc00 0x5595c554ab00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f0c147bf700 1 -- 172.21.15.78:0/1160862806 <== osd.1 v2:172.21.15.78:6809/35734 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 66 out=81b] v66'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556d59c13440 con 0x556d591fd000 2023-10-11T09:45:59.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:1 8.14 8:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ec9943f400 con 0x55ec9943f000 2023-10-11T09:45:59.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f0c16003840 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x556d591fd400 0x556d59208580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:4 9.2d 9:b6db6db6::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x556d591fd800 con 0x556d591fd400 2023-10-11T09:45:59.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f4f03f79840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ab46bb3400 0x55ab46c5b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f541ad3d700 1 -- 172.21.15.78:0/2226872998 <== osd.3 v2:172.21.15.78:6824/35738 2 ==== osd_op_reply(8 100000001fc.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv10 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x55caf9247440 con 0x55caf93f6800 2023-10-11T09:45:59.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f541c581840 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55caf93f6c00 0x55caf93e4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f541c581840 1 -- 172.21.15.78:0/2226872998 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:9 10.c 10:30604137:::100000001fc.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55caf93f7000 con 0x55caf93f6c00 2023-10-11T09:45:59.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f109e13e700 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x560b64118000 0x560b63d61180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f4f03f79840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab46bb3000 0x55ab46c5b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7fd2aaf83700 1 --2- 172.21.15.78:0/2359111291 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55ec9943f000 0x55ec9865c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f4f03f79840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab4799bc00 0x55ab46c5ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f4f02735700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab46bb3000 0x55ab46c5b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7fabe8f09700 1 -- 172.21.15.78:0/3728357160 <== osd.7 v2:172.21.15.133:6824/35901 9 ==== osd_op_reply(23 10000000000.00000000 [omap-set-vals] v66'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d605db3200 con 0x55d605372c00 2023-10-11T09:45:59.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f541a53c700 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55caf93f6c00 0x55caf93e4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7fabe9f4c840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/2 pointing to 0x0x100000001f7 2023-10-11T09:45:59.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:24 9.e 9:70000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55d605373400 con 0x55d605372c00 2023-10-11T09:45:59.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f4f02735700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab46bb3000 0x55ab46c5b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46526/0 (socket says 172.21.15.78:46526) 2023-10-11T09:45:59.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f4f02735700 1 -- 172.21.15.78:0/3361204897 learned_addr learned my addr 172.21.15.78:0/3361204897 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:59.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f0c13fbe700 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x556d591fd400 0x556d59208580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab46c32640 con 0x55ab46bb3000 2023-10-11T09:45:59.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7fd2abf85700 1 --2- 172.21.15.78:0/2359111291 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55ec989fa000 0x55ec98a00000 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x55ec99434b70 tx=0x55ec9871ade0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab46c32780 con 0x55ab4799bc00 2023-10-11T09:45:59.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f3a0faaf700 1 -- 172.21.15.78:0/4084151273 <== osd.7 v2:172.21.15.133:6824/35901 1 ==== osd_op_reply(7 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x5595c5ebb200 con 0x5595c552fc00 2023-10-11T09:45:59.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f3a10af2840 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-10-11T09:45:59.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f3a10af2840 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-10-11T09:45:59.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f3a10af2840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/17, will insert to root 2023-10-11T09:45:59.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:8 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c5560400 con 0x5595c552fc00 2023-10-11T09:45:59.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f109e13e700 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x560b64118000 0x560b63d61180 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f4f02f36700 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ab46bb3400 0x55ab46c5b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ab46c32500 con 0x55ab46bb3400 2023-10-11T09:45:59.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f0c13fbe700 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x556d591fd400 0x556d59208580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f4f01f34700 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab4799bc00 0x55ab46c5ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7fabe8f09700 1 -- 172.21.15.78:0/3728357160 <== osd.7 v2:172.21.15.133:6824/35901 10 ==== osd_op_reply(24 [pgnls-filter start_epoch 66 out=49b] v64'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d605db3200 con 0x55d605372c00 2023-10-11T09:45:59.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7fd2aaf83700 1 --2- 172.21.15.78:0/2359111291 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55ec9943f000 0x55ec9865c680 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7fabe3eff700 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:25 9.2e 9:74000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55d6053a8000 con 0x55d605372c00 2023-10-11T09:45:59.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f109e13e700 1 -- 172.21.15.78:0/1612535802 <== osd.0 v2:172.21.15.78:6800/35735 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x560b64b25200 con 0x560b64118000 2023-10-11T09:45:59.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f3a0faaf700 1 -- 172.21.15.78:0/4084151273 <== osd.7 v2:172.21.15.133:6824/35901 2 ==== osd_op_reply(8 10000000000.00000000 [omap-get-header out=274b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5595c5ebb200 con 0x5595c552fc00 2023-10-11T09:45:59.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f541a53c700 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55caf93f6c00 0x55caf93e4b00 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f109f181840 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x560b64118800 0x560b64120000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f3a10af2840 10 datascan.find_or_create_dirfrag: read_version = 18 2023-10-11T09:45:59.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f3a10af2840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-10-11T09:45:59.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:9 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c5560800 con 0x5595c552fc00 2023-10-11T09:45:59.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f4f02735700 1 -- 172.21.15.78:0/3361204897 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ab46bb3400 msgr2=0x55ab46c5b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f4f02735700 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ab46bb3400 0x55ab46c5b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f4f02735700 1 -- 172.21.15.78:0/3361204897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab4799bc00 msgr2=0x55ab46c5ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f4f02735700 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab4799bc00 0x55ab46c5ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f4f02735700 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab46bd4ea0 con 0x55ab46bb3000 2023-10-11T09:45:59.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f0c13fbe700 1 -- 172.21.15.78:0/1160862806 <== osd.2 v2:172.21.15.78:6808/35732 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 66 out=49b] v64'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d59c12fc0 con 0x556d591fd400 2023-10-11T09:45:59.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:3 9.0 9:00000000::::head [pgnls-filter start_epoch 66 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b64118c00 con 0x560b64118800 2023-10-11T09:45:59.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f0c0ffb6700 1 -- 172.21.15.78:0/1160862806 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:5 9.1d 9:b8000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x556d5921e000 con 0x556d591fc800 2023-10-11T09:45:59.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7fd2aaf83700 1 -- 172.21.15.78:0/2359111291 <== osd.4 v2:172.21.15.133:6808/35898 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv97 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x55ec99394fc0 con 0x55ec9943f000 2023-10-11T09:45:59.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7fd2acfc8840 1 --2- 172.21.15.78:0/2359111291 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55ec9943f800 0x55ec9865cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f4f02735700 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab46bb3000 0x55ab46c5b180 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x55ab46eb9770 tx=0x55ab47991ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7fabe8f09700 1 -- 172.21.15.78:0/3728357160 <== osd.7 v2:172.21.15.133:6824/35901 11 ==== osd_op_reply(25 [pgnls-filter start_epoch 66 out=49b] v64'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d605db3200 con 0x55d605372c00 2023-10-11T09:45:59.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7fabe36fe700 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:26 9.1e 9:78000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55d6053a8400 con 0x55d60521b800 2023-10-11T09:45:59.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f541a53c700 1 -- 172.21.15.78:0/2226872998 <== osd.7 v2:172.21.15.133:6824/35901 1 ==== osd_op_reply(9 100000001fc.00000000 [getxattr (30) out=30b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55caf9246fc0 con 0x55caf93f6c00 2023-10-11T09:45:59.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f541c581840 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-10-11T09:45:59.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f541c581840 10 datascan.inject_with_backtrace: inode: 0x0x100000001fc 2023-10-11T09:45:59.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f541c581840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/7 2023-10-11T09:45:59.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f541c581840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=7 2023-10-11T09:45:59.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f541c581840 1 -- 172.21.15.78:0/2226872998 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:10 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55caf93f7400 con 0x55caf93f6c00 2023-10-11T09:45:59.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.856+0000 7f109d93d700 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x560b64118800 0x560b64120000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:2 8.1f 8:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ec989fa400 con 0x55ec9943f800 2023-10-11T09:45:59.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f3a0faaf700 1 -- 172.21.15.78:0/4084151273 <== osd.7 v2:172.21.15.133:6824/35901 3 ==== osd_op_reply(9 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5595c5ebb200 con 0x5595c552fc00 2023-10-11T09:45:59.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f3a10af2840 20 datascan.read_dentry: oid=10000000000.00000000 dname=17 frag=*, r=0 2023-10-11T09:45:59.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f3a10af2840 20 datascan.read_dentry: 17_head not found in result 2023-10-11T09:45:59.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f3a10af2840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000206 at 0x0x10000000000/17 with size=6291456 bytes 2023-10-11T09:45:59.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:10 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c5560c00 con 0x5595c552fc00 2023-10-11T09:45:59.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f0c14fc0700 1 -- 172.21.15.78:0/1160862806 <== osd.0 v2:172.21.15.78:6800/35735 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 66 out=49b] v64'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d59c13200 con 0x556d591fc800 2023-10-11T09:45:59.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7fd2abf85700 1 --2- 172.21.15.78:0/2359111291 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55ec9943f800 0x55ec9865cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f0c0f7b5700 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x556d591fcc00 0x556d59209b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f0c0f7b5700 1 -- 172.21.15.78:0/1160862806 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:6 9.3d 9:bc000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x556d59228000 con 0x556d591fcc00 2023-10-11T09:45:59.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f541a53c700 1 -- 172.21.15.78:0/2226872998 <== osd.7 v2:172.21.15.133:6824/35901 2 ==== osd_op_reply(10 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55caf9246fc0 con 0x55caf93f6c00 2023-10-11T09:45:59.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f4efc729700 1 -- 172.21.15.78:0/3361204897 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ab46be2f20 con 0x55ab46bb3000 2023-10-11T09:45:59.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f4efc729700 1 -- 172.21.15.78:0/3361204897 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ab46bfe900 con 0x55ab46bb3000 2023-10-11T09:45:59.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f541c581840 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-10-11T09:45:59.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f541c581840 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-10-11T09:45:59.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f541c581840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/7, will insert to root 2023-10-11T09:45:59.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f4efc729700 1 -- 172.21.15.78:0/3361204897 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ab46be3080 con 0x55ab46bb3000 2023-10-11T09:45:59.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f541c581840 1 -- 172.21.15.78:0/2226872998 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:11 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55caf93f7800 con 0x55caf93f6c00 2023-10-11T09:45:59.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f109d93d700 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x560b64118800 0x560b64120000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ab46bd5520 con 0x55ab46bb3000 2023-10-11T09:45:59.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7fabe8f09700 1 -- 172.21.15.78:0/3728357160 <== osd.0 v2:172.21.15.78:6800/35735 7 ==== osd_op_reply(26 [pgnls-filter start_epoch 66 out=81b] v66'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d605db3200 con 0x55d60521b800 2023-10-11T09:45:59.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7fabe9f4c840 20 datascan.forall_objects: OSD matched oid 1000000020c.00000000 2023-10-11T09:45:59.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7fabe9f4c840 10 datascan.operator(): handling object 1000000020c.0 2023-10-11T09:45:59.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:27 9.1e 9:789b801b:::1000000020c.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55d60521b400 con 0x55d60521b800 2023-10-11T09:45:59.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7fd2abf85700 1 --2- 172.21.15.78:0/2359111291 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55ec9943f800 0x55ec9865cc00 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f0c14fc0700 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x556d591fcc00 0x556d59209b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ab46bd56c0 con 0x55ab46bb3000 2023-10-11T09:45:59.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f4f03f79840 4 datascan.init: resolving metadata pool 8 2023-10-11T09:45:59.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7fabe8f09700 1 -- 172.21.15.78:0/3728357160 <== osd.0 v2:172.21.15.78:6800/35735 8 ==== osd_op_reply(27 1000000020c.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv9 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x55d605db3200 con 0x55d60521b800 2023-10-11T09:45:59.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:28 10.1e 10:789b801b:::1000000020c.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55d605372400 con 0x55d60521b800 2023-10-11T09:45:59.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7fd2abf85700 1 -- 172.21.15.78:0/2359111291 <== osd.0 v2:172.21.15.78:6800/35735 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x55ec99395200 con 0x55ec9943f800 2023-10-11T09:45:59.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7fd2acfc8840 1 --2- 172.21.15.78:0/2359111291 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55ec989fa800 0x55ec9865d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:3 9.0 9:00000000::::head [pgnls-filter start_epoch 66 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ec989fac00 con 0x55ec989fa800 2023-10-11T09:45:59.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f0c14fc0700 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x556d591fcc00 0x556d59209b80 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f109d93d700 1 -- 172.21.15.78:0/1612535802 <== osd.1 v2:172.21.15.78:6809/35734 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 66 out=81b] v66'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x560b64b25440 con 0x560b64118800 2023-10-11T09:45:59.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f109f181840 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x560b64119000 0x560b64120580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:4 9.12 9:49249249::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b64119400 con 0x560b64119000 2023-10-11T09:45:59.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7fabe8f09700 1 -- 172.21.15.78:0/3728357160 <== osd.0 v2:172.21.15.78:6800/35735 9 ==== osd_op_reply(28 1000000020c.00000000 [getxattr (30) out=30b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55d605db3200 con 0x55d60521b800 2023-10-11T09:45:59.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7fabe9f4c840 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-10-11T09:45:59.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7fabe9f4c840 10 datascan.inject_with_backtrace: inode: 0x0x1000000020c 2023-10-11T09:45:59.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7fabe9f4c840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/23 2023-10-11T09:45:59.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7fabe9f4c840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=23 2023-10-11T09:45:59.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:29 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55d605373000 con 0x55d605372c00 2023-10-11T09:45:59.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7fd2ab784700 1 --2- 172.21.15.78:0/2359111291 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55ec989fa800 0x55ec9865d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f0c14fc0700 1 -- 172.21.15.78:0/1160862806 <== osd.7 v2:172.21.15.133:6824/35901 1 ==== osd_op_reply(6 [pgnls-filter start_epoch 66 out=81b] v66'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556d59c13200 con 0x556d591fcc00 2023-10-11T09:45:59.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f0c16003840 20 datascan.forall_objects: OSD matched oid 10000000205.00000000 2023-10-11T09:45:59.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f0c16003840 10 datascan.operator(): handling object 10000000205.0 2023-10-11T09:45:59.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:7 9.3d 9:bfcd3380:::10000000205.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x556d591fdc00 con 0x556d591fcc00 2023-10-11T09:45:59.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f109d13c700 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x560b64119000 0x560b64120580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7fd2ab784700 1 --2- 172.21.15.78:0/2359111291 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55ec989fa800 0x55ec9865d180 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f3a0faaf700 1 -- 172.21.15.78:0/4084151273 <== osd.7 v2:172.21.15.133:6824/35901 4 ==== osd_op_reply(10 10000000000.00000000 [omap-set-vals] v66'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5595c5ebb200 con 0x5595c552fc00 2023-10-11T09:45:59.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f3a10af2840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/17 pointing to 0x0x10000000206 2023-10-11T09:45:59.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:11 9.20 9:04000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c5561000 con 0x5595c552e400 2023-10-11T09:45:59.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f0c14fc0700 1 -- 172.21.15.78:0/1160862806 <== osd.7 v2:172.21.15.133:6824/35901 2 ==== osd_op_reply(7 10000000205.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x556d59c13200 con 0x556d591fcc00 2023-10-11T09:45:59.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:8 10.1d 10:bfcd3380:::10000000205.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x556d58d89400 con 0x556d591fc000 2023-10-11T09:45:59.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f541a53c700 1 -- 172.21.15.78:0/2226872998 <== osd.7 v2:172.21.15.133:6824/35901 3 ==== osd_op_reply(11 10000000000.00000000 [omap-get-header out=274b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55caf9246fc0 con 0x55caf93f6c00 2023-10-11T09:45:59.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f4efc729700 1 -- 172.21.15.78:0/3361204897 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x55ab46c10b00 con 0x55ab46bb3000 2023-10-11T09:45:59.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f541c581840 10 datascan.find_or_create_dirfrag: read_version = 19 2023-10-11T09:45:59.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f541c581840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-10-11T09:45:59.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f541c581840 1 -- 172.21.15.78:0/2226872998 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:12 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55caf93f7c00 con 0x55caf93f6c00 2023-10-11T09:45:59.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7f109d13c700 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x560b64119000 0x560b64120580 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7fabe8f09700 1 -- 172.21.15.78:0/3728357160 <== osd.7 v2:172.21.15.133:6824/35901 12 ==== osd_op_reply(29 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55d605db3200 con 0x55d605372c00 2023-10-11T09:45:59.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7fabe9f4c840 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-10-11T09:45:59.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7fabe9f4c840 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-10-11T09:45:59.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7fabe9f4c840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/23, will insert to root 2023-10-11T09:45:59.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:30 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55d605373c00 con 0x55d605372c00 2023-10-11T09:45:59.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.860+0000 7fd2ab784700 1 -- 172.21.15.78:0/2359111291 <== osd.1 v2:172.21.15.78:6809/35734 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 66 out=81b] v66'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ec99395440 con 0x55ec989fa800 2023-10-11T09:45:59.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:4 9.9 9:92492492::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ec989fb000 con 0x55ec9943f800 2023-10-11T09:45:59.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f3a0faaf700 1 -- 172.21.15.78:0/4084151273 <== osd.0 v2:172.21.15.78:6800/35735 2 ==== osd_op_reply(11 [pgnls-filter start_epoch 66 out=81b] v66'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5595c5ebb200 con 0x5595c552e400 2023-10-11T09:45:59.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f0c13fbe700 1 -- 172.21.15.78:0/1160862806 <== osd.4 v2:172.21.15.133:6808/35898 2 ==== osd_op_reply(8 10000000205.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x556d59c12fc0 con 0x556d591fc000 2023-10-11T09:45:59.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f0c16003840 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-10-11T09:45:59.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f0c16003840 10 datascan.inject_with_backtrace: inode: 0x0x10000000205 2023-10-11T09:45:59.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f0c16003840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/16 2023-10-11T09:45:59.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f0c16003840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=16 2023-10-11T09:45:59.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f3a10af2840 20 datascan.forall_objects: OSD matched oid 1000000020b.00000000 2023-10-11T09:45:59.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f3a10af2840 10 datascan.operator(): handling object 1000000020b.0 2023-10-11T09:45:59.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:12 9.20 9:05552577:::1000000020b.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c5561400 con 0x5595c552e400 2023-10-11T09:45:59.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:9 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x556d59228800 con 0x556d591fcc00 2023-10-11T09:45:59.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f541a53c700 1 -- 172.21.15.78:0/2226872998 <== osd.7 v2:172.21.15.133:6824/35901 4 ==== osd_op_reply(12 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55caf9246fc0 con 0x55caf93f6c00 2023-10-11T09:45:59.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f541c581840 20 datascan.read_dentry: oid=10000000000.00000000 dname=7 frag=*, r=0 2023-10-11T09:45:59.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f541c581840 20 datascan.read_dentry: 7_head not found in result 2023-10-11T09:45:59.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f541c581840 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fc at 0x0x10000000000/7 with size=6291456 bytes 2023-10-11T09:45:59.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f541c581840 1 -- 172.21.15.78:0/2226872998 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:13 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55caf92d3000 con 0x55caf93f6c00 2023-10-11T09:45:59.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f109d13c700 1 -- 172.21.15.78:0/1612535802 <== osd.5 v2:172.21.15.133:6816/35899 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 66 out=49b] v64'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560b64b24fc0 con 0x560b64119000 2023-10-11T09:45:59.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7fd2abf85700 1 -- 172.21.15.78:0/2359111291 <== osd.0 v2:172.21.15.78:6800/35735 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 66 out=49b] v64'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ec99395200 con 0x55ec9943f800 2023-10-11T09:45:59.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7fabe8f09700 1 -- 172.21.15.78:0/3728357160 <== osd.7 v2:172.21.15.133:6824/35901 13 ==== osd_op_reply(30 10000000000.00000000 [omap-get-header out=274b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55d605db3200 con 0x55d605372c00 2023-10-11T09:45:59.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7fd2a6f7b700 1 --2- 172.21.15.78:0/2359111291 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55ec989fa400 0x55ec98a00580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7fd2a6f7b700 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:5 9.29 9:94000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ec989fb000 con 0x55ec989fa400 2023-10-11T09:45:59.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7fabe9f4c840 10 datascan.find_or_create_dirfrag: read_version = 19 2023-10-11T09:45:59.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7fabe9f4c840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-10-11T09:45:59.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:31 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55d605373400 con 0x55d605372c00 2023-10-11T09:45:59.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f0c14fc0700 1 -- 172.21.15.78:0/1160862806 <== osd.7 v2:172.21.15.133:6824/35901 3 ==== osd_op_reply(9 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x556d59c13200 con 0x556d591fcc00 2023-10-11T09:45:59.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f3a0faaf700 1 -- 172.21.15.78:0/4084151273 <== osd.0 v2:172.21.15.78:6800/35735 3 ==== osd_op_reply(12 1000000020b.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x5595c5ebb200 con 0x5595c552e400 2023-10-11T09:45:59.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f0c16003840 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-10-11T09:45:59.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f0c16003840 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-10-11T09:45:59.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f0c16003840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/16, will insert to root 2023-10-11T09:45:59.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:10 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x556d59228c00 con 0x556d591fcc00 2023-10-11T09:45:59.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:13 10.0 10:05552577:::1000000020b.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c5561800 con 0x5595c552f000 2023-10-11T09:45:59.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f1099134700 1 -- 172.21.15.78:0/1612535802 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:5 9.32 9:4c000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b63ff1c00 con 0x560b63ff1800 2023-10-11T09:45:59.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f4efc729700 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55ab46f20400 0x55ab47010000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.114 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f4efc729700 1 -- 172.21.15.78:0/3361204897 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x55ab46beca80 con 0x55ab46bb3000 2023-10-11T09:45:59.114 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f4f03f79840 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-10-11T09:45:59.114 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7fd2aaf83700 1 --2- 172.21.15.78:0/2359111291 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55ec989fa400 0x55ec98a00580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f4f03f79840 4 datascan.main: data pool 'cephfs_data' has ID 9 2023-10-11T09:45:59.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f4f03f79840 4 datascan.main: opening data pool 'cephfs_data' 2023-10-11T09:45:59.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f4f03f79840 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55ab4799b800 0x55ab46c5a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f4f02f36700 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55ab46f20400 0x55ab47010000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f3a0eaad700 1 -- 172.21.15.78:0/4084151273 <== osd.6 v2:172.21.15.133:6800/35900 2 ==== osd_op_reply(13 1000000020b.00000000 [getxattr (30) out=30b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x5595c5ebb440 con 0x5595c552f000 2023-10-11T09:45:59.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f3a10af2840 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-10-11T09:45:59.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f3a10af2840 10 datascan.inject_with_backtrace: inode: 0x0x1000000020b 2023-10-11T09:45:59.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:1 8.14 8:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab4799b400 con 0x55ab4799b800 2023-10-11T09:45:59.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f3a10af2840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/22 2023-10-11T09:45:59.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f3a10af2840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=22 2023-10-11T09:45:59.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:14 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c5561c00 con 0x5595c552fc00 2023-10-11T09:45:59.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f109d13c700 1 -- 172.21.15.78:0/1612535802 <== osd.4 v2:172.21.15.133:6808/35898 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 66 out=49b] v64'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560b64b24fc0 con 0x560b63ff1800 2023-10-11T09:45:59.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f1098933700 1 -- 172.21.15.78:0/1612535802 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:6 9.a 9:50000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b64119400 con 0x560b64119000 2023-10-11T09:45:59.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f4f01f34700 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55ab4799b800 0x55ab46c5a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f4f02f36700 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55ab46f20400 0x55ab47010000 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x55ab46c09bc0 tx=0x55ab47990240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7fd2aaf83700 1 --2- 172.21.15.78:0/2359111291 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55ec989fa400 0x55ec98a00580 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f109d13c700 1 -- 172.21.15.78:0/1612535802 <== osd.5 v2:172.21.15.133:6816/35899 2 ==== osd_op_reply(6 [pgnls-filter start_epoch 66 out=49b] v64'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560b64b24fc0 con 0x560b64119000 2023-10-11T09:45:59.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f1099134700 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x560b64119800 0x560b64121b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f1099134700 1 -- 172.21.15.78:0/1612535802 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:7 9.2a 9:54000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b64119c00 con 0x560b64119800 2023-10-11T09:45:59.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f4f01f34700 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55ab4799b800 0x55ab46c5a680 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f109e13e700 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x560b64119800 0x560b64121b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7fd2aaf83700 1 -- 172.21.15.78:0/2359111291 <== osd.7 v2:172.21.15.133:6824/35901 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 66 out=49b] v64'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ec99394fc0 con 0x55ec989fa400 2023-10-11T09:45:59.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f4f01f34700 1 -- 172.21.15.78:0/3361204897 <== osd.4 v2:172.21.15.133:6808/35898 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv97 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x55ab479a6000 con 0x55ab4799b800 2023-10-11T09:45:59.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7fd2a677a700 1 --2- 172.21.15.78:0/2359111291 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55ec9943f400 0x55ec98a00b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7fd2a677a700 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:6 9.19 9:98000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ec989fb000 con 0x55ec9943f400 2023-10-11T09:45:59.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f541a53c700 1 -- 172.21.15.78:0/2226872998 <== osd.7 v2:172.21.15.133:6824/35901 5 ==== osd_op_reply(13 10000000000.00000000 [omap-set-vals] v66'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55caf9246fc0 con 0x55caf93f6c00 2023-10-11T09:45:59.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f541c581840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/7 pointing to 0x0x100000001fc 2023-10-11T09:45:59.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f541c581840 1 -- 172.21.15.78:0/2226872998 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:14 9.2c 9:34000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55caf92d3c00 con 0x55caf93f6800 2023-10-11T09:45:59.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f4f03f79840 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55ab4799b000 0x55ab46c59080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:2 8.1f 8:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab4799ac00 con 0x55ab4799b000 2023-10-11T09:45:59.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f109e13e700 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x560b64119800 0x560b64121b80 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7fabe8f09700 1 -- 172.21.15.78:0/3728357160 <== osd.7 v2:172.21.15.133:6824/35901 14 ==== osd_op_reply(31 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55d605db3200 con 0x55d605372c00 2023-10-11T09:45:59.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7fabe9f4c840 20 datascan.read_dentry: oid=10000000000.00000000 dname=23 frag=*, r=0 2023-10-11T09:45:59.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7fabe9f4c840 20 datascan.read_dentry: 23_head not found in result 2023-10-11T09:45:59.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7fabe9f4c840 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000020c at 0x0x10000000000/23 with size=6291456 bytes 2023-10-11T09:45:59.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:32 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55d6053a8000 con 0x55d605372c00 2023-10-11T09:45:59.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7fd2abf85700 1 --2- 172.21.15.78:0/2359111291 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55ec9943f400 0x55ec98a00b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f4f02f36700 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55ab4799b000 0x55ab46c59080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f0c14fc0700 1 -- 172.21.15.78:0/1160862806 <== osd.7 v2:172.21.15.133:6824/35901 4 ==== osd_op_reply(10 10000000000.00000000 [omap-get-header out=274b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x556d59c13200 con 0x556d591fcc00 2023-10-11T09:45:59.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f0c16003840 10 datascan.find_or_create_dirfrag: read_version = 20 2023-10-11T09:45:59.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f0c16003840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-10-11T09:45:59.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:11 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x556d59229000 con 0x556d591fcc00 2023-10-11T09:45:59.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f3a0faaf700 1 -- 172.21.15.78:0/4084151273 <== osd.7 v2:172.21.15.133:6824/35901 5 ==== osd_op_reply(14 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x5595c5ebb200 con 0x5595c552fc00 2023-10-11T09:45:59.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f3a10af2840 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-10-11T09:45:59.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f3a10af2840 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-10-11T09:45:59.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f3a10af2840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/22, will insert to root 2023-10-11T09:45:59.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:15 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c5560400 con 0x5595c552fc00 2023-10-11T09:45:59.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.864+0000 7f541ad3d700 1 -- 172.21.15.78:0/2226872998 <== osd.3 v2:172.21.15.78:6824/35738 3 ==== osd_op_reply(14 [pgnls-filter start_epoch 66 out=81b] v66'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55caf9247440 con 0x55caf93f6800 2023-10-11T09:45:59.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f541c581840 20 datascan.forall_objects: OSD matched oid 10000000209.00000000 2023-10-11T09:45:59.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f541c581840 10 datascan.operator(): handling object 10000000209.0 2023-10-11T09:45:59.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f541c581840 1 -- 172.21.15.78:0/2226872998 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:15 9.2c 9:37389ade:::10000000209.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55caf925d400 con 0x55caf93f6800 2023-10-11T09:45:59.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f109e13e700 1 -- 172.21.15.78:0/1612535802 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(7 [pgnls-filter start_epoch 66 out=101b] v66'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x560b64b25200 con 0x560b64119800 2023-10-11T09:45:59.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f109f181840 20 datascan.forall_objects: OSD matched oid 1000000020a.00000000 2023-10-11T09:45:59.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f109f181840 10 datascan.operator(): handling object 1000000020a.0 2023-10-11T09:45:59.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:8 9.2a 9:55a6cd83:::1000000020a.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b64118400 con 0x560b64119800 2023-10-11T09:45:59.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f4f02f36700 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55ab4799b000 0x55ab46c59080 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7fd2abf85700 1 --2- 172.21.15.78:0/2359111291 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55ec9943f400 0x55ec98a00b00 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f541ad3d700 1 -- 172.21.15.78:0/2226872998 <== osd.3 v2:172.21.15.78:6824/35738 4 ==== osd_op_reply(15 10000000209.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x55caf9247440 con 0x55caf93f6800 2023-10-11T09:45:59.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f541c581840 1 -- 172.21.15.78:0/2226872998 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:16 10.c 10:37389ade:::10000000209.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55caf93f7000 con 0x55caf93f6c00 2023-10-11T09:45:59.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f109e13e700 1 -- 172.21.15.78:0/1612535802 <== osd.3 v2:172.21.15.78:6824/35738 2 ==== osd_op_reply(8 1000000020a.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x560b64b25200 con 0x560b64119800 2023-10-11T09:45:59.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f109f181840 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x560b64119c00 0x560b64121080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:9 10.a 10:55a6cd83:::1000000020a.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b6413c000 con 0x560b64119c00 2023-10-11T09:45:59.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f4f02f36700 1 -- 172.21.15.78:0/3361204897 <== osd.0 v2:172.21.15.78:6800/35735 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x55ab479a6d80 con 0x55ab4799b000 2023-10-11T09:45:59.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f4f03f79840 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55ab46f20800 0x55ab46c5a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:3 9.0 9:00000000::::head [pgnls-filter start_epoch 66 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab46f20c00 con 0x55ab46f20800 2023-10-11T09:45:59.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7fd2abf85700 1 -- 172.21.15.78:0/2359111291 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(6 [pgnls-filter start_epoch 66 out=101b] v66'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55ec99395200 con 0x55ec9943f400 2023-10-11T09:45:59.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7fd2acfc8840 20 datascan.forall_objects: OSD matched oid 10000000200.00000000 2023-10-11T09:45:59.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7fd2acfc8840 10 datascan.operator(): handling object 10000000200.0 2023-10-11T09:45:59.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:7 9.19 9:992a6246:::10000000200.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ec989fb400 con 0x55ec9943f400 2023-10-11T09:45:59.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f541a53c700 1 -- 172.21.15.78:0/2226872998 <== osd.7 v2:172.21.15.133:6824/35901 6 ==== osd_op_reply(16 10000000209.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55caf9246fc0 con 0x55caf93f6c00 2023-10-11T09:45:59.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7fabe8f09700 1 -- 172.21.15.78:0/3728357160 <== osd.7 v2:172.21.15.133:6824/35901 15 ==== osd_op_reply(32 10000000000.00000000 [omap-set-vals] v66'21 uv21 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d605db3200 con 0x55d605372c00 2023-10-11T09:45:59.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7fabe9f4c840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/23 pointing to 0x0x1000000020c 2023-10-11T09:45:59.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7fabe9f4c840 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55d6053a8400 0x55d6053a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f541c581840 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-10-11T09:45:59.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f541c581840 10 datascan.inject_with_backtrace: inode: 0x0x10000000209 2023-10-11T09:45:59.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f541c581840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/20 2023-10-11T09:45:59.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f541c581840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=20 2023-10-11T09:45:59.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f541c581840 1 -- 172.21.15.78:0/2226872998 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:17 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55caf93f7400 con 0x55caf93f6c00 2023-10-11T09:45:59.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f109d93d700 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x560b64119c00 0x560b64121080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f4f02735700 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55ab46f20800 0x55ab46c5a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f3a0faaf700 1 -- 172.21.15.78:0/4084151273 <== osd.7 v2:172.21.15.133:6824/35901 6 ==== osd_op_reply(15 10000000000.00000000 [omap-get-header out=274b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5595c5ebb200 con 0x5595c552fc00 2023-10-11T09:45:59.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f3a10af2840 10 datascan.find_or_create_dirfrag: read_version = 21 2023-10-11T09:45:59.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f3a10af2840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-10-11T09:45:59.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:16 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c5560800 con 0x5595c552fc00 2023-10-11T09:45:59.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:33 9.3e 9:7c000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55d6053a8800 con 0x55d6053a8400 2023-10-11T09:45:59.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7fd2abf85700 1 -- 172.21.15.78:0/2359111291 <== osd.3 v2:172.21.15.78:6824/35738 2 ==== osd_op_reply(7 10000000200.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv18 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x55ec99395200 con 0x55ec9943f400 2023-10-11T09:45:59.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f0c14fc0700 1 -- 172.21.15.78:0/1160862806 <== osd.7 v2:172.21.15.133:6824/35901 5 ==== osd_op_reply(11 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x556d59c13200 con 0x556d591fcc00 2023-10-11T09:45:59.145 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:8 10.19 10:992a6246:::10000000200.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ec989fb800 con 0x55ec989fa800 2023-10-11T09:45:59.145 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f0c16003840 20 datascan.read_dentry: oid=10000000000.00000000 dname=16 frag=*, r=0 2023-10-11T09:45:59.145 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f0c16003840 20 datascan.read_dentry: 16_head not found in result 2023-10-11T09:45:59.145 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f0c16003840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000205 at 0x0x10000000000/16 with size=6291456 bytes 2023-10-11T09:45:59.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f541a53c700 1 -- 172.21.15.78:0/2226872998 <== osd.7 v2:172.21.15.133:6824/35901 7 ==== osd_op_reply(17 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55caf9246fc0 con 0x55caf93f6c00 2023-10-11T09:45:59.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7fabe8708700 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55d6053a8400 0x55d6053a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:12 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x556d59229400 con 0x556d591fcc00 2023-10-11T09:45:59.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f3a0faaf700 1 -- 172.21.15.78:0/4084151273 <== osd.7 v2:172.21.15.133:6824/35901 7 ==== osd_op_reply(16 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5595c5ebb200 con 0x5595c552fc00 2023-10-11T09:45:59.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f3a10af2840 20 datascan.read_dentry: oid=10000000000.00000000 dname=22 frag=*, r=0 2023-10-11T09:45:59.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f3a10af2840 20 datascan.read_dentry: 22_head not found in result 2023-10-11T09:45:59.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f3a10af2840 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000020b at 0x0x10000000000/22 with size=6291456 bytes 2023-10-11T09:45:59.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:17 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c5560c00 con 0x5595c552fc00 2023-10-11T09:45:59.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f541c581840 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-10-11T09:45:59.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f541c581840 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-10-11T09:45:59.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f541c581840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/20, will insert to root 2023-10-11T09:45:59.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f541c581840 1 -- 172.21.15.78:0/2226872998 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:18 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55caf93f7800 con 0x55caf93f6c00 2023-10-11T09:45:59.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7fd2ab784700 1 -- 172.21.15.78:0/2359111291 <== osd.1 v2:172.21.15.78:6809/35734 2 ==== osd_op_reply(8 10000000200.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55ec99395440 con 0x55ec989fa800 2023-10-11T09:45:59.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7fd2acfc8840 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-10-11T09:45:59.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7fd2acfc8840 10 datascan.inject_with_backtrace: inode: 0x0x10000000200 2023-10-11T09:45:59.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7fd2acfc8840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/11 2023-10-11T09:45:59.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7fd2acfc8840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=11 2023-10-11T09:45:59.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:9 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ec989fbc00 con 0x55ec989fa400 2023-10-11T09:45:59.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f109d93d700 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x560b64119c00 0x560b64121080 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f4f02735700 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55ab46f20800 0x55ab46c5a100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7fabe8708700 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55d6053a8400 0x55d6053a5b80 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f109d93d700 1 -- 172.21.15.78:0/1612535802 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== osd_op_reply(9 1000000020a.00000000 [getxattr (30) out=30b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x560b64b25440 con 0x560b64119c00 2023-10-11T09:45:59.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f109f181840 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-10-11T09:45:59.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f109f181840 10 datascan.inject_with_backtrace: inode: 0x0x1000000020a 2023-10-11T09:45:59.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f109f181840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/21 2023-10-11T09:45:59.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f109f181840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=21 2023-10-11T09:45:59.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f109f181840 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x560b6413c400 0x560b64121600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7fabe8708700 1 -- 172.21.15.78:0/3728357160 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(33 [pgnls-filter start_epoch 66 out=49b] v64'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d605db3440 con 0x55d6053a8400 2023-10-11T09:45:59.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:10 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b6413c800 con 0x560b6413c400 2023-10-11T09:45:59.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7fabe36fe700 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:34 9.1 9:80000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55d605c9c000 con 0x55d6053a8400 2023-10-11T09:45:59.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f4f02735700 1 -- 172.21.15.78:0/3361204897 <== osd.1 v2:172.21.15.78:6809/35734 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 66 out=81b] v66'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ab479a6fc0 con 0x55ab46f20800 2023-10-11T09:45:59.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f109d13c700 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x560b6413c400 0x560b64121600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f4f03f79840 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55ab46f21000 0x55ab47010580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7fabe8708700 1 -- 172.21.15.78:0/3728357160 <== osd.3 v2:172.21.15.78:6824/35738 2 ==== osd_op_reply(34 [pgnls-filter start_epoch 66 out=49b] v64'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d605db3440 con 0x55d6053a8400 2023-10-11T09:45:59.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7fabe3eff700 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55d605c9c800 0x55d6053a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7fabe3eff700 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:35 9.21 9:84000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55d605c9cc00 con 0x55d605c9c800 2023-10-11T09:45:59.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:4 9.1b 9:db6db6db::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab46f21400 con 0x55ab46f21000 2023-10-11T09:45:59.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f4f01f34700 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55ab46f21000 0x55ab47010580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7fabe7f07700 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55d605c9c800 0x55d6053a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f0c14fc0700 1 -- 172.21.15.78:0/1160862806 <== osd.7 v2:172.21.15.133:6824/35901 6 ==== osd_op_reply(12 10000000000.00000000 [omap-set-vals] v66'22 uv22 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556d59c13200 con 0x556d591fcc00 2023-10-11T09:45:59.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.868+0000 7f0c16003840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/16 pointing to 0x0x10000000205 2023-10-11T09:45:59.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:13 9.3 9:c0000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x556d59229800 con 0x556d591fcc00 2023-10-11T09:45:59.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7f541a53c700 1 -- 172.21.15.78:0/2226872998 <== osd.7 v2:172.21.15.133:6824/35901 8 ==== osd_op_reply(18 10000000000.00000000 [omap-get-header out=274b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55caf9246fc0 con 0x55caf93f6c00 2023-10-11T09:45:59.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7f541c581840 10 datascan.find_or_create_dirfrag: read_version = 22 2023-10-11T09:45:59.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7f541c581840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-10-11T09:45:59.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7f541c581840 1 -- 172.21.15.78:0/2226872998 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:19 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55caf93f7c00 con 0x55caf93f6c00 2023-10-11T09:45:59.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7f109d13c700 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x560b6413c400 0x560b64121600 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7f4f01f34700 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55ab46f21000 0x55ab47010580 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7fabe7f07700 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55d605c9c800 0x55d6053a4000 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7f0c14fc0700 1 -- 172.21.15.78:0/1160862806 <== osd.7 v2:172.21.15.133:6824/35901 7 ==== osd_op_reply(13 [pgnls-filter start_epoch 66 out=49b] v64'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d59c13200 con 0x556d591fcc00 2023-10-11T09:45:59.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7f0c0f7b5700 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x556d59228400 0x556d5920a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7f0c0f7b5700 1 -- 172.21.15.78:0/1160862806 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:14 9.23 9:c4000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x556d59228c00 con 0x556d59228400 2023-10-11T09:45:59.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7f4f01f34700 1 -- 172.21.15.78:0/3361204897 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 66 out=49b] v64'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ab479a6000 con 0x55ab46f21000 2023-10-11T09:45:59.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7f4efdf2c700 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55ab47036000 0x55ab47012c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7fabe7f07700 1 -- 172.21.15.78:0/3728357160 <== osd.2 v2:172.21.15.78:6808/35732 1 ==== osd_op_reply(35 [pgnls-filter start_epoch 66 out=49b] v64'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d605db2fc0 con 0x55d605c9c800 2023-10-11T09:45:59.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7fabe36fe700 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:36 9.11 9:88000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55d605c9c400 con 0x55d605372c00 2023-10-11T09:45:59.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7f0c147bf700 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x556d59228400 0x556d5920a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7f4efdf2c700 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:5 9.3b 9:dc000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab47036400 con 0x55ab47036000 2023-10-11T09:45:59.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7f4f02f36700 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55ab47036000 0x55ab47012c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7fabe8f09700 1 -- 172.21.15.78:0/3728357160 <== osd.7 v2:172.21.15.133:6824/35901 16 ==== osd_op_reply(36 [pgnls-filter start_epoch 66 out=49b] v64'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d605db3200 con 0x55d605372c00 2023-10-11T09:45:59.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7fabe3eff700 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:37 9.31 9:8c000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55d605372400 con 0x55d60521b000 2023-10-11T09:45:59.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7f0c147bf700 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x556d59228400 0x556d5920a100 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7f4f02f36700 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55ab47036000 0x55ab47012c00 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7f0c147bf700 1 -- 172.21.15.78:0/1160862806 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(14 [pgnls-filter start_epoch 66 out=49b] v64'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d59c13440 con 0x556d59228400 2023-10-11T09:45:59.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7f0c0ffb6700 1 -- 172.21.15.78:0/1160862806 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:15 9.13 9:c8000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x556d59229400 con 0x556d591fcc00 2023-10-11T09:45:59.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7fabe7f07700 1 -- 172.21.15.78:0/3728357160 <== osd.4 v2:172.21.15.133:6808/35898 2 ==== osd_op_reply(37 [pgnls-filter start_epoch 66 out=49b] v64'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d605db2fc0 con 0x55d60521b000 2023-10-11T09:45:59.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7fabe36fe700 1 -- 172.21.15.78:0/3728357160 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:38 9.9 9:90000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55d6053a8000 con 0x55d60521b800 2023-10-11T09:45:59.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7f4f02f36700 1 -- 172.21.15.78:0/3361204897 <== osd.2 v2:172.21.15.78:6808/35732 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 66 out=81b] v66'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ab479a6d80 con 0x55ab47036000 2023-10-11T09:45:59.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7f4f03f79840 20 datascan.forall_objects: OSD matched oid 100000001fe.00000000 2023-10-11T09:45:59.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7f4f03f79840 10 datascan.operator(): handling object 100000001fe.0 2023-10-11T09:45:59.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:6 9.3b 9:deccbd37:::100000001fe.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab46f21800 con 0x55ab47036000 2023-10-11T09:45:59.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7f0c14fc0700 1 -- 172.21.15.78:0/1160862806 <== osd.7 v2:172.21.15.133:6824/35901 8 ==== osd_op_reply(15 [pgnls-filter start_epoch 66 out=81b] v66'19 uv19 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556d59c13200 con 0x556d591fcc00 2023-10-11T09:45:59.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7fabe8f09700 1 -- 172.21.15.78:0/3728357160 <== osd.0 v2:172.21.15.78:6800/35735 10 ==== osd_op_reply(38 [pgnls-filter start_epoch 66 out=49b] v64'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d605db3200 con 0x55d60521b800 2023-10-11T09:45:59.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7f0c16003840 20 datascan.forall_objects: OSD matched oid 100000001fb.00000000 2023-10-11T09:45:59.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7f0c16003840 10 datascan.operator(): handling object 100000001fb.0 2023-10-11T09:45:59.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:16 9.13 9:ca8fdbba:::100000001fb.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x556d59229c00 con 0x556d591fcc00 2023-10-11T09:45:59.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55d60521b800 msgr2=0x55d604fcfb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7fabe9f4c840 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55d60521b800 0x55d604fcfb80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55d605372800 msgr2=0x55d604fcf600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7fabe9f4c840 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55d605372800 0x55d604fcf600 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55d605c9c800 msgr2=0x55d6053a4000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7f4f02f36700 1 -- 172.21.15.78:0/3361204897 <== osd.2 v2:172.21.15.78:6808/35732 2 ==== osd_op_reply(6 100000001fe.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv8 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x55ab479a6d80 con 0x55ab47036000 2023-10-11T09:45:59.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:7 10.1b 10:deccbd37:::100000001fe.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab46f21c00 con 0x55ab46f21000 2023-10-11T09:45:59.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7fabe9f4c840 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55d605c9c800 0x55d6053a4000 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55d6053a8400 msgr2=0x55d6053a5b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7fabe9f4c840 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55d6053a8400 0x55d6053a5b80 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55d60521b000 msgr2=0x55d604fd0100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7fabe9f4c840 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55d60521b000 0x55d604fd0100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55d605373800 msgr2=0x55d6053a4580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7fabe9f4c840 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55d605373800 0x55d6053a4580 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55d605372c00 msgr2=0x55d6053a4b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7fabe9f4c840 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55d605372c00 0x55d6053a4b00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7fabe8708700 1 -- 172.21.15.78:0/3728357160 reap_dead start 2023-10-11T09:45:59.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7fabe8708700 1 -- 172.21.15.78:0/3728357160 reap_dead start 2023-10-11T09:45:59.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7fabe8708700 1 -- 172.21.15.78:0/3728357160 reap_dead start 2023-10-11T09:45:59.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55d605372000 msgr2=0x55d604fd1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7fabe9f4c840 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55d605372000 0x55d604fd1700 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x55d604ff7350 tx=0x55d605233ec0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d60521a400 msgr2=0x55d604fd1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7fabe9f4c840 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d60521a400 0x55d604fd1180 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x55d605d15470 tx=0x55d60508e000 comp rx=0 tx=0).stop 2023-10-11T09:45:59.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7f0c14fc0700 1 -- 172.21.15.78:0/1160862806 <== osd.7 v2:172.21.15.133:6824/35901 9 ==== osd_op_reply(16 100000001fb.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv19 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x556d59c13200 con 0x556d591fcc00 2023-10-11T09:45:59.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:17 10.13 10:ca8fdbba:::100000001fb.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x556d59228000 con 0x556d591fd000 2023-10-11T09:45:59.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7f4f01f34700 1 -- 172.21.15.78:0/3361204897 <== osd.3 v2:172.21.15.78:6824/35738 2 ==== osd_op_reply(7 100000001fe.00000000 [getxattr (30) out=30b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55ab479a6000 con 0x55ab46f21000 2023-10-11T09:45:59.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7f4f03f79840 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-10-11T09:45:59.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7f4f03f79840 10 datascan.inject_with_backtrace: inode: 0x0x100000001fe 2023-10-11T09:45:59.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7f4f03f79840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/9 2023-10-11T09:45:59.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7f4f03f79840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=9 2023-10-11T09:45:59.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7f4f03f79840 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55ab47036c00 0x55ab47011080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:8 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab47037000 con 0x55ab47036c00 2023-10-11T09:45:59.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 shutdown_connections 2023-10-11T09:45:59.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7fabe9f4c840 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55d605372000 0x55d604fd1700 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7fabe9f4c840 1 --2- 172.21.15.78:0/3728357160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d60521a400 0x55d604fd1180 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.872+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 >> 172.21.15.78:0/3728357160 conn(0x55d604f28c00 msgr2=0x55d605015800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:59.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f3a0faaf700 1 -- 172.21.15.78:0/4084151273 <== osd.7 v2:172.21.15.133:6824/35901 8 ==== osd_op_reply(17 10000000000.00000000 [omap-set-vals] v66'23 uv23 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5595c5ebb200 con 0x5595c552fc00 2023-10-11T09:45:59.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f3a10af2840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/22 pointing to 0x0x1000000020b 2023-10-11T09:45:59.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:18 9.10 9:08000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c5560000 con 0x5595c5417c00 2023-10-11T09:45:59.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 shutdown_connections 2023-10-11T09:45:59.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7fd2aaf83700 1 -- 172.21.15.78:0/2359111291 <== osd.7 v2:172.21.15.133:6824/35901 2 ==== osd_op_reply(9 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55ec99394fc0 con 0x55ec989fa400 2023-10-11T09:45:59.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7fd2acfc8840 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-10-11T09:45:59.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7fd2acfc8840 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-10-11T09:45:59.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7fd2acfc8840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/11, will insert to root 2023-10-11T09:45:59.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:10 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ec98a3a000 con 0x55ec989fa400 2023-10-11T09:45:59.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f0c147bf700 1 -- 172.21.15.78:0/1160862806 <== osd.1 v2:172.21.15.78:6809/35734 2 ==== osd_op_reply(17 100000001fb.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x556d59c13440 con 0x556d591fd000 2023-10-11T09:45:59.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f0c16003840 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-10-11T09:45:59.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f0c16003840 10 datascan.inject_with_backtrace: inode: 0x0x100000001fb 2023-10-11T09:45:59.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f0c16003840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/6 2023-10-11T09:45:59.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f0c16003840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=6 2023-10-11T09:45:59.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:18 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x556d59228800 con 0x556d591fcc00 2023-10-11T09:45:59.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f4f02735700 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55ab47036c00 0x55ab47011080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3728357160 wait complete. 2023-10-11T09:45:59.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f109d13c700 1 -- 172.21.15.78:0/1612535802 <== osd.7 v2:172.21.15.133:6824/35901 1 ==== osd_op_reply(10 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x560b64b24fc0 con 0x560b6413c400 2023-10-11T09:45:59.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f109f181840 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-10-11T09:45:59.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f109f181840 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-10-11T09:45:59.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f109f181840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/21, will insert to root 2023-10-11T09:45:59.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:11 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b6413cc00 con 0x560b6413c400 2023-10-11T09:45:59.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f541a53c700 1 -- 172.21.15.78:0/2226872998 <== osd.7 v2:172.21.15.133:6824/35901 9 ==== osd_op_reply(19 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55caf9246fc0 con 0x55caf93f6c00 2023-10-11T09:45:59.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f541c581840 20 datascan.read_dentry: oid=10000000000.00000000 dname=20 frag=*, r=0 2023-10-11T09:45:59.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f541c581840 20 datascan.read_dentry: 20_head not found in result 2023-10-11T09:45:59.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f541c581840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000209 at 0x0x10000000000/20 with size=6291456 bytes 2023-10-11T09:45:59.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f541c581840 1 -- 172.21.15.78:0/2226872998 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:20 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55caf93f7000 con 0x55caf93f6c00 2023-10-11T09:45:59.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3669947308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d604f29400 msgr2=0x55d604fceb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7fabe9f4c840 1 --2- 172.21.15.78:0/3669947308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d604f29400 0x55d604fceb00 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x55d604ff73b0 tx=0x55d60508eea0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f0c14fc0700 1 -- 172.21.15.78:0/1160862806 <== osd.7 v2:172.21.15.133:6824/35901 10 ==== osd_op_reply(18 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x556d59c13200 con 0x556d591fcc00 2023-10-11T09:45:59.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f0c16003840 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-10-11T09:45:59.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f0c16003840 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-10-11T09:45:59.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f0c16003840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/6, will insert to root 2023-10-11T09:45:59.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:19 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x556d59228c00 con 0x556d591fcc00 2023-10-11T09:45:59.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f3a0f2ae700 1 -- 172.21.15.78:0/4084151273 <== osd.4 v2:172.21.15.133:6808/35898 2 ==== osd_op_reply(18 [pgnls-filter start_epoch 66 out=81b] v66'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5595c5ebafc0 con 0x5595c5417c00 2023-10-11T09:45:59.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3669947308 shutdown_connections 2023-10-11T09:45:59.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7fabe9f4c840 1 --2- 172.21.15.78:0/3669947308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d604f29000 0x55d604fcf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7fabe9f4c840 1 --2- 172.21.15.78:0/3669947308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d604f29400 0x55d604fceb00 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7fabe9f4c840 1 --2- 172.21.15.78:0/3669947308 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d604f29800 0x55d604fce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3669947308 >> 172.21.15.78:0/3669947308 conn(0x55d604f29c00 msgr2=0x55d605015000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:59.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f4f02735700 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55ab47036c00 0x55ab47011080 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7fd2aaf83700 1 -- 172.21.15.78:0/2359111291 <== osd.7 v2:172.21.15.133:6824/35901 3 ==== osd_op_reply(10 10000000000.00000000 [omap-get-header out=274b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55ec99394fc0 con 0x55ec989fa400 2023-10-11T09:45:59.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7fd2acfc8840 10 datascan.find_or_create_dirfrag: read_version = 23 2023-10-11T09:45:59.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7fd2acfc8840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-10-11T09:45:59.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:11 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ec98a3a400 con 0x55ec989fa400 2023-10-11T09:45:59.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f3a10af2840 20 datascan.forall_objects: OSD matched oid 100000001fd.00000000 2023-10-11T09:45:59.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f3a10af2840 10 datascan.operator(): handling object 100000001fd.0 2023-10-11T09:45:59.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:19 9.10 9:08b99fea:::100000001fd.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c552e800 con 0x5595c5417c00 2023-10-11T09:45:59.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f109d13c700 1 -- 172.21.15.78:0/1612535802 <== osd.7 v2:172.21.15.133:6824/35901 2 ==== osd_op_reply(11 10000000000.00000000 [omap-get-header out=274b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x560b64b24fc0 con 0x560b6413c400 2023-10-11T09:45:59.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f109f181840 10 datascan.find_or_create_dirfrag: read_version = 23 2023-10-11T09:45:59.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f109f181840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-10-11T09:45:59.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3669947308 shutdown_connections 2023-10-11T09:45:59.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:12 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b6413d000 con 0x560b6413c400 2023-10-11T09:45:59.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7fabe9f4c840 1 -- 172.21.15.78:0/3669947308 wait complete. 2023-10-11T09:45:59.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f3a0f2ae700 1 -- 172.21.15.78:0/4084151273 <== osd.4 v2:172.21.15.133:6808/35898 3 ==== osd_op_reply(19 100000001fd.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv10 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x5595c5ebafc0 con 0x5595c5417c00 2023-10-11T09:45:59.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:20 10.10 10:08b99fea:::100000001fd.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c5560000 con 0x5595c5417c00 2023-10-11T09:45:59.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f3a0f2ae700 1 -- 172.21.15.78:0/4084151273 <== osd.4 v2:172.21.15.133:6808/35898 4 ==== osd_op_reply(20 100000001fd.00000000 [getxattr (30) out=30b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x5595c5ebafc0 con 0x5595c5417c00 2023-10-11T09:45:59.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f3a10af2840 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-10-11T09:45:59.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f3a10af2840 10 datascan.inject_with_backtrace: inode: 0x0x100000001fd 2023-10-11T09:45:59.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f3a10af2840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/8 2023-10-11T09:45:59.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f3a10af2840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=8 2023-10-11T09:45:59.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:21 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c5561000 con 0x5595c552fc00 2023-10-11T09:45:59.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f541a53c700 1 -- 172.21.15.78:0/2226872998 <== osd.7 v2:172.21.15.133:6824/35901 10 ==== osd_op_reply(20 10000000000.00000000 [omap-set-vals] v66'24 uv24 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55caf9246fc0 con 0x55caf93f6c00 2023-10-11T09:45:59.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f541c581840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/20 pointing to 0x0x10000000209 2023-10-11T09:45:59.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.876+0000 7f541c581840 1 -- 172.21.15.78:0/2226872998 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:21 9.1c 9:38000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55caf93f7400 con 0x55caf925d800 2023-10-11T09:45:59.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f0c14fc0700 1 -- 172.21.15.78:0/1160862806 <== osd.7 v2:172.21.15.133:6824/35901 11 ==== osd_op_reply(19 10000000000.00000000 [omap-get-header out=274b] v0'0 uv24 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x556d59c13200 con 0x556d591fcc00 2023-10-11T09:45:59.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f0c16003840 10 datascan.find_or_create_dirfrag: read_version = 24 2023-10-11T09:45:59.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f0c16003840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-10-11T09:45:59.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:20 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x556d59228000 con 0x556d591fcc00 2023-10-11T09:45:59.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f4f02735700 1 -- 172.21.15.78:0/3361204897 <== osd.7 v2:172.21.15.133:6824/35901 1 ==== osd_op_reply(8 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55ab479a6fc0 con 0x55ab47036c00 2023-10-11T09:45:59.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f4f03f79840 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-10-11T09:45:59.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f4f03f79840 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-10-11T09:45:59.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f4f03f79840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/9, will insert to root 2023-10-11T09:45:59.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:9 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab47037400 con 0x55ab47036c00 2023-10-11T09:45:59.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f109d13c700 1 -- 172.21.15.78:0/1612535802 <== osd.7 v2:172.21.15.133:6824/35901 3 ==== osd_op_reply(12 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv24 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x560b64b24fc0 con 0x560b6413c400 2023-10-11T09:45:59.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f109f181840 20 datascan.read_dentry: oid=10000000000.00000000 dname=21 frag=*, r=0 2023-10-11T09:45:59.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f109f181840 20 datascan.read_dentry: 21_head not found in result 2023-10-11T09:45:59.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f109f181840 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000020a at 0x0x10000000000/21 with size=6291456 bytes 2023-10-11T09:45:59.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:13 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b6413d400 con 0x560b6413c400 2023-10-11T09:45:59.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f541b53e700 1 -- 172.21.15.78:0/2226872998 <== osd.0 v2:172.21.15.78:6800/35735 2 ==== osd_op_reply(21 [pgnls-filter start_epoch 66 out=49b] v64'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55caf9247200 con 0x55caf925d800 2023-10-11T09:45:59.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7fd2aaf83700 1 -- 172.21.15.78:0/2359111291 <== osd.7 v2:172.21.15.133:6824/35901 4 ==== osd_op_reply(11 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv24 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55ec99394fc0 con 0x55ec989fa400 2023-10-11T09:45:59.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7fd2acfc8840 20 datascan.read_dentry: oid=10000000000.00000000 dname=11 frag=*, r=0 2023-10-11T09:45:59.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7fd2acfc8840 20 datascan.read_dentry: 11_head not found in result 2023-10-11T09:45:59.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7fd2acfc8840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000200 at 0x0x10000000000/11 with size=6291456 bytes 2023-10-11T09:45:59.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:12 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ec98a3a800 con 0x55ec989fa400 2023-10-11T09:45:59.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f5415d33700 1 -- 172.21.15.78:0/2226872998 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:22 9.3c 9:3c000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55caf93f6400 con 0x55caf92d3800 2023-10-11T09:45:59.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f3a0faaf700 1 -- 172.21.15.78:0/4084151273 <== osd.7 v2:172.21.15.133:6824/35901 9 ==== osd_op_reply(21 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x5595c5ebb200 con 0x5595c552fc00 2023-10-11T09:45:59.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f3a10af2840 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-10-11T09:45:59.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f3a10af2840 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-10-11T09:45:59.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f3a10af2840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/8, will insert to root 2023-10-11T09:45:59.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:22 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c5561400 con 0x5595c552fc00 2023-10-11T09:45:59.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f0c14fc0700 1 -- 172.21.15.78:0/1160862806 <== osd.7 v2:172.21.15.133:6824/35901 12 ==== osd_op_reply(20 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv24 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x556d59c13200 con 0x556d591fcc00 2023-10-11T09:45:59.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f0c16003840 20 datascan.read_dentry: oid=10000000000.00000000 dname=6 frag=*, r=0 2023-10-11T09:45:59.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f0c16003840 20 datascan.read_dentry: 6_head not found in result 2023-10-11T09:45:59.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f0c16003840 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fb at 0x0x10000000000/6 with size=6291456 bytes 2023-10-11T09:45:59.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:21 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x556d591fc400 con 0x556d591fcc00 2023-10-11T09:45:59.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f4f02735700 1 -- 172.21.15.78:0/3361204897 <== osd.7 v2:172.21.15.133:6824/35901 2 ==== osd_op_reply(9 10000000000.00000000 [omap-get-header out=274b] v0'0 uv24 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55ab479a6fc0 con 0x55ab47036c00 2023-10-11T09:45:59.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f4f03f79840 10 datascan.find_or_create_dirfrag: read_version = 24 2023-10-11T09:45:59.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f4f03f79840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-10-11T09:45:59.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:10 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab47037800 con 0x55ab47036c00 2023-10-11T09:45:59.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f541b53e700 1 -- 172.21.15.78:0/2226872998 <== osd.5 v2:172.21.15.133:6816/35899 2 ==== osd_op_reply(22 [pgnls-filter start_epoch 66 out=49b] v64'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55caf9247200 con 0x55caf92d3800 2023-10-11T09:45:59.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f5416534700 1 -- 172.21.15.78:0/2226872998 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:23 9.2 9:40000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55caf93f6000 con 0x55caf93f6800 2023-10-11T09:45:59.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f541ad3d700 1 -- 172.21.15.78:0/2226872998 <== osd.3 v2:172.21.15.78:6824/35738 5 ==== osd_op_reply(23 [pgnls-filter start_epoch 66 out=49b] v64'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55caf9247440 con 0x55caf93f6800 2023-10-11T09:45:59.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f5415d33700 1 -- 172.21.15.78:0/2226872998 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:24 9.22 9:44000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55caf9406000 con 0x55caf93f6c00 2023-10-11T09:45:59.216 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--worker_n', '3', '--worker_m', '7'] in time 0:00:00.495768, result: 2023-10-11T09:45:59.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f541a53c700 1 -- 172.21.15.78:0/2226872998 <== osd.7 v2:172.21.15.133:6824/35901 11 ==== osd_op_reply(24 [pgnls-filter start_epoch 66 out=49b] v64'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55caf9246fc0 con 0x55caf93f6c00 2023-10-11T09:45:59.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f5416534700 1 -- 172.21.15.78:0/2226872998 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:25 9.12 9:48000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55caf93f7400 con 0x55caf92d3800 2023-10-11T09:45:59.218 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f541b53e700 1 -- 172.21.15.78:0/2226872998 <== osd.5 v2:172.21.15.133:6816/35899 3 ==== osd_op_reply(25 [pgnls-filter start_epoch 66 out=49b] v64'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55caf9247200 con 0x55caf92d3800 2023-10-11T09:45:59.218 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f541c581840 1 -- 172.21.15.78:0/2226872998 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55caf925d800 msgr2=0x55caf8ff7b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.218 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.880+0000 7f541c581840 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55caf925d800 0x55caf8ff7b80 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.218 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f541c581840 1 -- 172.21.15.78:0/2226872998 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55caf92d2c00 msgr2=0x55caf8ff7600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f541c581840 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55caf92d2c00 0x55caf8ff7600 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f541c581840 1 -- 172.21.15.78:0/2226872998 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55caf92d3400 msgr2=0x55caf93e4000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f541c581840 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55caf92d3400 0x55caf93e4000 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f541c581840 1 -- 172.21.15.78:0/2226872998 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55caf93f6800 msgr2=0x55caf93e6c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f541c581840 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55caf93f6800 0x55caf93e6c00 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f541c581840 1 -- 172.21.15.78:0/2226872998 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55caf925d000 msgr2=0x55caf8ff8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f541c581840 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55caf925d000 0x55caf8ff8100 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f541c581840 1 -- 172.21.15.78:0/2226872998 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55caf92d3800 msgr2=0x55caf93e6680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f541c581840 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55caf92d3800 0x55caf93e6680 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f541c581840 1 -- 172.21.15.78:0/2226872998 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55caf93f6c00 msgr2=0x55caf93e4b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f541c581840 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55caf93f6c00 0x55caf93e4b00 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f541a53c700 1 -- 172.21.15.78:0/2226872998 reap_dead start 2023-10-11T09:45:59.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f541a53c700 1 -- 172.21.15.78:0/2226872998 reap_dead start 2023-10-11T09:45:59.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f541a53c700 1 -- 172.21.15.78:0/2226872998 reap_dead start 2023-10-11T09:45:59.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f541a53c700 1 -- 172.21.15.78:0/2226872998 reap_dead start 2023-10-11T09:45:59.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f541a53c700 1 -- 172.21.15.78:0/2226872998 reap_dead start 2023-10-11T09:45:59.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f541c581840 1 -- 172.21.15.78:0/2226872998 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55caf92d2400 msgr2=0x55caf8ff9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f541c581840 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55caf92d2400 0x55caf8ff9700 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x55caf92507b0 tx=0x55caf9d2af60 comp rx=0 tx=0).stop 2023-10-11T09:45:59.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f541c581840 1 -- 172.21.15.78:0/2226872998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55caf925c800 msgr2=0x55caf8ff8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f541c581840 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55caf925c800 0x55caf8ff8c00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x55caf92514a0 tx=0x55caf9d2a180 comp rx=0 tx=0).stop 2023-10-11T09:45:59.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f109d13c700 1 -- 172.21.15.78:0/1612535802 <== osd.7 v2:172.21.15.133:6824/35901 4 ==== osd_op_reply(13 10000000000.00000000 [omap-set-vals] v66'25 uv25 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x560b64b24fc0 con 0x560b6413c400 2023-10-11T09:45:59.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f109f181840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/21 pointing to 0x0x1000000020a 2023-10-11T09:45:59.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:14 9.2a 9:57fe7535::::head [pgnls-filter start_epoch 66 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b6413d800 con 0x560b64119800 2023-10-11T09:45:59.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f541c581840 1 -- 172.21.15.78:0/2226872998 shutdown_connections 2023-10-11T09:45:59.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f541c581840 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55caf92d2400 0x55caf8ff9700 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f541c581840 1 --2- 172.21.15.78:0/2226872998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55caf925c800 0x55caf8ff8c00 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f541c581840 1 -- 172.21.15.78:0/2226872998 >> 172.21.15.78:0/2226872998 conn(0x55caf8f50c00 msgr2=0x55caf9031800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:59.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f541c581840 1 -- 172.21.15.78:0/2226872998 shutdown_connections 2023-10-11T09:45:59.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f541c581840 1 -- 172.21.15.78:0/2226872998 wait complete. 2023-10-11T09:45:59.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f109e13e700 1 -- 172.21.15.78:0/1612535802 <== osd.3 v2:172.21.15.78:6824/35738 3 ==== osd_op_reply(14 [pgnls-filter start_epoch 66 out=81b] v66'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x560b64b25200 con 0x560b64119800 2023-10-11T09:45:59.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f109f181840 20 datascan.forall_objects: OSD matched oid 100000001f6.00000000 2023-10-11T09:45:59.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f109f181840 10 datascan.operator(): handling object 100000001f6.0 2023-10-11T09:45:59.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:15 9.2a 9:57fe7535:::100000001f6.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b6413dc00 con 0x560b64119800 2023-10-11T09:45:59.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f541c581840 1 -- 172.21.15.78:0/302911193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf8f51800 msgr2=0x55caf8ff6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f541c581840 1 --2- 172.21.15.78:0/302911193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf8f51800 0x55caf8ff6580 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x55caf9051410 tx=0x55caf9d2afc0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f541c581840 1 -- 172.21.15.78:0/302911193 shutdown_connections 2023-10-11T09:45:59.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f541c581840 1 --2- 172.21.15.78:0/302911193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55caf8f51000 0x55caf8ff7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f541c581840 1 --2- 172.21.15.78:0/302911193 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55caf8f51400 0x55caf8ff6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f541c581840 1 --2- 172.21.15.78:0/302911193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf8f51800 0x55caf8ff6580 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f541c581840 1 -- 172.21.15.78:0/302911193 >> 172.21.15.78:0/302911193 conn(0x55caf8f51c00 msgr2=0x55caf9031000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:59.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f109e13e700 1 -- 172.21.15.78:0/1612535802 <== osd.3 v2:172.21.15.78:6824/35738 4 ==== osd_op_reply(15 100000001f6.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x560b64b25200 con 0x560b64119800 2023-10-11T09:45:59.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f541c581840 1 -- 172.21.15.78:0/302911193 shutdown_connections 2023-10-11T09:45:59.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f541c581840 1 -- 172.21.15.78:0/302911193 wait complete. 2023-10-11T09:45:59.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:16 10.a 10:57fe7535:::100000001f6.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b6413d800 con 0x560b64119c00 2023-10-11T09:45:59.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f109d93d700 1 -- 172.21.15.78:0/1612535802 <== osd.6 v2:172.21.15.133:6800/35900 2 ==== osd_op_reply(16 100000001f6.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x560b64b25440 con 0x560b64119c00 2023-10-11T09:45:59.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f109f181840 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-10-11T09:45:59.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f109f181840 10 datascan.inject_with_backtrace: inode: 0x0x100000001f6 2023-10-11T09:45:59.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f109f181840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/1 2023-10-11T09:45:59.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f109f181840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=1 2023-10-11T09:45:59.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:17 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b6413dc00 con 0x560b6413c400 2023-10-11T09:45:59.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7fd2aaf83700 1 -- 172.21.15.78:0/2359111291 <== osd.7 v2:172.21.15.133:6824/35901 5 ==== osd_op_reply(12 10000000000.00000000 [omap-set-vals] v66'26 uv26 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55ec99394fc0 con 0x55ec989fa400 2023-10-11T09:45:59.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7fd2acfc8840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/11 pointing to 0x0x10000000200 2023-10-11T09:45:59.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.884+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:13 9.19 9:997b7129::::head [pgnls-filter start_epoch 66 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ec98a3ac00 con 0x55ec9943f400 2023-10-11T09:45:59.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.888+0000 7f4f02735700 1 -- 172.21.15.78:0/3361204897 <== osd.7 v2:172.21.15.133:6824/35901 3 ==== osd_op_reply(10 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv26 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55ab479a6fc0 con 0x55ab47036c00 2023-10-11T09:45:59.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.888+0000 7f4f03f79840 20 datascan.read_dentry: oid=10000000000.00000000 dname=9 frag=*, r=0 2023-10-11T09:45:59.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.888+0000 7f4f03f79840 20 datascan.read_dentry: 9_head not found in result 2023-10-11T09:45:59.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.888+0000 7f4f03f79840 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fe at 0x0x10000000000/9 with size=6291456 bytes 2023-10-11T09:45:59.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.888+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:11 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab47037c00 con 0x55ab47036c00 2023-10-11T09:45:59.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.888+0000 7fd2abf85700 1 -- 172.21.15.78:0/2359111291 <== osd.3 v2:172.21.15.78:6824/35738 3 ==== osd_op_reply(13 [pgnls-filter start_epoch 66 out=81b] v66'18 uv18 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ec99395200 con 0x55ec9943f400 2023-10-11T09:45:59.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.888+0000 7fd2acfc8840 20 datascan.forall_objects: OSD matched oid 100000001f8.00000000 2023-10-11T09:45:59.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.888+0000 7fd2acfc8840 10 datascan.operator(): handling object 100000001f8.0 2023-10-11T09:45:59.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.888+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:14 9.19 9:997b7129:::100000001f8.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ec98a3b000 con 0x55ec9943f400 2023-10-11T09:45:59.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.888+0000 7fd2abf85700 1 -- 172.21.15.78:0/2359111291 <== osd.3 v2:172.21.15.78:6824/35738 4 ==== osd_op_reply(14 100000001f8.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x55ec99395200 con 0x55ec9943f400 2023-10-11T09:45:59.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.888+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:15 10.19 10:997b7129:::100000001f8.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ec98a3b400 con 0x55ec989fa800 2023-10-11T09:45:59.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.888+0000 7fd2ab784700 1 -- 172.21.15.78:0/2359111291 <== osd.1 v2:172.21.15.78:6809/35734 3 ==== osd_op_reply(15 100000001f8.00000000 [getxattr (30) out=30b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55ec99395440 con 0x55ec989fa800 2023-10-11T09:45:59.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.888+0000 7fd2acfc8840 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-10-11T09:45:59.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.888+0000 7fd2acfc8840 10 datascan.inject_with_backtrace: inode: 0x0x100000001f8 2023-10-11T09:45:59.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.888+0000 7fd2acfc8840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/3 2023-10-11T09:45:59.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.888+0000 7fd2acfc8840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=3 2023-10-11T09:45:59.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.888+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:16 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ec98a3b800 con 0x55ec989fa400 2023-10-11T09:45:59.239 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--worker_n', '1', '--worker_m', '7'] in time 0:00:00.519194, result: 2023-10-11T09:45:59.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.892+0000 7f0c14fc0700 1 -- 172.21.15.78:0/1160862806 <== osd.7 v2:172.21.15.133:6824/35901 13 ==== osd_op_reply(21 10000000000.00000000 [omap-set-vals] v66'27 uv27 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556d59c13200 con 0x556d591fcc00 2023-10-11T09:45:59.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.892+0000 7f0c16003840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/6 pointing to 0x0x100000001fb 2023-10-11T09:45:59.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.892+0000 7f0c16003840 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x556d5908d800 0x556d59209600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.892+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:22 9.33 9:cc000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x556d59229400 con 0x556d5908d800 2023-10-11T09:45:59.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.892+0000 7f3a0faaf700 1 -- 172.21.15.78:0/4084151273 <== osd.7 v2:172.21.15.133:6824/35901 10 ==== osd_op_reply(22 10000000000.00000000 [omap-get-header out=274b] v0'0 uv27 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5595c5ebb200 con 0x5595c552fc00 2023-10-11T09:45:59.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.892+0000 7f3a10af2840 10 datascan.find_or_create_dirfrag: read_version = 27 2023-10-11T09:45:59.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.892+0000 7f3a10af2840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-10-11T09:45:59.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.892+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:23 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c5561c00 con 0x5595c552fc00 2023-10-11T09:45:59.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.892+0000 7f109d13c700 1 -- 172.21.15.78:0/1612535802 <== osd.7 v2:172.21.15.133:6824/35901 5 ==== osd_op_reply(17 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x560b64b24fc0 con 0x560b6413c400 2023-10-11T09:45:59.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.892+0000 7f109f181840 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-10-11T09:45:59.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.892+0000 7f109f181840 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-10-11T09:45:59.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.892+0000 7f109f181840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/1, will insert to root 2023-10-11T09:45:59.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.892+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:18 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b6413c800 con 0x560b6413c400 2023-10-11T09:45:59.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.892+0000 7f0c13fbe700 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x556d5908d800 0x556d59209600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f0c13fbe700 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x556d5908d800 0x556d59209600 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f4f02735700 1 -- 172.21.15.78:0/3361204897 <== osd.7 v2:172.21.15.133:6824/35901 4 ==== osd_op_reply(11 10000000000.00000000 [omap-set-vals] v66'28 uv28 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55ab479a6fc0 con 0x55ab47036c00 2023-10-11T09:45:59.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f4f03f79840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/9 pointing to 0x0x100000001fe 2023-10-11T09:45:59.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f0c13fbe700 1 -- 172.21.15.78:0/1160862806 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== osd_op_reply(22 [pgnls-filter start_epoch 66 out=49b] v64'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d59c12fc0 con 0x556d5908d800 2023-10-11T09:45:59.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f4f03f79840 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55ab46f21400 0x55ab47011600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:12 9.7 9:e0000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab46f21c00 con 0x55ab46f21400 2023-10-11T09:45:59.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f0c0ffb6700 1 -- 172.21.15.78:0/1160862806 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:23 9.b 9:d0000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x556d59229800 con 0x556d591fc800 2023-10-11T09:45:59.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7fd2aaf83700 1 -- 172.21.15.78:0/2359111291 <== osd.7 v2:172.21.15.133:6824/35901 6 ==== osd_op_reply(16 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55ec99394fc0 con 0x55ec989fa400 2023-10-11T09:45:59.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7fd2acfc8840 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-10-11T09:45:59.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7fd2acfc8840 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-10-11T09:45:59.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7fd2acfc8840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/3, will insert to root 2023-10-11T09:45:59.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:17 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ec98a3bc00 con 0x55ec989fa400 2023-10-11T09:45:59.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f3a0faaf700 1 -- 172.21.15.78:0/4084151273 <== osd.7 v2:172.21.15.133:6824/35901 11 ==== osd_op_reply(23 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv28 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5595c5ebb200 con 0x5595c552fc00 2023-10-11T09:45:59.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f3a10af2840 20 datascan.read_dentry: oid=10000000000.00000000 dname=8 frag=*, r=0 2023-10-11T09:45:59.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f3a10af2840 20 datascan.read_dentry: 8_head not found in result 2023-10-11T09:45:59.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f3a10af2840 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fd at 0x0x10000000000/8 with size=6291456 bytes 2023-10-11T09:45:59.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:24 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c5560400 con 0x5595c552fc00 2023-10-11T09:45:59.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f109d13c700 1 -- 172.21.15.78:0/1612535802 <== osd.7 v2:172.21.15.133:6824/35901 6 ==== osd_op_reply(18 10000000000.00000000 [omap-get-header out=274b] v0'0 uv28 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x560b64b24fc0 con 0x560b6413c400 2023-10-11T09:45:59.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f109f181840 10 datascan.find_or_create_dirfrag: read_version = 28 2023-10-11T09:45:59.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f109f181840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-10-11T09:45:59.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:19 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b64118400 con 0x560b6413c400 2023-10-11T09:45:59.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f4f01f34700 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55ab46f21400 0x55ab47011600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f0c14fc0700 1 -- 172.21.15.78:0/1160862806 <== osd.0 v2:172.21.15.78:6800/35735 3 ==== osd_op_reply(23 [pgnls-filter start_epoch 66 out=49b] v64'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d59c13200 con 0x556d591fc800 2023-10-11T09:45:59.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7fd2aaf83700 1 -- 172.21.15.78:0/2359111291 <== osd.7 v2:172.21.15.133:6824/35901 7 ==== osd_op_reply(17 10000000000.00000000 [omap-get-header out=274b] v0'0 uv28 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55ec99394fc0 con 0x55ec989fa400 2023-10-11T09:45:59.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7fd2acfc8840 10 datascan.find_or_create_dirfrag: read_version = 28 2023-10-11T09:45:59.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7fd2acfc8840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-10-11T09:45:59.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f0c0f7b5700 1 -- 172.21.15.78:0/1160862806 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:24 9.2b 9:d4000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x556d59229000 con 0x556d591fcc00 2023-10-11T09:45:59.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:18 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ec98a3ac00 con 0x55ec989fa400 2023-10-11T09:45:59.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f4f01f34700 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55ab46f21400 0x55ab47011600 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f0c14fc0700 1 -- 172.21.15.78:0/1160862806 <== osd.7 v2:172.21.15.133:6824/35901 14 ==== osd_op_reply(24 [pgnls-filter start_epoch 66 out=81b] v66'15 uv15 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556d59c13200 con 0x556d591fcc00 2023-10-11T09:45:59.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f0c16003840 20 datascan.forall_objects: OSD matched oid 10000000203.00000000 2023-10-11T09:45:59.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f0c16003840 10 datascan.operator(): handling object 10000000203.0 2023-10-11T09:45:59.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:25 9.2b 9:d4a58307:::10000000203.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x556d59229c00 con 0x556d591fcc00 2023-10-11T09:45:59.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f0c14fc0700 1 -- 172.21.15.78:0/1160862806 <== osd.7 v2:172.21.15.133:6824/35901 15 ==== osd_op_reply(25 10000000203.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x556d59c13200 con 0x556d591fcc00 2023-10-11T09:45:59.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:26 10.b 10:d4a58307:::10000000203.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x556d59228800 con 0x556d5908d800 2023-10-11T09:45:59.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f4f01f34700 1 -- 172.21.15.78:0/3361204897 <== osd.5 v2:172.21.15.133:6816/35899 1 ==== osd_op_reply(12 [pgnls-filter start_epoch 66 out=101b] v66'13 uv13 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55ab479a6000 con 0x55ab46f21400 2023-10-11T09:45:59.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f4f03f79840 20 datascan.forall_objects: OSD matched oid 10000000207.00000000 2023-10-11T09:45:59.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f4f03f79840 10 datascan.operator(): handling object 10000000207.0 2023-10-11T09:45:59.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:13 9.7 9:e15a914f:::10000000207.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab4799b400 con 0x55ab46f21400 2023-10-11T09:45:59.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f3a0faaf700 1 -- 172.21.15.78:0/4084151273 <== osd.7 v2:172.21.15.133:6824/35901 12 ==== osd_op_reply(24 10000000000.00000000 [omap-set-vals] v66'29 uv29 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5595c5ebb200 con 0x5595c552fc00 2023-10-11T09:45:59.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f3a10af2840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/8 pointing to 0x0x100000001fd 2023-10-11T09:45:59.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:25 9.30 9:0c000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c552f400 con 0x5595c552e400 2023-10-11T09:45:59.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f0c13fbe700 1 -- 172.21.15.78:0/1160862806 <== osd.6 v2:172.21.15.133:6800/35900 2 ==== osd_op_reply(26 10000000203.00000000 [getxattr (30) out=30b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x556d59c12fc0 con 0x556d5908d800 2023-10-11T09:45:59.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f0c16003840 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-10-11T09:45:59.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f0c16003840 10 datascan.inject_with_backtrace: inode: 0x0x10000000203 2023-10-11T09:45:59.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f0c16003840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/14 2023-10-11T09:45:59.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f0c16003840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=14 2023-10-11T09:45:59.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.896+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:27 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x556d59228c00 con 0x556d591fcc00 2023-10-11T09:45:59.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f109d13c700 1 -- 172.21.15.78:0/1612535802 <== osd.7 v2:172.21.15.133:6824/35901 7 ==== osd_op_reply(19 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv29 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x560b64b24fc0 con 0x560b6413c400 2023-10-11T09:45:59.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f109f181840 20 datascan.read_dentry: oid=10000000000.00000000 dname=1 frag=*, r=0 2023-10-11T09:45:59.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f109f181840 20 datascan.read_dentry: 1_head not found in result 2023-10-11T09:45:59.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f109f181840 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f6 at 0x0x10000000000/1 with size=6291456 bytes 2023-10-11T09:45:59.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:20 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b64119400 con 0x560b6413c400 2023-10-11T09:45:59.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f4f01f34700 1 -- 172.21.15.78:0/3361204897 <== osd.5 v2:172.21.15.133:6816/35899 2 ==== osd_op_reply(13 10000000207.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x55ab479a6000 con 0x55ab46f21400 2023-10-11T09:45:59.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7fd2aaf83700 1 -- 172.21.15.78:0/2359111291 <== osd.7 v2:172.21.15.133:6824/35901 8 ==== osd_op_reply(18 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv29 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55ec99394fc0 con 0x55ec989fa400 2023-10-11T09:45:59.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f4f03f79840 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55ab47037000 0x55ab47012100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f3a0faaf700 1 -- 172.21.15.78:0/4084151273 <== osd.0 v2:172.21.15.78:6800/35735 4 ==== osd_op_reply(25 [pgnls-filter start_epoch 66 out=49b] v64'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5595c5ebb200 con 0x5595c552e400 2023-10-11T09:45:59.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7fd2acfc8840 20 datascan.read_dentry: oid=10000000000.00000000 dname=3 frag=*, r=0 2023-10-11T09:45:59.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7fd2acfc8840 20 datascan.read_dentry: 3_head not found in result 2023-10-11T09:45:59.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7fd2acfc8840 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f8 at 0x0x10000000000/3 with size=6291456 bytes 2023-10-11T09:45:59.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:19 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ec98a3a000 con 0x55ec989fa400 2023-10-11T09:45:59.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:14 10.7 10:e15a914f:::10000000207.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab47037400 con 0x55ab47037000 2023-10-11T09:45:59.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f0c14fc0700 1 -- 172.21.15.78:0/1160862806 <== osd.7 v2:172.21.15.133:6824/35901 16 ==== osd_op_reply(27 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x556d59c13200 con 0x556d591fcc00 2023-10-11T09:45:59.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f0c16003840 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-10-11T09:45:59.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f0c16003840 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-10-11T09:45:59.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f0c16003840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/14, will insert to root 2023-10-11T09:45:59.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f3a0a2a4700 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5595c5560800 0x5595c554a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:28 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x556d59228000 con 0x556d591fcc00 2023-10-11T09:45:59.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f3a0a2a4700 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:26 9.8 9:10000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c5560c00 con 0x5595c5560800 2023-10-11T09:45:59.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f4f02f36700 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55ab47037000 0x55ab47012100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f3a0f2ae700 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5595c5560800 0x5595c554a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f3a0f2ae700 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5595c5560800 0x5595c554a000 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f4f02f36700 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55ab47037000 0x55ab47012100 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f3a0f2ae700 1 -- 172.21.15.78:0/4084151273 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(26 [pgnls-filter start_epoch 66 out=49b] v64'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5595c5ebafc0 con 0x5595c5560800 2023-10-11T09:45:59.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f3a0aaa5700 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:27 9.28 9:14000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c5561400 con 0x5595c552f000 2023-10-11T09:45:59.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f4f02f36700 1 -- 172.21.15.78:0/3361204897 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== osd_op_reply(14 10000000207.00000000 [getxattr (30) out=30b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55ab479a6d80 con 0x55ab47037000 2023-10-11T09:45:59.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f4f03f79840 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-10-11T09:45:59.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f4f03f79840 10 datascan.inject_with_backtrace: inode: 0x0x10000000207 2023-10-11T09:45:59.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f4f03f79840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/18 2023-10-11T09:45:59.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f4f03f79840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=18 2023-10-11T09:45:59.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:15 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab46f20c00 con 0x55ab47036c00 2023-10-11T09:45:59.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f3a0eaad700 1 -- 172.21.15.78:0/4084151273 <== osd.6 v2:172.21.15.133:6800/35900 3 ==== osd_op_reply(27 [pgnls-filter start_epoch 66 out=101b] v66'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5595c5ebb440 con 0x5595c552f000 2023-10-11T09:45:59.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f3a10af2840 20 datascan.forall_objects: OSD matched oid 10000000202.00000000 2023-10-11T09:45:59.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f3a10af2840 10 datascan.operator(): handling object 10000000202.0 2023-10-11T09:45:59.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:28 9.28 9:15246602:::10000000202.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c552e000 con 0x5595c552f000 2023-10-11T09:45:59.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f3a0eaad700 1 -- 172.21.15.78:0/4084151273 <== osd.6 v2:172.21.15.133:6800/35900 4 ==== osd_op_reply(28 10000000202.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x5595c5ebb440 con 0x5595c552f000 2023-10-11T09:45:59.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:29 10.8 10:15246602:::10000000202.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c5561800 con 0x5595c552e400 2023-10-11T09:45:59.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f3a0faaf700 1 -- 172.21.15.78:0/4084151273 <== osd.0 v2:172.21.15.78:6800/35735 5 ==== osd_op_reply(29 10000000202.00000000 [getxattr (30) out=30b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x5595c5ebb200 con 0x5595c552e400 2023-10-11T09:45:59.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f3a10af2840 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-10-11T09:45:59.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f3a10af2840 10 datascan.inject_with_backtrace: inode: 0x0x10000000202 2023-10-11T09:45:59.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f3a10af2840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/13 2023-10-11T09:45:59.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f3a10af2840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=13 2023-10-11T09:45:59.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.900+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:30 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c5561400 con 0x5595c552fc00 2023-10-11T09:45:59.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.904+0000 7f109d13c700 1 -- 172.21.15.78:0/1612535802 <== osd.7 v2:172.21.15.133:6824/35901 8 ==== osd_op_reply(20 10000000000.00000000 [omap-set-vals] v66'30 uv30 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x560b64b24fc0 con 0x560b6413c400 2023-10-11T09:45:59.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.904+0000 7f109f181840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/1 pointing to 0x0x100000001f6 2023-10-11T09:45:59.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.904+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:21 9.1a 9:58000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b63ff1c00 con 0x560b64119800 2023-10-11T09:45:59.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.904+0000 7fd2aaf83700 1 -- 172.21.15.78:0/2359111291 <== osd.7 v2:172.21.15.133:6824/35901 9 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v66'31 uv31 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55ec99394fc0 con 0x55ec989fa400 2023-10-11T09:45:59.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.904+0000 7f109e13e700 1 -- 172.21.15.78:0/1612535802 <== osd.3 v2:172.21.15.78:6824/35738 5 ==== osd_op_reply(21 [pgnls-filter start_epoch 66 out=49b] v64'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560b64b25200 con 0x560b64119800 2023-10-11T09:45:59.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.904+0000 7fd2acfc8840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/3 pointing to 0x0x100000001f8 2023-10-11T09:45:59.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.904+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:20 9.39 9:9c000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ec98a3a400 con 0x55ec9943f000 2023-10-11T09:45:59.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.904+0000 7f1098933700 1 -- 172.21.15.78:0/1612535802 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:22 9.3a 9:5c000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b6413cc00 con 0x560b64118000 2023-10-11T09:45:59.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.904+0000 7f0c14fc0700 1 -- 172.21.15.78:0/1160862806 <== osd.7 v2:172.21.15.133:6824/35901 17 ==== osd_op_reply(28 10000000000.00000000 [omap-get-header out=274b] v0'0 uv31 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x556d59c13200 con 0x556d591fcc00 2023-10-11T09:45:59.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.904+0000 7f0c16003840 10 datascan.find_or_create_dirfrag: read_version = 31 2023-10-11T09:45:59.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.904+0000 7f0c16003840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-10-11T09:45:59.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.904+0000 7f4f02735700 1 -- 172.21.15.78:0/3361204897 <== osd.7 v2:172.21.15.133:6824/35901 5 ==== osd_op_reply(15 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55ab479a6fc0 con 0x55ab47036c00 2023-10-11T09:45:59.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.904+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:29 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x556d59229800 con 0x556d591fcc00 2023-10-11T09:45:59.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.904+0000 7f4f03f79840 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-10-11T09:45:59.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.904+0000 7f4f03f79840 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-10-11T09:45:59.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.904+0000 7f4f03f79840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/18, will insert to root 2023-10-11T09:45:59.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.904+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:16 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab47036400 con 0x55ab47036c00 2023-10-11T09:45:59.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.904+0000 7f3a0faaf700 1 -- 172.21.15.78:0/4084151273 <== osd.7 v2:172.21.15.133:6824/35901 13 ==== osd_op_reply(30 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x5595c5ebb200 con 0x5595c552fc00 2023-10-11T09:45:59.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.904+0000 7f3a10af2840 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-10-11T09:45:59.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.904+0000 7f3a10af2840 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-10-11T09:45:59.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.904+0000 7f3a10af2840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/13, will insert to root 2023-10-11T09:45:59.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:31 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c5561c00 con 0x5595c552fc00 2023-10-11T09:45:59.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7fd2aaf83700 1 -- 172.21.15.78:0/2359111291 <== osd.4 v2:172.21.15.133:6808/35898 2 ==== osd_op_reply(20 [pgnls-filter start_epoch 66 out=49b] v64'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ec99394fc0 con 0x55ec9943f000 2023-10-11T09:45:59.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7fd2a6f7b700 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:21 9.5 9:a0000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ec98a3a800 con 0x55ec9943f800 2023-10-11T09:45:59.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7f109e13e700 1 -- 172.21.15.78:0/1612535802 <== osd.0 v2:172.21.15.78:6800/35735 2 ==== osd_op_reply(22 [pgnls-filter start_epoch 66 out=49b] v64'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560b64b25200 con 0x560b64118000 2023-10-11T09:45:59.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7f0c14fc0700 1 -- 172.21.15.78:0/1160862806 <== osd.7 v2:172.21.15.133:6824/35901 18 ==== osd_op_reply(29 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv31 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x556d59c13200 con 0x556d591fcc00 2023-10-11T09:45:59.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7f1099134700 1 -- 172.21.15.78:0/1612535802 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:23 9.6 9:60000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b6413d000 con 0x560b64119000 2023-10-11T09:45:59.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7f0c16003840 20 datascan.read_dentry: oid=10000000000.00000000 dname=14 frag=*, r=0 2023-10-11T09:45:59.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7f0c16003840 20 datascan.read_dentry: 14_head not found in result 2023-10-11T09:45:59.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7f0c16003840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000203 at 0x0x10000000000/14 with size=6291456 bytes 2023-10-11T09:45:59.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:30 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x556d59229000 con 0x556d591fcc00 2023-10-11T09:45:59.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7f4f02735700 1 -- 172.21.15.78:0/3361204897 <== osd.7 v2:172.21.15.133:6824/35901 6 ==== osd_op_reply(16 10000000000.00000000 [omap-get-header out=274b] v0'0 uv31 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55ab479a6fc0 con 0x55ab47036c00 2023-10-11T09:45:59.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7f4f03f79840 10 datascan.find_or_create_dirfrag: read_version = 31 2023-10-11T09:45:59.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7f4f03f79840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-10-11T09:45:59.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:17 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab47037400 con 0x55ab47036c00 2023-10-11T09:45:59.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7f3a0faaf700 1 -- 172.21.15.78:0/4084151273 <== osd.7 v2:172.21.15.133:6824/35901 14 ==== osd_op_reply(31 10000000000.00000000 [omap-get-header out=274b] v0'0 uv31 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5595c5ebb200 con 0x5595c552fc00 2023-10-11T09:45:59.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7f3a10af2840 10 datascan.find_or_create_dirfrag: read_version = 31 2023-10-11T09:45:59.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7f3a10af2840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-10-11T09:45:59.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:32 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c552f800 con 0x5595c552fc00 2023-10-11T09:45:59.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7f109d13c700 1 -- 172.21.15.78:0/1612535802 <== osd.5 v2:172.21.15.133:6816/35899 3 ==== osd_op_reply(23 [pgnls-filter start_epoch 66 out=49b] v64'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560b64b24fc0 con 0x560b64119000 2023-10-11T09:45:59.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7f1098933700 1 -- 172.21.15.78:0/1612535802 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:24 9.26 9:64000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b64118c00 con 0x560b64119000 2023-10-11T09:45:59.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7fd2abf85700 1 -- 172.21.15.78:0/2359111291 <== osd.0 v2:172.21.15.78:6800/35735 3 ==== osd_op_reply(21 [pgnls-filter start_epoch 66 out=81b] v66'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ec99395200 con 0x55ec9943f800 2023-10-11T09:45:59.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7fd2acfc8840 20 datascan.forall_objects: OSD matched oid 10000000204.00000000 2023-10-11T09:45:59.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7fd2acfc8840 10 datascan.operator(): handling object 10000000204.0 2023-10-11T09:45:59.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:22 9.5 9:a0beda95:::10000000204.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ec989fb000 con 0x55ec9943f800 2023-10-11T09:45:59.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7f109d13c700 1 -- 172.21.15.78:0/1612535802 <== osd.5 v2:172.21.15.133:6816/35899 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 66 out=81b] v66'19 uv19 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x560b64b24fc0 con 0x560b64119000 2023-10-11T09:45:59.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7f109f181840 20 datascan.forall_objects: OSD matched oid 100000001ff.00000000 2023-10-11T09:45:59.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7f109f181840 10 datascan.operator(): handling object 100000001ff.0 2023-10-11T09:45:59.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:25 9.26 9:658f2455:::100000001ff.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b6413c800 con 0x560b64119000 2023-10-11T09:45:59.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7fd2abf85700 1 -- 172.21.15.78:0/2359111291 <== osd.0 v2:172.21.15.78:6800/35735 4 ==== osd_op_reply(22 10000000204.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv9 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x55ec99395200 con 0x55ec9943f800 2023-10-11T09:45:59.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7fd2acfc8840 1 --2- 172.21.15.78:0/2359111291 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55ec989fb400 0x55ec98a01b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:23 10.5 10:a0beda95:::10000000204.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ec989fbc00 con 0x55ec989fb400 2023-10-11T09:45:59.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7f109d13c700 1 -- 172.21.15.78:0/1612535802 <== osd.5 v2:172.21.15.133:6816/35899 5 ==== osd_op_reply(25 100000001ff.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv19 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x560b64b24fc0 con 0x560b64119000 2023-10-11T09:45:59.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:26 10.6 10:658f2455:::100000001ff.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b6413d000 con 0x560b64119000 2023-10-11T09:45:59.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7fd2ab784700 1 --2- 172.21.15.78:0/2359111291 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55ec989fb400 0x55ec98a01b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7f109d13c700 1 -- 172.21.15.78:0/1612535802 <== osd.5 v2:172.21.15.133:6816/35899 6 ==== osd_op_reply(26 100000001ff.00000000 [getxattr (30) out=30b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x560b64b24fc0 con 0x560b64119000 2023-10-11T09:45:59.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7f109f181840 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-10-11T09:45:59.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7f109f181840 10 datascan.inject_with_backtrace: inode: 0x0x100000001ff 2023-10-11T09:45:59.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7f109f181840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/10 2023-10-11T09:45:59.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7f109f181840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=10 2023-10-11T09:45:59.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:27 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b6413c000 con 0x560b6413c400 2023-10-11T09:45:59.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7fd2ab784700 1 --2- 172.21.15.78:0/2359111291 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55ec989fb400 0x55ec98a01b80 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7fd2ab784700 1 -- 172.21.15.78:0/2359111291 <== osd.2 v2:172.21.15.78:6808/35732 1 ==== osd_op_reply(23 10000000204.00000000 [getxattr (30) out=30b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55ec99395440 con 0x55ec989fb400 2023-10-11T09:45:59.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7fd2acfc8840 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-10-11T09:45:59.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7f0c14fc0700 1 -- 172.21.15.78:0/1160862806 <== osd.7 v2:172.21.15.133:6824/35901 19 ==== osd_op_reply(30 10000000000.00000000 [omap-set-vals] v66'32 uv32 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556d59c13200 con 0x556d591fcc00 2023-10-11T09:45:59.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7f0c16003840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/14 pointing to 0x0x10000000203 2023-10-11T09:45:59.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7fd2acfc8840 10 datascan.inject_with_backtrace: inode: 0x0x10000000204 2023-10-11T09:45:59.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7fd2acfc8840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/15 2023-10-11T09:45:59.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7fd2acfc8840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=15 2023-10-11T09:45:59.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:24 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ec98a3b400 con 0x55ec989fa400 2023-10-11T09:45:59.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.908+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:31 9.1b 9:d8000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x556d59229c00 con 0x556d59228400 2023-10-11T09:45:59.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f3a0faaf700 1 -- 172.21.15.78:0/4084151273 <== osd.7 v2:172.21.15.133:6824/35901 15 ==== osd_op_reply(32 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv32 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5595c5ebb200 con 0x5595c552fc00 2023-10-11T09:45:59.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f3a10af2840 20 datascan.read_dentry: oid=10000000000.00000000 dname=13 frag=*, r=0 2023-10-11T09:45:59.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f3a10af2840 20 datascan.read_dentry: 13_head not found in result 2023-10-11T09:45:59.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f3a10af2840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000202 at 0x0x10000000000/13 with size=6291456 bytes 2023-10-11T09:45:59.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:33 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c5417400 con 0x5595c552fc00 2023-10-11T09:45:59.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f4f02735700 1 -- 172.21.15.78:0/3361204897 <== osd.7 v2:172.21.15.133:6824/35901 7 ==== osd_op_reply(17 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv32 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55ab479a6fc0 con 0x55ab47036c00 2023-10-11T09:45:59.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f4f03f79840 20 datascan.read_dentry: oid=10000000000.00000000 dname=18 frag=*, r=0 2023-10-11T09:45:59.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f4f03f79840 20 datascan.read_dentry: 18_head not found in result 2023-10-11T09:45:59.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f4f03f79840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000207 at 0x0x10000000000/18 with size=6291456 bytes 2023-10-11T09:45:59.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:18 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab46f21800 con 0x55ab47036c00 2023-10-11T09:45:59.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f109d13c700 1 -- 172.21.15.78:0/1612535802 <== osd.7 v2:172.21.15.133:6824/35901 9 ==== osd_op_reply(27 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x560b64b24fc0 con 0x560b6413c400 2023-10-11T09:45:59.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f109f181840 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-10-11T09:45:59.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f109f181840 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-10-11T09:45:59.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f109f181840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/10, will insert to root 2023-10-11T09:45:59.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:28 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b6413d800 con 0x560b6413c400 2023-10-11T09:45:59.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7fd2aaf83700 1 -- 172.21.15.78:0/2359111291 <== osd.7 v2:172.21.15.133:6824/35901 10 ==== osd_op_reply(24 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55ec99394fc0 con 0x55ec989fa400 2023-10-11T09:45:59.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7fd2acfc8840 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-10-11T09:45:59.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7fd2acfc8840 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-10-11T09:45:59.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7fd2acfc8840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/15, will insert to root 2023-10-11T09:45:59.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f0c147bf700 1 -- 172.21.15.78:0/1160862806 <== osd.3 v2:172.21.15.78:6824/35738 2 ==== osd_op_reply(31 [pgnls-filter start_epoch 66 out=49b] v64'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d59c13440 con 0x556d59228400 2023-10-11T09:45:59.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:25 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ec98a3ac00 con 0x55ec989fa400 2023-10-11T09:45:59.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x556d591fc800 msgr2=0x556d58e31180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f0c16003840 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x556d591fc800 0x556d58e31180 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x556d591fd000 msgr2=0x556d59208000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f0c16003840 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x556d591fd000 0x556d59208000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x556d591fd400 msgr2=0x556d59208580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f0c16003840 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x556d591fd400 0x556d59208580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x556d59228400 msgr2=0x556d5920a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f0c16003840 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x556d59228400 0x556d5920a100 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x556d591fc000 msgr2=0x556d58e30c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f0c16003840 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x556d591fc000 0x556d58e30c00 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x556d5908d800 msgr2=0x556d59209600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f0c16003840 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x556d5908d800 0x556d59209600 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f0c13fbe700 1 -- 172.21.15.78:0/1160862806 reap_dead start 2023-10-11T09:45:59.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x556d591fcc00 msgr2=0x556d59209b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f0c16003840 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x556d591fcc00 0x556d59209b80 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f0c13fbe700 1 -- 172.21.15.78:0/1160862806 reap_dead start 2023-10-11T09:45:59.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f0c13fbe700 1 -- 172.21.15.78:0/1160862806 reap_dead start 2023-10-11T09:45:59.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f0c13fbe700 1 -- 172.21.15.78:0/1160862806 reap_dead start 2023-10-11T09:45:59.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x556d5908dc00 msgr2=0x556d58e31700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f0c16003840 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x556d5908dc00 0x556d58e31700 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x556d590a12c0 tx=0x556d59b80de0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d5908cc00 msgr2=0x556d58e30100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f0c16003840 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d5908cc00 0x556d58e30100 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x556d590a1320 tx=0x556d59b80ea0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 shutdown_connections 2023-10-11T09:45:59.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f0c16003840 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x556d5908dc00 0x556d58e31700 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f0c16003840 1 --2- 172.21.15.78:0/1160862806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d5908cc00 0x556d58e30100 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 >> 172.21.15.78:0/1160862806 conn(0x556d5908c400 msgr2=0x556d58e7d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:59.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 shutdown_connections 2023-10-11T09:45:59.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f0c16003840 1 -- 172.21.15.78:0/1160862806 wait complete. 2023-10-11T09:45:59.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f0c16003840 1 -- 172.21.15.78:0/299133903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d58d89000 msgr2=0x556d58e2eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f0c16003840 1 --2- 172.21.15.78:0/299133903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d58d89000 0x556d58e2eb00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x556d58e573b0 tx=0x556d59b80f60 comp rx=0 tx=0).stop 2023-10-11T09:45:59.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f0c16003840 1 -- 172.21.15.78:0/299133903 shutdown_connections 2023-10-11T09:45:59.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f0c16003840 1 --2- 172.21.15.78:0/299133903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d58d89c00 0x556d58e2f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f0c16003840 1 --2- 172.21.15.78:0/299133903 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556d58d89800 0x556d58e2f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f0c16003840 1 --2- 172.21.15.78:0/299133903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d58d89000 0x556d58e2eb00 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f0c16003840 1 -- 172.21.15.78:0/299133903 >> 172.21.15.78:0/299133903 conn(0x556d58d88c00 msgr2=0x556d58e7d000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:59.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f0c16003840 1 -- 172.21.15.78:0/299133903 shutdown_connections 2023-10-11T09:45:59.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.912+0000 7f0c16003840 1 -- 172.21.15.78:0/299133903 wait complete. 2023-10-11T09:45:59.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.916+0000 7f4f02735700 1 -- 172.21.15.78:0/3361204897 <== osd.7 v2:172.21.15.133:6824/35901 8 ==== osd_op_reply(18 10000000000.00000000 [omap-set-vals] v66'33 uv33 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55ab479a6fc0 con 0x55ab47036c00 2023-10-11T09:45:59.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.916+0000 7f4f03f79840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/18 pointing to 0x0x10000000207 2023-10-11T09:45:59.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.916+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:19 9.7 9:e2e07930::::head [pgnls-filter start_epoch 66 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab4799ac00 con 0x55ab46f21400 2023-10-11T09:45:59.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7f3a0faaf700 1 -- 172.21.15.78:0/4084151273 <== osd.7 v2:172.21.15.133:6824/35901 16 ==== osd_op_reply(33 10000000000.00000000 [omap-set-vals] v66'34 uv34 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5595c5ebb200 con 0x5595c552fc00 2023-10-11T09:45:59.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7f3a10af2840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/13 pointing to 0x0x10000000202 2023-10-11T09:45:59.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:34 9.28 9:177a762d::::head [pgnls-filter start_epoch 66 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c552f400 con 0x5595c552f000 2023-10-11T09:45:59.317 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--worker_n', '5', '--worker_m', '7'] in time 0:00:00.594844, result: 2023-10-11T09:45:59.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7f109d13c700 1 -- 172.21.15.78:0/1612535802 <== osd.7 v2:172.21.15.133:6824/35901 10 ==== osd_op_reply(28 10000000000.00000000 [omap-get-header out=274b] v0'0 uv34 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x560b64b24fc0 con 0x560b6413c400 2023-10-11T09:45:59.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7f109f181840 10 datascan.find_or_create_dirfrag: read_version = 34 2023-10-11T09:45:59.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7f109f181840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-10-11T09:45:59.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:29 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b64118400 con 0x560b6413c400 2023-10-11T09:45:59.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7f4f01f34700 1 -- 172.21.15.78:0/3361204897 <== osd.5 v2:172.21.15.133:6816/35899 3 ==== osd_op_reply(19 [pgnls-filter start_epoch 66 out=81b] v66'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ab479a6000 con 0x55ab46f21400 2023-10-11T09:45:59.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7fd2aaf83700 1 -- 172.21.15.78:0/2359111291 <== osd.7 v2:172.21.15.133:6824/35901 11 ==== osd_op_reply(25 10000000000.00000000 [omap-get-header out=274b] v0'0 uv34 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55ec99394fc0 con 0x55ec989fa400 2023-10-11T09:45:59.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7fd2acfc8840 10 datascan.find_or_create_dirfrag: read_version = 34 2023-10-11T09:45:59.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7fd2acfc8840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-10-11T09:45:59.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7f4f03f79840 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2023-10-11T09:45:59.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7f4f03f79840 10 datascan.operator(): handling object 10000000001.0 2023-10-11T09:45:59.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:26 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ec989fac00 con 0x55ec989fa400 2023-10-11T09:45:59.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:20 9.7 9:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab47037800 con 0x55ab46f21400 2023-10-11T09:45:59.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7f3a0eaad700 1 -- 172.21.15.78:0/4084151273 <== osd.6 v2:172.21.15.133:6800/35900 5 ==== osd_op_reply(34 [pgnls-filter start_epoch 66 out=81b] v66'14 uv14 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5595c5ebb440 con 0x5595c552f000 2023-10-11T09:45:59.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7f3a10af2840 20 datascan.forall_objects: OSD matched oid 10000000201.00000000 2023-10-11T09:45:59.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7f3a10af2840 10 datascan.operator(): handling object 10000000201.0 2023-10-11T09:45:59.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:35 9.28 9:177a762d:::10000000201.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c552f800 con 0x5595c552f000 2023-10-11T09:45:59.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7f109d13c700 1 -- 172.21.15.78:0/1612535802 <== osd.7 v2:172.21.15.133:6824/35901 11 ==== osd_op_reply(29 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv34 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x560b64b24fc0 con 0x560b6413c400 2023-10-11T09:45:59.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7f109f181840 20 datascan.read_dentry: oid=10000000000.00000000 dname=10 frag=*, r=0 2023-10-11T09:45:59.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7f109f181840 20 datascan.read_dentry: 10_head not found in result 2023-10-11T09:45:59.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7f109f181840 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001ff at 0x0x10000000000/10 with size=6291456 bytes 2023-10-11T09:45:59.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:30 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b64119400 con 0x560b6413c400 2023-10-11T09:45:59.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7f4f01f34700 1 -- 172.21.15.78:0/3361204897 <== osd.5 v2:172.21.15.133:6816/35899 4 ==== osd_op_reply(20 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x55ab479a6000 con 0x55ab46f21400 2023-10-11T09:45:59.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7fd2aaf83700 1 -- 172.21.15.78:0/2359111291 <== osd.7 v2:172.21.15.133:6824/35901 12 ==== osd_op_reply(26 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv34 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55ec99394fc0 con 0x55ec989fa400 2023-10-11T09:45:59.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7fd2acfc8840 20 datascan.read_dentry: oid=10000000000.00000000 dname=15 frag=*, r=0 2023-10-11T09:45:59.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7fd2acfc8840 20 datascan.read_dentry: 15_head not found in result 2023-10-11T09:45:59.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7fd2acfc8840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000204 at 0x0x10000000000/15 with size=6291456 bytes 2023-10-11T09:45:59.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:21 10.7 10:e2e07930:::10000000001.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab47037c00 con 0x55ab47037000 2023-10-11T09:45:59.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:27 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ec989fb800 con 0x55ec989fa400 2023-10-11T09:45:59.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7f3a0eaad700 1 -- 172.21.15.78:0/4084151273 <== osd.6 v2:172.21.15.133:6800/35900 6 ==== osd_op_reply(35 10000000201.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv14 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x5595c5ebb440 con 0x5595c552f000 2023-10-11T09:45:59.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:36 10.8 10:177a762d:::10000000201.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c5560400 con 0x5595c552e400 2023-10-11T09:45:59.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7f3a0faaf700 1 -- 172.21.15.78:0/4084151273 <== osd.0 v2:172.21.15.78:6800/35735 6 ==== osd_op_reply(36 10000000201.00000000 [getxattr (30) out=30b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x5595c5ebb200 con 0x5595c552e400 2023-10-11T09:45:59.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7f3a10af2840 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-10-11T09:45:59.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7f3a10af2840 10 datascan.inject_with_backtrace: inode: 0x0x10000000201 2023-10-11T09:45:59.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7f3a10af2840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/12 2023-10-11T09:45:59.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7f3a10af2840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=12 2023-10-11T09:45:59.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7f4f02f36700 1 -- 172.21.15.78:0/3361204897 <== osd.6 v2:172.21.15.133:6800/35900 2 ==== osd_op_reply(21 10000000001.00000000 [getxattr (30) out=30b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55ab479a6d80 con 0x55ab47037000 2023-10-11T09:45:59.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7f4f03f79840 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-10-11T09:45:59.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:37 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c5561800 con 0x5595c552fc00 2023-10-11T09:45:59.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7f4f03f79840 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2023-10-11T09:45:59.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7f4f03f79840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/0 2023-10-11T09:45:59.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7f4f03f79840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=0 2023-10-11T09:45:59.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.920+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:22 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab47036400 con 0x55ab47036c00 2023-10-11T09:45:59.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109d13c700 1 -- 172.21.15.78:0/1612535802 <== osd.7 v2:172.21.15.133:6824/35901 12 ==== osd_op_reply(30 10000000000.00000000 [omap-set-vals] v66'35 uv35 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x560b64b24fc0 con 0x560b6413c400 2023-10-11T09:45:59.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109f181840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/10 pointing to 0x0x100000001ff 2023-10-11T09:45:59.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:31 9.16 9:68000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b64118c00 con 0x560b63ff1800 2023-10-11T09:45:59.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109d13c700 1 -- 172.21.15.78:0/1612535802 <== osd.4 v2:172.21.15.133:6808/35898 3 ==== osd_op_reply(31 [pgnls-filter start_epoch 66 out=49b] v64'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560b64b24fc0 con 0x560b63ff1800 2023-10-11T09:45:59.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f1098933700 1 -- 172.21.15.78:0/1612535802 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:32 9.36 9:6c000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x560b6413c800 con 0x560b64118800 2023-10-11T09:45:59.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109d93d700 1 -- 172.21.15.78:0/1612535802 <== osd.1 v2:172.21.15.78:6809/35734 2 ==== osd_op_reply(32 [pgnls-filter start_epoch 66 out=101b] v66'23 uv23 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x560b64b25440 con 0x560b64118800 2023-10-11T09:45:59.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x560b64118000 msgr2=0x560b63d61180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109f181840 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x560b64118000 0x560b63d61180 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x560b64118800 msgr2=0x560b64120000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109f181840 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x560b64118800 0x560b64120000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x560b64119800 msgr2=0x560b64121b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109f181840 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x560b64119800 0x560b64121b80 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x560b63ff1800 msgr2=0x560b63d60c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109f181840 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x560b63ff1800 0x560b63d60c00 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x560b64119000 msgr2=0x560b64120580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109f181840 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x560b64119000 0x560b64120580 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x560b64119c00 msgr2=0x560b64121080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109f181840 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x560b64119c00 0x560b64121080 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x560b6413c400 msgr2=0x560b64121600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109f181840 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x560b6413c400 0x560b64121600 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109d93d700 1 -- 172.21.15.78:0/1612535802 reap_dead start 2023-10-11T09:45:59.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109d93d700 1 -- 172.21.15.78:0/1612535802 reap_dead start 2023-10-11T09:45:59.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x560b63ff1400 msgr2=0x560b63d61700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109f181840 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x560b63ff1400 0x560b63d61700 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x560b63dd0870 tx=0x560b63fb0fc0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b63ff0800 msgr2=0x560b63d60100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109f181840 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b63ff0800 0x560b63d60100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x560b63fae900 tx=0x560b64a9c2a0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 shutdown_connections 2023-10-11T09:45:59.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109f181840 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x560b6413c400 0x560b64121600 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109f181840 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x560b63ff1400 0x560b63d61700 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109f181840 1 --2- 172.21.15.78:0/1612535802 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b63ff0800 0x560b63d60100 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 >> 172.21.15.78:0/1612535802 conn(0x560b63ff0000 msgr2=0x560b63d75800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:59.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 shutdown_connections 2023-10-11T09:45:59.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109f181840 1 -- 172.21.15.78:0/1612535802 wait complete. 2023-10-11T09:45:59.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109f181840 1 -- 172.21.15.78:0/27808337 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b63cb9800 msgr2=0x560b63d5f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109f181840 1 --2- 172.21.15.78:0/27808337 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b63cb9800 0x560b63d5f080 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x560b63dd1260 tx=0x560b64a9c000 comp rx=0 tx=0).stop 2023-10-11T09:45:59.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109f181840 1 -- 172.21.15.78:0/27808337 shutdown_connections 2023-10-11T09:45:59.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109f181840 1 --2- 172.21.15.78:0/27808337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b63cb9c00 0x560b63d5f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109f181840 1 --2- 172.21.15.78:0/27808337 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560b63cb9800 0x560b63d5f080 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109f181840 1 --2- 172.21.15.78:0/27808337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b63cb9000 0x560b63d5eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109f181840 1 -- 172.21.15.78:0/27808337 >> 172.21.15.78:0/27808337 conn(0x560b63cb8c00 msgr2=0x560b63d75000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:59.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109f181840 1 -- 172.21.15.78:0/27808337 shutdown_connections 2023-10-11T09:45:59.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.924+0000 7f109f181840 1 -- 172.21.15.78:0/27808337 wait complete. 2023-10-11T09:45:59.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7fd2aaf83700 1 -- 172.21.15.78:0/2359111291 <== osd.7 v2:172.21.15.133:6824/35901 13 ==== osd_op_reply(27 10000000000.00000000 [omap-set-vals] v66'36 uv36 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55ec99394fc0 con 0x55ec989fa400 2023-10-11T09:45:59.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7fd2acfc8840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/15 pointing to 0x0x10000000204 2023-10-11T09:45:59.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:28 9.25 9:a4000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ec989fbc00 con 0x55ec9943f800 2023-10-11T09:45:59.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7f3a0faaf700 1 -- 172.21.15.78:0/4084151273 <== osd.7 v2:172.21.15.133:6824/35901 17 ==== osd_op_reply(37 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x5595c5ebb200 con 0x5595c552fc00 2023-10-11T09:45:59.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7f3a10af2840 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-10-11T09:45:59.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7f3a10af2840 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-10-11T09:45:59.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7f3a10af2840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/12, will insert to root 2023-10-11T09:45:59.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:38 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c5561400 con 0x5595c552fc00 2023-10-11T09:45:59.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7f4f02735700 1 -- 172.21.15.78:0/3361204897 <== osd.7 v2:172.21.15.133:6824/35901 9 ==== osd_op_reply(22 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55ab479a6fc0 con 0x55ab47036c00 2023-10-11T09:45:59.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7f4f03f79840 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-10-11T09:45:59.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7f4f03f79840 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-10-11T09:45:59.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7f4f03f79840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/0, will insert to root 2023-10-11T09:45:59.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:23 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab46f20c00 con 0x55ab47036c00 2023-10-11T09:45:59.347 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--worker_n', '2', '--worker_m', '7'] in time 0:00:00.626970, result: 2023-10-11T09:45:59.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7fd2abf85700 1 -- 172.21.15.78:0/2359111291 <== osd.0 v2:172.21.15.78:6800/35735 5 ==== osd_op_reply(28 [pgnls-filter start_epoch 66 out=49b] v64'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ec99395200 con 0x55ec9943f800 2023-10-11T09:45:59.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7fd2a6f7b700 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:29 9.15 9:a8000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ec98a3b800 con 0x55ec989fa400 2023-10-11T09:45:59.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7f3a0faaf700 1 -- 172.21.15.78:0/4084151273 <== osd.7 v2:172.21.15.133:6824/35901 18 ==== osd_op_reply(38 10000000000.00000000 [omap-get-header out=274b] v0'0 uv36 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5595c5ebb200 con 0x5595c552fc00 2023-10-11T09:45:59.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7f3a10af2840 10 datascan.find_or_create_dirfrag: read_version = 36 2023-10-11T09:45:59.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7f3a10af2840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-10-11T09:45:59.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:39 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c5561c00 con 0x5595c552fc00 2023-10-11T09:45:59.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7f4f02735700 1 -- 172.21.15.78:0/3361204897 <== osd.7 v2:172.21.15.133:6824/35901 10 ==== osd_op_reply(23 10000000000.00000000 [omap-get-header out=274b] v0'0 uv36 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55ab479a6fc0 con 0x55ab47036c00 2023-10-11T09:45:59.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7f4f03f79840 10 datascan.find_or_create_dirfrag: read_version = 36 2023-10-11T09:45:59.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7f4f03f79840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-10-11T09:45:59.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:24 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab47037800 con 0x55ab47036c00 2023-10-11T09:45:59.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7f3a0faaf700 1 -- 172.21.15.78:0/4084151273 <== osd.7 v2:172.21.15.133:6824/35901 19 ==== osd_op_reply(39 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv36 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5595c5ebb200 con 0x5595c552fc00 2023-10-11T09:45:59.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7f3a10af2840 20 datascan.read_dentry: oid=10000000000.00000000 dname=12 frag=*, r=0 2023-10-11T09:45:59.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7f3a10af2840 20 datascan.read_dentry: 12_head not found in result 2023-10-11T09:45:59.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7f3a10af2840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000201 at 0x0x10000000000/12 with size=6291456 bytes 2023-10-11T09:45:59.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:40 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c5556000 con 0x5595c552fc00 2023-10-11T09:45:59.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7f4f02735700 1 -- 172.21.15.78:0/3361204897 <== osd.7 v2:172.21.15.133:6824/35901 11 ==== osd_op_reply(24 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv36 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55ab479a6fc0 con 0x55ab47036c00 2023-10-11T09:45:59.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7f4f03f79840 20 datascan.read_dentry: oid=10000000000.00000000 dname=0 frag=*, r=0 2023-10-11T09:45:59.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7f4f03f79840 20 datascan.read_dentry: 0_head not found in result 2023-10-11T09:45:59.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7f4f03f79840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/0 with size=6291456 bytes 2023-10-11T09:45:59.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:25 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab4799b400 con 0x55ab47036c00 2023-10-11T09:45:59.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7fd2aaf83700 1 -- 172.21.15.78:0/2359111291 <== osd.7 v2:172.21.15.133:6824/35901 14 ==== osd_op_reply(29 [pgnls-filter start_epoch 66 out=81b] v66'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ec99394fc0 con 0x55ec989fa400 2023-10-11T09:45:59.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7fd2acfc8840 20 datascan.forall_objects: OSD matched oid 1000000020d.00000000 2023-10-11T09:45:59.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7fd2acfc8840 10 datascan.operator(): handling object 1000000020d.0 2023-10-11T09:45:59.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:30 9.15 9:a8e69ea8:::1000000020d.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ec989fb000 con 0x55ec989fa400 2023-10-11T09:45:59.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7fd2aaf83700 1 -- 172.21.15.78:0/2359111291 <== osd.7 v2:172.21.15.133:6824/35901 15 ==== osd_op_reply(30 1000000020d.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv10 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x55ec99394fc0 con 0x55ec989fa400 2023-10-11T09:45:59.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:31 10.15 10:a8e69ea8:::1000000020d.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ec989fbc00 con 0x55ec9943f400 2023-10-11T09:45:59.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7fd2abf85700 1 -- 172.21.15.78:0/2359111291 <== osd.3 v2:172.21.15.78:6824/35738 5 ==== osd_op_reply(31 1000000020d.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55ec99395200 con 0x55ec9943f400 2023-10-11T09:45:59.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7fd2acfc8840 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-10-11T09:45:59.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7fd2acfc8840 10 datascan.inject_with_backtrace: inode: 0x0x1000000020d 2023-10-11T09:45:59.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7fd2acfc8840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/24 2023-10-11T09:45:59.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7fd2acfc8840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=24 2023-10-11T09:45:59.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.932+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:32 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ec989fac00 con 0x55ec989fa400 2023-10-11T09:45:59.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7f3a0faaf700 1 -- 172.21.15.78:0/4084151273 <== osd.7 v2:172.21.15.133:6824/35901 20 ==== osd_op_reply(40 10000000000.00000000 [omap-set-vals] v66'37 uv37 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5595c5ebb200 con 0x5595c552fc00 2023-10-11T09:45:59.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7f3a10af2840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/12 pointing to 0x0x10000000201 2023-10-11T09:45:59.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:41 9.18 9:18000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c5556400 con 0x5595c552f000 2023-10-11T09:45:59.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7f3a0eaad700 1 -- 172.21.15.78:0/4084151273 <== osd.6 v2:172.21.15.133:6800/35900 7 ==== osd_op_reply(41 [pgnls-filter start_epoch 66 out=49b] v64'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5595c5ebb440 con 0x5595c552f000 2023-10-11T09:45:59.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7f3a0a2a4700 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:42 9.38 9:1c000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c5561000 con 0x5595c5560800 2023-10-11T09:45:59.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7f4f02735700 1 -- 172.21.15.78:0/3361204897 <== osd.7 v2:172.21.15.133:6824/35901 12 ==== osd_op_reply(25 10000000000.00000000 [omap-set-vals] v66'38 uv38 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55ab479a6fc0 con 0x55ab47036c00 2023-10-11T09:45:59.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7f3a0f2ae700 1 -- 172.21.15.78:0/4084151273 <== osd.3 v2:172.21.15.78:6824/35738 2 ==== osd_op_reply(42 [pgnls-filter start_epoch 66 out=49b] v64'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5595c5ebafc0 con 0x5595c5560800 2023-10-11T09:45:59.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7f4f03f79840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/0 pointing to 0x0x10000000001 2023-10-11T09:45:59.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:26 9.27 9:e4000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab4799ac00 con 0x55ab47036000 2023-10-11T09:45:59.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7f3a0aaa5700 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:43 9.4 9:20000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c5560000 con 0x5595c552ec00 2023-10-11T09:45:59.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7fd2aaf83700 1 -- 172.21.15.78:0/2359111291 <== osd.7 v2:172.21.15.133:6824/35901 16 ==== osd_op_reply(32 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55ec99394fc0 con 0x55ec989fa400 2023-10-11T09:45:59.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7fd2acfc8840 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-10-11T09:45:59.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7fd2acfc8840 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-10-11T09:45:59.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7fd2acfc8840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/24, will insert to root 2023-10-11T09:45:59.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:33 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ec98a3b000 con 0x55ec989fa400 2023-10-11T09:45:59.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7f4f02f36700 1 -- 172.21.15.78:0/3361204897 <== osd.2 v2:172.21.15.78:6808/35732 3 ==== osd_op_reply(26 [pgnls-filter start_epoch 66 out=81b] v66'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ab479a6d80 con 0x55ab47036000 2023-10-11T09:45:59.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7fd2aaf83700 1 -- 172.21.15.78:0/2359111291 <== osd.7 v2:172.21.15.133:6824/35901 17 ==== osd_op_reply(33 10000000000.00000000 [omap-get-header out=274b] v0'0 uv38 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55ec99394fc0 con 0x55ec989fa400 2023-10-11T09:45:59.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7fd2acfc8840 10 datascan.find_or_create_dirfrag: read_version = 38 2023-10-11T09:45:59.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7f4f03f79840 20 datascan.forall_objects: OSD matched oid 10000000208.00000000 2023-10-11T09:45:59.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7f4f03f79840 10 datascan.operator(): handling object 10000000208.0 2023-10-11T09:45:59.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7f3a0f2ae700 1 -- 172.21.15.78:0/4084151273 <== osd.1 v2:172.21.15.78:6809/35734 4 ==== osd_op_reply(43 [pgnls-filter start_epoch 66 out=49b] v64'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5595c5ebafc0 con 0x5595c552ec00 2023-10-11T09:45:59.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7fd2acfc8840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-10-11T09:45:59.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:34 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ec98a3a800 con 0x55ec989fa400 2023-10-11T09:45:59.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:27 9.27 9:e5993619:::10000000208.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab46f21c00 con 0x55ab47036000 2023-10-11T09:45:59.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7f3a0a2a4700 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x5595c5560c00 0x5595c554bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7f3a0a2a4700 1 -- 172.21.15.78:0/4084151273 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:44 9.24 9:24000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x5595c5561000 con 0x5595c5560c00 2023-10-11T09:45:59.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7f4f02f36700 1 -- 172.21.15.78:0/3361204897 <== osd.2 v2:172.21.15.78:6808/35732 4 ==== osd_op_reply(27 10000000208.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv6 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x55ab479a6d80 con 0x55ab47036000 2023-10-11T09:45:59.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7f3a0eaad700 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x5595c5560c00 0x5595c554bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:28 10.7 10:e5993619:::10000000208.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab47037400 con 0x55ab47037000 2023-10-11T09:45:59.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7fd2aaf83700 1 -- 172.21.15.78:0/2359111291 <== osd.7 v2:172.21.15.133:6824/35901 18 ==== osd_op_reply(34 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv38 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55ec99394fc0 con 0x55ec989fa400 2023-10-11T09:45:59.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7fd2acfc8840 20 datascan.read_dentry: oid=10000000000.00000000 dname=24 frag=*, r=0 2023-10-11T09:45:59.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7fd2acfc8840 20 datascan.read_dentry: 24_head not found in result 2023-10-11T09:45:59.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7fd2acfc8840 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000020d at 0x0x10000000000/24 with size=6291456 bytes 2023-10-11T09:45:59.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:35 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ec98a3a000 con 0x55ec989fa400 2023-10-11T09:45:59.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7f4f02f36700 1 -- 172.21.15.78:0/3361204897 <== osd.6 v2:172.21.15.133:6800/35900 3 ==== osd_op_reply(28 10000000208.00000000 [getxattr (30) out=30b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55ab479a6d80 con 0x55ab47037000 2023-10-11T09:45:59.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7f4f03f79840 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-10-11T09:45:59.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7f4f03f79840 10 datascan.inject_with_backtrace: inode: 0x0x10000000208 2023-10-11T09:45:59.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7f4f03f79840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/19 2023-10-11T09:45:59.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7f4f03f79840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=19 2023-10-11T09:45:59.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:29 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab47036400 con 0x55ab47036c00 2023-10-11T09:45:59.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.936+0000 7f3a0eaad700 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x5595c5560c00 0x5595c554bb80 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f3a0eaad700 1 -- 172.21.15.78:0/4084151273 <== osd.2 v2:172.21.15.78:6808/35732 1 ==== osd_op_reply(44 [pgnls-filter start_epoch 66 out=49b] v64'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5595c5ebb440 con 0x5595c5560c00 2023-10-11T09:45:59.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x5595c552e400 msgr2=0x5595c5177b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f3a10af2840 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x5595c552e400 0x5595c5177b80 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x5595c552ec00 msgr2=0x5595c5177600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f3a10af2840 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x5595c552ec00 0x5595c5177600 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x5595c5560c00 msgr2=0x5595c554bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f3a10af2840 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x5595c5560c00 0x5595c554bb80 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5595c5560800 msgr2=0x5595c554a000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f3a10af2840 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5595c5560800 0x5595c554a000 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x5595c5417c00 msgr2=0x5595c5178100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f3a10af2840 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x5595c5417c00 0x5595c5178100 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x5595c552f000 msgr2=0x5595c554a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f3a10af2840 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x5595c552f000 0x5595c554a580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x5595c552fc00 msgr2=0x5595c554ab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f3a10af2840 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x5595c552fc00 0x5595c554ab00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f3a0f2ae700 1 -- 172.21.15.78:0/4084151273 reap_dead start 2023-10-11T09:45:59.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f3a0f2ae700 1 -- 172.21.15.78:0/4084151273 reap_dead start 2023-10-11T09:45:59.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f3a0f2ae700 1 -- 172.21.15.78:0/4084151273 reap_dead start 2023-10-11T09:45:59.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f3a0f2ae700 1 -- 172.21.15.78:0/4084151273 reap_dead start 2023-10-11T09:45:59.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5595c5417800 msgr2=0x5595c5179700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f3a10af2840 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5595c5417800 0x5595c5179700 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x5595c51d89c0 tx=0x5595c5eb6e40 comp rx=0 tx=0).stop 2023-10-11T09:45:59.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595c5416c00 msgr2=0x5595c5178680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f3a10af2840 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595c5416c00 0x5595c5178680 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x5595c5ec7530 tx=0x5595c5eb2060 comp rx=0 tx=0).stop 2023-10-11T09:45:59.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 shutdown_connections 2023-10-11T09:45:59.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f3a10af2840 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5595c5417800 0x5595c5179700 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f3a10af2840 1 --2- 172.21.15.78:0/4084151273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595c5416c00 0x5595c5178680 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 >> 172.21.15.78:0/4084151273 conn(0x5595c5416000 msgr2=0x5595c51c5800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:59.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 shutdown_connections 2023-10-11T09:45:59.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f3a10af2840 1 -- 172.21.15.78:0/4084151273 wait complete. 2023-10-11T09:45:59.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f3a10af2840 1 -- 172.21.15.78:0/2018903296 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595c50d1400 msgr2=0x5595c5176b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f3a10af2840 1 --2- 172.21.15.78:0/2018903296 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595c50d1400 0x5595c5176b00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x5595c51d9500 tx=0x5595c5eb2000 comp rx=0 tx=0).stop 2023-10-11T09:45:59.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f3a10af2840 1 -- 172.21.15.78:0/2018903296 shutdown_connections 2023-10-11T09:45:59.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f3a10af2840 1 --2- 172.21.15.78:0/2018903296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595c50d1800 0x5595c5177080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f3a10af2840 1 --2- 172.21.15.78:0/2018903296 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595c50d1400 0x5595c5176b00 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f3a10af2840 1 --2- 172.21.15.78:0/2018903296 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5595c50d1000 0x5595c5176580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f3a10af2840 1 -- 172.21.15.78:0/2018903296 >> 172.21.15.78:0/2018903296 conn(0x5595c50d0c00 msgr2=0x5595c51c5000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:59.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f3a10af2840 1 -- 172.21.15.78:0/2018903296 shutdown_connections 2023-10-11T09:45:59.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f3a10af2840 1 -- 172.21.15.78:0/2018903296 wait complete. 2023-10-11T09:45:59.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7fd2aaf83700 1 -- 172.21.15.78:0/2359111291 <== osd.7 v2:172.21.15.133:6824/35901 19 ==== osd_op_reply(35 10000000000.00000000 [omap-set-vals] v66'39 uv39 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55ec99394fc0 con 0x55ec989fa400 2023-10-11T09:45:59.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7fd2acfc8840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/24 pointing to 0x0x1000000020d 2023-10-11T09:45:59.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:36 9.35 9:ac000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ec98a3a400 con 0x55ec989fa800 2023-10-11T09:45:59.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f4f02735700 1 -- 172.21.15.78:0/3361204897 <== osd.7 v2:172.21.15.133:6824/35901 13 ==== osd_op_reply(29 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55ab479a6fc0 con 0x55ab47036c00 2023-10-11T09:45:59.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f4f03f79840 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-10-11T09:45:59.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f4f03f79840 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-10-11T09:45:59.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f4f03f79840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/19, will insert to root 2023-10-11T09:45:59.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:30 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab47037800 con 0x55ab47036c00 2023-10-11T09:45:59.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7fd2ab784700 1 -- 172.21.15.78:0/2359111291 <== osd.1 v2:172.21.15.78:6809/35734 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 66 out=49b] v64'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ec99395440 con 0x55ec989fa800 2023-10-11T09:45:59.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f4f02735700 1 -- 172.21.15.78:0/3361204897 <== osd.7 v2:172.21.15.133:6824/35901 14 ==== osd_op_reply(30 10000000000.00000000 [omap-get-header out=274b] v0'0 uv39 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55ab479a6fc0 con 0x55ab47036c00 2023-10-11T09:45:59.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f4f03f79840 10 datascan.find_or_create_dirfrag: read_version = 39 2023-10-11T09:45:59.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7fd2a6f7b700 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:37 9.d 9:b0000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ec98a3bc00 con 0x55ec989fa400 2023-10-11T09:45:59.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f4f03f79840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-10-11T09:45:59.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.940+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:31 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab46f21800 con 0x55ab47036c00 2023-10-11T09:45:59.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7f4f02735700 1 -- 172.21.15.78:0/3361204897 <== osd.7 v2:172.21.15.133:6824/35901 15 ==== osd_op_reply(31 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv39 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55ab479a6fc0 con 0x55ab47036c00 2023-10-11T09:45:59.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7f4f03f79840 20 datascan.read_dentry: oid=10000000000.00000000 dname=19 frag=*, r=0 2023-10-11T09:45:59.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7f4f03f79840 20 datascan.read_dentry: 19_head not found in result 2023-10-11T09:45:59.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7f4f03f79840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000208 at 0x0x10000000000/19 with size=6291456 bytes 2023-10-11T09:45:59.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7fd2aaf83700 1 -- 172.21.15.78:0/2359111291 <== osd.7 v2:172.21.15.133:6824/35901 20 ==== osd_op_reply(37 [pgnls-filter start_epoch 66 out=49b] v64'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ec99394fc0 con 0x55ec989fa400 2023-10-11T09:45:59.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:32 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab46f20c00 con 0x55ab47036c00 2023-10-11T09:45:59.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7fd2a677a700 1 -- 172.21.15.78:0/2359111291 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:38 9.2d 9:b4000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ec989fb800 con 0x55ec989fb400 2023-10-11T09:45:59.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7fd2ab784700 1 -- 172.21.15.78:0/2359111291 <== osd.2 v2:172.21.15.78:6808/35732 2 ==== osd_op_reply(38 [pgnls-filter start_epoch 66 out=49b] v64'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ec99395440 con 0x55ec989fb400 2023-10-11T09:45:59.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55ec9943f800 msgr2=0x55ec9865cc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7fd2acfc8840 1 --2- 172.21.15.78:0/2359111291 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55ec9943f800 0x55ec9865cc00 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55ec989fa800 msgr2=0x55ec9865d180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7fd2acfc8840 1 --2- 172.21.15.78:0/2359111291 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55ec989fa800 0x55ec9865d180 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55ec989fb400 msgr2=0x55ec98a01b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7fd2acfc8840 1 --2- 172.21.15.78:0/2359111291 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55ec989fb400 0x55ec98a01b80 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55ec9943f400 msgr2=0x55ec98a00b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7fd2acfc8840 1 --2- 172.21.15.78:0/2359111291 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55ec9943f400 0x55ec98a00b00 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7fd2aaf83700 1 -- 172.21.15.78:0/2359111291 reap_dead start 2023-10-11T09:45:59.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55ec9943f000 msgr2=0x55ec9865c680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7fd2acfc8840 1 --2- 172.21.15.78:0/2359111291 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55ec9943f000 0x55ec9865c680 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7fd2aaf83700 1 -- 172.21.15.78:0/2359111291 reap_dead start 2023-10-11T09:45:59.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55ec989fa400 msgr2=0x55ec98a00580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7fd2acfc8840 1 --2- 172.21.15.78:0/2359111291 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55ec989fa400 0x55ec98a00580 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55ec989fa000 msgr2=0x55ec98a00000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7fd2acfc8840 1 --2- 172.21.15.78:0/2359111291 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55ec989fa000 0x55ec98a00000 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x55ec99434b70 tx=0x55ec9871ade0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec9943e800 msgr2=0x55ec9865a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7fd2acfc8840 1 --2- 172.21.15.78:0/2359111291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec9943e800 0x55ec9865a580 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x55ec98683740 tx=0x55ec9868b4a0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 shutdown_connections 2023-10-11T09:45:59.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7fd2acfc8840 1 --2- 172.21.15.78:0/2359111291 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55ec989fa400 0x55ec98a00580 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7fd2acfc8840 1 --2- 172.21.15.78:0/2359111291 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55ec9943f000 0x55ec9865c680 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7fd2acfc8840 1 --2- 172.21.15.78:0/2359111291 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55ec989fa000 0x55ec98a00000 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7fd2acfc8840 1 --2- 172.21.15.78:0/2359111291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec9943e800 0x55ec9865a580 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 >> 172.21.15.78:0/2359111291 conn(0x55ec985b4c00 msgr2=0x55ec986a1800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:59.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 shutdown_connections 2023-10-11T09:45:59.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7fd2acfc8840 1 -- 172.21.15.78:0/2359111291 wait complete. 2023-10-11T09:45:59.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7f4f02735700 1 -- 172.21.15.78:0/3361204897 <== osd.7 v2:172.21.15.133:6824/35901 16 ==== osd_op_reply(32 10000000000.00000000 [omap-set-vals] v66'40 uv40 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55ab479a6fc0 con 0x55ab47036c00 2023-10-11T09:45:59.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7f4f03f79840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/19 pointing to 0x0x10000000208 2023-10-11T09:45:59.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:33 9.17 9:e8000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab47044000 con 0x55ab47036c00 2023-10-11T09:45:59.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7fd2acfc8840 1 -- 172.21.15.78:0/3778569310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec985b5800 msgr2=0x55ec9865bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7fd2acfc8840 1 --2- 172.21.15.78:0/3778569310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec985b5800 0x55ec9865bb80 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x55ec98683380 tx=0x55ec9868b3e0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7fd2acfc8840 1 -- 172.21.15.78:0/3778569310 shutdown_connections 2023-10-11T09:45:59.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7fd2acfc8840 1 --2- 172.21.15.78:0/3778569310 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ec985b5000 0x55ec9865b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7fd2acfc8840 1 --2- 172.21.15.78:0/3778569310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec985b5400 0x55ec9865b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7fd2acfc8840 1 --2- 172.21.15.78:0/3778569310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec985b5800 0x55ec9865bb80 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7fd2acfc8840 1 -- 172.21.15.78:0/3778569310 >> 172.21.15.78:0/3778569310 conn(0x55ec985b5c00 msgr2=0x55ec986a1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:59.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7fd2acfc8840 1 -- 172.21.15.78:0/3778569310 shutdown_connections 2023-10-11T09:45:59.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7fd2acfc8840 1 -- 172.21.15.78:0/3778569310 wait complete. 2023-10-11T09:45:59.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7f4f02735700 1 -- 172.21.15.78:0/3361204897 <== osd.7 v2:172.21.15.133:6824/35901 17 ==== osd_op_reply(33 [pgnls-filter start_epoch 66 out=49b] v64'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ab479a6fc0 con 0x55ab47036c00 2023-10-11T09:45:59.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7f4efd72b700 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:34 9.37 9:ec000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab4799b400 con 0x55ab4799b800 2023-10-11T09:45:59.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7f4f01f34700 1 -- 172.21.15.78:0/3361204897 <== osd.4 v2:172.21.15.133:6808/35898 2 ==== osd_op_reply(34 [pgnls-filter start_epoch 66 out=49b] v64'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ab479a6000 con 0x55ab4799b800 2023-10-11T09:45:59.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.944+0000 7f4efdf2c700 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:35 9.f 9:f0000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab47036800 con 0x55ab46f21400 2023-10-11T09:45:59.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.948+0000 7f4f01f34700 1 -- 172.21.15.78:0/3361204897 <== osd.5 v2:172.21.15.133:6816/35899 5 ==== osd_op_reply(35 [pgnls-filter start_epoch 66 out=49b] v64'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ab479a6000 con 0x55ab46f21400 2023-10-11T09:45:59.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.948+0000 7f4efd72b700 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:36 9.2f 9:f4000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab47036400 con 0x55ab46f21000 2023-10-11T09:45:59.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.948+0000 7f4f01f34700 1 -- 172.21.15.78:0/3361204897 <== osd.3 v2:172.21.15.78:6824/35738 3 ==== osd_op_reply(36 [pgnls-filter start_epoch 66 out=49b] v64'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ab479a6000 con 0x55ab46f21000 2023-10-11T09:45:59.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.948+0000 7f4efdf2c700 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:37 9.1f 9:f8000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab47044000 con 0x55ab46f20800 2023-10-11T09:45:59.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.948+0000 7f4f02735700 1 -- 172.21.15.78:0/3361204897 <== osd.1 v2:172.21.15.78:6809/35734 2 ==== osd_op_reply(37 [pgnls-filter start_epoch 66 out=49b] v64'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ab479a6fc0 con 0x55ab46f20800 2023-10-11T09:45:59.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.948+0000 7f4efd72b700 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:38 9.3f 9:fc000000::::head [pgnls-filter start_epoch 66 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab47037800 con 0x55ab47036000 2023-10-11T09:45:59.404 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--worker_n', '0', '--worker_m', '7'] in time 0:00:00.685178, result: 2023-10-11T09:45:59.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.948+0000 7f4f02f36700 1 -- 172.21.15.78:0/3361204897 <== osd.2 v2:172.21.15.78:6808/35732 5 ==== osd_op_reply(38 [pgnls-filter start_epoch 66 out=81b] v66'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ab479a6d80 con 0x55ab47036000 2023-10-11T09:45:59.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.948+0000 7f4f03f79840 20 datascan.forall_objects: OSD matched oid 100000001fa.00000000 2023-10-11T09:45:59.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.948+0000 7f4f03f79840 10 datascan.operator(): handling object 100000001fa.0 2023-10-11T09:45:59.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.948+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:39 9.3f 9:feb1c803:::100000001fa.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab47044400 con 0x55ab47036000 2023-10-11T09:45:59.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.948+0000 7f4f02f36700 1 -- 172.21.15.78:0/3361204897 <== osd.2 v2:172.21.15.78:6808/35732 6 ==== osd_op_reply(39 100000001fa.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x55ab479a6d80 con 0x55ab47036000 2023-10-11T09:45:59.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.948+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:40 10.1f 10:feb1c803:::100000001fa.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab47044800 con 0x55ab47036c00 2023-10-11T09:45:59.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.952+0000 7f4f02735700 1 -- 172.21.15.78:0/3361204897 <== osd.7 v2:172.21.15.133:6824/35901 18 ==== osd_op_reply(40 100000001fa.00000000 [getxattr (30) out=30b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55ab479a6fc0 con 0x55ab47036c00 2023-10-11T09:45:59.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.952+0000 7f4f03f79840 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-10-11T09:45:59.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.952+0000 7f4f03f79840 10 datascan.inject_with_backtrace: inode: 0x0x100000001fa 2023-10-11T09:45:59.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.952+0000 7f4f03f79840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/5 2023-10-11T09:45:59.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.952+0000 7f4f03f79840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=5 2023-10-11T09:45:59.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.952+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:41 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab47044c00 con 0x55ab47036c00 2023-10-11T09:45:59.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.952+0000 7f4f02735700 1 -- 172.21.15.78:0/3361204897 <== osd.7 v2:172.21.15.133:6824/35901 19 ==== osd_op_reply(41 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55ab479a6fc0 con 0x55ab47036c00 2023-10-11T09:45:59.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.952+0000 7f4f03f79840 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-10-11T09:45:59.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.952+0000 7f4f03f79840 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-10-11T09:45:59.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.952+0000 7f4f03f79840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/5, will insert to root 2023-10-11T09:45:59.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.952+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:42 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab47045000 con 0x55ab47036c00 2023-10-11T09:45:59.410 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--worker_n', '4', '--worker_m', '7'] in time 0:00:00.689155, result: 2023-10-11T09:45:59.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.952+0000 7f4f02735700 1 -- 172.21.15.78:0/3361204897 <== osd.7 v2:172.21.15.133:6824/35901 20 ==== osd_op_reply(42 10000000000.00000000 [omap-get-header out=274b] v0'0 uv40 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55ab479a6fc0 con 0x55ab47036c00 2023-10-11T09:45:59.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.952+0000 7f4f03f79840 10 datascan.find_or_create_dirfrag: read_version = 40 2023-10-11T09:45:59.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.952+0000 7f4f03f79840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-10-11T09:45:59.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.952+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:43 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab47045400 con 0x55ab47036c00 2023-10-11T09:45:59.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.952+0000 7f4f02735700 1 -- 172.21.15.78:0/3361204897 <== osd.7 v2:172.21.15.133:6824/35901 21 ==== osd_op_reply(43 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv40 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55ab479a6fc0 con 0x55ab47036c00 2023-10-11T09:45:59.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.952+0000 7f4f03f79840 20 datascan.read_dentry: oid=10000000000.00000000 dname=5 frag=*, r=0 2023-10-11T09:45:59.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.952+0000 7f4f03f79840 20 datascan.read_dentry: 5_head not found in result 2023-10-11T09:45:59.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.952+0000 7f4f03f79840 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fa at 0x0x10000000000/5 with size=6291456 bytes 2023-10-11T09:45:59.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.952+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:44 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ab47045800 con 0x55ab47036c00 2023-10-11T09:45:59.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.956+0000 7f4f02735700 1 -- 172.21.15.78:0/3361204897 <== osd.7 v2:172.21.15.133:6824/35901 22 ==== osd_op_reply(44 10000000000.00000000 [omap-set-vals] v66'41 uv41 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55ab479a6fc0 con 0x55ab47036c00 2023-10-11T09:45:59.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.956+0000 7f4f03f79840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/5 pointing to 0x0x100000001fa 2023-10-11T09:45:59.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.956+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55ab4799b000 msgr2=0x55ab46c59080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.956+0000 7f4f03f79840 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55ab4799b000 0x55ab46c59080 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.956+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55ab46f20800 msgr2=0x55ab46c5a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.956+0000 7f4f03f79840 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55ab46f20800 0x55ab46c5a100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.956+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55ab47036000 msgr2=0x55ab47012c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.956+0000 7f4f03f79840 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55ab47036000 0x55ab47012c00 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.956+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55ab46f21000 msgr2=0x55ab47010580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.956+0000 7f4f03f79840 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55ab46f21000 0x55ab47010580 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.956+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55ab4799b800 msgr2=0x55ab46c5a680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.956+0000 7f4f03f79840 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55ab4799b800 0x55ab46c5a680 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.956+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55ab46f21400 msgr2=0x55ab47011600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.956+0000 7f4f03f79840 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55ab46f21400 0x55ab47011600 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.956+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55ab47037000 msgr2=0x55ab47012100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.956+0000 7f4f03f79840 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55ab47037000 0x55ab47012100 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.956+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55ab47036c00 msgr2=0x55ab47011080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.956+0000 7f4f03f79840 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55ab47036c00 0x55ab47011080 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.956+0000 7f4f02735700 1 -- 172.21.15.78:0/3361204897 reap_dead start 2023-10-11T09:45:59.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.956+0000 7f4f02735700 1 -- 172.21.15.78:0/3361204897 reap_dead start 2023-10-11T09:45:59.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.956+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55ab46f20400 msgr2=0x55ab47010000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.956+0000 7f4f03f79840 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55ab46f20400 0x55ab47010000 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x55ab46c09bc0 tx=0x55ab47990240 comp rx=0 tx=0).stop 2023-10-11T09:45:59.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.956+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab46bb3000 msgr2=0x55ab46c5b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.956+0000 7f4f03f79840 1 --2- 172.21.15.78:0/3361204897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab46bb3000 0x55ab46c5b180 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x55ab46eb9770 tx=0x55ab47991ec0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.956+0000 7f4f02735700 1 -- 172.21.15.78:0/3361204897 reap_dead start 2023-10-11T09:45:59.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.956+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 shutdown_connections 2023-10-11T09:45:59.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.956+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 >> 172.21.15.78:0/3361204897 conn(0x55ab46bb3800 msgr2=0x55ab46c6e800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:59.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.956+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 shutdown_connections 2023-10-11T09:45:59.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.956+0000 7f4f03f79840 1 -- 172.21.15.78:0/3361204897 wait complete. 2023-10-11T09:45:59.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.956+0000 7f4f03f79840 1 -- 172.21.15.78:0/2851354210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab46bb3c00 msgr2=0x55ab46c58b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.956+0000 7f4f03f79840 1 --2- 172.21.15.78:0/2851354210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab46bb3c00 0x55ab46c58b00 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x55ab46ccb260 tx=0x55ab47990060 comp rx=0 tx=0).stop 2023-10-11T09:45:59.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.956+0000 7f4f03f79840 1 -- 172.21.15.78:0/2851354210 shutdown_connections 2023-10-11T09:45:59.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.956+0000 7f4f03f79840 1 --2- 172.21.15.78:0/2851354210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab4799a800 0x55ab46c59b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.956+0000 7f4f03f79840 1 --2- 172.21.15.78:0/2851354210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab46bb3c00 0x55ab46c58b00 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.956+0000 7f4f03f79840 1 --2- 172.21.15.78:0/2851354210 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ab4799a400 0x55ab46c59600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.956+0000 7f4f03f79840 1 -- 172.21.15.78:0/2851354210 >> 172.21.15.78:0/2851354210 conn(0x55ab46bb2c00 msgr2=0x55ab46c70000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:59.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.956+0000 7f4f03f79840 1 -- 172.21.15.78:0/2851354210 shutdown_connections 2023-10-11T09:45:59.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:58.956+0000 7f4f03f79840 1 -- 172.21.15.78:0/2851354210 wait complete. 2023-10-11T09:45:59.423 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--worker_n', '6', '--worker_m', '7'] in time 0:00:00.700541, result: 2023-10-11T09:45:59.423 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2023-10-11T09:45:59.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.484+0000 7f445acc6840 1 -- start start 2023-10-11T09:45:59.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.484+0000 7f445acc6840 1 Processor -- start 2023-10-11T09:45:59.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.488+0000 7f445acc6840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5236f6400 0x55a5236a7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.488+0000 7f445acc6840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a523601c00 0x55a5236a6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.488+0000 7f445acc6840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5236f6800 0x55a5236a7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.488+0000 7f445acc6840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a523680a00 con 0x55a5236f6400 2023-10-11T09:45:59.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.488+0000 7f445acc6840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a523637e00 con 0x55a5236f6800 2023-10-11T09:45:59.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.488+0000 7f445acc6840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a523680280 con 0x55a523601c00 2023-10-11T09:45:59.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.488+0000 7f4459482700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5236f6400 0x55a5236a7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.488+0000 7f4459482700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5236f6400 0x55a5236a7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46538/0 (socket says 172.21.15.78:46538) 2023-10-11T09:45:59.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.488+0000 7f4459482700 1 -- 172.21.15.78:0/1847392375 learned_addr learned my addr 172.21.15.78:0/1847392375 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:59.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.488+0000 7f4459c83700 1 --2- 172.21.15.78:0/1847392375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5236f6800 0x55a5236a7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.488+0000 7f4458c81700 1 --2- 172.21.15.78:0/1847392375 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a523601c00 0x55a5236a6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.488+0000 7f4459482700 1 -- 172.21.15.78:0/1847392375 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a523601c00 msgr2=0x55a5236a6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:59.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.488+0000 7f4459482700 1 --2- 172.21.15.78:0/1847392375 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a523601c00 0x55a5236a6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.488+0000 7f4459482700 1 -- 172.21.15.78:0/1847392375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5236f6800 msgr2=0x55a5236a7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:59.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.488+0000 7f4459482700 1 --2- 172.21.15.78:0/1847392375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5236f6800 0x55a5236a7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.488+0000 7f4459482700 1 -- 172.21.15.78:0/1847392375 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a523622ea0 con 0x55a5236f6400 2023-10-11T09:45:59.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.488+0000 7f4459c83700 1 --2- 172.21.15.78:0/1847392375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5236f6800 0x55a5236a7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:45:59.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.488+0000 7f4459482700 1 --2- 172.21.15.78:0/1847392375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5236f6400 0x55a5236a7600 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x55a524369290 tx=0x55a5243f2ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c3ce1e5f15f4e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.488+0000 7f4456c7d700 1 -- 172.21.15.78:0/1847392375 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a523631600 con 0x55a5236f6400 2023-10-11T09:45:59.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.488+0000 7f4456c7d700 1 -- 172.21.15.78:0/1847392375 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a52364c480 con 0x55a5236f6400 2023-10-11T09:45:59.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.488+0000 7f445acc6840 1 -- 172.21.15.78:0/1847392375 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a523623520 con 0x55a5236f6400 2023-10-11T09:45:59.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.488+0000 7f445acc6840 1 -- 172.21.15.78:0/1847392375 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55a5236236c0 con 0x55a5236f6400 2023-10-11T09:45:59.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.488+0000 7f445acc6840 4 waiting for MDS map... 2023-10-11T09:45:59.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.488+0000 7f4456c7d700 1 -- 172.21.15.78:0/1847392375 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a523631080 con 0x55a5236f6400 2023-10-11T09:45:59.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.488+0000 7f4456c7d700 1 -- 172.21.15.78:0/1847392375 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x55a52363ac40 con 0x55a5236f6400 2023-10-11T09:45:59.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.488+0000 7f4456c7d700 1 -- 172.21.15.78:0/1847392375 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 43) v1 ==== 3118+0+0 (secure 0 0 0) 0x55a52448b200 con 0x55a5236f6400 2023-10-11T09:45:59.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.488+0000 7f445acc6840 4 Got MDS map 43 2023-10-11T09:45:59.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.488+0000 7f445acc6840 4 datascan.main: Using metadata pool output 2023-10-11T09:45:59.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.488+0000 7f445acc6840 4 datascan.main: connecting to RADOS... 2023-10-11T09:45:59.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.492+0000 7f445acc6840 1 Processor -- start 2023-10-11T09:45:59.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.492+0000 7f445acc6840 1 -- start start 2023-10-11T09:45:59.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.492+0000 7f445acc6840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a523601400 0x55a5236a7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.492+0000 7f445acc6840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a523601000 0x55a5236a8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.492+0000 7f445acc6840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5236f6c00 0x55a5236a8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.492+0000 7f445acc6840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5236803c0 con 0x55a523601400 2023-10-11T09:45:59.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.492+0000 7f445acc6840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a523680500 con 0x55a5236f6c00 2023-10-11T09:45:59.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.492+0000 7f445acc6840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a523680140 con 0x55a523601000 2023-10-11T09:45:59.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.492+0000 7f4459c83700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a523601400 0x55a5236a7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.492+0000 7f4459c83700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a523601400 0x55a5236a7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46552/0 (socket says 172.21.15.78:46552) 2023-10-11T09:45:59.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.492+0000 7f4459c83700 1 -- 172.21.15.78:0/3406585164 learned_addr learned my addr 172.21.15.78:0/3406585164 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:59.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.492+0000 7f4458c81700 1 --2- 172.21.15.78:0/3406585164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5236f6c00 0x55a5236a8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.492+0000 7f4459482700 1 --2- 172.21.15.78:0/3406585164 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a523601000 0x55a5236a8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.492+0000 7f4459c83700 1 -- 172.21.15.78:0/3406585164 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a523601000 msgr2=0x55a5236a8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:59.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.492+0000 7f4459c83700 1 --2- 172.21.15.78:0/3406585164 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a523601000 0x55a5236a8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.492+0000 7f4459c83700 1 -- 172.21.15.78:0/3406585164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5236f6c00 msgr2=0x55a5236a8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:59.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.492+0000 7f4459c83700 1 --2- 172.21.15.78:0/3406585164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5236f6c00 0x55a5236a8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.492+0000 7f4459c83700 1 -- 172.21.15.78:0/3406585164 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a523623860 con 0x55a523601400 2023-10-11T09:45:59.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.492+0000 7f4458c81700 1 --2- 172.21.15.78:0/3406585164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5236f6c00 0x55a5236a8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:45:59.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.492+0000 7f4459c83700 1 --2- 172.21.15.78:0/3406585164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a523601400 0x55a5236a7080 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x55a5239007e0 tx=0x55a5238fec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7dbe2db6aa953db server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.496+0000 7f4454c79700 1 -- 172.21.15.78:0/3406585164 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a523631600 con 0x55a523601400 2023-10-11T09:45:59.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.496+0000 7f4454c79700 1 -- 172.21.15.78:0/3406585164 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a52364c480 con 0x55a523601400 2023-10-11T09:45:59.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.496+0000 7f4454c79700 1 -- 172.21.15.78:0/3406585164 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a523631080 con 0x55a523601400 2023-10-11T09:45:59.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.496+0000 7f445acc6840 1 -- 172.21.15.78:0/3406585164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a523601400 msgr2=0x55a5236a7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.496+0000 7f445acc6840 1 --2- 172.21.15.78:0/3406585164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a523601400 0x55a5236a7080 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x55a5239007e0 tx=0x55a5238fec00 comp rx=0 tx=0).stop 2023-10-11T09:45:59.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.496+0000 7f445acc6840 1 -- 172.21.15.78:0/3406585164 shutdown_connections 2023-10-11T09:45:59.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.496+0000 7f445acc6840 1 --2- 172.21.15.78:0/3406585164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5236f6c00 0x55a5236a8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.496+0000 7f445acc6840 1 --2- 172.21.15.78:0/3406585164 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a523601000 0x55a5236a8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.496+0000 7f445acc6840 1 --2- 172.21.15.78:0/3406585164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a523601400 0x55a5236a7080 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.496+0000 7f445acc6840 1 -- 172.21.15.78:0/3406585164 >> 172.21.15.78:0/3406585164 conn(0x55a523601800 msgr2=0x55a5236cf000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:59.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.496+0000 7f445acc6840 1 -- 172.21.15.78:0/3406585164 shutdown_connections 2023-10-11T09:45:59.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.496+0000 7f445acc6840 1 -- 172.21.15.78:0/3406585164 wait complete. 2023-10-11T09:45:59.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.496+0000 7f445acc6840 1 Processor -- start 2023-10-11T09:45:59.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.496+0000 7f445acc6840 1 -- start start 2023-10-11T09:45:59.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.496+0000 7f445acc6840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a523601400 0x55a5236a9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.496+0000 7f445acc6840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a523601000 0x55a5236a9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.496+0000 7f445acc6840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5236f7c00 0x55a5236a8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.496+0000 7f445acc6840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a523680140 con 0x55a523601000 2023-10-11T09:45:59.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.496+0000 7f445acc6840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a523680500 con 0x55a5236f7c00 2023-10-11T09:45:59.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.496+0000 7f445acc6840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a5236803c0 con 0x55a523601400 2023-10-11T09:45:59.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.496+0000 7f4459482700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a523601000 0x55a5236a9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.496+0000 7f4459482700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a523601000 0x55a5236a9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46568/0 (socket says 172.21.15.78:46568) 2023-10-11T09:45:59.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.496+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 learned_addr learned my addr 172.21.15.78:0/203925639 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:59.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.496+0000 7f4459c83700 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a523601400 0x55a5236a9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.496+0000 7f4458c81700 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5236f7c00 0x55a5236a8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.496+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a523601400 msgr2=0x55a5236a9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.496+0000 7f4459482700 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a523601400 0x55a5236a9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.500+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5236f7c00 msgr2=0x55a5236a8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.500+0000 7f4459482700 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5236f7c00 0x55a5236a8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.500+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a523622ea0 con 0x55a523601000 2023-10-11T09:45:59.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.500+0000 7f4458c81700 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5236f7c00 0x55a5236a8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:45:59.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.500+0000 7f4459482700 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a523601000 0x55a5236a9180 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x55a523900750 tx=0x55a5243f2d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.500+0000 7f4453476700 1 -- 172.21.15.78:0/203925639 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a523630f20 con 0x55a523601000 2023-10-11T09:45:59.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.500+0000 7f4453476700 1 -- 172.21.15.78:0/203925639 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a524488a80 con 0x55a523601000 2023-10-11T09:45:59.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.500+0000 7f4453476700 1 -- 172.21.15.78:0/203925639 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a523631600 con 0x55a523601000 2023-10-11T09:45:59.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.500+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a523623520 con 0x55a523601000 2023-10-11T09:45:59.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.500+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a523623a00 con 0x55a523601000 2023-10-11T09:45:59.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.500+0000 7f445acc6840 4 datascan.init: resolving metadata pool 8 2023-10-11T09:45:59.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.500+0000 7f4453476700 1 -- 172.21.15.78:0/203925639 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x55a52365eb00 con 0x55a523601000 2023-10-11T09:45:59.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.504+0000 7f4453476700 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55a523966800 0x55a523a4e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.504+0000 7f4453476700 1 -- 172.21.15.78:0/203925639 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x55a52363aa80 con 0x55a523601000 2023-10-11T09:45:59.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.504+0000 7f445acc6840 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-10-11T09:45:59.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.504+0000 7f445acc6840 4 datascan.main: resolving metadata pool 8 2023-10-11T09:45:59.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.504+0000 7f4459c83700 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55a523966800 0x55a523a4e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.504+0000 7f445acc6840 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55a5236f7800 0x55a5236a8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.504+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:1 8.0 8:00000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a5236f7400 con 0x55a5236f7800 2023-10-11T09:45:59.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.504+0000 7f4458c81700 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55a5236f7800 0x55a5236a8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.504+0000 7f4459c83700 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55a523966800 0x55a523a4e000 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x55a5239007e0 tx=0x55a5243f2de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.504+0000 7f4458c81700 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55a5236f7800 0x55a5236a8680 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.504+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(1 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f0fc0 con 0x55a5236f7800 2023-10-11T09:45:59.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.504+0000 7f4458c81700 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55a5236f7400 0x55a523a4e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.504+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:2 8.10 8:08000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523966c00 con 0x55a5236f7400 2023-10-11T09:45:59.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.504+0000 7f4459c83700 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55a5236f7400 0x55a523a4e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.508+0000 7f4459c83700 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55a5236f7400 0x55a523a4e580 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.508+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 1 ==== osd_op_reply(2 [pgnls start_epoch 66 out=81b] v66'41 uv41 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.508+0000 7f4459c83700 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55a523966c00 0x55a5236a6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.508+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:3 8.8 8:10000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523967000 con 0x55a523966c00 2023-10-11T09:45:59.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.508+0000 7f4459482700 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55a523966c00 0x55a5236a6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.508+0000 7f4459482700 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55a523966c00 0x55a5236a6580 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.512+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 <== osd.5 v2:172.21.15.133:6816/35899 1 ==== osd_op_reply(3 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f1440 con 0x55a523966c00 2023-10-11T09:45:59.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.512+0000 7f4459482700 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55a523967000 0x55a523a4eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.512+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:4 8.18 8:18000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523966400 con 0x55a523967000 2023-10-11T09:45:59.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.512+0000 7f4458c81700 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55a523967000 0x55a523a4eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.512+0000 7f4458c81700 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55a523967000 0x55a523a4eb00 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.512+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 <== osd.4 v2:172.21.15.133:6808/35898 1 ==== osd_op_reply(4 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f0fc0 con 0x55a523967000 2023-10-11T09:45:59.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.512+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:5 8.4 8:20000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523966400 con 0x55a523966c00 2023-10-11T09:45:59.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.516+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 <== osd.5 v2:172.21.15.133:6816/35899 2 ==== osd_op_reply(5 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f1440 con 0x55a523966c00 2023-10-11T09:45:59.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.516+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:6 8.14 8:28000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523966400 con 0x55a523967000 2023-10-11T09:45:59.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.516+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 <== osd.4 v2:172.21.15.133:6808/35898 2 ==== osd_op_reply(6 [pgnls start_epoch 66 out=125b] v66'97 uv97 ondisk = 1) v8 ==== 144+0+125 (crc 0 0 0) 0x55a5243f0fc0 con 0x55a523967000 2023-10-11T09:45:59.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.516+0000 7f4458c81700 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55a523966400 0x55a523a4f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.516+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:7 8.c 8:30000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523967c00 con 0x55a523966400 2023-10-11T09:45:59.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.516+0000 7f4459c83700 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55a523966400 0x55a523a4f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.516+0000 7f4459c83700 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55a523966400 0x55a523a4f080 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.516+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.0 v2:172.21.15.78:6800/35735 1 ==== osd_op_reply(7 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f1200 con 0x55a523966400 2023-10-11T09:45:59.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.516+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:8 8.1c 8:38000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523967c00 con 0x55a523966c00 2023-10-11T09:45:59.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.520+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 <== osd.5 v2:172.21.15.133:6816/35899 3 ==== osd_op_reply(8 [pgnls start_epoch 66 out=49b] v65'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f1440 con 0x55a523966c00 2023-10-11T09:45:59.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.520+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:9 8.2 8:40000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523967c00 con 0x55a523967000 2023-10-11T09:45:59.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.520+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 <== osd.4 v2:172.21.15.133:6808/35898 3 ==== osd_op_reply(9 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f0fc0 con 0x55a523967000 2023-10-11T09:45:59.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.520+0000 7f4458c81700 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55a523967c00 0x55a523a4f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.520+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:10 8.12 8:48000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a70000 con 0x55a523967c00 2023-10-11T09:45:59.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.520+0000 7f4459482700 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55a523967c00 0x55a523a4f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.520+0000 7f4459482700 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55a523967c00 0x55a523a4f600 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.524+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== osd_op_reply(10 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f1440 con 0x55a523967c00 2023-10-11T09:45:59.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.524+0000 7f4459482700 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55a523a70000 0x55a523a4fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.524+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:11 8.a 8:50000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a70c00 con 0x55a523a70000 2023-10-11T09:45:59.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.524+0000 7f4458c81700 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55a523a70000 0x55a523a4fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.524+0000 7f4458c81700 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55a523a70000 0x55a523a4fb80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.524+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 <== osd.1 v2:172.21.15.78:6809/35734 1 ==== osd_op_reply(11 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f0fc0 con 0x55a523a70000 2023-10-11T09:45:59.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.524+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:12 8.1a 8:58000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a70c00 con 0x55a523966400 2023-10-11T09:45:59.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.524+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.0 v2:172.21.15.78:6800/35735 2 ==== osd_op_reply(12 [pgnls start_epoch 66 out=49b] v66'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f1200 con 0x55a523966400 2023-10-11T09:45:59.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.524+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:13 8.6 8:60000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a70c00 con 0x55a523966400 2023-10-11T09:45:59.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.524+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.0 v2:172.21.15.78:6800/35735 3 ==== osd_op_reply(13 [pgnls start_epoch 66 out=49b] v65'22 uv22 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f1200 con 0x55a523966400 2023-10-11T09:45:59.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.524+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:14 8.16 8:68000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a70c00 con 0x55a5236f7800 2023-10-11T09:45:59.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.528+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 <== osd.3 v2:172.21.15.78:6824/35738 2 ==== osd_op_reply(14 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f0fc0 con 0x55a5236f7800 2023-10-11T09:45:59.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.528+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:15 8.e 8:70000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a70c00 con 0x55a523966400 2023-10-11T09:45:59.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.528+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.0 v2:172.21.15.78:6800/35735 4 ==== osd_op_reply(15 [pgnls start_epoch 66 out=49b] v65'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f1200 con 0x55a523966400 2023-10-11T09:45:59.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.528+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:16 8.1e 8:78000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a70c00 con 0x55a5236f7800 2023-10-11T09:45:59.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.528+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 <== osd.3 v2:172.21.15.78:6824/35738 3 ==== osd_op_reply(16 [pgnls start_epoch 66 out=49b] v66'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f0fc0 con 0x55a5236f7800 2023-10-11T09:45:59.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.528+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:17 8.1 8:80000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a70c00 con 0x55a523a70000 2023-10-11T09:45:59.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.528+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 <== osd.1 v2:172.21.15.78:6809/35734 2 ==== osd_op_reply(17 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f0fc0 con 0x55a523a70000 2023-10-11T09:45:59.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.528+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:18 8.11 8:88000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a70c00 con 0x55a523966c00 2023-10-11T09:45:59.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.528+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 <== osd.5 v2:172.21.15.133:6816/35899 4 ==== osd_op_reply(18 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f1440 con 0x55a523966c00 2023-10-11T09:45:59.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.528+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:19 8.9 8:90000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a70c00 con 0x55a523967c00 2023-10-11T09:45:59.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.532+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 <== osd.6 v2:172.21.15.133:6800/35900 2 ==== osd_op_reply(19 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f1440 con 0x55a523967c00 2023-10-11T09:45:59.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.532+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:20 8.19 8:98000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a70c00 con 0x55a523966c00 2023-10-11T09:45:59.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.532+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 <== osd.5 v2:172.21.15.133:6816/35899 5 ==== osd_op_reply(20 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f1440 con 0x55a523966c00 2023-10-11T09:45:59.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.532+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:21 8.5 8:a0000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a70c00 con 0x55a523967c00 2023-10-11T09:45:59.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.532+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 <== osd.6 v2:172.21.15.133:6800/35900 3 ==== osd_op_reply(21 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f1440 con 0x55a523967c00 2023-10-11T09:45:59.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.532+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:22 8.15 8:a8000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a70c00 con 0x55a5236f7400 2023-10-11T09:45:59.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.532+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 2 ==== osd_op_reply(22 [pgnls start_epoch 66 out=73b] v65'6 uv6 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.532+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:23 8.d 8:b0000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a70c00 con 0x55a523966c00 2023-10-11T09:45:59.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.536+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 <== osd.5 v2:172.21.15.133:6816/35899 6 ==== osd_op_reply(23 [pgnls start_epoch 66 out=74b] v66'3 uv3 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55a5243f1440 con 0x55a523966c00 2023-10-11T09:45:59.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.536+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:24 8.1d 8:b8000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a70c00 con 0x55a523966400 2023-10-11T09:45:59.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.536+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.0 v2:172.21.15.78:6800/35735 5 ==== osd_op_reply(24 [pgnls start_epoch 66 out=49b] v65'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f1200 con 0x55a523966400 2023-10-11T09:45:59.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.536+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:25 8.3 8:c0000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a70c00 con 0x55a523a70000 2023-10-11T09:45:59.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.536+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 <== osd.1 v2:172.21.15.78:6809/35734 3 ==== osd_op_reply(25 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f0fc0 con 0x55a523a70000 2023-10-11T09:45:59.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.536+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:26 8.13 8:c8000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a70c00 con 0x55a5236f7800 2023-10-11T09:45:59.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.536+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 <== osd.3 v2:172.21.15.78:6824/35738 4 ==== osd_op_reply(26 [pgnls start_epoch 66 out=98b] v66'7 uv7 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55a5243f0fc0 con 0x55a5236f7800 2023-10-11T09:45:59.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.536+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:27 8.b 8:d0000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a70c00 con 0x55a5236f7800 2023-10-11T09:45:59.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.540+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 <== osd.3 v2:172.21.15.78:6824/35738 5 ==== osd_op_reply(27 [pgnls start_epoch 66 out=76b] v66'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55a5243f0fc0 con 0x55a5236f7800 2023-10-11T09:45:59.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.540+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:28 8.1b 8:d8000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a70c00 con 0x55a523967000 2023-10-11T09:45:59.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.540+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 <== osd.4 v2:172.21.15.133:6808/35898 4 ==== osd_op_reply(28 [pgnls start_epoch 66 out=49b] v66'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f0fc0 con 0x55a523967000 2023-10-11T09:45:59.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.540+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:29 8.7 8:e0000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a70c00 con 0x55a523a70000 2023-10-11T09:45:59.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.540+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 <== osd.1 v2:172.21.15.78:6809/35734 4 ==== osd_op_reply(29 [pgnls start_epoch 66 out=49b] v65'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f0fc0 con 0x55a523a70000 2023-10-11T09:45:59.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.540+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:30 8.17 8:e8000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a70c00 con 0x55a5236f7400 2023-10-11T09:45:59.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.540+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 3 ==== osd_op_reply(30 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.540+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:31 8.f 8:f0000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a70c00 con 0x55a5236f7800 2023-10-11T09:45:59.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 <== osd.3 v2:172.21.15.78:6824/35738 6 ==== osd_op_reply(31 [pgnls start_epoch 66 out=73b] v66'4 uv4 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55a5243f0fc0 con 0x55a5236f7800 2023-10-11T09:45:59.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:32 8.1f 8:f8000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a70c00 con 0x55a523966400 2023-10-11T09:45:59.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.0 v2:172.21.15.78:6800/35735 6 ==== osd_op_reply(32 [pgnls start_epoch 66 out=101b] v66'12 uv12 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55a5243f1200 con 0x55a523966400 2023-10-11T09:45:59.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2023-10-11T09:45:59.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:33 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a5236f7000 con 0x55a5236f7400 2023-10-11T09:45:59.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 4 ==== osd_op_reply(33 10000000000.00000000 [omap-get-vals out=12470b] v0'0 uv41 ondisk = 0) v8 ==== 164+0+12470 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2023-10-11T09:45:59.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-10-11T09:45:59.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 10 datascan.scan_links: step 1: handling object 200.00000000 2023-10-11T09:45:59.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-10-11T09:45:59.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 10 datascan.scan_links: step 1: handling object 200.00000001 2023-10-11T09:45:59.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-10-11T09:45:59.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 10 datascan.scan_links: step 1: handling object 500.00000000 2023-10-11T09:45:59.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-10-11T09:45:59.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 10 datascan.scan_links: step 1: handling object mds_snaptable 2023-10-11T09:45:59.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-10-11T09:45:59.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 10 datascan.scan_links: step 1: handling object mds0_inotable 2023-10-11T09:45:59.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-10-11T09:45:59.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 10 datascan.scan_links: step 1: handling object 100.00000000 2023-10-11T09:45:59.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:34 8.13 8:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a5236f6c00 con 0x55a5236f7800 2023-10-11T09:45:59.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 <== osd.3 v2:172.21.15.78:6824/35738 7 ==== osd_op_reply(34 100.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55a5243f0fc0 con 0x55a5236f7800 2023-10-11T09:45:59.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2023-10-11T09:45:59.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-10-11T09:45:59.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 10 datascan.scan_links: step 1: handling object 400.00000000 2023-10-11T09:45:59.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-10-11T09:45:59.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2023-10-11T09:45:59.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-10-11T09:45:59.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 10 datascan.scan_links: step 1: handling object 1.00000000 2023-10-11T09:45:59.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.544+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:35 8.1f 8:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71800 con 0x55a523966400 2023-10-11T09:45:59.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.548+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.0 v2:172.21.15.78:6800/35735 7 ==== osd_op_reply(35 1.00000000 [omap-get-vals out=508b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+508 (crc 0 0 0) 0x55a5243f1200 con 0x55a523966400 2023-10-11T09:45:59.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.548+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.548+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:36 8.0 8:00000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71c00 con 0x55a5236f7800 2023-10-11T09:45:59.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.548+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 <== osd.3 v2:172.21.15.78:6824/35738 8 ==== osd_op_reply(36 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f0fc0 con 0x55a5236f7800 2023-10-11T09:45:59.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.548+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:37 8.10 8:08000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71c00 con 0x55a5236f7400 2023-10-11T09:45:59.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.548+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 5 ==== osd_op_reply(37 [pgnls start_epoch 66 out=81b] v66'41 uv41 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.548+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:38 8.8 8:10000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71c00 con 0x55a523966c00 2023-10-11T09:45:59.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.552+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 <== osd.5 v2:172.21.15.133:6816/35899 7 ==== osd_op_reply(38 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f1440 con 0x55a523966c00 2023-10-11T09:45:59.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.552+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:39 8.18 8:18000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71c00 con 0x55a523967000 2023-10-11T09:45:59.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.552+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 <== osd.4 v2:172.21.15.133:6808/35898 5 ==== osd_op_reply(39 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f0fc0 con 0x55a523967000 2023-10-11T09:45:59.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.552+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:40 8.4 8:20000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71c00 con 0x55a523966c00 2023-10-11T09:45:59.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.552+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 <== osd.5 v2:172.21.15.133:6816/35899 8 ==== osd_op_reply(40 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f1440 con 0x55a523966c00 2023-10-11T09:45:59.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.552+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:41 8.14 8:28000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71c00 con 0x55a523967000 2023-10-11T09:45:59.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.552+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 <== osd.4 v2:172.21.15.133:6808/35898 6 ==== osd_op_reply(41 [pgnls start_epoch 66 out=125b] v66'97 uv97 ondisk = 1) v8 ==== 144+0+125 (crc 0 0 0) 0x55a5243f0fc0 con 0x55a523967000 2023-10-11T09:45:59.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.552+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:42 8.c 8:30000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71c00 con 0x55a523966400 2023-10-11T09:45:59.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.556+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.0 v2:172.21.15.78:6800/35735 8 ==== osd_op_reply(42 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f1200 con 0x55a523966400 2023-10-11T09:45:59.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.556+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:43 8.1c 8:38000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71c00 con 0x55a523966c00 2023-10-11T09:45:59.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.556+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 <== osd.5 v2:172.21.15.133:6816/35899 9 ==== osd_op_reply(43 [pgnls start_epoch 66 out=49b] v65'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f1440 con 0x55a523966c00 2023-10-11T09:45:59.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.556+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:44 8.2 8:40000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71c00 con 0x55a523967000 2023-10-11T09:45:59.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.556+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 <== osd.4 v2:172.21.15.133:6808/35898 7 ==== osd_op_reply(44 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f0fc0 con 0x55a523967000 2023-10-11T09:45:59.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.556+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:45 8.12 8:48000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71c00 con 0x55a523967c00 2023-10-11T09:45:59.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.560+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 <== osd.6 v2:172.21.15.133:6800/35900 4 ==== osd_op_reply(45 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f1440 con 0x55a523967c00 2023-10-11T09:45:59.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.560+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:46 8.a 8:50000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71c00 con 0x55a523a70000 2023-10-11T09:45:59.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.560+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 <== osd.1 v2:172.21.15.78:6809/35734 5 ==== osd_op_reply(46 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f0fc0 con 0x55a523a70000 2023-10-11T09:45:59.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.560+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:47 8.1a 8:58000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71c00 con 0x55a523966400 2023-10-11T09:45:59.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.560+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.0 v2:172.21.15.78:6800/35735 9 ==== osd_op_reply(47 [pgnls start_epoch 66 out=49b] v66'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f1200 con 0x55a523966400 2023-10-11T09:45:59.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.560+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:48 8.6 8:60000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71c00 con 0x55a523966400 2023-10-11T09:45:59.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.560+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.0 v2:172.21.15.78:6800/35735 10 ==== osd_op_reply(48 [pgnls start_epoch 66 out=49b] v65'22 uv22 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f1200 con 0x55a523966400 2023-10-11T09:45:59.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.560+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:49 8.16 8:68000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71c00 con 0x55a5236f7800 2023-10-11T09:45:59.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.560+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 <== osd.3 v2:172.21.15.78:6824/35738 9 ==== osd_op_reply(49 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f0fc0 con 0x55a5236f7800 2023-10-11T09:45:59.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.560+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:50 8.e 8:70000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71c00 con 0x55a523966400 2023-10-11T09:45:59.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.564+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.0 v2:172.21.15.78:6800/35735 11 ==== osd_op_reply(50 [pgnls start_epoch 66 out=49b] v65'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f1200 con 0x55a523966400 2023-10-11T09:45:59.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.564+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:51 8.1e 8:78000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71c00 con 0x55a5236f7800 2023-10-11T09:45:59.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.564+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 <== osd.3 v2:172.21.15.78:6824/35738 10 ==== osd_op_reply(51 [pgnls start_epoch 66 out=49b] v66'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f0fc0 con 0x55a5236f7800 2023-10-11T09:45:59.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.564+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:52 8.1 8:80000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71c00 con 0x55a523a70000 2023-10-11T09:45:59.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.564+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 <== osd.1 v2:172.21.15.78:6809/35734 6 ==== osd_op_reply(52 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f0fc0 con 0x55a523a70000 2023-10-11T09:45:59.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.564+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:53 8.11 8:88000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71c00 con 0x55a523966c00 2023-10-11T09:45:59.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.564+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 <== osd.5 v2:172.21.15.133:6816/35899 10 ==== osd_op_reply(53 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f1440 con 0x55a523966c00 2023-10-11T09:45:59.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.564+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:54 8.9 8:90000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71c00 con 0x55a523967c00 2023-10-11T09:45:59.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.568+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 <== osd.6 v2:172.21.15.133:6800/35900 5 ==== osd_op_reply(54 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f1440 con 0x55a523967c00 2023-10-11T09:45:59.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.568+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:55 8.19 8:98000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71c00 con 0x55a523966c00 2023-10-11T09:45:59.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.568+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 <== osd.5 v2:172.21.15.133:6816/35899 11 ==== osd_op_reply(55 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f1440 con 0x55a523966c00 2023-10-11T09:45:59.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.568+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:56 8.5 8:a0000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71c00 con 0x55a523967c00 2023-10-11T09:45:59.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.568+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 <== osd.6 v2:172.21.15.133:6800/35900 6 ==== osd_op_reply(56 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f1440 con 0x55a523967c00 2023-10-11T09:45:59.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.568+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:57 8.15 8:a8000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71c00 con 0x55a5236f7400 2023-10-11T09:45:59.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.568+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 6 ==== osd_op_reply(57 [pgnls start_epoch 66 out=73b] v65'6 uv6 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.568+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:58 8.d 8:b0000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71c00 con 0x55a523966c00 2023-10-11T09:45:59.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.572+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 <== osd.5 v2:172.21.15.133:6816/35899 12 ==== osd_op_reply(58 [pgnls start_epoch 66 out=74b] v66'3 uv3 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55a5243f1440 con 0x55a523966c00 2023-10-11T09:45:59.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.572+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:59 8.1d 8:b8000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71c00 con 0x55a523966400 2023-10-11T09:45:59.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.572+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.0 v2:172.21.15.78:6800/35735 12 ==== osd_op_reply(59 [pgnls start_epoch 66 out=49b] v65'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f1200 con 0x55a523966400 2023-10-11T09:45:59.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.572+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:60 8.3 8:c0000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71c00 con 0x55a523a70000 2023-10-11T09:45:59.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.572+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 <== osd.1 v2:172.21.15.78:6809/35734 7 ==== osd_op_reply(60 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f0fc0 con 0x55a523a70000 2023-10-11T09:45:59.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.572+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:61 8.13 8:c8000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71c00 con 0x55a5236f7800 2023-10-11T09:45:59.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.572+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 <== osd.3 v2:172.21.15.78:6824/35738 11 ==== osd_op_reply(61 [pgnls start_epoch 66 out=98b] v66'7 uv7 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55a5243f0fc0 con 0x55a5236f7800 2023-10-11T09:45:59.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.572+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:62 8.b 8:d0000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71c00 con 0x55a5236f7800 2023-10-11T09:45:59.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.576+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 <== osd.3 v2:172.21.15.78:6824/35738 12 ==== osd_op_reply(62 [pgnls start_epoch 66 out=76b] v66'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55a5243f0fc0 con 0x55a5236f7800 2023-10-11T09:45:59.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.576+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:63 8.1b 8:d8000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71c00 con 0x55a523967000 2023-10-11T09:45:59.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.576+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 <== osd.4 v2:172.21.15.133:6808/35898 8 ==== osd_op_reply(63 [pgnls start_epoch 66 out=49b] v66'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f0fc0 con 0x55a523967000 2023-10-11T09:45:59.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.576+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:64 8.7 8:e0000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71c00 con 0x55a523a70000 2023-10-11T09:45:59.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.576+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 <== osd.1 v2:172.21.15.78:6809/35734 8 ==== osd_op_reply(64 [pgnls start_epoch 66 out=49b] v65'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f0fc0 con 0x55a523a70000 2023-10-11T09:45:59.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.576+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:65 8.17 8:e8000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71c00 con 0x55a5236f7400 2023-10-11T09:45:59.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.576+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 7 ==== osd_op_reply(65 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.576+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:66 8.f 8:f0000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71c00 con 0x55a5236f7800 2023-10-11T09:45:59.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 <== osd.3 v2:172.21.15.78:6824/35738 13 ==== osd_op_reply(66 [pgnls start_epoch 66 out=73b] v66'4 uv4 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55a5243f0fc0 con 0x55a5236f7800 2023-10-11T09:45:59.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:67 8.1f 8:f8000000::::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71c00 con 0x55a523966400 2023-10-11T09:45:59.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.0 v2:172.21.15.78:6800/35735 13 ==== osd_op_reply(67 [pgnls start_epoch 66 out=101b] v66'12 uv12 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55a5243f1200 con 0x55a523966400 2023-10-11T09:45:59.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2023-10-11T09:45:59.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:68 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a70c00 con 0x55a5236f7400 2023-10-11T09:45:59.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 8 ==== osd_op_reply(68 10000000000.00000000 [omap-get-vals out=12470b] v0'0 uv41 ondisk = 0) v8 ==== 164+0+12470 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2023-10-11T09:45:59.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: adding 0x100000001ff for future processing to fix dnfirst 2023-10-11T09:45:59.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: adding 0x10000000200 for future processing to fix dnfirst 2023-10-11T09:45:59.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: adding 0x10000000201 for future processing to fix dnfirst 2023-10-11T09:45:59.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: adding 0x10000000202 for future processing to fix dnfirst 2023-10-11T09:45:59.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: adding 0x10000000203 for future processing to fix dnfirst 2023-10-11T09:45:59.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: adding 0x10000000204 for future processing to fix dnfirst 2023-10-11T09:45:59.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: adding 0x10000000205 for future processing to fix dnfirst 2023-10-11T09:45:59.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: adding 0x10000000206 for future processing to fix dnfirst 2023-10-11T09:45:59.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: adding 0x10000000207 for future processing to fix dnfirst 2023-10-11T09:45:59.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: adding 0x10000000208 for future processing to fix dnfirst 2023-10-11T09:45:59.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: adding 0x100000001f6 for future processing to fix dnfirst 2023-10-11T09:45:59.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: adding 0x10000000209 for future processing to fix dnfirst 2023-10-11T09:45:59.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: adding 0x1000000020a for future processing to fix dnfirst 2023-10-11T09:45:59.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: adding 0x1000000020b for future processing to fix dnfirst 2023-10-11T09:45:59.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: adding 0x1000000020c for future processing to fix dnfirst 2023-10-11T09:45:59.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: adding 0x1000000020d for future processing to fix dnfirst 2023-10-11T09:45:59.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2023-10-11T09:45:59.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: adding 0x100000001f8 for future processing to fix dnfirst 2023-10-11T09:45:59.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: adding 0x100000001f9 for future processing to fix dnfirst 2023-10-11T09:45:59.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: adding 0x100000001fa for future processing to fix dnfirst 2023-10-11T09:45:59.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: adding 0x100000001fb for future processing to fix dnfirst 2023-10-11T09:45:59.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: adding 0x100000001fc for future processing to fix dnfirst 2023-10-11T09:45:59.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: adding 0x100000001fd for future processing to fix dnfirst 2023-10-11T09:45:59.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 20 datascan.scan_links: adding 0x100000001fe for future processing to fix dnfirst 2023-10-11T09:45:59.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2023-10-11T09:45:59.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-10-11T09:45:59.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 10 datascan.scan_links: step 2: handling object 200.00000000 2023-10-11T09:45:59.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-10-11T09:45:59.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 10 datascan.scan_links: step 2: handling object 200.00000001 2023-10-11T09:45:59.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-10-11T09:45:59.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 10 datascan.scan_links: step 2: handling object 500.00000000 2023-10-11T09:45:59.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-10-11T09:45:59.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 10 datascan.scan_links: step 2: handling object mds_snaptable 2023-10-11T09:45:59.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-10-11T09:45:59.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 10 datascan.scan_links: step 2: handling object mds0_inotable 2023-10-11T09:45:59.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-10-11T09:45:59.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 10 datascan.scan_links: step 2: handling object 100.00000000 2023-10-11T09:45:59.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:69 8.13 8:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71800 con 0x55a5236f7800 2023-10-11T09:45:59.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 <== osd.3 v2:172.21.15.78:6824/35738 14 ==== osd_op_reply(69 100.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55a5243f0fc0 con 0x55a5236f7800 2023-10-11T09:45:59.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2023-10-11T09:45:59.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-10-11T09:45:59.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 10 datascan.scan_links: step 2: handling object 400.00000000 2023-10-11T09:45:59.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-10-11T09:45:59.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2023-10-11T09:45:59.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-10-11T09:45:59.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 10 datascan.scan_links: step 2: handling object 1.00000000 2023-10-11T09:45:59.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:70 8.1f 8:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a5236f7000 con 0x55a523966400 2023-10-11T09:45:59.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.580+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.0 v2:172.21.15.78:6800/35735 14 ==== osd_op_reply(70 1.00000000 [omap-get-vals out=508b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+508 (crc 0 0 0) 0x55a5243f1200 con 0x55a523966400 2023-10-11T09:45:59.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.584+0000 7f445acc6840 20 datascan.scan_links: injected ino detected 2023-10-11T09:45:59.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.584+0000 7f445acc6840 20 datascan.scan_links: adding 0x10000000000 for future processing to fix dnfirst 2023-10-11T09:45:59.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.584+0000 7f445acc6840 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2023-10-11T09:45:59.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.584+0000 7f445acc6840 10 datascan.scan_links: removing dup dentries from 0 objects 2023-10-11T09:45:59.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.584+0000 7f445acc6840 10 datascan.scan_links: processing 0 bad_nlink_inos 2023-10-11T09:45:59.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.584+0000 7f445acc6840 10 datascan.scan_links: processing 26 injected_inos 2023-10-11T09:45:59.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.584+0000 7f445acc6840 10 datascan.scan_links: handling injected_ino 0x10000000000 2023-10-11T09:45:59.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.584+0000 7f445acc6840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x1:subdir 2023-10-11T09:45:59.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.584+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:71 8.1f 8:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523967800 con 0x55a523966400 2023-10-11T09:45:59.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.584+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.0 v2:172.21.15.78:6800/35735 15 ==== osd_op_reply(71 1.00000000 [omap-get-vals-by-keys out=507b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+507 (crc 0 0 0) 0x55a5243f1200 con 0x55a523966400 2023-10-11T09:45:59.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.584+0000 7f445acc6840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:45:59.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.584+0000 7f445acc6840 20 datascan.scan_links: first is now 2 2023-10-11T09:45:59.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.584+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:72 8.1f 8:ff5b34d6:::1.00000000:head [omap-set-vals in=507b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523967400 con 0x55a523966400 2023-10-11T09:45:59.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.588+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.0 v2:172.21.15.78:6800/35735 16 ==== osd_op_reply(72 1.00000000 [omap-set-vals] v66'13 uv13 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x55a5243f1200 con 0x55a523966400 2023-10-11T09:45:59.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.588+0000 7f445acc6840 20 datascan.inject_linkage: Injected dentry 0x0x1/subdir pointing to 0x0x10000000000 2023-10-11T09:45:59.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.588+0000 7f445acc6840 10 datascan.scan_links: handling injected_ino 0x10000000001 2023-10-11T09:45:59.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.588+0000 7f445acc6840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:0 2023-10-11T09:45:59.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.588+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:73 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a9e000 con 0x55a5236f7400 2023-10-11T09:45:59.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.588+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 9 ==== osd_op_reply(73 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv41 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.588+0000 7f445acc6840 20 datascan.read_dentry: oid=10000000000.00000000 dname=0 frag=*, r=0 2023-10-11T09:45:59.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.588+0000 7f445acc6840 20 datascan.scan_links: first is now 2 2023-10-11T09:45:59.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.588+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:74 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a9e400 con 0x55a5236f7400 2023-10-11T09:45:59.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.592+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 10 ==== osd_op_reply(74 10000000000.00000000 [omap-set-vals] v66'42 uv42 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.592+0000 7f445acc6840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/0 pointing to 0x0x10000000001 2023-10-11T09:45:59.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.592+0000 7f445acc6840 10 datascan.scan_links: handling injected_ino 0x100000001f6 2023-10-11T09:45:59.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.592+0000 7f445acc6840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:1 2023-10-11T09:45:59.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.592+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:75 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a9e800 con 0x55a5236f7400 2023-10-11T09:45:59.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.592+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 11 ==== osd_op_reply(75 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv42 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.592+0000 7f445acc6840 20 datascan.read_dentry: oid=10000000000.00000000 dname=1 frag=*, r=0 2023-10-11T09:45:59.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.592+0000 7f445acc6840 20 datascan.scan_links: first is now 2 2023-10-11T09:45:59.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.592+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:76 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a9ec00 con 0x55a5236f7400 2023-10-11T09:45:59.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.592+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 12 ==== osd_op_reply(76 10000000000.00000000 [omap-set-vals] v66'43 uv43 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.592+0000 7f445acc6840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/1 pointing to 0x0x100000001f6 2023-10-11T09:45:59.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.592+0000 7f445acc6840 10 datascan.scan_links: handling injected_ino 0x100000001f7 2023-10-11T09:45:59.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.592+0000 7f445acc6840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:2 2023-10-11T09:45:59.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.592+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:77 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a9f000 con 0x55a5236f7400 2023-10-11T09:45:59.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.596+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 13 ==== osd_op_reply(77 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv43 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.596+0000 7f445acc6840 20 datascan.read_dentry: oid=10000000000.00000000 dname=2 frag=*, r=0 2023-10-11T09:45:59.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.596+0000 7f445acc6840 20 datascan.scan_links: first is now 2 2023-10-11T09:45:59.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.596+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:78 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a9f400 con 0x55a5236f7400 2023-10-11T09:45:59.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.596+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 14 ==== osd_op_reply(78 10000000000.00000000 [omap-set-vals] v66'44 uv44 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.596+0000 7f445acc6840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/2 pointing to 0x0x100000001f7 2023-10-11T09:45:59.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.596+0000 7f445acc6840 10 datascan.scan_links: handling injected_ino 0x100000001f8 2023-10-11T09:45:59.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.596+0000 7f445acc6840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:3 2023-10-11T09:45:59.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.596+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:79 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a9f800 con 0x55a5236f7400 2023-10-11T09:45:59.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.600+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 15 ==== osd_op_reply(79 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv44 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.600+0000 7f445acc6840 20 datascan.read_dentry: oid=10000000000.00000000 dname=3 frag=*, r=0 2023-10-11T09:45:59.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.600+0000 7f445acc6840 20 datascan.scan_links: first is now 2 2023-10-11T09:45:59.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.600+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:80 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a9fc00 con 0x55a5236f7400 2023-10-11T09:45:59.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.600+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 16 ==== osd_op_reply(80 10000000000.00000000 [omap-set-vals] v66'45 uv45 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.600+0000 7f445acc6840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/3 pointing to 0x0x100000001f8 2023-10-11T09:45:59.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.600+0000 7f445acc6840 10 datascan.scan_links: handling injected_ino 0x100000001f9 2023-10-11T09:45:59.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.600+0000 7f445acc6840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:4 2023-10-11T09:45:59.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.600+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:81 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523967800 con 0x55a5236f7400 2023-10-11T09:45:59.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.600+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 17 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv45 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.600+0000 7f445acc6840 20 datascan.read_dentry: oid=10000000000.00000000 dname=4 frag=*, r=0 2023-10-11T09:45:59.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.600+0000 7f445acc6840 20 datascan.scan_links: first is now 2 2023-10-11T09:45:59.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.600+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:82 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523967400 con 0x55a5236f7400 2023-10-11T09:45:59.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.604+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 18 ==== osd_op_reply(82 10000000000.00000000 [omap-set-vals] v66'46 uv46 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.604+0000 7f445acc6840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/4 pointing to 0x0x100000001f9 2023-10-11T09:45:59.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.604+0000 7f445acc6840 10 datascan.scan_links: handling injected_ino 0x100000001fa 2023-10-11T09:45:59.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.604+0000 7f445acc6840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:5 2023-10-11T09:45:59.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.604+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:83 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a5236f7000 con 0x55a5236f7400 2023-10-11T09:45:59.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.604+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 19 ==== osd_op_reply(83 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv46 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.604+0000 7f445acc6840 20 datascan.read_dentry: oid=10000000000.00000000 dname=5 frag=*, r=0 2023-10-11T09:45:59.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.604+0000 7f445acc6840 20 datascan.scan_links: first is now 2 2023-10-11T09:45:59.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.604+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:84 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71c00 con 0x55a5236f7400 2023-10-11T09:45:59.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.608+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 20 ==== osd_op_reply(84 10000000000.00000000 [omap-set-vals] v66'47 uv47 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.608+0000 7f445acc6840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/5 pointing to 0x0x100000001fa 2023-10-11T09:45:59.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.608+0000 7f445acc6840 10 datascan.scan_links: handling injected_ino 0x100000001fb 2023-10-11T09:45:59.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.608+0000 7f445acc6840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:6 2023-10-11T09:45:59.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.608+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:85 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a70c00 con 0x55a5236f7400 2023-10-11T09:45:59.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.608+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 21 ==== osd_op_reply(85 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv47 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.608+0000 7f445acc6840 20 datascan.read_dentry: oid=10000000000.00000000 dname=6 frag=*, r=0 2023-10-11T09:45:59.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.608+0000 7f445acc6840 20 datascan.scan_links: first is now 2 2023-10-11T09:45:59.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.608+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:86 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a9e000 con 0x55a5236f7400 2023-10-11T09:45:59.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.612+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 22 ==== osd_op_reply(86 10000000000.00000000 [omap-set-vals] v66'48 uv48 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.612+0000 7f445acc6840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/6 pointing to 0x0x100000001fb 2023-10-11T09:45:59.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.612+0000 7f445acc6840 10 datascan.scan_links: handling injected_ino 0x100000001fc 2023-10-11T09:45:59.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.612+0000 7f445acc6840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:7 2023-10-11T09:45:59.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.612+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:87 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a9e400 con 0x55a5236f7400 2023-10-11T09:45:59.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.612+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 23 ==== osd_op_reply(87 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv48 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.612+0000 7f445acc6840 20 datascan.read_dentry: oid=10000000000.00000000 dname=7 frag=*, r=0 2023-10-11T09:45:59.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.612+0000 7f445acc6840 20 datascan.scan_links: first is now 2 2023-10-11T09:45:59.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.612+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:88 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a9e800 con 0x55a5236f7400 2023-10-11T09:45:59.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.616+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 24 ==== osd_op_reply(88 10000000000.00000000 [omap-set-vals] v66'49 uv49 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.616+0000 7f445acc6840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/7 pointing to 0x0x100000001fc 2023-10-11T09:45:59.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.616+0000 7f445acc6840 10 datascan.scan_links: handling injected_ino 0x100000001fd 2023-10-11T09:45:59.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.616+0000 7f445acc6840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:8 2023-10-11T09:45:59.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.616+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:89 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a9ec00 con 0x55a5236f7400 2023-10-11T09:45:59.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.616+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 25 ==== osd_op_reply(89 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv49 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.616+0000 7f445acc6840 20 datascan.read_dentry: oid=10000000000.00000000 dname=8 frag=*, r=0 2023-10-11T09:45:59.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.616+0000 7f445acc6840 20 datascan.scan_links: first is now 2 2023-10-11T09:45:59.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.616+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:90 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a9f000 con 0x55a5236f7400 2023-10-11T09:45:59.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.620+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 26 ==== osd_op_reply(90 10000000000.00000000 [omap-set-vals] v66'50 uv50 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.620+0000 7f445acc6840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/8 pointing to 0x0x100000001fd 2023-10-11T09:45:59.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.620+0000 7f445acc6840 10 datascan.scan_links: handling injected_ino 0x100000001fe 2023-10-11T09:45:59.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.620+0000 7f445acc6840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:9 2023-10-11T09:45:59.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.620+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:91 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a9f400 con 0x55a5236f7400 2023-10-11T09:45:59.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.620+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 27 ==== osd_op_reply(91 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv50 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.620+0000 7f445acc6840 20 datascan.read_dentry: oid=10000000000.00000000 dname=9 frag=*, r=0 2023-10-11T09:45:59.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.620+0000 7f445acc6840 20 datascan.scan_links: first is now 2 2023-10-11T09:45:59.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.620+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:92 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a9f800 con 0x55a5236f7400 2023-10-11T09:45:59.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.620+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 28 ==== osd_op_reply(92 10000000000.00000000 [omap-set-vals] v66'51 uv51 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.624+0000 7f445acc6840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/9 pointing to 0x0x100000001fe 2023-10-11T09:45:59.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.624+0000 7f445acc6840 10 datascan.scan_links: handling injected_ino 0x100000001ff 2023-10-11T09:45:59.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.624+0000 7f445acc6840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:10 2023-10-11T09:45:59.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.624+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:93 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a9fc00 con 0x55a5236f7400 2023-10-11T09:45:59.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.624+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 29 ==== osd_op_reply(93 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv51 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.624+0000 7f445acc6840 20 datascan.read_dentry: oid=10000000000.00000000 dname=10 frag=*, r=0 2023-10-11T09:45:59.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.624+0000 7f445acc6840 20 datascan.scan_links: first is now 2 2023-10-11T09:45:59.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.624+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:94 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a9e000 con 0x55a5236f7400 2023-10-11T09:45:59.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.624+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 30 ==== osd_op_reply(94 10000000000.00000000 [omap-set-vals] v66'52 uv52 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.624+0000 7f445acc6840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/10 pointing to 0x0x100000001ff 2023-10-11T09:45:59.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.624+0000 7f445acc6840 10 datascan.scan_links: handling injected_ino 0x10000000200 2023-10-11T09:45:59.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.624+0000 7f445acc6840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:11 2023-10-11T09:45:59.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.624+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:95 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a9e400 con 0x55a5236f7400 2023-10-11T09:45:59.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.628+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 31 ==== osd_op_reply(95 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv52 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.628+0000 7f445acc6840 20 datascan.read_dentry: oid=10000000000.00000000 dname=11 frag=*, r=0 2023-10-11T09:45:59.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.628+0000 7f445acc6840 20 datascan.scan_links: first is now 2 2023-10-11T09:45:59.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.628+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:96 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a9e800 con 0x55a5236f7400 2023-10-11T09:45:59.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.628+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 32 ==== osd_op_reply(96 10000000000.00000000 [omap-set-vals] v66'53 uv53 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.628+0000 7f445acc6840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/11 pointing to 0x0x10000000200 2023-10-11T09:45:59.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.628+0000 7f445acc6840 10 datascan.scan_links: handling injected_ino 0x10000000201 2023-10-11T09:45:59.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.628+0000 7f445acc6840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:12 2023-10-11T09:45:59.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.628+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:97 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71c00 con 0x55a5236f7400 2023-10-11T09:45:59.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.628+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 33 ==== osd_op_reply(97 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv53 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.628+0000 7f445acc6840 20 datascan.read_dentry: oid=10000000000.00000000 dname=12 frag=*, r=0 2023-10-11T09:45:59.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.628+0000 7f445acc6840 20 datascan.scan_links: first is now 2 2023-10-11T09:45:59.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.628+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:98 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a70c00 con 0x55a5236f7400 2023-10-11T09:45:59.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.632+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 34 ==== osd_op_reply(98 10000000000.00000000 [omap-set-vals] v66'54 uv54 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.632+0000 7f445acc6840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/12 pointing to 0x0x10000000201 2023-10-11T09:45:59.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.632+0000 7f445acc6840 10 datascan.scan_links: handling injected_ino 0x10000000202 2023-10-11T09:45:59.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.632+0000 7f445acc6840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:13 2023-10-11T09:45:59.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.632+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:99 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a5236f7000 con 0x55a5236f7400 2023-10-11T09:45:59.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.632+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 35 ==== osd_op_reply(99 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv54 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.632+0000 7f445acc6840 20 datascan.read_dentry: oid=10000000000.00000000 dname=13 frag=*, r=0 2023-10-11T09:45:59.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.632+0000 7f445acc6840 20 datascan.scan_links: first is now 2 2023-10-11T09:45:59.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.632+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:100 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523967800 con 0x55a5236f7400 2023-10-11T09:45:59.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.636+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 36 ==== osd_op_reply(100 10000000000.00000000 [omap-set-vals] v66'55 uv55 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.636+0000 7f445acc6840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/13 pointing to 0x0x10000000202 2023-10-11T09:45:59.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.636+0000 7f445acc6840 10 datascan.scan_links: handling injected_ino 0x10000000203 2023-10-11T09:45:59.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.636+0000 7f445acc6840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:14 2023-10-11T09:45:59.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.636+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:101 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523967400 con 0x55a5236f7400 2023-10-11T09:45:59.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.636+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 37 ==== osd_op_reply(101 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv55 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.636+0000 7f445acc6840 20 datascan.read_dentry: oid=10000000000.00000000 dname=14 frag=*, r=0 2023-10-11T09:45:59.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.636+0000 7f445acc6840 20 datascan.scan_links: first is now 2 2023-10-11T09:45:59.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.636+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:102 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71c00 con 0x55a5236f7400 2023-10-11T09:45:59.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.640+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 38 ==== osd_op_reply(102 10000000000.00000000 [omap-set-vals] v66'56 uv56 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.640+0000 7f445acc6840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/14 pointing to 0x0x10000000203 2023-10-11T09:45:59.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.640+0000 7f445acc6840 10 datascan.scan_links: handling injected_ino 0x10000000204 2023-10-11T09:45:59.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.640+0000 7f445acc6840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:15 2023-10-11T09:45:59.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.640+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:103 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a9e000 con 0x55a5236f7400 2023-10-11T09:45:59.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.640+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 39 ==== osd_op_reply(103 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv56 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.640+0000 7f445acc6840 20 datascan.read_dentry: oid=10000000000.00000000 dname=15 frag=*, r=0 2023-10-11T09:45:59.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.640+0000 7f445acc6840 20 datascan.scan_links: first is now 2 2023-10-11T09:45:59.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.640+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:104 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a9e400 con 0x55a5236f7400 2023-10-11T09:45:59.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.644+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 40 ==== osd_op_reply(104 10000000000.00000000 [omap-set-vals] v66'57 uv57 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.644+0000 7f445acc6840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/15 pointing to 0x0x10000000204 2023-10-11T09:45:59.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.644+0000 7f445acc6840 10 datascan.scan_links: handling injected_ino 0x10000000205 2023-10-11T09:45:59.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.644+0000 7f445acc6840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:16 2023-10-11T09:45:59.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.644+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:105 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a9e800 con 0x55a5236f7400 2023-10-11T09:45:59.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.644+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 41 ==== osd_op_reply(105 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv57 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.644+0000 7f445acc6840 20 datascan.read_dentry: oid=10000000000.00000000 dname=16 frag=*, r=0 2023-10-11T09:45:59.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.644+0000 7f445acc6840 20 datascan.scan_links: first is now 2 2023-10-11T09:45:59.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.644+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:106 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a9ec00 con 0x55a5236f7400 2023-10-11T09:45:59.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.648+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 42 ==== osd_op_reply(106 10000000000.00000000 [omap-set-vals] v66'58 uv58 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.648+0000 7f445acc6840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/16 pointing to 0x0x10000000205 2023-10-11T09:45:59.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.648+0000 7f445acc6840 10 datascan.scan_links: handling injected_ino 0x10000000206 2023-10-11T09:45:59.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.648+0000 7f445acc6840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:17 2023-10-11T09:45:59.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.648+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:107 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a9f000 con 0x55a5236f7400 2023-10-11T09:45:59.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.648+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 43 ==== osd_op_reply(107 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv58 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.648+0000 7f445acc6840 20 datascan.read_dentry: oid=10000000000.00000000 dname=17 frag=*, r=0 2023-10-11T09:45:59.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.648+0000 7f445acc6840 20 datascan.scan_links: first is now 2 2023-10-11T09:45:59.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.648+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:108 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a9f400 con 0x55a5236f7400 2023-10-11T09:45:59.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.652+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 44 ==== osd_op_reply(108 10000000000.00000000 [omap-set-vals] v66'59 uv59 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.652+0000 7f445acc6840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/17 pointing to 0x0x10000000206 2023-10-11T09:45:59.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.652+0000 7f445acc6840 10 datascan.scan_links: handling injected_ino 0x10000000207 2023-10-11T09:45:59.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.652+0000 7f445acc6840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:18 2023-10-11T09:45:59.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.652+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:109 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a9f800 con 0x55a5236f7400 2023-10-11T09:45:59.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.652+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 45 ==== osd_op_reply(109 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv59 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.652+0000 7f445acc6840 20 datascan.read_dentry: oid=10000000000.00000000 dname=18 frag=*, r=0 2023-10-11T09:45:59.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.652+0000 7f445acc6840 20 datascan.scan_links: first is now 2 2023-10-11T09:45:59.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.652+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:110 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a9fc00 con 0x55a5236f7400 2023-10-11T09:45:59.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.652+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 46 ==== osd_op_reply(110 10000000000.00000000 [omap-set-vals] v66'60 uv60 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.652+0000 7f445acc6840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/18 pointing to 0x0x10000000207 2023-10-11T09:45:59.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.652+0000 7f445acc6840 10 datascan.scan_links: handling injected_ino 0x10000000208 2023-10-11T09:45:59.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.652+0000 7f445acc6840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:19 2023-10-11T09:45:59.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.652+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:111 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a9e000 con 0x55a5236f7400 2023-10-11T09:45:59.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.656+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 47 ==== osd_op_reply(111 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv60 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.656+0000 7f445acc6840 20 datascan.read_dentry: oid=10000000000.00000000 dname=19 frag=*, r=0 2023-10-11T09:45:59.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.656+0000 7f445acc6840 20 datascan.scan_links: first is now 2 2023-10-11T09:45:59.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.656+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:112 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a9e400 con 0x55a5236f7400 2023-10-11T09:45:59.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.656+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 48 ==== osd_op_reply(112 10000000000.00000000 [omap-set-vals] v66'61 uv61 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.656+0000 7f445acc6840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/19 pointing to 0x0x10000000208 2023-10-11T09:45:59.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.656+0000 7f445acc6840 10 datascan.scan_links: handling injected_ino 0x10000000209 2023-10-11T09:45:59.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.656+0000 7f445acc6840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:20 2023-10-11T09:45:59.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.656+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:113 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a9e800 con 0x55a5236f7400 2023-10-11T09:45:59.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.656+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 49 ==== osd_op_reply(113 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv61 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.656+0000 7f445acc6840 20 datascan.read_dentry: oid=10000000000.00000000 dname=20 frag=*, r=0 2023-10-11T09:45:59.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.656+0000 7f445acc6840 20 datascan.scan_links: first is now 2 2023-10-11T09:45:59.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.656+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:114 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a9ec00 con 0x55a5236f7400 2023-10-11T09:45:59.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.660+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 50 ==== osd_op_reply(114 10000000000.00000000 [omap-set-vals] v66'62 uv62 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.660+0000 7f445acc6840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/20 pointing to 0x0x10000000209 2023-10-11T09:45:59.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.660+0000 7f445acc6840 10 datascan.scan_links: handling injected_ino 0x1000000020a 2023-10-11T09:45:59.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.660+0000 7f445acc6840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:21 2023-10-11T09:45:59.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.660+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:115 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a9f000 con 0x55a5236f7400 2023-10-11T09:45:59.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.660+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 51 ==== osd_op_reply(115 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv62 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.660+0000 7f445acc6840 20 datascan.read_dentry: oid=10000000000.00000000 dname=21 frag=*, r=0 2023-10-11T09:45:59.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.660+0000 7f445acc6840 20 datascan.scan_links: first is now 2 2023-10-11T09:45:59.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.660+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:116 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a70c00 con 0x55a5236f7400 2023-10-11T09:45:59.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.664+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 52 ==== osd_op_reply(116 10000000000.00000000 [omap-set-vals] v66'63 uv63 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.664+0000 7f445acc6840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/21 pointing to 0x0x1000000020a 2023-10-11T09:45:59.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.664+0000 7f445acc6840 10 datascan.scan_links: handling injected_ino 0x1000000020b 2023-10-11T09:45:59.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.664+0000 7f445acc6840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:22 2023-10-11T09:45:59.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.664+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:117 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71c00 con 0x55a5236f7400 2023-10-11T09:45:59.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.664+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 53 ==== osd_op_reply(117 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv63 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.664+0000 7f445acc6840 20 datascan.read_dentry: oid=10000000000.00000000 dname=22 frag=*, r=0 2023-10-11T09:45:59.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.664+0000 7f445acc6840 20 datascan.scan_links: first is now 2 2023-10-11T09:45:59.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.664+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:118 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523967800 con 0x55a5236f7400 2023-10-11T09:45:59.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.668+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 54 ==== osd_op_reply(118 10000000000.00000000 [omap-set-vals] v66'64 uv64 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.668+0000 7f445acc6840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/22 pointing to 0x0x1000000020b 2023-10-11T09:45:59.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.668+0000 7f445acc6840 10 datascan.scan_links: handling injected_ino 0x1000000020c 2023-10-11T09:45:59.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.668+0000 7f445acc6840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:23 2023-10-11T09:45:59.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.668+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:119 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523967400 con 0x55a5236f7400 2023-10-11T09:45:59.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.668+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 55 ==== osd_op_reply(119 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv64 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.668+0000 7f445acc6840 20 datascan.read_dentry: oid=10000000000.00000000 dname=23 frag=*, r=0 2023-10-11T09:45:59.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.668+0000 7f445acc6840 20 datascan.scan_links: first is now 2 2023-10-11T09:45:59.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.668+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:120 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a5236f7000 con 0x55a5236f7400 2023-10-11T09:45:59.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.672+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 56 ==== osd_op_reply(120 10000000000.00000000 [omap-set-vals] v66'65 uv65 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.672+0000 7f445acc6840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/23 pointing to 0x0x1000000020c 2023-10-11T09:45:59.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.672+0000 7f445acc6840 10 datascan.scan_links: handling injected_ino 0x1000000020d 2023-10-11T09:45:59.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.672+0000 7f445acc6840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:24 2023-10-11T09:45:59.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.672+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:121 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523967800 con 0x55a5236f7400 2023-10-11T09:45:59.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.672+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 57 ==== osd_op_reply(121 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv65 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.672+0000 7f445acc6840 20 datascan.read_dentry: oid=10000000000.00000000 dname=24 frag=*, r=0 2023-10-11T09:45:59.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.672+0000 7f445acc6840 20 datascan.scan_links: first is now 2 2023-10-11T09:45:59.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.672+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:122 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a70c00 con 0x55a5236f7400 2023-10-11T09:45:59.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.676+0000 7f4459c83700 1 -- 172.21.15.78:0/203925639 <== osd.7 v2:172.21.15.133:6824/35901 58 ==== osd_op_reply(122 10000000000.00000000 [omap-set-vals] v66'66 uv66 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55a5243f1200 con 0x55a5236f7400 2023-10-11T09:45:59.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.676+0000 7f445acc6840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/24 pointing to 0x0x1000000020d 2023-10-11T09:45:59.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.676+0000 7f445acc6840 10 datascan.scan_links: updating inotable 2023-10-11T09:45:59.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.676+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:123 8.13 8:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a71c00 con 0x55a5236f7800 2023-10-11T09:45:59.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.676+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 <== osd.3 v2:172.21.15.78:6824/35738 15 ==== osd_op_reply(123 mds0_inotable [read 0~34 out=34b] v0'0 uv6 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x55a5243f0fc0 con 0x55a5236f7800 2023-10-11T09:45:59.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.676+0000 7f445acc6840 10 mds.0.inotable: skip_inos was [0x10000000000~0x10000000000] 2023-10-11T09:45:59.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.676+0000 7f445acc6840 10 mds.0.inotable: skip_inos now [0x1000000020e~0xfffffffdf2] 2023-10-11T09:45:59.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.676+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:124 8.13 8:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a70800 con 0x55a5236f7800 2023-10-11T09:45:59.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.680+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 <== osd.3 v2:172.21.15.78:6824/35738 16 ==== osd_op_reply(124 mds0_inotable [writefull 0~34] v66'8 uv8 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x55a5243f0fc0 con 0x55a5236f7800 2023-10-11T09:45:59.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.680+0000 7f445acc6840 10 datascan.scan_links: updating snaptable 2023-10-11T09:45:59.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.680+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:125 8.d 8:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a523a70400 con 0x55a523966c00 2023-10-11T09:45:59.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.680+0000 7f4459482700 1 -- 172.21.15.78:0/203925639 <== osd.5 v2:172.21.15.133:6816/35899 13 ==== osd_op_reply(125 mds_snaptable [read 0~70 out=70b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x55a5243f1440 con 0x55a523966c00 2023-10-11T09:45:59.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.680+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55a523966400 msgr2=0x55a523a4f080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.680+0000 7f445acc6840 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55a523966400 0x55a523a4f080 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.680+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55a523a70000 msgr2=0x55a523a4fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.680+0000 7f445acc6840 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55a523a70000 0x55a523a4fb80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.680+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55a5236f7800 msgr2=0x55a5236a8680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.680+0000 7f445acc6840 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55a5236f7800 0x55a5236a8680 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.680+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55a523967000 msgr2=0x55a523a4eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.680+0000 7f445acc6840 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55a523967000 0x55a523a4eb00 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.680+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55a523966c00 msgr2=0x55a5236a6580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.680+0000 7f445acc6840 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55a523966c00 0x55a5236a6580 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.680+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55a523967c00 msgr2=0x55a523a4f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.680+0000 7f445acc6840 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55a523967c00 0x55a523a4f600 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.680+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55a5236f7400 msgr2=0x55a523a4e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.680+0000 7f445acc6840 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55a5236f7400 0x55a523a4e580 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.680+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 reap_dead start 2023-10-11T09:45:59.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.680+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 reap_dead start 2023-10-11T09:45:59.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.680+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55a523966800 msgr2=0x55a523a4e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.680+0000 7f445acc6840 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55a523966800 0x55a523a4e000 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x55a5239007e0 tx=0x55a5243f2de0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.680+0000 7f4458c81700 1 -- 172.21.15.78:0/203925639 reap_dead start 2023-10-11T09:45:59.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.680+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a523601000 msgr2=0x55a5236a9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.680+0000 7f445acc6840 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a523601000 0x55a5236a9180 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x55a523900750 tx=0x55a5243f2d20 comp rx=0 tx=0).stop 2023-10-11T09:45:59.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.680+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 shutdown_connections 2023-10-11T09:45:59.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.680+0000 7f445acc6840 1 --2- 172.21.15.78:0/203925639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a523601000 0x55a5236a9180 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.680+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 >> 172.21.15.78:0/203925639 conn(0x55a523601800 msgr2=0x55a5236cf000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:59.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.680+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 shutdown_connections 2023-10-11T09:45:59.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.680+0000 7f445acc6840 1 -- 172.21.15.78:0/203925639 wait complete. 2023-10-11T09:45:59.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.680+0000 7f445acc6840 1 -- 172.21.15.78:0/1847392375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5236f6400 msgr2=0x55a5236a7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.680+0000 7f445acc6840 1 --2- 172.21.15.78:0/1847392375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5236f6400 0x55a5236a7600 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x55a524369290 tx=0x55a5243f2ea0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.680+0000 7f445acc6840 1 -- 172.21.15.78:0/1847392375 shutdown_connections 2023-10-11T09:45:59.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.680+0000 7f445acc6840 1 --2- 172.21.15.78:0/1847392375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5236f6800 0x55a5236a7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.680+0000 7f445acc6840 1 --2- 172.21.15.78:0/1847392375 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a523601c00 0x55a5236a6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.680+0000 7f445acc6840 1 --2- 172.21.15.78:0/1847392375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5236f6400 0x55a5236a7600 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.680+0000 7f445acc6840 1 -- 172.21.15.78:0/1847392375 >> 172.21.15.78:0/1847392375 conn(0x55a523600c00 msgr2=0x55a5236d0000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:59.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.684+0000 7f445acc6840 1 -- 172.21.15.78:0/1847392375 shutdown_connections 2023-10-11T09:45:59.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.684+0000 7f445acc6840 1 -- 172.21.15.78:0/1847392375 wait complete. 2023-10-11T09:45:59.694 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.270464, result: 2023-10-11T09:45:59.694 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-11T09:45:59.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.864+0000 7fb39acd1700 1 Processor -- start 2023-10-11T09:45:59.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.864+0000 7fb39acd1700 1 -- start start 2023-10-11T09:45:59.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.868+0000 7fb39acd1700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb3940cda10 0x7fb3940caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.868+0000 7fb39acd1700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fb394164d20 0x7fb39406bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.868+0000 7fb39acd1700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb394165390 con 0x7fb3940cdde0 2023-10-11T09:45:59.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.868+0000 7fb39acd1700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3941651a0 con 0x7fb394164d20 2023-10-11T09:45:59.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.868+0000 7fb39acd1700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3940ca440 con 0x7fb3940cda10 2023-10-11T09:45:59.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.868+0000 7fb3994ce700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fb3940cdde0 0x7fb394164610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:60342/0 (socket says 172.21.15.78:60342) 2023-10-11T09:45:59.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.868+0000 7fb3994ce700 1 -- 172.21.15.78:0/163194637 learned_addr learned my addr 172.21.15.78:0/163194637 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:45:59.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.868+0000 7fb39a4d0700 1 --2- 172.21.15.78:0/163194637 >> v2:172.21.15.78:3300/0 conn(0x7fb394164d20 0x7fb39406bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.868+0000 7fb399ccf700 1 --2- 172.21.15.78:0/163194637 >> v2:172.21.15.133:3300/0 conn(0x7fb3940cda10 0x7fb3940caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.868+0000 7fb398ccd700 1 -- 172.21.15.78:0/163194637 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 727154363 0 0) 0x7fb394165390 con 0x7fb3940cdde0 2023-10-11T09:45:59.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.868+0000 7fb398ccd700 1 -- 172.21.15.78:0/163194637 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3800035d0 con 0x7fb3940cdde0 2023-10-11T09:45:59.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.868+0000 7fb399ccf700 1 -- 172.21.15.78:0/163194637 >> v2:172.21.15.78:3300/0 conn(0x7fb394164d20 msgr2=0x7fb39406bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:45:59.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.868+0000 7fb399ccf700 1 --2- 172.21.15.78:0/163194637 >> v2:172.21.15.78:3300/0 conn(0x7fb394164d20 0x7fb39406bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.868+0000 7fb399ccf700 1 -- 172.21.15.78:0/163194637 >> v1:172.21.15.78:6789/0 conn(0x7fb3940cdde0 legacy=0x7fb394164610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.868+0000 7fb399ccf700 1 -- 172.21.15.78:0/163194637 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3940ca720 con 0x7fb3940cda10 2023-10-11T09:45:59.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.868+0000 7fb39a4d0700 1 --2- 172.21.15.78:0/163194637 >> v2:172.21.15.78:3300/0 conn(0x7fb394164d20 0x7fb39406bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:45:59.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.868+0000 7fb399ccf700 1 --2- 172.21.15.78:0/163194637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3940cda10 0x7fb3940caf20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb39406afb0 tx=0x7fb39000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8419d1d1660afb2 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.868+0000 7fb398ccd700 1 -- 172.21.15.78:0/163194637 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39000e070 con 0x7fb3940cda10 2023-10-11T09:45:59.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.872+0000 7fb398ccd700 1 -- 172.21.15.78:0/163194637 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb39000bb20 con 0x7fb3940cda10 2023-10-11T09:45:59.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.872+0000 7fb398ccd700 1 -- 172.21.15.78:0/163194637 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb390012430 con 0x7fb3940cda10 2023-10-11T09:45:59.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.872+0000 7fb39acd1700 1 -- 172.21.15.78:0/163194637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3940cda10 msgr2=0x7fb3940caf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.872+0000 7fb39acd1700 1 --2- 172.21.15.78:0/163194637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3940cda10 0x7fb3940caf20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb39406afb0 tx=0x7fb39000b330 comp rx=0 tx=0).stop 2023-10-11T09:45:59.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.872+0000 7fb39acd1700 1 -- 172.21.15.78:0/163194637 shutdown_connections 2023-10-11T09:45:59.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.872+0000 7fb39acd1700 1 --2- 172.21.15.78:0/163194637 >> v2:172.21.15.78:3300/0 conn(0x7fb394164d20 0x7fb39406bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.872+0000 7fb39acd1700 1 --2- 172.21.15.78:0/163194637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3940cda10 0x7fb3940caf20 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.872+0000 7fb39acd1700 1 -- 172.21.15.78:0/163194637 >> 172.21.15.78:0/163194637 conn(0x7fb3940cea60 msgr2=0x7fb3940cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:45:59.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.872+0000 7fb39acd1700 1 -- 172.21.15.78:0/163194637 shutdown_connections 2023-10-11T09:45:59.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.872+0000 7fb39acd1700 1 -- 172.21.15.78:0/163194637 wait complete. 2023-10-11T09:45:59.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.872+0000 7fb39acd1700 1 Processor -- start 2023-10-11T09:45:59.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.876+0000 7fb39acd1700 1 -- start start 2023-10-11T09:45:59.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.876+0000 7fb39acd1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3940cda10 0x7fb3941fa130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.876+0000 7fb39acd1700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3940cdde0 0x7fb39406bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.876+0000 7fb39acd1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb394164d20 0x7fb3941f5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.876+0000 7fb39acd1700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3941fa670 con 0x7fb394164d20 2023-10-11T09:45:59.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.876+0000 7fb39acd1700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3941620a0 con 0x7fb3940cda10 2023-10-11T09:45:59.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.876+0000 7fb39acd1700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb394162340 con 0x7fb3940cdde0 2023-10-11T09:45:59.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.876+0000 7fb39a4d0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb394164d20 0x7fb3941f5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.876+0000 7fb39a4d0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb394164d20 0x7fb3941f5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46596/0 (socket says 172.21.15.78:46596) 2023-10-11T09:45:59.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.876+0000 7fb39a4d0700 1 -- 172.21.15.78:0/1535092765 learned_addr learned my addr 172.21.15.78:0/1535092765 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:45:59.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.876+0000 7fb3994ce700 1 --2- 172.21.15.78:0/1535092765 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3940cdde0 0x7fb39406bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.876+0000 7fb399ccf700 1 --2- 172.21.15.78:0/1535092765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3940cda10 0x7fb3941fa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.876+0000 7fb39a4d0700 1 -- 172.21.15.78:0/1535092765 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3940cdde0 msgr2=0x7fb39406bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.876+0000 7fb39a4d0700 1 --2- 172.21.15.78:0/1535092765 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3940cdde0 0x7fb39406bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.876+0000 7fb39a4d0700 1 -- 172.21.15.78:0/1535092765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3940cda10 msgr2=0x7fb3941fa130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:45:59.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.876+0000 7fb39a4d0700 1 --2- 172.21.15.78:0/1535092765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3940cda10 0x7fb3941fa130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:45:59.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.876+0000 7fb39a4d0700 1 -- 172.21.15.78:0/1535092765 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb38c0094b0 con 0x7fb394164d20 2023-10-11T09:45:59.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.876+0000 7fb39a4d0700 1 --2- 172.21.15.78:0/1535092765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb394164d20 0x7fb3941f5600 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fb38c00eb60 tx=0x7fb38c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.876+0000 7fb38a7fc700 1 -- 172.21.15.78:0/1535092765 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38c004020 con 0x7fb394164d20 2023-10-11T09:45:59.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.876+0000 7fb38a7fc700 1 -- 172.21.15.78:0/1535092765 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb38c004180 con 0x7fb394164d20 2023-10-11T09:45:59.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.876+0000 7fb39acd1700 1 -- 172.21.15.78:0/1535092765 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb390009a70 con 0x7fb394164d20 2023-10-11T09:45:59.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.876+0000 7fb38a7fc700 1 -- 172.21.15.78:0/1535092765 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38c004530 con 0x7fb394164d20 2023-10-11T09:45:59.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.876+0000 7fb39acd1700 1 -- 172.21.15.78:0/1535092765 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3941626d0 con 0x7fb394164d20 2023-10-11T09:45:59.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.880+0000 7fb39acd1700 1 -- 172.21.15.78:0/1535092765 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3940b3820 con 0x7fb394164d20 2023-10-11T09:45:59.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.880+0000 7fb38a7fc700 1 -- 172.21.15.78:0/1535092765 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fb38c00c850 con 0x7fb394164d20 2023-10-11T09:45:59.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.884+0000 7fb38a7fc700 1 --2- 172.21.15.78:0/1535092765 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb378065ed0 0x7fb378068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:45:59.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.884+0000 7fb38a7fc700 1 -- 172.21.15.78:0/1535092765 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6638+0+0 (secure 0 0 0) 0x7fb38c0839a0 con 0x7fb394164d20 2023-10-11T09:45:59.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.884+0000 7fb399ccf700 1 --2- 172.21.15.78:0/1535092765 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb378065ed0 0x7fb378068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:45:59.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.884+0000 7fb399ccf700 1 --2- 172.21.15.78:0/1535092765 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb378065ed0 0x7fb378068380 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb390014040 tx=0x7fb390011000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:45:59.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:45:59.888+0000 7fb38a7fc700 1 -- 172.21.15.78:0/1535092765 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fb38c050d10 con 0x7fb394164d20 2023-10-11T09:45:59.947 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:45:59.955 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:46:00.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.128+0000 7fb39acd1700 1 -- 172.21.15.78:0/1535092765 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7fb394065d40 con 0x7fb394164d20 2023-10-11T09:46:00.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.400+0000 7fb38a7fc700 1 -- 172.21.15.78:0/1535092765 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 3:0 v44) v1 ==== 101+0+0 (secure 0 0 0) 0x7fb38c054330 con 0x7fb394164d20 2023-10-11T09:46:00.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.404+0000 7fb39acd1700 1 -- 172.21.15.78:0/1535092765 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb378065ed0 msgr2=0x7fb378068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:00.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.404+0000 7fb39acd1700 1 --2- 172.21.15.78:0/1535092765 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb378065ed0 0x7fb378068380 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb390014040 tx=0x7fb390011000 comp rx=0 tx=0).stop 2023-10-11T09:46:00.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.404+0000 7fb39acd1700 1 -- 172.21.15.78:0/1535092765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb394164d20 msgr2=0x7fb3941f5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:00.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.404+0000 7fb39acd1700 1 --2- 172.21.15.78:0/1535092765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb394164d20 0x7fb3941f5600 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fb38c00eb60 tx=0x7fb38c00ef20 comp rx=0 tx=0).stop 2023-10-11T09:46:00.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.404+0000 7fb39acd1700 1 -- 172.21.15.78:0/1535092765 shutdown_connections 2023-10-11T09:46:00.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.404+0000 7fb39acd1700 1 --2- 172.21.15.78:0/1535092765 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb378065ed0 0x7fb378068380 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:00.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.404+0000 7fb39acd1700 1 --2- 172.21.15.78:0/1535092765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb394164d20 0x7fb3941f5600 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:00.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.404+0000 7fb39acd1700 1 --2- 172.21.15.78:0/1535092765 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3940cdde0 0x7fb39406bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:00.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.404+0000 7fb39acd1700 1 --2- 172.21.15.78:0/1535092765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3940cda10 0x7fb3941fa130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:00.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.404+0000 7fb39acd1700 1 -- 172.21.15.78:0/1535092765 >> 172.21.15.78:0/1535092765 conn(0x7fb3940cea60 msgr2=0x7fb39406fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:46:00.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.404+0000 7fb39acd1700 1 -- 172.21.15.78:0/1535092765 shutdown_connections 2023-10-11T09:46:00.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.404+0000 7fb39acd1700 1 -- 172.21.15.78:0/1535092765 wait complete. 2023-10-11T09:46:00.411 INFO:teuthology.orchestra.run.smithi078.stderr:repaired: restoring rank 3:0 2023-10-11T09:46:00.415 INFO:tasks.ceph.mds.a.smithi078.stderr:2023-10-11T09:46:00.412+0000 7ff87e7c2700 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-10-11T09:46:00.435 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-11T09:46:00.435 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-11T09:46:00.435 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-11T09:46:00.435 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:46:00.436 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-11T09:46:00.436 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-11T09:46:00.436 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-11T09:46:00.437 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:46:00.437 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-11T09:46:00.437 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-11T09:46:00.437 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-11T09:46:00.438 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:46:00.438 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-11T09:46:00.438 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-11T09:46:00.439 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-11T09:46:00.439 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:46:00.456 INFO:tasks.ceph.mds.a:Stopped 2023-10-11T09:46:00.456 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-11T09:46:00.457 INFO:tasks.ceph.mds.c:Stopped 2023-10-11T09:46:00.458 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-11T09:46:00.459 INFO:tasks.ceph.mds.d:Stopped 2023-10-11T09:46:00.459 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-11T09:46:00.461 INFO:tasks.ceph.mds.b:Stopped 2023-10-11T09:46:00.461 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-11T09:46:00.508 INFO:tasks.ceph.mds.a:Started 2023-10-11T09:46:00.509 DEBUG:teuthology.parallel:result is None 2023-10-11T09:46:00.510 INFO:tasks.ceph.mds.d:Started 2023-10-11T09:46:00.511 DEBUG:teuthology.parallel:result is None 2023-10-11T09:46:00.514 INFO:tasks.ceph.mds.b:Started 2023-10-11T09:46:00.514 DEBUG:teuthology.parallel:result is None 2023-10-11T09:46:00.515 INFO:tasks.ceph.mds.c:Started 2023-10-11T09:46:00.516 DEBUG:teuthology.parallel:result is None 2023-10-11T09:46:00.516 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:46:00.626 INFO:tasks.ceph.mds.a.smithi078.stdout:starting mds.a at 2023-10-11T09:46:00.627 INFO:tasks.ceph.mds.c.smithi078.stdout:starting mds.c at 2023-10-11T09:46:00.630 INFO:tasks.ceph.mds.d.smithi133.stdout:starting mds.d at 2023-10-11T09:46:00.632 INFO:tasks.ceph.mds.b.smithi133.stdout:starting mds.b at 2023-10-11T09:46:00.638 INFO:tasks.ceph.mon.a.smithi078.stderr:2023-10-11T09:46:00.635+0000 7f90131f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-11T09:46:00.685 INFO:tasks.ceph.mds.a.smithi078.stderr:2023-10-11T09:46:00.679+0000 7f2933026700 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-10-11T09:46:00.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.683+0000 7fca3935e700 1 Processor -- start 2023-10-11T09:46:00.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.683+0000 7fca3935e700 1 -- start start 2023-10-11T09:46:00.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.683+0000 7fca3935e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca340b2310 0x7fca34161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:00.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.683+0000 7fca3935e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca340d11e0 con 0x7fca340b4b60 2023-10-11T09:46:00.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.683+0000 7fca3935e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca340b5370 con 0x7fca340b5f30 2023-10-11T09:46:00.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.683+0000 7fca3935e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca340d14a0 con 0x7fca340b2310 2023-10-11T09:46:00.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.683+0000 7fca327fc700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fca340b4b60 0x7fca340d1ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:60368/0 (socket says 172.21.15.78:60368) 2023-10-11T09:46:00.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.683+0000 7fca327fc700 1 -- 172.21.15.78:0/3541226063 learned_addr learned my addr 172.21.15.78:0/3541226063 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:46:00.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.683+0000 7fca32ffd700 1 --2- 172.21.15.78:0/3541226063 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca340b2310 0x7fca34161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:00.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.683+0000 7fca31ffb700 1 -- 172.21.15.78:0/3541226063 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3691999922 0 0) 0x7fca340d11e0 con 0x7fca340b4b60 2023-10-11T09:46:00.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.683+0000 7fca31ffb700 1 -- 172.21.15.78:0/3541226063 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca1c0035d0 con 0x7fca340b4b60 2023-10-11T09:46:00.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.683+0000 7fca31ffb700 1 -- 172.21.15.78:0/3541226063 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2162591356 0 0) 0x7fca340b5370 con 0x7fca340b5f30 2023-10-11T09:46:00.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.683+0000 7fca31ffb700 1 -- 172.21.15.78:0/3541226063 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca340d11e0 con 0x7fca340b5f30 2023-10-11T09:46:00.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.683+0000 7fca32ffd700 1 -- 172.21.15.78:0/3541226063 >> v1:172.21.15.133:6789/0 conn(0x7fca340b5f30 legacy=0x7fca340cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:00.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.687+0000 7fca32ffd700 1 -- 172.21.15.78:0/3541226063 >> v1:172.21.15.78:6789/0 conn(0x7fca340b4b60 legacy=0x7fca340d1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:00.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.687+0000 7fca31ffb700 1 -- 172.21.15.78:0/3541226063 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2235993157 0 0) 0x7fca1c0035d0 con 0x7fca340b4b60 2023-10-11T09:46:00.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.687+0000 7fca32ffd700 1 -- 172.21.15.78:0/3541226063 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca340d1780 con 0x7fca340b2310 2023-10-11T09:46:00.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.687+0000 7fca32ffd700 1 --2- 172.21.15.78:0/3541226063 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca340b2310 0x7fca34161150 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fca24009880 tx=0x7fca24009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d6b6f1cfaea5536 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:00.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.687+0000 7fca31ffb700 1 -- 172.21.15.78:0/3541226063 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca28001670 con 0x7fca340b2310 2023-10-11T09:46:00.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.687+0000 7fca31ffb700 1 -- 172.21.15.78:0/3541226063 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca24004030 con 0x7fca340b2310 2023-10-11T09:46:00.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.687+0000 7fca31ffb700 1 -- 172.21.15.78:0/3541226063 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca240043a0 con 0x7fca340b2310 2023-10-11T09:46:00.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.687+0000 7fca3935e700 1 -- 172.21.15.78:0/3541226063 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca340b2310 msgr2=0x7fca34161150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:00.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.687+0000 7fca3935e700 1 --2- 172.21.15.78:0/3541226063 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca340b2310 0x7fca34161150 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fca24009880 tx=0x7fca24009b90 comp rx=0 tx=0).stop 2023-10-11T09:46:00.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.687+0000 7fca3935e700 1 -- 172.21.15.78:0/3541226063 shutdown_connections 2023-10-11T09:46:00.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.687+0000 7fca3935e700 1 --2- 172.21.15.78:0/3541226063 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca340b2310 0x7fca34161150 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:00.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.687+0000 7fca3935e700 1 -- 172.21.15.78:0/3541226063 >> 172.21.15.78:0/3541226063 conn(0x7fca340cfb60 msgr2=0x7fca340cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:46:00.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.687+0000 7fca3935e700 1 -- 172.21.15.78:0/3541226063 shutdown_connections 2023-10-11T09:46:00.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.687+0000 7fca3935e700 1 -- 172.21.15.78:0/3541226063 wait complete. 2023-10-11T09:46:00.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.691+0000 7fca3935e700 1 Processor -- start 2023-10-11T09:46:00.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.691+0000 7fca3935e700 1 -- start start 2023-10-11T09:46:00.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.691+0000 7fca3935e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca340b2310 0x7fca340c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:00.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.691+0000 7fca3935e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca340b4b60 0x7fca34157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:00.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.691+0000 7fca3935e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca340b5f30 0x7fca34157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:00.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.691+0000 7fca3935e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca341589b0 con 0x7fca340b4b60 2023-10-11T09:46:00.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.691+0000 7fca3935e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca341455b0 con 0x7fca340b2310 2023-10-11T09:46:00.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.691+0000 7fca3935e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca340c6480 con 0x7fca340b5f30 2023-10-11T09:46:00.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.691+0000 7fca327fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca340b4b60 0x7fca34157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:00.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.691+0000 7fca327fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca340b4b60 0x7fca34157940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46632/0 (socket says 172.21.15.78:46632) 2023-10-11T09:46:00.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.691+0000 7fca327fc700 1 -- 172.21.15.78:0/593012034 learned_addr learned my addr 172.21.15.78:0/593012034 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:46:00.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.691+0000 7fca337fe700 1 --2- 172.21.15.78:0/593012034 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca340b5f30 0x7fca34157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:00.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.691+0000 7fca32ffd700 1 --2- 172.21.15.78:0/593012034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca340b2310 0x7fca340c5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:00.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.695+0000 7fca327fc700 1 -- 172.21.15.78:0/593012034 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca340b5f30 msgr2=0x7fca34157e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:00.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.695+0000 7fca327fc700 1 --2- 172.21.15.78:0/593012034 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca340b5f30 0x7fca34157e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:00.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.695+0000 7fca327fc700 1 -- 172.21.15.78:0/593012034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca340b2310 msgr2=0x7fca340c5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:00.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.695+0000 7fca327fc700 1 --2- 172.21.15.78:0/593012034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca340b2310 0x7fca340c5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:00.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.695+0000 7fca327fc700 1 -- 172.21.15.78:0/593012034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca240094b0 con 0x7fca340b4b60 2023-10-11T09:46:00.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.695+0000 7fca32ffd700 1 --2- 172.21.15.78:0/593012034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca340b2310 0x7fca340c5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:46:00.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.695+0000 7fca327fc700 1 --2- 172.21.15.78:0/593012034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca340b4b60 0x7fca34157940 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fca2800c780 tx=0x7fca2800ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:00.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.695+0000 7fca1b7fe700 1 -- 172.21.15.78:0/593012034 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca28004ca0 con 0x7fca340b4b60 2023-10-11T09:46:00.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.695+0000 7fca1b7fe700 1 -- 172.21.15.78:0/593012034 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca28004e40 con 0x7fca340b4b60 2023-10-11T09:46:00.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.695+0000 7fca1b7fe700 1 -- 172.21.15.78:0/593012034 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca28007310 con 0x7fca340b4b60 2023-10-11T09:46:00.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.695+0000 7fca3935e700 1 -- 172.21.15.78:0/593012034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca341586d0 con 0x7fca340b4b60 2023-10-11T09:46:00.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.695+0000 7fca3935e700 1 -- 172.21.15.78:0/593012034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca34145240 con 0x7fca340b4b60 2023-10-11T09:46:00.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.695+0000 7fca3935e700 1 -- 172.21.15.78:0/593012034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca34144dc0 con 0x7fca340b4b60 2023-10-11T09:46:00.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.695+0000 7fca1b7fe700 1 -- 172.21.15.78:0/593012034 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fca28007470 con 0x7fca340b4b60 2023-10-11T09:46:00.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.699+0000 7fca1b7fe700 1 --2- 172.21.15.78:0/593012034 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fca20065ed0 0x7fca20068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:00.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.699+0000 7fca1b7fe700 1 -- 172.21.15.78:0/593012034 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6724+0+0 (secure 0 0 0) 0x7fca28083730 con 0x7fca340b4b60 2023-10-11T09:46:00.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.699+0000 7fca32ffd700 1 --2- 172.21.15.78:0/593012034 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fca20065ed0 0x7fca20068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:00.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.699+0000 7fca32ffd700 1 --2- 172.21.15.78:0/593012034 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fca20065ed0 0x7fca20068380 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fca240090b0 tx=0x7fca24010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:00.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.703+0000 7fca1b7fe700 1 -- 172.21.15.78:0/593012034 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fca34144dc0 con 0x7fca340b4b60 2023-10-11T09:46:00.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.923+0000 7fca3935e700 1 -- 172.21.15.78:0/593012034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca34145760 con 0x7fca340b4b60 2023-10-11T09:46:00.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.923+0000 7fca1b7fe700 1 -- 172.21.15.78:0/593012034 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4658 (secure 0 0 0) 0x7fca34145760 con 0x7fca340b4b60 2023-10-11T09:46:00.929 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:46:00.929 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":47,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5579,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3954348479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":3954348479},{"type":"v1","addr":"172.21.15.133:6835","nonce":3954348479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":5783,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3328076595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":3328076595},{"type":"v1","addr":"172.21.15.78:6837","nonce":3328076595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":6664,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2877345875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2877345875},{"type":"v1","addr":"172.21.15.133:6833","nonce":2877345875}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:45:54.252958+0000","modified":"2023-10-11T09:46:00.649393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6897},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6897":{"gid":6897,"name":"a","rank":0,"incarnation":47,"state":"up:replay","state_seq":1,"addr":"172.21.15.78:6837/3498868661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":3498868661},{"type":"v1","addr":"172.21.15.78:6837","nonce":3498868661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9,10],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-11T09:46:00.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.927+0000 7fca3935e700 1 -- 172.21.15.78:0/593012034 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fca20065ed0 msgr2=0x7fca20068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:00.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.927+0000 7fca3935e700 1 --2- 172.21.15.78:0/593012034 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fca20065ed0 0x7fca20068380 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fca240090b0 tx=0x7fca24010000 comp rx=0 tx=0).stop 2023-10-11T09:46:00.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.927+0000 7fca3935e700 1 -- 172.21.15.78:0/593012034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca340b4b60 msgr2=0x7fca34157940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:00.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.927+0000 7fca3935e700 1 --2- 172.21.15.78:0/593012034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca340b4b60 0x7fca34157940 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fca2800c780 tx=0x7fca2800ca90 comp rx=0 tx=0).stop 2023-10-11T09:46:00.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.927+0000 7fca3935e700 1 -- 172.21.15.78:0/593012034 shutdown_connections 2023-10-11T09:46:00.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.927+0000 7fca3935e700 1 --2- 172.21.15.78:0/593012034 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fca20065ed0 0x7fca20068380 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:00.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.927+0000 7fca3935e700 1 --2- 172.21.15.78:0/593012034 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca340b5f30 0x7fca34157e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:00.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.927+0000 7fca3935e700 1 --2- 172.21.15.78:0/593012034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca340b4b60 0x7fca34157940 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:00.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.927+0000 7fca3935e700 1 --2- 172.21.15.78:0/593012034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca340b2310 0x7fca340c5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:00.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.927+0000 7fca3935e700 1 -- 172.21.15.78:0/593012034 >> 172.21.15.78:0/593012034 conn(0x7fca340cfb60 msgr2=0x7fca340d2530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:46:00.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.927+0000 7fca3935e700 1 -- 172.21.15.78:0/593012034 shutdown_connections 2023-10-11T09:46:00.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:00.927+0000 7fca3935e700 1 -- 172.21.15.78:0/593012034 wait complete. 2023-10-11T09:46:00.934 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 47 2023-10-11T09:46:01.358 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 47, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-11T09:45:54.252958+0000', 'modified': '2023-10-11T09:46:00.649393+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 67, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6897}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6897': {'gid': 6897, 'name': 'a', 'rank': 0, 'incarnation': 47, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.78:6837/3498868661', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6835', 'nonce': 3498868661}, {'type': 'v1', 'addr': '172.21.15.78:6837', 'nonce': 3498868661}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9, 10], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-11T09:46:01.358 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6897:up:replay 2023-10-11T09:46:02.360 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:46:02.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.531+0000 7f37289ba700 1 Processor -- start 2023-10-11T09:46:02.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.531+0000 7f37289ba700 1 -- start start 2023-10-11T09:46:02.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.535+0000 7f37289ba700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3724071040 0x7f37240cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:02.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.535+0000 7f37289ba700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f37240ce330 0x7f372406be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:02.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.535+0000 7f37289ba700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f372406ffa0 con 0x7f3724160d40 2023-10-11T09:46:02.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.535+0000 7f37289ba700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3724070c20 con 0x7f37240ce330 2023-10-11T09:46:02.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.535+0000 7f37289ba700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3724161110 con 0x7f3724071040 2023-10-11T09:46:02.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.535+0000 7f372259c700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f3724160d40 0x7f37240cd280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:60372/0 (socket says 172.21.15.78:60372) 2023-10-11T09:46:02.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.535+0000 7f372259c700 1 -- 172.21.15.78:0/3562716067 learned_addr learned my addr 172.21.15.78:0/3562716067 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:46:02.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.535+0000 7f3722d9d700 1 --2- 172.21.15.78:0/3562716067 >> v2:172.21.15.133:3300/0 conn(0x7f37240ce330 0x7f372406be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:02.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.535+0000 7f3721d9b700 1 --2- 172.21.15.78:0/3562716067 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3724071040 0x7f37240cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:02.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.535+0000 7f372159a700 1 -- 172.21.15.78:0/3562716067 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2456886959 0 0) 0x7f372406ffa0 con 0x7f3724160d40 2023-10-11T09:46:02.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.535+0000 7f372159a700 1 -- 172.21.15.78:0/3562716067 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f370c0035d0 con 0x7f3724160d40 2023-10-11T09:46:02.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.535+0000 7f3721d9b700 1 -- 172.21.15.78:0/3562716067 >> v2:172.21.15.133:3300/0 conn(0x7f37240ce330 msgr2=0x7f372406be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:02.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.535+0000 7f3721d9b700 1 --2- 172.21.15.78:0/3562716067 >> v2:172.21.15.133:3300/0 conn(0x7f37240ce330 0x7f372406be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:02.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.535+0000 7f3721d9b700 1 -- 172.21.15.78:0/3562716067 >> v1:172.21.15.78:6789/0 conn(0x7f3724160d40 legacy=0x7f37240cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:02.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.535+0000 7f3721d9b700 1 -- 172.21.15.78:0/3562716067 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3724070280 con 0x7f3724071040 2023-10-11T09:46:02.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.535+0000 7f3722d9d700 1 --2- 172.21.15.78:0/3562716067 >> v2:172.21.15.133:3300/0 conn(0x7f37240ce330 0x7f372406be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:46:02.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.535+0000 7f3721d9b700 1 --2- 172.21.15.78:0/3562716067 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3724071040 0x7f37240cddf0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3718009850 tx=0x7f3718009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfe675a086d287a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:02.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.535+0000 7f372159a700 1 -- 172.21.15.78:0/3562716067 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3718011830 con 0x7f3724071040 2023-10-11T09:46:02.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.535+0000 7f372159a700 1 -- 172.21.15.78:0/3562716067 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3718011990 con 0x7f3724071040 2023-10-11T09:46:02.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.535+0000 7f372159a700 1 -- 172.21.15.78:0/3562716067 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3718011d40 con 0x7f3724071040 2023-10-11T09:46:02.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.539+0000 7f37289ba700 1 -- 172.21.15.78:0/3562716067 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3724071040 msgr2=0x7f37240cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:02.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.539+0000 7f37289ba700 1 --2- 172.21.15.78:0/3562716067 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3724071040 0x7f37240cddf0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3718009850 tx=0x7f3718009b60 comp rx=0 tx=0).stop 2023-10-11T09:46:02.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.539+0000 7f37289ba700 1 -- 172.21.15.78:0/3562716067 shutdown_connections 2023-10-11T09:46:02.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.539+0000 7f37289ba700 1 --2- 172.21.15.78:0/3562716067 >> v2:172.21.15.133:3300/0 conn(0x7f37240ce330 0x7f372406be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:02.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.539+0000 7f37289ba700 1 --2- 172.21.15.78:0/3562716067 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3724071040 0x7f37240cddf0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:02.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.539+0000 7f37289ba700 1 -- 172.21.15.78:0/3562716067 >> 172.21.15.78:0/3562716067 conn(0x7f372406aca0 msgr2=0x7f37240b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:46:02.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.539+0000 7f37289ba700 1 -- 172.21.15.78:0/3562716067 shutdown_connections 2023-10-11T09:46:02.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.539+0000 7f37289ba700 1 -- 172.21.15.78:0/3562716067 wait complete. 2023-10-11T09:46:02.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.543+0000 7f37289ba700 1 Processor -- start 2023-10-11T09:46:02.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.543+0000 7f37289ba700 1 -- start start 2023-10-11T09:46:02.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.543+0000 7f37289ba700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3724071040 0x7f3724159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:02.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.543+0000 7f37289ba700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37240ce330 0x7f37240b4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:02.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.543+0000 7f37289ba700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3724160d40 0x7f372415c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:02.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.543+0000 7f37289ba700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3724157f40 con 0x7f37240ce330 2023-10-11T09:46:02.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.543+0000 7f37289ba700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37240b3600 con 0x7f3724071040 2023-10-11T09:46:02.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.543+0000 7f37289ba700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37240b2a80 con 0x7f3724160d40 2023-10-11T09:46:02.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.543+0000 7f3721d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37240ce330 0x7f37240b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:02.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.543+0000 7f3721d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37240ce330 0x7f37240b4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46646/0 (socket says 172.21.15.78:46646) 2023-10-11T09:46:02.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.543+0000 7f3721d9b700 1 -- 172.21.15.78:0/871275630 learned_addr learned my addr 172.21.15.78:0/871275630 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:46:02.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.543+0000 7f372259c700 1 --2- 172.21.15.78:0/871275630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3724071040 0x7f3724159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:02.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.543+0000 7f3722d9d700 1 --2- 172.21.15.78:0/871275630 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3724160d40 0x7f372415c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:02.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.543+0000 7f3721d9b700 1 -- 172.21.15.78:0/871275630 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3724160d40 msgr2=0x7f372415c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:02.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.543+0000 7f3721d9b700 1 --2- 172.21.15.78:0/871275630 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3724160d40 0x7f372415c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:02.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.543+0000 7f3721d9b700 1 -- 172.21.15.78:0/871275630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3724071040 msgr2=0x7f3724159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:02.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.543+0000 7f3721d9b700 1 --2- 172.21.15.78:0/871275630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3724071040 0x7f3724159f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:02.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.543+0000 7f3721d9b700 1 -- 172.21.15.78:0/871275630 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f371c009a70 con 0x7f37240ce330 2023-10-11T09:46:02.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.547+0000 7f3721d9b700 1 --2- 172.21.15.78:0/871275630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37240ce330 0x7f37240b4560 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f3718013000 tx=0x7f3718004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:02.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.547+0000 7f3712ffd700 1 -- 172.21.15.78:0/871275630 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37180042c0 con 0x7f37240ce330 2023-10-11T09:46:02.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.547+0000 7f3712ffd700 1 -- 172.21.15.78:0/871275630 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f371801f070 con 0x7f37240ce330 2023-10-11T09:46:02.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.547+0000 7f37289ba700 1 -- 172.21.15.78:0/871275630 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37180094b0 con 0x7f37240ce330 2023-10-11T09:46:02.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.547+0000 7f3712ffd700 1 -- 172.21.15.78:0/871275630 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37180036c0 con 0x7f37240ce330 2023-10-11T09:46:02.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.547+0000 7f37289ba700 1 -- 172.21.15.78:0/871275630 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37240b3040 con 0x7f37240ce330 2023-10-11T09:46:02.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.547+0000 7f37289ba700 1 -- 172.21.15.78:0/871275630 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3724072250 con 0x7f37240ce330 2023-10-11T09:46:02.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.551+0000 7f3712ffd700 1 -- 172.21.15.78:0/871275630 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f3718021030 con 0x7f37240ce330 2023-10-11T09:46:02.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.551+0000 7f3712ffd700 1 --2- 172.21.15.78:0/871275630 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3704065ed0 0x7f3704068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:02.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.551+0000 7f3712ffd700 1 -- 172.21.15.78:0/871275630 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f3718084c20 con 0x7f37240ce330 2023-10-11T09:46:02.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.551+0000 7f372259c700 1 --2- 172.21.15.78:0/871275630 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3704065ed0 0x7f3704068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:02.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.551+0000 7f372259c700 1 --2- 172.21.15.78:0/871275630 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3704065ed0 0x7f3704068380 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3714007fd0 tx=0x7f3714007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:02.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.555+0000 7f3712ffd700 1 -- 172.21.15.78:0/871275630 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f37180520e0 con 0x7f37240ce330 2023-10-11T09:46:02.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.763+0000 7f37289ba700 1 -- 172.21.15.78:0/871275630 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3724158090 con 0x7f37240ce330 2023-10-11T09:46:02.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.767+0000 7f3712ffd700 1 -- 172.21.15.78:0/871275630 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4655 (secure 0 0 0) 0x7f3724158090 con 0x7f37240ce330 2023-10-11T09:46:02.772 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:46:02.772 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":49,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6664,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2877345875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2877345875},{"type":"v1","addr":"172.21.15.133:6833","nonce":2877345875}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":6667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/792033922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":792033922},{"type":"v1","addr":"172.21.15.133:6835","nonce":792033922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":6894,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/1229638118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":1229638118},{"type":"v1","addr":"172.21.15.78:6836","nonce":1229638118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:45:54.252958+0000","modified":"2023-10-11T09:46:02.665103+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6897},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6897":{"gid":6897,"name":"a","rank":0,"incarnation":47,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.78:6837/3498868661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":3498868661},{"type":"v1","addr":"172.21.15.78:6837","nonce":3498868661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9,10],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-11T09:46:02.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.767+0000 7f37289ba700 1 -- 172.21.15.78:0/871275630 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3704065ed0 msgr2=0x7f3704068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:02.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.767+0000 7f37289ba700 1 --2- 172.21.15.78:0/871275630 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3704065ed0 0x7f3704068380 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3714007fd0 tx=0x7f3714007de0 comp rx=0 tx=0).stop 2023-10-11T09:46:02.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.767+0000 7f37289ba700 1 -- 172.21.15.78:0/871275630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37240ce330 msgr2=0x7f37240b4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:02.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.767+0000 7f37289ba700 1 --2- 172.21.15.78:0/871275630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37240ce330 0x7f37240b4560 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f3718013000 tx=0x7f3718004770 comp rx=0 tx=0).stop 2023-10-11T09:46:02.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.767+0000 7f37289ba700 1 -- 172.21.15.78:0/871275630 shutdown_connections 2023-10-11T09:46:02.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.767+0000 7f37289ba700 1 --2- 172.21.15.78:0/871275630 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3704065ed0 0x7f3704068380 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:02.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.767+0000 7f37289ba700 1 --2- 172.21.15.78:0/871275630 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3724160d40 0x7f372415c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:02.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.767+0000 7f37289ba700 1 --2- 172.21.15.78:0/871275630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37240ce330 0x7f37240b4560 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:02.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.767+0000 7f37289ba700 1 --2- 172.21.15.78:0/871275630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3724071040 0x7f3724159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:02.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.767+0000 7f37289ba700 1 -- 172.21.15.78:0/871275630 >> 172.21.15.78:0/871275630 conn(0x7f372406aca0 msgr2=0x7f37240cd280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:46:02.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.771+0000 7f37289ba700 1 -- 172.21.15.78:0/871275630 shutdown_connections 2023-10-11T09:46:02.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:02.771+0000 7f37289ba700 1 -- 172.21.15.78:0/871275630 wait complete. 2023-10-11T09:46:02.777 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 49 2023-10-11T09:46:03.198 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 49, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-11T09:45:54.252958+0000', 'modified': '2023-10-11T09:46:02.665103+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 67, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6897}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6897': {'gid': 6897, 'name': 'a', 'rank': 0, 'incarnation': 47, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.78:6837/3498868661', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6835', 'nonce': 3498868661}, {'type': 'v1', 'addr': '172.21.15.78:6837', 'nonce': 3498868661}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9, 10], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-11T09:46:03.198 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6897:up:rejoin 2023-10-11T09:46:04.200 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:46:04.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.371+0000 7f451a1ef700 1 Processor -- start 2023-10-11T09:46:04.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.371+0000 7f451a1ef700 1 -- start start 2023-10-11T09:46:04.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.375+0000 7f451a1ef700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f451406cd00 0x7f451406d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:04.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.375+0000 7f451a1ef700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f451406d690 con 0x7f451406fc70 2023-10-11T09:46:04.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.375+0000 7f451a1ef700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45140cd1c0 con 0x7f451413f740 2023-10-11T09:46:04.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.375+0000 7f451a1ef700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45140ce4c0 con 0x7f451406cd00 2023-10-11T09:46:04.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.375+0000 7f45199ee700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f451406fc70 0x7f45140700f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:60388/0 (socket says 172.21.15.78:60388) 2023-10-11T09:46:04.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.375+0000 7f45199ee700 1 -- 172.21.15.78:0/3991212391 learned_addr learned my addr 172.21.15.78:0/3991212391 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:46:04.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.375+0000 7f45189ec700 1 --2- 172.21.15.78:0/3991212391 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f451406cd00 0x7f451406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:04.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.375+0000 7f450bfff700 1 -- 172.21.15.78:0/3991212391 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2231483554 0 0) 0x7f450c001490 con 0x7f451406fc70 2023-10-11T09:46:04.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.375+0000 7f450bfff700 1 -- 172.21.15.78:0/3991212391 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45000035d0 con 0x7f451406fc70 2023-10-11T09:46:04.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.375+0000 7f450bfff700 1 -- 172.21.15.78:0/3991212391 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2457741286 0 0) 0x7f45140cd1c0 con 0x7f451413f740 2023-10-11T09:46:04.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.375+0000 7f450bfff700 1 -- 172.21.15.78:0/3991212391 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f450c001490 con 0x7f451413f740 2023-10-11T09:46:04.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.375+0000 7f450bfff700 1 -- 172.21.15.78:0/3991212391 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2234088724 0 0) 0x7f45000035d0 con 0x7f451406fc70 2023-10-11T09:46:04.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.375+0000 7f450bfff700 1 -- 172.21.15.78:0/3991212391 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f451406cd00 msgr2=0x7f451406d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:04.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.375+0000 7f450bfff700 1 --2- 172.21.15.78:0/3991212391 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f451406cd00 0x7f451406d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:04.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.375+0000 7f450bfff700 1 -- 172.21.15.78:0/3991212391 >> v1:172.21.15.133:6789/0 conn(0x7f451413f740 legacy=0x7f45140ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:04.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.375+0000 7f450bfff700 1 -- 172.21.15.78:0/3991212391 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45140cd420 con 0x7f451406fc70 2023-10-11T09:46:04.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.375+0000 7f450bfff700 1 -- 172.21.15.78:0/3991212391 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f450c001b80 con 0x7f451406fc70 2023-10-11T09:46:04.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.375+0000 7f450bfff700 1 -- 172.21.15.78:0/3991212391 >> v1:172.21.15.78:6789/0 conn(0x7f451406fc70 legacy=0x7f45140700f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:04.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.375+0000 7f450bfff700 1 --2- 172.21.15.78:0/3991212391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4500003be0 0x7f4500003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:04.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.375+0000 7f450bfff700 1 --2- 172.21.15.78:0/3991212391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4500004870 0x7f4500004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:04.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.375+0000 7f450bfff700 1 --2- 172.21.15.78:0/3991212391 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f451406cd00 0x7f4500005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:04.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.375+0000 7f450bfff700 1 -- 172.21.15.78:0/3991212391 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45140ce4c0 con 0x7f4500003be0 2023-10-11T09:46:04.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.375+0000 7f450bfff700 1 -- 172.21.15.78:0/3991212391 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450000bae0 con 0x7f4500004870 2023-10-11T09:46:04.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.375+0000 7f450bfff700 1 -- 172.21.15.78:0/3991212391 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f450000bc50 con 0x7f451406cd00 2023-10-11T09:46:04.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.379+0000 7f45191ed700 1 --2- 172.21.15.78:0/3991212391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4500003be0 0x7f4500003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:04.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.379+0000 7f45189ec700 1 --2- 172.21.15.78:0/3991212391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4500004870 0x7f4500004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:04.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.379+0000 7f45199ee700 1 --2- 172.21.15.78:0/3991212391 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f451406cd00 0x7f4500005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:04.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.379+0000 7f45191ed700 1 -- 172.21.15.78:0/3991212391 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f451406cd00 msgr2=0x7f4500005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:04.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.379+0000 7f45191ed700 1 --2- 172.21.15.78:0/3991212391 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f451406cd00 0x7f4500005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:04.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.379+0000 7f45191ed700 1 -- 172.21.15.78:0/3991212391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4500004870 msgr2=0x7f4500004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:04.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.379+0000 7f45191ed700 1 --2- 172.21.15.78:0/3991212391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4500004870 0x7f4500004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:04.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.379+0000 7f45191ed700 1 -- 172.21.15.78:0/3991212391 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45000035d0 con 0x7f4500003be0 2023-10-11T09:46:04.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.379+0000 7f45199ee700 1 --2- 172.21.15.78:0/3991212391 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f451406cd00 0x7f4500005220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:46:04.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.379+0000 7f45191ed700 1 --2- 172.21.15.78:0/3991212391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4500003be0 0x7f4500003fd0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f4510001960 tx=0x7f45000058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46fa2f190a6cb7a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:04.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.379+0000 7f450bfff700 1 -- 172.21.15.78:0/3991212391 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4510001450 con 0x7f4500003be0 2023-10-11T09:46:04.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.379+0000 7f450bfff700 1 -- 172.21.15.78:0/3991212391 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45100052f0 con 0x7f4500003be0 2023-10-11T09:46:04.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.379+0000 7f450bfff700 1 -- 172.21.15.78:0/3991212391 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45100036a0 con 0x7f4500003be0 2023-10-11T09:46:04.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.379+0000 7f451a1ef700 1 -- 172.21.15.78:0/3991212391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4500003be0 msgr2=0x7f4500003fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:04.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.379+0000 7f451a1ef700 1 --2- 172.21.15.78:0/3991212391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4500003be0 0x7f4500003fd0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f4510001960 tx=0x7f45000058a0 comp rx=0 tx=0).stop 2023-10-11T09:46:04.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.379+0000 7f45199ee700 1 -- 172.21.15.78:0/3991212391 reap_dead start 2023-10-11T09:46:04.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.379+0000 7f451a1ef700 1 -- 172.21.15.78:0/3991212391 shutdown_connections 2023-10-11T09:46:04.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.379+0000 7f451a1ef700 1 -- 172.21.15.78:0/3991212391 >> 172.21.15.78:0/3991212391 conn(0x7f45140cea60 msgr2=0x7f45140cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:46:04.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.379+0000 7f451a1ef700 1 -- 172.21.15.78:0/3991212391 shutdown_connections 2023-10-11T09:46:04.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.379+0000 7f451a1ef700 1 -- 172.21.15.78:0/3991212391 wait complete. 2023-10-11T09:46:04.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.383+0000 7f451a1ef700 1 Processor -- start 2023-10-11T09:46:04.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.383+0000 7f451a1ef700 1 -- start start 2023-10-11T09:46:04.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.383+0000 7f451a1ef700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4500003be0 0x7f451414c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:04.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.383+0000 7f451a1ef700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4514159870 0x7f45140b4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:04.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.383+0000 7f451a1ef700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45141f1210 0x7f45141f1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:04.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.383+0000 7f451a1ef700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451415cb00 con 0x7f4514159870 2023-10-11T09:46:04.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.383+0000 7f451a1ef700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45141f1bc0 con 0x7f45141f1210 2023-10-11T09:46:04.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.383+0000 7f451a1ef700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f451415bec0 con 0x7f4500003be0 2023-10-11T09:46:04.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.383+0000 7f45189ec700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4514159870 0x7f45140b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:04.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.383+0000 7f45189ec700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4514159870 0x7f45140b4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46664/0 (socket says 172.21.15.78:46664) 2023-10-11T09:46:04.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.383+0000 7f45189ec700 1 -- 172.21.15.78:0/3495980219 learned_addr learned my addr 172.21.15.78:0/3495980219 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:46:04.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.387+0000 7f45191ed700 1 --2- 172.21.15.78:0/3495980219 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4500003be0 0x7f451414c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:04.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.387+0000 7f45199ee700 1 --2- 172.21.15.78:0/3495980219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45141f1210 0x7f45141f1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:04.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.387+0000 7f45189ec700 1 -- 172.21.15.78:0/3495980219 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4500003be0 msgr2=0x7f451414c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:04.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.387+0000 7f45189ec700 1 --2- 172.21.15.78:0/3495980219 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4500003be0 0x7f451414c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:04.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.387+0000 7f45189ec700 1 -- 172.21.15.78:0/3495980219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45141f1210 msgr2=0x7f45141f1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:04.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.387+0000 7f45189ec700 1 --2- 172.21.15.78:0/3495980219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45141f1210 0x7f45141f1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:04.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.387+0000 7f45189ec700 1 -- 172.21.15.78:0/3495980219 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f451415c160 con 0x7f4514159870 2023-10-11T09:46:04.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.387+0000 7f45191ed700 1 --2- 172.21.15.78:0/3495980219 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4500003be0 0x7f451414c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:46:04.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.387+0000 7f45199ee700 1 --2- 172.21.15.78:0/3495980219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45141f1210 0x7f45141f1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:46:04.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.387+0000 7f45189ec700 1 --2- 172.21.15.78:0/3495980219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4514159870 0x7f45140b4410 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f4504011700 tx=0x7f4504011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:04.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.387+0000 7f4509ffb700 1 -- 172.21.15.78:0/3495980219 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4504015870 con 0x7f4514159870 2023-10-11T09:46:04.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.387+0000 7f4509ffb700 1 -- 172.21.15.78:0/3495980219 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45040159d0 con 0x7f4514159870 2023-10-11T09:46:04.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.387+0000 7f451a1ef700 1 -- 172.21.15.78:0/3495980219 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f451415c440 con 0x7f4514159870 2023-10-11T09:46:04.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.387+0000 7f4509ffb700 1 -- 172.21.15.78:0/3495980219 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4504015d40 con 0x7f4514159870 2023-10-11T09:46:04.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.387+0000 7f451a1ef700 1 -- 172.21.15.78:0/3495980219 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4514159ce0 con 0x7f4514159870 2023-10-11T09:46:04.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.391+0000 7f451a1ef700 1 -- 172.21.15.78:0/3495980219 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4514065d40 con 0x7f4514159870 2023-10-11T09:46:04.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.391+0000 7f4509ffb700 1 -- 172.21.15.78:0/3495980219 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f4504013400 con 0x7f4514159870 2023-10-11T09:46:04.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.391+0000 7f4509ffb700 1 --2- 172.21.15.78:0/3495980219 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f44f8065ed0 0x7f44f8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:04.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.391+0000 7f4509ffb700 1 -- 172.21.15.78:0/3495980219 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f4504084270 con 0x7f4514159870 2023-10-11T09:46:04.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.391+0000 7f45191ed700 1 --2- 172.21.15.78:0/3495980219 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f44f8065ed0 0x7f44f8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:04.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.395+0000 7f45191ed700 1 --2- 172.21.15.78:0/3495980219 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f44f8065ed0 0x7f44f8068380 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4510005040 tx=0x7f451000a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:04.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.399+0000 7f4509ffb700 1 -- 172.21.15.78:0/3495980219 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f4504051590 con 0x7f4514159870 2023-10-11T09:46:04.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.607+0000 7f451a1ef700 1 -- 172.21.15.78:0/3495980219 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45140ca4d0 con 0x7f4514159870 2023-10-11T09:46:04.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.607+0000 7f4509ffb700 1 -- 172.21.15.78:0/3495980219 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4655 (secure 0 0 0) 0x7f4504054bb0 con 0x7f4514159870 2023-10-11T09:46:04.613 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:46:04.613 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":50,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6664,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2877345875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2877345875},{"type":"v1","addr":"172.21.15.133:6833","nonce":2877345875}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":6667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/792033922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":792033922},{"type":"v1","addr":"172.21.15.133:6835","nonce":792033922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":6894,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/1229638118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":1229638118},{"type":"v1","addr":"172.21.15.78:6836","nonce":1229638118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:45:54.252958+0000","modified":"2023-10-11T09:46:03.678418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6897},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6897":{"gid":6897,"name":"a","rank":0,"incarnation":47,"state":"up:active","state_seq":4,"addr":"172.21.15.78:6837/3498868661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":3498868661},{"type":"v1","addr":"172.21.15.78:6837","nonce":3498868661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9,10],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-11T09:46:04.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.611+0000 7f451a1ef700 1 -- 172.21.15.78:0/3495980219 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f44f8065ed0 msgr2=0x7f44f8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:04.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.611+0000 7f451a1ef700 1 --2- 172.21.15.78:0/3495980219 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f44f8065ed0 0x7f44f8068380 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4510005040 tx=0x7f451000a000 comp rx=0 tx=0).stop 2023-10-11T09:46:04.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.611+0000 7f451a1ef700 1 -- 172.21.15.78:0/3495980219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4514159870 msgr2=0x7f45140b4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:04.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.611+0000 7f451a1ef700 1 --2- 172.21.15.78:0/3495980219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4514159870 0x7f45140b4410 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f4504011700 tx=0x7f4504011ac0 comp rx=0 tx=0).stop 2023-10-11T09:46:04.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.611+0000 7f451a1ef700 1 -- 172.21.15.78:0/3495980219 shutdown_connections 2023-10-11T09:46:04.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.611+0000 7f451a1ef700 1 --2- 172.21.15.78:0/3495980219 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f44f8065ed0 0x7f44f8068380 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:04.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.611+0000 7f451a1ef700 1 --2- 172.21.15.78:0/3495980219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45141f1210 0x7f45141f1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:04.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.611+0000 7f451a1ef700 1 --2- 172.21.15.78:0/3495980219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4514159870 0x7f45140b4410 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:04.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.611+0000 7f451a1ef700 1 --2- 172.21.15.78:0/3495980219 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4500003be0 0x7f451414c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:04.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.611+0000 7f451a1ef700 1 -- 172.21.15.78:0/3495980219 >> 172.21.15.78:0/3495980219 conn(0x7f45140cea60 msgr2=0x7f451413d040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:46:04.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.611+0000 7f451a1ef700 1 -- 172.21.15.78:0/3495980219 shutdown_connections 2023-10-11T09:46:04.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:04.611+0000 7f451a1ef700 1 -- 172.21.15.78:0/3495980219 wait complete. 2023-10-11T09:46:04.618 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 50 2023-10-11T09:46:05.037 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 50, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-11T09:45:54.252958+0000', 'modified': '2023-10-11T09:46:03.678418+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 67, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6897}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6897': {'gid': 6897, 'name': 'a', 'rank': 0, 'incarnation': 47, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.78:6837/3498868661', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6835', 'nonce': 3498868661}, {'type': 'v1', 'addr': '172.21.15.78:6837', 'nonce': 3498868661}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9, 10], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-11T09:46:05.038 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-11T09:46:05.038 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-11T09:46:05.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.211+0000 7fcba9116700 1 Processor -- start 2023-10-11T09:46:05.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.211+0000 7fcba9116700 1 -- start start 2023-10-11T09:46:05.218 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.211+0000 7fcba9116700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcba40ca950 0x7fcba41650f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:05.218 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.211+0000 7fcba9116700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcba415e010 con 0x7fcba415b8e0 2023-10-11T09:46:05.218 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.211+0000 7fcba9116700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcba415e280 con 0x7fcba415a940 2023-10-11T09:46:05.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.211+0000 7fcba9116700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba415e560 con 0x7fcba40ca950 2023-10-11T09:46:05.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.215+0000 7fcba3fff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fcba415b8e0 0x7fcba406d0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:39860/0 (socket says 172.21.15.78:39860) 2023-10-11T09:46:05.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.215+0000 7fcba3fff700 1 -- 172.21.15.78:0/1160093410 learned_addr learned my addr 172.21.15.78:0/1160093410 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:46:05.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.215+0000 7fcba8915700 1 --2- 172.21.15.78:0/1160093410 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcba40ca950 0x7fcba41650f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:05.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.215+0000 7fcba2ffd700 1 -- 172.21.15.78:0/1160093410 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 517860717 0 0) 0x7fcba415e010 con 0x7fcba415b8e0 2023-10-11T09:46:05.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.215+0000 7fcba2ffd700 1 -- 172.21.15.78:0/1160093410 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb8c0035d0 con 0x7fcba415b8e0 2023-10-11T09:46:05.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.215+0000 7fcba2ffd700 1 -- 172.21.15.78:0/1160093410 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 885076567 0 0) 0x7fcba415e280 con 0x7fcba415a940 2023-10-11T09:46:05.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.215+0000 7fcba2ffd700 1 -- 172.21.15.78:0/1160093410 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcba415e010 con 0x7fcba415a940 2023-10-11T09:46:05.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.215+0000 7fcba8915700 1 -- 172.21.15.78:0/1160093410 >> v1:172.21.15.133:6789/0 conn(0x7fcba415a940 legacy=0x7fcba40ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:05.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.215+0000 7fcba8915700 1 -- 172.21.15.78:0/1160093410 >> v1:172.21.15.78:6789/0 conn(0x7fcba415b8e0 legacy=0x7fcba406d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:05.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.215+0000 7fcba8915700 1 -- 172.21.15.78:0/1160093410 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba415e840 con 0x7fcba40ca950 2023-10-11T09:46:05.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.215+0000 7fcba8915700 1 --2- 172.21.15.78:0/1160093410 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcba40ca950 0x7fcba41650f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c009dc0 tx=0x7fcb9c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=346bde79da077656 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:05.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.215+0000 7fcba2ffd700 1 -- 172.21.15.78:0/1160093410 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c00e070 con 0x7fcba40ca950 2023-10-11T09:46:05.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.215+0000 7fcba2ffd700 1 -- 172.21.15.78:0/1160093410 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9c00bb20 con 0x7fcba40ca950 2023-10-11T09:46:05.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.215+0000 7fcba2ffd700 1 -- 172.21.15.78:0/1160093410 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c012430 con 0x7fcba40ca950 2023-10-11T09:46:05.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.219+0000 7fcba9116700 1 -- 172.21.15.78:0/1160093410 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcba40ca950 msgr2=0x7fcba41650f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:05.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.219+0000 7fcba9116700 1 --2- 172.21.15.78:0/1160093410 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcba40ca950 0x7fcba41650f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c009dc0 tx=0x7fcb9c00b330 comp rx=0 tx=0).stop 2023-10-11T09:46:05.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.219+0000 7fcba9116700 1 -- 172.21.15.78:0/1160093410 shutdown_connections 2023-10-11T09:46:05.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.219+0000 7fcba9116700 1 --2- 172.21.15.78:0/1160093410 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcba40ca950 0x7fcba41650f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.219+0000 7fcba9116700 1 -- 172.21.15.78:0/1160093410 >> 172.21.15.78:0/1160093410 conn(0x7fcba40cea60 msgr2=0x7fcba40cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:46:05.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.219+0000 7fcba9116700 1 -- 172.21.15.78:0/1160093410 shutdown_connections 2023-10-11T09:46:05.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.219+0000 7fcba9116700 1 -- 172.21.15.78:0/1160093410 wait complete. 2023-10-11T09:46:05.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.223+0000 7fcba9116700 1 Processor -- start 2023-10-11T09:46:05.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.223+0000 7fcba9116700 1 -- start start 2023-10-11T09:46:05.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.223+0000 7fcba9116700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcba40ca950 0x7fcba40b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:05.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.223+0000 7fcba9116700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcba415a940 0x7fcba41f7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:05.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.223+0000 7fcba9116700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcba415b8e0 0x7fcba413f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:05.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.223+0000 7fcba9116700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba40b40f0 con 0x7fcba40ca950 2023-10-11T09:46:05.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.223+0000 7fcba9116700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba40b1a10 con 0x7fcba415a940 2023-10-11T09:46:05.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.223+0000 7fcba9116700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba415a400 con 0x7fcba415b8e0 2023-10-11T09:46:05.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.223+0000 7fcba3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcba40ca950 0x7fcba40b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:05.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.223+0000 7fcba3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcba40ca950 0x7fcba40b33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43592/0 (socket says 172.21.15.78:43592) 2023-10-11T09:46:05.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.223+0000 7fcba3fff700 1 -- 172.21.15.78:0/2231099971 learned_addr learned my addr 172.21.15.78:0/2231099971 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:46:05.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.223+0000 7fcba8915700 1 --2- 172.21.15.78:0/2231099971 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcba415b8e0 0x7fcba413f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:05.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.223+0000 7fcba37fe700 1 --2- 172.21.15.78:0/2231099971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcba415a940 0x7fcba41f7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:05.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.227+0000 7fcba3fff700 1 -- 172.21.15.78:0/2231099971 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcba415b8e0 msgr2=0x7fcba413f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:05.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.227+0000 7fcba3fff700 1 --2- 172.21.15.78:0/2231099971 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcba415b8e0 0x7fcba413f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.227+0000 7fcba3fff700 1 -- 172.21.15.78:0/2231099971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcba415a940 msgr2=0x7fcba41f7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:05.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.227+0000 7fcba3fff700 1 --2- 172.21.15.78:0/2231099971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcba415a940 0x7fcba41f7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.227+0000 7fcba3fff700 1 -- 172.21.15.78:0/2231099971 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb9c009a70 con 0x7fcba40ca950 2023-10-11T09:46:05.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.227+0000 7fcba3fff700 1 --2- 172.21.15.78:0/2231099971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcba40ca950 0x7fcba40b33c0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fcb9400c4d0 tx=0x7fcb9400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:05.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.227+0000 7fcba0ff9700 1 -- 172.21.15.78:0/2231099971 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb94005d90 con 0x7fcba40ca950 2023-10-11T09:46:05.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.227+0000 7fcba0ff9700 1 -- 172.21.15.78:0/2231099971 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb940070d0 con 0x7fcba40ca950 2023-10-11T09:46:05.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.227+0000 7fcba9116700 1 -- 172.21.15.78:0/2231099971 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba40b3b30 con 0x7fcba40ca950 2023-10-11T09:46:05.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.227+0000 7fcba0ff9700 1 -- 172.21.15.78:0/2231099971 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb94007440 con 0x7fcba40ca950 2023-10-11T09:46:05.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.227+0000 7fcba9116700 1 -- 172.21.15.78:0/2231099971 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba40b1c20 con 0x7fcba40ca950 2023-10-11T09:46:05.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.231+0000 7fcba0ff9700 1 -- 172.21.15.78:0/2231099971 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fcb940075a0 con 0x7fcba40ca950 2023-10-11T09:46:05.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.231+0000 7fcba0ff9700 1 --2- 172.21.15.78:0/2231099971 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fcb90065ed0 0x7fcb90068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:05.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.231+0000 7fcba0ff9700 1 -- 172.21.15.78:0/2231099971 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6724+0+0 (secure 0 0 0) 0x7fcb94082db0 con 0x7fcba40ca950 2023-10-11T09:46:05.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.231+0000 7fcba37fe700 1 --2- 172.21.15.78:0/2231099971 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fcb90065ed0 0x7fcb90068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:05.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.231+0000 7fcba37fe700 1 --2- 172.21.15.78:0/2231099971 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fcb90065ed0 0x7fcb90068380 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcba41429b0 tx=0x7fcb98009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:05.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.231+0000 7fcbad600740 1 Processor -- start 2023-10-11T09:46:05.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.231+0000 7fcbad600740 1 -- start start 2023-10-11T09:46:05.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.235+0000 7fcbad600740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xcb6eb0 0xf6f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:05.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.235+0000 7fcbad600740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb6ae0 0xf6f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:05.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.235+0000 7fcbad600740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xcb6710 0xf36640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:05.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.235+0000 7fcbad600740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0xe964a0 con 0xcb6ae0 2023-10-11T09:46:05.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.235+0000 7fcbad600740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0xe93c30 con 0xcb6eb0 2023-10-11T09:46:05.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.235+0000 7fcbad600740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0xe8f520 con 0xcb6710 2023-10-11T09:46:05.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.235+0000 7fcba37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb6ae0 0xf6f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:05.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.235+0000 7fcba37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb6ae0 0xf6f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43606/0 (socket says 172.21.15.78:43606) 2023-10-11T09:46:05.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.235+0000 7fcba37fe700 1 -- 172.21.15.78:0/1666203265 learned_addr learned my addr 172.21.15.78:0/1666203265 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:46:05.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.235+0000 7fcba3fff700 1 --2- 172.21.15.78:0/1666203265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xcb6eb0 0xf6f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:05.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.235+0000 7fcba8915700 1 --2- 172.21.15.78:0/1666203265 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xcb6710 0xf36640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:05.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.235+0000 7fcba37fe700 1 -- 172.21.15.78:0/1666203265 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xcb6710 msgr2=0xf36640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:05.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.235+0000 7fcba37fe700 1 --2- 172.21.15.78:0/1666203265 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xcb6710 0xf36640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.235+0000 7fcba37fe700 1 -- 172.21.15.78:0/1666203265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xcb6eb0 msgr2=0xf6f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:05.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.235+0000 7fcba37fe700 1 --2- 172.21.15.78:0/1666203265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xcb6eb0 0xf6f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.235+0000 7fcba37fe700 1 -- 172.21.15.78:0/1666203265 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xd029e0 con 0xcb6ae0 2023-10-11T09:46:05.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.235+0000 7fcba37fe700 1 --2- 172.21.15.78:0/1666203265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb6ae0 0xf6f560 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fcb980052e0 tx=0x7fcb98011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef5de55d3d86ab8c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:05.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.235+0000 7fcb89ffb700 1 -- 172.21.15.78:0/1666203265 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb98003d40 con 0xcb6ae0 2023-10-11T09:46:05.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.235+0000 7fcb89ffb700 1 -- 172.21.15.78:0/1666203265 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb98021450 con 0xcb6ae0 2023-10-11T09:46:05.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.235+0000 7fcb89ffb700 1 -- 172.21.15.78:0/1666203265 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb98003ea0 con 0xcb6ae0 2023-10-11T09:46:05.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.235+0000 7fcbad600740 1 -- 172.21.15.78:0/1666203265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb6ae0 msgr2=0xf6f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:05.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.235+0000 7fcbad600740 1 --2- 172.21.15.78:0/1666203265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb6ae0 0xf6f560 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fcb980052e0 tx=0x7fcb98011000 comp rx=0 tx=0).stop 2023-10-11T09:46:05.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.235+0000 7fcbad600740 1 -- 172.21.15.78:0/1666203265 shutdown_connections 2023-10-11T09:46:05.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.235+0000 7fcbad600740 1 --2- 172.21.15.78:0/1666203265 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xcb6710 0xf36640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.235+0000 7fcbad600740 1 --2- 172.21.15.78:0/1666203265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb6ae0 0xf6f560 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.235+0000 7fcbad600740 1 --2- 172.21.15.78:0/1666203265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xcb6eb0 0xf6f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.235+0000 7fcbad600740 1 -- 172.21.15.78:0/1666203265 >> 172.21.15.78:0/1666203265 conn(0xf1efa0 msgr2=0xf50420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:46:05.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.235+0000 7fcbad600740 1 -- 172.21.15.78:0/1666203265 shutdown_connections 2023-10-11T09:46:05.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.235+0000 7fcbad600740 1 -- 172.21.15.78:0/1666203265 wait complete. 2023-10-11T09:46:05.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.239+0000 7fcbad600740 1 -- start start 2023-10-11T09:46:05.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.239+0000 7fcbad600740 1 Processor -- start 2023-10-11T09:46:05.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.239+0000 7fcbad600740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb6710 0xf656a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:05.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.239+0000 7fcbad600740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xcb6ae0 0xf77200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:05.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.239+0000 7fcbad600740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xcb6eb0 0xf77740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:05.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.239+0000 7fcbad600740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0xe52b00 con 0xcb6710 2023-10-11T09:46:05.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.239+0000 7fcbad600740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0xe527a0 con 0xcb6eb0 2023-10-11T09:46:05.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.239+0000 7fcbad600740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0xd3bb20 con 0xcb6ae0 2023-10-11T09:46:05.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.239+0000 7fcbad600740 10 client.0 _ll_register_callbacks cb 0xe88e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:46:05.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.239+0000 7fcba3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb6710 0xf656a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:05.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.239+0000 7fcba3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb6710 0xf656a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43620/0 (socket says 172.21.15.78:43620) 2023-10-11T09:46:05.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.239+0000 7fcba3fff700 1 -- 172.21.15.78:0/150049283 learned_addr learned my addr 172.21.15.78:0/150049283 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:46:05.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.239+0000 7fcba8915700 1 --2- 172.21.15.78:0/150049283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xcb6eb0 0xf77740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:05.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.239+0000 7fcba37fe700 1 --2- 172.21.15.78:0/150049283 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xcb6ae0 0xf77200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:05.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.239+0000 7fcba3fff700 1 -- 172.21.15.78:0/150049283 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xcb6ae0 msgr2=0xf77200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:05.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.239+0000 7fcba3fff700 1 --2- 172.21.15.78:0/150049283 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xcb6ae0 0xf77200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.239+0000 7fcba3fff700 1 -- 172.21.15.78:0/150049283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xcb6eb0 msgr2=0xf77740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:05.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.239+0000 7fcba3fff700 1 --2- 172.21.15.78:0/150049283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xcb6eb0 0xf77740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.239+0000 7fcba3fff700 1 -- 172.21.15.78:0/150049283 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb9800f040 con 0xcb6710 2023-10-11T09:46:05.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.239+0000 7fcba8915700 1 --2- 172.21.15.78:0/150049283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xcb6eb0 0xf77740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:46:05.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.239+0000 7fcba3fff700 1 --2- 172.21.15.78:0/150049283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb6710 0xf656a0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fcb9400cbe0 tx=0x7fcb94054410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12416ef342b10144 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:05.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.239+0000 7fcb82ffd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:46:05.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.239+0000 7fcb82ffd700 1 -- 172.21.15.78:0/150049283 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9405bb70 con 0xcb6710 2023-10-11T09:46:05.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.239+0000 7fcb82ffd700 1 -- 172.21.15.78:0/150049283 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 6724+0+0 (secure 0 0 0) 0x7fcb9405be00 con 0xcb6710 2023-10-11T09:46:05.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.239+0000 7fcbad600740 1 -- 172.21.15.78:0/150049283 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xe885a0 con 0xcb6710 2023-10-11T09:46:05.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.243+0000 7fcb82ffd700 1 -- 172.21.15.78:0/150049283 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb9406b170 con 0xcb6710 2023-10-11T09:46:05.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.243+0000 7fcbad600740 10 client.6933 fetch_fsmap learned FSMap version 50 2023-10-11T09:46:05.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.243+0000 7fcbad600740 1 -- 172.21.15.78:0/150049283 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0xe55750 con 0xcb6710 2023-10-11T09:46:05.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.243+0000 7fcb82ffd700 1 -- 172.21.15.78:0/150049283 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9406f080 con 0xcb6710 2023-10-11T09:46:05.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.243+0000 7fcb82ffd700 1 -- 172.21.15.78:0/150049283 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 50) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fcb9406b2d0 con 0xcb6710 2023-10-11T09:46:05.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.243+0000 7fcbad600740 10 client.6933 fetch_fsmap finished waiting for FSMap version 50 2023-10-11T09:46:05.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.243+0000 7fcbad600740 10 client.6933 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-11T09:46:05.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.243+0000 7fcbad600740 20 client.6933 populate_metadata read hostname 'smithi078' 2023-10-11T09:46:05.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.243+0000 7fcbad600740 1 --2- 172.21.15.78:0/150049283 >> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] conn(0xd01790 0xf81f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:05.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.243+0000 7fcbad600740 4 client.6933 mds_command: new command op to 6897 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-11T09:46:05.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.243+0000 7fcbad600740 1 -- 172.21.15.78:0/150049283 --> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0xe882c0 con 0xd01790 2023-10-11T09:46:05.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.243+0000 7fcba37fe700 1 --2- 172.21.15.78:0/150049283 >> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] conn(0xd01790 0xf81f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:05.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.243+0000 7fcba37fe700 1 --2- 172.21.15.78:0/150049283 >> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] conn(0xd01790 0xf81f90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9afd05fec737a33c server_cookie=dff1b99bedddf14b in_seq=0 out_seq=0 2023-10-11T09:46:05.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.243+0000 7fcb82ffd700 10 client.6933 ms_handle_connect on v2:172.21.15.78:6835/3498868661 2023-10-11T09:46:05.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.243+0000 7fcb82ffd700 1 -- 172.21.15.78:0/150049283 <== mds.0 v2:172.21.15.78:6835/3498868661 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0xe882c0 con 0xd01790 2023-10-11T09:46:05.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.243+0000 7fcb82ffd700 10 client.6933 handle_command_reply: tid=0 2023-10-11T09:46:05.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.243+0000 7fcbad600740 1 client.6933 shutdown 2023-10-11T09:46:05.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.247+0000 7fcbad600740 1 -- 172.21.15.78:0/150049283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb6710 msgr2=0xf656a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:05.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.247+0000 7fcbad600740 1 --2- 172.21.15.78:0/150049283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb6710 0xf656a0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fcb9400cbe0 tx=0x7fcb94054410 comp rx=0 tx=0).stop 2023-10-11T09:46:05.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.247+0000 7fcbad600740 1 -- 172.21.15.78:0/150049283 shutdown_connections 2023-10-11T09:46:05.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.247+0000 7fcbad600740 1 --2- 172.21.15.78:0/150049283 >> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] conn(0xd01790 0xf81f90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.247+0000 7fcbad600740 1 --2- 172.21.15.78:0/150049283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xcb6eb0 0xf77740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.247+0000 7fcbad600740 1 --2- 172.21.15.78:0/150049283 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xcb6ae0 0xf77200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.247+0000 7fcbad600740 1 --2- 172.21.15.78:0/150049283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb6710 0xf656a0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.247+0000 7fcb82ffd700 0 client.6933 ms_handle_reset on v2:172.21.15.78:6835/3498868661 2023-10-11T09:46:05.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.247+0000 7fcbad600740 1 -- 172.21.15.78:0/150049283 >> 172.21.15.78:0/150049283 conn(0xf1efa0 msgr2=0xf43b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:46:05.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.247+0000 7fcbad600740 1 -- 172.21.15.78:0/150049283 shutdown_connections 2023-10-11T09:46:05.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.247+0000 7fcbad600740 1 -- 172.21.15.78:0/150049283 wait complete. 2023-10-11T09:46:05.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.247+0000 7fcbad600740 20 client.6933 trim_cache size 0 max 16384 2023-10-11T09:46:05.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.263+0000 7fcbad600740 1 Processor -- start 2023-10-11T09:46:05.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.263+0000 7fcbad600740 1 -- start start 2023-10-11T09:46:05.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.267+0000 7fcbad600740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xcb6ae0 0xf7dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:05.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.267+0000 7fcbad600740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xcb6eb0 0xf7e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:05.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.267+0000 7fcbad600740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xf77750 0xf26080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:05.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.267+0000 7fcbad600740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0xf509d0 con 0xf77750 2023-10-11T09:46:05.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.267+0000 7fcbad600740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0xe49760 con 0xcb6eb0 2023-10-11T09:46:05.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.267+0000 7fcbad600740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0xe4cc90 con 0xcb6ae0 2023-10-11T09:46:05.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.267+0000 7fcba8915700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xf77750 0xf26080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:05.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.267+0000 7fcba8915700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xf77750 0xf26080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43622/0 (socket says 172.21.15.78:43622) 2023-10-11T09:46:05.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.267+0000 7fcba8915700 1 -- 172.21.15.78:0/2824237521 learned_addr learned my addr 172.21.15.78:0/2824237521 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:46:05.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.267+0000 7fcba3fff700 1 --2- 172.21.15.78:0/2824237521 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xcb6ae0 0xf7dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:05.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.267+0000 7fcba37fe700 1 --2- 172.21.15.78:0/2824237521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xcb6eb0 0xf7e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:05.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.267+0000 7fcba8915700 1 -- 172.21.15.78:0/2824237521 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xcb6ae0 msgr2=0xf7dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:05.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.267+0000 7fcba8915700 1 --2- 172.21.15.78:0/2824237521 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xcb6ae0 0xf7dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.267+0000 7fcba8915700 1 -- 172.21.15.78:0/2824237521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xcb6eb0 msgr2=0xf7e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:05.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.267+0000 7fcba8915700 1 --2- 172.21.15.78:0/2824237521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xcb6eb0 0xf7e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.267+0000 7fcba8915700 1 -- 172.21.15.78:0/2824237521 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba40ca3c0 con 0xf77750 2023-10-11T09:46:05.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.267+0000 7fcba8915700 1 --2- 172.21.15.78:0/2824237521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xf77750 0xf26080 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fcba415efe0 tx=0x7fcb9c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69d1ad386241b09f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:05.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.267+0000 7fcb8a7fc700 1 -- 172.21.15.78:0/2824237521 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c02a070 con 0xf77750 2023-10-11T09:46:05.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.267+0000 7fcb8a7fc700 1 -- 172.21.15.78:0/2824237521 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9c0125e0 con 0xf77750 2023-10-11T09:46:05.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.267+0000 7fcb8a7fc700 1 -- 172.21.15.78:0/2824237521 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c012950 con 0xf77750 2023-10-11T09:46:05.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.267+0000 7fcbad600740 1 -- 172.21.15.78:0/2824237521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xf77750 msgr2=0xf26080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:05.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.267+0000 7fcbad600740 1 --2- 172.21.15.78:0/2824237521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xf77750 0xf26080 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fcba415efe0 tx=0x7fcb9c010000 comp rx=0 tx=0).stop 2023-10-11T09:46:05.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.267+0000 7fcbad600740 1 -- 172.21.15.78:0/2824237521 shutdown_connections 2023-10-11T09:46:05.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.267+0000 7fcbad600740 1 --2- 172.21.15.78:0/2824237521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xf77750 0xf26080 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.267+0000 7fcbad600740 1 --2- 172.21.15.78:0/2824237521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xcb6eb0 0xf7e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.267+0000 7fcbad600740 1 --2- 172.21.15.78:0/2824237521 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xcb6ae0 0xf7dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.267+0000 7fcbad600740 1 -- 172.21.15.78:0/2824237521 >> 172.21.15.78:0/2824237521 conn(0xf1efa0 msgr2=0xf861c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:46:05.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.267+0000 7fcbad600740 1 -- 172.21.15.78:0/2824237521 shutdown_connections 2023-10-11T09:46:05.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.267+0000 7fcbad600740 1 -- 172.21.15.78:0/2824237521 wait complete. 2023-10-11T09:46:05.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.271+0000 7fcbad600740 1 -- start start 2023-10-11T09:46:05.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.271+0000 7fcbad600740 1 Processor -- start 2023-10-11T09:46:05.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.271+0000 7fcbad600740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xcb6ae0 0xf4c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:05.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.271+0000 7fcbad600740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb6eb0 0xf4ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:05.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.271+0000 7fcbad600740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xf77750 0xf25260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:05.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.271+0000 7fcbad600740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0xf29cd0 con 0xcb6eb0 2023-10-11T09:46:05.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.271+0000 7fcbad600740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0xe545c0 con 0xf77750 2023-10-11T09:46:05.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.271+0000 7fcbad600740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0xe5ecd0 con 0xcb6ae0 2023-10-11T09:46:05.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.271+0000 7fcbad600740 10 client.0 _ll_register_callbacks cb 0xe8dc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:46:05.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.271+0000 7fcba37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb6eb0 0xf4ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:05.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.271+0000 7fcba37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb6eb0 0xf4ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43626/0 (socket says 172.21.15.78:43626) 2023-10-11T09:46:05.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.271+0000 7fcba37fe700 1 -- 172.21.15.78:0/1024081237 learned_addr learned my addr 172.21.15.78:0/1024081237 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:46:05.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.271+0000 7fcba37fe700 1 -- 172.21.15.78:0/1024081237 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xcb6ae0 msgr2=0xf4c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:05.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.271+0000 7fcba8915700 1 --2- 172.21.15.78:0/1024081237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xf77750 0xf25260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:05.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.271+0000 7fcba3fff700 1 --2- 172.21.15.78:0/1024081237 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xcb6ae0 0xf4c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:05.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.271+0000 7fcba37fe700 1 --2- 172.21.15.78:0/1024081237 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xcb6ae0 0xf4c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.271+0000 7fcba37fe700 1 -- 172.21.15.78:0/1024081237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xf77750 msgr2=0xf25260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:05.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.271+0000 7fcba37fe700 1 --2- 172.21.15.78:0/1024081237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xf77750 0xf25260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.271+0000 7fcba37fe700 1 -- 172.21.15.78:0/1024081237 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba415e840 con 0xcb6eb0 2023-10-11T09:46:05.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.271+0000 7fcba8915700 1 --2- 172.21.15.78:0/1024081237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xf77750 0xf25260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:46:05.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.271+0000 7fcba37fe700 1 --2- 172.21.15.78:0/1024081237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb6eb0 0xf4ca30 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fcb980011e0 tx=0x7fcb98000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf446055473dc13 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:05.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.271+0000 7fcb82ffd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:46:05.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.271+0000 7fcb82ffd700 1 -- 172.21.15.78:0/1024081237 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb98028040 con 0xcb6eb0 2023-10-11T09:46:05.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.271+0000 7fcb82ffd700 1 -- 172.21.15.78:0/1024081237 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 6724+0+0 (secure 0 0 0) 0x7fcb980217d0 con 0xcb6eb0 2023-10-11T09:46:05.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.271+0000 7fcbad600740 1 -- 172.21.15.78:0/1024081237 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb98021450 con 0xcb6eb0 2023-10-11T09:46:05.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.271+0000 7fcb82ffd700 1 -- 172.21.15.78:0/1024081237 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb98021990 con 0xcb6eb0 2023-10-11T09:46:05.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.271+0000 7fcbad600740 10 client.6939 fetch_fsmap learned FSMap version 50 2023-10-11T09:46:05.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.271+0000 7fcbad600740 1 -- 172.21.15.78:0/1024081237 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0xe3b980 con 0xcb6eb0 2023-10-11T09:46:05.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.275+0000 7fcb82ffd700 1 -- 172.21.15.78:0/1024081237 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb98021450 con 0xcb6eb0 2023-10-11T09:46:05.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.275+0000 7fcb82ffd700 1 -- 172.21.15.78:0/1024081237 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 50) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fcb98021af0 con 0xcb6eb0 2023-10-11T09:46:05.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.275+0000 7fcbad600740 10 client.6939 fetch_fsmap finished waiting for FSMap version 50 2023-10-11T09:46:05.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.275+0000 7fcbad600740 10 client.6939 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-11T09:46:05.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.275+0000 7fcbad600740 20 client.6939 populate_metadata read hostname 'smithi078' 2023-10-11T09:46:05.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.275+0000 7fcbad600740 1 --2- 172.21.15.78:0/1024081237 >> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] conn(0xf25930 0xf25d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:05.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.275+0000 7fcbad600740 4 client.6939 mds_command: new command op to 6897 tid=0[{"prefix": "status"}] 2023-10-11T09:46:05.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.275+0000 7fcbad600740 1 -- 172.21.15.78:0/1024081237 --> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] -- command(tid 0: {"prefix": "status"}) v1 -- 0xd0e4e0 con 0xf25930 2023-10-11T09:46:05.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.275+0000 7fcba3fff700 1 --2- 172.21.15.78:0/1024081237 >> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] conn(0xf25930 0xf25d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:05.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.275+0000 7fcba3fff700 1 --2- 172.21.15.78:0/1024081237 >> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] conn(0xf25930 0xf25d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3a423decbd302b server_cookie=f7ec2aab841de884 in_seq=0 out_seq=0 2023-10-11T09:46:05.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.275+0000 7fcb82ffd700 10 client.6939 ms_handle_connect on v2:172.21.15.78:6835/3498868661 2023-10-11T09:46:05.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.275+0000 7fcb82ffd700 1 -- 172.21.15.78:0/1024081237 <== mds.0 v2:172.21.15.78:6835/3498868661 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0xd0e4e0 con 0xf25930 2023-10-11T09:46:05.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.275+0000 7fcb82ffd700 10 client.6939 handle_command_reply: tid=0 2023-10-11T09:46:05.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.275+0000 7fcbad600740 1 client.6939 shutdown 2023-10-11T09:46:05.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.275+0000 7fcbad600740 1 -- 172.21.15.78:0/1024081237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb6eb0 msgr2=0xf4ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:05.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.275+0000 7fcbad600740 1 --2- 172.21.15.78:0/1024081237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb6eb0 0xf4ca30 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fcb980011e0 tx=0x7fcb98000ed0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.275+0000 7fcbad600740 1 -- 172.21.15.78:0/1024081237 shutdown_connections 2023-10-11T09:46:05.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.275+0000 7fcbad600740 1 --2- 172.21.15.78:0/1024081237 >> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] conn(0xf25930 0xf25d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.275+0000 7fcbad600740 1 --2- 172.21.15.78:0/1024081237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xf77750 0xf25260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.275+0000 7fcbad600740 1 --2- 172.21.15.78:0/1024081237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb6eb0 0xf4ca30 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.275+0000 7fcbad600740 1 --2- 172.21.15.78:0/1024081237 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xcb6ae0 0xf4c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.275+0000 7fcb82ffd700 0 client.6939 ms_handle_reset on v2:172.21.15.78:6835/3498868661 2023-10-11T09:46:05.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.275+0000 7fcbad600740 1 -- 172.21.15.78:0/1024081237 >> 172.21.15.78:0/1024081237 conn(0xf1efa0 msgr2=0xf29230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:46:05.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.275+0000 7fcbad600740 1 -- 172.21.15.78:0/1024081237 shutdown_connections 2023-10-11T09:46:05.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.275+0000 7fcbad600740 1 -- 172.21.15.78:0/1024081237 wait complete. 2023-10-11T09:46:05.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.275+0000 7fcbad600740 20 client.6939 trim_cache size 0 max 16384 2023-10-11T09:46:05.294 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:46:05.295 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "dee99db9-7a72-406f-a1e2-917943f2b13d", 2023-10-11T09:46:05.295 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2023-10-11T09:46:05.295 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 6897, 2023-10-11T09:46:05.295 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2023-10-11T09:46:05.295 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2023-10-11T09:46:05.295 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2023-10-11T09:46:05.296 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 4.6177248449999997, 2023-10-11T09:46:05.296 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 50, 2023-10-11T09:46:05.296 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 70, 2023-10-11T09:46:05.296 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 70, 2023-10-11T09:46:05.297 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 4.6477537529999999 2023-10-11T09:46:05.297 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:46:05.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.279+0000 7fcba9116700 1 -- 172.21.15.78:0/2231099971 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fcb90065ed0 msgr2=0x7fcb90068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:05.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.279+0000 7fcba9116700 1 --2- 172.21.15.78:0/2231099971 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fcb90065ed0 0x7fcb90068380 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcba41429b0 tx=0x7fcb98009040 comp rx=0 tx=0).stop 2023-10-11T09:46:05.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.279+0000 7fcba9116700 1 -- 172.21.15.78:0/2231099971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcba40ca950 msgr2=0x7fcba40b33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:05.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.279+0000 7fcba9116700 1 --2- 172.21.15.78:0/2231099971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcba40ca950 0x7fcba40b33c0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fcb9400c4d0 tx=0x7fcb9400c7e0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.279+0000 7fcba9116700 1 -- 172.21.15.78:0/2231099971 shutdown_connections 2023-10-11T09:46:05.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.279+0000 7fcba9116700 1 --2- 172.21.15.78:0/2231099971 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fcb90065ed0 0x7fcb90068380 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.279+0000 7fcba9116700 1 --2- 172.21.15.78:0/2231099971 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcba415b8e0 0x7fcba413f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.279+0000 7fcba9116700 1 --2- 172.21.15.78:0/2231099971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcba415a940 0x7fcba41f7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.279+0000 7fcba9116700 1 --2- 172.21.15.78:0/2231099971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcba40ca950 0x7fcba40b33c0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.279+0000 7fcba9116700 1 -- 172.21.15.78:0/2231099971 >> 172.21.15.78:0/2231099971 conn(0x7fcba40cea60 msgr2=0x7fcba406d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:46:05.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.279+0000 7fcba9116700 1 -- 172.21.15.78:0/2231099971 shutdown_connections 2023-10-11T09:46:05.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.279+0000 7fcba9116700 1 -- 172.21.15.78:0/2231099971 wait complete. 2023-10-11T09:46:05.305 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:46:05.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.471+0000 7fbdd3137700 1 Processor -- start 2023-10-11T09:46:05.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.471+0000 7fbdd3137700 1 -- start start 2023-10-11T09:46:05.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.471+0000 7fbdd3137700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fbdcc15d050 0x7fbdcc0cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:05.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.471+0000 7fbdd3137700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbdcc160c60 0x7fbdcc15d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:05.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.471+0000 7fbdd3137700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdcc160210 con 0x7fbdcc15d960 2023-10-11T09:46:05.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.471+0000 7fbdd3137700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc0ccd60 con 0x7fbdcc15d050 2023-10-11T09:46:05.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.471+0000 7fbdd3137700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc0ccfc0 con 0x7fbdcc160c60 2023-10-11T09:46:05.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.471+0000 7fbdd2135700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fbdcc15d050 0x7fbdcc0cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:05.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.471+0000 7fbdd2135700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fbdcc15d050 0x7fbdcc0cc820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43640/0 (socket says 172.21.15.78:43640) 2023-10-11T09:46:05.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.471+0000 7fbdd2135700 1 -- 172.21.15.78:0/3112471363 learned_addr learned my addr 172.21.15.78:0/3112471363 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:46:05.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.471+0000 7fbdd1934700 1 --2- 172.21.15.78:0/3112471363 >> v2:172.21.15.133:3300/0 conn(0x7fbdcc160c60 0x7fbdcc15d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:05.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.471+0000 7fbdd2135700 1 -- 172.21.15.78:0/3112471363 >> v2:172.21.15.133:3300/0 conn(0x7fbdcc160c60 msgr2=0x7fbdcc15d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:05.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.471+0000 7fbdd2135700 1 --2- 172.21.15.78:0/3112471363 >> v2:172.21.15.133:3300/0 conn(0x7fbdcc160c60 0x7fbdcc15d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.471+0000 7fbdd2135700 1 -- 172.21.15.78:0/3112471363 >> v1:172.21.15.133:6789/0 conn(0x7fbdcc15d960 legacy=0x7fbdcc0b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:05.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.471+0000 7fbdd2135700 1 -- 172.21.15.78:0/3112471363 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdcc0cd2a0 con 0x7fbdcc15d050 2023-10-11T09:46:05.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.475+0000 7fbdd2135700 1 --2- 172.21.15.78:0/3112471363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdcc15d050 0x7fbdcc0cc820 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc06e260 tx=0x7fbdc8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1168bae1a022351 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:05.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.475+0000 7fbdd1133700 1 -- 172.21.15.78:0/3112471363 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8004230 con 0x7fbdcc15d050 2023-10-11T09:46:05.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.475+0000 7fbdd1133700 1 -- 172.21.15.78:0/3112471363 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc8004390 con 0x7fbdcc15d050 2023-10-11T09:46:05.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.475+0000 7fbdd1133700 1 -- 172.21.15.78:0/3112471363 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc80036c0 con 0x7fbdcc15d050 2023-10-11T09:46:05.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.475+0000 7fbdd3137700 1 -- 172.21.15.78:0/3112471363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdcc15d050 msgr2=0x7fbdcc0cc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:05.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.475+0000 7fbdd3137700 1 --2- 172.21.15.78:0/3112471363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdcc15d050 0x7fbdcc0cc820 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc06e260 tx=0x7fbdc8009bf0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.475+0000 7fbdd3137700 1 -- 172.21.15.78:0/3112471363 shutdown_connections 2023-10-11T09:46:05.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.475+0000 7fbdd3137700 1 --2- 172.21.15.78:0/3112471363 >> v2:172.21.15.133:3300/0 conn(0x7fbdcc160c60 0x7fbdcc15d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.475+0000 7fbdd3137700 1 --2- 172.21.15.78:0/3112471363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdcc15d050 0x7fbdcc0cc820 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.475+0000 7fbdd3137700 1 -- 172.21.15.78:0/3112471363 >> 172.21.15.78:0/3112471363 conn(0x7fbdcc0cea60 msgr2=0x7fbdcc0cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:46:05.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.475+0000 7fbdd3137700 1 -- 172.21.15.78:0/3112471363 shutdown_connections 2023-10-11T09:46:05.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.475+0000 7fbdd3137700 1 -- 172.21.15.78:0/3112471363 wait complete. 2023-10-11T09:46:05.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.479+0000 7fbdd3137700 1 Processor -- start 2023-10-11T09:46:05.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.479+0000 7fbdd3137700 1 -- start start 2023-10-11T09:46:05.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.479+0000 7fbdd3137700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdcc15d050 0x7fbdcc0b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:05.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.479+0000 7fbdd3137700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbdcc15d960 0x7fbdcc0b4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:05.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.479+0000 7fbdd3137700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbdcc160c60 0x7fbdcc147370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:05.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.479+0000 7fbdd3137700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc0b2650 con 0x7fbdcc15d050 2023-10-11T09:46:05.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.479+0000 7fbdd3137700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc0b2930 con 0x7fbdcc160c60 2023-10-11T09:46:05.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.479+0000 7fbdd3137700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc0b2c10 con 0x7fbdcc15d960 2023-10-11T09:46:05.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.479+0000 7fbdd2135700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdcc15d050 0x7fbdcc0b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:05.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.479+0000 7fbdd2135700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdcc15d050 0x7fbdcc0b39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43652/0 (socket says 172.21.15.78:43652) 2023-10-11T09:46:05.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.479+0000 7fbdd2135700 1 -- 172.21.15.78:0/1564118410 learned_addr learned my addr 172.21.15.78:0/1564118410 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:46:05.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.479+0000 7fbdd2936700 1 --2- 172.21.15.78:0/1564118410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbdcc160c60 0x7fbdcc147370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:05.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.479+0000 7fbdd1934700 1 --2- 172.21.15.78:0/1564118410 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbdcc15d960 0x7fbdcc0b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:05.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.479+0000 7fbdd2135700 1 -- 172.21.15.78:0/1564118410 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbdcc15d960 msgr2=0x7fbdcc0b4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:05.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.479+0000 7fbdd2135700 1 --2- 172.21.15.78:0/1564118410 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbdcc15d960 0x7fbdcc0b4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.479+0000 7fbdd2135700 1 -- 172.21.15.78:0/1564118410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbdcc160c60 msgr2=0x7fbdcc147370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:05.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.479+0000 7fbdd2135700 1 --2- 172.21.15.78:0/1564118410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbdcc160c60 0x7fbdcc147370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.479+0000 7fbdd2135700 1 -- 172.21.15.78:0/1564118410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc8009570 con 0x7fbdcc15d050 2023-10-11T09:46:05.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.483+0000 7fbdd2135700 1 --2- 172.21.15.78:0/1564118410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdcc15d050 0x7fbdcc0b39d0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8004000 tx=0x7fbdc8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:05.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.483+0000 7fbdc2ffd700 1 -- 172.21.15.78:0/1564118410 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8011830 con 0x7fbdcc15d050 2023-10-11T09:46:05.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.483+0000 7fbdc2ffd700 1 -- 172.21.15.78:0/1564118410 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc80119b0 con 0x7fbdcc15d050 2023-10-11T09:46:05.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.483+0000 7fbdd3137700 1 -- 172.21.15.78:0/1564118410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdcc0b2ef0 con 0x7fbdcc15d050 2023-10-11T09:46:05.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.483+0000 7fbdc2ffd700 1 -- 172.21.15.78:0/1564118410 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8011d60 con 0x7fbdcc15d050 2023-10-11T09:46:05.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.483+0000 7fbdd3137700 1 -- 172.21.15.78:0/1564118410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdcc0b34b0 con 0x7fbdcc15d050 2023-10-11T09:46:05.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.483+0000 7fbdd3137700 1 -- 172.21.15.78:0/1564118410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdcc065d40 con 0x7fbdcc15d050 2023-10-11T09:46:05.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.483+0000 7fbdc2ffd700 1 -- 172.21.15.78:0/1564118410 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fbdc801b440 con 0x7fbdcc15d050 2023-10-11T09:46:05.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.487+0000 7fbdc2ffd700 1 --2- 172.21.15.78:0/1564118410 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbdb0065ed0 0x7fbdb0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:05.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.487+0000 7fbdc2ffd700 1 -- 172.21.15.78:0/1564118410 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6724+0+0 (secure 0 0 0) 0x7fbdc8085240 con 0x7fbdcc15d050 2023-10-11T09:46:05.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.487+0000 7fbdd1934700 1 --2- 172.21.15.78:0/1564118410 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbdb0065ed0 0x7fbdb0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:05.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.487+0000 7fbdd1934700 1 --2- 172.21.15.78:0/1564118410 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbdb0065ed0 0x7fbdb0068380 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc155000 tx=0x7fbdbc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:05.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.491+0000 7fbdc2ffd700 1 -- 172.21.15.78:0/1564118410 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fbdc8052660 con 0x7fbdcc15d050 2023-10-11T09:46:05.659 INFO:tasks.ceph.mds.a.smithi078.stderr:2023-10-11T09:46:05.655+0000 7f293682d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-11T09:46:05.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.703+0000 7fbdd3137700 1 -- 172.21.15.78:0/1564118410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbdcc0cb340 con 0x7fbdcc15d050 2023-10-11T09:46:05.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.707+0000 7fbdc2ffd700 1 -- 172.21.15.78:0/1564118410 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4655 (secure 0 0 0) 0x7fbdc8003e60 con 0x7fbdcc15d050 2023-10-11T09:46:05.711 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:46:05.711 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":50,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6664,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2877345875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2877345875},{"type":"v1","addr":"172.21.15.133:6833","nonce":2877345875}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":6667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/792033922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":792033922},{"type":"v1","addr":"172.21.15.133:6835","nonce":792033922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":6894,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/1229638118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":1229638118},{"type":"v1","addr":"172.21.15.78:6836","nonce":1229638118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:45:54.252958+0000","modified":"2023-10-11T09:46:03.678418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6897},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6897":{"gid":6897,"name":"a","rank":0,"incarnation":47,"state":"up:active","state_seq":4,"addr":"172.21.15.78:6837/3498868661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":3498868661},{"type":"v1","addr":"172.21.15.78:6837","nonce":3498868661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9,10],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-11T09:46:05.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.707+0000 7fbdd3137700 1 -- 172.21.15.78:0/1564118410 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbdb0065ed0 msgr2=0x7fbdb0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:05.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.707+0000 7fbdd3137700 1 --2- 172.21.15.78:0/1564118410 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbdb0065ed0 0x7fbdb0068380 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc155000 tx=0x7fbdbc008040 comp rx=0 tx=0).stop 2023-10-11T09:46:05.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.707+0000 7fbdd3137700 1 -- 172.21.15.78:0/1564118410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdcc15d050 msgr2=0x7fbdcc0b39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:05.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.707+0000 7fbdd3137700 1 --2- 172.21.15.78:0/1564118410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdcc15d050 0x7fbdcc0b39d0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8004000 tx=0x7fbdc8004770 comp rx=0 tx=0).stop 2023-10-11T09:46:05.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.707+0000 7fbdd3137700 1 -- 172.21.15.78:0/1564118410 shutdown_connections 2023-10-11T09:46:05.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.707+0000 7fbdd3137700 1 --2- 172.21.15.78:0/1564118410 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbdb0065ed0 0x7fbdb0068380 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.707+0000 7fbdd3137700 1 --2- 172.21.15.78:0/1564118410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbdcc160c60 0x7fbdcc147370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.707+0000 7fbdd3137700 1 --2- 172.21.15.78:0/1564118410 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbdcc15d960 0x7fbdcc0b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.707+0000 7fbdd3137700 1 --2- 172.21.15.78:0/1564118410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdcc15d050 0x7fbdcc0b39d0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.707+0000 7fbdd3137700 1 -- 172.21.15.78:0/1564118410 >> 172.21.15.78:0/1564118410 conn(0x7fbdcc0cea60 msgr2=0x7fbdcc15f200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:46:05.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.707+0000 7fbdd3137700 1 -- 172.21.15.78:0/1564118410 shutdown_connections 2023-10-11T09:46:05.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.707+0000 7fbdd3137700 1 -- 172.21.15.78:0/1564118410 wait complete. 2023-10-11T09:46:05.718 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 50 2023-10-11T09:46:05.735 INFO:tasks.cephfs.test_data_scan:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 3, "epoch": 50, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 3, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-11T09:45:54.252958+0000", "damaged": [], "data_pools": [ 9, 10 ], "enabled": true, "epoch": 50, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_6897": { "addr": "172.21.15.78:6837/3498868661", "addrs": { "addrvec": [ { "addr": "172.21.15.78:6835", "nonce": 3498868661, "type": "v2" }, { "addr": "172.21.15.78:6837", "nonce": 3498868661, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6897, "incarnation": 47, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 4 } }, "last_failure": 0, "last_failure_osd_epoch": 67, "max_file_size": 1099511627776, "max_mds": 1, "metadata_pool": 8, "modified": "2023-10-11T09:46:03.678418+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6897 } } } ], "standbys": [ { "addr": "172.21.15.133:6833/2877345875", "addrs": { "addrvec": [ { "addr": "172.21.15.133:6832", "nonce": 2877345875, "type": "v2" }, { "addr": "172.21.15.133:6833", "nonce": 2877345875, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 47, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6664, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.133:6835/792033922", "addrs": { "addrvec": [ { "addr": "172.21.15.133:6834", "nonce": 792033922, "type": "v2" }, { "addr": "172.21.15.133:6835", "nonce": 792033922, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 48, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6667, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.78:6836/1229638118", "addrs": { "addrvec": [ { "addr": "172.21.15.78:6834", "nonce": 1229638118, "type": "v2" }, { "addr": "172.21.15.78:6836", "nonce": 1229638118, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 48, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6894, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-11T09:46:05.735 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-11T09:46:05.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.899+0000 7fd0aa1df700 1 Processor -- start 2023-10-11T09:46:05.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.899+0000 7fd0aa1df700 1 -- start start 2023-10-11T09:46:05.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.903+0000 7fd0aa1df700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd0a4071040 0x7fd0a40cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:05.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.903+0000 7fd0aa1df700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd0a40cba00 0x7fd0a406c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:05.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.903+0000 7fd0aa1df700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0a40720f0 con 0x7fd0a4154e00 2023-10-11T09:46:05.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.903+0000 7fd0aa1df700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0a40714e0 con 0x7fd0a40cba00 2023-10-11T09:46:05.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.903+0000 7fd0aa1df700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a40717c0 con 0x7fd0a4071040 2023-10-11T09:46:05.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.903+0000 7fd0a37fe700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fd0a4154e00 0x7fd0a415b650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:39864/0 (socket says 172.21.15.78:39864) 2023-10-11T09:46:05.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.903+0000 7fd0a37fe700 1 -- 172.21.15.78:0/2713717206 learned_addr learned my addr 172.21.15.78:0/2713717206 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:46:05.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.903+0000 7fd0a3fff700 1 --2- 172.21.15.78:0/2713717206 >> v2:172.21.15.133:3300/0 conn(0x7fd0a40cba00 0x7fd0a406c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:05.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.903+0000 7fd0a2ffd700 1 --2- 172.21.15.78:0/2713717206 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd0a4071040 0x7fd0a40cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:05.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.903+0000 7fd0a27fc700 1 -- 172.21.15.78:0/2713717206 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2284865886 0 0) 0x7fd0a40720f0 con 0x7fd0a4154e00 2023-10-11T09:46:05.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.903+0000 7fd0a27fc700 1 -- 172.21.15.78:0/2713717206 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd08c0035d0 con 0x7fd0a4154e00 2023-10-11T09:46:05.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.903+0000 7fd0a3fff700 1 -- 172.21.15.78:0/2713717206 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd0a4071040 msgr2=0x7fd0a40cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:05.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.903+0000 7fd0a3fff700 1 --2- 172.21.15.78:0/2713717206 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd0a4071040 0x7fd0a40cb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.907+0000 7fd0a3fff700 1 -- 172.21.15.78:0/2713717206 >> v1:172.21.15.78:6789/0 conn(0x7fd0a4154e00 legacy=0x7fd0a415b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:05.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.907+0000 7fd0a3fff700 1 -- 172.21.15.78:0/2713717206 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a4071aa0 con 0x7fd0a40cba00 2023-10-11T09:46:05.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.907+0000 7fd0a3fff700 1 --2- 172.21.15.78:0/2713717206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0a40cba00 0x7fd0a406c010 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd09c009d90 tx=0x7fd09c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=deb8f8c408c7eb40 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:05.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.907+0000 7fd0a27fc700 1 -- 172.21.15.78:0/2713717206 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09c00e070 con 0x7fd0a40cba00 2023-10-11T09:46:05.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.907+0000 7fd0a27fc700 1 -- 172.21.15.78:0/2713717206 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd09c00bb20 con 0x7fd0a40cba00 2023-10-11T09:46:05.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.907+0000 7fd0a27fc700 1 -- 172.21.15.78:0/2713717206 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09c012430 con 0x7fd0a40cba00 2023-10-11T09:46:05.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.907+0000 7fd0aa1df700 1 -- 172.21.15.78:0/2713717206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0a40cba00 msgr2=0x7fd0a406c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:05.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.907+0000 7fd0aa1df700 1 --2- 172.21.15.78:0/2713717206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0a40cba00 0x7fd0a406c010 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd09c009d90 tx=0x7fd09c00b330 comp rx=0 tx=0).stop 2023-10-11T09:46:05.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.907+0000 7fd0aa1df700 1 -- 172.21.15.78:0/2713717206 shutdown_connections 2023-10-11T09:46:05.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.907+0000 7fd0aa1df700 1 --2- 172.21.15.78:0/2713717206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0a40cba00 0x7fd0a406c010 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.907+0000 7fd0aa1df700 1 --2- 172.21.15.78:0/2713717206 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd0a4071040 0x7fd0a40cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.907+0000 7fd0aa1df700 1 -- 172.21.15.78:0/2713717206 >> 172.21.15.78:0/2713717206 conn(0x7fd0a406aca0 msgr2=0x7fd0a40b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:46:05.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.907+0000 7fd0aa1df700 1 -- 172.21.15.78:0/2713717206 shutdown_connections 2023-10-11T09:46:05.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.907+0000 7fd0aa1df700 1 -- 172.21.15.78:0/2713717206 wait complete. 2023-10-11T09:46:05.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.911+0000 7fd0aa1df700 1 Processor -- start 2023-10-11T09:46:05.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.911+0000 7fd0aa1df700 1 -- start start 2023-10-11T09:46:05.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.911+0000 7fd0aa1df700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0a4071040 0x7fd0a414c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:05.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.911+0000 7fd0aa1df700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0a40cba00 0x7fd0a41f11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:05.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.911+0000 7fd0aa1df700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd0a4154e00 0x7fd0a40afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:05.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.911+0000 7fd0aa1df700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a415ab50 con 0x7fd0a40cba00 2023-10-11T09:46:05.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.911+0000 7fd0aa1df700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a414cd40 con 0x7fd0a4071040 2023-10-11T09:46:05.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.911+0000 7fd0aa1df700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a41553c0 con 0x7fd0a4154e00 2023-10-11T09:46:05.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.911+0000 7fd0a2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0a40cba00 0x7fd0a41f11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:05.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.911+0000 7fd0a2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0a40cba00 0x7fd0a41f11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43654/0 (socket says 172.21.15.78:43654) 2023-10-11T09:46:05.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.911+0000 7fd0a2ffd700 1 -- 172.21.15.78:0/2689086389 learned_addr learned my addr 172.21.15.78:0/2689086389 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:46:05.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.911+0000 7fd0a37fe700 1 --2- 172.21.15.78:0/2689086389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0a4071040 0x7fd0a414c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:05.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.911+0000 7fd0a3fff700 1 --2- 172.21.15.78:0/2689086389 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd0a4154e00 0x7fd0a40afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:05.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.911+0000 7fd0a2ffd700 1 -- 172.21.15.78:0/2689086389 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd0a4154e00 msgr2=0x7fd0a40afaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:05.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.911+0000 7fd0a2ffd700 1 --2- 172.21.15.78:0/2689086389 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd0a4154e00 0x7fd0a40afaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.911+0000 7fd0a2ffd700 1 -- 172.21.15.78:0/2689086389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0a4071040 msgr2=0x7fd0a414c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:05.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.911+0000 7fd0a2ffd700 1 --2- 172.21.15.78:0/2689086389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0a4071040 0x7fd0a414c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:05.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.911+0000 7fd0a2ffd700 1 -- 172.21.15.78:0/2689086389 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd09c009a70 con 0x7fd0a40cba00 2023-10-11T09:46:05.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.911+0000 7fd0a2ffd700 1 --2- 172.21.15.78:0/2689086389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0a40cba00 0x7fd0a41f11e0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fd09800e9a0 tx=0x7fd09800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:05.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.915+0000 7fd08bfff700 1 -- 172.21.15.78:0/2689086389 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd098009720 con 0x7fd0a40cba00 2023-10-11T09:46:05.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.915+0000 7fd08bfff700 1 -- 172.21.15.78:0/2689086389 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd09800c690 con 0x7fd0a40cba00 2023-10-11T09:46:05.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.915+0000 7fd0aa1df700 1 -- 172.21.15.78:0/2689086389 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0a4155660 con 0x7fd0a40cba00 2023-10-11T09:46:05.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.915+0000 7fd08bfff700 1 -- 172.21.15.78:0/2689086389 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09800ca40 con 0x7fd0a40cba00 2023-10-11T09:46:05.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.915+0000 7fd0aa1df700 1 -- 172.21.15.78:0/2689086389 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0a4155c20 con 0x7fd0a40cba00 2023-10-11T09:46:05.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.915+0000 7fd0aa1df700 1 -- 172.21.15.78:0/2689086389 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0a404e530 con 0x7fd0a40cba00 2023-10-11T09:46:05.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.919+0000 7fd08bfff700 1 -- 172.21.15.78:0/2689086389 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fd09800cba0 con 0x7fd0a40cba00 2023-10-11T09:46:05.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.919+0000 7fd08bfff700 1 --2- 172.21.15.78:0/2689086389 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd090065ed0 0x7fd090068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:05.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.919+0000 7fd08bfff700 1 -- 172.21.15.78:0/2689086389 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6724+0+0 (secure 0 0 0) 0x7fd098083ae0 con 0x7fd0a40cba00 2023-10-11T09:46:05.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.919+0000 7fd0a37fe700 1 --2- 172.21.15.78:0/2689086389 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd090065ed0 0x7fd090068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:05.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.919+0000 7fd0a37fe700 1 --2- 172.21.15.78:0/2689086389 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd090065ed0 0x7fd090068380 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd094007fd0 tx=0x7fd094007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:05.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:05.927+0000 7fd08bfff700 1 -- 172.21.15.78:0/2689086389 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fd098050e90 con 0x7fd0a40cba00 2023-10-11T09:46:06.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:06.135+0000 7fd0aa1df700 1 -- 172.21.15.78:0/2689086389 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd0a415eec0 con 0x7fd0a40cba00 2023-10-11T09:46:06.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:06.139+0000 7fd08bfff700 1 -- 172.21.15.78:0/2689086389 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v50) v1 ==== 53+0+91 (secure 0 0 0) 0x7fd098021070 con 0x7fd0a40cba00 2023-10-11T09:46:06.143 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_extra ] 2023-10-11T09:46:06.145 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:06.139+0000 7fd0aa1df700 1 -- 172.21.15.78:0/2689086389 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd090065ed0 msgr2=0x7fd090068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:06.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:06.139+0000 7fd0aa1df700 1 --2- 172.21.15.78:0/2689086389 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd090065ed0 0x7fd090068380 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd094007fd0 tx=0x7fd094007de0 comp rx=0 tx=0).stop 2023-10-11T09:46:06.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:06.139+0000 7fd0aa1df700 1 -- 172.21.15.78:0/2689086389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0a40cba00 msgr2=0x7fd0a41f11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:06.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:06.139+0000 7fd0aa1df700 1 --2- 172.21.15.78:0/2689086389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0a40cba00 0x7fd0a41f11e0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fd09800e9a0 tx=0x7fd09800ecb0 comp rx=0 tx=0).stop 2023-10-11T09:46:06.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:06.139+0000 7fd0aa1df700 1 -- 172.21.15.78:0/2689086389 shutdown_connections 2023-10-11T09:46:06.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:06.139+0000 7fd0aa1df700 1 --2- 172.21.15.78:0/2689086389 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd090065ed0 0x7fd090068380 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:06.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:06.139+0000 7fd0aa1df700 1 --2- 172.21.15.78:0/2689086389 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd0a4154e00 0x7fd0a40afaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:06.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:06.139+0000 7fd0aa1df700 1 --2- 172.21.15.78:0/2689086389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0a40cba00 0x7fd0a41f11e0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:06.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:06.139+0000 7fd0aa1df700 1 --2- 172.21.15.78:0/2689086389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0a4071040 0x7fd0a414c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:06.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:06.139+0000 7fd0aa1df700 1 -- 172.21.15.78:0/2689086389 >> 172.21.15.78:0/2689086389 conn(0x7fd0a406aca0 msgr2=0x7fd0a415dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:46:06.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:06.139+0000 7fd0aa1df700 1 -- 172.21.15.78:0/2689086389 shutdown_connections 2023-10-11T09:46:06.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:06.139+0000 7fd0aa1df700 1 -- 172.21.15.78:0/2689086389 wait complete. 2023-10-11T09:46:06.166 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-11T09:46:06.167 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-11T09:46:06.167 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi078.front.sepia.ceph.com 2023-10-11T09:46:06.167 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-11T09:46:06.168 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-11T09:46:06.168 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-11T09:46:06.168 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-11T09:46:06.169 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-11T09:46:06.169 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-11T09:46:06.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:46:06.170 DEBUG:teuthology.orchestra.run.smithi078:> ip addr 2023-10-11T09:46:06.189 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-11T09:46:06.189 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-11T09:46:06.189 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2023-10-11T09:46:06.189 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:46:06.190 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2023-10-11T09:46:06.190 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:46:06.190 INFO:teuthology.orchestra.run.smithi078.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:46:06.190 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:22 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:46:06.190 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f0 2023-10-11T09:46:06.191 INFO:teuthology.orchestra.run.smithi078.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:46:06.191 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:23 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:46:06.191 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f1 2023-10-11T09:46:06.191 INFO:teuthology.orchestra.run.smithi078.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:46:06.191 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dc brd ff:ff:ff:ff:ff:ff 2023-10-11T09:46:06.191 INFO:teuthology.orchestra.run.smithi078.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-11T09:46:06.192 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dd brd ff:ff:ff:ff:ff:ff 2023-10-11T09:46:06.192 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.21.15.78/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-11T09:46:06.192 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft 42098sec preferred_lft 42098sec 2023-10-11T09:46:06.192 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::ec4:7aff:fe8f:d4dd/64 scope link 2023-10-11T09:46:06.192 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:46:06.193 INFO:teuthology.orchestra.run.smithi078.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-11T09:46:06.193 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 02:42:b6:e4:f8:0d brd ff:ff:ff:ff:ff:ff 2023-10-11T09:46:06.193 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-11T09:46:06.193 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:46:06.193 INFO:teuthology.orchestra.run.smithi078.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-11T09:46:06.193 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 7a:89:8a:a9:e1:23 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:46:06.194 INFO:teuthology.orchestra.run.smithi078.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-11T09:46:06.194 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:46:06.194 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::8ca:ebff:fea9:13f1/64 scope link 2023-10-11T09:46:06.194 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:46:06.194 INFO:teuthology.orchestra.run.smithi078.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-11T09:46:06.195 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 7a:89:8a:a9:e1:23 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-11T09:46:06.195 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::7889:8aff:fea9:e123/64 scope link 2023-10-11T09:46:06.195 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:46:06.195 INFO:teuthology.orchestra.run.smithi078.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-11T09:46:06.196 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether ca:ac:e8:27:51:0d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-11T09:46:06.196 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::c8ac:e8ff:fe27:510d/64 scope link 2023-10-11T09:46:06.196 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:46:06.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:46:06.197 DEBUG:teuthology.orchestra.run.smithi078:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-11T09:46:06.275 INFO:teuthology.orchestra.run.smithi078.stdout:1 2023-10-11T09:46:06.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:46:06.278 DEBUG:teuthology.orchestra.run.smithi078:> route 2023-10-11T09:46:06.338 INFO:teuthology.orchestra.run.smithi078.stdout:Kernel IP routing table 2023-10-11T09:46:06.339 INFO:teuthology.orchestra.run.smithi078.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-11T09:46:06.339 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-11T09:46:06.339 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-11T09:46:06.339 INFO:teuthology.orchestra.run.smithi078.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-11T09:46:06.340 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-10-11T09:46:06.340 INFO:teuthology.orchestra.run.smithi078.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-11T09:46:06.340 INFO:teuthology.orchestra.run.smithi078.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-11T09:46:06.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:46:06.342 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec bash -c ' 2023-10-11T09:46:06.342 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2023-10-11T09:46:06.342 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-11T09:46:06.343 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-11T09:46:06.343 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-11T09:46:06.343 DEBUG:teuthology.orchestra.run.smithi078:> ') 2023-10-11T09:46:06.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:46:06.476 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2023-10-11T09:46:06.493 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-11T09:46:06.493 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-11T09:46:06.494 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-11T09:46:06.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:46:06.495 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set brx.0 up 2023-10-11T09:46:06.577 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-11T09:46:06.577 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T09:46:06.578 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-11T09:46:06.635 INFO:teuthology.orchestra.run.smithi078.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-11T09:46:06.636 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T09:46:06.636 DEBUG:teuthology.orchestra.run.smithi078:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-11T09:46:06.699 DEBUG:teuthology.orchestra.run.smithi078:> sudo modprobe fuse 2023-10-11T09:46:06.780 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T09:46:06.781 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T09:46:06.855 INFO:teuthology.orchestra.run.smithi078.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T09:46:06.858 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T09:46:06.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:46:06.859 DEBUG:teuthology.orchestra.run.smithi078:> ls /sys/fs/fuse/connections 2023-10-11T09:46:06.917 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-11T09:46:06.918 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-11T09:46:06.964 DEBUG:teuthology.orchestra.run.smithi078:> sudo modprobe fuse 2023-10-11T09:46:06.998 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T09:46:06.999 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T09:46:07.078 INFO:teuthology.orchestra.run.smithi078.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T09:46:07.080 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T09:46:07.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:46:07.081 DEBUG:teuthology.orchestra.run.smithi078:> ls /sys/fs/fuse/connections 2023-10-11T09:46:07.098 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:2023-10-11T09:46:07.091+0000 7fd4246c5640 -1 init, newargv = 0x557086d84580 newargc=15 2023-10-11T09:46:07.099 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:ceph-fuse[51345]: starting ceph client 2023-10-11T09:46:07.110 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:ceph-fuse[51345]: starting fuse 2023-10-11T09:46:07.112 INFO:teuthology.orchestra.run.smithi078.stdout:49 2023-10-11T09:46:07.113 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2023-10-11T09:46:07.113 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:46:07.114 DEBUG:teuthology.orchestra.run.smithi078:> import glob 2023-10-11T09:46:07.114 DEBUG:teuthology.orchestra.run.smithi078:> import re 2023-10-11T09:46:07.115 DEBUG:teuthology.orchestra.run.smithi078:> import os 2023-10-11T09:46:07.115 DEBUG:teuthology.orchestra.run.smithi078:> import subprocess 2023-10-11T09:46:07.115 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:46:07.116 DEBUG:teuthology.orchestra.run.smithi078:> def _find_admin_socket(client_name): 2023-10-11T09:46:07.116 DEBUG:teuthology.orchestra.run.smithi078:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-11T09:46:07.117 DEBUG:teuthology.orchestra.run.smithi078:> files = glob.glob(asok_path) 2023-10-11T09:46:07.117 DEBUG:teuthology.orchestra.run.smithi078:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-11T09:46:07.117 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:46:07.118 DEBUG:teuthology.orchestra.run.smithi078:> # Given a non-glob path, it better be there 2023-10-11T09:46:07.118 DEBUG:teuthology.orchestra.run.smithi078:> if "*" not in asok_path: 2023-10-11T09:46:07.118 DEBUG:teuthology.orchestra.run.smithi078:> assert(len(files) == 1) 2023-10-11T09:46:07.119 DEBUG:teuthology.orchestra.run.smithi078:> return files[0] 2023-10-11T09:46:07.119 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:46:07.119 DEBUG:teuthology.orchestra.run.smithi078:> for f in files: 2023-10-11T09:46:07.120 DEBUG:teuthology.orchestra.run.smithi078:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-11T09:46:07.120 DEBUG:teuthology.orchestra.run.smithi078:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-11T09:46:07.121 DEBUG:teuthology.orchestra.run.smithi078:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-11T09:46:07.121 DEBUG:teuthology.orchestra.run.smithi078:> contents = proc_f.read() 2023-10-11T09:46:07.121 DEBUG:teuthology.orchestra.run.smithi078:> if mountpoint in contents: 2023-10-11T09:46:07.122 DEBUG:teuthology.orchestra.run.smithi078:> return f 2023-10-11T09:46:07.122 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-11T09:46:07.122 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:46:07.123 DEBUG:teuthology.orchestra.run.smithi078:> print(_find_admin_socket("client.0")) 2023-10-11T09:46:07.123 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:46:07.270 INFO:teuthology.orchestra.run.smithi078.stdout:/var/run/ceph/ceph-client.0.51345.asok 2023-10-11T09:46:07.455 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.51345.asok 2023-10-11T09:46:07.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:46:07.456 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.51345.asok status 2023-10-11T09:46:07.646 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:46:07.647 INFO:teuthology.orchestra.run.smithi078.stdout: "metadata": { 2023-10-11T09:46:07.647 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_sha1": "9a774796f0ecdd11a4b771de396e627aa1a187af", 2023-10-11T09:46:07.647 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_version": "ceph version 18.2.0-662-g9a774796 (9a774796f0ecdd11a4b771de396e627aa1a187af) reef (stable)", 2023-10-11T09:46:07.647 INFO:teuthology.orchestra.run.smithi078.stdout: "entity_id": "0", 2023-10-11T09:46:07.648 INFO:teuthology.orchestra.run.smithi078.stdout: "hostname": "smithi078", 2023-10-11T09:46:07.648 INFO:teuthology.orchestra.run.smithi078.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-11T09:46:07.648 INFO:teuthology.orchestra.run.smithi078.stdout: "pid": "51345", 2023-10-11T09:46:07.648 INFO:teuthology.orchestra.run.smithi078.stdout: "root": "/" 2023-10-11T09:46:07.648 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:46:07.649 INFO:teuthology.orchestra.run.smithi078.stdout: "dentry_count": 0, 2023-10-11T09:46:07.649 INFO:teuthology.orchestra.run.smithi078.stdout: "dentry_pinned_count": 0, 2023-10-11T09:46:07.649 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 6960, 2023-10-11T09:46:07.649 INFO:teuthology.orchestra.run.smithi078.stdout: "inst": { 2023-10-11T09:46:07.649 INFO:teuthology.orchestra.run.smithi078.stdout: "name": { 2023-10-11T09:46:07.649 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "client", 2023-10-11T09:46:07.650 INFO:teuthology.orchestra.run.smithi078.stdout: "num": 6960 2023-10-11T09:46:07.650 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:46:07.650 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": { 2023-10-11T09:46:07.650 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2023-10-11T09:46:07.650 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "192.168.0.1:0", 2023-10-11T09:46:07.651 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 2727670860 2023-10-11T09:46:07.651 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:46:07.651 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:46:07.651 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": { 2023-10-11T09:46:07.651 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2023-10-11T09:46:07.652 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "192.168.0.1:0", 2023-10-11T09:46:07.652 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 2727670860 2023-10-11T09:46:07.652 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:46:07.652 INFO:teuthology.orchestra.run.smithi078.stdout: "inst_str": "client.6960 192.168.0.1:0/2727670860", 2023-10-11T09:46:07.652 INFO:teuthology.orchestra.run.smithi078.stdout: "addr_str": "192.168.0.1:0/2727670860", 2023-10-11T09:46:07.652 INFO:teuthology.orchestra.run.smithi078.stdout: "inode_count": 1, 2023-10-11T09:46:07.653 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_epoch": 50, 2023-10-11T09:46:07.653 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_epoch": 70, 2023-10-11T09:46:07.653 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_epoch_barrier": 0, 2023-10-11T09:46:07.653 INFO:teuthology.orchestra.run.smithi078.stdout: "blocklisted": false, 2023-10-11T09:46:07.653 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs" 2023-10-11T09:46:07.653 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:46:07.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:46:07.661 DEBUG:teuthology.orchestra.run.smithi078:> stat --file-system '--printf=%T 2023-10-11T09:46:07.661 DEBUG:teuthology.orchestra.run.smithi078:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-11T09:46:07.681 INFO:teuthology.orchestra.run.smithi078.stdout:fuseblk 2023-10-11T09:46:07.682 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-11T09:46:07.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:46:07.682 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-11T09:46:07.757 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2023-10-11T09:46:07.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.975+0000 7fca1c881700 1 Processor -- start 2023-10-11T09:46:07.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.979+0000 7fca1c881700 1 -- start start 2023-10-11T09:46:07.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.979+0000 7fca1c881700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fca1806a770 0x7fca18165590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:07.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.979+0000 7fca1c881700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca180c5cb0 con 0x7fca181408d0 2023-10-11T09:46:07.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.979+0000 7fca1c881700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca180c5f90 con 0x7fca18143900 2023-10-11T09:46:07.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.979+0000 7fca1c881700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca1806e490 con 0x7fca1806a770 2023-10-11T09:46:07.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.979+0000 7fca16d9d700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fca181408d0 0x7fca1806dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:39872/0 (socket says 172.21.15.78:39872) 2023-10-11T09:46:07.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.983+0000 7fca16d9d700 1 -- 172.21.15.78:0/4188084363 learned_addr learned my addr 172.21.15.78:0/4188084363 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:46:07.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.983+0000 7fca1759e700 1 --2- 172.21.15.78:0/4188084363 >> v2:172.21.15.133:3300/0 conn(0x7fca1806a770 0x7fca18165590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:07.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.983+0000 7fca1659c700 1 -- 172.21.15.78:0/4188084363 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4254206535 0 0) 0x7fca180c5cb0 con 0x7fca181408d0 2023-10-11T09:46:07.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.983+0000 7fca1659c700 1 -- 172.21.15.78:0/4188084363 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca040035d0 con 0x7fca181408d0 2023-10-11T09:46:07.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.983+0000 7fca1659c700 1 -- 172.21.15.78:0/4188084363 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2948592392 0 0) 0x7fca180c5f90 con 0x7fca18143900 2023-10-11T09:46:07.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.983+0000 7fca1659c700 1 -- 172.21.15.78:0/4188084363 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca180c5cb0 con 0x7fca18143900 2023-10-11T09:46:07.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.983+0000 7fca1759e700 1 -- 172.21.15.78:0/4188084363 >> v1:172.21.15.133:6789/0 conn(0x7fca18143900 legacy=0x7fca1806a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:07.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.983+0000 7fca1659c700 1 -- 172.21.15.78:0/4188084363 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3903157076 0 0) 0x7fca040035d0 con 0x7fca181408d0 2023-10-11T09:46:07.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.983+0000 7fca1759e700 1 -- 172.21.15.78:0/4188084363 >> v1:172.21.15.78:6789/0 conn(0x7fca181408d0 legacy=0x7fca1806dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:07.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.983+0000 7fca1759e700 1 -- 172.21.15.78:0/4188084363 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca180c6270 con 0x7fca1806a770 2023-10-11T09:46:07.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.983+0000 7fca1759e700 1 --2- 172.21.15.78:0/4188084363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca1806a770 0x7fca18165590 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fca10009d90 tx=0x7fca1000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76c7c39a8ed85c3e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:07.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.983+0000 7fca1659c700 1 -- 172.21.15.78:0/4188084363 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1000e070 con 0x7fca1806a770 2023-10-11T09:46:07.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.983+0000 7fca1659c700 1 -- 172.21.15.78:0/4188084363 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca10003a40 con 0x7fca1806a770 2023-10-11T09:46:07.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.983+0000 7fca1659c700 1 -- 172.21.15.78:0/4188084363 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca08001670 con 0x7fca1806a770 2023-10-11T09:46:07.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.983+0000 7fca1c881700 1 -- 172.21.15.78:0/4188084363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca1806a770 msgr2=0x7fca18165590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:07.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.983+0000 7fca1c881700 1 --2- 172.21.15.78:0/4188084363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca1806a770 0x7fca18165590 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fca10009d90 tx=0x7fca1000b330 comp rx=0 tx=0).stop 2023-10-11T09:46:07.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.983+0000 7fca1c881700 1 -- 172.21.15.78:0/4188084363 shutdown_connections 2023-10-11T09:46:07.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.983+0000 7fca1c881700 1 --2- 172.21.15.78:0/4188084363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca1806a770 0x7fca18165590 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:07.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.983+0000 7fca1c881700 1 -- 172.21.15.78:0/4188084363 >> 172.21.15.78:0/4188084363 conn(0x7fca1806be00 msgr2=0x7fca180b1800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:46:07.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.983+0000 7fca1c881700 1 -- 172.21.15.78:0/4188084363 shutdown_connections 2023-10-11T09:46:07.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.987+0000 7fca1c881700 1 -- 172.21.15.78:0/4188084363 wait complete. 2023-10-11T09:46:07.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.987+0000 7fca1c881700 1 Processor -- start 2023-10-11T09:46:07.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.987+0000 7fca1c881700 1 -- start start 2023-10-11T09:46:07.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.991+0000 7fca1c881700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca1806a770 0x7fca180cec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:07.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.991+0000 7fca1c881700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca181408d0 0x7fca1813ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:07.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.991+0000 7fca1c881700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca18143900 0x7fca180b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:07.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.991+0000 7fca1c881700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca180cf1d0 con 0x7fca1806a770 2023-10-11T09:46:07.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.991+0000 7fca1c881700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca180cf3c0 con 0x7fca181408d0 2023-10-11T09:46:07.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.991+0000 7fca1c881700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca180b3bf0 con 0x7fca18143900 2023-10-11T09:46:07.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.991+0000 7fca16d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca1806a770 0x7fca180cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:07.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.991+0000 7fca16d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca1806a770 0x7fca180cec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43662/0 (socket says 172.21.15.78:43662) 2023-10-11T09:46:07.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.991+0000 7fca16d9d700 1 -- 172.21.15.78:0/3718169626 learned_addr learned my addr 172.21.15.78:0/3718169626 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:46:07.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.991+0000 7fca1759e700 1 --2- 172.21.15.78:0/3718169626 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca18143900 0x7fca180b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:07.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.991+0000 7fca0ffff700 1 --2- 172.21.15.78:0/3718169626 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca181408d0 0x7fca1813ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:07.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.991+0000 7fca16d9d700 1 -- 172.21.15.78:0/3718169626 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca18143900 msgr2=0x7fca180b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:07.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.991+0000 7fca16d9d700 1 --2- 172.21.15.78:0/3718169626 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca18143900 0x7fca180b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:07.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.991+0000 7fca16d9d700 1 -- 172.21.15.78:0/3718169626 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca181408d0 msgr2=0x7fca1813ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:07.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.991+0000 7fca16d9d700 1 --2- 172.21.15.78:0/3718169626 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca181408d0 0x7fca1813ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:07.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.991+0000 7fca16d9d700 1 -- 172.21.15.78:0/3718169626 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca10009a70 con 0x7fca1806a770 2023-10-11T09:46:07.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.991+0000 7fca0ffff700 1 --2- 172.21.15.78:0/3718169626 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca181408d0 0x7fca1813ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:46:07.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.991+0000 7fca16d9d700 1 --2- 172.21.15.78:0/3718169626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca1806a770 0x7fca180cec90 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fca0800c750 tx=0x7fca0800ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:07.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.991+0000 7fca0f7fe700 1 -- 172.21.15.78:0/3718169626 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca08004ca0 con 0x7fca1806a770 2023-10-11T09:46:07.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.991+0000 7fca0f7fe700 1 -- 172.21.15.78:0/3718169626 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca08004e00 con 0x7fca1806a770 2023-10-11T09:46:08.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.991+0000 7fca0f7fe700 1 -- 172.21.15.78:0/3718169626 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca08007310 con 0x7fca1806a770 2023-10-11T09:46:08.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.991+0000 7fca1c881700 1 -- 172.21.15.78:0/3718169626 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca180b3ed0 con 0x7fca1806a770 2023-10-11T09:46:08.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.991+0000 7fca1c881700 1 -- 172.21.15.78:0/3718169626 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca180cf6a0 con 0x7fca1806a770 2023-10-11T09:46:08.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.995+0000 7fca0f7fe700 1 -- 172.21.15.78:0/3718169626 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fca08007470 con 0x7fca1806a770 2023-10-11T09:46:08.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.995+0000 7fca0f7fe700 1 --2- 172.21.15.78:0/3718169626 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc9f8065ed0 0x7fc9f8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:08.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.995+0000 7fca0f7fe700 1 -- 172.21.15.78:0/3718169626 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6724+0+0 (secure 0 0 0) 0x7fca08083920 con 0x7fca1806a770 2023-10-11T09:46:08.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.999+0000 7fca0ffff700 1 --2- 172.21.15.78:0/3718169626 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc9f8065ed0 0x7fc9f8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:08.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.999+0000 7fca0ffff700 1 --2- 172.21.15.78:0/3718169626 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc9f8065ed0 0x7fc9f8068380 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fca18142e80 tx=0x7fca00009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:08.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.999+0000 7fca20317740 1 Processor -- start 2023-10-11T09:46:08.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.999+0000 7fca20317740 1 -- start start 2023-10-11T09:46:08.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.999+0000 7fca20317740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1545d80 0x17fafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:08.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.999+0000 7fca20317740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x15459b0 0x17fb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:08.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.999+0000 7fca20317740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1590a30 0x17fba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:08.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.999+0000 7fca20317740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x16d82f0 con 0x1545d80 2023-10-11T09:46:08.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.999+0000 7fca20317740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x1725610 con 0x1590a30 2023-10-11T09:46:08.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.999+0000 7fca20317740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x1722da0 con 0x15459b0 2023-10-11T09:46:08.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.999+0000 7fca16d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1545d80 0x17fafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:08.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.999+0000 7fca16d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1545d80 0x17fafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43670/0 (socket says 172.21.15.78:43670) 2023-10-11T09:46:08.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.999+0000 7fca16d9d700 1 -- 172.21.15.78:0/3905267066 learned_addr learned my addr 172.21.15.78:0/3905267066 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:46:08.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.999+0000 7fca1759e700 1 --2- 172.21.15.78:0/3905267066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1590a30 0x17fba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:08.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.003+0000 7fca16d9d700 1 -- 172.21.15.78:0/3905267066 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x15459b0 msgr2=0x17fb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:08.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:07.999+0000 7fca0ffff700 1 --2- 172.21.15.78:0/3905267066 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x15459b0 0x17fb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:08.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.003+0000 7fca16d9d700 1 --2- 172.21.15.78:0/3905267066 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x15459b0 0x17fb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.003+0000 7fca16d9d700 1 -- 172.21.15.78:0/3905267066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1590a30 msgr2=0x17fba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:08.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.003+0000 7fca16d9d700 1 --2- 172.21.15.78:0/3905267066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1590a30 0x17fba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.003+0000 7fca16d9d700 1 -- 172.21.15.78:0/3905267066 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1745b50 con 0x1545d80 2023-10-11T09:46:08.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.003+0000 7fca0ffff700 1 --2- 172.21.15.78:0/3905267066 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x15459b0 0x17fb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:46:08.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.003+0000 7fca1759e700 1 --2- 172.21.15.78:0/3905267066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1590a30 0x17fba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:46:08.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.003+0000 7fca16d9d700 1 --2- 172.21.15.78:0/3905267066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1545d80 0x17fafb0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fca080547f0 tx=0x7fca08007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ccd27312e5cbc8a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:08.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.003+0000 7fca0cff9700 1 -- 172.21.15.78:0/3905267066 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca08058690 con 0x1545d80 2023-10-11T09:46:08.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.003+0000 7fca0cff9700 1 -- 172.21.15.78:0/3905267066 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca0805c8d0 con 0x1545d80 2023-10-11T09:46:08.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.003+0000 7fca0cff9700 1 -- 172.21.15.78:0/3905267066 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca08054ac0 con 0x1545d80 2023-10-11T09:46:08.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.003+0000 7fca20317740 1 -- 172.21.15.78:0/3905267066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1545d80 msgr2=0x17fafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:08.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.003+0000 7fca20317740 1 --2- 172.21.15.78:0/3905267066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1545d80 0x17fafb0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fca080547f0 tx=0x7fca08007f90 comp rx=0 tx=0).stop 2023-10-11T09:46:08.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.003+0000 7fca20317740 1 -- 172.21.15.78:0/3905267066 shutdown_connections 2023-10-11T09:46:08.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.003+0000 7fca20317740 1 --2- 172.21.15.78:0/3905267066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1590a30 0x17fba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.003+0000 7fca20317740 1 --2- 172.21.15.78:0/3905267066 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x15459b0 0x17fb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.003+0000 7fca20317740 1 --2- 172.21.15.78:0/3905267066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1545d80 0x17fafb0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.003+0000 7fca20317740 1 -- 172.21.15.78:0/3905267066 >> 172.21.15.78:0/3905267066 conn(0x1546150 msgr2=0x17b5860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:46:08.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.003+0000 7fca20317740 1 -- 172.21.15.78:0/3905267066 shutdown_connections 2023-10-11T09:46:08.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.003+0000 7fca20317740 1 -- 172.21.15.78:0/3905267066 wait complete. 2023-10-11T09:46:08.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.003+0000 7fca20317740 1 -- start start 2023-10-11T09:46:08.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.003+0000 7fca20317740 1 Processor -- start 2023-10-11T09:46:08.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.007+0000 7fca20317740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x15459b0 0x17fa560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:08.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.007+0000 7fca20317740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1545d80 0x17b8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:08.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.007+0000 7fca20317740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1590a30 0x17b8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:08.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.007+0000 7fca20317740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x171e690 con 0x1545d80 2023-10-11T09:46:08.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.007+0000 7fca20317740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x16e1c50 con 0x1590a30 2023-10-11T09:46:08.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.007+0000 7fca20317740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x16e18f0 con 0x15459b0 2023-10-11T09:46:08.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.007+0000 7fca20317740 10 client.0 _ll_register_callbacks cb 0x1771a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:46:08.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.007+0000 7fca0ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1545d80 0x17b8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:08.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.007+0000 7fca0ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1545d80 0x17b8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43678/0 (socket says 172.21.15.78:43678) 2023-10-11T09:46:08.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.007+0000 7fca0ffff700 1 -- 172.21.15.78:0/705453228 learned_addr learned my addr 172.21.15.78:0/705453228 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:46:08.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.007+0000 7fca1759e700 1 --2- 172.21.15.78:0/705453228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1590a30 0x17b8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:08.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.007+0000 7fca16d9d700 1 --2- 172.21.15.78:0/705453228 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x15459b0 0x17fa560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:08.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.007+0000 7fca0ffff700 1 -- 172.21.15.78:0/705453228 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x15459b0 msgr2=0x17fa560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:08.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.007+0000 7fca0ffff700 1 --2- 172.21.15.78:0/705453228 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x15459b0 0x17fa560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.007+0000 7fca0ffff700 1 -- 172.21.15.78:0/705453228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1590a30 msgr2=0x17b8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:08.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.007+0000 7fca0ffff700 1 --2- 172.21.15.78:0/705453228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1590a30 0x17b8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.007+0000 7fca0ffff700 1 -- 172.21.15.78:0/705453228 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca180cf6a0 con 0x1545d80 2023-10-11T09:46:08.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.007+0000 7fca0ffff700 1 --2- 172.21.15.78:0/705453228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1545d80 0x17b8f00 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fca00005670 tx=0x7fca00000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92e7806cedd122a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:08.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.007+0000 7fc9f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:46:08.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.007+0000 7fc9f5ffb700 1 -- 172.21.15.78:0/705453228 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0000e8f0 con 0x1545d80 2023-10-11T09:46:08.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.007+0000 7fc9f5ffb700 1 -- 172.21.15.78:0/705453228 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 6724+0+0 (secure 0 0 0) 0x7fca0000ea50 con 0x1545d80 2023-10-11T09:46:08.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.007+0000 7fca20317740 1 -- 172.21.15.78:0/705453228 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1717710 con 0x1545d80 2023-10-11T09:46:08.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.007+0000 7fc9f5ffb700 1 -- 172.21.15.78:0/705453228 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca0000ec10 con 0x1545d80 2023-10-11T09:46:08.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.007+0000 7fca20317740 10 client.6972 fetch_fsmap learned FSMap version 50 2023-10-11T09:46:08.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.007+0000 7fca20317740 1 -- 172.21.15.78:0/705453228 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x1591c80 con 0x1545d80 2023-10-11T09:46:08.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.007+0000 7fc9f5ffb700 1 -- 172.21.15.78:0/705453228 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1717710 con 0x1545d80 2023-10-11T09:46:08.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.007+0000 7fc9f5ffb700 1 -- 172.21.15.78:0/705453228 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 50) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fca00022430 con 0x1545d80 2023-10-11T09:46:08.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.007+0000 7fca20317740 10 client.6972 fetch_fsmap finished waiting for FSMap version 50 2023-10-11T09:46:08.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.007+0000 7fca20317740 10 client.6972 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-10-11T09:46:08.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.007+0000 7fca20317740 20 client.6972 populate_metadata read hostname 'smithi078' 2023-10-11T09:46:08.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.007+0000 7fca20317740 1 --2- 172.21.15.78:0/705453228 >> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] conn(0x17e0a90 0x17fb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:08.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.007+0000 7fca20317740 4 client.6972 mds_command: new command op to 6897 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-11T09:46:08.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.007+0000 7fca20317740 1 -- 172.21.15.78:0/705453228 --> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1717430 con 0x17e0a90 2023-10-11T09:46:08.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.011+0000 7fca16d9d700 1 --2- 172.21.15.78:0/705453228 >> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] conn(0x17e0a90 0x17fb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:08.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.011+0000 7fc9f5ffb700 10 client.6972 ms_handle_connect on v2:172.21.15.78:6835/3498868661 2023-10-11T09:46:08.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.011+0000 7fca16d9d700 1 --2- 172.21.15.78:0/705453228 >> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] conn(0x17e0a90 0x17fb4a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb990a500a8907a server_cookie=d3a1bb346f8f643f in_seq=0 out_seq=0 2023-10-11T09:46:08.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.011+0000 7fc9f5ffb700 1 -- 172.21.15.78:0/705453228 <== mds.0 v2:172.21.15.78:6835/3498868661 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x1717430 con 0x17e0a90 2023-10-11T09:46:08.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.011+0000 7fc9f5ffb700 10 client.6972 handle_command_reply: tid=0 2023-10-11T09:46:08.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.011+0000 7fca20317740 1 client.6972 shutdown 2023-10-11T09:46:08.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.011+0000 7fca20317740 1 -- 172.21.15.78:0/705453228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1545d80 msgr2=0x17b8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:08.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.011+0000 7fca20317740 1 --2- 172.21.15.78:0/705453228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1545d80 0x17b8f00 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fca00005670 tx=0x7fca00000ed0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.011+0000 7fca20317740 1 -- 172.21.15.78:0/705453228 shutdown_connections 2023-10-11T09:46:08.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.011+0000 7fca20317740 1 --2- 172.21.15.78:0/705453228 >> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] conn(0x17e0a90 0x17fb4a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.011+0000 7fca20317740 1 --2- 172.21.15.78:0/705453228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1590a30 0x17b8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.011+0000 7fca20317740 1 --2- 172.21.15.78:0/705453228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1545d80 0x17b8f00 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.011+0000 7fca20317740 1 --2- 172.21.15.78:0/705453228 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x15459b0 0x17fa560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.011+0000 7fca20317740 1 -- 172.21.15.78:0/705453228 >> 172.21.15.78:0/705453228 conn(0x1546150 msgr2=0x17dde10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:46:08.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.011+0000 7fc9f5ffb700 0 client.6972 ms_handle_reset on v2:172.21.15.78:6835/3498868661 2023-10-11T09:46:08.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.011+0000 7fca20317740 1 -- 172.21.15.78:0/705453228 shutdown_connections 2023-10-11T09:46:08.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.011+0000 7fca20317740 1 -- 172.21.15.78:0/705453228 wait complete. 2023-10-11T09:46:08.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.011+0000 7fca20317740 20 client.6972 trim_cache size 0 max 16384 2023-10-11T09:46:08.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.031+0000 7fca20317740 1 Processor -- start 2023-10-11T09:46:08.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.031+0000 7fca20317740 1 -- start start 2023-10-11T09:46:08.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.031+0000 7fca20317740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x15459b0 0x17b7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:08.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.031+0000 7fca20317740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1545d80 0x17b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:08.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.031+0000 7fca20317740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1590a30 0x17b5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:08.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.031+0000 7fca20317740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x15cadc0 con 0x15459b0 2023-10-11T09:46:08.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.031+0000 7fca20317740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x16ede50 con 0x1545d80 2023-10-11T09:46:08.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.031+0000 7fca20317740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x171dea0 con 0x1590a30 2023-10-11T09:46:08.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.031+0000 7fca16d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x15459b0 0x17b7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:08.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.031+0000 7fca16d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x15459b0 0x17b7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43680/0 (socket says 172.21.15.78:43680) 2023-10-11T09:46:08.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.031+0000 7fca16d9d700 1 -- 172.21.15.78:0/4039118065 learned_addr learned my addr 172.21.15.78:0/4039118065 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:46:08.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.031+0000 7fca1759e700 1 --2- 172.21.15.78:0/4039118065 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1590a30 0x17b5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:08.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.031+0000 7fca0ffff700 1 --2- 172.21.15.78:0/4039118065 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1545d80 0x17b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:08.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.031+0000 7fca16d9d700 1 -- 172.21.15.78:0/4039118065 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1590a30 msgr2=0x17b5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:08.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.031+0000 7fca16d9d700 1 --2- 172.21.15.78:0/4039118065 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1590a30 0x17b5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.031+0000 7fca16d9d700 1 -- 172.21.15.78:0/4039118065 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1545d80 msgr2=0x17b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:08.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.031+0000 7fca16d9d700 1 --2- 172.21.15.78:0/4039118065 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1545d80 0x17b4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.031+0000 7fca16d9d700 1 -- 172.21.15.78:0/4039118065 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca1813f310 con 0x15459b0 2023-10-11T09:46:08.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.035+0000 7fca16d9d700 1 --2- 172.21.15.78:0/4039118065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x15459b0 0x17b7200 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fca08050d60 tx=0x7fca08007850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f785f8111abf667 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:08.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.035+0000 7fca0d7fa700 1 -- 172.21.15.78:0/4039118065 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca080543a0 con 0x15459b0 2023-10-11T09:46:08.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.035+0000 7fca0d7fa700 1 -- 172.21.15.78:0/4039118065 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca08052c00 con 0x15459b0 2023-10-11T09:46:08.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.035+0000 7fca0d7fa700 1 -- 172.21.15.78:0/4039118065 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca08053f50 con 0x15459b0 2023-10-11T09:46:08.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.035+0000 7fca20317740 1 -- 172.21.15.78:0/4039118065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x15459b0 msgr2=0x17b7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:08.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.035+0000 7fca20317740 1 --2- 172.21.15.78:0/4039118065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x15459b0 0x17b7200 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fca08050d60 tx=0x7fca08007850 comp rx=0 tx=0).stop 2023-10-11T09:46:08.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.035+0000 7fca20317740 1 -- 172.21.15.78:0/4039118065 shutdown_connections 2023-10-11T09:46:08.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.035+0000 7fca20317740 1 --2- 172.21.15.78:0/4039118065 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1590a30 0x17b5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.035+0000 7fca20317740 1 --2- 172.21.15.78:0/4039118065 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1545d80 0x17b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.035+0000 7fca20317740 1 --2- 172.21.15.78:0/4039118065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x15459b0 0x17b7200 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.035+0000 7fca20317740 1 -- 172.21.15.78:0/4039118065 >> 172.21.15.78:0/4039118065 conn(0x1546150 msgr2=0x180dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:46:08.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.035+0000 7fca20317740 1 -- 172.21.15.78:0/4039118065 shutdown_connections 2023-10-11T09:46:08.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.035+0000 7fca20317740 1 -- 172.21.15.78:0/4039118065 wait complete. 2023-10-11T09:46:08.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.035+0000 7fca20317740 1 -- start start 2023-10-11T09:46:08.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.035+0000 7fca20317740 1 Processor -- start 2023-10-11T09:46:08.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.039+0000 7fca20317740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x15459b0 0x185bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:08.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.039+0000 7fca20317740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1545d80 0x185c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:08.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.039+0000 7fca20317740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1590a30 0x185c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:08.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.039+0000 7fca20317740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x185d880 con 0x15459b0 2023-10-11T09:46:08.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.039+0000 7fca20317740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x180eb70 con 0x1545d80 2023-10-11T09:46:08.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.039+0000 7fca20317740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x185da60 con 0x1590a30 2023-10-11T09:46:08.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.039+0000 7fca20317740 10 client.0 _ll_register_callbacks cb 0x16d6d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:46:08.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.039+0000 7fca16d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x15459b0 0x185bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:08.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.039+0000 7fca16d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x15459b0 0x185bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43696/0 (socket says 172.21.15.78:43696) 2023-10-11T09:46:08.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.039+0000 7fca16d9d700 1 -- 172.21.15.78:0/3385391190 learned_addr learned my addr 172.21.15.78:0/3385391190 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:46:08.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.039+0000 7fca0ffff700 1 --2- 172.21.15.78:0/3385391190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1545d80 0x185c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:08.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.039+0000 7fca1759e700 1 --2- 172.21.15.78:0/3385391190 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1590a30 0x185c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:08.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.039+0000 7fca16d9d700 1 -- 172.21.15.78:0/3385391190 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1590a30 msgr2=0x185c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:08.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.039+0000 7fca16d9d700 1 --2- 172.21.15.78:0/3385391190 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1590a30 0x185c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.039+0000 7fca16d9d700 1 -- 172.21.15.78:0/3385391190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1545d80 msgr2=0x185c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:08.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.039+0000 7fca16d9d700 1 --2- 172.21.15.78:0/3385391190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1545d80 0x185c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.039+0000 7fca16d9d700 1 -- 172.21.15.78:0/3385391190 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca0806e040 con 0x15459b0 2023-10-11T09:46:08.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.039+0000 7fca16d9d700 1 --2- 172.21.15.78:0/3385391190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x15459b0 0x185bec0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fca08054560 tx=0x7fca0806a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d8c2b0a44ba5fd server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:08.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.039+0000 7fc9f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:46:08.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.039+0000 7fc9f5ffb700 1 -- 172.21.15.78:0/3385391190 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca080543a0 con 0x15459b0 2023-10-11T09:46:08.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.039+0000 7fc9f5ffb700 1 -- 172.21.15.78:0/3385391190 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 6724+0+0 (secure 0 0 0) 0x7fca080529a0 con 0x15459b0 2023-10-11T09:46:08.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.039+0000 7fca20317740 1 -- 172.21.15.78:0/3385391190 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x17b92d0 con 0x15459b0 2023-10-11T09:46:08.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.043+0000 7fc9f5ffb700 1 -- 172.21.15.78:0/3385391190 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca08007720 con 0x15459b0 2023-10-11T09:46:08.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.043+0000 7fca20317740 10 client.6978 fetch_fsmap learned FSMap version 50 2023-10-11T09:46:08.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.043+0000 7fca20317740 1 -- 172.21.15.78:0/3385391190 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x16e48e0 con 0x15459b0 2023-10-11T09:46:08.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.043+0000 7fc9f5ffb700 1 -- 172.21.15.78:0/3385391190 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x17b92d0 con 0x15459b0 2023-10-11T09:46:08.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.043+0000 7fc9f5ffb700 1 -- 172.21.15.78:0/3385391190 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 50) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fca08054ac0 con 0x15459b0 2023-10-11T09:46:08.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.043+0000 7fca20317740 10 client.6978 fetch_fsmap finished waiting for FSMap version 50 2023-10-11T09:46:08.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.043+0000 7fca20317740 10 client.6978 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-10-11T09:46:08.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.043+0000 7fca20317740 20 client.6978 populate_metadata read hostname 'smithi078' 2023-10-11T09:46:08.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.043+0000 7fca20317740 1 --2- 172.21.15.78:0/3385391190 >> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] conn(0x185a340 0x185a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:08.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.043+0000 7fca20317740 4 client.6978 mds_command: new command op to 6897 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2023-10-11T09:46:08.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.043+0000 7fca20317740 1 -- 172.21.15.78:0/3385391190 --> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x7fca0805c8d0 con 0x185a340 2023-10-11T09:46:08.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.043+0000 7fca0ffff700 1 --2- 172.21.15.78:0/3385391190 >> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] conn(0x185a340 0x185a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:08.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.043+0000 7fc9f5ffb700 10 client.6978 ms_handle_connect on v2:172.21.15.78:6835/3498868661 2023-10-11T09:46:08.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.043+0000 7fca0ffff700 1 --2- 172.21.15.78:0/3385391190 >> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] conn(0x185a340 0x185a740 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2c6fa2adf8b88f server_cookie=53e5d128da77cd82 in_seq=0 out_seq=0 2023-10-11T09:46:08.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.043+0000 7fc9f5ffb700 1 -- 172.21.15.78:0/3385391190 <== mds.0 v2:172.21.15.78:6835/3498868661 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x7fca0805c8d0 con 0x185a340 2023-10-11T09:46:08.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.043+0000 7fc9f5ffb700 10 client.6978 handle_command_reply: tid=0 2023-10-11T09:46:08.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.043+0000 7fca20317740 1 client.6978 shutdown 2023-10-11T09:46:08.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.043+0000 7fca20317740 1 -- 172.21.15.78:0/3385391190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x15459b0 msgr2=0x185bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:08.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.043+0000 7fca20317740 1 --2- 172.21.15.78:0/3385391190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x15459b0 0x185bec0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fca08054560 tx=0x7fca0806a000 comp rx=0 tx=0).stop 2023-10-11T09:46:08.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.043+0000 7fca20317740 1 -- 172.21.15.78:0/3385391190 shutdown_connections 2023-10-11T09:46:08.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.047+0000 7fca20317740 1 --2- 172.21.15.78:0/3385391190 >> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] conn(0x185a340 0x185a740 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.047+0000 7fca20317740 1 --2- 172.21.15.78:0/3385391190 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1590a30 0x185c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.047+0000 7fca20317740 1 --2- 172.21.15.78:0/3385391190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1545d80 0x185c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.047+0000 7fca20317740 1 --2- 172.21.15.78:0/3385391190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x15459b0 0x185bec0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.047+0000 7fca20317740 1 -- 172.21.15.78:0/3385391190 >> 172.21.15.78:0/3385391190 conn(0x1546150 msgr2=0x17f8750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:46:08.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.047+0000 7fc9f5ffb700 0 client.6978 ms_handle_reset on v2:172.21.15.78:6835/3498868661 2023-10-11T09:46:08.058 INFO:tasks.ceph.mds.a.smithi078.stderr:2023-10-11T09:46:08.047+0000 7f2933026700 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2023-10-11T09:46:08.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.047+0000 7fca20317740 1 -- 172.21.15.78:0/3385391190 shutdown_connections 2023-10-11T09:46:08.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.047+0000 7fca20317740 1 -- 172.21.15.78:0/3385391190 wait complete. 2023-10-11T09:46:08.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.047+0000 7fca20317740 20 client.6978 trim_cache size 0 max 16384 2023-10-11T09:46:08.059 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:46:08.059 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2023-10-11T09:46:08.060 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "df3143e9-5161-4e91-93ef-2c0b6488c02b", 2023-10-11T09:46:08.060 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2023-10-11T09:46:08.060 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:46:08.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.047+0000 7fca1c881700 1 -- 172.21.15.78:0/3718169626 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc9f8065ed0 msgr2=0x7fc9f8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:08.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.047+0000 7fca1c881700 1 --2- 172.21.15.78:0/3718169626 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc9f8065ed0 0x7fc9f8068380 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fca18142e80 tx=0x7fca00009040 comp rx=0 tx=0).stop 2023-10-11T09:46:08.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.047+0000 7fca1c881700 1 -- 172.21.15.78:0/3718169626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca1806a770 msgr2=0x7fca180cec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:08.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.047+0000 7fca1c881700 1 --2- 172.21.15.78:0/3718169626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca1806a770 0x7fca180cec90 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fca0800c750 tx=0x7fca0800ca60 comp rx=0 tx=0).stop 2023-10-11T09:46:08.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.047+0000 7fca1c881700 1 -- 172.21.15.78:0/3718169626 shutdown_connections 2023-10-11T09:46:08.062 INFO:tasks.ceph.mds.a.smithi078.stderr:2023-10-11T09:46:08.047+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /subdir/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55e01e8fe100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000000:[<0x1/subdir v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-10-11T09:46:08.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.047+0000 7fca1c881700 1 --2- 172.21.15.78:0/3718169626 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc9f8065ed0 0x7fc9f8068380 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.047+0000 7fca1c881700 1 --2- 172.21.15.78:0/3718169626 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca18143900 0x7fca180b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.047+0000 7fca1c881700 1 --2- 172.21.15.78:0/3718169626 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca181408d0 0x7fca1813ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.047+0000 7fca1c881700 1 --2- 172.21.15.78:0/3718169626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca1806a770 0x7fca180cec90 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.047+0000 7fca1c881700 1 -- 172.21.15.78:0/3718169626 >> 172.21.15.78:0/3718169626 conn(0x7fca1806be00 msgr2=0x7fca1806a060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:46:08.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.047+0000 7fca1c881700 1 -- 172.21.15.78:0/3718169626 shutdown_connections 2023-10-11T09:46:08.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.047+0000 7fca1c881700 1 -- 172.21.15.78:0/3718169626 wait complete. 2023-10-11T09:46:08.066 INFO:tasks.ceph.mds.a.smithi078.stderr:2023-10-11T09:46:08.051+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f8 [2,head] /subdir/3 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea1c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001f8:[<0x10000000000/3 v42>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x100000001f8:[<0x10000000000/3 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:46:08.067 INFO:tasks.ceph.mds.a.smithi078.stderr:2023-10-11T09:46:08.051+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fd [2,head] /subdir/8 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea1a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001fd:[<0x10000000000/8 v82>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x100000001fd:[<0x10000000000/8 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:46:08.067 INFO:tasks.ceph.mds.a.smithi078.stderr:2023-10-11T09:46:08.051+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020a [2,head] /subdir/21 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea22000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x1000000020a:[<0x10000000000/21 v188>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x1000000020a:[<0x10000000000/21 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:46:08.067 INFO:tasks.ceph.mds.a.smithi078.stderr:2023-10-11T09:46:08.051+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000208 [2,head] /subdir/19 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea23080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000208:[<0x10000000000/19 v170>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x10000000208:[<0x10000000000/19 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:46:08.067 INFO:tasks.ceph.mds.a.smithi078.stderr:2023-10-11T09:46:08.051+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fe [2,head] /subdir/9 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea1a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001fe:[<0x10000000000/9 v90>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x100000001fe:[<0x10000000000/9 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:46:08.068 INFO:tasks.ceph.mds.a.smithi078.stderr:2023-10-11T09:46:08.051+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000209 [2,head] /subdir/20 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea22580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000209:[<0x10000000000/20 v178>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x10000000209:[<0x10000000000/20 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:46:08.068 INFO:tasks.ceph.mds.a.smithi078.stderr:2023-10-11T09:46:08.051+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000201 [2,head] /subdir/12 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea25700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000201:[<0x10000000000/12 v116>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x10000000201:[<0x10000000000/12 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:46:08.068 INFO:tasks.ceph.mds.a.smithi078.stderr:2023-10-11T09:46:08.051+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000204 [2,head] /subdir/15 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea24680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000204:[<0x10000000000/15 v140>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x10000000204:[<0x10000000000/15 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:46:08.068 INFO:tasks.ceph.mds.a.smithi078.stderr:2023-10-11T09:46:08.051+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001ff [2,head] /subdir/10 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea2c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001ff:[<0x10000000000/10 v100>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x100000001ff:[<0x10000000000/10 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:46:08.068 INFO:tasks.ceph.mds.a.smithi078.stderr:2023-10-11T09:46:08.055+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f7 [2,head] /subdir/2 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea1c680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001f7:[<0x10000000000/2 v34>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x100000001f7:[<0x10000000000/2 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:46:08.069 INFO:tasks.ceph.mds.a.smithi078.stderr:2023-10-11T09:46:08.055+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000200 [2,head] /subdir/11 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea2c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000200:[<0x10000000000/11 v110>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x10000000200:[<0x10000000000/11 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:46:08.069 INFO:tasks.ceph.mds.a.smithi078.stderr:2023-10-11T09:46:08.055+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f6 [2,head] /subdir/1 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea22b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001f6:[<0x10000000000/1 v30>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x100000001f6:[<0x10000000000/1 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:46:08.069 INFO:tasks.ceph.mds.a.smithi078.stderr:2023-10-11T09:46:08.055+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f9 [2,head] /subdir/4 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea1bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001f9:[<0x10000000000/4 v52>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x100000001f9:[<0x10000000000/4 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:46:08.069 INFO:tasks.ceph.mds.a.smithi078.stderr:2023-10-11T09:46:08.055+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020d [2,head] /subdir/24 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea1cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x1000000020d:[<0x10000000000/24 v202>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x1000000020d:[<0x10000000000/24 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:46:08.069 INFO:tasks.ceph.mds.a.smithi078.stderr:2023-10-11T09:46:08.055+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000205 [2,head] /subdir/16 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea24100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000205:[<0x10000000000/16 v146>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x10000000205:[<0x10000000000/16 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:46:08.070 INFO:tasks.ceph.mds.a.smithi078.stderr:2023-10-11T09:46:08.055+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020b [2,head] /subdir/22 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea1d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x1000000020b:[<0x10000000000/22 v194>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x1000000020b:[<0x10000000000/22 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:46:08.070 INFO:tasks.ceph.mds.a.smithi078.stderr:2023-10-11T09:46:08.055+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/0 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea2cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000001:[<0x10000000000/0 v16>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x10000000001:[<0x10000000000/0 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:46:08.070 INFO:tasks.ceph.mds.a.smithi078.stderr:2023-10-11T09:46:08.055+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fa [2,head] /subdir/5 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea1b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001fa:[<0x10000000000/5 v58>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x100000001fa:[<0x10000000000/5 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:46:08.070 INFO:tasks.ceph.mds.a.smithi078.stderr:2023-10-11T09:46:08.055+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000203 [2,head] /subdir/14 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea24c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000203:[<0x10000000000/14 v130>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x10000000203:[<0x10000000000/14 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:46:08.070 INFO:tasks.ceph.mds.a.smithi078.stderr:2023-10-11T09:46:08.055+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000207 [2,head] /subdir/18 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea23600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000207:[<0x10000000000/18 v164>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x10000000207:[<0x10000000000/18 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:46:08.071 INFO:tasks.ceph.mds.a.smithi078.stderr:2023-10-11T09:46:08.055+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000202 [2,head] /subdir/13 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea25180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000202:[<0x10000000000/13 v122>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x10000000202:[<0x10000000000/13 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:46:08.071 INFO:tasks.ceph.mds.a.smithi078.stderr:2023-10-11T09:46:08.055+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020c [2,head] /subdir/23 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea1d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x1000000020c:[<0x10000000000/23 v200>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x1000000020c:[<0x10000000000/23 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:46:08.071 INFO:tasks.ceph.mds.a.smithi078.stderr:2023-10-11T09:46:08.055+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fc [2,head] /subdir/7 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea1ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001fc:[<0x10000000000/7 v76>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x100000001fc:[<0x10000000000/7 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:46:08.071 INFO:tasks.ceph.mds.a.smithi078.stderr:2023-10-11T09:46:08.055+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000206 [2,head] /subdir/17 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea23b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000206:[<0x10000000000/17 v154>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x10000000206:[<0x10000000000/17 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:46:08.072 INFO:tasks.ceph.mds.a.smithi078.stderr:2023-10-11T09:46:08.055+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fb [2,head] /subdir/6 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea1b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001fb:[<0x10000000000/6 v66>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x100000001fb:[<0x10000000000/6 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:46:08.072 INFO:tasks.ceph.mds.a.smithi078.stderr:2023-10-11T09:46:08.059+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v5 ap=5 snaprealm=0x55e01e901200 DIRTYPARENT f(v1 1=0+1) n(v0 rc2023-10-11T09:46:07.751414+0000 1=0+1)/n() (idft sync r=1) (inest sync r=1) (ifile sync->mix r=1) (iversion lock) caps={6960=pLsXs/-@3} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x55e01e8fd600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=0+1)","ondisk_value.rstat":"n(v0 2=0+2)","memory_value.dirstat":"f(v1 1=0+1)","memory_value.rstat":"n(v0 rc2023-10-11T09:46:07.751414+0000 1=0+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-10-11T09:46:08.076 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-10-11T09:46:08.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.247+0000 7f456d978700 1 Processor -- start 2023-10-11T09:46:08.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.247+0000 7f456d978700 1 -- start start 2023-10-11T09:46:08.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.251+0000 7f456d978700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f45680b84c0 0x7f45680cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:08.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.251+0000 7f456d978700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f45680b8890 0x7f4568160370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:08.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.251+0000 7f456d978700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45680cd690 con 0x7f45681608b0 2023-10-11T09:46:08.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.251+0000 7f456d978700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45680cdea0 con 0x7f45680b84c0 2023-10-11T09:46:08.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.251+0000 7f456d978700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45680d0c10 con 0x7f45680b8890 2023-10-11T09:46:08.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.251+0000 7f4566ffd700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f45680b84c0 0x7f45680cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:08.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.251+0000 7f4566ffd700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f45680b84c0 0x7f45680cfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43710/0 (socket says 172.21.15.78:43710) 2023-10-11T09:46:08.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.251+0000 7f4566ffd700 1 -- 172.21.15.78:0/4221395092 learned_addr learned my addr 172.21.15.78:0/4221395092 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:46:08.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.251+0000 7f45667fc700 1 --2- 172.21.15.78:0/4221395092 >> v2:172.21.15.133:3300/0 conn(0x7f45680b8890 0x7f4568160370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:08.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.251+0000 7f4566ffd700 1 -- 172.21.15.78:0/4221395092 >> v2:172.21.15.133:3300/0 conn(0x7f45680b8890 msgr2=0x7f4568160370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:08.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.251+0000 7f4566ffd700 1 --2- 172.21.15.78:0/4221395092 >> v2:172.21.15.133:3300/0 conn(0x7f45680b8890 0x7f4568160370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.251+0000 7f4566ffd700 1 -- 172.21.15.78:0/4221395092 >> v1:172.21.15.133:6789/0 conn(0x7f45681608b0 legacy=0x7f45680d2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:08.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.251+0000 7f4566ffd700 1 -- 172.21.15.78:0/4221395092 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45680d02e0 con 0x7f45680b84c0 2023-10-11T09:46:08.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.251+0000 7f4566ffd700 1 --2- 172.21.15.78:0/4221395092 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45680b84c0 0x7f45680cfda0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f456808bfc0 tx=0x7f4558009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dadc61ec89d3c4d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:08.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.251+0000 7f4565ffb700 1 -- 172.21.15.78:0/4221395092 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4558011830 con 0x7f45680b84c0 2023-10-11T09:46:08.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.251+0000 7f4565ffb700 1 -- 172.21.15.78:0/4221395092 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4558011990 con 0x7f45680b84c0 2023-10-11T09:46:08.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.251+0000 7f4565ffb700 1 -- 172.21.15.78:0/4221395092 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4558011d20 con 0x7f45680b84c0 2023-10-11T09:46:08.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.255+0000 7f456d978700 1 -- 172.21.15.78:0/4221395092 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45680b84c0 msgr2=0x7f45680cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:08.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.255+0000 7f456d978700 1 --2- 172.21.15.78:0/4221395092 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45680b84c0 0x7f45680cfda0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f456808bfc0 tx=0x7f4558009bf0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.255+0000 7f456d978700 1 -- 172.21.15.78:0/4221395092 shutdown_connections 2023-10-11T09:46:08.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.255+0000 7f456d978700 1 --2- 172.21.15.78:0/4221395092 >> v2:172.21.15.133:3300/0 conn(0x7f45680b8890 0x7f4568160370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.255+0000 7f456d978700 1 --2- 172.21.15.78:0/4221395092 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45680b84c0 0x7f45680cfda0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.255+0000 7f456d978700 1 -- 172.21.15.78:0/4221395092 >> 172.21.15.78:0/4221395092 conn(0x7f456806cb50 msgr2=0x7f456806cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:46:08.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.255+0000 7f456d978700 1 -- 172.21.15.78:0/4221395092 shutdown_connections 2023-10-11T09:46:08.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.255+0000 7f456d978700 1 -- 172.21.15.78:0/4221395092 wait complete. 2023-10-11T09:46:08.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.259+0000 7f456d978700 1 Processor -- start 2023-10-11T09:46:08.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.259+0000 7f456d978700 1 -- start start 2023-10-11T09:46:08.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.259+0000 7f456d978700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45680b84c0 0x7f45680c97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:08.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.259+0000 7f456d978700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45680b8890 0x7f45681e5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:08.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.259+0000 7f456d978700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f45681608b0 0x7f45681e6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:08.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.259+0000 7f456d978700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45680c7620 con 0x7f45680b84c0 2023-10-11T09:46:08.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.259+0000 7f456d978700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45680c6cf0 con 0x7f45680b8890 2023-10-11T09:46:08.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.259+0000 7f456d978700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45680c6fd0 con 0x7f45681608b0 2023-10-11T09:46:08.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.259+0000 7f4566ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45680b84c0 0x7f45680c97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:08.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.259+0000 7f4566ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45680b84c0 0x7f45680c97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43726/0 (socket says 172.21.15.78:43726) 2023-10-11T09:46:08.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.259+0000 7f4566ffd700 1 -- 172.21.15.78:0/708004576 learned_addr learned my addr 172.21.15.78:0/708004576 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:46:08.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.259+0000 7f45677fe700 1 --2- 172.21.15.78:0/708004576 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f45681608b0 0x7f45681e6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:08.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.259+0000 7f4566ffd700 1 -- 172.21.15.78:0/708004576 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f45681608b0 msgr2=0x7f45681e6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:08.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.259+0000 7f4566ffd700 1 --2- 172.21.15.78:0/708004576 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f45681608b0 0x7f45681e6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.259+0000 7f4566ffd700 1 -- 172.21.15.78:0/708004576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45680b8890 msgr2=0x7f45681e5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:08.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.259+0000 7f45667fc700 1 --2- 172.21.15.78:0/708004576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45680b8890 0x7f45681e5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:08.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.259+0000 7f4566ffd700 1 --2- 172.21.15.78:0/708004576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45680b8890 0x7f45681e5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.259+0000 7f4566ffd700 1 -- 172.21.15.78:0/708004576 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4558009570 con 0x7f45680b84c0 2023-10-11T09:46:08.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.259+0000 7f45677fe700 1 --2- 172.21.15.78:0/708004576 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f45681608b0 0x7f45681e6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:46:08.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.259+0000 7f4566ffd700 1 --2- 172.21.15.78:0/708004576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45680b84c0 0x7f45680c97b0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f4558000c00 tx=0x7f4558004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:08.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.259+0000 7f454ffff700 1 -- 172.21.15.78:0/708004576 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4558004610 con 0x7f45680b84c0 2023-10-11T09:46:08.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.259+0000 7f454ffff700 1 -- 172.21.15.78:0/708004576 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f455801e070 con 0x7f45680b84c0 2023-10-11T09:46:08.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.259+0000 7f454ffff700 1 -- 172.21.15.78:0/708004576 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45580038e0 con 0x7f45680b84c0 2023-10-11T09:46:08.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.263+0000 7f456d978700 1 -- 172.21.15.78:0/708004576 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45680c72b0 con 0x7f45680b84c0 2023-10-11T09:46:08.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.263+0000 7f456d978700 1 -- 172.21.15.78:0/708004576 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f456815c450 con 0x7f45680b84c0 2023-10-11T09:46:08.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.263+0000 7f454ffff700 1 -- 172.21.15.78:0/708004576 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f4558003a40 con 0x7f45680b84c0 2023-10-11T09:46:08.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.267+0000 7f4570c02740 1 Processor -- start 2023-10-11T09:46:08.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.267+0000 7f454ffff700 1 --2- 172.21.15.78:0/708004576 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4554065f00 0x7f45540683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:08.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.267+0000 7f4570c02740 1 -- start start 2023-10-11T09:46:08.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.267+0000 7f454ffff700 1 -- 172.21.15.78:0/708004576 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f45580849c0 con 0x7f45680b84c0 2023-10-11T09:46:08.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.267+0000 7f45677fe700 1 --2- 172.21.15.78:0/708004576 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4554065f00 0x7f45540683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:08.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.267+0000 7f4570c02740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c7ac10 0x1f105c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:08.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.267+0000 7f4570c02740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1c7a840 0x1f31bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:08.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.267+0000 7f4570c02740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1cc58c0 0x1f320f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:08.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.267+0000 7f4570c02740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e5a630 con 0x1c7ac10 2023-10-11T09:46:08.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.267+0000 7f4570c02740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e57dc0 con 0x1c7a840 2023-10-11T09:46:08.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.267+0000 7f4570c02740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e536b0 con 0x1cc58c0 2023-10-11T09:46:08.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.267+0000 7f45677fe700 1 --2- 172.21.15.78:0/708004576 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4554065f00 0x7f45540683b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f45600012e0 tx=0x7f4560009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:08.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.267+0000 7f4566ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c7ac10 0x1f105c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:08.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.267+0000 7f4566ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c7ac10 0x1f105c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43730/0 (socket says 172.21.15.78:43730) 2023-10-11T09:46:08.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.267+0000 7f4566ffd700 1 -- 172.21.15.78:0/3652085605 learned_addr learned my addr 172.21.15.78:0/3652085605 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:46:08.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.267+0000 7f45667fc700 1 --2- 172.21.15.78:0/3652085605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1c7a840 0x1f31bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:08.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.267+0000 7f45677fe700 1 --2- 172.21.15.78:0/3652085605 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1cc58c0 0x1f320f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:08.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.267+0000 7f4566ffd700 1 -- 172.21.15.78:0/3652085605 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1cc58c0 msgr2=0x1f320f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:08.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.267+0000 7f4566ffd700 1 --2- 172.21.15.78:0/3652085605 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1cc58c0 0x1f320f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.267+0000 7f4566ffd700 1 -- 172.21.15.78:0/3652085605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1c7a840 msgr2=0x1f31bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:08.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.267+0000 7f4566ffd700 1 --2- 172.21.15.78:0/3652085605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1c7a840 0x1f31bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.267+0000 7f4566ffd700 1 -- 172.21.15.78:0/3652085605 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1e795b0 con 0x1c7ac10 2023-10-11T09:46:08.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.267+0000 7f45677fe700 1 --2- 172.21.15.78:0/3652085605 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1cc58c0 0x1f320f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:46:08.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.267+0000 7f45667fc700 1 --2- 172.21.15.78:0/3652085605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1c7a840 0x1f31bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:46:08.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.271+0000 7f4566ffd700 1 --2- 172.21.15.78:0/3652085605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c7ac10 0x1f105c0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f4558051f20 tx=0x7f45580561a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45318bab5acc2e00 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:08.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.271+0000 7f454d7fa700 1 -- 172.21.15.78:0/3652085605 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455805da00 con 0x1c7ac10 2023-10-11T09:46:08.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.271+0000 7f454d7fa700 1 -- 172.21.15.78:0/3652085605 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4558053dc0 con 0x1c7ac10 2023-10-11T09:46:08.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.271+0000 7f454d7fa700 1 -- 172.21.15.78:0/3652085605 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4558055c80 con 0x1c7ac10 2023-10-11T09:46:08.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.271+0000 7f4570c02740 1 -- 172.21.15.78:0/3652085605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c7ac10 msgr2=0x1f105c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:08.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.271+0000 7f4570c02740 1 --2- 172.21.15.78:0/3652085605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c7ac10 0x1f105c0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f4558051f20 tx=0x7f45580561a0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.271+0000 7f4570c02740 1 -- 172.21.15.78:0/3652085605 shutdown_connections 2023-10-11T09:46:08.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.271+0000 7f4570c02740 1 --2- 172.21.15.78:0/3652085605 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1cc58c0 0x1f320f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.271+0000 7f4570c02740 1 --2- 172.21.15.78:0/3652085605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1c7a840 0x1f31bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.271+0000 7f4570c02740 1 --2- 172.21.15.78:0/3652085605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c7ac10 0x1f105c0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.271+0000 7f4570c02740 1 -- 172.21.15.78:0/3652085605 >> 172.21.15.78:0/3652085605 conn(0x1c7afe0 msgr2=0x1f13350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:46:08.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.271+0000 7f4570c02740 1 -- 172.21.15.78:0/3652085605 shutdown_connections 2023-10-11T09:46:08.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.271+0000 7f4570c02740 1 -- 172.21.15.78:0/3652085605 wait complete. 2023-10-11T09:46:08.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.271+0000 7f4570c02740 1 -- start start 2023-10-11T09:46:08.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.271+0000 7f4570c02740 1 Processor -- start 2023-10-11T09:46:08.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.275+0000 7f4570c02740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c7a840 0x1ee5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:08.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.275+0000 7f4570c02740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1c7ac10 0x1f08b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:08.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.275+0000 7f4570c02740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1cc58c0 0x1f090d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:08.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.275+0000 7f4570c02740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e16c10 con 0x1c7a840 2023-10-11T09:46:08.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.275+0000 7f4570c02740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e168b0 con 0x1cc58c0 2023-10-11T09:46:08.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.275+0000 7f4570c02740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x1cffc50 con 0x1c7ac10 2023-10-11T09:46:08.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.275+0000 7f4570c02740 10 client.0 _ll_register_callbacks cb 0x1e4cff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:46:08.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.275+0000 7f4566ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c7a840 0x1ee5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:08.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.275+0000 7f4566ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c7a840 0x1ee5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43736/0 (socket says 172.21.15.78:43736) 2023-10-11T09:46:08.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.275+0000 7f4566ffd700 1 -- 172.21.15.78:0/2111087849 learned_addr learned my addr 172.21.15.78:0/2111087849 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:46:08.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.275+0000 7f45677fe700 1 --2- 172.21.15.78:0/2111087849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1cc58c0 0x1f090d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:08.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.275+0000 7f45667fc700 1 --2- 172.21.15.78:0/2111087849 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1c7ac10 0x1f08b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:08.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.275+0000 7f4566ffd700 1 -- 172.21.15.78:0/2111087849 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1c7ac10 msgr2=0x1f08b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:08.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.275+0000 7f4566ffd700 1 --2- 172.21.15.78:0/2111087849 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1c7ac10 0x1f08b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.275+0000 7f4566ffd700 1 -- 172.21.15.78:0/2111087849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1cc58c0 msgr2=0x1f090d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:08.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.275+0000 7f4566ffd700 1 --2- 172.21.15.78:0/2111087849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1cc58c0 0x1f090d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.275+0000 7f4566ffd700 1 -- 172.21.15.78:0/2111087849 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f456815c450 con 0x1c7a840 2023-10-11T09:46:08.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.275+0000 7f4566ffd700 1 --2- 172.21.15.78:0/2111087849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c7a840 0x1ee5c10 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f4558006cf0 tx=0x7f4558084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c9670aeff27e7a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:08.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.275+0000 7f45467fc700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:46:08.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.275+0000 7f45467fc700 1 -- 172.21.15.78:0/2111087849 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455805da00 con 0x1c7a840 2023-10-11T09:46:08.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.275+0000 7f45467fc700 1 -- 172.21.15.78:0/2111087849 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f4558052130 con 0x1c7a840 2023-10-11T09:46:08.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.275+0000 7f4570c02740 1 -- 172.21.15.78:0/2111087849 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1e4c730 con 0x1c7a840 2023-10-11T09:46:08.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.275+0000 7f45467fc700 1 -- 172.21.15.78:0/2111087849 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45580522f0 con 0x1c7a840 2023-10-11T09:46:08.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.275+0000 7f4570c02740 10 client.6990 fetch_fsmap learned FSMap version 50 2023-10-11T09:46:08.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.275+0000 7f4570c02740 1 -- 172.21.15.78:0/2111087849 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x1cc6b10 con 0x1c7a840 2023-10-11T09:46:08.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.275+0000 7f45467fc700 1 -- 172.21.15.78:0/2111087849 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1e4c730 con 0x1c7a840 2023-10-11T09:46:08.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.275+0000 7f45467fc700 1 -- 172.21.15.78:0/2111087849 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 50) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f4558054f10 con 0x1c7a840 2023-10-11T09:46:08.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.275+0000 7f4570c02740 10 client.6990 fetch_fsmap finished waiting for FSMap version 50 2023-10-11T09:46:08.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.275+0000 7f4570c02740 10 client.6990 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-10-11T09:46:08.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.275+0000 7f4570c02740 20 client.6990 populate_metadata read hostname 'smithi078' 2023-10-11T09:46:08.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.275+0000 7f4570c02740 1 --2- 172.21.15.78:0/2111087849 >> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] conn(0x1ee7150 0x1ee7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:08.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.275+0000 7f4570c02740 4 client.6990 mds_command: new command op to 6897 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-11T09:46:08.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.275+0000 7f4570c02740 1 -- 172.21.15.78:0/2111087849 --> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1e4c450 con 0x1ee7150 2023-10-11T09:46:08.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.275+0000 7f45667fc700 1 --2- 172.21.15.78:0/2111087849 >> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] conn(0x1ee7150 0x1ee7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:08.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.275+0000 7f45667fc700 1 --2- 172.21.15.78:0/2111087849 >> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] conn(0x1ee7150 0x1ee7550 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea6f40b8324a9fcb server_cookie=df4b046beef600c0 in_seq=0 out_seq=0 2023-10-11T09:46:08.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.275+0000 7f45467fc700 10 client.6990 ms_handle_connect on v2:172.21.15.78:6835/3498868661 2023-10-11T09:46:08.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.279+0000 7f45467fc700 1 -- 172.21.15.78:0/2111087849 <== mds.0 v2:172.21.15.78:6835/3498868661 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x1e4c450 con 0x1ee7150 2023-10-11T09:46:08.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.279+0000 7f45467fc700 10 client.6990 handle_command_reply: tid=0 2023-10-11T09:46:08.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.279+0000 7f4570c02740 1 client.6990 shutdown 2023-10-11T09:46:08.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.279+0000 7f4570c02740 1 -- 172.21.15.78:0/2111087849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c7a840 msgr2=0x1ee5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:08.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.279+0000 7f4570c02740 1 --2- 172.21.15.78:0/2111087849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c7a840 0x1ee5c10 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f4558006cf0 tx=0x7f4558084040 comp rx=0 tx=0).stop 2023-10-11T09:46:08.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.279+0000 7f4570c02740 1 -- 172.21.15.78:0/2111087849 shutdown_connections 2023-10-11T09:46:08.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.279+0000 7f4570c02740 1 --2- 172.21.15.78:0/2111087849 >> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] conn(0x1ee7150 0x1ee7550 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.279+0000 7f4570c02740 1 --2- 172.21.15.78:0/2111087849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1cc58c0 0x1f090d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.279+0000 7f4570c02740 1 --2- 172.21.15.78:0/2111087849 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1c7ac10 0x1f08b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.279+0000 7f4570c02740 1 --2- 172.21.15.78:0/2111087849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c7a840 0x1ee5c10 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.279+0000 7f45467fc700 0 client.6990 ms_handle_reset on v2:172.21.15.78:6835/3498868661 2023-10-11T09:46:08.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.279+0000 7f4570c02740 1 -- 172.21.15.78:0/2111087849 >> 172.21.15.78:0/2111087849 conn(0x1c7afe0 msgr2=0x1f13710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:46:08.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.279+0000 7f4570c02740 1 -- 172.21.15.78:0/2111087849 shutdown_connections 2023-10-11T09:46:08.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.279+0000 7f4570c02740 1 -- 172.21.15.78:0/2111087849 wait complete. 2023-10-11T09:46:08.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.279+0000 7f4570c02740 20 client.6990 trim_cache size 0 max 16384 2023-10-11T09:46:08.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.299+0000 7f4570c02740 1 Processor -- start 2023-10-11T09:46:08.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.299+0000 7f4570c02740 1 -- start start 2023-10-11T09:46:08.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.299+0000 7f4570c02740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1c7a840 0x1f88550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:08.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.299+0000 7f4570c02740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1c7ac10 0x1f88a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:08.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.299+0000 7f4570c02740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1cc58c0 0x1f86260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:08.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.299+0000 7f4570c02740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e54c40 con 0x1cc58c0 2023-10-11T09:46:08.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.299+0000 7f4570c02740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e13b60 con 0x1c7a840 2023-10-11T09:46:08.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.299+0000 7f4570c02740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e20920 con 0x1c7ac10 2023-10-11T09:46:08.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.299+0000 7f4566ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1c7a840 0x1f88550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:08.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.299+0000 7f45677fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1cc58c0 0x1f86260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:08.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.299+0000 7f4566ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1c7a840 0x1f88550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:59586/0 (socket says 172.21.15.78:59586) 2023-10-11T09:46:08.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.299+0000 7f4566ffd700 1 -- 172.21.15.78:0/3896510317 learned_addr learned my addr 172.21.15.78:0/3896510317 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:46:08.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.299+0000 7f45667fc700 1 --2- 172.21.15.78:0/3896510317 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1c7ac10 0x1f88a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:08.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.299+0000 7f4566ffd700 1 -- 172.21.15.78:0/3896510317 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1c7ac10 msgr2=0x1f88a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:08.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.299+0000 7f4566ffd700 1 --2- 172.21.15.78:0/3896510317 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1c7ac10 0x1f88a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.299+0000 7f4566ffd700 1 -- 172.21.15.78:0/3896510317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1cc58c0 msgr2=0x1f86260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:08.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.299+0000 7f4566ffd700 1 --2- 172.21.15.78:0/3896510317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1cc58c0 0x1f86260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.299+0000 7f4566ffd700 1 -- 172.21.15.78:0/3896510317 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45681e6a10 con 0x1c7a840 2023-10-11T09:46:08.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.299+0000 7f4566ffd700 1 --2- 172.21.15.78:0/3896510317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1c7a840 0x1f88550 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4558003fd0 tx=0x7f455801c030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbac2d11dad9117e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:08.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.299+0000 7f454dffb700 1 -- 172.21.15.78:0/3896510317 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4558055380 con 0x1c7a840 2023-10-11T09:46:08.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.299+0000 7f454dffb700 1 -- 172.21.15.78:0/3896510317 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4558055a20 con 0x1c7a840 2023-10-11T09:46:08.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.299+0000 7f454dffb700 1 -- 172.21.15.78:0/3896510317 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455805da00 con 0x1c7a840 2023-10-11T09:46:08.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.303+0000 7f4570c02740 1 -- 172.21.15.78:0/3896510317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1c7a840 msgr2=0x1f88550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:08.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.303+0000 7f4570c02740 1 --2- 172.21.15.78:0/3896510317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1c7a840 0x1f88550 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4558003fd0 tx=0x7f455801c030 comp rx=0 tx=0).stop 2023-10-11T09:46:08.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.303+0000 7f4570c02740 1 -- 172.21.15.78:0/3896510317 shutdown_connections 2023-10-11T09:46:08.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.303+0000 7f4570c02740 1 --2- 172.21.15.78:0/3896510317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1cc58c0 0x1f86260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.303+0000 7f4570c02740 1 --2- 172.21.15.78:0/3896510317 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1c7ac10 0x1f88a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.303+0000 7f4570c02740 1 --2- 172.21.15.78:0/3896510317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1c7a840 0x1f88550 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.303+0000 7f4570c02740 1 -- 172.21.15.78:0/3896510317 >> 172.21.15.78:0/3896510317 conn(0x1c7afe0 msgr2=0x1f12ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:46:08.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.303+0000 7f4570c02740 1 -- 172.21.15.78:0/3896510317 shutdown_connections 2023-10-11T09:46:08.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.303+0000 7f4570c02740 1 -- 172.21.15.78:0/3896510317 wait complete. 2023-10-11T09:46:08.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.303+0000 7f4570c02740 1 -- start start 2023-10-11T09:46:08.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.303+0000 7f4570c02740 1 Processor -- start 2023-10-11T09:46:08.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.303+0000 7f4570c02740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1c7a840 0x1f3b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:08.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.303+0000 7f4570c02740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1c7ac10 0x1f85350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:08.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.303+0000 7f4570c02740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1cc58c0 0x1f85890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:08.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.303+0000 7f4570c02740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f85dd0 con 0x1cc58c0 2023-10-11T09:46:08.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.303+0000 7f4570c02740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f3ba10 con 0x1c7ac10 2023-10-11T09:46:08.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.303+0000 7f4570c02740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x1f3a9f0 con 0x1c7a840 2023-10-11T09:46:08.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.303+0000 7f4570c02740 10 client.0 _ll_register_callbacks cb 0x1e51e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:46:08.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.303+0000 7f45677fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1cc58c0 0x1f85890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:08.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.303+0000 7f45677fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1cc58c0 0x1f85890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43758/0 (socket says 172.21.15.78:43758) 2023-10-11T09:46:08.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.303+0000 7f45677fe700 1 -- 172.21.15.78:0/2676280227 learned_addr learned my addr 172.21.15.78:0/2676280227 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:46:08.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.303+0000 7f4566ffd700 1 --2- 172.21.15.78:0/2676280227 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1c7a840 0x1f3b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:08.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.303+0000 7f45667fc700 1 --2- 172.21.15.78:0/2676280227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1c7ac10 0x1f85350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:08.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.307+0000 7f45677fe700 1 -- 172.21.15.78:0/2676280227 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1c7a840 msgr2=0x1f3b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:08.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.307+0000 7f45677fe700 1 --2- 172.21.15.78:0/2676280227 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1c7a840 0x1f3b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.307+0000 7f45677fe700 1 -- 172.21.15.78:0/2676280227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1c7ac10 msgr2=0x1f85350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:08.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.307+0000 7f45677fe700 1 --2- 172.21.15.78:0/2676280227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1c7ac10 0x1f85350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.307+0000 7f45677fe700 1 -- 172.21.15.78:0/2676280227 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1cc6b10 con 0x1cc58c0 2023-10-11T09:46:08.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.307+0000 7f45677fe700 1 --2- 172.21.15.78:0/2676280227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1cc58c0 0x1f85890 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f45600149d0 tx=0x7f4560001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626fc35825462ca7 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:08.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.307+0000 7f45467fc700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:46:08.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.307+0000 7f45467fc700 1 -- 172.21.15.78:0/2676280227 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45600126e0 con 0x1cc58c0 2023-10-11T09:46:08.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.307+0000 7f4570c02740 1 -- 172.21.15.78:0/2676280227 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4558053dc0 con 0x1cc58c0 2023-10-11T09:46:08.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.307+0000 7f45467fc700 1 -- 172.21.15.78:0/2676280227 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f4560012840 con 0x1cc58c0 2023-10-11T09:46:08.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.307+0000 7f45467fc700 1 -- 172.21.15.78:0/2676280227 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4560012a00 con 0x1cc58c0 2023-10-11T09:46:08.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.307+0000 7f4570c02740 10 client.6996 fetch_fsmap learned FSMap version 50 2023-10-11T09:46:08.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.307+0000 7f4570c02740 1 -- 172.21.15.78:0/2676280227 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x1e19820 con 0x1cc58c0 2023-10-11T09:46:08.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.307+0000 7f45467fc700 1 -- 172.21.15.78:0/2676280227 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4558053dc0 con 0x1cc58c0 2023-10-11T09:46:08.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.307+0000 7f45467fc700 1 -- 172.21.15.78:0/2676280227 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 50) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f456001f050 con 0x1cc58c0 2023-10-11T09:46:08.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.307+0000 7f4570c02740 10 client.6996 fetch_fsmap finished waiting for FSMap version 50 2023-10-11T09:46:08.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.307+0000 7f4570c02740 10 client.6996 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-10-11T09:46:08.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.307+0000 7f4570c02740 20 client.6996 populate_metadata read hostname 'smithi078' 2023-10-11T09:46:08.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.307+0000 7f4570c02740 1 --2- 172.21.15.78:0/2676280227 >> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] conn(0x1f47220 0x1f47620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:08.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.307+0000 7f4570c02740 4 client.6996 mds_command: new command op to 6897 tid=0[{"prefix": "scrub status"}] 2023-10-11T09:46:08.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.307+0000 7f4570c02740 1 -- 172.21.15.78:0/2676280227 --> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x1cd2610 con 0x1f47220 2023-10-11T09:46:08.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.307+0000 7f4566ffd700 1 --2- 172.21.15.78:0/2676280227 >> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] conn(0x1f47220 0x1f47620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:08.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.307+0000 7f45467fc700 10 client.6996 ms_handle_connect on v2:172.21.15.78:6835/3498868661 2023-10-11T09:46:08.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.307+0000 7f4566ffd700 1 --2- 172.21.15.78:0/2676280227 >> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] conn(0x1f47220 0x1f47620 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=684a14c90e0e9052 server_cookie=63125de3895c3c21 in_seq=0 out_seq=0 2023-10-11T09:46:08.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.307+0000 7f45467fc700 1 -- 172.21.15.78:0/2676280227 <== mds.0 v2:172.21.15.78:6835/3498868661 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x1cd2610 con 0x1f47220 2023-10-11T09:46:08.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.307+0000 7f45467fc700 10 client.6996 handle_command_reply: tid=0 2023-10-11T09:46:08.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.307+0000 7f4570c02740 1 client.6996 shutdown 2023-10-11T09:46:08.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.307+0000 7f4570c02740 1 -- 172.21.15.78:0/2676280227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1cc58c0 msgr2=0x1f85890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:08.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.307+0000 7f4570c02740 1 --2- 172.21.15.78:0/2676280227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1cc58c0 0x1f85890 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f45600149d0 tx=0x7f4560001250 comp rx=0 tx=0).stop 2023-10-11T09:46:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.311+0000 7f4570c02740 1 -- 172.21.15.78:0/2676280227 shutdown_connections 2023-10-11T09:46:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.311+0000 7f4570c02740 1 --2- 172.21.15.78:0/2676280227 >> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] conn(0x1f47220 0x1f47620 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.311+0000 7f4570c02740 1 --2- 172.21.15.78:0/2676280227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1cc58c0 0x1f85890 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.311+0000 7f4570c02740 1 --2- 172.21.15.78:0/2676280227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1c7ac10 0x1f85350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.311+0000 7f45467fc700 0 client.6996 ms_handle_reset on v2:172.21.15.78:6835/3498868661 2023-10-11T09:46:08.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.311+0000 7f4570c02740 1 --2- 172.21.15.78:0/2676280227 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1c7a840 0x1f3b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.311+0000 7f4570c02740 1 -- 172.21.15.78:0/2676280227 >> 172.21.15.78:0/2676280227 conn(0x1c7afe0 msgr2=0x1f86260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:46:08.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.311+0000 7f4570c02740 1 -- 172.21.15.78:0/2676280227 shutdown_connections 2023-10-11T09:46:08.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.311+0000 7f4570c02740 1 -- 172.21.15.78:0/2676280227 wait complete. 2023-10-11T09:46:08.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.311+0000 7f4570c02740 20 client.6996 trim_cache size 0 max 16384 2023-10-11T09:46:08.327 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:46:08.327 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-11T09:46:08.328 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2023-10-11T09:46:08.328 INFO:teuthology.orchestra.run.smithi078.stdout: "df3143e9-5161-4e91-93ef-2c0b6488c02b": { 2023-10-11T09:46:08.328 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2023-10-11T09:46:08.328 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "df3143e9-5161-4e91-93ef-2c0b6488c02b", 2023-10-11T09:46:08.328 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,repair" 2023-10-11T09:46:08.329 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:46:08.329 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:46:08.329 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:46:08.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.311+0000 7f456d978700 1 -- 172.21.15.78:0/708004576 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4554065f00 msgr2=0x7f45540683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:08.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.311+0000 7f456d978700 1 --2- 172.21.15.78:0/708004576 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4554065f00 0x7f45540683b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f45600012e0 tx=0x7f4560009040 comp rx=0 tx=0).stop 2023-10-11T09:46:08.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.311+0000 7f456d978700 1 -- 172.21.15.78:0/708004576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45680b84c0 msgr2=0x7f45680c97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:08.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.311+0000 7f456d978700 1 --2- 172.21.15.78:0/708004576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45680b84c0 0x7f45680c97b0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f4558000c00 tx=0x7f4558004770 comp rx=0 tx=0).stop 2023-10-11T09:46:08.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.311+0000 7f456d978700 1 -- 172.21.15.78:0/708004576 shutdown_connections 2023-10-11T09:46:08.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.311+0000 7f456d978700 1 --2- 172.21.15.78:0/708004576 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4554065f00 0x7f45540683b0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.311+0000 7f456d978700 1 --2- 172.21.15.78:0/708004576 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f45681608b0 0x7f45681e6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.311+0000 7f456d978700 1 --2- 172.21.15.78:0/708004576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45680b8890 0x7f45681e5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.311+0000 7f456d978700 1 --2- 172.21.15.78:0/708004576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45680b84c0 0x7f45680c97b0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:08.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.311+0000 7f456d978700 1 -- 172.21.15.78:0/708004576 >> 172.21.15.78:0/708004576 conn(0x7f456806cb50 msgr2=0x7f456806b140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:46:08.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.311+0000 7f456d978700 1 -- 172.21.15.78:0/708004576 shutdown_connections 2023-10-11T09:46:08.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:08.311+0000 7f456d978700 1 -- 172.21.15.78:0/708004576 wait complete. 2023-10-11T09:46:08.338 INFO:tasks.cephfs.filesystem:scrub status for tag:df3143e9-5161-4e91-93ef-2c0b6488c02b - {'path': '/', 'tag': 'df3143e9-5161-4e91-93ef-2c0b6488c02b', 'options': 'recursive,repair'} 2023-10-11T09:46:30.036 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:46:30.043 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:46:38.339 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-10-11T09:46:38.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.519+0000 7fd194d37700 1 Processor -- start 2023-10-11T09:46:38.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.523+0000 7fd194d37700 1 -- start start 2023-10-11T09:46:38.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.523+0000 7fd194d37700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fd1900b3240 0x7fd1900d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:38.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.523+0000 7fd194d37700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd1900b3610 0x7fd1900b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:38.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.523+0000 7fd194d37700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd19015d040 0x7fd19015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:38.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.523+0000 7fd194d37700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1900b3fd0 con 0x7fd1900b3240 2023-10-11T09:46:38.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.523+0000 7fd194d37700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1900b47f0 con 0x7fd19015d040 2023-10-11T09:46:38.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.523+0000 7fd194d37700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1900b8130 con 0x7fd1900b3610 2023-10-11T09:46:38.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.523+0000 7fd18f7fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fd1900b3240 0x7fd1900d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:38.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.523+0000 7fd18f7fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fd1900b3240 0x7fd1900d29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35102/0 (socket says 172.21.15.78:35102) 2023-10-11T09:46:38.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.523+0000 7fd18f7fe700 1 -- 172.21.15.78:0/3150957799 learned_addr learned my addr 172.21.15.78:0/3150957799 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:46:38.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.523+0000 7fd18ffff700 1 --2- 172.21.15.78:0/3150957799 >> v2:172.21.15.133:3300/0 conn(0x7fd19015d040 0x7fd19015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:38.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.523+0000 7fd18effd700 1 --2- 172.21.15.78:0/3150957799 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd1900b3610 0x7fd1900b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:38.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.523+0000 7fd18f7fe700 1 -- 172.21.15.78:0/3150957799 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd1900b3610 msgr2=0x7fd1900b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:38.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.523+0000 7fd18f7fe700 1 --2- 172.21.15.78:0/3150957799 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd1900b3610 0x7fd1900b3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:38.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.523+0000 7fd18f7fe700 1 -- 172.21.15.78:0/3150957799 >> v2:172.21.15.133:3300/0 conn(0x7fd19015d040 msgr2=0x7fd19015d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:38.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.523+0000 7fd18f7fe700 1 --2- 172.21.15.78:0/3150957799 >> v2:172.21.15.133:3300/0 conn(0x7fd19015d040 0x7fd19015d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:38.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.523+0000 7fd18f7fe700 1 -- 172.21.15.78:0/3150957799 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1900b41e0 con 0x7fd1900b3240 2023-10-11T09:46:38.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.523+0000 7fd18ffff700 1 --2- 172.21.15.78:0/3150957799 >> v2:172.21.15.133:3300/0 conn(0x7fd19015d040 0x7fd19015d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:46:38.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.523+0000 7fd18f7fe700 1 --2- 172.21.15.78:0/3150957799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1900b3240 0x7fd1900d29d0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fd1900672a0 tx=0x7fd180009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aacc7e4983beda22 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:38.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.523+0000 7fd18e7fc700 1 -- 172.21.15.78:0/3150957799 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18000b8a0 con 0x7fd1900b3240 2023-10-11T09:46:38.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.523+0000 7fd18e7fc700 1 -- 172.21.15.78:0/3150957799 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18000ba00 con 0x7fd1900b3240 2023-10-11T09:46:38.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.523+0000 7fd18e7fc700 1 -- 172.21.15.78:0/3150957799 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18000bd90 con 0x7fd1900b3240 2023-10-11T09:46:38.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.527+0000 7fd194d37700 1 -- 172.21.15.78:0/3150957799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1900b3240 msgr2=0x7fd1900d29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:38.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.527+0000 7fd194d37700 1 --2- 172.21.15.78:0/3150957799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1900b3240 0x7fd1900d29d0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fd1900672a0 tx=0x7fd180009bf0 comp rx=0 tx=0).stop 2023-10-11T09:46:38.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.527+0000 7fd194d37700 1 -- 172.21.15.78:0/3150957799 shutdown_connections 2023-10-11T09:46:38.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.527+0000 7fd194d37700 1 --2- 172.21.15.78:0/3150957799 >> v2:172.21.15.133:3300/0 conn(0x7fd19015d040 0x7fd19015d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:38.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.527+0000 7fd194d37700 1 --2- 172.21.15.78:0/3150957799 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd1900b3610 0x7fd1900b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:38.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.527+0000 7fd194d37700 1 --2- 172.21.15.78:0/3150957799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1900b3240 0x7fd1900d29d0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:38.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.527+0000 7fd194d37700 1 -- 172.21.15.78:0/3150957799 >> 172.21.15.78:0/3150957799 conn(0x7fd19009c260 msgr2=0x7fd1900d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:46:38.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.527+0000 7fd194d37700 1 -- 172.21.15.78:0/3150957799 shutdown_connections 2023-10-11T09:46:38.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.527+0000 7fd194d37700 1 -- 172.21.15.78:0/3150957799 wait complete. 2023-10-11T09:46:38.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.527+0000 7fd194d37700 1 Processor -- start 2023-10-11T09:46:38.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.527+0000 7fd194d37700 1 -- start start 2023-10-11T09:46:38.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.527+0000 7fd194d37700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1900b3240 0x7fd19009fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:38.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.527+0000 7fd194d37700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1900b3610 0x7fd1901f5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:38.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.527+0000 7fd194d37700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd19015d040 0x7fd19009dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:38.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.527+0000 7fd194d37700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1901f5ff0 con 0x7fd1900b3610 2023-10-11T09:46:38.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.527+0000 7fd194d37700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1900a0170 con 0x7fd1900b3240 2023-10-11T09:46:38.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.527+0000 7fd194d37700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1900a0450 con 0x7fd19015d040 2023-10-11T09:46:38.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.531+0000 7fd18effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1900b3610 0x7fd1901f5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:38.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.531+0000 7fd18effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1900b3610 0x7fd1901f5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35112/0 (socket says 172.21.15.78:35112) 2023-10-11T09:46:38.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.531+0000 7fd18effd700 1 -- 172.21.15.78:0/1437808562 learned_addr learned my addr 172.21.15.78:0/1437808562 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:46:38.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.531+0000 7fd18f7fe700 1 --2- 172.21.15.78:0/1437808562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1900b3240 0x7fd19009fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:38.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.531+0000 7fd18ffff700 1 --2- 172.21.15.78:0/1437808562 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd19015d040 0x7fd19009dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:38.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.531+0000 7fd18effd700 1 -- 172.21.15.78:0/1437808562 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd19015d040 msgr2=0x7fd19009dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:38.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.531+0000 7fd18effd700 1 --2- 172.21.15.78:0/1437808562 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd19015d040 0x7fd19009dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:38.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.531+0000 7fd18effd700 1 -- 172.21.15.78:0/1437808562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1900b3240 msgr2=0x7fd19009fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:38.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.531+0000 7fd18effd700 1 --2- 172.21.15.78:0/1437808562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1900b3240 0x7fd19009fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:38.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.531+0000 7fd18effd700 1 -- 172.21.15.78:0/1437808562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd180009570 con 0x7fd1900b3610 2023-10-11T09:46:38.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.531+0000 7fd18effd700 1 --2- 172.21.15.78:0/1437808562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1900b3610 0x7fd1901f5ab0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fd18400d7d0 tx=0x7fd18400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:38.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.531+0000 7fd177fff700 1 -- 172.21.15.78:0/1437808562 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1840125f0 con 0x7fd1900b3610 2023-10-11T09:46:38.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.531+0000 7fd177fff700 1 -- 172.21.15.78:0/1437808562 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd184012750 con 0x7fd1900b3610 2023-10-11T09:46:38.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.531+0000 7fd194d37700 1 -- 172.21.15.78:0/1437808562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1900a0730 con 0x7fd1900b3610 2023-10-11T09:46:38.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.531+0000 7fd177fff700 1 -- 172.21.15.78:0/1437808562 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd184012ae0 con 0x7fd1900b3610 2023-10-11T09:46:38.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.531+0000 7fd194d37700 1 -- 172.21.15.78:0/1437808562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1900b13e0 con 0x7fd1900b3610 2023-10-11T09:46:38.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.535+0000 7fd177fff700 1 -- 172.21.15.78:0/1437808562 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fd184012c40 con 0x7fd1900b3610 2023-10-11T09:46:38.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.535+0000 7fd177fff700 1 --2- 172.21.15.78:0/1437808562 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd17c065ed0 0x7fd17c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:38.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.535+0000 7fd177fff700 1 -- 172.21.15.78:0/1437808562 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6724+0+0 (secure 0 0 0) 0x7fd184083a80 con 0x7fd1900b3610 2023-10-11T09:46:38.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.535+0000 7fd18f7fe700 1 --2- 172.21.15.78:0/1437808562 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd17c065ed0 0x7fd17c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:38.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.535+0000 7fd18f7fe700 1 --2- 172.21.15.78:0/1437808562 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd17c065ed0 0x7fd17c068380 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd180000c00 tx=0x7fd180008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:38.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.535+0000 7fd19922a740 1 Processor -- start 2023-10-11T09:46:38.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.539+0000 7fd19922a740 1 -- start start 2023-10-11T09:46:38.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.539+0000 7fd19922a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x163dc10 0x18d35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:38.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.539+0000 7fd19922a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x163d840 0x18f4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:38.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.539+0000 7fd19922a740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x16888c0 0x18f52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:38.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.539+0000 7fd19922a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x181d630 con 0x163dc10 2023-10-11T09:46:38.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.539+0000 7fd19922a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x181adc0 con 0x163d840 2023-10-11T09:46:38.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.539+0000 7fd19922a740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x18166b0 con 0x16888c0 2023-10-11T09:46:38.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.539+0000 7fd18f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x163dc10 0x18d35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:38.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.539+0000 7fd18f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x163dc10 0x18d35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35120/0 (socket says 172.21.15.78:35120) 2023-10-11T09:46:38.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.539+0000 7fd18f7fe700 1 -- 172.21.15.78:0/526142987 learned_addr learned my addr 172.21.15.78:0/526142987 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:46:38.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.539+0000 7fd18ffff700 1 --2- 172.21.15.78:0/526142987 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x16888c0 0x18f52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:38.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.539+0000 7fd18effd700 1 --2- 172.21.15.78:0/526142987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x163d840 0x18f4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:38.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.539+0000 7fd18f7fe700 1 -- 172.21.15.78:0/526142987 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x16888c0 msgr2=0x18f52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:38.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.539+0000 7fd18f7fe700 1 --2- 172.21.15.78:0/526142987 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x16888c0 0x18f52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:38.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.539+0000 7fd18f7fe700 1 -- 172.21.15.78:0/526142987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x163d840 msgr2=0x18f4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:38.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.539+0000 7fd18f7fe700 1 --2- 172.21.15.78:0/526142987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x163d840 0x18f4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:38.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.539+0000 7fd18f7fe700 1 -- 172.21.15.78:0/526142987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x183c5b0 con 0x163dc10 2023-10-11T09:46:38.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.539+0000 7fd18effd700 1 --2- 172.21.15.78:0/526142987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x163d840 0x18f4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:46:38.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.539+0000 7fd18f7fe700 1 --2- 172.21.15.78:0/526142987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x163dc10 0x18d35a0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fd180006e20 tx=0x7fd180010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f4bace8c55da54b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:38.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.539+0000 7fd1757fa700 1 -- 172.21.15.78:0/526142987 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18001f070 con 0x163dc10 2023-10-11T09:46:38.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.539+0000 7fd1757fa700 1 -- 172.21.15.78:0/526142987 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1800036c0 con 0x163dc10 2023-10-11T09:46:38.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.539+0000 7fd1757fa700 1 -- 172.21.15.78:0/526142987 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd180003a30 con 0x163dc10 2023-10-11T09:46:38.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.543+0000 7fd19922a740 1 -- 172.21.15.78:0/526142987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x163dc10 msgr2=0x18d35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:38.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.543+0000 7fd19922a740 1 --2- 172.21.15.78:0/526142987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x163dc10 0x18d35a0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fd180006e20 tx=0x7fd180010000 comp rx=0 tx=0).stop 2023-10-11T09:46:38.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.543+0000 7fd19922a740 1 -- 172.21.15.78:0/526142987 shutdown_connections 2023-10-11T09:46:38.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.543+0000 7fd19922a740 1 --2- 172.21.15.78:0/526142987 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x16888c0 0x18f52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:38.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.543+0000 7fd19922a740 1 --2- 172.21.15.78:0/526142987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x163d840 0x18f4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:38.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.543+0000 7fd19922a740 1 --2- 172.21.15.78:0/526142987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x163dc10 0x18d35a0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:38.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.543+0000 7fd19922a740 1 -- 172.21.15.78:0/526142987 >> 172.21.15.78:0/526142987 conn(0x163dfe0 msgr2=0x18d6350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:46:38.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.543+0000 7fd19922a740 1 -- 172.21.15.78:0/526142987 shutdown_connections 2023-10-11T09:46:38.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.543+0000 7fd19922a740 1 -- 172.21.15.78:0/526142987 wait complete. 2023-10-11T09:46:38.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.543+0000 7fd19922a740 1 -- start start 2023-10-11T09:46:38.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.543+0000 7fd19922a740 1 Processor -- start 2023-10-11T09:46:38.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.547+0000 7fd19922a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x163d840 0x18cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:38.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.547+0000 7fd19922a740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x163dc10 0x18d24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:38.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.547+0000 7fd19922a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16888c0 0x18d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:38.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.547+0000 7fd19922a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x17d9c10 con 0x16888c0 2023-10-11T09:46:38.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.547+0000 7fd19922a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x17d98b0 con 0x163d840 2023-10-11T09:46:38.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.547+0000 7fd19922a740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x16c2c50 con 0x163dc10 2023-10-11T09:46:38.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.547+0000 7fd19922a740 10 client.0 _ll_register_callbacks cb 0x180fff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:46:38.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.547+0000 7fd18ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16888c0 0x18d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:38.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.547+0000 7fd18ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16888c0 0x18d2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35134/0 (socket says 172.21.15.78:35134) 2023-10-11T09:46:38.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.547+0000 7fd18ffff700 1 -- 172.21.15.78:0/3605969178 learned_addr learned my addr 172.21.15.78:0/3605969178 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:46:38.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.547+0000 7fd18f7fe700 1 --2- 172.21.15.78:0/3605969178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x163d840 0x18cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:38.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.547+0000 7fd18effd700 1 --2- 172.21.15.78:0/3605969178 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x163dc10 0x18d24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:38.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.547+0000 7fd18ffff700 1 -- 172.21.15.78:0/3605969178 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x163dc10 msgr2=0x18d24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:38.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.547+0000 7fd18ffff700 1 --2- 172.21.15.78:0/3605969178 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x163dc10 0x18d24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:38.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.547+0000 7fd18ffff700 1 -- 172.21.15.78:0/3605969178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x163d840 msgr2=0x18cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:38.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.547+0000 7fd18ffff700 1 --2- 172.21.15.78:0/3605969178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x163d840 0x18cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:38.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.547+0000 7fd18ffff700 1 -- 172.21.15.78:0/3605969178 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1900b41e0 con 0x16888c0 2023-10-11T09:46:38.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.547+0000 7fd18effd700 1 --2- 172.21.15.78:0/3605969178 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x163dc10 0x18d24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:46:38.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.547+0000 7fd18f7fe700 1 --2- 172.21.15.78:0/3605969178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x163d840 0x18cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:46:38.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.547+0000 7fd18ffff700 1 --2- 172.21.15.78:0/3605969178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16888c0 0x18d2a00 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fd188004690 tx=0x7fd188004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d85844fdb578f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:38.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.547+0000 7fd16e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:46:38.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.547+0000 7fd16e7fc700 1 -- 172.21.15.78:0/3605969178 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1880079c0 con 0x16888c0 2023-10-11T09:46:38.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.547+0000 7fd19922a740 1 -- 172.21.15.78:0/3605969178 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x180f730 con 0x16888c0 2023-10-11T09:46:38.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.547+0000 7fd16e7fc700 1 -- 172.21.15.78:0/3605969178 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 6724+0+0 (secure 0 0 0) 0x7fd188007b20 con 0x16888c0 2023-10-11T09:46:38.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.547+0000 7fd16e7fc700 1 -- 172.21.15.78:0/3605969178 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd188007ce0 con 0x16888c0 2023-10-11T09:46:38.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.547+0000 7fd19922a740 10 client.7008 fetch_fsmap learned FSMap version 50 2023-10-11T09:46:38.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.547+0000 7fd19922a740 1 -- 172.21.15.78:0/3605969178 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x1689b10 con 0x16888c0 2023-10-11T09:46:38.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.547+0000 7fd16e7fc700 1 -- 172.21.15.78:0/3605969178 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x180f730 con 0x16888c0 2023-10-11T09:46:38.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.547+0000 7fd16e7fc700 1 -- 172.21.15.78:0/3605969178 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 50) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd188016640 con 0x16888c0 2023-10-11T09:46:38.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.547+0000 7fd19922a740 10 client.7008 fetch_fsmap finished waiting for FSMap version 50 2023-10-11T09:46:38.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.547+0000 7fd19922a740 10 client.7008 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-10-11T09:46:38.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.547+0000 7fd19922a740 20 client.7008 populate_metadata read hostname 'smithi078' 2023-10-11T09:46:38.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.547+0000 7fd19922a740 1 --2- 172.21.15.78:0/3605969178 >> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] conn(0x18b3810 0x1907c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:38.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.547+0000 7fd19922a740 4 client.7008 mds_command: new command op to 6897 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-11T09:46:38.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.547+0000 7fd19922a740 1 -- 172.21.15.78:0/3605969178 --> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x180f450 con 0x18b3810 2023-10-11T09:46:38.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.551+0000 7fd18f7fe700 1 --2- 172.21.15.78:0/3605969178 >> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] conn(0x18b3810 0x1907c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:38.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.551+0000 7fd16e7fc700 10 client.7008 ms_handle_connect on v2:172.21.15.78:6835/3498868661 2023-10-11T09:46:38.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.551+0000 7fd18f7fe700 1 --2- 172.21.15.78:0/3605969178 >> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] conn(0x18b3810 0x1907c90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24c151768c91d8c1 server_cookie=7bee106a253905a7 in_seq=0 out_seq=0 2023-10-11T09:46:38.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.551+0000 7fd16e7fc700 1 -- 172.21.15.78:0/3605969178 <== mds.0 v2:172.21.15.78:6835/3498868661 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x180f450 con 0x18b3810 2023-10-11T09:46:38.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.551+0000 7fd16e7fc700 10 client.7008 handle_command_reply: tid=0 2023-10-11T09:46:38.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.551+0000 7fd19922a740 1 client.7008 shutdown 2023-10-11T09:46:38.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.551+0000 7fd19922a740 1 -- 172.21.15.78:0/3605969178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16888c0 msgr2=0x18d2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:38.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.551+0000 7fd19922a740 1 --2- 172.21.15.78:0/3605969178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16888c0 0x18d2a00 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fd188004690 tx=0x7fd188004770 comp rx=0 tx=0).stop 2023-10-11T09:46:38.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.551+0000 7fd19922a740 1 -- 172.21.15.78:0/3605969178 shutdown_connections 2023-10-11T09:46:38.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.551+0000 7fd19922a740 1 --2- 172.21.15.78:0/3605969178 >> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] conn(0x18b3810 0x1907c90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:38.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.551+0000 7fd19922a740 1 --2- 172.21.15.78:0/3605969178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16888c0 0x18d2a00 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:38.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.551+0000 7fd19922a740 1 --2- 172.21.15.78:0/3605969178 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x163dc10 0x18d24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:38.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.551+0000 7fd19922a740 1 --2- 172.21.15.78:0/3605969178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x163d840 0x18cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:38.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.551+0000 7fd16e7fc700 0 client.7008 ms_handle_reset on v2:172.21.15.78:6835/3498868661 2023-10-11T09:46:38.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.551+0000 7fd19922a740 1 -- 172.21.15.78:0/3605969178 >> 172.21.15.78:0/3605969178 conn(0x163dfe0 msgr2=0x18d6750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:46:38.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.551+0000 7fd19922a740 1 -- 172.21.15.78:0/3605969178 shutdown_connections 2023-10-11T09:46:38.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.551+0000 7fd19922a740 1 -- 172.21.15.78:0/3605969178 wait complete. 2023-10-11T09:46:38.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.551+0000 7fd19922a740 20 client.7008 trim_cache size 0 max 16384 2023-10-11T09:46:38.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.575+0000 7fd19922a740 1 Processor -- start 2023-10-11T09:46:38.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.575+0000 7fd19922a740 1 -- start start 2023-10-11T09:46:38.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.575+0000 7fd19922a740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x163d840 0x18b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:38.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.575+0000 7fd19922a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x163dc10 0x18e05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:38.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.575+0000 7fd19922a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16888c0 0x18e0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:38.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.575+0000 7fd19922a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x17d6b60 con 0x16888c0 2023-10-11T09:46:38.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.575+0000 7fd19922a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x1815540 con 0x163dc10 2023-10-11T09:46:38.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.575+0000 7fd19922a740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x1819210 con 0x163d840 2023-10-11T09:46:38.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.575+0000 7fd18ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16888c0 0x18e0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:38.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.575+0000 7fd18ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16888c0 0x18e0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35138/0 (socket says 172.21.15.78:35138) 2023-10-11T09:46:38.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.575+0000 7fd18ffff700 1 -- 172.21.15.78:0/25761004 learned_addr learned my addr 172.21.15.78:0/25761004 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:46:38.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.575+0000 7fd18effd700 1 --2- 172.21.15.78:0/25761004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x163dc10 0x18e05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:38.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.575+0000 7fd18f7fe700 1 --2- 172.21.15.78:0/25761004 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x163d840 0x18b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:38.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.575+0000 7fd18ffff700 1 -- 172.21.15.78:0/25761004 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x163d840 msgr2=0x18b0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:38.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.575+0000 7fd18ffff700 1 --2- 172.21.15.78:0/25761004 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x163d840 0x18b0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:38.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.575+0000 7fd18ffff700 1 -- 172.21.15.78:0/25761004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x163dc10 msgr2=0x18e05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:38.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.575+0000 7fd18ffff700 1 --2- 172.21.15.78:0/25761004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x163dc10 0x18e05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:38.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.575+0000 7fd18ffff700 1 -- 172.21.15.78:0/25761004 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1900d0e20 con 0x16888c0 2023-10-11T09:46:38.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.579+0000 7fd18ffff700 1 --2- 172.21.15.78:0/25761004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16888c0 0x18e0b10 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fd188004250 tx=0x7fd188010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad3db5f7a7f6feef server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:38.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.579+0000 7fd175ffb700 1 -- 172.21.15.78:0/25761004 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd188009ce0 con 0x16888c0 2023-10-11T09:46:38.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.579+0000 7fd175ffb700 1 -- 172.21.15.78:0/25761004 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18800f040 con 0x16888c0 2023-10-11T09:46:38.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.579+0000 7fd175ffb700 1 -- 172.21.15.78:0/25761004 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd188016640 con 0x16888c0 2023-10-11T09:46:38.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.579+0000 7fd19922a740 1 -- 172.21.15.78:0/25761004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16888c0 msgr2=0x18e0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:38.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.579+0000 7fd19922a740 1 --2- 172.21.15.78:0/25761004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16888c0 0x18e0b10 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fd188004250 tx=0x7fd188010000 comp rx=0 tx=0).stop 2023-10-11T09:46:38.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.579+0000 7fd19922a740 1 -- 172.21.15.78:0/25761004 shutdown_connections 2023-10-11T09:46:38.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.579+0000 7fd19922a740 1 --2- 172.21.15.78:0/25761004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x16888c0 0x18e0b10 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:38.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.579+0000 7fd19922a740 1 --2- 172.21.15.78:0/25761004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x163dc10 0x18e05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:38.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.579+0000 7fd19922a740 1 --2- 172.21.15.78:0/25761004 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x163d840 0x18b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:38.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.579+0000 7fd19922a740 1 -- 172.21.15.78:0/25761004 >> 172.21.15.78:0/25761004 conn(0x163dfe0 msgr2=0x18d2780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:46:38.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.579+0000 7fd19922a740 1 -- 172.21.15.78:0/25761004 shutdown_connections 2023-10-11T09:46:38.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.579+0000 7fd19922a740 1 -- 172.21.15.78:0/25761004 wait complete. 2023-10-11T09:46:38.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.583+0000 7fd19922a740 1 -- start start 2023-10-11T09:46:38.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.583+0000 7fd19922a740 1 Processor -- start 2023-10-11T09:46:38.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.583+0000 7fd19922a740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x163d840 0x18cb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:38.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.583+0000 7fd19922a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x163dc10 0x1950610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:38.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.583+0000 7fd19922a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x16888c0 0x1950b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:38.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.583+0000 7fd19922a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x181e8c0 con 0x163dc10 2023-10-11T09:46:38.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.583+0000 7fd19922a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x1813ab0 con 0x16888c0 2023-10-11T09:46:38.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.583+0000 7fd19922a740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x1951090 con 0x163d840 2023-10-11T09:46:38.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.583+0000 7fd19922a740 10 client.0 _ll_register_callbacks cb 0x1814e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:46:38.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.587+0000 7fd18effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x163dc10 0x1950610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:38.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.587+0000 7fd18effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x163dc10 0x1950610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35154/0 (socket says 172.21.15.78:35154) 2023-10-11T09:46:38.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.587+0000 7fd18effd700 1 -- 172.21.15.78:0/4186435480 learned_addr learned my addr 172.21.15.78:0/4186435480 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:46:38.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.587+0000 7fd18f7fe700 1 --2- 172.21.15.78:0/4186435480 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x163d840 0x18cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:38.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.587+0000 7fd18ffff700 1 --2- 172.21.15.78:0/4186435480 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x16888c0 0x1950b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:38.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.587+0000 7fd18effd700 1 -- 172.21.15.78:0/4186435480 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x163d840 msgr2=0x18cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:38.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.587+0000 7fd18effd700 1 --2- 172.21.15.78:0/4186435480 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x163d840 0x18cb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:38.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.587+0000 7fd18effd700 1 -- 172.21.15.78:0/4186435480 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x16888c0 msgr2=0x1950b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:46:38.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.587+0000 7fd18effd700 1 --2- 172.21.15.78:0/4186435480 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x16888c0 0x1950b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:38.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.587+0000 7fd18effd700 1 -- 172.21.15.78:0/4186435480 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1689b10 con 0x163dc10 2023-10-11T09:46:38.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.587+0000 7fd18f7fe700 1 --2- 172.21.15.78:0/4186435480 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x163d840 0x18cb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:46:38.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.587+0000 7fd18effd700 1 --2- 172.21.15.78:0/4186435480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x163dc10 0x1950610 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fd184054b20 tx=0x7fd184054780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec7628a2d101f3b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:46:38.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.587+0000 7fd16e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:46:38.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.587+0000 7fd16e7fc700 1 -- 172.21.15.78:0/4186435480 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd184054280 con 0x163dc10 2023-10-11T09:46:38.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.587+0000 7fd16e7fc700 1 -- 172.21.15.78:0/4186435480 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 6724+0+0 (secure 0 0 0) 0x7fd184054b90 con 0x163dc10 2023-10-11T09:46:38.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.587+0000 7fd19922a740 1 -- 172.21.15.78:0/4186435480 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd1800036c0 con 0x163dc10 2023-10-11T09:46:38.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.587+0000 7fd16e7fc700 1 -- 172.21.15.78:0/4186435480 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd184058910 con 0x163dc10 2023-10-11T09:46:38.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.587+0000 7fd19922a740 10 client.7014 fetch_fsmap learned FSMap version 50 2023-10-11T09:46:38.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.587+0000 7fd19922a740 1 -- 172.21.15.78:0/4186435480 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x17dc820 con 0x163dc10 2023-10-11T09:46:38.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.587+0000 7fd16e7fc700 1 -- 172.21.15.78:0/4186435480 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd184076070 con 0x163dc10 2023-10-11T09:46:38.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.587+0000 7fd16e7fc700 1 -- 172.21.15.78:0/4186435480 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 50) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd18405c630 con 0x163dc10 2023-10-11T09:46:38.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.587+0000 7fd19922a740 10 client.7014 fetch_fsmap finished waiting for FSMap version 50 2023-10-11T09:46:38.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.587+0000 7fd19922a740 10 client.7014 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-10-11T09:46:38.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.587+0000 7fd19922a740 20 client.7014 populate_metadata read hostname 'smithi078' 2023-10-11T09:46:38.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.587+0000 7fd19922a740 1 --2- 172.21.15.78:0/4186435480 >> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] conn(0x1909c60 0x18ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:46:38.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.587+0000 7fd19922a740 4 client.7014 mds_command: new command op to 6897 tid=0[{"prefix": "scrub status"}] 2023-10-11T09:46:38.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.587+0000 7fd19922a740 1 -- 172.21.15.78:0/4186435480 --> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x1695610 con 0x1909c60 2023-10-11T09:46:38.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.587+0000 7fd18f7fe700 1 --2- 172.21.15.78:0/4186435480 >> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] conn(0x1909c60 0x18ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:46:38.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.587+0000 7fd16e7fc700 10 client.7014 ms_handle_connect on v2:172.21.15.78:6835/3498868661 2023-10-11T09:46:38.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.587+0000 7fd18f7fe700 1 --2- 172.21.15.78:0/4186435480 >> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] conn(0x1909c60 0x18ce600 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1852f82d7d23617 server_cookie=3785dde18d3d43ef in_seq=0 out_seq=0 2023-10-11T09:46:38.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.591+0000 7fd16e7fc700 1 -- 172.21.15.78:0/4186435480 <== mds.0 v2:172.21.15.78:6835/3498868661 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x1695610 con 0x1909c60 2023-10-11T09:46:38.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.591+0000 7fd16e7fc700 10 client.7014 handle_command_reply: tid=0 2023-10-11T09:46:38.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.591+0000 7fd19922a740 1 client.7014 shutdown 2023-10-11T09:46:38.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.591+0000 7fd19922a740 1 -- 172.21.15.78:0/4186435480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x163dc10 msgr2=0x1950610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:38.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.591+0000 7fd19922a740 1 --2- 172.21.15.78:0/4186435480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x163dc10 0x1950610 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fd184054b20 tx=0x7fd184054780 comp rx=0 tx=0).stop 2023-10-11T09:46:38.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.591+0000 7fd19922a740 1 -- 172.21.15.78:0/4186435480 shutdown_connections 2023-10-11T09:46:38.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.591+0000 7fd19922a740 1 --2- 172.21.15.78:0/4186435480 >> [v2:172.21.15.78:6835/3498868661,v1:172.21.15.78:6837/3498868661] conn(0x1909c60 0x18ce600 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:38.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.591+0000 7fd19922a740 1 --2- 172.21.15.78:0/4186435480 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x16888c0 0x1950b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:38.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.591+0000 7fd19922a740 1 --2- 172.21.15.78:0/4186435480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x163dc10 0x1950610 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:38.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.591+0000 7fd16e7fc700 0 client.7014 ms_handle_reset on v2:172.21.15.78:6835/3498868661 2023-10-11T09:46:38.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.591+0000 7fd19922a740 1 --2- 172.21.15.78:0/4186435480 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x163d840 0x18cb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:38.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.591+0000 7fd19922a740 1 -- 172.21.15.78:0/4186435480 >> 172.21.15.78:0/4186435480 conn(0x163dfe0 msgr2=0x18d2780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:46:38.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.591+0000 7fd19922a740 1 -- 172.21.15.78:0/4186435480 shutdown_connections 2023-10-11T09:46:38.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.591+0000 7fd19922a740 1 -- 172.21.15.78:0/4186435480 wait complete. 2023-10-11T09:46:38.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.591+0000 7fd19922a740 20 client.7014 trim_cache size 0 max 16384 2023-10-11T09:46:38.604 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:46:38.604 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2023-10-11T09:46:38.604 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2023-10-11T09:46:38.605 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:46:38.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.591+0000 7fd194d37700 1 -- 172.21.15.78:0/1437808562 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd17c065ed0 msgr2=0x7fd17c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:38.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.591+0000 7fd194d37700 1 --2- 172.21.15.78:0/1437808562 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd17c065ed0 0x7fd17c068380 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd180000c00 tx=0x7fd180008030 comp rx=0 tx=0).stop 2023-10-11T09:46:38.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.591+0000 7fd194d37700 1 -- 172.21.15.78:0/1437808562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1900b3610 msgr2=0x7fd1901f5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:46:38.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.591+0000 7fd194d37700 1 --2- 172.21.15.78:0/1437808562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1900b3610 0x7fd1901f5ab0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fd18400d7d0 tx=0x7fd18400dae0 comp rx=0 tx=0).stop 2023-10-11T09:46:38.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.591+0000 7fd194d37700 1 -- 172.21.15.78:0/1437808562 shutdown_connections 2023-10-11T09:46:38.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.591+0000 7fd194d37700 1 --2- 172.21.15.78:0/1437808562 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd17c065ed0 0x7fd17c068380 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:38.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.591+0000 7fd194d37700 1 --2- 172.21.15.78:0/1437808562 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd19015d040 0x7fd19009dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:38.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.591+0000 7fd194d37700 1 --2- 172.21.15.78:0/1437808562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1900b3610 0x7fd1901f5ab0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:38.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.591+0000 7fd194d37700 1 --2- 172.21.15.78:0/1437808562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1900b3240 0x7fd19009fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:46:38.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.591+0000 7fd194d37700 1 -- 172.21.15.78:0/1437808562 >> 172.21.15.78:0/1437808562 conn(0x7fd19009c260 msgr2=0x7fd1900b6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:46:38.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.591+0000 7fd194d37700 1 -- 172.21.15.78:0/1437808562 shutdown_connections 2023-10-11T09:46:38.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:46:38.591+0000 7fd194d37700 1 -- 172.21.15.78:0/1437808562 wait complete. 2023-10-11T09:46:38.620 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-11T09:46:38.620 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/0 2023-10-11T09:46:38.621 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:46:38.621 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:46:38.621 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/0" 2023-10-11T09:46:38.622 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:46:38.622 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:46:38.623 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 6291456: 2023-10-11T09:46:38.623 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:46:38.623 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 6291456 2023-10-11T09:46:38.624 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:46:38.624 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:46:38.624 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:46:38.625 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:46:38.625 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:46:38.625 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:46:41.928 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/1 2023-10-11T09:46:41.929 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:46:41.929 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:46:41.930 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/1" 2023-10-11T09:46:41.930 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:46:41.930 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:46:41.931 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 6291456: 2023-10-11T09:46:41.931 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:46:41.931 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 6291456 2023-10-11T09:46:41.932 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:46:41.932 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:46:41.932 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:46:41.933 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:46:41.933 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:46:41.933 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:46:45.072 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/2 2023-10-11T09:46:45.073 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:46:45.073 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:46:45.074 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/2" 2023-10-11T09:46:45.074 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:46:45.074 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:46:45.075 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 6291456: 2023-10-11T09:46:45.075 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:46:45.075 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 6291456 2023-10-11T09:46:45.076 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:46:45.076 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:46:45.076 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:46:45.077 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:46:45.077 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:46:45.077 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:46:48.209 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/3 2023-10-11T09:46:48.210 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:46:48.211 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:46:48.211 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/3" 2023-10-11T09:46:48.211 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:46:48.212 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:46:48.212 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 6291456: 2023-10-11T09:46:48.212 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:46:48.213 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 6291456 2023-10-11T09:46:48.213 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:46:48.213 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:46:48.214 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:46:48.214 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:46:48.215 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:46:48.215 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:46:51.557 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/4 2023-10-11T09:46:51.558 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:46:51.558 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:46:51.559 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/4" 2023-10-11T09:46:51.559 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:46:51.559 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:46:51.560 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 6291456: 2023-10-11T09:46:51.560 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:46:51.561 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 6291456 2023-10-11T09:46:51.561 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:46:51.561 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:46:51.562 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:46:51.562 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:46:51.562 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:46:51.563 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:46:54.702 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/5 2023-10-11T09:46:54.703 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:46:54.703 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:46:54.704 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/5" 2023-10-11T09:46:54.704 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:46:54.704 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:46:54.705 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 6291456: 2023-10-11T09:46:54.705 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:46:54.705 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 6291456 2023-10-11T09:46:54.706 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:46:54.706 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:46:54.706 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:46:54.707 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:46:54.707 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:46:54.707 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:46:57.845 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/6 2023-10-11T09:46:57.846 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:46:57.846 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:46:57.846 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/6" 2023-10-11T09:46:57.847 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:46:57.847 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:46:57.847 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 6291456: 2023-10-11T09:46:57.848 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:46:57.848 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 6291456 2023-10-11T09:46:57.848 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:46:57.849 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:46:57.849 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:46:57.849 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:46:57.850 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:46:57.850 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:47:00.115 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:47:00.123 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:47:01.185 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/7 2023-10-11T09:47:01.185 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:47:01.186 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:47:01.186 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/7" 2023-10-11T09:47:01.187 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:47:01.187 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:47:01.187 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 6291456: 2023-10-11T09:47:01.188 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:47:01.188 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 6291456 2023-10-11T09:47:01.188 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:47:01.189 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:47:01.189 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:47:01.190 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:47:01.190 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:47:01.190 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:47:04.288 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/8 2023-10-11T09:47:04.289 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:47:04.289 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:47:04.289 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/8" 2023-10-11T09:47:04.290 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:47:04.290 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:47:04.291 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 6291456: 2023-10-11T09:47:04.291 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:47:04.291 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 6291456 2023-10-11T09:47:04.292 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:47:04.292 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:47:04.292 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:47:04.293 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:47:04.293 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:47:04.293 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:47:07.432 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/9 2023-10-11T09:47:07.433 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:47:07.434 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:47:07.434 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/9" 2023-10-11T09:47:07.434 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:47:07.435 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:47:07.435 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 6291456: 2023-10-11T09:47:07.435 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:47:07.436 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 6291456 2023-10-11T09:47:07.436 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:47:07.436 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:47:07.437 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:47:07.437 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:47:07.437 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:47:07.438 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:47:10.573 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/10 2023-10-11T09:47:10.574 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:47:10.574 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:47:10.574 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/10" 2023-10-11T09:47:10.575 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:47:10.575 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:47:10.575 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 6291456: 2023-10-11T09:47:10.576 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:47:10.576 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 6291456 2023-10-11T09:47:10.576 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:47:10.577 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:47:10.577 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:47:10.577 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:47:10.578 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:47:10.578 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:47:13.715 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/11 2023-10-11T09:47:13.715 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:47:13.716 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:47:13.716 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/11" 2023-10-11T09:47:13.716 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:47:13.717 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:47:13.717 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 6291456: 2023-10-11T09:47:13.717 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:47:13.718 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 6291456 2023-10-11T09:47:13.718 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:47:13.718 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:47:13.719 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:47:13.719 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:47:13.720 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:47:13.720 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:47:16.856 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/12 2023-10-11T09:47:16.856 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:47:16.857 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:47:16.857 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/12" 2023-10-11T09:47:16.857 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:47:16.858 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:47:16.858 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 6291456: 2023-10-11T09:47:16.859 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:47:16.859 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 6291456 2023-10-11T09:47:16.859 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:47:16.860 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:47:16.860 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:47:16.860 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:47:16.861 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:47:16.861 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:47:20.201 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/13 2023-10-11T09:47:20.202 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:47:20.202 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:47:20.203 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/13" 2023-10-11T09:47:20.203 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:47:20.204 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:47:20.204 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 6291456: 2023-10-11T09:47:20.204 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:47:20.205 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 6291456 2023-10-11T09:47:20.205 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:47:20.205 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:47:20.206 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:47:20.206 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:47:20.206 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:47:20.207 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:47:23.544 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/14 2023-10-11T09:47:23.544 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:47:23.545 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:47:23.545 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/14" 2023-10-11T09:47:23.545 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:47:23.546 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:47:23.546 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 6291456: 2023-10-11T09:47:23.547 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:47:23.547 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 6291456 2023-10-11T09:47:23.547 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:47:23.548 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:47:23.548 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:47:23.548 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:47:23.549 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:47:23.549 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:47:26.689 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/15 2023-10-11T09:47:26.690 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:47:26.690 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:47:26.691 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/15" 2023-10-11T09:47:26.691 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:47:26.691 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:47:26.692 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 6291456: 2023-10-11T09:47:26.692 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:47:26.692 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 6291456 2023-10-11T09:47:26.693 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:47:26.693 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:47:26.693 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:47:26.694 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:47:26.694 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:47:26.694 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:47:30.034 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/16 2023-10-11T09:47:30.035 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:47:30.035 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:47:30.035 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/16" 2023-10-11T09:47:30.036 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:47:30.036 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:47:30.036 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 6291456: 2023-10-11T09:47:30.037 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:47:30.037 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 6291456 2023-10-11T09:47:30.037 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:47:30.038 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:47:30.038 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:47:30.038 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:47:30.039 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:47:30.039 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:47:30.194 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:47:30.202 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:47:33.176 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/17 2023-10-11T09:47:33.177 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:47:33.178 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:47:33.178 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/17" 2023-10-11T09:47:33.178 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:47:33.179 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:47:33.179 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 6291456: 2023-10-11T09:47:33.179 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:47:33.180 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 6291456 2023-10-11T09:47:33.180 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:47:33.180 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:47:33.181 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:47:33.181 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:47:33.181 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:47:33.182 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:47:36.281 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/18 2023-10-11T09:47:36.282 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:47:36.282 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:47:36.283 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/18" 2023-10-11T09:47:36.283 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:47:36.284 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:47:36.284 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 6291456: 2023-10-11T09:47:36.284 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:47:36.285 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 6291456 2023-10-11T09:47:36.285 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:47:36.285 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:47:36.286 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:47:36.286 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:47:36.286 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:47:36.287 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:47:39.425 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/19 2023-10-11T09:47:39.426 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:47:39.426 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:47:39.427 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/19" 2023-10-11T09:47:39.427 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:47:39.428 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:47:39.428 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 6291456: 2023-10-11T09:47:39.428 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:47:39.429 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 6291456 2023-10-11T09:47:39.429 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:47:39.429 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:47:39.430 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:47:39.430 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:47:39.430 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:47:39.431 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:47:42.772 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/20 2023-10-11T09:47:42.773 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:47:42.774 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:47:42.774 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/20" 2023-10-11T09:47:42.774 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:47:42.775 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:47:42.775 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 6291456: 2023-10-11T09:47:42.776 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:47:42.776 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 6291456 2023-10-11T09:47:42.776 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:47:42.777 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:47:42.777 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:47:42.777 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:47:42.778 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:47:42.778 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:47:45.918 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/21 2023-10-11T09:47:45.919 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:47:45.919 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:47:45.919 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/21" 2023-10-11T09:47:45.920 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:47:45.920 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:47:45.920 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 6291456: 2023-10-11T09:47:45.921 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:47:45.921 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 6291456 2023-10-11T09:47:45.921 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:47:45.922 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:47:45.922 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:47:45.922 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:47:45.923 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:47:45.923 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:47:49.065 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/22 2023-10-11T09:47:49.066 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:47:49.066 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:47:49.066 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/22" 2023-10-11T09:47:49.067 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:47:49.067 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:47:49.068 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 6291456: 2023-10-11T09:47:49.068 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:47:49.068 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 6291456 2023-10-11T09:47:49.069 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:47:49.069 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:47:49.069 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:47:49.070 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:47:49.070 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:47:49.070 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:47:52.410 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/23 2023-10-11T09:47:52.410 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:47:52.411 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:47:52.411 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/23" 2023-10-11T09:47:52.411 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:47:52.412 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:47:52.412 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 6291456: 2023-10-11T09:47:52.412 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:47:52.413 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 6291456 2023-10-11T09:47:52.413 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:47:52.413 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:47:52.414 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:47:52.414 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:47:52.414 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:47:52.415 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:47:55.552 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/24 2023-10-11T09:47:55.553 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:47:55.554 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:47:55.554 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/subdir/24" 2023-10-11T09:47:55.554 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:47:55.555 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:47:55.555 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 6291456: 2023-10-11T09:47:55.555 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:47:55.556 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 6291456 2023-10-11T09:47:55.556 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:47:55.557 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:47:55.557 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:47:55.557 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:47:55.558 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:47:55.558 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:47:58.697 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-11T09:47:58.698 DEBUG:teuthology.orchestra.run.smithi078:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-11T09:47:58.783 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-11T09:47:58.828 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-11T09:47:58.829 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:47:58.829 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:47:58.849 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:47:59.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.049+0000 7f3486769700 1 Processor -- start 2023-10-11T09:47:59.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.049+0000 7f3486769700 1 -- start start 2023-10-11T09:47:59.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.053+0000 7f3486769700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3480154e00 0x7f348015b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:47:59.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.053+0000 7f3486769700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f3480071040 0x7f34800cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:47:59.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.053+0000 7f3486769700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34800720f0 con 0x7f34800cba00 2023-10-11T09:47:59.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.053+0000 7f3486769700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f348015bb90 con 0x7f3480071040 2023-10-11T09:47:59.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.053+0000 7f3486769700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34800714e0 con 0x7f3480154e00 2023-10-11T09:47:59.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.053+0000 7f34777fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f3480071040 0x7f34800cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:47:59.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.053+0000 7f3484d06700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f34800cba00 0x7f348015e1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:41554/0 (socket says 172.21.15.78:41554) 2023-10-11T09:47:59.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.053+0000 7f3484d06700 1 -- 172.21.15.78:0/1574036547 learned_addr learned my addr 172.21.15.78:0/1574036547 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:47:59.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.053+0000 7f347ffff700 1 --2- 172.21.15.78:0/1574036547 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3480154e00 0x7f348015b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:47:59.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.053+0000 7f34777fe700 1 -- 172.21.15.78:0/1574036547 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3480154e00 msgr2=0x7f348015b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:47:59.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.053+0000 7f34777fe700 1 --2- 172.21.15.78:0/1574036547 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3480154e00 0x7f348015b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:47:59.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.053+0000 7f34777fe700 1 -- 172.21.15.78:0/1574036547 >> v1:172.21.15.78:6789/0 conn(0x7f34800cba00 legacy=0x7f348015e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:47:59.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.053+0000 7f34777fe700 1 -- 172.21.15.78:0/1574036547 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34800717c0 con 0x7f3480071040 2023-10-11T09:47:59.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.053+0000 7f347ffff700 1 --2- 172.21.15.78:0/1574036547 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3480154e00 0x7f348015b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:47:59.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.053+0000 7f34777fe700 1 --2- 172.21.15.78:0/1574036547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3480071040 0x7f34800cb4c0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f3468009850 tx=0x7f3468009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef7f49595d57c82c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:47:59.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.057+0000 7f347f7fe700 1 -- 172.21.15.78:0/1574036547 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3468011830 con 0x7f3480071040 2023-10-11T09:47:59.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.057+0000 7f347f7fe700 1 -- 172.21.15.78:0/1574036547 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3468011990 con 0x7f3480071040 2023-10-11T09:47:59.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.057+0000 7f347f7fe700 1 -- 172.21.15.78:0/1574036547 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3468011d20 con 0x7f3480071040 2023-10-11T09:47:59.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.057+0000 7f3486769700 1 -- 172.21.15.78:0/1574036547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3480071040 msgr2=0x7f34800cb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:47:59.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.057+0000 7f3486769700 1 --2- 172.21.15.78:0/1574036547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3480071040 0x7f34800cb4c0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f3468009850 tx=0x7f3468009b60 comp rx=0 tx=0).stop 2023-10-11T09:47:59.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.057+0000 7f3486769700 1 -- 172.21.15.78:0/1574036547 shutdown_connections 2023-10-11T09:47:59.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.057+0000 7f3486769700 1 --2- 172.21.15.78:0/1574036547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3480071040 0x7f34800cb4c0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:47:59.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.057+0000 7f3486769700 1 --2- 172.21.15.78:0/1574036547 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3480154e00 0x7f348015b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:47:59.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.057+0000 7f3486769700 1 -- 172.21.15.78:0/1574036547 >> 172.21.15.78:0/1574036547 conn(0x7f348006aca0 msgr2=0x7f34800b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:47:59.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.057+0000 7f3486769700 1 -- 172.21.15.78:0/1574036547 shutdown_connections 2023-10-11T09:47:59.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.057+0000 7f3486769700 1 -- 172.21.15.78:0/1574036547 wait complete. 2023-10-11T09:47:59.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.057+0000 7f3486769700 1 Processor -- start 2023-10-11T09:47:59.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.057+0000 7f3486769700 1 -- start start 2023-10-11T09:47:59.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.061+0000 7f3486769700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3480071040 0x7f34801f1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:47:59.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.061+0000 7f3486769700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34800cba00 0x7f34800cc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:47:59.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.061+0000 7f3486769700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3480154e00 0x7f34801f2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:47:59.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.061+0000 7f3486769700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34801f1dd0 con 0x7f34800cba00 2023-10-11T09:47:59.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.061+0000 7f3486769700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34800ccbd0 con 0x7f3480071040 2023-10-11T09:47:59.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.061+0000 7f3486769700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3480155da0 con 0x7f3480154e00 2023-10-11T09:47:59.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.061+0000 7f34777fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34800cba00 0x7f34800cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:47:59.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.061+0000 7f34777fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34800cba00 0x7f34800cc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48462/0 (socket says 172.21.15.78:48462) 2023-10-11T09:47:59.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.061+0000 7f34777fe700 1 -- 172.21.15.78:0/2047529689 learned_addr learned my addr 172.21.15.78:0/2047529689 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:47:59.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.061+0000 7f3484d06700 1 --2- 172.21.15.78:0/2047529689 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3480154e00 0x7f34801f2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:47:59.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.061+0000 7f347ffff700 1 --2- 172.21.15.78:0/2047529689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3480071040 0x7f34801f1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:47:59.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.061+0000 7f34777fe700 1 -- 172.21.15.78:0/2047529689 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3480154e00 msgr2=0x7f34801f2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:47:59.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.061+0000 7f34777fe700 1 --2- 172.21.15.78:0/2047529689 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3480154e00 0x7f34801f2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:47:59.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.061+0000 7f34777fe700 1 -- 172.21.15.78:0/2047529689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3480071040 msgr2=0x7f34801f1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:47:59.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.061+0000 7f34777fe700 1 --2- 172.21.15.78:0/2047529689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3480071040 0x7f34801f1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:47:59.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.061+0000 7f34777fe700 1 -- 172.21.15.78:0/2047529689 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34680094b0 con 0x7f34800cba00 2023-10-11T09:47:59.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.061+0000 7f347ffff700 1 --2- 172.21.15.78:0/2047529689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3480071040 0x7f34801f1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:47:59.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.061+0000 7f3484d06700 1 --2- 172.21.15.78:0/2047529689 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3480154e00 0x7f34801f2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:47:59.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.061+0000 7f34777fe700 1 --2- 172.21.15.78:0/2047529689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34800cba00 0x7f34800cc690 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f3468000c00 tx=0x7f3468004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:47:59.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.061+0000 7f347d7fa700 1 -- 172.21.15.78:0/2047529689 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3468004570 con 0x7f34800cba00 2023-10-11T09:47:59.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.061+0000 7f347d7fa700 1 -- 172.21.15.78:0/2047529689 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f346801f070 con 0x7f34800cba00 2023-10-11T09:47:59.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.061+0000 7f3486769700 1 -- 172.21.15.78:0/2047529689 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3480156040 con 0x7f34800cba00 2023-10-11T09:47:59.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.061+0000 7f347d7fa700 1 -- 172.21.15.78:0/2047529689 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3468003900 con 0x7f34800cba00 2023-10-11T09:47:59.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.061+0000 7f3486769700 1 -- 172.21.15.78:0/2047529689 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34801f22e0 con 0x7f34800cba00 2023-10-11T09:47:59.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.065+0000 7f3486769700 1 -- 172.21.15.78:0/2047529689 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f348004e530 con 0x7f34800cba00 2023-10-11T09:47:59.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.065+0000 7f347d7fa700 1 -- 172.21.15.78:0/2047529689 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f3468021030 con 0x7f34800cba00 2023-10-11T09:47:59.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.065+0000 7f347d7fa700 1 --2- 172.21.15.78:0/2047529689 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3460065ed0 0x7f3460068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:47:59.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.065+0000 7f347d7fa700 1 -- 172.21.15.78:0/2047529689 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f3468084f10 con 0x7f34800cba00 2023-10-11T09:47:59.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.065+0000 7f347ffff700 1 --2- 172.21.15.78:0/2047529689 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3460065ed0 0x7f3460068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:47:59.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.069+0000 7f347ffff700 1 --2- 172.21.15.78:0/2047529689 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3460065ed0 0x7f3460068380 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3480159590 tx=0x7f3470008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:47:59.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.073+0000 7f347d7fa700 1 -- 172.21.15.78:0/2047529689 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f34680522b0 con 0x7f34800cba00 2023-10-11T09:47:59.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.249+0000 7f3486769700 1 -- 172.21.15.78:0/2047529689 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f348006e1e0 con 0x7f34800cba00 2023-10-11T09:47:59.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.253+0000 7f347d7fa700 1 -- 172.21.15.78:0/2047529689 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+20052 (secure 0 0 0) 0x7f3468015090 con 0x7f34800cba00 2023-10-11T09:47:59.304 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:47:59.304 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":70,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:46:00.671033+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:44:05.198742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-11T09:44:05.868171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data_extra","create_time":"2023-10-11T09:44:16.049941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6836/1666964283":"2023-10-12T09:46:00.635561+0000","172.21.15.78:6834/1666964283":"2023-10-12T09:46:00.635561+0000","172.21.15.78:6836/1819646091":"2023-10-12T09:45:54.334448+0000","172.21.15.78:6834/1819646091":"2023-10-12T09:45:54.334448+0000","172.21.15.78:6837/852594750":"2023-10-12T09:45:53.299952+0000","172.21.15.78:6835/852594750":"2023-10-12T09:45:53.299952+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:47:59.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.257+0000 7f3486769700 1 -- 172.21.15.78:0/2047529689 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3460065ed0 msgr2=0x7f3460068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:47:59.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.257+0000 7f3486769700 1 --2- 172.21.15.78:0/2047529689 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3460065ed0 0x7f3460068380 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3480159590 tx=0x7f3470008040 comp rx=0 tx=0).stop 2023-10-11T09:47:59.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.257+0000 7f3486769700 1 -- 172.21.15.78:0/2047529689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34800cba00 msgr2=0x7f34800cc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:47:59.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.257+0000 7f3486769700 1 --2- 172.21.15.78:0/2047529689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34800cba00 0x7f34800cc690 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f3468000c00 tx=0x7f3468004770 comp rx=0 tx=0).stop 2023-10-11T09:47:59.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.257+0000 7f3486769700 1 -- 172.21.15.78:0/2047529689 shutdown_connections 2023-10-11T09:47:59.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.257+0000 7f3486769700 1 --2- 172.21.15.78:0/2047529689 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3460065ed0 0x7f3460068380 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:47:59.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.257+0000 7f3486769700 1 --2- 172.21.15.78:0/2047529689 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3480154e00 0x7f34801f2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:47:59.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.257+0000 7f3486769700 1 --2- 172.21.15.78:0/2047529689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34800cba00 0x7f34800cc690 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:47:59.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.257+0000 7f3486769700 1 --2- 172.21.15.78:0/2047529689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3480071040 0x7f34801f1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:47:59.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.257+0000 7f3486769700 1 -- 172.21.15.78:0/2047529689 >> 172.21.15.78:0/2047529689 conn(0x7f348006aca0 msgr2=0x7f348015d470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:47:59.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.257+0000 7f3486769700 1 -- 172.21.15.78:0/2047529689 shutdown_connections 2023-10-11T09:47:59.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.257+0000 7f3486769700 1 -- 172.21.15.78:0/2047529689 wait complete. 2023-10-11T09:47:59.311 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T09:44:05.198742+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T09:44:05.868171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-10-11T09:44:16.049941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:47:59.312 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:47:59.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.481+0000 7f1c2d7a7700 1 Processor -- start 2023-10-11T09:47:59.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.481+0000 7f1c2d7a7700 1 -- start start 2023-10-11T09:47:59.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.481+0000 7f1c2d7a7700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f1c28160cb0 0x7f1c280cb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:47:59.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.481+0000 7f1c2d7a7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c280b1030 0x7f1c2806ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:47:59.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.481+0000 7f1c2d7a7700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c28165570 con 0x7f1c2806d080 2023-10-11T09:47:59.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.481+0000 7f1c2d7a7700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c28161080 con 0x7f1c28160cb0 2023-10-11T09:47:59.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.481+0000 7f1c2d7a7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c28164750 con 0x7f1c280b1030 2023-10-11T09:47:59.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.485+0000 7f1c26ffd700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f1c28160cb0 0x7f1c280cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:47:59.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.485+0000 7f1c267fc700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f1c2806d080 0x7f1c280b0920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:41570/0 (socket says 172.21.15.78:41570) 2023-10-11T09:47:59.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.485+0000 7f1c267fc700 1 -- 172.21.15.78:0/843057906 learned_addr learned my addr 172.21.15.78:0/843057906 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:47:59.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.485+0000 7f1c277fe700 1 --2- 172.21.15.78:0/843057906 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c280b1030 0x7f1c2806ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:47:59.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.485+0000 7f1c25ffb700 1 -- 172.21.15.78:0/843057906 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3283801927 0 0) 0x7f1c28165570 con 0x7f1c2806d080 2023-10-11T09:47:59.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.485+0000 7f1c26ffd700 1 -- 172.21.15.78:0/843057906 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c280b1030 msgr2=0x7f1c2806ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:47:59.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.485+0000 7f1c26ffd700 1 --2- 172.21.15.78:0/843057906 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c280b1030 0x7f1c2806ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:47:59.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.485+0000 7f1c26ffd700 1 -- 172.21.15.78:0/843057906 >> v1:172.21.15.78:6789/0 conn(0x7f1c2806d080 legacy=0x7f1c280b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:47:59.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.485+0000 7f1c26ffd700 1 -- 172.21.15.78:0/843057906 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c28164960 con 0x7f1c28160cb0 2023-10-11T09:47:59.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.485+0000 7f1c26ffd700 1 --2- 172.21.15.78:0/843057906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c28160cb0 0x7f1c280cb100 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f1c18009850 tx=0x7f1c18009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c1f94438a0d3b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:47:59.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.485+0000 7f1c25ffb700 1 -- 172.21.15.78:0/843057906 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c18011830 con 0x7f1c28160cb0 2023-10-11T09:47:59.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.485+0000 7f1c25ffb700 1 -- 172.21.15.78:0/843057906 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c18011990 con 0x7f1c28160cb0 2023-10-11T09:47:59.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.485+0000 7f1c25ffb700 1 -- 172.21.15.78:0/843057906 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c18011d20 con 0x7f1c28160cb0 2023-10-11T09:47:59.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.485+0000 7f1c2d7a7700 1 -- 172.21.15.78:0/843057906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c28160cb0 msgr2=0x7f1c280cb100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:47:59.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.485+0000 7f1c2d7a7700 1 --2- 172.21.15.78:0/843057906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c28160cb0 0x7f1c280cb100 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f1c18009850 tx=0x7f1c18009b60 comp rx=0 tx=0).stop 2023-10-11T09:47:59.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.485+0000 7f1c2d7a7700 1 -- 172.21.15.78:0/843057906 shutdown_connections 2023-10-11T09:47:59.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.485+0000 7f1c2d7a7700 1 --2- 172.21.15.78:0/843057906 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c280b1030 0x7f1c2806ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:47:59.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.485+0000 7f1c2d7a7700 1 --2- 172.21.15.78:0/843057906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c28160cb0 0x7f1c280cb100 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:47:59.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.485+0000 7f1c2d7a7700 1 -- 172.21.15.78:0/843057906 >> 172.21.15.78:0/843057906 conn(0x7f1c280cecc0 msgr2=0x7f1c280cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:47:59.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.485+0000 7f1c2d7a7700 1 -- 172.21.15.78:0/843057906 shutdown_connections 2023-10-11T09:47:59.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.485+0000 7f1c2d7a7700 1 -- 172.21.15.78:0/843057906 wait complete. 2023-10-11T09:47:59.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.489+0000 7f1c2d7a7700 1 Processor -- start 2023-10-11T09:47:59.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.489+0000 7f1c2d7a7700 1 -- start start 2023-10-11T09:47:59.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.493+0000 7f1c2d7a7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c2806d080 0x7f1c28157fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:47:59.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.493+0000 7f1c2d7a7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c280b1030 0x7f1c28163320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:47:59.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.493+0000 7f1c2d7a7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c28160cb0 0x7f1c2814dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:47:59.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.493+0000 7f1c2d7a7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c28163860 con 0x7f1c280b1030 2023-10-11T09:47:59.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.493+0000 7f1c2d7a7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2814e4f0 con 0x7f1c2806d080 2023-10-11T09:47:59.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.493+0000 7f1c2d7a7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c280b1c90 con 0x7f1c28160cb0 2023-10-11T09:47:59.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.493+0000 7f1c267fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c280b1030 0x7f1c28163320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:47:59.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.493+0000 7f1c267fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c280b1030 0x7f1c28163320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48480/0 (socket says 172.21.15.78:48480) 2023-10-11T09:47:59.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.493+0000 7f1c267fc700 1 -- 172.21.15.78:0/244223708 learned_addr learned my addr 172.21.15.78:0/244223708 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:47:59.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.493+0000 7f1c26ffd700 1 --2- 172.21.15.78:0/244223708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c2806d080 0x7f1c28157fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:47:59.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.493+0000 7f1c267fc700 1 -- 172.21.15.78:0/244223708 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c28160cb0 msgr2=0x7f1c2814dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:47:59.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.493+0000 7f1c267fc700 1 --2- 172.21.15.78:0/244223708 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c28160cb0 0x7f1c2814dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:47:59.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.493+0000 7f1c267fc700 1 -- 172.21.15.78:0/244223708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c2806d080 msgr2=0x7f1c28157fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:47:59.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.493+0000 7f1c267fc700 1 --2- 172.21.15.78:0/244223708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c2806d080 0x7f1c28157fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:47:59.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.493+0000 7f1c267fc700 1 -- 172.21.15.78:0/244223708 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c180094b0 con 0x7f1c280b1030 2023-10-11T09:47:59.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.493+0000 7f1c26ffd700 1 --2- 172.21.15.78:0/244223708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c2806d080 0x7f1c28157fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:47:59.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.493+0000 7f1c267fc700 1 --2- 172.21.15.78:0/244223708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c280b1030 0x7f1c28163320 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c00c510 tx=0x7f1c1c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:47:59.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.493+0000 7f1c0f7fe700 1 -- 172.21.15.78:0/244223708 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c004ca0 con 0x7f1c280b1030 2023-10-11T09:47:59.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.493+0000 7f1c0f7fe700 1 -- 172.21.15.78:0/244223708 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1c004e20 con 0x7f1c280b1030 2023-10-11T09:47:59.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.493+0000 7f1c0f7fe700 1 -- 172.21.15.78:0/244223708 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c007310 con 0x7f1c280b1030 2023-10-11T09:47:59.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.493+0000 7f1c2d7a7700 1 -- 172.21.15.78:0/244223708 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c280b1f30 con 0x7f1c280b1030 2023-10-11T09:47:59.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.493+0000 7f1c2d7a7700 1 -- 172.21.15.78:0/244223708 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c280b24f0 con 0x7f1c280b1030 2023-10-11T09:47:59.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.497+0000 7f1c2d7a7700 1 -- 172.21.15.78:0/244223708 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c2806eb80 con 0x7f1c280b1030 2023-10-11T09:47:59.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.497+0000 7f1c0f7fe700 1 -- 172.21.15.78:0/244223708 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f1c1c007470 con 0x7f1c280b1030 2023-10-11T09:47:59.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.497+0000 7f1c0f7fe700 1 --2- 172.21.15.78:0/244223708 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1c14065ed0 0x7f1c14068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:47:59.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.497+0000 7f1c0f7fe700 1 -- 172.21.15.78:0/244223708 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f1c1c083860 con 0x7f1c280b1030 2023-10-11T09:47:59.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.497+0000 7f1c26ffd700 1 --2- 172.21.15.78:0/244223708 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1c14065ed0 0x7f1c14068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:47:59.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.501+0000 7f1c26ffd700 1 --2- 172.21.15.78:0/244223708 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1c14065ed0 0x7f1c14068380 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1c280b4e60 tx=0x7f1c18008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:47:59.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.505+0000 7f1c0f7fe700 1 -- 172.21.15.78:0/244223708 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f1c1c050b70 con 0x7f1c280b1030 2023-10-11T09:47:59.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.665+0000 7f1c2d7a7700 1 -- 172.21.15.78:0/244223708 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1c280c9110 con 0x7f1c280b1030 2023-10-11T09:47:59.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.665+0000 7f1c0f7fe700 1 -- 172.21.15.78:0/244223708 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1c1c007720 con 0x7f1c280b1030 2023-10-11T09:47:59.674 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:47:59.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.669+0000 7f1c2d7a7700 1 -- 172.21.15.78:0/244223708 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1c14065ed0 msgr2=0x7f1c14068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:47:59.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.669+0000 7f1c2d7a7700 1 --2- 172.21.15.78:0/244223708 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1c14065ed0 0x7f1c14068380 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1c280b4e60 tx=0x7f1c18008030 comp rx=0 tx=0).stop 2023-10-11T09:47:59.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.669+0000 7f1c2d7a7700 1 -- 172.21.15.78:0/244223708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c280b1030 msgr2=0x7f1c28163320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:47:59.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.669+0000 7f1c2d7a7700 1 --2- 172.21.15.78:0/244223708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c280b1030 0x7f1c28163320 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c00c510 tx=0x7f1c1c00c8d0 comp rx=0 tx=0).stop 2023-10-11T09:47:59.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.669+0000 7f1c2d7a7700 1 -- 172.21.15.78:0/244223708 shutdown_connections 2023-10-11T09:47:59.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.669+0000 7f1c2d7a7700 1 --2- 172.21.15.78:0/244223708 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1c14065ed0 0x7f1c14068380 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:47:59.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.669+0000 7f1c2d7a7700 1 --2- 172.21.15.78:0/244223708 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c28160cb0 0x7f1c2814dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:47:59.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.669+0000 7f1c2d7a7700 1 --2- 172.21.15.78:0/244223708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c280b1030 0x7f1c28163320 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:47:59.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.669+0000 7f1c2d7a7700 1 --2- 172.21.15.78:0/244223708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c2806d080 0x7f1c28157fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:47:59.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.669+0000 7f1c2d7a7700 1 -- 172.21.15.78:0/244223708 >> 172.21.15.78:0/244223708 conn(0x7f1c280cecc0 msgr2=0x7f1c280aeea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:47:59.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.669+0000 7f1c2d7a7700 1 -- 172.21.15.78:0/244223708 shutdown_connections 2023-10-11T09:47:59.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.669+0000 7f1c2d7a7700 1 -- 172.21.15.78:0/244223708 wait complete. 2023-10-11T09:47:59.700 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:47:59.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.861+0000 7f6a20862700 1 Processor -- start 2023-10-11T09:47:59.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.861+0000 7f6a20862700 1 -- start start 2023-10-11T09:47:59.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.865+0000 7f6a20862700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6a1c0b3a60 0x7f6a1c06d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:47:59.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.865+0000 7f6a20862700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f6a1c06d560 0x7f6a1c06bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:47:59.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.865+0000 7f6a20862700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a1c0b2d30 con 0x7f6a1c15a6b0 2023-10-11T09:47:59.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.865+0000 7f6a20862700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a1c06d960 con 0x7f6a1c06d560 2023-10-11T09:47:59.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.865+0000 7f6a20862700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a1c0cf360 con 0x7f6a1c0b3a60 2023-10-11T09:47:59.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.865+0000 7f6a1ad9d700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f6a1c15a6b0 0x7f6a1c0ce8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:41584/0 (socket says 172.21.15.78:41584) 2023-10-11T09:47:59.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.865+0000 7f6a1ad9d700 1 -- 172.21.15.78:0/3646850093 learned_addr learned my addr 172.21.15.78:0/3646850093 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:47:59.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.865+0000 7f6a1b59e700 1 --2- 172.21.15.78:0/3646850093 >> v2:172.21.15.78:3300/0 conn(0x7f6a1c06d560 0x7f6a1c06bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:47:59.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.865+0000 7f6a1a59c700 1 --2- 172.21.15.78:0/3646850093 >> v2:172.21.15.133:3300/0 conn(0x7f6a1c0b3a60 0x7f6a1c06d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:47:59.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.865+0000 7f6a19d9b700 1 -- 172.21.15.78:0/3646850093 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1413448118 0 0) 0x7f6a1c0b2d30 con 0x7f6a1c15a6b0 2023-10-11T09:47:59.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.865+0000 7f6a19d9b700 1 -- 172.21.15.78:0/3646850093 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a040035d0 con 0x7f6a1c15a6b0 2023-10-11T09:47:59.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.865+0000 7f6a1b59e700 1 -- 172.21.15.78:0/3646850093 >> v2:172.21.15.133:3300/0 conn(0x7f6a1c0b3a60 msgr2=0x7f6a1c06d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:47:59.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.865+0000 7f6a1b59e700 1 --2- 172.21.15.78:0/3646850093 >> v2:172.21.15.133:3300/0 conn(0x7f6a1c0b3a60 0x7f6a1c06d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:47:59.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.865+0000 7f6a1b59e700 1 -- 172.21.15.78:0/3646850093 >> v1:172.21.15.78:6789/0 conn(0x7f6a1c15a6b0 legacy=0x7f6a1c0ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:47:59.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.865+0000 7f6a1b59e700 1 -- 172.21.15.78:0/3646850093 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a1c0b2f80 con 0x7f6a1c06d560 2023-10-11T09:47:59.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.869+0000 7f6a1b59e700 1 --2- 172.21.15.78:0/3646850093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a1c06d560 0x7f6a1c06bbe0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c06fd70 tx=0x7f6a14009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf5db26bc365a8b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:47:59.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.869+0000 7f6a19d9b700 1 -- 172.21.15.78:0/3646850093 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a14011830 con 0x7f6a1c06d560 2023-10-11T09:47:59.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.869+0000 7f6a19d9b700 1 -- 172.21.15.78:0/3646850093 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a14011990 con 0x7f6a1c06d560 2023-10-11T09:47:59.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.869+0000 7f6a19d9b700 1 -- 172.21.15.78:0/3646850093 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a14011d20 con 0x7f6a1c06d560 2023-10-11T09:47:59.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.869+0000 7f6a20862700 1 -- 172.21.15.78:0/3646850093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a1c06d560 msgr2=0x7f6a1c06bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:47:59.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.869+0000 7f6a20862700 1 --2- 172.21.15.78:0/3646850093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a1c06d560 0x7f6a1c06bbe0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c06fd70 tx=0x7f6a14009b30 comp rx=0 tx=0).stop 2023-10-11T09:47:59.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.869+0000 7f6a20862700 1 -- 172.21.15.78:0/3646850093 shutdown_connections 2023-10-11T09:47:59.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.869+0000 7f6a20862700 1 --2- 172.21.15.78:0/3646850093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a1c06d560 0x7f6a1c06bbe0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:47:59.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.869+0000 7f6a20862700 1 --2- 172.21.15.78:0/3646850093 >> v2:172.21.15.133:3300/0 conn(0x7f6a1c0b3a60 0x7f6a1c06d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:47:59.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.869+0000 7f6a20862700 1 -- 172.21.15.78:0/3646850093 >> 172.21.15.78:0/3646850093 conn(0x7f6a1c06a060 msgr2=0x7f6a1c06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:47:59.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.869+0000 7f6a20862700 1 -- 172.21.15.78:0/3646850093 shutdown_connections 2023-10-11T09:47:59.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.869+0000 7f6a20862700 1 -- 172.21.15.78:0/3646850093 wait complete. 2023-10-11T09:47:59.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.873+0000 7f6a20862700 1 Processor -- start 2023-10-11T09:47:59.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.873+0000 7f6a20862700 1 -- start start 2023-10-11T09:47:59.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.873+0000 7f6a20862700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6a1c06d560 0x7f6a1c156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:47:59.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.873+0000 7f6a20862700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a1c0b3a60 0x7f6a1c0adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:47:59.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.873+0000 7f6a20862700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a1c15a6b0 0x7f6a1c14f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:47:59.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.873+0000 7f6a20862700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c15aa80 con 0x7f6a1c0b3a60 2023-10-11T09:47:59.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.873+0000 7f6a20862700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c157350 con 0x7f6a1c15a6b0 2023-10-11T09:47:59.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.873+0000 7f6a20862700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c157630 con 0x7f6a1c06d560 2023-10-11T09:47:59.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.873+0000 7f6a1a59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a1c0b3a60 0x7f6a1c0adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:47:59.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.873+0000 7f6a1a59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a1c0b3a60 0x7f6a1c0adcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48498/0 (socket says 172.21.15.78:48498) 2023-10-11T09:47:59.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.873+0000 7f6a1a59c700 1 -- 172.21.15.78:0/89731537 learned_addr learned my addr 172.21.15.78:0/89731537 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:47:59.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.873+0000 7f6a1ad9d700 1 --2- 172.21.15.78:0/89731537 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6a1c06d560 0x7f6a1c156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:47:59.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.873+0000 7f6a1b59e700 1 --2- 172.21.15.78:0/89731537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a1c15a6b0 0x7f6a1c14f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:47:59.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.873+0000 7f6a1a59c700 1 -- 172.21.15.78:0/89731537 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6a1c06d560 msgr2=0x7f6a1c156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:47:59.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.873+0000 7f6a1a59c700 1 --2- 172.21.15.78:0/89731537 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6a1c06d560 0x7f6a1c156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:47:59.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.873+0000 7f6a1a59c700 1 -- 172.21.15.78:0/89731537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a1c15a6b0 msgr2=0x7f6a1c14f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:47:59.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.873+0000 7f6a1a59c700 1 --2- 172.21.15.78:0/89731537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a1c15a6b0 0x7f6a1c14f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:47:59.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.873+0000 7f6a1a59c700 1 -- 172.21.15.78:0/89731537 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a140094b0 con 0x7f6a1c0b3a60 2023-10-11T09:47:59.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.873+0000 7f6a1ad9d700 1 --2- 172.21.15.78:0/89731537 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6a1c06d560 0x7f6a1c156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:47:59.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.873+0000 7f6a1a59c700 1 --2- 172.21.15.78:0/89731537 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a1c0b3a60 0x7f6a1c0adcd0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f6a10009fd0 tx=0x7f6a1000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:47:59.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.877+0000 7f6a037fe700 1 -- 172.21.15.78:0/89731537 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a100095a0 con 0x7f6a1c0b3a60 2023-10-11T09:47:59.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.877+0000 7f6a037fe700 1 -- 172.21.15.78:0/89731537 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a10013680 con 0x7f6a1c0b3a60 2023-10-11T09:47:59.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.877+0000 7f6a20862700 1 -- 172.21.15.78:0/89731537 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a1c157910 con 0x7f6a1c0b3a60 2023-10-11T09:47:59.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.877+0000 7f6a037fe700 1 -- 172.21.15.78:0/89731537 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a10013a30 con 0x7f6a1c0b3a60 2023-10-11T09:47:59.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.877+0000 7f6a20862700 1 -- 172.21.15.78:0/89731537 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a1c157040 con 0x7f6a1c0b3a60 2023-10-11T09:47:59.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.877+0000 7f6a20862700 1 -- 172.21.15.78:0/89731537 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a1c065d40 con 0x7f6a1c0b3a60 2023-10-11T09:47:59.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.877+0000 7f6a037fe700 1 -- 172.21.15.78:0/89731537 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f6a10013b90 con 0x7f6a1c0b3a60 2023-10-11T09:47:59.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.881+0000 7f6a037fe700 1 --2- 172.21.15.78:0/89731537 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6a08065ed0 0x7f6a08068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:47:59.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.881+0000 7f6a037fe700 1 -- 172.21.15.78:0/89731537 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f6a10083a40 con 0x7f6a1c0b3a60 2023-10-11T09:47:59.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.881+0000 7f6a1ad9d700 1 --2- 172.21.15.78:0/89731537 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6a08065ed0 0x7f6a08068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:47:59.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.881+0000 7f6a1ad9d700 1 --2- 172.21.15.78:0/89731537 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6a08065ed0 0x7f6a08068380 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c001560 tx=0x7f6a0c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:47:59.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:47:59.885+0000 7f6a037fe700 1 -- 172.21.15.78:0/89731537 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f6a10050de0 con 0x7f6a1c0b3a60 2023-10-11T09:48:00.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.053+0000 7f6a20862700 1 -- 172.21.15.78:0/89731537 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6a1c0cb6a0 con 0x7f6a1c0b3a60 2023-10-11T09:48:00.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.053+0000 7f6a037fe700 1 -- 172.21.15.78:0/89731537 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v70) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6a100170a0 con 0x7f6a1c0b3a60 2023-10-11T09:48:00.062 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:48:00.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.057+0000 7f6a20862700 1 -- 172.21.15.78:0/89731537 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6a08065ed0 msgr2=0x7f6a08068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:00.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.057+0000 7f6a20862700 1 --2- 172.21.15.78:0/89731537 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6a08065ed0 0x7f6a08068380 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c001560 tx=0x7f6a0c009040 comp rx=0 tx=0).stop 2023-10-11T09:48:00.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.057+0000 7f6a20862700 1 -- 172.21.15.78:0/89731537 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a1c0b3a60 msgr2=0x7f6a1c0adcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:00.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.057+0000 7f6a20862700 1 --2- 172.21.15.78:0/89731537 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a1c0b3a60 0x7f6a1c0adcd0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f6a10009fd0 tx=0x7f6a1000ebd0 comp rx=0 tx=0).stop 2023-10-11T09:48:00.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.057+0000 7f6a20862700 1 -- 172.21.15.78:0/89731537 shutdown_connections 2023-10-11T09:48:00.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.057+0000 7f6a20862700 1 --2- 172.21.15.78:0/89731537 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6a08065ed0 0x7f6a08068380 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:00.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.057+0000 7f6a20862700 1 --2- 172.21.15.78:0/89731537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a1c15a6b0 0x7f6a1c14f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:00.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.057+0000 7f6a20862700 1 --2- 172.21.15.78:0/89731537 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a1c0b3a60 0x7f6a1c0adcd0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:00.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.057+0000 7f6a20862700 1 --2- 172.21.15.78:0/89731537 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6a1c06d560 0x7f6a1c156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:00.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.057+0000 7f6a20862700 1 -- 172.21.15.78:0/89731537 >> 172.21.15.78:0/89731537 conn(0x7f6a1c06a060 msgr2=0x7f6a1c0ce5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:00.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.057+0000 7f6a20862700 1 -- 172.21.15.78:0/89731537 shutdown_connections 2023-10-11T09:48:00.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.057+0000 7f6a20862700 1 -- 172.21.15.78:0/89731537 wait complete. 2023-10-11T09:48:00.088 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T09:48:00.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.261+0000 7f20adf0e700 1 Processor -- start 2023-10-11T09:48:00.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.261+0000 7f20adf0e700 1 -- start start 2023-10-11T09:48:00.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.265+0000 7f20adf0e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20a80b3240 0x7f20a80d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:00.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.265+0000 7f20adf0e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f20a80b3a90 0x7f20a80b85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:00.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.265+0000 7f20adf0e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20a80b47f0 con 0x7f20a80b3610 2023-10-11T09:48:00.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.265+0000 7f20adf0e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20a80b8130 con 0x7f20a80b3a90 2023-10-11T09:48:00.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.265+0000 7f20adf0e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a80b3f10 con 0x7f20a80b3240 2023-10-11T09:48:00.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.265+0000 7f20acf0c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20a80b3240 0x7f20a80d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:00.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.265+0000 7f20ad70d700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f20a80b3a90 0x7f20a80b85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:00.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.265+0000 7f20acf0c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20a80b3240 0x7f20a80d29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:44442/0 (socket says 172.21.15.78:44442) 2023-10-11T09:48:00.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.265+0000 7f20a7fff700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f20a80b3610 0x7f20a815d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:38588/0 (socket says 172.21.15.78:38588) 2023-10-11T09:48:00.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.265+0000 7f20acf0c700 1 -- 172.21.15.78:0/1134108748 learned_addr learned my addr 172.21.15.78:0/1134108748 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:00.274 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:48:00.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.265+0000 7f20ad70d700 1 -- 172.21.15.78:0/1134108748 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20a80b3240 msgr2=0x7f20a80d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:00.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.265+0000 7f20ad70d700 1 --2- 172.21.15.78:0/1134108748 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20a80b3240 0x7f20a80d29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:00.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.265+0000 7f20ad70d700 1 -- 172.21.15.78:0/1134108748 >> v1:172.21.15.133:6789/0 conn(0x7f20a80b3610 legacy=0x7f20a815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:00.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.265+0000 7f20ad70d700 1 -- 172.21.15.78:0/1134108748 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a80b41e0 con 0x7f20a80b3a90 2023-10-11T09:48:00.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.265+0000 7f20acf0c700 1 --2- 172.21.15.78:0/1134108748 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20a80b3240 0x7f20a80d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:48:00.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.265+0000 7f20ad70d700 1 --2- 172.21.15.78:0/1134108748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20a80b3a90 0x7f20a80b85c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f20a0009850 tx=0x7f20a0009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f4c6696649675d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:00.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.269+0000 7f20a77fe700 1 -- 172.21.15.78:0/1134108748 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a0004230 con 0x7f20a80b3a90 2023-10-11T09:48:00.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.269+0000 7f20a77fe700 1 -- 172.21.15.78:0/1134108748 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a0004390 con 0x7f20a80b3a90 2023-10-11T09:48:00.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.269+0000 7f20a77fe700 1 -- 172.21.15.78:0/1134108748 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a00036c0 con 0x7f20a80b3a90 2023-10-11T09:48:00.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.269+0000 7f20adf0e700 1 -- 172.21.15.78:0/1134108748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20a80b3a90 msgr2=0x7f20a80b85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:00.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.269+0000 7f20adf0e700 1 --2- 172.21.15.78:0/1134108748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20a80b3a90 0x7f20a80b85c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f20a0009850 tx=0x7f20a0009b60 comp rx=0 tx=0).stop 2023-10-11T09:48:00.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.269+0000 7f20adf0e700 1 -- 172.21.15.78:0/1134108748 shutdown_connections 2023-10-11T09:48:00.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.269+0000 7f20adf0e700 1 --2- 172.21.15.78:0/1134108748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20a80b3a90 0x7f20a80b85c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:00.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.269+0000 7f20adf0e700 1 --2- 172.21.15.78:0/1134108748 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20a80b3240 0x7f20a80d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:00.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.269+0000 7f20adf0e700 1 -- 172.21.15.78:0/1134108748 >> 172.21.15.78:0/1134108748 conn(0x7f20a809c260 msgr2=0x7f20a80d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:00.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.269+0000 7f20adf0e700 1 -- 172.21.15.78:0/1134108748 shutdown_connections 2023-10-11T09:48:00.281 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:48:00.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.269+0000 7f20adf0e700 1 -- 172.21.15.78:0/1134108748 wait complete. 2023-10-11T09:48:00.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.273+0000 7f20adf0e700 1 Processor -- start 2023-10-11T09:48:00.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.273+0000 7f20adf0e700 1 -- start start 2023-10-11T09:48:00.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.273+0000 7f20adf0e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20a80b3240 0x7f20a814fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:00.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.273+0000 7f20adf0e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a80b3610 0x7f20a815f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:00.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.273+0000 7f20adf0e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20a80b3a90 0x7f20a80d2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:00.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.273+0000 7f20adf0e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a80c9280 con 0x7f20a80b3610 2023-10-11T09:48:00.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.273+0000 7f20adf0e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a81f1ea0 con 0x7f20a80b3a90 2023-10-11T09:48:00.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.273+0000 7f20adf0e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a815ce00 con 0x7f20a80b3240 2023-10-11T09:48:00.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.277+0000 7f20acf0c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20a80b3240 0x7f20a814fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:00.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.277+0000 7f20a7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a80b3610 0x7f20a815f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:00.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.277+0000 7f20acf0c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20a80b3240 0x7f20a814fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:44454/0 (socket says 172.21.15.78:44454) 2023-10-11T09:48:00.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.277+0000 7f20acf0c700 1 -- 172.21.15.78:0/2392031696 learned_addr learned my addr 172.21.15.78:0/2392031696 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:00.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.277+0000 7f20ad70d700 1 --2- 172.21.15.78:0/2392031696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20a80b3a90 0x7f20a80d2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:00.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.277+0000 7f20ad70d700 1 -- 172.21.15.78:0/2392031696 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20a80b3240 msgr2=0x7f20a814fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:00.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.277+0000 7f20ad70d700 1 --2- 172.21.15.78:0/2392031696 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20a80b3240 0x7f20a814fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:00.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.277+0000 7f20ad70d700 1 -- 172.21.15.78:0/2392031696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a80b3610 msgr2=0x7f20a815f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:00.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.277+0000 7f20ad70d700 1 --2- 172.21.15.78:0/2392031696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a80b3610 0x7f20a815f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:00.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.277+0000 7f20ad70d700 1 -- 172.21.15.78:0/2392031696 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f209c009b30 con 0x7f20a80b3a90 2023-10-11T09:48:00.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.277+0000 7f20ad70d700 1 --2- 172.21.15.78:0/2392031696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20a80b3a90 0x7f20a80d2fe0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f20a0015000 tx=0x7f20a0004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:00.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.277+0000 7f20a57fa700 1 -- 172.21.15.78:0/2392031696 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a0006ea0 con 0x7f20a80b3a90 2023-10-11T09:48:00.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.277+0000 7f20a57fa700 1 -- 172.21.15.78:0/2392031696 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a001f070 con 0x7f20a80b3a90 2023-10-11T09:48:00.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.277+0000 7f20adf0e700 1 -- 172.21.15.78:0/2392031696 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20a00094b0 con 0x7f20a80b3a90 2023-10-11T09:48:00.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.277+0000 7f20a57fa700 1 -- 172.21.15.78:0/2392031696 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a0011630 con 0x7f20a80b3a90 2023-10-11T09:48:00.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.277+0000 7f20adf0e700 1 -- 172.21.15.78:0/2392031696 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a815d050 con 0x7f20a80b3a90 2023-10-11T09:48:00.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.281+0000 7f20adf0e700 1 -- 172.21.15.78:0/2392031696 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20a8065d40 con 0x7f20a80b3a90 2023-10-11T09:48:00.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.281+0000 7f20a57fa700 1 -- 172.21.15.78:0/2392031696 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f20a0021030 con 0x7f20a80b3a90 2023-10-11T09:48:00.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.281+0000 7f20a57fa700 1 --2- 172.21.15.78:0/2392031696 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f208c065ed0 0x7f208c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:00.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.281+0000 7f20a57fa700 1 -- 172.21.15.78:0/2392031696 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f20a0084d00 con 0x7f20a80b3a90 2023-10-11T09:48:00.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.281+0000 7f20acf0c700 1 --2- 172.21.15.78:0/2392031696 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f208c065ed0 0x7f208c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:00.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.281+0000 7f20acf0c700 1 --2- 172.21.15.78:0/2392031696 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f208c065ed0 0x7f208c068380 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f209c009fd0 tx=0x7f209c00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:00.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.289+0000 7f20a57fa700 1 -- 172.21.15.78:0/2392031696 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f20a0052020 con 0x7f20a80b3a90 2023-10-11T09:48:00.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.465+0000 7f20adf0e700 1 -- 172.21.15.78:0/2392031696 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f20a8065d40 con 0x7f20a80b3a90 2023-10-11T09:48:00.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.469+0000 7f20a57fa700 1 -- 172.21.15.78:0/2392031696 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) v1 ==== 104+0+11 (secure 0 0 0) 0x7f20a0055640 con 0x7f20a80b3a90 2023-10-11T09:48:00.476 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:48:00.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.473+0000 7f20adf0e700 1 -- 172.21.15.78:0/2392031696 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f208c065ed0 msgr2=0x7f208c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:00.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.473+0000 7f20adf0e700 1 --2- 172.21.15.78:0/2392031696 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f208c065ed0 0x7f208c068380 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f209c009fd0 tx=0x7f209c00b040 comp rx=0 tx=0).stop 2023-10-11T09:48:00.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.473+0000 7f20adf0e700 1 -- 172.21.15.78:0/2392031696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20a80b3a90 msgr2=0x7f20a80d2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:00.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.473+0000 7f20adf0e700 1 --2- 172.21.15.78:0/2392031696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20a80b3a90 0x7f20a80d2fe0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f20a0015000 tx=0x7f20a0004770 comp rx=0 tx=0).stop 2023-10-11T09:48:00.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.473+0000 7f20adf0e700 1 -- 172.21.15.78:0/2392031696 shutdown_connections 2023-10-11T09:48:00.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.473+0000 7f20adf0e700 1 --2- 172.21.15.78:0/2392031696 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f208c065ed0 0x7f208c068380 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:00.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.473+0000 7f20adf0e700 1 --2- 172.21.15.78:0/2392031696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20a80b3a90 0x7f20a80d2fe0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:00.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.473+0000 7f20adf0e700 1 --2- 172.21.15.78:0/2392031696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a80b3610 0x7f20a815f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:00.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.473+0000 7f20adf0e700 1 --2- 172.21.15.78:0/2392031696 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20a80b3240 0x7f20a814fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:00.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.473+0000 7f20adf0e700 1 -- 172.21.15.78:0/2392031696 >> 172.21.15.78:0/2392031696 conn(0x7f20a809c260 msgr2=0x7f20a806c750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:00.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.473+0000 7f20adf0e700 1 -- 172.21.15.78:0/2392031696 shutdown_connections 2023-10-11T09:48:00.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.473+0000 7f20adf0e700 1 -- 172.21.15.78:0/2392031696 wait complete. 2023-10-11T09:48:00.501 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T09:48:00.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.673+0000 7f0fb113d700 1 Processor -- start 2023-10-11T09:48:00.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.673+0000 7f0fb113d700 1 -- start start 2023-10-11T09:48:00.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.673+0000 7f0fb113d700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0fac158430 0x7f0fac0710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:00.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.673+0000 7f0fb113d700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fac071610 con 0x7f0fac0caf90 2023-10-11T09:48:00.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.673+0000 7f0fb113d700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fac15e280 con 0x7f0fac06d340 2023-10-11T09:48:00.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.673+0000 7f0fb113d700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fac070c10 con 0x7f0fac158430 2023-10-11T09:48:00.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.677+0000 7f0fab59e700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f0fac0caf90 0x7f0fac15afe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:41596/0 (socket says 172.21.15.78:41596) 2023-10-11T09:48:00.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.677+0000 7f0fab59e700 1 -- 172.21.15.78:0/1681287063 learned_addr learned my addr 172.21.15.78:0/1681287063 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:00.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.677+0000 7f0faad9d700 1 --2- 172.21.15.78:0/1681287063 >> v2:172.21.15.133:3300/0 conn(0x7f0fac158430 0x7f0fac0710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:00.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.677+0000 7f0fa9d9b700 1 -- 172.21.15.78:0/1681287063 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4174314212 0 0) 0x7f0fac15e280 con 0x7f0fac06d340 2023-10-11T09:48:00.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.677+0000 7f0fa9d9b700 1 -- 172.21.15.78:0/1681287063 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f940035d0 con 0x7f0fac06d340 2023-10-11T09:48:00.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.677+0000 7f0fa9d9b700 1 -- 172.21.15.78:0/1681287063 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4149991819 0 0) 0x7f0fac071610 con 0x7f0fac0caf90 2023-10-11T09:48:00.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.677+0000 7f0fa9d9b700 1 -- 172.21.15.78:0/1681287063 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fac15e280 con 0x7f0fac0caf90 2023-10-11T09:48:00.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.677+0000 7f0faad9d700 1 -- 172.21.15.78:0/1681287063 >> v1:172.21.15.133:6789/0 conn(0x7f0fac06d340 legacy=0x7f0fac0ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:00.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.677+0000 7f0faad9d700 1 -- 172.21.15.78:0/1681287063 >> v1:172.21.15.78:6789/0 conn(0x7f0fac0caf90 legacy=0x7f0fac15afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:00.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.677+0000 7f0faad9d700 1 -- 172.21.15.78:0/1681287063 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fac15e560 con 0x7f0fac158430 2023-10-11T09:48:00.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.677+0000 7f0faad9d700 1 --2- 172.21.15.78:0/1681287063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fac158430 0x7f0fac0710d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c009850 tx=0x7f0f9c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1de1f26bc25db786 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:00.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.677+0000 7f0fa9d9b700 1 -- 172.21.15.78:0/1681287063 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c011830 con 0x7f0fac158430 2023-10-11T09:48:00.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.677+0000 7f0fa9d9b700 1 -- 172.21.15.78:0/1681287063 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f9c011990 con 0x7f0fac158430 2023-10-11T09:48:00.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.677+0000 7f0fa9d9b700 1 -- 172.21.15.78:0/1681287063 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c011d40 con 0x7f0fac158430 2023-10-11T09:48:00.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.677+0000 7f0fb113d700 1 -- 172.21.15.78:0/1681287063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fac158430 msgr2=0x7f0fac0710d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:00.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.677+0000 7f0fb113d700 1 --2- 172.21.15.78:0/1681287063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fac158430 0x7f0fac0710d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c009850 tx=0x7f0f9c009b60 comp rx=0 tx=0).stop 2023-10-11T09:48:00.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.677+0000 7f0fb113d700 1 -- 172.21.15.78:0/1681287063 shutdown_connections 2023-10-11T09:48:00.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.677+0000 7f0fb113d700 1 --2- 172.21.15.78:0/1681287063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fac158430 0x7f0fac0710d0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:00.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.677+0000 7f0fb113d700 1 -- 172.21.15.78:0/1681287063 >> 172.21.15.78:0/1681287063 conn(0x7f0fac0cecc0 msgr2=0x7f0fac0cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:00.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.677+0000 7f0fb113d700 1 -- 172.21.15.78:0/1681287063 shutdown_connections 2023-10-11T09:48:00.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.681+0000 7f0fb113d700 1 -- 172.21.15.78:0/1681287063 wait complete. 2023-10-11T09:48:00.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.681+0000 7f0fb113d700 1 Processor -- start 2023-10-11T09:48:00.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.685+0000 7f0fb113d700 1 -- start start 2023-10-11T09:48:00.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.685+0000 7f0fb113d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fac06d340 0x7f0fac155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:00.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.685+0000 7f0fb113d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0fac0caf90 0x7f0fac14ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:00.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.685+0000 7f0fb113d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0fac158430 0x7f0fac0ae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:00.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.685+0000 7f0fb113d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fac0cd8a0 con 0x7f0fac158430 2023-10-11T09:48:00.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.685+0000 7f0fb113d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fac0ccc90 con 0x7f0fac06d340 2023-10-11T09:48:00.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.685+0000 7f0fb113d700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fac0ccf70 con 0x7f0fac0caf90 2023-10-11T09:48:00.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.685+0000 7f0fab59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0fac158430 0x7f0fac0ae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:00.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.685+0000 7f0faad9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fac06d340 0x7f0fac155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:00.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.685+0000 7f0fab59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0fac158430 0x7f0fac0ae9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48512/0 (socket says 172.21.15.78:48512) 2023-10-11T09:48:00.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.685+0000 7f0fab59e700 1 -- 172.21.15.78:0/1350332534 learned_addr learned my addr 172.21.15.78:0/1350332534 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:00.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.685+0000 7f0faa59c700 1 --2- 172.21.15.78:0/1350332534 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0fac0caf90 0x7f0fac14ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:00.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.685+0000 7f0fab59e700 1 -- 172.21.15.78:0/1350332534 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0fac0caf90 msgr2=0x7f0fac14ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:00.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.685+0000 7f0fab59e700 1 --2- 172.21.15.78:0/1350332534 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0fac0caf90 0x7f0fac14ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:00.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.685+0000 7f0fab59e700 1 -- 172.21.15.78:0/1350332534 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fac06d340 msgr2=0x7f0fac155650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:00.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.685+0000 7f0fab59e700 1 --2- 172.21.15.78:0/1350332534 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fac06d340 0x7f0fac155650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:00.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.685+0000 7f0fab59e700 1 -- 172.21.15.78:0/1350332534 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f9c0094b0 con 0x7f0fac158430 2023-10-11T09:48:00.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.689+0000 7f0fab59e700 1 --2- 172.21.15.78:0/1350332534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0fac158430 0x7f0fac0ae9c0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f0fa400cae0 tx=0x7f0fa400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:00.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.689+0000 7f0f937fe700 1 -- 172.21.15.78:0/1350332534 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa4007480 con 0x7f0fac158430 2023-10-11T09:48:00.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.689+0000 7f0f937fe700 1 -- 172.21.15.78:0/1350332534 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fa40075e0 con 0x7f0fac158430 2023-10-11T09:48:00.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.689+0000 7f0f937fe700 1 -- 172.21.15.78:0/1350332534 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa4007990 con 0x7f0fac158430 2023-10-11T09:48:00.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.689+0000 7f0fb113d700 1 -- 172.21.15.78:0/1350332534 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fac0cd250 con 0x7f0fac158430 2023-10-11T09:48:00.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.689+0000 7f0fb113d700 1 -- 172.21.15.78:0/1350332534 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fac14f010 con 0x7f0fac158430 2023-10-11T09:48:00.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.689+0000 7f0fb113d700 1 -- 172.21.15.78:0/1350332534 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fac0cda50 con 0x7f0fac158430 2023-10-11T09:48:00.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.689+0000 7f0f937fe700 1 -- 172.21.15.78:0/1350332534 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f0fa401b3b0 con 0x7f0fac158430 2023-10-11T09:48:00.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.693+0000 7f0f937fe700 1 --2- 172.21.15.78:0/1350332534 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0f98065f00 0x7f0f980683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:00.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.693+0000 7f0f937fe700 1 -- 172.21.15.78:0/1350332534 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f0fa40833e0 con 0x7f0fac158430 2023-10-11T09:48:00.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.693+0000 7f0faad9d700 1 --2- 172.21.15.78:0/1350332534 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0f98065f00 0x7f0f980683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:00.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.693+0000 7f0faad9d700 1 --2- 172.21.15.78:0/1350332534 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0f98065f00 0x7f0f980683b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c0090b0 tx=0x7f0f9c01c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:00.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.697+0000 7f0f937fe700 1 -- 172.21.15.78:0/1350332534 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f0fac0cda50 con 0x7f0fac158430 2023-10-11T09:48:00.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.873+0000 7f0fb113d700 1 -- 172.21.15.78:0/1350332534 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0fac0cd9e0 con 0x7f0fac158430 2023-10-11T09:48:00.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.873+0000 7f0f937fe700 1 -- 172.21.15.78:0/1350332534 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v70) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0fa4054180 con 0x7f0fac158430 2023-10-11T09:48:00.883 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2023-10-11T09:48:00.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.881+0000 7f0fb113d700 1 -- 172.21.15.78:0/1350332534 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0f98065f00 msgr2=0x7f0f980683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:00.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.881+0000 7f0fb113d700 1 --2- 172.21.15.78:0/1350332534 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0f98065f00 0x7f0f980683b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c0090b0 tx=0x7f0f9c01c040 comp rx=0 tx=0).stop 2023-10-11T09:48:00.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.881+0000 7f0fb113d700 1 -- 172.21.15.78:0/1350332534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0fac158430 msgr2=0x7f0fac0ae9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:00.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.881+0000 7f0fb113d700 1 --2- 172.21.15.78:0/1350332534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0fac158430 0x7f0fac0ae9c0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f0fa400cae0 tx=0x7f0fa400cb10 comp rx=0 tx=0).stop 2023-10-11T09:48:00.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.881+0000 7f0fb113d700 1 -- 172.21.15.78:0/1350332534 shutdown_connections 2023-10-11T09:48:00.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.881+0000 7f0fb113d700 1 --2- 172.21.15.78:0/1350332534 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0f98065f00 0x7f0f980683b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:00.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.881+0000 7f0fb113d700 1 --2- 172.21.15.78:0/1350332534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0fac158430 0x7f0fac0ae9c0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:00.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.881+0000 7f0fb113d700 1 --2- 172.21.15.78:0/1350332534 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0fac0caf90 0x7f0fac14ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:00.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.881+0000 7f0fb113d700 1 --2- 172.21.15.78:0/1350332534 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fac06d340 0x7f0fac155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:00.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.881+0000 7f0fb113d700 1 -- 172.21.15.78:0/1350332534 >> 172.21.15.78:0/1350332534 conn(0x7f0fac0cecc0 msgr2=0x7f0fac0cf090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:00.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.881+0000 7f0fb113d700 1 -- 172.21.15.78:0/1350332534 shutdown_connections 2023-10-11T09:48:00.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:00.881+0000 7f0fb113d700 1 -- 172.21.15.78:0/1350332534 wait complete. 2023-10-11T09:48:00.909 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-10-11T09:48:01.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.085+0000 7f0d8bfff700 1 Processor -- start 2023-10-11T09:48:01.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.085+0000 7f0d8bfff700 1 -- start start 2023-10-11T09:48:01.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.085+0000 7f0d8bfff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0d8c15a6b0 0x7f0d8c0ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:01.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.085+0000 7f0d8bfff700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d8c0b2d30 con 0x7f0d8c0b3a60 2023-10-11T09:48:01.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.085+0000 7f0d8bfff700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d8c0cf360 con 0x7f0d8c06d730 2023-10-11T09:48:01.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.085+0000 7f0d8bfff700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d8c0b2f80 con 0x7f0d8c15a6b0 2023-10-11T09:48:01.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.085+0000 7f0d8affd700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0d8c15a6b0 0x7f0d8c0ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:01.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.085+0000 7f0d8affd700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0d8c15a6b0 0x7f0d8c0ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48524/0 (socket says 172.21.15.78:48524) 2023-10-11T09:48:01.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.085+0000 7f0d8affd700 1 -- 172.21.15.78:0/1696930744 learned_addr learned my addr 172.21.15.78:0/1696930744 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:01.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.085+0000 7f0d8a7fc700 1 --1- 172.21.15.78:0/1696930744 >> v1:172.21.15.78:6789/0 conn(0x7f0d8c0b3a60 0x7f0d8c06d020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:41612/0 (socket says 172.21.15.78:41612) 2023-10-11T09:48:01.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.089+0000 7f0d8affd700 1 -- 172.21.15.78:0/1696930744 >> v1:172.21.15.133:6789/0 conn(0x7f0d8c06d730 legacy=0x7f0d8c06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:01.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.089+0000 7f0d8affd700 1 -- 172.21.15.78:0/1696930744 >> v1:172.21.15.78:6789/0 conn(0x7f0d8c0b3a60 legacy=0x7f0d8c06d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:01.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.089+0000 7f0d8affd700 1 -- 172.21.15.78:0/1696930744 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d8c0b3260 con 0x7f0d8c15a6b0 2023-10-11T09:48:01.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.089+0000 7f0d89ffb700 1 -- 172.21.15.78:0/1696930744 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1788832443 0 0) 0x7f0d8c0b2d30 con 0x7f0d8c0b3a60 2023-10-11T09:48:01.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.089+0000 7f0d8affd700 1 --2- 172.21.15.78:0/1696930744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d8c15a6b0 0x7f0d8c0ce8a0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c009910 tx=0x7f0d7c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f742c8dda61b79f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:01.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.089+0000 7f0d89ffb700 1 -- 172.21.15.78:0/1696930744 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c00ba60 con 0x7f0d8c15a6b0 2023-10-11T09:48:01.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.089+0000 7f0d89ffb700 1 -- 172.21.15.78:0/1696930744 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7c00bbc0 con 0x7f0d8c15a6b0 2023-10-11T09:48:01.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.089+0000 7f0d89ffb700 1 -- 172.21.15.78:0/1696930744 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c009e60 con 0x7f0d8c15a6b0 2023-10-11T09:48:01.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.089+0000 7f0d8bfff700 1 -- 172.21.15.78:0/1696930744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d8c15a6b0 msgr2=0x7f0d8c0ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:01.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.089+0000 7f0d8bfff700 1 --2- 172.21.15.78:0/1696930744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d8c15a6b0 0x7f0d8c0ce8a0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c009910 tx=0x7f0d7c009c20 comp rx=0 tx=0).stop 2023-10-11T09:48:01.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.089+0000 7f0d8bfff700 1 -- 172.21.15.78:0/1696930744 shutdown_connections 2023-10-11T09:48:01.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.089+0000 7f0d8bfff700 1 --2- 172.21.15.78:0/1696930744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d8c15a6b0 0x7f0d8c0ce8a0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:01.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.089+0000 7f0d8bfff700 1 -- 172.21.15.78:0/1696930744 >> 172.21.15.78:0/1696930744 conn(0x7f0d8c06a060 msgr2=0x7f0d8c06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:01.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.089+0000 7f0d8bfff700 1 -- 172.21.15.78:0/1696930744 shutdown_connections 2023-10-11T09:48:01.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.089+0000 7f0d8bfff700 1 -- 172.21.15.78:0/1696930744 wait complete. 2023-10-11T09:48:01.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.093+0000 7f0d8bfff700 1 Processor -- start 2023-10-11T09:48:01.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.093+0000 7f0d8bfff700 1 -- start start 2023-10-11T09:48:01.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.097+0000 7f0d8bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d8c06d730 0x7f0d8c1458d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:01.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.097+0000 7f0d8bfff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d8c0b3a60 0x7f0d8c153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:01.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.097+0000 7f0d8bfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d8c15a6b0 0x7f0d8c1f2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:01.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.097+0000 7f0d8bfff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8c153fa0 con 0x7f0d8c06d730 2023-10-11T09:48:01.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.097+0000 7f0d8bfff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8c157450 con 0x7f0d8c15a6b0 2023-10-11T09:48:01.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.097+0000 7f0d8bfff700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8c159780 con 0x7f0d8c0b3a60 2023-10-11T09:48:01.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.097+0000 7f0d8affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d8c06d730 0x7f0d8c1458d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:01.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.097+0000 7f0d8affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d8c06d730 0x7f0d8c1458d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48526/0 (socket says 172.21.15.78:48526) 2023-10-11T09:48:01.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.097+0000 7f0d8affd700 1 -- 172.21.15.78:0/2161923817 learned_addr learned my addr 172.21.15.78:0/2161923817 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:01.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.097+0000 7f0d8a7fc700 1 --2- 172.21.15.78:0/2161923817 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d8c0b3a60 0x7f0d8c153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:01.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.097+0000 7f0d8b7fe700 1 --2- 172.21.15.78:0/2161923817 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d8c15a6b0 0x7f0d8c1f2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:01.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.097+0000 7f0d8affd700 1 -- 172.21.15.78:0/2161923817 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d8c0b3a60 msgr2=0x7f0d8c153a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:01.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.097+0000 7f0d8affd700 1 --2- 172.21.15.78:0/2161923817 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d8c0b3a60 0x7f0d8c153a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:01.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.097+0000 7f0d8affd700 1 -- 172.21.15.78:0/2161923817 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d8c15a6b0 msgr2=0x7f0d8c1f2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:01.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.097+0000 7f0d8affd700 1 --2- 172.21.15.78:0/2161923817 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d8c15a6b0 0x7f0d8c1f2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:01.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.097+0000 7f0d8affd700 1 -- 172.21.15.78:0/2161923817 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d7c009570 con 0x7f0d8c06d730 2023-10-11T09:48:01.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.097+0000 7f0d8affd700 1 --2- 172.21.15.78:0/2161923817 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d8c06d730 0x7f0d8c1458d0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c009c60 tx=0x7f0d7c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:01.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.097+0000 7f0d737fe700 1 -- 172.21.15.78:0/2161923817 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c0119f0 con 0x7f0d8c06d730 2023-10-11T09:48:01.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.097+0000 7f0d737fe700 1 -- 172.21.15.78:0/2161923817 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7c01e070 con 0x7f0d8c06d730 2023-10-11T09:48:01.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.097+0000 7f0d8bfff700 1 -- 172.21.15.78:0/2161923817 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d8c157710 con 0x7f0d8c06d730 2023-10-11T09:48:01.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.097+0000 7f0d737fe700 1 -- 172.21.15.78:0/2161923817 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c011d90 con 0x7f0d8c06d730 2023-10-11T09:48:01.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.097+0000 7f0d8bfff700 1 -- 172.21.15.78:0/2161923817 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d8c157cd0 con 0x7f0d8c06d730 2023-10-11T09:48:01.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.101+0000 7f0d8bfff700 1 -- 172.21.15.78:0/2161923817 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d8c065d40 con 0x7f0d8c06d730 2023-10-11T09:48:01.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.101+0000 7f0d737fe700 1 -- 172.21.15.78:0/2161923817 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f0d7c02d450 con 0x7f0d8c06d730 2023-10-11T09:48:01.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.101+0000 7f0d737fe700 1 --2- 172.21.15.78:0/2161923817 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0d78065ed0 0x7f0d78068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:01.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.101+0000 7f0d737fe700 1 -- 172.21.15.78:0/2161923817 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f0d7c084b10 con 0x7f0d8c06d730 2023-10-11T09:48:01.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.101+0000 7f0d8a7fc700 1 --2- 172.21.15.78:0/2161923817 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0d78065ed0 0x7f0d78068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:01.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.105+0000 7f0d8a7fc700 1 --2- 172.21.15.78:0/2161923817 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0d78065ed0 0x7f0d78068380 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c150380 tx=0x7f0d80007a80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:01.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.109+0000 7f0d737fe700 1 -- 172.21.15.78:0/2161923817 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f0d7c051ed0 con 0x7f0d8c06d730 2023-10-11T09:48:01.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.281+0000 7f0d8bfff700 1 -- 172.21.15.78:0/2161923817 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7f0d8c14edd0 con 0x7f0d8c06d730 2023-10-11T09:48:01.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.281+0000 7f0d737fe700 1 -- 172.21.15.78:0/2161923817 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v70) v1 ==== 106+0+11 (secure 0 0 0) 0x7f0d7c0150b0 con 0x7f0d8c06d730 2023-10-11T09:48:01.290 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:48:01.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.285+0000 7f0d8bfff700 1 -- 172.21.15.78:0/2161923817 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0d78065ed0 msgr2=0x7f0d78068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:01.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.285+0000 7f0d8bfff700 1 --2- 172.21.15.78:0/2161923817 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0d78065ed0 0x7f0d78068380 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c150380 tx=0x7f0d80007a80 comp rx=0 tx=0).stop 2023-10-11T09:48:01.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.285+0000 7f0d8bfff700 1 -- 172.21.15.78:0/2161923817 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d8c06d730 msgr2=0x7f0d8c1458d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:01.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.285+0000 7f0d8bfff700 1 --2- 172.21.15.78:0/2161923817 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d8c06d730 0x7f0d8c1458d0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c009c60 tx=0x7f0d7c004770 comp rx=0 tx=0).stop 2023-10-11T09:48:01.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.285+0000 7f0d8bfff700 1 -- 172.21.15.78:0/2161923817 shutdown_connections 2023-10-11T09:48:01.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.285+0000 7f0d8bfff700 1 --2- 172.21.15.78:0/2161923817 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0d78065ed0 0x7f0d78068380 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:01.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.285+0000 7f0d8bfff700 1 --2- 172.21.15.78:0/2161923817 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d8c15a6b0 0x7f0d8c1f2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:01.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.285+0000 7f0d8bfff700 1 --2- 172.21.15.78:0/2161923817 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d8c0b3a60 0x7f0d8c153a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:01.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.285+0000 7f0d8bfff700 1 --2- 172.21.15.78:0/2161923817 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d8c06d730 0x7f0d8c1458d0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:01.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.285+0000 7f0d8bfff700 1 -- 172.21.15.78:0/2161923817 >> 172.21.15.78:0/2161923817 conn(0x7f0d8c06a060 msgr2=0x7f0d8c0cc620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:01.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.285+0000 7f0d8bfff700 1 -- 172.21.15.78:0/2161923817 shutdown_connections 2023-10-11T09:48:01.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.285+0000 7f0d8bfff700 1 -- 172.21.15.78:0/2161923817 wait complete. 2023-10-11T09:48:01.314 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T09:48:01.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.481+0000 7f193359e700 1 Processor -- start 2023-10-11T09:48:01.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.481+0000 7f193359e700 1 -- start start 2023-10-11T09:48:01.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.481+0000 7f193359e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f19340ca950 0x7f19341650f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:01.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.481+0000 7f193359e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f193415e010 con 0x7f193415b8e0 2023-10-11T09:48:01.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.481+0000 7f193359e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f193415e280 con 0x7f193415a940 2023-10-11T09:48:01.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.481+0000 7f193359e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f193415e560 con 0x7f19340ca950 2023-10-11T09:48:01.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.481+0000 7f193259c700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f193415b8e0 0x7f193406d0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:41626/0 (socket says 172.21.15.78:41626) 2023-10-11T09:48:01.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.481+0000 7f193259c700 1 -- 172.21.15.78:0/3278031220 learned_addr learned my addr 172.21.15.78:0/3278031220 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:01.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.481+0000 7f1932d9d700 1 --2- 172.21.15.78:0/3278031220 >> v2:172.21.15.78:3300/0 conn(0x7f19340ca950 0x7f19341650f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:01.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.485+0000 7f1931d9b700 1 -- 172.21.15.78:0/3278031220 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2076170965 0 0) 0x7f193415e010 con 0x7f193415b8e0 2023-10-11T09:48:01.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.485+0000 7f1931d9b700 1 -- 172.21.15.78:0/3278031220 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19200035d0 con 0x7f193415b8e0 2023-10-11T09:48:01.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.485+0000 7f1931d9b700 1 -- 172.21.15.78:0/3278031220 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2417688873 0 0) 0x7f193415e280 con 0x7f193415a940 2023-10-11T09:48:01.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.485+0000 7f1931d9b700 1 -- 172.21.15.78:0/3278031220 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f193415e010 con 0x7f193415a940 2023-10-11T09:48:01.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.485+0000 7f1931d9b700 1 -- 172.21.15.78:0/3278031220 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3543169556 0 0) 0x7f19200035d0 con 0x7f193415b8e0 2023-10-11T09:48:01.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.485+0000 7f1932d9d700 1 -- 172.21.15.78:0/3278031220 >> v1:172.21.15.133:6789/0 conn(0x7f193415a940 legacy=0x7f19340ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:01.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.485+0000 7f1932d9d700 1 -- 172.21.15.78:0/3278031220 >> v1:172.21.15.78:6789/0 conn(0x7f193415b8e0 legacy=0x7f193406d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:01.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.485+0000 7f1932d9d700 1 -- 172.21.15.78:0/3278031220 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f193415e840 con 0x7f19340ca950 2023-10-11T09:48:01.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.485+0000 7f1932d9d700 1 --2- 172.21.15.78:0/3278031220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19340ca950 0x7f19341650f0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f192c009d90 tx=0x7f192c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce664588ff2898ce server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:01.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.485+0000 7f1931d9b700 1 -- 172.21.15.78:0/3278031220 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c00e070 con 0x7f19340ca950 2023-10-11T09:48:01.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.485+0000 7f1931d9b700 1 -- 172.21.15.78:0/3278031220 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f192c003a40 con 0x7f19340ca950 2023-10-11T09:48:01.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.485+0000 7f1931d9b700 1 -- 172.21.15.78:0/3278031220 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1924001610 con 0x7f19340ca950 2023-10-11T09:48:01.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.485+0000 7f193359e700 1 -- 172.21.15.78:0/3278031220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19340ca950 msgr2=0x7f19341650f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:01.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.485+0000 7f193359e700 1 --2- 172.21.15.78:0/3278031220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19340ca950 0x7f19341650f0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f192c009d90 tx=0x7f192c00b330 comp rx=0 tx=0).stop 2023-10-11T09:48:01.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.485+0000 7f193359e700 1 -- 172.21.15.78:0/3278031220 shutdown_connections 2023-10-11T09:48:01.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.485+0000 7f193359e700 1 --2- 172.21.15.78:0/3278031220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19340ca950 0x7f19341650f0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:01.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.485+0000 7f193359e700 1 -- 172.21.15.78:0/3278031220 >> 172.21.15.78:0/3278031220 conn(0x7f19340cea60 msgr2=0x7f19340cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:01.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.485+0000 7f193359e700 1 -- 172.21.15.78:0/3278031220 shutdown_connections 2023-10-11T09:48:01.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.485+0000 7f193359e700 1 -- 172.21.15.78:0/3278031220 wait complete. 2023-10-11T09:48:01.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.489+0000 7f193359e700 1 Processor -- start 2023-10-11T09:48:01.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.489+0000 7f193359e700 1 -- start start 2023-10-11T09:48:01.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.489+0000 7f193359e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19340ca950 0x7f19341428e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:01.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.489+0000 7f193359e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f193415a940 0x7f193413f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:01.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.489+0000 7f193359e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f193415b8e0 0x7f193413ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:01.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.489+0000 7f193359e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193413fa10 con 0x7f193415a940 2023-10-11T09:48:01.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.489+0000 7f193359e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19341431c0 con 0x7f19340ca950 2023-10-11T09:48:01.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.489+0000 7f193359e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f193413d9b0 con 0x7f193415b8e0 2023-10-11T09:48:01.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.489+0000 7f192bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f193415a940 0x7f193413f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:01.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.489+0000 7f192bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f193415a940 0x7f193413f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48536/0 (socket says 172.21.15.78:48536) 2023-10-11T09:48:01.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.489+0000 7f192bfff700 1 -- 172.21.15.78:0/3504566620 learned_addr learned my addr 172.21.15.78:0/3504566620 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:01.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.489+0000 7f193259c700 1 --2- 172.21.15.78:0/3504566620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19340ca950 0x7f19341428e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:01.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.493+0000 7f1932d9d700 1 --2- 172.21.15.78:0/3504566620 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f193415b8e0 0x7f193413ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:01.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.493+0000 7f192bfff700 1 -- 172.21.15.78:0/3504566620 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f193415b8e0 msgr2=0x7f193413ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:01.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.493+0000 7f192bfff700 1 --2- 172.21.15.78:0/3504566620 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f193415b8e0 0x7f193413ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:01.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.493+0000 7f192bfff700 1 -- 172.21.15.78:0/3504566620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19340ca950 msgr2=0x7f19341428e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:01.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.493+0000 7f192bfff700 1 --2- 172.21.15.78:0/3504566620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19340ca950 0x7f19341428e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:01.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.493+0000 7f192bfff700 1 -- 172.21.15.78:0/3504566620 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f192c009a70 con 0x7f193415a940 2023-10-11T09:48:01.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.493+0000 7f193259c700 1 --2- 172.21.15.78:0/3504566620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19340ca950 0x7f19341428e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:48:01.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.493+0000 7f192bfff700 1 --2- 172.21.15.78:0/3504566620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f193415a940 0x7f193413f4d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f191c00c730 tx=0x7f191c00caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:01.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.493+0000 7f192affd700 1 -- 172.21.15.78:0/3504566620 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191c0052d0 con 0x7f193415a940 2023-10-11T09:48:01.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.493+0000 7f192affd700 1 -- 172.21.15.78:0/3504566620 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f191c005450 con 0x7f193415a940 2023-10-11T09:48:01.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.493+0000 7f192affd700 1 -- 172.21.15.78:0/3504566620 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191c0038e0 con 0x7f193415a940 2023-10-11T09:48:01.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.493+0000 7f193359e700 1 -- 172.21.15.78:0/3504566620 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1934142ee0 con 0x7f193415a940 2023-10-11T09:48:01.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.493+0000 7f193359e700 1 -- 172.21.15.78:0/3504566620 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19340b3840 con 0x7f193415a940 2023-10-11T09:48:01.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.493+0000 7f193359e700 1 -- 172.21.15.78:0/3504566620 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1934065d40 con 0x7f193415a940 2023-10-11T09:48:01.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.497+0000 7f192affd700 1 -- 172.21.15.78:0/3504566620 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f191c003a40 con 0x7f193415a940 2023-10-11T09:48:01.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.497+0000 7f192affd700 1 --2- 172.21.15.78:0/3504566620 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1914065f00 0x7f19140683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:01.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.497+0000 7f192affd700 1 -- 172.21.15.78:0/3504566620 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f191c0830b0 con 0x7f193415a940 2023-10-11T09:48:01.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.497+0000 7f193259c700 1 --2- 172.21.15.78:0/3504566620 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1914065f00 0x7f19140683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:01.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.497+0000 7f193259c700 1 --2- 172.21.15.78:0/3504566620 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1914065f00 0x7f19140683b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1924001800 tx=0x7f1924001590 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:01.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.501+0000 7f192affd700 1 -- 172.21.15.78:0/3504566620 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f191c0503c0 con 0x7f193415a940 2023-10-11T09:48:01.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.689+0000 7f193359e700 1 -- 172.21.15.78:0/3504566620 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f193415eaf0 con 0x7f193415a940 2023-10-11T09:48:01.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.689+0000 7f192affd700 1 -- 172.21.15.78:0/3504566620 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v70) v1 ==== 80+0+364 (secure 0 0 0) 0x7f191c0539e0 con 0x7f193415a940 2023-10-11T09:48:01.697 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6836/1666964283 2023-10-12T09:46:00.635561+0000 2023-10-11T09:48:01.697 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6834/1666964283 2023-10-12T09:46:00.635561+0000 2023-10-11T09:48:01.697 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6836/1819646091 2023-10-12T09:45:54.334448+0000 2023-10-11T09:48:01.697 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6834/1819646091 2023-10-12T09:45:54.334448+0000 2023-10-11T09:48:01.697 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6837/852594750 2023-10-12T09:45:53.299952+0000 2023-10-11T09:48:01.698 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6835/852594750 2023-10-12T09:45:53.299952+0000 2023-10-11T09:48:01.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.689+0000 7f193359e700 1 -- 172.21.15.78:0/3504566620 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1914065f00 msgr2=0x7f19140683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:01.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.689+0000 7f193359e700 1 --2- 172.21.15.78:0/3504566620 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1914065f00 0x7f19140683b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1924001800 tx=0x7f1924001590 comp rx=0 tx=0).stop 2023-10-11T09:48:01.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.689+0000 7f193359e700 1 -- 172.21.15.78:0/3504566620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f193415a940 msgr2=0x7f193413f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:01.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.689+0000 7f193359e700 1 --2- 172.21.15.78:0/3504566620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f193415a940 0x7f193413f4d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f191c00c730 tx=0x7f191c00caf0 comp rx=0 tx=0).stop 2023-10-11T09:48:01.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.689+0000 7f193359e700 1 -- 172.21.15.78:0/3504566620 shutdown_connections 2023-10-11T09:48:01.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.689+0000 7f193359e700 1 --2- 172.21.15.78:0/3504566620 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1914065f00 0x7f19140683b0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:01.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.689+0000 7f193359e700 1 --2- 172.21.15.78:0/3504566620 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f193415b8e0 0x7f193413ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:01.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.689+0000 7f193359e700 1 --2- 172.21.15.78:0/3504566620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f193415a940 0x7f193413f4d0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:01.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.689+0000 7f193359e700 1 --2- 172.21.15.78:0/3504566620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19340ca950 0x7f19341428e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:01.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.689+0000 7f193359e700 1 -- 172.21.15.78:0/3504566620 >> 172.21.15.78:0/3504566620 conn(0x7f19340cea60 msgr2=0x7f1934143640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:01.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.693+0000 7f193359e700 1 -- 172.21.15.78:0/3504566620 shutdown_connections 2023-10-11T09:48:01.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:01.693+0000 7f193359e700 1 -- 172.21.15.78:0/3504566620 wait complete. 2023-10-11T09:48:01.701 INFO:teuthology.orchestra.run.smithi078.stderr:listed 6 entries 2023-10-11T09:48:01.720 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi078.front.sepia.ceph.com... 2023-10-11T09:48:01.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:48:01.721 DEBUG:teuthology.orchestra.run.smithi078:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-11T09:48:01.751 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:ceph-fuse[51345]: fuse finished with error 0 and tester_r 0 2023-10-11T09:48:01.787 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2023-10-11T09:48:01.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:48:01.788 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-11T09:48:01.855 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:48:01.855 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:48:01.875 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352292k,nr_inodes=4088073,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1430 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275376k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-11T09:48:01.876 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2023-10-11T09:48:01.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:48:01.877 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-11T09:48:01.935 INFO:teuthology.orchestra.run.smithi133.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-10-11T09:48:01.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T09:48:01.937 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:48:01.938 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:48:01.957 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352272k,nr_inodes=4088068,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17487 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-11T09:48:01.958 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2023-10-11T09:48:01.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:48:01.958 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-11T09:48:02.016 INFO:teuthology.orchestra.run.smithi078.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-11T09:48:02.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T09:48:02.018 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:48:02.019 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:48:02.036 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352292k,nr_inodes=4088073,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1430 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275376k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-11T09:48:02.036 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2023-10-11T09:48:02.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:48:02.037 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-11T09:48:02.101 INFO:teuthology.orchestra.run.smithi133.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-11T09:48:02.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T09:48:02.103 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:48:02.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.273+0000 7f3c1cdbf700 1 Processor -- start 2023-10-11T09:48:02.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.273+0000 7f3c1cdbf700 1 -- start start 2023-10-11T09:48:02.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.273+0000 7f3c1cdbf700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c1815c880 0x7f3c1806be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:02.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.273+0000 7f3c1cdbf700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3c180cd210 0x7f3c180cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:02.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.273+0000 7f3c1cdbf700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c180c93a0 con 0x7f3c18160e60 2023-10-11T09:48:02.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.273+0000 7f3c1cdbf700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c18161420 con 0x7f3c180cd210 2023-10-11T09:48:02.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.273+0000 7f3c1cdbf700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c180cca10 con 0x7f3c1815c880 2023-10-11T09:48:02.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.273+0000 7f3c16d9d700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f3c18160e60 0x7f3c180c9b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:41636/0 (socket says 172.21.15.78:41636) 2023-10-11T09:48:02.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.273+0000 7f3c16d9d700 1 -- 172.21.15.78:0/1858059137 learned_addr learned my addr 172.21.15.78:0/1858059137 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:02.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.273+0000 7f3c1659c700 1 --2- 172.21.15.78:0/1858059137 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c1815c880 0x7f3c1806be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:02.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.273+0000 7f3c15d9b700 1 --2- 172.21.15.78:0/1858059137 >> v2:172.21.15.133:3300/0 conn(0x7f3c180cd210 0x7f3c180cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:02.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.277+0000 7f3c1559a700 1 -- 172.21.15.78:0/1858059137 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3813714802 0 0) 0x7f3c180c93a0 con 0x7f3c18160e60 2023-10-11T09:48:02.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.277+0000 7f3c1559a700 1 -- 172.21.15.78:0/1858059137 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c000035d0 con 0x7f3c18160e60 2023-10-11T09:48:02.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.277+0000 7f3c1659c700 1 -- 172.21.15.78:0/1858059137 >> v2:172.21.15.133:3300/0 conn(0x7f3c180cd210 msgr2=0x7f3c180cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:02.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.277+0000 7f3c1659c700 1 --2- 172.21.15.78:0/1858059137 >> v2:172.21.15.133:3300/0 conn(0x7f3c180cd210 0x7f3c180cb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:02.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.277+0000 7f3c1659c700 1 -- 172.21.15.78:0/1858059137 >> v1:172.21.15.78:6789/0 conn(0x7f3c18160e60 legacy=0x7f3c180c9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:02.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.277+0000 7f3c1559a700 1 -- 172.21.15.78:0/1858059137 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3045077317 0 0) 0x7f3c000035d0 con 0x7f3c18160e60 2023-10-11T09:48:02.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.277+0000 7f3c1659c700 1 -- 172.21.15.78:0/1858059137 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c180cccb0 con 0x7f3c1815c880 2023-10-11T09:48:02.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.277+0000 7f3c1659c700 1 --2- 172.21.15.78:0/1858059137 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c1815c880 0x7f3c1806be60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3c08009850 tx=0x7f3c08009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b10a691780e50a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:02.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.277+0000 7f3c1559a700 1 -- 172.21.15.78:0/1858059137 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c08011830 con 0x7f3c1815c880 2023-10-11T09:48:02.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.277+0000 7f3c1559a700 1 -- 172.21.15.78:0/1858059137 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c08011990 con 0x7f3c1815c880 2023-10-11T09:48:02.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.277+0000 7f3c1559a700 1 -- 172.21.15.78:0/1858059137 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c08011d40 con 0x7f3c1815c880 2023-10-11T09:48:02.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.277+0000 7f3c1cdbf700 1 -- 172.21.15.78:0/1858059137 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c1815c880 msgr2=0x7f3c1806be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:02.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.277+0000 7f3c1cdbf700 1 --2- 172.21.15.78:0/1858059137 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c1815c880 0x7f3c1806be60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3c08009850 tx=0x7f3c08009b60 comp rx=0 tx=0).stop 2023-10-11T09:48:02.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.277+0000 7f3c1cdbf700 1 -- 172.21.15.78:0/1858059137 shutdown_connections 2023-10-11T09:48:02.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.277+0000 7f3c1cdbf700 1 --2- 172.21.15.78:0/1858059137 >> v2:172.21.15.133:3300/0 conn(0x7f3c180cd210 0x7f3c180cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:02.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.277+0000 7f3c1cdbf700 1 --2- 172.21.15.78:0/1858059137 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c1815c880 0x7f3c1806be60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:02.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.277+0000 7f3c1cdbf700 1 -- 172.21.15.78:0/1858059137 >> 172.21.15.78:0/1858059137 conn(0x7f3c1806aca0 msgr2=0x7f3c180b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:02.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.277+0000 7f3c1cdbf700 1 -- 172.21.15.78:0/1858059137 shutdown_connections 2023-10-11T09:48:02.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.277+0000 7f3c1cdbf700 1 -- 172.21.15.78:0/1858059137 wait complete. 2023-10-11T09:48:02.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.281+0000 7f3c1cdbf700 1 Processor -- start 2023-10-11T09:48:02.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.281+0000 7f3c1cdbf700 1 -- start start 2023-10-11T09:48:02.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.281+0000 7f3c1cdbf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c180cd210 0x7f3c1815fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:02.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.281+0000 7f3c1cdbf700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c1815c880 0x7f3c1814f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:02.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.281+0000 7f3c1cdbf700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c18160e60 0x7f3c1815d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:02.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.281+0000 7f3c1cdbf700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c181501a0 con 0x7f3c180cd210 2023-10-11T09:48:02.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.281+0000 7f3c1cdbf700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c181f6190 con 0x7f3c1815c880 2023-10-11T09:48:02.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.281+0000 7f3c1cdbf700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1815dde0 con 0x7f3c18160e60 2023-10-11T09:48:02.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.281+0000 7f3c1659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c180cd210 0x7f3c1815fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:02.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.285+0000 7f3c1659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c180cd210 0x7f3c1815fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48550/0 (socket says 172.21.15.78:48550) 2023-10-11T09:48:02.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.285+0000 7f3c1659c700 1 -- 172.21.15.78:0/2870495117 learned_addr learned my addr 172.21.15.78:0/2870495117 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:02.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.285+0000 7f3c16d9d700 1 --2- 172.21.15.78:0/2870495117 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c18160e60 0x7f3c1815d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:02.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.285+0000 7f3c15d9b700 1 --2- 172.21.15.78:0/2870495117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c1815c880 0x7f3c1814f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:02.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.285+0000 7f3c1659c700 1 -- 172.21.15.78:0/2870495117 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c18160e60 msgr2=0x7f3c1815d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:02.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.285+0000 7f3c1659c700 1 --2- 172.21.15.78:0/2870495117 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c18160e60 0x7f3c1815d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:02.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.285+0000 7f3c1659c700 1 -- 172.21.15.78:0/2870495117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c1815c880 msgr2=0x7f3c1814f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:02.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.285+0000 7f3c1659c700 1 --2- 172.21.15.78:0/2870495117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c1815c880 0x7f3c1814f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:02.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.285+0000 7f3c1659c700 1 -- 172.21.15.78:0/2870495117 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c080094b0 con 0x7f3c180cd210 2023-10-11T09:48:02.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.285+0000 7f3c1659c700 1 --2- 172.21.15.78:0/2870495117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c180cd210 0x7f3c1815fd40 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f3c080090b0 tx=0x7f3c08004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:02.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.285+0000 7f3c06ffd700 1 -- 172.21.15.78:0/2870495117 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0801a8d0 con 0x7f3c180cd210 2023-10-11T09:48:02.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.285+0000 7f3c06ffd700 1 -- 172.21.15.78:0/2870495117 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c0801f070 con 0x7f3c180cd210 2023-10-11T09:48:02.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.285+0000 7f3c06ffd700 1 -- 172.21.15.78:0/2870495117 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0801ac90 con 0x7f3c180cd210 2023-10-11T09:48:02.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.285+0000 7f3c1cdbf700 1 -- 172.21.15.78:0/2870495117 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c18160320 con 0x7f3c180cd210 2023-10-11T09:48:02.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.285+0000 7f3c1cdbf700 1 -- 172.21.15.78:0/2870495117 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c181608e0 con 0x7f3c180cd210 2023-10-11T09:48:02.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.285+0000 7f3c1cdbf700 1 -- 172.21.15.78:0/2870495117 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c1804e530 con 0x7f3c180cd210 2023-10-11T09:48:02.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.289+0000 7f3c06ffd700 1 -- 172.21.15.78:0/2870495117 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f3c0801b430 con 0x7f3c180cd210 2023-10-11T09:48:02.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.289+0000 7f3c06ffd700 1 --2- 172.21.15.78:0/2870495117 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3bf8065ed0 0x7f3bf8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:02.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.289+0000 7f3c06ffd700 1 -- 172.21.15.78:0/2870495117 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f3c08084bd0 con 0x7f3c180cd210 2023-10-11T09:48:02.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.289+0000 7f3c15d9b700 1 --2- 172.21.15.78:0/2870495117 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3bf8065ed0 0x7f3bf8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:02.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.289+0000 7f3c15d9b700 1 --2- 172.21.15.78:0/2870495117 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3bf8065ed0 0x7f3bf8068380 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3c181f97e0 tx=0x7f3c0c009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:02.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.297+0000 7f3c06ffd700 1 -- 172.21.15.78:0/2870495117 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f3c08051e50 con 0x7f3c180cd210 2023-10-11T09:48:02.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.509+0000 7f3c1cdbf700 1 -- 172.21.15.78:0/2870495117 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c180b1170 con 0x7f3c180cd210 2023-10-11T09:48:02.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.509+0000 7f3c06ffd700 1 -- 172.21.15.78:0/2870495117 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4655 (secure 0 0 0) 0x7f3c08021030 con 0x7f3c180cd210 2023-10-11T09:48:02.519 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:02.519 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":50,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6664,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2877345875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2877345875},{"type":"v1","addr":"172.21.15.133:6833","nonce":2877345875}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":6667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/792033922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":792033922},{"type":"v1","addr":"172.21.15.133:6835","nonce":792033922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":6894,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/1229638118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":1229638118},{"type":"v1","addr":"172.21.15.78:6836","nonce":1229638118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:45:54.252958+0000","modified":"2023-10-11T09:46:03.678418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6897},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6897":{"gid":6897,"name":"a","rank":0,"incarnation":47,"state":"up:active","state_seq":4,"addr":"172.21.15.78:6837/3498868661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":3498868661},{"type":"v1","addr":"172.21.15.78:6837","nonce":3498868661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9,10],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-11T09:48:02.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.513+0000 7f3c1cdbf700 1 -- 172.21.15.78:0/2870495117 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3bf8065ed0 msgr2=0x7f3bf8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:02.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.513+0000 7f3c1cdbf700 1 --2- 172.21.15.78:0/2870495117 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3bf8065ed0 0x7f3bf8068380 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3c181f97e0 tx=0x7f3c0c009240 comp rx=0 tx=0).stop 2023-10-11T09:48:02.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.513+0000 7f3c1cdbf700 1 -- 172.21.15.78:0/2870495117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c180cd210 msgr2=0x7f3c1815fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:02.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.513+0000 7f3c1cdbf700 1 --2- 172.21.15.78:0/2870495117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c180cd210 0x7f3c1815fd40 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f3c080090b0 tx=0x7f3c08004770 comp rx=0 tx=0).stop 2023-10-11T09:48:02.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.513+0000 7f3c1cdbf700 1 -- 172.21.15.78:0/2870495117 shutdown_connections 2023-10-11T09:48:02.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.513+0000 7f3c1cdbf700 1 --2- 172.21.15.78:0/2870495117 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3bf8065ed0 0x7f3bf8068380 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:02.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.513+0000 7f3c1cdbf700 1 --2- 172.21.15.78:0/2870495117 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c18160e60 0x7f3c1815d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:02.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.513+0000 7f3c1cdbf700 1 --2- 172.21.15.78:0/2870495117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c1815c880 0x7f3c1814f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:02.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.513+0000 7f3c1cdbf700 1 --2- 172.21.15.78:0/2870495117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c180cd210 0x7f3c1815fd40 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:02.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.513+0000 7f3c1cdbf700 1 -- 172.21.15.78:0/2870495117 >> 172.21.15.78:0/2870495117 conn(0x7f3c1806aca0 msgr2=0x7f3c180cb4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:02.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.513+0000 7f3c1cdbf700 1 -- 172.21.15.78:0/2870495117 shutdown_connections 2023-10-11T09:48:02.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.513+0000 7f3c1cdbf700 1 -- 172.21.15.78:0/2870495117 wait complete. 2023-10-11T09:48:02.525 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 50 2023-10-11T09:48:02.543 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:48:02.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.701+0000 7f8044e17700 1 Processor -- start 2023-10-11T09:48:02.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.701+0000 7f8044e17700 1 -- start start 2023-10-11T09:48:02.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.705+0000 7f8044e17700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8040154e00 0x7f804015b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:02.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.705+0000 7f8044e17700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80400720f0 con 0x7f80400cbbd0 2023-10-11T09:48:02.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.705+0000 7f8044e17700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f804015bb90 con 0x7f8040071040 2023-10-11T09:48:02.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.705+0000 7f8044e17700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80400714e0 con 0x7f8040154e00 2023-10-11T09:48:02.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.705+0000 7f803ed9d700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f80400cbbd0 0x7f804006c010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:41642/0 (socket says 172.21.15.78:41642) 2023-10-11T09:48:02.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.705+0000 7f803ed9d700 1 -- 172.21.15.78:0/3299739173 learned_addr learned my addr 172.21.15.78:0/3299739173 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:02.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.705+0000 7f803e59c700 1 --2- 172.21.15.78:0/3299739173 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8040154e00 0x7f804015b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:02.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.705+0000 7f803dd9b700 1 -- 172.21.15.78:0/3299739173 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2527649606 0 0) 0x7f80400720f0 con 0x7f80400cbbd0 2023-10-11T09:48:02.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.705+0000 7f803dd9b700 1 -- 172.21.15.78:0/3299739173 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f802c0035d0 con 0x7f80400cbbd0 2023-10-11T09:48:02.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.705+0000 7f803dd9b700 1 -- 172.21.15.78:0/3299739173 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3811600885 0 0) 0x7f804015bb90 con 0x7f8040071040 2023-10-11T09:48:02.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.705+0000 7f803dd9b700 1 -- 172.21.15.78:0/3299739173 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80400720f0 con 0x7f8040071040 2023-10-11T09:48:02.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.705+0000 7f803e59c700 1 -- 172.21.15.78:0/3299739173 >> v1:172.21.15.133:6789/0 conn(0x7f8040071040 legacy=0x7f80400cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:02.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.705+0000 7f803e59c700 1 -- 172.21.15.78:0/3299739173 >> v1:172.21.15.78:6789/0 conn(0x7f80400cbbd0 legacy=0x7f804006c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:02.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.705+0000 7f803e59c700 1 -- 172.21.15.78:0/3299739173 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80400717c0 con 0x7f8040154e00 2023-10-11T09:48:02.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.709+0000 7f803e59c700 1 --2- 172.21.15.78:0/3299739173 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8040154e00 0x7f804015b650 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8030009880 tx=0x7f8030009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74916a4b9b6f1c23 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:02.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.709+0000 7f803dd9b700 1 -- 172.21.15.78:0/3299739173 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8030011830 con 0x7f8040154e00 2023-10-11T09:48:02.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.709+0000 7f803dd9b700 1 -- 172.21.15.78:0/3299739173 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8030011990 con 0x7f8040154e00 2023-10-11T09:48:02.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.709+0000 7f803dd9b700 1 -- 172.21.15.78:0/3299739173 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8030011d20 con 0x7f8040154e00 2023-10-11T09:48:02.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.709+0000 7f8044e17700 1 -- 172.21.15.78:0/3299739173 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8040154e00 msgr2=0x7f804015b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:02.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.709+0000 7f8044e17700 1 --2- 172.21.15.78:0/3299739173 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8040154e00 0x7f804015b650 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8030009880 tx=0x7f8030009b90 comp rx=0 tx=0).stop 2023-10-11T09:48:02.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.709+0000 7f8044e17700 1 -- 172.21.15.78:0/3299739173 shutdown_connections 2023-10-11T09:48:02.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.709+0000 7f8044e17700 1 --2- 172.21.15.78:0/3299739173 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8040154e00 0x7f804015b650 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:02.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.709+0000 7f8044e17700 1 -- 172.21.15.78:0/3299739173 >> 172.21.15.78:0/3299739173 conn(0x7f804006aca0 msgr2=0x7f80400b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:02.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.709+0000 7f8044e17700 1 -- 172.21.15.78:0/3299739173 shutdown_connections 2023-10-11T09:48:02.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.709+0000 7f8044e17700 1 -- 172.21.15.78:0/3299739173 wait complete. 2023-10-11T09:48:02.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.713+0000 7f8044e17700 1 Processor -- start 2023-10-11T09:48:02.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.713+0000 7f8044e17700 1 -- start start 2023-10-11T09:48:02.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.713+0000 7f8044e17700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8040071040 0x7f80400ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:02.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.713+0000 7f8044e17700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80400cbbd0 0x7f804014b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:02.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.713+0000 7f8044e17700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8040154e00 0x7f804006c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:02.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.713+0000 7f8044e17700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804014cd40 con 0x7f80400cbbd0 2023-10-11T09:48:02.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.713+0000 7f8044e17700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80400aca70 con 0x7f8040154e00 2023-10-11T09:48:02.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.713+0000 7f8044e17700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80400cd300 con 0x7f8040071040 2023-10-11T09:48:02.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.713+0000 7f8037fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80400cbbd0 0x7f804014b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:02.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.713+0000 7f8037fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80400cbbd0 0x7f804014b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48552/0 (socket says 172.21.15.78:48552) 2023-10-11T09:48:02.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.713+0000 7f8037fff700 1 -- 172.21.15.78:0/1947711018 learned_addr learned my addr 172.21.15.78:0/1947711018 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:02.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.713+0000 7f803e59c700 1 --2- 172.21.15.78:0/1947711018 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8040071040 0x7f80400ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:02.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.713+0000 7f803ed9d700 1 --2- 172.21.15.78:0/1947711018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8040154e00 0x7f804006c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:02.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.713+0000 7f8037fff700 1 -- 172.21.15.78:0/1947711018 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8040071040 msgr2=0x7f80400ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:02.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.713+0000 7f8037fff700 1 --2- 172.21.15.78:0/1947711018 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8040071040 0x7f80400ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:02.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.713+0000 7f8037fff700 1 -- 172.21.15.78:0/1947711018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8040154e00 msgr2=0x7f804006c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:02.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.713+0000 7f8037fff700 1 --2- 172.21.15.78:0/1947711018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8040154e00 0x7f804006c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:02.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.713+0000 7f8037fff700 1 -- 172.21.15.78:0/1947711018 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80300094b0 con 0x7f80400cbbd0 2023-10-11T09:48:02.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.717+0000 7f8037fff700 1 --2- 172.21.15.78:0/1947711018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80400cbbd0 0x7f804014b0f0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f802800c500 tx=0x7f802800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:02.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.717+0000 7f8036ffd700 1 -- 172.21.15.78:0/1947711018 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80280052d0 con 0x7f80400cbbd0 2023-10-11T09:48:02.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.717+0000 7f8036ffd700 1 -- 172.21.15.78:0/1947711018 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8028005450 con 0x7f80400cbbd0 2023-10-11T09:48:02.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.717+0000 7f8044e17700 1 -- 172.21.15.78:0/1947711018 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f804014ca60 con 0x7f80400cbbd0 2023-10-11T09:48:02.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.717+0000 7f8036ffd700 1 -- 172.21.15.78:0/1947711018 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80280038e0 con 0x7f80400cbbd0 2023-10-11T09:48:02.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.717+0000 7f8044e17700 1 -- 172.21.15.78:0/1947711018 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80400acff0 con 0x7f80400cbbd0 2023-10-11T09:48:02.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.717+0000 7f8044e17700 1 -- 172.21.15.78:0/1947711018 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80400ac8d0 con 0x7f80400cbbd0 2023-10-11T09:48:02.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.717+0000 7f8036ffd700 1 -- 172.21.15.78:0/1947711018 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f8028003a40 con 0x7f80400cbbd0 2023-10-11T09:48:02.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.721+0000 7f8036ffd700 1 --2- 172.21.15.78:0/1947711018 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8020065ed0 0x7f8020068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:02.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.721+0000 7f8036ffd700 1 -- 172.21.15.78:0/1947711018 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f8028082ea0 con 0x7f80400cbbd0 2023-10-11T09:48:02.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.721+0000 7f803e59c700 1 --2- 172.21.15.78:0/1947711018 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8020065ed0 0x7f8020068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:02.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.721+0000 7f803e59c700 1 --2- 172.21.15.78:0/1947711018 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8020065ed0 0x7f8020068380 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f80300090b0 tx=0x7f803001c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:02.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.725+0000 7f8036ffd700 1 -- 172.21.15.78:0/1947711018 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f80400ac8d0 con 0x7f80400cbbd0 2023-10-11T09:48:02.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.897+0000 7f8044e17700 1 -- 172.21.15.78:0/1947711018 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f80400ac8d0 con 0x7f80400cbbd0 2023-10-11T09:48:02.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.901+0000 7f8036ffd700 1 -- 172.21.15.78:0/1947711018 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+20052 (secure 0 0 0) 0x7f80400ac8d0 con 0x7f80400cbbd0 2023-10-11T09:48:02.910 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:02.910 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":70,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:46:00.671033+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:44:05.198742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-11T09:44:05.868171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data_extra","create_time":"2023-10-11T09:44:16.049941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6836/1666964283":"2023-10-12T09:46:00.635561+0000","172.21.15.78:6834/1666964283":"2023-10-12T09:46:00.635561+0000","172.21.15.78:6836/1819646091":"2023-10-12T09:45:54.334448+0000","172.21.15.78:6834/1819646091":"2023-10-12T09:45:54.334448+0000","172.21.15.78:6837/852594750":"2023-10-12T09:45:53.299952+0000","172.21.15.78:6835/852594750":"2023-10-12T09:45:53.299952+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:48:02.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.905+0000 7f8044e17700 1 -- 172.21.15.78:0/1947711018 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8020065ed0 msgr2=0x7f8020068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:02.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.905+0000 7f8044e17700 1 --2- 172.21.15.78:0/1947711018 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8020065ed0 0x7f8020068380 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f80300090b0 tx=0x7f803001c040 comp rx=0 tx=0).stop 2023-10-11T09:48:02.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.905+0000 7f8044e17700 1 -- 172.21.15.78:0/1947711018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80400cbbd0 msgr2=0x7f804014b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:02.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.905+0000 7f8044e17700 1 --2- 172.21.15.78:0/1947711018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80400cbbd0 0x7f804014b0f0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f802800c500 tx=0x7f802800c810 comp rx=0 tx=0).stop 2023-10-11T09:48:02.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.905+0000 7f8044e17700 1 -- 172.21.15.78:0/1947711018 shutdown_connections 2023-10-11T09:48:02.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.905+0000 7f8044e17700 1 --2- 172.21.15.78:0/1947711018 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8020065ed0 0x7f8020068380 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:02.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.905+0000 7f8044e17700 1 --2- 172.21.15.78:0/1947711018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8040154e00 0x7f804006c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:02.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.905+0000 7f8044e17700 1 --2- 172.21.15.78:0/1947711018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80400cbbd0 0x7f804014b0f0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:02.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.905+0000 7f8044e17700 1 --2- 172.21.15.78:0/1947711018 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8040071040 0x7f80400ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:02.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.905+0000 7f8044e17700 1 -- 172.21.15.78:0/1947711018 >> 172.21.15.78:0/1947711018 conn(0x7f804006aca0 msgr2=0x7f8040159ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:02.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.905+0000 7f8044e17700 1 -- 172.21.15.78:0/1947711018 shutdown_connections 2023-10-11T09:48:02.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:02.905+0000 7f8044e17700 1 -- 172.21.15.78:0/1947711018 wait complete. 2023-10-11T09:48:02.933 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T09:44:05.198742+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T09:44:05.868171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-10-11T09:44:16.049941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:48:02.934 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:48:03.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.101+0000 7f8bbc92b700 1 Processor -- start 2023-10-11T09:48:03.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.101+0000 7f8bbc92b700 1 -- start start 2023-10-11T09:48:03.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.105+0000 7f8bbc92b700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8bb8146fe0 0x7f8bb806bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:03.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.105+0000 7f8bbc92b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f8bb80b36f0 0x7f8bb81629e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:03.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.105+0000 7f8bbc92b700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb806ccb0 con 0x7f8bb8161d90 2023-10-11T09:48:03.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.105+0000 7f8bbc92b700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bb806c410 con 0x7f8bb80b36f0 2023-10-11T09:48:03.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.105+0000 7f8bbc92b700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bb806c6f0 con 0x7f8bb8146fe0 2023-10-11T09:48:03.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.105+0000 7f8bb659c700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f8bb8161d90 0x7f8bb80b2fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:41648/0 (socket says 172.21.15.78:41648) 2023-10-11T09:48:03.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.105+0000 7f8bb659c700 1 -- 172.21.15.78:0/3767792814 learned_addr learned my addr 172.21.15.78:0/3767792814 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:03.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.105+0000 7f8bb759e700 1 --2- 172.21.15.78:0/3767792814 >> v2:172.21.15.78:3300/0 conn(0x7f8bb80b36f0 0x7f8bb81629e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:03.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.105+0000 7f8bb6d9d700 1 --2- 172.21.15.78:0/3767792814 >> v2:172.21.15.133:3300/0 conn(0x7f8bb8146fe0 0x7f8bb806bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:03.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.105+0000 7f8bb5d9b700 1 -- 172.21.15.78:0/3767792814 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 400935428 0 0) 0x7f8bac001110 con 0x7f8bb8161d90 2023-10-11T09:48:03.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.105+0000 7f8bb5d9b700 1 -- 172.21.15.78:0/3767792814 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ba00035d0 con 0x7f8bb8161d90 2023-10-11T09:48:03.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.105+0000 7f8bb5d9b700 1 -- 172.21.15.78:0/3767792814 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1588264231 0 0) 0x7f8ba00035d0 con 0x7f8bb8161d90 2023-10-11T09:48:03.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.105+0000 7f8bb5d9b700 1 -- 172.21.15.78:0/3767792814 >> v2:172.21.15.133:3300/0 conn(0x7f8bb8146fe0 msgr2=0x7f8bb806bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:03.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.105+0000 7f8bb5d9b700 1 --2- 172.21.15.78:0/3767792814 >> v2:172.21.15.133:3300/0 conn(0x7f8bb8146fe0 0x7f8bb806bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:03.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.105+0000 7f8bb5d9b700 1 -- 172.21.15.78:0/3767792814 >> v2:172.21.15.78:3300/0 conn(0x7f8bb80b36f0 msgr2=0x7f8bb81629e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:03.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.105+0000 7f8bb5d9b700 1 --2- 172.21.15.78:0/3767792814 >> v2:172.21.15.78:3300/0 conn(0x7f8bb80b36f0 0x7f8bb81629e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:03.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.105+0000 7f8bb5d9b700 1 -- 172.21.15.78:0/3767792814 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb806c9d0 con 0x7f8bb8161d90 2023-10-11T09:48:03.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.105+0000 7f8bb5d9b700 1 -- 172.21.15.78:0/3767792814 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f8bac001800 con 0x7f8bb8161d90 2023-10-11T09:48:03.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.105+0000 7f8bb5d9b700 1 -- 172.21.15.78:0/3767792814 >> v1:172.21.15.78:6789/0 conn(0x7f8bb8161d90 legacy=0x7f8bb80b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:03.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.105+0000 7f8bb5d9b700 1 --2- 172.21.15.78:0/3767792814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ba0003a20 0x7f8ba0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:03.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.105+0000 7f8bb5d9b700 1 --2- 172.21.15.78:0/3767792814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ba0004870 0x7f8ba0004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:03.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.105+0000 7f8bb5d9b700 1 --2- 172.21.15.78:0/3767792814 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8ba0005220 0x7f8ba000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:03.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.105+0000 7f8bb5d9b700 1 -- 172.21.15.78:0/3767792814 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb806c410 con 0x7f8ba0003a20 2023-10-11T09:48:03.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.105+0000 7f8bb5d9b700 1 -- 172.21.15.78:0/3767792814 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb806c6f0 con 0x7f8ba0004870 2023-10-11T09:48:03.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.105+0000 7f8bb5d9b700 1 -- 172.21.15.78:0/3767792814 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba000c020 con 0x7f8ba0005220 2023-10-11T09:48:03.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.105+0000 7f8bb759e700 1 --2- 172.21.15.78:0/3767792814 >> v2:172.21.15.78:3300/0 conn(0x7f8bb80b36f0 0x7f8bb81629e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:48:03.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.109+0000 7f8bb659c700 1 --2- 172.21.15.78:0/3767792814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ba0004870 0x7f8ba0004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:03.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.109+0000 7f8bb6d9d700 1 --2- 172.21.15.78:0/3767792814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ba0003a20 0x7f8ba0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:03.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.109+0000 7f8bb759e700 1 --2- 172.21.15.78:0/3767792814 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8ba0005220 0x7f8ba000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:03.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.109+0000 7f8bb6d9d700 1 -- 172.21.15.78:0/3767792814 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8ba0005220 msgr2=0x7f8ba000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:03.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.109+0000 7f8bb6d9d700 1 --2- 172.21.15.78:0/3767792814 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8ba0005220 0x7f8ba000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:03.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.109+0000 7f8bb6d9d700 1 -- 172.21.15.78:0/3767792814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ba0004870 msgr2=0x7f8ba0004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:03.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.109+0000 7f8bb6d9d700 1 --2- 172.21.15.78:0/3767792814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ba0004870 0x7f8ba0004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:03.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.109+0000 7f8bb759e700 1 -- 172.21.15.78:0/3767792814 reap_dead start 2023-10-11T09:48:03.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.109+0000 7f8bb6d9d700 1 -- 172.21.15.78:0/3767792814 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba00035d0 con 0x7f8ba0003a20 2023-10-11T09:48:03.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.109+0000 7f8bb6d9d700 1 --2- 172.21.15.78:0/3767792814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ba0003a20 0x7f8ba0003e30 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8009fc0 tx=0x7f8ba00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f170b9a679a1dc3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:03.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.109+0000 7f8bb5d9b700 1 -- 172.21.15.78:0/3767792814 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba800ec30 con 0x7f8ba0003a20 2023-10-11T09:48:03.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.109+0000 7f8bb5d9b700 1 -- 172.21.15.78:0/3767792814 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba800f040 con 0x7f8ba0003a20 2023-10-11T09:48:03.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.109+0000 7f8bb5d9b700 1 -- 172.21.15.78:0/3767792814 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8014480 con 0x7f8ba0003a20 2023-10-11T09:48:03.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.109+0000 7f8bbc92b700 1 -- 172.21.15.78:0/3767792814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ba0003a20 msgr2=0x7f8ba0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:03.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.109+0000 7f8bbc92b700 1 --2- 172.21.15.78:0/3767792814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ba0003a20 0x7f8ba0003e30 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8009fc0 tx=0x7f8ba00058a0 comp rx=0 tx=0).stop 2023-10-11T09:48:03.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.109+0000 7f8bbc92b700 1 -- 172.21.15.78:0/3767792814 shutdown_connections 2023-10-11T09:48:03.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.109+0000 7f8bbc92b700 1 --2- 172.21.15.78:0/3767792814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ba0003a20 0x7f8ba0003e30 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:03.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.109+0000 7f8bbc92b700 1 -- 172.21.15.78:0/3767792814 >> 172.21.15.78:0/3767792814 conn(0x7f8bb806a060 msgr2=0x7f8bb806a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:03.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.109+0000 7f8bbc92b700 1 -- 172.21.15.78:0/3767792814 shutdown_connections 2023-10-11T09:48:03.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.109+0000 7f8bbc92b700 1 -- 172.21.15.78:0/3767792814 wait complete. 2023-10-11T09:48:03.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.113+0000 7f8bbc92b700 1 Processor -- start 2023-10-11T09:48:03.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.113+0000 7f8bbc92b700 1 -- start start 2023-10-11T09:48:03.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.113+0000 7f8bbc92b700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8ba0003a20 0x7f8bb8145ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:03.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.113+0000 7f8bbc92b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8bb80cdb80 0x7f8bb80cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:03.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.117+0000 7f8bbc92b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bb815c930 0x7f8bb815cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:03.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.117+0000 7f8bbc92b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb80ca6a0 con 0x7f8bb80cdb80 2023-10-11T09:48:03.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.117+0000 7f8bbc92b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb80ce690 con 0x7f8bb815c930 2023-10-11T09:48:03.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.117+0000 7f8bbc92b700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb8157fb0 con 0x7f8ba0003a20 2023-10-11T09:48:03.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.117+0000 7f8bb659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8bb80cdb80 0x7f8bb80cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:03.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.117+0000 7f8bb659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8bb80cdb80 0x7f8bb80cdfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48566/0 (socket says 172.21.15.78:48566) 2023-10-11T09:48:03.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.117+0000 7f8bb659c700 1 -- 172.21.15.78:0/208124982 learned_addr learned my addr 172.21.15.78:0/208124982 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:03.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.117+0000 7f8bb759e700 1 --2- 172.21.15.78:0/208124982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bb815c930 0x7f8bb815cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:03.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.117+0000 7f8bb6d9d700 1 --2- 172.21.15.78:0/208124982 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8ba0003a20 0x7f8bb8145ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:03.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.117+0000 7f8bb659c700 1 -- 172.21.15.78:0/208124982 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8ba0003a20 msgr2=0x7f8bb8145ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:03.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.117+0000 7f8bb659c700 1 --2- 172.21.15.78:0/208124982 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8ba0003a20 0x7f8bb8145ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:03.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.117+0000 7f8bb659c700 1 -- 172.21.15.78:0/208124982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bb815c930 msgr2=0x7f8bb815cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:03.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.117+0000 7f8bb659c700 1 --2- 172.21.15.78:0/208124982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bb815c930 0x7f8bb815cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:03.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.117+0000 7f8bb659c700 1 -- 172.21.15.78:0/208124982 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb8158290 con 0x7f8bb80cdb80 2023-10-11T09:48:03.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.117+0000 7f8bb659c700 1 --2- 172.21.15.78:0/208124982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8bb80cdb80 0x7f8bb80cdfd0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0072c0 tx=0x7f8bac007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:03.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.117+0000 7f8b9f7fe700 1 -- 172.21.15.78:0/208124982 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac0077a0 con 0x7f8bb80cdb80 2023-10-11T09:48:03.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.117+0000 7f8b9f7fe700 1 -- 172.21.15.78:0/208124982 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bac00f870 con 0x7f8bb80cdb80 2023-10-11T09:48:03.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.117+0000 7f8b9f7fe700 1 -- 172.21.15.78:0/208124982 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac00fc20 con 0x7f8bb80cdb80 2023-10-11T09:48:03.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.117+0000 7f8bbc92b700 1 -- 172.21.15.78:0/208124982 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bb8158570 con 0x7f8bb80cdb80 2023-10-11T09:48:03.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.117+0000 7f8bbc92b700 1 -- 172.21.15.78:0/208124982 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bb815d2e0 con 0x7f8bb80cdb80 2023-10-11T09:48:03.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.121+0000 7f8bbc92b700 1 -- 172.21.15.78:0/208124982 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bb8065d40 con 0x7f8bb80cdb80 2023-10-11T09:48:03.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.121+0000 7f8b9f7fe700 1 -- 172.21.15.78:0/208124982 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f8bac00fd80 con 0x7f8bb80cdb80 2023-10-11T09:48:03.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.121+0000 7f8b9f7fe700 1 --2- 172.21.15.78:0/208124982 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8ba4065f00 0x7f8ba40683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:03.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.121+0000 7f8b9f7fe700 1 -- 172.21.15.78:0/208124982 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f8bac083890 con 0x7f8bb80cdb80 2023-10-11T09:48:03.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.121+0000 7f8bb6d9d700 1 --2- 172.21.15.78:0/208124982 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8ba4065f00 0x7f8ba40683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:03.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.125+0000 7f8bb6d9d700 1 --2- 172.21.15.78:0/208124982 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8ba4065f00 0x7f8ba40683b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8000f80 tx=0x7f8ba8013000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:03.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.129+0000 7f8b9f7fe700 1 -- 172.21.15.78:0/208124982 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f8bac050c40 con 0x7f8bb80cdb80 2023-10-11T09:48:03.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.301+0000 7f8bbc92b700 1 -- 172.21.15.78:0/208124982 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8bb806b570 con 0x7f8bb80cdb80 2023-10-11T09:48:03.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.301+0000 7f8b9f7fe700 1 -- 172.21.15.78:0/208124982 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8bac054260 con 0x7f8bb80cdb80 2023-10-11T09:48:03.308 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:48:03.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.305+0000 7f8bbc92b700 1 -- 172.21.15.78:0/208124982 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8ba4065f00 msgr2=0x7f8ba40683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:03.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.305+0000 7f8bbc92b700 1 --2- 172.21.15.78:0/208124982 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8ba4065f00 0x7f8ba40683b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8000f80 tx=0x7f8ba8013000 comp rx=0 tx=0).stop 2023-10-11T09:48:03.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.305+0000 7f8bbc92b700 1 -- 172.21.15.78:0/208124982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8bb80cdb80 msgr2=0x7f8bb80cdfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:03.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.305+0000 7f8bbc92b700 1 --2- 172.21.15.78:0/208124982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8bb80cdb80 0x7f8bb80cdfd0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0072c0 tx=0x7f8bac007680 comp rx=0 tx=0).stop 2023-10-11T09:48:03.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.305+0000 7f8bbc92b700 1 -- 172.21.15.78:0/208124982 shutdown_connections 2023-10-11T09:48:03.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.305+0000 7f8bbc92b700 1 --2- 172.21.15.78:0/208124982 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8ba4065f00 0x7f8ba40683b0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:03.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.305+0000 7f8bbc92b700 1 --2- 172.21.15.78:0/208124982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bb815c930 0x7f8bb815cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:03.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.305+0000 7f8bbc92b700 1 --2- 172.21.15.78:0/208124982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8bb80cdb80 0x7f8bb80cdfd0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:03.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.305+0000 7f8bbc92b700 1 --2- 172.21.15.78:0/208124982 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8ba0003a20 0x7f8bb8145ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:03.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.305+0000 7f8bbc92b700 1 -- 172.21.15.78:0/208124982 >> 172.21.15.78:0/208124982 conn(0x7f8bb806a060 msgr2=0x7f8bb80cbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:03.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.305+0000 7f8bbc92b700 1 -- 172.21.15.78:0/208124982 shutdown_connections 2023-10-11T09:48:03.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.305+0000 7f8bbc92b700 1 -- 172.21.15.78:0/208124982 wait complete. 2023-10-11T09:48:03.335 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:48:03.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.505+0000 7f240d2ee700 1 Processor -- start 2023-10-11T09:48:03.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.505+0000 7f240d2ee700 1 -- start start 2023-10-11T09:48:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.509+0000 7f240d2ee700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f24080c69c0 0x7f24080cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.509+0000 7f240d2ee700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24080ced20 con 0x7f240806a060 2023-10-11T09:48:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.509+0000 7f240d2ee700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f240806e210 con 0x7f240806dd80 2023-10-11T09:48:03.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.509+0000 7f240d2ee700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24080ceb90 con 0x7f24080c69c0 2023-10-11T09:48:03.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.509+0000 7f2406ffd700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f240806a060 0x7f240806a430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:41652/0 (socket says 172.21.15.78:41652) 2023-10-11T09:48:03.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.509+0000 7f2406ffd700 1 -- 172.21.15.78:0/2178880264 learned_addr learned my addr 172.21.15.78:0/2178880264 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:03.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.509+0000 7f24077fe700 1 --2- 172.21.15.78:0/2178880264 >> v2:172.21.15.78:3300/0 conn(0x7f24080c69c0 0x7f24080cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:03.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.509+0000 7f24077fe700 1 -- 172.21.15.78:0/2178880264 >> v1:172.21.15.133:6789/0 conn(0x7f240806dd80 legacy=0x7f2408160680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:03.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.509+0000 7f2405ffb700 1 -- 172.21.15.78:0/2178880264 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 284218318 0 0) 0x7f240806e210 con 0x7f240806dd80 2023-10-11T09:48:03.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.509+0000 7f24077fe700 1 -- 172.21.15.78:0/2178880264 >> v1:172.21.15.78:6789/0 conn(0x7f240806a060 legacy=0x7f240806a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:03.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.509+0000 7f24077fe700 1 -- 172.21.15.78:0/2178880264 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f240806e4f0 con 0x7f24080c69c0 2023-10-11T09:48:03.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.509+0000 7f24077fe700 1 --2- 172.21.15.78:0/2178880264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24080c69c0 0x7f24080cf1b0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f2400009d90 tx=0x7f240000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da910018e0fa42e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:03.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.509+0000 7f2405ffb700 1 -- 172.21.15.78:0/2178880264 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240000e070 con 0x7f24080c69c0 2023-10-11T09:48:03.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.509+0000 7f2405ffb700 1 -- 172.21.15.78:0/2178880264 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f240000bad0 con 0x7f24080c69c0 2023-10-11T09:48:03.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.509+0000 7f2405ffb700 1 -- 172.21.15.78:0/2178880264 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240000be80 con 0x7f24080c69c0 2023-10-11T09:48:03.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.509+0000 7f240d2ee700 1 -- 172.21.15.78:0/2178880264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24080c69c0 msgr2=0x7f24080cf1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:03.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.509+0000 7f240d2ee700 1 --2- 172.21.15.78:0/2178880264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24080c69c0 0x7f24080cf1b0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f2400009d90 tx=0x7f240000b330 comp rx=0 tx=0).stop 2023-10-11T09:48:03.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.509+0000 7f240d2ee700 1 -- 172.21.15.78:0/2178880264 shutdown_connections 2023-10-11T09:48:03.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.509+0000 7f240d2ee700 1 --2- 172.21.15.78:0/2178880264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24080c69c0 0x7f24080cf1b0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:03.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.509+0000 7f240d2ee700 1 -- 172.21.15.78:0/2178880264 >> 172.21.15.78:0/2178880264 conn(0x7f24080b24c0 msgr2=0x7f24080cc7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:03.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.509+0000 7f240d2ee700 1 -- 172.21.15.78:0/2178880264 shutdown_connections 2023-10-11T09:48:03.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.513+0000 7f240d2ee700 1 -- 172.21.15.78:0/2178880264 wait complete. 2023-10-11T09:48:03.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.513+0000 7f240d2ee700 1 Processor -- start 2023-10-11T09:48:03.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.513+0000 7f240d2ee700 1 -- start start 2023-10-11T09:48:03.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.517+0000 7f240d2ee700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f240806a060 0x7f24080c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:03.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.517+0000 7f240d2ee700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f240806dd80 0x7f240815b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:03.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.517+0000 7f240d2ee700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f24080c69c0 0x7f240815fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:03.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.517+0000 7f240d2ee700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24080c7350 con 0x7f240806a060 2023-10-11T09:48:03.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.517+0000 7f240d2ee700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240815ba60 con 0x7f240806dd80 2023-10-11T09:48:03.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.517+0000 7f240d2ee700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24081f3270 con 0x7f24080c69c0 2023-10-11T09:48:03.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.517+0000 7f2406ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f240806a060 0x7f24080c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:03.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.517+0000 7f2406ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f240806a060 0x7f24080c6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48580/0 (socket says 172.21.15.78:48580) 2023-10-11T09:48:03.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.517+0000 7f2406ffd700 1 -- 172.21.15.78:0/2943098700 learned_addr learned my addr 172.21.15.78:0/2943098700 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:03.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.517+0000 7f24067fc700 1 --2- 172.21.15.78:0/2943098700 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f240806dd80 0x7f240815b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:03.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.517+0000 7f24077fe700 1 --2- 172.21.15.78:0/2943098700 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f24080c69c0 0x7f240815fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:03.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.517+0000 7f2406ffd700 1 -- 172.21.15.78:0/2943098700 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f24080c69c0 msgr2=0x7f240815fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:03.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.517+0000 7f2406ffd700 1 --2- 172.21.15.78:0/2943098700 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f24080c69c0 0x7f240815fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:03.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.517+0000 7f2406ffd700 1 -- 172.21.15.78:0/2943098700 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f240806dd80 msgr2=0x7f240815b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:03.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.517+0000 7f2406ffd700 1 --2- 172.21.15.78:0/2943098700 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f240806dd80 0x7f240815b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:03.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.517+0000 7f2406ffd700 1 -- 172.21.15.78:0/2943098700 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2400009a70 con 0x7f240806a060 2023-10-11T09:48:03.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.517+0000 7f24067fc700 1 --2- 172.21.15.78:0/2943098700 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f240806dd80 0x7f240815b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:48:03.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.517+0000 7f2406ffd700 1 --2- 172.21.15.78:0/2943098700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f240806a060 0x7f24080c6e10 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f23f800c780 tx=0x7f23f800ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:03.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.517+0000 7f23ef7fe700 1 -- 172.21.15.78:0/2943098700 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f8004ca0 con 0x7f240806a060 2023-10-11T09:48:03.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.517+0000 7f23ef7fe700 1 -- 172.21.15.78:0/2943098700 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f8004e00 con 0x7f240806a060 2023-10-11T09:48:03.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.517+0000 7f240d2ee700 1 -- 172.21.15.78:0/2943098700 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24081f3550 con 0x7f240806a060 2023-10-11T09:48:03.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.517+0000 7f23ef7fe700 1 -- 172.21.15.78:0/2943098700 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f80072f0 con 0x7f240806a060 2023-10-11T09:48:03.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.517+0000 7f240d2ee700 1 -- 172.21.15.78:0/2943098700 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24081f3b10 con 0x7f240806a060 2023-10-11T09:48:03.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.521+0000 7f240d2ee700 1 -- 172.21.15.78:0/2943098700 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f240806fc50 con 0x7f240806a060 2023-10-11T09:48:03.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.521+0000 7f23ef7fe700 1 -- 172.21.15.78:0/2943098700 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f23f8007450 con 0x7f240806a060 2023-10-11T09:48:03.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.521+0000 7f23ef7fe700 1 --2- 172.21.15.78:0/2943098700 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f23f4065f00 0x7f23f40683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:03.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.521+0000 7f23ef7fe700 1 -- 172.21.15.78:0/2943098700 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f23f8083790 con 0x7f240806a060 2023-10-11T09:48:03.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.521+0000 7f24067fc700 1 --2- 172.21.15.78:0/2943098700 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f23f4065f00 0x7f23f40683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:03.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.525+0000 7f24067fc700 1 --2- 172.21.15.78:0/2943098700 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f23f4065f00 0x7f23f40683b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2408071260 tx=0x7f23fc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:03.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.529+0000 7f23ef7fe700 1 -- 172.21.15.78:0/2943098700 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f23f8050b40 con 0x7f240806a060 2023-10-11T09:48:03.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.697+0000 7f240d2ee700 1 -- 172.21.15.78:0/2943098700 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2408065db0 con 0x7f240806a060 2023-10-11T09:48:03.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.697+0000 7f23ef7fe700 1 -- 172.21.15.78:0/2943098700 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v70) v1 ==== 92+0+10 (secure 0 0 0) 0x7f23f8054160 con 0x7f240806a060 2023-10-11T09:48:03.705 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:48:03.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.701+0000 7f240d2ee700 1 -- 172.21.15.78:0/2943098700 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f23f4065f00 msgr2=0x7f23f40683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:03.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.701+0000 7f240d2ee700 1 --2- 172.21.15.78:0/2943098700 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f23f4065f00 0x7f23f40683b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2408071260 tx=0x7f23fc009040 comp rx=0 tx=0).stop 2023-10-11T09:48:03.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.701+0000 7f240d2ee700 1 -- 172.21.15.78:0/2943098700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f240806a060 msgr2=0x7f24080c6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:03.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.701+0000 7f240d2ee700 1 --2- 172.21.15.78:0/2943098700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f240806a060 0x7f24080c6e10 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f23f800c780 tx=0x7f23f800ca90 comp rx=0 tx=0).stop 2023-10-11T09:48:03.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.701+0000 7f240d2ee700 1 -- 172.21.15.78:0/2943098700 shutdown_connections 2023-10-11T09:48:03.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.701+0000 7f240d2ee700 1 --2- 172.21.15.78:0/2943098700 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f23f4065f00 0x7f23f40683b0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:03.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.701+0000 7f240d2ee700 1 --2- 172.21.15.78:0/2943098700 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f24080c69c0 0x7f240815fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:03.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.701+0000 7f240d2ee700 1 --2- 172.21.15.78:0/2943098700 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f240806dd80 0x7f240815b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:03.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.701+0000 7f240d2ee700 1 --2- 172.21.15.78:0/2943098700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f240806a060 0x7f24080c6e10 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:03.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.701+0000 7f240d2ee700 1 -- 172.21.15.78:0/2943098700 >> 172.21.15.78:0/2943098700 conn(0x7f24080b24c0 msgr2=0x7f240815c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:03.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.701+0000 7f240d2ee700 1 -- 172.21.15.78:0/2943098700 shutdown_connections 2023-10-11T09:48:03.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.701+0000 7f240d2ee700 1 -- 172.21.15.78:0/2943098700 wait complete. 2023-10-11T09:48:03.729 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T09:48:03.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.893+0000 7f00ab59e700 1 Processor -- start 2023-10-11T09:48:03.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.893+0000 7f00ab59e700 1 -- start start 2023-10-11T09:48:03.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.897+0000 7f00ab59e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f00ac0b3240 0x7f00ac0d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:03.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.897+0000 7f00ab59e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f00ac0b3a90 0x7f00ac0b85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:03.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.897+0000 7f00ab59e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00ac0b47f0 con 0x7f00ac0b3610 2023-10-11T09:48:03.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.897+0000 7f00ab59e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00ac0b8130 con 0x7f00ac0b3a90 2023-10-11T09:48:03.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.897+0000 7f00ab59e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00ac0b41e0 con 0x7f00ac0b3240 2023-10-11T09:48:03.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.897+0000 7f00aad9d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f00ac0b3a90 0x7f00ac0b85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:03.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.897+0000 7f00aad9d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f00ac0b3a90 0x7f00ac0b85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48596/0 (socket says 172.21.15.78:48596) 2023-10-11T09:48:03.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.897+0000 7f00aad9d700 1 -- 172.21.15.78:0/2498022041 learned_addr learned my addr 172.21.15.78:0/2498022041 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:03.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.897+0000 7f00aa59c700 1 --2- 172.21.15.78:0/2498022041 >> v2:172.21.15.133:3300/0 conn(0x7f00ac0b3240 0x7f00ac0d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:03.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.897+0000 7f00aad9d700 1 -- 172.21.15.78:0/2498022041 >> v2:172.21.15.133:3300/0 conn(0x7f00ac0b3240 msgr2=0x7f00ac0d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:03.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.897+0000 7f00aad9d700 1 --2- 172.21.15.78:0/2498022041 >> v2:172.21.15.133:3300/0 conn(0x7f00ac0b3240 0x7f00ac0d29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:03.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.897+0000 7f00aad9d700 1 -- 172.21.15.78:0/2498022041 >> v1:172.21.15.133:6789/0 conn(0x7f00ac0b3610 legacy=0x7f00ac15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:03.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.897+0000 7f00aad9d700 1 -- 172.21.15.78:0/2498022041 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ac0b7800 con 0x7f00ac0b3a90 2023-10-11T09:48:03.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.897+0000 7f00aad9d700 1 --2- 172.21.15.78:0/2498022041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00ac0b3a90 0x7f00ac0b85c0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f00ac06aa40 tx=0x7f00a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d286153f77b7406 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:03.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.901+0000 7f00a959a700 1 -- 172.21.15.78:0/2498022041 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a400e070 con 0x7f00ac0b3a90 2023-10-11T09:48:03.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.901+0000 7f00a959a700 1 -- 172.21.15.78:0/2498022041 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a4003a40 con 0x7f00ac0b3a90 2023-10-11T09:48:03.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.901+0000 7f00a959a700 1 -- 172.21.15.78:0/2498022041 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a4003df0 con 0x7f00ac0b3a90 2023-10-11T09:48:03.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.901+0000 7f00ab59e700 1 -- 172.21.15.78:0/2498022041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00ac0b3a90 msgr2=0x7f00ac0b85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:03.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.901+0000 7f00ab59e700 1 --2- 172.21.15.78:0/2498022041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00ac0b3a90 0x7f00ac0b85c0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f00ac06aa40 tx=0x7f00a400b330 comp rx=0 tx=0).stop 2023-10-11T09:48:03.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.901+0000 7f00ab59e700 1 -- 172.21.15.78:0/2498022041 shutdown_connections 2023-10-11T09:48:03.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.901+0000 7f00ab59e700 1 --2- 172.21.15.78:0/2498022041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00ac0b3a90 0x7f00ac0b85c0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:03.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.901+0000 7f00ab59e700 1 --2- 172.21.15.78:0/2498022041 >> v2:172.21.15.133:3300/0 conn(0x7f00ac0b3240 0x7f00ac0d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:03.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.901+0000 7f00ab59e700 1 -- 172.21.15.78:0/2498022041 >> 172.21.15.78:0/2498022041 conn(0x7f00ac09c260 msgr2=0x7f00ac0d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:03.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.901+0000 7f00ab59e700 1 -- 172.21.15.78:0/2498022041 shutdown_connections 2023-10-11T09:48:03.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.901+0000 7f00ab59e700 1 -- 172.21.15.78:0/2498022041 wait complete. 2023-10-11T09:48:03.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.905+0000 7f00ab59e700 1 Processor -- start 2023-10-11T09:48:03.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.905+0000 7f00ab59e700 1 -- start start 2023-10-11T09:48:03.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.905+0000 7f00ab59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00ac0b3240 0x7f00ac153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:03.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.905+0000 7f00ab59e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00ac0b3610 0x7f00ac15f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:03.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.905+0000 7f00ab59e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00ac0b3a90 0x7f00ac1522f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:03.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.905+0000 7f00ab59e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac15ee20 con 0x7f00ac0b3240 2023-10-11T09:48:03.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.905+0000 7f00ab59e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac154010 con 0x7f00ac0b3a90 2023-10-11T09:48:03.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.905+0000 7f00ab59e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac152e30 con 0x7f00ac0b3610 2023-10-11T09:48:03.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.905+0000 7f00aa59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00ac0b3240 0x7f00ac153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:03.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.905+0000 7f00aa59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00ac0b3240 0x7f00ac153ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48612/0 (socket says 172.21.15.78:48612) 2023-10-11T09:48:03.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.905+0000 7f00aa59c700 1 -- 172.21.15.78:0/1350528543 learned_addr learned my addr 172.21.15.78:0/1350528543 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:03.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.905+0000 7f00aa59c700 1 -- 172.21.15.78:0/1350528543 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00ac0b3610 msgr2=0x7f00ac15f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:03.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.905+0000 7f00aa59c700 1 --2- 172.21.15.78:0/1350528543 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00ac0b3610 0x7f00ac15f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:03.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.905+0000 7f00aa59c700 1 -- 172.21.15.78:0/1350528543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00ac0b3a90 msgr2=0x7f00ac1522f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:03.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.905+0000 7f00aad9d700 1 --2- 172.21.15.78:0/1350528543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00ac0b3a90 0x7f00ac1522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:03.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.905+0000 7f00aa59c700 1 --2- 172.21.15.78:0/1350528543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00ac0b3a90 0x7f00ac1522f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:03.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.905+0000 7f00aa59c700 1 -- 172.21.15.78:0/1350528543 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a4009b30 con 0x7f00ac0b3240 2023-10-11T09:48:03.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.905+0000 7f00aa59c700 1 --2- 172.21.15.78:0/1350528543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00ac0b3240 0x7f00ac153ad0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f009c00d800 tx=0x7f009c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:03.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.905+0000 7f009affd700 1 -- 172.21.15.78:0/1350528543 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009c0125f0 con 0x7f00ac0b3240 2023-10-11T09:48:03.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.905+0000 7f009affd700 1 -- 172.21.15.78:0/1350528543 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f009c012750 con 0x7f00ac0b3240 2023-10-11T09:48:03.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.905+0000 7f009affd700 1 -- 172.21.15.78:0/1350528543 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009c012ac0 con 0x7f00ac0b3240 2023-10-11T09:48:03.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.905+0000 7f00ab59e700 1 -- 172.21.15.78:0/1350528543 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00ac15e2a0 con 0x7f00ac0b3240 2023-10-11T09:48:03.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.905+0000 7f00ab59e700 1 -- 172.21.15.78:0/1350528543 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00ac15e860 con 0x7f00ac0b3240 2023-10-11T09:48:03.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.909+0000 7f00ab59e700 1 -- 172.21.15.78:0/1350528543 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00ac0d1d10 con 0x7f00ac0b3240 2023-10-11T09:48:03.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.909+0000 7f009affd700 1 -- 172.21.15.78:0/1350528543 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f009c012c20 con 0x7f00ac0b3240 2023-10-11T09:48:03.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.913+0000 7f009affd700 1 --2- 172.21.15.78:0/1350528543 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f008c065ed0 0x7f008c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:03.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.913+0000 7f009affd700 1 -- 172.21.15.78:0/1350528543 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f009c083e10 con 0x7f00ac0b3240 2023-10-11T09:48:03.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.913+0000 7f00a9d9b700 1 --2- 172.21.15.78:0/1350528543 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f008c065ed0 0x7f008c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:03.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.913+0000 7f00a9d9b700 1 --2- 172.21.15.78:0/1350528543 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f008c065ed0 0x7f008c068380 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f00ac1f54c0 tx=0x7f00a0006c70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:03.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:03.917+0000 7f009affd700 1 -- 172.21.15.78:0/1350528543 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f009c087030 con 0x7f00ac0b3240 2023-10-11T09:48:04.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.089+0000 7f00ab59e700 1 -- 172.21.15.78:0/1350528543 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f00ac0b7d90 con 0x7f00ac0b3240 2023-10-11T09:48:04.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.089+0000 7f009affd700 1 -- 172.21.15.78:0/1350528543 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) v1 ==== 104+0+11 (secure 0 0 0) 0x7f009c054c40 con 0x7f00ac0b3240 2023-10-11T09:48:04.097 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:48:04.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.093+0000 7f00ab59e700 1 -- 172.21.15.78:0/1350528543 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f008c065ed0 msgr2=0x7f008c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:04.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.093+0000 7f00ab59e700 1 --2- 172.21.15.78:0/1350528543 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f008c065ed0 0x7f008c068380 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f00ac1f54c0 tx=0x7f00a0006c70 comp rx=0 tx=0).stop 2023-10-11T09:48:04.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.093+0000 7f00ab59e700 1 -- 172.21.15.78:0/1350528543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00ac0b3240 msgr2=0x7f00ac153ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:04.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.093+0000 7f00ab59e700 1 --2- 172.21.15.78:0/1350528543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00ac0b3240 0x7f00ac153ad0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f009c00d800 tx=0x7f009c00dbc0 comp rx=0 tx=0).stop 2023-10-11T09:48:04.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.093+0000 7f00ab59e700 1 -- 172.21.15.78:0/1350528543 shutdown_connections 2023-10-11T09:48:04.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.093+0000 7f00ab59e700 1 --2- 172.21.15.78:0/1350528543 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f008c065ed0 0x7f008c068380 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:04.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.093+0000 7f00ab59e700 1 --2- 172.21.15.78:0/1350528543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00ac0b3a90 0x7f00ac1522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:04.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.093+0000 7f00ab59e700 1 --2- 172.21.15.78:0/1350528543 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00ac0b3610 0x7f00ac15f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:04.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.093+0000 7f00ab59e700 1 --2- 172.21.15.78:0/1350528543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00ac0b3240 0x7f00ac153ad0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:04.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.093+0000 7f00ab59e700 1 -- 172.21.15.78:0/1350528543 >> 172.21.15.78:0/1350528543 conn(0x7f00ac09c260 msgr2=0x7f00ac0afb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:04.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.093+0000 7f00ab59e700 1 -- 172.21.15.78:0/1350528543 shutdown_connections 2023-10-11T09:48:04.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.093+0000 7f00ab59e700 1 -- 172.21.15.78:0/1350528543 wait complete. 2023-10-11T09:48:04.123 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T09:48:04.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.293+0000 7fc4839df700 1 Processor -- start 2023-10-11T09:48:04.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.293+0000 7fc4839df700 1 -- start start 2023-10-11T09:48:04.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.293+0000 7fc4839df700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc47c0b84c0 0x7fc47c0cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:04.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.293+0000 7fc4839df700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc47c160a80 0x7fc47c0d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:04.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.293+0000 7fc4839df700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc47c0cd690 con 0x7fc47c0b8890 2023-10-11T09:48:04.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.293+0000 7fc4839df700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc47c0cdea0 con 0x7fc47c160a80 2023-10-11T09:48:04.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.293+0000 7fc4839df700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c0d0c10 con 0x7fc47c0b84c0 2023-10-11T09:48:04.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.297+0000 7fc481f7c700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc47c160a80 0x7fc47c0d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:04.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.297+0000 7fc481f7c700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc47c160a80 0x7fc47c0d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:48326/0 (socket says 172.21.15.78:48326) 2023-10-11T09:48:04.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.297+0000 7fc481f7c700 1 -- 172.21.15.78:0/3892679402 learned_addr learned my addr 172.21.15.78:0/3892679402 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:04.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.297+0000 7fc48177b700 1 --2- 172.21.15.78:0/3892679402 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc47c0b84c0 0x7fc47c0cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:04.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.297+0000 7fc481f7c700 1 -- 172.21.15.78:0/3892679402 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc47c0b84c0 msgr2=0x7fc47c0cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:04.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.297+0000 7fc481f7c700 1 --2- 172.21.15.78:0/3892679402 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc47c0b84c0 0x7fc47c0cfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:04.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.297+0000 7fc481f7c700 1 -- 172.21.15.78:0/3892679402 >> v1:172.21.15.133:6789/0 conn(0x7fc47c0b8890 legacy=0x7fc47c160370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:04.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.297+0000 7fc481f7c700 1 -- 172.21.15.78:0/3892679402 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc47c0d02e0 con 0x7fc47c160a80 2023-10-11T09:48:04.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.297+0000 7fc481f7c700 1 --2- 172.21.15.78:0/3892679402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc47c160a80 0x7fc47c0d1340 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc474009e50 tx=0x7fc47400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0918a77e3efdeef server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:04.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.297+0000 7fc473fff700 1 -- 172.21.15.78:0/3892679402 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47400e070 con 0x7fc47c160a80 2023-10-11T09:48:04.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.297+0000 7fc473fff700 1 -- 172.21.15.78:0/3892679402 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc47400b910 con 0x7fc47c160a80 2023-10-11T09:48:04.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.297+0000 7fc473fff700 1 -- 172.21.15.78:0/3892679402 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47400bcc0 con 0x7fc47c160a80 2023-10-11T09:48:04.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.301+0000 7fc4839df700 1 -- 172.21.15.78:0/3892679402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc47c160a80 msgr2=0x7fc47c0d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:04.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.301+0000 7fc4839df700 1 --2- 172.21.15.78:0/3892679402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc47c160a80 0x7fc47c0d1340 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc474009e50 tx=0x7fc47400b330 comp rx=0 tx=0).stop 2023-10-11T09:48:04.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.301+0000 7fc4839df700 1 -- 172.21.15.78:0/3892679402 shutdown_connections 2023-10-11T09:48:04.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.301+0000 7fc4839df700 1 --2- 172.21.15.78:0/3892679402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc47c160a80 0x7fc47c0d1340 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:04.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.301+0000 7fc4839df700 1 --2- 172.21.15.78:0/3892679402 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc47c0b84c0 0x7fc47c0cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:04.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.301+0000 7fc4839df700 1 -- 172.21.15.78:0/3892679402 >> 172.21.15.78:0/3892679402 conn(0x7fc47c06cb50 msgr2=0x7fc47c06cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:04.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.301+0000 7fc4839df700 1 -- 172.21.15.78:0/3892679402 shutdown_connections 2023-10-11T09:48:04.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.301+0000 7fc4839df700 1 -- 172.21.15.78:0/3892679402 wait complete. 2023-10-11T09:48:04.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.305+0000 7fc4839df700 1 Processor -- start 2023-10-11T09:48:04.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.305+0000 7fc4839df700 1 -- start start 2023-10-11T09:48:04.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.305+0000 7fc4839df700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc47c0b84c0 0x7fc47c0c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:04.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.305+0000 7fc4839df700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc47c0b8890 0x7fc47c1f1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:04.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.305+0000 7fc4839df700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc47c160a80 0x7fc47c1f24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:04.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.305+0000 7fc4839df700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c0c9b30 con 0x7fc47c160a80 2023-10-11T09:48:04.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.305+0000 7fc4839df700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c0c9e10 con 0x7fc47c0b8890 2023-10-11T09:48:04.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.305+0000 7fc4839df700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c0cb160 con 0x7fc47c0b84c0 2023-10-11T09:48:04.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.305+0000 7fc481f7c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc47c160a80 0x7fc47c1f24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:04.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.305+0000 7fc481f7c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc47c160a80 0x7fc47c1f24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48616/0 (socket says 172.21.15.78:48616) 2023-10-11T09:48:04.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.305+0000 7fc481f7c700 1 -- 172.21.15.78:0/1592548238 learned_addr learned my addr 172.21.15.78:0/1592548238 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:04.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.305+0000 7fc48177b700 1 --2- 172.21.15.78:0/1592548238 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc47c0b84c0 0x7fc47c0c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:04.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.305+0000 7fc480f7a700 1 --2- 172.21.15.78:0/1592548238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc47c0b8890 0x7fc47c1f1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:04.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.305+0000 7fc481f7c700 1 -- 172.21.15.78:0/1592548238 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc47c0b84c0 msgr2=0x7fc47c0c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:04.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.305+0000 7fc481f7c700 1 --2- 172.21.15.78:0/1592548238 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc47c0b84c0 0x7fc47c0c91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:04.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.305+0000 7fc481f7c700 1 -- 172.21.15.78:0/1592548238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc47c0b8890 msgr2=0x7fc47c1f1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:04.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.305+0000 7fc481f7c700 1 --2- 172.21.15.78:0/1592548238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc47c0b8890 0x7fc47c1f1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:04.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.305+0000 7fc481f7c700 1 -- 172.21.15.78:0/1592548238 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc474009b30 con 0x7fc47c160a80 2023-10-11T09:48:04.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.309+0000 7fc481f7c700 1 --2- 172.21.15.78:0/1592548238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc47c160a80 0x7fc47c1f24f0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fc47c067640 tx=0x7fc474004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:04.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.309+0000 7fc471ffb700 1 -- 172.21.15.78:0/1592548238 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc474004800 con 0x7fc47c160a80 2023-10-11T09:48:04.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.309+0000 7fc471ffb700 1 -- 172.21.15.78:0/1592548238 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4740092e0 con 0x7fc47c160a80 2023-10-11T09:48:04.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.309+0000 7fc4839df700 1 -- 172.21.15.78:0/1592548238 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc47c0cb440 con 0x7fc47c160a80 2023-10-11T09:48:04.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.309+0000 7fc471ffb700 1 -- 172.21.15.78:0/1592548238 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47401c630 con 0x7fc47c160a80 2023-10-11T09:48:04.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.309+0000 7fc4839df700 1 -- 172.21.15.78:0/1592548238 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc47c15f120 con 0x7fc47c160a80 2023-10-11T09:48:04.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.309+0000 7fc4839df700 1 -- 172.21.15.78:0/1592548238 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc47c0cb960 con 0x7fc47c160a80 2023-10-11T09:48:04.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.313+0000 7fc471ffb700 1 -- 172.21.15.78:0/1592548238 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fc47401c790 con 0x7fc47c160a80 2023-10-11T09:48:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.313+0000 7fc471ffb700 1 --2- 172.21.15.78:0/1592548238 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc460065f20 0x7fc4600683d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.313+0000 7fc471ffb700 1 -- 172.21.15.78:0/1592548238 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6724+0+0 (secure 0 0 0) 0x7fc474085310 con 0x7fc47c160a80 2023-10-11T09:48:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.313+0000 7fc48177b700 1 --2- 172.21.15.78:0/1592548238 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc460065f20 0x7fc4600683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.313+0000 7fc48177b700 1 --2- 172.21.15.78:0/1592548238 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc460065f20 0x7fc4600683d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc478009b50 tx=0x7fc478009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:04.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.317+0000 7fc471ffb700 1 -- 172.21.15.78:0/1592548238 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fc474052730 con 0x7fc47c160a80 2023-10-11T09:48:04.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.493+0000 7fc4839df700 1 -- 172.21.15.78:0/1592548238 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc47c08bff0 con 0x7fc47c160a80 2023-10-11T09:48:04.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.493+0000 7fc471ffb700 1 -- 172.21.15.78:0/1592548238 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v70) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc474055d50 con 0x7fc47c160a80 2023-10-11T09:48:04.502 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2023-10-11T09:48:04.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.501+0000 7fc4839df700 1 -- 172.21.15.78:0/1592548238 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc460065f20 msgr2=0x7fc4600683d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:04.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.501+0000 7fc4839df700 1 --2- 172.21.15.78:0/1592548238 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc460065f20 0x7fc4600683d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc478009b50 tx=0x7fc478009240 comp rx=0 tx=0).stop 2023-10-11T09:48:04.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.501+0000 7fc4839df700 1 -- 172.21.15.78:0/1592548238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc47c160a80 msgr2=0x7fc47c1f24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:04.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.501+0000 7fc4839df700 1 --2- 172.21.15.78:0/1592548238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc47c160a80 0x7fc47c1f24f0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fc47c067640 tx=0x7fc474004af0 comp rx=0 tx=0).stop 2023-10-11T09:48:04.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.501+0000 7fc4839df700 1 -- 172.21.15.78:0/1592548238 shutdown_connections 2023-10-11T09:48:04.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.501+0000 7fc4839df700 1 --2- 172.21.15.78:0/1592548238 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc460065f20 0x7fc4600683d0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:04.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.501+0000 7fc4839df700 1 --2- 172.21.15.78:0/1592548238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc47c160a80 0x7fc47c1f24f0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:04.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.501+0000 7fc4839df700 1 --2- 172.21.15.78:0/1592548238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc47c0b8890 0x7fc47c1f1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:04.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.501+0000 7fc4839df700 1 --2- 172.21.15.78:0/1592548238 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc47c0b84c0 0x7fc47c0c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:04.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.501+0000 7fc4839df700 1 -- 172.21.15.78:0/1592548238 >> 172.21.15.78:0/1592548238 conn(0x7fc47c06cb50 msgr2=0x7fc47c06b140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:04.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.501+0000 7fc4839df700 1 -- 172.21.15.78:0/1592548238 shutdown_connections 2023-10-11T09:48:04.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.501+0000 7fc4839df700 1 -- 172.21.15.78:0/1592548238 wait complete. 2023-10-11T09:48:04.526 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-10-11T09:48:04.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.689+0000 7fd0e2ebf700 1 Processor -- start 2023-10-11T09:48:04.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.689+0000 7fd0e2ebf700 1 -- start start 2023-10-11T09:48:04.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.693+0000 7fd0e2ebf700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd0dc0b8890 0x7fd0dc0d2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:04.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.693+0000 7fd0e2ebf700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0dc0cd690 con 0x7fd0dc0d3510 2023-10-11T09:48:04.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.693+0000 7fd0e2ebf700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0dc0cd8e0 con 0x7fd0dc0b84c0 2023-10-11T09:48:04.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.693+0000 7fd0e2ebf700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc0cdea0 con 0x7fd0dc0b8890 2023-10-11T09:48:04.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.693+0000 7fd0e145c700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fd0dc0d3510 0x7fd0dc0d1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:39920/0 (socket says 172.21.15.78:39920) 2023-10-11T09:48:04.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.693+0000 7fd0e145c700 1 -- 172.21.15.78:0/205343819 learned_addr learned my addr 172.21.15.78:0/205343819 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:04.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.693+0000 7fd0dbfff700 1 --2- 172.21.15.78:0/205343819 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd0dc0b8890 0x7fd0dc0d2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:04.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.693+0000 7fd0db7fe700 1 -- 172.21.15.78:0/205343819 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1061267651 0 0) 0x7fd0dc0cd690 con 0x7fd0dc0d3510 2023-10-11T09:48:04.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.693+0000 7fd0db7fe700 1 -- 172.21.15.78:0/205343819 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0cc0035d0 con 0x7fd0dc0d3510 2023-10-11T09:48:04.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.693+0000 7fd0db7fe700 1 -- 172.21.15.78:0/205343819 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2918564934 0 0) 0x7fd0dc0cd8e0 con 0x7fd0dc0b84c0 2023-10-11T09:48:04.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.693+0000 7fd0db7fe700 1 -- 172.21.15.78:0/205343819 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0dc0cd690 con 0x7fd0dc0b84c0 2023-10-11T09:48:04.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.693+0000 7fd0db7fe700 1 -- 172.21.15.78:0/205343819 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 334951304 0 0) 0x7fd0cc0035d0 con 0x7fd0dc0d3510 2023-10-11T09:48:04.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.693+0000 7fd0db7fe700 1 -- 172.21.15.78:0/205343819 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd0dc0b8890 msgr2=0x7fd0dc0d2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:04.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.693+0000 7fd0db7fe700 1 --2- 172.21.15.78:0/205343819 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd0dc0b8890 0x7fd0dc0d2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:04.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.693+0000 7fd0db7fe700 1 -- 172.21.15.78:0/205343819 >> v1:172.21.15.133:6789/0 conn(0x7fd0dc0b84c0 legacy=0x7fd0dc0cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:04.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.693+0000 7fd0db7fe700 1 -- 172.21.15.78:0/205343819 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0dc0cdbc0 con 0x7fd0dc0d3510 2023-10-11T09:48:04.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.693+0000 7fd0db7fe700 1 -- 172.21.15.78:0/205343819 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7fd0d40019f0 con 0x7fd0dc0d3510 2023-10-11T09:48:04.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.693+0000 7fd0db7fe700 1 -- 172.21.15.78:0/205343819 >> v1:172.21.15.78:6789/0 conn(0x7fd0dc0d3510 legacy=0x7fd0dc0d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:04.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.693+0000 7fd0db7fe700 1 --2- 172.21.15.78:0/205343819 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd0dc0b8890 0x7fd0cc003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:04.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.693+0000 7fd0db7fe700 1 --2- 172.21.15.78:0/205343819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0cc004190 0x7fd0cc004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:04.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.693+0000 7fd0db7fe700 1 --2- 172.21.15.78:0/205343819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0cc004db0 0x7fd0cc005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:04.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.693+0000 7fd0db7fe700 1 -- 172.21.15.78:0/205343819 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc0cdea0 con 0x7fd0cc004db0 2023-10-11T09:48:04.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.693+0000 7fd0db7fe700 1 -- 172.21.15.78:0/205343819 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc00bbc0 con 0x7fd0cc004190 2023-10-11T09:48:04.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.693+0000 7fd0db7fe700 1 -- 172.21.15.78:0/205343819 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc00bd80 con 0x7fd0dc0b8890 2023-10-11T09:48:04.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.697+0000 7fd0e0c5b700 1 --2- 172.21.15.78:0/205343819 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd0dc0b8890 0x7fd0cc003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:04.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.697+0000 7fd0dbfff700 1 --2- 172.21.15.78:0/205343819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0cc004190 0x7fd0cc004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:04.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.697+0000 7fd0e145c700 1 --2- 172.21.15.78:0/205343819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0cc004db0 0x7fd0cc005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:04.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.697+0000 7fd0e0c5b700 1 -- 172.21.15.78:0/205343819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0cc004190 msgr2=0x7fd0cc004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:04.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.697+0000 7fd0e0c5b700 1 --2- 172.21.15.78:0/205343819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0cc004190 0x7fd0cc004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:04.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.697+0000 7fd0e0c5b700 1 -- 172.21.15.78:0/205343819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0cc004db0 msgr2=0x7fd0cc005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:04.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.697+0000 7fd0e0c5b700 1 --2- 172.21.15.78:0/205343819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0cc004db0 0x7fd0cc005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:04.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.697+0000 7fd0e0c5b700 1 -- 172.21.15.78:0/205343819 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0cc0035d0 con 0x7fd0dc0b8890 2023-10-11T09:48:04.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.697+0000 7fd0e145c700 1 --2- 172.21.15.78:0/205343819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0cc004db0 0x7fd0cc005220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:48:04.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.697+0000 7fd0e0c5b700 1 --2- 172.21.15.78:0/205343819 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd0dc0b8890 0x7fd0cc003be0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0001960 tx=0x7fd0cc0058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cbcc5c547c64f41 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:04.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.697+0000 7fd0db7fe700 1 -- 172.21.15.78:0/205343819 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0001450 con 0x7fd0dc0b8890 2023-10-11T09:48:04.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.697+0000 7fd0db7fe700 1 -- 172.21.15.78:0/205343819 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d00052f0 con 0x7fd0dc0b8890 2023-10-11T09:48:04.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.697+0000 7fd0db7fe700 1 -- 172.21.15.78:0/205343819 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d00036a0 con 0x7fd0dc0b8890 2023-10-11T09:48:04.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.697+0000 7fd0e2ebf700 1 -- 172.21.15.78:0/205343819 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd0dc0b8890 msgr2=0x7fd0cc003be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:04.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.697+0000 7fd0e2ebf700 1 --2- 172.21.15.78:0/205343819 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd0dc0b8890 0x7fd0cc003be0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0001960 tx=0x7fd0cc0058a0 comp rx=0 tx=0).stop 2023-10-11T09:48:04.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.697+0000 7fd0e145c700 1 -- 172.21.15.78:0/205343819 reap_dead start 2023-10-11T09:48:04.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.697+0000 7fd0e2ebf700 1 -- 172.21.15.78:0/205343819 shutdown_connections 2023-10-11T09:48:04.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.697+0000 7fd0e2ebf700 1 -- 172.21.15.78:0/205343819 >> 172.21.15.78:0/205343819 conn(0x7fd0dc06cb50 msgr2=0x7fd0dc06cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:04.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.697+0000 7fd0e2ebf700 1 -- 172.21.15.78:0/205343819 shutdown_connections 2023-10-11T09:48:04.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.697+0000 7fd0e2ebf700 1 -- 172.21.15.78:0/205343819 wait complete. 2023-10-11T09:48:04.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.701+0000 7fd0e2ebf700 1 Processor -- start 2023-10-11T09:48:04.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.701+0000 7fd0e2ebf700 1 -- start start 2023-10-11T09:48:04.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.701+0000 7fd0e2ebf700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0dc0b8890 0x7fd0dc15e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:04.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.701+0000 7fd0e2ebf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0dc0c6ad0 0x7fd0dc1e9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:04.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.701+0000 7fd0e2ebf700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd0dc1483e0 0x7fd0dc148880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:04.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.701+0000 7fd0e2ebf700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc1493a0 con 0x7fd0dc0c6ad0 2023-10-11T09:48:04.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.701+0000 7fd0e2ebf700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc1e9e50 con 0x7fd0dc0b8890 2023-10-11T09:48:04.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.701+0000 7fd0e2ebf700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc1464f0 con 0x7fd0dc1483e0 2023-10-11T09:48:04.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.701+0000 7fd0dbfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0dc0c6ad0 0x7fd0dc1e9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:04.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.701+0000 7fd0dbfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0dc0c6ad0 0x7fd0dc1e9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40616/0 (socket says 172.21.15.78:40616) 2023-10-11T09:48:04.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.701+0000 7fd0dbfff700 1 -- 172.21.15.78:0/2110163932 learned_addr learned my addr 172.21.15.78:0/2110163932 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:04.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.701+0000 7fd0e0c5b700 1 --2- 172.21.15.78:0/2110163932 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0dc0b8890 0x7fd0dc15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:04.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.701+0000 7fd0e145c700 1 --2- 172.21.15.78:0/2110163932 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd0dc1483e0 0x7fd0dc148880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:04.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.701+0000 7fd0dbfff700 1 -- 172.21.15.78:0/2110163932 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd0dc1483e0 msgr2=0x7fd0dc148880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:04.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.701+0000 7fd0dbfff700 1 --2- 172.21.15.78:0/2110163932 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd0dc1483e0 0x7fd0dc148880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:04.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.701+0000 7fd0dbfff700 1 -- 172.21.15.78:0/2110163932 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0dc0b8890 msgr2=0x7fd0dc15e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:04.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.701+0000 7fd0dbfff700 1 --2- 172.21.15.78:0/2110163932 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0dc0b8890 0x7fd0dc15e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:04.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.701+0000 7fd0dbfff700 1 -- 172.21.15.78:0/2110163932 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0dc1467d0 con 0x7fd0dc0c6ad0 2023-10-11T09:48:04.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.705+0000 7fd0dbfff700 1 --2- 172.21.15.78:0/2110163932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0dc0c6ad0 0x7fd0dc1e9910 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8011a50 tx=0x7fd0c8011e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:04.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.705+0000 7fd0d97fa700 1 -- 172.21.15.78:0/2110163932 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c800f980 con 0x7fd0dc0c6ad0 2023-10-11T09:48:04.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.705+0000 7fd0d97fa700 1 -- 172.21.15.78:0/2110163932 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c800fae0 con 0x7fd0dc0c6ad0 2023-10-11T09:48:04.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.705+0000 7fd0d97fa700 1 -- 172.21.15.78:0/2110163932 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c800fe90 con 0x7fd0dc0c6ad0 2023-10-11T09:48:04.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.705+0000 7fd0e2ebf700 1 -- 172.21.15.78:0/2110163932 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0dc146ab0 con 0x7fd0dc0c6ad0 2023-10-11T09:48:04.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.705+0000 7fd0e2ebf700 1 -- 172.21.15.78:0/2110163932 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0dc15a580 con 0x7fd0dc0c6ad0 2023-10-11T09:48:04.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.705+0000 7fd0e2ebf700 1 -- 172.21.15.78:0/2110163932 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0dc15a7a0 con 0x7fd0dc0c6ad0 2023-10-11T09:48:04.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.705+0000 7fd0d97fa700 1 -- 172.21.15.78:0/2110163932 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fd0c8014440 con 0x7fd0dc0c6ad0 2023-10-11T09:48:04.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.709+0000 7fd0d97fa700 1 --2- 172.21.15.78:0/2110163932 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd0c0065f00 0x7fd0c00683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:04.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.709+0000 7fd0d97fa700 1 -- 172.21.15.78:0/2110163932 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6724+0+0 (secure 0 0 0) 0x7fd0c8084180 con 0x7fd0dc0c6ad0 2023-10-11T09:48:04.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.709+0000 7fd0e0c5b700 1 --2- 172.21.15.78:0/2110163932 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd0c0065f00 0x7fd0c00683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:04.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.709+0000 7fd0e0c5b700 1 --2- 172.21.15.78:0/2110163932 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd0c0065f00 0x7fd0c00683b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0015040 tx=0x7fd0d000a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:04.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.717+0000 7fd0d97fa700 1 -- 172.21.15.78:0/2110163932 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fd0dc15a7a0 con 0x7fd0dc0c6ad0 2023-10-11T09:48:04.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.893+0000 7fd0e2ebf700 1 -- 172.21.15.78:0/2110163932 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7fd0dc15a7a0 con 0x7fd0dc0c6ad0 2023-10-11T09:48:04.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.893+0000 7fd0d97fa700 1 -- 172.21.15.78:0/2110163932 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v70) v1 ==== 106+0+11 (secure 0 0 0) 0x7fd0dc15a7a0 con 0x7fd0dc0c6ad0 2023-10-11T09:48:04.900 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:48:04.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.897+0000 7fd0e2ebf700 1 -- 172.21.15.78:0/2110163932 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd0c0065f00 msgr2=0x7fd0c00683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:04.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.897+0000 7fd0e2ebf700 1 --2- 172.21.15.78:0/2110163932 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd0c0065f00 0x7fd0c00683b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0015040 tx=0x7fd0d000a000 comp rx=0 tx=0).stop 2023-10-11T09:48:04.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.897+0000 7fd0e2ebf700 1 -- 172.21.15.78:0/2110163932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0dc0c6ad0 msgr2=0x7fd0dc1e9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:04.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.897+0000 7fd0e2ebf700 1 --2- 172.21.15.78:0/2110163932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0dc0c6ad0 0x7fd0dc1e9910 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8011a50 tx=0x7fd0c8011e10 comp rx=0 tx=0).stop 2023-10-11T09:48:04.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.897+0000 7fd0e2ebf700 1 -- 172.21.15.78:0/2110163932 shutdown_connections 2023-10-11T09:48:04.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.897+0000 7fd0e2ebf700 1 --2- 172.21.15.78:0/2110163932 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd0c0065f00 0x7fd0c00683b0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:04.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.897+0000 7fd0e2ebf700 1 --2- 172.21.15.78:0/2110163932 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd0dc1483e0 0x7fd0dc148880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:04.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.897+0000 7fd0e2ebf700 1 --2- 172.21.15.78:0/2110163932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0dc0c6ad0 0x7fd0dc1e9910 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:04.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.897+0000 7fd0e2ebf700 1 --2- 172.21.15.78:0/2110163932 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0dc0b8890 0x7fd0dc15e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:04.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.897+0000 7fd0e2ebf700 1 -- 172.21.15.78:0/2110163932 >> 172.21.15.78:0/2110163932 conn(0x7fd0dc06cb50 msgr2=0x7fd0dc0b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:04.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.897+0000 7fd0e2ebf700 1 -- 172.21.15.78:0/2110163932 shutdown_connections 2023-10-11T09:48:04.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:04.897+0000 7fd0e2ebf700 1 -- 172.21.15.78:0/2110163932 wait complete. 2023-10-11T09:48:04.927 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:48:05.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.093+0000 7f9b19f90700 1 Processor -- start 2023-10-11T09:48:05.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.093+0000 7f9b19f90700 1 -- start start 2023-10-11T09:48:05.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.097+0000 7f9b19f90700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9b140cda10 0x7f9b140caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:05.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.097+0000 7f9b19f90700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f9b140cdde0 0x7f9b14164610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:05.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.097+0000 7f9b19f90700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b14165390 con 0x7f9b14164b50 2023-10-11T09:48:05.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.097+0000 7f9b19f90700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b140ca440 con 0x7f9b140cdde0 2023-10-11T09:48:05.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.097+0000 7f9b19f90700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b140ca720 con 0x7f9b140cda10 2023-10-11T09:48:05.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.097+0000 7f9b0bfff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f9b140cdde0 0x7f9b14164610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:05.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.097+0000 7f9b0bfff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f9b140cdde0 0x7f9b14164610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40628/0 (socket says 172.21.15.78:40628) 2023-10-11T09:48:05.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.097+0000 7f9b0bfff700 1 -- 172.21.15.78:0/758571656 learned_addr learned my addr 172.21.15.78:0/758571656 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:05.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.097+0000 7f9b18f8e700 1 --2- 172.21.15.78:0/758571656 >> v2:172.21.15.133:3300/0 conn(0x7f9b140cda10 0x7f9b140caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:05.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.097+0000 7f9b0bfff700 1 -- 172.21.15.78:0/758571656 >> v2:172.21.15.133:3300/0 conn(0x7f9b140cda10 msgr2=0x7f9b140caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:05.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.097+0000 7f9b0bfff700 1 --2- 172.21.15.78:0/758571656 >> v2:172.21.15.133:3300/0 conn(0x7f9b140cda10 0x7f9b140caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:05.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.097+0000 7f9b0bfff700 1 -- 172.21.15.78:0/758571656 >> v1:172.21.15.133:6789/0 conn(0x7f9b14164b50 legacy=0x7f9b1406bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:05.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.097+0000 7f9b0bfff700 1 -- 172.21.15.78:0/758571656 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b140caa00 con 0x7f9b140cdde0 2023-10-11T09:48:05.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.097+0000 7f9b0bfff700 1 --2- 172.21.15.78:0/758571656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b140cdde0 0x7f9b14164610 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f9b1406afb0 tx=0x7f9b04009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265af891237eb78e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:05.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.097+0000 7f9b0b7fe700 1 -- 172.21.15.78:0/758571656 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b04004230 con 0x7f9b140cdde0 2023-10-11T09:48:05.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.097+0000 7f9b0b7fe700 1 -- 172.21.15.78:0/758571656 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b04004390 con 0x7f9b140cdde0 2023-10-11T09:48:05.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.097+0000 7f9b0b7fe700 1 -- 172.21.15.78:0/758571656 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b040036c0 con 0x7f9b140cdde0 2023-10-11T09:48:05.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.097+0000 7f9b19f90700 1 -- 172.21.15.78:0/758571656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b140cdde0 msgr2=0x7f9b14164610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:05.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.097+0000 7f9b19f90700 1 --2- 172.21.15.78:0/758571656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b140cdde0 0x7f9b14164610 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f9b1406afb0 tx=0x7f9b04009bf0 comp rx=0 tx=0).stop 2023-10-11T09:48:05.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.097+0000 7f9b19f90700 1 -- 172.21.15.78:0/758571656 shutdown_connections 2023-10-11T09:48:05.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.097+0000 7f9b19f90700 1 --2- 172.21.15.78:0/758571656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b140cdde0 0x7f9b14164610 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:05.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.101+0000 7f9b19f90700 1 --2- 172.21.15.78:0/758571656 >> v2:172.21.15.133:3300/0 conn(0x7f9b140cda10 0x7f9b140caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:05.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.101+0000 7f9b19f90700 1 -- 172.21.15.78:0/758571656 >> 172.21.15.78:0/758571656 conn(0x7f9b140cea60 msgr2=0x7f9b140cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:05.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.101+0000 7f9b19f90700 1 -- 172.21.15.78:0/758571656 shutdown_connections 2023-10-11T09:48:05.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.101+0000 7f9b19f90700 1 -- 172.21.15.78:0/758571656 wait complete. 2023-10-11T09:48:05.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.101+0000 7f9b19f90700 1 Processor -- start 2023-10-11T09:48:05.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.101+0000 7f9b19f90700 1 -- start start 2023-10-11T09:48:05.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.105+0000 7f9b19f90700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b140cda10 0x7f9b14144d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:05.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.105+0000 7f9b19f90700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b140cdde0 0x7f9b140b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:05.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.105+0000 7f9b19f90700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9b14164b50 0x7f9b140af900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:05.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.105+0000 7f9b19f90700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b141452b0 con 0x7f9b140cda10 2023-10-11T09:48:05.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.105+0000 7f9b19f90700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b140b2cc0 con 0x7f9b140cdde0 2023-10-11T09:48:05.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.105+0000 7f9b19f90700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b140b4100 con 0x7f9b14164b50 2023-10-11T09:48:05.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.105+0000 7f9b18f8e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b140cda10 0x7f9b14144d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:05.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.105+0000 7f9b18f8e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b140cda10 0x7f9b14144d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40644/0 (socket says 172.21.15.78:40644) 2023-10-11T09:48:05.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.105+0000 7f9b18f8e700 1 -- 172.21.15.78:0/2762895217 learned_addr learned my addr 172.21.15.78:0/2762895217 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:05.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.105+0000 7f9b0bfff700 1 --2- 172.21.15.78:0/2762895217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b140cdde0 0x7f9b140b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:05.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.105+0000 7f9b1978f700 1 --2- 172.21.15.78:0/2762895217 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9b14164b50 0x7f9b140af900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:05.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.105+0000 7f9b18f8e700 1 -- 172.21.15.78:0/2762895217 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9b14164b50 msgr2=0x7f9b140af900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:05.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.105+0000 7f9b18f8e700 1 --2- 172.21.15.78:0/2762895217 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9b14164b50 0x7f9b140af900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:05.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.105+0000 7f9b18f8e700 1 -- 172.21.15.78:0/2762895217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b140cdde0 msgr2=0x7f9b140b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:05.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.105+0000 7f9b18f8e700 1 --2- 172.21.15.78:0/2762895217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b140cdde0 0x7f9b140b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:05.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.105+0000 7f9b18f8e700 1 -- 172.21.15.78:0/2762895217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b04009570 con 0x7f9b140cda10 2023-10-11T09:48:05.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.105+0000 7f9b18f8e700 1 --2- 172.21.15.78:0/2762895217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b140cda10 0x7f9b14144d70 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f9b1000e760 tx=0x7f9b1000ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:05.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.105+0000 7f9b097fa700 1 -- 172.21.15.78:0/2762895217 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b10013610 con 0x7f9b140cda10 2023-10-11T09:48:05.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.105+0000 7f9b097fa700 1 -- 172.21.15.78:0/2762895217 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b10013770 con 0x7f9b140cda10 2023-10-11T09:48:05.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.105+0000 7f9b097fa700 1 -- 172.21.15.78:0/2762895217 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b10013ae0 con 0x7f9b140cda10 2023-10-11T09:48:05.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.105+0000 7f9b19f90700 1 -- 172.21.15.78:0/2762895217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b140b4380 con 0x7f9b140cda10 2023-10-11T09:48:05.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.109+0000 7f9b19f90700 1 -- 172.21.15.78:0/2762895217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b140b2700 con 0x7f9b140cda10 2023-10-11T09:48:05.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.109+0000 7f9b19f90700 1 -- 172.21.15.78:0/2762895217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b14065d40 con 0x7f9b140cda10 2023-10-11T09:48:05.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.109+0000 7f9b097fa700 1 -- 172.21.15.78:0/2762895217 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f9b10013c40 con 0x7f9b140cda10 2023-10-11T09:48:05.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.113+0000 7f9b097fa700 1 --2- 172.21.15.78:0/2762895217 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9b00065ed0 0x7f9b00068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:05.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.113+0000 7f9b097fa700 1 -- 172.21.15.78:0/2762895217 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f9b100836f0 con 0x7f9b140cda10 2023-10-11T09:48:05.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.113+0000 7f9b0bfff700 1 --2- 172.21.15.78:0/2762895217 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9b00065ed0 0x7f9b00068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:05.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.113+0000 7f9b0bfff700 1 --2- 172.21.15.78:0/2762895217 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9b00065ed0 0x7f9b00068380 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9b1406afb0 tx=0x7f9b04008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:05.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.117+0000 7f9b097fa700 1 -- 172.21.15.78:0/2762895217 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f9b10050aa0 con 0x7f9b140cda10 2023-10-11T09:48:05.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.325+0000 7f9b19f90700 1 -- 172.21.15.78:0/2762895217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b140b4510 con 0x7f9b140cda10 2023-10-11T09:48:05.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.325+0000 7f9b097fa700 1 -- 172.21.15.78:0/2762895217 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4655 (secure 0 0 0) 0x7f9b140b4510 con 0x7f9b140cda10 2023-10-11T09:48:05.333 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:05.333 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":50,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6664,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2877345875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2877345875},{"type":"v1","addr":"172.21.15.133:6833","nonce":2877345875}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":6667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/792033922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":792033922},{"type":"v1","addr":"172.21.15.133:6835","nonce":792033922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":6894,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/1229638118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":1229638118},{"type":"v1","addr":"172.21.15.78:6836","nonce":1229638118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:45:54.252958+0000","modified":"2023-10-11T09:46:03.678418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6897},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6897":{"gid":6897,"name":"a","rank":0,"incarnation":47,"state":"up:active","state_seq":4,"addr":"172.21.15.78:6837/3498868661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":3498868661},{"type":"v1","addr":"172.21.15.78:6837","nonce":3498868661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9,10],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-11T09:48:05.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.329+0000 7f9b19f90700 1 -- 172.21.15.78:0/2762895217 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9b00065ed0 msgr2=0x7f9b00068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:05.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.329+0000 7f9b19f90700 1 --2- 172.21.15.78:0/2762895217 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9b00065ed0 0x7f9b00068380 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9b1406afb0 tx=0x7f9b04008030 comp rx=0 tx=0).stop 2023-10-11T09:48:05.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.329+0000 7f9b19f90700 1 -- 172.21.15.78:0/2762895217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b140cda10 msgr2=0x7f9b14144d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:05.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.329+0000 7f9b19f90700 1 --2- 172.21.15.78:0/2762895217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b140cda10 0x7f9b14144d70 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f9b1000e760 tx=0x7f9b1000ea70 comp rx=0 tx=0).stop 2023-10-11T09:48:05.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.329+0000 7f9b19f90700 1 -- 172.21.15.78:0/2762895217 shutdown_connections 2023-10-11T09:48:05.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.329+0000 7f9b19f90700 1 --2- 172.21.15.78:0/2762895217 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9b00065ed0 0x7f9b00068380 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:05.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.329+0000 7f9b19f90700 1 --2- 172.21.15.78:0/2762895217 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9b14164b50 0x7f9b140af900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:05.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.329+0000 7f9b19f90700 1 --2- 172.21.15.78:0/2762895217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b140cdde0 0x7f9b140b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:05.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.329+0000 7f9b19f90700 1 --2- 172.21.15.78:0/2762895217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b140cda10 0x7f9b14144d70 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:05.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.329+0000 7f9b19f90700 1 -- 172.21.15.78:0/2762895217 >> 172.21.15.78:0/2762895217 conn(0x7f9b140cea60 msgr2=0x7f9b14148dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:05.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.329+0000 7f9b19f90700 1 -- 172.21.15.78:0/2762895217 shutdown_connections 2023-10-11T09:48:05.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.329+0000 7f9b19f90700 1 -- 172.21.15.78:0/2762895217 wait complete. 2023-10-11T09:48:05.338 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 50 2023-10-11T09:48:05.357 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:48:05.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.513+0000 7f5b7cc3e700 1 Processor -- start 2023-10-11T09:48:05.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.513+0000 7f5b7cc3e700 1 -- start start 2023-10-11T09:48:05.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.517+0000 7f5b7cc3e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b78071040 0x7f5b780cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:05.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.517+0000 7f5b7cc3e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5b780cba00 0x7f5b7806c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:05.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.517+0000 7f5b7cc3e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b780720f0 con 0x7f5b78154e00 2023-10-11T09:48:05.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.517+0000 7f5b7cc3e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b780714e0 con 0x7f5b780cba00 2023-10-11T09:48:05.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.517+0000 7f5b7cc3e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b780717c0 con 0x7f5b78071040 2023-10-11T09:48:05.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.517+0000 7f5b7659c700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f5b78154e00 0x7f5b7815b650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:39936/0 (socket says 172.21.15.78:39936) 2023-10-11T09:48:05.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.517+0000 7f5b7659c700 1 -- 172.21.15.78:0/2114323800 learned_addr learned my addr 172.21.15.78:0/2114323800 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:05.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.517+0000 7f5b76d9d700 1 --2- 172.21.15.78:0/2114323800 >> v2:172.21.15.133:3300/0 conn(0x7f5b780cba00 0x7f5b7806c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:05.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.517+0000 7f5b6dd9b700 1 --2- 172.21.15.78:0/2114323800 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b78071040 0x7f5b780cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:05.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.517+0000 7f5b75d9b700 1 -- 172.21.15.78:0/2114323800 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3170902394 0 0) 0x7f5b780720f0 con 0x7f5b78154e00 2023-10-11T09:48:05.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.517+0000 7f5b75d9b700 1 -- 172.21.15.78:0/2114323800 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b580035d0 con 0x7f5b78154e00 2023-10-11T09:48:05.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.517+0000 7f5b75d9b700 1 -- 172.21.15.78:0/2114323800 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3953135726 0 0) 0x7f5b580035d0 con 0x7f5b78154e00 2023-10-11T09:48:05.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.517+0000 7f5b75d9b700 1 -- 172.21.15.78:0/2114323800 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b78071040 msgr2=0x7f5b780cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:05.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.517+0000 7f5b75d9b700 1 --2- 172.21.15.78:0/2114323800 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b78071040 0x7f5b780cb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:05.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.517+0000 7f5b75d9b700 1 -- 172.21.15.78:0/2114323800 >> v2:172.21.15.133:3300/0 conn(0x7f5b780cba00 msgr2=0x7f5b7806c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:05.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.517+0000 7f5b75d9b700 1 --2- 172.21.15.78:0/2114323800 >> v2:172.21.15.133:3300/0 conn(0x7f5b780cba00 0x7f5b7806c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:05.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.517+0000 7f5b75d9b700 1 -- 172.21.15.78:0/2114323800 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b78071aa0 con 0x7f5b78154e00 2023-10-11T09:48:05.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.517+0000 7f5b75d9b700 1 -- 172.21.15.78:0/2114323800 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f5b68001670 con 0x7f5b78154e00 2023-10-11T09:48:05.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.517+0000 7f5b75d9b700 1 -- 172.21.15.78:0/2114323800 >> v1:172.21.15.78:6789/0 conn(0x7f5b78154e00 legacy=0x7f5b7815b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:05.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.517+0000 7f5b75d9b700 1 --2- 172.21.15.78:0/2114323800 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b78071040 0x7f5b58003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:05.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.517+0000 7f5b75d9b700 1 --2- 172.21.15.78:0/2114323800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b58003fa0 0x7f5b58004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:05.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.517+0000 7f5b75d9b700 1 --2- 172.21.15.78:0/2114323800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b58004db0 0x7f5b58005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:05.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.517+0000 7f5b75d9b700 1 -- 172.21.15.78:0/2114323800 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b780714e0 con 0x7f5b58004db0 2023-10-11T09:48:05.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.517+0000 7f5b75d9b700 1 -- 172.21.15.78:0/2114323800 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b780717c0 con 0x7f5b58003fa0 2023-10-11T09:48:05.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.517+0000 7f5b75d9b700 1 -- 172.21.15.78:0/2114323800 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5800ba30 con 0x7f5b78071040 2023-10-11T09:48:05.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.517+0000 7f5b76d9d700 1 --2- 172.21.15.78:0/2114323800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b58004db0 0x7f5b58005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:05.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.517+0000 7f5b7659c700 1 --2- 172.21.15.78:0/2114323800 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b78071040 0x7f5b58003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:05.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.517+0000 7f5b6dd9b700 1 --2- 172.21.15.78:0/2114323800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b58003fa0 0x7f5b58004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:05.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.521+0000 7f5b76d9d700 1 -- 172.21.15.78:0/2114323800 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b78071040 msgr2=0x7f5b58003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:05.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.521+0000 7f5b76d9d700 1 --2- 172.21.15.78:0/2114323800 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b78071040 0x7f5b58003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:05.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.521+0000 7f5b76d9d700 1 -- 172.21.15.78:0/2114323800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b58003fa0 msgr2=0x7f5b58004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:05.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.521+0000 7f5b76d9d700 1 --2- 172.21.15.78:0/2114323800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b58003fa0 0x7f5b58004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:05.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.521+0000 7f5b76d9d700 1 -- 172.21.15.78:0/2114323800 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b580035d0 con 0x7f5b58004db0 2023-10-11T09:48:05.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.521+0000 7f5b7659c700 1 --2- 172.21.15.78:0/2114323800 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b78071040 0x7f5b58003a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:48:05.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.521+0000 7f5b76d9d700 1 --2- 172.21.15.78:0/2114323800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b58004db0 0x7f5b58005220 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009fc0 tx=0x7f5b580058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d665aef201f9a03 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:05.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.521+0000 7f5b75d9b700 1 -- 172.21.15.78:0/2114323800 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7000ec30 con 0x7f5b58004db0 2023-10-11T09:48:05.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.521+0000 7f5b75d9b700 1 -- 172.21.15.78:0/2114323800 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b7000f040 con 0x7f5b58004db0 2023-10-11T09:48:05.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.521+0000 7f5b75d9b700 1 -- 172.21.15.78:0/2114323800 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b70014480 con 0x7f5b58004db0 2023-10-11T09:48:05.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.521+0000 7f5b7cc3e700 1 -- 172.21.15.78:0/2114323800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b58004db0 msgr2=0x7f5b58005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:05.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.521+0000 7f5b7cc3e700 1 --2- 172.21.15.78:0/2114323800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b58004db0 0x7f5b58005220 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009fc0 tx=0x7f5b580058a0 comp rx=0 tx=0).stop 2023-10-11T09:48:05.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.521+0000 7f5b76d9d700 1 -- 172.21.15.78:0/2114323800 reap_dead start 2023-10-11T09:48:05.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.521+0000 7f5b7cc3e700 1 -- 172.21.15.78:0/2114323800 shutdown_connections 2023-10-11T09:48:05.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.521+0000 7f5b7cc3e700 1 -- 172.21.15.78:0/2114323800 >> 172.21.15.78:0/2114323800 conn(0x7f5b7806aca0 msgr2=0x7f5b780b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:05.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.521+0000 7f5b7cc3e700 1 -- 172.21.15.78:0/2114323800 shutdown_connections 2023-10-11T09:48:05.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.521+0000 7f5b7cc3e700 1 -- 172.21.15.78:0/2114323800 wait complete. 2023-10-11T09:48:05.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.525+0000 7f5b7cc3e700 1 Processor -- start 2023-10-11T09:48:05.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.525+0000 7f5b7cc3e700 1 -- start start 2023-10-11T09:48:05.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.525+0000 7f5b7cc3e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b58004db0 0x7f5b781479e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:05.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.525+0000 7f5b7cc3e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b780b3100 0x7f5b780b3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:05.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.525+0000 7f5b7cc3e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b7814a5e0 0x7f5b7814aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:05.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.525+0000 7f5b7cc3e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b780b3ab0 con 0x7f5b58004db0 2023-10-11T09:48:05.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.525+0000 7f5b7cc3e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7815b0a0 con 0x7f5b7814a5e0 2023-10-11T09:48:05.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.525+0000 7f5b7cc3e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7815b340 con 0x7f5b780b3100 2023-10-11T09:48:05.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.525+0000 7f5b7659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b58004db0 0x7f5b781479e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:05.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.525+0000 7f5b7659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b58004db0 0x7f5b781479e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40652/0 (socket says 172.21.15.78:40652) 2023-10-11T09:48:05.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.525+0000 7f5b7659c700 1 -- 172.21.15.78:0/1218855405 learned_addr learned my addr 172.21.15.78:0/1218855405 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:05.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.525+0000 7f5b76d9d700 1 --2- 172.21.15.78:0/1218855405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b7814a5e0 0x7f5b7814aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:05.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.525+0000 7f5b6dd9b700 1 --2- 172.21.15.78:0/1218855405 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b780b3100 0x7f5b780b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:05.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.525+0000 7f5b7659c700 1 -- 172.21.15.78:0/1218855405 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b780b3100 msgr2=0x7f5b780b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:05.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.529+0000 7f5b7659c700 1 --2- 172.21.15.78:0/1218855405 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b780b3100 0x7f5b780b3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:05.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.529+0000 7f5b7659c700 1 -- 172.21.15.78:0/1218855405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b7814a5e0 msgr2=0x7f5b7814aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:05.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.529+0000 7f5b7659c700 1 --2- 172.21.15.78:0/1218855405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b7814a5e0 0x7f5b7814aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:05.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.529+0000 7f5b7659c700 1 -- 172.21.15.78:0/1218855405 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b7815b620 con 0x7f5b58004db0 2023-10-11T09:48:05.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.529+0000 7f5b6dd9b700 1 --2- 172.21.15.78:0/1218855405 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b780b3100 0x7f5b780b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:48:05.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.529+0000 7f5b7659c700 1 --2- 172.21.15.78:0/1218855405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b58004db0 0x7f5b781479e0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f5b68007680 tx=0x7f5b6800f4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:05.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.529+0000 7f5b6f7fe700 1 -- 172.21.15.78:0/1218855405 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6800fac0 con 0x7f5b58004db0 2023-10-11T09:48:05.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.529+0000 7f5b6f7fe700 1 -- 172.21.15.78:0/1218855405 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b6800fc20 con 0x7f5b58004db0 2023-10-11T09:48:05.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.529+0000 7f5b7cc3e700 1 -- 172.21.15.78:0/1218855405 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b7815b900 con 0x7f5b58004db0 2023-10-11T09:48:05.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.529+0000 7f5b6f7fe700 1 -- 172.21.15.78:0/1218855405 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b680133f0 con 0x7f5b58004db0 2023-10-11T09:48:05.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.529+0000 7f5b7cc3e700 1 -- 172.21.15.78:0/1218855405 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b7814af90 con 0x7f5b58004db0 2023-10-11T09:48:05.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.529+0000 7f5b7cc3e700 1 -- 172.21.15.78:0/1218855405 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b7815bdc0 con 0x7f5b58004db0 2023-10-11T09:48:05.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.533+0000 7f5b6f7fe700 1 -- 172.21.15.78:0/1218855405 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f5b68013550 con 0x7f5b58004db0 2023-10-11T09:48:05.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.533+0000 7f5b6f7fe700 1 --2- 172.21.15.78:0/1218855405 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5b5c065f00 0x7f5b5c0683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:05.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.533+0000 7f5b6f7fe700 1 -- 172.21.15.78:0/1218855405 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f5b68083ed0 con 0x7f5b58004db0 2023-10-11T09:48:05.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.533+0000 7f5b6dd9b700 1 --2- 172.21.15.78:0/1218855405 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5b5c065f00 0x7f5b5c0683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:05.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.533+0000 7f5b6dd9b700 1 --2- 172.21.15.78:0/1218855405 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5b5c065f00 0x7f5b5c0683b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5b781f5930 tx=0x7f5b60008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:05.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.537+0000 7f5b6f7fe700 1 -- 172.21.15.78:0/1218855405 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f5b68051280 con 0x7f5b58004db0 2023-10-11T09:48:05.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.713+0000 7f5b7cc3e700 1 -- 172.21.15.78:0/1218855405 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b7804e530 con 0x7f5b58004db0 2023-10-11T09:48:05.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.717+0000 7f5b6f7fe700 1 -- 172.21.15.78:0/1218855405 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+20052 (secure 0 0 0) 0x7f5b680548a0 con 0x7f5b58004db0 2023-10-11T09:48:05.726 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:05.727 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":70,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:46:00.671033+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:44:05.198742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-11T09:44:05.868171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data_extra","create_time":"2023-10-11T09:44:16.049941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6836/1666964283":"2023-10-12T09:46:00.635561+0000","172.21.15.78:6834/1666964283":"2023-10-12T09:46:00.635561+0000","172.21.15.78:6836/1819646091":"2023-10-12T09:45:54.334448+0000","172.21.15.78:6834/1819646091":"2023-10-12T09:45:54.334448+0000","172.21.15.78:6837/852594750":"2023-10-12T09:45:53.299952+0000","172.21.15.78:6835/852594750":"2023-10-12T09:45:53.299952+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:48:05.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.721+0000 7f5b7cc3e700 1 -- 172.21.15.78:0/1218855405 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5b5c065f00 msgr2=0x7f5b5c0683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:05.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.721+0000 7f5b7cc3e700 1 --2- 172.21.15.78:0/1218855405 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5b5c065f00 0x7f5b5c0683b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5b781f5930 tx=0x7f5b60008040 comp rx=0 tx=0).stop 2023-10-11T09:48:05.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.721+0000 7f5b7cc3e700 1 -- 172.21.15.78:0/1218855405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b58004db0 msgr2=0x7f5b781479e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:05.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.721+0000 7f5b7cc3e700 1 --2- 172.21.15.78:0/1218855405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b58004db0 0x7f5b781479e0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f5b68007680 tx=0x7f5b6800f4b0 comp rx=0 tx=0).stop 2023-10-11T09:48:05.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.721+0000 7f5b7cc3e700 1 -- 172.21.15.78:0/1218855405 shutdown_connections 2023-10-11T09:48:05.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.721+0000 7f5b7cc3e700 1 --2- 172.21.15.78:0/1218855405 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5b5c065f00 0x7f5b5c0683b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:05.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.721+0000 7f5b7cc3e700 1 --2- 172.21.15.78:0/1218855405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b7814a5e0 0x7f5b7814aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:05.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.721+0000 7f5b7cc3e700 1 --2- 172.21.15.78:0/1218855405 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b780b3100 0x7f5b780b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:05.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.721+0000 7f5b7cc3e700 1 --2- 172.21.15.78:0/1218855405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b58004db0 0x7f5b781479e0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:05.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.721+0000 7f5b7cc3e700 1 -- 172.21.15.78:0/1218855405 >> 172.21.15.78:0/1218855405 conn(0x7f5b7806aca0 msgr2=0x7f5b7814f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:05.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.721+0000 7f5b7cc3e700 1 -- 172.21.15.78:0/1218855405 shutdown_connections 2023-10-11T09:48:05.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.721+0000 7f5b7cc3e700 1 -- 172.21.15.78:0/1218855405 wait complete. 2023-10-11T09:48:05.752 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-11T09:48:05.753 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:48:05.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.917+0000 7f83c5770700 1 Processor -- start 2023-10-11T09:48:05.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.917+0000 7f83c5770700 1 -- start start 2023-10-11T09:48:05.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.921+0000 7f83c5770700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f83c00d1160 0x7f83c00cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:05.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.921+0000 7f83c5770700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83c015dc10 con 0x7f83c00d1660 2023-10-11T09:48:05.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.921+0000 7f83c5770700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83c0160380 con 0x7f83c0156ce0 2023-10-11T09:48:05.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.921+0000 7f83c5770700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c015d930 con 0x7f83c00d1160 2023-10-11T09:48:05.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.921+0000 7f83bf7fe700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f83c00d1660 0x7f83c00d0a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:39952/0 (socket says 172.21.15.78:39952) 2023-10-11T09:48:05.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.921+0000 7f83bf7fe700 1 -- 172.21.15.78:0/59546068 learned_addr learned my addr 172.21.15.78:0/59546068 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:05.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.921+0000 7f83c4f6f700 1 --2- 172.21.15.78:0/59546068 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f83c00d1160 0x7f83c00cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:05.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.921+0000 7f83beffd700 1 -- 172.21.15.78:0/59546068 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1571031605 0 0) 0x7f83c015dc10 con 0x7f83c00d1660 2023-10-11T09:48:05.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.921+0000 7f83beffd700 1 -- 172.21.15.78:0/59546068 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83a80035d0 con 0x7f83c00d1660 2023-10-11T09:48:05.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.921+0000 7f83beffd700 1 -- 172.21.15.78:0/59546068 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 968780084 0 0) 0x7f83c0160380 con 0x7f83c0156ce0 2023-10-11T09:48:05.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.921+0000 7f83beffd700 1 -- 172.21.15.78:0/59546068 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83c015dc10 con 0x7f83c0156ce0 2023-10-11T09:48:05.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.921+0000 7f83beffd700 1 -- 172.21.15.78:0/59546068 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2641416674 0 0) 0x7f83a80035d0 con 0x7f83c00d1660 2023-10-11T09:48:05.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.921+0000 7f83c4f6f700 1 -- 172.21.15.78:0/59546068 >> v1:172.21.15.133:6789/0 conn(0x7f83c0156ce0 legacy=0x7f83c0160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:05.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.921+0000 7f83c4f6f700 1 -- 172.21.15.78:0/59546068 >> v1:172.21.15.78:6789/0 conn(0x7f83c00d1660 legacy=0x7f83c00d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:05.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.921+0000 7f83c4f6f700 1 -- 172.21.15.78:0/59546068 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c0160620 con 0x7f83c00d1160 2023-10-11T09:48:05.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.921+0000 7f83c4f6f700 1 --2- 172.21.15.78:0/59546068 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f83c00d1160 0x7f83c00cefc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f83b8009dc0 tx=0x7f83b800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14b1c91bf54dccf9 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:05.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.921+0000 7f83beffd700 1 -- 172.21.15.78:0/59546068 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b800e070 con 0x7f83c00d1160 2023-10-11T09:48:05.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.921+0000 7f83beffd700 1 -- 172.21.15.78:0/59546068 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b8003a40 con 0x7f83c00d1160 2023-10-11T09:48:05.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.921+0000 7f83beffd700 1 -- 172.21.15.78:0/59546068 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b4001670 con 0x7f83c00d1160 2023-10-11T09:48:05.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.925+0000 7f83c5770700 1 -- 172.21.15.78:0/59546068 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f83c00d1160 msgr2=0x7f83c00cefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:05.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.925+0000 7f83c5770700 1 --2- 172.21.15.78:0/59546068 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f83c00d1160 0x7f83c00cefc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f83b8009dc0 tx=0x7f83b800b330 comp rx=0 tx=0).stop 2023-10-11T09:48:05.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.925+0000 7f83c5770700 1 -- 172.21.15.78:0/59546068 shutdown_connections 2023-10-11T09:48:05.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.925+0000 7f83c5770700 1 --2- 172.21.15.78:0/59546068 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f83c00d1160 0x7f83c00cefc0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:05.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.925+0000 7f83c5770700 1 -- 172.21.15.78:0/59546068 >> 172.21.15.78:0/59546068 conn(0x7f83c006afb0 msgr2=0x7f83c006b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:05.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.925+0000 7f83c5770700 1 -- 172.21.15.78:0/59546068 shutdown_connections 2023-10-11T09:48:05.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.925+0000 7f83c5770700 1 -- 172.21.15.78:0/59546068 wait complete. 2023-10-11T09:48:05.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.925+0000 7f83c5770700 1 Processor -- start 2023-10-11T09:48:05.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.925+0000 7f83c5770700 1 -- start start 2023-10-11T09:48:05.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.929+0000 7f83c5770700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83c00d1160 0x7f83c00c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:05.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.929+0000 7f83c5770700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83c00d1660 0x7f83c00cb770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:05.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.929+0000 7f83c5770700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f83c0156ce0 0x7f83c00cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:05.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.929+0000 7f83c5770700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c00ca7a0 con 0x7f83c00d1660 2023-10-11T09:48:05.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.929+0000 7f83c5770700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c00caa80 con 0x7f83c00d1160 2023-10-11T09:48:05.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.929+0000 7f83c5770700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c00cad60 con 0x7f83c0156ce0 2023-10-11T09:48:05.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.929+0000 7f83bf7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83c00d1660 0x7f83c00cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:05.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.929+0000 7f83bf7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83c00d1660 0x7f83c00cb770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40654/0 (socket says 172.21.15.78:40654) 2023-10-11T09:48:05.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.929+0000 7f83bf7fe700 1 -- 172.21.15.78:0/2830625792 learned_addr learned my addr 172.21.15.78:0/2830625792 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:05.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.929+0000 7f83c4f6f700 1 --2- 172.21.15.78:0/2830625792 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f83c0156ce0 0x7f83c00cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:05.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.929+0000 7f83bffff700 1 --2- 172.21.15.78:0/2830625792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83c00d1160 0x7f83c00c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:05.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.929+0000 7f83bf7fe700 1 -- 172.21.15.78:0/2830625792 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f83c0156ce0 msgr2=0x7f83c00cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:05.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.929+0000 7f83bf7fe700 1 --2- 172.21.15.78:0/2830625792 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f83c0156ce0 0x7f83c00cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:05.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.929+0000 7f83bf7fe700 1 -- 172.21.15.78:0/2830625792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83c00d1160 msgr2=0x7f83c00c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:05.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.929+0000 7f83bf7fe700 1 --2- 172.21.15.78:0/2830625792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83c00d1160 0x7f83c00c9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:05.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.929+0000 7f83bf7fe700 1 -- 172.21.15.78:0/2830625792 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b8009a70 con 0x7f83c00d1660 2023-10-11T09:48:05.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.929+0000 7f83c4f6f700 1 --2- 172.21.15.78:0/2830625792 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f83c0156ce0 0x7f83c00cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:48:05.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.929+0000 7f83bffff700 1 --2- 172.21.15.78:0/2830625792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83c00d1160 0x7f83c00c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:48:05.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.929+0000 7f83bf7fe700 1 --2- 172.21.15.78:0/2830625792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83c00d1660 0x7f83c00cb770 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f83b400c750 tx=0x7f83b400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:05.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.929+0000 7f83bcff9700 1 -- 172.21.15.78:0/2830625792 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b4004ca0 con 0x7f83c00d1660 2023-10-11T09:48:05.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.929+0000 7f83bcff9700 1 -- 172.21.15.78:0/2830625792 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b4004e00 con 0x7f83c00d1660 2023-10-11T09:48:05.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.929+0000 7f83c5770700 1 -- 172.21.15.78:0/2830625792 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83c00cb040 con 0x7f83c00d1660 2023-10-11T09:48:05.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.929+0000 7f83bcff9700 1 -- 172.21.15.78:0/2830625792 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b40072f0 con 0x7f83c00d1660 2023-10-11T09:48:05.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.929+0000 7f83c5770700 1 -- 172.21.15.78:0/2830625792 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83c00ca490 con 0x7f83c00d1660 2023-10-11T09:48:05.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.933+0000 7f83c5770700 1 -- 172.21.15.78:0/2830625792 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83c00cb560 con 0x7f83c00d1660 2023-10-11T09:48:05.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.933+0000 7f83bcff9700 1 -- 172.21.15.78:0/2830625792 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f83b4007450 con 0x7f83c00d1660 2023-10-11T09:48:05.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.933+0000 7f83bcff9700 1 --2- 172.21.15.78:0/2830625792 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f83ac065ed0 0x7f83ac068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:05.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.933+0000 7f83bcff9700 1 -- 172.21.15.78:0/2830625792 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f83b4083120 con 0x7f83c00d1660 2023-10-11T09:48:05.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.933+0000 7f83bffff700 1 --2- 172.21.15.78:0/2830625792 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f83ac065ed0 0x7f83ac068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:05.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.937+0000 7f83bffff700 1 --2- 172.21.15.78:0/2830625792 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f83ac065ed0 0x7f83ac068380 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f83b00011e0 tx=0x7f83b0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:05.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:05.941+0000 7f83bcff9700 1 -- 172.21.15.78:0/2830625792 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f83c00cb560 con 0x7f83c00d1660 2023-10-11T09:48:06.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.153+0000 7f83c5770700 1 -- 172.21.15.78:0/2830625792 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83c00cb560 con 0x7f83c00d1660 2023-10-11T09:48:06.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.157+0000 7f83bcff9700 1 -- 172.21.15.78:0/2830625792 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4655 (secure 0 0 0) 0x7f83c00cb560 con 0x7f83c00d1660 2023-10-11T09:48:06.164 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:06.165 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":50,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6664,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2877345875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2877345875},{"type":"v1","addr":"172.21.15.133:6833","nonce":2877345875}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":6667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/792033922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":792033922},{"type":"v1","addr":"172.21.15.133:6835","nonce":792033922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":6894,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/1229638118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":1229638118},{"type":"v1","addr":"172.21.15.78:6836","nonce":1229638118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:45:54.252958+0000","modified":"2023-10-11T09:46:03.678418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6897},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6897":{"gid":6897,"name":"a","rank":0,"incarnation":47,"state":"up:active","state_seq":4,"addr":"172.21.15.78:6837/3498868661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":3498868661},{"type":"v1","addr":"172.21.15.78:6837","nonce":3498868661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9,10],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-11T09:48:06.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.157+0000 7f83c5770700 1 -- 172.21.15.78:0/2830625792 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f83ac065ed0 msgr2=0x7f83ac068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:06.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.157+0000 7f83c5770700 1 --2- 172.21.15.78:0/2830625792 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f83ac065ed0 0x7f83ac068380 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f83b00011e0 tx=0x7f83b0009040 comp rx=0 tx=0).stop 2023-10-11T09:48:06.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.157+0000 7f83c5770700 1 -- 172.21.15.78:0/2830625792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83c00d1660 msgr2=0x7f83c00cb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:06.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.157+0000 7f83c5770700 1 --2- 172.21.15.78:0/2830625792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83c00d1660 0x7f83c00cb770 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f83b400c750 tx=0x7f83b400cb10 comp rx=0 tx=0).stop 2023-10-11T09:48:06.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.157+0000 7f83c5770700 1 -- 172.21.15.78:0/2830625792 shutdown_connections 2023-10-11T09:48:06.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.157+0000 7f83c5770700 1 --2- 172.21.15.78:0/2830625792 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f83ac065ed0 0x7f83ac068380 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:06.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.157+0000 7f83c5770700 1 --2- 172.21.15.78:0/2830625792 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f83c0156ce0 0x7f83c00cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:06.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.157+0000 7f83c5770700 1 --2- 172.21.15.78:0/2830625792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83c00d1660 0x7f83c00cb770 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:06.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.157+0000 7f83c5770700 1 --2- 172.21.15.78:0/2830625792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83c00d1160 0x7f83c00c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:06.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.157+0000 7f83c5770700 1 -- 172.21.15.78:0/2830625792 >> 172.21.15.78:0/2830625792 conn(0x7f83c006afb0 msgr2=0x7f83c00d09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:06.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.161+0000 7f83c5770700 1 -- 172.21.15.78:0/2830625792 shutdown_connections 2023-10-11T09:48:06.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.161+0000 7f83c5770700 1 -- 172.21.15.78:0/2830625792 wait complete. 2023-10-11T09:48:06.171 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 50 2023-10-11T09:48:06.190 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:48:06.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.345+0000 7fac8721e700 1 Processor -- start 2023-10-11T09:48:06.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.345+0000 7fac8721e700 1 -- start start 2023-10-11T09:48:06.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.345+0000 7fac8721e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac800b5f30 0x7fac800b63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:06.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.345+0000 7fac8721e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac800d11e0 con 0x7fac80161150 2023-10-11T09:48:06.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.345+0000 7fac8721e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac800b5370 con 0x7fac800b2310 2023-10-11T09:48:06.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.345+0000 7fac8721e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac801615b0 con 0x7fac800b5f30 2023-10-11T09:48:06.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.349+0000 7fac857bb700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac800b5f30 0x7fac800b63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:06.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.349+0000 7fac77fff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fac80161150 0x7fac800d1ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:39968/0 (socket says 172.21.15.78:39968) 2023-10-11T09:48:06.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.349+0000 7fac84fba700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fac800b2310 0x7fac800b4b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:45760/0 (socket says 172.21.15.78:45760) 2023-10-11T09:48:06.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.349+0000 7fac77fff700 1 -- 172.21.15.78:0/366401589 learned_addr learned my addr 172.21.15.78:0/366401589 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:06.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.349+0000 7fac857bb700 1 -- 172.21.15.78:0/366401589 >> v1:172.21.15.133:6789/0 conn(0x7fac800b2310 legacy=0x7fac800b4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:06.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.349+0000 7fac857bb700 1 -- 172.21.15.78:0/366401589 >> v1:172.21.15.78:6789/0 conn(0x7fac80161150 legacy=0x7fac800d1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:06.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.349+0000 7fac857bb700 1 -- 172.21.15.78:0/366401589 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac800d14a0 con 0x7fac800b5f30 2023-10-11T09:48:06.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.349+0000 7fac857bb700 1 --2- 172.21.15.78:0/366401589 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac800b5f30 0x7fac800b63b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fac78009880 tx=0x7fac78009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57c5635860e661b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:06.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.349+0000 7fac777fe700 1 -- 172.21.15.78:0/366401589 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac78011830 con 0x7fac800b5f30 2023-10-11T09:48:06.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.349+0000 7fac777fe700 1 -- 172.21.15.78:0/366401589 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac78011990 con 0x7fac800b5f30 2023-10-11T09:48:06.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.349+0000 7fac777fe700 1 -- 172.21.15.78:0/366401589 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac78011d20 con 0x7fac800b5f30 2023-10-11T09:48:06.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.349+0000 7fac8721e700 1 -- 172.21.15.78:0/366401589 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac800b5f30 msgr2=0x7fac800b63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:06.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.349+0000 7fac8721e700 1 --2- 172.21.15.78:0/366401589 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac800b5f30 0x7fac800b63b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fac78009880 tx=0x7fac78009b90 comp rx=0 tx=0).stop 2023-10-11T09:48:06.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.349+0000 7fac8721e700 1 -- 172.21.15.78:0/366401589 shutdown_connections 2023-10-11T09:48:06.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.349+0000 7fac8721e700 1 --2- 172.21.15.78:0/366401589 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac800b5f30 0x7fac800b63b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:06.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.349+0000 7fac8721e700 1 -- 172.21.15.78:0/366401589 >> 172.21.15.78:0/366401589 conn(0x7fac800cfb60 msgr2=0x7fac800cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:06.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.349+0000 7fac8721e700 1 -- 172.21.15.78:0/366401589 shutdown_connections 2023-10-11T09:48:06.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.349+0000 7fac8721e700 1 -- 172.21.15.78:0/366401589 wait complete. 2023-10-11T09:48:06.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.353+0000 7fac8721e700 1 Processor -- start 2023-10-11T09:48:06.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.353+0000 7fac8721e700 1 -- start start 2023-10-11T09:48:06.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.353+0000 7fac8721e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac800b2310 0x7fac80143e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:06.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.353+0000 7fac8721e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac800b5f30 0x7fac80152c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:06.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.353+0000 7fac8721e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac80161150 0x7fac8015fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:06.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.353+0000 7fac8721e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac80141f00 con 0x7fac800b5f30 2023-10-11T09:48:06.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.353+0000 7fac8721e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac80141280 con 0x7fac800b2310 2023-10-11T09:48:06.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.353+0000 7fac8721e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac80141560 con 0x7fac80161150 2023-10-11T09:48:06.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.357+0000 7fac77fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac800b5f30 0x7fac80152c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:06.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.357+0000 7fac77fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac800b5f30 0x7fac80152c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40662/0 (socket says 172.21.15.78:40662) 2023-10-11T09:48:06.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.357+0000 7fac77fff700 1 -- 172.21.15.78:0/280552928 learned_addr learned my addr 172.21.15.78:0/280552928 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:06.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.357+0000 7fac84fba700 1 --2- 172.21.15.78:0/280552928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac800b2310 0x7fac80143e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:06.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.357+0000 7fac857bb700 1 --2- 172.21.15.78:0/280552928 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac80161150 0x7fac8015fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:06.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.357+0000 7fac77fff700 1 -- 172.21.15.78:0/280552928 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac80161150 msgr2=0x7fac8015fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:06.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.357+0000 7fac77fff700 1 --2- 172.21.15.78:0/280552928 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac80161150 0x7fac8015fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:06.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.357+0000 7fac77fff700 1 -- 172.21.15.78:0/280552928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac800b2310 msgr2=0x7fac80143e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:06.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.357+0000 7fac77fff700 1 --2- 172.21.15.78:0/280552928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac800b2310 0x7fac80143e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:06.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.357+0000 7fac77fff700 1 -- 172.21.15.78:0/280552928 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac780094b0 con 0x7fac800b5f30 2023-10-11T09:48:06.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.357+0000 7fac77fff700 1 --2- 172.21.15.78:0/280552928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac800b5f30 0x7fac80152c90 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fac7000c410 tx=0x7fac7000c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:06.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.357+0000 7fac757fa700 1 -- 172.21.15.78:0/280552928 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac70005d90 con 0x7fac800b5f30 2023-10-11T09:48:06.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.357+0000 7fac757fa700 1 -- 172.21.15.78:0/280552928 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac7000ce20 con 0x7fac800b5f30 2023-10-11T09:48:06.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.357+0000 7fac757fa700 1 -- 172.21.15.78:0/280552928 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac700072f0 con 0x7fac800b5f30 2023-10-11T09:48:06.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.357+0000 7fac8721e700 1 -- 172.21.15.78:0/280552928 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac80141840 con 0x7fac800b5f30 2023-10-11T09:48:06.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.357+0000 7fac8721e700 1 -- 172.21.15.78:0/280552928 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac8015ff90 con 0x7fac800b5f30 2023-10-11T09:48:06.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.361+0000 7fac8721e700 1 -- 172.21.15.78:0/280552928 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac8004e530 con 0x7fac800b5f30 2023-10-11T09:48:06.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.361+0000 7fac757fa700 1 -- 172.21.15.78:0/280552928 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fac70007450 con 0x7fac800b5f30 2023-10-11T09:48:06.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.361+0000 7fac757fa700 1 --2- 172.21.15.78:0/280552928 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fac6c065ed0 0x7fac6c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:06.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.361+0000 7fac757fa700 1 -- 172.21.15.78:0/280552928 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6724+0+0 (secure 0 0 0) 0x7fac70082da0 con 0x7fac800b5f30 2023-10-11T09:48:06.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.361+0000 7fac84fba700 1 --2- 172.21.15.78:0/280552928 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fac6c065ed0 0x7fac6c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:06.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.365+0000 7fac84fba700 1 --2- 172.21.15.78:0/280552928 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fac6c065ed0 0x7fac6c068380 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fac7c0012e0 tx=0x7fac7c001af0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:06.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.369+0000 7fac757fa700 1 -- 172.21.15.78:0/280552928 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fac70050150 con 0x7fac800b5f30 2023-10-11T09:48:06.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.581+0000 7fac8721e700 1 -- 172.21.15.78:0/280552928 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac800c6990 con 0x7fac800b5f30 2023-10-11T09:48:06.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.585+0000 7fac757fa700 1 -- 172.21.15.78:0/280552928 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4655 (secure 0 0 0) 0x7fac70017020 con 0x7fac800b5f30 2023-10-11T09:48:06.592 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:06.593 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":50,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6664,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2877345875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2877345875},{"type":"v1","addr":"172.21.15.133:6833","nonce":2877345875}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":6667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/792033922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":792033922},{"type":"v1","addr":"172.21.15.133:6835","nonce":792033922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":6894,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/1229638118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":1229638118},{"type":"v1","addr":"172.21.15.78:6836","nonce":1229638118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:45:54.252958+0000","modified":"2023-10-11T09:46:03.678418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6897},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6897":{"gid":6897,"name":"a","rank":0,"incarnation":47,"state":"up:active","state_seq":4,"addr":"172.21.15.78:6837/3498868661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":3498868661},{"type":"v1","addr":"172.21.15.78:6837","nonce":3498868661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9,10],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-11T09:48:06.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.585+0000 7fac8721e700 1 -- 172.21.15.78:0/280552928 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fac6c065ed0 msgr2=0x7fac6c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:06.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.585+0000 7fac8721e700 1 --2- 172.21.15.78:0/280552928 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fac6c065ed0 0x7fac6c068380 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fac7c0012e0 tx=0x7fac7c001af0 comp rx=0 tx=0).stop 2023-10-11T09:48:06.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.585+0000 7fac8721e700 1 -- 172.21.15.78:0/280552928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac800b5f30 msgr2=0x7fac80152c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:06.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.585+0000 7fac8721e700 1 --2- 172.21.15.78:0/280552928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac800b5f30 0x7fac80152c90 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fac7000c410 tx=0x7fac7000c720 comp rx=0 tx=0).stop 2023-10-11T09:48:06.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.589+0000 7fac8721e700 1 -- 172.21.15.78:0/280552928 shutdown_connections 2023-10-11T09:48:06.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.589+0000 7fac8721e700 1 --2- 172.21.15.78:0/280552928 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fac6c065ed0 0x7fac6c068380 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:06.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.589+0000 7fac8721e700 1 --2- 172.21.15.78:0/280552928 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac80161150 0x7fac8015fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:06.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.589+0000 7fac8721e700 1 --2- 172.21.15.78:0/280552928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac800b5f30 0x7fac80152c90 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:06.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.589+0000 7fac8721e700 1 --2- 172.21.15.78:0/280552928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac800b2310 0x7fac80143e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:06.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.589+0000 7fac8721e700 1 -- 172.21.15.78:0/280552928 >> 172.21.15.78:0/280552928 conn(0x7fac800cfb60 msgr2=0x7fac800b8540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:06.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.589+0000 7fac8721e700 1 -- 172.21.15.78:0/280552928 shutdown_connections 2023-10-11T09:48:06.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.589+0000 7fac8721e700 1 -- 172.21.15.78:0/280552928 wait complete. 2023-10-11T09:48:06.598 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 50 2023-10-11T09:48:06.616 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:48:06.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.781+0000 7f2db0977700 1 Processor -- start 2023-10-11T09:48:06.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.781+0000 7f2db0977700 1 -- start start 2023-10-11T09:48:06.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.785+0000 7f2db0977700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f2dac156ce0 0x7f2dac160b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:06.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.785+0000 7f2db0977700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2dac0d1660 0x7f2dac0d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:06.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.785+0000 7f2db0977700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2dac0d28e0 0x7f2dac0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:06.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.785+0000 7f2db0977700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dac15dc10 con 0x7f2dac156ce0 2023-10-11T09:48:06.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.785+0000 7f2db0977700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dac160380 con 0x7f2dac0d28e0 2023-10-11T09:48:06.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.785+0000 7f2db0977700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dac15d930 con 0x7f2dac0d1660 2023-10-11T09:48:06.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.785+0000 7f2daad9d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f2dac156ce0 0x7f2dac160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:06.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.785+0000 7f2daad9d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f2dac156ce0 0x7f2dac160b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40674/0 (socket says 172.21.15.78:40674) 2023-10-11T09:48:06.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.785+0000 7f2daad9d700 1 -- 172.21.15.78:0/3538969709 learned_addr learned my addr 172.21.15.78:0/3538969709 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:06.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.785+0000 7f2daa59c700 1 --2- 172.21.15.78:0/3538969709 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2dac0d1660 0x7f2dac0d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:06.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.785+0000 7f2dab59e700 1 --2- 172.21.15.78:0/3538969709 >> v2:172.21.15.133:3300/0 conn(0x7f2dac0d28e0 0x7f2dac0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:06.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.785+0000 7f2daad9d700 1 -- 172.21.15.78:0/3538969709 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2dac0d1660 msgr2=0x7f2dac0d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:06.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.785+0000 7f2daad9d700 1 --2- 172.21.15.78:0/3538969709 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2dac0d1660 0x7f2dac0d0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:06.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.785+0000 7f2daad9d700 1 -- 172.21.15.78:0/3538969709 >> v2:172.21.15.133:3300/0 conn(0x7f2dac0d28e0 msgr2=0x7f2dac0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:06.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.785+0000 7f2daad9d700 1 --2- 172.21.15.78:0/3538969709 >> v2:172.21.15.133:3300/0 conn(0x7f2dac0d28e0 0x7f2dac0d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:06.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.785+0000 7f2daad9d700 1 -- 172.21.15.78:0/3538969709 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dac160620 con 0x7f2dac156ce0 2023-10-11T09:48:06.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.785+0000 7f2daad9d700 1 --2- 172.21.15.78:0/3538969709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dac156ce0 0x7f2dac160b40 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f2dac160550 tx=0x7f2d9c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf3110295faf662 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:06.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.785+0000 7f2da9d9b700 1 -- 172.21.15.78:0/3538969709 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c004230 con 0x7f2dac156ce0 2023-10-11T09:48:06.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.785+0000 7f2da9d9b700 1 -- 172.21.15.78:0/3538969709 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9c004390 con 0x7f2dac156ce0 2023-10-11T09:48:06.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.785+0000 7f2da9d9b700 1 -- 172.21.15.78:0/3538969709 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c0036c0 con 0x7f2dac156ce0 2023-10-11T09:48:06.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.789+0000 7f2db0977700 1 -- 172.21.15.78:0/3538969709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dac156ce0 msgr2=0x7f2dac160b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:06.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.789+0000 7f2db0977700 1 --2- 172.21.15.78:0/3538969709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dac156ce0 0x7f2dac160b40 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f2dac160550 tx=0x7f2d9c009bf0 comp rx=0 tx=0).stop 2023-10-11T09:48:06.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.789+0000 7f2db0977700 1 -- 172.21.15.78:0/3538969709 shutdown_connections 2023-10-11T09:48:06.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.789+0000 7f2db0977700 1 --2- 172.21.15.78:0/3538969709 >> v2:172.21.15.133:3300/0 conn(0x7f2dac0d28e0 0x7f2dac0d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:06.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.789+0000 7f2db0977700 1 --2- 172.21.15.78:0/3538969709 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2dac0d1660 0x7f2dac0d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:06.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.789+0000 7f2db0977700 1 --2- 172.21.15.78:0/3538969709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dac156ce0 0x7f2dac160b40 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:06.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.789+0000 7f2db0977700 1 -- 172.21.15.78:0/3538969709 >> 172.21.15.78:0/3538969709 conn(0x7f2dac06afb0 msgr2=0x7f2dac06b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:06.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.789+0000 7f2db0977700 1 -- 172.21.15.78:0/3538969709 shutdown_connections 2023-10-11T09:48:06.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.789+0000 7f2db0977700 1 -- 172.21.15.78:0/3538969709 wait complete. 2023-10-11T09:48:06.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.793+0000 7f2db0977700 1 Processor -- start 2023-10-11T09:48:06.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.793+0000 7f2db0977700 1 -- start start 2023-10-11T09:48:06.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.793+0000 7f2db0977700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dac0d1660 0x7f2dac157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:06.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.793+0000 7f2db0977700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2dac0d28e0 0x7f2dac1f5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:06.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.793+0000 7f2db0977700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dac156ce0 0x7f2dac1f5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:06.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.793+0000 7f2db0977700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dac1490f0 con 0x7f2dac156ce0 2023-10-11T09:48:06.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.793+0000 7f2db0977700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dac0c9300 con 0x7f2dac0d1660 2023-10-11T09:48:06.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.793+0000 7f2db0977700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dac149590 con 0x7f2dac0d28e0 2023-10-11T09:48:06.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.793+0000 7f2dab59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dac156ce0 0x7f2dac1f5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:06.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.793+0000 7f2dab59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dac156ce0 0x7f2dac1f5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40684/0 (socket says 172.21.15.78:40684) 2023-10-11T09:48:06.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.793+0000 7f2dab59e700 1 -- 172.21.15.78:0/4071090926 learned_addr learned my addr 172.21.15.78:0/4071090926 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:06.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.793+0000 7f2daa59c700 1 --2- 172.21.15.78:0/4071090926 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2dac0d28e0 0x7f2dac1f5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:06.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.793+0000 7f2daad9d700 1 --2- 172.21.15.78:0/4071090926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dac0d1660 0x7f2dac157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:06.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.793+0000 7f2dab59e700 1 -- 172.21.15.78:0/4071090926 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2dac0d28e0 msgr2=0x7f2dac1f5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:06.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.793+0000 7f2dab59e700 1 --2- 172.21.15.78:0/4071090926 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2dac0d28e0 0x7f2dac1f5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:06.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.793+0000 7f2dab59e700 1 -- 172.21.15.78:0/4071090926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dac0d1660 msgr2=0x7f2dac157230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:06.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.793+0000 7f2dab59e700 1 --2- 172.21.15.78:0/4071090926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dac0d1660 0x7f2dac157230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:06.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.793+0000 7f2dab59e700 1 -- 172.21.15.78:0/4071090926 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d9c009570 con 0x7f2dac156ce0 2023-10-11T09:48:06.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.797+0000 7f2dab59e700 1 --2- 172.21.15.78:0/4071090926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dac156ce0 0x7f2dac1f5870 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f2da400efc0 tx=0x7f2da400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:06.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.797+0000 7f2d937fe700 1 -- 172.21.15.78:0/4071090926 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da4009ea0 con 0x7f2dac156ce0 2023-10-11T09:48:06.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.797+0000 7f2d937fe700 1 -- 172.21.15.78:0/4071090926 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da400f040 con 0x7f2dac156ce0 2023-10-11T09:48:06.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.797+0000 7f2d937fe700 1 -- 172.21.15.78:0/4071090926 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da400ced0 con 0x7f2dac156ce0 2023-10-11T09:48:06.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.797+0000 7f2db0977700 1 -- 172.21.15.78:0/4071090926 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dac0c8410 con 0x7f2dac156ce0 2023-10-11T09:48:06.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.797+0000 7f2db0977700 1 -- 172.21.15.78:0/4071090926 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dac0c89d0 con 0x7f2dac156ce0 2023-10-11T09:48:06.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.797+0000 7f2db0977700 1 -- 172.21.15.78:0/4071090926 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dac1431d0 con 0x7f2dac156ce0 2023-10-11T09:48:06.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.801+0000 7f2d937fe700 1 -- 172.21.15.78:0/4071090926 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f2da4014480 con 0x7f2dac156ce0 2023-10-11T09:48:06.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.801+0000 7f2d937fe700 1 --2- 172.21.15.78:0/4071090926 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2d98065f00 0x7f2d980683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:06.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.801+0000 7f2d937fe700 1 -- 172.21.15.78:0/4071090926 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f2da40842d0 con 0x7f2dac156ce0 2023-10-11T09:48:06.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.801+0000 7f2daad9d700 1 --2- 172.21.15.78:0/4071090926 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2d98065f00 0x7f2d980683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:06.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.801+0000 7f2daad9d700 1 --2- 172.21.15.78:0/4071090926 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2d98065f00 0x7f2d980683b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2dac160550 tx=0x7f2d9c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:06.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.805+0000 7f2d937fe700 1 -- 172.21.15.78:0/4071090926 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f2dac1431d0 con 0x7f2dac156ce0 2023-10-11T09:48:06.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.985+0000 7f2db0977700 1 -- 172.21.15.78:0/4071090926 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2dac1431d0 con 0x7f2dac156ce0 2023-10-11T09:48:06.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.989+0000 7f2d937fe700 1 -- 172.21.15.78:0/4071090926 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+20052 (secure 0 0 0) 0x7f2dac1431d0 con 0x7f2dac156ce0 2023-10-11T09:48:06.998 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:06.999 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":70,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:46:00.671033+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:44:05.198742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-11T09:44:05.868171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data_extra","create_time":"2023-10-11T09:44:16.049941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6836/1666964283":"2023-10-12T09:46:00.635561+0000","172.21.15.78:6834/1666964283":"2023-10-12T09:46:00.635561+0000","172.21.15.78:6836/1819646091":"2023-10-12T09:45:54.334448+0000","172.21.15.78:6834/1819646091":"2023-10-12T09:45:54.334448+0000","172.21.15.78:6837/852594750":"2023-10-12T09:45:53.299952+0000","172.21.15.78:6835/852594750":"2023-10-12T09:45:53.299952+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:48:07.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.993+0000 7f2db0977700 1 -- 172.21.15.78:0/4071090926 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2d98065f00 msgr2=0x7f2d980683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:07.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.993+0000 7f2db0977700 1 --2- 172.21.15.78:0/4071090926 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2d98065f00 0x7f2d980683b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2dac160550 tx=0x7f2d9c008030 comp rx=0 tx=0).stop 2023-10-11T09:48:07.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.993+0000 7f2db0977700 1 -- 172.21.15.78:0/4071090926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dac156ce0 msgr2=0x7f2dac1f5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:07.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.993+0000 7f2db0977700 1 --2- 172.21.15.78:0/4071090926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dac156ce0 0x7f2dac1f5870 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f2da400efc0 tx=0x7f2da400c3a0 comp rx=0 tx=0).stop 2023-10-11T09:48:07.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.993+0000 7f2db0977700 1 -- 172.21.15.78:0/4071090926 shutdown_connections 2023-10-11T09:48:07.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.993+0000 7f2db0977700 1 --2- 172.21.15.78:0/4071090926 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2d98065f00 0x7f2d980683b0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:07.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.993+0000 7f2db0977700 1 --2- 172.21.15.78:0/4071090926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dac156ce0 0x7f2dac1f5870 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:07.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.993+0000 7f2db0977700 1 --2- 172.21.15.78:0/4071090926 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2dac0d28e0 0x7f2dac1f5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:07.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.993+0000 7f2db0977700 1 --2- 172.21.15.78:0/4071090926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dac0d1660 0x7f2dac157230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:07.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.993+0000 7f2db0977700 1 -- 172.21.15.78:0/4071090926 >> 172.21.15.78:0/4071090926 conn(0x7f2dac06afb0 msgr2=0x7f2dac156260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:07.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.993+0000 7f2db0977700 1 -- 172.21.15.78:0/4071090926 shutdown_connections 2023-10-11T09:48:07.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:06.993+0000 7f2db0977700 1 -- 172.21.15.78:0/4071090926 wait complete. 2023-10-11T09:48:07.024 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-11T09:48:07.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.193+0000 7f6d39bc3700 1 Processor -- start 2023-10-11T09:48:07.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.193+0000 7f6d39bc3700 1 -- start start 2023-10-11T09:48:07.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.197+0000 7f6d39bc3700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d3415a940 0x7f6d340ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:07.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.197+0000 7f6d39bc3700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d3415e010 con 0x7f6d340ca780 2023-10-11T09:48:07.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.197+0000 7f6d39bc3700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d340cab80 con 0x7f6d3415b8e0 2023-10-11T09:48:07.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.197+0000 7f6d39bc3700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3415e280 con 0x7f6d3415a940 2023-10-11T09:48:07.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.197+0000 7f6d393c2700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f6d340ca780 0x7f6d3413fcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:39980/0 (socket says 172.21.15.78:39980) 2023-10-11T09:48:07.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.197+0000 7f6d393c2700 1 -- 172.21.15.78:0/3422048370 learned_addr learned my addr 172.21.15.78:0/3422048370 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:07.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.197+0000 7f6d2bfff700 1 --2- 172.21.15.78:0/3422048370 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d3415a940 0x7f6d340ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:07.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.197+0000 7f6d2bfff700 1 -- 172.21.15.78:0/3422048370 >> v1:172.21.15.133:6789/0 conn(0x7f6d3415b8e0 legacy=0x7f6d3406d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:07.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.197+0000 7f6d2bfff700 1 -- 172.21.15.78:0/3422048370 >> v1:172.21.15.78:6789/0 conn(0x7f6d340ca780 legacy=0x7f6d3413fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:07.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.197+0000 7f6d2b7fe700 1 -- 172.21.15.78:0/3422048370 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2651635521 0 0) 0x7f6d3415e010 con 0x7f6d340ca780 2023-10-11T09:48:07.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.197+0000 7f6d2bfff700 1 -- 172.21.15.78:0/3422048370 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d3415e560 con 0x7f6d3415a940 2023-10-11T09:48:07.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.197+0000 7f6d2bfff700 1 --2- 172.21.15.78:0/3422048370 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d3415a940 0x7f6d340ca240 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6d24009880 tx=0x7f6d24009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=521921d1979355de server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:07.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.197+0000 7f6d2b7fe700 1 -- 172.21.15.78:0/3422048370 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d24011830 con 0x7f6d3415a940 2023-10-11T09:48:07.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.197+0000 7f6d2b7fe700 1 -- 172.21.15.78:0/3422048370 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d24011990 con 0x7f6d3415a940 2023-10-11T09:48:07.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.197+0000 7f6d2b7fe700 1 -- 172.21.15.78:0/3422048370 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d24011d20 con 0x7f6d3415a940 2023-10-11T09:48:07.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.201+0000 7f6d39bc3700 1 -- 172.21.15.78:0/3422048370 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d3415a940 msgr2=0x7f6d340ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:07.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.201+0000 7f6d39bc3700 1 --2- 172.21.15.78:0/3422048370 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d3415a940 0x7f6d340ca240 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6d24009880 tx=0x7f6d24009b90 comp rx=0 tx=0).stop 2023-10-11T09:48:07.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.201+0000 7f6d39bc3700 1 -- 172.21.15.78:0/3422048370 shutdown_connections 2023-10-11T09:48:07.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.201+0000 7f6d39bc3700 1 --2- 172.21.15.78:0/3422048370 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d3415a940 0x7f6d340ca240 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:07.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.201+0000 7f6d39bc3700 1 -- 172.21.15.78:0/3422048370 >> 172.21.15.78:0/3422048370 conn(0x7f6d340cea60 msgr2=0x7f6d340cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:07.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.201+0000 7f6d39bc3700 1 -- 172.21.15.78:0/3422048370 shutdown_connections 2023-10-11T09:48:07.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.201+0000 7f6d39bc3700 1 -- 172.21.15.78:0/3422048370 wait complete. 2023-10-11T09:48:07.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.205+0000 7f6d39bc3700 1 Processor -- start 2023-10-11T09:48:07.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.205+0000 7f6d39bc3700 1 -- start start 2023-10-11T09:48:07.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.205+0000 7f6d39bc3700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d340ca780 0x7f6d34159220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:07.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.205+0000 7f6d39bc3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d3415a940 0x7f6d340cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:07.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.205+0000 7f6d39bc3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d3415b8e0 0x7f6d340c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:07.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.205+0000 7f6d39bc3700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d341412f0 con 0x7f6d3415a940 2023-10-11T09:48:07.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.205+0000 7f6d39bc3700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d34141590 con 0x7f6d3415b8e0 2023-10-11T09:48:07.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.205+0000 7f6d39bc3700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d34141870 con 0x7f6d340ca780 2023-10-11T09:48:07.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.205+0000 7f6d2bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d3415a940 0x7f6d340cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:07.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.205+0000 7f6d2bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d3415a940 0x7f6d340cdeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40692/0 (socket says 172.21.15.78:40692) 2023-10-11T09:48:07.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.205+0000 7f6d2bfff700 1 -- 172.21.15.78:0/2343233496 learned_addr learned my addr 172.21.15.78:0/2343233496 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:07.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.205+0000 7f6d393c2700 1 --2- 172.21.15.78:0/2343233496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d3415b8e0 0x7f6d340c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:07.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.205+0000 7f6d38bc1700 1 --2- 172.21.15.78:0/2343233496 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d340ca780 0x7f6d34159220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:07.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.205+0000 7f6d2bfff700 1 -- 172.21.15.78:0/2343233496 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d340ca780 msgr2=0x7f6d34159220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:07.218 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.205+0000 7f6d2bfff700 1 --2- 172.21.15.78:0/2343233496 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d340ca780 0x7f6d34159220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:07.218 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.205+0000 7f6d2bfff700 1 -- 172.21.15.78:0/2343233496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d3415b8e0 msgr2=0x7f6d340c7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:07.218 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.205+0000 7f6d2bfff700 1 --2- 172.21.15.78:0/2343233496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d3415b8e0 0x7f6d340c7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:07.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.205+0000 7f6d2bfff700 1 -- 172.21.15.78:0/2343233496 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d240094b0 con 0x7f6d3415a940 2023-10-11T09:48:07.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.205+0000 7f6d38bc1700 1 --2- 172.21.15.78:0/2343233496 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d340ca780 0x7f6d34159220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:48:07.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.205+0000 7f6d393c2700 1 --2- 172.21.15.78:0/2343233496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d3415b8e0 0x7f6d340c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:48:07.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.205+0000 7f6d2bfff700 1 --2- 172.21.15.78:0/2343233496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d3415a940 0x7f6d340cdeb0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f6d24014040 tx=0x7f6d24004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:07.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.205+0000 7f6d297fa700 1 -- 172.21.15.78:0/2343233496 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2400bc10 con 0x7f6d3415a940 2023-10-11T09:48:07.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.205+0000 7f6d297fa700 1 -- 172.21.15.78:0/2343233496 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d2401e070 con 0x7f6d3415a940 2023-10-11T09:48:07.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.205+0000 7f6d39bc3700 1 -- 172.21.15.78:0/2343233496 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d34141b50 con 0x7f6d3415a940 2023-10-11T09:48:07.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.209+0000 7f6d297fa700 1 -- 172.21.15.78:0/2343233496 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2401a420 con 0x7f6d3415a940 2023-10-11T09:48:07.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.209+0000 7f6d39bc3700 1 -- 172.21.15.78:0/2343233496 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d34142110 con 0x7f6d3415a940 2023-10-11T09:48:07.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.209+0000 7f6d39bc3700 1 -- 172.21.15.78:0/2343233496 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d34065d40 con 0x7f6d3415a940 2023-10-11T09:48:07.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.209+0000 7f6d297fa700 1 -- 172.21.15.78:0/2343233496 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f6d2401a580 con 0x7f6d3415a940 2023-10-11T09:48:07.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.213+0000 7f6d297fa700 1 --2- 172.21.15.78:0/2343233496 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6d20065ed0 0x7f6d20068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:07.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.213+0000 7f6d297fa700 1 -- 172.21.15.78:0/2343233496 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f6d24084ce0 con 0x7f6d3415a940 2023-10-11T09:48:07.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.213+0000 7f6d38bc1700 1 --2- 172.21.15.78:0/2343233496 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6d20065ed0 0x7f6d20068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:07.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.213+0000 7f6d38bc1700 1 --2- 172.21.15.78:0/2343233496 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6d20065ed0 0x7f6d20068380 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6d30000f60 tx=0x7f6d30009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:07.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.217+0000 7f6d297fa700 1 -- 172.21.15.78:0/2343233496 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f6d24052090 con 0x7f6d3415a940 2023-10-11T09:48:07.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.433+0000 7f6d39bc3700 1 -- 172.21.15.78:0/2343233496 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6d3413ea80 con 0x7f6d3415a940 2023-10-11T09:48:07.445 INFO:tasks.ceph.mon.a.smithi078.stderr:2023-10-11T09:48:07.437+0000 7f90131f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-11T09:48:07.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.453+0000 7f6d297fa700 1 -- 172.21.15.78:0/2343233496 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v51) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6d24015090 con 0x7f6d3415a940 2023-10-11T09:48:07.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.457+0000 7f6d39bc3700 1 -- 172.21.15.78:0/2343233496 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6d20065ed0 msgr2=0x7f6d20068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:07.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.457+0000 7f6d39bc3700 1 --2- 172.21.15.78:0/2343233496 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6d20065ed0 0x7f6d20068380 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6d30000f60 tx=0x7f6d30009040 comp rx=0 tx=0).stop 2023-10-11T09:48:07.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.457+0000 7f6d39bc3700 1 -- 172.21.15.78:0/2343233496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d3415a940 msgr2=0x7f6d340cdeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:07.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.457+0000 7f6d39bc3700 1 --2- 172.21.15.78:0/2343233496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d3415a940 0x7f6d340cdeb0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f6d24014040 tx=0x7f6d24004770 comp rx=0 tx=0).stop 2023-10-11T09:48:07.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.457+0000 7f6d39bc3700 1 -- 172.21.15.78:0/2343233496 shutdown_connections 2023-10-11T09:48:07.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.457+0000 7f6d39bc3700 1 --2- 172.21.15.78:0/2343233496 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6d20065ed0 0x7f6d20068380 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:07.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.457+0000 7f6d39bc3700 1 --2- 172.21.15.78:0/2343233496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d3415b8e0 0x7f6d340c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:07.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.457+0000 7f6d39bc3700 1 --2- 172.21.15.78:0/2343233496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d3415a940 0x7f6d340cdeb0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:07.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.457+0000 7f6d39bc3700 1 --2- 172.21.15.78:0/2343233496 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d340ca780 0x7f6d34159220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:07.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.457+0000 7f6d39bc3700 1 -- 172.21.15.78:0/2343233496 >> 172.21.15.78:0/2343233496 conn(0x7f6d340cea60 msgr2=0x7f6d340c9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:07.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.457+0000 7f6d39bc3700 1 -- 172.21.15.78:0/2343233496 shutdown_connections 2023-10-11T09:48:07.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.457+0000 7f6d39bc3700 1 -- 172.21.15.78:0/2343233496 wait complete. 2023-10-11T09:48:07.467 INFO:teuthology.orchestra.run.smithi078.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-11T09:48:07.476 INFO:tasks.ceph.mds.a.smithi078.stderr: -9452> 2023-10-11T09:46:05.655+0000 7f293682d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-11T09:48:07.477 INFO:tasks.ceph.mds.a.smithi078.stderr: -9055> 2023-10-11T09:46:08.047+0000 7f2933026700 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2023-10-11T09:48:07.477 INFO:tasks.ceph.mds.a.smithi078.stderr: -8897> 2023-10-11T09:46:08.047+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /subdir/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55e01e8fe100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000000:[<0x1/subdir v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-10-11T09:48:07.479 INFO:tasks.ceph.mds.a.smithi078.stderr: -8515> 2023-10-11T09:46:08.051+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f8 [2,head] /subdir/3 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea1c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001f8:[<0x10000000000/3 v42>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x100000001f8:[<0x10000000000/3 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:48:07.479 INFO:tasks.ceph.mds.a.smithi078.stderr: -8490> 2023-10-11T09:46:08.051+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fd [2,head] /subdir/8 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea1a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001fd:[<0x10000000000/8 v82>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x100000001fd:[<0x10000000000/8 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:48:07.479 INFO:tasks.ceph.mds.a.smithi078.stderr: -8282> 2023-10-11T09:46:08.051+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020a [2,head] /subdir/21 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea22000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x1000000020a:[<0x10000000000/21 v188>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x1000000020a:[<0x10000000000/21 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:48:07.480 INFO:tasks.ceph.mds.a.smithi078.stderr: -8259> 2023-10-11T09:46:08.051+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000208 [2,head] /subdir/19 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea23080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000208:[<0x10000000000/19 v170>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x10000000208:[<0x10000000000/19 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:48:07.480 INFO:tasks.ceph.mds.a.smithi078.stderr: -8237> 2023-10-11T09:46:08.051+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fe [2,head] /subdir/9 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea1a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001fe:[<0x10000000000/9 v90>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x100000001fe:[<0x10000000000/9 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:48:07.480 INFO:tasks.ceph.mds.a.smithi078.stderr: -8211> 2023-10-11T09:46:08.051+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000209 [2,head] /subdir/20 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea22580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000209:[<0x10000000000/20 v178>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x10000000209:[<0x10000000000/20 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:48:07.480 INFO:tasks.ceph.mds.a.smithi078.stderr: -8189> 2023-10-11T09:46:08.051+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000201 [2,head] /subdir/12 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea25700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000201:[<0x10000000000/12 v116>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x10000000201:[<0x10000000000/12 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:48:07.480 INFO:tasks.ceph.mds.a.smithi078.stderr: -8166> 2023-10-11T09:46:08.051+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000204 [2,head] /subdir/15 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea24680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000204:[<0x10000000000/15 v140>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x10000000204:[<0x10000000000/15 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:48:07.481 INFO:tasks.ceph.mds.a.smithi078.stderr: -8144> 2023-10-11T09:46:08.051+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001ff [2,head] /subdir/10 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea2c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001ff:[<0x10000000000/10 v100>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x100000001ff:[<0x10000000000/10 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:48:07.481 INFO:tasks.ceph.mds.a.smithi078.stderr: -8120> 2023-10-11T09:46:08.055+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f7 [2,head] /subdir/2 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea1c680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001f7:[<0x10000000000/2 v34>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x100000001f7:[<0x10000000000/2 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:48:07.481 INFO:tasks.ceph.mds.a.smithi078.stderr: -8096> 2023-10-11T09:46:08.055+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000200 [2,head] /subdir/11 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea2c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000200:[<0x10000000000/11 v110>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x10000000200:[<0x10000000000/11 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:48:07.481 INFO:tasks.ceph.mds.a.smithi078.stderr: -8074> 2023-10-11T09:46:08.055+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f6 [2,head] /subdir/1 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea22b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001f6:[<0x10000000000/1 v30>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x100000001f6:[<0x10000000000/1 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:48:07.482 INFO:tasks.ceph.mds.a.smithi078.stderr: -8052> 2023-10-11T09:46:08.055+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f9 [2,head] /subdir/4 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea1bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001f9:[<0x10000000000/4 v52>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x100000001f9:[<0x10000000000/4 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:48:07.482 INFO:tasks.ceph.mds.a.smithi078.stderr: -8028> 2023-10-11T09:46:08.055+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020d [2,head] /subdir/24 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea1cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x1000000020d:[<0x10000000000/24 v202>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x1000000020d:[<0x10000000000/24 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:48:07.482 INFO:tasks.ceph.mds.a.smithi078.stderr: -8006> 2023-10-11T09:46:08.055+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000205 [2,head] /subdir/16 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea24100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000205:[<0x10000000000/16 v146>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x10000000205:[<0x10000000000/16 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:48:07.482 INFO:tasks.ceph.mds.a.smithi078.stderr: -7982> 2023-10-11T09:46:08.055+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020b [2,head] /subdir/22 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea1d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x1000000020b:[<0x10000000000/22 v194>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x1000000020b:[<0x10000000000/22 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:48:07.482 INFO:tasks.ceph.mds.a.smithi078.stderr: -7960> 2023-10-11T09:46:08.055+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/0 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea2cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000001:[<0x10000000000/0 v16>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x10000000001:[<0x10000000000/0 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:48:07.483 INFO:tasks.ceph.mds.a.smithi078.stderr: -7937> 2023-10-11T09:46:08.055+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fa [2,head] /subdir/5 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea1b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001fa:[<0x10000000000/5 v58>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x100000001fa:[<0x10000000000/5 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:48:07.483 INFO:tasks.ceph.mds.a.smithi078.stderr: -7914> 2023-10-11T09:46:08.055+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000203 [2,head] /subdir/14 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea24c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000203:[<0x10000000000/14 v130>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x10000000203:[<0x10000000000/14 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:48:07.483 INFO:tasks.ceph.mds.a.smithi078.stderr: -7890> 2023-10-11T09:46:08.055+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000207 [2,head] /subdir/18 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea23600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000207:[<0x10000000000/18 v164>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x10000000207:[<0x10000000000/18 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:48:07.483 INFO:tasks.ceph.mds.a.smithi078.stderr: -7867> 2023-10-11T09:46:08.055+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000202 [2,head] /subdir/13 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea25180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000202:[<0x10000000000/13 v122>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x10000000202:[<0x10000000000/13 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:48:07.483 INFO:tasks.ceph.mds.a.smithi078.stderr: -7843> 2023-10-11T09:46:08.055+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020c [2,head] /subdir/23 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea1d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x1000000020c:[<0x10000000000/23 v200>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x1000000020c:[<0x10000000000/23 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:48:07.484 INFO:tasks.ceph.mds.a.smithi078.stderr: -7827> 2023-10-11T09:46:08.055+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fc [2,head] /subdir/7 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea1ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001fc:[<0x10000000000/7 v76>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x100000001fc:[<0x10000000000/7 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:48:07.484 INFO:tasks.ceph.mds.a.smithi078.stderr: -7810> 2023-10-11T09:46:08.055+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000206 [2,head] /subdir/17 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea23b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000206:[<0x10000000000/17 v154>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x10000000206:[<0x10000000000/17 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:48:07.484 INFO:tasks.ceph.mds.a.smithi078.stderr: -7793> 2023-10-11T09:46:08.055+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fb [2,head] /subdir/6 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e01ea1b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001fb:[<0x10000000000/6 v66>,<0x1/subdir v212>]//[9]","memoryvalue":"(10)0x100000001fb:[<0x10000000000/6 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:48:07.484 INFO:tasks.ceph.mds.a.smithi078.stderr: -7698> 2023-10-11T09:46:08.059+0000 7f2933026700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v5 ap=5 snaprealm=0x55e01e901200 DIRTYPARENT f(v1 1=0+1) n(v0 rc2023-10-11T09:46:07.751414+0000 1=0+1)/n() (idft sync r=1) (inest sync r=1) (ifile sync->mix r=1) (iversion lock) caps={6960=pLsXs/-@3} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x55e01e8fd600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=0+1)","ondisk_value.rstat":"n(v0 2=0+2)","memory_value.dirstat":"f(v1 1=0+1)","memory_value.rstat":"n(v0 rc2023-10-11T09:46:07.751414+0000 1=0+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-10-11T09:48:07.486 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-11T09:48:07.535 INFO:tasks.ceph.mds.a.smithi078.stdout:starting mds.a at 2023-10-11T09:48:07.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.653+0000 7f5a9e4dc700 1 Processor -- start 2023-10-11T09:48:07.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.653+0000 7f5a9e4dc700 1 -- start start 2023-10-11T09:48:07.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.657+0000 7f5a9e4dc700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f5a980b3610 0x7f5a980d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:07.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.657+0000 7f5a9e4dc700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5a980b3a90 0x7f5a980b85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:07.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.657+0000 7f5a9e4dc700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a980b47f0 con 0x7f5a980b3240 2023-10-11T09:48:07.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.657+0000 7f5a9e4dc700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a980b8130 con 0x7f5a980b3610 2023-10-11T09:48:07.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.657+0000 7f5a9e4dc700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a980b41e0 con 0x7f5a980b3a90 2023-10-11T09:48:07.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.657+0000 7f5a9ccd9700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f5a980b3610 0x7f5a980d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:07.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.657+0000 7f5a9dcdb700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5a980b3a90 0x7f5a980b85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:07.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.657+0000 7f5a9ccd9700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f5a980b3610 0x7f5a980d29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40716/0 (socket says 172.21.15.78:40716) 2023-10-11T09:48:07.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.657+0000 7f5a9ccd9700 1 -- 172.21.15.78:0/3669920482 learned_addr learned my addr 172.21.15.78:0/3669920482 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:07.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.657+0000 7f5a9ccd9700 1 -- 172.21.15.78:0/3669920482 >> v2:172.21.15.133:3300/0 conn(0x7f5a980b3a90 msgr2=0x7f5a980b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:07.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.657+0000 7f5a9ccd9700 1 --2- 172.21.15.78:0/3669920482 >> v2:172.21.15.133:3300/0 conn(0x7f5a980b3a90 0x7f5a980b85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:07.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.657+0000 7f5a9ccd9700 1 -- 172.21.15.78:0/3669920482 >> v1:172.21.15.133:6789/0 conn(0x7f5a980b3240 legacy=0x7f5a9815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:07.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.657+0000 7f5a9ccd9700 1 -- 172.21.15.78:0/3669920482 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a980b7800 con 0x7f5a980b3610 2023-10-11T09:48:07.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.657+0000 7f5a9ccd9700 1 --2- 172.21.15.78:0/3669920482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a980b3610 0x7f5a980d29d0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f5a98072b40 tx=0x7f5a88009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321140b94d1c58c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:07.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.657+0000 7f5a8ffff700 1 -- 172.21.15.78:0/3669920482 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a88011830 con 0x7f5a980b3610 2023-10-11T09:48:07.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.657+0000 7f5a8ffff700 1 -- 172.21.15.78:0/3669920482 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a88011990 con 0x7f5a980b3610 2023-10-11T09:48:07.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.657+0000 7f5a8ffff700 1 -- 172.21.15.78:0/3669920482 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a88011d20 con 0x7f5a980b3610 2023-10-11T09:48:07.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.657+0000 7f5a9e4dc700 1 -- 172.21.15.78:0/3669920482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a980b3610 msgr2=0x7f5a980d29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:07.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.657+0000 7f5a9e4dc700 1 --2- 172.21.15.78:0/3669920482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a980b3610 0x7f5a980d29d0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f5a98072b40 tx=0x7f5a88009bf0 comp rx=0 tx=0).stop 2023-10-11T09:48:07.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.661+0000 7f5a9e4dc700 1 -- 172.21.15.78:0/3669920482 shutdown_connections 2023-10-11T09:48:07.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.661+0000 7f5a9e4dc700 1 --2- 172.21.15.78:0/3669920482 >> v2:172.21.15.133:3300/0 conn(0x7f5a980b3a90 0x7f5a980b85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:07.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.661+0000 7f5a9e4dc700 1 --2- 172.21.15.78:0/3669920482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a980b3610 0x7f5a980d29d0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:07.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.661+0000 7f5a9e4dc700 1 -- 172.21.15.78:0/3669920482 >> 172.21.15.78:0/3669920482 conn(0x7f5a9809c260 msgr2=0x7f5a980d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:07.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.661+0000 7f5a9e4dc700 1 -- 172.21.15.78:0/3669920482 shutdown_connections 2023-10-11T09:48:07.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.661+0000 7f5a9e4dc700 1 -- 172.21.15.78:0/3669920482 wait complete. 2023-10-11T09:48:07.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.665+0000 7f5a9e4dc700 1 Processor -- start 2023-10-11T09:48:07.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.665+0000 7f5a9e4dc700 1 -- start start 2023-10-11T09:48:07.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.665+0000 7f5a9e4dc700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5a980b3240 0x7f5a9815faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:07.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.665+0000 7f5a9e4dc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a980b3610 0x7f5a98149400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:07.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.665+0000 7f5a9e4dc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a980b3a90 0x7f5a980c8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:07.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.665+0000 7f5a9e4dc700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9815e810 con 0x7f5a980b3a90 2023-10-11T09:48:07.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.665+0000 7f5a9e4dc700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a98149940 con 0x7f5a980b3610 2023-10-11T09:48:07.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.665+0000 7f5a9e4dc700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9815ea00 con 0x7f5a980b3240 2023-10-11T09:48:07.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.665+0000 7f5a9dcdb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a980b3a90 0x7f5a980c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:07.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.665+0000 7f5a9dcdb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a980b3a90 0x7f5a980c8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40726/0 (socket says 172.21.15.78:40726) 2023-10-11T09:48:07.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.665+0000 7f5a9dcdb700 1 -- 172.21.15.78:0/449554981 learned_addr learned my addr 172.21.15.78:0/449554981 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:07.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.665+0000 7f5a9ccd9700 1 --2- 172.21.15.78:0/449554981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a980b3610 0x7f5a98149400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:07.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.665+0000 7f5a9d4da700 1 --2- 172.21.15.78:0/449554981 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5a980b3240 0x7f5a9815faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:07.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.665+0000 7f5a9dcdb700 1 -- 172.21.15.78:0/449554981 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5a980b3240 msgr2=0x7f5a9815faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:07.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.665+0000 7f5a9dcdb700 1 --2- 172.21.15.78:0/449554981 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5a980b3240 0x7f5a9815faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:07.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.665+0000 7f5a9dcdb700 1 -- 172.21.15.78:0/449554981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a980b3610 msgr2=0x7f5a98149400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:07.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.665+0000 7f5a9dcdb700 1 --2- 172.21.15.78:0/449554981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a980b3610 0x7f5a98149400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:07.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.665+0000 7f5a9dcdb700 1 -- 172.21.15.78:0/449554981 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a88009570 con 0x7f5a980b3a90 2023-10-11T09:48:07.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.665+0000 7f5a9d4da700 1 --2- 172.21.15.78:0/449554981 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5a980b3240 0x7f5a9815faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:48:07.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.669+0000 7f5a9dcdb700 1 --2- 172.21.15.78:0/449554981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a980b3a90 0x7f5a980c8930 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f5a9000efc0 tx=0x7f5a9000c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:07.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.669+0000 7f5a8dffb700 1 -- 172.21.15.78:0/449554981 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9000cb80 con 0x7f5a980b3a90 2023-10-11T09:48:07.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.669+0000 7f5a8dffb700 1 -- 172.21.15.78:0/449554981 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a9000f040 con 0x7f5a980b3a90 2023-10-11T09:48:07.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.669+0000 7f5a8dffb700 1 -- 172.21.15.78:0/449554981 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a90014480 con 0x7f5a980b3a90 2023-10-11T09:48:07.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.669+0000 7f5a9e4dc700 1 -- 172.21.15.78:0/449554981 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a9815ece0 con 0x7f5a980b3a90 2023-10-11T09:48:07.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.669+0000 7f5a9e4dc700 1 -- 172.21.15.78:0/449554981 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a9815f2a0 con 0x7f5a980b3a90 2023-10-11T09:48:07.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.669+0000 7f5a9e4dc700 1 -- 172.21.15.78:0/449554981 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a98065d40 con 0x7f5a980b3a90 2023-10-11T09:48:07.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.669+0000 7f5a8dffb700 1 -- 172.21.15.78:0/449554981 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f5a900145e0 con 0x7f5a980b3a90 2023-10-11T09:48:07.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.673+0000 7f5a8dffb700 1 --2- 172.21.15.78:0/449554981 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5a7c065ed0 0x7f5a7c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:07.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.673+0000 7f5a8dffb700 1 -- 172.21.15.78:0/449554981 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6810+0+0 (secure 0 0 0) 0x7f5a90084340 con 0x7f5a980b3a90 2023-10-11T09:48:07.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.673+0000 7f5a9d4da700 1 --2- 172.21.15.78:0/449554981 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5a7c065ed0 0x7f5a7c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:07.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.673+0000 7f5a9d4da700 1 --2- 172.21.15.78:0/449554981 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5a7c065ed0 0x7f5a7c068380 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5a94000f60 tx=0x7f5a94009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:07.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.681+0000 7f5a8dffb700 1 -- 172.21.15.78:0/449554981 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f5a90051610 con 0x7f5a980b3a90 2023-10-11T09:48:07.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:07.897+0000 7f5a9e4dc700 1 -- 172.21.15.78:0/449554981 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5a980b71e0 con 0x7f5a980b3a90 2023-10-11T09:48:08.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.461+0000 7f5a8dffb700 1 -- 172.21.15.78:0/449554981 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v52) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5a90054c30 con 0x7f5a980b3a90 2023-10-11T09:48:08.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.465+0000 7f5a9e4dc700 1 -- 172.21.15.78:0/449554981 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5a7c065ed0 msgr2=0x7f5a7c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:08.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.465+0000 7f5a9e4dc700 1 --2- 172.21.15.78:0/449554981 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5a7c065ed0 0x7f5a7c068380 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5a94000f60 tx=0x7f5a94009040 comp rx=0 tx=0).stop 2023-10-11T09:48:08.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.465+0000 7f5a9e4dc700 1 -- 172.21.15.78:0/449554981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a980b3a90 msgr2=0x7f5a980c8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:08.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.465+0000 7f5a9e4dc700 1 --2- 172.21.15.78:0/449554981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a980b3a90 0x7f5a980c8930 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f5a9000efc0 tx=0x7f5a9000c0c0 comp rx=0 tx=0).stop 2023-10-11T09:48:08.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.465+0000 7f5a9e4dc700 1 -- 172.21.15.78:0/449554981 shutdown_connections 2023-10-11T09:48:08.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.465+0000 7f5a9e4dc700 1 --2- 172.21.15.78:0/449554981 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5a7c065ed0 0x7f5a7c068380 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:08.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.465+0000 7f5a9e4dc700 1 --2- 172.21.15.78:0/449554981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a980b3a90 0x7f5a980c8930 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:08.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.465+0000 7f5a9e4dc700 1 --2- 172.21.15.78:0/449554981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a980b3610 0x7f5a98149400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:08.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.465+0000 7f5a9e4dc700 1 --2- 172.21.15.78:0/449554981 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5a980b3240 0x7f5a9815faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:08.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.465+0000 7f5a9e4dc700 1 -- 172.21.15.78:0/449554981 >> 172.21.15.78:0/449554981 conn(0x7f5a9809c260 msgr2=0x7f5a980d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:08.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.465+0000 7f5a9e4dc700 1 -- 172.21.15.78:0/449554981 shutdown_connections 2023-10-11T09:48:08.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.465+0000 7f5a9e4dc700 1 -- 172.21.15.78:0/449554981 wait complete. 2023-10-11T09:48:08.496 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-11T09:48:08.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.673+0000 7f9eb9197700 1 Processor -- start 2023-10-11T09:48:08.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.673+0000 7f9eb9197700 1 -- start start 2023-10-11T09:48:08.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.677+0000 7f9eb9197700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9eb415a0f0 0x7f9eb40cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:08.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.677+0000 7f9eb9197700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f9eb40b52e0 0x7f9eb41608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:08.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.677+0000 7f9eb9197700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eb40d2bd0 con 0x7f9eb4160de0 2023-10-11T09:48:08.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.677+0000 7f9eb9197700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eb40cf630 con 0x7f9eb40b52e0 2023-10-11T09:48:08.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.677+0000 7f9eb9197700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb4160410 con 0x7f9eb415a0f0 2023-10-11T09:48:08.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.677+0000 7f9eb37fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f9eb40b52e0 0x7f9eb41608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:08.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.677+0000 7f9eb37fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f9eb40b52e0 0x7f9eb41608a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40742/0 (socket says 172.21.15.78:40742) 2023-10-11T09:48:08.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.677+0000 7f9eb37fe700 1 -- 172.21.15.78:0/3807151376 learned_addr learned my addr 172.21.15.78:0/3807151376 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:08.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.677+0000 7f9eb3fff700 1 --2- 172.21.15.78:0/3807151376 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9eb415a0f0 0x7f9eb40cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:08.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.677+0000 7f9eb37fe700 1 -- 172.21.15.78:0/3807151376 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9eb415a0f0 msgr2=0x7f9eb40cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:08.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.677+0000 7f9eb37fe700 1 --2- 172.21.15.78:0/3807151376 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9eb415a0f0 0x7f9eb40cefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:08.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.677+0000 7f9eb37fe700 1 -- 172.21.15.78:0/3807151376 >> v1:172.21.15.133:6789/0 conn(0x7f9eb4160de0 legacy=0x7f9eb415cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:08.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.677+0000 7f9eb37fe700 1 -- 172.21.15.78:0/3807151376 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb40cdb30 con 0x7f9eb40b52e0 2023-10-11T09:48:08.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.677+0000 7f9eb3fff700 1 --2- 172.21.15.78:0/3807151376 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9eb415a0f0 0x7f9eb40cefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:48:08.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.677+0000 7f9eb37fe700 1 --2- 172.21.15.78:0/3807151376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eb40b52e0 0x7f9eb41608a0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8009910 tx=0x7f9ea8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e4b3364fd3cd23 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:08.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.677+0000 7f9eb2ffd700 1 -- 172.21.15.78:0/3807151376 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea800ba60 con 0x7f9eb40b52e0 2023-10-11T09:48:08.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.677+0000 7f9eb2ffd700 1 -- 172.21.15.78:0/3807151376 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea800bbc0 con 0x7f9eb40b52e0 2023-10-11T09:48:08.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.677+0000 7f9eb2ffd700 1 -- 172.21.15.78:0/3807151376 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8009e60 con 0x7f9eb40b52e0 2023-10-11T09:48:08.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.677+0000 7f9eb9197700 1 -- 172.21.15.78:0/3807151376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eb40b52e0 msgr2=0x7f9eb41608a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:08.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.681+0000 7f9eb9197700 1 --2- 172.21.15.78:0/3807151376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eb40b52e0 0x7f9eb41608a0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8009910 tx=0x7f9ea8009c20 comp rx=0 tx=0).stop 2023-10-11T09:48:08.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.681+0000 7f9eb9197700 1 -- 172.21.15.78:0/3807151376 shutdown_connections 2023-10-11T09:48:08.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.681+0000 7f9eb9197700 1 --2- 172.21.15.78:0/3807151376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eb40b52e0 0x7f9eb41608a0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:08.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.681+0000 7f9eb9197700 1 --2- 172.21.15.78:0/3807151376 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9eb415a0f0 0x7f9eb40cefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:08.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.681+0000 7f9eb9197700 1 -- 172.21.15.78:0/3807151376 >> 172.21.15.78:0/3807151376 conn(0x7f9eb406afd0 msgr2=0x7f9eb406b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:08.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.681+0000 7f9eb9197700 1 -- 172.21.15.78:0/3807151376 shutdown_connections 2023-10-11T09:48:08.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.681+0000 7f9eb9197700 1 -- 172.21.15.78:0/3807151376 wait complete. 2023-10-11T09:48:08.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.681+0000 7f9eb9197700 1 Processor -- start 2023-10-11T09:48:08.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.681+0000 7f9eb9197700 1 -- start start 2023-10-11T09:48:08.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.681+0000 7f9eb9197700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9eb40b52e0 0x7f9eb41e6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:08.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.681+0000 7f9eb9197700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9eb415a0f0 0x7f9eb40c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:08.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.681+0000 7f9eb9197700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eb4160de0 0x7f9eb40cca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:08.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.681+0000 7f9eb9197700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb40c9ae0 con 0x7f9eb4160de0 2023-10-11T09:48:08.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.681+0000 7f9eb9197700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb40ccfa0 con 0x7f9eb40b52e0 2023-10-11T09:48:08.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.681+0000 7f9eb9197700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb41e7670 con 0x7f9eb415a0f0 2023-10-11T09:48:08.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.685+0000 7f9eb8996700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eb4160de0 0x7f9eb40cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:08.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.685+0000 7f9eb8996700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eb4160de0 0x7f9eb40cca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40756/0 (socket says 172.21.15.78:40756) 2023-10-11T09:48:08.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.685+0000 7f9eb8996700 1 -- 172.21.15.78:0/2186264931 learned_addr learned my addr 172.21.15.78:0/2186264931 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:08.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.685+0000 7f9eb37fe700 1 --2- 172.21.15.78:0/2186264931 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9eb415a0f0 0x7f9eb40c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:08.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.685+0000 7f9eb3fff700 1 --2- 172.21.15.78:0/2186264931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9eb40b52e0 0x7f9eb41e6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:08.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.685+0000 7f9eb8996700 1 -- 172.21.15.78:0/2186264931 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9eb415a0f0 msgr2=0x7f9eb40c95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:08.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.685+0000 7f9eb8996700 1 --2- 172.21.15.78:0/2186264931 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9eb415a0f0 0x7f9eb40c95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:08.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.685+0000 7f9eb8996700 1 -- 172.21.15.78:0/2186264931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9eb40b52e0 msgr2=0x7f9eb41e6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:08.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.685+0000 7f9eb8996700 1 --2- 172.21.15.78:0/2186264931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9eb40b52e0 0x7f9eb41e6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:08.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.685+0000 7f9eb8996700 1 -- 172.21.15.78:0/2186264931 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea8009570 con 0x7f9eb4160de0 2023-10-11T09:48:08.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.685+0000 7f9eb8996700 1 --2- 172.21.15.78:0/2186264931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eb4160de0 0x7f9eb40cca60 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f9eac00c480 tx=0x7f9eac00c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:08.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.685+0000 7f9eb0ff9700 1 -- 172.21.15.78:0/2186264931 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac0052d0 con 0x7f9eb4160de0 2023-10-11T09:48:08.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.685+0000 7f9eb0ff9700 1 -- 172.21.15.78:0/2186264931 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eac005430 con 0x7f9eb4160de0 2023-10-11T09:48:08.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.685+0000 7f9eb9197700 1 -- 172.21.15.78:0/2186264931 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb41e7950 con 0x7f9eb4160de0 2023-10-11T09:48:08.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.685+0000 7f9eb0ff9700 1 -- 172.21.15.78:0/2186264931 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac0055a0 con 0x7f9eb4160de0 2023-10-11T09:48:08.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.685+0000 7f9eb9197700 1 -- 172.21.15.78:0/2186264931 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb41e7f10 con 0x7f9eb4160de0 2023-10-11T09:48:08.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.685+0000 7f9eb9197700 1 -- 172.21.15.78:0/2186264931 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eb408bb90 con 0x7f9eb4160de0 2023-10-11T09:48:08.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.689+0000 7f9eb0ff9700 1 -- 172.21.15.78:0/2186264931 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f9eac0038e0 con 0x7f9eb4160de0 2023-10-11T09:48:08.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.689+0000 7f9eb0ff9700 1 --2- 172.21.15.78:0/2186264931 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9ea0065ed0 0x7f9ea0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:08.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.689+0000 7f9eb0ff9700 1 -- 172.21.15.78:0/2186264931 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6810+0+0 (secure 0 0 0) 0x7f9eac082ff0 con 0x7f9eb4160de0 2023-10-11T09:48:08.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.689+0000 7f9eb3fff700 1 --2- 172.21.15.78:0/2186264931 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9ea0065ed0 0x7f9ea0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:08.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.689+0000 7f9eb3fff700 1 --2- 172.21.15.78:0/2186264931 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9ea0065ed0 0x7f9ea0068380 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4009b20 tx=0x7f9ea4003aa0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:08.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.697+0000 7f9eb0ff9700 1 -- 172.21.15.78:0/2186264931 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f9eac0b3fa0 con 0x7f9eb4160de0 2023-10-11T09:48:08.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:08.873+0000 7f9eb9197700 1 -- 172.21.15.78:0/2186264931 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9eb4157030 con 0x7f9eb4160de0 2023-10-11T09:48:09.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.473+0000 7f9eb0ff9700 1 -- 172.21.15.78:0/2186264931 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v72) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9eac0170d0 con 0x7f9eb4160de0 2023-10-11T09:48:09.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.477+0000 7f9eb9197700 1 -- 172.21.15.78:0/2186264931 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9ea0065ed0 msgr2=0x7f9ea0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:09.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.477+0000 7f9eb9197700 1 --2- 172.21.15.78:0/2186264931 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9ea0065ed0 0x7f9ea0068380 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4009b20 tx=0x7f9ea4003aa0 comp rx=0 tx=0).stop 2023-10-11T09:48:09.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.477+0000 7f9eb9197700 1 -- 172.21.15.78:0/2186264931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eb4160de0 msgr2=0x7f9eb40cca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:09.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.477+0000 7f9eb9197700 1 --2- 172.21.15.78:0/2186264931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eb4160de0 0x7f9eb40cca60 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f9eac00c480 tx=0x7f9eac00c790 comp rx=0 tx=0).stop 2023-10-11T09:48:09.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.477+0000 7f9eb9197700 1 -- 172.21.15.78:0/2186264931 shutdown_connections 2023-10-11T09:48:09.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.477+0000 7f9eb9197700 1 --2- 172.21.15.78:0/2186264931 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9ea0065ed0 0x7f9ea0068380 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:09.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.477+0000 7f9eb9197700 1 --2- 172.21.15.78:0/2186264931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eb4160de0 0x7f9eb40cca60 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:09.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.477+0000 7f9eb9197700 1 --2- 172.21.15.78:0/2186264931 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9eb415a0f0 0x7f9eb40c95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:09.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.477+0000 7f9eb9197700 1 --2- 172.21.15.78:0/2186264931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9eb40b52e0 0x7f9eb41e6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:09.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.477+0000 7f9eb9197700 1 -- 172.21.15.78:0/2186264931 >> 172.21.15.78:0/2186264931 conn(0x7f9eb406afd0 msgr2=0x7f9eb415cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:09.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.477+0000 7f9eb9197700 1 -- 172.21.15.78:0/2186264931 shutdown_connections 2023-10-11T09:48:09.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.477+0000 7f9eb9197700 1 -- 172.21.15.78:0/2186264931 wait complete. 2023-10-11T09:48:09.488 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' removed 2023-10-11T09:48:09.508 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-11T09:48:09.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.685+0000 7f79cedd5700 1 Processor -- start 2023-10-11T09:48:09.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.689+0000 7f79cedd5700 1 -- start start 2023-10-11T09:48:09.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.689+0000 7f79cedd5700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f79c815a0f0 0x7f79c80cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:09.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.689+0000 7f79cedd5700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f79c815cbf0 0x7f79c815d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:09.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.689+0000 7f79cedd5700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79c80d2bd0 con 0x7f79c80b52e0 2023-10-11T09:48:09.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.689+0000 7f79cedd5700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79c815d5b0 con 0x7f79c815cbf0 2023-10-11T09:48:09.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.689+0000 7f79cedd5700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79c80cf630 con 0x7f79c815a0f0 2023-10-11T09:48:09.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.693+0000 7f79ce5d4700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f79c815cbf0 0x7f79c815d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:09.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.693+0000 7f79cddd3700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f79c815a0f0 0x7f79c80cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:09.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.693+0000 7f79cd5d2700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f79c80b52e0 0x7f79c81608a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:45808/0 (socket says 172.21.15.78:45808) 2023-10-11T09:48:09.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.693+0000 7f79cd5d2700 1 -- 172.21.15.78:0/891816943 learned_addr learned my addr 172.21.15.78:0/891816943 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:09.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.693+0000 7f79ce5d4700 1 -- 172.21.15.78:0/891816943 >> v2:172.21.15.133:3300/0 conn(0x7f79c815a0f0 msgr2=0x7f79c80cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:09.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.693+0000 7f79ce5d4700 1 --2- 172.21.15.78:0/891816943 >> v2:172.21.15.133:3300/0 conn(0x7f79c815a0f0 0x7f79c80cefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:09.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.693+0000 7f79ce5d4700 1 -- 172.21.15.78:0/891816943 >> v1:172.21.15.133:6789/0 conn(0x7f79c80b52e0 legacy=0x7f79c81608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:09.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.693+0000 7f79ce5d4700 1 -- 172.21.15.78:0/891816943 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c8160410 con 0x7f79c815cbf0 2023-10-11T09:48:09.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.693+0000 7f79ce5d4700 1 --2- 172.21.15.78:0/891816943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79c815cbf0 0x7f79c815d070 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f79c806cbb0 tx=0x7f79c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2793d9ab19201853 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:09.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.693+0000 7f79ccdd1700 1 -- 172.21.15.78:0/891816943 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c000e070 con 0x7f79c815cbf0 2023-10-11T09:48:09.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.693+0000 7f79ccdd1700 1 -- 172.21.15.78:0/891816943 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79c000bb20 con 0x7f79c815cbf0 2023-10-11T09:48:09.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.693+0000 7f79ccdd1700 1 -- 172.21.15.78:0/891816943 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c0012430 con 0x7f79c815cbf0 2023-10-11T09:48:09.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.693+0000 7f79cedd5700 1 -- 172.21.15.78:0/891816943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79c815cbf0 msgr2=0x7f79c815d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:09.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.693+0000 7f79cedd5700 1 --2- 172.21.15.78:0/891816943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79c815cbf0 0x7f79c815d070 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f79c806cbb0 tx=0x7f79c000b330 comp rx=0 tx=0).stop 2023-10-11T09:48:09.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.693+0000 7f79cedd5700 1 -- 172.21.15.78:0/891816943 shutdown_connections 2023-10-11T09:48:09.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.693+0000 7f79cedd5700 1 --2- 172.21.15.78:0/891816943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79c815cbf0 0x7f79c815d070 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:09.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.693+0000 7f79cedd5700 1 --2- 172.21.15.78:0/891816943 >> v2:172.21.15.133:3300/0 conn(0x7f79c815a0f0 0x7f79c80cefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:09.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.693+0000 7f79cedd5700 1 -- 172.21.15.78:0/891816943 >> 172.21.15.78:0/891816943 conn(0x7f79c806afd0 msgr2=0x7f79c806b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:09.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.693+0000 7f79cedd5700 1 -- 172.21.15.78:0/891816943 shutdown_connections 2023-10-11T09:48:09.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.693+0000 7f79cedd5700 1 -- 172.21.15.78:0/891816943 wait complete. 2023-10-11T09:48:09.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.697+0000 7f79cedd5700 1 Processor -- start 2023-10-11T09:48:09.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.697+0000 7f79cedd5700 1 -- start start 2023-10-11T09:48:09.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.697+0000 7f79cedd5700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79c80b52e0 0x7f79c80b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:09.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.697+0000 7f79cedd5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79c815a0f0 0x7f79c80b2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:09.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.697+0000 7f79cedd5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79c815cbf0 0x7f79c814f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:09.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.697+0000 7f79cedd5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c80b3910 con 0x7f79c815a0f0 2023-10-11T09:48:09.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.697+0000 7f79cedd5700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c80b2d00 con 0x7f79c815cbf0 2023-10-11T09:48:09.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.697+0000 7f79cedd5700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79c80b2fe0 con 0x7f79c80b52e0 2023-10-11T09:48:09.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.697+0000 7f79cd5d2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79c815a0f0 0x7f79c80b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:09.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.697+0000 7f79cd5d2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79c815a0f0 0x7f79c80b2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40774/0 (socket says 172.21.15.78:40774) 2023-10-11T09:48:09.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.697+0000 7f79cd5d2700 1 -- 172.21.15.78:0/2096008625 learned_addr learned my addr 172.21.15.78:0/2096008625 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:09.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.697+0000 7f79cddd3700 1 --2- 172.21.15.78:0/2096008625 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79c80b52e0 0x7f79c80b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:09.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.697+0000 7f79cd5d2700 1 -- 172.21.15.78:0/2096008625 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79c80b52e0 msgr2=0x7f79c80b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:09.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.697+0000 7f79ce5d4700 1 --2- 172.21.15.78:0/2096008625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79c815cbf0 0x7f79c814f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:09.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.697+0000 7f79cd5d2700 1 --2- 172.21.15.78:0/2096008625 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79c80b52e0 0x7f79c80b3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:09.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.697+0000 7f79cd5d2700 1 -- 172.21.15.78:0/2096008625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79c815cbf0 msgr2=0x7f79c814f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:09.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.697+0000 7f79cd5d2700 1 --2- 172.21.15.78:0/2096008625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79c815cbf0 0x7f79c814f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:09.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.697+0000 7f79cd5d2700 1 -- 172.21.15.78:0/2096008625 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c0009a70 con 0x7f79c815a0f0 2023-10-11T09:48:09.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.697+0000 7f79ce5d4700 1 --2- 172.21.15.78:0/2096008625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79c815cbf0 0x7f79c814f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:48:09.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.697+0000 7f79cddd3700 1 --2- 172.21.15.78:0/2096008625 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79c80b52e0 0x7f79c80b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:48:09.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.697+0000 7f79cd5d2700 1 --2- 172.21.15.78:0/2096008625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79c815a0f0 0x7f79c80b2670 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f79b800c700 tx=0x7f79b800cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:09.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.697+0000 7f79be7fc700 1 -- 172.21.15.78:0/2096008625 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b8005d90 con 0x7f79c815a0f0 2023-10-11T09:48:09.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.697+0000 7f79be7fc700 1 -- 172.21.15.78:0/2096008625 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b80070d0 con 0x7f79c815a0f0 2023-10-11T09:48:09.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.697+0000 7f79be7fc700 1 -- 172.21.15.78:0/2096008625 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b8007440 con 0x7f79c815a0f0 2023-10-11T09:48:09.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.701+0000 7f79cedd5700 1 -- 172.21.15.78:0/2096008625 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79c80b32c0 con 0x7f79c815a0f0 2023-10-11T09:48:09.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.701+0000 7f79cedd5700 1 -- 172.21.15.78:0/2096008625 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79c80b86d0 con 0x7f79c815a0f0 2023-10-11T09:48:09.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.701+0000 7f79cedd5700 1 -- 172.21.15.78:0/2096008625 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79c80b3ac0 con 0x7f79c815a0f0 2023-10-11T09:48:09.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.701+0000 7f79be7fc700 1 -- 172.21.15.78:0/2096008625 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f79b80075a0 con 0x7f79c815a0f0 2023-10-11T09:48:09.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.705+0000 7f79be7fc700 1 --2- 172.21.15.78:0/2096008625 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f79ac065ed0 0x7f79ac068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:09.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.705+0000 7f79be7fc700 1 -- 172.21.15.78:0/2096008625 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6372+0+0 (secure 0 0 0) 0x7f79b8082e00 con 0x7f79c815a0f0 2023-10-11T09:48:09.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.705+0000 7f79cddd3700 1 --2- 172.21.15.78:0/2096008625 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f79ac065ed0 0x7f79ac068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:09.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.705+0000 7f79cddd3700 1 --2- 172.21.15.78:0/2096008625 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f79ac065ed0 0x7f79ac068380 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f79c4009b70 tx=0x7f79c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:09.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.709+0000 7f79be7fc700 1 -- 172.21.15.78:0/2096008625 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f79b8050330 con 0x7f79c815a0f0 2023-10-11T09:48:09.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:09.881+0000 7f79cedd5700 1 -- 172.21.15.78:0/2096008625 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f79c808bb90 con 0x7f79c815a0f0 2023-10-11T09:48:10.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.481+0000 7f79be7fc700 1 -- 172.21.15.78:0/2096008625 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v73) v1 ==== 169+0+0 (secure 0 0 0) 0x7f79b8053950 con 0x7f79c815a0f0 2023-10-11T09:48:10.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.489+0000 7f79cedd5700 1 -- 172.21.15.78:0/2096008625 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f79ac065ed0 msgr2=0x7f79ac068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:10.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.489+0000 7f79cedd5700 1 --2- 172.21.15.78:0/2096008625 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f79ac065ed0 0x7f79ac068380 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f79c4009b70 tx=0x7f79c4008040 comp rx=0 tx=0).stop 2023-10-11T09:48:10.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.489+0000 7f79cedd5700 1 -- 172.21.15.78:0/2096008625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79c815a0f0 msgr2=0x7f79c80b2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:10.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.489+0000 7f79cedd5700 1 --2- 172.21.15.78:0/2096008625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79c815a0f0 0x7f79c80b2670 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f79b800c700 tx=0x7f79b800cac0 comp rx=0 tx=0).stop 2023-10-11T09:48:10.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.489+0000 7f79cedd5700 1 -- 172.21.15.78:0/2096008625 shutdown_connections 2023-10-11T09:48:10.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.489+0000 7f79cedd5700 1 --2- 172.21.15.78:0/2096008625 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f79ac065ed0 0x7f79ac068380 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:10.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.489+0000 7f79cedd5700 1 --2- 172.21.15.78:0/2096008625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79c815cbf0 0x7f79c814f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:10.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.489+0000 7f79cedd5700 1 --2- 172.21.15.78:0/2096008625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79c815a0f0 0x7f79c80b2670 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:10.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.489+0000 7f79cedd5700 1 --2- 172.21.15.78:0/2096008625 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79c80b52e0 0x7f79c80b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:10.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.489+0000 7f79cedd5700 1 -- 172.21.15.78:0/2096008625 >> 172.21.15.78:0/2096008625 conn(0x7f79c806afd0 msgr2=0x7f79c815f900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:10.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.489+0000 7f79cedd5700 1 -- 172.21.15.78:0/2096008625 shutdown_connections 2023-10-11T09:48:10.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.489+0000 7f79cedd5700 1 -- 172.21.15.78:0/2096008625 wait complete. 2023-10-11T09:48:10.500 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' removed 2023-10-11T09:48:10.520 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_extra cephfs_data_extra --yes-i-really-really-mean-it 2023-10-11T09:48:10.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.689+0000 7fea7b6bf700 1 Processor -- start 2023-10-11T09:48:10.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.689+0000 7fea7b6bf700 1 -- start start 2023-10-11T09:48:10.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.689+0000 7fea7b6bf700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fea7406c630 0x7fea740c2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:10.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.689+0000 7fea7b6bf700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea740cd3e0 con 0x7fea74160d90 2023-10-11T09:48:10.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.689+0000 7fea7b6bf700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea740710f0 con 0x7fea741649c0 2023-10-11T09:48:10.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.689+0000 7fea7b6bf700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea740cb8f0 con 0x7fea7406c630 2023-10-11T09:48:10.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.689+0000 7fea7945b700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fea74160d90 0x7fea740c1a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:39986/0 (socket says 172.21.15.78:39986) 2023-10-11T09:48:10.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.689+0000 7fea7945b700 1 -- 172.21.15.78:0/461731799 learned_addr learned my addr 172.21.15.78:0/461731799 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:10.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.693+0000 7fea78c5a700 1 --2- 172.21.15.78:0/461731799 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fea7406c630 0x7fea740c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:10.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.693+0000 7fea6bfff700 1 -- 172.21.15.78:0/461731799 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2616376963 0 0) 0x7fea740cd3e0 con 0x7fea74160d90 2023-10-11T09:48:10.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.693+0000 7fea6bfff700 1 -- 172.21.15.78:0/461731799 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea600035d0 con 0x7fea74160d90 2023-10-11T09:48:10.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.693+0000 7fea6bfff700 1 -- 172.21.15.78:0/461731799 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1956566093 0 0) 0x7fea740710f0 con 0x7fea741649c0 2023-10-11T09:48:10.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.693+0000 7fea6bfff700 1 -- 172.21.15.78:0/461731799 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea740cd3e0 con 0x7fea741649c0 2023-10-11T09:48:10.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.693+0000 7fea78c5a700 1 -- 172.21.15.78:0/461731799 >> v1:172.21.15.133:6789/0 conn(0x7fea741649c0 legacy=0x7fea74164e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:10.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.693+0000 7fea78c5a700 1 -- 172.21.15.78:0/461731799 >> v1:172.21.15.78:6789/0 conn(0x7fea74160d90 legacy=0x7fea740c1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:10.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.693+0000 7fea78c5a700 1 -- 172.21.15.78:0/461731799 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea74072300 con 0x7fea7406c630 2023-10-11T09:48:10.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.693+0000 7fea78c5a700 1 --2- 172.21.15.78:0/461731799 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fea7406c630 0x7fea740c2180 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fea64009870 tx=0x7fea64009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32ca507535a819dd server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:10.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.693+0000 7fea6bfff700 1 -- 172.21.15.78:0/461731799 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea64005e50 con 0x7fea7406c630 2023-10-11T09:48:10.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.693+0000 7fea6bfff700 1 -- 172.21.15.78:0/461731799 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea64004030 con 0x7fea7406c630 2023-10-11T09:48:10.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.693+0000 7fea6bfff700 1 -- 172.21.15.78:0/461731799 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea640043c0 con 0x7fea7406c630 2023-10-11T09:48:10.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.693+0000 7fea7b6bf700 1 -- 172.21.15.78:0/461731799 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fea7406c630 msgr2=0x7fea740c2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:10.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.693+0000 7fea7b6bf700 1 --2- 172.21.15.78:0/461731799 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fea7406c630 0x7fea740c2180 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fea64009870 tx=0x7fea64009b80 comp rx=0 tx=0).stop 2023-10-11T09:48:10.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.693+0000 7fea7b6bf700 1 -- 172.21.15.78:0/461731799 shutdown_connections 2023-10-11T09:48:10.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.693+0000 7fea7b6bf700 1 --2- 172.21.15.78:0/461731799 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fea7406c630 0x7fea740c2180 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:10.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.693+0000 7fea7b6bf700 1 -- 172.21.15.78:0/461731799 >> 172.21.15.78:0/461731799 conn(0x7fea7406a060 msgr2=0x7fea7406a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:10.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.693+0000 7fea7b6bf700 1 -- 172.21.15.78:0/461731799 shutdown_connections 2023-10-11T09:48:10.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.693+0000 7fea7b6bf700 1 -- 172.21.15.78:0/461731799 wait complete. 2023-10-11T09:48:10.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.697+0000 7fea7b6bf700 1 Processor -- start 2023-10-11T09:48:10.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.701+0000 7fea7b6bf700 1 -- start start 2023-10-11T09:48:10.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.701+0000 7fea7b6bf700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fea7406c630 0x7fea7414a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:10.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.701+0000 7fea7b6bf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea74160d90 0x7fea741ea770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:10.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.701+0000 7fea7b6bf700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea741649c0 0x7fea7415eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:10.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.701+0000 7fea7b6bf700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea740c6e80 con 0x7fea74160d90 2023-10-11T09:48:10.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.701+0000 7fea7b6bf700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea740c7610 con 0x7fea741649c0 2023-10-11T09:48:10.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.701+0000 7fea7b6bf700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea740c78f0 con 0x7fea7406c630 2023-10-11T09:48:10.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.701+0000 7fea78c5a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea74160d90 0x7fea741ea770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:10.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.701+0000 7fea78c5a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea74160d90 0x7fea741ea770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40784/0 (socket says 172.21.15.78:40784) 2023-10-11T09:48:10.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.701+0000 7fea78c5a700 1 -- 172.21.15.78:0/4188966266 learned_addr learned my addr 172.21.15.78:0/4188966266 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:10.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.701+0000 7fea7945b700 1 --2- 172.21.15.78:0/4188966266 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fea7406c630 0x7fea7414a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:10.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.701+0000 7fea79c5c700 1 --2- 172.21.15.78:0/4188966266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea741649c0 0x7fea7415eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:10.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.701+0000 7fea78c5a700 1 -- 172.21.15.78:0/4188966266 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fea7406c630 msgr2=0x7fea7414a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:10.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.701+0000 7fea78c5a700 1 --2- 172.21.15.78:0/4188966266 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fea7406c630 0x7fea7414a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:10.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.701+0000 7fea78c5a700 1 -- 172.21.15.78:0/4188966266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea741649c0 msgr2=0x7fea7415eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:10.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.701+0000 7fea78c5a700 1 --2- 172.21.15.78:0/4188966266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea741649c0 0x7fea7415eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:10.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.701+0000 7fea78c5a700 1 -- 172.21.15.78:0/4188966266 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea64009450 con 0x7fea74160d90 2023-10-11T09:48:10.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.701+0000 7fea78c5a700 1 --2- 172.21.15.78:0/4188966266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea74160d90 0x7fea741ea770 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fea64014040 tx=0x7fea64004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:10.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.701+0000 7fea69ffb700 1 -- 172.21.15.78:0/4188966266 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea64003d10 con 0x7fea74160d90 2023-10-11T09:48:10.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.701+0000 7fea69ffb700 1 -- 172.21.15.78:0/4188966266 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea64003e70 con 0x7fea74160d90 2023-10-11T09:48:10.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.701+0000 7fea69ffb700 1 -- 172.21.15.78:0/4188966266 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6401c640 con 0x7fea74160d90 2023-10-11T09:48:10.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.701+0000 7fea7b6bf700 1 -- 172.21.15.78:0/4188966266 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea740c7bd0 con 0x7fea74160d90 2023-10-11T09:48:10.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.705+0000 7fea7b6bf700 1 -- 172.21.15.78:0/4188966266 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea7414a970 con 0x7fea74160d90 2023-10-11T09:48:10.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.705+0000 7fea7b6bf700 1 -- 172.21.15.78:0/4188966266 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea740c80f0 con 0x7fea74160d90 2023-10-11T09:48:10.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.705+0000 7fea69ffb700 1 -- 172.21.15.78:0/4188966266 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fea6401c7a0 con 0x7fea74160d90 2023-10-11T09:48:10.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.709+0000 7fea69ffb700 1 --2- 172.21.15.78:0/4188966266 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fea58065ed0 0x7fea58068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:10.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.709+0000 7fea69ffb700 1 -- 172.21.15.78:0/4188966266 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5958+0+0 (secure 0 0 0) 0x7fea64006e40 con 0x7fea74160d90 2023-10-11T09:48:10.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.709+0000 7fea7945b700 1 --2- 172.21.15.78:0/4188966266 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fea58065ed0 0x7fea58068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:10.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.709+0000 7fea7945b700 1 --2- 172.21.15.78:0/4188966266 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fea58065ed0 0x7fea58068380 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fea70007fd0 tx=0x7fea70009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:10.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.713+0000 7fea69ffb700 1 -- 172.21.15.78:0/4188966266 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fea740c80f0 con 0x7fea74160d90 2023-10-11T09:48:10.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:10.901+0000 7fea7b6bf700 1 -- 172.21.15.78:0/4188966266 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_extra", "pool2": "cephfs_data_extra", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fea740c80f0 con 0x7fea74160d90 2023-10-11T09:48:11.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.505+0000 7fea69ffb700 1 -- 172.21.15.78:0/4188966266 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_extra", "pool2": "cephfs_data_extra", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_extra' removed v74) v1 ==== 187+0+0 (secure 0 0 0) 0x7fea740c80f0 con 0x7fea74160d90 2023-10-11T09:48:11.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.509+0000 7fea7b6bf700 1 -- 172.21.15.78:0/4188966266 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fea58065ed0 msgr2=0x7fea58068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:11.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.509+0000 7fea7b6bf700 1 --2- 172.21.15.78:0/4188966266 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fea58065ed0 0x7fea58068380 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fea70007fd0 tx=0x7fea70009040 comp rx=0 tx=0).stop 2023-10-11T09:48:11.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.509+0000 7fea7b6bf700 1 -- 172.21.15.78:0/4188966266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea74160d90 msgr2=0x7fea741ea770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:11.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.509+0000 7fea7b6bf700 1 --2- 172.21.15.78:0/4188966266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea74160d90 0x7fea741ea770 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fea64014040 tx=0x7fea64004770 comp rx=0 tx=0).stop 2023-10-11T09:48:11.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.509+0000 7fea7b6bf700 1 -- 172.21.15.78:0/4188966266 shutdown_connections 2023-10-11T09:48:11.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.509+0000 7fea7b6bf700 1 --2- 172.21.15.78:0/4188966266 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fea58065ed0 0x7fea58068380 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:11.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.509+0000 7fea7b6bf700 1 --2- 172.21.15.78:0/4188966266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea741649c0 0x7fea7415eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:11.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.509+0000 7fea7b6bf700 1 --2- 172.21.15.78:0/4188966266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea74160d90 0x7fea741ea770 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:11.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.509+0000 7fea7b6bf700 1 --2- 172.21.15.78:0/4188966266 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fea7406c630 0x7fea7414a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:11.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.509+0000 7fea7b6bf700 1 -- 172.21.15.78:0/4188966266 >> 172.21.15.78:0/4188966266 conn(0x7fea7406a060 msgr2=0x7fea740b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:11.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.509+0000 7fea7b6bf700 1 -- 172.21.15.78:0/4188966266 shutdown_connections 2023-10-11T09:48:11.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.509+0000 7fea7b6bf700 1 -- 172.21.15.78:0/4188966266 wait complete. 2023-10-11T09:48:11.520 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data_extra' removed 2023-10-11T09:48:11.540 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-11T09:48:11.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.709+0000 7fd5b8907700 1 Processor -- start 2023-10-11T09:48:11.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.709+0000 7fd5b8907700 1 -- start start 2023-10-11T09:48:11.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.713+0000 7fd5b8907700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd5b41601f0 0x7fd5b40ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:11.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.713+0000 7fd5b8907700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5b40cc420 con 0x7fd5b40b1820 2023-10-11T09:48:11.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.713+0000 7fd5b8907700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5b4160680 con 0x7fd5b413f9e0 2023-10-11T09:48:11.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.713+0000 7fd5b8907700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b413fdb0 con 0x7fd5b41601f0 2023-10-11T09:48:11.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.713+0000 7fd5b3fff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fd5b40b1820 0x7fd5b40cc8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:39996/0 (socket says 172.21.15.78:39996) 2023-10-11T09:48:11.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.713+0000 7fd5b3fff700 1 -- 172.21.15.78:0/3301351729 learned_addr learned my addr 172.21.15.78:0/3301351729 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:11.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.713+0000 7fd5aaffd700 1 --2- 172.21.15.78:0/3301351729 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd5b41601f0 0x7fd5b40ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:11.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.713+0000 7fd5b2ffd700 1 -- 172.21.15.78:0/3301351729 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3372636314 0 0) 0x7fd5b40cc420 con 0x7fd5b40b1820 2023-10-11T09:48:11.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.713+0000 7fd5b2ffd700 1 -- 172.21.15.78:0/3301351729 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5a00035d0 con 0x7fd5b40b1820 2023-10-11T09:48:11.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.713+0000 7fd5b2ffd700 1 -- 172.21.15.78:0/3301351729 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3811339332 0 0) 0x7fd5a00035d0 con 0x7fd5b40b1820 2023-10-11T09:48:11.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.713+0000 7fd5b2ffd700 1 -- 172.21.15.78:0/3301351729 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd5b41601f0 msgr2=0x7fd5b40ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:11.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.713+0000 7fd5b2ffd700 1 --2- 172.21.15.78:0/3301351729 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd5b41601f0 0x7fd5b40ca1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:11.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.713+0000 7fd5b2ffd700 1 -- 172.21.15.78:0/3301351729 >> v1:172.21.15.133:6789/0 conn(0x7fd5b413f9e0 legacy=0x7fd5b40c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:11.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.713+0000 7fd5b2ffd700 1 -- 172.21.15.78:0/3301351729 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b4160960 con 0x7fd5b40b1820 2023-10-11T09:48:11.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.713+0000 7fd5b2ffd700 1 -- 172.21.15.78:0/3301351729 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7fd5ac0019f0 con 0x7fd5b40b1820 2023-10-11T09:48:11.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.713+0000 7fd5b2ffd700 1 -- 172.21.15.78:0/3301351729 >> v1:172.21.15.78:6789/0 conn(0x7fd5b40b1820 legacy=0x7fd5b40cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:11.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.713+0000 7fd5b2ffd700 1 --2- 172.21.15.78:0/3301351729 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd5b41601f0 0x7fd5a0003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:11.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.713+0000 7fd5b2ffd700 1 --2- 172.21.15.78:0/3301351729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5a0004080 0x7fd5a0004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:11.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.717+0000 7fd5b2ffd700 1 --2- 172.21.15.78:0/3301351729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5a0004db0 0x7fd5a0005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:11.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.717+0000 7fd5b2ffd700 1 -- 172.21.15.78:0/3301351729 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b413fdb0 con 0x7fd5a0004080 2023-10-11T09:48:11.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.717+0000 7fd5b2ffd700 1 -- 172.21.15.78:0/3301351729 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a000bc50 con 0x7fd5a0004db0 2023-10-11T09:48:11.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.717+0000 7fd5b2ffd700 1 -- 172.21.15.78:0/3301351729 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a000be10 con 0x7fd5b41601f0 2023-10-11T09:48:11.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.717+0000 7fd5aaffd700 1 --2- 172.21.15.78:0/3301351729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5a0004080 0x7fd5a0004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:11.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.717+0000 7fd5b37fe700 1 --2- 172.21.15.78:0/3301351729 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd5b41601f0 0x7fd5a0003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:11.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.717+0000 7fd5b3fff700 1 --2- 172.21.15.78:0/3301351729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5a0004db0 0x7fd5a0005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:11.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.717+0000 7fd5aaffd700 1 -- 172.21.15.78:0/3301351729 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd5b41601f0 msgr2=0x7fd5a0003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:11.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.717+0000 7fd5aaffd700 1 --2- 172.21.15.78:0/3301351729 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd5b41601f0 0x7fd5a0003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:11.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.717+0000 7fd5aaffd700 1 -- 172.21.15.78:0/3301351729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5a0004db0 msgr2=0x7fd5a0005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:11.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.717+0000 7fd5aaffd700 1 --2- 172.21.15.78:0/3301351729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5a0004db0 0x7fd5a0005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:11.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.717+0000 7fd5aaffd700 1 -- 172.21.15.78:0/3301351729 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a00035d0 con 0x7fd5a0004080 2023-10-11T09:48:11.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.717+0000 7fd5b3fff700 1 --2- 172.21.15.78:0/3301351729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5a0004db0 0x7fd5a0005220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:48:11.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.717+0000 7fd5aaffd700 1 --2- 172.21.15.78:0/3301351729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5a0004080 0x7fd5a0004870 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fd59c0098b0 tx=0x7fd5a00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e25ddd177988a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:11.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.717+0000 7fd5b2ffd700 1 -- 172.21.15.78:0/3301351729 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c00c070 con 0x7fd5a0004080 2023-10-11T09:48:11.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.717+0000 7fd5b2ffd700 1 -- 172.21.15.78:0/3301351729 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd59c013610 con 0x7fd5a0004080 2023-10-11T09:48:11.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.717+0000 7fd5b2ffd700 1 -- 172.21.15.78:0/3301351729 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c0139c0 con 0x7fd5a0004080 2023-10-11T09:48:11.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.717+0000 7fd5b8907700 1 -- 172.21.15.78:0/3301351729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5a0004080 msgr2=0x7fd5a0004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:11.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.717+0000 7fd5b8907700 1 --2- 172.21.15.78:0/3301351729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5a0004080 0x7fd5a0004870 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fd59c0098b0 tx=0x7fd5a00058a0 comp rx=0 tx=0).stop 2023-10-11T09:48:11.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.717+0000 7fd5b3fff700 1 -- 172.21.15.78:0/3301351729 reap_dead start 2023-10-11T09:48:11.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.717+0000 7fd5b8907700 1 -- 172.21.15.78:0/3301351729 shutdown_connections 2023-10-11T09:48:11.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.717+0000 7fd5b8907700 1 -- 172.21.15.78:0/3301351729 >> 172.21.15.78:0/3301351729 conn(0x7fd5b406dd80 msgr2=0x7fd5b406e180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:11.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.717+0000 7fd5b8907700 1 -- 172.21.15.78:0/3301351729 shutdown_connections 2023-10-11T09:48:11.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.717+0000 7fd5b8907700 1 -- 172.21.15.78:0/3301351729 wait complete. 2023-10-11T09:48:11.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.721+0000 7fd5b8907700 1 Processor -- start 2023-10-11T09:48:11.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.721+0000 7fd5b8907700 1 -- start start 2023-10-11T09:48:11.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.721+0000 7fd5b8907700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd5a0004080 0x7fd5b40b3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:11.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.725+0000 7fd5b8907700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5b413a070 0x7fd5b40cb030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:11.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.725+0000 7fd5b8907700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5b413a460 0x7fd5b40cb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:11.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.725+0000 7fd5b8907700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b40b2b90 con 0x7fd5b413a070 2023-10-11T09:48:11.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.725+0000 7fd5b8907700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b40b2e70 con 0x7fd5b413a460 2023-10-11T09:48:11.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.725+0000 7fd5b8907700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b40b43c0 con 0x7fd5a0004080 2023-10-11T09:48:11.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.725+0000 7fd5aaffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5b413a070 0x7fd5b40cb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:11.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.725+0000 7fd5aaffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5b413a070 0x7fd5b40cb030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40796/0 (socket says 172.21.15.78:40796) 2023-10-11T09:48:11.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.725+0000 7fd5aaffd700 1 -- 172.21.15.78:0/1247554359 learned_addr learned my addr 172.21.15.78:0/1247554359 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:11.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.725+0000 7fd5b37fe700 1 --2- 172.21.15.78:0/1247554359 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd5a0004080 0x7fd5b40b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:11.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.725+0000 7fd5b3fff700 1 --2- 172.21.15.78:0/1247554359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5b413a460 0x7fd5b40cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:11.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.725+0000 7fd5aaffd700 1 -- 172.21.15.78:0/1247554359 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd5a0004080 msgr2=0x7fd5b40b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:11.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.725+0000 7fd5aaffd700 1 --2- 172.21.15.78:0/1247554359 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd5a0004080 0x7fd5b40b3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:11.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.725+0000 7fd5aaffd700 1 -- 172.21.15.78:0/1247554359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5b413a460 msgr2=0x7fd5b40cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:11.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.725+0000 7fd5aaffd700 1 --2- 172.21.15.78:0/1247554359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5b413a460 0x7fd5b40cb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:11.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.725+0000 7fd5aaffd700 1 -- 172.21.15.78:0/1247554359 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b40b46a0 con 0x7fd5b413a070 2023-10-11T09:48:11.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.725+0000 7fd5aaffd700 1 --2- 172.21.15.78:0/1247554359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5b413a070 0x7fd5b40cb030 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fd59c000c00 tx=0x7fd59c0047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:11.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.725+0000 7fd5b0ff9700 1 -- 172.21.15.78:0/1247554359 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c013c50 con 0x7fd5b413a070 2023-10-11T09:48:11.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.725+0000 7fd5b0ff9700 1 -- 172.21.15.78:0/1247554359 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd59c013db0 con 0x7fd5b413a070 2023-10-11T09:48:11.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.725+0000 7fd5b8907700 1 -- 172.21.15.78:0/1247554359 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5b40b4980 con 0x7fd5b413a070 2023-10-11T09:48:11.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.725+0000 7fd5b0ff9700 1 -- 172.21.15.78:0/1247554359 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c0038e0 con 0x7fd5b413a070 2023-10-11T09:48:11.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.725+0000 7fd5b8907700 1 -- 172.21.15.78:0/1247554359 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5b41584e0 con 0x7fd5b413a070 2023-10-11T09:48:11.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.725+0000 7fd5b8907700 1 -- 172.21.15.78:0/1247554359 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5b40b4bc0 con 0x7fd5b413a070 2023-10-11T09:48:11.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.729+0000 7fd5b0ff9700 1 -- 172.21.15.78:0/1247554359 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fd59c003a40 con 0x7fd5b413a070 2023-10-11T09:48:11.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.729+0000 7fd5b0ff9700 1 --2- 172.21.15.78:0/1247554359 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd594065f00 0x7fd5940683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:11.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.729+0000 7fd5b0ff9700 1 -- 172.21.15.78:0/1247554359 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fd59c0847b0 con 0x7fd5b413a070 2023-10-11T09:48:11.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.729+0000 7fd5b37fe700 1 --2- 172.21.15.78:0/1247554359 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd594065f00 0x7fd5940683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:11.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.733+0000 7fd5b37fe700 1 --2- 172.21.15.78:0/1247554359 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd594065f00 0x7fd5940683b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd5a40055b0 tx=0x7fd5a4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:11.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.737+0000 7fd5b0ff9700 1 -- 172.21.15.78:0/1247554359 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fd59c051f60 con 0x7fd5b413a070 2023-10-11T09:48:11.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.913+0000 7fd5b8907700 1 -- 172.21.15.78:0/1247554359 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd5b4065d40 con 0x7fd5b413a070 2023-10-11T09:48:11.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.921+0000 7fd5b0ff9700 1 -- 172.21.15.78:0/1247554359 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd59c055580 con 0x7fd5b413a070 2023-10-11T09:48:11.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.925+0000 7fd5b8907700 1 -- 172.21.15.78:0/1247554359 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd594065f00 msgr2=0x7fd5940683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:11.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.925+0000 7fd5b8907700 1 --2- 172.21.15.78:0/1247554359 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd594065f00 0x7fd5940683b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd5a40055b0 tx=0x7fd5a4009040 comp rx=0 tx=0).stop 2023-10-11T09:48:11.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.925+0000 7fd5b8907700 1 -- 172.21.15.78:0/1247554359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5b413a070 msgr2=0x7fd5b40cb030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:11.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.925+0000 7fd5b8907700 1 --2- 172.21.15.78:0/1247554359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5b413a070 0x7fd5b40cb030 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fd59c000c00 tx=0x7fd59c0047e0 comp rx=0 tx=0).stop 2023-10-11T09:48:11.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.925+0000 7fd5b8907700 1 -- 172.21.15.78:0/1247554359 shutdown_connections 2023-10-11T09:48:11.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.925+0000 7fd5b8907700 1 --2- 172.21.15.78:0/1247554359 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd594065f00 0x7fd5940683b0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:11.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.925+0000 7fd5b8907700 1 --2- 172.21.15.78:0/1247554359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5b413a460 0x7fd5b40cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:11.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.925+0000 7fd5b8907700 1 --2- 172.21.15.78:0/1247554359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5b413a070 0x7fd5b40cb030 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:11.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.925+0000 7fd5b8907700 1 --2- 172.21.15.78:0/1247554359 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd5a0004080 0x7fd5b40b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:11.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.925+0000 7fd5b8907700 1 -- 172.21.15.78:0/1247554359 >> 172.21.15.78:0/1247554359 conn(0x7fd5b406dd80 msgr2=0x7fd5b40b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:11.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.929+0000 7fd5b8907700 1 -- 172.21.15.78:0/1247554359 shutdown_connections 2023-10-11T09:48:11.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:11.929+0000 7fd5b8907700 1 -- 172.21.15.78:0/1247554359 wait complete. 2023-10-11T09:48:11.957 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_many_files_many_workers' 2023-10-11T09:48:12.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.113+0000 7f36eeae4700 1 Processor -- start 2023-10-11T09:48:12.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.117+0000 7f36eeae4700 1 -- start start 2023-10-11T09:48:12.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.117+0000 7f36eeae4700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f36e8156ce0 0x7f36e8160b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:12.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.117+0000 7f36eeae4700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36e80d1160 0x7f36e80cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:12.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.117+0000 7f36eeae4700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e815dc10 con 0x7f36e80d1660 2023-10-11T09:48:12.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.117+0000 7f36eeae4700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36e8160380 con 0x7f36e8156ce0 2023-10-11T09:48:12.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.117+0000 7f36eeae4700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e815d930 con 0x7f36e80d1160 2023-10-11T09:48:12.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.117+0000 7f36ed2e1700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f36e80d1660 0x7f36e80d0a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:40008/0 (socket says 172.21.15.78:40008) 2023-10-11T09:48:12.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.117+0000 7f36ed2e1700 1 -- 172.21.15.78:0/1199276595 learned_addr learned my addr 172.21.15.78:0/1199276595 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:12.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.117+0000 7f36edae2700 1 --2- 172.21.15.78:0/1199276595 >> v2:172.21.15.133:3300/0 conn(0x7f36e8156ce0 0x7f36e8160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:12.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.117+0000 7f36ee2e3700 1 --2- 172.21.15.78:0/1199276595 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36e80d1160 0x7f36e80cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:12.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.121+0000 7f36ecae0700 1 -- 172.21.15.78:0/1199276595 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2427332995 0 0) 0x7f36e815dc10 con 0x7f36e80d1660 2023-10-11T09:48:12.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.121+0000 7f36ecae0700 1 -- 172.21.15.78:0/1199276595 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36d40035d0 con 0x7f36e80d1660 2023-10-11T09:48:12.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.121+0000 7f36edae2700 1 -- 172.21.15.78:0/1199276595 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36e80d1160 msgr2=0x7f36e80cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:12.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.121+0000 7f36edae2700 1 --2- 172.21.15.78:0/1199276595 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36e80d1160 0x7f36e80cefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:12.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.121+0000 7f36edae2700 1 -- 172.21.15.78:0/1199276595 >> v1:172.21.15.78:6789/0 conn(0x7f36e80d1660 legacy=0x7f36e80d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:12.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.121+0000 7f36ecae0700 1 -- 172.21.15.78:0/1199276595 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2731953884 0 0) 0x7f36d40035d0 con 0x7f36e80d1660 2023-10-11T09:48:12.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.121+0000 7f36edae2700 1 -- 172.21.15.78:0/1199276595 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e8160620 con 0x7f36e8156ce0 2023-10-11T09:48:12.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.121+0000 7f36edae2700 1 --2- 172.21.15.78:0/1199276595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36e8156ce0 0x7f36e8160b40 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f36e4009850 tx=0x7f36e4009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6aed8ace76b7184 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:12.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.121+0000 7f36ecae0700 1 -- 172.21.15.78:0/1199276595 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d8001670 con 0x7f36e8156ce0 2023-10-11T09:48:12.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.121+0000 7f36ecae0700 1 -- 172.21.15.78:0/1199276595 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e4004030 con 0x7f36e8156ce0 2023-10-11T09:48:12.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.121+0000 7f36ecae0700 1 -- 172.21.15.78:0/1199276595 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e40043e0 con 0x7f36e8156ce0 2023-10-11T09:48:12.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.121+0000 7f36eeae4700 1 -- 172.21.15.78:0/1199276595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36e8156ce0 msgr2=0x7f36e8160b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:12.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.121+0000 7f36eeae4700 1 --2- 172.21.15.78:0/1199276595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36e8156ce0 0x7f36e8160b40 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f36e4009850 tx=0x7f36e4009b60 comp rx=0 tx=0).stop 2023-10-11T09:48:12.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.121+0000 7f36eeae4700 1 -- 172.21.15.78:0/1199276595 shutdown_connections 2023-10-11T09:48:12.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.121+0000 7f36eeae4700 1 --2- 172.21.15.78:0/1199276595 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36e80d1160 0x7f36e80cefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:12.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.121+0000 7f36eeae4700 1 --2- 172.21.15.78:0/1199276595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36e8156ce0 0x7f36e8160b40 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:12.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.121+0000 7f36eeae4700 1 -- 172.21.15.78:0/1199276595 >> 172.21.15.78:0/1199276595 conn(0x7f36e806afb0 msgr2=0x7f36e806b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:12.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.121+0000 7f36eeae4700 1 -- 172.21.15.78:0/1199276595 shutdown_connections 2023-10-11T09:48:12.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.121+0000 7f36eeae4700 1 -- 172.21.15.78:0/1199276595 wait complete. 2023-10-11T09:48:12.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.125+0000 7f36eeae4700 1 Processor -- start 2023-10-11T09:48:12.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.125+0000 7f36eeae4700 1 -- start start 2023-10-11T09:48:12.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.125+0000 7f36eeae4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36e80d1160 0x7f36e81f4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:12.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.125+0000 7f36eeae4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36e80d1660 0x7f36e80c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:12.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.125+0000 7f36eeae4700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36e8156ce0 0x7f36e81548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:12.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.125+0000 7f36eeae4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e80caa10 con 0x7f36e80d1160 2023-10-11T09:48:12.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.125+0000 7f36eeae4700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e80c6f10 con 0x7f36e80d1660 2023-10-11T09:48:12.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.125+0000 7f36eeae4700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e80c7160 con 0x7f36e8156ce0 2023-10-11T09:48:12.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.125+0000 7f36edae2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36e80d1160 0x7f36e81f4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:12.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.125+0000 7f36edae2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36e80d1160 0x7f36e81f4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40806/0 (socket says 172.21.15.78:40806) 2023-10-11T09:48:12.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.125+0000 7f36edae2700 1 -- 172.21.15.78:0/3504387389 learned_addr learned my addr 172.21.15.78:0/3504387389 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:12.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.125+0000 7f36ee2e3700 1 --2- 172.21.15.78:0/3504387389 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36e8156ce0 0x7f36e81548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:12.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.125+0000 7f36ed2e1700 1 --2- 172.21.15.78:0/3504387389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36e80d1660 0x7f36e80c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:12.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.125+0000 7f36edae2700 1 -- 172.21.15.78:0/3504387389 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36e8156ce0 msgr2=0x7f36e81548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:12.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.125+0000 7f36edae2700 1 --2- 172.21.15.78:0/3504387389 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36e8156ce0 0x7f36e81548a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:12.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.125+0000 7f36edae2700 1 -- 172.21.15.78:0/3504387389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36e80d1660 msgr2=0x7f36e80c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:12.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.125+0000 7f36edae2700 1 --2- 172.21.15.78:0/3504387389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36e80d1660 0x7f36e80c9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:12.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.125+0000 7f36edae2700 1 -- 172.21.15.78:0/3504387389 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e40094b0 con 0x7f36e80d1160 2023-10-11T09:48:12.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.129+0000 7f36edae2700 1 --2- 172.21.15.78:0/3504387389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36e80d1160 0x7f36e81f4930 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f36e4000c00 tx=0x7f36e4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:12.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.129+0000 7f36de7fc700 1 -- 172.21.15.78:0/3504387389 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e4003b00 con 0x7f36e80d1160 2023-10-11T09:48:12.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.129+0000 7f36de7fc700 1 -- 172.21.15.78:0/3504387389 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e401f070 con 0x7f36e80d1160 2023-10-11T09:48:12.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.129+0000 7f36eeae4700 1 -- 172.21.15.78:0/3504387389 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36e80c7440 con 0x7f36e80d1160 2023-10-11T09:48:12.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.129+0000 7f36de7fc700 1 -- 172.21.15.78:0/3504387389 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e4011630 con 0x7f36e80d1160 2023-10-11T09:48:12.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.129+0000 7f36eeae4700 1 -- 172.21.15.78:0/3504387389 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e80c7a00 con 0x7f36e80d1160 2023-10-11T09:48:12.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.129+0000 7f36eeae4700 1 -- 172.21.15.78:0/3504387389 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36e8065d40 con 0x7f36e80d1160 2023-10-11T09:48:12.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.133+0000 7f36de7fc700 1 -- 172.21.15.78:0/3504387389 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f36e4021030 con 0x7f36e80d1160 2023-10-11T09:48:12.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.133+0000 7f36de7fc700 1 --2- 172.21.15.78:0/3504387389 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f36cc065ed0 0x7f36cc068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:12.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.133+0000 7f36de7fc700 1 -- 172.21.15.78:0/3504387389 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f36e40848d0 con 0x7f36e80d1160 2023-10-11T09:48:12.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.133+0000 7f36ed2e1700 1 --2- 172.21.15.78:0/3504387389 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f36cc065ed0 0x7f36cc068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:12.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.133+0000 7f36ed2e1700 1 --2- 172.21.15.78:0/3504387389 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f36cc065ed0 0x7f36cc068380 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f36e80b87e0 tx=0x7f36d8001230 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:12.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.141+0000 7f36de7fc700 1 -- 172.21.15.78:0/3504387389 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f36e4052220 con 0x7f36e80d1160 2023-10-11T09:48:12.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.377+0000 7f36eeae4700 1 -- 172.21.15.78:0/3504387389 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_many_files_many_workers"]} v 0) v1 -- 0x7f36e806d310 con 0x7f36e80d1160 2023-10-11T09:48:12.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.505+0000 7f36de7fc700 1 -- 172.21.15.78:0/3504387389 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_many_files_many_workers"]}]=0 v337) v1 ==== 172+0+0 (secure 0 0 0) 0x7f36e4055840 con 0x7f36e80d1160 2023-10-11T09:48:12.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.509+0000 7f36eeae4700 1 -- 172.21.15.78:0/3504387389 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f36cc065ed0 msgr2=0x7f36cc068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:12.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.509+0000 7f36eeae4700 1 --2- 172.21.15.78:0/3504387389 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f36cc065ed0 0x7f36cc068380 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f36e80b87e0 tx=0x7f36d8001230 comp rx=0 tx=0).stop 2023-10-11T09:48:12.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.509+0000 7f36eeae4700 1 -- 172.21.15.78:0/3504387389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36e80d1160 msgr2=0x7f36e81f4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:12.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.509+0000 7f36eeae4700 1 --2- 172.21.15.78:0/3504387389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36e80d1160 0x7f36e81f4930 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f36e4000c00 tx=0x7f36e4004770 comp rx=0 tx=0).stop 2023-10-11T09:48:12.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.509+0000 7f36eeae4700 1 -- 172.21.15.78:0/3504387389 shutdown_connections 2023-10-11T09:48:12.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.509+0000 7f36eeae4700 1 --2- 172.21.15.78:0/3504387389 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f36cc065ed0 0x7f36cc068380 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:12.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.509+0000 7f36eeae4700 1 --2- 172.21.15.78:0/3504387389 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36e8156ce0 0x7f36e81548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:12.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.509+0000 7f36eeae4700 1 --2- 172.21.15.78:0/3504387389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36e80d1660 0x7f36e80c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:12.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.509+0000 7f36eeae4700 1 --2- 172.21.15.78:0/3504387389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36e80d1160 0x7f36e81f4930 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:12.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.509+0000 7f36eeae4700 1 -- 172.21.15.78:0/3504387389 >> 172.21.15.78:0/3504387389 conn(0x7f36e806afb0 msgr2=0x7f36e80b5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:12.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.509+0000 7f36eeae4700 1 -- 172.21.15.78:0/3504387389 shutdown_connections 2023-10-11T09:48:12.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.509+0000 7f36eeae4700 1 -- 172.21.15.78:0/3504387389 wait complete. 2023-10-11T09:48:12.541 INFO:tasks.cephfs_test_runner:test_extra_data_pool_rebuild_many_files_many_workers (tasks.cephfs.test_data_scan.TestDataScan) ... ok 2023-10-11T09:48:12.541 INFO:tasks.cephfs_test_runner:Starting test: test_extra_data_pool_rebuild_simple (tasks.cephfs.test_data_scan.TestDataScan) 2023-10-11T09:48:12.542 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_simple' 2023-10-11T09:48:12.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.717+0000 7fcaa951f700 1 Processor -- start 2023-10-11T09:48:12.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.717+0000 7fcaa951f700 1 -- start start 2023-10-11T09:48:12.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.721+0000 7fcaa951f700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcaa40ce500 0x7fcaa415d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:12.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.721+0000 7fcaa951f700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fcaa4070100 0x7fcaa4070560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:12.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.721+0000 7fcaa951f700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaa4159c50 con 0x7fcaa4159490 2023-10-11T09:48:12.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.721+0000 7fcaa951f700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaa4159ef0 con 0x7fcaa4070100 2023-10-11T09:48:12.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.721+0000 7fcaa951f700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaa415a1d0 con 0x7fcaa40ce500 2023-10-11T09:48:12.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.721+0000 7fcaa37fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fcaa4070100 0x7fcaa4070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:12.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.721+0000 7fcaa27fc700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcaa40ce500 0x7fcaa415d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:12.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.721+0000 7fcaa37fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fcaa4070100 0x7fcaa4070560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40822/0 (socket says 172.21.15.78:40822) 2023-10-11T09:48:12.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.721+0000 7fcaa37fe700 1 -- 172.21.15.78:0/3029693932 learned_addr learned my addr 172.21.15.78:0/3029693932 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:12.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.721+0000 7fcaa2ffd700 1 --1- 172.21.15.78:0/3029693932 >> v1:172.21.15.133:6789/0 conn(0x7fcaa4159490 0x7fcaa415d030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:45820/0 (socket says 172.21.15.78:45820) 2023-10-11T09:48:12.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.725+0000 7fcaa37fe700 1 -- 172.21.15.78:0/3029693932 >> v2:172.21.15.133:3300/0 conn(0x7fcaa40ce500 msgr2=0x7fcaa415d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:12.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.725+0000 7fcaa37fe700 1 --2- 172.21.15.78:0/3029693932 >> v2:172.21.15.133:3300/0 conn(0x7fcaa40ce500 0x7fcaa415d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:12.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.725+0000 7fcaa37fe700 1 -- 172.21.15.78:0/3029693932 >> v1:172.21.15.133:6789/0 conn(0x7fcaa4159490 legacy=0x7fcaa415d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:12.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.725+0000 7fcaa1ffb700 1 -- 172.21.15.78:0/3029693932 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2718728201 0 0) 0x7fcaa4159c50 con 0x7fcaa4159490 2023-10-11T09:48:12.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.725+0000 7fcaa37fe700 1 -- 172.21.15.78:0/3029693932 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa415a4b0 con 0x7fcaa4070100 2023-10-11T09:48:12.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.725+0000 7fcaa37fe700 1 --2- 172.21.15.78:0/3029693932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa4070100 0x7fcaa4070560 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fcaa406f8d0 tx=0x7fca9c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=716eb7f5f225cbb5 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:12.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.725+0000 7fcaa1ffb700 1 -- 172.21.15.78:0/3029693932 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c00e070 con 0x7fcaa4070100 2023-10-11T09:48:12.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.725+0000 7fcaa1ffb700 1 -- 172.21.15.78:0/3029693932 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca9c00b910 con 0x7fcaa4070100 2023-10-11T09:48:12.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.725+0000 7fcaa1ffb700 1 -- 172.21.15.78:0/3029693932 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c00bcc0 con 0x7fcaa4070100 2023-10-11T09:48:12.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.725+0000 7fcaa951f700 1 -- 172.21.15.78:0/3029693932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa4070100 msgr2=0x7fcaa4070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:12.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.725+0000 7fcaa951f700 1 --2- 172.21.15.78:0/3029693932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa4070100 0x7fcaa4070560 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fcaa406f8d0 tx=0x7fca9c00b330 comp rx=0 tx=0).stop 2023-10-11T09:48:12.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.725+0000 7fcaa951f700 1 -- 172.21.15.78:0/3029693932 shutdown_connections 2023-10-11T09:48:12.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.725+0000 7fcaa951f700 1 --2- 172.21.15.78:0/3029693932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa4070100 0x7fcaa4070560 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:12.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.725+0000 7fcaa951f700 1 --2- 172.21.15.78:0/3029693932 >> v2:172.21.15.133:3300/0 conn(0x7fcaa40ce500 0x7fcaa415d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:12.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.725+0000 7fcaa951f700 1 -- 172.21.15.78:0/3029693932 >> 172.21.15.78:0/3029693932 conn(0x7fcaa406aca0 msgr2=0x7fcaa40b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:12.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.725+0000 7fcaa951f700 1 -- 172.21.15.78:0/3029693932 shutdown_connections 2023-10-11T09:48:12.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.725+0000 7fcaa951f700 1 -- 172.21.15.78:0/3029693932 wait complete. 2023-10-11T09:48:12.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.729+0000 7fcaa951f700 1 Processor -- start 2023-10-11T09:48:12.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.729+0000 7fcaa951f700 1 -- start start 2023-10-11T09:48:12.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.729+0000 7fcaa951f700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcaa4070100 0x7fcaa415bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:12.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.729+0000 7fcaa951f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcaa40ce500 0x7fcaa40cc130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:12.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.729+0000 7fcaa951f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa4159490 0x7fcaa41406e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:12.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.729+0000 7fcaa951f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa40cd420 con 0x7fcaa4159490 2023-10-11T09:48:12.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.729+0000 7fcaa951f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa40cc670 con 0x7fcaa40ce500 2023-10-11T09:48:12.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.729+0000 7fcaa951f700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa40b27f0 con 0x7fcaa4070100 2023-10-11T09:48:12.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.729+0000 7fcaa37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa4159490 0x7fcaa41406e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:12.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.729+0000 7fcaa2ffd700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcaa4070100 0x7fcaa415bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:12.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.729+0000 7fcaa37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa4159490 0x7fcaa41406e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40836/0 (socket says 172.21.15.78:40836) 2023-10-11T09:48:12.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.729+0000 7fcaa37fe700 1 -- 172.21.15.78:0/803977352 learned_addr learned my addr 172.21.15.78:0/803977352 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:12.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.729+0000 7fcaa27fc700 1 --2- 172.21.15.78:0/803977352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcaa40ce500 0x7fcaa40cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:12.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.733+0000 7fcaa37fe700 1 -- 172.21.15.78:0/803977352 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcaa4070100 msgr2=0x7fcaa415bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:12.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.733+0000 7fcaa37fe700 1 --2- 172.21.15.78:0/803977352 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcaa4070100 0x7fcaa415bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:12.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.733+0000 7fcaa37fe700 1 -- 172.21.15.78:0/803977352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcaa40ce500 msgr2=0x7fcaa40cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:12.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.733+0000 7fcaa37fe700 1 --2- 172.21.15.78:0/803977352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcaa40ce500 0x7fcaa40cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:12.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.733+0000 7fcaa37fe700 1 -- 172.21.15.78:0/803977352 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca9c009b30 con 0x7fcaa4159490 2023-10-11T09:48:12.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.733+0000 7fcaa37fe700 1 --2- 172.21.15.78:0/803977352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa4159490 0x7fcaa41406e0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fca9c00b680 tx=0x7fca9c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:12.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.733+0000 7fca8b7fe700 1 -- 172.21.15.78:0/803977352 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c0048b0 con 0x7fcaa4159490 2023-10-11T09:48:12.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.733+0000 7fca8b7fe700 1 -- 172.21.15.78:0/803977352 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca9c0099b0 con 0x7fcaa4159490 2023-10-11T09:48:12.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.733+0000 7fca8b7fe700 1 -- 172.21.15.78:0/803977352 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c01c680 con 0x7fcaa4159490 2023-10-11T09:48:12.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.733+0000 7fcaa951f700 1 -- 172.21.15.78:0/803977352 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaa4140c20 con 0x7fcaa4159490 2023-10-11T09:48:12.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.733+0000 7fcaa951f700 1 -- 172.21.15.78:0/803977352 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa40ccb80 con 0x7fcaa4159490 2023-10-11T09:48:12.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.733+0000 7fcaa951f700 1 -- 172.21.15.78:0/803977352 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaa40b29c0 con 0x7fcaa4159490 2023-10-11T09:48:12.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.737+0000 7fca8b7fe700 1 -- 172.21.15.78:0/803977352 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fca9c01c7e0 con 0x7fcaa4159490 2023-10-11T09:48:12.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.737+0000 7fca8b7fe700 1 --2- 172.21.15.78:0/803977352 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fca90065f00 0x7fca900683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:12.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.737+0000 7fca8b7fe700 1 -- 172.21.15.78:0/803977352 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fca9c0854c0 con 0x7fcaa4159490 2023-10-11T09:48:12.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.737+0000 7fcaa2ffd700 1 --2- 172.21.15.78:0/803977352 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fca90065f00 0x7fca900683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:12.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.741+0000 7fcaa2ffd700 1 --2- 172.21.15.78:0/803977352 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fca90065f00 0x7fca900683b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fca94007c60 tx=0x7fca94007bb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:12.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.741+0000 7fca8b7fe700 1 -- 172.21.15.78:0/803977352 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fca9c052f90 con 0x7fcaa4159490 2023-10-11T09:48:12.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:12.989+0000 7fcaa951f700 1 -- 172.21.15.78:0/803977352 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_simple"]} v 0) v1 -- 0x7fcaa404e530 con 0x7fcaa4159490 2023-10-11T09:48:13.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.509+0000 7fca8b7fe700 1 -- 172.21.15.78:0/803977352 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_simple"]}]=0 v338) v1 ==== 158+0+0 (secure 0 0 0) 0x7fca9c0565b0 con 0x7fcaa4159490 2023-10-11T09:48:13.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.513+0000 7fcaa951f700 1 -- 172.21.15.78:0/803977352 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fca90065f00 msgr2=0x7fca900683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:13.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.513+0000 7fcaa951f700 1 --2- 172.21.15.78:0/803977352 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fca90065f00 0x7fca900683b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fca94007c60 tx=0x7fca94007bb0 comp rx=0 tx=0).stop 2023-10-11T09:48:13.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.513+0000 7fcaa951f700 1 -- 172.21.15.78:0/803977352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa4159490 msgr2=0x7fcaa41406e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:13.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.513+0000 7fcaa951f700 1 --2- 172.21.15.78:0/803977352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa4159490 0x7fcaa41406e0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fca9c00b680 tx=0x7fca9c004af0 comp rx=0 tx=0).stop 2023-10-11T09:48:13.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.513+0000 7fcaa951f700 1 -- 172.21.15.78:0/803977352 shutdown_connections 2023-10-11T09:48:13.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.513+0000 7fcaa951f700 1 --2- 172.21.15.78:0/803977352 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fca90065f00 0x7fca900683b0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:13.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.513+0000 7fcaa951f700 1 --2- 172.21.15.78:0/803977352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa4159490 0x7fcaa41406e0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:13.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.513+0000 7fcaa951f700 1 --2- 172.21.15.78:0/803977352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcaa40ce500 0x7fcaa40cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:13.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.513+0000 7fcaa951f700 1 --2- 172.21.15.78:0/803977352 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcaa4070100 0x7fcaa415bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:13.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.513+0000 7fcaa951f700 1 -- 172.21.15.78:0/803977352 >> 172.21.15.78:0/803977352 conn(0x7fcaa406aca0 msgr2=0x7fcaa406c190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:13.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.513+0000 7fcaa951f700 1 -- 172.21.15.78:0/803977352 shutdown_connections 2023-10-11T09:48:13.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.513+0000 7fcaa951f700 1 -- 172.21.15.78:0/803977352 wait complete. 2023-10-11T09:48:13.546 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-11T09:48:13.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.717+0000 7fe1a6a25700 1 Processor -- start 2023-10-11T09:48:13.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.717+0000 7fe1a6a25700 1 -- start start 2023-10-11T09:48:13.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.717+0000 7fe1a6a25700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe1a00b1030 0x7fe1a00cb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:13.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.717+0000 7fe1a6a25700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe1a006d080 0x7fe1a006ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:13.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.721+0000 7fe1a6a25700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1a0165570 con 0x7fe1a0160cb0 2023-10-11T09:48:13.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.721+0000 7fe1a6a25700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1a0161080 con 0x7fe1a00b1030 2023-10-11T09:48:13.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.721+0000 7fe1a6a25700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1a0164750 con 0x7fe1a006d080 2023-10-11T09:48:13.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.721+0000 7fe19ffff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fe1a0160cb0 0x7fe1a00b0920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:40012/0 (socket says 172.21.15.78:40012) 2023-10-11T09:48:13.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.721+0000 7fe19ffff700 1 -- 172.21.15.78:0/671143181 learned_addr learned my addr 172.21.15.78:0/671143181 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:13.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.721+0000 7fe19f7fe700 1 --2- 172.21.15.78:0/671143181 >> v2:172.21.15.78:3300/0 conn(0x7fe1a00b1030 0x7fe1a00cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:13.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.721+0000 7fe1a4fc2700 1 --2- 172.21.15.78:0/671143181 >> v2:172.21.15.133:3300/0 conn(0x7fe1a006d080 0x7fe1a006ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:13.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.721+0000 7fe19effd700 1 -- 172.21.15.78:0/671143181 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3684793548 0 0) 0x7fe1a0165570 con 0x7fe1a0160cb0 2023-10-11T09:48:13.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.721+0000 7fe19effd700 1 -- 172.21.15.78:0/671143181 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1880035d0 con 0x7fe1a0160cb0 2023-10-11T09:48:13.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.721+0000 7fe19f7fe700 1 -- 172.21.15.78:0/671143181 >> v2:172.21.15.133:3300/0 conn(0x7fe1a006d080 msgr2=0x7fe1a006ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:13.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.721+0000 7fe19f7fe700 1 --2- 172.21.15.78:0/671143181 >> v2:172.21.15.133:3300/0 conn(0x7fe1a006d080 0x7fe1a006ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:13.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.721+0000 7fe19f7fe700 1 -- 172.21.15.78:0/671143181 >> v1:172.21.15.78:6789/0 conn(0x7fe1a0160cb0 legacy=0x7fe1a00b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:13.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.721+0000 7fe19f7fe700 1 -- 172.21.15.78:0/671143181 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a0164960 con 0x7fe1a00b1030 2023-10-11T09:48:13.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.721+0000 7fe19f7fe700 1 --2- 172.21.15.78:0/671143181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1a00b1030 0x7fe1a00cb100 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fe1a00726a0 tx=0x7fe194009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fb8b3108f07933c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:13.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.721+0000 7fe19effd700 1 -- 172.21.15.78:0/671143181 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe194011830 con 0x7fe1a00b1030 2023-10-11T09:48:13.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.721+0000 7fe19effd700 1 -- 172.21.15.78:0/671143181 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe194011990 con 0x7fe1a00b1030 2023-10-11T09:48:13.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.721+0000 7fe19effd700 1 -- 172.21.15.78:0/671143181 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe194011d20 con 0x7fe1a00b1030 2023-10-11T09:48:13.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.721+0000 7fe1a6a25700 1 -- 172.21.15.78:0/671143181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1a00b1030 msgr2=0x7fe1a00cb100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:13.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.721+0000 7fe1a6a25700 1 --2- 172.21.15.78:0/671143181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1a00b1030 0x7fe1a00cb100 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fe1a00726a0 tx=0x7fe194009b30 comp rx=0 tx=0).stop 2023-10-11T09:48:13.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.721+0000 7fe1a6a25700 1 -- 172.21.15.78:0/671143181 shutdown_connections 2023-10-11T09:48:13.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.721+0000 7fe1a6a25700 1 --2- 172.21.15.78:0/671143181 >> v2:172.21.15.133:3300/0 conn(0x7fe1a006d080 0x7fe1a006ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:13.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.721+0000 7fe1a6a25700 1 --2- 172.21.15.78:0/671143181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1a00b1030 0x7fe1a00cb100 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:13.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.725+0000 7fe1a6a25700 1 -- 172.21.15.78:0/671143181 >> 172.21.15.78:0/671143181 conn(0x7fe1a00cecc0 msgr2=0x7fe1a00cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:13.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.725+0000 7fe1a6a25700 1 -- 172.21.15.78:0/671143181 shutdown_connections 2023-10-11T09:48:13.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.725+0000 7fe1a6a25700 1 -- 172.21.15.78:0/671143181 wait complete. 2023-10-11T09:48:13.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.725+0000 7fe1a6a25700 1 Processor -- start 2023-10-11T09:48:13.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.729+0000 7fe1a6a25700 1 -- start start 2023-10-11T09:48:13.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.729+0000 7fe1a6a25700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1a006d080 0x7fe1a01ea560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:13.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.729+0000 7fe1a6a25700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1a00b1030 0x7fe1a01ece80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:13.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.729+0000 7fe1a6a25700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1a0160cb0 0x7fe1a0148000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:13.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.729+0000 7fe1a6a25700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a0155730 con 0x7fe1a006d080 2023-10-11T09:48:13.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.729+0000 7fe1a6a25700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a01eab40 con 0x7fe1a00b1030 2023-10-11T09:48:13.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.729+0000 7fe1a6a25700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a01eae20 con 0x7fe1a0160cb0 2023-10-11T09:48:13.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.729+0000 7fe19ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1a006d080 0x7fe1a01ea560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:13.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.729+0000 7fe19ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1a006d080 0x7fe1a01ea560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40856/0 (socket says 172.21.15.78:40856) 2023-10-11T09:48:13.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.729+0000 7fe19ffff700 1 -- 172.21.15.78:0/4166182685 learned_addr learned my addr 172.21.15.78:0/4166182685 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:13.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.729+0000 7fe19f7fe700 1 --2- 172.21.15.78:0/4166182685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1a00b1030 0x7fe1a01ece80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:13.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.729+0000 7fe1a4fc2700 1 --2- 172.21.15.78:0/4166182685 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1a0160cb0 0x7fe1a0148000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:13.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.729+0000 7fe19ffff700 1 -- 172.21.15.78:0/4166182685 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1a0160cb0 msgr2=0x7fe1a0148000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:13.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.729+0000 7fe19ffff700 1 --2- 172.21.15.78:0/4166182685 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1a0160cb0 0x7fe1a0148000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:13.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.729+0000 7fe19ffff700 1 -- 172.21.15.78:0/4166182685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1a00b1030 msgr2=0x7fe1a01ece80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:13.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.729+0000 7fe19ffff700 1 --2- 172.21.15.78:0/4166182685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1a00b1030 0x7fe1a01ece80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:13.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.729+0000 7fe19ffff700 1 -- 172.21.15.78:0/4166182685 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1940094b0 con 0x7fe1a006d080 2023-10-11T09:48:13.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.729+0000 7fe19ffff700 1 --2- 172.21.15.78:0/4166182685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1a006d080 0x7fe1a01ea560 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fe19000c4d0 tx=0x7fe19000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:13.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.729+0000 7fe19cff9700 1 -- 172.21.15.78:0/4166182685 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe190005d90 con 0x7fe1a006d080 2023-10-11T09:48:13.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.729+0000 7fe19cff9700 1 -- 172.21.15.78:0/4166182685 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1900070d0 con 0x7fe1a006d080 2023-10-11T09:48:13.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.729+0000 7fe19cff9700 1 -- 172.21.15.78:0/4166182685 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe190007480 con 0x7fe1a006d080 2023-10-11T09:48:13.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.729+0000 7fe1a6a25700 1 -- 172.21.15.78:0/4166182685 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1a01eb100 con 0x7fe1a006d080 2023-10-11T09:48:13.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.733+0000 7fe1a6a25700 1 -- 172.21.15.78:0/4166182685 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1a01eb6c0 con 0x7fe1a006d080 2023-10-11T09:48:13.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.733+0000 7fe1a6a25700 1 -- 172.21.15.78:0/4166182685 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1a006eb80 con 0x7fe1a006d080 2023-10-11T09:48:13.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.733+0000 7fe19cff9700 1 -- 172.21.15.78:0/4166182685 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fe1900075e0 con 0x7fe1a006d080 2023-10-11T09:48:13.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.737+0000 7fe19cff9700 1 --2- 172.21.15.78:0/4166182685 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe18c065ed0 0x7fe18c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:13.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.737+0000 7fe19cff9700 1 -- 172.21.15.78:0/4166182685 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fe1900829f0 con 0x7fe1a006d080 2023-10-11T09:48:13.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.737+0000 7fe19f7fe700 1 --2- 172.21.15.78:0/4166182685 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe18c065ed0 0x7fe18c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:13.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.737+0000 7fe19f7fe700 1 --2- 172.21.15.78:0/4166182685 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe18c065ed0 0x7fe18c068380 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe1a00b1420 tx=0x7fe194008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:13.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.741+0000 7fe19cff9700 1 -- 172.21.15.78:0/4166182685 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fe190050240 con 0x7fe1a006d080 2023-10-11T09:48:13.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.917+0000 7fe1a6a25700 1 -- 172.21.15.78:0/4166182685 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe1a0071540 con 0x7fe1a006d080 2023-10-11T09:48:13.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.925+0000 7fe19cff9700 1 -- 172.21.15.78:0/4166182685 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe190053860 con 0x7fe1a006d080 2023-10-11T09:48:13.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.929+0000 7fe1a6a25700 1 -- 172.21.15.78:0/4166182685 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe18c065ed0 msgr2=0x7fe18c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:13.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.929+0000 7fe1a6a25700 1 --2- 172.21.15.78:0/4166182685 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe18c065ed0 0x7fe18c068380 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe1a00b1420 tx=0x7fe194008030 comp rx=0 tx=0).stop 2023-10-11T09:48:13.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.929+0000 7fe1a6a25700 1 -- 172.21.15.78:0/4166182685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1a006d080 msgr2=0x7fe1a01ea560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:13.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.929+0000 7fe1a6a25700 1 --2- 172.21.15.78:0/4166182685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1a006d080 0x7fe1a01ea560 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fe19000c4d0 tx=0x7fe19000c7e0 comp rx=0 tx=0).stop 2023-10-11T09:48:13.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.929+0000 7fe1a6a25700 1 -- 172.21.15.78:0/4166182685 shutdown_connections 2023-10-11T09:48:13.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.929+0000 7fe1a6a25700 1 --2- 172.21.15.78:0/4166182685 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe18c065ed0 0x7fe18c068380 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:13.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.929+0000 7fe1a6a25700 1 --2- 172.21.15.78:0/4166182685 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1a0160cb0 0x7fe1a0148000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:13.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.929+0000 7fe1a6a25700 1 --2- 172.21.15.78:0/4166182685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1a00b1030 0x7fe1a01ece80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:13.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.929+0000 7fe1a6a25700 1 --2- 172.21.15.78:0/4166182685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1a006d080 0x7fe1a01ea560 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:13.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.929+0000 7fe1a6a25700 1 -- 172.21.15.78:0/4166182685 >> 172.21.15.78:0/4166182685 conn(0x7fe1a00cecc0 msgr2=0x7fe1a00cba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:13.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.933+0000 7fe1a6a25700 1 -- 172.21.15.78:0/4166182685 shutdown_connections 2023-10-11T09:48:13.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:13.933+0000 7fe1a6a25700 1 -- 172.21.15.78:0/4166182685 wait complete. 2023-10-11T09:48:13.961 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-11T09:48:13.962 DEBUG:teuthology.orchestra.run.smithi078:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-11T09:48:14.002 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-11T09:48:14.044 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-11T09:48:14.045 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:48:14.045 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:48:14.065 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352272k,nr_inodes=4088068,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17487 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-11T09:48:14.066 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:48:14.066 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:48:14.101 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352292k,nr_inodes=4088073,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1430 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275376k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-11T09:48:14.102 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:48:14.102 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:48:14.129 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352272k,nr_inodes=4088068,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17487 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-11T09:48:14.130 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:48:14.130 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:48:14.163 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352292k,nr_inodes=4088073,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1430 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275376k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-11T09:48:14.164 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-11T09:48:14.164 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:48:14.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.333+0000 7fb0be5c3700 1 Processor -- start 2023-10-11T09:48:14.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.333+0000 7fb0be5c3700 1 -- start start 2023-10-11T09:48:14.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.333+0000 7fb0be5c3700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb0b80caf20 0x7fb0b8164610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:14.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.333+0000 7fb0be5c3700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fb0b8164b50 0x7fb0b806bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:14.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.333+0000 7fb0be5c3700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0b8165390 con 0x7fb0b80cda10 2023-10-11T09:48:14.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.333+0000 7fb0be5c3700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0b80cb3a0 con 0x7fb0b8164b50 2023-10-11T09:48:14.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.333+0000 7fb0be5c3700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0b80ca440 con 0x7fb0b80caf20 2023-10-11T09:48:14.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.337+0000 7fb0bddc2700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fb0b8164b50 0x7fb0b806bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:14.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.337+0000 7fb0bddc2700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fb0b8164b50 0x7fb0b806bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40866/0 (socket says 172.21.15.78:40866) 2023-10-11T09:48:14.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.337+0000 7fb0bcdc0700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb0b80caf20 0x7fb0b8164610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:14.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.337+0000 7fb0bddc2700 1 -- 172.21.15.78:0/3442150447 learned_addr learned my addr 172.21.15.78:0/3442150447 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:14.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.337+0000 7fb0bddc2700 1 -- 172.21.15.78:0/3442150447 >> v2:172.21.15.133:3300/0 conn(0x7fb0b80caf20 msgr2=0x7fb0b8164610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:14.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.337+0000 7fb0bddc2700 1 --2- 172.21.15.78:0/3442150447 >> v2:172.21.15.133:3300/0 conn(0x7fb0b80caf20 0x7fb0b8164610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:14.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.337+0000 7fb0bddc2700 1 -- 172.21.15.78:0/3442150447 >> v1:172.21.15.133:6789/0 conn(0x7fb0b80cda10 legacy=0x7fb0b80cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:14.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.337+0000 7fb0bddc2700 1 -- 172.21.15.78:0/3442150447 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b80ca720 con 0x7fb0b8164b50 2023-10-11T09:48:14.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.337+0000 7fb0bddc2700 1 --2- 172.21.15.78:0/3442150447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0b8164b50 0x7fb0b806bbf0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80cea10 tx=0x7fb0b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93de458dfbda2844 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:14.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.337+0000 7fb0affff700 1 -- 172.21.15.78:0/3442150447 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b000e070 con 0x7fb0b8164b50 2023-10-11T09:48:14.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.337+0000 7fb0affff700 1 -- 172.21.15.78:0/3442150447 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b000b910 con 0x7fb0b8164b50 2023-10-11T09:48:14.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.337+0000 7fb0affff700 1 -- 172.21.15.78:0/3442150447 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b000bcc0 con 0x7fb0b8164b50 2023-10-11T09:48:14.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.337+0000 7fb0be5c3700 1 -- 172.21.15.78:0/3442150447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0b8164b50 msgr2=0x7fb0b806bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:14.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.337+0000 7fb0be5c3700 1 --2- 172.21.15.78:0/3442150447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0b8164b50 0x7fb0b806bbf0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80cea10 tx=0x7fb0b000b330 comp rx=0 tx=0).stop 2023-10-11T09:48:14.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.337+0000 7fb0be5c3700 1 -- 172.21.15.78:0/3442150447 shutdown_connections 2023-10-11T09:48:14.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.337+0000 7fb0be5c3700 1 --2- 172.21.15.78:0/3442150447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0b8164b50 0x7fb0b806bbf0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:14.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.337+0000 7fb0be5c3700 1 --2- 172.21.15.78:0/3442150447 >> v2:172.21.15.133:3300/0 conn(0x7fb0b80caf20 0x7fb0b8164610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:14.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.337+0000 7fb0be5c3700 1 -- 172.21.15.78:0/3442150447 >> 172.21.15.78:0/3442150447 conn(0x7fb0b80cea60 msgr2=0x7fb0b80cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:14.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.341+0000 7fb0be5c3700 1 -- 172.21.15.78:0/3442150447 shutdown_connections 2023-10-11T09:48:14.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.341+0000 7fb0be5c3700 1 -- 172.21.15.78:0/3442150447 wait complete. 2023-10-11T09:48:14.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.341+0000 7fb0be5c3700 1 Processor -- start 2023-10-11T09:48:14.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.341+0000 7fb0be5c3700 1 -- start start 2023-10-11T09:48:14.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.345+0000 7fb0be5c3700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb0b80caf20 0x7fb0b813f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:14.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.345+0000 7fb0be5c3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0b80cda10 0x7fb0b80b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:14.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.345+0000 7fb0be5c3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0b8164b50 0x7fb0b80b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:14.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.345+0000 7fb0be5c3700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b81f63d0 con 0x7fb0b8164b50 2023-10-11T09:48:14.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.345+0000 7fb0be5c3700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b8163a50 con 0x7fb0b80cda10 2023-10-11T09:48:14.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.345+0000 7fb0be5c3700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b813fae0 con 0x7fb0b80caf20 2023-10-11T09:48:14.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.345+0000 7fb0bddc2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0b8164b50 0x7fb0b80b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:14.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.345+0000 7fb0bddc2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0b8164b50 0x7fb0b80b36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40872/0 (socket says 172.21.15.78:40872) 2023-10-11T09:48:14.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.345+0000 7fb0bddc2700 1 -- 172.21.15.78:0/307338211 learned_addr learned my addr 172.21.15.78:0/307338211 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:14.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.345+0000 7fb0bd5c1700 1 --2- 172.21.15.78:0/307338211 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb0b80caf20 0x7fb0b813f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:14.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.345+0000 7fb0bcdc0700 1 --2- 172.21.15.78:0/307338211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0b80cda10 0x7fb0b80b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:14.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.345+0000 7fb0bddc2700 1 -- 172.21.15.78:0/307338211 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb0b80caf20 msgr2=0x7fb0b813f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:14.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.345+0000 7fb0bddc2700 1 --2- 172.21.15.78:0/307338211 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb0b80caf20 0x7fb0b813f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:14.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.345+0000 7fb0bddc2700 1 -- 172.21.15.78:0/307338211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0b80cda10 msgr2=0x7fb0b80b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:14.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.345+0000 7fb0bddc2700 1 --2- 172.21.15.78:0/307338211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0b80cda10 0x7fb0b80b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:14.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.345+0000 7fb0bddc2700 1 -- 172.21.15.78:0/307338211 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b0009b30 con 0x7fb0b8164b50 2023-10-11T09:48:14.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.345+0000 7fb0bcdc0700 1 --2- 172.21.15.78:0/307338211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0b80cda10 0x7fb0b80b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:48:14.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.345+0000 7fb0bd5c1700 1 --2- 172.21.15.78:0/307338211 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb0b80caf20 0x7fb0b813f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:48:14.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.345+0000 7fb0bddc2700 1 --2- 172.21.15.78:0/307338211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0b8164b50 0x7fb0b80b36d0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fb0b000b680 tx=0x7fb0b0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:14.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.345+0000 7fb0adffb700 1 -- 172.21.15.78:0/307338211 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b00099b0 con 0x7fb0b8164b50 2023-10-11T09:48:14.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.345+0000 7fb0adffb700 1 -- 172.21.15.78:0/307338211 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b00092e0 con 0x7fb0b8164b50 2023-10-11T09:48:14.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.345+0000 7fb0be5c3700 1 -- 172.21.15.78:0/307338211 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0b8163490 con 0x7fb0b8164b50 2023-10-11T09:48:14.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.345+0000 7fb0adffb700 1 -- 172.21.15.78:0/307338211 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b001b630 con 0x7fb0b8164b50 2023-10-11T09:48:14.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.345+0000 7fb0be5c3700 1 -- 172.21.15.78:0/307338211 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0b81f65e0 con 0x7fb0b8164b50 2023-10-11T09:48:14.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.349+0000 7fb0be5c3700 1 -- 172.21.15.78:0/307338211 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0b8067f50 con 0x7fb0b8164b50 2023-10-11T09:48:14.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.349+0000 7fb0adffb700 1 -- 172.21.15.78:0/307338211 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fb0b001b790 con 0x7fb0b8164b50 2023-10-11T09:48:14.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.353+0000 7fb0adffb700 1 --2- 172.21.15.78:0/307338211 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb09c065ed0 0x7fb09c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:14.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.353+0000 7fb0adffb700 1 -- 172.21.15.78:0/307338211 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fb0b0084c00 con 0x7fb0b8164b50 2023-10-11T09:48:14.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.353+0000 7fb0bd5c1700 1 --2- 172.21.15.78:0/307338211 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb09c065ed0 0x7fb09c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:14.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.353+0000 7fb0bd5c1700 1 --2- 172.21.15.78:0/307338211 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb09c065ed0 0x7fb09c068380 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4000f60 tx=0x7fb0b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:14.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.357+0000 7fb0adffb700 1 -- 172.21.15.78:0/307338211 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fb0b00524e0 con 0x7fb0b8164b50 2023-10-11T09:48:14.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.565+0000 7fb0be5c3700 1 -- 172.21.15.78:0/307338211 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0b80c9680 con 0x7fb0b8164b50 2023-10-11T09:48:14.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.565+0000 7fb0adffb700 1 -- 172.21.15.78:0/307338211 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+3532 (secure 0 0 0) 0x7fb0b001fde0 con 0x7fb0b8164b50 2023-10-11T09:48:14.574 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:14.574 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":52,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6664,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2877345875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2877345875},{"type":"v1","addr":"172.21.15.133:6833","nonce":2877345875}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":6667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/792033922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":792033922},{"type":"v1","addr":"172.21.15.133:6835","nonce":792033922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":6894,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/1229638118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":1229638118},{"type":"v1","addr":"172.21.15.78:6836","nonce":1229638118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":7155,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/2797163841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":2797163841},{"type":"v1","addr":"172.21.15.78:6837","nonce":2797163841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[]} 2023-10-11T09:48:14.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.569+0000 7fb0be5c3700 1 -- 172.21.15.78:0/307338211 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb09c065ed0 msgr2=0x7fb09c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:14.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.569+0000 7fb0be5c3700 1 --2- 172.21.15.78:0/307338211 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb09c065ed0 0x7fb09c068380 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4000f60 tx=0x7fb0b4009040 comp rx=0 tx=0).stop 2023-10-11T09:48:14.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.569+0000 7fb0be5c3700 1 -- 172.21.15.78:0/307338211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0b8164b50 msgr2=0x7fb0b80b36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:14.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.569+0000 7fb0be5c3700 1 --2- 172.21.15.78:0/307338211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0b8164b50 0x7fb0b80b36d0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fb0b000b680 tx=0x7fb0b0004af0 comp rx=0 tx=0).stop 2023-10-11T09:48:14.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.569+0000 7fb0be5c3700 1 -- 172.21.15.78:0/307338211 shutdown_connections 2023-10-11T09:48:14.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.569+0000 7fb0be5c3700 1 --2- 172.21.15.78:0/307338211 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb09c065ed0 0x7fb09c068380 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:14.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.569+0000 7fb0be5c3700 1 --2- 172.21.15.78:0/307338211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0b8164b50 0x7fb0b80b36d0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:14.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.569+0000 7fb0be5c3700 1 --2- 172.21.15.78:0/307338211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0b80cda10 0x7fb0b80b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:14.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.569+0000 7fb0be5c3700 1 --2- 172.21.15.78:0/307338211 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb0b80caf20 0x7fb0b813f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:14.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.569+0000 7fb0be5c3700 1 -- 172.21.15.78:0/307338211 >> 172.21.15.78:0/307338211 conn(0x7fb0b80cea60 msgr2=0x7fb0b806fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:14.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.569+0000 7fb0be5c3700 1 -- 172.21.15.78:0/307338211 shutdown_connections 2023-10-11T09:48:14.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.569+0000 7fb0be5c3700 1 -- 172.21.15.78:0/307338211 wait complete. 2023-10-11T09:48:14.582 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 52 2023-10-11T09:48:14.601 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-11T09:48:14.708 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-11T09:48:14.709 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-11T09:48:14.709 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:48:14.709 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-11T09:48:14.712 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-11T09:48:14.713 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-11T09:48:14.713 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:48:14.714 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-11T09:48:14.715 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-11T09:48:14.716 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-11T09:48:14.716 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:48:14.718 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-11T09:48:14.718 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-11T09:48:14.719 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-11T09:48:14.719 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:48:14.734 INFO:tasks.ceph.mds.a:Stopped 2023-10-11T09:48:14.735 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-11T09:48:14.737 INFO:tasks.ceph.mds.c:Stopped 2023-10-11T09:48:14.737 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-11T09:48:14.743 INFO:tasks.ceph.mds.b:Stopped 2023-10-11T09:48:14.744 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-11T09:48:14.745 INFO:tasks.ceph.mds.d:Stopped 2023-10-11T09:48:14.745 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-11T09:48:14.786 INFO:tasks.ceph.mds.a:Started 2023-10-11T09:48:14.787 DEBUG:teuthology.parallel:result is None 2023-10-11T09:48:14.790 INFO:tasks.ceph.mds.c:Started 2023-10-11T09:48:14.790 DEBUG:teuthology.parallel:result is None 2023-10-11T09:48:14.796 INFO:tasks.ceph.mds.b:Started 2023-10-11T09:48:14.797 DEBUG:teuthology.parallel:result is None 2023-10-11T09:48:14.801 INFO:tasks.ceph.mds.d:Started 2023-10-11T09:48:14.802 DEBUG:teuthology.parallel:result is None 2023-10-11T09:48:14.802 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-11T09:48:14.901 INFO:tasks.ceph.mds.a.smithi078.stdout:starting mds.a at 2023-10-11T09:48:14.902 INFO:tasks.ceph.mds.c.smithi078.stdout:starting mds.c at 2023-10-11T09:48:14.919 INFO:tasks.ceph.mds.b.smithi133.stdout:starting mds.b at 2023-10-11T09:48:14.920 INFO:tasks.ceph.mds.d.smithi133.stdout:starting mds.d at 2023-10-11T09:48:14.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.961+0000 7fe290e8c700 1 Processor -- start 2023-10-11T09:48:14.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.961+0000 7fe290e8c700 1 -- start start 2023-10-11T09:48:14.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.961+0000 7fe290e8c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe28c0ce8c0 0x7fe28c0cb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:14.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.961+0000 7fe290e8c700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe28c0b2800 0x7fe28c0cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:14.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.961+0000 7fe290e8c700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe28c15ad70 con 0x7fe28c0cf1d0 2023-10-11T09:48:14.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.961+0000 7fe290e8c700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe28c0cf650 con 0x7fe28c0ce8c0 2023-10-11T09:48:14.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.961+0000 7fe290e8c700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe28c0d1c40 con 0x7fe28c0b2800 2023-10-11T09:48:14.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.961+0000 7fe28b7fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe28c0ce8c0 0x7fe28c0cb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:14.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.961+0000 7fe28bfff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fe28c0cf1d0 0x7fe28c06f190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:48096/0 (socket says 172.21.15.78:48096) 2023-10-11T09:48:14.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.961+0000 7fe28bfff700 1 -- 172.21.15.78:0/807717234 learned_addr learned my addr 172.21.15.78:0/807717234 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:14.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.965+0000 7fe28affd700 1 --2- 172.21.15.78:0/807717234 >> v2:172.21.15.133:3300/0 conn(0x7fe28c0b2800 0x7fe28c0cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:14.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.965+0000 7fe28b7fe700 1 -- 172.21.15.78:0/807717234 >> v2:172.21.15.133:3300/0 conn(0x7fe28c0b2800 msgr2=0x7fe28c0cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:14.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.965+0000 7fe28b7fe700 1 --2- 172.21.15.78:0/807717234 >> v2:172.21.15.133:3300/0 conn(0x7fe28c0b2800 0x7fe28c0cec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:14.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.965+0000 7fe28b7fe700 1 -- 172.21.15.78:0/807717234 >> v1:172.21.15.78:6789/0 conn(0x7fe28c0cf1d0 legacy=0x7fe28c06f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:14.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.965+0000 7fe28b7fe700 1 -- 172.21.15.78:0/807717234 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe28c06e0f0 con 0x7fe28c0ce8c0 2023-10-11T09:48:14.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.965+0000 7fe28a7fc700 1 -- 172.21.15.78:0/807717234 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3597565883 0 0) 0x7fe284001490 con 0x7fe28c0cf1d0 2023-10-11T09:48:14.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.965+0000 7fe28b7fe700 1 --2- 172.21.15.78:0/807717234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe28c0ce8c0 0x7fe28c0cb340 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fe28c071760 tx=0x7fe27c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bae1ba1f078076d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:14.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.965+0000 7fe28a7fc700 1 -- 172.21.15.78:0/807717234 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c005e50 con 0x7fe28c0ce8c0 2023-10-11T09:48:14.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.965+0000 7fe28a7fc700 1 -- 172.21.15.78:0/807717234 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe27c006030 con 0x7fe28c0ce8c0 2023-10-11T09:48:14.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.965+0000 7fe28a7fc700 1 -- 172.21.15.78:0/807717234 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c0063c0 con 0x7fe28c0ce8c0 2023-10-11T09:48:14.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.965+0000 7fe290e8c700 1 -- 172.21.15.78:0/807717234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe28c0ce8c0 msgr2=0x7fe28c0cb340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:14.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.965+0000 7fe290e8c700 1 --2- 172.21.15.78:0/807717234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe28c0ce8c0 0x7fe28c0cb340 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fe28c071760 tx=0x7fe27c009b20 comp rx=0 tx=0).stop 2023-10-11T09:48:14.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.965+0000 7fe290e8c700 1 -- 172.21.15.78:0/807717234 shutdown_connections 2023-10-11T09:48:14.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.965+0000 7fe290e8c700 1 --2- 172.21.15.78:0/807717234 >> v2:172.21.15.133:3300/0 conn(0x7fe28c0b2800 0x7fe28c0cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:14.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.965+0000 7fe290e8c700 1 --2- 172.21.15.78:0/807717234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe28c0ce8c0 0x7fe28c0cb340 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:14.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.965+0000 7fe290e8c700 1 -- 172.21.15.78:0/807717234 >> 172.21.15.78:0/807717234 conn(0x7fe28c09a130 msgr2=0x7fe28c0d0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:14.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.965+0000 7fe290e8c700 1 -- 172.21.15.78:0/807717234 shutdown_connections 2023-10-11T09:48:14.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.965+0000 7fe290e8c700 1 -- 172.21.15.78:0/807717234 wait complete. 2023-10-11T09:48:14.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.969+0000 7fe290e8c700 1 Processor -- start 2023-10-11T09:48:14.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.969+0000 7fe290e8c700 1 -- start start 2023-10-11T09:48:14.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.969+0000 7fe290e8c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe28c0b2800 0x7fe28c155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:14.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.969+0000 7fe290e8c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe28c0ce8c0 0x7fe28c0b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:14.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.969+0000 7fe290e8c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe28c0cf1d0 0x7fe28c0b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:14.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.969+0000 7fe290e8c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28c15b320 con 0x7fe28c0cf1d0 2023-10-11T09:48:14.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.969+0000 7fe290e8c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28c0cd340 con 0x7fe28c0b2800 2023-10-11T09:48:14.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.969+0000 7fe290e8c700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe28c0b4ce0 con 0x7fe28c0ce8c0 2023-10-11T09:48:14.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.969+0000 7fe28bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe28c0cf1d0 0x7fe28c0b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:14.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.969+0000 7fe28b7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe28c0b2800 0x7fe28c155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:14.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.969+0000 7fe28bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe28c0cf1d0 0x7fe28c0b40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33746/0 (socket says 172.21.15.78:33746) 2023-10-11T09:48:14.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.969+0000 7fe28bfff700 1 -- 172.21.15.78:0/2851515565 learned_addr learned my addr 172.21.15.78:0/2851515565 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:14.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.969+0000 7fe28affd700 1 --2- 172.21.15.78:0/2851515565 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe28c0ce8c0 0x7fe28c0b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:14.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.973+0000 7fe28bfff700 1 -- 172.21.15.78:0/2851515565 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe28c0ce8c0 msgr2=0x7fe28c0b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:14.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.973+0000 7fe28bfff700 1 --2- 172.21.15.78:0/2851515565 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe28c0ce8c0 0x7fe28c0b3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:14.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.973+0000 7fe28bfff700 1 -- 172.21.15.78:0/2851515565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe28c0b2800 msgr2=0x7fe28c155670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:14.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.973+0000 7fe28bfff700 1 --2- 172.21.15.78:0/2851515565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe28c0b2800 0x7fe28c155670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:14.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.973+0000 7fe28bfff700 1 -- 172.21.15.78:0/2851515565 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe27c009450 con 0x7fe28c0cf1d0 2023-10-11T09:48:14.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.973+0000 7fe28bfff700 1 --2- 172.21.15.78:0/2851515565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe28c0cf1d0 0x7fe28c0b40f0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fe28400cca0 tx=0x7fe28400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:14.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.973+0000 7fe273fff700 1 -- 172.21.15.78:0/2851515565 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28400ab10 con 0x7fe28c0cf1d0 2023-10-11T09:48:14.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.973+0000 7fe273fff700 1 -- 172.21.15.78:0/2851515565 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28400ac90 con 0x7fe28c0cf1d0 2023-10-11T09:48:14.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.973+0000 7fe273fff700 1 -- 172.21.15.78:0/2851515565 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2840070d0 con 0x7fe28c0cf1d0 2023-10-11T09:48:14.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.973+0000 7fe290e8c700 1 -- 172.21.15.78:0/2851515565 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe28c0b4720 con 0x7fe28c0cf1d0 2023-10-11T09:48:14.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.973+0000 7fe290e8c700 1 -- 172.21.15.78:0/2851515565 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe28c15afb0 con 0x7fe28c0cf1d0 2023-10-11T09:48:14.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.973+0000 7fe290e8c700 1 -- 172.21.15.78:0/2851515565 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe28c065d40 con 0x7fe28c0cf1d0 2023-10-11T09:48:14.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.977+0000 7fe273fff700 1 -- 172.21.15.78:0/2851515565 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fe284007250 con 0x7fe28c0cf1d0 2023-10-11T09:48:14.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.977+0000 7fe273fff700 1 --2- 172.21.15.78:0/2851515565 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe278065ed0 0x7fe278068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:14.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.977+0000 7fe273fff700 1 -- 172.21.15.78:0/2851515565 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fe284082ec0 con 0x7fe28c0cf1d0 2023-10-11T09:48:14.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.977+0000 7fe28b7fe700 1 --2- 172.21.15.78:0/2851515565 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe278065ed0 0x7fe278068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:14.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.977+0000 7fe28b7fe700 1 --2- 172.21.15.78:0/2851515565 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe278065ed0 0x7fe278068380 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe27c009b60 tx=0x7fe27c008070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:14.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:14.985+0000 7fe273fff700 1 -- 172.21.15.78:0/2851515565 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fe284050700 con 0x7fe28c0cf1d0 2023-10-11T09:48:15.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:15.169+0000 7fe290e8c700 1 -- 172.21.15.78:0/2851515565 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fe28c06e680 con 0x7fe28c0cf1d0 2023-10-11T09:48:15.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:15.961+0000 7fe273fff700 1 -- 172.21.15.78:0/2851515565 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v79) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe284018080 con 0x7fe28c0cf1d0 2023-10-11T09:48:15.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:15.965+0000 7fe290e8c700 1 -- 172.21.15.78:0/2851515565 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe278065ed0 msgr2=0x7fe278068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:15.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:15.965+0000 7fe290e8c700 1 --2- 172.21.15.78:0/2851515565 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe278065ed0 0x7fe278068380 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe27c009b60 tx=0x7fe27c008070 comp rx=0 tx=0).stop 2023-10-11T09:48:15.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:15.965+0000 7fe290e8c700 1 -- 172.21.15.78:0/2851515565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe28c0cf1d0 msgr2=0x7fe28c0b40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:15.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:15.965+0000 7fe290e8c700 1 --2- 172.21.15.78:0/2851515565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe28c0cf1d0 0x7fe28c0b40f0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fe28400cca0 tx=0x7fe28400a060 comp rx=0 tx=0).stop 2023-10-11T09:48:15.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:15.965+0000 7fe290e8c700 1 -- 172.21.15.78:0/2851515565 shutdown_connections 2023-10-11T09:48:15.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:15.965+0000 7fe290e8c700 1 --2- 172.21.15.78:0/2851515565 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe278065ed0 0x7fe278068380 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:15.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:15.965+0000 7fe290e8c700 1 --2- 172.21.15.78:0/2851515565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe28c0cf1d0 0x7fe28c0b40f0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:15.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:15.965+0000 7fe290e8c700 1 --2- 172.21.15.78:0/2851515565 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe28c0ce8c0 0x7fe28c0b3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:15.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:15.965+0000 7fe290e8c700 1 --2- 172.21.15.78:0/2851515565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe28c0b2800 0x7fe28c155670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:15.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:15.965+0000 7fe290e8c700 1 -- 172.21.15.78:0/2851515565 >> 172.21.15.78:0/2851515565 conn(0x7fe28c09a130 msgr2=0x7fe28c0ad990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:15.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:15.965+0000 7fe290e8c700 1 -- 172.21.15.78:0/2851515565 shutdown_connections 2023-10-11T09:48:15.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:15.965+0000 7fe290e8c700 1 -- 172.21.15.78:0/2851515565 wait complete. 2023-10-11T09:48:15.977 INFO:teuthology.orchestra.run.smithi078.stderr: removed all blocklist entries 2023-10-11T09:48:15.995 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-11T09:48:16.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.161+0000 7f782572c700 1 Processor -- start 2023-10-11T09:48:16.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.161+0000 7f782572c700 1 -- start start 2023-10-11T09:48:16.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.165+0000 7f782572c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f782015b8e0 0x7f782006d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:16.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.165+0000 7f782572c700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f782015e010 con 0x7f782006d610 2023-10-11T09:48:16.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.165+0000 7f782572c700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f782015e280 con 0x7f78200ca950 2023-10-11T09:48:16.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.165+0000 7f782572c700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f782015e560 con 0x7f782015b8e0 2023-10-11T09:48:16.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.165+0000 7f78167fc700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f782006d610 0x7f78200ca240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:48102/0 (socket says 172.21.15.78:48102) 2023-10-11T09:48:16.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.165+0000 7f78167fc700 1 -- 172.21.15.78:0/2618868378 learned_addr learned my addr 172.21.15.78:0/2618868378 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:16.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.165+0000 7f781effd700 1 --2- 172.21.15.78:0/2618868378 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f782015b8e0 0x7f782006d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:16.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.165+0000 7f781e7fc700 1 -- 172.21.15.78:0/2618868378 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1151096230 0 0) 0x7f782015e010 con 0x7f782006d610 2023-10-11T09:48:16.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.165+0000 7f781e7fc700 1 -- 172.21.15.78:0/2618868378 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f780c0035d0 con 0x7f782006d610 2023-10-11T09:48:16.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.165+0000 7f781effd700 1 -- 172.21.15.78:0/2618868378 >> v1:172.21.15.133:6789/0 conn(0x7f78200ca950 legacy=0x7f782013fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:16.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.165+0000 7f781effd700 1 -- 172.21.15.78:0/2618868378 >> v1:172.21.15.78:6789/0 conn(0x7f782006d610 legacy=0x7f78200ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:16.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.165+0000 7f781e7fc700 1 -- 172.21.15.78:0/2618868378 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2036841028 0 0) 0x7f782015e280 con 0x7f78200ca950 2023-10-11T09:48:16.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.165+0000 7f781effd700 1 -- 172.21.15.78:0/2618868378 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f782015e840 con 0x7f782015b8e0 2023-10-11T09:48:16.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.169+0000 7f781effd700 1 --2- 172.21.15.78:0/2618868378 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f782015b8e0 0x7f782006d0d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7810009dc0 tx=0x7f781000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ffd89ccdccf14e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:16.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.169+0000 7f781e7fc700 1 -- 172.21.15.78:0/2618868378 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781000e070 con 0x7f782015b8e0 2023-10-11T09:48:16.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.169+0000 7f781e7fc700 1 -- 172.21.15.78:0/2618868378 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f781000bb20 con 0x7f782015b8e0 2023-10-11T09:48:16.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.169+0000 7f781e7fc700 1 -- 172.21.15.78:0/2618868378 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781000bed0 con 0x7f782015b8e0 2023-10-11T09:48:16.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.169+0000 7f782572c700 1 -- 172.21.15.78:0/2618868378 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f782015b8e0 msgr2=0x7f782006d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:16.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.169+0000 7f782572c700 1 --2- 172.21.15.78:0/2618868378 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f782015b8e0 0x7f782006d0d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7810009dc0 tx=0x7f781000b330 comp rx=0 tx=0).stop 2023-10-11T09:48:16.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.169+0000 7f782572c700 1 -- 172.21.15.78:0/2618868378 shutdown_connections 2023-10-11T09:48:16.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.169+0000 7f782572c700 1 --2- 172.21.15.78:0/2618868378 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f782015b8e0 0x7f782006d0d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:16.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.169+0000 7f782572c700 1 -- 172.21.15.78:0/2618868378 >> 172.21.15.78:0/2618868378 conn(0x7f78200cea60 msgr2=0x7f78200cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:16.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.169+0000 7f782572c700 1 -- 172.21.15.78:0/2618868378 shutdown_connections 2023-10-11T09:48:16.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.169+0000 7f782572c700 1 -- 172.21.15.78:0/2618868378 wait complete. 2023-10-11T09:48:16.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.173+0000 7f782572c700 1 Processor -- start 2023-10-11T09:48:16.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.173+0000 7f782572c700 1 -- start start 2023-10-11T09:48:16.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.173+0000 7f782572c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f782006d610 0x7f7820141840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:16.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.173+0000 7f782572c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f78200ca950 0x7f78200b4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:16.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.173+0000 7f782572c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f782015b8e0 0x7f78201f1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:16.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.173+0000 7f782572c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78201e4dd0 con 0x7f782006d610 2023-10-11T09:48:16.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.173+0000 7f782572c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78200b4a80 con 0x7f782015b8e0 2023-10-11T09:48:16.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.173+0000 7f782572c700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78201e4150 con 0x7f78200ca950 2023-10-11T09:48:16.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.173+0000 7f781effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f782006d610 0x7f7820141840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:16.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.173+0000 7f781effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f782006d610 0x7f7820141840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33752/0 (socket says 172.21.15.78:33752) 2023-10-11T09:48:16.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.173+0000 7f781effd700 1 -- 172.21.15.78:0/549405383 learned_addr learned my addr 172.21.15.78:0/549405383 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:16.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.173+0000 7f78167fc700 1 --2- 172.21.15.78:0/549405383 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f78200ca950 0x7f78200b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:16.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.177+0000 7f781f7fe700 1 --2- 172.21.15.78:0/549405383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f782015b8e0 0x7f78201f1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:16.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.177+0000 7f781effd700 1 -- 172.21.15.78:0/549405383 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f78200ca950 msgr2=0x7f78200b4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:16.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.177+0000 7f781effd700 1 --2- 172.21.15.78:0/549405383 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f78200ca950 0x7f78200b4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:16.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.177+0000 7f781effd700 1 -- 172.21.15.78:0/549405383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f782015b8e0 msgr2=0x7f78201f1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:16.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.177+0000 7f781effd700 1 --2- 172.21.15.78:0/549405383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f782015b8e0 0x7f78201f1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:16.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.177+0000 7f781effd700 1 -- 172.21.15.78:0/549405383 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7810009a70 con 0x7f782006d610 2023-10-11T09:48:16.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.177+0000 7f781f7fe700 1 --2- 172.21.15.78:0/549405383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f782015b8e0 0x7f78201f1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:48:16.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.177+0000 7f781effd700 1 --2- 172.21.15.78:0/549405383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f782006d610 0x7f7820141840 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f7810009f30 tx=0x7f7810004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:16.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.177+0000 7f7817fff700 1 -- 172.21.15.78:0/549405383 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781001b8d0 con 0x7f782006d610 2023-10-11T09:48:16.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.177+0000 7f7817fff700 1 -- 172.21.15.78:0/549405383 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7810020070 con 0x7f782006d610 2023-10-11T09:48:16.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.177+0000 7f7817fff700 1 -- 172.21.15.78:0/549405383 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781001bc50 con 0x7f782006d610 2023-10-11T09:48:16.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.177+0000 7f782572c700 1 -- 172.21.15.78:0/549405383 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78201e4430 con 0x7f782006d610 2023-10-11T09:48:16.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.177+0000 7f782572c700 1 -- 172.21.15.78:0/549405383 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78201e49f0 con 0x7f782006d610 2023-10-11T09:48:16.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.177+0000 7f782572c700 1 -- 172.21.15.78:0/549405383 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78201e4f10 con 0x7f782006d610 2023-10-11T09:48:16.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.181+0000 7f7817fff700 1 -- 172.21.15.78:0/549405383 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f781001c400 con 0x7f782006d610 2023-10-11T09:48:16.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.181+0000 7f7817fff700 1 --2- 172.21.15.78:0/549405383 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7800065ed0 0x7f7800068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:16.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.181+0000 7f7817fff700 1 -- 172.21.15.78:0/549405383 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7810022070 con 0x7f782006d610 2023-10-11T09:48:16.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.181+0000 7f78167fc700 1 --2- 172.21.15.78:0/549405383 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7800065ed0 0x7f7800068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:16.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.181+0000 7f78167fc700 1 --2- 172.21.15.78:0/549405383 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7800065ed0 0x7f7800068380 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f78201f4e70 tx=0x7f7808009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:16.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.189+0000 7f7817fff700 1 -- 172.21.15.78:0/549405383 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f78201e4f10 con 0x7f782006d610 2023-10-11T09:48:16.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.421+0000 7f782572c700 1 -- 172.21.15.78:0/549405383 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f78201e4f10 con 0x7f782006d610 2023-10-11T09:48:16.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.421+0000 7f7817fff700 1 -- 172.21.15.78:0/549405383 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f78201e4f10 con 0x7f782006d610 2023-10-11T09:48:16.429 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:16.429 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:48:16.429 INFO:teuthology.orchestra.run.smithi078.stdout: "auth_dump": [ 2023-10-11T09:48:16.429 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:48:16.430 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.a", 2023-10-11T09:48:16.430 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZlL58cAxAALe+WtkV/aLopy1w9i6OVRg==", 2023-10-11T09:48:16.430 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:48:16.430 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:48:16.430 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2023-10-11T09:48:16.431 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2023-10-11T09:48:16.431 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:48:16.431 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:48:16.431 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:48:16.431 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:48:16.431 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.b", 2023-10-11T09:48:16.432 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZliPLmFhAA+6wegaD6yTUsttFzo9OrIg==", 2023-10-11T09:48:16.433 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:48:16.433 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:48:16.433 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2023-10-11T09:48:16.433 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2023-10-11T09:48:16.434 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:48:16.434 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:48:16.434 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:48:16.434 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:48:16.434 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.c", 2023-10-11T09:48:16.435 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZl9jJBDhAAjos7/02OqF0WldFm70/fVA==", 2023-10-11T09:48:16.435 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:48:16.435 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:48:16.435 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2023-10-11T09:48:16.435 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2023-10-11T09:48:16.436 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:48:16.436 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:48:16.436 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:48:16.436 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:48:16.436 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.d", 2023-10-11T09:48:16.436 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZlTb/4IRAAeXtcbWVB5TnSoYch+uMZWw==", 2023-10-11T09:48:16.437 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:48:16.437 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:48:16.437 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2023-10-11T09:48:16.437 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2023-10-11T09:48:16.437 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:48:16.438 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:48:16.438 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:48:16.438 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:48:16.438 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.0", 2023-10-11T09:48:16.439 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBYbSZldbrVFxAA6ogUpCcFO8LhHn4zdkudNQ==", 2023-10-11T09:48:16.439 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:48:16.439 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:48:16.439 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:48:16.439 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:48:16.439 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:48:16.440 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:48:16.440 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:48:16.440 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.1", 2023-10-11T09:48:16.440 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBbbSZl+mkPJxAA4Rrt0j5lzXapYjXhdulhNQ==", 2023-10-11T09:48:16.440 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:48:16.441 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:48:16.441 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:48:16.441 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:48:16.441 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:48:16.441 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:48:16.441 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:48:16.442 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.2", 2023-10-11T09:48:16.442 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBebSZlZ6lSNRAA3ltw4ZTkFNpudCPc2zTYzQ==", 2023-10-11T09:48:16.442 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:48:16.442 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:48:16.442 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:48:16.443 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:48:16.443 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:48:16.443 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:48:16.443 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:48:16.443 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.3", 2023-10-11T09:48:16.443 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBibSZlsE+jBxAA2Ix2JAFTUxljpHAwPpJeUA==", 2023-10-11T09:48:16.444 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:48:16.444 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:48:16.444 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:48:16.444 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:48:16.444 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:48:16.444 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:48:16.445 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:48:16.445 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.4", 2023-10-11T09:48:16.445 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBobSZlII4OGxAA0Z5wGuu4wOs/14tQRnHB1Q==", 2023-10-11T09:48:16.445 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:48:16.445 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:48:16.446 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:48:16.446 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:48:16.446 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:48:16.446 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:48:16.446 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:48:16.447 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.5", 2023-10-11T09:48:16.447 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBrbSZlG6aSKhAAZq179SfpkelcJJLaUcLDCg==", 2023-10-11T09:48:16.447 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:48:16.447 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:48:16.448 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:48:16.448 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:48:16.448 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:48:16.449 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:48:16.450 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:48:16.450 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.6", 2023-10-11T09:48:16.450 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBubSZl3W89NxAAgUrd0/yMOWlrhAOU1BpM9A==", 2023-10-11T09:48:16.450 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:48:16.451 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:48:16.451 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:48:16.451 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:48:16.451 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:48:16.451 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:48:16.452 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:48:16.452 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.7", 2023-10-11T09:48:16.452 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBybSZloIjyCRAAjmvyK9zMSEsV8bVebwzPxg==", 2023-10-11T09:48:16.452 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:48:16.452 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:48:16.453 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:48:16.453 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:48:16.453 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:48:16.453 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:48:16.453 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:48:16.454 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.0", 2023-10-11T09:48:16.454 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZlmGmCKBAAOI8EvyRwg8HZt+q/vUy5Ow==", 2023-10-11T09:48:16.454 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:48:16.454 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:48:16.454 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow r", 2023-10-11T09:48:16.455 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-11T09:48:16.455 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:48:16.455 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:48:16.455 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:48:16.455 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.1", 2023-10-11T09:48:16.456 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZlYvfHNBAAReum0efx4MmqpLMxFnTROQ==", 2023-10-11T09:48:16.456 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:48:16.456 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:48:16.456 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2023-10-11T09:48:16.456 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2023-10-11T09:48:16.457 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2023-10-11T09:48:16.457 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:48:16.457 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:48:16.457 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:48:16.457 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.2", 2023-10-11T09:48:16.458 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZlbd9lLhAA9LAjreTcxwAjhiNlKndl+Q==", 2023-10-11T09:48:16.458 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:48:16.458 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:48:16.458 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2023-10-11T09:48:16.458 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2023-10-11T09:48:16.459 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2023-10-11T09:48:16.459 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:48:16.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:48:16.459 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:48:16.460 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.3", 2023-10-11T09:48:16.460 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZlHUjeOhAASPKTRvsillob4gjdW0akhQ==", 2023-10-11T09:48:16.460 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:48:16.460 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:48:16.460 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2023-10-11T09:48:16.461 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2023-10-11T09:48:16.461 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2023-10-11T09:48:16.461 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:48:16.461 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:48:16.461 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:48:16.462 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.admin", 2023-10-11T09:48:16.462 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBTbSZlhqcOFRAA/lkGYL03++trOuIqNLS0kQ==", 2023-10-11T09:48:16.462 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:48:16.462 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2023-10-11T09:48:16.462 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2023-10-11T09:48:16.463 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2023-10-11T09:48:16.463 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:48:16.463 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:48:16.463 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:48:16.463 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:48:16.464 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mgr.x", 2023-10-11T09:48:16.464 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBTbSZlS/TjNxAAMkq9xRMwgsbLlT4FdobATw==", 2023-10-11T09:48:16.464 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:48:16.464 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2023-10-11T09:48:16.464 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile mgr", 2023-10-11T09:48:16.465 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:48:16.465 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:48:16.465 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:48:16.465 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:48:16.465 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mgr.y", 2023-10-11T09:48:16.465 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBTbSZlnv4yMRAAQh8w18k76KRjdCvSDJ4QbA==", 2023-10-11T09:48:16.466 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:48:16.466 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2023-10-11T09:48:16.466 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile mgr", 2023-10-11T09:48:16.466 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:48:16.466 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:48:16.467 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:48:16.467 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2023-10-11T09:48:16.467 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:48:16.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.421+0000 7f782572c700 1 -- 172.21.15.78:0/549405383 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7800065ed0 msgr2=0x7f7800068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:16.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.421+0000 7f782572c700 1 --2- 172.21.15.78:0/549405383 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7800065ed0 0x7f7800068380 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f78201f4e70 tx=0x7f7808009040 comp rx=0 tx=0).stop 2023-10-11T09:48:16.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.421+0000 7f782572c700 1 -- 172.21.15.78:0/549405383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f782006d610 msgr2=0x7f7820141840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:16.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.421+0000 7f782572c700 1 --2- 172.21.15.78:0/549405383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f782006d610 0x7f7820141840 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f7810009f30 tx=0x7f7810004af0 comp rx=0 tx=0).stop 2023-10-11T09:48:16.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.425+0000 7f782572c700 1 -- 172.21.15.78:0/549405383 shutdown_connections 2023-10-11T09:48:16.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.425+0000 7f782572c700 1 --2- 172.21.15.78:0/549405383 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7800065ed0 0x7f7800068380 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:16.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.425+0000 7f782572c700 1 --2- 172.21.15.78:0/549405383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f782015b8e0 0x7f78201f1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:16.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.425+0000 7f782572c700 1 --2- 172.21.15.78:0/549405383 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f78200ca950 0x7f78200b4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:16.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.425+0000 7f782572c700 1 --2- 172.21.15.78:0/549405383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f782006d610 0x7f7820141840 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:16.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.425+0000 7f782572c700 1 -- 172.21.15.78:0/549405383 >> 172.21.15.78:0/549405383 conn(0x7f78200cea60 msgr2=0x7f7820142770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:16.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.425+0000 7f782572c700 1 -- 172.21.15.78:0/549405383 shutdown_connections 2023-10-11T09:48:16.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.425+0000 7f782572c700 1 -- 172.21.15.78:0/549405383 wait complete. 2023-10-11T09:48:16.472 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:48:16.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.633+0000 7f48ca2eb700 1 Processor -- start 2023-10-11T09:48:16.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.633+0000 7f48ca2eb700 1 -- start start 2023-10-11T09:48:16.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.633+0000 7f48ca2eb700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f48c4156ce0 0x7f48c4160b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:16.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.633+0000 7f48ca2eb700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48c415dc10 con 0x7f48c40d1660 2023-10-11T09:48:16.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.633+0000 7f48ca2eb700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48c4160380 con 0x7f48c40d1160 2023-10-11T09:48:16.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.633+0000 7f48ca2eb700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48c415d930 con 0x7f48c4156ce0 2023-10-11T09:48:16.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.633+0000 7f48c92e9700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f48c4156ce0 0x7f48c4160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:16.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.633+0000 7f48c8ae8700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f48c40d1660 0x7f48c40d0a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:48106/0 (socket says 172.21.15.78:48106) 2023-10-11T09:48:16.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.633+0000 7f48c8ae8700 1 -- 172.21.15.78:0/2863060205 learned_addr learned my addr 172.21.15.78:0/2863060205 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:16.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.633+0000 7f48bbfff700 1 -- 172.21.15.78:0/2863060205 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 473753377 0 0) 0x7f48c415dc10 con 0x7f48c40d1660 2023-10-11T09:48:16.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.633+0000 7f48bbfff700 1 -- 172.21.15.78:0/2863060205 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48b00035d0 con 0x7f48c40d1660 2023-10-11T09:48:16.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.633+0000 7f48c92e9700 1 -- 172.21.15.78:0/2863060205 >> v1:172.21.15.133:6789/0 conn(0x7f48c40d1160 legacy=0x7f48c40cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:16.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.633+0000 7f48c92e9700 1 -- 172.21.15.78:0/2863060205 >> v1:172.21.15.78:6789/0 conn(0x7f48c40d1660 legacy=0x7f48c40d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:16.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.633+0000 7f48c92e9700 1 -- 172.21.15.78:0/2863060205 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c4160620 con 0x7f48c4156ce0 2023-10-11T09:48:16.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.637+0000 7f48c92e9700 1 --2- 172.21.15.78:0/2863060205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48c4156ce0 0x7f48c4160b40 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f48c0009850 tx=0x7f48c0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44c6d919a2cdcdf server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:16.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.637+0000 7f48bbfff700 1 -- 172.21.15.78:0/2863060205 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c0011830 con 0x7f48c4156ce0 2023-10-11T09:48:16.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.637+0000 7f48bbfff700 1 -- 172.21.15.78:0/2863060205 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c0011990 con 0x7f48c4156ce0 2023-10-11T09:48:16.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.637+0000 7f48bbfff700 1 -- 172.21.15.78:0/2863060205 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c0011d40 con 0x7f48c4156ce0 2023-10-11T09:48:16.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.637+0000 7f48ca2eb700 1 -- 172.21.15.78:0/2863060205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48c4156ce0 msgr2=0x7f48c4160b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:16.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.637+0000 7f48ca2eb700 1 --2- 172.21.15.78:0/2863060205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48c4156ce0 0x7f48c4160b40 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f48c0009850 tx=0x7f48c0009b60 comp rx=0 tx=0).stop 2023-10-11T09:48:16.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.637+0000 7f48ca2eb700 1 -- 172.21.15.78:0/2863060205 shutdown_connections 2023-10-11T09:48:16.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.637+0000 7f48ca2eb700 1 --2- 172.21.15.78:0/2863060205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48c4156ce0 0x7f48c4160b40 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:16.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.637+0000 7f48ca2eb700 1 -- 172.21.15.78:0/2863060205 >> 172.21.15.78:0/2863060205 conn(0x7f48c406afb0 msgr2=0x7f48c406b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:16.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.637+0000 7f48ca2eb700 1 -- 172.21.15.78:0/2863060205 shutdown_connections 2023-10-11T09:48:16.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.637+0000 7f48ca2eb700 1 -- 172.21.15.78:0/2863060205 wait complete. 2023-10-11T09:48:16.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.640+0000 7f48ca2eb700 1 Processor -- start 2023-10-11T09:48:16.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.640+0000 7f48ca2eb700 1 -- start start 2023-10-11T09:48:16.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.640+0000 7f48ca2eb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48c40d1160 0x7f48c414eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:16.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.640+0000 7f48ca2eb700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48c40d1660 0x7f48c40cf850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:16.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.640+0000 7f48ca2eb700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48c4156ce0 0x7f48c40cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:16.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.640+0000 7f48ca2eb700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c40caf80 con 0x7f48c40d1160 2023-10-11T09:48:16.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.640+0000 7f48ca2eb700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c40caca0 con 0x7f48c4156ce0 2023-10-11T09:48:16.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.640+0000 7f48ca2eb700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48c40c7af0 con 0x7f48c40d1660 2023-10-11T09:48:16.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.640+0000 7f48c92e9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48c40d1160 0x7f48c414eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:16.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.640+0000 7f48c92e9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48c40d1160 0x7f48c414eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33770/0 (socket says 172.21.15.78:33770) 2023-10-11T09:48:16.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.640+0000 7f48c92e9700 1 -- 172.21.15.78:0/2424876153 learned_addr learned my addr 172.21.15.78:0/2424876153 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:16.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.640+0000 7f48c8ae8700 1 --2- 172.21.15.78:0/2424876153 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48c40d1660 0x7f48c40cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:16.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.640+0000 7f48c92e9700 1 -- 172.21.15.78:0/2424876153 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48c40d1660 msgr2=0x7f48c40cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:16.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.640+0000 7f48c92e9700 1 --2- 172.21.15.78:0/2424876153 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48c40d1660 0x7f48c40cf850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:16.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.640+0000 7f48c92e9700 1 -- 172.21.15.78:0/2424876153 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48c4156ce0 msgr2=0x7f48c40cfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:16.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.640+0000 7f48c92e9700 1 --2- 172.21.15.78:0/2424876153 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48c4156ce0 0x7f48c40cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:16.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.640+0000 7f48c92e9700 1 -- 172.21.15.78:0/2424876153 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c00094b0 con 0x7f48c40d1160 2023-10-11T09:48:16.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.640+0000 7f48c92e9700 1 --2- 172.21.15.78:0/2424876153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48c40d1160 0x7f48c414eb10 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f48c0000c00 tx=0x7f48c0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:16.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.640+0000 7f48b9ffb700 1 -- 172.21.15.78:0/2424876153 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c001aac0 con 0x7f48c40d1160 2023-10-11T09:48:16.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.640+0000 7f48b9ffb700 1 -- 172.21.15.78:0/2424876153 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c001f070 con 0x7f48c40d1160 2023-10-11T09:48:16.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.640+0000 7f48ca2eb700 1 -- 172.21.15.78:0/2424876153 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48c40c7dd0 con 0x7f48c40d1160 2023-10-11T09:48:16.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.640+0000 7f48b9ffb700 1 -- 172.21.15.78:0/2424876153 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c001ae60 con 0x7f48c40d1160 2023-10-11T09:48:16.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.640+0000 7f48ca2eb700 1 -- 172.21.15.78:0/2424876153 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c40ca930 con 0x7f48c40d1160 2023-10-11T09:48:16.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.644+0000 7f48ca2eb700 1 -- 172.21.15.78:0/2424876153 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48c40c82f0 con 0x7f48c40d1160 2023-10-11T09:48:16.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.644+0000 7f48b9ffb700 1 -- 172.21.15.78:0/2424876153 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f48c001b430 con 0x7f48c40d1160 2023-10-11T09:48:16.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.644+0000 7f48b9ffb700 1 --2- 172.21.15.78:0/2424876153 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f48a8065ed0 0x7f48a8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:16.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.644+0000 7f48b9ffb700 1 -- 172.21.15.78:0/2424876153 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f48c00845d0 con 0x7f48c40d1160 2023-10-11T09:48:16.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.644+0000 7f48c8ae8700 1 --2- 172.21.15.78:0/2424876153 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f48a8065ed0 0x7f48a8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:16.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.648+0000 7f48c8ae8700 1 --2- 172.21.15.78:0/2424876153 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f48a8065ed0 0x7f48a8068380 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f48c4155ad0 tx=0x7f48b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:16.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.652+0000 7f48b9ffb700 1 -- 172.21.15.78:0/2424876153 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f48c40c82f0 con 0x7f48c40d1160 2023-10-11T09:48:16.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.824+0000 7f48ca2eb700 1 -- 172.21.15.78:0/2424876153 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48c40c82f0 con 0x7f48c40d1160 2023-10-11T09:48:16.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.824+0000 7f48b9ffb700 1 -- 172.21.15.78:0/2424876153 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+14172 (secure 0 0 0) 0x7f48c40c82f0 con 0x7f48c40d1160 2023-10-11T09:48:16.835 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:16.836 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":79,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:48:15.955456+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:48:16.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.828+0000 7f48ca2eb700 1 -- 172.21.15.78:0/2424876153 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f48a8065ed0 msgr2=0x7f48a8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:16.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.828+0000 7f48ca2eb700 1 --2- 172.21.15.78:0/2424876153 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f48a8065ed0 0x7f48a8068380 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f48c4155ad0 tx=0x7f48b4009040 comp rx=0 tx=0).stop 2023-10-11T09:48:16.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.828+0000 7f48ca2eb700 1 -- 172.21.15.78:0/2424876153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48c40d1160 msgr2=0x7f48c414eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:16.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.828+0000 7f48ca2eb700 1 --2- 172.21.15.78:0/2424876153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48c40d1160 0x7f48c414eb10 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f48c0000c00 tx=0x7f48c0004770 comp rx=0 tx=0).stop 2023-10-11T09:48:16.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.828+0000 7f48ca2eb700 1 -- 172.21.15.78:0/2424876153 shutdown_connections 2023-10-11T09:48:16.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.828+0000 7f48ca2eb700 1 --2- 172.21.15.78:0/2424876153 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f48a8065ed0 0x7f48a8068380 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:16.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.828+0000 7f48ca2eb700 1 --2- 172.21.15.78:0/2424876153 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48c4156ce0 0x7f48c40cfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:16.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.828+0000 7f48ca2eb700 1 --2- 172.21.15.78:0/2424876153 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48c40d1660 0x7f48c40cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:16.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.828+0000 7f48ca2eb700 1 --2- 172.21.15.78:0/2424876153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48c40d1160 0x7f48c414eb10 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:16.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.828+0000 7f48ca2eb700 1 -- 172.21.15.78:0/2424876153 >> 172.21.15.78:0/2424876153 conn(0x7f48c406afb0 msgr2=0x7f48c40d0780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:16.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.828+0000 7f48ca2eb700 1 -- 172.21.15.78:0/2424876153 shutdown_connections 2023-10-11T09:48:16.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:16.828+0000 7f48ca2eb700 1 -- 172.21.15.78:0/2424876153 wait complete. 2023-10-11T09:48:16.863 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:48:16.863 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:48:17.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.024+0000 7f07c0870700 1 Processor -- start 2023-10-11T09:48:17.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.024+0000 7f07c0870700 1 -- start start 2023-10-11T09:48:17.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.028+0000 7f07c0870700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f07bc15a6b0 0x7f07bc0ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:17.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.028+0000 7f07c0870700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07bc0b2d30 con 0x7f07bc0b3a60 2023-10-11T09:48:17.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.028+0000 7f07c0870700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07bc0cf360 con 0x7f07bc06d730 2023-10-11T09:48:17.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.028+0000 7f07c0870700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07bc0b2f80 con 0x7f07bc15a6b0 2023-10-11T09:48:17.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.028+0000 7f07b259c700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f07bc0b3a60 0x7f07bc06d020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:48116/0 (socket says 172.21.15.78:48116) 2023-10-11T09:48:17.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.028+0000 7f07b259c700 1 -- 172.21.15.78:0/337120216 learned_addr learned my addr 172.21.15.78:0/337120216 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:17.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.028+0000 7f07bad9d700 1 --2- 172.21.15.78:0/337120216 >> v2:172.21.15.78:3300/0 conn(0x7f07bc15a6b0 0x7f07bc0ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:17.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.028+0000 7f07ba59c700 1 -- 172.21.15.78:0/337120216 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3267768262 0 0) 0x7f07bc0b2d30 con 0x7f07bc0b3a60 2023-10-11T09:48:17.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.028+0000 7f07ba59c700 1 -- 172.21.15.78:0/337120216 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07a80035d0 con 0x7f07bc0b3a60 2023-10-11T09:48:17.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.028+0000 7f07bad9d700 1 -- 172.21.15.78:0/337120216 >> v1:172.21.15.133:6789/0 conn(0x7f07bc06d730 legacy=0x7f07bc06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:17.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.028+0000 7f07bad9d700 1 -- 172.21.15.78:0/337120216 >> v1:172.21.15.78:6789/0 conn(0x7f07bc0b3a60 legacy=0x7f07bc06d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:17.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.028+0000 7f07bad9d700 1 -- 172.21.15.78:0/337120216 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07bc0b3260 con 0x7f07bc15a6b0 2023-10-11T09:48:17.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.028+0000 7f07bad9d700 1 --2- 172.21.15.78:0/337120216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07bc15a6b0 0x7f07bc0ce8a0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f07ac009850 tx=0x7f07ac009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b44eba1c68070dd server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:17.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.028+0000 7f07ba59c700 1 -- 172.21.15.78:0/337120216 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ac011830 con 0x7f07bc15a6b0 2023-10-11T09:48:17.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.028+0000 7f07ba59c700 1 -- 172.21.15.78:0/337120216 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07ac011990 con 0x7f07bc15a6b0 2023-10-11T09:48:17.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.028+0000 7f07ba59c700 1 -- 172.21.15.78:0/337120216 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ac011d40 con 0x7f07bc15a6b0 2023-10-11T09:48:17.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.028+0000 7f07c0870700 1 -- 172.21.15.78:0/337120216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07bc15a6b0 msgr2=0x7f07bc0ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:17.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.028+0000 7f07c0870700 1 --2- 172.21.15.78:0/337120216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07bc15a6b0 0x7f07bc0ce8a0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f07ac009850 tx=0x7f07ac009b60 comp rx=0 tx=0).stop 2023-10-11T09:48:17.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.028+0000 7f07c0870700 1 -- 172.21.15.78:0/337120216 shutdown_connections 2023-10-11T09:48:17.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.028+0000 7f07c0870700 1 --2- 172.21.15.78:0/337120216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07bc15a6b0 0x7f07bc0ce8a0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:17.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.028+0000 7f07c0870700 1 -- 172.21.15.78:0/337120216 >> 172.21.15.78:0/337120216 conn(0x7f07bc06a060 msgr2=0x7f07bc06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:17.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.032+0000 7f07c0870700 1 -- 172.21.15.78:0/337120216 shutdown_connections 2023-10-11T09:48:17.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.032+0000 7f07c0870700 1 -- 172.21.15.78:0/337120216 wait complete. 2023-10-11T09:48:17.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.032+0000 7f07c0870700 1 Processor -- start 2023-10-11T09:48:17.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.036+0000 7f07c0870700 1 -- start start 2023-10-11T09:48:17.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.036+0000 7f07c0870700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07bc06d730 0x7f07bc13f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:17.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.036+0000 7f07c0870700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07bc0b3a60 0x7f07bc15b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:17.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.036+0000 7f07c0870700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07bc15a6b0 0x7f07bc157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:17.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.036+0000 7f07c0870700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07bc14c1b0 con 0x7f07bc15a6b0 2023-10-11T09:48:17.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.036+0000 7f07c0870700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07bc14b530 con 0x7f07bc06d730 2023-10-11T09:48:17.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.036+0000 7f07c0870700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07bc14b810 con 0x7f07bc0b3a60 2023-10-11T09:48:17.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.036+0000 7f07bb59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07bc15a6b0 0x7f07bc157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:17.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.036+0000 7f07bb59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07bc15a6b0 0x7f07bc157590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33794/0 (socket says 172.21.15.78:33794) 2023-10-11T09:48:17.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.036+0000 7f07bb59e700 1 -- 172.21.15.78:0/1874937853 learned_addr learned my addr 172.21.15.78:0/1874937853 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:17.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.036+0000 7f07b259c700 1 --2- 172.21.15.78:0/1874937853 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07bc0b3a60 0x7f07bc15b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:17.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.036+0000 7f07bad9d700 1 --2- 172.21.15.78:0/1874937853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07bc06d730 0x7f07bc13f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:17.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.036+0000 7f07bb59e700 1 -- 172.21.15.78:0/1874937853 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07bc0b3a60 msgr2=0x7f07bc15b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:17.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.036+0000 7f07bb59e700 1 --2- 172.21.15.78:0/1874937853 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07bc0b3a60 0x7f07bc15b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:17.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.036+0000 7f07bb59e700 1 -- 172.21.15.78:0/1874937853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07bc06d730 msgr2=0x7f07bc13f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:17.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.036+0000 7f07bb59e700 1 --2- 172.21.15.78:0/1874937853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07bc06d730 0x7f07bc13f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:17.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.036+0000 7f07bb59e700 1 -- 172.21.15.78:0/1874937853 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07ac0094b0 con 0x7f07bc15a6b0 2023-10-11T09:48:17.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.036+0000 7f07bb59e700 1 --2- 172.21.15.78:0/1874937853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07bc15a6b0 0x7f07bc157590 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f07b400c480 tx=0x7f07b400c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:17.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.040+0000 7f07b3fff700 1 -- 172.21.15.78:0/1874937853 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b40052d0 con 0x7f07bc15a6b0 2023-10-11T09:48:17.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.040+0000 7f07b3fff700 1 -- 172.21.15.78:0/1874937853 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07b400ce90 con 0x7f07bc15a6b0 2023-10-11T09:48:17.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.040+0000 7f07b3fff700 1 -- 172.21.15.78:0/1874937853 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b40036a0 con 0x7f07bc15a6b0 2023-10-11T09:48:17.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.040+0000 7f07c0870700 1 -- 172.21.15.78:0/1874937853 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07bc14baf0 con 0x7f07bc15a6b0 2023-10-11T09:48:17.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.040+0000 7f07c0870700 1 -- 172.21.15.78:0/1874937853 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07bc149a40 con 0x7f07bc15a6b0 2023-10-11T09:48:17.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.040+0000 7f07c0870700 1 -- 172.21.15.78:0/1874937853 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07bc14c2f0 con 0x7f07bc15a6b0 2023-10-11T09:48:17.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.040+0000 7f07b3fff700 1 -- 172.21.15.78:0/1874937853 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f07b4003800 con 0x7f07bc15a6b0 2023-10-11T09:48:17.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.044+0000 7f07b3fff700 1 --2- 172.21.15.78:0/1874937853 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f079c065ed0 0x7f079c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:17.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.044+0000 7f07b3fff700 1 -- 172.21.15.78:0/1874937853 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f07b4082770 con 0x7f07bc15a6b0 2023-10-11T09:48:17.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.044+0000 7f07bad9d700 1 --2- 172.21.15.78:0/1874937853 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f079c065ed0 0x7f079c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:17.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.044+0000 7f07bad9d700 1 --2- 172.21.15.78:0/1874937853 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f079c065ed0 0x7f079c068380 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f07ac0090b0 tx=0x7f07ac01c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:17.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.048+0000 7f07b3fff700 1 -- 172.21.15.78:0/1874937853 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f07bc14c2f0 con 0x7f07bc15a6b0 2023-10-11T09:48:17.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.224+0000 7f07c0870700 1 -- 172.21.15.78:0/1874937853 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f07bc14c2f0 con 0x7f07bc15a6b0 2023-10-11T09:48:17.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.224+0000 7f07b3fff700 1 -- 172.21.15.78:0/1874937853 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f07bc14c2f0 con 0x7f07bc15a6b0 2023-10-11T09:48:17.235 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:48:17.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.228+0000 7f07c0870700 1 -- 172.21.15.78:0/1874937853 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f079c065ed0 msgr2=0x7f079c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:17.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.228+0000 7f07c0870700 1 --2- 172.21.15.78:0/1874937853 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f079c065ed0 0x7f079c068380 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f07ac0090b0 tx=0x7f07ac01c040 comp rx=0 tx=0).stop 2023-10-11T09:48:17.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.228+0000 7f07c0870700 1 -- 172.21.15.78:0/1874937853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07bc15a6b0 msgr2=0x7f07bc157590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:17.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.228+0000 7f07c0870700 1 --2- 172.21.15.78:0/1874937853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07bc15a6b0 0x7f07bc157590 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f07b400c480 tx=0x7f07b400c790 comp rx=0 tx=0).stop 2023-10-11T09:48:17.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.228+0000 7f07c0870700 1 -- 172.21.15.78:0/1874937853 shutdown_connections 2023-10-11T09:48:17.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.232+0000 7f07c0870700 1 --2- 172.21.15.78:0/1874937853 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f079c065ed0 0x7f079c068380 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:17.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.232+0000 7f07c0870700 1 --2- 172.21.15.78:0/1874937853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07bc15a6b0 0x7f07bc157590 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:17.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.232+0000 7f07c0870700 1 --2- 172.21.15.78:0/1874937853 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07bc0b3a60 0x7f07bc15b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:17.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.232+0000 7f07c0870700 1 --2- 172.21.15.78:0/1874937853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07bc06d730 0x7f07bc13f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:17.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.232+0000 7f07c0870700 1 -- 172.21.15.78:0/1874937853 >> 172.21.15.78:0/1874937853 conn(0x7f07bc06a060 msgr2=0x7f07bc14f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:17.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.232+0000 7f07c0870700 1 -- 172.21.15.78:0/1874937853 shutdown_connections 2023-10-11T09:48:17.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.232+0000 7f07c0870700 1 -- 172.21.15.78:0/1874937853 wait complete. 2023-10-11T09:48:17.262 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:48:17.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.424+0000 7f949359e700 1 Processor -- start 2023-10-11T09:48:17.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.424+0000 7f949359e700 1 -- start start 2023-10-11T09:48:17.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.428+0000 7f949359e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f949415c900 0x7f94940b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:17.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.428+0000 7f949359e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f94940ad290 0x7f9494071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:17.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.428+0000 7f949359e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f949406fde0 con 0x7f94940ce7f0 2023-10-11T09:48:17.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.428+0000 7f949359e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94940705f0 con 0x7f94940ad290 2023-10-11T09:48:17.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.428+0000 7f949359e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9494071fc0 con 0x7f949415c900 2023-10-11T09:48:17.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.428+0000 7f9489d9b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f94940ad290 0x7f9494071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:17.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.428+0000 7f9489d9b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f94940ad290 0x7f9494071a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33798/0 (socket says 172.21.15.78:33798) 2023-10-11T09:48:17.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.428+0000 7f9489d9b700 1 -- 172.21.15.78:0/2069541946 learned_addr learned my addr 172.21.15.78:0/2069541946 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:17.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.428+0000 7f949259c700 1 --2- 172.21.15.78:0/2069541946 >> v2:172.21.15.133:3300/0 conn(0x7f949415c900 0x7f94940b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:17.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.428+0000 7f9489d9b700 1 -- 172.21.15.78:0/2069541946 >> v2:172.21.15.133:3300/0 conn(0x7f949415c900 msgr2=0x7f94940b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:17.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.428+0000 7f9489d9b700 1 --2- 172.21.15.78:0/2069541946 >> v2:172.21.15.133:3300/0 conn(0x7f949415c900 0x7f94940b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:17.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.428+0000 7f9489d9b700 1 -- 172.21.15.78:0/2069541946 >> v1:172.21.15.133:6789/0 conn(0x7f94940ce7f0 legacy=0x7f949406bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:17.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.428+0000 7f9489d9b700 1 -- 172.21.15.78:0/2069541946 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94940ae000 con 0x7f94940ad290 2023-10-11T09:48:17.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.428+0000 7f9489d9b700 1 --2- 172.21.15.78:0/2069541946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94940ad290 0x7f9494071a80 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f94940724b0 tx=0x7f947c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff4c3f51d0751bb6 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:17.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.428+0000 7f9491d9b700 1 -- 172.21.15.78:0/2069541946 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c011830 con 0x7f94940ad290 2023-10-11T09:48:17.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.432+0000 7f9491d9b700 1 -- 172.21.15.78:0/2069541946 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f947c011990 con 0x7f94940ad290 2023-10-11T09:48:17.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.432+0000 7f9491d9b700 1 -- 172.21.15.78:0/2069541946 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c011d20 con 0x7f94940ad290 2023-10-11T09:48:17.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.432+0000 7f949359e700 1 -- 172.21.15.78:0/2069541946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94940ad290 msgr2=0x7f9494071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:17.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.432+0000 7f949359e700 1 --2- 172.21.15.78:0/2069541946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94940ad290 0x7f9494071a80 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f94940724b0 tx=0x7f947c009bf0 comp rx=0 tx=0).stop 2023-10-11T09:48:17.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.432+0000 7f949359e700 1 -- 172.21.15.78:0/2069541946 shutdown_connections 2023-10-11T09:48:17.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.432+0000 7f949359e700 1 --2- 172.21.15.78:0/2069541946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94940ad290 0x7f9494071a80 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:17.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.432+0000 7f949359e700 1 --2- 172.21.15.78:0/2069541946 >> v2:172.21.15.133:3300/0 conn(0x7f949415c900 0x7f94940b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:17.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.432+0000 7f949359e700 1 -- 172.21.15.78:0/2069541946 >> 172.21.15.78:0/2069541946 conn(0x7f949406a060 msgr2=0x7f949406a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:17.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.432+0000 7f949359e700 1 -- 172.21.15.78:0/2069541946 shutdown_connections 2023-10-11T09:48:17.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.432+0000 7f949359e700 1 -- 172.21.15.78:0/2069541946 wait complete. 2023-10-11T09:48:17.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.432+0000 7f949359e700 1 Processor -- start 2023-10-11T09:48:17.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.432+0000 7f949359e700 1 -- start start 2023-10-11T09:48:17.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.436+0000 7f949359e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94940ad290 0x7f94941470e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:17.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.436+0000 7f949359e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f94940ce7f0 0x7f94940ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:17.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.436+0000 7f949359e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f949415c900 0x7f9494159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:17.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.436+0000 7f949359e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9494147620 con 0x7f94940ad290 2023-10-11T09:48:17.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.436+0000 7f949359e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94940aebc0 con 0x7f949415c900 2023-10-11T09:48:17.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.436+0000 7f949359e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9494148020 con 0x7f94940ce7f0 2023-10-11T09:48:17.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.436+0000 7f949259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94940ad290 0x7f94941470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:17.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.436+0000 7f949259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94940ad290 0x7f94941470e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33806/0 (socket says 172.21.15.78:33806) 2023-10-11T09:48:17.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.436+0000 7f949259c700 1 -- 172.21.15.78:0/2941696385 learned_addr learned my addr 172.21.15.78:0/2941696385 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:17.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.436+0000 7f9489d9b700 1 --2- 172.21.15.78:0/2941696385 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f94940ce7f0 0x7f94940ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:17.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.436+0000 7f9492d9d700 1 --2- 172.21.15.78:0/2941696385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f949415c900 0x7f9494159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:17.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.436+0000 7f949259c700 1 -- 172.21.15.78:0/2941696385 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f94940ce7f0 msgr2=0x7f94940ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:17.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.436+0000 7f949259c700 1 --2- 172.21.15.78:0/2941696385 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f94940ce7f0 0x7f94940ae680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:17.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.436+0000 7f949259c700 1 -- 172.21.15.78:0/2941696385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f949415c900 msgr2=0x7f9494159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:17.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.436+0000 7f949259c700 1 --2- 172.21.15.78:0/2941696385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f949415c900 0x7f9494159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:17.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.436+0000 7f949259c700 1 -- 172.21.15.78:0/2941696385 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f947c009570 con 0x7f94940ad290 2023-10-11T09:48:17.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.436+0000 7f9489d9b700 1 --2- 172.21.15.78:0/2941696385 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f94940ce7f0 0x7f94940ae680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:48:17.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.436+0000 7f949259c700 1 --2- 172.21.15.78:0/2941696385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94940ad290 0x7f94941470e0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f948400ea00 tx=0x7f948400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:17.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.436+0000 7f948b7fe700 1 -- 172.21.15.78:0/2941696385 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9484009720 con 0x7f94940ad290 2023-10-11T09:48:17.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.436+0000 7f948b7fe700 1 -- 172.21.15.78:0/2941696385 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948400c850 con 0x7f94940ad290 2023-10-11T09:48:17.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.436+0000 7f949359e700 1 -- 172.21.15.78:0/2941696385 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94940b1550 con 0x7f94940ad290 2023-10-11T09:48:17.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.436+0000 7f948b7fe700 1 -- 172.21.15.78:0/2941696385 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948400cbc0 con 0x7f94940ad290 2023-10-11T09:48:17.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.436+0000 7f949359e700 1 -- 172.21.15.78:0/2941696385 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94940af0b0 con 0x7f94940ad290 2023-10-11T09:48:17.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.440+0000 7f949359e700 1 -- 172.21.15.78:0/2941696385 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9494065d40 con 0x7f94940ad290 2023-10-11T09:48:17.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.440+0000 7f948b7fe700 1 -- 172.21.15.78:0/2941696385 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f948400cd20 con 0x7f94940ad290 2023-10-11T09:48:17.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.444+0000 7f948b7fe700 1 --2- 172.21.15.78:0/2941696385 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9474065ed0 0x7f9474068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:17.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.444+0000 7f948b7fe700 1 -- 172.21.15.78:0/2941696385 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9484083440 con 0x7f94940ad290 2023-10-11T09:48:17.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.444+0000 7f9489d9b700 1 --2- 172.21.15.78:0/2941696385 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9474065ed0 0x7f9474068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:17.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.444+0000 7f9489d9b700 1 --2- 172.21.15.78:0/2941696385 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9474065ed0 0x7f9474068380 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f947c000c00 tx=0x7f947c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:17.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.448+0000 7f948b7fe700 1 -- 172.21.15.78:0/2941696385 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f9484050dd0 con 0x7f94940ad290 2023-10-11T09:48:17.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.624+0000 7f949359e700 1 -- 172.21.15.78:0/2941696385 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f94940b1550 con 0x7f94940ad290 2023-10-11T09:48:17.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.624+0000 7f948b7fe700 1 -- 172.21.15.78:0/2941696385 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v79) v1 ==== 92+0+10 (secure 0 0 0) 0x7f94940b1550 con 0x7f94940ad290 2023-10-11T09:48:17.634 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:48:17.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.628+0000 7f949359e700 1 -- 172.21.15.78:0/2941696385 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9474065ed0 msgr2=0x7f9474068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:17.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.628+0000 7f949359e700 1 --2- 172.21.15.78:0/2941696385 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9474065ed0 0x7f9474068380 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f947c000c00 tx=0x7f947c008030 comp rx=0 tx=0).stop 2023-10-11T09:48:17.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.628+0000 7f949359e700 1 -- 172.21.15.78:0/2941696385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94940ad290 msgr2=0x7f94941470e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:17.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.628+0000 7f949359e700 1 --2- 172.21.15.78:0/2941696385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94940ad290 0x7f94941470e0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f948400ea00 tx=0x7f948400edc0 comp rx=0 tx=0).stop 2023-10-11T09:48:17.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.628+0000 7f949359e700 1 -- 172.21.15.78:0/2941696385 shutdown_connections 2023-10-11T09:48:17.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.628+0000 7f949359e700 1 --2- 172.21.15.78:0/2941696385 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9474065ed0 0x7f9474068380 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:17.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.628+0000 7f949359e700 1 --2- 172.21.15.78:0/2941696385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f949415c900 0x7f9494159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:17.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.628+0000 7f949359e700 1 --2- 172.21.15.78:0/2941696385 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f94940ce7f0 0x7f94940ae680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:17.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.628+0000 7f949359e700 1 --2- 172.21.15.78:0/2941696385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94940ad290 0x7f94941470e0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:17.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.628+0000 7f949359e700 1 -- 172.21.15.78:0/2941696385 >> 172.21.15.78:0/2941696385 conn(0x7f949406a060 msgr2=0x7f949406a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:17.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.628+0000 7f949359e700 1 -- 172.21.15.78:0/2941696385 shutdown_connections 2023-10-11T09:48:17.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.628+0000 7f949359e700 1 -- 172.21.15.78:0/2941696385 wait complete. 2023-10-11T09:48:17.656 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-11T09:48:17.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.812+0000 7f085cda5700 1 Processor -- start 2023-10-11T09:48:17.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.812+0000 7f085cda5700 1 -- start start 2023-10-11T09:48:17.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.816+0000 7f085cda5700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f085815c880 0x7f085806be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:17.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.816+0000 7f085cda5700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f08580cbd30 0x7f08580c9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:17.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.816+0000 7f085cda5700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08580c93a0 con 0x7f08580cd210 2023-10-11T09:48:17.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.816+0000 7f085cda5700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0858161420 con 0x7f08580cbd30 2023-10-11T09:48:17.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.816+0000 7f085cda5700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08580cca10 con 0x7f085815c880 2023-10-11T09:48:17.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.816+0000 7f0856d9d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f08580cbd30 0x7f08580c9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:17.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.816+0000 7f085659c700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f085815c880 0x7f085806be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:17.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.816+0000 7f085659c700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f085815c880 0x7f085806be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:54794/0 (socket says 172.21.15.78:54794) 2023-10-11T09:48:17.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.816+0000 7f085659c700 1 -- 172.21.15.78:0/2024064469 learned_addr learned my addr 172.21.15.78:0/2024064469 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:17.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.816+0000 7f0856d9d700 1 -- 172.21.15.78:0/2024064469 >> v2:172.21.15.133:3300/0 conn(0x7f085815c880 msgr2=0x7f085806be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:17.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.816+0000 7f0856d9d700 1 --2- 172.21.15.78:0/2024064469 >> v2:172.21.15.133:3300/0 conn(0x7f085815c880 0x7f085806be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:17.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.816+0000 7f0856d9d700 1 -- 172.21.15.78:0/2024064469 >> v1:172.21.15.78:6789/0 conn(0x7f08580cd210 legacy=0x7f08580cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:17.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.816+0000 7f0856d9d700 1 -- 172.21.15.78:0/2024064469 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08580cccb0 con 0x7f08580cbd30 2023-10-11T09:48:17.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.820+0000 7f0856d9d700 1 --2- 172.21.15.78:0/2024064469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08580cbd30 0x7f08580c9b40 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f085806d8e0 tx=0x7f0850009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=438b9ad3f6978fd server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:17.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.820+0000 7f085559a700 1 -- 172.21.15.78:0/2024064469 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0850011830 con 0x7f08580cbd30 2023-10-11T09:48:17.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.820+0000 7f085559a700 1 -- 172.21.15.78:0/2024064469 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0850011990 con 0x7f08580cbd30 2023-10-11T09:48:17.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.820+0000 7f085559a700 1 -- 172.21.15.78:0/2024064469 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0850011d20 con 0x7f08580cbd30 2023-10-11T09:48:17.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.820+0000 7f085cda5700 1 -- 172.21.15.78:0/2024064469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08580cbd30 msgr2=0x7f08580c9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:17.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.820+0000 7f085cda5700 1 --2- 172.21.15.78:0/2024064469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08580cbd30 0x7f08580c9b40 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f085806d8e0 tx=0x7f0850009b30 comp rx=0 tx=0).stop 2023-10-11T09:48:17.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.820+0000 7f085cda5700 1 -- 172.21.15.78:0/2024064469 shutdown_connections 2023-10-11T09:48:17.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.820+0000 7f085cda5700 1 --2- 172.21.15.78:0/2024064469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08580cbd30 0x7f08580c9b40 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:17.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.820+0000 7f085cda5700 1 --2- 172.21.15.78:0/2024064469 >> v2:172.21.15.133:3300/0 conn(0x7f085815c880 0x7f085806be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:17.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.820+0000 7f085cda5700 1 -- 172.21.15.78:0/2024064469 >> 172.21.15.78:0/2024064469 conn(0x7f085806aca0 msgr2=0x7f08580b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:17.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.820+0000 7f085cda5700 1 -- 172.21.15.78:0/2024064469 shutdown_connections 2023-10-11T09:48:17.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.820+0000 7f085cda5700 1 -- 172.21.15.78:0/2024064469 wait complete. 2023-10-11T09:48:17.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.824+0000 7f085cda5700 1 Processor -- start 2023-10-11T09:48:17.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.824+0000 7f085cda5700 1 -- start start 2023-10-11T09:48:17.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.824+0000 7f085cda5700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08580cbd30 0x7f0858151b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:17.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.824+0000 7f085cda5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08580cd210 0x7f08580af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:17.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.824+0000 7f085cda5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f085815c880 0x7f0858160140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:17.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.824+0000 7f085cda5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08580b4180 con 0x7f085815c880 2023-10-11T09:48:17.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.824+0000 7f085cda5700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08580b4420 con 0x7f08580cd210 2023-10-11T09:48:17.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.824+0000 7f085cda5700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08580b4700 con 0x7f08580cbd30 2023-10-11T09:48:17.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.824+0000 7f0856d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f085815c880 0x7f0858160140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:17.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.824+0000 7f0856d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f085815c880 0x7f0858160140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33830/0 (socket says 172.21.15.78:33830) 2023-10-11T09:48:17.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.824+0000 7f0856d9d700 1 -- 172.21.15.78:0/927312244 learned_addr learned my addr 172.21.15.78:0/927312244 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:17.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.824+0000 7f085659c700 1 --2- 172.21.15.78:0/927312244 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08580cbd30 0x7f0858151b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:17.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.824+0000 7f0855d9b700 1 --2- 172.21.15.78:0/927312244 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08580cd210 0x7f08580af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:17.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.824+0000 7f0856d9d700 1 -- 172.21.15.78:0/927312244 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08580cbd30 msgr2=0x7f0858151b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:17.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.824+0000 7f0856d9d700 1 --2- 172.21.15.78:0/927312244 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08580cbd30 0x7f0858151b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:17.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.824+0000 7f0856d9d700 1 -- 172.21.15.78:0/927312244 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08580cd210 msgr2=0x7f08580af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:17.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.824+0000 7f0856d9d700 1 --2- 172.21.15.78:0/927312244 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08580cd210 0x7f08580af7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:17.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.824+0000 7f0856d9d700 1 -- 172.21.15.78:0/927312244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08500094b0 con 0x7f085815c880 2023-10-11T09:48:17.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.824+0000 7f0856d9d700 1 --2- 172.21.15.78:0/927312244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f085815c880 0x7f0858160140 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f0850009b70 tx=0x7f0850004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:17.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.824+0000 7f0846ffd700 1 -- 172.21.15.78:0/927312244 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0850004380 con 0x7f085815c880 2023-10-11T09:48:17.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.824+0000 7f0846ffd700 1 -- 172.21.15.78:0/927312244 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f085001f070 con 0x7f085815c880 2023-10-11T09:48:17.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.824+0000 7f0846ffd700 1 -- 172.21.15.78:0/927312244 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08500036c0 con 0x7f085815c880 2023-10-11T09:48:17.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.824+0000 7f085cda5700 1 -- 172.21.15.78:0/927312244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08580b49e0 con 0x7f085815c880 2023-10-11T09:48:17.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.828+0000 7f085cda5700 1 -- 172.21.15.78:0/927312244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0858158440 con 0x7f085815c880 2023-10-11T09:48:17.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.828+0000 7f085cda5700 1 -- 172.21.15.78:0/927312244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08580cd6f0 con 0x7f085815c880 2023-10-11T09:48:17.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.828+0000 7f0846ffd700 1 -- 172.21.15.78:0/927312244 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f0850021030 con 0x7f085815c880 2023-10-11T09:48:17.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.832+0000 7f0846ffd700 1 --2- 172.21.15.78:0/927312244 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0838065ed0 0x7f0838068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:17.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.832+0000 7f0846ffd700 1 -- 172.21.15.78:0/927312244 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0850084750 con 0x7f085815c880 2023-10-11T09:48:17.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.832+0000 7f085659c700 1 --2- 172.21.15.78:0/927312244 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0838065ed0 0x7f0838068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:17.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.832+0000 7f085659c700 1 --2- 172.21.15.78:0/927312244 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0838065ed0 0x7f0838068380 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f08580cad60 tx=0x7f084800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:17.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:17.836+0000 7f0846ffd700 1 -- 172.21.15.78:0/927312244 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f0850052100 con 0x7f085815c880 2023-10-11T09:48:18.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.008+0000 7f085cda5700 1 -- 172.21.15.78:0/927312244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f085804e530 con 0x7f085815c880 2023-10-11T09:48:18.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.012+0000 7f0846ffd700 1 -- 172.21.15.78:0/927312244 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) v1 ==== 84+0+126 (secure 0 0 0) 0x7f0850055720 con 0x7f085815c880 2023-10-11T09:48:18.020 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:18.020 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2023-10-11T09:48:18.020 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:48:18.020 INFO:teuthology.orchestra.run.smithi078.stdout: "poolnum": 1, 2023-10-11T09:48:18.021 INFO:teuthology.orchestra.run.smithi078.stdout: "poolname": ".mgr" 2023-10-11T09:48:18.021 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:48:18.021 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:48:18.021 INFO:teuthology.orchestra.run.smithi078.stdout: "poolnum": 2, 2023-10-11T09:48:18.021 INFO:teuthology.orchestra.run.smithi078.stdout: "poolname": "rbd" 2023-10-11T09:48:18.022 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:48:18.022 INFO:teuthology.orchestra.run.smithi078.stdout:] 2023-10-11T09:48:18.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.012+0000 7f085cda5700 1 -- 172.21.15.78:0/927312244 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0838065ed0 msgr2=0x7f0838068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:18.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.012+0000 7f085cda5700 1 --2- 172.21.15.78:0/927312244 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0838065ed0 0x7f0838068380 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f08580cad60 tx=0x7f084800b040 comp rx=0 tx=0).stop 2023-10-11T09:48:18.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.012+0000 7f085cda5700 1 -- 172.21.15.78:0/927312244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f085815c880 msgr2=0x7f0858160140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:18.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.012+0000 7f085cda5700 1 --2- 172.21.15.78:0/927312244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f085815c880 0x7f0858160140 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f0850009b70 tx=0x7f0850004770 comp rx=0 tx=0).stop 2023-10-11T09:48:18.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.012+0000 7f085cda5700 1 -- 172.21.15.78:0/927312244 shutdown_connections 2023-10-11T09:48:18.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.012+0000 7f085cda5700 1 --2- 172.21.15.78:0/927312244 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0838065ed0 0x7f0838068380 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:18.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.012+0000 7f085cda5700 1 --2- 172.21.15.78:0/927312244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f085815c880 0x7f0858160140 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:18.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.012+0000 7f085cda5700 1 --2- 172.21.15.78:0/927312244 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08580cd210 0x7f08580af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:18.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.012+0000 7f085cda5700 1 --2- 172.21.15.78:0/927312244 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08580cbd30 0x7f0858151b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:18.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.012+0000 7f085cda5700 1 -- 172.21.15.78:0/927312244 >> 172.21.15.78:0/927312244 conn(0x7f085806aca0 msgr2=0x7f0858066010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:18.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.012+0000 7f085cda5700 1 -- 172.21.15.78:0/927312244 shutdown_connections 2023-10-11T09:48:18.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.012+0000 7f085cda5700 1 -- 172.21.15.78:0/927312244 wait complete. 2023-10-11T09:48:18.043 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-11T09:48:18.044 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-11T09:48:18.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.216+0000 7ff6a17df700 1 Processor -- start 2023-10-11T09:48:18.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.216+0000 7ff6a17df700 1 -- start start 2023-10-11T09:48:18.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.220+0000 7ff6a17df700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff69c160d90 0x7ff69c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:18.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.220+0000 7ff6a17df700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff69c1649c0 0x7ff69c164e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:18.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.220+0000 7ff6a17df700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff69c0cd3e0 con 0x7ff69c0c1a70 2023-10-11T09:48:18.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.220+0000 7ff6a17df700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff69c0710f0 con 0x7ff69c1649c0 2023-10-11T09:48:18.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.220+0000 7ff6a17df700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c0cb8f0 con 0x7ff69c160d90 2023-10-11T09:48:18.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.220+0000 7ff69affd700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff69c160d90 0x7ff69c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:18.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.220+0000 7ff69b7fe700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff69c1649c0 0x7ff69c164e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:18.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.220+0000 7ff69affd700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff69c160d90 0x7ff69c06c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:36950/0 (socket says 172.21.15.78:36950) 2023-10-11T09:48:18.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.220+0000 7ff69affd700 1 -- 172.21.15.78:0/3955563645 learned_addr learned my addr 172.21.15.78:0/3955563645 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:18.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.220+0000 7ff69affd700 1 -- 172.21.15.78:0/3955563645 >> v2:172.21.15.133:3300/0 conn(0x7ff69c1649c0 msgr2=0x7ff69c164e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:18.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.220+0000 7ff69affd700 1 --2- 172.21.15.78:0/3955563645 >> v2:172.21.15.133:3300/0 conn(0x7ff69c1649c0 0x7ff69c164e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:18.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.220+0000 7ff69affd700 1 -- 172.21.15.78:0/3955563645 >> v1:172.21.15.133:6789/0 conn(0x7ff69c0c1a70 legacy=0x7ff69c0c1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:18.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.220+0000 7ff69affd700 1 -- 172.21.15.78:0/3955563645 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff69c072300 con 0x7ff69c160d90 2023-10-11T09:48:18.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.224+0000 7ff69affd700 1 --2- 172.21.15.78:0/3955563645 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff69c160d90 0x7ff69c06c630 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff68c009910 tx=0x7ff68c009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac5a2ea877f47430 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:18.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.224+0000 7ff69a7fc700 1 -- 172.21.15.78:0/3955563645 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68c011830 con 0x7ff69c160d90 2023-10-11T09:48:18.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.224+0000 7ff69a7fc700 1 -- 172.21.15.78:0/3955563645 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff68c011990 con 0x7ff69c160d90 2023-10-11T09:48:18.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.224+0000 7ff69a7fc700 1 -- 172.21.15.78:0/3955563645 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68c011d40 con 0x7ff69c160d90 2023-10-11T09:48:18.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.224+0000 7ff6a17df700 1 -- 172.21.15.78:0/3955563645 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff69c160d90 msgr2=0x7ff69c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:18.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.224+0000 7ff6a17df700 1 --2- 172.21.15.78:0/3955563645 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff69c160d90 0x7ff69c06c630 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff68c009910 tx=0x7ff68c009c20 comp rx=0 tx=0).stop 2023-10-11T09:48:18.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.224+0000 7ff6a17df700 1 -- 172.21.15.78:0/3955563645 shutdown_connections 2023-10-11T09:48:18.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.224+0000 7ff6a17df700 1 --2- 172.21.15.78:0/3955563645 >> v2:172.21.15.133:3300/0 conn(0x7ff69c1649c0 0x7ff69c164e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:18.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.224+0000 7ff6a17df700 1 --2- 172.21.15.78:0/3955563645 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff69c160d90 0x7ff69c06c630 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:18.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.224+0000 7ff6a17df700 1 -- 172.21.15.78:0/3955563645 >> 172.21.15.78:0/3955563645 conn(0x7ff69c06a060 msgr2=0x7ff69c06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:18.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.224+0000 7ff6a17df700 1 -- 172.21.15.78:0/3955563645 shutdown_connections 2023-10-11T09:48:18.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.224+0000 7ff6a17df700 1 -- 172.21.15.78:0/3955563645 wait complete. 2023-10-11T09:48:18.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.228+0000 7ff6a17df700 1 Processor -- start 2023-10-11T09:48:18.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.228+0000 7ff6a17df700 1 -- start start 2023-10-11T09:48:18.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.228+0000 7ff6a17df700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff69c0c1a70 0x7ff69c0af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:18.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.228+0000 7ff6a17df700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff69c160d90 0x7ff69c15f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:18.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.228+0000 7ff6a17df700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff69c1649c0 0x7ff69c0afc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:18.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.228+0000 7ff6a17df700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c161300 con 0x7ff69c1649c0 2023-10-11T09:48:18.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.228+0000 7ff6a17df700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c0ae900 con 0x7ff69c0c1a70 2023-10-11T09:48:18.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.228+0000 7ff6a17df700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c0aebe0 con 0x7ff69c160d90 2023-10-11T09:48:18.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.228+0000 7ff69b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff69c1649c0 0x7ff69c0afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:18.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.228+0000 7ff69b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff69c1649c0 0x7ff69c0afc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33834/0 (socket says 172.21.15.78:33834) 2023-10-11T09:48:18.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.228+0000 7ff69b7fe700 1 -- 172.21.15.78:0/2381785284 learned_addr learned my addr 172.21.15.78:0/2381785284 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:18.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.228+0000 7ff69affd700 1 --2- 172.21.15.78:0/2381785284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff69c0c1a70 0x7ff69c0af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:18.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.228+0000 7ff693fff700 1 --2- 172.21.15.78:0/2381785284 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff69c160d90 0x7ff69c15f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:18.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.228+0000 7ff69b7fe700 1 -- 172.21.15.78:0/2381785284 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff69c160d90 msgr2=0x7ff69c15f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:18.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.228+0000 7ff69b7fe700 1 --2- 172.21.15.78:0/2381785284 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff69c160d90 0x7ff69c15f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:18.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.228+0000 7ff69b7fe700 1 -- 172.21.15.78:0/2381785284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff69c0c1a70 msgr2=0x7ff69c0af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:18.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.228+0000 7ff69b7fe700 1 --2- 172.21.15.78:0/2381785284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff69c0c1a70 0x7ff69c0af6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:18.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.228+0000 7ff69b7fe700 1 -- 172.21.15.78:0/2381785284 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68c009570 con 0x7ff69c1649c0 2023-10-11T09:48:18.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.228+0000 7ff69b7fe700 1 --2- 172.21.15.78:0/2381785284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff69c1649c0 0x7ff69c0afc90 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7ff69400efc0 tx=0x7ff69400c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:18.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.232+0000 7ff6937fe700 1 -- 172.21.15.78:0/2381785284 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69400cb80 con 0x7ff69c1649c0 2023-10-11T09:48:18.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.232+0000 7ff6937fe700 1 -- 172.21.15.78:0/2381785284 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69400f040 con 0x7ff69c1649c0 2023-10-11T09:48:18.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.232+0000 7ff6937fe700 1 -- 172.21.15.78:0/2381785284 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff694014480 con 0x7ff69c1649c0 2023-10-11T09:48:18.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.232+0000 7ff6a17df700 1 -- 172.21.15.78:0/2381785284 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff69c0aeec0 con 0x7ff69c1649c0 2023-10-11T09:48:18.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.232+0000 7ff6a17df700 1 -- 172.21.15.78:0/2381785284 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff69c15fce0 con 0x7ff69c1649c0 2023-10-11T09:48:18.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.232+0000 7ff6a17df700 1 -- 172.21.15.78:0/2381785284 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff69c04e530 con 0x7ff69c1649c0 2023-10-11T09:48:18.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.232+0000 7ff6937fe700 1 -- 172.21.15.78:0/2381785284 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7ff6940145e0 con 0x7ff69c1649c0 2023-10-11T09:48:18.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.236+0000 7ff6937fe700 1 --2- 172.21.15.78:0/2381785284 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff67c065f00 0x7ff67c0683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:18.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.236+0000 7ff6937fe700 1 -- 172.21.15.78:0/2381785284 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff694083c50 con 0x7ff69c1649c0 2023-10-11T09:48:18.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.236+0000 7ff69affd700 1 --2- 172.21.15.78:0/2381785284 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff67c065f00 0x7ff67c0683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:18.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.236+0000 7ff69affd700 1 --2- 172.21.15.78:0/2381785284 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff67c065f00 0x7ff67c0683b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff68c000c00 tx=0x7ff68c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:18.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.240+0000 7ff6937fe700 1 -- 172.21.15.78:0/2381785284 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7ff6940515e0 con 0x7ff69c1649c0 2023-10-11T09:48:18.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.424+0000 7ff6a17df700 1 -- 172.21.15.78:0/2381785284 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff69c06c410 con 0x7ff69c1649c0 2023-10-11T09:48:18.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.540+0000 7ff6937fe700 1 -- 172.21.15.78:0/2381785284 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff694054c00 con 0x7ff69c1649c0 2023-10-11T09:48:18.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.544+0000 7ff6a17df700 1 -- 172.21.15.78:0/2381785284 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff67c065f00 msgr2=0x7ff67c0683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:18.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.544+0000 7ff6a17df700 1 --2- 172.21.15.78:0/2381785284 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff67c065f00 0x7ff67c0683b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff68c000c00 tx=0x7ff68c008030 comp rx=0 tx=0).stop 2023-10-11T09:48:18.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.544+0000 7ff6a17df700 1 -- 172.21.15.78:0/2381785284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff69c1649c0 msgr2=0x7ff69c0afc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:18.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.544+0000 7ff6a17df700 1 --2- 172.21.15.78:0/2381785284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff69c1649c0 0x7ff69c0afc90 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7ff69400efc0 tx=0x7ff69400c0c0 comp rx=0 tx=0).stop 2023-10-11T09:48:18.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.544+0000 7ff6a17df700 1 -- 172.21.15.78:0/2381785284 shutdown_connections 2023-10-11T09:48:18.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.544+0000 7ff6a17df700 1 --2- 172.21.15.78:0/2381785284 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff67c065f00 0x7ff67c0683b0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:18.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.544+0000 7ff6a17df700 1 --2- 172.21.15.78:0/2381785284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff69c1649c0 0x7ff69c0afc90 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:18.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.544+0000 7ff6a17df700 1 --2- 172.21.15.78:0/2381785284 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff69c160d90 0x7ff69c15f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:18.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.544+0000 7ff6a17df700 1 --2- 172.21.15.78:0/2381785284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff69c0c1a70 0x7ff69c0af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:18.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.544+0000 7ff6a17df700 1 -- 172.21.15.78:0/2381785284 >> 172.21.15.78:0/2381785284 conn(0x7ff69c06a060 msgr2=0x7ff69c0b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:18.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.548+0000 7ff6a17df700 1 -- 172.21.15.78:0/2381785284 shutdown_connections 2023-10-11T09:48:18.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.548+0000 7ff6a17df700 1 -- 172.21.15.78:0/2381785284 wait complete. 2023-10-11T09:48:18.560 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' created 2023-10-11T09:48:18.577 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-11T09:48:18.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.744+0000 7f842759e700 1 Processor -- start 2023-10-11T09:48:18.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.744+0000 7f842759e700 1 -- start start 2023-10-11T09:48:18.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.748+0000 7f842759e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84280ce240 0x7f8428161210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:18.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.748+0000 7f842759e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f84280c1b00 0x7f8428165060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:18.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.748+0000 7f842759e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f84280b0570 0x7f84280b09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:18.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.748+0000 7f842759e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84280b0ef0 con 0x7f84280c1b00 2023-10-11T09:48:18.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.748+0000 7f842759e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8428161b40 con 0x7f84280b0570 2023-10-11T09:48:18.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.748+0000 7f842759e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8428161e20 con 0x7f84280ce240 2023-10-11T09:48:18.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.748+0000 7f842659c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84280ce240 0x7f8428161210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:18.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.748+0000 7f8425d9b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f84280c1b00 0x7f8428165060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:18.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.748+0000 7f842659c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84280ce240 0x7f8428161210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:36954/0 (socket says 172.21.15.78:36954) 2023-10-11T09:48:18.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.748+0000 7f842659c700 1 -- 172.21.15.78:0/523195243 learned_addr learned my addr 172.21.15.78:0/523195243 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:18.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.748+0000 7f8426d9d700 1 --2- 172.21.15.78:0/523195243 >> v2:172.21.15.133:3300/0 conn(0x7f84280b0570 0x7f84280b09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:18.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.748+0000 7f842659c700 1 -- 172.21.15.78:0/523195243 >> v2:172.21.15.133:3300/0 conn(0x7f84280b0570 msgr2=0x7f84280b09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:18.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.748+0000 7f842659c700 1 --2- 172.21.15.78:0/523195243 >> v2:172.21.15.133:3300/0 conn(0x7f84280b0570 0x7f84280b09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:18.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.748+0000 7f842659c700 1 -- 172.21.15.78:0/523195243 >> v2:172.21.15.78:3300/0 conn(0x7f84280c1b00 msgr2=0x7f8428165060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:18.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.748+0000 7f842659c700 1 --2- 172.21.15.78:0/523195243 >> v2:172.21.15.78:3300/0 conn(0x7f84280c1b00 0x7f8428165060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:18.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.748+0000 7f842659c700 1 -- 172.21.15.78:0/523195243 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8428162100 con 0x7f84280ce240 2023-10-11T09:48:18.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.752+0000 7f842659c700 1 --2- 172.21.15.78:0/523195243 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84280ce240 0x7f8428161210 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f84280ae480 tx=0x7f841800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=969b0660dd15e111 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:18.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.752+0000 7f842559a700 1 -- 172.21.15.78:0/523195243 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841800e070 con 0x7f84280ce240 2023-10-11T09:48:18.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.752+0000 7f842559a700 1 -- 172.21.15.78:0/523195243 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f841800b910 con 0x7f84280ce240 2023-10-11T09:48:18.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.752+0000 7f842559a700 1 -- 172.21.15.78:0/523195243 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841800bcc0 con 0x7f84280ce240 2023-10-11T09:48:18.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.752+0000 7f842759e700 1 -- 172.21.15.78:0/523195243 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84280ce240 msgr2=0x7f8428161210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:18.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.752+0000 7f842759e700 1 --2- 172.21.15.78:0/523195243 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84280ce240 0x7f8428161210 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f84280ae480 tx=0x7f841800b330 comp rx=0 tx=0).stop 2023-10-11T09:48:18.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.752+0000 7f842759e700 1 -- 172.21.15.78:0/523195243 shutdown_connections 2023-10-11T09:48:18.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.752+0000 7f842759e700 1 --2- 172.21.15.78:0/523195243 >> v2:172.21.15.133:3300/0 conn(0x7f84280b0570 0x7f84280b09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:18.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.752+0000 7f842759e700 1 --2- 172.21.15.78:0/523195243 >> v2:172.21.15.78:3300/0 conn(0x7f84280c1b00 0x7f8428165060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:18.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.752+0000 7f842759e700 1 --2- 172.21.15.78:0/523195243 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84280ce240 0x7f8428161210 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:18.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.752+0000 7f842759e700 1 -- 172.21.15.78:0/523195243 >> 172.21.15.78:0/523195243 conn(0x7f842806acb0 msgr2=0x7f84280cb570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:18.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.752+0000 7f842759e700 1 -- 172.21.15.78:0/523195243 shutdown_connections 2023-10-11T09:48:18.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.752+0000 7f842759e700 1 -- 172.21.15.78:0/523195243 wait complete. 2023-10-11T09:48:18.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.756+0000 7f842759e700 1 Processor -- start 2023-10-11T09:48:18.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.756+0000 7f842759e700 1 -- start start 2023-10-11T09:48:18.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.756+0000 7f842759e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84280b0570 0x7f842814aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:18.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.756+0000 7f842759e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84280c1b00 0x7f84280b1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:18.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.756+0000 7f842759e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84280ce240 0x7f84280b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:18.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.756+0000 7f842759e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842814e3d0 con 0x7f84280c1b00 2023-10-11T09:48:18.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.756+0000 7f842759e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84281fa260 con 0x7f84280ce240 2023-10-11T09:48:18.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.756+0000 7f842759e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f842814e0f0 con 0x7f84280b0570 2023-10-11T09:48:18.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.760+0000 7f8425d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84280c1b00 0x7f84280b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:18.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.760+0000 7f8425d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84280c1b00 0x7f84280b1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33842/0 (socket says 172.21.15.78:33842) 2023-10-11T09:48:18.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.760+0000 7f8425d9b700 1 -- 172.21.15.78:0/1388805727 learned_addr learned my addr 172.21.15.78:0/1388805727 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:18.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.760+0000 7f842659c700 1 --2- 172.21.15.78:0/1388805727 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84280b0570 0x7f842814aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:18.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.760+0000 7f8426d9d700 1 --2- 172.21.15.78:0/1388805727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84280ce240 0x7f84280b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:18.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.760+0000 7f8425d9b700 1 -- 172.21.15.78:0/1388805727 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84280b0570 msgr2=0x7f842814aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:18.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.760+0000 7f8425d9b700 1 --2- 172.21.15.78:0/1388805727 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84280b0570 0x7f842814aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:18.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.760+0000 7f8425d9b700 1 -- 172.21.15.78:0/1388805727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84280ce240 msgr2=0x7f84280b1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:18.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.760+0000 7f8425d9b700 1 --2- 172.21.15.78:0/1388805727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84280ce240 0x7f84280b1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:18.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.760+0000 7f8425d9b700 1 -- 172.21.15.78:0/1388805727 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8418009b30 con 0x7f84280c1b00 2023-10-11T09:48:18.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.760+0000 7f8425d9b700 1 --2- 172.21.15.78:0/1388805727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84280c1b00 0x7f84280b1920 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f841c00e9d0 tx=0x7f841c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:18.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.760+0000 7f8416ffd700 1 -- 172.21.15.78:0/1388805727 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841c009720 con 0x7f84280c1b00 2023-10-11T09:48:18.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.760+0000 7f8416ffd700 1 -- 172.21.15.78:0/1388805727 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f841c00ee70 con 0x7f84280c1b00 2023-10-11T09:48:18.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.760+0000 7f8416ffd700 1 -- 172.21.15.78:0/1388805727 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841c00ca50 con 0x7f84280c1b00 2023-10-11T09:48:18.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.760+0000 7f842759e700 1 -- 172.21.15.78:0/1388805727 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84281f9ba0 con 0x7f84280c1b00 2023-10-11T09:48:18.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.760+0000 7f842759e700 1 -- 172.21.15.78:0/1388805727 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f842814b000 con 0x7f84280c1b00 2023-10-11T09:48:18.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.764+0000 7f842759e700 1 -- 172.21.15.78:0/1388805727 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84281f9a00 con 0x7f84280c1b00 2023-10-11T09:48:18.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.764+0000 7f8416ffd700 1 -- 172.21.15.78:0/1388805727 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f841c00cbb0 con 0x7f84280c1b00 2023-10-11T09:48:18.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.768+0000 7f8416ffd700 1 --2- 172.21.15.78:0/1388805727 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8408065ed0 0x7f8408068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:18.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.768+0000 7f8416ffd700 1 -- 172.21.15.78:0/1388805727 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f841c083530 con 0x7f84280c1b00 2023-10-11T09:48:18.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.768+0000 7f842659c700 1 --2- 172.21.15.78:0/1388805727 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8408065ed0 0x7f8408068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:18.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.768+0000 7f842659c700 1 --2- 172.21.15.78:0/1388805727 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8408065ed0 0x7f8408068380 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f841800b370 tx=0x7f8418009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:18.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.772+0000 7f8416ffd700 1 -- 172.21.15.78:0/1388805727 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f84281f9a00 con 0x7f84280c1b00 2023-10-11T09:48:18.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:18.948+0000 7f842759e700 1 -- 172.21.15.78:0/1388805727 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f84281f9a00 con 0x7f84280c1b00 2023-10-11T09:48:19.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.552+0000 7f8416ffd700 1 -- 172.21.15.78:0/1388805727 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v81) v1 ==== 170+0+0 (secure 0 0 0) 0x7f84281f9a00 con 0x7f84280c1b00 2023-10-11T09:48:19.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.564+0000 7f842759e700 1 -- 172.21.15.78:0/1388805727 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8408065ed0 msgr2=0x7f8408068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:19.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.564+0000 7f842759e700 1 --2- 172.21.15.78:0/1388805727 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8408065ed0 0x7f8408068380 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f841800b370 tx=0x7f8418009000 comp rx=0 tx=0).stop 2023-10-11T09:48:19.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.564+0000 7f842759e700 1 -- 172.21.15.78:0/1388805727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84280c1b00 msgr2=0x7f84280b1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:19.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.564+0000 7f842759e700 1 --2- 172.21.15.78:0/1388805727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84280c1b00 0x7f84280b1920 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f841c00e9d0 tx=0x7f841c00ece0 comp rx=0 tx=0).stop 2023-10-11T09:48:19.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.568+0000 7f842759e700 1 -- 172.21.15.78:0/1388805727 shutdown_connections 2023-10-11T09:48:19.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.568+0000 7f842759e700 1 --2- 172.21.15.78:0/1388805727 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8408065ed0 0x7f8408068380 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:19.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.568+0000 7f842759e700 1 --2- 172.21.15.78:0/1388805727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84280ce240 0x7f84280b1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:19.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.568+0000 7f842759e700 1 --2- 172.21.15.78:0/1388805727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84280c1b00 0x7f84280b1920 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:19.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.568+0000 7f842759e700 1 --2- 172.21.15.78:0/1388805727 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84280b0570 0x7f842814aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:19.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.568+0000 7f842759e700 1 -- 172.21.15.78:0/1388805727 >> 172.21.15.78:0/1388805727 conn(0x7f842806acb0 msgr2=0x7f84280afde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:19.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.568+0000 7f842759e700 1 -- 172.21.15.78:0/1388805727 shutdown_connections 2023-10-11T09:48:19.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.572+0000 7f842759e700 1 -- 172.21.15.78:0/1388805727 wait complete. 2023-10-11T09:48:19.582 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' created 2023-10-11T09:48:19.601 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-11T09:48:19.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.768+0000 7f64f55c3700 1 Processor -- start 2023-10-11T09:48:19.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.768+0000 7f64f55c3700 1 -- start start 2023-10-11T09:48:19.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.772+0000 7f64f55c3700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f64f015c900 0x7f64f00b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:19.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.772+0000 7f64f55c3700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64f00ce7f0 0x7f64f00cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:19.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.772+0000 7f64f55c3700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64f006fde0 con 0x7f64f00ad290 2023-10-11T09:48:19.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.772+0000 7f64f55c3700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64f00705f0 con 0x7f64f015c900 2023-10-11T09:48:19.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.772+0000 7f64f55c3700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64f00cf1b0 con 0x7f64f00ce7f0 2023-10-11T09:48:19.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.772+0000 7f64effff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f64f015c900 0x7f64f00b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:19.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.772+0000 7f64effff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f64f015c900 0x7f64f00b1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33850/0 (socket says 172.21.15.78:33850) 2023-10-11T09:48:19.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.772+0000 7f64effff700 1 -- 172.21.15.78:0/809700186 learned_addr learned my addr 172.21.15.78:0/809700186 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:19.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.772+0000 7f64f4dc2700 1 --2- 172.21.15.78:0/809700186 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64f00ce7f0 0x7f64f00cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:19.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.772+0000 7f64effff700 1 -- 172.21.15.78:0/809700186 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64f00ce7f0 msgr2=0x7f64f00cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:19.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.772+0000 7f64effff700 1 --2- 172.21.15.78:0/809700186 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64f00ce7f0 0x7f64f00cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:19.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.772+0000 7f64effff700 1 -- 172.21.15.78:0/809700186 >> v1:172.21.15.133:6789/0 conn(0x7f64f00ad290 legacy=0x7f64f0071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:19.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.772+0000 7f64effff700 1 -- 172.21.15.78:0/809700186 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f00ae000 con 0x7f64f015c900 2023-10-11T09:48:19.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.772+0000 7f64effff700 1 --2- 172.21.15.78:0/809700186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64f015c900 0x7f64f00b1a10 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f64e0009910 tx=0x7f64e0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76546fa5e98bf99b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:19.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.772+0000 7f64eeffd700 1 -- 172.21.15.78:0/809700186 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e0004230 con 0x7f64f015c900 2023-10-11T09:48:19.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.772+0000 7f64eeffd700 1 -- 172.21.15.78:0/809700186 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e0004390 con 0x7f64f015c900 2023-10-11T09:48:19.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.772+0000 7f64eeffd700 1 -- 172.21.15.78:0/809700186 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e0009e60 con 0x7f64f015c900 2023-10-11T09:48:19.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.776+0000 7f64f55c3700 1 -- 172.21.15.78:0/809700186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64f015c900 msgr2=0x7f64f00b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:19.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.776+0000 7f64f55c3700 1 --2- 172.21.15.78:0/809700186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64f015c900 0x7f64f00b1a10 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f64e0009910 tx=0x7f64e0009c20 comp rx=0 tx=0).stop 2023-10-11T09:48:19.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.776+0000 7f64f55c3700 1 -- 172.21.15.78:0/809700186 shutdown_connections 2023-10-11T09:48:19.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.776+0000 7f64f55c3700 1 --2- 172.21.15.78:0/809700186 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64f00ce7f0 0x7f64f00cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:19.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.776+0000 7f64f55c3700 1 --2- 172.21.15.78:0/809700186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64f015c900 0x7f64f00b1a10 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:19.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.776+0000 7f64f55c3700 1 -- 172.21.15.78:0/809700186 >> 172.21.15.78:0/809700186 conn(0x7f64f006a060 msgr2=0x7f64f006a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:19.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.776+0000 7f64f55c3700 1 -- 172.21.15.78:0/809700186 shutdown_connections 2023-10-11T09:48:19.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.776+0000 7f64f55c3700 1 -- 172.21.15.78:0/809700186 wait complete. 2023-10-11T09:48:19.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.776+0000 7f64f55c3700 1 Processor -- start 2023-10-11T09:48:19.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.780+0000 7f64f55c3700 1 -- start start 2023-10-11T09:48:19.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.780+0000 7f64f55c3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64f00ad290 0x7f64f00ca530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:19.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.780+0000 7f64f55c3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64f00ce7f0 0x7f64f0151ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:19.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.780+0000 7f64f55c3700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64f015c900 0x7f64f01e7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:19.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.780+0000 7f64f55c3700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f01e5af0 con 0x7f64f00ce7f0 2023-10-11T09:48:19.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.780+0000 7f64f55c3700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f01e6370 con 0x7f64f00ad290 2023-10-11T09:48:19.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.780+0000 7f64f55c3700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64f01e5540 con 0x7f64f015c900 2023-10-11T09:48:19.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.780+0000 7f64ef7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64f00ce7f0 0x7f64f0151ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:19.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.780+0000 7f64effff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64f00ad290 0x7f64f00ca530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:19.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.780+0000 7f64ef7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64f00ce7f0 0x7f64f0151ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33860/0 (socket says 172.21.15.78:33860) 2023-10-11T09:48:19.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.780+0000 7f64ef7fe700 1 -- 172.21.15.78:0/2937446483 learned_addr learned my addr 172.21.15.78:0/2937446483 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:19.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.780+0000 7f64f4dc2700 1 --2- 172.21.15.78:0/2937446483 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64f015c900 0x7f64f01e7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:19.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.780+0000 7f64ef7fe700 1 -- 172.21.15.78:0/2937446483 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64f015c900 msgr2=0x7f64f01e7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:19.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.780+0000 7f64ef7fe700 1 --2- 172.21.15.78:0/2937446483 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64f015c900 0x7f64f01e7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:19.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.780+0000 7f64ef7fe700 1 -- 172.21.15.78:0/2937446483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64f00ad290 msgr2=0x7f64f00ca530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:19.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.780+0000 7f64ef7fe700 1 --2- 172.21.15.78:0/2937446483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64f00ad290 0x7f64f00ca530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:19.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.780+0000 7f64ef7fe700 1 -- 172.21.15.78:0/2937446483 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e0009570 con 0x7f64f00ce7f0 2023-10-11T09:48:19.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.780+0000 7f64ef7fe700 1 --2- 172.21.15.78:0/2937446483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64f00ce7f0 0x7f64f0151ae0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f64e400b4a0 tx=0x7f64e400b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:19.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.780+0000 7f64ecff9700 1 -- 172.21.15.78:0/2937446483 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e4011830 con 0x7f64f00ce7f0 2023-10-11T09:48:19.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.784+0000 7f64ecff9700 1 -- 172.21.15.78:0/2937446483 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e4011990 con 0x7f64f00ce7f0 2023-10-11T09:48:19.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.784+0000 7f64ecff9700 1 -- 172.21.15.78:0/2937446483 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e4011d40 con 0x7f64f00ce7f0 2023-10-11T09:48:19.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.784+0000 7f64f55c3700 1 -- 172.21.15.78:0/2937446483 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f01e5d20 con 0x7f64f00ce7f0 2023-10-11T09:48:19.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.784+0000 7f64f55c3700 1 -- 172.21.15.78:0/2937446483 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f01e5160 con 0x7f64f00ce7f0 2023-10-11T09:48:19.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.784+0000 7f64f55c3700 1 -- 172.21.15.78:0/2937446483 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64f0065d40 con 0x7f64f00ce7f0 2023-10-11T09:48:19.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.784+0000 7f64ecff9700 1 -- 172.21.15.78:0/2937446483 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f64e401a400 con 0x7f64f00ce7f0 2023-10-11T09:48:19.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.788+0000 7f64ecff9700 1 --2- 172.21.15.78:0/2937446483 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f64dc065ed0 0x7f64dc068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:19.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.788+0000 7f64ecff9700 1 -- 172.21.15.78:0/2937446483 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f64e4083460 con 0x7f64f00ce7f0 2023-10-11T09:48:19.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.788+0000 7f64effff700 1 --2- 172.21.15.78:0/2937446483 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f64dc065ed0 0x7f64dc068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:19.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.788+0000 7f64effff700 1 --2- 172.21.15.78:0/2937446483 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f64dc065ed0 0x7f64dc068380 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f64e0009910 tx=0x7f64e0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:19.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:19.796+0000 7f64ecff9700 1 -- 172.21.15.78:0/2937446483 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f64e4050a70 con 0x7f64f00ce7f0 2023-10-11T09:48:20.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.016+0000 7f64f55c3700 1 -- 172.21.15.78:0/2937446483 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f64f00cf5b0 con 0x7f64f00ce7f0 2023-10-11T09:48:20.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.028+0000 7f64ecff9700 1 -- 172.21.15.78:0/2937446483 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v54) v1 ==== 174+0+0 (secure 0 0 0) 0x7f64e4054090 con 0x7f64f00ce7f0 2023-10-11T09:48:20.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.036+0000 7f64f55c3700 1 -- 172.21.15.78:0/2937446483 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f64dc065ed0 msgr2=0x7f64dc068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:20.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.036+0000 7f64f55c3700 1 --2- 172.21.15.78:0/2937446483 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f64dc065ed0 0x7f64dc068380 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f64e0009910 tx=0x7f64e0008030 comp rx=0 tx=0).stop 2023-10-11T09:48:20.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.036+0000 7f64f55c3700 1 -- 172.21.15.78:0/2937446483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64f00ce7f0 msgr2=0x7f64f0151ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:20.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.036+0000 7f64f55c3700 1 --2- 172.21.15.78:0/2937446483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64f00ce7f0 0x7f64f0151ae0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f64e400b4a0 tx=0x7f64e400b7b0 comp rx=0 tx=0).stop 2023-10-11T09:48:20.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.040+0000 7f64f55c3700 1 -- 172.21.15.78:0/2937446483 shutdown_connections 2023-10-11T09:48:20.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.060+0000 7f64f55c3700 1 --2- 172.21.15.78:0/2937446483 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f64dc065ed0 0x7f64dc068380 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:20.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.060+0000 7f64f55c3700 1 --2- 172.21.15.78:0/2937446483 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64f015c900 0x7f64f01e7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:20.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.060+0000 7f64f55c3700 1 --2- 172.21.15.78:0/2937446483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64f00ce7f0 0x7f64f0151ae0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:20.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.060+0000 7f64f55c3700 1 --2- 172.21.15.78:0/2937446483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64f00ad290 0x7f64f00ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:20.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.060+0000 7f64f55c3700 1 -- 172.21.15.78:0/2937446483 >> 172.21.15.78:0/2937446483 conn(0x7f64f006a060 msgr2=0x7f64f00b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:20.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.060+0000 7f64f55c3700 1 -- 172.21.15.78:0/2937446483 shutdown_connections 2023-10-11T09:48:20.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.060+0000 7f64f55c3700 1 -- 172.21.15.78:0/2937446483 wait complete. 2023-10-11T09:48:20.073 INFO:teuthology.orchestra.run.smithi078.stderr:new fs with metadata pool 11 and data pool 12 2023-10-11T09:48:20.090 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:48:20.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.248+0000 7f9e3eabd700 1 Processor -- start 2023-10-11T09:48:20.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.252+0000 7f9e3eabd700 1 -- start start 2023-10-11T09:48:20.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.252+0000 7f9e3eabd700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f9e3815b8e0 0x7f9e3806d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:20.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.252+0000 7f9e3eabd700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9e380ca950 0x7f9e3813fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:20.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.252+0000 7f9e3eabd700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e3815e010 con 0x7f9e3806d610 2023-10-11T09:48:20.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.252+0000 7f9e3eabd700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e3815e280 con 0x7f9e3815b8e0 2023-10-11T09:48:20.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.252+0000 7f9e3eabd700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e3815e560 con 0x7f9e380ca950 2023-10-11T09:48:20.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.252+0000 7f9e3dabb700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f9e3815b8e0 0x7f9e3806d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:20.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.252+0000 7f9e3dabb700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f9e3815b8e0 0x7f9e3806d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33874/0 (socket says 172.21.15.78:33874) 2023-10-11T09:48:20.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.252+0000 7f9e3e2bc700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9e380ca950 0x7f9e3813fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:20.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.252+0000 7f9e3dabb700 1 -- 172.21.15.78:0/2678846683 learned_addr learned my addr 172.21.15.78:0/2678846683 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:20.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.256+0000 7f9e3dabb700 1 -- 172.21.15.78:0/2678846683 >> v2:172.21.15.133:3300/0 conn(0x7f9e380ca950 msgr2=0x7f9e3813fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:20.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.256+0000 7f9e3dabb700 1 --2- 172.21.15.78:0/2678846683 >> v2:172.21.15.133:3300/0 conn(0x7f9e380ca950 0x7f9e3813fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:20.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.256+0000 7f9e3dabb700 1 -- 172.21.15.78:0/2678846683 >> v1:172.21.15.133:6789/0 conn(0x7f9e3806d610 legacy=0x7f9e380ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:20.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.256+0000 7f9e3dabb700 1 -- 172.21.15.78:0/2678846683 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3815e840 con 0x7f9e3815b8e0 2023-10-11T09:48:20.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.256+0000 7f9e3dabb700 1 --2- 172.21.15.78:0/2678846683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e3815b8e0 0x7f9e3806d0d0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f9e3815efb0 tx=0x7f9e34009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e14bea2678c6da37 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:20.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.256+0000 7f9e3cab9700 1 -- 172.21.15.78:0/2678846683 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e34011830 con 0x7f9e3815b8e0 2023-10-11T09:48:20.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.256+0000 7f9e3cab9700 1 -- 172.21.15.78:0/2678846683 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e34011990 con 0x7f9e3815b8e0 2023-10-11T09:48:20.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.256+0000 7f9e3cab9700 1 -- 172.21.15.78:0/2678846683 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e34011d20 con 0x7f9e3815b8e0 2023-10-11T09:48:20.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.256+0000 7f9e3eabd700 1 -- 172.21.15.78:0/2678846683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e3815b8e0 msgr2=0x7f9e3806d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:20.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.256+0000 7f9e3eabd700 1 --2- 172.21.15.78:0/2678846683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e3815b8e0 0x7f9e3806d0d0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f9e3815efb0 tx=0x7f9e34009bf0 comp rx=0 tx=0).stop 2023-10-11T09:48:20.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.256+0000 7f9e3eabd700 1 -- 172.21.15.78:0/2678846683 shutdown_connections 2023-10-11T09:48:20.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.256+0000 7f9e3eabd700 1 --2- 172.21.15.78:0/2678846683 >> v2:172.21.15.133:3300/0 conn(0x7f9e380ca950 0x7f9e3813fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:20.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.256+0000 7f9e3eabd700 1 --2- 172.21.15.78:0/2678846683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e3815b8e0 0x7f9e3806d0d0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:20.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.256+0000 7f9e3eabd700 1 -- 172.21.15.78:0/2678846683 >> 172.21.15.78:0/2678846683 conn(0x7f9e380cea60 msgr2=0x7f9e380cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:20.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.256+0000 7f9e3eabd700 1 -- 172.21.15.78:0/2678846683 shutdown_connections 2023-10-11T09:48:20.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.256+0000 7f9e3eabd700 1 -- 172.21.15.78:0/2678846683 wait complete. 2023-10-11T09:48:20.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.260+0000 7f9e3eabd700 1 Processor -- start 2023-10-11T09:48:20.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.260+0000 7f9e3eabd700 1 -- start start 2023-10-11T09:48:20.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.260+0000 7f9e3eabd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e3806d610 0x7f9e380cc820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:20.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.260+0000 7f9e3eabd700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9e380ca950 0x7f9e3813bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:20.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.260+0000 7f9e3eabd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e3815b8e0 0x7f9e38138620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:20.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.260+0000 7f9e3eabd700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3813c1f0 con 0x7f9e3815b8e0 2023-10-11T09:48:20.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.260+0000 7f9e3eabd700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e380cceb0 con 0x7f9e3806d610 2023-10-11T09:48:20.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.260+0000 7f9e3eabd700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e380cd190 con 0x7f9e380ca950 2023-10-11T09:48:20.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.260+0000 7f9e3e2bc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e3815b8e0 0x7f9e38138620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:20.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.260+0000 7f9e3e2bc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e3815b8e0 0x7f9e38138620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33876/0 (socket says 172.21.15.78:33876) 2023-10-11T09:48:20.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.260+0000 7f9e3e2bc700 1 -- 172.21.15.78:0/3238620952 learned_addr learned my addr 172.21.15.78:0/3238620952 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:20.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.260+0000 7f9e3dabb700 1 --2- 172.21.15.78:0/3238620952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e3806d610 0x7f9e380cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:20.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.260+0000 7f9e3d2ba700 1 --2- 172.21.15.78:0/3238620952 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9e380ca950 0x7f9e3813bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:20.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.264+0000 7f9e3e2bc700 1 -- 172.21.15.78:0/3238620952 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9e380ca950 msgr2=0x7f9e3813bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:20.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.264+0000 7f9e3e2bc700 1 --2- 172.21.15.78:0/3238620952 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9e380ca950 0x7f9e3813bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:20.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.264+0000 7f9e3e2bc700 1 -- 172.21.15.78:0/3238620952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e3806d610 msgr2=0x7f9e380cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:20.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.264+0000 7f9e3e2bc700 1 --2- 172.21.15.78:0/3238620952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e3806d610 0x7f9e380cc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:20.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.264+0000 7f9e3e2bc700 1 -- 172.21.15.78:0/3238620952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e34009570 con 0x7f9e3815b8e0 2023-10-11T09:48:20.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.264+0000 7f9e3e2bc700 1 --2- 172.21.15.78:0/3238620952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e3815b8e0 0x7f9e38138620 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000efc0 tx=0x7f9e3000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:20.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.264+0000 7f9e2e7fc700 1 -- 172.21.15.78:0/3238620952 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3000cc90 con 0x7f9e3815b8e0 2023-10-11T09:48:20.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.264+0000 7f9e2e7fc700 1 -- 172.21.15.78:0/3238620952 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3000f040 con 0x7f9e3815b8e0 2023-10-11T09:48:20.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.264+0000 7f9e2e7fc700 1 -- 172.21.15.78:0/3238620952 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e300146c0 con 0x7f9e3815b8e0 2023-10-11T09:48:20.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.264+0000 7f9e3eabd700 1 -- 172.21.15.78:0/3238620952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e380cd470 con 0x7f9e3815b8e0 2023-10-11T09:48:20.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.264+0000 7f9e3eabd700 1 -- 172.21.15.78:0/3238620952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e380cda30 con 0x7f9e3815b8e0 2023-10-11T09:48:20.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.264+0000 7f9e3eabd700 1 -- 172.21.15.78:0/3238620952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e38065d40 con 0x7f9e3815b8e0 2023-10-11T09:48:20.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.268+0000 7f9e2e7fc700 1 -- 172.21.15.78:0/3238620952 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f9e30014820 con 0x7f9e3815b8e0 2023-10-11T09:48:20.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.268+0000 7f9e2e7fc700 1 --2- 172.21.15.78:0/3238620952 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9e1c065ed0 0x7f9e1c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:20.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.268+0000 7f9e2e7fc700 1 -- 172.21.15.78:0/3238620952 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9e300843d0 con 0x7f9e3815b8e0 2023-10-11T09:48:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.268+0000 7f9e3dabb700 1 --2- 172.21.15.78:0/3238620952 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9e1c065ed0 0x7f9e1c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.268+0000 7f9e3dabb700 1 --2- 172.21.15.78:0/3238620952 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9e1c065ed0 0x7f9e1c068380 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9e34000c00 tx=0x7f9e34008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:20.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.276+0000 7f9e2e7fc700 1 -- 172.21.15.78:0/3238620952 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f9e30051980 con 0x7f9e3815b8e0 2023-10-11T09:48:20.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.452+0000 7f9e3eabd700 1 -- 172.21.15.78:0/3238620952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e3815cdd0 con 0x7f9e3815b8e0 2023-10-11T09:48:20.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.456+0000 7f9e2e7fc700 1 -- 172.21.15.78:0/3238620952 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+17899 (secure 0 0 0) 0x7f9e30009ca0 con 0x7f9e3815b8e0 2023-10-11T09:48:20.466 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:20.466 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":82,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:48:20.021500+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:48:18.491556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-11T09:48:19.018937+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:48:20.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.460+0000 7f9e3eabd700 1 -- 172.21.15.78:0/3238620952 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9e1c065ed0 msgr2=0x7f9e1c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:20.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.460+0000 7f9e3eabd700 1 --2- 172.21.15.78:0/3238620952 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9e1c065ed0 0x7f9e1c068380 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9e34000c00 tx=0x7f9e34008030 comp rx=0 tx=0).stop 2023-10-11T09:48:20.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.460+0000 7f9e3eabd700 1 -- 172.21.15.78:0/3238620952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e3815b8e0 msgr2=0x7f9e38138620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:20.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.460+0000 7f9e3eabd700 1 --2- 172.21.15.78:0/3238620952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e3815b8e0 0x7f9e38138620 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000efc0 tx=0x7f9e3000c3a0 comp rx=0 tx=0).stop 2023-10-11T09:48:20.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.460+0000 7f9e3eabd700 1 -- 172.21.15.78:0/3238620952 shutdown_connections 2023-10-11T09:48:20.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.460+0000 7f9e3eabd700 1 --2- 172.21.15.78:0/3238620952 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9e1c065ed0 0x7f9e1c068380 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:20.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.460+0000 7f9e3eabd700 1 --2- 172.21.15.78:0/3238620952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e3815b8e0 0x7f9e38138620 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:20.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.460+0000 7f9e3eabd700 1 --2- 172.21.15.78:0/3238620952 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9e380ca950 0x7f9e3813bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:20.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.460+0000 7f9e3eabd700 1 --2- 172.21.15.78:0/3238620952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e3806d610 0x7f9e380cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:20.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.460+0000 7f9e3eabd700 1 -- 172.21.15.78:0/3238620952 >> 172.21.15.78:0/3238620952 conn(0x7f9e380cea60 msgr2=0x7f9e380cb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:20.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.460+0000 7f9e3eabd700 1 -- 172.21.15.78:0/3238620952 shutdown_connections 2023-10-11T09:48:20.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.460+0000 7f9e3eabd700 1 -- 172.21.15.78:0/3238620952 wait complete. 2023-10-11T09:48:20.490 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:48:20.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.652+0000 7f0caef69700 1 Processor -- start 2023-10-11T09:48:20.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.652+0000 7f0caef69700 1 -- start start 2023-10-11T09:48:20.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.656+0000 7f0caef69700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ca8160d40 0x7f0ca80cd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:20.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.656+0000 7f0caef69700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0ca8071040 0x7f0ca80cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:20.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.656+0000 7f0caef69700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ca806ffa0 con 0x7f0ca80cd7c0 2023-10-11T09:48:20.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.656+0000 7f0caef69700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ca8070c20 con 0x7f0ca8071040 2023-10-11T09:48:20.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.656+0000 7f0caef69700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca8161110 con 0x7f0ca8160d40 2023-10-11T09:48:20.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.656+0000 7f0ca7fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0ca8071040 0x7f0ca80cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:20.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.656+0000 7f0ca7fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0ca8071040 0x7f0ca80cddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33884/0 (socket says 172.21.15.78:33884) 2023-10-11T09:48:20.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.656+0000 7f0ca7fff700 1 -- 172.21.15.78:0/2276336501 learned_addr learned my addr 172.21.15.78:0/2276336501 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:20.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.656+0000 7f0cacd05700 1 --2- 172.21.15.78:0/2276336501 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ca8160d40 0x7f0ca80cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:20.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.656+0000 7f0ca7fff700 1 -- 172.21.15.78:0/2276336501 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ca8160d40 msgr2=0x7f0ca80cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:20.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.656+0000 7f0ca7fff700 1 --2- 172.21.15.78:0/2276336501 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ca8160d40 0x7f0ca80cd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:20.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.656+0000 7f0ca7fff700 1 -- 172.21.15.78:0/2276336501 >> v1:172.21.15.133:6789/0 conn(0x7f0ca80cd7c0 legacy=0x7f0ca81647f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:20.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.656+0000 7f0ca7fff700 1 -- 172.21.15.78:0/2276336501 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca8070280 con 0x7f0ca8071040 2023-10-11T09:48:20.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.656+0000 7f0cacd05700 1 --2- 172.21.15.78:0/2276336501 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ca8160d40 0x7f0ca80cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:48:20.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.656+0000 7f0ca7fff700 1 --2- 172.21.15.78:0/2276336501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ca8071040 0x7f0ca80cddf0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f0c94009910 tx=0x7f0c94009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2edc027c210763b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:20.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.656+0000 7f0ca77fe700 1 -- 172.21.15.78:0/2276336501 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9400ba60 con 0x7f0ca8071040 2023-10-11T09:48:20.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.656+0000 7f0ca77fe700 1 -- 172.21.15.78:0/2276336501 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c9400bbc0 con 0x7f0ca8071040 2023-10-11T09:48:20.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.656+0000 7f0ca77fe700 1 -- 172.21.15.78:0/2276336501 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c94009e60 con 0x7f0ca8071040 2023-10-11T09:48:20.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.660+0000 7f0caef69700 1 -- 172.21.15.78:0/2276336501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ca8071040 msgr2=0x7f0ca80cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:20.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.660+0000 7f0caef69700 1 --2- 172.21.15.78:0/2276336501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ca8071040 0x7f0ca80cddf0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f0c94009910 tx=0x7f0c94009c20 comp rx=0 tx=0).stop 2023-10-11T09:48:20.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.660+0000 7f0caef69700 1 -- 172.21.15.78:0/2276336501 shutdown_connections 2023-10-11T09:48:20.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.660+0000 7f0caef69700 1 --2- 172.21.15.78:0/2276336501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ca8071040 0x7f0ca80cddf0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:20.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.660+0000 7f0caef69700 1 --2- 172.21.15.78:0/2276336501 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ca8160d40 0x7f0ca80cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:20.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.660+0000 7f0caef69700 1 -- 172.21.15.78:0/2276336501 >> 172.21.15.78:0/2276336501 conn(0x7f0ca806aca0 msgr2=0x7f0ca80b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:20.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.660+0000 7f0caef69700 1 -- 172.21.15.78:0/2276336501 shutdown_connections 2023-10-11T09:48:20.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.660+0000 7f0caef69700 1 -- 172.21.15.78:0/2276336501 wait complete. 2023-10-11T09:48:20.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.660+0000 7f0caef69700 1 Processor -- start 2023-10-11T09:48:20.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.660+0000 7f0caef69700 1 -- start start 2023-10-11T09:48:20.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.664+0000 7f0caef69700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ca8071040 0x7f0ca80c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:20.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.664+0000 7f0caef69700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ca80cd7c0 0x7f0ca8152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:20.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.664+0000 7f0caef69700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ca8160d40 0x7f0ca81625b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:20.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.664+0000 7f0caef69700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca80c9120 con 0x7f0ca80cd7c0 2023-10-11T09:48:20.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.664+0000 7f0caef69700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca8153520 con 0x7f0ca8071040 2023-10-11T09:48:20.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.664+0000 7f0caef69700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca81facd0 con 0x7f0ca8160d40 2023-10-11T09:48:20.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.664+0000 7f0ca7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ca80cd7c0 0x7f0ca8152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:20.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.664+0000 7f0ca7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ca80cd7c0 0x7f0ca8152fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33900/0 (socket says 172.21.15.78:33900) 2023-10-11T09:48:20.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.664+0000 7f0ca7fff700 1 -- 172.21.15.78:0/1468760846 learned_addr learned my addr 172.21.15.78:0/1468760846 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:20.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.664+0000 7f0cacd05700 1 --2- 172.21.15.78:0/1468760846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ca8071040 0x7f0ca80c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:20.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.664+0000 7f0cad506700 1 --2- 172.21.15.78:0/1468760846 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ca8160d40 0x7f0ca81625b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:20.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.664+0000 7f0ca7fff700 1 -- 172.21.15.78:0/1468760846 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ca8160d40 msgr2=0x7f0ca81625b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:20.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.664+0000 7f0ca7fff700 1 --2- 172.21.15.78:0/1468760846 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ca8160d40 0x7f0ca81625b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:20.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.664+0000 7f0ca7fff700 1 -- 172.21.15.78:0/1468760846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ca8071040 msgr2=0x7f0ca80c8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:20.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.664+0000 7f0ca7fff700 1 --2- 172.21.15.78:0/1468760846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ca8071040 0x7f0ca80c8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:20.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.664+0000 7f0ca7fff700 1 -- 172.21.15.78:0/1468760846 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c94009570 con 0x7f0ca80cd7c0 2023-10-11T09:48:20.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.664+0000 7f0cad506700 1 --2- 172.21.15.78:0/1468760846 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ca8160d40 0x7f0ca81625b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:48:20.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.664+0000 7f0ca7fff700 1 --2- 172.21.15.78:0/1468760846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ca80cd7c0 0x7f0ca8152fe0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f0c94000c00 tx=0x7f0c94004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:20.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.664+0000 7f0ca57fa700 1 -- 172.21.15.78:0/1468760846 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c94004510 con 0x7f0ca80cd7c0 2023-10-11T09:48:20.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.664+0000 7f0ca57fa700 1 -- 172.21.15.78:0/1468760846 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c9401e070 con 0x7f0ca80cd7c0 2023-10-11T09:48:20.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.664+0000 7f0ca57fa700 1 -- 172.21.15.78:0/1468760846 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c94003900 con 0x7f0ca80cd7c0 2023-10-11T09:48:20.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.664+0000 7f0caef69700 1 -- 172.21.15.78:0/1468760846 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ca81fa150 con 0x7f0ca80cd7c0 2023-10-11T09:48:20.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.664+0000 7f0caef69700 1 -- 172.21.15.78:0/1468760846 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ca81fa710 con 0x7f0ca80cd7c0 2023-10-11T09:48:20.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.668+0000 7f0caef69700 1 -- 172.21.15.78:0/1468760846 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ca8072250 con 0x7f0ca80cd7c0 2023-10-11T09:48:20.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.668+0000 7f0ca57fa700 1 -- 172.21.15.78:0/1468760846 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f0c94003a60 con 0x7f0ca80cd7c0 2023-10-11T09:48:20.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.668+0000 7f0ca57fa700 1 --2- 172.21.15.78:0/1468760846 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0c8c065ed0 0x7f0c8c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:20.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.668+0000 7f0ca57fa700 1 -- 172.21.15.78:0/1468760846 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0c94020070 con 0x7f0ca80cd7c0 2023-10-11T09:48:20.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.668+0000 7f0cacd05700 1 --2- 172.21.15.78:0/1468760846 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0c8c065ed0 0x7f0c8c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:20.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.672+0000 7f0cacd05700 1 --2- 172.21.15.78:0/1468760846 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0c8c065ed0 0x7f0c8c068380 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c0099f0 tx=0x7f0c9c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:20.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.676+0000 7f0ca57fa700 1 -- 172.21.15.78:0/1468760846 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f0c94052270 con 0x7f0ca80cd7c0 2023-10-11T09:48:20.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.852+0000 7f0caef69700 1 -- 172.21.15.78:0/1468760846 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ca80c3c70 con 0x7f0ca80cd7c0 2023-10-11T09:48:20.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.856+0000 7f0ca57fa700 1 -- 172.21.15.78:0/1468760846 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+17899 (secure 0 0 0) 0x7f0ca80c3c70 con 0x7f0ca80cd7c0 2023-10-11T09:48:20.866 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:20.866 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":82,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:48:20.021500+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:48:18.491556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-11T09:48:19.018937+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:48:20.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.860+0000 7f0caef69700 1 -- 172.21.15.78:0/1468760846 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0c8c065ed0 msgr2=0x7f0c8c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:20.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.860+0000 7f0caef69700 1 --2- 172.21.15.78:0/1468760846 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0c8c065ed0 0x7f0c8c068380 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c0099f0 tx=0x7f0c9c008040 comp rx=0 tx=0).stop 2023-10-11T09:48:20.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.860+0000 7f0caef69700 1 -- 172.21.15.78:0/1468760846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ca80cd7c0 msgr2=0x7f0ca8152fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:20.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.860+0000 7f0caef69700 1 --2- 172.21.15.78:0/1468760846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ca80cd7c0 0x7f0ca8152fe0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f0c94000c00 tx=0x7f0c94004770 comp rx=0 tx=0).stop 2023-10-11T09:48:20.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.860+0000 7f0caef69700 1 -- 172.21.15.78:0/1468760846 shutdown_connections 2023-10-11T09:48:20.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.860+0000 7f0caef69700 1 --2- 172.21.15.78:0/1468760846 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0c8c065ed0 0x7f0c8c068380 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:20.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.860+0000 7f0caef69700 1 --2- 172.21.15.78:0/1468760846 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ca8160d40 0x7f0ca81625b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:20.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.860+0000 7f0caef69700 1 --2- 172.21.15.78:0/1468760846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ca80cd7c0 0x7f0ca8152fe0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:20.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.860+0000 7f0caef69700 1 --2- 172.21.15.78:0/1468760846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ca8071040 0x7f0ca80c8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:20.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.860+0000 7f0caef69700 1 -- 172.21.15.78:0/1468760846 >> 172.21.15.78:0/1468760846 conn(0x7f0ca806aca0 msgr2=0x7f0ca80cec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:20.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.860+0000 7f0caef69700 1 -- 172.21.15.78:0/1468760846 shutdown_connections 2023-10-11T09:48:20.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:20.860+0000 7f0caef69700 1 -- 172.21.15.78:0/1468760846 wait complete. 2023-10-11T09:48:20.891 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-11T09:48:21.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.052+0000 7f62dd26a700 1 Processor -- start 2023-10-11T09:48:21.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.052+0000 7f62dd26a700 1 -- start start 2023-10-11T09:48:21.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.052+0000 7f62dd26a700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f62d80b3610 0x7f62d80d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:21.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.052+0000 7f62dd26a700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62d80b3a90 0x7f62d80b85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:21.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.052+0000 7f62dd26a700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62d80b47f0 con 0x7f62d80b3240 2023-10-11T09:48:21.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.052+0000 7f62dd26a700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62d80b8130 con 0x7f62d80b3610 2023-10-11T09:48:21.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.052+0000 7f62dd26a700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d80b3f30 con 0x7f62d80b3a90 2023-10-11T09:48:21.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.052+0000 7f62d67fc700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f62d80b3610 0x7f62d80d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:21.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.052+0000 7f62d67fc700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f62d80b3610 0x7f62d80d29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33902/0 (socket says 172.21.15.78:33902) 2023-10-11T09:48:21.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.052+0000 7f62d67fc700 1 -- 172.21.15.78:0/1842360340 learned_addr learned my addr 172.21.15.78:0/1842360340 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:21.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.052+0000 7f62d77fe700 1 --2- 172.21.15.78:0/1842360340 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62d80b3a90 0x7f62d80b85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:21.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.052+0000 7f62d67fc700 1 -- 172.21.15.78:0/1842360340 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62d80b3a90 msgr2=0x7f62d80b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:21.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.052+0000 7f62d67fc700 1 --2- 172.21.15.78:0/1842360340 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62d80b3a90 0x7f62d80b85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:21.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.052+0000 7f62d67fc700 1 -- 172.21.15.78:0/1842360340 >> v1:172.21.15.133:6789/0 conn(0x7f62d80b3240 legacy=0x7f62d815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:21.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.052+0000 7f62d67fc700 1 -- 172.21.15.78:0/1842360340 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d80b41e0 con 0x7f62d80b3610 2023-10-11T09:48:21.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.056+0000 7f62d67fc700 1 --2- 172.21.15.78:0/1842360340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62d80b3610 0x7f62d80d29d0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009910 tx=0x7f62cc009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c75edc4d2c451db server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:21.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.056+0000 7f62d5ffb700 1 -- 172.21.15.78:0/1842360340 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc00ba60 con 0x7f62d80b3610 2023-10-11T09:48:21.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.056+0000 7f62d5ffb700 1 -- 172.21.15.78:0/1842360340 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62cc00bbc0 con 0x7f62d80b3610 2023-10-11T09:48:21.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.056+0000 7f62d5ffb700 1 -- 172.21.15.78:0/1842360340 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc009e60 con 0x7f62d80b3610 2023-10-11T09:48:21.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.056+0000 7f62dd26a700 1 -- 172.21.15.78:0/1842360340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62d80b3610 msgr2=0x7f62d80d29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:21.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.056+0000 7f62dd26a700 1 --2- 172.21.15.78:0/1842360340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62d80b3610 0x7f62d80d29d0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009910 tx=0x7f62cc009c20 comp rx=0 tx=0).stop 2023-10-11T09:48:21.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.056+0000 7f62dd26a700 1 -- 172.21.15.78:0/1842360340 shutdown_connections 2023-10-11T09:48:21.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.056+0000 7f62dd26a700 1 --2- 172.21.15.78:0/1842360340 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62d80b3a90 0x7f62d80b85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:21.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.056+0000 7f62dd26a700 1 --2- 172.21.15.78:0/1842360340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62d80b3610 0x7f62d80d29d0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:21.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.056+0000 7f62dd26a700 1 -- 172.21.15.78:0/1842360340 >> 172.21.15.78:0/1842360340 conn(0x7f62d809c260 msgr2=0x7f62d80d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:21.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.056+0000 7f62dd26a700 1 -- 172.21.15.78:0/1842360340 shutdown_connections 2023-10-11T09:48:21.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.056+0000 7f62dd26a700 1 -- 172.21.15.78:0/1842360340 wait complete. 2023-10-11T09:48:21.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.060+0000 7f62dd26a700 1 Processor -- start 2023-10-11T09:48:21.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.060+0000 7f62dd26a700 1 -- start start 2023-10-11T09:48:21.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.060+0000 7f62dd26a700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62d80b3240 0x7f62d80a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:21.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.060+0000 7f62dd26a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62d80b3610 0x7f62d815e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:21.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.060+0000 7f62dd26a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62d80b3a90 0x7f62d81e65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:21.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.060+0000 7f62dd26a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d815bd60 con 0x7f62d80b3a90 2023-10-11T09:48:21.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.060+0000 7f62dd26a700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d80c7050 con 0x7f62d80b3610 2023-10-11T09:48:21.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.060+0000 7f62dd26a700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d80c90e0 con 0x7f62d80b3240 2023-10-11T09:48:21.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.060+0000 7f62d77fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62d80b3a90 0x7f62d81e65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:21.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.060+0000 7f62d6ffd700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62d80b3240 0x7f62d80a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:21.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.060+0000 7f62d6ffd700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62d80b3240 0x7f62d80a0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:37020/0 (socket says 172.21.15.78:37020) 2023-10-11T09:48:21.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.060+0000 7f62d6ffd700 1 -- 172.21.15.78:0/33959684 learned_addr learned my addr 172.21.15.78:0/33959684 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:21.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.060+0000 7f62d67fc700 1 --2- 172.21.15.78:0/33959684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62d80b3610 0x7f62d815e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:21.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.060+0000 7f62d77fe700 1 -- 172.21.15.78:0/33959684 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62d80b3240 msgr2=0x7f62d80a0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:21.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.060+0000 7f62d77fe700 1 --2- 172.21.15.78:0/33959684 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62d80b3240 0x7f62d80a0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:21.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.060+0000 7f62d77fe700 1 -- 172.21.15.78:0/33959684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62d80b3610 msgr2=0x7f62d815e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:21.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.060+0000 7f62d77fe700 1 --2- 172.21.15.78:0/33959684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62d80b3610 0x7f62d815e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:21.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.060+0000 7f62d77fe700 1 -- 172.21.15.78:0/33959684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62cc009570 con 0x7f62d80b3a90 2023-10-11T09:48:21.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.060+0000 7f62d77fe700 1 --2- 172.21.15.78:0/33959684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62d80b3a90 0x7f62d81e65d0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f62d0007dc0 tx=0x7f62d000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:21.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.064+0000 7f62bf7fe700 1 -- 172.21.15.78:0/33959684 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d0018070 con 0x7f62d80b3a90 2023-10-11T09:48:21.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.064+0000 7f62bf7fe700 1 -- 172.21.15.78:0/33959684 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d000eb30 con 0x7f62d80b3a90 2023-10-11T09:48:21.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.064+0000 7f62dd26a700 1 -- 172.21.15.78:0/33959684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d80c9380 con 0x7f62d80b3a90 2023-10-11T09:48:21.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.064+0000 7f62bf7fe700 1 -- 172.21.15.78:0/33959684 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d000eec0 con 0x7f62d80b3a90 2023-10-11T09:48:21.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.064+0000 7f62dd26a700 1 -- 172.21.15.78:0/33959684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d80c9940 con 0x7f62d80b3a90 2023-10-11T09:48:21.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.064+0000 7f62dd26a700 1 -- 172.21.15.78:0/33959684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d80c6ea0 con 0x7f62d80b3a90 2023-10-11T09:48:21.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.064+0000 7f62bf7fe700 1 -- 172.21.15.78:0/33959684 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f62d0003a40 con 0x7f62d80b3a90 2023-10-11T09:48:21.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.068+0000 7f62bf7fe700 1 --2- 172.21.15.78:0/33959684 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f62c4065ed0 0x7f62c4068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:21.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.068+0000 7f62bf7fe700 1 -- 172.21.15.78:0/33959684 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f62d0083760 con 0x7f62d80b3a90 2023-10-11T09:48:21.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.068+0000 7f62d6ffd700 1 --2- 172.21.15.78:0/33959684 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f62c4065ed0 0x7f62c4068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:21.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.068+0000 7f62d6ffd700 1 --2- 172.21.15.78:0/33959684 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f62c4065ed0 0x7f62c4068380 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f62c80018a0 tx=0x7f62c8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:21.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.072+0000 7f62bf7fe700 1 -- 172.21.15.78:0/33959684 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f62d0050da0 con 0x7f62d80b3a90 2023-10-11T09:48:21.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:21.284+0000 7f62dd26a700 1 -- 172.21.15.78:0/33959684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f62d8065d40 con 0x7f62d80b3a90 2023-10-11T09:48:22.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.044+0000 7f62bf7fe700 1 -- 172.21.15.78:0/33959684 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v57) v1 ==== 118+0+0 (secure 0 0 0) 0x7f62d0009950 con 0x7f62d80b3a90 2023-10-11T09:48:22.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.048+0000 7f62dd26a700 1 -- 172.21.15.78:0/33959684 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f62c4065ed0 msgr2=0x7f62c4068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:22.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.048+0000 7f62dd26a700 1 --2- 172.21.15.78:0/33959684 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f62c4065ed0 0x7f62c4068380 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f62c80018a0 tx=0x7f62c8009040 comp rx=0 tx=0).stop 2023-10-11T09:48:22.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.048+0000 7f62dd26a700 1 -- 172.21.15.78:0/33959684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62d80b3a90 msgr2=0x7f62d81e65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:22.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.048+0000 7f62dd26a700 1 --2- 172.21.15.78:0/33959684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62d80b3a90 0x7f62d81e65d0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f62d0007dc0 tx=0x7f62d000e3c0 comp rx=0 tx=0).stop 2023-10-11T09:48:22.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.048+0000 7f62dd26a700 1 -- 172.21.15.78:0/33959684 shutdown_connections 2023-10-11T09:48:22.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.048+0000 7f62dd26a700 1 --2- 172.21.15.78:0/33959684 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f62c4065ed0 0x7f62c4068380 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:22.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.048+0000 7f62dd26a700 1 --2- 172.21.15.78:0/33959684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62d80b3a90 0x7f62d81e65d0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:22.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.048+0000 7f62dd26a700 1 --2- 172.21.15.78:0/33959684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62d80b3610 0x7f62d815e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:22.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.048+0000 7f62dd26a700 1 --2- 172.21.15.78:0/33959684 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62d80b3240 0x7f62d80a0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:22.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.048+0000 7f62dd26a700 1 -- 172.21.15.78:0/33959684 >> 172.21.15.78:0/33959684 conn(0x7f62d809c260 msgr2=0x7f62d815c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:22.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.048+0000 7f62dd26a700 1 -- 172.21.15.78:0/33959684 shutdown_connections 2023-10-11T09:48:22.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.048+0000 7f62dd26a700 1 -- 172.21.15.78:0/33959684 wait complete. 2023-10-11T09:48:22.078 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-11T09:48:22.078 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-11T09:48:22.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.248+0000 7f3430b45700 1 Processor -- start 2023-10-11T09:48:22.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.248+0000 7f3430b45700 1 -- start start 2023-10-11T09:48:22.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.248+0000 7f3430b45700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f342c0b3240 0x7f342c0d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:22.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.248+0000 7f3430b45700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f342c0b3a90 0x7f342c0b85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:22.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.248+0000 7f3430b45700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f342c0b47f0 con 0x7f342c0b3610 2023-10-11T09:48:22.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.248+0000 7f3430b45700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f342c0b8130 con 0x7f342c0b3240 2023-10-11T09:48:22.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.248+0000 7f3430b45700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f342c0b3f30 con 0x7f342c0b3a90 2023-10-11T09:48:22.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.248+0000 7f342b7fe700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f342c0b3240 0x7f342c0d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:22.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.248+0000 7f342bfff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f342c0b3a90 0x7f342c0b85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:22.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.248+0000 7f342b7fe700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f342c0b3240 0x7f342c0d29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:54908/0 (socket says 172.21.15.78:54908) 2023-10-11T09:48:22.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.248+0000 7f342b7fe700 1 -- 172.21.15.78:0/3219720605 learned_addr learned my addr 172.21.15.78:0/3219720605 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:22.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.252+0000 7f342b7fe700 1 -- 172.21.15.78:0/3219720605 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f342c0b3a90 msgr2=0x7f342c0b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:22.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.252+0000 7f342b7fe700 1 --2- 172.21.15.78:0/3219720605 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f342c0b3a90 0x7f342c0b85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:22.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.252+0000 7f342b7fe700 1 -- 172.21.15.78:0/3219720605 >> v1:172.21.15.133:6789/0 conn(0x7f342c0b3610 legacy=0x7f342c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:22.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.252+0000 7f342b7fe700 1 -- 172.21.15.78:0/3219720605 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f342c0b41e0 con 0x7f342c0b3240 2023-10-11T09:48:22.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.252+0000 7f342a7fc700 1 -- 172.21.15.78:0/3219720605 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2392766471 0 0) 0x7f3420001110 con 0x7f342c0b3610 2023-10-11T09:48:22.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.252+0000 7f342bfff700 1 --2- 172.21.15.78:0/3219720605 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f342c0b3a90 0x7f342c0b85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:48:22.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.252+0000 7f342b7fe700 1 --2- 172.21.15.78:0/3219720605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f342c0b3240 0x7f342c0d29d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f341c009910 tx=0x7f341c009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31171a488321681c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:22.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.252+0000 7f342a7fc700 1 -- 172.21.15.78:0/3219720605 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c011830 con 0x7f342c0b3240 2023-10-11T09:48:22.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.252+0000 7f342a7fc700 1 -- 172.21.15.78:0/3219720605 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f341c011990 con 0x7f342c0b3240 2023-10-11T09:48:22.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.252+0000 7f342a7fc700 1 -- 172.21.15.78:0/3219720605 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c011d40 con 0x7f342c0b3240 2023-10-11T09:48:22.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.252+0000 7f3430b45700 1 -- 172.21.15.78:0/3219720605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f342c0b3240 msgr2=0x7f342c0d29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:22.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.252+0000 7f3430b45700 1 --2- 172.21.15.78:0/3219720605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f342c0b3240 0x7f342c0d29d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f341c009910 tx=0x7f341c009c20 comp rx=0 tx=0).stop 2023-10-11T09:48:22.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.252+0000 7f3430b45700 1 -- 172.21.15.78:0/3219720605 shutdown_connections 2023-10-11T09:48:22.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.252+0000 7f3430b45700 1 --2- 172.21.15.78:0/3219720605 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f342c0b3a90 0x7f342c0b85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:22.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.252+0000 7f3430b45700 1 --2- 172.21.15.78:0/3219720605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f342c0b3240 0x7f342c0d29d0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:22.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.252+0000 7f3430b45700 1 -- 172.21.15.78:0/3219720605 >> 172.21.15.78:0/3219720605 conn(0x7f342c09c260 msgr2=0x7f342c0d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:22.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.256+0000 7f3430b45700 1 -- 172.21.15.78:0/3219720605 shutdown_connections 2023-10-11T09:48:22.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.256+0000 7f3430b45700 1 -- 172.21.15.78:0/3219720605 wait complete. 2023-10-11T09:48:22.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.256+0000 7f3430b45700 1 Processor -- start 2023-10-11T09:48:22.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.260+0000 7f3430b45700 1 -- start start 2023-10-11T09:48:22.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.260+0000 7f3430b45700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f342c0b3240 0x7f342c0d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:22.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.260+0000 7f3430b45700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f342c0b3610 0x7f342c0c7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:22.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.260+0000 7f3430b45700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f342c0b3a90 0x7f342c0c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:22.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.260+0000 7f3430b45700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c15cea0 con 0x7f342c0b3a90 2023-10-11T09:48:22.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.260+0000 7f3430b45700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c15d140 con 0x7f342c0b3610 2023-10-11T09:48:22.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.260+0000 7f3430b45700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f342c15d420 con 0x7f342c0b3240 2023-10-11T09:48:22.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.260+0000 7f342bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f342c0b3a90 0x7f342c0c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:22.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.260+0000 7f342bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f342c0b3a90 0x7f342c0c7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33916/0 (socket says 172.21.15.78:33916) 2023-10-11T09:48:22.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.260+0000 7f342bfff700 1 -- 172.21.15.78:0/3929412020 learned_addr learned my addr 172.21.15.78:0/3929412020 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:22.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.260+0000 7f342b7fe700 1 --2- 172.21.15.78:0/3929412020 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f342c0b3240 0x7f342c0d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:22.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.260+0000 7f342affd700 1 --2- 172.21.15.78:0/3929412020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f342c0b3610 0x7f342c0c7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:22.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.260+0000 7f342bfff700 1 -- 172.21.15.78:0/3929412020 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f342c0b3240 msgr2=0x7f342c0d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:22.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.260+0000 7f342bfff700 1 --2- 172.21.15.78:0/3929412020 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f342c0b3240 0x7f342c0d31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:22.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.260+0000 7f342bfff700 1 -- 172.21.15.78:0/3929412020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f342c0b3610 msgr2=0x7f342c0c7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:22.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.260+0000 7f342bfff700 1 --2- 172.21.15.78:0/3929412020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f342c0b3610 0x7f342c0c7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:22.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.260+0000 7f342bfff700 1 -- 172.21.15.78:0/3929412020 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3424009a70 con 0x7f342c0b3a90 2023-10-11T09:48:22.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.260+0000 7f342bfff700 1 --2- 172.21.15.78:0/3929412020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f342c0b3a90 0x7f342c0c7f90 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f342400efc0 tx=0x7f342400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:22.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.260+0000 7f3413fff700 1 -- 172.21.15.78:0/3929412020 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3424009d70 con 0x7f342c0b3a90 2023-10-11T09:48:22.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.260+0000 7f3413fff700 1 -- 172.21.15.78:0/3929412020 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f342400f040 con 0x7f342c0b3a90 2023-10-11T09:48:22.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.260+0000 7f3430b45700 1 -- 172.21.15.78:0/3929412020 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f341c009570 con 0x7f342c0b3a90 2023-10-11T09:48:22.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.260+0000 7f3413fff700 1 -- 172.21.15.78:0/3929412020 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342400ce00 con 0x7f342c0b3a90 2023-10-11T09:48:22.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.260+0000 7f3430b45700 1 -- 172.21.15.78:0/3929412020 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f342c15d9e0 con 0x7f342c0b3a90 2023-10-11T09:48:22.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.264+0000 7f3430b45700 1 -- 172.21.15.78:0/3929412020 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f342c065d40 con 0x7f342c0b3a90 2023-10-11T09:48:22.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.264+0000 7f3413fff700 1 -- 172.21.15.78:0/3929412020 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f3424014480 con 0x7f342c0b3a90 2023-10-11T09:48:22.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.268+0000 7f3413fff700 1 --2- 172.21.15.78:0/3929412020 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3418065ed0 0x7f3418068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:22.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.268+0000 7f3413fff700 1 -- 172.21.15.78:0/3929412020 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3424083ff0 con 0x7f342c0b3a90 2023-10-11T09:48:22.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.268+0000 7f342b7fe700 1 --2- 172.21.15.78:0/3929412020 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3418065ed0 0x7f3418068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:22.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.268+0000 7f342b7fe700 1 --2- 172.21.15.78:0/3929412020 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3418065ed0 0x7f3418068380 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f341c014040 tx=0x7f341c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:22.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.276+0000 7f3413fff700 1 -- 172.21.15.78:0/3929412020 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f34240b4fa0 con 0x7f342c0b3a90 2023-10-11T09:48:22.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:22.488+0000 7f3430b45700 1 -- 172.21.15.78:0/3929412020 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f342c0b77d0 con 0x7f342c0b3a90 2023-10-11T09:48:23.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.052+0000 7f3413fff700 1 -- 172.21.15.78:0/3929412020 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v58) v1 ==== 122+0+0 (secure 0 0 0) 0x7f34240515c0 con 0x7f342c0b3a90 2023-10-11T09:48:23.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.056+0000 7f3430b45700 1 -- 172.21.15.78:0/3929412020 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3418065ed0 msgr2=0x7f3418068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:23.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.056+0000 7f3430b45700 1 --2- 172.21.15.78:0/3929412020 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3418065ed0 0x7f3418068380 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f341c014040 tx=0x7f341c008030 comp rx=0 tx=0).stop 2023-10-11T09:48:23.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.056+0000 7f3430b45700 1 -- 172.21.15.78:0/3929412020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f342c0b3a90 msgr2=0x7f342c0c7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:23.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.056+0000 7f3430b45700 1 --2- 172.21.15.78:0/3929412020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f342c0b3a90 0x7f342c0c7f90 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f342400efc0 tx=0x7f342400c3a0 comp rx=0 tx=0).stop 2023-10-11T09:48:23.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.056+0000 7f3430b45700 1 -- 172.21.15.78:0/3929412020 shutdown_connections 2023-10-11T09:48:23.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.056+0000 7f3430b45700 1 --2- 172.21.15.78:0/3929412020 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3418065ed0 0x7f3418068380 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:23.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.056+0000 7f3430b45700 1 --2- 172.21.15.78:0/3929412020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f342c0b3a90 0x7f342c0c7f90 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:23.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.056+0000 7f3430b45700 1 --2- 172.21.15.78:0/3929412020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f342c0b3610 0x7f342c0c7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:23.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.056+0000 7f3430b45700 1 --2- 172.21.15.78:0/3929412020 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f342c0b3240 0x7f342c0d31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:23.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.056+0000 7f3430b45700 1 -- 172.21.15.78:0/3929412020 >> 172.21.15.78:0/3929412020 conn(0x7f342c09c260 msgr2=0x7f342c15c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:23.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.060+0000 7f3430b45700 1 -- 172.21.15.78:0/3929412020 shutdown_connections 2023-10-11T09:48:23.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.060+0000 7f3430b45700 1 -- 172.21.15.78:0/3929412020 wait complete. 2023-10-11T09:48:23.092 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-11T09:48:23.092 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:48:23.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.256+0000 7f4fd3420700 1 Processor -- start 2023-10-11T09:48:23.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.256+0000 7f4fd3420700 1 -- start start 2023-10-11T09:48:23.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.260+0000 7f4fd3420700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4fcc15b8e0 0x7f4fcc06d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:23.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.260+0000 7f4fd3420700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fcc06d610 0x7f4fcc0ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:23.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.260+0000 7f4fd3420700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fcc15e010 con 0x7f4fcc15a940 2023-10-11T09:48:23.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.260+0000 7f4fd3420700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fcc15e280 con 0x7f4fcc15b8e0 2023-10-11T09:48:23.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.260+0000 7f4fd3420700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc15e560 con 0x7f4fcc06d610 2023-10-11T09:48:23.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.260+0000 7f4fd2c1f700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f4fcc15a940 0x7f4fcc0c9020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:48148/0 (socket says 172.21.15.78:48148) 2023-10-11T09:48:23.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.260+0000 7f4fd2c1f700 1 -- 172.21.15.78:0/116706948 learned_addr learned my addr 172.21.15.78:0/116706948 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:23.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.260+0000 7f4fd1c1d700 1 --2- 172.21.15.78:0/116706948 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fcc06d610 0x7f4fcc0ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:23.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.260+0000 7f4fd241e700 1 --2- 172.21.15.78:0/116706948 >> v2:172.21.15.133:3300/0 conn(0x7f4fcc15b8e0 0x7f4fcc06d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:23.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.260+0000 7f4fd1c1d700 1 -- 172.21.15.78:0/116706948 >> v2:172.21.15.133:3300/0 conn(0x7f4fcc15b8e0 msgr2=0x7f4fcc06d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:23.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.260+0000 7f4fd1c1d700 1 --2- 172.21.15.78:0/116706948 >> v2:172.21.15.133:3300/0 conn(0x7f4fcc15b8e0 0x7f4fcc06d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:23.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.260+0000 7f4fd1c1d700 1 -- 172.21.15.78:0/116706948 >> v1:172.21.15.78:6789/0 conn(0x7f4fcc15a940 legacy=0x7f4fcc0c9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:23.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.260+0000 7f4fd141c700 1 -- 172.21.15.78:0/116706948 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2905327786 0 0) 0x7f4fcc15e010 con 0x7f4fcc15a940 2023-10-11T09:48:23.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.260+0000 7f4fd1c1d700 1 -- 172.21.15.78:0/116706948 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fcc15e840 con 0x7f4fcc06d610 2023-10-11T09:48:23.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.260+0000 7f4fd1c1d700 1 --2- 172.21.15.78:0/116706948 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fcc06d610 0x7f4fcc0ca240 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc009850 tx=0x7f4fbc009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd26aba53706c1ad server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:23.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.260+0000 7f4fd141c700 1 -- 172.21.15.78:0/116706948 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc011830 con 0x7f4fcc06d610 2023-10-11T09:48:23.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.260+0000 7f4fd141c700 1 -- 172.21.15.78:0/116706948 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fbc011990 con 0x7f4fcc06d610 2023-10-11T09:48:23.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.260+0000 7f4fd141c700 1 -- 172.21.15.78:0/116706948 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc011d40 con 0x7f4fcc06d610 2023-10-11T09:48:23.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.264+0000 7f4fd3420700 1 -- 172.21.15.78:0/116706948 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fcc06d610 msgr2=0x7f4fcc0ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:23.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.264+0000 7f4fd3420700 1 --2- 172.21.15.78:0/116706948 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fcc06d610 0x7f4fcc0ca240 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc009850 tx=0x7f4fbc009b60 comp rx=0 tx=0).stop 2023-10-11T09:48:23.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.264+0000 7f4fd3420700 1 -- 172.21.15.78:0/116706948 shutdown_connections 2023-10-11T09:48:23.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.264+0000 7f4fd3420700 1 --2- 172.21.15.78:0/116706948 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fcc06d610 0x7f4fcc0ca240 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:23.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.264+0000 7f4fd3420700 1 --2- 172.21.15.78:0/116706948 >> v2:172.21.15.133:3300/0 conn(0x7f4fcc15b8e0 0x7f4fcc06d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:23.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.264+0000 7f4fd3420700 1 -- 172.21.15.78:0/116706948 >> 172.21.15.78:0/116706948 conn(0x7f4fcc0cea60 msgr2=0x7f4fcc0cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:23.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.264+0000 7f4fd3420700 1 -- 172.21.15.78:0/116706948 shutdown_connections 2023-10-11T09:48:23.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.264+0000 7f4fd3420700 1 -- 172.21.15.78:0/116706948 wait complete. 2023-10-11T09:48:23.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.264+0000 7f4fd3420700 1 Processor -- start 2023-10-11T09:48:23.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.268+0000 7f4fd3420700 1 -- start start 2023-10-11T09:48:23.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.268+0000 7f4fd3420700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fcc06d610 0x7f4fcc1f10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:23.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.268+0000 7f4fd3420700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fcc15a940 0x7f4fcc0c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:23.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.268+0000 7f4fd3420700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fcc15b8e0 0x7f4fcc0c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:23.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.268+0000 7f4fd3420700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc1419c0 con 0x7f4fcc06d610 2023-10-11T09:48:23.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.268+0000 7f4fd3420700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc141de0 con 0x7f4fcc15a940 2023-10-11T09:48:23.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.268+0000 7f4fd3420700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc140db0 con 0x7f4fcc15b8e0 2023-10-11T09:48:23.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.268+0000 7f4fd241e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fcc06d610 0x7f4fcc1f10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:23.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.268+0000 7f4fd241e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fcc06d610 0x7f4fcc1f10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33924/0 (socket says 172.21.15.78:33924) 2023-10-11T09:48:23.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.268+0000 7f4fd241e700 1 -- 172.21.15.78:0/2674671752 learned_addr learned my addr 172.21.15.78:0/2674671752 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:23.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.268+0000 7f4fd2c1f700 1 --2- 172.21.15.78:0/2674671752 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fcc15b8e0 0x7f4fcc0c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:23.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.268+0000 7f4fd1c1d700 1 --2- 172.21.15.78:0/2674671752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fcc15a940 0x7f4fcc0c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:23.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.268+0000 7f4fd241e700 1 -- 172.21.15.78:0/2674671752 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fcc15b8e0 msgr2=0x7f4fcc0c82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:23.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.268+0000 7f4fd241e700 1 --2- 172.21.15.78:0/2674671752 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fcc15b8e0 0x7f4fcc0c82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:23.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.268+0000 7f4fd241e700 1 -- 172.21.15.78:0/2674671752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fcc15a940 msgr2=0x7f4fcc0c7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:23.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.268+0000 7f4fd241e700 1 --2- 172.21.15.78:0/2674671752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fcc15a940 0x7f4fcc0c7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:23.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.268+0000 7f4fd241e700 1 -- 172.21.15.78:0/2674671752 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fbc0094b0 con 0x7f4fcc06d610 2023-10-11T09:48:23.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.268+0000 7f4fd241e700 1 --2- 172.21.15.78:0/2674671752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fcc06d610 0x7f4fcc1f10e0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f4fc800ea30 tx=0x7f4fc800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:23.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.268+0000 7f4fc2ffd700 1 -- 172.21.15.78:0/2674671752 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8009720 con 0x7f4fcc06d610 2023-10-11T09:48:23.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.268+0000 7f4fc2ffd700 1 -- 172.21.15.78:0/2674671752 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc800c910 con 0x7f4fcc06d610 2023-10-11T09:48:23.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.268+0000 7f4fd3420700 1 -- 172.21.15.78:0/2674671752 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fcc141020 con 0x7f4fcc06d610 2023-10-11T09:48:23.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.268+0000 7f4fc2ffd700 1 -- 172.21.15.78:0/2674671752 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc800cc80 con 0x7f4fcc06d610 2023-10-11T09:48:23.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.268+0000 7f4fd3420700 1 -- 172.21.15.78:0/2674671752 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fcc1415e0 con 0x7f4fcc06d610 2023-10-11T09:48:23.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.272+0000 7f4fd3420700 1 -- 172.21.15.78:0/2674671752 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fcc141c30 con 0x7f4fcc06d610 2023-10-11T09:48:23.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.272+0000 7f4fc2ffd700 1 -- 172.21.15.78:0/2674671752 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f4fc8013470 con 0x7f4fcc06d610 2023-10-11T09:48:23.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.272+0000 7f4fc2ffd700 1 --2- 172.21.15.78:0/2674671752 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4fb0065ed0 0x7f4fb0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:23.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.276+0000 7f4fc2ffd700 1 -- 172.21.15.78:0/2674671752 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4fc80839a0 con 0x7f4fcc06d610 2023-10-11T09:48:23.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.276+0000 7f4fd1c1d700 1 --2- 172.21.15.78:0/2674671752 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4fb0065ed0 0x7f4fb0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:23.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.276+0000 7f4fd1c1d700 1 --2- 172.21.15.78:0/2674671752 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4fb0065ed0 0x7f4fb0068380 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc0090b0 tx=0x7f4fbc01c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:23.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.280+0000 7f4fc2ffd700 1 -- 172.21.15.78:0/2674671752 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f4fc8050f70 con 0x7f4fcc06d610 2023-10-11T09:48:23.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.488+0000 7f4fd3420700 1 -- 172.21.15.78:0/2674671752 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4fcc065d40 con 0x7f4fcc06d610 2023-10-11T09:48:23.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.492+0000 7f4fc2ffd700 1 -- 172.21.15.78:0/2674671752 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4650 (secure 0 0 0) 0x7f4fc8054590 con 0x7f4fcc06d610 2023-10-11T09:48:23.500 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:23.500 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":58,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6961,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/1080544187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1080544187},{"type":"v1","addr":"172.21.15.133:6834","nonce":1080544187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/367194191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":367194191},{"type":"v1","addr":"172.21.15.133:6835","nonce":367194191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":7224,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/3463904224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3463904224},{"type":"v1","addr":"172.21.15.78:6836","nonce":3463904224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:48:20.022298+0000","modified":"2023-10-11T09:48:23.045010+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7227},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7227":{"gid":7227,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6837/951312831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":951312831},{"type":"v1","addr":"172.21.15.78:6837","nonce":951312831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-11T09:48:23.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.496+0000 7f4fd3420700 1 -- 172.21.15.78:0/2674671752 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4fb0065ed0 msgr2=0x7f4fb0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:23.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.496+0000 7f4fd3420700 1 --2- 172.21.15.78:0/2674671752 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4fb0065ed0 0x7f4fb0068380 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc0090b0 tx=0x7f4fbc01c040 comp rx=0 tx=0).stop 2023-10-11T09:48:23.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.496+0000 7f4fd3420700 1 -- 172.21.15.78:0/2674671752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fcc06d610 msgr2=0x7f4fcc1f10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:23.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.496+0000 7f4fd3420700 1 --2- 172.21.15.78:0/2674671752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fcc06d610 0x7f4fcc1f10e0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f4fc800ea30 tx=0x7f4fc800edf0 comp rx=0 tx=0).stop 2023-10-11T09:48:23.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.496+0000 7f4fd3420700 1 -- 172.21.15.78:0/2674671752 shutdown_connections 2023-10-11T09:48:23.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.496+0000 7f4fd3420700 1 --2- 172.21.15.78:0/2674671752 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4fb0065ed0 0x7f4fb0068380 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:23.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.496+0000 7f4fd3420700 1 --2- 172.21.15.78:0/2674671752 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fcc15b8e0 0x7f4fcc0c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:23.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.496+0000 7f4fd3420700 1 --2- 172.21.15.78:0/2674671752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fcc15a940 0x7f4fcc0c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:23.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.496+0000 7f4fd3420700 1 --2- 172.21.15.78:0/2674671752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fcc06d610 0x7f4fcc1f10e0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:23.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.496+0000 7f4fd3420700 1 -- 172.21.15.78:0/2674671752 >> 172.21.15.78:0/2674671752 conn(0x7f4fcc0cea60 msgr2=0x7f4fcc0714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:23.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.496+0000 7f4fd3420700 1 -- 172.21.15.78:0/2674671752 shutdown_connections 2023-10-11T09:48:23.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.496+0000 7f4fd3420700 1 -- 172.21.15.78:0/2674671752 wait complete. 2023-10-11T09:48:23.506 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 58 2023-10-11T09:48:23.526 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:48:23.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.684+0000 7fd4c2f29700 1 Processor -- start 2023-10-11T09:48:23.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.684+0000 7fd4c2f29700 1 -- start start 2023-10-11T09:48:23.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.684+0000 7fd4c2f29700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd4bc0cda10 0x7fd4bc0caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:23.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.684+0000 7fd4c2f29700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4bc0cdde0 0x7fd4bc164610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:23.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.684+0000 7fd4c2f29700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4bc165390 con 0x7fd4bc164b50 2023-10-11T09:48:23.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.684+0000 7fd4c2f29700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4bc0ca440 con 0x7fd4bc0cda10 2023-10-11T09:48:23.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.684+0000 7fd4c2f29700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4bc0ca720 con 0x7fd4bc0cdde0 2023-10-11T09:48:23.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.684+0000 7fd4c1f27700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd4bc0cda10 0x7fd4bc0caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:23.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.684+0000 7fd4c1726700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4bc0cdde0 0x7fd4bc164610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:23.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.684+0000 7fd4c2728700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fd4bc164b50 0x7fd4bc06bbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:43324/0 (socket says 172.21.15.78:43324) 2023-10-11T09:48:23.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.684+0000 7fd4c2728700 1 -- 172.21.15.78:0/512302833 learned_addr learned my addr 172.21.15.78:0/512302833 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:23.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.688+0000 7fd4c1f27700 1 -- 172.21.15.78:0/512302833 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4bc0cdde0 msgr2=0x7fd4bc164610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:23.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.688+0000 7fd4c1f27700 1 --2- 172.21.15.78:0/512302833 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4bc0cdde0 0x7fd4bc164610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:23.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.688+0000 7fd4c1f27700 1 -- 172.21.15.78:0/512302833 >> v1:172.21.15.133:6789/0 conn(0x7fd4bc164b50 legacy=0x7fd4bc06bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:23.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.688+0000 7fd4c1f27700 1 -- 172.21.15.78:0/512302833 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4bc0caa00 con 0x7fd4bc0cda10 2023-10-11T09:48:23.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.688+0000 7fd4c0f25700 1 -- 172.21.15.78:0/512302833 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3242091610 0 0) 0x7fd4bc165390 con 0x7fd4bc164b50 2023-10-11T09:48:23.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.688+0000 7fd4c1f27700 1 --2- 172.21.15.78:0/512302833 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4bc0cda10 0x7fd4bc0caf20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8009850 tx=0x7fd4b8009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6a8fb21455d84bd server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:23.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.688+0000 7fd4c0f25700 1 -- 172.21.15.78:0/512302833 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8011830 con 0x7fd4bc0cda10 2023-10-11T09:48:23.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.688+0000 7fd4c0f25700 1 -- 172.21.15.78:0/512302833 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b8011990 con 0x7fd4bc0cda10 2023-10-11T09:48:23.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.688+0000 7fd4c0f25700 1 -- 172.21.15.78:0/512302833 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8011d40 con 0x7fd4bc0cda10 2023-10-11T09:48:23.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.688+0000 7fd4c2f29700 1 -- 172.21.15.78:0/512302833 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4bc0cda10 msgr2=0x7fd4bc0caf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:23.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.688+0000 7fd4c2f29700 1 --2- 172.21.15.78:0/512302833 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4bc0cda10 0x7fd4bc0caf20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8009850 tx=0x7fd4b8009b60 comp rx=0 tx=0).stop 2023-10-11T09:48:23.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.688+0000 7fd4c2f29700 1 -- 172.21.15.78:0/512302833 shutdown_connections 2023-10-11T09:48:23.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.688+0000 7fd4c2f29700 1 --2- 172.21.15.78:0/512302833 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4bc0cdde0 0x7fd4bc164610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:23.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.688+0000 7fd4c2f29700 1 --2- 172.21.15.78:0/512302833 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4bc0cda10 0x7fd4bc0caf20 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:23.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.688+0000 7fd4c2f29700 1 -- 172.21.15.78:0/512302833 >> 172.21.15.78:0/512302833 conn(0x7fd4bc0cea60 msgr2=0x7fd4bc0cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:23.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.688+0000 7fd4c2f29700 1 -- 172.21.15.78:0/512302833 shutdown_connections 2023-10-11T09:48:23.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.688+0000 7fd4c2f29700 1 -- 172.21.15.78:0/512302833 wait complete. 2023-10-11T09:48:23.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.692+0000 7fd4c2f29700 1 Processor -- start 2023-10-11T09:48:23.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.692+0000 7fd4c2f29700 1 -- start start 2023-10-11T09:48:23.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.696+0000 7fd4c2f29700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4bc0cda10 0x7fd4bc144b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:23.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.696+0000 7fd4c2f29700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4bc0cdde0 0x7fd4bc0b3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:23.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.696+0000 7fd4c2f29700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4bc164b50 0x7fd4bc163030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:23.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.696+0000 7fd4c2f29700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4bc161ef0 con 0x7fd4bc164b50 2023-10-11T09:48:23.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.696+0000 7fd4c2f29700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4bc1621d0 con 0x7fd4bc0cdde0 2023-10-11T09:48:23.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.696+0000 7fd4c2f29700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4bc1624b0 con 0x7fd4bc0cda10 2023-10-11T09:48:23.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.696+0000 7fd4c2728700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4bc164b50 0x7fd4bc163030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:23.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.696+0000 7fd4c2728700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4bc164b50 0x7fd4bc163030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33930/0 (socket says 172.21.15.78:33930) 2023-10-11T09:48:23.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.696+0000 7fd4c2728700 1 -- 172.21.15.78:0/1410384310 learned_addr learned my addr 172.21.15.78:0/1410384310 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:23.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.696+0000 7fd4c1f27700 1 --2- 172.21.15.78:0/1410384310 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4bc0cda10 0x7fd4bc144b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:23.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.696+0000 7fd4c1726700 1 --2- 172.21.15.78:0/1410384310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4bc0cdde0 0x7fd4bc0b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:23.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.696+0000 7fd4c2728700 1 -- 172.21.15.78:0/1410384310 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4bc0cda10 msgr2=0x7fd4bc144b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:23.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.696+0000 7fd4c2728700 1 --2- 172.21.15.78:0/1410384310 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4bc0cda10 0x7fd4bc144b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:23.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.696+0000 7fd4c2728700 1 -- 172.21.15.78:0/1410384310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4bc0cdde0 msgr2=0x7fd4bc0b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:23.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.696+0000 7fd4c2728700 1 --2- 172.21.15.78:0/1410384310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4bc0cdde0 0x7fd4bc0b3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:23.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.696+0000 7fd4c2728700 1 -- 172.21.15.78:0/1410384310 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b80094b0 con 0x7fd4bc164b50 2023-10-11T09:48:23.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.696+0000 7fd4c1726700 1 --2- 172.21.15.78:0/1410384310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4bc0cdde0 0x7fd4bc0b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:48:23.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.696+0000 7fd4c2728700 1 --2- 172.21.15.78:0/1410384310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4bc164b50 0x7fd4bc163030 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fd4b4002230 tx=0x7fd4b400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:23.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.696+0000 7fd4b27fc700 1 -- 172.21.15.78:0/1410384310 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b400ab70 con 0x7fd4bc164b50 2023-10-11T09:48:23.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.696+0000 7fd4b27fc700 1 -- 172.21.15.78:0/1410384310 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b400acf0 con 0x7fd4bc164b50 2023-10-11T09:48:23.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.696+0000 7fd4b27fc700 1 -- 172.21.15.78:0/1410384310 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b4007310 con 0x7fd4bc164b50 2023-10-11T09:48:23.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.696+0000 7fd4c2f29700 1 -- 172.21.15.78:0/1410384310 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4bc0ce3b0 con 0x7fd4bc164b50 2023-10-11T09:48:23.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.696+0000 7fd4c2f29700 1 -- 172.21.15.78:0/1410384310 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4bc162a70 con 0x7fd4bc164b50 2023-10-11T09:48:23.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.700+0000 7fd4c2f29700 1 -- 172.21.15.78:0/1410384310 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4bc065d40 con 0x7fd4bc164b50 2023-10-11T09:48:23.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.700+0000 7fd4b27fc700 1 -- 172.21.15.78:0/1410384310 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fd4b4007470 con 0x7fd4bc164b50 2023-10-11T09:48:23.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.700+0000 7fd4b27fc700 1 --2- 172.21.15.78:0/1410384310 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd4a0065ed0 0x7fd4a0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:23.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.700+0000 7fd4b27fc700 1 -- 172.21.15.78:0/1410384310 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd4b40832a0 con 0x7fd4bc164b50 2023-10-11T09:48:23.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.700+0000 7fd4c1f27700 1 --2- 172.21.15.78:0/1410384310 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd4a0065ed0 0x7fd4a0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:23.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.704+0000 7fd4c1f27700 1 --2- 172.21.15.78:0/1410384310 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd4a0065ed0 0x7fd4a0068380 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd4b80090b0 tx=0x7fd4b801c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:23.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.708+0000 7fd4b27fc700 1 -- 172.21.15.78:0/1410384310 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fd4b40508f0 con 0x7fd4bc164b50 2023-10-11T09:48:23.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.880+0000 7fd4c2f29700 1 -- 172.21.15.78:0/1410384310 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4bc0c8f10 con 0x7fd4bc164b50 2023-10-11T09:48:23.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.884+0000 7fd4b27fc700 1 -- 172.21.15.78:0/1410384310 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+17886 (secure 0 0 0) 0x7fd4b400ae60 con 0x7fd4bc164b50 2023-10-11T09:48:23.894 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:23.895 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":83,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:48:21.028496+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:48:18.491556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-11T09:48:19.018937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:48:23.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.888+0000 7fd4c2f29700 1 -- 172.21.15.78:0/1410384310 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd4a0065ed0 msgr2=0x7fd4a0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:23.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.888+0000 7fd4c2f29700 1 --2- 172.21.15.78:0/1410384310 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd4a0065ed0 0x7fd4a0068380 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd4b80090b0 tx=0x7fd4b801c040 comp rx=0 tx=0).stop 2023-10-11T09:48:23.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.888+0000 7fd4c2f29700 1 -- 172.21.15.78:0/1410384310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4bc164b50 msgr2=0x7fd4bc163030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:23.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.888+0000 7fd4c2f29700 1 --2- 172.21.15.78:0/1410384310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4bc164b50 0x7fd4bc163030 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fd4b4002230 tx=0x7fd4b400cf50 comp rx=0 tx=0).stop 2023-10-11T09:48:23.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.888+0000 7fd4c2f29700 1 -- 172.21.15.78:0/1410384310 shutdown_connections 2023-10-11T09:48:23.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.888+0000 7fd4c2f29700 1 --2- 172.21.15.78:0/1410384310 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd4a0065ed0 0x7fd4a0068380 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:23.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.888+0000 7fd4c2f29700 1 --2- 172.21.15.78:0/1410384310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4bc164b50 0x7fd4bc163030 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:23.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.888+0000 7fd4c2f29700 1 --2- 172.21.15.78:0/1410384310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4bc0cdde0 0x7fd4bc0b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:23.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.888+0000 7fd4c2f29700 1 --2- 172.21.15.78:0/1410384310 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4bc0cda10 0x7fd4bc144b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:23.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.888+0000 7fd4c2f29700 1 -- 172.21.15.78:0/1410384310 >> 172.21.15.78:0/1410384310 conn(0x7fd4bc0cea60 msgr2=0x7fd4bc06fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:23.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.888+0000 7fd4c2f29700 1 -- 172.21.15.78:0/1410384310 shutdown_connections 2023-10-11T09:48:23.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:23.888+0000 7fd4c2f29700 1 -- 172.21.15.78:0/1410384310 wait complete. 2023-10-11T09:48:23.919 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-11T09:48:23.920 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-11T09:48:24.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.080+0000 7fdc25c6d700 1 Processor -- start 2023-10-11T09:48:24.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.080+0000 7fdc25c6d700 1 -- start start 2023-10-11T09:48:24.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.084+0000 7fdc25c6d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdc2015d050 0x7fdc200cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:24.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.084+0000 7fdc25c6d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fdc200b06a0 0x7fdc200b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:24.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.084+0000 7fdc25c6d700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc20160210 con 0x7fdc20160c60 2023-10-11T09:48:24.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.084+0000 7fdc25c6d700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc200b1020 con 0x7fdc200b06a0 2023-10-11T09:48:24.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.084+0000 7fdc25c6d700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc200ccd60 con 0x7fdc2015d050 2023-10-11T09:48:24.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.084+0000 7fdc2546c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fdc200b06a0 0x7fdc200b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:24.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.084+0000 7fdc2546c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fdc200b06a0 0x7fdc200b0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33940/0 (socket says 172.21.15.78:33940) 2023-10-11T09:48:24.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.084+0000 7fdc2546c700 1 -- 172.21.15.78:0/2021756474 learned_addr learned my addr 172.21.15.78:0/2021756474 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:24.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.084+0000 7fdc24c6b700 1 --2- 172.21.15.78:0/2021756474 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdc2015d050 0x7fdc200cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:24.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.084+0000 7fdc2546c700 1 -- 172.21.15.78:0/2021756474 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdc2015d050 msgr2=0x7fdc200cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:24.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.084+0000 7fdc2546c700 1 --2- 172.21.15.78:0/2021756474 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdc2015d050 0x7fdc200cc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:24.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.084+0000 7fdc2546c700 1 -- 172.21.15.78:0/2021756474 >> v1:172.21.15.78:6789/0 conn(0x7fdc20160c60 legacy=0x7fdc2015d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:24.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.084+0000 7fdc2546c700 1 -- 172.21.15.78:0/2021756474 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc200ccfc0 con 0x7fdc200b06a0 2023-10-11T09:48:24.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.084+0000 7fdc2546c700 1 --2- 172.21.15.78:0/2021756474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc200b06a0 0x7fdc200b0ae0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fdc18009910 tx=0x7fdc18009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd0fb94c326ac6b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:24.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.084+0000 7fdc177fe700 1 -- 172.21.15.78:0/2021756474 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc18011830 con 0x7fdc200b06a0 2023-10-11T09:48:24.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.084+0000 7fdc177fe700 1 -- 172.21.15.78:0/2021756474 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc18011990 con 0x7fdc200b06a0 2023-10-11T09:48:24.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.084+0000 7fdc177fe700 1 -- 172.21.15.78:0/2021756474 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc18011d40 con 0x7fdc200b06a0 2023-10-11T09:48:24.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.084+0000 7fdc25c6d700 1 -- 172.21.15.78:0/2021756474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc200b06a0 msgr2=0x7fdc200b0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:24.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.084+0000 7fdc25c6d700 1 --2- 172.21.15.78:0/2021756474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc200b06a0 0x7fdc200b0ae0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fdc18009910 tx=0x7fdc18009c20 comp rx=0 tx=0).stop 2023-10-11T09:48:24.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.084+0000 7fdc25c6d700 1 -- 172.21.15.78:0/2021756474 shutdown_connections 2023-10-11T09:48:24.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.084+0000 7fdc25c6d700 1 --2- 172.21.15.78:0/2021756474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc200b06a0 0x7fdc200b0ae0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:24.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.084+0000 7fdc25c6d700 1 --2- 172.21.15.78:0/2021756474 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdc2015d050 0x7fdc200cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:24.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.084+0000 7fdc25c6d700 1 -- 172.21.15.78:0/2021756474 >> 172.21.15.78:0/2021756474 conn(0x7fdc200cea60 msgr2=0x7fdc200cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:24.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.088+0000 7fdc25c6d700 1 -- 172.21.15.78:0/2021756474 shutdown_connections 2023-10-11T09:48:24.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.088+0000 7fdc25c6d700 1 -- 172.21.15.78:0/2021756474 wait complete. 2023-10-11T09:48:24.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.088+0000 7fdc25c6d700 1 Processor -- start 2023-10-11T09:48:24.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.088+0000 7fdc25c6d700 1 -- start start 2023-10-11T09:48:24.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.092+0000 7fdc25c6d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc200b06a0 0x7fdc200afca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:24.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.092+0000 7fdc25c6d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdc2015d050 0x7fdc201445e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:24.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.092+0000 7fdc25c6d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc20160c60 0x7fdc200b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:24.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.092+0000 7fdc25c6d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc20144b20 con 0x7fdc20160c60 2023-10-11T09:48:24.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.092+0000 7fdc25c6d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2006fc70 con 0x7fdc200b06a0 2023-10-11T09:48:24.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.092+0000 7fdc25c6d700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc200b2c70 con 0x7fdc2015d050 2023-10-11T09:48:24.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.092+0000 7fdc2546c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc20160c60 0x7fdc200b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:24.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.092+0000 7fdc2546c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc20160c60 0x7fdc200b1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33946/0 (socket says 172.21.15.78:33946) 2023-10-11T09:48:24.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.092+0000 7fdc2546c700 1 -- 172.21.15.78:0/2916690410 learned_addr learned my addr 172.21.15.78:0/2916690410 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:24.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.092+0000 7fdc17fff700 1 --2- 172.21.15.78:0/2916690410 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdc2015d050 0x7fdc201445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:24.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.092+0000 7fdc24c6b700 1 --2- 172.21.15.78:0/2916690410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc200b06a0 0x7fdc200afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:24.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.092+0000 7fdc2546c700 1 -- 172.21.15.78:0/2916690410 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdc2015d050 msgr2=0x7fdc201445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:24.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.092+0000 7fdc2546c700 1 --2- 172.21.15.78:0/2916690410 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdc2015d050 0x7fdc201445e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:24.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.092+0000 7fdc2546c700 1 -- 172.21.15.78:0/2916690410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc200b06a0 msgr2=0x7fdc200afca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:24.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.092+0000 7fdc2546c700 1 --2- 172.21.15.78:0/2916690410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc200b06a0 0x7fdc200afca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:24.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.092+0000 7fdc2546c700 1 -- 172.21.15.78:0/2916690410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc18009570 con 0x7fdc20160c60 2023-10-11T09:48:24.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.096+0000 7fdc2546c700 1 --2- 172.21.15.78:0/2916690410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc20160c60 0x7fdc200b1a10 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fdc20072690 tx=0x7fdc18004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:24.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.096+0000 7fdc157fa700 1 -- 172.21.15.78:0/2916690410 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc18004510 con 0x7fdc20160c60 2023-10-11T09:48:24.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.096+0000 7fdc157fa700 1 -- 172.21.15.78:0/2916690410 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1801e070 con 0x7fdc20160c60 2023-10-11T09:48:24.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.096+0000 7fdc25c6d700 1 -- 172.21.15.78:0/2916690410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc200b2f50 con 0x7fdc20160c60 2023-10-11T09:48:24.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.096+0000 7fdc157fa700 1 -- 172.21.15.78:0/2916690410 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc18003900 con 0x7fdc20160c60 2023-10-11T09:48:24.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.096+0000 7fdc25c6d700 1 -- 172.21.15.78:0/2916690410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc2006ff10 con 0x7fdc20160c60 2023-10-11T09:48:24.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.096+0000 7fdc25c6d700 1 -- 172.21.15.78:0/2916690410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc20070100 con 0x7fdc20160c60 2023-10-11T09:48:24.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.100+0000 7fdc157fa700 1 -- 172.21.15.78:0/2916690410 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fdc18003a60 con 0x7fdc20160c60 2023-10-11T09:48:24.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.100+0000 7fdc157fa700 1 --2- 172.21.15.78:0/2916690410 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdc0c065ed0 0x7fdc0c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:24.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.100+0000 7fdc157fa700 1 -- 172.21.15.78:0/2916690410 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdc18084a70 con 0x7fdc20160c60 2023-10-11T09:48:24.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.100+0000 7fdc24c6b700 1 --2- 172.21.15.78:0/2916690410 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdc0c065ed0 0x7fdc0c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:24.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.100+0000 7fdc24c6b700 1 --2- 172.21.15.78:0/2916690410 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdc0c065ed0 0x7fdc0c068380 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c011340 tx=0x7fdc1c00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:24.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.104+0000 7fdc157fa700 1 -- 172.21.15.78:0/2916690410 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fdc18052150 con 0x7fdc20160c60 2023-10-11T09:48:24.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.276+0000 7fdc25c6d700 1 -- 172.21.15.78:0/2916690410 --> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdc200b3470 con 0x7fdc0c065ed0 2023-10-11T09:48:24.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.292+0000 7fdc157fa700 1 -- 172.21.15.78:0/2916690410 <== mgr.4101 v2:172.21.15.78:6832/35606 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+294243 (secure 0 0 0) 0x7fdc1c00f040 con 0x7fdc0c065ed0 2023-10-11T09:48:24.304 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:24.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.300+0000 7fdc25c6d700 1 -- 172.21.15.78:0/2916690410 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdc0c065ed0 msgr2=0x7fdc0c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:24.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.300+0000 7fdc25c6d700 1 --2- 172.21.15.78:0/2916690410 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdc0c065ed0 0x7fdc0c068380 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c011340 tx=0x7fdc1c00b040 comp rx=0 tx=0).stop 2023-10-11T09:48:24.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.300+0000 7fdc25c6d700 1 -- 172.21.15.78:0/2916690410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc20160c60 msgr2=0x7fdc200b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:24.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.300+0000 7fdc25c6d700 1 --2- 172.21.15.78:0/2916690410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc20160c60 0x7fdc200b1a10 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fdc20072690 tx=0x7fdc18004770 comp rx=0 tx=0).stop 2023-10-11T09:48:24.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.300+0000 7fdc25c6d700 1 -- 172.21.15.78:0/2916690410 shutdown_connections 2023-10-11T09:48:24.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.300+0000 7fdc25c6d700 1 --2- 172.21.15.78:0/2916690410 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdc0c065ed0 0x7fdc0c068380 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:24.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.300+0000 7fdc25c6d700 1 --2- 172.21.15.78:0/2916690410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc20160c60 0x7fdc200b1a10 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:24.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.300+0000 7fdc25c6d700 1 --2- 172.21.15.78:0/2916690410 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdc2015d050 0x7fdc201445e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:24.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.300+0000 7fdc25c6d700 1 --2- 172.21.15.78:0/2916690410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc200b06a0 0x7fdc200afca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:24.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.300+0000 7fdc25c6d700 1 -- 172.21.15.78:0/2916690410 >> 172.21.15.78:0/2916690410 conn(0x7fdc200cea60 msgr2=0x7fdc20155da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:24.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.300+0000 7fdc25c6d700 1 -- 172.21.15.78:0/2916690410 shutdown_connections 2023-10-11T09:48:24.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.300+0000 7fdc25c6d700 1 -- 172.21.15.78:0/2916690410 wait complete. 2023-10-11T09:48:24.312 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2023-10-11T09:48:24.330 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":313,"stamp":"2023-10-11T09:48:22.487517+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":252968,"kb_used_data":8680,"kb_used_omap":0,"kb_used_meta":244224,"kb_avail":754721752,"statfs":{"total":773094113280,"available":772835074048,"internally_reserved":0,"allocated":8888320,"data_stored":4798352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":250085376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001008"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:18.531756+0000","last_change":"2023-10-11T09:48:18.531756+0000","last_active":"2023-10-11T09:48:18.531756+0000","last_peered":"2023-10-11T09:48:18.531756+0000","last_clean":"2023-10-11T09:48:18.531756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:18.531756+0000","last_undegraded":"2023-10-11T09:48:18.531756+0000","last_fullsized":"2023-10-11T09:48:18.531756+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:18.531756+0000","last_change":"2023-10-11T09:48:18.531756+0000","last_active":"2023-10-11T09:48:18.531756+0000","last_peered":"2023-10-11T09:48:18.531756+0000","last_clean":"2023-10-11T09:48:18.531756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:18.531756+0000","last_undegraded":"2023-10-11T09:48:18.531756+0000","last_fullsized":"2023-10-11T09:48:18.531756+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:18.531756+0000","last_change":"2023-10-11T09:48:18.531756+0000","last_active":"2023-10-11T09:48:18.531756+0000","last_peered":"2023-10-11T09:48:18.531756+0000","last_clean":"2023-10-11T09:48:18.531756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:18.531756+0000","last_undegraded":"2023-10-11T09:48:18.531756+0000","last_fullsized":"2023-10-11T09:48:18.531756+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:18.531756+0000","last_change":"2023-10-11T09:48:18.531756+0000","last_active":"2023-10-11T09:48:18.531756+0000","last_peered":"2023-10-11T09:48:18.531756+0000","last_clean":"2023-10-11T09:48:18.531756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:18.531756+0000","last_undegraded":"2023-10-11T09:48:18.531756+0000","last_fullsized":"2023-10-11T09:48:18.531756+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:18.531756+0000","last_change":"2023-10-11T09:48:18.531756+0000","last_active":"2023-10-11T09:48:18.531756+0000","last_peered":"2023-10-11T09:48:18.531756+0000","last_clean":"2023-10-11T09:48:18.531756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:18.531756+0000","last_undegraded":"2023-10-11T09:48:18.531756+0000","last_fullsized":"2023-10-11T09:48:18.531756+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:18.531756+0000","last_change":"2023-10-11T09:48:18.531756+0000","last_active":"2023-10-11T09:48:18.531756+0000","last_peered":"2023-10-11T09:48:18.531756+0000","last_clean":"2023-10-11T09:48:18.531756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:18.531756+0000","last_undegraded":"2023-10-11T09:48:18.531756+0000","last_fullsized":"2023-10-11T09:48:18.531756+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:18.531756+0000","last_change":"2023-10-11T09:48:18.531756+0000","last_active":"2023-10-11T09:48:18.531756+0000","last_peered":"2023-10-11T09:48:18.531756+0000","last_clean":"2023-10-11T09:48:18.531756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:18.531756+0000","last_undegraded":"2023-10-11T09:48:18.531756+0000","last_fullsized":"2023-10-11T09:48:18.531756+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:18.531756+0000","last_change":"2023-10-11T09:48:18.531756+0000","last_active":"2023-10-11T09:48:18.531756+0000","last_peered":"2023-10-11T09:48:18.531756+0000","last_clean":"2023-10-11T09:48:18.531756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:18.531756+0000","last_undegraded":"2023-10-11T09:48:18.531756+0000","last_fullsized":"2023-10-11T09:48:18.531756+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:18.531756+0000","last_change":"2023-10-11T09:48:18.531756+0000","last_active":"2023-10-11T09:48:18.531756+0000","last_peered":"2023-10-11T09:48:18.531756+0000","last_clean":"2023-10-11T09:48:18.531756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:18.531756+0000","last_undegraded":"2023-10-11T09:48:18.531756+0000","last_fullsized":"2023-10-11T09:48:18.531756+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:18.531756+0000","last_change":"2023-10-11T09:48:18.531756+0000","last_active":"2023-10-11T09:48:18.531756+0000","last_peered":"2023-10-11T09:48:18.531756+0000","last_clean":"2023-10-11T09:48:18.531756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:18.531756+0000","last_undegraded":"2023-10-11T09:48:18.531756+0000","last_fullsized":"2023-10-11T09:48:18.531756+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:18.531756+0000","last_change":"2023-10-11T09:48:18.531756+0000","last_active":"2023-10-11T09:48:18.531756+0000","last_peered":"2023-10-11T09:48:18.531756+0000","last_clean":"2023-10-11T09:48:18.531756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:18.531756+0000","last_undegraded":"2023-10-11T09:48:18.531756+0000","last_fullsized":"2023-10-11T09:48:18.531756+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:18.531756+0000","last_change":"2023-10-11T09:48:18.531756+0000","last_active":"2023-10-11T09:48:18.531756+0000","last_peered":"2023-10-11T09:48:18.531756+0000","last_clean":"2023-10-11T09:48:18.531756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:18.531756+0000","last_undegraded":"2023-10-11T09:48:18.531756+0000","last_fullsized":"2023-10-11T09:48:18.531756+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:18.531756+0000","last_change":"2023-10-11T09:48:18.531756+0000","last_active":"2023-10-11T09:48:18.531756+0000","last_peered":"2023-10-11T09:48:18.531756+0000","last_clean":"2023-10-11T09:48:18.531756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:18.531756+0000","last_undegraded":"2023-10-11T09:48:18.531756+0000","last_fullsized":"2023-10-11T09:48:18.531756+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:18.531756+0000","last_change":"2023-10-11T09:48:18.531756+0000","last_active":"2023-10-11T09:48:18.531756+0000","last_peered":"2023-10-11T09:48:18.531756+0000","last_clean":"2023-10-11T09:48:18.531756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:18.531756+0000","last_undegraded":"2023-10-11T09:48:18.531756+0000","last_fullsized":"2023-10-11T09:48:18.531756+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:18.531756+0000","last_change":"2023-10-11T09:48:18.531756+0000","last_active":"2023-10-11T09:48:18.531756+0000","last_peered":"2023-10-11T09:48:18.531756+0000","last_clean":"2023-10-11T09:48:18.531756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:18.531756+0000","last_undegraded":"2023-10-11T09:48:18.531756+0000","last_fullsized":"2023-10-11T09:48:18.531756+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:18.531756+0000","last_change":"2023-10-11T09:48:18.531756+0000","last_active":"2023-10-11T09:48:18.531756+0000","last_peered":"2023-10-11T09:48:18.531756+0000","last_clean":"2023-10-11T09:48:18.531756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:18.531756+0000","last_undegraded":"2023-10-11T09:48:18.531756+0000","last_fullsized":"2023-10-11T09:48:18.531756+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:18.531756+0000","last_change":"2023-10-11T09:48:18.531756+0000","last_active":"2023-10-11T09:48:18.531756+0000","last_peered":"2023-10-11T09:48:18.531756+0000","last_clean":"2023-10-11T09:48:18.531756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:18.531756+0000","last_undegraded":"2023-10-11T09:48:18.531756+0000","last_fullsized":"2023-10-11T09:48:18.531756+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:18.531756+0000","last_change":"2023-10-11T09:48:18.531756+0000","last_active":"2023-10-11T09:48:18.531756+0000","last_peered":"2023-10-11T09:48:18.531756+0000","last_clean":"2023-10-11T09:48:18.531756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:18.531756+0000","last_undegraded":"2023-10-11T09:48:18.531756+0000","last_fullsized":"2023-10-11T09:48:18.531756+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:18.531756+0000","last_change":"2023-10-11T09:48:18.531756+0000","last_active":"2023-10-11T09:48:18.531756+0000","last_peered":"2023-10-11T09:48:18.531756+0000","last_clean":"2023-10-11T09:48:18.531756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:18.531756+0000","last_undegraded":"2023-10-11T09:48:18.531756+0000","last_fullsized":"2023-10-11T09:48:18.531756+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:18.531756+0000","last_change":"2023-10-11T09:48:18.531756+0000","last_active":"2023-10-11T09:48:18.531756+0000","last_peered":"2023-10-11T09:48:18.531756+0000","last_clean":"2023-10-11T09:48:18.531756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:18.531756+0000","last_undegraded":"2023-10-11T09:48:18.531756+0000","last_fullsized":"2023-10-11T09:48:18.531756+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:18.531756+0000","last_change":"2023-10-11T09:48:18.531756+0000","last_active":"2023-10-11T09:48:18.531756+0000","last_peered":"2023-10-11T09:48:18.531756+0000","last_clean":"2023-10-11T09:48:18.531756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:18.531756+0000","last_undegraded":"2023-10-11T09:48:18.531756+0000","last_fullsized":"2023-10-11T09:48:18.531756+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:18.531756+0000","last_change":"2023-10-11T09:48:18.531756+0000","last_active":"2023-10-11T09:48:18.531756+0000","last_peered":"2023-10-11T09:48:18.531756+0000","last_clean":"2023-10-11T09:48:18.531756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:18.531756+0000","last_undegraded":"2023-10-11T09:48:18.531756+0000","last_fullsized":"2023-10-11T09:48:18.531756+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:18.531756+0000","last_change":"2023-10-11T09:48:18.531756+0000","last_active":"2023-10-11T09:48:18.531756+0000","last_peered":"2023-10-11T09:48:18.531756+0000","last_clean":"2023-10-11T09:48:18.531756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:18.531756+0000","last_undegraded":"2023-10-11T09:48:18.531756+0000","last_fullsized":"2023-10-11T09:48:18.531756+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:18.531756+0000","last_change":"2023-10-11T09:48:18.531756+0000","last_active":"2023-10-11T09:48:18.531756+0000","last_peered":"2023-10-11T09:48:18.531756+0000","last_clean":"2023-10-11T09:48:18.531756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:18.531756+0000","last_undegraded":"2023-10-11T09:48:18.531756+0000","last_fullsized":"2023-10-11T09:48:18.531756+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:18.531756+0000","last_change":"2023-10-11T09:48:18.531756+0000","last_active":"2023-10-11T09:48:18.531756+0000","last_peered":"2023-10-11T09:48:18.531756+0000","last_clean":"2023-10-11T09:48:18.531756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:18.531756+0000","last_undegraded":"2023-10-11T09:48:18.531756+0000","last_fullsized":"2023-10-11T09:48:18.531756+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":56,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-11T09:48:15.341934+0000","last_change":"2023-10-11T09:40:36.151839+0000","last_active":"2023-10-11T09:48:15.341934+0000","last_peered":"2023-10-11T09:48:15.341934+0000","last_clean":"2023-10-11T09:48:15.341934+0000","last_became_active":"2023-10-11T09:40:36.151179+0000","last_became_peered":"2023-10-11T09:40:36.151179+0000","last_unstale":"2023-10-11T09:48:15.341934+0000","last_undegraded":"2023-10-11T09:48:15.341934+0000","last_fullsized":"2023-10-11T09:48:15.341934+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:18.531756+0000","last_change":"2023-10-11T09:48:18.531756+0000","last_active":"2023-10-11T09:48:18.531756+0000","last_peered":"2023-10-11T09:48:18.531756+0000","last_clean":"2023-10-11T09:48:18.531756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:18.531756+0000","last_undegraded":"2023-10-11T09:48:18.531756+0000","last_fullsized":"2023-10-11T09:48:18.531756+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-11T09:48:15.339618+0000","last_change":"2023-10-11T09:40:36.150224+0000","last_active":"2023-10-11T09:48:15.339618+0000","last_peered":"2023-10-11T09:48:15.339618+0000","last_clean":"2023-10-11T09:48:15.339618+0000","last_became_active":"2023-10-11T09:40:36.149919+0000","last_became_peered":"2023-10-11T09:40:36.149919+0000","last_unstale":"2023-10-11T09:48:15.339618+0000","last_undegraded":"2023-10-11T09:48:15.339618+0000","last_fullsized":"2023-10-11T09:48:15.339618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:18.531756+0000","last_change":"2023-10-11T09:48:18.531756+0000","last_active":"2023-10-11T09:48:18.531756+0000","last_peered":"2023-10-11T09:48:18.531756+0000","last_clean":"2023-10-11T09:48:18.531756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:18.531756+0000","last_undegraded":"2023-10-11T09:48:18.531756+0000","last_fullsized":"2023-10-11T09:48:18.531756+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":73,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-11T09:48:14.954765+0000","last_change":"2023-10-11T09:40:36.162174+0000","last_active":"2023-10-11T09:48:14.954765+0000","last_peered":"2023-10-11T09:48:14.954765+0000","last_clean":"2023-10-11T09:48:14.954765+0000","last_became_active":"2023-10-11T09:40:36.161710+0000","last_became_peered":"2023-10-11T09:40:36.161710+0000","last_unstale":"2023-10-11T09:48:14.954765+0000","last_undegraded":"2023-10-11T09:48:14.954765+0000","last_fullsized":"2023-10-11T09:48:14.954765+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:57:03.630878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:18.531756+0000","last_change":"2023-10-11T09:48:18.531756+0000","last_active":"2023-10-11T09:48:18.531756+0000","last_peered":"2023-10-11T09:48:18.531756+0000","last_clean":"2023-10-11T09:48:18.531756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:18.531756+0000","last_undegraded":"2023-10-11T09:48:18.531756+0000","last_fullsized":"2023-10-11T09:48:18.531756+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-11T09:48:15.339611+0000","last_change":"2023-10-11T09:40:36.160620+0000","last_active":"2023-10-11T09:48:15.339611+0000","last_peered":"2023-10-11T09:48:15.339611+0000","last_clean":"2023-10-11T09:48:15.339611+0000","last_became_active":"2023-10-11T09:40:36.160171+0000","last_became_peered":"2023-10-11T09:40:36.160171+0000","last_unstale":"2023-10-11T09:48:15.339611+0000","last_undegraded":"2023-10-11T09:48:15.339611+0000","last_fullsized":"2023-10-11T09:48:15.339611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:18.531756+0000","last_change":"2023-10-11T09:48:18.531756+0000","last_active":"2023-10-11T09:48:18.531756+0000","last_peered":"2023-10-11T09:48:18.531756+0000","last_clean":"2023-10-11T09:48:18.531756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:18.531756+0000","last_undegraded":"2023-10-11T09:48:18.531756+0000","last_fullsized":"2023-10-11T09:48:18.531756+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-11T09:48:14.955349+0000","last_change":"2023-10-11T09:40:36.151710+0000","last_active":"2023-10-11T09:48:14.955349+0000","last_peered":"2023-10-11T09:48:14.955349+0000","last_clean":"2023-10-11T09:48:14.955349+0000","last_became_active":"2023-10-11T09:40:36.151219+0000","last_became_peered":"2023-10-11T09:40:36.151219+0000","last_unstale":"2023-10-11T09:48:14.955349+0000","last_undegraded":"2023-10-11T09:48:14.955349+0000","last_fullsized":"2023-10-11T09:48:14.955349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:18.531756+0000","last_change":"2023-10-11T09:48:18.531756+0000","last_active":"2023-10-11T09:48:18.531756+0000","last_peered":"2023-10-11T09:48:18.531756+0000","last_clean":"2023-10-11T09:48:18.531756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:18.531756+0000","last_undegraded":"2023-10-11T09:48:18.531756+0000","last_fullsized":"2023-10-11T09:48:18.531756+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":60,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-11T09:48:14.955207+0000","last_change":"2023-10-11T09:40:36.216248+0000","last_active":"2023-10-11T09:48:14.955207+0000","last_peered":"2023-10-11T09:48:14.955207+0000","last_clean":"2023-10-11T09:48:14.955207+0000","last_became_active":"2023-10-11T09:40:36.215890+0000","last_became_peered":"2023-10-11T09:40:36.215890+0000","last_unstale":"2023-10-11T09:48:14.955207+0000","last_undegraded":"2023-10-11T09:48:14.955207+0000","last_fullsized":"2023-10-11T09:48:14.955207+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:18.531756+0000","last_change":"2023-10-11T09:48:18.531756+0000","last_active":"2023-10-11T09:48:18.531756+0000","last_peered":"2023-10-11T09:48:18.531756+0000","last_clean":"2023-10-11T09:48:18.531756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:18.531756+0000","last_undegraded":"2023-10-11T09:48:18.531756+0000","last_fullsized":"2023-10-11T09:48:18.531756+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":73,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-11T09:48:14.954721+0000","last_change":"2023-10-11T09:40:36.149739+0000","last_active":"2023-10-11T09:48:14.954721+0000","last_peered":"2023-10-11T09:48:14.954721+0000","last_clean":"2023-10-11T09:48:14.954721+0000","last_became_active":"2023-10-11T09:40:36.149419+0000","last_became_peered":"2023-10-11T09:40:36.149419+0000","last_unstale":"2023-10-11T09:48:14.954721+0000","last_undegraded":"2023-10-11T09:48:14.954721+0000","last_fullsized":"2023-10-11T09:48:14.954721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:54:50.640362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:19.543459+0000","last_change":"2023-10-11T09:48:19.543459+0000","last_active":"2023-10-11T09:48:19.543459+0000","last_peered":"2023-10-11T09:48:19.543459+0000","last_clean":"2023-10-11T09:48:19.543459+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:19.543459+0000","last_undegraded":"2023-10-11T09:48:19.543459+0000","last_fullsized":"2023-10-11T09:48:19.543459+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:48:18.531756+0000","last_change":"2023-10-11T09:48:18.531756+0000","last_active":"2023-10-11T09:48:18.531756+0000","last_peered":"2023-10-11T09:48:18.531756+0000","last_clean":"2023-10-11T09:48:18.531756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:48:18.531756+0000","last_undegraded":"2023-10-11T09:48:18.531756+0000","last_fullsized":"2023-10-11T09:48:18.531756+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-11T09:48:14.955331+0000","last_change":"2023-10-11T09:40:36.148773+0000","last_active":"2023-10-11T09:48:14.955331+0000","last_peered":"2023-10-11T09:48:14.955331+0000","last_clean":"2023-10-11T09:48:14.955331+0000","last_became_active":"2023-10-11T09:40:36.148489+0000","last_became_peered":"2023-10-11T09:40:36.148489+0000","last_unstale":"2023-10-11T09:48:14.955331+0000","last_undegraded":"2023-10-11T09:48:14.955331+0000","last_fullsized":"2023-10-11T09:48:14.955331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":226,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-11T09:48:14.954756+0000","last_change":"2023-10-11T09:40:29.874372+0000","last_active":"2023-10-11T09:48:14.954756+0000","last_peered":"2023-10-11T09:48:14.954756+0000","last_clean":"2023-10-11T09:48:14.954756+0000","last_became_active":"2023-10-11T09:40:29.873621+0000","last_became_peered":"2023-10-11T09:40:29.873621+0000","last_unstale":"2023-10-11T09:48:14.954756+0000","last_undegraded":"2023-10-11T09:48:14.954756+0000","last_fullsized":"2023-10-11T09:48:14.954756+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_clean_scrub_stamp":"2023-10-11T09:40:28.742276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:22:07.985311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574946,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32568,"kb_used_data":1520,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339272,"statfs":{"total":96636764160,"available":96603414528,"internally_reserved":0,"allocated":1556480,"data_stored":1042570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:47:45 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.97999999999999998,"15min":1.1180000000000001},"min":{"1min":0.56499999999999995,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.2689999999999999,"5min":1.3799999999999999,"15min":1.903},"last":1.409},{"interface":"front","average":{"1min":0.88200000000000001,"5min":1.0209999999999999,"15min":1.1100000000000001},"min":{"1min":0.61899999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.323,"5min":1.5760000000000001,"15min":1.696},"last":1.373}]},{"osd":1,"last update":"Wed Oct 11 09:47:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.083,"15min":1.1240000000000001},"min":{"1min":0.72399999999999998,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":2.25},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.88900000000000001,"5min":1.026,"15min":1.109},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.155,"5min":1.508,"15min":1.696},"last":0.96599999999999997}]},{"osd":2,"last update":"Wed Oct 11 09:47:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.079,"15min":1.1140000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.58999999999999997},"max":{"1min":1.361,"5min":1.6339999999999999,"15min":1.873},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.1299999999999999,"15min":1.1439999999999999},"min":{"1min":0.77300000000000002,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.3879999999999999,"5min":1.5680000000000001,"15min":1.6779999999999999},"last":0.81200000000000006}]},{"osd":3,"last update":"Wed Oct 11 09:47:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1579999999999999,"15min":1.2330000000000001},"min":{"1min":0.80400000000000005,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.4430000000000001,"5min":1.552,"15min":1.724},"last":1.6459999999999999},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.238,"15min":1.2849999999999999},"min":{"1min":1.02,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.4299999999999999,"5min":1.661,"15min":2.1579999999999999},"last":1.5529999999999999}]},{"osd":4,"last update":"Wed Oct 11 09:47:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0589999999999999,"15min":1.1579999999999999},"min":{"1min":0.68300000000000005,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.4390000000000001,"5min":1.893,"15min":2.2010000000000001},"last":1.073},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1819999999999999,"15min":1.2450000000000001},"min":{"1min":0.70299999999999996,"5min":0.66600000000000004,"15min":0.63500000000000001},"max":{"1min":1.3859999999999999,"5min":1.8640000000000001,"15min":1.9339999999999999},"last":1.702}]},{"osd":5,"last update":"Wed Oct 11 09:47:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.258,"15min":1.2869999999999999},"min":{"1min":0.89600000000000002,"5min":0.85999999999999999,"15min":0.73499999999999999},"max":{"1min":1.5589999999999999,"5min":1.7250000000000001,"15min":1.744},"last":1.466},{"interface":"front","average":{"1min":1.139,"5min":1.1870000000000001,"15min":1.325},"min":{"1min":0.78300000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.3120000000000001,"5min":1.9690000000000001,"15min":3.0750000000000002},"last":1.335}]},{"osd":6,"last update":"Wed Oct 11 09:47:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.335,"15min":1.4279999999999999},"min":{"1min":0.95799999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.4530000000000001,"5min":2.0289999999999999,"15min":3.137},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.284,"5min":1.3380000000000001,"15min":1.401},"min":{"1min":0.93200000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":1.5009999999999999,"5min":1.8080000000000001,"15min":1.839},"last":1.5840000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574945,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":962560,"data_stored":452202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:47:38 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.997,"15min":0.99199999999999999},"min":{"1min":0.499,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.2450000000000001,"5min":1.589,"15min":2.1520000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.028,"5min":0.996,"15min":1.032},"min":{"1min":0.61599999999999999,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3839999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.9159999999999999}]},{"osd":1,"last update":"Wed Oct 11 09:47:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.1479999999999999,"15min":1.179},"min":{"1min":0.68600000000000005,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.613,"5min":1.794,"15min":1.9379999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.137,"15min":1.2210000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3620000000000001,"5min":1.52,"15min":1.8260000000000001},"last":1.881}]},{"osd":2,"last update":"Wed Oct 11 09:47:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.1200000000000001,"15min":1.149},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.508,"5min":1.7090000000000001,"15min":2.0859999999999999},"last":1.847},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.123,"15min":1.22},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.371,"5min":1.6419999999999999,"15min":2.605},"last":1.9590000000000001}]},{"osd":3,"last update":"Wed Oct 11 09:47:38 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.075,"15min":1.151},"min":{"1min":0.73399999999999999,"5min":0.627,"15min":0.627},"max":{"1min":1.2729999999999999,"5min":1.5589999999999999,"15min":2.0299999999999998},"last":1.9930000000000001},{"interface":"front","average":{"1min":1.085,"5min":1.137,"15min":1.248},"min":{"1min":0.76400000000000001,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.427,"5min":1.7509999999999999,"15min":3.0819999999999999},"last":1.135}]},{"osd":4,"last update":"Wed Oct 11 09:47:38 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.22,"15min":1.292},"min":{"1min":0.72099999999999997,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.486,"5min":1.7310000000000001,"15min":1.978},"last":0.69599999999999995},{"interface":"front","average":{"1min":1.194,"5min":1.2070000000000001,"15min":1.3280000000000001},"min":{"1min":0.84399999999999997,"5min":0.70599999999999996,"15min":0.69099999999999995},"max":{"1min":1.452,"5min":2.141,"15min":2.6920000000000002},"last":0.82299999999999995}]},{"osd":5,"last update":"Wed Oct 11 09:47:38 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.216,"15min":1.262},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.58099999999999996},"max":{"1min":2.4009999999999998,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":2.1560000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.2569999999999999,"15min":1.3720000000000001},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.077,"5min":2.1640000000000001,"15min":2.7290000000000001},"last":0.58999999999999997}]},{"osd":7,"last update":"Wed Oct 11 09:47:38 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.276,"15min":1.373},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.613,"5min":1.895,"15min":2.1899999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.321,"5min":1.393,"15min":1.4530000000000001},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.6919999999999999,"5min":1.859,"15min":2.3050000000000002},"last":0.96399999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574945,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27892,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343948,"statfs":{"total":96636764160,"available":96608202752,"internally_reserved":0,"allocated":962560,"data_stored":452202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":1.004,"15min":0.98799999999999999},"min":{"1min":0.53400000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.306,"5min":1.516,"15min":1.516},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.026,"15min":0.98499999999999999},"min":{"1min":0.625,"5min":0.55300000000000005,"15min":0.46500000000000002},"max":{"1min":1.3109999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":0.997}]},{"osd":1,"last update":"Wed Oct 11 09:47:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0449999999999999,"15min":0.97199999999999998},"min":{"1min":0.67800000000000005,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.083,"15min":1.0640000000000001},"min":{"1min":0.65200000000000002,"5min":0.60299999999999998,"15min":0.437},"max":{"1min":1.3899999999999999,"5min":1.607,"15min":1.607},"last":1.0469999999999999}]},{"osd":2,"last update":"Wed Oct 11 09:47:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.155,"15min":1.1859999999999999},"min":{"1min":0.76500000000000001,"5min":0.747,"15min":0.73899999999999999},"max":{"1min":1.5589999999999999,"5min":1.6379999999999999,"15min":1.831},"last":1.22},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.177,"15min":1.2210000000000001},"min":{"1min":0.77100000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.4119999999999999,"5min":1.6879999999999999,"15min":1.8819999999999999},"last":1.246}]},{"osd":3,"last update":"Wed Oct 11 09:47:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.1040000000000001,"15min":1.129},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.56999999999999995},"max":{"1min":1.409,"5min":1.8500000000000001,"15min":1.921},"last":0.84999999999999998},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.147,"15min":1.1559999999999999},"min":{"1min":0.71099999999999997,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3700000000000001,"5min":1.5900000000000001,"15min":2.0209999999999999},"last":1.0820000000000001}]},{"osd":4,"last update":"Wed Oct 11 09:47:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.179,"15min":1.1859999999999999},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.72399999999999998},"max":{"1min":1.4570000000000001,"5min":2.117,"15min":2.117},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.194,"5min":1.161,"15min":1.2},"min":{"1min":0.89700000000000002,"5min":0.62,"15min":0.62},"max":{"1min":1.4990000000000001,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":0.71799999999999997}]},{"osd":6,"last update":"Wed Oct 11 09:47:40 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.244,"15min":1.274},"min":{"1min":0.92600000000000005,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.6060000000000001,"5min":1.764,"15min":2.4529999999999998},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3240000000000001,"15min":1.3109999999999999},"min":{"1min":0.80200000000000005,"5min":0.753,"15min":0.753},"max":{"1min":1.663,"5min":1.923,"15min":1.9630000000000001},"last":1.179}]},{"osd":7,"last update":"Wed Oct 11 09:47:40 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.2829999999999999,"15min":1.3799999999999999},"min":{"1min":1.0149999999999999,"5min":0.82399999999999995,"15min":0.57699999999999996},"max":{"1min":1.7090000000000001,"5min":1.9770000000000001,"15min":3.6120000000000001},"last":1.139},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.379,"15min":1.45},"min":{"1min":0.97099999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.712,"5min":1.8220000000000001,"15min":3.2109999999999999},"last":1.2969999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574945,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31988,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":962560,"data_stored":452202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":1.034,"15min":1.054},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.341,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":1.2649999999999999},{"interface":"front","average":{"1min":0.97199999999999998,"5min":1.0609999999999999,"15min":1.089},"min":{"1min":0.72199999999999998,"5min":0.64900000000000002,"15min":0.54700000000000004},"max":{"1min":1.333,"5min":1.5900000000000001,"15min":2.1160000000000001},"last":0.60199999999999998}]},{"osd":1,"last update":"Wed Oct 11 09:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":1.04,"15min":1.079},"min":{"1min":0.66500000000000004,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.2529999999999999,"5min":1.46,"15min":1.46},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.085,"5min":1.109,"15min":1.151},"min":{"1min":0.70599999999999996,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.429,"5min":1.635,"15min":1.8220000000000001},"last":1.03}]},{"osd":2,"last update":"Wed Oct 11 09:47:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1539999999999999,"15min":1.1839999999999999},"min":{"1min":0.75,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.556,"5min":1.556,"15min":1.919},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.1559999999999999,"15min":1.2090000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.8859999999999999},"last":1.1579999999999999}]},{"osd":3,"last update":"Wed Oct 11 09:47:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.089,"15min":1.135},"min":{"1min":0.76000000000000001,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.3580000000000001,"5min":1.3959999999999999,"15min":1.6930000000000001},"last":0.999},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.1379999999999999,"15min":1.2769999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.72599999999999998},"max":{"1min":1.405,"5min":1.532,"15min":2.3340000000000001},"last":1.075}]},{"osd":5,"last update":"Wed Oct 11 09:47:47 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.1379999999999999,"15min":1.202},"min":{"1min":0.80600000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.5129999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.754},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.1479999999999999,"15min":1.2030000000000001},"min":{"1min":0.80300000000000005,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.454,"5min":1.649,"15min":1.8520000000000001},"last":0.68899999999999995}]},{"osd":6,"last update":"Wed Oct 11 09:47:47 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.198,"15min":1.268},"min":{"1min":0.93000000000000005,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.639,"5min":1.639,"15min":1.8},"last":0.88100000000000001},{"interface":"front","average":{"1min":1.306,"5min":1.278,"15min":1.3280000000000001},"min":{"1min":0.95099999999999996,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.694,"5min":1.694,"15min":2.1120000000000001},"last":1.115}]},{"osd":7,"last update":"Wed Oct 11 09:47:47 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.2330000000000001,"15min":1.3280000000000001},"min":{"1min":0.88,"5min":0.73299999999999998,"15min":0.60899999999999999},"max":{"1min":1.595,"5min":1.5960000000000001,"15min":1.9470000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.337,"5min":1.321,"15min":1.4450000000000001},"min":{"1min":1.091,"5min":0.82299999999999995,"15min":0.75900000000000001},"max":{"1min":1.7210000000000001,"5min":1.784,"15min":1.9990000000000001},"last":0.97999999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574944,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":962560,"data_stored":452202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:47:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.70499999999999996,"15min":0.82899999999999996},"min":{"1min":0.499,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.28,"5min":1.28,"15min":1.462},"last":1.038},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.77000000000000002,"15min":0.84599999999999997},"min":{"1min":0.49199999999999999,"5min":0.435,"15min":0.41099999999999998},"max":{"1min":0.96999999999999997,"5min":1.256,"15min":1.4950000000000001},"last":0.73499999999999999}]},{"osd":1,"last update":"Wed Oct 11 09:47:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.78900000000000003,"15min":0.81100000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0149999999999999,"5min":1.335,"15min":1.335},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.871,"15min":0.93200000000000005},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.105,"5min":1.486,"15min":1.486},"last":0.96299999999999997}]},{"osd":2,"last update":"Wed Oct 11 09:47:42 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.93700000000000006,"15min":0.97199999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.2649999999999999,"5min":1.4219999999999999,"15min":1.5389999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.95299999999999996,"15min":1.0840000000000001},"min":{"1min":0.58499999999999996,"5min":0.432,"15min":0.432},"max":{"1min":1.377,"5min":1.444,"15min":2.5870000000000002},"last":1.1200000000000001}]},{"osd":4,"last update":"Wed Oct 11 09:47:42 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.085,"15min":1.1970000000000001},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.5209999999999999,"5min":1.7210000000000001,"15min":2.395},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.163,"15min":1.2849999999999999},"min":{"1min":0.94899999999999995,"5min":0.80900000000000005,"15min":0.73599999999999999},"max":{"1min":1.488,"5min":1.621,"15min":3.1579999999999999},"last":1}]},{"osd":5,"last update":"Wed Oct 11 09:47:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.194,"15min":1.3109999999999999},"min":{"1min":0.84199999999999997,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.5880000000000001,"5min":1.7430000000000001,"15min":3.0009999999999999},"last":1.089},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.198,"15min":1.393},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.67800000000000005},"max":{"1min":1.444,"5min":1.5840000000000001,"15min":3.8050000000000002},"last":1.1819999999999999}]},{"osd":6,"last update":"Wed Oct 11 09:47:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.1839999999999999,"15min":1.2829999999999999},"min":{"1min":0.86899999999999999,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.5740000000000001,"5min":1.663,"15min":2.1949999999999998},"last":1.143},{"interface":"front","average":{"1min":1.236,"5min":1.2509999999999999,"15min":1.377},"min":{"1min":0.97299999999999998,"5min":0.96599999999999997,"15min":0.84499999999999997},"max":{"1min":1.5720000000000001,"5min":1.663,"15min":3.2549999999999999},"last":1.2589999999999999}]},{"osd":7,"last update":"Wed Oct 11 09:47:42 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.262,"15min":1.431},"min":{"1min":0.98099999999999998,"5min":0.89600000000000002,"15min":0.76700000000000002},"max":{"1min":1.5149999999999999,"5min":1.591,"15min":3.7490000000000001},"last":1.222},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.242,"15min":1.365},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.4650000000000001,"5min":1.661,"15min":2.625},"last":1.0660000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574946,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":962560,"data_stored":452202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.77700000000000002,"15min":0.81399999999999995},"min":{"1min":0.52800000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":2.0600000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.79200000000000004,"15min":0.92700000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.2030000000000001,"5min":1.571,"15min":2.8919999999999999},"last":0.75800000000000001}]},{"osd":1,"last update":"Wed Oct 11 09:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.84499999999999997,"15min":1},"min":{"1min":0.52300000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.383,"5min":1.4990000000000001,"15min":3.7650000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.93600000000000005,"15min":0.98099999999999998},"min":{"1min":0.59199999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.675,"5min":1.675,"15min":2.5339999999999998},"last":0.98299999999999998}]},{"osd":3,"last update":"Wed Oct 11 09:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.92400000000000004,"15min":1.0700000000000001},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1240000000000001,"5min":1.4059999999999999,"15min":2.403},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.97199999999999998,"15min":1.056},"min":{"1min":0.70099999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.169,"5min":1.4690000000000001,"15min":2.7589999999999999},"last":0.89600000000000002}]},{"osd":4,"last update":"Wed Oct 11 09:47:41 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.135,"15min":1.204},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":2.2549999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":1.22,"5min":1.2450000000000001,"15min":1.385},"min":{"1min":0.86799999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.464,"5min":1.651,"15min":2.6859999999999999},"last":0.78800000000000003}]},{"osd":5,"last update":"Wed Oct 11 09:47:41 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.212,"15min":1.3320000000000001},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.4450000000000001,"5min":1.6799999999999999,"15min":3.044},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.181,"15min":1.3009999999999999},"min":{"1min":0.72799999999999998,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.837,"5min":1.843,"15min":3.5790000000000002},"last":0.85999999999999999}]},{"osd":6,"last update":"Wed Oct 11 09:47:41 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.1990000000000001,"15min":1.3799999999999999},"min":{"1min":0.78800000000000003,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.494,"5min":1.8049999999999999,"15min":3.1600000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2609999999999999,"15min":1.3460000000000001},"min":{"1min":0.97299999999999998,"5min":0.623,"15min":0.623},"max":{"1min":1.518,"5min":1.806,"15min":2.839},"last":1.1060000000000001}]},{"osd":7,"last update":"Wed Oct 11 09:47:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.248,"15min":1.397},"min":{"1min":0.749,"5min":0.747,"15min":0.747},"max":{"1min":1.726,"5min":1.9079999999999999,"15min":3.1230000000000002},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.335,"5min":1.3400000000000001,"15min":1.488},"min":{"1min":1.0680000000000001,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.6850000000000001,"5min":1.9570000000000001,"15min":2.988},"last":0.879}]}]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":962560,"data_stored":452202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.746,"15min":0.78600000000000003},"min":{"1min":0.52700000000000002,"5min":0.375,"15min":0.375},"max":{"1min":1.109,"5min":1.236,"15min":1.27},"last":1.008},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.77900000000000003,"15min":0.82799999999999996},"min":{"1min":0.51000000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0960000000000001,"5min":1.3160000000000001,"15min":1.3899999999999999},"last":0.70999999999999996}]},{"osd":2,"last update":"Wed Oct 11 09:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79300000000000004,"15min":0.85699999999999998},"min":{"1min":0.437,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.88500000000000001,"15min":0.97999999999999998},"min":{"1min":0.59899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.218,"5min":1.403,"15min":1.6619999999999999},"last":1.252}]},{"osd":3,"last update":"Wed Oct 11 09:47:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":0.98799999999999999,"15min":1.0209999999999999},"min":{"1min":0.57799999999999996,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.3460000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":1.4299999999999999},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.93999999999999995,"15min":0.98099999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.282,"5min":1.544,"15min":1.589},"last":1.1519999999999999}]},{"osd":4,"last update":"Wed Oct 11 09:47:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.0880000000000001,"15min":1.167},"min":{"1min":0.78000000000000003,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.476,"5min":1.482,"15min":2.016},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2,"15min":1.296},"min":{"1min":0.88800000000000001,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":2.323},"last":1.5620000000000001}]},{"osd":5,"last update":"Wed Oct 11 09:47:41 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1950000000000001,"15min":1.302},"min":{"1min":0.86499999999999999,"5min":0.84199999999999997,"15min":0.65800000000000003},"max":{"1min":1.458,"5min":1.7070000000000001,"15min":2.4079999999999999},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.181,"5min":1.1659999999999999,"15min":1.256},"min":{"1min":0.91200000000000003,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":2.1349999999999998},"last":1.677}]},{"osd":6,"last update":"Wed Oct 11 09:47:41 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.22,"15min":1.355},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.71499999999999997},"max":{"1min":1.603,"5min":1.657,"15min":2.8940000000000001},"last":1.619},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.238,"15min":1.355},"min":{"1min":0.98699999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":3.3719999999999999},"last":1.29}]},{"osd":7,"last update":"Wed Oct 11 09:47:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.3049999999999999,"15min":1.4039999999999999},"min":{"1min":0.94299999999999995,"5min":0.89700000000000002,"15min":0.86099999999999999},"max":{"1min":1.7929999999999999,"5min":1.915,"15min":2.194},"last":1.5880000000000001},{"interface":"front","average":{"1min":1.268,"5min":1.3149999999999999,"15min":1.417},"min":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.89600000000000002},"max":{"1min":1.53,"5min":1.7050000000000001,"15min":1.968},"last":1.526}]}]},{"osd":0,"up_from":13,"seq":55834574944,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32568,"kb_used_data":1520,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339272,"statfs":{"total":96636764160,"available":96603414528,"internally_reserved":0,"allocated":1556480,"data_stored":1042570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 11 09:47:33 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.77300000000000002,"15min":0.80500000000000005},"min":{"1min":0.48999999999999999,"5min":0.48599999999999999,"15min":0.38500000000000001},"max":{"1min":0.98599999999999999,"5min":1.1850000000000001,"15min":1.2989999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86499999999999999,"15min":0.92100000000000004},"min":{"1min":0.52600000000000002,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":2.3100000000000001},"last":1.1719999999999999}]},{"osd":2,"last update":"Wed Oct 11 09:47:33 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.89900000000000002,"15min":0.94199999999999995},"min":{"1min":0.53100000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0569999999999999,"5min":1.395,"15min":1.7509999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.876,"5min":0.94099999999999995,"15min":0.96599999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.9450000000000001},"last":0.64100000000000001}]},{"osd":3,"last update":"Wed Oct 11 09:47:33 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.78200000000000003,"15min":0.871},"min":{"1min":0.56000000000000005,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":2.7639999999999998},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.91200000000000003,"15min":1.032},"min":{"1min":0.60899999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.2010000000000001,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":0.85099999999999998}]},{"osd":4,"last update":"Wed Oct 11 09:47:33 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.165,"15min":1.194},"min":{"1min":0.82699999999999996,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.107,"15min":1.204},"min":{"1min":0.88,"5min":0.82299999999999995,"15min":0.69799999999999995},"max":{"1min":1.3089999999999999,"5min":1.5620000000000001,"15min":2.6629999999999998},"last":1.1339999999999999}]},{"osd":5,"last update":"Wed Oct 11 09:47:33 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.202,"15min":1.345},"min":{"1min":0.93700000000000006,"5min":0.878,"15min":0.878},"max":{"1min":1.3109999999999999,"5min":1.734,"15min":3.0150000000000001},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.2110000000000001,"15min":1.3180000000000001},"min":{"1min":0.96199999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"max":{"1min":1.7070000000000001,"5min":1.7070000000000001,"15min":3.601},"last":1.232}]},{"osd":6,"last update":"Wed Oct 11 09:47:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1639999999999999,"15min":1.23},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.78000000000000003},"max":{"1min":1.359,"5min":1.492,"15min":1.825},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.238,"15min":1.333},"min":{"1min":1.028,"5min":0.996,"15min":0.93899999999999995},"max":{"1min":1.4370000000000001,"5min":1.6599999999999999,"15min":2.1440000000000001},"last":1.3380000000000001}]},{"osd":7,"last update":"Wed Oct 11 09:47:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2609999999999999,"15min":1.327},"min":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91400000000000003},"max":{"1min":1.75,"5min":1.75,"15min":1.8759999999999999},"last":1.05},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.226,"15min":1.339},"min":{"1min":0.99399999999999999,"5min":0.92300000000000004,"15min":0.73899999999999999},"max":{"1min":1.482,"5min":1.5349999999999999,"15min":2.5419999999999998},"last":1.196}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-11T09:48:24.336 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-11T09:48:24.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.544+0000 7f1b13045700 1 Processor -- start 2023-10-11T09:48:24.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.544+0000 7f1b13045700 1 -- start start 2023-10-11T09:48:24.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.544+0000 7f1b13045700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1b0c0ce4f0 0x7f1b0c0acff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:24.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.544+0000 7f1b13045700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b0c0cc8e0 con 0x7f1b0c0cf600 2023-10-11T09:48:24.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.544+0000 7f1b13045700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b0c0ad660 con 0x7f1b0c15c960 2023-10-11T09:48:24.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.544+0000 7f1b13045700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b0c15cd30 con 0x7f1b0c0ce4f0 2023-10-11T09:48:24.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.544+0000 7f1b11842700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f1b0c0cf600 0x7f1b0c06fc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:48166/0 (socket says 172.21.15.78:48166) 2023-10-11T09:48:24.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.544+0000 7f1b11842700 1 -- 172.21.15.78:0/688901454 learned_addr learned my addr 172.21.15.78:0/688901454 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:24.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.544+0000 7f1b12844700 1 --2- 172.21.15.78:0/688901454 >> v2:172.21.15.133:3300/0 conn(0x7f1b0c0ce4f0 0x7f1b0c0acff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:24.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.548+0000 7f1b11041700 1 -- 172.21.15.78:0/688901454 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 44971605 0 0) 0x7f1b0c0cc8e0 con 0x7f1b0c0cf600 2023-10-11T09:48:24.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.548+0000 7f1b11041700 1 -- 172.21.15.78:0/688901454 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1af80035d0 con 0x7f1b0c0cf600 2023-10-11T09:48:24.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.548+0000 7f1b11041700 1 -- 172.21.15.78:0/688901454 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1817841601 0 0) 0x7f1b0c0ad660 con 0x7f1b0c15c960 2023-10-11T09:48:24.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.548+0000 7f1b11041700 1 -- 172.21.15.78:0/688901454 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b0c0cc8e0 con 0x7f1b0c15c960 2023-10-11T09:48:24.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.548+0000 7f1b12844700 1 -- 172.21.15.78:0/688901454 >> v1:172.21.15.133:6789/0 conn(0x7f1b0c15c960 legacy=0x7f1b0c0ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:24.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.548+0000 7f1b12844700 1 -- 172.21.15.78:0/688901454 >> v1:172.21.15.78:6789/0 conn(0x7f1b0c0cf600 legacy=0x7f1b0c06fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:24.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.548+0000 7f1b12844700 1 -- 172.21.15.78:0/688901454 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b0c0ad940 con 0x7f1b0c0ce4f0 2023-10-11T09:48:24.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.548+0000 7f1b12844700 1 --2- 172.21.15.78:0/688901454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b0c0ce4f0 0x7f1b0c0acff0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1b04009d90 tx=0x7f1b0400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee2aff4f591741eb server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:24.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.548+0000 7f1b11041700 1 -- 172.21.15.78:0/688901454 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0400e070 con 0x7f1b0c0ce4f0 2023-10-11T09:48:24.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.548+0000 7f1b11041700 1 -- 172.21.15.78:0/688901454 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0400bb20 con 0x7f1b0c0ce4f0 2023-10-11T09:48:24.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.548+0000 7f1b11041700 1 -- 172.21.15.78:0/688901454 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b04012430 con 0x7f1b0c0ce4f0 2023-10-11T09:48:24.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.548+0000 7f1b13045700 1 -- 172.21.15.78:0/688901454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b0c0ce4f0 msgr2=0x7f1b0c0acff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:24.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.548+0000 7f1b13045700 1 --2- 172.21.15.78:0/688901454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b0c0ce4f0 0x7f1b0c0acff0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1b04009d90 tx=0x7f1b0400b330 comp rx=0 tx=0).stop 2023-10-11T09:48:24.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.548+0000 7f1b13045700 1 -- 172.21.15.78:0/688901454 shutdown_connections 2023-10-11T09:48:24.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.548+0000 7f1b13045700 1 --2- 172.21.15.78:0/688901454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b0c0ce4f0 0x7f1b0c0acff0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:24.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.548+0000 7f1b13045700 1 -- 172.21.15.78:0/688901454 >> 172.21.15.78:0/688901454 conn(0x7f1b0c0cc070 msgr2=0x7f1b0c0ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:24.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.548+0000 7f1b13045700 1 -- 172.21.15.78:0/688901454 shutdown_connections 2023-10-11T09:48:24.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.548+0000 7f1b13045700 1 -- 172.21.15.78:0/688901454 wait complete. 2023-10-11T09:48:24.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.552+0000 7f1b13045700 1 Processor -- start 2023-10-11T09:48:24.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.552+0000 7f1b13045700 1 -- start start 2023-10-11T09:48:24.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.552+0000 7f1b13045700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b0c0ce4f0 0x7f1b0c0aff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:24.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.552+0000 7f1b13045700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b0c0cf600 0x7f1b0c0b4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:24.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.552+0000 7f1b13045700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b0c15c960 0x7f1b0c1f1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:24.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.552+0000 7f1b13045700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c0c2b80 con 0x7f1b0c0cf600 2023-10-11T09:48:24.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.556+0000 7f1b13045700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c0b0870 con 0x7f1b0c0ce4f0 2023-10-11T09:48:24.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.556+0000 7f1b13045700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c0b45d0 con 0x7f1b0c15c960 2023-10-11T09:48:24.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.556+0000 7f1b11842700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b0c0cf600 0x7f1b0c0b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:24.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.556+0000 7f1b11842700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b0c0cf600 0x7f1b0c0b4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57332/0 (socket says 172.21.15.78:57332) 2023-10-11T09:48:24.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.556+0000 7f1b11842700 1 -- 172.21.15.78:0/1190229162 learned_addr learned my addr 172.21.15.78:0/1190229162 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:24.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.556+0000 7f1b12043700 1 --2- 172.21.15.78:0/1190229162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b0c0ce4f0 0x7f1b0c0aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:24.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.556+0000 7f1b12844700 1 --2- 172.21.15.78:0/1190229162 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b0c15c960 0x7f1b0c1f1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:24.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.556+0000 7f1b11842700 1 -- 172.21.15.78:0/1190229162 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b0c15c960 msgr2=0x7f1b0c1f1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:24.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.556+0000 7f1b11842700 1 --2- 172.21.15.78:0/1190229162 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b0c15c960 0x7f1b0c1f1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:24.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.556+0000 7f1b11842700 1 -- 172.21.15.78:0/1190229162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b0c0ce4f0 msgr2=0x7f1b0c0aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:24.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.556+0000 7f1b11842700 1 --2- 172.21.15.78:0/1190229162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b0c0ce4f0 0x7f1b0c0aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:24.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.556+0000 7f1b11842700 1 -- 172.21.15.78:0/1190229162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b04009a70 con 0x7f1b0c0cf600 2023-10-11T09:48:24.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.556+0000 7f1b12844700 1 --2- 172.21.15.78:0/1190229162 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b0c15c960 0x7f1b0c1f1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:48:24.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.556+0000 7f1b11842700 1 --2- 172.21.15.78:0/1190229162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b0c0cf600 0x7f1b0c0b4090 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f1afc00c530 tx=0x7f1afc00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:24.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.556+0000 7f1b02ffd700 1 -- 172.21.15.78:0/1190229162 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc005e50 con 0x7f1b0c0cf600 2023-10-11T09:48:24.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.556+0000 7f1b02ffd700 1 -- 172.21.15.78:0/1190229162 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1afc0070d0 con 0x7f1b0c0cf600 2023-10-11T09:48:24.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.556+0000 7f1b13045700 1 -- 172.21.15.78:0/1190229162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b0c0b0590 con 0x7f1b0c0cf600 2023-10-11T09:48:24.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.556+0000 7f1b02ffd700 1 -- 172.21.15.78:0/1190229162 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc007460 con 0x7f1b0c0cf600 2023-10-11T09:48:24.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.556+0000 7f1b13045700 1 -- 172.21.15.78:0/1190229162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b0c0c27a0 con 0x7f1b0c0cf600 2023-10-11T09:48:24.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.556+0000 7f1b13045700 1 -- 172.21.15.78:0/1190229162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b0c0c2cc0 con 0x7f1b0c0cf600 2023-10-11T09:48:24.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.560+0000 7f1b02ffd700 1 -- 172.21.15.78:0/1190229162 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f1afc0075c0 con 0x7f1b0c0cf600 2023-10-11T09:48:24.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.560+0000 7f1b02ffd700 1 --2- 172.21.15.78:0/1190229162 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1af0065ed0 0x7f1af0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:24.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.560+0000 7f1b02ffd700 1 -- 172.21.15.78:0/1190229162 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1afc082db0 con 0x7f1b0c0cf600 2023-10-11T09:48:24.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.560+0000 7f1b12043700 1 --2- 172.21.15.78:0/1190229162 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1af0065ed0 0x7f1af0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:24.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.564+0000 7f1b12043700 1 --2- 172.21.15.78:0/1190229162 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1af0065ed0 0x7f1af0068380 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1b080011e0 tx=0x7f1b08009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:24.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.568+0000 7f1b02ffd700 1 -- 172.21.15.78:0/1190229162 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f1b0c0c2cc0 con 0x7f1b0c0cf600 2023-10-11T09:48:24.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.736+0000 7f1b13045700 1 -- 172.21.15.78:0/1190229162 --> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1b0c0cdf60 con 0x7f1af0065ed0 2023-10-11T09:48:24.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.752+0000 7f1b02ffd700 1 -- 172.21.15.78:0/1190229162 <== mgr.4101 v2:172.21.15.78:6832/35606 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308963 (secure 0 0 0) 0x7f1b08001cf0 con 0x7f1af0065ed0 2023-10-11T09:48:24.761 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:24.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.760+0000 7f1b13045700 1 -- 172.21.15.78:0/1190229162 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1af0065ed0 msgr2=0x7f1af0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:24.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.760+0000 7f1b13045700 1 --2- 172.21.15.78:0/1190229162 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1af0065ed0 0x7f1af0068380 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1b080011e0 tx=0x7f1b08009040 comp rx=0 tx=0).stop 2023-10-11T09:48:24.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.760+0000 7f1b13045700 1 -- 172.21.15.78:0/1190229162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b0c0cf600 msgr2=0x7f1b0c0b4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:24.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.760+0000 7f1b13045700 1 --2- 172.21.15.78:0/1190229162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b0c0cf600 0x7f1b0c0b4090 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f1afc00c530 tx=0x7f1afc00c840 comp rx=0 tx=0).stop 2023-10-11T09:48:24.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.760+0000 7f1b13045700 1 -- 172.21.15.78:0/1190229162 shutdown_connections 2023-10-11T09:48:24.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.760+0000 7f1b13045700 1 --2- 172.21.15.78:0/1190229162 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1af0065ed0 0x7f1af0068380 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:24.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.760+0000 7f1b13045700 1 --2- 172.21.15.78:0/1190229162 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b0c15c960 0x7f1b0c1f1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:24.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.760+0000 7f1b13045700 1 --2- 172.21.15.78:0/1190229162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b0c0cf600 0x7f1b0c0b4090 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:24.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.760+0000 7f1b13045700 1 --2- 172.21.15.78:0/1190229162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b0c0ce4f0 0x7f1b0c0aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:24.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.760+0000 7f1b13045700 1 -- 172.21.15.78:0/1190229162 >> 172.21.15.78:0/1190229162 conn(0x7f1b0c0cc070 msgr2=0x7f1b0c0b1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:24.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.760+0000 7f1b13045700 1 -- 172.21.15.78:0/1190229162 shutdown_connections 2023-10-11T09:48:24.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:24.760+0000 7f1b13045700 1 -- 172.21.15.78:0/1190229162 wait complete. 2023-10-11T09:48:24.772 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2023-10-11T09:48:24.790 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2023-10-11T09:48:24.488094+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":205,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257512,"kb_used_data":9128,"kb_used_omap":0,"kb_used_meta":248320,"kb_avail":754717208,"statfs":{"total":773094113280,"available":772830420992,"internally_reserved":0,"allocated":9347072,"data_stored":5015708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":254279680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.942582"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.047408+0000","last_change":"2023-10-11T09:48:20.061620+0000","last_active":"2023-10-11T09:48:21.047408+0000","last_peered":"2023-10-11T09:48:21.047408+0000","last_clean":"2023-10-11T09:48:21.047408+0000","last_became_active":"2023-10-11T09:48:20.059781+0000","last_became_peered":"2023-10-11T09:48:20.059781+0000","last_unstale":"2023-10-11T09:48:21.047408+0000","last_undegraded":"2023-10-11T09:48:21.047408+0000","last_fullsized":"2023-10-11T09:48:21.047408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:43:21.058457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.135236+0000","last_change":"2023-10-11T09:48:20.049344+0000","last_active":"2023-10-11T09:48:22.135236+0000","last_peered":"2023-10-11T09:48:22.135236+0000","last_clean":"2023-10-11T09:48:22.135236+0000","last_became_active":"2023-10-11T09:48:20.049114+0000","last_became_peered":"2023-10-11T09:48:20.049114+0000","last_unstale":"2023-10-11T09:48:22.135236+0000","last_undegraded":"2023-10-11T09:48:22.135236+0000","last_fullsized":"2023-10-11T09:48:22.135236+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:48:23.600595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.054087+0000","last_change":"2023-10-11T09:48:20.057873+0000","last_active":"2023-10-11T09:48:22.054087+0000","last_peered":"2023-10-11T09:48:22.054087+0000","last_clean":"2023-10-11T09:48:22.054087+0000","last_became_active":"2023-10-11T09:48:20.057639+0000","last_became_peered":"2023-10-11T09:48:20.057639+0000","last_unstale":"2023-10-11T09:48:22.054087+0000","last_undegraded":"2023-10-11T09:48:22.054087+0000","last_fullsized":"2023-10-11T09:48:22.054087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:16:08.127920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.136947+0000","last_change":"2023-10-11T09:48:20.048154+0000","last_active":"2023-10-11T09:48:22.136947+0000","last_peered":"2023-10-11T09:48:22.136947+0000","last_clean":"2023-10-11T09:48:22.136947+0000","last_became_active":"2023-10-11T09:48:20.047861+0000","last_became_peered":"2023-10-11T09:48:20.047861+0000","last_unstale":"2023-10-11T09:48:22.136947+0000","last_undegraded":"2023-10-11T09:48:22.136947+0000","last_fullsized":"2023-10-11T09:48:22.136947+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:20:01.062840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.536596+0000","last_change":"2023-10-11T09:48:20.057685+0000","last_active":"2023-10-11T09:48:21.536596+0000","last_peered":"2023-10-11T09:48:21.536596+0000","last_clean":"2023-10-11T09:48:21.536596+0000","last_became_active":"2023-10-11T09:48:20.057438+0000","last_became_peered":"2023-10-11T09:48:20.057438+0000","last_unstale":"2023-10-11T09:48:21.536596+0000","last_undegraded":"2023-10-11T09:48:21.536596+0000","last_fullsized":"2023-10-11T09:48:21.536596+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:04:48.144323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.043488+0000","last_change":"2023-10-11T09:48:20.061599+0000","last_active":"2023-10-11T09:48:21.043488+0000","last_peered":"2023-10-11T09:48:21.043488+0000","last_clean":"2023-10-11T09:48:21.043488+0000","last_became_active":"2023-10-11T09:48:20.061336+0000","last_became_peered":"2023-10-11T09:48:20.061336+0000","last_unstale":"2023-10-11T09:48:21.043488+0000","last_undegraded":"2023-10-11T09:48:21.043488+0000","last_fullsized":"2023-10-11T09:48:21.043488+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:48:31.632403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.137520+0000","last_change":"2023-10-11T09:48:20.055891+0000","last_active":"2023-10-11T09:48:22.137520+0000","last_peered":"2023-10-11T09:48:22.137520+0000","last_clean":"2023-10-11T09:48:22.137520+0000","last_became_active":"2023-10-11T09:48:20.055569+0000","last_became_peered":"2023-10-11T09:48:20.055569+0000","last_unstale":"2023-10-11T09:48:22.137520+0000","last_undegraded":"2023-10-11T09:48:22.137520+0000","last_fullsized":"2023-10-11T09:48:22.137520+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:14:16.684501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.536001+0000","last_change":"2023-10-11T09:48:20.053085+0000","last_active":"2023-10-11T09:48:21.536001+0000","last_peered":"2023-10-11T09:48:21.536001+0000","last_clean":"2023-10-11T09:48:21.536001+0000","last_became_active":"2023-10-11T09:48:20.052830+0000","last_became_peered":"2023-10-11T09:48:20.052830+0000","last_unstale":"2023-10-11T09:48:21.536001+0000","last_undegraded":"2023-10-11T09:48:21.536001+0000","last_fullsized":"2023-10-11T09:48:21.536001+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:04:09.959390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.047298+0000","last_change":"2023-10-11T09:48:20.054724+0000","last_active":"2023-10-11T09:48:21.047298+0000","last_peered":"2023-10-11T09:48:21.047298+0000","last_clean":"2023-10-11T09:48:21.047298+0000","last_became_active":"2023-10-11T09:48:20.054427+0000","last_became_peered":"2023-10-11T09:48:20.054427+0000","last_unstale":"2023-10-11T09:48:21.047298+0000","last_undegraded":"2023-10-11T09:48:21.047298+0000","last_fullsized":"2023-10-11T09:48:21.047298+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:40:23.432848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.136807+0000","last_change":"2023-10-11T09:48:20.062025+0000","last_active":"2023-10-11T09:48:22.136807+0000","last_peered":"2023-10-11T09:48:22.136807+0000","last_clean":"2023-10-11T09:48:22.136807+0000","last_became_active":"2023-10-11T09:48:20.061649+0000","last_became_peered":"2023-10-11T09:48:20.061649+0000","last_unstale":"2023-10-11T09:48:22.136807+0000","last_undegraded":"2023-10-11T09:48:22.136807+0000","last_fullsized":"2023-10-11T09:48:22.136807+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:00:34.688895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.134818+0000","last_change":"2023-10-11T09:48:20.048308+0000","last_active":"2023-10-11T09:48:22.134818+0000","last_peered":"2023-10-11T09:48:22.134818+0000","last_clean":"2023-10-11T09:48:22.134818+0000","last_became_active":"2023-10-11T09:48:20.048039+0000","last_became_peered":"2023-10-11T09:48:20.048039+0000","last_unstale":"2023-10-11T09:48:22.134818+0000","last_undegraded":"2023-10-11T09:48:22.134818+0000","last_fullsized":"2023-10-11T09:48:22.134818+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:42:29.468628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.137178+0000","last_change":"2023-10-11T09:48:20.056504+0000","last_active":"2023-10-11T09:48:22.137178+0000","last_peered":"2023-10-11T09:48:22.137178+0000","last_clean":"2023-10-11T09:48:22.137178+0000","last_became_active":"2023-10-11T09:48:20.056238+0000","last_became_peered":"2023-10-11T09:48:20.056238+0000","last_unstale":"2023-10-11T09:48:22.137178+0000","last_undegraded":"2023-10-11T09:48:22.137178+0000","last_fullsized":"2023-10-11T09:48:22.137178+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:49:39.448060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.536575+0000","last_change":"2023-10-11T09:48:20.062853+0000","last_active":"2023-10-11T09:48:21.536575+0000","last_peered":"2023-10-11T09:48:21.536575+0000","last_clean":"2023-10-11T09:48:21.536575+0000","last_became_active":"2023-10-11T09:48:20.062587+0000","last_became_peered":"2023-10-11T09:48:20.062587+0000","last_unstale":"2023-10-11T09:48:21.536575+0000","last_undegraded":"2023-10-11T09:48:21.536575+0000","last_fullsized":"2023-10-11T09:48:21.536575+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:21:58.739423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.043447+0000","last_change":"2023-10-11T09:48:20.060851+0000","last_active":"2023-10-11T09:48:21.043447+0000","last_peered":"2023-10-11T09:48:21.043447+0000","last_clean":"2023-10-11T09:48:21.043447+0000","last_became_active":"2023-10-11T09:48:20.060585+0000","last_became_peered":"2023-10-11T09:48:20.060585+0000","last_unstale":"2023-10-11T09:48:21.043447+0000","last_undegraded":"2023-10-11T09:48:21.043447+0000","last_fullsized":"2023-10-11T09:48:21.043447+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:43:21.058457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.134344+0000","last_change":"2023-10-11T09:48:20.050372+0000","last_active":"2023-10-11T09:48:22.134344+0000","last_peered":"2023-10-11T09:48:22.134344+0000","last_clean":"2023-10-11T09:48:22.134344+0000","last_became_active":"2023-10-11T09:48:20.050101+0000","last_became_peered":"2023-10-11T09:48:20.050101+0000","last_unstale":"2023-10-11T09:48:22.134344+0000","last_undegraded":"2023-10-11T09:48:22.134344+0000","last_fullsized":"2023-10-11T09:48:22.134344+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:42:07.010445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.045948+0000","last_change":"2023-10-11T09:48:20.051413+0000","last_active":"2023-10-11T09:48:21.045948+0000","last_peered":"2023-10-11T09:48:21.045948+0000","last_clean":"2023-10-11T09:48:21.045948+0000","last_became_active":"2023-10-11T09:48:20.051117+0000","last_became_peered":"2023-10-11T09:48:20.051117+0000","last_unstale":"2023-10-11T09:48:21.045948+0000","last_undegraded":"2023-10-11T09:48:21.045948+0000","last_fullsized":"2023-10-11T09:48:21.045948+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:20:15.431729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.536918+0000","last_change":"2023-10-11T09:48:20.064427+0000","last_active":"2023-10-11T09:48:21.536918+0000","last_peered":"2023-10-11T09:48:21.536918+0000","last_clean":"2023-10-11T09:48:21.536918+0000","last_became_active":"2023-10-11T09:48:20.063738+0000","last_became_peered":"2023-10-11T09:48:20.063738+0000","last_unstale":"2023-10-11T09:48:21.536918+0000","last_undegraded":"2023-10-11T09:48:21.536918+0000","last_fullsized":"2023-10-11T09:48:21.536918+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:39:09.258068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.054037+0000","last_change":"2023-10-11T09:48:20.059642+0000","last_active":"2023-10-11T09:48:22.054037+0000","last_peered":"2023-10-11T09:48:22.054037+0000","last_clean":"2023-10-11T09:48:22.054037+0000","last_became_active":"2023-10-11T09:48:20.059355+0000","last_became_peered":"2023-10-11T09:48:20.059355+0000","last_unstale":"2023-10-11T09:48:22.054037+0000","last_undegraded":"2023-10-11T09:48:22.054037+0000","last_fullsized":"2023-10-11T09:48:22.054037+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:54:27.307834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.043383+0000","last_change":"2023-10-11T09:48:20.061003+0000","last_active":"2023-10-11T09:48:21.043383+0000","last_peered":"2023-10-11T09:48:21.043383+0000","last_clean":"2023-10-11T09:48:21.043383+0000","last_became_active":"2023-10-11T09:48:20.060678+0000","last_became_peered":"2023-10-11T09:48:20.060678+0000","last_unstale":"2023-10-11T09:48:21.043383+0000","last_undegraded":"2023-10-11T09:48:21.043383+0000","last_fullsized":"2023-10-11T09:48:21.043383+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:14:28.773445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.536754+0000","last_change":"2023-10-11T09:48:20.048847+0000","last_active":"2023-10-11T09:48:21.536754+0000","last_peered":"2023-10-11T09:48:21.536754+0000","last_clean":"2023-10-11T09:48:21.536754+0000","last_became_active":"2023-10-11T09:48:20.048614+0000","last_became_peered":"2023-10-11T09:48:20.048614+0000","last_unstale":"2023-10-11T09:48:21.536754+0000","last_undegraded":"2023-10-11T09:48:21.536754+0000","last_fullsized":"2023-10-11T09:48:21.536754+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:28:35.667386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.046957+0000","last_change":"2023-10-11T09:48:20.049954+0000","last_active":"2023-10-11T09:48:21.046957+0000","last_peered":"2023-10-11T09:48:21.046957+0000","last_clean":"2023-10-11T09:48:21.046957+0000","last_became_active":"2023-10-11T09:48:20.049545+0000","last_became_peered":"2023-10-11T09:48:20.049545+0000","last_unstale":"2023-10-11T09:48:21.046957+0000","last_undegraded":"2023-10-11T09:48:21.046957+0000","last_fullsized":"2023-10-11T09:48:21.046957+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:17:12.933706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.536901+0000","last_change":"2023-10-11T09:48:20.067309+0000","last_active":"2023-10-11T09:48:21.536901+0000","last_peered":"2023-10-11T09:48:21.536901+0000","last_clean":"2023-10-11T09:48:21.536901+0000","last_became_active":"2023-10-11T09:48:20.067042+0000","last_became_peered":"2023-10-11T09:48:20.067042+0000","last_unstale":"2023-10-11T09:48:21.536901+0000","last_undegraded":"2023-10-11T09:48:21.536901+0000","last_fullsized":"2023-10-11T09:48:21.536901+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:51:05.341212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.046488+0000","last_change":"2023-10-11T09:48:20.051434+0000","last_active":"2023-10-11T09:48:21.046488+0000","last_peered":"2023-10-11T09:48:21.046488+0000","last_clean":"2023-10-11T09:48:21.046488+0000","last_became_active":"2023-10-11T09:48:20.050896+0000","last_became_peered":"2023-10-11T09:48:20.050896+0000","last_unstale":"2023-10-11T09:48:21.046488+0000","last_undegraded":"2023-10-11T09:48:21.046488+0000","last_fullsized":"2023-10-11T09:48:21.046488+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:43:21.058457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.134207+0000","last_change":"2023-10-11T09:48:20.046333+0000","last_active":"2023-10-11T09:48:22.134207+0000","last_peered":"2023-10-11T09:48:22.134207+0000","last_clean":"2023-10-11T09:48:22.134207+0000","last_became_active":"2023-10-11T09:48:20.046091+0000","last_became_peered":"2023-10-11T09:48:20.046091+0000","last_unstale":"2023-10-11T09:48:22.134207+0000","last_undegraded":"2023-10-11T09:48:22.134207+0000","last_fullsized":"2023-10-11T09:48:22.134207+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:29:51.829530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.135113+0000","last_change":"2023-10-11T09:48:20.047093+0000","last_active":"2023-10-11T09:48:22.135113+0000","last_peered":"2023-10-11T09:48:22.135113+0000","last_clean":"2023-10-11T09:48:22.135113+0000","last_became_active":"2023-10-11T09:48:20.046860+0000","last_became_peered":"2023-10-11T09:48:20.046860+0000","last_unstale":"2023-10-11T09:48:22.135113+0000","last_undegraded":"2023-10-11T09:48:22.135113+0000","last_fullsized":"2023-10-11T09:48:22.135113+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:58:41.596598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.137007+0000","last_change":"2023-10-11T09:48:20.070270+0000","last_active":"2023-10-11T09:48:22.137007+0000","last_peered":"2023-10-11T09:48:22.137007+0000","last_clean":"2023-10-11T09:48:22.137007+0000","last_became_active":"2023-10-11T09:48:20.069954+0000","last_became_peered":"2023-10-11T09:48:20.069954+0000","last_unstale":"2023-10-11T09:48:22.137007+0000","last_undegraded":"2023-10-11T09:48:22.137007+0000","last_fullsized":"2023-10-11T09:48:22.137007+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:49:24.914673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.047111+0000","last_change":"2023-10-11T09:48:20.052119+0000","last_active":"2023-10-11T09:48:21.047111+0000","last_peered":"2023-10-11T09:48:21.047111+0000","last_clean":"2023-10-11T09:48:21.047111+0000","last_became_active":"2023-10-11T09:48:20.051526+0000","last_became_peered":"2023-10-11T09:48:20.051526+0000","last_unstale":"2023-10-11T09:48:21.047111+0000","last_undegraded":"2023-10-11T09:48:21.047111+0000","last_fullsized":"2023-10-11T09:48:21.047111+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:47:43.381718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.536660+0000","last_change":"2023-10-11T09:48:20.054097+0000","last_active":"2023-10-11T09:48:21.536660+0000","last_peered":"2023-10-11T09:48:21.536660+0000","last_clean":"2023-10-11T09:48:21.536660+0000","last_became_active":"2023-10-11T09:48:20.053793+0000","last_became_peered":"2023-10-11T09:48:20.053793+0000","last_unstale":"2023-10-11T09:48:21.536660+0000","last_undegraded":"2023-10-11T09:48:21.536660+0000","last_fullsized":"2023-10-11T09:48:21.536660+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:35:26.753917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.134929+0000","last_change":"2023-10-11T09:48:20.062569+0000","last_active":"2023-10-11T09:48:22.134929+0000","last_peered":"2023-10-11T09:48:22.134929+0000","last_clean":"2023-10-11T09:48:22.134929+0000","last_became_active":"2023-10-11T09:48:20.062313+0000","last_became_peered":"2023-10-11T09:48:20.062313+0000","last_unstale":"2023-10-11T09:48:22.134929+0000","last_undegraded":"2023-10-11T09:48:22.134929+0000","last_fullsized":"2023-10-11T09:48:22.134929+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:05:25.940628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.046410+0000","last_change":"2023-10-11T09:48:20.057398+0000","last_active":"2023-10-11T09:48:21.046410+0000","last_peered":"2023-10-11T09:48:21.046410+0000","last_clean":"2023-10-11T09:48:21.046410+0000","last_became_active":"2023-10-11T09:48:20.057138+0000","last_became_peered":"2023-10-11T09:48:20.057138+0000","last_unstale":"2023-10-11T09:48:21.046410+0000","last_undegraded":"2023-10-11T09:48:21.046410+0000","last_fullsized":"2023-10-11T09:48:21.046410+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:48:31.632403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.053610+0000","last_change":"2023-10-11T09:48:20.064561+0000","last_active":"2023-10-11T09:48:22.053610+0000","last_peered":"2023-10-11T09:48:22.053610+0000","last_clean":"2023-10-11T09:48:22.053610+0000","last_became_active":"2023-10-11T09:48:20.063806+0000","last_became_peered":"2023-10-11T09:48:20.063806+0000","last_unstale":"2023-10-11T09:48:22.053610+0000","last_undegraded":"2023-10-11T09:48:22.053610+0000","last_fullsized":"2023-10-11T09:48:22.053610+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:46:26.451162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.045929+0000","last_change":"2023-10-11T09:48:20.059745+0000","last_active":"2023-10-11T09:48:21.045929+0000","last_peered":"2023-10-11T09:48:21.045929+0000","last_clean":"2023-10-11T09:48:21.045929+0000","last_became_active":"2023-10-11T09:48:20.059503+0000","last_became_peered":"2023-10-11T09:48:20.059503+0000","last_unstale":"2023-10-11T09:48:21.045929+0000","last_undegraded":"2023-10-11T09:48:21.045929+0000","last_fullsized":"2023-10-11T09:48:21.045929+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:17:12.933706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.053594+0000","last_change":"2023-10-11T09:48:20.054303+0000","last_active":"2023-10-11T09:48:22.053594+0000","last_peered":"2023-10-11T09:48:22.053594+0000","last_clean":"2023-10-11T09:48:22.053594+0000","last_became_active":"2023-10-11T09:48:20.054030+0000","last_became_peered":"2023-10-11T09:48:20.054030+0000","last_unstale":"2023-10-11T09:48:22.053594+0000","last_undegraded":"2023-10-11T09:48:22.053594+0000","last_fullsized":"2023-10-11T09:48:22.053594+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:26:14.797995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"82'4","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.536936+0000","last_change":"2023-10-11T09:48:19.579178+0000","last_active":"2023-10-11T09:48:21.536936+0000","last_peered":"2023-10-11T09:48:21.536936+0000","last_clean":"2023-10-11T09:48:21.536936+0000","last_became_active":"2023-10-11T09:48:19.578915+0000","last_became_peered":"2023-10-11T09:48:19.578915+0000","last_unstale":"2023-10-11T09:48:21.536936+0000","last_undegraded":"2023-10-11T09:48:21.536936+0000","last_fullsized":"2023-10-11T09:48:21.536936+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:02:27.507873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.134328+0000","last_change":"2023-10-11T09:48:20.054505+0000","last_active":"2023-10-11T09:48:22.134328+0000","last_peered":"2023-10-11T09:48:22.134328+0000","last_clean":"2023-10-11T09:48:22.134328+0000","last_became_active":"2023-10-11T09:48:20.054238+0000","last_became_peered":"2023-10-11T09:48:20.054238+0000","last_unstale":"2023-10-11T09:48:22.134328+0000","last_undegraded":"2023-10-11T09:48:22.134328+0000","last_fullsized":"2023-10-11T09:48:22.134328+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:18:08.594482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"82'1","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.047295+0000","last_change":"2023-10-11T09:48:19.579252+0000","last_active":"2023-10-11T09:48:21.047295+0000","last_peered":"2023-10-11T09:48:21.047295+0000","last_clean":"2023-10-11T09:48:21.047295+0000","last_became_active":"2023-10-11T09:48:19.578590+0000","last_became_peered":"2023-10-11T09:48:19.578590+0000","last_unstale":"2023-10-11T09:48:21.047295+0000","last_undegraded":"2023-10-11T09:48:21.047295+0000","last_fullsized":"2023-10-11T09:48:21.047295+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:48:40.989918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.046487+0000","last_change":"2023-10-11T09:48:20.064581+0000","last_active":"2023-10-11T09:48:21.046487+0000","last_peered":"2023-10-11T09:48:21.046487+0000","last_clean":"2023-10-11T09:48:21.046487+0000","last_became_active":"2023-10-11T09:48:20.063856+0000","last_became_peered":"2023-10-11T09:48:20.063856+0000","last_unstale":"2023-10-11T09:48:21.046487+0000","last_undegraded":"2023-10-11T09:48:21.046487+0000","last_fullsized":"2023-10-11T09:48:21.046487+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:40:37.192929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"82'2","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.047113+0000","last_change":"2023-10-11T09:48:19.573361+0000","last_active":"2023-10-11T09:48:21.047113+0000","last_peered":"2023-10-11T09:48:21.047113+0000","last_clean":"2023-10-11T09:48:21.047113+0000","last_became_active":"2023-10-11T09:48:19.573065+0000","last_became_peered":"2023-10-11T09:48:19.573065+0000","last_unstale":"2023-10-11T09:48:21.047113+0000","last_undegraded":"2023-10-11T09:48:21.047113+0000","last_fullsized":"2023-10-11T09:48:21.047113+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:31:43.945622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.046777+0000","last_change":"2023-10-11T09:48:20.054354+0000","last_active":"2023-10-11T09:48:21.046777+0000","last_peered":"2023-10-11T09:48:21.046777+0000","last_clean":"2023-10-11T09:48:21.046777+0000","last_became_active":"2023-10-11T09:48:20.053992+0000","last_became_peered":"2023-10-11T09:48:20.053992+0000","last_unstale":"2023-10-11T09:48:21.046777+0000","last_undegraded":"2023-10-11T09:48:21.046777+0000","last_fullsized":"2023-10-11T09:48:21.046777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:45:38.018297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"82'2","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.138120+0000","last_change":"2023-10-11T09:48:19.577409+0000","last_active":"2023-10-11T09:48:22.138120+0000","last_peered":"2023-10-11T09:48:22.138120+0000","last_clean":"2023-10-11T09:48:22.138120+0000","last_became_active":"2023-10-11T09:48:19.576818+0000","last_became_peered":"2023-10-11T09:48:19.576818+0000","last_unstale":"2023-10-11T09:48:22.138120+0000","last_undegraded":"2023-10-11T09:48:22.138120+0000","last_fullsized":"2023-10-11T09:48:22.138120+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:33:37.037919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.134918+0000","last_change":"2023-10-11T09:48:20.048810+0000","last_active":"2023-10-11T09:48:22.134918+0000","last_peered":"2023-10-11T09:48:22.134918+0000","last_clean":"2023-10-11T09:48:22.134918+0000","last_became_active":"2023-10-11T09:48:20.048459+0000","last_became_peered":"2023-10-11T09:48:20.048459+0000","last_unstale":"2023-10-11T09:48:22.134918+0000","last_undegraded":"2023-10-11T09:48:22.134918+0000","last_fullsized":"2023-10-11T09:48:22.134918+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:15:34.109824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"82'1","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.046919+0000","last_change":"2023-10-11T09:48:19.574151+0000","last_active":"2023-10-11T09:48:21.046919+0000","last_peered":"2023-10-11T09:48:21.046919+0000","last_clean":"2023-10-11T09:48:21.046919+0000","last_became_active":"2023-10-11T09:48:19.573887+0000","last_became_peered":"2023-10-11T09:48:19.573887+0000","last_unstale":"2023-10-11T09:48:21.046919+0000","last_undegraded":"2023-10-11T09:48:21.046919+0000","last_fullsized":"2023-10-11T09:48:21.046919+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:57:57.259949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.134954+0000","last_change":"2023-10-11T09:48:20.048174+0000","last_active":"2023-10-11T09:48:22.134954+0000","last_peered":"2023-10-11T09:48:22.134954+0000","last_clean":"2023-10-11T09:48:22.134954+0000","last_became_active":"2023-10-11T09:48:20.047884+0000","last_became_peered":"2023-10-11T09:48:20.047884+0000","last_unstale":"2023-10-11T09:48:22.134954+0000","last_undegraded":"2023-10-11T09:48:22.134954+0000","last_fullsized":"2023-10-11T09:48:22.134954+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:41:12.945778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"82'1","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.136516+0000","last_change":"2023-10-11T09:48:19.573108+0000","last_active":"2023-10-11T09:48:22.136516+0000","last_peered":"2023-10-11T09:48:22.136516+0000","last_clean":"2023-10-11T09:48:22.136516+0000","last_became_active":"2023-10-11T09:48:19.572789+0000","last_became_peered":"2023-10-11T09:48:19.572789+0000","last_unstale":"2023-10-11T09:48:22.136516+0000","last_undegraded":"2023-10-11T09:48:22.136516+0000","last_fullsized":"2023-10-11T09:48:22.136516+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:40:36.181226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.047116+0000","last_change":"2023-10-11T09:48:20.056920+0000","last_active":"2023-10-11T09:48:21.047116+0000","last_peered":"2023-10-11T09:48:21.047116+0000","last_clean":"2023-10-11T09:48:21.047116+0000","last_became_active":"2023-10-11T09:48:20.056400+0000","last_became_peered":"2023-10-11T09:48:20.056400+0000","last_unstale":"2023-10-11T09:48:21.047116+0000","last_undegraded":"2023-10-11T09:48:21.047116+0000","last_fullsized":"2023-10-11T09:48:21.047116+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:19:47.840158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.134439+0000","last_change":"2023-10-11T09:48:19.572106+0000","last_active":"2023-10-11T09:48:22.134439+0000","last_peered":"2023-10-11T09:48:22.134439+0000","last_clean":"2023-10-11T09:48:22.134439+0000","last_became_active":"2023-10-11T09:48:19.571850+0000","last_became_peered":"2023-10-11T09:48:19.571850+0000","last_unstale":"2023-10-11T09:48:22.134439+0000","last_undegraded":"2023-10-11T09:48:22.134439+0000","last_fullsized":"2023-10-11T09:48:22.134439+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:37:36.623392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.138450+0000","last_change":"2023-10-11T09:48:20.065172+0000","last_active":"2023-10-11T09:48:22.138450+0000","last_peered":"2023-10-11T09:48:22.138450+0000","last_clean":"2023-10-11T09:48:22.138450+0000","last_became_active":"2023-10-11T09:48:20.064882+0000","last_became_peered":"2023-10-11T09:48:20.064882+0000","last_unstale":"2023-10-11T09:48:22.138450+0000","last_undegraded":"2023-10-11T09:48:22.138450+0000","last_fullsized":"2023-10-11T09:48:22.138450+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:25:13.916751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.536050+0000","last_change":"2023-10-11T09:48:19.578411+0000","last_active":"2023-10-11T09:48:21.536050+0000","last_peered":"2023-10-11T09:48:21.536050+0000","last_clean":"2023-10-11T09:48:21.536050+0000","last_became_active":"2023-10-11T09:48:19.578145+0000","last_became_peered":"2023-10-11T09:48:19.578145+0000","last_unstale":"2023-10-11T09:48:21.536050+0000","last_undegraded":"2023-10-11T09:48:21.536050+0000","last_fullsized":"2023-10-11T09:48:21.536050+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:22:29.039499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.134519+0000","last_change":"2023-10-11T09:48:20.055127+0000","last_active":"2023-10-11T09:48:22.134519+0000","last_peered":"2023-10-11T09:48:22.134519+0000","last_clean":"2023-10-11T09:48:22.134519+0000","last_became_active":"2023-10-11T09:48:20.054863+0000","last_became_peered":"2023-10-11T09:48:20.054863+0000","last_unstale":"2023-10-11T09:48:22.134519+0000","last_undegraded":"2023-10-11T09:48:22.134519+0000","last_fullsized":"2023-10-11T09:48:22.134519+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:49:39.448060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.047453+0000","last_change":"2023-10-11T09:48:19.579660+0000","last_active":"2023-10-11T09:48:21.047453+0000","last_peered":"2023-10-11T09:48:21.047453+0000","last_clean":"2023-10-11T09:48:21.047453+0000","last_became_active":"2023-10-11T09:48:19.579377+0000","last_became_peered":"2023-10-11T09:48:19.579377+0000","last_unstale":"2023-10-11T09:48:21.047453+0000","last_undegraded":"2023-10-11T09:48:21.047453+0000","last_fullsized":"2023-10-11T09:48:21.047453+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:21:08.022527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.136575+0000","last_change":"2023-10-11T09:48:20.053393+0000","last_active":"2023-10-11T09:48:22.136575+0000","last_peered":"2023-10-11T09:48:22.136575+0000","last_clean":"2023-10-11T09:48:22.136575+0000","last_became_active":"2023-10-11T09:48:20.053118+0000","last_became_peered":"2023-10-11T09:48:20.053118+0000","last_unstale":"2023-10-11T09:48:22.136575+0000","last_undegraded":"2023-10-11T09:48:22.136575+0000","last_fullsized":"2023-10-11T09:48:22.136575+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:21:50.863599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.137156+0000","last_change":"2023-10-11T09:48:19.578202+0000","last_active":"2023-10-11T09:48:22.137156+0000","last_peered":"2023-10-11T09:48:22.137156+0000","last_clean":"2023-10-11T09:48:22.137156+0000","last_became_active":"2023-10-11T09:48:19.577934+0000","last_became_peered":"2023-10-11T09:48:19.577934+0000","last_unstale":"2023-10-11T09:48:22.137156+0000","last_undegraded":"2023-10-11T09:48:22.137156+0000","last_fullsized":"2023-10-11T09:48:22.137156+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:40:22.421145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.043533+0000","last_change":"2023-10-11T09:48:20.056138+0000","last_active":"2023-10-11T09:48:21.043533+0000","last_peered":"2023-10-11T09:48:21.043533+0000","last_clean":"2023-10-11T09:48:21.043533+0000","last_became_active":"2023-10-11T09:48:20.055893+0000","last_became_peered":"2023-10-11T09:48:20.055893+0000","last_unstale":"2023-10-11T09:48:21.043533+0000","last_undegraded":"2023-10-11T09:48:21.043533+0000","last_fullsized":"2023-10-11T09:48:21.043533+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:47:43.381718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"82'3","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.537055+0000","last_change":"2023-10-11T09:48:19.577614+0000","last_active":"2023-10-11T09:48:21.537055+0000","last_peered":"2023-10-11T09:48:21.537055+0000","last_clean":"2023-10-11T09:48:21.537055+0000","last_became_active":"2023-10-11T09:48:19.577141+0000","last_became_peered":"2023-10-11T09:48:19.577141+0000","last_unstale":"2023-10-11T09:48:21.537055+0000","last_undegraded":"2023-10-11T09:48:21.537055+0000","last_fullsized":"2023-10-11T09:48:21.537055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:41:34.362183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.053982+0000","last_change":"2023-10-11T09:48:20.051606+0000","last_active":"2023-10-11T09:48:22.053982+0000","last_peered":"2023-10-11T09:48:22.053982+0000","last_clean":"2023-10-11T09:48:22.053982+0000","last_became_active":"2023-10-11T09:48:20.051022+0000","last_became_peered":"2023-10-11T09:48:20.051022+0000","last_unstale":"2023-10-11T09:48:22.053982+0000","last_undegraded":"2023-10-11T09:48:22.053982+0000","last_fullsized":"2023-10-11T09:48:22.053982+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:28:53.169535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"82'6","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.138139+0000","last_change":"2023-10-11T09:48:19.577593+0000","last_active":"2023-10-11T09:48:22.138139+0000","last_peered":"2023-10-11T09:48:22.138139+0000","last_clean":"2023-10-11T09:48:22.138139+0000","last_became_active":"2023-10-11T09:48:19.577146+0000","last_became_peered":"2023-10-11T09:48:19.577146+0000","last_unstale":"2023-10-11T09:48:22.138139+0000","last_undegraded":"2023-10-11T09:48:22.138139+0000","last_fullsized":"2023-10-11T09:48:22.138139+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:31:18.086995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.137151+0000","last_change":"2023-10-11T09:48:20.047535+0000","last_active":"2023-10-11T09:48:22.137151+0000","last_peered":"2023-10-11T09:48:22.137151+0000","last_clean":"2023-10-11T09:48:22.137151+0000","last_became_active":"2023-10-11T09:48:20.047041+0000","last_became_peered":"2023-10-11T09:48:20.047041+0000","last_unstale":"2023-10-11T09:48:22.137151+0000","last_undegraded":"2023-10-11T09:48:22.137151+0000","last_fullsized":"2023-10-11T09:48:22.137151+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:11:29.660137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"82'2","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.136518+0000","last_change":"2023-10-11T09:48:19.575547+0000","last_active":"2023-10-11T09:48:22.136518+0000","last_peered":"2023-10-11T09:48:22.136518+0000","last_clean":"2023-10-11T09:48:22.136518+0000","last_became_active":"2023-10-11T09:48:19.575211+0000","last_became_peered":"2023-10-11T09:48:19.575211+0000","last_unstale":"2023-10-11T09:48:22.136518+0000","last_undegraded":"2023-10-11T09:48:22.136518+0000","last_fullsized":"2023-10-11T09:48:22.136518+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:54:34.822974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.138177+0000","last_change":"2023-10-11T09:48:20.058769+0000","last_active":"2023-10-11T09:48:22.138177+0000","last_peered":"2023-10-11T09:48:22.138177+0000","last_clean":"2023-10-11T09:48:22.138177+0000","last_became_active":"2023-10-11T09:48:20.056338+0000","last_became_peered":"2023-10-11T09:48:20.056338+0000","last_unstale":"2023-10-11T09:48:22.138177+0000","last_undegraded":"2023-10-11T09:48:22.138177+0000","last_fullsized":"2023-10-11T09:48:22.138177+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:53:34.128929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.046760+0000","last_change":"2023-10-11T09:48:19.579252+0000","last_active":"2023-10-11T09:48:21.046760+0000","last_peered":"2023-10-11T09:48:21.046760+0000","last_clean":"2023-10-11T09:48:21.046760+0000","last_became_active":"2023-10-11T09:48:19.578888+0000","last_became_peered":"2023-10-11T09:48:19.578888+0000","last_unstale":"2023-10-11T09:48:21.046760+0000","last_undegraded":"2023-10-11T09:48:21.046760+0000","last_fullsized":"2023-10-11T09:48:21.046760+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:34:00.812070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.138314+0000","last_change":"2023-10-11T09:48:20.064762+0000","last_active":"2023-10-11T09:48:22.138314+0000","last_peered":"2023-10-11T09:48:22.138314+0000","last_clean":"2023-10-11T09:48:22.138314+0000","last_became_active":"2023-10-11T09:48:20.064451+0000","last_became_peered":"2023-10-11T09:48:20.064451+0000","last_unstale":"2023-10-11T09:48:22.138314+0000","last_undegraded":"2023-10-11T09:48:22.138314+0000","last_fullsized":"2023-10-11T09:48:22.138314+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T21:10:56.642508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.536288+0000","last_change":"2023-10-11T09:48:19.581504+0000","last_active":"2023-10-11T09:48:21.536288+0000","last_peered":"2023-10-11T09:48:21.536288+0000","last_clean":"2023-10-11T09:48:21.536288+0000","last_became_active":"2023-10-11T09:48:19.581001+0000","last_became_peered":"2023-10-11T09:48:19.581001+0000","last_unstale":"2023-10-11T09:48:21.536288+0000","last_undegraded":"2023-10-11T09:48:21.536288+0000","last_fullsized":"2023-10-11T09:48:21.536288+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:27:53.680109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.138465+0000","last_change":"2023-10-11T09:48:20.065724+0000","last_active":"2023-10-11T09:48:22.138465+0000","last_peered":"2023-10-11T09:48:22.138465+0000","last_clean":"2023-10-11T09:48:22.138465+0000","last_became_active":"2023-10-11T09:48:20.065439+0000","last_became_peered":"2023-10-11T09:48:20.065439+0000","last_unstale":"2023-10-11T09:48:22.138465+0000","last_undegraded":"2023-10-11T09:48:22.138465+0000","last_fullsized":"2023-10-11T09:48:22.138465+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:51:03.409007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"82'8","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.134499+0000","last_change":"2023-10-11T09:48:19.576212+0000","last_active":"2023-10-11T09:48:22.134499+0000","last_peered":"2023-10-11T09:48:22.134499+0000","last_clean":"2023-10-11T09:48:22.134499+0000","last_became_active":"2023-10-11T09:48:19.575898+0000","last_became_peered":"2023-10-11T09:48:19.575898+0000","last_unstale":"2023-10-11T09:48:22.134499+0000","last_undegraded":"2023-10-11T09:48:22.134499+0000","last_fullsized":"2023-10-11T09:48:22.134499+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:20:55.627010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.046359+0000","last_change":"2023-10-11T09:48:20.052876+0000","last_active":"2023-10-11T09:48:21.046359+0000","last_peered":"2023-10-11T09:48:21.046359+0000","last_clean":"2023-10-11T09:48:21.046359+0000","last_became_active":"2023-10-11T09:48:20.052561+0000","last_became_peered":"2023-10-11T09:48:20.052561+0000","last_unstale":"2023-10-11T09:48:21.046359+0000","last_undegraded":"2023-10-11T09:48:21.046359+0000","last_fullsized":"2023-10-11T09:48:21.046359+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:48:31.632403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"82'2","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.136835+0000","last_change":"2023-10-11T09:48:19.575554+0000","last_active":"2023-10-11T09:48:22.136835+0000","last_peered":"2023-10-11T09:48:22.136835+0000","last_clean":"2023-10-11T09:48:22.136835+0000","last_became_active":"2023-10-11T09:48:19.575221+0000","last_became_peered":"2023-10-11T09:48:19.575221+0000","last_unstale":"2023-10-11T09:48:22.136835+0000","last_undegraded":"2023-10-11T09:48:22.136835+0000","last_fullsized":"2023-10-11T09:48:22.136835+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:19:46.828454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.137551+0000","last_change":"2023-10-11T09:48:20.064317+0000","last_active":"2023-10-11T09:48:22.137551+0000","last_peered":"2023-10-11T09:48:22.137551+0000","last_clean":"2023-10-11T09:48:22.137551+0000","last_became_active":"2023-10-11T09:48:20.060327+0000","last_became_peered":"2023-10-11T09:48:20.060327+0000","last_unstale":"2023-10-11T09:48:22.137551+0000","last_undegraded":"2023-10-11T09:48:22.137551+0000","last_fullsized":"2023-10-11T09:48:22.137551+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:51:13.157565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.135385+0000","last_change":"2023-10-11T09:48:19.581885+0000","last_active":"2023-10-11T09:48:22.135385+0000","last_peered":"2023-10-11T09:48:22.135385+0000","last_clean":"2023-10-11T09:48:22.135385+0000","last_became_active":"2023-10-11T09:48:19.581609+0000","last_became_peered":"2023-10-11T09:48:19.581609+0000","last_unstale":"2023-10-11T09:48:22.135385+0000","last_undegraded":"2023-10-11T09:48:22.135385+0000","last_fullsized":"2023-10-11T09:48:22.135385+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:28:57.113866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.046606+0000","last_change":"2023-10-11T09:48:20.062864+0000","last_active":"2023-10-11T09:48:21.046606+0000","last_peered":"2023-10-11T09:48:21.046606+0000","last_clean":"2023-10-11T09:48:21.046606+0000","last_became_active":"2023-10-11T09:48:20.061145+0000","last_became_peered":"2023-10-11T09:48:20.061145+0000","last_unstale":"2023-10-11T09:48:21.046606+0000","last_undegraded":"2023-10-11T09:48:21.046606+0000","last_fullsized":"2023-10-11T09:48:21.046606+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:54:35.834677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.047310+0000","last_change":"2023-10-11T09:48:19.574768+0000","last_active":"2023-10-11T09:48:21.047310+0000","last_peered":"2023-10-11T09:48:21.047310+0000","last_clean":"2023-10-11T09:48:21.047310+0000","last_became_active":"2023-10-11T09:48:19.573306+0000","last_became_peered":"2023-10-11T09:48:19.573306+0000","last_unstale":"2023-10-11T09:48:21.047310+0000","last_undegraded":"2023-10-11T09:48:21.047310+0000","last_fullsized":"2023-10-11T09:48:21.047310+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:24:05.536905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.135159+0000","last_change":"2023-10-11T09:48:20.055053+0000","last_active":"2023-10-11T09:48:22.135159+0000","last_peered":"2023-10-11T09:48:22.135159+0000","last_clean":"2023-10-11T09:48:22.135159+0000","last_became_active":"2023-10-11T09:48:20.054785+0000","last_became_peered":"2023-10-11T09:48:20.054785+0000","last_unstale":"2023-10-11T09:48:22.135159+0000","last_undegraded":"2023-10-11T09:48:22.135159+0000","last_fullsized":"2023-10-11T09:48:22.135159+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:21:50.863599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.047118+0000","last_change":"2023-10-11T09:48:19.579196+0000","last_active":"2023-10-11T09:48:21.047118+0000","last_peered":"2023-10-11T09:48:21.047118+0000","last_clean":"2023-10-11T09:48:21.047118+0000","last_became_active":"2023-10-11T09:48:19.578865+0000","last_became_peered":"2023-10-11T09:48:19.578865+0000","last_unstale":"2023-10-11T09:48:21.047118+0000","last_undegraded":"2023-10-11T09:48:21.047118+0000","last_fullsized":"2023-10-11T09:48:21.047118+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:39:55.952816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.137880+0000","last_change":"2023-10-11T09:48:20.059145+0000","last_active":"2023-10-11T09:48:22.137880+0000","last_peered":"2023-10-11T09:48:22.137880+0000","last_clean":"2023-10-11T09:48:22.137880+0000","last_became_active":"2023-10-11T09:48:20.058851+0000","last_became_peered":"2023-10-11T09:48:20.058851+0000","last_unstale":"2023-10-11T09:48:22.137880+0000","last_undegraded":"2023-10-11T09:48:22.137880+0000","last_fullsized":"2023-10-11T09:48:22.137880+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:09:08.896230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.137779+0000","last_change":"2023-10-11T09:48:19.580143+0000","last_active":"2023-10-11T09:48:22.137779+0000","last_peered":"2023-10-11T09:48:22.137779+0000","last_clean":"2023-10-11T09:48:22.137779+0000","last_became_active":"2023-10-11T09:48:19.579725+0000","last_became_peered":"2023-10-11T09:48:19.579725+0000","last_unstale":"2023-10-11T09:48:22.137779+0000","last_undegraded":"2023-10-11T09:48:22.137779+0000","last_fullsized":"2023-10-11T09:48:22.137779+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:22:17.577022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.138328+0000","last_change":"2023-10-11T09:48:20.064875+0000","last_active":"2023-10-11T09:48:22.138328+0000","last_peered":"2023-10-11T09:48:22.138328+0000","last_clean":"2023-10-11T09:48:22.138328+0000","last_became_active":"2023-10-11T09:48:20.064611+0000","last_became_peered":"2023-10-11T09:48:20.064611+0000","last_unstale":"2023-10-11T09:48:22.138328+0000","last_undegraded":"2023-10-11T09:48:22.138328+0000","last_fullsized":"2023-10-11T09:48:22.138328+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:12:05.821898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.536762+0000","last_change":"2023-10-11T09:48:19.582483+0000","last_active":"2023-10-11T09:48:21.536762+0000","last_peered":"2023-10-11T09:48:21.536762+0000","last_clean":"2023-10-11T09:48:21.536762+0000","last_became_active":"2023-10-11T09:48:19.582205+0000","last_became_peered":"2023-10-11T09:48:19.582205+0000","last_unstale":"2023-10-11T09:48:21.536762+0000","last_undegraded":"2023-10-11T09:48:21.536762+0000","last_fullsized":"2023-10-11T09:48:21.536762+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:12:59.590334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.054231+0000","last_change":"2023-10-11T09:48:20.064498+0000","last_active":"2023-10-11T09:48:22.054231+0000","last_peered":"2023-10-11T09:48:22.054231+0000","last_clean":"2023-10-11T09:48:22.054231+0000","last_became_active":"2023-10-11T09:48:20.064231+0000","last_became_peered":"2023-10-11T09:48:20.064231+0000","last_unstale":"2023-10-11T09:48:22.054231+0000","last_undegraded":"2023-10-11T09:48:22.054231+0000","last_fullsized":"2023-10-11T09:48:22.054231+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:17:03.223268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.137801+0000","last_change":"2023-10-11T09:48:19.577356+0000","last_active":"2023-10-11T09:48:22.137801+0000","last_peered":"2023-10-11T09:48:22.137801+0000","last_clean":"2023-10-11T09:48:22.137801+0000","last_became_active":"2023-10-11T09:48:19.577048+0000","last_became_peered":"2023-10-11T09:48:19.577048+0000","last_unstale":"2023-10-11T09:48:22.137801+0000","last_undegraded":"2023-10-11T09:48:22.137801+0000","last_fullsized":"2023-10-11T09:48:22.137801+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:51:49.495573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.054179+0000","last_change":"2023-10-11T09:48:20.058936+0000","last_active":"2023-10-11T09:48:22.054179+0000","last_peered":"2023-10-11T09:48:22.054179+0000","last_clean":"2023-10-11T09:48:22.054179+0000","last_became_active":"2023-10-11T09:48:20.058593+0000","last_became_peered":"2023-10-11T09:48:20.058593+0000","last_unstale":"2023-10-11T09:48:22.054179+0000","last_undegraded":"2023-10-11T09:48:22.054179+0000","last_fullsized":"2023-10-11T09:48:22.054179+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:53:51.832854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.053564+0000","last_change":"2023-10-11T09:48:19.567901+0000","last_active":"2023-10-11T09:48:22.053564+0000","last_peered":"2023-10-11T09:48:22.053564+0000","last_clean":"2023-10-11T09:48:22.053564+0000","last_became_active":"2023-10-11T09:48:19.567408+0000","last_became_peered":"2023-10-11T09:48:19.567408+0000","last_unstale":"2023-10-11T09:48:22.053564+0000","last_undegraded":"2023-10-11T09:48:22.053564+0000","last_fullsized":"2023-10-11T09:48:22.053564+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:53:33.117225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.536076+0000","last_change":"2023-10-11T09:48:20.064109+0000","last_active":"2023-10-11T09:48:21.536076+0000","last_peered":"2023-10-11T09:48:21.536076+0000","last_clean":"2023-10-11T09:48:21.536076+0000","last_became_active":"2023-10-11T09:48:20.063490+0000","last_became_peered":"2023-10-11T09:48:20.063490+0000","last_unstale":"2023-10-11T09:48:21.536076+0000","last_undegraded":"2023-10-11T09:48:21.536076+0000","last_fullsized":"2023-10-11T09:48:21.536076+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:29:19.217653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"82'2","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.047216+0000","last_change":"2023-10-11T09:48:19.573204+0000","last_active":"2023-10-11T09:48:21.047216+0000","last_peered":"2023-10-11T09:48:21.047216+0000","last_clean":"2023-10-11T09:48:21.047216+0000","last_became_active":"2023-10-11T09:48:19.572889+0000","last_became_peered":"2023-10-11T09:48:19.572889+0000","last_unstale":"2023-10-11T09:48:21.047216+0000","last_undegraded":"2023-10-11T09:48:21.047216+0000","last_fullsized":"2023-10-11T09:48:21.047216+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:00:01.766804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":60,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-11T09:48:20.051138+0000","last_change":"2023-10-11T09:40:36.151839+0000","last_active":"2023-10-11T09:48:20.051138+0000","last_peered":"2023-10-11T09:48:20.051138+0000","last_clean":"2023-10-11T09:48:20.051138+0000","last_became_active":"2023-10-11T09:40:36.151179+0000","last_became_peered":"2023-10-11T09:40:36.151179+0000","last_unstale":"2023-10-11T09:48:20.051138+0000","last_undegraded":"2023-10-11T09:48:20.051138+0000","last_fullsized":"2023-10-11T09:48:20.051138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.043079+0000","last_change":"2023-10-11T09:48:20.058096+0000","last_active":"2023-10-11T09:48:21.043079+0000","last_peered":"2023-10-11T09:48:21.043079+0000","last_clean":"2023-10-11T09:48:21.043079+0000","last_became_active":"2023-10-11T09:48:20.056657+0000","last_became_peered":"2023-10-11T09:48:20.056657+0000","last_unstale":"2023-10-11T09:48:21.043079+0000","last_undegraded":"2023-10-11T09:48:21.043079+0000","last_fullsized":"2023-10-11T09:48:21.043079+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:11:29.660137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"82'1","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.047497+0000","last_change":"2023-10-11T09:48:19.571886+0000","last_active":"2023-10-11T09:48:21.047497+0000","last_peered":"2023-10-11T09:48:21.047497+0000","last_clean":"2023-10-11T09:48:21.047497+0000","last_became_active":"2023-10-11T09:48:19.571380+0000","last_became_peered":"2023-10-11T09:48:19.571380+0000","last_unstale":"2023-10-11T09:48:21.047497+0000","last_undegraded":"2023-10-11T09:48:21.047497+0000","last_fullsized":"2023-10-11T09:48:21.047497+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:57:57.259949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":65,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-11T09:48:20.040462+0000","last_change":"2023-10-11T09:40:36.150224+0000","last_active":"2023-10-11T09:48:20.040462+0000","last_peered":"2023-10-11T09:48:20.040462+0000","last_clean":"2023-10-11T09:48:20.040462+0000","last_became_active":"2023-10-11T09:40:36.149919+0000","last_became_peered":"2023-10-11T09:40:36.149919+0000","last_unstale":"2023-10-11T09:48:20.040462+0000","last_undegraded":"2023-10-11T09:48:20.040462+0000","last_fullsized":"2023-10-11T09:48:20.040462+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.043273+0000","last_change":"2023-10-11T09:48:20.060778+0000","last_active":"2023-10-11T09:48:21.043273+0000","last_peered":"2023-10-11T09:48:21.043273+0000","last_clean":"2023-10-11T09:48:21.043273+0000","last_became_active":"2023-10-11T09:48:20.060498+0000","last_became_peered":"2023-10-11T09:48:20.060498+0000","last_unstale":"2023-10-11T09:48:21.043273+0000","last_undegraded":"2023-10-11T09:48:21.043273+0000","last_fullsized":"2023-10-11T09:48:21.043273+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:20:15.431729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.053826+0000","last_change":"2023-10-11T09:48:19.573134+0000","last_active":"2023-10-11T09:48:22.053826+0000","last_peered":"2023-10-11T09:48:22.053826+0000","last_clean":"2023-10-11T09:48:22.053826+0000","last_became_active":"2023-10-11T09:48:19.572886+0000","last_became_peered":"2023-10-11T09:48:19.572886+0000","last_unstale":"2023-10-11T09:48:22.053826+0000","last_undegraded":"2023-10-11T09:48:22.053826+0000","last_fullsized":"2023-10-11T09:48:22.053826+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:09:07.884527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":77,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-11T09:48:20.043817+0000","last_change":"2023-10-11T09:40:36.162174+0000","last_active":"2023-10-11T09:48:20.043817+0000","last_peered":"2023-10-11T09:48:20.043817+0000","last_clean":"2023-10-11T09:48:20.043817+0000","last_became_active":"2023-10-11T09:40:36.161710+0000","last_became_peered":"2023-10-11T09:40:36.161710+0000","last_unstale":"2023-10-11T09:48:20.043817+0000","last_undegraded":"2023-10-11T09:48:20.043817+0000","last_fullsized":"2023-10-11T09:48:20.043817+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:57:03.630878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.536216+0000","last_change":"2023-10-11T09:48:20.069400+0000","last_active":"2023-10-11T09:48:21.536216+0000","last_peered":"2023-10-11T09:48:21.536216+0000","last_clean":"2023-10-11T09:48:21.536216+0000","last_became_active":"2023-10-11T09:48:20.069078+0000","last_became_peered":"2023-10-11T09:48:20.069078+0000","last_unstale":"2023-10-11T09:48:21.536216+0000","last_undegraded":"2023-10-11T09:48:21.536216+0000","last_fullsized":"2023-10-11T09:48:21.536216+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:09:08.896230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"82'1","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.537147+0000","last_change":"2023-10-11T09:48:19.574822+0000","last_active":"2023-10-11T09:48:21.537147+0000","last_peered":"2023-10-11T09:48:21.537147+0000","last_clean":"2023-10-11T09:48:21.537147+0000","last_became_active":"2023-10-11T09:48:19.574488+0000","last_became_peered":"2023-10-11T09:48:19.574488+0000","last_unstale":"2023-10-11T09:48:21.537147+0000","last_undegraded":"2023-10-11T09:48:21.537147+0000","last_fullsized":"2023-10-11T09:48:21.537147+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:11:22.058692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":65,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-11T09:48:20.038205+0000","last_change":"2023-10-11T09:40:36.160620+0000","last_active":"2023-10-11T09:48:20.038205+0000","last_peered":"2023-10-11T09:48:20.038205+0000","last_clean":"2023-10-11T09:48:20.038205+0000","last_became_active":"2023-10-11T09:40:36.160171+0000","last_became_peered":"2023-10-11T09:40:36.160171+0000","last_unstale":"2023-10-11T09:48:20.038205+0000","last_undegraded":"2023-10-11T09:48:20.038205+0000","last_fullsized":"2023-10-11T09:48:20.038205+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.134892+0000","last_change":"2023-10-11T09:48:20.045827+0000","last_active":"2023-10-11T09:48:22.134892+0000","last_peered":"2023-10-11T09:48:22.134892+0000","last_clean":"2023-10-11T09:48:22.134892+0000","last_became_active":"2023-10-11T09:48:20.045522+0000","last_became_peered":"2023-10-11T09:48:20.045522+0000","last_unstale":"2023-10-11T09:48:22.134892+0000","last_undegraded":"2023-10-11T09:48:22.134892+0000","last_fullsized":"2023-10-11T09:48:22.134892+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:55:47.212435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"82'2","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.043263+0000","last_change":"2023-10-11T09:48:19.565160+0000","last_active":"2023-10-11T09:48:21.043263+0000","last_peered":"2023-10-11T09:48:21.043263+0000","last_clean":"2023-10-11T09:48:21.043263+0000","last_became_active":"2023-10-11T09:48:19.564879+0000","last_became_peered":"2023-10-11T09:48:19.564879+0000","last_unstale":"2023-10-11T09:48:21.043263+0000","last_undegraded":"2023-10-11T09:48:21.043263+0000","last_fullsized":"2023-10-11T09:48:21.043263+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:39:55.952816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":65,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-11T09:48:20.036734+0000","last_change":"2023-10-11T09:40:36.151710+0000","last_active":"2023-10-11T09:48:20.036734+0000","last_peered":"2023-10-11T09:48:20.036734+0000","last_clean":"2023-10-11T09:48:20.036734+0000","last_became_active":"2023-10-11T09:40:36.151219+0000","last_became_peered":"2023-10-11T09:40:36.151219+0000","last_unstale":"2023-10-11T09:48:20.036734+0000","last_undegraded":"2023-10-11T09:48:20.036734+0000","last_fullsized":"2023-10-11T09:48:20.036734+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.138738+0000","last_change":"2023-10-11T09:48:20.058687+0000","last_active":"2023-10-11T09:48:22.138738+0000","last_peered":"2023-10-11T09:48:22.138738+0000","last_clean":"2023-10-11T09:48:22.138738+0000","last_became_active":"2023-10-11T09:48:20.058336+0000","last_became_peered":"2023-10-11T09:48:20.058336+0000","last_unstale":"2023-10-11T09:48:22.138738+0000","last_undegraded":"2023-10-11T09:48:22.138738+0000","last_fullsized":"2023-10-11T09:48:22.138738+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:18:49.991491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.043453+0000","last_change":"2023-10-11T09:48:19.576792+0000","last_active":"2023-10-11T09:48:21.043453+0000","last_peered":"2023-10-11T09:48:21.043453+0000","last_clean":"2023-10-11T09:48:21.043453+0000","last_became_active":"2023-10-11T09:48:19.576293+0000","last_became_peered":"2023-10-11T09:48:19.576293+0000","last_unstale":"2023-10-11T09:48:21.043453+0000","last_undegraded":"2023-10-11T09:48:21.043453+0000","last_fullsized":"2023-10-11T09:48:21.043453+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:20:55.627010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":64,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-11T09:48:20.037424+0000","last_change":"2023-10-11T09:40:36.216248+0000","last_active":"2023-10-11T09:48:20.037424+0000","last_peered":"2023-10-11T09:48:20.037424+0000","last_clean":"2023-10-11T09:48:20.037424+0000","last_became_active":"2023-10-11T09:40:36.215890+0000","last_became_peered":"2023-10-11T09:40:36.215890+0000","last_unstale":"2023-10-11T09:48:20.037424+0000","last_undegraded":"2023-10-11T09:48:20.037424+0000","last_fullsized":"2023-10-11T09:48:20.037424+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:22.135398+0000","last_change":"2023-10-11T09:48:20.062576+0000","last_active":"2023-10-11T09:48:22.135398+0000","last_peered":"2023-10-11T09:48:22.135398+0000","last_clean":"2023-10-11T09:48:22.135398+0000","last_became_active":"2023-10-11T09:48:20.062281+0000","last_became_peered":"2023-10-11T09:48:20.062281+0000","last_unstale":"2023-10-11T09:48:22.135398+0000","last_undegraded":"2023-10-11T09:48:22.135398+0000","last_fullsized":"2023-10-11T09:48:22.135398+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:00:34.688895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.043267+0000","last_change":"2023-10-11T09:48:19.577629+0000","last_active":"2023-10-11T09:48:21.043267+0000","last_peered":"2023-10-11T09:48:21.043267+0000","last_clean":"2023-10-11T09:48:21.043267+0000","last_became_active":"2023-10-11T09:48:19.577254+0000","last_became_peered":"2023-10-11T09:48:19.577254+0000","last_unstale":"2023-10-11T09:48:21.043267+0000","last_undegraded":"2023-10-11T09:48:21.043267+0000","last_fullsized":"2023-10-11T09:48:21.043267+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:28:57.113866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":77,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-11T09:48:20.042809+0000","last_change":"2023-10-11T09:40:36.149739+0000","last_active":"2023-10-11T09:48:20.042809+0000","last_peered":"2023-10-11T09:48:20.042809+0000","last_clean":"2023-10-11T09:48:20.042809+0000","last_became_active":"2023-10-11T09:40:36.149419+0000","last_became_peered":"2023-10-11T09:40:36.149419+0000","last_unstale":"2023-10-11T09:48:20.042809+0000","last_undegraded":"2023-10-11T09:48:20.042809+0000","last_fullsized":"2023-10-11T09:48:20.042809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:54:50.640362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.047254+0000","last_change":"2023-10-11T09:48:20.052287+0000","last_active":"2023-10-11T09:48:21.047254+0000","last_peered":"2023-10-11T09:48:21.047254+0000","last_clean":"2023-10-11T09:48:21.047254+0000","last_became_active":"2023-10-11T09:48:20.051817+0000","last_became_peered":"2023-10-11T09:48:20.051817+0000","last_unstale":"2023-10-11T09:48:21.047254+0000","last_undegraded":"2023-10-11T09:48:21.047254+0000","last_fullsized":"2023-10-11T09:48:21.047254+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:19.543459+0000","last_clean_scrub_stamp":"2023-10-11T09:48:19.543459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:45:38.018297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-11T09:48:21.043584+0000","last_change":"2023-10-11T09:48:19.576550+0000","last_active":"2023-10-11T09:48:21.043584+0000","last_peered":"2023-10-11T09:48:21.043584+0000","last_clean":"2023-10-11T09:48:21.043584+0000","last_became_active":"2023-10-11T09:48:19.576115+0000","last_became_peered":"2023-10-11T09:48:19.576115+0000","last_unstale":"2023-10-11T09:48:21.043584+0000","last_undegraded":"2023-10-11T09:48:21.043584+0000","last_fullsized":"2023-10-11T09:48:21.043584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:48:18.531756+0000","last_clean_scrub_stamp":"2023-10-11T09:48:18.531756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:37:36.623392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":65,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-11T09:48:20.037207+0000","last_change":"2023-10-11T09:40:36.148773+0000","last_active":"2023-10-11T09:48:20.037207+0000","last_peered":"2023-10-11T09:48:20.037207+0000","last_clean":"2023-10-11T09:48:20.037207+0000","last_became_active":"2023-10-11T09:40:36.148489+0000","last_became_peered":"2023-10-11T09:40:36.148489+0000","last_unstale":"2023-10-11T09:48:20.037207+0000","last_undegraded":"2023-10-11T09:48:20.037207+0000","last_fullsized":"2023-10-11T09:48:20.037207+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":230,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-10-11T09:48:20.042927+0000","last_change":"2023-10-11T09:40:29.874372+0000","last_active":"2023-10-11T09:48:20.042927+0000","last_peered":"2023-10-11T09:48:20.042927+0000","last_clean":"2023-10-11T09:48:20.042927+0000","last_became_active":"2023-10-11T09:40:29.873621+0000","last_became_peered":"2023-10-11T09:40:29.873621+0000","last_unstale":"2023-10-11T09:48:20.042927+0000","last_undegraded":"2023-10-11T09:48:20.042927+0000","last_fullsized":"2023-10-11T09:48:20.042927+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_clean_scrub_stamp":"2023-10-11T09:40:28.742276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:22:07.985311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574947,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32620,"kb_used_data":1572,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1609728,"data_stored":1069171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:47:45 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.97999999999999998,"15min":1.1180000000000001},"min":{"1min":0.56499999999999995,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.2689999999999999,"5min":1.3799999999999999,"15min":1.903},"last":1.0269999999999999},{"interface":"front","average":{"1min":0.88200000000000001,"5min":1.0209999999999999,"15min":1.1100000000000001},"min":{"1min":0.61899999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.323,"5min":1.5760000000000001,"15min":1.696},"last":0.98899999999999999}]},{"osd":1,"last update":"Wed Oct 11 09:47:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.083,"15min":1.1240000000000001},"min":{"1min":0.72399999999999998,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":2.25},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.88900000000000001,"5min":1.026,"15min":1.109},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.155,"5min":1.508,"15min":1.696},"last":0.88500000000000001}]},{"osd":2,"last update":"Wed Oct 11 09:47:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.079,"15min":1.1140000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.58999999999999997},"max":{"1min":1.361,"5min":1.6339999999999999,"15min":1.873},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.1299999999999999,"15min":1.1439999999999999},"min":{"1min":0.77300000000000002,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.3879999999999999,"5min":1.5680000000000001,"15min":1.6779999999999999},"last":1.1140000000000001}]},{"osd":3,"last update":"Wed Oct 11 09:47:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1579999999999999,"15min":1.2330000000000001},"min":{"1min":0.80400000000000005,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.4430000000000001,"5min":1.552,"15min":1.724},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.238,"15min":1.2849999999999999},"min":{"1min":1.02,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.4299999999999999,"5min":1.661,"15min":2.1579999999999999},"last":0.747}]},{"osd":4,"last update":"Wed Oct 11 09:47:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0589999999999999,"15min":1.1579999999999999},"min":{"1min":0.68300000000000005,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.4390000000000001,"5min":1.893,"15min":2.2010000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1819999999999999,"15min":1.2450000000000001},"min":{"1min":0.70299999999999996,"5min":0.66600000000000004,"15min":0.63500000000000001},"max":{"1min":1.3859999999999999,"5min":1.8640000000000001,"15min":1.9339999999999999},"last":1.1299999999999999}]},{"osd":5,"last update":"Wed Oct 11 09:47:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.258,"15min":1.2869999999999999},"min":{"1min":0.89600000000000002,"5min":0.85999999999999999,"15min":0.73499999999999999},"max":{"1min":1.5589999999999999,"5min":1.7250000000000001,"15min":1.744},"last":1.071},{"interface":"front","average":{"1min":1.139,"5min":1.1870000000000001,"15min":1.325},"min":{"1min":0.78300000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.3120000000000001,"5min":1.9690000000000001,"15min":3.0750000000000002},"last":1.0529999999999999}]},{"osd":6,"last update":"Wed Oct 11 09:47:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.335,"15min":1.4279999999999999},"min":{"1min":0.95799999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.4530000000000001,"5min":2.0289999999999999,"15min":3.137},"last":1.202},{"interface":"front","average":{"1min":1.284,"5min":1.3380000000000001,"15min":1.401},"min":{"1min":0.93200000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":1.5009999999999999,"5min":1.8080000000000001,"15min":1.839},"last":1.161}]}]},{"osd":6,"up_from":13,"seq":55834574946,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":1024000,"data_stored":480345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:47:38 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.997,"15min":0.99199999999999999},"min":{"1min":0.499,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.2450000000000001,"5min":1.589,"15min":2.1520000000000001},"last":7.0730000000000004},{"interface":"front","average":{"1min":1.028,"5min":0.996,"15min":1.032},"min":{"1min":0.61599999999999999,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3839999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":7.1189999999999998}]},{"osd":1,"last update":"Wed Oct 11 09:47:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.1479999999999999,"15min":1.179},"min":{"1min":0.68600000000000005,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.613,"5min":1.794,"15min":1.9379999999999999},"last":5.0279999999999996},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.137,"15min":1.2210000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3620000000000001,"5min":1.52,"15min":1.8260000000000001},"last":2.2869999999999999}]},{"osd":2,"last update":"Wed Oct 11 09:47:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.1200000000000001,"15min":1.149},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.508,"5min":1.7090000000000001,"15min":2.0859999999999999},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.123,"15min":1.22},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.371,"5min":1.6419999999999999,"15min":2.605},"last":2}]},{"osd":3,"last update":"Wed Oct 11 09:47:38 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.075,"15min":1.151},"min":{"1min":0.73399999999999999,"5min":0.627,"15min":0.627},"max":{"1min":1.2729999999999999,"5min":1.5589999999999999,"15min":2.0299999999999998},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.085,"5min":1.137,"15min":1.248},"min":{"1min":0.76400000000000001,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.427,"5min":1.7509999999999999,"15min":3.0819999999999999},"last":0.61599999999999999}]},{"osd":4,"last update":"Wed Oct 11 09:47:38 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.22,"15min":1.292},"min":{"1min":0.72099999999999997,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.486,"5min":1.7310000000000001,"15min":1.978},"last":7.5599999999999996},{"interface":"front","average":{"1min":1.194,"5min":1.2070000000000001,"15min":1.3280000000000001},"min":{"1min":0.84399999999999997,"5min":0.70599999999999996,"15min":0.69099999999999995},"max":{"1min":1.452,"5min":2.141,"15min":2.6920000000000002},"last":7.1779999999999999}]},{"osd":5,"last update":"Wed Oct 11 09:47:38 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.216,"15min":1.262},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.58099999999999996},"max":{"1min":2.4009999999999998,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":0.57199999999999995},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.2569999999999999,"15min":1.3720000000000001},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.077,"5min":2.1640000000000001,"15min":2.7290000000000001},"last":2.0489999999999999}]},{"osd":7,"last update":"Wed Oct 11 09:47:38 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.276,"15min":1.373},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.613,"5min":1.895,"15min":2.1899999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.321,"5min":1.393,"15min":1.4530000000000001},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.6919999999999999,"5min":1.859,"15min":2.3050000000000002},"last":0.65200000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574946,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":1015808,"data_stored":478815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":1.004,"15min":0.98799999999999999},"min":{"1min":0.53400000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.306,"5min":1.516,"15min":1.516},"last":1.052},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.026,"15min":0.98499999999999999},"min":{"1min":0.625,"5min":0.55300000000000005,"15min":0.46500000000000002},"max":{"1min":1.3109999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":0.93300000000000005}]},{"osd":1,"last update":"Wed Oct 11 09:47:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0449999999999999,"15min":0.97199999999999998},"min":{"1min":0.67800000000000005,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.083,"15min":1.0640000000000001},"min":{"1min":0.65200000000000002,"5min":0.60299999999999998,"15min":0.437},"max":{"1min":1.3899999999999999,"5min":1.607,"15min":1.607},"last":1.2509999999999999}]},{"osd":2,"last update":"Wed Oct 11 09:47:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.155,"15min":1.1859999999999999},"min":{"1min":0.76500000000000001,"5min":0.747,"15min":0.73899999999999999},"max":{"1min":1.5589999999999999,"5min":1.6379999999999999,"15min":1.831},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.177,"15min":1.2210000000000001},"min":{"1min":0.77100000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.4119999999999999,"5min":1.6879999999999999,"15min":1.8819999999999999},"last":1.486}]},{"osd":3,"last update":"Wed Oct 11 09:47:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.1040000000000001,"15min":1.129},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.56999999999999995},"max":{"1min":1.409,"5min":1.8500000000000001,"15min":1.921},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.147,"15min":1.1559999999999999},"min":{"1min":0.71099999999999997,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3700000000000001,"5min":1.5900000000000001,"15min":2.0209999999999999},"last":1.218}]},{"osd":4,"last update":"Wed Oct 11 09:47:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.179,"15min":1.1859999999999999},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.72399999999999998},"max":{"1min":1.4570000000000001,"5min":2.117,"15min":2.117},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.194,"5min":1.161,"15min":1.2},"min":{"1min":0.89700000000000002,"5min":0.62,"15min":0.62},"max":{"1min":1.4990000000000001,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.3}]},{"osd":6,"last update":"Wed Oct 11 09:47:40 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.244,"15min":1.274},"min":{"1min":0.92600000000000005,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.6060000000000001,"5min":1.764,"15min":2.4529999999999998},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3240000000000001,"15min":1.3109999999999999},"min":{"1min":0.80200000000000005,"5min":0.753,"15min":0.753},"max":{"1min":1.663,"5min":1.923,"15min":1.9630000000000001},"last":1.5449999999999999}]},{"osd":7,"last update":"Wed Oct 11 09:47:40 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.2829999999999999,"15min":1.3799999999999999},"min":{"1min":1.0149999999999999,"5min":0.82399999999999995,"15min":0.57699999999999996},"max":{"1min":1.7090000000000001,"5min":1.9770000000000001,"15min":3.6120000000000001},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.379,"15min":1.45},"min":{"1min":0.97099999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.712,"5min":1.8220000000000001,"15min":3.2109999999999999},"last":1.6870000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574946,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32056,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339784,"statfs":{"total":96636764160,"available":96603938816,"internally_reserved":0,"allocated":1032192,"data_stored":480449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":1.034,"15min":1.054},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.341,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.97199999999999998,"5min":1.0609999999999999,"15min":1.089},"min":{"1min":0.72199999999999998,"5min":0.64900000000000002,"15min":0.54700000000000004},"max":{"1min":1.333,"5min":1.5900000000000001,"15min":2.1160000000000001},"last":1.0549999999999999}]},{"osd":1,"last update":"Wed Oct 11 09:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":1.04,"15min":1.079},"min":{"1min":0.66500000000000004,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.2529999999999999,"5min":1.46,"15min":1.46},"last":1.353},{"interface":"front","average":{"1min":1.085,"5min":1.109,"15min":1.151},"min":{"1min":0.70599999999999996,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.429,"5min":1.635,"15min":1.8220000000000001},"last":1.1499999999999999}]},{"osd":2,"last update":"Wed Oct 11 09:47:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1539999999999999,"15min":1.1839999999999999},"min":{"1min":0.75,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.556,"5min":1.556,"15min":1.919},"last":1.6120000000000001},{"interface":"front","average":{"1min":1.083,"5min":1.1559999999999999,"15min":1.2090000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.8859999999999999},"last":1.8140000000000001}]},{"osd":3,"last update":"Wed Oct 11 09:47:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.089,"15min":1.135},"min":{"1min":0.76000000000000001,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.3580000000000001,"5min":1.3959999999999999,"15min":1.6930000000000001},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.1379999999999999,"15min":1.2769999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.72599999999999998},"max":{"1min":1.405,"5min":1.532,"15min":2.3340000000000001},"last":1.099}]},{"osd":5,"last update":"Wed Oct 11 09:47:47 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.1379999999999999,"15min":1.202},"min":{"1min":0.80600000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.5129999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.4730000000000001},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.1479999999999999,"15min":1.2030000000000001},"min":{"1min":0.80300000000000005,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.454,"5min":1.649,"15min":1.8520000000000001},"last":1.514}]},{"osd":6,"last update":"Wed Oct 11 09:47:47 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.198,"15min":1.268},"min":{"1min":0.93000000000000005,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.639,"5min":1.639,"15min":1.8},"last":1.196},{"interface":"front","average":{"1min":1.306,"5min":1.278,"15min":1.3280000000000001},"min":{"1min":0.95099999999999996,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.694,"5min":1.694,"15min":2.1120000000000001},"last":1.3919999999999999}]},{"osd":7,"last update":"Wed Oct 11 09:47:47 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.2330000000000001,"15min":1.3280000000000001},"min":{"1min":0.88,"5min":0.73299999999999998,"15min":0.60899999999999999},"max":{"1min":1.595,"5min":1.5960000000000001,"15min":1.9470000000000001},"last":1.302},{"interface":"front","average":{"1min":1.337,"5min":1.321,"15min":1.4450000000000001},"min":{"1min":1.091,"5min":0.82299999999999995,"15min":0.75900000000000001},"max":{"1min":1.7210000000000001,"5min":1.784,"15min":1.9990000000000001},"last":1.546}]}]},{"osd":3,"up_from":13,"seq":55834574945,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":1015808,"data_stored":479363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:47:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.70499999999999996,"15min":0.82899999999999996},"min":{"1min":0.499,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.28,"5min":1.28,"15min":1.462},"last":0.75},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.77000000000000002,"15min":0.84599999999999997},"min":{"1min":0.49199999999999999,"5min":0.435,"15min":0.41099999999999998},"max":{"1min":0.96999999999999997,"5min":1.256,"15min":1.4950000000000001},"last":0.84799999999999998}]},{"osd":1,"last update":"Wed Oct 11 09:47:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.78900000000000003,"15min":0.81100000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0149999999999999,"5min":1.335,"15min":1.335},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.871,"15min":0.93200000000000005},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.105,"5min":1.486,"15min":1.486},"last":1.2}]},{"osd":2,"last update":"Wed Oct 11 09:47:42 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.93700000000000006,"15min":0.97199999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.2649999999999999,"5min":1.4219999999999999,"15min":1.5389999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.95299999999999996,"15min":1.0840000000000001},"min":{"1min":0.58499999999999996,"5min":0.432,"15min":0.432},"max":{"1min":1.377,"5min":1.444,"15min":2.5870000000000002},"last":1.0069999999999999}]},{"osd":4,"last update":"Wed Oct 11 09:47:42 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.085,"15min":1.1970000000000001},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.5209999999999999,"5min":1.7210000000000001,"15min":2.395},"last":1.349},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.163,"15min":1.2849999999999999},"min":{"1min":0.94899999999999995,"5min":0.80900000000000005,"15min":0.73599999999999999},"max":{"1min":1.488,"5min":1.621,"15min":3.1579999999999999},"last":1.369}]},{"osd":5,"last update":"Wed Oct 11 09:47:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.194,"15min":1.3109999999999999},"min":{"1min":0.84199999999999997,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.5880000000000001,"5min":1.7430000000000001,"15min":3.0009999999999999},"last":1.502},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.198,"15min":1.393},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.67800000000000005},"max":{"1min":1.444,"5min":1.5840000000000001,"15min":3.8050000000000002},"last":1.4750000000000001}]},{"osd":6,"last update":"Wed Oct 11 09:47:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.1839999999999999,"15min":1.2829999999999999},"min":{"1min":0.86899999999999999,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.5740000000000001,"5min":1.663,"15min":2.1949999999999998},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.236,"5min":1.2509999999999999,"15min":1.377},"min":{"1min":0.97299999999999998,"5min":0.96599999999999997,"15min":0.84499999999999997},"max":{"1min":1.5720000000000001,"5min":1.663,"15min":3.2549999999999999},"last":1.159}]},{"osd":7,"last update":"Wed Oct 11 09:47:42 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.262,"15min":1.431},"min":{"1min":0.98099999999999998,"5min":0.89600000000000002,"15min":0.76700000000000002},"max":{"1min":1.5149999999999999,"5min":1.591,"15min":3.7490000000000001},"last":1.407},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.242,"15min":1.365},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.4650000000000001,"5min":1.661,"15min":2.625},"last":1.448}]}]},{"osd":2,"up_from":13,"seq":55834574947,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":1024000,"data_stored":479523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.77700000000000002,"15min":0.81399999999999995},"min":{"1min":0.52800000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":2.0600000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.79200000000000004,"15min":0.92700000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.2030000000000001,"5min":1.571,"15min":2.8919999999999999},"last":0.75700000000000001}]},{"osd":1,"last update":"Wed Oct 11 09:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.84499999999999997,"15min":1},"min":{"1min":0.52300000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.383,"5min":1.4990000000000001,"15min":3.7650000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.93600000000000005,"15min":0.98099999999999998},"min":{"1min":0.59199999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.675,"5min":1.675,"15min":2.5339999999999998},"last":0.90100000000000002}]},{"osd":3,"last update":"Wed Oct 11 09:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.92400000000000004,"15min":1.0700000000000001},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1240000000000001,"5min":1.4059999999999999,"15min":2.403},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.97199999999999998,"15min":1.056},"min":{"1min":0.70099999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.169,"5min":1.4690000000000001,"15min":2.7589999999999999},"last":1.2909999999999999}]},{"osd":4,"last update":"Wed Oct 11 09:47:41 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.135,"15min":1.204},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":2.2549999999999999},"last":1.478},{"interface":"front","average":{"1min":1.22,"5min":1.2450000000000001,"15min":1.385},"min":{"1min":0.86799999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.464,"5min":1.651,"15min":2.6859999999999999},"last":1.3779999999999999}]},{"osd":5,"last update":"Wed Oct 11 09:47:41 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.212,"15min":1.3320000000000001},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.4450000000000001,"5min":1.6799999999999999,"15min":3.044},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.181,"15min":1.3009999999999999},"min":{"1min":0.72799999999999998,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.837,"5min":1.843,"15min":3.5790000000000002},"last":1.337}]},{"osd":6,"last update":"Wed Oct 11 09:47:41 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.1990000000000001,"15min":1.3799999999999999},"min":{"1min":0.78800000000000003,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.494,"5min":1.8049999999999999,"15min":3.1600000000000001},"last":1.26},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2609999999999999,"15min":1.3460000000000001},"min":{"1min":0.97299999999999998,"5min":0.623,"15min":0.623},"max":{"1min":1.518,"5min":1.806,"15min":2.839},"last":1.5049999999999999}]},{"osd":7,"last update":"Wed Oct 11 09:47:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.248,"15min":1.397},"min":{"1min":0.749,"5min":0.747,"15min":0.747},"max":{"1min":1.726,"5min":1.9079999999999999,"15min":3.1230000000000002},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.335,"5min":1.3400000000000001,"15min":1.488},"min":{"1min":1.0680000000000001,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.6850000000000001,"5min":1.9570000000000001,"15min":2.988},"last":1.5329999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574946,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":1019904,"data_stored":478893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.746,"15min":0.78600000000000003},"min":{"1min":0.52700000000000002,"5min":0.375,"15min":0.375},"max":{"1min":1.109,"5min":1.236,"15min":1.27},"last":0.878},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.77900000000000003,"15min":0.82799999999999996},"min":{"1min":0.51000000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0960000000000001,"5min":1.3160000000000001,"15min":1.3899999999999999},"last":0.67200000000000004}]},{"osd":2,"last update":"Wed Oct 11 09:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79300000000000004,"15min":0.85699999999999998},"min":{"1min":0.437,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.88500000000000001,"15min":0.97999999999999998},"min":{"1min":0.59899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.218,"5min":1.403,"15min":1.6619999999999999},"last":1.206}]},{"osd":3,"last update":"Wed Oct 11 09:47:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":0.98799999999999999,"15min":1.0209999999999999},"min":{"1min":0.57799999999999996,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.3460000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":1.0569999999999999},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.93999999999999995,"15min":0.98099999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.282,"5min":1.544,"15min":1.589},"last":0.83399999999999996}]},{"osd":4,"last update":"Wed Oct 11 09:47:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.0880000000000001,"15min":1.167},"min":{"1min":0.78000000000000003,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.476,"5min":1.482,"15min":2.016},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2,"15min":1.296},"min":{"1min":0.88800000000000001,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":2.323},"last":1.552}]},{"osd":5,"last update":"Wed Oct 11 09:47:41 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1950000000000001,"15min":1.302},"min":{"1min":0.86499999999999999,"5min":0.84199999999999997,"15min":0.65800000000000003},"max":{"1min":1.458,"5min":1.7070000000000001,"15min":2.4079999999999999},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.181,"5min":1.1659999999999999,"15min":1.256},"min":{"1min":0.91200000000000003,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":2.1349999999999998},"last":1.167}]},{"osd":6,"last update":"Wed Oct 11 09:47:41 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.22,"15min":1.355},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.71499999999999997},"max":{"1min":1.603,"5min":1.657,"15min":2.8940000000000001},"last":1.101},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.238,"15min":1.355},"min":{"1min":0.98699999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":3.3719999999999999},"last":1.26}]},{"osd":7,"last update":"Wed Oct 11 09:47:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.3049999999999999,"15min":1.4039999999999999},"min":{"1min":0.94299999999999995,"5min":0.89700000000000002,"15min":0.86099999999999999},"max":{"1min":1.7929999999999999,"5min":1.915,"15min":2.194},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.268,"5min":1.3149999999999999,"15min":1.417},"min":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.89600000000000002},"max":{"1min":1.53,"5min":1.7050000000000001,"15min":1.968},"last":1.2290000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574945,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32616,"kb_used_data":1568,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339224,"statfs":{"total":96636764160,"available":96603365376,"internally_reserved":0,"allocated":1605632,"data_stored":1069149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 11 09:47:33 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.77300000000000002,"15min":0.80500000000000005},"min":{"1min":0.48999999999999999,"5min":0.48599999999999999,"15min":0.38500000000000001},"max":{"1min":0.98599999999999999,"5min":1.1850000000000001,"15min":1.2989999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86499999999999999,"15min":0.92100000000000004},"min":{"1min":0.52600000000000002,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":2.3100000000000001},"last":0.91000000000000003}]},{"osd":2,"last update":"Wed Oct 11 09:47:33 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.89900000000000002,"15min":0.94199999999999995},"min":{"1min":0.53100000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0569999999999999,"5min":1.395,"15min":1.7509999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.876,"5min":0.94099999999999995,"15min":0.96599999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.9450000000000001},"last":0.752}]},{"osd":3,"last update":"Wed Oct 11 09:47:33 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.78200000000000003,"15min":0.871},"min":{"1min":0.56000000000000005,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":2.7639999999999998},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.91200000000000003,"15min":1.032},"min":{"1min":0.60899999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.2010000000000001,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":0.88200000000000001}]},{"osd":4,"last update":"Wed Oct 11 09:47:33 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.165,"15min":1.194},"min":{"1min":0.82699999999999996,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":1.363},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.107,"15min":1.204},"min":{"1min":0.88,"5min":0.82299999999999995,"15min":0.69799999999999995},"max":{"1min":1.3089999999999999,"5min":1.5620000000000001,"15min":2.6629999999999998},"last":1.173}]},{"osd":5,"last update":"Wed Oct 11 09:47:33 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.202,"15min":1.345},"min":{"1min":0.93700000000000006,"5min":0.878,"15min":0.878},"max":{"1min":1.3109999999999999,"5min":1.734,"15min":3.0150000000000001},"last":1.482},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.2110000000000001,"15min":1.3180000000000001},"min":{"1min":0.96199999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"max":{"1min":1.7070000000000001,"5min":1.7070000000000001,"15min":3.601},"last":1.1220000000000001}]},{"osd":6,"last update":"Wed Oct 11 09:47:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1639999999999999,"15min":1.23},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.78000000000000003},"max":{"1min":1.359,"5min":1.492,"15min":1.825},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.238,"15min":1.333},"min":{"1min":1.028,"5min":0.996,"15min":0.93899999999999995},"max":{"1min":1.4370000000000001,"5min":1.6599999999999999,"15min":2.1440000000000001},"last":1.391}]},{"osd":7,"last update":"Wed Oct 11 09:47:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2609999999999999,"15min":1.327},"min":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91400000000000003},"max":{"1min":1.75,"5min":1.75,"15min":1.8759999999999999},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.226,"15min":1.339},"min":{"1min":0.99399999999999999,"5min":0.92300000000000004,"15min":0.73899999999999999},"max":{"1min":1.482,"5min":1.5349999999999999,"15min":2.5419999999999998},"last":1.093}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-11T09:48:24.797 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-11T09:48:24.798 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-11T09:48:25.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.008+0000 7f2f06716700 1 Processor -- start 2023-10-11T09:48:25.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.008+0000 7f2f06716700 1 -- start start 2023-10-11T09:48:25.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.012+0000 7f2f06716700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f000b3240 0x7f2f000d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:25.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.012+0000 7f2f06716700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2f000b3a90 0x7f2f000b85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:25.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.012+0000 7f2f06716700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f000b47f0 con 0x7f2f000b3610 2023-10-11T09:48:25.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.012+0000 7f2f06716700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f000b8130 con 0x7f2f000b3a90 2023-10-11T09:48:25.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.012+0000 7f2f06716700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f000b3f10 con 0x7f2f000b3240 2023-10-11T09:48:25.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.012+0000 7f2f04f13700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f2f000b3610 0x7f2f0015d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:42900/0 (socket says 172.21.15.78:42900) 2023-10-11T09:48:25.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.012+0000 7f2f04f13700 1 -- 172.21.15.78:0/3000682017 learned_addr learned my addr 172.21.15.78:0/3000682017 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:25.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.012+0000 7f2f05714700 1 --2- 172.21.15.78:0/3000682017 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f000b3240 0x7f2f000d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:25.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.012+0000 7f2f05f15700 1 --2- 172.21.15.78:0/3000682017 >> v2:172.21.15.133:3300/0 conn(0x7f2f000b3a90 0x7f2f000b85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:25.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.012+0000 7f2ef7fff700 1 -- 172.21.15.78:0/3000682017 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 921121488 0 0) 0x7f2ef0001110 con 0x7f2f000b3610 2023-10-11T09:48:25.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.012+0000 7f2ef7fff700 1 -- 172.21.15.78:0/3000682017 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2eec0035d0 con 0x7f2f000b3610 2023-10-11T09:48:25.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.016+0000 7f2f05714700 1 -- 172.21.15.78:0/3000682017 >> v2:172.21.15.133:3300/0 conn(0x7f2f000b3a90 msgr2=0x7f2f000b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:25.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.016+0000 7f2f05714700 1 --2- 172.21.15.78:0/3000682017 >> v2:172.21.15.133:3300/0 conn(0x7f2f000b3a90 0x7f2f000b85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:25.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.016+0000 7f2f05714700 1 -- 172.21.15.78:0/3000682017 >> v1:172.21.15.78:6789/0 conn(0x7f2f000b3610 legacy=0x7f2f0015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:25.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.016+0000 7f2f05714700 1 -- 172.21.15.78:0/3000682017 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f000b41e0 con 0x7f2f000b3240 2023-10-11T09:48:25.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.016+0000 7f2f05714700 1 --2- 172.21.15.78:0/3000682017 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f000b3240 0x7f2f000d29d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2efc009850 tx=0x7f2efc009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed9aca04642278a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:25.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.016+0000 7f2ef7fff700 1 -- 172.21.15.78:0/3000682017 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2efc011830 con 0x7f2f000b3240 2023-10-11T09:48:25.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.016+0000 7f2ef7fff700 1 -- 172.21.15.78:0/3000682017 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2efc011990 con 0x7f2f000b3240 2023-10-11T09:48:25.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.016+0000 7f2ef7fff700 1 -- 172.21.15.78:0/3000682017 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2efc011d40 con 0x7f2f000b3240 2023-10-11T09:48:25.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.016+0000 7f2f06716700 1 -- 172.21.15.78:0/3000682017 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f000b3240 msgr2=0x7f2f000d29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:25.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.016+0000 7f2f06716700 1 --2- 172.21.15.78:0/3000682017 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f000b3240 0x7f2f000d29d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2efc009850 tx=0x7f2efc009b60 comp rx=0 tx=0).stop 2023-10-11T09:48:25.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.016+0000 7f2f06716700 1 -- 172.21.15.78:0/3000682017 shutdown_connections 2023-10-11T09:48:25.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.016+0000 7f2f06716700 1 --2- 172.21.15.78:0/3000682017 >> v2:172.21.15.133:3300/0 conn(0x7f2f000b3a90 0x7f2f000b85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:25.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.016+0000 7f2f06716700 1 --2- 172.21.15.78:0/3000682017 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f000b3240 0x7f2f000d29d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:25.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.016+0000 7f2f06716700 1 -- 172.21.15.78:0/3000682017 >> 172.21.15.78:0/3000682017 conn(0x7f2f0009c260 msgr2=0x7f2f000d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:25.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.016+0000 7f2f06716700 1 -- 172.21.15.78:0/3000682017 shutdown_connections 2023-10-11T09:48:25.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.016+0000 7f2f06716700 1 -- 172.21.15.78:0/3000682017 wait complete. 2023-10-11T09:48:25.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.020+0000 7f2f06716700 1 Processor -- start 2023-10-11T09:48:25.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.020+0000 7f2f06716700 1 -- start start 2023-10-11T09:48:25.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.020+0000 7f2f06716700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f000b3240 0x7f2f0015e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:25.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.020+0000 7f2f06716700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f000b3610 0x7f2f0015a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:25.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.020+0000 7f2f06716700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f000b3a90 0x7f2f0015ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:25.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.020+0000 7f2f06716700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0015de80 con 0x7f2f000b3240 2023-10-11T09:48:25.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.020+0000 7f2f06716700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0009fde0 con 0x7f2f000b3a90 2023-10-11T09:48:25.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.020+0000 7f2f06716700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0009f7f0 con 0x7f2f000b3610 2023-10-11T09:48:25.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.020+0000 7f2f05714700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f000b3240 0x7f2f0015e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:25.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.020+0000 7f2f05714700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f000b3240 0x7f2f0015e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57346/0 (socket says 172.21.15.78:57346) 2023-10-11T09:48:25.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.020+0000 7f2f05714700 1 -- 172.21.15.78:0/3755769654 learned_addr learned my addr 172.21.15.78:0/3755769654 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:25.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.020+0000 7f2f05f15700 1 --2- 172.21.15.78:0/3755769654 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f000b3a90 0x7f2f0015ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:25.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.020+0000 7f2f04f13700 1 --2- 172.21.15.78:0/3755769654 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f000b3610 0x7f2f0015a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:25.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.020+0000 7f2f05714700 1 -- 172.21.15.78:0/3755769654 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f000b3610 msgr2=0x7f2f0015a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:25.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.020+0000 7f2f05714700 1 --2- 172.21.15.78:0/3755769654 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f000b3610 0x7f2f0015a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:25.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.020+0000 7f2f05714700 1 -- 172.21.15.78:0/3755769654 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f000b3a90 msgr2=0x7f2f0015ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:25.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.020+0000 7f2f05714700 1 --2- 172.21.15.78:0/3755769654 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f000b3a90 0x7f2f0015ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:25.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.020+0000 7f2f05714700 1 -- 172.21.15.78:0/3755769654 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2efc0094b0 con 0x7f2f000b3240 2023-10-11T09:48:25.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.024+0000 7f2f05714700 1 --2- 172.21.15.78:0/3755769654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f000b3240 0x7f2f0015e410 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f2efc0090b0 tx=0x7f2efc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:25.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.024+0000 7f2ef5ffb700 1 -- 172.21.15.78:0/3755769654 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2efc004450 con 0x7f2f000b3240 2023-10-11T09:48:25.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.024+0000 7f2ef5ffb700 1 -- 172.21.15.78:0/3755769654 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2efc01e070 con 0x7f2f000b3240 2023-10-11T09:48:25.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.024+0000 7f2f06716700 1 -- 172.21.15.78:0/3755769654 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f0015d8c0 con 0x7f2f000b3240 2023-10-11T09:48:25.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.024+0000 7f2ef5ffb700 1 -- 172.21.15.78:0/3755769654 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2efc003900 con 0x7f2f000b3240 2023-10-11T09:48:25.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.024+0000 7f2f06716700 1 -- 172.21.15.78:0/3755769654 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f0009fa70 con 0x7f2f000b3240 2023-10-11T09:48:25.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.024+0000 7f2f06716700 1 -- 172.21.15.78:0/3755769654 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f00065d40 con 0x7f2f000b3240 2023-10-11T09:48:25.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.028+0000 7f2ef5ffb700 1 -- 172.21.15.78:0/3755769654 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f2efc003a60 con 0x7f2f000b3240 2023-10-11T09:48:25.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.028+0000 7f2ef5ffb700 1 --2- 172.21.15.78:0/3755769654 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2ee4065ed0 0x7f2ee4068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:25.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.028+0000 7f2ef5ffb700 1 -- 172.21.15.78:0/3755769654 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2efc084550 con 0x7f2f000b3240 2023-10-11T09:48:25.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.028+0000 7f2f04f13700 1 --2- 172.21.15.78:0/3755769654 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2ee4065ed0 0x7f2ee4068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:25.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.028+0000 7f2f04f13700 1 --2- 172.21.15.78:0/3755769654 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2ee4065ed0 0x7f2ee4068380 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2f0015bf10 tx=0x7f2ef0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:25.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.032+0000 7f2ef5ffb700 1 -- 172.21.15.78:0/3755769654 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f2efc051b90 con 0x7f2f000b3240 2023-10-11T09:48:25.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.272+0000 7f2f06716700 1 -- 172.21.15.78:0/3755769654 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f2f000b7250 con 0x7f2f000b3240 2023-10-11T09:48:25.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.280+0000 7f2ef5ffb700 1 -- 172.21.15.78:0/3755769654 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+0 (secure 0 0 0) 0x7f2efc0551b0 con 0x7f2f000b3240 2023-10-11T09:48:25.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.284+0000 7f2f06716700 1 -- 172.21.15.78:0/3755769654 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2ee4065ed0 msgr2=0x7f2ee4068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:25.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.284+0000 7f2f06716700 1 --2- 172.21.15.78:0/3755769654 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2ee4065ed0 0x7f2ee4068380 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2f0015bf10 tx=0x7f2ef0009040 comp rx=0 tx=0).stop 2023-10-11T09:48:25.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.284+0000 7f2f06716700 1 -- 172.21.15.78:0/3755769654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f000b3240 msgr2=0x7f2f0015e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:25.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.284+0000 7f2f06716700 1 --2- 172.21.15.78:0/3755769654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f000b3240 0x7f2f0015e410 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f2efc0090b0 tx=0x7f2efc004770 comp rx=0 tx=0).stop 2023-10-11T09:48:25.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.284+0000 7f2f06716700 1 -- 172.21.15.78:0/3755769654 shutdown_connections 2023-10-11T09:48:25.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.284+0000 7f2f06716700 1 --2- 172.21.15.78:0/3755769654 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2ee4065ed0 0x7f2ee4068380 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:25.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.284+0000 7f2f06716700 1 --2- 172.21.15.78:0/3755769654 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f000b3a90 0x7f2f0015ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:25.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.284+0000 7f2f06716700 1 --2- 172.21.15.78:0/3755769654 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f000b3610 0x7f2f0015a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:25.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.284+0000 7f2f06716700 1 --2- 172.21.15.78:0/3755769654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f000b3240 0x7f2f0015e410 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:25.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.284+0000 7f2f06716700 1 -- 172.21.15.78:0/3755769654 >> 172.21.15.78:0/3755769654 conn(0x7f2f0009c260 msgr2=0x7f2f000d0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:25.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.284+0000 7f2f06716700 1 -- 172.21.15.78:0/3755769654 shutdown_connections 2023-10-11T09:48:25.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.284+0000 7f2f06716700 1 -- 172.21.15.78:0/3755769654 wait complete. 2023-10-11T09:48:25.297 INFO:teuthology.orchestra.run.smithi078.stderr:updated caps for client.0 2023-10-11T09:48:25.317 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:48:25.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.476+0000 7fbc115c6700 1 Processor -- start 2023-10-11T09:48:25.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.480+0000 7fbc115c6700 1 -- start start 2023-10-11T09:48:25.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.480+0000 7fbc115c6700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbc0c15dba0 0x7fbc0c0af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:25.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.480+0000 7fbc115c6700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc0c0caef0 con 0x7fbc0c159500 2023-10-11T09:48:25.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.480+0000 7fbc115c6700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc0c0cccd0 con 0x7fbc0c0b2760 2023-10-11T09:48:25.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.480+0000 7fbc115c6700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc0c0afd10 con 0x7fbc0c15dba0 2023-10-11T09:48:25.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.480+0000 7fbc0affd700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fbc0c159500 0x7fbc0c0b2050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:42904/0 (socket says 172.21.15.78:42904) 2023-10-11T09:48:25.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.480+0000 7fbc0affd700 1 -- 172.21.15.78:0/829833663 learned_addr learned my addr 172.21.15.78:0/829833663 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:25.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.480+0000 7fbc027fc700 1 --2- 172.21.15.78:0/829833663 >> v2:172.21.15.133:3300/0 conn(0x7fbc0c15dba0 0x7fbc0c0af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:25.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.484+0000 7fbc0a7fc700 1 -- 172.21.15.78:0/829833663 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2921115813 0 0) 0x7fbc0c0caef0 con 0x7fbc0c159500 2023-10-11T09:48:25.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.484+0000 7fbc0a7fc700 1 -- 172.21.15.78:0/829833663 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbf80035d0 con 0x7fbc0c159500 2023-10-11T09:48:25.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.484+0000 7fbc027fc700 1 -- 172.21.15.78:0/829833663 >> v1:172.21.15.133:6789/0 conn(0x7fbc0c0b2760 legacy=0x7fbc0c0ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:25.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.484+0000 7fbc027fc700 1 -- 172.21.15.78:0/829833663 >> v1:172.21.15.78:6789/0 conn(0x7fbc0c159500 legacy=0x7fbc0c0b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:25.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.484+0000 7fbc027fc700 1 -- 172.21.15.78:0/829833663 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc0c0ccfb0 con 0x7fbc0c15dba0 2023-10-11T09:48:25.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.484+0000 7fbc027fc700 1 --2- 172.21.15.78:0/829833663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc0c15dba0 0x7fbc0c0af7d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4009850 tx=0x7fbbf4009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ac92a45eec94d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:25.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.484+0000 7fbc0a7fc700 1 -- 172.21.15.78:0/829833663 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4011830 con 0x7fbc0c15dba0 2023-10-11T09:48:25.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.484+0000 7fbc0a7fc700 1 -- 172.21.15.78:0/829833663 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf4011990 con 0x7fbc0c15dba0 2023-10-11T09:48:25.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.484+0000 7fbc0a7fc700 1 -- 172.21.15.78:0/829833663 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4011d40 con 0x7fbc0c15dba0 2023-10-11T09:48:25.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.484+0000 7fbc115c6700 1 -- 172.21.15.78:0/829833663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc0c15dba0 msgr2=0x7fbc0c0af7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:25.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.484+0000 7fbc115c6700 1 --2- 172.21.15.78:0/829833663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc0c15dba0 0x7fbc0c0af7d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4009850 tx=0x7fbbf4009b60 comp rx=0 tx=0).stop 2023-10-11T09:48:25.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.484+0000 7fbc115c6700 1 -- 172.21.15.78:0/829833663 shutdown_connections 2023-10-11T09:48:25.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.484+0000 7fbc115c6700 1 --2- 172.21.15.78:0/829833663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc0c15dba0 0x7fbc0c0af7d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:25.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.484+0000 7fbc115c6700 1 -- 172.21.15.78:0/829833663 >> 172.21.15.78:0/829833663 conn(0x7fbc0c06dd80 msgr2=0x7fbc0c06e150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:25.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.484+0000 7fbc115c6700 1 -- 172.21.15.78:0/829833663 shutdown_connections 2023-10-11T09:48:25.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.484+0000 7fbc115c6700 1 -- 172.21.15.78:0/829833663 wait complete. 2023-10-11T09:48:25.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.488+0000 7fbc115c6700 1 Processor -- start 2023-10-11T09:48:25.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.488+0000 7fbc115c6700 1 -- start start 2023-10-11T09:48:25.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.488+0000 7fbc115c6700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbc0c0b2760 0x7fbc0c0aeee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:25.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.488+0000 7fbc115c6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc0c159500 0x7fbc0c154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:25.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.488+0000 7fbc115c6700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc0c15dba0 0x7fbc0c06b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:25.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.488+0000 7fbc115c6700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0c06c7d0 con 0x7fbc0c159500 2023-10-11T09:48:25.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.488+0000 7fbc115c6700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0c1554b0 con 0x7fbc0c15dba0 2023-10-11T09:48:25.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.488+0000 7fbc115c6700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0c0ac8d0 con 0x7fbc0c0b2760 2023-10-11T09:48:25.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.492+0000 7fbc027fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc0c159500 0x7fbc0c154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:25.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.492+0000 7fbc027fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc0c159500 0x7fbc0c154f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57350/0 (socket says 172.21.15.78:57350) 2023-10-11T09:48:25.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.492+0000 7fbc027fc700 1 -- 172.21.15.78:0/3791957253 learned_addr learned my addr 172.21.15.78:0/3791957253 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:25.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.492+0000 7fbc0bfff700 1 --2- 172.21.15.78:0/3791957253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc0c15dba0 0x7fbc0c06b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:25.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.492+0000 7fbc0affd700 1 --2- 172.21.15.78:0/3791957253 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbc0c0b2760 0x7fbc0c0aeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:25.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.492+0000 7fbc027fc700 1 -- 172.21.15.78:0/3791957253 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbc0c0b2760 msgr2=0x7fbc0c0aeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:25.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.492+0000 7fbc027fc700 1 --2- 172.21.15.78:0/3791957253 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbc0c0b2760 0x7fbc0c0aeee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:25.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.492+0000 7fbc027fc700 1 -- 172.21.15.78:0/3791957253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc0c15dba0 msgr2=0x7fbc0c06b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:25.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.492+0000 7fbc027fc700 1 --2- 172.21.15.78:0/3791957253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc0c15dba0 0x7fbc0c06b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:25.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.492+0000 7fbc027fc700 1 -- 172.21.15.78:0/3791957253 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf40094b0 con 0x7fbc0c159500 2023-10-11T09:48:25.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.492+0000 7fbc0bfff700 1 --2- 172.21.15.78:0/3791957253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc0c15dba0 0x7fbc0c06b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:48:25.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.492+0000 7fbc027fc700 1 --2- 172.21.15.78:0/3791957253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc0c159500 0x7fbc0c154f70 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4000c00 tx=0x7fbbf4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:25.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.492+0000 7fbc03fff700 1 -- 172.21.15.78:0/3791957253 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4019ab0 con 0x7fbc0c159500 2023-10-11T09:48:25.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.492+0000 7fbc03fff700 1 -- 172.21.15.78:0/3791957253 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf401e070 con 0x7fbc0c159500 2023-10-11T09:48:25.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.492+0000 7fbc115c6700 1 -- 172.21.15.78:0/3791957253 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc0c0ad2e0 con 0x7fbc0c159500 2023-10-11T09:48:25.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.492+0000 7fbc03fff700 1 -- 172.21.15.78:0/3791957253 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4019e50 con 0x7fbc0c159500 2023-10-11T09:48:25.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.492+0000 7fbc115c6700 1 -- 172.21.15.78:0/3791957253 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc0c0ace70 con 0x7fbc0c159500 2023-10-11T09:48:25.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.492+0000 7fbc115c6700 1 -- 172.21.15.78:0/3791957253 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc0c04e530 con 0x7fbc0c159500 2023-10-11T09:48:25.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.496+0000 7fbc03fff700 1 -- 172.21.15.78:0/3791957253 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fbbf401a420 con 0x7fbc0c159500 2023-10-11T09:48:25.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.496+0000 7fbc03fff700 1 --2- 172.21.15.78:0/3791957253 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbbec065ed0 0x7fbbec068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:25.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.496+0000 7fbc03fff700 1 -- 172.21.15.78:0/3791957253 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbbf4084980 con 0x7fbc0c159500 2023-10-11T09:48:25.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.496+0000 7fbc0affd700 1 --2- 172.21.15.78:0/3791957253 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbbec065ed0 0x7fbbec068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:25.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.500+0000 7fbc0affd700 1 --2- 172.21.15.78:0/3791957253 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbbec065ed0 0x7fbbec068380 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc0011e0 tx=0x7fbbfc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:25.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.504+0000 7fbc03fff700 1 -- 172.21.15.78:0/3791957253 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fbbf4051fc0 con 0x7fbc0c159500 2023-10-11T09:48:25.534 INFO:tasks.ceph.mds.a.smithi078.stderr:2023-10-11T09:48:25.524+0000 7f41f4542700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-11T09:48:25.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.708+0000 7fbc115c6700 1 -- 172.21.15.78:0/3791957253 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc0c0ca7f0 con 0x7fbc0c159500 2023-10-11T09:48:25.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.712+0000 7fbc03fff700 1 -- 172.21.15.78:0/3791957253 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4650 (secure 0 0 0) 0x7fbbf4015020 con 0x7fbc0c159500 2023-10-11T09:48:25.720 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:25.720 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":58,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6961,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/1080544187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1080544187},{"type":"v1","addr":"172.21.15.133:6834","nonce":1080544187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/367194191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":367194191},{"type":"v1","addr":"172.21.15.133:6835","nonce":367194191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":7224,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/3463904224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3463904224},{"type":"v1","addr":"172.21.15.78:6836","nonce":3463904224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:48:20.022298+0000","modified":"2023-10-11T09:48:23.045010+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7227},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7227":{"gid":7227,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6837/951312831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":951312831},{"type":"v1","addr":"172.21.15.78:6837","nonce":951312831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-11T09:48:25.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.712+0000 7fbc115c6700 1 -- 172.21.15.78:0/3791957253 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbbec065ed0 msgr2=0x7fbbec068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:25.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.712+0000 7fbc115c6700 1 --2- 172.21.15.78:0/3791957253 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbbec065ed0 0x7fbbec068380 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc0011e0 tx=0x7fbbfc009040 comp rx=0 tx=0).stop 2023-10-11T09:48:25.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.712+0000 7fbc115c6700 1 -- 172.21.15.78:0/3791957253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc0c159500 msgr2=0x7fbc0c154f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:25.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.712+0000 7fbc115c6700 1 --2- 172.21.15.78:0/3791957253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc0c159500 0x7fbc0c154f70 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4000c00 tx=0x7fbbf4004770 comp rx=0 tx=0).stop 2023-10-11T09:48:25.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.712+0000 7fbc115c6700 1 -- 172.21.15.78:0/3791957253 shutdown_connections 2023-10-11T09:48:25.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.712+0000 7fbc115c6700 1 --2- 172.21.15.78:0/3791957253 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbbec065ed0 0x7fbbec068380 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:25.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.712+0000 7fbc115c6700 1 --2- 172.21.15.78:0/3791957253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc0c15dba0 0x7fbc0c06b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:25.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.712+0000 7fbc115c6700 1 --2- 172.21.15.78:0/3791957253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc0c159500 0x7fbc0c154f70 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:25.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.712+0000 7fbc115c6700 1 --2- 172.21.15.78:0/3791957253 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbc0c0b2760 0x7fbc0c0aeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:25.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.712+0000 7fbc115c6700 1 -- 172.21.15.78:0/3791957253 >> 172.21.15.78:0/3791957253 conn(0x7fbc0c06dd80 msgr2=0x7fbc0c0cb3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:25.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.712+0000 7fbc115c6700 1 -- 172.21.15.78:0/3791957253 shutdown_connections 2023-10-11T09:48:25.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:25.712+0000 7fbc115c6700 1 -- 172.21.15.78:0/3791957253 wait complete. 2023-10-11T09:48:25.725 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 58 2023-10-11T09:48:26.146 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 58, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-11T09:48:20.022298+0000', 'modified': '2023-10-11T09:48:23.045010+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7227}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7227': {'gid': 7227, 'name': 'a', 'rank': 0, 'incarnation': 54, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.78:6837/951312831', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6835', 'nonce': 951312831}, {'type': 'v1', 'addr': '172.21.15.78:6837', 'nonce': 951312831}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-11T09:48:26.147 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-11T09:48:26.147 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-11T09:48:26.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.316+0000 7f40c4fd2700 1 Processor -- start 2023-10-11T09:48:26.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.316+0000 7f40c4fd2700 1 -- start start 2023-10-11T09:48:26.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.320+0000 7f40c4fd2700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f40c015b8e0 0x7f40c006d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:26.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.320+0000 7f40c4fd2700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40c00ca950 0x7f40c013fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:26.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.320+0000 7f40c4fd2700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40c015e010 con 0x7f40c006d610 2023-10-11T09:48:26.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.320+0000 7f40c4fd2700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40c015e280 con 0x7f40c015b8e0 2023-10-11T09:48:26.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.320+0000 7f40c4fd2700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c015e560 con 0x7f40c00ca950 2023-10-11T09:48:26.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.320+0000 7f40bffff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40c00ca950 0x7f40c013fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:26.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.320+0000 7f40bffff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40c00ca950 0x7f40c013fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:48996/0 (socket says 172.21.15.78:48996) 2023-10-11T09:48:26.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.320+0000 7f40bffff700 1 -- 172.21.15.78:0/677760022 learned_addr learned my addr 172.21.15.78:0/677760022 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:26.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.320+0000 7f40bf7fe700 1 --2- 172.21.15.78:0/677760022 >> v2:172.21.15.133:3300/0 conn(0x7f40c015b8e0 0x7f40c006d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:26.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.320+0000 7f40bffff700 1 -- 172.21.15.78:0/677760022 >> v2:172.21.15.133:3300/0 conn(0x7f40c015b8e0 msgr2=0x7f40c006d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:26.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.320+0000 7f40bffff700 1 --2- 172.21.15.78:0/677760022 >> v2:172.21.15.133:3300/0 conn(0x7f40c015b8e0 0x7f40c006d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.320+0000 7f40bffff700 1 -- 172.21.15.78:0/677760022 >> v1:172.21.15.133:6789/0 conn(0x7f40c006d610 legacy=0x7f40c00ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:26.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.320+0000 7f40bffff700 1 -- 172.21.15.78:0/677760022 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c015e840 con 0x7f40c00ca950 2023-10-11T09:48:26.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.320+0000 7f40bffff700 1 --2- 172.21.15.78:0/677760022 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40c00ca950 0x7f40c013fcb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f40b8009e50 tx=0x7f40b800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b2ec517ca62aba9 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:26.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.328+0000 7f40be7fc700 1 -- 172.21.15.78:0/677760022 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b800e070 con 0x7f40c00ca950 2023-10-11T09:48:26.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.328+0000 7f40be7fc700 1 -- 172.21.15.78:0/677760022 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b800b910 con 0x7f40c00ca950 2023-10-11T09:48:26.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.328+0000 7f40be7fc700 1 -- 172.21.15.78:0/677760022 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b800bcc0 con 0x7f40c00ca950 2023-10-11T09:48:26.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.328+0000 7f40c4fd2700 1 -- 172.21.15.78:0/677760022 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40c00ca950 msgr2=0x7f40c013fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:26.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.328+0000 7f40c4fd2700 1 --2- 172.21.15.78:0/677760022 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40c00ca950 0x7f40c013fcb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f40b8009e50 tx=0x7f40b800b330 comp rx=0 tx=0).stop 2023-10-11T09:48:26.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.328+0000 7f40c4fd2700 1 -- 172.21.15.78:0/677760022 shutdown_connections 2023-10-11T09:48:26.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.328+0000 7f40c4fd2700 1 --2- 172.21.15.78:0/677760022 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40c00ca950 0x7f40c013fcb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.328+0000 7f40c4fd2700 1 --2- 172.21.15.78:0/677760022 >> v2:172.21.15.133:3300/0 conn(0x7f40c015b8e0 0x7f40c006d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.328+0000 7f40c4fd2700 1 -- 172.21.15.78:0/677760022 >> 172.21.15.78:0/677760022 conn(0x7f40c00cea60 msgr2=0x7f40c00cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:26.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.328+0000 7f40c4fd2700 1 -- 172.21.15.78:0/677760022 shutdown_connections 2023-10-11T09:48:26.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.328+0000 7f40c4fd2700 1 -- 172.21.15.78:0/677760022 wait complete. 2023-10-11T09:48:26.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.332+0000 7f40c4fd2700 1 Processor -- start 2023-10-11T09:48:26.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.332+0000 7f40c4fd2700 1 -- start start 2023-10-11T09:48:26.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.336+0000 7f40c4fd2700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40c006d610 0x7f40c01f16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:26.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.336+0000 7f40c4fd2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40c00ca950 0x7f40c012f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:26.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.336+0000 7f40c4fd2700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40c015b8e0 0x7f40c0130bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:26.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.336+0000 7f40c4fd2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c013f880 con 0x7f40c00ca950 2023-10-11T09:48:26.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.336+0000 7f40c4fd2700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c012ffe0 con 0x7f40c006d610 2023-10-11T09:48:26.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.336+0000 7f40c4fd2700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c01302c0 con 0x7f40c015b8e0 2023-10-11T09:48:26.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.336+0000 7f40beffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40c00ca950 0x7f40c012f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:26.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.336+0000 7f40bf7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40c006d610 0x7f40c01f16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:26.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.336+0000 7f40beffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40c00ca950 0x7f40c012f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57358/0 (socket says 172.21.15.78:57358) 2023-10-11T09:48:26.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.336+0000 7f40bf7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40c006d610 0x7f40c01f16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:39366/0 (socket says 172.21.15.78:39366) 2023-10-11T09:48:26.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.336+0000 7f40bf7fe700 1 -- 172.21.15.78:0/4281599023 learned_addr learned my addr 172.21.15.78:0/4281599023 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:26.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.336+0000 7f40bffff700 1 --2- 172.21.15.78:0/4281599023 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40c015b8e0 0x7f40c0130bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:26.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.336+0000 7f40bf7fe700 1 -- 172.21.15.78:0/4281599023 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40c015b8e0 msgr2=0x7f40c0130bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:26.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.336+0000 7f40bf7fe700 1 --2- 172.21.15.78:0/4281599023 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40c015b8e0 0x7f40c0130bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.336+0000 7f40bf7fe700 1 -- 172.21.15.78:0/4281599023 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40c00ca950 msgr2=0x7f40c012f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:26.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.336+0000 7f40bf7fe700 1 --2- 172.21.15.78:0/4281599023 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40c00ca950 0x7f40c012f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.336+0000 7f40bf7fe700 1 -- 172.21.15.78:0/4281599023 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b8009b30 con 0x7f40c006d610 2023-10-11T09:48:26.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.336+0000 7f40bf7fe700 1 --2- 172.21.15.78:0/4281599023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40c006d610 0x7f40c01f16e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f40b000e9d0 tx=0x7f40b000ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:26.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.336+0000 7f40a7fff700 1 -- 172.21.15.78:0/4281599023 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b0009720 con 0x7f40c006d610 2023-10-11T09:48:26.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.336+0000 7f40a7fff700 1 -- 172.21.15.78:0/4281599023 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b000ee70 con 0x7f40c006d610 2023-10-11T09:48:26.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.336+0000 7f40a7fff700 1 -- 172.21.15.78:0/4281599023 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b000ca50 con 0x7f40c006d610 2023-10-11T09:48:26.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.336+0000 7f40c4fd2700 1 -- 172.21.15.78:0/4281599023 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c01305a0 con 0x7f40c006d610 2023-10-11T09:48:26.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.336+0000 7f40c4fd2700 1 -- 172.21.15.78:0/4281599023 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c0134e00 con 0x7f40c006d610 2023-10-11T09:48:26.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.340+0000 7f40a7fff700 1 -- 172.21.15.78:0/4281599023 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f40b000cbb0 con 0x7f40c006d610 2023-10-11T09:48:26.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.344+0000 7f40a7fff700 1 --2- 172.21.15.78:0/4281599023 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f40ac065f00 0x7f40ac0683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:26.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.344+0000 7f40a7fff700 1 -- 172.21.15.78:0/4281599023 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f40b0083750 con 0x7f40c006d610 2023-10-11T09:48:26.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.344+0000 7f40beffd700 1 --2- 172.21.15.78:0/4281599023 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f40ac065f00 0x7f40ac0683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:26.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.344+0000 7f40beffd700 1 --2- 172.21.15.78:0/4281599023 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f40ac065f00 0x7f40ac0683b0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f40c015d220 tx=0x7f40b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:26.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.344+0000 7f40c94c3740 1 Processor -- start 2023-10-11T09:48:26.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.344+0000 7f40c94c3740 1 -- start start 2023-10-11T09:48:26.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.348+0000 7f40c94c3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2363eb0 0x261c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:26.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.348+0000 7f40c94c3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2363ae0 0x261c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:26.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.348+0000 7f40c94c3740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2363710 0x25e3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:26.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.348+0000 7f40c94c3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x25434a0 con 0x2363eb0 2023-10-11T09:48:26.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.348+0000 7f40c94c3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x2540c30 con 0x2363ae0 2023-10-11T09:48:26.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.348+0000 7f40c94c3740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x253c520 con 0x2363710 2023-10-11T09:48:26.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.348+0000 7f40bf7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2363eb0 0x261c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:26.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.348+0000 7f40bf7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2363eb0 0x261c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57370/0 (socket says 172.21.15.78:57370) 2023-10-11T09:48:26.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.348+0000 7f40bf7fe700 1 -- 172.21.15.78:0/691194741 learned_addr learned my addr 172.21.15.78:0/691194741 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:26.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.348+0000 7f40beffd700 1 --2- 172.21.15.78:0/691194741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2363ae0 0x261c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:26.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.348+0000 7f40bffff700 1 --2- 172.21.15.78:0/691194741 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2363710 0x25e3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:26.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.348+0000 7f40bf7fe700 1 -- 172.21.15.78:0/691194741 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2363710 msgr2=0x25e3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:26.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.348+0000 7f40bf7fe700 1 --2- 172.21.15.78:0/691194741 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2363710 0x25e3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.348+0000 7f40bf7fe700 1 -- 172.21.15.78:0/691194741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2363ae0 msgr2=0x261c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:26.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.348+0000 7f40bf7fe700 1 --2- 172.21.15.78:0/691194741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2363ae0 0x261c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.348+0000 7f40bf7fe700 1 -- 172.21.15.78:0/691194741 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x23af9e0 con 0x2363eb0 2023-10-11T09:48:26.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.348+0000 7f40bffff700 1 --2- 172.21.15.78:0/691194741 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2363710 0x25e3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:48:26.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.348+0000 7f40bf7fe700 1 --2- 172.21.15.78:0/691194741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2363eb0 0x261c020 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f40b0081ac0 tx=0x7f40b000cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50eb3d14c442c0ac server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:26.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.348+0000 7f40a57fa700 1 -- 172.21.15.78:0/691194741 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b005ca40 con 0x2363eb0 2023-10-11T09:48:26.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.348+0000 7f40a57fa700 1 -- 172.21.15.78:0/691194741 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b0052e00 con 0x2363eb0 2023-10-11T09:48:26.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.348+0000 7f40a57fa700 1 -- 172.21.15.78:0/691194741 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b0054f00 con 0x2363eb0 2023-10-11T09:48:26.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.348+0000 7f40c94c3740 1 -- 172.21.15.78:0/691194741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2363eb0 msgr2=0x261c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:26.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.348+0000 7f40c94c3740 1 --2- 172.21.15.78:0/691194741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2363eb0 0x261c020 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f40b0081ac0 tx=0x7f40b000cf30 comp rx=0 tx=0).stop 2023-10-11T09:48:26.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.348+0000 7f40c94c3740 1 -- 172.21.15.78:0/691194741 shutdown_connections 2023-10-11T09:48:26.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.348+0000 7f40c94c3740 1 --2- 172.21.15.78:0/691194741 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2363710 0x25e3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.348+0000 7f40c94c3740 1 --2- 172.21.15.78:0/691194741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2363ae0 0x261c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.348+0000 7f40c94c3740 1 --2- 172.21.15.78:0/691194741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2363eb0 0x261c020 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.348+0000 7f40c94c3740 1 -- 172.21.15.78:0/691194741 >> 172.21.15.78:0/691194741 conn(0x25cbfa0 msgr2=0x25fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:26.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.348+0000 7f40c94c3740 1 -- 172.21.15.78:0/691194741 shutdown_connections 2023-10-11T09:48:26.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.348+0000 7f40c94c3740 1 -- 172.21.15.78:0/691194741 wait complete. 2023-10-11T09:48:26.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.352+0000 7f40c94c3740 1 -- start start 2023-10-11T09:48:26.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.352+0000 7f40c94c3740 1 Processor -- start 2023-10-11T09:48:26.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.352+0000 7f40c94c3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2363710 0x25f4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:26.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.352+0000 7f40c94c3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2363ae0 0x2610bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:26.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.352+0000 7f40c94c3740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2363eb0 0x2611110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:26.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.352+0000 7f40c94c3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x24ffb00 con 0x2363710 2023-10-11T09:48:26.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.352+0000 7f40c94c3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x24ff7a0 con 0x2363ae0 2023-10-11T09:48:26.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.352+0000 7f40c94c3740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x23e8b20 con 0x2363eb0 2023-10-11T09:48:26.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.352+0000 7f40c94c3740 10 client.0 _ll_register_callbacks cb 0x2535e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:48:26.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.352+0000 7f40bf7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2363710 0x25f4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:26.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.352+0000 7f40bf7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2363710 0x25f4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57378/0 (socket says 172.21.15.78:57378) 2023-10-11T09:48:26.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.352+0000 7f40bf7fe700 1 -- 172.21.15.78:0/3090627564 learned_addr learned my addr 172.21.15.78:0/3090627564 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:26.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.352+0000 7f40beffd700 1 --2- 172.21.15.78:0/3090627564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2363ae0 0x2610bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:26.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.352+0000 7f40bffff700 1 --2- 172.21.15.78:0/3090627564 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2363eb0 0x2611110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:26.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.352+0000 7f40bf7fe700 1 -- 172.21.15.78:0/3090627564 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2363eb0 msgr2=0x2611110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:26.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.352+0000 7f40bf7fe700 1 --2- 172.21.15.78:0/3090627564 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2363eb0 0x2611110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.352+0000 7f40bf7fe700 1 -- 172.21.15.78:0/3090627564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2363ae0 msgr2=0x2610bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:26.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.352+0000 7f40bf7fe700 1 --2- 172.21.15.78:0/3090627564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2363ae0 0x2610bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.352+0000 7f40bf7fe700 1 -- 172.21.15.78:0/3090627564 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c0134e00 con 0x2363710 2023-10-11T09:48:26.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.352+0000 7f40bf7fe700 1 --2- 172.21.15.78:0/3090627564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2363710 0x25f4b80 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f40b006c1d0 tx=0x7f40b0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=250e440902f0eee9 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:26.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.352+0000 7f409e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:48:26.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.352+0000 7f409e7fc700 1 -- 172.21.15.78:0/3090627564 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b0050da0 con 0x2363710 2023-10-11T09:48:26.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.352+0000 7f40c94c3740 1 -- 172.21.15.78:0/3090627564 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x25355a0 con 0x2363710 2023-10-11T09:48:26.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.352+0000 7f409e7fc700 1 -- 172.21.15.78:0/3090627564 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f40b005ca40 con 0x2363710 2023-10-11T09:48:26.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.352+0000 7f409e7fc700 1 -- 172.21.15.78:0/3090627564 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40b005c5b0 con 0x2363710 2023-10-11T09:48:26.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.352+0000 7f40c94c3740 10 client.7359 fetch_fsmap learned FSMap version 58 2023-10-11T09:48:26.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.352+0000 7f40c94c3740 1 -- 172.21.15.78:0/3090627564 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x2502750 con 0x2363710 2023-10-11T09:48:26.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.352+0000 7f409e7fc700 1 -- 172.21.15.78:0/3090627564 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x25355a0 con 0x2363710 2023-10-11T09:48:26.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.356+0000 7f409e7fc700 1 -- 172.21.15.78:0/3090627564 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 58) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f40b0072b20 con 0x2363710 2023-10-11T09:48:26.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.356+0000 7f40c94c3740 10 client.7359 fetch_fsmap finished waiting for FSMap version 58 2023-10-11T09:48:26.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.356+0000 7f40c94c3740 10 client.7359 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-11T09:48:26.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.356+0000 7f40c94c3740 20 client.7359 populate_metadata read hostname 'smithi078' 2023-10-11T09:48:26.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.356+0000 7f40c94c3740 1 --2- 172.21.15.78:0/3090627564 >> [v2:172.21.15.78:6835/951312831,v1:172.21.15.78:6837/951312831] conn(0x23ae790 0x26255a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:26.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.356+0000 7f40c94c3740 4 client.7359 mds_command: new command op to 7227 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-11T09:48:26.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.356+0000 7f40c94c3740 1 -- 172.21.15.78:0/3090627564 --> [v2:172.21.15.78:6835/951312831,v1:172.21.15.78:6837/951312831] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x25352c0 con 0x23ae790 2023-10-11T09:48:26.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.356+0000 7f40beffd700 1 --2- 172.21.15.78:0/3090627564 >> [v2:172.21.15.78:6835/951312831,v1:172.21.15.78:6837/951312831] conn(0x23ae790 0x26255a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:26.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.356+0000 7f40beffd700 1 --2- 172.21.15.78:0/3090627564 >> [v2:172.21.15.78:6835/951312831,v1:172.21.15.78:6837/951312831] conn(0x23ae790 0x26255a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ea8a04706c50766 server_cookie=e773412661ab0687 in_seq=0 out_seq=0 2023-10-11T09:48:26.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.356+0000 7f409e7fc700 10 client.7359 ms_handle_connect on v2:172.21.15.78:6835/951312831 2023-10-11T09:48:26.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.356+0000 7f409e7fc700 1 -- 172.21.15.78:0/3090627564 <== mds.0 v2:172.21.15.78:6835/951312831 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x25352c0 con 0x23ae790 2023-10-11T09:48:26.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.356+0000 7f409e7fc700 10 client.7359 handle_command_reply: tid=0 2023-10-11T09:48:26.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.356+0000 7f40c94c3740 1 client.7359 shutdown 2023-10-11T09:48:26.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.356+0000 7f40c94c3740 1 -- 172.21.15.78:0/3090627564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2363710 msgr2=0x25f4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:26.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.356+0000 7f40c94c3740 1 --2- 172.21.15.78:0/3090627564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2363710 0x25f4b80 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f40b006c1d0 tx=0x7f40b0000bc0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.356+0000 7f40c94c3740 1 -- 172.21.15.78:0/3090627564 shutdown_connections 2023-10-11T09:48:26.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.356+0000 7f40c94c3740 1 --2- 172.21.15.78:0/3090627564 >> [v2:172.21.15.78:6835/951312831,v1:172.21.15.78:6837/951312831] conn(0x23ae790 0x26255a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.356+0000 7f40c94c3740 1 --2- 172.21.15.78:0/3090627564 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2363eb0 0x2611110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.356+0000 7f409e7fc700 0 client.7359 ms_handle_reset on v2:172.21.15.78:6835/951312831 2023-10-11T09:48:26.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.356+0000 7f40c94c3740 1 --2- 172.21.15.78:0/3090627564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2363ae0 0x2610bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.356+0000 7f40c94c3740 1 --2- 172.21.15.78:0/3090627564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2363710 0x25f4b80 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.356+0000 7f40c94c3740 1 -- 172.21.15.78:0/3090627564 >> 172.21.15.78:0/3090627564 conn(0x25cbfa0 msgr2=0x25f0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:26.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.356+0000 7f40c94c3740 1 -- 172.21.15.78:0/3090627564 shutdown_connections 2023-10-11T09:48:26.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.356+0000 7f40c94c3740 1 -- 172.21.15.78:0/3090627564 wait complete. 2023-10-11T09:48:26.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.356+0000 7f40c94c3740 20 client.7359 trim_cache size 0 max 16384 2023-10-11T09:48:26.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.376+0000 7f40c94c3740 1 Processor -- start 2023-10-11T09:48:26.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.380+0000 7f40c94c3740 1 -- start start 2023-10-11T09:48:26.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.380+0000 7f40c94c3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2363710 0x25f3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:26.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.380+0000 7f40c94c3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2363ae0 0x260edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:26.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.380+0000 7f40c94c3740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2363eb0 0x260f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:26.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.380+0000 7f40c94c3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x2506a20 con 0x2363ae0 2023-10-11T09:48:26.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.380+0000 7f40c94c3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x24e4550 con 0x2363710 2023-10-11T09:48:26.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.380+0000 7f40c94c3740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x2504d00 con 0x2363eb0 2023-10-11T09:48:26.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.380+0000 7f40beffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2363ae0 0x260edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:26.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.380+0000 7f40beffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2363ae0 0x260edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57382/0 (socket says 172.21.15.78:57382) 2023-10-11T09:48:26.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.380+0000 7f40beffd700 1 -- 172.21.15.78:0/3800745481 learned_addr learned my addr 172.21.15.78:0/3800745481 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:26.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.380+0000 7f40bffff700 1 --2- 172.21.15.78:0/3800745481 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2363eb0 0x260f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:26.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.380+0000 7f40bf7fe700 1 --2- 172.21.15.78:0/3800745481 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2363710 0x25f3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:26.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.380+0000 7f40beffd700 1 -- 172.21.15.78:0/3800745481 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2363eb0 msgr2=0x260f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:26.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.380+0000 7f40beffd700 1 --2- 172.21.15.78:0/3800745481 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2363eb0 0x260f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.380+0000 7f40beffd700 1 -- 172.21.15.78:0/3800745481 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2363710 msgr2=0x25f3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:26.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.380+0000 7f40beffd700 1 --2- 172.21.15.78:0/3800745481 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2363710 0x25f3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.380+0000 7f40beffd700 1 -- 172.21.15.78:0/3800745481 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c0131490 con 0x2363ae0 2023-10-11T09:48:26.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.380+0000 7f40bffff700 1 --2- 172.21.15.78:0/3800745481 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2363eb0 0x260f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:48:26.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.380+0000 7f40beffd700 1 --2- 172.21.15.78:0/3800745481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2363ae0 0x260edc0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f40c0142c90 tx=0x7f40b4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436eb8ed26a40746 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:26.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.380+0000 7f40a5ffb700 1 -- 172.21.15.78:0/3800745481 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b400fbb0 con 0x2363ae0 2023-10-11T09:48:26.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.384+0000 7f40a5ffb700 1 -- 172.21.15.78:0/3800745481 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b400fd10 con 0x2363ae0 2023-10-11T09:48:26.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.384+0000 7f40a5ffb700 1 -- 172.21.15.78:0/3800745481 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b400fe80 con 0x2363ae0 2023-10-11T09:48:26.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.384+0000 7f40c94c3740 1 -- 172.21.15.78:0/3800745481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2363ae0 msgr2=0x260edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:26.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.384+0000 7f40c94c3740 1 --2- 172.21.15.78:0/3800745481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2363ae0 0x260edc0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f40c0142c90 tx=0x7f40b4000ed0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.384+0000 7f40c94c3740 1 -- 172.21.15.78:0/3800745481 shutdown_connections 2023-10-11T09:48:26.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.384+0000 7f40c94c3740 1 --2- 172.21.15.78:0/3800745481 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2363eb0 0x260f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.384+0000 7f40c94c3740 1 --2- 172.21.15.78:0/3800745481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2363ae0 0x260edc0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.384+0000 7f40c94c3740 1 --2- 172.21.15.78:0/3800745481 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2363710 0x25f3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.384+0000 7f40c94c3740 1 -- 172.21.15.78:0/3800745481 >> 172.21.15.78:0/3800745481 conn(0x25cbfa0 msgr2=0x262b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:26.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.384+0000 7f40c94c3740 1 -- 172.21.15.78:0/3800745481 shutdown_connections 2023-10-11T09:48:26.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.384+0000 7f40c94c3740 1 -- 172.21.15.78:0/3800745481 wait complete. 2023-10-11T09:48:26.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.384+0000 7f40c94c3740 1 -- start start 2023-10-11T09:48:26.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.388+0000 7f40c94c3740 1 Processor -- start 2023-10-11T09:48:26.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.388+0000 7f40c94c3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2363710 0x267fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:26.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.388+0000 7f40c94c3740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2363ae0 0x262f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:26.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.388+0000 7f40c94c3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2363eb0 0x262fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:26.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.388+0000 7f40c94c3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x25fdfa0 con 0x2363710 2023-10-11T09:48:26.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.388+0000 7f40c94c3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x26310d0 con 0x2363eb0 2023-10-11T09:48:26.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.388+0000 7f40c94c3740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x2630250 con 0x2363ae0 2023-10-11T09:48:26.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.388+0000 7f40c94c3740 10 client.0 _ll_register_callbacks cb 0x253ac80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:48:26.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.388+0000 7f40bf7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2363710 0x267fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:26.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.388+0000 7f40bf7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2363710 0x267fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57388/0 (socket says 172.21.15.78:57388) 2023-10-11T09:48:26.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.388+0000 7f40bf7fe700 1 -- 172.21.15.78:0/2617116001 learned_addr learned my addr 172.21.15.78:0/2617116001 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:26.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.388+0000 7f40beffd700 1 --2- 172.21.15.78:0/2617116001 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2363ae0 0x262f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:26.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.388+0000 7f40bf7fe700 1 -- 172.21.15.78:0/2617116001 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2363ae0 msgr2=0x262f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:26.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.388+0000 7f40bf7fe700 1 --2- 172.21.15.78:0/2617116001 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2363ae0 0x262f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.388+0000 7f40bf7fe700 1 -- 172.21.15.78:0/2617116001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2363eb0 msgr2=0x262fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:26.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.388+0000 7f40bf7fe700 1 --2- 172.21.15.78:0/2617116001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2363eb0 0x262fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.388+0000 7f40bf7fe700 1 -- 172.21.15.78:0/2617116001 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c015e840 con 0x2363710 2023-10-11T09:48:26.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.388+0000 7f40bf7fe700 1 --2- 172.21.15.78:0/2617116001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2363710 0x267fbb0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f40b00551e0 tx=0x7f40b00548e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be3a1160b3c3736 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:26.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.388+0000 7f409e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:48:26.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.388+0000 7f409e7fc700 1 -- 172.21.15.78:0/2617116001 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b0050a50 con 0x2363710 2023-10-11T09:48:26.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.388+0000 7f409e7fc700 1 -- 172.21.15.78:0/2617116001 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f40b0050bb0 con 0x2363710 2023-10-11T09:48:26.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.388+0000 7f40c94c3740 1 -- 172.21.15.78:0/2617116001 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f40b0052e00 con 0x2363710 2023-10-11T09:48:26.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.388+0000 7f409e7fc700 1 -- 172.21.15.78:0/2617116001 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40b0050da0 con 0x2363710 2023-10-11T09:48:26.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.388+0000 7f40c94c3740 10 client.7365 fetch_fsmap learned FSMap version 58 2023-10-11T09:48:26.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.388+0000 7f40c94c3740 1 -- 172.21.15.78:0/2617116001 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x7f40b401d040 con 0x2363710 2023-10-11T09:48:26.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.392+0000 7f409e7fc700 1 -- 172.21.15.78:0/2617116001 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b0052e00 con 0x2363710 2023-10-11T09:48:26.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.392+0000 7f409e7fc700 1 -- 172.21.15.78:0/2617116001 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 58) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f40b00539c0 con 0x2363710 2023-10-11T09:48:26.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.392+0000 7f40c94c3740 10 client.7365 fetch_fsmap finished waiting for FSMap version 58 2023-10-11T09:48:26.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.392+0000 7f40c94c3740 10 client.7365 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-11T09:48:26.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.392+0000 7f40c94c3740 20 client.7365 populate_metadata read hostname 'smithi078' 2023-10-11T09:48:26.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.392+0000 7f40c94c3740 1 --2- 172.21.15.78:0/2617116001 >> [v2:172.21.15.78:6835/951312831,v1:172.21.15.78:6837/951312831] conn(0x260e080 0x2631480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:26.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.392+0000 7f40c94c3740 4 client.7365 mds_command: new command op to 7227 tid=0[{"prefix": "status"}] 2023-10-11T09:48:26.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.392+0000 7f40c94c3740 1 -- 172.21.15.78:0/2617116001 --> [v2:172.21.15.78:6835/951312831,v1:172.21.15.78:6837/951312831] -- command(tid 0: {"prefix": "status"}) v1 -- 0x23bb4e0 con 0x260e080 2023-10-11T09:48:26.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.392+0000 7f40beffd700 1 --2- 172.21.15.78:0/2617116001 >> [v2:172.21.15.78:6835/951312831,v1:172.21.15.78:6837/951312831] conn(0x260e080 0x2631480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:26.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.392+0000 7f40beffd700 1 --2- 172.21.15.78:0/2617116001 >> [v2:172.21.15.78:6835/951312831,v1:172.21.15.78:6837/951312831] conn(0x260e080 0x2631480 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92e824a141db9d6b server_cookie=757688109be58700 in_seq=0 out_seq=0 2023-10-11T09:48:26.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.392+0000 7f409e7fc700 10 client.7365 ms_handle_connect on v2:172.21.15.78:6835/951312831 2023-10-11T09:48:26.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.392+0000 7f409e7fc700 1 -- 172.21.15.78:0/2617116001 <== mds.0 v2:172.21.15.78:6835/951312831 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x23bb4e0 con 0x260e080 2023-10-11T09:48:26.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.392+0000 7f409e7fc700 10 client.7365 handle_command_reply: tid=0 2023-10-11T09:48:26.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.392+0000 7f40c94c3740 1 client.7365 shutdown 2023-10-11T09:48:26.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.392+0000 7f40c94c3740 1 -- 172.21.15.78:0/2617116001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2363710 msgr2=0x267fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:26.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.392+0000 7f40c94c3740 1 --2- 172.21.15.78:0/2617116001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2363710 0x267fbb0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f40b00551e0 tx=0x7f40b00548e0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.392+0000 7f40c94c3740 1 -- 172.21.15.78:0/2617116001 shutdown_connections 2023-10-11T09:48:26.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.392+0000 7f40c94c3740 1 --2- 172.21.15.78:0/2617116001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2363eb0 0x262fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.392+0000 7f40c94c3740 1 --2- 172.21.15.78:0/2617116001 >> [v2:172.21.15.78:6835/951312831,v1:172.21.15.78:6837/951312831] conn(0x260e080 0x2631480 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.392+0000 7f40c94c3740 1 --2- 172.21.15.78:0/2617116001 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2363ae0 0x262f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.392+0000 7f409e7fc700 0 client.7365 ms_handle_reset on v2:172.21.15.78:6835/951312831 2023-10-11T09:48:26.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.392+0000 7f40c94c3740 1 --2- 172.21.15.78:0/2617116001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2363710 0x267fbb0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.392+0000 7f40c94c3740 1 -- 172.21.15.78:0/2617116001 >> 172.21.15.78:0/2617116001 conn(0x25cbfa0 msgr2=0x25f3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:26.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.392+0000 7f40c94c3740 1 -- 172.21.15.78:0/2617116001 shutdown_connections 2023-10-11T09:48:26.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.392+0000 7f40c94c3740 1 -- 172.21.15.78:0/2617116001 wait complete. 2023-10-11T09:48:26.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.392+0000 7f40c94c3740 20 client.7365 trim_cache size 0 max 16384 2023-10-11T09:48:26.411 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:48:26.411 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "dee99db9-7a72-406f-a1e2-917943f2b13d", 2023-10-11T09:48:26.411 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2023-10-11T09:48:26.411 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 7227, 2023-10-11T09:48:26.412 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2023-10-11T09:48:26.412 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2023-10-11T09:48:26.412 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2023-10-11T09:48:26.412 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 6.3695202039999996, 2023-10-11T09:48:26.412 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 58, 2023-10-11T09:48:26.413 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 83, 2023-10-11T09:48:26.413 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 83, 2023-10-11T09:48:26.413 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 11.495011784000001 2023-10-11T09:48:26.413 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:48:26.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.392+0000 7f40c4fd2700 1 -- 172.21.15.78:0/4281599023 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f40ac065f00 msgr2=0x7f40ac0683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:26.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.392+0000 7f40c4fd2700 1 --2- 172.21.15.78:0/4281599023 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f40ac065f00 0x7f40ac0683b0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f40c015d220 tx=0x7f40b4009040 comp rx=0 tx=0).stop 2023-10-11T09:48:26.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.392+0000 7f40c4fd2700 1 -- 172.21.15.78:0/4281599023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40c006d610 msgr2=0x7f40c01f16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:26.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.392+0000 7f40c4fd2700 1 --2- 172.21.15.78:0/4281599023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40c006d610 0x7f40c01f16e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f40b000e9d0 tx=0x7f40b000ece0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.392+0000 7f40c4fd2700 1 -- 172.21.15.78:0/4281599023 shutdown_connections 2023-10-11T09:48:26.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.392+0000 7f40c4fd2700 1 --2- 172.21.15.78:0/4281599023 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f40ac065f00 0x7f40ac0683b0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.392+0000 7f40c4fd2700 1 --2- 172.21.15.78:0/4281599023 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40c015b8e0 0x7f40c0130bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.392+0000 7f40c4fd2700 1 --2- 172.21.15.78:0/4281599023 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40c00ca950 0x7f40c012f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.392+0000 7f40c4fd2700 1 --2- 172.21.15.78:0/4281599023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40c006d610 0x7f40c01f16e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.392+0000 7f40c4fd2700 1 -- 172.21.15.78:0/4281599023 >> 172.21.15.78:0/4281599023 conn(0x7f40c00cea60 msgr2=0x7f40c00ca1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:26.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.396+0000 7f40c4fd2700 1 -- 172.21.15.78:0/4281599023 shutdown_connections 2023-10-11T09:48:26.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.396+0000 7f40c4fd2700 1 -- 172.21.15.78:0/4281599023 wait complete. 2023-10-11T09:48:26.426 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-11T09:48:26.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.588+0000 7f6ae63fb700 1 Processor -- start 2023-10-11T09:48:26.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.588+0000 7f6ae63fb700 1 -- start start 2023-10-11T09:48:26.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.592+0000 7f6ae63fb700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f6ae00caf20 0x7f6ae0164610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:26.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.592+0000 7f6ae63fb700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6ae0164b50 0x7f6ae006bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:26.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.592+0000 7f6ae63fb700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ae0165390 con 0x7f6ae00cda10 2023-10-11T09:48:26.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.592+0000 7f6ae63fb700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ae00cb3a0 con 0x7f6ae00caf20 2023-10-11T09:48:26.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.592+0000 7f6ae63fb700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae00ca440 con 0x7f6ae0164b50 2023-10-11T09:48:26.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.592+0000 7f6ae4bf8700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f6ae00caf20 0x7f6ae0164610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:26.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.592+0000 7f6ae53f9700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f6ae00cda10 0x7f6ae00cdde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:50976/0 (socket says 172.21.15.78:50976) 2023-10-11T09:48:26.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.592+0000 7f6ae53f9700 1 -- 172.21.15.78:0/1600771610 learned_addr learned my addr 172.21.15.78:0/1600771610 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:26.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.592+0000 7f6ae5bfa700 1 --2- 172.21.15.78:0/1600771610 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6ae0164b50 0x7f6ae006bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:26.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.592+0000 7f6ae4bf8700 1 -- 172.21.15.78:0/1600771610 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6ae0164b50 msgr2=0x7f6ae006bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:26.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.592+0000 7f6ae4bf8700 1 --2- 172.21.15.78:0/1600771610 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6ae0164b50 0x7f6ae006bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.592+0000 7f6ae4bf8700 1 -- 172.21.15.78:0/1600771610 >> v1:172.21.15.133:6789/0 conn(0x7f6ae00cda10 legacy=0x7f6ae00cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:26.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.592+0000 7f6ae4bf8700 1 -- 172.21.15.78:0/1600771610 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae00ca720 con 0x7f6ae00caf20 2023-10-11T09:48:26.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.592+0000 7f6ae4bf8700 1 --2- 172.21.15.78:0/1600771610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ae00caf20 0x7f6ae0164610 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0009850 tx=0x7f6ad0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70bec0e233d92338 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:26.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.592+0000 7f6ad7fff700 1 -- 172.21.15.78:0/1600771610 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0011830 con 0x7f6ae00caf20 2023-10-11T09:48:26.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.592+0000 7f6ad7fff700 1 -- 172.21.15.78:0/1600771610 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad0011990 con 0x7f6ae00caf20 2023-10-11T09:48:26.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.592+0000 7f6ad7fff700 1 -- 172.21.15.78:0/1600771610 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0011d40 con 0x7f6ae00caf20 2023-10-11T09:48:26.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.596+0000 7f6ae63fb700 1 -- 172.21.15.78:0/1600771610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ae00caf20 msgr2=0x7f6ae0164610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:26.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.596+0000 7f6ae63fb700 1 --2- 172.21.15.78:0/1600771610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ae00caf20 0x7f6ae0164610 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0009850 tx=0x7f6ad0009b60 comp rx=0 tx=0).stop 2023-10-11T09:48:26.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.596+0000 7f6ae63fb700 1 -- 172.21.15.78:0/1600771610 shutdown_connections 2023-10-11T09:48:26.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.596+0000 7f6ae63fb700 1 --2- 172.21.15.78:0/1600771610 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6ae0164b50 0x7f6ae006bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.596+0000 7f6ae63fb700 1 --2- 172.21.15.78:0/1600771610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ae00caf20 0x7f6ae0164610 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.596+0000 7f6ae63fb700 1 -- 172.21.15.78:0/1600771610 >> 172.21.15.78:0/1600771610 conn(0x7f6ae00cea60 msgr2=0x7f6ae00cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:26.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.596+0000 7f6ae63fb700 1 -- 172.21.15.78:0/1600771610 shutdown_connections 2023-10-11T09:48:26.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.596+0000 7f6ae63fb700 1 -- 172.21.15.78:0/1600771610 wait complete. 2023-10-11T09:48:26.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.600+0000 7f6ae63fb700 1 Processor -- start 2023-10-11T09:48:26.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.600+0000 7f6ae63fb700 1 -- start start 2023-10-11T09:48:26.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.600+0000 7f6ae63fb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ae00caf20 0x7f6ae01f9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:26.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.600+0000 7f6ae63fb700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6ae00cda10 0x7f6ae00b2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:26.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.600+0000 7f6ae63fb700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ae0164b50 0x7f6ae01f9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:26.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.600+0000 7f6ae63fb700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae01f9640 con 0x7f6ae00caf20 2023-10-11T09:48:26.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.600+0000 7f6ae63fb700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae00b2e70 con 0x7f6ae0164b50 2023-10-11T09:48:26.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.600+0000 7f6ae63fb700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae00b3150 con 0x7f6ae00cda10 2023-10-11T09:48:26.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.600+0000 7f6ae53f9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ae00caf20 0x7f6ae01f9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:26.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.600+0000 7f6ae53f9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ae00caf20 0x7f6ae01f9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57406/0 (socket says 172.21.15.78:57406) 2023-10-11T09:48:26.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.600+0000 7f6ae53f9700 1 -- 172.21.15.78:0/3438293426 learned_addr learned my addr 172.21.15.78:0/3438293426 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:26.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.600+0000 7f6ae4bf8700 1 --2- 172.21.15.78:0/3438293426 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6ae00cda10 0x7f6ae00b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:26.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.600+0000 7f6ae5bfa700 1 --2- 172.21.15.78:0/3438293426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ae0164b50 0x7f6ae01f9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:26.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.600+0000 7f6ae53f9700 1 -- 172.21.15.78:0/3438293426 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6ae00cda10 msgr2=0x7f6ae00b2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:26.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.600+0000 7f6ae53f9700 1 --2- 172.21.15.78:0/3438293426 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6ae00cda10 0x7f6ae00b2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.600+0000 7f6ae53f9700 1 -- 172.21.15.78:0/3438293426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ae0164b50 msgr2=0x7f6ae01f9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:26.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.600+0000 7f6ae53f9700 1 --2- 172.21.15.78:0/3438293426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ae0164b50 0x7f6ae01f9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.600+0000 7f6ae53f9700 1 -- 172.21.15.78:0/3438293426 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad00094b0 con 0x7f6ae00caf20 2023-10-11T09:48:26.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.600+0000 7f6ae53f9700 1 --2- 172.21.15.78:0/3438293426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ae00caf20 0x7f6ae01f9100 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f6adc00c700 tx=0x7f6adc00cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:26.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.600+0000 7f6ad5ffb700 1 -- 172.21.15.78:0/3438293426 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc005d90 con 0x7f6ae00caf20 2023-10-11T09:48:26.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.600+0000 7f6ad5ffb700 1 -- 172.21.15.78:0/3438293426 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6adc0070d0 con 0x7f6ae00caf20 2023-10-11T09:48:26.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.600+0000 7f6ad5ffb700 1 -- 172.21.15.78:0/3438293426 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc007440 con 0x7f6ae00caf20 2023-10-11T09:48:26.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.600+0000 7f6ae63fb700 1 -- 172.21.15.78:0/3438293426 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ae00b3430 con 0x7f6ae00caf20 2023-10-11T09:48:26.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.604+0000 7f6ae63fb700 1 -- 172.21.15.78:0/3438293426 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ae00b39f0 con 0x7f6ae00caf20 2023-10-11T09:48:26.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.604+0000 7f6ae63fb700 1 -- 172.21.15.78:0/3438293426 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ae0067f50 con 0x7f6ae00caf20 2023-10-11T09:48:26.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.604+0000 7f6ad5ffb700 1 -- 172.21.15.78:0/3438293426 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f6adc0075a0 con 0x7f6ae00caf20 2023-10-11T09:48:26.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.608+0000 7f6ad5ffb700 1 --2- 172.21.15.78:0/3438293426 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6ac4065ed0 0x7f6ac4068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:26.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.608+0000 7f6ad5ffb700 1 -- 172.21.15.78:0/3438293426 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6adc082c00 con 0x7f6ae00caf20 2023-10-11T09:48:26.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.608+0000 7f6ae4bf8700 1 --2- 172.21.15.78:0/3438293426 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6ac4065ed0 0x7f6ac4068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:26.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.608+0000 7f6ae4bf8700 1 --2- 172.21.15.78:0/3438293426 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6ac4065ed0 0x7f6ac4068380 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00090b0 tx=0x7f6ad0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:26.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.612+0000 7f6ad5ffb700 1 -- 172.21.15.78:0/3438293426 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f6adc050310 con 0x7f6ae00caf20 2023-10-11T09:48:26.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.824+0000 7f6ae63fb700 1 -- 172.21.15.78:0/3438293426 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6ae0065d40 con 0x7f6ae00caf20 2023-10-11T09:48:26.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.828+0000 7f6ad5ffb700 1 -- 172.21.15.78:0/3438293426 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v58) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6adc053930 con 0x7f6ae00caf20 2023-10-11T09:48:26.835 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-11T09:48:26.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.828+0000 7f6ae63fb700 1 -- 172.21.15.78:0/3438293426 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6ac4065ed0 msgr2=0x7f6ac4068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:26.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.828+0000 7f6ae63fb700 1 --2- 172.21.15.78:0/3438293426 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6ac4065ed0 0x7f6ac4068380 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00090b0 tx=0x7f6ad0008030 comp rx=0 tx=0).stop 2023-10-11T09:48:26.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.828+0000 7f6ae63fb700 1 -- 172.21.15.78:0/3438293426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ae00caf20 msgr2=0x7f6ae01f9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:26.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.828+0000 7f6ae63fb700 1 --2- 172.21.15.78:0/3438293426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ae00caf20 0x7f6ae01f9100 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f6adc00c700 tx=0x7f6adc00cac0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.828+0000 7f6ae63fb700 1 -- 172.21.15.78:0/3438293426 shutdown_connections 2023-10-11T09:48:26.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.828+0000 7f6ae63fb700 1 --2- 172.21.15.78:0/3438293426 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6ac4065ed0 0x7f6ac4068380 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.828+0000 7f6ae63fb700 1 --2- 172.21.15.78:0/3438293426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ae0164b50 0x7f6ae01f9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.828+0000 7f6ae63fb700 1 --2- 172.21.15.78:0/3438293426 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6ae00cda10 0x7f6ae00b2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.828+0000 7f6ae63fb700 1 --2- 172.21.15.78:0/3438293426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ae00caf20 0x7f6ae01f9100 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:26.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.828+0000 7f6ae63fb700 1 -- 172.21.15.78:0/3438293426 >> 172.21.15.78:0/3438293426 conn(0x7f6ae00cea60 msgr2=0x7f6ae00cb4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:26.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.828+0000 7f6ae63fb700 1 -- 172.21.15.78:0/3438293426 shutdown_connections 2023-10-11T09:48:26.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:26.828+0000 7f6ae63fb700 1 -- 172.21.15.78:0/3438293426 wait complete. 2023-10-11T09:48:26.860 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-11T09:48:26.860 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-11T09:48:26.860 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi078.front.sepia.ceph.com 2023-10-11T09:48:26.861 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-11T09:48:26.861 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-11T09:48:26.861 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-11T09:48:26.862 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-11T09:48:26.862 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-11T09:48:26.862 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-11T09:48:26.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:48:26.863 DEBUG:teuthology.orchestra.run.smithi078:> ip addr 2023-10-11T09:48:26.881 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-11T09:48:26.881 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-11T09:48:26.881 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2023-10-11T09:48:26.882 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:48:26.882 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2023-10-11T09:48:26.882 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:48:26.882 INFO:teuthology.orchestra.run.smithi078.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:48:26.882 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:22 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:48:26.883 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f0 2023-10-11T09:48:26.883 INFO:teuthology.orchestra.run.smithi078.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:48:26.883 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:23 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:48:26.883 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f1 2023-10-11T09:48:26.883 INFO:teuthology.orchestra.run.smithi078.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:48:26.884 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dc brd ff:ff:ff:ff:ff:ff 2023-10-11T09:48:26.884 INFO:teuthology.orchestra.run.smithi078.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-11T09:48:26.884 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dd brd ff:ff:ff:ff:ff:ff 2023-10-11T09:48:26.884 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.21.15.78/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-11T09:48:26.884 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft 41957sec preferred_lft 41957sec 2023-10-11T09:48:26.885 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::ec4:7aff:fe8f:d4dd/64 scope link 2023-10-11T09:48:26.885 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:48:26.885 INFO:teuthology.orchestra.run.smithi078.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-11T09:48:26.885 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 02:42:b6:e4:f8:0d brd ff:ff:ff:ff:ff:ff 2023-10-11T09:48:26.885 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-11T09:48:26.886 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:48:26.886 INFO:teuthology.orchestra.run.smithi078.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-11T09:48:26.886 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 7a:89:8a:a9:e1:23 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:48:26.886 INFO:teuthology.orchestra.run.smithi078.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-11T09:48:26.886 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:48:26.887 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::8ca:ebff:fea9:13f1/64 scope link 2023-10-11T09:48:26.887 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:48:26.887 INFO:teuthology.orchestra.run.smithi078.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-11T09:48:26.887 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 7a:89:8a:a9:e1:23 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-11T09:48:26.887 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::7889:8aff:fea9:e123/64 scope link 2023-10-11T09:48:26.887 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:48:26.888 INFO:teuthology.orchestra.run.smithi078.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-11T09:48:26.888 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether ca:ac:e8:27:51:0d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-11T09:48:26.888 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::c8ac:e8ff:fe27:510d/64 scope link 2023-10-11T09:48:26.888 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:48:26.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:48:26.890 DEBUG:teuthology.orchestra.run.smithi078:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-11T09:48:26.968 INFO:teuthology.orchestra.run.smithi078.stdout:1 2023-10-11T09:48:26.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:48:26.971 DEBUG:teuthology.orchestra.run.smithi078:> route 2023-10-11T09:48:27.035 INFO:teuthology.orchestra.run.smithi078.stdout:Kernel IP routing table 2023-10-11T09:48:27.035 INFO:teuthology.orchestra.run.smithi078.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-11T09:48:27.036 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-11T09:48:27.036 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-11T09:48:27.036 INFO:teuthology.orchestra.run.smithi078.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-11T09:48:27.036 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-10-11T09:48:27.036 INFO:teuthology.orchestra.run.smithi078.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-11T09:48:27.037 INFO:teuthology.orchestra.run.smithi078.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-11T09:48:27.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:48:27.038 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec bash -c ' 2023-10-11T09:48:27.038 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2023-10-11T09:48:27.039 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-11T09:48:27.039 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-11T09:48:27.040 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-11T09:48:27.040 DEBUG:teuthology.orchestra.run.smithi078:> ') 2023-10-11T09:48:27.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:48:27.175 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2023-10-11T09:48:27.193 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-11T09:48:27.193 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-11T09:48:27.194 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-11T09:48:27.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:48:27.195 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set brx.0 up 2023-10-11T09:48:27.273 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-11T09:48:27.274 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T09:48:27.274 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-11T09:48:27.338 INFO:teuthology.orchestra.run.smithi078.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-11T09:48:27.338 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T09:48:27.339 DEBUG:teuthology.orchestra.run.smithi078:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-11T09:48:27.401 DEBUG:teuthology.orchestra.run.smithi078:> sudo modprobe fuse 2023-10-11T09:48:27.478 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T09:48:27.478 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T09:48:27.557 INFO:teuthology.orchestra.run.smithi078.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T09:48:27.560 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T09:48:27.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:48:27.561 DEBUG:teuthology.orchestra.run.smithi078:> ls /sys/fs/fuse/connections 2023-10-11T09:48:27.625 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-11T09:48:27.626 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-11T09:48:27.677 DEBUG:teuthology.orchestra.run.smithi078:> sudo modprobe fuse 2023-10-11T09:48:27.712 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T09:48:27.712 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T09:48:27.790 INFO:teuthology.orchestra.run.smithi078.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T09:48:27.792 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T09:48:27.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:48:27.793 DEBUG:teuthology.orchestra.run.smithi078:> ls /sys/fs/fuse/connections 2023-10-11T09:48:27.812 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:2023-10-11T09:48:27.804+0000 7febff0e1640 -1 init, newargv = 0x560973ece9b0 newargc=15 2023-10-11T09:48:27.813 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:ceph-fuse[53166]: starting ceph client 2023-10-11T09:48:27.824 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:ceph-fuse[53166]: starting fuse 2023-10-11T09:48:27.827 INFO:teuthology.orchestra.run.smithi078.stdout:49 2023-10-11T09:48:27.828 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2023-10-11T09:48:27.828 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:48:27.829 DEBUG:teuthology.orchestra.run.smithi078:> import glob 2023-10-11T09:48:27.829 DEBUG:teuthology.orchestra.run.smithi078:> import re 2023-10-11T09:48:27.829 DEBUG:teuthology.orchestra.run.smithi078:> import os 2023-10-11T09:48:27.830 DEBUG:teuthology.orchestra.run.smithi078:> import subprocess 2023-10-11T09:48:27.830 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:48:27.831 DEBUG:teuthology.orchestra.run.smithi078:> def _find_admin_socket(client_name): 2023-10-11T09:48:27.831 DEBUG:teuthology.orchestra.run.smithi078:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-11T09:48:27.831 DEBUG:teuthology.orchestra.run.smithi078:> files = glob.glob(asok_path) 2023-10-11T09:48:27.832 DEBUG:teuthology.orchestra.run.smithi078:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-11T09:48:27.832 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:48:27.833 DEBUG:teuthology.orchestra.run.smithi078:> # Given a non-glob path, it better be there 2023-10-11T09:48:27.833 DEBUG:teuthology.orchestra.run.smithi078:> if "*" not in asok_path: 2023-10-11T09:48:27.833 DEBUG:teuthology.orchestra.run.smithi078:> assert(len(files) == 1) 2023-10-11T09:48:27.834 DEBUG:teuthology.orchestra.run.smithi078:> return files[0] 2023-10-11T09:48:27.834 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:48:27.835 DEBUG:teuthology.orchestra.run.smithi078:> for f in files: 2023-10-11T09:48:27.835 DEBUG:teuthology.orchestra.run.smithi078:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-11T09:48:27.835 DEBUG:teuthology.orchestra.run.smithi078:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-11T09:48:27.836 DEBUG:teuthology.orchestra.run.smithi078:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-11T09:48:27.836 DEBUG:teuthology.orchestra.run.smithi078:> contents = proc_f.read() 2023-10-11T09:48:27.836 DEBUG:teuthology.orchestra.run.smithi078:> if mountpoint in contents: 2023-10-11T09:48:27.837 DEBUG:teuthology.orchestra.run.smithi078:> return f 2023-10-11T09:48:27.837 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-11T09:48:27.837 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:48:27.838 DEBUG:teuthology.orchestra.run.smithi078:> print(_find_admin_socket("client.0")) 2023-10-11T09:48:27.838 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:48:27.986 INFO:teuthology.orchestra.run.smithi078.stdout:/var/run/ceph/ceph-client.0.53166.asok 2023-10-11T09:48:28.171 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.53166.asok 2023-10-11T09:48:28.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:48:28.172 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.53166.asok status 2023-10-11T09:48:28.371 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:48:28.371 INFO:teuthology.orchestra.run.smithi078.stdout: "metadata": { 2023-10-11T09:48:28.371 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_sha1": "9a774796f0ecdd11a4b771de396e627aa1a187af", 2023-10-11T09:48:28.371 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_version": "ceph version 18.2.0-662-g9a774796 (9a774796f0ecdd11a4b771de396e627aa1a187af) reef (stable)", 2023-10-11T09:48:28.372 INFO:teuthology.orchestra.run.smithi078.stdout: "entity_id": "0", 2023-10-11T09:48:28.372 INFO:teuthology.orchestra.run.smithi078.stdout: "hostname": "smithi078", 2023-10-11T09:48:28.372 INFO:teuthology.orchestra.run.smithi078.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-11T09:48:28.372 INFO:teuthology.orchestra.run.smithi078.stdout: "pid": "53166", 2023-10-11T09:48:28.372 INFO:teuthology.orchestra.run.smithi078.stdout: "root": "/" 2023-10-11T09:48:28.373 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:48:28.373 INFO:teuthology.orchestra.run.smithi078.stdout: "dentry_count": 0, 2023-10-11T09:48:28.373 INFO:teuthology.orchestra.run.smithi078.stdout: "dentry_pinned_count": 0, 2023-10-11T09:48:28.373 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 7377, 2023-10-11T09:48:28.373 INFO:teuthology.orchestra.run.smithi078.stdout: "inst": { 2023-10-11T09:48:28.374 INFO:teuthology.orchestra.run.smithi078.stdout: "name": { 2023-10-11T09:48:28.374 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "client", 2023-10-11T09:48:28.374 INFO:teuthology.orchestra.run.smithi078.stdout: "num": 7377 2023-10-11T09:48:28.374 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:48:28.374 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": { 2023-10-11T09:48:28.374 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2023-10-11T09:48:28.375 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "192.168.0.1:0", 2023-10-11T09:48:28.375 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 532063503 2023-10-11T09:48:28.375 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:48:28.375 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:48:28.375 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": { 2023-10-11T09:48:28.376 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2023-10-11T09:48:28.376 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "192.168.0.1:0", 2023-10-11T09:48:28.376 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 532063503 2023-10-11T09:48:28.376 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:48:28.376 INFO:teuthology.orchestra.run.smithi078.stdout: "inst_str": "client.7377 192.168.0.1:0/532063503", 2023-10-11T09:48:28.377 INFO:teuthology.orchestra.run.smithi078.stdout: "addr_str": "192.168.0.1:0/532063503", 2023-10-11T09:48:28.377 INFO:teuthology.orchestra.run.smithi078.stdout: "inode_count": 1, 2023-10-11T09:48:28.377 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_epoch": 58, 2023-10-11T09:48:28.377 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_epoch": 83, 2023-10-11T09:48:28.377 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_epoch_barrier": 0, 2023-10-11T09:48:28.378 INFO:teuthology.orchestra.run.smithi078.stdout: "blocklisted": false, 2023-10-11T09:48:28.378 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs" 2023-10-11T09:48:28.378 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:48:28.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:48:28.387 DEBUG:teuthology.orchestra.run.smithi078:> stat --file-system '--printf=%T 2023-10-11T09:48:28.387 DEBUG:teuthology.orchestra.run.smithi078:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-11T09:48:28.407 INFO:teuthology.orchestra.run.smithi078.stdout:fuseblk 2023-10-11T09:48:28.407 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-11T09:48:28.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:48:28.408 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-11T09:48:28.489 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_extra --pg_num_min 64 2023-10-11T09:48:28.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.692+0000 7f079b45d700 1 Processor -- start 2023-10-11T09:48:28.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.692+0000 7f079b45d700 1 -- start start 2023-10-11T09:48:28.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.696+0000 7f079b45d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07940cdbd0 0x7f079406cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:28.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.696+0000 7f079b45d700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0794165140 con 0x7f07940b0eb0 2023-10-11T09:48:28.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.696+0000 7f079b45d700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07940cd470 con 0x7f07941655d0 2023-10-11T09:48:28.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.696+0000 7f079b45d700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07941648e0 con 0x7f07940cdbd0 2023-10-11T09:48:28.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.696+0000 7f079a45b700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07940cdbd0 0x7f079406cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:28.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.696+0000 7f079ac5c700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f07940b0eb0 0x7f07940ca2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:42914/0 (socket says 172.21.15.78:42914) 2023-10-11T09:48:28.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.696+0000 7f079ac5c700 1 -- 172.21.15.78:0/553956190 learned_addr learned my addr 172.21.15.78:0/553956190 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:28.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.696+0000 7f0799c5a700 1 --1- 172.21.15.78:0/553956190 >> v1:172.21.15.133:6789/0 conn(0x7f07941655d0 0x7f07940cdfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:50988/0 (socket says 172.21.15.78:50988) 2023-10-11T09:48:28.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.696+0000 7f0799459700 1 -- 172.21.15.78:0/553956190 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2236208718 0 0) 0x7f07940cd470 con 0x7f07941655d0 2023-10-11T09:48:28.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.696+0000 7f0799459700 1 -- 172.21.15.78:0/553956190 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07800035d0 con 0x7f07941655d0 2023-10-11T09:48:28.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.696+0000 7f0799459700 1 -- 172.21.15.78:0/553956190 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 241144382 0 0) 0x7f0794165140 con 0x7f07940b0eb0 2023-10-11T09:48:28.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.696+0000 7f0799459700 1 -- 172.21.15.78:0/553956190 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07940cd470 con 0x7f07940b0eb0 2023-10-11T09:48:28.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.696+0000 7f079a45b700 1 -- 172.21.15.78:0/553956190 >> v1:172.21.15.133:6789/0 conn(0x7f07941655d0 legacy=0x7f07940cdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:28.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.696+0000 7f079a45b700 1 -- 172.21.15.78:0/553956190 >> v1:172.21.15.78:6789/0 conn(0x7f07940b0eb0 legacy=0x7f07940ca2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:28.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.696+0000 7f079a45b700 1 -- 172.21.15.78:0/553956190 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0794164af0 con 0x7f07940cdbd0 2023-10-11T09:48:28.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.700+0000 7f079a45b700 1 --2- 172.21.15.78:0/553956190 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07940cdbd0 0x7f079406cfa0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0790009880 tx=0x7f0790009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b83f493124c246fe server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:28.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.700+0000 7f0799459700 1 -- 172.21.15.78:0/553956190 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0790011830 con 0x7f07940cdbd0 2023-10-11T09:48:28.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.700+0000 7f0799459700 1 -- 172.21.15.78:0/553956190 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0790011990 con 0x7f07940cdbd0 2023-10-11T09:48:28.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.700+0000 7f0799459700 1 -- 172.21.15.78:0/553956190 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0790011d20 con 0x7f07940cdbd0 2023-10-11T09:48:28.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.700+0000 7f079b45d700 1 -- 172.21.15.78:0/553956190 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07940cdbd0 msgr2=0x7f079406cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:28.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.700+0000 7f079b45d700 1 --2- 172.21.15.78:0/553956190 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07940cdbd0 0x7f079406cfa0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0790009880 tx=0x7f0790009b90 comp rx=0 tx=0).stop 2023-10-11T09:48:28.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.700+0000 7f079b45d700 1 -- 172.21.15.78:0/553956190 shutdown_connections 2023-10-11T09:48:28.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.700+0000 7f079b45d700 1 --2- 172.21.15.78:0/553956190 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07940cdbd0 0x7f079406cfa0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:28.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.700+0000 7f079b45d700 1 -- 172.21.15.78:0/553956190 >> 172.21.15.78:0/553956190 conn(0x7f079406acf0 msgr2=0x7f07940b0710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:28.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.700+0000 7f079b45d700 1 -- 172.21.15.78:0/553956190 shutdown_connections 2023-10-11T09:48:28.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.700+0000 7f079b45d700 1 -- 172.21.15.78:0/553956190 wait complete. 2023-10-11T09:48:28.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.704+0000 7f079b45d700 1 Processor -- start 2023-10-11T09:48:28.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.704+0000 7f079b45d700 1 -- start start 2023-10-11T09:48:28.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.704+0000 7f079b45d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07940b0eb0 0x7f07940c87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:28.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.704+0000 7f079b45d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07940cdbd0 0x7f07941572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:28.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.704+0000 7f079b45d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07941655d0 0x7f07941f67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:28.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.704+0000 7f079b45d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07940c8d20 con 0x7f07941655d0 2023-10-11T09:48:28.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.704+0000 7f079b45d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07940c7a20 con 0x7f07940b0eb0 2023-10-11T09:48:28.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.704+0000 7f079b45d700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07940c7d00 con 0x7f07940cdbd0 2023-10-11T09:48:28.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.704+0000 7f079ac5c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07941655d0 0x7f07941f67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:28.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.708+0000 7f079ac5c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07941655d0 0x7f07941f67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57414/0 (socket says 172.21.15.78:57414) 2023-10-11T09:48:28.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.708+0000 7f079ac5c700 1 -- 172.21.15.78:0/4113875162 learned_addr learned my addr 172.21.15.78:0/4113875162 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:28.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.708+0000 7f079a45b700 1 --2- 172.21.15.78:0/4113875162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07940b0eb0 0x7f07940c87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:28.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.708+0000 7f0799c5a700 1 --2- 172.21.15.78:0/4113875162 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07940cdbd0 0x7f07941572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:28.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.708+0000 7f079ac5c700 1 -- 172.21.15.78:0/4113875162 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07940cdbd0 msgr2=0x7f07941572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:28.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.708+0000 7f079ac5c700 1 --2- 172.21.15.78:0/4113875162 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07940cdbd0 0x7f07941572e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:28.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.708+0000 7f079ac5c700 1 -- 172.21.15.78:0/4113875162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07940b0eb0 msgr2=0x7f07940c87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:28.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.708+0000 7f079ac5c700 1 --2- 172.21.15.78:0/4113875162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07940b0eb0 0x7f07940c87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:28.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.708+0000 7f079ac5c700 1 -- 172.21.15.78:0/4113875162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07900094b0 con 0x7f07941655d0 2023-10-11T09:48:28.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.708+0000 7f079ac5c700 1 --2- 172.21.15.78:0/4113875162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07941655d0 0x7f07941f67e0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f078c00caa0 tx=0x7f078c00cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:28.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.708+0000 7f078affd700 1 -- 172.21.15.78:0/4113875162 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c00a7b0 con 0x7f07941655d0 2023-10-11T09:48:28.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.708+0000 7f078affd700 1 -- 172.21.15.78:0/4113875162 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078c00a910 con 0x7f07941655d0 2023-10-11T09:48:28.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.708+0000 7f079b45d700 1 -- 172.21.15.78:0/4113875162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07940c7fe0 con 0x7f07941655d0 2023-10-11T09:48:28.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.708+0000 7f078affd700 1 -- 172.21.15.78:0/4113875162 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c00aca0 con 0x7f07941655d0 2023-10-11T09:48:28.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.708+0000 7f079b45d700 1 -- 172.21.15.78:0/4113875162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07940c2650 con 0x7f07941655d0 2023-10-11T09:48:28.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.708+0000 7f079b45d700 1 -- 172.21.15.78:0/4113875162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07940c2840 con 0x7f07941655d0 2023-10-11T09:48:28.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.712+0000 7f078affd700 1 -- 172.21.15.78:0/4113875162 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f078c01b3b0 con 0x7f07941655d0 2023-10-11T09:48:28.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.712+0000 7f078affd700 1 --2- 172.21.15.78:0/4113875162 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0778065ed0 0x7f0778068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:28.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.712+0000 7f078affd700 1 -- 172.21.15.78:0/4113875162 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f078c083270 con 0x7f07941655d0 2023-10-11T09:48:28.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.716+0000 7f079a45b700 1 --2- 172.21.15.78:0/4113875162 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0778065ed0 0x7f0778068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:28.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.716+0000 7f079a45b700 1 --2- 172.21.15.78:0/4113875162 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0778065ed0 0x7f0778068380 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f07900090b0 tx=0x7f079001c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:28.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.720+0000 7f078affd700 1 -- 172.21.15.78:0/4113875162 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f078c0508b0 con 0x7f07941655d0 2023-10-11T09:48:28.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:28.892+0000 7f079b45d700 1 -- 172.21.15.78:0/4113875162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_extra", "pg_num_min": 64} v 0) v1 -- 0x7f0794065db0 con 0x7f07941655d0 2023-10-11T09:48:29.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.352+0000 7f078affd700 1 -- 172.21.15.78:0/4113875162 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_extra", "pg_num_min": 64}]=0 pool 'cephfs_data_extra' created v84) v1 ==== 142+0+0 (secure 0 0 0) 0x7f078c053ed0 con 0x7f07941655d0 2023-10-11T09:48:29.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.364+0000 7f079b45d700 1 -- 172.21.15.78:0/4113875162 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0778065ed0 msgr2=0x7f0778068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:29.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.364+0000 7f079b45d700 1 --2- 172.21.15.78:0/4113875162 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0778065ed0 0x7f0778068380 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f07900090b0 tx=0x7f079001c040 comp rx=0 tx=0).stop 2023-10-11T09:48:29.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.364+0000 7f079b45d700 1 -- 172.21.15.78:0/4113875162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07941655d0 msgr2=0x7f07941f67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:29.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.364+0000 7f079b45d700 1 --2- 172.21.15.78:0/4113875162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07941655d0 0x7f07941f67e0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f078c00caa0 tx=0x7f078c00cb80 comp rx=0 tx=0).stop 2023-10-11T09:48:29.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.364+0000 7f079b45d700 1 -- 172.21.15.78:0/4113875162 shutdown_connections 2023-10-11T09:48:29.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.364+0000 7f079b45d700 1 --2- 172.21.15.78:0/4113875162 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0778065ed0 0x7f0778068380 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:29.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.364+0000 7f079b45d700 1 --2- 172.21.15.78:0/4113875162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07941655d0 0x7f07941f67e0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:29.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.364+0000 7f079b45d700 1 --2- 172.21.15.78:0/4113875162 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07940cdbd0 0x7f07941572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:29.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.364+0000 7f079b45d700 1 --2- 172.21.15.78:0/4113875162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07940b0eb0 0x7f07940c87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:29.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.364+0000 7f079b45d700 1 -- 172.21.15.78:0/4113875162 >> 172.21.15.78:0/4113875162 conn(0x7f079406acf0 msgr2=0x7f07940cdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:29.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.364+0000 7f079b45d700 1 -- 172.21.15.78:0/4113875162 shutdown_connections 2023-10-11T09:48:29.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.364+0000 7f079b45d700 1 -- 172.21.15.78:0/4113875162 wait complete. 2023-10-11T09:48:29.375 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data_extra' created 2023-10-11T09:48:29.393 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_extra 2023-10-11T09:48:29.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.560+0000 7f40c32f5700 1 Processor -- start 2023-10-11T09:48:29.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.560+0000 7f40c32f5700 1 -- start start 2023-10-11T09:48:29.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.564+0000 7f40c32f5700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f40bc146fe0 0x7f40bc06bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:29.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.564+0000 7f40c32f5700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f40bc161d90 0x7f40bc0b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:29.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.564+0000 7f40c32f5700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40bc06ccb0 con 0x7f40bc0b3520 2023-10-11T09:48:29.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.564+0000 7f40c32f5700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40bc06c410 con 0x7f40bc161d90 2023-10-11T09:48:29.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.564+0000 7f40c32f5700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40bc06c6f0 con 0x7f40bc146fe0 2023-10-11T09:48:29.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.564+0000 7f40c1af2700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f40bc161d90 0x7f40bc0b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:29.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.564+0000 7f40c22f3700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f40bc146fe0 0x7f40bc06bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:29.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.564+0000 7f40c22f3700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f40bc146fe0 0x7f40bc06bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:39436/0 (socket says 172.21.15.78:39436) 2023-10-11T09:48:29.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.564+0000 7f40c22f3700 1 -- 172.21.15.78:0/1043812693 learned_addr learned my addr 172.21.15.78:0/1043812693 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:29.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.564+0000 7f40c1af2700 1 -- 172.21.15.78:0/1043812693 >> v2:172.21.15.133:3300/0 conn(0x7f40bc146fe0 msgr2=0x7f40bc06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:29.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.564+0000 7f40c1af2700 1 --2- 172.21.15.78:0/1043812693 >> v2:172.21.15.133:3300/0 conn(0x7f40bc146fe0 0x7f40bc06bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:29.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.564+0000 7f40c1af2700 1 -- 172.21.15.78:0/1043812693 >> v1:172.21.15.133:6789/0 conn(0x7f40bc0b3520 legacy=0x7f40bc1647e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:29.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.564+0000 7f40c1af2700 1 -- 172.21.15.78:0/1043812693 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40bc06c9d0 con 0x7f40bc161d90 2023-10-11T09:48:29.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.564+0000 7f40c22f3700 1 --2- 172.21.15.78:0/1043812693 >> v2:172.21.15.133:3300/0 conn(0x7f40bc146fe0 0x7f40bc06bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:48:29.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.564+0000 7f40c1af2700 1 --2- 172.21.15.78:0/1043812693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40bc161d90 0x7f40bc0b2fe0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f40bc06f7f0 tx=0x7f40ac009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3538fb248afcfff1 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:29.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.564+0000 7f40c12f1700 1 -- 172.21.15.78:0/1043812693 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ac011830 con 0x7f40bc161d90 2023-10-11T09:48:29.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.568+0000 7f40c12f1700 1 -- 172.21.15.78:0/1043812693 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40ac011990 con 0x7f40bc161d90 2023-10-11T09:48:29.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.568+0000 7f40c12f1700 1 -- 172.21.15.78:0/1043812693 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ac011d20 con 0x7f40bc161d90 2023-10-11T09:48:29.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.568+0000 7f40c32f5700 1 -- 172.21.15.78:0/1043812693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40bc161d90 msgr2=0x7f40bc0b2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:29.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.568+0000 7f40c32f5700 1 --2- 172.21.15.78:0/1043812693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40bc161d90 0x7f40bc0b2fe0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f40bc06f7f0 tx=0x7f40ac009b30 comp rx=0 tx=0).stop 2023-10-11T09:48:29.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.568+0000 7f40c32f5700 1 -- 172.21.15.78:0/1043812693 shutdown_connections 2023-10-11T09:48:29.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.568+0000 7f40c32f5700 1 --2- 172.21.15.78:0/1043812693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40bc161d90 0x7f40bc0b2fe0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:29.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.568+0000 7f40c32f5700 1 --2- 172.21.15.78:0/1043812693 >> v2:172.21.15.133:3300/0 conn(0x7f40bc146fe0 0x7f40bc06bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:29.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.568+0000 7f40c32f5700 1 -- 172.21.15.78:0/1043812693 >> 172.21.15.78:0/1043812693 conn(0x7f40bc06a060 msgr2=0x7f40bc06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:29.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.568+0000 7f40c32f5700 1 -- 172.21.15.78:0/1043812693 shutdown_connections 2023-10-11T09:48:29.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.568+0000 7f40c32f5700 1 -- 172.21.15.78:0/1043812693 wait complete. 2023-10-11T09:48:29.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.572+0000 7f40c32f5700 1 Processor -- start 2023-10-11T09:48:29.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.572+0000 7f40c32f5700 1 -- start start 2023-10-11T09:48:29.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.572+0000 7f40c32f5700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40bc0b3520 0x7f40bc0aed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:29.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.572+0000 7f40c32f5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40bc146fe0 0x7f40bc145fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:29.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.572+0000 7f40c32f5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40bc161d90 0x7f40bc0a8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:29.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.572+0000 7f40c32f5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc0cd680 con 0x7f40bc146fe0 2023-10-11T09:48:29.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.572+0000 7f40c32f5700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc0a8a10 con 0x7f40bc161d90 2023-10-11T09:48:29.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.572+0000 7f40c32f5700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc0ccce0 con 0x7f40bc0b3520 2023-10-11T09:48:29.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.572+0000 7f40c1af2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40bc146fe0 0x7f40bc145fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:29.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.572+0000 7f40c1af2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40bc146fe0 0x7f40bc145fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57420/0 (socket says 172.21.15.78:57420) 2023-10-11T09:48:29.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.572+0000 7f40c1af2700 1 -- 172.21.15.78:0/1871425650 learned_addr learned my addr 172.21.15.78:0/1871425650 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:29.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.572+0000 7f40c1af2700 1 -- 172.21.15.78:0/1871425650 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40bc0b3520 msgr2=0x7f40bc0aed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:29.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.572+0000 7f40c1af2700 1 --2- 172.21.15.78:0/1871425650 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40bc0b3520 0x7f40bc0aed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:29.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.572+0000 7f40c2af4700 1 --2- 172.21.15.78:0/1871425650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40bc161d90 0x7f40bc0a8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:29.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.572+0000 7f40c1af2700 1 -- 172.21.15.78:0/1871425650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40bc161d90 msgr2=0x7f40bc0a8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:29.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.572+0000 7f40c1af2700 1 --2- 172.21.15.78:0/1871425650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40bc161d90 0x7f40bc0a8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:29.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.572+0000 7f40c1af2700 1 -- 172.21.15.78:0/1871425650 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b8009570 con 0x7f40bc146fe0 2023-10-11T09:48:29.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.572+0000 7f40c2af4700 1 --2- 172.21.15.78:0/1871425650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40bc161d90 0x7f40bc0a8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:48:29.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.572+0000 7f40c1af2700 1 --2- 172.21.15.78:0/1871425650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40bc146fe0 0x7f40bc145fd0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f40ac014040 tx=0x7f40ac004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:29.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.572+0000 7f40b2ffd700 1 -- 172.21.15.78:0/1871425650 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ac0036c0 con 0x7f40bc146fe0 2023-10-11T09:48:29.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.576+0000 7f40b2ffd700 1 -- 172.21.15.78:0/1871425650 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40ac01e070 con 0x7f40bc146fe0 2023-10-11T09:48:29.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.576+0000 7f40c32f5700 1 -- 172.21.15.78:0/1871425650 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40ac0094b0 con 0x7f40bc146fe0 2023-10-11T09:48:29.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.576+0000 7f40b2ffd700 1 -- 172.21.15.78:0/1871425650 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ac003a40 con 0x7f40bc146fe0 2023-10-11T09:48:29.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.576+0000 7f40c32f5700 1 -- 172.21.15.78:0/1871425650 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40bc0cd2a0 con 0x7f40bc146fe0 2023-10-11T09:48:29.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.576+0000 7f40c32f5700 1 -- 172.21.15.78:0/1871425650 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40bc0cb050 con 0x7f40bc146fe0 2023-10-11T09:48:29.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.576+0000 7f40b2ffd700 1 -- 172.21.15.78:0/1871425650 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f40ac003ba0 con 0x7f40bc146fe0 2023-10-11T09:48:29.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.580+0000 7f40b2ffd700 1 --2- 172.21.15.78:0/1871425650 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f40a0065ed0 0x7f40a0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:29.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.580+0000 7f40b2ffd700 1 -- 172.21.15.78:0/1871425650 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6434+0+0 (secure 0 0 0) 0x7f40ac0849a0 con 0x7f40bc146fe0 2023-10-11T09:48:29.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.580+0000 7f40c22f3700 1 --2- 172.21.15.78:0/1871425650 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f40a0065ed0 0x7f40a0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:29.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.580+0000 7f40c22f3700 1 --2- 172.21.15.78:0/1871425650 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f40a0065ed0 0x7f40a0068380 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f40b8006fd0 tx=0x7f40b8006f40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:29.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.588+0000 7f40b2ffd700 1 -- 172.21.15.78:0/1871425650 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f40ac051f00 con 0x7f40bc146fe0 2023-10-11T09:48:29.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.812+0000 7f40c32f5700 1 -- 172.21.15.78:0/1871425650 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_extra"} v 0) v1 -- 0x7f40bc065d40 con 0x7f40bc146fe0 2023-10-11T09:48:29.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.824+0000 7f40b2ffd700 1 -- 172.21.15.78:0/1871425650 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_extra"}]=0 added data pool 13 to fsmap v59) v1 ==== 141+0+0 (secure 0 0 0) 0x7f40ac0200b0 con 0x7f40bc146fe0 2023-10-11T09:48:29.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.832+0000 7f40c32f5700 1 -- 172.21.15.78:0/1871425650 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f40a0065ed0 msgr2=0x7f40a0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:29.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.832+0000 7f40c32f5700 1 --2- 172.21.15.78:0/1871425650 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f40a0065ed0 0x7f40a0068380 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f40b8006fd0 tx=0x7f40b8006f40 comp rx=0 tx=0).stop 2023-10-11T09:48:29.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.832+0000 7f40c32f5700 1 -- 172.21.15.78:0/1871425650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40bc146fe0 msgr2=0x7f40bc145fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:29.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.832+0000 7f40c32f5700 1 --2- 172.21.15.78:0/1871425650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40bc146fe0 0x7f40bc145fd0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f40ac014040 tx=0x7f40ac004770 comp rx=0 tx=0).stop 2023-10-11T09:48:29.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.832+0000 7f40c32f5700 1 -- 172.21.15.78:0/1871425650 shutdown_connections 2023-10-11T09:48:29.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.832+0000 7f40c32f5700 1 --2- 172.21.15.78:0/1871425650 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f40a0065ed0 0x7f40a0068380 secure :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f40b8006fd0 tx=0x7f40b8006f40 comp rx=0 tx=0).stop 2023-10-11T09:48:29.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.832+0000 7f40c32f5700 1 --2- 172.21.15.78:0/1871425650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40bc161d90 0x7f40bc0a8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:29.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.832+0000 7f40c32f5700 1 --2- 172.21.15.78:0/1871425650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40bc146fe0 0x7f40bc145fd0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:29.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.832+0000 7f40c32f5700 1 --2- 172.21.15.78:0/1871425650 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40bc0b3520 0x7f40bc0aed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:29.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.832+0000 7f40c32f5700 1 -- 172.21.15.78:0/1871425650 >> 172.21.15.78:0/1871425650 conn(0x7f40bc06a060 msgr2=0x7f40bc0ae660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:29.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.832+0000 7f40c32f5700 1 -- 172.21.15.78:0/1871425650 shutdown_connections 2023-10-11T09:48:29.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:29.836+0000 7f40c32f5700 1 -- 172.21.15.78:0/1871425650 wait complete. 2023-10-11T09:48:29.848 INFO:teuthology.orchestra.run.smithi078.stderr:added data pool 13 to fsmap 2023-10-11T09:48:29.866 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:48:30.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.024+0000 7f58b8b97700 1 Processor -- start 2023-10-11T09:48:30.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.024+0000 7f58b8b97700 1 -- start start 2023-10-11T09:48:30.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.028+0000 7f58b8b97700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58b415d050 0x7f58b40cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:30.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.028+0000 7f58b8b97700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f58b4160c60 0x7f58b415d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:30.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.028+0000 7f58b8b97700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f58b415d960 0x7f58b40b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:30.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.028+0000 7f58b8b97700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58b4160210 con 0x7f58b415d960 2023-10-11T09:48:30.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.028+0000 7f58b8b97700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58b40ccd60 con 0x7f58b4160c60 2023-10-11T09:48:30.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.028+0000 7f58b8b97700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b40ccfc0 con 0x7f58b415d050 2023-10-11T09:48:30.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.028+0000 7f58b3fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f58b415d960 0x7f58b40b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:30.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.028+0000 7f58b3fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f58b415d960 0x7f58b40b06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57426/0 (socket says 172.21.15.78:57426) 2023-10-11T09:48:30.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.028+0000 7f58b3fff700 1 -- 172.21.15.78:0/2935535394 learned_addr learned my addr 172.21.15.78:0/2935535394 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:30.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.028+0000 7f58b37fe700 1 --2- 172.21.15.78:0/2935535394 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58b415d050 0x7f58b40cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:30.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.028+0000 7f58aaffd700 1 --2- 172.21.15.78:0/2935535394 >> v2:172.21.15.133:3300/0 conn(0x7f58b4160c60 0x7f58b415d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:30.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.028+0000 7f58b3fff700 1 -- 172.21.15.78:0/2935535394 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58b415d050 msgr2=0x7f58b40cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:30.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.028+0000 7f58b3fff700 1 --2- 172.21.15.78:0/2935535394 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58b415d050 0x7f58b40cc820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:30.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.028+0000 7f58b3fff700 1 -- 172.21.15.78:0/2935535394 >> v2:172.21.15.133:3300/0 conn(0x7f58b4160c60 msgr2=0x7f58b415d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:30.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.028+0000 7f58b3fff700 1 --2- 172.21.15.78:0/2935535394 >> v2:172.21.15.133:3300/0 conn(0x7f58b4160c60 0x7f58b415d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:30.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.028+0000 7f58b3fff700 1 -- 172.21.15.78:0/2935535394 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b40cd2a0 con 0x7f58b415d960 2023-10-11T09:48:30.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.028+0000 7f58aaffd700 1 --2- 172.21.15.78:0/2935535394 >> v2:172.21.15.133:3300/0 conn(0x7f58b4160c60 0x7f58b415d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:48:30.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.028+0000 7f58b3fff700 1 --2- 172.21.15.78:0/2935535394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58b415d960 0x7f58b40b06a0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f58b406f960 tx=0x7f58ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43862a09ac4c4f1b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:30.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.028+0000 7f58b2ffd700 1 -- 172.21.15.78:0/2935535394 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac00e070 con 0x7f58b415d960 2023-10-11T09:48:30.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.028+0000 7f58b2ffd700 1 -- 172.21.15.78:0/2935535394 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ac00b9c0 con 0x7f58b415d960 2023-10-11T09:48:30.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.028+0000 7f58b2ffd700 1 -- 172.21.15.78:0/2935535394 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac00bd70 con 0x7f58b415d960 2023-10-11T09:48:30.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.028+0000 7f58b8b97700 1 -- 172.21.15.78:0/2935535394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58b415d960 msgr2=0x7f58b40b06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:30.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.028+0000 7f58b8b97700 1 --2- 172.21.15.78:0/2935535394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58b415d960 0x7f58b40b06a0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f58b406f960 tx=0x7f58ac00b330 comp rx=0 tx=0).stop 2023-10-11T09:48:30.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.028+0000 7f58b8b97700 1 -- 172.21.15.78:0/2935535394 shutdown_connections 2023-10-11T09:48:30.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.028+0000 7f58b8b97700 1 --2- 172.21.15.78:0/2935535394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58b415d960 0x7f58b40b06a0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:30.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.028+0000 7f58b8b97700 1 --2- 172.21.15.78:0/2935535394 >> v2:172.21.15.133:3300/0 conn(0x7f58b4160c60 0x7f58b415d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:30.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.028+0000 7f58b8b97700 1 --2- 172.21.15.78:0/2935535394 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58b415d050 0x7f58b40cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:30.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.028+0000 7f58b8b97700 1 -- 172.21.15.78:0/2935535394 >> 172.21.15.78:0/2935535394 conn(0x7f58b40cea60 msgr2=0x7f58b40cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:30.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.032+0000 7f58b8b97700 1 -- 172.21.15.78:0/2935535394 shutdown_connections 2023-10-11T09:48:30.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.032+0000 7f58b8b97700 1 -- 172.21.15.78:0/2935535394 wait complete. 2023-10-11T09:48:30.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.032+0000 7f58b8b97700 1 Processor -- start 2023-10-11T09:48:30.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.036+0000 7f58b8b97700 1 -- start start 2023-10-11T09:48:30.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.036+0000 7f58b8b97700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58b415d050 0x7f58b4140730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:30.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.036+0000 7f58b8b97700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58b415d960 0x7f58b40b1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:30.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.036+0000 7f58b8b97700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58b4160c60 0x7f58b4155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:30.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.036+0000 7f58b8b97700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b41f5ed0 con 0x7f58b415d960 2023-10-11T09:48:30.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.036+0000 7f58b8b97700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b41f5290 con 0x7f58b4160c60 2023-10-11T09:48:30.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.036+0000 7f58b8b97700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b4140c70 con 0x7f58b415d050 2023-10-11T09:48:30.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.036+0000 7f58aaffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58b415d960 0x7f58b40b1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:30.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.036+0000 7f58aaffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58b415d960 0x7f58b40b1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57438/0 (socket says 172.21.15.78:57438) 2023-10-11T09:48:30.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.036+0000 7f58aaffd700 1 -- 172.21.15.78:0/260896158 learned_addr learned my addr 172.21.15.78:0/260896158 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:30.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.036+0000 7f58b37fe700 1 --2- 172.21.15.78:0/260896158 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58b415d050 0x7f58b4140730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:30.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.036+0000 7f58b3fff700 1 --2- 172.21.15.78:0/260896158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58b4160c60 0x7f58b4155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:30.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.036+0000 7f58aaffd700 1 -- 172.21.15.78:0/260896158 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58b415d050 msgr2=0x7f58b4140730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:30.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.036+0000 7f58aaffd700 1 --2- 172.21.15.78:0/260896158 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58b415d050 0x7f58b4140730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:30.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.036+0000 7f58aaffd700 1 -- 172.21.15.78:0/260896158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58b4160c60 msgr2=0x7f58b4155030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:30.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.036+0000 7f58aaffd700 1 --2- 172.21.15.78:0/260896158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58b4160c60 0x7f58b4155030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:30.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.036+0000 7f58aaffd700 1 -- 172.21.15.78:0/260896158 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ac009b30 con 0x7f58b415d960 2023-10-11T09:48:30.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.036+0000 7f58aaffd700 1 --2- 172.21.15.78:0/260896158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58b415d960 0x7f58b40b1c50 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f589c00c770 tx=0x7f589c00ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:30.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.036+0000 7f58b0ff9700 1 -- 172.21.15.78:0/260896158 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c011800 con 0x7f58b415d960 2023-10-11T09:48:30.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.036+0000 7f58b0ff9700 1 -- 172.21.15.78:0/260896158 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589c011960 con 0x7f58b415d960 2023-10-11T09:48:30.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.036+0000 7f58b0ff9700 1 -- 172.21.15.78:0/260896158 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c011d10 con 0x7f58b415d960 2023-10-11T09:48:30.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.036+0000 7f58b8b97700 1 -- 172.21.15.78:0/260896158 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58b41f5530 con 0x7f58b415d960 2023-10-11T09:48:30.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.040+0000 7f58b8b97700 1 -- 172.21.15.78:0/260896158 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b41f5af0 con 0x7f58b415d960 2023-10-11T09:48:30.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.040+0000 7f58b8b97700 1 -- 172.21.15.78:0/260896158 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58b4065d40 con 0x7f58b415d960 2023-10-11T09:48:30.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.040+0000 7f58b0ff9700 1 -- 172.21.15.78:0/260896158 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f589c018360 con 0x7f58b415d960 2023-10-11T09:48:30.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.044+0000 7f58b0ff9700 1 --2- 172.21.15.78:0/260896158 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5894065ed0 0x7f5894068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:30.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.044+0000 7f58b0ff9700 1 -- 172.21.15.78:0/260896158 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f589c083810 con 0x7f58b415d960 2023-10-11T09:48:30.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.044+0000 7f58b37fe700 1 --2- 172.21.15.78:0/260896158 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5894065ed0 0x7f5894068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:30.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.044+0000 7f58b37fe700 1 --2- 172.21.15.78:0/260896158 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5894065ed0 0x7f5894068380 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f58b4153960 tx=0x7f58a4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:30.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.048+0000 7f58b0ff9700 1 -- 172.21.15.78:0/260896158 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f589c050cb0 con 0x7f58b415d960 2023-10-11T09:48:30.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.256+0000 7f58b8b97700 1 -- 172.21.15.78:0/260896158 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58b40b1310 con 0x7f58b415d960 2023-10-11T09:48:30.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.256+0000 7f58b0ff9700 1 -- 172.21.15.78:0/260896158 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4653 (secure 0 0 0) 0x7f589c0542d0 con 0x7f58b415d960 2023-10-11T09:48:30.266 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:30.267 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":59,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6961,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/1080544187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1080544187},{"type":"v1","addr":"172.21.15.133:6834","nonce":1080544187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/367194191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":367194191},{"type":"v1","addr":"172.21.15.133:6835","nonce":367194191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":7224,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/3463904224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3463904224},{"type":"v1","addr":"172.21.15.78:6836","nonce":3463904224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:48:20.022298+0000","modified":"2023-10-11T09:48:29.819506+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7227},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7227":{"gid":7227,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6837/951312831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":951312831},{"type":"v1","addr":"172.21.15.78:6837","nonce":951312831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12,13],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-11T09:48:30.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.260+0000 7f58b8b97700 1 -- 172.21.15.78:0/260896158 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5894065ed0 msgr2=0x7f5894068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:30.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.260+0000 7f58b8b97700 1 --2- 172.21.15.78:0/260896158 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5894065ed0 0x7f5894068380 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f58b4153960 tx=0x7f58a4008040 comp rx=0 tx=0).stop 2023-10-11T09:48:30.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.260+0000 7f58b8b97700 1 -- 172.21.15.78:0/260896158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58b415d960 msgr2=0x7f58b40b1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:30.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.260+0000 7f58b8b97700 1 --2- 172.21.15.78:0/260896158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58b415d960 0x7f58b40b1c50 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f589c00c770 tx=0x7f589c00ca80 comp rx=0 tx=0).stop 2023-10-11T09:48:30.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.260+0000 7f58b8b97700 1 -- 172.21.15.78:0/260896158 shutdown_connections 2023-10-11T09:48:30.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.260+0000 7f58b8b97700 1 --2- 172.21.15.78:0/260896158 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5894065ed0 0x7f5894068380 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:30.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.260+0000 7f58b8b97700 1 --2- 172.21.15.78:0/260896158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58b4160c60 0x7f58b4155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:30.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.260+0000 7f58b8b97700 1 --2- 172.21.15.78:0/260896158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58b415d960 0x7f58b40b1c50 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:30.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.260+0000 7f58b8b97700 1 --2- 172.21.15.78:0/260896158 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58b415d050 0x7f58b4140730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:30.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.260+0000 7f58b8b97700 1 -- 172.21.15.78:0/260896158 >> 172.21.15.78:0/260896158 conn(0x7f58b40cea60 msgr2=0x7f58b406c950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:30.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.260+0000 7f58b8b97700 1 -- 172.21.15.78:0/260896158 shutdown_connections 2023-10-11T09:48:30.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.260+0000 7f58b8b97700 1 -- 172.21.15.78:0/260896158 wait complete. 2023-10-11T09:48:30.271 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 59 2023-10-11T09:48:30.290 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:48:30.345 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:48:30.352 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:48:30.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.452+0000 7f4e3d590700 1 Processor -- start 2023-10-11T09:48:30.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.452+0000 7f4e3d590700 1 -- start start 2023-10-11T09:48:30.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.452+0000 7f4e3d590700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e3815b8e0 0x7f4e3806d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:30.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.452+0000 7f4e3d590700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e3815e010 con 0x7f4e3806d610 2023-10-11T09:48:30.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.452+0000 7f4e3d590700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e3815e280 con 0x7f4e380ca950 2023-10-11T09:48:30.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.452+0000 7f4e3d590700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3815e560 con 0x7f4e3815b8e0 2023-10-11T09:48:30.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.452+0000 7f4e377fe700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f4e3806d610 0x7f4e380ca240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:42924/0 (socket says 172.21.15.78:42924) 2023-10-11T09:48:30.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.452+0000 7f4e377fe700 1 -- 172.21.15.78:0/2692073143 learned_addr learned my addr 172.21.15.78:0/2692073143 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:30.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.456+0000 7f4e37fff700 1 --2- 172.21.15.78:0/2692073143 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e3815b8e0 0x7f4e3806d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:30.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.456+0000 7f4e36ffd700 1 -- 172.21.15.78:0/2692073143 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1220862670 0 0) 0x7f4e3815e010 con 0x7f4e3806d610 2023-10-11T09:48:30.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.456+0000 7f4e36ffd700 1 -- 172.21.15.78:0/2692073143 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e200035d0 con 0x7f4e3806d610 2023-10-11T09:48:30.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.456+0000 7f4e36ffd700 1 -- 172.21.15.78:0/2692073143 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4055041532 0 0) 0x7f4e3815e280 con 0x7f4e380ca950 2023-10-11T09:48:30.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.456+0000 7f4e36ffd700 1 -- 172.21.15.78:0/2692073143 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e3815e010 con 0x7f4e380ca950 2023-10-11T09:48:30.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.456+0000 7f4e36ffd700 1 -- 172.21.15.78:0/2692073143 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3590627286 0 0) 0x7f4e200035d0 con 0x7f4e3806d610 2023-10-11T09:48:30.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.456+0000 7f4e36ffd700 1 -- 172.21.15.78:0/2692073143 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e3815b8e0 msgr2=0x7f4e3806d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:30.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.456+0000 7f4e36ffd700 1 --2- 172.21.15.78:0/2692073143 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e3815b8e0 0x7f4e3806d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:30.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.456+0000 7f4e36ffd700 1 -- 172.21.15.78:0/2692073143 >> v1:172.21.15.133:6789/0 conn(0x7f4e380ca950 legacy=0x7f4e3813fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:30.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.456+0000 7f4e36ffd700 1 -- 172.21.15.78:0/2692073143 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e3815e840 con 0x7f4e3806d610 2023-10-11T09:48:30.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.456+0000 7f4e36ffd700 1 -- 172.21.15.78:0/2692073143 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f4e2c001670 con 0x7f4e3806d610 2023-10-11T09:48:30.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.456+0000 7f4e36ffd700 1 -- 172.21.15.78:0/2692073143 >> v1:172.21.15.78:6789/0 conn(0x7f4e3806d610 legacy=0x7f4e380ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:30.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.456+0000 7f4e36ffd700 1 --2- 172.21.15.78:0/2692073143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e20003be0 0x7f4e20003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:30.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.456+0000 7f4e36ffd700 1 --2- 172.21.15.78:0/2692073143 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e3815b8e0 0x7f4e20004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:30.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.456+0000 7f4e36ffd700 1 --2- 172.21.15.78:0/2692073143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e20004db0 0x7f4e20005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:30.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.456+0000 7f4e36ffd700 1 -- 172.21.15.78:0/2692073143 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3815e560 con 0x7f4e20004db0 2023-10-11T09:48:30.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.456+0000 7f4e36ffd700 1 -- 172.21.15.78:0/2692073143 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2000bae0 con 0x7f4e20003be0 2023-10-11T09:48:30.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.456+0000 7f4e36ffd700 1 -- 172.21.15.78:0/2692073143 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2000bc50 con 0x7f4e3815b8e0 2023-10-11T09:48:30.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.456+0000 7f4e37fff700 1 --2- 172.21.15.78:0/2692073143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e20003be0 0x7f4e20003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:30.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.456+0000 7f4e3cd8f700 1 --2- 172.21.15.78:0/2692073143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e20004db0 0x7f4e20005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:30.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.456+0000 7f4e377fe700 1 --2- 172.21.15.78:0/2692073143 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e3815b8e0 0x7f4e20004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:30.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.456+0000 7f4e37fff700 1 -- 172.21.15.78:0/2692073143 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e3815b8e0 msgr2=0x7f4e20004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:30.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.456+0000 7f4e37fff700 1 --2- 172.21.15.78:0/2692073143 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e3815b8e0 0x7f4e20004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:30.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.456+0000 7f4e37fff700 1 -- 172.21.15.78:0/2692073143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e20004db0 msgr2=0x7f4e20005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:30.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.456+0000 7f4e37fff700 1 --2- 172.21.15.78:0/2692073143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e20004db0 0x7f4e20005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:30.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.456+0000 7f4e37fff700 1 -- 172.21.15.78:0/2692073143 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e3815e840 con 0x7f4e20003be0 2023-10-11T09:48:30.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.456+0000 7f4e37fff700 1 --2- 172.21.15.78:0/2692073143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e20003be0 0x7f4e20003fd0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009ba0 tx=0x7f4e200058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=209b8a4a84babdc8 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:30.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.456+0000 7f4e36ffd700 1 -- 172.21.15.78:0/2692073143 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2800e6c0 con 0x7f4e20003be0 2023-10-11T09:48:30.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.456+0000 7f4e36ffd700 1 -- 172.21.15.78:0/2692073143 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e2800e820 con 0x7f4e20003be0 2023-10-11T09:48:30.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.456+0000 7f4e36ffd700 1 -- 172.21.15.78:0/2692073143 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2800eb90 con 0x7f4e20003be0 2023-10-11T09:48:30.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.460+0000 7f4e3d590700 1 -- 172.21.15.78:0/2692073143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e20003be0 msgr2=0x7f4e20003fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:30.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.460+0000 7f4e3d590700 1 --2- 172.21.15.78:0/2692073143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e20003be0 0x7f4e20003fd0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009ba0 tx=0x7f4e200058a0 comp rx=0 tx=0).stop 2023-10-11T09:48:30.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.460+0000 7f4e3cd8f700 1 -- 172.21.15.78:0/2692073143 reap_dead start 2023-10-11T09:48:30.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.460+0000 7f4e3d590700 1 -- 172.21.15.78:0/2692073143 shutdown_connections 2023-10-11T09:48:30.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.460+0000 7f4e3d590700 1 -- 172.21.15.78:0/2692073143 >> 172.21.15.78:0/2692073143 conn(0x7f4e380cea60 msgr2=0x7f4e380cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:30.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.460+0000 7f4e3d590700 1 -- 172.21.15.78:0/2692073143 shutdown_connections 2023-10-11T09:48:30.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.460+0000 7f4e3d590700 1 -- 172.21.15.78:0/2692073143 wait complete. 2023-10-11T09:48:30.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.460+0000 7f4e3d590700 1 Processor -- start 2023-10-11T09:48:30.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.464+0000 7f4e3d590700 1 -- start start 2023-10-11T09:48:30.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.464+0000 7f4e3d590700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e20003be0 0x7f4e380ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:30.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.464+0000 7f4e3d590700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e380c3b40 0x7f4e380cb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:30.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.464+0000 7f4e3d590700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e38159820 0x7f4e38159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:30.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.464+0000 7f4e3d590700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e38141da0 con 0x7f4e20003be0 2023-10-11T09:48:30.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.464+0000 7f4e3d590700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3815a200 con 0x7f4e380c3b40 2023-10-11T09:48:30.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.464+0000 7f4e3d590700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e38141120 con 0x7f4e38159820 2023-10-11T09:48:30.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.464+0000 7f4e37fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e20003be0 0x7f4e380ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:30.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.464+0000 7f4e37fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e20003be0 0x7f4e380ccc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57452/0 (socket says 172.21.15.78:57452) 2023-10-11T09:48:30.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.464+0000 7f4e37fff700 1 -- 172.21.15.78:0/167289236 learned_addr learned my addr 172.21.15.78:0/167289236 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:30.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.464+0000 7f4e377fe700 1 --2- 172.21.15.78:0/167289236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e380c3b40 0x7f4e380cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:30.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.464+0000 7f4e3cd8f700 1 --2- 172.21.15.78:0/167289236 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e38159820 0x7f4e38159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:30.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.464+0000 7f4e37fff700 1 -- 172.21.15.78:0/167289236 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e38159820 msgr2=0x7f4e38159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:30.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.464+0000 7f4e37fff700 1 --2- 172.21.15.78:0/167289236 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e38159820 0x7f4e38159cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:30.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.464+0000 7f4e37fff700 1 -- 172.21.15.78:0/167289236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e380c3b40 msgr2=0x7f4e380cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:30.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.464+0000 7f4e37fff700 1 --2- 172.21.15.78:0/167289236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e380c3b40 0x7f4e380cb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:30.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.464+0000 7f4e37fff700 1 -- 172.21.15.78:0/167289236 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e38141400 con 0x7f4e20003be0 2023-10-11T09:48:30.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.464+0000 7f4e37fff700 1 --2- 172.21.15.78:0/167289236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e20003be0 0x7f4e380ccc50 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f4e280090b0 tx=0x7f4e280047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:30.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.464+0000 7f4e34ff9700 1 -- 172.21.15.78:0/167289236 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2800e6c0 con 0x7f4e20003be0 2023-10-11T09:48:30.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.464+0000 7f4e34ff9700 1 -- 172.21.15.78:0/167289236 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e28013e10 con 0x7f4e20003be0 2023-10-11T09:48:30.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.464+0000 7f4e3d590700 1 -- 172.21.15.78:0/167289236 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e381416e0 con 0x7f4e20003be0 2023-10-11T09:48:30.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.464+0000 7f4e34ff9700 1 -- 172.21.15.78:0/167289236 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e28004270 con 0x7f4e20003be0 2023-10-11T09:48:30.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.468+0000 7f4e3d590700 1 -- 172.21.15.78:0/167289236 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e381421c0 con 0x7f4e20003be0 2023-10-11T09:48:30.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.468+0000 7f4e3d590700 1 -- 172.21.15.78:0/167289236 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e38065d40 con 0x7f4e20003be0 2023-10-11T09:48:30.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.468+0000 7f4e34ff9700 1 -- 172.21.15.78:0/167289236 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f4e280043d0 con 0x7f4e20003be0 2023-10-11T09:48:30.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.468+0000 7f4e34ff9700 1 --2- 172.21.15.78:0/167289236 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4e24065ed0 0x7f4e24068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:30.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.468+0000 7f4e34ff9700 1 -- 172.21.15.78:0/167289236 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f4e28085480 con 0x7f4e20003be0 2023-10-11T09:48:30.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.472+0000 7f4e377fe700 1 --2- 172.21.15.78:0/167289236 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4e24065ed0 0x7f4e24068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:30.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.472+0000 7f4e377fe700 1 --2- 172.21.15.78:0/167289236 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4e24065ed0 0x7f4e24068380 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4e3813f910 tx=0x7f4e2c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:30.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.476+0000 7f4e34ff9700 1 -- 172.21.15.78:0/167289236 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f4e280528a0 con 0x7f4e20003be0 2023-10-11T09:48:30.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.652+0000 7f4e3d590700 1 -- 172.21.15.78:0/167289236 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e381439c0 con 0x7f4e20003be0 2023-10-11T09:48:30.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.656+0000 7f4e34ff9700 1 -- 172.21.15.78:0/167289236 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+19684 (secure 0 0 0) 0x7f4e28055ec0 con 0x7f4e20003be0 2023-10-11T09:48:30.665 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:30.665 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":85,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:48:29.818615+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:48:18.491556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-11T09:48:19.018937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data_extra","create_time":"2023-10-11T09:48:28.963956+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:48:30.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.656+0000 7f4e3d590700 1 -- 172.21.15.78:0/167289236 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4e24065ed0 msgr2=0x7f4e24068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:30.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.656+0000 7f4e3d590700 1 --2- 172.21.15.78:0/167289236 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4e24065ed0 0x7f4e24068380 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4e3813f910 tx=0x7f4e2c009040 comp rx=0 tx=0).stop 2023-10-11T09:48:30.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.656+0000 7f4e3d590700 1 -- 172.21.15.78:0/167289236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e20003be0 msgr2=0x7f4e380ccc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:30.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.656+0000 7f4e3d590700 1 --2- 172.21.15.78:0/167289236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e20003be0 0x7f4e380ccc50 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f4e280090b0 tx=0x7f4e280047e0 comp rx=0 tx=0).stop 2023-10-11T09:48:30.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.660+0000 7f4e3d590700 1 -- 172.21.15.78:0/167289236 shutdown_connections 2023-10-11T09:48:30.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.660+0000 7f4e3d590700 1 --2- 172.21.15.78:0/167289236 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4e24065ed0 0x7f4e24068380 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:30.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.660+0000 7f4e3d590700 1 --2- 172.21.15.78:0/167289236 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e38159820 0x7f4e38159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:30.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.660+0000 7f4e3d590700 1 --2- 172.21.15.78:0/167289236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e380c3b40 0x7f4e380cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:30.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.660+0000 7f4e3d590700 1 --2- 172.21.15.78:0/167289236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e20003be0 0x7f4e380ccc50 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:30.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.660+0000 7f4e3d590700 1 -- 172.21.15.78:0/167289236 >> 172.21.15.78:0/167289236 conn(0x7f4e380cea60 msgr2=0x7f4e3815d640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:30.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.660+0000 7f4e3d590700 1 -- 172.21.15.78:0/167289236 shutdown_connections 2023-10-11T09:48:30.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:30.660+0000 7f4e3d590700 1 -- 172.21.15.78:0/167289236 wait complete. 2023-10-11T09:48:30.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:48:30.690 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v cephfs_data_extra .) 2023-10-11T09:48:30.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:48:30.723 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2023-10-11T09:48:30.797 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:48:30.798 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:48:30.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:48:30.852 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=subdir/sixmegs bs=1M conv=fdatasync count=6 seek=0) 2023-10-11T09:48:31.049 INFO:teuthology.orchestra.run.smithi078.stderr:6+0 records in 2023-10-11T09:48:31.049 INFO:teuthology.orchestra.run.smithi078.stderr:6+0 records out 2023-10-11T09:48:31.049 INFO:teuthology.orchestra.run.smithi078.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.129367 s, 48.6 MB/s 2023-10-11T09:48:31.050 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:48:31.051 DEBUG:teuthology.orchestra.run.smithi078:> import os 2023-10-11T09:48:31.051 DEBUG:teuthology.orchestra.run.smithi078:> import stat 2023-10-11T09:48:31.051 DEBUG:teuthology.orchestra.run.smithi078:> import json 2023-10-11T09:48:31.052 DEBUG:teuthology.orchestra.run.smithi078:> import sys 2023-10-11T09:48:31.052 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:48:31.052 DEBUG:teuthology.orchestra.run.smithi078:> try: 2023-10-11T09:48:31.053 DEBUG:teuthology.orchestra.run.smithi078:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2023-10-11T09:48:31.053 DEBUG:teuthology.orchestra.run.smithi078:> except OSError as e: 2023-10-11T09:48:31.054 DEBUG:teuthology.orchestra.run.smithi078:> sys.exit(e.errno) 2023-10-11T09:48:31.054 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:48:31.054 DEBUG:teuthology.orchestra.run.smithi078:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2023-10-11T09:48:31.055 DEBUG:teuthology.orchestra.run.smithi078:> print(json.dumps( 2023-10-11T09:48:31.055 DEBUG:teuthology.orchestra.run.smithi078:> dict([(a, getattr(s, a)) for a in attrs]), 2023-10-11T09:48:31.055 DEBUG:teuthology.orchestra.run.smithi078:> indent=2)) 2023-10-11T09:48:31.056 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:48:31.157 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:48:31.158 INFO:teuthology.orchestra.run.smithi078.stdout: "st_mode": 33204, 2023-10-11T09:48:31.158 INFO:teuthology.orchestra.run.smithi078.stdout: "st_ino": 1099511627777, 2023-10-11T09:48:31.158 INFO:teuthology.orchestra.run.smithi078.stdout: "st_dev": 49, 2023-10-11T09:48:31.159 INFO:teuthology.orchestra.run.smithi078.stdout: "st_nlink": 1, 2023-10-11T09:48:31.159 INFO:teuthology.orchestra.run.smithi078.stdout: "st_uid": 1000, 2023-10-11T09:48:31.159 INFO:teuthology.orchestra.run.smithi078.stdout: "st_gid": 1267, 2023-10-11T09:48:31.159 INFO:teuthology.orchestra.run.smithi078.stdout: "st_size": 6291456, 2023-10-11T09:48:31.159 INFO:teuthology.orchestra.run.smithi078.stdout: "st_atime": 1697017710.9142084, 2023-10-11T09:48:31.159 INFO:teuthology.orchestra.run.smithi078.stdout: "st_mtime": 1697017711.0025866, 2023-10-11T09:48:31.160 INFO:teuthology.orchestra.run.smithi078.stdout: "st_ctime": 1697017711.0025866 2023-10-11T09:48:31.160 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:48:31.339 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:48:31.340 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:48:31.404 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:48:31.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.612+0000 7fb0055d5700 1 Processor -- start 2023-10-11T09:48:31.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.612+0000 7fb0055d5700 1 -- start start 2023-10-11T09:48:31.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.616+0000 7fb0055d5700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fb00015d050 0x7fb0000cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:31.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.616+0000 7fb0055d5700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb000160c60 0x7fb00015d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:31.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.616+0000 7fb0055d5700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb000160210 con 0x7fb00015d960 2023-10-11T09:48:31.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.616+0000 7fb0055d5700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0000ccd60 con 0x7fb00015d050 2023-10-11T09:48:31.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.616+0000 7fb0055d5700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0000ccfc0 con 0x7fb000160c60 2023-10-11T09:48:31.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.616+0000 7faffffff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fb00015d050 0x7fb0000cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:31.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.616+0000 7faffffff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fb00015d050 0x7fb0000cc820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57458/0 (socket says 172.21.15.78:57458) 2023-10-11T09:48:31.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.616+0000 7faffffff700 1 -- 172.21.15.78:0/1360836200 learned_addr learned my addr 172.21.15.78:0/1360836200 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:31.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.616+0000 7fb004dd4700 1 --1- 172.21.15.78:0/1360836200 >> v1:172.21.15.78:6789/0 conn(0x7fb00015d960 0x7fb0000b06a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:42936/0 (socket says 172.21.15.78:42936) 2023-10-11T09:48:31.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.616+0000 7faff77fe700 1 --2- 172.21.15.78:0/1360836200 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb000160c60 0x7fb00015d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:31.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.616+0000 7faffffff700 1 -- 172.21.15.78:0/1360836200 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb000160c60 msgr2=0x7fb00015d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:31.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.616+0000 7faffffff700 1 --2- 172.21.15.78:0/1360836200 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb000160c60 0x7fb00015d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:31.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.616+0000 7faffffff700 1 -- 172.21.15.78:0/1360836200 >> v1:172.21.15.78:6789/0 conn(0x7fb00015d960 legacy=0x7fb0000b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:31.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.616+0000 7faffffff700 1 -- 172.21.15.78:0/1360836200 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0000cd2a0 con 0x7fb00015d050 2023-10-11T09:48:31.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.616+0000 7faffffff700 1 --2- 172.21.15.78:0/1360836200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb00015d050 0x7fb0000cc820 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7faff0009910 tx=0x7faff0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10345cdcfaa84128 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:31.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.616+0000 7fafff7fe700 1 -- 172.21.15.78:0/1360836200 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff000ba60 con 0x7fb00015d050 2023-10-11T09:48:31.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.616+0000 7fafff7fe700 1 -- 172.21.15.78:0/1360836200 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff000bbc0 con 0x7fb00015d050 2023-10-11T09:48:31.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.616+0000 7fafff7fe700 1 -- 172.21.15.78:0/1360836200 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff0009e60 con 0x7fb00015d050 2023-10-11T09:48:31.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.616+0000 7fb0055d5700 1 -- 172.21.15.78:0/1360836200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb00015d050 msgr2=0x7fb0000cc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:31.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.616+0000 7fb0055d5700 1 --2- 172.21.15.78:0/1360836200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb00015d050 0x7fb0000cc820 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7faff0009910 tx=0x7faff0009c20 comp rx=0 tx=0).stop 2023-10-11T09:48:31.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.616+0000 7fb0055d5700 1 -- 172.21.15.78:0/1360836200 shutdown_connections 2023-10-11T09:48:31.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.616+0000 7fb0055d5700 1 --2- 172.21.15.78:0/1360836200 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb000160c60 0x7fb00015d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:31.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.616+0000 7fb0055d5700 1 --2- 172.21.15.78:0/1360836200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb00015d050 0x7fb0000cc820 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:31.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.616+0000 7fb0055d5700 1 -- 172.21.15.78:0/1360836200 >> 172.21.15.78:0/1360836200 conn(0x7fb0000cea60 msgr2=0x7fb0000cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:31.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.616+0000 7fb0055d5700 1 -- 172.21.15.78:0/1360836200 shutdown_connections 2023-10-11T09:48:31.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.616+0000 7fb0055d5700 1 -- 172.21.15.78:0/1360836200 wait complete. 2023-10-11T09:48:31.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.620+0000 7fb0055d5700 1 Processor -- start 2023-10-11T09:48:31.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.620+0000 7fb0055d5700 1 -- start start 2023-10-11T09:48:31.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.624+0000 7fb0055d5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb00015d050 0x7fb000071440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:31.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.624+0000 7fb0055d5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb00015d960 0x7fb0001f18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:31.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.624+0000 7fb0055d5700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb000160c60 0x7fb000142be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:31.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.624+0000 7fb0055d5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0000b49a0 con 0x7fb00015d050 2023-10-11T09:48:31.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.624+0000 7fb0055d5700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb000070680 con 0x7fb00015d960 2023-10-11T09:48:31.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.624+0000 7fb0055d5700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb000070960 con 0x7fb000160c60 2023-10-11T09:48:31.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.624+0000 7faffffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb00015d050 0x7fb000071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:31.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.624+0000 7faffffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb00015d050 0x7fb000071440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57464/0 (socket says 172.21.15.78:57464) 2023-10-11T09:48:31.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.624+0000 7faffffff700 1 -- 172.21.15.78:0/1669003172 learned_addr learned my addr 172.21.15.78:0/1669003172 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:31.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.624+0000 7faff77fe700 1 --2- 172.21.15.78:0/1669003172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb00015d960 0x7fb0001f18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:31.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.624+0000 7faffffff700 1 -- 172.21.15.78:0/1669003172 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb000160c60 msgr2=0x7fb000142be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:31.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.624+0000 7faffffff700 1 --2- 172.21.15.78:0/1669003172 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb000160c60 0x7fb000142be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:31.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.624+0000 7faffffff700 1 -- 172.21.15.78:0/1669003172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb00015d960 msgr2=0x7fb0001f18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:31.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.624+0000 7faffffff700 1 --2- 172.21.15.78:0/1669003172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb00015d960 0x7fb0001f18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:31.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.624+0000 7faffffff700 1 -- 172.21.15.78:0/1669003172 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff0009570 con 0x7fb00015d050 2023-10-11T09:48:31.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.624+0000 7faffffff700 1 --2- 172.21.15.78:0/1669003172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb00015d050 0x7fb000071440 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7faff0014040 tx=0x7faff0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:31.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.624+0000 7faffd7fa700 1 -- 172.21.15.78:0/1669003172 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff0004610 con 0x7fb00015d050 2023-10-11T09:48:31.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.624+0000 7faffd7fa700 1 -- 172.21.15.78:0/1669003172 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff001e070 con 0x7fb00015d050 2023-10-11T09:48:31.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.624+0000 7fb0055d5700 1 -- 172.21.15.78:0/1669003172 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb000070c40 con 0x7fb00015d050 2023-10-11T09:48:31.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.624+0000 7faffd7fa700 1 -- 172.21.15.78:0/1669003172 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff0003920 con 0x7fb00015d050 2023-10-11T09:48:31.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.624+0000 7fb0055d5700 1 -- 172.21.15.78:0/1669003172 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0001f1e20 con 0x7fb00015d050 2023-10-11T09:48:31.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.628+0000 7fb0055d5700 1 -- 172.21.15.78:0/1669003172 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb000071160 con 0x7fb00015d050 2023-10-11T09:48:31.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.628+0000 7faffd7fa700 1 -- 172.21.15.78:0/1669003172 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7faff0003a80 con 0x7fb00015d050 2023-10-11T09:48:31.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.628+0000 7faffd7fa700 1 --2- 172.21.15.78:0/1669003172 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fafe4065ed0 0x7fafe4068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:31.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.628+0000 7faffd7fa700 1 -- 172.21.15.78:0/1669003172 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6466+0+0 (secure 0 0 0) 0x7faff0020070 con 0x7fb00015d050 2023-10-11T09:48:31.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.628+0000 7faff77fe700 1 --2- 172.21.15.78:0/1669003172 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fafe4065ed0 0x7fafe4068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:31.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.632+0000 7faff77fe700 1 --2- 172.21.15.78:0/1669003172 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fafe4065ed0 0x7fafe4068380 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb0000c9ad0 tx=0x7fafe8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:31.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.636+0000 7faffd7fa700 1 -- 172.21.15.78:0/1669003172 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7faff0052600 con 0x7fb00015d050 2023-10-11T09:48:31.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.816+0000 7fb0055d5700 1 -- 172.21.15.78:0/1669003172 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb000071160 con 0x7fb00015d050 2023-10-11T09:48:31.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.820+0000 7faffd7fa700 1 -- 172.21.15.78:0/1669003172 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+19671 (secure 0 0 0) 0x7fb000071160 con 0x7fb00015d050 2023-10-11T09:48:31.830 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:31.830 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":86,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:48:30.824033+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:48:18.491556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-11T09:48:19.018937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data_extra","create_time":"2023-10-11T09:48:28.963956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:48:31.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.824+0000 7fb0055d5700 1 -- 172.21.15.78:0/1669003172 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fafe4065ed0 msgr2=0x7fafe4068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:31.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.824+0000 7fb0055d5700 1 --2- 172.21.15.78:0/1669003172 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fafe4065ed0 0x7fafe4068380 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb0000c9ad0 tx=0x7fafe8008040 comp rx=0 tx=0).stop 2023-10-11T09:48:31.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.824+0000 7fb0055d5700 1 -- 172.21.15.78:0/1669003172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb00015d050 msgr2=0x7fb000071440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:31.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.824+0000 7fb0055d5700 1 --2- 172.21.15.78:0/1669003172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb00015d050 0x7fb000071440 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7faff0014040 tx=0x7faff0004770 comp rx=0 tx=0).stop 2023-10-11T09:48:31.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.824+0000 7fb0055d5700 1 -- 172.21.15.78:0/1669003172 shutdown_connections 2023-10-11T09:48:31.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.824+0000 7fb0055d5700 1 --2- 172.21.15.78:0/1669003172 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fafe4065ed0 0x7fafe4068380 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:31.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.824+0000 7fb0055d5700 1 --2- 172.21.15.78:0/1669003172 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb000160c60 0x7fb000142be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:31.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.824+0000 7fb0055d5700 1 --2- 172.21.15.78:0/1669003172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb00015d960 0x7fb0001f18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:31.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.824+0000 7fb0055d5700 1 --2- 172.21.15.78:0/1669003172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb00015d050 0x7fb000071440 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:31.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.824+0000 7fb0055d5700 1 -- 172.21.15.78:0/1669003172 >> 172.21.15.78:0/1669003172 conn(0x7fb0000cea60 msgr2=0x7fb00015de30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:31.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.824+0000 7fb0055d5700 1 -- 172.21.15.78:0/1669003172 shutdown_connections 2023-10-11T09:48:31.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:31.824+0000 7fb0055d5700 1 -- 172.21.15.78:0/1669003172 wait complete. 2023-10-11T09:48:31.855 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T09:48:18.491556+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T09:48:19.018937+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-10-11T09:48:28.963956+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:48:31.856 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:48:32.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.028+0000 7f2649251700 1 Processor -- start 2023-10-11T09:48:32.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.028+0000 7f2649251700 1 -- start start 2023-10-11T09:48:32.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.032+0000 7f2649251700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f26440b1030 0x7f26440cb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:32.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.032+0000 7f2649251700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f264406d080 0x7f264406ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:32.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.032+0000 7f2649251700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2644165570 con 0x7f2644160cb0 2023-10-11T09:48:32.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.032+0000 7f2649251700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2644161080 con 0x7f26440b1030 2023-10-11T09:48:32.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.032+0000 7f2649251700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2644164750 con 0x7f264406d080 2023-10-11T09:48:32.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.032+0000 7f264259c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f26440b1030 0x7f26440cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:32.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.032+0000 7f264259c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f26440b1030 0x7f26440cb100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57466/0 (socket says 172.21.15.78:57466) 2023-10-11T09:48:32.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.032+0000 7f264259c700 1 -- 172.21.15.78:0/3703389498 learned_addr learned my addr 172.21.15.78:0/3703389498 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:32.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.032+0000 7f264359e700 1 --2- 172.21.15.78:0/3703389498 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f264406d080 0x7f264406ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:32.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.032+0000 7f264259c700 1 -- 172.21.15.78:0/3703389498 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f264406d080 msgr2=0x7f264406ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:32.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.032+0000 7f264259c700 1 --2- 172.21.15.78:0/3703389498 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f264406d080 0x7f264406ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:32.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.032+0000 7f264259c700 1 -- 172.21.15.78:0/3703389498 >> v1:172.21.15.133:6789/0 conn(0x7f2644160cb0 legacy=0x7f26440b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:32.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.032+0000 7f264259c700 1 -- 172.21.15.78:0/3703389498 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2644164960 con 0x7f26440b1030 2023-10-11T09:48:32.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.032+0000 7f264259c700 1 --2- 172.21.15.78:0/3703389498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26440b1030 0x7f26440cb100 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f2638009910 tx=0x7f2638009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6f07cc5d1cd19 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:32.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.032+0000 7f2641d9b700 1 -- 172.21.15.78:0/3703389498 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2638004230 con 0x7f26440b1030 2023-10-11T09:48:32.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.032+0000 7f2641d9b700 1 -- 172.21.15.78:0/3703389498 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2638004390 con 0x7f26440b1030 2023-10-11T09:48:32.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.032+0000 7f2641d9b700 1 -- 172.21.15.78:0/3703389498 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2638009e60 con 0x7f26440b1030 2023-10-11T09:48:32.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.032+0000 7f2649251700 1 -- 172.21.15.78:0/3703389498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26440b1030 msgr2=0x7f26440cb100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:32.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.032+0000 7f2649251700 1 --2- 172.21.15.78:0/3703389498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26440b1030 0x7f26440cb100 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f2638009910 tx=0x7f2638009c20 comp rx=0 tx=0).stop 2023-10-11T09:48:32.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.032+0000 7f2649251700 1 -- 172.21.15.78:0/3703389498 shutdown_connections 2023-10-11T09:48:32.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.032+0000 7f2649251700 1 --2- 172.21.15.78:0/3703389498 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f264406d080 0x7f264406ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:32.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.032+0000 7f2649251700 1 --2- 172.21.15.78:0/3703389498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26440b1030 0x7f26440cb100 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:32.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.032+0000 7f2649251700 1 -- 172.21.15.78:0/3703389498 >> 172.21.15.78:0/3703389498 conn(0x7f26440cecc0 msgr2=0x7f26440cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:32.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.032+0000 7f2649251700 1 -- 172.21.15.78:0/3703389498 shutdown_connections 2023-10-11T09:48:32.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.032+0000 7f2649251700 1 -- 172.21.15.78:0/3703389498 wait complete. 2023-10-11T09:48:32.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.036+0000 7f2649251700 1 Processor -- start 2023-10-11T09:48:32.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.036+0000 7f2649251700 1 -- start start 2023-10-11T09:48:32.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.040+0000 7f2649251700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f264406d080 0x7f2644154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:32.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.040+0000 7f2649251700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26440b1030 0x7f26440b4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:32.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.040+0000 7f2649251700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2644160cb0 0x7f2644151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:32.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.040+0000 7f2649251700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26440b46a0 con 0x7f2644160cb0 2023-10-11T09:48:32.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.040+0000 7f2649251700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26440b30c0 con 0x7f26440b1030 2023-10-11T09:48:32.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.040+0000 7f2649251700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26440b33a0 con 0x7f264406d080 2023-10-11T09:48:32.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.040+0000 7f264359e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2644160cb0 0x7f2644151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:32.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.040+0000 7f264359e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2644160cb0 0x7f2644151fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57476/0 (socket says 172.21.15.78:57476) 2023-10-11T09:48:32.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.040+0000 7f264359e700 1 -- 172.21.15.78:0/3255397380 learned_addr learned my addr 172.21.15.78:0/3255397380 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:32.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.040+0000 7f264259c700 1 --2- 172.21.15.78:0/3255397380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26440b1030 0x7f26440b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:32.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.040+0000 7f2642d9d700 1 --2- 172.21.15.78:0/3255397380 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f264406d080 0x7f2644154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:32.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.040+0000 7f264359e700 1 -- 172.21.15.78:0/3255397380 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f264406d080 msgr2=0x7f2644154fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:32.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.040+0000 7f264359e700 1 --2- 172.21.15.78:0/3255397380 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f264406d080 0x7f2644154fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:32.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.040+0000 7f264359e700 1 -- 172.21.15.78:0/3255397380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26440b1030 msgr2=0x7f26440b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:32.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.040+0000 7f264359e700 1 --2- 172.21.15.78:0/3255397380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26440b1030 0x7f26440b4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:32.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.040+0000 7f264359e700 1 -- 172.21.15.78:0/3255397380 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2638009570 con 0x7f2644160cb0 2023-10-11T09:48:32.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.040+0000 7f264359e700 1 --2- 172.21.15.78:0/3255397380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2644160cb0 0x7f2644151fa0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f263c007e60 tx=0x7f263c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:32.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.040+0000 7f262b7fe700 1 -- 172.21.15.78:0/3255397380 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263c00ede0 con 0x7f2644160cb0 2023-10-11T09:48:32.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.040+0000 7f262b7fe700 1 -- 172.21.15.78:0/3255397380 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f263c014420 con 0x7f2644160cb0 2023-10-11T09:48:32.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.040+0000 7f2649251700 1 -- 172.21.15.78:0/3255397380 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26440b3680 con 0x7f2644160cb0 2023-10-11T09:48:32.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.040+0000 7f262b7fe700 1 -- 172.21.15.78:0/3255397380 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263c0147d0 con 0x7f2644160cb0 2023-10-11T09:48:32.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.040+0000 7f2649251700 1 -- 172.21.15.78:0/3255397380 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26440b3c40 con 0x7f2644160cb0 2023-10-11T09:48:32.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.040+0000 7f2649251700 1 -- 172.21.15.78:0/3255397380 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f264406eb80 con 0x7f2644160cb0 2023-10-11T09:48:32.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.044+0000 7f262b7fe700 1 -- 172.21.15.78:0/3255397380 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f263c014930 con 0x7f2644160cb0 2023-10-11T09:48:32.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.044+0000 7f262b7fe700 1 --2- 172.21.15.78:0/3255397380 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2630065f00 0x7f26300683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:32.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.044+0000 7f262b7fe700 1 -- 172.21.15.78:0/3255397380 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f263c084100 con 0x7f2644160cb0 2023-10-11T09:48:32.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.044+0000 7f2642d9d700 1 --2- 172.21.15.78:0/3255397380 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2630065f00 0x7f26300683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:32.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.048+0000 7f2642d9d700 1 --2- 172.21.15.78:0/3255397380 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2630065f00 0x7f26300683b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2634009890 tx=0x7f2634008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:32.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.052+0000 7f262b7fe700 1 -- 172.21.15.78:0/3255397380 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f263c051510 con 0x7f2644160cb0 2023-10-11T09:48:32.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.224+0000 7f2649251700 1 -- 172.21.15.78:0/3255397380 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2644065db0 con 0x7f2644160cb0 2023-10-11T09:48:32.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.224+0000 7f262b7fe700 1 -- 172.21.15.78:0/3255397380 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f263c054b30 con 0x7f2644160cb0 2023-10-11T09:48:32.233 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:48:32.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.228+0000 7f2649251700 1 -- 172.21.15.78:0/3255397380 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2630065f00 msgr2=0x7f26300683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:32.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.228+0000 7f2649251700 1 --2- 172.21.15.78:0/3255397380 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2630065f00 0x7f26300683b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2634009890 tx=0x7f2634008040 comp rx=0 tx=0).stop 2023-10-11T09:48:32.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.228+0000 7f2649251700 1 -- 172.21.15.78:0/3255397380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2644160cb0 msgr2=0x7f2644151fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:32.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.228+0000 7f2649251700 1 --2- 172.21.15.78:0/3255397380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2644160cb0 0x7f2644151fa0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f263c007e60 tx=0x7f263c00e3c0 comp rx=0 tx=0).stop 2023-10-11T09:48:32.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.228+0000 7f2649251700 1 -- 172.21.15.78:0/3255397380 shutdown_connections 2023-10-11T09:48:32.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.228+0000 7f2649251700 1 --2- 172.21.15.78:0/3255397380 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2630065f00 0x7f26300683b0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:32.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.228+0000 7f2649251700 1 --2- 172.21.15.78:0/3255397380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2644160cb0 0x7f2644151fa0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:32.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.228+0000 7f2649251700 1 --2- 172.21.15.78:0/3255397380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26440b1030 0x7f26440b4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:32.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.228+0000 7f2649251700 1 --2- 172.21.15.78:0/3255397380 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f264406d080 0x7f2644154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:32.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.228+0000 7f2649251700 1 -- 172.21.15.78:0/3255397380 >> 172.21.15.78:0/3255397380 conn(0x7f26440cecc0 msgr2=0x7f26440cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:32.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.228+0000 7f2649251700 1 -- 172.21.15.78:0/3255397380 shutdown_connections 2023-10-11T09:48:32.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.228+0000 7f2649251700 1 -- 172.21.15.78:0/3255397380 wait complete. 2023-10-11T09:48:32.256 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:48:32.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.428+0000 7f10a2b78700 1 Processor -- start 2023-10-11T09:48:32.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.428+0000 7f10a2b78700 1 -- start start 2023-10-11T09:48:32.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.428+0000 7f10a2b78700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f109c0c9b90 0x7f109c15db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:32.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.428+0000 7f10a2b78700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f109c06cf70 0x7f109c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:32.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.428+0000 7f10a2b78700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f109c06d900 con 0x7f109c1596b0 2023-10-11T09:48:32.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.428+0000 7f10a2b78700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f109c0c9280 con 0x7f109c06cf70 2023-10-11T09:48:32.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.428+0000 7f10a2b78700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f109c0cd430 con 0x7f109c0c9b90 2023-10-11T09:48:32.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.432+0000 7f10a1115700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f109c06cf70 0x7f109c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:32.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.432+0000 7f10a1115700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f109c06cf70 0x7f109c06d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:39520/0 (socket says 172.21.15.78:39520) 2023-10-11T09:48:32.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.432+0000 7f10a1115700 1 -- 172.21.15.78:0/262200135 learned_addr learned my addr 172.21.15.78:0/262200135 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:32.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.432+0000 7f10a0914700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f109c1596b0 0x7f109c0ccd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:51006/0 (socket says 172.21.15.78:51006) 2023-10-11T09:48:32.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.432+0000 7f109bfff700 1 --2- 172.21.15.78:0/262200135 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f109c0c9b90 0x7f109c15db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:32.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.432+0000 7f10a1115700 1 -- 172.21.15.78:0/262200135 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f109c0c9b90 msgr2=0x7f109c15db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:32.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.432+0000 7f10a1115700 1 --2- 172.21.15.78:0/262200135 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f109c0c9b90 0x7f109c15db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:32.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.432+0000 7f10a1115700 1 -- 172.21.15.78:0/262200135 >> v1:172.21.15.133:6789/0 conn(0x7f109c1596b0 legacy=0x7f109c0ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:32.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.432+0000 7f10a1115700 1 -- 172.21.15.78:0/262200135 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f109c0c9fe0 con 0x7f109c06cf70 2023-10-11T09:48:32.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.432+0000 7f10a1115700 1 --2- 172.21.15.78:0/262200135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f109c06cf70 0x7f109c06d3c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1094009e50 tx=0x7f109400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21fbe0e836e66138 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:32.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.432+0000 7f109b7fe700 1 -- 172.21.15.78:0/262200135 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109400e070 con 0x7f109c06cf70 2023-10-11T09:48:32.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.432+0000 7f109b7fe700 1 -- 172.21.15.78:0/262200135 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109400b910 con 0x7f109c06cf70 2023-10-11T09:48:32.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.432+0000 7f109b7fe700 1 -- 172.21.15.78:0/262200135 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109400bcc0 con 0x7f109c06cf70 2023-10-11T09:48:32.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.432+0000 7f10a2b78700 1 -- 172.21.15.78:0/262200135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f109c06cf70 msgr2=0x7f109c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:32.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.432+0000 7f10a2b78700 1 --2- 172.21.15.78:0/262200135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f109c06cf70 0x7f109c06d3c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1094009e50 tx=0x7f109400b330 comp rx=0 tx=0).stop 2023-10-11T09:48:32.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.432+0000 7f10a2b78700 1 -- 172.21.15.78:0/262200135 shutdown_connections 2023-10-11T09:48:32.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.432+0000 7f10a2b78700 1 --2- 172.21.15.78:0/262200135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f109c06cf70 0x7f109c06d3c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:32.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.432+0000 7f10a2b78700 1 --2- 172.21.15.78:0/262200135 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f109c0c9b90 0x7f109c15db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:32.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.432+0000 7f10a2b78700 1 -- 172.21.15.78:0/262200135 >> 172.21.15.78:0/262200135 conn(0x7f109c0cecc0 msgr2=0x7f109c0cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:32.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.432+0000 7f10a2b78700 1 -- 172.21.15.78:0/262200135 shutdown_connections 2023-10-11T09:48:32.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.432+0000 7f10a2b78700 1 -- 172.21.15.78:0/262200135 wait complete. 2023-10-11T09:48:32.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.436+0000 7f10a2b78700 1 Processor -- start 2023-10-11T09:48:32.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.436+0000 7f10a2b78700 1 -- start start 2023-10-11T09:48:32.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.440+0000 7f10a2b78700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f109c06cf70 0x7f109c1f22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:32.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.440+0000 7f10a2b78700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f109c0c9b90 0x7f109c15afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:32.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.440+0000 7f10a2b78700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f109c1596b0 0x7f109c157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:32.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.440+0000 7f10a2b78700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c1470a0 con 0x7f109c0c9b90 2023-10-11T09:48:32.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.440+0000 7f10a2b78700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c15c970 con 0x7f109c1596b0 2023-10-11T09:48:32.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.440+0000 7f10a2b78700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f109c15b8d0 con 0x7f109c06cf70 2023-10-11T09:48:32.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.440+0000 7f109bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f109c0c9b90 0x7f109c15afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:32.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.440+0000 7f109bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f109c0c9b90 0x7f109c15afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57492/0 (socket says 172.21.15.78:57492) 2023-10-11T09:48:32.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.440+0000 7f109bfff700 1 -- 172.21.15.78:0/1973306566 learned_addr learned my addr 172.21.15.78:0/1973306566 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:32.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.440+0000 7f10a0914700 1 --2- 172.21.15.78:0/1973306566 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f109c06cf70 0x7f109c1f22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:32.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.440+0000 7f10a1115700 1 --2- 172.21.15.78:0/1973306566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f109c1596b0 0x7f109c157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:32.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.440+0000 7f109bfff700 1 -- 172.21.15.78:0/1973306566 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f109c06cf70 msgr2=0x7f109c1f22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:32.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.440+0000 7f109bfff700 1 --2- 172.21.15.78:0/1973306566 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f109c06cf70 0x7f109c1f22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:32.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.440+0000 7f109bfff700 1 -- 172.21.15.78:0/1973306566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f109c1596b0 msgr2=0x7f109c157cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:32.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.440+0000 7f109bfff700 1 --2- 172.21.15.78:0/1973306566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f109c1596b0 0x7f109c157cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:32.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.440+0000 7f109bfff700 1 -- 172.21.15.78:0/1973306566 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1094009b30 con 0x7f109c0c9b90 2023-10-11T09:48:32.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.440+0000 7f109bfff700 1 --2- 172.21.15.78:0/1973306566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f109c0c9b90 0x7f109c15afa0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f108800e9d0 tx=0x7f108800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:32.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.440+0000 7f10997fa700 1 -- 172.21.15.78:0/1973306566 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1088009720 con 0x7f109c0c9b90 2023-10-11T09:48:32.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.440+0000 7f10997fa700 1 -- 172.21.15.78:0/1973306566 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108800ee70 con 0x7f109c0c9b90 2023-10-11T09:48:32.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.440+0000 7f10a2b78700 1 -- 172.21.15.78:0/1973306566 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f109c15bbb0 con 0x7f109c0c9b90 2023-10-11T09:48:32.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.440+0000 7f10997fa700 1 -- 172.21.15.78:0/1973306566 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108800ca50 con 0x7f109c0c9b90 2023-10-11T09:48:32.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.444+0000 7f10a2b78700 1 -- 172.21.15.78:0/1973306566 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f109c15c170 con 0x7f109c0c9b90 2023-10-11T09:48:32.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.444+0000 7f10a2b78700 1 -- 172.21.15.78:0/1973306566 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f109c0cb250 con 0x7f109c0c9b90 2023-10-11T09:48:32.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.444+0000 7f10997fa700 1 -- 172.21.15.78:0/1973306566 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f108800cbb0 con 0x7f109c0c9b90 2023-10-11T09:48:32.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.448+0000 7f10997fa700 1 --2- 172.21.15.78:0/1973306566 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1080065f00 0x7f10800683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:32.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.448+0000 7f10997fa700 1 -- 172.21.15.78:0/1973306566 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f1088083990 con 0x7f109c0c9b90 2023-10-11T09:48:32.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.448+0000 7f10a0914700 1 --2- 172.21.15.78:0/1973306566 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1080065f00 0x7f10800683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:32.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.448+0000 7f10a0914700 1 --2- 172.21.15.78:0/1973306566 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1080065f00 0x7f10800683b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1090000f60 tx=0x7f1090007ef0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:32.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.452+0000 7f10997fa700 1 -- 172.21.15.78:0/1973306566 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f1088050da0 con 0x7f109c0c9b90 2023-10-11T09:48:32.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.624+0000 7f10a2b78700 1 -- 172.21.15.78:0/1973306566 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f109c0719a0 con 0x7f109c0c9b90 2023-10-11T09:48:32.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.628+0000 7f10997fa700 1 -- 172.21.15.78:0/1973306566 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v86) v1 ==== 92+0+10 (secure 0 0 0) 0x7f108800ce60 con 0x7f109c0c9b90 2023-10-11T09:48:32.636 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:48:32.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.628+0000 7f10a2b78700 1 -- 172.21.15.78:0/1973306566 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1080065f00 msgr2=0x7f10800683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:32.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.628+0000 7f10a2b78700 1 --2- 172.21.15.78:0/1973306566 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1080065f00 0x7f10800683b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1090000f60 tx=0x7f1090007ef0 comp rx=0 tx=0).stop 2023-10-11T09:48:32.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.628+0000 7f10a2b78700 1 -- 172.21.15.78:0/1973306566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f109c0c9b90 msgr2=0x7f109c15afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:32.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.628+0000 7f10a2b78700 1 --2- 172.21.15.78:0/1973306566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f109c0c9b90 0x7f109c15afa0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f108800e9d0 tx=0x7f108800ece0 comp rx=0 tx=0).stop 2023-10-11T09:48:32.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.628+0000 7f10a2b78700 1 -- 172.21.15.78:0/1973306566 shutdown_connections 2023-10-11T09:48:32.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.628+0000 7f10a2b78700 1 --2- 172.21.15.78:0/1973306566 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1080065f00 0x7f10800683b0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:32.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.628+0000 7f10a2b78700 1 --2- 172.21.15.78:0/1973306566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f109c1596b0 0x7f109c157cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:32.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.628+0000 7f10a2b78700 1 --2- 172.21.15.78:0/1973306566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f109c0c9b90 0x7f109c15afa0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:32.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.628+0000 7f10a2b78700 1 --2- 172.21.15.78:0/1973306566 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f109c06cf70 0x7f109c1f22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:32.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.628+0000 7f10a2b78700 1 -- 172.21.15.78:0/1973306566 >> 172.21.15.78:0/1973306566 conn(0x7f109c0cecc0 msgr2=0x7f109c1488c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:32.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.628+0000 7f10a2b78700 1 -- 172.21.15.78:0/1973306566 shutdown_connections 2023-10-11T09:48:32.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.628+0000 7f10a2b78700 1 -- 172.21.15.78:0/1973306566 wait complete. 2023-10-11T09:48:32.659 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T09:48:32.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.832+0000 7fd83b7a7700 1 Processor -- start 2023-10-11T09:48:32.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.832+0000 7fd83b7a7700 1 -- start start 2023-10-11T09:48:32.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.836+0000 7fd83b7a7700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd834160cb0 0x7fd8340cb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:32.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.836+0000 7fd83b7a7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd83406d080 0x7fd8340b0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:32.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.836+0000 7fd83b7a7700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd834165570 con 0x7fd8340b0e60 2023-10-11T09:48:32.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.836+0000 7fd83b7a7700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd834161080 con 0x7fd834160cb0 2023-10-11T09:48:32.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.836+0000 7fd83b7a7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd834164750 con 0x7fd83406d080 2023-10-11T09:48:32.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.836+0000 7fd839543700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd834160cb0 0x7fd8340cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:32.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.836+0000 7fd839543700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd834160cb0 0x7fd8340cb100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:39540/0 (socket says 172.21.15.78:39540) 2023-10-11T09:48:32.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.836+0000 7fd839543700 1 -- 172.21.15.78:0/2842639077 learned_addr learned my addr 172.21.15.78:0/2842639077 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:32.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.836+0000 7fd838d42700 1 --2- 172.21.15.78:0/2842639077 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd83406d080 0x7fd8340b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:32.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.836+0000 7fd839543700 1 -- 172.21.15.78:0/2842639077 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd83406d080 msgr2=0x7fd8340b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:32.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.836+0000 7fd839543700 1 --2- 172.21.15.78:0/2842639077 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd83406d080 0x7fd8340b0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:32.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.836+0000 7fd839543700 1 -- 172.21.15.78:0/2842639077 >> v1:172.21.15.133:6789/0 conn(0x7fd8340b0e60 legacy=0x7fd83406be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:32.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.836+0000 7fd839543700 1 -- 172.21.15.78:0/2842639077 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd834164960 con 0x7fd834160cb0 2023-10-11T09:48:32.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.836+0000 7fd838d42700 1 --2- 172.21.15.78:0/2842639077 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd83406d080 0x7fd8340b0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:48:32.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.836+0000 7fd82bfff700 1 -- 172.21.15.78:0/2842639077 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3280872691 0 0) 0x7fd834165570 con 0x7fd8340b0e60 2023-10-11T09:48:32.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.836+0000 7fd839543700 1 --2- 172.21.15.78:0/2842639077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd834160cb0 0x7fd8340cb100 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd830009910 tx=0x7fd830009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=692aeb3cdcc9eac1 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:32.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.836+0000 7fd82bfff700 1 -- 172.21.15.78:0/2842639077 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83000ba60 con 0x7fd834160cb0 2023-10-11T09:48:32.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.840+0000 7fd82bfff700 1 -- 172.21.15.78:0/2842639077 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd83000bbc0 con 0x7fd834160cb0 2023-10-11T09:48:32.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.840+0000 7fd82bfff700 1 -- 172.21.15.78:0/2842639077 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd830009e60 con 0x7fd834160cb0 2023-10-11T09:48:32.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.840+0000 7fd83b7a7700 1 -- 172.21.15.78:0/2842639077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd834160cb0 msgr2=0x7fd8340cb100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:32.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.840+0000 7fd83b7a7700 1 --2- 172.21.15.78:0/2842639077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd834160cb0 0x7fd8340cb100 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd830009910 tx=0x7fd830009c20 comp rx=0 tx=0).stop 2023-10-11T09:48:32.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.840+0000 7fd83b7a7700 1 -- 172.21.15.78:0/2842639077 shutdown_connections 2023-10-11T09:48:32.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.840+0000 7fd83b7a7700 1 --2- 172.21.15.78:0/2842639077 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd83406d080 0x7fd8340b0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:32.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.840+0000 7fd83b7a7700 1 --2- 172.21.15.78:0/2842639077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd834160cb0 0x7fd8340cb100 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:32.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.840+0000 7fd83b7a7700 1 -- 172.21.15.78:0/2842639077 >> 172.21.15.78:0/2842639077 conn(0x7fd8340cecc0 msgr2=0x7fd8340cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:32.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.840+0000 7fd83b7a7700 1 -- 172.21.15.78:0/2842639077 shutdown_connections 2023-10-11T09:48:32.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.840+0000 7fd83b7a7700 1 -- 172.21.15.78:0/2842639077 wait complete. 2023-10-11T09:48:32.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.844+0000 7fd83b7a7700 1 Processor -- start 2023-10-11T09:48:32.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.844+0000 7fd83b7a7700 1 -- start start 2023-10-11T09:48:32.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.844+0000 7fd83b7a7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd83406d080 0x7fd8341517e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:32.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.844+0000 7fd83b7a7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8340b0e60 0x7fd83415b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:32.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.844+0000 7fd83b7a7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd834160cb0 0x7fd83415baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:32.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.844+0000 7fd83b7a7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8340b4870 con 0x7fd83406d080 2023-10-11T09:48:32.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.844+0000 7fd83b7a7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8341615f0 con 0x7fd834160cb0 2023-10-11T09:48:32.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.844+0000 7fd83b7a7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd83415cd30 con 0x7fd8340b0e60 2023-10-11T09:48:32.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.844+0000 7fd839543700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd83406d080 0x7fd8341517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:32.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.844+0000 7fd839543700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd83406d080 0x7fd8341517e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57502/0 (socket says 172.21.15.78:57502) 2023-10-11T09:48:32.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.844+0000 7fd839543700 1 -- 172.21.15.78:0/2849060199 learned_addr learned my addr 172.21.15.78:0/2849060199 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:32.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.844+0000 7fd839d44700 1 --2- 172.21.15.78:0/2849060199 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd834160cb0 0x7fd83415baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:32.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.844+0000 7fd838d42700 1 --2- 172.21.15.78:0/2849060199 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8340b0e60 0x7fd83415b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:32.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.844+0000 7fd839543700 1 -- 172.21.15.78:0/2849060199 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8340b0e60 msgr2=0x7fd83415b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:32.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.848+0000 7fd839543700 1 --2- 172.21.15.78:0/2849060199 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8340b0e60 0x7fd83415b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:32.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.848+0000 7fd839543700 1 -- 172.21.15.78:0/2849060199 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd834160cb0 msgr2=0x7fd83415baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:32.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.848+0000 7fd839543700 1 --2- 172.21.15.78:0/2849060199 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd834160cb0 0x7fd83415baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:32.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.848+0000 7fd839543700 1 -- 172.21.15.78:0/2849060199 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd830009570 con 0x7fd83406d080 2023-10-11T09:48:32.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.848+0000 7fd839d44700 1 --2- 172.21.15.78:0/2849060199 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd834160cb0 0x7fd83415baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:48:32.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.848+0000 7fd838d42700 1 --2- 172.21.15.78:0/2849060199 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8340b0e60 0x7fd83415b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:48:32.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.848+0000 7fd839543700 1 --2- 172.21.15.78:0/2849060199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd83406d080 0x7fd8341517e0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fd830009150 tx=0x7fd830004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:32.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.848+0000 7fd829ffb700 1 -- 172.21.15.78:0/2849060199 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8300036c0 con 0x7fd83406d080 2023-10-11T09:48:32.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.848+0000 7fd829ffb700 1 -- 172.21.15.78:0/2849060199 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd83001e070 con 0x7fd83406d080 2023-10-11T09:48:32.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.848+0000 7fd83b7a7700 1 -- 172.21.15.78:0/2849060199 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd83415d010 con 0x7fd83406d080 2023-10-11T09:48:32.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.848+0000 7fd829ffb700 1 -- 172.21.15.78:0/2849060199 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd830003a60 con 0x7fd83406d080 2023-10-11T09:48:32.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.848+0000 7fd83b7a7700 1 -- 172.21.15.78:0/2849060199 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd83415d5d0 con 0x7fd83406d080 2023-10-11T09:48:32.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.848+0000 7fd83b7a7700 1 -- 172.21.15.78:0/2849060199 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd83406eb80 con 0x7fd83406d080 2023-10-11T09:48:32.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.848+0000 7fd829ffb700 1 -- 172.21.15.78:0/2849060199 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fd830003bc0 con 0x7fd83406d080 2023-10-11T09:48:32.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.852+0000 7fd829ffb700 1 --2- 172.21.15.78:0/2849060199 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd818065ed0 0x7fd818068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:32.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.852+0000 7fd829ffb700 1 -- 172.21.15.78:0/2849060199 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6466+0+0 (secure 0 0 0) 0x7fd830020070 con 0x7fd83406d080 2023-10-11T09:48:32.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.852+0000 7fd838d42700 1 --2- 172.21.15.78:0/2849060199 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd818065ed0 0x7fd818068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:32.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.852+0000 7fd838d42700 1 --2- 172.21.15.78:0/2849060199 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd818065ed0 0x7fd818068380 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd824009980 tx=0x7fd824008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:32.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:32.860+0000 7fd829ffb700 1 -- 172.21.15.78:0/2849060199 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fd830052400 con 0x7fd83406d080 2023-10-11T09:48:33.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.040+0000 7fd83b7a7700 1 -- 172.21.15.78:0/2849060199 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd834065db0 con 0x7fd83406d080 2023-10-11T09:48:33.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.040+0000 7fd829ffb700 1 -- 172.21.15.78:0/2849060199 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd830055a20 con 0x7fd83406d080 2023-10-11T09:48:33.051 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:48:33.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.044+0000 7fd83b7a7700 1 -- 172.21.15.78:0/2849060199 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd818065ed0 msgr2=0x7fd818068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:33.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.044+0000 7fd83b7a7700 1 --2- 172.21.15.78:0/2849060199 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd818065ed0 0x7fd818068380 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd824009980 tx=0x7fd824008040 comp rx=0 tx=0).stop 2023-10-11T09:48:33.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.044+0000 7fd83b7a7700 1 -- 172.21.15.78:0/2849060199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd83406d080 msgr2=0x7fd8341517e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:33.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.044+0000 7fd83b7a7700 1 --2- 172.21.15.78:0/2849060199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd83406d080 0x7fd8341517e0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fd830009150 tx=0x7fd830004770 comp rx=0 tx=0).stop 2023-10-11T09:48:33.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.044+0000 7fd83b7a7700 1 -- 172.21.15.78:0/2849060199 shutdown_connections 2023-10-11T09:48:33.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.044+0000 7fd83b7a7700 1 --2- 172.21.15.78:0/2849060199 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd818065ed0 0x7fd818068380 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:33.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.044+0000 7fd83b7a7700 1 --2- 172.21.15.78:0/2849060199 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd834160cb0 0x7fd83415baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:33.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.044+0000 7fd83b7a7700 1 --2- 172.21.15.78:0/2849060199 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8340b0e60 0x7fd83415b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:33.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.044+0000 7fd83b7a7700 1 --2- 172.21.15.78:0/2849060199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd83406d080 0x7fd8341517e0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:33.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.044+0000 7fd83b7a7700 1 -- 172.21.15.78:0/2849060199 >> 172.21.15.78:0/2849060199 conn(0x7fd8340cecc0 msgr2=0x7fd834070cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:33.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.044+0000 7fd83b7a7700 1 -- 172.21.15.78:0/2849060199 shutdown_connections 2023-10-11T09:48:33.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.044+0000 7fd83b7a7700 1 -- 172.21.15.78:0/2849060199 wait complete. 2023-10-11T09:48:33.074 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T09:48:33.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.240+0000 7f824565d700 1 Processor -- start 2023-10-11T09:48:33.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.240+0000 7f824565d700 1 -- start start 2023-10-11T09:48:33.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.244+0000 7f824565d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8240071e00 0x7f8240164d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:33.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.244+0000 7f824565d700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8240165260 0x7f82400cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:33.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.244+0000 7f824565d700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82401645d0 con 0x7f8240146e30 2023-10-11T09:48:33.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.244+0000 7f824565d700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8240072250 con 0x7f8240165260 2023-10-11T09:48:33.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.244+0000 7f824565d700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8240161830 con 0x7f8240071e00 2023-10-11T09:48:33.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.244+0000 7f8244e5c700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8240165260 0x7f82400cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:33.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.244+0000 7f82377fe700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8240071e00 0x7f8240164d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:33.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.244+0000 7f823ffff700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f8240146e30 0x7f8240160fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:51022/0 (socket says 172.21.15.78:51022) 2023-10-11T09:48:33.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.244+0000 7f823ffff700 1 -- 172.21.15.78:0/3389818255 learned_addr learned my addr 172.21.15.78:0/3389818255 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:33.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.244+0000 7f82377fe700 1 -- 172.21.15.78:0/3389818255 >> v2:172.21.15.133:3300/0 conn(0x7f8240165260 msgr2=0x7f82400cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:33.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.244+0000 7f82377fe700 1 --2- 172.21.15.78:0/3389818255 >> v2:172.21.15.133:3300/0 conn(0x7f8240165260 0x7f82400cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:33.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.244+0000 7f82377fe700 1 -- 172.21.15.78:0/3389818255 >> v1:172.21.15.133:6789/0 conn(0x7f8240146e30 legacy=0x7f8240160fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:33.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.244+0000 7f82377fe700 1 -- 172.21.15.78:0/3389818255 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8240161b10 con 0x7f8240071e00 2023-10-11T09:48:33.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.248+0000 7f82377fe700 1 --2- 172.21.15.78:0/3389818255 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8240071e00 0x7f8240164d20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8228009d90 tx=0x7f822800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f92a9c48d46ff1d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:33.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.248+0000 7f823f7fe700 1 -- 172.21.15.78:0/3389818255 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822800e070 con 0x7f8240071e00 2023-10-11T09:48:33.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.248+0000 7f823f7fe700 1 -- 172.21.15.78:0/3389818255 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822800bb20 con 0x7f8240071e00 2023-10-11T09:48:33.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.248+0000 7f823f7fe700 1 -- 172.21.15.78:0/3389818255 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8228012430 con 0x7f8240071e00 2023-10-11T09:48:33.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.248+0000 7f824565d700 1 -- 172.21.15.78:0/3389818255 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8240071e00 msgr2=0x7f8240164d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:33.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.248+0000 7f824565d700 1 --2- 172.21.15.78:0/3389818255 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8240071e00 0x7f8240164d20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8228009d90 tx=0x7f822800b330 comp rx=0 tx=0).stop 2023-10-11T09:48:33.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.248+0000 7f824565d700 1 -- 172.21.15.78:0/3389818255 shutdown_connections 2023-10-11T09:48:33.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.248+0000 7f824565d700 1 --2- 172.21.15.78:0/3389818255 >> v2:172.21.15.133:3300/0 conn(0x7f8240165260 0x7f82400cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:33.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.248+0000 7f824565d700 1 --2- 172.21.15.78:0/3389818255 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8240071e00 0x7f8240164d20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:33.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.248+0000 7f824565d700 1 -- 172.21.15.78:0/3389818255 >> 172.21.15.78:0/3389818255 conn(0x7f824006dd80 msgr2=0x7f824006e180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:33.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.248+0000 7f824565d700 1 -- 172.21.15.78:0/3389818255 shutdown_connections 2023-10-11T09:48:33.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.248+0000 7f824565d700 1 -- 172.21.15.78:0/3389818255 wait complete. 2023-10-11T09:48:33.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.252+0000 7f824565d700 1 Processor -- start 2023-10-11T09:48:33.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.252+0000 7f824565d700 1 -- start start 2023-10-11T09:48:33.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.252+0000 7f824565d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8240071e00 0x7f82400b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:33.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.252+0000 7f824565d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8240146e30 0x7f824015daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:33.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.252+0000 7f824565d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8240165260 0x7f8240149870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:33.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.252+0000 7f824565d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82400b39b0 con 0x7f8240071e00 2023-10-11T09:48:33.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.252+0000 7f824565d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82400ccb40 con 0x7f8240146e30 2023-10-11T09:48:33.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.252+0000 7f824565d700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82400cce20 con 0x7f8240165260 2023-10-11T09:48:33.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.252+0000 7f823ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8240071e00 0x7f82400b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:33.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.252+0000 7f8244e5c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8240165260 0x7f8240149870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:33.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.252+0000 7f823ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8240071e00 0x7f82400b3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57516/0 (socket says 172.21.15.78:57516) 2023-10-11T09:48:33.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.252+0000 7f823ffff700 1 -- 172.21.15.78:0/3092852692 learned_addr learned my addr 172.21.15.78:0/3092852692 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:33.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.252+0000 7f82377fe700 1 --2- 172.21.15.78:0/3092852692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8240146e30 0x7f824015daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:33.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.252+0000 7f823ffff700 1 -- 172.21.15.78:0/3092852692 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8240165260 msgr2=0x7f8240149870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:33.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.252+0000 7f823ffff700 1 --2- 172.21.15.78:0/3092852692 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8240165260 0x7f8240149870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:33.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.252+0000 7f823ffff700 1 -- 172.21.15.78:0/3092852692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8240146e30 msgr2=0x7f824015daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:33.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.252+0000 7f823ffff700 1 --2- 172.21.15.78:0/3092852692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8240146e30 0x7f824015daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:33.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.252+0000 7f823ffff700 1 -- 172.21.15.78:0/3092852692 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8228009a70 con 0x7f8240071e00 2023-10-11T09:48:33.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.252+0000 7f823ffff700 1 --2- 172.21.15.78:0/3092852692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8240071e00 0x7f82400b3e40 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f823000c500 tx=0x7f823000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:33.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.256+0000 7f823d7fa700 1 -- 172.21.15.78:0/3092852692 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8230005d90 con 0x7f8240071e00 2023-10-11T09:48:33.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.256+0000 7f823d7fa700 1 -- 172.21.15.78:0/3092852692 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82300070d0 con 0x7f8240071e00 2023-10-11T09:48:33.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.256+0000 7f824565d700 1 -- 172.21.15.78:0/3092852692 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82400cd100 con 0x7f8240071e00 2023-10-11T09:48:33.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.256+0000 7f823d7fa700 1 -- 172.21.15.78:0/3092852692 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8230007460 con 0x7f8240071e00 2023-10-11T09:48:33.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.256+0000 7f824565d700 1 -- 172.21.15.78:0/3092852692 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8240149db0 con 0x7f8240071e00 2023-10-11T09:48:33.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.256+0000 7f824565d700 1 -- 172.21.15.78:0/3092852692 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8240065d40 con 0x7f8240071e00 2023-10-11T09:48:33.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.256+0000 7f823d7fa700 1 -- 172.21.15.78:0/3092852692 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f82300075c0 con 0x7f8240071e00 2023-10-11T09:48:33.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.260+0000 7f823d7fa700 1 --2- 172.21.15.78:0/3092852692 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8220065ed0 0x7f8220068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:33.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.260+0000 7f823d7fa700 1 -- 172.21.15.78:0/3092852692 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f8230082dc0 con 0x7f8240071e00 2023-10-11T09:48:33.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.260+0000 7f82377fe700 1 --2- 172.21.15.78:0/3092852692 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8220065ed0 0x7f8220068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:33.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.260+0000 7f82377fe700 1 --2- 172.21.15.78:0/3092852692 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8220065ed0 0x7f8220068380 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8228014040 tx=0x7f8228011000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:33.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.264+0000 7f823d7fa700 1 -- 172.21.15.78:0/3092852692 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f82300501d0 con 0x7f8240071e00 2023-10-11T09:48:33.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.444+0000 7f824565d700 1 -- 172.21.15.78:0/3092852692 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f82400cfd10 con 0x7f8240071e00 2023-10-11T09:48:33.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.444+0000 7f823d7fa700 1 -- 172.21.15.78:0/3092852692 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8230017020 con 0x7f8240071e00 2023-10-11T09:48:33.452 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2023-10-11T09:48:33.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.444+0000 7f824565d700 1 -- 172.21.15.78:0/3092852692 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8220065ed0 msgr2=0x7f8220068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:33.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.444+0000 7f824565d700 1 --2- 172.21.15.78:0/3092852692 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8220065ed0 0x7f8220068380 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8228014040 tx=0x7f8228011000 comp rx=0 tx=0).stop 2023-10-11T09:48:33.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.444+0000 7f824565d700 1 -- 172.21.15.78:0/3092852692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8240071e00 msgr2=0x7f82400b3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:33.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.444+0000 7f824565d700 1 --2- 172.21.15.78:0/3092852692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8240071e00 0x7f82400b3e40 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f823000c500 tx=0x7f823000c810 comp rx=0 tx=0).stop 2023-10-11T09:48:33.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.448+0000 7f824565d700 1 -- 172.21.15.78:0/3092852692 shutdown_connections 2023-10-11T09:48:33.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.448+0000 7f824565d700 1 --2- 172.21.15.78:0/3092852692 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8220065ed0 0x7f8220068380 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:33.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.448+0000 7f824565d700 1 --2- 172.21.15.78:0/3092852692 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8240165260 0x7f8240149870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:33.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.448+0000 7f824565d700 1 --2- 172.21.15.78:0/3092852692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8240146e30 0x7f824015daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:33.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.448+0000 7f824565d700 1 --2- 172.21.15.78:0/3092852692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8240071e00 0x7f82400b3e40 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:33.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.448+0000 7f824565d700 1 -- 172.21.15.78:0/3092852692 >> 172.21.15.78:0/3092852692 conn(0x7f824006dd80 msgr2=0x7f824006a060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:33.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.448+0000 7f824565d700 1 -- 172.21.15.78:0/3092852692 shutdown_connections 2023-10-11T09:48:33.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.448+0000 7f824565d700 1 -- 172.21.15.78:0/3092852692 wait complete. 2023-10-11T09:48:33.476 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-10-11T09:48:33.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.644+0000 7f09de545700 1 Processor -- start 2023-10-11T09:48:33.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.644+0000 7f09de545700 1 -- start start 2023-10-11T09:48:33.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.648+0000 7f09de545700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09d80b3a60 0x7f09d806d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:33.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.648+0000 7f09de545700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09d806d9c0 con 0x7f09d815a6b0 2023-10-11T09:48:33.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.648+0000 7f09de545700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09d80b2d30 con 0x7f09d806d560 2023-10-11T09:48:33.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.648+0000 7f09de545700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d80cf360 con 0x7f09d80b3a60 2023-10-11T09:48:33.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.648+0000 7f09dd543700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f09d815a6b0 0x7f09d80ce8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:42950/0 (socket says 172.21.15.78:42950) 2023-10-11T09:48:33.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.648+0000 7f09dd543700 1 -- 172.21.15.78:0/51613401 learned_addr learned my addr 172.21.15.78:0/51613401 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:33.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.648+0000 7f09dcd42700 1 --2- 172.21.15.78:0/51613401 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09d80b3a60 0x7f09d806d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:33.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.648+0000 7f09cffff700 1 -- 172.21.15.78:0/51613401 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1329020723 0 0) 0x7f09d806d9c0 con 0x7f09d815a6b0 2023-10-11T09:48:33.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.648+0000 7f09cffff700 1 -- 172.21.15.78:0/51613401 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09c40035d0 con 0x7f09d815a6b0 2023-10-11T09:48:33.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.648+0000 7f09cffff700 1 -- 172.21.15.78:0/51613401 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2487654811 0 0) 0x7f09d80b2d30 con 0x7f09d806d560 2023-10-11T09:48:33.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.648+0000 7f09cffff700 1 -- 172.21.15.78:0/51613401 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09d806d9c0 con 0x7f09d806d560 2023-10-11T09:48:33.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.648+0000 7f09dcd42700 1 -- 172.21.15.78:0/51613401 >> v1:172.21.15.133:6789/0 conn(0x7f09d806d560 legacy=0x7f09d806bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:33.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.648+0000 7f09dcd42700 1 -- 172.21.15.78:0/51613401 >> v1:172.21.15.78:6789/0 conn(0x7f09d815a6b0 legacy=0x7f09d80ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:33.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.648+0000 7f09cffff700 1 -- 172.21.15.78:0/51613401 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3921191208 0 0) 0x7f09c40035d0 con 0x7f09d815a6b0 2023-10-11T09:48:33.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.648+0000 7f09dcd42700 1 -- 172.21.15.78:0/51613401 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d80b2f80 con 0x7f09d80b3a60 2023-10-11T09:48:33.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.652+0000 7f09dcd42700 1 --2- 172.21.15.78:0/51613401 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09d80b3a60 0x7f09d806d020 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f09c8009880 tx=0x7f09c8009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4323cf6f3b810e8d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:33.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.652+0000 7f09cffff700 1 -- 172.21.15.78:0/51613401 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d4001670 con 0x7f09d80b3a60 2023-10-11T09:48:33.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.652+0000 7f09cffff700 1 -- 172.21.15.78:0/51613401 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c8004030 con 0x7f09d80b3a60 2023-10-11T09:48:33.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.652+0000 7f09cffff700 1 -- 172.21.15.78:0/51613401 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c80043c0 con 0x7f09d80b3a60 2023-10-11T09:48:33.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.652+0000 7f09de545700 1 -- 172.21.15.78:0/51613401 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09d80b3a60 msgr2=0x7f09d806d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:33.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.652+0000 7f09de545700 1 --2- 172.21.15.78:0/51613401 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09d80b3a60 0x7f09d806d020 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f09c8009880 tx=0x7f09c8009b90 comp rx=0 tx=0).stop 2023-10-11T09:48:33.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.652+0000 7f09de545700 1 -- 172.21.15.78:0/51613401 shutdown_connections 2023-10-11T09:48:33.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.652+0000 7f09de545700 1 --2- 172.21.15.78:0/51613401 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09d80b3a60 0x7f09d806d020 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:33.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.652+0000 7f09de545700 1 -- 172.21.15.78:0/51613401 >> 172.21.15.78:0/51613401 conn(0x7f09d806a060 msgr2=0x7f09d806a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:33.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.652+0000 7f09de545700 1 -- 172.21.15.78:0/51613401 shutdown_connections 2023-10-11T09:48:33.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.652+0000 7f09de545700 1 -- 172.21.15.78:0/51613401 wait complete. 2023-10-11T09:48:33.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.656+0000 7f09de545700 1 Processor -- start 2023-10-11T09:48:33.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.656+0000 7f09de545700 1 -- start start 2023-10-11T09:48:33.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.656+0000 7f09de545700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09d806d560 0x7f09d8158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:33.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.656+0000 7f09de545700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09d80b3a60 0x7f09d81453a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:33.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.656+0000 7f09de545700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d815a6b0 0x7f09d81458e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:33.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.656+0000 7f09de545700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d81f5540 con 0x7f09d80b3a60 2023-10-11T09:48:33.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.656+0000 7f09de545700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d81f63a0 con 0x7f09d815a6b0 2023-10-11T09:48:33.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.656+0000 7f09de545700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d81f5720 con 0x7f09d806d560 2023-10-11T09:48:33.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.656+0000 7f09dcd42700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09d80b3a60 0x7f09d81453a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:33.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.656+0000 7f09dcd42700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09d80b3a60 0x7f09d81453a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57530/0 (socket says 172.21.15.78:57530) 2023-10-11T09:48:33.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.656+0000 7f09dcd42700 1 -- 172.21.15.78:0/3681130225 learned_addr learned my addr 172.21.15.78:0/3681130225 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:33.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.656+0000 7f09dd543700 1 --2- 172.21.15.78:0/3681130225 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09d806d560 0x7f09d8158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:33.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.656+0000 7f09ddd44700 1 --2- 172.21.15.78:0/3681130225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d815a6b0 0x7f09d81458e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:33.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.660+0000 7f09dcd42700 1 -- 172.21.15.78:0/3681130225 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09d806d560 msgr2=0x7f09d8158470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:33.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.660+0000 7f09dcd42700 1 --2- 172.21.15.78:0/3681130225 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09d806d560 0x7f09d8158470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:33.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.660+0000 7f09dcd42700 1 -- 172.21.15.78:0/3681130225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d815a6b0 msgr2=0x7f09d81458e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:33.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.660+0000 7f09dcd42700 1 --2- 172.21.15.78:0/3681130225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d815a6b0 0x7f09d81458e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:33.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.660+0000 7f09dcd42700 1 -- 172.21.15.78:0/3681130225 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c80094b0 con 0x7f09d80b3a60 2023-10-11T09:48:33.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.660+0000 7f09ddd44700 1 --2- 172.21.15.78:0/3681130225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d815a6b0 0x7f09d81458e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:48:33.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.660+0000 7f09dcd42700 1 --2- 172.21.15.78:0/3681130225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09d80b3a60 0x7f09d81453a0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f09c8014040 tx=0x7f09c8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:33.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.660+0000 7f09cdffb700 1 -- 172.21.15.78:0/3681130225 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c8003c90 con 0x7f09d80b3a60 2023-10-11T09:48:33.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.660+0000 7f09cdffb700 1 -- 172.21.15.78:0/3681130225 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c8003df0 con 0x7f09d80b3a60 2023-10-11T09:48:33.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.660+0000 7f09cdffb700 1 -- 172.21.15.78:0/3681130225 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c8011870 con 0x7f09d80b3a60 2023-10-11T09:48:33.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.660+0000 7f09de545700 1 -- 172.21.15.78:0/3681130225 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09d81f5a00 con 0x7f09d80b3a60 2023-10-11T09:48:33.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.660+0000 7f09de545700 1 -- 172.21.15.78:0/3681130225 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d81f5fc0 con 0x7f09d80b3a60 2023-10-11T09:48:33.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.660+0000 7f09de545700 1 -- 172.21.15.78:0/3681130225 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09d81f64e0 con 0x7f09d80b3a60 2023-10-11T09:48:33.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.664+0000 7f09cdffb700 1 -- 172.21.15.78:0/3681130225 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f09c8018050 con 0x7f09d80b3a60 2023-10-11T09:48:33.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.664+0000 7f09cdffb700 1 --2- 172.21.15.78:0/3681130225 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f09bc065ed0 0x7f09bc068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:33.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.664+0000 7f09cdffb700 1 -- 172.21.15.78:0/3681130225 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f09c80843f0 con 0x7f09d80b3a60 2023-10-11T09:48:33.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.664+0000 7f09dd543700 1 --2- 172.21.15.78:0/3681130225 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f09bc065ed0 0x7f09bc068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:33.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.668+0000 7f09dd543700 1 --2- 172.21.15.78:0/3681130225 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f09bc065ed0 0x7f09bc068380 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f09d4000f60 tx=0x7f09d4001690 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:33.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.672+0000 7f09cdffb700 1 -- 172.21.15.78:0/3681130225 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f09d81f64e0 con 0x7f09d80b3a60 2023-10-11T09:48:33.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.848+0000 7f09de545700 1 -- 172.21.15.78:0/3681130225 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7f09d81f64e0 con 0x7f09d80b3a60 2023-10-11T09:48:33.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.848+0000 7f09cdffb700 1 -- 172.21.15.78:0/3681130225 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v86) v1 ==== 106+0+11 (secure 0 0 0) 0x7f09d81f64e0 con 0x7f09d80b3a60 2023-10-11T09:48:33.859 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:48:33.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.852+0000 7f09de545700 1 -- 172.21.15.78:0/3681130225 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f09bc065ed0 msgr2=0x7f09bc068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:33.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.852+0000 7f09de545700 1 --2- 172.21.15.78:0/3681130225 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f09bc065ed0 0x7f09bc068380 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f09d4000f60 tx=0x7f09d4001690 comp rx=0 tx=0).stop 2023-10-11T09:48:33.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.852+0000 7f09de545700 1 -- 172.21.15.78:0/3681130225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09d80b3a60 msgr2=0x7f09d81453a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:33.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.852+0000 7f09de545700 1 --2- 172.21.15.78:0/3681130225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09d80b3a60 0x7f09d81453a0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f09c8014040 tx=0x7f09c8004770 comp rx=0 tx=0).stop 2023-10-11T09:48:33.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.852+0000 7f09de545700 1 -- 172.21.15.78:0/3681130225 shutdown_connections 2023-10-11T09:48:33.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.852+0000 7f09de545700 1 --2- 172.21.15.78:0/3681130225 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f09bc065ed0 0x7f09bc068380 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:33.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.852+0000 7f09de545700 1 --2- 172.21.15.78:0/3681130225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d815a6b0 0x7f09d81458e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:33.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.852+0000 7f09de545700 1 --2- 172.21.15.78:0/3681130225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09d80b3a60 0x7f09d81453a0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:33.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.852+0000 7f09de545700 1 --2- 172.21.15.78:0/3681130225 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09d806d560 0x7f09d8158470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:33.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.852+0000 7f09de545700 1 -- 172.21.15.78:0/3681130225 >> 172.21.15.78:0/3681130225 conn(0x7f09d806a060 msgr2=0x7f09d806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:33.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.852+0000 7f09de545700 1 -- 172.21.15.78:0/3681130225 shutdown_connections 2023-10-11T09:48:33.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:33.852+0000 7f09de545700 1 -- 172.21.15.78:0/3681130225 wait complete. 2023-10-11T09:48:33.882 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T09:48:34.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.048+0000 7fc3248a3700 1 Processor -- start 2023-10-11T09:48:34.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.048+0000 7fc3248a3700 1 -- start start 2023-10-11T09:48:34.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.052+0000 7fc3248a3700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc320154e00 0x7fc32015b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:34.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.052+0000 7fc3248a3700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fc3200cbbd0 0x7fc32006c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:34.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.052+0000 7fc3248a3700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3200720f0 con 0x7fc320071040 2023-10-11T09:48:34.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.052+0000 7fc3248a3700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc32015bb90 con 0x7fc3200cbbd0 2023-10-11T09:48:34.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.052+0000 7fc3248a3700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3200714e0 con 0x7fc320154e00 2023-10-11T09:48:34.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.052+0000 7fc31e59c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc320154e00 0x7fc32015b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:34.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.052+0000 7fc317fff700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fc320071040 0x7fc3200cb4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:51042/0 (socket says 172.21.15.78:51042) 2023-10-11T09:48:34.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.052+0000 7fc31ed9d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fc3200cbbd0 0x7fc32006c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:34.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.052+0000 7fc317fff700 1 -- 172.21.15.78:0/4190969489 learned_addr learned my addr 172.21.15.78:0/4190969489 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:34.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.052+0000 7fc31dd9b700 1 -- 172.21.15.78:0/4190969489 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2958978425 0 0) 0x7fc3200720f0 con 0x7fc320071040 2023-10-11T09:48:34.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.052+0000 7fc31dd9b700 1 -- 172.21.15.78:0/4190969489 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc30c0035d0 con 0x7fc320071040 2023-10-11T09:48:34.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.056+0000 7fc31ed9d700 1 -- 172.21.15.78:0/4190969489 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc320154e00 msgr2=0x7fc32015b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:34.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.056+0000 7fc31ed9d700 1 --2- 172.21.15.78:0/4190969489 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc320154e00 0x7fc32015b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:34.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.056+0000 7fc31ed9d700 1 -- 172.21.15.78:0/4190969489 >> v1:172.21.15.133:6789/0 conn(0x7fc320071040 legacy=0x7fc3200cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:34.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.056+0000 7fc31ed9d700 1 -- 172.21.15.78:0/4190969489 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3200717c0 con 0x7fc3200cbbd0 2023-10-11T09:48:34.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.056+0000 7fc31e59c700 1 --2- 172.21.15.78:0/4190969489 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc320154e00 0x7fc32015b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:48:34.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.056+0000 7fc31ed9d700 1 --2- 172.21.15.78:0/4190969489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3200cbbd0 0x7fc32006c010 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fc318009d90 tx=0x7fc31800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76db552e9835075 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:34.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.056+0000 7fc31dd9b700 1 -- 172.21.15.78:0/4190969489 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31800e070 con 0x7fc3200cbbd0 2023-10-11T09:48:34.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.056+0000 7fc31dd9b700 1 -- 172.21.15.78:0/4190969489 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc31800bb20 con 0x7fc3200cbbd0 2023-10-11T09:48:34.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.056+0000 7fc31dd9b700 1 -- 172.21.15.78:0/4190969489 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc318012430 con 0x7fc3200cbbd0 2023-10-11T09:48:34.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.056+0000 7fc3248a3700 1 -- 172.21.15.78:0/4190969489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3200cbbd0 msgr2=0x7fc32006c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:34.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.056+0000 7fc3248a3700 1 --2- 172.21.15.78:0/4190969489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3200cbbd0 0x7fc32006c010 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fc318009d90 tx=0x7fc31800b330 comp rx=0 tx=0).stop 2023-10-11T09:48:34.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.056+0000 7fc3248a3700 1 -- 172.21.15.78:0/4190969489 shutdown_connections 2023-10-11T09:48:34.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.056+0000 7fc3248a3700 1 --2- 172.21.15.78:0/4190969489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3200cbbd0 0x7fc32006c010 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:34.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.056+0000 7fc3248a3700 1 --2- 172.21.15.78:0/4190969489 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc320154e00 0x7fc32015b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:34.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.056+0000 7fc3248a3700 1 -- 172.21.15.78:0/4190969489 >> 172.21.15.78:0/4190969489 conn(0x7fc32006aca0 msgr2=0x7fc3200b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:34.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.056+0000 7fc3248a3700 1 -- 172.21.15.78:0/4190969489 shutdown_connections 2023-10-11T09:48:34.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.056+0000 7fc3248a3700 1 -- 172.21.15.78:0/4190969489 wait complete. 2023-10-11T09:48:34.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.060+0000 7fc3248a3700 1 Processor -- start 2023-10-11T09:48:34.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.060+0000 7fc3248a3700 1 -- start start 2023-10-11T09:48:34.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.060+0000 7fc3248a3700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc320071040 0x7fc320155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:34.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.060+0000 7fc3248a3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3200cbbd0 0x7fc3201f1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:34.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.060+0000 7fc3248a3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc320154e00 0x7fc3200ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:34.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.060+0000 7fc3248a3700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3201f1d70 con 0x7fc3200cbbd0 2023-10-11T09:48:34.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.060+0000 7fc3248a3700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3201584e0 con 0x7fc320154e00 2023-10-11T09:48:34.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.060+0000 7fc3248a3700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc32014f5c0 con 0x7fc320071040 2023-10-11T09:48:34.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.060+0000 7fc317fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3200cbbd0 0x7fc3201f1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:34.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.060+0000 7fc317fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3200cbbd0 0x7fc3201f1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57546/0 (socket says 172.21.15.78:57546) 2023-10-11T09:48:34.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.060+0000 7fc317fff700 1 -- 172.21.15.78:0/730173810 learned_addr learned my addr 172.21.15.78:0/730173810 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:34.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.060+0000 7fc31ed9d700 1 --2- 172.21.15.78:0/730173810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc320154e00 0x7fc3200ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:34.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.060+0000 7fc31e59c700 1 --2- 172.21.15.78:0/730173810 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc320071040 0x7fc320155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:34.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.064+0000 7fc317fff700 1 -- 172.21.15.78:0/730173810 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc320071040 msgr2=0x7fc320155260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:34.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.064+0000 7fc317fff700 1 --2- 172.21.15.78:0/730173810 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc320071040 0x7fc320155260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:34.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.064+0000 7fc317fff700 1 -- 172.21.15.78:0/730173810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc320154e00 msgr2=0x7fc3200ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:34.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.064+0000 7fc317fff700 1 --2- 172.21.15.78:0/730173810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc320154e00 0x7fc3200ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:34.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.064+0000 7fc317fff700 1 -- 172.21.15.78:0/730173810 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3100094b0 con 0x7fc3200cbbd0 2023-10-11T09:48:34.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.064+0000 7fc31e59c700 1 --2- 172.21.15.78:0/730173810 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc320071040 0x7fc320155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:48:34.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.064+0000 7fc317fff700 1 --2- 172.21.15.78:0/730173810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3200cbbd0 0x7fc3201f1830 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fc308005930 tx=0x7fc30800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:34.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.064+0000 7fc316ffd700 1 -- 172.21.15.78:0/730173810 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc308005d90 con 0x7fc3200cbbd0 2023-10-11T09:48:34.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.064+0000 7fc316ffd700 1 -- 172.21.15.78:0/730173810 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3080070d0 con 0x7fc3200cbbd0 2023-10-11T09:48:34.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.064+0000 7fc316ffd700 1 -- 172.21.15.78:0/730173810 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc308007480 con 0x7fc3200cbbd0 2023-10-11T09:48:34.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.064+0000 7fc3248a3700 1 -- 172.21.15.78:0/730173810 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc318009a70 con 0x7fc3200cbbd0 2023-10-11T09:48:34.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.064+0000 7fc3248a3700 1 -- 172.21.15.78:0/730173810 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc320155bd0 con 0x7fc3200cbbd0 2023-10-11T09:48:34.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.064+0000 7fc3248a3700 1 -- 172.21.15.78:0/730173810 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc32004e530 con 0x7fc3200cbbd0 2023-10-11T09:48:34.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.068+0000 7fc316ffd700 1 -- 172.21.15.78:0/730173810 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fc3080075e0 con 0x7fc3200cbbd0 2023-10-11T09:48:34.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.068+0000 7fc316ffd700 1 --2- 172.21.15.78:0/730173810 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc300065ed0 0x7fc300068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:34.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.068+0000 7fc316ffd700 1 -- 172.21.15.78:0/730173810 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6466+0+0 (secure 0 0 0) 0x7fc308082e70 con 0x7fc3200cbbd0 2023-10-11T09:48:34.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.068+0000 7fc31e59c700 1 --2- 172.21.15.78:0/730173810 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc300065ed0 0x7fc300068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:34.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.068+0000 7fc31e59c700 1 --2- 172.21.15.78:0/730173810 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc300065ed0 0x7fc300068380 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc310009dd0 tx=0x7fc310008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:34.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.072+0000 7fc316ffd700 1 -- 172.21.15.78:0/730173810 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fc308050280 con 0x7fc3200cbbd0 2023-10-11T09:48:34.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.260+0000 7fc3248a3700 1 -- 172.21.15.78:0/730173810 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc32006e1e0 con 0x7fc3200cbbd0 2023-10-11T09:48:34.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.260+0000 7fc316ffd700 1 -- 172.21.15.78:0/730173810 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v86) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc308017080 con 0x7fc3200cbbd0 2023-10-11T09:48:34.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.260+0000 7fc3248a3700 1 -- 172.21.15.78:0/730173810 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc300065ed0 msgr2=0x7fc300068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:34.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.260+0000 7fc3248a3700 1 --2- 172.21.15.78:0/730173810 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc300065ed0 0x7fc300068380 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc310009dd0 tx=0x7fc310008040 comp rx=0 tx=0).stop 2023-10-11T09:48:34.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.260+0000 7fc3248a3700 1 -- 172.21.15.78:0/730173810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3200cbbd0 msgr2=0x7fc3201f1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:34.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.260+0000 7fc3248a3700 1 --2- 172.21.15.78:0/730173810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3200cbbd0 0x7fc3201f1830 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fc308005930 tx=0x7fc30800c7b0 comp rx=0 tx=0).stop 2023-10-11T09:48:34.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.260+0000 7fc3248a3700 1 -- 172.21.15.78:0/730173810 shutdown_connections 2023-10-11T09:48:34.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.260+0000 7fc3248a3700 1 --2- 172.21.15.78:0/730173810 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc300065ed0 0x7fc300068380 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:34.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.260+0000 7fc3248a3700 1 --2- 172.21.15.78:0/730173810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc320154e00 0x7fc3200ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:34.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.260+0000 7fc3248a3700 1 --2- 172.21.15.78:0/730173810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3200cbbd0 0x7fc3201f1830 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:34.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.260+0000 7fc3248a3700 1 --2- 172.21.15.78:0/730173810 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc320071040 0x7fc320155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:34.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.260+0000 7fc3248a3700 1 -- 172.21.15.78:0/730173810 >> 172.21.15.78:0/730173810 conn(0x7fc32006aca0 msgr2=0x7fc3200adc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:34.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.264+0000 7fc3248a3700 1 -- 172.21.15.78:0/730173810 shutdown_connections 2023-10-11T09:48:34.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.264+0000 7fc3248a3700 1 -- 172.21.15.78:0/730173810 wait complete. 2023-10-11T09:48:34.274 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2023-10-11T09:48:34.296 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:48:34.296 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:48:34.312 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:48:34.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.516+0000 7efd211a8700 1 Processor -- start 2023-10-11T09:48:34.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.516+0000 7efd211a8700 1 -- start start 2023-10-11T09:48:34.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.520+0000 7efd211a8700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efd1c15a940 0x7efd1c0ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:34.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.520+0000 7efd211a8700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd1c0ca780 0x7efd1c13fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:34.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.520+0000 7efd211a8700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd1c0cabc0 con 0x7efd1c15b8e0 2023-10-11T09:48:34.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.520+0000 7efd211a8700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd1c15e010 con 0x7efd1c15a940 2023-10-11T09:48:34.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.520+0000 7efd211a8700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c15e280 con 0x7efd1c0ca780 2023-10-11T09:48:34.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.520+0000 7efd209a7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd1c0ca780 0x7efd1c13fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:34.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.520+0000 7efd1bfff700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7efd1c15b8e0 0x7efd1c06d0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:51046/0 (socket says 172.21.15.78:51046) 2023-10-11T09:48:34.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.520+0000 7efd1bfff700 1 -- 172.21.15.78:0/2827155605 learned_addr learned my addr 172.21.15.78:0/2827155605 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:34.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.520+0000 7efd137fe700 1 --2- 172.21.15.78:0/2827155605 >> v2:172.21.15.133:3300/0 conn(0x7efd1c15a940 0x7efd1c0ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:34.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.520+0000 7efd209a7700 1 -- 172.21.15.78:0/2827155605 >> v2:172.21.15.133:3300/0 conn(0x7efd1c15a940 msgr2=0x7efd1c0ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:34.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.520+0000 7efd209a7700 1 --2- 172.21.15.78:0/2827155605 >> v2:172.21.15.133:3300/0 conn(0x7efd1c15a940 0x7efd1c0ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:34.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.520+0000 7efd209a7700 1 -- 172.21.15.78:0/2827155605 >> v1:172.21.15.133:6789/0 conn(0x7efd1c15b8e0 legacy=0x7efd1c06d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:34.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.520+0000 7efd209a7700 1 -- 172.21.15.78:0/2827155605 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd1c15e560 con 0x7efd1c0ca780 2023-10-11T09:48:34.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.520+0000 7efd1b7fe700 1 -- 172.21.15.78:0/2827155605 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1320977966 0 0) 0x7efd1c0cabc0 con 0x7efd1c15b8e0 2023-10-11T09:48:34.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.520+0000 7efd137fe700 1 --2- 172.21.15.78:0/2827155605 >> v2:172.21.15.133:3300/0 conn(0x7efd1c15a940 0x7efd1c0ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:48:34.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.524+0000 7efd209a7700 1 --2- 172.21.15.78:0/2827155605 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd1c0ca780 0x7efd1c13fcb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efd14009850 tx=0x7efd14009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9a6fd98fe0e5b08 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:34.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.524+0000 7efd1b7fe700 1 -- 172.21.15.78:0/2827155605 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd14011830 con 0x7efd1c0ca780 2023-10-11T09:48:34.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.524+0000 7efd1b7fe700 1 -- 172.21.15.78:0/2827155605 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd14011990 con 0x7efd1c0ca780 2023-10-11T09:48:34.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.524+0000 7efd1b7fe700 1 -- 172.21.15.78:0/2827155605 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd14011d40 con 0x7efd1c0ca780 2023-10-11T09:48:34.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.524+0000 7efd211a8700 1 -- 172.21.15.78:0/2827155605 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd1c0ca780 msgr2=0x7efd1c13fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:34.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.524+0000 7efd211a8700 1 --2- 172.21.15.78:0/2827155605 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd1c0ca780 0x7efd1c13fcb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efd14009850 tx=0x7efd14009b60 comp rx=0 tx=0).stop 2023-10-11T09:48:34.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.524+0000 7efd211a8700 1 -- 172.21.15.78:0/2827155605 shutdown_connections 2023-10-11T09:48:34.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.524+0000 7efd211a8700 1 --2- 172.21.15.78:0/2827155605 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd1c0ca780 0x7efd1c13fcb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:34.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.524+0000 7efd211a8700 1 --2- 172.21.15.78:0/2827155605 >> v2:172.21.15.133:3300/0 conn(0x7efd1c15a940 0x7efd1c0ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:34.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.524+0000 7efd211a8700 1 -- 172.21.15.78:0/2827155605 >> 172.21.15.78:0/2827155605 conn(0x7efd1c0cea60 msgr2=0x7efd1c0cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:34.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.524+0000 7efd211a8700 1 -- 172.21.15.78:0/2827155605 shutdown_connections 2023-10-11T09:48:34.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.524+0000 7efd211a8700 1 -- 172.21.15.78:0/2827155605 wait complete. 2023-10-11T09:48:34.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.528+0000 7efd211a8700 1 Processor -- start 2023-10-11T09:48:34.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.528+0000 7efd211a8700 1 -- start start 2023-10-11T09:48:34.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.528+0000 7efd211a8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd1c0ca780 0x7efd1c0c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:34.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.528+0000 7efd211a8700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd1c15a940 0x7efd1c0cd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:34.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.528+0000 7efd211a8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd1c15b8e0 0x7efd1c159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:34.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.528+0000 7efd211a8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c13f210 con 0x7efd1c0ca780 2023-10-11T09:48:34.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.528+0000 7efd211a8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c12f5c0 con 0x7efd1c15b8e0 2023-10-11T09:48:34.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.528+0000 7efd211a8700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c0ce3c0 con 0x7efd1c15a940 2023-10-11T09:48:34.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.532+0000 7efd1bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd1c0ca780 0x7efd1c0c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:34.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.532+0000 7efd1bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd1c0ca780 0x7efd1c0c8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57560/0 (socket says 172.21.15.78:57560) 2023-10-11T09:48:34.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.532+0000 7efd1bfff700 1 -- 172.21.15.78:0/2744241782 learned_addr learned my addr 172.21.15.78:0/2744241782 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:34.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.532+0000 7efd209a7700 1 --2- 172.21.15.78:0/2744241782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd1c15b8e0 0x7efd1c159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:34.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.532+0000 7efd137fe700 1 --2- 172.21.15.78:0/2744241782 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd1c15a940 0x7efd1c0cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:34.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.532+0000 7efd1bfff700 1 -- 172.21.15.78:0/2744241782 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd1c15a940 msgr2=0x7efd1c0cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:34.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.532+0000 7efd1bfff700 1 --2- 172.21.15.78:0/2744241782 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd1c15a940 0x7efd1c0cd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:34.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.532+0000 7efd1bfff700 1 -- 172.21.15.78:0/2744241782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd1c15b8e0 msgr2=0x7efd1c159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:34.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.532+0000 7efd1bfff700 1 --2- 172.21.15.78:0/2744241782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd1c15b8e0 0x7efd1c159820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:34.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.532+0000 7efd1bfff700 1 -- 172.21.15.78:0/2744241782 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd04009a70 con 0x7efd1c0ca780 2023-10-11T09:48:34.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.532+0000 7efd1bfff700 1 --2- 172.21.15.78:0/2744241782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd1c0ca780 0x7efd1c0c8000 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7efd0c00c4e0 tx=0x7efd0c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:34.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.532+0000 7efd197fa700 1 -- 172.21.15.78:0/2744241782 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0c004ca0 con 0x7efd1c0ca780 2023-10-11T09:48:34.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.532+0000 7efd197fa700 1 -- 172.21.15.78:0/2744241782 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd0c004e20 con 0x7efd1c0ca780 2023-10-11T09:48:34.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.532+0000 7efd211a8700 1 -- 172.21.15.78:0/2744241782 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd140094b0 con 0x7efd1c0ca780 2023-10-11T09:48:34.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.532+0000 7efd197fa700 1 -- 172.21.15.78:0/2744241782 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0c007310 con 0x7efd1c0ca780 2023-10-11T09:48:34.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.532+0000 7efd211a8700 1 -- 172.21.15.78:0/2744241782 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd1c12f250 con 0x7efd1c0ca780 2023-10-11T09:48:34.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.532+0000 7efd211a8700 1 -- 172.21.15.78:0/2744241782 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd1c12edd0 con 0x7efd1c0ca780 2023-10-11T09:48:34.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.536+0000 7efd197fa700 1 -- 172.21.15.78:0/2744241782 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7efd0c007470 con 0x7efd1c0ca780 2023-10-11T09:48:34.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.536+0000 7efd197fa700 1 --2- 172.21.15.78:0/2744241782 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efcfc065ed0 0x7efcfc068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:34.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.536+0000 7efd197fa700 1 -- 172.21.15.78:0/2744241782 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6466+0+0 (secure 0 0 0) 0x7efd0c083890 con 0x7efd1c0ca780 2023-10-11T09:48:34.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.536+0000 7efd137fe700 1 --2- 172.21.15.78:0/2744241782 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efcfc065ed0 0x7efcfc068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:34.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.540+0000 7efd137fe700 1 --2- 172.21.15.78:0/2744241782 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efcfc065ed0 0x7efcfc068380 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7efd1c0c8950 tx=0x7efd0400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:34.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.544+0000 7efd197fa700 1 -- 172.21.15.78:0/2744241782 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7efd0c050eb0 con 0x7efd1c0ca780 2023-10-11T09:48:34.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.716+0000 7efd211a8700 1 -- 172.21.15.78:0/2744241782 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efd1c12edd0 con 0x7efd1c0ca780 2023-10-11T09:48:34.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.720+0000 7efd197fa700 1 -- 172.21.15.78:0/2744241782 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+19671 (secure 0 0 0) 0x7efd1c12edd0 con 0x7efd1c0ca780 2023-10-11T09:48:34.728 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:34.729 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":86,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:48:30.824033+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:48:18.491556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-11T09:48:19.018937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data_extra","create_time":"2023-10-11T09:48:28.963956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:48:34.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.720+0000 7efd211a8700 1 -- 172.21.15.78:0/2744241782 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efcfc065ed0 msgr2=0x7efcfc068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:34.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.720+0000 7efd211a8700 1 --2- 172.21.15.78:0/2744241782 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efcfc065ed0 0x7efcfc068380 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7efd1c0c8950 tx=0x7efd0400b040 comp rx=0 tx=0).stop 2023-10-11T09:48:34.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.720+0000 7efd211a8700 1 -- 172.21.15.78:0/2744241782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd1c0ca780 msgr2=0x7efd1c0c8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:34.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.720+0000 7efd211a8700 1 --2- 172.21.15.78:0/2744241782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd1c0ca780 0x7efd1c0c8000 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7efd0c00c4e0 tx=0x7efd0c00c7f0 comp rx=0 tx=0).stop 2023-10-11T09:48:34.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.720+0000 7efd211a8700 1 -- 172.21.15.78:0/2744241782 shutdown_connections 2023-10-11T09:48:34.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.720+0000 7efd211a8700 1 --2- 172.21.15.78:0/2744241782 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efcfc065ed0 0x7efcfc068380 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:34.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.720+0000 7efd211a8700 1 --2- 172.21.15.78:0/2744241782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd1c15b8e0 0x7efd1c159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:34.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.720+0000 7efd211a8700 1 --2- 172.21.15.78:0/2744241782 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd1c15a940 0x7efd1c0cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:34.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.720+0000 7efd211a8700 1 --2- 172.21.15.78:0/2744241782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd1c0ca780 0x7efd1c0c8000 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:34.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.720+0000 7efd211a8700 1 -- 172.21.15.78:0/2744241782 >> 172.21.15.78:0/2744241782 conn(0x7efd1c0cea60 msgr2=0x7efd1c0c99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:34.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.720+0000 7efd211a8700 1 -- 172.21.15.78:0/2744241782 shutdown_connections 2023-10-11T09:48:34.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.720+0000 7efd211a8700 1 -- 172.21.15.78:0/2744241782 wait complete. 2023-10-11T09:48:34.752 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T09:48:18.491556+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T09:48:19.018937+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-10-11T09:48:28.963956+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:48:34.753 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:48:34.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.916+0000 7f8e4b59e700 1 Processor -- start 2023-10-11T09:48:34.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.916+0000 7f8e4b59e700 1 -- start start 2023-10-11T09:48:34.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.920+0000 7f8e4b59e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8e4c15a6b0 0x7f8e4c0ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:34.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.920+0000 7f8e4b59e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8e4c0b3a60 0x7f8e4c06d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:34.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.920+0000 7f8e4b59e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e4c0b2d30 con 0x7f8e4c0cede0 2023-10-11T09:48:34.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.920+0000 7f8e4b59e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e4c0cf360 con 0x7f8e4c15a6b0 2023-10-11T09:48:34.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.920+0000 7f8e4b59e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c0b2f80 con 0x7f8e4c0b3a60 2023-10-11T09:48:34.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.920+0000 7f8e4ad9d700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f8e4c0cede0 0x7f8e4c0b1a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:50056/0 (socket says 172.21.15.78:50056) 2023-10-11T09:48:34.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.920+0000 7f8e4ad9d700 1 -- 172.21.15.78:0/1652129341 learned_addr learned my addr 172.21.15.78:0/1652129341 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:34.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.920+0000 7f8e4a59c700 1 --2- 172.21.15.78:0/1652129341 >> v2:172.21.15.133:3300/0 conn(0x7f8e4c15a6b0 0x7f8e4c0ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:34.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.920+0000 7f8e49d9b700 1 --2- 172.21.15.78:0/1652129341 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8e4c0b3a60 0x7f8e4c06d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:34.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.920+0000 7f8e4959a700 1 -- 172.21.15.78:0/1652129341 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2522825850 0 0) 0x7f8e4c0b2d30 con 0x7f8e4c0cede0 2023-10-11T09:48:34.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.920+0000 7f8e4959a700 1 -- 172.21.15.78:0/1652129341 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e340035d0 con 0x7f8e4c0cede0 2023-10-11T09:48:34.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.920+0000 7f8e4a59c700 1 -- 172.21.15.78:0/1652129341 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8e4c0b3a60 msgr2=0x7f8e4c06d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:34.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.920+0000 7f8e4a59c700 1 --2- 172.21.15.78:0/1652129341 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8e4c0b3a60 0x7f8e4c06d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:34.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.920+0000 7f8e4a59c700 1 -- 172.21.15.78:0/1652129341 >> v1:172.21.15.78:6789/0 conn(0x7f8e4c0cede0 legacy=0x7f8e4c0b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:34.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.920+0000 7f8e4a59c700 1 -- 172.21.15.78:0/1652129341 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4c0b3260 con 0x7f8e4c15a6b0 2023-10-11T09:48:34.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.924+0000 7f8e4a59c700 1 --2- 172.21.15.78:0/1652129341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e4c15a6b0 0x7f8e4c0ce8a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c009d90 tx=0x7f8e3c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea0f2a23eaf65ca1 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:34.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.924+0000 7f8e4959a700 1 -- 172.21.15.78:0/1652129341 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c00e070 con 0x7f8e4c15a6b0 2023-10-11T09:48:34.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.924+0000 7f8e4959a700 1 -- 172.21.15.78:0/1652129341 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e3c00bb20 con 0x7f8e4c15a6b0 2023-10-11T09:48:34.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.924+0000 7f8e4959a700 1 -- 172.21.15.78:0/1652129341 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c012430 con 0x7f8e4c15a6b0 2023-10-11T09:48:34.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.924+0000 7f8e4b59e700 1 -- 172.21.15.78:0/1652129341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e4c15a6b0 msgr2=0x7f8e4c0ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:34.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.924+0000 7f8e4b59e700 1 --2- 172.21.15.78:0/1652129341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e4c15a6b0 0x7f8e4c0ce8a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c009d90 tx=0x7f8e3c00b330 comp rx=0 tx=0).stop 2023-10-11T09:48:34.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.924+0000 7f8e4b59e700 1 -- 172.21.15.78:0/1652129341 shutdown_connections 2023-10-11T09:48:34.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.924+0000 7f8e4b59e700 1 --2- 172.21.15.78:0/1652129341 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8e4c0b3a60 0x7f8e4c06d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:34.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.924+0000 7f8e4b59e700 1 --2- 172.21.15.78:0/1652129341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e4c15a6b0 0x7f8e4c0ce8a0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:34.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.924+0000 7f8e4b59e700 1 -- 172.21.15.78:0/1652129341 >> 172.21.15.78:0/1652129341 conn(0x7f8e4c06a060 msgr2=0x7f8e4c06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:34.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.924+0000 7f8e4b59e700 1 -- 172.21.15.78:0/1652129341 shutdown_connections 2023-10-11T09:48:34.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.924+0000 7f8e4b59e700 1 -- 172.21.15.78:0/1652129341 wait complete. 2023-10-11T09:48:34.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.928+0000 7f8e4b59e700 1 Processor -- start 2023-10-11T09:48:34.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.928+0000 7f8e4b59e700 1 -- start start 2023-10-11T09:48:34.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.932+0000 7f8e4b59e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8e4c0b3a60 0x7f8e4c145a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:34.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.932+0000 7f8e4b59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e4c0cede0 0x7f8e4c146af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:34.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.932+0000 7f8e4b59e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e4c15a6b0 0x7f8e4c0aee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:34.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.932+0000 7f8e4b59e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c145fa0 con 0x7f8e4c0cede0 2023-10-11T09:48:34.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.932+0000 7f8e4b59e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c1575b0 con 0x7f8e4c15a6b0 2023-10-11T09:48:34.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.932+0000 7f8e4b59e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c157820 con 0x7f8e4c0b3a60 2023-10-11T09:48:34.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.932+0000 7f8e49d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e4c0cede0 0x7f8e4c146af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:34.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.932+0000 7f8e49d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e4c0cede0 0x7f8e4c146af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44102/0 (socket says 172.21.15.78:44102) 2023-10-11T09:48:34.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.932+0000 7f8e49d9b700 1 -- 172.21.15.78:0/1689260476 learned_addr learned my addr 172.21.15.78:0/1689260476 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:34.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.932+0000 7f8e4a59c700 1 --2- 172.21.15.78:0/1689260476 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8e4c0b3a60 0x7f8e4c145a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:34.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.932+0000 7f8e4ad9d700 1 --2- 172.21.15.78:0/1689260476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e4c15a6b0 0x7f8e4c0aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:34.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.932+0000 7f8e49d9b700 1 -- 172.21.15.78:0/1689260476 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8e4c0b3a60 msgr2=0x7f8e4c145a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:34.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.932+0000 7f8e49d9b700 1 --2- 172.21.15.78:0/1689260476 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8e4c0b3a60 0x7f8e4c145a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:34.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.932+0000 7f8e49d9b700 1 -- 172.21.15.78:0/1689260476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e4c15a6b0 msgr2=0x7f8e4c0aee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:34.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.932+0000 7f8e49d9b700 1 --2- 172.21.15.78:0/1689260476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e4c15a6b0 0x7f8e4c0aee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:34.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.932+0000 7f8e49d9b700 1 -- 172.21.15.78:0/1689260476 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e3c009a70 con 0x7f8e4c0cede0 2023-10-11T09:48:34.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.932+0000 7f8e49d9b700 1 --2- 172.21.15.78:0/1689260476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e4c0cede0 0x7f8e4c146af0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000e9a0 tx=0x7f8e4000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:34.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.932+0000 7f8e3affd700 1 -- 172.21.15.78:0/1689260476 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e40009720 con 0x7f8e4c0cede0 2023-10-11T09:48:34.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.932+0000 7f8e3affd700 1 -- 172.21.15.78:0/1689260476 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e4000c690 con 0x7f8e4c0cede0 2023-10-11T09:48:34.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.932+0000 7f8e4b59e700 1 -- 172.21.15.78:0/1689260476 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e4c157b00 con 0x7f8e4c0cede0 2023-10-11T09:48:34.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.932+0000 7f8e3affd700 1 -- 172.21.15.78:0/1689260476 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4000ca40 con 0x7f8e4c0cede0 2023-10-11T09:48:34.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.932+0000 7f8e4b59e700 1 -- 172.21.15.78:0/1689260476 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e4c1463b0 con 0x7f8e4c0cede0 2023-10-11T09:48:34.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.936+0000 7f8e4b59e700 1 -- 172.21.15.78:0/1689260476 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e4c065d40 con 0x7f8e4c0cede0 2023-10-11T09:48:34.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.936+0000 7f8e3affd700 1 -- 172.21.15.78:0/1689260476 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f8e4000cba0 con 0x7f8e4c0cede0 2023-10-11T09:48:34.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.940+0000 7f8e3affd700 1 --2- 172.21.15.78:0/1689260476 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8e2c065ed0 0x7f8e2c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:34.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.940+0000 7f8e3affd700 1 -- 172.21.15.78:0/1689260476 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f8e40083950 con 0x7f8e4c0cede0 2023-10-11T09:48:34.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.940+0000 7f8e4a59c700 1 --2- 172.21.15.78:0/1689260476 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8e2c065ed0 0x7f8e2c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:34.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.940+0000 7f8e4a59c700 1 --2- 172.21.15.78:0/1689260476 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8e2c065ed0 0x7f8e2c068380 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c014040 tx=0x7f8e3c011000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:34.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:34.948+0000 7f8e3affd700 1 -- 172.21.15.78:0/1689260476 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f8e40050df0 con 0x7f8e4c0cede0 2023-10-11T09:48:35.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.116+0000 7f8e4b59e700 1 -- 172.21.15.78:0/1689260476 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8e4c0cb6a0 con 0x7f8e4c0cede0 2023-10-11T09:48:35.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.116+0000 7f8e3affd700 1 -- 172.21.15.78:0/1689260476 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8e40054410 con 0x7f8e4c0cede0 2023-10-11T09:48:35.127 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:48:35.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.124+0000 7f8e4b59e700 1 -- 172.21.15.78:0/1689260476 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8e2c065ed0 msgr2=0x7f8e2c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:35.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.124+0000 7f8e4b59e700 1 --2- 172.21.15.78:0/1689260476 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8e2c065ed0 0x7f8e2c068380 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c014040 tx=0x7f8e3c011000 comp rx=0 tx=0).stop 2023-10-11T09:48:35.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.124+0000 7f8e4b59e700 1 -- 172.21.15.78:0/1689260476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e4c0cede0 msgr2=0x7f8e4c146af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:35.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.124+0000 7f8e4b59e700 1 --2- 172.21.15.78:0/1689260476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e4c0cede0 0x7f8e4c146af0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000e9a0 tx=0x7f8e4000ecb0 comp rx=0 tx=0).stop 2023-10-11T09:48:35.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.124+0000 7f8e4b59e700 1 -- 172.21.15.78:0/1689260476 shutdown_connections 2023-10-11T09:48:35.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.124+0000 7f8e4b59e700 1 --2- 172.21.15.78:0/1689260476 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8e2c065ed0 0x7f8e2c068380 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:35.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.124+0000 7f8e4b59e700 1 --2- 172.21.15.78:0/1689260476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e4c15a6b0 0x7f8e4c0aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:35.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.124+0000 7f8e4b59e700 1 --2- 172.21.15.78:0/1689260476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e4c0cede0 0x7f8e4c146af0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:35.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.124+0000 7f8e4b59e700 1 --2- 172.21.15.78:0/1689260476 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8e4c0b3a60 0x7f8e4c145a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:35.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.124+0000 7f8e4b59e700 1 -- 172.21.15.78:0/1689260476 >> 172.21.15.78:0/1689260476 conn(0x7f8e4c06a060 msgr2=0x7f8e4c06a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:35.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.124+0000 7f8e4b59e700 1 -- 172.21.15.78:0/1689260476 shutdown_connections 2023-10-11T09:48:35.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.124+0000 7f8e4b59e700 1 -- 172.21.15.78:0/1689260476 wait complete. 2023-10-11T09:48:35.154 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:48:35.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.320+0000 7f10a3fff700 1 Processor -- start 2023-10-11T09:48:35.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.320+0000 7f10a3fff700 1 -- start start 2023-10-11T09:48:35.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.324+0000 7f10a3fff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10a40b3610 0x7f10a40d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:35.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.324+0000 7f10a3fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f10a40b3a90 0x7f10a40b85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:35.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.324+0000 7f10a3fff700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10a40b47f0 con 0x7f10a40b3240 2023-10-11T09:48:35.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.324+0000 7f10a3fff700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10a40b8130 con 0x7f10a40b3a90 2023-10-11T09:48:35.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.324+0000 7f10a3fff700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a40b3f10 con 0x7f10a40b3610 2023-10-11T09:48:35.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.324+0000 7f10a2ffd700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f10a40b3240 0x7f10a415d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:50066/0 (socket says 172.21.15.78:50066) 2023-10-11T09:48:35.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.324+0000 7f10a2ffd700 1 -- 172.21.15.78:0/608265770 learned_addr learned my addr 172.21.15.78:0/608265770 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:35.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.324+0000 7f10a37fe700 1 --2- 172.21.15.78:0/608265770 >> v2:172.21.15.78:3300/0 conn(0x7f10a40b3a90 0x7f10a40b85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:35.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.324+0000 7f10a27fc700 1 --2- 172.21.15.78:0/608265770 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10a40b3610 0x7f10a40d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:35.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.324+0000 7f10a1ffb700 1 -- 172.21.15.78:0/608265770 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1653135208 0 0) 0x7f1094001110 con 0x7f10a40b3240 2023-10-11T09:48:35.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.324+0000 7f10a1ffb700 1 -- 172.21.15.78:0/608265770 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f108c0035d0 con 0x7f10a40b3240 2023-10-11T09:48:35.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.324+0000 7f10a37fe700 1 -- 172.21.15.78:0/608265770 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10a40b3610 msgr2=0x7f10a40d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:35.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.324+0000 7f10a37fe700 1 --2- 172.21.15.78:0/608265770 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10a40b3610 0x7f10a40d29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:35.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.324+0000 7f10a37fe700 1 -- 172.21.15.78:0/608265770 >> v1:172.21.15.78:6789/0 conn(0x7f10a40b3240 legacy=0x7f10a415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:35.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.324+0000 7f10a37fe700 1 -- 172.21.15.78:0/608265770 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a40b41e0 con 0x7f10a40b3a90 2023-10-11T09:48:35.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.324+0000 7f10a37fe700 1 --2- 172.21.15.78:0/608265770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10a40b3a90 0x7f10a40b85c0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f109c009d90 tx=0x7f109c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f10dc325ba13c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:35.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.324+0000 7f10a1ffb700 1 -- 172.21.15.78:0/608265770 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c00e070 con 0x7f10a40b3a90 2023-10-11T09:48:35.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.324+0000 7f10a1ffb700 1 -- 172.21.15.78:0/608265770 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109c00bb20 con 0x7f10a40b3a90 2023-10-11T09:48:35.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.324+0000 7f10a1ffb700 1 -- 172.21.15.78:0/608265770 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c00be90 con 0x7f10a40b3a90 2023-10-11T09:48:35.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.328+0000 7f10a3fff700 1 -- 172.21.15.78:0/608265770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10a40b3a90 msgr2=0x7f10a40b85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:35.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.328+0000 7f10a3fff700 1 --2- 172.21.15.78:0/608265770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10a40b3a90 0x7f10a40b85c0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f109c009d90 tx=0x7f109c00b330 comp rx=0 tx=0).stop 2023-10-11T09:48:35.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.328+0000 7f10a3fff700 1 -- 172.21.15.78:0/608265770 shutdown_connections 2023-10-11T09:48:35.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.328+0000 7f10a3fff700 1 --2- 172.21.15.78:0/608265770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10a40b3a90 0x7f10a40b85c0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:35.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.328+0000 7f10a3fff700 1 --2- 172.21.15.78:0/608265770 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10a40b3610 0x7f10a40d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:35.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.328+0000 7f10a3fff700 1 -- 172.21.15.78:0/608265770 >> 172.21.15.78:0/608265770 conn(0x7f10a409c260 msgr2=0x7f10a40d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:35.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.328+0000 7f10a3fff700 1 -- 172.21.15.78:0/608265770 shutdown_connections 2023-10-11T09:48:35.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.328+0000 7f10a3fff700 1 -- 172.21.15.78:0/608265770 wait complete. 2023-10-11T09:48:35.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.328+0000 7f10a3fff700 1 Processor -- start 2023-10-11T09:48:35.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.328+0000 7f10a3fff700 1 -- start start 2023-10-11T09:48:35.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.332+0000 7f10a3fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10a40b3240 0x7f10a414f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:35.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.332+0000 7f10a3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10a40b3610 0x7f10a40d0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:35.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.332+0000 7f10a3fff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10a40b3a90 0x7f10a40d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:35.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.332+0000 7f10a3fff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a40c6c60 con 0x7f10a40b3610 2023-10-11T09:48:35.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.332+0000 7f10a3fff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a40c6f00 con 0x7f10a40b3240 2023-10-11T09:48:35.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.332+0000 7f10a3fff700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a40c71e0 con 0x7f10a40b3a90 2023-10-11T09:48:35.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.332+0000 7f10a27fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10a40b3610 0x7f10a40d0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:35.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.332+0000 7f10a27fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10a40b3610 0x7f10a40d0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44130/0 (socket says 172.21.15.78:44130) 2023-10-11T09:48:35.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.332+0000 7f10a27fc700 1 -- 172.21.15.78:0/3312677069 learned_addr learned my addr 172.21.15.78:0/3312677069 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:35.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.332+0000 7f10a37fe700 1 --2- 172.21.15.78:0/3312677069 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10a40b3a90 0x7f10a40d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:35.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.332+0000 7f10a2ffd700 1 --2- 172.21.15.78:0/3312677069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10a40b3240 0x7f10a414f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:35.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.332+0000 7f10a27fc700 1 -- 172.21.15.78:0/3312677069 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10a40b3a90 msgr2=0x7f10a40d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:35.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.332+0000 7f10a27fc700 1 --2- 172.21.15.78:0/3312677069 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10a40b3a90 0x7f10a40d10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:35.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.332+0000 7f10a27fc700 1 -- 172.21.15.78:0/3312677069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10a40b3240 msgr2=0x7f10a414f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:35.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.332+0000 7f10a27fc700 1 --2- 172.21.15.78:0/3312677069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10a40b3240 0x7f10a414f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:35.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.332+0000 7f10a27fc700 1 -- 172.21.15.78:0/3312677069 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f109c009a70 con 0x7f10a40b3610 2023-10-11T09:48:35.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.332+0000 7f10a2ffd700 1 --2- 172.21.15.78:0/3312677069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10a40b3240 0x7f10a414f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:48:35.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.332+0000 7f10a27fc700 1 --2- 172.21.15.78:0/3312677069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10a40b3610 0x7f10a40d0b60 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f1098009fd0 tx=0x7f109800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:35.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.332+0000 7f108bfff700 1 -- 172.21.15.78:0/3312677069 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10980095a0 con 0x7f10a40b3610 2023-10-11T09:48:35.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.332+0000 7f108bfff700 1 -- 172.21.15.78:0/3312677069 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1098013680 con 0x7f10a40b3610 2023-10-11T09:48:35.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.332+0000 7f10a3fff700 1 -- 172.21.15.78:0/3312677069 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10a40c74c0 con 0x7f10a40b3610 2023-10-11T09:48:35.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.332+0000 7f108bfff700 1 -- 172.21.15.78:0/3312677069 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1098013a30 con 0x7f10a40b3610 2023-10-11T09:48:35.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.332+0000 7f10a3fff700 1 -- 172.21.15.78:0/3312677069 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a414cbe0 con 0x7f10a40b3610 2023-10-11T09:48:35.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.332+0000 7f10a3fff700 1 -- 172.21.15.78:0/3312677069 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10a40b2bb0 con 0x7f10a40b3610 2023-10-11T09:48:35.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.336+0000 7f108bfff700 1 -- 172.21.15.78:0/3312677069 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f1098013b90 con 0x7f10a40b3610 2023-10-11T09:48:35.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.336+0000 7f108bfff700 1 --2- 172.21.15.78:0/3312677069 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1090065ed0 0x7f1090068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:35.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.336+0000 7f108bfff700 1 -- 172.21.15.78:0/3312677069 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f10980839d0 con 0x7f10a40b3610 2023-10-11T09:48:35.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.336+0000 7f10a2ffd700 1 --2- 172.21.15.78:0/3312677069 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1090065ed0 0x7f1090068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:35.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.340+0000 7f10a2ffd700 1 --2- 172.21.15.78:0/3312677069 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1090065ed0 0x7f1090068380 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1094007fd0 tx=0x7f1094009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:35.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.344+0000 7f108bfff700 1 -- 172.21.15.78:0/3312677069 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f1098050de0 con 0x7f10a40b3610 2023-10-11T09:48:35.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.520+0000 7f10a3fff700 1 -- 172.21.15.78:0/3312677069 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f10a408bf90 con 0x7f10a40b3610 2023-10-11T09:48:35.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.520+0000 7f108bfff700 1 -- 172.21.15.78:0/3312677069 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v86) v1 ==== 92+0+10 (secure 0 0 0) 0x7f10980170a0 con 0x7f10a40b3610 2023-10-11T09:48:35.530 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:48:35.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.524+0000 7f10a3fff700 1 -- 172.21.15.78:0/3312677069 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1090065ed0 msgr2=0x7f1090068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:35.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.524+0000 7f10a3fff700 1 --2- 172.21.15.78:0/3312677069 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1090065ed0 0x7f1090068380 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1094007fd0 tx=0x7f1094009040 comp rx=0 tx=0).stop 2023-10-11T09:48:35.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.524+0000 7f10a3fff700 1 -- 172.21.15.78:0/3312677069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10a40b3610 msgr2=0x7f10a40d0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:35.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.524+0000 7f10a3fff700 1 --2- 172.21.15.78:0/3312677069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10a40b3610 0x7f10a40d0b60 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f1098009fd0 tx=0x7f109800ebd0 comp rx=0 tx=0).stop 2023-10-11T09:48:35.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.524+0000 7f10a3fff700 1 -- 172.21.15.78:0/3312677069 shutdown_connections 2023-10-11T09:48:35.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.524+0000 7f10a3fff700 1 --2- 172.21.15.78:0/3312677069 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1090065ed0 0x7f1090068380 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:35.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.524+0000 7f10a3fff700 1 --2- 172.21.15.78:0/3312677069 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10a40b3a90 0x7f10a40d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:35.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.524+0000 7f10a3fff700 1 --2- 172.21.15.78:0/3312677069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10a40b3610 0x7f10a40d0b60 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:35.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.524+0000 7f10a3fff700 1 --2- 172.21.15.78:0/3312677069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10a40b3240 0x7f10a414f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:35.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.524+0000 7f10a3fff700 1 -- 172.21.15.78:0/3312677069 >> 172.21.15.78:0/3312677069 conn(0x7f10a409c260 msgr2=0x7f10a40d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:35.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.524+0000 7f10a3fff700 1 -- 172.21.15.78:0/3312677069 shutdown_connections 2023-10-11T09:48:35.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.524+0000 7f10a3fff700 1 -- 172.21.15.78:0/3312677069 wait complete. 2023-10-11T09:48:35.554 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T09:48:35.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.724+0000 7f791e1a8700 1 Processor -- start 2023-10-11T09:48:35.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.724+0000 7f791e1a8700 1 -- start start 2023-10-11T09:48:35.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.728+0000 7f791e1a8700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79180b3240 0x7f79180d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:35.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.728+0000 7f791e1a8700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f79180b3610 0x7f79180b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:35.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.728+0000 7f791e1a8700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79180b47f0 con 0x7f791815d040 2023-10-11T09:48:35.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.728+0000 7f791e1a8700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79180b3fd0 con 0x7f79180b3610 2023-10-11T09:48:35.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.728+0000 7f791e1a8700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79180b8130 con 0x7f79180b3240 2023-10-11T09:48:35.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.728+0000 7f791c9a5700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f79180b3610 0x7f79180b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:35.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.728+0000 7f791d9a7700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f791815d040 0x7f79180b85c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:50072/0 (socket says 172.21.15.78:50072) 2023-10-11T09:48:35.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.728+0000 7f791d9a7700 1 -- 172.21.15.78:0/1781243327 learned_addr learned my addr 172.21.15.78:0/1781243327 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:35.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.728+0000 7f791d1a6700 1 --2- 172.21.15.78:0/1781243327 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79180b3240 0x7f79180d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:35.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.728+0000 7f791c9a5700 1 -- 172.21.15.78:0/1781243327 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79180b3240 msgr2=0x7f79180d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:35.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.728+0000 7f791c9a5700 1 --2- 172.21.15.78:0/1781243327 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79180b3240 0x7f79180d29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:35.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.728+0000 7f791c9a5700 1 -- 172.21.15.78:0/1781243327 >> v1:172.21.15.78:6789/0 conn(0x7f791815d040 legacy=0x7f79180b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:35.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.728+0000 7f791c9a5700 1 -- 172.21.15.78:0/1781243327 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79180b41e0 con 0x7f79180b3610 2023-10-11T09:48:35.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.728+0000 7f791c9a5700 1 --2- 172.21.15.78:0/1781243327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79180b3610 0x7f79180b3a90 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f7908009850 tx=0x7f7908009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13920cbf64fa88a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:35.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.728+0000 7f790ffff700 1 -- 172.21.15.78:0/1781243327 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7908011830 con 0x7f79180b3610 2023-10-11T09:48:35.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.728+0000 7f790ffff700 1 -- 172.21.15.78:0/1781243327 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7908011990 con 0x7f79180b3610 2023-10-11T09:48:35.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.728+0000 7f790ffff700 1 -- 172.21.15.78:0/1781243327 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7908011d20 con 0x7f79180b3610 2023-10-11T09:48:35.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.728+0000 7f791e1a8700 1 -- 172.21.15.78:0/1781243327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79180b3610 msgr2=0x7f79180b3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:35.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.728+0000 7f791e1a8700 1 --2- 172.21.15.78:0/1781243327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79180b3610 0x7f79180b3a90 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f7908009850 tx=0x7f7908009b60 comp rx=0 tx=0).stop 2023-10-11T09:48:35.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.728+0000 7f791e1a8700 1 -- 172.21.15.78:0/1781243327 shutdown_connections 2023-10-11T09:48:35.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.728+0000 7f791e1a8700 1 --2- 172.21.15.78:0/1781243327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79180b3610 0x7f79180b3a90 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:35.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.728+0000 7f791e1a8700 1 --2- 172.21.15.78:0/1781243327 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79180b3240 0x7f79180d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:35.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.728+0000 7f791e1a8700 1 -- 172.21.15.78:0/1781243327 >> 172.21.15.78:0/1781243327 conn(0x7f791809c260 msgr2=0x7f79180d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:35.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.728+0000 7f791e1a8700 1 -- 172.21.15.78:0/1781243327 shutdown_connections 2023-10-11T09:48:35.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.728+0000 7f791e1a8700 1 -- 172.21.15.78:0/1781243327 wait complete. 2023-10-11T09:48:35.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.732+0000 7f791e1a8700 1 Processor -- start 2023-10-11T09:48:35.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.732+0000 7f791e1a8700 1 -- start start 2023-10-11T09:48:35.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.736+0000 7f791e1a8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79180b3240 0x7f79180c8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:35.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.736+0000 7f791e1a8700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79180b3610 0x7f79180c9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:35.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.736+0000 7f791e1a8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f791815d040 0x7f791809fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:35.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.736+0000 7f791e1a8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79180c78a0 con 0x7f791815d040 2023-10-11T09:48:35.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.736+0000 7f791e1a8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79180c7b80 con 0x7f79180b3240 2023-10-11T09:48:35.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.736+0000 7f791e1a8700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79180c7e60 con 0x7f79180b3610 2023-10-11T09:48:35.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.736+0000 7f791d9a7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f791815d040 0x7f791809fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:35.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.736+0000 7f791d9a7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f791815d040 0x7f791809fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44154/0 (socket says 172.21.15.78:44154) 2023-10-11T09:48:35.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.736+0000 7f791d9a7700 1 -- 172.21.15.78:0/1903420975 learned_addr learned my addr 172.21.15.78:0/1903420975 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:35.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.736+0000 7f791c9a5700 1 --2- 172.21.15.78:0/1903420975 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79180b3610 0x7f79180c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:35.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.736+0000 7f791d1a6700 1 --2- 172.21.15.78:0/1903420975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79180b3240 0x7f79180c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:35.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.736+0000 7f791d9a7700 1 -- 172.21.15.78:0/1903420975 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79180b3610 msgr2=0x7f79180c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:35.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.736+0000 7f791d9a7700 1 --2- 172.21.15.78:0/1903420975 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79180b3610 0x7f79180c9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:35.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.736+0000 7f791d9a7700 1 -- 172.21.15.78:0/1903420975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79180b3240 msgr2=0x7f79180c8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:35.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.736+0000 7f791d9a7700 1 --2- 172.21.15.78:0/1903420975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79180b3240 0x7f79180c8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:35.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.736+0000 7f791d9a7700 1 -- 172.21.15.78:0/1903420975 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79080094b0 con 0x7f791815d040 2023-10-11T09:48:35.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.736+0000 7f791d9a7700 1 --2- 172.21.15.78:0/1903420975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f791815d040 0x7f791809fe10 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f791000c4d0 tx=0x7f791000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:35.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.736+0000 7f790dffb700 1 -- 172.21.15.78:0/1903420975 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7910005d90 con 0x7f791815d040 2023-10-11T09:48:35.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.740+0000 7f790dffb700 1 -- 172.21.15.78:0/1903420975 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79100070d0 con 0x7f791815d040 2023-10-11T09:48:35.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.740+0000 7f791e1a8700 1 -- 172.21.15.78:0/1903420975 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79180c8140 con 0x7f791815d040 2023-10-11T09:48:35.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.740+0000 7f790dffb700 1 -- 172.21.15.78:0/1903420975 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7910007440 con 0x7f791815d040 2023-10-11T09:48:35.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.740+0000 7f791e1a8700 1 -- 172.21.15.78:0/1903420975 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79180c8e60 con 0x7f791815d040 2023-10-11T09:48:35.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.740+0000 7f791e1a8700 1 -- 172.21.15.78:0/1903420975 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7918065d40 con 0x7f791815d040 2023-10-11T09:48:35.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.740+0000 7f790dffb700 1 -- 172.21.15.78:0/1903420975 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f79100075a0 con 0x7f791815d040 2023-10-11T09:48:35.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.744+0000 7f790dffb700 1 --2- 172.21.15.78:0/1903420975 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f78fc065ed0 0x7f78fc068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:35.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.744+0000 7f790dffb700 1 -- 172.21.15.78:0/1903420975 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f7910082b30 con 0x7f791815d040 2023-10-11T09:48:35.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.744+0000 7f791d1a6700 1 --2- 172.21.15.78:0/1903420975 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f78fc065ed0 0x7f78fc068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:35.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.744+0000 7f791d1a6700 1 --2- 172.21.15.78:0/1903420975 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f78fc065ed0 0x7f78fc068380 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f7914003b50 tx=0x7f7914009830 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:35.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.748+0000 7f790dffb700 1 -- 172.21.15.78:0/1903420975 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f791004fff0 con 0x7f791815d040 2023-10-11T09:48:35.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.924+0000 7f791e1a8700 1 -- 172.21.15.78:0/1903420975 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f79180b47f0 con 0x7f791815d040 2023-10-11T09:48:35.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.928+0000 7f790dffb700 1 -- 172.21.15.78:0/1903420975 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7910017020 con 0x7f791815d040 2023-10-11T09:48:35.936 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:48:35.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.932+0000 7f791e1a8700 1 -- 172.21.15.78:0/1903420975 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f78fc065ed0 msgr2=0x7f78fc068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:35.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.932+0000 7f791e1a8700 1 --2- 172.21.15.78:0/1903420975 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f78fc065ed0 0x7f78fc068380 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f7914003b50 tx=0x7f7914009830 comp rx=0 tx=0).stop 2023-10-11T09:48:35.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.932+0000 7f791e1a8700 1 -- 172.21.15.78:0/1903420975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f791815d040 msgr2=0x7f791809fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:35.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.932+0000 7f791e1a8700 1 --2- 172.21.15.78:0/1903420975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f791815d040 0x7f791809fe10 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f791000c4d0 tx=0x7f791000c7e0 comp rx=0 tx=0).stop 2023-10-11T09:48:35.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.932+0000 7f791e1a8700 1 -- 172.21.15.78:0/1903420975 shutdown_connections 2023-10-11T09:48:35.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.932+0000 7f791e1a8700 1 --2- 172.21.15.78:0/1903420975 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f78fc065ed0 0x7f78fc068380 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:35.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.932+0000 7f791e1a8700 1 --2- 172.21.15.78:0/1903420975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f791815d040 0x7f791809fe10 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:35.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.932+0000 7f791e1a8700 1 --2- 172.21.15.78:0/1903420975 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79180b3610 0x7f79180c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:35.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.932+0000 7f791e1a8700 1 --2- 172.21.15.78:0/1903420975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79180b3240 0x7f79180c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:35.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.932+0000 7f791e1a8700 1 -- 172.21.15.78:0/1903420975 >> 172.21.15.78:0/1903420975 conn(0x7f791809c260 msgr2=0x7f791815fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:35.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.932+0000 7f791e1a8700 1 -- 172.21.15.78:0/1903420975 shutdown_connections 2023-10-11T09:48:35.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:35.932+0000 7f791e1a8700 1 -- 172.21.15.78:0/1903420975 wait complete. 2023-10-11T09:48:35.963 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T09:48:36.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.132+0000 7f2eae298700 1 Processor -- start 2023-10-11T09:48:36.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.132+0000 7f2eae298700 1 -- start start 2023-10-11T09:48:36.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.136+0000 7f2eae298700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2ea8146fe0 0x7f2ea806bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:36.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.136+0000 7f2eae298700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f2ea80b36f0 0x7f2ea81629e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:36.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.136+0000 7f2eae298700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea806ccb0 con 0x7f2ea8161d90 2023-10-11T09:48:36.145 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.136+0000 7f2eae298700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ea806c410 con 0x7f2ea80b36f0 2023-10-11T09:48:36.145 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.136+0000 7f2eae298700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ea806c6f0 con 0x7f2ea8146fe0 2023-10-11T09:48:36.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.136+0000 7f2eada97700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f2ea80b36f0 0x7f2ea81629e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:36.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.136+0000 7f2eada97700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f2ea80b36f0 0x7f2ea81629e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44166/0 (socket says 172.21.15.78:44166) 2023-10-11T09:48:36.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.136+0000 7f2eada97700 1 -- 172.21.15.78:0/2892793468 learned_addr learned my addr 172.21.15.78:0/2892793468 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:36.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.136+0000 7f2eaca95700 1 --1- 172.21.15.78:0/2892793468 >> v1:172.21.15.78:6789/0 conn(0x7f2ea8161d90 0x7f2ea80b2fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:50086/0 (socket says 172.21.15.78:50086) 2023-10-11T09:48:36.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.136+0000 7f2ead296700 1 --2- 172.21.15.78:0/2892793468 >> v2:172.21.15.133:3300/0 conn(0x7f2ea8146fe0 0x7f2ea806bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:36.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.136+0000 7f2eada97700 1 -- 172.21.15.78:0/2892793468 >> v2:172.21.15.133:3300/0 conn(0x7f2ea8146fe0 msgr2=0x7f2ea806bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:36.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.136+0000 7f2eada97700 1 --2- 172.21.15.78:0/2892793468 >> v2:172.21.15.133:3300/0 conn(0x7f2ea8146fe0 0x7f2ea806bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:36.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.136+0000 7f2eada97700 1 -- 172.21.15.78:0/2892793468 >> v1:172.21.15.78:6789/0 conn(0x7f2ea8161d90 legacy=0x7f2ea80b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:36.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.136+0000 7f2eada97700 1 -- 172.21.15.78:0/2892793468 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea806c9d0 con 0x7f2ea80b36f0 2023-10-11T09:48:36.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.136+0000 7f2e9ffff700 1 -- 172.21.15.78:0/2892793468 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3176038856 0 0) 0x7f2e98001110 con 0x7f2ea8161d90 2023-10-11T09:48:36.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.136+0000 7f2eada97700 1 --2- 172.21.15.78:0/2892793468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ea80b36f0 0x7f2ea81629e0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f2ea806f7f0 tx=0x7f2ea000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad4259910905d5d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:36.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.136+0000 7f2e9ffff700 1 -- 172.21.15.78:0/2892793468 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea000e070 con 0x7f2ea80b36f0 2023-10-11T09:48:36.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.136+0000 7f2e9ffff700 1 -- 172.21.15.78:0/2892793468 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea000b9c0 con 0x7f2ea80b36f0 2023-10-11T09:48:36.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.136+0000 7f2e9ffff700 1 -- 172.21.15.78:0/2892793468 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea000bd70 con 0x7f2ea80b36f0 2023-10-11T09:48:36.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.140+0000 7f2eae298700 1 -- 172.21.15.78:0/2892793468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ea80b36f0 msgr2=0x7f2ea81629e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:36.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.140+0000 7f2eae298700 1 --2- 172.21.15.78:0/2892793468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ea80b36f0 0x7f2ea81629e0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f2ea806f7f0 tx=0x7f2ea000b330 comp rx=0 tx=0).stop 2023-10-11T09:48:36.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.140+0000 7f2eae298700 1 -- 172.21.15.78:0/2892793468 shutdown_connections 2023-10-11T09:48:36.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.140+0000 7f2eae298700 1 --2- 172.21.15.78:0/2892793468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ea80b36f0 0x7f2ea81629e0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:36.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.140+0000 7f2eae298700 1 --2- 172.21.15.78:0/2892793468 >> v2:172.21.15.133:3300/0 conn(0x7f2ea8146fe0 0x7f2ea806bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:36.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.140+0000 7f2eae298700 1 -- 172.21.15.78:0/2892793468 >> 172.21.15.78:0/2892793468 conn(0x7f2ea806a060 msgr2=0x7f2ea806a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:36.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.140+0000 7f2eae298700 1 -- 172.21.15.78:0/2892793468 shutdown_connections 2023-10-11T09:48:36.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.140+0000 7f2eae298700 1 -- 172.21.15.78:0/2892793468 wait complete. 2023-10-11T09:48:36.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.144+0000 7f2eae298700 1 Processor -- start 2023-10-11T09:48:36.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.144+0000 7f2eae298700 1 -- start start 2023-10-11T09:48:36.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.144+0000 7f2eae298700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ea80b36f0 0x7f2ea80a8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:36.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.144+0000 7f2eae298700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ea8146fe0 0x7f2ea8146060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:36.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.144+0000 7f2eae298700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ea8161d90 0x7f2ea80abc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:36.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.144+0000 7f2eae298700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea80ab840 con 0x7f2ea8146fe0 2023-10-11T09:48:36.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.144+0000 7f2eae298700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea80a9790 con 0x7f2ea8161d90 2023-10-11T09:48:36.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.144+0000 7f2eae298700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea80a9a70 con 0x7f2ea80b36f0 2023-10-11T09:48:36.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.144+0000 7f2eaca95700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ea8146fe0 0x7f2ea8146060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:36.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.144+0000 7f2eaca95700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ea8146fe0 0x7f2ea8146060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44172/0 (socket says 172.21.15.78:44172) 2023-10-11T09:48:36.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.144+0000 7f2eaca95700 1 -- 172.21.15.78:0/306210314 learned_addr learned my addr 172.21.15.78:0/306210314 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:36.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.144+0000 7f2eada97700 1 --2- 172.21.15.78:0/306210314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ea8161d90 0x7f2ea80abc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:36.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.144+0000 7f2ead296700 1 --2- 172.21.15.78:0/306210314 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ea80b36f0 0x7f2ea80a8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:36.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.144+0000 7f2eaca95700 1 -- 172.21.15.78:0/306210314 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ea80b36f0 msgr2=0x7f2ea80a8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:36.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.144+0000 7f2eaca95700 1 --2- 172.21.15.78:0/306210314 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ea80b36f0 0x7f2ea80a8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:36.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.144+0000 7f2eaca95700 1 -- 172.21.15.78:0/306210314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ea8161d90 msgr2=0x7f2ea80abc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:36.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.144+0000 7f2eaca95700 1 --2- 172.21.15.78:0/306210314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ea8161d90 0x7f2ea80abc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:36.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.144+0000 7f2eaca95700 1 -- 172.21.15.78:0/306210314 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea0009b30 con 0x7f2ea8146fe0 2023-10-11T09:48:36.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.144+0000 7f2eada97700 1 --2- 172.21.15.78:0/306210314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ea8161d90 0x7f2ea80abc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:48:36.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.144+0000 7f2ead296700 1 --2- 172.21.15.78:0/306210314 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ea80b36f0 0x7f2ea80a8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:48:36.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.144+0000 7f2eaca95700 1 --2- 172.21.15.78:0/306210314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ea8146fe0 0x7f2ea8146060 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f2e9800c4e0 tx=0x7f2e9800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:36.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.144+0000 7f2e9dffb700 1 -- 172.21.15.78:0/306210314 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e98004ca0 con 0x7f2ea8146fe0 2023-10-11T09:48:36.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.144+0000 7f2e9dffb700 1 -- 172.21.15.78:0/306210314 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e98004e20 con 0x7f2ea8146fe0 2023-10-11T09:48:36.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.144+0000 7f2eae298700 1 -- 172.21.15.78:0/306210314 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ea80a9d50 con 0x7f2ea8146fe0 2023-10-11T09:48:36.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.144+0000 7f2e9dffb700 1 -- 172.21.15.78:0/306210314 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e98007310 con 0x7f2ea8146fe0 2023-10-11T09:48:36.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.148+0000 7f2eae298700 1 -- 172.21.15.78:0/306210314 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ea80aa310 con 0x7f2ea8146fe0 2023-10-11T09:48:36.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.148+0000 7f2eae298700 1 -- 172.21.15.78:0/306210314 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ea80ab9b0 con 0x7f2ea8146fe0 2023-10-11T09:48:36.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.148+0000 7f2e9dffb700 1 -- 172.21.15.78:0/306210314 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f2e98007470 con 0x7f2ea8146fe0 2023-10-11T09:48:36.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.152+0000 7f2e9dffb700 1 --2- 172.21.15.78:0/306210314 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2e8c065f00 0x7f2e8c0683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:36.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.152+0000 7f2e9dffb700 1 -- 172.21.15.78:0/306210314 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f2e980834c0 con 0x7f2ea8146fe0 2023-10-11T09:48:36.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.152+0000 7f2ead296700 1 --2- 172.21.15.78:0/306210314 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2e8c065f00 0x7f2e8c0683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:36.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.152+0000 7f2ead296700 1 --2- 172.21.15.78:0/306210314 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2e8c065f00 0x7f2e8c0683b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2ea80cde90 tx=0x7f2ea4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:36.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.160+0000 7f2e9dffb700 1 -- 172.21.15.78:0/306210314 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f2ea80ab9b0 con 0x7f2ea8146fe0 2023-10-11T09:48:36.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.328+0000 7f2eae298700 1 -- 172.21.15.78:0/306210314 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2ea80ab980 con 0x7f2ea8146fe0 2023-10-11T09:48:36.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.332+0000 7f2e9dffb700 1 -- 172.21.15.78:0/306210314 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2e98050960 con 0x7f2ea8146fe0 2023-10-11T09:48:36.340 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2023-10-11T09:48:36.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.336+0000 7f2eae298700 1 -- 172.21.15.78:0/306210314 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2e8c065f00 msgr2=0x7f2e8c0683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:36.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.336+0000 7f2eae298700 1 --2- 172.21.15.78:0/306210314 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2e8c065f00 0x7f2e8c0683b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2ea80cde90 tx=0x7f2ea4008040 comp rx=0 tx=0).stop 2023-10-11T09:48:36.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.336+0000 7f2eae298700 1 -- 172.21.15.78:0/306210314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ea8146fe0 msgr2=0x7f2ea8146060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:36.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.336+0000 7f2eae298700 1 --2- 172.21.15.78:0/306210314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ea8146fe0 0x7f2ea8146060 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f2e9800c4e0 tx=0x7f2e9800c8a0 comp rx=0 tx=0).stop 2023-10-11T09:48:36.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.336+0000 7f2eae298700 1 -- 172.21.15.78:0/306210314 shutdown_connections 2023-10-11T09:48:36.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.336+0000 7f2eae298700 1 --2- 172.21.15.78:0/306210314 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2e8c065f00 0x7f2e8c0683b0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:36.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.336+0000 7f2eae298700 1 --2- 172.21.15.78:0/306210314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ea8161d90 0x7f2ea80abc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:36.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.336+0000 7f2eae298700 1 --2- 172.21.15.78:0/306210314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ea8146fe0 0x7f2ea8146060 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:36.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.336+0000 7f2eae298700 1 --2- 172.21.15.78:0/306210314 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ea80b36f0 0x7f2ea80a8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:36.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.336+0000 7f2eae298700 1 -- 172.21.15.78:0/306210314 >> 172.21.15.78:0/306210314 conn(0x7f2ea806a060 msgr2=0x7f2ea80ae660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:36.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.336+0000 7f2eae298700 1 -- 172.21.15.78:0/306210314 shutdown_connections 2023-10-11T09:48:36.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.336+0000 7f2eae298700 1 -- 172.21.15.78:0/306210314 wait complete. 2023-10-11T09:48:36.367 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-10-11T09:48:36.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.536+0000 7f9c317a7700 1 Processor -- start 2023-10-11T09:48:36.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.540+0000 7f9c317a7700 1 -- start start 2023-10-11T09:48:36.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.540+0000 7f9c317a7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c2c0b3240 0x7f9c2c0d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:36.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.540+0000 7f9c317a7700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f9c2c0b3610 0x7f9c2c0b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:36.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.540+0000 7f9c317a7700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c2c0b47f0 con 0x7f9c2c15d040 2023-10-11T09:48:36.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.540+0000 7f9c317a7700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c2c0b3fd0 con 0x7f9c2c0b3610 2023-10-11T09:48:36.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.540+0000 7f9c317a7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c0b8130 con 0x7f9c2c0b3240 2023-10-11T09:48:36.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.540+0000 7f9c2a7fc700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f9c2c0b3610 0x7f9c2c0b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:36.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.540+0000 7f9c2a7fc700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f9c2c0b3610 0x7f9c2c0b3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44178/0 (socket says 172.21.15.78:44178) 2023-10-11T09:48:36.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.540+0000 7f9c2a7fc700 1 -- 172.21.15.78:0/1655480277 learned_addr learned my addr 172.21.15.78:0/1655480277 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:36.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.540+0000 7f9c2affd700 1 --2- 172.21.15.78:0/1655480277 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c2c0b3240 0x7f9c2c0d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:36.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.544+0000 7f9c2a7fc700 1 -- 172.21.15.78:0/1655480277 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c2c0b3240 msgr2=0x7f9c2c0d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:36.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.544+0000 7f9c2a7fc700 1 --2- 172.21.15.78:0/1655480277 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c2c0b3240 0x7f9c2c0d29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:36.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.544+0000 7f9c2a7fc700 1 -- 172.21.15.78:0/1655480277 >> v1:172.21.15.133:6789/0 conn(0x7f9c2c15d040 legacy=0x7f9c2c0b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:36.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.544+0000 7f9c2a7fc700 1 -- 172.21.15.78:0/1655480277 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c2c0b41e0 con 0x7f9c2c0b3610 2023-10-11T09:48:36.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.544+0000 7f9c2a7fc700 1 --2- 172.21.15.78:0/1655480277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c2c0b3610 0x7f9c2c0b3a90 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f9c20009910 tx=0x7f9c20009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d406bc32bc4f324 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:36.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.544+0000 7f9c29ffb700 1 -- 172.21.15.78:0/1655480277 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c20011830 con 0x7f9c2c0b3610 2023-10-11T09:48:36.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.544+0000 7f9c29ffb700 1 -- 172.21.15.78:0/1655480277 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c20011990 con 0x7f9c2c0b3610 2023-10-11T09:48:36.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.544+0000 7f9c29ffb700 1 -- 172.21.15.78:0/1655480277 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c20011d40 con 0x7f9c2c0b3610 2023-10-11T09:48:36.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.544+0000 7f9c317a7700 1 -- 172.21.15.78:0/1655480277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c2c0b3610 msgr2=0x7f9c2c0b3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:36.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.544+0000 7f9c317a7700 1 --2- 172.21.15.78:0/1655480277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c2c0b3610 0x7f9c2c0b3a90 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f9c20009910 tx=0x7f9c20009c20 comp rx=0 tx=0).stop 2023-10-11T09:48:36.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.544+0000 7f9c317a7700 1 -- 172.21.15.78:0/1655480277 shutdown_connections 2023-10-11T09:48:36.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.544+0000 7f9c317a7700 1 --2- 172.21.15.78:0/1655480277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c2c0b3610 0x7f9c2c0b3a90 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:36.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.544+0000 7f9c317a7700 1 --2- 172.21.15.78:0/1655480277 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c2c0b3240 0x7f9c2c0d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:36.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.544+0000 7f9c317a7700 1 -- 172.21.15.78:0/1655480277 >> 172.21.15.78:0/1655480277 conn(0x7f9c2c09c260 msgr2=0x7f9c2c0d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:36.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.544+0000 7f9c317a7700 1 -- 172.21.15.78:0/1655480277 shutdown_connections 2023-10-11T09:48:36.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.544+0000 7f9c317a7700 1 -- 172.21.15.78:0/1655480277 wait complete. 2023-10-11T09:48:36.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.548+0000 7f9c317a7700 1 Processor -- start 2023-10-11T09:48:36.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.548+0000 7f9c317a7700 1 -- start start 2023-10-11T09:48:36.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.552+0000 7f9c317a7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c2c0b3240 0x7f9c2c0a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:36.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.552+0000 7f9c317a7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c2c0b3610 0x7f9c2c1f5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:36.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.552+0000 7f9c317a7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c2c15d040 0x7f9c2c1f27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:36.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.552+0000 7f9c317a7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c0a0d70 con 0x7f9c2c0b3240 2023-10-11T09:48:36.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.552+0000 7f9c317a7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c1f5d70 con 0x7f9c2c0b3610 2023-10-11T09:48:36.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.552+0000 7f9c317a7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c1f6050 con 0x7f9c2c15d040 2023-10-11T09:48:36.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.552+0000 7f9c2affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c2c0b3240 0x7f9c2c0a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:36.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.552+0000 7f9c2affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c2c0b3240 0x7f9c2c0a0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44184/0 (socket says 172.21.15.78:44184) 2023-10-11T09:48:36.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.552+0000 7f9c2affd700 1 -- 172.21.15.78:0/3814175996 learned_addr learned my addr 172.21.15.78:0/3814175996 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:36.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.552+0000 7f9c2a7fc700 1 --2- 172.21.15.78:0/3814175996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c2c0b3610 0x7f9c2c1f5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:36.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.552+0000 7f9c2b7fe700 1 --2- 172.21.15.78:0/3814175996 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c2c15d040 0x7f9c2c1f27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:36.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.552+0000 7f9c2affd700 1 -- 172.21.15.78:0/3814175996 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c2c15d040 msgr2=0x7f9c2c1f27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:36.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.552+0000 7f9c2affd700 1 --2- 172.21.15.78:0/3814175996 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c2c15d040 0x7f9c2c1f27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:36.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.552+0000 7f9c2affd700 1 -- 172.21.15.78:0/3814175996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c2c0b3610 msgr2=0x7f9c2c1f5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:36.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.552+0000 7f9c2affd700 1 --2- 172.21.15.78:0/3814175996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c2c0b3610 0x7f9c2c1f5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:36.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.552+0000 7f9c2affd700 1 -- 172.21.15.78:0/3814175996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c20009570 con 0x7f9c2c0b3240 2023-10-11T09:48:36.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.552+0000 7f9c2a7fc700 1 --2- 172.21.15.78:0/3814175996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c2c0b3610 0x7f9c2c1f5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:48:36.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.552+0000 7f9c2affd700 1 --2- 172.21.15.78:0/3814175996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c2c0b3240 0x7f9c2c0a0830 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00e870 tx=0x7f9c1c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:36.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.552+0000 7f9c137fe700 1 -- 172.21.15.78:0/3814175996 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c0095c0 con 0x7f9c2c0b3240 2023-10-11T09:48:36.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.552+0000 7f9c137fe700 1 -- 172.21.15.78:0/3814175996 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c1c00ee70 con 0x7f9c2c0b3240 2023-10-11T09:48:36.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.552+0000 7f9c317a7700 1 -- 172.21.15.78:0/3814175996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c2c1f6330 con 0x7f9c2c0b3240 2023-10-11T09:48:36.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.552+0000 7f9c137fe700 1 -- 172.21.15.78:0/3814175996 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c00ca70 con 0x7f9c2c0b3240 2023-10-11T09:48:36.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.552+0000 7f9c317a7700 1 -- 172.21.15.78:0/3814175996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c2c1f2290 con 0x7f9c2c0b3240 2023-10-11T09:48:36.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.556+0000 7f9c317a7700 1 -- 172.21.15.78:0/3814175996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c2c065d40 con 0x7f9c2c0b3240 2023-10-11T09:48:36.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.556+0000 7f9c137fe700 1 -- 172.21.15.78:0/3814175996 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f9c1c00cbd0 con 0x7f9c2c0b3240 2023-10-11T09:48:36.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.556+0000 7f9c137fe700 1 --2- 172.21.15.78:0/3814175996 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9c18065ed0 0x7f9c18068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:36.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.556+0000 7f9c137fe700 1 -- 172.21.15.78:0/3814175996 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f9c1c083940 con 0x7f9c2c0b3240 2023-10-11T09:48:36.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.556+0000 7f9c2a7fc700 1 --2- 172.21.15.78:0/3814175996 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9c18065ed0 0x7f9c18068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:36.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.560+0000 7f9c2a7fc700 1 --2- 172.21.15.78:0/3814175996 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9c18065ed0 0x7f9c18068380 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c0b3a40 tx=0x7f9c20008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:36.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.560+0000 7f9c137fe700 1 -- 172.21.15.78:0/3814175996 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f9c1c050e80 con 0x7f9c2c0b3240 2023-10-11T09:48:36.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.736+0000 7f9c317a7700 1 -- 172.21.15.78:0/3814175996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7f9c2c0c8380 con 0x7f9c2c0b3240 2023-10-11T09:48:36.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.736+0000 7f9c137fe700 1 -- 172.21.15.78:0/3814175996 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v86) v1 ==== 106+0+11 (secure 0 0 0) 0x7f9c1c0544a0 con 0x7f9c2c0b3240 2023-10-11T09:48:36.745 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:48:36.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.740+0000 7f9c317a7700 1 -- 172.21.15.78:0/3814175996 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9c18065ed0 msgr2=0x7f9c18068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:36.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.740+0000 7f9c317a7700 1 --2- 172.21.15.78:0/3814175996 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9c18065ed0 0x7f9c18068380 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c0b3a40 tx=0x7f9c20008030 comp rx=0 tx=0).stop 2023-10-11T09:48:36.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.740+0000 7f9c317a7700 1 -- 172.21.15.78:0/3814175996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c2c0b3240 msgr2=0x7f9c2c0a0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:36.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.740+0000 7f9c317a7700 1 --2- 172.21.15.78:0/3814175996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c2c0b3240 0x7f9c2c0a0830 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00e870 tx=0x7f9c1c00ec30 comp rx=0 tx=0).stop 2023-10-11T09:48:36.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.740+0000 7f9c317a7700 1 -- 172.21.15.78:0/3814175996 shutdown_connections 2023-10-11T09:48:36.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.740+0000 7f9c317a7700 1 --2- 172.21.15.78:0/3814175996 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9c18065ed0 0x7f9c18068380 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:36.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.740+0000 7f9c317a7700 1 --2- 172.21.15.78:0/3814175996 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c2c15d040 0x7f9c2c1f27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:36.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.740+0000 7f9c317a7700 1 --2- 172.21.15.78:0/3814175996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c2c0b3610 0x7f9c2c1f5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:36.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.740+0000 7f9c317a7700 1 --2- 172.21.15.78:0/3814175996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c2c0b3240 0x7f9c2c0a0830 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:36.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.740+0000 7f9c317a7700 1 -- 172.21.15.78:0/3814175996 >> 172.21.15.78:0/3814175996 conn(0x7f9c2c09c260 msgr2=0x7f9c2c15fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:36.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.740+0000 7f9c317a7700 1 -- 172.21.15.78:0/3814175996 shutdown_connections 2023-10-11T09:48:36.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.740+0000 7f9c317a7700 1 -- 172.21.15.78:0/3814175996 wait complete. 2023-10-11T09:48:36.768 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T09:48:36.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.936+0000 7f5c2cccf700 1 Processor -- start 2023-10-11T09:48:36.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.936+0000 7f5c2cccf700 1 -- start start 2023-10-11T09:48:36.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.936+0000 7f5c2cccf700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5c28154e00 0x7f5c2815b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:36.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.936+0000 7f5c2cccf700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c28071040 0x7f5c280cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:36.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.936+0000 7f5c2cccf700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c280720f0 con 0x7f5c280cba00 2023-10-11T09:48:36.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.936+0000 7f5c2cccf700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c2815bb90 con 0x7f5c28154e00 2023-10-11T09:48:36.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.936+0000 7f5c2cccf700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c280714e0 con 0x7f5c28071040 2023-10-11T09:48:36.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.940+0000 7f5c26d9d700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f5c280cba00 0x7f5c2815e1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:50092/0 (socket says 172.21.15.78:50092) 2023-10-11T09:48:36.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.940+0000 7f5c26d9d700 1 -- 172.21.15.78:0/2953946024 learned_addr learned my addr 172.21.15.78:0/2953946024 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:36.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.940+0000 7f5c1ffff700 1 --2- 172.21.15.78:0/2953946024 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c28071040 0x7f5c280cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:36.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.940+0000 7f5c2659c700 1 --2- 172.21.15.78:0/2953946024 >> v2:172.21.15.133:3300/0 conn(0x7f5c28154e00 0x7f5c2815b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:36.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.940+0000 7f5c25d9b700 1 -- 172.21.15.78:0/2953946024 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2067377951 0 0) 0x7f5c280720f0 con 0x7f5c280cba00 2023-10-11T09:48:36.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.940+0000 7f5c25d9b700 1 -- 172.21.15.78:0/2953946024 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c140035d0 con 0x7f5c280cba00 2023-10-11T09:48:36.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.940+0000 7f5c1ffff700 1 -- 172.21.15.78:0/2953946024 >> v2:172.21.15.133:3300/0 conn(0x7f5c28154e00 msgr2=0x7f5c2815b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:36.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.940+0000 7f5c1ffff700 1 --2- 172.21.15.78:0/2953946024 >> v2:172.21.15.133:3300/0 conn(0x7f5c28154e00 0x7f5c2815b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:36.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.940+0000 7f5c1ffff700 1 -- 172.21.15.78:0/2953946024 >> v1:172.21.15.78:6789/0 conn(0x7f5c280cba00 legacy=0x7f5c2815e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:36.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.940+0000 7f5c1ffff700 1 -- 172.21.15.78:0/2953946024 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c280717c0 con 0x7f5c28071040 2023-10-11T09:48:36.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.940+0000 7f5c1ffff700 1 --2- 172.21.15.78:0/2953946024 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c28071040 0x7f5c280cb4c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5c10009850 tx=0x7f5c10009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efab9e6f19b26110 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:36.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.940+0000 7f5c25d9b700 1 -- 172.21.15.78:0/2953946024 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c10011830 con 0x7f5c28071040 2023-10-11T09:48:36.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.940+0000 7f5c25d9b700 1 -- 172.21.15.78:0/2953946024 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c10011990 con 0x7f5c28071040 2023-10-11T09:48:36.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.940+0000 7f5c25d9b700 1 -- 172.21.15.78:0/2953946024 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c10011d00 con 0x7f5c28071040 2023-10-11T09:48:36.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.940+0000 7f5c2cccf700 1 -- 172.21.15.78:0/2953946024 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c28071040 msgr2=0x7f5c280cb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:36.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.940+0000 7f5c2cccf700 1 --2- 172.21.15.78:0/2953946024 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c28071040 0x7f5c280cb4c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5c10009850 tx=0x7f5c10009b60 comp rx=0 tx=0).stop 2023-10-11T09:48:36.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.940+0000 7f5c2cccf700 1 -- 172.21.15.78:0/2953946024 shutdown_connections 2023-10-11T09:48:36.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.940+0000 7f5c2cccf700 1 --2- 172.21.15.78:0/2953946024 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c28071040 0x7f5c280cb4c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:36.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.940+0000 7f5c2cccf700 1 --2- 172.21.15.78:0/2953946024 >> v2:172.21.15.133:3300/0 conn(0x7f5c28154e00 0x7f5c2815b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:36.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.940+0000 7f5c2cccf700 1 -- 172.21.15.78:0/2953946024 >> 172.21.15.78:0/2953946024 conn(0x7f5c2806aca0 msgr2=0x7f5c280b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:36.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.940+0000 7f5c2cccf700 1 -- 172.21.15.78:0/2953946024 shutdown_connections 2023-10-11T09:48:36.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.940+0000 7f5c2cccf700 1 -- 172.21.15.78:0/2953946024 wait complete. 2023-10-11T09:48:36.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.944+0000 7f5c2cccf700 1 Processor -- start 2023-10-11T09:48:36.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.944+0000 7f5c2cccf700 1 -- start start 2023-10-11T09:48:36.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.948+0000 7f5c2cccf700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c28071040 0x7f5c2813feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:36.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.948+0000 7f5c2cccf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c280cba00 0x7f5c280cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:36.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.948+0000 7f5c2cccf700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c28154e00 0x7f5c280b0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:36.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.948+0000 7f5c2cccf700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c280b0ab0 con 0x7f5c280cba00 2023-10-11T09:48:36.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.948+0000 7f5c2cccf700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c280ccf10 con 0x7f5c28154e00 2023-10-11T09:48:36.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.948+0000 7f5c2cccf700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c280b0cc0 con 0x7f5c28071040 2023-10-11T09:48:36.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.948+0000 7f5c1ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c280cba00 0x7f5c280cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:36.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.948+0000 7f5c1ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c280cba00 0x7f5c280cd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44186/0 (socket says 172.21.15.78:44186) 2023-10-11T09:48:36.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.948+0000 7f5c1ffff700 1 -- 172.21.15.78:0/3506374466 learned_addr learned my addr 172.21.15.78:0/3506374466 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:36.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.948+0000 7f5c2659c700 1 --2- 172.21.15.78:0/3506374466 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c28071040 0x7f5c2813feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:36.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.948+0000 7f5c26d9d700 1 --2- 172.21.15.78:0/3506374466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c28154e00 0x7f5c280b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:36.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.948+0000 7f5c1ffff700 1 -- 172.21.15.78:0/3506374466 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c28071040 msgr2=0x7f5c2813feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:36.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.948+0000 7f5c1ffff700 1 --2- 172.21.15.78:0/3506374466 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c28071040 0x7f5c2813feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:36.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.948+0000 7f5c1ffff700 1 -- 172.21.15.78:0/3506374466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c28154e00 msgr2=0x7f5c280b0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:36.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.948+0000 7f5c1ffff700 1 --2- 172.21.15.78:0/3506374466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c28154e00 0x7f5c280b0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:36.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.948+0000 7f5c1ffff700 1 -- 172.21.15.78:0/3506374466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c100094b0 con 0x7f5c280cba00 2023-10-11T09:48:36.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.948+0000 7f5c1ffff700 1 --2- 172.21.15.78:0/3506374466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c280cba00 0x7f5c280cd3a0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f5c10000c00 tx=0x7f5c10004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:36.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.948+0000 7f5c1effd700 1 -- 172.21.15.78:0/3506374466 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c10004590 con 0x7f5c280cba00 2023-10-11T09:48:36.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.948+0000 7f5c1effd700 1 -- 172.21.15.78:0/3506374466 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1001f070 con 0x7f5c280cba00 2023-10-11T09:48:36.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.948+0000 7f5c1effd700 1 -- 172.21.15.78:0/3506374466 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c10003900 con 0x7f5c280cba00 2023-10-11T09:48:36.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.952+0000 7f5c2cccf700 1 -- 172.21.15.78:0/3506374466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c280b0fa0 con 0x7f5c280cba00 2023-10-11T09:48:36.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.952+0000 7f5c2cccf700 1 -- 172.21.15.78:0/3506374466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c280ccba0 con 0x7f5c280cba00 2023-10-11T09:48:36.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.952+0000 7f5c2cccf700 1 -- 172.21.15.78:0/3506374466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c280cd080 con 0x7f5c280cba00 2023-10-11T09:48:36.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.956+0000 7f5c1effd700 1 -- 172.21.15.78:0/3506374466 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f5c10003a60 con 0x7f5c280cba00 2023-10-11T09:48:36.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.956+0000 7f5c1effd700 1 --2- 172.21.15.78:0/3506374466 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5c08065ed0 0x7f5c08068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:36.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.956+0000 7f5c1effd700 1 -- 172.21.15.78:0/3506374466 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f5c10021070 con 0x7f5c280cba00 2023-10-11T09:48:36.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.956+0000 7f5c2659c700 1 --2- 172.21.15.78:0/3506374466 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5c08065ed0 0x7f5c08068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:36.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.956+0000 7f5c2659c700 1 --2- 172.21.15.78:0/3506374466 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5c08065ed0 0x7f5c08068380 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5c281583b0 tx=0x7f5c18009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:36.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:36.960+0000 7f5c1effd700 1 -- 172.21.15.78:0/3506374466 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f5c10052640 con 0x7f5c280cba00 2023-10-11T09:48:37.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:37.144+0000 7f5c2cccf700 1 -- 172.21.15.78:0/3506374466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5c2804e530 con 0x7f5c280cba00 2023-10-11T09:48:37.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:37.148+0000 7f5c1effd700 1 -- 172.21.15.78:0/3506374466 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v86) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5c10055c60 con 0x7f5c280cba00 2023-10-11T09:48:37.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:37.148+0000 7f5c2cccf700 1 -- 172.21.15.78:0/3506374466 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5c08065ed0 msgr2=0x7f5c08068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:37.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:37.148+0000 7f5c2cccf700 1 --2- 172.21.15.78:0/3506374466 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5c08065ed0 0x7f5c08068380 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5c281583b0 tx=0x7f5c18009240 comp rx=0 tx=0).stop 2023-10-11T09:48:37.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:37.148+0000 7f5c2cccf700 1 -- 172.21.15.78:0/3506374466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c280cba00 msgr2=0x7f5c280cd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:37.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:37.148+0000 7f5c2cccf700 1 --2- 172.21.15.78:0/3506374466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c280cba00 0x7f5c280cd3a0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f5c10000c00 tx=0x7f5c10004770 comp rx=0 tx=0).stop 2023-10-11T09:48:37.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:37.148+0000 7f5c2cccf700 1 -- 172.21.15.78:0/3506374466 shutdown_connections 2023-10-11T09:48:37.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:37.148+0000 7f5c2cccf700 1 --2- 172.21.15.78:0/3506374466 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5c08065ed0 0x7f5c08068380 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:37.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:37.148+0000 7f5c2cccf700 1 --2- 172.21.15.78:0/3506374466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c28154e00 0x7f5c280b0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:37.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:37.148+0000 7f5c2cccf700 1 --2- 172.21.15.78:0/3506374466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c280cba00 0x7f5c280cd3a0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:37.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:37.148+0000 7f5c2cccf700 1 --2- 172.21.15.78:0/3506374466 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c28071040 0x7f5c2813feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:37.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:37.148+0000 7f5c2cccf700 1 -- 172.21.15.78:0/3506374466 >> 172.21.15.78:0/3506374466 conn(0x7f5c2806aca0 msgr2=0x7f5c2814f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:37.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:37.148+0000 7f5c2cccf700 1 -- 172.21.15.78:0/3506374466 shutdown_connections 2023-10-11T09:48:37.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:37.148+0000 7f5c2cccf700 1 -- 172.21.15.78:0/3506374466 wait complete. 2023-10-11T09:48:37.162 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2023-10-11T09:48:37.180 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi078.front.sepia.ceph.com... 2023-10-11T09:48:37.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:48:37.181 DEBUG:teuthology.orchestra.run.smithi078:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-11T09:48:37.212 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:ceph-fuse[53166]: fuse finished with error 0 and tester_r 0 2023-10-11T09:48:37.217 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:48:43.318 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2023-10-11T09:48:43.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:48:43.320 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-11T09:48:43.341 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:48:43.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.544+0000 7fde86d0e700 1 Processor -- start 2023-10-11T09:48:43.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.544+0000 7fde86d0e700 1 -- start start 2023-10-11T09:48:43.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.548+0000 7fde86d0e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fde800cda10 0x7fde800caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:43.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.548+0000 7fde86d0e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde80165390 con 0x7fde800cdde0 2023-10-11T09:48:43.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.548+0000 7fde86d0e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde800ca440 con 0x7fde80164d20 2023-10-11T09:48:43.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.548+0000 7fde86d0e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde801651a0 con 0x7fde800cda10 2023-10-11T09:48:43.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.548+0000 7fde8550b700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fde800cdde0 0x7fde80164610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:50108/0 (socket says 172.21.15.78:50108) 2023-10-11T09:48:43.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.548+0000 7fde8550b700 1 -- 172.21.15.78:0/2689603734 learned_addr learned my addr 172.21.15.78:0/2689603734 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:43.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.548+0000 7fde85d0c700 1 --2- 172.21.15.78:0/2689603734 >> v2:172.21.15.133:3300/0 conn(0x7fde800cda10 0x7fde800caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:43.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.548+0000 7fde84d0a700 1 -- 172.21.15.78:0/2689603734 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2290090370 0 0) 0x7fde80165390 con 0x7fde800cdde0 2023-10-11T09:48:43.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.548+0000 7fde84d0a700 1 -- 172.21.15.78:0/2689603734 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde6c0035d0 con 0x7fde800cdde0 2023-10-11T09:48:43.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.548+0000 7fde84d0a700 1 -- 172.21.15.78:0/2689603734 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3412259827 0 0) 0x7fde800ca440 con 0x7fde80164d20 2023-10-11T09:48:43.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.548+0000 7fde84d0a700 1 -- 172.21.15.78:0/2689603734 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde80165390 con 0x7fde80164d20 2023-10-11T09:48:43.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.548+0000 7fde84d0a700 1 -- 172.21.15.78:0/2689603734 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1124995622 0 0) 0x7fde6c0035d0 con 0x7fde800cdde0 2023-10-11T09:48:43.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.548+0000 7fde84d0a700 1 -- 172.21.15.78:0/2689603734 >> v2:172.21.15.133:3300/0 conn(0x7fde800cda10 msgr2=0x7fde800caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:43.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.548+0000 7fde84d0a700 1 --2- 172.21.15.78:0/2689603734 >> v2:172.21.15.133:3300/0 conn(0x7fde800cda10 0x7fde800caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:43.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.548+0000 7fde84d0a700 1 -- 172.21.15.78:0/2689603734 >> v1:172.21.15.133:6789/0 conn(0x7fde80164d20 legacy=0x7fde8006bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:43.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.548+0000 7fde84d0a700 1 -- 172.21.15.78:0/2689603734 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde800ca720 con 0x7fde800cdde0 2023-10-11T09:48:43.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.548+0000 7fde84d0a700 1 -- 172.21.15.78:0/2689603734 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7fde70001670 con 0x7fde800cdde0 2023-10-11T09:48:43.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.548+0000 7fde84d0a700 1 -- 172.21.15.78:0/2689603734 >> v1:172.21.15.78:6789/0 conn(0x7fde800cdde0 legacy=0x7fde80164610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:43.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.548+0000 7fde84d0a700 1 --2- 172.21.15.78:0/2689603734 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fde6c003bb0 0x7fde6c003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:43.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.548+0000 7fde84d0a700 1 --2- 172.21.15.78:0/2689603734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde6c004870 0x7fde6c004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:43.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.548+0000 7fde84d0a700 1 --2- 172.21.15.78:0/2689603734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde6c005220 0x7fde6c00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:43.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.548+0000 7fde84d0a700 1 -- 172.21.15.78:0/2689603734 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c00c050 con 0x7fde6c005220 2023-10-11T09:48:43.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.548+0000 7fde84d0a700 1 -- 172.21.15.78:0/2689603734 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c00c1c0 con 0x7fde6c004870 2023-10-11T09:48:43.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.548+0000 7fde84d0a700 1 -- 172.21.15.78:0/2689603734 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c00c380 con 0x7fde6c003bb0 2023-10-11T09:48:43.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.548+0000 7fde8650d700 1 --2- 172.21.15.78:0/2689603734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde6c005220 0x7fde6c00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:43.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.548+0000 7fde85d0c700 1 --2- 172.21.15.78:0/2689603734 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fde6c003bb0 0x7fde6c003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:43.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.548+0000 7fde8550b700 1 --2- 172.21.15.78:0/2689603734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde6c004870 0x7fde6c004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:43.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.552+0000 7fde85d0c700 1 -- 172.21.15.78:0/2689603734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde6c004870 msgr2=0x7fde6c004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:43.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.552+0000 7fde85d0c700 1 --2- 172.21.15.78:0/2689603734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde6c004870 0x7fde6c004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:43.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.552+0000 7fde85d0c700 1 -- 172.21.15.78:0/2689603734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde6c005220 msgr2=0x7fde6c00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:43.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.552+0000 7fde85d0c700 1 --2- 172.21.15.78:0/2689603734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde6c005220 0x7fde6c00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:43.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.552+0000 7fde85d0c700 1 -- 172.21.15.78:0/2689603734 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde6c0035d0 con 0x7fde6c003bb0 2023-10-11T09:48:43.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.552+0000 7fde8650d700 1 --2- 172.21.15.78:0/2689603734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde6c005220 0x7fde6c00ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:48:43.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.552+0000 7fde8550b700 1 --2- 172.21.15.78:0/2689603734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde6c004870 0x7fde6c004ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:48:43.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.552+0000 7fde8650d700 1 -- 172.21.15.78:0/2689603734 reap_dead start 2023-10-11T09:48:43.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.552+0000 7fde85d0c700 1 --2- 172.21.15.78:0/2689603734 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fde6c003bb0 0x7fde6c003fa0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fde7c009b70 tx=0x7fde6c0058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27f0878320dfe122 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:43.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.552+0000 7fde84d0a700 1 -- 172.21.15.78:0/2689603734 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7c00e6c0 con 0x7fde6c003bb0 2023-10-11T09:48:43.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.552+0000 7fde84d0a700 1 -- 172.21.15.78:0/2689603734 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde7c00e820 con 0x7fde6c003bb0 2023-10-11T09:48:43.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.552+0000 7fde84d0a700 1 -- 172.21.15.78:0/2689603734 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7c00eb90 con 0x7fde6c003bb0 2023-10-11T09:48:43.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.552+0000 7fde86d0e700 1 -- 172.21.15.78:0/2689603734 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fde6c003bb0 msgr2=0x7fde6c003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:43.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.552+0000 7fde86d0e700 1 --2- 172.21.15.78:0/2689603734 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fde6c003bb0 0x7fde6c003fa0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fde7c009b70 tx=0x7fde6c0058a0 comp rx=0 tx=0).stop 2023-10-11T09:48:43.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.552+0000 7fde86d0e700 1 -- 172.21.15.78:0/2689603734 shutdown_connections 2023-10-11T09:48:43.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.552+0000 7fde86d0e700 1 --2- 172.21.15.78:0/2689603734 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fde6c003bb0 0x7fde6c003fa0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:43.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.552+0000 7fde86d0e700 1 -- 172.21.15.78:0/2689603734 >> 172.21.15.78:0/2689603734 conn(0x7fde800cea60 msgr2=0x7fde800cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:43.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.552+0000 7fde86d0e700 1 -- 172.21.15.78:0/2689603734 shutdown_connections 2023-10-11T09:48:43.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.552+0000 7fde86d0e700 1 -- 172.21.15.78:0/2689603734 wait complete. 2023-10-11T09:48:43.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.556+0000 7fde86d0e700 1 Processor -- start 2023-10-11T09:48:43.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.556+0000 7fde86d0e700 1 -- start start 2023-10-11T09:48:43.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.560+0000 7fde86d0e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde6c003bb0 0x7fde8013c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:43.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.560+0000 7fde86d0e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde800709c0 0x7fde8015c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:43.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.560+0000 7fde86d0e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fde801f6b30 0x7fde801f6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:43.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.560+0000 7fde86d0e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8013d4a0 con 0x7fde6c003bb0 2023-10-11T09:48:43.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.560+0000 7fde86d0e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde801f6710 con 0x7fde800709c0 2023-10-11T09:48:43.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.560+0000 7fde86d0e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde800b3e00 con 0x7fde801f6b30 2023-10-11T09:48:43.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.560+0000 7fde85d0c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde6c003bb0 0x7fde8013c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:43.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.560+0000 7fde85d0c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde6c003bb0 0x7fde8013c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44212/0 (socket says 172.21.15.78:44212) 2023-10-11T09:48:43.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.560+0000 7fde85d0c700 1 -- 172.21.15.78:0/1491160893 learned_addr learned my addr 172.21.15.78:0/1491160893 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:43.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.560+0000 7fde8550b700 1 --2- 172.21.15.78:0/1491160893 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde800709c0 0x7fde8015c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:43.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.560+0000 7fde8650d700 1 --2- 172.21.15.78:0/1491160893 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fde801f6b30 0x7fde801f6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:43.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.560+0000 7fde85d0c700 1 -- 172.21.15.78:0/1491160893 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fde801f6b30 msgr2=0x7fde801f6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:43.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.560+0000 7fde85d0c700 1 --2- 172.21.15.78:0/1491160893 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fde801f6b30 0x7fde801f6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:43.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.560+0000 7fde85d0c700 1 -- 172.21.15.78:0/1491160893 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde800709c0 msgr2=0x7fde8015c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:43.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.560+0000 7fde85d0c700 1 --2- 172.21.15.78:0/1491160893 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde800709c0 0x7fde8015c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:43.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.560+0000 7fde85d0c700 1 -- 172.21.15.78:0/1491160893 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde8013cc00 con 0x7fde6c003bb0 2023-10-11T09:48:43.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.560+0000 7fde8650d700 1 --2- 172.21.15.78:0/1491160893 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fde801f6b30 0x7fde801f6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:48:43.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.560+0000 7fde8550b700 1 --2- 172.21.15.78:0/1491160893 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde800709c0 0x7fde8015c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:48:43.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.560+0000 7fde85d0c700 1 --2- 172.21.15.78:0/1491160893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde6c003bb0 0x7fde8013c620 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fde6c00c510 tx=0x7fde7c0047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:43.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.560+0000 7fde767fc700 1 -- 172.21.15.78:0/1491160893 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7c00e6c0 con 0x7fde6c003bb0 2023-10-11T09:48:43.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.560+0000 7fde767fc700 1 -- 172.21.15.78:0/1491160893 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde7c003af0 con 0x7fde6c003bb0 2023-10-11T09:48:43.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.560+0000 7fde86d0e700 1 -- 172.21.15.78:0/1491160893 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde8013cee0 con 0x7fde6c003bb0 2023-10-11T09:48:43.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.560+0000 7fde767fc700 1 -- 172.21.15.78:0/1491160893 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7c003ea0 con 0x7fde6c003bb0 2023-10-11T09:48:43.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.560+0000 7fde86d0e700 1 -- 172.21.15.78:0/1491160893 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde8015ce70 con 0x7fde6c003bb0 2023-10-11T09:48:43.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.560+0000 7fde86d0e700 1 -- 172.21.15.78:0/1491160893 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde8013d400 con 0x7fde6c003bb0 2023-10-11T09:48:43.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.564+0000 7fde767fc700 1 -- 172.21.15.78:0/1491160893 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fde7c013470 con 0x7fde6c003bb0 2023-10-11T09:48:43.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.564+0000 7fde767fc700 1 --2- 172.21.15.78:0/1491160893 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fde64065ed0 0x7fde64068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:43.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.564+0000 7fde767fc700 1 -- 172.21.15.78:0/1491160893 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6466+0+0 (secure 0 0 0) 0x7fde7c084f50 con 0x7fde6c003bb0 2023-10-11T09:48:43.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.564+0000 7fde8550b700 1 --2- 172.21.15.78:0/1491160893 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fde64065ed0 0x7fde64068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:43.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.564+0000 7fde8550b700 1 --2- 172.21.15.78:0/1491160893 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fde64065ed0 0x7fde64068380 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fde8013e730 tx=0x7fde70001470 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:43.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.572+0000 7fde767fc700 1 -- 172.21.15.78:0/1491160893 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fde7c052510 con 0x7fde6c003bb0 2023-10-11T09:48:43.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.780+0000 7fde86d0e700 1 -- 172.21.15.78:0/1491160893 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde80065d40 con 0x7fde6c003bb0 2023-10-11T09:48:43.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.780+0000 7fde767fc700 1 -- 172.21.15.78:0/1491160893 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4653 (secure 0 0 0) 0x7fde7c055b30 con 0x7fde6c003bb0 2023-10-11T09:48:43.790 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:43.790 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":59,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6961,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/1080544187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1080544187},{"type":"v1","addr":"172.21.15.133:6834","nonce":1080544187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/367194191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":367194191},{"type":"v1","addr":"172.21.15.133:6835","nonce":367194191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":7224,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/3463904224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3463904224},{"type":"v1","addr":"172.21.15.78:6836","nonce":3463904224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:48:20.022298+0000","modified":"2023-10-11T09:48:29.819506+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7227},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7227":{"gid":7227,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6837/951312831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":951312831},{"type":"v1","addr":"172.21.15.78:6837","nonce":951312831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12,13],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-11T09:48:43.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.784+0000 7fde86d0e700 1 -- 172.21.15.78:0/1491160893 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fde64065ed0 msgr2=0x7fde64068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:43.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.784+0000 7fde86d0e700 1 --2- 172.21.15.78:0/1491160893 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fde64065ed0 0x7fde64068380 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fde8013e730 tx=0x7fde70001470 comp rx=0 tx=0).stop 2023-10-11T09:48:43.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.784+0000 7fde86d0e700 1 -- 172.21.15.78:0/1491160893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde6c003bb0 msgr2=0x7fde8013c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:43.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.784+0000 7fde86d0e700 1 --2- 172.21.15.78:0/1491160893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde6c003bb0 0x7fde8013c620 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fde6c00c510 tx=0x7fde7c0047e0 comp rx=0 tx=0).stop 2023-10-11T09:48:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.784+0000 7fde86d0e700 1 -- 172.21.15.78:0/1491160893 shutdown_connections 2023-10-11T09:48:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.784+0000 7fde86d0e700 1 --2- 172.21.15.78:0/1491160893 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fde64065ed0 0x7fde64068380 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:43.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.784+0000 7fde86d0e700 1 --2- 172.21.15.78:0/1491160893 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fde801f6b30 0x7fde801f6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:43.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.784+0000 7fde86d0e700 1 --2- 172.21.15.78:0/1491160893 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde800709c0 0x7fde8015c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:43.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.784+0000 7fde86d0e700 1 --2- 172.21.15.78:0/1491160893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde6c003bb0 0x7fde8013c620 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:43.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.784+0000 7fde86d0e700 1 -- 172.21.15.78:0/1491160893 >> 172.21.15.78:0/1491160893 conn(0x7fde800cea60 msgr2=0x7fde800713a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:43.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.784+0000 7fde86d0e700 1 -- 172.21.15.78:0/1491160893 shutdown_connections 2023-10-11T09:48:43.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:43.784+0000 7fde86d0e700 1 -- 172.21.15.78:0/1491160893 wait complete. 2023-10-11T09:48:43.795 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 59 2023-10-11T09:48:43.813 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2023-10-11T09:48:44.013 INFO:teuthology.orchestra.run.smithi078.stdout:{"message":"","return_code":0} 2023-10-11T09:48:44.013 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-10-11T09:48:44.014 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-11T09:48:44.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.184+0000 7fa8a6c53700 1 Processor -- start 2023-10-11T09:48:44.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.184+0000 7fa8a6c53700 1 -- start start 2023-10-11T09:48:44.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.184+0000 7fa8a6c53700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fa8a00caf20 0x7fa8a0164610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:44.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.184+0000 7fa8a6c53700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa8a0164b50 0x7fa8a006bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:44.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.188+0000 7fa8a6c53700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8a0165390 con 0x7fa8a00cda10 2023-10-11T09:48:44.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.188+0000 7fa8a6c53700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8a00cb3a0 con 0x7fa8a00caf20 2023-10-11T09:48:44.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.188+0000 7fa8a6c53700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8a00ca440 con 0x7fa8a0164b50 2023-10-11T09:48:44.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.188+0000 7fa8a5450700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fa8a00caf20 0x7fa8a0164610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:44.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.188+0000 7fa8a5450700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fa8a00caf20 0x7fa8a0164610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44214/0 (socket says 172.21.15.78:44214) 2023-10-11T09:48:44.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.188+0000 7fa8a5450700 1 -- 172.21.15.78:0/1735462158 learned_addr learned my addr 172.21.15.78:0/1735462158 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:44.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.188+0000 7fa8a6452700 1 --2- 172.21.15.78:0/1735462158 >> v2:172.21.15.133:3300/0 conn(0x7fa8a0164b50 0x7fa8a006bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:44.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.188+0000 7fa8a5450700 1 -- 172.21.15.78:0/1735462158 >> v2:172.21.15.133:3300/0 conn(0x7fa8a0164b50 msgr2=0x7fa8a006bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:44.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.188+0000 7fa8a5450700 1 --2- 172.21.15.78:0/1735462158 >> v2:172.21.15.133:3300/0 conn(0x7fa8a0164b50 0x7fa8a006bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:44.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.188+0000 7fa8a5450700 1 -- 172.21.15.78:0/1735462158 >> v1:172.21.15.78:6789/0 conn(0x7fa8a00cda10 legacy=0x7fa8a00cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:44.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.188+0000 7fa8a5450700 1 -- 172.21.15.78:0/1735462158 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a00ca720 con 0x7fa8a00caf20 2023-10-11T09:48:44.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.188+0000 7fa8a4c4f700 1 -- 172.21.15.78:0/1735462158 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1125656234 0 0) 0x7fa8a0165390 con 0x7fa8a00cda10 2023-10-11T09:48:44.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.188+0000 7fa8a6452700 1 --2- 172.21.15.78:0/1735462158 >> v2:172.21.15.133:3300/0 conn(0x7fa8a0164b50 0x7fa8a006bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:48:44.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.188+0000 7fa8a5450700 1 --2- 172.21.15.78:0/1735462158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8a00caf20 0x7fa8a0164610 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fa8a00ce4f0 tx=0x7fa890009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c1a279cc1d691ed server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:44.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.188+0000 7fa8a4c4f700 1 -- 172.21.15.78:0/1735462158 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89000b8a0 con 0x7fa8a00caf20 2023-10-11T09:48:44.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.188+0000 7fa8a4c4f700 1 -- 172.21.15.78:0/1735462158 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa89000ba00 con 0x7fa8a00caf20 2023-10-11T09:48:44.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.188+0000 7fa8a4c4f700 1 -- 172.21.15.78:0/1735462158 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89000bd90 con 0x7fa8a00caf20 2023-10-11T09:48:44.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.188+0000 7fa8a6c53700 1 -- 172.21.15.78:0/1735462158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8a00caf20 msgr2=0x7fa8a0164610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:44.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.188+0000 7fa8a6c53700 1 --2- 172.21.15.78:0/1735462158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8a00caf20 0x7fa8a0164610 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fa8a00ce4f0 tx=0x7fa890009bf0 comp rx=0 tx=0).stop 2023-10-11T09:48:44.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.188+0000 7fa8a6c53700 1 -- 172.21.15.78:0/1735462158 shutdown_connections 2023-10-11T09:48:44.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.188+0000 7fa8a6c53700 1 --2- 172.21.15.78:0/1735462158 >> v2:172.21.15.133:3300/0 conn(0x7fa8a0164b50 0x7fa8a006bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:44.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.188+0000 7fa8a6c53700 1 --2- 172.21.15.78:0/1735462158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8a00caf20 0x7fa8a0164610 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:44.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.188+0000 7fa8a6c53700 1 -- 172.21.15.78:0/1735462158 >> 172.21.15.78:0/1735462158 conn(0x7fa8a00cea60 msgr2=0x7fa8a00cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:44.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.192+0000 7fa8a6c53700 1 -- 172.21.15.78:0/1735462158 shutdown_connections 2023-10-11T09:48:44.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.192+0000 7fa8a6c53700 1 -- 172.21.15.78:0/1735462158 wait complete. 2023-10-11T09:48:44.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.192+0000 7fa8a6c53700 1 Processor -- start 2023-10-11T09:48:44.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.196+0000 7fa8a6c53700 1 -- start start 2023-10-11T09:48:44.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.196+0000 7fa8a6c53700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8a00caf20 0x7fa8a015fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:44.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.196+0000 7fa8a6c53700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8a00cda10 0x7fa8a0070830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:44.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.196+0000 7fa8a6c53700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa8a0164b50 0x7fa8a0070d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:44.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.196+0000 7fa8a6c53700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a00b1a10 con 0x7fa8a00caf20 2023-10-11T09:48:44.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.196+0000 7fa8a6c53700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a0148f40 con 0x7fa8a00cda10 2023-10-11T09:48:44.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.196+0000 7fa8a6c53700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a0149220 con 0x7fa8a0164b50 2023-10-11T09:48:44.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.196+0000 7fa8a5c51700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8a00caf20 0x7fa8a015fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:44.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.196+0000 7fa8a5c51700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8a00caf20 0x7fa8a015fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44216/0 (socket says 172.21.15.78:44216) 2023-10-11T09:48:44.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.196+0000 7fa8a5c51700 1 -- 172.21.15.78:0/3214406014 learned_addr learned my addr 172.21.15.78:0/3214406014 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:44.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.196+0000 7fa8a5450700 1 --2- 172.21.15.78:0/3214406014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8a00cda10 0x7fa8a0070830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:44.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.196+0000 7fa8a6452700 1 --2- 172.21.15.78:0/3214406014 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa8a0164b50 0x7fa8a0070d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:44.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.196+0000 7fa8a5c51700 1 -- 172.21.15.78:0/3214406014 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa8a0164b50 msgr2=0x7fa8a0070d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:44.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.196+0000 7fa8a5c51700 1 --2- 172.21.15.78:0/3214406014 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa8a0164b50 0x7fa8a0070d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:44.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.196+0000 7fa8a5c51700 1 -- 172.21.15.78:0/3214406014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8a00cda10 msgr2=0x7fa8a0070830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:44.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.196+0000 7fa8a5c51700 1 --2- 172.21.15.78:0/3214406014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8a00cda10 0x7fa8a0070830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:44.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.196+0000 7fa8a5c51700 1 -- 172.21.15.78:0/3214406014 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa890009570 con 0x7fa8a00caf20 2023-10-11T09:48:44.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.196+0000 7fa8a5c51700 1 --2- 172.21.15.78:0/3214406014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8a00caf20 0x7fa8a015fe70 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fa89c0011e0 tx=0x7fa89c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:44.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.196+0000 7fa8967fc700 1 -- 172.21.15.78:0/3214406014 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c0054e0 con 0x7fa8a00caf20 2023-10-11T09:48:44.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.200+0000 7fa8967fc700 1 -- 172.21.15.78:0/3214406014 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa89c0036a0 con 0x7fa8a00caf20 2023-10-11T09:48:44.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.200+0000 7fa8a6c53700 1 -- 172.21.15.78:0/3214406014 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8a0149500 con 0x7fa8a00caf20 2023-10-11T09:48:44.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.200+0000 7fa8967fc700 1 -- 172.21.15.78:0/3214406014 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c003a50 con 0x7fa8a00caf20 2023-10-11T09:48:44.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.200+0000 7fa8a6c53700 1 -- 172.21.15.78:0/3214406014 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8a01481a0 con 0x7fa8a00caf20 2023-10-11T09:48:44.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.200+0000 7fa8a6c53700 1 -- 172.21.15.78:0/3214406014 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8a00b1f10 con 0x7fa8a00caf20 2023-10-11T09:48:44.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.200+0000 7fa8967fc700 1 -- 172.21.15.78:0/3214406014 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fa89c003bb0 con 0x7fa8a00caf20 2023-10-11T09:48:44.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.204+0000 7fa8967fc700 1 --2- 172.21.15.78:0/3214406014 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa884065ed0 0x7fa884068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:44.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.204+0000 7fa8967fc700 1 -- 172.21.15.78:0/3214406014 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6466+0+0 (secure 0 0 0) 0x7fa89c082f20 con 0x7fa8a00caf20 2023-10-11T09:48:44.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.204+0000 7fa8a5450700 1 --2- 172.21.15.78:0/3214406014 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa884065ed0 0x7fa884068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:44.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.204+0000 7fa8a5450700 1 --2- 172.21.15.78:0/3214406014 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa884065ed0 0x7fa884068380 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa890000c00 tx=0x7fa890008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:44.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.212+0000 7fa8967fc700 1 -- 172.21.15.78:0/3214406014 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fa8a00b1f10 con 0x7fa8a00caf20 2023-10-11T09:48:44.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.428+0000 7fa8a6c53700 1 -- 172.21.15.78:0/3214406014 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa8a00b1f10 con 0x7fa8a00caf20 2023-10-11T09:48:44.438 INFO:tasks.ceph.mon.a.smithi078.stderr:2023-10-11T09:48:44.428+0000 7f90131f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-11T09:48:44.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.444+0000 7fa8967fc700 1 -- 172.21.15.78:0/3214406014 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v60) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa8a00b1f10 con 0x7fa8a00caf20 2023-10-11T09:48:44.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.448+0000 7fa8a6c53700 1 -- 172.21.15.78:0/3214406014 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa884065ed0 msgr2=0x7fa884068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:44.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.448+0000 7fa8a6c53700 1 --2- 172.21.15.78:0/3214406014 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa884065ed0 0x7fa884068380 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa890000c00 tx=0x7fa890008030 comp rx=0 tx=0).stop 2023-10-11T09:48:44.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.448+0000 7fa8a6c53700 1 -- 172.21.15.78:0/3214406014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8a00caf20 msgr2=0x7fa8a015fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:44.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.448+0000 7fa8a6c53700 1 --2- 172.21.15.78:0/3214406014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8a00caf20 0x7fa8a015fe70 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fa89c0011e0 tx=0x7fa89c00cae0 comp rx=0 tx=0).stop 2023-10-11T09:48:44.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.448+0000 7fa8a6c53700 1 -- 172.21.15.78:0/3214406014 shutdown_connections 2023-10-11T09:48:44.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.448+0000 7fa8a6c53700 1 --2- 172.21.15.78:0/3214406014 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa884065ed0 0x7fa884068380 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:44.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.448+0000 7fa8a6c53700 1 --2- 172.21.15.78:0/3214406014 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa8a0164b50 0x7fa8a0070d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:44.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.448+0000 7fa8a6c53700 1 --2- 172.21.15.78:0/3214406014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8a00cda10 0x7fa8a0070830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:44.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.448+0000 7fa8a6c53700 1 --2- 172.21.15.78:0/3214406014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8a00caf20 0x7fa8a015fe70 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:44.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.448+0000 7fa8a6c53700 1 -- 172.21.15.78:0/3214406014 >> 172.21.15.78:0/3214406014 conn(0x7fa8a00cea60 msgr2=0x7fa8a006bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:44.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.448+0000 7fa8a6c53700 1 -- 172.21.15.78:0/3214406014 shutdown_connections 2023-10-11T09:48:44.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.448+0000 7fa8a6c53700 1 -- 172.21.15.78:0/3214406014 wait complete. 2023-10-11T09:48:44.460 INFO:teuthology.orchestra.run.smithi078.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-11T09:48:44.466 INFO:tasks.ceph.mds.a.smithi078.stderr: -3417> 2023-10-11T09:48:25.524+0000 7f41f4542700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-11T09:48:44.477 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-11T09:48:44.481 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-11T09:48:44.532 INFO:tasks.ceph.mds.a.smithi078.stdout:starting mds.a at 2023-10-11T09:48:44.544 INFO:teuthology.orchestra.run.smithi078.stdout:[global] 2023-10-11T09:48:44.544 INFO:teuthology.orchestra.run.smithi078.stdout: chdir = "" 2023-10-11T09:48:44.545 INFO:teuthology.orchestra.run.smithi078.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-11T09:48:44.545 INFO:teuthology.orchestra.run.smithi078.stdout: auth supported = cephx 2023-10-11T09:48:44.545 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.545 INFO:teuthology.orchestra.run.smithi078.stdout: filestore xattr use omap = true 2023-10-11T09:48:44.545 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.545 INFO:teuthology.orchestra.run.smithi078.stdout: mon clock drift allowed = 1.000 2023-10-11T09:48:44.546 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.546 INFO:teuthology.orchestra.run.smithi078.stdout: osd crush chooseleaf type = 0 2023-10-11T09:48:44.546 INFO:teuthology.orchestra.run.smithi078.stdout: auth debug = true 2023-10-11T09:48:44.546 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.546 INFO:teuthology.orchestra.run.smithi078.stdout: ms die on old message = true 2023-10-11T09:48:44.547 INFO:teuthology.orchestra.run.smithi078.stdout: ms die on bug = true 2023-10-11T09:48:44.547 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.547 INFO:teuthology.orchestra.run.smithi078.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-11T09:48:44.547 INFO:teuthology.orchestra.run.smithi078.stdout: mon pg warn max object skew = 0 2023-10-11T09:48:44.547 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.547 INFO:teuthology.orchestra.run.smithi078.stdout: # disable pg_autoscaler by default for new pools 2023-10-11T09:48:44.548 INFO:teuthology.orchestra.run.smithi078.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-11T09:48:44.548 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.548 INFO:teuthology.orchestra.run.smithi078.stdout: osd pool default size = 2 2023-10-11T09:48:44.548 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.548 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd allow primary affinity = true 2023-10-11T09:48:44.548 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd allow pg remap = true 2023-10-11T09:48:44.549 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on legacy crush tunables = false 2023-10-11T09:48:44.549 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on crush straw calc version zero = false 2023-10-11T09:48:44.549 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on no sortbitwise = false 2023-10-11T09:48:44.549 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on osd down out interval zero = false 2023-10-11T09:48:44.549 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on too few osds = false 2023-10-11T09:48:44.550 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-11T09:48:44.550 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-11T09:48:44.550 INFO:teuthology.orchestra.run.smithi078.stdout: mon_allow_pool_size_one = true 2023-10-11T09:48:44.550 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.550 INFO:teuthology.orchestra.run.smithi078.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-11T09:48:44.551 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.551 INFO:teuthology.orchestra.run.smithi078.stdout: osd default data pool replay window = 5 2023-10-11T09:48:44.551 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.551 INFO:teuthology.orchestra.run.smithi078.stdout: mon allow pool delete = true 2023-10-11T09:48:44.551 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.551 INFO:teuthology.orchestra.run.smithi078.stdout: mon cluster log file level = debug 2023-10-11T09:48:44.552 INFO:teuthology.orchestra.run.smithi078.stdout: debug asserts on shutdown = true 2023-10-11T09:48:44.552 INFO:teuthology.orchestra.run.smithi078.stdout: mon health detail to clog = false 2023-10-11T09:48:44.552 INFO:teuthology.orchestra.run.smithi078.stdout: mon host = "172.21.15.78,172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790]" 2023-10-11T09:48:44.552 INFO:teuthology.orchestra.run.smithi078.stdout: fsid = dee99db9-7a72-406f-a1e2-917943f2b13d 2023-10-11T09:48:44.552 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.553 INFO:teuthology.orchestra.run.smithi078.stdout:[osd] 2023-10-11T09:48:44.553 INFO:teuthology.orchestra.run.smithi078.stdout: osd journal size = 100 2023-10-11T09:48:44.553 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.553 INFO:teuthology.orchestra.run.smithi078.stdout: osd scrub load threshold = 5.0 2023-10-11T09:48:44.553 INFO:teuthology.orchestra.run.smithi078.stdout: osd scrub max interval = 600 2023-10-11T09:48:44.553 INFO:teuthology.orchestra.run.smithi078.stdout: osd mclock profile = high_recovery_ops 2023-10-11T09:48:44.554 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.554 INFO:teuthology.orchestra.run.smithi078.stdout: osd recover clone overlap = true 2023-10-11T09:48:44.554 INFO:teuthology.orchestra.run.smithi078.stdout: osd recovery max chunk = 1048576 2023-10-11T09:48:44.554 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.554 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug shutdown = true 2023-10-11T09:48:44.555 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug op order = true 2023-10-11T09:48:44.555 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug verify stray on activate = true 2023-10-11T09:48:44.555 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.555 INFO:teuthology.orchestra.run.smithi078.stdout: osd open classes on start = true 2023-10-11T09:48:44.555 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug pg log writeout = true 2023-10-11T09:48:44.555 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.556 INFO:teuthology.orchestra.run.smithi078.stdout: osd deep scrub update digest min age = 30 2023-10-11T09:48:44.556 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.556 INFO:teuthology.orchestra.run.smithi078.stdout: osd map max advance = 10 2023-10-11T09:48:44.556 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.556 INFO:teuthology.orchestra.run.smithi078.stdout: journal zero on create = true 2023-10-11T09:48:44.557 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.557 INFO:teuthology.orchestra.run.smithi078.stdout: filestore ondisk finisher threads = 3 2023-10-11T09:48:44.557 INFO:teuthology.orchestra.run.smithi078.stdout: filestore apply finisher threads = 3 2023-10-11T09:48:44.557 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.557 INFO:teuthology.orchestra.run.smithi078.stdout: bdev debug aio = true 2023-10-11T09:48:44.558 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug misdirected ops = true 2023-10-11T09:48:44.558 INFO:teuthology.orchestra.run.smithi078.stdout: bdev async discard = True 2023-10-11T09:48:44.558 INFO:teuthology.orchestra.run.smithi078.stdout: bdev enable discard = True 2023-10-11T09:48:44.558 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore allocator = bitmap 2023-10-11T09:48:44.558 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore block size = 96636764160 2023-10-11T09:48:44.559 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore fsck on mount = True 2023-10-11T09:48:44.559 INFO:teuthology.orchestra.run.smithi078.stdout: debug bluefs = 1/20 2023-10-11T09:48:44.559 INFO:teuthology.orchestra.run.smithi078.stdout: debug bluestore = 1/20 2023-10-11T09:48:44.559 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:48:44.559 INFO:teuthology.orchestra.run.smithi078.stdout: debug osd = 20 2023-10-11T09:48:44.559 INFO:teuthology.orchestra.run.smithi078.stdout: debug rocksdb = 4/10 2023-10-11T09:48:44.560 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd backfillfull_ratio = 0.85 2023-10-11T09:48:44.560 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd full ratio = 0.9 2023-10-11T09:48:44.560 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd nearfull ratio = 0.8 2023-10-11T09:48:44.560 INFO:teuthology.orchestra.run.smithi078.stdout: osd failsafe full ratio = 0.95 2023-10-11T09:48:44.560 INFO:teuthology.orchestra.run.smithi078.stdout: osd objectstore = bluestore 2023-10-11T09:48:44.561 INFO:teuthology.orchestra.run.smithi078.stdout: osd op complaint time = 180 2023-10-11T09:48:44.561 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.561 INFO:teuthology.orchestra.run.smithi078.stdout:[mgr] 2023-10-11T09:48:44.561 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:48:44.561 INFO:teuthology.orchestra.run.smithi078.stdout: debug mgr = 20 2023-10-11T09:48:44.562 INFO:teuthology.orchestra.run.smithi078.stdout: debug mon = 20 2023-10-11T09:48:44.562 INFO:teuthology.orchestra.run.smithi078.stdout: debug auth = 20 2023-10-11T09:48:44.562 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:48:44.562 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:48:44.562 INFO:teuthology.orchestra.run.smithi078.stdout: mgr/telemetry/nag = false 2023-10-11T09:48:44.563 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.563 INFO:teuthology.orchestra.run.smithi078.stdout:[mon] 2023-10-11T09:48:44.563 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:48:44.563 INFO:teuthology.orchestra.run.smithi078.stdout: debug mon = 20 2023-10-11T09:48:44.563 INFO:teuthology.orchestra.run.smithi078.stdout: debug paxos = 20 2023-10-11T09:48:44.563 INFO:teuthology.orchestra.run.smithi078.stdout: debug auth = 20 2023-10-11T09:48:44.564 INFO:teuthology.orchestra.run.smithi078.stdout: mon data avail warn = 5 2023-10-11T09:48:44.564 INFO:teuthology.orchestra.run.smithi078.stdout: mon mgr mkfs grace = 240 2023-10-11T09:48:44.564 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:48:44.564 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd reporter subtree level = osd 2023-10-11T09:48:44.564 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd prime pg temp = true 2023-10-11T09:48:44.564 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:48:44.565 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.565 INFO:teuthology.orchestra.run.smithi078.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-11T09:48:44.565 INFO:teuthology.orchestra.run.smithi078.stdout: auth mon ticket ttl = 660 # 11m 2023-10-11T09:48:44.565 INFO:teuthology.orchestra.run.smithi078.stdout: auth service ticket ttl = 240 # 4m 2023-10-11T09:48:44.565 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.565 INFO:teuthology.orchestra.run.smithi078.stdout: # don't complain about insecure global_id in the test suite 2023-10-11T09:48:44.566 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-11T09:48:44.566 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-11T09:48:44.566 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.566 INFO:teuthology.orchestra.run.smithi078.stdout: # 1m isn't quite enough 2023-10-11T09:48:44.566 INFO:teuthology.orchestra.run.smithi078.stdout: mon_down_mkfs_grace = 2m 2023-10-11T09:48:44.566 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.567 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_filestore_osds = false 2023-10-11T09:48:44.567 INFO:teuthology.orchestra.run.smithi078.stdout: mon op complaint time = 120 2023-10-11T09:48:44.567 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.567 INFO:teuthology.orchestra.run.smithi078.stdout:[client] 2023-10-11T09:48:44.567 INFO:teuthology.orchestra.run.smithi078.stdout: rgw cache enabled = true 2023-10-11T09:48:44.568 INFO:teuthology.orchestra.run.smithi078.stdout: rgw enable ops log = true 2023-10-11T09:48:44.568 INFO:teuthology.orchestra.run.smithi078.stdout: rgw enable usage log = true 2023-10-11T09:48:44.568 INFO:teuthology.orchestra.run.smithi078.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-11T09:48:44.568 INFO:teuthology.orchestra.run.smithi078.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-11T09:48:44.568 INFO:teuthology.orchestra.run.smithi078.stdout: client mount timeout = 600 2023-10-11T09:48:44.569 INFO:teuthology.orchestra.run.smithi078.stdout: debug client = 20 2023-10-11T09:48:44.569 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:48:44.569 INFO:teuthology.orchestra.run.smithi078.stdout: pid file = "" 2023-10-11T09:48:44.569 INFO:teuthology.orchestra.run.smithi078.stdout: rados mon op timeout = 900 2023-10-11T09:48:44.569 INFO:teuthology.orchestra.run.smithi078.stdout: rados osd op timeout = 900 2023-10-11T09:48:44.569 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.a] 2023-10-11T09:48:44.570 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.b] 2023-10-11T09:48:44.570 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.c] 2023-10-11T09:48:44.570 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.a] 2023-10-11T09:48:44.570 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.c] 2023-10-11T09:48:44.570 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.b] 2023-10-11T09:48:44.571 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.d] 2023-10-11T09:48:44.571 INFO:teuthology.orchestra.run.smithi078.stdout:[mds] 2023-10-11T09:48:44.571 INFO:teuthology.orchestra.run.smithi078.stdout: debug mds = 20 2023-10-11T09:48:44.571 INFO:teuthology.orchestra.run.smithi078.stdout: debug mds balancer = 20 2023-10-11T09:48:44.571 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:48:44.571 INFO:teuthology.orchestra.run.smithi078.stdout: mds debug frag = True 2023-10-11T09:48:44.572 INFO:teuthology.orchestra.run.smithi078.stdout: mds debug scatterstat = False 2023-10-11T09:48:44.572 INFO:teuthology.orchestra.run.smithi078.stdout: mds op complaint time = 180 2023-10-11T09:48:44.572 INFO:teuthology.orchestra.run.smithi078.stdout: mds verify scatter = False 2023-10-11T09:48:44.572 INFO:teuthology.orchestra.run.smithi078.stdout: osd op complaint time = 180 2023-10-11T09:48:44.572 INFO:teuthology.orchestra.run.smithi078.stdout: rados mon op timeout = 900 2023-10-11T09:48:44.572 INFO:teuthology.orchestra.run.smithi078.stdout: rados osd op timeout = 900 2023-10-11T09:48:44.573 INFO:teuthology.orchestra.run.smithi133.stdout:[global] 2023-10-11T09:48:44.573 INFO:teuthology.orchestra.run.smithi133.stdout: chdir = "" 2023-10-11T09:48:44.574 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-11T09:48:44.574 INFO:teuthology.orchestra.run.smithi133.stdout: auth supported = cephx 2023-10-11T09:48:44.574 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.574 INFO:teuthology.orchestra.run.smithi133.stdout: filestore xattr use omap = true 2023-10-11T09:48:44.574 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.575 INFO:teuthology.orchestra.run.smithi133.stdout: mon clock drift allowed = 1.000 2023-10-11T09:48:44.575 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.575 INFO:teuthology.orchestra.run.smithi133.stdout: osd crush chooseleaf type = 0 2023-10-11T09:48:44.575 INFO:teuthology.orchestra.run.smithi133.stdout: auth debug = true 2023-10-11T09:48:44.575 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.575 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on old message = true 2023-10-11T09:48:44.576 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on bug = true 2023-10-11T09:48:44.576 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.576 INFO:teuthology.orchestra.run.smithi133.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-11T09:48:44.576 INFO:teuthology.orchestra.run.smithi133.stdout: mon pg warn max object skew = 0 2023-10-11T09:48:44.576 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.576 INFO:teuthology.orchestra.run.smithi133.stdout: # disable pg_autoscaler by default for new pools 2023-10-11T09:48:44.577 INFO:teuthology.orchestra.run.smithi133.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-11T09:48:44.577 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.577 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default size = 2 2023-10-11T09:48:44.577 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.577 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow primary affinity = true 2023-10-11T09:48:44.577 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow pg remap = true 2023-10-11T09:48:44.578 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on legacy crush tunables = false 2023-10-11T09:48:44.578 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on crush straw calc version zero = false 2023-10-11T09:48:44.578 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on no sortbitwise = false 2023-10-11T09:48:44.578 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on osd down out interval zero = false 2023-10-11T09:48:44.578 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on too few osds = false 2023-10-11T09:48:44.579 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-11T09:48:44.579 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-11T09:48:44.579 INFO:teuthology.orchestra.run.smithi133.stdout: mon_allow_pool_size_one = true 2023-10-11T09:48:44.579 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.579 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-11T09:48:44.580 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.580 INFO:teuthology.orchestra.run.smithi133.stdout: osd default data pool replay window = 5 2023-10-11T09:48:44.580 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.580 INFO:teuthology.orchestra.run.smithi133.stdout: mon allow pool delete = true 2023-10-11T09:48:44.580 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.580 INFO:teuthology.orchestra.run.smithi133.stdout: mon cluster log file level = debug 2023-10-11T09:48:44.581 INFO:teuthology.orchestra.run.smithi133.stdout: debug asserts on shutdown = true 2023-10-11T09:48:44.581 INFO:teuthology.orchestra.run.smithi133.stdout: mon health detail to clog = false 2023-10-11T09:48:44.581 INFO:teuthology.orchestra.run.smithi133.stdout: mon host = "172.21.15.78,172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790]" 2023-10-11T09:48:44.581 INFO:teuthology.orchestra.run.smithi133.stdout: fsid = dee99db9-7a72-406f-a1e2-917943f2b13d 2023-10-11T09:48:44.581 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.581 INFO:teuthology.orchestra.run.smithi133.stdout:[osd] 2023-10-11T09:48:44.582 INFO:teuthology.orchestra.run.smithi133.stdout: osd journal size = 100 2023-10-11T09:48:44.582 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.582 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub load threshold = 5.0 2023-10-11T09:48:44.582 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub max interval = 600 2023-10-11T09:48:44.582 INFO:teuthology.orchestra.run.smithi133.stdout: osd mclock profile = high_recovery_ops 2023-10-11T09:48:44.582 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.583 INFO:teuthology.orchestra.run.smithi133.stdout: osd recover clone overlap = true 2023-10-11T09:48:44.583 INFO:teuthology.orchestra.run.smithi133.stdout: osd recovery max chunk = 1048576 2023-10-11T09:48:44.583 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.583 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug shutdown = true 2023-10-11T09:48:44.583 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug op order = true 2023-10-11T09:48:44.583 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug verify stray on activate = true 2023-10-11T09:48:44.584 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.584 INFO:teuthology.orchestra.run.smithi133.stdout: osd open classes on start = true 2023-10-11T09:48:44.584 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug pg log writeout = true 2023-10-11T09:48:44.584 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.584 INFO:teuthology.orchestra.run.smithi133.stdout: osd deep scrub update digest min age = 30 2023-10-11T09:48:44.585 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.585 INFO:teuthology.orchestra.run.smithi133.stdout: osd map max advance = 10 2023-10-11T09:48:44.585 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.585 INFO:teuthology.orchestra.run.smithi133.stdout: journal zero on create = true 2023-10-11T09:48:44.585 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.586 INFO:teuthology.orchestra.run.smithi133.stdout: filestore ondisk finisher threads = 3 2023-10-11T09:48:44.586 INFO:teuthology.orchestra.run.smithi133.stdout: filestore apply finisher threads = 3 2023-10-11T09:48:44.586 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.586 INFO:teuthology.orchestra.run.smithi133.stdout: bdev debug aio = true 2023-10-11T09:48:44.586 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug misdirected ops = true 2023-10-11T09:48:44.587 INFO:teuthology.orchestra.run.smithi133.stdout: bdev async discard = True 2023-10-11T09:48:44.587 INFO:teuthology.orchestra.run.smithi133.stdout: bdev enable discard = True 2023-10-11T09:48:44.587 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore allocator = bitmap 2023-10-11T09:48:44.587 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore block size = 96636764160 2023-10-11T09:48:44.587 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore fsck on mount = True 2023-10-11T09:48:44.587 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluefs = 1/20 2023-10-11T09:48:44.588 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluestore = 1/20 2023-10-11T09:48:44.588 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:48:44.588 INFO:teuthology.orchestra.run.smithi133.stdout: debug osd = 20 2023-10-11T09:48:44.588 INFO:teuthology.orchestra.run.smithi133.stdout: debug rocksdb = 4/10 2023-10-11T09:48:44.588 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd backfillfull_ratio = 0.85 2023-10-11T09:48:44.589 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd full ratio = 0.9 2023-10-11T09:48:44.589 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd nearfull ratio = 0.8 2023-10-11T09:48:44.589 INFO:teuthology.orchestra.run.smithi133.stdout: osd failsafe full ratio = 0.95 2023-10-11T09:48:44.589 INFO:teuthology.orchestra.run.smithi133.stdout: osd objectstore = bluestore 2023-10-11T09:48:44.589 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2023-10-11T09:48:44.590 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.590 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr] 2023-10-11T09:48:44.590 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:48:44.590 INFO:teuthology.orchestra.run.smithi133.stdout: debug mgr = 20 2023-10-11T09:48:44.590 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2023-10-11T09:48:44.591 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2023-10-11T09:48:44.591 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:48:44.591 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:48:44.591 INFO:teuthology.orchestra.run.smithi133.stdout: mgr/telemetry/nag = false 2023-10-11T09:48:44.591 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.592 INFO:teuthology.orchestra.run.smithi133.stdout:[mon] 2023-10-11T09:48:44.592 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:48:44.592 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2023-10-11T09:48:44.592 INFO:teuthology.orchestra.run.smithi133.stdout: debug paxos = 20 2023-10-11T09:48:44.592 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2023-10-11T09:48:44.592 INFO:teuthology.orchestra.run.smithi133.stdout: mon data avail warn = 5 2023-10-11T09:48:44.593 INFO:teuthology.orchestra.run.smithi133.stdout: mon mgr mkfs grace = 240 2023-10-11T09:48:44.593 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:48:44.593 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd reporter subtree level = osd 2023-10-11T09:48:44.593 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd prime pg temp = true 2023-10-11T09:48:44.593 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:48:44.594 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.594 INFO:teuthology.orchestra.run.smithi133.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-11T09:48:44.594 INFO:teuthology.orchestra.run.smithi133.stdout: auth mon ticket ttl = 660 # 11m 2023-10-11T09:48:44.594 INFO:teuthology.orchestra.run.smithi133.stdout: auth service ticket ttl = 240 # 4m 2023-10-11T09:48:44.594 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.595 INFO:teuthology.orchestra.run.smithi133.stdout: # don't complain about insecure global_id in the test suite 2023-10-11T09:48:44.595 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-11T09:48:44.595 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-11T09:48:44.595 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.595 INFO:teuthology.orchestra.run.smithi133.stdout: # 1m isn't quite enough 2023-10-11T09:48:44.595 INFO:teuthology.orchestra.run.smithi133.stdout: mon_down_mkfs_grace = 2m 2023-10-11T09:48:44.596 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.596 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_filestore_osds = false 2023-10-11T09:48:44.596 INFO:teuthology.orchestra.run.smithi133.stdout: mon op complaint time = 120 2023-10-11T09:48:44.596 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.596 INFO:teuthology.orchestra.run.smithi133.stdout:[client] 2023-10-11T09:48:44.597 INFO:teuthology.orchestra.run.smithi133.stdout: rgw cache enabled = true 2023-10-11T09:48:44.597 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable ops log = true 2023-10-11T09:48:44.597 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable usage log = true 2023-10-11T09:48:44.597 INFO:teuthology.orchestra.run.smithi133.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-11T09:48:44.597 INFO:teuthology.orchestra.run.smithi133.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-11T09:48:44.598 INFO:teuthology.orchestra.run.smithi133.stdout: client mount timeout = 600 2023-10-11T09:48:44.598 INFO:teuthology.orchestra.run.smithi133.stdout: debug client = 20 2023-10-11T09:48:44.598 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:48:44.598 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = "" 2023-10-11T09:48:44.598 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2023-10-11T09:48:44.599 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2023-10-11T09:48:44.599 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.a] 2023-10-11T09:48:44.599 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.b] 2023-10-11T09:48:44.599 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.c] 2023-10-11T09:48:44.599 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.a] 2023-10-11T09:48:44.599 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.c] 2023-10-11T09:48:44.600 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.b] 2023-10-11T09:48:44.600 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.d] 2023-10-11T09:48:44.600 INFO:teuthology.orchestra.run.smithi133.stdout:[mds] 2023-10-11T09:48:44.600 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds = 20 2023-10-11T09:48:44.600 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds balancer = 20 2023-10-11T09:48:44.600 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:48:44.601 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug frag = True 2023-10-11T09:48:44.601 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug scatterstat = False 2023-10-11T09:48:44.601 INFO:teuthology.orchestra.run.smithi133.stdout: mds op complaint time = 180 2023-10-11T09:48:44.601 INFO:teuthology.orchestra.run.smithi133.stdout: mds verify scatter = False 2023-10-11T09:48:44.601 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2023-10-11T09:48:44.602 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2023-10-11T09:48:44.602 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2023-10-11T09:48:44.604 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-11T09:48:44.649 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-11T09:48:44.716 INFO:teuthology.orchestra.run.smithi078.stdout:[global] 2023-10-11T09:48:44.716 INFO:teuthology.orchestra.run.smithi078.stdout: chdir = "" 2023-10-11T09:48:44.716 INFO:teuthology.orchestra.run.smithi078.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-11T09:48:44.717 INFO:teuthology.orchestra.run.smithi078.stdout: auth supported = cephx 2023-10-11T09:48:44.717 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.717 INFO:teuthology.orchestra.run.smithi078.stdout: filestore xattr use omap = true 2023-10-11T09:48:44.717 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.717 INFO:teuthology.orchestra.run.smithi078.stdout: mon clock drift allowed = 1.000 2023-10-11T09:48:44.718 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.718 INFO:teuthology.orchestra.run.smithi078.stdout: osd crush chooseleaf type = 0 2023-10-11T09:48:44.718 INFO:teuthology.orchestra.run.smithi078.stdout: auth debug = true 2023-10-11T09:48:44.718 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.718 INFO:teuthology.orchestra.run.smithi078.stdout: ms die on old message = true 2023-10-11T09:48:44.719 INFO:teuthology.orchestra.run.smithi078.stdout: ms die on bug = true 2023-10-11T09:48:44.719 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.719 INFO:teuthology.orchestra.run.smithi078.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-11T09:48:44.719 INFO:teuthology.orchestra.run.smithi078.stdout: mon pg warn max object skew = 0 2023-10-11T09:48:44.719 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.719 INFO:teuthology.orchestra.run.smithi078.stdout: # disable pg_autoscaler by default for new pools 2023-10-11T09:48:44.720 INFO:teuthology.orchestra.run.smithi078.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-11T09:48:44.720 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.720 INFO:teuthology.orchestra.run.smithi078.stdout: osd pool default size = 2 2023-10-11T09:48:44.720 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.720 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd allow primary affinity = true 2023-10-11T09:48:44.721 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd allow pg remap = true 2023-10-11T09:48:44.721 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on legacy crush tunables = false 2023-10-11T09:48:44.721 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on crush straw calc version zero = false 2023-10-11T09:48:44.721 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on no sortbitwise = false 2023-10-11T09:48:44.721 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on osd down out interval zero = false 2023-10-11T09:48:44.721 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on too few osds = false 2023-10-11T09:48:44.722 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-11T09:48:44.722 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-11T09:48:44.722 INFO:teuthology.orchestra.run.smithi078.stdout: mon_allow_pool_size_one = true 2023-10-11T09:48:44.722 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.722 INFO:teuthology.orchestra.run.smithi078.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-11T09:48:44.723 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.723 INFO:teuthology.orchestra.run.smithi078.stdout: osd default data pool replay window = 5 2023-10-11T09:48:44.723 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.723 INFO:teuthology.orchestra.run.smithi078.stdout: mon allow pool delete = true 2023-10-11T09:48:44.723 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.724 INFO:teuthology.orchestra.run.smithi078.stdout: mon cluster log file level = debug 2023-10-11T09:48:44.724 INFO:teuthology.orchestra.run.smithi078.stdout: debug asserts on shutdown = true 2023-10-11T09:48:44.724 INFO:teuthology.orchestra.run.smithi078.stdout: mon health detail to clog = false 2023-10-11T09:48:44.724 INFO:teuthology.orchestra.run.smithi078.stdout: mon host = "172.21.15.78,172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790]" 2023-10-11T09:48:44.724 INFO:teuthology.orchestra.run.smithi078.stdout: fsid = dee99db9-7a72-406f-a1e2-917943f2b13d 2023-10-11T09:48:44.724 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.725 INFO:teuthology.orchestra.run.smithi078.stdout:[osd] 2023-10-11T09:48:44.725 INFO:teuthology.orchestra.run.smithi078.stdout: osd journal size = 100 2023-10-11T09:48:44.725 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.725 INFO:teuthology.orchestra.run.smithi078.stdout: osd scrub load threshold = 5.0 2023-10-11T09:48:44.725 INFO:teuthology.orchestra.run.smithi078.stdout: osd scrub max interval = 600 2023-10-11T09:48:44.726 INFO:teuthology.orchestra.run.smithi078.stdout: osd mclock profile = high_recovery_ops 2023-10-11T09:48:44.726 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.726 INFO:teuthology.orchestra.run.smithi078.stdout: osd recover clone overlap = true 2023-10-11T09:48:44.726 INFO:teuthology.orchestra.run.smithi078.stdout: osd recovery max chunk = 1048576 2023-10-11T09:48:44.727 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.727 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug shutdown = true 2023-10-11T09:48:44.727 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug op order = true 2023-10-11T09:48:44.727 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug verify stray on activate = true 2023-10-11T09:48:44.727 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.728 INFO:teuthology.orchestra.run.smithi078.stdout: osd open classes on start = true 2023-10-11T09:48:44.728 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug pg log writeout = true 2023-10-11T09:48:44.728 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.728 INFO:teuthology.orchestra.run.smithi078.stdout: osd deep scrub update digest min age = 30 2023-10-11T09:48:44.728 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.729 INFO:teuthology.orchestra.run.smithi078.stdout: osd map max advance = 10 2023-10-11T09:48:44.729 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.729 INFO:teuthology.orchestra.run.smithi078.stdout: journal zero on create = true 2023-10-11T09:48:44.729 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.729 INFO:teuthology.orchestra.run.smithi078.stdout: filestore ondisk finisher threads = 3 2023-10-11T09:48:44.730 INFO:teuthology.orchestra.run.smithi078.stdout: filestore apply finisher threads = 3 2023-10-11T09:48:44.730 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.730 INFO:teuthology.orchestra.run.smithi078.stdout: bdev debug aio = true 2023-10-11T09:48:44.730 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug misdirected ops = true 2023-10-11T09:48:44.731 INFO:teuthology.orchestra.run.smithi078.stdout: bdev async discard = True 2023-10-11T09:48:44.731 INFO:teuthology.orchestra.run.smithi078.stdout: bdev enable discard = True 2023-10-11T09:48:44.731 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore allocator = bitmap 2023-10-11T09:48:44.731 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore block size = 96636764160 2023-10-11T09:48:44.731 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore fsck on mount = True 2023-10-11T09:48:44.732 INFO:teuthology.orchestra.run.smithi078.stdout: debug bluefs = 1/20 2023-10-11T09:48:44.732 INFO:teuthology.orchestra.run.smithi078.stdout: debug bluestore = 1/20 2023-10-11T09:48:44.732 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:48:44.732 INFO:teuthology.orchestra.run.smithi078.stdout: debug osd = 20 2023-10-11T09:48:44.732 INFO:teuthology.orchestra.run.smithi078.stdout: debug rocksdb = 4/10 2023-10-11T09:48:44.733 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd backfillfull_ratio = 0.85 2023-10-11T09:48:44.733 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd full ratio = 0.9 2023-10-11T09:48:44.733 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd nearfull ratio = 0.8 2023-10-11T09:48:44.733 INFO:teuthology.orchestra.run.smithi078.stdout: osd failsafe full ratio = 0.95 2023-10-11T09:48:44.734 INFO:teuthology.orchestra.run.smithi078.stdout: osd objectstore = bluestore 2023-10-11T09:48:44.734 INFO:teuthology.orchestra.run.smithi078.stdout: osd op complaint time = 180 2023-10-11T09:48:44.734 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.734 INFO:teuthology.orchestra.run.smithi078.stdout:[mgr] 2023-10-11T09:48:44.734 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:48:44.735 INFO:teuthology.orchestra.run.smithi078.stdout: debug mgr = 20 2023-10-11T09:48:44.735 INFO:teuthology.orchestra.run.smithi078.stdout: debug mon = 20 2023-10-11T09:48:44.735 INFO:teuthology.orchestra.run.smithi078.stdout: debug auth = 20 2023-10-11T09:48:44.735 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:48:44.736 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:48:44.736 INFO:teuthology.orchestra.run.smithi078.stdout: mgr/telemetry/nag = false 2023-10-11T09:48:44.736 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.736 INFO:teuthology.orchestra.run.smithi078.stdout:[mon] 2023-10-11T09:48:44.736 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:48:44.737 INFO:teuthology.orchestra.run.smithi078.stdout: debug mon = 20 2023-10-11T09:48:44.737 INFO:teuthology.orchestra.run.smithi078.stdout: debug paxos = 20 2023-10-11T09:48:44.737 INFO:teuthology.orchestra.run.smithi078.stdout: debug auth = 20 2023-10-11T09:48:44.737 INFO:teuthology.orchestra.run.smithi078.stdout: mon data avail warn = 5 2023-10-11T09:48:44.737 INFO:teuthology.orchestra.run.smithi078.stdout: mon mgr mkfs grace = 240 2023-10-11T09:48:44.738 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:48:44.738 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd reporter subtree level = osd 2023-10-11T09:48:44.738 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd prime pg temp = true 2023-10-11T09:48:44.738 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:48:44.739 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.739 INFO:teuthology.orchestra.run.smithi078.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-11T09:48:44.739 INFO:teuthology.orchestra.run.smithi078.stdout: auth mon ticket ttl = 660 # 11m 2023-10-11T09:48:44.739 INFO:teuthology.orchestra.run.smithi078.stdout: auth service ticket ttl = 240 # 4m 2023-10-11T09:48:44.739 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.739 INFO:teuthology.orchestra.run.smithi078.stdout: # don't complain about insecure global_id in the test suite 2023-10-11T09:48:44.740 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-11T09:48:44.740 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-11T09:48:44.740 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.740 INFO:teuthology.orchestra.run.smithi078.stdout: # 1m isn't quite enough 2023-10-11T09:48:44.740 INFO:teuthology.orchestra.run.smithi078.stdout: mon_down_mkfs_grace = 2m 2023-10-11T09:48:44.741 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.741 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_filestore_osds = false 2023-10-11T09:48:44.741 INFO:teuthology.orchestra.run.smithi078.stdout: mon op complaint time = 120 2023-10-11T09:48:44.741 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:44.741 INFO:teuthology.orchestra.run.smithi078.stdout:[client] 2023-10-11T09:48:44.742 INFO:teuthology.orchestra.run.smithi078.stdout: rgw cache enabled = true 2023-10-11T09:48:44.742 INFO:teuthology.orchestra.run.smithi078.stdout: rgw enable ops log = true 2023-10-11T09:48:44.742 INFO:teuthology.orchestra.run.smithi078.stdout: rgw enable usage log = true 2023-10-11T09:48:44.742 INFO:teuthology.orchestra.run.smithi078.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-11T09:48:44.742 INFO:teuthology.orchestra.run.smithi078.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-11T09:48:44.742 INFO:teuthology.orchestra.run.smithi078.stdout: client mount timeout = 600 2023-10-11T09:48:44.743 INFO:teuthology.orchestra.run.smithi078.stdout: debug client = 20 2023-10-11T09:48:44.743 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:48:44.743 INFO:teuthology.orchestra.run.smithi078.stdout: pid file = "" 2023-10-11T09:48:44.743 INFO:teuthology.orchestra.run.smithi078.stdout: rados mon op timeout = 900 2023-10-11T09:48:44.743 INFO:teuthology.orchestra.run.smithi078.stdout: rados osd op timeout = 900 2023-10-11T09:48:44.744 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.a] 2023-10-11T09:48:44.744 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.b] 2023-10-11T09:48:44.744 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.c] 2023-10-11T09:48:44.744 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.a] 2023-10-11T09:48:44.744 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.c] 2023-10-11T09:48:44.745 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.b] 2023-10-11T09:48:44.745 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.d] 2023-10-11T09:48:44.745 INFO:teuthology.orchestra.run.smithi078.stdout:[mds] 2023-10-11T09:48:44.745 INFO:teuthology.orchestra.run.smithi078.stdout: debug mds = 20 2023-10-11T09:48:44.745 INFO:teuthology.orchestra.run.smithi078.stdout: debug mds balancer = 20 2023-10-11T09:48:44.745 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:48:44.746 INFO:teuthology.orchestra.run.smithi078.stdout: mds debug frag = True 2023-10-11T09:48:44.746 INFO:teuthology.orchestra.run.smithi078.stdout: mds debug scatterstat = False 2023-10-11T09:48:44.746 INFO:teuthology.orchestra.run.smithi078.stdout: mds op complaint time = 180 2023-10-11T09:48:44.746 INFO:teuthology.orchestra.run.smithi078.stdout: mds verify scatter = False 2023-10-11T09:48:44.746 INFO:teuthology.orchestra.run.smithi078.stdout: osd op complaint time = 180 2023-10-11T09:48:44.747 INFO:teuthology.orchestra.run.smithi078.stdout: rados mon op timeout = 900 2023-10-11T09:48:44.747 INFO:teuthology.orchestra.run.smithi078.stdout: rados osd op timeout = 900 2023-10-11T09:48:44.748 INFO:teuthology.orchestra.run.smithi133.stdout:[global] 2023-10-11T09:48:44.748 INFO:teuthology.orchestra.run.smithi133.stdout: chdir = "" 2023-10-11T09:48:44.748 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-11T09:48:44.749 INFO:teuthology.orchestra.run.smithi133.stdout: auth supported = cephx 2023-10-11T09:48:44.749 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.749 INFO:teuthology.orchestra.run.smithi133.stdout: filestore xattr use omap = true 2023-10-11T09:48:44.749 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.750 INFO:teuthology.orchestra.run.smithi133.stdout: mon clock drift allowed = 1.000 2023-10-11T09:48:44.750 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.750 INFO:teuthology.orchestra.run.smithi133.stdout: osd crush chooseleaf type = 0 2023-10-11T09:48:44.750 INFO:teuthology.orchestra.run.smithi133.stdout: auth debug = true 2023-10-11T09:48:44.750 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.751 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on old message = true 2023-10-11T09:48:44.751 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on bug = true 2023-10-11T09:48:44.751 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.751 INFO:teuthology.orchestra.run.smithi133.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-11T09:48:44.751 INFO:teuthology.orchestra.run.smithi133.stdout: mon pg warn max object skew = 0 2023-10-11T09:48:44.752 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.752 INFO:teuthology.orchestra.run.smithi133.stdout: # disable pg_autoscaler by default for new pools 2023-10-11T09:48:44.752 INFO:teuthology.orchestra.run.smithi133.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-11T09:48:44.752 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.752 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default size = 2 2023-10-11T09:48:44.752 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.753 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow primary affinity = true 2023-10-11T09:48:44.753 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow pg remap = true 2023-10-11T09:48:44.753 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on legacy crush tunables = false 2023-10-11T09:48:44.753 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on crush straw calc version zero = false 2023-10-11T09:48:44.753 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on no sortbitwise = false 2023-10-11T09:48:44.754 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on osd down out interval zero = false 2023-10-11T09:48:44.754 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on too few osds = false 2023-10-11T09:48:44.754 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-11T09:48:44.754 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-11T09:48:44.755 INFO:teuthology.orchestra.run.smithi133.stdout: mon_allow_pool_size_one = true 2023-10-11T09:48:44.755 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.755 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-11T09:48:44.755 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.755 INFO:teuthology.orchestra.run.smithi133.stdout: osd default data pool replay window = 5 2023-10-11T09:48:44.756 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.756 INFO:teuthology.orchestra.run.smithi133.stdout: mon allow pool delete = true 2023-10-11T09:48:44.756 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.756 INFO:teuthology.orchestra.run.smithi133.stdout: mon cluster log file level = debug 2023-10-11T09:48:44.756 INFO:teuthology.orchestra.run.smithi133.stdout: debug asserts on shutdown = true 2023-10-11T09:48:44.756 INFO:teuthology.orchestra.run.smithi133.stdout: mon health detail to clog = false 2023-10-11T09:48:44.757 INFO:teuthology.orchestra.run.smithi133.stdout: mon host = "172.21.15.78,172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790]" 2023-10-11T09:48:44.757 INFO:teuthology.orchestra.run.smithi133.stdout: fsid = dee99db9-7a72-406f-a1e2-917943f2b13d 2023-10-11T09:48:44.757 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.757 INFO:teuthology.orchestra.run.smithi133.stdout:[osd] 2023-10-11T09:48:44.758 INFO:teuthology.orchestra.run.smithi133.stdout: osd journal size = 100 2023-10-11T09:48:44.758 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.758 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub load threshold = 5.0 2023-10-11T09:48:44.758 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub max interval = 600 2023-10-11T09:48:44.758 INFO:teuthology.orchestra.run.smithi133.stdout: osd mclock profile = high_recovery_ops 2023-10-11T09:48:44.759 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.759 INFO:teuthology.orchestra.run.smithi133.stdout: osd recover clone overlap = true 2023-10-11T09:48:44.759 INFO:teuthology.orchestra.run.smithi133.stdout: osd recovery max chunk = 1048576 2023-10-11T09:48:44.759 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.760 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug shutdown = true 2023-10-11T09:48:44.760 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug op order = true 2023-10-11T09:48:44.760 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug verify stray on activate = true 2023-10-11T09:48:44.760 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.760 INFO:teuthology.orchestra.run.smithi133.stdout: osd open classes on start = true 2023-10-11T09:48:44.761 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug pg log writeout = true 2023-10-11T09:48:44.761 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.761 INFO:teuthology.orchestra.run.smithi133.stdout: osd deep scrub update digest min age = 30 2023-10-11T09:48:44.761 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.761 INFO:teuthology.orchestra.run.smithi133.stdout: osd map max advance = 10 2023-10-11T09:48:44.761 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.762 INFO:teuthology.orchestra.run.smithi133.stdout: journal zero on create = true 2023-10-11T09:48:44.762 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.762 INFO:teuthology.orchestra.run.smithi133.stdout: filestore ondisk finisher threads = 3 2023-10-11T09:48:44.762 INFO:teuthology.orchestra.run.smithi133.stdout: filestore apply finisher threads = 3 2023-10-11T09:48:44.762 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.763 INFO:teuthology.orchestra.run.smithi133.stdout: bdev debug aio = true 2023-10-11T09:48:44.763 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug misdirected ops = true 2023-10-11T09:48:44.763 INFO:teuthology.orchestra.run.smithi133.stdout: bdev async discard = True 2023-10-11T09:48:44.763 INFO:teuthology.orchestra.run.smithi133.stdout: bdev enable discard = True 2023-10-11T09:48:44.763 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore allocator = bitmap 2023-10-11T09:48:44.763 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore block size = 96636764160 2023-10-11T09:48:44.764 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore fsck on mount = True 2023-10-11T09:48:44.764 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluefs = 1/20 2023-10-11T09:48:44.764 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluestore = 1/20 2023-10-11T09:48:44.764 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:48:44.764 INFO:teuthology.orchestra.run.smithi133.stdout: debug osd = 20 2023-10-11T09:48:44.765 INFO:teuthology.orchestra.run.smithi133.stdout: debug rocksdb = 4/10 2023-10-11T09:48:44.765 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd backfillfull_ratio = 0.85 2023-10-11T09:48:44.765 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd full ratio = 0.9 2023-10-11T09:48:44.765 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd nearfull ratio = 0.8 2023-10-11T09:48:44.765 INFO:teuthology.orchestra.run.smithi133.stdout: osd failsafe full ratio = 0.95 2023-10-11T09:48:44.765 INFO:teuthology.orchestra.run.smithi133.stdout: osd objectstore = bluestore 2023-10-11T09:48:44.766 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2023-10-11T09:48:44.766 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.766 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr] 2023-10-11T09:48:44.766 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:48:44.767 INFO:teuthology.orchestra.run.smithi133.stdout: debug mgr = 20 2023-10-11T09:48:44.767 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2023-10-11T09:48:44.767 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2023-10-11T09:48:44.767 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:48:44.767 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:48:44.767 INFO:teuthology.orchestra.run.smithi133.stdout: mgr/telemetry/nag = false 2023-10-11T09:48:44.768 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.768 INFO:teuthology.orchestra.run.smithi133.stdout:[mon] 2023-10-11T09:48:44.768 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:48:44.768 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2023-10-11T09:48:44.768 INFO:teuthology.orchestra.run.smithi133.stdout: debug paxos = 20 2023-10-11T09:48:44.769 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2023-10-11T09:48:44.769 INFO:teuthology.orchestra.run.smithi133.stdout: mon data avail warn = 5 2023-10-11T09:48:44.769 INFO:teuthology.orchestra.run.smithi133.stdout: mon mgr mkfs grace = 240 2023-10-11T09:48:44.769 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:48:44.769 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd reporter subtree level = osd 2023-10-11T09:48:44.769 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd prime pg temp = true 2023-10-11T09:48:44.770 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:48:44.770 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.770 INFO:teuthology.orchestra.run.smithi133.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-11T09:48:44.770 INFO:teuthology.orchestra.run.smithi133.stdout: auth mon ticket ttl = 660 # 11m 2023-10-11T09:48:44.770 INFO:teuthology.orchestra.run.smithi133.stdout: auth service ticket ttl = 240 # 4m 2023-10-11T09:48:44.771 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.771 INFO:teuthology.orchestra.run.smithi133.stdout: # don't complain about insecure global_id in the test suite 2023-10-11T09:48:44.771 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-11T09:48:44.771 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-11T09:48:44.771 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.771 INFO:teuthology.orchestra.run.smithi133.stdout: # 1m isn't quite enough 2023-10-11T09:48:44.772 INFO:teuthology.orchestra.run.smithi133.stdout: mon_down_mkfs_grace = 2m 2023-10-11T09:48:44.772 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.772 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_filestore_osds = false 2023-10-11T09:48:44.772 INFO:teuthology.orchestra.run.smithi133.stdout: mon op complaint time = 120 2023-10-11T09:48:44.772 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:48:44.773 INFO:teuthology.orchestra.run.smithi133.stdout:[client] 2023-10-11T09:48:44.773 INFO:teuthology.orchestra.run.smithi133.stdout: rgw cache enabled = true 2023-10-11T09:48:44.773 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable ops log = true 2023-10-11T09:48:44.773 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable usage log = true 2023-10-11T09:48:44.773 INFO:teuthology.orchestra.run.smithi133.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-11T09:48:44.773 INFO:teuthology.orchestra.run.smithi133.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-11T09:48:44.774 INFO:teuthology.orchestra.run.smithi133.stdout: client mount timeout = 600 2023-10-11T09:48:44.774 INFO:teuthology.orchestra.run.smithi133.stdout: debug client = 20 2023-10-11T09:48:44.774 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:48:44.774 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = "" 2023-10-11T09:48:44.774 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2023-10-11T09:48:44.775 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2023-10-11T09:48:44.775 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.a] 2023-10-11T09:48:44.775 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.b] 2023-10-11T09:48:44.775 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.c] 2023-10-11T09:48:44.776 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.a] 2023-10-11T09:48:44.776 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.c] 2023-10-11T09:48:44.776 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.b] 2023-10-11T09:48:44.776 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.d] 2023-10-11T09:48:44.776 INFO:teuthology.orchestra.run.smithi133.stdout:[mds] 2023-10-11T09:48:44.776 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds = 20 2023-10-11T09:48:44.777 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds balancer = 20 2023-10-11T09:48:44.777 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:48:44.777 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug frag = True 2023-10-11T09:48:44.777 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug scatterstat = False 2023-10-11T09:48:44.777 INFO:teuthology.orchestra.run.smithi133.stdout: mds op complaint time = 180 2023-10-11T09:48:44.778 INFO:teuthology.orchestra.run.smithi133.stdout: mds verify scatter = False 2023-10-11T09:48:44.778 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2023-10-11T09:48:44.778 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2023-10-11T09:48:44.778 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2023-10-11T09:48:44.780 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph purge cephfs_metadata --yes-i-really-really-mean-it 2023-10-11T09:48:44.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.840+0000 7f20f00e3140 1 Processor -- start 2023-10-11T09:48:44.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.840+0000 7f20f00e3140 1 -- start start 2023-10-11T09:48:44.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.840+0000 7f20f00e3140 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627bfbc7ec0 0x5627bfc73f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:44.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.840+0000 7f20f00e3140 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5627bfbc9360 0x5627bfb99d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:44.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.840+0000 7f20f00e3140 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627bfc70a70 0x5627bfbc3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:44.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.840+0000 7f20f00e3140 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627bfb28ce0 con 0x5627bfbc7ec0 2023-10-11T09:48:44.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.840+0000 7f20f00e3140 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627bfae5250 con 0x5627bfc70a70 2023-10-11T09:48:44.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.840+0000 7f20f00e3140 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x5627bfae4ef0 con 0x5627bfbc9360 2023-10-11T09:48:44.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.844+0000 7f20edc18700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627bfbc7ec0 0x5627bfc73f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:44.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.844+0000 7f20edc18700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627bfbc7ec0 0x5627bfc73f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35376/0 (socket says 172.21.15.78:35376) 2023-10-11T09:48:44.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.844+0000 7f20edc18700 1 -- 172.21.15.78:0/740077165 learned_addr learned my addr 172.21.15.78:0/740077165 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:44.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.844+0000 7f20ed417700 1 --2- 172.21.15.78:0/740077165 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5627bfbc9360 0x5627bfb99d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:44.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.844+0000 7f20ee419700 1 --2- 172.21.15.78:0/740077165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627bfc70a70 0x5627bfbc3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:44.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.844+0000 7f20edc18700 1 -- 172.21.15.78:0/740077165 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5627bfbc9360 msgr2=0x5627bfb99d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:44.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.844+0000 7f20edc18700 1 --2- 172.21.15.78:0/740077165 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5627bfbc9360 0x5627bfb99d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:44.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.844+0000 7f20edc18700 1 -- 172.21.15.78:0/740077165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627bfc70a70 msgr2=0x5627bfbc3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:44.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.844+0000 7f20edc18700 1 --2- 172.21.15.78:0/740077165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627bfc70a70 0x5627bfbc3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:44.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.844+0000 7f20edc18700 1 -- 172.21.15.78:0/740077165 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d800a1c0 con 0x5627bfbc7ec0 2023-10-11T09:48:44.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.844+0000 7f20ee419700 1 --2- 172.21.15.78:0/740077165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627bfc70a70 0x5627bfbc3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:48:44.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.844+0000 7f20edc18700 1 --2- 172.21.15.78:0/740077165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627bfbc7ec0 0x5627bfc73f70 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f20e0001e90 tx=0x7f20e000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d29a6995e4b667f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:44.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.844+0000 7f20e7fff700 1 -- 172.21.15.78:0/740077165 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e000f5a0 con 0x5627bfbc7ec0 2023-10-11T09:48:44.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.844+0000 7f20e7fff700 1 -- 172.21.15.78:0/740077165 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e000a040 con 0x5627bfbc7ec0 2023-10-11T09:48:44.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.844+0000 7f20e7fff700 1 -- 172.21.15.78:0/740077165 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e000f940 con 0x5627bfbc7ec0 2023-10-11T09:48:44.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.844+0000 7f20f00e3140 1 -- 172.21.15.78:0/740077165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627bfbc7ec0 msgr2=0x5627bfc73f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:44.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.844+0000 7f20f00e3140 1 --2- 172.21.15.78:0/740077165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627bfbc7ec0 0x5627bfc73f70 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f20e0001e90 tx=0x7f20e000b4a0 comp rx=0 tx=0).stop 2023-10-11T09:48:44.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.844+0000 7f20f00e3140 1 -- 172.21.15.78:0/740077165 shutdown_connections 2023-10-11T09:48:44.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.844+0000 7f20f00e3140 1 --2- 172.21.15.78:0/740077165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627bfc70a70 0x5627bfbc3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:44.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.844+0000 7f20f00e3140 1 --2- 172.21.15.78:0/740077165 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5627bfbc9360 0x5627bfb99d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:44.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.844+0000 7f20f00e3140 1 --2- 172.21.15.78:0/740077165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627bfbc7ec0 0x5627bfc73f70 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:44.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.844+0000 7f20f00e3140 1 -- 172.21.15.78:0/740077165 >> 172.21.15.78:0/740077165 conn(0x5627bfb9abe0 msgr2=0x5627bfc73860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:44.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.844+0000 7f20f00e3140 1 -- 172.21.15.78:0/740077165 shutdown_connections 2023-10-11T09:48:44.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.844+0000 7f20f00e3140 1 -- 172.21.15.78:0/740077165 wait complete. 2023-10-11T09:48:44.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.848+0000 7f20f00e3140 1 Processor -- start 2023-10-11T09:48:44.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.848+0000 7f20f00e3140 1 -- start start 2023-10-11T09:48:44.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.848+0000 7f20f00e3140 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5627bfbc7ec0 0x5627bfc61f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:44.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.848+0000 7f20f00e3140 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627bfbc9360 0x5627bfbdf320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:44.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.848+0000 7f20f00e3140 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627bfc70a70 0x5627bfc6fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:44.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.848+0000 7f20f00e3140 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627bfabb240 con 0x5627bfbc9360 2023-10-11T09:48:44.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.848+0000 7f20f00e3140 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627bfc5efb0 con 0x5627bfc70a70 2023-10-11T09:48:44.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.848+0000 7f20f00e3140 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x5627bfc5f250 con 0x5627bfbc7ec0 2023-10-11T09:48:44.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.848+0000 7f20ed417700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627bfbc9360 0x5627bfbdf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:44.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.848+0000 7f20ed417700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627bfbc9360 0x5627bfbdf320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35392/0 (socket says 172.21.15.78:35392) 2023-10-11T09:48:44.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.848+0000 7f20ed417700 1 -- 172.21.15.78:0/1295674937 learned_addr learned my addr 172.21.15.78:0/1295674937 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:44.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.848+0000 7f20edc18700 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5627bfbc7ec0 0x5627bfc61f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:44.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.848+0000 7f20ee419700 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627bfc70a70 0x5627bfc6fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:44.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.848+0000 7f20ed417700 1 -- 172.21.15.78:0/1295674937 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5627bfbc7ec0 msgr2=0x5627bfc61f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:44.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.848+0000 7f20ed417700 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5627bfbc7ec0 0x5627bfc61f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:44.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.848+0000 7f20ed417700 1 -- 172.21.15.78:0/1295674937 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627bfc70a70 msgr2=0x5627bfc6fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:44.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.848+0000 7f20ed417700 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627bfc70a70 0x5627bfc6fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:44.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.848+0000 7f20ed417700 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e0001a90 con 0x5627bfbc9360 2023-10-11T09:48:44.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.848+0000 7f20ed417700 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627bfbc9360 0x5627bfbdf320 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f20d8000f80 tx=0x7f20d8003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:44.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.848+0000 7f20e6ffd700 1 -- 172.21.15.78:0/1295674937 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d8010a90 con 0x5627bfbc9360 2023-10-11T09:48:44.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.848+0000 7f20e6ffd700 1 -- 172.21.15.78:0/1295674937 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d8009040 con 0x5627bfbc9360 2023-10-11T09:48:44.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.848+0000 7f20e6ffd700 1 -- 172.21.15.78:0/1295674937 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d8010e30 con 0x5627bfbc9360 2023-10-11T09:48:44.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.848+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5627bfac68b0 con 0x5627bfbc9360 2023-10-11T09:48:44.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.848+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5627bfac6310 con 0x5627bfbc9360 2023-10-11T09:48:44.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.852+0000 7f20e6ffd700 1 -- 172.21.15.78:0/1295674937 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f20d8017030 con 0x5627bfbc9360 2023-10-11T09:48:44.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.852+0000 7f20e6ffd700 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f20d0064830 0x7f20d0066ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:44.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.856+0000 7f20e6ffd700 1 -- 172.21.15.78:0/1295674937 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6552+0+0 (secure 0 0 0) 0x7f20d8084070 con 0x5627bfbc9360 2023-10-11T09:48:44.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.856+0000 7f20edc18700 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f20d0064830 0x7f20d0066ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:44.868 INFO:teuthology.orchestra.run.smithi078.stdout:Warning: using slow linear search 2023-10-11T09:48:44.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.856+0000 7f20f00e3140 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x5627bfc6ff50 0x5627bfc5fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:44.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.856+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:1 11.0 11:00000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x5627bfc5eae0 con 0x5627bfc6ff50 2023-10-11T09:48:44.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.856+0000 7f20ee419700 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x5627bfc6ff50 0x5627bfc5fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:44.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.856+0000 7f20edc18700 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f20d0064830 0x7f20d0066ce0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f20e000b4e0 tx=0x7f20e0002000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:44.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.856+0000 7f20ee419700 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x5627bfc6ff50 0x5627bfc5fed0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:44.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.856+0000 7f20ee419700 1 -- 172.21.15.78:0/1295674937 <== osd.1 v2:172.21.15.78:6809/35734 1 ==== osd_op_reply(1 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f20e800e5b0 con 0x5627bfc6ff50 2023-10-11T09:48:44.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.856+0000 7f20ee419700 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x7f20e80108f0 0x7f20e800f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:44.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.856+0000 7f20ee419700 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:2 11.10 11:08000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x5627bfc5eae0 con 0x7f20e80108f0 2023-10-11T09:48:44.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.856+0000 7f20edc18700 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x7f20e80108f0 0x7f20e800f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:44.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.856+0000 7f20edc18700 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x7f20e80108f0 0x7f20e800f370 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:44.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.860+0000 7f20edc18700 1 -- 172.21.15.78:0/1295674937 <== osd.0 v2:172.21.15.78:6800/35735 1 ==== osd_op_reply(2 [pgnls start_epoch 87 out=177b] v86'10 uv10 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x7f20e0018070 con 0x7f20e80108f0 2023-10-11T09:48:44.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.860+0000 7f20edc18700 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x7f20e0010070 0x7f20e0010950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:44.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.860+0000 7f20edc18700 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:3 11.8 11:10000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x5627bfc5eae0 con 0x7f20e0010070 2023-10-11T09:48:44.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.860+0000 7f20ed417700 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x7f20e0010070 0x7f20e0010950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:44.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.860+0000 7f20ed417700 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x7f20e0010070 0x7f20e0010950 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:44.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.860+0000 7f20ed417700 1 -- 172.21.15.78:0/1295674937 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(3 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f20d80822c0 con 0x7f20e0010070 2023-10-11T09:48:44.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.860+0000 7f20ed417700 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x7f20d8010070 0x7f20d8055b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:44.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.860+0000 7f20ed417700 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:4 11.18 11:18000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x5627bfc5eae0 con 0x7f20d8010070 2023-10-11T09:48:44.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.860+0000 7f20ee419700 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x7f20d8010070 0x7f20d8055b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:44.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.860+0000 7f20ee419700 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x7f20d8010070 0x7f20d8055b10 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:44.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.864+0000 7f20ee419700 1 -- 172.21.15.78:0/1295674937 <== osd.2 v2:172.21.15.78:6808/35732 1 ==== osd_op_reply(4 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f20e800e5b0 con 0x7f20d8010070 2023-10-11T09:48:44.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.864+0000 7f20ee419700 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x7f20e8013970 0x7f20e800a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:44.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.864+0000 7f20ee419700 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:5 11.4 11:20000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x5627bfc5eae0 con 0x7f20e8013970 2023-10-11T09:48:44.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.864+0000 7f20edc18700 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x7f20e8013970 0x7f20e800a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:44.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.864+0000 7f20edc18700 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x7f20e8013970 0x7f20e800a2c0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:44.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.864+0000 7f20edc18700 1 -- 172.21.15.78:0/1295674937 <== osd.7 v2:172.21.15.133:6824/35901 1 ==== osd_op_reply(5 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f20e0018070 con 0x7f20e8013970 2023-10-11T09:48:44.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.864+0000 7f20edc18700 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x7f20e0028b30 0x7f20e001d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:44.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.864+0000 7f20edc18700 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:6 11.14 11:28000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x5627bfc5eae0 con 0x7f20e0028b30 2023-10-11T09:48:44.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.868+0000 7f20ed417700 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x7f20e0028b30 0x7f20e001d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:44.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.868+0000 7f20ed417700 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x7f20e0028b30 0x7f20e001d910 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:44.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.868+0000 7f20ed417700 1 -- 172.21.15.78:0/1295674937 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== osd_op_reply(6 [pgnls start_epoch 87 out=149b] v86'21 uv21 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x7f20d80822c0 con 0x7f20e0028b30 2023-10-11T09:48:44.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.868+0000 7f20ed417700 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:7 11.c 11:30000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x5627bfc5eae0 con 0x5627bfc6ff50 2023-10-11T09:48:44.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.872+0000 7f20ee419700 1 -- 172.21.15.78:0/1295674937 <== osd.1 v2:172.21.15.78:6809/35734 2 ==== osd_op_reply(7 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f20e800e5b0 con 0x5627bfc6ff50 2023-10-11T09:48:44.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.872+0000 7f20ee419700 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:8 11.1c 11:38000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x5627bfc5eae0 con 0x7f20e0028b30 2023-10-11T09:48:44.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.872+0000 7f20ed417700 1 -- 172.21.15.78:0/1295674937 <== osd.6 v2:172.21.15.133:6800/35900 2 ==== osd_op_reply(8 [pgnls start_epoch 87 out=73b] v82'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f20d80822c0 con 0x7f20e0028b30 2023-10-11T09:48:44.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.872+0000 7f20ed417700 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:9 11.2 11:40000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x5627bfc5eae0 con 0x7f20d8010070 2023-10-11T09:48:44.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.872+0000 7f20ee419700 1 -- 172.21.15.78:0/1295674937 <== osd.2 v2:172.21.15.78:6808/35732 2 ==== osd_op_reply(9 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f20e800e5b0 con 0x7f20d8010070 2023-10-11T09:48:44.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.872+0000 7f20ee419700 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:10 11.12 11:48000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x5627bfc5eae0 con 0x7f20e8013970 2023-10-11T09:48:44.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.872+0000 7f20edc18700 1 -- 172.21.15.78:0/1295674937 <== osd.7 v2:172.21.15.133:6824/35901 2 ==== osd_op_reply(10 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f20e0018070 con 0x7f20e8013970 2023-10-11T09:48:44.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.872+0000 7f20edc18700 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:11 11.a 11:50000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x5627bfc5eae0 con 0x7f20e0010070 2023-10-11T09:48:44.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.872+0000 7f20ed417700 1 -- 172.21.15.78:0/1295674937 <== osd.3 v2:172.21.15.78:6824/35738 2 ==== osd_op_reply(11 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f20d80822c0 con 0x7f20e0010070 2023-10-11T09:48:44.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.872+0000 7f20ed417700 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x7f20d80553e0 0x7f20d806c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:44.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.872+0000 7f20ed417700 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:12 11.1a 11:58000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x5627bfc5eae0 con 0x7f20d80553e0 2023-10-11T09:48:44.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.876+0000 7f20ee419700 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x7f20d80553e0 0x7f20d806c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:44.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.876+0000 7f20ee419700 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x7f20d80553e0 0x7f20d806c980 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:44.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.876+0000 7f20ee419700 1 -- 172.21.15.78:0/1295674937 <== osd.5 v2:172.21.15.133:6816/35899 1 ==== osd_op_reply(12 [pgnls start_epoch 87 out=49b] v82'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f20e800e5b0 con 0x7f20d80553e0 2023-10-11T09:48:44.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.876+0000 7f20ee419700 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:13 11.6 11:60000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x5627bfc5eae0 con 0x7f20e80108f0 2023-10-11T09:48:44.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.876+0000 7f20edc18700 1 -- 172.21.15.78:0/1295674937 <== osd.0 v2:172.21.15.78:6800/35735 2 ==== osd_op_reply(13 [pgnls start_epoch 87 out=77b] v86'4 uv4 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x7f20e0018070 con 0x7f20e80108f0 2023-10-11T09:48:44.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.876+0000 7f20edc18700 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:14 11.16 11:68000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x5627bfc5eae0 con 0x7f20d80553e0 2023-10-11T09:48:44.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.876+0000 7f20ee419700 1 -- 172.21.15.78:0/1295674937 <== osd.5 v2:172.21.15.133:6816/35899 2 ==== osd_op_reply(14 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f20e800e5b0 con 0x7f20d80553e0 2023-10-11T09:48:44.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.876+0000 7f20ee419700 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x7f20e8029910 0x7f20e80273e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:44.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.880+0000 7f20ee419700 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:15 11.e 11:70000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x5627bfc5eae0 con 0x7f20e8029910 2023-10-11T09:48:44.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.880+0000 7f20edc18700 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x7f20e8029910 0x7f20e80273e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:44.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.880+0000 7f20edc18700 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x7f20e8029910 0x7f20e80273e0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:44.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.880+0000 7f20edc18700 1 -- 172.21.15.78:0/1295674937 <== osd.4 v2:172.21.15.133:6808/35898 1 ==== osd_op_reply(15 [pgnls start_epoch 87 out=73b] v82'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f20e0018070 con 0x7f20e8029910 2023-10-11T09:48:44.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.880+0000 7f20edc18700 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:16 11.1e 11:78000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x5627bfc5eae0 con 0x7f20e8013970 2023-10-11T09:48:44.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.884+0000 7f20edc18700 1 -- 172.21.15.78:0/1295674937 <== osd.7 v2:172.21.15.133:6824/35901 3 ==== osd_op_reply(16 [pgnls start_epoch 87 out=49b] v82'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f20e0018070 con 0x7f20e8013970 2023-10-11T09:48:44.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.884+0000 7f20edc18700 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:17 11.1 11:80000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x5627bfc5eae0 con 0x7f20e0028b30 2023-10-11T09:48:44.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.884+0000 7f20ed417700 1 -- 172.21.15.78:0/1295674937 <== osd.6 v2:172.21.15.133:6800/35900 3 ==== osd_op_reply(17 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f20d80822c0 con 0x7f20e0028b30 2023-10-11T09:48:44.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.884+0000 7f20ed417700 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:18 11.11 11:88000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x5627bfc5eae0 con 0x7f20d8010070 2023-10-11T09:48:44.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.884+0000 7f20ee419700 1 -- 172.21.15.78:0/1295674937 <== osd.2 v2:172.21.15.78:6808/35732 3 ==== osd_op_reply(18 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f20e800e5b0 con 0x7f20d8010070 2023-10-11T09:48:44.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.884+0000 7f20ee419700 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:19 11.9 11:90000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x5627bfc5eae0 con 0x7f20e0010070 2023-10-11T09:48:44.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.884+0000 7f20ed417700 1 -- 172.21.15.78:0/1295674937 <== osd.3 v2:172.21.15.78:6824/35738 3 ==== osd_op_reply(19 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f20d80822c0 con 0x7f20e0010070 2023-10-11T09:48:44.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.884+0000 7f20ed417700 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:20 11.19 11:98000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x5627bfc5eae0 con 0x7f20e80108f0 2023-10-11T09:48:44.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.884+0000 7f20edc18700 1 -- 172.21.15.78:0/1295674937 <== osd.0 v2:172.21.15.78:6800/35735 3 ==== osd_op_reply(20 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f20e0018070 con 0x7f20e80108f0 2023-10-11T09:48:44.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.884+0000 7f20edc18700 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:21 11.5 11:a0000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x5627bfc5eae0 con 0x7f20e8013970 2023-10-11T09:48:44.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.888+0000 7f20edc18700 1 -- 172.21.15.78:0/1295674937 <== osd.7 v2:172.21.15.133:6824/35901 4 ==== osd_op_reply(21 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f20e0018070 con 0x7f20e8013970 2023-10-11T09:48:44.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.888+0000 7f20edc18700 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:22 11.15 11:a8000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x5627bfc5eae0 con 0x7f20d8010070 2023-10-11T09:48:44.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.888+0000 7f20ee419700 1 -- 172.21.15.78:0/1295674937 <== osd.2 v2:172.21.15.78:6808/35732 4 ==== osd_op_reply(22 [pgnls start_epoch 87 out=97b] v82'3 uv3 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x7f20e800e5b0 con 0x7f20d8010070 2023-10-11T09:48:44.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.888+0000 7f20ee419700 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:23 11.d 11:b0000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x5627bfc5eae0 con 0x7f20d8010070 2023-10-11T09:48:44.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.888+0000 7f20ee419700 1 -- 172.21.15.78:0/1295674937 <== osd.2 v2:172.21.15.78:6808/35732 5 ==== osd_op_reply(23 [pgnls start_epoch 87 out=74b] v82'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f20e800e5b0 con 0x7f20d8010070 2023-10-11T09:48:44.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.888+0000 7f20ee419700 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:24 11.1d 11:b8000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x5627bfc5eae0 con 0x7f20e8013970 2023-10-11T09:48:44.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.888+0000 7f20edc18700 1 -- 172.21.15.78:0/1295674937 <== osd.7 v2:172.21.15.133:6824/35901 5 ==== osd_op_reply(24 [pgnls start_epoch 87 out=73b] v82'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f20e0018070 con 0x7f20e8013970 2023-10-11T09:48:44.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.888+0000 7f20edc18700 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:25 11.3 11:c0000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x5627bfc5eae0 con 0x7f20e0028b30 2023-10-11T09:48:44.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.892+0000 7f20ed417700 1 -- 172.21.15.78:0/1295674937 <== osd.6 v2:172.21.15.133:6800/35900 4 ==== osd_op_reply(25 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f20d80822c0 con 0x7f20e0028b30 2023-10-11T09:48:44.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.892+0000 7f20ed417700 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:26 11.13 11:c8000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x5627bfc5eae0 con 0x7f20d80553e0 2023-10-11T09:48:44.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.892+0000 7f20ee419700 1 -- 172.21.15.78:0/1295674937 <== osd.5 v2:172.21.15.133:6816/35899 3 ==== osd_op_reply(26 [pgnls start_epoch 87 out=98b] v86'3 uv3 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x7f20e800e5b0 con 0x7f20d80553e0 2023-10-11T09:48:44.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.892+0000 7f20ee419700 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:27 11.b 11:d0000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x5627bfc5eae0 con 0x7f20e0010070 2023-10-11T09:48:44.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.892+0000 7f20ed417700 1 -- 172.21.15.78:0/1295674937 <== osd.3 v2:172.21.15.78:6824/35738 4 ==== osd_op_reply(27 [pgnls start_epoch 87 out=76b] v86'3 uv3 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x7f20d80822c0 con 0x7f20e0010070 2023-10-11T09:48:44.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.892+0000 7f20ed417700 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:28 11.1b 11:d8000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x5627bfc5eae0 con 0x7f20e8029910 2023-10-11T09:48:44.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.892+0000 7f20edc18700 1 -- 172.21.15.78:0/1295674937 <== osd.4 v2:172.21.15.133:6808/35898 2 ==== osd_op_reply(28 [pgnls start_epoch 87 out=49b] v82'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f20e0018070 con 0x7f20e8029910 2023-10-11T09:48:44.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.892+0000 7f20edc18700 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:29 11.7 11:e0000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x5627bfc5eae0 con 0x7f20d80553e0 2023-10-11T09:48:44.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.896+0000 7f20ee419700 1 -- 172.21.15.78:0/1295674937 <== osd.5 v2:172.21.15.133:6816/35899 4 ==== osd_op_reply(29 [pgnls start_epoch 87 out=73b] v82'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f20e800e5b0 con 0x7f20d80553e0 2023-10-11T09:48:44.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.896+0000 7f20ee419700 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:30 11.17 11:e8000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x5627bfc5eae0 con 0x7f20e8013970 2023-10-11T09:48:44.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.896+0000 7f20edc18700 1 -- 172.21.15.78:0/1295674937 <== osd.7 v2:172.21.15.133:6824/35901 6 ==== osd_op_reply(30 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f20e0018070 con 0x7f20e8013970 2023-10-11T09:48:44.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.896+0000 7f20edc18700 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:31 11.f 11:f0000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x5627bfc5eae0 con 0x7f20e8013970 2023-10-11T09:48:44.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.896+0000 7f20edc18700 1 -- 172.21.15.78:0/1295674937 <== osd.7 v2:172.21.15.133:6824/35901 7 ==== osd_op_reply(31 [pgnls start_epoch 87 out=73b] v82'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f20e004b040 con 0x7f20e8013970 2023-10-11T09:48:44.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.896+0000 7f20edc18700 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:32 11.1f 11:f8000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x5627bfc5eae0 con 0x7f20d8010070 2023-10-11T09:48:44.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.900+0000 7f20ee419700 1 -- 172.21.15.78:0/1295674937 <== osd.2 v2:172.21.15.78:6808/35732 6 ==== osd_op_reply(32 [pgnls start_epoch 87 out=101b] v86'6 uv6 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x7f20e800e5b0 con 0x7f20d8010070 2023-10-11T09:48:44.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.900+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:33 11.10 11:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x5627bfc60650 con 0x7f20e80108f0 2023-10-11T09:48:44.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.900+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:34 11.10 11:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x5627bfc5de20 con 0x7f20e80108f0 2023-10-11T09:48:44.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.900+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:35 11.10 11:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x5627bfd13640 con 0x7f20e80108f0 2023-10-11T09:48:44.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.900+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:36 11.10 11:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x5627bfd13ee0 con 0x7f20e80108f0 2023-10-11T09:48:44.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.900+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:37 11.10 11:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x5627bfd14780 con 0x7f20e80108f0 2023-10-11T09:48:44.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.900+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:38 11.14 11:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x5627bfd15020 con 0x7f20e0028b30 2023-10-11T09:48:44.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.900+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:39 11.14 11:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x5627bfd158c0 con 0x7f20e0028b30 2023-10-11T09:48:44.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.900+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:40 11.14 11:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x5627bfd16160 con 0x7f20e0028b30 2023-10-11T09:48:44.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.900+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:41 11.14 11:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x5627bfd16a00 con 0x7f20e0028b30 2023-10-11T09:48:44.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.900+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:42 11.1c 11:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x5627bfd172a0 con 0x7f20e0028b30 2023-10-11T09:48:44.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.900+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:43 11.6 11:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x5627bfd17b40 con 0x7f20e80108f0 2023-10-11T09:48:44.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.900+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:44 11.e 11:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x5627bfd18460 con 0x7f20e8029910 2023-10-11T09:48:44.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.900+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:45 11.15 11:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x5627bfd18fa0 con 0x7f20d8010070 2023-10-11T09:48:44.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.900+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:46 11.15 11:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x5627bfd19ae0 con 0x7f20d8010070 2023-10-11T09:48:44.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.900+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:47 11.d 11:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x5627bfd1a600 con 0x7f20d8010070 2023-10-11T09:48:44.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.900+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:48 11.1d 11:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x5627bfd1b140 con 0x7f20e8013970 2023-10-11T09:48:44.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.904+0000 7f20ee419700 1 -- 172.21.15.78:0/1295674937 <== osd.2 v2:172.21.15.78:6808/35732 7 ==== osd_op_reply(45 604.00000000 [delete] v87'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f20e800e5b0 con 0x7f20d8010070 2023-10-11T09:48:44.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.904+0000 7f20edc18700 1 -- 172.21.15.78:0/1295674937 <== osd.0 v2:172.21.15.78:6800/35735 4 ==== osd_op_reply(43 mds0_openfiles.0 [delete] v87'5 uv5 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f20e004b040 con 0x7f20e80108f0 2023-10-11T09:48:44.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.904+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:49 11.13 11:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x5627bfd18a50 con 0x7f20d80553e0 2023-10-11T09:48:44.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.904+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:50 11.13 11:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x5627bfd1b650 con 0x7f20d80553e0 2023-10-11T09:48:44.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.904+0000 7f20ed417700 1 -- 172.21.15.78:0/1295674937 <== osd.6 v2:172.21.15.133:6800/35900 5 ==== osd_op_reply(38 1.00000000.inode [delete] v87'22 uv22 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f20d80822c0 con 0x7f20e0028b30 2023-10-11T09:48:44.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.904+0000 7f20edc18700 1 -- 172.21.15.78:0/1295674937 <== osd.0 v2:172.21.15.78:6800/35735 5 ==== osd_op_reply(33 601.00000000 [delete] v87'11 uv11 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f20e004b040 con 0x7f20e80108f0 2023-10-11T09:48:44.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.904+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:51 11.b 11:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x5627bfd1bb00 con 0x7f20e0010070 2023-10-11T09:48:44.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.904+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:52 11.7 11:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x5627bfd106b0 con 0x7f20d80553e0 2023-10-11T09:48:44.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.904+0000 7f20edc18700 1 -- 172.21.15.78:0/1295674937 <== osd.4 v2:172.21.15.133:6808/35898 3 ==== osd_op_reply(44 608.00000000 [delete] v87'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f20e004b040 con 0x7f20e8029910 2023-10-11T09:48:44.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.908+0000 7f20ed417700 1 -- 172.21.15.78:0/1295674937 <== osd.6 v2:172.21.15.133:6800/35900 6 ==== osd_op_reply(39 200.00000000 [delete] v87'23 uv23 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f20d80822c0 con 0x7f20e0028b30 2023-10-11T09:48:44.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.908+0000 7f20ee419700 1 -- 172.21.15.78:0/1295674937 <== osd.2 v2:172.21.15.78:6808/35732 8 ==== osd_op_reply(47 mds_snaptable [delete] v87'2 uv2 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f20e800e5b0 con 0x7f20d8010070 2023-10-11T09:48:44.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.908+0000 7f20edc18700 1 -- 172.21.15.78:0/1295674937 <== osd.7 v2:172.21.15.133:6824/35901 8 ==== osd_op_reply(48 605.00000000 [delete] v87'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f20e004b040 con 0x7f20e8013970 2023-10-11T09:48:44.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.908+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:53 11.f 11:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x5627bfd106b0 con 0x7f20e8013970 2023-10-11T09:48:44.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.908+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:54 11.1f 11:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x5627bfd1a060 con 0x7f20d8010070 2023-10-11T09:48:44.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.908+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:55 11.1f 11:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x5627bfd1c3a0 con 0x7f20d8010070 2023-10-11T09:48:44.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.908+0000 7f20edc18700 1 -- 172.21.15.78:0/1295674937 <== osd.0 v2:172.21.15.78:6800/35735 6 ==== osd_op_reply(34 602.00000000 [delete] v87'12 uv12 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f20e004b040 con 0x7f20e80108f0 2023-10-11T09:48:44.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.908+0000 7f20ed417700 1 -- 172.21.15.78:0/1295674937 <== osd.6 v2:172.21.15.133:6800/35900 7 ==== osd_op_reply(42 607.00000000 [delete] v87'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f20d80822c0 con 0x7f20e0028b30 2023-10-11T09:48:44.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.908+0000 7f20ee419700 1 -- 172.21.15.78:0/1295674937 <== osd.5 v2:172.21.15.133:6816/35899 5 ==== osd_op_reply(49 mds0_inotable [delete] v87'4 uv4 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f20e800e5b0 con 0x7f20d80553e0 2023-10-11T09:48:44.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.908+0000 7f20ed417700 1 -- 172.21.15.78:0/1295674937 <== osd.6 v2:172.21.15.133:6800/35900 8 ==== osd_op_reply(40 200.00000001 [delete] v87'24 uv24 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f20d80822c0 con 0x7f20e0028b30 2023-10-11T09:48:44.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.908+0000 7f20edc18700 1 -- 172.21.15.78:0/1295674937 <== osd.0 v2:172.21.15.78:6800/35735 7 ==== osd_op_reply(35 10000000000.00000000 [delete] v87'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f20e004b040 con 0x7f20e80108f0 2023-10-11T09:48:44.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.908+0000 7f20edc18700 1 -- 172.21.15.78:0/1295674937 <== osd.0 v2:172.21.15.78:6800/35735 8 ==== osd_op_reply(36 600.00000000 [delete] v87'14 uv14 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f20e004b040 con 0x7f20e80108f0 2023-10-11T09:48:44.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.908+0000 7f20edc18700 1 -- 172.21.15.78:0/1295674937 <== osd.0 v2:172.21.15.78:6800/35735 9 ==== osd_op_reply(37 603.00000000 [delete] v87'15 uv15 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f20e004b040 con 0x7f20e80108f0 2023-10-11T09:48:44.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.908+0000 7f20ed417700 1 -- 172.21.15.78:0/1295674937 <== osd.6 v2:172.21.15.133:6800/35900 9 ==== osd_op_reply(41 606.00000000 [delete] v87'25 uv25 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f20d80822c0 con 0x7f20e0028b30 2023-10-11T09:48:44.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.908+0000 7f20ee419700 1 -- 172.21.15.78:0/1295674937 <== osd.5 v2:172.21.15.133:6816/35899 6 ==== osd_op_reply(50 100.00000000 [delete] v87'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f20e800e5b0 con 0x7f20d80553e0 2023-10-11T09:48:44.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.908+0000 7f20ee419700 1 -- 172.21.15.78:0/1295674937 <== osd.2 v2:172.21.15.78:6808/35732 9 ==== osd_op_reply(46 500.00000000 [delete] v87'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f20e800e5b0 con 0x7f20d8010070 2023-10-11T09:48:44.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.912+0000 7f20ee419700 1 -- 172.21.15.78:0/1295674937 <== osd.2 v2:172.21.15.78:6808/35732 10 ==== osd_op_reply(54 100.00000000.inode [delete] v87'7 uv7 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f20e800e5b0 con 0x7f20d8010070 2023-10-11T09:48:44.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.912+0000 7f20ed417700 1 -- 172.21.15.78:0/1295674937 <== osd.3 v2:172.21.15.78:6824/35738 5 ==== osd_op_reply(51 mds0_sessionmap [delete] v87'4 uv4 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f20d80822c0 con 0x7f20e0010070 2023-10-11T09:48:44.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.912+0000 7f20ee419700 1 -- 172.21.15.78:0/1295674937 <== osd.5 v2:172.21.15.133:6816/35899 7 ==== osd_op_reply(52 609.00000000 [delete] v87'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f20e800e5b0 con 0x7f20d80553e0 2023-10-11T09:48:44.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.912+0000 7f20ee419700 1 -- 172.21.15.78:0/1295674937 <== osd.2 v2:172.21.15.78:6808/35732 11 ==== osd_op_reply(55 1.00000000 [delete] v87'8 uv8 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f20e800e5b0 con 0x7f20d8010070 2023-10-11T09:48:44.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.912+0000 7f20edc18700 1 -- 172.21.15.78:0/1295674937 <== osd.7 v2:172.21.15.133:6824/35901 9 ==== osd_op_reply(53 400.00000000 [delete] v87'2 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f20e004b040 con 0x7f20e8013970 2023-10-11T09:48:44.925 INFO:teuthology.orchestra.run.smithi078.stdout:Removed 23 objects 2023-10-11T09:48:44.925 INFO:teuthology.orchestra.run.smithi078.stdout:successfully purged pool cephfs_metadata 2023-10-11T09:48:44.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.912+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x7f20e80108f0 msgr2=0x7f20e800f370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:44.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.912+0000 7f20f00e3140 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x7f20e80108f0 0x7f20e800f370 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:44.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.912+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x5627bfc6ff50 msgr2=0x5627bfc5fed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:44.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.912+0000 7f20f00e3140 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x5627bfc6ff50 0x5627bfc5fed0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:44.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.912+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x7f20d8010070 msgr2=0x7f20d8055b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:44.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.912+0000 7f20f00e3140 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x7f20d8010070 0x7f20d8055b10 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:44.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.912+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x7f20e0010070 msgr2=0x7f20e0010950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:44.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.912+0000 7f20f00e3140 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x7f20e0010070 0x7f20e0010950 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:44.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.912+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x7f20e8029910 msgr2=0x7f20e80273e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:44.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.912+0000 7f20f00e3140 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x7f20e8029910 0x7f20e80273e0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:44.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.912+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x7f20d80553e0 msgr2=0x7f20d806c980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:44.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.912+0000 7f20f00e3140 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x7f20d80553e0 0x7f20d806c980 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:44.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.912+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x7f20e0028b30 msgr2=0x7f20e001d910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:44.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.912+0000 7f20f00e3140 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x7f20e0028b30 0x7f20e001d910 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:44.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.912+0000 7f20ee419700 1 -- 172.21.15.78:0/1295674937 reap_dead start 2023-10-11T09:48:44.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.912+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x7f20e8013970 msgr2=0x7f20e800a2c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:44.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.912+0000 7f20f00e3140 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x7f20e8013970 0x7f20e800a2c0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:44.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.916+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f20d0064830 msgr2=0x7f20d0066ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:44.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.916+0000 7f20ee419700 1 -- 172.21.15.78:0/1295674937 reap_dead start 2023-10-11T09:48:44.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.916+0000 7f20f00e3140 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f20d0064830 0x7f20d0066ce0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f20e000b4e0 tx=0x7f20e0002000 comp rx=0 tx=0).stop 2023-10-11T09:48:44.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.916+0000 7f20ee419700 1 -- 172.21.15.78:0/1295674937 reap_dead start 2023-10-11T09:48:44.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.916+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627bfbc9360 msgr2=0x5627bfbdf320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:44.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.916+0000 7f20f00e3140 1 --2- 172.21.15.78:0/1295674937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627bfbc9360 0x5627bfbdf320 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f20d8000f80 tx=0x7f20d8003330 comp rx=0 tx=0).stop 2023-10-11T09:48:44.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.916+0000 7f20ee419700 1 -- 172.21.15.78:0/1295674937 reap_dead start 2023-10-11T09:48:44.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.916+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 shutdown_connections 2023-10-11T09:48:44.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.916+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 >> 172.21.15.78:0/1295674937 conn(0x5627bfb9abe0 msgr2=0x5627bfbe29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:44.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.916+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 shutdown_connections 2023-10-11T09:48:44.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:44.916+0000 7f20f00e3140 1 -- 172.21.15.78:0/1295674937 wait complete. 2023-10-11T09:48:44.933 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs reset cephfs --yes-i-really-mean-it 2023-10-11T09:48:45.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.108+0000 7f95690bf700 1 Processor -- start 2023-10-11T09:48:45.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.108+0000 7f95690bf700 1 -- start start 2023-10-11T09:48:45.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.108+0000 7f95690bf700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9564158430 0x7f95640710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:45.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.108+0000 7f95690bf700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9564071610 con 0x7f956406d340 2023-10-11T09:48:45.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.108+0000 7f95690bf700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f956415e280 con 0x7f95640caf90 2023-10-11T09:48:45.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.108+0000 7f95690bf700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9564070c10 con 0x7f9564158430 2023-10-11T09:48:45.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.108+0000 7f956259c700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f956406d340 0x7f95640ca880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:54472/0 (socket says 172.21.15.78:54472) 2023-10-11T09:48:45.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.108+0000 7f956259c700 1 -- 172.21.15.78:0/3985135956 learned_addr learned my addr 172.21.15.78:0/3985135956 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:45.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.108+0000 7f9561d9b700 1 -- 172.21.15.78:0/3985135956 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2745722743 0 0) 0x7f9564071610 con 0x7f956406d340 2023-10-11T09:48:45.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.108+0000 7f9562d9d700 1 --2- 172.21.15.78:0/3985135956 >> v2:172.21.15.133:3300/0 conn(0x7f9564158430 0x7f95640710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:45.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.108+0000 7f9561d9b700 1 -- 172.21.15.78:0/3985135956 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f954c0035d0 con 0x7f956406d340 2023-10-11T09:48:45.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.108+0000 7f9561d9b700 1 -- 172.21.15.78:0/3985135956 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2364650824 0 0) 0x7f954c0035d0 con 0x7f956406d340 2023-10-11T09:48:45.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.108+0000 7f9561d9b700 1 -- 172.21.15.78:0/3985135956 >> v2:172.21.15.133:3300/0 conn(0x7f9564158430 msgr2=0x7f95640710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:45.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.108+0000 7f9561d9b700 1 --2- 172.21.15.78:0/3985135956 >> v2:172.21.15.133:3300/0 conn(0x7f9564158430 0x7f95640710d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:45.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.108+0000 7f9561d9b700 1 -- 172.21.15.78:0/3985135956 >> v1:172.21.15.133:6789/0 conn(0x7f95640caf90 legacy=0x7f956415afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:45.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.108+0000 7f9561d9b700 1 -- 172.21.15.78:0/3985135956 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f956415e560 con 0x7f956406d340 2023-10-11T09:48:45.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.108+0000 7f9561d9b700 1 -- 172.21.15.78:0/3985135956 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f9558001670 con 0x7f956406d340 2023-10-11T09:48:45.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.108+0000 7f9561d9b700 1 -- 172.21.15.78:0/3985135956 >> v1:172.21.15.78:6789/0 conn(0x7f956406d340 legacy=0x7f95640ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:45.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.108+0000 7f9561d9b700 1 --2- 172.21.15.78:0/3985135956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f954c003a70 0x7f954c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:45.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.108+0000 7f9561d9b700 1 --2- 172.21.15.78:0/3985135956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f954c004870 0x7f954c004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:45.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.108+0000 7f9561d9b700 1 --2- 172.21.15.78:0/3985135956 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f954c005220 0x7f954c00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:45.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.108+0000 7f9561d9b700 1 -- 172.21.15.78:0/3985135956 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954c00c100 con 0x7f954c004870 2023-10-11T09:48:45.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.108+0000 7f9561d9b700 1 -- 172.21.15.78:0/3985135956 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954c00c240 con 0x7f954c003a70 2023-10-11T09:48:45.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.108+0000 7f9561d9b700 1 -- 172.21.15.78:0/3985135956 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f954c00c400 con 0x7f954c005220 2023-10-11T09:48:45.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.112+0000 7f956259c700 1 --2- 172.21.15.78:0/3985135956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f954c004870 0x7f954c004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:45.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.112+0000 7f956359e700 1 --2- 172.21.15.78:0/3985135956 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f954c005220 0x7f954c00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:45.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.112+0000 7f9562d9d700 1 --2- 172.21.15.78:0/3985135956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f954c003a70 0x7f954c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:45.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.112+0000 7f956259c700 1 -- 172.21.15.78:0/3985135956 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f954c005220 msgr2=0x7f954c00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:45.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.112+0000 7f956259c700 1 --2- 172.21.15.78:0/3985135956 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f954c005220 0x7f954c00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:45.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.112+0000 7f956259c700 1 -- 172.21.15.78:0/3985135956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f954c003a70 msgr2=0x7f954c003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:45.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.112+0000 7f956259c700 1 --2- 172.21.15.78:0/3985135956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f954c003a70 0x7f954c003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:45.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.112+0000 7f956259c700 1 -- 172.21.15.78:0/3985135956 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f954c0035d0 con 0x7f954c004870 2023-10-11T09:48:45.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.112+0000 7f956359e700 1 -- 172.21.15.78:0/3985135956 reap_dead start 2023-10-11T09:48:45.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.112+0000 7f956259c700 1 --2- 172.21.15.78:0/3985135956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f954c004870 0x7f954c004ce0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f955800c160 tx=0x7f954c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd5e2199e892552 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:45.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.112+0000 7f9561d9b700 1 -- 172.21.15.78:0/3985135956 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9558007b50 con 0x7f954c004870 2023-10-11T09:48:45.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.112+0000 7f9561d9b700 1 -- 172.21.15.78:0/3985135956 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f955800cd00 con 0x7f954c004870 2023-10-11T09:48:45.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.112+0000 7f9561d9b700 1 -- 172.21.15.78:0/3985135956 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9558005bc0 con 0x7f954c004870 2023-10-11T09:48:45.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.112+0000 7f95690bf700 1 -- 172.21.15.78:0/3985135956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f954c004870 msgr2=0x7f954c004ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:45.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.112+0000 7f95690bf700 1 --2- 172.21.15.78:0/3985135956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f954c004870 0x7f954c004ce0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f955800c160 tx=0x7f954c0058a0 comp rx=0 tx=0).stop 2023-10-11T09:48:45.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.112+0000 7f95690bf700 1 -- 172.21.15.78:0/3985135956 shutdown_connections 2023-10-11T09:48:45.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.112+0000 7f95690bf700 1 --2- 172.21.15.78:0/3985135956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f954c004870 0x7f954c004ce0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:45.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.112+0000 7f95690bf700 1 -- 172.21.15.78:0/3985135956 >> 172.21.15.78:0/3985135956 conn(0x7f95640cecc0 msgr2=0x7f95640cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:45.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.112+0000 7f95690bf700 1 -- 172.21.15.78:0/3985135956 shutdown_connections 2023-10-11T09:48:45.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.112+0000 7f95690bf700 1 -- 172.21.15.78:0/3985135956 wait complete. 2023-10-11T09:48:45.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.116+0000 7f95690bf700 1 Processor -- start 2023-10-11T09:48:45.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.116+0000 7f95690bf700 1 -- start start 2023-10-11T09:48:45.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.116+0000 7f95690bf700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f954c004870 0x7f9564149440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:45.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.116+0000 7f95690bf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95640cdfe0 0x7f9564158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:45.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.116+0000 7f95690bf700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95640ce450 0x7f95641591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:45.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.116+0000 7f95690bf700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956415bcf0 con 0x7f95640cdfe0 2023-10-11T09:48:45.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.116+0000 7f95690bf700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956415b0e0 con 0x7f95640ce450 2023-10-11T09:48:45.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.116+0000 7f95690bf700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f956415b3c0 con 0x7f954c004870 2023-10-11T09:48:45.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.116+0000 7f956259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95640cdfe0 0x7f9564158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:45.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.116+0000 7f956259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95640cdfe0 0x7f9564158c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35406/0 (socket says 172.21.15.78:35406) 2023-10-11T09:48:45.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.116+0000 7f956259c700 1 -- 172.21.15.78:0/2007267159 learned_addr learned my addr 172.21.15.78:0/2007267159 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:45.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.116+0000 7f956359e700 1 --2- 172.21.15.78:0/2007267159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95640ce450 0x7f95641591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:45.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.116+0000 7f9562d9d700 1 --2- 172.21.15.78:0/2007267159 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f954c004870 0x7f9564149440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:45.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.116+0000 7f956259c700 1 -- 172.21.15.78:0/2007267159 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f954c004870 msgr2=0x7f9564149440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:45.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.116+0000 7f956259c700 1 --2- 172.21.15.78:0/2007267159 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f954c004870 0x7f9564149440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:45.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.116+0000 7f956259c700 1 -- 172.21.15.78:0/2007267159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95640ce450 msgr2=0x7f95641591b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:45.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.116+0000 7f956259c700 1 --2- 172.21.15.78:0/2007267159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95640ce450 0x7f95641591b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:45.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.116+0000 7f956259c700 1 -- 172.21.15.78:0/2007267159 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f956415b6a0 con 0x7f95640cdfe0 2023-10-11T09:48:45.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.120+0000 7f956259c700 1 --2- 172.21.15.78:0/2007267159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95640cdfe0 0x7f9564158c70 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f9558004ac0 tx=0x7f956415f0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:45.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.120+0000 7f954b7fe700 1 -- 172.21.15.78:0/2007267159 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9558000ed0 con 0x7f95640cdfe0 2023-10-11T09:48:45.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.120+0000 7f954b7fe700 1 -- 172.21.15.78:0/2007267159 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f955800ce70 con 0x7f95640cdfe0 2023-10-11T09:48:45.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.120+0000 7f95690bf700 1 -- 172.21.15.78:0/2007267159 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f956415b980 con 0x7f95640cdfe0 2023-10-11T09:48:45.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.120+0000 7f954b7fe700 1 -- 172.21.15.78:0/2007267159 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95580072d0 con 0x7f95640cdfe0 2023-10-11T09:48:45.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.120+0000 7f95690bf700 1 -- 172.21.15.78:0/2007267159 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95640cc800 con 0x7f95640cdfe0 2023-10-11T09:48:45.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.120+0000 7f95690bf700 1 -- 172.21.15.78:0/2007267159 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f956406eb80 con 0x7f95640cdfe0 2023-10-11T09:48:45.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.124+0000 7f954b7fe700 1 -- 172.21.15.78:0/2007267159 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f9558007430 con 0x7f95640cdfe0 2023-10-11T09:48:45.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.124+0000 7f954b7fe700 1 --2- 172.21.15.78:0/2007267159 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9550065ed0 0x7f9550068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:45.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.124+0000 7f954b7fe700 1 -- 172.21.15.78:0/2007267159 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6552+0+0 (secure 0 0 0) 0x7f95580848b0 con 0x7f95640cdfe0 2023-10-11T09:48:45.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.124+0000 7f9562d9d700 1 --2- 172.21.15.78:0/2007267159 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9550065ed0 0x7f9550068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:45.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.124+0000 7f9562d9d700 1 --2- 172.21.15.78:0/2007267159 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9550065ed0 0x7f9550068380 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f956404f770 tx=0x7f9554008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:45.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.132+0000 7f954b7fe700 1 -- 172.21.15.78:0/2007267159 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f9558051d00 con 0x7f95640cdfe0 2023-10-11T09:48:45.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.344+0000 7f95690bf700 1 -- 172.21.15.78:0/2007267159 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs reset", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f956414bf40 con 0x7f95640cdfe0 2023-10-11T09:48:45.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.460+0000 7f954b7fe700 1 -- 172.21.15.78:0/2007267159 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs reset", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v61) v1 ==== 107+0+0 (secure 0 0 0) 0x7f9558055320 con 0x7f95640cdfe0 2023-10-11T09:48:45.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.464+0000 7f95690bf700 1 -- 172.21.15.78:0/2007267159 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9550065ed0 msgr2=0x7f9550068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:45.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.464+0000 7f95690bf700 1 --2- 172.21.15.78:0/2007267159 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9550065ed0 0x7f9550068380 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f956404f770 tx=0x7f9554008040 comp rx=0 tx=0).stop 2023-10-11T09:48:45.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.464+0000 7f95690bf700 1 -- 172.21.15.78:0/2007267159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95640cdfe0 msgr2=0x7f9564158c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:45.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.464+0000 7f95690bf700 1 --2- 172.21.15.78:0/2007267159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95640cdfe0 0x7f9564158c70 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f9558004ac0 tx=0x7f956415f0e0 comp rx=0 tx=0).stop 2023-10-11T09:48:45.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.464+0000 7f95690bf700 1 -- 172.21.15.78:0/2007267159 shutdown_connections 2023-10-11T09:48:45.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.464+0000 7f95690bf700 1 --2- 172.21.15.78:0/2007267159 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9550065ed0 0x7f9550068380 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:45.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.464+0000 7f95690bf700 1 --2- 172.21.15.78:0/2007267159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95640ce450 0x7f95641591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:45.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.464+0000 7f95690bf700 1 --2- 172.21.15.78:0/2007267159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95640cdfe0 0x7f9564158c70 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:45.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.464+0000 7f95690bf700 1 --2- 172.21.15.78:0/2007267159 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f954c004870 0x7f9564149440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:45.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.464+0000 7f95690bf700 1 -- 172.21.15.78:0/2007267159 >> 172.21.15.78:0/2007267159 conn(0x7f95640cecc0 msgr2=0x7f9564157760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:45.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.464+0000 7f95690bf700 1 -- 172.21.15.78:0/2007267159 shutdown_connections 2023-10-11T09:48:45.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.464+0000 7f95690bf700 1 -- 172.21.15.78:0/2007267159 wait complete. 2023-10-11T09:48:45.483 INFO:tasks.ceph.mds.c.smithi078.stderr:2023-10-11T09:48:45.476+0000 7fe70ebcf700 -1 mds.0.sessionmap _load_finish got (2) No such file or directory 2023-10-11T09:48:45.484 INFO:tasks.ceph.mds.c.smithi078.stderr:2023-10-11T09:48:45.476+0000 7fe70ebcf700 -1 log_channel(cluster) log [ERR] : error reading sessionmap 'mds0_sessionmap' -2 ((2) No such file or directory) 2023-10-11T09:48:45.485 INFO:tasks.ceph.mon.a.smithi078.stderr:2023-10-11T09:48:45.476+0000 7f90131f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-11T09:48:45.485 INFO:tasks.ceph.mon.a.smithi078.stderr:2023-10-11T09:48:45.476+0000 7f90131f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-10-11T09:48:45.500 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-11T09:48:45.506 INFO:tasks.ceph.mds.c.smithi078.stderr: -26> 2023-10-11T09:48:45.476+0000 7fe70ebcf700 -1 mds.0.sessionmap _load_finish got (2) No such file or directory 2023-10-11T09:48:45.507 INFO:tasks.ceph.mds.c.smithi078.stderr: -25> 2023-10-11T09:48:45.476+0000 7fe70ebcf700 -1 log_channel(cluster) log [ERR] : error reading sessionmap 'mds0_sessionmap' -2 ((2) No such file or directory) 2023-10-11T09:48:45.553 INFO:tasks.ceph.mds.c.smithi078.stdout:starting mds.c at 2023-10-11T09:48:45.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.672+0000 7f856ffff700 1 Processor -- start 2023-10-11T09:48:45.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.672+0000 7f856ffff700 1 -- start start 2023-10-11T09:48:45.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.676+0000 7f856ffff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f85700b3610 0x7f85700d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:45.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.676+0000 7f856ffff700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f85700b3a90 0x7f85700b85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:45.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.676+0000 7f856ffff700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85700b47f0 con 0x7f85700b3240 2023-10-11T09:48:45.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.676+0000 7f856ffff700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85700b8130 con 0x7f85700b3610 2023-10-11T09:48:45.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.676+0000 7f856ffff700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85700b41e0 con 0x7f85700b3a90 2023-10-11T09:48:45.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.676+0000 7f856e7fc700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f85700b3610 0x7f85700d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:45.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.676+0000 7f856e7fc700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f85700b3610 0x7f85700d29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35432/0 (socket says 172.21.15.78:35432) 2023-10-11T09:48:45.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.676+0000 7f856e7fc700 1 -- 172.21.15.78:0/1302340434 learned_addr learned my addr 172.21.15.78:0/1302340434 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:45.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.676+0000 7f856f7fe700 1 --2- 172.21.15.78:0/1302340434 >> v2:172.21.15.133:3300/0 conn(0x7f85700b3a90 0x7f85700b85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:45.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.676+0000 7f856e7fc700 1 -- 172.21.15.78:0/1302340434 >> v2:172.21.15.133:3300/0 conn(0x7f85700b3a90 msgr2=0x7f85700b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:45.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.676+0000 7f856e7fc700 1 --2- 172.21.15.78:0/1302340434 >> v2:172.21.15.133:3300/0 conn(0x7f85700b3a90 0x7f85700b85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:45.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.676+0000 7f856e7fc700 1 -- 172.21.15.78:0/1302340434 >> v1:172.21.15.78:6789/0 conn(0x7f85700b3240 legacy=0x7f857015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:45.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.676+0000 7f856e7fc700 1 -- 172.21.15.78:0/1302340434 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85700b7800 con 0x7f85700b3610 2023-10-11T09:48:45.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.676+0000 7f856e7fc700 1 --2- 172.21.15.78:0/1302340434 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85700b3610 0x7f85700d29d0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f8570072b40 tx=0x7f8564009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93f61c71b5b4f245 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:45.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.676+0000 7f856dffb700 1 -- 172.21.15.78:0/1302340434 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856400b8a0 con 0x7f85700b3610 2023-10-11T09:48:45.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.676+0000 7f856dffb700 1 -- 172.21.15.78:0/1302340434 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f856400ba00 con 0x7f85700b3610 2023-10-11T09:48:45.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.676+0000 7f856dffb700 1 -- 172.21.15.78:0/1302340434 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856400bd90 con 0x7f85700b3610 2023-10-11T09:48:45.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.676+0000 7f856ffff700 1 -- 172.21.15.78:0/1302340434 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85700b3610 msgr2=0x7f85700d29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:45.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.676+0000 7f856ffff700 1 --2- 172.21.15.78:0/1302340434 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85700b3610 0x7f85700d29d0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f8570072b40 tx=0x7f8564009bf0 comp rx=0 tx=0).stop 2023-10-11T09:48:45.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.676+0000 7f856ffff700 1 -- 172.21.15.78:0/1302340434 shutdown_connections 2023-10-11T09:48:45.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.676+0000 7f856ffff700 1 --2- 172.21.15.78:0/1302340434 >> v2:172.21.15.133:3300/0 conn(0x7f85700b3a90 0x7f85700b85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:45.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.676+0000 7f856ffff700 1 --2- 172.21.15.78:0/1302340434 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85700b3610 0x7f85700d29d0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:45.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.676+0000 7f856ffff700 1 -- 172.21.15.78:0/1302340434 >> 172.21.15.78:0/1302340434 conn(0x7f857009c260 msgr2=0x7f85700d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:45.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.676+0000 7f856ffff700 1 -- 172.21.15.78:0/1302340434 shutdown_connections 2023-10-11T09:48:45.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.676+0000 7f856ffff700 1 -- 172.21.15.78:0/1302340434 wait complete. 2023-10-11T09:48:45.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.680+0000 7f856ffff700 1 Processor -- start 2023-10-11T09:48:45.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.680+0000 7f856ffff700 1 -- start start 2023-10-11T09:48:45.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.680+0000 7f856ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85700b3240 0x7f8570158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:45.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.680+0000 7f856ffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85700b3610 0x7f857009d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:45.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.680+0000 7f856ffff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f85700b3a90 0x7f857015ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:45.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.680+0000 7f856ffff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857009e2c0 con 0x7f85700b3240 2023-10-11T09:48:45.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.680+0000 7f856ffff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857009e5a0 con 0x7f85700b3610 2023-10-11T09:48:45.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.680+0000 7f856ffff700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f857009e880 con 0x7f85700b3a90 2023-10-11T09:48:45.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.680+0000 7f856effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85700b3240 0x7f8570158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:45.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.680+0000 7f856effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85700b3240 0x7f8570158f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35444/0 (socket says 172.21.15.78:35444) 2023-10-11T09:48:45.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.680+0000 7f856effd700 1 -- 172.21.15.78:0/2090303557 learned_addr learned my addr 172.21.15.78:0/2090303557 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:45.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.680+0000 7f856effd700 1 -- 172.21.15.78:0/2090303557 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f85700b3a90 msgr2=0x7f857015ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:45.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.680+0000 7f856effd700 1 --2- 172.21.15.78:0/2090303557 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f85700b3a90 0x7f857015ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:45.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.680+0000 7f856e7fc700 1 --2- 172.21.15.78:0/2090303557 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85700b3610 0x7f857009d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:45.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.680+0000 7f856effd700 1 -- 172.21.15.78:0/2090303557 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85700b3610 msgr2=0x7f857009d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:45.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.680+0000 7f856effd700 1 --2- 172.21.15.78:0/2090303557 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85700b3610 0x7f857009d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:45.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.680+0000 7f856effd700 1 -- 172.21.15.78:0/2090303557 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8564009570 con 0x7f85700b3240 2023-10-11T09:48:45.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.680+0000 7f856e7fc700 1 --2- 172.21.15.78:0/2090303557 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85700b3610 0x7f857009d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:48:45.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.680+0000 7f856effd700 1 --2- 172.21.15.78:0/2090303557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85700b3240 0x7f8570158f90 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f856000c410 tx=0x7f856000c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:45.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.680+0000 7f85577fe700 1 -- 172.21.15.78:0/2090303557 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85600052d0 con 0x7f85700b3240 2023-10-11T09:48:45.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.680+0000 7f85577fe700 1 -- 172.21.15.78:0/2090303557 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8560005430 con 0x7f85700b3240 2023-10-11T09:48:45.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.680+0000 7f856ffff700 1 -- 172.21.15.78:0/2090303557 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f857009eb60 con 0x7f85700b3240 2023-10-11T09:48:45.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.680+0000 7f85577fe700 1 -- 172.21.15.78:0/2090303557 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85600055a0 con 0x7f85700b3240 2023-10-11T09:48:45.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.680+0000 7f856ffff700 1 -- 172.21.15.78:0/2090303557 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85701594d0 con 0x7f85700b3240 2023-10-11T09:48:45.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.684+0000 7f856ffff700 1 -- 172.21.15.78:0/2090303557 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8570065d40 con 0x7f85700b3240 2023-10-11T09:48:45.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.684+0000 7f85577fe700 1 -- 172.21.15.78:0/2090303557 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f85600036a0 con 0x7f85700b3240 2023-10-11T09:48:45.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.688+0000 7f85577fe700 1 --2- 172.21.15.78:0/2090303557 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f855c065ed0 0x7f855c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:45.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.688+0000 7f85577fe700 1 -- 172.21.15.78:0/2090303557 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6638+0+0 (secure 0 0 0) 0x7f8560082c30 con 0x7f85700b3240 2023-10-11T09:48:45.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.688+0000 7f856e7fc700 1 --2- 172.21.15.78:0/2090303557 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f855c065ed0 0x7f855c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:45.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.688+0000 7f856e7fc700 1 --2- 172.21.15.78:0/2090303557 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f855c065ed0 0x7f855c068380 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f85700c64c0 tx=0x7f8564008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:45.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.692+0000 7f85577fe700 1 -- 172.21.15.78:0/2090303557 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f856004ff10 con 0x7f85700b3240 2023-10-11T09:48:45.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:45.900+0000 7f856ffff700 1 -- 172.21.15.78:0/2090303557 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f85700b47f0 con 0x7f85700b3240 2023-10-11T09:48:46.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.500+0000 7f85577fe700 1 -- 172.21.15.78:0/2090303557 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v64) v1 ==== 162+0+0 (secure 0 0 0) 0x7f8560053530 con 0x7f85700b3240 2023-10-11T09:48:46.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.504+0000 7f856ffff700 1 -- 172.21.15.78:0/2090303557 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f855c065ed0 msgr2=0x7f855c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:46.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.504+0000 7f856ffff700 1 --2- 172.21.15.78:0/2090303557 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f855c065ed0 0x7f855c068380 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f85700c64c0 tx=0x7f8564008030 comp rx=0 tx=0).stop 2023-10-11T09:48:46.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.504+0000 7f856ffff700 1 -- 172.21.15.78:0/2090303557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85700b3240 msgr2=0x7f8570158f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:46.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.504+0000 7f856ffff700 1 --2- 172.21.15.78:0/2090303557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85700b3240 0x7f8570158f90 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f856000c410 tx=0x7f856000c7d0 comp rx=0 tx=0).stop 2023-10-11T09:48:46.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.504+0000 7f856ffff700 1 -- 172.21.15.78:0/2090303557 shutdown_connections 2023-10-11T09:48:46.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.504+0000 7f856ffff700 1 --2- 172.21.15.78:0/2090303557 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f855c065ed0 0x7f855c068380 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:46.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.504+0000 7f856ffff700 1 --2- 172.21.15.78:0/2090303557 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f85700b3a90 0x7f857015ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:46.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.504+0000 7f856ffff700 1 --2- 172.21.15.78:0/2090303557 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85700b3610 0x7f857009d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:46.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.504+0000 7f856ffff700 1 --2- 172.21.15.78:0/2090303557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85700b3240 0x7f8570158f90 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:46.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.504+0000 7f856ffff700 1 -- 172.21.15.78:0/2090303557 >> 172.21.15.78:0/2090303557 conn(0x7f857009c260 msgr2=0x7f85700d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:46.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.504+0000 7f856ffff700 1 -- 172.21.15.78:0/2090303557 shutdown_connections 2023-10-11T09:48:46.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.504+0000 7f856ffff700 1 -- 172.21.15.78:0/2090303557 wait complete. 2023-10-11T09:48:46.514 INFO:teuthology.orchestra.run.smithi078.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-11T09:48:46.536 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:48:46.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.752+0000 7fa8b3fb8700 1 Processor -- start 2023-10-11T09:48:46.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.756+0000 7fa8b3fb8700 1 -- start start 2023-10-11T09:48:46.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.756+0000 7fa8b3fb8700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa8ac154e00 0x7fa8ac15b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:46.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.756+0000 7fa8b3fb8700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fa8ac071040 0x7fa8ac0cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:46.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.756+0000 7fa8b3fb8700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8ac0720f0 con 0x7fa8ac0cba00 2023-10-11T09:48:46.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.756+0000 7fa8b3fb8700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8ac15bb90 con 0x7fa8ac071040 2023-10-11T09:48:46.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.756+0000 7fa8b3fb8700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8ac0714e0 con 0x7fa8ac154e00 2023-10-11T09:48:46.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.756+0000 7fa8b2555700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fa8ac0cba00 0x7fa8ac15e1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:54490/0 (socket says 172.21.15.78:54490) 2023-10-11T09:48:46.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.756+0000 7fa8b2555700 1 -- 172.21.15.78:0/2146286123 learned_addr learned my addr 172.21.15.78:0/2146286123 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:46.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.756+0000 7fa8b1553700 1 --2- 172.21.15.78:0/2146286123 >> v2:172.21.15.78:3300/0 conn(0x7fa8ac071040 0x7fa8ac0cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:46.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.756+0000 7fa8b1d54700 1 --2- 172.21.15.78:0/2146286123 >> v2:172.21.15.133:3300/0 conn(0x7fa8ac154e00 0x7fa8ac15b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:46.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.756+0000 7fa8b1553700 1 -- 172.21.15.78:0/2146286123 >> v2:172.21.15.133:3300/0 conn(0x7fa8ac154e00 msgr2=0x7fa8ac15b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:46.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.756+0000 7fa8b1553700 1 --2- 172.21.15.78:0/2146286123 >> v2:172.21.15.133:3300/0 conn(0x7fa8ac154e00 0x7fa8ac15b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:46.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.756+0000 7fa8b1553700 1 -- 172.21.15.78:0/2146286123 >> v1:172.21.15.78:6789/0 conn(0x7fa8ac0cba00 legacy=0x7fa8ac15e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:46.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.756+0000 7fa8b0d52700 1 -- 172.21.15.78:0/2146286123 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 499711602 0 0) 0x7fa8ac0720f0 con 0x7fa8ac0cba00 2023-10-11T09:48:46.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.756+0000 7fa8b1553700 1 -- 172.21.15.78:0/2146286123 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8ac0717c0 con 0x7fa8ac071040 2023-10-11T09:48:46.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.760+0000 7fa8b1553700 1 --2- 172.21.15.78:0/2146286123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8ac071040 0x7fa8ac0cb4c0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac06f7c0 tx=0x7fa89c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb296e6a4d7e3e0d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:46.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.760+0000 7fa8b0d52700 1 -- 172.21.15.78:0/2146286123 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c011830 con 0x7fa8ac071040 2023-10-11T09:48:46.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.760+0000 7fa8b0d52700 1 -- 172.21.15.78:0/2146286123 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa89c011990 con 0x7fa8ac071040 2023-10-11T09:48:46.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.760+0000 7fa8b0d52700 1 -- 172.21.15.78:0/2146286123 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c011d20 con 0x7fa8ac071040 2023-10-11T09:48:46.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.760+0000 7fa8b3fb8700 1 -- 172.21.15.78:0/2146286123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8ac071040 msgr2=0x7fa8ac0cb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:46.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.760+0000 7fa8b3fb8700 1 --2- 172.21.15.78:0/2146286123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8ac071040 0x7fa8ac0cb4c0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac06f7c0 tx=0x7fa89c009b30 comp rx=0 tx=0).stop 2023-10-11T09:48:46.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.760+0000 7fa8b3fb8700 1 -- 172.21.15.78:0/2146286123 shutdown_connections 2023-10-11T09:48:46.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.760+0000 7fa8b3fb8700 1 --2- 172.21.15.78:0/2146286123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8ac071040 0x7fa8ac0cb4c0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:46.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.760+0000 7fa8b3fb8700 1 --2- 172.21.15.78:0/2146286123 >> v2:172.21.15.133:3300/0 conn(0x7fa8ac154e00 0x7fa8ac15b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:46.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.760+0000 7fa8b3fb8700 1 -- 172.21.15.78:0/2146286123 >> 172.21.15.78:0/2146286123 conn(0x7fa8ac06aca0 msgr2=0x7fa8ac0b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:46.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.760+0000 7fa8b3fb8700 1 -- 172.21.15.78:0/2146286123 shutdown_connections 2023-10-11T09:48:46.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.760+0000 7fa8b3fb8700 1 -- 172.21.15.78:0/2146286123 wait complete. 2023-10-11T09:48:46.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.760+0000 7fa8b3fb8700 1 Processor -- start 2023-10-11T09:48:46.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.764+0000 7fa8b3fb8700 1 -- start start 2023-10-11T09:48:46.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.764+0000 7fa8b3fb8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8ac071040 0x7fa8ac143a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:46.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.764+0000 7fa8b3fb8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8ac0cba00 0x7fa8ac0b3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:46.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.764+0000 7fa8b3fb8700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa8ac154e00 0x7fa8ac0b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:46.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.764+0000 7fa8b3fb8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac0b2000 con 0x7fa8ac0cba00 2023-10-11T09:48:46.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.764+0000 7fa8b3fb8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac0b22e0 con 0x7fa8ac071040 2023-10-11T09:48:46.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.764+0000 7fa8b3fb8700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac0b25c0 con 0x7fa8ac154e00 2023-10-11T09:48:46.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.764+0000 7fa8b1553700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8ac0cba00 0x7fa8ac0b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:46.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.764+0000 7fa8b1553700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8ac0cba00 0x7fa8ac0b3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35460/0 (socket says 172.21.15.78:35460) 2023-10-11T09:48:46.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.764+0000 7fa8b1553700 1 -- 172.21.15.78:0/4269291780 learned_addr learned my addr 172.21.15.78:0/4269291780 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:46.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.764+0000 7fa8b1d54700 1 --2- 172.21.15.78:0/4269291780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8ac071040 0x7fa8ac143a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:46.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.764+0000 7fa8b2555700 1 --2- 172.21.15.78:0/4269291780 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa8ac154e00 0x7fa8ac0b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:46.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.764+0000 7fa8b1553700 1 -- 172.21.15.78:0/4269291780 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa8ac154e00 msgr2=0x7fa8ac0b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:46.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.764+0000 7fa8b1553700 1 --2- 172.21.15.78:0/4269291780 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa8ac154e00 0x7fa8ac0b38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:46.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.764+0000 7fa8b1553700 1 -- 172.21.15.78:0/4269291780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8ac071040 msgr2=0x7fa8ac143a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:46.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.764+0000 7fa8b1553700 1 --2- 172.21.15.78:0/4269291780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8ac071040 0x7fa8ac143a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:46.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.764+0000 7fa8b1553700 1 -- 172.21.15.78:0/4269291780 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa89c0094b0 con 0x7fa8ac0cba00 2023-10-11T09:48:46.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.764+0000 7fa8b1553700 1 --2- 172.21.15.78:0/4269291780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8ac0cba00 0x7fa8ac0b3370 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fa89c009fd0 tx=0x7fa89c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:46.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.764+0000 7fa8a27fc700 1 -- 172.21.15.78:0/4269291780 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c0045b0 con 0x7fa8ac0cba00 2023-10-11T09:48:46.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.764+0000 7fa8a27fc700 1 -- 172.21.15.78:0/4269291780 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa89c01f070 con 0x7fa8ac0cba00 2023-10-11T09:48:46.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.764+0000 7fa8a27fc700 1 -- 172.21.15.78:0/4269291780 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c0038e0 con 0x7fa8ac0cba00 2023-10-11T09:48:46.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.764+0000 7fa8b3fb8700 1 -- 172.21.15.78:0/4269291780 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8ac0b28a0 con 0x7fa8ac0cba00 2023-10-11T09:48:46.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.764+0000 7fa8b3fb8700 1 -- 172.21.15.78:0/4269291780 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8ac0b1c90 con 0x7fa8ac0cba00 2023-10-11T09:48:46.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.768+0000 7fa8b3fb8700 1 -- 172.21.15.78:0/4269291780 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8ac0b2dc0 con 0x7fa8ac0cba00 2023-10-11T09:48:46.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.768+0000 7fa8a27fc700 1 -- 172.21.15.78:0/4269291780 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fa89c021030 con 0x7fa8ac0cba00 2023-10-11T09:48:46.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.768+0000 7fa8a27fc700 1 --2- 172.21.15.78:0/4269291780 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa890065ed0 0x7fa890068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:46.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.768+0000 7fa8a27fc700 1 -- 172.21.15.78:0/4269291780 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6638+0+0 (secure 0 0 0) 0x7fa89c084e00 con 0x7fa8ac0cba00 2023-10-11T09:48:46.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.768+0000 7fa8b1d54700 1 --2- 172.21.15.78:0/4269291780 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa890065ed0 0x7fa890068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:46.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.772+0000 7fa8b1d54700 1 --2- 172.21.15.78:0/4269291780 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa890065ed0 0x7fa890068380 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa8a80073f0 tx=0x7fa8a800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:46.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.776+0000 7fa8a27fc700 1 -- 172.21.15.78:0/4269291780 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fa8ac0b2dc0 con 0x7fa8ac0cba00 2023-10-11T09:48:46.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.984+0000 7fa8b3fb8700 1 -- 172.21.15.78:0/4269291780 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8ac0b2dc0 con 0x7fa8ac0cba00 2023-10-11T09:48:46.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.988+0000 7fa8a27fc700 1 -- 172.21.15.78:0/4269291780 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4648 (secure 0 0 0) 0x7fa8ac0b2dc0 con 0x7fa8ac0cba00 2023-10-11T09:48:46.992 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:46.993 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":64,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6961,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/1080544187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1080544187},{"type":"v1","addr":"172.21.15.133:6834","nonce":1080544187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/367194191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":367194191},{"type":"v1","addr":"172.21.15.133:6835","nonce":367194191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":7216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/2688829449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":2688829449},{"type":"v1","addr":"172.21.15.78:6837","nonce":2688829449}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":7533,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/1560261029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":1560261029},{"type":"v1","addr":"172.21.15.78:6836","nonce":1560261029}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:48:45.351822+0000","modified":"2023-10-11T09:48:46.494562+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[12,13],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-11T09:48:46.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.988+0000 7fa8b3fb8700 1 -- 172.21.15.78:0/4269291780 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa890065ed0 msgr2=0x7fa890068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:46.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.988+0000 7fa8b3fb8700 1 --2- 172.21.15.78:0/4269291780 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa890065ed0 0x7fa890068380 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa8a80073f0 tx=0x7fa8a800b040 comp rx=0 tx=0).stop 2023-10-11T09:48:46.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.988+0000 7fa8b3fb8700 1 -- 172.21.15.78:0/4269291780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8ac0cba00 msgr2=0x7fa8ac0b3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:46.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.988+0000 7fa8b3fb8700 1 --2- 172.21.15.78:0/4269291780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8ac0cba00 0x7fa8ac0b3370 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fa89c009fd0 tx=0x7fa89c004770 comp rx=0 tx=0).stop 2023-10-11T09:48:46.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.988+0000 7fa8b3fb8700 1 -- 172.21.15.78:0/4269291780 shutdown_connections 2023-10-11T09:48:46.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.988+0000 7fa8b3fb8700 1 --2- 172.21.15.78:0/4269291780 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa890065ed0 0x7fa890068380 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:46.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.988+0000 7fa8b3fb8700 1 --2- 172.21.15.78:0/4269291780 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa8ac154e00 0x7fa8ac0b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:46.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.988+0000 7fa8b3fb8700 1 --2- 172.21.15.78:0/4269291780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8ac0cba00 0x7fa8ac0b3370 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:46.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.988+0000 7fa8b3fb8700 1 --2- 172.21.15.78:0/4269291780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8ac071040 0x7fa8ac143a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:46.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.988+0000 7fa8b3fb8700 1 -- 172.21.15.78:0/4269291780 >> 172.21.15.78:0/4269291780 conn(0x7fa8ac06aca0 msgr2=0x7fa8ac06cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:46.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.988+0000 7fa8b3fb8700 1 -- 172.21.15.78:0/4269291780 shutdown_connections 2023-10-11T09:48:46.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:46.988+0000 7fa8b3fb8700 1 -- 172.21.15.78:0/4269291780 wait complete. 2023-10-11T09:48:46.998 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 64 2023-10-11T09:48:47.018 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-11T09:48:47.019 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:48:47.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.184+0000 7f5d2bed8700 1 Processor -- start 2023-10-11T09:48:47.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.184+0000 7f5d2bed8700 1 -- start start 2023-10-11T09:48:47.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.188+0000 7f5d2bed8700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f5d24071040 0x7f5d240cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:47.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.188+0000 7f5d2bed8700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d240ce790 con 0x7f5d240ce330 2023-10-11T09:48:47.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.188+0000 7f5d2bed8700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d2406ffa0 con 0x7f5d24160d40 2023-10-11T09:48:47.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.188+0000 7f5d2bed8700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d24070c20 con 0x7f5d24071040 2023-10-11T09:48:47.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.188+0000 7f5d29473700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f5d24071040 0x7f5d240cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:47.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.188+0000 7f5d29473700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f5d24071040 0x7f5d240cddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35470/0 (socket says 172.21.15.78:35470) 2023-10-11T09:48:47.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.188+0000 7f5d29473700 1 -- 172.21.15.78:0/2762684591 learned_addr learned my addr 172.21.15.78:0/2762684591 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:47.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.188+0000 7f5d29473700 1 -- 172.21.15.78:0/2762684591 >> v1:172.21.15.133:6789/0 conn(0x7f5d24160d40 legacy=0x7f5d240cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:47.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.188+0000 7f5d29473700 1 -- 172.21.15.78:0/2762684591 >> v1:172.21.15.78:6789/0 conn(0x7f5d240ce330 legacy=0x7f5d2406be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:47.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.188+0000 7f5d29473700 1 -- 172.21.15.78:0/2762684591 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d24070280 con 0x7f5d24071040 2023-10-11T09:48:47.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.188+0000 7f5d29473700 1 --2- 172.21.15.78:0/2762684591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d24071040 0x7f5d240cddf0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f5d14009910 tx=0x7f5d14009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f5d2b477e40f90 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:47.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.188+0000 7f5d28c72700 1 -- 172.21.15.78:0/2762684591 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1400ba60 con 0x7f5d24071040 2023-10-11T09:48:47.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.188+0000 7f5d28c72700 1 -- 172.21.15.78:0/2762684591 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d1400bbc0 con 0x7f5d24071040 2023-10-11T09:48:47.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.188+0000 7f5d28c72700 1 -- 172.21.15.78:0/2762684591 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d14009e60 con 0x7f5d24071040 2023-10-11T09:48:47.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.192+0000 7f5d2bed8700 1 -- 172.21.15.78:0/2762684591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d24071040 msgr2=0x7f5d240cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:47.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.192+0000 7f5d2bed8700 1 --2- 172.21.15.78:0/2762684591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d24071040 0x7f5d240cddf0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f5d14009910 tx=0x7f5d14009c20 comp rx=0 tx=0).stop 2023-10-11T09:48:47.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.192+0000 7f5d2bed8700 1 -- 172.21.15.78:0/2762684591 shutdown_connections 2023-10-11T09:48:47.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.192+0000 7f5d2bed8700 1 --2- 172.21.15.78:0/2762684591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d24071040 0x7f5d240cddf0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:47.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.192+0000 7f5d2bed8700 1 -- 172.21.15.78:0/2762684591 >> 172.21.15.78:0/2762684591 conn(0x7f5d2406aca0 msgr2=0x7f5d240b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:47.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.192+0000 7f5d2bed8700 1 -- 172.21.15.78:0/2762684591 shutdown_connections 2023-10-11T09:48:47.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.192+0000 7f5d2bed8700 1 -- 172.21.15.78:0/2762684591 wait complete. 2023-10-11T09:48:47.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.192+0000 7f5d2bed8700 1 Processor -- start 2023-10-11T09:48:47.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.192+0000 7f5d2bed8700 1 -- start start 2023-10-11T09:48:47.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.192+0000 7f5d2bed8700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5d24071040 0x7f5d240b1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:47.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.192+0000 7f5d2bed8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d240ce330 0x7f5d24159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:47.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.192+0000 7f5d2bed8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d24160d40 0x7f5d240ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:47.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.192+0000 7f5d2bed8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2415a510 con 0x7f5d24160d40 2023-10-11T09:48:47.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.192+0000 7f5d2bed8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d240b2390 con 0x7f5d240ce330 2023-10-11T09:48:47.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.192+0000 7f5d2bed8700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d240b2670 con 0x7f5d24071040 2023-10-11T09:48:47.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.192+0000 7f5d2a475700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d24160d40 0x7f5d240ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:47.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.192+0000 7f5d2a475700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d24160d40 0x7f5d240ca5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35486/0 (socket says 172.21.15.78:35486) 2023-10-11T09:48:47.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.196+0000 7f5d2a475700 1 -- 172.21.15.78:0/2136814474 learned_addr learned my addr 172.21.15.78:0/2136814474 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:47.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.196+0000 7f5d29473700 1 --2- 172.21.15.78:0/2136814474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d240ce330 0x7f5d24159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:47.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.196+0000 7f5d29c74700 1 --2- 172.21.15.78:0/2136814474 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5d24071040 0x7f5d240b1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:47.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.196+0000 7f5d2a475700 1 -- 172.21.15.78:0/2136814474 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5d24071040 msgr2=0x7f5d240b1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:47.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.196+0000 7f5d2a475700 1 --2- 172.21.15.78:0/2136814474 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5d24071040 0x7f5d240b1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:47.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.196+0000 7f5d2a475700 1 -- 172.21.15.78:0/2136814474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d240ce330 msgr2=0x7f5d24159410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:47.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.196+0000 7f5d2a475700 1 --2- 172.21.15.78:0/2136814474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d240ce330 0x7f5d24159410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:47.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.196+0000 7f5d2a475700 1 -- 172.21.15.78:0/2136814474 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d14009570 con 0x7f5d24160d40 2023-10-11T09:48:47.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.196+0000 7f5d2a475700 1 --2- 172.21.15.78:0/2136814474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d24160d40 0x7f5d240ca5d0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00cc50 tx=0x7f5d1c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:47.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.196+0000 7f5d1a7fc700 1 -- 172.21.15.78:0/2136814474 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c00ad80 con 0x7f5d24160d40 2023-10-11T09:48:47.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.196+0000 7f5d1a7fc700 1 -- 172.21.15.78:0/2136814474 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d1c00ce50 con 0x7f5d24160d40 2023-10-11T09:48:47.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.196+0000 7f5d1a7fc700 1 -- 172.21.15.78:0/2136814474 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c003c60 con 0x7f5d24160d40 2023-10-11T09:48:47.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.196+0000 7f5d2bed8700 1 -- 172.21.15.78:0/2136814474 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d240b2950 con 0x7f5d24160d40 2023-10-11T09:48:47.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.196+0000 7f5d2bed8700 1 -- 172.21.15.78:0/2136814474 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d240b2f10 con 0x7f5d24160d40 2023-10-11T09:48:47.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.196+0000 7f5d2bed8700 1 -- 172.21.15.78:0/2136814474 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d24072250 con 0x7f5d24160d40 2023-10-11T09:48:47.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.200+0000 7f5d1a7fc700 1 -- 172.21.15.78:0/2136814474 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f5d1c0050d0 con 0x7f5d24160d40 2023-10-11T09:48:47.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.200+0000 7f5d1a7fc700 1 --2- 172.21.15.78:0/2136814474 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5d08065ed0 0x7f5d08068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:47.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.200+0000 7f5d1a7fc700 1 -- 172.21.15.78:0/2136814474 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6638+0+0 (secure 0 0 0) 0x7f5d1c083be0 con 0x7f5d24160d40 2023-10-11T09:48:47.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.200+0000 7f5d29c74700 1 --2- 172.21.15.78:0/2136814474 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5d08065ed0 0x7f5d08068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:47.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.200+0000 7f5d29c74700 1 --2- 172.21.15.78:0/2136814474 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5d08065ed0 0x7f5d08068380 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5d20000f60 tx=0x7f5d20009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:47.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.204+0000 7f5d1a7fc700 1 -- 172.21.15.78:0/2136814474 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f5d1c050f50 con 0x7f5d24160d40 2023-10-11T09:48:47.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.412+0000 7f5d2bed8700 1 -- 172.21.15.78:0/2136814474 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d2404e530 con 0x7f5d24160d40 2023-10-11T09:48:47.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.416+0000 7f5d1a7fc700 1 -- 172.21.15.78:0/2136814474 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4648 (secure 0 0 0) 0x7f5d1c054570 con 0x7f5d24160d40 2023-10-11T09:48:47.421 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:47.421 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":64,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6961,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/1080544187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1080544187},{"type":"v1","addr":"172.21.15.133:6834","nonce":1080544187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/367194191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":367194191},{"type":"v1","addr":"172.21.15.133:6835","nonce":367194191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":7216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/2688829449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":2688829449},{"type":"v1","addr":"172.21.15.78:6837","nonce":2688829449}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":7533,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/1560261029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":1560261029},{"type":"v1","addr":"172.21.15.78:6836","nonce":1560261029}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:48:45.351822+0000","modified":"2023-10-11T09:48:46.494562+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[12,13],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-11T09:48:47.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.416+0000 7f5d2bed8700 1 -- 172.21.15.78:0/2136814474 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5d08065ed0 msgr2=0x7f5d08068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:47.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.416+0000 7f5d2bed8700 1 --2- 172.21.15.78:0/2136814474 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5d08065ed0 0x7f5d08068380 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5d20000f60 tx=0x7f5d20009040 comp rx=0 tx=0).stop 2023-10-11T09:48:47.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.416+0000 7f5d2bed8700 1 -- 172.21.15.78:0/2136814474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d24160d40 msgr2=0x7f5d240ca5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:47.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.416+0000 7f5d2bed8700 1 --2- 172.21.15.78:0/2136814474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d24160d40 0x7f5d240ca5d0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00cc50 tx=0x7f5d1c00a340 comp rx=0 tx=0).stop 2023-10-11T09:48:47.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.416+0000 7f5d2bed8700 1 -- 172.21.15.78:0/2136814474 shutdown_connections 2023-10-11T09:48:47.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.416+0000 7f5d2bed8700 1 --2- 172.21.15.78:0/2136814474 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5d08065ed0 0x7f5d08068380 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:47.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.416+0000 7f5d2bed8700 1 --2- 172.21.15.78:0/2136814474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d24160d40 0x7f5d240ca5d0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:47.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.416+0000 7f5d2bed8700 1 --2- 172.21.15.78:0/2136814474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d240ce330 0x7f5d24159410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:47.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.416+0000 7f5d2bed8700 1 --2- 172.21.15.78:0/2136814474 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5d24071040 0x7f5d240b1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:47.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.416+0000 7f5d2bed8700 1 -- 172.21.15.78:0/2136814474 >> 172.21.15.78:0/2136814474 conn(0x7f5d2406aca0 msgr2=0x7f5d240cdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:47.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.416+0000 7f5d2bed8700 1 -- 172.21.15.78:0/2136814474 shutdown_connections 2023-10-11T09:48:47.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.416+0000 7f5d2bed8700 1 -- 172.21.15.78:0/2136814474 wait complete. 2023-10-11T09:48:47.426 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 64 2023-10-11T09:48:47.442 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:48:47.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.608+0000 7f266c5f0700 1 Processor -- start 2023-10-11T09:48:47.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.608+0000 7f266c5f0700 1 -- start start 2023-10-11T09:48:47.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.608+0000 7f266c5f0700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f266415c880 0x7f266406be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:47.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.608+0000 7f266c5f0700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f26640cbd30 0x7f26640c9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:47.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.608+0000 7f266c5f0700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26640c93a0 con 0x7f26640cd210 2023-10-11T09:48:47.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.608+0000 7f266c5f0700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2664161420 con 0x7f266415c880 2023-10-11T09:48:47.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.608+0000 7f266c5f0700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26640cca10 con 0x7f26640cbd30 2023-10-11T09:48:47.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.612+0000 7f2669b8b700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f26640cd210 0x7f26640cb620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:54494/0 (socket says 172.21.15.78:54494) 2023-10-11T09:48:47.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.612+0000 7f2669b8b700 1 -- 172.21.15.78:0/1106372532 learned_addr learned my addr 172.21.15.78:0/1106372532 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:47.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.612+0000 7f266a38c700 1 --2- 172.21.15.78:0/1106372532 >> v2:172.21.15.78:3300/0 conn(0x7f266415c880 0x7f266406be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:47.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.612+0000 7f266ab8d700 1 --2- 172.21.15.78:0/1106372532 >> v2:172.21.15.133:3300/0 conn(0x7f26640cbd30 0x7f26640c9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:47.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.612+0000 7f266938a700 1 -- 172.21.15.78:0/1106372532 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2658681830 0 0) 0x7f26640c93a0 con 0x7f26640cd210 2023-10-11T09:48:47.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.612+0000 7f266938a700 1 -- 172.21.15.78:0/1106372532 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26500035d0 con 0x7f26640cd210 2023-10-11T09:48:47.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.612+0000 7f266a38c700 1 -- 172.21.15.78:0/1106372532 >> v2:172.21.15.133:3300/0 conn(0x7f26640cbd30 msgr2=0x7f26640c9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:47.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.612+0000 7f266a38c700 1 --2- 172.21.15.78:0/1106372532 >> v2:172.21.15.133:3300/0 conn(0x7f26640cbd30 0x7f26640c9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:47.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.612+0000 7f266a38c700 1 -- 172.21.15.78:0/1106372532 >> v1:172.21.15.78:6789/0 conn(0x7f26640cd210 legacy=0x7f26640cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:47.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.612+0000 7f266a38c700 1 -- 172.21.15.78:0/1106372532 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26640cccb0 con 0x7f266415c880 2023-10-11T09:48:47.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.612+0000 7f266ab8d700 1 --2- 172.21.15.78:0/1106372532 >> v2:172.21.15.133:3300/0 conn(0x7f26640cbd30 0x7f26640c9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:48:47.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.612+0000 7f266a38c700 1 --2- 172.21.15.78:0/1106372532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f266415c880 0x7f266406be60 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f26640cc640 tx=0x7f2660009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cdb42b40a83c348 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:47.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.612+0000 7f266938a700 1 -- 172.21.15.78:0/1106372532 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2660011830 con 0x7f266415c880 2023-10-11T09:48:47.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.612+0000 7f266938a700 1 -- 172.21.15.78:0/1106372532 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2660011990 con 0x7f266415c880 2023-10-11T09:48:47.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.612+0000 7f266938a700 1 -- 172.21.15.78:0/1106372532 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2660011d20 con 0x7f266415c880 2023-10-11T09:48:47.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.612+0000 7f266c5f0700 1 -- 172.21.15.78:0/1106372532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f266415c880 msgr2=0x7f266406be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:47.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.612+0000 7f266c5f0700 1 --2- 172.21.15.78:0/1106372532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f266415c880 0x7f266406be60 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f26640cc640 tx=0x7f2660009b30 comp rx=0 tx=0).stop 2023-10-11T09:48:47.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.612+0000 7f266c5f0700 1 -- 172.21.15.78:0/1106372532 shutdown_connections 2023-10-11T09:48:47.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.612+0000 7f266c5f0700 1 --2- 172.21.15.78:0/1106372532 >> v2:172.21.15.133:3300/0 conn(0x7f26640cbd30 0x7f26640c9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:47.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.612+0000 7f266c5f0700 1 --2- 172.21.15.78:0/1106372532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f266415c880 0x7f266406be60 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:47.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.612+0000 7f266c5f0700 1 -- 172.21.15.78:0/1106372532 >> 172.21.15.78:0/1106372532 conn(0x7f266406aca0 msgr2=0x7f26640b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:47.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.612+0000 7f266c5f0700 1 -- 172.21.15.78:0/1106372532 shutdown_connections 2023-10-11T09:48:47.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.616+0000 7f266c5f0700 1 -- 172.21.15.78:0/1106372532 wait complete. 2023-10-11T09:48:47.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.616+0000 7f266c5f0700 1 Processor -- start 2023-10-11T09:48:47.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.616+0000 7f266c5f0700 1 -- start start 2023-10-11T09:48:47.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.620+0000 7f266c5f0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26640cbd30 0x7f26640b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:47.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.620+0000 7f266c5f0700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f26640cd210 0x7f26641501e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:47.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.620+0000 7f266c5f0700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f266415c880 0x7f26640ae850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:47.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.620+0000 7f266c5f0700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26640b1d30 con 0x7f26640cbd30 2023-10-11T09:48:47.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.620+0000 7f266c5f0700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26640af560 con 0x7f266415c880 2023-10-11T09:48:47.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.620+0000 7f266c5f0700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26640af840 con 0x7f26640cd210 2023-10-11T09:48:47.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.620+0000 7f266a38c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26640cbd30 0x7f26640b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:47.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.620+0000 7f266a38c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26640cbd30 0x7f26640b46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35506/0 (socket says 172.21.15.78:35506) 2023-10-11T09:48:47.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.620+0000 7f266a38c700 1 -- 172.21.15.78:0/2095864109 learned_addr learned my addr 172.21.15.78:0/2095864109 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:47.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.620+0000 7f266ab8d700 1 --2- 172.21.15.78:0/2095864109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f266415c880 0x7f26640ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:47.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.620+0000 7f266a38c700 1 -- 172.21.15.78:0/2095864109 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f26640cd210 msgr2=0x7f26641501e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:47.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.620+0000 7f266a38c700 1 --2- 172.21.15.78:0/2095864109 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f26640cd210 0x7f26641501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:47.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.620+0000 7f266a38c700 1 -- 172.21.15.78:0/2095864109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f266415c880 msgr2=0x7f26640ae850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:47.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.620+0000 7f266a38c700 1 --2- 172.21.15.78:0/2095864109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f266415c880 0x7f26640ae850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:47.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.620+0000 7f266a38c700 1 -- 172.21.15.78:0/2095864109 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f265c009a70 con 0x7f26640cbd30 2023-10-11T09:48:47.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.620+0000 7f266a38c700 1 --2- 172.21.15.78:0/2095864109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26640cbd30 0x7f26640b46a0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f2660014040 tx=0x7f2660004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:47.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.620+0000 7f265affd700 1 -- 172.21.15.78:0/2095864109 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2660004610 con 0x7f26640cbd30 2023-10-11T09:48:47.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.620+0000 7f265affd700 1 -- 172.21.15.78:0/2095864109 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f266001e070 con 0x7f26640cbd30 2023-10-11T09:48:47.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.620+0000 7f266c5f0700 1 -- 172.21.15.78:0/2095864109 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26600094b0 con 0x7f26640cbd30 2023-10-11T09:48:47.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.620+0000 7f265affd700 1 -- 172.21.15.78:0/2095864109 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26600038e0 con 0x7f26640cbd30 2023-10-11T09:48:47.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.620+0000 7f266c5f0700 1 -- 172.21.15.78:0/2095864109 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26640afe00 con 0x7f26640cbd30 2023-10-11T09:48:47.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.624+0000 7f266c5f0700 1 -- 172.21.15.78:0/2095864109 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26640b1ea0 con 0x7f26640cbd30 2023-10-11T09:48:47.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.624+0000 7f265affd700 1 -- 172.21.15.78:0/2095864109 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f2660003a40 con 0x7f26640cbd30 2023-10-11T09:48:47.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.624+0000 7f265affd700 1 --2- 172.21.15.78:0/2095864109 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2648065ed0 0x7f2648068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:47.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.624+0000 7f265affd700 1 -- 172.21.15.78:0/2095864109 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6638+0+0 (secure 0 0 0) 0x7f2660084a20 con 0x7f26640cbd30 2023-10-11T09:48:47.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.624+0000 7f2669b8b700 1 --2- 172.21.15.78:0/2095864109 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2648065ed0 0x7f2648068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:47.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.624+0000 7f2669b8b700 1 --2- 172.21.15.78:0/2095864109 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2648065ed0 0x7f2648068380 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f26640c8810 tx=0x7f26540079e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:47.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.628+0000 7f265affd700 1 -- 172.21.15.78:0/2095864109 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f2660051e20 con 0x7f26640cbd30 2023-10-11T09:48:47.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.836+0000 7f266c5f0700 1 -- 172.21.15.78:0/2095864109 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f266404e530 con 0x7f26640cbd30 2023-10-11T09:48:47.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.836+0000 7f265affd700 1 -- 172.21.15.78:0/2095864109 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4648 (secure 0 0 0) 0x7f2660055440 con 0x7f26640cbd30 2023-10-11T09:48:47.841 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:47.841 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":64,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6961,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/1080544187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1080544187},{"type":"v1","addr":"172.21.15.133:6834","nonce":1080544187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/367194191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":367194191},{"type":"v1","addr":"172.21.15.133:6835","nonce":367194191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":7216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/2688829449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":2688829449},{"type":"v1","addr":"172.21.15.78:6837","nonce":2688829449}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":7533,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/1560261029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":1560261029},{"type":"v1","addr":"172.21.15.78:6836","nonce":1560261029}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:48:45.351822+0000","modified":"2023-10-11T09:48:46.494562+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[12,13],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-11T09:48:47.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.840+0000 7f266c5f0700 1 -- 172.21.15.78:0/2095864109 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2648065ed0 msgr2=0x7f2648068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:47.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.840+0000 7f266c5f0700 1 --2- 172.21.15.78:0/2095864109 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2648065ed0 0x7f2648068380 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f26640c8810 tx=0x7f26540079e0 comp rx=0 tx=0).stop 2023-10-11T09:48:47.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.840+0000 7f266c5f0700 1 -- 172.21.15.78:0/2095864109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26640cbd30 msgr2=0x7f26640b46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:47.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.840+0000 7f266c5f0700 1 --2- 172.21.15.78:0/2095864109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26640cbd30 0x7f26640b46a0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f2660014040 tx=0x7f2660004770 comp rx=0 tx=0).stop 2023-10-11T09:48:47.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.840+0000 7f266c5f0700 1 -- 172.21.15.78:0/2095864109 shutdown_connections 2023-10-11T09:48:47.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.840+0000 7f266c5f0700 1 --2- 172.21.15.78:0/2095864109 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2648065ed0 0x7f2648068380 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:47.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.840+0000 7f266c5f0700 1 --2- 172.21.15.78:0/2095864109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f266415c880 0x7f26640ae850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:47.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.840+0000 7f266c5f0700 1 --2- 172.21.15.78:0/2095864109 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f26640cd210 0x7f26641501e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:47.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.840+0000 7f266c5f0700 1 --2- 172.21.15.78:0/2095864109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26640cbd30 0x7f26640b46a0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:47.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.840+0000 7f266c5f0700 1 -- 172.21.15.78:0/2095864109 >> 172.21.15.78:0/2095864109 conn(0x7f266406aca0 msgr2=0x7f2664071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:47.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.840+0000 7f266c5f0700 1 -- 172.21.15.78:0/2095864109 shutdown_connections 2023-10-11T09:48:47.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:47.840+0000 7f266c5f0700 1 -- 172.21.15.78:0/2095864109 wait complete. 2023-10-11T09:48:47.847 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 64 2023-10-11T09:48:47.865 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:48:48.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.024+0000 7f34eb1e8700 1 Processor -- start 2023-10-11T09:48:48.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.024+0000 7f34eb1e8700 1 -- start start 2023-10-11T09:48:48.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.028+0000 7f34eb1e8700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f34e415b8e0 0x7f34e406d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:48.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.028+0000 7f34eb1e8700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f34e406d610 0x7f34e40ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:48.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.028+0000 7f34eb1e8700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f34e415a940 0x7f34e40ca780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:48.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.028+0000 7f34eb1e8700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34e415e010 con 0x7f34e406d610 2023-10-11T09:48:48.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.028+0000 7f34eb1e8700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34e415e280 con 0x7f34e415b8e0 2023-10-11T09:48:48.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.028+0000 7f34eb1e8700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34e415e560 con 0x7f34e415a940 2023-10-11T09:48:48.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.028+0000 7f34e99e5700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f34e406d610 0x7f34e40ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:48.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.028+0000 7f34e99e5700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f34e406d610 0x7f34e40ca240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35520/0 (socket says 172.21.15.78:35520) 2023-10-11T09:48:48.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.028+0000 7f34e99e5700 1 -- 172.21.15.78:0/3462786246 learned_addr learned my addr 172.21.15.78:0/3462786246 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:48.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.028+0000 7f34ea1e6700 1 --2- 172.21.15.78:0/3462786246 >> v2:172.21.15.133:3300/0 conn(0x7f34e415b8e0 0x7f34e406d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:48.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.028+0000 7f34ea9e7700 1 --2- 172.21.15.78:0/3462786246 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f34e415a940 0x7f34e40ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:48.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.028+0000 7f34e99e5700 1 -- 172.21.15.78:0/3462786246 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f34e415a940 msgr2=0x7f34e40ca780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:48.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.028+0000 7f34e99e5700 1 --2- 172.21.15.78:0/3462786246 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f34e415a940 0x7f34e40ca780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.028+0000 7f34e99e5700 1 -- 172.21.15.78:0/3462786246 >> v2:172.21.15.133:3300/0 conn(0x7f34e415b8e0 msgr2=0x7f34e406d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:48.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.028+0000 7f34e99e5700 1 --2- 172.21.15.78:0/3462786246 >> v2:172.21.15.133:3300/0 conn(0x7f34e415b8e0 0x7f34e406d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.028+0000 7f34e99e5700 1 -- 172.21.15.78:0/3462786246 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e415e840 con 0x7f34e406d610 2023-10-11T09:48:48.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.028+0000 7f34e99e5700 1 --2- 172.21.15.78:0/3462786246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34e406d610 0x7f34e40ca240 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f34e40b4cd0 tx=0x7f34d4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc05660c3ccb053b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:48.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.028+0000 7f34e91e4700 1 -- 172.21.15.78:0/3462786246 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d4011830 con 0x7f34e406d610 2023-10-11T09:48:48.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.028+0000 7f34e91e4700 1 -- 172.21.15.78:0/3462786246 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d4011990 con 0x7f34e406d610 2023-10-11T09:48:48.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.028+0000 7f34e91e4700 1 -- 172.21.15.78:0/3462786246 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d4011d20 con 0x7f34e406d610 2023-10-11T09:48:48.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.028+0000 7f34eb1e8700 1 -- 172.21.15.78:0/3462786246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34e406d610 msgr2=0x7f34e40ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:48.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.028+0000 7f34eb1e8700 1 --2- 172.21.15.78:0/3462786246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34e406d610 0x7f34e40ca240 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f34e40b4cd0 tx=0x7f34d4009bf0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.032+0000 7f34eb1e8700 1 -- 172.21.15.78:0/3462786246 shutdown_connections 2023-10-11T09:48:48.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.032+0000 7f34eb1e8700 1 --2- 172.21.15.78:0/3462786246 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f34e415a940 0x7f34e40ca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.032+0000 7f34eb1e8700 1 --2- 172.21.15.78:0/3462786246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34e406d610 0x7f34e40ca240 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.032+0000 7f34eb1e8700 1 --2- 172.21.15.78:0/3462786246 >> v2:172.21.15.133:3300/0 conn(0x7f34e415b8e0 0x7f34e406d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.032+0000 7f34eb1e8700 1 -- 172.21.15.78:0/3462786246 >> 172.21.15.78:0/3462786246 conn(0x7f34e40cea60 msgr2=0x7f34e40cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:48.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.032+0000 7f34eb1e8700 1 -- 172.21.15.78:0/3462786246 shutdown_connections 2023-10-11T09:48:48.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.032+0000 7f34eb1e8700 1 -- 172.21.15.78:0/3462786246 wait complete. 2023-10-11T09:48:48.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.036+0000 7f34eb1e8700 1 Processor -- start 2023-10-11T09:48:48.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.036+0000 7f34eb1e8700 1 -- start start 2023-10-11T09:48:48.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.036+0000 7f34eb1e8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34e406d610 0x7f34e40cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:48.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.036+0000 7f34eb1e8700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f34e415a940 0x7f34e40cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:48.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.036+0000 7f34eb1e8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34e415b8e0 0x7f34e4154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:48.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.036+0000 7f34eb1e8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e4142b50 con 0x7f34e406d610 2023-10-11T09:48:48.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.036+0000 7f34eb1e8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e4142da0 con 0x7f34e415b8e0 2023-10-11T09:48:48.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.036+0000 7f34eb1e8700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34e40cdc50 con 0x7f34e415a940 2023-10-11T09:48:48.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.036+0000 7f34ea1e6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34e406d610 0x7f34e40cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:48.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.036+0000 7f34ea1e6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34e406d610 0x7f34e40cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35536/0 (socket says 172.21.15.78:35536) 2023-10-11T09:48:48.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.036+0000 7f34ea1e6700 1 -- 172.21.15.78:0/2512204981 learned_addr learned my addr 172.21.15.78:0/2512204981 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:48.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.036+0000 7f34ea9e7700 1 --2- 172.21.15.78:0/2512204981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34e415b8e0 0x7f34e4154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:48.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.036+0000 7f34e99e5700 1 --2- 172.21.15.78:0/2512204981 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f34e415a940 0x7f34e40cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:48.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.036+0000 7f34ea1e6700 1 -- 172.21.15.78:0/2512204981 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f34e415a940 msgr2=0x7f34e40cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:48.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.036+0000 7f34ea1e6700 1 --2- 172.21.15.78:0/2512204981 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f34e415a940 0x7f34e40cbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.036+0000 7f34ea1e6700 1 -- 172.21.15.78:0/2512204981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34e415b8e0 msgr2=0x7f34e4154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:48.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.036+0000 7f34ea1e6700 1 --2- 172.21.15.78:0/2512204981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34e415b8e0 0x7f34e4154310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.036+0000 7f34ea1e6700 1 -- 172.21.15.78:0/2512204981 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d4009570 con 0x7f34e406d610 2023-10-11T09:48:48.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.040+0000 7f34ea1e6700 1 --2- 172.21.15.78:0/2512204981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34e406d610 0x7f34e40cb5e0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f34e000e9d0 tx=0x7f34e000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:48.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.040+0000 7f34daffd700 1 -- 172.21.15.78:0/2512204981 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e0009720 con 0x7f34e406d610 2023-10-11T09:48:48.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.040+0000 7f34daffd700 1 -- 172.21.15.78:0/2512204981 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e000c850 con 0x7f34e406d610 2023-10-11T09:48:48.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.040+0000 7f34eb1e8700 1 -- 172.21.15.78:0/2512204981 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34e40cdf30 con 0x7f34e406d610 2023-10-11T09:48:48.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.040+0000 7f34daffd700 1 -- 172.21.15.78:0/2512204981 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e000cbc0 con 0x7f34e406d610 2023-10-11T09:48:48.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.040+0000 7f34eb1e8700 1 -- 172.21.15.78:0/2512204981 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34e40b3db0 con 0x7f34e406d610 2023-10-11T09:48:48.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.040+0000 7f34eb1e8700 1 -- 172.21.15.78:0/2512204981 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34e4065d40 con 0x7f34e406d610 2023-10-11T09:48:48.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.040+0000 7f34daffd700 1 -- 172.21.15.78:0/2512204981 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f34e000cd20 con 0x7f34e406d610 2023-10-11T09:48:48.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.044+0000 7f34daffd700 1 --2- 172.21.15.78:0/2512204981 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f34c8065ed0 0x7f34c8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:48.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.044+0000 7f34daffd700 1 -- 172.21.15.78:0/2512204981 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6638+0+0 (secure 0 0 0) 0x7f34e0083ab0 con 0x7f34e406d610 2023-10-11T09:48:48.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.044+0000 7f34e99e5700 1 --2- 172.21.15.78:0/2512204981 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f34c8065ed0 0x7f34c8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:48.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.044+0000 7f34e99e5700 1 --2- 172.21.15.78:0/2512204981 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f34c8065ed0 0x7f34c8068380 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f34d4000c00 tx=0x7f34d4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:48.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.048+0000 7f34daffd700 1 -- 172.21.15.78:0/2512204981 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f34e0050e20 con 0x7f34e406d610 2023-10-11T09:48:48.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.260+0000 7f34eb1e8700 1 -- 172.21.15.78:0/2512204981 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34e41432f0 con 0x7f34e406d610 2023-10-11T09:48:48.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.260+0000 7f34daffd700 1 -- 172.21.15.78:0/2512204981 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4648 (secure 0 0 0) 0x7f34e0054440 con 0x7f34e406d610 2023-10-11T09:48:48.266 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:48.266 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":64,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6961,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/1080544187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1080544187},{"type":"v1","addr":"172.21.15.133:6834","nonce":1080544187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/367194191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":367194191},{"type":"v1","addr":"172.21.15.133:6835","nonce":367194191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":7216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/2688829449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":2688829449},{"type":"v1","addr":"172.21.15.78:6837","nonce":2688829449}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":7533,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/1560261029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":1560261029},{"type":"v1","addr":"172.21.15.78:6836","nonce":1560261029}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:48:45.351822+0000","modified":"2023-10-11T09:48:46.494562+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[12,13],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-11T09:48:48.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.264+0000 7f34eb1e8700 1 -- 172.21.15.78:0/2512204981 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f34c8065ed0 msgr2=0x7f34c8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:48.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.264+0000 7f34eb1e8700 1 --2- 172.21.15.78:0/2512204981 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f34c8065ed0 0x7f34c8068380 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f34d4000c00 tx=0x7f34d4008030 comp rx=0 tx=0).stop 2023-10-11T09:48:48.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.264+0000 7f34eb1e8700 1 -- 172.21.15.78:0/2512204981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34e406d610 msgr2=0x7f34e40cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:48.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.264+0000 7f34eb1e8700 1 --2- 172.21.15.78:0/2512204981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34e406d610 0x7f34e40cb5e0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f34e000e9d0 tx=0x7f34e000ece0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.264+0000 7f34eb1e8700 1 -- 172.21.15.78:0/2512204981 shutdown_connections 2023-10-11T09:48:48.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.264+0000 7f34eb1e8700 1 --2- 172.21.15.78:0/2512204981 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f34c8065ed0 0x7f34c8068380 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.264+0000 7f34eb1e8700 1 --2- 172.21.15.78:0/2512204981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34e415b8e0 0x7f34e4154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.264+0000 7f34eb1e8700 1 --2- 172.21.15.78:0/2512204981 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f34e415a940 0x7f34e40cbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.264+0000 7f34eb1e8700 1 --2- 172.21.15.78:0/2512204981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34e406d610 0x7f34e40cb5e0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.264+0000 7f34eb1e8700 1 -- 172.21.15.78:0/2512204981 >> 172.21.15.78:0/2512204981 conn(0x7f34e40cea60 msgr2=0x7f34e415c810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:48.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.264+0000 7f34eb1e8700 1 -- 172.21.15.78:0/2512204981 shutdown_connections 2023-10-11T09:48:48.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.264+0000 7f34eb1e8700 1 -- 172.21.15.78:0/2512204981 wait complete. 2023-10-11T09:48:48.271 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 64 2023-10-11T09:48:48.290 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:48:48.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.452+0000 7f0605578700 1 Processor -- start 2023-10-11T09:48:48.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.452+0000 7f0605578700 1 -- start start 2023-10-11T09:48:48.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.452+0000 7f0605578700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f06000ce500 0x7f060016bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:48.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.452+0000 7f0605578700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f060006ffa0 con 0x7f0600160d40 2023-10-11T09:48:48.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.452+0000 7f0605578700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0600070280 con 0x7f0600071040 2023-10-11T09:48:48.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.452+0000 7f0605578700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0600070c20 con 0x7f06000ce500 2023-10-11T09:48:48.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.452+0000 7f05ff7fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f06000ce500 0x7f060016bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:48.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.452+0000 7f05ff7fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f06000ce500 0x7f060016bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35548/0 (socket says 172.21.15.78:35548) 2023-10-11T09:48:48.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.452+0000 7f05ff7fe700 1 -- 172.21.15.78:0/3508563058 learned_addr learned my addr 172.21.15.78:0/3508563058 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:48.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.452+0000 7f05ff7fe700 1 -- 172.21.15.78:0/3508563058 >> v1:172.21.15.133:6789/0 conn(0x7f0600071040 legacy=0x7f06000cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:48.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.452+0000 7f05ff7fe700 1 -- 172.21.15.78:0/3508563058 >> v1:172.21.15.78:6789/0 conn(0x7f0600160d40 legacy=0x7f06000cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:48.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.452+0000 7f05ff7fe700 1 -- 172.21.15.78:0/3508563058 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0600070560 con 0x7f06000ce500 2023-10-11T09:48:48.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.456+0000 7f05ff7fe700 1 --2- 172.21.15.78:0/3508563058 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06000ce500 0x7f060016bb80 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f05f8009e50 tx=0x7f05f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68612cee6616bbc1 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:48.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.456+0000 7f05fe7fc700 1 -- 172.21.15.78:0/3508563058 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f800e070 con 0x7f06000ce500 2023-10-11T09:48:48.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.456+0000 7f05fe7fc700 1 -- 172.21.15.78:0/3508563058 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f800b9c0 con 0x7f06000ce500 2023-10-11T09:48:48.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.456+0000 7f05fe7fc700 1 -- 172.21.15.78:0/3508563058 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f800bd70 con 0x7f06000ce500 2023-10-11T09:48:48.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.456+0000 7f0605578700 1 -- 172.21.15.78:0/3508563058 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06000ce500 msgr2=0x7f060016bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:48.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.456+0000 7f0605578700 1 --2- 172.21.15.78:0/3508563058 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06000ce500 0x7f060016bb80 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f05f8009e50 tx=0x7f05f800b330 comp rx=0 tx=0).stop 2023-10-11T09:48:48.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.456+0000 7f0605578700 1 -- 172.21.15.78:0/3508563058 shutdown_connections 2023-10-11T09:48:48.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.456+0000 7f0605578700 1 --2- 172.21.15.78:0/3508563058 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06000ce500 0x7f060016bb80 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.456+0000 7f0605578700 1 -- 172.21.15.78:0/3508563058 >> 172.21.15.78:0/3508563058 conn(0x7f060006aca0 msgr2=0x7f06000b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:48.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.456+0000 7f0605578700 1 -- 172.21.15.78:0/3508563058 shutdown_connections 2023-10-11T09:48:48.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.456+0000 7f0605578700 1 -- 172.21.15.78:0/3508563058 wait complete. 2023-10-11T09:48:48.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.460+0000 7f0605578700 1 Processor -- start 2023-10-11T09:48:48.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.460+0000 7f0605578700 1 -- start start 2023-10-11T09:48:48.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.460+0000 7f0605578700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0600071040 0x7f06000b3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:48.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.460+0000 7f0605578700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f06000ce500 0x7f060014be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:48.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.460+0000 7f0605578700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0600160d40 0x7f06000c8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:48.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.460+0000 7f0605578700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06000b2170 con 0x7f0600071040 2023-10-11T09:48:48.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.460+0000 7f0605578700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06000b2450 con 0x7f0600160d40 2023-10-11T09:48:48.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.460+0000 7f0605578700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06000b2730 con 0x7f06000ce500 2023-10-11T09:48:48.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.460+0000 7f05feffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0600071040 0x7f06000b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:48.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.460+0000 7f05feffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0600071040 0x7f06000b3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35562/0 (socket says 172.21.15.78:35562) 2023-10-11T09:48:48.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.460+0000 7f05feffd700 1 -- 172.21.15.78:0/3467896370 learned_addr learned my addr 172.21.15.78:0/3467896370 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:48.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.460+0000 7f05ff7fe700 1 --2- 172.21.15.78:0/3467896370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0600160d40 0x7f06000c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:48.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.460+0000 7f05feffd700 1 -- 172.21.15.78:0/3467896370 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f06000ce500 msgr2=0x7f060014be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:48.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.460+0000 7f05feffd700 1 --2- 172.21.15.78:0/3467896370 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f06000ce500 0x7f060014be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.460+0000 7f05feffd700 1 -- 172.21.15.78:0/3467896370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0600160d40 msgr2=0x7f06000c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:48.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.460+0000 7f05feffd700 1 --2- 172.21.15.78:0/3467896370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0600160d40 0x7f06000c8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.460+0000 7f05feffd700 1 -- 172.21.15.78:0/3467896370 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f8009b30 con 0x7f0600071040 2023-10-11T09:48:48.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.460+0000 7f05ff7fe700 1 --2- 172.21.15.78:0/3467896370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0600160d40 0x7f06000c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:48:48.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.460+0000 7f05feffd700 1 --2- 172.21.15.78:0/3467896370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0600071040 0x7f06000b3210 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f05f000c500 tx=0x7f05f000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:48.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.460+0000 7f05f7fff700 1 -- 172.21.15.78:0/3467896370 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f00052d0 con 0x7f0600071040 2023-10-11T09:48:48.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.460+0000 7f05f7fff700 1 -- 172.21.15.78:0/3467896370 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f0005430 con 0x7f0600071040 2023-10-11T09:48:48.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.460+0000 7f05f7fff700 1 -- 172.21.15.78:0/3467896370 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f00055a0 con 0x7f0600071040 2023-10-11T09:48:48.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.460+0000 7f0605578700 1 -- 172.21.15.78:0/3467896370 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06000b2a10 con 0x7f0600071040 2023-10-11T09:48:48.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.464+0000 7f0605578700 1 -- 172.21.15.78:0/3467896370 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06000b3750 con 0x7f0600071040 2023-10-11T09:48:48.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.464+0000 7f0605578700 1 -- 172.21.15.78:0/3467896370 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0600072250 con 0x7f0600071040 2023-10-11T09:48:48.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.464+0000 7f05f7fff700 1 -- 172.21.15.78:0/3467896370 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f05f00038e0 con 0x7f0600071040 2023-10-11T09:48:48.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.468+0000 7f05f7fff700 1 --2- 172.21.15.78:0/3467896370 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f05e0065ed0 0x7f05e0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:48.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.468+0000 7f05f7fff700 1 -- 172.21.15.78:0/3467896370 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6638+0+0 (secure 0 0 0) 0x7f05f0082fd0 con 0x7f0600071040 2023-10-11T09:48:48.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.468+0000 7f05f67fc700 1 --2- 172.21.15.78:0/3467896370 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f05e0065ed0 0x7f05e0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:48.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.468+0000 7f05f67fc700 1 --2- 172.21.15.78:0/3467896370 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f05e0065ed0 0x7f05e0068380 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f05e8000f60 tx=0x7f05e8007a60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:48.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.476+0000 7f05f7fff700 1 -- 172.21.15.78:0/3467896370 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f05f0050350 con 0x7f0600071040 2023-10-11T09:48:48.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.696+0000 7f0605578700 1 -- 172.21.15.78:0/3467896370 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06000cf8f0 con 0x7f0600071040 2023-10-11T09:48:48.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.696+0000 7f05f7fff700 1 -- 172.21.15.78:0/3467896370 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4648 (secure 0 0 0) 0x7f05f0017090 con 0x7f0600071040 2023-10-11T09:48:48.702 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:48.703 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":64,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6961,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/1080544187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1080544187},{"type":"v1","addr":"172.21.15.133:6834","nonce":1080544187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/367194191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":367194191},{"type":"v1","addr":"172.21.15.133:6835","nonce":367194191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":7216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/2688829449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":2688829449},{"type":"v1","addr":"172.21.15.78:6837","nonce":2688829449}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":7533,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/1560261029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":1560261029},{"type":"v1","addr":"172.21.15.78:6836","nonce":1560261029}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:48:45.351822+0000","modified":"2023-10-11T09:48:46.494562+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[12,13],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-11T09:48:48.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.700+0000 7f0605578700 1 -- 172.21.15.78:0/3467896370 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f05e0065ed0 msgr2=0x7f05e0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:48.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.700+0000 7f0605578700 1 --2- 172.21.15.78:0/3467896370 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f05e0065ed0 0x7f05e0068380 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f05e8000f60 tx=0x7f05e8007a60 comp rx=0 tx=0).stop 2023-10-11T09:48:48.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.700+0000 7f0605578700 1 -- 172.21.15.78:0/3467896370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0600071040 msgr2=0x7f06000b3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:48.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.700+0000 7f0605578700 1 --2- 172.21.15.78:0/3467896370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0600071040 0x7f06000b3210 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f05f000c500 tx=0x7f05f000c8c0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.700+0000 7f0605578700 1 -- 172.21.15.78:0/3467896370 shutdown_connections 2023-10-11T09:48:48.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.700+0000 7f0605578700 1 --2- 172.21.15.78:0/3467896370 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f05e0065ed0 0x7f05e0068380 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.700+0000 7f0605578700 1 --2- 172.21.15.78:0/3467896370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0600160d40 0x7f06000c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.700+0000 7f0605578700 1 --2- 172.21.15.78:0/3467896370 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f06000ce500 0x7f060014be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.700+0000 7f0605578700 1 --2- 172.21.15.78:0/3467896370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0600071040 0x7f06000b3210 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.700+0000 7f0605578700 1 -- 172.21.15.78:0/3467896370 >> 172.21.15.78:0/3467896370 conn(0x7f060006aca0 msgr2=0x7f060006d120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:48.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.700+0000 7f0605578700 1 -- 172.21.15.78:0/3467896370 shutdown_connections 2023-10-11T09:48:48.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.700+0000 7f0605578700 1 -- 172.21.15.78:0/3467896370 wait complete. 2023-10-11T09:48:48.708 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 64 2023-10-11T09:48:48.729 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset session 2023-10-11T09:48:48.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.788+0000 7fe030c51380 1 -- start start 2023-10-11T09:48:48.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.788+0000 7fe030c51380 1 Processor -- start 2023-10-11T09:48:48.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.788+0000 7fe030c51380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55de31741800 0x55de317e6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:48.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.788+0000 7fe030c51380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de31741400 0x55de317e6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:48.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.788+0000 7fe030c51380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de31741000 0x55de317e7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:48.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.788+0000 7fe030c51380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de31777e00 con 0x55de31741400 2023-10-11T09:48:48.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.788+0000 7fe030c51380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de317c0a00 con 0x55de31741000 2023-10-11T09:48:48.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.788+0000 7fe030c51380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55de317c0640 con 0x55de31741800 2023-10-11T09:48:48.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.788+0000 7fe02ec0a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de31741400 0x55de317e6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:48.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.788+0000 7fe02ec0a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de31741400 0x55de317e6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35586/0 (socket says 172.21.15.78:35586) 2023-10-11T09:48:48.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.788+0000 7fe02ec0a700 1 -- 172.21.15.78:0/4179734229 learned_addr learned my addr 172.21.15.78:0/4179734229 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:48.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.788+0000 7fe02f40b700 1 --2- 172.21.15.78:0/4179734229 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55de31741800 0x55de317e6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:48.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.788+0000 7fe02fc0c700 1 --2- 172.21.15.78:0/4179734229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de31741000 0x55de317e7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:48.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.788+0000 7fe02ec0a700 1 -- 172.21.15.78:0/4179734229 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55de31741800 msgr2=0x55de317e6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:48.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.788+0000 7fe02ec0a700 1 --2- 172.21.15.78:0/4179734229 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55de31741800 0x55de317e6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.788+0000 7fe02ec0a700 1 -- 172.21.15.78:0/4179734229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de31741000 msgr2=0x55de317e7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:48.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.788+0000 7fe02ec0a700 1 --2- 172.21.15.78:0/4179734229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de31741000 0x55de317e7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.788+0000 7fe02ec0a700 1 -- 172.21.15.78:0/4179734229 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de31762b60 con 0x55de31741400 2023-10-11T09:48:48.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.788+0000 7fe02fc0c700 1 --2- 172.21.15.78:0/4179734229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de31741000 0x55de317e7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:48:48.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.792+0000 7fe02ec0a700 1 --2- 172.21.15.78:0/4179734229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de31741400 0x55de317e6b00 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x55de318a2ba0 tx=0x55de3251afc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dfc723ad96d9677 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:48.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.792+0000 7fe02cc06700 1 -- 172.21.15.78:0/4179734229 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55de31771600 con 0x55de31741400 2023-10-11T09:48:48.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.792+0000 7fe02cc06700 1 -- 172.21.15.78:0/4179734229 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55de3178c480 con 0x55de31741400 2023-10-11T09:48:48.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.792+0000 7fe02cc06700 1 -- 172.21.15.78:0/4179734229 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55de317711e0 con 0x55de31741400 2023-10-11T09:48:48.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.792+0000 7fe030c51380 1 -- 172.21.15.78:0/4179734229 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de31762340 con 0x55de31741400 2023-10-11T09:48:48.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.792+0000 7fe030c51380 1 -- 172.21.15.78:0/4179734229 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55de31762000 con 0x55de31741400 2023-10-11T09:48:48.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.792+0000 7fe030c51380 4 waiting for MDS map... 2023-10-11T09:48:48.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.792+0000 7fe02cc06700 1 -- 172.21.15.78:0/4179734229 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(88..88 src has 1..88) v4 ==== 6638+0+0 (secure 0 0 0) 0x55de3177ac40 con 0x55de31741400 2023-10-11T09:48:48.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.792+0000 7fe02cc06700 1 -- 172.21.15.78:0/4179734229 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 64) v1 ==== 3118+0+0 (secure 0 0 0) 0x55de325b8000 con 0x55de31741400 2023-10-11T09:48:48.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.792+0000 7fe030c51380 4 Got MDS map 64 2023-10-11T09:48:48.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.792+0000 7fe030c51380 10 main: main 2023-10-11T09:48:48.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.792+0000 7fe030c51380 4 main: connecting to RADOS... 2023-10-11T09:48:48.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.792+0000 7fe030c51380 1 Processor -- start 2023-10-11T09:48:48.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.792+0000 7fe030c51380 1 -- start start 2023-10-11T09:48:48.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.792+0000 7fe030c51380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de31a36400 0x55de317e7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:48.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.792+0000 7fe030c51380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55de31a36800 0x55de317e7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:48.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.792+0000 7fe030c51380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de31a36c00 0x55de317e8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:48.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.792+0000 7fe030c51380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de317c0500 con 0x55de31a36400 2023-10-11T09:48:48.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.792+0000 7fe030c51380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de317c03c0 con 0x55de31a36c00 2023-10-11T09:48:48.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.792+0000 7fe030c51380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55de317c0a00 con 0x55de31a36800 2023-10-11T09:48:48.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.792+0000 7fe02fc0c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de31a36400 0x55de317e7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:48.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.796+0000 7fe02fc0c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de31a36400 0x55de317e7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35592/0 (socket says 172.21.15.78:35592) 2023-10-11T09:48:48.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.796+0000 7fe02fc0c700 1 -- 172.21.15.78:0/1606986105 learned_addr learned my addr 172.21.15.78:0/1606986105 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:48.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.796+0000 7fe02ec0a700 1 --2- 172.21.15.78:0/1606986105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de31a36c00 0x55de317e8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:48.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.796+0000 7fe02f40b700 1 --2- 172.21.15.78:0/1606986105 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55de31a36800 0x55de317e7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:48.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.796+0000 7fe02fc0c700 1 -- 172.21.15.78:0/1606986105 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55de31a36800 msgr2=0x55de317e7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:48.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.796+0000 7fe02fc0c700 1 --2- 172.21.15.78:0/1606986105 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55de31a36800 0x55de317e7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.796+0000 7fe02fc0c700 1 -- 172.21.15.78:0/1606986105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de31a36c00 msgr2=0x55de317e8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:48.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.796+0000 7fe02fc0c700 1 --2- 172.21.15.78:0/1606986105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de31a36c00 0x55de317e8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.796+0000 7fe02fc0c700 1 -- 172.21.15.78:0/1606986105 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de31762ea0 con 0x55de31a36400 2023-10-11T09:48:48.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.796+0000 7fe02fc0c700 1 --2- 172.21.15.78:0/1606986105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de31a36400 0x55de317e7600 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x55de31a860f0 tx=0x55de3251a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cc0da737f889f08 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:48.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.796+0000 7fe02ac02700 1 -- 172.21.15.78:0/1606986105 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55de31771080 con 0x55de31a36400 2023-10-11T09:48:48.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.796+0000 7fe02ac02700 1 -- 172.21.15.78:0/1606986105 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55de32528900 con 0x55de31a36400 2023-10-11T09:48:48.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.796+0000 7fe02ac02700 1 -- 172.21.15.78:0/1606986105 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55de31771600 con 0x55de31a36400 2023-10-11T09:48:48.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.796+0000 7fe030c51380 1 -- 172.21.15.78:0/1606986105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de31a36400 msgr2=0x55de317e7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:48.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.796+0000 7fe030c51380 1 --2- 172.21.15.78:0/1606986105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de31a36400 0x55de317e7600 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x55de31a860f0 tx=0x55de3251a000 comp rx=0 tx=0).stop 2023-10-11T09:48:48.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.796+0000 7fe030c51380 1 -- 172.21.15.78:0/1606986105 shutdown_connections 2023-10-11T09:48:48.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.796+0000 7fe030c51380 1 --2- 172.21.15.78:0/1606986105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de31a36c00 0x55de317e8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.796+0000 7fe030c51380 1 --2- 172.21.15.78:0/1606986105 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55de31a36800 0x55de317e7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.796+0000 7fe030c51380 1 --2- 172.21.15.78:0/1606986105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de31a36400 0x55de317e7600 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.796+0000 7fe030c51380 1 -- 172.21.15.78:0/1606986105 >> 172.21.15.78:0/1606986105 conn(0x55de31740c00 msgr2=0x55de3182d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:48.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.796+0000 7fe030c51380 1 -- 172.21.15.78:0/1606986105 shutdown_connections 2023-10-11T09:48:48.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.796+0000 7fe030c51380 1 -- 172.21.15.78:0/1606986105 wait complete. 2023-10-11T09:48:48.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.796+0000 7fe030c51380 1 Processor -- start 2023-10-11T09:48:48.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.796+0000 7fe030c51380 1 -- start start 2023-10-11T09:48:48.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.800+0000 7fe030c51380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de31a36400 0x55de317e9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:48.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.800+0000 7fe030c51380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55de31a36800 0x55de317e8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:48.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.800+0000 7fe030c51380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de31a36c00 0x55de317e8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:48.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.800+0000 7fe030c51380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de317c0280 con 0x55de31a36400 2023-10-11T09:48:48.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.800+0000 7fe030c51380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de317c0140 con 0x55de31a36c00 2023-10-11T09:48:48.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.800+0000 7fe030c51380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55de317c0000 con 0x55de31a36800 2023-10-11T09:48:48.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.800+0000 7fe02fc0c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de31a36400 0x55de317e9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:48.818 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.800+0000 7fe02fc0c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de31a36400 0x55de317e9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35596/0 (socket says 172.21.15.78:35596) 2023-10-11T09:48:48.818 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.800+0000 7fe02fc0c700 1 -- 172.21.15.78:0/3272740764 learned_addr learned my addr 172.21.15.78:0/3272740764 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:48.818 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.800+0000 7fe02ec0a700 1 --2- 172.21.15.78:0/3272740764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de31a36c00 0x55de317e8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:48.818 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.800+0000 7fe02f40b700 1 --2- 172.21.15.78:0/3272740764 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55de31a36800 0x55de317e8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:48.818 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.800+0000 7fe02fc0c700 1 -- 172.21.15.78:0/3272740764 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55de31a36800 msgr2=0x55de317e8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:48.819 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.800+0000 7fe02fc0c700 1 --2- 172.21.15.78:0/3272740764 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55de31a36800 0x55de317e8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.819 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.800+0000 7fe02fc0c700 1 -- 172.21.15.78:0/3272740764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de31a36c00 msgr2=0x55de317e8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:48.819 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.800+0000 7fe02fc0c700 1 --2- 172.21.15.78:0/3272740764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de31a36c00 0x55de317e8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.819 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.800+0000 7fe02fc0c700 1 -- 172.21.15.78:0/3272740764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de317631e0 con 0x55de31a36400 2023-10-11T09:48:48.819 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.800+0000 7fe02ec0a700 1 --2- 172.21.15.78:0/3272740764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de31a36c00 0x55de317e8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:48:48.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.800+0000 7fe02fc0c700 1 --2- 172.21.15.78:0/3272740764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de31a36400 0x55de317e9180 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x55de318a27b0 tx=0x55de3251a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:48.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.800+0000 7fe0293ff700 1 -- 172.21.15.78:0/3272740764 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55de317711e0 con 0x55de31a36400 2023-10-11T09:48:48.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.800+0000 7fe0293ff700 1 -- 172.21.15.78:0/3272740764 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55de3178c900 con 0x55de31a36400 2023-10-11T09:48:48.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.800+0000 7fe030c51380 1 -- 172.21.15.78:0/3272740764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55de31762ea0 con 0x55de31a36400 2023-10-11T09:48:48.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.800+0000 7fe0293ff700 1 -- 172.21.15.78:0/3272740764 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55de31770dc0 con 0x55de31a36400 2023-10-11T09:48:48.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.800+0000 7fe030c51380 1 -- 172.21.15.78:0/3272740764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de31762b60 con 0x55de31a36400 2023-10-11T09:48:48.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.800+0000 7fe030c51380 4 main: resolving pool 11 2023-10-11T09:48:48.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.804+0000 7fe0293ff700 1 -- 172.21.15.78:0/3272740764 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x55de317be840 con 0x55de31a36400 2023-10-11T09:48:48.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.804+0000 7fe0293ff700 1 --2- 172.21.15.78:0/3272740764 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55de31b82000 0x55de317e9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:48.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.804+0000 7fe0293ff700 1 -- 172.21.15.78:0/3272740764 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6638+0+0 (secure 0 0 0) 0x55de3177aa80 con 0x55de31a36400 2023-10-11T09:48:48.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.804+0000 7fe030c51380 4 main: creating IoCtx.. 2023-10-11T09:48:48.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.804+0000 7fe030c51380 1 --2- 172.21.15.78:0/3272740764 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55de31a37000 0x55de317e8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:48.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.804+0000 7fe030c51380 1 -- 172.21.15.78:0/3272740764 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:1 11.b 11:d0630e4c:::mds0_sessionmap:head [omap-clear,omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x55de31a37400 con 0x55de31a37000 2023-10-11T09:48:48.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.804+0000 7fe02f40b700 1 --2- 172.21.15.78:0/3272740764 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55de31b82000 0x55de317e9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:48.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.804+0000 7fe02ec0a700 1 --2- 172.21.15.78:0/3272740764 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55de31a37000 0x55de317e8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:48.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.804+0000 7fe02f40b700 1 --2- 172.21.15.78:0/3272740764 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55de31b82000 0x55de317e9700 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x55de318a2270 tx=0x55de31a3bc80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:48.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.804+0000 7fe02ec0a700 1 --2- 172.21.15.78:0/3272740764 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55de31a37000 0x55de317e8100 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:48.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.808+0000 7fe02ec0a700 1 -- 172.21.15.78:0/3272740764 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(1 mds0_sessionmap [omap-clear,omap-set-header] v88'5 uv5 ondisk = 0) v8 ==== 201+0+0 (crc 0 0 0) 0x55de318a6fc0 con 0x55de31a37000 2023-10-11T09:48:48.823 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:48:48.824 INFO:teuthology.orchestra.run.smithi078.stdout: "0": { 2023-10-11T09:48:48.824 INFO:teuthology.orchestra.run.smithi078.stdout: "data": {}, 2023-10-11T09:48:48.824 INFO:teuthology.orchestra.run.smithi078.stdout: "result": 0 2023-10-11T09:48:48.824 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:48:48.824 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:48:48.825 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:48.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.808+0000 7fe030c51380 1 -- 172.21.15.78:0/3272740764 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55de31a37000 msgr2=0x55de317e8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:48.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.808+0000 7fe030c51380 1 --2- 172.21.15.78:0/3272740764 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55de31a37000 0x55de317e8100 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.808+0000 7fe030c51380 1 -- 172.21.15.78:0/3272740764 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55de31b82000 msgr2=0x55de317e9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:48.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.808+0000 7fe030c51380 1 --2- 172.21.15.78:0/3272740764 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55de31b82000 0x55de317e9700 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x55de318a2270 tx=0x55de31a3bc80 comp rx=0 tx=0).stop 2023-10-11T09:48:48.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.808+0000 7fe030c51380 1 -- 172.21.15.78:0/3272740764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de31a36400 msgr2=0x55de317e9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:48.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.808+0000 7fe030c51380 1 --2- 172.21.15.78:0/3272740764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de31a36400 0x55de317e9180 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x55de318a27b0 tx=0x55de3251a1e0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.808+0000 7fe02f40b700 1 -- 172.21.15.78:0/3272740764 reap_dead start 2023-10-11T09:48:48.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.808+0000 7fe030c51380 1 -- 172.21.15.78:0/3272740764 shutdown_connections 2023-10-11T09:48:48.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.808+0000 7fe030c51380 1 -- 172.21.15.78:0/3272740764 >> 172.21.15.78:0/3272740764 conn(0x55de31740c00 msgr2=0x55de3182d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:48.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.808+0000 7fe030c51380 1 -- 172.21.15.78:0/3272740764 shutdown_connections 2023-10-11T09:48:48.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.808+0000 7fe030c51380 1 -- 172.21.15.78:0/3272740764 wait complete. 2023-10-11T09:48:48.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.808+0000 7fe030c51380 1 -- 172.21.15.78:0/4179734229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de31741400 msgr2=0x55de317e6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:48.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.808+0000 7fe030c51380 1 --2- 172.21.15.78:0/4179734229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de31741400 0x55de317e6b00 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x55de318a2ba0 tx=0x55de3251afc0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.808+0000 7fe030c51380 1 -- 172.21.15.78:0/4179734229 shutdown_connections 2023-10-11T09:48:48.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.808+0000 7fe030c51380 1 --2- 172.21.15.78:0/4179734229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de31741000 0x55de317e7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.808+0000 7fe030c51380 1 --2- 172.21.15.78:0/4179734229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de31741400 0x55de317e6b00 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.808+0000 7fe030c51380 1 --2- 172.21.15.78:0/4179734229 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55de31741800 0x55de317e6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.808+0000 7fe030c51380 1 -- 172.21.15.78:0/4179734229 >> 172.21.15.78:0/4179734229 conn(0x55de31741c00 msgr2=0x55de3182d000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:48.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.808+0000 7fe030c51380 1 -- 172.21.15.78:0/4179734229 shutdown_connections 2023-10-11T09:48:48.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.808+0000 7fe030c51380 1 -- 172.21.15.78:0/4179734229 wait complete. 2023-10-11T09:48:48.831 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'session'] in time 0:00:00.102093, result: { "0": { "data": {}, "result": 0 } } 2023-10-11T09:48:48.831 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset snap 2023-10-11T09:48:48.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.888+0000 7fdd0ff00380 1 -- start start 2023-10-11T09:48:48.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.888+0000 7fdd0ff00380 1 Processor -- start 2023-10-11T09:48:48.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.888+0000 7fdd0ff00380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556aa00c1800 0x556aa0166580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:48.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.888+0000 7fdd0ff00380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa00c1400 0x556aa0166b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:48.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.888+0000 7fdd0ff00380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556aa00c1000 0x556aa0167080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:48.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.888+0000 7fdd0ff00380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aa0140a00 con 0x556aa00c1400 2023-10-11T09:48:48.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.888+0000 7fdd0ff00380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aa0140780 con 0x556aa00c1000 2023-10-11T09:48:48.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.888+0000 7fdd0ff00380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x556aa0140640 con 0x556aa00c1800 2023-10-11T09:48:48.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.888+0000 7fdd0deb9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa00c1400 0x556aa0166b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:48.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.888+0000 7fdd0deb9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa00c1400 0x556aa0166b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35622/0 (socket says 172.21.15.78:35622) 2023-10-11T09:48:48.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.888+0000 7fdd0deb9700 1 -- 172.21.15.78:0/2495627332 learned_addr learned my addr 172.21.15.78:0/2495627332 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:48.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.888+0000 7fdd0eebb700 1 --2- 172.21.15.78:0/2495627332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556aa00c1000 0x556aa0167080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:48.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.888+0000 7fdd0e6ba700 1 --2- 172.21.15.78:0/2495627332 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556aa00c1800 0x556aa0166580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:48.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.888+0000 7fdd0deb9700 1 -- 172.21.15.78:0/2495627332 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556aa00c1800 msgr2=0x556aa0166580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:48.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.888+0000 7fdd0deb9700 1 --2- 172.21.15.78:0/2495627332 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556aa00c1800 0x556aa0166580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.888+0000 7fdd0deb9700 1 -- 172.21.15.78:0/2495627332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556aa00c1000 msgr2=0x556aa0167080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:48.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.888+0000 7fdd0deb9700 1 --2- 172.21.15.78:0/2495627332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556aa00c1000 0x556aa0167080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.888+0000 7fdd0deb9700 1 -- 172.21.15.78:0/2495627332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556aa00e2b60 con 0x556aa00c1400 2023-10-11T09:48:48.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.888+0000 7fdd0deb9700 1 --2- 172.21.15.78:0/2495627332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa00c1400 0x556aa0166b00 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x556aa0e9cb70 tx=0x556aa0ea6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b123d6230af29202 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:48.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.888+0000 7fdd0beb5700 1 -- 172.21.15.78:0/2495627332 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556aa00f1080 con 0x556aa00c1400 2023-10-11T09:48:48.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.888+0000 7fdd0ff00380 1 -- 172.21.15.78:0/2495627332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556aa00e31e0 con 0x556aa00c1400 2023-10-11T09:48:48.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.888+0000 7fdd0beb5700 1 -- 172.21.15.78:0/2495627332 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556aa0ebc300 con 0x556aa00c1400 2023-10-11T09:48:48.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.892+0000 7fdd0ff00380 1 -- 172.21.15.78:0/2495627332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x556aa00e2b60 con 0x556aa00c1400 2023-10-11T09:48:48.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.892+0000 7fdd0ff00380 4 waiting for MDS map... 2023-10-11T09:48:48.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.892+0000 7fdd0beb5700 1 -- 172.21.15.78:0/2495627332 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556aa00f1600 con 0x556aa00c1400 2023-10-11T09:48:48.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.892+0000 7fdd0beb5700 1 -- 172.21.15.78:0/2495627332 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(88..88 src has 1..88) v4 ==== 6638+0+0 (secure 0 0 0) 0x556aa00fac40 con 0x556aa00c1400 2023-10-11T09:48:48.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.892+0000 7fdd0beb5700 1 -- 172.21.15.78:0/2495627332 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 64) v1 ==== 3118+0+0 (secure 0 0 0) 0x556aa0102f00 con 0x556aa00c1400 2023-10-11T09:48:48.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.892+0000 7fdd0ff00380 4 Got MDS map 64 2023-10-11T09:48:48.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.892+0000 7fdd0ff00380 10 main: main 2023-10-11T09:48:48.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.892+0000 7fdd0ff00380 4 main: connecting to RADOS... 2023-10-11T09:48:48.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.892+0000 7fdd0ff00380 1 Processor -- start 2023-10-11T09:48:48.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.892+0000 7fdd0ff00380 1 -- start start 2023-10-11T09:48:48.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.892+0000 7fdd0ff00380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556aa0ea4400 0x556aa0167600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:48.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.892+0000 7fdd0ff00380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556aa0ea4800 0x556aa0167b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:48.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.892+0000 7fdd0ff00380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0ea4c00 0x556aa0168100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:48.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.892+0000 7fdd0ff00380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aa00f7e00 con 0x556aa0ea4c00 2023-10-11T09:48:48.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.892+0000 7fdd0ff00380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aa0140780 con 0x556aa0ea4400 2023-10-11T09:48:48.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.892+0000 7fdd0ff00380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x556aa0140a00 con 0x556aa0ea4800 2023-10-11T09:48:48.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.892+0000 7fdd0deb9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0ea4c00 0x556aa0168100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:48.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.892+0000 7fdd0deb9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0ea4c00 0x556aa0168100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35630/0 (socket says 172.21.15.78:35630) 2023-10-11T09:48:48.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.892+0000 7fdd0deb9700 1 -- 172.21.15.78:0/3738681469 learned_addr learned my addr 172.21.15.78:0/3738681469 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:48.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.896+0000 7fdd0e6ba700 1 --2- 172.21.15.78:0/3738681469 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556aa0ea4800 0x556aa0167b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:48.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.896+0000 7fdd0eebb700 1 --2- 172.21.15.78:0/3738681469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556aa0ea4400 0x556aa0167600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:48.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.896+0000 7fdd0deb9700 1 -- 172.21.15.78:0/3738681469 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556aa0ea4800 msgr2=0x556aa0167b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:48.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.896+0000 7fdd0deb9700 1 --2- 172.21.15.78:0/3738681469 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556aa0ea4800 0x556aa0167b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.896+0000 7fdd0deb9700 1 -- 172.21.15.78:0/3738681469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556aa0ea4400 msgr2=0x556aa0167600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:48.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.896+0000 7fdd0deb9700 1 --2- 172.21.15.78:0/3738681469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556aa0ea4400 0x556aa0167600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.896+0000 7fdd0deb9700 1 -- 172.21.15.78:0/3738681469 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556aa00e2340 con 0x556aa0ea4c00 2023-10-11T09:48:48.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.896+0000 7fdd0deb9700 1 --2- 172.21.15.78:0/3738681469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0ea4c00 0x556aa0168100 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x556aa0e9a0c0 tx=0x556aa0ea6060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=180fd7c40ffe04fd server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:48.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.896+0000 7fdd09eb1700 1 -- 172.21.15.78:0/3738681469 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556aa00f11e0 con 0x556aa0ea4c00 2023-10-11T09:48:48.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.896+0000 7fdd09eb1700 1 -- 172.21.15.78:0/3738681469 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556aa0ebcd80 con 0x556aa0ea4c00 2023-10-11T09:48:48.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.896+0000 7fdd09eb1700 1 -- 172.21.15.78:0/3738681469 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556aa00f0f20 con 0x556aa0ea4c00 2023-10-11T09:48:48.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.896+0000 7fdd0ff00380 1 -- 172.21.15.78:0/3738681469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0ea4c00 msgr2=0x556aa0168100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:48.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.896+0000 7fdd0ff00380 1 --2- 172.21.15.78:0/3738681469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0ea4c00 0x556aa0168100 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x556aa0e9a0c0 tx=0x556aa0ea6060 comp rx=0 tx=0).stop 2023-10-11T09:48:48.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.896+0000 7fdd0ff00380 1 -- 172.21.15.78:0/3738681469 shutdown_connections 2023-10-11T09:48:48.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.896+0000 7fdd0ff00380 1 --2- 172.21.15.78:0/3738681469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0ea4c00 0x556aa0168100 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.896+0000 7fdd0ff00380 1 --2- 172.21.15.78:0/3738681469 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556aa0ea4800 0x556aa0167b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.896+0000 7fdd0ff00380 1 --2- 172.21.15.78:0/3738681469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556aa0ea4400 0x556aa0167600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.896+0000 7fdd0ff00380 1 -- 172.21.15.78:0/3738681469 >> 172.21.15.78:0/3738681469 conn(0x556aa00c0c00 msgr2=0x556aa01ad800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:48.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.896+0000 7fdd0ff00380 1 -- 172.21.15.78:0/3738681469 shutdown_connections 2023-10-11T09:48:48.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.896+0000 7fdd0ff00380 1 -- 172.21.15.78:0/3738681469 wait complete. 2023-10-11T09:48:48.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.896+0000 7fdd0ff00380 1 Processor -- start 2023-10-11T09:48:48.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.896+0000 7fdd0ff00380 1 -- start start 2023-10-11T09:48:48.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.900+0000 7fdd0ff00380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556aa0ea4400 0x556aa0169180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:48.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.900+0000 7fdd0ff00380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0ea4800 0x556aa0168c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:48.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.900+0000 7fdd0ff00380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556aa0ea4c00 0x556aa0168680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:48.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.900+0000 7fdd0ff00380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aa0140500 con 0x556aa0ea4800 2023-10-11T09:48:48.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.900+0000 7fdd0ff00380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aa01403c0 con 0x556aa0ea4400 2023-10-11T09:48:48.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.900+0000 7fdd0ff00380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x556aa0140280 con 0x556aa0ea4c00 2023-10-11T09:48:48.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.900+0000 7fdd0e6ba700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0ea4800 0x556aa0168c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:48.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.900+0000 7fdd0e6ba700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0ea4800 0x556aa0168c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35634/0 (socket says 172.21.15.78:35634) 2023-10-11T09:48:48.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.900+0000 7fdd0e6ba700 1 -- 172.21.15.78:0/3605856714 learned_addr learned my addr 172.21.15.78:0/3605856714 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:48.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.900+0000 7fdd0eebb700 1 --2- 172.21.15.78:0/3605856714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556aa0ea4400 0x556aa0169180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:48.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.900+0000 7fdd0deb9700 1 --2- 172.21.15.78:0/3605856714 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556aa0ea4c00 0x556aa0168680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:48.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.900+0000 7fdd0e6ba700 1 -- 172.21.15.78:0/3605856714 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556aa0ea4c00 msgr2=0x556aa0168680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:48.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.900+0000 7fdd0e6ba700 1 --2- 172.21.15.78:0/3605856714 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556aa0ea4c00 0x556aa0168680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.900+0000 7fdd0e6ba700 1 -- 172.21.15.78:0/3605856714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556aa0ea4400 msgr2=0x556aa0169180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:48.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.900+0000 7fdd0e6ba700 1 --2- 172.21.15.78:0/3605856714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556aa0ea4400 0x556aa0169180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.900+0000 7fdd0e6ba700 1 -- 172.21.15.78:0/3605856714 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556aa00e31e0 con 0x556aa0ea4800 2023-10-11T09:48:48.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.900+0000 7fdd0deb9700 1 --2- 172.21.15.78:0/3605856714 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556aa0ea4c00 0x556aa0168680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:48:48.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.900+0000 7fdd0e6ba700 1 --2- 172.21.15.78:0/3605856714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0ea4800 0x556aa0168c00 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x556aa0e9af60 tx=0x556aa0eb8e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:48.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.900+0000 7fdd086ae700 1 -- 172.21.15.78:0/3605856714 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556aa00f11e0 con 0x556aa0ea4800 2023-10-11T09:48:48.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.900+0000 7fdd086ae700 1 -- 172.21.15.78:0/3605856714 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556aa010c180 con 0x556aa0ea4800 2023-10-11T09:48:48.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.900+0000 7fdd086ae700 1 -- 172.21.15.78:0/3605856714 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556aa00f0f20 con 0x556aa0ea4800 2023-10-11T09:48:48.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.900+0000 7fdd0ff00380 1 -- 172.21.15.78:0/3605856714 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556aa00e2b60 con 0x556aa0ea4800 2023-10-11T09:48:48.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.900+0000 7fdd0ff00380 1 -- 172.21.15.78:0/3605856714 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556aa00e2340 con 0x556aa0ea4800 2023-10-11T09:48:48.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.900+0000 7fdd0ff00380 4 main: resolving pool 11 2023-10-11T09:48:48.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.904+0000 7fdd086ae700 1 -- 172.21.15.78:0/3605856714 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x556aa013e840 con 0x556aa0ea4800 2023-10-11T09:48:48.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.904+0000 7fdd086ae700 1 --2- 172.21.15.78:0/3605856714 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x556aa04fc000 0x556aa0169700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:48.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.904+0000 7fdd086ae700 1 -- 172.21.15.78:0/3605856714 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6638+0+0 (secure 0 0 0) 0x556aa00faa80 con 0x556aa0ea4800 2023-10-11T09:48:48.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.904+0000 7fdd0ff00380 4 main: creating IoCtx.. 2023-10-11T09:48:48.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.904+0000 7fdd0ff00380 1 --2- 172.21.15.78:0/3605856714 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x556aa0ea5000 0x556aa0168100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:48.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.904+0000 7fdd0eebb700 1 --2- 172.21.15.78:0/3605856714 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x556aa04fc000 0x556aa0169700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:48.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.904+0000 7fdd0ff00380 1 -- 172.21.15.78:0/3605856714 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:1 11.d 11:b50e409b:::mds_snaptable:head [writefull 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x556aa0ea5400 con 0x556aa0ea5000 2023-10-11T09:48:48.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.904+0000 7fdd0deb9700 1 --2- 172.21.15.78:0/3605856714 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x556aa0ea5000 0x556aa0168100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:48.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.904+0000 7fdd0eebb700 1 --2- 172.21.15.78:0/3605856714 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x556aa04fc000 0x556aa0169700 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x556aa0e9c090 tx=0x556aa0eb92c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:48.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.904+0000 7fdd0deb9700 1 --2- 172.21.15.78:0/3605856714 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x556aa0ea5000 0x556aa0168100 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:48.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.908+0000 7fdd0deb9700 1 -- 172.21.15.78:0/3605856714 <== osd.2 v2:172.21.15.78:6808/35732 1 ==== osd_op_reply(1 mds_snaptable [writefull 0~70] v88'3 uv3 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x556aa0f4afc0 con 0x556aa0ea5000 2023-10-11T09:48:48.920 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:48:48.920 INFO:teuthology.orchestra.run.smithi078.stdout: "result": 0 2023-10-11T09:48:48.921 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:48:48.921 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:48.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.908+0000 7fdd0ff00380 1 -- 172.21.15.78:0/3605856714 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x556aa0ea5000 msgr2=0x556aa0168100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:48.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.908+0000 7fdd0ff00380 1 --2- 172.21.15.78:0/3605856714 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x556aa0ea5000 0x556aa0168100 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.908+0000 7fdd0ff00380 1 -- 172.21.15.78:0/3605856714 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x556aa04fc000 msgr2=0x556aa0169700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:48.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.908+0000 7fdd0ff00380 1 --2- 172.21.15.78:0/3605856714 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x556aa04fc000 0x556aa0169700 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x556aa0e9c090 tx=0x556aa0eb92c0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.908+0000 7fdd0ff00380 1 -- 172.21.15.78:0/3605856714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0ea4800 msgr2=0x556aa0168c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:48.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.908+0000 7fdd0ff00380 1 --2- 172.21.15.78:0/3605856714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0ea4800 0x556aa0168c00 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x556aa0e9af60 tx=0x556aa0eb8e40 comp rx=0 tx=0).stop 2023-10-11T09:48:48.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.908+0000 7fdd0e6ba700 1 -- 172.21.15.78:0/3605856714 reap_dead start 2023-10-11T09:48:48.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.908+0000 7fdd0ff00380 1 -- 172.21.15.78:0/3605856714 shutdown_connections 2023-10-11T09:48:48.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.908+0000 7fdd0ff00380 1 -- 172.21.15.78:0/3605856714 >> 172.21.15.78:0/3605856714 conn(0x556aa00c0c00 msgr2=0x556aa01ad800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:48.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.908+0000 7fdd0ff00380 1 -- 172.21.15.78:0/3605856714 shutdown_connections 2023-10-11T09:48:48.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.908+0000 7fdd0ff00380 1 -- 172.21.15.78:0/3605856714 wait complete. 2023-10-11T09:48:48.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.912+0000 7fdd0ff00380 1 -- 172.21.15.78:0/2495627332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa00c1400 msgr2=0x556aa0166b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:48.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.912+0000 7fdd0ff00380 1 --2- 172.21.15.78:0/2495627332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa00c1400 0x556aa0166b00 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x556aa0e9cb70 tx=0x556aa0ea6000 comp rx=0 tx=0).stop 2023-10-11T09:48:48.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.912+0000 7fdd0ff00380 1 -- 172.21.15.78:0/2495627332 shutdown_connections 2023-10-11T09:48:48.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.912+0000 7fdd0ff00380 1 --2- 172.21.15.78:0/2495627332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556aa00c1000 0x556aa0167080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.912+0000 7fdd0ff00380 1 --2- 172.21.15.78:0/2495627332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa00c1400 0x556aa0166b00 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.912+0000 7fdd0ff00380 1 --2- 172.21.15.78:0/2495627332 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556aa00c1800 0x556aa0166580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.912+0000 7fdd0ff00380 1 -- 172.21.15.78:0/2495627332 >> 172.21.15.78:0/2495627332 conn(0x556aa00c1c00 msgr2=0x556aa01ad000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:48.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.912+0000 7fdd0ff00380 1 -- 172.21.15.78:0/2495627332 shutdown_connections 2023-10-11T09:48:48.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.912+0000 7fdd0ff00380 1 -- 172.21.15.78:0/2495627332 wait complete. 2023-10-11T09:48:48.927 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'snap'] in time 0:00:00.096006, result: { "result": 0 } 2023-10-11T09:48:48.927 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset inode 2023-10-11T09:48:48.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.988+0000 7fb2cd57c380 1 -- start start 2023-10-11T09:48:48.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.988+0000 7fb2cd57c380 1 Processor -- start 2023-10-11T09:48:48.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.988+0000 7fb2cd57c380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e67bcf800 0x563e67c74580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:48.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.988+0000 7fb2cd57c380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e67bcf400 0x563e67c74b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:48.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.988+0000 7fb2cd57c380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563e67bcf000 0x563e67c75080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:48.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.988+0000 7fb2cd57c380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e67c05e00 con 0x563e67bcf800 2023-10-11T09:48:48.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.988+0000 7fb2cd57c380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e67c4ea00 con 0x563e67bcf400 2023-10-11T09:48:48.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.988+0000 7fb2cd57c380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x563e67c4e640 con 0x563e67bcf000 2023-10-11T09:48:48.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.988+0000 7fb2cbd36700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e67bcf800 0x563e67c74580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:48.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.988+0000 7fb2cbd36700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e67bcf800 0x563e67c74580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35642/0 (socket says 172.21.15.78:35642) 2023-10-11T09:48:48.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.988+0000 7fb2cbd36700 1 -- 172.21.15.78:0/1305751894 learned_addr learned my addr 172.21.15.78:0/1305751894 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:48.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.988+0000 7fb2cb535700 1 --2- 172.21.15.78:0/1305751894 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e67bcf400 0x563e67c74b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:48.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.988+0000 7fb2cc537700 1 --2- 172.21.15.78:0/1305751894 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563e67bcf000 0x563e67c75080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:48.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.988+0000 7fb2cbd36700 1 -- 172.21.15.78:0/1305751894 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563e67bcf000 msgr2=0x563e67c75080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:48.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.988+0000 7fb2cbd36700 1 --2- 172.21.15.78:0/1305751894 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563e67bcf000 0x563e67c75080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.988+0000 7fb2cbd36700 1 -- 172.21.15.78:0/1305751894 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e67bcf400 msgr2=0x563e67c74b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:48.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.988+0000 7fb2cbd36700 1 --2- 172.21.15.78:0/1305751894 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e67bcf400 0x563e67c74b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:48.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.988+0000 7fb2cbd36700 1 -- 172.21.15.78:0/1305751894 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e67bf0b60 con 0x563e67bcf800 2023-10-11T09:48:48.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.988+0000 7fb2cc537700 1 --2- 172.21.15.78:0/1305751894 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563e67bcf000 0x563e67c75080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:48:48.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.988+0000 7fb2cbd36700 1 --2- 172.21.15.78:0/1305751894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e67bcf800 0x563e67c74580 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x563e67d30ba0 tx=0x563e689a8f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=908fa2d349d9ab77 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:48.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.992+0000 7fb2c9531700 1 -- 172.21.15.78:0/1305751894 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563e67bff600 con 0x563e67bcf800 2023-10-11T09:48:48.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.992+0000 7fb2c9531700 1 -- 172.21.15.78:0/1305751894 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563e67c1a480 con 0x563e67bcf800 2023-10-11T09:48:48.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.992+0000 7fb2cd57c380 1 -- 172.21.15.78:0/1305751894 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e67bf0340 con 0x563e67bcf800 2023-10-11T09:48:48.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.992+0000 7fb2cd57c380 1 -- 172.21.15.78:0/1305751894 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x563e67bf0000 con 0x563e67bcf800 2023-10-11T09:48:49.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.992+0000 7fb2cd57c380 4 waiting for MDS map... 2023-10-11T09:48:49.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.992+0000 7fb2c9531700 1 -- 172.21.15.78:0/1305751894 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563e67bff1e0 con 0x563e67bcf800 2023-10-11T09:48:49.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.992+0000 7fb2c9531700 1 -- 172.21.15.78:0/1305751894 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(88..88 src has 1..88) v4 ==== 6638+0+0 (secure 0 0 0) 0x563e67c08c40 con 0x563e67bcf800 2023-10-11T09:48:49.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.992+0000 7fb2c9531700 1 -- 172.21.15.78:0/1305751894 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 64) v1 ==== 3118+0+0 (secure 0 0 0) 0x563e67eb8300 con 0x563e67bcf800 2023-10-11T09:48:49.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.992+0000 7fb2cd57c380 4 Got MDS map 64 2023-10-11T09:48:49.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.992+0000 7fb2cd57c380 10 main: main 2023-10-11T09:48:49.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.992+0000 7fb2cd57c380 4 main: connecting to RADOS... 2023-10-11T09:48:49.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.992+0000 7fb2cd57c380 1 Processor -- start 2023-10-11T09:48:49.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.992+0000 7fb2cd57c380 1 -- start start 2023-10-11T09:48:49.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.992+0000 7fb2cd57c380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e67ec0400 0x563e67c75600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.992+0000 7fb2cd57c380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e67ec0800 0x563e67c75b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.992+0000 7fb2cd57c380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563e67ec0c00 0x563e67c76100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.992+0000 7fb2cd57c380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e67c4e500 con 0x563e67ec0400 2023-10-11T09:48:49.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.992+0000 7fb2cd57c380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e67c4e3c0 con 0x563e67ec0800 2023-10-11T09:48:49.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.992+0000 7fb2cd57c380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x563e67c4ea00 con 0x563e67ec0c00 2023-10-11T09:48:49.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.996+0000 7fb2cc537700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e67ec0400 0x563e67c75600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.996+0000 7fb2cc537700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e67ec0400 0x563e67c75600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35650/0 (socket says 172.21.15.78:35650) 2023-10-11T09:48:49.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.996+0000 7fb2cc537700 1 -- 172.21.15.78:0/1392109354 learned_addr learned my addr 172.21.15.78:0/1392109354 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:49.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.996+0000 7fb2cbd36700 1 --2- 172.21.15.78:0/1392109354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e67ec0800 0x563e67c75b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.996+0000 7fb2cb535700 1 --2- 172.21.15.78:0/1392109354 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563e67ec0c00 0x563e67c76100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.996+0000 7fb2cc537700 1 -- 172.21.15.78:0/1392109354 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563e67ec0c00 msgr2=0x563e67c76100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:49.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.996+0000 7fb2cc537700 1 --2- 172.21.15.78:0/1392109354 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563e67ec0c00 0x563e67c76100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.996+0000 7fb2cc537700 1 -- 172.21.15.78:0/1392109354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e67ec0800 msgr2=0x563e67c75b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:49.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.996+0000 7fb2cc537700 1 --2- 172.21.15.78:0/1392109354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e67ec0800 0x563e67c75b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.996+0000 7fb2cc537700 1 -- 172.21.15.78:0/1392109354 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e67bf0ea0 con 0x563e67ec0400 2023-10-11T09:48:49.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.996+0000 7fb2cc537700 1 --2- 172.21.15.78:0/1392109354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e67ec0400 0x563e67c75600 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x563e67f140f0 tx=0x563e67ecccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8df3b4fabe3dff3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.996+0000 7fb2c752d700 1 -- 172.21.15.78:0/1392109354 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563e67bff600 con 0x563e67ec0400 2023-10-11T09:48:49.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.996+0000 7fb2c752d700 1 -- 172.21.15.78:0/1392109354 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563e67c1a180 con 0x563e67ec0400 2023-10-11T09:48:49.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.996+0000 7fb2c752d700 1 -- 172.21.15.78:0/1392109354 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563e67bff1e0 con 0x563e67ec0400 2023-10-11T09:48:49.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.996+0000 7fb2cd57c380 1 -- 172.21.15.78:0/1392109354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e67ec0400 msgr2=0x563e67c75600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.996+0000 7fb2cd57c380 1 --2- 172.21.15.78:0/1392109354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e67ec0400 0x563e67c75600 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x563e67f140f0 tx=0x563e67ecccc0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.996+0000 7fb2cd57c380 1 -- 172.21.15.78:0/1392109354 shutdown_connections 2023-10-11T09:48:49.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.996+0000 7fb2cd57c380 1 --2- 172.21.15.78:0/1392109354 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563e67ec0c00 0x563e67c76100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.996+0000 7fb2cd57c380 1 --2- 172.21.15.78:0/1392109354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e67ec0800 0x563e67c75b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.996+0000 7fb2cd57c380 1 --2- 172.21.15.78:0/1392109354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e67ec0400 0x563e67c75600 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.996+0000 7fb2cd57c380 1 -- 172.21.15.78:0/1392109354 >> 172.21.15.78:0/1392109354 conn(0x563e67bcec00 msgr2=0x563e67cbb800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:49.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.996+0000 7fb2cd57c380 1 -- 172.21.15.78:0/1392109354 shutdown_connections 2023-10-11T09:48:49.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.996+0000 7fb2cd57c380 1 -- 172.21.15.78:0/1392109354 wait complete. 2023-10-11T09:48:49.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.996+0000 7fb2cd57c380 1 Processor -- start 2023-10-11T09:48:49.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:48.996+0000 7fb2cd57c380 1 -- start start 2023-10-11T09:48:49.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.000+0000 7fb2cd57c380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e67ec0400 0x563e67c77180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.000+0000 7fb2cd57c380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e67ec0800 0x563e67c76c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.000+0000 7fb2cd57c380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563e67ec0c00 0x563e67c76680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.000+0000 7fb2cd57c380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e67c4e280 con 0x563e67ec0800 2023-10-11T09:48:49.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.000+0000 7fb2cd57c380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e67c4e140 con 0x563e67ec0400 2023-10-11T09:48:49.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.000+0000 7fb2cd57c380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x563e67c4e000 con 0x563e67ec0c00 2023-10-11T09:48:49.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.000+0000 7fb2cbd36700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e67ec0800 0x563e67c76c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.000+0000 7fb2cbd36700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e67ec0800 0x563e67c76c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35660/0 (socket says 172.21.15.78:35660) 2023-10-11T09:48:49.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.000+0000 7fb2cbd36700 1 -- 172.21.15.78:0/3241875898 learned_addr learned my addr 172.21.15.78:0/3241875898 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:49.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.000+0000 7fb2cc537700 1 --2- 172.21.15.78:0/3241875898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e67ec0400 0x563e67c77180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.000+0000 7fb2cb535700 1 --2- 172.21.15.78:0/3241875898 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563e67ec0c00 0x563e67c76680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.000+0000 7fb2cbd36700 1 -- 172.21.15.78:0/3241875898 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563e67ec0c00 msgr2=0x563e67c76680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.000+0000 7fb2cbd36700 1 --2- 172.21.15.78:0/3241875898 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563e67ec0c00 0x563e67c76680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.000+0000 7fb2cbd36700 1 -- 172.21.15.78:0/3241875898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e67ec0400 msgr2=0x563e67c77180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.000+0000 7fb2cbd36700 1 --2- 172.21.15.78:0/3241875898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e67ec0400 0x563e67c77180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.000+0000 7fb2cbd36700 1 -- 172.21.15.78:0/3241875898 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e67bf0b60 con 0x563e67ec0800 2023-10-11T09:48:49.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.000+0000 7fb2cc537700 1 --2- 172.21.15.78:0/3241875898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e67ec0400 0x563e67c77180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:48:49.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.000+0000 7fb2cbd36700 1 --2- 172.21.15.78:0/3241875898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e67ec0800 0x563e67c76c00 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x563e67f14d20 tx=0x563e689a8de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.000+0000 7fb2c5d2a700 1 -- 172.21.15.78:0/3241875898 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563e67bfef20 con 0x563e67ec0800 2023-10-11T09:48:49.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.000+0000 7fb2c5d2a700 1 -- 172.21.15.78:0/3241875898 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563e689ae180 con 0x563e67ec0800 2023-10-11T09:48:49.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.000+0000 7fb2c5d2a700 1 -- 172.21.15.78:0/3241875898 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563e67bff600 con 0x563e67ec0800 2023-10-11T09:48:49.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.000+0000 7fb2cd57c380 1 -- 172.21.15.78:0/3241875898 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563e67bf0340 con 0x563e67ec0800 2023-10-11T09:48:49.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.000+0000 7fb2cd57c380 1 -- 172.21.15.78:0/3241875898 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e67bf0000 con 0x563e67ec0800 2023-10-11T09:48:49.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.000+0000 7fb2cd57c380 4 main: resolving pool 11 2023-10-11T09:48:49.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.004+0000 7fb2c5d2a700 1 -- 172.21.15.78:0/3241875898 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x563e67c4c840 con 0x563e67ec0800 2023-10-11T09:48:49.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.004+0000 7fb2c5d2a700 1 --2- 172.21.15.78:0/3241875898 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x563e68014000 0x563e67c77700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.004+0000 7fb2c5d2a700 1 -- 172.21.15.78:0/3241875898 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6638+0+0 (secure 0 0 0) 0x563e67c08a80 con 0x563e67ec0800 2023-10-11T09:48:49.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.004+0000 7fb2cd57c380 4 main: creating IoCtx.. 2023-10-11T09:48:49.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.004+0000 7fb2cc537700 1 --2- 172.21.15.78:0/3241875898 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x563e68014000 0x563e67c77700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.004+0000 7fb2cd57c380 1 --2- 172.21.15.78:0/3241875898 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x563e67ec1000 0x563e67c76100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.004+0000 7fb2cd57c380 1 -- 172.21.15.78:0/3241875898 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:1 11.13 11:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x563e67ec1400 con 0x563e67ec1000 2023-10-11T09:48:49.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.004+0000 7fb2cb535700 1 --2- 172.21.15.78:0/3241875898 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x563e67ec1000 0x563e67c76100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.004+0000 7fb2cc537700 1 --2- 172.21.15.78:0/3241875898 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x563e68014000 0x563e67c77700 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x563e67f140f0 tx=0x563e67eccc00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.008+0000 7fb2cb535700 1 --2- 172.21.15.78:0/3241875898 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x563e67ec1000 0x563e67c76100 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.008+0000 7fb2cb535700 1 -- 172.21.15.78:0/3241875898 <== osd.5 v2:172.21.15.133:6816/35899 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v88'6 uv6 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x563e68a4afc0 con 0x563e67ec1000 2023-10-11T09:48:49.019 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:48:49.020 INFO:teuthology.orchestra.run.smithi078.stdout: "0": { 2023-10-11T09:48:49.020 INFO:teuthology.orchestra.run.smithi078.stdout: "data": {}, 2023-10-11T09:48:49.020 INFO:teuthology.orchestra.run.smithi078.stdout: "result": 0 2023-10-11T09:48:49.020 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:48:49.020 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:48:49.021 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:49.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.008+0000 7fb2cd57c380 1 -- 172.21.15.78:0/3241875898 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x563e67ec1000 msgr2=0x563e67c76100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.008+0000 7fb2cd57c380 1 --2- 172.21.15.78:0/3241875898 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x563e67ec1000 0x563e67c76100 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.008+0000 7fb2cd57c380 1 -- 172.21.15.78:0/3241875898 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x563e68014000 msgr2=0x563e67c77700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.008+0000 7fb2cd57c380 1 --2- 172.21.15.78:0/3241875898 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x563e68014000 0x563e67c77700 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x563e67f140f0 tx=0x563e67eccc00 comp rx=0 tx=0).stop 2023-10-11T09:48:49.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.008+0000 7fb2cd57c380 1 -- 172.21.15.78:0/3241875898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e67ec0800 msgr2=0x563e67c76c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.008+0000 7fb2cd57c380 1 --2- 172.21.15.78:0/3241875898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e67ec0800 0x563e67c76c00 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x563e67f14d20 tx=0x563e689a8de0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.008+0000 7fb2cb535700 1 -- 172.21.15.78:0/3241875898 reap_dead start 2023-10-11T09:48:49.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.008+0000 7fb2cd57c380 1 -- 172.21.15.78:0/3241875898 shutdown_connections 2023-10-11T09:48:49.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.008+0000 7fb2cd57c380 1 -- 172.21.15.78:0/3241875898 >> 172.21.15.78:0/3241875898 conn(0x563e67bcec00 msgr2=0x563e67cbb800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:49.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.012+0000 7fb2cd57c380 1 -- 172.21.15.78:0/3241875898 shutdown_connections 2023-10-11T09:48:49.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.012+0000 7fb2cd57c380 1 -- 172.21.15.78:0/3241875898 wait complete. 2023-10-11T09:48:49.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.012+0000 7fb2cd57c380 1 -- 172.21.15.78:0/1305751894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e67bcf800 msgr2=0x563e67c74580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.012+0000 7fb2cd57c380 1 --2- 172.21.15.78:0/1305751894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e67bcf800 0x563e67c74580 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x563e67d30ba0 tx=0x563e689a8f00 comp rx=0 tx=0).stop 2023-10-11T09:48:49.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.012+0000 7fb2cd57c380 1 -- 172.21.15.78:0/1305751894 shutdown_connections 2023-10-11T09:48:49.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.012+0000 7fb2cd57c380 1 --2- 172.21.15.78:0/1305751894 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563e67bcf000 0x563e67c75080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.012+0000 7fb2cd57c380 1 --2- 172.21.15.78:0/1305751894 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e67bcf400 0x563e67c74b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.012+0000 7fb2cd57c380 1 --2- 172.21.15.78:0/1305751894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e67bcf800 0x563e67c74580 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.012+0000 7fb2cd57c380 1 -- 172.21.15.78:0/1305751894 >> 172.21.15.78:0/1305751894 conn(0x563e67bcfc00 msgr2=0x563e67cbb000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:49.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.012+0000 7fb2cd57c380 1 -- 172.21.15.78:0/1305751894 shutdown_connections 2023-10-11T09:48:49.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.012+0000 7fb2cd57c380 1 -- 172.21.15.78:0/1305751894 wait complete. 2023-10-11T09:48:49.027 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'inode'] in time 0:00:00.099178, result: { "0": { "data": {}, "result": 0 } } 2023-10-11T09:48:49.027 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2023-10-11T09:48:49.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.088+0000 7f2de2f3e380 1 -- start start 2023-10-11T09:48:49.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.088+0000 7f2de2f3e380 1 Processor -- start 2023-10-11T09:48:49.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.088+0000 7f2de2f3e380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352b601000 0x56352b6a6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.088+0000 7f2de2f3e380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56352b601400 0x56352b6a6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.088+0000 7f2de2f3e380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56352b601800 0x56352b6a7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.088+0000 7f2de2f3e380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56352b680a00 con 0x56352b601800 2023-10-11T09:48:49.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.088+0000 7f2de2f3e380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56352b680780 con 0x56352b601000 2023-10-11T09:48:49.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.088+0000 7f2de2f3e380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x56352b680640 con 0x56352b601400 2023-10-11T09:48:49.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.088+0000 7f2de1ef9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56352b601800 0x56352b6a7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.088+0000 7f2de1ef9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56352b601800 0x56352b6a7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35674/0 (socket says 172.21.15.78:35674) 2023-10-11T09:48:49.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.088+0000 7f2de1ef9700 1 -- 172.21.15.78:0/1842934256 learned_addr learned my addr 172.21.15.78:0/1842934256 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:49.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.088+0000 7f2de0ef7700 1 --2- 172.21.15.78:0/1842934256 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56352b601400 0x56352b6a6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.088+0000 7f2de16f8700 1 --2- 172.21.15.78:0/1842934256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352b601000 0x56352b6a6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.088+0000 7f2de1ef9700 1 -- 172.21.15.78:0/1842934256 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56352b601400 msgr2=0x56352b6a6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:49.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.088+0000 7f2de1ef9700 1 --2- 172.21.15.78:0/1842934256 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56352b601400 0x56352b6a6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.088+0000 7f2de1ef9700 1 -- 172.21.15.78:0/1842934256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352b601000 msgr2=0x56352b6a6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:49.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.088+0000 7f2de1ef9700 1 --2- 172.21.15.78:0/1842934256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352b601000 0x56352b6a6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.088+0000 7f2de1ef9700 1 -- 172.21.15.78:0/1842934256 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56352b622b60 con 0x56352b601800 2023-10-11T09:48:49.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.092+0000 7f2de1ef9700 1 --2- 172.21.15.78:0/1842934256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56352b601800 0x56352b6a7080 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x56352c36ee40 tx=0x56352c3ee480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa577a2c00d090a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.092+0000 7f2ddeef3700 1 -- 172.21.15.78:0/1842934256 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56352b631600 con 0x56352b601800 2023-10-11T09:48:49.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.092+0000 7f2ddeef3700 1 -- 172.21.15.78:0/1842934256 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56352b64c180 con 0x56352b601800 2023-10-11T09:48:49.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.092+0000 7f2ddeef3700 1 -- 172.21.15.78:0/1842934256 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56352b6311e0 con 0x56352b601800 2023-10-11T09:48:49.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.092+0000 7f2de2f3e380 1 -- 172.21.15.78:0/1842934256 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56352b622340 con 0x56352b601800 2023-10-11T09:48:49.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.092+0000 7f2de2f3e380 1 -- 172.21.15.78:0/1842934256 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x56352b622000 con 0x56352b601800 2023-10-11T09:48:49.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.092+0000 7f2de2f3e380 4 waiting for MDS map... 2023-10-11T09:48:49.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.092+0000 7f2ddeef3700 1 -- 172.21.15.78:0/1842934256 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(88..88 src has 1..88) v4 ==== 6638+0+0 (secure 0 0 0) 0x56352b63ac40 con 0x56352b601800 2023-10-11T09:48:49.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.092+0000 7f2ddeef3700 1 -- 172.21.15.78:0/1842934256 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 64) v1 ==== 3118+0+0 (secure 0 0 0) 0x56352c3e0300 con 0x56352b601800 2023-10-11T09:48:49.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.092+0000 7f2de2f3e380 4 Got MDS map 64 2023-10-11T09:48:49.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.092+0000 7f2de2f3e380 10 main: JournalTool::main 2023-10-11T09:48:49.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.092+0000 7f2de2f3e380 4 main: JournalTool: connecting to RADOS... 2023-10-11T09:48:49.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.092+0000 7f2de2f3e380 1 Processor -- start 2023-10-11T09:48:49.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.092+0000 7f2de2f3e380 1 -- start start 2023-10-11T09:48:49.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.092+0000 7f2de2f3e380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352b956400 0x56352b6a7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.092+0000 7f2de2f3e380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56352b956800 0x56352b6a7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.092+0000 7f2de2f3e380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56352b956c00 0x56352b6a8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.092+0000 7f2de2f3e380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56352b637e00 con 0x56352b956800 2023-10-11T09:48:49.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.092+0000 7f2de2f3e380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56352b680780 con 0x56352b956400 2023-10-11T09:48:49.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.092+0000 7f2de2f3e380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x56352b680500 con 0x56352b956c00 2023-10-11T09:48:49.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.096+0000 7f2de16f8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56352b956800 0x56352b6a7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.096+0000 7f2de16f8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56352b956800 0x56352b6a7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35680/0 (socket says 172.21.15.78:35680) 2023-10-11T09:48:49.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.096+0000 7f2de16f8700 1 -- 172.21.15.78:0/4246617367 learned_addr learned my addr 172.21.15.78:0/4246617367 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:49.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.096+0000 7f2de0ef7700 1 --2- 172.21.15.78:0/4246617367 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56352b956c00 0x56352b6a8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.096+0000 7f2de0ef7700 1 --2- 172.21.15.78:0/4246617367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352b956400 0x56352b6a7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.096+0000 7f2de16f8700 1 -- 172.21.15.78:0/4246617367 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56352b956c00 msgr2=0x56352b6a8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:49.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.096+0000 7f2de16f8700 1 --2- 172.21.15.78:0/4246617367 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56352b956c00 0x56352b6a8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.096+0000 7f2de16f8700 1 -- 172.21.15.78:0/4246617367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352b956400 msgr2=0x56352b6a7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:49.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.096+0000 7f2de16f8700 1 --2- 172.21.15.78:0/4246617367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352b956400 0x56352b6a7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.096+0000 7f2de16f8700 1 -- 172.21.15.78:0/4246617367 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56352b622ea0 con 0x56352b956800 2023-10-11T09:48:49.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.096+0000 7f2de16f8700 1 --2- 172.21.15.78:0/4246617367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56352b956800 0x56352b6a7b80 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x56352b964090 tx=0x56352c3ee060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538c15735579aa2f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.096+0000 7f2ddceef700 1 -- 172.21.15.78:0/4246617367 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56352b631080 con 0x56352b956800 2023-10-11T09:48:49.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.096+0000 7f2ddceef700 1 -- 172.21.15.78:0/4246617367 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56352b64c900 con 0x56352b956800 2023-10-11T09:48:49.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.096+0000 7f2ddceef700 1 -- 172.21.15.78:0/4246617367 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56352b631600 con 0x56352b956800 2023-10-11T09:48:49.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.096+0000 7f2de2f3e380 1 -- 172.21.15.78:0/4246617367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56352b956800 msgr2=0x56352b6a7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.096+0000 7f2de2f3e380 1 --2- 172.21.15.78:0/4246617367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56352b956800 0x56352b6a7b80 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x56352b964090 tx=0x56352c3ee060 comp rx=0 tx=0).stop 2023-10-11T09:48:49.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.096+0000 7f2de2f3e380 1 -- 172.21.15.78:0/4246617367 shutdown_connections 2023-10-11T09:48:49.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.096+0000 7f2de2f3e380 1 --2- 172.21.15.78:0/4246617367 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56352b956c00 0x56352b6a8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.096+0000 7f2de2f3e380 1 --2- 172.21.15.78:0/4246617367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56352b956800 0x56352b6a7b80 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.096+0000 7f2de2f3e380 1 --2- 172.21.15.78:0/4246617367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352b956400 0x56352b6a7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.096+0000 7f2de2f3e380 1 -- 172.21.15.78:0/4246617367 >> 172.21.15.78:0/4246617367 conn(0x56352b956000 msgr2=0x56352b6f1800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:49.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.096+0000 7f2de2f3e380 1 -- 172.21.15.78:0/4246617367 shutdown_connections 2023-10-11T09:48:49.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.096+0000 7f2de2f3e380 1 -- 172.21.15.78:0/4246617367 wait complete. 2023-10-11T09:48:49.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.096+0000 7f2de2f3e380 1 Processor -- start 2023-10-11T09:48:49.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.096+0000 7f2de2f3e380 1 -- start start 2023-10-11T09:48:49.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.100+0000 7f2de2f3e380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352b956400 0x56352b6a9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.100+0000 7f2de2f3e380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56352b956800 0x56352b6a8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.100+0000 7f2de2f3e380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56352b956c00 0x56352b6a8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.100+0000 7f2de2f3e380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56352b6803c0 con 0x56352b956c00 2023-10-11T09:48:49.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.100+0000 7f2de2f3e380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56352b680280 con 0x56352b956400 2023-10-11T09:48:49.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.100+0000 7f2de2f3e380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x56352b680140 con 0x56352b956800 2023-10-11T09:48:49.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.100+0000 7f2de0ef7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56352b956c00 0x56352b6a8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.100+0000 7f2de0ef7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56352b956c00 0x56352b6a8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35688/0 (socket says 172.21.15.78:35688) 2023-10-11T09:48:49.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.100+0000 7f2de0ef7700 1 -- 172.21.15.78:0/1592087488 learned_addr learned my addr 172.21.15.78:0/1592087488 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:49.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.100+0000 7f2de16f8700 1 --2- 172.21.15.78:0/1592087488 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56352b956800 0x56352b6a8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.100+0000 7f2de0ef7700 1 --2- 172.21.15.78:0/1592087488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352b956400 0x56352b6a9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.100+0000 7f2de0ef7700 1 -- 172.21.15.78:0/1592087488 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56352b956800 msgr2=0x56352b6a8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.100+0000 7f2de0ef7700 1 --2- 172.21.15.78:0/1592087488 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56352b956800 0x56352b6a8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.100+0000 7f2de0ef7700 1 -- 172.21.15.78:0/1592087488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352b956400 msgr2=0x56352b6a9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.100+0000 7f2de0ef7700 1 --2- 172.21.15.78:0/1592087488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352b956400 0x56352b6a9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.100+0000 7f2de0ef7700 1 -- 172.21.15.78:0/1592087488 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56352b6231e0 con 0x56352b956c00 2023-10-11T09:48:49.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.100+0000 7f2de16f8700 1 --2- 172.21.15.78:0/1592087488 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56352b956800 0x56352b6a8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:48:49.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.100+0000 7f2de0ef7700 1 --2- 172.21.15.78:0/1592087488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56352b956c00 0x56352b6a8680 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x56352b9654d0 tx=0x56352c3ee0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.100+0000 7f2ddb6ec700 1 -- 172.21.15.78:0/1592087488 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56352b631080 con 0x56352b956c00 2023-10-11T09:48:49.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.100+0000 7f2ddb6ec700 1 -- 172.21.15.78:0/1592087488 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56352c3ff800 con 0x56352b956c00 2023-10-11T09:48:49.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.100+0000 7f2ddb6ec700 1 -- 172.21.15.78:0/1592087488 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56352b631600 con 0x56352b956c00 2023-10-11T09:48:49.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.100+0000 7f2de2f3e380 1 -- 172.21.15.78:0/1592087488 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56352b622ea0 con 0x56352b956c00 2023-10-11T09:48:49.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.100+0000 7f2de2f3e380 1 -- 172.21.15.78:0/1592087488 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56352b622b60 con 0x56352b956c00 2023-10-11T09:48:49.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.100+0000 7f2de2f3e380 4 main: JournalTool: resolving pool 11 2023-10-11T09:48:49.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.104+0000 7f2ddb6ec700 1 -- 172.21.15.78:0/1592087488 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x56352b67e840 con 0x56352b956c00 2023-10-11T09:48:49.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.104+0000 7f2ddb6ec700 1 --2- 172.21.15.78:0/1592087488 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x56352b957400 0x56352b6a9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.104+0000 7f2ddb6ec700 1 -- 172.21.15.78:0/1592087488 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6638+0+0 (secure 0 0 0) 0x56352b63aa80 con 0x56352b956c00 2023-10-11T09:48:49.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.104+0000 7f2de2f3e380 4 main: JournalTool: creating IoCtx.. 2023-10-11T09:48:49.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.104+0000 7f2de2f3e380 4 main: Executing for rank 0 2023-10-11T09:48:49.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.104+0000 7f2de16f8700 1 --2- 172.21.15.78:0/1592087488 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x56352b957400 0x56352b6a9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.104+0000 7f2de16f8700 1 --2- 172.21.15.78:0/1592087488 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x56352b957400 0x56352b6a9700 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x56352b964090 tx=0x56352c3ee1e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.104+0000 7f2de2f3e380 1 -- start start 2023-10-11T09:48:49.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.108+0000 7f2de2f3e380 1 Processor -- start 2023-10-11T09:48:49.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.108+0000 7f2de2f3e380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352b957c00 0x56352b6a8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.108+0000 7f2de2f3e380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56352baba000 0x56352b6a7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.108+0000 7f2de2f3e380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56352baba400 0x56352b6a7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.108+0000 7f2de2f3e380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56352b680140 con 0x56352baba000 2023-10-11T09:48:49.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.108+0000 7f2de2f3e380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56352b680280 con 0x56352b957c00 2023-10-11T09:48:49.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.108+0000 7f2de2f3e380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x56352b680500 con 0x56352baba400 2023-10-11T09:48:49.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.108+0000 7f2de16f8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56352baba000 0x56352b6a7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.108+0000 7f2de16f8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56352baba000 0x56352b6a7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35696/0 (socket says 172.21.15.78:35696) 2023-10-11T09:48:49.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.108+0000 7f2de16f8700 1 -- 172.21.15.78:0/3912093410 learned_addr learned my addr 172.21.15.78:0/3912093410 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:49.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.108+0000 7f2de0ef7700 1 --2- 172.21.15.78:0/3912093410 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56352baba400 0x56352b6a7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.108+0000 7f2de1ef9700 1 --2- 172.21.15.78:0/3912093410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352b957c00 0x56352b6a8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.108+0000 7f2de16f8700 1 -- 172.21.15.78:0/3912093410 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56352baba400 msgr2=0x56352b6a7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:49.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.108+0000 7f2de16f8700 1 --2- 172.21.15.78:0/3912093410 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56352baba400 0x56352b6a7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.108+0000 7f2de16f8700 1 -- 172.21.15.78:0/3912093410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352b957c00 msgr2=0x56352b6a8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:49.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.108+0000 7f2de16f8700 1 --2- 172.21.15.78:0/3912093410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352b957c00 0x56352b6a8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.108+0000 7f2de16f8700 1 -- 172.21.15.78:0/3912093410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56352b9a2b60 con 0x56352baba000 2023-10-11T09:48:49.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.108+0000 7f2de1ef9700 1 --2- 172.21.15.78:0/3912093410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352b957c00 0x56352b6a8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:48:49.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.108+0000 7f2de0ef7700 1 --2- 172.21.15.78:0/3912093410 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56352baba400 0x56352b6a7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:48:49.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.108+0000 7f2de16f8700 1 --2- 172.21.15.78:0/3912093410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56352baba000 0x56352b6a7b80 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x56352ba26ea0 tx=0x56352c3ee180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be0ee88a1150e7ae server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.108+0000 7f2dd86e6700 1 -- 172.21.15.78:0/3912093410 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56352b6311e0 con 0x56352baba000 2023-10-11T09:48:49.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.108+0000 7f2dd86e6700 1 -- 172.21.15.78:0/3912093410 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56352b64c480 con 0x56352baba000 2023-10-11T09:48:49.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.108+0000 7f2dd86e6700 1 -- 172.21.15.78:0/3912093410 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56352b630dc0 con 0x56352baba000 2023-10-11T09:48:49.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.108+0000 7f2de2f3e380 1 -- 172.21.15.78:0/3912093410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56352b9a29c0 con 0x56352baba000 2023-10-11T09:48:49.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.108+0000 7f2de2f3e380 1 -- 172.21.15.78:0/3912093410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x56352b9a2820 con 0x56352baba000 2023-10-11T09:48:49.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.108+0000 7f2de2f3e380 4 waiting for MDS map... 2023-10-11T09:48:49.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.108+0000 7f2dd86e6700 1 -- 172.21.15.78:0/3912093410 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(88..88 src has 1..88) v4 ==== 6638+0+0 (secure 0 0 0) 0x56352b63a8c0 con 0x56352baba000 2023-10-11T09:48:49.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.112+0000 7f2dd86e6700 1 -- 172.21.15.78:0/3912093410 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 64) v1 ==== 3118+0+0 (secure 0 0 0) 0x56352ba55b00 con 0x56352baba000 2023-10-11T09:48:49.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.112+0000 7f2de2f3e380 4 Got MDS map 64 2023-10-11T09:48:49.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.112+0000 7f2de2f3e380 1 --2- 172.21.15.78:0/3912093410 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x56352baba800 0x56352bad6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.112+0000 7f2de2f3e380 1 -- 172.21.15.78:0/3912093410 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:1 11.f 11:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x56352babac00 con 0x56352baba800 2023-10-11T09:48:49.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.112+0000 7f2de1ef9700 1 --2- 172.21.15.78:0/3912093410 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x56352baba800 0x56352bad6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.112+0000 7f2de1ef9700 1 --2- 172.21.15.78:0/3912093410 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x56352baba800 0x56352bad6000 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=c344dd65af3648a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.116+0000 7f2de1ef9700 1 -- 172.21.15.78:0/3912093410 <== osd.7 v2:172.21.15.133:6824/35901 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v88'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x56352c3fb200 con 0x56352baba800 2023-10-11T09:48:49.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.116+0000 7f2de2f3e380 1 --2- 172.21.15.78:0/3912093410 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x56352babb000 0x56352bad6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.116+0000 7f2de2f3e380 1 -- 172.21.15.78:0/3912093410 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:2 11.14 11:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x56352babb400 con 0x56352babb000 2023-10-11T09:48:49.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.116+0000 7f2de0ef7700 1 --2- 172.21.15.78:0/3912093410 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x56352babb000 0x56352bad6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.116+0000 7f2de0ef7700 1 --2- 172.21.15.78:0/3912093410 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x56352babb000 0x56352bad6580 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=29023467fb78a29 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.120+0000 7f2de0ef7700 1 -- 172.21.15.78:0/3912093410 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v88'26 uv26 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x56352c3fb440 con 0x56352babb000 2023-10-11T09:48:49.131 INFO:teuthology.orchestra.run.smithi078.stdout:writing EResetJournal entry 2023-10-11T09:48:49.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.120+0000 7f2de2f3e380 1 -- 172.21.15.78:0/3912093410 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:3 11.14 11:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x56352babbc00 con 0x56352babb000 2023-10-11T09:48:49.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.120+0000 7f2de2f3e380 1 --2- 172.21.15.78:0/3912093410 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x56352bae6000 0x56352bad6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.120+0000 7f2de2f3e380 1 -- 172.21.15.78:0/3912093410 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:4 11.1b 11:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x56352bae6400 con 0x56352bae6000 2023-10-11T09:48:49.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.120+0000 7f2de2f3e380 1 --2- 172.21.15.78:0/3912093410 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x56352bae6800 0x56352bad7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.120+0000 7f2de2f3e380 1 -- 172.21.15.78:0/3912093410 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:5 11.b 11:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x56352bae6c00 con 0x56352bae6800 2023-10-11T09:48:49.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.120+0000 7f2de2f3e380 1 --2- 172.21.15.78:0/3912093410 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x56352bae7000 0x56352bad7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.120+0000 7f2de2f3e380 1 -- 172.21.15.78:0/3912093410 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:6 11.1f 11:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x56352bae7400 con 0x56352bae7000 2023-10-11T09:48:49.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.120+0000 7f2de2f3e380 1 --2- 172.21.15.78:0/3912093410 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x56352bae7800 0x56352bad7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.120+0000 7f2de2f3e380 1 -- 172.21.15.78:0/3912093410 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:7 11.1a 11:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x56352bae7c00 con 0x56352bae7800 2023-10-11T09:48:49.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.120+0000 7f2de2f3e380 1 -- 172.21.15.78:0/3912093410 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:8 11.1e 11:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x56352baf2000 con 0x56352baba800 2023-10-11T09:48:49.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.120+0000 7f2de1ef9700 1 --2- 172.21.15.78:0/3912093410 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x56352bae6000 0x56352bad6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.120+0000 7f2de0ef7700 1 --2- 172.21.15.78:0/3912093410 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x56352bae7000 0x56352bad7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.120+0000 7f2de16f8700 1 --2- 172.21.15.78:0/3912093410 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x56352bae6800 0x56352bad7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.124+0000 7f2de1ef9700 1 --2- 172.21.15.78:0/3912093410 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x56352bae7800 0x56352bad7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.124+0000 7f2de0ef7700 1 --2- 172.21.15.78:0/3912093410 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x56352bae7000 0x56352bad7600 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=77b104092e3dca44 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.124+0000 7f2de1ef9700 1 --2- 172.21.15.78:0/3912093410 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x56352bae6000 0x56352bad6b00 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=14d0977ac1d0118f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.124+0000 7f2de16f8700 1 --2- 172.21.15.78:0/3912093410 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x56352bae6800 0x56352bad7080 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=ee58339187e6232d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.124+0000 7f2de1ef9700 1 --2- 172.21.15.78:0/3912093410 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x56352bae7800 0x56352bad7b80 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=45f09b5fd7101671 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.124+0000 7f2de0ef7700 1 -- 172.21.15.78:0/3912093410 <== osd.6 v2:172.21.15.133:6800/35900 2 ==== osd_op_reply(3 200.00000001 [delete] v88'27 uv26 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x56352c3fb440 con 0x56352babb000 2023-10-11T09:48:49.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.124+0000 7f2de1ef9700 1 -- 172.21.15.78:0/3912093410 <== osd.7 v2:172.21.15.133:6824/35901 2 ==== osd_op_reply(8 200.00000006 [delete] v88'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x56352c3fb200 con 0x56352baba800 2023-10-11T09:48:49.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.124+0000 7f2de1ef9700 1 -- 172.21.15.78:0/3912093410 <== osd.4 v2:172.21.15.133:6808/35898 1 ==== osd_op_reply(4 200.00000002 [delete] v88'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x56352c3fb200 con 0x56352bae6000 2023-10-11T09:48:49.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.124+0000 7f2de0ef7700 1 -- 172.21.15.78:0/3912093410 <== osd.2 v2:172.21.15.78:6808/35732 1 ==== osd_op_reply(6 200.00000004 [delete] v88'9 uv8 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x56352c3fb440 con 0x56352bae7000 2023-10-11T09:48:49.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.124+0000 7f2dd6ee3700 1 -- 172.21.15.78:0/3912093410 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:9 11.14 11:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x56352baf2000 con 0x56352babb000 2023-10-11T09:48:49.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.128+0000 7f2de16f8700 1 -- 172.21.15.78:0/3912093410 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(5 200.00000003 [delete] v88'6 uv5 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x56352bb02000 con 0x56352bae6800 2023-10-11T09:48:49.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.128+0000 7f2de1ef9700 1 -- 172.21.15.78:0/3912093410 <== osd.5 v2:172.21.15.133:6816/35899 1 ==== osd_op_reply(7 200.00000005 [delete] v88'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x56352c3fb200 con 0x56352bae7800 2023-10-11T09:48:49.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.128+0000 7f2de0ef7700 1 -- 172.21.15.78:0/3912093410 <== osd.6 v2:172.21.15.133:6800/35900 3 ==== osd_op_reply(9 200.00000001 [write 0~48] v88'28 uv28 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x56352c3fb440 con 0x56352babb000 2023-10-11T09:48:49.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.128+0000 7f2de2f3e380 4 Successfully wrote new journal pointer and header for rank 4:0 2023-10-11T09:48:49.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.128+0000 7f2de2f3e380 1 -- 172.21.15.78:0/3912093410 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x56352bae7000 msgr2=0x56352bad7600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.128+0000 7f2de2f3e380 1 --2- 172.21.15.78:0/3912093410 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x56352bae7000 0x56352bad7600 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.128+0000 7f2de2f3e380 1 -- 172.21.15.78:0/3912093410 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x56352bae6800 msgr2=0x56352bad7080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.128+0000 7f2de2f3e380 1 --2- 172.21.15.78:0/3912093410 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x56352bae6800 0x56352bad7080 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.128+0000 7f2de2f3e380 1 -- 172.21.15.78:0/3912093410 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x56352bae6000 msgr2=0x56352bad6b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.128+0000 7f2de2f3e380 1 --2- 172.21.15.78:0/3912093410 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x56352bae6000 0x56352bad6b00 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.128+0000 7f2de2f3e380 1 -- 172.21.15.78:0/3912093410 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x56352bae7800 msgr2=0x56352bad7b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.128+0000 7f2de2f3e380 1 --2- 172.21.15.78:0/3912093410 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x56352bae7800 0x56352bad7b80 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.128+0000 7f2de2f3e380 1 -- 172.21.15.78:0/3912093410 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x56352babb000 msgr2=0x56352bad6580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.128+0000 7f2de2f3e380 1 --2- 172.21.15.78:0/3912093410 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x56352babb000 0x56352bad6580 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.145 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.128+0000 7f2de2f3e380 1 -- 172.21.15.78:0/3912093410 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x56352baba800 msgr2=0x56352bad6000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.145 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.128+0000 7f2de2f3e380 1 --2- 172.21.15.78:0/3912093410 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x56352baba800 0x56352bad6000 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.145 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.128+0000 7f2de0ef7700 1 -- 172.21.15.78:0/3912093410 reap_dead start 2023-10-11T09:48:49.145 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.128+0000 7f2de0ef7700 1 -- 172.21.15.78:0/3912093410 reap_dead start 2023-10-11T09:48:49.145 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.128+0000 7f2de2f3e380 1 -- 172.21.15.78:0/3912093410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56352baba000 msgr2=0x56352b6a7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.128+0000 7f2de2f3e380 1 --2- 172.21.15.78:0/3912093410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56352baba000 0x56352b6a7b80 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x56352ba26ea0 tx=0x56352c3ee180 comp rx=0 tx=0).stop 2023-10-11T09:48:49.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.128+0000 7f2de0ef7700 1 -- 172.21.15.78:0/3912093410 reap_dead start 2023-10-11T09:48:49.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.128+0000 7f2de2f3e380 1 -- 172.21.15.78:0/3912093410 shutdown_connections 2023-10-11T09:48:49.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.128+0000 7f2de2f3e380 1 -- 172.21.15.78:0/3912093410 >> 172.21.15.78:0/3912093410 conn(0x56352b957800 msgr2=0x56352b6f2000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:49.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.128+0000 7f2de2f3e380 1 -- 172.21.15.78:0/3912093410 shutdown_connections 2023-10-11T09:48:49.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.128+0000 7f2de2f3e380 1 -- 172.21.15.78:0/3912093410 wait complete. 2023-10-11T09:48:49.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.128+0000 7f2de2f3e380 1 -- 172.21.15.78:0/1592087488 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x56352b957400 msgr2=0x56352b6a9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.128+0000 7f2de2f3e380 1 --2- 172.21.15.78:0/1592087488 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x56352b957400 0x56352b6a9700 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x56352b964090 tx=0x56352c3ee1e0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.128+0000 7f2de2f3e380 1 -- 172.21.15.78:0/1592087488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56352b956c00 msgr2=0x56352b6a8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.128+0000 7f2de2f3e380 1 --2- 172.21.15.78:0/1592087488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56352b956c00 0x56352b6a8680 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x56352b9654d0 tx=0x56352c3ee0c0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.132+0000 7f2de2f3e380 1 -- 172.21.15.78:0/1592087488 shutdown_connections 2023-10-11T09:48:49.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.132+0000 7f2de2f3e380 1 --2- 172.21.15.78:0/1592087488 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x56352b957400 0x56352b6a9700 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.132+0000 7f2de2f3e380 1 --2- 172.21.15.78:0/1592087488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56352b956c00 0x56352b6a8680 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.132+0000 7f2de2f3e380 1 --2- 172.21.15.78:0/1592087488 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56352b956800 0x56352b6a8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.132+0000 7f2de2f3e380 1 --2- 172.21.15.78:0/1592087488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352b956400 0x56352b6a9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.132+0000 7f2de2f3e380 1 -- 172.21.15.78:0/1592087488 >> 172.21.15.78:0/1592087488 conn(0x56352b956000 msgr2=0x56352b6f1800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:49.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.132+0000 7f2de2f3e380 1 -- 172.21.15.78:0/1592087488 shutdown_connections 2023-10-11T09:48:49.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.132+0000 7f2de2f3e380 1 -- 172.21.15.78:0/1592087488 wait complete. 2023-10-11T09:48:49.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.132+0000 7f2de2f3e380 1 -- 172.21.15.78:0/1842934256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56352b601800 msgr2=0x56352b6a7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.132+0000 7f2de2f3e380 1 --2- 172.21.15.78:0/1842934256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56352b601800 0x56352b6a7080 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x56352c36ee40 tx=0x56352c3ee480 comp rx=0 tx=0).stop 2023-10-11T09:48:49.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.132+0000 7f2de2f3e380 1 -- 172.21.15.78:0/1842934256 shutdown_connections 2023-10-11T09:48:49.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.132+0000 7f2de2f3e380 1 --2- 172.21.15.78:0/1842934256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56352b601800 0x56352b6a7080 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.132+0000 7f2de2f3e380 1 --2- 172.21.15.78:0/1842934256 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56352b601400 0x56352b6a6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.132+0000 7f2de2f3e380 1 --2- 172.21.15.78:0/1842934256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56352b601000 0x56352b6a6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.132+0000 7f2de2f3e380 1 -- 172.21.15.78:0/1842934256 >> 172.21.15.78:0/1842934256 conn(0x56352b600c00 msgr2=0x56352b6f1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:49.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.132+0000 7f2de2f3e380 1 -- 172.21.15.78:0/1842934256 shutdown_connections 2023-10-11T09:48:49.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.132+0000 7f2de2f3e380 1 -- 172.21.15.78:0/1842934256 wait complete. 2023-10-11T09:48:49.152 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.125106, result: writing EResetJournal entry 2023-10-11T09:48:49.153 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 init 2023-10-11T09:48:49.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.212+0000 7ff3670e5840 1 -- start start 2023-10-11T09:48:49.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.212+0000 7ff3670e5840 1 Processor -- start 2023-10-11T09:48:49.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.212+0000 7ff3670e5840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5572aebe7800 0x5572aec8c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.212+0000 7ff3670e5840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5572aebe7400 0x5572aec8cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.212+0000 7ff3670e5840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572aebe7000 0x5572aec8d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.212+0000 7ff3670e5840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572aec66780 con 0x5572aebe7000 2023-10-11T09:48:49.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.212+0000 7ff3670e5840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572aec66a00 con 0x5572aebe7400 2023-10-11T09:48:49.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.212+0000 7ff3670e5840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x5572aec66640 con 0x5572aebe7800 2023-10-11T09:48:49.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.212+0000 7ff3660a2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572aebe7000 0x5572aec8d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.212+0000 7ff3660a2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572aebe7000 0x5572aec8d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35704/0 (socket says 172.21.15.78:35704) 2023-10-11T09:48:49.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.212+0000 7ff3660a2700 1 -- 172.21.15.78:0/1015222077 learned_addr learned my addr 172.21.15.78:0/1015222077 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:49.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.212+0000 7ff3658a1700 1 --2- 172.21.15.78:0/1015222077 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5572aebe7800 0x5572aec8c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.212+0000 7ff3650a0700 1 --2- 172.21.15.78:0/1015222077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5572aebe7400 0x5572aec8cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.212+0000 7ff3658a1700 1 -- 172.21.15.78:0/1015222077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5572aebe7400 msgr2=0x5572aec8cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:49.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.212+0000 7ff3658a1700 1 --2- 172.21.15.78:0/1015222077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5572aebe7400 0x5572aec8cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.212+0000 7ff3658a1700 1 -- 172.21.15.78:0/1015222077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572aebe7000 msgr2=0x5572aec8d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:49.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.212+0000 7ff3658a1700 1 --2- 172.21.15.78:0/1015222077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572aebe7000 0x5572aec8d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.212+0000 7ff3658a1700 1 -- 172.21.15.78:0/1015222077 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572aec08b60 con 0x5572aebe7800 2023-10-11T09:48:49.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.212+0000 7ff3660a2700 1 --2- 172.21.15.78:0/1015222077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572aebe7000 0x5572aec8d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:48:49.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.216+0000 7ff3658a1700 1 --2- 172.21.15.78:0/1015222077 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5572aebe7800 0x5572aec8c580 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x5572aed433e0 tx=0x5572af9daea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=470448c9387d348b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.216+0000 7ff36309c700 1 -- 172.21.15.78:0/1015222077 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5572aec17600 con 0x5572aebe7800 2023-10-11T09:48:49.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.216+0000 7ff36309c700 1 -- 172.21.15.78:0/1015222077 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5572aec32480 con 0x5572aebe7800 2023-10-11T09:48:49.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.216+0000 7ff36309c700 1 -- 172.21.15.78:0/1015222077 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5572aec171e0 con 0x5572aebe7800 2023-10-11T09:48:49.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.216+0000 7ff3670e5840 1 -- 172.21.15.78:0/1015222077 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5572aec08340 con 0x5572aebe7800 2023-10-11T09:48:49.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.216+0000 7ff3670e5840 1 -- 172.21.15.78:0/1015222077 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5572aec08000 con 0x5572aebe7800 2023-10-11T09:48:49.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.216+0000 7ff3670e5840 4 waiting for MDS map... 2023-10-11T09:48:49.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.216+0000 7ff36309c700 1 -- 172.21.15.78:0/1015222077 <== mon.2 v2:172.21.15.133:3301/0 4 ==== osd_map(88..88 src has 1..88) v4 ==== 6638+0+0 (secure 0 0 0) 0x5572aec20c40 con 0x5572aebe7800 2023-10-11T09:48:49.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.216+0000 7ff36309c700 1 -- 172.21.15.78:0/1015222077 <== mon.2 v2:172.21.15.133:3301/0 5 ==== fsmap(e 64) v1 ==== 3118+0+0 (secure 0 0 0) 0x5572af9e4000 con 0x5572aebe7800 2023-10-11T09:48:49.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.216+0000 7ff3670e5840 4 Got MDS map 64 2023-10-11T09:48:49.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.216+0000 7ff3670e5840 4 datascan.main: Using metadata pool output 2023-10-11T09:48:49.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.216+0000 7ff3670e5840 4 datascan.main: connecting to RADOS... 2023-10-11T09:48:49.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.216+0000 7ff3670e5840 1 Processor -- start 2023-10-11T09:48:49.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.216+0000 7ff3670e5840 1 -- start start 2023-10-11T09:48:49.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.220+0000 7ff3670e5840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5572aeeda800 0x5572aec8d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.220+0000 7ff3670e5840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572aeedac00 0x5572aec8db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.220+0000 7ff3670e5840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5572aeedb000 0x5572aec8e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.220+0000 7ff3670e5840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572aec1de00 con 0x5572aeedac00 2023-10-11T09:48:49.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.220+0000 7ff3670e5840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572aec66780 con 0x5572aeeda800 2023-10-11T09:48:49.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.220+0000 7ff3670e5840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x5572aec66500 con 0x5572aeedb000 2023-10-11T09:48:49.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.220+0000 7ff3658a1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572aeedac00 0x5572aec8db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.220+0000 7ff3658a1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572aeedac00 0x5572aec8db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35714/0 (socket says 172.21.15.78:35714) 2023-10-11T09:48:49.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.220+0000 7ff3658a1700 1 -- 172.21.15.78:0/600087020 learned_addr learned my addr 172.21.15.78:0/600087020 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:49.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.220+0000 7ff3660a2700 1 --2- 172.21.15.78:0/600087020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5572aeeda800 0x5572aec8d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.220+0000 7ff3650a0700 1 --2- 172.21.15.78:0/600087020 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5572aeedb000 0x5572aec8e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.220+0000 7ff3658a1700 1 -- 172.21.15.78:0/600087020 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5572aeedb000 msgr2=0x5572aec8e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:49.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.220+0000 7ff3658a1700 1 --2- 172.21.15.78:0/600087020 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5572aeedb000 0x5572aec8e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.220+0000 7ff3658a1700 1 -- 172.21.15.78:0/600087020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5572aeeda800 msgr2=0x5572aec8d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:49.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.220+0000 7ff3658a1700 1 --2- 172.21.15.78:0/600087020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5572aeeda800 0x5572aec8d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.220+0000 7ff3658a1700 1 -- 172.21.15.78:0/600087020 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572aec08ea0 con 0x5572aeedac00 2023-10-11T09:48:49.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.220+0000 7ff3650a0700 1 --2- 172.21.15.78:0/600087020 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5572aeedb000 0x5572aec8e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:48:49.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.220+0000 7ff3658a1700 1 --2- 172.21.15.78:0/600087020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572aeedac00 0x5572aec8db80 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x5572aeed88d0 tx=0x5572af9dafc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99a47a159406b3db server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.220+0000 7ff361098700 1 -- 172.21.15.78:0/600087020 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5572aec16f20 con 0x5572aeedac00 2023-10-11T09:48:49.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.220+0000 7ff361098700 1 -- 172.21.15.78:0/600087020 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5572af9dcd80 con 0x5572aeedac00 2023-10-11T09:48:49.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.220+0000 7ff361098700 1 -- 172.21.15.78:0/600087020 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5572aec17600 con 0x5572aeedac00 2023-10-11T09:48:49.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.220+0000 7ff3670e5840 1 -- 172.21.15.78:0/600087020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572aeedac00 msgr2=0x5572aec8db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.220+0000 7ff3670e5840 1 --2- 172.21.15.78:0/600087020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572aeedac00 0x5572aec8db80 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x5572aeed88d0 tx=0x5572af9dafc0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.220+0000 7ff3670e5840 1 -- 172.21.15.78:0/600087020 shutdown_connections 2023-10-11T09:48:49.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.220+0000 7ff3670e5840 1 --2- 172.21.15.78:0/600087020 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5572aeedb000 0x5572aec8e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.220+0000 7ff3670e5840 1 --2- 172.21.15.78:0/600087020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572aeedac00 0x5572aec8db80 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.220+0000 7ff3670e5840 1 --2- 172.21.15.78:0/600087020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5572aeeda800 0x5572aec8d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.220+0000 7ff3670e5840 1 -- 172.21.15.78:0/600087020 >> 172.21.15.78:0/600087020 conn(0x5572aebe6c00 msgr2=0x5572aecb5800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:49.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.220+0000 7ff3670e5840 1 -- 172.21.15.78:0/600087020 shutdown_connections 2023-10-11T09:48:49.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.220+0000 7ff3670e5840 1 -- 172.21.15.78:0/600087020 wait complete. 2023-10-11T09:48:49.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.224+0000 7ff3670e5840 1 Processor -- start 2023-10-11T09:48:49.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.224+0000 7ff3670e5840 1 -- start start 2023-10-11T09:48:49.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.224+0000 7ff3670e5840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5572aeeda800 0x5572aec8f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.224+0000 7ff3670e5840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572aeedac00 0x5572aec8ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.224+0000 7ff3670e5840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5572aeedb000 0x5572aec8e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.224+0000 7ff3670e5840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572aec663c0 con 0x5572aeedac00 2023-10-11T09:48:49.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.224+0000 7ff3670e5840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572aec66280 con 0x5572aeedb000 2023-10-11T09:48:49.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.224+0000 7ff3670e5840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x5572aec66140 con 0x5572aeeda800 2023-10-11T09:48:49.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.224+0000 7ff3658a1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572aeedac00 0x5572aec8ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.224+0000 7ff3658a1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572aeedac00 0x5572aec8ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35726/0 (socket says 172.21.15.78:35726) 2023-10-11T09:48:49.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.224+0000 7ff3658a1700 1 -- 172.21.15.78:0/3250609143 learned_addr learned my addr 172.21.15.78:0/3250609143 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:49.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.224+0000 7ff3660a2700 1 --2- 172.21.15.78:0/3250609143 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5572aeeda800 0x5572aec8f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.224+0000 7ff3650a0700 1 --2- 172.21.15.78:0/3250609143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5572aeedb000 0x5572aec8e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.224+0000 7ff3658a1700 1 -- 172.21.15.78:0/3250609143 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5572aeeda800 msgr2=0x5572aec8f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.224+0000 7ff3658a1700 1 --2- 172.21.15.78:0/3250609143 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5572aeeda800 0x5572aec8f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.224+0000 7ff3658a1700 1 -- 172.21.15.78:0/3250609143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5572aeedb000 msgr2=0x5572aec8e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.224+0000 7ff3658a1700 1 --2- 172.21.15.78:0/3250609143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5572aeedb000 0x5572aec8e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.224+0000 7ff3658a1700 1 -- 172.21.15.78:0/3250609143 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572aec08b60 con 0x5572aeedac00 2023-10-11T09:48:49.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.228+0000 7ff3658a1700 1 --2- 172.21.15.78:0/3250609143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572aeedac00 0x5572aec8ec00 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x5572aeed9590 tx=0x5572af9daf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.228+0000 7ff35f895700 1 -- 172.21.15.78:0/3250609143 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5572aec171e0 con 0x5572aeedac00 2023-10-11T09:48:49.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.228+0000 7ff35f895700 1 -- 172.21.15.78:0/3250609143 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5572af9dca80 con 0x5572aeedac00 2023-10-11T09:48:49.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.228+0000 7ff3670e5840 1 -- 172.21.15.78:0/3250609143 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5572aec08340 con 0x5572aeedac00 2023-10-11T09:48:49.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.228+0000 7ff35f895700 1 -- 172.21.15.78:0/3250609143 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5572aec16dc0 con 0x5572aeedac00 2023-10-11T09:48:49.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.228+0000 7ff3670e5840 1 -- 172.21.15.78:0/3250609143 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5572aec09520 con 0x5572aeedac00 2023-10-11T09:48:49.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.228+0000 7ff3670e5840 4 datascan.init: resolving metadata pool 11 2023-10-11T09:48:49.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.228+0000 7ff35f895700 1 -- 172.21.15.78:0/3250609143 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x5572aec44840 con 0x5572aeedac00 2023-10-11T09:48:49.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.232+0000 7ff35f895700 1 --2- 172.21.15.78:0/3250609143 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5572aef48800 0x5572aec8f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.232+0000 7ff35f895700 1 -- 172.21.15.78:0/3250609143 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6638+0+0 (secure 0 0 0) 0x5572aec20a80 con 0x5572aeedac00 2023-10-11T09:48:49.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.232+0000 7ff3670e5840 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-10-11T09:48:49.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.232+0000 7ff3670e5840 1 --2- 172.21.15.78:0/3250609143 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x5572aeedb400 0x5572aec8e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.232+0000 7ff3660a2700 1 --2- 172.21.15.78:0/3250609143 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5572aef48800 0x5572aec8f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.232+0000 7ff3670e5840 1 -- 172.21.15.78:0/3250609143 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:1 11.14 11:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x5572aeedb800 con 0x5572aeedb400 2023-10-11T09:48:49.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.232+0000 7ff3650a0700 1 --2- 172.21.15.78:0/3250609143 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x5572aeedb400 0x5572aec8e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.232+0000 7ff3660a2700 1 --2- 172.21.15.78:0/3250609143 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5572aef48800 0x5572aec8f700 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x5572aed433b0 tx=0x5572aeedcf00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.232+0000 7ff3650a0700 1 --2- 172.21.15.78:0/3250609143 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x5572aeedb400 0x5572aec8e100 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.232+0000 7ff3650a0700 1 -- 172.21.15.78:0/3250609143 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== osd_op_reply(1 1.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 160+0+0 (crc 0 0 0) 0x5572afa70fc0 con 0x5572aeedb400 2023-10-11T09:48:49.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.232+0000 7ff3670e5840 1 -- 172.21.15.78:0/3250609143 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:2 11.14 11:287034c4:::1.00000000.inode:head [writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x5572aeedbc00 con 0x5572aeedb400 2023-10-11T09:48:49.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.236+0000 7ff3650a0700 1 -- 172.21.15.78:0/3250609143 <== osd.6 v2:172.21.15.133:6800/35900 2 ==== osd_op_reply(2 1.00000000.inode [writefull 0~582] v88'29 uv29 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x5572afa70fc0 con 0x5572aeedb400 2023-10-11T09:48:49.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.236+0000 7ff3670e5840 1 --2- 172.21.15.78:0/3250609143 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x5572aef48c00 0x5572aec8db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.236+0000 7ff3670e5840 1 -- 172.21.15.78:0/3250609143 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:3 11.1f 11:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x5572aef49000 con 0x5572aef48c00 2023-10-11T09:48:49.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.236+0000 7ff3660a2700 1 --2- 172.21.15.78:0/3250609143 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x5572aef48c00 0x5572aec8db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.236+0000 7ff3660a2700 1 --2- 172.21.15.78:0/3250609143 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x5572aef48c00 0x5572aec8db80 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.240+0000 7ff3660a2700 1 -- 172.21.15.78:0/3250609143 <== osd.2 v2:172.21.15.78:6808/35732 1 ==== osd_op_reply(3 100.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 162+0+0 (crc 0 0 0) 0x5572afa71680 con 0x5572aef48c00 2023-10-11T09:48:49.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.240+0000 7ff3670e5840 1 -- 172.21.15.78:0/3250609143 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:4 11.1f 11:fe07bba1:::100.00000000.inode:head [writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x5572aef49400 con 0x5572aef48c00 2023-10-11T09:48:49.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.240+0000 7ff3660a2700 1 -- 172.21.15.78:0/3250609143 <== osd.2 v2:172.21.15.78:6808/35732 2 ==== osd_op_reply(4 100.00000000.inode [writefull 0~582] v88'10 uv10 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x5572afa71680 con 0x5572aef48c00 2023-10-11T09:48:49.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.240+0000 7ff3670e5840 1 --2- 172.21.15.78:0/3250609143 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x5572aef49800 0x5572aec8d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.240+0000 7ff3670e5840 1 -- 172.21.15.78:0/3250609143 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:5 11.13 11:cd5a64a3:::100.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x5572aef49c00 con 0x5572aef49800 2023-10-11T09:48:49.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.244+0000 7ff3658a1700 1 --2- 172.21.15.78:0/3250609143 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x5572aef49800 0x5572aec8d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.244+0000 7ff3658a1700 1 --2- 172.21.15.78:0/3250609143 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x5572aef49800 0x5572aec8d600 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.244+0000 7ff3658a1700 1 -- 172.21.15.78:0/3250609143 <== osd.5 v2:172.21.15.133:6816/35899 1 ==== osd_op_reply(5 100.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5572afa71d40 con 0x5572aef49800 2023-10-11T09:48:49.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.244+0000 7ff3670e5840 10 datascan.find_or_create_dirfrag: read_version = 0 2023-10-11T09:48:49.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.244+0000 7ff3670e5840 1 -- 172.21.15.78:0/3250609143 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:6 11.13 11:cd5a64a3:::100.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x5572aeedb800 con 0x5572aef49800 2023-10-11T09:48:49.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.248+0000 7ff3658a1700 1 -- 172.21.15.78:0/3250609143 <== osd.5 v2:172.21.15.133:6816/35899 2 ==== osd_op_reply(6 100.00000000 [create,omap-set-header] v88'7 uv7 ondisk = 0) v8 ==== 198+0+0 (crc 0 0 0) 0x5572afa71d40 con 0x5572aef49800 2023-10-11T09:48:49.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.248+0000 7ff3670e5840 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x100 2023-10-11T09:48:49.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.248+0000 7ff3670e5840 1 -- 172.21.15.78:0/3250609143 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x5572aef48c00 msgr2=0x5572aec8db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.248+0000 7ff3670e5840 1 --2- 172.21.15.78:0/3250609143 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x5572aef48c00 0x5572aec8db80 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.248+0000 7ff3670e5840 1 -- 172.21.15.78:0/3250609143 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x5572aef49800 msgr2=0x5572aec8d600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.248+0000 7ff3670e5840 1 --2- 172.21.15.78:0/3250609143 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x5572aef49800 0x5572aec8d600 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.248+0000 7ff3670e5840 1 -- 172.21.15.78:0/3250609143 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x5572aeedb400 msgr2=0x5572aec8e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.248+0000 7ff3670e5840 1 --2- 172.21.15.78:0/3250609143 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x5572aeedb400 0x5572aec8e100 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.248+0000 7ff3650a0700 1 -- 172.21.15.78:0/3250609143 reap_dead start 2023-10-11T09:48:49.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.248+0000 7ff3670e5840 1 -- 172.21.15.78:0/3250609143 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5572aef48800 msgr2=0x5572aec8f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.248+0000 7ff3670e5840 1 --2- 172.21.15.78:0/3250609143 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5572aef48800 0x5572aec8f700 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x5572aed433b0 tx=0x5572aeedcf00 comp rx=0 tx=0).stop 2023-10-11T09:48:49.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.248+0000 7ff3670e5840 1 -- 172.21.15.78:0/3250609143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572aeedac00 msgr2=0x5572aec8ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.248+0000 7ff3670e5840 1 --2- 172.21.15.78:0/3250609143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572aeedac00 0x5572aec8ec00 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x5572aeed9590 tx=0x5572af9daf60 comp rx=0 tx=0).stop 2023-10-11T09:48:49.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.248+0000 7ff3670e5840 1 -- 172.21.15.78:0/3250609143 shutdown_connections 2023-10-11T09:48:49.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.248+0000 7ff3670e5840 1 --2- 172.21.15.78:0/3250609143 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x5572aef48800 0x5572aec8f700 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.248+0000 7ff3670e5840 1 --2- 172.21.15.78:0/3250609143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572aeedac00 0x5572aec8ec00 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.248+0000 7ff3670e5840 1 -- 172.21.15.78:0/3250609143 >> 172.21.15.78:0/3250609143 conn(0x5572aebe6c00 msgr2=0x5572aecb5800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:49.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.248+0000 7ff3670e5840 1 -- 172.21.15.78:0/3250609143 shutdown_connections 2023-10-11T09:48:49.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.248+0000 7ff3670e5840 1 -- 172.21.15.78:0/3250609143 wait complete. 2023-10-11T09:48:49.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.248+0000 7ff3670e5840 1 -- 172.21.15.78:0/1015222077 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5572aebe7800 msgr2=0x5572aec8c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.248+0000 7ff3670e5840 1 --2- 172.21.15.78:0/1015222077 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5572aebe7800 0x5572aec8c580 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x5572aed433e0 tx=0x5572af9daea0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.248+0000 7ff3670e5840 1 -- 172.21.15.78:0/1015222077 shutdown_connections 2023-10-11T09:48:49.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.248+0000 7ff3670e5840 1 --2- 172.21.15.78:0/1015222077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572aebe7000 0x5572aec8d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.248+0000 7ff3670e5840 1 --2- 172.21.15.78:0/1015222077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5572aebe7400 0x5572aec8cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.248+0000 7ff3670e5840 1 --2- 172.21.15.78:0/1015222077 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5572aebe7800 0x5572aec8c580 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.248+0000 7ff3670e5840 1 -- 172.21.15.78:0/1015222077 >> 172.21.15.78:0/1015222077 conn(0x5572aebe7c00 msgr2=0x5572aecb5000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:49.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.248+0000 7ff3670e5840 1 -- 172.21.15.78:0/1015222077 shutdown_connections 2023-10-11T09:48:49.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.248+0000 7ff3670e5840 1 -- 172.21.15.78:0/1015222077 wait complete. 2023-10-11T09:48:49.263 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'init'] in time 0:00:00.109855, result: 2023-10-11T09:48:49.263 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents 2023-10-11T09:48:49.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.324+0000 7f0bceb44840 1 -- start start 2023-10-11T09:48:49.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.324+0000 7f0bceb44840 1 Processor -- start 2023-10-11T09:48:49.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.324+0000 7f0bceb44840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559ba04e5800 0x559ba058a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.324+0000 7f0bceb44840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ba04e5400 0x559ba058ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.324+0000 7f0bceb44840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559ba04e5000 0x559ba058b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.324+0000 7f0bceb44840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ba051be00 con 0x559ba04e5400 2023-10-11T09:48:49.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.324+0000 7f0bceb44840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ba0564a00 con 0x559ba04e5000 2023-10-11T09:48:49.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.324+0000 7f0bceb44840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ba0564640 con 0x559ba04e5800 2023-10-11T09:48:49.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.328+0000 7f0bccaff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ba04e5400 0x559ba058ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.328+0000 7f0bccaff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ba04e5400 0x559ba058ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35738/0 (socket says 172.21.15.78:35738) 2023-10-11T09:48:49.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.328+0000 7f0bccaff700 1 -- 172.21.15.78:0/1975239613 learned_addr learned my addr 172.21.15.78:0/1975239613 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:49.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.328+0000 7f0bcdb01700 1 --2- 172.21.15.78:0/1975239613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559ba04e5000 0x559ba058b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.328+0000 7f0bcd300700 1 --2- 172.21.15.78:0/1975239613 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559ba04e5800 0x559ba058a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.328+0000 7f0bccaff700 1 -- 172.21.15.78:0/1975239613 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559ba04e5800 msgr2=0x559ba058a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:49.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.328+0000 7f0bccaff700 1 --2- 172.21.15.78:0/1975239613 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559ba04e5800 0x559ba058a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.328+0000 7f0bccaff700 1 -- 172.21.15.78:0/1975239613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559ba04e5000 msgr2=0x559ba058b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:49.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.328+0000 7f0bccaff700 1 --2- 172.21.15.78:0/1975239613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559ba04e5000 0x559ba058b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.328+0000 7f0bccaff700 1 -- 172.21.15.78:0/1975239613 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ba0506b60 con 0x559ba04e5400 2023-10-11T09:48:49.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.328+0000 7f0bccaff700 1 --2- 172.21.15.78:0/1975239613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ba04e5400 0x559ba058ab00 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x559ba124d3e0 tx=0x559ba12cad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168a193d2ed61d0d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.328+0000 7f0bcaafb700 1 -- 172.21.15.78:0/1975239613 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559ba0515600 con 0x559ba04e5400 2023-10-11T09:48:49.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.328+0000 7f0bcaafb700 1 -- 172.21.15.78:0/1975239613 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559ba0530480 con 0x559ba04e5400 2023-10-11T09:48:49.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.328+0000 7f0bcaafb700 1 -- 172.21.15.78:0/1975239613 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559ba05151e0 con 0x559ba04e5400 2023-10-11T09:48:49.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.328+0000 7f0bceb44840 1 -- 172.21.15.78:0/1975239613 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559ba0506340 con 0x559ba04e5400 2023-10-11T09:48:49.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.328+0000 7f0bceb44840 1 -- 172.21.15.78:0/1975239613 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x559ba0506000 con 0x559ba04e5400 2023-10-11T09:48:49.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.328+0000 7f0bceb44840 4 waiting for MDS map... 2023-10-11T09:48:49.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.328+0000 7f0bcaafb700 1 -- 172.21.15.78:0/1975239613 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(88..88 src has 1..88) v4 ==== 6638+0+0 (secure 0 0 0) 0x559ba051ec40 con 0x559ba04e5400 2023-10-11T09:48:49.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.328+0000 7f0bcaafb700 1 -- 172.21.15.78:0/1975239613 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 64) v1 ==== 3118+0+0 (secure 0 0 0) 0x559ba1362000 con 0x559ba04e5400 2023-10-11T09:48:49.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.328+0000 7f0bceb44840 4 Got MDS map 64 2023-10-11T09:48:49.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.328+0000 7f0bceb44840 4 datascan.main: Using metadata pool output 2023-10-11T09:48:49.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.328+0000 7f0bceb44840 4 datascan.main: connecting to RADOS... 2023-10-11T09:48:49.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.328+0000 7f0bceb44840 1 Processor -- start 2023-10-11T09:48:49.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.328+0000 7f0bceb44840 1 -- start start 2023-10-11T09:48:49.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.332+0000 7f0bceb44840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ba07e2400 0x559ba058b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.332+0000 7f0bceb44840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559ba07e2800 0x559ba058bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.332+0000 7f0bceb44840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559ba07e2c00 0x559ba058c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.332+0000 7f0bceb44840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ba0564500 con 0x559ba07e2400 2023-10-11T09:48:49.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.332+0000 7f0bceb44840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ba05643c0 con 0x559ba07e2c00 2023-10-11T09:48:49.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.332+0000 7f0bceb44840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ba0564a00 con 0x559ba07e2800 2023-10-11T09:48:49.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.332+0000 7f0bcdb01700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ba07e2400 0x559ba058b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.332+0000 7f0bcdb01700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ba07e2400 0x559ba058b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35740/0 (socket says 172.21.15.78:35740) 2023-10-11T09:48:49.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.332+0000 7f0bcdb01700 1 -- 172.21.15.78:0/3467720587 learned_addr learned my addr 172.21.15.78:0/3467720587 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:49.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.332+0000 7f0bccaff700 1 --2- 172.21.15.78:0/3467720587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559ba07e2c00 0x559ba058c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.332+0000 7f0bcd300700 1 --2- 172.21.15.78:0/3467720587 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559ba07e2800 0x559ba058bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.332+0000 7f0bcdb01700 1 -- 172.21.15.78:0/3467720587 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559ba07e2800 msgr2=0x559ba058bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:49.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.332+0000 7f0bcdb01700 1 --2- 172.21.15.78:0/3467720587 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559ba07e2800 0x559ba058bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.332+0000 7f0bcdb01700 1 -- 172.21.15.78:0/3467720587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559ba07e2c00 msgr2=0x559ba058c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:49.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.332+0000 7f0bcdb01700 1 --2- 172.21.15.78:0/3467720587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559ba07e2c00 0x559ba058c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.332+0000 7f0bcdb01700 1 -- 172.21.15.78:0/3467720587 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ba0506ea0 con 0x559ba07e2400 2023-10-11T09:48:49.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.332+0000 7f0bcdb01700 1 --2- 172.21.15.78:0/3467720587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ba07e2400 0x559ba058b600 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x559ba07d4840 tx=0x559ba07d6c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e18d1c95e86e7d35 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.332+0000 7f0bc8af7700 1 -- 172.21.15.78:0/3467720587 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559ba0515600 con 0x559ba07e2400 2023-10-11T09:48:49.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.332+0000 7f0bc8af7700 1 -- 172.21.15.78:0/3467720587 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559ba0530900 con 0x559ba07e2400 2023-10-11T09:48:49.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.332+0000 7f0bc8af7700 1 -- 172.21.15.78:0/3467720587 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559ba05151e0 con 0x559ba07e2400 2023-10-11T09:48:49.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.332+0000 7f0bceb44840 1 -- 172.21.15.78:0/3467720587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ba07e2400 msgr2=0x559ba058b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.332+0000 7f0bceb44840 1 --2- 172.21.15.78:0/3467720587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ba07e2400 0x559ba058b600 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x559ba07d4840 tx=0x559ba07d6c00 comp rx=0 tx=0).stop 2023-10-11T09:48:49.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.332+0000 7f0bceb44840 1 -- 172.21.15.78:0/3467720587 shutdown_connections 2023-10-11T09:48:49.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.332+0000 7f0bceb44840 1 --2- 172.21.15.78:0/3467720587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559ba07e2c00 0x559ba058c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.332+0000 7f0bceb44840 1 --2- 172.21.15.78:0/3467720587 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559ba07e2800 0x559ba058bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.332+0000 7f0bceb44840 1 --2- 172.21.15.78:0/3467720587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ba07e2400 0x559ba058b600 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.332+0000 7f0bceb44840 1 -- 172.21.15.78:0/3467720587 >> 172.21.15.78:0/3467720587 conn(0x559ba04e4c00 msgr2=0x559ba05b3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:49.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.332+0000 7f0bceb44840 1 -- 172.21.15.78:0/3467720587 shutdown_connections 2023-10-11T09:48:49.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.332+0000 7f0bceb44840 1 -- 172.21.15.78:0/3467720587 wait complete. 2023-10-11T09:48:49.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.336+0000 7f0bceb44840 1 Processor -- start 2023-10-11T09:48:49.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.336+0000 7f0bceb44840 1 -- start start 2023-10-11T09:48:49.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.336+0000 7f0bceb44840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559ba07e2400 0x559ba058d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.336+0000 7f0bceb44840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559ba07e2800 0x559ba058cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.336+0000 7f0bceb44840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ba07e2c00 0x559ba058c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.336+0000 7f0bceb44840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ba0564280 con 0x559ba07e2c00 2023-10-11T09:48:49.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.336+0000 7f0bceb44840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ba0564140 con 0x559ba07e2800 2023-10-11T09:48:49.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.336+0000 7f0bceb44840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ba0564000 con 0x559ba07e2400 2023-10-11T09:48:49.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.336+0000 7f0bccaff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ba07e2c00 0x559ba058c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.336+0000 7f0bccaff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ba07e2c00 0x559ba058c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35750/0 (socket says 172.21.15.78:35750) 2023-10-11T09:48:49.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.336+0000 7f0bccaff700 1 -- 172.21.15.78:0/2409963846 learned_addr learned my addr 172.21.15.78:0/2409963846 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:49.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.336+0000 7f0bcd300700 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559ba07e2800 0x559ba058cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.336+0000 7f0bcdb01700 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559ba07e2400 0x559ba058d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.336+0000 7f0bccaff700 1 -- 172.21.15.78:0/2409963846 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559ba07e2400 msgr2=0x559ba058d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.336+0000 7f0bccaff700 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559ba07e2400 0x559ba058d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.336+0000 7f0bccaff700 1 -- 172.21.15.78:0/2409963846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559ba07e2800 msgr2=0x559ba058cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.336+0000 7f0bccaff700 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559ba07e2800 0x559ba058cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.336+0000 7f0bccaff700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ba0506b60 con 0x559ba07e2c00 2023-10-11T09:48:49.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.336+0000 7f0bccaff700 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ba07e2c00 0x559ba058c680 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x559ba07d46f0 tx=0x559ba12caba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.340+0000 7f0bc72f4700 1 -- 172.21.15.78:0/2409963846 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559ba0514f20 con 0x559ba07e2c00 2023-10-11T09:48:49.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.340+0000 7f0bc72f4700 1 -- 172.21.15.78:0/2409963846 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559ba05dac00 con 0x559ba07e2c00 2023-10-11T09:48:49.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.340+0000 7f0bc72f4700 1 -- 172.21.15.78:0/2409963846 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559ba0515600 con 0x559ba07e2c00 2023-10-11T09:48:49.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.340+0000 7f0bceb44840 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559ba0506340 con 0x559ba07e2c00 2023-10-11T09:48:49.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.340+0000 7f0bceb44840 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559ba0507520 con 0x559ba07e2c00 2023-10-11T09:48:49.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.340+0000 7f0bceb44840 4 datascan.init: resolving metadata pool 11 2023-10-11T09:48:49.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.340+0000 7f0bc72f4700 1 -- 172.21.15.78:0/2409963846 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x559ba0542840 con 0x559ba07e2c00 2023-10-11T09:48:49.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.340+0000 7f0bc72f4700 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x559ba0938000 0x559ba058d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.344+0000 7f0bc72f4700 1 -- 172.21.15.78:0/2409963846 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6638+0+0 (secure 0 0 0) 0x559ba051ea80 con 0x559ba07e2c00 2023-10-11T09:48:49.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.344+0000 7f0bceb44840 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-10-11T09:48:49.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.344+0000 7f0bceb44840 4 datascan.main: data pool 'cephfs_data' has ID 12 2023-10-11T09:48:49.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.344+0000 7f0bceb44840 4 datascan.main: opening data pool 'cephfs_data' 2023-10-11T09:48:49.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.344+0000 7f0bceb44840 4 datascan.main: data pool 'cephfs_data_extra' has ID 13 2023-10-11T09:48:49.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.344+0000 7f0bceb44840 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-10-11T09:48:49.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.344+0000 7f0bceb44840 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x559ba07e3000 0x559ba058c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.344+0000 7f0bceb44840 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:1 12.0 12:00000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba07e3400 con 0x559ba07e3000 2023-10-11T09:48:49.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.344+0000 7f0bcdb01700 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x559ba0938000 0x559ba058d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.344+0000 7f0bcd300700 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x559ba07e3000 0x559ba058c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.344+0000 7f0bcdb01700 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x559ba0938000 0x559ba058d700 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x559ba07d4840 tx=0x559ba12cad20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.344+0000 7f0bcd300700 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x559ba07e3000 0x559ba058c100 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.344+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.4 v2:172.21.15.133:6808/35898 1 ==== osd_op_reply(1 [pgnls start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba07e3000 2023-10-11T09:48:49.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.344+0000 7f0bc82f6700 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x559ba0938400 0x559ba094e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.344+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:2 12.20 12:04000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba0938800 con 0x559ba0938400 2023-10-11T09:48:49.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.348+0000 7f0bcdb01700 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x559ba0938400 0x559ba094e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.348+0000 7f0bcdb01700 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x559ba0938400 0x559ba094e000 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.348+0000 7f0bcdb01700 1 -- 172.21.15.78:0/2409963846 <== osd.7 v2:172.21.15.133:6824/35901 1 ==== osd_op_reply(2 [pgnls start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359200 con 0x559ba0938400 2023-10-11T09:48:49.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.348+0000 7f0bc8af7700 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x559ba0939000 0x559ba094e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.348+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:3 12.10 12:08000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba0939400 con 0x559ba0939000 2023-10-11T09:48:49.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.348+0000 7f0bcd300700 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x559ba0939000 0x559ba094e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.348+0000 7f0bcd300700 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x559ba0939000 0x559ba094e580 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.352+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.0 v2:172.21.15.78:6800/35735 1 ==== osd_op_reply(3 [pgnls start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba0939000 2023-10-11T09:48:49.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.352+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:4 12.30 12:0c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba0938c00 con 0x559ba0938400 2023-10-11T09:48:49.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.352+0000 7f0bcdb01700 1 -- 172.21.15.78:0/2409963846 <== osd.7 v2:172.21.15.133:6824/35901 2 ==== osd_op_reply(4 [pgnls start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359200 con 0x559ba0938400 2023-10-11T09:48:49.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.352+0000 7f0bc8af7700 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x559ba0939800 0x559ba094eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.352+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:5 12.8 12:10000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba0938800 con 0x559ba0939800 2023-10-11T09:48:49.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.352+0000 7f0bccaff700 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x559ba0939800 0x559ba094eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.352+0000 7f0bccaff700 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x559ba0939800 0x559ba094eb00 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.352+0000 7f0bccaff700 1 -- 172.21.15.78:0/2409963846 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(5 [pgnls start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359440 con 0x559ba0939800 2023-10-11T09:48:49.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.352+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:6 12.28 12:14000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba0939400 con 0x559ba0939000 2023-10-11T09:48:49.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.352+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.0 v2:172.21.15.78:6800/35735 2 ==== osd_op_reply(6 [pgnls start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba0939000 2023-10-11T09:48:49.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.356+0000 7f0bc8af7700 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x559ba0938c00 0x559ba094f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.356+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:7 12.18 12:18000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba07e3400 con 0x559ba0938c00 2023-10-11T09:48:49.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.356+0000 7f0bcdb01700 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x559ba0938c00 0x559ba094f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.356+0000 7f0bcdb01700 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x559ba0938c00 0x559ba094f080 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.356+0000 7f0bcdb01700 1 -- 172.21.15.78:0/2409963846 <== osd.1 v2:172.21.15.78:6809/35734 1 ==== osd_op_reply(7 [pgnls start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359200 con 0x559ba0938c00 2023-10-11T09:48:49.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.356+0000 7f0bc82f6700 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x559ba0939c00 0x559ba094f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.356+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:8 12.38 12:1c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba096a000 con 0x559ba0939c00 2023-10-11T09:48:49.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.356+0000 7f0bcd300700 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x559ba0939c00 0x559ba094f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.356+0000 7f0bcd300700 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x559ba0939c00 0x559ba094f600 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.360+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.2 v2:172.21.15.78:6808/35732 1 ==== osd_op_reply(8 [pgnls start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba0939c00 2023-10-11T09:48:49.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.360+0000 7f0bc8af7700 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x559ba096b000 0x559ba0950100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.360+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:9 12.4 12:20000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba096b400 con 0x559ba096b000 2023-10-11T09:48:49.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.360+0000 7f0bccaff700 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x559ba096b000 0x559ba0950100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.360+0000 7f0bccaff700 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x559ba096b000 0x559ba0950100 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.360+0000 7f0bccaff700 1 -- 172.21.15.78:0/2409963846 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== osd_op_reply(9 [pgnls start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359440 con 0x559ba096b000 2023-10-11T09:48:49.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.360+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:10 12.24 12:24000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba096a400 con 0x559ba0939c00 2023-10-11T09:48:49.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.360+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.2 v2:172.21.15.78:6808/35732 2 ==== osd_op_reply(10 [pgnls start_epoch 88 out=49b] v86'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba0939c00 2023-10-11T09:48:49.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.364+0000 7f0bc8af7700 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x559ba096b800 0x559ba0950680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.364+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:11 12.14 12:28000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba096bc00 con 0x559ba096b800 2023-10-11T09:48:49.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.364+0000 7f0bcdb01700 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x559ba096b800 0x559ba0950680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.364+0000 7f0bcdb01700 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x559ba096b800 0x559ba0950680 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.364+0000 7f0bcdb01700 1 -- 172.21.15.78:0/2409963846 <== osd.5 v2:172.21.15.133:6816/35899 1 ==== osd_op_reply(11 [pgnls start_epoch 88 out=49b] v86'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359200 con 0x559ba096b800 2023-10-11T09:48:49.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.364+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:12 12.34 12:2c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba096a800 con 0x559ba096b800 2023-10-11T09:48:49.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.364+0000 7f0bcdb01700 1 -- 172.21.15.78:0/2409963846 <== osd.5 v2:172.21.15.133:6816/35899 2 ==== osd_op_reply(12 [pgnls start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359200 con 0x559ba096b800 2023-10-11T09:48:49.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.364+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:13 12.c 12:30000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba096bc00 con 0x559ba0939000 2023-10-11T09:48:49.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.368+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.0 v2:172.21.15.78:6800/35735 3 ==== osd_op_reply(13 [pgnls start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba0939000 2023-10-11T09:48:49.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.368+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:14 12.2c 12:34000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba096ac00 con 0x559ba0939c00 2023-10-11T09:48:49.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.368+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.2 v2:172.21.15.78:6808/35732 3 ==== osd_op_reply(14 [pgnls start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba0939c00 2023-10-11T09:48:49.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.368+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:15 12.1c 12:38000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba096a800 con 0x559ba0939000 2023-10-11T09:48:49.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.368+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.0 v2:172.21.15.78:6800/35735 4 ==== osd_op_reply(15 [pgnls start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba0939000 2023-10-11T09:48:49.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.368+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:16 12.3c 12:3c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba096a400 con 0x559ba096b800 2023-10-11T09:48:49.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.368+0000 7f0bcdb01700 1 -- 172.21.15.78:0/2409963846 <== osd.5 v2:172.21.15.133:6816/35899 3 ==== osd_op_reply(16 [pgnls start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359200 con 0x559ba096b800 2023-10-11T09:48:49.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.368+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:17 12.2 12:40000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba096a000 con 0x559ba07e3000 2023-10-11T09:48:49.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.372+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.4 v2:172.21.15.133:6808/35898 2 ==== osd_op_reply(17 [pgnls start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba07e3000 2023-10-11T09:48:49.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.372+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:18 12.22 12:44000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba096bc00 con 0x559ba0938400 2023-10-11T09:48:49.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.372+0000 7f0bcdb01700 1 -- 172.21.15.78:0/2409963846 <== osd.7 v2:172.21.15.133:6824/35901 3 ==== osd_op_reply(18 [pgnls start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359200 con 0x559ba0938400 2023-10-11T09:48:49.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.372+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:19 12.12 12:48000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba0939400 con 0x559ba0939800 2023-10-11T09:48:49.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.372+0000 7f0bccaff700 1 -- 172.21.15.78:0/2409963846 <== osd.3 v2:172.21.15.78:6824/35738 2 ==== osd_op_reply(19 [pgnls start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359440 con 0x559ba0939800 2023-10-11T09:48:49.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.372+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:20 12.32 12:4c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba096ac00 con 0x559ba0939800 2023-10-11T09:48:49.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.372+0000 7f0bccaff700 1 -- 172.21.15.78:0/2409963846 <== osd.3 v2:172.21.15.78:6824/35738 3 ==== osd_op_reply(20 [pgnls start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359440 con 0x559ba0939800 2023-10-11T09:48:49.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.372+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:21 12.a 12:50000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba07e3400 con 0x559ba0939c00 2023-10-11T09:48:49.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.372+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.2 v2:172.21.15.78:6808/35732 4 ==== osd_op_reply(21 [pgnls start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba0939c00 2023-10-11T09:48:49.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.372+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:22 12.2a 12:54000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba096a800 con 0x559ba0939c00 2023-10-11T09:48:49.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.372+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.2 v2:172.21.15.78:6808/35732 5 ==== osd_op_reply(22 [pgnls start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba0939c00 2023-10-11T09:48:49.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.372+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:23 12.1a 12:58000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba096b400 con 0x559ba07e3000 2023-10-11T09:48:49.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.376+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.4 v2:172.21.15.133:6808/35898 3 ==== osd_op_reply(23 [pgnls start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba07e3000 2023-10-11T09:48:49.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.376+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:24 12.3a 12:5c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba096a000 con 0x559ba0939800 2023-10-11T09:48:49.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.376+0000 7f0bccaff700 1 -- 172.21.15.78:0/2409963846 <== osd.3 v2:172.21.15.78:6824/35738 4 ==== osd_op_reply(24 [pgnls start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359440 con 0x559ba0939800 2023-10-11T09:48:49.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.376+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:25 12.6 12:60000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba096ac00 con 0x559ba0938c00 2023-10-11T09:48:49.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.376+0000 7f0bcdb01700 1 -- 172.21.15.78:0/2409963846 <== osd.1 v2:172.21.15.78:6809/35734 2 ==== osd_op_reply(25 [pgnls start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359200 con 0x559ba0938c00 2023-10-11T09:48:49.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.376+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:26 12.26 12:64000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba097a000 con 0x559ba096b800 2023-10-11T09:48:49.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.376+0000 7f0bcdb01700 1 -- 172.21.15.78:0/2409963846 <== osd.5 v2:172.21.15.133:6816/35899 4 ==== osd_op_reply(26 [pgnls start_epoch 88 out=49b] v86'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359200 con 0x559ba096b800 2023-10-11T09:48:49.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.376+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:27 12.16 12:68000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba0938800 con 0x559ba096b000 2023-10-11T09:48:49.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.380+0000 7f0bccaff700 1 -- 172.21.15.78:0/2409963846 <== osd.6 v2:172.21.15.133:6800/35900 2 ==== osd_op_reply(27 [pgnls start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359440 con 0x559ba096b000 2023-10-11T09:48:49.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.380+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:28 12.36 12:6c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba097a400 con 0x559ba096b800 2023-10-11T09:48:49.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.380+0000 7f0bcdb01700 1 -- 172.21.15.78:0/2409963846 <== osd.5 v2:172.21.15.133:6816/35899 5 ==== osd_op_reply(28 [pgnls start_epoch 88 out=49b] v86'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359200 con 0x559ba096b800 2023-10-11T09:48:49.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.380+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:29 12.e 12:70000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba0939400 con 0x559ba0939000 2023-10-11T09:48:49.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.380+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.0 v2:172.21.15.78:6800/35735 5 ==== osd_op_reply(29 [pgnls start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba0939000 2023-10-11T09:48:49.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.380+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:30 12.2e 12:74000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba097a800 con 0x559ba0938c00 2023-10-11T09:48:49.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.380+0000 7f0bcdb01700 1 -- 172.21.15.78:0/2409963846 <== osd.1 v2:172.21.15.78:6809/35734 3 ==== osd_op_reply(30 [pgnls start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359200 con 0x559ba0938c00 2023-10-11T09:48:49.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.380+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:31 12.1e 12:78000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba097a000 con 0x559ba07e3000 2023-10-11T09:48:49.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.384+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.4 v2:172.21.15.133:6808/35898 4 ==== osd_op_reply(31 [pgnls start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba07e3000 2023-10-11T09:48:49.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.384+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:32 12.3e 12:7c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba097ac00 con 0x559ba0939000 2023-10-11T09:48:49.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.384+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.0 v2:172.21.15.78:6800/35735 6 ==== osd_op_reply(32 [pgnls start_epoch 88 out=49b] v86'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba0939000 2023-10-11T09:48:49.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.384+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:33 12.1 12:80000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba097bc00 con 0x559ba096b000 2023-10-11T09:48:49.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.384+0000 7f0bccaff700 1 -- 172.21.15.78:0/2409963846 <== osd.6 v2:172.21.15.133:6800/35900 3 ==== osd_op_reply(33 [pgnls start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359440 con 0x559ba096b000 2023-10-11T09:48:49.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.384+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:34 12.21 12:84000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba097b000 con 0x559ba0938c00 2023-10-11T09:48:49.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.384+0000 7f0bcdb01700 1 -- 172.21.15.78:0/2409963846 <== osd.1 v2:172.21.15.78:6809/35734 4 ==== osd_op_reply(34 [pgnls start_epoch 88 out=49b] v86'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359200 con 0x559ba0938c00 2023-10-11T09:48:49.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.384+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:35 12.11 12:88000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba096a400 con 0x559ba096b800 2023-10-11T09:48:49.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.388+0000 7f0bcdb01700 1 -- 172.21.15.78:0/2409963846 <== osd.5 v2:172.21.15.133:6816/35899 6 ==== osd_op_reply(35 [pgnls start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359200 con 0x559ba096b800 2023-10-11T09:48:49.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.388+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:36 12.31 12:8c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba097b400 con 0x559ba0939000 2023-10-11T09:48:49.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.388+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.0 v2:172.21.15.78:6800/35735 7 ==== osd_op_reply(36 [pgnls start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba0939000 2023-10-11T09:48:49.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.388+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:37 12.9 12:90000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba096bc00 con 0x559ba0939c00 2023-10-11T09:48:49.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.388+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.2 v2:172.21.15.78:6808/35732 6 ==== osd_op_reply(37 [pgnls start_epoch 88 out=49b] v86'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba0939c00 2023-10-11T09:48:49.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.388+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:38 12.29 12:94000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba097b800 con 0x559ba07e3000 2023-10-11T09:48:49.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.388+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.4 v2:172.21.15.133:6808/35898 5 ==== osd_op_reply(38 [pgnls start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba07e3000 2023-10-11T09:48:49.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.388+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:39 12.19 12:98000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba096ac00 con 0x559ba0939000 2023-10-11T09:48:49.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.388+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.0 v2:172.21.15.78:6800/35735 8 ==== osd_op_reply(39 [pgnls start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba0939000 2023-10-11T09:48:49.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.388+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:40 12.39 12:9c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba097a000 con 0x559ba096b000 2023-10-11T09:48:49.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.392+0000 7f0bccaff700 1 -- 172.21.15.78:0/2409963846 <== osd.6 v2:172.21.15.133:6800/35900 4 ==== osd_op_reply(40 [pgnls start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359440 con 0x559ba096b000 2023-10-11T09:48:49.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.392+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:41 12.5 12:a0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba097c000 con 0x559ba096b000 2023-10-11T09:48:49.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.392+0000 7f0bccaff700 1 -- 172.21.15.78:0/2409963846 <== osd.6 v2:172.21.15.133:6800/35900 5 ==== osd_op_reply(41 [pgnls start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359440 con 0x559ba096b000 2023-10-11T09:48:49.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.392+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:42 12.25 12:a4000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba0939400 con 0x559ba0938400 2023-10-11T09:48:49.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.392+0000 7f0bcdb01700 1 -- 172.21.15.78:0/2409963846 <== osd.7 v2:172.21.15.133:6824/35901 4 ==== osd_op_reply(42 [pgnls start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359200 con 0x559ba0938400 2023-10-11T09:48:49.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.392+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:43 12.15 12:a8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba097c400 con 0x559ba096b000 2023-10-11T09:48:49.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.396+0000 7f0bccaff700 1 -- 172.21.15.78:0/2409963846 <== osd.6 v2:172.21.15.133:6800/35900 6 ==== osd_op_reply(43 [pgnls start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359440 con 0x559ba096b000 2023-10-11T09:48:49.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.396+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:44 12.35 12:ac000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba096a800 con 0x559ba0939000 2023-10-11T09:48:49.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.396+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.0 v2:172.21.15.78:6800/35735 9 ==== osd_op_reply(44 [pgnls start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba0939000 2023-10-11T09:48:49.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.396+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:45 12.d 12:b0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba097c800 con 0x559ba0938400 2023-10-11T09:48:49.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.396+0000 7f0bcdb01700 1 -- 172.21.15.78:0/2409963846 <== osd.7 v2:172.21.15.133:6824/35901 5 ==== osd_op_reply(45 [pgnls start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359200 con 0x559ba0938400 2023-10-11T09:48:49.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.396+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:46 12.2d 12:b4000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba096b400 con 0x559ba0939800 2023-10-11T09:48:49.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.396+0000 7f0bccaff700 1 -- 172.21.15.78:0/2409963846 <== osd.3 v2:172.21.15.78:6824/35738 5 ==== osd_op_reply(46 [pgnls start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359440 con 0x559ba0939800 2023-10-11T09:48:49.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.396+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:47 12.1d 12:b8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba097cc00 con 0x559ba0939000 2023-10-11T09:48:49.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.400+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.0 v2:172.21.15.78:6800/35735 10 ==== osd_op_reply(47 [pgnls start_epoch 88 out=49b] v86'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba0939000 2023-10-11T09:48:49.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.400+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:48 12.3d 12:bc000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba07e3400 con 0x559ba0938c00 2023-10-11T09:48:49.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.400+0000 7f0bcdb01700 1 -- 172.21.15.78:0/2409963846 <== osd.1 v2:172.21.15.78:6809/35734 5 ==== osd_op_reply(48 [pgnls start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359200 con 0x559ba0938c00 2023-10-11T09:48:49.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.400+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:49 12.3 12:c0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba097d000 con 0x559ba0939000 2023-10-11T09:48:49.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.400+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.0 v2:172.21.15.78:6800/35735 11 ==== osd_op_reply(49 [pgnls start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba0939000 2023-10-11T09:48:49.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.400+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:50 12.23 12:c4000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba097c000 con 0x559ba0939000 2023-10-11T09:48:49.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.400+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.0 v2:172.21.15.78:6800/35735 12 ==== osd_op_reply(50 [pgnls start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba0939000 2023-10-11T09:48:49.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.400+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:51 12.13 12:c8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba097d400 con 0x559ba0938c00 2023-10-11T09:48:49.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.400+0000 7f0bcdb01700 1 -- 172.21.15.78:0/2409963846 <== osd.1 v2:172.21.15.78:6809/35734 6 ==== osd_op_reply(51 [pgnls start_epoch 88 out=49b] v86'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359200 con 0x559ba0938c00 2023-10-11T09:48:49.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.400+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:52 12.33 12:cc000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba0939400 con 0x559ba0939c00 2023-10-11T09:48:49.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.404+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.2 v2:172.21.15.78:6808/35732 7 ==== osd_op_reply(52 [pgnls start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba0939c00 2023-10-11T09:48:49.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.404+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:53 12.b 12:d0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba097d800 con 0x559ba0939800 2023-10-11T09:48:49.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.404+0000 7f0bccaff700 1 -- 172.21.15.78:0/2409963846 <== osd.3 v2:172.21.15.78:6824/35738 6 ==== osd_op_reply(53 [pgnls start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359440 con 0x559ba0939800 2023-10-11T09:48:49.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.404+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:54 12.2b 12:d4000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba0938800 con 0x559ba07e3000 2023-10-11T09:48:49.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.404+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.4 v2:172.21.15.133:6808/35898 6 ==== osd_op_reply(54 [pgnls start_epoch 88 out=49b] v86'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba07e3000 2023-10-11T09:48:49.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.404+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:55 12.1b 12:d8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba097dc00 con 0x559ba07e3000 2023-10-11T09:48:49.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.404+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.4 v2:172.21.15.133:6808/35898 7 ==== osd_op_reply(55 [pgnls start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba07e3000 2023-10-11T09:48:49.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.404+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:56 12.3b 12:dc000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba096bc00 con 0x559ba0939c00 2023-10-11T09:48:49.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.408+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.2 v2:172.21.15.78:6808/35732 8 ==== osd_op_reply(56 [pgnls start_epoch 88 out=49b] v86'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba0939c00 2023-10-11T09:48:49.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.408+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:57 12.7 12:e0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba096bc00 con 0x559ba0938c00 2023-10-11T09:48:49.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.408+0000 7f0bcdb01700 1 -- 172.21.15.78:0/2409963846 <== osd.1 v2:172.21.15.78:6809/35734 7 ==== osd_op_reply(57 [pgnls start_epoch 88 out=81b] v86'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x559ba1359200 con 0x559ba0938c00 2023-10-11T09:48:49.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.408+0000 7f0bceb44840 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:58 12.7 12:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba07e3800 con 0x559ba0938c00 2023-10-11T09:48:49.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.408+0000 7f0bcdb01700 1 -- 172.21.15.78:0/2409963846 <== osd.1 v2:172.21.15.78:6809/35734 8 ==== osd_op_reply(58 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x559ba1359200 con 0x559ba0938c00 2023-10-11T09:48:49.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.408+0000 7f0bceb44840 10 datascan.operator(): handling object 1099511627777.0 2023-10-11T09:48:49.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.408+0000 7f0bceb44840 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:59 12.7 12:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba0980000 con 0x559ba0938c00 2023-10-11T09:48:49.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.412+0000 7f0bcdb01700 1 -- 172.21.15.78:0/2409963846 <== osd.1 v2:172.21.15.78:6809/35734 9 ==== osd_op_reply(59 10000000001.00000000 [call] v88'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x559ba1359200 con 0x559ba0938c00 2023-10-11T09:48:49.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.412+0000 7f0bceb44840 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:60 12.27 12:e4000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba0980400 con 0x559ba0939000 2023-10-11T09:48:49.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.412+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.0 v2:172.21.15.78:6800/35735 13 ==== osd_op_reply(60 [pgnls start_epoch 88 out=49b] v86'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba0939000 2023-10-11T09:48:49.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.412+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:61 12.17 12:e8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba097cc00 con 0x559ba096b000 2023-10-11T09:48:49.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.412+0000 7f0bccaff700 1 -- 172.21.15.78:0/2409963846 <== osd.6 v2:172.21.15.133:6800/35900 7 ==== osd_op_reply(61 [pgnls start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359440 con 0x559ba096b000 2023-10-11T09:48:49.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.412+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:62 12.37 12:ec000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba096ac00 con 0x559ba07e3000 2023-10-11T09:48:49.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.416+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.4 v2:172.21.15.133:6808/35898 8 ==== osd_op_reply(62 [pgnls start_epoch 88 out=49b] v86'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba07e3000 2023-10-11T09:48:49.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.416+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:63 12.f 12:f0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba097d000 con 0x559ba096b000 2023-10-11T09:48:49.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.416+0000 7f0bccaff700 1 -- 172.21.15.78:0/2409963846 <== osd.6 v2:172.21.15.133:6800/35900 8 ==== osd_op_reply(63 [pgnls start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359440 con 0x559ba096b000 2023-10-11T09:48:49.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.416+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:64 12.2f 12:f4000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba096a800 con 0x559ba0939c00 2023-10-11T09:48:49.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.416+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.2 v2:172.21.15.78:6808/35732 9 ==== osd_op_reply(64 [pgnls start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba0939c00 2023-10-11T09:48:49.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.416+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:65 12.1f 12:f8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba097c000 con 0x559ba096b000 2023-10-11T09:48:49.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.416+0000 7f0bccaff700 1 -- 172.21.15.78:0/2409963846 <== osd.6 v2:172.21.15.133:6800/35900 9 ==== osd_op_reply(65 [pgnls start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359440 con 0x559ba096b000 2023-10-11T09:48:49.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.416+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:66 12.3f 12:fc000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba096a400 con 0x559ba0938400 2023-10-11T09:48:49.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.420+0000 7f0bcdb01700 1 -- 172.21.15.78:0/2409963846 <== osd.7 v2:172.21.15.133:6824/35901 6 ==== osd_op_reply(66 [pgnls start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359200 con 0x559ba0938400 2023-10-11T09:48:49.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.420+0000 7f0bceb44840 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:67 13.0 13:00000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba0980800 con 0x559ba0938400 2023-10-11T09:48:49.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.420+0000 7f0bcdb01700 1 -- 172.21.15.78:0/2409963846 <== osd.7 v2:172.21.15.133:6824/35901 7 ==== osd_op_reply(67 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359200 con 0x559ba0938400 2023-10-11T09:48:49.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.420+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:68 13.10 13:08000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba096a000 con 0x559ba0938c00 2023-10-11T09:48:49.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.420+0000 7f0bcdb01700 1 -- 172.21.15.78:0/2409963846 <== osd.1 v2:172.21.15.78:6809/35734 10 ==== osd_op_reply(68 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359200 con 0x559ba0938c00 2023-10-11T09:48:49.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.420+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:69 13.8 13:10000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba097dc00 con 0x559ba0939c00 2023-10-11T09:48:49.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.424+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.2 v2:172.21.15.78:6808/35732 10 ==== osd_op_reply(69 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba0939c00 2023-10-11T09:48:49.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.424+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:70 13.18 13:18000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba07e3400 con 0x559ba0938400 2023-10-11T09:48:49.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.424+0000 7f0bcdb01700 1 -- 172.21.15.78:0/2409963846 <== osd.7 v2:172.21.15.133:6824/35901 8 ==== osd_op_reply(70 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359200 con 0x559ba0938400 2023-10-11T09:48:49.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.424+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:71 13.4 13:20000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba0939400 con 0x559ba096b000 2023-10-11T09:48:49.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.424+0000 7f0bccaff700 1 -- 172.21.15.78:0/2409963846 <== osd.6 v2:172.21.15.133:6800/35900 10 ==== osd_op_reply(71 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359440 con 0x559ba096b000 2023-10-11T09:48:49.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.424+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:72 13.14 13:28000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba096b400 con 0x559ba0939000 2023-10-11T09:48:49.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.424+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.0 v2:172.21.15.78:6800/35735 14 ==== osd_op_reply(72 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba0939000 2023-10-11T09:48:49.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.424+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:73 13.c 13:30000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba0938800 con 0x559ba0938c00 2023-10-11T09:48:49.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.428+0000 7f0bcdb01700 1 -- 172.21.15.78:0/2409963846 <== osd.1 v2:172.21.15.78:6809/35734 11 ==== osd_op_reply(73 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359200 con 0x559ba0938c00 2023-10-11T09:48:49.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.428+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:74 13.1c 13:38000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba096bc00 con 0x559ba096b000 2023-10-11T09:48:49.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.428+0000 7f0bccaff700 1 -- 172.21.15.78:0/2409963846 <== osd.6 v2:172.21.15.133:6800/35900 11 ==== osd_op_reply(74 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359440 con 0x559ba096b000 2023-10-11T09:48:49.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.428+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:75 13.2 13:40000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba0980000 con 0x559ba096b800 2023-10-11T09:48:49.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.428+0000 7f0bcdb01700 1 -- 172.21.15.78:0/2409963846 <== osd.5 v2:172.21.15.133:6816/35899 7 ==== osd_op_reply(75 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359200 con 0x559ba096b800 2023-10-11T09:48:49.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.428+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:76 13.12 13:48000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba097c400 con 0x559ba0939000 2023-10-11T09:48:49.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.428+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.0 v2:172.21.15.78:6800/35735 15 ==== osd_op_reply(76 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba0939000 2023-10-11T09:48:49.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.428+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:77 13.a 13:50000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba0981c00 con 0x559ba0939c00 2023-10-11T09:48:49.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.428+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.2 v2:172.21.15.78:6808/35732 11 ==== osd_op_reply(77 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba0939c00 2023-10-11T09:48:49.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.428+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:78 13.1a 13:58000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba097d800 con 0x559ba0938c00 2023-10-11T09:48:49.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.432+0000 7f0bcdb01700 1 -- 172.21.15.78:0/2409963846 <== osd.1 v2:172.21.15.78:6809/35734 12 ==== osd_op_reply(78 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359200 con 0x559ba0938c00 2023-10-11T09:48:49.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.432+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:79 13.6 13:60000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba07e3800 con 0x559ba07e3000 2023-10-11T09:48:49.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.432+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.4 v2:172.21.15.133:6808/35898 9 ==== osd_op_reply(79 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba07e3000 2023-10-11T09:48:49.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.432+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:80 13.16 13:68000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba097cc00 con 0x559ba0939800 2023-10-11T09:48:49.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.432+0000 7f0bccaff700 1 -- 172.21.15.78:0/2409963846 <== osd.3 v2:172.21.15.78:6824/35738 7 ==== osd_op_reply(80 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359440 con 0x559ba0939800 2023-10-11T09:48:49.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.432+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:81 13.e 13:70000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba0980400 con 0x559ba0938400 2023-10-11T09:48:49.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.436+0000 7f0bcdb01700 1 -- 172.21.15.78:0/2409963846 <== osd.7 v2:172.21.15.133:6824/35901 9 ==== osd_op_reply(81 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359200 con 0x559ba0938400 2023-10-11T09:48:49.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.436+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:82 13.1e 13:78000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba097d000 con 0x559ba07e3000 2023-10-11T09:48:49.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.436+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.4 v2:172.21.15.133:6808/35898 10 ==== osd_op_reply(82 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba07e3000 2023-10-11T09:48:49.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.436+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:83 13.1 13:80000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba0981c00 con 0x559ba096b000 2023-10-11T09:48:49.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.436+0000 7f0bccaff700 1 -- 172.21.15.78:0/2409963846 <== osd.6 v2:172.21.15.133:6800/35900 12 ==== osd_op_reply(83 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359440 con 0x559ba096b000 2023-10-11T09:48:49.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.436+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:84 13.11 13:88000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba097c000 con 0x559ba0938c00 2023-10-11T09:48:49.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.436+0000 7f0bcdb01700 1 -- 172.21.15.78:0/2409963846 <== osd.1 v2:172.21.15.78:6809/35734 13 ==== osd_op_reply(84 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359200 con 0x559ba0938c00 2023-10-11T09:48:49.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.436+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:85 13.9 13:90000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba0980800 con 0x559ba096b800 2023-10-11T09:48:49.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.440+0000 7f0bcdb01700 1 -- 172.21.15.78:0/2409963846 <== osd.5 v2:172.21.15.133:6816/35899 8 ==== osd_op_reply(85 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359200 con 0x559ba096b800 2023-10-11T09:48:49.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.440+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:86 13.19 13:98000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba097c800 con 0x559ba0939800 2023-10-11T09:48:49.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.440+0000 7f0bccaff700 1 -- 172.21.15.78:0/2409963846 <== osd.3 v2:172.21.15.78:6824/35738 8 ==== osd_op_reply(86 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359440 con 0x559ba0939800 2023-10-11T09:48:49.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.440+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:87 13.5 13:a0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba0938800 con 0x559ba07e3000 2023-10-11T09:48:49.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.440+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.4 v2:172.21.15.133:6808/35898 11 ==== osd_op_reply(87 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba07e3000 2023-10-11T09:48:49.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.440+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:88 13.15 13:a8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba097d400 con 0x559ba07e3000 2023-10-11T09:48:49.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.440+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.4 v2:172.21.15.133:6808/35898 12 ==== osd_op_reply(88 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba07e3000 2023-10-11T09:48:49.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.440+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:89 13.d 13:b0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba07e3800 con 0x559ba07e3000 2023-10-11T09:48:49.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.444+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.4 v2:172.21.15.133:6808/35898 13 ==== osd_op_reply(89 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba07e3000 2023-10-11T09:48:49.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.444+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:90 13.1d 13:b8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba097dc00 con 0x559ba0938400 2023-10-11T09:48:49.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.444+0000 7f0bcdb01700 1 -- 172.21.15.78:0/2409963846 <== osd.7 v2:172.21.15.133:6824/35901 10 ==== osd_op_reply(90 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359200 con 0x559ba0938400 2023-10-11T09:48:49.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.444+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:91 13.3 13:c0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba07e3400 con 0x559ba0939000 2023-10-11T09:48:49.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.444+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.0 v2:172.21.15.78:6800/35735 16 ==== osd_op_reply(91 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba0939000 2023-10-11T09:48:49.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.444+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:92 13.13 13:c8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba097c400 con 0x559ba0938400 2023-10-11T09:48:49.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.444+0000 7f0bcdb01700 1 -- 172.21.15.78:0/2409963846 <== osd.7 v2:172.21.15.133:6824/35901 11 ==== osd_op_reply(92 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359200 con 0x559ba0938400 2023-10-11T09:48:49.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.444+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:93 13.b 13:d0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba096ac00 con 0x559ba096b800 2023-10-11T09:48:49.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.448+0000 7f0bcdb01700 1 -- 172.21.15.78:0/2409963846 <== osd.5 v2:172.21.15.133:6816/35899 9 ==== osd_op_reply(93 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359200 con 0x559ba096b800 2023-10-11T09:48:49.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.448+0000 7f0bc82f6700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:94 13.1b 13:d8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba097d800 con 0x559ba07e3000 2023-10-11T09:48:49.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.448+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 <== osd.4 v2:172.21.15.133:6808/35898 14 ==== osd_op_reply(94 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1358fc0 con 0x559ba07e3000 2023-10-11T09:48:49.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.448+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:95 13.7 13:e0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba096a800 con 0x559ba0938400 2023-10-11T09:48:49.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.448+0000 7f0bcdb01700 1 -- 172.21.15.78:0/2409963846 <== osd.7 v2:172.21.15.133:6824/35901 12 ==== osd_op_reply(95 [pgnls start_epoch 88 out=81b] v86'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x559ba1359200 con 0x559ba0938400 2023-10-11T09:48:49.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.448+0000 7f0bceb44840 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:96 13.7 13:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba0980c00 con 0x559ba0938400 2023-10-11T09:48:49.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.448+0000 7f0bcdb01700 1 -- 172.21.15.78:0/2409963846 <== osd.7 v2:172.21.15.133:6824/35901 13 ==== osd_op_reply(96 10000000001.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x559ba1359200 con 0x559ba0938400 2023-10-11T09:48:49.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.448+0000 7f0bceb44840 10 datascan.operator(): handling object 1099511627777.0 2023-10-11T09:48:49.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.448+0000 7f0bceb44840 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:97 12.7 12:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba0981000 con 0x559ba0938c00 2023-10-11T09:48:49.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.452+0000 7f0bcdb01700 1 -- 172.21.15.78:0/2409963846 <== osd.1 v2:172.21.15.78:6809/35734 14 ==== osd_op_reply(97 10000000001.00000000 [call,setxattr (8)] v88'8 uv8 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x559ba1359200 con 0x559ba0938c00 2023-10-11T09:48:49.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.452+0000 7f0bceb44840 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:98 13.17 13:e8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba0981400 con 0x559ba096b800 2023-10-11T09:48:49.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.452+0000 7f0bcdb01700 1 -- 172.21.15.78:0/2409963846 <== osd.5 v2:172.21.15.133:6816/35899 10 ==== osd_op_reply(98 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359200 con 0x559ba096b800 2023-10-11T09:48:49.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.452+0000 7f0bc8af7700 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:99 13.f 13:f0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba096b400 con 0x559ba096b000 2023-10-11T09:48:49.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.456+0000 7f0bccaff700 1 -- 172.21.15.78:0/2409963846 <== osd.6 v2:172.21.15.133:6800/35900 13 ==== osd_op_reply(99 [pgnls start_epoch 88 out=81b] v86'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x559ba1359440 con 0x559ba096b000 2023-10-11T09:48:49.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.456+0000 7f0bceb44840 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:100 13.f 13:f526278d:::10000000001.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba0981800 con 0x559ba096b000 2023-10-11T09:48:49.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.456+0000 7f0bccaff700 1 -- 172.21.15.78:0/2409963846 <== osd.6 v2:172.21.15.133:6800/35900 14 ==== osd_op_reply(100 10000000001.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x559ba1359440 con 0x559ba096b000 2023-10-11T09:48:49.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.456+0000 7f0bceb44840 10 datascan.operator(): handling object 1099511627777.1 2023-10-11T09:48:49.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.456+0000 7f0bceb44840 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:101 12.7 12:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba096bc00 con 0x559ba0938c00 2023-10-11T09:48:49.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bcdb01700 1 -- 172.21.15.78:0/2409963846 <== osd.1 v2:172.21.15.78:6809/35734 15 ==== osd_op_reply(101 10000000001.00000000 [call,setxattr (8)] v88'9 uv9 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x559ba1359200 con 0x559ba0938c00 2023-10-11T09:48:49.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bceb44840 1 -- 172.21.15.78:0/2409963846 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:102 13.1f 13:f8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x559ba096b400 con 0x559ba096b000 2023-10-11T09:48:49.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bccaff700 1 -- 172.21.15.78:0/2409963846 <== osd.6 v2:172.21.15.133:6800/35900 15 ==== osd_op_reply(102 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559ba1359440 con 0x559ba096b000 2023-10-11T09:48:49.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bceb44840 1 -- 172.21.15.78:0/2409963846 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x559ba0939000 msgr2=0x559ba094e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bceb44840 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x559ba0939000 0x559ba094e580 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bceb44840 1 -- 172.21.15.78:0/2409963846 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x559ba0938c00 msgr2=0x559ba094f080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bceb44840 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x559ba0938c00 0x559ba094f080 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bceb44840 1 -- 172.21.15.78:0/2409963846 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x559ba0939c00 msgr2=0x559ba094f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bceb44840 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x559ba0939c00 0x559ba094f600 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bceb44840 1 -- 172.21.15.78:0/2409963846 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x559ba0939800 msgr2=0x559ba094eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bceb44840 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x559ba0939800 0x559ba094eb00 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bceb44840 1 -- 172.21.15.78:0/2409963846 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x559ba07e3000 msgr2=0x559ba058c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bceb44840 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x559ba07e3000 0x559ba058c100 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bceb44840 1 -- 172.21.15.78:0/2409963846 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x559ba096b800 msgr2=0x559ba0950680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bceb44840 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x559ba096b800 0x559ba0950680 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bceb44840 1 -- 172.21.15.78:0/2409963846 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x559ba096b000 msgr2=0x559ba0950100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bceb44840 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x559ba096b000 0x559ba0950100 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bceb44840 1 -- 172.21.15.78:0/2409963846 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x559ba0938400 msgr2=0x559ba094e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bceb44840 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x559ba0938400 0x559ba094e000 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 reap_dead start 2023-10-11T09:48:49.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 reap_dead start 2023-10-11T09:48:49.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 reap_dead start 2023-10-11T09:48:49.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bcd300700 1 -- 172.21.15.78:0/2409963846 reap_dead start 2023-10-11T09:48:49.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bceb44840 1 -- 172.21.15.78:0/2409963846 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x559ba0938000 msgr2=0x559ba058d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bceb44840 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x559ba0938000 0x559ba058d700 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x559ba07d4840 tx=0x559ba12cad20 comp rx=0 tx=0).stop 2023-10-11T09:48:49.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bceb44840 1 -- 172.21.15.78:0/2409963846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ba07e2c00 msgr2=0x559ba058c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bceb44840 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ba07e2c00 0x559ba058c680 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x559ba07d46f0 tx=0x559ba12caba0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bceb44840 1 -- 172.21.15.78:0/2409963846 shutdown_connections 2023-10-11T09:48:49.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bceb44840 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x559ba0938000 0x559ba058d700 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bceb44840 1 --2- 172.21.15.78:0/2409963846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ba07e2c00 0x559ba058c680 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bceb44840 1 -- 172.21.15.78:0/2409963846 >> 172.21.15.78:0/2409963846 conn(0x559ba04e4c00 msgr2=0x559ba05b3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:49.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bceb44840 1 -- 172.21.15.78:0/2409963846 shutdown_connections 2023-10-11T09:48:49.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bceb44840 1 -- 172.21.15.78:0/2409963846 wait complete. 2023-10-11T09:48:49.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bceb44840 1 -- 172.21.15.78:0/1975239613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ba04e5400 msgr2=0x559ba058ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bceb44840 1 --2- 172.21.15.78:0/1975239613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ba04e5400 0x559ba058ab00 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x559ba124d3e0 tx=0x559ba12cad80 comp rx=0 tx=0).stop 2023-10-11T09:48:49.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bceb44840 1 -- 172.21.15.78:0/1975239613 shutdown_connections 2023-10-11T09:48:49.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bceb44840 1 --2- 172.21.15.78:0/1975239613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559ba04e5000 0x559ba058b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bceb44840 1 --2- 172.21.15.78:0/1975239613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ba04e5400 0x559ba058ab00 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bceb44840 1 --2- 172.21.15.78:0/1975239613 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559ba04e5800 0x559ba058a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bceb44840 1 -- 172.21.15.78:0/1975239613 >> 172.21.15.78:0/1975239613 conn(0x559ba04e5c00 msgr2=0x559ba05b3000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:49.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bceb44840 1 -- 172.21.15.78:0/1975239613 shutdown_connections 2023-10-11T09:48:49.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.460+0000 7f0bceb44840 1 -- 172.21.15.78:0/1975239613 wait complete. 2023-10-11T09:48:49.475 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents'] in time 0:00:00.212041, result: 2023-10-11T09:48:49.475 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes 2023-10-11T09:48:49.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.532+0000 7fe33f2ec840 1 -- start start 2023-10-11T09:48:49.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.532+0000 7fe33f2ec840 1 Processor -- start 2023-10-11T09:48:49.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.536+0000 7fe33f2ec840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d388bf800 0x562d38964580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.536+0000 7fe33f2ec840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562d388bf400 0x562d38964b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.536+0000 7fe33f2ec840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d388bf000 0x562d38965080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.536+0000 7fe33f2ec840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d388f5e00 con 0x562d388bf800 2023-10-11T09:48:49.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.536+0000 7fe33f2ec840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d3893ea00 con 0x562d388bf000 2023-10-11T09:48:49.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.536+0000 7fe33f2ec840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d3893e640 con 0x562d388bf400 2023-10-11T09:48:49.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.536+0000 7fe33daa8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d388bf800 0x562d38964580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.536+0000 7fe33daa8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d388bf800 0x562d38964580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35764/0 (socket says 172.21.15.78:35764) 2023-10-11T09:48:49.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.536+0000 7fe33daa8700 1 -- 172.21.15.78:0/1492595867 learned_addr learned my addr 172.21.15.78:0/1492595867 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:49.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.536+0000 7fe33d2a7700 1 --2- 172.21.15.78:0/1492595867 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562d388bf400 0x562d38964b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.536+0000 7fe33e2a9700 1 --2- 172.21.15.78:0/1492595867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d388bf000 0x562d38965080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.536+0000 7fe33daa8700 1 -- 172.21.15.78:0/1492595867 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562d388bf400 msgr2=0x562d38964b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:49.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.536+0000 7fe33daa8700 1 --2- 172.21.15.78:0/1492595867 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562d388bf400 0x562d38964b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.536+0000 7fe33daa8700 1 -- 172.21.15.78:0/1492595867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d388bf000 msgr2=0x562d38965080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:49.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.536+0000 7fe33daa8700 1 --2- 172.21.15.78:0/1492595867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d388bf000 0x562d38965080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.536+0000 7fe33daa8700 1 -- 172.21.15.78:0/1492595867 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d388e0b60 con 0x562d388bf800 2023-10-11T09:48:49.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.536+0000 7fe33daa8700 1 --2- 172.21.15.78:0/1492595867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d388bf800 0x562d38964580 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x562d3898d3e0 tx=0x562d396a6d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1cc52b6d3670600 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.536+0000 7fe33b2a3700 1 -- 172.21.15.78:0/1492595867 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562d388ef600 con 0x562d388bf800 2023-10-11T09:48:49.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.536+0000 7fe33b2a3700 1 -- 172.21.15.78:0/1492595867 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562d3890a180 con 0x562d388bf800 2023-10-11T09:48:49.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.536+0000 7fe33f2ec840 1 -- 172.21.15.78:0/1492595867 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d388e0340 con 0x562d388bf800 2023-10-11T09:48:49.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.536+0000 7fe33f2ec840 1 -- 172.21.15.78:0/1492595867 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x562d388e0000 con 0x562d388bf800 2023-10-11T09:48:49.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.536+0000 7fe33f2ec840 4 waiting for MDS map... 2023-10-11T09:48:49.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.536+0000 7fe33b2a3700 1 -- 172.21.15.78:0/1492595867 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562d388ef1e0 con 0x562d388bf800 2023-10-11T09:48:49.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.536+0000 7fe33b2a3700 1 -- 172.21.15.78:0/1492595867 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(88..88 src has 1..88) v4 ==== 6638+0+0 (secure 0 0 0) 0x562d388f8c40 con 0x562d388bf800 2023-10-11T09:48:49.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.536+0000 7fe33b2a3700 1 -- 172.21.15.78:0/1492595867 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 64) v1 ==== 3118+0+0 (secure 0 0 0) 0x562d39744300 con 0x562d388bf800 2023-10-11T09:48:49.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.536+0000 7fe33f2ec840 4 Got MDS map 64 2023-10-11T09:48:49.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.536+0000 7fe33f2ec840 4 datascan.main: Using metadata pool output 2023-10-11T09:48:49.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.536+0000 7fe33f2ec840 4 datascan.main: connecting to RADOS... 2023-10-11T09:48:49.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.536+0000 7fe33f2ec840 1 Processor -- start 2023-10-11T09:48:49.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.536+0000 7fe33f2ec840 1 -- start start 2023-10-11T09:48:49.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.540+0000 7fe33f2ec840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d39748400 0x562d38965600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.540+0000 7fe33f2ec840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d39748800 0x562d38965b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.540+0000 7fe33f2ec840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562d39748c00 0x562d38966100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.540+0000 7fe33f2ec840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d3893e500 con 0x562d39748800 2023-10-11T09:48:49.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.540+0000 7fe33f2ec840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d3893e3c0 con 0x562d39748400 2023-10-11T09:48:49.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.540+0000 7fe33f2ec840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d3893ea00 con 0x562d39748c00 2023-10-11T09:48:49.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.540+0000 7fe33daa8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d39748800 0x562d38965b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.540+0000 7fe33daa8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d39748800 0x562d38965b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35768/0 (socket says 172.21.15.78:35768) 2023-10-11T09:48:49.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.540+0000 7fe33daa8700 1 -- 172.21.15.78:0/3528107322 learned_addr learned my addr 172.21.15.78:0/3528107322 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:49.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.540+0000 7fe33e2a9700 1 --2- 172.21.15.78:0/3528107322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d39748400 0x562d38965600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.540+0000 7fe33daa8700 1 -- 172.21.15.78:0/3528107322 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562d39748c00 msgr2=0x562d38966100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:49.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.540+0000 7fe33daa8700 1 --2- 172.21.15.78:0/3528107322 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562d39748c00 0x562d38966100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.540+0000 7fe33daa8700 1 -- 172.21.15.78:0/3528107322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d39748400 msgr2=0x562d38965600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:49.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.540+0000 7fe33daa8700 1 --2- 172.21.15.78:0/3528107322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d39748400 0x562d38965600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.540+0000 7fe33daa8700 1 -- 172.21.15.78:0/3528107322 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d388e0ea0 con 0x562d39748800 2023-10-11T09:48:49.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.540+0000 7fe33e2a9700 1 --2- 172.21.15.78:0/3528107322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d39748400 0x562d38965600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:48:49.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.540+0000 7fe33daa8700 1 --2- 172.21.15.78:0/3528107322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d39748800 0x562d38965b80 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x562d397406f0 tx=0x562d396a6ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb1803445ea0d2ff server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.540+0000 7fe33929f700 1 -- 172.21.15.78:0/3528107322 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562d388eef20 con 0x562d39748800 2023-10-11T09:48:49.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.540+0000 7fe33929f700 1 -- 172.21.15.78:0/3528107322 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562d396a4d80 con 0x562d39748800 2023-10-11T09:48:49.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.540+0000 7fe33929f700 1 -- 172.21.15.78:0/3528107322 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562d388ef600 con 0x562d39748800 2023-10-11T09:48:49.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.540+0000 7fe33f2ec840 1 -- 172.21.15.78:0/3528107322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d39748800 msgr2=0x562d38965b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.540+0000 7fe33f2ec840 1 --2- 172.21.15.78:0/3528107322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d39748800 0x562d38965b80 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x562d397406f0 tx=0x562d396a6ba0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.540+0000 7fe33f2ec840 1 -- 172.21.15.78:0/3528107322 shutdown_connections 2023-10-11T09:48:49.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.540+0000 7fe33f2ec840 1 --2- 172.21.15.78:0/3528107322 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562d39748c00 0x562d38966100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.540+0000 7fe33f2ec840 1 --2- 172.21.15.78:0/3528107322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d39748800 0x562d38965b80 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.540+0000 7fe33f2ec840 1 --2- 172.21.15.78:0/3528107322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d39748400 0x562d38965600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.540+0000 7fe33f2ec840 1 -- 172.21.15.78:0/3528107322 >> 172.21.15.78:0/3528107322 conn(0x562d388bec00 msgr2=0x562d389af800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:49.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.540+0000 7fe33f2ec840 1 -- 172.21.15.78:0/3528107322 shutdown_connections 2023-10-11T09:48:49.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.540+0000 7fe33f2ec840 1 -- 172.21.15.78:0/3528107322 wait complete. 2023-10-11T09:48:49.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.540+0000 7fe33f2ec840 1 Processor -- start 2023-10-11T09:48:49.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.544+0000 7fe33f2ec840 1 -- start start 2023-10-11T09:48:49.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.544+0000 7fe33f2ec840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d39748400 0x562d38967180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.544+0000 7fe33f2ec840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562d39748800 0x562d38966c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.544+0000 7fe33f2ec840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d39748c00 0x562d38966680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.544+0000 7fe33f2ec840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d3893e280 con 0x562d39748400 2023-10-11T09:48:49.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.544+0000 7fe33f2ec840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d3893e140 con 0x562d39748c00 2023-10-11T09:48:49.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.544+0000 7fe33f2ec840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d3893e000 con 0x562d39748800 2023-10-11T09:48:49.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.544+0000 7fe33e2a9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d39748400 0x562d38967180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.544+0000 7fe33e2a9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d39748400 0x562d38967180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35774/0 (socket says 172.21.15.78:35774) 2023-10-11T09:48:49.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.544+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 learned_addr learned my addr 172.21.15.78:0/3149944686 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:49.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.544+0000 7fe33daa8700 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562d39748800 0x562d38966c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.544+0000 7fe33d2a7700 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d39748c00 0x562d38966680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.544+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562d39748800 msgr2=0x562d38966c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.544+0000 7fe33e2a9700 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562d39748800 0x562d38966c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.544+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d39748c00 msgr2=0x562d38966680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.544+0000 7fe33e2a9700 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d39748c00 0x562d38966680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.544+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d388e0b60 con 0x562d39748400 2023-10-11T09:48:49.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.544+0000 7fe33daa8700 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562d39748800 0x562d38966c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:48:49.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.544+0000 7fe33d2a7700 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d39748c00 0x562d38966680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:48:49.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.544+0000 7fe33e2a9700 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d39748400 0x562d38967180 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x562d397413b0 tx=0x562d38bb8e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.544+0000 7fe337a9c700 1 -- 172.21.15.78:0/3149944686 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562d388eef20 con 0x562d39748400 2023-10-11T09:48:49.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.544+0000 7fe337a9c700 1 -- 172.21.15.78:0/3149944686 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562d3890a480 con 0x562d39748400 2023-10-11T09:48:49.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.544+0000 7fe33f2ec840 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562d388e0340 con 0x562d39748400 2023-10-11T09:48:49.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.544+0000 7fe337a9c700 1 -- 172.21.15.78:0/3149944686 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562d388ef600 con 0x562d39748400 2023-10-11T09:48:49.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.544+0000 7fe33f2ec840 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d388e1520 con 0x562d39748400 2023-10-11T09:48:49.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.544+0000 7fe33f2ec840 4 datascan.init: resolving metadata pool 11 2023-10-11T09:48:49.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.548+0000 7fe337a9c700 1 -- 172.21.15.78:0/3149944686 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x562d3891c840 con 0x562d39748400 2023-10-11T09:48:49.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.548+0000 7fe337a9c700 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x562d38d0a000 0x562d38967700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.548+0000 7fe337a9c700 1 -- 172.21.15.78:0/3149944686 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6638+0+0 (secure 0 0 0) 0x562d388f8a80 con 0x562d39748400 2023-10-11T09:48:49.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.548+0000 7fe33f2ec840 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-10-11T09:48:49.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.548+0000 7fe33f2ec840 4 datascan.main: data pool 'cephfs_data' has ID 12 2023-10-11T09:48:49.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.548+0000 7fe33f2ec840 4 datascan.main: opening data pool 'cephfs_data' 2023-10-11T09:48:49.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.548+0000 7fe33f2ec840 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x562d39749000 0x562d38966100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.548+0000 7fe33f2ec840 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:1 11.14 11:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d39749400 con 0x562d39749000 2023-10-11T09:48:49.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.548+0000 7fe33daa8700 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x562d38d0a000 0x562d38967700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.548+0000 7fe33d2a7700 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x562d39749000 0x562d38966100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.548+0000 7fe33daa8700 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x562d38d0a000 0x562d38967700 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x562d397406f0 tx=0x562d38bb8c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.548+0000 7fe33d2a7700 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x562d39749000 0x562d38966100 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.552+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv29 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x562d3962cfc0 con 0x562d39749000 2023-10-11T09:48:49.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.552+0000 7fe33f2ec840 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x562d39749800 0x562d38965b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.552+0000 7fe33f2ec840 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:2 11.1f 11:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d0a400 con 0x562d39749800 2023-10-11T09:48:49.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.552+0000 7fe33e2a9700 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x562d39749800 0x562d38965b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.552+0000 7fe33e2a9700 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x562d39749800 0x562d38965b80 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.552+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.2 v2:172.21.15.78:6808/35732 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x562d3962d200 con 0x562d39749800 2023-10-11T09:48:49.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.552+0000 7fe33f2ec840 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x562d38d0a800 0x562d38965600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.552+0000 7fe33f2ec840 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:3 12.0 12:00000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d0ac00 con 0x562d38d0a800 2023-10-11T09:48:49.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.552+0000 7fe33daa8700 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x562d38d0a800 0x562d38965600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.556+0000 7fe33daa8700 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x562d38d0a800 0x562d38965600 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.556+0000 7fe33daa8700 1 -- 172.21.15.78:0/3149944686 <== osd.4 v2:172.21.15.133:6808/35898 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d440 con 0x562d38d0a800 2023-10-11T09:48:49.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.556+0000 7fe338a9e700 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x562d38d38000 0x562d38d3e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.556+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:4 12.20 12:04000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d38400 con 0x562d38d38000 2023-10-11T09:48:49.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.556+0000 7fe33d2a7700 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x562d38d38000 0x562d38d3e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.556+0000 7fe33d2a7700 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x562d38d38000 0x562d38d3e000 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.560+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.7 v2:172.21.15.133:6824/35901 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962cfc0 con 0x562d38d38000 2023-10-11T09:48:49.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.560+0000 7fe33929f700 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x562d39749400 0x562d38d3e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.560+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:5 12.10 12:08000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d38400 con 0x562d39749400 2023-10-11T09:48:49.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.560+0000 7fe33e2a9700 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x562d39749400 0x562d38d3e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.560+0000 7fe33e2a9700 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x562d39749400 0x562d38d3e580 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.560+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.0 v2:172.21.15.78:6800/35735 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749400 2023-10-11T09:48:49.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.560+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:6 12.30 12:0c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d38800 con 0x562d38d38000 2023-10-11T09:48:49.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.560+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.7 v2:172.21.15.133:6824/35901 2 ==== osd_op_reply(6 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962cfc0 con 0x562d38d38000 2023-10-11T09:48:49.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.560+0000 7fe33929f700 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x562d38d38c00 0x562d38d3eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.560+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:7 12.8 12:10000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d39000 con 0x562d38d38c00 2023-10-11T09:48:49.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.564+0000 7fe33daa8700 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x562d38d38c00 0x562d38d3eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.564+0000 7fe33daa8700 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x562d38d38c00 0x562d38d3eb00 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.564+0000 7fe33daa8700 1 -- 172.21.15.78:0/3149944686 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(7 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d440 con 0x562d38d38c00 2023-10-11T09:48:49.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.564+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:8 12.28 12:14000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d39c00 con 0x562d39749400 2023-10-11T09:48:49.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.564+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.0 v2:172.21.15.78:6800/35735 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749400 2023-10-11T09:48:49.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.564+0000 7fe33929f700 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x562d38d39400 0x562d38d3f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.564+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:9 12.18 12:18000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d39800 con 0x562d38d39400 2023-10-11T09:48:49.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.564+0000 7fe33d2a7700 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x562d38d39400 0x562d38d3f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.564+0000 7fe33d2a7700 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x562d38d39400 0x562d38d3f080 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.568+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.1 v2:172.21.15.78:6809/35734 1 ==== osd_op_reply(9 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962cfc0 con 0x562d38d39400 2023-10-11T09:48:49.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.568+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:10 12.38 12:1c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d38400 con 0x562d39749800 2023-10-11T09:48:49.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.568+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.2 v2:172.21.15.78:6808/35732 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749800 2023-10-11T09:48:49.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.568+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:11 12.4 12:20000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4e000 con 0x562d39749000 2023-10-11T09:48:49.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.568+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.6 v2:172.21.15.133:6800/35900 2 ==== osd_op_reply(11 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962cfc0 con 0x562d39749000 2023-10-11T09:48:49.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.568+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:12 12.24 12:24000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d39c00 con 0x562d39749800 2023-10-11T09:48:49.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.568+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.2 v2:172.21.15.78:6808/35732 3 ==== osd_op_reply(12 [pgnls-filter start_epoch 88 out=49b] v86'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749800 2023-10-11T09:48:49.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.568+0000 7fe33929f700 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x562d38d4e400 0x562d38d3f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.568+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:13 12.14 12:28000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4e800 con 0x562d38d4e400 2023-10-11T09:48:49.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.568+0000 7fe33e2a9700 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x562d38d4e400 0x562d38d3f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.572+0000 7fe33e2a9700 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x562d38d4e400 0x562d38d3f600 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.572+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.5 v2:172.21.15.133:6816/35899 1 ==== osd_op_reply(13 [pgnls-filter start_epoch 88 out=49b] v86'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d38d4e400 2023-10-11T09:48:49.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.572+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:14 12.34 12:2c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d0a400 con 0x562d38d4e400 2023-10-11T09:48:49.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.572+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.5 v2:172.21.15.133:6816/35899 2 ==== osd_op_reply(14 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d38d4e400 2023-10-11T09:48:49.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.572+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:15 12.c 12:30000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4ec00 con 0x562d39749400 2023-10-11T09:48:49.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.572+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.0 v2:172.21.15.78:6800/35735 3 ==== osd_op_reply(15 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749400 2023-10-11T09:48:49.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.572+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:16 12.2c 12:34000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d38400 con 0x562d39749800 2023-10-11T09:48:49.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.576+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.2 v2:172.21.15.78:6808/35732 4 ==== osd_op_reply(16 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749800 2023-10-11T09:48:49.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.576+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:17 12.1c 12:38000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4e800 con 0x562d39749400 2023-10-11T09:48:49.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.576+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.0 v2:172.21.15.78:6800/35735 4 ==== osd_op_reply(17 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749400 2023-10-11T09:48:49.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.576+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:18 12.3c 12:3c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d39c00 con 0x562d38d4e400 2023-10-11T09:48:49.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.576+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.5 v2:172.21.15.133:6816/35899 3 ==== osd_op_reply(18 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d38d4e400 2023-10-11T09:48:49.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.576+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:19 12.2 12:40000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4e000 con 0x562d38d0a800 2023-10-11T09:48:49.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.576+0000 7fe33daa8700 1 -- 172.21.15.78:0/3149944686 <== osd.4 v2:172.21.15.133:6808/35898 2 ==== osd_op_reply(19 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d440 con 0x562d38d0a800 2023-10-11T09:48:49.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.576+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:20 12.22 12:44000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d38800 con 0x562d38d38000 2023-10-11T09:48:49.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.580+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.7 v2:172.21.15.133:6824/35901 3 ==== osd_op_reply(20 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962cfc0 con 0x562d38d38000 2023-10-11T09:48:49.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.580+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:21 12.12 12:48000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4f000 con 0x562d38d38c00 2023-10-11T09:48:49.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.580+0000 7fe33daa8700 1 -- 172.21.15.78:0/3149944686 <== osd.3 v2:172.21.15.78:6824/35738 2 ==== osd_op_reply(21 [pgnls-filter start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d440 con 0x562d38d38c00 2023-10-11T09:48:49.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.580+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:22 12.32 12:4c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d39800 con 0x562d38d38c00 2023-10-11T09:48:49.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.580+0000 7fe33daa8700 1 -- 172.21.15.78:0/3149944686 <== osd.3 v2:172.21.15.78:6824/35738 3 ==== osd_op_reply(22 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d440 con 0x562d38d38c00 2023-10-11T09:48:49.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.580+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:23 12.a 12:50000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4f400 con 0x562d39749800 2023-10-11T09:48:49.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.580+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.2 v2:172.21.15.78:6808/35732 5 ==== osd_op_reply(23 [pgnls-filter start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749800 2023-10-11T09:48:49.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.580+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:24 12.2a 12:54000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d0a400 con 0x562d39749800 2023-10-11T09:48:49.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.580+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.2 v2:172.21.15.78:6808/35732 6 ==== osd_op_reply(24 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749800 2023-10-11T09:48:49.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.580+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:25 12.1a 12:58000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4f800 con 0x562d38d0a800 2023-10-11T09:48:49.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.584+0000 7fe33daa8700 1 -- 172.21.15.78:0/3149944686 <== osd.4 v2:172.21.15.133:6808/35898 3 ==== osd_op_reply(25 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d440 con 0x562d38d0a800 2023-10-11T09:48:49.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.584+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:26 12.3a 12:5c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4ec00 con 0x562d38d38c00 2023-10-11T09:48:49.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.584+0000 7fe33daa8700 1 -- 172.21.15.78:0/3149944686 <== osd.3 v2:172.21.15.78:6824/35738 4 ==== osd_op_reply(26 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d440 con 0x562d38d38c00 2023-10-11T09:48:49.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.584+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:27 12.6 12:60000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4f000 con 0x562d38d39400 2023-10-11T09:48:49.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.584+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.1 v2:172.21.15.78:6809/35734 2 ==== osd_op_reply(27 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962cfc0 con 0x562d38d39400 2023-10-11T09:48:49.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.584+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:28 12.26 12:64000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4e800 con 0x562d38d4e400 2023-10-11T09:48:49.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.584+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.5 v2:172.21.15.133:6816/35899 4 ==== osd_op_reply(28 [pgnls-filter start_epoch 88 out=49b] v86'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d38d4e400 2023-10-11T09:48:49.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.584+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:29 12.16 12:68000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4f800 con 0x562d39749000 2023-10-11T09:48:49.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.588+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.6 v2:172.21.15.133:6800/35900 3 ==== osd_op_reply(29 [pgnls-filter start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962cfc0 con 0x562d39749000 2023-10-11T09:48:49.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.588+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:30 12.36 12:6c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d38400 con 0x562d38d4e400 2023-10-11T09:48:49.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.588+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.5 v2:172.21.15.133:6816/35899 5 ==== osd_op_reply(30 [pgnls-filter start_epoch 88 out=49b] v86'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d38d4e400 2023-10-11T09:48:49.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.588+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:31 12.e 12:70000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4ec00 con 0x562d39749400 2023-10-11T09:48:49.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.588+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.0 v2:172.21.15.78:6800/35735 5 ==== osd_op_reply(31 [pgnls-filter start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749400 2023-10-11T09:48:49.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.588+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:32 12.2e 12:74000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d39c00 con 0x562d38d39400 2023-10-11T09:48:49.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.588+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.1 v2:172.21.15.78:6809/35734 3 ==== osd_op_reply(32 [pgnls-filter start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962cfc0 con 0x562d38d39400 2023-10-11T09:48:49.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.588+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:33 12.1e 12:78000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4f400 con 0x562d38d0a800 2023-10-11T09:48:49.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.592+0000 7fe33daa8700 1 -- 172.21.15.78:0/3149944686 <== osd.4 v2:172.21.15.133:6808/35898 4 ==== osd_op_reply(33 [pgnls-filter start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d440 con 0x562d38d0a800 2023-10-11T09:48:49.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.592+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:34 12.3e 12:7c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d39000 con 0x562d39749400 2023-10-11T09:48:49.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.592+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.0 v2:172.21.15.78:6800/35735 6 ==== osd_op_reply(34 [pgnls-filter start_epoch 88 out=49b] v86'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749400 2023-10-11T09:48:49.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.592+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:35 12.1 12:80000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4e000 con 0x562d39749000 2023-10-11T09:48:49.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.592+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.6 v2:172.21.15.133:6800/35900 4 ==== osd_op_reply(35 [pgnls-filter start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962cfc0 con 0x562d39749000 2023-10-11T09:48:49.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.592+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:36 12.21 12:84000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d0ac00 con 0x562d38d39400 2023-10-11T09:48:49.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.592+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.1 v2:172.21.15.78:6809/35734 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 88 out=49b] v86'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962cfc0 con 0x562d38d39400 2023-10-11T09:48:49.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.592+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:37 12.11 12:88000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d39800 con 0x562d38d4e400 2023-10-11T09:48:49.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.596+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.5 v2:172.21.15.133:6816/35899 6 ==== osd_op_reply(37 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d38d4e400 2023-10-11T09:48:49.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.596+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:38 12.31 12:8c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4e800 con 0x562d39749400 2023-10-11T09:48:49.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.596+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.0 v2:172.21.15.78:6800/35735 7 ==== osd_op_reply(38 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749400 2023-10-11T09:48:49.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.596+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:39 12.9 12:90000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4ec00 con 0x562d39749800 2023-10-11T09:48:49.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.596+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.2 v2:172.21.15.78:6808/35732 7 ==== osd_op_reply(39 [pgnls-filter start_epoch 88 out=49b] v86'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749800 2023-10-11T09:48:49.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.596+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:40 12.29 12:94000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d38400 con 0x562d38d0a800 2023-10-11T09:48:49.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.600+0000 7fe33daa8700 1 -- 172.21.15.78:0/3149944686 <== osd.4 v2:172.21.15.133:6808/35898 5 ==== osd_op_reply(40 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d440 con 0x562d38d0a800 2023-10-11T09:48:49.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.600+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:41 12.19 12:98000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4f000 con 0x562d39749400 2023-10-11T09:48:49.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.600+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.0 v2:172.21.15.78:6800/35735 8 ==== osd_op_reply(41 [pgnls-filter start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749400 2023-10-11T09:48:49.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.600+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:42 12.39 12:9c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d39000 con 0x562d39749000 2023-10-11T09:48:49.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.600+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.6 v2:172.21.15.133:6800/35900 5 ==== osd_op_reply(42 [pgnls-filter start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962cfc0 con 0x562d39749000 2023-10-11T09:48:49.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.600+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:43 12.5 12:a0000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4f800 con 0x562d39749000 2023-10-11T09:48:49.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.600+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.6 v2:172.21.15.133:6800/35900 6 ==== osd_op_reply(43 [pgnls-filter start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962cfc0 con 0x562d39749000 2023-10-11T09:48:49.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.600+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:44 12.25 12:a4000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d39800 con 0x562d38d38000 2023-10-11T09:48:49.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.604+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.7 v2:172.21.15.133:6824/35901 4 ==== osd_op_reply(44 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962cfc0 con 0x562d38d38000 2023-10-11T09:48:49.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.604+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:45 12.15 12:a8000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4fc00 con 0x562d39749000 2023-10-11T09:48:49.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.604+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.6 v2:172.21.15.133:6800/35900 7 ==== osd_op_reply(45 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962cfc0 con 0x562d39749000 2023-10-11T09:48:49.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.604+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:46 12.35 12:ac000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d38800 con 0x562d39749400 2023-10-11T09:48:49.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.604+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.0 v2:172.21.15.78:6800/35735 9 ==== osd_op_reply(46 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749400 2023-10-11T09:48:49.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.604+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:47 12.d 12:b0000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d39000 con 0x562d38d38000 2023-10-11T09:48:49.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.604+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.7 v2:172.21.15.133:6824/35901 5 ==== osd_op_reply(47 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962cfc0 con 0x562d38d38000 2023-10-11T09:48:49.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.608+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:48 12.2d 12:b4000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d39c00 con 0x562d38d38c00 2023-10-11T09:48:49.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.608+0000 7fe33daa8700 1 -- 172.21.15.78:0/3149944686 <== osd.3 v2:172.21.15.78:6824/35738 5 ==== osd_op_reply(48 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d440 con 0x562d38d38c00 2023-10-11T09:48:49.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.608+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:49 12.1d 12:b8000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d39800 con 0x562d39749400 2023-10-11T09:48:49.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.608+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.0 v2:172.21.15.78:6800/35735 10 ==== osd_op_reply(49 [pgnls-filter start_epoch 88 out=49b] v86'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749400 2023-10-11T09:48:49.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.608+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:50 12.3d 12:bc000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d0a400 con 0x562d38d39400 2023-10-11T09:48:49.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.608+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.1 v2:172.21.15.78:6809/35734 5 ==== osd_op_reply(50 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962cfc0 con 0x562d38d39400 2023-10-11T09:48:49.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.608+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:51 12.3 12:c0000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4e000 con 0x562d39749400 2023-10-11T09:48:49.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.608+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.0 v2:172.21.15.78:6800/35735 11 ==== osd_op_reply(51 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749400 2023-10-11T09:48:49.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.608+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:52 12.23 12:c4000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d5e000 con 0x562d39749400 2023-10-11T09:48:49.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.612+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.0 v2:172.21.15.78:6800/35735 12 ==== osd_op_reply(52 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749400 2023-10-11T09:48:49.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.612+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:53 12.13 12:c8000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4f800 con 0x562d38d39400 2023-10-11T09:48:49.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.612+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.1 v2:172.21.15.78:6809/35734 6 ==== osd_op_reply(53 [pgnls-filter start_epoch 88 out=49b] v86'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962cfc0 con 0x562d38d39400 2023-10-11T09:48:49.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.612+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:54 12.33 12:cc000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d5e400 con 0x562d39749800 2023-10-11T09:48:49.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.612+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.2 v2:172.21.15.78:6808/35732 8 ==== osd_op_reply(54 [pgnls-filter start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749800 2023-10-11T09:48:49.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.612+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:55 12.b 12:d0000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d0ac00 con 0x562d38d38c00 2023-10-11T09:48:49.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.612+0000 7fe33daa8700 1 -- 172.21.15.78:0/3149944686 <== osd.3 v2:172.21.15.78:6824/35738 6 ==== osd_op_reply(55 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d440 con 0x562d38d38c00 2023-10-11T09:48:49.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.612+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:56 12.2b 12:d4000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d5e800 con 0x562d38d0a800 2023-10-11T09:48:49.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.616+0000 7fe33daa8700 1 -- 172.21.15.78:0/3149944686 <== osd.4 v2:172.21.15.133:6808/35898 6 ==== osd_op_reply(56 [pgnls-filter start_epoch 88 out=49b] v86'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d440 con 0x562d38d0a800 2023-10-11T09:48:49.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.616+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:57 12.1b 12:d8000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4f400 con 0x562d38d0a800 2023-10-11T09:48:49.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.616+0000 7fe33daa8700 1 -- 172.21.15.78:0/3149944686 <== osd.4 v2:172.21.15.133:6808/35898 7 ==== osd_op_reply(57 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d440 con 0x562d38d0a800 2023-10-11T09:48:49.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.616+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:58 12.3b 12:dc000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d5ec00 con 0x562d39749800 2023-10-11T09:48:49.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.616+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.2 v2:172.21.15.78:6808/35732 9 ==== osd_op_reply(58 [pgnls-filter start_epoch 88 out=49b] v86'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749800 2023-10-11T09:48:49.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.616+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:59 12.7 12:e0000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4e800 con 0x562d38d39400 2023-10-11T09:48:49.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.616+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.1 v2:172.21.15.78:6809/35734 7 ==== osd_op_reply(59 [pgnls-filter start_epoch 88 out=81b] v88'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562d3962cfc0 con 0x562d38d39400 2023-10-11T09:48:49.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.616+0000 7fe33f2ec840 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:60 12.0 12:00000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d0b000 con 0x562d38d0a800 2023-10-11T09:48:49.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.620+0000 7fe33daa8700 1 -- 172.21.15.78:0/3149944686 <== osd.4 v2:172.21.15.133:6808/35898 8 ==== osd_op_reply(60 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d440 con 0x562d38d0a800 2023-10-11T09:48:49.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.620+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:61 12.20 12:04000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4ec00 con 0x562d38d38000 2023-10-11T09:48:49.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.620+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.7 v2:172.21.15.133:6824/35901 6 ==== osd_op_reply(61 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962cfc0 con 0x562d38d38000 2023-10-11T09:48:49.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.620+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:62 12.10 12:08000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d5f000 con 0x562d39749400 2023-10-11T09:48:49.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.620+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.0 v2:172.21.15.78:6800/35735 13 ==== osd_op_reply(62 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749400 2023-10-11T09:48:49.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.620+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:63 12.30 12:0c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4f000 con 0x562d38d38000 2023-10-11T09:48:49.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.620+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.7 v2:172.21.15.133:6824/35901 7 ==== osd_op_reply(63 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962cfc0 con 0x562d38d38000 2023-10-11T09:48:49.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.620+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:64 12.8 12:10000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d5f400 con 0x562d38d38c00 2023-10-11T09:48:49.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.624+0000 7fe33daa8700 1 -- 172.21.15.78:0/3149944686 <== osd.3 v2:172.21.15.78:6824/35738 7 ==== osd_op_reply(64 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d440 con 0x562d38d38c00 2023-10-11T09:48:49.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.624+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:65 12.28 12:14000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d38400 con 0x562d39749400 2023-10-11T09:48:49.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.624+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.0 v2:172.21.15.78:6800/35735 14 ==== osd_op_reply(65 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749400 2023-10-11T09:48:49.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.624+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:66 12.18 12:18000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d5f800 con 0x562d38d39400 2023-10-11T09:48:49.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.624+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.1 v2:172.21.15.78:6809/35734 8 ==== osd_op_reply(66 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962cfc0 con 0x562d38d39400 2023-10-11T09:48:49.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.624+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:67 12.38 12:1c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d39c00 con 0x562d39749800 2023-10-11T09:48:49.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.624+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.2 v2:172.21.15.78:6808/35732 10 ==== osd_op_reply(67 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749800 2023-10-11T09:48:49.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.624+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:68 12.4 12:20000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d5fc00 con 0x562d39749000 2023-10-11T09:48:49.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.628+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.6 v2:172.21.15.133:6800/35900 8 ==== osd_op_reply(68 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962cfc0 con 0x562d39749000 2023-10-11T09:48:49.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.628+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:69 12.24 12:24000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d38800 con 0x562d39749800 2023-10-11T09:48:49.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.628+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.2 v2:172.21.15.78:6808/35732 11 ==== osd_op_reply(69 [pgnls-filter start_epoch 88 out=49b] v86'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749800 2023-10-11T09:48:49.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.628+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:70 12.14 12:28000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d5e800 con 0x562d38d4e400 2023-10-11T09:48:49.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.628+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.5 v2:172.21.15.133:6816/35899 7 ==== osd_op_reply(70 [pgnls-filter start_epoch 88 out=49b] v86'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d38d4e400 2023-10-11T09:48:49.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.628+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:71 12.34 12:2c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d39800 con 0x562d38d4e400 2023-10-11T09:48:49.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.628+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.5 v2:172.21.15.133:6816/35899 8 ==== osd_op_reply(71 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d38d4e400 2023-10-11T09:48:49.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.628+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:72 12.c 12:30000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d5e000 con 0x562d39749400 2023-10-11T09:48:49.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.632+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.0 v2:172.21.15.78:6800/35735 15 ==== osd_op_reply(72 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749400 2023-10-11T09:48:49.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.632+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:73 12.2c 12:34000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d0ac00 con 0x562d39749800 2023-10-11T09:48:49.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.632+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.2 v2:172.21.15.78:6808/35732 12 ==== osd_op_reply(73 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749800 2023-10-11T09:48:49.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.632+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:74 12.1c 12:38000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4fc00 con 0x562d39749400 2023-10-11T09:48:49.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.632+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.0 v2:172.21.15.78:6800/35735 16 ==== osd_op_reply(74 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749400 2023-10-11T09:48:49.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.632+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:75 12.3c 12:3c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d38400 con 0x562d38d4e400 2023-10-11T09:48:49.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.632+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.5 v2:172.21.15.133:6816/35899 9 ==== osd_op_reply(75 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d38d4e400 2023-10-11T09:48:49.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.632+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:76 12.2 12:40000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4f800 con 0x562d38d0a800 2023-10-11T09:48:49.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.636+0000 7fe33daa8700 1 -- 172.21.15.78:0/3149944686 <== osd.4 v2:172.21.15.133:6808/35898 9 ==== osd_op_reply(76 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d440 con 0x562d38d0a800 2023-10-11T09:48:49.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.636+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:77 12.22 12:44000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d39c00 con 0x562d38d38000 2023-10-11T09:48:49.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.636+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.7 v2:172.21.15.133:6824/35901 8 ==== osd_op_reply(77 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962cfc0 con 0x562d38d38000 2023-10-11T09:48:49.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.636+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:78 12.12 12:48000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4e800 con 0x562d38d38c00 2023-10-11T09:48:49.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.636+0000 7fe33daa8700 1 -- 172.21.15.78:0/3149944686 <== osd.3 v2:172.21.15.78:6824/35738 8 ==== osd_op_reply(78 [pgnls-filter start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d440 con 0x562d38d38c00 2023-10-11T09:48:49.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.636+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:79 12.32 12:4c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d38800 con 0x562d38d38c00 2023-10-11T09:48:49.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.636+0000 7fe33daa8700 1 -- 172.21.15.78:0/3149944686 <== osd.3 v2:172.21.15.78:6824/35738 9 ==== osd_op_reply(79 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d440 con 0x562d38d38c00 2023-10-11T09:48:49.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.636+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:80 12.a 12:50000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4ec00 con 0x562d39749800 2023-10-11T09:48:49.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.640+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.2 v2:172.21.15.78:6808/35732 13 ==== osd_op_reply(80 [pgnls-filter start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749800 2023-10-11T09:48:49.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.640+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:81 12.2a 12:54000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d39800 con 0x562d39749800 2023-10-11T09:48:49.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.640+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.2 v2:172.21.15.78:6808/35732 14 ==== osd_op_reply(81 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749800 2023-10-11T09:48:49.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.640+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:82 12.1a 12:58000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4e000 con 0x562d38d0a800 2023-10-11T09:48:49.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.640+0000 7fe33daa8700 1 -- 172.21.15.78:0/3149944686 <== osd.4 v2:172.21.15.133:6808/35898 10 ==== osd_op_reply(82 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d440 con 0x562d38d0a800 2023-10-11T09:48:49.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.640+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:83 12.3a 12:5c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d5e400 con 0x562d38d38c00 2023-10-11T09:48:49.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.640+0000 7fe33daa8700 1 -- 172.21.15.78:0/3149944686 <== osd.3 v2:172.21.15.78:6824/35738 10 ==== osd_op_reply(83 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d440 con 0x562d38d38c00 2023-10-11T09:48:49.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.640+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:84 12.6 12:60000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d0a400 con 0x562d38d39400 2023-10-11T09:48:49.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.640+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.1 v2:172.21.15.78:6809/35734 9 ==== osd_op_reply(84 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962cfc0 con 0x562d38d39400 2023-10-11T09:48:49.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.640+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:85 12.26 12:64000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d5ec00 con 0x562d38d4e400 2023-10-11T09:48:49.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.644+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.5 v2:172.21.15.133:6816/35899 10 ==== osd_op_reply(85 [pgnls-filter start_epoch 88 out=49b] v86'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d38d4e400 2023-10-11T09:48:49.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.644+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:86 12.16 12:68000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d39000 con 0x562d39749000 2023-10-11T09:48:49.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.644+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.6 v2:172.21.15.133:6800/35900 9 ==== osd_op_reply(86 [pgnls-filter start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962cfc0 con 0x562d39749000 2023-10-11T09:48:49.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.644+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:87 12.36 12:6c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d5f000 con 0x562d38d4e400 2023-10-11T09:48:49.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.644+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.5 v2:172.21.15.133:6816/35899 11 ==== osd_op_reply(87 [pgnls-filter start_epoch 88 out=49b] v86'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d38d4e400 2023-10-11T09:48:49.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.644+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:88 12.e 12:70000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d38800 con 0x562d39749400 2023-10-11T09:48:49.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.648+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.0 v2:172.21.15.78:6800/35735 17 ==== osd_op_reply(88 [pgnls-filter start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749400 2023-10-11T09:48:49.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.648+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:89 12.2e 12:74000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d5e800 con 0x562d38d39400 2023-10-11T09:48:49.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.648+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.1 v2:172.21.15.78:6809/35734 10 ==== osd_op_reply(89 [pgnls-filter start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962cfc0 con 0x562d38d39400 2023-10-11T09:48:49.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.648+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:90 12.1e 12:78000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4f400 con 0x562d38d0a800 2023-10-11T09:48:49.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.648+0000 7fe33daa8700 1 -- 172.21.15.78:0/3149944686 <== osd.4 v2:172.21.15.133:6808/35898 11 ==== osd_op_reply(90 [pgnls-filter start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d440 con 0x562d38d0a800 2023-10-11T09:48:49.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.648+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:91 12.3e 12:7c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d5f800 con 0x562d39749400 2023-10-11T09:48:49.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.648+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.0 v2:172.21.15.78:6800/35735 18 ==== osd_op_reply(91 [pgnls-filter start_epoch 88 out=49b] v86'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749400 2023-10-11T09:48:49.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.648+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:92 12.1 12:80000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4f800 con 0x562d39749000 2023-10-11T09:48:49.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.648+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.6 v2:172.21.15.133:6800/35900 10 ==== osd_op_reply(92 [pgnls-filter start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962cfc0 con 0x562d39749000 2023-10-11T09:48:49.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.648+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:93 12.21 12:84000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d5fc00 con 0x562d38d39400 2023-10-11T09:48:49.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.652+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.1 v2:172.21.15.78:6809/35734 11 ==== osd_op_reply(93 [pgnls-filter start_epoch 88 out=49b] v86'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962cfc0 con 0x562d38d39400 2023-10-11T09:48:49.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.652+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:94 12.11 12:88000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4e800 con 0x562d38d4e400 2023-10-11T09:48:49.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.652+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.5 v2:172.21.15.133:6816/35899 12 ==== osd_op_reply(94 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d38d4e400 2023-10-11T09:48:49.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.652+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:95 12.31 12:8c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d5e800 con 0x562d39749400 2023-10-11T09:48:49.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.652+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.0 v2:172.21.15.78:6800/35735 19 ==== osd_op_reply(95 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749400 2023-10-11T09:48:49.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.652+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:96 12.9 12:90000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d5f400 con 0x562d39749800 2023-10-11T09:48:49.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.652+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.2 v2:172.21.15.78:6808/35732 15 ==== osd_op_reply(96 [pgnls-filter start_epoch 88 out=49b] v86'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749800 2023-10-11T09:48:49.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.652+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:97 12.29 12:94000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d5e000 con 0x562d38d0a800 2023-10-11T09:48:49.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.656+0000 7fe33daa8700 1 -- 172.21.15.78:0/3149944686 <== osd.4 v2:172.21.15.133:6808/35898 12 ==== osd_op_reply(97 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d440 con 0x562d38d0a800 2023-10-11T09:48:49.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.656+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:98 12.19 12:98000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d0b000 con 0x562d39749400 2023-10-11T09:48:49.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.656+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.0 v2:172.21.15.78:6800/35735 20 ==== osd_op_reply(98 [pgnls-filter start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749400 2023-10-11T09:48:49.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.656+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:99 12.39 12:9c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d5ec00 con 0x562d39749000 2023-10-11T09:48:49.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.656+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.6 v2:172.21.15.133:6800/35900 11 ==== osd_op_reply(99 [pgnls-filter start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962cfc0 con 0x562d39749000 2023-10-11T09:48:49.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.656+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:100 12.5 12:a0000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d39c00 con 0x562d39749000 2023-10-11T09:48:49.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.656+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.6 v2:172.21.15.133:6800/35900 12 ==== osd_op_reply(100 [pgnls-filter start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962cfc0 con 0x562d39749000 2023-10-11T09:48:49.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.656+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:101 12.25 12:a4000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d5f000 con 0x562d38d38000 2023-10-11T09:48:49.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.660+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.7 v2:172.21.15.133:6824/35901 9 ==== osd_op_reply(101 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962cfc0 con 0x562d38d38000 2023-10-11T09:48:49.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.660+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:102 12.15 12:a8000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d39000 con 0x562d39749000 2023-10-11T09:48:49.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.660+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.6 v2:172.21.15.133:6800/35900 13 ==== osd_op_reply(102 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962cfc0 con 0x562d39749000 2023-10-11T09:48:49.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.660+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:103 12.35 12:ac000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4f000 con 0x562d39749400 2023-10-11T09:48:49.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.660+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.0 v2:172.21.15.78:6800/35735 21 ==== osd_op_reply(103 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749400 2023-10-11T09:48:49.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.660+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:104 12.d 12:b0000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d38400 con 0x562d38d38000 2023-10-11T09:48:49.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.660+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.7 v2:172.21.15.133:6824/35901 10 ==== osd_op_reply(104 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962cfc0 con 0x562d38d38000 2023-10-11T09:48:49.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.660+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:105 12.2d 12:b4000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4fc00 con 0x562d38d38c00 2023-10-11T09:48:49.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.664+0000 7fe33daa8700 1 -- 172.21.15.78:0/3149944686 <== osd.3 v2:172.21.15.78:6824/35738 11 ==== osd_op_reply(105 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d440 con 0x562d38d38c00 2023-10-11T09:48:49.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.664+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:106 12.1d 12:b8000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d39800 con 0x562d39749400 2023-10-11T09:48:49.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.664+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.0 v2:172.21.15.78:6800/35735 22 ==== osd_op_reply(106 [pgnls-filter start_epoch 88 out=49b] v86'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749400 2023-10-11T09:48:49.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.664+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:107 12.3d 12:bc000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4ec00 con 0x562d38d39400 2023-10-11T09:48:49.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.664+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.1 v2:172.21.15.78:6809/35734 12 ==== osd_op_reply(107 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962cfc0 con 0x562d38d39400 2023-10-11T09:48:49.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.664+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:108 12.3 12:c0000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d38800 con 0x562d39749400 2023-10-11T09:48:49.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.664+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.0 v2:172.21.15.78:6800/35735 23 ==== osd_op_reply(108 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749400 2023-10-11T09:48:49.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.664+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:109 12.23 12:c4000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d0a400 con 0x562d39749400 2023-10-11T09:48:49.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.664+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.0 v2:172.21.15.78:6800/35735 24 ==== osd_op_reply(109 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749400 2023-10-11T09:48:49.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.664+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:110 12.13 12:c8000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4f800 con 0x562d38d39400 2023-10-11T09:48:49.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.668+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.1 v2:172.21.15.78:6809/35734 13 ==== osd_op_reply(110 [pgnls-filter start_epoch 88 out=49b] v86'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962cfc0 con 0x562d38d39400 2023-10-11T09:48:49.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.668+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:111 12.33 12:cc000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d0ac00 con 0x562d39749800 2023-10-11T09:48:49.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.668+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.2 v2:172.21.15.78:6808/35732 16 ==== osd_op_reply(111 [pgnls-filter start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749800 2023-10-11T09:48:49.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.668+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:112 12.b 12:d0000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4ec00 con 0x562d38d38c00 2023-10-11T09:48:49.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.668+0000 7fe33daa8700 1 -- 172.21.15.78:0/3149944686 <== osd.3 v2:172.21.15.78:6824/35738 12 ==== osd_op_reply(112 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d440 con 0x562d38d38c00 2023-10-11T09:48:49.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.668+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:113 12.2b 12:d4000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d0b000 con 0x562d38d0a800 2023-10-11T09:48:49.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.668+0000 7fe33daa8700 1 -- 172.21.15.78:0/3149944686 <== osd.4 v2:172.21.15.133:6808/35898 13 ==== osd_op_reply(113 [pgnls-filter start_epoch 88 out=49b] v86'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d440 con 0x562d38d0a800 2023-10-11T09:48:49.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.668+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:114 12.1b 12:d8000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d39c00 con 0x562d38d0a800 2023-10-11T09:48:49.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.668+0000 7fe33daa8700 1 -- 172.21.15.78:0/3149944686 <== osd.4 v2:172.21.15.133:6808/35898 14 ==== osd_op_reply(114 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d440 con 0x562d38d0a800 2023-10-11T09:48:49.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.668+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:115 12.3b 12:dc000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d0a400 con 0x562d39749800 2023-10-11T09:48:49.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.672+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.2 v2:172.21.15.78:6808/35732 17 ==== osd_op_reply(115 [pgnls-filter start_epoch 88 out=49b] v86'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749800 2023-10-11T09:48:49.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.672+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:116 12.7 12:e0000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d39000 con 0x562d38d39400 2023-10-11T09:48:49.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.672+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.1 v2:172.21.15.78:6809/35734 14 ==== osd_op_reply(116 [pgnls-filter start_epoch 88 out=81b] v88'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562d3962cfc0 con 0x562d38d39400 2023-10-11T09:48:49.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.672+0000 7fe33f2ec840 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2023-10-11T09:48:49.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.672+0000 7fe33f2ec840 10 datascan.operator(): handling object 10000000001.0 2023-10-11T09:48:49.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.672+0000 7fe33f2ec840 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:117 12.7 12:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d0b400 con 0x562d38d39400 2023-10-11T09:48:49.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.672+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.1 v2:172.21.15.78:6809/35734 15 ==== osd_op_reply(117 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (103) out=103b,getxattr,getxattr] v0'0 uv9 ondisk = 0) v8 ==== 416+0+149 (crc 0 0 0) 0x562d3962cfc0 con 0x562d38d39400 2023-10-11T09:48:49.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.672+0000 7fe33f2ec840 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:118 13.7 13:e2e07930:::10000000001.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d0b800 con 0x562d38d38000 2023-10-11T09:48:49.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.672+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.7 v2:172.21.15.133:6824/35901 11 ==== osd_op_reply(118 10000000001.00000000 [getxattr (30) out=30b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x562d3962cfc0 con 0x562d38d38000 2023-10-11T09:48:49.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.672+0000 7fe33f2ec840 20 datascan.operator(): loaded layout from xattr: pi: 13 os: 4194304 sc: 1 su: 4194304 2023-10-11T09:48:49.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.672+0000 7fe33f2ec840 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2023-10-11T09:48:49.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.672+0000 7fe33f2ec840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/sixmegs 2023-10-11T09:48:49.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.672+0000 7fe33f2ec840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=sixmegs 2023-10-11T09:48:49.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.672+0000 7fe33f2ec840 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:119 11.10 11:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d0bc00 con 0x562d39749400 2023-10-11T09:48:49.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.672+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.0 v2:172.21.15.78:6800/35735 25 ==== osd_op_reply(119 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x562d3962d200 con 0x562d39749400 2023-10-11T09:48:49.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.672+0000 7fe33f2ec840 4 datascan.get_frag_of: Unexpected error on '10000000000.00000000': (2) No such file or directory 2023-10-11T09:48:49.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.672+0000 7fe33f2ec840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/sixmegs, will insert to root 2023-10-11T09:48:49.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.672+0000 7fe33f2ec840 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:120 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d0b000 con 0x562d39749400 2023-10-11T09:48:49.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.676+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.0 v2:172.21.15.78:6800/35735 26 ==== osd_op_reply(120 10000000000.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x562d3962d200 con 0x562d39749400 2023-10-11T09:48:49.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.676+0000 7fe33f2ec840 10 datascan.find_or_create_dirfrag: read_version = 0 2023-10-11T09:48:49.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.676+0000 7fe33f2ec840 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:121 11.10 11:0cf6ad0f:::10000000000.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4e000 con 0x562d39749400 2023-10-11T09:48:49.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.676+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.0 v2:172.21.15.78:6800/35735 27 ==== osd_op_reply(121 10000000000.00000000 [create,omap-set-header] v88'16 uv16 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x562d3962d200 con 0x562d39749400 2023-10-11T09:48:49.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.676+0000 7fe33f2ec840 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x10000000000 2023-10-11T09:48:49.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.676+0000 7fe33f2ec840 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:122 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4f400 con 0x562d39749400 2023-10-11T09:48:49.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.680+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.0 v2:172.21.15.78:6800/35735 28 ==== osd_op_reply(122 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x562d3962d200 con 0x562d39749400 2023-10-11T09:48:49.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.680+0000 7fe33f2ec840 20 datascan.read_dentry: oid=10000000000.00000000 dname=sixmegs frag=*, r=0 2023-10-11T09:48:49.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.680+0000 7fe33f2ec840 20 datascan.read_dentry: sixmegs_head not found in result 2023-10-11T09:48:49.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.680+0000 7fe33f2ec840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/sixmegs with size=6291456 bytes 2023-10-11T09:48:49.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.680+0000 7fe33f2ec840 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:123 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=508b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4fc00 con 0x562d39749400 2023-10-11T09:48:49.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.680+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.0 v2:172.21.15.78:6800/35735 29 ==== osd_op_reply(123 10000000000.00000000 [omap-set-vals] v88'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562d3962d200 con 0x562d39749400 2023-10-11T09:48:49.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.680+0000 7fe33f2ec840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/sixmegs pointing to 0x0x10000000001 2023-10-11T09:48:49.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.680+0000 7fe33f2ec840 10 datascan.inject_with_backtrace: backptr: 0x0x1/subdir 2023-10-11T09:48:49.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.680+0000 7fe33f2ec840 20 datascan.get_frag_of: dirino=0x1 target_dname=subdir 2023-10-11T09:48:49.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.680+0000 7fe33f2ec840 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:124 11.1f 11:ff5b34d6:::1.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4ec00 con 0x562d39749800 2023-10-11T09:48:49.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.684+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.2 v2:172.21.15.78:6808/35732 18 ==== osd_op_reply(124 1.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x562d3962d200 con 0x562d39749800 2023-10-11T09:48:49.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.684+0000 7fe33f2ec840 4 datascan.get_frag_of: Unexpected error on '1.00000000': (2) No such file or directory 2023-10-11T09:48:49.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.684+0000 7fe33f2ec840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x1/subdir, will insert to root 2023-10-11T09:48:49.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.684+0000 7fe33f2ec840 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:125 11.1f 11:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d5e400 con 0x562d39749800 2023-10-11T09:48:49.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.684+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.2 v2:172.21.15.78:6808/35732 19 ==== osd_op_reply(125 1.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x562d3962d200 con 0x562d39749800 2023-10-11T09:48:49.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.684+0000 7fe33f2ec840 10 datascan.find_or_create_dirfrag: read_version = 0 2023-10-11T09:48:49.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.684+0000 7fe33f2ec840 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:126 11.1f 11:ff5b34d6:::1.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d5e000 con 0x562d39749800 2023-10-11T09:48:49.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.684+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.2 v2:172.21.15.78:6808/35732 20 ==== osd_op_reply(126 1.00000000 [create,omap-set-header] v88'11 uv11 ondisk = 0) v8 ==== 196+0+0 (crc 0 0 0) 0x562d3962d200 con 0x562d39749800 2023-10-11T09:48:49.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.684+0000 7fe33f2ec840 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x1 2023-10-11T09:48:49.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.684+0000 7fe33f2ec840 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:127 11.1f 11:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4a000 con 0x562d39749800 2023-10-11T09:48:49.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.684+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.2 v2:172.21.15.78:6808/35732 21 ==== osd_op_reply(127 1.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv11 ondisk = 0) v8 ==== 154+0+4 (crc 0 0 0) 0x562d3962d200 con 0x562d39749800 2023-10-11T09:48:49.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.684+0000 7fe33f2ec840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:48:49.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.684+0000 7fe33f2ec840 20 datascan.read_dentry: subdir_head not found in result 2023-10-11T09:48:49.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.684+0000 7fe33f2ec840 10 datascan.inject_with_backtrace: Linking ancestor directory of inode 0x0x10000000000 at 0x0x1:subdir 2023-10-11T09:48:49.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.684+0000 7fe33f2ec840 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:128 11.1f 11:ff5b34d6:::1.00000000:head [omap-set-vals in=507b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4a400 con 0x562d39749800 2023-10-11T09:48:49.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.688+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.2 v2:172.21.15.78:6808/35732 22 ==== osd_op_reply(128 1.00000000 [omap-set-vals] v88'12 uv12 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x562d3962d200 con 0x562d39749800 2023-10-11T09:48:49.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.688+0000 7fe33f2ec840 20 datascan.inject_linkage: Injected dentry 0x0x1/subdir pointing to 0x0x10000000000 2023-10-11T09:48:49.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.688+0000 7fe33f2ec840 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:129 12.27 12:e4000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d4a800 con 0x562d39749400 2023-10-11T09:48:49.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.688+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.0 v2:172.21.15.78:6800/35735 30 ==== osd_op_reply(129 [pgnls-filter start_epoch 88 out=49b] v86'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749400 2023-10-11T09:48:49.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.688+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:130 12.17 12:e8000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d38400 con 0x562d39749000 2023-10-11T09:48:49.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.692+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.6 v2:172.21.15.133:6800/35900 14 ==== osd_op_reply(130 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962cfc0 con 0x562d39749000 2023-10-11T09:48:49.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.692+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:131 12.37 12:ec000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d0ac00 con 0x562d38d0a800 2023-10-11T09:48:49.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.692+0000 7fe33daa8700 1 -- 172.21.15.78:0/3149944686 <== osd.4 v2:172.21.15.133:6808/35898 15 ==== osd_op_reply(131 [pgnls-filter start_epoch 88 out=49b] v86'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d440 con 0x562d38d0a800 2023-10-11T09:48:49.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.692+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:132 12.f 12:f0000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d5fc00 con 0x562d39749000 2023-10-11T09:48:49.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.692+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.6 v2:172.21.15.133:6800/35900 15 ==== osd_op_reply(132 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962cfc0 con 0x562d39749000 2023-10-11T09:48:49.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.692+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:133 12.2f 12:f4000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d39800 con 0x562d39749800 2023-10-11T09:48:49.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.692+0000 7fe33e2a9700 1 -- 172.21.15.78:0/3149944686 <== osd.2 v2:172.21.15.78:6808/35732 23 ==== osd_op_reply(133 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962d200 con 0x562d39749800 2023-10-11T09:48:49.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.692+0000 7fe338a9e700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:134 12.1f 12:f8000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d5ec00 con 0x562d39749000 2023-10-11T09:48:49.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.696+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.6 v2:172.21.15.133:6800/35900 16 ==== osd_op_reply(134 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962cfc0 con 0x562d39749000 2023-10-11T09:48:49.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.696+0000 7fe33929f700 1 -- 172.21.15.78:0/3149944686 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:135 12.3f 12:fc000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x562d38d38800 con 0x562d38d38000 2023-10-11T09:48:49.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.696+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 <== osd.7 v2:172.21.15.133:6824/35901 12 ==== osd_op_reply(135 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562d3962cfc0 con 0x562d38d38000 2023-10-11T09:48:49.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.696+0000 7fe33f2ec840 1 -- 172.21.15.78:0/3149944686 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x562d39749400 msgr2=0x562d38d3e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.696+0000 7fe33f2ec840 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x562d39749400 0x562d38d3e580 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.696+0000 7fe33f2ec840 1 -- 172.21.15.78:0/3149944686 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x562d38d39400 msgr2=0x562d38d3f080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.696+0000 7fe33f2ec840 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x562d38d39400 0x562d38d3f080 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.696+0000 7fe33f2ec840 1 -- 172.21.15.78:0/3149944686 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x562d39749800 msgr2=0x562d38965b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.696+0000 7fe33f2ec840 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x562d39749800 0x562d38965b80 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.696+0000 7fe33f2ec840 1 -- 172.21.15.78:0/3149944686 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x562d38d38c00 msgr2=0x562d38d3eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.696+0000 7fe33f2ec840 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x562d38d38c00 0x562d38d3eb00 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.696+0000 7fe33f2ec840 1 -- 172.21.15.78:0/3149944686 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x562d38d0a800 msgr2=0x562d38965600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.696+0000 7fe33f2ec840 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x562d38d0a800 0x562d38965600 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.696+0000 7fe33f2ec840 1 -- 172.21.15.78:0/3149944686 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x562d38d4e400 msgr2=0x562d38d3f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.696+0000 7fe33f2ec840 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x562d38d4e400 0x562d38d3f600 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.696+0000 7fe33f2ec840 1 -- 172.21.15.78:0/3149944686 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x562d39749000 msgr2=0x562d38966100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.696+0000 7fe33f2ec840 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x562d39749000 0x562d38966100 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.696+0000 7fe33f2ec840 1 -- 172.21.15.78:0/3149944686 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x562d38d38000 msgr2=0x562d38d3e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.696+0000 7fe33f2ec840 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x562d38d38000 0x562d38d3e000 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.696+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 reap_dead start 2023-10-11T09:48:49.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.696+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 reap_dead start 2023-10-11T09:48:49.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.696+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 reap_dead start 2023-10-11T09:48:49.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.696+0000 7fe33d2a7700 1 -- 172.21.15.78:0/3149944686 reap_dead start 2023-10-11T09:48:49.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.696+0000 7fe33f2ec840 1 -- 172.21.15.78:0/3149944686 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x562d38d0a000 msgr2=0x562d38967700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.696+0000 7fe33f2ec840 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x562d38d0a000 0x562d38967700 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x562d397406f0 tx=0x562d38bb8c00 comp rx=0 tx=0).stop 2023-10-11T09:48:49.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.696+0000 7fe33f2ec840 1 -- 172.21.15.78:0/3149944686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d39748400 msgr2=0x562d38967180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.696+0000 7fe33f2ec840 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d39748400 0x562d38967180 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x562d397413b0 tx=0x562d38bb8e40 comp rx=0 tx=0).stop 2023-10-11T09:48:49.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.696+0000 7fe33f2ec840 1 -- 172.21.15.78:0/3149944686 shutdown_connections 2023-10-11T09:48:49.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.696+0000 7fe33f2ec840 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x562d38d0a000 0x562d38967700 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.696+0000 7fe33f2ec840 1 --2- 172.21.15.78:0/3149944686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d39748400 0x562d38967180 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.696+0000 7fe33f2ec840 1 -- 172.21.15.78:0/3149944686 >> 172.21.15.78:0/3149944686 conn(0x562d388bec00 msgr2=0x562d389af800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:49.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.696+0000 7fe33f2ec840 1 -- 172.21.15.78:0/3149944686 shutdown_connections 2023-10-11T09:48:49.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.696+0000 7fe33f2ec840 1 -- 172.21.15.78:0/3149944686 wait complete. 2023-10-11T09:48:49.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.696+0000 7fe33f2ec840 1 -- 172.21.15.78:0/1492595867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d388bf800 msgr2=0x562d38964580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.700+0000 7fe33f2ec840 1 --2- 172.21.15.78:0/1492595867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d388bf800 0x562d38964580 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x562d3898d3e0 tx=0x562d396a6d80 comp rx=0 tx=0).stop 2023-10-11T09:48:49.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.700+0000 7fe33f2ec840 1 -- 172.21.15.78:0/1492595867 shutdown_connections 2023-10-11T09:48:49.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.700+0000 7fe33f2ec840 1 --2- 172.21.15.78:0/1492595867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d388bf000 0x562d38965080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.700+0000 7fe33f2ec840 1 --2- 172.21.15.78:0/1492595867 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562d388bf400 0x562d38964b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.700+0000 7fe33f2ec840 1 --2- 172.21.15.78:0/1492595867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d388bf800 0x562d38964580 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.700+0000 7fe33f2ec840 1 -- 172.21.15.78:0/1492595867 >> 172.21.15.78:0/1492595867 conn(0x562d388bfc00 msgr2=0x562d389af000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:49.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.700+0000 7fe33f2ec840 1 -- 172.21.15.78:0/1492595867 shutdown_connections 2023-10-11T09:48:49.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.700+0000 7fe33f2ec840 1 -- 172.21.15.78:0/1492595867 wait complete. 2023-10-11T09:48:49.712 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes'] in time 0:00:00.237142, result: 2023-10-11T09:48:49.713 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2023-10-11T09:48:49.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.768+0000 7f02a6cf5840 1 -- start start 2023-10-11T09:48:49.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.768+0000 7f02a6cf5840 1 Processor -- start 2023-10-11T09:48:49.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.772+0000 7f02a6cf5840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55edde76b000 0x55edde810580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.772+0000 7f02a6cf5840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55edde76b400 0x55edde810b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.772+0000 7f02a6cf5840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55edde76b800 0x55edde811080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.772+0000 7f02a6cf5840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edde7eaa00 con 0x55edde76b400 2023-10-11T09:48:49.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.772+0000 7f02a6cf5840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edde7ea780 con 0x55edde76b000 2023-10-11T09:48:49.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.772+0000 7f02a6cf5840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55edde7ea640 con 0x55edde76b800 2023-10-11T09:48:49.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.772+0000 7f02a4cb0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55edde76b400 0x55edde810b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.772+0000 7f02a4cb0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55edde76b400 0x55edde810b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35786/0 (socket says 172.21.15.78:35786) 2023-10-11T09:48:49.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.772+0000 7f02a4cb0700 1 -- 172.21.15.78:0/1027891103 learned_addr learned my addr 172.21.15.78:0/1027891103 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:49.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.772+0000 7f02a54b1700 1 --2- 172.21.15.78:0/1027891103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55edde76b000 0x55edde810580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.772+0000 7f02a5cb2700 1 --2- 172.21.15.78:0/1027891103 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55edde76b800 0x55edde811080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.772+0000 7f02a4cb0700 1 -- 172.21.15.78:0/1027891103 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55edde76b800 msgr2=0x55edde811080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:49.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.772+0000 7f02a4cb0700 1 --2- 172.21.15.78:0/1027891103 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55edde76b800 0x55edde811080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.772+0000 7f02a4cb0700 1 -- 172.21.15.78:0/1027891103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55edde76b000 msgr2=0x55edde810580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:49.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.772+0000 7f02a4cb0700 1 --2- 172.21.15.78:0/1027891103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55edde76b000 0x55edde810580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.772+0000 7f02a4cb0700 1 -- 172.21.15.78:0/1027891103 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55edde78cb60 con 0x55edde76b400 2023-10-11T09:48:49.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.772+0000 7f02a4cb0700 1 --2- 172.21.15.78:0/1027891103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55edde76b400 0x55edde810b00 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x55edde879560 tx=0x55eddf550480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a3b005c663f2573 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.772+0000 7f02a2cac700 1 -- 172.21.15.78:0/1027891103 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55edde79b600 con 0x55edde76b400 2023-10-11T09:48:49.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.772+0000 7f02a2cac700 1 -- 172.21.15.78:0/1027891103 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55edde7b6480 con 0x55edde76b400 2023-10-11T09:48:49.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.772+0000 7f02a6cf5840 1 -- 172.21.15.78:0/1027891103 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55edde78c340 con 0x55edde76b400 2023-10-11T09:48:49.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.772+0000 7f02a6cf5840 1 -- 172.21.15.78:0/1027891103 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55edde78c000 con 0x55edde76b400 2023-10-11T09:48:49.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.772+0000 7f02a6cf5840 4 waiting for MDS map... 2023-10-11T09:48:49.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.772+0000 7f02a2cac700 1 -- 172.21.15.78:0/1027891103 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55edde79b1e0 con 0x55edde76b400 2023-10-11T09:48:49.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.772+0000 7f02a2cac700 1 -- 172.21.15.78:0/1027891103 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(88..88 src has 1..88) v4 ==== 6638+0+0 (secure 0 0 0) 0x55edde7a4c40 con 0x55edde76b400 2023-10-11T09:48:49.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.772+0000 7f02a2cac700 1 -- 172.21.15.78:0/1027891103 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 64) v1 ==== 3118+0+0 (secure 0 0 0) 0x55eddf54e300 con 0x55edde76b400 2023-10-11T09:48:49.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.772+0000 7f02a6cf5840 4 Got MDS map 64 2023-10-11T09:48:49.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.772+0000 7f02a6cf5840 4 datascan.main: Using metadata pool output 2023-10-11T09:48:49.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.772+0000 7f02a6cf5840 4 datascan.main: connecting to RADOS... 2023-10-11T09:48:49.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.776+0000 7f02a6cf5840 1 Processor -- start 2023-10-11T09:48:49.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.776+0000 7f02a6cf5840 1 -- start start 2023-10-11T09:48:49.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.776+0000 7f02a6cf5840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eddeab2400 0x55edde811600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.776+0000 7f02a6cf5840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55eddeab2800 0x55edde811b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.776+0000 7f02a6cf5840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eddeab2c00 0x55edde812100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.776+0000 7f02a6cf5840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edde7a1e00 con 0x55eddeab2c00 2023-10-11T09:48:49.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.776+0000 7f02a6cf5840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edde7ea780 con 0x55eddeab2400 2023-10-11T09:48:49.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.776+0000 7f02a6cf5840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55edde7ea500 con 0x55eddeab2800 2023-10-11T09:48:49.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.776+0000 7f02a4cb0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eddeab2c00 0x55edde812100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.776+0000 7f02a4cb0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eddeab2c00 0x55edde812100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35798/0 (socket says 172.21.15.78:35798) 2023-10-11T09:48:49.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.776+0000 7f02a4cb0700 1 -- 172.21.15.78:0/3333995082 learned_addr learned my addr 172.21.15.78:0/3333995082 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:49.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.776+0000 7f02a54b1700 1 --2- 172.21.15.78:0/3333995082 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55eddeab2800 0x55edde811b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.776+0000 7f02a5cb2700 1 --2- 172.21.15.78:0/3333995082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eddeab2400 0x55edde811600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.776+0000 7f02a4cb0700 1 -- 172.21.15.78:0/3333995082 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55eddeab2800 msgr2=0x55edde811b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:49.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.776+0000 7f02a4cb0700 1 --2- 172.21.15.78:0/3333995082 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55eddeab2800 0x55edde811b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.776+0000 7f02a4cb0700 1 -- 172.21.15.78:0/3333995082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eddeab2400 msgr2=0x55edde811600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:49.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.776+0000 7f02a4cb0700 1 --2- 172.21.15.78:0/3333995082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eddeab2400 0x55edde811600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.776+0000 7f02a4cb0700 1 -- 172.21.15.78:0/3333995082 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55edde78cea0 con 0x55eddeab2c00 2023-10-11T09:48:49.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.776+0000 7f02a5cb2700 1 --2- 172.21.15.78:0/3333995082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eddeab2400 0x55edde811600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:48:49.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.776+0000 7f02a4cb0700 1 --2- 172.21.15.78:0/3333995082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eddeab2c00 0x55edde812100 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x55eddf5529f0 tx=0x55eddf550120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e5c89dd217a38c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.776+0000 7f02a0ca8700 1 -- 172.21.15.78:0/3333995082 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55edde79af20 con 0x55eddeab2c00 2023-10-11T09:48:49.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.776+0000 7f02a0ca8700 1 -- 172.21.15.78:0/3333995082 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eddf55b080 con 0x55eddeab2c00 2023-10-11T09:48:49.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.776+0000 7f02a0ca8700 1 -- 172.21.15.78:0/3333995082 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55edde79b600 con 0x55eddeab2c00 2023-10-11T09:48:49.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.780+0000 7f02a6cf5840 1 -- 172.21.15.78:0/3333995082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eddeab2c00 msgr2=0x55edde812100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.780+0000 7f02a6cf5840 1 --2- 172.21.15.78:0/3333995082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eddeab2c00 0x55edde812100 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x55eddf5529f0 tx=0x55eddf550120 comp rx=0 tx=0).stop 2023-10-11T09:48:49.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.780+0000 7f02a6cf5840 1 -- 172.21.15.78:0/3333995082 shutdown_connections 2023-10-11T09:48:49.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.780+0000 7f02a6cf5840 1 --2- 172.21.15.78:0/3333995082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eddeab2c00 0x55edde812100 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.780+0000 7f02a6cf5840 1 --2- 172.21.15.78:0/3333995082 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55eddeab2800 0x55edde811b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.780+0000 7f02a6cf5840 1 --2- 172.21.15.78:0/3333995082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eddeab2400 0x55edde811600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.780+0000 7f02a6cf5840 1 -- 172.21.15.78:0/3333995082 >> 172.21.15.78:0/3333995082 conn(0x55eddeab2000 msgr2=0x55edde85b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:49.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.780+0000 7f02a6cf5840 1 -- 172.21.15.78:0/3333995082 shutdown_connections 2023-10-11T09:48:49.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.780+0000 7f02a6cf5840 1 -- 172.21.15.78:0/3333995082 wait complete. 2023-10-11T09:48:49.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.780+0000 7f02a6cf5840 1 Processor -- start 2023-10-11T09:48:49.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.780+0000 7f02a6cf5840 1 -- start start 2023-10-11T09:48:49.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.780+0000 7f02a6cf5840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55eddeab2400 0x55edde813180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.780+0000 7f02a6cf5840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eddeab2800 0x55edde812c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.780+0000 7f02a6cf5840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eddeab2c00 0x55edde812680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.780+0000 7f02a6cf5840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edde7ea3c0 con 0x55eddeab2c00 2023-10-11T09:48:49.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.780+0000 7f02a6cf5840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edde7ea280 con 0x55eddeab2800 2023-10-11T09:48:49.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.780+0000 7f02a6cf5840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55edde7ea140 con 0x55eddeab2400 2023-10-11T09:48:49.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.780+0000 7f02a4cb0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eddeab2c00 0x55edde812680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.780+0000 7f02a4cb0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eddeab2c00 0x55edde812680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35812/0 (socket says 172.21.15.78:35812) 2023-10-11T09:48:49.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.780+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 learned_addr learned my addr 172.21.15.78:0/2381226368 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:49.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.780+0000 7f02a5cb2700 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55eddeab2400 0x55edde813180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.780+0000 7f02a54b1700 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eddeab2800 0x55edde812c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.780+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55eddeab2400 msgr2=0x55edde813180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.780+0000 7f02a4cb0700 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55eddeab2400 0x55edde813180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.780+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eddeab2800 msgr2=0x55edde812c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.780+0000 7f02a4cb0700 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eddeab2800 0x55edde812c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.780+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55edde78cb60 con 0x55eddeab2c00 2023-10-11T09:48:49.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.780+0000 7f02a54b1700 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eddeab2800 0x55edde812c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:48:49.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.780+0000 7f02a5cb2700 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55eddeab2400 0x55edde813180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:48:49.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.784+0000 7f02a4cb0700 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eddeab2c00 0x55edde812680 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x55eddf552c60 tx=0x55eddf556c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.784+0000 7f029f4a5700 1 -- 172.21.15.78:0/2381226368 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55edde79b1e0 con 0x55eddeab2c00 2023-10-11T09:48:49.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.784+0000 7f029f4a5700 1 -- 172.21.15.78:0/2381226368 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eddf55af00 con 0x55eddeab2c00 2023-10-11T09:48:49.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.784+0000 7f029f4a5700 1 -- 172.21.15.78:0/2381226368 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55edde79adc0 con 0x55eddeab2c00 2023-10-11T09:48:49.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.784+0000 7f02a6cf5840 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55edde78c340 con 0x55eddeab2c00 2023-10-11T09:48:49.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.784+0000 7f02a6cf5840 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55edde78d520 con 0x55eddeab2c00 2023-10-11T09:48:49.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.784+0000 7f02a6cf5840 4 datascan.init: resolving metadata pool 11 2023-10-11T09:48:49.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.784+0000 7f029f4a5700 1 -- 172.21.15.78:0/2381226368 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x55edde7c8840 con 0x55eddeab2c00 2023-10-11T09:48:49.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.788+0000 7f029f4a5700 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55eddeab3800 0x55edde813700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.788+0000 7f029f4a5700 1 -- 172.21.15.78:0/2381226368 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6638+0+0 (secure 0 0 0) 0x55edde7a4a80 con 0x55eddeab2c00 2023-10-11T09:48:49.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.788+0000 7f02a6cf5840 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-10-11T09:48:49.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.788+0000 7f02a6cf5840 4 datascan.main: resolving metadata pool 11 2023-10-11T09:48:49.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.788+0000 7f02a5cb2700 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55eddeab3800 0x55edde813700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.788+0000 7f02a6cf5840 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55eddeab3c00 0x55edde812100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.788+0000 7f02a6cf5840 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:1 11.0 11:00000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc4000 con 0x55eddeab3c00 2023-10-11T09:48:49.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.788+0000 7f02a54b1700 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55eddeab3c00 0x55edde812100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.788+0000 7f02a5cb2700 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55eddeab3800 0x55edde813700 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x55edde8789f0 tx=0x55eddf556cc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.788+0000 7f02a54b1700 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55eddeab3c00 0x55edde812100 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.788+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 <== osd.1 v2:172.21.15.78:6809/35734 1 ==== osd_op_reply(1 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f2fc0 con 0x55eddeab3c00 2023-10-11T09:48:49.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.788+0000 7f02a54b1700 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55eddebc4000 0x55eddebce000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.788+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:2 11.10 11:08000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc5800 con 0x55eddebc4000 2023-10-11T09:48:49.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.788+0000 7f02a5cb2700 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55eddebc4000 0x55eddebce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.788+0000 7f02a5cb2700 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55eddebc4000 0x55eddebce000 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.792+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 <== osd.0 v2:172.21.15.78:6800/35735 1 ==== osd_op_reply(2 [pgnls start_epoch 88 out=81b] v88'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55eddf5f3200 con 0x55eddebc4000 2023-10-11T09:48:49.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.792+0000 7f02a5cb2700 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55eddebc5800 0x55eddebce580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.792+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:3 11.8 11:10000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc5c00 con 0x55eddebc5800 2023-10-11T09:48:49.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.792+0000 7f02a54b1700 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55eddebc5800 0x55eddebce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.792+0000 7f02a54b1700 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55eddebc5800 0x55eddebce580 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.792+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(3 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f2fc0 con 0x55eddebc5800 2023-10-11T09:48:49.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.792+0000 7f02a54b1700 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55eddebc5c00 0x55eddebceb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.792+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:4 11.18 11:18000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebd8000 con 0x55eddebc5c00 2023-10-11T09:48:49.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.792+0000 7f02a4cb0700 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55eddebc5c00 0x55eddebceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.792+0000 7f02a4cb0700 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55eddebc5c00 0x55eddebceb00 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.792+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 <== osd.2 v2:172.21.15.78:6808/35732 1 ==== osd_op_reply(4 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f3440 con 0x55eddebc5c00 2023-10-11T09:48:49.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.792+0000 7f02a4cb0700 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55eddebd8000 0x55eddebcf600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.792+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:5 11.4 11:20000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddeab3400 con 0x55eddebd8000 2023-10-11T09:48:49.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.796+0000 7f02a5cb2700 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55eddebd8000 0x55eddebcf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.796+0000 7f02a5cb2700 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55eddebd8000 0x55eddebcf600 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.796+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 <== osd.7 v2:172.21.15.133:6824/35901 1 ==== osd_op_reply(5 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f3200 con 0x55eddebd8000 2023-10-11T09:48:49.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.796+0000 7f02a5cb2700 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55eddeab3400 0x55eddebcfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.796+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:6 11.14 11:28000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebd8800 con 0x55eddeab3400 2023-10-11T09:48:49.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.796+0000 7f02a54b1700 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55eddeab3400 0x55eddebcfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.800+0000 7f02a54b1700 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55eddeab3400 0x55eddebcfb80 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.800+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== osd_op_reply(6 [pgnls start_epoch 88 out=125b] v88'29 uv29 ondisk = 1) v8 ==== 144+0+125 (crc 0 0 0) 0x55eddf5f2fc0 con 0x55eddeab3400 2023-10-11T09:48:49.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.800+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:7 11.c 11:30000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebd8800 con 0x55eddeab3c00 2023-10-11T09:48:49.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.800+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 <== osd.1 v2:172.21.15.78:6809/35734 2 ==== osd_op_reply(7 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f2fc0 con 0x55eddeab3c00 2023-10-11T09:48:49.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.800+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:8 11.1c 11:38000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebd8800 con 0x55eddeab3400 2023-10-11T09:48:49.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.800+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 <== osd.6 v2:172.21.15.133:6800/35900 2 ==== osd_op_reply(8 [pgnls start_epoch 88 out=49b] v87'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f2fc0 con 0x55eddeab3400 2023-10-11T09:48:49.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.800+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:9 11.2 11:40000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebd8800 con 0x55eddebc5c00 2023-10-11T09:48:49.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.800+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 <== osd.2 v2:172.21.15.78:6808/35732 2 ==== osd_op_reply(9 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f3440 con 0x55eddebc5c00 2023-10-11T09:48:49.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.800+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:10 11.12 11:48000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebd8800 con 0x55eddebd8000 2023-10-11T09:48:49.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.800+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 <== osd.7 v2:172.21.15.133:6824/35901 2 ==== osd_op_reply(10 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f3200 con 0x55eddebd8000 2023-10-11T09:48:49.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.804+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:11 11.a 11:50000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebd8800 con 0x55eddebc5800 2023-10-11T09:48:49.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.804+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 <== osd.3 v2:172.21.15.78:6824/35738 2 ==== osd_op_reply(11 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f2fc0 con 0x55eddebc5800 2023-10-11T09:48:49.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.804+0000 7f02a54b1700 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55eddebd8800 0x55eddebcf080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.804+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:12 11.1a 11:58000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebd8400 con 0x55eddebd8800 2023-10-11T09:48:49.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.804+0000 7f02a4cb0700 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55eddebd8800 0x55eddebcf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.804+0000 7f02a4cb0700 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55eddebd8800 0x55eddebcf080 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.804+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 <== osd.5 v2:172.21.15.133:6816/35899 1 ==== osd_op_reply(12 [pgnls start_epoch 88 out=49b] v88'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f3440 con 0x55eddebd8800 2023-10-11T09:48:49.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.804+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:13 11.6 11:60000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebd8400 con 0x55eddebc4000 2023-10-11T09:48:49.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.808+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 <== osd.0 v2:172.21.15.78:6800/35735 2 ==== osd_op_reply(13 [pgnls start_epoch 88 out=49b] v87'5 uv5 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f3200 con 0x55eddebc4000 2023-10-11T09:48:49.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.808+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:14 11.16 11:68000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebd8400 con 0x55eddebd8800 2023-10-11T09:48:49.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.808+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 <== osd.5 v2:172.21.15.133:6816/35899 2 ==== osd_op_reply(14 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f3440 con 0x55eddebd8800 2023-10-11T09:48:49.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.808+0000 7f02a4cb0700 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55eddebd8400 0x55eddebd0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:49.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.808+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:15 11.e 11:70000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebd9000 con 0x55eddebd8400 2023-10-11T09:48:49.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.808+0000 7f02a5cb2700 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55eddebd8400 0x55eddebd0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:49.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.808+0000 7f02a5cb2700 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55eddebd8400 0x55eddebd0680 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:49.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.808+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 <== osd.4 v2:172.21.15.133:6808/35898 1 ==== osd_op_reply(15 [pgnls start_epoch 88 out=49b] v87'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f3200 con 0x55eddebd8400 2023-10-11T09:48:49.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.808+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:16 11.1e 11:78000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebd9000 con 0x55eddebd8000 2023-10-11T09:48:49.818 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.812+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 <== osd.7 v2:172.21.15.133:6824/35901 3 ==== osd_op_reply(16 [pgnls start_epoch 88 out=49b] v88'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f3200 con 0x55eddebd8000 2023-10-11T09:48:49.818 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.812+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:17 11.1 11:80000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebd9000 con 0x55eddeab3400 2023-10-11T09:48:49.819 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.812+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 <== osd.6 v2:172.21.15.133:6800/35900 3 ==== osd_op_reply(17 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f2fc0 con 0x55eddeab3400 2023-10-11T09:48:49.819 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.812+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:18 11.11 11:88000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebd9000 con 0x55eddebc5c00 2023-10-11T09:48:49.819 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.812+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 <== osd.2 v2:172.21.15.78:6808/35732 3 ==== osd_op_reply(18 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f3440 con 0x55eddebc5c00 2023-10-11T09:48:49.819 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.812+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:19 11.9 11:90000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebd9000 con 0x55eddebc5800 2023-10-11T09:48:49.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.812+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 <== osd.3 v2:172.21.15.78:6824/35738 3 ==== osd_op_reply(19 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f2fc0 con 0x55eddebc5800 2023-10-11T09:48:49.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.812+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:20 11.19 11:98000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebd9000 con 0x55eddebc4000 2023-10-11T09:48:49.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.816+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 <== osd.0 v2:172.21.15.78:6800/35735 3 ==== osd_op_reply(20 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f3200 con 0x55eddebc4000 2023-10-11T09:48:49.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.816+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:21 11.5 11:a0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebd9000 con 0x55eddebd8000 2023-10-11T09:48:49.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.816+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 <== osd.7 v2:172.21.15.133:6824/35901 4 ==== osd_op_reply(21 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f3200 con 0x55eddebd8000 2023-10-11T09:48:49.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.816+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:22 11.15 11:a8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebd9000 con 0x55eddebc5c00 2023-10-11T09:48:49.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.816+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 <== osd.2 v2:172.21.15.78:6808/35732 4 ==== osd_op_reply(22 [pgnls start_epoch 88 out=73b] v87'6 uv6 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55eddf5f3440 con 0x55eddebc5c00 2023-10-11T09:48:49.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.816+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:23 11.d 11:b0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebd9000 con 0x55eddebc5c00 2023-10-11T09:48:49.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.816+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 <== osd.2 v2:172.21.15.78:6808/35732 5 ==== osd_op_reply(23 [pgnls start_epoch 88 out=74b] v88'3 uv3 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55eddf5f3440 con 0x55eddebc5c00 2023-10-11T09:48:49.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.816+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:24 11.1d 11:b8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebd9000 con 0x55eddebd8000 2023-10-11T09:48:49.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.816+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 <== osd.7 v2:172.21.15.133:6824/35901 5 ==== osd_op_reply(24 [pgnls start_epoch 88 out=49b] v87'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f3200 con 0x55eddebd8000 2023-10-11T09:48:49.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.816+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:25 11.3 11:c0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebd9000 con 0x55eddeab3400 2023-10-11T09:48:49.824 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.820+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 <== osd.6 v2:172.21.15.133:6800/35900 4 ==== osd_op_reply(25 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f2fc0 con 0x55eddeab3400 2023-10-11T09:48:49.824 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.820+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:26 11.13 11:c8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebd9000 con 0x55eddebd8800 2023-10-11T09:48:49.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.820+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 <== osd.5 v2:172.21.15.133:6816/35899 3 ==== osd_op_reply(26 [pgnls start_epoch 88 out=98b] v88'7 uv7 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55eddf5f3440 con 0x55eddebd8800 2023-10-11T09:48:49.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.820+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:27 11.b 11:d0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebd9000 con 0x55eddebc5800 2023-10-11T09:48:49.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.820+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 <== osd.3 v2:172.21.15.78:6824/35738 4 ==== osd_op_reply(27 [pgnls start_epoch 88 out=76b] v88'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55eddf5f2fc0 con 0x55eddebc5800 2023-10-11T09:48:49.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.820+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:28 11.1b 11:d8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebd9000 con 0x55eddebd8400 2023-10-11T09:48:49.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.820+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 <== osd.4 v2:172.21.15.133:6808/35898 2 ==== osd_op_reply(28 [pgnls start_epoch 88 out=49b] v88'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f3200 con 0x55eddebd8400 2023-10-11T09:48:49.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.820+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:29 11.7 11:e0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebd9000 con 0x55eddebd8800 2023-10-11T09:48:49.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.824+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 <== osd.5 v2:172.21.15.133:6816/35899 4 ==== osd_op_reply(29 [pgnls start_epoch 88 out=49b] v87'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f3440 con 0x55eddebd8800 2023-10-11T09:48:49.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.824+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:30 11.17 11:e8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebd9000 con 0x55eddebd8000 2023-10-11T09:48:49.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.824+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 <== osd.7 v2:172.21.15.133:6824/35901 6 ==== osd_op_reply(30 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f3200 con 0x55eddebd8000 2023-10-11T09:48:49.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.824+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:31 11.f 11:f0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebd9000 con 0x55eddebd8000 2023-10-11T09:48:49.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.824+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 <== osd.7 v2:172.21.15.133:6824/35901 7 ==== osd_op_reply(31 [pgnls start_epoch 88 out=73b] v88'4 uv4 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55eddf5f3200 con 0x55eddebd8000 2023-10-11T09:48:49.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.824+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:32 11.1f 11:f8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebd9000 con 0x55eddebc5c00 2023-10-11T09:48:49.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.824+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 <== osd.2 v2:172.21.15.78:6808/35732 6 ==== osd_op_reply(32 [pgnls start_epoch 88 out=101b] v88'12 uv12 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55eddf5f3440 con 0x55eddebc5c00 2023-10-11T09:48:49.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.824+0000 7f02a6cf5840 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2023-10-11T09:48:49.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.824+0000 7f02a6cf5840 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:33 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc4400 con 0x55eddebc4000 2023-10-11T09:48:49.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.828+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 <== osd.0 v2:172.21.15.78:6800/35735 4 ==== osd_op_reply(33 10000000000.00000000 [omap-get-vals out=509b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+509 (crc 0 0 0) 0x55eddf5f3200 con 0x55eddebc4000 2023-10-11T09:48:49.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.828+0000 7f02a6cf5840 20 datascan.scan_links: injected ino detected 2023-10-11T09:48:49.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.828+0000 7f02a6cf5840 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2023-10-11T09:48:49.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.828+0000 7f02a6cf5840 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-10-11T09:48:49.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.828+0000 7f02a6cf5840 10 datascan.scan_links: step 1: handling object 200.00000000 2023-10-11T09:48:49.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.828+0000 7f02a6cf5840 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-10-11T09:48:49.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.828+0000 7f02a6cf5840 10 datascan.scan_links: step 1: handling object 200.00000001 2023-10-11T09:48:49.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.828+0000 7f02a6cf5840 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-10-11T09:48:49.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.828+0000 7f02a6cf5840 10 datascan.scan_links: step 1: handling object 500.00000000 2023-10-11T09:48:49.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.828+0000 7f02a6cf5840 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-10-11T09:48:49.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.828+0000 7f02a6cf5840 10 datascan.scan_links: step 1: handling object mds_snaptable 2023-10-11T09:48:49.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.828+0000 7f02a6cf5840 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-10-11T09:48:49.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.828+0000 7f02a6cf5840 10 datascan.scan_links: step 1: handling object mds0_inotable 2023-10-11T09:48:49.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.828+0000 7f02a6cf5840 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-10-11T09:48:49.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.828+0000 7f02a6cf5840 10 datascan.scan_links: step 1: handling object 100.00000000 2023-10-11T09:48:49.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.828+0000 7f02a6cf5840 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:34 11.13 11:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc4800 con 0x55eddebd8800 2023-10-11T09:48:49.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.828+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 <== osd.5 v2:172.21.15.133:6816/35899 5 ==== osd_op_reply(34 100.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55eddf5f3440 con 0x55eddebd8800 2023-10-11T09:48:49.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.828+0000 7f02a6cf5840 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2023-10-11T09:48:49.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.828+0000 7f02a6cf5840 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-10-11T09:48:49.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.828+0000 7f02a6cf5840 10 datascan.scan_links: step 1: handling object 400.00000000 2023-10-11T09:48:49.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.828+0000 7f02a6cf5840 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-10-11T09:48:49.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.828+0000 7f02a6cf5840 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2023-10-11T09:48:49.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.828+0000 7f02a6cf5840 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-10-11T09:48:49.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.828+0000 7f02a6cf5840 10 datascan.scan_links: step 1: handling object 1.00000000 2023-10-11T09:48:49.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.828+0000 7f02a6cf5840 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:35 11.1f 11:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc4c00 con 0x55eddebc5c00 2023-10-11T09:48:49.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.828+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 <== osd.2 v2:172.21.15.78:6808/35732 7 ==== osd_op_reply(35 1.00000000 [omap-get-vals out=508b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+508 (crc 0 0 0) 0x55eddf5f3440 con 0x55eddebc5c00 2023-10-11T09:48:49.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.828+0000 7f02a6cf5840 20 datascan.scan_links: injected ino detected 2023-10-11T09:48:49.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.828+0000 7f02a6cf5840 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:36 11.0 11:00000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc5000 con 0x55eddeab3c00 2023-10-11T09:48:49.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.832+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 <== osd.1 v2:172.21.15.78:6809/35734 3 ==== osd_op_reply(36 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f2fc0 con 0x55eddeab3c00 2023-10-11T09:48:49.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.832+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:37 11.10 11:08000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc5000 con 0x55eddebc4000 2023-10-11T09:48:49.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.832+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 <== osd.0 v2:172.21.15.78:6800/35735 5 ==== osd_op_reply(37 [pgnls start_epoch 88 out=81b] v88'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55eddf5f3200 con 0x55eddebc4000 2023-10-11T09:48:49.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.832+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:38 11.8 11:10000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc5000 con 0x55eddebc5800 2023-10-11T09:48:49.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.832+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 <== osd.3 v2:172.21.15.78:6824/35738 5 ==== osd_op_reply(38 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f2fc0 con 0x55eddebc5800 2023-10-11T09:48:49.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.832+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:39 11.18 11:18000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc5000 con 0x55eddebc5c00 2023-10-11T09:48:49.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.832+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 <== osd.2 v2:172.21.15.78:6808/35732 8 ==== osd_op_reply(39 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f3440 con 0x55eddebc5c00 2023-10-11T09:48:49.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.832+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:40 11.4 11:20000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc5000 con 0x55eddebd8000 2023-10-11T09:48:49.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.832+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 <== osd.7 v2:172.21.15.133:6824/35901 8 ==== osd_op_reply(40 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f3200 con 0x55eddebd8000 2023-10-11T09:48:49.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.832+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:41 11.14 11:28000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc5000 con 0x55eddeab3400 2023-10-11T09:48:49.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.836+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 <== osd.6 v2:172.21.15.133:6800/35900 5 ==== osd_op_reply(41 [pgnls start_epoch 88 out=125b] v88'29 uv29 ondisk = 1) v8 ==== 144+0+125 (crc 0 0 0) 0x55eddf5f2fc0 con 0x55eddeab3400 2023-10-11T09:48:49.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.836+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:42 11.c 11:30000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc5000 con 0x55eddeab3c00 2023-10-11T09:48:49.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.836+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 <== osd.1 v2:172.21.15.78:6809/35734 4 ==== osd_op_reply(42 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f2fc0 con 0x55eddeab3c00 2023-10-11T09:48:49.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.836+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:43 11.1c 11:38000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc5000 con 0x55eddeab3400 2023-10-11T09:48:49.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.836+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 <== osd.6 v2:172.21.15.133:6800/35900 6 ==== osd_op_reply(43 [pgnls start_epoch 88 out=49b] v87'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f2fc0 con 0x55eddeab3400 2023-10-11T09:48:49.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.836+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:44 11.2 11:40000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc5000 con 0x55eddebc5c00 2023-10-11T09:48:49.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.836+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 <== osd.2 v2:172.21.15.78:6808/35732 9 ==== osd_op_reply(44 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f3440 con 0x55eddebc5c00 2023-10-11T09:48:49.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.836+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:45 11.12 11:48000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc5000 con 0x55eddebd8000 2023-10-11T09:48:49.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.840+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 <== osd.7 v2:172.21.15.133:6824/35901 9 ==== osd_op_reply(45 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f3200 con 0x55eddebd8000 2023-10-11T09:48:49.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.840+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:46 11.a 11:50000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc5000 con 0x55eddebc5800 2023-10-11T09:48:49.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.840+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 <== osd.3 v2:172.21.15.78:6824/35738 6 ==== osd_op_reply(46 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f2fc0 con 0x55eddebc5800 2023-10-11T09:48:49.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.840+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:47 11.1a 11:58000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc5000 con 0x55eddebd8800 2023-10-11T09:48:49.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.840+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 <== osd.5 v2:172.21.15.133:6816/35899 6 ==== osd_op_reply(47 [pgnls start_epoch 88 out=49b] v88'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f3440 con 0x55eddebd8800 2023-10-11T09:48:49.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.840+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:48 11.6 11:60000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc5000 con 0x55eddebc4000 2023-10-11T09:48:49.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.840+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 <== osd.0 v2:172.21.15.78:6800/35735 6 ==== osd_op_reply(48 [pgnls start_epoch 88 out=49b] v87'5 uv5 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f3200 con 0x55eddebc4000 2023-10-11T09:48:49.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.840+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:49 11.16 11:68000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc5000 con 0x55eddebd8800 2023-10-11T09:48:49.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.844+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 <== osd.5 v2:172.21.15.133:6816/35899 7 ==== osd_op_reply(49 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f3440 con 0x55eddebd8800 2023-10-11T09:48:49.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.844+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:50 11.e 11:70000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc5000 con 0x55eddebd8400 2023-10-11T09:48:49.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.844+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 <== osd.4 v2:172.21.15.133:6808/35898 3 ==== osd_op_reply(50 [pgnls start_epoch 88 out=49b] v87'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f3200 con 0x55eddebd8400 2023-10-11T09:48:49.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.844+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:51 11.1e 11:78000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc5000 con 0x55eddebd8000 2023-10-11T09:48:49.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.844+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 <== osd.7 v2:172.21.15.133:6824/35901 10 ==== osd_op_reply(51 [pgnls start_epoch 88 out=49b] v88'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f3200 con 0x55eddebd8000 2023-10-11T09:48:49.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.844+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:52 11.1 11:80000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc5000 con 0x55eddeab3400 2023-10-11T09:48:49.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.844+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 <== osd.6 v2:172.21.15.133:6800/35900 7 ==== osd_op_reply(52 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f2fc0 con 0x55eddeab3400 2023-10-11T09:48:49.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.844+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:53 11.11 11:88000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc5000 con 0x55eddebc5c00 2023-10-11T09:48:49.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.848+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 <== osd.2 v2:172.21.15.78:6808/35732 10 ==== osd_op_reply(53 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f3440 con 0x55eddebc5c00 2023-10-11T09:48:49.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.848+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:54 11.9 11:90000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc5000 con 0x55eddebc5800 2023-10-11T09:48:49.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.848+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 <== osd.3 v2:172.21.15.78:6824/35738 7 ==== osd_op_reply(54 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f2fc0 con 0x55eddebc5800 2023-10-11T09:48:49.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.848+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:55 11.19 11:98000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc5000 con 0x55eddebc4000 2023-10-11T09:48:49.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.848+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 <== osd.0 v2:172.21.15.78:6800/35735 7 ==== osd_op_reply(55 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f3200 con 0x55eddebc4000 2023-10-11T09:48:49.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.848+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:56 11.5 11:a0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc5000 con 0x55eddebd8000 2023-10-11T09:48:49.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.848+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 <== osd.7 v2:172.21.15.133:6824/35901 11 ==== osd_op_reply(56 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f3200 con 0x55eddebd8000 2023-10-11T09:48:49.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.848+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:57 11.15 11:a8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc5000 con 0x55eddebc5c00 2023-10-11T09:48:49.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.848+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 <== osd.2 v2:172.21.15.78:6808/35732 11 ==== osd_op_reply(57 [pgnls start_epoch 88 out=73b] v87'6 uv6 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55eddf5f3440 con 0x55eddebc5c00 2023-10-11T09:48:49.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.848+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:58 11.d 11:b0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc5000 con 0x55eddebc5c00 2023-10-11T09:48:49.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.852+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 <== osd.2 v2:172.21.15.78:6808/35732 12 ==== osd_op_reply(58 [pgnls start_epoch 88 out=74b] v88'3 uv3 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55eddf5f3440 con 0x55eddebc5c00 2023-10-11T09:48:49.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.852+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:59 11.1d 11:b8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc5000 con 0x55eddebd8000 2023-10-11T09:48:49.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.852+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 <== osd.7 v2:172.21.15.133:6824/35901 12 ==== osd_op_reply(59 [pgnls start_epoch 88 out=49b] v87'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f3200 con 0x55eddebd8000 2023-10-11T09:48:49.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.852+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:60 11.3 11:c0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc5000 con 0x55eddeab3400 2023-10-11T09:48:49.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.852+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 <== osd.6 v2:172.21.15.133:6800/35900 8 ==== osd_op_reply(60 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f2fc0 con 0x55eddeab3400 2023-10-11T09:48:49.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.852+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:61 11.13 11:c8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc5000 con 0x55eddebd8800 2023-10-11T09:48:49.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.852+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 <== osd.5 v2:172.21.15.133:6816/35899 8 ==== osd_op_reply(61 [pgnls start_epoch 88 out=98b] v88'7 uv7 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55eddf5f3440 con 0x55eddebd8800 2023-10-11T09:48:49.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.852+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:62 11.b 11:d0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc5000 con 0x55eddebc5800 2023-10-11T09:48:49.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.856+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 <== osd.3 v2:172.21.15.78:6824/35738 8 ==== osd_op_reply(62 [pgnls start_epoch 88 out=76b] v88'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55eddf5f2fc0 con 0x55eddebc5800 2023-10-11T09:48:49.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.856+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:63 11.1b 11:d8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc5000 con 0x55eddebd8400 2023-10-11T09:48:49.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.856+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 <== osd.4 v2:172.21.15.133:6808/35898 4 ==== osd_op_reply(63 [pgnls start_epoch 88 out=49b] v88'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f3200 con 0x55eddebd8400 2023-10-11T09:48:49.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.856+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:64 11.7 11:e0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc5000 con 0x55eddebd8800 2023-10-11T09:48:49.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.856+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 <== osd.5 v2:172.21.15.133:6816/35899 9 ==== osd_op_reply(64 [pgnls start_epoch 88 out=49b] v87'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f3440 con 0x55eddebd8800 2023-10-11T09:48:49.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.856+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:65 11.17 11:e8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc5000 con 0x55eddebd8000 2023-10-11T09:48:49.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.856+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 <== osd.7 v2:172.21.15.133:6824/35901 13 ==== osd_op_reply(65 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eddf5f3200 con 0x55eddebd8000 2023-10-11T09:48:49.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.856+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:66 11.f 11:f0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc5000 con 0x55eddebd8000 2023-10-11T09:48:49.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 <== osd.7 v2:172.21.15.133:6824/35901 14 ==== osd_op_reply(66 [pgnls start_epoch 88 out=73b] v88'4 uv4 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55eddf5f3200 con 0x55eddebd8000 2023-10-11T09:48:49.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:67 11.1f 11:f8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc5000 con 0x55eddebc5c00 2023-10-11T09:48:49.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 <== osd.2 v2:172.21.15.78:6808/35732 13 ==== osd_op_reply(67 [pgnls start_epoch 88 out=101b] v88'12 uv12 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55eddf5f3440 con 0x55eddebc5c00 2023-10-11T09:48:49.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a6cf5840 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2023-10-11T09:48:49.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a6cf5840 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:68 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc5400 con 0x55eddebc4000 2023-10-11T09:48:49.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 <== osd.0 v2:172.21.15.78:6800/35735 8 ==== osd_op_reply(68 10000000000.00000000 [omap-get-vals out=509b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+509 (crc 0 0 0) 0x55eddf5f3200 con 0x55eddebc4000 2023-10-11T09:48:49.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a6cf5840 20 datascan.scan_links: injected ino detected 2023-10-11T09:48:49.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a6cf5840 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2023-10-11T09:48:49.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a6cf5840 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2023-10-11T09:48:49.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a6cf5840 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-10-11T09:48:49.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a6cf5840 10 datascan.scan_links: step 2: handling object 200.00000000 2023-10-11T09:48:49.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a6cf5840 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-10-11T09:48:49.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a6cf5840 10 datascan.scan_links: step 2: handling object 200.00000001 2023-10-11T09:48:49.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a6cf5840 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-10-11T09:48:49.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a6cf5840 10 datascan.scan_links: step 2: handling object 500.00000000 2023-10-11T09:48:49.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a6cf5840 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-10-11T09:48:49.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a6cf5840 10 datascan.scan_links: step 2: handling object mds_snaptable 2023-10-11T09:48:49.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a6cf5840 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-10-11T09:48:49.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a6cf5840 10 datascan.scan_links: step 2: handling object mds0_inotable 2023-10-11T09:48:49.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a6cf5840 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-10-11T09:48:49.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a6cf5840 10 datascan.scan_links: step 2: handling object 100.00000000 2023-10-11T09:48:49.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a6cf5840 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:69 11.13 11:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebd9800 con 0x55eddebd8800 2023-10-11T09:48:49.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 <== osd.5 v2:172.21.15.133:6816/35899 10 ==== osd_op_reply(69 100.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55eddf5f3440 con 0x55eddebd8800 2023-10-11T09:48:49.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a6cf5840 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2023-10-11T09:48:49.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a6cf5840 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-10-11T09:48:49.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a6cf5840 10 datascan.scan_links: step 2: handling object 400.00000000 2023-10-11T09:48:49.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a6cf5840 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-10-11T09:48:49.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a6cf5840 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2023-10-11T09:48:49.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a6cf5840 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-10-11T09:48:49.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a6cf5840 10 datascan.scan_links: step 2: handling object 1.00000000 2023-10-11T09:48:49.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a6cf5840 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:70 11.1f 11:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebd9400 con 0x55eddebc5c00 2023-10-11T09:48:49.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 <== osd.2 v2:172.21.15.78:6808/35732 14 ==== osd_op_reply(70 1.00000000 [omap-get-vals out=508b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+508 (crc 0 0 0) 0x55eddf5f3440 con 0x55eddebc5c00 2023-10-11T09:48:49.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a6cf5840 20 datascan.scan_links: injected ino detected 2023-10-11T09:48:49.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a6cf5840 20 datascan.scan_links: adding 0x10000000000 for future processing to fix dnfirst 2023-10-11T09:48:49.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a6cf5840 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2023-10-11T09:48:49.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a6cf5840 10 datascan.scan_links: removing dup dentries from 0 objects 2023-10-11T09:48:49.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a6cf5840 10 datascan.scan_links: processing 0 bad_nlink_inos 2023-10-11T09:48:49.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a6cf5840 10 datascan.scan_links: processing 2 injected_inos 2023-10-11T09:48:49.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a6cf5840 10 datascan.scan_links: handling injected_ino 0x10000000000 2023-10-11T09:48:49.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a6cf5840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x1:subdir 2023-10-11T09:48:49.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.860+0000 7f02a6cf5840 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:71 11.1f 11:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebd9000 con 0x55eddebc5c00 2023-10-11T09:48:49.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.864+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 <== osd.2 v2:172.21.15.78:6808/35732 15 ==== osd_op_reply(71 1.00000000 [omap-get-vals-by-keys out=507b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+507 (crc 0 0 0) 0x55eddf5f3440 con 0x55eddebc5c00 2023-10-11T09:48:49.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.864+0000 7f02a6cf5840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:48:49.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.864+0000 7f02a6cf5840 20 datascan.scan_links: first is now 2 2023-10-11T09:48:49.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.864+0000 7f02a6cf5840 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:72 11.1f 11:ff5b34d6:::1.00000000:head [omap-set-vals in=507b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebd9c00 con 0x55eddebc5c00 2023-10-11T09:48:49.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.864+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 <== osd.2 v2:172.21.15.78:6808/35732 16 ==== osd_op_reply(72 1.00000000 [omap-set-vals] v88'13 uv13 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x55eddf5f3440 con 0x55eddebc5c00 2023-10-11T09:48:49.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.864+0000 7f02a6cf5840 20 datascan.inject_linkage: Injected dentry 0x0x1/subdir pointing to 0x0x10000000000 2023-10-11T09:48:49.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.864+0000 7f02a6cf5840 10 datascan.scan_links: handling injected_ino 0x10000000001 2023-10-11T09:48:49.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.864+0000 7f02a6cf5840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:sixmegs 2023-10-11T09:48:49.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.864+0000 7f02a6cf5840 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:73 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc4800 con 0x55eddebc4000 2023-10-11T09:48:49.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.864+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 <== osd.0 v2:172.21.15.78:6800/35735 9 ==== osd_op_reply(73 10000000000.00000000 [omap-get-vals-by-keys out=508b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+508 (crc 0 0 0) 0x55eddf5f3200 con 0x55eddebc4000 2023-10-11T09:48:49.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.864+0000 7f02a6cf5840 20 datascan.read_dentry: oid=10000000000.00000000 dname=sixmegs frag=*, r=0 2023-10-11T09:48:49.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.864+0000 7f02a6cf5840 20 datascan.scan_links: first is now 2 2023-10-11T09:48:49.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.864+0000 7f02a6cf5840 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:74 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=508b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc4c00 con 0x55eddebc4000 2023-10-11T09:48:49.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.868+0000 7f02a5cb2700 1 -- 172.21.15.78:0/2381226368 <== osd.0 v2:172.21.15.78:6800/35735 10 ==== osd_op_reply(74 10000000000.00000000 [omap-set-vals] v88'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55eddf5f3200 con 0x55eddebc4000 2023-10-11T09:48:49.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.868+0000 7f02a6cf5840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/sixmegs pointing to 0x0x10000000001 2023-10-11T09:48:49.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.868+0000 7f02a6cf5840 10 datascan.scan_links: updating inotable 2023-10-11T09:48:49.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.868+0000 7f02a6cf5840 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:75 11.13 11:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc4400 con 0x55eddebd8800 2023-10-11T09:48:49.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.872+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 <== osd.5 v2:172.21.15.133:6816/35899 11 ==== osd_op_reply(75 mds0_inotable [read 0~34 out=34b] v0'0 uv6 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x55eddf5f3440 con 0x55eddebd8800 2023-10-11T09:48:49.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.872+0000 7f02a6cf5840 10 mds.0.inotable: skip_inos was [0x10000000000~0x10000000000] 2023-10-11T09:48:49.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.872+0000 7f02a6cf5840 10 mds.0.inotable: skip_inos now [0x10000000002~0xfffffffffe] 2023-10-11T09:48:49.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.872+0000 7f02a6cf5840 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:76 11.13 11:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc5400 con 0x55eddebd8800 2023-10-11T09:48:49.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.872+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 <== osd.5 v2:172.21.15.133:6816/35899 12 ==== osd_op_reply(76 mds0_inotable [writefull 0~34] v88'8 uv8 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x55eddf5f3440 con 0x55eddebd8800 2023-10-11T09:48:49.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.872+0000 7f02a6cf5840 10 datascan.scan_links: updating snaptable 2023-10-11T09:48:49.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.872+0000 7f02a6cf5840 1 -- 172.21.15.78:0/2381226368 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:77 11.d 11:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x55eddebc4800 con 0x55eddebc5c00 2023-10-11T09:48:49.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a4cb0700 1 -- 172.21.15.78:0/2381226368 <== osd.2 v2:172.21.15.78:6808/35732 17 ==== osd_op_reply(77 mds_snaptable [read 0~70 out=70b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x55eddf5f3440 con 0x55eddebc5c00 2023-10-11T09:48:49.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a6cf5840 1 -- 172.21.15.78:0/2381226368 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55eddebc4000 msgr2=0x55eddebce000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a6cf5840 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55eddebc4000 0x55eddebce000 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a6cf5840 1 -- 172.21.15.78:0/2381226368 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55eddeab3c00 msgr2=0x55edde812100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a6cf5840 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55eddeab3c00 0x55edde812100 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a6cf5840 1 -- 172.21.15.78:0/2381226368 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55eddebc5c00 msgr2=0x55eddebceb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a6cf5840 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55eddebc5c00 0x55eddebceb00 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a6cf5840 1 -- 172.21.15.78:0/2381226368 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55eddebc5800 msgr2=0x55eddebce580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a6cf5840 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55eddebc5800 0x55eddebce580 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a6cf5840 1 -- 172.21.15.78:0/2381226368 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55eddebd8400 msgr2=0x55eddebd0680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a6cf5840 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55eddebd8400 0x55eddebd0680 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a6cf5840 1 -- 172.21.15.78:0/2381226368 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55eddebd8800 msgr2=0x55eddebcf080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a6cf5840 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55eddebd8800 0x55eddebcf080 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a6cf5840 1 -- 172.21.15.78:0/2381226368 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55eddeab3400 msgr2=0x55eddebcfb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a6cf5840 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55eddeab3400 0x55eddebcfb80 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a6cf5840 1 -- 172.21.15.78:0/2381226368 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55eddebd8000 msgr2=0x55eddebcf600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a6cf5840 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55eddebd8000 0x55eddebcf600 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 reap_dead start 2023-10-11T09:48:49.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 reap_dead start 2023-10-11T09:48:49.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a6cf5840 1 -- 172.21.15.78:0/2381226368 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55eddeab3800 msgr2=0x55edde813700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a6cf5840 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55eddeab3800 0x55edde813700 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x55edde8789f0 tx=0x55eddf556cc0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a6cf5840 1 -- 172.21.15.78:0/2381226368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eddeab2c00 msgr2=0x55edde812680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a6cf5840 1 --2- 172.21.15.78:0/2381226368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eddeab2c00 0x55edde812680 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x55eddf552c60 tx=0x55eddf556c00 comp rx=0 tx=0).stop 2023-10-11T09:48:49.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 reap_dead start 2023-10-11T09:48:49.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 reap_dead start 2023-10-11T09:48:49.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a54b1700 1 -- 172.21.15.78:0/2381226368 reap_dead start 2023-10-11T09:48:49.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a6cf5840 1 -- 172.21.15.78:0/2381226368 shutdown_connections 2023-10-11T09:48:49.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a6cf5840 1 -- 172.21.15.78:0/2381226368 >> 172.21.15.78:0/2381226368 conn(0x55eddeab2000 msgr2=0x55edde85b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:49.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a6cf5840 1 -- 172.21.15.78:0/2381226368 shutdown_connections 2023-10-11T09:48:49.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a6cf5840 1 -- 172.21.15.78:0/2381226368 wait complete. 2023-10-11T09:48:49.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a6cf5840 1 -- 172.21.15.78:0/1027891103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55edde76b400 msgr2=0x55edde810b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:49.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a6cf5840 1 --2- 172.21.15.78:0/1027891103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55edde76b400 0x55edde810b00 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x55edde879560 tx=0x55eddf550480 comp rx=0 tx=0).stop 2023-10-11T09:48:49.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a6cf5840 1 -- 172.21.15.78:0/1027891103 shutdown_connections 2023-10-11T09:48:49.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a6cf5840 1 --2- 172.21.15.78:0/1027891103 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55edde76b800 0x55edde811080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a6cf5840 1 --2- 172.21.15.78:0/1027891103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55edde76b400 0x55edde810b00 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a6cf5840 1 --2- 172.21.15.78:0/1027891103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55edde76b000 0x55edde810580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:49.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a6cf5840 1 -- 172.21.15.78:0/1027891103 >> 172.21.15.78:0/1027891103 conn(0x55edde76ac00 msgr2=0x55edde85b000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:49.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a6cf5840 1 -- 172.21.15.78:0/1027891103 shutdown_connections 2023-10-11T09:48:49.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:49.876+0000 7f02a6cf5840 1 -- 172.21.15.78:0/1027891103 wait complete. 2023-10-11T09:48:49.891 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.178330, result: 2023-10-11T09:48:49.891 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-11T09:48:50.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.064+0000 7f0c81fef700 1 Processor -- start 2023-10-11T09:48:50.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.064+0000 7f0c81fef700 1 -- start start 2023-10-11T09:48:50.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.068+0000 7f0c81fef700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c7c0d1660 0x7f0c7c0d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:50.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.068+0000 7f0c81fef700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c7c15dc10 con 0x7f0c7c156ce0 2023-10-11T09:48:50.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.068+0000 7f0c81fef700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c7c160380 con 0x7f0c7c0d28e0 2023-10-11T09:48:50.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.068+0000 7f0c81fef700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c15d930 con 0x7f0c7c0d1660 2023-10-11T09:48:50.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.068+0000 7f0c80fed700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f0c7c156ce0 0x7f0c7c160b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:54554/0 (socket says 172.21.15.78:54554) 2023-10-11T09:48:50.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.068+0000 7f0c80fed700 1 -- 172.21.15.78:0/863554158 learned_addr learned my addr 172.21.15.78:0/863554158 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:50.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.068+0000 7f0c7bfff700 1 --2- 172.21.15.78:0/863554158 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c7c0d1660 0x7f0c7c0d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:50.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.068+0000 7f0c7b7fe700 1 -- 172.21.15.78:0/863554158 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1932232031 0 0) 0x7f0c7c15dc10 con 0x7f0c7c156ce0 2023-10-11T09:48:50.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.068+0000 7f0c7b7fe700 1 -- 172.21.15.78:0/863554158 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c6c0035d0 con 0x7f0c7c156ce0 2023-10-11T09:48:50.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.068+0000 7f0c7b7fe700 1 -- 172.21.15.78:0/863554158 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2722942102 0 0) 0x7f0c6c0035d0 con 0x7f0c7c156ce0 2023-10-11T09:48:50.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.068+0000 7f0c7b7fe700 1 -- 172.21.15.78:0/863554158 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c7c0d1660 msgr2=0x7f0c7c0d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:50.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.068+0000 7f0c7b7fe700 1 --2- 172.21.15.78:0/863554158 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c7c0d1660 0x7f0c7c0d0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:50.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.068+0000 7f0c7b7fe700 1 -- 172.21.15.78:0/863554158 >> v1:172.21.15.133:6789/0 conn(0x7f0c7c0d28e0 legacy=0x7f0c7c0cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:50.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.068+0000 7f0c7b7fe700 1 -- 172.21.15.78:0/863554158 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7c160620 con 0x7f0c7c156ce0 2023-10-11T09:48:50.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.068+0000 7f0c7b7fe700 1 -- 172.21.15.78:0/863554158 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f0c70001670 con 0x7f0c7c156ce0 2023-10-11T09:48:50.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.068+0000 7f0c7b7fe700 1 -- 172.21.15.78:0/863554158 >> v1:172.21.15.78:6789/0 conn(0x7f0c7c156ce0 legacy=0x7f0c7c160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:50.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.068+0000 7f0c7b7fe700 1 --2- 172.21.15.78:0/863554158 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c7c0d1660 0x7f0c6c003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:50.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.068+0000 7f0c7b7fe700 1 --2- 172.21.15.78:0/863554158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c6c004020 0x7f0c6c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:50.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.068+0000 7f0c7b7fe700 1 --2- 172.21.15.78:0/863554158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c6c004db0 0x7f0c6c005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:50.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.068+0000 7f0c7b7fe700 1 -- 172.21.15.78:0/863554158 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c15d930 con 0x7f0c6c004020 2023-10-11T09:48:50.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.068+0000 7f0c7b7fe700 1 -- 172.21.15.78:0/863554158 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6c00bbf0 con 0x7f0c6c004db0 2023-10-11T09:48:50.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.068+0000 7f0c7b7fe700 1 -- 172.21.15.78:0/863554158 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6c00bdb0 con 0x7f0c7c0d1660 2023-10-11T09:48:50.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.068+0000 7f0c7bfff700 1 --2- 172.21.15.78:0/863554158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c6c004020 0x7f0c6c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:50.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.072+0000 7f0c80fed700 1 --2- 172.21.15.78:0/863554158 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c7c0d1660 0x7f0c6c003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:50.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.072+0000 7f0c817ee700 1 --2- 172.21.15.78:0/863554158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c6c004db0 0x7f0c6c005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:50.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.072+0000 7f0c7bfff700 1 -- 172.21.15.78:0/863554158 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c7c0d1660 msgr2=0x7f0c6c003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:50.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.072+0000 7f0c7bfff700 1 --2- 172.21.15.78:0/863554158 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c7c0d1660 0x7f0c6c003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:50.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.072+0000 7f0c7bfff700 1 -- 172.21.15.78:0/863554158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c6c004db0 msgr2=0x7f0c6c005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:50.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.072+0000 7f0c7bfff700 1 --2- 172.21.15.78:0/863554158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c6c004db0 0x7f0c6c005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:50.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.072+0000 7f0c7bfff700 1 -- 172.21.15.78:0/863554158 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c6c0035d0 con 0x7f0c6c004020 2023-10-11T09:48:50.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.072+0000 7f0c80fed700 1 --2- 172.21.15.78:0/863554158 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c7c0d1660 0x7f0c6c003a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:48:50.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.072+0000 7f0c817ee700 1 --2- 172.21.15.78:0/863554158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c6c004db0 0x7f0c6c005220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:48:50.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.072+0000 7f0c7bfff700 1 --2- 172.21.15.78:0/863554158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c6c004020 0x7f0c6c004870 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f0c68009b40 tx=0x7f0c6c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc3501d307444e3c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:50.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.072+0000 7f0c7b7fe700 1 -- 172.21.15.78:0/863554158 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6800e6c0 con 0x7f0c6c004020 2023-10-11T09:48:50.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.072+0000 7f0c7b7fe700 1 -- 172.21.15.78:0/863554158 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c6800e820 con 0x7f0c6c004020 2023-10-11T09:48:50.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.072+0000 7f0c7b7fe700 1 -- 172.21.15.78:0/863554158 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6800eb90 con 0x7f0c6c004020 2023-10-11T09:48:50.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.072+0000 7f0c81fef700 1 -- 172.21.15.78:0/863554158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c6c004020 msgr2=0x7f0c6c004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:50.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.072+0000 7f0c81fef700 1 --2- 172.21.15.78:0/863554158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c6c004020 0x7f0c6c004870 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f0c68009b40 tx=0x7f0c6c0058a0 comp rx=0 tx=0).stop 2023-10-11T09:48:50.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.072+0000 7f0c817ee700 1 -- 172.21.15.78:0/863554158 reap_dead start 2023-10-11T09:48:50.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.072+0000 7f0c81fef700 1 -- 172.21.15.78:0/863554158 shutdown_connections 2023-10-11T09:48:50.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.072+0000 7f0c81fef700 1 -- 172.21.15.78:0/863554158 >> 172.21.15.78:0/863554158 conn(0x7f0c7c06afb0 msgr2=0x7f0c7c06b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:50.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.072+0000 7f0c81fef700 1 -- 172.21.15.78:0/863554158 shutdown_connections 2023-10-11T09:48:50.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.072+0000 7f0c81fef700 1 -- 172.21.15.78:0/863554158 wait complete. 2023-10-11T09:48:50.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.076+0000 7f0c81fef700 1 Processor -- start 2023-10-11T09:48:50.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.076+0000 7f0c81fef700 1 -- start start 2023-10-11T09:48:50.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.076+0000 7f0c81fef700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c6c004020 0x7f0c7c15f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:50.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.076+0000 7f0c81fef700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c7c151b60 0x7f0c7c0c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:50.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.076+0000 7f0c81fef700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c7c0c8fc0 0x7f0c7c0c9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:50.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.076+0000 7f0c81fef700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c1f5330 con 0x7f0c6c004020 2023-10-11T09:48:50.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.076+0000 7f0c81fef700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c15f950 con 0x7f0c7c151b60 2023-10-11T09:48:50.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.076+0000 7f0c81fef700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c1f5580 con 0x7f0c7c0c8fc0 2023-10-11T09:48:50.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.076+0000 7f0c80fed700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c6c004020 0x7f0c7c15f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:50.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.076+0000 7f0c80fed700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c6c004020 0x7f0c7c15f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35828/0 (socket says 172.21.15.78:35828) 2023-10-11T09:48:50.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.076+0000 7f0c80fed700 1 -- 172.21.15.78:0/2218018423 learned_addr learned my addr 172.21.15.78:0/2218018423 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:50.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.076+0000 7f0c7bfff700 1 --2- 172.21.15.78:0/2218018423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c7c151b60 0x7f0c7c0c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:50.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.076+0000 7f0c817ee700 1 --2- 172.21.15.78:0/2218018423 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c7c0c8fc0 0x7f0c7c0c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:50.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.080+0000 7f0c80fed700 1 -- 172.21.15.78:0/2218018423 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c7c0c8fc0 msgr2=0x7f0c7c0c9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:50.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.080+0000 7f0c80fed700 1 --2- 172.21.15.78:0/2218018423 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c7c0c8fc0 0x7f0c7c0c9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:50.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.080+0000 7f0c80fed700 1 -- 172.21.15.78:0/2218018423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c7c151b60 msgr2=0x7f0c7c0c72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:50.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.080+0000 7f0c80fed700 1 --2- 172.21.15.78:0/2218018423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c7c151b60 0x7f0c7c0c72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:50.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.080+0000 7f0c80fed700 1 -- 172.21.15.78:0/2218018423 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7c1f5860 con 0x7f0c6c004020 2023-10-11T09:48:50.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.080+0000 7f0c80fed700 1 --2- 172.21.15.78:0/2218018423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c6c004020 0x7f0c7c15f410 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000dae0 tx=0x7f0c7000ddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:50.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.080+0000 7f0c797fa700 1 -- 172.21.15.78:0/2218018423 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c70004ca0 con 0x7f0c6c004020 2023-10-11T09:48:50.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.080+0000 7f0c797fa700 1 -- 172.21.15.78:0/2218018423 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c70004e00 con 0x7f0c6c004020 2023-10-11T09:48:50.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.080+0000 7f0c81fef700 1 -- 172.21.15.78:0/2218018423 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c7c1f5b40 con 0x7f0c6c004020 2023-10-11T09:48:50.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.080+0000 7f0c797fa700 1 -- 172.21.15.78:0/2218018423 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c700072d0 con 0x7f0c6c004020 2023-10-11T09:48:50.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.080+0000 7f0c81fef700 1 -- 172.21.15.78:0/2218018423 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c7c0c9970 con 0x7f0c6c004020 2023-10-11T09:48:50.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.080+0000 7f0c81fef700 1 -- 172.21.15.78:0/2218018423 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c7c065d40 con 0x7f0c6c004020 2023-10-11T09:48:50.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.084+0000 7f0c797fa700 1 -- 172.21.15.78:0/2218018423 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f0c70007430 con 0x7f0c6c004020 2023-10-11T09:48:50.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.084+0000 7f0c797fa700 1 --2- 172.21.15.78:0/2218018423 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0c60065ed0 0x7f0c60068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:50.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.084+0000 7f0c797fa700 1 -- 172.21.15.78:0/2218018423 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6638+0+0 (secure 0 0 0) 0x7f0c700830d0 con 0x7f0c6c004020 2023-10-11T09:48:50.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.084+0000 7f0c7bfff700 1 --2- 172.21.15.78:0/2218018423 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0c60065ed0 0x7f0c60068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:50.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.088+0000 7f0c7bfff700 1 --2- 172.21.15.78:0/2218018423 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0c60065ed0 0x7f0c60068380 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0c680090b0 tx=0x7f0c6801e040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:50.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.092+0000 7f0c797fa700 1 -- 172.21.15.78:0/2218018423 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f0c70050570 con 0x7f0c6c004020 2023-10-11T09:48:50.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.332+0000 7f0c81fef700 1 -- 172.21.15.78:0/2218018423 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f0c7c06b630 con 0x7f0c6c004020 2023-10-11T09:48:50.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.540+0000 7f0c797fa700 1 -- 172.21.15.78:0/2218018423 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 4:0 v65) v1 ==== 101+0+0 (secure 0 0 0) 0x7f0c70053b90 con 0x7f0c6c004020 2023-10-11T09:48:50.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.544+0000 7f0c81fef700 1 -- 172.21.15.78:0/2218018423 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0c60065ed0 msgr2=0x7f0c60068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:50.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.544+0000 7f0c81fef700 1 --2- 172.21.15.78:0/2218018423 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0c60065ed0 0x7f0c60068380 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0c680090b0 tx=0x7f0c6801e040 comp rx=0 tx=0).stop 2023-10-11T09:48:50.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.544+0000 7f0c81fef700 1 -- 172.21.15.78:0/2218018423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c6c004020 msgr2=0x7f0c7c15f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:50.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.544+0000 7f0c81fef700 1 --2- 172.21.15.78:0/2218018423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c6c004020 0x7f0c7c15f410 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000dae0 tx=0x7f0c7000ddf0 comp rx=0 tx=0).stop 2023-10-11T09:48:50.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.544+0000 7f0c81fef700 1 -- 172.21.15.78:0/2218018423 shutdown_connections 2023-10-11T09:48:50.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.544+0000 7f0c81fef700 1 --2- 172.21.15.78:0/2218018423 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0c60065ed0 0x7f0c60068380 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:50.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.544+0000 7f0c81fef700 1 --2- 172.21.15.78:0/2218018423 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c7c0c8fc0 0x7f0c7c0c9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:50.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.544+0000 7f0c81fef700 1 --2- 172.21.15.78:0/2218018423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c7c151b60 0x7f0c7c0c72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:50.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.544+0000 7f0c81fef700 1 --2- 172.21.15.78:0/2218018423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c6c004020 0x7f0c7c15f410 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:50.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.544+0000 7f0c81fef700 1 -- 172.21.15.78:0/2218018423 >> 172.21.15.78:0/2218018423 conn(0x7f0c7c06afb0 msgr2=0x7f0c7c0cfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:50.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.544+0000 7f0c81fef700 1 -- 172.21.15.78:0/2218018423 shutdown_connections 2023-10-11T09:48:50.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.544+0000 7f0c81fef700 1 -- 172.21.15.78:0/2218018423 wait complete. 2023-10-11T09:48:50.553 INFO:teuthology.orchestra.run.smithi078.stderr:repaired: restoring rank 4:0 2023-10-11T09:48:50.557 INFO:tasks.ceph.mds.c.smithi078.stderr:2023-10-11T09:48:50.552+0000 7fd6ccd43700 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-10-11T09:48:50.574 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-11T09:48:50.574 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-11T09:48:50.575 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-11T09:48:50.575 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:48:50.575 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-11T09:48:50.575 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-11T09:48:50.576 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-11T09:48:50.576 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:48:50.576 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-11T09:48:50.576 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-11T09:48:50.577 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-11T09:48:50.577 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:48:50.577 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-11T09:48:50.578 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-11T09:48:50.578 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-11T09:48:50.578 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:48:50.596 INFO:tasks.ceph.mds.a:Stopped 2023-10-11T09:48:50.597 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-11T09:48:50.598 INFO:tasks.ceph.mds.b:Stopped 2023-10-11T09:48:50.598 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-11T09:48:50.599 INFO:tasks.ceph.mds.d:Stopped 2023-10-11T09:48:50.599 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-11T09:48:50.601 INFO:tasks.ceph.mds.c:Stopped 2023-10-11T09:48:50.601 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-11T09:48:50.648 INFO:tasks.ceph.mds.b:Started 2023-10-11T09:48:50.648 DEBUG:teuthology.parallel:result is None 2023-10-11T09:48:50.649 INFO:tasks.ceph.mds.a:Started 2023-10-11T09:48:50.649 DEBUG:teuthology.parallel:result is None 2023-10-11T09:48:50.650 INFO:tasks.ceph.mds.d:Started 2023-10-11T09:48:50.651 DEBUG:teuthology.parallel:result is None 2023-10-11T09:48:50.655 INFO:tasks.ceph.mds.c:Started 2023-10-11T09:48:50.656 DEBUG:teuthology.parallel:result is None 2023-10-11T09:48:50.656 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:48:50.763 INFO:tasks.ceph.mds.b.smithi133.stdout:starting mds.b at 2023-10-11T09:48:50.763 INFO:tasks.ceph.mds.d.smithi133.stdout:starting mds.d at 2023-10-11T09:48:50.768 INFO:tasks.ceph.mds.c.smithi078.stdout:starting mds.c at 2023-10-11T09:48:50.768 INFO:tasks.ceph.mds.a.smithi078.stdout:starting mds.a at 2023-10-11T09:48:50.808 INFO:tasks.ceph.mon.a.smithi078.stderr:2023-10-11T09:48:50.804+0000 7f90109ef700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-11T09:48:50.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.824+0000 7ffa70770700 1 Processor -- start 2023-10-11T09:48:50.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.824+0000 7ffa70770700 1 -- start start 2023-10-11T09:48:50.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.828+0000 7ffa70770700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ffa6815a780 0x7ffa680cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:50.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.828+0000 7ffa70770700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa680cf4c0 con 0x7ffa680cefb0 2023-10-11T09:48:50.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.828+0000 7ffa70770700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa680cf7a0 con 0x7ffa6815ab50 2023-10-11T09:48:50.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.828+0000 7ffa70770700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa680cfa80 con 0x7ffa6815a780 2023-10-11T09:48:50.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.828+0000 7ffa6e50c700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ffa6815a780 0x7ffa680cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:50.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.828+0000 7ffa6e50c700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ffa6815a780 0x7ffa680cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:33968/0 (socket says 172.21.15.78:33968) 2023-10-11T09:48:50.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.828+0000 7ffa6e50c700 1 -- 172.21.15.78:0/2004131189 learned_addr learned my addr 172.21.15.78:0/2004131189 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:50.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.828+0000 7ffa6e50c700 1 -- 172.21.15.78:0/2004131189 >> v1:172.21.15.133:6789/0 conn(0x7ffa6815ab50 legacy=0x7ffa680ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:50.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.828+0000 7ffa6e50c700 1 -- 172.21.15.78:0/2004131189 >> v1:172.21.15.78:6789/0 conn(0x7ffa680cefb0 legacy=0x7ffa680b3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:50.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.828+0000 7ffa6d50a700 1 -- 172.21.15.78:0/2004131189 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1678805581 0 0) 0x7ffa680cf4c0 con 0x7ffa680cefb0 2023-10-11T09:48:50.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.828+0000 7ffa6e50c700 1 -- 172.21.15.78:0/2004131189 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa680cc760 con 0x7ffa6815a780 2023-10-11T09:48:50.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.828+0000 7ffa6e50c700 1 --2- 172.21.15.78:0/2004131189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa6815a780 0x7ffa680cea70 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ffa64009910 tx=0x7ffa64009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f8e68261e764110 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:50.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.832+0000 7ffa6d50a700 1 -- 172.21.15.78:0/2004131189 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6400ba60 con 0x7ffa6815a780 2023-10-11T09:48:50.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.832+0000 7ffa6d50a700 1 -- 172.21.15.78:0/2004131189 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa6400bbc0 con 0x7ffa6815a780 2023-10-11T09:48:50.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.832+0000 7ffa6d50a700 1 -- 172.21.15.78:0/2004131189 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa64009e60 con 0x7ffa6815a780 2023-10-11T09:48:50.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.832+0000 7ffa70770700 1 -- 172.21.15.78:0/2004131189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa6815a780 msgr2=0x7ffa680cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:50.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.832+0000 7ffa70770700 1 --2- 172.21.15.78:0/2004131189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa6815a780 0x7ffa680cea70 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ffa64009910 tx=0x7ffa64009c20 comp rx=0 tx=0).stop 2023-10-11T09:48:50.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.832+0000 7ffa70770700 1 -- 172.21.15.78:0/2004131189 shutdown_connections 2023-10-11T09:48:50.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.832+0000 7ffa70770700 1 --2- 172.21.15.78:0/2004131189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa6815a780 0x7ffa680cea70 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:50.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.832+0000 7ffa70770700 1 -- 172.21.15.78:0/2004131189 >> 172.21.15.78:0/2004131189 conn(0x7ffa680d0f30 msgr2=0x7ffa680d1330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:50.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.832+0000 7ffa70770700 1 -- 172.21.15.78:0/2004131189 shutdown_connections 2023-10-11T09:48:50.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.832+0000 7ffa70770700 1 -- 172.21.15.78:0/2004131189 wait complete. 2023-10-11T09:48:50.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.836+0000 7ffa70770700 1 Processor -- start 2023-10-11T09:48:50.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.840+0000 7ffa70770700 1 -- start start 2023-10-11T09:48:50.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.840+0000 7ffa70770700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffa680cefb0 0x7ffa680b3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:50.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.840+0000 7ffa70770700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa6815a780 0x7ffa681419f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:50.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.840+0000 7ffa70770700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa6815ab50 0x7ffa68141f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:50.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.840+0000 7ffa70770700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa680b42a0 con 0x7ffa6815a780 2023-10-11T09:48:50.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.840+0000 7ffa70770700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa680b6d90 con 0x7ffa6815ab50 2023-10-11T09:48:50.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.840+0000 7ffa70770700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa68142470 con 0x7ffa680cefb0 2023-10-11T09:48:50.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.840+0000 7ffa6e50c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffa680cefb0 0x7ffa680b3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:50.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.840+0000 7ffa6dd0b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa6815a780 0x7ffa681419f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:50.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.840+0000 7ffa6e50c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffa680cefb0 0x7ffa680b3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:46472/0 (socket says 172.21.15.78:46472) 2023-10-11T09:48:50.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.840+0000 7ffa6e50c700 1 -- 172.21.15.78:0/2640131022 learned_addr learned my addr 172.21.15.78:0/2640131022 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:50.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.840+0000 7ffa6ed0d700 1 --2- 172.21.15.78:0/2640131022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa6815ab50 0x7ffa68141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:50.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.840+0000 7ffa6dd0b700 1 -- 172.21.15.78:0/2640131022 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffa680cefb0 msgr2=0x7ffa680b3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:50.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.840+0000 7ffa6dd0b700 1 --2- 172.21.15.78:0/2640131022 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffa680cefb0 0x7ffa680b3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:50.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.840+0000 7ffa6dd0b700 1 -- 172.21.15.78:0/2640131022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa6815ab50 msgr2=0x7ffa68141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:50.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.840+0000 7ffa6dd0b700 1 --2- 172.21.15.78:0/2640131022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa6815ab50 0x7ffa68141f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:50.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.840+0000 7ffa6dd0b700 1 -- 172.21.15.78:0/2640131022 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa64009570 con 0x7ffa6815a780 2023-10-11T09:48:50.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.840+0000 7ffa6e50c700 1 --2- 172.21.15.78:0/2640131022 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffa680cefb0 0x7ffa680b3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:48:50.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.840+0000 7ffa6dd0b700 1 --2- 172.21.15.78:0/2640131022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa6815a780 0x7ffa681419f0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7ffa5800c410 tx=0x7ffa58001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:50.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.844+0000 7ffa5effd700 1 -- 172.21.15.78:0/2640131022 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa580052d0 con 0x7ffa6815a780 2023-10-11T09:48:50.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.844+0000 7ffa5effd700 1 -- 172.21.15.78:0/2640131022 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa58005430 con 0x7ffa6815a780 2023-10-11T09:48:50.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.844+0000 7ffa70770700 1 -- 172.21.15.78:0/2640131022 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa6815da20 con 0x7ffa6815a780 2023-10-11T09:48:50.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.844+0000 7ffa5effd700 1 -- 172.21.15.78:0/2640131022 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa580055a0 con 0x7ffa6815a780 2023-10-11T09:48:50.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.844+0000 7ffa70770700 1 -- 172.21.15.78:0/2640131022 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa680b6a20 con 0x7ffa6815a780 2023-10-11T09:48:50.852 INFO:tasks.ceph.mds.b.smithi133.stderr:2023-10-11T09:48:50.845+0000 7f575d918700 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-10-11T09:48:50.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.844+0000 7ffa70770700 1 -- 172.21.15.78:0/2640131022 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa680b6ed0 con 0x7ffa6815a780 2023-10-11T09:48:50.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.848+0000 7ffa5effd700 1 -- 172.21.15.78:0/2640131022 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7ffa580038e0 con 0x7ffa6815a780 2023-10-11T09:48:50.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.848+0000 7ffa5effd700 1 --2- 172.21.15.78:0/2640131022 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffa4c065ed0 0x7ffa4c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:50.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.848+0000 7ffa5effd700 1 -- 172.21.15.78:0/2640131022 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6724+0+0 (secure 0 0 0) 0x7ffa58082cd0 con 0x7ffa6815a780 2023-10-11T09:48:50.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.848+0000 7ffa6e50c700 1 --2- 172.21.15.78:0/2640131022 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffa4c065ed0 0x7ffa4c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:50.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.848+0000 7ffa6e50c700 1 --2- 172.21.15.78:0/2640131022 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffa4c065ed0 0x7ffa4c068380 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ffa64009150 tx=0x7ffa64008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:50.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:50.852+0000 7ffa5effd700 1 -- 172.21.15.78:0/2640131022 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7ffa680b6ed0 con 0x7ffa6815a780 2023-10-11T09:48:51.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:51.056+0000 7ffa70770700 1 -- 172.21.15.78:0/2640131022 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa680b6ed0 con 0x7ffa6815a780 2023-10-11T09:48:51.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:51.056+0000 7ffa5effd700 1 -- 172.21.15.78:0/2640131022 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4657 (secure 0 0 0) 0x7ffa680b6ed0 con 0x7ffa6815a780 2023-10-11T09:48:51.063 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:51.063 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":68,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6581,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/238181631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":238181631},{"type":"v1","addr":"172.21.15.78:6837","nonce":238181631}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":7378,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/2416080754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2416080754},{"type":"v1","addr":"172.21.15.133:6834","nonce":2416080754}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":7384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/3016626706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3016626706},{"type":"v1","addr":"172.21.15.78:6836","nonce":3016626706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:48:45.351822+0000","modified":"2023-10-11T09:48:50.827853+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7725":{"gid":7725,"name":"b","rank":0,"incarnation":68,"state":"up:replay","state_seq":1,"addr":"172.21.15.133:6835/3922016223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":3922016223},{"type":"v1","addr":"172.21.15.133:6835","nonce":3922016223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12,13],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-11T09:48:51.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:51.060+0000 7ffa70770700 1 -- 172.21.15.78:0/2640131022 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffa4c065ed0 msgr2=0x7ffa4c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:51.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:51.060+0000 7ffa70770700 1 --2- 172.21.15.78:0/2640131022 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffa4c065ed0 0x7ffa4c068380 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ffa64009150 tx=0x7ffa64008030 comp rx=0 tx=0).stop 2023-10-11T09:48:51.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:51.060+0000 7ffa70770700 1 -- 172.21.15.78:0/2640131022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa6815a780 msgr2=0x7ffa681419f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:51.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:51.060+0000 7ffa70770700 1 --2- 172.21.15.78:0/2640131022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa6815a780 0x7ffa681419f0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7ffa5800c410 tx=0x7ffa58001e70 comp rx=0 tx=0).stop 2023-10-11T09:48:51.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:51.060+0000 7ffa70770700 1 -- 172.21.15.78:0/2640131022 shutdown_connections 2023-10-11T09:48:51.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:51.060+0000 7ffa70770700 1 --2- 172.21.15.78:0/2640131022 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffa4c065ed0 0x7ffa4c068380 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:51.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:51.060+0000 7ffa70770700 1 --2- 172.21.15.78:0/2640131022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa6815ab50 0x7ffa68141f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:51.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:51.060+0000 7ffa70770700 1 --2- 172.21.15.78:0/2640131022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa6815a780 0x7ffa681419f0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:51.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:51.060+0000 7ffa70770700 1 --2- 172.21.15.78:0/2640131022 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffa680cefb0 0x7ffa680b3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:51.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:51.060+0000 7ffa70770700 1 -- 172.21.15.78:0/2640131022 >> 172.21.15.78:0/2640131022 conn(0x7ffa680d0f30 msgr2=0x7ffa680b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:51.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:51.064+0000 7ffa70770700 1 -- 172.21.15.78:0/2640131022 shutdown_connections 2023-10-11T09:48:51.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:51.064+0000 7ffa70770700 1 -- 172.21.15.78:0/2640131022 wait complete. 2023-10-11T09:48:51.070 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 68 2023-10-11T09:48:51.493 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-11T09:48:45.351822+0000', 'modified': '2023-10-11T09:48:50.827853+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 92, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7725}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7725': {'gid': 7725, 'name': 'b', 'rank': 0, 'incarnation': 68, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.133:6835/3922016223', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6833', 'nonce': 3922016223}, {'type': 'v1', 'addr': '172.21.15.133:6835', 'nonce': 3922016223}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [12, 13], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-11T09:48:51.494 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7725:up:replay 2023-10-11T09:48:52.495 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:48:52.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.660+0000 7fb64b482700 1 Processor -- start 2023-10-11T09:48:52.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.660+0000 7fb64b482700 1 -- start start 2023-10-11T09:48:52.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.664+0000 7fb64b482700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb644160d40 0x7fb6440cd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:52.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.664+0000 7fb64b482700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb644071040 0x7fb6440cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:52.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.664+0000 7fb64b482700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb64406ffa0 con 0x7fb6440cd7c0 2023-10-11T09:48:52.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.664+0000 7fb64b482700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb644070c20 con 0x7fb644071040 2023-10-11T09:48:52.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.664+0000 7fb64b482700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb644161110 con 0x7fb644160d40 2023-10-11T09:48:52.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.664+0000 7fb648a1d700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb644071040 0x7fb6440cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:52.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.664+0000 7fb64921e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb644160d40 0x7fb6440cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:52.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.664+0000 7fb649a1f700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fb6440cd7c0 0x7fb6441647f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:60414/0 (socket says 172.21.15.78:60414) 2023-10-11T09:48:52.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.664+0000 7fb64921e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb644160d40 0x7fb6440cd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:46488/0 (socket says 172.21.15.78:46488) 2023-10-11T09:48:52.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.664+0000 7fb648a1d700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb644071040 0x7fb6440cddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:33994/0 (socket says 172.21.15.78:33994) 2023-10-11T09:48:52.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.664+0000 7fb649a1f700 1 -- 172.21.15.78:0/953891834 learned_addr learned my addr 172.21.15.78:0/953891834 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:52.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.664+0000 7fb648a1d700 1 -- 172.21.15.78:0/953891834 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb644160d40 msgr2=0x7fb6440cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:52.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.664+0000 7fb648a1d700 1 --2- 172.21.15.78:0/953891834 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb644160d40 0x7fb6440cd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:52.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.664+0000 7fb648a1d700 1 -- 172.21.15.78:0/953891834 >> v1:172.21.15.133:6789/0 conn(0x7fb6440cd7c0 legacy=0x7fb6441647f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:52.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.664+0000 7fb648a1d700 1 -- 172.21.15.78:0/953891834 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb644070280 con 0x7fb644071040 2023-10-11T09:48:52.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.664+0000 7fb64921e700 1 --2- 172.21.15.78:0/953891834 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb644160d40 0x7fb6440cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:48:52.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.668+0000 7fb648a1d700 1 --2- 172.21.15.78:0/953891834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb644071040 0x7fb6440cddf0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb634009850 tx=0x7fb634009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=430a6e85b7938e74 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:52.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.668+0000 7fb63bfff700 1 -- 172.21.15.78:0/953891834 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb634011830 con 0x7fb644071040 2023-10-11T09:48:52.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.668+0000 7fb63bfff700 1 -- 172.21.15.78:0/953891834 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb634011990 con 0x7fb644071040 2023-10-11T09:48:52.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.668+0000 7fb63bfff700 1 -- 172.21.15.78:0/953891834 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb634011d40 con 0x7fb644071040 2023-10-11T09:48:52.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.668+0000 7fb64b482700 1 -- 172.21.15.78:0/953891834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb644071040 msgr2=0x7fb6440cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:52.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.668+0000 7fb64b482700 1 --2- 172.21.15.78:0/953891834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb644071040 0x7fb6440cddf0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb634009850 tx=0x7fb634009b60 comp rx=0 tx=0).stop 2023-10-11T09:48:52.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.668+0000 7fb64b482700 1 -- 172.21.15.78:0/953891834 shutdown_connections 2023-10-11T09:48:52.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.668+0000 7fb64b482700 1 --2- 172.21.15.78:0/953891834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb644071040 0x7fb6440cddf0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:52.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.668+0000 7fb64b482700 1 --2- 172.21.15.78:0/953891834 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb644160d40 0x7fb6440cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:52.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.668+0000 7fb64b482700 1 -- 172.21.15.78:0/953891834 >> 172.21.15.78:0/953891834 conn(0x7fb64406aca0 msgr2=0x7fb6440b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:52.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.668+0000 7fb64b482700 1 -- 172.21.15.78:0/953891834 shutdown_connections 2023-10-11T09:48:52.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.668+0000 7fb64b482700 1 -- 172.21.15.78:0/953891834 wait complete. 2023-10-11T09:48:52.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.672+0000 7fb64b482700 1 Processor -- start 2023-10-11T09:48:52.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.672+0000 7fb64b482700 1 -- start start 2023-10-11T09:48:52.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.672+0000 7fb64b482700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb644071040 0x7fb644159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:52.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.672+0000 7fb64b482700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6440cd7c0 0x7fb64415cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:52.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.672+0000 7fb64b482700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb644160d40 0x7fb6441eafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:52.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.672+0000 7fb64b482700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64415d830 con 0x7fb6440cd7c0 2023-10-11T09:48:52.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.672+0000 7fb64b482700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64415db10 con 0x7fb644071040 2023-10-11T09:48:52.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.672+0000 7fb64b482700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6441ea7b0 con 0x7fb644160d40 2023-10-11T09:48:52.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.676+0000 7fb648a1d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6440cd7c0 0x7fb64415cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:52.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.676+0000 7fb648a1d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6440cd7c0 0x7fb64415cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35882/0 (socket says 172.21.15.78:35882) 2023-10-11T09:48:52.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.676+0000 7fb648a1d700 1 -- 172.21.15.78:0/2795948946 learned_addr learned my addr 172.21.15.78:0/2795948946 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:52.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.676+0000 7fb64921e700 1 --2- 172.21.15.78:0/2795948946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb644071040 0x7fb644159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:52.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.676+0000 7fb649a1f700 1 --2- 172.21.15.78:0/2795948946 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb644160d40 0x7fb6441eafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:52.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.676+0000 7fb648a1d700 1 -- 172.21.15.78:0/2795948946 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb644160d40 msgr2=0x7fb6441eafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:52.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.676+0000 7fb648a1d700 1 --2- 172.21.15.78:0/2795948946 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb644160d40 0x7fb6441eafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:52.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.676+0000 7fb648a1d700 1 -- 172.21.15.78:0/2795948946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb644071040 msgr2=0x7fb644159b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:52.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.676+0000 7fb648a1d700 1 --2- 172.21.15.78:0/2795948946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb644071040 0x7fb644159b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:52.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.676+0000 7fb648a1d700 1 -- 172.21.15.78:0/2795948946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb640009a70 con 0x7fb6440cd7c0 2023-10-11T09:48:52.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.676+0000 7fb648a1d700 1 --2- 172.21.15.78:0/2795948946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6440cd7c0 0x7fb64415cd60 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fb6340090b0 tx=0x7fb634004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:52.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.676+0000 7fb639ffb700 1 -- 172.21.15.78:0/2795948946 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb634004480 con 0x7fb6440cd7c0 2023-10-11T09:48:52.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.676+0000 7fb639ffb700 1 -- 172.21.15.78:0/2795948946 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb63401f070 con 0x7fb6440cd7c0 2023-10-11T09:48:52.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.676+0000 7fb64b482700 1 -- 172.21.15.78:0/2795948946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6340094b0 con 0x7fb6440cd7c0 2023-10-11T09:48:52.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.676+0000 7fb639ffb700 1 -- 172.21.15.78:0/2795948946 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb634003900 con 0x7fb6440cd7c0 2023-10-11T09:48:52.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.676+0000 7fb64b482700 1 -- 172.21.15.78:0/2795948946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb64415a060 con 0x7fb6440cd7c0 2023-10-11T09:48:52.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.680+0000 7fb64b482700 1 -- 172.21.15.78:0/2795948946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb644072250 con 0x7fb6440cd7c0 2023-10-11T09:48:52.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.680+0000 7fb639ffb700 1 -- 172.21.15.78:0/2795948946 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fb634021030 con 0x7fb6440cd7c0 2023-10-11T09:48:52.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.680+0000 7fb639ffb700 1 --2- 172.21.15.78:0/2795948946 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb628065f00 0x7fb6280683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:52.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.680+0000 7fb639ffb700 1 -- 172.21.15.78:0/2795948946 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6724+0+0 (secure 0 0 0) 0x7fb634084d10 con 0x7fb6440cd7c0 2023-10-11T09:48:52.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.680+0000 7fb64921e700 1 --2- 172.21.15.78:0/2795948946 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb628065f00 0x7fb6280683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:52.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.684+0000 7fb64921e700 1 --2- 172.21.15.78:0/2795948946 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb628065f00 0x7fb6280683b0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb6400114e0 tx=0x7fb64000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:52.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.688+0000 7fb639ffb700 1 -- 172.21.15.78:0/2795948946 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fb63408a020 con 0x7fb6440cd7c0 2023-10-11T09:48:52.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.896+0000 7fb64b482700 1 -- 172.21.15.78:0/2795948946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb644070530 con 0x7fb6440cd7c0 2023-10-11T09:48:52.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.896+0000 7fb639ffb700 1 -- 172.21.15.78:0/2795948946 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4657 (secure 0 0 0) 0x7fb634015090 con 0x7fb6440cd7c0 2023-10-11T09:48:52.903 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:52.903 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":70,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6581,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/238181631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":238181631},{"type":"v1","addr":"172.21.15.78:6837","nonce":238181631}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":7378,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/2416080754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2416080754},{"type":"v1","addr":"172.21.15.133:6834","nonce":2416080754}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":7384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/3016626706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3016626706},{"type":"v1","addr":"172.21.15.78:6836","nonce":3016626706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:48:45.351822+0000","modified":"2023-10-11T09:48:52.840256+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7725":{"gid":7725,"name":"b","rank":0,"incarnation":68,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.133:6835/3922016223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":3922016223},{"type":"v1","addr":"172.21.15.133:6835","nonce":3922016223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12,13],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-11T09:48:52.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.900+0000 7fb64b482700 1 -- 172.21.15.78:0/2795948946 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb628065f00 msgr2=0x7fb6280683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:52.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.900+0000 7fb64b482700 1 --2- 172.21.15.78:0/2795948946 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb628065f00 0x7fb6280683b0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb6400114e0 tx=0x7fb64000b040 comp rx=0 tx=0).stop 2023-10-11T09:48:52.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.900+0000 7fb64b482700 1 -- 172.21.15.78:0/2795948946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6440cd7c0 msgr2=0x7fb64415cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:52.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.900+0000 7fb64b482700 1 --2- 172.21.15.78:0/2795948946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6440cd7c0 0x7fb64415cd60 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fb6340090b0 tx=0x7fb634004770 comp rx=0 tx=0).stop 2023-10-11T09:48:52.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.900+0000 7fb64b482700 1 -- 172.21.15.78:0/2795948946 shutdown_connections 2023-10-11T09:48:52.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.900+0000 7fb64b482700 1 --2- 172.21.15.78:0/2795948946 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb628065f00 0x7fb6280683b0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:52.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.900+0000 7fb64b482700 1 --2- 172.21.15.78:0/2795948946 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb644160d40 0x7fb6441eafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:52.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.900+0000 7fb64b482700 1 --2- 172.21.15.78:0/2795948946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6440cd7c0 0x7fb64415cd60 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:52.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.900+0000 7fb64b482700 1 --2- 172.21.15.78:0/2795948946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb644071040 0x7fb644159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:52.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.900+0000 7fb64b482700 1 -- 172.21.15.78:0/2795948946 >> 172.21.15.78:0/2795948946 conn(0x7fb64406aca0 msgr2=0x7fb6441612f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:52.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.900+0000 7fb64b482700 1 -- 172.21.15.78:0/2795948946 shutdown_connections 2023-10-11T09:48:52.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:52.900+0000 7fb64b482700 1 -- 172.21.15.78:0/2795948946 wait complete. 2023-10-11T09:48:52.909 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 70 2023-10-11T09:48:53.333 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 70, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-11T09:48:45.351822+0000', 'modified': '2023-10-11T09:48:52.840256+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 92, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7725}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7725': {'gid': 7725, 'name': 'b', 'rank': 0, 'incarnation': 68, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.133:6835/3922016223', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6833', 'nonce': 3922016223}, {'type': 'v1', 'addr': '172.21.15.133:6835', 'nonce': 3922016223}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [12, 13], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-11T09:48:53.333 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7725:up:rejoin 2023-10-11T09:48:54.334 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:48:54.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.508+0000 7fda22fb4700 1 Processor -- start 2023-10-11T09:48:54.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.508+0000 7fda22fb4700 1 -- start start 2023-10-11T09:48:54.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.512+0000 7fda22fb4700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fda1c15a940 0x7fda1c0ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:54.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.512+0000 7fda22fb4700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fda1c0ca780 0x7fda1c13fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:54.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.512+0000 7fda22fb4700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda1c0cabc0 con 0x7fda1c15b8e0 2023-10-11T09:48:54.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.512+0000 7fda22fb4700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda1c15e010 con 0x7fda1c15a940 2023-10-11T09:48:54.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.512+0000 7fda22fb4700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda1c15e280 con 0x7fda1c0ca780 2023-10-11T09:48:54.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.512+0000 7fda217b1700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fda1c15a940 0x7fda1c0ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:54.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.512+0000 7fda21fb2700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fda1c15b8e0 0x7fda1c06d0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:60428/0 (socket says 172.21.15.78:60428) 2023-10-11T09:48:54.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.512+0000 7fda21fb2700 1 -- 172.21.15.78:0/3181537523 learned_addr learned my addr 172.21.15.78:0/3181537523 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:54.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.512+0000 7fda227b3700 1 --2- 172.21.15.78:0/3181537523 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fda1c0ca780 0x7fda1c13fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:54.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.512+0000 7fda217b1700 1 -- 172.21.15.78:0/3181537523 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fda1c0ca780 msgr2=0x7fda1c13fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:54.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.512+0000 7fda217b1700 1 --2- 172.21.15.78:0/3181537523 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fda1c0ca780 0x7fda1c13fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:54.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.512+0000 7fda217b1700 1 -- 172.21.15.78:0/3181537523 >> v1:172.21.15.133:6789/0 conn(0x7fda1c15b8e0 legacy=0x7fda1c06d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:54.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.512+0000 7fda217b1700 1 -- 172.21.15.78:0/3181537523 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda1c15e560 con 0x7fda1c15a940 2023-10-11T09:48:54.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.512+0000 7fda20fb0700 1 -- 172.21.15.78:0/3181537523 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2701947065 0 0) 0x7fda1c0cabc0 con 0x7fda1c15b8e0 2023-10-11T09:48:54.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.512+0000 7fda227b3700 1 --2- 172.21.15.78:0/3181537523 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fda1c0ca780 0x7fda1c13fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:48:54.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.512+0000 7fda217b1700 1 --2- 172.21.15.78:0/3181537523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda1c15a940 0x7fda1c0ca240 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fda0c009850 tx=0x7fda0c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f58cc46a06fd58 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:54.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.512+0000 7fda20fb0700 1 -- 172.21.15.78:0/3181537523 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0c011830 con 0x7fda1c15a940 2023-10-11T09:48:54.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.512+0000 7fda20fb0700 1 -- 172.21.15.78:0/3181537523 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda0c011990 con 0x7fda1c15a940 2023-10-11T09:48:54.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.512+0000 7fda20fb0700 1 -- 172.21.15.78:0/3181537523 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0c011d40 con 0x7fda1c15a940 2023-10-11T09:48:54.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.516+0000 7fda22fb4700 1 -- 172.21.15.78:0/3181537523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda1c15a940 msgr2=0x7fda1c0ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:54.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.516+0000 7fda22fb4700 1 --2- 172.21.15.78:0/3181537523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda1c15a940 0x7fda1c0ca240 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fda0c009850 tx=0x7fda0c009b60 comp rx=0 tx=0).stop 2023-10-11T09:48:54.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.516+0000 7fda22fb4700 1 -- 172.21.15.78:0/3181537523 shutdown_connections 2023-10-11T09:48:54.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.516+0000 7fda22fb4700 1 --2- 172.21.15.78:0/3181537523 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fda1c0ca780 0x7fda1c13fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:54.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.516+0000 7fda22fb4700 1 --2- 172.21.15.78:0/3181537523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda1c15a940 0x7fda1c0ca240 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:54.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.516+0000 7fda22fb4700 1 -- 172.21.15.78:0/3181537523 >> 172.21.15.78:0/3181537523 conn(0x7fda1c0cea60 msgr2=0x7fda1c0cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:54.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.516+0000 7fda22fb4700 1 -- 172.21.15.78:0/3181537523 shutdown_connections 2023-10-11T09:48:54.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.516+0000 7fda22fb4700 1 -- 172.21.15.78:0/3181537523 wait complete. 2023-10-11T09:48:54.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.516+0000 7fda22fb4700 1 Processor -- start 2023-10-11T09:48:54.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.520+0000 7fda22fb4700 1 -- start start 2023-10-11T09:48:54.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.520+0000 7fda22fb4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda1c0ca780 0x7fda1c13baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:54.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.520+0000 7fda22fb4700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fda1c15a940 0x7fda1c159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:54.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.520+0000 7fda22fb4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda1c15b8e0 0x7fda1c1596b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:54.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.520+0000 7fda22fb4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1c1edcc0 con 0x7fda1c0ca780 2023-10-11T09:48:54.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.520+0000 7fda22fb4700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1c13c630 con 0x7fda1c15b8e0 2023-10-11T09:48:54.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.520+0000 7fda22fb4700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda1c1ed390 con 0x7fda1c15a940 2023-10-11T09:48:54.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.520+0000 7fda21fb2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda1c0ca780 0x7fda1c13baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:54.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.520+0000 7fda21fb2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda1c0ca780 0x7fda1c13baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35896/0 (socket says 172.21.15.78:35896) 2023-10-11T09:48:54.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.520+0000 7fda21fb2700 1 -- 172.21.15.78:0/1354616142 learned_addr learned my addr 172.21.15.78:0/1354616142 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:54.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.520+0000 7fda227b3700 1 --2- 172.21.15.78:0/1354616142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda1c15b8e0 0x7fda1c1596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:54.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.520+0000 7fda217b1700 1 --2- 172.21.15.78:0/1354616142 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fda1c15a940 0x7fda1c159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:54.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.520+0000 7fda21fb2700 1 -- 172.21.15.78:0/1354616142 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fda1c15a940 msgr2=0x7fda1c159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:54.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.520+0000 7fda21fb2700 1 --2- 172.21.15.78:0/1354616142 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fda1c15a940 0x7fda1c159170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:54.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.520+0000 7fda21fb2700 1 -- 172.21.15.78:0/1354616142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda1c15b8e0 msgr2=0x7fda1c1596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:54.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.520+0000 7fda21fb2700 1 --2- 172.21.15.78:0/1354616142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda1c15b8e0 0x7fda1c1596b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:54.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.520+0000 7fda21fb2700 1 -- 172.21.15.78:0/1354616142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda14009a70 con 0x7fda1c0ca780 2023-10-11T09:48:54.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.520+0000 7fda21fb2700 1 --2- 172.21.15.78:0/1354616142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda1c0ca780 0x7fda1c13baa0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fda1800c4e0 tx=0x7fda1800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:54.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.520+0000 7fda127fc700 1 -- 172.21.15.78:0/1354616142 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda18004ca0 con 0x7fda1c0ca780 2023-10-11T09:48:54.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.520+0000 7fda127fc700 1 -- 172.21.15.78:0/1354616142 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda18004e20 con 0x7fda1c0ca780 2023-10-11T09:48:54.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.520+0000 7fda127fc700 1 -- 172.21.15.78:0/1354616142 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda18007310 con 0x7fda1c0ca780 2023-10-11T09:48:54.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.520+0000 7fda22fb4700 1 -- 172.21.15.78:0/1354616142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda0c0094b0 con 0x7fda1c0ca780 2023-10-11T09:48:54.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.520+0000 7fda22fb4700 1 -- 172.21.15.78:0/1354616142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda1c1ed950 con 0x7fda1c0ca780 2023-10-11T09:48:54.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.524+0000 7fda22fb4700 1 -- 172.21.15.78:0/1354616142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda1c065d40 con 0x7fda1c0ca780 2023-10-11T09:48:54.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.524+0000 7fda127fc700 1 -- 172.21.15.78:0/1354616142 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fda18007470 con 0x7fda1c0ca780 2023-10-11T09:48:54.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.528+0000 7fda127fc700 1 --2- 172.21.15.78:0/1354616142 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fda00065ed0 0x7fda00068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:54.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.528+0000 7fda127fc700 1 -- 172.21.15.78:0/1354616142 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6724+0+0 (secure 0 0 0) 0x7fda180838d0 con 0x7fda1c0ca780 2023-10-11T09:48:54.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.528+0000 7fda217b1700 1 --2- 172.21.15.78:0/1354616142 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fda00065ed0 0x7fda00068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:54.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.528+0000 7fda217b1700 1 --2- 172.21.15.78:0/1354616142 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fda00065ed0 0x7fda00068380 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fda0c0090b0 tx=0x7fda0c01c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:54.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.532+0000 7fda127fc700 1 -- 172.21.15.78:0/1354616142 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fda18050b60 con 0x7fda1c0ca780 2023-10-11T09:48:54.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.744+0000 7fda22fb4700 1 -- 172.21.15.78:0/1354616142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda1c1ee130 con 0x7fda1c0ca780 2023-10-11T09:48:54.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.744+0000 7fda127fc700 1 -- 172.21.15.78:0/1354616142 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4657 (secure 0 0 0) 0x7fda18007720 con 0x7fda1c0ca780 2023-10-11T09:48:54.750 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:54.750 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":71,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6581,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/238181631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":238181631},{"type":"v1","addr":"172.21.15.78:6837","nonce":238181631}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":7378,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/2416080754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2416080754},{"type":"v1","addr":"172.21.15.133:6834","nonce":2416080754}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":7384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/3016626706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3016626706},{"type":"v1","addr":"172.21.15.78:6836","nonce":3016626706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:48:45.351822+0000","modified":"2023-10-11T09:48:53.854831+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7725":{"gid":7725,"name":"b","rank":0,"incarnation":68,"state":"up:active","state_seq":4,"addr":"172.21.15.133:6835/3922016223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":3922016223},{"type":"v1","addr":"172.21.15.133:6835","nonce":3922016223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12,13],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-11T09:48:54.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.748+0000 7fda22fb4700 1 -- 172.21.15.78:0/1354616142 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fda00065ed0 msgr2=0x7fda00068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:54.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.748+0000 7fda22fb4700 1 --2- 172.21.15.78:0/1354616142 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fda00065ed0 0x7fda00068380 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fda0c0090b0 tx=0x7fda0c01c040 comp rx=0 tx=0).stop 2023-10-11T09:48:54.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.748+0000 7fda22fb4700 1 -- 172.21.15.78:0/1354616142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda1c0ca780 msgr2=0x7fda1c13baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:54.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.748+0000 7fda22fb4700 1 --2- 172.21.15.78:0/1354616142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda1c0ca780 0x7fda1c13baa0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fda1800c4e0 tx=0x7fda1800c8a0 comp rx=0 tx=0).stop 2023-10-11T09:48:54.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.748+0000 7fda22fb4700 1 -- 172.21.15.78:0/1354616142 shutdown_connections 2023-10-11T09:48:54.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.748+0000 7fda22fb4700 1 --2- 172.21.15.78:0/1354616142 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fda00065ed0 0x7fda00068380 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:54.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.748+0000 7fda22fb4700 1 --2- 172.21.15.78:0/1354616142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda1c15b8e0 0x7fda1c1596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:54.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.748+0000 7fda22fb4700 1 --2- 172.21.15.78:0/1354616142 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fda1c15a940 0x7fda1c159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:54.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.748+0000 7fda22fb4700 1 --2- 172.21.15.78:0/1354616142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda1c0ca780 0x7fda1c13baa0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:54.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.748+0000 7fda22fb4700 1 -- 172.21.15.78:0/1354616142 >> 172.21.15.78:0/1354616142 conn(0x7fda1c0cea60 msgr2=0x7fda1c15be70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:54.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.748+0000 7fda22fb4700 1 -- 172.21.15.78:0/1354616142 shutdown_connections 2023-10-11T09:48:54.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:54.748+0000 7fda22fb4700 1 -- 172.21.15.78:0/1354616142 wait complete. 2023-10-11T09:48:54.754 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 71 2023-10-11T09:48:55.177 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-11T09:48:45.351822+0000', 'modified': '2023-10-11T09:48:53.854831+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 92, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7725}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7725': {'gid': 7725, 'name': 'b', 'rank': 0, 'incarnation': 68, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.133:6835/3922016223', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6833', 'nonce': 3922016223}, {'type': 'v1', 'addr': '172.21.15.133:6835', 'nonce': 3922016223}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [12, 13], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-11T09:48:55.178 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-11T09:48:55.179 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-11T09:48:55.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.356+0000 7f01b0e34700 1 Processor -- start 2023-10-11T09:48:55.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.356+0000 7f01b0e34700 1 -- start start 2023-10-11T09:48:55.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.360+0000 7f01b0e34700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f01ac15c880 0x7f01ac06be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:55.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.360+0000 7f01b0e34700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01ac0c93a0 con 0x7f01ac0cbd30 2023-10-11T09:48:55.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.360+0000 7f01b0e34700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01ac0cca10 con 0x7f01ac0cd210 2023-10-11T09:48:55.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.360+0000 7f01b0e34700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01ac161420 con 0x7f01ac15c880 2023-10-11T09:48:55.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.360+0000 7f01aa59c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f01ac15c880 0x7f01ac06be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:55.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.360+0000 7f01aa59c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f01ac15c880 0x7f01ac06be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37588/0 (socket says 172.21.15.78:37588) 2023-10-11T09:48:55.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.360+0000 7f01aa59c700 1 -- 172.21.15.78:0/3712841997 learned_addr learned my addr 172.21.15.78:0/3712841997 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:55.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.360+0000 7f01aad9d700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f01ac0cbd30 0x7f01ac0c9b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:45820/0 (socket says 172.21.15.78:45820) 2023-10-11T09:48:55.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.360+0000 7f01aa59c700 1 -- 172.21.15.78:0/3712841997 >> v1:172.21.15.133:6789/0 conn(0x7f01ac0cd210 legacy=0x7f01ac0cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:55.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.360+0000 7f01aa59c700 1 -- 172.21.15.78:0/3712841997 >> v1:172.21.15.78:6789/0 conn(0x7f01ac0cbd30 legacy=0x7f01ac0c9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:55.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.360+0000 7f01aa59c700 1 -- 172.21.15.78:0/3712841997 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ac0cccb0 con 0x7f01ac15c880 2023-10-11T09:48:55.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.360+0000 7f01a959a700 1 -- 172.21.15.78:0/3712841997 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2676125874 0 0) 0x7f01ac0c93a0 con 0x7f01ac0cbd30 2023-10-11T09:48:55.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.360+0000 7f01aa59c700 1 --2- 172.21.15.78:0/3712841997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01ac15c880 0x7f01ac06be60 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f019c009910 tx=0x7f019c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6569929a568adec8 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:55.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.360+0000 7f01a959a700 1 -- 172.21.15.78:0/3712841997 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c00ba60 con 0x7f01ac15c880 2023-10-11T09:48:55.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.360+0000 7f01a959a700 1 -- 172.21.15.78:0/3712841997 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f019c00bbc0 con 0x7f01ac15c880 2023-10-11T09:48:55.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.360+0000 7f01a959a700 1 -- 172.21.15.78:0/3712841997 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c009e60 con 0x7f01ac15c880 2023-10-11T09:48:55.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.364+0000 7f01b0e34700 1 -- 172.21.15.78:0/3712841997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01ac15c880 msgr2=0x7f01ac06be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:55.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.364+0000 7f01b0e34700 1 --2- 172.21.15.78:0/3712841997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01ac15c880 0x7f01ac06be60 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f019c009910 tx=0x7f019c009c20 comp rx=0 tx=0).stop 2023-10-11T09:48:55.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.364+0000 7f01b0e34700 1 -- 172.21.15.78:0/3712841997 shutdown_connections 2023-10-11T09:48:55.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.364+0000 7f01b0e34700 1 --2- 172.21.15.78:0/3712841997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01ac15c880 0x7f01ac06be60 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.364+0000 7f01b0e34700 1 -- 172.21.15.78:0/3712841997 >> 172.21.15.78:0/3712841997 conn(0x7f01ac06aca0 msgr2=0x7f01ac0b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:55.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.364+0000 7f01b0e34700 1 -- 172.21.15.78:0/3712841997 shutdown_connections 2023-10-11T09:48:55.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.364+0000 7f01b0e34700 1 -- 172.21.15.78:0/3712841997 wait complete. 2023-10-11T09:48:55.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.368+0000 7f01b0e34700 1 Processor -- start 2023-10-11T09:48:55.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.368+0000 7f01b0e34700 1 -- start start 2023-10-11T09:48:55.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.368+0000 7f01b0e34700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01ac0cbd30 0x7f01ac15f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:55.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.368+0000 7f01b0e34700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f01ac0cd210 0x7f01ac0b1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:55.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.368+0000 7f01b0e34700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01ac15c880 0x7f01ac13c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:55.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.368+0000 7f01b0e34700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac0b3120 con 0x7f01ac0cbd30 2023-10-11T09:48:55.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.368+0000 7f01b0e34700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac0b4410 con 0x7f01ac15c880 2023-10-11T09:48:55.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.368+0000 7f01b0e34700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac14e650 con 0x7f01ac0cd210 2023-10-11T09:48:55.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.368+0000 7f01aa59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01ac0cbd30 0x7f01ac15f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:55.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.368+0000 7f01aa59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01ac0cbd30 0x7f01ac15f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37594/0 (socket says 172.21.15.78:37594) 2023-10-11T09:48:55.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.368+0000 7f01aa59c700 1 -- 172.21.15.78:0/194942225 learned_addr learned my addr 172.21.15.78:0/194942225 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:55.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.368+0000 7f01aad9d700 1 --2- 172.21.15.78:0/194942225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01ac15c880 0x7f01ac13c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:55.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.368+0000 7f01a9d9b700 1 --2- 172.21.15.78:0/194942225 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f01ac0cd210 0x7f01ac0b1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:55.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.368+0000 7f01aa59c700 1 -- 172.21.15.78:0/194942225 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f01ac0cd210 msgr2=0x7f01ac0b1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:55.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.368+0000 7f01aa59c700 1 --2- 172.21.15.78:0/194942225 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f01ac0cd210 0x7f01ac0b1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.368+0000 7f01aa59c700 1 -- 172.21.15.78:0/194942225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01ac15c880 msgr2=0x7f01ac13c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:55.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.368+0000 7f01aa59c700 1 --2- 172.21.15.78:0/194942225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01ac15c880 0x7f01ac13c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.368+0000 7f01aa59c700 1 -- 172.21.15.78:0/194942225 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f019c009570 con 0x7f01ac0cbd30 2023-10-11T09:48:55.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.372+0000 7f01aa59c700 1 --2- 172.21.15.78:0/194942225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01ac0cbd30 0x7f01ac15f6c0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f019c009c60 tx=0x7f019c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:55.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.372+0000 7f019affd700 1 -- 172.21.15.78:0/194942225 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c0119f0 con 0x7f01ac0cbd30 2023-10-11T09:48:55.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.372+0000 7f019affd700 1 -- 172.21.15.78:0/194942225 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f019c01e070 con 0x7f01ac0cbd30 2023-10-11T09:48:55.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.372+0000 7f01b0e34700 1 -- 172.21.15.78:0/194942225 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01ac0b4660 con 0x7f01ac0cbd30 2023-10-11T09:48:55.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.372+0000 7f019affd700 1 -- 172.21.15.78:0/194942225 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c011d90 con 0x7f01ac0cbd30 2023-10-11T09:48:55.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.372+0000 7f01b0e34700 1 -- 172.21.15.78:0/194942225 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01ac15fc00 con 0x7f01ac0cbd30 2023-10-11T09:48:55.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.376+0000 7f019affd700 1 -- 172.21.15.78:0/194942225 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f019c02d450 con 0x7f01ac0cbd30 2023-10-11T09:48:55.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.380+0000 7f019affd700 1 --2- 172.21.15.78:0/194942225 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f018c065ed0 0x7f018c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:55.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.380+0000 7f019affd700 1 -- 172.21.15.78:0/194942225 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f019c084b10 con 0x7f01ac0cbd30 2023-10-11T09:48:55.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.380+0000 7f01a9d9b700 1 --2- 172.21.15.78:0/194942225 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f018c065ed0 0x7f018c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:55.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.380+0000 7f01b40c2740 1 Processor -- start 2023-10-11T09:48:55.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.380+0000 7f01b40c2740 1 -- start start 2023-10-11T09:48:55.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.380+0000 7f01a9d9b700 1 --2- 172.21.15.78:0/194942225 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f018c065ed0 0x7f018c068380 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f01ac152740 tx=0x7f01a0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:55.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.380+0000 7f01b40c2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xe56eb0 0x11063f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:55.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.380+0000 7f01b40c2740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xe56ae0 0x110ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:55.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.380+0000 7f01b40c2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xe56710 0x110f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:55.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.380+0000 7f01b40c2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x10364a0 con 0xe56eb0 2023-10-11T09:48:55.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.380+0000 7f01b40c2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x1033c30 con 0xe56710 2023-10-11T09:48:55.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.380+0000 7f01b40c2740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x102f520 con 0xe56ae0 2023-10-11T09:48:55.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.380+0000 7f01aa59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xe56eb0 0x11063f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:55.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.380+0000 7f01aa59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xe56eb0 0x11063f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37596/0 (socket says 172.21.15.78:37596) 2023-10-11T09:48:55.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.380+0000 7f01aa59c700 1 -- 172.21.15.78:0/1016956958 learned_addr learned my addr 172.21.15.78:0/1016956958 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:55.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.380+0000 7f01a9d9b700 1 --2- 172.21.15.78:0/1016956958 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xe56ae0 0x110ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:55.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.380+0000 7f01aa59c700 1 -- 172.21.15.78:0/1016956958 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xe56ae0 msgr2=0x110ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:55.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.380+0000 7f01aad9d700 1 --2- 172.21.15.78:0/1016956958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xe56710 0x110f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:55.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.380+0000 7f01aa59c700 1 --2- 172.21.15.78:0/1016956958 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xe56ae0 0x110ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.380+0000 7f01aa59c700 1 -- 172.21.15.78:0/1016956958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xe56710 msgr2=0x110f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:55.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.380+0000 7f01aa59c700 1 --2- 172.21.15.78:0/1016956958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xe56710 0x110f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.380+0000 7f01aa59c700 1 -- 172.21.15.78:0/1016956958 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xea29e0 con 0xe56eb0 2023-10-11T09:48:55.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.380+0000 7f01a9d9b700 1 --2- 172.21.15.78:0/1016956958 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xe56ae0 0x110ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:48:55.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.380+0000 7f01aad9d700 1 --2- 172.21.15.78:0/1016956958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xe56710 0x110f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:48:55.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.380+0000 7f01aa59c700 1 --2- 172.21.15.78:0/1016956958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xe56eb0 0x11063f0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f019c05d620 tx=0x7f019c0092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6efd34d2273a74cb server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:55.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.384+0000 7f018bfff700 1 -- 172.21.15.78:0/1016956958 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c055b70 con 0xe56eb0 2023-10-11T09:48:55.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.384+0000 7f018bfff700 1 -- 172.21.15.78:0/1016956958 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f019c0599a0 con 0xe56eb0 2023-10-11T09:48:55.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.384+0000 7f018bfff700 1 -- 172.21.15.78:0/1016956958 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c05db50 con 0xe56eb0 2023-10-11T09:48:55.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.384+0000 7f01b40c2740 1 -- 172.21.15.78:0/1016956958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xe56eb0 msgr2=0x11063f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:55.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.384+0000 7f01b40c2740 1 --2- 172.21.15.78:0/1016956958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xe56eb0 0x11063f0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f019c05d620 tx=0x7f019c0092e0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.384+0000 7f01b40c2740 1 -- 172.21.15.78:0/1016956958 shutdown_connections 2023-10-11T09:48:55.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.384+0000 7f01b40c2740 1 --2- 172.21.15.78:0/1016956958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xe56710 0x110f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.384+0000 7f01b40c2740 1 --2- 172.21.15.78:0/1016956958 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xe56ae0 0x110ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.384+0000 7f01b40c2740 1 --2- 172.21.15.78:0/1016956958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xe56eb0 0x11063f0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.384+0000 7f01b40c2740 1 -- 172.21.15.78:0/1016956958 >> 172.21.15.78:0/1016956958 conn(0x10befa0 msgr2=0x10f0210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:55.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.384+0000 7f01b40c2740 1 -- 172.21.15.78:0/1016956958 shutdown_connections 2023-10-11T09:48:55.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.384+0000 7f01b40c2740 1 -- 172.21.15.78:0/1016956958 wait complete. 2023-10-11T09:48:55.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.384+0000 7f01b40c2740 1 -- start start 2023-10-11T09:48:55.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.384+0000 7f01b40c2740 1 Processor -- start 2023-10-11T09:48:55.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.384+0000 7f01b40c2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xe56710 0x10c37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:55.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.384+0000 7f01b40c2740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xe56ae0 0x10c4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:55.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.384+0000 7f01b40c2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xe56eb0 0x11184d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:55.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.384+0000 7f01b40c2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0xff2b00 con 0xe56eb0 2023-10-11T09:48:55.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.384+0000 7f01b40c2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0xff27a0 con 0xe56710 2023-10-11T09:48:55.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.384+0000 7f01b40c2740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0xedbb20 con 0xe56ae0 2023-10-11T09:48:55.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.384+0000 7f01b40c2740 10 client.0 _ll_register_callbacks cb 0x1028e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:48:55.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.388+0000 7f01aad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xe56eb0 0x11184d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:55.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.388+0000 7f01aad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xe56eb0 0x11184d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37608/0 (socket says 172.21.15.78:37608) 2023-10-11T09:48:55.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.388+0000 7f01aad9d700 1 -- 172.21.15.78:0/3324253509 learned_addr learned my addr 172.21.15.78:0/3324253509 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:55.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.388+0000 7f01aa59c700 1 --2- 172.21.15.78:0/3324253509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xe56710 0x10c37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:55.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.388+0000 7f01a9d9b700 1 --2- 172.21.15.78:0/3324253509 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xe56ae0 0x10c4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:55.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.388+0000 7f01aad9d700 1 -- 172.21.15.78:0/3324253509 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xe56ae0 msgr2=0x10c4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:55.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.388+0000 7f01aad9d700 1 --2- 172.21.15.78:0/3324253509 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xe56ae0 0x10c4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.388+0000 7f01aad9d700 1 -- 172.21.15.78:0/3324253509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xe56710 msgr2=0x10c37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:55.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.388+0000 7f01aad9d700 1 --2- 172.21.15.78:0/3324253509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xe56710 0x10c37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.388+0000 7f01aad9d700 1 -- 172.21.15.78:0/3324253509 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01ac15fc00 con 0xe56eb0 2023-10-11T09:48:55.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.388+0000 7f01aa59c700 1 --2- 172.21.15.78:0/3324253509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xe56710 0x10c37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:48:55.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.388+0000 7f01aad9d700 1 --2- 172.21.15.78:0/3324253509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xe56eb0 0x11184d0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f01a40078c0 tx=0x7f01a4007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e87ae7d4182c3fe2 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:55.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.388+0000 7f01897fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:48:55.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.388+0000 7f01897fa700 1 -- 172.21.15.78:0/3324253509 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a4010870 con 0xe56eb0 2023-10-11T09:48:55.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.388+0000 7f01b40c2740 1 -- 172.21.15.78:0/3324253509 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x10285a0 con 0xe56eb0 2023-10-11T09:48:55.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.388+0000 7f01897fa700 1 -- 172.21.15.78:0/3324253509 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f01a40109d0 con 0xe56eb0 2023-10-11T09:48:55.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.388+0000 7f01897fa700 1 -- 172.21.15.78:0/3324253509 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01a4010b90 con 0xe56eb0 2023-10-11T09:48:55.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.388+0000 7f01b40c2740 10 client.7761 fetch_fsmap learned FSMap version 71 2023-10-11T09:48:55.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.388+0000 7f01b40c2740 1 -- 172.21.15.78:0/3324253509 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0xff5750 con 0xe56eb0 2023-10-11T09:48:55.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.388+0000 7f01897fa700 1 -- 172.21.15.78:0/3324253509 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x10285a0 con 0xe56eb0 2023-10-11T09:48:55.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.388+0000 7f01897fa700 1 -- 172.21.15.78:0/3324253509 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 71) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f01a4010cf0 con 0xe56eb0 2023-10-11T09:48:55.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.388+0000 7f01b40c2740 10 client.7761 fetch_fsmap finished waiting for FSMap version 71 2023-10-11T09:48:55.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.388+0000 7f01b40c2740 10 client.7761 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-11T09:48:55.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.388+0000 7f01b40c2740 20 client.7761 populate_metadata read hostname 'smithi078' 2023-10-11T09:48:55.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.388+0000 7f01b40c2740 1 --2- 172.21.15.78:0/3324253509 >> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] conn(0xea1790 0x10ecab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:55.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.388+0000 7f01b40c2740 4 client.7761 mds_command: new command op to 7725 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-11T09:48:55.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.388+0000 7f01b40c2740 1 -- 172.21.15.78:0/3324253509 --> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x10282c0 con 0xea1790 2023-10-11T09:48:55.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.388+0000 7f01aa59c700 1 --2- 172.21.15.78:0/3324253509 >> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] conn(0xea1790 0x10ecab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:55.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.392+0000 7f01aa59c700 1 --2- 172.21.15.78:0/3324253509 >> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] conn(0xea1790 0x10ecab0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e816901ea2490484 server_cookie=d5a1c11006906a3f in_seq=0 out_seq=0 2023-10-11T09:48:55.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.392+0000 7f01897fa700 10 client.7761 ms_handle_connect on v2:172.21.15.133:6833/3922016223 2023-10-11T09:48:55.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.392+0000 7f01897fa700 1 -- 172.21.15.78:0/3324253509 <== mds.0 v2:172.21.15.133:6833/3922016223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x10282c0 con 0xea1790 2023-10-11T09:48:55.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.392+0000 7f01897fa700 10 client.7761 handle_command_reply: tid=0 2023-10-11T09:48:55.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.392+0000 7f01b40c2740 1 client.7761 shutdown 2023-10-11T09:48:55.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.392+0000 7f01b40c2740 1 -- 172.21.15.78:0/3324253509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xe56eb0 msgr2=0x11184d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:55.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.392+0000 7f01b40c2740 1 --2- 172.21.15.78:0/3324253509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xe56eb0 0x11184d0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f01a40078c0 tx=0x7f01a4007bd0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.392+0000 7f01b40c2740 1 -- 172.21.15.78:0/3324253509 shutdown_connections 2023-10-11T09:48:55.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.392+0000 7f01b40c2740 1 --2- 172.21.15.78:0/3324253509 >> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] conn(0xea1790 0x10ecab0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.392+0000 7f01b40c2740 1 --2- 172.21.15.78:0/3324253509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xe56eb0 0x11184d0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.392+0000 7f01b40c2740 1 --2- 172.21.15.78:0/3324253509 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xe56ae0 0x10c4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.392+0000 7f01b40c2740 1 --2- 172.21.15.78:0/3324253509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xe56710 0x10c37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.392+0000 7f01897fa700 0 client.7761 ms_handle_reset on v2:172.21.15.133:6833/3922016223 2023-10-11T09:48:55.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.392+0000 7f01b40c2740 1 -- 172.21.15.78:0/3324253509 >> 172.21.15.78:0/3324253509 conn(0x10befa0 msgr2=0x110e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:55.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.392+0000 7f01b40c2740 1 -- 172.21.15.78:0/3324253509 shutdown_connections 2023-10-11T09:48:55.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.392+0000 7f01b40c2740 1 -- 172.21.15.78:0/3324253509 wait complete. 2023-10-11T09:48:55.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.396+0000 7f01b40c2740 20 client.7761 trim_cache size 0 max 16384 2023-10-11T09:48:55.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.416+0000 7f01b40c2740 1 Processor -- start 2023-10-11T09:48:55.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.416+0000 7f01b40c2740 1 -- start start 2023-10-11T09:48:55.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.416+0000 7f01b40c2740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xe56710 0x10efbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:55.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.416+0000 7f01b40c2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xe56ae0 0x10c3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:55.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.416+0000 7f01b40c2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xe56eb0 0x10c3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:55.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.416+0000 7f01b40c2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x108e250 con 0xe56eb0 2023-10-11T09:48:55.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.416+0000 7f01b40c2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0xfd5d90 con 0xe56ae0 2023-10-11T09:48:55.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.416+0000 7f01b40c2740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0xfdcda0 con 0xe56710 2023-10-11T09:48:55.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.416+0000 7f01aad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xe56eb0 0x10c3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:55.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.416+0000 7f01aad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xe56eb0 0x10c3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37616/0 (socket says 172.21.15.78:37616) 2023-10-11T09:48:55.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.416+0000 7f01aad9d700 1 -- 172.21.15.78:0/398361903 learned_addr learned my addr 172.21.15.78:0/398361903 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:55.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.416+0000 7f01a9d9b700 1 --2- 172.21.15.78:0/398361903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xe56ae0 0x10c3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:55.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.416+0000 7f01aa59c700 1 --2- 172.21.15.78:0/398361903 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xe56710 0x10efbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:55.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.416+0000 7f01aad9d700 1 -- 172.21.15.78:0/398361903 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xe56710 msgr2=0x10efbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:55.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.416+0000 7f01aad9d700 1 --2- 172.21.15.78:0/398361903 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xe56710 0x10efbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.416+0000 7f01aad9d700 1 -- 172.21.15.78:0/398361903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xe56ae0 msgr2=0x10c3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:55.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.416+0000 7f01aad9d700 1 --2- 172.21.15.78:0/398361903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xe56ae0 0x10c3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.416+0000 7f01aad9d700 1 -- 172.21.15.78:0/398361903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xfdb980 con 0xe56eb0 2023-10-11T09:48:55.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.416+0000 7f01aad9d700 1 --2- 172.21.15.78:0/398361903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xe56eb0 0x10c3680 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f01a4001320 tx=0x7f01a400c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b94b3b85129b5d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:55.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.416+0000 7f0198ff9700 1 -- 172.21.15.78:0/398361903 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a402a070 con 0xe56eb0 2023-10-11T09:48:55.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.416+0000 7f0198ff9700 1 -- 172.21.15.78:0/398361903 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a401c490 con 0xe56eb0 2023-10-11T09:48:55.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.416+0000 7f0198ff9700 1 -- 172.21.15.78:0/398361903 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a401c8d0 con 0xe56eb0 2023-10-11T09:48:55.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.420+0000 7f01b40c2740 1 -- 172.21.15.78:0/398361903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xe56eb0 msgr2=0x10c3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:55.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.420+0000 7f01b40c2740 1 --2- 172.21.15.78:0/398361903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xe56eb0 0x10c3680 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f01a4001320 tx=0x7f01a400c000 comp rx=0 tx=0).stop 2023-10-11T09:48:55.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.420+0000 7f01b40c2740 1 -- 172.21.15.78:0/398361903 shutdown_connections 2023-10-11T09:48:55.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.420+0000 7f01b40c2740 1 --2- 172.21.15.78:0/398361903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xe56eb0 0x10c3680 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.420+0000 7f01b40c2740 1 --2- 172.21.15.78:0/398361903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xe56ae0 0x10c3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.420+0000 7f01b40c2740 1 --2- 172.21.15.78:0/398361903 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xe56710 0x10efbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.420+0000 7f01b40c2740 1 -- 172.21.15.78:0/398361903 >> 172.21.15.78:0/398361903 conn(0x10befa0 msgr2=0x10c5580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:55.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.420+0000 7f01b40c2740 1 -- 172.21.15.78:0/398361903 shutdown_connections 2023-10-11T09:48:55.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.420+0000 7f01b40c2740 1 -- 172.21.15.78:0/398361903 wait complete. 2023-10-11T09:48:55.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.420+0000 7f01b40c2740 1 -- start start 2023-10-11T09:48:55.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.420+0000 7f01b40c2740 1 Processor -- start 2023-10-11T09:48:55.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.424+0000 7f01b40c2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xe56710 0x116a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:55.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.424+0000 7f01b40c2740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xe56ae0 0x1119d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:55.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.424+0000 7f01b40c2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xe56eb0 0x111a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:55.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.424+0000 7f01b40c2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x1119910 con 0xe56eb0 2023-10-11T09:48:55.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.424+0000 7f01b40c2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x10eb530 con 0xe56710 2023-10-11T09:48:55.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.424+0000 7f01b40c2740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x10eb290 con 0xe56ae0 2023-10-11T09:48:55.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.424+0000 7f01b40c2740 10 client.0 _ll_register_callbacks cb 0x7f019c006d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:48:55.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.424+0000 7f01aad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xe56eb0 0x111a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:55.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.424+0000 7f01aad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xe56eb0 0x111a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37620/0 (socket says 172.21.15.78:37620) 2023-10-11T09:48:55.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.424+0000 7f01aad9d700 1 -- 172.21.15.78:0/1621098976 learned_addr learned my addr 172.21.15.78:0/1621098976 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:55.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.424+0000 7f01a9d9b700 1 --2- 172.21.15.78:0/1621098976 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xe56ae0 0x1119d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:55.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.424+0000 7f01aa59c700 1 --2- 172.21.15.78:0/1621098976 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xe56710 0x116a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:55.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.424+0000 7f01aad9d700 1 -- 172.21.15.78:0/1621098976 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xe56ae0 msgr2=0x1119d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:55.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.424+0000 7f01aad9d700 1 --2- 172.21.15.78:0/1621098976 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xe56ae0 0x1119d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.424+0000 7f01aad9d700 1 -- 172.21.15.78:0/1621098976 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xe56710 msgr2=0x116a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:55.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.424+0000 7f01aad9d700 1 --2- 172.21.15.78:0/1621098976 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xe56710 0x116a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.424+0000 7f01aad9d700 1 -- 172.21.15.78:0/1621098976 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0xff5750 con 0xe56eb0 2023-10-11T09:48:55.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.424+0000 7f01aa59c700 1 --2- 172.21.15.78:0/1621098976 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xe56710 0x116a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:48:55.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.424+0000 7f01a9d9b700 1 --2- 172.21.15.78:0/1621098976 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xe56ae0 0x1119d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:48:55.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.428+0000 7f01aad9d700 1 --2- 172.21.15.78:0/1621098976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xe56eb0 0x111a280 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f01a4009010 tx=0x7f01a4001400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2b6966517c0f34b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:55.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.428+0000 7f01897fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:48:55.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.428+0000 7f01897fa700 1 -- 172.21.15.78:0/1621098976 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a402a040 con 0xe56eb0 2023-10-11T09:48:55.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.428+0000 7f01897fa700 1 -- 172.21.15.78:0/1621098976 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f01a4016af0 con 0xe56eb0 2023-10-11T09:48:55.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.428+0000 7f01b40c2740 1 -- 172.21.15.78:0/1621098976 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f019c0599a0 con 0xe56eb0 2023-10-11T09:48:55.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.428+0000 7f01897fa700 1 -- 172.21.15.78:0/1621098976 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01a4016cb0 con 0xe56eb0 2023-10-11T09:48:55.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.428+0000 7f01b40c2740 10 client.7767 fetch_fsmap learned FSMap version 71 2023-10-11T09:48:55.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.428+0000 7f01b40c2740 1 -- 172.21.15.78:0/1621098976 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0xfcfc50 con 0xe56eb0 2023-10-11T09:48:55.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.428+0000 7f01897fa700 1 -- 172.21.15.78:0/1621098976 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f019c0599a0 con 0xe56eb0 2023-10-11T09:48:55.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.428+0000 7f01897fa700 1 -- 172.21.15.78:0/1621098976 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 71) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f01a4042070 con 0xe56eb0 2023-10-11T09:48:55.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.428+0000 7f01b40c2740 10 client.7767 fetch_fsmap finished waiting for FSMap version 71 2023-10-11T09:48:55.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.428+0000 7f01b40c2740 10 client.7767 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-11T09:48:55.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.428+0000 7f01b40c2740 20 client.7767 populate_metadata read hostname 'smithi078' 2023-10-11T09:48:55.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.428+0000 7f01b40c2740 1 --2- 172.21.15.78:0/1621098976 >> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] conn(0x111ce90 0x111d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:55.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.428+0000 7f01b40c2740 4 client.7767 mds_command: new command op to 7725 tid=0[{"prefix": "status"}] 2023-10-11T09:48:55.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.428+0000 7f01b40c2740 1 -- 172.21.15.78:0/1621098976 --> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] -- command(tid 0: {"prefix": "status"}) v1 -- 0xeae4e0 con 0x111ce90 2023-10-11T09:48:55.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.428+0000 7f01aa59c700 1 --2- 172.21.15.78:0/1621098976 >> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] conn(0x111ce90 0x111d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:55.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.428+0000 7f01897fa700 10 client.7767 ms_handle_connect on v2:172.21.15.133:6833/3922016223 2023-10-11T09:48:55.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.428+0000 7f01aa59c700 1 --2- 172.21.15.78:0/1621098976 >> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] conn(0x111ce90 0x111d260 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f0910361bed01ee server_cookie=4b45a558e5bfaa89 in_seq=0 out_seq=0 2023-10-11T09:48:55.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.432+0000 7f01897fa700 1 -- 172.21.15.78:0/1621098976 <== mds.0 v2:172.21.15.133:6833/3922016223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0xeae4e0 con 0x111ce90 2023-10-11T09:48:55.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.432+0000 7f01897fa700 10 client.7767 handle_command_reply: tid=0 2023-10-11T09:48:55.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.432+0000 7f01b40c2740 1 client.7767 shutdown 2023-10-11T09:48:55.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.432+0000 7f01b40c2740 1 -- 172.21.15.78:0/1621098976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xe56eb0 msgr2=0x111a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:55.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.432+0000 7f01b40c2740 1 --2- 172.21.15.78:0/1621098976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xe56eb0 0x111a280 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f01a4009010 tx=0x7f01a4001400 comp rx=0 tx=0).stop 2023-10-11T09:48:55.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.432+0000 7f01b40c2740 1 -- 172.21.15.78:0/1621098976 shutdown_connections 2023-10-11T09:48:55.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.432+0000 7f01b40c2740 1 --2- 172.21.15.78:0/1621098976 >> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] conn(0x111ce90 0x111d260 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.432+0000 7f01b40c2740 1 --2- 172.21.15.78:0/1621098976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xe56eb0 0x111a280 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.432+0000 7f01b40c2740 1 --2- 172.21.15.78:0/1621098976 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xe56ae0 0x1119d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.432+0000 7f01b40c2740 1 --2- 172.21.15.78:0/1621098976 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xe56710 0x116a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.432+0000 7f01b40c2740 1 -- 172.21.15.78:0/1621098976 >> 172.21.15.78:0/1621098976 conn(0x10befa0 msgr2=0x10ecab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:55.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.432+0000 7f01897fa700 0 client.7767 ms_handle_reset on v2:172.21.15.133:6833/3922016223 2023-10-11T09:48:55.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.432+0000 7f01b40c2740 1 -- 172.21.15.78:0/1621098976 shutdown_connections 2023-10-11T09:48:55.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.432+0000 7f01b40c2740 1 -- 172.21.15.78:0/1621098976 wait complete. 2023-10-11T09:48:55.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.432+0000 7f01b40c2740 20 client.7767 trim_cache size 0 max 16384 2023-10-11T09:48:55.444 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:48:55.445 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "dee99db9-7a72-406f-a1e2-917943f2b13d", 2023-10-11T09:48:55.445 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2023-10-11T09:48:55.445 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 7725, 2023-10-11T09:48:55.445 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2023-10-11T09:48:55.445 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2023-10-11T09:48:55.446 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2023-10-11T09:48:55.446 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 4.5966583270000001, 2023-10-11T09:48:55.446 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 71, 2023-10-11T09:48:55.446 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 92, 2023-10-11T09:48:55.446 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 92, 2023-10-11T09:48:55.447 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 4.6661431909999997 2023-10-11T09:48:55.447 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:48:55.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.432+0000 7f01b0e34700 1 -- 172.21.15.78:0/194942225 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f018c065ed0 msgr2=0x7f018c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:55.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.432+0000 7f01b0e34700 1 --2- 172.21.15.78:0/194942225 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f018c065ed0 0x7f018c068380 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f01ac152740 tx=0x7f01a0009040 comp rx=0 tx=0).stop 2023-10-11T09:48:55.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.432+0000 7f01b0e34700 1 -- 172.21.15.78:0/194942225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01ac0cbd30 msgr2=0x7f01ac15f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:55.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.432+0000 7f01b0e34700 1 --2- 172.21.15.78:0/194942225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01ac0cbd30 0x7f01ac15f6c0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f019c009c60 tx=0x7f019c004770 comp rx=0 tx=0).stop 2023-10-11T09:48:55.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.432+0000 7f01b0e34700 1 -- 172.21.15.78:0/194942225 shutdown_connections 2023-10-11T09:48:55.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.432+0000 7f01b0e34700 1 --2- 172.21.15.78:0/194942225 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f018c065ed0 0x7f018c068380 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.432+0000 7f01b0e34700 1 --2- 172.21.15.78:0/194942225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01ac15c880 0x7f01ac13c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.432+0000 7f01b0e34700 1 --2- 172.21.15.78:0/194942225 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f01ac0cd210 0x7f01ac0b1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.432+0000 7f01b0e34700 1 --2- 172.21.15.78:0/194942225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01ac0cbd30 0x7f01ac15f6c0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.432+0000 7f01b0e34700 1 -- 172.21.15.78:0/194942225 >> 172.21.15.78:0/194942225 conn(0x7f01ac06aca0 msgr2=0x7f01ac1606f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:55.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.432+0000 7f01b0e34700 1 -- 172.21.15.78:0/194942225 shutdown_connections 2023-10-11T09:48:55.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.432+0000 7f01b0e34700 1 -- 172.21.15.78:0/194942225 wait complete. 2023-10-11T09:48:55.460 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:48:55.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.632+0000 7fae19039700 1 Processor -- start 2023-10-11T09:48:55.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.632+0000 7fae19039700 1 -- start start 2023-10-11T09:48:55.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.632+0000 7fae19039700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fae140cda10 0x7fae140caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:55.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.632+0000 7fae19039700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae140cdde0 0x7fae14164610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:55.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.632+0000 7fae19039700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae14165390 con 0x7fae14164b50 2023-10-11T09:48:55.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.632+0000 7fae19039700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae140ca440 con 0x7fae140cda10 2023-10-11T09:48:55.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.632+0000 7fae19039700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae140ca720 con 0x7fae140cdde0 2023-10-11T09:48:55.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.632+0000 7fae137fe700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae140cdde0 0x7fae14164610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:55.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.632+0000 7fae137fe700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae140cdde0 0x7fae14164610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:45986/0 (socket says 172.21.15.78:45986) 2023-10-11T09:48:55.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.632+0000 7fae137fe700 1 -- 172.21.15.78:0/2821166904 learned_addr learned my addr 172.21.15.78:0/2821166904 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:55.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.632+0000 7fae13fff700 1 --2- 172.21.15.78:0/2821166904 >> v2:172.21.15.133:3300/0 conn(0x7fae140cda10 0x7fae140caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:55.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.636+0000 7fae137fe700 1 -- 172.21.15.78:0/2821166904 >> v2:172.21.15.133:3300/0 conn(0x7fae140cda10 msgr2=0x7fae140caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:55.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.636+0000 7fae137fe700 1 --2- 172.21.15.78:0/2821166904 >> v2:172.21.15.133:3300/0 conn(0x7fae140cda10 0x7fae140caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.636+0000 7fae137fe700 1 -- 172.21.15.78:0/2821166904 >> v1:172.21.15.133:6789/0 conn(0x7fae14164b50 legacy=0x7fae1406bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:55.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.636+0000 7fae137fe700 1 -- 172.21.15.78:0/2821166904 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae140caa00 con 0x7fae140cdde0 2023-10-11T09:48:55.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.636+0000 7fae137fe700 1 --2- 172.21.15.78:0/2821166904 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae140cdde0 0x7fae14164610 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fae08009910 tx=0x7fae08009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9375ce5568a5c804 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:55.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.636+0000 7fae12ffd700 1 -- 172.21.15.78:0/2821166904 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae08011830 con 0x7fae140cdde0 2023-10-11T09:48:55.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.636+0000 7fae12ffd700 1 -- 172.21.15.78:0/2821166904 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae08011990 con 0x7fae140cdde0 2023-10-11T09:48:55.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.636+0000 7fae12ffd700 1 -- 172.21.15.78:0/2821166904 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae08011d40 con 0x7fae140cdde0 2023-10-11T09:48:55.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.636+0000 7fae19039700 1 -- 172.21.15.78:0/2821166904 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae140cdde0 msgr2=0x7fae14164610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:55.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.636+0000 7fae19039700 1 --2- 172.21.15.78:0/2821166904 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae140cdde0 0x7fae14164610 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fae08009910 tx=0x7fae08009c20 comp rx=0 tx=0).stop 2023-10-11T09:48:55.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.636+0000 7fae19039700 1 -- 172.21.15.78:0/2821166904 shutdown_connections 2023-10-11T09:48:55.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.636+0000 7fae19039700 1 --2- 172.21.15.78:0/2821166904 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae140cdde0 0x7fae14164610 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.636+0000 7fae19039700 1 --2- 172.21.15.78:0/2821166904 >> v2:172.21.15.133:3300/0 conn(0x7fae140cda10 0x7fae140caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.636+0000 7fae19039700 1 -- 172.21.15.78:0/2821166904 >> 172.21.15.78:0/2821166904 conn(0x7fae140cea60 msgr2=0x7fae140cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:55.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.636+0000 7fae19039700 1 -- 172.21.15.78:0/2821166904 shutdown_connections 2023-10-11T09:48:55.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.636+0000 7fae19039700 1 -- 172.21.15.78:0/2821166904 wait complete. 2023-10-11T09:48:55.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.640+0000 7fae19039700 1 Processor -- start 2023-10-11T09:48:55.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.640+0000 7fae19039700 1 -- start start 2023-10-11T09:48:55.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.640+0000 7fae19039700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae140cda10 0x7fae14161050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:55.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.640+0000 7fae19039700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae140cdde0 0x7fae14163cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:55.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.640+0000 7fae19039700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae14164b50 0x7fae1413bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:55.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.640+0000 7fae19039700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae14164230 con 0x7fae140cdde0 2023-10-11T09:48:55.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.640+0000 7fae19039700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1413c220 con 0x7fae140cda10 2023-10-11T09:48:55.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.640+0000 7fae19039700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae14162340 con 0x7fae14164b50 2023-10-11T09:48:55.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.640+0000 7fae137fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae140cdde0 0x7fae14163cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:55.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.644+0000 7fae137fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae140cdde0 0x7fae14163cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37636/0 (socket says 172.21.15.78:37636) 2023-10-11T09:48:55.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.644+0000 7fae137fe700 1 -- 172.21.15.78:0/324755005 learned_addr learned my addr 172.21.15.78:0/324755005 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:55.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.644+0000 7fae18838700 1 --2- 172.21.15.78:0/324755005 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae14164b50 0x7fae1413bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:55.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.644+0000 7fae13fff700 1 --2- 172.21.15.78:0/324755005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae140cda10 0x7fae14161050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:55.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.644+0000 7fae137fe700 1 -- 172.21.15.78:0/324755005 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae14164b50 msgr2=0x7fae1413bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:55.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.644+0000 7fae137fe700 1 --2- 172.21.15.78:0/324755005 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae14164b50 0x7fae1413bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.644+0000 7fae137fe700 1 -- 172.21.15.78:0/324755005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae140cda10 msgr2=0x7fae14161050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:55.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.644+0000 7fae137fe700 1 --2- 172.21.15.78:0/324755005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae140cda10 0x7fae14161050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.644+0000 7fae137fe700 1 -- 172.21.15.78:0/324755005 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae08009570 con 0x7fae140cdde0 2023-10-11T09:48:55.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.644+0000 7fae137fe700 1 --2- 172.21.15.78:0/324755005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae140cdde0 0x7fae14163cf0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fae140ca860 tx=0x7fae08004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:55.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.644+0000 7fae10ff9700 1 -- 172.21.15.78:0/324755005 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae08004510 con 0x7fae140cdde0 2023-10-11T09:48:55.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.644+0000 7fae10ff9700 1 -- 172.21.15.78:0/324755005 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae080036c0 con 0x7fae140cdde0 2023-10-11T09:48:55.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.644+0000 7fae10ff9700 1 -- 172.21.15.78:0/324755005 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae08003a70 con 0x7fae140cdde0 2023-10-11T09:48:55.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.644+0000 7fae19039700 1 -- 172.21.15.78:0/324755005 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae140ce3b0 con 0x7fae140cdde0 2023-10-11T09:48:55.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.644+0000 7fae19039700 1 -- 172.21.15.78:0/324755005 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae14161a10 con 0x7fae140cdde0 2023-10-11T09:48:55.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.644+0000 7fae19039700 1 -- 172.21.15.78:0/324755005 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae14065d40 con 0x7fae140cdde0 2023-10-11T09:48:55.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.648+0000 7fae10ff9700 1 -- 172.21.15.78:0/324755005 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fae08003bd0 con 0x7fae140cdde0 2023-10-11T09:48:55.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.648+0000 7fae10ff9700 1 --2- 172.21.15.78:0/324755005 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fae00065ed0 0x7fae00068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:55.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.648+0000 7fae10ff9700 1 -- 172.21.15.78:0/324755005 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6724+0+0 (secure 0 0 0) 0x7fae08021030 con 0x7fae140cdde0 2023-10-11T09:48:55.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.648+0000 7fae13fff700 1 --2- 172.21.15.78:0/324755005 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fae00065ed0 0x7fae00068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:55.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.648+0000 7fae13fff700 1 --2- 172.21.15.78:0/324755005 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fae00065ed0 0x7fae00068380 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fae04009b50 tx=0x7fae04009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:55.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.656+0000 7fae10ff9700 1 -- 172.21.15.78:0/324755005 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fae080523c0 con 0x7fae140cdde0 2023-10-11T09:48:55.838 INFO:tasks.ceph.mds.b.smithi133.stderr:2023-10-11T09:48:55.833+0000 7f576111f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-11T09:48:55.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.868+0000 7fae19039700 1 -- 172.21.15.78:0/324755005 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae14162480 con 0x7fae140cdde0 2023-10-11T09:48:55.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.868+0000 7fae10ff9700 1 -- 172.21.15.78:0/324755005 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4657 (secure 0 0 0) 0x7fae0802b090 con 0x7fae140cdde0 2023-10-11T09:48:55.875 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:48:55.876 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":71,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6581,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/238181631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":238181631},{"type":"v1","addr":"172.21.15.78:6837","nonce":238181631}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":7378,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/2416080754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2416080754},{"type":"v1","addr":"172.21.15.133:6834","nonce":2416080754}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":7384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/3016626706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3016626706},{"type":"v1","addr":"172.21.15.78:6836","nonce":3016626706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:48:45.351822+0000","modified":"2023-10-11T09:48:53.854831+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7725":{"gid":7725,"name":"b","rank":0,"incarnation":68,"state":"up:active","state_seq":4,"addr":"172.21.15.133:6835/3922016223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":3922016223},{"type":"v1","addr":"172.21.15.133:6835","nonce":3922016223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12,13],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-11T09:48:55.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.872+0000 7fae19039700 1 -- 172.21.15.78:0/324755005 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fae00065ed0 msgr2=0x7fae00068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:55.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.872+0000 7fae19039700 1 --2- 172.21.15.78:0/324755005 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fae00065ed0 0x7fae00068380 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fae04009b50 tx=0x7fae04009240 comp rx=0 tx=0).stop 2023-10-11T09:48:55.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.872+0000 7fae19039700 1 -- 172.21.15.78:0/324755005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae140cdde0 msgr2=0x7fae14163cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:55.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.872+0000 7fae19039700 1 --2- 172.21.15.78:0/324755005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae140cdde0 0x7fae14163cf0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fae140ca860 tx=0x7fae08004770 comp rx=0 tx=0).stop 2023-10-11T09:48:55.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.872+0000 7fae19039700 1 -- 172.21.15.78:0/324755005 shutdown_connections 2023-10-11T09:48:55.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.872+0000 7fae19039700 1 --2- 172.21.15.78:0/324755005 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fae00065ed0 0x7fae00068380 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.872+0000 7fae19039700 1 --2- 172.21.15.78:0/324755005 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae14164b50 0x7fae1413bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.872+0000 7fae19039700 1 --2- 172.21.15.78:0/324755005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae140cdde0 0x7fae14163cf0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.872+0000 7fae19039700 1 --2- 172.21.15.78:0/324755005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae140cda10 0x7fae14161050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:55.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.872+0000 7fae19039700 1 -- 172.21.15.78:0/324755005 >> 172.21.15.78:0/324755005 conn(0x7fae140cea60 msgr2=0x7fae140b0550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:55.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.872+0000 7fae19039700 1 -- 172.21.15.78:0/324755005 shutdown_connections 2023-10-11T09:48:55.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:55.872+0000 7fae19039700 1 -- 172.21.15.78:0/324755005 wait complete. 2023-10-11T09:48:55.880 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 71 2023-10-11T09:48:55.897 INFO:tasks.cephfs.test_data_scan:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 4, "epoch": 71, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 4, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-11T09:48:45.351822+0000", "damaged": [], "data_pools": [ 12, 13 ], "enabled": true, "epoch": 71, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_7725": { "addr": "172.21.15.133:6835/3922016223", "addrs": { "addrvec": [ { "addr": "172.21.15.133:6833", "nonce": 3922016223, "type": "v2" }, { "addr": "172.21.15.133:6835", "nonce": 3922016223, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7725, "incarnation": 68, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:active", "state_seq": 4 } }, "last_failure": 0, "last_failure_osd_epoch": 92, "max_file_size": 1099511627776, "max_mds": 1, "metadata_pool": 11, "modified": "2023-10-11T09:48:53.854831+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7725 } } } ], "standbys": [ { "addr": "172.21.15.78:6837/238181631", "addrs": { "addrvec": [ { "addr": "172.21.15.78:6835", "nonce": 238181631, "type": "v2" }, { "addr": "172.21.15.78:6837", "nonce": 238181631, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 67, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6581, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.133:6834/2416080754", "addrs": { "addrvec": [ { "addr": "172.21.15.133:6832", "nonce": 2416080754, "type": "v2" }, { "addr": "172.21.15.133:6834", "nonce": 2416080754, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 67, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7378, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.78:6836/3016626706", "addrs": { "addrvec": [ { "addr": "172.21.15.78:6834", "nonce": 3016626706, "type": "v2" }, { "addr": "172.21.15.78:6836", "nonce": 3016626706, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 67, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7384, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-11T09:48:55.897 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-11T09:48:56.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.068+0000 7ff44d6c6700 1 Processor -- start 2023-10-11T09:48:56.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.068+0000 7ff44d6c6700 1 -- start start 2023-10-11T09:48:56.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.072+0000 7ff44d6c6700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff448071040 0x7ff4480cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:56.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.072+0000 7ff44d6c6700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4480ce330 0x7ff44806be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:56.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.072+0000 7ff44d6c6700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff44806ffa0 con 0x7ff448160d40 2023-10-11T09:48:56.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.072+0000 7ff44d6c6700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff448070c20 con 0x7ff448071040 2023-10-11T09:48:56.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.072+0000 7ff44d6c6700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff448161110 con 0x7ff4480ce330 2023-10-11T09:48:56.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.072+0000 7ff43e7fc700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff448071040 0x7ff4480cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:56.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.072+0000 7ff4477fe700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4480ce330 0x7ff44806be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:56.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.072+0000 7ff446ffd700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff448160d40 0x7ff4480cd280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:36044/0 (socket says 172.21.15.78:36044) 2023-10-11T09:48:56.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.072+0000 7ff446ffd700 1 -- 172.21.15.78:0/1470219114 learned_addr learned my addr 172.21.15.78:0/1470219114 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:56.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.072+0000 7ff4477fe700 1 -- 172.21.15.78:0/1470219114 >> v2:172.21.15.133:3300/0 conn(0x7ff448071040 msgr2=0x7ff4480cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:56.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.072+0000 7ff4477fe700 1 --2- 172.21.15.78:0/1470219114 >> v2:172.21.15.133:3300/0 conn(0x7ff448071040 0x7ff4480cddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:56.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.072+0000 7ff4477fe700 1 -- 172.21.15.78:0/1470219114 >> v1:172.21.15.133:6789/0 conn(0x7ff448160d40 legacy=0x7ff4480cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:56.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.072+0000 7ff4477fe700 1 -- 172.21.15.78:0/1470219114 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff448070280 con 0x7ff4480ce330 2023-10-11T09:48:56.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.072+0000 7ff4477fe700 1 --2- 172.21.15.78:0/1470219114 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4480ce330 0x7ff44806be60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff440009d90 tx=0x7ff44000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=670857540d1aec36 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:56.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.072+0000 7ff4467fc700 1 -- 172.21.15.78:0/1470219114 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44000e070 con 0x7ff4480ce330 2023-10-11T09:48:56.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.072+0000 7ff4467fc700 1 -- 172.21.15.78:0/1470219114 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff44000bb20 con 0x7ff4480ce330 2023-10-11T09:48:56.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.072+0000 7ff4467fc700 1 -- 172.21.15.78:0/1470219114 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff440012430 con 0x7ff4480ce330 2023-10-11T09:48:56.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.076+0000 7ff44d6c6700 1 -- 172.21.15.78:0/1470219114 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4480ce330 msgr2=0x7ff44806be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:56.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.076+0000 7ff44d6c6700 1 --2- 172.21.15.78:0/1470219114 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4480ce330 0x7ff44806be60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff440009d90 tx=0x7ff44000b330 comp rx=0 tx=0).stop 2023-10-11T09:48:56.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.076+0000 7ff44d6c6700 1 -- 172.21.15.78:0/1470219114 shutdown_connections 2023-10-11T09:48:56.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.076+0000 7ff44d6c6700 1 --2- 172.21.15.78:0/1470219114 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4480ce330 0x7ff44806be60 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:56.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.076+0000 7ff44d6c6700 1 --2- 172.21.15.78:0/1470219114 >> v2:172.21.15.133:3300/0 conn(0x7ff448071040 0x7ff4480cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:56.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.076+0000 7ff44d6c6700 1 -- 172.21.15.78:0/1470219114 >> 172.21.15.78:0/1470219114 conn(0x7ff44806aca0 msgr2=0x7ff4480b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:56.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.076+0000 7ff44d6c6700 1 -- 172.21.15.78:0/1470219114 shutdown_connections 2023-10-11T09:48:56.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.076+0000 7ff44d6c6700 1 -- 172.21.15.78:0/1470219114 wait complete. 2023-10-11T09:48:56.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.080+0000 7ff44d6c6700 1 Processor -- start 2023-10-11T09:48:56.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.080+0000 7ff44d6c6700 1 -- start start 2023-10-11T09:48:56.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.080+0000 7ff44d6c6700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff448071040 0x7ff44814c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:56.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.080+0000 7ff44d6c6700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4480ce330 0x7ff44815bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:56.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.080+0000 7ff44d6c6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff448160d40 0x7ff44815c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:56.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.080+0000 7ff44d6c6700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44815dba0 con 0x7ff448160d40 2023-10-11T09:48:56.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.080+0000 7ff44d6c6700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44814c5d0 con 0x7ff4480ce330 2023-10-11T09:48:56.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.080+0000 7ff44d6c6700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff44815cb00 con 0x7ff448071040 2023-10-11T09:48:56.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.080+0000 7ff4477fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff448160d40 0x7ff44815c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:56.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.080+0000 7ff4477fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff448160d40 0x7ff44815c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37648/0 (socket says 172.21.15.78:37648) 2023-10-11T09:48:56.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.080+0000 7ff4477fe700 1 -- 172.21.15.78:0/3908539743 learned_addr learned my addr 172.21.15.78:0/3908539743 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:56.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.080+0000 7ff446ffd700 1 --2- 172.21.15.78:0/3908539743 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff448071040 0x7ff44814c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:56.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.080+0000 7ff43e7fc700 1 --2- 172.21.15.78:0/3908539743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4480ce330 0x7ff44815bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:56.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.084+0000 7ff4477fe700 1 -- 172.21.15.78:0/3908539743 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff448071040 msgr2=0x7ff44814c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:56.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.084+0000 7ff4477fe700 1 --2- 172.21.15.78:0/3908539743 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff448071040 0x7ff44814c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:56.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.084+0000 7ff4477fe700 1 -- 172.21.15.78:0/3908539743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4480ce330 msgr2=0x7ff44815bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:56.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.084+0000 7ff4477fe700 1 --2- 172.21.15.78:0/3908539743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4480ce330 0x7ff44815bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:56.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.084+0000 7ff4477fe700 1 -- 172.21.15.78:0/3908539743 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff440009a70 con 0x7ff448160d40 2023-10-11T09:48:56.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.084+0000 7ff4477fe700 1 --2- 172.21.15.78:0/3908539743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff448160d40 0x7ff44815c4f0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7ff440014040 tx=0x7ff440004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:56.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.084+0000 7ff43ffff700 1 -- 172.21.15.78:0/3908539743 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff440012430 con 0x7ff448160d40 2023-10-11T09:48:56.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.084+0000 7ff43ffff700 1 -- 172.21.15.78:0/3908539743 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff440020070 con 0x7ff448160d40 2023-10-11T09:48:56.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.084+0000 7ff44d6c6700 1 -- 172.21.15.78:0/3908539743 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff44815cde0 con 0x7ff448160d40 2023-10-11T09:48:56.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.084+0000 7ff43ffff700 1 -- 172.21.15.78:0/3908539743 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4400098f0 con 0x7ff448160d40 2023-10-11T09:48:56.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.084+0000 7ff44d6c6700 1 -- 172.21.15.78:0/3908539743 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff44815d3a0 con 0x7ff448160d40 2023-10-11T09:48:56.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.084+0000 7ff44d6c6700 1 -- 172.21.15.78:0/3908539743 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff448072250 con 0x7ff448160d40 2023-10-11T09:48:56.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.088+0000 7ff43ffff700 1 -- 172.21.15.78:0/3908539743 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7ff440004040 con 0x7ff448160d40 2023-10-11T09:48:56.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.088+0000 7ff43ffff700 1 --2- 172.21.15.78:0/3908539743 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff42c065ed0 0x7ff42c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:56.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.088+0000 7ff43ffff700 1 -- 172.21.15.78:0/3908539743 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6724+0+0 (secure 0 0 0) 0x7ff440022070 con 0x7ff448160d40 2023-10-11T09:48:56.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.088+0000 7ff446ffd700 1 --2- 172.21.15.78:0/3908539743 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff42c065ed0 0x7ff42c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:56.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.088+0000 7ff446ffd700 1 --2- 172.21.15.78:0/3908539743 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff42c065ed0 0x7ff42c068380 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff438005930 tx=0x7ff438009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:56.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.096+0000 7ff43ffff700 1 -- 172.21.15.78:0/3908539743 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7ff440052a50 con 0x7ff448160d40 2023-10-11T09:48:56.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.304+0000 7ff44d6c6700 1 -- 172.21.15.78:0/3908539743 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff44804e530 con 0x7ff448160d40 2023-10-11T09:48:56.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.308+0000 7ff43ffff700 1 -- 172.21.15.78:0/3908539743 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v71) v1 ==== 53+0+91 (secure 0 0 0) 0x7ff44002c070 con 0x7ff448160d40 2023-10-11T09:48:56.312 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_extra ] 2023-10-11T09:48:56.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.308+0000 7ff44d6c6700 1 -- 172.21.15.78:0/3908539743 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff42c065ed0 msgr2=0x7ff42c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:56.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.308+0000 7ff44d6c6700 1 --2- 172.21.15.78:0/3908539743 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff42c065ed0 0x7ff42c068380 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff438005930 tx=0x7ff438009040 comp rx=0 tx=0).stop 2023-10-11T09:48:56.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.308+0000 7ff44d6c6700 1 -- 172.21.15.78:0/3908539743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff448160d40 msgr2=0x7ff44815c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:56.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.308+0000 7ff44d6c6700 1 --2- 172.21.15.78:0/3908539743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff448160d40 0x7ff44815c4f0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7ff440014040 tx=0x7ff440004af0 comp rx=0 tx=0).stop 2023-10-11T09:48:56.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.308+0000 7ff44d6c6700 1 -- 172.21.15.78:0/3908539743 shutdown_connections 2023-10-11T09:48:56.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.308+0000 7ff44d6c6700 1 --2- 172.21.15.78:0/3908539743 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff42c065ed0 0x7ff42c068380 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:56.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.308+0000 7ff44d6c6700 1 --2- 172.21.15.78:0/3908539743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff448160d40 0x7ff44815c4f0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:56.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.308+0000 7ff44d6c6700 1 --2- 172.21.15.78:0/3908539743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4480ce330 0x7ff44815bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:56.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.308+0000 7ff44d6c6700 1 --2- 172.21.15.78:0/3908539743 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff448071040 0x7ff44814c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:56.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.308+0000 7ff44d6c6700 1 -- 172.21.15.78:0/3908539743 >> 172.21.15.78:0/3908539743 conn(0x7ff44806aca0 msgr2=0x7ff4480ca5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:56.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.308+0000 7ff44d6c6700 1 -- 172.21.15.78:0/3908539743 shutdown_connections 2023-10-11T09:48:56.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:56.308+0000 7ff44d6c6700 1 -- 172.21.15.78:0/3908539743 wait complete. 2023-10-11T09:48:56.335 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-11T09:48:56.335 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-11T09:48:56.336 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi078.front.sepia.ceph.com 2023-10-11T09:48:56.336 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-11T09:48:56.336 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-11T09:48:56.337 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-11T09:48:56.337 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-11T09:48:56.337 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-11T09:48:56.337 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-11T09:48:56.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:48:56.338 DEBUG:teuthology.orchestra.run.smithi078:> ip addr 2023-10-11T09:48:56.354 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-11T09:48:56.354 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-11T09:48:56.354 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2023-10-11T09:48:56.355 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:48:56.355 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2023-10-11T09:48:56.355 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:48:56.355 INFO:teuthology.orchestra.run.smithi078.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:48:56.355 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:22 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:48:56.356 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f0 2023-10-11T09:48:56.356 INFO:teuthology.orchestra.run.smithi078.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:48:56.356 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:23 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:48:56.356 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f1 2023-10-11T09:48:56.356 INFO:teuthology.orchestra.run.smithi078.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:48:56.357 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dc brd ff:ff:ff:ff:ff:ff 2023-10-11T09:48:56.357 INFO:teuthology.orchestra.run.smithi078.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-11T09:48:56.357 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dd brd ff:ff:ff:ff:ff:ff 2023-10-11T09:48:56.357 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.21.15.78/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-11T09:48:56.357 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft 41928sec preferred_lft 41928sec 2023-10-11T09:48:56.357 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::ec4:7aff:fe8f:d4dd/64 scope link 2023-10-11T09:48:56.358 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:48:56.358 INFO:teuthology.orchestra.run.smithi078.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-11T09:48:56.358 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 02:42:b6:e4:f8:0d brd ff:ff:ff:ff:ff:ff 2023-10-11T09:48:56.358 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-11T09:48:56.359 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:48:56.359 INFO:teuthology.orchestra.run.smithi078.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-11T09:48:56.359 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 7a:89:8a:a9:e1:23 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:48:56.359 INFO:teuthology.orchestra.run.smithi078.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-11T09:48:56.359 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:48:56.360 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::8ca:ebff:fea9:13f1/64 scope link 2023-10-11T09:48:56.360 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:48:56.360 INFO:teuthology.orchestra.run.smithi078.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-11T09:48:56.360 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 7a:89:8a:a9:e1:23 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-11T09:48:56.360 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::7889:8aff:fea9:e123/64 scope link 2023-10-11T09:48:56.360 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:48:56.361 INFO:teuthology.orchestra.run.smithi078.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-11T09:48:56.361 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether ca:ac:e8:27:51:0d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-11T09:48:56.361 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::c8ac:e8ff:fe27:510d/64 scope link 2023-10-11T09:48:56.361 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:48:56.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:48:56.363 DEBUG:teuthology.orchestra.run.smithi078:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-11T09:48:56.441 INFO:teuthology.orchestra.run.smithi078.stdout:1 2023-10-11T09:48:56.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:48:56.444 DEBUG:teuthology.orchestra.run.smithi078:> route 2023-10-11T09:48:56.507 INFO:teuthology.orchestra.run.smithi078.stdout:Kernel IP routing table 2023-10-11T09:48:56.507 INFO:teuthology.orchestra.run.smithi078.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-11T09:48:56.508 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-11T09:48:56.508 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-11T09:48:56.508 INFO:teuthology.orchestra.run.smithi078.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-11T09:48:56.508 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-10-11T09:48:56.508 INFO:teuthology.orchestra.run.smithi078.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-11T09:48:56.509 INFO:teuthology.orchestra.run.smithi078.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-11T09:48:56.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:48:56.509 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec bash -c ' 2023-10-11T09:48:56.510 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2023-10-11T09:48:56.510 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-11T09:48:56.511 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-11T09:48:56.511 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-11T09:48:56.511 DEBUG:teuthology.orchestra.run.smithi078:> ') 2023-10-11T09:48:56.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:48:56.644 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2023-10-11T09:48:56.660 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-11T09:48:56.661 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-11T09:48:56.662 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-11T09:48:56.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:48:56.663 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set brx.0 up 2023-10-11T09:48:56.738 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-11T09:48:56.738 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T09:48:56.739 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-11T09:48:56.798 INFO:teuthology.orchestra.run.smithi078.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-11T09:48:56.798 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T09:48:56.799 DEBUG:teuthology.orchestra.run.smithi078:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-11T09:48:56.856 DEBUG:teuthology.orchestra.run.smithi078:> sudo modprobe fuse 2023-10-11T09:48:56.941 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T09:48:56.942 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T09:48:57.020 INFO:teuthology.orchestra.run.smithi078.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T09:48:57.023 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T09:48:57.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:48:57.025 DEBUG:teuthology.orchestra.run.smithi078:> ls /sys/fs/fuse/connections 2023-10-11T09:48:57.084 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-11T09:48:57.084 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-11T09:48:57.134 DEBUG:teuthology.orchestra.run.smithi078:> sudo modprobe fuse 2023-10-11T09:48:57.170 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T09:48:57.171 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T09:48:57.246 INFO:teuthology.orchestra.run.smithi078.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T09:48:57.249 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T09:48:57.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:48:57.250 DEBUG:teuthology.orchestra.run.smithi078:> ls /sys/fs/fuse/connections 2023-10-11T09:48:57.271 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:2023-10-11T09:48:57.264+0000 7f038cffb640 -1 init, newargv = 0x55666dfd27b0 newargc=15 2023-10-11T09:48:57.272 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:ceph-fuse[54576]: starting ceph client 2023-10-11T09:48:57.284 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:ceph-fuse[54576]: starting fuse 2023-10-11T09:48:57.287 INFO:teuthology.orchestra.run.smithi078.stdout:49 2023-10-11T09:48:57.288 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2023-10-11T09:48:57.288 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:48:57.288 DEBUG:teuthology.orchestra.run.smithi078:> import glob 2023-10-11T09:48:57.289 DEBUG:teuthology.orchestra.run.smithi078:> import re 2023-10-11T09:48:57.289 DEBUG:teuthology.orchestra.run.smithi078:> import os 2023-10-11T09:48:57.290 DEBUG:teuthology.orchestra.run.smithi078:> import subprocess 2023-10-11T09:48:57.290 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:48:57.290 DEBUG:teuthology.orchestra.run.smithi078:> def _find_admin_socket(client_name): 2023-10-11T09:48:57.291 DEBUG:teuthology.orchestra.run.smithi078:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-11T09:48:57.291 DEBUG:teuthology.orchestra.run.smithi078:> files = glob.glob(asok_path) 2023-10-11T09:48:57.291 DEBUG:teuthology.orchestra.run.smithi078:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-11T09:48:57.292 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:48:57.292 DEBUG:teuthology.orchestra.run.smithi078:> # Given a non-glob path, it better be there 2023-10-11T09:48:57.292 DEBUG:teuthology.orchestra.run.smithi078:> if "*" not in asok_path: 2023-10-11T09:48:57.293 DEBUG:teuthology.orchestra.run.smithi078:> assert(len(files) == 1) 2023-10-11T09:48:57.293 DEBUG:teuthology.orchestra.run.smithi078:> return files[0] 2023-10-11T09:48:57.293 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:48:57.294 DEBUG:teuthology.orchestra.run.smithi078:> for f in files: 2023-10-11T09:48:57.294 DEBUG:teuthology.orchestra.run.smithi078:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-11T09:48:57.294 DEBUG:teuthology.orchestra.run.smithi078:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-11T09:48:57.295 DEBUG:teuthology.orchestra.run.smithi078:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-11T09:48:57.295 DEBUG:teuthology.orchestra.run.smithi078:> contents = proc_f.read() 2023-10-11T09:48:57.295 DEBUG:teuthology.orchestra.run.smithi078:> if mountpoint in contents: 2023-10-11T09:48:57.296 DEBUG:teuthology.orchestra.run.smithi078:> return f 2023-10-11T09:48:57.296 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-11T09:48:57.296 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:48:57.297 DEBUG:teuthology.orchestra.run.smithi078:> print(_find_admin_socket("client.0")) 2023-10-11T09:48:57.297 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:48:57.446 INFO:teuthology.orchestra.run.smithi078.stdout:/var/run/ceph/ceph-client.0.54576.asok 2023-10-11T09:48:57.631 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.54576.asok 2023-10-11T09:48:57.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:48:57.632 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.54576.asok status 2023-10-11T09:48:57.824 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:48:57.825 INFO:teuthology.orchestra.run.smithi078.stdout: "metadata": { 2023-10-11T09:48:57.825 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_sha1": "9a774796f0ecdd11a4b771de396e627aa1a187af", 2023-10-11T09:48:57.825 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_version": "ceph version 18.2.0-662-g9a774796 (9a774796f0ecdd11a4b771de396e627aa1a187af) reef (stable)", 2023-10-11T09:48:57.825 INFO:teuthology.orchestra.run.smithi078.stdout: "entity_id": "0", 2023-10-11T09:48:57.826 INFO:teuthology.orchestra.run.smithi078.stdout: "hostname": "smithi078", 2023-10-11T09:48:57.826 INFO:teuthology.orchestra.run.smithi078.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-11T09:48:57.826 INFO:teuthology.orchestra.run.smithi078.stdout: "pid": "54576", 2023-10-11T09:48:57.826 INFO:teuthology.orchestra.run.smithi078.stdout: "root": "/" 2023-10-11T09:48:57.826 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:48:57.827 INFO:teuthology.orchestra.run.smithi078.stdout: "dentry_count": 0, 2023-10-11T09:48:57.827 INFO:teuthology.orchestra.run.smithi078.stdout: "dentry_pinned_count": 0, 2023-10-11T09:48:57.827 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 7776, 2023-10-11T09:48:57.827 INFO:teuthology.orchestra.run.smithi078.stdout: "inst": { 2023-10-11T09:48:57.827 INFO:teuthology.orchestra.run.smithi078.stdout: "name": { 2023-10-11T09:48:57.828 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "client", 2023-10-11T09:48:57.828 INFO:teuthology.orchestra.run.smithi078.stdout: "num": 7776 2023-10-11T09:48:57.828 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:48:57.828 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": { 2023-10-11T09:48:57.828 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2023-10-11T09:48:57.829 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "192.168.0.1:0", 2023-10-11T09:48:57.829 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 527063791 2023-10-11T09:48:57.829 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:48:57.829 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:48:57.829 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": { 2023-10-11T09:48:57.829 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2023-10-11T09:48:57.830 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "192.168.0.1:0", 2023-10-11T09:48:57.830 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 527063791 2023-10-11T09:48:57.830 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:48:57.830 INFO:teuthology.orchestra.run.smithi078.stdout: "inst_str": "client.7776 192.168.0.1:0/527063791", 2023-10-11T09:48:57.830 INFO:teuthology.orchestra.run.smithi078.stdout: "addr_str": "192.168.0.1:0/527063791", 2023-10-11T09:48:57.831 INFO:teuthology.orchestra.run.smithi078.stdout: "inode_count": 1, 2023-10-11T09:48:57.831 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_epoch": 71, 2023-10-11T09:48:57.831 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_epoch": 92, 2023-10-11T09:48:57.831 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_epoch_barrier": 0, 2023-10-11T09:48:57.831 INFO:teuthology.orchestra.run.smithi078.stdout: "blocklisted": false, 2023-10-11T09:48:57.832 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs" 2023-10-11T09:48:57.832 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:48:57.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:48:57.839 DEBUG:teuthology.orchestra.run.smithi078:> stat --file-system '--printf=%T 2023-10-11T09:48:57.839 DEBUG:teuthology.orchestra.run.smithi078:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-11T09:48:57.860 INFO:teuthology.orchestra.run.smithi078.stdout:fuseblk 2023-10-11T09:48:57.860 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-11T09:48:57.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:48:57.861 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-11T09:48:57.939 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub start / repair,recursive 2023-10-11T09:48:58.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.160+0000 7fa8765a8700 1 Processor -- start 2023-10-11T09:48:58.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.160+0000 7fa8765a8700 1 -- start start 2023-10-11T09:48:58.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.160+0000 7fa8765a8700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fa87015a7b0 0x7fa8700cb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.160+0000 7fa8765a8700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa8700b1a10 0x7fa8700b1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.160+0000 7fa8765a8700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8700cbba0 con 0x7fa87006c840 2023-10-11T09:48:58.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.160+0000 7fa8765a8700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8700cb9b0 con 0x7fa87015a7b0 2023-10-11T09:48:58.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.160+0000 7fa8765a8700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8700cbe80 con 0x7fa8700b1a10 2023-10-11T09:48:58.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.164+0000 7fa8755a6700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fa87015a7b0 0x7fa8700cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.164+0000 7fa874da5700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa87006c840 0x7fa870071810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:36046/0 (socket says 172.21.15.78:36046) 2023-10-11T09:48:58.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.164+0000 7fa874da5700 1 -- 172.21.15.78:0/1696500478 learned_addr learned my addr 172.21.15.78:0/1696500478 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:58.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.164+0000 7fa875da7700 1 --2- 172.21.15.78:0/1696500478 >> v2:172.21.15.133:3300/0 conn(0x7fa8700b1a10 0x7fa8700b1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.164+0000 7fa867fff700 1 -- 172.21.15.78:0/1696500478 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3467858730 0 0) 0x7fa8700cbba0 con 0x7fa87006c840 2023-10-11T09:48:58.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.164+0000 7fa867fff700 1 -- 172.21.15.78:0/1696500478 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa85c0035d0 con 0x7fa87006c840 2023-10-11T09:48:58.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.164+0000 7fa8755a6700 1 -- 172.21.15.78:0/1696500478 >> v2:172.21.15.133:3300/0 conn(0x7fa8700b1a10 msgr2=0x7fa8700b1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:58.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.164+0000 7fa8755a6700 1 --2- 172.21.15.78:0/1696500478 >> v2:172.21.15.133:3300/0 conn(0x7fa8700b1a10 0x7fa8700b1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.164+0000 7fa8755a6700 1 -- 172.21.15.78:0/1696500478 >> v1:172.21.15.133:6789/0 conn(0x7fa87006c840 legacy=0x7fa870071810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:58.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.164+0000 7fa8755a6700 1 -- 172.21.15.78:0/1696500478 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa870071010 con 0x7fa87015a7b0 2023-10-11T09:48:58.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.164+0000 7fa8755a6700 1 --2- 172.21.15.78:0/1696500478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa87015a7b0 0x7fa8700cb470 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fa86c009850 tx=0x7fa86c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b641ae364b2f07b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:58.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.164+0000 7fa867fff700 1 -- 172.21.15.78:0/1696500478 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c011830 con 0x7fa87015a7b0 2023-10-11T09:48:58.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.164+0000 7fa867fff700 1 -- 172.21.15.78:0/1696500478 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86c011990 con 0x7fa87015a7b0 2023-10-11T09:48:58.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.164+0000 7fa867fff700 1 -- 172.21.15.78:0/1696500478 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c011d20 con 0x7fa87015a7b0 2023-10-11T09:48:58.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.164+0000 7fa8765a8700 1 -- 172.21.15.78:0/1696500478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa87015a7b0 msgr2=0x7fa8700cb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:58.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.164+0000 7fa8765a8700 1 --2- 172.21.15.78:0/1696500478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa87015a7b0 0x7fa8700cb470 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fa86c009850 tx=0x7fa86c009b60 comp rx=0 tx=0).stop 2023-10-11T09:48:58.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.164+0000 7fa8765a8700 1 -- 172.21.15.78:0/1696500478 shutdown_connections 2023-10-11T09:48:58.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.164+0000 7fa8765a8700 1 --2- 172.21.15.78:0/1696500478 >> v2:172.21.15.133:3300/0 conn(0x7fa8700b1a10 0x7fa8700b1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.164+0000 7fa8765a8700 1 --2- 172.21.15.78:0/1696500478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa87015a7b0 0x7fa8700cb470 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.164+0000 7fa8765a8700 1 -- 172.21.15.78:0/1696500478 >> 172.21.15.78:0/1696500478 conn(0x7fa87006acf0 msgr2=0x7fa8700b0710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:58.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.164+0000 7fa8765a8700 1 -- 172.21.15.78:0/1696500478 shutdown_connections 2023-10-11T09:48:58.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.168+0000 7fa8765a8700 1 -- 172.21.15.78:0/1696500478 wait complete. 2023-10-11T09:48:58.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.168+0000 7fa8765a8700 1 Processor -- start 2023-10-11T09:48:58.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.168+0000 7fa8765a8700 1 -- start start 2023-10-11T09:48:58.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.172+0000 7fa8765a8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa87006c840 0x7fa870155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.172+0000 7fa8765a8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8700b1a10 0x7fa8700c21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.172+0000 7fa8765a8700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa87015a7b0 0x7fa8700c2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.172+0000 7fa8765a8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8700c44e0 con 0x7fa8700b1a10 2023-10-11T09:48:58.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.172+0000 7fa8765a8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8700c3be0 con 0x7fa87006c840 2023-10-11T09:48:58.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.172+0000 7fa8765a8700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8700c8280 con 0x7fa87015a7b0 2023-10-11T09:48:58.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.172+0000 7fa874da5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8700b1a10 0x7fa8700c21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.172+0000 7fa874da5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8700b1a10 0x7fa8700c21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37676/0 (socket says 172.21.15.78:37676) 2023-10-11T09:48:58.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.172+0000 7fa874da5700 1 -- 172.21.15.78:0/3709057860 learned_addr learned my addr 172.21.15.78:0/3709057860 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:58.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.172+0000 7fa8755a6700 1 --2- 172.21.15.78:0/3709057860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa87006c840 0x7fa870155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.172+0000 7fa875da7700 1 --2- 172.21.15.78:0/3709057860 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa87015a7b0 0x7fa8700c2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.172+0000 7fa874da5700 1 -- 172.21.15.78:0/3709057860 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa87015a7b0 msgr2=0x7fa8700c2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:58.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.172+0000 7fa874da5700 1 --2- 172.21.15.78:0/3709057860 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa87015a7b0 0x7fa8700c2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.172+0000 7fa874da5700 1 -- 172.21.15.78:0/3709057860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa87006c840 msgr2=0x7fa870155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:58.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.172+0000 7fa874da5700 1 --2- 172.21.15.78:0/3709057860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa87006c840 0x7fa870155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.172+0000 7fa874da5700 1 -- 172.21.15.78:0/3709057860 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86c0094b0 con 0x7fa8700b1a10 2023-10-11T09:48:58.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.172+0000 7fa874da5700 1 --2- 172.21.15.78:0/3709057860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8700b1a10 0x7fa8700c21f0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fa86000c700 tx=0x7fa86000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:58.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.172+0000 7fa865ffb700 1 -- 172.21.15.78:0/3709057860 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa860005d90 con 0x7fa8700b1a10 2023-10-11T09:48:58.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.172+0000 7fa865ffb700 1 -- 172.21.15.78:0/3709057860 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8600070d0 con 0x7fa8700b1a10 2023-10-11T09:48:58.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.172+0000 7fa8765a8700 1 -- 172.21.15.78:0/3709057860 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8700c3e20 con 0x7fa8700b1a10 2023-10-11T09:48:58.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.172+0000 7fa865ffb700 1 -- 172.21.15.78:0/3709057860 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa860007480 con 0x7fa8700b1a10 2023-10-11T09:48:58.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.172+0000 7fa8765a8700 1 -- 172.21.15.78:0/3709057860 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8700c4900 con 0x7fa8700b1a10 2023-10-11T09:48:58.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.176+0000 7fa865ffb700 1 -- 172.21.15.78:0/3709057860 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fa8600075e0 con 0x7fa8700b1a10 2023-10-11T09:48:58.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.180+0000 7fa865ffb700 1 --2- 172.21.15.78:0/3709057860 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa854065ed0 0x7fa854068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.180+0000 7fa865ffb700 1 -- 172.21.15.78:0/3709057860 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6724+0+0 (secure 0 0 0) 0x7fa860082f00 con 0x7fa8700b1a10 2023-10-11T09:48:58.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.180+0000 7fa8755a6700 1 --2- 172.21.15.78:0/3709057860 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa854065ed0 0x7fa854068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.180+0000 7fa87aa92740 1 Processor -- start 2023-10-11T09:48:58.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.180+0000 7fa87aa92740 1 -- start start 2023-10-11T09:48:58.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.180+0000 7fa8755a6700 1 --2- 172.21.15.78:0/3709057860 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa854065ed0 0x7fa854068380 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa86c000c00 tx=0x7fa86c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:58.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.180+0000 7fa87aa92740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2666d80 0x291bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.180+0000 7fa87aa92740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x26669b0 0x291c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.180+0000 7fa87aa92740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26b1a30 0x291ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.180+0000 7fa87aa92740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x27f92f0 con 0x26b1a30 2023-10-11T09:48:58.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.180+0000 7fa87aa92740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x2846610 con 0x26669b0 2023-10-11T09:48:58.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.180+0000 7fa87aa92740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x2843da0 con 0x2666d80 2023-10-11T09:48:58.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.180+0000 7fa875da7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26b1a30 0x291ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.180+0000 7fa875da7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26b1a30 0x291ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37692/0 (socket says 172.21.15.78:37692) 2023-10-11T09:48:58.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.180+0000 7fa875da7700 1 -- 172.21.15.78:0/1513053364 learned_addr learned my addr 172.21.15.78:0/1513053364 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:58.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.180+0000 7fa8755a6700 1 --2- 172.21.15.78:0/1513053364 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2666d80 0x291bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.180+0000 7fa874da5700 1 --2- 172.21.15.78:0/1513053364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x26669b0 0x291c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.184+0000 7fa875da7700 1 -- 172.21.15.78:0/1513053364 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2666d80 msgr2=0x291bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:58.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.184+0000 7fa875da7700 1 --2- 172.21.15.78:0/1513053364 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2666d80 0x291bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.184+0000 7fa875da7700 1 -- 172.21.15.78:0/1513053364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x26669b0 msgr2=0x291c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:58.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.184+0000 7fa875da7700 1 --2- 172.21.15.78:0/1513053364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x26669b0 0x291c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.184+0000 7fa875da7700 1 -- 172.21.15.78:0/1513053364 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2866b50 con 0x26b1a30 2023-10-11T09:48:58.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.184+0000 7fa875da7700 1 --2- 172.21.15.78:0/1513053364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26b1a30 0x291ca30 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fa868011500 tx=0x7fa868009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8e5279335ad54d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:58.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.184+0000 7fa85affd700 1 -- 172.21.15.78:0/1513053364 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86800e040 con 0x26b1a30 2023-10-11T09:48:58.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.184+0000 7fa85affd700 1 -- 172.21.15.78:0/1513053364 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86800f040 con 0x26b1a30 2023-10-11T09:48:58.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.184+0000 7fa85affd700 1 -- 172.21.15.78:0/1513053364 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa868015a40 con 0x26b1a30 2023-10-11T09:48:58.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.184+0000 7fa87aa92740 1 -- 172.21.15.78:0/1513053364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26b1a30 msgr2=0x291ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:58.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.184+0000 7fa87aa92740 1 --2- 172.21.15.78:0/1513053364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26b1a30 0x291ca30 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fa868011500 tx=0x7fa868009750 comp rx=0 tx=0).stop 2023-10-11T09:48:58.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.184+0000 7fa87aa92740 1 -- 172.21.15.78:0/1513053364 shutdown_connections 2023-10-11T09:48:58.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.184+0000 7fa87aa92740 1 --2- 172.21.15.78:0/1513053364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26b1a30 0x291ca30 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.184+0000 7fa87aa92740 1 --2- 172.21.15.78:0/1513053364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x26669b0 0x291c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.184+0000 7fa87aa92740 1 --2- 172.21.15.78:0/1513053364 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2666d80 0x291bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.184+0000 7fa87aa92740 1 -- 172.21.15.78:0/1513053364 >> 172.21.15.78:0/1513053364 conn(0x2667150 msgr2=0x28d6860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:58.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.184+0000 7fa87aa92740 1 -- 172.21.15.78:0/1513053364 shutdown_connections 2023-10-11T09:48:58.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.184+0000 7fa87aa92740 1 -- 172.21.15.78:0/1513053364 wait complete. 2023-10-11T09:48:58.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.184+0000 7fa87aa92740 1 -- start start 2023-10-11T09:48:58.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.184+0000 7fa87aa92740 1 Processor -- start 2023-10-11T09:48:58.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.188+0000 7fa87aa92740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x26669b0 0x2928f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.188+0000 7fa87aa92740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2666d80 0x29294c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.188+0000 7fa87aa92740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x26b1a30 0x28fe1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.188+0000 7fa87aa92740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x283f690 con 0x2666d80 2023-10-11T09:48:58.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.188+0000 7fa87aa92740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x2802c50 con 0x26669b0 2023-10-11T09:48:58.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.188+0000 7fa87aa92740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x28028f0 con 0x26b1a30 2023-10-11T09:48:58.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.188+0000 7fa87aa92740 10 client.0 _ll_register_callbacks cb 0x2892a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:48:58.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.188+0000 7fa874da5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2666d80 0x29294c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.188+0000 7fa874da5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2666d80 0x29294c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37708/0 (socket says 172.21.15.78:37708) 2023-10-11T09:48:58.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.188+0000 7fa874da5700 1 -- 172.21.15.78:0/2870504722 learned_addr learned my addr 172.21.15.78:0/2870504722 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:58.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.188+0000 7fa8755a6700 1 --2- 172.21.15.78:0/2870504722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x26669b0 0x2928f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.188+0000 7fa875da7700 1 --2- 172.21.15.78:0/2870504722 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x26b1a30 0x28fe1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.188+0000 7fa874da5700 1 -- 172.21.15.78:0/2870504722 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x26b1a30 msgr2=0x28fe1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:58.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.188+0000 7fa874da5700 1 --2- 172.21.15.78:0/2870504722 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x26b1a30 0x28fe1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.188+0000 7fa874da5700 1 -- 172.21.15.78:0/2870504722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x26669b0 msgr2=0x2928f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:58.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.188+0000 7fa874da5700 1 --2- 172.21.15.78:0/2870504722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x26669b0 0x2928f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.188+0000 7fa874da5700 1 -- 172.21.15.78:0/2870504722 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa868011230 con 0x2666d80 2023-10-11T09:48:58.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.188+0000 7fa875da7700 1 --2- 172.21.15.78:0/2870504722 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x26b1a30 0x28fe1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:48:58.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.188+0000 7fa874da5700 1 --2- 172.21.15.78:0/2870504722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2666d80 0x29294c0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fa860007fd0 tx=0x7fa8600544a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b855c8452e53bbe server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:58.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.188+0000 7fa84ffff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:48:58.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.188+0000 7fa84ffff700 1 -- 172.21.15.78:0/2870504722 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86005bcb0 con 0x2666d80 2023-10-11T09:48:58.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.188+0000 7fa87aa92740 1 -- 172.21.15.78:0/2870504722 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x2838710 con 0x2666d80 2023-10-11T09:48:58.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.188+0000 7fa84ffff700 1 -- 172.21.15.78:0/2870504722 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6724+0+0 (secure 0 0 0) 0x7fa860053f60 con 0x2666d80 2023-10-11T09:48:58.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.192+0000 7fa84ffff700 1 -- 172.21.15.78:0/2870504722 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa860057d90 con 0x2666d80 2023-10-11T09:48:58.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.192+0000 7fa87aa92740 10 client.7788 fetch_fsmap learned FSMap version 71 2023-10-11T09:48:58.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.192+0000 7fa87aa92740 1 -- 172.21.15.78:0/2870504722 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x26b2c80 con 0x2666d80 2023-10-11T09:48:58.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.192+0000 7fa84ffff700 1 -- 172.21.15.78:0/2870504722 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86006f020 con 0x2666d80 2023-10-11T09:48:58.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.192+0000 7fa84ffff700 1 -- 172.21.15.78:0/2870504722 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 71) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fa86004fd50 con 0x2666d80 2023-10-11T09:48:58.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.192+0000 7fa87aa92740 10 client.7788 fetch_fsmap finished waiting for FSMap version 71 2023-10-11T09:48:58.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.192+0000 7fa87aa92740 10 client.7788 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2023-10-11T09:48:58.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.192+0000 7fa87aa92740 20 client.7788 populate_metadata read hostname 'smithi078' 2023-10-11T09:48:58.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.192+0000 7fa87aa92740 1 --2- 172.21.15.78:0/2870504722 >> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] conn(0x291baa0 0x28d8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.192+0000 7fa87aa92740 4 client.7788 mds_command: new command op to 7725 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-11T09:48:58.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.192+0000 7fa87aa92740 1 -- 172.21.15.78:0/2870504722 --> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x2838430 con 0x291baa0 2023-10-11T09:48:58.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.192+0000 7fa8755a6700 1 --2- 172.21.15.78:0/2870504722 >> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] conn(0x291baa0 0x28d8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.192+0000 7fa84ffff700 10 client.7788 ms_handle_connect on v2:172.21.15.133:6833/3922016223 2023-10-11T09:48:58.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.192+0000 7fa8755a6700 1 --2- 172.21.15.78:0/2870504722 >> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] conn(0x291baa0 0x28d8d50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de609551345752a7 server_cookie=86f291fc22d47ccf in_seq=0 out_seq=0 2023-10-11T09:48:58.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.196+0000 7fa84ffff700 1 -- 172.21.15.78:0/2870504722 <== mds.0 v2:172.21.15.133:6833/3922016223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x2838430 con 0x291baa0 2023-10-11T09:48:58.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.196+0000 7fa84ffff700 10 client.7788 handle_command_reply: tid=0 2023-10-11T09:48:58.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.196+0000 7fa87aa92740 1 client.7788 shutdown 2023-10-11T09:48:58.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.196+0000 7fa87aa92740 1 -- 172.21.15.78:0/2870504722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2666d80 msgr2=0x29294c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:58.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.196+0000 7fa87aa92740 1 --2- 172.21.15.78:0/2870504722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2666d80 0x29294c0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fa860007fd0 tx=0x7fa8600544a0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.196+0000 7fa87aa92740 1 -- 172.21.15.78:0/2870504722 shutdown_connections 2023-10-11T09:48:58.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.196+0000 7fa87aa92740 1 --2- 172.21.15.78:0/2870504722 >> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] conn(0x291baa0 0x28d8d50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.196+0000 7fa87aa92740 1 --2- 172.21.15.78:0/2870504722 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x26b1a30 0x28fe1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.196+0000 7fa87aa92740 1 --2- 172.21.15.78:0/2870504722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2666d80 0x29294c0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.196+0000 7fa87aa92740 1 --2- 172.21.15.78:0/2870504722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x26669b0 0x2928f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.196+0000 7fa84ffff700 0 client.7788 ms_handle_reset on v2:172.21.15.133:6833/3922016223 2023-10-11T09:48:58.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.196+0000 7fa87aa92740 1 -- 172.21.15.78:0/2870504722 >> 172.21.15.78:0/2870504722 conn(0x2667150 msgr2=0x28fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:58.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.196+0000 7fa87aa92740 1 -- 172.21.15.78:0/2870504722 shutdown_connections 2023-10-11T09:48:58.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.196+0000 7fa87aa92740 1 -- 172.21.15.78:0/2870504722 wait complete. 2023-10-11T09:48:58.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.196+0000 7fa87aa92740 20 client.7788 trim_cache size 0 max 16384 2023-10-11T09:48:58.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.212+0000 7fa87aa92740 1 Processor -- start 2023-10-11T09:48:58.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.216+0000 7fa87aa92740 1 -- start start 2023-10-11T09:48:58.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.216+0000 7fa87aa92740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x26669b0 0x2976ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.216+0000 7fa87aa92740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2666d80 0x2976ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.216+0000 7fa87aa92740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26b1a30 0x2919c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.216+0000 7fa87aa92740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x26ebdc0 con 0x26b1a30 2023-10-11T09:48:58.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.216+0000 7fa87aa92740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x280ee50 con 0x2666d80 2023-10-11T09:48:58.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.216+0000 7fa87aa92740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x283ff40 con 0x26669b0 2023-10-11T09:48:58.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.216+0000 7fa875da7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26b1a30 0x2919c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.216+0000 7fa875da7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26b1a30 0x2919c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37714/0 (socket says 172.21.15.78:37714) 2023-10-11T09:48:58.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.216+0000 7fa8755a6700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x26669b0 0x2976ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.216+0000 7fa875da7700 1 -- 172.21.15.78:0/1708414202 learned_addr learned my addr 172.21.15.78:0/1708414202 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:58.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.216+0000 7fa874da5700 1 --2- 172.21.15.78:0/1708414202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2666d80 0x2976ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.220+0000 7fa875da7700 1 -- 172.21.15.78:0/1708414202 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x26669b0 msgr2=0x2976ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:58.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.220+0000 7fa875da7700 1 --2- 172.21.15.78:0/1708414202 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x26669b0 0x2976ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.220+0000 7fa875da7700 1 -- 172.21.15.78:0/1708414202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2666d80 msgr2=0x2976ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:58.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.220+0000 7fa875da7700 1 --2- 172.21.15.78:0/1708414202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2666d80 0x2976ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.220+0000 7fa875da7700 1 -- 172.21.15.78:0/1708414202 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8700c2fd0 con 0x26b1a30 2023-10-11T09:48:58.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.220+0000 7fa8755a6700 1 --2- 172.21.15.78:0/1708414202 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x26669b0 0x2976ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:48:58.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.220+0000 7fa875da7700 1 --2- 172.21.15.78:0/1708414202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26b1a30 0x2919c10 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fa868019000 tx=0x7fa868016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c56a24ffe1a0cb57 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:58.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.220+0000 7fa85b7fe700 1 -- 172.21.15.78:0/1708414202 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86800e040 con 0x26b1a30 2023-10-11T09:48:58.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.220+0000 7fa85b7fe700 1 -- 172.21.15.78:0/1708414202 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa868003c40 con 0x26b1a30 2023-10-11T09:48:58.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.220+0000 7fa85b7fe700 1 -- 172.21.15.78:0/1708414202 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa868011e50 con 0x26b1a30 2023-10-11T09:48:58.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.220+0000 7fa87aa92740 1 -- 172.21.15.78:0/1708414202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26b1a30 msgr2=0x2919c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:58.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.220+0000 7fa87aa92740 1 --2- 172.21.15.78:0/1708414202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26b1a30 0x2919c10 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fa868019000 tx=0x7fa868016000 comp rx=0 tx=0).stop 2023-10-11T09:48:58.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.220+0000 7fa87aa92740 1 -- 172.21.15.78:0/1708414202 shutdown_connections 2023-10-11T09:48:58.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.220+0000 7fa87aa92740 1 --2- 172.21.15.78:0/1708414202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26b1a30 0x2919c10 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.220+0000 7fa87aa92740 1 --2- 172.21.15.78:0/1708414202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2666d80 0x2976ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.220+0000 7fa87aa92740 1 --2- 172.21.15.78:0/1708414202 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x26669b0 0x2976ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.220+0000 7fa87aa92740 1 -- 172.21.15.78:0/1708414202 >> 172.21.15.78:0/1708414202 conn(0x2667150 msgr2=0x2927820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:58.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.220+0000 7fa87aa92740 1 -- 172.21.15.78:0/1708414202 shutdown_connections 2023-10-11T09:48:58.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.220+0000 7fa87aa92740 1 -- 172.21.15.78:0/1708414202 wait complete. 2023-10-11T09:48:58.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.220+0000 7fa87aa92740 1 -- start start 2023-10-11T09:48:58.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.220+0000 7fa87aa92740 1 Processor -- start 2023-10-11T09:48:58.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.224+0000 7fa87aa92740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x26669b0 0x2973e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.224+0000 7fa87aa92740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2666d80 0x2929a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.224+0000 7fa87aa92740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26b1a30 0x2929f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.224+0000 7fa87aa92740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x28fe1b0 con 0x26b1a30 2023-10-11T09:48:58.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.224+0000 7fa87aa92740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x29280a0 con 0x2666d80 2023-10-11T09:48:58.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.224+0000 7fa87aa92740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x2924c70 con 0x26669b0 2023-10-11T09:48:58.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.224+0000 7fa87aa92740 10 client.0 _ll_register_callbacks cb 0x27f7d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:48:58.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.224+0000 7fa875da7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26b1a30 0x2929f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.224+0000 7fa875da7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26b1a30 0x2929f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37720/0 (socket says 172.21.15.78:37720) 2023-10-11T09:48:58.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.224+0000 7fa875da7700 1 -- 172.21.15.78:0/2075126486 learned_addr learned my addr 172.21.15.78:0/2075126486 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:58.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.224+0000 7fa8755a6700 1 --2- 172.21.15.78:0/2075126486 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x26669b0 0x2973e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.224+0000 7fa874da5700 1 --2- 172.21.15.78:0/2075126486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2666d80 0x2929a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.224+0000 7fa875da7700 1 -- 172.21.15.78:0/2075126486 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x26669b0 msgr2=0x2973e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:58.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.224+0000 7fa875da7700 1 --2- 172.21.15.78:0/2075126486 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x26669b0 0x2973e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.224+0000 7fa875da7700 1 -- 172.21.15.78:0/2075126486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2666d80 msgr2=0x2929a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:58.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.224+0000 7fa875da7700 1 --2- 172.21.15.78:0/2075126486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2666d80 0x2929a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.224+0000 7fa875da7700 1 -- 172.21.15.78:0/2075126486 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa870071010 con 0x26b1a30 2023-10-11T09:48:58.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.224+0000 7fa874da5700 1 --2- 172.21.15.78:0/2075126486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2666d80 0x2929a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:48:58.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.224+0000 7fa875da7700 1 --2- 172.21.15.78:0/2075126486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26b1a30 0x2929f40 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fa86800b010 tx=0x7fa868004b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a8f3242542fd54 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:58.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.224+0000 7fa84ffff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:48:58.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.228+0000 7fa84ffff700 1 -- 172.21.15.78:0/2075126486 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86800e030 con 0x26b1a30 2023-10-11T09:48:58.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.228+0000 7fa87aa92740 1 -- 172.21.15.78:0/2075126486 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa86800f040 con 0x26b1a30 2023-10-11T09:48:58.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.228+0000 7fa84ffff700 1 -- 172.21.15.78:0/2075126486 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6724+0+0 (secure 0 0 0) 0x7fa868024070 con 0x26b1a30 2023-10-11T09:48:58.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.228+0000 7fa84ffff700 1 -- 172.21.15.78:0/2075126486 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa868009d60 con 0x26b1a30 2023-10-11T09:48:58.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.228+0000 7fa87aa92740 10 client.7794 fetch_fsmap learned FSMap version 71 2023-10-11T09:48:58.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.228+0000 7fa87aa92740 1 -- 172.21.15.78:0/2075126486 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x2866b50 con 0x26b1a30 2023-10-11T09:48:58.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.228+0000 7fa84ffff700 1 -- 172.21.15.78:0/2075126486 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86800f040 con 0x26b1a30 2023-10-11T09:48:58.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.228+0000 7fa84ffff700 1 -- 172.21.15.78:0/2075126486 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 71) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fa86801b020 con 0x26b1a30 2023-10-11T09:48:58.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.228+0000 7fa87aa92740 10 client.7794 fetch_fsmap finished waiting for FSMap version 71 2023-10-11T09:48:58.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.228+0000 7fa87aa92740 10 client.7794 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2023-10-11T09:48:58.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.228+0000 7fa87aa92740 20 client.7794 populate_metadata read hostname 'smithi078' 2023-10-11T09:48:58.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.228+0000 7fa87aa92740 1 --2- 172.21.15.78:0/2075126486 >> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] conn(0x292a480 0x28fc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.228+0000 7fa87aa92740 4 client.7794 mds_command: new command op to 7725 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2023-10-11T09:48:58.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.228+0000 7fa87aa92740 1 -- 172.21.15.78:0/2075126486 --> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x26be780 con 0x292a480 2023-10-11T09:48:58.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.228+0000 7fa8755a6700 1 --2- 172.21.15.78:0/2075126486 >> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] conn(0x292a480 0x28fc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.228+0000 7fa84ffff700 10 client.7794 ms_handle_connect on v2:172.21.15.133:6833/3922016223 2023-10-11T09:48:58.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.228+0000 7fa8755a6700 1 --2- 172.21.15.78:0/2075126486 >> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] conn(0x292a480 0x28fc610 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b3b42393205f54d server_cookie=71517e17f516776b in_seq=0 out_seq=0 2023-10-11T09:48:58.243 INFO:tasks.ceph.mds.b.smithi133.stderr:2023-10-11T09:48:58.229+0000 7f575d918700 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2023-10-11T09:48:58.243 INFO:tasks.ceph.mds.b.smithi133.stderr:2023-10-11T09:48:58.233+0000 7f575d918700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /subdir/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x562310908580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(11)0x10000000000:[<0x1/subdir v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-10-11T09:48:58.243 INFO:tasks.ceph.mds.b.smithi133.stderr:2023-10-11T09:48:58.233+0000 7f575d918700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/sixmegs auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x56231098e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(13)0x10000000001:[<0x10000000000/sixmegs v8>,<0x1/subdir v10>]//[12]","memoryvalue":"(13)0x10000000001:[<0x10000000000/sixmegs v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:48:58.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.232+0000 7fa84ffff700 1 -- 172.21.15.78:0/2075126486 <== mds.0 v2:172.21.15.133:6833/3922016223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x26be780 con 0x292a480 2023-10-11T09:48:58.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.232+0000 7fa84ffff700 10 client.7794 handle_command_reply: tid=0 2023-10-11T09:48:58.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.232+0000 7fa87aa92740 1 client.7794 shutdown 2023-10-11T09:48:58.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.232+0000 7fa87aa92740 1 -- 172.21.15.78:0/2075126486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26b1a30 msgr2=0x2929f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:58.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.232+0000 7fa87aa92740 1 --2- 172.21.15.78:0/2075126486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26b1a30 0x2929f40 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fa86800b010 tx=0x7fa868004b60 comp rx=0 tx=0).stop 2023-10-11T09:48:58.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.232+0000 7fa87aa92740 1 -- 172.21.15.78:0/2075126486 shutdown_connections 2023-10-11T09:48:58.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.232+0000 7fa87aa92740 1 --2- 172.21.15.78:0/2075126486 >> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] conn(0x292a480 0x28fc610 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.232+0000 7fa87aa92740 1 --2- 172.21.15.78:0/2075126486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26b1a30 0x2929f40 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.232+0000 7fa87aa92740 1 --2- 172.21.15.78:0/2075126486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2666d80 0x2929a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.232+0000 7fa84ffff700 0 client.7794 ms_handle_reset on v2:172.21.15.133:6833/3922016223 2023-10-11T09:48:58.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.232+0000 7fa87aa92740 1 --2- 172.21.15.78:0/2075126486 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x26669b0 0x2973e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.232+0000 7fa87aa92740 1 -- 172.21.15.78:0/2075126486 >> 172.21.15.78:0/2075126486 conn(0x2667150 msgr2=0x2923ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:58.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.232+0000 7fa87aa92740 1 -- 172.21.15.78:0/2075126486 shutdown_connections 2023-10-11T09:48:58.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.232+0000 7fa87aa92740 1 -- 172.21.15.78:0/2075126486 wait complete. 2023-10-11T09:48:58.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.232+0000 7fa87aa92740 20 client.7794 trim_cache size 0 max 16384 2023-10-11T09:48:58.248 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:48:58.248 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2023-10-11T09:48:58.248 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "3625d844-7f25-43ba-8fe4-f63d2736b540", 2023-10-11T09:48:58.248 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2023-10-11T09:48:58.248 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:48:58.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.232+0000 7fa8765a8700 1 -- 172.21.15.78:0/3709057860 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa854065ed0 msgr2=0x7fa854068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:58.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.232+0000 7fa8765a8700 1 --2- 172.21.15.78:0/3709057860 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa854065ed0 0x7fa854068380 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa86c000c00 tx=0x7fa86c008030 comp rx=0 tx=0).stop 2023-10-11T09:48:58.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.232+0000 7fa8765a8700 1 -- 172.21.15.78:0/3709057860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8700b1a10 msgr2=0x7fa8700c21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:58.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.232+0000 7fa8765a8700 1 --2- 172.21.15.78:0/3709057860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8700b1a10 0x7fa8700c21f0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fa86000c700 tx=0x7fa86000ca10 comp rx=0 tx=0).stop 2023-10-11T09:48:58.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.232+0000 7fa8765a8700 1 -- 172.21.15.78:0/3709057860 shutdown_connections 2023-10-11T09:48:58.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.232+0000 7fa8765a8700 1 --2- 172.21.15.78:0/3709057860 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa854065ed0 0x7fa854068380 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.232+0000 7fa8765a8700 1 --2- 172.21.15.78:0/3709057860 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa87015a7b0 0x7fa8700c2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.232+0000 7fa8765a8700 1 --2- 172.21.15.78:0/3709057860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8700b1a10 0x7fa8700c21f0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.232+0000 7fa8765a8700 1 --2- 172.21.15.78:0/3709057860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa87006c840 0x7fa870155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.232+0000 7fa8765a8700 1 -- 172.21.15.78:0/3709057860 >> 172.21.15.78:0/3709057860 conn(0x7fa87006acf0 msgr2=0x7fa8700ca4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:58.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.232+0000 7fa8765a8700 1 -- 172.21.15.78:0/3709057860 shutdown_connections 2023-10-11T09:48:58.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.232+0000 7fa8765a8700 1 -- 172.21.15.78:0/3709057860 wait complete. 2023-10-11T09:48:58.252 INFO:tasks.ceph.mds.b.smithi133.stderr:2023-10-11T09:48:58.241+0000 7f575d918700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v5 ap=5 snaprealm=0x562310937200 DIRTYPARENT f(v1 1=0+1) n(v0 rc2023-10-11T09:48:57.930520+0000 1=0+1)/n() (idft sync r=1) (inest sync r=1) (ifile sync->mix r=1) (iversion lock) caps={7776=pLsXs/-@3} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x56231090a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(11)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=0+1)","ondisk_value.rstat":"n(v0 2=0+2)","memory_value.dirstat":"f(v1 1=0+1)","memory_value.rstat":"n(v0 rc2023-10-11T09:48:57.930520+0000 1=0+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-10-11T09:48:58.261 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2023-10-11T09:48:58.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.440+0000 7f1fe61b5700 1 Processor -- start 2023-10-11T09:48:58.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.440+0000 7f1fe61b5700 1 -- start start 2023-10-11T09:48:58.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.444+0000 7f1fe61b5700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1fe006d730 0x7f1fe0165240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.444+0000 7f1fe61b5700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fe00b2d30 con 0x7f1fe00b3a60 2023-10-11T09:48:58.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.444+0000 7f1fe61b5700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fe00cf360 con 0x7f1fe015a6b0 2023-10-11T09:48:58.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.444+0000 7f1fe61b5700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fe00b2f80 con 0x7f1fe006d730 2023-10-11T09:48:58.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.444+0000 7f1fe49b2700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f1fe00b3a60 0x7f1fe006d020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:45824/0 (socket says 172.21.15.78:45824) 2023-10-11T09:48:58.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.444+0000 7f1fe49b2700 1 -- 172.21.15.78:0/1692478251 learned_addr learned my addr 172.21.15.78:0/1692478251 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:48:58.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.444+0000 7f1fe59b4700 1 --2- 172.21.15.78:0/1692478251 >> v2:172.21.15.133:3300/0 conn(0x7f1fe006d730 0x7f1fe0165240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.444+0000 7f1fd7fff700 1 -- 172.21.15.78:0/1692478251 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1812309434 0 0) 0x7f1fe00b2d30 con 0x7f1fe00b3a60 2023-10-11T09:48:58.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.444+0000 7f1fd7fff700 1 -- 172.21.15.78:0/1692478251 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fcc0035d0 con 0x7f1fe00b3a60 2023-10-11T09:48:58.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.444+0000 7f1fe59b4700 1 -- 172.21.15.78:0/1692478251 >> v1:172.21.15.133:6789/0 conn(0x7f1fe015a6b0 legacy=0x7f1fe00ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:58.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.444+0000 7f1fe59b4700 1 -- 172.21.15.78:0/1692478251 >> v1:172.21.15.78:6789/0 conn(0x7f1fe00b3a60 legacy=0x7f1fe006d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:58.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.444+0000 7f1fe59b4700 1 -- 172.21.15.78:0/1692478251 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe00b3260 con 0x7f1fe006d730 2023-10-11T09:48:58.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.444+0000 7f1fe59b4700 1 --2- 172.21.15.78:0/1692478251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fe006d730 0x7f1fe0165240 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8009850 tx=0x7f1fd8009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=696b2d74513f33e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:58.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.444+0000 7f1fd7fff700 1 -- 172.21.15.78:0/1692478251 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8011830 con 0x7f1fe006d730 2023-10-11T09:48:58.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.448+0000 7f1fd7fff700 1 -- 172.21.15.78:0/1692478251 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd8011990 con 0x7f1fe006d730 2023-10-11T09:48:58.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.448+0000 7f1fd7fff700 1 -- 172.21.15.78:0/1692478251 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8011d40 con 0x7f1fe006d730 2023-10-11T09:48:58.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.448+0000 7f1fe61b5700 1 -- 172.21.15.78:0/1692478251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fe006d730 msgr2=0x7f1fe0165240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:58.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.448+0000 7f1fe61b5700 1 --2- 172.21.15.78:0/1692478251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fe006d730 0x7f1fe0165240 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8009850 tx=0x7f1fd8009b60 comp rx=0 tx=0).stop 2023-10-11T09:48:58.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.448+0000 7f1fe61b5700 1 -- 172.21.15.78:0/1692478251 shutdown_connections 2023-10-11T09:48:58.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.448+0000 7f1fe61b5700 1 --2- 172.21.15.78:0/1692478251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fe006d730 0x7f1fe0165240 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.448+0000 7f1fe61b5700 1 -- 172.21.15.78:0/1692478251 >> 172.21.15.78:0/1692478251 conn(0x7f1fe006a060 msgr2=0x7f1fe006a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:58.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.448+0000 7f1fe61b5700 1 -- 172.21.15.78:0/1692478251 shutdown_connections 2023-10-11T09:48:58.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.448+0000 7f1fe61b5700 1 -- 172.21.15.78:0/1692478251 wait complete. 2023-10-11T09:48:58.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.448+0000 7f1fe61b5700 1 Processor -- start 2023-10-11T09:48:58.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.448+0000 7f1fe61b5700 1 -- start start 2023-10-11T09:48:58.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.452+0000 7f1fe61b5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fe006d730 0x7f1fe014e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.452+0000 7f1fe61b5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fe00b3a60 0x7f1fe00ad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.452+0000 7f1fe61b5700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1fe015a6b0 0x7f1fe00ad700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.452+0000 7f1fe61b5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe00af710 con 0x7f1fe006d730 2023-10-11T09:48:58.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.452+0000 7f1fe61b5700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe00af9f0 con 0x7f1fe00b3a60 2023-10-11T09:48:58.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.452+0000 7f1fe61b5700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe00afcd0 con 0x7f1fe015a6b0 2023-10-11T09:48:58.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.452+0000 7f1fe51b3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fe006d730 0x7f1fe014e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.452+0000 7f1fe51b3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fe006d730 0x7f1fe014e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37722/0 (socket says 172.21.15.78:37722) 2023-10-11T09:48:58.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.452+0000 7f1fe51b3700 1 -- 172.21.15.78:0/1334251748 learned_addr learned my addr 172.21.15.78:0/1334251748 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:58.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.452+0000 7f1fe59b4700 1 --2- 172.21.15.78:0/1334251748 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1fe015a6b0 0x7f1fe00ad700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.452+0000 7f1fe49b2700 1 --2- 172.21.15.78:0/1334251748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fe00b3a60 0x7f1fe00ad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.452+0000 7f1fe51b3700 1 -- 172.21.15.78:0/1334251748 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1fe015a6b0 msgr2=0x7f1fe00ad700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:58.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.452+0000 7f1fe51b3700 1 --2- 172.21.15.78:0/1334251748 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1fe015a6b0 0x7f1fe00ad700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.452+0000 7f1fe51b3700 1 -- 172.21.15.78:0/1334251748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fe00b3a60 msgr2=0x7f1fe00ad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:58.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.452+0000 7f1fe51b3700 1 --2- 172.21.15.78:0/1334251748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fe00b3a60 0x7f1fe00ad1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.452+0000 7f1fe51b3700 1 -- 172.21.15.78:0/1334251748 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd80094b0 con 0x7f1fe006d730 2023-10-11T09:48:58.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.452+0000 7f1fe59b4700 1 --2- 172.21.15.78:0/1334251748 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1fe015a6b0 0x7f1fe00ad700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:48:58.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.452+0000 7f1fe51b3700 1 --2- 172.21.15.78:0/1334251748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fe006d730 0x7f1fe014e740 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc001ed0 tx=0x7f1fdc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:58.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.452+0000 7f1fd5ffb700 1 -- 172.21.15.78:0/1334251748 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc00ade0 con 0x7f1fe006d730 2023-10-11T09:48:58.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.452+0000 7f1fe61b5700 1 -- 172.21.15.78:0/1334251748 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe00affb0 con 0x7f1fe006d730 2023-10-11T09:48:58.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.452+0000 7f1fd5ffb700 1 -- 172.21.15.78:0/1334251748 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fdc0020d0 con 0x7f1fe006d730 2023-10-11T09:48:58.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.452+0000 7f1fd5ffb700 1 -- 172.21.15.78:0/1334251748 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc003c40 con 0x7f1fe006d730 2023-10-11T09:48:58.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.452+0000 7f1fe61b5700 1 -- 172.21.15.78:0/1334251748 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe014ec80 con 0x7f1fe006d730 2023-10-11T09:48:58.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.456+0000 7f1fd5ffb700 1 -- 172.21.15.78:0/1334251748 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f1fdc0050d0 con 0x7f1fe006d730 2023-10-11T09:48:58.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.460+0000 7f1fd5ffb700 1 --2- 172.21.15.78:0/1334251748 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1fc4065ed0 0x7f1fc4068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.460+0000 7f1fd5ffb700 1 -- 172.21.15.78:0/1334251748 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f1fdc083c70 con 0x7f1fe006d730 2023-10-11T09:48:58.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.460+0000 7f1fe49b2700 1 --2- 172.21.15.78:0/1334251748 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1fc4065ed0 0x7f1fc4068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.460+0000 7f1fea6ab740 1 Processor -- start 2023-10-11T09:48:58.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.460+0000 7f1fea6ab740 1 -- start start 2023-10-11T09:48:58.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.460+0000 7f1fe49b2700 1 --2- 172.21.15.78:0/1334251748 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1fc4065ed0 0x7f1fc4068380 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f1fe00ccfe0 tx=0x7f1fd0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:58.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.460+0000 7f1fea6ab740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c97c10 0x1f2d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.460+0000 7f1fea6ab740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1c97840 0x1f4ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.460+0000 7f1fea6ab740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1ce28c0 0x1f4f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.460+0000 7f1fea6ab740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e77630 con 0x1c97c10 2023-10-11T09:48:58.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.460+0000 7f1fea6ab740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e74dc0 con 0x1c97840 2023-10-11T09:48:58.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.460+0000 7f1fea6ab740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e706b0 con 0x1ce28c0 2023-10-11T09:48:58.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.464+0000 7f1fe51b3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c97c10 0x1f2d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.464+0000 7f1fe51b3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c97c10 0x1f2d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37726/0 (socket says 172.21.15.78:37726) 2023-10-11T09:48:58.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.464+0000 7f1fe51b3700 1 -- 172.21.15.78:0/2370599679 learned_addr learned my addr 172.21.15.78:0/2370599679 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:58.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.464+0000 7f1fe59b4700 1 --2- 172.21.15.78:0/2370599679 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1ce28c0 0x1f4f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.464+0000 7f1fe51b3700 1 -- 172.21.15.78:0/2370599679 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1ce28c0 msgr2=0x1f4f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:58.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.464+0000 7f1fe51b3700 1 --2- 172.21.15.78:0/2370599679 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1ce28c0 0x1f4f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.464+0000 7f1fe49b2700 1 --2- 172.21.15.78:0/2370599679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1c97840 0x1f4ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.464+0000 7f1fe51b3700 1 -- 172.21.15.78:0/2370599679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1c97840 msgr2=0x1f4ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:58.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.464+0000 7f1fe51b3700 1 --2- 172.21.15.78:0/2370599679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1c97840 0x1f4ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.464+0000 7f1fe51b3700 1 -- 172.21.15.78:0/2370599679 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1e965b0 con 0x1c97c10 2023-10-11T09:48:58.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.464+0000 7f1fe49b2700 1 --2- 172.21.15.78:0/2370599679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1c97840 0x1f4ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:48:58.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.464+0000 7f1fe51b3700 1 --2- 172.21.15.78:0/2370599679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c97c10 0x1f2d5a0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc05c700 tx=0x7f1fdc054630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b5fe40d7b6d3f52 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:58.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.464+0000 7f1fcaffd700 1 -- 172.21.15.78:0/2370599679 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc058a80 con 0x1c97c10 2023-10-11T09:48:58.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.464+0000 7f1fcaffd700 1 -- 172.21.15.78:0/2370599679 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fdc05cc30 con 0x1c97c10 2023-10-11T09:48:58.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.464+0000 7f1fcaffd700 1 -- 172.21.15.78:0/2370599679 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc081e10 con 0x1c97c10 2023-10-11T09:48:58.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.464+0000 7f1fea6ab740 1 -- 172.21.15.78:0/2370599679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c97c10 msgr2=0x1f2d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:58.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.464+0000 7f1fea6ab740 1 --2- 172.21.15.78:0/2370599679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c97c10 0x1f2d5a0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc05c700 tx=0x7f1fdc054630 comp rx=0 tx=0).stop 2023-10-11T09:48:58.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.464+0000 7f1fea6ab740 1 -- 172.21.15.78:0/2370599679 shutdown_connections 2023-10-11T09:48:58.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.464+0000 7f1fea6ab740 1 --2- 172.21.15.78:0/2370599679 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1ce28c0 0x1f4f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.464+0000 7f1fea6ab740 1 --2- 172.21.15.78:0/2370599679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1c97840 0x1f4ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.464+0000 7f1fea6ab740 1 --2- 172.21.15.78:0/2370599679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c97c10 0x1f2d5a0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.464+0000 7f1fea6ab740 1 -- 172.21.15.78:0/2370599679 >> 172.21.15.78:0/2370599679 conn(0x1c97fe0 msgr2=0x1f30350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:58.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.464+0000 7f1fea6ab740 1 -- 172.21.15.78:0/2370599679 shutdown_connections 2023-10-11T09:48:58.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.464+0000 7f1fea6ab740 1 -- 172.21.15.78:0/2370599679 wait complete. 2023-10-11T09:48:58.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.464+0000 7f1fea6ab740 1 -- start start 2023-10-11T09:48:58.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.464+0000 7f1fea6ab740 1 Processor -- start 2023-10-11T09:48:58.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.468+0000 7f1fea6ab740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1c97840 0x1f0c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.468+0000 7f1fea6ab740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c97c10 0x1f2b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.468+0000 7f1fea6ab740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1ce28c0 0x1f4de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.468+0000 7f1fea6ab740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e33c10 con 0x1c97c10 2023-10-11T09:48:58.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.468+0000 7f1fea6ab740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e338b0 con 0x1ce28c0 2023-10-11T09:48:58.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.468+0000 7f1fea6ab740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x1d1cc50 con 0x1c97840 2023-10-11T09:48:58.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.468+0000 7f1fea6ab740 10 client.0 _ll_register_callbacks cb 0x1e69ff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:48:58.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.468+0000 7f1fe49b2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c97c10 0x1f2b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.468+0000 7f1fe49b2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c97c10 0x1f2b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37740/0 (socket says 172.21.15.78:37740) 2023-10-11T09:48:58.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.468+0000 7f1fe49b2700 1 -- 172.21.15.78:0/1705964718 learned_addr learned my addr 172.21.15.78:0/1705964718 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:58.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.468+0000 7f1fe51b3700 1 --2- 172.21.15.78:0/1705964718 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1c97840 0x1f0c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.468+0000 7f1fe59b4700 1 --2- 172.21.15.78:0/1705964718 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1ce28c0 0x1f4de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.468+0000 7f1fe49b2700 1 -- 172.21.15.78:0/1705964718 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1c97840 msgr2=0x1f0c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:58.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.468+0000 7f1fe49b2700 1 --2- 172.21.15.78:0/1705964718 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1c97840 0x1f0c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.468+0000 7f1fe49b2700 1 -- 172.21.15.78:0/1705964718 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1ce28c0 msgr2=0x1f4de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:58.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.468+0000 7f1fe49b2700 1 --2- 172.21.15.78:0/1705964718 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1ce28c0 0x1f4de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.468+0000 7f1fe49b2700 1 -- 172.21.15.78:0/1705964718 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe014ec80 con 0x1c97c10 2023-10-11T09:48:58.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.468+0000 7f1fe59b4700 1 --2- 172.21.15.78:0/1705964718 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1ce28c0 0x1f4de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:48:58.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.468+0000 7f1fe49b2700 1 --2- 172.21.15.78:0/1705964718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c97c10 0x1f2b720 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f1fd00073d0 tx=0x7f1fd0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdfcc90f45f8e3a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:58.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.468+0000 7f1fbffff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:48:58.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.468+0000 7f1fbffff700 1 -- 172.21.15.78:0/1705964718 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0007840 con 0x1c97c10 2023-10-11T09:48:58.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.468+0000 7f1fea6ab740 1 -- 172.21.15.78:0/1705964718 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1e69730 con 0x1c97c10 2023-10-11T09:48:58.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.468+0000 7f1fbffff700 1 -- 172.21.15.78:0/1705964718 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f1fd00158d0 con 0x1c97c10 2023-10-11T09:48:58.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.468+0000 7f1fbffff700 1 -- 172.21.15.78:0/1705964718 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fd0005e80 con 0x1c97c10 2023-10-11T09:48:58.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.468+0000 7f1fea6ab740 10 client.7806 fetch_fsmap learned FSMap version 71 2023-10-11T09:48:58.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.468+0000 7f1fea6ab740 1 -- 172.21.15.78:0/1705964718 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x1ce3b10 con 0x1c97c10 2023-10-11T09:48:58.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.468+0000 7f1fbffff700 1 -- 172.21.15.78:0/1705964718 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1e69730 con 0x1c97c10 2023-10-11T09:48:58.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.468+0000 7f1fbffff700 1 -- 172.21.15.78:0/1705964718 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 71) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f1fd0015a90 con 0x1c97c10 2023-10-11T09:48:58.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.468+0000 7f1fea6ab740 10 client.7806 fetch_fsmap finished waiting for FSMap version 71 2023-10-11T09:48:58.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.468+0000 7f1fea6ab740 10 client.7806 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2023-10-11T09:48:58.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.468+0000 7f1fea6ab740 20 client.7806 populate_metadata read hostname 'smithi078' 2023-10-11T09:48:58.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.472+0000 7f1fea6ab740 1 --2- 172.21.15.78:0/1705964718 >> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] conn(0x1f0acf0 0x1f0b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.472+0000 7f1fea6ab740 4 client.7806 mds_command: new command op to 7725 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-11T09:48:58.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.472+0000 7f1fea6ab740 1 -- 172.21.15.78:0/1705964718 --> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1e69450 con 0x1f0acf0 2023-10-11T09:48:58.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.472+0000 7f1fe51b3700 1 --2- 172.21.15.78:0/1705964718 >> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] conn(0x1f0acf0 0x1f0b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.472+0000 7f1fe51b3700 1 --2- 172.21.15.78:0/1705964718 >> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] conn(0x1f0acf0 0x1f0b0f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f119e5d3f226385 server_cookie=395eec91287f94b5 in_seq=0 out_seq=0 2023-10-11T09:48:58.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.472+0000 7f1fbffff700 10 client.7806 ms_handle_connect on v2:172.21.15.133:6833/3922016223 2023-10-11T09:48:58.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.472+0000 7f1fbffff700 1 -- 172.21.15.78:0/1705964718 <== mds.0 v2:172.21.15.133:6833/3922016223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x1e69450 con 0x1f0acf0 2023-10-11T09:48:58.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.472+0000 7f1fbffff700 10 client.7806 handle_command_reply: tid=0 2023-10-11T09:48:58.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.472+0000 7f1fea6ab740 1 client.7806 shutdown 2023-10-11T09:48:58.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.472+0000 7f1fea6ab740 1 -- 172.21.15.78:0/1705964718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c97c10 msgr2=0x1f2b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:58.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.472+0000 7f1fea6ab740 1 --2- 172.21.15.78:0/1705964718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c97c10 0x1f2b720 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f1fd00073d0 tx=0x7f1fd0000ed0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.472+0000 7f1fea6ab740 1 -- 172.21.15.78:0/1705964718 shutdown_connections 2023-10-11T09:48:58.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.472+0000 7f1fea6ab740 1 --2- 172.21.15.78:0/1705964718 >> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] conn(0x1f0acf0 0x1f0b0f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.472+0000 7f1fea6ab740 1 --2- 172.21.15.78:0/1705964718 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1ce28c0 0x1f4de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.472+0000 7f1fea6ab740 1 --2- 172.21.15.78:0/1705964718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c97c10 0x1f2b720 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.472+0000 7f1fea6ab740 1 --2- 172.21.15.78:0/1705964718 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1c97840 0x1f0c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.472+0000 7f1fbffff700 0 client.7806 ms_handle_reset on v2:172.21.15.133:6833/3922016223 2023-10-11T09:48:58.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.472+0000 7f1fea6ab740 1 -- 172.21.15.78:0/1705964718 >> 172.21.15.78:0/1705964718 conn(0x1c97fe0 msgr2=0x1f30750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:58.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.476+0000 7f1fea6ab740 1 -- 172.21.15.78:0/1705964718 shutdown_connections 2023-10-11T09:48:58.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.476+0000 7f1fea6ab740 1 -- 172.21.15.78:0/1705964718 wait complete. 2023-10-11T09:48:58.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.476+0000 7f1fea6ab740 20 client.7806 trim_cache size 0 max 16384 2023-10-11T09:48:58.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.492+0000 7f1fea6ab740 1 Processor -- start 2023-10-11T09:48:58.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.492+0000 7f1fea6ab740 1 -- start start 2023-10-11T09:48:58.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.496+0000 7f1fea6ab740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c97840 0x1f4a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.496+0000 7f1fea6ab740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1c97c10 0x1f4ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.496+0000 7f1fea6ab740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1ce28c0 0x1f4b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.496+0000 7f1fea6ab740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e73210 con 0x1c97840 2023-10-11T09:48:58.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.496+0000 7f1fea6ab740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e1d650 con 0x1c97c10 2023-10-11T09:48:58.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.496+0000 7f1fea6ab740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e6fec0 con 0x1ce28c0 2023-10-11T09:48:58.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.496+0000 7f1fe51b3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c97840 0x1f4a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.496+0000 7f1fe51b3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c97840 0x1f4a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37746/0 (socket says 172.21.15.78:37746) 2023-10-11T09:48:58.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.496+0000 7f1fe51b3700 1 -- 172.21.15.78:0/3162704387 learned_addr learned my addr 172.21.15.78:0/3162704387 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:58.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.496+0000 7f1fe59b4700 1 --2- 172.21.15.78:0/3162704387 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1ce28c0 0x1f4b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.496+0000 7f1fe49b2700 1 --2- 172.21.15.78:0/3162704387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1c97c10 0x1f4ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.496+0000 7f1fe51b3700 1 -- 172.21.15.78:0/3162704387 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1ce28c0 msgr2=0x1f4b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:58.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.496+0000 7f1fe51b3700 1 --2- 172.21.15.78:0/3162704387 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1ce28c0 0x1f4b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.496+0000 7f1fe51b3700 1 -- 172.21.15.78:0/3162704387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1c97c10 msgr2=0x1f4ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:58.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.496+0000 7f1fe51b3700 1 --2- 172.21.15.78:0/3162704387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1c97c10 0x1f4ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.496+0000 7f1fe51b3700 1 -- 172.21.15.78:0/3162704387 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe00cc980 con 0x1c97840 2023-10-11T09:48:58.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.496+0000 7f1fe59b4700 1 --2- 172.21.15.78:0/3162704387 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1ce28c0 0x1f4b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:48:58.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.496+0000 7f1fe51b3700 1 --2- 172.21.15.78:0/3162704387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c97840 0x1f4a840 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc052fa0 tx=0x7f1fdc01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a66e119a727e21 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:58.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.496+0000 7f1fcb7fe700 1 -- 172.21.15.78:0/3162704387 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc058b00 con 0x1c97840 2023-10-11T09:48:58.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.496+0000 7f1fcb7fe700 1 -- 172.21.15.78:0/3162704387 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fdc052ff0 con 0x1c97840 2023-10-11T09:48:58.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.496+0000 7f1fcb7fe700 1 -- 172.21.15.78:0/3162704387 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc071230 con 0x1c97840 2023-10-11T09:48:58.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.496+0000 7f1fea6ab740 1 -- 172.21.15.78:0/3162704387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c97840 msgr2=0x1f4a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:58.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.496+0000 7f1fea6ab740 1 --2- 172.21.15.78:0/3162704387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c97840 0x1f4a840 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc052fa0 tx=0x7f1fdc01d000 comp rx=0 tx=0).stop 2023-10-11T09:48:58.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.496+0000 7f1fea6ab740 1 -- 172.21.15.78:0/3162704387 shutdown_connections 2023-10-11T09:48:58.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.496+0000 7f1fea6ab740 1 --2- 172.21.15.78:0/3162704387 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1ce28c0 0x1f4b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.496+0000 7f1fea6ab740 1 --2- 172.21.15.78:0/3162704387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1c97c10 0x1f4ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.496+0000 7f1fea6ab740 1 --2- 172.21.15.78:0/3162704387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c97840 0x1f4a840 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.496+0000 7f1fea6ab740 1 -- 172.21.15.78:0/3162704387 >> 172.21.15.78:0/3162704387 conn(0x1c97fe0 msgr2=0x1f2fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:58.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.496+0000 7f1fea6ab740 1 -- 172.21.15.78:0/3162704387 shutdown_connections 2023-10-11T09:48:58.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.496+0000 7f1fea6ab740 1 -- 172.21.15.78:0/3162704387 wait complete. 2023-10-11T09:48:58.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.500+0000 7f1fea6ab740 1 -- start start 2023-10-11T09:48:58.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.500+0000 7f1fea6ab740 1 Processor -- start 2023-10-11T09:48:58.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.500+0000 7f1fea6ab740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1c97840 0x1f092f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.500+0000 7f1fea6ab740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c97c10 0x1faa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.500+0000 7f1fea6ab740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1ce28c0 0x1faac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.500+0000 7f1fea6ab740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f02b80 con 0x1c97c10 2023-10-11T09:48:58.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.500+0000 7f1fea6ab740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f02950 con 0x1ce28c0 2023-10-11T09:48:58.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.500+0000 7f1fea6ab740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x1f2b910 con 0x1c97840 2023-10-11T09:48:58.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.500+0000 7f1fea6ab740 10 client.0 _ll_register_callbacks cb 0x1e6ee10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:48:58.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.500+0000 7f1fe49b2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c97c10 0x1faa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.500+0000 7f1fe51b3700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1c97840 0x1f092f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.500+0000 7f1fe49b2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c97c10 0x1faa6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37756/0 (socket says 172.21.15.78:37756) 2023-10-11T09:48:58.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.500+0000 7f1fe49b2700 1 -- 172.21.15.78:0/1190987555 learned_addr learned my addr 172.21.15.78:0/1190987555 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:48:58.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.500+0000 7f1fe59b4700 1 --2- 172.21.15.78:0/1190987555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1ce28c0 0x1faac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.500+0000 7f1fe49b2700 1 -- 172.21.15.78:0/1190987555 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1c97840 msgr2=0x1f092f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:58.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.500+0000 7f1fe49b2700 1 --2- 172.21.15.78:0/1190987555 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1c97840 0x1f092f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.500+0000 7f1fe49b2700 1 -- 172.21.15.78:0/1190987555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1ce28c0 msgr2=0x1faac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:48:58.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.500+0000 7f1fe49b2700 1 --2- 172.21.15.78:0/1190987555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1ce28c0 0x1faac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.500+0000 7f1fe49b2700 1 -- 172.21.15.78:0/1190987555 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1e965b0 con 0x1c97c10 2023-10-11T09:48:58.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.504+0000 7f1fe49b2700 1 --2- 172.21.15.78:0/1190987555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c97c10 0x1faa6c0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0001220 tx=0x7f1fd0007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b177aa818b39d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:48:58.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.504+0000 7f1fbffff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:48:58.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.504+0000 7f1fbffff700 1 -- 172.21.15.78:0/1190987555 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd00013b0 con 0x1c97c10 2023-10-11T09:48:58.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.504+0000 7f1fea6ab740 1 -- 172.21.15.78:0/1190987555 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1fdc05cc30 con 0x1c97c10 2023-10-11T09:48:58.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.504+0000 7f1fbffff700 1 -- 172.21.15.78:0/1190987555 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f1fd002c430 con 0x1c97c10 2023-10-11T09:48:58.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.504+0000 7f1fbffff700 1 -- 172.21.15.78:0/1190987555 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fd0015bd0 con 0x1c97c10 2023-10-11T09:48:58.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.504+0000 7f1fea6ab740 10 client.7812 fetch_fsmap learned FSMap version 71 2023-10-11T09:48:58.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.504+0000 7f1fea6ab740 1 -- 172.21.15.78:0/1190987555 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x1e36820 con 0x1c97c10 2023-10-11T09:48:58.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.504+0000 7f1fbffff700 1 -- 172.21.15.78:0/1190987555 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fdc05cc30 con 0x1c97c10 2023-10-11T09:48:58.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.504+0000 7f1fbffff700 1 -- 172.21.15.78:0/1190987555 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 71) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f1fd002c5f0 con 0x1c97c10 2023-10-11T09:48:58.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.504+0000 7f1fea6ab740 10 client.7812 fetch_fsmap finished waiting for FSMap version 71 2023-10-11T09:48:58.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.504+0000 7f1fea6ab740 10 client.7812 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2023-10-11T09:48:58.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.504+0000 7f1fea6ab740 20 client.7812 populate_metadata read hostname 'smithi078' 2023-10-11T09:48:58.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.504+0000 7f1fea6ab740 1 --2- 172.21.15.78:0/1190987555 >> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] conn(0x1fab140 0x1f04eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:48:58.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.504+0000 7f1fea6ab740 4 client.7812 mds_command: new command op to 7725 tid=0[{"prefix": "scrub status"}] 2023-10-11T09:48:58.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.504+0000 7f1fea6ab740 1 -- 172.21.15.78:0/1190987555 --> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x1cef610 con 0x1fab140 2023-10-11T09:48:58.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.504+0000 7f1fe51b3700 1 --2- 172.21.15.78:0/1190987555 >> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] conn(0x1fab140 0x1f04eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:48:58.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.504+0000 7f1fe51b3700 1 --2- 172.21.15.78:0/1190987555 >> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] conn(0x1fab140 0x1f04eb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c14d712a815f77c server_cookie=cdec1222d3888ad5 in_seq=0 out_seq=0 2023-10-11T09:48:58.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.504+0000 7f1fbffff700 10 client.7812 ms_handle_connect on v2:172.21.15.133:6833/3922016223 2023-10-11T09:48:58.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.504+0000 7f1fbffff700 1 -- 172.21.15.78:0/1190987555 <== mds.0 v2:172.21.15.133:6833/3922016223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x1cef610 con 0x1fab140 2023-10-11T09:48:58.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.504+0000 7f1fbffff700 10 client.7812 handle_command_reply: tid=0 2023-10-11T09:48:58.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.504+0000 7f1fea6ab740 1 client.7812 shutdown 2023-10-11T09:48:58.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.504+0000 7f1fea6ab740 1 -- 172.21.15.78:0/1190987555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c97c10 msgr2=0x1faa6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:58.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.504+0000 7f1fea6ab740 1 --2- 172.21.15.78:0/1190987555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c97c10 0x1faa6c0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0001220 tx=0x7f1fd0007000 comp rx=0 tx=0).stop 2023-10-11T09:48:58.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.504+0000 7f1fea6ab740 1 -- 172.21.15.78:0/1190987555 shutdown_connections 2023-10-11T09:48:58.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.504+0000 7f1fea6ab740 1 --2- 172.21.15.78:0/1190987555 >> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] conn(0x1fab140 0x1f04eb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.504+0000 7f1fea6ab740 1 --2- 172.21.15.78:0/1190987555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1ce28c0 0x1faac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.504+0000 7f1fea6ab740 1 --2- 172.21.15.78:0/1190987555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1c97c10 0x1faa6c0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.504+0000 7f1fbffff700 0 client.7812 ms_handle_reset on v2:172.21.15.133:6833/3922016223 2023-10-11T09:48:58.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.504+0000 7f1fea6ab740 1 --2- 172.21.15.78:0/1190987555 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1c97840 0x1f092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.504+0000 7f1fea6ab740 1 -- 172.21.15.78:0/1190987555 >> 172.21.15.78:0/1190987555 conn(0x1c97fe0 msgr2=0x1f4cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:58.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.508+0000 7f1fea6ab740 1 -- 172.21.15.78:0/1190987555 shutdown_connections 2023-10-11T09:48:58.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.508+0000 7f1fea6ab740 1 -- 172.21.15.78:0/1190987555 wait complete. 2023-10-11T09:48:58.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.508+0000 7f1fea6ab740 20 client.7812 trim_cache size 0 max 16384 2023-10-11T09:48:58.521 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:48:58.521 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-11T09:48:58.521 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2023-10-11T09:48:58.521 INFO:teuthology.orchestra.run.smithi078.stdout: "3625d844-7f25-43ba-8fe4-f63d2736b540": { 2023-10-11T09:48:58.522 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2023-10-11T09:48:58.522 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "3625d844-7f25-43ba-8fe4-f63d2736b540", 2023-10-11T09:48:58.522 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,repair" 2023-10-11T09:48:58.522 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:48:58.522 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:48:58.523 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:48:58.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.508+0000 7f1fe61b5700 1 -- 172.21.15.78:0/1334251748 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1fc4065ed0 msgr2=0x7f1fc4068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:58.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.508+0000 7f1fe61b5700 1 --2- 172.21.15.78:0/1334251748 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1fc4065ed0 0x7f1fc4068380 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f1fe00ccfe0 tx=0x7f1fd0009040 comp rx=0 tx=0).stop 2023-10-11T09:48:58.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.508+0000 7f1fe61b5700 1 -- 172.21.15.78:0/1334251748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fe006d730 msgr2=0x7f1fe014e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:48:58.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.508+0000 7f1fe61b5700 1 --2- 172.21.15.78:0/1334251748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fe006d730 0x7f1fe014e740 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc001ed0 tx=0x7f1fdc00a340 comp rx=0 tx=0).stop 2023-10-11T09:48:58.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.508+0000 7f1fe61b5700 1 -- 172.21.15.78:0/1334251748 shutdown_connections 2023-10-11T09:48:58.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.508+0000 7f1fe61b5700 1 --2- 172.21.15.78:0/1334251748 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1fc4065ed0 0x7f1fc4068380 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.508+0000 7f1fe61b5700 1 --2- 172.21.15.78:0/1334251748 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1fe015a6b0 0x7f1fe00ad700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.508+0000 7f1fe61b5700 1 --2- 172.21.15.78:0/1334251748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fe00b3a60 0x7f1fe00ad1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.508+0000 7f1fe61b5700 1 --2- 172.21.15.78:0/1334251748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fe006d730 0x7f1fe014e740 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:48:58.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.508+0000 7f1fe61b5700 1 -- 172.21.15.78:0/1334251748 >> 172.21.15.78:0/1334251748 conn(0x7f1fe006a060 msgr2=0x7f1fe00ce5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:48:58.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.508+0000 7f1fe61b5700 1 -- 172.21.15.78:0/1334251748 shutdown_connections 2023-10-11T09:48:58.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:48:58.508+0000 7f1fe61b5700 1 -- 172.21.15.78:0/1334251748 wait complete. 2023-10-11T09:48:58.535 INFO:tasks.cephfs.filesystem:scrub status for tag:3625d844-7f25-43ba-8fe4-f63d2736b540 - {'path': '/', 'tag': '3625d844-7f25-43ba-8fe4-f63d2736b540', 'options': 'recursive,repair'} 2023-10-11T09:49:00.424 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:49:00.431 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:49:28.537 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2023-10-11T09:49:28.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.715+0000 7f0c7ec5e700 1 Processor -- start 2023-10-11T09:49:28.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.715+0000 7f0c7ec5e700 1 -- start start 2023-10-11T09:49:28.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.715+0000 7f0c7ec5e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0c780b84c0 0x7f0c780cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:28.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.715+0000 7f0c7ec5e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c780b8890 0x7f0c78160370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:28.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.715+0000 7f0c7ec5e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0c781608b0 0x7f0c78160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:28.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.715+0000 7f0c7ec5e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c78161220 con 0x7f0c781608b0 2023-10-11T09:49:28.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.715+0000 7f0c7ec5e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c780cdea0 con 0x7f0c780b84c0 2023-10-11T09:49:28.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.715+0000 7f0c7ec5e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c780d0c10 con 0x7f0c780b8890 2023-10-11T09:49:28.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.719+0000 7f0c7d1fb700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0c781608b0 0x7f0c78160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:28.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.719+0000 7f0c77fff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c780b8890 0x7f0c78160370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:28.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.719+0000 7f0c7c9fa700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0c780b84c0 0x7f0c780cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:28.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.719+0000 7f0c77fff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c780b8890 0x7f0c78160370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:39924/0 (socket says 172.21.15.78:39924) 2023-10-11T09:49:28.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.719+0000 7f0c77fff700 1 -- 172.21.15.78:0/199109519 learned_addr learned my addr 172.21.15.78:0/199109519 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:28.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.719+0000 7f0c77fff700 1 -- 172.21.15.78:0/199109519 >> v2:172.21.15.133:3300/0 conn(0x7f0c780b84c0 msgr2=0x7f0c780cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:28.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.719+0000 7f0c77fff700 1 --2- 172.21.15.78:0/199109519 >> v2:172.21.15.133:3300/0 conn(0x7f0c780b84c0 0x7f0c780cfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:28.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.719+0000 7f0c77fff700 1 -- 172.21.15.78:0/199109519 >> v2:172.21.15.78:3300/0 conn(0x7f0c781608b0 msgr2=0x7f0c78160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:28.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.719+0000 7f0c77fff700 1 --2- 172.21.15.78:0/199109519 >> v2:172.21.15.78:3300/0 conn(0x7f0c781608b0 0x7f0c78160ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:28.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.719+0000 7f0c77fff700 1 -- 172.21.15.78:0/199109519 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c780cd690 con 0x7f0c780b8890 2023-10-11T09:49:28.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.719+0000 7f0c77fff700 1 --2- 172.21.15.78:0/199109519 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c780b8890 0x7f0c78160370 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0c780675c0 tx=0x7f0c64009bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45d36a4cde832dd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:28.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.719+0000 7f0c777fe700 1 -- 172.21.15.78:0/199109519 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c64011830 con 0x7f0c780b8890 2023-10-11T09:49:28.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.719+0000 7f0c777fe700 1 -- 172.21.15.78:0/199109519 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c64011990 con 0x7f0c780b8890 2023-10-11T09:49:28.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.719+0000 7f0c777fe700 1 -- 172.21.15.78:0/199109519 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c64011d20 con 0x7f0c780b8890 2023-10-11T09:49:28.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.719+0000 7f0c7ec5e700 1 -- 172.21.15.78:0/199109519 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c780b8890 msgr2=0x7f0c78160370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:28.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.719+0000 7f0c7ec5e700 1 --2- 172.21.15.78:0/199109519 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c780b8890 0x7f0c78160370 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0c780675c0 tx=0x7f0c64009bf0 comp rx=0 tx=0).stop 2023-10-11T09:49:28.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.723+0000 7f0c7ec5e700 1 -- 172.21.15.78:0/199109519 shutdown_connections 2023-10-11T09:49:28.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.723+0000 7f0c7ec5e700 1 --2- 172.21.15.78:0/199109519 >> v2:172.21.15.78:3300/0 conn(0x7f0c781608b0 0x7f0c78160ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:28.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.723+0000 7f0c7ec5e700 1 --2- 172.21.15.78:0/199109519 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c780b8890 0x7f0c78160370 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:28.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.723+0000 7f0c7ec5e700 1 --2- 172.21.15.78:0/199109519 >> v2:172.21.15.133:3300/0 conn(0x7f0c780b84c0 0x7f0c780cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:28.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.723+0000 7f0c7ec5e700 1 -- 172.21.15.78:0/199109519 >> 172.21.15.78:0/199109519 conn(0x7f0c7806cb50 msgr2=0x7f0c7806cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:28.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.723+0000 7f0c7ec5e700 1 -- 172.21.15.78:0/199109519 shutdown_connections 2023-10-11T09:49:28.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.723+0000 7f0c7ec5e700 1 -- 172.21.15.78:0/199109519 wait complete. 2023-10-11T09:49:28.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.723+0000 7f0c7ec5e700 1 Processor -- start 2023-10-11T09:49:28.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.727+0000 7f0c7ec5e700 1 -- start start 2023-10-11T09:49:28.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.727+0000 7f0c7ec5e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c780b84c0 0x7f0c7814d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:28.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.727+0000 7f0c7ec5e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c780b8890 0x7f0c7815f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:28.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.727+0000 7f0c7ec5e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c781608b0 0x7f0c7806b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:28.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.727+0000 7f0c7ec5e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c780c8230 con 0x7f0c780b84c0 2023-10-11T09:49:28.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.727+0000 7f0c7ec5e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7815d900 con 0x7f0c780b8890 2023-10-11T09:49:28.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.727+0000 7f0c7ec5e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7815dbe0 con 0x7f0c781608b0 2023-10-11T09:49:28.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.727+0000 7f0c7c9fa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c780b84c0 0x7f0c7814d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:28.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.727+0000 7f0c7c9fa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c780b84c0 0x7f0c7814d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39670/0 (socket says 172.21.15.78:39670) 2023-10-11T09:49:28.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.727+0000 7f0c7c9fa700 1 -- 172.21.15.78:0/1074789300 learned_addr learned my addr 172.21.15.78:0/1074789300 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:28.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.727+0000 7f0c77fff700 1 --2- 172.21.15.78:0/1074789300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c780b8890 0x7f0c7815f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:28.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.727+0000 7f0c7d1fb700 1 --2- 172.21.15.78:0/1074789300 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c781608b0 0x7f0c7806b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:28.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.727+0000 7f0c7c9fa700 1 -- 172.21.15.78:0/1074789300 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c781608b0 msgr2=0x7f0c7806b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:28.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.727+0000 7f0c7c9fa700 1 --2- 172.21.15.78:0/1074789300 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c781608b0 0x7f0c7806b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:28.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.727+0000 7f0c7c9fa700 1 -- 172.21.15.78:0/1074789300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c780b8890 msgr2=0x7f0c7815f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:28.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.727+0000 7f0c7c9fa700 1 --2- 172.21.15.78:0/1074789300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c780b8890 0x7f0c7815f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:28.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.727+0000 7f0c7c9fa700 1 -- 172.21.15.78:0/1074789300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c64009570 con 0x7f0c780b84c0 2023-10-11T09:49:28.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.727+0000 7f0c77fff700 1 --2- 172.21.15.78:0/1074789300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c780b8890 0x7f0c7815f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:49:28.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.727+0000 7f0c7c9fa700 1 --2- 172.21.15.78:0/1074789300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c780b84c0 0x7f0c7814d350 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c00efc0 tx=0x7f0c6c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:28.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.731+0000 7f0c757fa700 1 -- 172.21.15.78:0/1074789300 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c00cdc0 con 0x7f0c780b84c0 2023-10-11T09:49:28.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.731+0000 7f0c757fa700 1 -- 172.21.15.78:0/1074789300 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c6c00f040 con 0x7f0c780b84c0 2023-10-11T09:49:28.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.731+0000 7f0c7ec5e700 1 -- 172.21.15.78:0/1074789300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c780c84d0 con 0x7f0c780b84c0 2023-10-11T09:49:28.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.731+0000 7f0c757fa700 1 -- 172.21.15.78:0/1074789300 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c0146a0 con 0x7f0c780b84c0 2023-10-11T09:49:28.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.731+0000 7f0c7ec5e700 1 -- 172.21.15.78:0/1074789300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c7815dec0 con 0x7f0c780b84c0 2023-10-11T09:49:28.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.735+0000 7f0c757fa700 1 -- 172.21.15.78:0/1074789300 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f0c6c014800 con 0x7f0c780b84c0 2023-10-11T09:49:28.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.735+0000 7f0c757fa700 1 --2- 172.21.15.78:0/1074789300 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0c5c065ed0 0x7f0c5c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:28.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.735+0000 7f0c757fa700 1 -- 172.21.15.78:0/1074789300 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f0c6c084450 con 0x7f0c780b84c0 2023-10-11T09:49:28.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.735+0000 7f0c77fff700 1 --2- 172.21.15.78:0/1074789300 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0c5c065ed0 0x7f0c5c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:28.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.739+0000 7f0c77fff700 1 --2- 172.21.15.78:0/1074789300 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0c5c065ed0 0x7f0c5c068380 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f0c64000c00 tx=0x7f0c64008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:28.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.739+0000 7f0c81ef5740 1 Processor -- start 2023-10-11T09:49:28.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.739+0000 7f0c81ef5740 1 -- start start 2023-10-11T09:49:28.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.739+0000 7f0c81ef5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x213dc10 0x23d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:28.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.739+0000 7f0c81ef5740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x213d840 0x23f4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:28.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.739+0000 7f0c81ef5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x21888c0 0x23f50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:28.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.739+0000 7f0c81ef5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x231d630 con 0x21888c0 2023-10-11T09:49:28.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.739+0000 7f0c81ef5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x231adc0 con 0x213dc10 2023-10-11T09:49:28.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.739+0000 7f0c81ef5740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x23166b0 con 0x213d840 2023-10-11T09:49:28.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.739+0000 7f0c7d1fb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x21888c0 0x23f50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:28.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.739+0000 7f0c7d1fb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x21888c0 0x23f50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39678/0 (socket says 172.21.15.78:39678) 2023-10-11T09:49:28.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.739+0000 7f0c7d1fb700 1 -- 172.21.15.78:0/2768065379 learned_addr learned my addr 172.21.15.78:0/2768065379 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:28.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.739+0000 7f0c77fff700 1 --2- 172.21.15.78:0/2768065379 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x213d840 0x23f4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:28.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.739+0000 7f0c7c9fa700 1 --2- 172.21.15.78:0/2768065379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x213dc10 0x23d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:28.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.739+0000 7f0c7d1fb700 1 -- 172.21.15.78:0/2768065379 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x213d840 msgr2=0x23f4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:28.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.739+0000 7f0c7d1fb700 1 --2- 172.21.15.78:0/2768065379 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x213d840 0x23f4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:28.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.739+0000 7f0c7d1fb700 1 -- 172.21.15.78:0/2768065379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x213dc10 msgr2=0x23d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:28.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.739+0000 7f0c7d1fb700 1 --2- 172.21.15.78:0/2768065379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x213dc10 0x23d35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:28.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.739+0000 7f0c7d1fb700 1 -- 172.21.15.78:0/2768065379 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x233c5b0 con 0x21888c0 2023-10-11T09:49:28.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.739+0000 7f0c7d1fb700 1 --2- 172.21.15.78:0/2768065379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x21888c0 0x23f50f0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f0c70009b20 tx=0x7f0c70009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d971e46d22d37fb0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:28.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.739+0000 7f0c627fc700 1 -- 172.21.15.78:0/2768065379 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c70011ae0 con 0x21888c0 2023-10-11T09:49:28.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.739+0000 7f0c627fc700 1 -- 172.21.15.78:0/2768065379 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c70011c40 con 0x21888c0 2023-10-11T09:49:28.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.739+0000 7f0c627fc700 1 -- 172.21.15.78:0/2768065379 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c70014440 con 0x21888c0 2023-10-11T09:49:28.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.743+0000 7f0c81ef5740 1 -- 172.21.15.78:0/2768065379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x21888c0 msgr2=0x23f50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:28.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.743+0000 7f0c81ef5740 1 --2- 172.21.15.78:0/2768065379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x21888c0 0x23f50f0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f0c70009b20 tx=0x7f0c70009e30 comp rx=0 tx=0).stop 2023-10-11T09:49:28.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.743+0000 7f0c81ef5740 1 -- 172.21.15.78:0/2768065379 shutdown_connections 2023-10-11T09:49:28.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.743+0000 7f0c81ef5740 1 --2- 172.21.15.78:0/2768065379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x21888c0 0x23f50f0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:28.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.743+0000 7f0c81ef5740 1 --2- 172.21.15.78:0/2768065379 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x213d840 0x23f4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:28.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.743+0000 7f0c81ef5740 1 --2- 172.21.15.78:0/2768065379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x213dc10 0x23d35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:28.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.743+0000 7f0c81ef5740 1 -- 172.21.15.78:0/2768065379 >> 172.21.15.78:0/2768065379 conn(0x213dfe0 msgr2=0x23d6350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:28.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.743+0000 7f0c81ef5740 1 -- 172.21.15.78:0/2768065379 shutdown_connections 2023-10-11T09:49:28.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.743+0000 7f0c81ef5740 1 -- 172.21.15.78:0/2768065379 wait complete. 2023-10-11T09:49:28.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.743+0000 7f0c81ef5740 1 -- start start 2023-10-11T09:49:28.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.743+0000 7f0c81ef5740 1 Processor -- start 2023-10-11T09:49:28.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.747+0000 7f0c81ef5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x213d840 0x23acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:28.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.747+0000 7f0c81ef5740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x213dc10 0x23add30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:28.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.747+0000 7f0c81ef5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x21888c0 0x23af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:28.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.747+0000 7f0c81ef5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x22d9c10 con 0x213d840 2023-10-11T09:49:28.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.747+0000 7f0c81ef5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x22d98b0 con 0x21888c0 2023-10-11T09:49:28.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.747+0000 7f0c81ef5740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x21c2c50 con 0x213dc10 2023-10-11T09:49:28.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.747+0000 7f0c81ef5740 10 client.0 _ll_register_callbacks cb 0x230fff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:49:28.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.747+0000 7f0c7c9fa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x213d840 0x23acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:28.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.747+0000 7f0c7c9fa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x213d840 0x23acd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39686/0 (socket says 172.21.15.78:39686) 2023-10-11T09:49:28.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.747+0000 7f0c7c9fa700 1 -- 172.21.15.78:0/477024699 learned_addr learned my addr 172.21.15.78:0/477024699 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:28.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.747+0000 7f0c77fff700 1 --2- 172.21.15.78:0/477024699 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x213dc10 0x23add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:28.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.747+0000 7f0c7d1fb700 1 --2- 172.21.15.78:0/477024699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x21888c0 0x23af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:28.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.747+0000 7f0c7c9fa700 1 -- 172.21.15.78:0/477024699 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x213dc10 msgr2=0x23add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:28.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.747+0000 7f0c7c9fa700 1 --2- 172.21.15.78:0/477024699 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x213dc10 0x23add30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:28.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.747+0000 7f0c7c9fa700 1 -- 172.21.15.78:0/477024699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x21888c0 msgr2=0x23af8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:28.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.747+0000 7f0c7c9fa700 1 --2- 172.21.15.78:0/477024699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x21888c0 0x23af8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:28.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.747+0000 7f0c7c9fa700 1 -- 172.21.15.78:0/477024699 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c700097d0 con 0x213d840 2023-10-11T09:49:28.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.747+0000 7f0c7d1fb700 1 --2- 172.21.15.78:0/477024699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x21888c0 0x23af8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:49:28.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.747+0000 7f0c7c9fa700 1 --2- 172.21.15.78:0/477024699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x213d840 0x23acd30 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c0553b0 tx=0x7f0c6c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9cba43849e9e223 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:28.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.747+0000 7f0c577fe700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:49:28.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.747+0000 7f0c577fe700 1 -- 172.21.15.78:0/477024699 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c054b10 con 0x213d840 2023-10-11T09:49:28.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.747+0000 7f0c81ef5740 1 -- 172.21.15.78:0/477024699 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x230f730 con 0x213d840 2023-10-11T09:49:28.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.747+0000 7f0c577fe700 1 -- 172.21.15.78:0/477024699 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f0c6c055420 con 0x213d840 2023-10-11T09:49:28.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.747+0000 7f0c577fe700 1 -- 172.21.15.78:0/477024699 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c6c059250 con 0x213d840 2023-10-11T09:49:28.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.747+0000 7f0c81ef5740 10 client.7824 fetch_fsmap learned FSMap version 71 2023-10-11T09:49:28.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.747+0000 7f0c81ef5740 1 -- 172.21.15.78:0/477024699 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x2189b10 con 0x213d840 2023-10-11T09:49:28.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.747+0000 7f0c577fe700 1 -- 172.21.15.78:0/477024699 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c6c070020 con 0x213d840 2023-10-11T09:49:28.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.747+0000 7f0c577fe700 1 -- 172.21.15.78:0/477024699 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 71) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f0c6c082080 con 0x213d840 2023-10-11T09:49:28.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.747+0000 7f0c81ef5740 10 client.7824 fetch_fsmap finished waiting for FSMap version 71 2023-10-11T09:49:28.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.747+0000 7f0c81ef5740 10 client.7824 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2023-10-11T09:49:28.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.747+0000 7f0c81ef5740 20 client.7824 populate_metadata read hostname 'smithi078' 2023-10-11T09:49:28.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.747+0000 7f0c81ef5740 1 --2- 172.21.15.78:0/477024699 >> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] conn(0x23d22a0 0x2407c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:28.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.747+0000 7f0c81ef5740 4 client.7824 mds_command: new command op to 7725 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-11T09:49:28.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.747+0000 7f0c81ef5740 1 -- 172.21.15.78:0/477024699 --> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x230f450 con 0x23d22a0 2023-10-11T09:49:28.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.747+0000 7f0c77fff700 1 --2- 172.21.15.78:0/477024699 >> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] conn(0x23d22a0 0x2407c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:28.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.751+0000 7f0c77fff700 1 --2- 172.21.15.78:0/477024699 >> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] conn(0x23d22a0 0x2407c00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8d2e2cfd709d17e server_cookie=bf7ecf15bea2effa in_seq=0 out_seq=0 2023-10-11T09:49:28.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.751+0000 7f0c577fe700 10 client.7824 ms_handle_connect on v2:172.21.15.133:6833/3922016223 2023-10-11T09:49:28.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.751+0000 7f0c577fe700 1 -- 172.21.15.78:0/477024699 <== mds.0 v2:172.21.15.133:6833/3922016223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x230f450 con 0x23d22a0 2023-10-11T09:49:28.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.751+0000 7f0c577fe700 10 client.7824 handle_command_reply: tid=0 2023-10-11T09:49:28.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.751+0000 7f0c81ef5740 1 client.7824 shutdown 2023-10-11T09:49:28.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.751+0000 7f0c81ef5740 1 -- 172.21.15.78:0/477024699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x213d840 msgr2=0x23acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:28.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.751+0000 7f0c81ef5740 1 --2- 172.21.15.78:0/477024699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x213d840 0x23acd30 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c0553b0 tx=0x7f0c6c013000 comp rx=0 tx=0).stop 2023-10-11T09:49:28.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.751+0000 7f0c81ef5740 1 -- 172.21.15.78:0/477024699 shutdown_connections 2023-10-11T09:49:28.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.751+0000 7f0c81ef5740 1 --2- 172.21.15.78:0/477024699 >> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] conn(0x23d22a0 0x2407c00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:28.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.751+0000 7f0c81ef5740 1 --2- 172.21.15.78:0/477024699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x21888c0 0x23af8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:28.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.751+0000 7f0c81ef5740 1 --2- 172.21.15.78:0/477024699 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x213dc10 0x23add30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:28.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.751+0000 7f0c81ef5740 1 --2- 172.21.15.78:0/477024699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x213d840 0x23acd30 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:28.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.751+0000 7f0c577fe700 0 client.7824 ms_handle_reset on v2:172.21.15.133:6833/3922016223 2023-10-11T09:49:28.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.751+0000 7f0c81ef5740 1 -- 172.21.15.78:0/477024699 >> 172.21.15.78:0/477024699 conn(0x213dfe0 msgr2=0x23d6a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:28.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.751+0000 7f0c81ef5740 1 -- 172.21.15.78:0/477024699 shutdown_connections 2023-10-11T09:49:28.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.751+0000 7f0c81ef5740 1 -- 172.21.15.78:0/477024699 wait complete. 2023-10-11T09:49:28.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.751+0000 7f0c81ef5740 20 client.7824 trim_cache size 0 max 16384 2023-10-11T09:49:28.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.775+0000 7f0c81ef5740 1 Processor -- start 2023-10-11T09:49:28.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.775+0000 7f0c81ef5740 1 -- start start 2023-10-11T09:49:28.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.775+0000 7f0c81ef5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x213d840 0x2403be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:28.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.775+0000 7f0c81ef5740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x213dc10 0x23f96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:28.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.775+0000 7f0c81ef5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x21888c0 0x23f9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:28.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.775+0000 7f0c81ef5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x23d60a0 con 0x21888c0 2023-10-11T09:49:28.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.775+0000 7f0c81ef5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x23aeff0 con 0x213d840 2023-10-11T09:49:28.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.775+0000 7f0c81ef5740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x22bb7c0 con 0x213dc10 2023-10-11T09:49:28.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.775+0000 7f0c7d1fb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x21888c0 0x23f9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:28.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.775+0000 7f0c7d1fb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x21888c0 0x23f9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39700/0 (socket says 172.21.15.78:39700) 2023-10-11T09:49:28.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.775+0000 7f0c7d1fb700 1 -- 172.21.15.78:0/3874939325 learned_addr learned my addr 172.21.15.78:0/3874939325 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:28.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.775+0000 7f0c77fff700 1 --2- 172.21.15.78:0/3874939325 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x213dc10 0x23f96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:28.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.775+0000 7f0c7c9fa700 1 --2- 172.21.15.78:0/3874939325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x213d840 0x2403be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:28.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.775+0000 7f0c7d1fb700 1 -- 172.21.15.78:0/3874939325 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x213dc10 msgr2=0x23f96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:28.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.775+0000 7f0c7d1fb700 1 --2- 172.21.15.78:0/3874939325 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x213dc10 0x23f96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:28.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.775+0000 7f0c7d1fb700 1 -- 172.21.15.78:0/3874939325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x213d840 msgr2=0x2403be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:28.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.775+0000 7f0c7d1fb700 1 --2- 172.21.15.78:0/3874939325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x213d840 0x2403be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:28.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.775+0000 7f0c7d1fb700 1 -- 172.21.15.78:0/3874939325 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7806b9e0 con 0x21888c0 2023-10-11T09:49:28.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.775+0000 7f0c77fff700 1 --2- 172.21.15.78:0/3874939325 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x213dc10 0x23f96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:49:28.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.775+0000 7f0c7d1fb700 1 --2- 172.21.15.78:0/3874939325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x21888c0 0x23f9be0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x229ff50 tx=0x7f0c70016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6512cc5255feb59f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:28.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.775+0000 7f0c62ffd700 1 -- 172.21.15.78:0/3874939325 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c70011ae0 con 0x21888c0 2023-10-11T09:49:28.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.775+0000 7f0c62ffd700 1 -- 172.21.15.78:0/3874939325 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c70009620 con 0x21888c0 2023-10-11T09:49:28.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.775+0000 7f0c62ffd700 1 -- 172.21.15.78:0/3874939325 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c70004270 con 0x21888c0 2023-10-11T09:49:28.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.779+0000 7f0c81ef5740 1 -- 172.21.15.78:0/3874939325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x21888c0 msgr2=0x23f9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:28.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.779+0000 7f0c81ef5740 1 --2- 172.21.15.78:0/3874939325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x21888c0 0x23f9be0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x229ff50 tx=0x7f0c70016000 comp rx=0 tx=0).stop 2023-10-11T09:49:28.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.779+0000 7f0c81ef5740 1 -- 172.21.15.78:0/3874939325 shutdown_connections 2023-10-11T09:49:28.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.779+0000 7f0c81ef5740 1 --2- 172.21.15.78:0/3874939325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x21888c0 0x23f9be0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:28.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.779+0000 7f0c81ef5740 1 --2- 172.21.15.78:0/3874939325 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x213dc10 0x23f96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:28.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.779+0000 7f0c81ef5740 1 --2- 172.21.15.78:0/3874939325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x213d840 0x2403be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:28.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.779+0000 7f0c81ef5740 1 -- 172.21.15.78:0/3874939325 >> 172.21.15.78:0/3874939325 conn(0x213dfe0 msgr2=0x23fc540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:28.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.779+0000 7f0c81ef5740 1 -- 172.21.15.78:0/3874939325 shutdown_connections 2023-10-11T09:49:28.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.779+0000 7f0c81ef5740 1 -- 172.21.15.78:0/3874939325 wait complete. 2023-10-11T09:49:28.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.779+0000 7f0c81ef5740 1 -- start start 2023-10-11T09:49:28.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.779+0000 7f0c81ef5740 1 Processor -- start 2023-10-11T09:49:28.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.779+0000 7f0c81ef5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x213d840 0x2450cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:28.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.779+0000 7f0c81ef5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x213dc10 0x23cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:28.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.779+0000 7f0c81ef5740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x21888c0 0x23d0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:28.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.779+0000 7f0c81ef5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x24511f0 con 0x213dc10 2023-10-11T09:49:28.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.779+0000 7f0c81ef5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x23d0670 con 0x213d840 2023-10-11T09:49:28.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.779+0000 7f0c81ef5740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x23ada70 con 0x21888c0 2023-10-11T09:49:28.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.779+0000 7f0c81ef5740 10 client.0 _ll_register_callbacks cb 0x2314e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:49:28.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.779+0000 7f0c77fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x213dc10 0x23cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:28.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.779+0000 7f0c77fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x213dc10 0x23cfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39704/0 (socket says 172.21.15.78:39704) 2023-10-11T09:49:28.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.779+0000 7f0c77fff700 1 -- 172.21.15.78:0/1902699731 learned_addr learned my addr 172.21.15.78:0/1902699731 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:28.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.779+0000 7f0c7d1fb700 1 --2- 172.21.15.78:0/1902699731 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x21888c0 0x23d0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:28.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.779+0000 7f0c7c9fa700 1 --2- 172.21.15.78:0/1902699731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x213d840 0x2450cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:28.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.783+0000 7f0c77fff700 1 -- 172.21.15.78:0/1902699731 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x21888c0 msgr2=0x23d0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:28.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.783+0000 7f0c77fff700 1 --2- 172.21.15.78:0/1902699731 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x21888c0 0x23d0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:28.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.783+0000 7f0c77fff700 1 -- 172.21.15.78:0/1902699731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x213d840 msgr2=0x2450cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:28.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.783+0000 7f0c77fff700 1 --2- 172.21.15.78:0/1902699731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x213d840 0x2450cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:28.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.783+0000 7f0c77fff700 1 -- 172.21.15.78:0/1902699731 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x233c5b0 con 0x213dc10 2023-10-11T09:49:28.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.783+0000 7f0c77fff700 1 --2- 172.21.15.78:0/1902699731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x213dc10 0x23cfbf0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f0c64013000 tx=0x7f0c6400e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b677a98dfa618fd server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:28.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.783+0000 7f0c577fe700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:49:28.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.783+0000 7f0c577fe700 1 -- 172.21.15.78:0/1902699731 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6402f040 con 0x213dc10 2023-10-11T09:49:28.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.783+0000 7f0c81ef5740 1 -- 172.21.15.78:0/1902699731 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x23ac220 con 0x213dc10 2023-10-11T09:49:28.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.783+0000 7f0c577fe700 1 -- 172.21.15.78:0/1902699731 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f0c640038c0 con 0x213dc10 2023-10-11T09:49:28.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.783+0000 7f0c577fe700 1 -- 172.21.15.78:0/1902699731 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c64003a80 con 0x213dc10 2023-10-11T09:49:28.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.783+0000 7f0c81ef5740 10 client.7830 fetch_fsmap learned FSMap version 71 2023-10-11T09:49:28.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.783+0000 7f0c81ef5740 1 -- 172.21.15.78:0/1902699731 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x22dc820 con 0x213dc10 2023-10-11T09:49:28.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.783+0000 7f0c577fe700 1 -- 172.21.15.78:0/1902699731 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x23ac220 con 0x213dc10 2023-10-11T09:49:28.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.783+0000 7f0c577fe700 1 -- 172.21.15.78:0/1902699731 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 71) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f0c64003be0 con 0x213dc10 2023-10-11T09:49:28.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.783+0000 7f0c81ef5740 10 client.7830 fetch_fsmap finished waiting for FSMap version 71 2023-10-11T09:49:28.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.783+0000 7f0c81ef5740 10 client.7830 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2023-10-11T09:49:28.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.783+0000 7f0c81ef5740 20 client.7830 populate_metadata read hostname 'smithi078' 2023-10-11T09:49:28.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.783+0000 7f0c81ef5740 1 --2- 172.21.15.78:0/1902699731 >> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] conn(0x23cc320 0x23cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:28.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.783+0000 7f0c81ef5740 4 client.7830 mds_command: new command op to 7725 tid=0[{"prefix": "scrub status"}] 2023-10-11T09:49:28.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.783+0000 7f0c81ef5740 1 -- 172.21.15.78:0/1902699731 --> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f0c70011c40 con 0x23cc320 2023-10-11T09:49:28.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.783+0000 7f0c7c9fa700 1 --2- 172.21.15.78:0/1902699731 >> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] conn(0x23cc320 0x23cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:28.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.783+0000 7f0c7c9fa700 1 --2- 172.21.15.78:0/1902699731 >> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] conn(0x23cc320 0x23cc720 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d38dc96cb5126a00 server_cookie=a1eaea3c83c75e44 in_seq=0 out_seq=0 2023-10-11T09:49:28.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.787+0000 7f0c577fe700 10 client.7830 ms_handle_connect on v2:172.21.15.133:6833/3922016223 2023-10-11T09:49:28.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.787+0000 7f0c577fe700 1 -- 172.21.15.78:0/1902699731 <== mds.0 v2:172.21.15.133:6833/3922016223 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7f0c70011c40 con 0x23cc320 2023-10-11T09:49:28.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.787+0000 7f0c577fe700 10 client.7830 handle_command_reply: tid=0 2023-10-11T09:49:28.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.787+0000 7f0c81ef5740 1 client.7830 shutdown 2023-10-11T09:49:28.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.787+0000 7f0c81ef5740 1 -- 172.21.15.78:0/1902699731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x213dc10 msgr2=0x23cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:28.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.787+0000 7f0c81ef5740 1 --2- 172.21.15.78:0/1902699731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x213dc10 0x23cfbf0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f0c64013000 tx=0x7f0c6400e000 comp rx=0 tx=0).stop 2023-10-11T09:49:28.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.787+0000 7f0c81ef5740 1 -- 172.21.15.78:0/1902699731 shutdown_connections 2023-10-11T09:49:28.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.787+0000 7f0c81ef5740 1 --2- 172.21.15.78:0/1902699731 >> [v2:172.21.15.133:6833/3922016223,v1:172.21.15.133:6835/3922016223] conn(0x23cc320 0x23cc720 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:28.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.787+0000 7f0c81ef5740 1 --2- 172.21.15.78:0/1902699731 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x21888c0 0x23d0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:28.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.787+0000 7f0c81ef5740 1 --2- 172.21.15.78:0/1902699731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x213dc10 0x23cfbf0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:28.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.787+0000 7f0c577fe700 0 client.7830 ms_handle_reset on v2:172.21.15.133:6833/3922016223 2023-10-11T09:49:28.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.787+0000 7f0c81ef5740 1 --2- 172.21.15.78:0/1902699731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x213d840 0x2450cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:28.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.787+0000 7f0c81ef5740 1 -- 172.21.15.78:0/1902699731 >> 172.21.15.78:0/1902699731 conn(0x213dfe0 msgr2=0x23cab70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:28.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.787+0000 7f0c81ef5740 1 -- 172.21.15.78:0/1902699731 shutdown_connections 2023-10-11T09:49:28.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.787+0000 7f0c81ef5740 1 -- 172.21.15.78:0/1902699731 wait complete. 2023-10-11T09:49:28.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.787+0000 7f0c81ef5740 20 client.7830 trim_cache size 0 max 16384 2023-10-11T09:49:28.803 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:49:28.803 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2023-10-11T09:49:28.803 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2023-10-11T09:49:28.804 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:49:28.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.787+0000 7f0c7ec5e700 1 -- 172.21.15.78:0/1074789300 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0c5c065ed0 msgr2=0x7f0c5c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:28.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.787+0000 7f0c7ec5e700 1 --2- 172.21.15.78:0/1074789300 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0c5c065ed0 0x7f0c5c068380 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f0c64000c00 tx=0x7f0c64008030 comp rx=0 tx=0).stop 2023-10-11T09:49:28.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.787+0000 7f0c7ec5e700 1 -- 172.21.15.78:0/1074789300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c780b84c0 msgr2=0x7f0c7814d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:28.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.787+0000 7f0c7ec5e700 1 --2- 172.21.15.78:0/1074789300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c780b84c0 0x7f0c7814d350 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c00efc0 tx=0x7f0c6c00c3a0 comp rx=0 tx=0).stop 2023-10-11T09:49:28.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.787+0000 7f0c7ec5e700 1 -- 172.21.15.78:0/1074789300 shutdown_connections 2023-10-11T09:49:28.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.787+0000 7f0c7ec5e700 1 --2- 172.21.15.78:0/1074789300 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0c5c065ed0 0x7f0c5c068380 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:28.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.787+0000 7f0c7ec5e700 1 --2- 172.21.15.78:0/1074789300 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c781608b0 0x7f0c7806b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:28.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.787+0000 7f0c7ec5e700 1 --2- 172.21.15.78:0/1074789300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c780b8890 0x7f0c7815f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:28.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.787+0000 7f0c7ec5e700 1 --2- 172.21.15.78:0/1074789300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c780b84c0 0x7f0c7814d350 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:28.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.787+0000 7f0c7ec5e700 1 -- 172.21.15.78:0/1074789300 >> 172.21.15.78:0/1074789300 conn(0x7f0c7806cb50 msgr2=0x7f0c780cfbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:28.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.787+0000 7f0c7ec5e700 1 -- 172.21.15.78:0/1074789300 shutdown_connections 2023-10-11T09:49:28.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:28.787+0000 7f0c7ec5e700 1 -- 172.21.15.78:0/1074789300 wait complete. 2023-10-11T09:49:28.817 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-11T09:49:28.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:49:28.819 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo ls subdir) 2023-10-11T09:49:28.868 INFO:teuthology.orchestra.run.smithi078.stdout:sixmegs 2023-10-11T09:49:28.872 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:49:28.873 DEBUG:teuthology.orchestra.run.smithi078:> import os 2023-10-11T09:49:28.873 DEBUG:teuthology.orchestra.run.smithi078:> import stat 2023-10-11T09:49:28.874 DEBUG:teuthology.orchestra.run.smithi078:> import json 2023-10-11T09:49:28.874 DEBUG:teuthology.orchestra.run.smithi078:> import sys 2023-10-11T09:49:28.874 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:49:28.875 DEBUG:teuthology.orchestra.run.smithi078:> try: 2023-10-11T09:49:28.875 DEBUG:teuthology.orchestra.run.smithi078:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2023-10-11T09:49:28.875 DEBUG:teuthology.orchestra.run.smithi078:> except OSError as e: 2023-10-11T09:49:28.876 DEBUG:teuthology.orchestra.run.smithi078:> sys.exit(e.errno) 2023-10-11T09:49:28.876 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:49:28.876 DEBUG:teuthology.orchestra.run.smithi078:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2023-10-11T09:49:28.877 DEBUG:teuthology.orchestra.run.smithi078:> print(json.dumps( 2023-10-11T09:49:28.877 DEBUG:teuthology.orchestra.run.smithi078:> dict([(a, getattr(s, a)) for a in attrs]), 2023-10-11T09:49:28.877 DEBUG:teuthology.orchestra.run.smithi078:> indent=2)) 2023-10-11T09:49:28.878 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:49:29.034 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:49:29.034 INFO:teuthology.orchestra.run.smithi078.stdout: "st_mode": 33088, 2023-10-11T09:49:29.034 INFO:teuthology.orchestra.run.smithi078.stdout: "st_ino": 1099511627777, 2023-10-11T09:49:29.035 INFO:teuthology.orchestra.run.smithi078.stdout: "st_dev": 49, 2023-10-11T09:49:29.035 INFO:teuthology.orchestra.run.smithi078.stdout: "st_nlink": 1, 2023-10-11T09:49:29.035 INFO:teuthology.orchestra.run.smithi078.stdout: "st_uid": 0, 2023-10-11T09:49:29.035 INFO:teuthology.orchestra.run.smithi078.stdout: "st_gid": 0, 2023-10-11T09:49:29.035 INFO:teuthology.orchestra.run.smithi078.stdout: "st_size": 6291456, 2023-10-11T09:49:29.037 INFO:teuthology.orchestra.run.smithi078.stdout: "st_atime": 1697017723.0, 2023-10-11T09:49:29.037 INFO:teuthology.orchestra.run.smithi078.stdout: "st_mtime": 1697017723.0, 2023-10-11T09:49:29.038 INFO:teuthology.orchestra.run.smithi078.stdout: "st_ctime": 1697017723.0 2023-10-11T09:49:29.038 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:49:29.220 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-11T09:49:29.220 DEBUG:teuthology.orchestra.run.smithi078:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-11T09:49:29.307 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-11T09:49:29.349 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-11T09:49:29.350 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:49:29.350 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:49:29.375 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:49:29.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.587+0000 7f6e3a445700 1 Processor -- start 2023-10-11T09:49:29.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.591+0000 7f6e3a445700 1 -- start start 2023-10-11T09:49:29.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.591+0000 7f6e3a445700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6e340cc820 0x7f6e34160c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:29.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.591+0000 7f6e3a445700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f6e340b06a0 0x7f6e340b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:29.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.591+0000 7f6e3a445700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e34160210 con 0x7f6e3415d050 2023-10-11T09:49:29.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.591+0000 7f6e3a445700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e341611a0 con 0x7f6e340b06a0 2023-10-11T09:49:29.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.591+0000 7f6e3a445700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e340b1000 con 0x7f6e340cc820 2023-10-11T09:49:29.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.591+0000 7f6e39c44700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f6e340b06a0 0x7f6e340b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:29.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.591+0000 7f6e39c44700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f6e340b06a0 0x7f6e340b0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39714/0 (socket says 172.21.15.78:39714) 2023-10-11T09:49:29.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.591+0000 7f6e39c44700 1 -- 172.21.15.78:0/157682258 learned_addr learned my addr 172.21.15.78:0/157682258 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:29.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.591+0000 7f6e38c42700 1 --2- 172.21.15.78:0/157682258 >> v2:172.21.15.133:3300/0 conn(0x7f6e340cc820 0x7f6e34160c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:29.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.591+0000 7f6e39c44700 1 -- 172.21.15.78:0/157682258 >> v2:172.21.15.133:3300/0 conn(0x7f6e340cc820 msgr2=0x7f6e34160c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:29.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.591+0000 7f6e39c44700 1 --2- 172.21.15.78:0/157682258 >> v2:172.21.15.133:3300/0 conn(0x7f6e340cc820 0x7f6e34160c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:29.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.591+0000 7f6e39c44700 1 -- 172.21.15.78:0/157682258 >> v1:172.21.15.133:6789/0 conn(0x7f6e3415d050 legacy=0x7f6e3415d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:29.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.591+0000 7f6e39c44700 1 -- 172.21.15.78:0/157682258 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e340ccfc0 con 0x7f6e340b06a0 2023-10-11T09:49:29.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.595+0000 7f6e39c44700 1 --2- 172.21.15.78:0/157682258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e340b06a0 0x7f6e340b0ac0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f6e3406e260 tx=0x7f6e2c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4dc3cb5fabe7be9 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:29.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.595+0000 7f6e33fff700 1 -- 172.21.15.78:0/157682258 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c011830 con 0x7f6e340b06a0 2023-10-11T09:49:29.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.595+0000 7f6e33fff700 1 -- 172.21.15.78:0/157682258 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e2c011990 con 0x7f6e340b06a0 2023-10-11T09:49:29.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.595+0000 7f6e33fff700 1 -- 172.21.15.78:0/157682258 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c011d20 con 0x7f6e340b06a0 2023-10-11T09:49:29.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.595+0000 7f6e3a445700 1 -- 172.21.15.78:0/157682258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e340b06a0 msgr2=0x7f6e340b0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:29.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.595+0000 7f6e3a445700 1 --2- 172.21.15.78:0/157682258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e340b06a0 0x7f6e340b0ac0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f6e3406e260 tx=0x7f6e2c009bf0 comp rx=0 tx=0).stop 2023-10-11T09:49:29.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.595+0000 7f6e3a445700 1 -- 172.21.15.78:0/157682258 shutdown_connections 2023-10-11T09:49:29.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.595+0000 7f6e3a445700 1 --2- 172.21.15.78:0/157682258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e340b06a0 0x7f6e340b0ac0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:29.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.595+0000 7f6e3a445700 1 --2- 172.21.15.78:0/157682258 >> v2:172.21.15.133:3300/0 conn(0x7f6e340cc820 0x7f6e34160c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:29.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.595+0000 7f6e3a445700 1 -- 172.21.15.78:0/157682258 >> 172.21.15.78:0/157682258 conn(0x7f6e340cea60 msgr2=0x7f6e340cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:29.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.595+0000 7f6e3a445700 1 -- 172.21.15.78:0/157682258 shutdown_connections 2023-10-11T09:49:29.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.595+0000 7f6e3a445700 1 -- 172.21.15.78:0/157682258 wait complete. 2023-10-11T09:49:29.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.599+0000 7f6e3a445700 1 Processor -- start 2023-10-11T09:49:29.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.599+0000 7f6e3a445700 1 -- start start 2023-10-11T09:49:29.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.599+0000 7f6e3a445700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e340b06a0 0x7f6e341525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:29.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.599+0000 7f6e3a445700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e340cc820 0x7f6e34070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:29.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.599+0000 7f6e3a445700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e3415d050 0x7f6e34071330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:29.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.599+0000 7f6e3a445700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e340b1ab0 con 0x7f6e3415d050 2023-10-11T09:49:29.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.599+0000 7f6e3a445700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3415f6e0 con 0x7f6e340cc820 2023-10-11T09:49:29.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.599+0000 7f6e3a445700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3415f9c0 con 0x7f6e340b06a0 2023-10-11T09:49:29.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.599+0000 7f6e39c44700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e3415d050 0x7f6e34071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:29.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.599+0000 7f6e39c44700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e3415d050 0x7f6e34071330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39728/0 (socket says 172.21.15.78:39728) 2023-10-11T09:49:29.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.599+0000 7f6e39c44700 1 -- 172.21.15.78:0/2834113053 learned_addr learned my addr 172.21.15.78:0/2834113053 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:29.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.599+0000 7f6e38c42700 1 --2- 172.21.15.78:0/2834113053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e340cc820 0x7f6e34070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:29.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.599+0000 7f6e39443700 1 --2- 172.21.15.78:0/2834113053 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e340b06a0 0x7f6e341525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:29.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.599+0000 7f6e39c44700 1 -- 172.21.15.78:0/2834113053 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e340b06a0 msgr2=0x7f6e341525a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:29.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.599+0000 7f6e39c44700 1 --2- 172.21.15.78:0/2834113053 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e340b06a0 0x7f6e341525a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:29.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.603+0000 7f6e39c44700 1 -- 172.21.15.78:0/2834113053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e340cc820 msgr2=0x7f6e34070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:29.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.603+0000 7f6e39c44700 1 --2- 172.21.15.78:0/2834113053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e340cc820 0x7f6e34070df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:29.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.603+0000 7f6e39c44700 1 -- 172.21.15.78:0/2834113053 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e2c009570 con 0x7f6e3415d050 2023-10-11T09:49:29.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.603+0000 7f6e39c44700 1 --2- 172.21.15.78:0/2834113053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e3415d050 0x7f6e34071330 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c014040 tx=0x7f6e2c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:29.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.603+0000 7f6e31ffb700 1 -- 172.21.15.78:0/2834113053 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c004450 con 0x7f6e3415d050 2023-10-11T09:49:29.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.603+0000 7f6e31ffb700 1 -- 172.21.15.78:0/2834113053 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e2c01f070 con 0x7f6e3415d050 2023-10-11T09:49:29.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.603+0000 7f6e3a445700 1 -- 172.21.15.78:0/2834113053 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e340b1d50 con 0x7f6e3415d050 2023-10-11T09:49:29.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.603+0000 7f6e31ffb700 1 -- 172.21.15.78:0/2834113053 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c003900 con 0x7f6e3415d050 2023-10-11T09:49:29.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.603+0000 7f6e3a445700 1 -- 172.21.15.78:0/2834113053 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e34152ae0 con 0x7f6e3415d050 2023-10-11T09:49:29.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.603+0000 7f6e3a445700 1 -- 172.21.15.78:0/2834113053 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e34065d40 con 0x7f6e3415d050 2023-10-11T09:49:29.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.607+0000 7f6e31ffb700 1 -- 172.21.15.78:0/2834113053 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f6e2c003a60 con 0x7f6e3415d050 2023-10-11T09:49:29.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.607+0000 7f6e31ffb700 1 --2- 172.21.15.78:0/2834113053 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6e18065ed0 0x7f6e18068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:29.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.607+0000 7f6e31ffb700 1 -- 172.21.15.78:0/2834113053 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f6e2c021070 con 0x7f6e3415d050 2023-10-11T09:49:29.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.607+0000 7f6e39443700 1 --2- 172.21.15.78:0/2834113053 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6e18065ed0 0x7f6e18068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:29.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.611+0000 7f6e39443700 1 --2- 172.21.15.78:0/2834113053 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6e18065ed0 0x7f6e18068380 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6e2800f290 tx=0x7f6e28009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:29.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.615+0000 7f6e31ffb700 1 -- 172.21.15.78:0/2834113053 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f6e2c0523c0 con 0x7f6e3415d050 2023-10-11T09:49:29.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.787+0000 7f6e3a445700 1 -- 172.21.15.78:0/2834113053 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e340b4a40 con 0x7f6e3415d050 2023-10-11T09:49:29.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.791+0000 7f6e31ffb700 1 -- 172.21.15.78:0/2834113053 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+20058 (secure 0 0 0) 0x7f6e2c015090 con 0x7f6e3415d050 2023-10-11T09:49:29.798 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:49:29.798 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":92,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:48:50.804275+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:48:18.491556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-11T09:48:19.018937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data_extra","create_time":"2023-10-11T09:48:28.963956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6834/1560261029":"2023-10-12T09:48:50.804239+0000","172.21.15.78:6836/1560261029":"2023-10-12T09:48:50.804239+0000","172.21.15.78:6836/3463904224":"2023-10-12T09:48:45.480946+0000","172.21.15.78:6834/3463904224":"2023-10-12T09:48:45.480946+0000","172.21.15.78:6837/951312831":"2023-10-12T09:48:44.433517+0000","172.21.15.78:6835/951312831":"2023-10-12T09:48:44.433517+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:49:29.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.795+0000 7f6e3a445700 1 -- 172.21.15.78:0/2834113053 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6e18065ed0 msgr2=0x7f6e18068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:29.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.795+0000 7f6e3a445700 1 --2- 172.21.15.78:0/2834113053 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6e18065ed0 0x7f6e18068380 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6e2800f290 tx=0x7f6e28009040 comp rx=0 tx=0).stop 2023-10-11T09:49:29.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.795+0000 7f6e3a445700 1 -- 172.21.15.78:0/2834113053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e3415d050 msgr2=0x7f6e34071330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:29.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.795+0000 7f6e3a445700 1 --2- 172.21.15.78:0/2834113053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e3415d050 0x7f6e34071330 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c014040 tx=0x7f6e2c004770 comp rx=0 tx=0).stop 2023-10-11T09:49:29.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.795+0000 7f6e3a445700 1 -- 172.21.15.78:0/2834113053 shutdown_connections 2023-10-11T09:49:29.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.795+0000 7f6e3a445700 1 --2- 172.21.15.78:0/2834113053 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6e18065ed0 0x7f6e18068380 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:29.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.795+0000 7f6e3a445700 1 --2- 172.21.15.78:0/2834113053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e3415d050 0x7f6e34071330 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:29.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.795+0000 7f6e3a445700 1 --2- 172.21.15.78:0/2834113053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e340cc820 0x7f6e34070df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:29.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.795+0000 7f6e3a445700 1 --2- 172.21.15.78:0/2834113053 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e340b06a0 0x7f6e341525a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:29.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.795+0000 7f6e3a445700 1 -- 172.21.15.78:0/2834113053 >> 172.21.15.78:0/2834113053 conn(0x7f6e340cea60 msgr2=0x7f6e340cbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:29.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.795+0000 7f6e3a445700 1 -- 172.21.15.78:0/2834113053 shutdown_connections 2023-10-11T09:49:29.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.795+0000 7f6e3a445700 1 -- 172.21.15.78:0/2834113053 wait complete. 2023-10-11T09:49:29.824 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T09:48:18.491556+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T09:48:19.018937+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-10-11T09:48:28.963956+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:49:29.825 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:49:30.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.995+0000 7fe8220ca700 1 Processor -- start 2023-10-11T09:49:30.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.995+0000 7fe8220ca700 1 -- start start 2023-10-11T09:49:30.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.995+0000 7fe8220ca700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe81c161d90 0x7fe81c0b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:30.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.995+0000 7fe8220ca700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe81c06ccb0 con 0x7fe81c146fe0 2023-10-11T09:49:30.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.999+0000 7fe8220ca700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe81c06c410 con 0x7fe81c0b3520 2023-10-11T09:49:30.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.999+0000 7fe8220ca700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c06c6f0 con 0x7fe81c161d90 2023-10-11T09:49:30.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.999+0000 7fe8210c8700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fe81c146fe0 0x7fe81c06bbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:56544/0 (socket says 172.21.15.78:56544) 2023-10-11T09:49:30.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.999+0000 7fe8210c8700 1 -- 172.21.15.78:0/2149999497 learned_addr learned my addr 172.21.15.78:0/2149999497 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:49:30.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.999+0000 7fe8208c7700 1 --2- 172.21.15.78:0/2149999497 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe81c161d90 0x7fe81c0b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:30.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.999+0000 7fe8208c7700 1 -- 172.21.15.78:0/2149999497 >> v1:172.21.15.133:6789/0 conn(0x7fe81c0b3520 legacy=0x7fe81c1629e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:30.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.999+0000 7fe8208c7700 1 -- 172.21.15.78:0/2149999497 >> v1:172.21.15.78:6789/0 conn(0x7fe81c146fe0 legacy=0x7fe81c06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:30.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.999+0000 7fe8208c7700 1 -- 172.21.15.78:0/2149999497 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe81c06c9d0 con 0x7fe81c161d90 2023-10-11T09:49:30.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.999+0000 7fe813fff700 1 -- 172.21.15.78:0/2149999497 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 793570921 0 0) 0x7fe818001040 con 0x7fe81c146fe0 2023-10-11T09:49:30.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.999+0000 7fe8208c7700 1 --2- 172.21.15.78:0/2149999497 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe81c161d90 0x7fe81c0b2fe0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe80c009800 tx=0x7fe80c009b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db88d8a5fc6f03a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:30.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.999+0000 7fe813fff700 1 -- 172.21.15.78:0/2149999497 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80c004230 con 0x7fe81c161d90 2023-10-11T09:49:30.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.999+0000 7fe813fff700 1 -- 172.21.15.78:0/2149999497 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe80c004390 con 0x7fe81c161d90 2023-10-11T09:49:30.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.999+0000 7fe813fff700 1 -- 172.21.15.78:0/2149999497 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80c0036c0 con 0x7fe81c161d90 2023-10-11T09:49:30.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.999+0000 7fe8220ca700 1 -- 172.21.15.78:0/2149999497 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe81c161d90 msgr2=0x7fe81c0b2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:30.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:29.999+0000 7fe8220ca700 1 --2- 172.21.15.78:0/2149999497 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe81c161d90 0x7fe81c0b2fe0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe80c009800 tx=0x7fe80c009b10 comp rx=0 tx=0).stop 2023-10-11T09:49:30.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.003+0000 7fe8220ca700 1 -- 172.21.15.78:0/2149999497 shutdown_connections 2023-10-11T09:49:30.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.003+0000 7fe8220ca700 1 --2- 172.21.15.78:0/2149999497 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe81c161d90 0x7fe81c0b2fe0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:30.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.003+0000 7fe8220ca700 1 -- 172.21.15.78:0/2149999497 >> 172.21.15.78:0/2149999497 conn(0x7fe81c06a060 msgr2=0x7fe81c06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:30.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.003+0000 7fe8220ca700 1 -- 172.21.15.78:0/2149999497 shutdown_connections 2023-10-11T09:49:30.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.003+0000 7fe8220ca700 1 -- 172.21.15.78:0/2149999497 wait complete. 2023-10-11T09:49:30.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.003+0000 7fe8220ca700 1 Processor -- start 2023-10-11T09:49:30.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.007+0000 7fe8220ca700 1 -- start start 2023-10-11T09:49:30.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.007+0000 7fe8220ca700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe81c0b3520 0x7fe81c0adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:30.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.007+0000 7fe8220ca700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe81c146fe0 0x7fe81c0cd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:30.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.007+0000 7fe8220ca700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe81c161d90 0x7fe81c0cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:30.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.007+0000 7fe8220ca700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c1f6360 con 0x7fe81c0b3520 2023-10-11T09:49:30.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.007+0000 7fe8220ca700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c15c080 con 0x7fe81c146fe0 2023-10-11T09:49:30.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.007+0000 7fe8220ca700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c0ae210 con 0x7fe81c161d90 2023-10-11T09:49:30.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.007+0000 7fe8210c8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe81c0b3520 0x7fe81c0adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:30.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.007+0000 7fe8210c8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe81c0b3520 0x7fe81c0adcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39738/0 (socket says 172.21.15.78:39738) 2023-10-11T09:49:30.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.007+0000 7fe8210c8700 1 -- 172.21.15.78:0/2970030891 learned_addr learned my addr 172.21.15.78:0/2970030891 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:30.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.007+0000 7fe8210c8700 1 -- 172.21.15.78:0/2970030891 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe81c161d90 msgr2=0x7fe81c0cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:30.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.007+0000 7fe8208c7700 1 --2- 172.21.15.78:0/2970030891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe81c146fe0 0x7fe81c0cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:30.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.007+0000 7fe8218c9700 1 --2- 172.21.15.78:0/2970030891 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe81c161d90 0x7fe81c0cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:30.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.007+0000 7fe8210c8700 1 --2- 172.21.15.78:0/2970030891 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe81c161d90 0x7fe81c0cd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:30.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.007+0000 7fe8210c8700 1 -- 172.21.15.78:0/2970030891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe81c146fe0 msgr2=0x7fe81c0cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:30.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.007+0000 7fe8210c8700 1 --2- 172.21.15.78:0/2970030891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe81c146fe0 0x7fe81c0cd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:30.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.007+0000 7fe8210c8700 1 -- 172.21.15.78:0/2970030891 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe80c0094b0 con 0x7fe81c0b3520 2023-10-11T09:49:30.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.007+0000 7fe8218c9700 1 --2- 172.21.15.78:0/2970030891 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe81c161d90 0x7fe81c0cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:49:30.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.007+0000 7fe8208c7700 1 --2- 172.21.15.78:0/2970030891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe81c146fe0 0x7fe81c0cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:49:30.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.007+0000 7fe8210c8700 1 --2- 172.21.15.78:0/2970030891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe81c0b3520 0x7fe81c0adcd0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fe81800c550 tx=0x7fe81800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:30.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.007+0000 7fe811ffb700 1 -- 172.21.15.78:0/2970030891 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe818004ca0 con 0x7fe81c0b3520 2023-10-11T09:49:30.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.007+0000 7fe811ffb700 1 -- 172.21.15.78:0/2970030891 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe818004e00 con 0x7fe81c0b3520 2023-10-11T09:49:30.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.007+0000 7fe8220ca700 1 -- 172.21.15.78:0/2970030891 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe81c15b500 con 0x7fe81c0b3520 2023-10-11T09:49:30.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.007+0000 7fe811ffb700 1 -- 172.21.15.78:0/2970030891 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8180072d0 con 0x7fe81c0b3520 2023-10-11T09:49:30.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.007+0000 7fe8220ca700 1 -- 172.21.15.78:0/2970030891 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe81c15bac0 con 0x7fe81c0b3520 2023-10-11T09:49:30.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.011+0000 7fe8220ca700 1 -- 172.21.15.78:0/2970030891 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe81c065d40 con 0x7fe81c0b3520 2023-10-11T09:49:30.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.011+0000 7fe811ffb700 1 -- 172.21.15.78:0/2970030891 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fe818007430 con 0x7fe81c0b3520 2023-10-11T09:49:30.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.011+0000 7fe811ffb700 1 --2- 172.21.15.78:0/2970030891 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe800065f00 0x7fe8000683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:30.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.011+0000 7fe811ffb700 1 -- 172.21.15.78:0/2970030891 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6724+0+0 (secure 0 0 0) 0x7fe818083620 con 0x7fe81c0b3520 2023-10-11T09:49:30.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.011+0000 7fe8208c7700 1 --2- 172.21.15.78:0/2970030891 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe800065f00 0x7fe8000683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:30.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.015+0000 7fe8208c7700 1 --2- 172.21.15.78:0/2970030891 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe800065f00 0x7fe8000683b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe80c009800 tx=0x7fe80c013040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:30.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.015+0000 7fe811ffb700 1 -- 172.21.15.78:0/2970030891 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fe8180509e0 con 0x7fe81c0b3520 2023-10-11T09:49:30.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.191+0000 7fe8220ca700 1 -- 172.21.15.78:0/2970030891 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe81c06b570 con 0x7fe81c0b3520 2023-10-11T09:49:30.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.191+0000 7fe811ffb700 1 -- 172.21.15.78:0/2970030891 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe818054000 con 0x7fe81c0b3520 2023-10-11T09:49:30.198 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:49:30.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.195+0000 7fe8220ca700 1 -- 172.21.15.78:0/2970030891 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe800065f00 msgr2=0x7fe8000683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:30.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.195+0000 7fe8220ca700 1 --2- 172.21.15.78:0/2970030891 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe800065f00 0x7fe8000683b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe80c009800 tx=0x7fe80c013040 comp rx=0 tx=0).stop 2023-10-11T09:49:30.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.195+0000 7fe8220ca700 1 -- 172.21.15.78:0/2970030891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe81c0b3520 msgr2=0x7fe81c0adcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:30.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.195+0000 7fe8220ca700 1 --2- 172.21.15.78:0/2970030891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe81c0b3520 0x7fe81c0adcd0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fe81800c550 tx=0x7fe81800c910 comp rx=0 tx=0).stop 2023-10-11T09:49:30.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.195+0000 7fe8220ca700 1 -- 172.21.15.78:0/2970030891 shutdown_connections 2023-10-11T09:49:30.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.195+0000 7fe8220ca700 1 --2- 172.21.15.78:0/2970030891 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe800065f00 0x7fe8000683b0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:30.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.195+0000 7fe8220ca700 1 --2- 172.21.15.78:0/2970030891 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe81c161d90 0x7fe81c0cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:30.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.195+0000 7fe8220ca700 1 --2- 172.21.15.78:0/2970030891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe81c146fe0 0x7fe81c0cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:30.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.195+0000 7fe8220ca700 1 --2- 172.21.15.78:0/2970030891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe81c0b3520 0x7fe81c0adcd0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:30.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.195+0000 7fe8220ca700 1 -- 172.21.15.78:0/2970030891 >> 172.21.15.78:0/2970030891 conn(0x7fe81c06a060 msgr2=0x7fe81c06bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:30.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.199+0000 7fe8220ca700 1 -- 172.21.15.78:0/2970030891 shutdown_connections 2023-10-11T09:49:30.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.199+0000 7fe8220ca700 1 -- 172.21.15.78:0/2970030891 wait complete. 2023-10-11T09:49:30.224 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:49:30.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.391+0000 7f5936313700 1 Processor -- start 2023-10-11T09:49:30.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.391+0000 7f5936313700 1 -- start start 2023-10-11T09:49:30.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.391+0000 7f5936313700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f593006d340 0x7f59300ca880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:30.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.391+0000 7f5936313700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f593015e280 con 0x7f5930158430 2023-10-11T09:49:30.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.391+0000 7f5936313700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f593015e560 con 0x7f59300cadc0 2023-10-11T09:49:30.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.391+0000 7f5936313700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5930070c10 con 0x7f593006d340 2023-10-11T09:49:30.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.391+0000 7f592ffff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f5930158430 0x7f59300710d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:56554/0 (socket says 172.21.15.78:56554) 2023-10-11T09:49:30.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.391+0000 7f592ffff700 1 -- 172.21.15.78:0/2361272125 learned_addr learned my addr 172.21.15.78:0/2361272125 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:49:30.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.391+0000 7f592f7fe700 1 --2- 172.21.15.78:0/2361272125 >> v2:172.21.15.78:3300/0 conn(0x7f593006d340 0x7f59300ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:30.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.391+0000 7f592effd700 1 -- 172.21.15.78:0/2361272125 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2184280157 0 0) 0x7f593015e280 con 0x7f5930158430 2023-10-11T09:49:30.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.391+0000 7f592effd700 1 -- 172.21.15.78:0/2361272125 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59180035d0 con 0x7f5930158430 2023-10-11T09:49:30.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.391+0000 7f592effd700 1 -- 172.21.15.78:0/2361272125 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1788908734 0 0) 0x7f593015e560 con 0x7f59300cadc0 2023-10-11T09:49:30.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.391+0000 7f592effd700 1 -- 172.21.15.78:0/2361272125 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f593015e280 con 0x7f59300cadc0 2023-10-11T09:49:30.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.391+0000 7f592f7fe700 1 -- 172.21.15.78:0/2361272125 >> v1:172.21.15.133:6789/0 conn(0x7f59300cadc0 legacy=0x7f593015afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:30.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.391+0000 7f592f7fe700 1 -- 172.21.15.78:0/2361272125 >> v1:172.21.15.78:6789/0 conn(0x7f5930158430 legacy=0x7f59300710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:30.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.391+0000 7f592effd700 1 -- 172.21.15.78:0/2361272125 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3071324869 0 0) 0x7f59180035d0 con 0x7f5930158430 2023-10-11T09:49:30.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.395+0000 7f592f7fe700 1 -- 172.21.15.78:0/2361272125 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f593015e840 con 0x7f593006d340 2023-10-11T09:49:30.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.395+0000 7f592f7fe700 1 --2- 172.21.15.78:0/2361272125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f593006d340 0x7f59300ca880 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f5924009850 tx=0x7f5924009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ac1ff7981fab95 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:30.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.395+0000 7f592effd700 1 -- 172.21.15.78:0/2361272125 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5920001670 con 0x7f593006d340 2023-10-11T09:49:30.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.395+0000 7f592effd700 1 -- 172.21.15.78:0/2361272125 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5924004030 con 0x7f593006d340 2023-10-11T09:49:30.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.395+0000 7f592effd700 1 -- 172.21.15.78:0/2361272125 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59240043a0 con 0x7f593006d340 2023-10-11T09:49:30.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.395+0000 7f5936313700 1 -- 172.21.15.78:0/2361272125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f593006d340 msgr2=0x7f59300ca880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:30.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.395+0000 7f5936313700 1 --2- 172.21.15.78:0/2361272125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f593006d340 0x7f59300ca880 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f5924009850 tx=0x7f5924009b60 comp rx=0 tx=0).stop 2023-10-11T09:49:30.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.395+0000 7f5936313700 1 -- 172.21.15.78:0/2361272125 shutdown_connections 2023-10-11T09:49:30.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.395+0000 7f5936313700 1 --2- 172.21.15.78:0/2361272125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f593006d340 0x7f59300ca880 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:30.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.395+0000 7f5936313700 1 -- 172.21.15.78:0/2361272125 >> 172.21.15.78:0/2361272125 conn(0x7f59300cecc0 msgr2=0x7f59300cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:30.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.395+0000 7f5936313700 1 -- 172.21.15.78:0/2361272125 shutdown_connections 2023-10-11T09:49:30.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.395+0000 7f5936313700 1 -- 172.21.15.78:0/2361272125 wait complete. 2023-10-11T09:49:30.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.399+0000 7f5936313700 1 Processor -- start 2023-10-11T09:49:30.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.399+0000 7f5936313700 1 -- start start 2023-10-11T09:49:30.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.399+0000 7f5936313700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f593006d340 0x7f5930155d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:30.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.399+0000 7f5936313700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59300cadc0 0x7f5930147500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:30.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.399+0000 7f5936313700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5930158430 0x7f5930147a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:30.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.399+0000 7f5936313700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59301565d0 con 0x7f5930158430 2023-10-11T09:49:30.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.399+0000 7f5936313700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59301568b0 con 0x7f593006d340 2023-10-11T09:49:30.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.399+0000 7f5936313700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5930156b90 con 0x7f59300cadc0 2023-10-11T09:49:30.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.399+0000 7f59348b0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5930158430 0x7f5930147a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:30.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.403+0000 7f59348b0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5930158430 0x7f5930147a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39758/0 (socket says 172.21.15.78:39758) 2023-10-11T09:49:30.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.403+0000 7f59348b0700 1 -- 172.21.15.78:0/1608632429 learned_addr learned my addr 172.21.15.78:0/1608632429 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:30.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.403+0000 7f592ffff700 1 --2- 172.21.15.78:0/1608632429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f593006d340 0x7f5930155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:30.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.403+0000 7f592f7fe700 1 --2- 172.21.15.78:0/1608632429 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59300cadc0 0x7f5930147500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:30.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.403+0000 7f59348b0700 1 -- 172.21.15.78:0/1608632429 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59300cadc0 msgr2=0x7f5930147500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:30.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.403+0000 7f59348b0700 1 --2- 172.21.15.78:0/1608632429 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59300cadc0 0x7f5930147500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:30.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.403+0000 7f59348b0700 1 -- 172.21.15.78:0/1608632429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f593006d340 msgr2=0x7f5930155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:30.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.403+0000 7f59348b0700 1 --2- 172.21.15.78:0/1608632429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f593006d340 0x7f5930155d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:30.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.403+0000 7f59348b0700 1 -- 172.21.15.78:0/1608632429 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59240094b0 con 0x7f5930158430 2023-10-11T09:49:30.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.403+0000 7f59348b0700 1 --2- 172.21.15.78:0/1608632429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5930158430 0x7f5930147a40 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f592800cc20 tx=0x7f592800cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:30.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.403+0000 7f592cff9700 1 -- 172.21.15.78:0/1608632429 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5928003c20 con 0x7f5930158430 2023-10-11T09:49:30.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.403+0000 7f592cff9700 1 -- 172.21.15.78:0/1608632429 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5928003d80 con 0x7f5930158430 2023-10-11T09:49:30.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.403+0000 7f592cff9700 1 -- 172.21.15.78:0/1608632429 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592800aae0 con 0x7f5930158430 2023-10-11T09:49:30.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.403+0000 7f5936313700 1 -- 172.21.15.78:0/1608632429 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5930156e70 con 0x7f5930158430 2023-10-11T09:49:30.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.403+0000 7f5936313700 1 -- 172.21.15.78:0/1608632429 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5930156290 con 0x7f5930158430 2023-10-11T09:49:30.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.403+0000 7f5936313700 1 -- 172.21.15.78:0/1608632429 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5930157390 con 0x7f5930158430 2023-10-11T09:49:30.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.407+0000 7f592cff9700 1 -- 172.21.15.78:0/1608632429 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f592800ac40 con 0x7f5930158430 2023-10-11T09:49:30.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.407+0000 7f592cff9700 1 --2- 172.21.15.78:0/1608632429 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f591c065ed0 0x7f591c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:30.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.407+0000 7f592cff9700 1 -- 172.21.15.78:0/1608632429 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f5928083aa0 con 0x7f5930158430 2023-10-11T09:49:30.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.407+0000 7f592ffff700 1 --2- 172.21.15.78:0/1608632429 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f591c065ed0 0x7f591c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:30.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.407+0000 7f592ffff700 1 --2- 172.21.15.78:0/1608632429 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f591c065ed0 0x7f591c068380 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5920001380 tx=0x7f5920007ba0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:30.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.415+0000 7f592cff9700 1 -- 172.21.15.78:0/1608632429 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f5930157390 con 0x7f5930158430 2023-10-11T09:49:30.514 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:49:30.520 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:49:30.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.587+0000 7f5936313700 1 -- 172.21.15.78:0/1608632429 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5930157390 con 0x7f5930158430 2023-10-11T09:49:30.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.587+0000 7f592cff9700 1 -- 172.21.15.78:0/1608632429 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v92) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5930157390 con 0x7f5930158430 2023-10-11T09:49:30.596 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:49:30.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.591+0000 7f5936313700 1 -- 172.21.15.78:0/1608632429 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f591c065ed0 msgr2=0x7f591c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:30.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.591+0000 7f5936313700 1 --2- 172.21.15.78:0/1608632429 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f591c065ed0 0x7f591c068380 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5920001380 tx=0x7f5920007ba0 comp rx=0 tx=0).stop 2023-10-11T09:49:30.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.591+0000 7f5936313700 1 -- 172.21.15.78:0/1608632429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5930158430 msgr2=0x7f5930147a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:30.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.591+0000 7f5936313700 1 --2- 172.21.15.78:0/1608632429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5930158430 0x7f5930147a40 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f592800cc20 tx=0x7f592800cf30 comp rx=0 tx=0).stop 2023-10-11T09:49:30.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.591+0000 7f5936313700 1 -- 172.21.15.78:0/1608632429 shutdown_connections 2023-10-11T09:49:30.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.591+0000 7f5936313700 1 --2- 172.21.15.78:0/1608632429 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f591c065ed0 0x7f591c068380 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:30.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.591+0000 7f5936313700 1 --2- 172.21.15.78:0/1608632429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5930158430 0x7f5930147a40 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:30.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.591+0000 7f5936313700 1 --2- 172.21.15.78:0/1608632429 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59300cadc0 0x7f5930147500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:30.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.591+0000 7f5936313700 1 --2- 172.21.15.78:0/1608632429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f593006d340 0x7f5930155d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:30.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.591+0000 7f5936313700 1 -- 172.21.15.78:0/1608632429 >> 172.21.15.78:0/1608632429 conn(0x7f59300cecc0 msgr2=0x7f5930071090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:30.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.591+0000 7f5936313700 1 -- 172.21.15.78:0/1608632429 shutdown_connections 2023-10-11T09:49:30.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.591+0000 7f5936313700 1 -- 172.21.15.78:0/1608632429 wait complete. 2023-10-11T09:49:30.619 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T09:49:30.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.831+0000 7fae8549f700 1 Processor -- start 2023-10-11T09:49:30.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.831+0000 7fae8549f700 1 -- start start 2023-10-11T09:49:30.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.835+0000 7fae8549f700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fae8006d400 0x7fae8015a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:30.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.835+0000 7fae8549f700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fae800ce240 0x7fae800705b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:30.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.835+0000 7fae8549f700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae80070af0 con 0x7fae80157130 2023-10-11T09:49:30.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.835+0000 7fae8549f700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae800b1aa0 con 0x7fae8006d400 2023-10-11T09:49:30.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.835+0000 7fae8549f700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae800ce6c0 con 0x7fae800ce240 2023-10-11T09:49:30.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.835+0000 7fae777fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fae8006d400 0x7fae8015a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:30.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.835+0000 7fae777fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fae8006d400 0x7fae8015a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39762/0 (socket says 172.21.15.78:39762) 2023-10-11T09:49:30.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.835+0000 7fae777fe700 1 -- 172.21.15.78:0/2149784828 learned_addr learned my addr 172.21.15.78:0/2149784828 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:30.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.835+0000 7fae84c9e700 1 --2- 172.21.15.78:0/2149784828 >> v2:172.21.15.133:3300/0 conn(0x7fae800ce240 0x7fae800705b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:30.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.835+0000 7fae777fe700 1 -- 172.21.15.78:0/2149784828 >> v2:172.21.15.133:3300/0 conn(0x7fae800ce240 msgr2=0x7fae800705b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:30.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.835+0000 7fae777fe700 1 --2- 172.21.15.78:0/2149784828 >> v2:172.21.15.133:3300/0 conn(0x7fae800ce240 0x7fae800705b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:30.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.835+0000 7fae777fe700 1 -- 172.21.15.78:0/2149784828 >> v1:172.21.15.133:6789/0 conn(0x7fae80157130 legacy=0x7fae8006ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:30.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.835+0000 7fae777fe700 1 -- 172.21.15.78:0/2149784828 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae80157500 con 0x7fae8006d400 2023-10-11T09:49:30.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.835+0000 7fae777fe700 1 --2- 172.21.15.78:0/2149784828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae8006d400 0x7fae8015a680 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fae8015e170 tx=0x7fae68009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c174cdfcb4ed68 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:30.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.835+0000 7fae7f7fe700 1 -- 172.21.15.78:0/2149784828 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6800b8a0 con 0x7fae8006d400 2023-10-11T09:49:30.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.835+0000 7fae7f7fe700 1 -- 172.21.15.78:0/2149784828 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae6800ba00 con 0x7fae8006d400 2023-10-11T09:49:30.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.835+0000 7fae7f7fe700 1 -- 172.21.15.78:0/2149784828 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6800bd90 con 0x7fae8006d400 2023-10-11T09:49:30.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.839+0000 7fae8549f700 1 -- 172.21.15.78:0/2149784828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae8006d400 msgr2=0x7fae8015a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:30.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.839+0000 7fae8549f700 1 --2- 172.21.15.78:0/2149784828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae8006d400 0x7fae8015a680 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fae8015e170 tx=0x7fae68009bf0 comp rx=0 tx=0).stop 2023-10-11T09:49:30.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.839+0000 7fae8549f700 1 -- 172.21.15.78:0/2149784828 shutdown_connections 2023-10-11T09:49:30.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.839+0000 7fae8549f700 1 --2- 172.21.15.78:0/2149784828 >> v2:172.21.15.133:3300/0 conn(0x7fae800ce240 0x7fae800705b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:30.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.839+0000 7fae8549f700 1 --2- 172.21.15.78:0/2149784828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae8006d400 0x7fae8015a680 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:30.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.839+0000 7fae8549f700 1 -- 172.21.15.78:0/2149784828 >> 172.21.15.78:0/2149784828 conn(0x7fae8006a060 msgr2=0x7fae8006a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:30.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.839+0000 7fae8549f700 1 -- 172.21.15.78:0/2149784828 shutdown_connections 2023-10-11T09:49:30.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.839+0000 7fae8549f700 1 -- 172.21.15.78:0/2149784828 wait complete. 2023-10-11T09:49:30.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.839+0000 7fae8549f700 1 Processor -- start 2023-10-11T09:49:30.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.839+0000 7fae8549f700 1 -- start start 2023-10-11T09:49:30.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.843+0000 7fae8549f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae8006d400 0x7fae80148a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:30.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.843+0000 7fae8549f700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae800ce240 0x7fae8006c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:30.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.843+0000 7fae8549f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae80157130 0x7fae8006c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:30.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.843+0000 7fae8549f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8013fbc0 con 0x7fae8006d400 2023-10-11T09:49:30.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.843+0000 7fae8549f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae800ad6c0 con 0x7fae80157130 2023-10-11T09:49:30.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.843+0000 7fae8549f700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae80157730 con 0x7fae800ce240 2023-10-11T09:49:30.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.843+0000 7fae7ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae8006d400 0x7fae80148a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:30.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.843+0000 7fae7ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae8006d400 0x7fae80148a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39770/0 (socket says 172.21.15.78:39770) 2023-10-11T09:49:30.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.843+0000 7fae7ffff700 1 -- 172.21.15.78:0/2838322339 learned_addr learned my addr 172.21.15.78:0/2838322339 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:30.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.843+0000 7fae84c9e700 1 --2- 172.21.15.78:0/2838322339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae80157130 0x7fae8006c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:30.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.843+0000 7fae777fe700 1 --2- 172.21.15.78:0/2838322339 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae800ce240 0x7fae8006c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:30.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.843+0000 7fae7ffff700 1 -- 172.21.15.78:0/2838322339 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae800ce240 msgr2=0x7fae8006c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:30.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.843+0000 7fae7ffff700 1 --2- 172.21.15.78:0/2838322339 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae800ce240 0x7fae8006c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:30.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.843+0000 7fae7ffff700 1 -- 172.21.15.78:0/2838322339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae80157130 msgr2=0x7fae8006c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:30.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.843+0000 7fae7ffff700 1 --2- 172.21.15.78:0/2838322339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae80157130 0x7fae8006c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:30.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.843+0000 7fae7ffff700 1 -- 172.21.15.78:0/2838322339 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae68009570 con 0x7fae8006d400 2023-10-11T09:49:30.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.843+0000 7fae84c9e700 1 --2- 172.21.15.78:0/2838322339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae80157130 0x7fae8006c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:49:30.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.843+0000 7fae777fe700 1 --2- 172.21.15.78:0/2838322339 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae800ce240 0x7fae8006c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:49:30.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.843+0000 7fae7ffff700 1 --2- 172.21.15.78:0/2838322339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae8006d400 0x7fae80148a50 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fae7000c500 tx=0x7fae7000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:30.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.843+0000 7fae7d7fa700 1 -- 172.21.15.78:0/2838322339 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae700052d0 con 0x7fae8006d400 2023-10-11T09:49:30.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.843+0000 7fae7d7fa700 1 -- 172.21.15.78:0/2838322339 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae70005430 con 0x7fae8006d400 2023-10-11T09:49:30.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.843+0000 7fae7d7fa700 1 -- 172.21.15.78:0/2838322339 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae700055a0 con 0x7fae8006d400 2023-10-11T09:49:30.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.843+0000 7fae8549f700 1 -- 172.21.15.78:0/2838322339 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae80157a10 con 0x7fae8006d400 2023-10-11T09:49:30.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.843+0000 7fae8549f700 1 -- 172.21.15.78:0/2838322339 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae80157fd0 con 0x7fae8006d400 2023-10-11T09:49:30.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.847+0000 7fae8549f700 1 -- 172.21.15.78:0/2838322339 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae8015c6d0 con 0x7fae8006d400 2023-10-11T09:49:30.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.847+0000 7fae7d7fa700 1 -- 172.21.15.78:0/2838322339 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fae700038e0 con 0x7fae8006d400 2023-10-11T09:49:30.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.847+0000 7fae7d7fa700 1 --2- 172.21.15.78:0/2838322339 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fae60065ed0 0x7fae60068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:30.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.847+0000 7fae7d7fa700 1 -- 172.21.15.78:0/2838322339 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6724+0+0 (secure 0 0 0) 0x7fae70083000 con 0x7fae8006d400 2023-10-11T09:49:30.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.847+0000 7fae777fe700 1 --2- 172.21.15.78:0/2838322339 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fae60065ed0 0x7fae60068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:30.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.851+0000 7fae777fe700 1 --2- 172.21.15.78:0/2838322339 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fae60065ed0 0x7fae60068380 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fae8006d7f0 tx=0x7fae68008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:30.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:30.855+0000 7fae7d7fa700 1 -- 172.21.15.78:0/2838322339 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fae70050330 con 0x7fae8006d400 2023-10-11T09:49:31.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.035+0000 7fae8549f700 1 -- 172.21.15.78:0/2838322339 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fae800cf7d0 con 0x7fae8006d400 2023-10-11T09:49:31.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.035+0000 7fae7d7fa700 1 -- 172.21.15.78:0/2838322339 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7fae70053950 con 0x7fae8006d400 2023-10-11T09:49:31.041 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:49:31.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.035+0000 7fae8549f700 1 -- 172.21.15.78:0/2838322339 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fae60065ed0 msgr2=0x7fae60068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:31.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.035+0000 7fae8549f700 1 --2- 172.21.15.78:0/2838322339 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fae60065ed0 0x7fae60068380 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fae8006d7f0 tx=0x7fae68008030 comp rx=0 tx=0).stop 2023-10-11T09:49:31.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.035+0000 7fae8549f700 1 -- 172.21.15.78:0/2838322339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae8006d400 msgr2=0x7fae80148a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:31.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.035+0000 7fae8549f700 1 --2- 172.21.15.78:0/2838322339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae8006d400 0x7fae80148a50 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fae7000c500 tx=0x7fae7000c8c0 comp rx=0 tx=0).stop 2023-10-11T09:49:31.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.039+0000 7fae8549f700 1 -- 172.21.15.78:0/2838322339 shutdown_connections 2023-10-11T09:49:31.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.039+0000 7fae8549f700 1 --2- 172.21.15.78:0/2838322339 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fae60065ed0 0x7fae60068380 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:31.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.039+0000 7fae8549f700 1 --2- 172.21.15.78:0/2838322339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae80157130 0x7fae8006c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:31.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.039+0000 7fae8549f700 1 --2- 172.21.15.78:0/2838322339 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae800ce240 0x7fae8006c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:31.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.039+0000 7fae8549f700 1 --2- 172.21.15.78:0/2838322339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae8006d400 0x7fae80148a50 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:31.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.039+0000 7fae8549f700 1 -- 172.21.15.78:0/2838322339 >> 172.21.15.78:0/2838322339 conn(0x7fae8006a060 msgr2=0x7fae80159950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:31.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.039+0000 7fae8549f700 1 -- 172.21.15.78:0/2838322339 shutdown_connections 2023-10-11T09:49:31.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.039+0000 7fae8549f700 1 -- 172.21.15.78:0/2838322339 wait complete. 2023-10-11T09:49:31.065 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T09:49:31.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.239+0000 7fabad769700 1 Processor -- start 2023-10-11T09:49:31.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.239+0000 7fabad769700 1 -- start start 2023-10-11T09:49:31.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.243+0000 7fabad769700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7faba80b8890 0x7faba80d2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:31.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.243+0000 7fabad769700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faba80d3510 0x7faba80d3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:31.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.243+0000 7fabad769700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faba80d3e80 con 0x7faba80b84c0 2023-10-11T09:49:31.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.243+0000 7fabad769700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faba80cdea0 con 0x7faba80b8890 2023-10-11T09:49:31.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.243+0000 7fabad769700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faba80d0c10 con 0x7faba80d3510 2023-10-11T09:49:31.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.243+0000 7faba67fc700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7faba80b8890 0x7faba80d2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:31.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.243+0000 7faba67fc700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7faba80b8890 0x7faba80d2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39786/0 (socket says 172.21.15.78:39786) 2023-10-11T09:49:31.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.243+0000 7faba67fc700 1 -- 172.21.15.78:0/4193060318 learned_addr learned my addr 172.21.15.78:0/4193060318 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:31.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.243+0000 7faba6ffd700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7faba80b84c0 0x7faba80cfda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:56558/0 (socket says 172.21.15.78:56558) 2023-10-11T09:49:31.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.243+0000 7faba77fe700 1 --2- 172.21.15.78:0/4193060318 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faba80d3510 0x7faba80d3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:31.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.243+0000 7faba67fc700 1 -- 172.21.15.78:0/4193060318 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faba80d3510 msgr2=0x7faba80d3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:31.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.243+0000 7faba67fc700 1 --2- 172.21.15.78:0/4193060318 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faba80d3510 0x7faba80d3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:31.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.243+0000 7faba67fc700 1 -- 172.21.15.78:0/4193060318 >> v1:172.21.15.78:6789/0 conn(0x7faba80b84c0 legacy=0x7faba80cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:31.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.243+0000 7faba67fc700 1 -- 172.21.15.78:0/4193060318 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba80cd690 con 0x7faba80b8890 2023-10-11T09:49:31.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.243+0000 7faba5ffb700 1 -- 172.21.15.78:0/4193060318 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3988399179 0 0) 0x7faba80d3e80 con 0x7faba80b84c0 2023-10-11T09:49:31.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.243+0000 7faba67fc700 1 --2- 172.21.15.78:0/4193060318 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faba80b8890 0x7faba80d2fd0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fab9c009910 tx=0x7fab9c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9745f6428a7c6e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:31.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.243+0000 7faba5ffb700 1 -- 172.21.15.78:0/4193060318 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c011830 con 0x7faba80b8890 2023-10-11T09:49:31.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.243+0000 7faba5ffb700 1 -- 172.21.15.78:0/4193060318 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9c011990 con 0x7faba80b8890 2023-10-11T09:49:31.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.243+0000 7faba5ffb700 1 -- 172.21.15.78:0/4193060318 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c011d40 con 0x7faba80b8890 2023-10-11T09:49:31.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.243+0000 7fabad769700 1 -- 172.21.15.78:0/4193060318 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faba80b8890 msgr2=0x7faba80d2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:31.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.243+0000 7fabad769700 1 --2- 172.21.15.78:0/4193060318 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faba80b8890 0x7faba80d2fd0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fab9c009910 tx=0x7fab9c009c20 comp rx=0 tx=0).stop 2023-10-11T09:49:31.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.247+0000 7fabad769700 1 -- 172.21.15.78:0/4193060318 shutdown_connections 2023-10-11T09:49:31.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.247+0000 7fabad769700 1 --2- 172.21.15.78:0/4193060318 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faba80d3510 0x7faba80d3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:31.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.247+0000 7fabad769700 1 --2- 172.21.15.78:0/4193060318 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faba80b8890 0x7faba80d2fd0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:31.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.247+0000 7fabad769700 1 -- 172.21.15.78:0/4193060318 >> 172.21.15.78:0/4193060318 conn(0x7faba806cb50 msgr2=0x7faba806cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:31.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.247+0000 7fabad769700 1 -- 172.21.15.78:0/4193060318 shutdown_connections 2023-10-11T09:49:31.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.247+0000 7fabad769700 1 -- 172.21.15.78:0/4193060318 wait complete. 2023-10-11T09:49:31.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.251+0000 7fabad769700 1 Processor -- start 2023-10-11T09:49:31.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.251+0000 7fabad769700 1 -- start start 2023-10-11T09:49:31.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.251+0000 7fabad769700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faba80b84c0 0x7faba8156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:31.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.251+0000 7fabad769700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faba80b8890 0x7faba815bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:31.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.251+0000 7fabad769700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faba80d3510 0x7faba81e69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:31.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.251+0000 7fabad769700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba80c7780 con 0x7faba80b84c0 2023-10-11T09:49:31.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.251+0000 7fabad769700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba80c8b40 con 0x7faba80d3510 2023-10-11T09:49:31.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.251+0000 7fabad769700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faba80c9660 con 0x7faba80b8890 2023-10-11T09:49:31.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.251+0000 7faba6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faba80b84c0 0x7faba8156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:31.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.251+0000 7faba6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faba80b84c0 0x7faba8156ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39792/0 (socket says 172.21.15.78:39792) 2023-10-11T09:49:31.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.251+0000 7faba6ffd700 1 -- 172.21.15.78:0/1166464146 learned_addr learned my addr 172.21.15.78:0/1166464146 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:31.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.251+0000 7faba67fc700 1 --2- 172.21.15.78:0/1166464146 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faba80b8890 0x7faba815bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:31.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.251+0000 7faba6ffd700 1 -- 172.21.15.78:0/1166464146 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faba80b8890 msgr2=0x7faba815bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:31.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.251+0000 7faba6ffd700 1 --2- 172.21.15.78:0/1166464146 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faba80b8890 0x7faba815bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:31.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.251+0000 7faba6ffd700 1 -- 172.21.15.78:0/1166464146 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faba80d3510 msgr2=0x7faba81e69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:31.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.251+0000 7faba6ffd700 1 --2- 172.21.15.78:0/1166464146 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faba80d3510 0x7faba81e69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:31.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.251+0000 7faba6ffd700 1 -- 172.21.15.78:0/1166464146 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab9c009570 con 0x7faba80b84c0 2023-10-11T09:49:31.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.251+0000 7faba67fc700 1 --2- 172.21.15.78:0/1166464146 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faba80b8890 0x7faba815bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:49:31.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.251+0000 7faba6ffd700 1 --2- 172.21.15.78:0/1166464146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faba80b84c0 0x7faba8156ea0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fab9800c510 tx=0x7fab9800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:31.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.251+0000 7fab8f7fe700 1 -- 172.21.15.78:0/1166464146 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab98004ca0 con 0x7faba80b84c0 2023-10-11T09:49:31.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.251+0000 7fab8f7fe700 1 -- 172.21.15.78:0/1166464146 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab98004e20 con 0x7faba80b84c0 2023-10-11T09:49:31.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.251+0000 7fabad769700 1 -- 172.21.15.78:0/1166464146 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba80cce00 con 0x7faba80b84c0 2023-10-11T09:49:31.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.251+0000 7fab8f7fe700 1 -- 172.21.15.78:0/1166464146 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab98007310 con 0x7faba80b84c0 2023-10-11T09:49:31.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.251+0000 7fabad769700 1 -- 172.21.15.78:0/1166464146 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faba80c8d70 con 0x7faba80b84c0 2023-10-11T09:49:31.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.255+0000 7fabad769700 1 -- 172.21.15.78:0/1166464146 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faba81e6f10 con 0x7faba80b84c0 2023-10-11T09:49:31.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.255+0000 7fab8f7fe700 1 -- 172.21.15.78:0/1166464146 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fab98007470 con 0x7faba80b84c0 2023-10-11T09:49:31.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.255+0000 7fab8f7fe700 1 --2- 172.21.15.78:0/1166464146 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fab94065ed0 0x7fab94068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:31.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.255+0000 7fab8f7fe700 1 -- 172.21.15.78:0/1166464146 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6724+0+0 (secure 0 0 0) 0x7fab98083760 con 0x7faba80b84c0 2023-10-11T09:49:31.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.255+0000 7faba67fc700 1 --2- 172.21.15.78:0/1166464146 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fab94065ed0 0x7fab94068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:31.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.259+0000 7faba67fc700 1 --2- 172.21.15.78:0/1166464146 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fab94065ed0 0x7fab94068380 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7faba80c6970 tx=0x7fab9c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:31.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.263+0000 7fab8f7fe700 1 -- 172.21.15.78:0/1166464146 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fab98050b20 con 0x7faba80b84c0 2023-10-11T09:49:31.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.439+0000 7fabad769700 1 -- 172.21.15.78:0/1166464146 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faba808bff0 con 0x7faba80b84c0 2023-10-11T09:49:31.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.439+0000 7fab8f7fe700 1 -- 172.21.15.78:0/1166464146 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7fab98007720 con 0x7faba80b84c0 2023-10-11T09:49:31.445 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2023-10-11T09:49:31.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.443+0000 7fabad769700 1 -- 172.21.15.78:0/1166464146 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fab94065ed0 msgr2=0x7fab94068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:31.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.443+0000 7fabad769700 1 --2- 172.21.15.78:0/1166464146 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fab94065ed0 0x7fab94068380 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7faba80c6970 tx=0x7fab9c008030 comp rx=0 tx=0).stop 2023-10-11T09:49:31.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.443+0000 7fabad769700 1 -- 172.21.15.78:0/1166464146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faba80b84c0 msgr2=0x7faba8156ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:31.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.443+0000 7fabad769700 1 --2- 172.21.15.78:0/1166464146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faba80b84c0 0x7faba8156ea0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fab9800c510 tx=0x7fab9800c8d0 comp rx=0 tx=0).stop 2023-10-11T09:49:31.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.443+0000 7fabad769700 1 -- 172.21.15.78:0/1166464146 shutdown_connections 2023-10-11T09:49:31.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.443+0000 7fabad769700 1 --2- 172.21.15.78:0/1166464146 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fab94065ed0 0x7fab94068380 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:31.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.443+0000 7fabad769700 1 --2- 172.21.15.78:0/1166464146 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faba80d3510 0x7faba81e69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:31.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.443+0000 7fabad769700 1 --2- 172.21.15.78:0/1166464146 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faba80b8890 0x7faba815bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:31.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.443+0000 7fabad769700 1 --2- 172.21.15.78:0/1166464146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faba80b84c0 0x7faba8156ea0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:31.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.443+0000 7fabad769700 1 -- 172.21.15.78:0/1166464146 >> 172.21.15.78:0/1166464146 conn(0x7faba806cb50 msgr2=0x7faba81e7410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:31.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.443+0000 7fabad769700 1 -- 172.21.15.78:0/1166464146 shutdown_connections 2023-10-11T09:49:31.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.443+0000 7fabad769700 1 -- 172.21.15.78:0/1166464146 wait complete. 2023-10-11T09:49:31.470 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-10-11T09:49:31.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.651+0000 7f49470ee700 1 Processor -- start 2023-10-11T09:49:31.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.651+0000 7f49470ee700 1 -- start start 2023-10-11T09:49:31.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.651+0000 7f49470ee700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f49400b84c0 0x7f49400cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:31.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.651+0000 7f49470ee700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f49400b8890 0x7f4940160370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:31.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.651+0000 7f49470ee700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f49401608b0 0x7f4940160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:31.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.651+0000 7f49470ee700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4940161220 con 0x7f49400b84c0 2023-10-11T09:49:31.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.651+0000 7f49470ee700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49400cdea0 con 0x7f49401608b0 2023-10-11T09:49:31.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.651+0000 7f49470ee700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49400d0c10 con 0x7f49400b8890 2023-10-11T09:49:31.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.651+0000 7f4944e8a700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f49400b84c0 0x7f49400cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:31.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.651+0000 7f493ffff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f49400b8890 0x7f4940160370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:31.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.651+0000 7f4944e8a700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f49400b84c0 0x7f49400cfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39802/0 (socket says 172.21.15.78:39802) 2023-10-11T09:49:31.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.651+0000 7f4944e8a700 1 -- 172.21.15.78:0/2616936725 learned_addr learned my addr 172.21.15.78:0/2616936725 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:31.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.651+0000 7f494568b700 1 --2- 172.21.15.78:0/2616936725 >> v2:172.21.15.133:3300/0 conn(0x7f49401608b0 0x7f4940160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:31.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.655+0000 7f4944e8a700 1 -- 172.21.15.78:0/2616936725 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f49400b8890 msgr2=0x7f4940160370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:31.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.655+0000 7f4944e8a700 1 --2- 172.21.15.78:0/2616936725 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f49400b8890 0x7f4940160370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:31.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.655+0000 7f4944e8a700 1 -- 172.21.15.78:0/2616936725 >> v2:172.21.15.133:3300/0 conn(0x7f49401608b0 msgr2=0x7f4940160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:31.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.655+0000 7f4944e8a700 1 --2- 172.21.15.78:0/2616936725 >> v2:172.21.15.133:3300/0 conn(0x7f49401608b0 0x7f4940160ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:31.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.655+0000 7f4944e8a700 1 -- 172.21.15.78:0/2616936725 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49400cd690 con 0x7f49400b84c0 2023-10-11T09:49:31.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.655+0000 7f4944e8a700 1 --2- 172.21.15.78:0/2616936725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49400b84c0 0x7f49400cfda0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f49400c6ab0 tx=0x7f4934009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753867afbf5ec95c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:31.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.655+0000 7f493f7fe700 1 -- 172.21.15.78:0/2616936725 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4934011830 con 0x7f49400b84c0 2023-10-11T09:49:31.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.655+0000 7f493f7fe700 1 -- 172.21.15.78:0/2616936725 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4934011990 con 0x7f49400b84c0 2023-10-11T09:49:31.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.655+0000 7f493f7fe700 1 -- 172.21.15.78:0/2616936725 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4934011d20 con 0x7f49400b84c0 2023-10-11T09:49:31.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.655+0000 7f49470ee700 1 -- 172.21.15.78:0/2616936725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49400b84c0 msgr2=0x7f49400cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:31.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.655+0000 7f49470ee700 1 --2- 172.21.15.78:0/2616936725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49400b84c0 0x7f49400cfda0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f49400c6ab0 tx=0x7f4934009bf0 comp rx=0 tx=0).stop 2023-10-11T09:49:31.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.655+0000 7f49470ee700 1 -- 172.21.15.78:0/2616936725 shutdown_connections 2023-10-11T09:49:31.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.655+0000 7f49470ee700 1 --2- 172.21.15.78:0/2616936725 >> v2:172.21.15.133:3300/0 conn(0x7f49401608b0 0x7f4940160ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:31.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.655+0000 7f49470ee700 1 --2- 172.21.15.78:0/2616936725 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f49400b8890 0x7f4940160370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:31.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.655+0000 7f49470ee700 1 --2- 172.21.15.78:0/2616936725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49400b84c0 0x7f49400cfda0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:31.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.655+0000 7f49470ee700 1 -- 172.21.15.78:0/2616936725 >> 172.21.15.78:0/2616936725 conn(0x7f494006cb50 msgr2=0x7f494006cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:31.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.655+0000 7f49470ee700 1 -- 172.21.15.78:0/2616936725 shutdown_connections 2023-10-11T09:49:31.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.655+0000 7f49470ee700 1 -- 172.21.15.78:0/2616936725 wait complete. 2023-10-11T09:49:31.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.659+0000 7f49470ee700 1 Processor -- start 2023-10-11T09:49:31.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.659+0000 7f49470ee700 1 -- start start 2023-10-11T09:49:31.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.659+0000 7f49470ee700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49400b84c0 0x7f494015a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:31.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.659+0000 7f49470ee700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f49400b8890 0x7f49400cc170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:31.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.659+0000 7f49470ee700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49401608b0 0x7f49400ca110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:31.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.659+0000 7f49470ee700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494015ab90 con 0x7f49400b84c0 2023-10-11T09:49:31.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.659+0000 7f49470ee700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49400b6030 con 0x7f49401608b0 2023-10-11T09:49:31.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.659+0000 7f49470ee700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49400cd480 con 0x7f49400b8890 2023-10-11T09:49:31.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.659+0000 7f4944e8a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49400b84c0 0x7f494015a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:31.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.659+0000 7f4944e8a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49400b84c0 0x7f494015a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39812/0 (socket says 172.21.15.78:39812) 2023-10-11T09:49:31.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.659+0000 7f4944e8a700 1 -- 172.21.15.78:0/1458496934 learned_addr learned my addr 172.21.15.78:0/1458496934 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:31.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.663+0000 7f493ffff700 1 --2- 172.21.15.78:0/1458496934 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f49400b8890 0x7f49400cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:31.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.663+0000 7f494568b700 1 --2- 172.21.15.78:0/1458496934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49401608b0 0x7f49400ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:31.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.663+0000 7f4944e8a700 1 -- 172.21.15.78:0/1458496934 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f49400b8890 msgr2=0x7f49400cc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:31.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.663+0000 7f4944e8a700 1 --2- 172.21.15.78:0/1458496934 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f49400b8890 0x7f49400cc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:31.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.663+0000 7f4944e8a700 1 -- 172.21.15.78:0/1458496934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49401608b0 msgr2=0x7f49400ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:31.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.663+0000 7f4944e8a700 1 --2- 172.21.15.78:0/1458496934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49401608b0 0x7f49400ca110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:31.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.663+0000 7f4944e8a700 1 -- 172.21.15.78:0/1458496934 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4934009570 con 0x7f49400b84c0 2023-10-11T09:49:31.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.663+0000 7f4944e8a700 1 --2- 172.21.15.78:0/1458496934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49400b84c0 0x7f494015a650 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f4940067640 tx=0x7f4934004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:31.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.663+0000 7f493d7fa700 1 -- 172.21.15.78:0/1458496934 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49340044d0 con 0x7f49400b84c0 2023-10-11T09:49:31.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.663+0000 7f493d7fa700 1 -- 172.21.15.78:0/1458496934 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493401f070 con 0x7f49400b84c0 2023-10-11T09:49:31.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.663+0000 7f493d7fa700 1 -- 172.21.15.78:0/1458496934 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4934003900 con 0x7f49400b84c0 2023-10-11T09:49:31.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.663+0000 7f49470ee700 1 -- 172.21.15.78:0/1458496934 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49400ccbe0 con 0x7f49400b84c0 2023-10-11T09:49:31.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.663+0000 7f49470ee700 1 -- 172.21.15.78:0/1458496934 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49400cd1a0 con 0x7f49400b84c0 2023-10-11T09:49:31.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.663+0000 7f49470ee700 1 -- 172.21.15.78:0/1458496934 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f494008bff0 con 0x7f49400b84c0 2023-10-11T09:49:31.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.667+0000 7f493d7fa700 1 -- 172.21.15.78:0/1458496934 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f4934003a60 con 0x7f49400b84c0 2023-10-11T09:49:31.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.667+0000 7f493d7fa700 1 --2- 172.21.15.78:0/1458496934 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4924065ed0 0x7f4924068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:31.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.667+0000 7f493d7fa700 1 -- 172.21.15.78:0/1458496934 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f4934021070 con 0x7f49400b84c0 2023-10-11T09:49:31.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.667+0000 7f493ffff700 1 --2- 172.21.15.78:0/1458496934 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4924065ed0 0x7f4924068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:31.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.667+0000 7f493ffff700 1 --2- 172.21.15.78:0/1458496934 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4924065ed0 0x7f4924068380 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f49400b2620 tx=0x7f492c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:31.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.675+0000 7f493d7fa700 1 -- 172.21.15.78:0/1458496934 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f4934052150 con 0x7f49400b84c0 2023-10-11T09:49:31.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.847+0000 7f49470ee700 1 -- 172.21.15.78:0/1458496934 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7f4940065db0 con 0x7f49400b84c0 2023-10-11T09:49:31.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.847+0000 7f493d7fa700 1 -- 172.21.15.78:0/1458496934 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v92) v1 ==== 106+0+11 (secure 0 0 0) 0x7f4934015020 con 0x7f49400b84c0 2023-10-11T09:49:31.855 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:49:31.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.851+0000 7f49470ee700 1 -- 172.21.15.78:0/1458496934 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4924065ed0 msgr2=0x7f4924068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:31.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.851+0000 7f49470ee700 1 --2- 172.21.15.78:0/1458496934 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4924065ed0 0x7f4924068380 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f49400b2620 tx=0x7f492c008040 comp rx=0 tx=0).stop 2023-10-11T09:49:31.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.851+0000 7f49470ee700 1 -- 172.21.15.78:0/1458496934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49400b84c0 msgr2=0x7f494015a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:31.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.851+0000 7f49470ee700 1 --2- 172.21.15.78:0/1458496934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49400b84c0 0x7f494015a650 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f4940067640 tx=0x7f4934004770 comp rx=0 tx=0).stop 2023-10-11T09:49:31.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.851+0000 7f49470ee700 1 -- 172.21.15.78:0/1458496934 shutdown_connections 2023-10-11T09:49:31.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.851+0000 7f49470ee700 1 --2- 172.21.15.78:0/1458496934 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4924065ed0 0x7f4924068380 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:31.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.851+0000 7f49470ee700 1 --2- 172.21.15.78:0/1458496934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49401608b0 0x7f49400ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:31.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.851+0000 7f49470ee700 1 --2- 172.21.15.78:0/1458496934 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f49400b8890 0x7f49400cc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:31.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.851+0000 7f49470ee700 1 --2- 172.21.15.78:0/1458496934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49400b84c0 0x7f494015a650 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:31.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.851+0000 7f49470ee700 1 -- 172.21.15.78:0/1458496934 >> 172.21.15.78:0/1458496934 conn(0x7f494006cb50 msgr2=0x7f49400b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:31.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.851+0000 7f49470ee700 1 -- 172.21.15.78:0/1458496934 shutdown_connections 2023-10-11T09:49:31.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:31.851+0000 7f49470ee700 1 -- 172.21.15.78:0/1458496934 wait complete. 2023-10-11T09:49:31.880 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T09:49:32.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.047+0000 7f383359e700 1 Processor -- start 2023-10-11T09:49:32.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.047+0000 7f383359e700 1 -- start start 2023-10-11T09:49:32.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.051+0000 7f383359e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f383406dd80 0x7f383406e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:32.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.051+0000 7f383359e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f38340b2800 0x7f38340cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:32.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.051+0000 7f383359e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38340cf6e0 con 0x7f38340d1a40 2023-10-11T09:49:32.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.051+0000 7f383359e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38340ce8c0 con 0x7f383406dd80 2023-10-11T09:49:32.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.051+0000 7f383359e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f383406e690 con 0x7f38340b2800 2023-10-11T09:49:32.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.051+0000 7f3831d9b700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f38340d1a40 0x7f38340cb340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:56562/0 (socket says 172.21.15.78:56562) 2023-10-11T09:49:32.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.051+0000 7f3831d9b700 1 -- 172.21.15.78:0/2200673159 learned_addr learned my addr 172.21.15.78:0/2200673159 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:49:32.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.051+0000 7f383259c700 1 --2- 172.21.15.78:0/2200673159 >> v2:172.21.15.133:3300/0 conn(0x7f383406dd80 0x7f383406e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:32.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.051+0000 7f3832d9d700 1 --2- 172.21.15.78:0/2200673159 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f38340b2800 0x7f38340cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:32.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.051+0000 7f383159a700 1 -- 172.21.15.78:0/2200673159 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2441166681 0 0) 0x7f38340cf6e0 con 0x7f38340d1a40 2023-10-11T09:49:32.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.051+0000 7f383159a700 1 -- 172.21.15.78:0/2200673159 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f381c0035d0 con 0x7f38340d1a40 2023-10-11T09:49:32.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.051+0000 7f383159a700 1 -- 172.21.15.78:0/2200673159 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1381639892 0 0) 0x7f381c0035d0 con 0x7f38340d1a40 2023-10-11T09:49:32.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.051+0000 7f383159a700 1 -- 172.21.15.78:0/2200673159 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f38340b2800 msgr2=0x7f38340cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:32.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.051+0000 7f383159a700 1 --2- 172.21.15.78:0/2200673159 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f38340b2800 0x7f38340cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:32.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.051+0000 7f383159a700 1 -- 172.21.15.78:0/2200673159 >> v2:172.21.15.133:3300/0 conn(0x7f383406dd80 msgr2=0x7f383406e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:32.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.051+0000 7f383159a700 1 --2- 172.21.15.78:0/2200673159 >> v2:172.21.15.133:3300/0 conn(0x7f383406dd80 0x7f383406e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:32.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.051+0000 7f383159a700 1 -- 172.21.15.78:0/2200673159 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38340cead0 con 0x7f38340d1a40 2023-10-11T09:49:32.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.051+0000 7f383159a700 1 -- 172.21.15.78:0/2200673159 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f3828001670 con 0x7f38340d1a40 2023-10-11T09:49:32.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.051+0000 7f383159a700 1 -- 172.21.15.78:0/2200673159 >> v1:172.21.15.78:6789/0 conn(0x7f38340d1a40 legacy=0x7f38340cb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:32.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.051+0000 7f383159a700 1 --2- 172.21.15.78:0/2200673159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f381c003a20 0x7f381c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:32.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.051+0000 7f383159a700 1 --2- 172.21.15.78:0/2200673159 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f38340b2800 0x7f381c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:32.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.051+0000 7f383159a700 1 --2- 172.21.15.78:0/2200673159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f381c004db0 0x7f381c005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:32.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.051+0000 7f383159a700 1 -- 172.21.15.78:0/2200673159 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38340ce8c0 con 0x7f381c004db0 2023-10-11T09:49:32.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.051+0000 7f383159a700 1 -- 172.21.15.78:0/2200673159 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f383406e690 con 0x7f381c003a20 2023-10-11T09:49:32.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.051+0000 7f383159a700 1 -- 172.21.15.78:0/2200673159 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f381c00ba30 con 0x7f38340b2800 2023-10-11T09:49:32.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.055+0000 7f3832d9d700 1 --2- 172.21.15.78:0/2200673159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f381c004db0 0x7f381c005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:32.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.055+0000 7f3832d9d700 1 -- 172.21.15.78:0/2200673159 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f38340b2800 msgr2=0x7f381c004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:32.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.055+0000 7f3832d9d700 1 --2- 172.21.15.78:0/2200673159 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f38340b2800 0x7f381c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:32.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.055+0000 7f3832d9d700 1 -- 172.21.15.78:0/2200673159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f381c003a20 msgr2=0x7f381c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:32.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.055+0000 7f3832d9d700 1 --2- 172.21.15.78:0/2200673159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f381c003a20 0x7f381c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:32.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.055+0000 7f3832d9d700 1 -- 172.21.15.78:0/2200673159 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f381c0035d0 con 0x7f381c004db0 2023-10-11T09:49:32.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.055+0000 7f3832d9d700 1 --2- 172.21.15.78:0/2200673159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f381c004db0 0x7f381c005220 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f382c009f90 tx=0x7f381c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb0baeeb8be6e638 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:32.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.055+0000 7f383159a700 1 -- 172.21.15.78:0/2200673159 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c00ec30 con 0x7f381c004db0 2023-10-11T09:49:32.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.055+0000 7f383159a700 1 -- 172.21.15.78:0/2200673159 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f382c00f040 con 0x7f381c004db0 2023-10-11T09:49:32.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.055+0000 7f383159a700 1 -- 172.21.15.78:0/2200673159 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c014480 con 0x7f381c004db0 2023-10-11T09:49:32.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.055+0000 7f383359e700 1 -- 172.21.15.78:0/2200673159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f381c004db0 msgr2=0x7f381c005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:32.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.055+0000 7f383359e700 1 --2- 172.21.15.78:0/2200673159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f381c004db0 0x7f381c005220 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f382c009f90 tx=0x7f381c0058a0 comp rx=0 tx=0).stop 2023-10-11T09:49:32.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.055+0000 7f3832d9d700 1 -- 172.21.15.78:0/2200673159 reap_dead start 2023-10-11T09:49:32.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.055+0000 7f383359e700 1 -- 172.21.15.78:0/2200673159 shutdown_connections 2023-10-11T09:49:32.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.055+0000 7f383359e700 1 -- 172.21.15.78:0/2200673159 >> 172.21.15.78:0/2200673159 conn(0x7f383409a130 msgr2=0x7f38340d0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:32.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.055+0000 7f383359e700 1 -- 172.21.15.78:0/2200673159 shutdown_connections 2023-10-11T09:49:32.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.055+0000 7f383359e700 1 -- 172.21.15.78:0/2200673159 wait complete. 2023-10-11T09:49:32.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.059+0000 7f383359e700 1 Processor -- start 2023-10-11T09:49:32.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.059+0000 7f383359e700 1 -- start start 2023-10-11T09:49:32.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.059+0000 7f383359e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f381c004db0 0x7f38340c4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:32.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.059+0000 7f383359e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f38340b6700 0x7f383415ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:32.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.059+0000 7f383359e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f383415b1c0 0x7f38340ae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:32.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.059+0000 7f383359e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38340c5d70 con 0x7f381c004db0 2023-10-11T09:49:32.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.059+0000 7f383359e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f383415b630 con 0x7f383415b1c0 2023-10-11T09:49:32.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.059+0000 7f383359e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38340b6b70 con 0x7f38340b6700 2023-10-11T09:49:32.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.059+0000 7f383259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f381c004db0 0x7f38340c4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:32.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.059+0000 7f383259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f381c004db0 0x7f38340c4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39822/0 (socket says 172.21.15.78:39822) 2023-10-11T09:49:32.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.059+0000 7f383259c700 1 -- 172.21.15.78:0/3799413364 learned_addr learned my addr 172.21.15.78:0/3799413364 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:32.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.059+0000 7f3831d9b700 1 --2- 172.21.15.78:0/3799413364 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f38340b6700 0x7f383415ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:32.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.059+0000 7f3832d9d700 1 --2- 172.21.15.78:0/3799413364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f383415b1c0 0x7f38340ae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:32.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.063+0000 7f383259c700 1 -- 172.21.15.78:0/3799413364 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f38340b6700 msgr2=0x7f383415ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:32.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.063+0000 7f383259c700 1 --2- 172.21.15.78:0/3799413364 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f38340b6700 0x7f383415ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:32.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.063+0000 7f383259c700 1 -- 172.21.15.78:0/3799413364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f383415b1c0 msgr2=0x7f38340ae9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:32.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.063+0000 7f383259c700 1 --2- 172.21.15.78:0/3799413364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f383415b1c0 0x7f38340ae9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:32.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.063+0000 7f383259c700 1 -- 172.21.15.78:0/3799413364 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38340c5160 con 0x7f381c004db0 2023-10-11T09:49:32.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.063+0000 7f383259c700 1 --2- 172.21.15.78:0/3799413364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f381c004db0 0x7f38340c4a80 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f382400e810 tx=0x7f382400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:32.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.063+0000 7f3822ffd700 1 -- 172.21.15.78:0/3799413364 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3824013680 con 0x7f381c004db0 2023-10-11T09:49:32.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.063+0000 7f3822ffd700 1 -- 172.21.15.78:0/3799413364 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38240137e0 con 0x7f381c004db0 2023-10-11T09:49:32.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.063+0000 7f383359e700 1 -- 172.21.15.78:0/3799413364 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38340c5440 con 0x7f381c004db0 2023-10-11T09:49:32.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.063+0000 7f3822ffd700 1 -- 172.21.15.78:0/3799413364 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3824013b90 con 0x7f381c004db0 2023-10-11T09:49:32.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.063+0000 7f383359e700 1 -- 172.21.15.78:0/3799413364 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38340c5a00 con 0x7f381c004db0 2023-10-11T09:49:32.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.063+0000 7f383359e700 1 -- 172.21.15.78:0/3799413364 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3834065d40 con 0x7f381c004db0 2023-10-11T09:49:32.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.067+0000 7f3822ffd700 1 -- 172.21.15.78:0/3799413364 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f3824013cf0 con 0x7f381c004db0 2023-10-11T09:49:32.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.067+0000 7f3822ffd700 1 --2- 172.21.15.78:0/3799413364 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3814065ed0 0x7f3814068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:32.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.067+0000 7f3822ffd700 1 -- 172.21.15.78:0/3799413364 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f3824083d70 con 0x7f381c004db0 2023-10-11T09:49:32.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.067+0000 7f3831d9b700 1 --2- 172.21.15.78:0/3799413364 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3814065ed0 0x7f3814068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:32.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.067+0000 7f3831d9b700 1 --2- 172.21.15.78:0/3799413364 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3814065ed0 0x7f3814068380 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3834152c30 tx=0x7f3828001a00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:32.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.071+0000 7f3822ffd700 1 -- 172.21.15.78:0/3799413364 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f3824051080 con 0x7f381c004db0 2023-10-11T09:49:32.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.255+0000 7f383359e700 1 -- 172.21.15.78:0/3799413364 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f38340cd570 con 0x7f381c004db0 2023-10-11T09:49:32.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.259+0000 7f3822ffd700 1 -- 172.21.15.78:0/3799413364 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v92) v1 ==== 80+0+364 (secure 0 0 0) 0x7f3824021080 con 0x7f381c004db0 2023-10-11T09:49:32.265 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6834/1560261029 2023-10-12T09:48:50.804239+0000 2023-10-11T09:49:32.265 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6836/1560261029 2023-10-12T09:48:50.804239+0000 2023-10-11T09:49:32.265 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6836/3463904224 2023-10-12T09:48:45.480946+0000 2023-10-11T09:49:32.265 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6834/3463904224 2023-10-12T09:48:45.480946+0000 2023-10-11T09:49:32.265 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6837/951312831 2023-10-12T09:48:44.433517+0000 2023-10-11T09:49:32.266 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6835/951312831 2023-10-12T09:48:44.433517+0000 2023-10-11T09:49:32.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.259+0000 7f383359e700 1 -- 172.21.15.78:0/3799413364 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3814065ed0 msgr2=0x7f3814068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:32.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.259+0000 7f383359e700 1 --2- 172.21.15.78:0/3799413364 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3814065ed0 0x7f3814068380 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3834152c30 tx=0x7f3828001a00 comp rx=0 tx=0).stop 2023-10-11T09:49:32.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.259+0000 7f383359e700 1 -- 172.21.15.78:0/3799413364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f381c004db0 msgr2=0x7f38340c4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:32.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.259+0000 7f383359e700 1 --2- 172.21.15.78:0/3799413364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f381c004db0 0x7f38340c4a80 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f382400e810 tx=0x7f382400eb20 comp rx=0 tx=0).stop 2023-10-11T09:49:32.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.263+0000 7f383359e700 1 -- 172.21.15.78:0/3799413364 shutdown_connections 2023-10-11T09:49:32.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.263+0000 7f383359e700 1 --2- 172.21.15.78:0/3799413364 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3814065ed0 0x7f3814068380 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:32.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.263+0000 7f383359e700 1 --2- 172.21.15.78:0/3799413364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f383415b1c0 0x7f38340ae9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:32.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.263+0000 7f383359e700 1 --2- 172.21.15.78:0/3799413364 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f38340b6700 0x7f383415ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:32.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.263+0000 7f383359e700 1 --2- 172.21.15.78:0/3799413364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f381c004db0 0x7f38340c4a80 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:32.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.263+0000 7f383359e700 1 -- 172.21.15.78:0/3799413364 >> 172.21.15.78:0/3799413364 conn(0x7f383409a130 msgr2=0x7f38340d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:32.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.263+0000 7f383359e700 1 -- 172.21.15.78:0/3799413364 shutdown_connections 2023-10-11T09:49:32.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.263+0000 7f383359e700 1 -- 172.21.15.78:0/3799413364 wait complete. 2023-10-11T09:49:32.270 INFO:teuthology.orchestra.run.smithi078.stderr:listed 6 entries 2023-10-11T09:49:32.292 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi078.front.sepia.ceph.com... 2023-10-11T09:49:32.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:49:32.293 DEBUG:teuthology.orchestra.run.smithi078:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-11T09:49:32.324 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:ceph-fuse[54576]: fuse finished with error 0 and tester_r 0 2023-10-11T09:49:32.329 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2023-10-11T09:49:32.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:49:32.330 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-11T09:49:32.504 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:49:32.504 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:49:32.524 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352292k,nr_inodes=4088073,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1430 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275376k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-11T09:49:32.524 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2023-10-11T09:49:32.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:49:32.525 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-11T09:49:32.582 INFO:teuthology.orchestra.run.smithi133.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-10-11T09:49:32.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T09:49:32.584 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:49:32.584 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:49:32.602 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352272k,nr_inodes=4088068,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17487 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-11T09:49:32.603 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2023-10-11T09:49:32.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:49:32.603 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-11T09:49:32.663 INFO:teuthology.orchestra.run.smithi078.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-11T09:49:32.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T09:49:32.665 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:49:32.665 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:49:32.684 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352292k,nr_inodes=4088073,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1430 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275376k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-11T09:49:32.684 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2023-10-11T09:49:32.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:49:32.685 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-11T09:49:32.749 INFO:teuthology.orchestra.run.smithi133.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-11T09:49:32.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T09:49:32.751 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:49:32.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.923+0000 7f0acfbc6700 1 Processor -- start 2023-10-11T09:49:32.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.923+0000 7f0acfbc6700 1 -- start start 2023-10-11T09:49:32.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.927+0000 7f0acfbc6700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ac8071040 0x7f0ac80cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:32.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.927+0000 7f0acfbc6700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0ac80ce330 0x7f0ac806be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:32.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.927+0000 7f0acfbc6700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac806ffa0 con 0x7f0ac8160d40 2023-10-11T09:49:32.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.927+0000 7f0acfbc6700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ac8070c20 con 0x7f0ac80ce330 2023-10-11T09:49:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.927+0000 7f0acfbc6700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac8161110 con 0x7f0ac8071040 2023-10-11T09:49:32.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.927+0000 7f0acd962700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f0ac8160d40 0x7f0ac80cd280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:56576/0 (socket says 172.21.15.78:56576) 2023-10-11T09:49:32.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.927+0000 7f0acd962700 1 -- 172.21.15.78:0/1230858364 learned_addr learned my addr 172.21.15.78:0/1230858364 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:49:32.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.927+0000 7f0acd161700 1 --2- 172.21.15.78:0/1230858364 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ac8071040 0x7f0ac80cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:32.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.927+0000 7f0ace163700 1 --2- 172.21.15.78:0/1230858364 >> v2:172.21.15.133:3300/0 conn(0x7f0ac80ce330 0x7f0ac806be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:32.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.927+0000 7f0acc960700 1 -- 172.21.15.78:0/1230858364 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2739781790 0 0) 0x7f0ac806ffa0 con 0x7f0ac8160d40 2023-10-11T09:49:32.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.927+0000 7f0acc960700 1 -- 172.21.15.78:0/1230858364 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ab40035d0 con 0x7f0ac8160d40 2023-10-11T09:49:32.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.927+0000 7f0ace163700 1 -- 172.21.15.78:0/1230858364 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ac8071040 msgr2=0x7f0ac80cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:32.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.927+0000 7f0ace163700 1 --2- 172.21.15.78:0/1230858364 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ac8071040 0x7f0ac80cddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:32.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.927+0000 7f0ace163700 1 -- 172.21.15.78:0/1230858364 >> v1:172.21.15.78:6789/0 conn(0x7f0ac8160d40 legacy=0x7f0ac80cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:32.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.927+0000 7f0ace163700 1 -- 172.21.15.78:0/1230858364 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac8070280 con 0x7f0ac80ce330 2023-10-11T09:49:32.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.927+0000 7f0acd161700 1 --2- 172.21.15.78:0/1230858364 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ac8071040 0x7f0ac80cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:49:32.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.927+0000 7f0ace163700 1 --2- 172.21.15.78:0/1230858364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ac80ce330 0x7f0ac806be60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0009d90 tx=0x7f0ac000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0ffb10f41894f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:32.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.931+0000 7f0acc960700 1 -- 172.21.15.78:0/1230858364 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac000e070 con 0x7f0ac80ce330 2023-10-11T09:49:32.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.931+0000 7f0acc960700 1 -- 172.21.15.78:0/1230858364 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ac000bb20 con 0x7f0ac80ce330 2023-10-11T09:49:32.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.931+0000 7f0acc960700 1 -- 172.21.15.78:0/1230858364 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0012430 con 0x7f0ac80ce330 2023-10-11T09:49:32.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.931+0000 7f0acfbc6700 1 -- 172.21.15.78:0/1230858364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ac80ce330 msgr2=0x7f0ac806be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:32.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.931+0000 7f0acfbc6700 1 --2- 172.21.15.78:0/1230858364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ac80ce330 0x7f0ac806be60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0009d90 tx=0x7f0ac000b330 comp rx=0 tx=0).stop 2023-10-11T09:49:32.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.931+0000 7f0acfbc6700 1 -- 172.21.15.78:0/1230858364 shutdown_connections 2023-10-11T09:49:32.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.931+0000 7f0acfbc6700 1 --2- 172.21.15.78:0/1230858364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ac80ce330 0x7f0ac806be60 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:32.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.931+0000 7f0acfbc6700 1 --2- 172.21.15.78:0/1230858364 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ac8071040 0x7f0ac80cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:32.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.931+0000 7f0acfbc6700 1 -- 172.21.15.78:0/1230858364 >> 172.21.15.78:0/1230858364 conn(0x7f0ac806aca0 msgr2=0x7f0ac80b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:32.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.931+0000 7f0acfbc6700 1 -- 172.21.15.78:0/1230858364 shutdown_connections 2023-10-11T09:49:32.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.931+0000 7f0acfbc6700 1 -- 172.21.15.78:0/1230858364 wait complete. 2023-10-11T09:49:32.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.935+0000 7f0acfbc6700 1 Processor -- start 2023-10-11T09:49:32.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.935+0000 7f0acfbc6700 1 -- start start 2023-10-11T09:49:32.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.935+0000 7f0acfbc6700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ac8071040 0x7f0ac81f5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:32.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.935+0000 7f0acfbc6700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ac80ce330 0x7f0ac80c3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:32.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.935+0000 7f0acfbc6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ac8160d40 0x7f0ac8163380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:32.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.935+0000 7f0acfbc6700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac80c5730 con 0x7f0ac8160d40 2023-10-11T09:49:32.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.935+0000 7f0acfbc6700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac81f6350 con 0x7f0ac8071040 2023-10-11T09:49:32.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.935+0000 7f0acfbc6700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac81f9b30 con 0x7f0ac80ce330 2023-10-11T09:49:32.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.935+0000 7f0ace163700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ac8160d40 0x7f0ac8163380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:32.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.935+0000 7f0ace163700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ac8160d40 0x7f0ac8163380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39830/0 (socket says 172.21.15.78:39830) 2023-10-11T09:49:32.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.935+0000 7f0ace163700 1 -- 172.21.15.78:0/3231042790 learned_addr learned my addr 172.21.15.78:0/3231042790 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:32.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.935+0000 7f0acd161700 1 --2- 172.21.15.78:0/3231042790 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ac80ce330 0x7f0ac80c3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:32.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.935+0000 7f0acd962700 1 --2- 172.21.15.78:0/3231042790 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ac8071040 0x7f0ac81f5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:32.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.935+0000 7f0ace163700 1 -- 172.21.15.78:0/3231042790 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ac80ce330 msgr2=0x7f0ac80c3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:32.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.935+0000 7f0ace163700 1 --2- 172.21.15.78:0/3231042790 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ac80ce330 0x7f0ac80c3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:32.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.935+0000 7f0ace163700 1 -- 172.21.15.78:0/3231042790 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ac8071040 msgr2=0x7f0ac81f5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:32.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.935+0000 7f0ace163700 1 --2- 172.21.15.78:0/3231042790 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ac8071040 0x7f0ac81f5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:32.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.935+0000 7f0ace163700 1 -- 172.21.15.78:0/3231042790 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab80094b0 con 0x7f0ac8160d40 2023-10-11T09:49:32.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.935+0000 7f0ace163700 1 --2- 172.21.15.78:0/3231042790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ac8160d40 0x7f0ac8163380 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0014040 tx=0x7f0ac0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:32.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.935+0000 7f0abe7fc700 1 -- 172.21.15.78:0/3231042790 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0004780 con 0x7f0ac8160d40 2023-10-11T09:49:32.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.935+0000 7f0abe7fc700 1 -- 172.21.15.78:0/3231042790 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ac0020070 con 0x7f0ac8160d40 2023-10-11T09:49:32.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.935+0000 7f0acfbc6700 1 -- 172.21.15.78:0/3231042790 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac0009a70 con 0x7f0ac8160d40 2023-10-11T09:49:32.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.935+0000 7f0abe7fc700 1 -- 172.21.15.78:0/3231042790 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac00276c0 con 0x7f0ac8160d40 2023-10-11T09:49:32.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.935+0000 7f0acfbc6700 1 -- 172.21.15.78:0/3231042790 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac81fa0f0 con 0x7f0ac8160d40 2023-10-11T09:49:32.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.939+0000 7f0acfbc6700 1 -- 172.21.15.78:0/3231042790 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac8072250 con 0x7f0ac8160d40 2023-10-11T09:49:32.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.939+0000 7f0abe7fc700 1 -- 172.21.15.78:0/3231042790 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f0ac0012430 con 0x7f0ac8160d40 2023-10-11T09:49:32.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.939+0000 7f0abe7fc700 1 --2- 172.21.15.78:0/3231042790 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0aac065ed0 0x7f0aac068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:32.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.939+0000 7f0abe7fc700 1 -- 172.21.15.78:0/3231042790 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f0ac0022070 con 0x7f0ac8160d40 2023-10-11T09:49:32.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.939+0000 7f0acd962700 1 --2- 172.21.15.78:0/3231042790 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0aac065ed0 0x7f0aac068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:32.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.943+0000 7f0acd962700 1 --2- 172.21.15.78:0/3231042790 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0aac065ed0 0x7f0aac068380 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4007fd0 tx=0x7f0ac4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:32.947+0000 7f0abe7fc700 1 -- 172.21.15.78:0/3231042790 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f0ac0052ae0 con 0x7f0ac8160d40 2023-10-11T09:49:33.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.151+0000 7f0acfbc6700 1 -- 172.21.15.78:0/3231042790 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ac81fa8f0 con 0x7f0ac8160d40 2023-10-11T09:49:33.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.155+0000 7f0abe7fc700 1 -- 172.21.15.78:0/3231042790 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4657 (secure 0 0 0) 0x7f0ac81fa8f0 con 0x7f0ac8160d40 2023-10-11T09:49:33.161 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:49:33.161 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":71,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6581,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/238181631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":238181631},{"type":"v1","addr":"172.21.15.78:6837","nonce":238181631}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":7378,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/2416080754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2416080754},{"type":"v1","addr":"172.21.15.133:6834","nonce":2416080754}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":7384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/3016626706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3016626706},{"type":"v1","addr":"172.21.15.78:6836","nonce":3016626706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:48:45.351822+0000","modified":"2023-10-11T09:48:53.854831+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7725":{"gid":7725,"name":"b","rank":0,"incarnation":68,"state":"up:active","state_seq":4,"addr":"172.21.15.133:6835/3922016223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":3922016223},{"type":"v1","addr":"172.21.15.133:6835","nonce":3922016223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12,13],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-11T09:49:33.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.155+0000 7f0acfbc6700 1 -- 172.21.15.78:0/3231042790 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0aac065ed0 msgr2=0x7f0aac068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:33.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.155+0000 7f0acfbc6700 1 --2- 172.21.15.78:0/3231042790 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0aac065ed0 0x7f0aac068380 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4007fd0 tx=0x7f0ac4009040 comp rx=0 tx=0).stop 2023-10-11T09:49:33.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.155+0000 7f0acfbc6700 1 -- 172.21.15.78:0/3231042790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ac8160d40 msgr2=0x7f0ac8163380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:33.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.155+0000 7f0acfbc6700 1 --2- 172.21.15.78:0/3231042790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ac8160d40 0x7f0ac8163380 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0014040 tx=0x7f0ac0004af0 comp rx=0 tx=0).stop 2023-10-11T09:49:33.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.155+0000 7f0acfbc6700 1 -- 172.21.15.78:0/3231042790 shutdown_connections 2023-10-11T09:49:33.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.155+0000 7f0acfbc6700 1 --2- 172.21.15.78:0/3231042790 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0aac065ed0 0x7f0aac068380 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:33.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.155+0000 7f0acfbc6700 1 --2- 172.21.15.78:0/3231042790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ac8160d40 0x7f0ac8163380 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:33.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.155+0000 7f0acfbc6700 1 --2- 172.21.15.78:0/3231042790 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ac80ce330 0x7f0ac80c3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:33.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.155+0000 7f0acfbc6700 1 --2- 172.21.15.78:0/3231042790 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ac8071040 0x7f0ac81f5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:33.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.155+0000 7f0acfbc6700 1 -- 172.21.15.78:0/3231042790 >> 172.21.15.78:0/3231042790 conn(0x7f0ac806aca0 msgr2=0x7f0ac815ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:33.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.159+0000 7f0acfbc6700 1 -- 172.21.15.78:0/3231042790 shutdown_connections 2023-10-11T09:49:33.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.159+0000 7f0acfbc6700 1 -- 172.21.15.78:0/3231042790 wait complete. 2023-10-11T09:49:33.166 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 71 2023-10-11T09:49:33.184 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:49:33.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.347+0000 7f286be6a700 1 Processor -- start 2023-10-11T09:49:33.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.347+0000 7f286be6a700 1 -- start start 2023-10-11T09:49:33.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.351+0000 7f286be6a700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f2864160d40 0x7f28640cd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:33.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.351+0000 7f286be6a700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f28640ce500 0x7f286406be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:33.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.351+0000 7f286be6a700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f286406ffa0 con 0x7f2864071040 2023-10-11T09:49:33.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.351+0000 7f286be6a700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2864070c20 con 0x7f2864160d40 2023-10-11T09:49:33.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.351+0000 7f286be6a700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2864161110 con 0x7f28640ce500 2023-10-11T09:49:33.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.351+0000 7f2869c06700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f2864160d40 0x7f28640cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:33.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.351+0000 7f2869c06700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f2864160d40 0x7f28640cd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39836/0 (socket says 172.21.15.78:39836) 2023-10-11T09:49:33.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.351+0000 7f2869c06700 1 -- 172.21.15.78:0/1262319631 learned_addr learned my addr 172.21.15.78:0/1262319631 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:33.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.351+0000 7f286a407700 1 --2- 172.21.15.78:0/1262319631 >> v2:172.21.15.133:3300/0 conn(0x7f28640ce500 0x7f286406be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:33.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.351+0000 7f2869c06700 1 -- 172.21.15.78:0/1262319631 >> v2:172.21.15.133:3300/0 conn(0x7f28640ce500 msgr2=0x7f286406be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:33.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.351+0000 7f2869c06700 1 --2- 172.21.15.78:0/1262319631 >> v2:172.21.15.133:3300/0 conn(0x7f28640ce500 0x7f286406be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:33.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.351+0000 7f2869c06700 1 -- 172.21.15.78:0/1262319631 >> v1:172.21.15.133:6789/0 conn(0x7f2864071040 legacy=0x7f28640cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:33.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.351+0000 7f2869c06700 1 -- 172.21.15.78:0/1262319631 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2864070280 con 0x7f2864160d40 2023-10-11T09:49:33.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.351+0000 7f2869c06700 1 --2- 172.21.15.78:0/1262319631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2864160d40 0x7f28640cd280 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f286406f890 tx=0x7f2860009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b8d13db47f284de server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:33.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.351+0000 7f2868c04700 1 -- 172.21.15.78:0/1262319631 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2860011830 con 0x7f2864160d40 2023-10-11T09:49:33.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.351+0000 7f2868c04700 1 -- 172.21.15.78:0/1262319631 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2860011990 con 0x7f2864160d40 2023-10-11T09:49:33.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.351+0000 7f2868c04700 1 -- 172.21.15.78:0/1262319631 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2860011d20 con 0x7f2864160d40 2023-10-11T09:49:33.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.355+0000 7f286be6a700 1 -- 172.21.15.78:0/1262319631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2864160d40 msgr2=0x7f28640cd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:33.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.355+0000 7f286be6a700 1 --2- 172.21.15.78:0/1262319631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2864160d40 0x7f28640cd280 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f286406f890 tx=0x7f2860009bf0 comp rx=0 tx=0).stop 2023-10-11T09:49:33.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.355+0000 7f286be6a700 1 -- 172.21.15.78:0/1262319631 shutdown_connections 2023-10-11T09:49:33.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.355+0000 7f286be6a700 1 --2- 172.21.15.78:0/1262319631 >> v2:172.21.15.133:3300/0 conn(0x7f28640ce500 0x7f286406be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:33.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.355+0000 7f286be6a700 1 --2- 172.21.15.78:0/1262319631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2864160d40 0x7f28640cd280 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:33.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.355+0000 7f286be6a700 1 -- 172.21.15.78:0/1262319631 >> 172.21.15.78:0/1262319631 conn(0x7f286406aca0 msgr2=0x7f28640b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:33.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.355+0000 7f286be6a700 1 -- 172.21.15.78:0/1262319631 shutdown_connections 2023-10-11T09:49:33.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.355+0000 7f286be6a700 1 -- 172.21.15.78:0/1262319631 wait complete. 2023-10-11T09:49:33.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.359+0000 7f286be6a700 1 Processor -- start 2023-10-11T09:49:33.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.359+0000 7f286be6a700 1 -- start start 2023-10-11T09:49:33.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.359+0000 7f286be6a700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2864071040 0x7f28640c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:33.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.359+0000 7f286be6a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28640ce500 0x7f2864151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:33.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.359+0000 7f286be6a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2864160d40 0x7f28640c6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:33.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.359+0000 7f286be6a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286415fb70 con 0x7f2864160d40 2023-10-11T09:49:33.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.359+0000 7f286be6a700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28640c7220 con 0x7f28640ce500 2023-10-11T09:49:33.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.359+0000 7f286be6a700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28640c7500 con 0x7f2864071040 2023-10-11T09:49:33.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.359+0000 7f286a407700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2864160d40 0x7f28640c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:33.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.359+0000 7f286a407700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2864160d40 0x7f28640c6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39840/0 (socket says 172.21.15.78:39840) 2023-10-11T09:49:33.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.359+0000 7f286a407700 1 -- 172.21.15.78:0/2016094655 learned_addr learned my addr 172.21.15.78:0/2016094655 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:33.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.359+0000 7f2869405700 1 --2- 172.21.15.78:0/2016094655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28640ce500 0x7f2864151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:33.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.359+0000 7f2869c06700 1 --2- 172.21.15.78:0/2016094655 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2864071040 0x7f28640c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:33.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.359+0000 7f286a407700 1 -- 172.21.15.78:0/2016094655 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2864071040 msgr2=0x7f28640c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:33.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.359+0000 7f286a407700 1 --2- 172.21.15.78:0/2016094655 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2864071040 0x7f28640c82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:33.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.359+0000 7f286a407700 1 -- 172.21.15.78:0/2016094655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28640ce500 msgr2=0x7f2864151ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:33.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.359+0000 7f286a407700 1 --2- 172.21.15.78:0/2016094655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28640ce500 0x7f2864151ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:33.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.359+0000 7f286a407700 1 -- 172.21.15.78:0/2016094655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2860009570 con 0x7f2864160d40 2023-10-11T09:49:33.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.363+0000 7f286a407700 1 --2- 172.21.15.78:0/2016094655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2864160d40 0x7f28640c6640 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f285c00efc0 tx=0x7f285c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:33.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.363+0000 7f285a7fc700 1 -- 172.21.15.78:0/2016094655 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285c009ca0 con 0x7f2864160d40 2023-10-11T09:49:33.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.363+0000 7f286be6a700 1 -- 172.21.15.78:0/2016094655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28640c77e0 con 0x7f2864160d40 2023-10-11T09:49:33.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.363+0000 7f285a7fc700 1 -- 172.21.15.78:0/2016094655 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f285c00f040 con 0x7f2864160d40 2023-10-11T09:49:33.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.363+0000 7f285a7fc700 1 -- 172.21.15.78:0/2016094655 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285c014480 con 0x7f2864160d40 2023-10-11T09:49:33.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.363+0000 7f286be6a700 1 -- 172.21.15.78:0/2016094655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28640c7da0 con 0x7f2864160d40 2023-10-11T09:49:33.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.363+0000 7f286be6a700 1 -- 172.21.15.78:0/2016094655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2864072250 con 0x7f2864160d40 2023-10-11T09:49:33.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.367+0000 7f285a7fc700 1 -- 172.21.15.78:0/2016094655 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f285c0145e0 con 0x7f2864160d40 2023-10-11T09:49:33.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.367+0000 7f285a7fc700 1 --2- 172.21.15.78:0/2016094655 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2848065ed0 0x7f2848068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:33.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.367+0000 7f285a7fc700 1 -- 172.21.15.78:0/2016094655 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f285c0849d0 con 0x7f2864160d40 2023-10-11T09:49:33.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.367+0000 7f2869c06700 1 --2- 172.21.15.78:0/2016094655 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2848065ed0 0x7f2848068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:33.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.367+0000 7f2869c06700 1 --2- 172.21.15.78:0/2016094655 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2848065ed0 0x7f2848068380 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2860000c00 tx=0x7f2860008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:33.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.371+0000 7f285a7fc700 1 -- 172.21.15.78:0/2016094655 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f285c051bd0 con 0x7f2864160d40 2023-10-11T09:49:33.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.543+0000 7f286be6a700 1 -- 172.21.15.78:0/2016094655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28640c7fc0 con 0x7f2864160d40 2023-10-11T09:49:33.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.547+0000 7f285a7fc700 1 -- 172.21.15.78:0/2016094655 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+20058 (secure 0 0 0) 0x7f285c018090 con 0x7f2864160d40 2023-10-11T09:49:33.555 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:49:33.555 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":92,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:48:50.804275+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:48:18.491556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-11T09:48:19.018937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data_extra","create_time":"2023-10-11T09:48:28.963956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6834/1560261029":"2023-10-12T09:48:50.804239+0000","172.21.15.78:6836/1560261029":"2023-10-12T09:48:50.804239+0000","172.21.15.78:6836/3463904224":"2023-10-12T09:48:45.480946+0000","172.21.15.78:6834/3463904224":"2023-10-12T09:48:45.480946+0000","172.21.15.78:6837/951312831":"2023-10-12T09:48:44.433517+0000","172.21.15.78:6835/951312831":"2023-10-12T09:48:44.433517+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:49:33.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.551+0000 7f286be6a700 1 -- 172.21.15.78:0/2016094655 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2848065ed0 msgr2=0x7f2848068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:33.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.551+0000 7f286be6a700 1 --2- 172.21.15.78:0/2016094655 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2848065ed0 0x7f2848068380 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2860000c00 tx=0x7f2860008030 comp rx=0 tx=0).stop 2023-10-11T09:49:33.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.551+0000 7f286be6a700 1 -- 172.21.15.78:0/2016094655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2864160d40 msgr2=0x7f28640c6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:33.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.551+0000 7f286be6a700 1 --2- 172.21.15.78:0/2016094655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2864160d40 0x7f28640c6640 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f285c00efc0 tx=0x7f285c00c3a0 comp rx=0 tx=0).stop 2023-10-11T09:49:33.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.551+0000 7f286be6a700 1 -- 172.21.15.78:0/2016094655 shutdown_connections 2023-10-11T09:49:33.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.551+0000 7f286be6a700 1 --2- 172.21.15.78:0/2016094655 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2848065ed0 0x7f2848068380 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:33.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.551+0000 7f286be6a700 1 --2- 172.21.15.78:0/2016094655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2864160d40 0x7f28640c6640 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:33.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.551+0000 7f286be6a700 1 --2- 172.21.15.78:0/2016094655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28640ce500 0x7f2864151ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:33.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.551+0000 7f286be6a700 1 --2- 172.21.15.78:0/2016094655 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2864071040 0x7f28640c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:33.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.551+0000 7f286be6a700 1 -- 172.21.15.78:0/2016094655 >> 172.21.15.78:0/2016094655 conn(0x7f286406aca0 msgr2=0x7f286406c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:33.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.551+0000 7f286be6a700 1 -- 172.21.15.78:0/2016094655 shutdown_connections 2023-10-11T09:49:33.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.551+0000 7f286be6a700 1 -- 172.21.15.78:0/2016094655 wait complete. 2023-10-11T09:49:33.581 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T09:48:18.491556+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T09:48:19.018937+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-10-11T09:48:28.963956+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:49:33.582 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:49:33.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.755+0000 7f950f197700 1 Processor -- start 2023-10-11T09:49:33.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.755+0000 7f950f197700 1 -- start start 2023-10-11T09:49:33.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.755+0000 7f950f197700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f950815a6b0 0x7f95080ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:33.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.759+0000 7f950f197700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95080b3a60 0x7f950806d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:33.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.759+0000 7f950f197700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f95080cede0 0x7f950806d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:33.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.759+0000 7f950f197700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95080b2d30 con 0x7f95080cede0 2023-10-11T09:49:33.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.759+0000 7f950f197700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95080cf360 con 0x7f950815a6b0 2023-10-11T09:49:33.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.759+0000 7f950f197700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95080b2f80 con 0x7f95080b3a60 2023-10-11T09:49:33.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.759+0000 7f950e996700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f95080cede0 0x7f950806d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:33.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.759+0000 7f950e996700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f95080cede0 0x7f950806d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39846/0 (socket says 172.21.15.78:39846) 2023-10-11T09:49:33.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.759+0000 7f950e195700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f950815a6b0 0x7f95080ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:33.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.759+0000 7f950e996700 1 -- 172.21.15.78:0/1476293751 learned_addr learned my addr 172.21.15.78:0/1476293751 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:33.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.759+0000 7f950d994700 1 --2- 172.21.15.78:0/1476293751 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95080b3a60 0x7f950806d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:33.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.759+0000 7f950e996700 1 -- 172.21.15.78:0/1476293751 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95080b3a60 msgr2=0x7f950806d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:33.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.759+0000 7f950e996700 1 --2- 172.21.15.78:0/1476293751 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95080b3a60 0x7f950806d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:33.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.759+0000 7f950e996700 1 -- 172.21.15.78:0/1476293751 >> v2:172.21.15.133:3300/0 conn(0x7f950815a6b0 msgr2=0x7f95080ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:33.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.759+0000 7f950e996700 1 --2- 172.21.15.78:0/1476293751 >> v2:172.21.15.133:3300/0 conn(0x7f950815a6b0 0x7f95080ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:33.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.759+0000 7f950e996700 1 -- 172.21.15.78:0/1476293751 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95080b3260 con 0x7f95080cede0 2023-10-11T09:49:33.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.759+0000 7f950e996700 1 --2- 172.21.15.78:0/1476293751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95080cede0 0x7f950806d560 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f9508072550 tx=0x7f950000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3adcafbbf4fd42 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:33.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.759+0000 7f950d193700 1 -- 172.21.15.78:0/1476293751 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950000e070 con 0x7f95080cede0 2023-10-11T09:49:33.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.759+0000 7f950d193700 1 -- 172.21.15.78:0/1476293751 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f950000b910 con 0x7f95080cede0 2023-10-11T09:49:33.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.759+0000 7f950d193700 1 -- 172.21.15.78:0/1476293751 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950000bcc0 con 0x7f95080cede0 2023-10-11T09:49:33.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.759+0000 7f950f197700 1 -- 172.21.15.78:0/1476293751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95080cede0 msgr2=0x7f950806d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:33.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.759+0000 7f950f197700 1 --2- 172.21.15.78:0/1476293751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95080cede0 0x7f950806d560 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f9508072550 tx=0x7f950000b330 comp rx=0 tx=0).stop 2023-10-11T09:49:33.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.763+0000 7f950f197700 1 -- 172.21.15.78:0/1476293751 shutdown_connections 2023-10-11T09:49:33.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.763+0000 7f950f197700 1 --2- 172.21.15.78:0/1476293751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95080cede0 0x7f950806d560 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:33.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.763+0000 7f950f197700 1 --2- 172.21.15.78:0/1476293751 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95080b3a60 0x7f950806d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:33.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.763+0000 7f950f197700 1 --2- 172.21.15.78:0/1476293751 >> v2:172.21.15.133:3300/0 conn(0x7f950815a6b0 0x7f95080ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:33.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.763+0000 7f950f197700 1 -- 172.21.15.78:0/1476293751 >> 172.21.15.78:0/1476293751 conn(0x7f950806a060 msgr2=0x7f950806a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:33.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.763+0000 7f950f197700 1 -- 172.21.15.78:0/1476293751 shutdown_connections 2023-10-11T09:49:33.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.763+0000 7f950f197700 1 -- 172.21.15.78:0/1476293751 wait complete. 2023-10-11T09:49:33.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.763+0000 7f950f197700 1 Processor -- start 2023-10-11T09:49:33.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.763+0000 7f950f197700 1 -- start start 2023-10-11T09:49:33.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.767+0000 7f950f197700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95080b3a60 0x7f9508142f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:33.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.767+0000 7f950f197700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95080cede0 0x7f95081f1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:33.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.767+0000 7f950f197700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f950815a6b0 0x7f95081f51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:33.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.767+0000 7f950f197700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f950815d350 con 0x7f950815a6b0 2023-10-11T09:49:33.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.767+0000 7f950f197700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95080b0fa0 con 0x7f95080b3a60 2023-10-11T09:49:33.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.767+0000 7f950f197700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f950815ca20 con 0x7f95080cede0 2023-10-11T09:49:33.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.767+0000 7f950e996700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f950815a6b0 0x7f95081f51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:33.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.767+0000 7f950d994700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95080cede0 0x7f95081f1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:33.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.767+0000 7f950e996700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f950815a6b0 0x7f95081f51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39850/0 (socket says 172.21.15.78:39850) 2023-10-11T09:49:33.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.767+0000 7f950e996700 1 -- 172.21.15.78:0/2291817352 learned_addr learned my addr 172.21.15.78:0/2291817352 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:33.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.767+0000 7f950e195700 1 --2- 172.21.15.78:0/2291817352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95080b3a60 0x7f9508142f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:33.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.767+0000 7f950e996700 1 -- 172.21.15.78:0/2291817352 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95080cede0 msgr2=0x7f95081f1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:33.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.767+0000 7f950e996700 1 --2- 172.21.15.78:0/2291817352 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95080cede0 0x7f95081f1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:33.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.767+0000 7f950e996700 1 -- 172.21.15.78:0/2291817352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95080b3a60 msgr2=0x7f9508142f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:33.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.767+0000 7f950e996700 1 --2- 172.21.15.78:0/2291817352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95080b3a60 0x7f9508142f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:33.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.767+0000 7f950e996700 1 -- 172.21.15.78:0/2291817352 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9500009b30 con 0x7f950815a6b0 2023-10-11T09:49:33.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.767+0000 7f950e996700 1 --2- 172.21.15.78:0/2291817352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f950815a6b0 0x7f95081f51d0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f95080725d0 tx=0x7f9500004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:33.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.767+0000 7f94feffd700 1 -- 172.21.15.78:0/2291817352 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95000099b0 con 0x7f950815a6b0 2023-10-11T09:49:33.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.767+0000 7f950f197700 1 -- 172.21.15.78:0/2291817352 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f950815cd00 con 0x7f950815a6b0 2023-10-11T09:49:33.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.767+0000 7f94feffd700 1 -- 172.21.15.78:0/2291817352 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f950000ce80 con 0x7f950815a6b0 2023-10-11T09:49:33.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.767+0000 7f94feffd700 1 -- 172.21.15.78:0/2291817352 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9500004490 con 0x7f950815a6b0 2023-10-11T09:49:33.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.767+0000 7f950f197700 1 -- 172.21.15.78:0/2291817352 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95081f1880 con 0x7f950815a6b0 2023-10-11T09:49:33.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.771+0000 7f950f197700 1 -- 172.21.15.78:0/2291817352 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f950815d4c0 con 0x7f950815a6b0 2023-10-11T09:49:33.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.771+0000 7f94feffd700 1 -- 172.21.15.78:0/2291817352 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f95000045f0 con 0x7f950815a6b0 2023-10-11T09:49:33.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.771+0000 7f94feffd700 1 --2- 172.21.15.78:0/2291817352 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f94ec065ed0 0x7f94ec068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:33.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.771+0000 7f94feffd700 1 -- 172.21.15.78:0/2291817352 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f9500084e30 con 0x7f950815a6b0 2023-10-11T09:49:33.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.775+0000 7f950e195700 1 --2- 172.21.15.78:0/2291817352 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f94ec065ed0 0x7f94ec068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:33.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.775+0000 7f950e195700 1 --2- 172.21.15.78:0/2291817352 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f94ec065ed0 0x7f94ec068380 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9504009ad0 tx=0x7f9504009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:33.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.779+0000 7f94feffd700 1 -- 172.21.15.78:0/2291817352 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f9500052200 con 0x7f950815a6b0 2023-10-11T09:49:33.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.955+0000 7f950f197700 1 -- 172.21.15.78:0/2291817352 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9508065d40 con 0x7f950815a6b0 2023-10-11T09:49:33.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.959+0000 7f94feffd700 1 -- 172.21.15.78:0/2291817352 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f95000048a0 con 0x7f950815a6b0 2023-10-11T09:49:33.964 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:49:33.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.963+0000 7f950f197700 1 -- 172.21.15.78:0/2291817352 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f94ec065ed0 msgr2=0x7f94ec068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:33.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.963+0000 7f950f197700 1 --2- 172.21.15.78:0/2291817352 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f94ec065ed0 0x7f94ec068380 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9504009ad0 tx=0x7f9504009240 comp rx=0 tx=0).stop 2023-10-11T09:49:33.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.963+0000 7f950f197700 1 -- 172.21.15.78:0/2291817352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f950815a6b0 msgr2=0x7f95081f51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:33.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.963+0000 7f950f197700 1 --2- 172.21.15.78:0/2291817352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f950815a6b0 0x7f95081f51d0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f95080725d0 tx=0x7f9500004af0 comp rx=0 tx=0).stop 2023-10-11T09:49:33.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.963+0000 7f950f197700 1 -- 172.21.15.78:0/2291817352 shutdown_connections 2023-10-11T09:49:33.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.963+0000 7f950f197700 1 --2- 172.21.15.78:0/2291817352 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f94ec065ed0 0x7f94ec068380 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:33.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.963+0000 7f950f197700 1 --2- 172.21.15.78:0/2291817352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f950815a6b0 0x7f95081f51d0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:33.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.963+0000 7f950f197700 1 --2- 172.21.15.78:0/2291817352 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95080cede0 0x7f95081f1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:33.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.963+0000 7f950f197700 1 --2- 172.21.15.78:0/2291817352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95080b3a60 0x7f9508142f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:33.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.963+0000 7f950f197700 1 -- 172.21.15.78:0/2291817352 >> 172.21.15.78:0/2291817352 conn(0x7f950806a060 msgr2=0x7f95080b1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:33.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.963+0000 7f950f197700 1 -- 172.21.15.78:0/2291817352 shutdown_connections 2023-10-11T09:49:33.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:33.963+0000 7f950f197700 1 -- 172.21.15.78:0/2291817352 wait complete. 2023-10-11T09:49:33.989 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:49:34.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.159+0000 7fc24d9a0700 1 Processor -- start 2023-10-11T09:49:34.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.159+0000 7fc24d9a0700 1 -- start start 2023-10-11T09:49:34.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.163+0000 7fc24d9a0700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc24815a810 0x7fc2480b2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:34.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.163+0000 7fc24d9a0700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fc24806fe50 0x7fc2480b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:34.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.163+0000 7fc24d9a0700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc2480cec00 0x7fc2480cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:34.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.163+0000 7fc24d9a0700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc24815abe0 con 0x7fc24806fe50 2023-10-11T09:49:34.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.163+0000 7fc24d9a0700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc24806f9c0 con 0x7fc2480cec00 2023-10-11T09:49:34.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.163+0000 7fc24d9a0700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc24806edb0 con 0x7fc24815a810 2023-10-11T09:49:34.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.163+0000 7fc23e7fc700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fc24806fe50 0x7fc2480b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:34.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.163+0000 7fc23e7fc700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fc24806fe50 0x7fc2480b2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39852/0 (socket says 172.21.15.78:39852) 2023-10-11T09:49:34.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.163+0000 7fc23e7fc700 1 -- 172.21.15.78:0/3217693348 learned_addr learned my addr 172.21.15.78:0/3217693348 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:34.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.163+0000 7fc246ffd700 1 --2- 172.21.15.78:0/3217693348 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc24815a810 0x7fc2480b2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:34.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.163+0000 7fc2477fe700 1 --2- 172.21.15.78:0/3217693348 >> v2:172.21.15.133:3300/0 conn(0x7fc2480cec00 0x7fc2480cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:34.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.163+0000 7fc23e7fc700 1 -- 172.21.15.78:0/3217693348 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc24815a810 msgr2=0x7fc2480b2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:34.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.163+0000 7fc23e7fc700 1 --2- 172.21.15.78:0/3217693348 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc24815a810 0x7fc2480b2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:34.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.163+0000 7fc23e7fc700 1 -- 172.21.15.78:0/3217693348 >> v2:172.21.15.133:3300/0 conn(0x7fc2480cec00 msgr2=0x7fc2480cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:34.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.163+0000 7fc23e7fc700 1 --2- 172.21.15.78:0/3217693348 >> v2:172.21.15.133:3300/0 conn(0x7fc2480cec00 0x7fc2480cf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:34.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.163+0000 7fc23e7fc700 1 -- 172.21.15.78:0/3217693348 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc24806f090 con 0x7fc24806fe50 2023-10-11T09:49:34.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.163+0000 7fc2477fe700 1 --2- 172.21.15.78:0/3217693348 >> v2:172.21.15.133:3300/0 conn(0x7fc2480cec00 0x7fc2480cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:49:34.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.163+0000 7fc23e7fc700 1 --2- 172.21.15.78:0/3217693348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc24806fe50 0x7fc2480b2f80 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fc24809a940 tx=0x7fc230009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ad9dcfe5f1670a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:34.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.163+0000 7fc2467fc700 1 -- 172.21.15.78:0/3217693348 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23000b8a0 con 0x7fc24806fe50 2023-10-11T09:49:34.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.163+0000 7fc2467fc700 1 -- 172.21.15.78:0/3217693348 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc23000ba00 con 0x7fc24806fe50 2023-10-11T09:49:34.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.163+0000 7fc2467fc700 1 -- 172.21.15.78:0/3217693348 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23000bd90 con 0x7fc24806fe50 2023-10-11T09:49:34.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.167+0000 7fc24d9a0700 1 -- 172.21.15.78:0/3217693348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc24806fe50 msgr2=0x7fc2480b2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:34.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.167+0000 7fc24d9a0700 1 --2- 172.21.15.78:0/3217693348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc24806fe50 0x7fc2480b2f80 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fc24809a940 tx=0x7fc230009bf0 comp rx=0 tx=0).stop 2023-10-11T09:49:34.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.167+0000 7fc24d9a0700 1 -- 172.21.15.78:0/3217693348 shutdown_connections 2023-10-11T09:49:34.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.167+0000 7fc24d9a0700 1 --2- 172.21.15.78:0/3217693348 >> v2:172.21.15.133:3300/0 conn(0x7fc2480cec00 0x7fc2480cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:34.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.167+0000 7fc24d9a0700 1 --2- 172.21.15.78:0/3217693348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc24806fe50 0x7fc2480b2f80 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:34.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.167+0000 7fc24d9a0700 1 --2- 172.21.15.78:0/3217693348 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc24815a810 0x7fc2480b2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:34.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.167+0000 7fc24d9a0700 1 -- 172.21.15.78:0/3217693348 >> 172.21.15.78:0/3217693348 conn(0x7fc24809a230 msgr2=0x7fc24808aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:34.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.167+0000 7fc24d9a0700 1 -- 172.21.15.78:0/3217693348 shutdown_connections 2023-10-11T09:49:34.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.167+0000 7fc24d9a0700 1 -- 172.21.15.78:0/3217693348 wait complete. 2023-10-11T09:49:34.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.171+0000 7fc24d9a0700 1 Processor -- start 2023-10-11T09:49:34.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.171+0000 7fc24d9a0700 1 -- start start 2023-10-11T09:49:34.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.171+0000 7fc24d9a0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc24806fe50 0x7fc2480c7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:34.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.171+0000 7fc24d9a0700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2480cec00 0x7fc248159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:34.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.171+0000 7fc24d9a0700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc24815a810 0x7fc248159580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:34.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.171+0000 7fc24d9a0700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2480c51a0 con 0x7fc24806fe50 2023-10-11T09:49:34.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.171+0000 7fc24d9a0700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2481f6c30 con 0x7fc24815a810 2023-10-11T09:49:34.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.171+0000 7fc24d9a0700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2480c7020 con 0x7fc2480cec00 2023-10-11T09:49:34.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.171+0000 7fc246ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc24806fe50 0x7fc2480c7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:34.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.171+0000 7fc246ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc24806fe50 0x7fc2480c7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39868/0 (socket says 172.21.15.78:39868) 2023-10-11T09:49:34.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.171+0000 7fc246ffd700 1 -- 172.21.15.78:0/3020915158 learned_addr learned my addr 172.21.15.78:0/3020915158 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:34.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.171+0000 7fc23e7fc700 1 --2- 172.21.15.78:0/3020915158 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2480cec00 0x7fc248159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:34.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.171+0000 7fc2477fe700 1 --2- 172.21.15.78:0/3020915158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc24815a810 0x7fc248159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:34.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.175+0000 7fc246ffd700 1 -- 172.21.15.78:0/3020915158 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2480cec00 msgr2=0x7fc248159040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:34.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.175+0000 7fc246ffd700 1 --2- 172.21.15.78:0/3020915158 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2480cec00 0x7fc248159040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:34.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.175+0000 7fc246ffd700 1 -- 172.21.15.78:0/3020915158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc24815a810 msgr2=0x7fc248159580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:34.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.175+0000 7fc246ffd700 1 --2- 172.21.15.78:0/3020915158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc24815a810 0x7fc248159580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:34.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.175+0000 7fc246ffd700 1 -- 172.21.15.78:0/3020915158 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc230009570 con 0x7fc24806fe50 2023-10-11T09:49:34.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.175+0000 7fc2477fe700 1 --2- 172.21.15.78:0/3020915158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc24815a810 0x7fc248159580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:49:34.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.175+0000 7fc246ffd700 1 --2- 172.21.15.78:0/3020915158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc24806fe50 0x7fc2480c7460 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fc238007ce0 tx=0x7fc23800e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:34.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.175+0000 7fc244ff9700 1 -- 172.21.15.78:0/3020915158 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23800ebc0 con 0x7fc24806fe50 2023-10-11T09:49:34.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.175+0000 7fc244ff9700 1 -- 172.21.15.78:0/3020915158 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc23800ed20 con 0x7fc24806fe50 2023-10-11T09:49:34.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.175+0000 7fc244ff9700 1 -- 172.21.15.78:0/3020915158 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc238014660 con 0x7fc24806fe50 2023-10-11T09:49:34.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.175+0000 7fc24d9a0700 1 -- 172.21.15.78:0/3020915158 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2480c79a0 con 0x7fc24806fe50 2023-10-11T09:49:34.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.175+0000 7fc24d9a0700 1 -- 172.21.15.78:0/3020915158 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2480c5670 con 0x7fc24806fe50 2023-10-11T09:49:34.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.175+0000 7fc24d9a0700 1 -- 172.21.15.78:0/3020915158 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc248070570 con 0x7fc24806fe50 2023-10-11T09:49:34.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.179+0000 7fc244ff9700 1 -- 172.21.15.78:0/3020915158 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fc2380147c0 con 0x7fc24806fe50 2023-10-11T09:49:34.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.179+0000 7fc244ff9700 1 --2- 172.21.15.78:0/3020915158 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc228065ed0 0x7fc228068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:34.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.179+0000 7fc244ff9700 1 -- 172.21.15.78:0/3020915158 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6724+0+0 (secure 0 0 0) 0x7fc238083ee0 con 0x7fc24806fe50 2023-10-11T09:49:34.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.179+0000 7fc23e7fc700 1 --2- 172.21.15.78:0/3020915158 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc228065ed0 0x7fc228068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:34.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.179+0000 7fc23e7fc700 1 --2- 172.21.15.78:0/3020915158 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc228065ed0 0x7fc228068380 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc24809a940 tx=0x7fc230008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:34.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.183+0000 7fc244ff9700 1 -- 172.21.15.78:0/3020915158 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fc238051200 con 0x7fc24806fe50 2023-10-11T09:49:34.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.367+0000 7fc24d9a0700 1 -- 172.21.15.78:0/3020915158 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc2480c79a0 con 0x7fc24806fe50 2023-10-11T09:49:34.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.367+0000 7fc244ff9700 1 -- 172.21.15.78:0/3020915158 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v92) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc2480c79a0 con 0x7fc24806fe50 2023-10-11T09:49:34.373 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:49:34.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.367+0000 7fc24d9a0700 1 -- 172.21.15.78:0/3020915158 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc228065ed0 msgr2=0x7fc228068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:34.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.367+0000 7fc24d9a0700 1 --2- 172.21.15.78:0/3020915158 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc228065ed0 0x7fc228068380 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc24809a940 tx=0x7fc230008030 comp rx=0 tx=0).stop 2023-10-11T09:49:34.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.367+0000 7fc24d9a0700 1 -- 172.21.15.78:0/3020915158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc24806fe50 msgr2=0x7fc2480c7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:34.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.367+0000 7fc24d9a0700 1 --2- 172.21.15.78:0/3020915158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc24806fe50 0x7fc2480c7460 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fc238007ce0 tx=0x7fc23800e0e0 comp rx=0 tx=0).stop 2023-10-11T09:49:34.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.367+0000 7fc24d9a0700 1 -- 172.21.15.78:0/3020915158 shutdown_connections 2023-10-11T09:49:34.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.367+0000 7fc24d9a0700 1 --2- 172.21.15.78:0/3020915158 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc228065ed0 0x7fc228068380 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:34.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.367+0000 7fc24d9a0700 1 --2- 172.21.15.78:0/3020915158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc24815a810 0x7fc248159580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:34.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.371+0000 7fc24d9a0700 1 --2- 172.21.15.78:0/3020915158 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2480cec00 0x7fc248159040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:34.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.371+0000 7fc24d9a0700 1 --2- 172.21.15.78:0/3020915158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc24806fe50 0x7fc2480c7460 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:34.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.371+0000 7fc24d9a0700 1 -- 172.21.15.78:0/3020915158 >> 172.21.15.78:0/3020915158 conn(0x7fc24809a230 msgr2=0x7fc2480b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:34.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.371+0000 7fc24d9a0700 1 -- 172.21.15.78:0/3020915158 shutdown_connections 2023-10-11T09:49:34.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.371+0000 7fc24d9a0700 1 -- 172.21.15.78:0/3020915158 wait complete. 2023-10-11T09:49:34.395 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T09:49:34.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.559+0000 7f453a8b1700 1 Processor -- start 2023-10-11T09:49:34.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.559+0000 7f453a8b1700 1 -- start start 2023-10-11T09:49:34.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.563+0000 7f453a8b1700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4534146fe0 0x7f453406bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:34.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.563+0000 7f453a8b1700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f453406ccb0 con 0x7f4534161d90 2023-10-11T09:49:34.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.563+0000 7f453a8b1700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f453406c410 con 0x7f45340b36f0 2023-10-11T09:49:34.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.563+0000 7f453a8b1700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f453406c6f0 con 0x7f4534146fe0 2023-10-11T09:49:34.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.563+0000 7f45390ae700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f4534161d90 0x7f45340b2fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:49474/0 (socket says 172.21.15.78:49474) 2023-10-11T09:49:34.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.563+0000 7f45390ae700 1 -- 172.21.15.78:0/805229311 learned_addr learned my addr 172.21.15.78:0/805229311 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:49:34.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.563+0000 7f45398af700 1 --2- 172.21.15.78:0/805229311 >> v2:172.21.15.133:3300/0 conn(0x7f4534146fe0 0x7f453406bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:34.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.563+0000 7f45388ad700 1 -- 172.21.15.78:0/805229311 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1860987411 0 0) 0x7f4524001110 con 0x7f4534161d90 2023-10-11T09:49:34.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.563+0000 7f45388ad700 1 -- 172.21.15.78:0/805229311 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45200035d0 con 0x7f4534161d90 2023-10-11T09:49:34.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.563+0000 7f45398af700 1 -- 172.21.15.78:0/805229311 >> v1:172.21.15.133:6789/0 conn(0x7f45340b36f0 legacy=0x7f45341629e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:34.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.563+0000 7f45398af700 1 -- 172.21.15.78:0/805229311 >> v1:172.21.15.78:6789/0 conn(0x7f4534161d90 legacy=0x7f45340b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:34.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.563+0000 7f45398af700 1 -- 172.21.15.78:0/805229311 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f453406c9d0 con 0x7f4534146fe0 2023-10-11T09:49:34.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.567+0000 7f45398af700 1 --2- 172.21.15.78:0/805229311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4534146fe0 0x7f453406bbe0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4530009850 tx=0x7f4530009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6d3027b13471566 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:34.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.567+0000 7f45388ad700 1 -- 172.21.15.78:0/805229311 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4530011830 con 0x7f4534146fe0 2023-10-11T09:49:34.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.567+0000 7f45388ad700 1 -- 172.21.15.78:0/805229311 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4530011990 con 0x7f4534146fe0 2023-10-11T09:49:34.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.567+0000 7f45388ad700 1 -- 172.21.15.78:0/805229311 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4530011d40 con 0x7f4534146fe0 2023-10-11T09:49:34.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.567+0000 7f453a8b1700 1 -- 172.21.15.78:0/805229311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4534146fe0 msgr2=0x7f453406bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:34.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.567+0000 7f453a8b1700 1 --2- 172.21.15.78:0/805229311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4534146fe0 0x7f453406bbe0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4530009850 tx=0x7f4530009b60 comp rx=0 tx=0).stop 2023-10-11T09:49:34.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.567+0000 7f453a8b1700 1 -- 172.21.15.78:0/805229311 shutdown_connections 2023-10-11T09:49:34.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.567+0000 7f453a8b1700 1 --2- 172.21.15.78:0/805229311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4534146fe0 0x7f453406bbe0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:34.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.567+0000 7f453a8b1700 1 -- 172.21.15.78:0/805229311 >> 172.21.15.78:0/805229311 conn(0x7f453406a060 msgr2=0x7f453406a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:34.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.567+0000 7f453a8b1700 1 -- 172.21.15.78:0/805229311 shutdown_connections 2023-10-11T09:49:34.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.567+0000 7f453a8b1700 1 -- 172.21.15.78:0/805229311 wait complete. 2023-10-11T09:49:34.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.571+0000 7f453a8b1700 1 Processor -- start 2023-10-11T09:49:34.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.571+0000 7f453a8b1700 1 -- start start 2023-10-11T09:49:34.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.571+0000 7f453a8b1700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f45340b36f0 0x7f453415ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:34.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.571+0000 7f453a8b1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4534146fe0 0x7f45341431c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:34.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.571+0000 7f453a8b1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4534161d90 0x7f45340b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:34.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.571+0000 7f453a8b1700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453415a870 con 0x7f4534146fe0 2023-10-11T09:49:34.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.571+0000 7f453a8b1700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45341599a0 con 0x7f4534161d90 2023-10-11T09:49:34.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.571+0000 7f453a8b1700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4534159bf0 con 0x7f45340b36f0 2023-10-11T09:49:34.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.571+0000 7f45390ae700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4534146fe0 0x7f45341431c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:34.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.571+0000 7f45390ae700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4534146fe0 0x7f45341431c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38484/0 (socket says 172.21.15.78:38484) 2023-10-11T09:49:34.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.571+0000 7f45390ae700 1 -- 172.21.15.78:0/3055840013 learned_addr learned my addr 172.21.15.78:0/3055840013 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:34.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.575+0000 7f45398af700 1 --2- 172.21.15.78:0/3055840013 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f45340b36f0 0x7f453415ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:34.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.575+0000 7f453a0b0700 1 --2- 172.21.15.78:0/3055840013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4534161d90 0x7f45340b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:34.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.575+0000 7f45390ae700 1 -- 172.21.15.78:0/3055840013 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f45340b36f0 msgr2=0x7f453415ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:34.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.575+0000 7f45390ae700 1 --2- 172.21.15.78:0/3055840013 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f45340b36f0 0x7f453415ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:34.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.575+0000 7f45390ae700 1 -- 172.21.15.78:0/3055840013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4534161d90 msgr2=0x7f45340b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:34.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.575+0000 7f45390ae700 1 --2- 172.21.15.78:0/3055840013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4534161d90 0x7f45340b05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:34.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.575+0000 7f45390ae700 1 -- 172.21.15.78:0/3055840013 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45300094b0 con 0x7f4534146fe0 2023-10-11T09:49:34.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.575+0000 7f45390ae700 1 --2- 172.21.15.78:0/3055840013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4534146fe0 0x7f45341431c0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f452400c4f0 tx=0x7f452400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:34.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.575+0000 7f452a7fc700 1 -- 172.21.15.78:0/3055840013 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4524005d90 con 0x7f4534146fe0 2023-10-11T09:49:34.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.575+0000 7f452a7fc700 1 -- 172.21.15.78:0/3055840013 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45240070d0 con 0x7f4534146fe0 2023-10-11T09:49:34.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.575+0000 7f453a8b1700 1 -- 172.21.15.78:0/3055840013 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4534159ed0 con 0x7f4534146fe0 2023-10-11T09:49:34.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.575+0000 7f452a7fc700 1 -- 172.21.15.78:0/3055840013 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4524007480 con 0x7f4534146fe0 2023-10-11T09:49:34.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.575+0000 7f453a8b1700 1 -- 172.21.15.78:0/3055840013 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f453415a490 con 0x7f4534146fe0 2023-10-11T09:49:34.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.575+0000 7f453a8b1700 1 -- 172.21.15.78:0/3055840013 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f453415a620 con 0x7f4534146fe0 2023-10-11T09:49:34.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.579+0000 7f452a7fc700 1 -- 172.21.15.78:0/3055840013 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f45240075e0 con 0x7f4534146fe0 2023-10-11T09:49:34.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.579+0000 7f452a7fc700 1 --2- 172.21.15.78:0/3055840013 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4518065ed0 0x7f4518068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:34.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.579+0000 7f452a7fc700 1 -- 172.21.15.78:0/3055840013 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f4524082eb0 con 0x7f4534146fe0 2023-10-11T09:49:34.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.579+0000 7f45398af700 1 --2- 172.21.15.78:0/3055840013 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4518065ed0 0x7f4518068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:34.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.579+0000 7f45398af700 1 --2- 172.21.15.78:0/3055840013 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4518065ed0 0x7f4518068380 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f45300090b0 tx=0x7f4530008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:34.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.587+0000 7f452a7fc700 1 -- 172.21.15.78:0/3055840013 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f45240501c0 con 0x7f4534146fe0 2023-10-11T09:49:34.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.755+0000 7f453a8b1700 1 -- 172.21.15.78:0/3055840013 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4534065d40 con 0x7f4534146fe0 2023-10-11T09:49:34.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.759+0000 7f452a7fc700 1 -- 172.21.15.78:0/3055840013 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4524017020 con 0x7f4534146fe0 2023-10-11T09:49:34.765 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:49:34.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.759+0000 7f453a8b1700 1 -- 172.21.15.78:0/3055840013 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4518065ed0 msgr2=0x7f4518068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:34.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.759+0000 7f453a8b1700 1 --2- 172.21.15.78:0/3055840013 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4518065ed0 0x7f4518068380 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f45300090b0 tx=0x7f4530008030 comp rx=0 tx=0).stop 2023-10-11T09:49:34.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.759+0000 7f453a8b1700 1 -- 172.21.15.78:0/3055840013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4534146fe0 msgr2=0x7f45341431c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:34.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.759+0000 7f453a8b1700 1 --2- 172.21.15.78:0/3055840013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4534146fe0 0x7f45341431c0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f452400c4f0 tx=0x7f452400c8b0 comp rx=0 tx=0).stop 2023-10-11T09:49:34.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.763+0000 7f453a8b1700 1 -- 172.21.15.78:0/3055840013 shutdown_connections 2023-10-11T09:49:34.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.763+0000 7f453a8b1700 1 --2- 172.21.15.78:0/3055840013 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4518065ed0 0x7f4518068380 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:34.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.763+0000 7f453a8b1700 1 --2- 172.21.15.78:0/3055840013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4534161d90 0x7f45340b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:34.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.763+0000 7f453a8b1700 1 --2- 172.21.15.78:0/3055840013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4534146fe0 0x7f45341431c0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:34.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.763+0000 7f453a8b1700 1 --2- 172.21.15.78:0/3055840013 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f45340b36f0 0x7f453415ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:34.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.763+0000 7f453a8b1700 1 -- 172.21.15.78:0/3055840013 >> 172.21.15.78:0/3055840013 conn(0x7f453406a060 msgr2=0x7f45340ae610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:34.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.763+0000 7f453a8b1700 1 -- 172.21.15.78:0/3055840013 shutdown_connections 2023-10-11T09:49:34.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.763+0000 7f453a8b1700 1 -- 172.21.15.78:0/3055840013 wait complete. 2023-10-11T09:49:34.791 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T09:49:34.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.955+0000 7f6902ab0700 1 Processor -- start 2023-10-11T09:49:34.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.955+0000 7f6902ab0700 1 -- start start 2023-10-11T09:49:34.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.959+0000 7f6902ab0700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f68fc0b3240 0x7f68fc0d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:34.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.959+0000 7f6902ab0700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f68fc0b3610 0x7f68fc0b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:34.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.959+0000 7f6902ab0700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68fc0b47f0 con 0x7f68fc15d040 2023-10-11T09:49:34.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.959+0000 7f6902ab0700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68fc0b3fd0 con 0x7f68fc0b3240 2023-10-11T09:49:34.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.959+0000 7f6902ab0700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68fc0b8130 con 0x7f68fc0b3610 2023-10-11T09:49:34.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.959+0000 7f6901aae700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f68fc0b3240 0x7f68fc0d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:34.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.959+0000 7f6901aae700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f68fc0b3240 0x7f68fc0d29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38490/0 (socket says 172.21.15.78:38490) 2023-10-11T09:49:34.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.959+0000 7f6901aae700 1 -- 172.21.15.78:0/2100820766 learned_addr learned my addr 172.21.15.78:0/2100820766 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:34.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.959+0000 7f69012ad700 1 --2- 172.21.15.78:0/2100820766 >> v2:172.21.15.133:3300/0 conn(0x7f68fc0b3610 0x7f68fc0b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:34.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.959+0000 7f6901aae700 1 -- 172.21.15.78:0/2100820766 >> v2:172.21.15.133:3300/0 conn(0x7f68fc0b3610 msgr2=0x7f68fc0b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:34.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.959+0000 7f6901aae700 1 --2- 172.21.15.78:0/2100820766 >> v2:172.21.15.133:3300/0 conn(0x7f68fc0b3610 0x7f68fc0b3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:34.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.959+0000 7f6901aae700 1 -- 172.21.15.78:0/2100820766 >> v1:172.21.15.133:6789/0 conn(0x7f68fc15d040 legacy=0x7f68fc0b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:34.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.959+0000 7f6901aae700 1 -- 172.21.15.78:0/2100820766 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68fc0b41e0 con 0x7f68fc0b3240 2023-10-11T09:49:34.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.959+0000 7f6901aae700 1 --2- 172.21.15.78:0/2100820766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68fc0b3240 0x7f68fc0d29d0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f68fc0d0a90 tx=0x7f68f8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77bc4e90f8a4abfa server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:34.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.959+0000 7f6900aac700 1 -- 172.21.15.78:0/2100820766 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f8011830 con 0x7f68fc0b3240 2023-10-11T09:49:34.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.959+0000 7f6900aac700 1 -- 172.21.15.78:0/2100820766 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f8011990 con 0x7f68fc0b3240 2023-10-11T09:49:34.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.959+0000 7f6900aac700 1 -- 172.21.15.78:0/2100820766 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f8011d20 con 0x7f68fc0b3240 2023-10-11T09:49:34.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.963+0000 7f6902ab0700 1 -- 172.21.15.78:0/2100820766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68fc0b3240 msgr2=0x7f68fc0d29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:34.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.963+0000 7f6902ab0700 1 --2- 172.21.15.78:0/2100820766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68fc0b3240 0x7f68fc0d29d0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f68fc0d0a90 tx=0x7f68f8009bf0 comp rx=0 tx=0).stop 2023-10-11T09:49:34.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.963+0000 7f6902ab0700 1 -- 172.21.15.78:0/2100820766 shutdown_connections 2023-10-11T09:49:34.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.963+0000 7f6902ab0700 1 --2- 172.21.15.78:0/2100820766 >> v2:172.21.15.133:3300/0 conn(0x7f68fc0b3610 0x7f68fc0b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:34.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.963+0000 7f6902ab0700 1 --2- 172.21.15.78:0/2100820766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68fc0b3240 0x7f68fc0d29d0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:34.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.963+0000 7f6902ab0700 1 -- 172.21.15.78:0/2100820766 >> 172.21.15.78:0/2100820766 conn(0x7f68fc09c260 msgr2=0x7f68fc0d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:34.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.963+0000 7f6902ab0700 1 -- 172.21.15.78:0/2100820766 shutdown_connections 2023-10-11T09:49:34.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.963+0000 7f6902ab0700 1 -- 172.21.15.78:0/2100820766 wait complete. 2023-10-11T09:49:34.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.967+0000 7f6902ab0700 1 Processor -- start 2023-10-11T09:49:34.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.967+0000 7f6902ab0700 1 -- start start 2023-10-11T09:49:34.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.967+0000 7f6902ab0700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f68fc0b3240 0x7f68fc1f1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:34.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.967+0000 7f6902ab0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68fc0b3610 0x7f68fc0b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:34.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.967+0000 7f6902ab0700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68fc15d040 0x7f68fc0b2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:34.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.967+0000 7f6902ab0700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc09fdb0 con 0x7f68fc0b3610 2023-10-11T09:49:34.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.967+0000 7f6902ab0700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc0a0090 con 0x7f68fc15d040 2023-10-11T09:49:34.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.967+0000 7f6902ab0700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc0a0370 con 0x7f68fc0b3240 2023-10-11T09:49:34.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.967+0000 7f69012ad700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68fc0b3610 0x7f68fc0b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:34.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.967+0000 7f69012ad700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68fc0b3610 0x7f68fc0b1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38498/0 (socket says 172.21.15.78:38498) 2023-10-11T09:49:34.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.967+0000 7f69012ad700 1 -- 172.21.15.78:0/3222579559 learned_addr learned my addr 172.21.15.78:0/3222579559 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:34.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.967+0000 7f6901aae700 1 --2- 172.21.15.78:0/3222579559 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f68fc0b3240 0x7f68fc1f1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:34.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.971+0000 7f69022af700 1 --2- 172.21.15.78:0/3222579559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68fc15d040 0x7f68fc0b2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:34.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.971+0000 7f69012ad700 1 -- 172.21.15.78:0/3222579559 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f68fc0b3240 msgr2=0x7f68fc1f1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:34.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.971+0000 7f69012ad700 1 --2- 172.21.15.78:0/3222579559 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f68fc0b3240 0x7f68fc1f1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:34.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.971+0000 7f69012ad700 1 -- 172.21.15.78:0/3222579559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68fc15d040 msgr2=0x7f68fc0b2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:34.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.971+0000 7f69012ad700 1 --2- 172.21.15.78:0/3222579559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68fc15d040 0x7f68fc0b2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:34.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.971+0000 7f69012ad700 1 -- 172.21.15.78:0/3222579559 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f8009570 con 0x7f68fc0b3610 2023-10-11T09:49:34.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.971+0000 7f6901aae700 1 --2- 172.21.15.78:0/3222579559 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f68fc0b3240 0x7f68fc1f1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:49:34.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.971+0000 7f69012ad700 1 --2- 172.21.15.78:0/3222579559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68fc0b3610 0x7f68fc0b1d20 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f68ec00e840 tx=0x7f68ec00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:34.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.971+0000 7f68f27fc700 1 -- 172.21.15.78:0/3222579559 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ec0095a0 con 0x7f68fc0b3610 2023-10-11T09:49:34.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.971+0000 7f68f27fc700 1 -- 172.21.15.78:0/3222579559 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68ec013680 con 0x7f68fc0b3610 2023-10-11T09:49:34.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.971+0000 7f6902ab0700 1 -- 172.21.15.78:0/3222579559 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68fc0a0650 con 0x7f68fc0b3610 2023-10-11T09:49:34.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.971+0000 7f68f27fc700 1 -- 172.21.15.78:0/3222579559 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ec0139f0 con 0x7f68fc0b3610 2023-10-11T09:49:34.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.971+0000 7f6902ab0700 1 -- 172.21.15.78:0/3222579559 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68fc09faa0 con 0x7f68fc0b3610 2023-10-11T09:49:34.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.971+0000 7f6902ab0700 1 -- 172.21.15.78:0/3222579559 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68fc08b980 con 0x7f68fc0b3610 2023-10-11T09:49:34.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.975+0000 7f68f27fc700 1 -- 172.21.15.78:0/3222579559 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f68ec013b50 con 0x7f68fc0b3610 2023-10-11T09:49:34.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.975+0000 7f68f27fc700 1 --2- 172.21.15.78:0/3222579559 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f68e0065ed0 0x7f68e0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:34.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.975+0000 7f68f27fc700 1 -- 172.21.15.78:0/3222579559 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f68ec0838a0 con 0x7f68fc0b3610 2023-10-11T09:49:34.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.975+0000 7f6901aae700 1 --2- 172.21.15.78:0/3222579559 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f68e0065ed0 0x7f68e0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:34.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.979+0000 7f6901aae700 1 --2- 172.21.15.78:0/3222579559 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f68e0065ed0 0x7f68e0068380 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f68f8000c00 tx=0x7f68f8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:34.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:34.979+0000 7f68f27fc700 1 -- 172.21.15.78:0/3222579559 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f68ec050c40 con 0x7f68fc0b3610 2023-10-11T09:49:35.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.155+0000 7f6902ab0700 1 -- 172.21.15.78:0/3222579559 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f68fc0b8d60 con 0x7f68fc0b3610 2023-10-11T09:49:35.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.155+0000 7f68f27fc700 1 -- 172.21.15.78:0/3222579559 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f68ec017020 con 0x7f68fc0b3610 2023-10-11T09:49:35.162 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2023-10-11T09:49:35.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.159+0000 7f6902ab0700 1 -- 172.21.15.78:0/3222579559 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f68e0065ed0 msgr2=0x7f68e0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:35.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.159+0000 7f6902ab0700 1 --2- 172.21.15.78:0/3222579559 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f68e0065ed0 0x7f68e0068380 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f68f8000c00 tx=0x7f68f8008030 comp rx=0 tx=0).stop 2023-10-11T09:49:35.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.159+0000 7f6902ab0700 1 -- 172.21.15.78:0/3222579559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68fc0b3610 msgr2=0x7f68fc0b1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:35.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.159+0000 7f6902ab0700 1 --2- 172.21.15.78:0/3222579559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68fc0b3610 0x7f68fc0b1d20 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f68ec00e840 tx=0x7f68ec00eb50 comp rx=0 tx=0).stop 2023-10-11T09:49:35.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.159+0000 7f6902ab0700 1 -- 172.21.15.78:0/3222579559 shutdown_connections 2023-10-11T09:49:35.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.159+0000 7f6902ab0700 1 --2- 172.21.15.78:0/3222579559 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f68e0065ed0 0x7f68e0068380 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:35.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.159+0000 7f6902ab0700 1 --2- 172.21.15.78:0/3222579559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68fc15d040 0x7f68fc0b2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:35.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.159+0000 7f6902ab0700 1 --2- 172.21.15.78:0/3222579559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68fc0b3610 0x7f68fc0b1d20 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:35.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.159+0000 7f6902ab0700 1 --2- 172.21.15.78:0/3222579559 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f68fc0b3240 0x7f68fc1f1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:35.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.159+0000 7f6902ab0700 1 -- 172.21.15.78:0/3222579559 >> 172.21.15.78:0/3222579559 conn(0x7f68fc09c260 msgr2=0x7f68fc06c810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:35.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.159+0000 7f6902ab0700 1 -- 172.21.15.78:0/3222579559 shutdown_connections 2023-10-11T09:49:35.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.159+0000 7f6902ab0700 1 -- 172.21.15.78:0/3222579559 wait complete. 2023-10-11T09:49:35.189 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-10-11T09:49:35.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.359+0000 7f269eb30700 1 Processor -- start 2023-10-11T09:49:35.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.359+0000 7f269eb30700 1 -- start start 2023-10-11T09:49:35.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.363+0000 7f269eb30700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f26980b3610 0x7f26980d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:35.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.363+0000 7f269eb30700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f26980b3a90 0x7f26980b85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:35.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.363+0000 7f269eb30700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26980b47f0 con 0x7f26980b3240 2023-10-11T09:49:35.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.363+0000 7f269eb30700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26980b8130 con 0x7f26980b3a90 2023-10-11T09:49:35.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.363+0000 7f269eb30700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26980b3f10 con 0x7f26980b3610 2023-10-11T09:49:35.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.363+0000 7f269e32f700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f26980b3a90 0x7f26980b85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:35.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.363+0000 7f269e32f700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f26980b3a90 0x7f26980b85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38514/0 (socket says 172.21.15.78:38514) 2023-10-11T09:49:35.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.363+0000 7f269e32f700 1 -- 172.21.15.78:0/1153092721 learned_addr learned my addr 172.21.15.78:0/1153092721 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:35.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.363+0000 7f269d32d700 1 --2- 172.21.15.78:0/1153092721 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f26980b3610 0x7f26980d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:35.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.363+0000 7f269e32f700 1 -- 172.21.15.78:0/1153092721 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f26980b3610 msgr2=0x7f26980d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:35.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.363+0000 7f269e32f700 1 --2- 172.21.15.78:0/1153092721 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f26980b3610 0x7f26980d29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:35.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.363+0000 7f269e32f700 1 -- 172.21.15.78:0/1153092721 >> v1:172.21.15.133:6789/0 conn(0x7f26980b3240 legacy=0x7f269815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:35.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.363+0000 7f269e32f700 1 -- 172.21.15.78:0/1153092721 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26980b41e0 con 0x7f26980b3a90 2023-10-11T09:49:35.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.363+0000 7f269e32f700 1 --2- 172.21.15.78:0/1153092721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26980b3a90 0x7f26980b85c0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f2690009e50 tx=0x7f269000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad805252d13b0427 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:35.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.363+0000 7f269cb2c700 1 -- 172.21.15.78:0/1153092721 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269000e070 con 0x7f26980b3a90 2023-10-11T09:49:35.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.363+0000 7f269cb2c700 1 -- 172.21.15.78:0/1153092721 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f269000b910 con 0x7f26980b3a90 2023-10-11T09:49:35.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.363+0000 7f269cb2c700 1 -- 172.21.15.78:0/1153092721 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269000bcc0 con 0x7f26980b3a90 2023-10-11T09:49:35.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.367+0000 7f269eb30700 1 -- 172.21.15.78:0/1153092721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26980b3a90 msgr2=0x7f26980b85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:35.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.367+0000 7f269eb30700 1 --2- 172.21.15.78:0/1153092721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26980b3a90 0x7f26980b85c0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f2690009e50 tx=0x7f269000b330 comp rx=0 tx=0).stop 2023-10-11T09:49:35.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.367+0000 7f269eb30700 1 -- 172.21.15.78:0/1153092721 shutdown_connections 2023-10-11T09:49:35.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.367+0000 7f269eb30700 1 --2- 172.21.15.78:0/1153092721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26980b3a90 0x7f26980b85c0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:35.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.367+0000 7f269eb30700 1 --2- 172.21.15.78:0/1153092721 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f26980b3610 0x7f26980d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:35.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.367+0000 7f269eb30700 1 -- 172.21.15.78:0/1153092721 >> 172.21.15.78:0/1153092721 conn(0x7f269809c260 msgr2=0x7f26980d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:35.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.367+0000 7f269eb30700 1 -- 172.21.15.78:0/1153092721 shutdown_connections 2023-10-11T09:49:35.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.367+0000 7f269eb30700 1 -- 172.21.15.78:0/1153092721 wait complete. 2023-10-11T09:49:35.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.371+0000 7f269eb30700 1 Processor -- start 2023-10-11T09:49:35.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.371+0000 7f269eb30700 1 -- start start 2023-10-11T09:49:35.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.371+0000 7f269eb30700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26980b3240 0x7f269815f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:35.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.371+0000 7f269eb30700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26980b3610 0x7f2698159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:35.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.371+0000 7f269eb30700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f26980b3a90 0x7f269815e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:35.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.371+0000 7f269eb30700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269815eb70 con 0x7f26980b3240 2023-10-11T09:49:35.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.371+0000 7f269eb30700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269815fce0 con 0x7f26980b3610 2023-10-11T09:49:35.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.371+0000 7f269eb30700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f269815a360 con 0x7f26980b3a90 2023-10-11T09:49:35.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.371+0000 7f269db2e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26980b3240 0x7f269815f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:35.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.371+0000 7f269db2e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26980b3240 0x7f269815f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38518/0 (socket says 172.21.15.78:38518) 2023-10-11T09:49:35.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.371+0000 7f269db2e700 1 -- 172.21.15.78:0/464602158 learned_addr learned my addr 172.21.15.78:0/464602158 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:35.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.371+0000 7f269d32d700 1 --2- 172.21.15.78:0/464602158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26980b3610 0x7f2698159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:35.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.371+0000 7f269e32f700 1 --2- 172.21.15.78:0/464602158 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f26980b3a90 0x7f269815e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:35.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.371+0000 7f269db2e700 1 -- 172.21.15.78:0/464602158 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f26980b3a90 msgr2=0x7f269815e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:35.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.371+0000 7f269db2e700 1 --2- 172.21.15.78:0/464602158 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f26980b3a90 0x7f269815e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:35.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.375+0000 7f269db2e700 1 -- 172.21.15.78:0/464602158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26980b3610 msgr2=0x7f2698159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:35.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.375+0000 7f269db2e700 1 --2- 172.21.15.78:0/464602158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26980b3610 0x7f2698159e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:35.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.375+0000 7f269db2e700 1 -- 172.21.15.78:0/464602158 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2690009b30 con 0x7f26980b3240 2023-10-11T09:49:35.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.375+0000 7f269db2e700 1 --2- 172.21.15.78:0/464602158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26980b3240 0x7f269815f7a0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f269400c500 tx=0x7f269400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:35.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.375+0000 7f268e7fc700 1 -- 172.21.15.78:0/464602158 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26940052d0 con 0x7f26980b3240 2023-10-11T09:49:35.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.375+0000 7f268e7fc700 1 -- 172.21.15.78:0/464602158 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2694005430 con 0x7f26980b3240 2023-10-11T09:49:35.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.375+0000 7f268e7fc700 1 -- 172.21.15.78:0/464602158 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26940055a0 con 0x7f26980b3240 2023-10-11T09:49:35.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.375+0000 7f269eb30700 1 -- 172.21.15.78:0/464602158 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f269815e8d0 con 0x7f26980b3240 2023-10-11T09:49:35.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.375+0000 7f269eb30700 1 -- 172.21.15.78:0/464602158 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26980c7150 con 0x7f26980b3240 2023-10-11T09:49:35.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.375+0000 7f269eb30700 1 -- 172.21.15.78:0/464602158 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2698065d40 con 0x7f26980b3240 2023-10-11T09:49:35.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.379+0000 7f268e7fc700 1 -- 172.21.15.78:0/464602158 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f26940038e0 con 0x7f26980b3240 2023-10-11T09:49:35.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.379+0000 7f268e7fc700 1 --2- 172.21.15.78:0/464602158 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f267c065ed0 0x7f267c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:35.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.379+0000 7f268e7fc700 1 -- 172.21.15.78:0/464602158 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f2694082cb0 con 0x7f26980b3240 2023-10-11T09:49:35.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.379+0000 7f269d32d700 1 --2- 172.21.15.78:0/464602158 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f267c065ed0 0x7f267c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:35.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.379+0000 7f269d32d700 1 --2- 172.21.15.78:0/464602158 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f267c065ed0 0x7f267c068380 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f269815c1d0 tx=0x7f2688009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:35.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.387+0000 7f268e7fc700 1 -- 172.21.15.78:0/464602158 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f269404ffc0 con 0x7f26980b3240 2023-10-11T09:49:35.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.555+0000 7f269eb30700 1 -- 172.21.15.78:0/464602158 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7f269815fe30 con 0x7f26980b3240 2023-10-11T09:49:35.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.559+0000 7f268e7fc700 1 -- 172.21.15.78:0/464602158 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v92) v1 ==== 106+0+11 (secure 0 0 0) 0x7f26940535e0 con 0x7f26980b3240 2023-10-11T09:49:35.564 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:49:35.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.559+0000 7f269eb30700 1 -- 172.21.15.78:0/464602158 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f267c065ed0 msgr2=0x7f267c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:35.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.559+0000 7f269eb30700 1 --2- 172.21.15.78:0/464602158 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f267c065ed0 0x7f267c068380 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f269815c1d0 tx=0x7f2688009240 comp rx=0 tx=0).stop 2023-10-11T09:49:35.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.559+0000 7f269eb30700 1 -- 172.21.15.78:0/464602158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26980b3240 msgr2=0x7f269815f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:35.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.559+0000 7f269eb30700 1 --2- 172.21.15.78:0/464602158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26980b3240 0x7f269815f7a0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f269400c500 tx=0x7f269400c810 comp rx=0 tx=0).stop 2023-10-11T09:49:35.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.559+0000 7f269eb30700 1 -- 172.21.15.78:0/464602158 shutdown_connections 2023-10-11T09:49:35.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.559+0000 7f269eb30700 1 --2- 172.21.15.78:0/464602158 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f267c065ed0 0x7f267c068380 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:35.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.559+0000 7f269eb30700 1 --2- 172.21.15.78:0/464602158 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f26980b3a90 0x7f269815e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:35.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.559+0000 7f269eb30700 1 --2- 172.21.15.78:0/464602158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26980b3610 0x7f2698159e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:35.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.559+0000 7f269eb30700 1 --2- 172.21.15.78:0/464602158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26980b3240 0x7f269815f7a0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:35.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.559+0000 7f269eb30700 1 -- 172.21.15.78:0/464602158 >> 172.21.15.78:0/464602158 conn(0x7f269809c260 msgr2=0x7f26980afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:35.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.559+0000 7f269eb30700 1 -- 172.21.15.78:0/464602158 shutdown_connections 2023-10-11T09:49:35.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.563+0000 7f269eb30700 1 -- 172.21.15.78:0/464602158 wait complete. 2023-10-11T09:49:35.587 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:49:35.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.751+0000 7ff5dad13700 1 Processor -- start 2023-10-11T09:49:35.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.751+0000 7ff5dad13700 1 -- start start 2023-10-11T09:49:35.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.751+0000 7ff5dad13700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ff5d4071040 0x7ff5d40cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:35.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.751+0000 7ff5dad13700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5d40720f0 con 0x7ff5d4154e00 2023-10-11T09:49:35.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.751+0000 7ff5dad13700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5d40714e0 con 0x7ff5d40cba00 2023-10-11T09:49:35.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.751+0000 7ff5dad13700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5d40717c0 con 0x7ff5d4071040 2023-10-11T09:49:35.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.751+0000 7ff5d8aaf700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7ff5d4154e00 0x7ff5d415b650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:49480/0 (socket says 172.21.15.78:49480) 2023-10-11T09:49:35.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.751+0000 7ff5d8aaf700 1 -- 172.21.15.78:0/3368845697 learned_addr learned my addr 172.21.15.78:0/3368845697 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:49:35.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.751+0000 7ff5cbfff700 1 --2- 172.21.15.78:0/3368845697 >> v2:172.21.15.78:3300/0 conn(0x7ff5d4071040 0x7ff5d40cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:35.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.751+0000 7ff5cb7fe700 1 -- 172.21.15.78:0/3368845697 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3756159067 0 0) 0x7ff5d40720f0 con 0x7ff5d4154e00 2023-10-11T09:49:35.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.751+0000 7ff5cbfff700 1 -- 172.21.15.78:0/3368845697 >> v1:172.21.15.133:6789/0 conn(0x7ff5d40cba00 legacy=0x7ff5d406c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:35.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.751+0000 7ff5cbfff700 1 -- 172.21.15.78:0/3368845697 >> v1:172.21.15.78:6789/0 conn(0x7ff5d4154e00 legacy=0x7ff5d415b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:35.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.755+0000 7ff5cbfff700 1 -- 172.21.15.78:0/3368845697 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d4071aa0 con 0x7ff5d4071040 2023-10-11T09:49:35.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.755+0000 7ff5cbfff700 1 --2- 172.21.15.78:0/3368845697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5d4071040 0x7ff5d40cb4c0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4009850 tx=0x7ff5c4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57dca6174ec76a04 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:35.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.755+0000 7ff5cb7fe700 1 -- 172.21.15.78:0/3368845697 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4011830 con 0x7ff5d4071040 2023-10-11T09:49:35.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.755+0000 7ff5cb7fe700 1 -- 172.21.15.78:0/3368845697 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c4011990 con 0x7ff5d4071040 2023-10-11T09:49:35.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.755+0000 7ff5cb7fe700 1 -- 172.21.15.78:0/3368845697 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4011d20 con 0x7ff5d4071040 2023-10-11T09:49:35.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.755+0000 7ff5dad13700 1 -- 172.21.15.78:0/3368845697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5d4071040 msgr2=0x7ff5d40cb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:35.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.755+0000 7ff5dad13700 1 --2- 172.21.15.78:0/3368845697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5d4071040 0x7ff5d40cb4c0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4009850 tx=0x7ff5c4009b60 comp rx=0 tx=0).stop 2023-10-11T09:49:35.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.755+0000 7ff5dad13700 1 -- 172.21.15.78:0/3368845697 shutdown_connections 2023-10-11T09:49:35.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.755+0000 7ff5dad13700 1 --2- 172.21.15.78:0/3368845697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5d4071040 0x7ff5d40cb4c0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:35.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.755+0000 7ff5dad13700 1 -- 172.21.15.78:0/3368845697 >> 172.21.15.78:0/3368845697 conn(0x7ff5d406aca0 msgr2=0x7ff5d40b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:35.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.755+0000 7ff5dad13700 1 -- 172.21.15.78:0/3368845697 shutdown_connections 2023-10-11T09:49:35.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.755+0000 7ff5dad13700 1 -- 172.21.15.78:0/3368845697 wait complete. 2023-10-11T09:49:35.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.759+0000 7ff5dad13700 1 Processor -- start 2023-10-11T09:49:35.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.759+0000 7ff5dad13700 1 -- start start 2023-10-11T09:49:35.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.759+0000 7ff5dad13700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff5d4071040 0x7ff5d414c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:35.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.759+0000 7ff5dad13700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5d40cba00 0x7ff5d415ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:35.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.759+0000 7ff5dad13700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5d4154e00 0x7ff5d415e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:35.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.759+0000 7ff5dad13700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d415c840 con 0x7ff5d4154e00 2023-10-11T09:49:35.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.759+0000 7ff5dad13700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d414cf00 con 0x7ff5d40cba00 2023-10-11T09:49:35.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.759+0000 7ff5dad13700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d406c990 con 0x7ff5d4071040 2023-10-11T09:49:35.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.759+0000 7ff5d92b0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5d4154e00 0x7ff5d415e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:35.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.759+0000 7ff5d92b0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5d4154e00 0x7ff5d415e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38548/0 (socket says 172.21.15.78:38548) 2023-10-11T09:49:35.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.759+0000 7ff5d92b0700 1 -- 172.21.15.78:0/4285041073 learned_addr learned my addr 172.21.15.78:0/4285041073 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:35.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.759+0000 7ff5d8aaf700 1 --2- 172.21.15.78:0/4285041073 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff5d4071040 0x7ff5d414c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:35.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.759+0000 7ff5cbfff700 1 --2- 172.21.15.78:0/4285041073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5d40cba00 0x7ff5d415ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:35.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.763+0000 7ff5d92b0700 1 -- 172.21.15.78:0/4285041073 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff5d4071040 msgr2=0x7ff5d414c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:35.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.763+0000 7ff5d92b0700 1 --2- 172.21.15.78:0/4285041073 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff5d4071040 0x7ff5d414c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:35.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.763+0000 7ff5d92b0700 1 -- 172.21.15.78:0/4285041073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5d40cba00 msgr2=0x7ff5d415ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:35.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.763+0000 7ff5d92b0700 1 --2- 172.21.15.78:0/4285041073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5d40cba00 0x7ff5d415ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:35.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.763+0000 7ff5d92b0700 1 -- 172.21.15.78:0/4285041073 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c40094b0 con 0x7ff5d4154e00 2023-10-11T09:49:35.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.763+0000 7ff5d8aaf700 1 --2- 172.21.15.78:0/4285041073 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff5d4071040 0x7ff5d414c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:49:35.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.763+0000 7ff5d92b0700 1 --2- 172.21.15.78:0/4285041073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5d4154e00 0x7ff5d415e190 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc00cc80 tx=0x7ff5cc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:35.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.763+0000 7ff5c97fa700 1 -- 172.21.15.78:0/4285041073 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc00ad40 con 0x7ff5d4154e00 2023-10-11T09:49:35.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.763+0000 7ff5c97fa700 1 -- 172.21.15.78:0/4285041073 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5cc00ce80 con 0x7ff5d4154e00 2023-10-11T09:49:35.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.763+0000 7ff5c97fa700 1 -- 172.21.15.78:0/4285041073 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc00aea0 con 0x7ff5d4154e00 2023-10-11T09:49:35.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.763+0000 7ff5dad13700 1 -- 172.21.15.78:0/4285041073 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5d406cc70 con 0x7ff5d4154e00 2023-10-11T09:49:35.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.763+0000 7ff5dad13700 1 -- 172.21.15.78:0/4285041073 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5d406d230 con 0x7ff5d4154e00 2023-10-11T09:49:35.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.763+0000 7ff5dad13700 1 -- 172.21.15.78:0/4285041073 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5d404e530 con 0x7ff5d4154e00 2023-10-11T09:49:35.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.763+0000 7ff5c97fa700 1 -- 172.21.15.78:0/4285041073 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7ff5cc003c60 con 0x7ff5d4154e00 2023-10-11T09:49:35.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.767+0000 7ff5c97fa700 1 --2- 172.21.15.78:0/4285041073 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff5c0065ed0 0x7ff5c0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:35.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.767+0000 7ff5c97fa700 1 -- 172.21.15.78:0/4285041073 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6724+0+0 (secure 0 0 0) 0x7ff5cc083b80 con 0x7ff5d4154e00 2023-10-11T09:49:35.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.767+0000 7ff5d8aaf700 1 --2- 172.21.15.78:0/4285041073 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff5c0065ed0 0x7ff5c0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:35.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.767+0000 7ff5d8aaf700 1 --2- 172.21.15.78:0/4285041073 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff5c0065ed0 0x7ff5c0068380 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0000f60 tx=0x7ff5d0007e70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:35.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.775+0000 7ff5c97fa700 1 -- 172.21.15.78:0/4285041073 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7ff5cc050f10 con 0x7ff5d4154e00 2023-10-11T09:49:35.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.987+0000 7ff5dad13700 1 -- 172.21.15.78:0/4285041073 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5d406e1e0 con 0x7ff5d4154e00 2023-10-11T09:49:35.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.987+0000 7ff5c97fa700 1 -- 172.21.15.78:0/4285041073 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4657 (secure 0 0 0) 0x7ff5cc054530 con 0x7ff5d4154e00 2023-10-11T09:49:35.995 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:49:35.996 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":71,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6581,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/238181631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":238181631},{"type":"v1","addr":"172.21.15.78:6837","nonce":238181631}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":7378,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/2416080754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2416080754},{"type":"v1","addr":"172.21.15.133:6834","nonce":2416080754}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":7384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/3016626706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3016626706},{"type":"v1","addr":"172.21.15.78:6836","nonce":3016626706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:48:45.351822+0000","modified":"2023-10-11T09:48:53.854831+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7725":{"gid":7725,"name":"b","rank":0,"incarnation":68,"state":"up:active","state_seq":4,"addr":"172.21.15.133:6835/3922016223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":3922016223},{"type":"v1","addr":"172.21.15.133:6835","nonce":3922016223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12,13],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-11T09:49:35.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.991+0000 7ff5dad13700 1 -- 172.21.15.78:0/4285041073 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff5c0065ed0 msgr2=0x7ff5c0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:35.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.991+0000 7ff5dad13700 1 --2- 172.21.15.78:0/4285041073 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff5c0065ed0 0x7ff5c0068380 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0000f60 tx=0x7ff5d0007e70 comp rx=0 tx=0).stop 2023-10-11T09:49:35.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.991+0000 7ff5dad13700 1 -- 172.21.15.78:0/4285041073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5d4154e00 msgr2=0x7ff5d415e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:35.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.991+0000 7ff5dad13700 1 --2- 172.21.15.78:0/4285041073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5d4154e00 0x7ff5d415e190 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc00cc80 tx=0x7ff5cc00a340 comp rx=0 tx=0).stop 2023-10-11T09:49:36.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.991+0000 7ff5dad13700 1 -- 172.21.15.78:0/4285041073 shutdown_connections 2023-10-11T09:49:36.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.991+0000 7ff5dad13700 1 --2- 172.21.15.78:0/4285041073 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff5c0065ed0 0x7ff5c0068380 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:36.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.991+0000 7ff5dad13700 1 --2- 172.21.15.78:0/4285041073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5d4154e00 0x7ff5d415e190 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:36.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.991+0000 7ff5dad13700 1 --2- 172.21.15.78:0/4285041073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5d40cba00 0x7ff5d415ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:36.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.991+0000 7ff5dad13700 1 --2- 172.21.15.78:0/4285041073 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff5d4071040 0x7ff5d414c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:36.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.991+0000 7ff5dad13700 1 -- 172.21.15.78:0/4285041073 >> 172.21.15.78:0/4285041073 conn(0x7ff5d406aca0 msgr2=0x7ff5d40aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:36.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.991+0000 7ff5dad13700 1 -- 172.21.15.78:0/4285041073 shutdown_connections 2023-10-11T09:49:36.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:35.991+0000 7ff5dad13700 1 -- 172.21.15.78:0/4285041073 wait complete. 2023-10-11T09:49:36.002 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 71 2023-10-11T09:49:36.020 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:49:36.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.183+0000 7f5ecf59e700 1 Processor -- start 2023-10-11T09:49:36.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.183+0000 7f5ecf59e700 1 -- start start 2023-10-11T09:49:36.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.187+0000 7f5ecf59e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5ed00cda10 0x7f5ed00caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:36.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.187+0000 7f5ecf59e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f5ed0164d20 0x7f5ed006bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:36.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.187+0000 7f5ecf59e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed0165390 con 0x7f5ed00cdde0 2023-10-11T09:49:36.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.187+0000 7f5ecf59e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ed01651a0 con 0x7f5ed0164d20 2023-10-11T09:49:36.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.187+0000 7f5ecf59e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ed00ca440 con 0x7f5ed00cda10 2023-10-11T09:49:36.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.187+0000 7f5ec5d9b700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f5ed00cdde0 0x7f5ed0164610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:49486/0 (socket says 172.21.15.78:49486) 2023-10-11T09:49:36.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.187+0000 7f5ec5d9b700 1 -- 172.21.15.78:0/3185295704 learned_addr learned my addr 172.21.15.78:0/3185295704 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:49:36.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.187+0000 7f5eced9d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f5ed0164d20 0x7f5ed006bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:36.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.187+0000 7f5ece59c700 1 --2- 172.21.15.78:0/3185295704 >> v2:172.21.15.133:3300/0 conn(0x7f5ed00cda10 0x7f5ed00caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:36.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.187+0000 7f5ecdd9b700 1 -- 172.21.15.78:0/3185295704 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4248146693 0 0) 0x7f5ed0165390 con 0x7f5ed00cdde0 2023-10-11T09:49:36.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.191+0000 7f5ecdd9b700 1 -- 172.21.15.78:0/3185295704 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ebc0035d0 con 0x7f5ed00cdde0 2023-10-11T09:49:36.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.191+0000 7f5eced9d700 1 -- 172.21.15.78:0/3185295704 >> v2:172.21.15.133:3300/0 conn(0x7f5ed00cda10 msgr2=0x7f5ed00caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:36.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.191+0000 7f5eced9d700 1 --2- 172.21.15.78:0/3185295704 >> v2:172.21.15.133:3300/0 conn(0x7f5ed00cda10 0x7f5ed00caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:36.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.191+0000 7f5eced9d700 1 -- 172.21.15.78:0/3185295704 >> v1:172.21.15.78:6789/0 conn(0x7f5ed00cdde0 legacy=0x7f5ed0164610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:36.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.191+0000 7f5eced9d700 1 -- 172.21.15.78:0/3185295704 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed00ca720 con 0x7f5ed0164d20 2023-10-11T09:49:36.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.191+0000 7f5ece59c700 1 --2- 172.21.15.78:0/3185295704 >> v2:172.21.15.133:3300/0 conn(0x7f5ed00cda10 0x7f5ed00caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:49:36.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.191+0000 7f5eced9d700 1 --2- 172.21.15.78:0/3185295704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ed0164d20 0x7f5ed006bbf0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f5ed006afb0 tx=0x7f5ec800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f88adc98e70fc64 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:36.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.191+0000 7f5ecdd9b700 1 -- 172.21.15.78:0/3185295704 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec800e070 con 0x7f5ed0164d20 2023-10-11T09:49:36.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.191+0000 7f5ecdd9b700 1 -- 172.21.15.78:0/3185295704 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec800bb20 con 0x7f5ed0164d20 2023-10-11T09:49:36.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.191+0000 7f5ecdd9b700 1 -- 172.21.15.78:0/3185295704 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec8012430 con 0x7f5ed0164d20 2023-10-11T09:49:36.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.191+0000 7f5ecf59e700 1 -- 172.21.15.78:0/3185295704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ed0164d20 msgr2=0x7f5ed006bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:36.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.191+0000 7f5ecf59e700 1 --2- 172.21.15.78:0/3185295704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ed0164d20 0x7f5ed006bbf0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f5ed006afb0 tx=0x7f5ec800b330 comp rx=0 tx=0).stop 2023-10-11T09:49:36.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.191+0000 7f5ecf59e700 1 -- 172.21.15.78:0/3185295704 shutdown_connections 2023-10-11T09:49:36.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.191+0000 7f5ecf59e700 1 --2- 172.21.15.78:0/3185295704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ed0164d20 0x7f5ed006bbf0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:36.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.191+0000 7f5ecf59e700 1 --2- 172.21.15.78:0/3185295704 >> v2:172.21.15.133:3300/0 conn(0x7f5ed00cda10 0x7f5ed00caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:36.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.191+0000 7f5ecf59e700 1 -- 172.21.15.78:0/3185295704 >> 172.21.15.78:0/3185295704 conn(0x7f5ed00cea60 msgr2=0x7f5ed00cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:36.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.191+0000 7f5ecf59e700 1 -- 172.21.15.78:0/3185295704 shutdown_connections 2023-10-11T09:49:36.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.191+0000 7f5ecf59e700 1 -- 172.21.15.78:0/3185295704 wait complete. 2023-10-11T09:49:36.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.195+0000 7f5ecf59e700 1 Processor -- start 2023-10-11T09:49:36.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.195+0000 7f5ecf59e700 1 -- start start 2023-10-11T09:49:36.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.195+0000 7f5ecf59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ed00cda10 0x7f5ed00b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:36.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.195+0000 7f5ecf59e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5ed00cdde0 0x7f5ed0162760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:36.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.195+0000 7f5ecf59e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ed0164d20 0x7f5ed0162ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:36.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.195+0000 7f5ecf59e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed00b3150 con 0x7f5ed00cda10 2023-10-11T09:49:36.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.195+0000 7f5ecf59e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed00b24d0 con 0x7f5ed0164d20 2023-10-11T09:49:36.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.195+0000 7f5ecf59e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed00b27b0 con 0x7f5ed00cdde0 2023-10-11T09:49:36.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.195+0000 7f5ece59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ed00cda10 0x7f5ed00b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:36.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.195+0000 7f5ece59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ed00cda10 0x7f5ed00b1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38554/0 (socket says 172.21.15.78:38554) 2023-10-11T09:49:36.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.195+0000 7f5ece59c700 1 -- 172.21.15.78:0/2997278684 learned_addr learned my addr 172.21.15.78:0/2997278684 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:36.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.195+0000 7f5eced9d700 1 --2- 172.21.15.78:0/2997278684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ed0164d20 0x7f5ed0162ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:36.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.199+0000 7f5ec5d9b700 1 --2- 172.21.15.78:0/2997278684 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5ed00cdde0 0x7f5ed0162760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:36.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.199+0000 7f5ece59c700 1 -- 172.21.15.78:0/2997278684 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5ed00cdde0 msgr2=0x7f5ed0162760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:36.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.199+0000 7f5ece59c700 1 --2- 172.21.15.78:0/2997278684 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5ed00cdde0 0x7f5ed0162760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:36.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.199+0000 7f5ece59c700 1 -- 172.21.15.78:0/2997278684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ed0164d20 msgr2=0x7f5ed0162ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:36.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.199+0000 7f5ece59c700 1 --2- 172.21.15.78:0/2997278684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ed0164d20 0x7f5ed0162ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:36.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.199+0000 7f5ece59c700 1 -- 172.21.15.78:0/2997278684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec00094b0 con 0x7f5ed00cda10 2023-10-11T09:49:36.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.199+0000 7f5ec5d9b700 1 --2- 172.21.15.78:0/2997278684 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5ed00cdde0 0x7f5ed0162760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:49:36.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.199+0000 7f5ece59c700 1 --2- 172.21.15.78:0/2997278684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ed00cda10 0x7f5ed00b1a10 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000ebe0 tx=0x7f5ec0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:36.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.199+0000 7f5ec77fe700 1 -- 172.21.15.78:0/2997278684 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec000c8b0 con 0x7f5ed00cda10 2023-10-11T09:49:36.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.199+0000 7f5ec77fe700 1 -- 172.21.15.78:0/2997278684 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec000ca10 con 0x7f5ed00cda10 2023-10-11T09:49:36.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.199+0000 7f5ecf59e700 1 -- 172.21.15.78:0/2997278684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ec8009a70 con 0x7f5ed00cda10 2023-10-11T09:49:36.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.199+0000 7f5ec77fe700 1 -- 172.21.15.78:0/2997278684 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec000cda0 con 0x7f5ed00cda10 2023-10-11T09:49:36.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.199+0000 7f5ecf59e700 1 -- 172.21.15.78:0/2997278684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ed00b2d70 con 0x7f5ed00cda10 2023-10-11T09:49:36.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.199+0000 7f5ecf59e700 1 -- 172.21.15.78:0/2997278684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ed0065d40 con 0x7f5ed00cda10 2023-10-11T09:49:36.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.203+0000 7f5ec77fe700 1 -- 172.21.15.78:0/2997278684 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f5ec0013470 con 0x7f5ed00cda10 2023-10-11T09:49:36.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.203+0000 7f5ec77fe700 1 --2- 172.21.15.78:0/2997278684 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5eb0065ed0 0x7f5eb0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:36.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.203+0000 7f5ec77fe700 1 -- 172.21.15.78:0/2997278684 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f5ec0083c60 con 0x7f5ed00cda10 2023-10-11T09:49:36.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.203+0000 7f5ec5d9b700 1 --2- 172.21.15.78:0/2997278684 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5eb0065ed0 0x7f5eb0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:36.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.203+0000 7f5ec5d9b700 1 --2- 172.21.15.78:0/2997278684 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5eb0065ed0 0x7f5eb0068380 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5ed013fc80 tx=0x7f5eb8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:36.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.211+0000 7f5ec77fe700 1 -- 172.21.15.78:0/2997278684 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f5ec0050f70 con 0x7f5ed00cda10 2023-10-11T09:49:36.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.391+0000 7f5ecf59e700 1 -- 172.21.15.78:0/2997278684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ed00b1480 con 0x7f5ed00cda10 2023-10-11T09:49:36.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.395+0000 7f5ec77fe700 1 -- 172.21.15.78:0/2997278684 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+20058 (secure 0 0 0) 0x7f5ec0017090 con 0x7f5ed00cda10 2023-10-11T09:49:36.402 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:49:36.402 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":92,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:48:50.804275+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:48:18.491556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-11T09:48:19.018937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data_extra","create_time":"2023-10-11T09:48:28.963956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6834/1560261029":"2023-10-12T09:48:50.804239+0000","172.21.15.78:6836/1560261029":"2023-10-12T09:48:50.804239+0000","172.21.15.78:6836/3463904224":"2023-10-12T09:48:45.480946+0000","172.21.15.78:6834/3463904224":"2023-10-12T09:48:45.480946+0000","172.21.15.78:6837/951312831":"2023-10-12T09:48:44.433517+0000","172.21.15.78:6835/951312831":"2023-10-12T09:48:44.433517+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:49:36.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.399+0000 7f5ecf59e700 1 -- 172.21.15.78:0/2997278684 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5eb0065ed0 msgr2=0x7f5eb0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:36.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.399+0000 7f5ecf59e700 1 --2- 172.21.15.78:0/2997278684 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5eb0065ed0 0x7f5eb0068380 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5ed013fc80 tx=0x7f5eb8009040 comp rx=0 tx=0).stop 2023-10-11T09:49:36.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.399+0000 7f5ecf59e700 1 -- 172.21.15.78:0/2997278684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ed00cda10 msgr2=0x7f5ed00b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:36.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.399+0000 7f5ecf59e700 1 --2- 172.21.15.78:0/2997278684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ed00cda10 0x7f5ed00b1a10 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000ebe0 tx=0x7f5ec0009e30 comp rx=0 tx=0).stop 2023-10-11T09:49:36.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.399+0000 7f5ecf59e700 1 -- 172.21.15.78:0/2997278684 shutdown_connections 2023-10-11T09:49:36.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.399+0000 7f5ecf59e700 1 --2- 172.21.15.78:0/2997278684 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5eb0065ed0 0x7f5eb0068380 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:36.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.399+0000 7f5ecf59e700 1 --2- 172.21.15.78:0/2997278684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ed0164d20 0x7f5ed0162ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:36.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.399+0000 7f5ecf59e700 1 --2- 172.21.15.78:0/2997278684 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5ed00cdde0 0x7f5ed0162760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:36.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.399+0000 7f5ecf59e700 1 --2- 172.21.15.78:0/2997278684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ed00cda10 0x7f5ed00b1a10 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:36.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.399+0000 7f5ecf59e700 1 -- 172.21.15.78:0/2997278684 >> 172.21.15.78:0/2997278684 conn(0x7f5ed00cea60 msgr2=0x7f5ed006bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:36.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.399+0000 7f5ecf59e700 1 -- 172.21.15.78:0/2997278684 shutdown_connections 2023-10-11T09:49:36.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.399+0000 7f5ecf59e700 1 -- 172.21.15.78:0/2997278684 wait complete. 2023-10-11T09:49:36.429 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-11T09:49:36.429 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:49:36.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.587+0000 7f62c2a39700 1 Processor -- start 2023-10-11T09:49:36.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.587+0000 7f62c2a39700 1 -- start start 2023-10-11T09:49:36.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.591+0000 7f62c2a39700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62bc0caf20 0x7f62bc164610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:36.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.591+0000 7f62c2a39700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f62bc164b50 0x7f62bc06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:36.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.591+0000 7f62c2a39700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62bc165390 con 0x7f62bc0cda10 2023-10-11T09:49:36.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.591+0000 7f62c2a39700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62bc0cb370 con 0x7f62bc164b50 2023-10-11T09:49:36.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.591+0000 7f62c2a39700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc0ca440 con 0x7f62bc0caf20 2023-10-11T09:49:36.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.591+0000 7f62c1a37700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f62bc0cda10 0x7f62bc0cdde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:49502/0 (socket says 172.21.15.78:49502) 2023-10-11T09:49:36.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.591+0000 7f62c1a37700 1 -- 172.21.15.78:0/3954602770 learned_addr learned my addr 172.21.15.78:0/3954602770 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:49:36.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.591+0000 7f62c1236700 1 --2- 172.21.15.78:0/3954602770 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62bc0caf20 0x7f62bc164610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:36.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.591+0000 7f62c2238700 1 --2- 172.21.15.78:0/3954602770 >> v2:172.21.15.133:3300/0 conn(0x7f62bc164b50 0x7f62bc06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:36.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.591+0000 7f62c0a35700 1 -- 172.21.15.78:0/3954602770 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1665302889 0 0) 0x7f62bc165390 con 0x7f62bc0cda10 2023-10-11T09:49:36.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.591+0000 7f62c0a35700 1 -- 172.21.15.78:0/3954602770 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62a80035d0 con 0x7f62bc0cda10 2023-10-11T09:49:36.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.595+0000 7f62c0a35700 1 -- 172.21.15.78:0/3954602770 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 654249854 0 0) 0x7f62a80035d0 con 0x7f62bc0cda10 2023-10-11T09:49:36.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.595+0000 7f62c0a35700 1 -- 172.21.15.78:0/3954602770 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62bc0caf20 msgr2=0x7f62bc164610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:36.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.595+0000 7f62c0a35700 1 --2- 172.21.15.78:0/3954602770 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62bc0caf20 0x7f62bc164610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:36.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.595+0000 7f62c0a35700 1 -- 172.21.15.78:0/3954602770 >> v2:172.21.15.133:3300/0 conn(0x7f62bc164b50 msgr2=0x7f62bc06bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:36.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.595+0000 7f62c0a35700 1 --2- 172.21.15.78:0/3954602770 >> v2:172.21.15.133:3300/0 conn(0x7f62bc164b50 0x7f62bc06bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:36.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.595+0000 7f62c0a35700 1 -- 172.21.15.78:0/3954602770 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62bc0ca720 con 0x7f62bc0cda10 2023-10-11T09:49:36.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.595+0000 7f62c0a35700 1 -- 172.21.15.78:0/3954602770 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f62b80019f0 con 0x7f62bc0cda10 2023-10-11T09:49:36.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.595+0000 7f62c0a35700 1 -- 172.21.15.78:0/3954602770 >> v1:172.21.15.78:6789/0 conn(0x7f62bc0cda10 legacy=0x7f62bc0cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:36.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.595+0000 7f62c0a35700 1 --2- 172.21.15.78:0/3954602770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62a8003a20 0x7f62a8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:36.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.595+0000 7f62c0a35700 1 --2- 172.21.15.78:0/3954602770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62a8004450 0x7f62a8004910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:36.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.595+0000 7f62c0a35700 1 --2- 172.21.15.78:0/3954602770 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62bc0caf20 0x7f62a8004fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:36.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.595+0000 7f62c0a35700 1 -- 172.21.15.78:0/3954602770 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc0cb370 con 0x7f62a8004450 2023-10-11T09:49:36.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.595+0000 7f62c0a35700 1 -- 172.21.15.78:0/3954602770 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc0ca440 con 0x7f62a8003a20 2023-10-11T09:49:36.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.595+0000 7f62c0a35700 1 -- 172.21.15.78:0/3954602770 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62a800ba30 con 0x7f62bc0caf20 2023-10-11T09:49:36.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.595+0000 7f62c1236700 1 --2- 172.21.15.78:0/3954602770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62a8004450 0x7f62a8004910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:36.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.595+0000 7f62c1a37700 1 --2- 172.21.15.78:0/3954602770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62a8003a20 0x7f62a8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:36.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.595+0000 7f62c2238700 1 --2- 172.21.15.78:0/3954602770 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62bc0caf20 0x7f62a8004fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:36.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.595+0000 7f62c1236700 1 -- 172.21.15.78:0/3954602770 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62bc0caf20 msgr2=0x7f62a8004fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:36.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.595+0000 7f62c1236700 1 --2- 172.21.15.78:0/3954602770 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62bc0caf20 0x7f62a8004fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:36.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.595+0000 7f62c1236700 1 -- 172.21.15.78:0/3954602770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62a8003a20 msgr2=0x7f62a8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:36.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.595+0000 7f62c1236700 1 --2- 172.21.15.78:0/3954602770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62a8003a20 0x7f62a8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:36.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.595+0000 7f62c1236700 1 -- 172.21.15.78:0/3954602770 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a80035d0 con 0x7f62a8004450 2023-10-11T09:49:36.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.595+0000 7f62c1236700 1 --2- 172.21.15.78:0/3954602770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62a8004450 0x7f62a8004910 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f62ac009b70 tx=0x7f62a80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e9049da0bb54ce server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:36.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.595+0000 7f62c0a35700 1 -- 172.21.15.78:0/3954602770 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ac00e6c0 con 0x7f62a8004450 2023-10-11T09:49:36.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.595+0000 7f62c0a35700 1 -- 172.21.15.78:0/3954602770 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62ac00e820 con 0x7f62a8004450 2023-10-11T09:49:36.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.595+0000 7f62c0a35700 1 -- 172.21.15.78:0/3954602770 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ac00eb90 con 0x7f62a8004450 2023-10-11T09:49:36.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.595+0000 7f62c2a39700 1 -- 172.21.15.78:0/3954602770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62a8004450 msgr2=0x7f62a8004910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:36.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.595+0000 7f62c2a39700 1 --2- 172.21.15.78:0/3954602770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62a8004450 0x7f62a8004910 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f62ac009b70 tx=0x7f62a80058a0 comp rx=0 tx=0).stop 2023-10-11T09:49:36.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.595+0000 7f62c2238700 1 -- 172.21.15.78:0/3954602770 reap_dead start 2023-10-11T09:49:36.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.595+0000 7f62c2a39700 1 -- 172.21.15.78:0/3954602770 shutdown_connections 2023-10-11T09:49:36.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.595+0000 7f62c2a39700 1 -- 172.21.15.78:0/3954602770 >> 172.21.15.78:0/3954602770 conn(0x7f62bc0cea60 msgr2=0x7f62bc0cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:36.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.599+0000 7f62c2a39700 1 -- 172.21.15.78:0/3954602770 shutdown_connections 2023-10-11T09:49:36.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.599+0000 7f62c2a39700 1 -- 172.21.15.78:0/3954602770 wait complete. 2023-10-11T09:49:36.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.599+0000 7f62c2a39700 1 Processor -- start 2023-10-11T09:49:36.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.599+0000 7f62c2a39700 1 -- start start 2023-10-11T09:49:36.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.603+0000 7f62c2a39700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62a8004450 0x7f62bc161010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:36.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.603+0000 7f62c2a39700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62bc0b4270 0x7f62bc144de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:36.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.603+0000 7f62c2a39700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62bc0b2070 0x7f62bc0b2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:36.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.603+0000 7f62c2a39700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc0b2a50 con 0x7f62a8004450 2023-10-11T09:49:36.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.603+0000 7f62c2a39700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc1495b0 con 0x7f62bc0b2070 2023-10-11T09:49:36.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.603+0000 7f62c2a39700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc161710 con 0x7f62bc0b4270 2023-10-11T09:49:36.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.603+0000 7f62c1a37700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62a8004450 0x7f62bc161010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:36.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.603+0000 7f62c1a37700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62a8004450 0x7f62bc161010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38560/0 (socket says 172.21.15.78:38560) 2023-10-11T09:49:36.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.603+0000 7f62c1a37700 1 -- 172.21.15.78:0/2339760160 learned_addr learned my addr 172.21.15.78:0/2339760160 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:36.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.603+0000 7f62c1236700 1 --2- 172.21.15.78:0/2339760160 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62bc0b4270 0x7f62bc144de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:36.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.603+0000 7f62c2238700 1 --2- 172.21.15.78:0/2339760160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62bc0b2070 0x7f62bc0b2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:36.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.603+0000 7f62c1a37700 1 -- 172.21.15.78:0/2339760160 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62bc0b4270 msgr2=0x7f62bc144de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:36.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.603+0000 7f62c1a37700 1 --2- 172.21.15.78:0/2339760160 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62bc0b4270 0x7f62bc144de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:36.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.603+0000 7f62c1a37700 1 -- 172.21.15.78:0/2339760160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62bc0b2070 msgr2=0x7f62bc0b2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:36.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.603+0000 7f62c1a37700 1 --2- 172.21.15.78:0/2339760160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62bc0b2070 0x7f62bc0b2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:36.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.603+0000 7f62c1a37700 1 -- 172.21.15.78:0/2339760160 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62bc1619f0 con 0x7f62a8004450 2023-10-11T09:49:36.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.603+0000 7f62c1236700 1 --2- 172.21.15.78:0/2339760160 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62bc0b4270 0x7f62bc144de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:49:36.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.603+0000 7f62c1a37700 1 --2- 172.21.15.78:0/2339760160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62a8004450 0x7f62bc161010 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f62b800afa0 tx=0x7f62b8007110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:36.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.603+0000 7f62b27fc700 1 -- 172.21.15.78:0/2339760160 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b8007a20 con 0x7f62a8004450 2023-10-11T09:49:36.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.603+0000 7f62b27fc700 1 -- 172.21.15.78:0/2339760160 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62b8007ba0 con 0x7f62a8004450 2023-10-11T09:49:36.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.603+0000 7f62c2a39700 1 -- 172.21.15.78:0/2339760160 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62bc161cd0 con 0x7f62a8004450 2023-10-11T09:49:36.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.603+0000 7f62b27fc700 1 -- 172.21.15.78:0/2339760160 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b8013630 con 0x7f62a8004450 2023-10-11T09:49:36.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.603+0000 7f62c2a39700 1 -- 172.21.15.78:0/2339760160 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62bc162290 con 0x7f62a8004450 2023-10-11T09:49:36.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.607+0000 7f62c2a39700 1 -- 172.21.15.78:0/2339760160 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62bc072070 con 0x7f62a8004450 2023-10-11T09:49:36.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.607+0000 7f62b27fc700 1 -- 172.21.15.78:0/2339760160 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f62b8013790 con 0x7f62a8004450 2023-10-11T09:49:36.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.607+0000 7f62b27fc700 1 --2- 172.21.15.78:0/2339760160 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f62a0065f00 0x7f62a00683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:36.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.607+0000 7f62b27fc700 1 -- 172.21.15.78:0/2339760160 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f62b8084790 con 0x7f62a8004450 2023-10-11T09:49:36.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.607+0000 7f62c1236700 1 --2- 172.21.15.78:0/2339760160 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f62a0065f00 0x7f62a00683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:36.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.611+0000 7f62c1236700 1 --2- 172.21.15.78:0/2339760160 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f62a0065f00 0x7f62a00683b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f62ac0090b0 tx=0x7f62ac008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:36.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.615+0000 7f62b27fc700 1 -- 172.21.15.78:0/2339760160 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f62b8051b30 con 0x7f62a8004450 2023-10-11T09:49:36.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.819+0000 7f62c2a39700 1 -- 172.21.15.78:0/2339760160 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62bc065d40 con 0x7f62a8004450 2023-10-11T09:49:36.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.819+0000 7f62b27fc700 1 -- 172.21.15.78:0/2339760160 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4657 (secure 0 0 0) 0x7f62b8017090 con 0x7f62a8004450 2023-10-11T09:49:36.827 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:49:36.827 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":71,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6581,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/238181631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":238181631},{"type":"v1","addr":"172.21.15.78:6837","nonce":238181631}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":7378,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/2416080754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2416080754},{"type":"v1","addr":"172.21.15.133:6834","nonce":2416080754}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":7384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/3016626706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3016626706},{"type":"v1","addr":"172.21.15.78:6836","nonce":3016626706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:48:45.351822+0000","modified":"2023-10-11T09:48:53.854831+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7725":{"gid":7725,"name":"b","rank":0,"incarnation":68,"state":"up:active","state_seq":4,"addr":"172.21.15.133:6835/3922016223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":3922016223},{"type":"v1","addr":"172.21.15.133:6835","nonce":3922016223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12,13],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-11T09:49:36.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.823+0000 7f62c2a39700 1 -- 172.21.15.78:0/2339760160 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f62a0065f00 msgr2=0x7f62a00683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:36.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.823+0000 7f62c2a39700 1 --2- 172.21.15.78:0/2339760160 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f62a0065f00 0x7f62a00683b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f62ac0090b0 tx=0x7f62ac008030 comp rx=0 tx=0).stop 2023-10-11T09:49:36.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.823+0000 7f62c2a39700 1 -- 172.21.15.78:0/2339760160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62a8004450 msgr2=0x7f62bc161010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:36.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.823+0000 7f62c2a39700 1 --2- 172.21.15.78:0/2339760160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62a8004450 0x7f62bc161010 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f62b800afa0 tx=0x7f62b8007110 comp rx=0 tx=0).stop 2023-10-11T09:49:36.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.823+0000 7f62c2a39700 1 -- 172.21.15.78:0/2339760160 shutdown_connections 2023-10-11T09:49:36.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.823+0000 7f62c2a39700 1 --2- 172.21.15.78:0/2339760160 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f62a0065f00 0x7f62a00683b0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:36.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.823+0000 7f62c2a39700 1 --2- 172.21.15.78:0/2339760160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62bc0b2070 0x7f62bc0b2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:36.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.823+0000 7f62c2a39700 1 --2- 172.21.15.78:0/2339760160 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62bc0b4270 0x7f62bc144de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:36.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.823+0000 7f62c2a39700 1 --2- 172.21.15.78:0/2339760160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62a8004450 0x7f62bc161010 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:36.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.823+0000 7f62c2a39700 1 -- 172.21.15.78:0/2339760160 >> 172.21.15.78:0/2339760160 conn(0x7f62bc0cea60 msgr2=0x7f62bc0cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:36.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.823+0000 7f62c2a39700 1 -- 172.21.15.78:0/2339760160 shutdown_connections 2023-10-11T09:49:36.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:36.823+0000 7f62c2a39700 1 -- 172.21.15.78:0/2339760160 wait complete. 2023-10-11T09:49:36.832 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 71 2023-10-11T09:49:36.850 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:49:37.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.011+0000 7f00f7b8b700 1 Processor -- start 2023-10-11T09:49:37.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.015+0000 7f00f7b8b700 1 -- start start 2023-10-11T09:49:37.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.015+0000 7f00f7b8b700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f00f0160d40 0x7f00f00cd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:37.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.015+0000 7f00f7b8b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f00f0071040 0x7f00f00cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:37.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.015+0000 7f00f7b8b700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00f006ffa0 con 0x7f00f00cd7c0 2023-10-11T09:49:37.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.015+0000 7f00f7b8b700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00f0070c20 con 0x7f00f0071040 2023-10-11T09:49:37.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.015+0000 7f00f7b8b700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00f0161110 con 0x7f00f0160d40 2023-10-11T09:49:37.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.015+0000 7f00f5126700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f00f0071040 0x7f00f00cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:37.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.015+0000 7f00f5126700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f00f0071040 0x7f00f00cddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38564/0 (socket says 172.21.15.78:38564) 2023-10-11T09:49:37.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.015+0000 7f00f5126700 1 -- 172.21.15.78:0/1497963309 learned_addr learned my addr 172.21.15.78:0/1497963309 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:37.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.015+0000 7f00f5927700 1 --2- 172.21.15.78:0/1497963309 >> v2:172.21.15.133:3300/0 conn(0x7f00f0160d40 0x7f00f00cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:37.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.015+0000 7f00f5126700 1 -- 172.21.15.78:0/1497963309 >> v2:172.21.15.133:3300/0 conn(0x7f00f0160d40 msgr2=0x7f00f00cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:37.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.015+0000 7f00f5126700 1 --2- 172.21.15.78:0/1497963309 >> v2:172.21.15.133:3300/0 conn(0x7f00f0160d40 0x7f00f00cd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:37.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.015+0000 7f00f5126700 1 -- 172.21.15.78:0/1497963309 >> v1:172.21.15.133:6789/0 conn(0x7f00f00cd7c0 legacy=0x7f00f01647f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:37.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.015+0000 7f00f5126700 1 -- 172.21.15.78:0/1497963309 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f0070280 con 0x7f00f0071040 2023-10-11T09:49:37.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.019+0000 7f00f5126700 1 --2- 172.21.15.78:0/1497963309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00f0071040 0x7f00f00cddf0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f00f00b1a50 tx=0x7f00e0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c859d187e08207 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:37.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.019+0000 7f00f4925700 1 -- 172.21.15.78:0/1497963309 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e0011830 con 0x7f00f0071040 2023-10-11T09:49:37.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.019+0000 7f00f4925700 1 -- 172.21.15.78:0/1497963309 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00e0011990 con 0x7f00f0071040 2023-10-11T09:49:37.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.019+0000 7f00f4925700 1 -- 172.21.15.78:0/1497963309 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e0011d20 con 0x7f00f0071040 2023-10-11T09:49:37.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.019+0000 7f00f7b8b700 1 -- 172.21.15.78:0/1497963309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00f0071040 msgr2=0x7f00f00cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:37.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.019+0000 7f00f7b8b700 1 --2- 172.21.15.78:0/1497963309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00f0071040 0x7f00f00cddf0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f00f00b1a50 tx=0x7f00e0009bf0 comp rx=0 tx=0).stop 2023-10-11T09:49:37.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.019+0000 7f00f7b8b700 1 -- 172.21.15.78:0/1497963309 shutdown_connections 2023-10-11T09:49:37.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.019+0000 7f00f7b8b700 1 --2- 172.21.15.78:0/1497963309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00f0071040 0x7f00f00cddf0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:37.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.019+0000 7f00f7b8b700 1 --2- 172.21.15.78:0/1497963309 >> v2:172.21.15.133:3300/0 conn(0x7f00f0160d40 0x7f00f00cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:37.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.019+0000 7f00f7b8b700 1 -- 172.21.15.78:0/1497963309 >> 172.21.15.78:0/1497963309 conn(0x7f00f006aca0 msgr2=0x7f00f00b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:37.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.019+0000 7f00f7b8b700 1 -- 172.21.15.78:0/1497963309 shutdown_connections 2023-10-11T09:49:37.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.019+0000 7f00f7b8b700 1 -- 172.21.15.78:0/1497963309 wait complete. 2023-10-11T09:49:37.030 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.023+0000 7f00f7b8b700 1 Processor -- start 2023-10-11T09:49:37.030 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.023+0000 7f00f7b8b700 1 -- start start 2023-10-11T09:49:37.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.023+0000 7f00f7b8b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00f0071040 0x7f00f015f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:37.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.023+0000 7f00f7b8b700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00f00cd7c0 0x7f00f00a9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:37.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.023+0000 7f00f7b8b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00f0160d40 0x7f00f00a95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:37.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.023+0000 7f00f7b8b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f00c4cf0 con 0x7f00f0160d40 2023-10-11T09:49:37.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.023+0000 7f00f7b8b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f00c8670 con 0x7f00f0071040 2023-10-11T09:49:37.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.023+0000 7f00f7b8b700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00f00c4f60 con 0x7f00f00cd7c0 2023-10-11T09:49:37.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.023+0000 7f00f6128700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00f0160d40 0x7f00f00a95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:37.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.023+0000 7f00f6128700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00f0160d40 0x7f00f00a95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38572/0 (socket says 172.21.15.78:38572) 2023-10-11T09:49:37.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.023+0000 7f00f6128700 1 -- 172.21.15.78:0/2472507332 learned_addr learned my addr 172.21.15.78:0/2472507332 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:37.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.027+0000 7f00f5126700 1 --2- 172.21.15.78:0/2472507332 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00f00cd7c0 0x7f00f00a9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:37.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.027+0000 7f00f5927700 1 --2- 172.21.15.78:0/2472507332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00f0071040 0x7f00f015f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:37.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.027+0000 7f00f6128700 1 -- 172.21.15.78:0/2472507332 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00f00cd7c0 msgr2=0x7f00f00a9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:37.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.027+0000 7f00f6128700 1 --2- 172.21.15.78:0/2472507332 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00f00cd7c0 0x7f00f00a9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:37.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.027+0000 7f00f6128700 1 -- 172.21.15.78:0/2472507332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00f0071040 msgr2=0x7f00f015f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:37.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.027+0000 7f00f6128700 1 --2- 172.21.15.78:0/2472507332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00f0071040 0x7f00f015f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:37.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.027+0000 7f00f6128700 1 -- 172.21.15.78:0/2472507332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e0009570 con 0x7f00f0160d40 2023-10-11T09:49:37.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.027+0000 7f00f6128700 1 --2- 172.21.15.78:0/2472507332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00f0160d40 0x7f00f00a95c0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f00e800c450 tx=0x7f00e800c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:37.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.027+0000 7f00e67fc700 1 -- 172.21.15.78:0/2472507332 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e80052d0 con 0x7f00f0160d40 2023-10-11T09:49:37.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.027+0000 7f00e67fc700 1 -- 172.21.15.78:0/2472507332 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00e8005430 con 0x7f00f0160d40 2023-10-11T09:49:37.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.027+0000 7f00f7b8b700 1 -- 172.21.15.78:0/2472507332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00f00c5240 con 0x7f00f0160d40 2023-10-11T09:49:37.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.027+0000 7f00e67fc700 1 -- 172.21.15.78:0/2472507332 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e80055a0 con 0x7f00f0160d40 2023-10-11T09:49:37.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.027+0000 7f00f7b8b700 1 -- 172.21.15.78:0/2472507332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00f00ab1d0 con 0x7f00f0160d40 2023-10-11T09:49:37.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.027+0000 7f00f7b8b700 1 -- 172.21.15.78:0/2472507332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00f00c73d0 con 0x7f00f0160d40 2023-10-11T09:49:37.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.031+0000 7f00e67fc700 1 -- 172.21.15.78:0/2472507332 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f00e80038e0 con 0x7f00f0160d40 2023-10-11T09:49:37.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.031+0000 7f00e67fc700 1 --2- 172.21.15.78:0/2472507332 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f00d4065ed0 0x7f00d4068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:37.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.031+0000 7f00e67fc700 1 -- 172.21.15.78:0/2472507332 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f00e8082e20 con 0x7f00f0160d40 2023-10-11T09:49:37.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.031+0000 7f00f5927700 1 --2- 172.21.15.78:0/2472507332 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f00d4065ed0 0x7f00d4068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:37.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.035+0000 7f00f5927700 1 --2- 172.21.15.78:0/2472507332 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f00d4065ed0 0x7f00d4068380 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f00ec0113e0 tx=0x7f00ec00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:37.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.039+0000 7f00e67fc700 1 -- 172.21.15.78:0/2472507332 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f00e8050130 con 0x7f00f0160d40 2023-10-11T09:49:37.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.243+0000 7f00f7b8b700 1 -- 172.21.15.78:0/2472507332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00f0072250 con 0x7f00f0160d40 2023-10-11T09:49:37.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.247+0000 7f00e67fc700 1 -- 172.21.15.78:0/2472507332 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4657 (secure 0 0 0) 0x7f00e8017090 con 0x7f00f0160d40 2023-10-11T09:49:37.253 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:49:37.253 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":71,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6581,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/238181631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":238181631},{"type":"v1","addr":"172.21.15.78:6837","nonce":238181631}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":7378,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/2416080754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2416080754},{"type":"v1","addr":"172.21.15.133:6834","nonce":2416080754}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":7384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/3016626706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3016626706},{"type":"v1","addr":"172.21.15.78:6836","nonce":3016626706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:48:45.351822+0000","modified":"2023-10-11T09:48:53.854831+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7725":{"gid":7725,"name":"b","rank":0,"incarnation":68,"state":"up:active","state_seq":4,"addr":"172.21.15.133:6835/3922016223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":3922016223},{"type":"v1","addr":"172.21.15.133:6835","nonce":3922016223}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12,13],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-11T09:49:37.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.247+0000 7f00f7b8b700 1 -- 172.21.15.78:0/2472507332 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f00d4065ed0 msgr2=0x7f00d4068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:37.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.247+0000 7f00f7b8b700 1 --2- 172.21.15.78:0/2472507332 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f00d4065ed0 0x7f00d4068380 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f00ec0113e0 tx=0x7f00ec00b040 comp rx=0 tx=0).stop 2023-10-11T09:49:37.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.247+0000 7f00f7b8b700 1 -- 172.21.15.78:0/2472507332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00f0160d40 msgr2=0x7f00f00a95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:37.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.247+0000 7f00f7b8b700 1 --2- 172.21.15.78:0/2472507332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00f0160d40 0x7f00f00a95c0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f00e800c450 tx=0x7f00e800c760 comp rx=0 tx=0).stop 2023-10-11T09:49:37.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.251+0000 7f00f7b8b700 1 -- 172.21.15.78:0/2472507332 shutdown_connections 2023-10-11T09:49:37.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.251+0000 7f00f7b8b700 1 --2- 172.21.15.78:0/2472507332 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f00d4065ed0 0x7f00d4068380 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:37.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.251+0000 7f00f7b8b700 1 --2- 172.21.15.78:0/2472507332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00f0160d40 0x7f00f00a95c0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:37.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.251+0000 7f00f7b8b700 1 --2- 172.21.15.78:0/2472507332 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00f00cd7c0 0x7f00f00a9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:37.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.251+0000 7f00f7b8b700 1 --2- 172.21.15.78:0/2472507332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00f0071040 0x7f00f015f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:37.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.251+0000 7f00f7b8b700 1 -- 172.21.15.78:0/2472507332 >> 172.21.15.78:0/2472507332 conn(0x7f00f006aca0 msgr2=0x7f00f00cdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:37.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.251+0000 7f00f7b8b700 1 -- 172.21.15.78:0/2472507332 shutdown_connections 2023-10-11T09:49:37.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.251+0000 7f00f7b8b700 1 -- 172.21.15.78:0/2472507332 wait complete. 2023-10-11T09:49:37.258 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 71 2023-10-11T09:49:37.275 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:49:37.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.439+0000 7f5639a7e700 1 Processor -- start 2023-10-11T09:49:37.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.443+0000 7f5639a7e700 1 -- start start 2023-10-11T09:49:37.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.443+0000 7f5639a7e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f563415b8e0 0x7f563406d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:37.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.443+0000 7f5639a7e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f56340ca950 0x7f563413fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:37.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.443+0000 7f5639a7e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f563415e010 con 0x7f563406d610 2023-10-11T09:49:37.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.443+0000 7f5639a7e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f563415e280 con 0x7f56340ca950 2023-10-11T09:49:37.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.443+0000 7f5639a7e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563415e560 con 0x7f563415b8e0 2023-10-11T09:49:37.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.443+0000 7f562bfff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f563406d610 0x7f56340ca240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:49516/0 (socket says 172.21.15.78:49516) 2023-10-11T09:49:37.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.443+0000 7f562bfff700 1 -- 172.21.15.78:0/3595221646 learned_addr learned my addr 172.21.15.78:0/3595221646 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:49:37.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.443+0000 7f5638a7c700 1 --2- 172.21.15.78:0/3595221646 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f563415b8e0 0x7f563406d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:37.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.443+0000 7f563927d700 1 --2- 172.21.15.78:0/3595221646 >> v2:172.21.15.133:3300/0 conn(0x7f56340ca950 0x7f563413fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:37.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.443+0000 7f562b7fe700 1 -- 172.21.15.78:0/3595221646 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2483304639 0 0) 0x7f563415e010 con 0x7f563406d610 2023-10-11T09:49:37.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.447+0000 7f562b7fe700 1 -- 172.21.15.78:0/3595221646 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f561c0035d0 con 0x7f563406d610 2023-10-11T09:49:37.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.447+0000 7f5638a7c700 1 -- 172.21.15.78:0/3595221646 >> v2:172.21.15.133:3300/0 conn(0x7f56340ca950 msgr2=0x7f563413fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:37.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.447+0000 7f5638a7c700 1 --2- 172.21.15.78:0/3595221646 >> v2:172.21.15.133:3300/0 conn(0x7f56340ca950 0x7f563413fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:37.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.447+0000 7f5638a7c700 1 -- 172.21.15.78:0/3595221646 >> v1:172.21.15.78:6789/0 conn(0x7f563406d610 legacy=0x7f56340ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:37.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.447+0000 7f5638a7c700 1 -- 172.21.15.78:0/3595221646 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563415e840 con 0x7f563415b8e0 2023-10-11T09:49:37.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.447+0000 7f5638a7c700 1 --2- 172.21.15.78:0/3595221646 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f563415b8e0 0x7f563406d0d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5630009850 tx=0x7f5630009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=624a7e3d082361b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:37.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.447+0000 7f562b7fe700 1 -- 172.21.15.78:0/3595221646 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5630011830 con 0x7f563415b8e0 2023-10-11T09:49:37.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.447+0000 7f562b7fe700 1 -- 172.21.15.78:0/3595221646 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5630011990 con 0x7f563415b8e0 2023-10-11T09:49:37.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.447+0000 7f562b7fe700 1 -- 172.21.15.78:0/3595221646 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5630011d00 con 0x7f563415b8e0 2023-10-11T09:49:37.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.447+0000 7f5639a7e700 1 -- 172.21.15.78:0/3595221646 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f563415b8e0 msgr2=0x7f563406d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:37.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.447+0000 7f5639a7e700 1 --2- 172.21.15.78:0/3595221646 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f563415b8e0 0x7f563406d0d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5630009850 tx=0x7f5630009b60 comp rx=0 tx=0).stop 2023-10-11T09:49:37.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.447+0000 7f5639a7e700 1 -- 172.21.15.78:0/3595221646 shutdown_connections 2023-10-11T09:49:37.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.447+0000 7f5639a7e700 1 --2- 172.21.15.78:0/3595221646 >> v2:172.21.15.133:3300/0 conn(0x7f56340ca950 0x7f563413fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:37.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.447+0000 7f5639a7e700 1 --2- 172.21.15.78:0/3595221646 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f563415b8e0 0x7f563406d0d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:37.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.447+0000 7f5639a7e700 1 -- 172.21.15.78:0/3595221646 >> 172.21.15.78:0/3595221646 conn(0x7f56340cea60 msgr2=0x7f56340cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:37.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.447+0000 7f5639a7e700 1 -- 172.21.15.78:0/3595221646 shutdown_connections 2023-10-11T09:49:37.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.447+0000 7f5639a7e700 1 -- 172.21.15.78:0/3595221646 wait complete. 2023-10-11T09:49:37.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.451+0000 7f5639a7e700 1 Processor -- start 2023-10-11T09:49:37.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.451+0000 7f5639a7e700 1 -- start start 2023-10-11T09:49:37.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.451+0000 7f5639a7e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f563406d610 0x7f5634159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:37.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.451+0000 7f5639a7e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56340ca950 0x7f56340c53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:37.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.451+0000 7f5639a7e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f563415b8e0 0x7f56341421d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:37.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.451+0000 7f5639a7e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56341407c0 con 0x7f56340ca950 2023-10-11T09:49:37.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.451+0000 7f5639a7e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56340c5bd0 con 0x7f563406d610 2023-10-11T09:49:37.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.451+0000 7f5639a7e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56340c5eb0 con 0x7f563415b8e0 2023-10-11T09:49:37.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.451+0000 7f562bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56340ca950 0x7f56340c53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:37.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.451+0000 7f562bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56340ca950 0x7f56340c53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38582/0 (socket says 172.21.15.78:38582) 2023-10-11T09:49:37.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.451+0000 7f562bfff700 1 -- 172.21.15.78:0/2487467536 learned_addr learned my addr 172.21.15.78:0/2487467536 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:37.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.451+0000 7f563927d700 1 --2- 172.21.15.78:0/2487467536 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f563415b8e0 0x7f56341421d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:37.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.451+0000 7f5638a7c700 1 --2- 172.21.15.78:0/2487467536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f563406d610 0x7f5634159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:37.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.451+0000 7f562bfff700 1 -- 172.21.15.78:0/2487467536 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f563415b8e0 msgr2=0x7f56341421d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:37.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.451+0000 7f562bfff700 1 --2- 172.21.15.78:0/2487467536 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f563415b8e0 0x7f56341421d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:37.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.451+0000 7f562bfff700 1 -- 172.21.15.78:0/2487467536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f563406d610 msgr2=0x7f5634159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:37.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.451+0000 7f562bfff700 1 --2- 172.21.15.78:0/2487467536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f563406d610 0x7f5634159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:37.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.451+0000 7f562bfff700 1 -- 172.21.15.78:0/2487467536 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56300094b0 con 0x7f56340ca950 2023-10-11T09:49:37.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.451+0000 7f562bfff700 1 --2- 172.21.15.78:0/2487467536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56340ca950 0x7f56340c53a0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f562400c4f0 tx=0x7f562400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:37.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.455+0000 7f56297fa700 1 -- 172.21.15.78:0/2487467536 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5624005d90 con 0x7f56340ca950 2023-10-11T09:49:37.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.455+0000 7f56297fa700 1 -- 172.21.15.78:0/2487467536 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56240070d0 con 0x7f56340ca950 2023-10-11T09:49:37.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.455+0000 7f56297fa700 1 -- 172.21.15.78:0/2487467536 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5624007440 con 0x7f56340ca950 2023-10-11T09:49:37.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.455+0000 7f5639a7e700 1 -- 172.21.15.78:0/2487467536 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5634140170 con 0x7f56340ca950 2023-10-11T09:49:37.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.455+0000 7f5639a7e700 1 -- 172.21.15.78:0/2487467536 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56340c58e0 con 0x7f56340ca950 2023-10-11T09:49:37.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.455+0000 7f5639a7e700 1 -- 172.21.15.78:0/2487467536 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5634065d40 con 0x7f56340ca950 2023-10-11T09:49:37.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.455+0000 7f56297fa700 1 -- 172.21.15.78:0/2487467536 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f56240075a0 con 0x7f56340ca950 2023-10-11T09:49:37.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.459+0000 7f56297fa700 1 --2- 172.21.15.78:0/2487467536 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5620065f00 0x7f56200683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:37.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.459+0000 7f56297fa700 1 -- 172.21.15.78:0/2487467536 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f5624082d10 con 0x7f56340ca950 2023-10-11T09:49:37.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.459+0000 7f5638a7c700 1 --2- 172.21.15.78:0/2487467536 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5620065f00 0x7f56200683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:37.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.459+0000 7f5638a7c700 1 --2- 172.21.15.78:0/2487467536 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5620065f00 0x7f56200683b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f56300090b0 tx=0x7f5630008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:37.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.463+0000 7f56297fa700 1 -- 172.21.15.78:0/2487467536 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f5624050170 con 0x7f56340ca950 2023-10-11T09:49:37.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.643+0000 7f5639a7e700 1 -- 172.21.15.78:0/2487467536 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f563413e740 con 0x7f56340ca950 2023-10-11T09:49:37.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.643+0000 7f56297fa700 1 -- 172.21.15.78:0/2487467536 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+20058 (secure 0 0 0) 0x7f5624017020 con 0x7f56340ca950 2023-10-11T09:49:37.652 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:49:37.652 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":92,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:48:50.804275+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:48:18.491556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-11T09:48:19.018937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data_extra","create_time":"2023-10-11T09:48:28.963956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6834/1560261029":"2023-10-12T09:48:50.804239+0000","172.21.15.78:6836/1560261029":"2023-10-12T09:48:50.804239+0000","172.21.15.78:6836/3463904224":"2023-10-12T09:48:45.480946+0000","172.21.15.78:6834/3463904224":"2023-10-12T09:48:45.480946+0000","172.21.15.78:6837/951312831":"2023-10-12T09:48:44.433517+0000","172.21.15.78:6835/951312831":"2023-10-12T09:48:44.433517+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:49:37.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.647+0000 7f5639a7e700 1 -- 172.21.15.78:0/2487467536 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5620065f00 msgr2=0x7f56200683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:37.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.647+0000 7f5639a7e700 1 --2- 172.21.15.78:0/2487467536 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5620065f00 0x7f56200683b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f56300090b0 tx=0x7f5630008030 comp rx=0 tx=0).stop 2023-10-11T09:49:37.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.647+0000 7f5639a7e700 1 -- 172.21.15.78:0/2487467536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56340ca950 msgr2=0x7f56340c53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:37.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.647+0000 7f5639a7e700 1 --2- 172.21.15.78:0/2487467536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56340ca950 0x7f56340c53a0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f562400c4f0 tx=0x7f562400c800 comp rx=0 tx=0).stop 2023-10-11T09:49:37.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.647+0000 7f5639a7e700 1 -- 172.21.15.78:0/2487467536 shutdown_connections 2023-10-11T09:49:37.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.647+0000 7f5639a7e700 1 --2- 172.21.15.78:0/2487467536 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5620065f00 0x7f56200683b0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:37.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.647+0000 7f5639a7e700 1 --2- 172.21.15.78:0/2487467536 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f563415b8e0 0x7f56341421d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:37.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.647+0000 7f5639a7e700 1 --2- 172.21.15.78:0/2487467536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56340ca950 0x7f56340c53a0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:37.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.647+0000 7f5639a7e700 1 --2- 172.21.15.78:0/2487467536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f563406d610 0x7f5634159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:37.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.647+0000 7f5639a7e700 1 -- 172.21.15.78:0/2487467536 >> 172.21.15.78:0/2487467536 conn(0x7f56340cea60 msgr2=0x7f56340cee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:37.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.647+0000 7f5639a7e700 1 -- 172.21.15.78:0/2487467536 shutdown_connections 2023-10-11T09:49:37.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.647+0000 7f5639a7e700 1 -- 172.21.15.78:0/2487467536 wait complete. 2023-10-11T09:49:37.676 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-11T09:49:37.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.843+0000 7f6686f70700 1 Processor -- start 2023-10-11T09:49:37.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.843+0000 7f6686f70700 1 -- start start 2023-10-11T09:49:37.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.847+0000 7f6686f70700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f66800d1660 0x7f66800d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:37.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.847+0000 7f6686f70700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f668015dc10 con 0x7f6680156ce0 2023-10-11T09:49:37.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.847+0000 7f6686f70700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6680160380 con 0x7f66800d28e0 2023-10-11T09:49:37.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.847+0000 7f6686f70700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f668015d930 con 0x7f66800d1660 2023-10-11T09:49:37.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.847+0000 7f6685f6e700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f6680156ce0 0x7f6680160b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:49520/0 (socket says 172.21.15.78:49520) 2023-10-11T09:49:37.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.847+0000 7f6685f6e700 1 -- 172.21.15.78:0/3355937777 learned_addr learned my addr 172.21.15.78:0/3355937777 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:49:37.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.847+0000 7f668576d700 1 --2- 172.21.15.78:0/3355937777 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f66800d1660 0x7f66800d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:37.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.847+0000 7f6684f6c700 1 -- 172.21.15.78:0/3355937777 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3677178658 0 0) 0x7f668015dc10 con 0x7f6680156ce0 2023-10-11T09:49:37.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.847+0000 7f6684f6c700 1 -- 172.21.15.78:0/3355937777 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f666c0035d0 con 0x7f6680156ce0 2023-10-11T09:49:37.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.847+0000 7f668576d700 1 -- 172.21.15.78:0/3355937777 >> v1:172.21.15.133:6789/0 conn(0x7f66800d28e0 legacy=0x7f66800cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:37.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.847+0000 7f6684f6c700 1 -- 172.21.15.78:0/3355937777 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3621935015 0 0) 0x7f6680160380 con 0x7f66800d28e0 2023-10-11T09:49:37.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.847+0000 7f668576d700 1 -- 172.21.15.78:0/3355937777 >> v1:172.21.15.78:6789/0 conn(0x7f6680156ce0 legacy=0x7f6680160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:37.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.847+0000 7f668576d700 1 -- 172.21.15.78:0/3355937777 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6680160620 con 0x7f66800d1660 2023-10-11T09:49:37.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.847+0000 7f668576d700 1 --2- 172.21.15.78:0/3355937777 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f66800d1660 0x7f66800d0a50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6670009880 tx=0x7f6670009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f34bc53119d2dedc server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:37.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.847+0000 7f6684f6c700 1 -- 172.21.15.78:0/3355937777 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6670011830 con 0x7f66800d1660 2023-10-11T09:49:37.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.851+0000 7f6684f6c700 1 -- 172.21.15.78:0/3355937777 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6670011990 con 0x7f66800d1660 2023-10-11T09:49:37.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.851+0000 7f6684f6c700 1 -- 172.21.15.78:0/3355937777 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6670011d20 con 0x7f66800d1660 2023-10-11T09:49:37.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.851+0000 7f6686f70700 1 -- 172.21.15.78:0/3355937777 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f66800d1660 msgr2=0x7f66800d0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:37.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.851+0000 7f6686f70700 1 --2- 172.21.15.78:0/3355937777 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f66800d1660 0x7f66800d0a50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6670009880 tx=0x7f6670009b90 comp rx=0 tx=0).stop 2023-10-11T09:49:37.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.851+0000 7f6686f70700 1 -- 172.21.15.78:0/3355937777 shutdown_connections 2023-10-11T09:49:37.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.851+0000 7f6686f70700 1 --2- 172.21.15.78:0/3355937777 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f66800d1660 0x7f66800d0a50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:37.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.851+0000 7f6686f70700 1 -- 172.21.15.78:0/3355937777 >> 172.21.15.78:0/3355937777 conn(0x7f668006afb0 msgr2=0x7f668006b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:37.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.851+0000 7f6686f70700 1 -- 172.21.15.78:0/3355937777 shutdown_connections 2023-10-11T09:49:37.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.851+0000 7f6686f70700 1 -- 172.21.15.78:0/3355937777 wait complete. 2023-10-11T09:49:37.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.851+0000 7f6686f70700 1 Processor -- start 2023-10-11T09:49:37.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.851+0000 7f6686f70700 1 -- start start 2023-10-11T09:49:37.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.855+0000 7f6686f70700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66800d1660 0x7f66800cc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:37.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.855+0000 7f6686f70700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f66800d28e0 0x7f66800c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:37.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.855+0000 7f6686f70700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6680156ce0 0x7f66801f1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:37.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.855+0000 7f6686f70700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66800c7c60 con 0x7f6680156ce0 2023-10-11T09:49:37.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.855+0000 7f6686f70700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66801f28d0 con 0x7f66800d1660 2023-10-11T09:49:37.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.855+0000 7f6686f70700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66801f2bb0 con 0x7f66800d28e0 2023-10-11T09:49:37.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.855+0000 7f668676f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6680156ce0 0x7f66801f1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:37.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.855+0000 7f668676f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6680156ce0 0x7f66801f1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38584/0 (socket says 172.21.15.78:38584) 2023-10-11T09:49:37.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.855+0000 7f668676f700 1 -- 172.21.15.78:0/4272961190 learned_addr learned my addr 172.21.15.78:0/4272961190 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:37.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.855+0000 7f668576d700 1 --2- 172.21.15.78:0/4272961190 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f66800d28e0 0x7f66800c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:37.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.855+0000 7f6685f6e700 1 --2- 172.21.15.78:0/4272961190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66800d1660 0x7f66800cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:37.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.855+0000 7f668676f700 1 -- 172.21.15.78:0/4272961190 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f66800d28e0 msgr2=0x7f66800c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:37.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.855+0000 7f668676f700 1 --2- 172.21.15.78:0/4272961190 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f66800d28e0 0x7f66800c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:37.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.855+0000 7f668676f700 1 -- 172.21.15.78:0/4272961190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66800d1660 msgr2=0x7f66800cc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:37.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.855+0000 7f668676f700 1 --2- 172.21.15.78:0/4272961190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66800d1660 0x7f66800cc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:37.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.855+0000 7f668676f700 1 -- 172.21.15.78:0/4272961190 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66700094b0 con 0x7f6680156ce0 2023-10-11T09:49:37.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.855+0000 7f6685f6e700 1 --2- 172.21.15.78:0/4272961190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66800d1660 0x7f66800cc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:49:37.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.855+0000 7f668676f700 1 --2- 172.21.15.78:0/4272961190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6680156ce0 0x7f66801f1d80 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f667800cc70 tx=0x7f667800cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:37.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.855+0000 7f66767fc700 1 -- 172.21.15.78:0/4272961190 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6678003e50 con 0x7f6680156ce0 2023-10-11T09:49:37.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.855+0000 7f66767fc700 1 -- 172.21.15.78:0/4272961190 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f667800a7a0 con 0x7f6680156ce0 2023-10-11T09:49:37.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.855+0000 7f6686f70700 1 -- 172.21.15.78:0/4272961190 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66800c7eb0 con 0x7f6680156ce0 2023-10-11T09:49:37.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.855+0000 7f66767fc700 1 -- 172.21.15.78:0/4272961190 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667800ab50 con 0x7f6680156ce0 2023-10-11T09:49:37.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.855+0000 7f6686f70700 1 -- 172.21.15.78:0/4272961190 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66801f55e0 con 0x7f6680156ce0 2023-10-11T09:49:37.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.859+0000 7f6686f70700 1 -- 172.21.15.78:0/4272961190 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66800c83d0 con 0x7f6680156ce0 2023-10-11T09:49:37.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.859+0000 7f66767fc700 1 -- 172.21.15.78:0/4272961190 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f667800acb0 con 0x7f6680156ce0 2023-10-11T09:49:37.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.863+0000 7f66767fc700 1 --2- 172.21.15.78:0/4272961190 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6664065ed0 0x7f6664068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:37.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.863+0000 7f66767fc700 1 -- 172.21.15.78:0/4272961190 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f66780835c0 con 0x7f6680156ce0 2023-10-11T09:49:37.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.863+0000 7f6685f6e700 1 --2- 172.21.15.78:0/4272961190 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6664065ed0 0x7f6664068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:37.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.863+0000 7f6685f6e700 1 --2- 172.21.15.78:0/4272961190 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6664065ed0 0x7f6664068380 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f667c005930 tx=0x7f667c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:37.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:37.867+0000 7f66767fc700 1 -- 172.21.15.78:0/4272961190 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f66800c83d0 con 0x7f6680156ce0 2023-10-11T09:49:38.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.079+0000 7f6686f70700 1 -- 172.21.15.78:0/4272961190 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f66800c83d0 con 0x7f6680156ce0 2023-10-11T09:49:38.088 INFO:tasks.ceph.mon.a.smithi078.stderr:2023-10-11T09:49:38.083+0000 7f90131f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-11T09:49:38.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.099+0000 7f66767fc700 1 -- 172.21.15.78:0/4272961190 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v72) v1 ==== 161+0+0 (secure 0 0 0) 0x7f66800c83d0 con 0x7f6680156ce0 2023-10-11T09:49:38.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.103+0000 7f6686f70700 1 -- 172.21.15.78:0/4272961190 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6664065ed0 msgr2=0x7f6664068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:38.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.103+0000 7f6686f70700 1 --2- 172.21.15.78:0/4272961190 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6664065ed0 0x7f6664068380 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f667c005930 tx=0x7f667c009040 comp rx=0 tx=0).stop 2023-10-11T09:49:38.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.103+0000 7f6686f70700 1 -- 172.21.15.78:0/4272961190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6680156ce0 msgr2=0x7f66801f1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:38.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.103+0000 7f6686f70700 1 --2- 172.21.15.78:0/4272961190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6680156ce0 0x7f66801f1d80 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f667800cc70 tx=0x7f667800cf80 comp rx=0 tx=0).stop 2023-10-11T09:49:38.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.103+0000 7f6686f70700 1 -- 172.21.15.78:0/4272961190 shutdown_connections 2023-10-11T09:49:38.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.103+0000 7f6686f70700 1 --2- 172.21.15.78:0/4272961190 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6664065ed0 0x7f6664068380 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:38.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.103+0000 7f6686f70700 1 --2- 172.21.15.78:0/4272961190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6680156ce0 0x7f66801f1d80 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:38.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.103+0000 7f6686f70700 1 --2- 172.21.15.78:0/4272961190 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f66800d28e0 0x7f66800c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:38.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.103+0000 7f6686f70700 1 --2- 172.21.15.78:0/4272961190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66800d1660 0x7f66800cc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:38.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.103+0000 7f6686f70700 1 -- 172.21.15.78:0/4272961190 >> 172.21.15.78:0/4272961190 conn(0x7f668006afb0 msgr2=0x7f668006b620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:38.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.103+0000 7f6686f70700 1 -- 172.21.15.78:0/4272961190 shutdown_connections 2023-10-11T09:49:38.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.103+0000 7f6686f70700 1 -- 172.21.15.78:0/4272961190 wait complete. 2023-10-11T09:49:38.111 INFO:tasks.ceph.mds.b.smithi133.stderr: -4503> 2023-10-11T09:48:50.845+0000 7f575d918700 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-10-11T09:49:38.111 INFO:teuthology.orchestra.run.smithi078.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-11T09:49:38.114 INFO:tasks.ceph.mds.b.smithi133.stderr: -3064> 2023-10-11T09:48:55.833+0000 7f576111f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-11T09:49:38.115 INFO:tasks.ceph.mds.b.smithi133.stderr: -2676> 2023-10-11T09:48:58.229+0000 7f575d918700 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2023-10-11T09:49:38.116 INFO:tasks.ceph.mds.b.smithi133.stderr: -2506> 2023-10-11T09:48:58.233+0000 7f575d918700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /subdir/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x562310908580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(11)0x10000000000:[<0x1/subdir v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-10-11T09:49:38.116 INFO:tasks.ceph.mds.b.smithi133.stderr: -2452> 2023-10-11T09:48:58.233+0000 7f575d918700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/sixmegs auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x56231098e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(13)0x10000000001:[<0x10000000000/sixmegs v8>,<0x1/subdir v10>]//[12]","memoryvalue":"(13)0x10000000001:[<0x10000000000/sixmegs v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:49:38.117 INFO:tasks.ceph.mds.b.smithi133.stderr: -2158> 2023-10-11T09:48:58.241+0000 7f575d918700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v5 ap=5 snaprealm=0x562310937200 DIRTYPARENT f(v1 1=0+1) n(v0 rc2023-10-11T09:48:57.930520+0000 1=0+1)/n() (idft sync r=1) (inest sync r=1) (ifile sync->mix r=1) (iversion lock) caps={7776=pLsXs/-@3} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x56231090a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(11)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=0+1)","ondisk_value.rstat":"n(v0 2=0+2)","memory_value.dirstat":"f(v1 1=0+1)","memory_value.rstat":"n(v0 rc2023-10-11T09:48:57.930520+0000 1=0+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-10-11T09:49:38.129 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-11T09:49:38.171 INFO:tasks.ceph.mds.b.smithi133.stdout:starting mds.b at 2023-10-11T09:49:38.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.299+0000 7f1a4275d700 1 Processor -- start 2023-10-11T09:49:38.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.303+0000 7f1a4275d700 1 -- start start 2023-10-11T09:49:38.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.303+0000 7f1a4275d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a3c157130 0x7f1a3c15a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:38.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.303+0000 7f1a4275d700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1a3c0ce240 0x7f1a3c0705b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:38.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.303+0000 7f1a4275d700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a3c070af0 con 0x7f1a3c06ccf0 2023-10-11T09:49:38.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.303+0000 7f1a4275d700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a3c0b1aa0 con 0x7f1a3c0ce240 2023-10-11T09:49:38.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.303+0000 7f1a4275d700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c0ce690 con 0x7f1a3c157130 2023-10-11T09:49:38.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.307+0000 7f1a41f5c700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f1a3c06ccf0 0x7f1a3c0b1ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:49532/0 (socket says 172.21.15.78:49532) 2023-10-11T09:49:38.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.307+0000 7f1a41f5c700 1 -- 172.21.15.78:0/3750987953 learned_addr learned my addr 172.21.15.78:0/3750987953 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:49:38.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.307+0000 7f1a40f5a700 1 --2- 172.21.15.78:0/3750987953 >> v2:172.21.15.133:3300/0 conn(0x7f1a3c0ce240 0x7f1a3c0705b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:38.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.307+0000 7f1a4175b700 1 --2- 172.21.15.78:0/3750987953 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a3c157130 0x7f1a3c15a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:38.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.307+0000 7f1a33fff700 1 -- 172.21.15.78:0/3750987953 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2655771928 0 0) 0x7f1a3c070af0 con 0x7f1a3c06ccf0 2023-10-11T09:49:38.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.307+0000 7f1a33fff700 1 -- 172.21.15.78:0/3750987953 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a280035d0 con 0x7f1a3c06ccf0 2023-10-11T09:49:38.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.307+0000 7f1a33fff700 1 -- 172.21.15.78:0/3750987953 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4289491255 0 0) 0x7f1a280035d0 con 0x7f1a3c06ccf0 2023-10-11T09:49:38.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.307+0000 7f1a4175b700 1 -- 172.21.15.78:0/3750987953 >> v2:172.21.15.133:3300/0 conn(0x7f1a3c0ce240 msgr2=0x7f1a3c0705b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:38.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.307+0000 7f1a4175b700 1 --2- 172.21.15.78:0/3750987953 >> v2:172.21.15.133:3300/0 conn(0x7f1a3c0ce240 0x7f1a3c0705b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:38.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.307+0000 7f1a4175b700 1 -- 172.21.15.78:0/3750987953 >> v1:172.21.15.78:6789/0 conn(0x7f1a3c06ccf0 legacy=0x7f1a3c0b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:38.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.307+0000 7f1a4175b700 1 -- 172.21.15.78:0/3750987953 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a3c157500 con 0x7f1a3c157130 2023-10-11T09:49:38.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.307+0000 7f1a4175b700 1 --2- 172.21.15.78:0/3750987953 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a3c157130 0x7f1a3c15a680 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1a38009850 tx=0x7f1a38009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf8cde7514919234 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:38.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.307+0000 7f1a33fff700 1 -- 172.21.15.78:0/3750987953 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a340019f0 con 0x7f1a3c157130 2023-10-11T09:49:38.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.307+0000 7f1a33fff700 1 -- 172.21.15.78:0/3750987953 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a38004030 con 0x7f1a3c157130 2023-10-11T09:49:38.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.307+0000 7f1a33fff700 1 -- 172.21.15.78:0/3750987953 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a380043e0 con 0x7f1a3c157130 2023-10-11T09:49:38.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.307+0000 7f1a4275d700 1 -- 172.21.15.78:0/3750987953 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a3c157130 msgr2=0x7f1a3c15a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:38.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.307+0000 7f1a4275d700 1 --2- 172.21.15.78:0/3750987953 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a3c157130 0x7f1a3c15a680 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1a38009850 tx=0x7f1a38009b60 comp rx=0 tx=0).stop 2023-10-11T09:49:38.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.307+0000 7f1a4275d700 1 -- 172.21.15.78:0/3750987953 shutdown_connections 2023-10-11T09:49:38.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.307+0000 7f1a4275d700 1 --2- 172.21.15.78:0/3750987953 >> v2:172.21.15.133:3300/0 conn(0x7f1a3c0ce240 0x7f1a3c0705b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:38.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.307+0000 7f1a4275d700 1 --2- 172.21.15.78:0/3750987953 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a3c157130 0x7f1a3c15a680 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:38.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.307+0000 7f1a4275d700 1 -- 172.21.15.78:0/3750987953 >> 172.21.15.78:0/3750987953 conn(0x7f1a3c06a060 msgr2=0x7f1a3c06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:38.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.311+0000 7f1a4275d700 1 -- 172.21.15.78:0/3750987953 shutdown_connections 2023-10-11T09:49:38.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.311+0000 7f1a4275d700 1 -- 172.21.15.78:0/3750987953 wait complete. 2023-10-11T09:49:38.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.311+0000 7f1a4275d700 1 Processor -- start 2023-10-11T09:49:38.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.311+0000 7f1a4275d700 1 -- start start 2023-10-11T09:49:38.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.315+0000 7f1a4275d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a3c06ccf0 0x7f1a3c156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:38.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.315+0000 7f1a4275d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a3c0ce240 0x7f1a3c13fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:38.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.315+0000 7f1a4275d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a3c157130 0x7f1a3c14bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:38.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.315+0000 7f1a4275d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c14c6f0 con 0x7f1a3c0ce240 2023-10-11T09:49:38.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.315+0000 7f1a4275d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c13f3f0 con 0x7f1a3c157130 2023-10-11T09:49:38.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.315+0000 7f1a4275d700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c13f6d0 con 0x7f1a3c06ccf0 2023-10-11T09:49:38.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.315+0000 7f1a40f5a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a3c0ce240 0x7f1a3c13fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:38.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.315+0000 7f1a40f5a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a3c0ce240 0x7f1a3c13fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38586/0 (socket says 172.21.15.78:38586) 2023-10-11T09:49:38.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.315+0000 7f1a40f5a700 1 -- 172.21.15.78:0/632711138 learned_addr learned my addr 172.21.15.78:0/632711138 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:38.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.315+0000 7f1a4175b700 1 --2- 172.21.15.78:0/632711138 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a3c06ccf0 0x7f1a3c156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:38.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.315+0000 7f1a41f5c700 1 --2- 172.21.15.78:0/632711138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a3c157130 0x7f1a3c14bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:38.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.315+0000 7f1a40f5a700 1 -- 172.21.15.78:0/632711138 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a3c06ccf0 msgr2=0x7f1a3c156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:38.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.315+0000 7f1a40f5a700 1 --2- 172.21.15.78:0/632711138 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a3c06ccf0 0x7f1a3c156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:38.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.315+0000 7f1a40f5a700 1 -- 172.21.15.78:0/632711138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a3c157130 msgr2=0x7f1a3c14bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:38.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.315+0000 7f1a40f5a700 1 --2- 172.21.15.78:0/632711138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a3c157130 0x7f1a3c14bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:38.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.315+0000 7f1a40f5a700 1 -- 172.21.15.78:0/632711138 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a380094b0 con 0x7f1a3c0ce240 2023-10-11T09:49:38.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.315+0000 7f1a41f5c700 1 --2- 172.21.15.78:0/632711138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a3c157130 0x7f1a3c14bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:49:38.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.315+0000 7f1a40f5a700 1 --2- 172.21.15.78:0/632711138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a3c0ce240 0x7f1a3c13fed0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c00e840 tx=0x7f1a2c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:38.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.315+0000 7f1a31ffb700 1 -- 172.21.15.78:0/632711138 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c0095a0 con 0x7f1a3c0ce240 2023-10-11T09:49:38.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.315+0000 7f1a31ffb700 1 -- 172.21.15.78:0/632711138 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a2c00c690 con 0x7f1a3c0ce240 2023-10-11T09:49:38.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.315+0000 7f1a4275d700 1 -- 172.21.15.78:0/632711138 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a3c13f9b0 con 0x7f1a3c0ce240 2023-10-11T09:49:38.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.315+0000 7f1a31ffb700 1 -- 172.21.15.78:0/632711138 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c00ca00 con 0x7f1a3c0ce240 2023-10-11T09:49:38.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.315+0000 7f1a4275d700 1 -- 172.21.15.78:0/632711138 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a3c13f010 con 0x7f1a3c0ce240 2023-10-11T09:49:38.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.319+0000 7f1a4275d700 1 -- 172.21.15.78:0/632711138 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a3c071af0 con 0x7f1a3c0ce240 2023-10-11T09:49:38.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.319+0000 7f1a31ffb700 1 -- 172.21.15.78:0/632711138 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f1a2c00cb60 con 0x7f1a3c0ce240 2023-10-11T09:49:38.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.319+0000 7f1a31ffb700 1 --2- 172.21.15.78:0/632711138 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1a20065ed0 0x7f1a20068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:38.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.323+0000 7f1a31ffb700 1 -- 172.21.15.78:0/632711138 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6810+0+0 (secure 0 0 0) 0x7f1a2c083b80 con 0x7f1a3c0ce240 2023-10-11T09:49:38.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.323+0000 7f1a4175b700 1 --2- 172.21.15.78:0/632711138 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1a20065ed0 0x7f1a20068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:38.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.323+0000 7f1a4175b700 1 --2- 172.21.15.78:0/632711138 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1a20065ed0 0x7f1a20068380 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1a380090b0 tx=0x7f1a38013040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:38.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.327+0000 7f1a31ffb700 1 -- 172.21.15.78:0/632711138 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f1a2c050dc0 con 0x7f1a3c0ce240 2023-10-11T09:49:38.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:38.547+0000 7f1a4275d700 1 -- 172.21.15.78:0/632711138 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1a3c071d30 con 0x7f1a3c0ce240 2023-10-11T09:49:39.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.111+0000 7f1a31ffb700 1 -- 172.21.15.78:0/632711138 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v73) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1a2c0543e0 con 0x7f1a3c0ce240 2023-10-11T09:49:39.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.115+0000 7f1a4275d700 1 -- 172.21.15.78:0/632711138 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1a20065ed0 msgr2=0x7f1a20068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:39.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.115+0000 7f1a4275d700 1 --2- 172.21.15.78:0/632711138 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1a20065ed0 0x7f1a20068380 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1a380090b0 tx=0x7f1a38013040 comp rx=0 tx=0).stop 2023-10-11T09:49:39.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.115+0000 7f1a4275d700 1 -- 172.21.15.78:0/632711138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a3c0ce240 msgr2=0x7f1a3c13fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:39.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.115+0000 7f1a4275d700 1 --2- 172.21.15.78:0/632711138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a3c0ce240 0x7f1a3c13fed0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c00e840 tx=0x7f1a2c00ec00 comp rx=0 tx=0).stop 2023-10-11T09:49:39.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.115+0000 7f1a4275d700 1 -- 172.21.15.78:0/632711138 shutdown_connections 2023-10-11T09:49:39.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.115+0000 7f1a4275d700 1 --2- 172.21.15.78:0/632711138 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1a20065ed0 0x7f1a20068380 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:39.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.115+0000 7f1a4275d700 1 --2- 172.21.15.78:0/632711138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a3c157130 0x7f1a3c14bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:39.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.115+0000 7f1a4275d700 1 --2- 172.21.15.78:0/632711138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a3c0ce240 0x7f1a3c13fed0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:39.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.115+0000 7f1a4275d700 1 --2- 172.21.15.78:0/632711138 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a3c06ccf0 0x7f1a3c156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:39.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.115+0000 7f1a4275d700 1 -- 172.21.15.78:0/632711138 >> 172.21.15.78:0/632711138 conn(0x7f1a3c06a060 msgr2=0x7f1a3c15c640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:39.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.115+0000 7f1a4275d700 1 -- 172.21.15.78:0/632711138 shutdown_connections 2023-10-11T09:49:39.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.115+0000 7f1a4275d700 1 -- 172.21.15.78:0/632711138 wait complete. 2023-10-11T09:49:39.145 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-11T09:49:39.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.311+0000 7f3668cf7700 1 Processor -- start 2023-10-11T09:49:39.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.311+0000 7f3668cf7700 1 -- start start 2023-10-11T09:49:39.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.311+0000 7f3668cf7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3664160b60 0x7f3664165190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:39.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.311+0000 7f3668cf7700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f36640d20b0 0x7f36640d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:39.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.311+0000 7f3668cf7700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36640d2c90 con 0x7f36640d2500 2023-10-11T09:49:39.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.311+0000 7f3668cf7700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36641646f0 con 0x7f36640d20b0 2023-10-11T09:49:39.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.311+0000 7f3668cf7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36640d29b0 con 0x7f3664160b60 2023-10-11T09:49:39.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.315+0000 7f3662ffd700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f36640d20b0 0x7f36640d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:39.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.315+0000 7f3662ffd700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f36640d20b0 0x7f36640d0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38596/0 (socket says 172.21.15.78:38596) 2023-10-11T09:49:39.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.315+0000 7f3662ffd700 1 -- 172.21.15.78:0/1942419550 learned_addr learned my addr 172.21.15.78:0/1942419550 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:39.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.315+0000 7f36637fe700 1 --2- 172.21.15.78:0/1942419550 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3664160b60 0x7f3664165190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:39.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.315+0000 7f3662ffd700 1 -- 172.21.15.78:0/1942419550 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3664160b60 msgr2=0x7f3664165190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:39.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.315+0000 7f3662ffd700 1 --2- 172.21.15.78:0/1942419550 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3664160b60 0x7f3664165190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:39.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.315+0000 7f3662ffd700 1 -- 172.21.15.78:0/1942419550 >> v1:172.21.15.133:6789/0 conn(0x7f36640d2500 legacy=0x7f36640d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:39.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.315+0000 7f3662ffd700 1 -- 172.21.15.78:0/1942419550 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3664164990 con 0x7f36640d20b0 2023-10-11T09:49:39.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.315+0000 7f36637fe700 1 --2- 172.21.15.78:0/1942419550 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3664160b60 0x7f3664165190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:49:39.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.315+0000 7f3662ffd700 1 --2- 172.21.15.78:0/1942419550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36640d20b0 0x7f36640d0a50 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f3658009910 tx=0x7f3658009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a3804535e8040f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:39.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.315+0000 7f36627fc700 1 -- 172.21.15.78:0/1942419550 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3658004230 con 0x7f36640d20b0 2023-10-11T09:49:39.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.315+0000 7f36627fc700 1 -- 172.21.15.78:0/1942419550 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3658004390 con 0x7f36640d20b0 2023-10-11T09:49:39.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.315+0000 7f36627fc700 1 -- 172.21.15.78:0/1942419550 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3658009e60 con 0x7f36640d20b0 2023-10-11T09:49:39.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.315+0000 7f3668cf7700 1 -- 172.21.15.78:0/1942419550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36640d20b0 msgr2=0x7f36640d0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:39.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.315+0000 7f3668cf7700 1 --2- 172.21.15.78:0/1942419550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36640d20b0 0x7f36640d0a50 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f3658009910 tx=0x7f3658009c20 comp rx=0 tx=0).stop 2023-10-11T09:49:39.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.315+0000 7f3668cf7700 1 -- 172.21.15.78:0/1942419550 shutdown_connections 2023-10-11T09:49:39.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.315+0000 7f3668cf7700 1 --2- 172.21.15.78:0/1942419550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36640d20b0 0x7f36640d0a50 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:39.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.315+0000 7f3668cf7700 1 --2- 172.21.15.78:0/1942419550 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3664160b60 0x7f3664165190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:39.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.315+0000 7f3668cf7700 1 -- 172.21.15.78:0/1942419550 >> 172.21.15.78:0/1942419550 conn(0x7f366406afd0 msgr2=0x7f366406b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:39.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.315+0000 7f3668cf7700 1 -- 172.21.15.78:0/1942419550 shutdown_connections 2023-10-11T09:49:39.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.315+0000 7f3668cf7700 1 -- 172.21.15.78:0/1942419550 wait complete. 2023-10-11T09:49:39.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.319+0000 7f3668cf7700 1 Processor -- start 2023-10-11T09:49:39.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.319+0000 7f3668cf7700 1 -- start start 2023-10-11T09:49:39.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.319+0000 7f3668cf7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36640d20b0 0x7f36640c9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:39.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.319+0000 7f3668cf7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36640d2500 0x7f36640b7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:39.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.319+0000 7f3668cf7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3664160b60 0x7f366414bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:39.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.319+0000 7f3668cf7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36640b7da0 con 0x7f36640d20b0 2023-10-11T09:49:39.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.319+0000 7f3668cf7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36640ca330 con 0x7f3664160b60 2023-10-11T09:49:39.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.319+0000 7f3668cf7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36640ca610 con 0x7f36640d2500 2023-10-11T09:49:39.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.319+0000 7f36637fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36640d20b0 0x7f36640c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:39.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.319+0000 7f36637fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36640d20b0 0x7f36640c9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38606/0 (socket says 172.21.15.78:38606) 2023-10-11T09:49:39.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.319+0000 7f36637fe700 1 -- 172.21.15.78:0/1248979984 learned_addr learned my addr 172.21.15.78:0/1248979984 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:39.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.319+0000 7f3662ffd700 1 --2- 172.21.15.78:0/1248979984 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36640d2500 0x7f36640b7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:39.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.319+0000 7f36637fe700 1 -- 172.21.15.78:0/1248979984 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36640d2500 msgr2=0x7f36640b7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:39.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.319+0000 7f3663fff700 1 --2- 172.21.15.78:0/1248979984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3664160b60 0x7f366414bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:39.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.319+0000 7f36637fe700 1 --2- 172.21.15.78:0/1248979984 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36640d2500 0x7f36640b7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:39.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.319+0000 7f36637fe700 1 -- 172.21.15.78:0/1248979984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3664160b60 msgr2=0x7f366414bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:39.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.319+0000 7f36637fe700 1 --2- 172.21.15.78:0/1248979984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3664160b60 0x7f366414bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:39.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.319+0000 7f36637fe700 1 -- 172.21.15.78:0/1248979984 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3658009570 con 0x7f36640d20b0 2023-10-11T09:49:39.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.319+0000 7f3662ffd700 1 --2- 172.21.15.78:0/1248979984 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36640d2500 0x7f36640b7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:49:39.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.319+0000 7f36637fe700 1 --2- 172.21.15.78:0/1248979984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36640d20b0 0x7f36640c9980 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f365400c7c0 tx=0x7f365400cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:39.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.319+0000 7f364bfff700 1 -- 172.21.15.78:0/1248979984 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3654011800 con 0x7f36640d20b0 2023-10-11T09:49:39.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.323+0000 7f364bfff700 1 -- 172.21.15.78:0/1248979984 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3654011960 con 0x7f36640d20b0 2023-10-11T09:49:39.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.323+0000 7f3668cf7700 1 -- 172.21.15.78:0/1248979984 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36640b7060 con 0x7f36640d20b0 2023-10-11T09:49:39.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.323+0000 7f364bfff700 1 -- 172.21.15.78:0/1248979984 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3654011cf0 con 0x7f36640d20b0 2023-10-11T09:49:39.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.323+0000 7f3668cf7700 1 -- 172.21.15.78:0/1248979984 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36640b6d70 con 0x7f36640d20b0 2023-10-11T09:49:39.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.323+0000 7f3668cf7700 1 -- 172.21.15.78:0/1248979984 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f366415de50 con 0x7f36640d20b0 2023-10-11T09:49:39.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.323+0000 7f364bfff700 1 -- 172.21.15.78:0/1248979984 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f3654021350 con 0x7f36640d20b0 2023-10-11T09:49:39.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.323+0000 7f364bfff700 1 --2- 172.21.15.78:0/1248979984 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3650065ed0 0x7f3650068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:39.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.323+0000 7f364bfff700 1 -- 172.21.15.78:0/1248979984 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6810+0+0 (secure 0 0 0) 0x7f3654083ac0 con 0x7f36640d20b0 2023-10-11T09:49:39.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.323+0000 7f3662ffd700 1 --2- 172.21.15.78:0/1248979984 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3650065ed0 0x7f3650068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:39.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.327+0000 7f3662ffd700 1 --2- 172.21.15.78:0/1248979984 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3650065ed0 0x7f3650068380 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3658009910 tx=0x7f3658008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:39.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.327+0000 7f364bfff700 1 -- 172.21.15.78:0/1248979984 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f3654050d90 con 0x7f36640d20b0 2023-10-11T09:49:39.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:39.507+0000 7f3668cf7700 1 -- 172.21.15.78:0/1248979984 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f36640ca7d0 con 0x7f36640d20b0 2023-10-11T09:49:40.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.367+0000 7f364bfff700 1 -- 172.21.15.78:0/1248979984 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v94) v1 ==== 181+0+0 (secure 0 0 0) 0x7f36540543b0 con 0x7f36640d20b0 2023-10-11T09:49:40.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.371+0000 7f3668cf7700 1 -- 172.21.15.78:0/1248979984 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3650065ed0 msgr2=0x7f3650068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:40.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.371+0000 7f3668cf7700 1 --2- 172.21.15.78:0/1248979984 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3650065ed0 0x7f3650068380 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3658009910 tx=0x7f3658008030 comp rx=0 tx=0).stop 2023-10-11T09:49:40.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.371+0000 7f3668cf7700 1 -- 172.21.15.78:0/1248979984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36640d20b0 msgr2=0x7f36640c9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:40.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.371+0000 7f3668cf7700 1 --2- 172.21.15.78:0/1248979984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36640d20b0 0x7f36640c9980 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f365400c7c0 tx=0x7f365400cb80 comp rx=0 tx=0).stop 2023-10-11T09:49:40.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.371+0000 7f3668cf7700 1 -- 172.21.15.78:0/1248979984 shutdown_connections 2023-10-11T09:49:40.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.371+0000 7f3668cf7700 1 --2- 172.21.15.78:0/1248979984 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3650065ed0 0x7f3650068380 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:40.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.371+0000 7f3668cf7700 1 --2- 172.21.15.78:0/1248979984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3664160b60 0x7f366414bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:40.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.371+0000 7f3668cf7700 1 --2- 172.21.15.78:0/1248979984 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36640d2500 0x7f36640b7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:40.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.371+0000 7f3668cf7700 1 --2- 172.21.15.78:0/1248979984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36640d20b0 0x7f36640c9980 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:40.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.371+0000 7f3668cf7700 1 -- 172.21.15.78:0/1248979984 >> 172.21.15.78:0/1248979984 conn(0x7f366406afd0 msgr2=0x7f366415b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:40.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.375+0000 7f3668cf7700 1 -- 172.21.15.78:0/1248979984 shutdown_connections 2023-10-11T09:49:40.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.375+0000 7f3668cf7700 1 -- 172.21.15.78:0/1248979984 wait complete. 2023-10-11T09:49:40.382 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' removed 2023-10-11T09:49:40.402 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-11T09:49:40.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.575+0000 7f8050c7e700 1 Processor -- start 2023-10-11T09:49:40.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.579+0000 7f8050c7e700 1 -- start start 2023-10-11T09:49:40.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.579+0000 7f8050c7e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f804c0cefc0 0x7f804c0b52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:40.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.579+0000 7f8050c7e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f804c15cbf0 0x7f804c15d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:40.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.579+0000 7f8050c7e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f804c0d2bd0 con 0x7f804c15a0f0 2023-10-11T09:49:40.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.579+0000 7f8050c7e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f804c15d5b0 con 0x7f804c15cbf0 2023-10-11T09:49:40.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.579+0000 7f8050c7e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f804c0cf430 con 0x7f804c0cefc0 2023-10-11T09:49:40.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.579+0000 7f804b7fe700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f804c15a0f0 0x7f804c1608a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:49538/0 (socket says 172.21.15.78:49538) 2023-10-11T09:49:40.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.579+0000 7f804b7fe700 1 -- 172.21.15.78:0/428941124 learned_addr learned my addr 172.21.15.78:0/428941124 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:49:40.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.579+0000 7f804bfff700 1 --2- 172.21.15.78:0/428941124 >> v2:172.21.15.78:3300/0 conn(0x7f804c15cbf0 0x7f804c15d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:40.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.579+0000 7f8043fff700 1 --2- 172.21.15.78:0/428941124 >> v2:172.21.15.133:3300/0 conn(0x7f804c0cefc0 0x7f804c0b52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:40.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.579+0000 7f804affd700 1 -- 172.21.15.78:0/428941124 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2685017976 0 0) 0x7f804c0d2bd0 con 0x7f804c15a0f0 2023-10-11T09:49:40.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.579+0000 7f804affd700 1 -- 172.21.15.78:0/428941124 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80380035d0 con 0x7f804c15a0f0 2023-10-11T09:49:40.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.579+0000 7f804affd700 1 -- 172.21.15.78:0/428941124 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3909774133 0 0) 0x7f80380035d0 con 0x7f804c15a0f0 2023-10-11T09:49:40.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.579+0000 7f804affd700 1 -- 172.21.15.78:0/428941124 >> v2:172.21.15.133:3300/0 conn(0x7f804c0cefc0 msgr2=0x7f804c0b52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:40.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.579+0000 7f804affd700 1 --2- 172.21.15.78:0/428941124 >> v2:172.21.15.133:3300/0 conn(0x7f804c0cefc0 0x7f804c0b52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:40.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.579+0000 7f804affd700 1 -- 172.21.15.78:0/428941124 >> v2:172.21.15.78:3300/0 conn(0x7f804c15cbf0 msgr2=0x7f804c15d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:40.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.579+0000 7f804affd700 1 --2- 172.21.15.78:0/428941124 >> v2:172.21.15.78:3300/0 conn(0x7f804c15cbf0 0x7f804c15d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:40.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.579+0000 7f804affd700 1 -- 172.21.15.78:0/428941124 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f804c0cf630 con 0x7f804c15a0f0 2023-10-11T09:49:40.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.579+0000 7f804affd700 1 -- 172.21.15.78:0/428941124 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f803c001670 con 0x7f804c15a0f0 2023-10-11T09:49:40.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.583+0000 7f804affd700 1 -- 172.21.15.78:0/428941124 >> v1:172.21.15.78:6789/0 conn(0x7f804c15a0f0 legacy=0x7f804c1608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:40.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.583+0000 7f804affd700 1 --2- 172.21.15.78:0/428941124 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8038003a20 0x7f8038003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:40.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.583+0000 7f804affd700 1 --2- 172.21.15.78:0/428941124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8038004870 0x7f8038004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:40.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.583+0000 7f804affd700 1 --2- 172.21.15.78:0/428941124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8038005220 0x7f803800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:40.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.583+0000 7f804affd700 1 -- 172.21.15.78:0/428941124 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804c15d5b0 con 0x7f8038004870 2023-10-11T09:49:40.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.583+0000 7f804affd700 1 -- 172.21.15.78:0/428941124 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804c0cf430 con 0x7f8038005220 2023-10-11T09:49:40.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.583+0000 7f804affd700 1 -- 172.21.15.78:0/428941124 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f803800bfa0 con 0x7f8038003a20 2023-10-11T09:49:40.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.583+0000 7f804bfff700 1 --2- 172.21.15.78:0/428941124 >> v2:172.21.15.78:3300/0 conn(0x7f804c15cbf0 0x7f804c15d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:49:40.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.583+0000 7f8043fff700 1 --2- 172.21.15.78:0/428941124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8038004870 0x7f8038004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:40.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.583+0000 7f804b7fe700 1 --2- 172.21.15.78:0/428941124 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8038003a20 0x7f8038003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:40.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.583+0000 7f8043fff700 1 -- 172.21.15.78:0/428941124 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8038003a20 msgr2=0x7f8038003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:40.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.583+0000 7f804bfff700 1 --2- 172.21.15.78:0/428941124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8038005220 0x7f803800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:40.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.583+0000 7f8043fff700 1 --2- 172.21.15.78:0/428941124 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8038003a20 0x7f8038003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:40.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.583+0000 7f8043fff700 1 -- 172.21.15.78:0/428941124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8038005220 msgr2=0x7f803800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:40.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.583+0000 7f8043fff700 1 --2- 172.21.15.78:0/428941124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8038005220 0x7f803800ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:40.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.583+0000 7f804bfff700 1 -- 172.21.15.78:0/428941124 reap_dead start 2023-10-11T09:49:40.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.583+0000 7f8043fff700 1 -- 172.21.15.78:0/428941124 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80380035d0 con 0x7f8038004870 2023-10-11T09:49:40.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.583+0000 7f8043fff700 1 --2- 172.21.15.78:0/428941124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8038004870 0x7f8038004ce0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f8034009b70 tx=0x7f80380058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27fb15af0aad7de5 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:40.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.583+0000 7f804affd700 1 -- 172.21.15.78:0/428941124 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803400e640 con 0x7f8038004870 2023-10-11T09:49:40.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.583+0000 7f804affd700 1 -- 172.21.15.78:0/428941124 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803400e7a0 con 0x7f8038004870 2023-10-11T09:49:40.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.583+0000 7f804affd700 1 -- 172.21.15.78:0/428941124 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803400eb10 con 0x7f8038004870 2023-10-11T09:49:40.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.583+0000 7f8050c7e700 1 -- 172.21.15.78:0/428941124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8038004870 msgr2=0x7f8038004ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:40.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.583+0000 7f8050c7e700 1 --2- 172.21.15.78:0/428941124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8038004870 0x7f8038004ce0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f8034009b70 tx=0x7f80380058a0 comp rx=0 tx=0).stop 2023-10-11T09:49:40.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.583+0000 7f8050c7e700 1 -- 172.21.15.78:0/428941124 shutdown_connections 2023-10-11T09:49:40.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.583+0000 7f8050c7e700 1 --2- 172.21.15.78:0/428941124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8038004870 0x7f8038004ce0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:40.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.583+0000 7f8050c7e700 1 -- 172.21.15.78:0/428941124 >> 172.21.15.78:0/428941124 conn(0x7f804c06afd0 msgr2=0x7f804c06b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:40.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.583+0000 7f8050c7e700 1 -- 172.21.15.78:0/428941124 shutdown_connections 2023-10-11T09:49:40.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.583+0000 7f8050c7e700 1 -- 172.21.15.78:0/428941124 wait complete. 2023-10-11T09:49:40.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.587+0000 7f8050c7e700 1 Processor -- start 2023-10-11T09:49:40.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.587+0000 7f8050c7e700 1 -- start start 2023-10-11T09:49:40.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.587+0000 7f8050c7e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8038004870 0x7f804c1508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:40.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.587+0000 7f8050c7e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f804c15c160 0x7f804c157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:40.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.587+0000 7f8050c7e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f804c15c5b0 0x7f804c157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:40.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.587+0000 7f8050c7e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804c0c8410 con 0x7f804c15c5b0 2023-10-11T09:49:40.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.587+0000 7f8050c7e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804c15b390 con 0x7f804c15c160 2023-10-11T09:49:40.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.587+0000 7f8050c7e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f804c0b6b30 con 0x7f8038004870 2023-10-11T09:49:40.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.587+0000 7f804bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f804c15c5b0 0x7f804c157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:40.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.587+0000 7f804bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f804c15c5b0 0x7f804c157a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38640/0 (socket says 172.21.15.78:38640) 2023-10-11T09:49:40.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.587+0000 7f804bfff700 1 -- 172.21.15.78:0/3584338699 learned_addr learned my addr 172.21.15.78:0/3584338699 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:40.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.587+0000 7f8043fff700 1 --2- 172.21.15.78:0/3584338699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f804c15c160 0x7f804c157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:40.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.591+0000 7f804b7fe700 1 --2- 172.21.15.78:0/3584338699 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8038004870 0x7f804c1508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:40.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.591+0000 7f804bfff700 1 -- 172.21.15.78:0/3584338699 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8038004870 msgr2=0x7f804c1508d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:40.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.591+0000 7f804bfff700 1 --2- 172.21.15.78:0/3584338699 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8038004870 0x7f804c1508d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:40.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.591+0000 7f804bfff700 1 -- 172.21.15.78:0/3584338699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f804c15c160 msgr2=0x7f804c157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:40.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.591+0000 7f804bfff700 1 --2- 172.21.15.78:0/3584338699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f804c15c160 0x7f804c157520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:40.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.591+0000 7f804bfff700 1 -- 172.21.15.78:0/3584338699 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f804c15a4c0 con 0x7f804c15c5b0 2023-10-11T09:49:40.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.591+0000 7f804bfff700 1 --2- 172.21.15.78:0/3584338699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f804c15c5b0 0x7f804c157a60 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f804c06dbb0 tx=0x7f804400f660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:40.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.591+0000 7f8048ff9700 1 -- 172.21.15.78:0/3584338699 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8044009ea0 con 0x7f804c15c5b0 2023-10-11T09:49:40.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.591+0000 7f8048ff9700 1 -- 172.21.15.78:0/3584338699 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f804400fec0 con 0x7f804c15c5b0 2023-10-11T09:49:40.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.591+0000 7f8050c7e700 1 -- 172.21.15.78:0/3584338699 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f804c15a780 con 0x7f804c15c5b0 2023-10-11T09:49:40.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.591+0000 7f8048ff9700 1 -- 172.21.15.78:0/3584338699 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8044003c80 con 0x7f804c15c5b0 2023-10-11T09:49:40.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.591+0000 7f8050c7e700 1 -- 172.21.15.78:0/3584338699 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f804c15ad40 con 0x7f804c15c5b0 2023-10-11T09:49:40.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.591+0000 7f8050c7e700 1 -- 172.21.15.78:0/3584338699 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f804c08bb90 con 0x7f804c15c5b0 2023-10-11T09:49:40.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.591+0000 7f8048ff9700 1 -- 172.21.15.78:0/3584338699 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f8044004030 con 0x7f804c15c5b0 2023-10-11T09:49:40.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.595+0000 7f8048ff9700 1 --2- 172.21.15.78:0/3584338699 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f802c065f00 0x7f802c0683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:40.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.595+0000 7f8048ff9700 1 -- 172.21.15.78:0/3584338699 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6372+0+0 (secure 0 0 0) 0x7f80440841a0 con 0x7f804c15c5b0 2023-10-11T09:49:40.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.595+0000 7f804b7fe700 1 --2- 172.21.15.78:0/3584338699 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f802c065f00 0x7f802c0683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:40.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.595+0000 7f804b7fe700 1 --2- 172.21.15.78:0/3584338699 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f802c065f00 0x7f802c0683b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f804c0694e0 tx=0x7f803c007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:40.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.599+0000 7f8048ff9700 1 -- 172.21.15.78:0/3584338699 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f80440516a0 con 0x7f804c15c5b0 2023-10-11T09:49:40.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:40.771+0000 7f8050c7e700 1 -- 172.21.15.78:0/3584338699 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f804c15b4d0 con 0x7f804c15c5b0 2023-10-11T09:49:41.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.375+0000 7f8048ff9700 1 -- 172.21.15.78:0/3584338699 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v95) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8044054cc0 con 0x7f804c15c5b0 2023-10-11T09:49:41.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.383+0000 7f8050c7e700 1 -- 172.21.15.78:0/3584338699 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f802c065f00 msgr2=0x7f802c0683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:41.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.383+0000 7f8050c7e700 1 --2- 172.21.15.78:0/3584338699 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f802c065f00 0x7f802c0683b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f804c0694e0 tx=0x7f803c007040 comp rx=0 tx=0).stop 2023-10-11T09:49:41.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.383+0000 7f8050c7e700 1 -- 172.21.15.78:0/3584338699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f804c15c5b0 msgr2=0x7f804c157a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:41.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.383+0000 7f8050c7e700 1 --2- 172.21.15.78:0/3584338699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f804c15c5b0 0x7f804c157a60 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f804c06dbb0 tx=0x7f804400f660 comp rx=0 tx=0).stop 2023-10-11T09:49:41.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.383+0000 7f8050c7e700 1 -- 172.21.15.78:0/3584338699 shutdown_connections 2023-10-11T09:49:41.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.383+0000 7f8050c7e700 1 --2- 172.21.15.78:0/3584338699 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f802c065f00 0x7f802c0683b0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:41.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.383+0000 7f8050c7e700 1 --2- 172.21.15.78:0/3584338699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f804c15c5b0 0x7f804c157a60 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:41.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.383+0000 7f8050c7e700 1 --2- 172.21.15.78:0/3584338699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f804c15c160 0x7f804c157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:41.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.383+0000 7f8050c7e700 1 --2- 172.21.15.78:0/3584338699 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8038004870 0x7f804c1508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:41.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.383+0000 7f8050c7e700 1 -- 172.21.15.78:0/3584338699 >> 172.21.15.78:0/3584338699 conn(0x7f804c06afd0 msgr2=0x7f804c152550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:41.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.383+0000 7f8050c7e700 1 -- 172.21.15.78:0/3584338699 shutdown_connections 2023-10-11T09:49:41.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.383+0000 7f8050c7e700 1 -- 172.21.15.78:0/3584338699 wait complete. 2023-10-11T09:49:41.392 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' removed 2023-10-11T09:49:41.412 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_extra cephfs_data_extra --yes-i-really-really-mean-it 2023-10-11T09:49:41.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.583+0000 7f732e5d5700 1 Processor -- start 2023-10-11T09:49:41.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.583+0000 7f732e5d5700 1 -- start start 2023-10-11T09:49:41.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.583+0000 7f732e5d5700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f73280cefc0 0x7f73280b52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:41.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.583+0000 7f732e5d5700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f732815cbf0 0x7f732815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:41.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.583+0000 7f732e5d5700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73280d2bd0 con 0x7f732815a0f0 2023-10-11T09:49:41.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.583+0000 7f732e5d5700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73280cf430 con 0x7f732815cbf0 2023-10-11T09:49:41.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.583+0000 7f732e5d5700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73280b5820 con 0x7f73280cefc0 2023-10-11T09:49:41.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.583+0000 7f732cdd2700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f73280cefc0 0x7f73280b52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:41.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.583+0000 7f732cdd2700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f73280cefc0 0x7f73280b52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:45482/0 (socket says 172.21.15.78:45482) 2023-10-11T09:49:41.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.583+0000 7f732cdd2700 1 -- 172.21.15.78:0/1356434231 learned_addr learned my addr 172.21.15.78:0/1356434231 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:41.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.583+0000 7f732ddd4700 1 --2- 172.21.15.78:0/1356434231 >> v2:172.21.15.133:3300/0 conn(0x7f732815cbf0 0x7f732815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:41.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.587+0000 7f732cdd2700 1 -- 172.21.15.78:0/1356434231 >> v2:172.21.15.133:3300/0 conn(0x7f732815cbf0 msgr2=0x7f732815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:41.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.587+0000 7f732cdd2700 1 --2- 172.21.15.78:0/1356434231 >> v2:172.21.15.133:3300/0 conn(0x7f732815cbf0 0x7f732815d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:41.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.587+0000 7f732cdd2700 1 -- 172.21.15.78:0/1356434231 >> v1:172.21.15.133:6789/0 conn(0x7f732815a0f0 legacy=0x7f73281608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:41.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.587+0000 7f732cdd2700 1 -- 172.21.15.78:0/1356434231 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73280cf630 con 0x7f73280cefc0 2023-10-11T09:49:41.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.587+0000 7f732cdd2700 1 --2- 172.21.15.78:0/1356434231 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f73280cefc0 0x7f73280b52e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7318009910 tx=0x7f7318009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b994af523ab4b0e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:41.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.587+0000 7f731ffff700 1 -- 172.21.15.78:0/1356434231 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7318011830 con 0x7f73280cefc0 2023-10-11T09:49:41.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.587+0000 7f731ffff700 1 -- 172.21.15.78:0/1356434231 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7318011990 con 0x7f73280cefc0 2023-10-11T09:49:41.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.587+0000 7f731ffff700 1 -- 172.21.15.78:0/1356434231 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7318011d40 con 0x7f73280cefc0 2023-10-11T09:49:41.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.587+0000 7f732e5d5700 1 -- 172.21.15.78:0/1356434231 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f73280cefc0 msgr2=0x7f73280b52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:41.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.587+0000 7f732e5d5700 1 --2- 172.21.15.78:0/1356434231 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f73280cefc0 0x7f73280b52e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7318009910 tx=0x7f7318009c20 comp rx=0 tx=0).stop 2023-10-11T09:49:41.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.587+0000 7f732e5d5700 1 -- 172.21.15.78:0/1356434231 shutdown_connections 2023-10-11T09:49:41.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.587+0000 7f732e5d5700 1 --2- 172.21.15.78:0/1356434231 >> v2:172.21.15.133:3300/0 conn(0x7f732815cbf0 0x7f732815d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:41.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.587+0000 7f732e5d5700 1 --2- 172.21.15.78:0/1356434231 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f73280cefc0 0x7f73280b52e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:41.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.587+0000 7f732e5d5700 1 -- 172.21.15.78:0/1356434231 >> 172.21.15.78:0/1356434231 conn(0x7f732806afd0 msgr2=0x7f732806b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:41.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.587+0000 7f732e5d5700 1 -- 172.21.15.78:0/1356434231 shutdown_connections 2023-10-11T09:49:41.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.587+0000 7f732e5d5700 1 -- 172.21.15.78:0/1356434231 wait complete. 2023-10-11T09:49:41.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.591+0000 7f732e5d5700 1 Processor -- start 2023-10-11T09:49:41.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.591+0000 7f732e5d5700 1 -- start start 2023-10-11T09:49:41.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.591+0000 7f732e5d5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73280cefc0 0x7f73280b85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:41.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.591+0000 7f732e5d5700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f732815a0f0 0x7f73280cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:41.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.591+0000 7f732e5d5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f732815cbf0 0x7f73281f21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:41.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.591+0000 7f732e5d5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73280b8b30 con 0x7f732815cbf0 2023-10-11T09:49:41.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.591+0000 7f732e5d5700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73280cc4c0 con 0x7f73280cefc0 2023-10-11T09:49:41.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.591+0000 7f732e5d5700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73280cc7a0 con 0x7f732815a0f0 2023-10-11T09:49:41.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.591+0000 7f732ddd4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f732815cbf0 0x7f73281f21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:41.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.591+0000 7f732ddd4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f732815cbf0 0x7f73281f21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38654/0 (socket says 172.21.15.78:38654) 2023-10-11T09:49:41.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.591+0000 7f732d5d3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73280cefc0 0x7f73280b85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:41.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.591+0000 7f732ddd4700 1 -- 172.21.15.78:0/4272765819 learned_addr learned my addr 172.21.15.78:0/4272765819 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:41.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.591+0000 7f732cdd2700 1 --2- 172.21.15.78:0/4272765819 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f732815a0f0 0x7f73280cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:41.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.591+0000 7f732ddd4700 1 -- 172.21.15.78:0/4272765819 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f732815a0f0 msgr2=0x7f73280cbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:41.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.591+0000 7f732ddd4700 1 --2- 172.21.15.78:0/4272765819 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f732815a0f0 0x7f73280cbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:41.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.591+0000 7f732ddd4700 1 -- 172.21.15.78:0/4272765819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73280cefc0 msgr2=0x7f73280b85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:41.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.591+0000 7f732ddd4700 1 --2- 172.21.15.78:0/4272765819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73280cefc0 0x7f73280b85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:41.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.591+0000 7f732ddd4700 1 -- 172.21.15.78:0/4272765819 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7318009570 con 0x7f732815cbf0 2023-10-11T09:49:41.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.591+0000 7f732d5d3700 1 --2- 172.21.15.78:0/4272765819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73280cefc0 0x7f73280b85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:49:41.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.595+0000 7f732ddd4700 1 --2- 172.21.15.78:0/4272765819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f732815cbf0 0x7f73281f21f0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f732000efc0 tx=0x7f732000c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:41.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.595+0000 7f731dffb700 1 -- 172.21.15.78:0/4272765819 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732000cad0 con 0x7f732815cbf0 2023-10-11T09:49:41.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.595+0000 7f731dffb700 1 -- 172.21.15.78:0/4272765819 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732000f040 con 0x7f732815cbf0 2023-10-11T09:49:41.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.595+0000 7f731dffb700 1 -- 172.21.15.78:0/4272765819 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732000ce70 con 0x7f732815cbf0 2023-10-11T09:49:41.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.595+0000 7f732e5d5700 1 -- 172.21.15.78:0/4272765819 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73280cca80 con 0x7f732815cbf0 2023-10-11T09:49:41.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.595+0000 7f732e5d5700 1 -- 172.21.15.78:0/4272765819 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73280cc010 con 0x7f732815cbf0 2023-10-11T09:49:41.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.595+0000 7f732e5d5700 1 -- 172.21.15.78:0/4272765819 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f732808bb90 con 0x7f732815cbf0 2023-10-11T09:49:41.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.599+0000 7f731dffb700 1 -- 172.21.15.78:0/4272765819 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f7320014480 con 0x7f732815cbf0 2023-10-11T09:49:41.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.599+0000 7f731dffb700 1 --2- 172.21.15.78:0/4272765819 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f730c065ed0 0x7f730c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:41.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.599+0000 7f731dffb700 1 -- 172.21.15.78:0/4272765819 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f7320083f00 con 0x7f732815cbf0 2023-10-11T09:49:41.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.599+0000 7f732d5d3700 1 --2- 172.21.15.78:0/4272765819 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f730c065ed0 0x7f730c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:41.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.599+0000 7f732d5d3700 1 --2- 172.21.15.78:0/4272765819 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f730c065ed0 0x7f730c068380 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7324000f60 tx=0x7f7324009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:41.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.607+0000 7f731dffb700 1 -- 172.21.15.78:0/4272765819 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f7320051650 con 0x7f732815cbf0 2023-10-11T09:49:41.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:41.779+0000 7f732e5d5700 1 -- 172.21.15.78:0/4272765819 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_extra", "pool2": "cephfs_data_extra", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f73280cc200 con 0x7f732815cbf0 2023-10-11T09:49:42.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.387+0000 7f731dffb700 1 -- 172.21.15.78:0/4272765819 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_extra", "pool2": "cephfs_data_extra", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_extra' removed v96) v1 ==== 187+0+0 (secure 0 0 0) 0x7f73200180e0 con 0x7f732815cbf0 2023-10-11T09:49:42.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.391+0000 7f732e5d5700 1 -- 172.21.15.78:0/4272765819 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f730c065ed0 msgr2=0x7f730c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:42.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.391+0000 7f732e5d5700 1 --2- 172.21.15.78:0/4272765819 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f730c065ed0 0x7f730c068380 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7324000f60 tx=0x7f7324009040 comp rx=0 tx=0).stop 2023-10-11T09:49:42.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.391+0000 7f732e5d5700 1 -- 172.21.15.78:0/4272765819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f732815cbf0 msgr2=0x7f73281f21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:42.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.391+0000 7f732e5d5700 1 --2- 172.21.15.78:0/4272765819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f732815cbf0 0x7f73281f21f0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f732000efc0 tx=0x7f732000c0c0 comp rx=0 tx=0).stop 2023-10-11T09:49:42.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.391+0000 7f732e5d5700 1 -- 172.21.15.78:0/4272765819 shutdown_connections 2023-10-11T09:49:42.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.391+0000 7f732e5d5700 1 --2- 172.21.15.78:0/4272765819 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f730c065ed0 0x7f730c068380 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:42.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.391+0000 7f732e5d5700 1 --2- 172.21.15.78:0/4272765819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f732815cbf0 0x7f73281f21f0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:42.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.391+0000 7f732e5d5700 1 --2- 172.21.15.78:0/4272765819 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f732815a0f0 0x7f73280cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:42.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.391+0000 7f732e5d5700 1 --2- 172.21.15.78:0/4272765819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73280cefc0 0x7f73280b85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:42.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.391+0000 7f732e5d5700 1 -- 172.21.15.78:0/4272765819 >> 172.21.15.78:0/4272765819 conn(0x7f732806afd0 msgr2=0x7f7328159050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:42.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.391+0000 7f732e5d5700 1 -- 172.21.15.78:0/4272765819 shutdown_connections 2023-10-11T09:49:42.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.391+0000 7f732e5d5700 1 -- 172.21.15.78:0/4272765819 wait complete. 2023-10-11T09:49:42.401 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data_extra' removed 2023-10-11T09:49:42.421 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-11T09:49:42.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.595+0000 7f05e6ba8700 1 Processor -- start 2023-10-11T09:49:42.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.595+0000 7f05e6ba8700 1 -- start start 2023-10-11T09:49:42.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.595+0000 7f05e6ba8700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f05e00b3240 0x7f05e00d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:42.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.595+0000 7f05e6ba8700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f05e00b3610 0x7f05e00b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:42.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.599+0000 7f05e6ba8700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f05e015d040 0x7f05e015d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:42.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.599+0000 7f05e6ba8700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05e00b3fd0 con 0x7f05e00b3610 2023-10-11T09:49:42.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.599+0000 7f05e6ba8700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05e00b47f0 con 0x7f05e00b3240 2023-10-11T09:49:42.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.599+0000 7f05e6ba8700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e00b8130 con 0x7f05e015d040 2023-10-11T09:49:42.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.599+0000 7f05e53a5700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f05e00b3610 0x7f05e00b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:42.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.599+0000 7f05e53a5700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f05e00b3610 0x7f05e00b3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38662/0 (socket says 172.21.15.78:38662) 2023-10-11T09:49:42.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.599+0000 7f05e53a5700 1 -- 172.21.15.78:0/4203886991 learned_addr learned my addr 172.21.15.78:0/4203886991 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:42.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.599+0000 7f05e63a7700 1 --2- 172.21.15.78:0/4203886991 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f05e015d040 0x7f05e015d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:42.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.599+0000 7f05e5ba6700 1 --2- 172.21.15.78:0/4203886991 >> v2:172.21.15.133:3300/0 conn(0x7f05e00b3240 0x7f05e00d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:42.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.599+0000 7f05e53a5700 1 -- 172.21.15.78:0/4203886991 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f05e015d040 msgr2=0x7f05e015d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:42.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.599+0000 7f05e53a5700 1 --2- 172.21.15.78:0/4203886991 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f05e015d040 0x7f05e015d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:42.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.599+0000 7f05e53a5700 1 -- 172.21.15.78:0/4203886991 >> v2:172.21.15.133:3300/0 conn(0x7f05e00b3240 msgr2=0x7f05e00d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:42.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.599+0000 7f05e53a5700 1 --2- 172.21.15.78:0/4203886991 >> v2:172.21.15.133:3300/0 conn(0x7f05e00b3240 0x7f05e00d29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:42.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.599+0000 7f05e53a5700 1 -- 172.21.15.78:0/4203886991 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e00b41e0 con 0x7f05e00b3610 2023-10-11T09:49:42.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.599+0000 7f05e53a5700 1 --2- 172.21.15.78:0/4203886991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05e00b3610 0x7f05e00b3a90 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f05e006aa40 tx=0x7f05d0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee18011f1bc025f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:42.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.599+0000 7f05e4ba4700 1 -- 172.21.15.78:0/4203886991 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d0011830 con 0x7f05e00b3610 2023-10-11T09:49:42.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.599+0000 7f05e4ba4700 1 -- 172.21.15.78:0/4203886991 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d0011990 con 0x7f05e00b3610 2023-10-11T09:49:42.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.599+0000 7f05e4ba4700 1 -- 172.21.15.78:0/4203886991 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d0011d20 con 0x7f05e00b3610 2023-10-11T09:49:42.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.599+0000 7f05e6ba8700 1 -- 172.21.15.78:0/4203886991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05e00b3610 msgr2=0x7f05e00b3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:42.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.599+0000 7f05e6ba8700 1 --2- 172.21.15.78:0/4203886991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05e00b3610 0x7f05e00b3a90 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f05e006aa40 tx=0x7f05d0009bf0 comp rx=0 tx=0).stop 2023-10-11T09:49:42.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.603+0000 7f05e6ba8700 1 -- 172.21.15.78:0/4203886991 shutdown_connections 2023-10-11T09:49:42.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.603+0000 7f05e6ba8700 1 --2- 172.21.15.78:0/4203886991 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f05e015d040 0x7f05e015d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:42.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.603+0000 7f05e6ba8700 1 --2- 172.21.15.78:0/4203886991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05e00b3610 0x7f05e00b3a90 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:42.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.603+0000 7f05e6ba8700 1 --2- 172.21.15.78:0/4203886991 >> v2:172.21.15.133:3300/0 conn(0x7f05e00b3240 0x7f05e00d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:42.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.603+0000 7f05e6ba8700 1 -- 172.21.15.78:0/4203886991 >> 172.21.15.78:0/4203886991 conn(0x7f05e009c260 msgr2=0x7f05e00d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:42.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.603+0000 7f05e6ba8700 1 -- 172.21.15.78:0/4203886991 shutdown_connections 2023-10-11T09:49:42.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.603+0000 7f05e6ba8700 1 -- 172.21.15.78:0/4203886991 wait complete. 2023-10-11T09:49:42.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.603+0000 7f05e6ba8700 1 Processor -- start 2023-10-11T09:49:42.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.603+0000 7f05e6ba8700 1 -- start start 2023-10-11T09:49:42.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.607+0000 7f05e6ba8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05e00b3240 0x7f05e015ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:42.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.607+0000 7f05e6ba8700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f05e00b3610 0x7f05e01592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:42.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.607+0000 7f05e6ba8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05e015d040 0x7f05e01f5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:42.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.607+0000 7f05e6ba8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e00b29f0 con 0x7f05e015d040 2023-10-11T09:49:42.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.607+0000 7f05e6ba8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e015b080 con 0x7f05e00b3240 2023-10-11T09:49:42.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.607+0000 7f05e6ba8700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e0159810 con 0x7f05e00b3610 2023-10-11T09:49:42.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.607+0000 7f05e63a7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05e015d040 0x7f05e01f5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:42.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.607+0000 7f05e63a7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05e015d040 0x7f05e01f5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38666/0 (socket says 172.21.15.78:38666) 2023-10-11T09:49:42.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.607+0000 7f05e63a7700 1 -- 172.21.15.78:0/84283980 learned_addr learned my addr 172.21.15.78:0/84283980 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:42.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.607+0000 7f05e5ba6700 1 --2- 172.21.15.78:0/84283980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05e00b3240 0x7f05e015ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:42.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.607+0000 7f05e53a5700 1 --2- 172.21.15.78:0/84283980 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f05e00b3610 0x7f05e01592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:42.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.607+0000 7f05e63a7700 1 -- 172.21.15.78:0/84283980 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f05e00b3610 msgr2=0x7f05e01592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:42.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.607+0000 7f05e63a7700 1 --2- 172.21.15.78:0/84283980 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f05e00b3610 0x7f05e01592d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:42.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.607+0000 7f05e63a7700 1 -- 172.21.15.78:0/84283980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05e00b3240 msgr2=0x7f05e015ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:42.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.607+0000 7f05e63a7700 1 --2- 172.21.15.78:0/84283980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05e00b3240 0x7f05e015ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:42.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.607+0000 7f05e63a7700 1 -- 172.21.15.78:0/84283980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d0009570 con 0x7f05e015d040 2023-10-11T09:49:42.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.607+0000 7f05e53a5700 1 --2- 172.21.15.78:0/84283980 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f05e00b3610 0x7f05e01592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:49:42.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.607+0000 7f05e63a7700 1 --2- 172.21.15.78:0/84283980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05e015d040 0x7f05e01f5250 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f05d800efc0 tx=0x7f05d800c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:42.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.607+0000 7f05d67fc700 1 -- 172.21.15.78:0/84283980 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d800cb80 con 0x7f05e015d040 2023-10-11T09:49:42.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.607+0000 7f05d67fc700 1 -- 172.21.15.78:0/84283980 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d800f040 con 0x7f05e015d040 2023-10-11T09:49:42.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.607+0000 7f05e6ba8700 1 -- 172.21.15.78:0/84283980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05e00b2430 con 0x7f05e015d040 2023-10-11T09:49:42.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.607+0000 7f05d67fc700 1 -- 172.21.15.78:0/84283980 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d8014480 con 0x7f05e015d040 2023-10-11T09:49:42.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.607+0000 7f05e6ba8700 1 -- 172.21.15.78:0/84283980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05e00c7cb0 con 0x7f05e015d040 2023-10-11T09:49:42.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.611+0000 7f05e6ba8700 1 -- 172.21.15.78:0/84283980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05e00c7b10 con 0x7f05e015d040 2023-10-11T09:49:42.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.611+0000 7f05d67fc700 1 -- 172.21.15.78:0/84283980 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f05d80145e0 con 0x7f05e015d040 2023-10-11T09:49:42.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.615+0000 7f05d67fc700 1 --2- 172.21.15.78:0/84283980 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f05c4065ed0 0x7f05c4068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:42.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.615+0000 7f05d67fc700 1 -- 172.21.15.78:0/84283980 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f05d8084110 con 0x7f05e015d040 2023-10-11T09:49:42.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.615+0000 7f05e5ba6700 1 --2- 172.21.15.78:0/84283980 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f05c4065ed0 0x7f05c4068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:42.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.615+0000 7f05e5ba6700 1 --2- 172.21.15.78:0/84283980 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f05c4065ed0 0x7f05c4068380 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f05dc009ad0 tx=0x7f05dc009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:42.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.619+0000 7f05d67fc700 1 -- 172.21.15.78:0/84283980 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f05e00c7b10 con 0x7f05e015d040 2023-10-11T09:49:42.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.791+0000 7f05e6ba8700 1 -- 172.21.15.78:0/84283980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f05e00c7b10 con 0x7f05e015d040 2023-10-11T09:49:42.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.799+0000 7f05d67fc700 1 -- 172.21.15.78:0/84283980 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f05e00c7b10 con 0x7f05e015d040 2023-10-11T09:49:42.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.803+0000 7f05e6ba8700 1 -- 172.21.15.78:0/84283980 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f05c4065ed0 msgr2=0x7f05c4068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:42.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.803+0000 7f05e6ba8700 1 --2- 172.21.15.78:0/84283980 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f05c4065ed0 0x7f05c4068380 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f05dc009ad0 tx=0x7f05dc009240 comp rx=0 tx=0).stop 2023-10-11T09:49:42.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.803+0000 7f05e6ba8700 1 -- 172.21.15.78:0/84283980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05e015d040 msgr2=0x7f05e01f5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:42.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.803+0000 7f05e6ba8700 1 --2- 172.21.15.78:0/84283980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05e015d040 0x7f05e01f5250 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f05d800efc0 tx=0x7f05d800c0c0 comp rx=0 tx=0).stop 2023-10-11T09:49:42.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.803+0000 7f05e6ba8700 1 -- 172.21.15.78:0/84283980 shutdown_connections 2023-10-11T09:49:42.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.803+0000 7f05e6ba8700 1 --2- 172.21.15.78:0/84283980 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f05c4065ed0 0x7f05c4068380 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:42.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.803+0000 7f05e6ba8700 1 --2- 172.21.15.78:0/84283980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05e015d040 0x7f05e01f5250 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:42.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.803+0000 7f05e6ba8700 1 --2- 172.21.15.78:0/84283980 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f05e00b3610 0x7f05e01592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:42.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.803+0000 7f05e6ba8700 1 --2- 172.21.15.78:0/84283980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05e00b3240 0x7f05e015ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:42.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.803+0000 7f05e6ba8700 1 -- 172.21.15.78:0/84283980 >> 172.21.15.78:0/84283980 conn(0x7f05e009c260 msgr2=0x7f05e00b8460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:42.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.803+0000 7f05e6ba8700 1 -- 172.21.15.78:0/84283980 shutdown_connections 2023-10-11T09:49:42.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.803+0000 7f05e6ba8700 1 -- 172.21.15.78:0/84283980 wait complete. 2023-10-11T09:49:42.833 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_simple' 2023-10-11T09:49:43.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.999+0000 7fb1e6ea3700 1 Processor -- start 2023-10-11T09:49:43.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.999+0000 7fb1e6ea3700 1 -- start start 2023-10-11T09:49:43.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.999+0000 7fb1e6ea3700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb1e0071040 0x7fb1e00cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:43.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.999+0000 7fb1e6ea3700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb1e00cba00 0x7fb1e006c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:43.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.999+0000 7fb1e6ea3700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1e00720f0 con 0x7fb1e0154e00 2023-10-11T09:49:43.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.999+0000 7fb1e6ea3700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1e00714e0 con 0x7fb1e0071040 2023-10-11T09:49:43.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:42.999+0000 7fb1e6ea3700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e00717c0 con 0x7fb1e00cba00 2023-10-11T09:49:43.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.003+0000 7fb1e5440700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb1e00cba00 0x7fb1e006c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:43.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.003+0000 7fb1e4c3f700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fb1e0154e00 0x7fb1e015b650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:43032/0 (socket says 172.21.15.78:43032) 2023-10-11T09:49:43.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.003+0000 7fb1e4c3f700 1 -- 172.21.15.78:0/3291052683 learned_addr learned my addr 172.21.15.78:0/3291052683 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:49:43.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.003+0000 7fb1d7fff700 1 --2- 172.21.15.78:0/3291052683 >> v2:172.21.15.133:3300/0 conn(0x7fb1e0071040 0x7fb1e00cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:43.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.003+0000 7fb1d7fff700 1 -- 172.21.15.78:0/3291052683 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb1e00cba00 msgr2=0x7fb1e006c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:43.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.003+0000 7fb1d7fff700 1 --2- 172.21.15.78:0/3291052683 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb1e00cba00 0x7fb1e006c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:43.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.003+0000 7fb1d7fff700 1 -- 172.21.15.78:0/3291052683 >> v1:172.21.15.133:6789/0 conn(0x7fb1e0154e00 legacy=0x7fb1e015b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:43.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.003+0000 7fb1d7fff700 1 -- 172.21.15.78:0/3291052683 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e0071aa0 con 0x7fb1e0071040 2023-10-11T09:49:43.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.003+0000 7fb1d7fff700 1 --2- 172.21.15.78:0/3291052683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1e0071040 0x7fb1e00cb4c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0009850 tx=0x7fb1d0009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1a53177c213260a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:43.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.003+0000 7fb1d77fe700 1 -- 172.21.15.78:0/3291052683 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d0011830 con 0x7fb1e0071040 2023-10-11T09:49:43.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.003+0000 7fb1d77fe700 1 -- 172.21.15.78:0/3291052683 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d0011990 con 0x7fb1e0071040 2023-10-11T09:49:43.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.003+0000 7fb1d77fe700 1 -- 172.21.15.78:0/3291052683 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d0011d40 con 0x7fb1e0071040 2023-10-11T09:49:43.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.007+0000 7fb1e6ea3700 1 -- 172.21.15.78:0/3291052683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1e0071040 msgr2=0x7fb1e00cb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:43.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.007+0000 7fb1e6ea3700 1 --2- 172.21.15.78:0/3291052683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1e0071040 0x7fb1e00cb4c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0009850 tx=0x7fb1d0009b60 comp rx=0 tx=0).stop 2023-10-11T09:49:43.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.007+0000 7fb1e6ea3700 1 -- 172.21.15.78:0/3291052683 shutdown_connections 2023-10-11T09:49:43.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.007+0000 7fb1e6ea3700 1 --2- 172.21.15.78:0/3291052683 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb1e00cba00 0x7fb1e006c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:43.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.007+0000 7fb1e6ea3700 1 --2- 172.21.15.78:0/3291052683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1e0071040 0x7fb1e00cb4c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:43.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.007+0000 7fb1e6ea3700 1 -- 172.21.15.78:0/3291052683 >> 172.21.15.78:0/3291052683 conn(0x7fb1e006aca0 msgr2=0x7fb1e00b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:43.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.007+0000 7fb1e6ea3700 1 -- 172.21.15.78:0/3291052683 shutdown_connections 2023-10-11T09:49:43.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.007+0000 7fb1e6ea3700 1 -- 172.21.15.78:0/3291052683 wait complete. 2023-10-11T09:49:43.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.011+0000 7fb1e6ea3700 1 Processor -- start 2023-10-11T09:49:43.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.011+0000 7fb1e6ea3700 1 -- start start 2023-10-11T09:49:43.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.011+0000 7fb1e6ea3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1e0071040 0x7fb1e0157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:43.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.011+0000 7fb1e6ea3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1e00cba00 0x7fb1e015b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:43.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.011+0000 7fb1e6ea3700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb1e0154e00 0x7fb1e00aec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:43.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.011+0000 7fb1e6ea3700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e00b2a90 con 0x7fb1e00cba00 2023-10-11T09:49:43.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.011+0000 7fb1e6ea3700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e00b1e10 con 0x7fb1e0071040 2023-10-11T09:49:43.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.011+0000 7fb1e6ea3700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e00b20f0 con 0x7fb1e0154e00 2023-10-11T09:49:43.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.011+0000 7fb1d7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1e00cba00 0x7fb1e015b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:43.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.011+0000 7fb1d7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1e00cba00 0x7fb1e015b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38672/0 (socket says 172.21.15.78:38672) 2023-10-11T09:49:43.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.011+0000 7fb1d7fff700 1 -- 172.21.15.78:0/3505182062 learned_addr learned my addr 172.21.15.78:0/3505182062 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:43.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.011+0000 7fb1e4c3f700 1 --2- 172.21.15.78:0/3505182062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1e0071040 0x7fb1e0157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:43.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.011+0000 7fb1e5440700 1 --2- 172.21.15.78:0/3505182062 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb1e0154e00 0x7fb1e00aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:43.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.011+0000 7fb1d7fff700 1 -- 172.21.15.78:0/3505182062 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb1e0154e00 msgr2=0x7fb1e00aec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:43.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.011+0000 7fb1d7fff700 1 --2- 172.21.15.78:0/3505182062 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb1e0154e00 0x7fb1e00aec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:43.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.011+0000 7fb1d7fff700 1 -- 172.21.15.78:0/3505182062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1e0071040 msgr2=0x7fb1e0157970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:43.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.011+0000 7fb1d7fff700 1 --2- 172.21.15.78:0/3505182062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1e0071040 0x7fb1e0157970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:43.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.011+0000 7fb1d7fff700 1 -- 172.21.15.78:0/3505182062 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d00094b0 con 0x7fb1e00cba00 2023-10-11T09:49:43.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.015+0000 7fb1d7fff700 1 --2- 172.21.15.78:0/3505182062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1e00cba00 0x7fb1e015b230 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fb1d00090b0 tx=0x7fb1d0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:43.030 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.015+0000 7fb1d57fa700 1 -- 172.21.15.78:0/3505182062 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d0004480 con 0x7fb1e00cba00 2023-10-11T09:49:43.030 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.015+0000 7fb1d57fa700 1 -- 172.21.15.78:0/3505182062 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d001f070 con 0x7fb1e00cba00 2023-10-11T09:49:43.030 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.015+0000 7fb1e6ea3700 1 -- 172.21.15.78:0/3505182062 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1e00b23d0 con 0x7fb1e00cba00 2023-10-11T09:49:43.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.015+0000 7fb1d57fa700 1 -- 172.21.15.78:0/3505182062 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d0003900 con 0x7fb1e00cba00 2023-10-11T09:49:43.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.015+0000 7fb1e6ea3700 1 -- 172.21.15.78:0/3505182062 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e0157eb0 con 0x7fb1e00cba00 2023-10-11T09:49:43.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.015+0000 7fb1e6ea3700 1 -- 172.21.15.78:0/3505182062 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1e004e530 con 0x7fb1e00cba00 2023-10-11T09:49:43.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.019+0000 7fb1d57fa700 1 -- 172.21.15.78:0/3505182062 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fb1d0021030 con 0x7fb1e00cba00 2023-10-11T09:49:43.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.019+0000 7fb1d57fa700 1 --2- 172.21.15.78:0/3505182062 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb1cc065ed0 0x7fb1cc068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:43.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.019+0000 7fb1d57fa700 1 -- 172.21.15.78:0/3505182062 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fb1d0084940 con 0x7fb1e00cba00 2023-10-11T09:49:43.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.019+0000 7fb1e4c3f700 1 --2- 172.21.15.78:0/3505182062 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb1cc065ed0 0x7fb1cc068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:43.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.019+0000 7fb1e4c3f700 1 --2- 172.21.15.78:0/3505182062 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb1cc065ed0 0x7fb1cc068380 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc005930 tx=0x7fb1dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:43.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.023+0000 7fb1d57fa700 1 -- 172.21.15.78:0/3505182062 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fb1d0052290 con 0x7fb1e00cba00 2023-10-11T09:49:43.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.271+0000 7fb1e6ea3700 1 -- 172.21.15.78:0/3505182062 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_simple"]} v 0) v1 -- 0x7fb1e006e1e0 con 0x7fb1e00cba00 2023-10-11T09:49:43.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.399+0000 7fb1d57fa700 1 -- 172.21.15.78:0/3505182062 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_simple"]}]=0 v406) v1 ==== 155+0+0 (secure 0 0 0) 0x7fb1d00558b0 con 0x7fb1e00cba00 2023-10-11T09:49:43.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.403+0000 7fb1e6ea3700 1 -- 172.21.15.78:0/3505182062 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb1cc065ed0 msgr2=0x7fb1cc068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:43.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.403+0000 7fb1e6ea3700 1 --2- 172.21.15.78:0/3505182062 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb1cc065ed0 0x7fb1cc068380 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc005930 tx=0x7fb1dc009040 comp rx=0 tx=0).stop 2023-10-11T09:49:43.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.403+0000 7fb1e6ea3700 1 -- 172.21.15.78:0/3505182062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1e00cba00 msgr2=0x7fb1e015b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:43.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.403+0000 7fb1e6ea3700 1 --2- 172.21.15.78:0/3505182062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1e00cba00 0x7fb1e015b230 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fb1d00090b0 tx=0x7fb1d0004770 comp rx=0 tx=0).stop 2023-10-11T09:49:43.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.403+0000 7fb1e6ea3700 1 -- 172.21.15.78:0/3505182062 shutdown_connections 2023-10-11T09:49:43.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.403+0000 7fb1e6ea3700 1 --2- 172.21.15.78:0/3505182062 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb1cc065ed0 0x7fb1cc068380 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:43.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.403+0000 7fb1e6ea3700 1 --2- 172.21.15.78:0/3505182062 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb1e0154e00 0x7fb1e00aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:43.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.403+0000 7fb1e6ea3700 1 --2- 172.21.15.78:0/3505182062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1e00cba00 0x7fb1e015b230 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:43.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.403+0000 7fb1e6ea3700 1 --2- 172.21.15.78:0/3505182062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1e0071040 0x7fb1e0157970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:43.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.403+0000 7fb1e6ea3700 1 -- 172.21.15.78:0/3505182062 >> 172.21.15.78:0/3505182062 conn(0x7fb1e006aca0 msgr2=0x7fb1e0159f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:43.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.403+0000 7fb1e6ea3700 1 -- 172.21.15.78:0/3505182062 shutdown_connections 2023-10-11T09:49:43.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.403+0000 7fb1e6ea3700 1 -- 172.21.15.78:0/3505182062 wait complete. 2023-10-11T09:49:43.433 INFO:tasks.cephfs_test_runner:test_extra_data_pool_rebuild_simple (tasks.cephfs.test_data_scan.TestDataScan) ... ok 2023-10-11T09:49:43.433 INFO:tasks.cephfs_test_runner:Starting test: test_extra_data_pool_stashed_layout (tasks.cephfs.test_data_scan.TestDataScan) 2023-10-11T09:49:43.434 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_stashed_layout' 2023-10-11T09:49:43.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.595+0000 7efc9dcd5700 1 Processor -- start 2023-10-11T09:49:43.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.595+0000 7efc9dcd5700 1 -- start start 2023-10-11T09:49:43.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.595+0000 7efc9dcd5700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7efc9815b8e0 0x7efc9806d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:43.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.595+0000 7efc9dcd5700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efc980ca950 0x7efc9813fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:43.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.595+0000 7efc9dcd5700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc9815e010 con 0x7efc9806d610 2023-10-11T09:49:43.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.595+0000 7efc9dcd5700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc9815e280 con 0x7efc9815b8e0 2023-10-11T09:49:43.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.595+0000 7efc9dcd5700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc9815e560 con 0x7efc980ca950 2023-10-11T09:49:43.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.595+0000 7efc9ccd3700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7efc9815b8e0 0x7efc9806d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:43.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.595+0000 7efc9ccd3700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7efc9815b8e0 0x7efc9806d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38680/0 (socket says 172.21.15.78:38680) 2023-10-11T09:49:43.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.595+0000 7efc9ccd3700 1 -- 172.21.15.78:0/874648646 learned_addr learned my addr 172.21.15.78:0/874648646 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:43.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.595+0000 7efc9d4d4700 1 --2- 172.21.15.78:0/874648646 >> v2:172.21.15.133:3300/0 conn(0x7efc980ca950 0x7efc9813fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:43.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.599+0000 7efc9ccd3700 1 -- 172.21.15.78:0/874648646 >> v2:172.21.15.133:3300/0 conn(0x7efc980ca950 msgr2=0x7efc9813fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:43.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.599+0000 7efc9ccd3700 1 --2- 172.21.15.78:0/874648646 >> v2:172.21.15.133:3300/0 conn(0x7efc980ca950 0x7efc9813fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:43.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.599+0000 7efc9ccd3700 1 -- 172.21.15.78:0/874648646 >> v1:172.21.15.133:6789/0 conn(0x7efc9806d610 legacy=0x7efc980ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:43.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.599+0000 7efc9ccd3700 1 -- 172.21.15.78:0/874648646 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc9815e840 con 0x7efc9815b8e0 2023-10-11T09:49:43.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.599+0000 7efc9d4d4700 1 --2- 172.21.15.78:0/874648646 >> v2:172.21.15.133:3300/0 conn(0x7efc980ca950 0x7efc9813fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:49:43.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.599+0000 7efc9ccd3700 1 --2- 172.21.15.78:0/874648646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc9815b8e0 0x7efc9806d0d0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7efc980b4cd0 tx=0x7efc94009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb0c9d0f6620333 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:43.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.599+0000 7efc8f7fe700 1 -- 172.21.15.78:0/874648646 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc94011830 con 0x7efc9815b8e0 2023-10-11T09:49:43.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.599+0000 7efc8f7fe700 1 -- 172.21.15.78:0/874648646 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc94011990 con 0x7efc9815b8e0 2023-10-11T09:49:43.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.599+0000 7efc8f7fe700 1 -- 172.21.15.78:0/874648646 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc94011d20 con 0x7efc9815b8e0 2023-10-11T09:49:43.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.599+0000 7efc9dcd5700 1 -- 172.21.15.78:0/874648646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc9815b8e0 msgr2=0x7efc9806d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:43.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.599+0000 7efc9dcd5700 1 --2- 172.21.15.78:0/874648646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc9815b8e0 0x7efc9806d0d0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7efc980b4cd0 tx=0x7efc94009bf0 comp rx=0 tx=0).stop 2023-10-11T09:49:43.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.599+0000 7efc9dcd5700 1 -- 172.21.15.78:0/874648646 shutdown_connections 2023-10-11T09:49:43.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.599+0000 7efc9dcd5700 1 --2- 172.21.15.78:0/874648646 >> v2:172.21.15.133:3300/0 conn(0x7efc980ca950 0x7efc9813fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:43.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.599+0000 7efc9dcd5700 1 --2- 172.21.15.78:0/874648646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc9815b8e0 0x7efc9806d0d0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:43.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.599+0000 7efc9dcd5700 1 -- 172.21.15.78:0/874648646 >> 172.21.15.78:0/874648646 conn(0x7efc980cea60 msgr2=0x7efc980cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:43.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.599+0000 7efc9dcd5700 1 -- 172.21.15.78:0/874648646 shutdown_connections 2023-10-11T09:49:43.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.599+0000 7efc9dcd5700 1 -- 172.21.15.78:0/874648646 wait complete. 2023-10-11T09:49:43.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.603+0000 7efc9dcd5700 1 Processor -- start 2023-10-11T09:49:43.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.603+0000 7efc9dcd5700 1 -- start start 2023-10-11T09:49:43.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.603+0000 7efc9dcd5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc9806d610 0x7efc980b3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:43.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.603+0000 7efc9dcd5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc980ca950 0x7efc981ef2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:43.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.603+0000 7efc9dcd5700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efc9815b8e0 0x7efc980c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:43.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.603+0000 7efc9dcd5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc98141ef0 con 0x7efc9806d610 2023-10-11T09:49:43.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.603+0000 7efc9dcd5700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc980b4ab0 con 0x7efc980ca950 2023-10-11T09:49:43.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.603+0000 7efc9dcd5700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc980b3bb0 con 0x7efc9815b8e0 2023-10-11T09:49:43.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.603+0000 7efc9ccd3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc9806d610 0x7efc980b3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:43.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.603+0000 7efc9ccd3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc9806d610 0x7efc980b3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38682/0 (socket says 172.21.15.78:38682) 2023-10-11T09:49:43.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.603+0000 7efc9ccd3700 1 -- 172.21.15.78:0/3557566553 learned_addr learned my addr 172.21.15.78:0/3557566553 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:43.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.603+0000 7efc9d4d4700 1 --2- 172.21.15.78:0/3557566553 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efc9815b8e0 0x7efc980c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:43.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.603+0000 7efc8ffff700 1 --2- 172.21.15.78:0/3557566553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc980ca950 0x7efc981ef2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:43.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.603+0000 7efc9ccd3700 1 -- 172.21.15.78:0/3557566553 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efc9815b8e0 msgr2=0x7efc980c7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:43.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.603+0000 7efc9ccd3700 1 --2- 172.21.15.78:0/3557566553 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efc9815b8e0 0x7efc980c7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:43.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.603+0000 7efc9ccd3700 1 -- 172.21.15.78:0/3557566553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc980ca950 msgr2=0x7efc981ef2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:43.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.603+0000 7efc9ccd3700 1 --2- 172.21.15.78:0/3557566553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc980ca950 0x7efc981ef2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:43.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.603+0000 7efc9ccd3700 1 -- 172.21.15.78:0/3557566553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc94009570 con 0x7efc9806d610 2023-10-11T09:49:43.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.603+0000 7efc8ffff700 1 --2- 172.21.15.78:0/3557566553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc980ca950 0x7efc981ef2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:49:43.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.603+0000 7efc9d4d4700 1 --2- 172.21.15.78:0/3557566553 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efc9815b8e0 0x7efc980c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:49:43.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.603+0000 7efc9ccd3700 1 --2- 172.21.15.78:0/3557566553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc9806d610 0x7efc980b3670 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7efc94009150 tx=0x7efc94004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:43.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.607+0000 7efc8d7fa700 1 -- 172.21.15.78:0/3557566553 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc94003ad0 con 0x7efc9806d610 2023-10-11T09:49:43.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.607+0000 7efc8d7fa700 1 -- 172.21.15.78:0/3557566553 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc9401e070 con 0x7efc9806d610 2023-10-11T09:49:43.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.607+0000 7efc9dcd5700 1 -- 172.21.15.78:0/3557566553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc981f2dd0 con 0x7efc9806d610 2023-10-11T09:49:43.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.607+0000 7efc8d7fa700 1 -- 172.21.15.78:0/3557566553 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc94003e70 con 0x7efc9806d610 2023-10-11T09:49:43.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.607+0000 7efc9dcd5700 1 -- 172.21.15.78:0/3557566553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc981f2af0 con 0x7efc9806d610 2023-10-11T09:49:43.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.607+0000 7efc9dcd5700 1 -- 172.21.15.78:0/3557566553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc981430e0 con 0x7efc9806d610 2023-10-11T09:49:43.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.607+0000 7efc8d7fa700 1 -- 172.21.15.78:0/3557566553 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7efc9401a420 con 0x7efc9806d610 2023-10-11T09:49:43.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.611+0000 7efc8d7fa700 1 --2- 172.21.15.78:0/3557566553 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efc84065ed0 0x7efc84068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:43.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.611+0000 7efc8d7fa700 1 -- 172.21.15.78:0/3557566553 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5554+0+0 (secure 0 0 0) 0x7efc94084ca0 con 0x7efc9806d610 2023-10-11T09:49:43.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.611+0000 7efc8ffff700 1 --2- 172.21.15.78:0/3557566553 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efc84065ed0 0x7efc84068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:43.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.611+0000 7efc8ffff700 1 --2- 172.21.15.78:0/3557566553 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efc84065ed0 0x7efc84068380 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7efc9812f430 tx=0x7efc88009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:43.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.619+0000 7efc8d7fa700 1 -- 172.21.15.78:0/3557566553 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7efc94052590 con 0x7efc9806d610 2023-10-11T09:49:43.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:43.871+0000 7efc9dcd5700 1 -- 172.21.15.78:0/3557566553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_stashed_layout"]} v 0) v1 -- 0x7efc98065d40 con 0x7efc9806d610 2023-10-11T09:49:44.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.407+0000 7efc8d7fa700 1 -- 172.21.15.78:0/3557566553 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_stashed_layout"]}]=0 v407) v1 ==== 158+0+0 (secure 0 0 0) 0x7efc94055bb0 con 0x7efc9806d610 2023-10-11T09:49:44.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.411+0000 7efc9dcd5700 1 -- 172.21.15.78:0/3557566553 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efc84065ed0 msgr2=0x7efc84068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:44.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.411+0000 7efc9dcd5700 1 --2- 172.21.15.78:0/3557566553 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efc84065ed0 0x7efc84068380 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7efc9812f430 tx=0x7efc88009040 comp rx=0 tx=0).stop 2023-10-11T09:49:44.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.411+0000 7efc9dcd5700 1 -- 172.21.15.78:0/3557566553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc9806d610 msgr2=0x7efc980b3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:44.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.411+0000 7efc9dcd5700 1 --2- 172.21.15.78:0/3557566553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc9806d610 0x7efc980b3670 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7efc94009150 tx=0x7efc94004770 comp rx=0 tx=0).stop 2023-10-11T09:49:44.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.411+0000 7efc9dcd5700 1 -- 172.21.15.78:0/3557566553 shutdown_connections 2023-10-11T09:49:44.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.411+0000 7efc9dcd5700 1 --2- 172.21.15.78:0/3557566553 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7efc84065ed0 0x7efc84068380 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:44.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.411+0000 7efc9dcd5700 1 --2- 172.21.15.78:0/3557566553 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efc9815b8e0 0x7efc980c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:44.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.411+0000 7efc9dcd5700 1 --2- 172.21.15.78:0/3557566553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc980ca950 0x7efc981ef2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:44.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.411+0000 7efc9dcd5700 1 --2- 172.21.15.78:0/3557566553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc9806d610 0x7efc980b3670 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:44.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.411+0000 7efc9dcd5700 1 -- 172.21.15.78:0/3557566553 >> 172.21.15.78:0/3557566553 conn(0x7efc980cea60 msgr2=0x7efc980c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:44.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.411+0000 7efc9dcd5700 1 -- 172.21.15.78:0/3557566553 shutdown_connections 2023-10-11T09:49:44.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.411+0000 7efc9dcd5700 1 -- 172.21.15.78:0/3557566553 wait complete. 2023-10-11T09:49:44.445 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-11T09:49:44.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.619+0000 7fb7f6d5e700 1 Processor -- start 2023-10-11T09:49:44.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.619+0000 7fb7f6d5e700 1 -- start start 2023-10-11T09:49:44.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.619+0000 7fb7f6d5e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb7f0158430 0x7fb7f00710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:44.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.619+0000 7fb7f6d5e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fb7f006d340 0x7fb7f00ca880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:44.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.619+0000 7fb7f6d5e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7f00cadc0 0x7fb7f015afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:44.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.619+0000 7fb7f6d5e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7f0070c10 con 0x7fb7f006d340 2023-10-11T09:49:44.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.619+0000 7fb7f6d5e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7f00cb240 con 0x7fb7f0158430 2023-10-11T09:49:44.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.619+0000 7fb7f6d5e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f0071610 con 0x7fb7f00cadc0 2023-10-11T09:49:44.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.619+0000 7fb7f4afa700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb7f0158430 0x7fb7f00710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:44.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.619+0000 7fb7effff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fb7f006d340 0x7fb7f00ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:44.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.619+0000 7fb7f4afa700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb7f0158430 0x7fb7f00710d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:40536/0 (socket says 172.21.15.78:40536) 2023-10-11T09:49:44.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.619+0000 7fb7f4afa700 1 -- 172.21.15.78:0/3354672006 learned_addr learned my addr 172.21.15.78:0/3354672006 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:44.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.619+0000 7fb7f52fb700 1 --2- 172.21.15.78:0/3354672006 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7f00cadc0 0x7fb7f015afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:44.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.623+0000 7fb7f4afa700 1 -- 172.21.15.78:0/3354672006 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7f00cadc0 msgr2=0x7fb7f015afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:44.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.623+0000 7fb7f4afa700 1 --2- 172.21.15.78:0/3354672006 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7f00cadc0 0x7fb7f015afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:44.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.623+0000 7fb7f4afa700 1 -- 172.21.15.78:0/3354672006 >> v2:172.21.15.78:3300/0 conn(0x7fb7f006d340 msgr2=0x7fb7f00ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:44.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.623+0000 7fb7f4afa700 1 --2- 172.21.15.78:0/3354672006 >> v2:172.21.15.78:3300/0 conn(0x7fb7f006d340 0x7fb7f00ca880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:44.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.623+0000 7fb7f4afa700 1 -- 172.21.15.78:0/3354672006 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f015e280 con 0x7fb7f0158430 2023-10-11T09:49:44.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.623+0000 7fb7f52fb700 1 --2- 172.21.15.78:0/3354672006 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7f00cadc0 0x7fb7f015afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:49:44.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.623+0000 7fb7effff700 1 --2- 172.21.15.78:0/3354672006 >> v2:172.21.15.78:3300/0 conn(0x7fb7f006d340 0x7fb7f00ca880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:49:44.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.623+0000 7fb7f4afa700 1 --2- 172.21.15.78:0/3354672006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7f0158430 0x7fb7f00710d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb7f006fd00 tx=0x7fb7e400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b7efcf16e271d5a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:44.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.623+0000 7fb7ef7fe700 1 -- 172.21.15.78:0/3354672006 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e400e070 con 0x7fb7f0158430 2023-10-11T09:49:44.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.623+0000 7fb7ef7fe700 1 -- 172.21.15.78:0/3354672006 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e400b910 con 0x7fb7f0158430 2023-10-11T09:49:44.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.623+0000 7fb7ef7fe700 1 -- 172.21.15.78:0/3354672006 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e400bcc0 con 0x7fb7f0158430 2023-10-11T09:49:44.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.623+0000 7fb7f6d5e700 1 -- 172.21.15.78:0/3354672006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7f0158430 msgr2=0x7fb7f00710d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:44.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.623+0000 7fb7f6d5e700 1 --2- 172.21.15.78:0/3354672006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7f0158430 0x7fb7f00710d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb7f006fd00 tx=0x7fb7e400b330 comp rx=0 tx=0).stop 2023-10-11T09:49:44.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.623+0000 7fb7f6d5e700 1 -- 172.21.15.78:0/3354672006 shutdown_connections 2023-10-11T09:49:44.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.623+0000 7fb7f6d5e700 1 --2- 172.21.15.78:0/3354672006 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7f00cadc0 0x7fb7f015afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:44.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.623+0000 7fb7f6d5e700 1 --2- 172.21.15.78:0/3354672006 >> v2:172.21.15.78:3300/0 conn(0x7fb7f006d340 0x7fb7f00ca880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:44.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.623+0000 7fb7f6d5e700 1 --2- 172.21.15.78:0/3354672006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7f0158430 0x7fb7f00710d0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:44.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.623+0000 7fb7f6d5e700 1 -- 172.21.15.78:0/3354672006 >> 172.21.15.78:0/3354672006 conn(0x7fb7f00cecc0 msgr2=0x7fb7f00cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:44.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.623+0000 7fb7f6d5e700 1 -- 172.21.15.78:0/3354672006 shutdown_connections 2023-10-11T09:49:44.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.623+0000 7fb7f6d5e700 1 -- 172.21.15.78:0/3354672006 wait complete. 2023-10-11T09:49:44.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.627+0000 7fb7f6d5e700 1 Processor -- start 2023-10-11T09:49:44.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.627+0000 7fb7f6d5e700 1 -- start start 2023-10-11T09:49:44.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.631+0000 7fb7f6d5e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7f006d340 0x7fb7f00cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:44.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.631+0000 7fb7f6d5e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7f00cadc0 0x7fb7f00cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:44.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.631+0000 7fb7f6d5e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7f0158430 0x7fb7f00cb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:44.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.631+0000 7fb7f6d5e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f015b580 con 0x7fb7f00cadc0 2023-10-11T09:49:44.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.631+0000 7fb7f6d5e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f015b860 con 0x7fb7f0158430 2023-10-11T09:49:44.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.631+0000 7fb7f6d5e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f015bb40 con 0x7fb7f006d340 2023-10-11T09:49:44.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.631+0000 7fb7effff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7f00cadc0 0x7fb7f00cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:44.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.631+0000 7fb7effff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7f00cadc0 0x7fb7f00cb3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41218/0 (socket says 172.21.15.78:41218) 2023-10-11T09:49:44.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.631+0000 7fb7effff700 1 -- 172.21.15.78:0/1472040928 learned_addr learned my addr 172.21.15.78:0/1472040928 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:44.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.631+0000 7fb7f4afa700 1 --2- 172.21.15.78:0/1472040928 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7f006d340 0x7fb7f00cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:44.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.631+0000 7fb7f52fb700 1 --2- 172.21.15.78:0/1472040928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7f0158430 0x7fb7f00cb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:44.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.631+0000 7fb7effff700 1 -- 172.21.15.78:0/1472040928 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7f006d340 msgr2=0x7fb7f00cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:44.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.631+0000 7fb7effff700 1 --2- 172.21.15.78:0/1472040928 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7f006d340 0x7fb7f00cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:44.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.631+0000 7fb7effff700 1 -- 172.21.15.78:0/1472040928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7f0158430 msgr2=0x7fb7f00cb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:44.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.631+0000 7fb7effff700 1 --2- 172.21.15.78:0/1472040928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7f0158430 0x7fb7f00cb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:44.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.631+0000 7fb7effff700 1 -- 172.21.15.78:0/1472040928 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7dc0094b0 con 0x7fb7f00cadc0 2023-10-11T09:49:44.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.631+0000 7fb7f52fb700 1 --2- 172.21.15.78:0/1472040928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7f0158430 0x7fb7f00cb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:49:44.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.631+0000 7fb7effff700 1 --2- 172.21.15.78:0/1472040928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7f00cadc0 0x7fb7f00cb3b0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc00ebe0 tx=0x7fb7dc00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:44.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.631+0000 7fb7ed7fa700 1 -- 172.21.15.78:0/1472040928 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc009790 con 0x7fb7f00cadc0 2023-10-11T09:49:44.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.631+0000 7fb7ed7fa700 1 -- 172.21.15.78:0/1472040928 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7dc00c850 con 0x7fb7f00cadc0 2023-10-11T09:49:44.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.631+0000 7fb7ed7fa700 1 -- 172.21.15.78:0/1472040928 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc00cbe0 con 0x7fb7f00cadc0 2023-10-11T09:49:44.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.631+0000 7fb7f6d5e700 1 -- 172.21.15.78:0/1472040928 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7e80094b0 con 0x7fb7f00cadc0 2023-10-11T09:49:44.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.631+0000 7fb7f6d5e700 1 -- 172.21.15.78:0/1472040928 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7e4009b30 con 0x7fb7f00cadc0 2023-10-11T09:49:44.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.635+0000 7fb7f6d5e700 1 -- 172.21.15.78:0/1472040928 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7f006eb80 con 0x7fb7f00cadc0 2023-10-11T09:49:44.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.635+0000 7fb7ed7fa700 1 -- 172.21.15.78:0/1472040928 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fb7dc00cd40 con 0x7fb7f00cadc0 2023-10-11T09:49:44.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.639+0000 7fb7ed7fa700 1 --2- 172.21.15.78:0/1472040928 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb7d4065ed0 0x7fb7d4068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:44.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.639+0000 7fb7ed7fa700 1 -- 172.21.15.78:0/1472040928 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fb7dc0836d0 con 0x7fb7f00cadc0 2023-10-11T09:49:44.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.639+0000 7fb7f4afa700 1 --2- 172.21.15.78:0/1472040928 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb7d4065ed0 0x7fb7d4068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:44.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.639+0000 7fb7f4afa700 1 --2- 172.21.15.78:0/1472040928 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb7d4065ed0 0x7fb7d4068380 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb7e400b370 tx=0x7fb7e4009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:44.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.643+0000 7fb7ed7fa700 1 -- 172.21.15.78:0/1472040928 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fb7dc050f00 con 0x7fb7f00cadc0 2023-10-11T09:49:44.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.819+0000 7fb7f6d5e700 1 -- 172.21.15.78:0/1472040928 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb7f015b6c0 con 0x7fb7f00cadc0 2023-10-11T09:49:44.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.827+0000 7fb7ed7fa700 1 -- 172.21.15.78:0/1472040928 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb7f015b6c0 con 0x7fb7f00cadc0 2023-10-11T09:49:44.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.831+0000 7fb7f6d5e700 1 -- 172.21.15.78:0/1472040928 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb7d4065ed0 msgr2=0x7fb7d4068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:44.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.831+0000 7fb7f6d5e700 1 --2- 172.21.15.78:0/1472040928 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb7d4065ed0 0x7fb7d4068380 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb7e400b370 tx=0x7fb7e4009000 comp rx=0 tx=0).stop 2023-10-11T09:49:44.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.831+0000 7fb7f6d5e700 1 -- 172.21.15.78:0/1472040928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7f00cadc0 msgr2=0x7fb7f00cb3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:44.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.831+0000 7fb7f6d5e700 1 --2- 172.21.15.78:0/1472040928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7f00cadc0 0x7fb7f00cb3b0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc00ebe0 tx=0x7fb7dc00eef0 comp rx=0 tx=0).stop 2023-10-11T09:49:44.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.835+0000 7fb7f6d5e700 1 -- 172.21.15.78:0/1472040928 shutdown_connections 2023-10-11T09:49:44.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.835+0000 7fb7f6d5e700 1 --2- 172.21.15.78:0/1472040928 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb7d4065ed0 0x7fb7d4068380 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:44.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.835+0000 7fb7f6d5e700 1 --2- 172.21.15.78:0/1472040928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7f0158430 0x7fb7f00cb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:44.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.835+0000 7fb7f6d5e700 1 --2- 172.21.15.78:0/1472040928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7f00cadc0 0x7fb7f00cb3b0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:44.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.835+0000 7fb7f6d5e700 1 --2- 172.21.15.78:0/1472040928 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7f006d340 0x7fb7f00cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:44.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.835+0000 7fb7f6d5e700 1 -- 172.21.15.78:0/1472040928 >> 172.21.15.78:0/1472040928 conn(0x7fb7f00cecc0 msgr2=0x7fb7f015daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:44.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.835+0000 7fb7f6d5e700 1 -- 172.21.15.78:0/1472040928 shutdown_connections 2023-10-11T09:49:44.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:44.835+0000 7fb7f6d5e700 1 -- 172.21.15.78:0/1472040928 wait complete. 2023-10-11T09:49:44.862 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-11T09:49:44.862 DEBUG:teuthology.orchestra.run.smithi078:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-11T09:49:44.905 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-11T09:49:44.949 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-11T09:49:44.950 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:49:44.950 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:49:44.970 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352272k,nr_inodes=4088068,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17487 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-11T09:49:44.971 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:49:44.972 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:49:45.012 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352292k,nr_inodes=4088073,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1430 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275376k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-11T09:49:45.012 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:49:45.013 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:49:45.030 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352272k,nr_inodes=4088068,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17487 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-11T09:49:45.030 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:49:45.031 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:49:45.070 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352292k,nr_inodes=4088073,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1430 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275376k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-11T09:49:45.071 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-11T09:49:45.071 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:49:45.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.243+0000 7f7c47701700 1 Processor -- start 2023-10-11T09:49:45.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.247+0000 7f7c47701700 1 -- start start 2023-10-11T09:49:45.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.247+0000 7f7c47701700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7c40071040 0x7f7c400cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:45.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.247+0000 7f7c47701700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c400720f0 con 0x7f7c40154e00 2023-10-11T09:49:45.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.247+0000 7f7c47701700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c400714e0 con 0x7f7c400cba00 2023-10-11T09:49:45.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.247+0000 7f7c47701700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c400717c0 con 0x7f7c40071040 2023-10-11T09:49:45.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.247+0000 7f7c4549d700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f7c40154e00 0x7f7c4015b650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:42186/0 (socket says 172.21.15.78:42186) 2023-10-11T09:49:45.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.247+0000 7f7c4549d700 1 -- 172.21.15.78:0/3038847583 learned_addr learned my addr 172.21.15.78:0/3038847583 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:49:45.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.247+0000 7f7c37fff700 1 -- 172.21.15.78:0/3038847583 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3982687121 0 0) 0x7f7c400720f0 con 0x7f7c40154e00 2023-10-11T09:49:45.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.247+0000 7f7c44c9c700 1 --2- 172.21.15.78:0/3038847583 >> v2:172.21.15.133:3300/0 conn(0x7f7c40071040 0x7f7c400cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:45.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.247+0000 7f7c37fff700 1 -- 172.21.15.78:0/3038847583 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c2c0035d0 con 0x7f7c40154e00 2023-10-11T09:49:45.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.247+0000 7f7c37fff700 1 -- 172.21.15.78:0/3038847583 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3549179657 0 0) 0x7f7c2c0035d0 con 0x7f7c40154e00 2023-10-11T09:49:45.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.247+0000 7f7c37fff700 1 -- 172.21.15.78:0/3038847583 >> v2:172.21.15.133:3300/0 conn(0x7f7c40071040 msgr2=0x7f7c400cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:45.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.251+0000 7f7c37fff700 1 --2- 172.21.15.78:0/3038847583 >> v2:172.21.15.133:3300/0 conn(0x7f7c40071040 0x7f7c400cb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:45.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.251+0000 7f7c37fff700 1 -- 172.21.15.78:0/3038847583 >> v1:172.21.15.133:6789/0 conn(0x7f7c400cba00 legacy=0x7f7c4006c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:45.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.251+0000 7f7c37fff700 1 -- 172.21.15.78:0/3038847583 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c40071aa0 con 0x7f7c40154e00 2023-10-11T09:49:45.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.251+0000 7f7c37fff700 1 -- 172.21.15.78:0/3038847583 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f7c3c001670 con 0x7f7c40154e00 2023-10-11T09:49:45.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.251+0000 7f7c37fff700 1 -- 172.21.15.78:0/3038847583 >> v1:172.21.15.78:6789/0 conn(0x7f7c40154e00 legacy=0x7f7c4015b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:45.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.251+0000 7f7c37fff700 1 --2- 172.21.15.78:0/3038847583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c2c003a20 0x7f7c2c003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:45.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.251+0000 7f7c37fff700 1 --2- 172.21.15.78:0/3038847583 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c2c004870 0x7f7c2c004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:45.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.251+0000 7f7c37fff700 1 --2- 172.21.15.78:0/3038847583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c2c005220 0x7f7c2c00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:45.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.251+0000 7f7c37fff700 1 -- 172.21.15.78:0/3038847583 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c400717c0 con 0x7f7c2c005220 2023-10-11T09:49:45.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.251+0000 7f7c37fff700 1 -- 172.21.15.78:0/3038847583 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c00c080 con 0x7f7c2c003a20 2023-10-11T09:49:45.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.251+0000 7f7c37fff700 1 -- 172.21.15.78:0/3038847583 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c00c240 con 0x7f7c2c004870 2023-10-11T09:49:45.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.251+0000 7f7c4549d700 1 --2- 172.21.15.78:0/3038847583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c2c003a20 0x7f7c2c003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:45.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.251+0000 7f7c45c9e700 1 --2- 172.21.15.78:0/3038847583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c2c005220 0x7f7c2c00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:45.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.251+0000 7f7c44c9c700 1 --2- 172.21.15.78:0/3038847583 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c2c004870 0x7f7c2c004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:45.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.251+0000 7f7c4549d700 1 -- 172.21.15.78:0/3038847583 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c2c004870 msgr2=0x7f7c2c004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:45.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.251+0000 7f7c4549d700 1 --2- 172.21.15.78:0/3038847583 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c2c004870 0x7f7c2c004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:45.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.251+0000 7f7c4549d700 1 -- 172.21.15.78:0/3038847583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c2c005220 msgr2=0x7f7c2c00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:45.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.251+0000 7f7c4549d700 1 --2- 172.21.15.78:0/3038847583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c2c005220 0x7f7c2c00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:45.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.251+0000 7f7c4549d700 1 -- 172.21.15.78:0/3038847583 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c2c0035d0 con 0x7f7c2c003a20 2023-10-11T09:49:45.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.251+0000 7f7c45c9e700 1 --2- 172.21.15.78:0/3038847583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c2c005220 0x7f7c2c00ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:49:45.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.251+0000 7f7c45c9e700 1 -- 172.21.15.78:0/3038847583 reap_dead start 2023-10-11T09:49:45.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.251+0000 7f7c4549d700 1 --2- 172.21.15.78:0/3038847583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c2c003a20 0x7f7c2c003df0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c005040 tx=0x7f7c2c0058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b7fb6976d5c04ce server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:45.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.251+0000 7f7c37fff700 1 -- 172.21.15.78:0/3038847583 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c005b80 con 0x7f7c2c003a20 2023-10-11T09:49:45.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.251+0000 7f7c37fff700 1 -- 172.21.15.78:0/3038847583 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c3c005d00 con 0x7f7c2c003a20 2023-10-11T09:49:45.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.251+0000 7f7c37fff700 1 -- 172.21.15.78:0/3038847583 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c008320 con 0x7f7c2c003a20 2023-10-11T09:49:45.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.251+0000 7f7c47701700 1 -- 172.21.15.78:0/3038847583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c2c003a20 msgr2=0x7f7c2c003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:45.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.251+0000 7f7c47701700 1 --2- 172.21.15.78:0/3038847583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c2c003a20 0x7f7c2c003df0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c005040 tx=0x7f7c2c0058a0 comp rx=0 tx=0).stop 2023-10-11T09:49:45.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.255+0000 7f7c47701700 1 -- 172.21.15.78:0/3038847583 shutdown_connections 2023-10-11T09:49:45.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.255+0000 7f7c47701700 1 --2- 172.21.15.78:0/3038847583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c2c003a20 0x7f7c2c003df0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:45.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.255+0000 7f7c47701700 1 -- 172.21.15.78:0/3038847583 >> 172.21.15.78:0/3038847583 conn(0x7f7c4006aca0 msgr2=0x7f7c400b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:45.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.255+0000 7f7c47701700 1 -- 172.21.15.78:0/3038847583 shutdown_connections 2023-10-11T09:49:45.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.255+0000 7f7c47701700 1 -- 172.21.15.78:0/3038847583 wait complete. 2023-10-11T09:49:45.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.255+0000 7f7c47701700 1 Processor -- start 2023-10-11T09:49:45.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.255+0000 7f7c47701700 1 -- start start 2023-10-11T09:49:45.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.259+0000 7f7c47701700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c2c003a20 0x7f7c40128030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:45.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.259+0000 7f7c47701700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c40145d60 0x7f7c40146620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:45.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.259+0000 7f7c47701700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c401461b0 0x7f7c40146b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:45.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.259+0000 7f7c47701700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c400b1c90 con 0x7f7c2c003a20 2023-10-11T09:49:45.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.259+0000 7f7c47701700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c40149660 con 0x7f7c401461b0 2023-10-11T09:49:45.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.259+0000 7f7c47701700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c400b1ea0 con 0x7f7c40145d60 2023-10-11T09:49:45.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.259+0000 7f7c4549d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c2c003a20 0x7f7c40128030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:45.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.259+0000 7f7c4549d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c2c003a20 0x7f7c40128030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41246/0 (socket says 172.21.15.78:41246) 2023-10-11T09:49:45.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.259+0000 7f7c4549d700 1 -- 172.21.15.78:0/1546688902 learned_addr learned my addr 172.21.15.78:0/1546688902 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:45.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.259+0000 7f7c45c9e700 1 --2- 172.21.15.78:0/1546688902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c401461b0 0x7f7c40146b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:45.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.259+0000 7f7c44c9c700 1 --2- 172.21.15.78:0/1546688902 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c40145d60 0x7f7c40146620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:45.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.259+0000 7f7c4549d700 1 -- 172.21.15.78:0/1546688902 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c40145d60 msgr2=0x7f7c40146620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:45.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.259+0000 7f7c4549d700 1 --2- 172.21.15.78:0/1546688902 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c40145d60 0x7f7c40146620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:45.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.259+0000 7f7c4549d700 1 -- 172.21.15.78:0/1546688902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c401461b0 msgr2=0x7f7c40146b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:45.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.259+0000 7f7c4549d700 1 --2- 172.21.15.78:0/1546688902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c401461b0 0x7f7c40146b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:45.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.259+0000 7f7c4549d700 1 -- 172.21.15.78:0/1546688902 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c400b2180 con 0x7f7c2c003a20 2023-10-11T09:49:45.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.259+0000 7f7c4549d700 1 --2- 172.21.15.78:0/1546688902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c2c003a20 0x7f7c40128030 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c005630 tx=0x7f7c3c001de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:45.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.259+0000 7f7c35ffb700 1 -- 172.21.15.78:0/1546688902 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c005e70 con 0x7f7c2c003a20 2023-10-11T09:49:45.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.259+0000 7f7c35ffb700 1 -- 172.21.15.78:0/1546688902 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c3c000ed0 con 0x7f7c2c003a20 2023-10-11T09:49:45.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.259+0000 7f7c47701700 1 -- 172.21.15.78:0/1546688902 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c400b2460 con 0x7f7c2c003a20 2023-10-11T09:49:45.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.259+0000 7f7c35ffb700 1 -- 172.21.15.78:0/1546688902 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c008320 con 0x7f7c2c003a20 2023-10-11T09:49:45.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.263+0000 7f7c47701700 1 -- 172.21.15.78:0/1546688902 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c40148af0 con 0x7f7c2c003a20 2023-10-11T09:49:45.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.263+0000 7f7c47701700 1 -- 172.21.15.78:0/1546688902 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c400b26a0 con 0x7f7c2c003a20 2023-10-11T09:49:45.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.263+0000 7f7c35ffb700 1 -- 172.21.15.78:0/1546688902 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f7c3c008480 con 0x7f7c2c003a20 2023-10-11T09:49:45.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.267+0000 7f7c35ffb700 1 --2- 172.21.15.78:0/1546688902 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7c24065ed0 0x7f7c24068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:45.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.267+0000 7f7c35ffb700 1 -- 172.21.15.78:0/1546688902 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f7c3c0849e0 con 0x7f7c2c003a20 2023-10-11T09:49:45.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.267+0000 7f7c44c9c700 1 --2- 172.21.15.78:0/1546688902 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7c24065ed0 0x7f7c24068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:45.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.267+0000 7f7c44c9c700 1 --2- 172.21.15.78:0/1546688902 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7c24065ed0 0x7f7c24068380 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f7c400b4020 tx=0x7f7c30008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:45.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.271+0000 7f7c35ffb700 1 -- 172.21.15.78:0/1546688902 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f7c400b26a0 con 0x7f7c2c003a20 2023-10-11T09:49:45.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.483+0000 7f7c47701700 1 -- 172.21.15.78:0/1546688902 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c400b26a0 con 0x7f7c2c003a20 2023-10-11T09:49:45.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.483+0000 7f7c35ffb700 1 -- 172.21.15.78:0/1546688902 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f7c400b26a0 con 0x7f7c2c003a20 2023-10-11T09:49:45.491 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:49:45.492 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":73,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6581,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/238181631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":238181631},{"type":"v1","addr":"172.21.15.78:6837","nonce":238181631}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":7378,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/2416080754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2416080754},{"type":"v1","addr":"172.21.15.133:6834","nonce":2416080754}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":7384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/3016626706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3016626706},{"type":"v1","addr":"172.21.15.78:6836","nonce":3016626706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":7636,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1002332198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":1002332198},{"type":"v1","addr":"172.21.15.133:6835","nonce":1002332198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[]} 2023-10-11T09:49:45.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.487+0000 7f7c47701700 1 -- 172.21.15.78:0/1546688902 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7c24065ed0 msgr2=0x7f7c24068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:45.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.487+0000 7f7c47701700 1 --2- 172.21.15.78:0/1546688902 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7c24065ed0 0x7f7c24068380 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f7c400b4020 tx=0x7f7c30008040 comp rx=0 tx=0).stop 2023-10-11T09:49:45.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.487+0000 7f7c47701700 1 -- 172.21.15.78:0/1546688902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c2c003a20 msgr2=0x7f7c40128030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:45.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.487+0000 7f7c47701700 1 --2- 172.21.15.78:0/1546688902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c2c003a20 0x7f7c40128030 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c005630 tx=0x7f7c3c001de0 comp rx=0 tx=0).stop 2023-10-11T09:49:45.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.487+0000 7f7c47701700 1 -- 172.21.15.78:0/1546688902 shutdown_connections 2023-10-11T09:49:45.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.487+0000 7f7c47701700 1 --2- 172.21.15.78:0/1546688902 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7c24065ed0 0x7f7c24068380 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:45.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.487+0000 7f7c47701700 1 --2- 172.21.15.78:0/1546688902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c401461b0 0x7f7c40146b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:45.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.487+0000 7f7c47701700 1 --2- 172.21.15.78:0/1546688902 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c40145d60 0x7f7c40146620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:45.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.487+0000 7f7c47701700 1 --2- 172.21.15.78:0/1546688902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c2c003a20 0x7f7c40128030 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:45.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.487+0000 7f7c47701700 1 -- 172.21.15.78:0/1546688902 >> 172.21.15.78:0/1546688902 conn(0x7f7c4006aca0 msgr2=0x7f7c4006ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:45.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.487+0000 7f7c47701700 1 -- 172.21.15.78:0/1546688902 shutdown_connections 2023-10-11T09:49:45.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.487+0000 7f7c47701700 1 -- 172.21.15.78:0/1546688902 wait complete. 2023-10-11T09:49:45.497 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 73 2023-10-11T09:49:45.516 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-11T09:49:45.516 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-11T09:49:45.517 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-11T09:49:45.517 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:49:45.517 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-11T09:49:45.517 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-11T09:49:45.518 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-11T09:49:45.518 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:49:45.518 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-11T09:49:45.519 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-11T09:49:45.519 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-11T09:49:45.519 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:49:45.520 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-11T09:49:45.520 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-11T09:49:45.520 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-11T09:49:45.520 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:49:45.542 INFO:tasks.ceph.mds.c:Stopped 2023-10-11T09:49:45.542 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-11T09:49:45.543 INFO:tasks.ceph.mds.b:Stopped 2023-10-11T09:49:45.543 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-11T09:49:45.544 INFO:tasks.ceph.mds.a:Stopped 2023-10-11T09:49:45.544 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-11T09:49:45.546 INFO:tasks.ceph.mds.d:Stopped 2023-10-11T09:49:45.546 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-11T09:49:45.589 INFO:tasks.ceph.mds.c:Started 2023-10-11T09:49:45.590 DEBUG:teuthology.parallel:result is None 2023-10-11T09:49:45.592 INFO:tasks.ceph.mds.a:Started 2023-10-11T09:49:45.592 DEBUG:teuthology.parallel:result is None 2023-10-11T09:49:45.595 INFO:tasks.ceph.mds.b:Started 2023-10-11T09:49:45.596 DEBUG:teuthology.parallel:result is None 2023-10-11T09:49:45.602 INFO:tasks.ceph.mds.d:Started 2023-10-11T09:49:45.602 DEBUG:teuthology.parallel:result is None 2023-10-11T09:49:45.603 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-11T09:49:45.699 INFO:tasks.ceph.mds.a.smithi078.stdout:starting mds.a at 2023-10-11T09:49:45.701 INFO:tasks.ceph.mds.c.smithi078.stdout:starting mds.c at 2023-10-11T09:49:45.711 INFO:tasks.ceph.mds.d.smithi133.stdout:starting mds.d at 2023-10-11T09:49:45.722 INFO:tasks.ceph.mds.b.smithi133.stdout:starting mds.b at 2023-10-11T09:49:45.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.771+0000 7f8f2d19a700 1 Processor -- start 2023-10-11T09:49:45.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.771+0000 7f8f2d19a700 1 -- start start 2023-10-11T09:49:45.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.775+0000 7f8f2d19a700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8f280cc840 0x7f8f280b1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:45.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.775+0000 7f8f2d19a700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f28160460 con 0x7f8f2815cb80 2023-10-11T09:49:45.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.775+0000 7f8f2d19a700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f28160700 con 0x7f8f2815c7b0 2023-10-11T09:49:45.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.775+0000 7f8f2d19a700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f281609e0 con 0x7f8f280cc840 2023-10-11T09:49:45.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.775+0000 7f8f27fff700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f8f2815c7b0 0x7f8f280cc130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:50690/0 (socket says 172.21.15.78:50690) 2023-10-11T09:49:45.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.775+0000 7f8f27fff700 1 -- 172.21.15.78:0/1497984209 learned_addr learned my addr 172.21.15.78:0/1497984209 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:49:45.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.775+0000 7f8f2c999700 1 --2- 172.21.15.78:0/1497984209 >> v2:172.21.15.133:3300/0 conn(0x7f8f280cc840 0x7f8f280b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:45.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.775+0000 7f8f277fe700 1 -- 172.21.15.78:0/1497984209 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1827984706 0 0) 0x7f8f28160700 con 0x7f8f2815c7b0 2023-10-11T09:49:45.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.775+0000 7f8f277fe700 1 -- 172.21.15.78:0/1497984209 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f140035d0 con 0x7f8f2815c7b0 2023-10-11T09:49:45.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.775+0000 7f8f277fe700 1 -- 172.21.15.78:0/1497984209 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2244330354 0 0) 0x7f8f28160460 con 0x7f8f2815cb80 2023-10-11T09:49:45.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.775+0000 7f8f277fe700 1 -- 172.21.15.78:0/1497984209 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f28160700 con 0x7f8f2815cb80 2023-10-11T09:49:45.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.775+0000 7f8f2c999700 1 -- 172.21.15.78:0/1497984209 >> v1:172.21.15.133:6789/0 conn(0x7f8f2815c7b0 legacy=0x7f8f280cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:45.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.775+0000 7f8f2c999700 1 -- 172.21.15.78:0/1497984209 >> v1:172.21.15.78:6789/0 conn(0x7f8f2815cb80 legacy=0x7f8f280b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:45.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.775+0000 7f8f2c999700 1 -- 172.21.15.78:0/1497984209 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f28160cc0 con 0x7f8f280cc840 2023-10-11T09:49:45.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.775+0000 7f8f2c999700 1 --2- 172.21.15.78:0/1497984209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f280cc840 0x7f8f280b1910 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8f20009d90 tx=0x7f8f2000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cea82a49ca5fb13 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:45.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.775+0000 7f8f277fe700 1 -- 172.21.15.78:0/1497984209 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2000e070 con 0x7f8f280cc840 2023-10-11T09:49:45.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.775+0000 7f8f277fe700 1 -- 172.21.15.78:0/1497984209 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f2000bb20 con 0x7f8f280cc840 2023-10-11T09:49:45.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.775+0000 7f8f277fe700 1 -- 172.21.15.78:0/1497984209 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f20012430 con 0x7f8f280cc840 2023-10-11T09:49:45.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.779+0000 7f8f2d19a700 1 -- 172.21.15.78:0/1497984209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f280cc840 msgr2=0x7f8f280b1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:45.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.779+0000 7f8f2d19a700 1 --2- 172.21.15.78:0/1497984209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f280cc840 0x7f8f280b1910 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8f20009d90 tx=0x7f8f2000b330 comp rx=0 tx=0).stop 2023-10-11T09:49:45.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.779+0000 7f8f2d19a700 1 -- 172.21.15.78:0/1497984209 shutdown_connections 2023-10-11T09:49:45.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.779+0000 7f8f2d19a700 1 --2- 172.21.15.78:0/1497984209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f280cc840 0x7f8f280b1910 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:45.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.779+0000 7f8f2d19a700 1 -- 172.21.15.78:0/1497984209 >> 172.21.15.78:0/1497984209 conn(0x7f8f2806c590 msgr2=0x7f8f2806c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:45.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.779+0000 7f8f2d19a700 1 -- 172.21.15.78:0/1497984209 shutdown_connections 2023-10-11T09:49:45.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.779+0000 7f8f2d19a700 1 -- 172.21.15.78:0/1497984209 wait complete. 2023-10-11T09:49:45.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.783+0000 7f8f2d19a700 1 Processor -- start 2023-10-11T09:49:45.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.783+0000 7f8f2d19a700 1 -- start start 2023-10-11T09:49:45.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.783+0000 7f8f2d19a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f280cc840 0x7f8f28153690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:45.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.783+0000 7f8f2d19a700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8f2815c7b0 0x7f8f2815e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:45.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.783+0000 7f8f2d19a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f281f6010 0x7f8f2815eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:45.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.783+0000 7f8f2d19a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2815f440 con 0x7f8f280cc840 2023-10-11T09:49:45.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.783+0000 7f8f2d19a700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f28153bd0 con 0x7f8f281f6010 2023-10-11T09:49:45.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.783+0000 7f8f2d19a700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f280c5dd0 con 0x7f8f2815c7b0 2023-10-11T09:49:45.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.783+0000 7f8f27fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f280cc840 0x7f8f28153690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:45.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.783+0000 7f8f27fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f280cc840 0x7f8f28153690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41296/0 (socket says 172.21.15.78:41296) 2023-10-11T09:49:45.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.783+0000 7f8f27fff700 1 -- 172.21.15.78:0/3377687812 learned_addr learned my addr 172.21.15.78:0/3377687812 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:45.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.783+0000 7f8f2c999700 1 --2- 172.21.15.78:0/3377687812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f281f6010 0x7f8f2815eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:45.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.783+0000 7f8f1ffff700 1 --2- 172.21.15.78:0/3377687812 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8f2815c7b0 0x7f8f2815e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:45.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.787+0000 7f8f27fff700 1 -- 172.21.15.78:0/3377687812 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8f2815c7b0 msgr2=0x7f8f2815e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:45.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.787+0000 7f8f27fff700 1 --2- 172.21.15.78:0/3377687812 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8f2815c7b0 0x7f8f2815e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:45.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.787+0000 7f8f27fff700 1 -- 172.21.15.78:0/3377687812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f281f6010 msgr2=0x7f8f2815eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:45.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.787+0000 7f8f27fff700 1 --2- 172.21.15.78:0/3377687812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f281f6010 0x7f8f2815eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:45.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.787+0000 7f8f27fff700 1 -- 172.21.15.78:0/3377687812 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f20009a70 con 0x7f8f280cc840 2023-10-11T09:49:45.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.787+0000 7f8f27fff700 1 --2- 172.21.15.78:0/3377687812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f280cc840 0x7f8f28153690 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f8f1800c560 tx=0x7f8f1800c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:45.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.787+0000 7f8f257fa700 1 -- 172.21.15.78:0/3377687812 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f18005d90 con 0x7f8f280cc840 2023-10-11T09:49:45.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.787+0000 7f8f257fa700 1 -- 172.21.15.78:0/3377687812 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f180070d0 con 0x7f8f280cc840 2023-10-11T09:49:45.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.787+0000 7f8f2d19a700 1 -- 172.21.15.78:0/3377687812 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f2815f160 con 0x7f8f280cc840 2023-10-11T09:49:45.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.787+0000 7f8f257fa700 1 -- 172.21.15.78:0/3377687812 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f18007460 con 0x7f8f280cc840 2023-10-11T09:49:45.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.787+0000 7f8f2d19a700 1 -- 172.21.15.78:0/3377687812 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f281f5810 con 0x7f8f280cc840 2023-10-11T09:49:45.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.787+0000 7f8f2d19a700 1 -- 172.21.15.78:0/3377687812 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f281f5390 con 0x7f8f280cc840 2023-10-11T09:49:45.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.791+0000 7f8f257fa700 1 -- 172.21.15.78:0/3377687812 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f8f180075c0 con 0x7f8f280cc840 2023-10-11T09:49:45.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.795+0000 7f8f257fa700 1 --2- 172.21.15.78:0/3377687812 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8f08065ed0 0x7f8f08068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:45.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.795+0000 7f8f257fa700 1 -- 172.21.15.78:0/3377687812 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f8f18082b30 con 0x7f8f280cc840 2023-10-11T09:49:45.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.795+0000 7f8f1ffff700 1 --2- 172.21.15.78:0/3377687812 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8f08065ed0 0x7f8f08068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:45.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.795+0000 7f8f1ffff700 1 --2- 172.21.15.78:0/3377687812 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8f08065ed0 0x7f8f08068380 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8f28159fe0 tx=0x7f8f10009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:45.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.799+0000 7f8f257fa700 1 -- 172.21.15.78:0/3377687812 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f8f281f5390 con 0x7f8f280cc840 2023-10-11T09:49:45.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:45.987+0000 7f8f2d19a700 1 -- 172.21.15.78:0/3377687812 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8f281f5390 con 0x7f8f280cc840 2023-10-11T09:49:46.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.758+0000 7f8f257fa700 1 -- 172.21.15.78:0/3377687812 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v101) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8f281f5390 con 0x7f8f280cc840 2023-10-11T09:49:46.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.762+0000 7f8f2d19a700 1 -- 172.21.15.78:0/3377687812 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8f08065ed0 msgr2=0x7f8f08068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:46.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.762+0000 7f8f2d19a700 1 --2- 172.21.15.78:0/3377687812 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8f08065ed0 0x7f8f08068380 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8f28159fe0 tx=0x7f8f10009040 comp rx=0 tx=0).stop 2023-10-11T09:49:46.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.762+0000 7f8f2d19a700 1 -- 172.21.15.78:0/3377687812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f280cc840 msgr2=0x7f8f28153690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:46.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.762+0000 7f8f2d19a700 1 --2- 172.21.15.78:0/3377687812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f280cc840 0x7f8f28153690 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f8f1800c560 tx=0x7f8f1800c870 comp rx=0 tx=0).stop 2023-10-11T09:49:46.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.762+0000 7f8f2d19a700 1 -- 172.21.15.78:0/3377687812 shutdown_connections 2023-10-11T09:49:46.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.762+0000 7f8f2d19a700 1 --2- 172.21.15.78:0/3377687812 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8f08065ed0 0x7f8f08068380 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:46.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.762+0000 7f8f2d19a700 1 --2- 172.21.15.78:0/3377687812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f281f6010 0x7f8f2815eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:46.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.762+0000 7f8f2d19a700 1 --2- 172.21.15.78:0/3377687812 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8f2815c7b0 0x7f8f2815e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:46.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.762+0000 7f8f2d19a700 1 --2- 172.21.15.78:0/3377687812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f280cc840 0x7f8f28153690 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:46.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.762+0000 7f8f2d19a700 1 -- 172.21.15.78:0/3377687812 >> 172.21.15.78:0/3377687812 conn(0x7f8f2806c590 msgr2=0x7f8f280b0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:46.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.762+0000 7f8f2d19a700 1 -- 172.21.15.78:0/3377687812 shutdown_connections 2023-10-11T09:49:46.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.762+0000 7f8f2d19a700 1 -- 172.21.15.78:0/3377687812 wait complete. 2023-10-11T09:49:46.771 INFO:teuthology.orchestra.run.smithi078.stderr: removed all blocklist entries 2023-10-11T09:49:46.790 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-11T09:49:46.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.958+0000 7f7f7723c700 1 Processor -- start 2023-10-11T09:49:46.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.958+0000 7f7f7723c700 1 -- start start 2023-10-11T09:49:46.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.962+0000 7f7f7723c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f7f70160d40 0x7f7f700cd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:46.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.962+0000 7f7f7723c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f70071040 0x7f7f700cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:46.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.962+0000 7f7f7723c700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7f700cd7c0 0x7f7f700ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:46.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.962+0000 7f7f7723c700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f70070c20 con 0x7f7f70160d40 2023-10-11T09:49:46.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.962+0000 7f7f7723c700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f70161110 con 0x7f7f700cd7c0 2023-10-11T09:49:46.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.962+0000 7f7f7723c700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7006ffa0 con 0x7f7f70071040 2023-10-11T09:49:46.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.962+0000 7f7f74fd8700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f7f70160d40 0x7f7f700cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:46.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.962+0000 7f7f74fd8700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f7f70160d40 0x7f7f700cd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41306/0 (socket says 172.21.15.78:41306) 2023-10-11T09:49:46.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.962+0000 7f7f74fd8700 1 -- 172.21.15.78:0/1760434177 learned_addr learned my addr 172.21.15.78:0/1760434177 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:46.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.962+0000 7f7f67fff700 1 --2- 172.21.15.78:0/1760434177 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f70071040 0x7f7f700cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:46.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.962+0000 7f7f757d9700 1 --2- 172.21.15.78:0/1760434177 >> v2:172.21.15.133:3300/0 conn(0x7f7f700cd7c0 0x7f7f700ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:46.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.962+0000 7f7f74fd8700 1 -- 172.21.15.78:0/1760434177 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f70071040 msgr2=0x7f7f700cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:46.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.962+0000 7f7f74fd8700 1 --2- 172.21.15.78:0/1760434177 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f70071040 0x7f7f700cddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:46.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.962+0000 7f7f74fd8700 1 -- 172.21.15.78:0/1760434177 >> v2:172.21.15.133:3300/0 conn(0x7f7f700cd7c0 msgr2=0x7f7f700ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:46.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.962+0000 7f7f74fd8700 1 --2- 172.21.15.78:0/1760434177 >> v2:172.21.15.133:3300/0 conn(0x7f7f700cd7c0 0x7f7f700ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:46.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.962+0000 7f7f74fd8700 1 -- 172.21.15.78:0/1760434177 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f70070280 con 0x7f7f70160d40 2023-10-11T09:49:46.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.962+0000 7f7f74fd8700 1 --2- 172.21.15.78:0/1760434177 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f70160d40 0x7f7f700cd280 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f7f700cec50 tx=0x7f7f6c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c230ef112c844d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:46.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.962+0000 7f7f677fe700 1 -- 172.21.15.78:0/1760434177 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c00b8a0 con 0x7f7f70160d40 2023-10-11T09:49:46.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.962+0000 7f7f677fe700 1 -- 172.21.15.78:0/1760434177 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f6c00ba00 con 0x7f7f70160d40 2023-10-11T09:49:46.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.962+0000 7f7f677fe700 1 -- 172.21.15.78:0/1760434177 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c00bd90 con 0x7f7f70160d40 2023-10-11T09:49:46.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.962+0000 7f7f7723c700 1 -- 172.21.15.78:0/1760434177 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f70160d40 msgr2=0x7f7f700cd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:46.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.962+0000 7f7f7723c700 1 --2- 172.21.15.78:0/1760434177 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f70160d40 0x7f7f700cd280 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f7f700cec50 tx=0x7f7f6c009bf0 comp rx=0 tx=0).stop 2023-10-11T09:49:46.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.962+0000 7f7f7723c700 1 -- 172.21.15.78:0/1760434177 shutdown_connections 2023-10-11T09:49:46.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.962+0000 7f7f7723c700 1 --2- 172.21.15.78:0/1760434177 >> v2:172.21.15.133:3300/0 conn(0x7f7f700cd7c0 0x7f7f700ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:46.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.962+0000 7f7f7723c700 1 --2- 172.21.15.78:0/1760434177 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f70071040 0x7f7f700cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:46.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.962+0000 7f7f7723c700 1 --2- 172.21.15.78:0/1760434177 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f70160d40 0x7f7f700cd280 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:46.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.962+0000 7f7f7723c700 1 -- 172.21.15.78:0/1760434177 >> 172.21.15.78:0/1760434177 conn(0x7f7f7006aca0 msgr2=0x7f7f700b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:46.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.962+0000 7f7f7723c700 1 -- 172.21.15.78:0/1760434177 shutdown_connections 2023-10-11T09:49:46.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.966+0000 7f7f7723c700 1 -- 172.21.15.78:0/1760434177 wait complete. 2023-10-11T09:49:46.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.966+0000 7f7f7723c700 1 Processor -- start 2023-10-11T09:49:46.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.966+0000 7f7f7723c700 1 -- start start 2023-10-11T09:49:46.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.966+0000 7f7f7723c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f70071040 0x7f7f70158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:46.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.966+0000 7f7f7723c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f700cd7c0 0x7f7f70155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:46.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.966+0000 7f7f7723c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f70160d40 0x7f7f701562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:46.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.966+0000 7f7f7723c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f700b2910 con 0x7f7f700cd7c0 2023-10-11T09:49:46.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.966+0000 7f7f7723c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f700b1c90 con 0x7f7f70071040 2023-10-11T09:49:46.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.966+0000 7f7f7723c700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f700b1f70 con 0x7f7f70160d40 2023-10-11T09:49:46.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.970+0000 7f7f67fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f700cd7c0 0x7f7f70155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:46.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.970+0000 7f7f67fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f700cd7c0 0x7f7f70155db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41318/0 (socket says 172.21.15.78:41318) 2023-10-11T09:49:46.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.970+0000 7f7f67fff700 1 -- 172.21.15.78:0/4255456222 learned_addr learned my addr 172.21.15.78:0/4255456222 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:46.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.970+0000 7f7f757d9700 1 --2- 172.21.15.78:0/4255456222 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f70160d40 0x7f7f701562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:46.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.970+0000 7f7f74fd8700 1 --2- 172.21.15.78:0/4255456222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f70071040 0x7f7f70158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:46.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.970+0000 7f7f67fff700 1 -- 172.21.15.78:0/4255456222 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f70160d40 msgr2=0x7f7f701562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:46.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.970+0000 7f7f67fff700 1 --2- 172.21.15.78:0/4255456222 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f70160d40 0x7f7f701562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:46.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.970+0000 7f7f67fff700 1 -- 172.21.15.78:0/4255456222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f70071040 msgr2=0x7f7f70158490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:46.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.970+0000 7f7f67fff700 1 --2- 172.21.15.78:0/4255456222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f70071040 0x7f7f70158490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:46.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.970+0000 7f7f67fff700 1 -- 172.21.15.78:0/4255456222 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6c009570 con 0x7f7f700cd7c0 2023-10-11T09:49:46.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.970+0000 7f7f67fff700 1 --2- 172.21.15.78:0/4255456222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f700cd7c0 0x7f7f70155db0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f7f6000e7e0 tx=0x7f7f6000eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:46.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.970+0000 7f7f657fa700 1 -- 172.21.15.78:0/4255456222 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6000c070 con 0x7f7f700cd7c0 2023-10-11T09:49:46.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.970+0000 7f7f657fa700 1 -- 172.21.15.78:0/4255456222 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f60013610 con 0x7f7f700cd7c0 2023-10-11T09:49:46.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.970+0000 7f7f657fa700 1 -- 172.21.15.78:0/4255456222 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f600139c0 con 0x7f7f700cd7c0 2023-10-11T09:49:46.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.970+0000 7f7f7723c700 1 -- 172.21.15.78:0/4255456222 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f700b2250 con 0x7f7f700cd7c0 2023-10-11T09:49:46.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.970+0000 7f7f7723c700 1 -- 172.21.15.78:0/4255456222 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f701589d0 con 0x7f7f700cd7c0 2023-10-11T09:49:46.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.970+0000 7f7f7723c700 1 -- 172.21.15.78:0/4255456222 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f70072250 con 0x7f7f700cd7c0 2023-10-11T09:49:46.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.974+0000 7f7f657fa700 1 -- 172.21.15.78:0/4255456222 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f7f60013b20 con 0x7f7f700cd7c0 2023-10-11T09:49:46.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.974+0000 7f7f657fa700 1 --2- 172.21.15.78:0/4255456222 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7f5c065ed0 0x7f7f5c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:46.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.974+0000 7f7f657fa700 1 -- 172.21.15.78:0/4255456222 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7f60083160 con 0x7f7f700cd7c0 2023-10-11T09:49:46.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.974+0000 7f7f74fd8700 1 --2- 172.21.15.78:0/4255456222 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7f5c065ed0 0x7f7f5c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:46.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.974+0000 7f7f74fd8700 1 --2- 172.21.15.78:0/4255456222 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7f5c065ed0 0x7f7f5c068380 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c000c00 tx=0x7f7f6c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:46.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:46.978+0000 7f7f657fa700 1 -- 172.21.15.78:0/4255456222 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f7f60050a70 con 0x7f7f700cd7c0 2023-10-11T09:49:47.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.210+0000 7f7f7723c700 1 -- 172.21.15.78:0/4255456222 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f7f700b2a50 con 0x7f7f700cd7c0 2023-10-11T09:49:47.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.210+0000 7f7f657fa700 1 -- 172.21.15.78:0/4255456222 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f7f700b2a50 con 0x7f7f700cd7c0 2023-10-11T09:49:47.218 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:49:47.218 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:49:47.219 INFO:teuthology.orchestra.run.smithi078.stdout: "auth_dump": [ 2023-10-11T09:49:47.219 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:49:47.219 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.a", 2023-10-11T09:49:47.219 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZlL58cAxAALe+WtkV/aLopy1w9i6OVRg==", 2023-10-11T09:49:47.219 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:49:47.219 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:49:47.220 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2023-10-11T09:49:47.220 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2023-10-11T09:49:47.220 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:49:47.220 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:49:47.220 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:49:47.221 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:49:47.221 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.b", 2023-10-11T09:49:47.221 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZliPLmFhAA+6wegaD6yTUsttFzo9OrIg==", 2023-10-11T09:49:47.221 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:49:47.221 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:49:47.222 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2023-10-11T09:49:47.222 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2023-10-11T09:49:47.222 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:49:47.222 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:49:47.223 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:49:47.223 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:49:47.223 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.c", 2023-10-11T09:49:47.223 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZl9jJBDhAAjos7/02OqF0WldFm70/fVA==", 2023-10-11T09:49:47.223 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:49:47.224 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:49:47.224 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2023-10-11T09:49:47.224 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2023-10-11T09:49:47.224 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:49:47.224 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:49:47.224 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:49:47.225 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:49:47.225 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.d", 2023-10-11T09:49:47.225 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZlTb/4IRAAeXtcbWVB5TnSoYch+uMZWw==", 2023-10-11T09:49:47.225 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:49:47.225 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:49:47.225 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2023-10-11T09:49:47.226 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2023-10-11T09:49:47.226 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:49:47.226 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:49:47.226 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:49:47.226 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:49:47.227 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.0", 2023-10-11T09:49:47.227 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBYbSZldbrVFxAA6ogUpCcFO8LhHn4zdkudNQ==", 2023-10-11T09:49:47.227 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:49:47.227 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:49:47.227 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:49:47.228 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:49:47.228 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:49:47.228 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:49:47.228 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:49:47.228 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.1", 2023-10-11T09:49:47.228 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBbbSZl+mkPJxAA4Rrt0j5lzXapYjXhdulhNQ==", 2023-10-11T09:49:47.229 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:49:47.229 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:49:47.229 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:49:47.229 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:49:47.229 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:49:47.229 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:49:47.230 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:49:47.230 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.2", 2023-10-11T09:49:47.230 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBebSZlZ6lSNRAA3ltw4ZTkFNpudCPc2zTYzQ==", 2023-10-11T09:49:47.230 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:49:47.230 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:49:47.230 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:49:47.231 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:49:47.231 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:49:47.231 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:49:47.231 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:49:47.231 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.3", 2023-10-11T09:49:47.231 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBibSZlsE+jBxAA2Ix2JAFTUxljpHAwPpJeUA==", 2023-10-11T09:49:47.232 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:49:47.232 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:49:47.232 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:49:47.232 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:49:47.232 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:49:47.232 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:49:47.232 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:49:47.233 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.4", 2023-10-11T09:49:47.233 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBobSZlII4OGxAA0Z5wGuu4wOs/14tQRnHB1Q==", 2023-10-11T09:49:47.233 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:49:47.233 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:49:47.233 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:49:47.234 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:49:47.234 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:49:47.234 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:49:47.234 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:49:47.234 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.5", 2023-10-11T09:49:47.235 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBrbSZlG6aSKhAAZq179SfpkelcJJLaUcLDCg==", 2023-10-11T09:49:47.235 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:49:47.235 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:49:47.235 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:49:47.235 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:49:47.235 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:49:47.236 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:49:47.236 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:49:47.236 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.6", 2023-10-11T09:49:47.236 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBubSZl3W89NxAAgUrd0/yMOWlrhAOU1BpM9A==", 2023-10-11T09:49:47.236 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:49:47.236 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:49:47.236 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:49:47.237 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:49:47.237 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:49:47.237 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:49:47.237 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:49:47.237 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.7", 2023-10-11T09:49:47.238 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBybSZloIjyCRAAjmvyK9zMSEsV8bVebwzPxg==", 2023-10-11T09:49:47.238 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:49:47.238 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:49:47.238 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:49:47.238 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:49:47.239 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:49:47.239 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:49:47.239 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:49:47.239 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.0", 2023-10-11T09:49:47.239 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZlmGmCKBAAOI8EvyRwg8HZt+q/vUy5Ow==", 2023-10-11T09:49:47.239 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:49:47.239 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:49:47.240 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow r", 2023-10-11T09:49:47.240 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-11T09:49:47.240 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:49:47.240 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:49:47.240 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:49:47.240 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.1", 2023-10-11T09:49:47.241 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZlYvfHNBAAReum0efx4MmqpLMxFnTROQ==", 2023-10-11T09:49:47.241 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:49:47.241 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:49:47.241 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2023-10-11T09:49:47.241 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2023-10-11T09:49:47.241 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2023-10-11T09:49:47.242 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:49:47.242 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:49:47.242 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:49:47.242 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.2", 2023-10-11T09:49:47.242 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZlbd9lLhAA9LAjreTcxwAjhiNlKndl+Q==", 2023-10-11T09:49:47.243 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:49:47.243 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:49:47.243 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2023-10-11T09:49:47.243 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2023-10-11T09:49:47.243 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2023-10-11T09:49:47.243 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:49:47.244 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:49:47.244 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:49:47.244 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.3", 2023-10-11T09:49:47.244 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZlHUjeOhAASPKTRvsillob4gjdW0akhQ==", 2023-10-11T09:49:47.244 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:49:47.244 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:49:47.244 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2023-10-11T09:49:47.245 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2023-10-11T09:49:47.245 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2023-10-11T09:49:47.245 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:49:47.245 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:49:47.245 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:49:47.245 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.admin", 2023-10-11T09:49:47.246 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBTbSZlhqcOFRAA/lkGYL03++trOuIqNLS0kQ==", 2023-10-11T09:49:47.246 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:49:47.246 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2023-10-11T09:49:47.246 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2023-10-11T09:49:47.246 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2023-10-11T09:49:47.247 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:49:47.247 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:49:47.247 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:49:47.247 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:49:47.247 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mgr.x", 2023-10-11T09:49:47.247 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBTbSZlS/TjNxAAMkq9xRMwgsbLlT4FdobATw==", 2023-10-11T09:49:47.248 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:49:47.248 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2023-10-11T09:49:47.248 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile mgr", 2023-10-11T09:49:47.248 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:49:47.248 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:49:47.249 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:49:47.249 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:49:47.249 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mgr.y", 2023-10-11T09:49:47.249 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBTbSZlnv4yMRAAQh8w18k76KRjdCvSDJ4QbA==", 2023-10-11T09:49:47.249 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:49:47.249 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2023-10-11T09:49:47.250 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile mgr", 2023-10-11T09:49:47.250 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:49:47.250 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:49:47.250 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:49:47.250 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2023-10-11T09:49:47.251 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:49:47.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.214+0000 7f7f7723c700 1 -- 172.21.15.78:0/4255456222 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7f5c065ed0 msgr2=0x7f7f5c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:47.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.214+0000 7f7f7723c700 1 --2- 172.21.15.78:0/4255456222 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7f5c065ed0 0x7f7f5c068380 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c000c00 tx=0x7f7f6c008030 comp rx=0 tx=0).stop 2023-10-11T09:49:47.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.214+0000 7f7f7723c700 1 -- 172.21.15.78:0/4255456222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f700cd7c0 msgr2=0x7f7f70155db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:47.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.214+0000 7f7f7723c700 1 --2- 172.21.15.78:0/4255456222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f700cd7c0 0x7f7f70155db0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f7f6000e7e0 tx=0x7f7f6000eaf0 comp rx=0 tx=0).stop 2023-10-11T09:49:47.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.214+0000 7f7f7723c700 1 -- 172.21.15.78:0/4255456222 shutdown_connections 2023-10-11T09:49:47.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.214+0000 7f7f7723c700 1 --2- 172.21.15.78:0/4255456222 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7f5c065ed0 0x7f7f5c068380 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:47.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.214+0000 7f7f7723c700 1 --2- 172.21.15.78:0/4255456222 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f70160d40 0x7f7f701562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:47.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.214+0000 7f7f7723c700 1 --2- 172.21.15.78:0/4255456222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f700cd7c0 0x7f7f70155db0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:47.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.214+0000 7f7f7723c700 1 --2- 172.21.15.78:0/4255456222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f70071040 0x7f7f70158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:47.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.214+0000 7f7f7723c700 1 -- 172.21.15.78:0/4255456222 >> 172.21.15.78:0/4255456222 conn(0x7f7f7006aca0 msgr2=0x7f7f700cb710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:47.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.214+0000 7f7f7723c700 1 -- 172.21.15.78:0/4255456222 shutdown_connections 2023-10-11T09:49:47.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.214+0000 7f7f7723c700 1 -- 172.21.15.78:0/4255456222 wait complete. 2023-10-11T09:49:47.255 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:49:47.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.422+0000 7f5f3731d700 1 Processor -- start 2023-10-11T09:49:47.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.422+0000 7f5f3731d700 1 -- start start 2023-10-11T09:49:47.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.422+0000 7f5f3731d700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5f300cc820 0x7f5f30160c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:47.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.422+0000 7f5f3731d700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f30160210 con 0x7f5f300b06a0 2023-10-11T09:49:47.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.422+0000 7f5f3731d700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f301611a0 con 0x7f5f3015d050 2023-10-11T09:49:47.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.422+0000 7f5f3731d700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f300ccfc0 con 0x7f5f300cc820 2023-10-11T09:49:47.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.426+0000 7f5f36b1c700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f5f300b06a0 0x7f5f300b0ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:42206/0 (socket says 172.21.15.78:42206) 2023-10-11T09:49:47.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.426+0000 7f5f36b1c700 1 -- 172.21.15.78:0/4075121167 learned_addr learned my addr 172.21.15.78:0/4075121167 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:49:47.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.426+0000 7f5f35b1a700 1 --2- 172.21.15.78:0/4075121167 >> v2:172.21.15.133:3300/0 conn(0x7f5f300cc820 0x7f5f30160c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:47.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.426+0000 7f5f35b1a700 1 -- 172.21.15.78:0/4075121167 >> v1:172.21.15.133:6789/0 conn(0x7f5f3015d050 legacy=0x7f5f3015d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:47.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.426+0000 7f5f35b1a700 1 -- 172.21.15.78:0/4075121167 >> v1:172.21.15.78:6789/0 conn(0x7f5f300b06a0 legacy=0x7f5f300b0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:47.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.426+0000 7f5f35b1a700 1 -- 172.21.15.78:0/4075121167 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f300cd2a0 con 0x7f5f300cc820 2023-10-11T09:49:47.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.426+0000 7f5f35319700 1 -- 172.21.15.78:0/4075121167 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4136755712 0 0) 0x7f5f30160210 con 0x7f5f300b06a0 2023-10-11T09:49:47.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.426+0000 7f5f35b1a700 1 --2- 172.21.15.78:0/4075121167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f300cc820 0x7f5f30160c60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009850 tx=0x7f5f20009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dd7e2c49972bc36 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:47.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.426+0000 7f5f35319700 1 -- 172.21.15.78:0/4075121167 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f20011830 con 0x7f5f300cc820 2023-10-11T09:49:47.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.426+0000 7f5f35319700 1 -- 172.21.15.78:0/4075121167 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f20011990 con 0x7f5f300cc820 2023-10-11T09:49:47.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.426+0000 7f5f35319700 1 -- 172.21.15.78:0/4075121167 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f20011d20 con 0x7f5f300cc820 2023-10-11T09:49:47.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.426+0000 7f5f3731d700 1 -- 172.21.15.78:0/4075121167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f300cc820 msgr2=0x7f5f30160c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:47.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.426+0000 7f5f3731d700 1 --2- 172.21.15.78:0/4075121167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f300cc820 0x7f5f30160c60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009850 tx=0x7f5f20009b60 comp rx=0 tx=0).stop 2023-10-11T09:49:47.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.426+0000 7f5f3731d700 1 -- 172.21.15.78:0/4075121167 shutdown_connections 2023-10-11T09:49:47.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.426+0000 7f5f3731d700 1 --2- 172.21.15.78:0/4075121167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f300cc820 0x7f5f30160c60 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:47.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.426+0000 7f5f3731d700 1 -- 172.21.15.78:0/4075121167 >> 172.21.15.78:0/4075121167 conn(0x7f5f300cea60 msgr2=0x7f5f300cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:47.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.426+0000 7f5f3731d700 1 -- 172.21.15.78:0/4075121167 shutdown_connections 2023-10-11T09:49:47.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.426+0000 7f5f3731d700 1 -- 172.21.15.78:0/4075121167 wait complete. 2023-10-11T09:49:47.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.430+0000 7f5f3731d700 1 Processor -- start 2023-10-11T09:49:47.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.430+0000 7f5f3731d700 1 -- start start 2023-10-11T09:49:47.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.430+0000 7f5f3731d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5f300b06a0 0x7f5f30144550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:47.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.430+0000 7f5f3731d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f300cc820 0x7f5f300b4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:47.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.430+0000 7f5f3731d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f3015d050 0x7f5f300b4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:47.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.430+0000 7f5f3731d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f30144a90 con 0x7f5f300cc820 2023-10-11T09:49:47.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.430+0000 7f5f3731d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f30070250 con 0x7f5f3015d050 2023-10-11T09:49:47.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.430+0000 7f5f3731d700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3015e710 con 0x7f5f300b06a0 2023-10-11T09:49:47.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.434+0000 7f5f35b1a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f300cc820 0x7f5f300b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:47.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.434+0000 7f5f35b1a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f300cc820 0x7f5f300b4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41326/0 (socket says 172.21.15.78:41326) 2023-10-11T09:49:47.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.434+0000 7f5f35b1a700 1 -- 172.21.15.78:0/2283153647 learned_addr learned my addr 172.21.15.78:0/2283153647 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:47.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.434+0000 7f5f3631b700 1 --2- 172.21.15.78:0/2283153647 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5f300b06a0 0x7f5f30144550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:47.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.434+0000 7f5f36b1c700 1 --2- 172.21.15.78:0/2283153647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f3015d050 0x7f5f300b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:47.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.434+0000 7f5f35b1a700 1 -- 172.21.15.78:0/2283153647 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5f300b06a0 msgr2=0x7f5f30144550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:47.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.434+0000 7f5f35b1a700 1 --2- 172.21.15.78:0/2283153647 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5f300b06a0 0x7f5f30144550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:47.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.434+0000 7f5f35b1a700 1 -- 172.21.15.78:0/2283153647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f3015d050 msgr2=0x7f5f300b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:47.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.434+0000 7f5f35b1a700 1 --2- 172.21.15.78:0/2283153647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f3015d050 0x7f5f300b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:47.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.434+0000 7f5f35b1a700 1 -- 172.21.15.78:0/2283153647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f200094b0 con 0x7f5f300cc820 2023-10-11T09:49:47.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.434+0000 7f5f35b1a700 1 --2- 172.21.15.78:0/2283153647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f300cc820 0x7f5f300b4220 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009fd0 tx=0x7f5f20004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:47.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.438+0000 7f5f26ffd700 1 -- 172.21.15.78:0/2283153647 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f200036c0 con 0x7f5f300cc820 2023-10-11T09:49:47.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.438+0000 7f5f3731d700 1 -- 172.21.15.78:0/2283153647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f3015e9f0 con 0x7f5f300cc820 2023-10-11T09:49:47.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.438+0000 7f5f26ffd700 1 -- 172.21.15.78:0/2283153647 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f2001e070 con 0x7f5f300cc820 2023-10-11T09:49:47.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.438+0000 7f5f26ffd700 1 -- 172.21.15.78:0/2283153647 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f20003a60 con 0x7f5f300cc820 2023-10-11T09:49:47.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.438+0000 7f5f3731d700 1 -- 172.21.15.78:0/2283153647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f30070510 con 0x7f5f300cc820 2023-10-11T09:49:47.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.438+0000 7f5f3731d700 1 -- 172.21.15.78:0/2283153647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f30065d40 con 0x7f5f300cc820 2023-10-11T09:49:47.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.442+0000 7f5f26ffd700 1 -- 172.21.15.78:0/2283153647 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f5f20003bc0 con 0x7f5f300cc820 2023-10-11T09:49:47.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.442+0000 7f5f26ffd700 1 --2- 172.21.15.78:0/2283153647 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5f14065ed0 0x7f5f14068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:47.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.442+0000 7f5f26ffd700 1 -- 172.21.15.78:0/2283153647 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5f200846d0 con 0x7f5f300cc820 2023-10-11T09:49:47.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.442+0000 7f5f3631b700 1 --2- 172.21.15.78:0/2283153647 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5f14065ed0 0x7f5f14068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:47.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.442+0000 7f5f3631b700 1 --2- 172.21.15.78:0/2283153647 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5f14065ed0 0x7f5f14068380 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c000f60 tx=0x7f5f2c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:47.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.450+0000 7f5f26ffd700 1 -- 172.21.15.78:0/2283153647 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f5f20051fe0 con 0x7f5f300cc820 2023-10-11T09:49:47.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.630+0000 7f5f3731d700 1 -- 172.21.15.78:0/2283153647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f30070700 con 0x7f5f300cc820 2023-10-11T09:49:47.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.634+0000 7f5f26ffd700 1 -- 172.21.15.78:0/2283153647 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+14173 (secure 0 0 0) 0x7f5f20084070 con 0x7f5f300cc820 2023-10-11T09:49:47.641 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:49:47.641 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":101,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:49:46.749939+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:49:47.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.634+0000 7f5f3731d700 1 -- 172.21.15.78:0/2283153647 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5f14065ed0 msgr2=0x7f5f14068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:47.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.634+0000 7f5f3731d700 1 --2- 172.21.15.78:0/2283153647 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5f14065ed0 0x7f5f14068380 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c000f60 tx=0x7f5f2c009040 comp rx=0 tx=0).stop 2023-10-11T09:49:47.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.634+0000 7f5f3731d700 1 -- 172.21.15.78:0/2283153647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f300cc820 msgr2=0x7f5f300b4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:47.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.634+0000 7f5f3731d700 1 --2- 172.21.15.78:0/2283153647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f300cc820 0x7f5f300b4220 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009fd0 tx=0x7f5f20004770 comp rx=0 tx=0).stop 2023-10-11T09:49:47.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.638+0000 7f5f3731d700 1 -- 172.21.15.78:0/2283153647 shutdown_connections 2023-10-11T09:49:47.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.638+0000 7f5f3731d700 1 --2- 172.21.15.78:0/2283153647 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5f14065ed0 0x7f5f14068380 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:47.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.638+0000 7f5f3731d700 1 --2- 172.21.15.78:0/2283153647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f3015d050 0x7f5f300b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:47.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.638+0000 7f5f3731d700 1 --2- 172.21.15.78:0/2283153647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f300cc820 0x7f5f300b4220 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:47.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.638+0000 7f5f3731d700 1 --2- 172.21.15.78:0/2283153647 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5f300b06a0 0x7f5f30144550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:47.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.638+0000 7f5f3731d700 1 -- 172.21.15.78:0/2283153647 >> 172.21.15.78:0/2283153647 conn(0x7f5f300cea60 msgr2=0x7f5f3006c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:47.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.638+0000 7f5f3731d700 1 -- 172.21.15.78:0/2283153647 shutdown_connections 2023-10-11T09:49:47.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.638+0000 7f5f3731d700 1 -- 172.21.15.78:0/2283153647 wait complete. 2023-10-11T09:49:47.664 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:49:47.665 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:49:47.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.842+0000 7ff452397700 1 Processor -- start 2023-10-11T09:49:47.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.842+0000 7ff452397700 1 -- start start 2023-10-11T09:49:47.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.846+0000 7ff452397700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff44c0ccf20 0x7ff44c15cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:47.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.846+0000 7ff452397700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff44c15a0d0 con 0x7ff44c159c50 2023-10-11T09:49:47.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.846+0000 7ff452397700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff44c0cfe00 con 0x7ff44c0ceca0 2023-10-11T09:49:47.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.846+0000 7ff452397700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff44c0737d0 con 0x7ff44c0ccf20 2023-10-11T09:49:47.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.846+0000 7ff450b94700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7ff44c159c50 0x7ff44c0cf6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:42214/0 (socket says 172.21.15.78:42214) 2023-10-11T09:49:47.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.846+0000 7ff450b94700 1 -- 172.21.15.78:0/1564461851 learned_addr learned my addr 172.21.15.78:0/1564461851 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:49:47.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.846+0000 7ff451b96700 1 --2- 172.21.15.78:0/1564461851 >> v2:172.21.15.133:3300/0 conn(0x7ff44c0ccf20 0x7ff44c15cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:47.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.846+0000 7ff443fff700 1 -- 172.21.15.78:0/1564461851 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3598310306 0 0) 0x7ff44c15a0d0 con 0x7ff44c159c50 2023-10-11T09:49:47.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.846+0000 7ff443fff700 1 -- 172.21.15.78:0/1564461851 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4380035d0 con 0x7ff44c159c50 2023-10-11T09:49:47.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.846+0000 7ff443fff700 1 -- 172.21.15.78:0/1564461851 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3778761211 0 0) 0x7ff44c0cfe00 con 0x7ff44c0ceca0 2023-10-11T09:49:47.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.846+0000 7ff443fff700 1 -- 172.21.15.78:0/1564461851 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff44c15a0d0 con 0x7ff44c0ceca0 2023-10-11T09:49:47.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.846+0000 7ff443fff700 1 -- 172.21.15.78:0/1564461851 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4250263641 0 0) 0x7ff4380035d0 con 0x7ff44c159c50 2023-10-11T09:49:47.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.846+0000 7ff443fff700 1 -- 172.21.15.78:0/1564461851 >> v2:172.21.15.133:3300/0 conn(0x7ff44c0ccf20 msgr2=0x7ff44c15cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:47.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.846+0000 7ff443fff700 1 --2- 172.21.15.78:0/1564461851 >> v2:172.21.15.133:3300/0 conn(0x7ff44c0ccf20 0x7ff44c15cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:47.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.846+0000 7ff443fff700 1 -- 172.21.15.78:0/1564461851 >> v1:172.21.15.133:6789/0 conn(0x7ff44c0ceca0 legacy=0x7ff44c0730c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:47.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.846+0000 7ff443fff700 1 -- 172.21.15.78:0/1564461851 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44c15c860 con 0x7ff44c159c50 2023-10-11T09:49:47.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.846+0000 7ff443fff700 1 -- 172.21.15.78:0/1564461851 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7ff43c001670 con 0x7ff44c159c50 2023-10-11T09:49:47.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.846+0000 7ff443fff700 1 -- 172.21.15.78:0/1564461851 >> v1:172.21.15.78:6789/0 conn(0x7ff44c159c50 legacy=0x7ff44c0cf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:47.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.846+0000 7ff443fff700 1 --2- 172.21.15.78:0/1564461851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff438003bb0 0x7ff438003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:47.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.846+0000 7ff443fff700 1 --2- 172.21.15.78:0/1564461851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff438004870 0x7ff438004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:47.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.846+0000 7ff443fff700 1 --2- 172.21.15.78:0/1564461851 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff438005220 0x7ff438010040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:47.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.846+0000 7ff443fff700 1 -- 172.21.15.78:0/1564461851 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c0737d0 con 0x7ff438003bb0 2023-10-11T09:49:47.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.846+0000 7ff443fff700 1 -- 172.21.15.78:0/1564461851 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff438010580 con 0x7ff438004870 2023-10-11T09:49:47.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.846+0000 7ff443fff700 1 -- 172.21.15.78:0/1564461851 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4380106c0 con 0x7ff438005220 2023-10-11T09:49:47.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.846+0000 7ff451395700 1 --2- 172.21.15.78:0/1564461851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff438003bb0 0x7ff438003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:47.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.846+0000 7ff451395700 1 -- 172.21.15.78:0/1564461851 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff438005220 msgr2=0x7ff438010040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:47.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.846+0000 7ff451395700 1 --2- 172.21.15.78:0/1564461851 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff438005220 0x7ff438010040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:47.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.846+0000 7ff451395700 1 -- 172.21.15.78:0/1564461851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff438004870 msgr2=0x7ff438004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:47.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.846+0000 7ff451395700 1 --2- 172.21.15.78:0/1564461851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff438004870 0x7ff438004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:47.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.846+0000 7ff451395700 1 -- 172.21.15.78:0/1564461851 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4380035d0 con 0x7ff438003bb0 2023-10-11T09:49:47.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.846+0000 7ff451b96700 1 -- 172.21.15.78:0/1564461851 reap_dead start 2023-10-11T09:49:47.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.846+0000 7ff451395700 1 --2- 172.21.15.78:0/1564461851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff438003bb0 0x7ff438003fa0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7ff44800c2f0 tx=0x7ff4380058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74c115078d94aaac server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:47.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.850+0000 7ff443fff700 1 -- 172.21.15.78:0/1564461851 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff448001f30 con 0x7ff438003bb0 2023-10-11T09:49:47.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.850+0000 7ff443fff700 1 -- 172.21.15.78:0/1564461851 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff44800ce90 con 0x7ff438003bb0 2023-10-11T09:49:47.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.850+0000 7ff443fff700 1 -- 172.21.15.78:0/1564461851 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff448003c60 con 0x7ff438003bb0 2023-10-11T09:49:47.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.850+0000 7ff452397700 1 -- 172.21.15.78:0/1564461851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff438003bb0 msgr2=0x7ff438003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:47.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.850+0000 7ff452397700 1 --2- 172.21.15.78:0/1564461851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff438003bb0 0x7ff438003fa0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7ff44800c2f0 tx=0x7ff4380058a0 comp rx=0 tx=0).stop 2023-10-11T09:49:47.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.850+0000 7ff452397700 1 -- 172.21.15.78:0/1564461851 shutdown_connections 2023-10-11T09:49:47.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.850+0000 7ff452397700 1 --2- 172.21.15.78:0/1564461851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff438003bb0 0x7ff438003fa0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:47.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.850+0000 7ff452397700 1 -- 172.21.15.78:0/1564461851 >> 172.21.15.78:0/1564461851 conn(0x7ff44c0b0f40 msgr2=0x7ff44c0b1310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:47.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.850+0000 7ff452397700 1 -- 172.21.15.78:0/1564461851 shutdown_connections 2023-10-11T09:49:47.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.850+0000 7ff452397700 1 -- 172.21.15.78:0/1564461851 wait complete. 2023-10-11T09:49:47.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.850+0000 7ff452397700 1 Processor -- start 2023-10-11T09:49:47.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.850+0000 7ff452397700 1 -- start start 2023-10-11T09:49:47.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.854+0000 7ff452397700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff438003bb0 0x7ff44c153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:47.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.854+0000 7ff452397700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff44c0b4090 0x7ff44c0b4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:47.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.854+0000 7ff452397700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff44c1e3630 0x7ff44c1e3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:47.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.854+0000 7ff452397700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c153bd0 con 0x7ff44c1e3630 2023-10-11T09:49:47.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.854+0000 7ff452397700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c0b4a40 con 0x7ff438003bb0 2023-10-11T09:49:47.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.854+0000 7ff452397700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c1e3fe0 con 0x7ff44c0b4090 2023-10-11T09:49:47.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.854+0000 7ff451b96700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff44c1e3630 0x7ff44c1e3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:47.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.854+0000 7ff451b96700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff44c1e3630 0x7ff44c1e3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41338/0 (socket says 172.21.15.78:41338) 2023-10-11T09:49:47.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.854+0000 7ff451b96700 1 -- 172.21.15.78:0/1449625605 learned_addr learned my addr 172.21.15.78:0/1449625605 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:47.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.854+0000 7ff451395700 1 --2- 172.21.15.78:0/1449625605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff438003bb0 0x7ff44c153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:47.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.854+0000 7ff450b94700 1 --2- 172.21.15.78:0/1449625605 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff44c0b4090 0x7ff44c0b4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:47.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.854+0000 7ff451b96700 1 -- 172.21.15.78:0/1449625605 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff44c0b4090 msgr2=0x7ff44c0b4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:47.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.854+0000 7ff451b96700 1 --2- 172.21.15.78:0/1449625605 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff44c0b4090 0x7ff44c0b4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:47.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.854+0000 7ff451b96700 1 -- 172.21.15.78:0/1449625605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff438003bb0 msgr2=0x7ff44c153000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:47.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.854+0000 7ff451b96700 1 --2- 172.21.15.78:0/1449625605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff438003bb0 0x7ff44c153000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:47.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.854+0000 7ff451b96700 1 -- 172.21.15.78:0/1449625605 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44c1e4240 con 0x7ff44c1e3630 2023-10-11T09:49:47.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.854+0000 7ff450b94700 1 --2- 172.21.15.78:0/1449625605 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff44c0b4090 0x7ff44c0b4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:49:47.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.854+0000 7ff451395700 1 --2- 172.21.15.78:0/1449625605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff438003bb0 0x7ff44c153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:49:47.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.854+0000 7ff451b96700 1 --2- 172.21.15.78:0/1449625605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff44c1e3630 0x7ff44c1e3aa0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7ff44c0cac00 tx=0x7ff44400ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:47.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.854+0000 7ff441ffb700 1 -- 172.21.15.78:0/1449625605 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff444013680 con 0x7ff44c1e3630 2023-10-11T09:49:47.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.854+0000 7ff441ffb700 1 -- 172.21.15.78:0/1449625605 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff444013800 con 0x7ff44c1e3630 2023-10-11T09:49:47.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.854+0000 7ff452397700 1 -- 172.21.15.78:0/1449625605 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff44c0b22e0 con 0x7ff44c1e3630 2023-10-11T09:49:47.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.854+0000 7ff441ffb700 1 -- 172.21.15.78:0/1449625605 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff444013bb0 con 0x7ff44c1e3630 2023-10-11T09:49:47.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.854+0000 7ff452397700 1 -- 172.21.15.78:0/1449625605 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff44c0b28a0 con 0x7ff44c1e3630 2023-10-11T09:49:47.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.858+0000 7ff452397700 1 -- 172.21.15.78:0/1449625605 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff44c065d40 con 0x7ff44c1e3630 2023-10-11T09:49:47.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.858+0000 7ff441ffb700 1 -- 172.21.15.78:0/1449625605 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7ff444013d10 con 0x7ff44c1e3630 2023-10-11T09:49:47.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.858+0000 7ff441ffb700 1 --2- 172.21.15.78:0/1449625605 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff430065f00 0x7ff4300683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:47.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.858+0000 7ff441ffb700 1 -- 172.21.15.78:0/1449625605 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff444082a90 con 0x7ff44c1e3630 2023-10-11T09:49:47.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.858+0000 7ff451395700 1 --2- 172.21.15.78:0/1449625605 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff430065f00 0x7ff4300683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:47.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.862+0000 7ff451395700 1 --2- 172.21.15.78:0/1449625605 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff430065f00 0x7ff4300683b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ff448015040 tx=0x7ff44800a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:47.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:47.866+0000 7ff441ffb700 1 -- 172.21.15.78:0/1449625605 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7ff4440503a0 con 0x7ff44c1e3630 2023-10-11T09:49:48.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.038+0000 7ff452397700 1 -- 172.21.15.78:0/1449625605 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff44c0b13b0 con 0x7ff44c1e3630 2023-10-11T09:49:48.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.038+0000 7ff441ffb700 1 -- 172.21.15.78:0/1449625605 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff4440539c0 con 0x7ff44c1e3630 2023-10-11T09:49:48.045 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:49:48.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.042+0000 7ff452397700 1 -- 172.21.15.78:0/1449625605 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff430065f00 msgr2=0x7ff4300683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:48.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.042+0000 7ff452397700 1 --2- 172.21.15.78:0/1449625605 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff430065f00 0x7ff4300683b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ff448015040 tx=0x7ff44800a000 comp rx=0 tx=0).stop 2023-10-11T09:49:48.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.042+0000 7ff452397700 1 -- 172.21.15.78:0/1449625605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff44c1e3630 msgr2=0x7ff44c1e3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:48.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.042+0000 7ff452397700 1 --2- 172.21.15.78:0/1449625605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff44c1e3630 0x7ff44c1e3aa0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7ff44c0cac00 tx=0x7ff44400ea50 comp rx=0 tx=0).stop 2023-10-11T09:49:48.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.042+0000 7ff452397700 1 -- 172.21.15.78:0/1449625605 shutdown_connections 2023-10-11T09:49:48.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.042+0000 7ff452397700 1 --2- 172.21.15.78:0/1449625605 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff430065f00 0x7ff4300683b0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:48.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.042+0000 7ff452397700 1 --2- 172.21.15.78:0/1449625605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff44c1e3630 0x7ff44c1e3aa0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:48.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.042+0000 7ff452397700 1 --2- 172.21.15.78:0/1449625605 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff44c0b4090 0x7ff44c0b4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:48.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.042+0000 7ff452397700 1 --2- 172.21.15.78:0/1449625605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff438003bb0 0x7ff44c153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:48.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.042+0000 7ff452397700 1 -- 172.21.15.78:0/1449625605 >> 172.21.15.78:0/1449625605 conn(0x7ff44c0b0f40 msgr2=0x7ff44c0730c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:48.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.042+0000 7ff452397700 1 -- 172.21.15.78:0/1449625605 shutdown_connections 2023-10-11T09:49:48.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.042+0000 7ff452397700 1 -- 172.21.15.78:0/1449625605 wait complete. 2023-10-11T09:49:48.069 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:49:48.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.238+0000 7f33b2d58700 1 Processor -- start 2023-10-11T09:49:48.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.238+0000 7f33b2d58700 1 -- start start 2023-10-11T09:49:48.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.242+0000 7f33b2d58700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f33ac06d2f0 0x7f33ac16b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:48.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.242+0000 7f33b2d58700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33ac0ae280 con 0x7f33ac0caf90 2023-10-11T09:49:48.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.242+0000 7f33b2d58700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33ac06be50 con 0x7f33ac160c50 2023-10-11T09:49:48.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.242+0000 7f33b2d58700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac06b520 con 0x7f33ac06d2f0 2023-10-11T09:49:48.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.242+0000 7f33b1555700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f33ac0caf90 0x7f33ac0cd7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:42230/0 (socket says 172.21.15.78:42230) 2023-10-11T09:49:48.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.242+0000 7f33b1555700 1 -- 172.21.15.78:0/2432327713 learned_addr learned my addr 172.21.15.78:0/2432327713 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:49:48.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.242+0000 7f33b2557700 1 --2- 172.21.15.78:0/2432327713 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f33ac06d2f0 0x7f33ac16b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:48.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.242+0000 7f33b0d54700 1 -- 172.21.15.78:0/2432327713 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1921595698 0 0) 0x7f339c001110 con 0x7f33ac0caf90 2023-10-11T09:49:48.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.242+0000 7f33b0d54700 1 -- 172.21.15.78:0/2432327713 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33980035d0 con 0x7f33ac0caf90 2023-10-11T09:49:48.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.242+0000 7f33b0d54700 1 -- 172.21.15.78:0/2432327713 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 365883244 0 0) 0x7f33ac06be50 con 0x7f33ac160c50 2023-10-11T09:49:48.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.242+0000 7f33b0d54700 1 -- 172.21.15.78:0/2432327713 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f339c001110 con 0x7f33ac160c50 2023-10-11T09:49:48.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.242+0000 7f33b2557700 1 -- 172.21.15.78:0/2432327713 >> v1:172.21.15.133:6789/0 conn(0x7f33ac160c50 legacy=0x7f33ac0ad9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:48.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.242+0000 7f33b2557700 1 -- 172.21.15.78:0/2432327713 >> v1:172.21.15.78:6789/0 conn(0x7f33ac0caf90 legacy=0x7f33ac0cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:48.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.242+0000 7f33b2557700 1 -- 172.21.15.78:0/2432327713 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ac06b800 con 0x7f33ac06d2f0 2023-10-11T09:49:48.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.242+0000 7f33b2557700 1 --2- 172.21.15.78:0/2432327713 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f33ac06d2f0 0x7f33ac16b7f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f33a4009dc0 tx=0x7f33a400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a526a4e683a9b7f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:48.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.246+0000 7f33b0d54700 1 -- 172.21.15.78:0/2432327713 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a400e070 con 0x7f33ac06d2f0 2023-10-11T09:49:48.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.246+0000 7f33b0d54700 1 -- 172.21.15.78:0/2432327713 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a400bb20 con 0x7f33ac06d2f0 2023-10-11T09:49:48.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.246+0000 7f33b0d54700 1 -- 172.21.15.78:0/2432327713 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a4012430 con 0x7f33ac06d2f0 2023-10-11T09:49:48.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.246+0000 7f33b2d58700 1 -- 172.21.15.78:0/2432327713 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f33ac06d2f0 msgr2=0x7f33ac16b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:48.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.246+0000 7f33b2d58700 1 --2- 172.21.15.78:0/2432327713 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f33ac06d2f0 0x7f33ac16b7f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f33a4009dc0 tx=0x7f33a400b330 comp rx=0 tx=0).stop 2023-10-11T09:49:48.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.246+0000 7f33b2d58700 1 -- 172.21.15.78:0/2432327713 shutdown_connections 2023-10-11T09:49:48.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.246+0000 7f33b2d58700 1 --2- 172.21.15.78:0/2432327713 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f33ac06d2f0 0x7f33ac16b7f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:48.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.246+0000 7f33b2d58700 1 -- 172.21.15.78:0/2432327713 >> 172.21.15.78:0/2432327713 conn(0x7f33ac0cc070 msgr2=0x7f33ac0ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:48.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.246+0000 7f33b2d58700 1 -- 172.21.15.78:0/2432327713 shutdown_connections 2023-10-11T09:49:48.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.246+0000 7f33b2d58700 1 -- 172.21.15.78:0/2432327713 wait complete. 2023-10-11T09:49:48.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.246+0000 7f33b2d58700 1 Processor -- start 2023-10-11T09:49:48.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.246+0000 7f33b2d58700 1 -- start start 2023-10-11T09:49:48.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.250+0000 7f33b2d58700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f33ac0caf90 0x7f33ac150e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:48.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.250+0000 7f33b2d58700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33ac160c50 0x7f33ac0c6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:48.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.250+0000 7f33b2d58700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33ac0b37c0 0x7f33ac0b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:48.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.250+0000 7f33b2d58700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac0b41a0 con 0x7f33ac160c50 2023-10-11T09:49:48.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.250+0000 7f33b2d58700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac1517b0 con 0x7f33ac0b37c0 2023-10-11T09:49:48.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.250+0000 7f33b2d58700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac0aeef0 con 0x7f33ac0caf90 2023-10-11T09:49:48.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.250+0000 7f33b1555700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33ac160c50 0x7f33ac0c6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:48.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.250+0000 7f33b1555700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33ac160c50 0x7f33ac0c6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41342/0 (socket says 172.21.15.78:41342) 2023-10-11T09:49:48.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.250+0000 7f33b1555700 1 -- 172.21.15.78:0/1999722766 learned_addr learned my addr 172.21.15.78:0/1999722766 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:48.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.250+0000 7f33b1d56700 1 --2- 172.21.15.78:0/1999722766 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f33ac0caf90 0x7f33ac150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:48.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.250+0000 7f33b2557700 1 --2- 172.21.15.78:0/1999722766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33ac0b37c0 0x7f33ac0b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:48.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.250+0000 7f33b1555700 1 -- 172.21.15.78:0/1999722766 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f33ac0caf90 msgr2=0x7f33ac150e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:48.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.250+0000 7f33b1555700 1 --2- 172.21.15.78:0/1999722766 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f33ac0caf90 0x7f33ac150e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:48.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.250+0000 7f33b1555700 1 -- 172.21.15.78:0/1999722766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33ac0b37c0 msgr2=0x7f33ac0b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:48.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.250+0000 7f33b1555700 1 --2- 172.21.15.78:0/1999722766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33ac0b37c0 0x7f33ac0b3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:48.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.250+0000 7f33b1555700 1 -- 172.21.15.78:0/1999722766 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a4009a70 con 0x7f33ac160c50 2023-10-11T09:49:48.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.250+0000 7f33b1555700 1 --2- 172.21.15.78:0/1999722766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33ac160c50 0x7f33ac0c6400 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f339c00c700 tx=0x7f339c00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:48.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.250+0000 7f33a27fc700 1 -- 172.21.15.78:0/1999722766 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339c005d90 con 0x7f33ac160c50 2023-10-11T09:49:48.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.250+0000 7f33a27fc700 1 -- 172.21.15.78:0/1999722766 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f339c0070d0 con 0x7f33ac160c50 2023-10-11T09:49:48.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.250+0000 7f33a27fc700 1 -- 172.21.15.78:0/1999722766 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339c007480 con 0x7f33ac160c50 2023-10-11T09:49:48.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.250+0000 7f33b2d58700 1 -- 172.21.15.78:0/1999722766 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33ac1513c0 con 0x7f33ac160c50 2023-10-11T09:49:48.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.250+0000 7f33b2d58700 1 -- 172.21.15.78:0/1999722766 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33ac0af420 con 0x7f33ac160c50 2023-10-11T09:49:48.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.254+0000 7f33b2d58700 1 -- 172.21.15.78:0/1999722766 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33ac04ed80 con 0x7f33ac160c50 2023-10-11T09:49:48.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.254+0000 7f33a27fc700 1 -- 172.21.15.78:0/1999722766 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f339c0075e0 con 0x7f33ac160c50 2023-10-11T09:49:48.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.254+0000 7f33a27fc700 1 --2- 172.21.15.78:0/1999722766 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3390065f00 0x7f33900683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:48.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.254+0000 7f33a27fc700 1 -- 172.21.15.78:0/1999722766 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f339c082910 con 0x7f33ac160c50 2023-10-11T09:49:48.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.254+0000 7f33b1d56700 1 --2- 172.21.15.78:0/1999722766 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3390065f00 0x7f33900683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:48.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.258+0000 7f33b1d56700 1 --2- 172.21.15.78:0/1999722766 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3390065f00 0x7f33900683b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f33a80011e0 tx=0x7f33a8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:48.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.262+0000 7f33a27fc700 1 -- 172.21.15.78:0/1999722766 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f339c050210 con 0x7f33ac160c50 2023-10-11T09:49:48.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.430+0000 7f33b2d58700 1 -- 172.21.15.78:0/1999722766 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f33ac0ae280 con 0x7f33ac160c50 2023-10-11T09:49:48.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.434+0000 7f33a27fc700 1 -- 172.21.15.78:0/1999722766 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v101) v1 ==== 92+0+10 (secure 0 0 0) 0x7f33ac0ae280 con 0x7f33ac160c50 2023-10-11T09:49:48.440 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:49:48.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.438+0000 7f33b2d58700 1 -- 172.21.15.78:0/1999722766 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3390065f00 msgr2=0x7f33900683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:48.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.438+0000 7f33b2d58700 1 --2- 172.21.15.78:0/1999722766 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3390065f00 0x7f33900683b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f33a80011e0 tx=0x7f33a8009040 comp rx=0 tx=0).stop 2023-10-11T09:49:48.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.438+0000 7f33b2d58700 1 -- 172.21.15.78:0/1999722766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33ac160c50 msgr2=0x7f33ac0c6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:48.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.438+0000 7f33b2d58700 1 --2- 172.21.15.78:0/1999722766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33ac160c50 0x7f33ac0c6400 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f339c00c700 tx=0x7f339c00ca10 comp rx=0 tx=0).stop 2023-10-11T09:49:48.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.438+0000 7f33b2d58700 1 -- 172.21.15.78:0/1999722766 shutdown_connections 2023-10-11T09:49:48.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.438+0000 7f33b2d58700 1 --2- 172.21.15.78:0/1999722766 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3390065f00 0x7f33900683b0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:48.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.438+0000 7f33b2d58700 1 --2- 172.21.15.78:0/1999722766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33ac0b37c0 0x7f33ac0b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:48.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.438+0000 7f33b2d58700 1 --2- 172.21.15.78:0/1999722766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33ac160c50 0x7f33ac0c6400 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:48.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.438+0000 7f33b2d58700 1 --2- 172.21.15.78:0/1999722766 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f33ac0caf90 0x7f33ac150e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:48.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.438+0000 7f33b2d58700 1 -- 172.21.15.78:0/1999722766 >> 172.21.15.78:0/1999722766 conn(0x7f33ac0cc070 msgr2=0x7f33ac06c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:48.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.438+0000 7f33b2d58700 1 -- 172.21.15.78:0/1999722766 shutdown_connections 2023-10-11T09:49:48.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.438+0000 7f33b2d58700 1 -- 172.21.15.78:0/1999722766 wait complete. 2023-10-11T09:49:48.465 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-11T09:49:48.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.626+0000 7fb33edeb700 1 Processor -- start 2023-10-11T09:49:48.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.630+0000 7fb33edeb700 1 -- start start 2023-10-11T09:49:48.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.630+0000 7fb33edeb700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3380b2310 0x7fb338161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:48.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.630+0000 7fb33edeb700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb3380b4b60 0x7fb3380d1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:48.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.630+0000 7fb33edeb700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fb3380d21e0 0x7fb3380b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:48.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.630+0000 7fb33edeb700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3380d11e0 con 0x7fb3380d21e0 2023-10-11T09:49:48.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.630+0000 7fb33edeb700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3380b5370 con 0x7fb3380b4b60 2023-10-11T09:49:48.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.630+0000 7fb33edeb700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3380d14a0 con 0x7fb3380b2310 2023-10-11T09:49:48.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.630+0000 7fb33d388700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fb3380d21e0 0x7fb3380b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:48.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.630+0000 7fb33d388700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fb3380d21e0 0x7fb3380b5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41358/0 (socket says 172.21.15.78:41358) 2023-10-11T09:49:48.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.630+0000 7fb33d388700 1 -- 172.21.15.78:0/3204189512 learned_addr learned my addr 172.21.15.78:0/3204189512 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:48.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.634+0000 7fb33cb87700 1 --2- 172.21.15.78:0/3204189512 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3380b2310 0x7fb338161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:48.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.634+0000 7fb32ffff700 1 --2- 172.21.15.78:0/3204189512 >> v2:172.21.15.133:3300/0 conn(0x7fb3380b4b60 0x7fb3380d1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:48.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.634+0000 7fb33d388700 1 -- 172.21.15.78:0/3204189512 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3380b2310 msgr2=0x7fb338161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:48.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.634+0000 7fb33d388700 1 --2- 172.21.15.78:0/3204189512 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3380b2310 0x7fb338161150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:48.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.634+0000 7fb33d388700 1 -- 172.21.15.78:0/3204189512 >> v2:172.21.15.133:3300/0 conn(0x7fb3380b4b60 msgr2=0x7fb3380d1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:48.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.634+0000 7fb33d388700 1 --2- 172.21.15.78:0/3204189512 >> v2:172.21.15.133:3300/0 conn(0x7fb3380b4b60 0x7fb3380d1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:48.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.634+0000 7fb33d388700 1 -- 172.21.15.78:0/3204189512 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3380d1780 con 0x7fb3380d21e0 2023-10-11T09:49:48.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.634+0000 7fb32ffff700 1 --2- 172.21.15.78:0/3204189512 >> v2:172.21.15.133:3300/0 conn(0x7fb3380b4b60 0x7fb3380d1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:49:48.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.634+0000 7fb33d388700 1 --2- 172.21.15.78:0/3204189512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3380d21e0 0x7fb3380b5f30 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fb33808be40 tx=0x7fb33000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85bd20c17070d129 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:48.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.634+0000 7fb32f7fe700 1 -- 172.21.15.78:0/3204189512 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33000e070 con 0x7fb3380d21e0 2023-10-11T09:49:48.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.634+0000 7fb32f7fe700 1 -- 172.21.15.78:0/3204189512 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33000b910 con 0x7fb3380d21e0 2023-10-11T09:49:48.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.634+0000 7fb32f7fe700 1 -- 172.21.15.78:0/3204189512 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33000bcc0 con 0x7fb3380d21e0 2023-10-11T09:49:48.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.634+0000 7fb33edeb700 1 -- 172.21.15.78:0/3204189512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3380d21e0 msgr2=0x7fb3380b5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:48.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.634+0000 7fb33edeb700 1 --2- 172.21.15.78:0/3204189512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3380d21e0 0x7fb3380b5f30 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fb33808be40 tx=0x7fb33000b330 comp rx=0 tx=0).stop 2023-10-11T09:49:48.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.634+0000 7fb33edeb700 1 -- 172.21.15.78:0/3204189512 shutdown_connections 2023-10-11T09:49:48.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.634+0000 7fb33edeb700 1 --2- 172.21.15.78:0/3204189512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3380d21e0 0x7fb3380b5f30 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:48.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.634+0000 7fb33edeb700 1 --2- 172.21.15.78:0/3204189512 >> v2:172.21.15.133:3300/0 conn(0x7fb3380b4b60 0x7fb3380d1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:48.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.634+0000 7fb33edeb700 1 --2- 172.21.15.78:0/3204189512 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3380b2310 0x7fb338161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:48.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.634+0000 7fb33edeb700 1 -- 172.21.15.78:0/3204189512 >> 172.21.15.78:0/3204189512 conn(0x7fb3380cfb60 msgr2=0x7fb3380cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:48.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.634+0000 7fb33edeb700 1 -- 172.21.15.78:0/3204189512 shutdown_connections 2023-10-11T09:49:48.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.634+0000 7fb33edeb700 1 -- 172.21.15.78:0/3204189512 wait complete. 2023-10-11T09:49:48.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.638+0000 7fb33edeb700 1 Processor -- start 2023-10-11T09:49:48.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.638+0000 7fb33edeb700 1 -- start start 2023-10-11T09:49:48.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.638+0000 7fb33edeb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3380b2310 0x7fb3380ca880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:48.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.638+0000 7fb33edeb700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3380b4b60 0x7fb33815e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:48.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.638+0000 7fb33edeb700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3380d21e0 0x7fb33815e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:48.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.638+0000 7fb33edeb700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb338149b60 con 0x7fb3380b2310 2023-10-11T09:49:48.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.638+0000 7fb33edeb700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb338148f20 con 0x7fb3380d21e0 2023-10-11T09:49:48.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.638+0000 7fb33edeb700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3380cadc0 con 0x7fb3380b4b60 2023-10-11T09:49:48.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.642+0000 7fb33cb87700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3380b2310 0x7fb3380ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:48.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.642+0000 7fb33cb87700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3380b2310 0x7fb3380ca880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41368/0 (socket says 172.21.15.78:41368) 2023-10-11T09:49:48.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.642+0000 7fb33cb87700 1 -- 172.21.15.78:0/3267272730 learned_addr learned my addr 172.21.15.78:0/3267272730 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:48.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.642+0000 7fb32ffff700 1 --2- 172.21.15.78:0/3267272730 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3380b4b60 0x7fb33815e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:48.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.642+0000 7fb33d388700 1 --2- 172.21.15.78:0/3267272730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3380d21e0 0x7fb33815e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:48.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.642+0000 7fb33cb87700 1 -- 172.21.15.78:0/3267272730 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3380b4b60 msgr2=0x7fb33815e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:48.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.642+0000 7fb33cb87700 1 --2- 172.21.15.78:0/3267272730 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3380b4b60 0x7fb33815e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:48.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.642+0000 7fb33cb87700 1 -- 172.21.15.78:0/3267272730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3380d21e0 msgr2=0x7fb33815e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:48.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.642+0000 7fb33cb87700 1 --2- 172.21.15.78:0/3267272730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3380d21e0 0x7fb33815e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:48.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.642+0000 7fb33cb87700 1 -- 172.21.15.78:0/3267272730 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb330009b30 con 0x7fb3380b2310 2023-10-11T09:49:48.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.642+0000 7fb33cb87700 1 --2- 172.21.15.78:0/3267272730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3380b2310 0x7fb3380ca880 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fb33400e960 tx=0x7fb33400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:48.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.642+0000 7fb32d7fa700 1 -- 172.21.15.78:0/3267272730 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3340096c0 con 0x7fb3380b2310 2023-10-11T09:49:48.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.642+0000 7fb32d7fa700 1 -- 172.21.15.78:0/3267272730 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb334013680 con 0x7fb3380b2310 2023-10-11T09:49:48.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.642+0000 7fb33edeb700 1 -- 172.21.15.78:0/3267272730 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3381491c0 con 0x7fb3380b2310 2023-10-11T09:49:48.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.642+0000 7fb32d7fa700 1 -- 172.21.15.78:0/3267272730 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb334013a10 con 0x7fb3380b2310 2023-10-11T09:49:48.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.642+0000 7fb33edeb700 1 -- 172.21.15.78:0/3267272730 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb338149780 con 0x7fb3380b2310 2023-10-11T09:49:48.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.642+0000 7fb33edeb700 1 -- 172.21.15.78:0/3267272730 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb33804e530 con 0x7fb3380b2310 2023-10-11T09:49:48.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.646+0000 7fb32d7fa700 1 -- 172.21.15.78:0/3267272730 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fb334013b90 con 0x7fb3380b2310 2023-10-11T09:49:48.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.646+0000 7fb32d7fa700 1 --2- 172.21.15.78:0/3267272730 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb324065ed0 0x7fb324068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:48.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.646+0000 7fb32d7fa700 1 -- 172.21.15.78:0/3267272730 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fb3340837a0 con 0x7fb3380b2310 2023-10-11T09:49:48.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.646+0000 7fb32ffff700 1 --2- 172.21.15.78:0/3267272730 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb324065ed0 0x7fb324068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:48.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.650+0000 7fb32ffff700 1 --2- 172.21.15.78:0/3267272730 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb324065ed0 0x7fb324068380 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb33806bd90 tx=0x7fb328008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:48.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.654+0000 7fb32d7fa700 1 -- 172.21.15.78:0/3267272730 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fb334051140 con 0x7fb3380b2310 2023-10-11T09:49:48.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.830+0000 7fb33edeb700 1 -- 172.21.15.78:0/3267272730 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb3380d29c0 con 0x7fb3380b2310 2023-10-11T09:49:48.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.830+0000 7fb32d7fa700 1 -- 172.21.15.78:0/3267272730 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v101) v1 ==== 84+0+126 (secure 0 0 0) 0x7fb334021090 con 0x7fb3380b2310 2023-10-11T09:49:48.837 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:49:48.837 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2023-10-11T09:49:48.838 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:49:48.838 INFO:teuthology.orchestra.run.smithi078.stdout: "poolnum": 1, 2023-10-11T09:49:48.838 INFO:teuthology.orchestra.run.smithi078.stdout: "poolname": ".mgr" 2023-10-11T09:49:48.838 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:49:48.839 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:49:48.839 INFO:teuthology.orchestra.run.smithi078.stdout: "poolnum": 2, 2023-10-11T09:49:48.839 INFO:teuthology.orchestra.run.smithi078.stdout: "poolname": "rbd" 2023-10-11T09:49:48.839 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:49:48.839 INFO:teuthology.orchestra.run.smithi078.stdout:] 2023-10-11T09:49:48.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.830+0000 7fb33edeb700 1 -- 172.21.15.78:0/3267272730 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb324065ed0 msgr2=0x7fb324068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:48.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.830+0000 7fb33edeb700 1 --2- 172.21.15.78:0/3267272730 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb324065ed0 0x7fb324068380 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb33806bd90 tx=0x7fb328008040 comp rx=0 tx=0).stop 2023-10-11T09:49:48.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.830+0000 7fb33edeb700 1 -- 172.21.15.78:0/3267272730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3380b2310 msgr2=0x7fb3380ca880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:48.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.830+0000 7fb33edeb700 1 --2- 172.21.15.78:0/3267272730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3380b2310 0x7fb3380ca880 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fb33400e960 tx=0x7fb33400ec70 comp rx=0 tx=0).stop 2023-10-11T09:49:48.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.834+0000 7fb33edeb700 1 -- 172.21.15.78:0/3267272730 shutdown_connections 2023-10-11T09:49:48.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.834+0000 7fb33edeb700 1 --2- 172.21.15.78:0/3267272730 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb324065ed0 0x7fb324068380 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:48.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.834+0000 7fb33edeb700 1 --2- 172.21.15.78:0/3267272730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3380d21e0 0x7fb33815e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:48.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.834+0000 7fb33edeb700 1 --2- 172.21.15.78:0/3267272730 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3380b4b60 0x7fb33815e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:48.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.834+0000 7fb33edeb700 1 --2- 172.21.15.78:0/3267272730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3380b2310 0x7fb3380ca880 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:48.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.834+0000 7fb33edeb700 1 -- 172.21.15.78:0/3267272730 >> 172.21.15.78:0/3267272730 conn(0x7fb3380cfb60 msgr2=0x7fb33806af50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:48.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.834+0000 7fb33edeb700 1 -- 172.21.15.78:0/3267272730 shutdown_connections 2023-10-11T09:49:48.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:48.834+0000 7fb33edeb700 1 -- 172.21.15.78:0/3267272730 wait complete. 2023-10-11T09:49:48.863 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-11T09:49:48.864 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-11T09:49:49.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.034+0000 7f5571b07700 1 Processor -- start 2023-10-11T09:49:49.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.034+0000 7f5571b07700 1 -- start start 2023-10-11T09:49:49.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.038+0000 7f5571b07700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f556c0cefc0 0x7f556c0b52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:49.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.038+0000 7f5571b07700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f556c15cbf0 0x7f556c15d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:49.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.038+0000 7f5571b07700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f556c0d2bd0 con 0x7f556c15a0f0 2023-10-11T09:49:49.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.038+0000 7f5571b07700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f556c15d5b0 con 0x7f556c0cefc0 2023-10-11T09:49:49.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.038+0000 7f5571b07700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f556c0cf430 con 0x7f556c15cbf0 2023-10-11T09:49:49.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.038+0000 7f556bfff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f556c0cefc0 0x7f556c0b52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:49.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.038+0000 7f556bfff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f556c0cefc0 0x7f556c0b52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41384/0 (socket says 172.21.15.78:41384) 2023-10-11T09:49:49.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.038+0000 7f556bfff700 1 -- 172.21.15.78:0/3012291714 learned_addr learned my addr 172.21.15.78:0/3012291714 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:49.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.038+0000 7f5571306700 1 --2- 172.21.15.78:0/3012291714 >> v2:172.21.15.133:3300/0 conn(0x7f556c15cbf0 0x7f556c15d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:49.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.038+0000 7f556bfff700 1 -- 172.21.15.78:0/3012291714 >> v2:172.21.15.133:3300/0 conn(0x7f556c15cbf0 msgr2=0x7f556c15d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:49.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.038+0000 7f556bfff700 1 --2- 172.21.15.78:0/3012291714 >> v2:172.21.15.133:3300/0 conn(0x7f556c15cbf0 0x7f556c15d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:49.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.038+0000 7f556bfff700 1 -- 172.21.15.78:0/3012291714 >> v1:172.21.15.78:6789/0 conn(0x7f556c15a0f0 legacy=0x7f556c1608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:49.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.038+0000 7f556bfff700 1 -- 172.21.15.78:0/3012291714 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f556c0cf630 con 0x7f556c0cefc0 2023-10-11T09:49:49.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.038+0000 7f556bfff700 1 --2- 172.21.15.78:0/3012291714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f556c0cefc0 0x7f556c0b52e0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f556c06bea0 tx=0x7f5558009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b449e368ecbd3d79 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:49.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.038+0000 7f556b7fe700 1 -- 172.21.15.78:0/3012291714 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555800b8a0 con 0x7f556c0cefc0 2023-10-11T09:49:49.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.038+0000 7f556b7fe700 1 -- 172.21.15.78:0/3012291714 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f555800ba00 con 0x7f556c0cefc0 2023-10-11T09:49:49.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.038+0000 7f556b7fe700 1 -- 172.21.15.78:0/3012291714 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555800bd90 con 0x7f556c0cefc0 2023-10-11T09:49:49.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.038+0000 7f5571b07700 1 -- 172.21.15.78:0/3012291714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f556c0cefc0 msgr2=0x7f556c0b52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:49.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.038+0000 7f5571b07700 1 --2- 172.21.15.78:0/3012291714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f556c0cefc0 0x7f556c0b52e0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f556c06bea0 tx=0x7f5558009bf0 comp rx=0 tx=0).stop 2023-10-11T09:49:49.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.038+0000 7f5571b07700 1 -- 172.21.15.78:0/3012291714 shutdown_connections 2023-10-11T09:49:49.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.038+0000 7f5571b07700 1 --2- 172.21.15.78:0/3012291714 >> v2:172.21.15.133:3300/0 conn(0x7f556c15cbf0 0x7f556c15d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:49.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.038+0000 7f5571b07700 1 --2- 172.21.15.78:0/3012291714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f556c0cefc0 0x7f556c0b52e0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:49.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.038+0000 7f5571b07700 1 -- 172.21.15.78:0/3012291714 >> 172.21.15.78:0/3012291714 conn(0x7f556c06afd0 msgr2=0x7f556c06b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:49.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.042+0000 7f5571b07700 1 -- 172.21.15.78:0/3012291714 shutdown_connections 2023-10-11T09:49:49.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.042+0000 7f5571b07700 1 -- 172.21.15.78:0/3012291714 wait complete. 2023-10-11T09:49:49.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.042+0000 7f5571b07700 1 Processor -- start 2023-10-11T09:49:49.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.042+0000 7f5571b07700 1 -- start start 2023-10-11T09:49:49.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.042+0000 7f5571b07700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f556c0cefc0 0x7f556c0c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:49.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.042+0000 7f5571b07700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f556c15a0f0 0x7f556c0b7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:49.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.042+0000 7f5571b07700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f556c15cbf0 0x7f556c14f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:49.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.042+0000 7f5571b07700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c0c8560 con 0x7f556c15a0f0 2023-10-11T09:49:49.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.042+0000 7f5571b07700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c0b7850 con 0x7f556c0cefc0 2023-10-11T09:49:49.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.042+0000 7f5571b07700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f556c0c9360 con 0x7f556c15cbf0 2023-10-11T09:49:49.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.042+0000 7f556bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f556c15a0f0 0x7f556c0b7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:49.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.046+0000 7f556bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f556c15a0f0 0x7f556c0b7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41392/0 (socket says 172.21.15.78:41392) 2023-10-11T09:49:49.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.046+0000 7f556bfff700 1 -- 172.21.15.78:0/1746251303 learned_addr learned my addr 172.21.15.78:0/1746251303 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:49.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.046+0000 7f5571306700 1 --2- 172.21.15.78:0/1746251303 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f556c15cbf0 0x7f556c14f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:49.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.046+0000 7f5570b05700 1 --2- 172.21.15.78:0/1746251303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f556c0cefc0 0x7f556c0c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:49.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.046+0000 7f556bfff700 1 -- 172.21.15.78:0/1746251303 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f556c15cbf0 msgr2=0x7f556c14f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:49.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.046+0000 7f556bfff700 1 --2- 172.21.15.78:0/1746251303 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f556c15cbf0 0x7f556c14f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:49.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.046+0000 7f556bfff700 1 -- 172.21.15.78:0/1746251303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f556c0cefc0 msgr2=0x7f556c0c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:49.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.046+0000 7f556bfff700 1 --2- 172.21.15.78:0/1746251303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f556c0cefc0 0x7f556c0c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:49.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.046+0000 7f556bfff700 1 -- 172.21.15.78:0/1746251303 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5558009570 con 0x7f556c15a0f0 2023-10-11T09:49:49.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.046+0000 7f556bfff700 1 --2- 172.21.15.78:0/1746251303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f556c15a0f0 0x7f556c0b7310 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f555800b0e0 tx=0x7f5558004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:49.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.046+0000 7f55697fa700 1 -- 172.21.15.78:0/1746251303 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55580036c0 con 0x7f556c15a0f0 2023-10-11T09:49:49.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.046+0000 7f55697fa700 1 -- 172.21.15.78:0/1746251303 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f555801f070 con 0x7f556c15a0f0 2023-10-11T09:49:49.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.046+0000 7f55697fa700 1 -- 172.21.15.78:0/1746251303 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5558003a60 con 0x7f556c15a0f0 2023-10-11T09:49:49.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.046+0000 7f5571b07700 1 -- 172.21.15.78:0/1746251303 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f556c0c97f0 con 0x7f556c15a0f0 2023-10-11T09:49:49.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.046+0000 7f5571b07700 1 -- 172.21.15.78:0/1746251303 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f556c0c8a30 con 0x7f556c15a0f0 2023-10-11T09:49:49.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.046+0000 7f5571b07700 1 -- 172.21.15.78:0/1746251303 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f556c08bb90 con 0x7f556c15a0f0 2023-10-11T09:49:49.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.050+0000 7f55697fa700 1 -- 172.21.15.78:0/1746251303 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f5558021030 con 0x7f556c15a0f0 2023-10-11T09:49:49.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.050+0000 7f55697fa700 1 --2- 172.21.15.78:0/1746251303 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5550065ed0 0x7f5550068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:49.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.050+0000 7f55697fa700 1 -- 172.21.15.78:0/1746251303 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5558084ad0 con 0x7f556c15a0f0 2023-10-11T09:49:49.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.050+0000 7f5570b05700 1 --2- 172.21.15.78:0/1746251303 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5550065ed0 0x7f5550068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:49.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.050+0000 7f5570b05700 1 --2- 172.21.15.78:0/1746251303 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5550065ed0 0x7f5550068380 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f55600018a0 tx=0x7f5560009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:49.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.054+0000 7f55697fa700 1 -- 172.21.15.78:0/1746251303 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f5558052460 con 0x7f556c15a0f0 2023-10-11T09:49:49.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.226+0000 7f5571b07700 1 -- 172.21.15.78:0/1746251303 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f556c065db0 con 0x7f556c15a0f0 2023-10-11T09:49:49.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.470+0000 7f55697fa700 1 -- 172.21.15.78:0/1746251303 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v102) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5558055a80 con 0x7f556c15a0f0 2023-10-11T09:49:49.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.482+0000 7f5571b07700 1 -- 172.21.15.78:0/1746251303 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5550065ed0 msgr2=0x7f5550068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:49.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.482+0000 7f5571b07700 1 --2- 172.21.15.78:0/1746251303 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5550065ed0 0x7f5550068380 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f55600018a0 tx=0x7f5560009040 comp rx=0 tx=0).stop 2023-10-11T09:49:49.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.482+0000 7f5571b07700 1 -- 172.21.15.78:0/1746251303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f556c15a0f0 msgr2=0x7f556c0b7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:49.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.482+0000 7f5571b07700 1 --2- 172.21.15.78:0/1746251303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f556c15a0f0 0x7f556c0b7310 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f555800b0e0 tx=0x7f5558004770 comp rx=0 tx=0).stop 2023-10-11T09:49:49.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.482+0000 7f5571b07700 1 -- 172.21.15.78:0/1746251303 shutdown_connections 2023-10-11T09:49:49.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.482+0000 7f5571b07700 1 --2- 172.21.15.78:0/1746251303 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5550065ed0 0x7f5550068380 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:49.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.482+0000 7f5571b07700 1 --2- 172.21.15.78:0/1746251303 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f556c15cbf0 0x7f556c14f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:49.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.482+0000 7f5571b07700 1 --2- 172.21.15.78:0/1746251303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f556c15a0f0 0x7f556c0b7310 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:49.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.482+0000 7f5571b07700 1 --2- 172.21.15.78:0/1746251303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f556c0cefc0 0x7f556c0c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:49.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.482+0000 7f5571b07700 1 -- 172.21.15.78:0/1746251303 >> 172.21.15.78:0/1746251303 conn(0x7f556c06afd0 msgr2=0x7f556c15c250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:49.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.482+0000 7f5571b07700 1 -- 172.21.15.78:0/1746251303 shutdown_connections 2023-10-11T09:49:49.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.482+0000 7f5571b07700 1 -- 172.21.15.78:0/1746251303 wait complete. 2023-10-11T09:49:49.493 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' created 2023-10-11T09:49:49.511 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-11T09:49:49.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.686+0000 7f3871f30700 1 Processor -- start 2023-10-11T09:49:49.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.686+0000 7f3871f30700 1 -- start start 2023-10-11T09:49:49.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.690+0000 7f3871f30700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f386c0b4940 0x7f386c0cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:49.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.690+0000 7f3871f30700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f386c0b4fd0 con 0x7f386c0cf8d0 2023-10-11T09:49:49.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.690+0000 7f3871f30700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f386c0b52b0 con 0x7f386c160870 2023-10-11T09:49:49.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.690+0000 7f3871f30700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f386c0b5870 con 0x7f386c0b4940 2023-10-11T09:49:49.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.690+0000 7f3870f2e700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f386c0cf8d0 0x7f386c0cfca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:42238/0 (socket says 172.21.15.78:42238) 2023-10-11T09:49:49.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.690+0000 7f3870f2e700 1 -- 172.21.15.78:0/3989073050 learned_addr learned my addr 172.21.15.78:0/3989073050 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:49:49.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.690+0000 7f386bfff700 1 --2- 172.21.15.78:0/3989073050 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f386c0b4940 0x7f386c0cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:49.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.690+0000 7f386b7fe700 1 -- 172.21.15.78:0/3989073050 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3517047984 0 0) 0x7f386c0b4fd0 con 0x7f386c0cf8d0 2023-10-11T09:49:49.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.690+0000 7f386b7fe700 1 -- 172.21.15.78:0/3989073050 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38540035d0 con 0x7f386c0cf8d0 2023-10-11T09:49:49.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.690+0000 7f386b7fe700 1 -- 172.21.15.78:0/3989073050 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2156932539 0 0) 0x7f386c0b52b0 con 0x7f386c160870 2023-10-11T09:49:49.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.690+0000 7f386bfff700 1 -- 172.21.15.78:0/3989073050 >> v1:172.21.15.133:6789/0 conn(0x7f386c160870 legacy=0x7f386c15cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:49.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.690+0000 7f386bfff700 1 -- 172.21.15.78:0/3989073050 >> v1:172.21.15.78:6789/0 conn(0x7f386c0cf8d0 legacy=0x7f386c0cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:49.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.690+0000 7f386bfff700 1 -- 172.21.15.78:0/3989073050 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f386c0b5590 con 0x7f386c0b4940 2023-10-11T09:49:49.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.690+0000 7f386bfff700 1 --2- 172.21.15.78:0/3989073050 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f386c0b4940 0x7f386c0cd5f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f385c009880 tx=0x7f385c009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f35a925eb1187692 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:49.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.694+0000 7f386b7fe700 1 -- 172.21.15.78:0/3989073050 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c011830 con 0x7f386c0b4940 2023-10-11T09:49:49.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.694+0000 7f386b7fe700 1 -- 172.21.15.78:0/3989073050 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385c011990 con 0x7f386c0b4940 2023-10-11T09:49:49.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.694+0000 7f386b7fe700 1 -- 172.21.15.78:0/3989073050 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c011d20 con 0x7f386c0b4940 2023-10-11T09:49:49.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.694+0000 7f3871f30700 1 -- 172.21.15.78:0/3989073050 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f386c0b4940 msgr2=0x7f386c0cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:49.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.694+0000 7f3871f30700 1 --2- 172.21.15.78:0/3989073050 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f386c0b4940 0x7f386c0cd5f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f385c009880 tx=0x7f385c009b90 comp rx=0 tx=0).stop 2023-10-11T09:49:49.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.694+0000 7f3871f30700 1 -- 172.21.15.78:0/3989073050 shutdown_connections 2023-10-11T09:49:49.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.694+0000 7f3871f30700 1 --2- 172.21.15.78:0/3989073050 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f386c0b4940 0x7f386c0cd5f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:49.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.694+0000 7f3871f30700 1 -- 172.21.15.78:0/3989073050 >> 172.21.15.78:0/3989073050 conn(0x7f386c06b010 msgr2=0x7f386c06b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:49.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.694+0000 7f3871f30700 1 -- 172.21.15.78:0/3989073050 shutdown_connections 2023-10-11T09:49:49.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.694+0000 7f3871f30700 1 -- 172.21.15.78:0/3989073050 wait complete. 2023-10-11T09:49:49.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.694+0000 7f3871f30700 1 Processor -- start 2023-10-11T09:49:49.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.694+0000 7f3871f30700 1 -- start start 2023-10-11T09:49:49.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.698+0000 7f3871f30700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f386c0b4940 0x7f386c1556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:49.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.698+0000 7f3871f30700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f386c0cf8d0 0x7f386c0cb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:49.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.698+0000 7f3871f30700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f386c160870 0x7f386c150240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:49.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.698+0000 7f3871f30700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386c0cc240 con 0x7f386c0cf8d0 2023-10-11T09:49:49.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.698+0000 7f3871f30700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386c0cc520 con 0x7f386c160870 2023-10-11T09:49:49.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.698+0000 7f3871f30700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f386c0cc800 con 0x7f386c0b4940 2023-10-11T09:49:49.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.698+0000 7f386bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f386c0cf8d0 0x7f386c0cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:49.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.698+0000 7f386bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f386c0cf8d0 0x7f386c0cb740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41396/0 (socket says 172.21.15.78:41396) 2023-10-11T09:49:49.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.698+0000 7f386bfff700 1 -- 172.21.15.78:0/1869801264 learned_addr learned my addr 172.21.15.78:0/1869801264 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:49.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.698+0000 7f3870f2e700 1 --2- 172.21.15.78:0/1869801264 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f386c0b4940 0x7f386c1556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:49.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.698+0000 7f387172f700 1 --2- 172.21.15.78:0/1869801264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f386c160870 0x7f386c150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:49.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.698+0000 7f386bfff700 1 -- 172.21.15.78:0/1869801264 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f386c0b4940 msgr2=0x7f386c1556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:49.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.698+0000 7f386bfff700 1 --2- 172.21.15.78:0/1869801264 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f386c0b4940 0x7f386c1556e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:49.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.698+0000 7f386bfff700 1 -- 172.21.15.78:0/1869801264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f386c160870 msgr2=0x7f386c150240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:49.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.698+0000 7f386bfff700 1 --2- 172.21.15.78:0/1869801264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f386c160870 0x7f386c150240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:49.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.698+0000 7f386bfff700 1 -- 172.21.15.78:0/1869801264 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f385c0094b0 con 0x7f386c0cf8d0 2023-10-11T09:49:49.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.698+0000 7f387172f700 1 --2- 172.21.15.78:0/1869801264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f386c160870 0x7f386c150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:49:49.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.698+0000 7f386bfff700 1 --2- 172.21.15.78:0/1869801264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f386c0cf8d0 0x7f386c0cb740 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f385c013040 tx=0x7f385c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:49.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.698+0000 7f38697fa700 1 -- 172.21.15.78:0/1869801264 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c019a80 con 0x7f386c0cf8d0 2023-10-11T09:49:49.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.698+0000 7f38697fa700 1 -- 172.21.15.78:0/1869801264 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385c01e070 con 0x7f386c0cf8d0 2023-10-11T09:49:49.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.698+0000 7f3871f30700 1 -- 172.21.15.78:0/1869801264 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f386c0ccae0 con 0x7f386c0cf8d0 2023-10-11T09:49:49.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.698+0000 7f38697fa700 1 -- 172.21.15.78:0/1869801264 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c019e00 con 0x7f386c0cf8d0 2023-10-11T09:49:49.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.698+0000 7f3871f30700 1 -- 172.21.15.78:0/1869801264 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f386c1e5c10 con 0x7f386c0cf8d0 2023-10-11T09:49:49.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.702+0000 7f3871f30700 1 -- 172.21.15.78:0/1869801264 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f386c1f2020 con 0x7f386c0cf8d0 2023-10-11T09:49:49.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.702+0000 7f38697fa700 1 -- 172.21.15.78:0/1869801264 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f385c02d450 con 0x7f386c0cf8d0 2023-10-11T09:49:49.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.702+0000 7f38697fa700 1 --2- 172.21.15.78:0/1869801264 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3858065ed0 0x7f3858068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:49.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.706+0000 7f38697fa700 1 -- 172.21.15.78:0/1869801264 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f385c0846b0 con 0x7f386c0cf8d0 2023-10-11T09:49:49.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.706+0000 7f3870f2e700 1 --2- 172.21.15.78:0/1869801264 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3858065ed0 0x7f3858068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:49.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.706+0000 7f3870f2e700 1 --2- 172.21.15.78:0/1869801264 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3858065ed0 0x7f3858068380 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f3860005930 tx=0x7f3860009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:49.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.710+0000 7f38697fa700 1 -- 172.21.15.78:0/1869801264 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f385c051f80 con 0x7f386c0cf8d0 2023-10-11T09:49:49.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:49.890+0000 7f3871f30700 1 -- 172.21.15.78:0/1869801264 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f386c065d40 con 0x7f386c0cf8d0 2023-10-11T09:49:50.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.490+0000 7f38697fa700 1 -- 172.21.15.78:0/1869801264 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v103) v1 ==== 170+0+0 (secure 0 0 0) 0x7f385c0555a0 con 0x7f386c0cf8d0 2023-10-11T09:49:50.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.506+0000 7f3871f30700 1 -- 172.21.15.78:0/1869801264 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3858065ed0 msgr2=0x7f3858068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:50.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.506+0000 7f3871f30700 1 --2- 172.21.15.78:0/1869801264 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3858065ed0 0x7f3858068380 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f3860005930 tx=0x7f3860009040 comp rx=0 tx=0).stop 2023-10-11T09:49:50.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.506+0000 7f3871f30700 1 -- 172.21.15.78:0/1869801264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f386c0cf8d0 msgr2=0x7f386c0cb740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:50.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.506+0000 7f3871f30700 1 --2- 172.21.15.78:0/1869801264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f386c0cf8d0 0x7f386c0cb740 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f385c013040 tx=0x7f385c004770 comp rx=0 tx=0).stop 2023-10-11T09:49:50.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.510+0000 7f3871f30700 1 -- 172.21.15.78:0/1869801264 shutdown_connections 2023-10-11T09:49:50.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.510+0000 7f3871f30700 1 --2- 172.21.15.78:0/1869801264 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3858065ed0 0x7f3858068380 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:50.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.510+0000 7f3871f30700 1 --2- 172.21.15.78:0/1869801264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f386c160870 0x7f386c150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:50.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.510+0000 7f3871f30700 1 --2- 172.21.15.78:0/1869801264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f386c0cf8d0 0x7f386c0cb740 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:50.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.510+0000 7f3871f30700 1 --2- 172.21.15.78:0/1869801264 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f386c0b4940 0x7f386c1556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:50.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.510+0000 7f3871f30700 1 -- 172.21.15.78:0/1869801264 >> 172.21.15.78:0/1869801264 conn(0x7f386c06b010 msgr2=0x7f386c0cfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:50.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.514+0000 7f3871f30700 1 -- 172.21.15.78:0/1869801264 shutdown_connections 2023-10-11T09:49:50.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.514+0000 7f3871f30700 1 -- 172.21.15.78:0/1869801264 wait complete. 2023-10-11T09:49:50.531 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' created 2023-10-11T09:49:50.549 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-11T09:49:50.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.710+0000 7f0f023df700 1 Processor -- start 2023-10-11T09:49:50.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.710+0000 7f0f023df700 1 -- start start 2023-10-11T09:49:50.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.710+0000 7f0f023df700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0efc158430 0x7f0efc0710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:50.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.710+0000 7f0f023df700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0efc06d340 0x7f0efc0ca880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:50.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.714+0000 7f0f023df700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0efc0cadc0 0x7f0efc15afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:50.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.714+0000 7f0f023df700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0efc070c10 con 0x7f0efc0cadc0 2023-10-11T09:49:50.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.714+0000 7f0f023df700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0efc0cb240 con 0x7f0efc158430 2023-10-11T09:49:50.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.714+0000 7f0f023df700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc071610 con 0x7f0efc06d340 2023-10-11T09:49:50.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.714+0000 7f0f0097c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0efc0cadc0 0x7f0efc15afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:50.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.714+0000 7f0f0097c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0efc0cadc0 0x7f0efc15afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41398/0 (socket says 172.21.15.78:41398) 2023-10-11T09:49:50.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.714+0000 7f0f0097c700 1 -- 172.21.15.78:0/409922074 learned_addr learned my addr 172.21.15.78:0/409922074 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:50.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.714+0000 7f0efbfff700 1 --2- 172.21.15.78:0/409922074 >> v2:172.21.15.133:3300/0 conn(0x7f0efc158430 0x7f0efc0710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:50.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.714+0000 7f0ef3fff700 1 --2- 172.21.15.78:0/409922074 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0efc06d340 0x7f0efc0ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:50.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.714+0000 7f0f0097c700 1 -- 172.21.15.78:0/409922074 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0efc06d340 msgr2=0x7f0efc0ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:50.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.714+0000 7f0f0097c700 1 --2- 172.21.15.78:0/409922074 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0efc06d340 0x7f0efc0ca880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:50.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.714+0000 7f0f0097c700 1 -- 172.21.15.78:0/409922074 >> v2:172.21.15.133:3300/0 conn(0x7f0efc158430 msgr2=0x7f0efc0710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:50.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.714+0000 7f0f0097c700 1 --2- 172.21.15.78:0/409922074 >> v2:172.21.15.133:3300/0 conn(0x7f0efc158430 0x7f0efc0710d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:50.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.714+0000 7f0f0097c700 1 -- 172.21.15.78:0/409922074 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0efc15e280 con 0x7f0efc0cadc0 2023-10-11T09:49:50.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.714+0000 7f0f0097c700 1 --2- 172.21.15.78:0/409922074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0efc0cadc0 0x7f0efc15afe0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f0efc06fd00 tx=0x7f0ef400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f84a1849b1b2c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:50.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.714+0000 7f0efb7fe700 1 -- 172.21.15.78:0/409922074 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef400e070 con 0x7f0efc0cadc0 2023-10-11T09:49:50.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.714+0000 7f0efb7fe700 1 -- 172.21.15.78:0/409922074 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ef400b910 con 0x7f0efc0cadc0 2023-10-11T09:49:50.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.714+0000 7f0efb7fe700 1 -- 172.21.15.78:0/409922074 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef400bcc0 con 0x7f0efc0cadc0 2023-10-11T09:49:50.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.718+0000 7f0f023df700 1 -- 172.21.15.78:0/409922074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0efc0cadc0 msgr2=0x7f0efc15afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:50.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.718+0000 7f0f023df700 1 --2- 172.21.15.78:0/409922074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0efc0cadc0 0x7f0efc15afe0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f0efc06fd00 tx=0x7f0ef400b330 comp rx=0 tx=0).stop 2023-10-11T09:49:50.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.718+0000 7f0f023df700 1 -- 172.21.15.78:0/409922074 shutdown_connections 2023-10-11T09:49:50.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.718+0000 7f0f023df700 1 --2- 172.21.15.78:0/409922074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0efc0cadc0 0x7f0efc15afe0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:50.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.718+0000 7f0f023df700 1 --2- 172.21.15.78:0/409922074 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0efc06d340 0x7f0efc0ca880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:50.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.718+0000 7f0f023df700 1 --2- 172.21.15.78:0/409922074 >> v2:172.21.15.133:3300/0 conn(0x7f0efc158430 0x7f0efc0710d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:50.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.718+0000 7f0f023df700 1 -- 172.21.15.78:0/409922074 >> 172.21.15.78:0/409922074 conn(0x7f0efc0cecc0 msgr2=0x7f0efc0cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:50.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.718+0000 7f0f023df700 1 -- 172.21.15.78:0/409922074 shutdown_connections 2023-10-11T09:49:50.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.718+0000 7f0f023df700 1 -- 172.21.15.78:0/409922074 wait complete. 2023-10-11T09:49:50.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.718+0000 7f0f023df700 1 Processor -- start 2023-10-11T09:49:50.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.718+0000 7f0f023df700 1 -- start start 2023-10-11T09:49:50.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.718+0000 7f0f023df700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0efc06d340 0x7f0efc143600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:50.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.718+0000 7f0f023df700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0efc0cadc0 0x7f0efc155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:50.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.718+0000 7f0f023df700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0efc158430 0x7f0efc1558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:50.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.718+0000 7f0f023df700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc15d110 con 0x7f0efc0cadc0 2023-10-11T09:49:50.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.718+0000 7f0f023df700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc15c070 con 0x7f0efc158430 2023-10-11T09:49:50.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.718+0000 7f0f023df700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc15c350 con 0x7f0efc06d340 2023-10-11T09:49:50.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.722+0000 7f0ef3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0efc0cadc0 0x7f0efc155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:50.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.722+0000 7f0ef3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0efc0cadc0 0x7f0efc155390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41404/0 (socket says 172.21.15.78:41404) 2023-10-11T09:49:50.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.722+0000 7f0ef3fff700 1 -- 172.21.15.78:0/3243535680 learned_addr learned my addr 172.21.15.78:0/3243535680 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:50.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.722+0000 7f0efbfff700 1 --2- 172.21.15.78:0/3243535680 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0efc06d340 0x7f0efc143600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:50.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.722+0000 7f0f0097c700 1 --2- 172.21.15.78:0/3243535680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0efc158430 0x7f0efc1558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:50.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.722+0000 7f0ef3fff700 1 -- 172.21.15.78:0/3243535680 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0efc06d340 msgr2=0x7f0efc143600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:50.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.722+0000 7f0ef3fff700 1 --2- 172.21.15.78:0/3243535680 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0efc06d340 0x7f0efc143600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:50.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.722+0000 7f0ef3fff700 1 -- 172.21.15.78:0/3243535680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0efc158430 msgr2=0x7f0efc1558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:50.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.722+0000 7f0ef3fff700 1 --2- 172.21.15.78:0/3243535680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0efc158430 0x7f0efc1558d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:50.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.722+0000 7f0ef3fff700 1 -- 172.21.15.78:0/3243535680 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef4009b30 con 0x7f0efc0cadc0 2023-10-11T09:49:50.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.722+0000 7f0efbfff700 1 --2- 172.21.15.78:0/3243535680 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0efc06d340 0x7f0efc143600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:49:50.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.722+0000 7f0ef3fff700 1 --2- 172.21.15.78:0/3243535680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0efc0cadc0 0x7f0efc155390 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f0ee400e870 tx=0x7f0ee400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:50.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.722+0000 7f0ef97fa700 1 -- 172.21.15.78:0/3243535680 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4009540 con 0x7f0efc0cadc0 2023-10-11T09:49:50.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.722+0000 7f0ef97fa700 1 -- 172.21.15.78:0/3243535680 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ee400c640 con 0x7f0efc0cadc0 2023-10-11T09:49:50.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.722+0000 7f0f023df700 1 -- 172.21.15.78:0/3243535680 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0efc15c630 con 0x7f0efc0cadc0 2023-10-11T09:49:50.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.722+0000 7f0ef97fa700 1 -- 172.21.15.78:0/3243535680 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee400c9b0 con 0x7f0efc0cadc0 2023-10-11T09:49:50.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.722+0000 7f0f023df700 1 -- 172.21.15.78:0/3243535680 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0efc15cbf0 con 0x7f0efc0cadc0 2023-10-11T09:49:50.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.722+0000 7f0f023df700 1 -- 172.21.15.78:0/3243535680 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0efc06eb80 con 0x7f0efc0cadc0 2023-10-11T09:49:50.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.726+0000 7f0ef97fa700 1 -- 172.21.15.78:0/3243535680 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f0ee400cb10 con 0x7f0efc0cadc0 2023-10-11T09:49:50.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.726+0000 7f0ef97fa700 1 --2- 172.21.15.78:0/3243535680 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0edc065ed0 0x7f0edc068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:50.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.726+0000 7f0ef97fa700 1 -- 172.21.15.78:0/3243535680 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f0ee4083830 con 0x7f0efc0cadc0 2023-10-11T09:49:50.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.726+0000 7f0efbfff700 1 --2- 172.21.15.78:0/3243535680 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0edc065ed0 0x7f0edc068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:50.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.730+0000 7f0efbfff700 1 --2- 172.21.15.78:0/3243535680 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0edc065ed0 0x7f0edc068380 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f0eec009c40 tx=0x7f0eec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:50.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.734+0000 7f0ef97fa700 1 -- 172.21.15.78:0/3243535680 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f0ee4050e40 con 0x7f0efc0cadc0 2023-10-11T09:49:50.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.950+0000 7f0f023df700 1 -- 172.21.15.78:0/3243535680 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f0efc065db0 con 0x7f0efc0cadc0 2023-10-11T09:49:50.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.970+0000 7f0ef97fa700 1 -- 172.21.15.78:0/3243535680 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v75) v1 ==== 174+0+0 (secure 0 0 0) 0x7f0ee4054460 con 0x7f0efc0cadc0 2023-10-11T09:49:50.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.978+0000 7f0f023df700 1 -- 172.21.15.78:0/3243535680 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0edc065ed0 msgr2=0x7f0edc068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:50.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.978+0000 7f0f023df700 1 --2- 172.21.15.78:0/3243535680 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0edc065ed0 0x7f0edc068380 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f0eec009c40 tx=0x7f0eec008040 comp rx=0 tx=0).stop 2023-10-11T09:49:50.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.978+0000 7f0f023df700 1 -- 172.21.15.78:0/3243535680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0efc0cadc0 msgr2=0x7f0efc155390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:50.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.978+0000 7f0f023df700 1 --2- 172.21.15.78:0/3243535680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0efc0cadc0 0x7f0efc155390 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f0ee400e870 tx=0x7f0ee400ec30 comp rx=0 tx=0).stop 2023-10-11T09:49:50.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.982+0000 7f0f023df700 1 -- 172.21.15.78:0/3243535680 shutdown_connections 2023-10-11T09:49:50.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.982+0000 7f0f023df700 1 --2- 172.21.15.78:0/3243535680 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0edc065ed0 0x7f0edc068380 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:50.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.982+0000 7f0f023df700 1 --2- 172.21.15.78:0/3243535680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0efc158430 0x7f0efc1558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:50.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.982+0000 7f0f023df700 1 --2- 172.21.15.78:0/3243535680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0efc0cadc0 0x7f0efc155390 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:50.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.982+0000 7f0f023df700 1 --2- 172.21.15.78:0/3243535680 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0efc06d340 0x7f0efc143600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:50.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.982+0000 7f0f023df700 1 -- 172.21.15.78:0/3243535680 >> 172.21.15.78:0/3243535680 conn(0x7f0efc0cecc0 msgr2=0x7f0efc158ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:50.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.986+0000 7f0f023df700 1 -- 172.21.15.78:0/3243535680 shutdown_connections 2023-10-11T09:49:50.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:50.986+0000 7f0f023df700 1 -- 172.21.15.78:0/3243535680 wait complete. 2023-10-11T09:49:51.007 INFO:teuthology.orchestra.run.smithi078.stderr:new fs with metadata pool 14 and data pool 15 2023-10-11T09:49:51.024 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:49:51.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.186+0000 7f8de31dc700 1 Processor -- start 2023-10-11T09:49:51.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.190+0000 7f8de31dc700 1 -- start start 2023-10-11T09:49:51.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.190+0000 7f8de31dc700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f8ddc15d050 0x7f8ddc0cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:51.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.190+0000 7f8de31dc700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8ddc160c60 0x7f8ddc15d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:51.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.190+0000 7f8de31dc700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8ddc15d960 0x7f8ddc0b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:51.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.190+0000 7f8de31dc700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ddc160210 con 0x7f8ddc15d050 2023-10-11T09:49:51.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.190+0000 7f8de31dc700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ddc0ccd60 con 0x7f8ddc160c60 2023-10-11T09:49:51.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.190+0000 7f8de31dc700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc0ccfc0 con 0x7f8ddc15d960 2023-10-11T09:49:51.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.190+0000 7f8de21da700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f8ddc15d050 0x7f8ddc0cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:51.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.190+0000 7f8de21da700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f8ddc15d050 0x7f8ddc0cc820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41418/0 (socket says 172.21.15.78:41418) 2023-10-11T09:49:51.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.190+0000 7f8de21da700 1 -- 172.21.15.78:0/789737503 learned_addr learned my addr 172.21.15.78:0/789737503 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:51.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.190+0000 7f8de19d9700 1 --2- 172.21.15.78:0/789737503 >> v2:172.21.15.133:3300/0 conn(0x7f8ddc160c60 0x7f8ddc15d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:51.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.190+0000 7f8de29db700 1 --2- 172.21.15.78:0/789737503 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8ddc15d960 0x7f8ddc0b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:51.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.190+0000 7f8de21da700 1 -- 172.21.15.78:0/789737503 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8ddc15d960 msgr2=0x7f8ddc0b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:51.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.190+0000 7f8de21da700 1 --2- 172.21.15.78:0/789737503 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8ddc15d960 0x7f8ddc0b06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:51.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.190+0000 7f8de21da700 1 -- 172.21.15.78:0/789737503 >> v2:172.21.15.133:3300/0 conn(0x7f8ddc160c60 msgr2=0x7f8ddc15d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:51.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.190+0000 7f8de21da700 1 --2- 172.21.15.78:0/789737503 >> v2:172.21.15.133:3300/0 conn(0x7f8ddc160c60 0x7f8ddc15d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:51.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.190+0000 7f8de21da700 1 -- 172.21.15.78:0/789737503 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ddc0cd2a0 con 0x7f8ddc15d050 2023-10-11T09:49:51.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.190+0000 7f8de19d9700 1 --2- 172.21.15.78:0/789737503 >> v2:172.21.15.133:3300/0 conn(0x7f8ddc160c60 0x7f8ddc15d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:49:51.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.190+0000 7f8de29db700 1 --2- 172.21.15.78:0/789737503 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8ddc15d960 0x7f8ddc0b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:49:51.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.190+0000 7f8de21da700 1 --2- 172.21.15.78:0/789737503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ddc15d050 0x7f8ddc0cc820 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc06e260 tx=0x7f8dd8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a813e832c5b747b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:51.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.190+0000 7f8de11d8700 1 -- 172.21.15.78:0/789737503 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd800b8a0 con 0x7f8ddc15d050 2023-10-11T09:49:51.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.194+0000 7f8de11d8700 1 -- 172.21.15.78:0/789737503 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dd800ba00 con 0x7f8ddc15d050 2023-10-11T09:49:51.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.194+0000 7f8de11d8700 1 -- 172.21.15.78:0/789737503 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd800bd90 con 0x7f8ddc15d050 2023-10-11T09:49:51.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.194+0000 7f8de31dc700 1 -- 172.21.15.78:0/789737503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ddc15d050 msgr2=0x7f8ddc0cc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:51.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.194+0000 7f8de31dc700 1 --2- 172.21.15.78:0/789737503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ddc15d050 0x7f8ddc0cc820 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc06e260 tx=0x7f8dd8009bf0 comp rx=0 tx=0).stop 2023-10-11T09:49:51.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.194+0000 7f8de31dc700 1 -- 172.21.15.78:0/789737503 shutdown_connections 2023-10-11T09:49:51.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.194+0000 7f8de31dc700 1 --2- 172.21.15.78:0/789737503 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8ddc15d960 0x7f8ddc0b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:51.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.194+0000 7f8de31dc700 1 --2- 172.21.15.78:0/789737503 >> v2:172.21.15.133:3300/0 conn(0x7f8ddc160c60 0x7f8ddc15d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:51.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.194+0000 7f8de31dc700 1 --2- 172.21.15.78:0/789737503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ddc15d050 0x7f8ddc0cc820 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:51.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.194+0000 7f8de31dc700 1 -- 172.21.15.78:0/789737503 >> 172.21.15.78:0/789737503 conn(0x7f8ddc0cea60 msgr2=0x7f8ddc0cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:51.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.194+0000 7f8de31dc700 1 -- 172.21.15.78:0/789737503 shutdown_connections 2023-10-11T09:49:51.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.194+0000 7f8de31dc700 1 -- 172.21.15.78:0/789737503 wait complete. 2023-10-11T09:49:51.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.194+0000 7f8de31dc700 1 Processor -- start 2023-10-11T09:49:51.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.198+0000 7f8de31dc700 1 -- start start 2023-10-11T09:49:51.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.198+0000 7f8de31dc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ddc15d050 0x7f8ddc14bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:51.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.198+0000 7f8de31dc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ddc15d960 0x7f8ddc0702d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:51.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.198+0000 7f8de31dc700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8ddc160c60 0x7f8ddc070810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:51.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.198+0000 7f8de31dc700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc150580 con 0x7f8ddc15d050 2023-10-11T09:49:51.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.198+0000 7f8de31dc700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc154540 con 0x7f8ddc15d960 2023-10-11T09:49:51.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.198+0000 7f8de31dc700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc154820 con 0x7f8ddc160c60 2023-10-11T09:49:51.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.198+0000 7f8de21da700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ddc15d050 0x7f8ddc14bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:51.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.198+0000 7f8de21da700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ddc15d050 0x7f8ddc14bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41434/0 (socket says 172.21.15.78:41434) 2023-10-11T09:49:51.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.198+0000 7f8de21da700 1 -- 172.21.15.78:0/1197203409 learned_addr learned my addr 172.21.15.78:0/1197203409 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:51.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.198+0000 7f8de19d9700 1 --2- 172.21.15.78:0/1197203409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ddc15d960 0x7f8ddc0702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:51.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.198+0000 7f8de29db700 1 --2- 172.21.15.78:0/1197203409 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8ddc160c60 0x7f8ddc070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:51.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.198+0000 7f8de21da700 1 -- 172.21.15.78:0/1197203409 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8ddc160c60 msgr2=0x7f8ddc070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:51.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.198+0000 7f8de21da700 1 --2- 172.21.15.78:0/1197203409 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8ddc160c60 0x7f8ddc070810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:51.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.198+0000 7f8de21da700 1 -- 172.21.15.78:0/1197203409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ddc15d960 msgr2=0x7f8ddc0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:51.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.198+0000 7f8de21da700 1 --2- 172.21.15.78:0/1197203409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ddc15d960 0x7f8ddc0702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:51.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.198+0000 7f8de21da700 1 -- 172.21.15.78:0/1197203409 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd8009570 con 0x7f8ddc15d050 2023-10-11T09:49:51.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.198+0000 7f8de21da700 1 --2- 172.21.15.78:0/1197203409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ddc15d050 0x7f8ddc14bc70 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8000c00 tx=0x7f8dd8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:51.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.198+0000 7f8dd2ffd700 1 -- 172.21.15.78:0/1197203409 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd80038c0 con 0x7f8ddc15d050 2023-10-11T09:49:51.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.198+0000 7f8dd2ffd700 1 -- 172.21.15.78:0/1197203409 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dd801e070 con 0x7f8ddc15d050 2023-10-11T09:49:51.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.198+0000 7f8de31dc700 1 -- 172.21.15.78:0/1197203409 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ddc1507d0 con 0x7f8ddc15d050 2023-10-11T09:49:51.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.198+0000 7f8dd2ffd700 1 -- 172.21.15.78:0/1197203409 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8003c80 con 0x7f8ddc15d050 2023-10-11T09:49:51.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.198+0000 7f8de31dc700 1 -- 172.21.15.78:0/1197203409 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ddc14c1b0 con 0x7f8ddc15d050 2023-10-11T09:49:51.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.202+0000 7f8de31dc700 1 -- 172.21.15.78:0/1197203409 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ddc065d40 con 0x7f8ddc15d050 2023-10-11T09:49:51.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.202+0000 7f8dd2ffd700 1 -- 172.21.15.78:0/1197203409 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f8dd8011420 con 0x7f8ddc15d050 2023-10-11T09:49:51.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.202+0000 7f8dd2ffd700 1 --2- 172.21.15.78:0/1197203409 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8dc0065f00 0x7f8dc00683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:51.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.202+0000 7f8dd2ffd700 1 -- 172.21.15.78:0/1197203409 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8dd8020070 con 0x7f8ddc15d050 2023-10-11T09:49:51.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.206+0000 7f8de19d9700 1 --2- 172.21.15.78:0/1197203409 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8dc0065f00 0x7f8dc00683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:51.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.206+0000 7f8de19d9700 1 --2- 172.21.15.78:0/1197203409 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8dc0065f00 0x7f8dc00683b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc15cf60 tx=0x7f8dcc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:51.218 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.210+0000 7f8dd2ffd700 1 -- 172.21.15.78:0/1197203409 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f8dd80525f0 con 0x7f8ddc15d050 2023-10-11T09:49:51.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.386+0000 7f8de31dc700 1 -- 172.21.15.78:0/1197203409 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ddc0b1310 con 0x7f8ddc15d050 2023-10-11T09:49:51.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.390+0000 7f8dd2ffd700 1 -- 172.21.15.78:0/1197203409 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+17846 (secure 0 0 0) 0x7f8dd8055c10 con 0x7f8ddc15d050 2023-10-11T09:49:51.398 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:49:51.398 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":104,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:49:50.955036+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:49:49.297597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-11T09:49:49.955104+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:49:51.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.394+0000 7f8de31dc700 1 -- 172.21.15.78:0/1197203409 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8dc0065f00 msgr2=0x7f8dc00683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:51.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.394+0000 7f8de31dc700 1 --2- 172.21.15.78:0/1197203409 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8dc0065f00 0x7f8dc00683b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc15cf60 tx=0x7f8dcc008040 comp rx=0 tx=0).stop 2023-10-11T09:49:51.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.394+0000 7f8de31dc700 1 -- 172.21.15.78:0/1197203409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ddc15d050 msgr2=0x7f8ddc14bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:51.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.394+0000 7f8de31dc700 1 --2- 172.21.15.78:0/1197203409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ddc15d050 0x7f8ddc14bc70 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8000c00 tx=0x7f8dd8004770 comp rx=0 tx=0).stop 2023-10-11T09:49:51.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.394+0000 7f8de31dc700 1 -- 172.21.15.78:0/1197203409 shutdown_connections 2023-10-11T09:49:51.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.394+0000 7f8de31dc700 1 --2- 172.21.15.78:0/1197203409 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8dc0065f00 0x7f8dc00683b0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:51.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.394+0000 7f8de31dc700 1 --2- 172.21.15.78:0/1197203409 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8ddc160c60 0x7f8ddc070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:51.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.394+0000 7f8de31dc700 1 --2- 172.21.15.78:0/1197203409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ddc15d960 0x7f8ddc0702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:51.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.394+0000 7f8de31dc700 1 --2- 172.21.15.78:0/1197203409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ddc15d050 0x7f8ddc14bc70 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:51.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.394+0000 7f8de31dc700 1 -- 172.21.15.78:0/1197203409 >> 172.21.15.78:0/1197203409 conn(0x7f8ddc0cea60 msgr2=0x7f8ddc155b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:51.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.394+0000 7f8de31dc700 1 -- 172.21.15.78:0/1197203409 shutdown_connections 2023-10-11T09:49:51.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.394+0000 7f8de31dc700 1 -- 172.21.15.78:0/1197203409 wait complete. 2023-10-11T09:49:51.422 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:49:51.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.586+0000 7ff48aa64700 1 Processor -- start 2023-10-11T09:49:51.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.586+0000 7ff48aa64700 1 -- start start 2023-10-11T09:49:51.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.590+0000 7ff48aa64700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ff4840caf20 0x7ff484164610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:51.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.590+0000 7ff48aa64700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff484164b50 0x7ff48406bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:51.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.590+0000 7ff48aa64700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff484165390 con 0x7ff4840cda10 2023-10-11T09:49:51.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.590+0000 7ff48aa64700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4840cb3a0 con 0x7ff4840caf20 2023-10-11T09:49:51.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.590+0000 7ff48aa64700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4840ca440 con 0x7ff484164b50 2023-10-11T09:49:51.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.590+0000 7ff489261700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ff4840caf20 0x7ff484164610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:51.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.590+0000 7ff489261700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ff4840caf20 0x7ff484164610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41438/0 (socket says 172.21.15.78:41438) 2023-10-11T09:49:51.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.590+0000 7ff489261700 1 -- 172.21.15.78:0/3018713559 learned_addr learned my addr 172.21.15.78:0/3018713559 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:51.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.590+0000 7ff489261700 1 -- 172.21.15.78:0/3018713559 >> v2:172.21.15.133:3300/0 conn(0x7ff484164b50 msgr2=0x7ff48406bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:51.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.590+0000 7ff489261700 1 --2- 172.21.15.78:0/3018713559 >> v2:172.21.15.133:3300/0 conn(0x7ff484164b50 0x7ff48406bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:51.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.590+0000 7ff489261700 1 -- 172.21.15.78:0/3018713559 >> v1:172.21.15.133:6789/0 conn(0x7ff4840cda10 legacy=0x7ff4840cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:51.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.590+0000 7ff489261700 1 -- 172.21.15.78:0/3018713559 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4840ca720 con 0x7ff4840caf20 2023-10-11T09:49:51.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.590+0000 7ff489261700 1 --2- 172.21.15.78:0/3018713559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4840caf20 0x7ff484164610 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7ff4840ce4f0 tx=0x7ff474009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca269a203022681 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:51.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.590+0000 7ff488a60700 1 -- 172.21.15.78:0/3018713559 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff474004230 con 0x7ff4840caf20 2023-10-11T09:49:51.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.594+0000 7ff488a60700 1 -- 172.21.15.78:0/3018713559 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff474004390 con 0x7ff4840caf20 2023-10-11T09:49:51.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.594+0000 7ff488a60700 1 -- 172.21.15.78:0/3018713559 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4740036c0 con 0x7ff4840caf20 2023-10-11T09:49:51.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.594+0000 7ff48aa64700 1 -- 172.21.15.78:0/3018713559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4840caf20 msgr2=0x7ff484164610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:51.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.594+0000 7ff48aa64700 1 --2- 172.21.15.78:0/3018713559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4840caf20 0x7ff484164610 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7ff4840ce4f0 tx=0x7ff474009bf0 comp rx=0 tx=0).stop 2023-10-11T09:49:51.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.594+0000 7ff48aa64700 1 -- 172.21.15.78:0/3018713559 shutdown_connections 2023-10-11T09:49:51.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.594+0000 7ff48aa64700 1 --2- 172.21.15.78:0/3018713559 >> v2:172.21.15.133:3300/0 conn(0x7ff484164b50 0x7ff48406bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:51.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.594+0000 7ff48aa64700 1 --2- 172.21.15.78:0/3018713559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4840caf20 0x7ff484164610 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:51.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.594+0000 7ff48aa64700 1 -- 172.21.15.78:0/3018713559 >> 172.21.15.78:0/3018713559 conn(0x7ff4840cea60 msgr2=0x7ff4840cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:51.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.594+0000 7ff48aa64700 1 -- 172.21.15.78:0/3018713559 shutdown_connections 2023-10-11T09:49:51.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.594+0000 7ff48aa64700 1 -- 172.21.15.78:0/3018713559 wait complete. 2023-10-11T09:49:51.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.598+0000 7ff48aa64700 1 Processor -- start 2023-10-11T09:49:51.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.598+0000 7ff48aa64700 1 -- start start 2023-10-11T09:49:51.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.598+0000 7ff48aa64700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4840caf20 0x7ff484152d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:51.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.598+0000 7ff48aa64700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4840cda10 0x7ff4841fa200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:51.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.598+0000 7ff48aa64700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff484164b50 0x7ff484148030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:51.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.598+0000 7ff48aa64700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff484147000 con 0x7ff4840caf20 2023-10-11T09:49:51.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.598+0000 7ff48aa64700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4841fa740 con 0x7ff4840cda10 2023-10-11T09:49:51.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.598+0000 7ff48aa64700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff484147270 con 0x7ff484164b50 2023-10-11T09:49:51.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.598+0000 7ff489a62700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4840caf20 0x7ff484152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:51.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.598+0000 7ff489261700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4840cda10 0x7ff4841fa200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:51.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.598+0000 7ff489a62700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4840caf20 0x7ff484152d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41444/0 (socket says 172.21.15.78:41444) 2023-10-11T09:49:51.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.598+0000 7ff489a62700 1 -- 172.21.15.78:0/3516969697 learned_addr learned my addr 172.21.15.78:0/3516969697 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:51.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.598+0000 7ff48a263700 1 --2- 172.21.15.78:0/3516969697 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff484164b50 0x7ff484148030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:51.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.598+0000 7ff489a62700 1 -- 172.21.15.78:0/3516969697 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff484164b50 msgr2=0x7ff484148030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:51.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.598+0000 7ff489a62700 1 --2- 172.21.15.78:0/3516969697 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff484164b50 0x7ff484148030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:51.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.598+0000 7ff489a62700 1 -- 172.21.15.78:0/3516969697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4840cda10 msgr2=0x7ff4841fa200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:51.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.598+0000 7ff489a62700 1 --2- 172.21.15.78:0/3516969697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4840cda10 0x7ff4841fa200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:51.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.598+0000 7ff489a62700 1 -- 172.21.15.78:0/3516969697 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff474009570 con 0x7ff4840caf20 2023-10-11T09:49:51.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.598+0000 7ff48a263700 1 --2- 172.21.15.78:0/3516969697 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff484164b50 0x7ff484148030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:49:51.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.598+0000 7ff489a62700 1 --2- 172.21.15.78:0/3516969697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4840caf20 0x7ff484152d00 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7ff48000c4e0 tx=0x7ff48000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:51.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.602+0000 7ff47a7fc700 1 -- 172.21.15.78:0/3516969697 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff480011790 con 0x7ff4840caf20 2023-10-11T09:49:51.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.602+0000 7ff47a7fc700 1 -- 172.21.15.78:0/3516969697 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff480011910 con 0x7ff4840caf20 2023-10-11T09:49:51.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.602+0000 7ff48aa64700 1 -- 172.21.15.78:0/3516969697 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff484147550 con 0x7ff4840caf20 2023-10-11T09:49:51.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.602+0000 7ff47a7fc700 1 -- 172.21.15.78:0/3516969697 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff480011cc0 con 0x7ff4840caf20 2023-10-11T09:49:51.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.602+0000 7ff48aa64700 1 -- 172.21.15.78:0/3516969697 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff484147b10 con 0x7ff4840caf20 2023-10-11T09:49:51.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.602+0000 7ff48aa64700 1 -- 172.21.15.78:0/3516969697 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff484065d40 con 0x7ff4840caf20 2023-10-11T09:49:51.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.602+0000 7ff47a7fc700 1 -- 172.21.15.78:0/3516969697 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7ff48001c460 con 0x7ff4840caf20 2023-10-11T09:49:51.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.606+0000 7ff47a7fc700 1 --2- 172.21.15.78:0/3516969697 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff468065ed0 0x7ff468068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:51.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.606+0000 7ff47a7fc700 1 -- 172.21.15.78:0/3516969697 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff480083660 con 0x7ff4840caf20 2023-10-11T09:49:51.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.606+0000 7ff489261700 1 --2- 172.21.15.78:0/3516969697 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff468065ed0 0x7ff468068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:51.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.606+0000 7ff489261700 1 --2- 172.21.15.78:0/3516969697 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff468065ed0 0x7ff468068380 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff4840ce4f0 tx=0x7ff474008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:51.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.610+0000 7ff47a7fc700 1 -- 172.21.15.78:0/3516969697 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7ff480050c10 con 0x7ff4840caf20 2023-10-11T09:49:51.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.790+0000 7ff48aa64700 1 -- 172.21.15.78:0/3516969697 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4840b1480 con 0x7ff4840caf20 2023-10-11T09:49:51.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.794+0000 7ff47a7fc700 1 -- 172.21.15.78:0/3516969697 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+17846 (secure 0 0 0) 0x7ff480054230 con 0x7ff4840caf20 2023-10-11T09:49:51.802 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:49:51.802 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":104,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:49:50.955036+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:49:49.297597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-11T09:49:49.955104+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:49:51.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.798+0000 7ff48aa64700 1 -- 172.21.15.78:0/3516969697 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff468065ed0 msgr2=0x7ff468068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:51.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.798+0000 7ff48aa64700 1 --2- 172.21.15.78:0/3516969697 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff468065ed0 0x7ff468068380 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff4840ce4f0 tx=0x7ff474008030 comp rx=0 tx=0).stop 2023-10-11T09:49:51.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.798+0000 7ff48aa64700 1 -- 172.21.15.78:0/3516969697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4840caf20 msgr2=0x7ff484152d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:51.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.798+0000 7ff48aa64700 1 --2- 172.21.15.78:0/3516969697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4840caf20 0x7ff484152d00 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7ff48000c4e0 tx=0x7ff48000c7f0 comp rx=0 tx=0).stop 2023-10-11T09:49:51.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.798+0000 7ff48aa64700 1 -- 172.21.15.78:0/3516969697 shutdown_connections 2023-10-11T09:49:51.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.798+0000 7ff48aa64700 1 --2- 172.21.15.78:0/3516969697 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff468065ed0 0x7ff468068380 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:51.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.798+0000 7ff48aa64700 1 --2- 172.21.15.78:0/3516969697 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff484164b50 0x7ff484148030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:51.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.798+0000 7ff48aa64700 1 --2- 172.21.15.78:0/3516969697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4840cda10 0x7ff4841fa200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:51.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.798+0000 7ff48aa64700 1 --2- 172.21.15.78:0/3516969697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4840caf20 0x7ff484152d00 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:51.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.798+0000 7ff48aa64700 1 -- 172.21.15.78:0/3516969697 >> 172.21.15.78:0/3516969697 conn(0x7ff4840cea60 msgr2=0x7ff4840cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:51.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.798+0000 7ff48aa64700 1 -- 172.21.15.78:0/3516969697 shutdown_connections 2023-10-11T09:49:51.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.798+0000 7ff48aa64700 1 -- 172.21.15.78:0/3516969697 wait complete. 2023-10-11T09:49:51.825 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-11T09:49:52.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.994+0000 7f4986145700 1 Processor -- start 2023-10-11T09:49:52.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.994+0000 7f4986145700 1 -- start start 2023-10-11T09:49:52.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.998+0000 7f4986145700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f49800b3240 0x7f49800d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:52.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.998+0000 7f4986145700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f49800b3610 0x7f49800b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:52.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.998+0000 7f4986145700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49800b47f0 con 0x7f498015d040 2023-10-11T09:49:52.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.998+0000 7f4986145700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49800b3fd0 con 0x7f49800b3610 2023-10-11T09:49:52.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.998+0000 7f4986145700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49800b8130 con 0x7f49800b3240 2023-10-11T09:49:52.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.998+0000 7f4984942700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f49800b3610 0x7f49800b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:52.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.998+0000 7f4984942700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f49800b3610 0x7f49800b3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41446/0 (socket says 172.21.15.78:41446) 2023-10-11T09:49:52.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.998+0000 7f4984942700 1 -- 172.21.15.78:0/59460282 learned_addr learned my addr 172.21.15.78:0/59460282 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:52.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.998+0000 7f4984942700 1 -- 172.21.15.78:0/59460282 >> v2:172.21.15.133:3300/0 conn(0x7f49800b3240 msgr2=0x7f49800d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:52.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.998+0000 7f4984942700 1 --2- 172.21.15.78:0/59460282 >> v2:172.21.15.133:3300/0 conn(0x7f49800b3240 0x7f49800d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:52.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.998+0000 7f4984942700 1 -- 172.21.15.78:0/59460282 >> v1:172.21.15.78:6789/0 conn(0x7f498015d040 legacy=0x7f49800b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:52.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.998+0000 7f4984942700 1 -- 172.21.15.78:0/59460282 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49800b41e0 con 0x7f49800b3610 2023-10-11T09:49:52.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.998+0000 7f4984942700 1 --2- 172.21.15.78:0/59460282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49800b3610 0x7f49800b3a90 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f498006aa40 tx=0x7f4970009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18373cc3c4103726 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:52.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.998+0000 7f4977fff700 1 -- 172.21.15.78:0/59460282 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4970004230 con 0x7f49800b3610 2023-10-11T09:49:52.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.998+0000 7f4977fff700 1 -- 172.21.15.78:0/59460282 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4970004390 con 0x7f49800b3610 2023-10-11T09:49:52.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.998+0000 7f4977fff700 1 -- 172.21.15.78:0/59460282 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49700036c0 con 0x7f49800b3610 2023-10-11T09:49:52.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.998+0000 7f4986145700 1 -- 172.21.15.78:0/59460282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49800b3610 msgr2=0x7f49800b3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:52.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.998+0000 7f4986145700 1 --2- 172.21.15.78:0/59460282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49800b3610 0x7f49800b3a90 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f498006aa40 tx=0x7f4970009bf0 comp rx=0 tx=0).stop 2023-10-11T09:49:52.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.998+0000 7f4986145700 1 -- 172.21.15.78:0/59460282 shutdown_connections 2023-10-11T09:49:52.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.998+0000 7f4986145700 1 --2- 172.21.15.78:0/59460282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49800b3610 0x7f49800b3a90 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:52.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.998+0000 7f4986145700 1 --2- 172.21.15.78:0/59460282 >> v2:172.21.15.133:3300/0 conn(0x7f49800b3240 0x7f49800d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:52.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:51.998+0000 7f4986145700 1 -- 172.21.15.78:0/59460282 >> 172.21.15.78:0/59460282 conn(0x7f498009c260 msgr2=0x7f49800d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:52.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.002+0000 7f4986145700 1 -- 172.21.15.78:0/59460282 shutdown_connections 2023-10-11T09:49:52.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.002+0000 7f4986145700 1 -- 172.21.15.78:0/59460282 wait complete. 2023-10-11T09:49:52.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.006+0000 7f4986145700 1 Processor -- start 2023-10-11T09:49:52.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.006+0000 7f4986145700 1 -- start start 2023-10-11T09:49:52.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.006+0000 7f4986145700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49800b3240 0x7f498009d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:52.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.006+0000 7f4986145700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f49800b3610 0x7f498009fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:52.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.006+0000 7f4986145700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f498015d040 0x7f498015a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:52.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.006+0000 7f4986145700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49800c7fb0 con 0x7f49800b3240 2023-10-11T09:49:52.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.006+0000 7f4986145700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498009df10 con 0x7f498015d040 2023-10-11T09:49:52.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.006+0000 7f4986145700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49800c7c60 con 0x7f49800b3610 2023-10-11T09:49:52.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.006+0000 7f4985143700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49800b3240 0x7f498009d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:52.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.006+0000 7f4985143700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49800b3240 0x7f498009d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41460/0 (socket says 172.21.15.78:41460) 2023-10-11T09:49:52.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.006+0000 7f4985143700 1 -- 172.21.15.78:0/1681076436 learned_addr learned my addr 172.21.15.78:0/1681076436 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:52.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.006+0000 7f4984942700 1 --2- 172.21.15.78:0/1681076436 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f49800b3610 0x7f498009fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:52.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.006+0000 7f4985944700 1 --2- 172.21.15.78:0/1681076436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f498015d040 0x7f498015a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:52.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.006+0000 7f4985143700 1 -- 172.21.15.78:0/1681076436 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f49800b3610 msgr2=0x7f498009fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:52.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.006+0000 7f4985143700 1 --2- 172.21.15.78:0/1681076436 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f49800b3610 0x7f498009fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:52.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.006+0000 7f4985143700 1 -- 172.21.15.78:0/1681076436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f498015d040 msgr2=0x7f498015a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:52.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.006+0000 7f4985143700 1 --2- 172.21.15.78:0/1681076436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f498015d040 0x7f498015a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:52.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.006+0000 7f4985143700 1 -- 172.21.15.78:0/1681076436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4970009570 con 0x7f49800b3240 2023-10-11T09:49:52.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.006+0000 7f4985944700 1 --2- 172.21.15.78:0/1681076436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f498015d040 0x7f498015a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:49:52.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.006+0000 7f4984942700 1 --2- 172.21.15.78:0/1681076436 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f49800b3610 0x7f498009fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:49:52.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.006+0000 7f4985143700 1 --2- 172.21.15.78:0/1681076436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49800b3240 0x7f498009d9d0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f497c00b550 tx=0x7f497c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:52.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.006+0000 7f4975ffb700 1 -- 172.21.15.78:0/1681076436 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c011830 con 0x7f49800b3240 2023-10-11T09:49:52.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.006+0000 7f4975ffb700 1 -- 172.21.15.78:0/1681076436 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497c011990 con 0x7f49800b3240 2023-10-11T09:49:52.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.006+0000 7f4975ffb700 1 -- 172.21.15.78:0/1681076436 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c011d40 con 0x7f49800b3240 2023-10-11T09:49:52.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.006+0000 7f4986145700 1 -- 172.21.15.78:0/1681076436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49800a0300 con 0x7f49800b3240 2023-10-11T09:49:52.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.010+0000 7f4986145700 1 -- 172.21.15.78:0/1681076436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49800a0840 con 0x7f49800b3240 2023-10-11T09:49:52.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.010+0000 7f4986145700 1 -- 172.21.15.78:0/1681076436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4980065d40 con 0x7f49800b3240 2023-10-11T09:49:52.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.010+0000 7f4975ffb700 1 -- 172.21.15.78:0/1681076436 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f497c01a400 con 0x7f49800b3240 2023-10-11T09:49:52.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.014+0000 7f4975ffb700 1 --2- 172.21.15.78:0/1681076436 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4964065ed0 0x7f4964068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:52.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.014+0000 7f4975ffb700 1 -- 172.21.15.78:0/1681076436 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f497c0837c0 con 0x7f49800b3240 2023-10-11T09:49:52.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.014+0000 7f4984942700 1 --2- 172.21.15.78:0/1681076436 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4964065ed0 0x7f4964068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:52.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.014+0000 7f4984942700 1 --2- 172.21.15.78:0/1681076436 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4964065ed0 0x7f4964068380 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f498006aa40 tx=0x7f4970008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:52.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.022+0000 7f4975ffb700 1 -- 172.21.15.78:0/1681076436 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f497c050ea0 con 0x7f49800b3240 2023-10-11T09:49:52.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.238+0000 7f4986145700 1 -- 172.21.15.78:0/1681076436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f498006a060 con 0x7f49800b3240 2023-10-11T09:49:52.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.978+0000 7f4975ffb700 1 -- 172.21.15.78:0/1681076436 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v78) v1 ==== 118+0+0 (secure 0 0 0) 0x7f497c0544c0 con 0x7f49800b3240 2023-10-11T09:49:52.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.982+0000 7f4986145700 1 -- 172.21.15.78:0/1681076436 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4964065ed0 msgr2=0x7f4964068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:52.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.982+0000 7f4986145700 1 --2- 172.21.15.78:0/1681076436 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4964065ed0 0x7f4964068380 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f498006aa40 tx=0x7f4970008030 comp rx=0 tx=0).stop 2023-10-11T09:49:52.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.982+0000 7f4986145700 1 -- 172.21.15.78:0/1681076436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49800b3240 msgr2=0x7f498009d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:52.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.982+0000 7f4986145700 1 --2- 172.21.15.78:0/1681076436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49800b3240 0x7f498009d9d0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f497c00b550 tx=0x7f497c00b910 comp rx=0 tx=0).stop 2023-10-11T09:49:52.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.982+0000 7f4986145700 1 -- 172.21.15.78:0/1681076436 shutdown_connections 2023-10-11T09:49:52.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.982+0000 7f4986145700 1 --2- 172.21.15.78:0/1681076436 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4964065ed0 0x7f4964068380 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:52.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.982+0000 7f4986145700 1 --2- 172.21.15.78:0/1681076436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f498015d040 0x7f498015a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:52.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.982+0000 7f4986145700 1 --2- 172.21.15.78:0/1681076436 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f49800b3610 0x7f498009fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:52.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.982+0000 7f4986145700 1 --2- 172.21.15.78:0/1681076436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49800b3240 0x7f498009d9d0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:52.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.982+0000 7f4986145700 1 -- 172.21.15.78:0/1681076436 >> 172.21.15.78:0/1681076436 conn(0x7f498009c260 msgr2=0x7f498015fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:52.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.982+0000 7f4986145700 1 -- 172.21.15.78:0/1681076436 shutdown_connections 2023-10-11T09:49:52.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:52.982+0000 7f4986145700 1 -- 172.21.15.78:0/1681076436 wait complete. 2023-10-11T09:49:53.012 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-11T09:49:53.013 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-11T09:49:53.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.186+0000 7fd7c59b7700 1 Processor -- start 2023-10-11T09:49:53.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.186+0000 7fd7c59b7700 1 -- start start 2023-10-11T09:49:53.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.186+0000 7fd7c59b7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7c015a950 0x7fd7c00ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:53.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.186+0000 7fd7c59b7700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd7c00b28f0 0x7fd7c00d0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:53.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.186+0000 7fd7c59b7700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fd7c00d1340 0x7fd7c015ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:53.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.186+0000 7fd7c59b7700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7c00ceef0 con 0x7fd7c00d1340 2023-10-11T09:49:53.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.186+0000 7fd7c59b7700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7c00d1760 con 0x7fd7c00b28f0 2023-10-11T09:49:53.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.186+0000 7fd7c59b7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c00cc470 con 0x7fd7c015a950 2023-10-11T09:49:53.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.190+0000 7fd7c51b6700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fd7c00d1340 0x7fd7c015ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:53.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.190+0000 7fd7c49b5700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7c015a950 0x7fd7c00ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:53.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.190+0000 7fd7c51b6700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fd7c00d1340 0x7fd7c015ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41466/0 (socket says 172.21.15.78:41466) 2023-10-11T09:49:53.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.190+0000 7fd7c51b6700 1 -- 172.21.15.78:0/4176423725 learned_addr learned my addr 172.21.15.78:0/4176423725 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:53.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.190+0000 7fd7b7fff700 1 --2- 172.21.15.78:0/4176423725 >> v2:172.21.15.133:3300/0 conn(0x7fd7c00b28f0 0x7fd7c00d0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:53.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.190+0000 7fd7c51b6700 1 -- 172.21.15.78:0/4176423725 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7c015a950 msgr2=0x7fd7c00ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:53.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.190+0000 7fd7c51b6700 1 --2- 172.21.15.78:0/4176423725 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7c015a950 0x7fd7c00ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:53.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.190+0000 7fd7c51b6700 1 -- 172.21.15.78:0/4176423725 >> v2:172.21.15.133:3300/0 conn(0x7fd7c00b28f0 msgr2=0x7fd7c00d0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:53.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.190+0000 7fd7c51b6700 1 --2- 172.21.15.78:0/4176423725 >> v2:172.21.15.133:3300/0 conn(0x7fd7c00b28f0 0x7fd7c00d0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:53.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.190+0000 7fd7c51b6700 1 -- 172.21.15.78:0/4176423725 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c00cc6f0 con 0x7fd7c00d1340 2023-10-11T09:49:53.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.190+0000 7fd7c51b6700 1 --2- 172.21.15.78:0/4176423725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7c00d1340 0x7fd7c015ad20 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0071de0 tx=0x7fd7b8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adada2528e43e2b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:53.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.190+0000 7fd7b77fe700 1 -- 172.21.15.78:0/4176423725 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b8011830 con 0x7fd7c00d1340 2023-10-11T09:49:53.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.190+0000 7fd7b77fe700 1 -- 172.21.15.78:0/4176423725 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b8011990 con 0x7fd7c00d1340 2023-10-11T09:49:53.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.190+0000 7fd7b77fe700 1 -- 172.21.15.78:0/4176423725 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b8011d20 con 0x7fd7c00d1340 2023-10-11T09:49:53.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.194+0000 7fd7c59b7700 1 -- 172.21.15.78:0/4176423725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7c00d1340 msgr2=0x7fd7c015ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:53.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.194+0000 7fd7c59b7700 1 --2- 172.21.15.78:0/4176423725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7c00d1340 0x7fd7c015ad20 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0071de0 tx=0x7fd7b8009bf0 comp rx=0 tx=0).stop 2023-10-11T09:49:53.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.194+0000 7fd7c59b7700 1 -- 172.21.15.78:0/4176423725 shutdown_connections 2023-10-11T09:49:53.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.194+0000 7fd7c59b7700 1 --2- 172.21.15.78:0/4176423725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7c00d1340 0x7fd7c015ad20 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:53.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.194+0000 7fd7c59b7700 1 --2- 172.21.15.78:0/4176423725 >> v2:172.21.15.133:3300/0 conn(0x7fd7c00b28f0 0x7fd7c00d0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:53.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.194+0000 7fd7c59b7700 1 --2- 172.21.15.78:0/4176423725 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7c015a950 0x7fd7c00ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:53.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.194+0000 7fd7c59b7700 1 -- 172.21.15.78:0/4176423725 >> 172.21.15.78:0/4176423725 conn(0x7fd7c006a060 msgr2=0x7fd7c006a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:53.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.194+0000 7fd7c59b7700 1 -- 172.21.15.78:0/4176423725 shutdown_connections 2023-10-11T09:49:53.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.194+0000 7fd7c59b7700 1 -- 172.21.15.78:0/4176423725 wait complete. 2023-10-11T09:49:53.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.194+0000 7fd7c59b7700 1 Processor -- start 2023-10-11T09:49:53.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.194+0000 7fd7c59b7700 1 -- start start 2023-10-11T09:49:53.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.198+0000 7fd7c59b7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7c00b28f0 0x7fd7c0151400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:53.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.198+0000 7fd7c59b7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7c00d1340 0x7fd7c00c6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:53.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.198+0000 7fd7c59b7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7c015a950 0x7fd7c00c68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:53.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.198+0000 7fd7c59b7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c006b170 con 0x7fd7c00b28f0 2023-10-11T09:49:53.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.198+0000 7fd7c59b7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c0151940 con 0x7fd7c00d1340 2023-10-11T09:49:53.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.198+0000 7fd7c59b7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c0142210 con 0x7fd7c015a950 2023-10-11T09:49:53.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.198+0000 7fd7c49b5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7c00b28f0 0x7fd7c0151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:53.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.198+0000 7fd7c49b5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7c00b28f0 0x7fd7c0151400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41480/0 (socket says 172.21.15.78:41480) 2023-10-11T09:49:53.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.198+0000 7fd7c49b5700 1 -- 172.21.15.78:0/2185120486 learned_addr learned my addr 172.21.15.78:0/2185120486 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:53.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.198+0000 7fd7b7fff700 1 --2- 172.21.15.78:0/2185120486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7c00d1340 0x7fd7c00c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:53.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.198+0000 7fd7c51b6700 1 --2- 172.21.15.78:0/2185120486 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7c015a950 0x7fd7c00c68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:53.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.198+0000 7fd7c49b5700 1 -- 172.21.15.78:0/2185120486 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7c015a950 msgr2=0x7fd7c00c68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:53.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.198+0000 7fd7c49b5700 1 --2- 172.21.15.78:0/2185120486 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7c015a950 0x7fd7c00c68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:53.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.198+0000 7fd7c49b5700 1 -- 172.21.15.78:0/2185120486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7c00d1340 msgr2=0x7fd7c00c6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:53.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.198+0000 7fd7c49b5700 1 --2- 172.21.15.78:0/2185120486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7c00d1340 0x7fd7c00c6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:53.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.198+0000 7fd7c49b5700 1 -- 172.21.15.78:0/2185120486 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b8009570 con 0x7fd7c00b28f0 2023-10-11T09:49:53.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.198+0000 7fd7c49b5700 1 --2- 172.21.15.78:0/2185120486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7c00b28f0 0x7fd7c0151400 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc00efc0 tx=0x7fd7bc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:53.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.202+0000 7fd7b57fa700 1 -- 172.21.15.78:0/2185120486 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc00cdc0 con 0x7fd7c00b28f0 2023-10-11T09:49:53.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.202+0000 7fd7c59b7700 1 -- 172.21.15.78:0/2185120486 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c01e3e40 con 0x7fd7c00b28f0 2023-10-11T09:49:53.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.202+0000 7fd7b57fa700 1 -- 172.21.15.78:0/2185120486 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7bc00f040 con 0x7fd7c00b28f0 2023-10-11T09:49:53.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.202+0000 7fd7b57fa700 1 -- 172.21.15.78:0/2185120486 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc0146c0 con 0x7fd7c00b28f0 2023-10-11T09:49:53.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.202+0000 7fd7c59b7700 1 -- 172.21.15.78:0/2185120486 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c01e4400 con 0x7fd7c00b28f0 2023-10-11T09:49:53.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.202+0000 7fd7c59b7700 1 -- 172.21.15.78:0/2185120486 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7c0065d40 con 0x7fd7c00b28f0 2023-10-11T09:49:53.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.206+0000 7fd7b57fa700 1 -- 172.21.15.78:0/2185120486 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fd7bc014820 con 0x7fd7c00b28f0 2023-10-11T09:49:53.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.206+0000 7fd7b57fa700 1 --2- 172.21.15.78:0/2185120486 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd7ac065ed0 0x7fd7ac068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:53.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.206+0000 7fd7b57fa700 1 -- 172.21.15.78:0/2185120486 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd7bc084510 con 0x7fd7c00b28f0 2023-10-11T09:49:53.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.206+0000 7fd7b7fff700 1 --2- 172.21.15.78:0/2185120486 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd7ac065ed0 0x7fd7ac068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:53.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.206+0000 7fd7b7fff700 1 --2- 172.21.15.78:0/2185120486 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd7ac065ed0 0x7fd7ac068380 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd7c00c7510 tx=0x7fd7b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:53.218 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.210+0000 7fd7b57fa700 1 -- 172.21.15.78:0/2185120486 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fd7bc051a40 con 0x7fd7c00b28f0 2023-10-11T09:49:53.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.422+0000 7fd7c59b7700 1 -- 172.21.15.78:0/2185120486 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd7c006a4d0 con 0x7fd7c00b28f0 2023-10-11T09:49:53.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.990+0000 7fd7b57fa700 1 -- 172.21.15.78:0/2185120486 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v79) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd7bc055060 con 0x7fd7c00b28f0 2023-10-11T09:49:54.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.994+0000 7fd7c59b7700 1 -- 172.21.15.78:0/2185120486 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd7ac065ed0 msgr2=0x7fd7ac068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:54.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.994+0000 7fd7c59b7700 1 --2- 172.21.15.78:0/2185120486 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd7ac065ed0 0x7fd7ac068380 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd7c00c7510 tx=0x7fd7b0008040 comp rx=0 tx=0).stop 2023-10-11T09:49:54.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.994+0000 7fd7c59b7700 1 -- 172.21.15.78:0/2185120486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7c00b28f0 msgr2=0x7fd7c0151400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:54.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.994+0000 7fd7c59b7700 1 --2- 172.21.15.78:0/2185120486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7c00b28f0 0x7fd7c0151400 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc00efc0 tx=0x7fd7bc00c3a0 comp rx=0 tx=0).stop 2023-10-11T09:49:54.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.994+0000 7fd7c59b7700 1 -- 172.21.15.78:0/2185120486 shutdown_connections 2023-10-11T09:49:54.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.994+0000 7fd7c59b7700 1 --2- 172.21.15.78:0/2185120486 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd7ac065ed0 0x7fd7ac068380 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:54.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.994+0000 7fd7c59b7700 1 --2- 172.21.15.78:0/2185120486 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7c015a950 0x7fd7c00c68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:54.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.994+0000 7fd7c59b7700 1 --2- 172.21.15.78:0/2185120486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7c00d1340 0x7fd7c00c6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:54.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.994+0000 7fd7c59b7700 1 --2- 172.21.15.78:0/2185120486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7c00b28f0 0x7fd7c0151400 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:54.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.994+0000 7fd7c59b7700 1 -- 172.21.15.78:0/2185120486 >> 172.21.15.78:0/2185120486 conn(0x7fd7c006a060 msgr2=0x7fd7c00c48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:54.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.994+0000 7fd7c59b7700 1 -- 172.21.15.78:0/2185120486 shutdown_connections 2023-10-11T09:49:54.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:53.994+0000 7fd7c59b7700 1 -- 172.21.15.78:0/2185120486 wait complete. 2023-10-11T09:49:54.026 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-11T09:49:54.027 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:49:54.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.198+0000 7f07ea95e700 1 Processor -- start 2023-10-11T09:49:54.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.198+0000 7f07ea95e700 1 -- start start 2023-10-11T09:49:54.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.202+0000 7f07ea95e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07e4154e00 0x7f07e415b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:54.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.202+0000 7f07ea95e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f07e4071040 0x7f07e40cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:54.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.202+0000 7f07ea95e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e40720f0 con 0x7f07e40cba00 2023-10-11T09:49:54.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.202+0000 7f07ea95e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07e415bb90 con 0x7f07e4071040 2023-10-11T09:49:54.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.202+0000 7f07ea95e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e40714e0 con 0x7f07e4154e00 2023-10-11T09:49:54.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.202+0000 7f07db7fe700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f07e4071040 0x7f07e40cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:54.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.202+0000 7f07e3fff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07e4154e00 0x7f07e415b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:54.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.202+0000 7f07e8efb700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f07e40cba00 0x7f07e415e1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:50736/0 (socket says 172.21.15.78:50736) 2023-10-11T09:49:54.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.202+0000 7f07e8efb700 1 -- 172.21.15.78:0/3132167417 learned_addr learned my addr 172.21.15.78:0/3132167417 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:49:54.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.206+0000 7f07e37fe700 1 -- 172.21.15.78:0/3132167417 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 273044836 0 0) 0x7f07e40720f0 con 0x7f07e40cba00 2023-10-11T09:49:54.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.206+0000 7f07e37fe700 1 -- 172.21.15.78:0/3132167417 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07d00035d0 con 0x7f07e40cba00 2023-10-11T09:49:54.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.206+0000 7f07db7fe700 1 -- 172.21.15.78:0/3132167417 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07e4154e00 msgr2=0x7f07e415b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:54.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.206+0000 7f07db7fe700 1 --2- 172.21.15.78:0/3132167417 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07e4154e00 0x7f07e415b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:54.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.206+0000 7f07db7fe700 1 -- 172.21.15.78:0/3132167417 >> v1:172.21.15.133:6789/0 conn(0x7f07e40cba00 legacy=0x7f07e415e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:54.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.206+0000 7f07db7fe700 1 -- 172.21.15.78:0/3132167417 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e40717c0 con 0x7f07e4071040 2023-10-11T09:49:54.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.206+0000 7f07db7fe700 1 --2- 172.21.15.78:0/3132167417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e4071040 0x7f07e40cb4c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f07cc009850 tx=0x7f07cc009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97a2016bbb91144d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:54.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.206+0000 7f07e37fe700 1 -- 172.21.15.78:0/3132167417 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07cc011830 con 0x7f07e4071040 2023-10-11T09:49:54.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.206+0000 7f07e37fe700 1 -- 172.21.15.78:0/3132167417 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07cc011990 con 0x7f07e4071040 2023-10-11T09:49:54.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.206+0000 7f07e37fe700 1 -- 172.21.15.78:0/3132167417 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07cc011d40 con 0x7f07e4071040 2023-10-11T09:49:54.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.206+0000 7f07ea95e700 1 -- 172.21.15.78:0/3132167417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e4071040 msgr2=0x7f07e40cb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:54.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.206+0000 7f07ea95e700 1 --2- 172.21.15.78:0/3132167417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e4071040 0x7f07e40cb4c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f07cc009850 tx=0x7f07cc009b60 comp rx=0 tx=0).stop 2023-10-11T09:49:54.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.206+0000 7f07ea95e700 1 -- 172.21.15.78:0/3132167417 shutdown_connections 2023-10-11T09:49:54.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.206+0000 7f07ea95e700 1 --2- 172.21.15.78:0/3132167417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e4071040 0x7f07e40cb4c0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:54.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.206+0000 7f07ea95e700 1 --2- 172.21.15.78:0/3132167417 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07e4154e00 0x7f07e415b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:54.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.206+0000 7f07ea95e700 1 -- 172.21.15.78:0/3132167417 >> 172.21.15.78:0/3132167417 conn(0x7f07e406aca0 msgr2=0x7f07e40b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:54.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.206+0000 7f07ea95e700 1 -- 172.21.15.78:0/3132167417 shutdown_connections 2023-10-11T09:49:54.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.206+0000 7f07ea95e700 1 -- 172.21.15.78:0/3132167417 wait complete. 2023-10-11T09:49:54.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.210+0000 7f07ea95e700 1 Processor -- start 2023-10-11T09:49:54.218 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.210+0000 7f07ea95e700 1 -- start start 2023-10-11T09:49:54.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.210+0000 7f07ea95e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e4071040 0x7f07e4157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:54.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.210+0000 7f07ea95e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07e40cba00 0x7f07e413b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:54.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.210+0000 7f07ea95e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07e4154e00 0x7f07e40b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:54.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.210+0000 7f07ea95e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e40b1dc0 con 0x7f07e40cba00 2023-10-11T09:49:54.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.210+0000 7f07ea95e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e40b20a0 con 0x7f07e4071040 2023-10-11T09:49:54.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.210+0000 7f07ea95e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e40b2380 con 0x7f07e4154e00 2023-10-11T09:49:54.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.210+0000 7f07db7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07e40cba00 0x7f07e413b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:54.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.210+0000 7f07db7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07e40cba00 0x7f07e413b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41488/0 (socket says 172.21.15.78:41488) 2023-10-11T09:49:54.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.210+0000 7f07db7fe700 1 -- 172.21.15.78:0/1674735213 learned_addr learned my addr 172.21.15.78:0/1674735213 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:54.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.210+0000 7f07e3fff700 1 --2- 172.21.15.78:0/1674735213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e4071040 0x7f07e4157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:54.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.210+0000 7f07e8efb700 1 --2- 172.21.15.78:0/1674735213 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07e4154e00 0x7f07e40b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:54.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.210+0000 7f07db7fe700 1 -- 172.21.15.78:0/1674735213 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07e4154e00 msgr2=0x7f07e40b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:54.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.210+0000 7f07db7fe700 1 --2- 172.21.15.78:0/1674735213 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07e4154e00 0x7f07e40b2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:54.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.210+0000 7f07db7fe700 1 -- 172.21.15.78:0/1674735213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e4071040 msgr2=0x7f07e4157b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:54.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.210+0000 7f07db7fe700 1 --2- 172.21.15.78:0/1674735213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e4071040 0x7f07e4157b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:54.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.210+0000 7f07db7fe700 1 -- 172.21.15.78:0/1674735213 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07cc0094b0 con 0x7f07e40cba00 2023-10-11T09:49:54.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.210+0000 7f07e3fff700 1 --2- 172.21.15.78:0/1674735213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e4071040 0x7f07e4157b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:49:54.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.214+0000 7f07db7fe700 1 --2- 172.21.15.78:0/1674735213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07e40cba00 0x7f07e413b620 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f07cc0090b0 tx=0x7f07cc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:54.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.214+0000 7f07e17fa700 1 -- 172.21.15.78:0/1674735213 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07cc004380 con 0x7f07e40cba00 2023-10-11T09:49:54.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.214+0000 7f07e17fa700 1 -- 172.21.15.78:0/1674735213 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07cc01f070 con 0x7f07e40cba00 2023-10-11T09:49:54.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.214+0000 7f07ea95e700 1 -- 172.21.15.78:0/1674735213 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07e40b2660 con 0x7f07e40cba00 2023-10-11T09:49:54.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.214+0000 7f07e17fa700 1 -- 172.21.15.78:0/1674735213 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07cc0036c0 con 0x7f07e40cba00 2023-10-11T09:49:54.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.214+0000 7f07ea95e700 1 -- 172.21.15.78:0/1674735213 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e4141ca0 con 0x7f07e40cba00 2023-10-11T09:49:54.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.214+0000 7f07ea95e700 1 -- 172.21.15.78:0/1674735213 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07e404e530 con 0x7f07e40cba00 2023-10-11T09:49:54.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.214+0000 7f07e17fa700 1 -- 172.21.15.78:0/1674735213 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f07cc021030 con 0x7f07e40cba00 2023-10-11T09:49:54.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.218+0000 7f07e17fa700 1 --2- 172.21.15.78:0/1674735213 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f07c4065ed0 0x7f07c4068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:54.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.218+0000 7f07e17fa700 1 -- 172.21.15.78:0/1674735213 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f07cc0849b0 con 0x7f07e40cba00 2023-10-11T09:49:54.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.218+0000 7f07e3fff700 1 --2- 172.21.15.78:0/1674735213 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f07c4065ed0 0x7f07c4068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:54.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.218+0000 7f07e3fff700 1 --2- 172.21.15.78:0/1674735213 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f07c4065ed0 0x7f07c4068380 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f07e415dcf0 tx=0x7f07d4009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:54.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.226+0000 7f07e17fa700 1 -- 172.21.15.78:0/1674735213 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f07cc052110 con 0x7f07e40cba00 2023-10-11T09:49:54.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.434+0000 7f07ea95e700 1 -- 172.21.15.78:0/1674735213 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07e406e1e0 con 0x7f07e40cba00 2023-10-11T09:49:54.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.434+0000 7f07e17fa700 1 -- 172.21.15.78:0/1674735213 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4656 (secure 0 0 0) 0x7f07cc015090 con 0x7f07e40cba00 2023-10-11T09:49:54.444 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:49:54.444 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":79,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6812,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/3697741427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3697741427},{"type":"v1","addr":"172.21.15.133:6833","nonce":3697741427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":6815,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2640710924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2640710924},{"type":"v1","addr":"172.21.15.133:6835","nonce":2640710924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8028,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/1538320637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":1538320637},{"type":"v1","addr":"172.21.15.78:6835","nonce":1538320637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:49:50.956153+0000","modified":"2023-10-11T09:49:53.977383+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8031},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8031":{"gid":8031,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6837/1286798022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1286798022},{"type":"v1","addr":"172.21.15.78:6837","nonce":1286798022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-11T09:49:54.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.438+0000 7f07ea95e700 1 -- 172.21.15.78:0/1674735213 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f07c4065ed0 msgr2=0x7f07c4068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:54.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.438+0000 7f07ea95e700 1 --2- 172.21.15.78:0/1674735213 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f07c4065ed0 0x7f07c4068380 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f07e415dcf0 tx=0x7f07d4009240 comp rx=0 tx=0).stop 2023-10-11T09:49:54.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.438+0000 7f07ea95e700 1 -- 172.21.15.78:0/1674735213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07e40cba00 msgr2=0x7f07e413b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:54.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.438+0000 7f07ea95e700 1 --2- 172.21.15.78:0/1674735213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07e40cba00 0x7f07e413b620 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f07cc0090b0 tx=0x7f07cc004770 comp rx=0 tx=0).stop 2023-10-11T09:49:54.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.438+0000 7f07ea95e700 1 -- 172.21.15.78:0/1674735213 shutdown_connections 2023-10-11T09:49:54.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.438+0000 7f07ea95e700 1 --2- 172.21.15.78:0/1674735213 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f07c4065ed0 0x7f07c4068380 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:54.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.438+0000 7f07ea95e700 1 --2- 172.21.15.78:0/1674735213 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07e4154e00 0x7f07e40b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:54.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.438+0000 7f07ea95e700 1 --2- 172.21.15.78:0/1674735213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07e40cba00 0x7f07e413b620 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:54.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.438+0000 7f07ea95e700 1 --2- 172.21.15.78:0/1674735213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e4071040 0x7f07e4157b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:54.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.438+0000 7f07ea95e700 1 -- 172.21.15.78:0/1674735213 >> 172.21.15.78:0/1674735213 conn(0x7f07e406aca0 msgr2=0x7f07e40ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:54.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.438+0000 7f07ea95e700 1 -- 172.21.15.78:0/1674735213 shutdown_connections 2023-10-11T09:49:54.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.438+0000 7f07ea95e700 1 -- 172.21.15.78:0/1674735213 wait complete. 2023-10-11T09:49:54.449 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 79 2023-10-11T09:49:54.466 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:49:54.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.626+0000 7f00a3076700 1 Processor -- start 2023-10-11T09:49:54.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.630+0000 7f00a3076700 1 -- start start 2023-10-11T09:49:54.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.630+0000 7f00a3076700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f009c0cda10 0x7f009c0caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:54.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.630+0000 7f00a3076700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f009c0cdde0 0x7f009c164610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:54.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.630+0000 7f00a3076700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f009c164b50 0x7f009c06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:54.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.630+0000 7f00a3076700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f009c165390 con 0x7f009c164b50 2023-10-11T09:49:54.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.630+0000 7f00a3076700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f009c0ca440 con 0x7f009c0cda10 2023-10-11T09:49:54.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.630+0000 7f00a3076700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f009c0ca720 con 0x7f009c0cdde0 2023-10-11T09:49:54.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.630+0000 7f00a2875700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f009c164b50 0x7f009c06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:54.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.630+0000 7f00a2074700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f009c0cda10 0x7f009c0caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:54.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.630+0000 7f00a2875700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f009c164b50 0x7f009c06bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36096/0 (socket says 172.21.15.78:36096) 2023-10-11T09:49:54.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.630+0000 7f00a2875700 1 -- 172.21.15.78:0/401724917 learned_addr learned my addr 172.21.15.78:0/401724917 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:54.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.630+0000 7f00a2074700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f009c0cda10 0x7f009c0caf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:47718/0 (socket says 172.21.15.78:47718) 2023-10-11T09:49:54.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.630+0000 7f00a1873700 1 --2- 172.21.15.78:0/401724917 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f009c0cdde0 0x7f009c164610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:54.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.634+0000 7f00a2875700 1 -- 172.21.15.78:0/401724917 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f009c0cdde0 msgr2=0x7f009c164610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:54.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.634+0000 7f00a2875700 1 --2- 172.21.15.78:0/401724917 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f009c0cdde0 0x7f009c164610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:54.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.634+0000 7f00a2875700 1 -- 172.21.15.78:0/401724917 >> v2:172.21.15.133:3300/0 conn(0x7f009c0cda10 msgr2=0x7f009c0caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:54.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.634+0000 7f00a2875700 1 --2- 172.21.15.78:0/401724917 >> v2:172.21.15.133:3300/0 conn(0x7f009c0cda10 0x7f009c0caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:54.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.634+0000 7f00a2875700 1 -- 172.21.15.78:0/401724917 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f009c0caa00 con 0x7f009c164b50 2023-10-11T09:49:54.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.634+0000 7f00a2074700 1 --2- 172.21.15.78:0/401724917 >> v2:172.21.15.133:3300/0 conn(0x7f009c0cda10 0x7f009c0caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:49:54.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.634+0000 7f00a2875700 1 --2- 172.21.15.78:0/401724917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f009c164b50 0x7f009c06bbf0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f009c06afb0 tx=0x7f009400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b79ade58d28e9a40 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:54.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.634+0000 7f00a1072700 1 -- 172.21.15.78:0/401724917 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009400e070 con 0x7f009c164b50 2023-10-11T09:49:54.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.634+0000 7f00a1072700 1 -- 172.21.15.78:0/401724917 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f009400b910 con 0x7f009c164b50 2023-10-11T09:49:54.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.634+0000 7f00a1072700 1 -- 172.21.15.78:0/401724917 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009400bcc0 con 0x7f009c164b50 2023-10-11T09:49:54.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.634+0000 7f00a3076700 1 -- 172.21.15.78:0/401724917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f009c164b50 msgr2=0x7f009c06bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:54.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.634+0000 7f00a3076700 1 --2- 172.21.15.78:0/401724917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f009c164b50 0x7f009c06bbf0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f009c06afb0 tx=0x7f009400b330 comp rx=0 tx=0).stop 2023-10-11T09:49:54.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.634+0000 7f00a3076700 1 -- 172.21.15.78:0/401724917 shutdown_connections 2023-10-11T09:49:54.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.634+0000 7f00a3076700 1 --2- 172.21.15.78:0/401724917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f009c164b50 0x7f009c06bbf0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:54.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.634+0000 7f00a3076700 1 --2- 172.21.15.78:0/401724917 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f009c0cdde0 0x7f009c164610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:54.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.634+0000 7f00a3076700 1 --2- 172.21.15.78:0/401724917 >> v2:172.21.15.133:3300/0 conn(0x7f009c0cda10 0x7f009c0caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:54.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.634+0000 7f00a3076700 1 -- 172.21.15.78:0/401724917 >> 172.21.15.78:0/401724917 conn(0x7f009c0cea60 msgr2=0x7f009c0cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:54.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.634+0000 7f00a3076700 1 -- 172.21.15.78:0/401724917 shutdown_connections 2023-10-11T09:49:54.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.634+0000 7f00a3076700 1 -- 172.21.15.78:0/401724917 wait complete. 2023-10-11T09:49:54.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.638+0000 7f00a3076700 1 Processor -- start 2023-10-11T09:49:54.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.638+0000 7f00a3076700 1 -- start start 2023-10-11T09:49:54.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.638+0000 7f00a3076700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f009c0cda10 0x7f009c137a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:54.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.638+0000 7f00a3076700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f009c0cdde0 0x7f009c139800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:54.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.638+0000 7f00a3076700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f009c164b50 0x7f009c163640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:54.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.638+0000 7f00a3076700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009c1470b0 con 0x7f009c0cdde0 2023-10-11T09:49:54.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.638+0000 7f00a3076700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009c147390 con 0x7f009c0cda10 2023-10-11T09:49:54.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.638+0000 7f00a3076700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f009c147670 con 0x7f009c164b50 2023-10-11T09:49:54.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.638+0000 7f00a1873700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f009c0cdde0 0x7f009c139800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:54.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.638+0000 7f00a1873700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f009c0cdde0 0x7f009c139800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36102/0 (socket says 172.21.15.78:36102) 2023-10-11T09:49:54.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.638+0000 7f00a1873700 1 -- 172.21.15.78:0/3886786600 learned_addr learned my addr 172.21.15.78:0/3886786600 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:54.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.638+0000 7f00a2875700 1 --2- 172.21.15.78:0/3886786600 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f009c164b50 0x7f009c163640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:54.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.642+0000 7f00a2074700 1 --2- 172.21.15.78:0/3886786600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f009c0cda10 0x7f009c137a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:54.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.642+0000 7f00a1873700 1 -- 172.21.15.78:0/3886786600 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f009c164b50 msgr2=0x7f009c163640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:54.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.642+0000 7f00a1873700 1 --2- 172.21.15.78:0/3886786600 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f009c164b50 0x7f009c163640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:54.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.642+0000 7f00a1873700 1 -- 172.21.15.78:0/3886786600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f009c0cda10 msgr2=0x7f009c137a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:54.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.642+0000 7f00a1873700 1 --2- 172.21.15.78:0/3886786600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f009c0cda10 0x7f009c137a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:54.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.642+0000 7f00a1873700 1 -- 172.21.15.78:0/3886786600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00980094b0 con 0x7f009c0cdde0 2023-10-11T09:49:54.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.642+0000 7f00a1873700 1 --2- 172.21.15.78:0/3886786600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f009c0cdde0 0x7f009c139800 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f008c00ebe0 tx=0x7f008c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:54.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.642+0000 7f0092ffd700 1 -- 172.21.15.78:0/3886786600 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008c009740 con 0x7f009c0cdde0 2023-10-11T09:49:54.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.642+0000 7f0092ffd700 1 -- 172.21.15.78:0/3886786600 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f008c00c850 con 0x7f009c0cdde0 2023-10-11T09:49:54.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.642+0000 7f0092ffd700 1 -- 172.21.15.78:0/3886786600 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008c00cbe0 con 0x7f009c0cdde0 2023-10-11T09:49:54.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.642+0000 7f00a3076700 1 -- 172.21.15.78:0/3886786600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0094009b30 con 0x7f009c0cdde0 2023-10-11T09:49:54.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.642+0000 7f00a3076700 1 -- 172.21.15.78:0/3886786600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f009c0ce3b0 con 0x7f009c0cdde0 2023-10-11T09:49:54.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.642+0000 7f00a3076700 1 -- 172.21.15.78:0/3886786600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f009c065d40 con 0x7f009c0cdde0 2023-10-11T09:49:54.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.646+0000 7f0092ffd700 1 -- 172.21.15.78:0/3886786600 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f008c00cd40 con 0x7f009c0cdde0 2023-10-11T09:49:54.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.646+0000 7f0092ffd700 1 --2- 172.21.15.78:0/3886786600 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0080065ed0 0x7f0080068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:54.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.646+0000 7f0092ffd700 1 -- 172.21.15.78:0/3886786600 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f008c083820 con 0x7f009c0cdde0 2023-10-11T09:49:54.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.646+0000 7f00a2074700 1 --2- 172.21.15.78:0/3886786600 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0080065ed0 0x7f0080068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:54.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.646+0000 7f00a2074700 1 --2- 172.21.15.78:0/3886786600 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0080065ed0 0x7f0080068380 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f009c0cb2c0 tx=0x7f0098009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:54.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.654+0000 7f0092ffd700 1 -- 172.21.15.78:0/3886786600 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f008c050e80 con 0x7f009c0cdde0 2023-10-11T09:49:54.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.826+0000 7f00a3076700 1 -- 172.21.15.78:0/3886786600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f009c0b1480 con 0x7f009c0cdde0 2023-10-11T09:49:54.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.830+0000 7f0092ffd700 1 -- 172.21.15.78:0/3886786600 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+17833 (secure 0 0 0) 0x7f008c0544a0 con 0x7f009c0cdde0 2023-10-11T09:49:54.838 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:49:54.838 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":105,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:49:51.961930+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:49:49.297597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-11T09:49:49.955104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:49:54.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.830+0000 7f00a3076700 1 -- 172.21.15.78:0/3886786600 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0080065ed0 msgr2=0x7f0080068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:54.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.830+0000 7f00a3076700 1 --2- 172.21.15.78:0/3886786600 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0080065ed0 0x7f0080068380 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f009c0cb2c0 tx=0x7f0098009240 comp rx=0 tx=0).stop 2023-10-11T09:49:54.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.830+0000 7f00a3076700 1 -- 172.21.15.78:0/3886786600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f009c0cdde0 msgr2=0x7f009c139800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:54.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.830+0000 7f00a3076700 1 --2- 172.21.15.78:0/3886786600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f009c0cdde0 0x7f009c139800 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f008c00ebe0 tx=0x7f008c00eef0 comp rx=0 tx=0).stop 2023-10-11T09:49:54.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.834+0000 7f00a3076700 1 -- 172.21.15.78:0/3886786600 shutdown_connections 2023-10-11T09:49:54.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.834+0000 7f00a3076700 1 --2- 172.21.15.78:0/3886786600 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0080065ed0 0x7f0080068380 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:54.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.834+0000 7f00a3076700 1 --2- 172.21.15.78:0/3886786600 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f009c164b50 0x7f009c163640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:54.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.834+0000 7f00a3076700 1 --2- 172.21.15.78:0/3886786600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f009c0cdde0 0x7f009c139800 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:54.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.834+0000 7f00a3076700 1 --2- 172.21.15.78:0/3886786600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f009c0cda10 0x7f009c137a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:54.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.834+0000 7f00a3076700 1 -- 172.21.15.78:0/3886786600 >> 172.21.15.78:0/3886786600 conn(0x7f009c0cea60 msgr2=0x7f009c06bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:54.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.834+0000 7f00a3076700 1 -- 172.21.15.78:0/3886786600 shutdown_connections 2023-10-11T09:49:54.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:54.834+0000 7f00a3076700 1 -- 172.21.15.78:0/3886786600 wait complete. 2023-10-11T09:49:54.863 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-11T09:49:54.864 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-11T09:49:55.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.030+0000 7f5e371b5700 1 Processor -- start 2023-10-11T09:49:55.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.030+0000 7f5e371b5700 1 -- start start 2023-10-11T09:49:55.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.034+0000 7f5e371b5700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5e3015d050 0x7f5e300cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:55.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.034+0000 7f5e371b5700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f5e30160c60 0x7f5e3015d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:55.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.034+0000 7f5e371b5700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e30160210 con 0x7f5e3015d960 2023-10-11T09:49:55.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.034+0000 7f5e371b5700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e300ccd60 con 0x7f5e30160c60 2023-10-11T09:49:55.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.034+0000 7f5e371b5700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e300ccfc0 con 0x7f5e3015d050 2023-10-11T09:49:55.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.034+0000 7f5e359b2700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f5e30160c60 0x7f5e3015d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:55.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.034+0000 7f5e359b2700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f5e30160c60 0x7f5e3015d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36106/0 (socket says 172.21.15.78:36106) 2023-10-11T09:49:55.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.034+0000 7f5e359b2700 1 -- 172.21.15.78:0/3620164010 learned_addr learned my addr 172.21.15.78:0/3620164010 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:55.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.034+0000 7f5e361b3700 1 --2- 172.21.15.78:0/3620164010 >> v2:172.21.15.133:3300/0 conn(0x7f5e3015d050 0x7f5e300cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:55.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.034+0000 7f5e359b2700 1 -- 172.21.15.78:0/3620164010 >> v2:172.21.15.133:3300/0 conn(0x7f5e3015d050 msgr2=0x7f5e300cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:55.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.034+0000 7f5e359b2700 1 --2- 172.21.15.78:0/3620164010 >> v2:172.21.15.133:3300/0 conn(0x7f5e3015d050 0x7f5e300cc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:55.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.034+0000 7f5e359b2700 1 -- 172.21.15.78:0/3620164010 >> v1:172.21.15.133:6789/0 conn(0x7f5e3015d960 legacy=0x7f5e300b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:55.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.034+0000 7f5e359b2700 1 -- 172.21.15.78:0/3620164010 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e300cd2a0 con 0x7f5e30160c60 2023-10-11T09:49:55.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.034+0000 7f5e359b2700 1 --2- 172.21.15.78:0/3620164010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e30160c60 0x7f5e3015d420 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f5e300cf8b0 tx=0x7f5e20009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4c3bf48eb81594 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:55.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.034+0000 7f5e351b1700 1 -- 172.21.15.78:0/3620164010 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e20011830 con 0x7f5e30160c60 2023-10-11T09:49:55.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.034+0000 7f5e351b1700 1 -- 172.21.15.78:0/3620164010 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e20011990 con 0x7f5e30160c60 2023-10-11T09:49:55.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.034+0000 7f5e351b1700 1 -- 172.21.15.78:0/3620164010 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e20011d20 con 0x7f5e30160c60 2023-10-11T09:49:55.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.038+0000 7f5e371b5700 1 -- 172.21.15.78:0/3620164010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e30160c60 msgr2=0x7f5e3015d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:55.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.038+0000 7f5e371b5700 1 --2- 172.21.15.78:0/3620164010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e30160c60 0x7f5e3015d420 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f5e300cf8b0 tx=0x7f5e20009bf0 comp rx=0 tx=0).stop 2023-10-11T09:49:55.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.038+0000 7f5e371b5700 1 -- 172.21.15.78:0/3620164010 shutdown_connections 2023-10-11T09:49:55.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.038+0000 7f5e371b5700 1 --2- 172.21.15.78:0/3620164010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e30160c60 0x7f5e3015d420 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:55.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.038+0000 7f5e371b5700 1 --2- 172.21.15.78:0/3620164010 >> v2:172.21.15.133:3300/0 conn(0x7f5e3015d050 0x7f5e300cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:55.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.038+0000 7f5e371b5700 1 -- 172.21.15.78:0/3620164010 >> 172.21.15.78:0/3620164010 conn(0x7f5e300cea60 msgr2=0x7f5e300cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:55.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.038+0000 7f5e371b5700 1 -- 172.21.15.78:0/3620164010 shutdown_connections 2023-10-11T09:49:55.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.038+0000 7f5e371b5700 1 -- 172.21.15.78:0/3620164010 wait complete. 2023-10-11T09:49:55.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.038+0000 7f5e371b5700 1 Processor -- start 2023-10-11T09:49:55.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.038+0000 7f5e371b5700 1 -- start start 2023-10-11T09:49:55.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.042+0000 7f5e371b5700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e3015d050 0x7f5e3015bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:55.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.042+0000 7f5e371b5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e3015d960 0x7f5e300c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:55.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.042+0000 7f5e371b5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e30160c60 0x7f5e301572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:55.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.042+0000 7f5e371b5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e301f1f60 con 0x7f5e3015d960 2023-10-11T09:49:55.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.042+0000 7f5e371b5700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e300b2a60 con 0x7f5e30160c60 2023-10-11T09:49:55.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.042+0000 7f5e371b5700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e300b3300 con 0x7f5e3015d050 2023-10-11T09:49:55.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.042+0000 7f5e359b2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e3015d960 0x7f5e300c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:55.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.042+0000 7f5e359b2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e3015d960 0x7f5e300c95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36118/0 (socket says 172.21.15.78:36118) 2023-10-11T09:49:55.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.042+0000 7f5e359b2700 1 -- 172.21.15.78:0/1853825881 learned_addr learned my addr 172.21.15.78:0/1853825881 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:55.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.042+0000 7f5e369b4700 1 --2- 172.21.15.78:0/1853825881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e30160c60 0x7f5e301572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:55.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.042+0000 7f5e361b3700 1 --2- 172.21.15.78:0/1853825881 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e3015d050 0x7f5e3015bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:55.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.042+0000 7f5e359b2700 1 -- 172.21.15.78:0/1853825881 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e3015d050 msgr2=0x7f5e3015bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:55.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.042+0000 7f5e359b2700 1 --2- 172.21.15.78:0/1853825881 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e3015d050 0x7f5e3015bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:55.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.042+0000 7f5e359b2700 1 -- 172.21.15.78:0/1853825881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e30160c60 msgr2=0x7f5e301572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:55.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.042+0000 7f5e359b2700 1 --2- 172.21.15.78:0/1853825881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e30160c60 0x7f5e301572e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:55.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.042+0000 7f5e359b2700 1 -- 172.21.15.78:0/1853825881 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e20009570 con 0x7f5e3015d960 2023-10-11T09:49:55.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.042+0000 7f5e361b3700 1 --2- 172.21.15.78:0/1853825881 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e3015d050 0x7f5e3015bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:49:55.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.042+0000 7f5e359b2700 1 --2- 172.21.15.78:0/1853825881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e3015d960 0x7f5e300c95a0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f5e30072690 tx=0x7f5e20004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:55.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.042+0000 7f5e26ffd700 1 -- 172.21.15.78:0/1853825881 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e20004550 con 0x7f5e3015d960 2023-10-11T09:49:55.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.042+0000 7f5e26ffd700 1 -- 172.21.15.78:0/1853825881 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2001f070 con 0x7f5e3015d960 2023-10-11T09:49:55.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.042+0000 7f5e26ffd700 1 -- 172.21.15.78:0/1853825881 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e20003900 con 0x7f5e3015d960 2023-10-11T09:49:55.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.042+0000 7f5e371b5700 1 -- 172.21.15.78:0/1853825881 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e300c9c80 con 0x7f5e3015d960 2023-10-11T09:49:55.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.042+0000 7f5e371b5700 1 -- 172.21.15.78:0/1853825881 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e300b2f90 con 0x7f5e3015d960 2023-10-11T09:49:55.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.046+0000 7f5e371b5700 1 -- 172.21.15.78:0/1853825881 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e300b34b0 con 0x7f5e3015d960 2023-10-11T09:49:55.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.046+0000 7f5e26ffd700 1 -- 172.21.15.78:0/1853825881 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f5e20003a60 con 0x7f5e3015d960 2023-10-11T09:49:55.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.046+0000 7f5e26ffd700 1 --2- 172.21.15.78:0/1853825881 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5e14065ed0 0x7f5e14068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:55.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.046+0000 7f5e26ffd700 1 -- 172.21.15.78:0/1853825881 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5e20021070 con 0x7f5e3015d960 2023-10-11T09:49:55.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.050+0000 7f5e361b3700 1 --2- 172.21.15.78:0/1853825881 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5e14065ed0 0x7f5e14068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:55.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.050+0000 7f5e361b3700 1 --2- 172.21.15.78:0/1853825881 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5e14065ed0 0x7f5e14068380 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009ba0 tx=0x7f5e2c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:55.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.054+0000 7f5e26ffd700 1 -- 172.21.15.78:0/1853825881 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f5e200522d0 con 0x7f5e3015d960 2023-10-11T09:49:55.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.226+0000 7f5e371b5700 1 -- 172.21.15.78:0/1853825881 --> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5e30160740 con 0x7f5e14065ed0 2023-10-11T09:49:55.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.242+0000 7f5e26ffd700 1 -- 172.21.15.78:0/1853825881 <== mgr.4101 v2:172.21.15.78:6832/35606 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309550 (secure 0 0 0) 0x7f5e2c011800 con 0x7f5e14065ed0 2023-10-11T09:49:55.254 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:49:55.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.250+0000 7f5e371b5700 1 -- 172.21.15.78:0/1853825881 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5e14065ed0 msgr2=0x7f5e14068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:55.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.250+0000 7f5e371b5700 1 --2- 172.21.15.78:0/1853825881 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5e14065ed0 0x7f5e14068380 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009ba0 tx=0x7f5e2c008040 comp rx=0 tx=0).stop 2023-10-11T09:49:55.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.250+0000 7f5e371b5700 1 -- 172.21.15.78:0/1853825881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e3015d960 msgr2=0x7f5e300c95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:55.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.250+0000 7f5e371b5700 1 --2- 172.21.15.78:0/1853825881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e3015d960 0x7f5e300c95a0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f5e30072690 tx=0x7f5e20004770 comp rx=0 tx=0).stop 2023-10-11T09:49:55.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.250+0000 7f5e371b5700 1 -- 172.21.15.78:0/1853825881 shutdown_connections 2023-10-11T09:49:55.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.250+0000 7f5e371b5700 1 --2- 172.21.15.78:0/1853825881 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5e14065ed0 0x7f5e14068380 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:55.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.250+0000 7f5e371b5700 1 --2- 172.21.15.78:0/1853825881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e30160c60 0x7f5e301572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:55.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.250+0000 7f5e371b5700 1 --2- 172.21.15.78:0/1853825881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e3015d960 0x7f5e300c95a0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:55.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.250+0000 7f5e371b5700 1 --2- 172.21.15.78:0/1853825881 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e3015d050 0x7f5e3015bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:55.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.250+0000 7f5e371b5700 1 -- 172.21.15.78:0/1853825881 >> 172.21.15.78:0/1853825881 conn(0x7f5e300cea60 msgr2=0x7f5e3015ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:55.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.250+0000 7f5e371b5700 1 -- 172.21.15.78:0/1853825881 shutdown_connections 2023-10-11T09:49:55.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.250+0000 7f5e371b5700 1 -- 172.21.15.78:0/1853825881 wait complete. 2023-10-11T09:49:55.260 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2023-10-11T09:49:55.278 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":381,"stamp":"2023-10-11T09:49:54.520995+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":173,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259684,"kb_used_data":11300,"kb_used_omap":0,"kb_used_meta":248320,"kb_avail":754715036,"statfs":{"total":773094113280,"available":772828196864,"internally_reserved":0,"allocated":11571200,"data_stored":6152573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":254279680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.045612"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.860765+0000","last_change":"2023-10-11T09:49:50.989786+0000","last_active":"2023-10-11T09:49:52.860765+0000","last_peered":"2023-10-11T09:49:52.860765+0000","last_clean":"2023-10-11T09:49:52.860765+0000","last_became_active":"2023-10-11T09:49:50.989515+0000","last_became_peered":"2023-10-11T09:49:50.989515+0000","last_unstale":"2023-10-11T09:49:52.860765+0000","last_undegraded":"2023-10-11T09:49:52.860765+0000","last_fullsized":"2023-10-11T09:49:52.860765+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:37:08.369025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.976764+0000","last_change":"2023-10-11T09:49:50.979308+0000","last_active":"2023-10-11T09:49:51.976764+0000","last_peered":"2023-10-11T09:49:51.976764+0000","last_clean":"2023-10-11T09:49:51.976764+0000","last_became_active":"2023-10-11T09:49:50.979017+0000","last_became_peered":"2023-10-11T09:49:50.979017+0000","last_unstale":"2023-10-11T09:49:51.976764+0000","last_undegraded":"2023-10-11T09:49:51.976764+0000","last_fullsized":"2023-10-11T09:49:51.976764+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:10:48.960792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.860974+0000","last_change":"2023-10-11T09:49:50.988605+0000","last_active":"2023-10-11T09:49:52.860974+0000","last_peered":"2023-10-11T09:49:52.860974+0000","last_clean":"2023-10-11T09:49:52.860974+0000","last_became_active":"2023-10-11T09:49:50.988286+0000","last_became_peered":"2023-10-11T09:49:50.988286+0000","last_unstale":"2023-10-11T09:49:52.860974+0000","last_undegraded":"2023-10-11T09:49:52.860974+0000","last_fullsized":"2023-10-11T09:49:52.860974+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:23:52.163820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.862224+0000","last_change":"2023-10-11T09:49:50.987680+0000","last_active":"2023-10-11T09:49:52.862224+0000","last_peered":"2023-10-11T09:49:52.862224+0000","last_clean":"2023-10-11T09:49:52.862224+0000","last_became_active":"2023-10-11T09:49:50.987385+0000","last_became_peered":"2023-10-11T09:49:50.987385+0000","last_unstale":"2023-10-11T09:49:52.862224+0000","last_undegraded":"2023-10-11T09:49:52.862224+0000","last_fullsized":"2023-10-11T09:49:52.862224+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:55:30.217353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.862376+0000","last_change":"2023-10-11T09:49:50.995325+0000","last_active":"2023-10-11T09:49:52.862376+0000","last_peered":"2023-10-11T09:49:52.862376+0000","last_clean":"2023-10-11T09:49:52.862376+0000","last_became_active":"2023-10-11T09:49:50.995036+0000","last_became_peered":"2023-10-11T09:49:50.995036+0000","last_unstale":"2023-10-11T09:49:52.862376+0000","last_undegraded":"2023-10-11T09:49:52.862376+0000","last_fullsized":"2023-10-11T09:49:52.862376+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:59:46.488123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.980264+0000","last_change":"2023-10-11T09:49:50.981217+0000","last_active":"2023-10-11T09:49:51.980264+0000","last_peered":"2023-10-11T09:49:51.980264+0000","last_clean":"2023-10-11T09:49:51.980264+0000","last_became_active":"2023-10-11T09:49:50.980922+0000","last_became_peered":"2023-10-11T09:49:50.980922+0000","last_unstale":"2023-10-11T09:49:51.980264+0000","last_undegraded":"2023-10-11T09:49:51.980264+0000","last_fullsized":"2023-10-11T09:49:51.980264+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:15:13.595478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.980116+0000","last_change":"2023-10-11T09:49:50.978839+0000","last_active":"2023-10-11T09:49:51.980116+0000","last_peered":"2023-10-11T09:49:51.980116+0000","last_clean":"2023-10-11T09:49:51.980116+0000","last_became_active":"2023-10-11T09:49:50.978525+0000","last_became_peered":"2023-10-11T09:49:50.978525+0000","last_unstale":"2023-10-11T09:49:51.980116+0000","last_undegraded":"2023-10-11T09:49:51.980116+0000","last_fullsized":"2023-10-11T09:49:51.980116+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:18:09.722589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.976094+0000","last_change":"2023-10-11T09:49:50.986499+0000","last_active":"2023-10-11T09:49:51.976094+0000","last_peered":"2023-10-11T09:49:51.976094+0000","last_clean":"2023-10-11T09:49:51.976094+0000","last_became_active":"2023-10-11T09:49:50.982208+0000","last_became_peered":"2023-10-11T09:49:50.982208+0000","last_unstale":"2023-10-11T09:49:51.976094+0000","last_undegraded":"2023-10-11T09:49:51.976094+0000","last_fullsized":"2023-10-11T09:49:51.976094+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:22:44.533534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.979969+0000","last_change":"2023-10-11T09:49:50.997182+0000","last_active":"2023-10-11T09:49:51.979969+0000","last_peered":"2023-10-11T09:49:51.979969+0000","last_clean":"2023-10-11T09:49:51.979969+0000","last_became_active":"2023-10-11T09:49:50.996715+0000","last_became_peered":"2023-10-11T09:49:50.996715+0000","last_unstale":"2023-10-11T09:49:51.979969+0000","last_undegraded":"2023-10-11T09:49:51.979969+0000","last_fullsized":"2023-10-11T09:49:51.979969+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:02:09.315636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.861599+0000","last_change":"2023-10-11T09:49:50.997926+0000","last_active":"2023-10-11T09:49:52.861599+0000","last_peered":"2023-10-11T09:49:52.861599+0000","last_clean":"2023-10-11T09:49:52.861599+0000","last_became_active":"2023-10-11T09:49:50.997643+0000","last_became_peered":"2023-10-11T09:49:50.997643+0000","last_unstale":"2023-10-11T09:49:52.861599+0000","last_undegraded":"2023-10-11T09:49:52.861599+0000","last_fullsized":"2023-10-11T09:49:52.861599+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:33:20.865574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.980442+0000","last_change":"2023-10-11T09:49:50.995300+0000","last_active":"2023-10-11T09:49:51.980442+0000","last_peered":"2023-10-11T09:49:51.980442+0000","last_clean":"2023-10-11T09:49:51.980442+0000","last_became_active":"2023-10-11T09:49:50.994518+0000","last_became_peered":"2023-10-11T09:49:50.994518+0000","last_unstale":"2023-10-11T09:49:51.980442+0000","last_undegraded":"2023-10-11T09:49:51.980442+0000","last_fullsized":"2023-10-11T09:49:51.980442+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:59:16.220357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.979625+0000","last_change":"2023-10-11T09:49:50.998757+0000","last_active":"2023-10-11T09:49:51.979625+0000","last_peered":"2023-10-11T09:49:51.979625+0000","last_clean":"2023-10-11T09:49:51.979625+0000","last_became_active":"2023-10-11T09:49:50.998484+0000","last_became_peered":"2023-10-11T09:49:50.998484+0000","last_unstale":"2023-10-11T09:49:51.979625+0000","last_undegraded":"2023-10-11T09:49:51.979625+0000","last_fullsized":"2023-10-11T09:49:51.979625+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:34:10.165727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.862029+0000","last_change":"2023-10-11T09:49:50.995089+0000","last_active":"2023-10-11T09:49:52.862029+0000","last_peered":"2023-10-11T09:49:52.862029+0000","last_clean":"2023-10-11T09:49:52.862029+0000","last_became_active":"2023-10-11T09:49:50.994807+0000","last_became_peered":"2023-10-11T09:49:50.994807+0000","last_unstale":"2023-10-11T09:49:52.862029+0000","last_undegraded":"2023-10-11T09:49:52.862029+0000","last_fullsized":"2023-10-11T09:49:52.862029+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T21:14:55.909654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.761608+0000","last_change":"2023-10-11T09:49:50.988965+0000","last_active":"2023-10-11T09:49:52.761608+0000","last_peered":"2023-10-11T09:49:52.761608+0000","last_clean":"2023-10-11T09:49:52.761608+0000","last_became_active":"2023-10-11T09:49:50.988720+0000","last_became_peered":"2023-10-11T09:49:50.988720+0000","last_unstale":"2023-10-11T09:49:52.761608+0000","last_undegraded":"2023-10-11T09:49:52.761608+0000","last_fullsized":"2023-10-11T09:49:52.761608+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:18:24.742598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.862360+0000","last_change":"2023-10-11T09:49:50.990497+0000","last_active":"2023-10-11T09:49:52.862360+0000","last_peered":"2023-10-11T09:49:52.862360+0000","last_clean":"2023-10-11T09:49:52.862360+0000","last_became_active":"2023-10-11T09:49:50.990259+0000","last_became_peered":"2023-10-11T09:49:50.990259+0000","last_unstale":"2023-10-11T09:49:52.862360+0000","last_undegraded":"2023-10-11T09:49:52.862360+0000","last_fullsized":"2023-10-11T09:49:52.862360+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:21:03.166998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.979627+0000","last_change":"2023-10-11T09:49:50.995978+0000","last_active":"2023-10-11T09:49:51.979627+0000","last_peered":"2023-10-11T09:49:51.979627+0000","last_clean":"2023-10-11T09:49:51.979627+0000","last_became_active":"2023-10-11T09:49:50.995663+0000","last_became_peered":"2023-10-11T09:49:50.995663+0000","last_unstale":"2023-10-11T09:49:51.979627+0000","last_undegraded":"2023-10-11T09:49:51.979627+0000","last_fullsized":"2023-10-11T09:49:51.979627+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:02:09.315636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.976892+0000","last_change":"2023-10-11T09:49:50.985645+0000","last_active":"2023-10-11T09:49:51.976892+0000","last_peered":"2023-10-11T09:49:51.976892+0000","last_clean":"2023-10-11T09:49:51.976892+0000","last_became_active":"2023-10-11T09:49:50.982055+0000","last_became_peered":"2023-10-11T09:49:50.982055+0000","last_unstale":"2023-10-11T09:49:51.976892+0000","last_undegraded":"2023-10-11T09:49:51.976892+0000","last_fullsized":"2023-10-11T09:49:51.976892+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:59:58.229994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.862749+0000","last_change":"2023-10-11T09:49:50.985744+0000","last_active":"2023-10-11T09:49:52.862749+0000","last_peered":"2023-10-11T09:49:52.862749+0000","last_clean":"2023-10-11T09:49:52.862749+0000","last_became_active":"2023-10-11T09:49:50.985453+0000","last_became_peered":"2023-10-11T09:49:50.985453+0000","last_unstale":"2023-10-11T09:49:52.862749+0000","last_undegraded":"2023-10-11T09:49:52.862749+0000","last_fullsized":"2023-10-11T09:49:52.862749+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:05:08.173043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.762398+0000","last_change":"2023-10-11T09:49:50.991940+0000","last_active":"2023-10-11T09:49:52.762398+0000","last_peered":"2023-10-11T09:49:52.762398+0000","last_clean":"2023-10-11T09:49:52.762398+0000","last_became_active":"2023-10-11T09:49:50.991649+0000","last_became_peered":"2023-10-11T09:49:50.991649+0000","last_unstale":"2023-10-11T09:49:52.762398+0000","last_undegraded":"2023-10-11T09:49:52.762398+0000","last_fullsized":"2023-10-11T09:49:52.762398+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:40:12.518424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.979638+0000","last_change":"2023-10-11T09:49:50.997023+0000","last_active":"2023-10-11T09:49:51.979638+0000","last_peered":"2023-10-11T09:49:51.979638+0000","last_clean":"2023-10-11T09:49:51.979638+0000","last_became_active":"2023-10-11T09:49:50.996746+0000","last_became_peered":"2023-10-11T09:49:50.996746+0000","last_unstale":"2023-10-11T09:49:51.979638+0000","last_undegraded":"2023-10-11T09:49:51.979638+0000","last_fullsized":"2023-10-11T09:49:51.979638+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:59:16.220357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.761990+0000","last_change":"2023-10-11T09:49:50.993502+0000","last_active":"2023-10-11T09:49:52.761990+0000","last_peered":"2023-10-11T09:49:52.761990+0000","last_clean":"2023-10-11T09:49:52.761990+0000","last_became_active":"2023-10-11T09:49:50.993252+0000","last_became_peered":"2023-10-11T09:49:50.993252+0000","last_unstale":"2023-10-11T09:49:52.761990+0000","last_undegraded":"2023-10-11T09:49:52.761990+0000","last_fullsized":"2023-10-11T09:49:52.761990+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:28:32.732527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.976087+0000","last_change":"2023-10-11T09:49:50.986490+0000","last_active":"2023-10-11T09:49:51.976087+0000","last_peered":"2023-10-11T09:49:51.976087+0000","last_clean":"2023-10-11T09:49:51.976087+0000","last_became_active":"2023-10-11T09:49:50.986130+0000","last_became_peered":"2023-10-11T09:49:50.986130+0000","last_unstale":"2023-10-11T09:49:51.976087+0000","last_undegraded":"2023-10-11T09:49:51.976087+0000","last_fullsized":"2023-10-11T09:49:51.976087+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:49:51.518585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.976117+0000","last_change":"2023-10-11T09:49:50.986408+0000","last_active":"2023-10-11T09:49:51.976117+0000","last_peered":"2023-10-11T09:49:51.976117+0000","last_clean":"2023-10-11T09:49:51.976117+0000","last_became_active":"2023-10-11T09:49:50.982073+0000","last_became_peered":"2023-10-11T09:49:50.982073+0000","last_unstale":"2023-10-11T09:49:51.976117+0000","last_undegraded":"2023-10-11T09:49:51.976117+0000","last_fullsized":"2023-10-11T09:49:51.976117+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:58:51.631450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.979754+0000","last_change":"2023-10-11T09:49:50.995424+0000","last_active":"2023-10-11T09:49:51.979754+0000","last_peered":"2023-10-11T09:49:51.979754+0000","last_clean":"2023-10-11T09:49:51.979754+0000","last_became_active":"2023-10-11T09:49:50.993764+0000","last_became_peered":"2023-10-11T09:49:50.993764+0000","last_unstale":"2023-10-11T09:49:51.979754+0000","last_undegraded":"2023-10-11T09:49:51.979754+0000","last_fullsized":"2023-10-11T09:49:51.979754+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:21:51.584219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.862247+0000","last_change":"2023-10-11T09:49:51.001328+0000","last_active":"2023-10-11T09:49:52.862247+0000","last_peered":"2023-10-11T09:49:52.862247+0000","last_clean":"2023-10-11T09:49:52.862247+0000","last_became_active":"2023-10-11T09:49:51.000975+0000","last_became_peered":"2023-10-11T09:49:51.000975+0000","last_unstale":"2023-10-11T09:49:52.862247+0000","last_undegraded":"2023-10-11T09:49:52.862247+0000","last_fullsized":"2023-10-11T09:49:52.862247+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:09:32.742677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.976890+0000","last_change":"2023-10-11T09:49:50.989370+0000","last_active":"2023-10-11T09:49:51.976890+0000","last_peered":"2023-10-11T09:49:51.976890+0000","last_clean":"2023-10-11T09:49:51.976890+0000","last_became_active":"2023-10-11T09:49:50.988703+0000","last_became_peered":"2023-10-11T09:49:50.988703+0000","last_unstale":"2023-10-11T09:49:51.976890+0000","last_undegraded":"2023-10-11T09:49:51.976890+0000","last_fullsized":"2023-10-11T09:49:51.976890+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:39:08.209288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.980663+0000","last_change":"2023-10-11T09:49:50.986873+0000","last_active":"2023-10-11T09:49:51.980663+0000","last_peered":"2023-10-11T09:49:51.980663+0000","last_clean":"2023-10-11T09:49:51.980663+0000","last_became_active":"2023-10-11T09:49:50.986584+0000","last_became_peered":"2023-10-11T09:49:50.986584+0000","last_unstale":"2023-10-11T09:49:51.980663+0000","last_undegraded":"2023-10-11T09:49:51.980663+0000","last_fullsized":"2023-10-11T09:49:51.980663+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:44:02.419155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.861589+0000","last_change":"2023-10-11T09:49:50.988505+0000","last_active":"2023-10-11T09:49:52.861589+0000","last_peered":"2023-10-11T09:49:52.861589+0000","last_clean":"2023-10-11T09:49:52.861589+0000","last_became_active":"2023-10-11T09:49:50.987725+0000","last_became_peered":"2023-10-11T09:49:50.987725+0000","last_unstale":"2023-10-11T09:49:52.861589+0000","last_undegraded":"2023-10-11T09:49:52.861589+0000","last_fullsized":"2023-10-11T09:49:52.861589+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:23:52.163820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.976473+0000","last_change":"2023-10-11T09:49:50.992552+0000","last_active":"2023-10-11T09:49:51.976473+0000","last_peered":"2023-10-11T09:49:51.976473+0000","last_clean":"2023-10-11T09:49:51.976473+0000","last_became_active":"2023-10-11T09:49:50.992261+0000","last_became_peered":"2023-10-11T09:49:50.992261+0000","last_unstale":"2023-10-11T09:49:51.976473+0000","last_undegraded":"2023-10-11T09:49:51.976473+0000","last_fullsized":"2023-10-11T09:49:51.976473+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T21:15:14.070871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.761596+0000","last_change":"2023-10-11T09:49:50.991964+0000","last_active":"2023-10-11T09:49:52.761596+0000","last_peered":"2023-10-11T09:49:52.761596+0000","last_clean":"2023-10-11T09:49:52.761596+0000","last_became_active":"2023-10-11T09:49:50.991675+0000","last_became_peered":"2023-10-11T09:49:50.991675+0000","last_unstale":"2023-10-11T09:49:52.761596+0000","last_undegraded":"2023-10-11T09:49:52.761596+0000","last_fullsized":"2023-10-11T09:49:52.761596+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:03:50.944882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.862144+0000","last_change":"2023-10-11T09:49:50.986112+0000","last_active":"2023-10-11T09:49:52.862144+0000","last_peered":"2023-10-11T09:49:52.862144+0000","last_clean":"2023-10-11T09:49:52.862144+0000","last_became_active":"2023-10-11T09:49:50.985872+0000","last_became_peered":"2023-10-11T09:49:50.985872+0000","last_unstale":"2023-10-11T09:49:52.862144+0000","last_undegraded":"2023-10-11T09:49:52.862144+0000","last_fullsized":"2023-10-11T09:49:52.862144+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:09:54.785090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.859955+0000","last_change":"2023-10-11T09:49:51.000196+0000","last_active":"2023-10-11T09:49:52.859955+0000","last_peered":"2023-10-11T09:49:52.859955+0000","last_clean":"2023-10-11T09:49:52.859955+0000","last_became_active":"2023-10-11T09:49:50.999958+0000","last_became_peered":"2023-10-11T09:49:50.999958+0000","last_unstale":"2023-10-11T09:49:52.859955+0000","last_undegraded":"2023-10-11T09:49:52.859955+0000","last_fullsized":"2023-10-11T09:49:52.859955+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:33:20.865574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.762407+0000","last_change":"2023-10-11T09:49:50.998343+0000","last_active":"2023-10-11T09:49:52.762407+0000","last_peered":"2023-10-11T09:49:52.762407+0000","last_clean":"2023-10-11T09:49:52.762407+0000","last_became_active":"2023-10-11T09:49:50.998030+0000","last_became_peered":"2023-10-11T09:49:50.998030+0000","last_unstale":"2023-10-11T09:49:52.762407+0000","last_undegraded":"2023-10-11T09:49:52.762407+0000","last_fullsized":"2023-10-11T09:49:52.762407+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:05:46.041960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"104'4","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.976980+0000","last_change":"2023-10-11T09:49:50.514133+0000","last_active":"2023-10-11T09:49:51.976980+0000","last_peered":"2023-10-11T09:49:51.976980+0000","last_clean":"2023-10-11T09:49:51.976980+0000","last_became_active":"2023-10-11T09:49:50.513884+0000","last_became_peered":"2023-10-11T09:49:50.513884+0000","last_unstale":"2023-10-11T09:49:51.976980+0000","last_undegraded":"2023-10-11T09:49:51.976980+0000","last_fullsized":"2023-10-11T09:49:51.976980+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:21:50.568824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.862666+0000","last_change":"2023-10-11T09:49:50.985757+0000","last_active":"2023-10-11T09:49:52.862666+0000","last_peered":"2023-10-11T09:49:52.862666+0000","last_clean":"2023-10-11T09:49:52.862666+0000","last_became_active":"2023-10-11T09:49:50.985508+0000","last_became_peered":"2023-10-11T09:49:50.985508+0000","last_unstale":"2023-10-11T09:49:52.862666+0000","last_undegraded":"2023-10-11T09:49:52.862666+0000","last_fullsized":"2023-10-11T09:49:52.862666+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:58:29.902050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"104'1","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.980135+0000","last_change":"2023-10-11T09:49:50.526472+0000","last_active":"2023-10-11T09:49:51.980135+0000","last_peered":"2023-10-11T09:49:51.980135+0000","last_clean":"2023-10-11T09:49:51.980135+0000","last_became_active":"2023-10-11T09:49:50.526161+0000","last_became_peered":"2023-10-11T09:49:50.526161+0000","last_unstale":"2023-10-11T09:49:51.980135+0000","last_undegraded":"2023-10-11T09:49:51.980135+0000","last_fullsized":"2023-10-11T09:49:51.980135+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:44:58.855585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.860917+0000","last_change":"2023-10-11T09:49:50.999107+0000","last_active":"2023-10-11T09:49:52.860917+0000","last_peered":"2023-10-11T09:49:52.860917+0000","last_clean":"2023-10-11T09:49:52.860917+0000","last_became_active":"2023-10-11T09:49:50.996673+0000","last_became_peered":"2023-10-11T09:49:50.996673+0000","last_unstale":"2023-10-11T09:49:52.860917+0000","last_undegraded":"2023-10-11T09:49:52.860917+0000","last_fullsized":"2023-10-11T09:49:52.860917+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:09:32.742677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"104'2","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.976745+0000","last_change":"2023-10-11T09:49:50.520875+0000","last_active":"2023-10-11T09:49:51.976745+0000","last_peered":"2023-10-11T09:49:51.976745+0000","last_clean":"2023-10-11T09:49:51.976745+0000","last_became_active":"2023-10-11T09:49:50.519768+0000","last_became_peered":"2023-10-11T09:49:50.519768+0000","last_unstale":"2023-10-11T09:49:51.976745+0000","last_undegraded":"2023-10-11T09:49:51.976745+0000","last_fullsized":"2023-10-11T09:49:51.976745+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:05:20.960304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.980198+0000","last_change":"2023-10-11T09:49:50.994717+0000","last_active":"2023-10-11T09:49:51.980198+0000","last_peered":"2023-10-11T09:49:51.980198+0000","last_clean":"2023-10-11T09:49:51.980198+0000","last_became_active":"2023-10-11T09:49:50.994269+0000","last_became_peered":"2023-10-11T09:49:50.994269+0000","last_unstale":"2023-10-11T09:49:51.980198+0000","last_undegraded":"2023-10-11T09:49:51.980198+0000","last_fullsized":"2023-10-11T09:49:51.980198+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:18:09.722589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"104'2","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.976480+0000","last_change":"2023-10-11T09:49:50.508175+0000","last_active":"2023-10-11T09:49:51.976480+0000","last_peered":"2023-10-11T09:49:51.976480+0000","last_clean":"2023-10-11T09:49:51.976480+0000","last_became_active":"2023-10-11T09:49:50.506935+0000","last_became_peered":"2023-10-11T09:49:50.506935+0000","last_unstale":"2023-10-11T09:49:51.976480+0000","last_undegraded":"2023-10-11T09:49:51.976480+0000","last_fullsized":"2023-10-11T09:49:51.976480+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:44:01.403761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.861733+0000","last_change":"2023-10-11T09:49:50.985826+0000","last_active":"2023-10-11T09:49:52.861733+0000","last_peered":"2023-10-11T09:49:52.861733+0000","last_clean":"2023-10-11T09:49:52.861733+0000","last_became_active":"2023-10-11T09:49:50.985535+0000","last_became_peered":"2023-10-11T09:49:50.985535+0000","last_unstale":"2023-10-11T09:49:52.861733+0000","last_undegraded":"2023-10-11T09:49:52.861733+0000","last_fullsized":"2023-10-11T09:49:52.861733+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:58:29.902050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"104'1","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.860625+0000","last_change":"2023-10-11T09:49:50.520797+0000","last_active":"2023-10-11T09:49:52.860625+0000","last_peered":"2023-10-11T09:49:52.860625+0000","last_clean":"2023-10-11T09:49:52.860625+0000","last_became_active":"2023-10-11T09:49:50.520541+0000","last_became_peered":"2023-10-11T09:49:50.520541+0000","last_unstale":"2023-10-11T09:49:52.860625+0000","last_undegraded":"2023-10-11T09:49:52.860625+0000","last_fullsized":"2023-10-11T09:49:52.860625+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:03:49.929488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.980289+0000","last_change":"2023-10-11T09:49:50.987056+0000","last_active":"2023-10-11T09:49:51.980289+0000","last_peered":"2023-10-11T09:49:51.980289+0000","last_clean":"2023-10-11T09:49:51.980289+0000","last_became_active":"2023-10-11T09:49:50.986763+0000","last_became_peered":"2023-10-11T09:49:50.986763+0000","last_unstale":"2023-10-11T09:49:51.980289+0000","last_undegraded":"2023-10-11T09:49:51.980289+0000","last_fullsized":"2023-10-11T09:49:51.980289+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:34:10.165727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"104'1","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.976226+0000","last_change":"2023-10-11T09:49:50.523783+0000","last_active":"2023-10-11T09:49:51.976226+0000","last_peered":"2023-10-11T09:49:51.976226+0000","last_clean":"2023-10-11T09:49:51.976226+0000","last_became_active":"2023-10-11T09:49:50.523502+0000","last_became_peered":"2023-10-11T09:49:50.523502+0000","last_unstale":"2023-10-11T09:49:51.976226+0000","last_undegraded":"2023-10-11T09:49:51.976226+0000","last_fullsized":"2023-10-11T09:49:51.976226+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:15:32.883615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.861811+0000","last_change":"2023-10-11T09:49:50.988703+0000","last_active":"2023-10-11T09:49:52.861811+0000","last_peered":"2023-10-11T09:49:52.861811+0000","last_clean":"2023-10-11T09:49:52.861811+0000","last_became_active":"2023-10-11T09:49:50.988467+0000","last_became_peered":"2023-10-11T09:49:50.988467+0000","last_unstale":"2023-10-11T09:49:52.861811+0000","last_undegraded":"2023-10-11T09:49:52.861811+0000","last_fullsized":"2023-10-11T09:49:52.861811+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:27:04.170534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.862530+0000","last_change":"2023-10-11T09:49:50.513707+0000","last_active":"2023-10-11T09:49:52.862530+0000","last_peered":"2023-10-11T09:49:52.862530+0000","last_clean":"2023-10-11T09:49:52.862530+0000","last_became_active":"2023-10-11T09:49:50.513395+0000","last_became_peered":"2023-10-11T09:49:50.513395+0000","last_unstale":"2023-10-11T09:49:52.862530+0000","last_undegraded":"2023-10-11T09:49:52.862530+0000","last_fullsized":"2023-10-11T09:49:52.862530+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:05:45.026565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.979044+0000","last_change":"2023-10-11T09:49:50.992115+0000","last_active":"2023-10-11T09:49:51.979044+0000","last_peered":"2023-10-11T09:49:51.979044+0000","last_clean":"2023-10-11T09:49:51.979044+0000","last_became_active":"2023-10-11T09:49:50.991820+0000","last_became_peered":"2023-10-11T09:49:50.991820+0000","last_unstale":"2023-10-11T09:49:51.979044+0000","last_undegraded":"2023-10-11T09:49:51.979044+0000","last_fullsized":"2023-10-11T09:49:51.979044+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:27:04.170534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.860117+0000","last_change":"2023-10-11T09:49:50.522730+0000","last_active":"2023-10-11T09:49:52.860117+0000","last_peered":"2023-10-11T09:49:52.860117+0000","last_clean":"2023-10-11T09:49:52.860117+0000","last_became_active":"2023-10-11T09:49:50.522449+0000","last_became_peered":"2023-10-11T09:49:50.522449+0000","last_unstale":"2023-10-11T09:49:52.860117+0000","last_undegraded":"2023-10-11T09:49:52.860117+0000","last_fullsized":"2023-10-11T09:49:52.860117+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:44:48.334155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.976948+0000","last_change":"2023-10-11T09:49:50.994254+0000","last_active":"2023-10-11T09:49:51.976948+0000","last_peered":"2023-10-11T09:49:51.976948+0000","last_clean":"2023-10-11T09:49:51.976948+0000","last_became_active":"2023-10-11T09:49:50.994012+0000","last_became_peered":"2023-10-11T09:49:50.994012+0000","last_unstale":"2023-10-11T09:49:51.976948+0000","last_undegraded":"2023-10-11T09:49:51.976948+0000","last_fullsized":"2023-10-11T09:49:51.976948+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:44:01.377967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.861109+0000","last_change":"2023-10-11T09:49:50.522467+0000","last_active":"2023-10-11T09:49:52.861109+0000","last_peered":"2023-10-11T09:49:52.861109+0000","last_clean":"2023-10-11T09:49:52.861109+0000","last_became_active":"2023-10-11T09:49:50.522020+0000","last_became_peered":"2023-10-11T09:49:50.522020+0000","last_unstale":"2023-10-11T09:49:52.861109+0000","last_undegraded":"2023-10-11T09:49:52.861109+0000","last_fullsized":"2023-10-11T09:49:52.861109+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:28:31.717133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.980155+0000","last_change":"2023-10-11T09:49:50.996976+0000","last_active":"2023-10-11T09:49:51.980155+0000","last_peered":"2023-10-11T09:49:51.980155+0000","last_clean":"2023-10-11T09:49:51.980155+0000","last_became_active":"2023-10-11T09:49:50.996379+0000","last_became_peered":"2023-10-11T09:49:50.996379+0000","last_unstale":"2023-10-11T09:49:51.980155+0000","last_undegraded":"2023-10-11T09:49:51.980155+0000","last_fullsized":"2023-10-11T09:49:51.980155+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:52:13.301627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.862855+0000","last_change":"2023-10-11T09:49:50.517731+0000","last_active":"2023-10-11T09:49:52.862855+0000","last_peered":"2023-10-11T09:49:52.862855+0000","last_clean":"2023-10-11T09:49:52.862855+0000","last_became_active":"2023-10-11T09:49:50.512755+0000","last_became_peered":"2023-10-11T09:49:50.512755+0000","last_unstale":"2023-10-11T09:49:52.862855+0000","last_undegraded":"2023-10-11T09:49:52.862855+0000","last_fullsized":"2023-10-11T09:49:52.862855+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:37:07.353631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.861540+0000","last_change":"2023-10-11T09:49:51.000726+0000","last_active":"2023-10-11T09:49:52.861540+0000","last_peered":"2023-10-11T09:49:52.861540+0000","last_clean":"2023-10-11T09:49:52.861540+0000","last_became_active":"2023-10-11T09:49:51.000456+0000","last_became_peered":"2023-10-11T09:49:51.000456+0000","last_unstale":"2023-10-11T09:49:52.861540+0000","last_undegraded":"2023-10-11T09:49:52.861540+0000","last_fullsized":"2023-10-11T09:49:52.861540+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:34:36.672892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"104'3","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.980561+0000","last_change":"2023-10-11T09:49:50.512385+0000","last_active":"2023-10-11T09:49:51.980561+0000","last_peered":"2023-10-11T09:49:51.980561+0000","last_clean":"2023-10-11T09:49:51.980561+0000","last_became_active":"2023-10-11T09:49:50.511261+0000","last_became_peered":"2023-10-11T09:49:50.511261+0000","last_unstale":"2023-10-11T09:49:51.980561+0000","last_undegraded":"2023-10-11T09:49:51.980561+0000","last_fullsized":"2023-10-11T09:49:51.980561+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:15:44.292008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.980425+0000","last_change":"2023-10-11T09:49:50.995456+0000","last_active":"2023-10-11T09:49:51.980425+0000","last_peered":"2023-10-11T09:49:51.980425+0000","last_clean":"2023-10-11T09:49:51.980425+0000","last_became_active":"2023-10-11T09:49:50.994836+0000","last_became_peered":"2023-10-11T09:49:50.994836+0000","last_unstale":"2023-10-11T09:49:51.980425+0000","last_undegraded":"2023-10-11T09:49:51.980425+0000","last_fullsized":"2023-10-11T09:49:51.980425+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:21:03.166998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"104'6","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.979823+0000","last_change":"2023-10-11T09:49:50.512003+0000","last_active":"2023-10-11T09:49:51.979823+0000","last_peered":"2023-10-11T09:49:51.979823+0000","last_clean":"2023-10-11T09:49:51.979823+0000","last_became_active":"2023-10-11T09:49:50.511514+0000","last_became_peered":"2023-10-11T09:49:50.511514+0000","last_unstale":"2023-10-11T09:49:51.979823+0000","last_undegraded":"2023-10-11T09:49:51.979823+0000","last_fullsized":"2023-10-11T09:49:51.979823+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:32:20.720114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.861999+0000","last_change":"2023-10-11T09:49:50.981207+0000","last_active":"2023-10-11T09:49:52.861999+0000","last_peered":"2023-10-11T09:49:52.861999+0000","last_clean":"2023-10-11T09:49:52.861999+0000","last_became_active":"2023-10-11T09:49:50.980894+0000","last_became_peered":"2023-10-11T09:49:50.980894+0000","last_unstale":"2023-10-11T09:49:52.861999+0000","last_undegraded":"2023-10-11T09:49:52.861999+0000","last_fullsized":"2023-10-11T09:49:52.861999+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:43:24.307597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"104'2","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.762222+0000","last_change":"2023-10-11T09:49:50.513870+0000","last_active":"2023-10-11T09:49:52.762222+0000","last_peered":"2023-10-11T09:49:52.762222+0000","last_clean":"2023-10-11T09:49:52.762222+0000","last_became_active":"2023-10-11T09:49:50.513624+0000","last_became_peered":"2023-10-11T09:49:50.513624+0000","last_unstale":"2023-10-11T09:49:52.762222+0000","last_undegraded":"2023-10-11T09:49:52.762222+0000","last_fullsized":"2023-10-11T09:49:52.762222+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:53:29.159730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.860484+0000","last_change":"2023-10-11T09:49:50.989100+0000","last_active":"2023-10-11T09:49:52.860484+0000","last_peered":"2023-10-11T09:49:52.860484+0000","last_clean":"2023-10-11T09:49:52.860484+0000","last_became_active":"2023-10-11T09:49:50.988861+0000","last_became_peered":"2023-10-11T09:49:50.988861+0000","last_unstale":"2023-10-11T09:49:52.860484+0000","last_undegraded":"2023-10-11T09:49:52.860484+0000","last_fullsized":"2023-10-11T09:49:52.860484+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:08:47.044323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.762118+0000","last_change":"2023-10-11T09:49:50.523983+0000","last_active":"2023-10-11T09:49:52.762118+0000","last_peered":"2023-10-11T09:49:52.762118+0000","last_clean":"2023-10-11T09:49:52.762118+0000","last_became_active":"2023-10-11T09:49:50.523590+0000","last_became_peered":"2023-10-11T09:49:50.523590+0000","last_unstale":"2023-10-11T09:49:52.762118+0000","last_undegraded":"2023-10-11T09:49:52.762118+0000","last_fullsized":"2023-10-11T09:49:52.762118+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:59:35.713273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.979034+0000","last_change":"2023-10-11T09:49:50.991115+0000","last_active":"2023-10-11T09:49:51.979034+0000","last_peered":"2023-10-11T09:49:51.979034+0000","last_clean":"2023-10-11T09:49:51.979034+0000","last_became_active":"2023-10-11T09:49:50.990841+0000","last_became_peered":"2023-10-11T09:49:50.990841+0000","last_unstale":"2023-10-11T09:49:51.979034+0000","last_undegraded":"2023-10-11T09:49:51.979034+0000","last_fullsized":"2023-10-11T09:49:51.979034+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:21:03.166998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.979242+0000","last_change":"2023-10-11T09:49:50.515961+0000","last_active":"2023-10-11T09:49:51.979242+0000","last_peered":"2023-10-11T09:49:51.979242+0000","last_clean":"2023-10-11T09:49:51.979242+0000","last_became_active":"2023-10-11T09:49:50.515600+0000","last_became_peered":"2023-10-11T09:49:50.515600+0000","last_unstale":"2023-10-11T09:49:51.979242+0000","last_undegraded":"2023-10-11T09:49:51.979242+0000","last_fullsized":"2023-10-11T09:49:51.979242+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:15:44.292008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.761609+0000","last_change":"2023-10-11T09:49:50.997959+0000","last_active":"2023-10-11T09:49:52.761609+0000","last_peered":"2023-10-11T09:49:52.761609+0000","last_clean":"2023-10-11T09:49:52.761609+0000","last_became_active":"2023-10-11T09:49:50.997573+0000","last_became_peered":"2023-10-11T09:49:50.997573+0000","last_unstale":"2023-10-11T09:49:52.761609+0000","last_undegraded":"2023-10-11T09:49:52.761609+0000","last_fullsized":"2023-10-11T09:49:52.761609+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:04:56.068527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"104'8","reported_seq":19,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.860114+0000","last_change":"2023-10-11T09:49:50.520314+0000","last_active":"2023-10-11T09:49:52.860114+0000","last_peered":"2023-10-11T09:49:52.860114+0000","last_clean":"2023-10-11T09:49:52.860114+0000","last_became_active":"2023-10-11T09:49:50.519985+0000","last_became_peered":"2023-10-11T09:49:50.519985+0000","last_unstale":"2023-10-11T09:49:52.860114+0000","last_undegraded":"2023-10-11T09:49:52.860114+0000","last_fullsized":"2023-10-11T09:49:52.860114+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:04:55.053132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.977025+0000","last_change":"2023-10-11T09:49:50.989554+0000","last_active":"2023-10-11T09:49:51.977025+0000","last_peered":"2023-10-11T09:49:51.977025+0000","last_clean":"2023-10-11T09:49:51.977025+0000","last_became_active":"2023-10-11T09:49:50.989037+0000","last_became_peered":"2023-10-11T09:49:50.989037+0000","last_unstale":"2023-10-11T09:49:51.977025+0000","last_undegraded":"2023-10-11T09:49:51.977025+0000","last_fullsized":"2023-10-11T09:49:51.977025+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:05:09.217513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"104'2","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.862418+0000","last_change":"2023-10-11T09:49:50.509971+0000","last_active":"2023-10-11T09:49:52.862418+0000","last_peered":"2023-10-11T09:49:52.862418+0000","last_clean":"2023-10-11T09:49:52.862418+0000","last_became_active":"2023-10-11T09:49:50.509678+0000","last_became_peered":"2023-10-11T09:49:50.509678+0000","last_unstale":"2023-10-11T09:49:52.862418+0000","last_undegraded":"2023-10-11T09:49:52.862418+0000","last_fullsized":"2023-10-11T09:49:52.862418+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:32:20.720114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.860994+0000","last_change":"2023-10-11T09:49:50.991735+0000","last_active":"2023-10-11T09:49:52.860994+0000","last_peered":"2023-10-11T09:49:52.860994+0000","last_clean":"2023-10-11T09:49:52.860994+0000","last_became_active":"2023-10-11T09:49:50.991493+0000","last_became_peered":"2023-10-11T09:49:50.991493+0000","last_unstale":"2023-10-11T09:49:52.860994+0000","last_undegraded":"2023-10-11T09:49:52.860994+0000","last_fullsized":"2023-10-11T09:49:52.860994+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:31:14.207396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.977049+0000","last_change":"2023-10-11T09:49:50.517413+0000","last_active":"2023-10-11T09:49:51.977049+0000","last_peered":"2023-10-11T09:49:51.977049+0000","last_clean":"2023-10-11T09:49:51.977049+0000","last_became_active":"2023-10-11T09:49:50.517154+0000","last_became_peered":"2023-10-11T09:49:50.517154+0000","last_unstale":"2023-10-11T09:49:51.977049+0000","last_undegraded":"2023-10-11T09:49:51.977049+0000","last_fullsized":"2023-10-11T09:49:51.977049+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:18:03.093467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.860803+0000","last_change":"2023-10-11T09:49:50.986853+0000","last_active":"2023-10-11T09:49:52.860803+0000","last_peered":"2023-10-11T09:49:52.860803+0000","last_clean":"2023-10-11T09:49:52.860803+0000","last_became_active":"2023-10-11T09:49:50.986594+0000","last_became_peered":"2023-10-11T09:49:50.986594+0000","last_unstale":"2023-10-11T09:49:52.860803+0000","last_undegraded":"2023-10-11T09:49:52.860803+0000","last_fullsized":"2023-10-11T09:49:52.860803+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:34:36.672892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.976767+0000","last_change":"2023-10-11T09:49:50.508629+0000","last_active":"2023-10-11T09:49:51.976767+0000","last_peered":"2023-10-11T09:49:51.976767+0000","last_clean":"2023-10-11T09:49:51.976767+0000","last_became_active":"2023-10-11T09:49:50.508382+0000","last_became_peered":"2023-10-11T09:49:50.508382+0000","last_unstale":"2023-10-11T09:49:51.976767+0000","last_undegraded":"2023-10-11T09:49:51.976767+0000","last_fullsized":"2023-10-11T09:49:51.976767+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:02:08.300241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.861751+0000","last_change":"2023-10-11T09:49:50.990194+0000","last_active":"2023-10-11T09:49:52.861751+0000","last_peered":"2023-10-11T09:49:52.861751+0000","last_clean":"2023-10-11T09:49:52.861751+0000","last_became_active":"2023-10-11T09:49:50.988413+0000","last_became_peered":"2023-10-11T09:49:50.988413+0000","last_unstale":"2023-10-11T09:49:52.861751+0000","last_undegraded":"2023-10-11T09:49:52.861751+0000","last_fullsized":"2023-10-11T09:49:52.861751+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:30:56.971861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.861754+0000","last_change":"2023-10-11T09:49:50.515303+0000","last_active":"2023-10-11T09:49:52.861754+0000","last_peered":"2023-10-11T09:49:52.861754+0000","last_clean":"2023-10-11T09:49:52.861754+0000","last_became_active":"2023-10-11T09:49:50.514991+0000","last_became_peered":"2023-10-11T09:49:50.514991+0000","last_unstale":"2023-10-11T09:49:52.861754+0000","last_undegraded":"2023-10-11T09:49:52.861754+0000","last_fullsized":"2023-10-11T09:49:52.861754+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:43:53.637800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.761937+0000","last_change":"2023-10-11T09:49:50.998446+0000","last_active":"2023-10-11T09:49:52.761937+0000","last_peered":"2023-10-11T09:49:52.761937+0000","last_clean":"2023-10-11T09:49:52.761937+0000","last_became_active":"2023-10-11T09:49:50.996224+0000","last_became_peered":"2023-10-11T09:49:50.996224+0000","last_unstale":"2023-10-11T09:49:52.761937+0000","last_undegraded":"2023-10-11T09:49:52.761937+0000","last_fullsized":"2023-10-11T09:49:52.761937+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:55:36.981091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.976366+0000","last_change":"2023-10-11T09:49:50.520625+0000","last_active":"2023-10-11T09:49:51.976366+0000","last_peered":"2023-10-11T09:49:51.976366+0000","last_clean":"2023-10-11T09:49:51.976366+0000","last_became_active":"2023-10-11T09:49:50.520251+0000","last_became_peered":"2023-10-11T09:49:50.520251+0000","last_unstale":"2023-10-11T09:49:51.976366+0000","last_undegraded":"2023-10-11T09:49:51.976366+0000","last_fullsized":"2023-10-11T09:49:51.976366+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:59:15.204962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.979386+0000","last_change":"2023-10-11T09:49:50.990452+0000","last_active":"2023-10-11T09:49:51.979386+0000","last_peered":"2023-10-11T09:49:51.979386+0000","last_clean":"2023-10-11T09:49:51.979386+0000","last_became_active":"2023-10-11T09:49:50.990168+0000","last_became_peered":"2023-10-11T09:49:50.990168+0000","last_unstale":"2023-10-11T09:49:51.979386+0000","last_undegraded":"2023-10-11T09:49:51.979386+0000","last_fullsized":"2023-10-11T09:49:51.979386+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:15:13.595478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.861231+0000","last_change":"2023-10-11T09:49:50.524094+0000","last_active":"2023-10-11T09:49:52.861231+0000","last_peered":"2023-10-11T09:49:52.861231+0000","last_clean":"2023-10-11T09:49:52.861231+0000","last_became_active":"2023-10-11T09:49:50.523738+0000","last_became_peered":"2023-10-11T09:49:50.523738+0000","last_unstale":"2023-10-11T09:49:52.861231+0000","last_undegraded":"2023-10-11T09:49:52.861231+0000","last_fullsized":"2023-10-11T09:49:52.861231+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:11:41.951896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.761593+0000","last_change":"2023-10-11T09:49:50.997443+0000","last_active":"2023-10-11T09:49:52.761593+0000","last_peered":"2023-10-11T09:49:52.761593+0000","last_clean":"2023-10-11T09:49:52.761593+0000","last_became_active":"2023-10-11T09:49:50.997179+0000","last_became_peered":"2023-10-11T09:49:50.997179+0000","last_unstale":"2023-10-11T09:49:52.761593+0000","last_undegraded":"2023-10-11T09:49:52.761593+0000","last_fullsized":"2023-10-11T09:49:52.761593+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:44:49.349550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.862331+0000","last_change":"2023-10-11T09:49:50.523257+0000","last_active":"2023-10-11T09:49:52.862331+0000","last_peered":"2023-10-11T09:49:52.862331+0000","last_clean":"2023-10-11T09:49:52.862331+0000","last_became_active":"2023-10-11T09:49:50.522563+0000","last_became_peered":"2023-10-11T09:49:50.522563+0000","last_unstale":"2023-10-11T09:49:52.862331+0000","last_undegraded":"2023-10-11T09:49:52.862331+0000","last_fullsized":"2023-10-11T09:49:52.862331+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:06:33.179642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.861114+0000","last_change":"2023-10-11T09:49:50.995440+0000","last_active":"2023-10-11T09:49:52.861114+0000","last_peered":"2023-10-11T09:49:52.861114+0000","last_clean":"2023-10-11T09:49:52.861114+0000","last_became_active":"2023-10-11T09:49:50.995157+0000","last_became_peered":"2023-10-11T09:49:50.995157+0000","last_unstale":"2023-10-11T09:49:52.861114+0000","last_undegraded":"2023-10-11T09:49:52.861114+0000","last_fullsized":"2023-10-11T09:49:52.861114+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:08:47.044323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.761792+0000","last_change":"2023-10-11T09:49:50.516673+0000","last_active":"2023-10-11T09:49:52.761792+0000","last_peered":"2023-10-11T09:49:52.761792+0000","last_clean":"2023-10-11T09:49:52.761792+0000","last_became_active":"2023-10-11T09:49:50.516247+0000","last_became_peered":"2023-10-11T09:49:50.516247+0000","last_unstale":"2023-10-11T09:49:52.761792+0000","last_undegraded":"2023-10-11T09:49:52.761792+0000","last_fullsized":"2023-10-11T09:49:52.761792+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:14:35.600428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.761908+0000","last_change":"2023-10-11T09:49:50.993829+0000","last_active":"2023-10-11T09:49:52.761908+0000","last_peered":"2023-10-11T09:49:52.761908+0000","last_clean":"2023-10-11T09:49:52.761908+0000","last_became_active":"2023-10-11T09:49:50.993547+0000","last_became_peered":"2023-10-11T09:49:50.993547+0000","last_unstale":"2023-10-11T09:49:52.761908+0000","last_undegraded":"2023-10-11T09:49:52.761908+0000","last_fullsized":"2023-10-11T09:49:52.761908+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:44:00.902594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"104'2","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.976220+0000","last_change":"2023-10-11T09:49:50.516828+0000","last_active":"2023-10-11T09:49:51.976220+0000","last_peered":"2023-10-11T09:49:51.976220+0000","last_clean":"2023-10-11T09:49:51.976220+0000","last_became_active":"2023-10-11T09:49:50.516569+0000","last_became_peered":"2023-10-11T09:49:50.516569+0000","last_unstale":"2023-10-11T09:49:51.976220+0000","last_undegraded":"2023-10-11T09:49:51.976220+0000","last_fullsized":"2023-10-11T09:49:51.976220+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:02:50.778526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":77,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-11T09:49:50.978451+0000","last_change":"2023-10-11T09:40:36.151839+0000","last_active":"2023-10-11T09:49:50.978451+0000","last_peered":"2023-10-11T09:49:50.978451+0000","last_clean":"2023-10-11T09:49:50.978451+0000","last_became_active":"2023-10-11T09:40:36.151179+0000","last_became_peered":"2023-10-11T09:40:36.151179+0000","last_unstale":"2023-10-11T09:49:50.978451+0000","last_undegraded":"2023-10-11T09:49:50.978451+0000","last_fullsized":"2023-10-11T09:49:50.978451+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.762118+0000","last_change":"2023-10-11T09:49:51.000188+0000","last_active":"2023-10-11T09:49:52.762118+0000","last_peered":"2023-10-11T09:49:52.762118+0000","last_clean":"2023-10-11T09:49:52.762118+0000","last_became_active":"2023-10-11T09:49:50.999952+0000","last_became_peered":"2023-10-11T09:49:50.999952+0000","last_unstale":"2023-10-11T09:49:52.762118+0000","last_undegraded":"2023-10-11T09:49:52.762118+0000","last_fullsized":"2023-10-11T09:49:52.762118+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:00:58.856002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.861770+0000","last_change":"2023-10-11T09:49:50.524923+0000","last_active":"2023-10-11T09:49:52.861770+0000","last_peered":"2023-10-11T09:49:52.861770+0000","last_clean":"2023-10-11T09:49:52.861770+0000","last_became_active":"2023-10-11T09:49:50.524643+0000","last_became_peered":"2023-10-11T09:49:50.524643+0000","last_unstale":"2023-10-11T09:49:52.861770+0000","last_undegraded":"2023-10-11T09:49:52.861770+0000","last_fullsized":"2023-10-11T09:49:52.861770+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:37:36.785205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":82,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-11T09:49:50.974920+0000","last_change":"2023-10-11T09:40:36.150224+0000","last_active":"2023-10-11T09:49:50.974920+0000","last_peered":"2023-10-11T09:49:50.974920+0000","last_clean":"2023-10-11T09:49:50.974920+0000","last_became_active":"2023-10-11T09:40:36.149919+0000","last_became_peered":"2023-10-11T09:40:36.149919+0000","last_unstale":"2023-10-11T09:49:50.974920+0000","last_undegraded":"2023-10-11T09:49:50.974920+0000","last_fullsized":"2023-10-11T09:49:50.974920+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.761519+0000","last_change":"2023-10-11T09:49:50.994027+0000","last_active":"2023-10-11T09:49:52.761519+0000","last_peered":"2023-10-11T09:49:52.761519+0000","last_clean":"2023-10-11T09:49:52.761519+0000","last_became_active":"2023-10-11T09:49:50.993781+0000","last_became_peered":"2023-10-11T09:49:50.993781+0000","last_unstale":"2023-10-11T09:49:52.761519+0000","last_undegraded":"2023-10-11T09:49:52.761519+0000","last_fullsized":"2023-10-11T09:49:52.761519+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:14:59.605167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.862078+0000","last_change":"2023-10-11T09:49:50.522433+0000","last_active":"2023-10-11T09:49:52.862078+0000","last_peered":"2023-10-11T09:49:52.862078+0000","last_clean":"2023-10-11T09:49:52.862078+0000","last_became_active":"2023-10-11T09:49:50.522041+0000","last_became_peered":"2023-10-11T09:49:50.522041+0000","last_unstale":"2023-10-11T09:49:52.862078+0000","last_undegraded":"2023-10-11T09:49:52.862078+0000","last_fullsized":"2023-10-11T09:49:52.862078+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:00:54.423451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":100,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-11T09:49:50.973102+0000","last_change":"2023-10-11T09:40:36.162174+0000","last_active":"2023-10-11T09:49:50.973102+0000","last_peered":"2023-10-11T09:49:50.973102+0000","last_clean":"2023-10-11T09:49:50.973102+0000","last_became_active":"2023-10-11T09:40:36.161710+0000","last_became_peered":"2023-10-11T09:40:36.161710+0000","last_unstale":"2023-10-11T09:49:50.973102+0000","last_undegraded":"2023-10-11T09:49:50.973102+0000","last_fullsized":"2023-10-11T09:49:50.973102+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:57:03.630878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.861017+0000","last_change":"2023-10-11T09:49:50.991770+0000","last_active":"2023-10-11T09:49:52.861017+0000","last_peered":"2023-10-11T09:49:52.861017+0000","last_clean":"2023-10-11T09:49:52.861017+0000","last_became_active":"2023-10-11T09:49:50.991408+0000","last_became_peered":"2023-10-11T09:49:50.991408+0000","last_unstale":"2023-10-11T09:49:52.861017+0000","last_undegraded":"2023-10-11T09:49:52.861017+0000","last_fullsized":"2023-10-11T09:49:52.861017+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:31:14.207396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.976217+0000","last_change":"2023-10-11T09:49:50.524013+0000","last_active":"2023-10-11T09:49:51.976217+0000","last_peered":"2023-10-11T09:49:51.976217+0000","last_clean":"2023-10-11T09:49:51.976217+0000","last_became_active":"2023-10-11T09:49:50.523692+0000","last_became_peered":"2023-10-11T09:49:50.523692+0000","last_unstale":"2023-10-11T09:49:51.976217+0000","last_undegraded":"2023-10-11T09:49:51.976217+0000","last_fullsized":"2023-10-11T09:49:51.976217+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T21:34:46.814409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":82,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-11T09:49:50.976184+0000","last_change":"2023-10-11T09:40:36.160620+0000","last_active":"2023-10-11T09:49:50.976184+0000","last_peered":"2023-10-11T09:49:50.976184+0000","last_clean":"2023-10-11T09:49:50.976184+0000","last_became_active":"2023-10-11T09:40:36.160171+0000","last_became_peered":"2023-10-11T09:40:36.160171+0000","last_unstale":"2023-10-11T09:49:50.976184+0000","last_undegraded":"2023-10-11T09:49:50.976184+0000","last_fullsized":"2023-10-11T09:49:50.976184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.862297+0000","last_change":"2023-10-11T09:49:50.999791+0000","last_active":"2023-10-11T09:49:52.862297+0000","last_peered":"2023-10-11T09:49:52.862297+0000","last_clean":"2023-10-11T09:49:52.862297+0000","last_became_active":"2023-10-11T09:49:50.999488+0000","last_became_peered":"2023-10-11T09:49:50.999488+0000","last_unstale":"2023-10-11T09:49:52.862297+0000","last_undegraded":"2023-10-11T09:49:52.862297+0000","last_fullsized":"2023-10-11T09:49:52.862297+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:44:00.902594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"104'2","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.979613+0000","last_change":"2023-10-11T09:49:50.516923+0000","last_active":"2023-10-11T09:49:51.979613+0000","last_peered":"2023-10-11T09:49:51.979613+0000","last_clean":"2023-10-11T09:49:51.979613+0000","last_became_active":"2023-10-11T09:49:50.516630+0000","last_became_peered":"2023-10-11T09:49:50.516630+0000","last_unstale":"2023-10-11T09:49:51.979613+0000","last_undegraded":"2023-10-11T09:49:51.979613+0000","last_fullsized":"2023-10-11T09:49:51.979613+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:44:58.855585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":82,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-11T09:49:50.973127+0000","last_change":"2023-10-11T09:40:36.151710+0000","last_active":"2023-10-11T09:49:50.973127+0000","last_peered":"2023-10-11T09:49:50.973127+0000","last_clean":"2023-10-11T09:49:50.973127+0000","last_became_active":"2023-10-11T09:40:36.151219+0000","last_became_peered":"2023-10-11T09:40:36.151219+0000","last_unstale":"2023-10-11T09:49:50.973127+0000","last_undegraded":"2023-10-11T09:49:50.973127+0000","last_fullsized":"2023-10-11T09:49:50.973127+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.862125+0000","last_change":"2023-10-11T09:49:50.987205+0000","last_active":"2023-10-11T09:49:52.862125+0000","last_peered":"2023-10-11T09:49:52.862125+0000","last_clean":"2023-10-11T09:49:52.862125+0000","last_became_active":"2023-10-11T09:49:50.986839+0000","last_became_peered":"2023-10-11T09:49:50.986839+0000","last_unstale":"2023-10-11T09:49:52.862125+0000","last_undegraded":"2023-10-11T09:49:52.862125+0000","last_fullsized":"2023-10-11T09:49:52.862125+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:05:08.173043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"104'1","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.861906+0000","last_change":"2023-10-11T09:49:50.512199+0000","last_active":"2023-10-11T09:49:52.861906+0000","last_peered":"2023-10-11T09:49:52.861906+0000","last_clean":"2023-10-11T09:49:52.861906+0000","last_became_active":"2023-10-11T09:49:50.511867+0000","last_became_peered":"2023-10-11T09:49:50.511867+0000","last_unstale":"2023-10-11T09:49:52.861906+0000","last_undegraded":"2023-10-11T09:49:52.861906+0000","last_fullsized":"2023-10-11T09:49:52.861906+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:27:42.811643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":82,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-11T09:49:50.978606+0000","last_change":"2023-10-11T09:40:36.216248+0000","last_active":"2023-10-11T09:49:50.978606+0000","last_peered":"2023-10-11T09:49:50.978606+0000","last_clean":"2023-10-11T09:49:50.978606+0000","last_became_active":"2023-10-11T09:40:36.215890+0000","last_became_peered":"2023-10-11T09:40:36.215890+0000","last_unstale":"2023-10-11T09:49:50.978606+0000","last_undegraded":"2023-10-11T09:49:50.978606+0000","last_fullsized":"2023-10-11T09:49:50.978606+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.980483+0000","last_change":"2023-10-11T09:49:50.995121+0000","last_active":"2023-10-11T09:49:51.980483+0000","last_peered":"2023-10-11T09:49:51.980483+0000","last_clean":"2023-10-11T09:49:51.980483+0000","last_became_active":"2023-10-11T09:49:50.994168+0000","last_became_peered":"2023-10-11T09:49:50.994168+0000","last_unstale":"2023-10-11T09:49:51.980483+0000","last_undegraded":"2023-10-11T09:49:51.980483+0000","last_fullsized":"2023-10-11T09:49:51.980483+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:52:13.301627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.976526+0000","last_change":"2023-10-11T09:49:50.509104+0000","last_active":"2023-10-11T09:49:51.976526+0000","last_peered":"2023-10-11T09:49:51.976526+0000","last_clean":"2023-10-11T09:49:51.976526+0000","last_became_active":"2023-10-11T09:49:50.508835+0000","last_became_peered":"2023-10-11T09:49:50.508835+0000","last_unstale":"2023-10-11T09:49:51.976526+0000","last_undegraded":"2023-10-11T09:49:51.976526+0000","last_fullsized":"2023-10-11T09:49:51.976526+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:34:09.150332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":100,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-11T09:49:50.973570+0000","last_change":"2023-10-11T09:40:36.149739+0000","last_active":"2023-10-11T09:49:50.973570+0000","last_peered":"2023-10-11T09:49:50.973570+0000","last_clean":"2023-10-11T09:49:50.973570+0000","last_became_active":"2023-10-11T09:40:36.149419+0000","last_became_peered":"2023-10-11T09:40:36.149419+0000","last_unstale":"2023-10-11T09:49:50.973570+0000","last_undegraded":"2023-10-11T09:49:50.973570+0000","last_fullsized":"2023-10-11T09:49:50.973570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:54:50.640362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.762380+0000","last_change":"2023-10-11T09:49:50.994097+0000","last_active":"2023-10-11T09:49:52.762380+0000","last_peered":"2023-10-11T09:49:52.762380+0000","last_clean":"2023-10-11T09:49:52.762380+0000","last_became_active":"2023-10-11T09:49:50.993772+0000","last_became_peered":"2023-10-11T09:49:50.993772+0000","last_unstale":"2023-10-11T09:49:52.762380+0000","last_undegraded":"2023-10-11T09:49:52.762380+0000","last_fullsized":"2023-10-11T09:49:52.762380+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:12:29.979310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"104'1","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.980376+0000","last_change":"2023-10-11T09:49:50.524655+0000","last_active":"2023-10-11T09:49:51.980376+0000","last_peered":"2023-10-11T09:49:51.980376+0000","last_clean":"2023-10-11T09:49:51.980376+0000","last_became_active":"2023-10-11T09:49:50.524340+0000","last_became_peered":"2023-10-11T09:49:50.524340+0000","last_unstale":"2023-10-11T09:49:51.980376+0000","last_undegraded":"2023-10-11T09:49:51.980376+0000","last_fullsized":"2023-10-11T09:49:51.980376+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:54:27.367170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":82,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-11T09:49:50.971387+0000","last_change":"2023-10-11T09:40:36.148773+0000","last_active":"2023-10-11T09:49:50.971387+0000","last_peered":"2023-10-11T09:49:50.971387+0000","last_clean":"2023-10-11T09:49:50.971387+0000","last_became_active":"2023-10-11T09:40:36.148489+0000","last_became_peered":"2023-10-11T09:40:36.148489+0000","last_unstale":"2023-10-11T09:49:50.971387+0000","last_undegraded":"2023-10-11T09:49:50.971387+0000","last_fullsized":"2023-10-11T09:49:50.971387+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":253,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-11T09:49:50.973719+0000","last_change":"2023-10-11T09:40:29.874372+0000","last_active":"2023-10-11T09:49:50.973719+0000","last_peered":"2023-10-11T09:49:50.973719+0000","last_clean":"2023-10-11T09:49:50.973719+0000","last_became_active":"2023-10-11T09:40:29.873621+0000","last_became_peered":"2023-10-11T09:40:29.873621+0000","last_unstale":"2023-10-11T09:49:50.973719+0000","last_undegraded":"2023-10-11T09:49:50.973719+0000","last_fullsized":"2023-10-11T09:49:50.973719+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_clean_scrub_stamp":"2023-10-11T09:40:28.742276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:22:07.985311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574965,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32900,"kb_used_data":1852,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338940,"statfs":{"total":96636764160,"available":96603074560,"internally_reserved":0,"allocated":1896448,"data_stored":1213864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:49:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.0049999999999999,"15min":1.0980000000000001},"min":{"1min":0.68600000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.4770000000000001,"5min":1.7390000000000001,"15min":1.903},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.131,"5min":1.01,"15min":1.0980000000000001},"min":{"1min":0.86399999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3899999999999999,"5min":1.5760000000000001,"15min":1.696},"last":1.554}]},{"osd":1,"last update":"Wed Oct 11 09:49:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.079,"15min":1.1140000000000001},"min":{"1min":0.84199999999999997,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.4370000000000001,"5min":1.645,"15min":2.25},"last":1.466},{"interface":"front","average":{"1min":1.054,"5min":1.016,"15min":1.0960000000000001},"min":{"1min":0.80400000000000005,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.5369999999999999,"5min":1.587,"15min":1.696},"last":1.115}]},{"osd":2,"last update":"Wed Oct 11 09:49:49 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.0800000000000001,"15min":1.109},"min":{"1min":0.68500000000000005,"5min":0.59499999999999997,"15min":0.58999999999999997},"max":{"1min":1.466,"5min":1.6339999999999999,"15min":1.873},"last":1.486},{"interface":"front","average":{"1min":1.206,"5min":1.1499999999999999,"15min":1.149},"min":{"1min":0.73599999999999999,"5min":0.66400000000000003,"15min":0.65500000000000003},"max":{"1min":1.569,"5min":1.78,"15min":1.78},"last":1.335}]},{"osd":3,"last update":"Wed Oct 11 09:49:49 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.159,"15min":1.2230000000000001},"min":{"1min":0.80000000000000004,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.53},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.246,"15min":1.2809999999999999},"min":{"1min":0.97199999999999998,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.8089999999999999,"5min":1.8720000000000001,"15min":2.1579999999999999},"last":1.514}]},{"osd":4,"last update":"Wed Oct 11 09:49:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0569999999999999,"15min":1.1379999999999999},"min":{"1min":0.752,"5min":0.64900000000000002,"15min":0.53300000000000003},"max":{"1min":1.4219999999999999,"5min":1.661,"15min":2.2010000000000001},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.202,"15min":1.2370000000000001},"min":{"1min":0.88200000000000001,"5min":0.67200000000000004,"15min":0.63500000000000001},"max":{"1min":1.768,"5min":1.768,"15min":1.9339999999999999},"last":1.4330000000000001}]},{"osd":5,"last update":"Wed Oct 11 09:49:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.28,"15min":1.2829999999999999},"min":{"1min":0.95699999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.744},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.2070000000000001,"15min":1.302},"min":{"1min":0.98199999999999998,"5min":0.78300000000000003,"15min":0.65900000000000003},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":3.0750000000000002},"last":1.196}]},{"osd":6,"last update":"Wed Oct 11 09:49:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.345,"15min":1.411},"min":{"1min":1.2010000000000001,"5min":0.77000000000000002,"15min":0.70999999999999996},"max":{"1min":1.835,"5min":1.919,"15min":3.137},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.417,"5min":1.3560000000000001,"15min":1.395},"min":{"1min":1.032,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.784,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.5760000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574964,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32308,"kb_used_data":1260,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339532,"statfs":{"total":96636764160,"available":96603680768,"internally_reserved":0,"allocated":1290240,"data_stored":617383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:49:40 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.085,"15min":1.028},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.6200000000000001,"5min":7.0730000000000004,"15min":7.0730000000000004},"last":1.3240000000000001},{"interface":"front","average":{"1min":0.998,"5min":1.165,"15min":1.079},"min":{"1min":0.63100000000000001,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.409,"5min":8.8450000000000006,"15min":8.8450000000000006},"last":1.081}]},{"osd":1,"last update":"Wed Oct 11 09:49:40 2023","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.3009999999999999,"15min":1.2290000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.4370000000000001,"5min":15.523,"15min":15.523},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.141,"5min":1.2909999999999999,"15min":1.2569999999999999},"min":{"1min":0.59199999999999997,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.516,"5min":15.253,"15min":15.253},"last":1.179}]},{"osd":2,"last update":"Wed Oct 11 09:49:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.111,"15min":1.141},"min":{"1min":0.57899999999999996,"5min":0.434,"15min":0.434},"max":{"1min":1.526,"5min":2.7919999999999998,"15min":2.7919999999999998},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.121,"5min":1.153,"15min":1.214},"min":{"1min":0.71599999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.48,"5min":5.0990000000000002,"15min":5.0990000000000002},"last":0.77800000000000002}]},{"osd":3,"last update":"Wed Oct 11 09:49:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.1850000000000001,"15min":1.1739999999999999},"min":{"1min":0.70599999999999996,"5min":0.627,"15min":0.627},"max":{"1min":1.4750000000000001,"5min":11.945,"15min":11.945},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.129,"5min":1.1180000000000001,"15min":1.222},"min":{"1min":0.57499999999999996,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.3999999999999999,"5min":1.5980000000000001,"15min":3.0819999999999999},"last":1.1120000000000001}]},{"osd":4,"last update":"Wed Oct 11 09:49:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.276,"15min":1.296},"min":{"1min":0.69899999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.6319999999999999,"5min":7.5599999999999996,"15min":7.5599999999999996},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.306,"15min":1.341},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.841,"5min":8.7520000000000007,"15min":8.7520000000000007},"last":1.204}]},{"osd":5,"last update":"Wed Oct 11 09:49:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1839999999999999,"15min":1.2430000000000001},"min":{"1min":0.77700000000000002,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.5900000000000001,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.25,"15min":1.349},"min":{"1min":0.80400000000000005,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.998,"5min":2.4390000000000001,"15min":2.7290000000000001},"last":0.997}]},{"osd":7,"last update":"Wed Oct 11 09:49:40 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.323,"15min":1.369},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.8089999999999999,"5min":8.7970000000000006,"15min":8.7970000000000006},"last":1.246},{"interface":"front","average":{"1min":1.341,"5min":1.3600000000000001,"15min":1.4330000000000001},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.605,"5min":2.641,"15min":2.641},"last":1.29}]}]},{"osd":5,"up_from":13,"seq":55834574964,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1268,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1298432,"data_stored":623454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:49:42 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.98699999999999999,"15min":0.99299999999999999},"min":{"1min":0.64100000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.3859999999999999,"5min":1.5049999999999999,"15min":1.516},"last":1.006},{"interface":"front","average":{"1min":1.085,"5min":1.0369999999999999,"15min":1.0029999999999999},"min":{"1min":0.69599999999999995,"5min":0.55300000000000005,"15min":0.46500000000000002},"max":{"1min":1.4370000000000001,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":1.0600000000000001}]},{"osd":1,"last update":"Wed Oct 11 09:49:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.0629999999999999,"15min":0.996},"min":{"1min":0.68700000000000006,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.7190000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.032,"5min":1.0880000000000001,"15min":1.0700000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.437},"max":{"1min":1.5629999999999999,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.1759999999999999}]},{"osd":2,"last update":"Wed Oct 11 09:49:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.153,"15min":1.181},"min":{"1min":0.78100000000000003,"5min":0.76500000000000001,"15min":0.73899999999999999},"max":{"1min":1.708,"5min":2.133,"15min":2.133},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.155,"15min":1.212},"min":{"1min":0.72599999999999998,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.673,"5min":1.6879999999999999,"15min":1.8819999999999999},"last":1.6240000000000001}]},{"osd":3,"last update":"Wed Oct 11 09:49:42 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.1060000000000001,"15min":1.1259999999999999},"min":{"1min":0.76500000000000001,"5min":0.59799999999999998,"15min":0.56999999999999995},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":1.921},"last":1.53},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1639999999999999,"15min":1.1599999999999999},"min":{"1min":0.77600000000000002,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":2.0209999999999999},"last":1.5860000000000001}]},{"osd":4,"last update":"Wed Oct 11 09:49:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1879999999999999,"15min":1.1830000000000001},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.772,"5min":2.117,"15min":2.117},"last":1.8560000000000001},{"interface":"front","average":{"1min":1.095,"5min":1.1619999999999999,"15min":1.1870000000000001},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.9690000000000001,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.887}]},{"osd":6,"last update":"Wed Oct 11 09:49:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.252,"15min":1.2649999999999999},"min":{"1min":0.69499999999999995,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":2.4529999999999998},"last":1.6899999999999999},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.3080000000000001,"15min":1.3089999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.869,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":1.764}]},{"osd":7,"last update":"Wed Oct 11 09:49:42 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.298,"15min":1.3640000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.57699999999999996},"max":{"1min":2.028,"5min":2.347,"15min":3.6120000000000001},"last":2.004},{"interface":"front","average":{"1min":1.409,"5min":1.405,"15min":1.4430000000000001},"min":{"1min":0.89100000000000001,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":2.2679999999999998,"5min":2.2679999999999998,"15min":3.2109999999999999},"last":1.8080000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574964,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32332,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339508,"statfs":{"total":96636764160,"available":96603656192,"internally_reserved":0,"allocated":1314816,"data_stored":625072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:49:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.0389999999999999,"15min":1.056},"min":{"1min":0.67700000000000005,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.3939999999999999,"5min":1.579,"15min":1.7170000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.0720000000000001,"15min":1.0880000000000001},"min":{"1min":0.63700000000000001,"5min":0.60199999999999998,"15min":0.54700000000000004},"max":{"1min":1.669,"5min":1.669,"15min":2.1160000000000001},"last":1.1220000000000001}]},{"osd":1,"last update":"Wed Oct 11 09:49:50 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.0720000000000001,"15min":1.083},"min":{"1min":0.84499999999999997,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.095,"15min":1.143},"min":{"1min":0.73399999999999999,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.8220000000000001},"last":1.075}]},{"osd":2,"last update":"Wed Oct 11 09:49:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.202,"15min":1.1919999999999999},"min":{"1min":0.93000000000000005,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":1.919},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.21,"15min":1.214},"min":{"1min":0.84499999999999997,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.5620000000000001,"5min":1.8300000000000001,"15min":1.8859999999999999},"last":0.78800000000000003}]},{"osd":3,"last update":"Wed Oct 11 09:49:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.097,"15min":1.131},"min":{"1min":0.872,"5min":0.72599999999999998,"15min":0.60199999999999998},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6930000000000001},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.147,"15min":1.25},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":2.3340000000000001},"last":1.1739999999999999}]},{"osd":5,"last update":"Wed Oct 11 09:49:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.153,"15min":1.1899999999999999},"min":{"1min":0.81200000000000006,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.593,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1519999999999999,"15min":1.1910000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.61199999999999999},"max":{"1min":1.732,"5min":1.732,"15min":1.8520000000000001},"last":1.3779999999999999}]},{"osd":6,"last update":"Wed Oct 11 09:49:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.222,"15min":1.256},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.746,"5min":1.746,"15min":1.8},"last":1.298},{"interface":"front","average":{"1min":1.355,"5min":1.294,"15min":1.323},"min":{"1min":1.073,"5min":0.77200000000000002,"15min":0.63500000000000001},"max":{"1min":1.6970000000000001,"5min":1.9399999999999999,"15min":2.1120000000000001},"last":1.218}]},{"osd":7,"last update":"Wed Oct 11 09:49:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.2609999999999999,"15min":1.3120000000000001},"min":{"1min":0.78400000000000003,"5min":0.73299999999999998,"15min":0.60899999999999999},"max":{"1min":1.766,"5min":1.794,"15min":1.9470000000000001},"last":1.337},{"interface":"front","average":{"1min":1.351,"5min":1.3480000000000001,"15min":1.425},"min":{"1min":0.875,"5min":0.82299999999999995,"15min":0.75900000000000001},"max":{"1min":1.8360000000000001,"5min":1.962,"15min":1.9990000000000001},"last":1.395}]}]},{"osd":3,"up_from":13,"seq":55834574963,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1268,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1298432,"data_stored":618317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:49:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.72699999999999998,"15min":0.80900000000000005},"min":{"1min":0.45700000000000002,"5min":0.442,"15min":0.33000000000000002},"max":{"1min":1.286,"5min":1.286,"15min":1.462},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.78300000000000003,"15min":0.83799999999999997},"min":{"1min":0.59099999999999997,"5min":0.48499999999999999,"15min":0.41099999999999998},"max":{"1min":1.363,"5min":1.363,"15min":1.4950000000000001},"last":0.78200000000000003}]},{"osd":1,"last update":"Wed Oct 11 09:49:43 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.78300000000000003,"15min":0.80800000000000005},"min":{"1min":0.53900000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":1.2150000000000001},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.88400000000000001,"15min":0.92400000000000004},"min":{"1min":0.53100000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.94999999999999996}]},{"osd":2,"last update":"Wed Oct 11 09:49:43 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":0.93500000000000005,"15min":0.96899999999999997},"min":{"1min":0.65000000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.044,"5min":0.98999999999999999,"15min":1.0669999999999999},"min":{"1min":0.63700000000000001,"5min":0.58499999999999996,"15min":0.432},"max":{"1min":1.462,"5min":1.607,"15min":2.5870000000000002},"last":1.0880000000000001}]},{"osd":4,"last update":"Wed Oct 11 09:49:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.0980000000000001,"15min":1.1739999999999999},"min":{"1min":0.82999999999999996,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.4119999999999999,"5min":1.7210000000000001,"15min":2.395},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.157,"15min":1.2549999999999999},"min":{"1min":0.94399999999999995,"5min":0.78100000000000003,"15min":0.73599999999999999},"max":{"1min":1.5169999999999999,"5min":1.621,"15min":3.1579999999999999},"last":1.1259999999999999}]},{"osd":5,"last update":"Wed Oct 11 09:49:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.204,"15min":1.288},"min":{"1min":0.99399999999999999,"5min":0.84199999999999997,"15min":0.76500000000000001},"max":{"1min":1.5680000000000001,"5min":1.6699999999999999,"15min":3.0009999999999999},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.25,"5min":1.224,"15min":1.3560000000000001},"min":{"1min":0.90300000000000002,"5min":0.86199999999999999,"15min":0.67800000000000005},"max":{"1min":1.5780000000000001,"5min":1.5840000000000001,"15min":3.8050000000000002},"last":1.282}]},{"osd":6,"last update":"Wed Oct 11 09:49:43 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.1990000000000001,"15min":1.2649999999999999},"min":{"1min":0.92400000000000004,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.4259999999999999,"5min":1.587,"15min":2.1949999999999998},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.2549999999999999,"15min":1.351},"min":{"1min":1.024,"5min":0.95599999999999996,"15min":0.84499999999999997},"max":{"1min":1.6899999999999999,"5min":1.7649999999999999,"15min":3.2549999999999999},"last":1.381}]},{"osd":7,"last update":"Wed Oct 11 09:49:43 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.2789999999999999,"15min":1.399},"min":{"1min":0.97699999999999998,"5min":0.95199999999999996,"15min":0.76700000000000002},"max":{"1min":1.544,"5min":1.7150000000000001,"15min":3.7490000000000001},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.246,"15min":1.3400000000000001},"min":{"1min":0.97499999999999998,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.5049999999999999,"5min":1.661,"15min":2.625},"last":1.024}]}]},{"osd":2,"up_from":13,"seq":55834574965,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":1256,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1286144,"data_stored":616765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.77000000000000002,"15min":0.80500000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.38500000000000001},"max":{"1min":1.25,"5min":1.5920000000000001,"15min":2.0600000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.82399999999999995,"15min":0.90800000000000003},"min":{"1min":0.64100000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.288,"5min":1.571,"15min":2.8919999999999999},"last":0.92700000000000005}]},{"osd":1,"last update":"Wed Oct 11 09:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.84699999999999998,"15min":0.96599999999999997},"min":{"1min":0.59999999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.149,"5min":1.8069999999999999,"15min":3.7650000000000001},"last":0.62},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94099999999999995,"15min":0.97299999999999998},"min":{"1min":0.51500000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.4239999999999999,"5min":1.675,"15min":2.5339999999999998},"last":1.0680000000000001}]},{"osd":3,"last update":"Wed Oct 11 09:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.90400000000000003,"15min":1.034},"min":{"1min":0.58299999999999996,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.591,"5min":1.591,"15min":2.403},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.044,"5min":0.98699999999999999,"15min":1.0469999999999999},"min":{"1min":0.69199999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":2.7589999999999999},"last":1.177}]},{"osd":4,"last update":"Wed Oct 11 09:49:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.1479999999999999,"15min":1.1910000000000001},"min":{"1min":0.745,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.5349999999999999,"5min":1.7809999999999999,"15min":2.2549999999999999},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.3480000000000001},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.73399999999999999},"max":{"1min":1.4470000000000001,"5min":1.5700000000000001,"15min":2.6859999999999999},"last":1.149}]},{"osd":5,"last update":"Wed Oct 11 09:49:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.218,"15min":1.3100000000000001},"min":{"1min":0.86899999999999999,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.6499999999999999,"5min":1.76,"15min":3.044},"last":1.246},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.1779999999999999,"15min":1.272},"min":{"1min":0.78500000000000003,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.6890000000000001,"5min":1.837,"15min":3.5790000000000002},"last":1.105}]},{"osd":6,"last update":"Wed Oct 11 09:49:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.206,"15min":1.3400000000000001},"min":{"1min":0.873,"5min":0.75,"15min":0.75},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":3.1600000000000001},"last":1.367},{"interface":"front","average":{"1min":1.321,"5min":1.2889999999999999,"15min":1.337},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.623},"max":{"1min":1.8560000000000001,"5min":1.869,"15min":2.839},"last":1.3400000000000001}]},{"osd":7,"last update":"Wed Oct 11 09:49:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.252,"15min":1.365},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.774,"5min":1.774,"15min":3.1230000000000002},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.329,"15min":1.4510000000000001},"min":{"1min":0.93000000000000005,"5min":0.84899999999999998,"15min":0.80000000000000004},"max":{"1min":1.7909999999999999,"5min":1.7909999999999999,"15min":2.988},"last":1.2829999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574964,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32312,"kb_used_data":1264,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339528,"statfs":{"total":96636764160,"available":96603676672,"internally_reserved":0,"allocated":1294336,"data_stored":623384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.78500000000000003,"15min":0.78900000000000003},"min":{"1min":0.60599999999999998,"5min":0.375,"15min":0.375},"max":{"1min":1.3109999999999999,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.78900000000000003,"15min":0.82099999999999995},"min":{"1min":0.59199999999999997,"5min":0.40100000000000002,"15min":0.36699999999999999},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":0.95099999999999996}]},{"osd":2,"last update":"Wed Oct 11 09:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.81399999999999995,"15min":0.84799999999999998},"min":{"1min":0.60099999999999998,"5min":0.437,"15min":0.36199999999999999},"max":{"1min":1.306,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.89400000000000002,"15min":0.96099999999999997},"min":{"1min":0.56000000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.6619999999999999},"last":0.81999999999999995}]},{"osd":3,"last update":"Wed Oct 11 09:49:41 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":0.995,"15min":1.014},"min":{"1min":0.73399999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.714,"5min":1.714,"15min":1.714},"last":0.61599999999999999},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.95199999999999996,"15min":0.97599999999999998},"min":{"1min":0.76700000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.84999999999999998}]},{"osd":4,"last update":"Wed Oct 11 09:49:41 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.117,"15min":1.157},"min":{"1min":0.84499999999999997,"5min":0.755,"15min":0.73599999999999999},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":2.016},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.2110000000000001,"15min":1.2769999999999999},"min":{"1min":0.93799999999999994,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.55,"5min":1.665,"15min":2.323},"last":1.1919999999999999}]},{"osd":5,"last update":"Wed Oct 11 09:49:41 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.21,"15min":1.2829999999999999},"min":{"1min":0.86099999999999999,"5min":0.79300000000000004,"15min":0.65800000000000003},"max":{"1min":1.722,"5min":1.722,"15min":2.4079999999999999},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.236,"5min":1.1950000000000001,"15min":1.2410000000000001},"min":{"1min":0.96099999999999997,"5min":0.86299999999999999,"15min":0.81200000000000006},"max":{"1min":1.5660000000000001,"5min":1.677,"15min":2.1349999999999998},"last":1.4359999999999999}]},{"osd":6,"last update":"Wed Oct 11 09:49:41 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.2490000000000001,"15min":1.3320000000000001},"min":{"1min":0.996,"5min":0.84599999999999997,"15min":0.71499999999999997},"max":{"1min":1.853,"5min":2.4620000000000002,"15min":2.8940000000000001},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2569999999999999,"15min":1.329},"min":{"1min":0.93000000000000005,"5min":0.90300000000000002,"15min":0.79200000000000004},"max":{"1min":1.673,"5min":1.8129999999999999,"15min":3.3719999999999999},"last":1.371}]},{"osd":7,"last update":"Wed Oct 11 09:49:41 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3100000000000001,"15min":1.3819999999999999},"min":{"1min":0.95699999999999996,"5min":0.89700000000000002,"15min":0.86099999999999999},"max":{"1min":1.639,"5min":1.835,"15min":2.194},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.395,"5min":1.3160000000000001,"15min":1.397},"min":{"1min":1.0549999999999999,"5min":0.93300000000000005,"15min":0.89600000000000002},"max":{"1min":1.857,"5min":1.857,"15min":1.968},"last":1.399}]}]},{"osd":0,"up_from":13,"seq":55834574963,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32896,"kb_used_data":1848,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338944,"statfs":{"total":96636764160,"available":96603078656,"internally_reserved":0,"allocated":1892352,"data_stored":1214334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 11 09:49:36 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.78900000000000003,"15min":0.80200000000000005},"min":{"1min":0.53800000000000003,"5min":0.48999999999999999,"15min":0.38500000000000001},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.87,"15min":0.91300000000000003},"min":{"1min":0.66900000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.3180000000000001,"5min":1.5249999999999999,"15min":2.3100000000000001},"last":0.68799999999999994}]},{"osd":2,"last update":"Wed Oct 11 09:49:36 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.86599999999999999,"15min":0.93400000000000005},"min":{"1min":0.61699999999999999,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.357,"5min":1.357,"15min":1.7509999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.91600000000000004,"15min":0.95999999999999996},"min":{"1min":0.68799999999999994,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.2549999999999999,"5min":1.5820000000000001,"15min":1.9450000000000001},"last":0.92300000000000004}]},{"osd":3,"last update":"Wed Oct 11 09:49:36 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.79600000000000004,"15min":0.85599999999999998},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.45600000000000002},"max":{"1min":1.22,"5min":1.22,"15min":2.7639999999999998},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90300000000000002,"15min":1.0029999999999999},"min":{"1min":0.61299999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.2649999999999999,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.026}]},{"osd":4,"last update":"Wed Oct 11 09:49:36 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.147,"15min":1.1830000000000001},"min":{"1min":0.94599999999999995,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.5529999999999999,"5min":1.792,"15min":1.792},"last":1.079},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1120000000000001,"15min":1.1850000000000001},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.69799999999999995},"max":{"1min":1.411,"5min":1.5620000000000001,"15min":2.6629999999999998},"last":1.321}]},{"osd":5,"last update":"Wed Oct 11 09:49:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.1970000000000001,"15min":1.3160000000000001},"min":{"1min":0.94799999999999995,"5min":0.878,"15min":0.878},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":3.0150000000000001},"last":1.133},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.1819999999999999,"15min":1.29},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.448,"5min":1.7070000000000001,"15min":3.601},"last":1.155}]},{"osd":6,"last update":"Wed Oct 11 09:49:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.1559999999999999,"15min":1.214},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.78000000000000003},"max":{"1min":1.502,"5min":1.502,"15min":1.825},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.268,"5min":1.238,"15min":1.3149999999999999},"min":{"1min":0.95299999999999996,"5min":0.93999999999999995,"15min":0.93899999999999995},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":2.1440000000000001},"last":1.3600000000000001}]},{"osd":7,"last update":"Wed Oct 11 09:49:36 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.252,"15min":1.3100000000000001},"min":{"1min":0.76900000000000002,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.4510000000000001,"5min":1.75,"15min":1.8759999999999999},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.2370000000000001,"15min":1.3180000000000001},"min":{"1min":0.96699999999999997,"5min":0.92300000000000004,"15min":0.73899999999999999},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":2.5419999999999998},"last":1.0489999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-11T09:49:55.284 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-11T09:49:55.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.486+0000 7fe05cb70700 1 Processor -- start 2023-10-11T09:49:55.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.486+0000 7fe05cb70700 1 -- start start 2023-10-11T09:49:55.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.490+0000 7fe05cb70700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe05815a940 0x7fe0580ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:55.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.490+0000 7fe05cb70700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe05815e010 con 0x7fe0580ca780 2023-10-11T09:49:55.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.490+0000 7fe05cb70700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0580cab70 con 0x7fe05815b8e0 2023-10-11T09:49:55.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.490+0000 7fe05cb70700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe05815e280 con 0x7fe05815a940 2023-10-11T09:49:55.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.490+0000 7fe057fff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fe0580ca780 0x7fe05813fcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:54992/0 (socket says 172.21.15.78:54992) 2023-10-11T09:49:55.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.490+0000 7fe057fff700 1 -- 172.21.15.78:0/1747602395 learned_addr learned my addr 172.21.15.78:0/1747602395 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:49:55.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.490+0000 7fe04effd700 1 --2- 172.21.15.78:0/1747602395 >> v2:172.21.15.133:3300/0 conn(0x7fe05815a940 0x7fe0580ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:55.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.490+0000 7fe056ffd700 1 -- 172.21.15.78:0/1747602395 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 886373828 0 0) 0x7fe05815e010 con 0x7fe0580ca780 2023-10-11T09:49:55.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.490+0000 7fe056ffd700 1 -- 172.21.15.78:0/1747602395 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0380035d0 con 0x7fe0580ca780 2023-10-11T09:49:55.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.490+0000 7fe056ffd700 1 -- 172.21.15.78:0/1747602395 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4285628204 0 0) 0x7fe0580cab70 con 0x7fe05815b8e0 2023-10-11T09:49:55.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.490+0000 7fe056ffd700 1 -- 172.21.15.78:0/1747602395 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe05815e010 con 0x7fe05815b8e0 2023-10-11T09:49:55.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.490+0000 7fe056ffd700 1 -- 172.21.15.78:0/1747602395 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 39282819 0 0) 0x7fe0380035d0 con 0x7fe0580ca780 2023-10-11T09:49:55.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.490+0000 7fe056ffd700 1 -- 172.21.15.78:0/1747602395 >> v2:172.21.15.133:3300/0 conn(0x7fe05815a940 msgr2=0x7fe0580ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:55.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.490+0000 7fe056ffd700 1 --2- 172.21.15.78:0/1747602395 >> v2:172.21.15.133:3300/0 conn(0x7fe05815a940 0x7fe0580ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:55.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.490+0000 7fe056ffd700 1 -- 172.21.15.78:0/1747602395 >> v1:172.21.15.133:6789/0 conn(0x7fe05815b8e0 legacy=0x7fe05806d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:55.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.490+0000 7fe056ffd700 1 -- 172.21.15.78:0/1747602395 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe05815e560 con 0x7fe0580ca780 2023-10-11T09:49:55.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.490+0000 7fe056ffd700 1 -- 172.21.15.78:0/1747602395 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7fe0500019f0 con 0x7fe0580ca780 2023-10-11T09:49:55.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.490+0000 7fe056ffd700 1 -- 172.21.15.78:0/1747602395 >> v1:172.21.15.78:6789/0 conn(0x7fe0580ca780 legacy=0x7fe05813fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:55.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.490+0000 7fe056ffd700 1 --2- 172.21.15.78:0/1747602395 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe038003c00 0x7fe038003ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:55.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.490+0000 7fe056ffd700 1 --2- 172.21.15.78:0/1747602395 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe038004870 0x7fe038004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:55.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.490+0000 7fe056ffd700 1 --2- 172.21.15.78:0/1747602395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe038005220 0x7fe03800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:55.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.490+0000 7fe056ffd700 1 -- 172.21.15.78:0/1747602395 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05815e280 con 0x7fe038005220 2023-10-11T09:49:55.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.490+0000 7fe056ffd700 1 -- 172.21.15.78:0/1747602395 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03800c100 con 0x7fe038004870 2023-10-11T09:49:55.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.490+0000 7fe056ffd700 1 -- 172.21.15.78:0/1747602395 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe03800c290 con 0x7fe038003c00 2023-10-11T09:49:55.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.490+0000 7fe057fff700 1 --2- 172.21.15.78:0/1747602395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe038005220 0x7fe03800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:55.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.490+0000 7fe0577fe700 1 --2- 172.21.15.78:0/1747602395 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe038003c00 0x7fe038003ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:55.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.490+0000 7fe04effd700 1 --2- 172.21.15.78:0/1747602395 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe038004870 0x7fe038004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:55.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.490+0000 7fe057fff700 1 -- 172.21.15.78:0/1747602395 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe038003c00 msgr2=0x7fe038003ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:55.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.490+0000 7fe057fff700 1 --2- 172.21.15.78:0/1747602395 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe038003c00 0x7fe038003ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:55.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.490+0000 7fe057fff700 1 -- 172.21.15.78:0/1747602395 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe038004870 msgr2=0x7fe038004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:55.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.490+0000 7fe057fff700 1 --2- 172.21.15.78:0/1747602395 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe038004870 0x7fe038004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:55.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.490+0000 7fe057fff700 1 -- 172.21.15.78:0/1747602395 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0380035d0 con 0x7fe038005220 2023-10-11T09:49:55.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.490+0000 7fe057fff700 1 -- 172.21.15.78:0/1747602395 reap_dead start 2023-10-11T09:49:55.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.494+0000 7fe057fff700 1 --2- 172.21.15.78:0/1747602395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe038005220 0x7fe03800ba30 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fe05000a6e0 tx=0x7fe0380058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=379b19b7f0ff5386 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:55.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.494+0000 7fe056ffd700 1 -- 172.21.15.78:0/1747602395 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0500070d0 con 0x7fe038005220 2023-10-11T09:49:55.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.494+0000 7fe056ffd700 1 -- 172.21.15.78:0/1747602395 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe050007230 con 0x7fe038005220 2023-10-11T09:49:55.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.494+0000 7fe056ffd700 1 -- 172.21.15.78:0/1747602395 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0500075e0 con 0x7fe038005220 2023-10-11T09:49:55.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.494+0000 7fe05cb70700 1 -- 172.21.15.78:0/1747602395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe038005220 msgr2=0x7fe03800ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:55.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.494+0000 7fe05cb70700 1 --2- 172.21.15.78:0/1747602395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe038005220 0x7fe03800ba30 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fe05000a6e0 tx=0x7fe0380058a0 comp rx=0 tx=0).stop 2023-10-11T09:49:55.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.494+0000 7fe05cb70700 1 -- 172.21.15.78:0/1747602395 shutdown_connections 2023-10-11T09:49:55.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.494+0000 7fe05cb70700 1 --2- 172.21.15.78:0/1747602395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe038005220 0x7fe03800ba30 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:55.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.494+0000 7fe05cb70700 1 -- 172.21.15.78:0/1747602395 >> 172.21.15.78:0/1747602395 conn(0x7fe0580cea60 msgr2=0x7fe0580cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:55.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.494+0000 7fe05cb70700 1 -- 172.21.15.78:0/1747602395 shutdown_connections 2023-10-11T09:49:55.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.494+0000 7fe05cb70700 1 -- 172.21.15.78:0/1747602395 wait complete. 2023-10-11T09:49:55.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.498+0000 7fe05cb70700 1 Processor -- start 2023-10-11T09:49:55.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.498+0000 7fe05cb70700 1 -- start start 2023-10-11T09:49:55.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.498+0000 7fe05cb70700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe038005220 0x7fe0581f1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:55.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.498+0000 7fe05cb70700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe05815cd70 0x7fe05815d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:55.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.498+0000 7fe05cb70700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe058132410 0x7fe058132880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:55.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.498+0000 7fe05cb70700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0581f2fd0 con 0x7fe05815cd70 2023-10-11T09:49:55.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.498+0000 7fe05cb70700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05815d720 con 0x7fe038005220 2023-10-11T09:49:55.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.498+0000 7fe05cb70700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0581f24d0 con 0x7fe058132410 2023-10-11T09:49:55.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.498+0000 7fe04effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe05815cd70 0x7fe05815d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:55.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.498+0000 7fe04effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe05815cd70 0x7fe05815d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36130/0 (socket says 172.21.15.78:36130) 2023-10-11T09:49:55.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.498+0000 7fe04effd700 1 -- 172.21.15.78:0/4042502913 learned_addr learned my addr 172.21.15.78:0/4042502913 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:55.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.498+0000 7fe0577fe700 1 --2- 172.21.15.78:0/4042502913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe038005220 0x7fe0581f1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:55.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.498+0000 7fe057fff700 1 --2- 172.21.15.78:0/4042502913 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe058132410 0x7fe058132880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:55.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.498+0000 7fe04effd700 1 -- 172.21.15.78:0/4042502913 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe058132410 msgr2=0x7fe058132880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:55.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.498+0000 7fe04effd700 1 --2- 172.21.15.78:0/4042502913 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe058132410 0x7fe058132880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:55.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.498+0000 7fe04effd700 1 -- 172.21.15.78:0/4042502913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe038005220 msgr2=0x7fe0581f1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:55.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.498+0000 7fe04effd700 1 --2- 172.21.15.78:0/4042502913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe038005220 0x7fe0581f1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:55.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.498+0000 7fe04effd700 1 -- 172.21.15.78:0/4042502913 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0581f2730 con 0x7fe05815cd70 2023-10-11T09:49:55.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.498+0000 7fe04effd700 1 --2- 172.21.15.78:0/4042502913 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe05815cd70 0x7fe05815d1e0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fe0480108f0 tx=0x7fe048010c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:55.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.498+0000 7fe054ff9700 1 -- 172.21.15.78:0/4042502913 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe048016670 con 0x7fe05815cd70 2023-10-11T09:49:55.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.498+0000 7fe054ff9700 1 -- 172.21.15.78:0/4042502913 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0480167d0 con 0x7fe05815cd70 2023-10-11T09:49:55.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.498+0000 7fe054ff9700 1 -- 172.21.15.78:0/4042502913 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe048016b40 con 0x7fe05815cd70 2023-10-11T09:49:55.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.498+0000 7fe05cb70700 1 -- 172.21.15.78:0/4042502913 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0581f2a10 con 0x7fe05815cd70 2023-10-11T09:49:55.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.502+0000 7fe05cb70700 1 -- 172.21.15.78:0/4042502913 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0581301c0 con 0x7fe05815cd70 2023-10-11T09:49:55.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.502+0000 7fe05cb70700 1 -- 172.21.15.78:0/4042502913 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe058130020 con 0x7fe05815cd70 2023-10-11T09:49:55.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.502+0000 7fe054ff9700 1 -- 172.21.15.78:0/4042502913 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fe048016ca0 con 0x7fe05815cd70 2023-10-11T09:49:55.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.502+0000 7fe054ff9700 1 --2- 172.21.15.78:0/4042502913 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe03c065f00 0x7fe03c0683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:55.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.502+0000 7fe054ff9700 1 -- 172.21.15.78:0/4042502913 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe0480839a0 con 0x7fe05815cd70 2023-10-11T09:49:55.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.506+0000 7fe0577fe700 1 --2- 172.21.15.78:0/4042502913 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe03c065f00 0x7fe03c0683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:55.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.506+0000 7fe0577fe700 1 --2- 172.21.15.78:0/4042502913 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe03c065f00 0x7fe03c0683b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe040000c00 tx=0x7fe040009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:55.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.510+0000 7fe054ff9700 1 -- 172.21.15.78:0/4042502913 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fe058130020 con 0x7fe05815cd70 2023-10-11T09:49:55.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.682+0000 7fe05cb70700 1 -- 172.21.15.78:0/4042502913 --> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe05815eb20 con 0x7fe03c065f00 2023-10-11T09:49:55.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.690+0000 7fe054ff9700 1 -- 172.21.15.78:0/4042502913 <== mgr.4101 v2:172.21.15.78:6832/35606 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309550 (secure 0 0 0) 0x7fe040001690 con 0x7fe03c065f00 2023-10-11T09:49:55.702 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:49:55.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.698+0000 7fe05cb70700 1 -- 172.21.15.78:0/4042502913 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe03c065f00 msgr2=0x7fe03c0683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:55.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.698+0000 7fe05cb70700 1 --2- 172.21.15.78:0/4042502913 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe03c065f00 0x7fe03c0683b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe040000c00 tx=0x7fe040009040 comp rx=0 tx=0).stop 2023-10-11T09:49:55.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.698+0000 7fe05cb70700 1 -- 172.21.15.78:0/4042502913 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe05815cd70 msgr2=0x7fe05815d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:55.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.698+0000 7fe05cb70700 1 --2- 172.21.15.78:0/4042502913 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe05815cd70 0x7fe05815d1e0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fe0480108f0 tx=0x7fe048010c00 comp rx=0 tx=0).stop 2023-10-11T09:49:55.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.698+0000 7fe05cb70700 1 -- 172.21.15.78:0/4042502913 shutdown_connections 2023-10-11T09:49:55.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.698+0000 7fe05cb70700 1 --2- 172.21.15.78:0/4042502913 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe03c065f00 0x7fe03c0683b0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:55.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.698+0000 7fe05cb70700 1 --2- 172.21.15.78:0/4042502913 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe058132410 0x7fe058132880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:55.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.698+0000 7fe05cb70700 1 --2- 172.21.15.78:0/4042502913 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe05815cd70 0x7fe05815d1e0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:55.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.698+0000 7fe05cb70700 1 --2- 172.21.15.78:0/4042502913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe038005220 0x7fe0581f1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:55.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.698+0000 7fe05cb70700 1 -- 172.21.15.78:0/4042502913 >> 172.21.15.78:0/4042502913 conn(0x7fe0580cea60 msgr2=0x7fe0581432a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:55.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.698+0000 7fe05cb70700 1 -- 172.21.15.78:0/4042502913 shutdown_connections 2023-10-11T09:49:55.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.698+0000 7fe05cb70700 1 -- 172.21.15.78:0/4042502913 wait complete. 2023-10-11T09:49:55.711 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2023-10-11T09:49:55.729 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":381,"stamp":"2023-10-11T09:49:54.520995+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":173,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259684,"kb_used_data":11300,"kb_used_omap":0,"kb_used_meta":248320,"kb_avail":754715036,"statfs":{"total":773094113280,"available":772828196864,"internally_reserved":0,"allocated":11571200,"data_stored":6152573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":254279680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.045612"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.860765+0000","last_change":"2023-10-11T09:49:50.989786+0000","last_active":"2023-10-11T09:49:52.860765+0000","last_peered":"2023-10-11T09:49:52.860765+0000","last_clean":"2023-10-11T09:49:52.860765+0000","last_became_active":"2023-10-11T09:49:50.989515+0000","last_became_peered":"2023-10-11T09:49:50.989515+0000","last_unstale":"2023-10-11T09:49:52.860765+0000","last_undegraded":"2023-10-11T09:49:52.860765+0000","last_fullsized":"2023-10-11T09:49:52.860765+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:37:08.369025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.976764+0000","last_change":"2023-10-11T09:49:50.979308+0000","last_active":"2023-10-11T09:49:51.976764+0000","last_peered":"2023-10-11T09:49:51.976764+0000","last_clean":"2023-10-11T09:49:51.976764+0000","last_became_active":"2023-10-11T09:49:50.979017+0000","last_became_peered":"2023-10-11T09:49:50.979017+0000","last_unstale":"2023-10-11T09:49:51.976764+0000","last_undegraded":"2023-10-11T09:49:51.976764+0000","last_fullsized":"2023-10-11T09:49:51.976764+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:10:48.960792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.860974+0000","last_change":"2023-10-11T09:49:50.988605+0000","last_active":"2023-10-11T09:49:52.860974+0000","last_peered":"2023-10-11T09:49:52.860974+0000","last_clean":"2023-10-11T09:49:52.860974+0000","last_became_active":"2023-10-11T09:49:50.988286+0000","last_became_peered":"2023-10-11T09:49:50.988286+0000","last_unstale":"2023-10-11T09:49:52.860974+0000","last_undegraded":"2023-10-11T09:49:52.860974+0000","last_fullsized":"2023-10-11T09:49:52.860974+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:23:52.163820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.862224+0000","last_change":"2023-10-11T09:49:50.987680+0000","last_active":"2023-10-11T09:49:52.862224+0000","last_peered":"2023-10-11T09:49:52.862224+0000","last_clean":"2023-10-11T09:49:52.862224+0000","last_became_active":"2023-10-11T09:49:50.987385+0000","last_became_peered":"2023-10-11T09:49:50.987385+0000","last_unstale":"2023-10-11T09:49:52.862224+0000","last_undegraded":"2023-10-11T09:49:52.862224+0000","last_fullsized":"2023-10-11T09:49:52.862224+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:55:30.217353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.862376+0000","last_change":"2023-10-11T09:49:50.995325+0000","last_active":"2023-10-11T09:49:52.862376+0000","last_peered":"2023-10-11T09:49:52.862376+0000","last_clean":"2023-10-11T09:49:52.862376+0000","last_became_active":"2023-10-11T09:49:50.995036+0000","last_became_peered":"2023-10-11T09:49:50.995036+0000","last_unstale":"2023-10-11T09:49:52.862376+0000","last_undegraded":"2023-10-11T09:49:52.862376+0000","last_fullsized":"2023-10-11T09:49:52.862376+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:59:46.488123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.980264+0000","last_change":"2023-10-11T09:49:50.981217+0000","last_active":"2023-10-11T09:49:51.980264+0000","last_peered":"2023-10-11T09:49:51.980264+0000","last_clean":"2023-10-11T09:49:51.980264+0000","last_became_active":"2023-10-11T09:49:50.980922+0000","last_became_peered":"2023-10-11T09:49:50.980922+0000","last_unstale":"2023-10-11T09:49:51.980264+0000","last_undegraded":"2023-10-11T09:49:51.980264+0000","last_fullsized":"2023-10-11T09:49:51.980264+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:15:13.595478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.980116+0000","last_change":"2023-10-11T09:49:50.978839+0000","last_active":"2023-10-11T09:49:51.980116+0000","last_peered":"2023-10-11T09:49:51.980116+0000","last_clean":"2023-10-11T09:49:51.980116+0000","last_became_active":"2023-10-11T09:49:50.978525+0000","last_became_peered":"2023-10-11T09:49:50.978525+0000","last_unstale":"2023-10-11T09:49:51.980116+0000","last_undegraded":"2023-10-11T09:49:51.980116+0000","last_fullsized":"2023-10-11T09:49:51.980116+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:18:09.722589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.976094+0000","last_change":"2023-10-11T09:49:50.986499+0000","last_active":"2023-10-11T09:49:51.976094+0000","last_peered":"2023-10-11T09:49:51.976094+0000","last_clean":"2023-10-11T09:49:51.976094+0000","last_became_active":"2023-10-11T09:49:50.982208+0000","last_became_peered":"2023-10-11T09:49:50.982208+0000","last_unstale":"2023-10-11T09:49:51.976094+0000","last_undegraded":"2023-10-11T09:49:51.976094+0000","last_fullsized":"2023-10-11T09:49:51.976094+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:22:44.533534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.979969+0000","last_change":"2023-10-11T09:49:50.997182+0000","last_active":"2023-10-11T09:49:51.979969+0000","last_peered":"2023-10-11T09:49:51.979969+0000","last_clean":"2023-10-11T09:49:51.979969+0000","last_became_active":"2023-10-11T09:49:50.996715+0000","last_became_peered":"2023-10-11T09:49:50.996715+0000","last_unstale":"2023-10-11T09:49:51.979969+0000","last_undegraded":"2023-10-11T09:49:51.979969+0000","last_fullsized":"2023-10-11T09:49:51.979969+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:02:09.315636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.861599+0000","last_change":"2023-10-11T09:49:50.997926+0000","last_active":"2023-10-11T09:49:52.861599+0000","last_peered":"2023-10-11T09:49:52.861599+0000","last_clean":"2023-10-11T09:49:52.861599+0000","last_became_active":"2023-10-11T09:49:50.997643+0000","last_became_peered":"2023-10-11T09:49:50.997643+0000","last_unstale":"2023-10-11T09:49:52.861599+0000","last_undegraded":"2023-10-11T09:49:52.861599+0000","last_fullsized":"2023-10-11T09:49:52.861599+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:33:20.865574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.980442+0000","last_change":"2023-10-11T09:49:50.995300+0000","last_active":"2023-10-11T09:49:51.980442+0000","last_peered":"2023-10-11T09:49:51.980442+0000","last_clean":"2023-10-11T09:49:51.980442+0000","last_became_active":"2023-10-11T09:49:50.994518+0000","last_became_peered":"2023-10-11T09:49:50.994518+0000","last_unstale":"2023-10-11T09:49:51.980442+0000","last_undegraded":"2023-10-11T09:49:51.980442+0000","last_fullsized":"2023-10-11T09:49:51.980442+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:59:16.220357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.979625+0000","last_change":"2023-10-11T09:49:50.998757+0000","last_active":"2023-10-11T09:49:51.979625+0000","last_peered":"2023-10-11T09:49:51.979625+0000","last_clean":"2023-10-11T09:49:51.979625+0000","last_became_active":"2023-10-11T09:49:50.998484+0000","last_became_peered":"2023-10-11T09:49:50.998484+0000","last_unstale":"2023-10-11T09:49:51.979625+0000","last_undegraded":"2023-10-11T09:49:51.979625+0000","last_fullsized":"2023-10-11T09:49:51.979625+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:34:10.165727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.862029+0000","last_change":"2023-10-11T09:49:50.995089+0000","last_active":"2023-10-11T09:49:52.862029+0000","last_peered":"2023-10-11T09:49:52.862029+0000","last_clean":"2023-10-11T09:49:52.862029+0000","last_became_active":"2023-10-11T09:49:50.994807+0000","last_became_peered":"2023-10-11T09:49:50.994807+0000","last_unstale":"2023-10-11T09:49:52.862029+0000","last_undegraded":"2023-10-11T09:49:52.862029+0000","last_fullsized":"2023-10-11T09:49:52.862029+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T21:14:55.909654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.761608+0000","last_change":"2023-10-11T09:49:50.988965+0000","last_active":"2023-10-11T09:49:52.761608+0000","last_peered":"2023-10-11T09:49:52.761608+0000","last_clean":"2023-10-11T09:49:52.761608+0000","last_became_active":"2023-10-11T09:49:50.988720+0000","last_became_peered":"2023-10-11T09:49:50.988720+0000","last_unstale":"2023-10-11T09:49:52.761608+0000","last_undegraded":"2023-10-11T09:49:52.761608+0000","last_fullsized":"2023-10-11T09:49:52.761608+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:18:24.742598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.862360+0000","last_change":"2023-10-11T09:49:50.990497+0000","last_active":"2023-10-11T09:49:52.862360+0000","last_peered":"2023-10-11T09:49:52.862360+0000","last_clean":"2023-10-11T09:49:52.862360+0000","last_became_active":"2023-10-11T09:49:50.990259+0000","last_became_peered":"2023-10-11T09:49:50.990259+0000","last_unstale":"2023-10-11T09:49:52.862360+0000","last_undegraded":"2023-10-11T09:49:52.862360+0000","last_fullsized":"2023-10-11T09:49:52.862360+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:21:03.166998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.979627+0000","last_change":"2023-10-11T09:49:50.995978+0000","last_active":"2023-10-11T09:49:51.979627+0000","last_peered":"2023-10-11T09:49:51.979627+0000","last_clean":"2023-10-11T09:49:51.979627+0000","last_became_active":"2023-10-11T09:49:50.995663+0000","last_became_peered":"2023-10-11T09:49:50.995663+0000","last_unstale":"2023-10-11T09:49:51.979627+0000","last_undegraded":"2023-10-11T09:49:51.979627+0000","last_fullsized":"2023-10-11T09:49:51.979627+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:02:09.315636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.976892+0000","last_change":"2023-10-11T09:49:50.985645+0000","last_active":"2023-10-11T09:49:51.976892+0000","last_peered":"2023-10-11T09:49:51.976892+0000","last_clean":"2023-10-11T09:49:51.976892+0000","last_became_active":"2023-10-11T09:49:50.982055+0000","last_became_peered":"2023-10-11T09:49:50.982055+0000","last_unstale":"2023-10-11T09:49:51.976892+0000","last_undegraded":"2023-10-11T09:49:51.976892+0000","last_fullsized":"2023-10-11T09:49:51.976892+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:59:58.229994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.862749+0000","last_change":"2023-10-11T09:49:50.985744+0000","last_active":"2023-10-11T09:49:52.862749+0000","last_peered":"2023-10-11T09:49:52.862749+0000","last_clean":"2023-10-11T09:49:52.862749+0000","last_became_active":"2023-10-11T09:49:50.985453+0000","last_became_peered":"2023-10-11T09:49:50.985453+0000","last_unstale":"2023-10-11T09:49:52.862749+0000","last_undegraded":"2023-10-11T09:49:52.862749+0000","last_fullsized":"2023-10-11T09:49:52.862749+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:05:08.173043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.762398+0000","last_change":"2023-10-11T09:49:50.991940+0000","last_active":"2023-10-11T09:49:52.762398+0000","last_peered":"2023-10-11T09:49:52.762398+0000","last_clean":"2023-10-11T09:49:52.762398+0000","last_became_active":"2023-10-11T09:49:50.991649+0000","last_became_peered":"2023-10-11T09:49:50.991649+0000","last_unstale":"2023-10-11T09:49:52.762398+0000","last_undegraded":"2023-10-11T09:49:52.762398+0000","last_fullsized":"2023-10-11T09:49:52.762398+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:40:12.518424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.979638+0000","last_change":"2023-10-11T09:49:50.997023+0000","last_active":"2023-10-11T09:49:51.979638+0000","last_peered":"2023-10-11T09:49:51.979638+0000","last_clean":"2023-10-11T09:49:51.979638+0000","last_became_active":"2023-10-11T09:49:50.996746+0000","last_became_peered":"2023-10-11T09:49:50.996746+0000","last_unstale":"2023-10-11T09:49:51.979638+0000","last_undegraded":"2023-10-11T09:49:51.979638+0000","last_fullsized":"2023-10-11T09:49:51.979638+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:59:16.220357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.761990+0000","last_change":"2023-10-11T09:49:50.993502+0000","last_active":"2023-10-11T09:49:52.761990+0000","last_peered":"2023-10-11T09:49:52.761990+0000","last_clean":"2023-10-11T09:49:52.761990+0000","last_became_active":"2023-10-11T09:49:50.993252+0000","last_became_peered":"2023-10-11T09:49:50.993252+0000","last_unstale":"2023-10-11T09:49:52.761990+0000","last_undegraded":"2023-10-11T09:49:52.761990+0000","last_fullsized":"2023-10-11T09:49:52.761990+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:28:32.732527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.976087+0000","last_change":"2023-10-11T09:49:50.986490+0000","last_active":"2023-10-11T09:49:51.976087+0000","last_peered":"2023-10-11T09:49:51.976087+0000","last_clean":"2023-10-11T09:49:51.976087+0000","last_became_active":"2023-10-11T09:49:50.986130+0000","last_became_peered":"2023-10-11T09:49:50.986130+0000","last_unstale":"2023-10-11T09:49:51.976087+0000","last_undegraded":"2023-10-11T09:49:51.976087+0000","last_fullsized":"2023-10-11T09:49:51.976087+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:49:51.518585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.976117+0000","last_change":"2023-10-11T09:49:50.986408+0000","last_active":"2023-10-11T09:49:51.976117+0000","last_peered":"2023-10-11T09:49:51.976117+0000","last_clean":"2023-10-11T09:49:51.976117+0000","last_became_active":"2023-10-11T09:49:50.982073+0000","last_became_peered":"2023-10-11T09:49:50.982073+0000","last_unstale":"2023-10-11T09:49:51.976117+0000","last_undegraded":"2023-10-11T09:49:51.976117+0000","last_fullsized":"2023-10-11T09:49:51.976117+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:58:51.631450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.979754+0000","last_change":"2023-10-11T09:49:50.995424+0000","last_active":"2023-10-11T09:49:51.979754+0000","last_peered":"2023-10-11T09:49:51.979754+0000","last_clean":"2023-10-11T09:49:51.979754+0000","last_became_active":"2023-10-11T09:49:50.993764+0000","last_became_peered":"2023-10-11T09:49:50.993764+0000","last_unstale":"2023-10-11T09:49:51.979754+0000","last_undegraded":"2023-10-11T09:49:51.979754+0000","last_fullsized":"2023-10-11T09:49:51.979754+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:21:51.584219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.862247+0000","last_change":"2023-10-11T09:49:51.001328+0000","last_active":"2023-10-11T09:49:52.862247+0000","last_peered":"2023-10-11T09:49:52.862247+0000","last_clean":"2023-10-11T09:49:52.862247+0000","last_became_active":"2023-10-11T09:49:51.000975+0000","last_became_peered":"2023-10-11T09:49:51.000975+0000","last_unstale":"2023-10-11T09:49:52.862247+0000","last_undegraded":"2023-10-11T09:49:52.862247+0000","last_fullsized":"2023-10-11T09:49:52.862247+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:09:32.742677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.976890+0000","last_change":"2023-10-11T09:49:50.989370+0000","last_active":"2023-10-11T09:49:51.976890+0000","last_peered":"2023-10-11T09:49:51.976890+0000","last_clean":"2023-10-11T09:49:51.976890+0000","last_became_active":"2023-10-11T09:49:50.988703+0000","last_became_peered":"2023-10-11T09:49:50.988703+0000","last_unstale":"2023-10-11T09:49:51.976890+0000","last_undegraded":"2023-10-11T09:49:51.976890+0000","last_fullsized":"2023-10-11T09:49:51.976890+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:39:08.209288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.980663+0000","last_change":"2023-10-11T09:49:50.986873+0000","last_active":"2023-10-11T09:49:51.980663+0000","last_peered":"2023-10-11T09:49:51.980663+0000","last_clean":"2023-10-11T09:49:51.980663+0000","last_became_active":"2023-10-11T09:49:50.986584+0000","last_became_peered":"2023-10-11T09:49:50.986584+0000","last_unstale":"2023-10-11T09:49:51.980663+0000","last_undegraded":"2023-10-11T09:49:51.980663+0000","last_fullsized":"2023-10-11T09:49:51.980663+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:44:02.419155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.861589+0000","last_change":"2023-10-11T09:49:50.988505+0000","last_active":"2023-10-11T09:49:52.861589+0000","last_peered":"2023-10-11T09:49:52.861589+0000","last_clean":"2023-10-11T09:49:52.861589+0000","last_became_active":"2023-10-11T09:49:50.987725+0000","last_became_peered":"2023-10-11T09:49:50.987725+0000","last_unstale":"2023-10-11T09:49:52.861589+0000","last_undegraded":"2023-10-11T09:49:52.861589+0000","last_fullsized":"2023-10-11T09:49:52.861589+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:23:52.163820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.976473+0000","last_change":"2023-10-11T09:49:50.992552+0000","last_active":"2023-10-11T09:49:51.976473+0000","last_peered":"2023-10-11T09:49:51.976473+0000","last_clean":"2023-10-11T09:49:51.976473+0000","last_became_active":"2023-10-11T09:49:50.992261+0000","last_became_peered":"2023-10-11T09:49:50.992261+0000","last_unstale":"2023-10-11T09:49:51.976473+0000","last_undegraded":"2023-10-11T09:49:51.976473+0000","last_fullsized":"2023-10-11T09:49:51.976473+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T21:15:14.070871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.761596+0000","last_change":"2023-10-11T09:49:50.991964+0000","last_active":"2023-10-11T09:49:52.761596+0000","last_peered":"2023-10-11T09:49:52.761596+0000","last_clean":"2023-10-11T09:49:52.761596+0000","last_became_active":"2023-10-11T09:49:50.991675+0000","last_became_peered":"2023-10-11T09:49:50.991675+0000","last_unstale":"2023-10-11T09:49:52.761596+0000","last_undegraded":"2023-10-11T09:49:52.761596+0000","last_fullsized":"2023-10-11T09:49:52.761596+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:03:50.944882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.862144+0000","last_change":"2023-10-11T09:49:50.986112+0000","last_active":"2023-10-11T09:49:52.862144+0000","last_peered":"2023-10-11T09:49:52.862144+0000","last_clean":"2023-10-11T09:49:52.862144+0000","last_became_active":"2023-10-11T09:49:50.985872+0000","last_became_peered":"2023-10-11T09:49:50.985872+0000","last_unstale":"2023-10-11T09:49:52.862144+0000","last_undegraded":"2023-10-11T09:49:52.862144+0000","last_fullsized":"2023-10-11T09:49:52.862144+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:09:54.785090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.859955+0000","last_change":"2023-10-11T09:49:51.000196+0000","last_active":"2023-10-11T09:49:52.859955+0000","last_peered":"2023-10-11T09:49:52.859955+0000","last_clean":"2023-10-11T09:49:52.859955+0000","last_became_active":"2023-10-11T09:49:50.999958+0000","last_became_peered":"2023-10-11T09:49:50.999958+0000","last_unstale":"2023-10-11T09:49:52.859955+0000","last_undegraded":"2023-10-11T09:49:52.859955+0000","last_fullsized":"2023-10-11T09:49:52.859955+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:33:20.865574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.762407+0000","last_change":"2023-10-11T09:49:50.998343+0000","last_active":"2023-10-11T09:49:52.762407+0000","last_peered":"2023-10-11T09:49:52.762407+0000","last_clean":"2023-10-11T09:49:52.762407+0000","last_became_active":"2023-10-11T09:49:50.998030+0000","last_became_peered":"2023-10-11T09:49:50.998030+0000","last_unstale":"2023-10-11T09:49:52.762407+0000","last_undegraded":"2023-10-11T09:49:52.762407+0000","last_fullsized":"2023-10-11T09:49:52.762407+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:05:46.041960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"104'4","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.976980+0000","last_change":"2023-10-11T09:49:50.514133+0000","last_active":"2023-10-11T09:49:51.976980+0000","last_peered":"2023-10-11T09:49:51.976980+0000","last_clean":"2023-10-11T09:49:51.976980+0000","last_became_active":"2023-10-11T09:49:50.513884+0000","last_became_peered":"2023-10-11T09:49:50.513884+0000","last_unstale":"2023-10-11T09:49:51.976980+0000","last_undegraded":"2023-10-11T09:49:51.976980+0000","last_fullsized":"2023-10-11T09:49:51.976980+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:21:50.568824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.862666+0000","last_change":"2023-10-11T09:49:50.985757+0000","last_active":"2023-10-11T09:49:52.862666+0000","last_peered":"2023-10-11T09:49:52.862666+0000","last_clean":"2023-10-11T09:49:52.862666+0000","last_became_active":"2023-10-11T09:49:50.985508+0000","last_became_peered":"2023-10-11T09:49:50.985508+0000","last_unstale":"2023-10-11T09:49:52.862666+0000","last_undegraded":"2023-10-11T09:49:52.862666+0000","last_fullsized":"2023-10-11T09:49:52.862666+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:58:29.902050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"104'1","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.980135+0000","last_change":"2023-10-11T09:49:50.526472+0000","last_active":"2023-10-11T09:49:51.980135+0000","last_peered":"2023-10-11T09:49:51.980135+0000","last_clean":"2023-10-11T09:49:51.980135+0000","last_became_active":"2023-10-11T09:49:50.526161+0000","last_became_peered":"2023-10-11T09:49:50.526161+0000","last_unstale":"2023-10-11T09:49:51.980135+0000","last_undegraded":"2023-10-11T09:49:51.980135+0000","last_fullsized":"2023-10-11T09:49:51.980135+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:44:58.855585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.860917+0000","last_change":"2023-10-11T09:49:50.999107+0000","last_active":"2023-10-11T09:49:52.860917+0000","last_peered":"2023-10-11T09:49:52.860917+0000","last_clean":"2023-10-11T09:49:52.860917+0000","last_became_active":"2023-10-11T09:49:50.996673+0000","last_became_peered":"2023-10-11T09:49:50.996673+0000","last_unstale":"2023-10-11T09:49:52.860917+0000","last_undegraded":"2023-10-11T09:49:52.860917+0000","last_fullsized":"2023-10-11T09:49:52.860917+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:09:32.742677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"104'2","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.976745+0000","last_change":"2023-10-11T09:49:50.520875+0000","last_active":"2023-10-11T09:49:51.976745+0000","last_peered":"2023-10-11T09:49:51.976745+0000","last_clean":"2023-10-11T09:49:51.976745+0000","last_became_active":"2023-10-11T09:49:50.519768+0000","last_became_peered":"2023-10-11T09:49:50.519768+0000","last_unstale":"2023-10-11T09:49:51.976745+0000","last_undegraded":"2023-10-11T09:49:51.976745+0000","last_fullsized":"2023-10-11T09:49:51.976745+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:05:20.960304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.980198+0000","last_change":"2023-10-11T09:49:50.994717+0000","last_active":"2023-10-11T09:49:51.980198+0000","last_peered":"2023-10-11T09:49:51.980198+0000","last_clean":"2023-10-11T09:49:51.980198+0000","last_became_active":"2023-10-11T09:49:50.994269+0000","last_became_peered":"2023-10-11T09:49:50.994269+0000","last_unstale":"2023-10-11T09:49:51.980198+0000","last_undegraded":"2023-10-11T09:49:51.980198+0000","last_fullsized":"2023-10-11T09:49:51.980198+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:18:09.722589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"104'2","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.976480+0000","last_change":"2023-10-11T09:49:50.508175+0000","last_active":"2023-10-11T09:49:51.976480+0000","last_peered":"2023-10-11T09:49:51.976480+0000","last_clean":"2023-10-11T09:49:51.976480+0000","last_became_active":"2023-10-11T09:49:50.506935+0000","last_became_peered":"2023-10-11T09:49:50.506935+0000","last_unstale":"2023-10-11T09:49:51.976480+0000","last_undegraded":"2023-10-11T09:49:51.976480+0000","last_fullsized":"2023-10-11T09:49:51.976480+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:44:01.403761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.861733+0000","last_change":"2023-10-11T09:49:50.985826+0000","last_active":"2023-10-11T09:49:52.861733+0000","last_peered":"2023-10-11T09:49:52.861733+0000","last_clean":"2023-10-11T09:49:52.861733+0000","last_became_active":"2023-10-11T09:49:50.985535+0000","last_became_peered":"2023-10-11T09:49:50.985535+0000","last_unstale":"2023-10-11T09:49:52.861733+0000","last_undegraded":"2023-10-11T09:49:52.861733+0000","last_fullsized":"2023-10-11T09:49:52.861733+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:58:29.902050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"104'1","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.860625+0000","last_change":"2023-10-11T09:49:50.520797+0000","last_active":"2023-10-11T09:49:52.860625+0000","last_peered":"2023-10-11T09:49:52.860625+0000","last_clean":"2023-10-11T09:49:52.860625+0000","last_became_active":"2023-10-11T09:49:50.520541+0000","last_became_peered":"2023-10-11T09:49:50.520541+0000","last_unstale":"2023-10-11T09:49:52.860625+0000","last_undegraded":"2023-10-11T09:49:52.860625+0000","last_fullsized":"2023-10-11T09:49:52.860625+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:03:49.929488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.980289+0000","last_change":"2023-10-11T09:49:50.987056+0000","last_active":"2023-10-11T09:49:51.980289+0000","last_peered":"2023-10-11T09:49:51.980289+0000","last_clean":"2023-10-11T09:49:51.980289+0000","last_became_active":"2023-10-11T09:49:50.986763+0000","last_became_peered":"2023-10-11T09:49:50.986763+0000","last_unstale":"2023-10-11T09:49:51.980289+0000","last_undegraded":"2023-10-11T09:49:51.980289+0000","last_fullsized":"2023-10-11T09:49:51.980289+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:34:10.165727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"104'1","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.976226+0000","last_change":"2023-10-11T09:49:50.523783+0000","last_active":"2023-10-11T09:49:51.976226+0000","last_peered":"2023-10-11T09:49:51.976226+0000","last_clean":"2023-10-11T09:49:51.976226+0000","last_became_active":"2023-10-11T09:49:50.523502+0000","last_became_peered":"2023-10-11T09:49:50.523502+0000","last_unstale":"2023-10-11T09:49:51.976226+0000","last_undegraded":"2023-10-11T09:49:51.976226+0000","last_fullsized":"2023-10-11T09:49:51.976226+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:15:32.883615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.861811+0000","last_change":"2023-10-11T09:49:50.988703+0000","last_active":"2023-10-11T09:49:52.861811+0000","last_peered":"2023-10-11T09:49:52.861811+0000","last_clean":"2023-10-11T09:49:52.861811+0000","last_became_active":"2023-10-11T09:49:50.988467+0000","last_became_peered":"2023-10-11T09:49:50.988467+0000","last_unstale":"2023-10-11T09:49:52.861811+0000","last_undegraded":"2023-10-11T09:49:52.861811+0000","last_fullsized":"2023-10-11T09:49:52.861811+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:27:04.170534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.862530+0000","last_change":"2023-10-11T09:49:50.513707+0000","last_active":"2023-10-11T09:49:52.862530+0000","last_peered":"2023-10-11T09:49:52.862530+0000","last_clean":"2023-10-11T09:49:52.862530+0000","last_became_active":"2023-10-11T09:49:50.513395+0000","last_became_peered":"2023-10-11T09:49:50.513395+0000","last_unstale":"2023-10-11T09:49:52.862530+0000","last_undegraded":"2023-10-11T09:49:52.862530+0000","last_fullsized":"2023-10-11T09:49:52.862530+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:05:45.026565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.979044+0000","last_change":"2023-10-11T09:49:50.992115+0000","last_active":"2023-10-11T09:49:51.979044+0000","last_peered":"2023-10-11T09:49:51.979044+0000","last_clean":"2023-10-11T09:49:51.979044+0000","last_became_active":"2023-10-11T09:49:50.991820+0000","last_became_peered":"2023-10-11T09:49:50.991820+0000","last_unstale":"2023-10-11T09:49:51.979044+0000","last_undegraded":"2023-10-11T09:49:51.979044+0000","last_fullsized":"2023-10-11T09:49:51.979044+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:27:04.170534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.860117+0000","last_change":"2023-10-11T09:49:50.522730+0000","last_active":"2023-10-11T09:49:52.860117+0000","last_peered":"2023-10-11T09:49:52.860117+0000","last_clean":"2023-10-11T09:49:52.860117+0000","last_became_active":"2023-10-11T09:49:50.522449+0000","last_became_peered":"2023-10-11T09:49:50.522449+0000","last_unstale":"2023-10-11T09:49:52.860117+0000","last_undegraded":"2023-10-11T09:49:52.860117+0000","last_fullsized":"2023-10-11T09:49:52.860117+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:44:48.334155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.976948+0000","last_change":"2023-10-11T09:49:50.994254+0000","last_active":"2023-10-11T09:49:51.976948+0000","last_peered":"2023-10-11T09:49:51.976948+0000","last_clean":"2023-10-11T09:49:51.976948+0000","last_became_active":"2023-10-11T09:49:50.994012+0000","last_became_peered":"2023-10-11T09:49:50.994012+0000","last_unstale":"2023-10-11T09:49:51.976948+0000","last_undegraded":"2023-10-11T09:49:51.976948+0000","last_fullsized":"2023-10-11T09:49:51.976948+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:44:01.377967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.861109+0000","last_change":"2023-10-11T09:49:50.522467+0000","last_active":"2023-10-11T09:49:52.861109+0000","last_peered":"2023-10-11T09:49:52.861109+0000","last_clean":"2023-10-11T09:49:52.861109+0000","last_became_active":"2023-10-11T09:49:50.522020+0000","last_became_peered":"2023-10-11T09:49:50.522020+0000","last_unstale":"2023-10-11T09:49:52.861109+0000","last_undegraded":"2023-10-11T09:49:52.861109+0000","last_fullsized":"2023-10-11T09:49:52.861109+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:28:31.717133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.980155+0000","last_change":"2023-10-11T09:49:50.996976+0000","last_active":"2023-10-11T09:49:51.980155+0000","last_peered":"2023-10-11T09:49:51.980155+0000","last_clean":"2023-10-11T09:49:51.980155+0000","last_became_active":"2023-10-11T09:49:50.996379+0000","last_became_peered":"2023-10-11T09:49:50.996379+0000","last_unstale":"2023-10-11T09:49:51.980155+0000","last_undegraded":"2023-10-11T09:49:51.980155+0000","last_fullsized":"2023-10-11T09:49:51.980155+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:52:13.301627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.862855+0000","last_change":"2023-10-11T09:49:50.517731+0000","last_active":"2023-10-11T09:49:52.862855+0000","last_peered":"2023-10-11T09:49:52.862855+0000","last_clean":"2023-10-11T09:49:52.862855+0000","last_became_active":"2023-10-11T09:49:50.512755+0000","last_became_peered":"2023-10-11T09:49:50.512755+0000","last_unstale":"2023-10-11T09:49:52.862855+0000","last_undegraded":"2023-10-11T09:49:52.862855+0000","last_fullsized":"2023-10-11T09:49:52.862855+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:37:07.353631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.861540+0000","last_change":"2023-10-11T09:49:51.000726+0000","last_active":"2023-10-11T09:49:52.861540+0000","last_peered":"2023-10-11T09:49:52.861540+0000","last_clean":"2023-10-11T09:49:52.861540+0000","last_became_active":"2023-10-11T09:49:51.000456+0000","last_became_peered":"2023-10-11T09:49:51.000456+0000","last_unstale":"2023-10-11T09:49:52.861540+0000","last_undegraded":"2023-10-11T09:49:52.861540+0000","last_fullsized":"2023-10-11T09:49:52.861540+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:34:36.672892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"104'3","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.980561+0000","last_change":"2023-10-11T09:49:50.512385+0000","last_active":"2023-10-11T09:49:51.980561+0000","last_peered":"2023-10-11T09:49:51.980561+0000","last_clean":"2023-10-11T09:49:51.980561+0000","last_became_active":"2023-10-11T09:49:50.511261+0000","last_became_peered":"2023-10-11T09:49:50.511261+0000","last_unstale":"2023-10-11T09:49:51.980561+0000","last_undegraded":"2023-10-11T09:49:51.980561+0000","last_fullsized":"2023-10-11T09:49:51.980561+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:15:44.292008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.980425+0000","last_change":"2023-10-11T09:49:50.995456+0000","last_active":"2023-10-11T09:49:51.980425+0000","last_peered":"2023-10-11T09:49:51.980425+0000","last_clean":"2023-10-11T09:49:51.980425+0000","last_became_active":"2023-10-11T09:49:50.994836+0000","last_became_peered":"2023-10-11T09:49:50.994836+0000","last_unstale":"2023-10-11T09:49:51.980425+0000","last_undegraded":"2023-10-11T09:49:51.980425+0000","last_fullsized":"2023-10-11T09:49:51.980425+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:21:03.166998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"104'6","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.979823+0000","last_change":"2023-10-11T09:49:50.512003+0000","last_active":"2023-10-11T09:49:51.979823+0000","last_peered":"2023-10-11T09:49:51.979823+0000","last_clean":"2023-10-11T09:49:51.979823+0000","last_became_active":"2023-10-11T09:49:50.511514+0000","last_became_peered":"2023-10-11T09:49:50.511514+0000","last_unstale":"2023-10-11T09:49:51.979823+0000","last_undegraded":"2023-10-11T09:49:51.979823+0000","last_fullsized":"2023-10-11T09:49:51.979823+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:32:20.720114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.861999+0000","last_change":"2023-10-11T09:49:50.981207+0000","last_active":"2023-10-11T09:49:52.861999+0000","last_peered":"2023-10-11T09:49:52.861999+0000","last_clean":"2023-10-11T09:49:52.861999+0000","last_became_active":"2023-10-11T09:49:50.980894+0000","last_became_peered":"2023-10-11T09:49:50.980894+0000","last_unstale":"2023-10-11T09:49:52.861999+0000","last_undegraded":"2023-10-11T09:49:52.861999+0000","last_fullsized":"2023-10-11T09:49:52.861999+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:43:24.307597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"104'2","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.762222+0000","last_change":"2023-10-11T09:49:50.513870+0000","last_active":"2023-10-11T09:49:52.762222+0000","last_peered":"2023-10-11T09:49:52.762222+0000","last_clean":"2023-10-11T09:49:52.762222+0000","last_became_active":"2023-10-11T09:49:50.513624+0000","last_became_peered":"2023-10-11T09:49:50.513624+0000","last_unstale":"2023-10-11T09:49:52.762222+0000","last_undegraded":"2023-10-11T09:49:52.762222+0000","last_fullsized":"2023-10-11T09:49:52.762222+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:53:29.159730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.860484+0000","last_change":"2023-10-11T09:49:50.989100+0000","last_active":"2023-10-11T09:49:52.860484+0000","last_peered":"2023-10-11T09:49:52.860484+0000","last_clean":"2023-10-11T09:49:52.860484+0000","last_became_active":"2023-10-11T09:49:50.988861+0000","last_became_peered":"2023-10-11T09:49:50.988861+0000","last_unstale":"2023-10-11T09:49:52.860484+0000","last_undegraded":"2023-10-11T09:49:52.860484+0000","last_fullsized":"2023-10-11T09:49:52.860484+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:08:47.044323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.762118+0000","last_change":"2023-10-11T09:49:50.523983+0000","last_active":"2023-10-11T09:49:52.762118+0000","last_peered":"2023-10-11T09:49:52.762118+0000","last_clean":"2023-10-11T09:49:52.762118+0000","last_became_active":"2023-10-11T09:49:50.523590+0000","last_became_peered":"2023-10-11T09:49:50.523590+0000","last_unstale":"2023-10-11T09:49:52.762118+0000","last_undegraded":"2023-10-11T09:49:52.762118+0000","last_fullsized":"2023-10-11T09:49:52.762118+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:59:35.713273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.979034+0000","last_change":"2023-10-11T09:49:50.991115+0000","last_active":"2023-10-11T09:49:51.979034+0000","last_peered":"2023-10-11T09:49:51.979034+0000","last_clean":"2023-10-11T09:49:51.979034+0000","last_became_active":"2023-10-11T09:49:50.990841+0000","last_became_peered":"2023-10-11T09:49:50.990841+0000","last_unstale":"2023-10-11T09:49:51.979034+0000","last_undegraded":"2023-10-11T09:49:51.979034+0000","last_fullsized":"2023-10-11T09:49:51.979034+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:21:03.166998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.979242+0000","last_change":"2023-10-11T09:49:50.515961+0000","last_active":"2023-10-11T09:49:51.979242+0000","last_peered":"2023-10-11T09:49:51.979242+0000","last_clean":"2023-10-11T09:49:51.979242+0000","last_became_active":"2023-10-11T09:49:50.515600+0000","last_became_peered":"2023-10-11T09:49:50.515600+0000","last_unstale":"2023-10-11T09:49:51.979242+0000","last_undegraded":"2023-10-11T09:49:51.979242+0000","last_fullsized":"2023-10-11T09:49:51.979242+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:15:44.292008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.761609+0000","last_change":"2023-10-11T09:49:50.997959+0000","last_active":"2023-10-11T09:49:52.761609+0000","last_peered":"2023-10-11T09:49:52.761609+0000","last_clean":"2023-10-11T09:49:52.761609+0000","last_became_active":"2023-10-11T09:49:50.997573+0000","last_became_peered":"2023-10-11T09:49:50.997573+0000","last_unstale":"2023-10-11T09:49:52.761609+0000","last_undegraded":"2023-10-11T09:49:52.761609+0000","last_fullsized":"2023-10-11T09:49:52.761609+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:04:56.068527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"104'8","reported_seq":19,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.860114+0000","last_change":"2023-10-11T09:49:50.520314+0000","last_active":"2023-10-11T09:49:52.860114+0000","last_peered":"2023-10-11T09:49:52.860114+0000","last_clean":"2023-10-11T09:49:52.860114+0000","last_became_active":"2023-10-11T09:49:50.519985+0000","last_became_peered":"2023-10-11T09:49:50.519985+0000","last_unstale":"2023-10-11T09:49:52.860114+0000","last_undegraded":"2023-10-11T09:49:52.860114+0000","last_fullsized":"2023-10-11T09:49:52.860114+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:04:55.053132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.977025+0000","last_change":"2023-10-11T09:49:50.989554+0000","last_active":"2023-10-11T09:49:51.977025+0000","last_peered":"2023-10-11T09:49:51.977025+0000","last_clean":"2023-10-11T09:49:51.977025+0000","last_became_active":"2023-10-11T09:49:50.989037+0000","last_became_peered":"2023-10-11T09:49:50.989037+0000","last_unstale":"2023-10-11T09:49:51.977025+0000","last_undegraded":"2023-10-11T09:49:51.977025+0000","last_fullsized":"2023-10-11T09:49:51.977025+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:05:09.217513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"104'2","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.862418+0000","last_change":"2023-10-11T09:49:50.509971+0000","last_active":"2023-10-11T09:49:52.862418+0000","last_peered":"2023-10-11T09:49:52.862418+0000","last_clean":"2023-10-11T09:49:52.862418+0000","last_became_active":"2023-10-11T09:49:50.509678+0000","last_became_peered":"2023-10-11T09:49:50.509678+0000","last_unstale":"2023-10-11T09:49:52.862418+0000","last_undegraded":"2023-10-11T09:49:52.862418+0000","last_fullsized":"2023-10-11T09:49:52.862418+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:32:20.720114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.860994+0000","last_change":"2023-10-11T09:49:50.991735+0000","last_active":"2023-10-11T09:49:52.860994+0000","last_peered":"2023-10-11T09:49:52.860994+0000","last_clean":"2023-10-11T09:49:52.860994+0000","last_became_active":"2023-10-11T09:49:50.991493+0000","last_became_peered":"2023-10-11T09:49:50.991493+0000","last_unstale":"2023-10-11T09:49:52.860994+0000","last_undegraded":"2023-10-11T09:49:52.860994+0000","last_fullsized":"2023-10-11T09:49:52.860994+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:31:14.207396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.977049+0000","last_change":"2023-10-11T09:49:50.517413+0000","last_active":"2023-10-11T09:49:51.977049+0000","last_peered":"2023-10-11T09:49:51.977049+0000","last_clean":"2023-10-11T09:49:51.977049+0000","last_became_active":"2023-10-11T09:49:50.517154+0000","last_became_peered":"2023-10-11T09:49:50.517154+0000","last_unstale":"2023-10-11T09:49:51.977049+0000","last_undegraded":"2023-10-11T09:49:51.977049+0000","last_fullsized":"2023-10-11T09:49:51.977049+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:18:03.093467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.860803+0000","last_change":"2023-10-11T09:49:50.986853+0000","last_active":"2023-10-11T09:49:52.860803+0000","last_peered":"2023-10-11T09:49:52.860803+0000","last_clean":"2023-10-11T09:49:52.860803+0000","last_became_active":"2023-10-11T09:49:50.986594+0000","last_became_peered":"2023-10-11T09:49:50.986594+0000","last_unstale":"2023-10-11T09:49:52.860803+0000","last_undegraded":"2023-10-11T09:49:52.860803+0000","last_fullsized":"2023-10-11T09:49:52.860803+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:34:36.672892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.976767+0000","last_change":"2023-10-11T09:49:50.508629+0000","last_active":"2023-10-11T09:49:51.976767+0000","last_peered":"2023-10-11T09:49:51.976767+0000","last_clean":"2023-10-11T09:49:51.976767+0000","last_became_active":"2023-10-11T09:49:50.508382+0000","last_became_peered":"2023-10-11T09:49:50.508382+0000","last_unstale":"2023-10-11T09:49:51.976767+0000","last_undegraded":"2023-10-11T09:49:51.976767+0000","last_fullsized":"2023-10-11T09:49:51.976767+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:02:08.300241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.861751+0000","last_change":"2023-10-11T09:49:50.990194+0000","last_active":"2023-10-11T09:49:52.861751+0000","last_peered":"2023-10-11T09:49:52.861751+0000","last_clean":"2023-10-11T09:49:52.861751+0000","last_became_active":"2023-10-11T09:49:50.988413+0000","last_became_peered":"2023-10-11T09:49:50.988413+0000","last_unstale":"2023-10-11T09:49:52.861751+0000","last_undegraded":"2023-10-11T09:49:52.861751+0000","last_fullsized":"2023-10-11T09:49:52.861751+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:30:56.971861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.861754+0000","last_change":"2023-10-11T09:49:50.515303+0000","last_active":"2023-10-11T09:49:52.861754+0000","last_peered":"2023-10-11T09:49:52.861754+0000","last_clean":"2023-10-11T09:49:52.861754+0000","last_became_active":"2023-10-11T09:49:50.514991+0000","last_became_peered":"2023-10-11T09:49:50.514991+0000","last_unstale":"2023-10-11T09:49:52.861754+0000","last_undegraded":"2023-10-11T09:49:52.861754+0000","last_fullsized":"2023-10-11T09:49:52.861754+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:43:53.637800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.761937+0000","last_change":"2023-10-11T09:49:50.998446+0000","last_active":"2023-10-11T09:49:52.761937+0000","last_peered":"2023-10-11T09:49:52.761937+0000","last_clean":"2023-10-11T09:49:52.761937+0000","last_became_active":"2023-10-11T09:49:50.996224+0000","last_became_peered":"2023-10-11T09:49:50.996224+0000","last_unstale":"2023-10-11T09:49:52.761937+0000","last_undegraded":"2023-10-11T09:49:52.761937+0000","last_fullsized":"2023-10-11T09:49:52.761937+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:55:36.981091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.976366+0000","last_change":"2023-10-11T09:49:50.520625+0000","last_active":"2023-10-11T09:49:51.976366+0000","last_peered":"2023-10-11T09:49:51.976366+0000","last_clean":"2023-10-11T09:49:51.976366+0000","last_became_active":"2023-10-11T09:49:50.520251+0000","last_became_peered":"2023-10-11T09:49:50.520251+0000","last_unstale":"2023-10-11T09:49:51.976366+0000","last_undegraded":"2023-10-11T09:49:51.976366+0000","last_fullsized":"2023-10-11T09:49:51.976366+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:59:15.204962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.979386+0000","last_change":"2023-10-11T09:49:50.990452+0000","last_active":"2023-10-11T09:49:51.979386+0000","last_peered":"2023-10-11T09:49:51.979386+0000","last_clean":"2023-10-11T09:49:51.979386+0000","last_became_active":"2023-10-11T09:49:50.990168+0000","last_became_peered":"2023-10-11T09:49:50.990168+0000","last_unstale":"2023-10-11T09:49:51.979386+0000","last_undegraded":"2023-10-11T09:49:51.979386+0000","last_fullsized":"2023-10-11T09:49:51.979386+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:15:13.595478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.861231+0000","last_change":"2023-10-11T09:49:50.524094+0000","last_active":"2023-10-11T09:49:52.861231+0000","last_peered":"2023-10-11T09:49:52.861231+0000","last_clean":"2023-10-11T09:49:52.861231+0000","last_became_active":"2023-10-11T09:49:50.523738+0000","last_became_peered":"2023-10-11T09:49:50.523738+0000","last_unstale":"2023-10-11T09:49:52.861231+0000","last_undegraded":"2023-10-11T09:49:52.861231+0000","last_fullsized":"2023-10-11T09:49:52.861231+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:11:41.951896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.761593+0000","last_change":"2023-10-11T09:49:50.997443+0000","last_active":"2023-10-11T09:49:52.761593+0000","last_peered":"2023-10-11T09:49:52.761593+0000","last_clean":"2023-10-11T09:49:52.761593+0000","last_became_active":"2023-10-11T09:49:50.997179+0000","last_became_peered":"2023-10-11T09:49:50.997179+0000","last_unstale":"2023-10-11T09:49:52.761593+0000","last_undegraded":"2023-10-11T09:49:52.761593+0000","last_fullsized":"2023-10-11T09:49:52.761593+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:44:49.349550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.862331+0000","last_change":"2023-10-11T09:49:50.523257+0000","last_active":"2023-10-11T09:49:52.862331+0000","last_peered":"2023-10-11T09:49:52.862331+0000","last_clean":"2023-10-11T09:49:52.862331+0000","last_became_active":"2023-10-11T09:49:50.522563+0000","last_became_peered":"2023-10-11T09:49:50.522563+0000","last_unstale":"2023-10-11T09:49:52.862331+0000","last_undegraded":"2023-10-11T09:49:52.862331+0000","last_fullsized":"2023-10-11T09:49:52.862331+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:06:33.179642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.861114+0000","last_change":"2023-10-11T09:49:50.995440+0000","last_active":"2023-10-11T09:49:52.861114+0000","last_peered":"2023-10-11T09:49:52.861114+0000","last_clean":"2023-10-11T09:49:52.861114+0000","last_became_active":"2023-10-11T09:49:50.995157+0000","last_became_peered":"2023-10-11T09:49:50.995157+0000","last_unstale":"2023-10-11T09:49:52.861114+0000","last_undegraded":"2023-10-11T09:49:52.861114+0000","last_fullsized":"2023-10-11T09:49:52.861114+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:08:47.044323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.761792+0000","last_change":"2023-10-11T09:49:50.516673+0000","last_active":"2023-10-11T09:49:52.761792+0000","last_peered":"2023-10-11T09:49:52.761792+0000","last_clean":"2023-10-11T09:49:52.761792+0000","last_became_active":"2023-10-11T09:49:50.516247+0000","last_became_peered":"2023-10-11T09:49:50.516247+0000","last_unstale":"2023-10-11T09:49:52.761792+0000","last_undegraded":"2023-10-11T09:49:52.761792+0000","last_fullsized":"2023-10-11T09:49:52.761792+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:14:35.600428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.761908+0000","last_change":"2023-10-11T09:49:50.993829+0000","last_active":"2023-10-11T09:49:52.761908+0000","last_peered":"2023-10-11T09:49:52.761908+0000","last_clean":"2023-10-11T09:49:52.761908+0000","last_became_active":"2023-10-11T09:49:50.993547+0000","last_became_peered":"2023-10-11T09:49:50.993547+0000","last_unstale":"2023-10-11T09:49:52.761908+0000","last_undegraded":"2023-10-11T09:49:52.761908+0000","last_fullsized":"2023-10-11T09:49:52.761908+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:44:00.902594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"104'2","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.976220+0000","last_change":"2023-10-11T09:49:50.516828+0000","last_active":"2023-10-11T09:49:51.976220+0000","last_peered":"2023-10-11T09:49:51.976220+0000","last_clean":"2023-10-11T09:49:51.976220+0000","last_became_active":"2023-10-11T09:49:50.516569+0000","last_became_peered":"2023-10-11T09:49:50.516569+0000","last_unstale":"2023-10-11T09:49:51.976220+0000","last_undegraded":"2023-10-11T09:49:51.976220+0000","last_fullsized":"2023-10-11T09:49:51.976220+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:02:50.778526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":77,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-11T09:49:50.978451+0000","last_change":"2023-10-11T09:40:36.151839+0000","last_active":"2023-10-11T09:49:50.978451+0000","last_peered":"2023-10-11T09:49:50.978451+0000","last_clean":"2023-10-11T09:49:50.978451+0000","last_became_active":"2023-10-11T09:40:36.151179+0000","last_became_peered":"2023-10-11T09:40:36.151179+0000","last_unstale":"2023-10-11T09:49:50.978451+0000","last_undegraded":"2023-10-11T09:49:50.978451+0000","last_fullsized":"2023-10-11T09:49:50.978451+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.762118+0000","last_change":"2023-10-11T09:49:51.000188+0000","last_active":"2023-10-11T09:49:52.762118+0000","last_peered":"2023-10-11T09:49:52.762118+0000","last_clean":"2023-10-11T09:49:52.762118+0000","last_became_active":"2023-10-11T09:49:50.999952+0000","last_became_peered":"2023-10-11T09:49:50.999952+0000","last_unstale":"2023-10-11T09:49:52.762118+0000","last_undegraded":"2023-10-11T09:49:52.762118+0000","last_fullsized":"2023-10-11T09:49:52.762118+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:00:58.856002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.861770+0000","last_change":"2023-10-11T09:49:50.524923+0000","last_active":"2023-10-11T09:49:52.861770+0000","last_peered":"2023-10-11T09:49:52.861770+0000","last_clean":"2023-10-11T09:49:52.861770+0000","last_became_active":"2023-10-11T09:49:50.524643+0000","last_became_peered":"2023-10-11T09:49:50.524643+0000","last_unstale":"2023-10-11T09:49:52.861770+0000","last_undegraded":"2023-10-11T09:49:52.861770+0000","last_fullsized":"2023-10-11T09:49:52.861770+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:37:36.785205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":82,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-11T09:49:50.974920+0000","last_change":"2023-10-11T09:40:36.150224+0000","last_active":"2023-10-11T09:49:50.974920+0000","last_peered":"2023-10-11T09:49:50.974920+0000","last_clean":"2023-10-11T09:49:50.974920+0000","last_became_active":"2023-10-11T09:40:36.149919+0000","last_became_peered":"2023-10-11T09:40:36.149919+0000","last_unstale":"2023-10-11T09:49:50.974920+0000","last_undegraded":"2023-10-11T09:49:50.974920+0000","last_fullsized":"2023-10-11T09:49:50.974920+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.761519+0000","last_change":"2023-10-11T09:49:50.994027+0000","last_active":"2023-10-11T09:49:52.761519+0000","last_peered":"2023-10-11T09:49:52.761519+0000","last_clean":"2023-10-11T09:49:52.761519+0000","last_became_active":"2023-10-11T09:49:50.993781+0000","last_became_peered":"2023-10-11T09:49:50.993781+0000","last_unstale":"2023-10-11T09:49:52.761519+0000","last_undegraded":"2023-10-11T09:49:52.761519+0000","last_fullsized":"2023-10-11T09:49:52.761519+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:14:59.605167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.862078+0000","last_change":"2023-10-11T09:49:50.522433+0000","last_active":"2023-10-11T09:49:52.862078+0000","last_peered":"2023-10-11T09:49:52.862078+0000","last_clean":"2023-10-11T09:49:52.862078+0000","last_became_active":"2023-10-11T09:49:50.522041+0000","last_became_peered":"2023-10-11T09:49:50.522041+0000","last_unstale":"2023-10-11T09:49:52.862078+0000","last_undegraded":"2023-10-11T09:49:52.862078+0000","last_fullsized":"2023-10-11T09:49:52.862078+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:00:54.423451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":100,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-11T09:49:50.973102+0000","last_change":"2023-10-11T09:40:36.162174+0000","last_active":"2023-10-11T09:49:50.973102+0000","last_peered":"2023-10-11T09:49:50.973102+0000","last_clean":"2023-10-11T09:49:50.973102+0000","last_became_active":"2023-10-11T09:40:36.161710+0000","last_became_peered":"2023-10-11T09:40:36.161710+0000","last_unstale":"2023-10-11T09:49:50.973102+0000","last_undegraded":"2023-10-11T09:49:50.973102+0000","last_fullsized":"2023-10-11T09:49:50.973102+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:57:03.630878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.861017+0000","last_change":"2023-10-11T09:49:50.991770+0000","last_active":"2023-10-11T09:49:52.861017+0000","last_peered":"2023-10-11T09:49:52.861017+0000","last_clean":"2023-10-11T09:49:52.861017+0000","last_became_active":"2023-10-11T09:49:50.991408+0000","last_became_peered":"2023-10-11T09:49:50.991408+0000","last_unstale":"2023-10-11T09:49:52.861017+0000","last_undegraded":"2023-10-11T09:49:52.861017+0000","last_fullsized":"2023-10-11T09:49:52.861017+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:31:14.207396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.976217+0000","last_change":"2023-10-11T09:49:50.524013+0000","last_active":"2023-10-11T09:49:51.976217+0000","last_peered":"2023-10-11T09:49:51.976217+0000","last_clean":"2023-10-11T09:49:51.976217+0000","last_became_active":"2023-10-11T09:49:50.523692+0000","last_became_peered":"2023-10-11T09:49:50.523692+0000","last_unstale":"2023-10-11T09:49:51.976217+0000","last_undegraded":"2023-10-11T09:49:51.976217+0000","last_fullsized":"2023-10-11T09:49:51.976217+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T21:34:46.814409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":82,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-11T09:49:50.976184+0000","last_change":"2023-10-11T09:40:36.160620+0000","last_active":"2023-10-11T09:49:50.976184+0000","last_peered":"2023-10-11T09:49:50.976184+0000","last_clean":"2023-10-11T09:49:50.976184+0000","last_became_active":"2023-10-11T09:40:36.160171+0000","last_became_peered":"2023-10-11T09:40:36.160171+0000","last_unstale":"2023-10-11T09:49:50.976184+0000","last_undegraded":"2023-10-11T09:49:50.976184+0000","last_fullsized":"2023-10-11T09:49:50.976184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.862297+0000","last_change":"2023-10-11T09:49:50.999791+0000","last_active":"2023-10-11T09:49:52.862297+0000","last_peered":"2023-10-11T09:49:52.862297+0000","last_clean":"2023-10-11T09:49:52.862297+0000","last_became_active":"2023-10-11T09:49:50.999488+0000","last_became_peered":"2023-10-11T09:49:50.999488+0000","last_unstale":"2023-10-11T09:49:52.862297+0000","last_undegraded":"2023-10-11T09:49:52.862297+0000","last_fullsized":"2023-10-11T09:49:52.862297+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:44:00.902594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"104'2","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.979613+0000","last_change":"2023-10-11T09:49:50.516923+0000","last_active":"2023-10-11T09:49:51.979613+0000","last_peered":"2023-10-11T09:49:51.979613+0000","last_clean":"2023-10-11T09:49:51.979613+0000","last_became_active":"2023-10-11T09:49:50.516630+0000","last_became_peered":"2023-10-11T09:49:50.516630+0000","last_unstale":"2023-10-11T09:49:51.979613+0000","last_undegraded":"2023-10-11T09:49:51.979613+0000","last_fullsized":"2023-10-11T09:49:51.979613+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:44:58.855585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":82,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-11T09:49:50.973127+0000","last_change":"2023-10-11T09:40:36.151710+0000","last_active":"2023-10-11T09:49:50.973127+0000","last_peered":"2023-10-11T09:49:50.973127+0000","last_clean":"2023-10-11T09:49:50.973127+0000","last_became_active":"2023-10-11T09:40:36.151219+0000","last_became_peered":"2023-10-11T09:40:36.151219+0000","last_unstale":"2023-10-11T09:49:50.973127+0000","last_undegraded":"2023-10-11T09:49:50.973127+0000","last_fullsized":"2023-10-11T09:49:50.973127+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.862125+0000","last_change":"2023-10-11T09:49:50.987205+0000","last_active":"2023-10-11T09:49:52.862125+0000","last_peered":"2023-10-11T09:49:52.862125+0000","last_clean":"2023-10-11T09:49:52.862125+0000","last_became_active":"2023-10-11T09:49:50.986839+0000","last_became_peered":"2023-10-11T09:49:50.986839+0000","last_unstale":"2023-10-11T09:49:52.862125+0000","last_undegraded":"2023-10-11T09:49:52.862125+0000","last_fullsized":"2023-10-11T09:49:52.862125+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:05:08.173043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"104'1","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.861906+0000","last_change":"2023-10-11T09:49:50.512199+0000","last_active":"2023-10-11T09:49:52.861906+0000","last_peered":"2023-10-11T09:49:52.861906+0000","last_clean":"2023-10-11T09:49:52.861906+0000","last_became_active":"2023-10-11T09:49:50.511867+0000","last_became_peered":"2023-10-11T09:49:50.511867+0000","last_unstale":"2023-10-11T09:49:52.861906+0000","last_undegraded":"2023-10-11T09:49:52.861906+0000","last_fullsized":"2023-10-11T09:49:52.861906+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:27:42.811643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":82,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-11T09:49:50.978606+0000","last_change":"2023-10-11T09:40:36.216248+0000","last_active":"2023-10-11T09:49:50.978606+0000","last_peered":"2023-10-11T09:49:50.978606+0000","last_clean":"2023-10-11T09:49:50.978606+0000","last_became_active":"2023-10-11T09:40:36.215890+0000","last_became_peered":"2023-10-11T09:40:36.215890+0000","last_unstale":"2023-10-11T09:49:50.978606+0000","last_undegraded":"2023-10-11T09:49:50.978606+0000","last_fullsized":"2023-10-11T09:49:50.978606+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.980483+0000","last_change":"2023-10-11T09:49:50.995121+0000","last_active":"2023-10-11T09:49:51.980483+0000","last_peered":"2023-10-11T09:49:51.980483+0000","last_clean":"2023-10-11T09:49:51.980483+0000","last_became_active":"2023-10-11T09:49:50.994168+0000","last_became_peered":"2023-10-11T09:49:50.994168+0000","last_unstale":"2023-10-11T09:49:51.980483+0000","last_undegraded":"2023-10-11T09:49:51.980483+0000","last_fullsized":"2023-10-11T09:49:51.980483+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:52:13.301627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.976526+0000","last_change":"2023-10-11T09:49:50.509104+0000","last_active":"2023-10-11T09:49:51.976526+0000","last_peered":"2023-10-11T09:49:51.976526+0000","last_clean":"2023-10-11T09:49:51.976526+0000","last_became_active":"2023-10-11T09:49:50.508835+0000","last_became_peered":"2023-10-11T09:49:50.508835+0000","last_unstale":"2023-10-11T09:49:51.976526+0000","last_undegraded":"2023-10-11T09:49:51.976526+0000","last_fullsized":"2023-10-11T09:49:51.976526+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:34:09.150332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":100,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-11T09:49:50.973570+0000","last_change":"2023-10-11T09:40:36.149739+0000","last_active":"2023-10-11T09:49:50.973570+0000","last_peered":"2023-10-11T09:49:50.973570+0000","last_clean":"2023-10-11T09:49:50.973570+0000","last_became_active":"2023-10-11T09:40:36.149419+0000","last_became_peered":"2023-10-11T09:40:36.149419+0000","last_unstale":"2023-10-11T09:49:50.973570+0000","last_undegraded":"2023-10-11T09:49:50.973570+0000","last_fullsized":"2023-10-11T09:49:50.973570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:54:50.640362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:52.762380+0000","last_change":"2023-10-11T09:49:50.994097+0000","last_active":"2023-10-11T09:49:52.762380+0000","last_peered":"2023-10-11T09:49:52.762380+0000","last_clean":"2023-10-11T09:49:52.762380+0000","last_became_active":"2023-10-11T09:49:50.993772+0000","last_became_peered":"2023-10-11T09:49:50.993772+0000","last_unstale":"2023-10-11T09:49:52.762380+0000","last_undegraded":"2023-10-11T09:49:52.762380+0000","last_fullsized":"2023-10-11T09:49:52.762380+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:50.474094+0000","last_clean_scrub_stamp":"2023-10-11T09:49:50.474094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:12:29.979310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"104'1","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-11T09:49:51.980376+0000","last_change":"2023-10-11T09:49:50.524655+0000","last_active":"2023-10-11T09:49:51.980376+0000","last_peered":"2023-10-11T09:49:51.980376+0000","last_clean":"2023-10-11T09:49:51.980376+0000","last_became_active":"2023-10-11T09:49:50.524340+0000","last_became_peered":"2023-10-11T09:49:50.524340+0000","last_unstale":"2023-10-11T09:49:51.980376+0000","last_undegraded":"2023-10-11T09:49:51.980376+0000","last_fullsized":"2023-10-11T09:49:51.980376+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:49:49.458699+0000","last_clean_scrub_stamp":"2023-10-11T09:49:49.458699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:54:27.367170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":82,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-11T09:49:50.971387+0000","last_change":"2023-10-11T09:40:36.148773+0000","last_active":"2023-10-11T09:49:50.971387+0000","last_peered":"2023-10-11T09:49:50.971387+0000","last_clean":"2023-10-11T09:49:50.971387+0000","last_became_active":"2023-10-11T09:40:36.148489+0000","last_became_peered":"2023-10-11T09:40:36.148489+0000","last_unstale":"2023-10-11T09:49:50.971387+0000","last_undegraded":"2023-10-11T09:49:50.971387+0000","last_fullsized":"2023-10-11T09:49:50.971387+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":253,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-11T09:49:50.973719+0000","last_change":"2023-10-11T09:40:29.874372+0000","last_active":"2023-10-11T09:49:50.973719+0000","last_peered":"2023-10-11T09:49:50.973719+0000","last_clean":"2023-10-11T09:49:50.973719+0000","last_became_active":"2023-10-11T09:40:29.873621+0000","last_became_peered":"2023-10-11T09:40:29.873621+0000","last_unstale":"2023-10-11T09:49:50.973719+0000","last_undegraded":"2023-10-11T09:49:50.973719+0000","last_fullsized":"2023-10-11T09:49:50.973719+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_clean_scrub_stamp":"2023-10-11T09:40:28.742276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:22:07.985311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574965,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32900,"kb_used_data":1852,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338940,"statfs":{"total":96636764160,"available":96603074560,"internally_reserved":0,"allocated":1896448,"data_stored":1213864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:49:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.0049999999999999,"15min":1.0980000000000001},"min":{"1min":0.68600000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.4770000000000001,"5min":1.7390000000000001,"15min":1.903},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.131,"5min":1.01,"15min":1.0980000000000001},"min":{"1min":0.86399999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3899999999999999,"5min":1.5760000000000001,"15min":1.696},"last":1.554}]},{"osd":1,"last update":"Wed Oct 11 09:49:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.079,"15min":1.1140000000000001},"min":{"1min":0.84199999999999997,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.4370000000000001,"5min":1.645,"15min":2.25},"last":1.466},{"interface":"front","average":{"1min":1.054,"5min":1.016,"15min":1.0960000000000001},"min":{"1min":0.80400000000000005,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.5369999999999999,"5min":1.587,"15min":1.696},"last":1.115}]},{"osd":2,"last update":"Wed Oct 11 09:49:49 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.0800000000000001,"15min":1.109},"min":{"1min":0.68500000000000005,"5min":0.59499999999999997,"15min":0.58999999999999997},"max":{"1min":1.466,"5min":1.6339999999999999,"15min":1.873},"last":1.486},{"interface":"front","average":{"1min":1.206,"5min":1.1499999999999999,"15min":1.149},"min":{"1min":0.73599999999999999,"5min":0.66400000000000003,"15min":0.65500000000000003},"max":{"1min":1.569,"5min":1.78,"15min":1.78},"last":1.335}]},{"osd":3,"last update":"Wed Oct 11 09:49:49 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.159,"15min":1.2230000000000001},"min":{"1min":0.80000000000000004,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.53},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.246,"15min":1.2809999999999999},"min":{"1min":0.97199999999999998,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.8089999999999999,"5min":1.8720000000000001,"15min":2.1579999999999999},"last":1.514}]},{"osd":4,"last update":"Wed Oct 11 09:49:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0569999999999999,"15min":1.1379999999999999},"min":{"1min":0.752,"5min":0.64900000000000002,"15min":0.53300000000000003},"max":{"1min":1.4219999999999999,"5min":1.661,"15min":2.2010000000000001},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.202,"15min":1.2370000000000001},"min":{"1min":0.88200000000000001,"5min":0.67200000000000004,"15min":0.63500000000000001},"max":{"1min":1.768,"5min":1.768,"15min":1.9339999999999999},"last":1.4330000000000001}]},{"osd":5,"last update":"Wed Oct 11 09:49:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.28,"15min":1.2829999999999999},"min":{"1min":0.95699999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.744},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.2070000000000001,"15min":1.302},"min":{"1min":0.98199999999999998,"5min":0.78300000000000003,"15min":0.65900000000000003},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":3.0750000000000002},"last":1.196}]},{"osd":6,"last update":"Wed Oct 11 09:49:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.345,"15min":1.411},"min":{"1min":1.2010000000000001,"5min":0.77000000000000002,"15min":0.70999999999999996},"max":{"1min":1.835,"5min":1.919,"15min":3.137},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.417,"5min":1.3560000000000001,"15min":1.395},"min":{"1min":1.032,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.784,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.5760000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574964,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32308,"kb_used_data":1260,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339532,"statfs":{"total":96636764160,"available":96603680768,"internally_reserved":0,"allocated":1290240,"data_stored":617383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:49:40 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.085,"15min":1.028},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.6200000000000001,"5min":7.0730000000000004,"15min":7.0730000000000004},"last":1.3240000000000001},{"interface":"front","average":{"1min":0.998,"5min":1.165,"15min":1.079},"min":{"1min":0.63100000000000001,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.409,"5min":8.8450000000000006,"15min":8.8450000000000006},"last":1.081}]},{"osd":1,"last update":"Wed Oct 11 09:49:40 2023","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.3009999999999999,"15min":1.2290000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.4370000000000001,"5min":15.523,"15min":15.523},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.141,"5min":1.2909999999999999,"15min":1.2569999999999999},"min":{"1min":0.59199999999999997,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.516,"5min":15.253,"15min":15.253},"last":1.179}]},{"osd":2,"last update":"Wed Oct 11 09:49:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.111,"15min":1.141},"min":{"1min":0.57899999999999996,"5min":0.434,"15min":0.434},"max":{"1min":1.526,"5min":2.7919999999999998,"15min":2.7919999999999998},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.121,"5min":1.153,"15min":1.214},"min":{"1min":0.71599999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.48,"5min":5.0990000000000002,"15min":5.0990000000000002},"last":0.77800000000000002}]},{"osd":3,"last update":"Wed Oct 11 09:49:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.1850000000000001,"15min":1.1739999999999999},"min":{"1min":0.70599999999999996,"5min":0.627,"15min":0.627},"max":{"1min":1.4750000000000001,"5min":11.945,"15min":11.945},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.129,"5min":1.1180000000000001,"15min":1.222},"min":{"1min":0.57499999999999996,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.3999999999999999,"5min":1.5980000000000001,"15min":3.0819999999999999},"last":1.1120000000000001}]},{"osd":4,"last update":"Wed Oct 11 09:49:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.276,"15min":1.296},"min":{"1min":0.69899999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.6319999999999999,"5min":7.5599999999999996,"15min":7.5599999999999996},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.306,"15min":1.341},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.841,"5min":8.7520000000000007,"15min":8.7520000000000007},"last":1.204}]},{"osd":5,"last update":"Wed Oct 11 09:49:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1839999999999999,"15min":1.2430000000000001},"min":{"1min":0.77700000000000002,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.5900000000000001,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.25,"15min":1.349},"min":{"1min":0.80400000000000005,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.998,"5min":2.4390000000000001,"15min":2.7290000000000001},"last":0.997}]},{"osd":7,"last update":"Wed Oct 11 09:49:40 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.323,"15min":1.369},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.8089999999999999,"5min":8.7970000000000006,"15min":8.7970000000000006},"last":1.246},{"interface":"front","average":{"1min":1.341,"5min":1.3600000000000001,"15min":1.4330000000000001},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.605,"5min":2.641,"15min":2.641},"last":1.29}]}]},{"osd":5,"up_from":13,"seq":55834574964,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1268,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1298432,"data_stored":623454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:49:42 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.98699999999999999,"15min":0.99299999999999999},"min":{"1min":0.64100000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.3859999999999999,"5min":1.5049999999999999,"15min":1.516},"last":1.006},{"interface":"front","average":{"1min":1.085,"5min":1.0369999999999999,"15min":1.0029999999999999},"min":{"1min":0.69599999999999995,"5min":0.55300000000000005,"15min":0.46500000000000002},"max":{"1min":1.4370000000000001,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":1.0600000000000001}]},{"osd":1,"last update":"Wed Oct 11 09:49:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.0629999999999999,"15min":0.996},"min":{"1min":0.68700000000000006,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.7190000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.032,"5min":1.0880000000000001,"15min":1.0700000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.437},"max":{"1min":1.5629999999999999,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.1759999999999999}]},{"osd":2,"last update":"Wed Oct 11 09:49:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.153,"15min":1.181},"min":{"1min":0.78100000000000003,"5min":0.76500000000000001,"15min":0.73899999999999999},"max":{"1min":1.708,"5min":2.133,"15min":2.133},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.155,"15min":1.212},"min":{"1min":0.72599999999999998,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.673,"5min":1.6879999999999999,"15min":1.8819999999999999},"last":1.6240000000000001}]},{"osd":3,"last update":"Wed Oct 11 09:49:42 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.1060000000000001,"15min":1.1259999999999999},"min":{"1min":0.76500000000000001,"5min":0.59799999999999998,"15min":0.56999999999999995},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":1.921},"last":1.53},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1639999999999999,"15min":1.1599999999999999},"min":{"1min":0.77600000000000002,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":2.0209999999999999},"last":1.5860000000000001}]},{"osd":4,"last update":"Wed Oct 11 09:49:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1879999999999999,"15min":1.1830000000000001},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.772,"5min":2.117,"15min":2.117},"last":1.8560000000000001},{"interface":"front","average":{"1min":1.095,"5min":1.1619999999999999,"15min":1.1870000000000001},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.9690000000000001,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.887}]},{"osd":6,"last update":"Wed Oct 11 09:49:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.252,"15min":1.2649999999999999},"min":{"1min":0.69499999999999995,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":2.4529999999999998},"last":1.6899999999999999},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.3080000000000001,"15min":1.3089999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.869,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":1.764}]},{"osd":7,"last update":"Wed Oct 11 09:49:42 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.298,"15min":1.3640000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.57699999999999996},"max":{"1min":2.028,"5min":2.347,"15min":3.6120000000000001},"last":2.004},{"interface":"front","average":{"1min":1.409,"5min":1.405,"15min":1.4430000000000001},"min":{"1min":0.89100000000000001,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":2.2679999999999998,"5min":2.2679999999999998,"15min":3.2109999999999999},"last":1.8080000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574964,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32332,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339508,"statfs":{"total":96636764160,"available":96603656192,"internally_reserved":0,"allocated":1314816,"data_stored":625072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:49:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.0389999999999999,"15min":1.056},"min":{"1min":0.67700000000000005,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.3939999999999999,"5min":1.579,"15min":1.7170000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.0720000000000001,"15min":1.0880000000000001},"min":{"1min":0.63700000000000001,"5min":0.60199999999999998,"15min":0.54700000000000004},"max":{"1min":1.669,"5min":1.669,"15min":2.1160000000000001},"last":1.1220000000000001}]},{"osd":1,"last update":"Wed Oct 11 09:49:50 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.0720000000000001,"15min":1.083},"min":{"1min":0.84499999999999997,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.095,"15min":1.143},"min":{"1min":0.73399999999999999,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.8220000000000001},"last":1.075}]},{"osd":2,"last update":"Wed Oct 11 09:49:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.202,"15min":1.1919999999999999},"min":{"1min":0.93000000000000005,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":1.919},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.21,"15min":1.214},"min":{"1min":0.84499999999999997,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.5620000000000001,"5min":1.8300000000000001,"15min":1.8859999999999999},"last":0.78800000000000003}]},{"osd":3,"last update":"Wed Oct 11 09:49:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.097,"15min":1.131},"min":{"1min":0.872,"5min":0.72599999999999998,"15min":0.60199999999999998},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6930000000000001},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.147,"15min":1.25},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":2.3340000000000001},"last":1.1739999999999999}]},{"osd":5,"last update":"Wed Oct 11 09:49:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.153,"15min":1.1899999999999999},"min":{"1min":0.81200000000000006,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.593,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1519999999999999,"15min":1.1910000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.61199999999999999},"max":{"1min":1.732,"5min":1.732,"15min":1.8520000000000001},"last":1.3779999999999999}]},{"osd":6,"last update":"Wed Oct 11 09:49:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.222,"15min":1.256},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.746,"5min":1.746,"15min":1.8},"last":1.298},{"interface":"front","average":{"1min":1.355,"5min":1.294,"15min":1.323},"min":{"1min":1.073,"5min":0.77200000000000002,"15min":0.63500000000000001},"max":{"1min":1.6970000000000001,"5min":1.9399999999999999,"15min":2.1120000000000001},"last":1.218}]},{"osd":7,"last update":"Wed Oct 11 09:49:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.2609999999999999,"15min":1.3120000000000001},"min":{"1min":0.78400000000000003,"5min":0.73299999999999998,"15min":0.60899999999999999},"max":{"1min":1.766,"5min":1.794,"15min":1.9470000000000001},"last":1.337},{"interface":"front","average":{"1min":1.351,"5min":1.3480000000000001,"15min":1.425},"min":{"1min":0.875,"5min":0.82299999999999995,"15min":0.75900000000000001},"max":{"1min":1.8360000000000001,"5min":1.962,"15min":1.9990000000000001},"last":1.395}]}]},{"osd":3,"up_from":13,"seq":55834574963,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1268,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1298432,"data_stored":618317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:49:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.72699999999999998,"15min":0.80900000000000005},"min":{"1min":0.45700000000000002,"5min":0.442,"15min":0.33000000000000002},"max":{"1min":1.286,"5min":1.286,"15min":1.462},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.78300000000000003,"15min":0.83799999999999997},"min":{"1min":0.59099999999999997,"5min":0.48499999999999999,"15min":0.41099999999999998},"max":{"1min":1.363,"5min":1.363,"15min":1.4950000000000001},"last":0.78200000000000003}]},{"osd":1,"last update":"Wed Oct 11 09:49:43 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.78300000000000003,"15min":0.80800000000000005},"min":{"1min":0.53900000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":1.2150000000000001},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.88400000000000001,"15min":0.92400000000000004},"min":{"1min":0.53100000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.94999999999999996}]},{"osd":2,"last update":"Wed Oct 11 09:49:43 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":0.93500000000000005,"15min":0.96899999999999997},"min":{"1min":0.65000000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.044,"5min":0.98999999999999999,"15min":1.0669999999999999},"min":{"1min":0.63700000000000001,"5min":0.58499999999999996,"15min":0.432},"max":{"1min":1.462,"5min":1.607,"15min":2.5870000000000002},"last":1.0880000000000001}]},{"osd":4,"last update":"Wed Oct 11 09:49:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.0980000000000001,"15min":1.1739999999999999},"min":{"1min":0.82999999999999996,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.4119999999999999,"5min":1.7210000000000001,"15min":2.395},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.157,"15min":1.2549999999999999},"min":{"1min":0.94399999999999995,"5min":0.78100000000000003,"15min":0.73599999999999999},"max":{"1min":1.5169999999999999,"5min":1.621,"15min":3.1579999999999999},"last":1.1259999999999999}]},{"osd":5,"last update":"Wed Oct 11 09:49:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.204,"15min":1.288},"min":{"1min":0.99399999999999999,"5min":0.84199999999999997,"15min":0.76500000000000001},"max":{"1min":1.5680000000000001,"5min":1.6699999999999999,"15min":3.0009999999999999},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.25,"5min":1.224,"15min":1.3560000000000001},"min":{"1min":0.90300000000000002,"5min":0.86199999999999999,"15min":0.67800000000000005},"max":{"1min":1.5780000000000001,"5min":1.5840000000000001,"15min":3.8050000000000002},"last":1.282}]},{"osd":6,"last update":"Wed Oct 11 09:49:43 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.1990000000000001,"15min":1.2649999999999999},"min":{"1min":0.92400000000000004,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.4259999999999999,"5min":1.587,"15min":2.1949999999999998},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.2549999999999999,"15min":1.351},"min":{"1min":1.024,"5min":0.95599999999999996,"15min":0.84499999999999997},"max":{"1min":1.6899999999999999,"5min":1.7649999999999999,"15min":3.2549999999999999},"last":1.381}]},{"osd":7,"last update":"Wed Oct 11 09:49:43 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.2789999999999999,"15min":1.399},"min":{"1min":0.97699999999999998,"5min":0.95199999999999996,"15min":0.76700000000000002},"max":{"1min":1.544,"5min":1.7150000000000001,"15min":3.7490000000000001},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.246,"15min":1.3400000000000001},"min":{"1min":0.97499999999999998,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.5049999999999999,"5min":1.661,"15min":2.625},"last":1.024}]}]},{"osd":2,"up_from":13,"seq":55834574965,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":1256,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1286144,"data_stored":616765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.77000000000000002,"15min":0.80500000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.38500000000000001},"max":{"1min":1.25,"5min":1.5920000000000001,"15min":2.0600000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.82399999999999995,"15min":0.90800000000000003},"min":{"1min":0.64100000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.288,"5min":1.571,"15min":2.8919999999999999},"last":0.92700000000000005}]},{"osd":1,"last update":"Wed Oct 11 09:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.84699999999999998,"15min":0.96599999999999997},"min":{"1min":0.59999999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.149,"5min":1.8069999999999999,"15min":3.7650000000000001},"last":0.62},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94099999999999995,"15min":0.97299999999999998},"min":{"1min":0.51500000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.4239999999999999,"5min":1.675,"15min":2.5339999999999998},"last":1.0680000000000001}]},{"osd":3,"last update":"Wed Oct 11 09:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.90400000000000003,"15min":1.034},"min":{"1min":0.58299999999999996,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.591,"5min":1.591,"15min":2.403},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.044,"5min":0.98699999999999999,"15min":1.0469999999999999},"min":{"1min":0.69199999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":2.7589999999999999},"last":1.177}]},{"osd":4,"last update":"Wed Oct 11 09:49:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.1479999999999999,"15min":1.1910000000000001},"min":{"1min":0.745,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.5349999999999999,"5min":1.7809999999999999,"15min":2.2549999999999999},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.3480000000000001},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.73399999999999999},"max":{"1min":1.4470000000000001,"5min":1.5700000000000001,"15min":2.6859999999999999},"last":1.149}]},{"osd":5,"last update":"Wed Oct 11 09:49:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.218,"15min":1.3100000000000001},"min":{"1min":0.86899999999999999,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.6499999999999999,"5min":1.76,"15min":3.044},"last":1.246},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.1779999999999999,"15min":1.272},"min":{"1min":0.78500000000000003,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.6890000000000001,"5min":1.837,"15min":3.5790000000000002},"last":1.105}]},{"osd":6,"last update":"Wed Oct 11 09:49:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.206,"15min":1.3400000000000001},"min":{"1min":0.873,"5min":0.75,"15min":0.75},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":3.1600000000000001},"last":1.367},{"interface":"front","average":{"1min":1.321,"5min":1.2889999999999999,"15min":1.337},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.623},"max":{"1min":1.8560000000000001,"5min":1.869,"15min":2.839},"last":1.3400000000000001}]},{"osd":7,"last update":"Wed Oct 11 09:49:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.252,"15min":1.365},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.774,"5min":1.774,"15min":3.1230000000000002},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.329,"15min":1.4510000000000001},"min":{"1min":0.93000000000000005,"5min":0.84899999999999998,"15min":0.80000000000000004},"max":{"1min":1.7909999999999999,"5min":1.7909999999999999,"15min":2.988},"last":1.2829999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574964,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32312,"kb_used_data":1264,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339528,"statfs":{"total":96636764160,"available":96603676672,"internally_reserved":0,"allocated":1294336,"data_stored":623384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.78500000000000003,"15min":0.78900000000000003},"min":{"1min":0.60599999999999998,"5min":0.375,"15min":0.375},"max":{"1min":1.3109999999999999,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.78900000000000003,"15min":0.82099999999999995},"min":{"1min":0.59199999999999997,"5min":0.40100000000000002,"15min":0.36699999999999999},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":0.95099999999999996}]},{"osd":2,"last update":"Wed Oct 11 09:49:41 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.81399999999999995,"15min":0.84799999999999998},"min":{"1min":0.60099999999999998,"5min":0.437,"15min":0.36199999999999999},"max":{"1min":1.306,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.89400000000000002,"15min":0.96099999999999997},"min":{"1min":0.56000000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.6619999999999999},"last":0.81999999999999995}]},{"osd":3,"last update":"Wed Oct 11 09:49:41 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":0.995,"15min":1.014},"min":{"1min":0.73399999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.714,"5min":1.714,"15min":1.714},"last":0.61599999999999999},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.95199999999999996,"15min":0.97599999999999998},"min":{"1min":0.76700000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.84999999999999998}]},{"osd":4,"last update":"Wed Oct 11 09:49:41 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.117,"15min":1.157},"min":{"1min":0.84499999999999997,"5min":0.755,"15min":0.73599999999999999},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":2.016},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.2110000000000001,"15min":1.2769999999999999},"min":{"1min":0.93799999999999994,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.55,"5min":1.665,"15min":2.323},"last":1.1919999999999999}]},{"osd":5,"last update":"Wed Oct 11 09:49:41 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.21,"15min":1.2829999999999999},"min":{"1min":0.86099999999999999,"5min":0.79300000000000004,"15min":0.65800000000000003},"max":{"1min":1.722,"5min":1.722,"15min":2.4079999999999999},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.236,"5min":1.1950000000000001,"15min":1.2410000000000001},"min":{"1min":0.96099999999999997,"5min":0.86299999999999999,"15min":0.81200000000000006},"max":{"1min":1.5660000000000001,"5min":1.677,"15min":2.1349999999999998},"last":1.4359999999999999}]},{"osd":6,"last update":"Wed Oct 11 09:49:41 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.2490000000000001,"15min":1.3320000000000001},"min":{"1min":0.996,"5min":0.84599999999999997,"15min":0.71499999999999997},"max":{"1min":1.853,"5min":2.4620000000000002,"15min":2.8940000000000001},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2569999999999999,"15min":1.329},"min":{"1min":0.93000000000000005,"5min":0.90300000000000002,"15min":0.79200000000000004},"max":{"1min":1.673,"5min":1.8129999999999999,"15min":3.3719999999999999},"last":1.371}]},{"osd":7,"last update":"Wed Oct 11 09:49:41 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3100000000000001,"15min":1.3819999999999999},"min":{"1min":0.95699999999999996,"5min":0.89700000000000002,"15min":0.86099999999999999},"max":{"1min":1.639,"5min":1.835,"15min":2.194},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.395,"5min":1.3160000000000001,"15min":1.397},"min":{"1min":1.0549999999999999,"5min":0.93300000000000005,"15min":0.89600000000000002},"max":{"1min":1.857,"5min":1.857,"15min":1.968},"last":1.399}]}]},{"osd":0,"up_from":13,"seq":55834574963,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32896,"kb_used_data":1848,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338944,"statfs":{"total":96636764160,"available":96603078656,"internally_reserved":0,"allocated":1892352,"data_stored":1214334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 11 09:49:36 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.78900000000000003,"15min":0.80200000000000005},"min":{"1min":0.53800000000000003,"5min":0.48999999999999999,"15min":0.38500000000000001},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.87,"15min":0.91300000000000003},"min":{"1min":0.66900000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.3180000000000001,"5min":1.5249999999999999,"15min":2.3100000000000001},"last":0.68799999999999994}]},{"osd":2,"last update":"Wed Oct 11 09:49:36 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.86599999999999999,"15min":0.93400000000000005},"min":{"1min":0.61699999999999999,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.357,"5min":1.357,"15min":1.7509999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.91600000000000004,"15min":0.95999999999999996},"min":{"1min":0.68799999999999994,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.2549999999999999,"5min":1.5820000000000001,"15min":1.9450000000000001},"last":0.92300000000000004}]},{"osd":3,"last update":"Wed Oct 11 09:49:36 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.79600000000000004,"15min":0.85599999999999998},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.45600000000000002},"max":{"1min":1.22,"5min":1.22,"15min":2.7639999999999998},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90300000000000002,"15min":1.0029999999999999},"min":{"1min":0.61299999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.2649999999999999,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.026}]},{"osd":4,"last update":"Wed Oct 11 09:49:36 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.147,"15min":1.1830000000000001},"min":{"1min":0.94599999999999995,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.5529999999999999,"5min":1.792,"15min":1.792},"last":1.079},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1120000000000001,"15min":1.1850000000000001},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.69799999999999995},"max":{"1min":1.411,"5min":1.5620000000000001,"15min":2.6629999999999998},"last":1.321}]},{"osd":5,"last update":"Wed Oct 11 09:49:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.1970000000000001,"15min":1.3160000000000001},"min":{"1min":0.94799999999999995,"5min":0.878,"15min":0.878},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":3.0150000000000001},"last":1.133},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.1819999999999999,"15min":1.29},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.448,"5min":1.7070000000000001,"15min":3.601},"last":1.155}]},{"osd":6,"last update":"Wed Oct 11 09:49:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.1559999999999999,"15min":1.214},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.78000000000000003},"max":{"1min":1.502,"5min":1.502,"15min":1.825},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.268,"5min":1.238,"15min":1.3149999999999999},"min":{"1min":0.95299999999999996,"5min":0.93999999999999995,"15min":0.93899999999999995},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":2.1440000000000001},"last":1.3600000000000001}]},{"osd":7,"last update":"Wed Oct 11 09:49:36 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.252,"15min":1.3100000000000001},"min":{"1min":0.76900000000000002,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.4510000000000001,"5min":1.75,"15min":1.8759999999999999},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.2370000000000001,"15min":1.3180000000000001},"min":{"1min":0.96699999999999997,"5min":0.92300000000000004,"15min":0.73899999999999999},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":2.5419999999999998},"last":1.0489999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-11T09:49:55.736 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-11T09:49:55.737 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-11T09:49:55.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.950+0000 7f6e58ef7700 1 Processor -- start 2023-10-11T09:49:55.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.950+0000 7f6e58ef7700 1 -- start start 2023-10-11T09:49:55.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.954+0000 7f6e58ef7700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6e540cf8d0 0x7f6e540b4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:55.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.954+0000 7f6e58ef7700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f6e54160870 0x7f6e54160c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:55.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.954+0000 7f6e58ef7700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e540b4fd0 con 0x7f6e540cfca0 2023-10-11T09:49:55.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.954+0000 7f6e58ef7700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e540b5870 con 0x7f6e54160870 2023-10-11T09:49:55.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.954+0000 7f6e58ef7700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e54161180 con 0x7f6e540cf8d0 2023-10-11T09:49:55.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.954+0000 7f6e52ffd700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f6e540cfca0 0x7f6e540cd5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:55000/0 (socket says 172.21.15.78:55000) 2023-10-11T09:49:55.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.954+0000 7f6e52ffd700 1 -- 172.21.15.78:0/3151746549 learned_addr learned my addr 172.21.15.78:0/3151746549 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:49:55.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.954+0000 7f6e537fe700 1 --2- 172.21.15.78:0/3151746549 >> v2:172.21.15.133:3300/0 conn(0x7f6e540cf8d0 0x7f6e540b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:55.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.954+0000 7f6e53fff700 1 --2- 172.21.15.78:0/3151746549 >> v2:172.21.15.78:3300/0 conn(0x7f6e54160870 0x7f6e54160c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:55.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.958+0000 7f6e527fc700 1 -- 172.21.15.78:0/3151746549 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1992973729 0 0) 0x7f6e540b4fd0 con 0x7f6e540cfca0 2023-10-11T09:49:55.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.958+0000 7f6e527fc700 1 -- 172.21.15.78:0/3151746549 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e3c0035d0 con 0x7f6e540cfca0 2023-10-11T09:49:55.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.958+0000 7f6e53fff700 1 -- 172.21.15.78:0/3151746549 >> v2:172.21.15.133:3300/0 conn(0x7f6e540cf8d0 msgr2=0x7f6e540b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:55.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.958+0000 7f6e53fff700 1 --2- 172.21.15.78:0/3151746549 >> v2:172.21.15.133:3300/0 conn(0x7f6e540cf8d0 0x7f6e540b4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:55.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.958+0000 7f6e53fff700 1 -- 172.21.15.78:0/3151746549 >> v1:172.21.15.78:6789/0 conn(0x7f6e540cfca0 legacy=0x7f6e540cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:55.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.958+0000 7f6e53fff700 1 -- 172.21.15.78:0/3151746549 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e540b52b0 con 0x7f6e54160870 2023-10-11T09:49:55.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.958+0000 7f6e53fff700 1 --2- 172.21.15.78:0/3151746549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e54160870 0x7f6e54160c40 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f6e540b5ab0 tx=0x7f6e4c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8244862b38791a98 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:55.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.958+0000 7f6e527fc700 1 -- 172.21.15.78:0/3151746549 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c00e070 con 0x7f6e54160870 2023-10-11T09:49:55.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.958+0000 7f6e527fc700 1 -- 172.21.15.78:0/3151746549 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4c00bb20 con 0x7f6e54160870 2023-10-11T09:49:55.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.958+0000 7f6e527fc700 1 -- 172.21.15.78:0/3151746549 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c012430 con 0x7f6e54160870 2023-10-11T09:49:55.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.958+0000 7f6e58ef7700 1 -- 172.21.15.78:0/3151746549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e54160870 msgr2=0x7f6e54160c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:55.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.958+0000 7f6e58ef7700 1 --2- 172.21.15.78:0/3151746549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e54160870 0x7f6e54160c40 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f6e540b5ab0 tx=0x7f6e4c00b330 comp rx=0 tx=0).stop 2023-10-11T09:49:55.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.958+0000 7f6e58ef7700 1 -- 172.21.15.78:0/3151746549 shutdown_connections 2023-10-11T09:49:55.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.958+0000 7f6e58ef7700 1 --2- 172.21.15.78:0/3151746549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e54160870 0x7f6e54160c40 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:55.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.958+0000 7f6e58ef7700 1 --2- 172.21.15.78:0/3151746549 >> v2:172.21.15.133:3300/0 conn(0x7f6e540cf8d0 0x7f6e540b4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:55.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.958+0000 7f6e58ef7700 1 -- 172.21.15.78:0/3151746549 >> 172.21.15.78:0/3151746549 conn(0x7f6e5406b010 msgr2=0x7f6e5406b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:55.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.958+0000 7f6e58ef7700 1 -- 172.21.15.78:0/3151746549 shutdown_connections 2023-10-11T09:49:55.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.958+0000 7f6e58ef7700 1 -- 172.21.15.78:0/3151746549 wait complete. 2023-10-11T09:49:55.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.962+0000 7f6e58ef7700 1 Processor -- start 2023-10-11T09:49:55.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.962+0000 7f6e58ef7700 1 -- start start 2023-10-11T09:49:55.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.962+0000 7f6e58ef7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e540cf8d0 0x7f6e541f66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:55.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.962+0000 7f6e58ef7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e540cfca0 0x7f6e541f9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:55.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.962+0000 7f6e58ef7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e54160870 0x7f6e5415e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:55.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.962+0000 7f6e58ef7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e541f5d00 con 0x7f6e540cf8d0 2023-10-11T09:49:55.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.962+0000 7f6e58ef7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e541fa270 con 0x7f6e540cfca0 2023-10-11T09:49:55.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.962+0000 7f6e58ef7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e541f9cb0 con 0x7f6e54160870 2023-10-11T09:49:55.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.962+0000 7f6e537fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e540cf8d0 0x7f6e541f66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:55.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.962+0000 7f6e537fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e540cf8d0 0x7f6e541f66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36148/0 (socket says 172.21.15.78:36148) 2023-10-11T09:49:55.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.962+0000 7f6e537fe700 1 -- 172.21.15.78:0/2411663738 learned_addr learned my addr 172.21.15.78:0/2411663738 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:55.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.966+0000 7f6e52ffd700 1 --2- 172.21.15.78:0/2411663738 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e540cfca0 0x7f6e541f9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:55.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.966+0000 7f6e53fff700 1 --2- 172.21.15.78:0/2411663738 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e54160870 0x7f6e5415e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:55.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.966+0000 7f6e537fe700 1 -- 172.21.15.78:0/2411663738 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e54160870 msgr2=0x7f6e5415e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:55.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.966+0000 7f6e537fe700 1 --2- 172.21.15.78:0/2411663738 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e54160870 0x7f6e5415e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:55.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.966+0000 7f6e537fe700 1 -- 172.21.15.78:0/2411663738 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e540cfca0 msgr2=0x7f6e541f9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:55.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.966+0000 7f6e537fe700 1 --2- 172.21.15.78:0/2411663738 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e540cfca0 0x7f6e541f9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:55.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.966+0000 7f6e537fe700 1 -- 172.21.15.78:0/2411663738 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e4c009a70 con 0x7f6e540cf8d0 2023-10-11T09:49:55.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.966+0000 7f6e537fe700 1 --2- 172.21.15.78:0/2411663738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e540cf8d0 0x7f6e541f66f0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f6e4400e840 tx=0x7f6e4400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:55.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.966+0000 7f6e3bfff700 1 -- 172.21.15.78:0/2411663738 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e440095a0 con 0x7f6e540cf8d0 2023-10-11T09:49:55.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.966+0000 7f6e3bfff700 1 -- 172.21.15.78:0/2411663738 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4400c690 con 0x7f6e540cf8d0 2023-10-11T09:49:55.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.966+0000 7f6e58ef7700 1 -- 172.21.15.78:0/2411663738 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e541f9f90 con 0x7f6e540cf8d0 2023-10-11T09:49:55.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.966+0000 7f6e3bfff700 1 -- 172.21.15.78:0/2411663738 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4400ca00 con 0x7f6e540cf8d0 2023-10-11T09:49:55.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.966+0000 7f6e58ef7700 1 -- 172.21.15.78:0/2411663738 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e541f61d0 con 0x7f6e540cf8d0 2023-10-11T09:49:55.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.966+0000 7f6e58ef7700 1 -- 172.21.15.78:0/2411663738 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e54065d40 con 0x7f6e540cf8d0 2023-10-11T09:49:55.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.970+0000 7f6e3bfff700 1 -- 172.21.15.78:0/2411663738 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f6e4400cb60 con 0x7f6e540cf8d0 2023-10-11T09:49:55.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.970+0000 7f6e3bfff700 1 --2- 172.21.15.78:0/2411663738 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6e40065ed0 0x7f6e40068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:55.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.970+0000 7f6e3bfff700 1 -- 172.21.15.78:0/2411663738 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6e44083920 con 0x7f6e540cf8d0 2023-10-11T09:49:55.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.970+0000 7f6e52ffd700 1 --2- 172.21.15.78:0/2411663738 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6e40065ed0 0x7f6e40068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:55.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.970+0000 7f6e52ffd700 1 --2- 172.21.15.78:0/2411663738 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6e40065ed0 0x7f6e40068380 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f6e54072160 tx=0x7f6e48009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:55.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:55.978+0000 7f6e3bfff700 1 -- 172.21.15.78:0/2411663738 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f6e44050e50 con 0x7f6e540cf8d0 2023-10-11T09:49:56.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.214+0000 7f6e58ef7700 1 -- 172.21.15.78:0/2411663738 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f6e540c9800 con 0x7f6e540cf8d0 2023-10-11T09:49:56.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.226+0000 7f6e3bfff700 1 -- 172.21.15.78:0/2411663738 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+0 (secure 0 0 0) 0x7f6e44054470 con 0x7f6e540cf8d0 2023-10-11T09:49:56.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.230+0000 7f6e58ef7700 1 -- 172.21.15.78:0/2411663738 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6e40065ed0 msgr2=0x7f6e40068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:56.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.230+0000 7f6e58ef7700 1 --2- 172.21.15.78:0/2411663738 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6e40065ed0 0x7f6e40068380 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f6e54072160 tx=0x7f6e48009040 comp rx=0 tx=0).stop 2023-10-11T09:49:56.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.230+0000 7f6e58ef7700 1 -- 172.21.15.78:0/2411663738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e540cf8d0 msgr2=0x7f6e541f66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:56.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.230+0000 7f6e58ef7700 1 --2- 172.21.15.78:0/2411663738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e540cf8d0 0x7f6e541f66f0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f6e4400e840 tx=0x7f6e4400ec00 comp rx=0 tx=0).stop 2023-10-11T09:49:56.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.230+0000 7f6e58ef7700 1 -- 172.21.15.78:0/2411663738 shutdown_connections 2023-10-11T09:49:56.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.230+0000 7f6e58ef7700 1 --2- 172.21.15.78:0/2411663738 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6e40065ed0 0x7f6e40068380 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:56.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.230+0000 7f6e58ef7700 1 --2- 172.21.15.78:0/2411663738 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e54160870 0x7f6e5415e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:56.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.230+0000 7f6e58ef7700 1 --2- 172.21.15.78:0/2411663738 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e540cfca0 0x7f6e541f9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:56.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.230+0000 7f6e58ef7700 1 --2- 172.21.15.78:0/2411663738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e540cf8d0 0x7f6e541f66f0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:56.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.230+0000 7f6e58ef7700 1 -- 172.21.15.78:0/2411663738 >> 172.21.15.78:0/2411663738 conn(0x7f6e5406b010 msgr2=0x7f6e540cd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:56.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.230+0000 7f6e58ef7700 1 -- 172.21.15.78:0/2411663738 shutdown_connections 2023-10-11T09:49:56.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.230+0000 7f6e58ef7700 1 -- 172.21.15.78:0/2411663738 wait complete. 2023-10-11T09:49:56.241 INFO:teuthology.orchestra.run.smithi078.stderr:updated caps for client.0 2023-10-11T09:49:56.260 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:49:56.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.430+0000 7f3e290d5700 1 Processor -- start 2023-10-11T09:49:56.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.430+0000 7f3e290d5700 1 -- start start 2023-10-11T09:49:56.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.434+0000 7f3e290d5700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f3e2415b8e0 0x7f3e2406d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:56.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.434+0000 7f3e290d5700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3e240ca950 0x7f3e2413fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:56.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.434+0000 7f3e290d5700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e2415e010 con 0x7f3e2406d610 2023-10-11T09:49:56.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.434+0000 7f3e290d5700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e2415e280 con 0x7f3e2415b8e0 2023-10-11T09:49:56.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.434+0000 7f3e290d5700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e2415e560 con 0x7f3e240ca950 2023-10-11T09:49:56.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.434+0000 7f3e23fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f3e2415b8e0 0x7f3e2406d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:56.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.434+0000 7f3e23fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f3e2415b8e0 0x7f3e2406d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36154/0 (socket says 172.21.15.78:36154) 2023-10-11T09:49:56.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.434+0000 7f3e23fff700 1 -- 172.21.15.78:0/4055577053 learned_addr learned my addr 172.21.15.78:0/4055577053 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:56.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.434+0000 7f3e288d4700 1 --2- 172.21.15.78:0/4055577053 >> v2:172.21.15.133:3300/0 conn(0x7f3e240ca950 0x7f3e2413fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:56.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.434+0000 7f3e23fff700 1 -- 172.21.15.78:0/4055577053 >> v2:172.21.15.133:3300/0 conn(0x7f3e240ca950 msgr2=0x7f3e2413fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:56.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.434+0000 7f3e23fff700 1 --2- 172.21.15.78:0/4055577053 >> v2:172.21.15.133:3300/0 conn(0x7f3e240ca950 0x7f3e2413fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:56.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.434+0000 7f3e23fff700 1 -- 172.21.15.78:0/4055577053 >> v1:172.21.15.78:6789/0 conn(0x7f3e2406d610 legacy=0x7f3e240ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:56.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.434+0000 7f3e23fff700 1 -- 172.21.15.78:0/4055577053 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e2415e840 con 0x7f3e2415b8e0 2023-10-11T09:49:56.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.434+0000 7f3e23fff700 1 --2- 172.21.15.78:0/4055577053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e2415b8e0 0x7f3e2406d0d0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f3e2415efb0 tx=0x7f3e14009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e310af982cb144fc server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:56.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.434+0000 7f3e237fe700 1 -- 172.21.15.78:0/4055577053 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1400b8a0 con 0x7f3e2415b8e0 2023-10-11T09:49:56.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.434+0000 7f3e237fe700 1 -- 172.21.15.78:0/4055577053 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e1400ba00 con 0x7f3e2415b8e0 2023-10-11T09:49:56.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.434+0000 7f3e237fe700 1 -- 172.21.15.78:0/4055577053 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1400bd90 con 0x7f3e2415b8e0 2023-10-11T09:49:56.444 INFO:tasks.ceph.mds.c.smithi078.stderr:2023-10-11T09:49:56.434+0000 7fecdb37f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-11T09:49:56.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.434+0000 7f3e290d5700 1 -- 172.21.15.78:0/4055577053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e2415b8e0 msgr2=0x7f3e2406d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:56.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.434+0000 7f3e290d5700 1 --2- 172.21.15.78:0/4055577053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e2415b8e0 0x7f3e2406d0d0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f3e2415efb0 tx=0x7f3e14009bf0 comp rx=0 tx=0).stop 2023-10-11T09:49:56.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.434+0000 7f3e290d5700 1 -- 172.21.15.78:0/4055577053 shutdown_connections 2023-10-11T09:49:56.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.434+0000 7f3e290d5700 1 --2- 172.21.15.78:0/4055577053 >> v2:172.21.15.133:3300/0 conn(0x7f3e240ca950 0x7f3e2413fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:56.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.434+0000 7f3e290d5700 1 --2- 172.21.15.78:0/4055577053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e2415b8e0 0x7f3e2406d0d0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:56.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.434+0000 7f3e290d5700 1 -- 172.21.15.78:0/4055577053 >> 172.21.15.78:0/4055577053 conn(0x7f3e240cea60 msgr2=0x7f3e240cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:56.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.434+0000 7f3e290d5700 1 -- 172.21.15.78:0/4055577053 shutdown_connections 2023-10-11T09:49:56.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.434+0000 7f3e290d5700 1 -- 172.21.15.78:0/4055577053 wait complete. 2023-10-11T09:49:56.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.438+0000 7f3e290d5700 1 Processor -- start 2023-10-11T09:49:56.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.438+0000 7f3e290d5700 1 -- start start 2023-10-11T09:49:56.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.442+0000 7f3e290d5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e2406d610 0x7f3e24159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:56.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.442+0000 7f3e290d5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e240ca950 0x7f3e240ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:56.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.442+0000 7f3e290d5700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3e2415b8e0 0x7f3e240cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:56.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.442+0000 7f3e290d5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2415a300 con 0x7f3e2406d610 2023-10-11T09:49:56.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.442+0000 7f3e290d5700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e240ae2a0 con 0x7f3e240ca950 2023-10-11T09:49:56.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.442+0000 7f3e290d5700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e240ad200 con 0x7f3e2415b8e0 2023-10-11T09:49:56.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.442+0000 7f3e23fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e2406d610 0x7f3e24159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:56.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.442+0000 7f3e23fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e2406d610 0x7f3e24159dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36160/0 (socket says 172.21.15.78:36160) 2023-10-11T09:49:56.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.442+0000 7f3e23fff700 1 -- 172.21.15.78:0/3857797745 learned_addr learned my addr 172.21.15.78:0/3857797745 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:56.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.442+0000 7f3e288d4700 1 --2- 172.21.15.78:0/3857797745 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3e2415b8e0 0x7f3e240cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:56.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.442+0000 7f3e1b7fe700 1 --2- 172.21.15.78:0/3857797745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e240ca950 0x7f3e240ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:56.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.442+0000 7f3e23fff700 1 -- 172.21.15.78:0/3857797745 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3e2415b8e0 msgr2=0x7f3e240cb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:56.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.442+0000 7f3e23fff700 1 --2- 172.21.15.78:0/3857797745 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3e2415b8e0 0x7f3e240cb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:56.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.442+0000 7f3e23fff700 1 -- 172.21.15.78:0/3857797745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e240ca950 msgr2=0x7f3e240ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:56.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.442+0000 7f3e23fff700 1 --2- 172.21.15.78:0/3857797745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e240ca950 0x7f3e240ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:56.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.442+0000 7f3e23fff700 1 -- 172.21.15.78:0/3857797745 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e14009570 con 0x7f3e2406d610 2023-10-11T09:49:56.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.442+0000 7f3e288d4700 1 --2- 172.21.15.78:0/3857797745 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3e2415b8e0 0x7f3e240cb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:49:56.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.442+0000 7f3e1b7fe700 1 --2- 172.21.15.78:0/3857797745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e240ca950 0x7f3e240ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:49:56.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.442+0000 7f3e23fff700 1 --2- 172.21.15.78:0/3857797745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e2406d610 0x7f3e24159dc0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f3e14000c00 tx=0x7f3e14004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:56.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.442+0000 7f3e217fa700 1 -- 172.21.15.78:0/3857797745 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e14003ad0 con 0x7f3e2406d610 2023-10-11T09:49:56.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.442+0000 7f3e217fa700 1 -- 172.21.15.78:0/3857797745 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e1401e070 con 0x7f3e2406d610 2023-10-11T09:49:56.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.442+0000 7f3e217fa700 1 -- 172.21.15.78:0/3857797745 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e14003e90 con 0x7f3e2406d610 2023-10-11T09:49:56.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.442+0000 7f3e290d5700 1 -- 172.21.15.78:0/3857797745 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e240ad4e0 con 0x7f3e2406d610 2023-10-11T09:49:56.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.446+0000 7f3e290d5700 1 -- 172.21.15.78:0/3857797745 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e240adaa0 con 0x7f3e2406d610 2023-10-11T09:49:56.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.446+0000 7f3e290d5700 1 -- 172.21.15.78:0/3857797745 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e24065d40 con 0x7f3e2406d610 2023-10-11T09:49:56.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.446+0000 7f3e217fa700 1 -- 172.21.15.78:0/3857797745 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f3e14011420 con 0x7f3e2406d610 2023-10-11T09:49:56.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.450+0000 7f3e217fa700 1 --2- 172.21.15.78:0/3857797745 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3e04065ed0 0x7f3e04068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:56.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.450+0000 7f3e217fa700 1 -- 172.21.15.78:0/3857797745 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3e14020070 con 0x7f3e2406d610 2023-10-11T09:49:56.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.450+0000 7f3e1b7fe700 1 --2- 172.21.15.78:0/3857797745 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3e04065ed0 0x7f3e04068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:56.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.450+0000 7f3e1b7fe700 1 --2- 172.21.15.78:0/3857797745 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3e04065ed0 0x7f3e04068380 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f3e240ae700 tx=0x7f3e0c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:56.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.454+0000 7f3e217fa700 1 -- 172.21.15.78:0/3857797745 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f3e140526d0 con 0x7f3e2406d610 2023-10-11T09:49:56.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.666+0000 7f3e290d5700 1 -- 172.21.15.78:0/3857797745 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e240b1030 con 0x7f3e2406d610 2023-10-11T09:49:56.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.666+0000 7f3e217fa700 1 -- 172.21.15.78:0/3857797745 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4656 (secure 0 0 0) 0x7f3e14055cf0 con 0x7f3e2406d610 2023-10-11T09:49:56.675 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:49:56.675 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":79,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6812,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/3697741427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3697741427},{"type":"v1","addr":"172.21.15.133:6833","nonce":3697741427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":6815,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2640710924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2640710924},{"type":"v1","addr":"172.21.15.133:6835","nonce":2640710924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8028,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/1538320637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":1538320637},{"type":"v1","addr":"172.21.15.78:6835","nonce":1538320637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:49:50.956153+0000","modified":"2023-10-11T09:49:53.977383+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8031},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8031":{"gid":8031,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6837/1286798022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1286798022},{"type":"v1","addr":"172.21.15.78:6837","nonce":1286798022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-11T09:49:56.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.670+0000 7f3e290d5700 1 -- 172.21.15.78:0/3857797745 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3e04065ed0 msgr2=0x7f3e04068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:56.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.670+0000 7f3e290d5700 1 --2- 172.21.15.78:0/3857797745 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3e04065ed0 0x7f3e04068380 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f3e240ae700 tx=0x7f3e0c009040 comp rx=0 tx=0).stop 2023-10-11T09:49:56.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.670+0000 7f3e290d5700 1 -- 172.21.15.78:0/3857797745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e2406d610 msgr2=0x7f3e24159dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:56.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.670+0000 7f3e290d5700 1 --2- 172.21.15.78:0/3857797745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e2406d610 0x7f3e24159dc0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f3e14000c00 tx=0x7f3e14004770 comp rx=0 tx=0).stop 2023-10-11T09:49:56.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.670+0000 7f3e290d5700 1 -- 172.21.15.78:0/3857797745 shutdown_connections 2023-10-11T09:49:56.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.670+0000 7f3e290d5700 1 --2- 172.21.15.78:0/3857797745 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3e04065ed0 0x7f3e04068380 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:56.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.670+0000 7f3e290d5700 1 --2- 172.21.15.78:0/3857797745 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3e2415b8e0 0x7f3e240cb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:56.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.670+0000 7f3e290d5700 1 --2- 172.21.15.78:0/3857797745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e240ca950 0x7f3e240ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:56.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.670+0000 7f3e290d5700 1 --2- 172.21.15.78:0/3857797745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e2406d610 0x7f3e24159dc0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:56.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.670+0000 7f3e290d5700 1 -- 172.21.15.78:0/3857797745 >> 172.21.15.78:0/3857797745 conn(0x7f3e240cea60 msgr2=0x7f3e240cee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:56.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.670+0000 7f3e290d5700 1 -- 172.21.15.78:0/3857797745 shutdown_connections 2023-10-11T09:49:56.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:56.670+0000 7f3e290d5700 1 -- 172.21.15.78:0/3857797745 wait complete. 2023-10-11T09:49:56.680 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 79 2023-10-11T09:49:57.101 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 79, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-11T09:49:50.956153+0000', 'modified': '2023-10-11T09:49:53.977383+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8031}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8031': {'gid': 8031, 'name': 'c', 'rank': 0, 'incarnation': 75, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.78:6837/1286798022', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6836', 'nonce': 1286798022}, {'type': 'v1', 'addr': '172.21.15.78:6837', 'nonce': 1286798022}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-11T09:49:57.102 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-11T09:49:57.102 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-11T09:49:57.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.274+0000 7f7d26d37700 1 Processor -- start 2023-10-11T09:49:57.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.274+0000 7f7d26d37700 1 -- start start 2023-10-11T09:49:57.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.278+0000 7f7d26d37700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f7d200d1660 0x7f7d200d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:57.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.278+0000 7f7d26d37700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d2015dc10 con 0x7f7d200d28e0 2023-10-11T09:49:57.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.278+0000 7f7d26d37700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d20160380 con 0x7f7d20156ce0 2023-10-11T09:49:57.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.278+0000 7f7d26d37700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d2015d930 con 0x7f7d200d1660 2023-10-11T09:49:57.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.278+0000 7f7d25534700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f7d200d1660 0x7f7d200d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:57.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.278+0000 7f7d25534700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f7d200d1660 0x7f7d200d0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36168/0 (socket says 172.21.15.78:36168) 2023-10-11T09:49:57.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.278+0000 7f7d25534700 1 -- 172.21.15.78:0/3917511816 learned_addr learned my addr 172.21.15.78:0/3917511816 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:57.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.278+0000 7f7d26536700 1 --1- 172.21.15.78:0/3917511816 >> v1:172.21.15.78:6789/0 conn(0x7f7d200d28e0 0x7f7d200cefc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:55012/0 (socket says 172.21.15.78:55012) 2023-10-11T09:49:57.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.278+0000 7f7d25534700 1 -- 172.21.15.78:0/3917511816 >> v1:172.21.15.133:6789/0 conn(0x7f7d20156ce0 legacy=0x7f7d20160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:57.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.278+0000 7f7d25534700 1 -- 172.21.15.78:0/3917511816 >> v1:172.21.15.78:6789/0 conn(0x7f7d200d28e0 legacy=0x7f7d200cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:57.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.278+0000 7f7d25534700 1 -- 172.21.15.78:0/3917511816 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d20160620 con 0x7f7d200d1660 2023-10-11T09:49:57.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.278+0000 7f7d25534700 1 --2- 172.21.15.78:0/3917511816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d200d1660 0x7f7d200d0a50 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009910 tx=0x7f7d10009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d9d0b4b0c6709b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:57.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.278+0000 7f7d24d33700 1 -- 172.21.15.78:0/3917511816 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1000ba60 con 0x7f7d200d1660 2023-10-11T09:49:57.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.278+0000 7f7d24d33700 1 -- 172.21.15.78:0/3917511816 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1000bbc0 con 0x7f7d200d1660 2023-10-11T09:49:57.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.278+0000 7f7d24d33700 1 -- 172.21.15.78:0/3917511816 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d10009e60 con 0x7f7d200d1660 2023-10-11T09:49:57.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.282+0000 7f7d26d37700 1 -- 172.21.15.78:0/3917511816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d200d1660 msgr2=0x7f7d200d0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:57.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.282+0000 7f7d26d37700 1 --2- 172.21.15.78:0/3917511816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d200d1660 0x7f7d200d0a50 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009910 tx=0x7f7d10009c20 comp rx=0 tx=0).stop 2023-10-11T09:49:57.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.282+0000 7f7d26d37700 1 -- 172.21.15.78:0/3917511816 shutdown_connections 2023-10-11T09:49:57.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.282+0000 7f7d26d37700 1 --2- 172.21.15.78:0/3917511816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d200d1660 0x7f7d200d0a50 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.282+0000 7f7d26d37700 1 -- 172.21.15.78:0/3917511816 >> 172.21.15.78:0/3917511816 conn(0x7f7d2006afb0 msgr2=0x7f7d2006b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:57.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.282+0000 7f7d26d37700 1 -- 172.21.15.78:0/3917511816 shutdown_connections 2023-10-11T09:49:57.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.282+0000 7f7d26d37700 1 -- 172.21.15.78:0/3917511816 wait complete. 2023-10-11T09:49:57.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.282+0000 7f7d26d37700 1 Processor -- start 2023-10-11T09:49:57.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.282+0000 7f7d26d37700 1 -- start start 2023-10-11T09:49:57.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.282+0000 7f7d26d37700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d200d1660 0x7f7d200c69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:57.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.282+0000 7f7d26d37700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7d200d28e0 0x7f7d200c8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:57.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.282+0000 7f7d26d37700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d20156ce0 0x7f7d20152730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:57.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.282+0000 7f7d26d37700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d200c6f90 con 0x7f7d200d1660 2023-10-11T09:49:57.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.282+0000 7f7d26d37700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d200c7270 con 0x7f7d20156ce0 2023-10-11T09:49:57.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.282+0000 7f7d26d37700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d200c7550 con 0x7f7d200d28e0 2023-10-11T09:49:57.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.286+0000 7f7d25d35700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d200d1660 0x7f7d200c69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:57.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.286+0000 7f7d25d35700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d200d1660 0x7f7d200c69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36170/0 (socket says 172.21.15.78:36170) 2023-10-11T09:49:57.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.286+0000 7f7d25d35700 1 -- 172.21.15.78:0/1855996802 learned_addr learned my addr 172.21.15.78:0/1855996802 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:57.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.286+0000 7f7d25534700 1 --2- 172.21.15.78:0/1855996802 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7d200d28e0 0x7f7d200c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:57.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.286+0000 7f7d26536700 1 --2- 172.21.15.78:0/1855996802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d20156ce0 0x7f7d20152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:57.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.286+0000 7f7d25d35700 1 -- 172.21.15.78:0/1855996802 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7d200d28e0 msgr2=0x7f7d200c8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:57.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.286+0000 7f7d25d35700 1 --2- 172.21.15.78:0/1855996802 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7d200d28e0 0x7f7d200c8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.286+0000 7f7d25d35700 1 -- 172.21.15.78:0/1855996802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d20156ce0 msgr2=0x7f7d20152730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:57.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.286+0000 7f7d25d35700 1 --2- 172.21.15.78:0/1855996802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d20156ce0 0x7f7d20152730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.286+0000 7f7d25d35700 1 -- 172.21.15.78:0/1855996802 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d10009570 con 0x7f7d200d1660 2023-10-11T09:49:57.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.286+0000 7f7d26536700 1 --2- 172.21.15.78:0/1855996802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d20156ce0 0x7f7d20152730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:49:57.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.286+0000 7f7d25d35700 1 --2- 172.21.15.78:0/1855996802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d200d1660 0x7f7d200c69b0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c00c500 tx=0x7f7d1c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:57.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.286+0000 7f7d167fc700 1 -- 172.21.15.78:0/1855996802 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c0052d0 con 0x7f7d200d1660 2023-10-11T09:49:57.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.286+0000 7f7d167fc700 1 -- 172.21.15.78:0/1855996802 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1c005430 con 0x7f7d200d1660 2023-10-11T09:49:57.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.286+0000 7f7d167fc700 1 -- 172.21.15.78:0/1855996802 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c0055a0 con 0x7f7d200d1660 2023-10-11T09:49:57.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.286+0000 7f7d26d37700 1 -- 172.21.15.78:0/1855996802 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d200c7830 con 0x7f7d200d1660 2023-10-11T09:49:57.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.286+0000 7f7d26d37700 1 -- 172.21.15.78:0/1855996802 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d20152c70 con 0x7f7d200d1660 2023-10-11T09:49:57.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.290+0000 7f7d167fc700 1 -- 172.21.15.78:0/1855996802 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f7d1c0038e0 con 0x7f7d200d1660 2023-10-11T09:49:57.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.290+0000 7f7d167fc700 1 --2- 172.21.15.78:0/1855996802 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7d04065ed0 0x7f7d04068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:57.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.290+0000 7f7d167fc700 1 -- 172.21.15.78:0/1855996802 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7d1c082c60 con 0x7f7d200d1660 2023-10-11T09:49:57.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.290+0000 7f7d25534700 1 --2- 172.21.15.78:0/1855996802 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7d04065ed0 0x7f7d04068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:57.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.294+0000 7f7d25534700 1 --2- 172.21.15.78:0/1855996802 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7d04065ed0 0x7f7d04068380 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7d2008c200 tx=0x7f7d10008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:57.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.294+0000 7f7d2b224740 1 Processor -- start 2023-10-11T09:49:57.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.294+0000 7f7d2b224740 1 -- start start 2023-10-11T09:49:57.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.294+0000 7f7d2b224740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2469eb0 0x2722020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:57.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.294+0000 7f7d2b224740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2469ae0 0x2722560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:57.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.294+0000 7f7d2b224740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2469710 0x26e9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:57.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.294+0000 7f7d2b224740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x26494a0 con 0x2469710 2023-10-11T09:49:57.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.294+0000 7f7d2b224740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x2646c30 con 0x2469ae0 2023-10-11T09:49:57.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.294+0000 7f7d2b224740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x2642520 con 0x2469eb0 2023-10-11T09:49:57.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.294+0000 7f7d26536700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2469710 0x26e9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:57.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.294+0000 7f7d26536700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2469710 0x26e9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36172/0 (socket says 172.21.15.78:36172) 2023-10-11T09:49:57.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.294+0000 7f7d26536700 1 -- 172.21.15.78:0/1147536974 learned_addr learned my addr 172.21.15.78:0/1147536974 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:57.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.294+0000 7f7d25d35700 1 --2- 172.21.15.78:0/1147536974 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2469eb0 0x2722020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:57.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.294+0000 7f7d26536700 1 -- 172.21.15.78:0/1147536974 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2469eb0 msgr2=0x2722020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:57.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.294+0000 7f7d25534700 1 --2- 172.21.15.78:0/1147536974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2469ae0 0x2722560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:57.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.294+0000 7f7d26536700 1 --2- 172.21.15.78:0/1147536974 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2469eb0 0x2722020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.294+0000 7f7d26536700 1 -- 172.21.15.78:0/1147536974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2469ae0 msgr2=0x2722560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:57.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.294+0000 7f7d26536700 1 --2- 172.21.15.78:0/1147536974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2469ae0 0x2722560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.294+0000 7f7d26536700 1 -- 172.21.15.78:0/1147536974 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x24b59e0 con 0x2469710 2023-10-11T09:49:57.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.294+0000 7f7d25d35700 1 --2- 172.21.15.78:0/1147536974 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2469eb0 0x2722020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:49:57.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.294+0000 7f7d25534700 1 --2- 172.21.15.78:0/1147536974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2469ae0 0x2722560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:49:57.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.294+0000 7f7d26536700 1 --2- 172.21.15.78:0/1147536974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2469710 0x26e9640 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f7d18002230 tx=0x7f7d18007330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=555b19b34a7da68a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:57.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.298+0000 7f7d037fe700 1 -- 172.21.15.78:0/1147536974 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d18007a60 con 0x2469710 2023-10-11T09:49:57.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.298+0000 7f7d037fe700 1 -- 172.21.15.78:0/1147536974 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d180114f0 con 0x2469710 2023-10-11T09:49:57.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.298+0000 7f7d037fe700 1 -- 172.21.15.78:0/1147536974 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d18011860 con 0x2469710 2023-10-11T09:49:57.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.298+0000 7f7d2b224740 1 -- 172.21.15.78:0/1147536974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2469710 msgr2=0x26e9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:57.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.298+0000 7f7d2b224740 1 --2- 172.21.15.78:0/1147536974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2469710 0x26e9640 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f7d18002230 tx=0x7f7d18007330 comp rx=0 tx=0).stop 2023-10-11T09:49:57.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.298+0000 7f7d2b224740 1 -- 172.21.15.78:0/1147536974 shutdown_connections 2023-10-11T09:49:57.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.298+0000 7f7d2b224740 1 --2- 172.21.15.78:0/1147536974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2469710 0x26e9640 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.298+0000 7f7d2b224740 1 --2- 172.21.15.78:0/1147536974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2469ae0 0x2722560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.298+0000 7f7d2b224740 1 --2- 172.21.15.78:0/1147536974 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2469eb0 0x2722020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.298+0000 7f7d2b224740 1 -- 172.21.15.78:0/1147536974 >> 172.21.15.78:0/1147536974 conn(0x26d1fa0 msgr2=0x2703420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:57.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.298+0000 7f7d2b224740 1 -- 172.21.15.78:0/1147536974 shutdown_connections 2023-10-11T09:49:57.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.298+0000 7f7d2b224740 1 -- 172.21.15.78:0/1147536974 wait complete. 2023-10-11T09:49:57.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.298+0000 7f7d2b224740 1 -- start start 2023-10-11T09:49:57.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.298+0000 7f7d2b224740 1 Processor -- start 2023-10-11T09:49:57.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.298+0000 7f7d2b224740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2469710 0x26fbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:57.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.298+0000 7f7d2b224740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2469ae0 0x2717ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:57.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.298+0000 7f7d2b224740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2469eb0 0x2717ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:57.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.298+0000 7f7d2b224740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x2605b00 con 0x2469eb0 2023-10-11T09:49:57.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.298+0000 7f7d2b224740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x26057a0 con 0x2469710 2023-10-11T09:49:57.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.298+0000 7f7d2b224740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x24eeb20 con 0x2469ae0 2023-10-11T09:49:57.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.298+0000 7f7d2b224740 10 client.0 _ll_register_callbacks cb 0x263be60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:49:57.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.302+0000 7f7d26536700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2469eb0 0x2717ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:57.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.302+0000 7f7d26536700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2469eb0 0x2717ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36188/0 (socket says 172.21.15.78:36188) 2023-10-11T09:49:57.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.302+0000 7f7d26536700 1 -- 172.21.15.78:0/3609375011 learned_addr learned my addr 172.21.15.78:0/3609375011 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:57.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.302+0000 7f7d25d35700 1 --2- 172.21.15.78:0/3609375011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2469710 0x26fbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:57.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.302+0000 7f7d25534700 1 --2- 172.21.15.78:0/3609375011 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2469ae0 0x2717ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:57.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.302+0000 7f7d26536700 1 -- 172.21.15.78:0/3609375011 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2469ae0 msgr2=0x2717ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:57.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.302+0000 7f7d26536700 1 --2- 172.21.15.78:0/3609375011 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2469ae0 0x2717ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.302+0000 7f7d26536700 1 -- 172.21.15.78:0/3609375011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2469710 msgr2=0x26fbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:57.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.302+0000 7f7d26536700 1 --2- 172.21.15.78:0/3609375011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2469710 0x26fbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.302+0000 7f7d26536700 1 -- 172.21.15.78:0/3609375011 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d1800f040 con 0x2469eb0 2023-10-11T09:49:57.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.302+0000 7f7d26536700 1 --2- 172.21.15.78:0/3609375011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2469eb0 0x2717ff0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f7d18001940 tx=0x7f7d18001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a022494fd571dfd server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:57.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.302+0000 7f7d00ff9700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:49:57.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.302+0000 7f7d00ff9700 1 -- 172.21.15.78:0/3609375011 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d18007a60 con 0x2469eb0 2023-10-11T09:49:57.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.302+0000 7f7d2b224740 1 -- 172.21.15.78:0/3609375011 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x263b5a0 con 0x2469eb0 2023-10-11T09:49:57.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.302+0000 7f7d00ff9700 1 -- 172.21.15.78:0/3609375011 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7d18011cd0 con 0x2469eb0 2023-10-11T09:49:57.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.302+0000 7f7d00ff9700 1 -- 172.21.15.78:0/3609375011 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d18011e90 con 0x2469eb0 2023-10-11T09:49:57.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.302+0000 7f7d2b224740 10 client.8181 fetch_fsmap learned FSMap version 79 2023-10-11T09:49:57.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.302+0000 7f7d2b224740 1 -- 172.21.15.78:0/3609375011 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x2608750 con 0x2469eb0 2023-10-11T09:49:57.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.302+0000 7f7d00ff9700 1 -- 172.21.15.78:0/3609375011 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x263b5a0 con 0x2469eb0 2023-10-11T09:49:57.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.302+0000 7f7d00ff9700 1 -- 172.21.15.78:0/3609375011 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 79) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f7d18014480 con 0x2469eb0 2023-10-11T09:49:57.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.302+0000 7f7d2b224740 10 client.8181 fetch_fsmap finished waiting for FSMap version 79 2023-10-11T09:49:57.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.302+0000 7f7d2b224740 10 client.8181 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-11T09:49:57.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.302+0000 7f7d2b224740 20 client.8181 populate_metadata read hostname 'smithi078' 2023-10-11T09:49:57.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.302+0000 7f7d2b224740 1 --2- 172.21.15.78:0/3609375011 >> [v2:172.21.15.78:6836/1286798022,v1:172.21.15.78:6837/1286798022] conn(0x24b4790 0x26ff250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:57.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.302+0000 7f7d2b224740 4 client.8181 mds_command: new command op to 8031 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-11T09:49:57.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.302+0000 7f7d2b224740 1 -- 172.21.15.78:0/3609375011 --> [v2:172.21.15.78:6836/1286798022,v1:172.21.15.78:6837/1286798022] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x263b2c0 con 0x24b4790 2023-10-11T09:49:57.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.302+0000 7f7d25d35700 1 --2- 172.21.15.78:0/3609375011 >> [v2:172.21.15.78:6836/1286798022,v1:172.21.15.78:6837/1286798022] conn(0x24b4790 0x26ff250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:57.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.302+0000 7f7d00ff9700 10 client.8181 ms_handle_connect on v2:172.21.15.78:6836/1286798022 2023-10-11T09:49:57.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.302+0000 7f7d25d35700 1 --2- 172.21.15.78:0/3609375011 >> [v2:172.21.15.78:6836/1286798022,v1:172.21.15.78:6837/1286798022] conn(0x24b4790 0x26ff250 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e342f920f6f2f7d server_cookie=c85f1dea97d1450 in_seq=0 out_seq=0 2023-10-11T09:49:57.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.306+0000 7f7d00ff9700 1 -- 172.21.15.78:0/3609375011 <== mds.0 v2:172.21.15.78:6836/1286798022 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x263b2c0 con 0x24b4790 2023-10-11T09:49:57.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.306+0000 7f7d00ff9700 10 client.8181 handle_command_reply: tid=0 2023-10-11T09:49:57.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.306+0000 7f7d2b224740 1 client.8181 shutdown 2023-10-11T09:49:57.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.306+0000 7f7d2b224740 1 -- 172.21.15.78:0/3609375011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2469eb0 msgr2=0x2717ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:57.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.306+0000 7f7d2b224740 1 --2- 172.21.15.78:0/3609375011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2469eb0 0x2717ff0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f7d18001940 tx=0x7f7d18001250 comp rx=0 tx=0).stop 2023-10-11T09:49:57.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.306+0000 7f7d2b224740 1 -- 172.21.15.78:0/3609375011 shutdown_connections 2023-10-11T09:49:57.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.306+0000 7f7d2b224740 1 --2- 172.21.15.78:0/3609375011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2469eb0 0x2717ff0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.306+0000 7f7d2b224740 1 --2- 172.21.15.78:0/3609375011 >> [v2:172.21.15.78:6836/1286798022,v1:172.21.15.78:6837/1286798022] conn(0x24b4790 0x26ff250 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.306+0000 7f7d2b224740 1 --2- 172.21.15.78:0/3609375011 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2469ae0 0x2717ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.306+0000 7f7d2b224740 1 --2- 172.21.15.78:0/3609375011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2469710 0x26fbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.306+0000 7f7d2b224740 1 -- 172.21.15.78:0/3609375011 >> 172.21.15.78:0/3609375011 conn(0x26d1fa0 msgr2=0x26f6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:57.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.306+0000 7f7d00ff9700 0 client.8181 ms_handle_reset on v2:172.21.15.78:6836/1286798022 2023-10-11T09:49:57.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.306+0000 7f7d2b224740 1 -- 172.21.15.78:0/3609375011 shutdown_connections 2023-10-11T09:49:57.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.306+0000 7f7d2b224740 1 -- 172.21.15.78:0/3609375011 wait complete. 2023-10-11T09:49:57.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.306+0000 7f7d2b224740 20 client.8181 trim_cache size 0 max 16384 2023-10-11T09:49:57.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.326+0000 7f7d2b224740 1 Processor -- start 2023-10-11T09:49:57.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.326+0000 7f7d2b224740 1 -- start start 2023-10-11T09:49:57.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.326+0000 7f7d2b224740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2469710 0x26db900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:57.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.326+0000 7f7d2b224740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2469ae0 0x26dbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:57.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.326+0000 7f7d2b224740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2469eb0 0x27176f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:57.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.326+0000 7f7d2b224740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x27039d0 con 0x2469710 2023-10-11T09:49:57.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.326+0000 7f7d2b224740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x2645e00 con 0x2469eb0 2023-10-11T09:49:57.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.326+0000 7f7d2b224740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x2641d30 con 0x2469ae0 2023-10-11T09:49:57.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.326+0000 7f7d26536700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2469eb0 0x27176f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:57.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.326+0000 7f7d25d35700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2469710 0x26db900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:57.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.326+0000 7f7d26536700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2469eb0 0x27176f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:47832/0 (socket says 172.21.15.78:47832) 2023-10-11T09:49:57.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.326+0000 7f7d26536700 1 -- 172.21.15.78:0/1236359317 learned_addr learned my addr 172.21.15.78:0/1236359317 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:57.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.326+0000 7f7d25534700 1 --2- 172.21.15.78:0/1236359317 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2469ae0 0x26dbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:57.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.326+0000 7f7d26536700 1 -- 172.21.15.78:0/1236359317 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2469ae0 msgr2=0x26dbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:57.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.326+0000 7f7d26536700 1 --2- 172.21.15.78:0/1236359317 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2469ae0 0x26dbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.326+0000 7f7d26536700 1 -- 172.21.15.78:0/1236359317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2469710 msgr2=0x26db900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:57.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.326+0000 7f7d26536700 1 --2- 172.21.15.78:0/1236359317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2469710 0x26db900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.326+0000 7f7d26536700 1 -- 172.21.15.78:0/1236359317 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d2015ea70 con 0x2469eb0 2023-10-11T09:49:57.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.330+0000 7f7d26536700 1 --2- 172.21.15.78:0/1236359317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2469eb0 0x27176f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7d18000f80 tx=0x7f7d1801e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c85e4fec2e41437 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:57.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.330+0000 7f7d03fff700 1 -- 172.21.15.78:0/1236359317 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d18001970 con 0x2469eb0 2023-10-11T09:49:57.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.330+0000 7f7d03fff700 1 -- 172.21.15.78:0/1236359317 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1800b040 con 0x2469eb0 2023-10-11T09:49:57.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.330+0000 7f7d03fff700 1 -- 172.21.15.78:0/1236359317 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d18011940 con 0x2469eb0 2023-10-11T09:49:57.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.330+0000 7f7d2b224740 1 -- 172.21.15.78:0/1236359317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2469eb0 msgr2=0x27176f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:57.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.330+0000 7f7d2b224740 1 --2- 172.21.15.78:0/1236359317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2469eb0 0x27176f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7d18000f80 tx=0x7f7d1801e000 comp rx=0 tx=0).stop 2023-10-11T09:49:57.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.330+0000 7f7d2b224740 1 -- 172.21.15.78:0/1236359317 shutdown_connections 2023-10-11T09:49:57.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.330+0000 7f7d2b224740 1 --2- 172.21.15.78:0/1236359317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2469eb0 0x27176f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.330+0000 7f7d2b224740 1 --2- 172.21.15.78:0/1236359317 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2469ae0 0x26dbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.330+0000 7f7d2b224740 1 --2- 172.21.15.78:0/1236359317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2469710 0x26db900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.330+0000 7f7d2b224740 1 -- 172.21.15.78:0/1236359317 >> 172.21.15.78:0/1236359317 conn(0x26d1fa0 msgr2=0x2729890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:57.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.330+0000 7f7d2b224740 1 -- 172.21.15.78:0/1236359317 shutdown_connections 2023-10-11T09:49:57.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.330+0000 7f7d2b224740 1 -- 172.21.15.78:0/1236359317 wait complete. 2023-10-11T09:49:57.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.330+0000 7f7d2b224740 1 -- start start 2023-10-11T09:49:57.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.334+0000 7f7d2b224740 1 Processor -- start 2023-10-11T09:49:57.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.334+0000 7f7d2b224740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2469710 0x27265d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:57.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.334+0000 7f7d2b224740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2469ae0 0x277d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:57.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.334+0000 7f7d2b224740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2469eb0 0x277d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:57.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.334+0000 7f7d2b224740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x26fdc40 con 0x2469ae0 2023-10-11T09:49:57.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.334+0000 7f7d2b224740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x2726b10 con 0x2469eb0 2023-10-11T09:49:57.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.334+0000 7f7d2b224740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x2718d30 con 0x2469710 2023-10-11T09:49:57.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.334+0000 7f7d2b224740 10 client.0 _ll_register_callbacks cb 0x2640c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:49:57.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.334+0000 7f7d25534700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2469ae0 0x277d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:57.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.334+0000 7f7d25534700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2469ae0 0x277d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36202/0 (socket says 172.21.15.78:36202) 2023-10-11T09:49:57.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.334+0000 7f7d25534700 1 -- 172.21.15.78:0/1805845416 learned_addr learned my addr 172.21.15.78:0/1805845416 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:57.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.334+0000 7f7d25d35700 1 --2- 172.21.15.78:0/1805845416 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2469710 0x27265d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:57.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.334+0000 7f7d26536700 1 --2- 172.21.15.78:0/1805845416 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2469eb0 0x277d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:57.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.334+0000 7f7d25534700 1 -- 172.21.15.78:0/1805845416 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2469710 msgr2=0x27265d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:57.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.334+0000 7f7d25534700 1 --2- 172.21.15.78:0/1805845416 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2469710 0x27265d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.334+0000 7f7d25534700 1 -- 172.21.15.78:0/1805845416 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2469eb0 msgr2=0x277d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:57.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.334+0000 7f7d25534700 1 --2- 172.21.15.78:0/1805845416 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2469eb0 0x277d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.338+0000 7f7d25534700 1 -- 172.21.15.78:0/1805845416 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x2608750 con 0x2469ae0 2023-10-11T09:49:57.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.338+0000 7f7d25534700 1 --2- 172.21.15.78:0/1805845416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2469ae0 0x277d0c0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f7d10006eb0 tx=0x7f7d100047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=393e4532e969d7c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:57.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.338+0000 7f7d00ff9700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:49:57.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.338+0000 7f7d00ff9700 1 -- 172.21.15.78:0/1805845416 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1002d070 con 0x2469ae0 2023-10-11T09:49:57.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.338+0000 7f7d00ff9700 1 -- 172.21.15.78:0/1805845416 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7d10003b20 con 0x2469ae0 2023-10-11T09:49:57.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.338+0000 7f7d2b224740 1 -- 172.21.15.78:0/1805845416 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d180114f0 con 0x2469ae0 2023-10-11T09:49:57.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.338+0000 7f7d00ff9700 1 -- 172.21.15.78:0/1805845416 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d10020070 con 0x2469ae0 2023-10-11T09:49:57.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.338+0000 7f7d2b224740 10 client.8187 fetch_fsmap learned FSMap version 79 2023-10-11T09:49:57.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.338+0000 7f7d2b224740 1 -- 172.21.15.78:0/1805845416 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x25ee980 con 0x2469ae0 2023-10-11T09:49:57.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.338+0000 7f7d00ff9700 1 -- 172.21.15.78:0/1805845416 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d180114f0 con 0x2469ae0 2023-10-11T09:49:57.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.338+0000 7f7d00ff9700 1 -- 172.21.15.78:0/1805845416 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 79) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f7d10003ce0 con 0x2469ae0 2023-10-11T09:49:57.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.338+0000 7f7d2b224740 10 client.8187 fetch_fsmap finished waiting for FSMap version 79 2023-10-11T09:49:57.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.338+0000 7f7d2b224740 10 client.8187 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-11T09:49:57.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.338+0000 7f7d2b224740 20 client.8187 populate_metadata read hostname 'smithi078' 2023-10-11T09:49:57.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.338+0000 7f7d2b224740 1 --2- 172.21.15.78:0/1805845416 >> [v2:172.21.15.78:6836/1286798022,v1:172.21.15.78:6837/1286798022] conn(0x2737e10 0x27381e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:57.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.338+0000 7f7d2b224740 4 client.8187 mds_command: new command op to 8031 tid=0[{"prefix": "status"}] 2023-10-11T09:49:57.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.338+0000 7f7d2b224740 1 -- 172.21.15.78:0/1805845416 --> [v2:172.21.15.78:6836/1286798022,v1:172.21.15.78:6837/1286798022] -- command(tid 0: {"prefix": "status"}) v1 -- 0x24c14e0 con 0x2737e10 2023-10-11T09:49:57.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.338+0000 7f7d25d35700 1 --2- 172.21.15.78:0/1805845416 >> [v2:172.21.15.78:6836/1286798022,v1:172.21.15.78:6837/1286798022] conn(0x2737e10 0x27381e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:57.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.338+0000 7f7d25d35700 1 --2- 172.21.15.78:0/1805845416 >> [v2:172.21.15.78:6836/1286798022,v1:172.21.15.78:6837/1286798022] conn(0x2737e10 0x27381e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c436a712af4e1c9a server_cookie=2beb47d617c5f635 in_seq=0 out_seq=0 2023-10-11T09:49:57.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.338+0000 7f7d00ff9700 10 client.8187 ms_handle_connect on v2:172.21.15.78:6836/1286798022 2023-10-11T09:49:57.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.338+0000 7f7d00ff9700 1 -- 172.21.15.78:0/1805845416 <== mds.0 v2:172.21.15.78:6836/1286798022 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x24c14e0 con 0x2737e10 2023-10-11T09:49:57.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.338+0000 7f7d00ff9700 10 client.8187 handle_command_reply: tid=0 2023-10-11T09:49:57.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.338+0000 7f7d2b224740 1 client.8187 shutdown 2023-10-11T09:49:57.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.342+0000 7f7d2b224740 1 -- 172.21.15.78:0/1805845416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2469ae0 msgr2=0x277d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:57.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.342+0000 7f7d2b224740 1 --2- 172.21.15.78:0/1805845416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2469ae0 0x277d0c0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f7d10006eb0 tx=0x7f7d100047e0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.342+0000 7f7d2b224740 1 -- 172.21.15.78:0/1805845416 shutdown_connections 2023-10-11T09:49:57.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.342+0000 7f7d2b224740 1 --2- 172.21.15.78:0/1805845416 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2469eb0 0x277d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.342+0000 7f7d2b224740 1 --2- 172.21.15.78:0/1805845416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2469ae0 0x277d0c0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.342+0000 7f7d2b224740 1 --2- 172.21.15.78:0/1805845416 >> [v2:172.21.15.78:6836/1286798022,v1:172.21.15.78:6837/1286798022] conn(0x2737e10 0x27381e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.342+0000 7f7d2b224740 1 --2- 172.21.15.78:0/1805845416 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2469710 0x27265d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.342+0000 7f7d2b224740 1 -- 172.21.15.78:0/1805845416 >> 172.21.15.78:0/1805845416 conn(0x26d1fa0 msgr2=0x26dc190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:57.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.342+0000 7f7d00ff9700 0 client.8187 ms_handle_reset on v2:172.21.15.78:6836/1286798022 2023-10-11T09:49:57.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.342+0000 7f7d2b224740 1 -- 172.21.15.78:0/1805845416 shutdown_connections 2023-10-11T09:49:57.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.342+0000 7f7d2b224740 1 -- 172.21.15.78:0/1805845416 wait complete. 2023-10-11T09:49:57.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.342+0000 7f7d2b224740 20 client.8187 trim_cache size 0 max 16384 2023-10-11T09:49:57.355 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:49:57.356 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "dee99db9-7a72-406f-a1e2-917943f2b13d", 2023-10-11T09:49:57.356 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2023-10-11T09:49:57.356 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 8031, 2023-10-11T09:49:57.356 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2023-10-11T09:49:57.356 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2023-10-11T09:49:57.356 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2023-10-11T09:49:57.357 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 6.3810755950000004, 2023-10-11T09:49:57.357 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 79, 2023-10-11T09:49:57.357 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 105, 2023-10-11T09:49:57.357 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 105, 2023-10-11T09:49:57.357 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 11.641040658 2023-10-11T09:49:57.357 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:49:57.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.342+0000 7f7d26d37700 1 -- 172.21.15.78:0/1855996802 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7d04065ed0 msgr2=0x7f7d04068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:57.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.342+0000 7f7d26d37700 1 --2- 172.21.15.78:0/1855996802 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7d04065ed0 0x7f7d04068380 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7d2008c200 tx=0x7f7d10008030 comp rx=0 tx=0).stop 2023-10-11T09:49:57.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.342+0000 7f7d26d37700 1 -- 172.21.15.78:0/1855996802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d200d1660 msgr2=0x7f7d200c69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:57.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.342+0000 7f7d26d37700 1 --2- 172.21.15.78:0/1855996802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d200d1660 0x7f7d200c69b0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c00c500 tx=0x7f7d1c00c810 comp rx=0 tx=0).stop 2023-10-11T09:49:57.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.342+0000 7f7d26d37700 1 -- 172.21.15.78:0/1855996802 shutdown_connections 2023-10-11T09:49:57.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.342+0000 7f7d26d37700 1 --2- 172.21.15.78:0/1855996802 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7d04065ed0 0x7f7d04068380 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.342+0000 7f7d26d37700 1 --2- 172.21.15.78:0/1855996802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d20156ce0 0x7f7d20152730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.342+0000 7f7d26d37700 1 --2- 172.21.15.78:0/1855996802 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7d200d28e0 0x7f7d200c8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.342+0000 7f7d26d37700 1 --2- 172.21.15.78:0/1855996802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d200d1660 0x7f7d200c69b0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.342+0000 7f7d26d37700 1 -- 172.21.15.78:0/1855996802 >> 172.21.15.78:0/1855996802 conn(0x7f7d2006afb0 msgr2=0x7f7d2015f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:57.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.342+0000 7f7d26d37700 1 -- 172.21.15.78:0/1855996802 shutdown_connections 2023-10-11T09:49:57.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.342+0000 7f7d26d37700 1 -- 172.21.15.78:0/1855996802 wait complete. 2023-10-11T09:49:57.371 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-11T09:49:57.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.534+0000 7f9e1aa5d700 1 Processor -- start 2023-10-11T09:49:57.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.534+0000 7f9e1aa5d700 1 -- start start 2023-10-11T09:49:57.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.538+0000 7f9e1aa5d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f9e1415b8e0 0x7f9e1406d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:57.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.538+0000 7f9e1aa5d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9e1406d610 0x7f9e140ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:57.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.538+0000 7f9e1aa5d700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9e1415a940 0x7f9e140ca780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:57.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.538+0000 7f9e1aa5d700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e1415e010 con 0x7f9e1415b8e0 2023-10-11T09:49:57.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.538+0000 7f9e1aa5d700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e1415e280 con 0x7f9e1415a940 2023-10-11T09:49:57.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.538+0000 7f9e1aa5d700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1415e560 con 0x7f9e1406d610 2023-10-11T09:49:57.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.538+0000 7f9e19a5b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f9e1415b8e0 0x7f9e1406d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:57.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.538+0000 7f9e19a5b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f9e1415b8e0 0x7f9e1406d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36214/0 (socket says 172.21.15.78:36214) 2023-10-11T09:49:57.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.538+0000 7f9e19a5b700 1 -- 172.21.15.78:0/1892008624 learned_addr learned my addr 172.21.15.78:0/1892008624 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:57.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.538+0000 7f9e1a25c700 1 --2- 172.21.15.78:0/1892008624 >> v2:172.21.15.133:3300/0 conn(0x7f9e1415a940 0x7f9e140ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:57.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.538+0000 7f9e1925a700 1 --2- 172.21.15.78:0/1892008624 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9e1406d610 0x7f9e140ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:57.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.538+0000 7f9e19a5b700 1 -- 172.21.15.78:0/1892008624 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9e1406d610 msgr2=0x7f9e140ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:57.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.538+0000 7f9e19a5b700 1 --2- 172.21.15.78:0/1892008624 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9e1406d610 0x7f9e140ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.538+0000 7f9e19a5b700 1 -- 172.21.15.78:0/1892008624 >> v2:172.21.15.133:3300/0 conn(0x7f9e1415a940 msgr2=0x7f9e140ca780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:57.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.538+0000 7f9e19a5b700 1 --2- 172.21.15.78:0/1892008624 >> v2:172.21.15.133:3300/0 conn(0x7f9e1415a940 0x7f9e140ca780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.538+0000 7f9e19a5b700 1 -- 172.21.15.78:0/1892008624 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e1415e840 con 0x7f9e1415b8e0 2023-10-11T09:49:57.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.538+0000 7f9e19a5b700 1 --2- 172.21.15.78:0/1892008624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e1415b8e0 0x7f9e1406d0d0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f9e140cf800 tx=0x7f9e10009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72efe517fbc0126 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:57.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.542+0000 7f9e18a59700 1 -- 172.21.15.78:0/1892008624 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e10011830 con 0x7f9e1415b8e0 2023-10-11T09:49:57.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.542+0000 7f9e18a59700 1 -- 172.21.15.78:0/1892008624 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e10011990 con 0x7f9e1415b8e0 2023-10-11T09:49:57.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.542+0000 7f9e18a59700 1 -- 172.21.15.78:0/1892008624 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e10011d20 con 0x7f9e1415b8e0 2023-10-11T09:49:57.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.542+0000 7f9e1aa5d700 1 -- 172.21.15.78:0/1892008624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e1415b8e0 msgr2=0x7f9e1406d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:57.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.542+0000 7f9e1aa5d700 1 --2- 172.21.15.78:0/1892008624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e1415b8e0 0x7f9e1406d0d0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f9e140cf800 tx=0x7f9e10009bf0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.542+0000 7f9e1aa5d700 1 -- 172.21.15.78:0/1892008624 shutdown_connections 2023-10-11T09:49:57.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.542+0000 7f9e1aa5d700 1 --2- 172.21.15.78:0/1892008624 >> v2:172.21.15.133:3300/0 conn(0x7f9e1415a940 0x7f9e140ca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.542+0000 7f9e1aa5d700 1 --2- 172.21.15.78:0/1892008624 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9e1406d610 0x7f9e140ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.542+0000 7f9e1aa5d700 1 --2- 172.21.15.78:0/1892008624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e1415b8e0 0x7f9e1406d0d0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.542+0000 7f9e1aa5d700 1 -- 172.21.15.78:0/1892008624 >> 172.21.15.78:0/1892008624 conn(0x7f9e140cea60 msgr2=0x7f9e140cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:57.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.542+0000 7f9e1aa5d700 1 -- 172.21.15.78:0/1892008624 shutdown_connections 2023-10-11T09:49:57.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.542+0000 7f9e1aa5d700 1 -- 172.21.15.78:0/1892008624 wait complete. 2023-10-11T09:49:57.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.546+0000 7f9e1aa5d700 1 Processor -- start 2023-10-11T09:49:57.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.546+0000 7f9e1aa5d700 1 -- start start 2023-10-11T09:49:57.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.546+0000 7f9e1aa5d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e1406d610 0x7f9e141f2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:57.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.546+0000 7f9e1aa5d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9e1415a940 0x7f9e14141ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:57.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.546+0000 7f9e1aa5d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e1415b8e0 0x7f9e141f4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:57.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.546+0000 7f9e1aa5d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e141f19f0 con 0x7f9e1415b8e0 2023-10-11T09:49:57.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.546+0000 7f9e1aa5d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e141f5ef0 con 0x7f9e1406d610 2023-10-11T09:49:57.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.546+0000 7f9e1aa5d700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e141f5930 con 0x7f9e1415a940 2023-10-11T09:49:57.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.546+0000 7f9e1a25c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e1415b8e0 0x7f9e141f4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:57.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.546+0000 7f9e1a25c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e1415b8e0 0x7f9e141f4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36220/0 (socket says 172.21.15.78:36220) 2023-10-11T09:49:57.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.546+0000 7f9e1a25c700 1 -- 172.21.15.78:0/3657046833 learned_addr learned my addr 172.21.15.78:0/3657046833 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:57.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.546+0000 7f9e1925a700 1 --2- 172.21.15.78:0/3657046833 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9e1415a940 0x7f9e14141ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:57.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.546+0000 7f9e19a5b700 1 --2- 172.21.15.78:0/3657046833 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e1406d610 0x7f9e141f2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:57.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.546+0000 7f9e1a25c700 1 -- 172.21.15.78:0/3657046833 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9e1415a940 msgr2=0x7f9e14141ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:57.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.546+0000 7f9e1a25c700 1 --2- 172.21.15.78:0/3657046833 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9e1415a940 0x7f9e14141ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.546+0000 7f9e1a25c700 1 -- 172.21.15.78:0/3657046833 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e1406d610 msgr2=0x7f9e141f2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:57.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.546+0000 7f9e1a25c700 1 --2- 172.21.15.78:0/3657046833 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e1406d610 0x7f9e141f2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.546+0000 7f9e1a25c700 1 -- 172.21.15.78:0/3657046833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e10009570 con 0x7f9e1415b8e0 2023-10-11T09:49:57.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.546+0000 7f9e1a25c700 1 --2- 172.21.15.78:0/3657046833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e1415b8e0 0x7f9e141f4fe0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c00efc0 tx=0x7f9e0c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:57.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.546+0000 7f9e0a7fc700 1 -- 172.21.15.78:0/3657046833 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c00cd40 con 0x7f9e1415b8e0 2023-10-11T09:49:57.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.546+0000 7f9e0a7fc700 1 -- 172.21.15.78:0/3657046833 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e0c00f040 con 0x7f9e1415b8e0 2023-10-11T09:49:57.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.546+0000 7f9e1aa5d700 1 -- 172.21.15.78:0/3657046833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e141f5c10 con 0x7f9e1415b8e0 2023-10-11T09:49:57.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.546+0000 7f9e0a7fc700 1 -- 172.21.15.78:0/3657046833 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c0146c0 con 0x7f9e1415b8e0 2023-10-11T09:49:57.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.546+0000 7f9e1aa5d700 1 -- 172.21.15.78:0/3657046833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e141f1ee0 con 0x7f9e1415b8e0 2023-10-11T09:49:57.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.550+0000 7f9e1aa5d700 1 -- 172.21.15.78:0/3657046833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e141f2100 con 0x7f9e1415b8e0 2023-10-11T09:49:57.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.550+0000 7f9e0a7fc700 1 -- 172.21.15.78:0/3657046833 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f9e0c014820 con 0x7f9e1415b8e0 2023-10-11T09:49:57.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.550+0000 7f9e0a7fc700 1 --2- 172.21.15.78:0/3657046833 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9df8065ed0 0x7f9df8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:57.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.550+0000 7f9e0a7fc700 1 -- 172.21.15.78:0/3657046833 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9e0c083ff0 con 0x7f9e1415b8e0 2023-10-11T09:49:57.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.554+0000 7f9e19a5b700 1 --2- 172.21.15.78:0/3657046833 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9df8065ed0 0x7f9df8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:57.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.554+0000 7f9e19a5b700 1 --2- 172.21.15.78:0/3657046833 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9df8065ed0 0x7f9df8068380 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9e10000c00 tx=0x7f9e10008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:57.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.558+0000 7f9e0a7fc700 1 -- 172.21.15.78:0/3657046833 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f9e0c0516d0 con 0x7f9e1415b8e0 2023-10-11T09:49:57.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.770+0000 7f9e1aa5d700 1 -- 172.21.15.78:0/3657046833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9e14065d40 con 0x7f9e1415b8e0 2023-10-11T09:49:57.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.774+0000 7f9e0a7fc700 1 -- 172.21.15.78:0/3657046833 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v79) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9e0c00cea0 con 0x7f9e1415b8e0 2023-10-11T09:49:57.781 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-11T09:49:57.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.774+0000 7f9e1aa5d700 1 -- 172.21.15.78:0/3657046833 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9df8065ed0 msgr2=0x7f9df8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:57.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.774+0000 7f9e1aa5d700 1 --2- 172.21.15.78:0/3657046833 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9df8065ed0 0x7f9df8068380 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9e10000c00 tx=0x7f9e10008030 comp rx=0 tx=0).stop 2023-10-11T09:49:57.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.774+0000 7f9e1aa5d700 1 -- 172.21.15.78:0/3657046833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e1415b8e0 msgr2=0x7f9e141f4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:57.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.774+0000 7f9e1aa5d700 1 --2- 172.21.15.78:0/3657046833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e1415b8e0 0x7f9e141f4fe0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c00efc0 tx=0x7f9e0c00c3a0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.774+0000 7f9e1aa5d700 1 -- 172.21.15.78:0/3657046833 shutdown_connections 2023-10-11T09:49:57.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.774+0000 7f9e1aa5d700 1 --2- 172.21.15.78:0/3657046833 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9df8065ed0 0x7f9df8068380 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.774+0000 7f9e1aa5d700 1 --2- 172.21.15.78:0/3657046833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e1415b8e0 0x7f9e141f4fe0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.774+0000 7f9e1aa5d700 1 --2- 172.21.15.78:0/3657046833 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9e1415a940 0x7f9e14141ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.774+0000 7f9e1aa5d700 1 --2- 172.21.15.78:0/3657046833 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e1406d610 0x7f9e141f2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:57.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.774+0000 7f9e1aa5d700 1 -- 172.21.15.78:0/3657046833 >> 172.21.15.78:0/3657046833 conn(0x7f9e140cea60 msgr2=0x7f9e140b0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:57.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.774+0000 7f9e1aa5d700 1 -- 172.21.15.78:0/3657046833 shutdown_connections 2023-10-11T09:49:57.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:57.774+0000 7f9e1aa5d700 1 -- 172.21.15.78:0/3657046833 wait complete. 2023-10-11T09:49:57.804 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-11T09:49:57.804 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-11T09:49:57.805 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi078.front.sepia.ceph.com 2023-10-11T09:49:57.805 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-11T09:49:57.805 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-11T09:49:57.806 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-11T09:49:57.806 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-11T09:49:57.807 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-11T09:49:57.807 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-11T09:49:57.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:49:57.808 DEBUG:teuthology.orchestra.run.smithi078:> ip addr 2023-10-11T09:49:57.824 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-11T09:49:57.824 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-11T09:49:57.824 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2023-10-11T09:49:57.824 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:49:57.825 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2023-10-11T09:49:57.825 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:49:57.825 INFO:teuthology.orchestra.run.smithi078.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:49:57.825 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:22 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:49:57.825 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f0 2023-10-11T09:49:57.826 INFO:teuthology.orchestra.run.smithi078.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:49:57.826 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:23 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:49:57.826 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f1 2023-10-11T09:49:57.826 INFO:teuthology.orchestra.run.smithi078.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:49:57.826 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dc brd ff:ff:ff:ff:ff:ff 2023-10-11T09:49:57.827 INFO:teuthology.orchestra.run.smithi078.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-11T09:49:57.827 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dd brd ff:ff:ff:ff:ff:ff 2023-10-11T09:49:57.827 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.21.15.78/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-11T09:49:57.827 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft 41866sec preferred_lft 41866sec 2023-10-11T09:49:57.827 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::ec4:7aff:fe8f:d4dd/64 scope link 2023-10-11T09:49:57.828 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:49:57.828 INFO:teuthology.orchestra.run.smithi078.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-11T09:49:57.828 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 02:42:b6:e4:f8:0d brd ff:ff:ff:ff:ff:ff 2023-10-11T09:49:57.828 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-11T09:49:57.828 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:49:57.828 INFO:teuthology.orchestra.run.smithi078.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-11T09:49:57.829 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 7a:89:8a:a9:e1:23 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:49:57.829 INFO:teuthology.orchestra.run.smithi078.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-11T09:49:57.829 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:49:57.829 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::8ca:ebff:fea9:13f1/64 scope link 2023-10-11T09:49:57.829 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:49:57.830 INFO:teuthology.orchestra.run.smithi078.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-11T09:49:57.830 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 7a:89:8a:a9:e1:23 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-11T09:49:57.830 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::7889:8aff:fea9:e123/64 scope link 2023-10-11T09:49:57.830 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:49:57.830 INFO:teuthology.orchestra.run.smithi078.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-11T09:49:57.831 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether ca:ac:e8:27:51:0d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-11T09:49:57.831 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::c8ac:e8ff:fe27:510d/64 scope link 2023-10-11T09:49:57.831 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:49:57.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:49:57.832 DEBUG:teuthology.orchestra.run.smithi078:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-11T09:49:57.909 INFO:teuthology.orchestra.run.smithi078.stdout:1 2023-10-11T09:49:57.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:49:57.913 DEBUG:teuthology.orchestra.run.smithi078:> route 2023-10-11T09:49:57.976 INFO:teuthology.orchestra.run.smithi078.stdout:Kernel IP routing table 2023-10-11T09:49:57.976 INFO:teuthology.orchestra.run.smithi078.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-11T09:49:57.977 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-11T09:49:57.977 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-11T09:49:57.977 INFO:teuthology.orchestra.run.smithi078.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-11T09:49:57.977 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-10-11T09:49:57.977 INFO:teuthology.orchestra.run.smithi078.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-11T09:49:57.978 INFO:teuthology.orchestra.run.smithi078.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-11T09:49:57.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:49:57.979 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec bash -c ' 2023-10-11T09:49:57.980 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2023-10-11T09:49:57.980 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-11T09:49:57.980 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-11T09:49:57.981 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-11T09:49:57.981 DEBUG:teuthology.orchestra.run.smithi078:> ') 2023-10-11T09:49:58.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:49:58.116 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2023-10-11T09:49:58.133 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-11T09:49:58.133 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-11T09:49:58.134 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-11T09:49:58.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:49:58.135 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set brx.0 up 2023-10-11T09:49:58.211 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-11T09:49:58.212 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T09:49:58.212 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-11T09:49:58.272 INFO:teuthology.orchestra.run.smithi078.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-11T09:49:58.273 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T09:49:58.273 DEBUG:teuthology.orchestra.run.smithi078:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-11T09:49:58.332 DEBUG:teuthology.orchestra.run.smithi078:> sudo modprobe fuse 2023-10-11T09:49:58.415 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T09:49:58.416 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T09:49:58.491 INFO:teuthology.orchestra.run.smithi078.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T09:49:58.495 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T09:49:58.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:49:58.496 DEBUG:teuthology.orchestra.run.smithi078:> ls /sys/fs/fuse/connections 2023-10-11T09:49:58.559 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-11T09:49:58.560 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-11T09:49:58.610 DEBUG:teuthology.orchestra.run.smithi078:> sudo modprobe fuse 2023-10-11T09:49:58.642 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T09:49:58.642 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T09:49:58.717 INFO:teuthology.orchestra.run.smithi078.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T09:49:58.719 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T09:49:58.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:49:58.720 DEBUG:teuthology.orchestra.run.smithi078:> ls /sys/fs/fuse/connections 2023-10-11T09:49:58.754 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:2023-10-11T09:49:58.746+0000 7f3d65288640 -1 init, newargv = 0x55e65934d7d0 newargc=15 2023-10-11T09:49:58.754 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:ceph-fuse[56273]: starting ceph client 2023-10-11T09:49:58.765 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:ceph-fuse[56273]: starting fuse 2023-10-11T09:49:58.768 INFO:teuthology.orchestra.run.smithi078.stdout:49 2023-10-11T09:49:58.769 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2023-10-11T09:49:58.769 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:49:58.770 DEBUG:teuthology.orchestra.run.smithi078:> import glob 2023-10-11T09:49:58.770 DEBUG:teuthology.orchestra.run.smithi078:> import re 2023-10-11T09:49:58.771 DEBUG:teuthology.orchestra.run.smithi078:> import os 2023-10-11T09:49:58.771 DEBUG:teuthology.orchestra.run.smithi078:> import subprocess 2023-10-11T09:49:58.771 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:49:58.772 DEBUG:teuthology.orchestra.run.smithi078:> def _find_admin_socket(client_name): 2023-10-11T09:49:58.772 DEBUG:teuthology.orchestra.run.smithi078:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-11T09:49:58.773 DEBUG:teuthology.orchestra.run.smithi078:> files = glob.glob(asok_path) 2023-10-11T09:49:58.773 DEBUG:teuthology.orchestra.run.smithi078:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-11T09:49:58.773 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:49:58.774 DEBUG:teuthology.orchestra.run.smithi078:> # Given a non-glob path, it better be there 2023-10-11T09:49:58.774 DEBUG:teuthology.orchestra.run.smithi078:> if "*" not in asok_path: 2023-10-11T09:49:58.775 DEBUG:teuthology.orchestra.run.smithi078:> assert(len(files) == 1) 2023-10-11T09:49:58.775 DEBUG:teuthology.orchestra.run.smithi078:> return files[0] 2023-10-11T09:49:58.775 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:49:58.776 DEBUG:teuthology.orchestra.run.smithi078:> for f in files: 2023-10-11T09:49:58.776 DEBUG:teuthology.orchestra.run.smithi078:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-11T09:49:58.777 DEBUG:teuthology.orchestra.run.smithi078:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-11T09:49:58.777 DEBUG:teuthology.orchestra.run.smithi078:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-11T09:49:58.777 DEBUG:teuthology.orchestra.run.smithi078:> contents = proc_f.read() 2023-10-11T09:49:58.778 DEBUG:teuthology.orchestra.run.smithi078:> if mountpoint in contents: 2023-10-11T09:49:58.778 DEBUG:teuthology.orchestra.run.smithi078:> return f 2023-10-11T09:49:58.779 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-11T09:49:58.779 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:49:58.780 DEBUG:teuthology.orchestra.run.smithi078:> print(_find_admin_socket("client.0")) 2023-10-11T09:49:58.780 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:49:58.931 INFO:teuthology.orchestra.run.smithi078.stdout:/var/run/ceph/ceph-client.0.56273.asok 2023-10-11T09:49:59.116 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.56273.asok 2023-10-11T09:49:59.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:49:59.117 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.56273.asok status 2023-10-11T09:49:59.312 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:49:59.313 INFO:teuthology.orchestra.run.smithi078.stdout: "metadata": { 2023-10-11T09:49:59.313 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_sha1": "9a774796f0ecdd11a4b771de396e627aa1a187af", 2023-10-11T09:49:59.313 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_version": "ceph version 18.2.0-662-g9a774796 (9a774796f0ecdd11a4b771de396e627aa1a187af) reef (stable)", 2023-10-11T09:49:59.313 INFO:teuthology.orchestra.run.smithi078.stdout: "entity_id": "0", 2023-10-11T09:49:59.314 INFO:teuthology.orchestra.run.smithi078.stdout: "hostname": "smithi078", 2023-10-11T09:49:59.314 INFO:teuthology.orchestra.run.smithi078.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-11T09:49:59.314 INFO:teuthology.orchestra.run.smithi078.stdout: "pid": "56273", 2023-10-11T09:49:59.314 INFO:teuthology.orchestra.run.smithi078.stdout: "root": "/" 2023-10-11T09:49:59.314 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:49:59.315 INFO:teuthology.orchestra.run.smithi078.stdout: "dentry_count": 0, 2023-10-11T09:49:59.315 INFO:teuthology.orchestra.run.smithi078.stdout: "dentry_pinned_count": 0, 2023-10-11T09:49:59.315 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 8199, 2023-10-11T09:49:59.315 INFO:teuthology.orchestra.run.smithi078.stdout: "inst": { 2023-10-11T09:49:59.316 INFO:teuthology.orchestra.run.smithi078.stdout: "name": { 2023-10-11T09:49:59.316 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "client", 2023-10-11T09:49:59.316 INFO:teuthology.orchestra.run.smithi078.stdout: "num": 8199 2023-10-11T09:49:59.316 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:49:59.316 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": { 2023-10-11T09:49:59.316 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2023-10-11T09:49:59.317 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "192.168.0.1:0", 2023-10-11T09:49:59.317 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 2630231585 2023-10-11T09:49:59.317 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:49:59.317 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:49:59.317 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": { 2023-10-11T09:49:59.318 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2023-10-11T09:49:59.318 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "192.168.0.1:0", 2023-10-11T09:49:59.318 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 2630231585 2023-10-11T09:49:59.318 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:49:59.318 INFO:teuthology.orchestra.run.smithi078.stdout: "inst_str": "client.8199 192.168.0.1:0/2630231585", 2023-10-11T09:49:59.319 INFO:teuthology.orchestra.run.smithi078.stdout: "addr_str": "192.168.0.1:0/2630231585", 2023-10-11T09:49:59.319 INFO:teuthology.orchestra.run.smithi078.stdout: "inode_count": 1, 2023-10-11T09:49:59.319 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_epoch": 79, 2023-10-11T09:49:59.319 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_epoch": 105, 2023-10-11T09:49:59.319 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_epoch_barrier": 0, 2023-10-11T09:49:59.319 INFO:teuthology.orchestra.run.smithi078.stdout: "blocklisted": false, 2023-10-11T09:49:59.320 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs" 2023-10-11T09:49:59.320 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:49:59.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:49:59.328 DEBUG:teuthology.orchestra.run.smithi078:> stat --file-system '--printf=%T 2023-10-11T09:49:59.328 DEBUG:teuthology.orchestra.run.smithi078:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-11T09:49:59.348 INFO:teuthology.orchestra.run.smithi078.stdout:fuseblk 2023-10-11T09:49:59.349 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-11T09:49:59.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:49:59.349 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-11T09:49:59.426 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_extra --pg_num_min 64 2023-10-11T09:49:59.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.642+0000 7fd269b13700 1 Processor -- start 2023-10-11T09:49:59.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.642+0000 7fd269b13700 1 -- start start 2023-10-11T09:49:59.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.646+0000 7fd269b13700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd26406c630 0x7fd2640c2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:59.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.646+0000 7fd269b13700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd2641649c0 0x7fd264164e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:59.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.646+0000 7fd269b13700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2640cd3e0 con 0x7fd264160d90 2023-10-11T09:49:59.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.646+0000 7fd269b13700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2640710f0 con 0x7fd26406c630 2023-10-11T09:49:59.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.646+0000 7fd269b13700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2640cb8f0 con 0x7fd2641649c0 2023-10-11T09:49:59.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.646+0000 7fd2637fe700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fd264160d90 0x7fd2640c1a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:55022/0 (socket says 172.21.15.78:55022) 2023-10-11T09:49:59.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.646+0000 7fd2637fe700 1 -- 172.21.15.78:0/1172613443 learned_addr learned my addr 172.21.15.78:0/1172613443 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:49:59.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.646+0000 7fd262ffd700 1 --2- 172.21.15.78:0/1172613443 >> v2:172.21.15.133:3300/0 conn(0x7fd26406c630 0x7fd2640c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:59.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.646+0000 7fd263fff700 1 --2- 172.21.15.78:0/1172613443 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd2641649c0 0x7fd264164e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:59.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.646+0000 7fd2627fc700 1 -- 172.21.15.78:0/1172613443 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2456984617 0 0) 0x7fd2640cd3e0 con 0x7fd264160d90 2023-10-11T09:49:59.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.646+0000 7fd2627fc700 1 -- 172.21.15.78:0/1172613443 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd24c0035d0 con 0x7fd264160d90 2023-10-11T09:49:59.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.646+0000 7fd263fff700 1 -- 172.21.15.78:0/1172613443 >> v2:172.21.15.133:3300/0 conn(0x7fd26406c630 msgr2=0x7fd2640c2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:49:59.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.646+0000 7fd263fff700 1 --2- 172.21.15.78:0/1172613443 >> v2:172.21.15.133:3300/0 conn(0x7fd26406c630 0x7fd2640c2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:59.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.646+0000 7fd2627fc700 1 -- 172.21.15.78:0/1172613443 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2504452647 0 0) 0x7fd24c0035d0 con 0x7fd264160d90 2023-10-11T09:49:59.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.646+0000 7fd263fff700 1 -- 172.21.15.78:0/1172613443 >> v1:172.21.15.78:6789/0 conn(0x7fd264160d90 legacy=0x7fd2640c1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:59.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.646+0000 7fd263fff700 1 -- 172.21.15.78:0/1172613443 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd264072300 con 0x7fd2641649c0 2023-10-11T09:49:59.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.646+0000 7fd263fff700 1 --2- 172.21.15.78:0/1172613443 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd2641649c0 0x7fd264164e20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd25c009d90 tx=0x7fd25c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57d9f0a8eb276c90 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:59.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.650+0000 7fd2627fc700 1 -- 172.21.15.78:0/1172613443 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25c00e070 con 0x7fd2641649c0 2023-10-11T09:49:59.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.650+0000 7fd2627fc700 1 -- 172.21.15.78:0/1172613443 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd25c003a40 con 0x7fd2641649c0 2023-10-11T09:49:59.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.650+0000 7fd2627fc700 1 -- 172.21.15.78:0/1172613443 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd254001690 con 0x7fd2641649c0 2023-10-11T09:49:59.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.650+0000 7fd269b13700 1 -- 172.21.15.78:0/1172613443 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd2641649c0 msgr2=0x7fd264164e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:59.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.650+0000 7fd269b13700 1 --2- 172.21.15.78:0/1172613443 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd2641649c0 0x7fd264164e20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd25c009d90 tx=0x7fd25c00b330 comp rx=0 tx=0).stop 2023-10-11T09:49:59.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.650+0000 7fd269b13700 1 -- 172.21.15.78:0/1172613443 shutdown_connections 2023-10-11T09:49:59.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.650+0000 7fd269b13700 1 --2- 172.21.15.78:0/1172613443 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd2641649c0 0x7fd264164e20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:59.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.650+0000 7fd269b13700 1 --2- 172.21.15.78:0/1172613443 >> v2:172.21.15.133:3300/0 conn(0x7fd26406c630 0x7fd2640c2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:59.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.650+0000 7fd269b13700 1 -- 172.21.15.78:0/1172613443 >> 172.21.15.78:0/1172613443 conn(0x7fd26406a060 msgr2=0x7fd26406a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:49:59.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.650+0000 7fd269b13700 1 -- 172.21.15.78:0/1172613443 shutdown_connections 2023-10-11T09:49:59.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.650+0000 7fd269b13700 1 -- 172.21.15.78:0/1172613443 wait complete. 2023-10-11T09:49:59.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.654+0000 7fd269b13700 1 Processor -- start 2023-10-11T09:49:59.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.654+0000 7fd269b13700 1 -- start start 2023-10-11T09:49:59.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.654+0000 7fd269b13700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd26406c630 0x7fd264158480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:59.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.654+0000 7fd269b13700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd264160d90 0x7fd264163eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:59.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.654+0000 7fd269b13700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2641649c0 0x7fd2640afe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:59.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.654+0000 7fd269b13700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2640b0bb0 con 0x7fd264160d90 2023-10-11T09:49:59.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.654+0000 7fd269b13700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2641643f0 con 0x7fd2641649c0 2023-10-11T09:49:59.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.654+0000 7fd269b13700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2640c8740 con 0x7fd26406c630 2023-10-11T09:49:59.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.654+0000 7fd262ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd264160d90 0x7fd264163eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:59.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.654+0000 7fd262ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd264160d90 0x7fd264163eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36224/0 (socket says 172.21.15.78:36224) 2023-10-11T09:49:59.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.654+0000 7fd262ffd700 1 -- 172.21.15.78:0/3691615764 learned_addr learned my addr 172.21.15.78:0/3691615764 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:49:59.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.654+0000 7fd263fff700 1 --2- 172.21.15.78:0/3691615764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2641649c0 0x7fd2640afe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:59.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.658+0000 7fd2637fe700 1 --2- 172.21.15.78:0/3691615764 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd26406c630 0x7fd264158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:59.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.658+0000 7fd262ffd700 1 -- 172.21.15.78:0/3691615764 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd26406c630 msgr2=0x7fd264158480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:59.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.658+0000 7fd262ffd700 1 --2- 172.21.15.78:0/3691615764 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd26406c630 0x7fd264158480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:59.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.658+0000 7fd262ffd700 1 -- 172.21.15.78:0/3691615764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2641649c0 msgr2=0x7fd2640afe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:49:59.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.658+0000 7fd262ffd700 1 --2- 172.21.15.78:0/3691615764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2641649c0 0x7fd2640afe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:49:59.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.658+0000 7fd262ffd700 1 -- 172.21.15.78:0/3691615764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd25c009a70 con 0x7fd264160d90 2023-10-11T09:49:59.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.658+0000 7fd262ffd700 1 --2- 172.21.15.78:0/3691615764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd264160d90 0x7fd264163eb0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fd25800e9b0 tx=0x7fd25800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:59.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.658+0000 7fd24bfff700 1 -- 172.21.15.78:0/3691615764 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd258009710 con 0x7fd264160d90 2023-10-11T09:49:59.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.658+0000 7fd269b13700 1 -- 172.21.15.78:0/3691615764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2640af380 con 0x7fd264160d90 2023-10-11T09:49:59.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.658+0000 7fd24bfff700 1 -- 172.21.15.78:0/3691615764 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd25800ee50 con 0x7fd264160d90 2023-10-11T09:49:59.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.658+0000 7fd24bfff700 1 -- 172.21.15.78:0/3691615764 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25800ca50 con 0x7fd264160d90 2023-10-11T09:49:59.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.658+0000 7fd269b13700 1 -- 172.21.15.78:0/3691615764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2640af940 con 0x7fd264160d90 2023-10-11T09:49:59.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.658+0000 7fd269b13700 1 -- 172.21.15.78:0/3691615764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2640af1e0 con 0x7fd264160d90 2023-10-11T09:49:59.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.662+0000 7fd24bfff700 1 -- 172.21.15.78:0/3691615764 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fd25800cbb0 con 0x7fd264160d90 2023-10-11T09:49:59.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.662+0000 7fd24bfff700 1 --2- 172.21.15.78:0/3691615764 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd250065ed0 0x7fd250068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:49:59.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.662+0000 7fd24bfff700 1 -- 172.21.15.78:0/3691615764 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd258083cb0 con 0x7fd264160d90 2023-10-11T09:49:59.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.662+0000 7fd2637fe700 1 --2- 172.21.15.78:0/3691615764 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd250065ed0 0x7fd250068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:49:59.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.662+0000 7fd2637fe700 1 --2- 172.21.15.78:0/3691615764 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd250065ed0 0x7fd250068380 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd254000f60 tx=0x7fd254001250 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:49:59.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.670+0000 7fd24bfff700 1 -- 172.21.15.78:0/3691615764 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fd2640af1e0 con 0x7fd264160d90 2023-10-11T09:49:59.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:49:59.846+0000 7fd269b13700 1 -- 172.21.15.78:0/3691615764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_extra", "pg_num_min": 64} v 0) v1 -- 0x7fd2640af1e0 con 0x7fd264160d90 2023-10-11T09:50:00.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.422+0000 7fd24bfff700 1 -- 172.21.15.78:0/3691615764 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_extra", "pg_num_min": 64}]=0 pool 'cephfs_data_extra' created v106) v1 ==== 142+0+0 (secure 0 0 0) 0x7fd2640af1e0 con 0x7fd264160d90 2023-10-11T09:50:00.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.434+0000 7fd269b13700 1 -- 172.21.15.78:0/3691615764 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd250065ed0 msgr2=0x7fd250068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:00.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.434+0000 7fd269b13700 1 --2- 172.21.15.78:0/3691615764 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd250065ed0 0x7fd250068380 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd254000f60 tx=0x7fd254001250 comp rx=0 tx=0).stop 2023-10-11T09:50:00.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.434+0000 7fd269b13700 1 -- 172.21.15.78:0/3691615764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd264160d90 msgr2=0x7fd264163eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:00.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.434+0000 7fd269b13700 1 --2- 172.21.15.78:0/3691615764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd264160d90 0x7fd264163eb0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fd25800e9b0 tx=0x7fd25800ecc0 comp rx=0 tx=0).stop 2023-10-11T09:50:00.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.434+0000 7fd269b13700 1 -- 172.21.15.78:0/3691615764 shutdown_connections 2023-10-11T09:50:00.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.434+0000 7fd269b13700 1 --2- 172.21.15.78:0/3691615764 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd250065ed0 0x7fd250068380 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:00.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.434+0000 7fd269b13700 1 --2- 172.21.15.78:0/3691615764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2641649c0 0x7fd2640afe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:00.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.434+0000 7fd269b13700 1 --2- 172.21.15.78:0/3691615764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd264160d90 0x7fd264163eb0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:00.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.434+0000 7fd269b13700 1 --2- 172.21.15.78:0/3691615764 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd26406c630 0x7fd264158480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:00.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.434+0000 7fd269b13700 1 -- 172.21.15.78:0/3691615764 >> 172.21.15.78:0/3691615764 conn(0x7fd26406a060 msgr2=0x7fd2641602f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:00.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.434+0000 7fd269b13700 1 -- 172.21.15.78:0/3691615764 shutdown_connections 2023-10-11T09:50:00.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.434+0000 7fd269b13700 1 -- 172.21.15.78:0/3691615764 wait complete. 2023-10-11T09:50:00.443 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data_extra' created 2023-10-11T09:50:00.460 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_extra 2023-10-11T09:50:00.585 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:50:00.591 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:50:00.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.630+0000 7f0d7359e700 1 Processor -- start 2023-10-11T09:50:00.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.630+0000 7f0d7359e700 1 -- start start 2023-10-11T09:50:00.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.634+0000 7f0d7359e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0d7413f9e0 0x7f0d740ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:00.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.634+0000 7f0d7359e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0d741601f0 0x7f0d740c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:00.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.634+0000 7f0d7359e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d740b1820 0x7f0d740b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:00.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.634+0000 7f0d7359e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d7413fdb0 con 0x7f0d741601f0 2023-10-11T09:50:00.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.634+0000 7f0d7359e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d740b21e0 con 0x7f0d7413f9e0 2023-10-11T09:50:00.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.634+0000 7f0d7359e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d740cc420 con 0x7f0d740b1820 2023-10-11T09:50:00.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.634+0000 7f0d69d9b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0d741601f0 0x7f0d740c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:00.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.634+0000 7f0d7259c700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0d7413f9e0 0x7f0d740ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:00.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.634+0000 7f0d7259c700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0d7413f9e0 0x7f0d740ca1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:47888/0 (socket says 172.21.15.78:47888) 2023-10-11T09:50:00.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.634+0000 7f0d69d9b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0d741601f0 0x7f0d740c8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36230/0 (socket says 172.21.15.78:36230) 2023-10-11T09:50:00.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.634+0000 7f0d7259c700 1 -- 172.21.15.78:0/3484006199 learned_addr learned my addr 172.21.15.78:0/3484006199 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:00.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.634+0000 7f0d72d9d700 1 --2- 172.21.15.78:0/3484006199 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d740b1820 0x7f0d740b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:00.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.634+0000 7f0d69d9b700 1 -- 172.21.15.78:0/3484006199 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d740b1820 msgr2=0x7f0d740b1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:00.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.634+0000 7f0d69d9b700 1 --2- 172.21.15.78:0/3484006199 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d740b1820 0x7f0d740b1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:00.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.634+0000 7f0d69d9b700 1 -- 172.21.15.78:0/3484006199 >> v2:172.21.15.133:3300/0 conn(0x7f0d7413f9e0 msgr2=0x7f0d740ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:00.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.634+0000 7f0d69d9b700 1 --2- 172.21.15.78:0/3484006199 >> v2:172.21.15.133:3300/0 conn(0x7f0d7413f9e0 0x7f0d740ca1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:00.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.634+0000 7f0d69d9b700 1 -- 172.21.15.78:0/3484006199 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d74160680 con 0x7f0d741601f0 2023-10-11T09:50:00.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.634+0000 7f0d69d9b700 1 --2- 172.21.15.78:0/3484006199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d741601f0 0x7f0d740c8fc0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f0d74073e90 tx=0x7f0d5c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c44499afd75bf1 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:00.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.634+0000 7f0d71d9b700 1 -- 172.21.15.78:0/3484006199 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c011830 con 0x7f0d741601f0 2023-10-11T09:50:00.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.634+0000 7f0d71d9b700 1 -- 172.21.15.78:0/3484006199 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d5c011990 con 0x7f0d741601f0 2023-10-11T09:50:00.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.634+0000 7f0d71d9b700 1 -- 172.21.15.78:0/3484006199 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c011d20 con 0x7f0d741601f0 2023-10-11T09:50:00.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.638+0000 7f0d7359e700 1 -- 172.21.15.78:0/3484006199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d741601f0 msgr2=0x7f0d740c8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:00.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.638+0000 7f0d7359e700 1 --2- 172.21.15.78:0/3484006199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d741601f0 0x7f0d740c8fc0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f0d74073e90 tx=0x7f0d5c009b30 comp rx=0 tx=0).stop 2023-10-11T09:50:00.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.638+0000 7f0d7359e700 1 -- 172.21.15.78:0/3484006199 shutdown_connections 2023-10-11T09:50:00.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.638+0000 7f0d7359e700 1 --2- 172.21.15.78:0/3484006199 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d740b1820 0x7f0d740b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:00.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.638+0000 7f0d7359e700 1 --2- 172.21.15.78:0/3484006199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d741601f0 0x7f0d740c8fc0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:00.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.638+0000 7f0d7359e700 1 --2- 172.21.15.78:0/3484006199 >> v2:172.21.15.133:3300/0 conn(0x7f0d7413f9e0 0x7f0d740ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:00.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.638+0000 7f0d7359e700 1 -- 172.21.15.78:0/3484006199 >> 172.21.15.78:0/3484006199 conn(0x7f0d7406dd80 msgr2=0x7f0d7406e180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:00.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.638+0000 7f0d7359e700 1 -- 172.21.15.78:0/3484006199 shutdown_connections 2023-10-11T09:50:00.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.638+0000 7f0d7359e700 1 -- 172.21.15.78:0/3484006199 wait complete. 2023-10-11T09:50:00.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.642+0000 7f0d7359e700 1 Processor -- start 2023-10-11T09:50:00.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.642+0000 7f0d7359e700 1 -- start start 2023-10-11T09:50:00.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.642+0000 7f0d7359e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d740b1820 0x7f0d740b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:00.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.642+0000 7f0d7359e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d7413f9e0 0x7f0d740b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:00.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.642+0000 7f0d7359e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d741601f0 0x7f0d741f1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:00.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.642+0000 7f0d7359e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d740b32e0 con 0x7f0d741601f0 2023-10-11T09:50:00.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.642+0000 7f0d7359e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d741f1d80 con 0x7f0d7413f9e0 2023-10-11T09:50:00.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.642+0000 7f0d7359e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d74159670 con 0x7f0d740b1820 2023-10-11T09:50:00.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.642+0000 7f0d72d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d741601f0 0x7f0d741f1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:00.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.642+0000 7f0d72d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d741601f0 0x7f0d741f1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36238/0 (socket says 172.21.15.78:36238) 2023-10-11T09:50:00.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.642+0000 7f0d72d9d700 1 -- 172.21.15.78:0/3063935919 learned_addr learned my addr 172.21.15.78:0/3063935919 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:00.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.642+0000 7f0d7259c700 1 --2- 172.21.15.78:0/3063935919 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d740b1820 0x7f0d740b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:00.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.642+0000 7f0d69d9b700 1 --2- 172.21.15.78:0/3063935919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d7413f9e0 0x7f0d740b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:00.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.642+0000 7f0d7259c700 1 -- 172.21.15.78:0/3063935919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d7413f9e0 msgr2=0x7f0d740b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:00.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.642+0000 7f0d7259c700 1 --2- 172.21.15.78:0/3063935919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d7413f9e0 0x7f0d740b2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:00.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.642+0000 7f0d7259c700 1 -- 172.21.15.78:0/3063935919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d741601f0 msgr2=0x7f0d741f1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:00.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.642+0000 7f0d7259c700 1 --2- 172.21.15.78:0/3063935919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d741601f0 0x7f0d741f1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:00.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.642+0000 7f0d7259c700 1 -- 172.21.15.78:0/3063935919 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d5c0094b0 con 0x7f0d740b1820 2023-10-11T09:50:00.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.642+0000 7f0d7259c700 1 --2- 172.21.15.78:0/3063935919 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d740b1820 0x7f0d740b34a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0d6400e820 tx=0x7f0d6400eb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:00.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.646+0000 7f0d6b7fe700 1 -- 172.21.15.78:0/3063935919 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d64013680 con 0x7f0d740b1820 2023-10-11T09:50:00.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.646+0000 7f0d6b7fe700 1 -- 172.21.15.78:0/3063935919 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d640137e0 con 0x7f0d740b1820 2023-10-11T09:50:00.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.646+0000 7f0d6b7fe700 1 -- 172.21.15.78:0/3063935919 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d64013b90 con 0x7f0d740b1820 2023-10-11T09:50:00.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.646+0000 7f0d7359e700 1 -- 172.21.15.78:0/3063935919 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d74159950 con 0x7f0d740b1820 2023-10-11T09:50:00.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.646+0000 7f0d7359e700 1 -- 172.21.15.78:0/3063935919 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d74159f10 con 0x7f0d740b1820 2023-10-11T09:50:00.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.646+0000 7f0d7359e700 1 -- 172.21.15.78:0/3063935919 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d7415a100 con 0x7f0d740b1820 2023-10-11T09:50:00.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.650+0000 7f0d6b7fe700 1 -- 172.21.15.78:0/3063935919 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f0d64013cf0 con 0x7f0d740b1820 2023-10-11T09:50:00.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.650+0000 7f0d6b7fe700 1 --2- 172.21.15.78:0/3063935919 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0d54065ed0 0x7f0d54068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:00.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.650+0000 7f0d6b7fe700 1 -- 172.21.15.78:0/3063935919 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6434+0+0 (secure 0 0 0) 0x7f0d64083cb0 con 0x7f0d740b1820 2023-10-11T09:50:00.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.650+0000 7f0d69d9b700 1 --2- 172.21.15.78:0/3063935919 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0d54065ed0 0x7f0d54068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:00.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.650+0000 7f0d6b7fe700 1 -- 172.21.15.78:0/3063935919 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f0d640511f0 con 0x7f0d740b1820 2023-10-11T09:50:00.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.650+0000 7f0d69d9b700 1 --2- 172.21.15.78:0/3063935919 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0d54065ed0 0x7f0d54068380 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c000c00 tx=0x7f0d5c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:00.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.870+0000 7f0d7359e700 1 -- 172.21.15.78:0/3063935919 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_extra"} v 0) v1 -- 0x7f0d74065d40 con 0x7f0d740b1820 2023-10-11T09:50:00.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.890+0000 7f0d6b7fe700 1 -- 172.21.15.78:0/3063935919 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_extra"}]=0 added data pool 16 to fsmap v80) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0d640170b0 con 0x7f0d740b1820 2023-10-11T09:50:00.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.898+0000 7f0d7359e700 1 -- 172.21.15.78:0/3063935919 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0d54065ed0 msgr2=0x7f0d54068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:00.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.898+0000 7f0d7359e700 1 --2- 172.21.15.78:0/3063935919 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0d54065ed0 0x7f0d54068380 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c000c00 tx=0x7f0d5c008030 comp rx=0 tx=0).stop 2023-10-11T09:50:00.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.898+0000 7f0d7359e700 1 -- 172.21.15.78:0/3063935919 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d740b1820 msgr2=0x7f0d740b34a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:00.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.898+0000 7f0d7359e700 1 --2- 172.21.15.78:0/3063935919 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d740b1820 0x7f0d740b34a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0d6400e820 tx=0x7f0d6400eb30 comp rx=0 tx=0).stop 2023-10-11T09:50:00.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.898+0000 7f0d7359e700 1 -- 172.21.15.78:0/3063935919 shutdown_connections 2023-10-11T09:50:00.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.898+0000 7f0d7359e700 1 --2- 172.21.15.78:0/3063935919 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0d54065ed0 0x7f0d54068380 secure :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c000c00 tx=0x7f0d5c008030 comp rx=0 tx=0).stop 2023-10-11T09:50:00.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.898+0000 7f0d7359e700 1 --2- 172.21.15.78:0/3063935919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d741601f0 0x7f0d741f1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:00.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.898+0000 7f0d7359e700 1 --2- 172.21.15.78:0/3063935919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d7413f9e0 0x7f0d740b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:00.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.898+0000 7f0d7359e700 1 --2- 172.21.15.78:0/3063935919 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d740b1820 0x7f0d740b34a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:00.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.898+0000 7f0d7359e700 1 -- 172.21.15.78:0/3063935919 >> 172.21.15.78:0/3063935919 conn(0x7f0d7406dd80 msgr2=0x7f0d740cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:00.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.902+0000 7f0d7359e700 1 -- 172.21.15.78:0/3063935919 shutdown_connections 2023-10-11T09:50:00.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:00.902+0000 7f0d7359e700 1 -- 172.21.15.78:0/3063935919 wait complete. 2023-10-11T09:50:00.913 INFO:teuthology.orchestra.run.smithi078.stderr:added data pool 16 to fsmap 2023-10-11T09:50:00.929 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:50:01.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.090+0000 7f292087b700 1 Processor -- start 2023-10-11T09:50:01.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.090+0000 7f292087b700 1 -- start start 2023-10-11T09:50:01.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.090+0000 7f292087b700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f291c0cbd30 0x7f291c06be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:01.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.090+0000 7f292087b700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f291c0c93a0 con 0x7f291c15c880 2023-10-11T09:50:01.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.090+0000 7f292087b700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f291c0cca10 con 0x7f291c0cd210 2023-10-11T09:50:01.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.090+0000 7f292087b700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f291c161420 con 0x7f291c0cbd30 2023-10-11T09:50:01.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.094+0000 7f291a59c700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f291c15c880 0x7f291c0cb620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:55024/0 (socket says 172.21.15.78:55024) 2023-10-11T09:50:01.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.094+0000 7f291a59c700 1 -- 172.21.15.78:0/1397502138 learned_addr learned my addr 172.21.15.78:0/1397502138 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:50:01.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.094+0000 7f2911d9b700 1 --2- 172.21.15.78:0/1397502138 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f291c0cbd30 0x7f291c06be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:01.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.094+0000 7f2919d9b700 1 -- 172.21.15.78:0/1397502138 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1590967807 0 0) 0x7f291c0c93a0 con 0x7f291c15c880 2023-10-11T09:50:01.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.094+0000 7f2919d9b700 1 -- 172.21.15.78:0/1397502138 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29080035d0 con 0x7f291c15c880 2023-10-11T09:50:01.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.094+0000 7f2919d9b700 1 -- 172.21.15.78:0/1397502138 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3618498834 0 0) 0x7f291c0cca10 con 0x7f291c0cd210 2023-10-11T09:50:01.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.094+0000 7f2919d9b700 1 -- 172.21.15.78:0/1397502138 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f291c0c93a0 con 0x7f291c0cd210 2023-10-11T09:50:01.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.094+0000 7f2911d9b700 1 -- 172.21.15.78:0/1397502138 >> v1:172.21.15.133:6789/0 conn(0x7f291c0cd210 legacy=0x7f291c0c9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:01.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.094+0000 7f2911d9b700 1 -- 172.21.15.78:0/1397502138 >> v1:172.21.15.78:6789/0 conn(0x7f291c15c880 legacy=0x7f291c0cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:01.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.094+0000 7f2911d9b700 1 -- 172.21.15.78:0/1397502138 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f291c0cccb0 con 0x7f291c0cbd30 2023-10-11T09:50:01.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.094+0000 7f2911d9b700 1 --2- 172.21.15.78:0/1397502138 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f291c0cbd30 0x7f291c06be60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2904009880 tx=0x7f2904009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=deee5ae2e96be3c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:01.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.094+0000 7f2919d9b700 1 -- 172.21.15.78:0/1397502138 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2904011830 con 0x7f291c0cbd30 2023-10-11T09:50:01.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.094+0000 7f2919d9b700 1 -- 172.21.15.78:0/1397502138 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2904011990 con 0x7f291c0cbd30 2023-10-11T09:50:01.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.094+0000 7f2919d9b700 1 -- 172.21.15.78:0/1397502138 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2904011d00 con 0x7f291c0cbd30 2023-10-11T09:50:01.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.094+0000 7f292087b700 1 -- 172.21.15.78:0/1397502138 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f291c0cbd30 msgr2=0x7f291c06be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:01.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.094+0000 7f292087b700 1 --2- 172.21.15.78:0/1397502138 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f291c0cbd30 0x7f291c06be60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2904009880 tx=0x7f2904009b90 comp rx=0 tx=0).stop 2023-10-11T09:50:01.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.094+0000 7f292087b700 1 -- 172.21.15.78:0/1397502138 shutdown_connections 2023-10-11T09:50:01.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.094+0000 7f292087b700 1 --2- 172.21.15.78:0/1397502138 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f291c0cbd30 0x7f291c06be60 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:01.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.094+0000 7f292087b700 1 -- 172.21.15.78:0/1397502138 >> 172.21.15.78:0/1397502138 conn(0x7f291c06aca0 msgr2=0x7f291c0b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:01.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.094+0000 7f292087b700 1 -- 172.21.15.78:0/1397502138 shutdown_connections 2023-10-11T09:50:01.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.094+0000 7f292087b700 1 -- 172.21.15.78:0/1397502138 wait complete. 2023-10-11T09:50:01.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.098+0000 7f292087b700 1 Processor -- start 2023-10-11T09:50:01.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.102+0000 7f292087b700 1 -- start start 2023-10-11T09:50:01.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.102+0000 7f292087b700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f291c0cbd30 0x7f291c153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:01.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.102+0000 7f292087b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f291c0cd210 0x7f291c146c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:01.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.102+0000 7f292087b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f291c15c880 0x7f291c0b3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:01.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.102+0000 7f292087b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f291c1577a0 con 0x7f291c15c880 2023-10-11T09:50:01.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.102+0000 7f292087b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f291c147140 con 0x7f291c0cd210 2023-10-11T09:50:01.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.102+0000 7f292087b700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f291c0b3040 con 0x7f291c0cbd30 2023-10-11T09:50:01.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.102+0000 7f291ad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f291c15c880 0x7f291c0b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:01.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.102+0000 7f291ad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f291c15c880 0x7f291c0b3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36242/0 (socket says 172.21.15.78:36242) 2023-10-11T09:50:01.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.102+0000 7f291ad9d700 1 -- 172.21.15.78:0/3212920007 learned_addr learned my addr 172.21.15.78:0/3212920007 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:01.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.102+0000 7f291a59c700 1 --2- 172.21.15.78:0/3212920007 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f291c0cbd30 0x7f291c153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:01.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.102+0000 7f2911d9b700 1 --2- 172.21.15.78:0/3212920007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f291c0cd210 0x7f291c146c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:01.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.102+0000 7f291ad9d700 1 -- 172.21.15.78:0/3212920007 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f291c0cbd30 msgr2=0x7f291c153b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:01.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.102+0000 7f291ad9d700 1 --2- 172.21.15.78:0/3212920007 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f291c0cbd30 0x7f291c153b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:01.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.102+0000 7f291ad9d700 1 -- 172.21.15.78:0/3212920007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f291c0cd210 msgr2=0x7f291c146c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:01.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.102+0000 7f291ad9d700 1 --2- 172.21.15.78:0/3212920007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f291c0cd210 0x7f291c146c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:01.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.102+0000 7f291ad9d700 1 -- 172.21.15.78:0/3212920007 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29040094b0 con 0x7f291c15c880 2023-10-11T09:50:01.114 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.102+0000 7f291ad9d700 1 --2- 172.21.15.78:0/3212920007 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f291c15c880 0x7f291c0b3380 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f291400cc80 tx=0x7f291400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:01.114 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.102+0000 7f29137fe700 1 -- 172.21.15.78:0/3212920007 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291400ad60 con 0x7f291c15c880 2023-10-11T09:50:01.114 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.102+0000 7f29137fe700 1 -- 172.21.15.78:0/3212920007 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291400ce80 con 0x7f291c15c880 2023-10-11T09:50:01.114 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.102+0000 7f29137fe700 1 -- 172.21.15.78:0/3212920007 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2914003c60 con 0x7f291c15c880 2023-10-11T09:50:01.114 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.102+0000 7f292087b700 1 -- 172.21.15.78:0/3212920007 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f291c0b2d60 con 0x7f291c15c880 2023-10-11T09:50:01.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.102+0000 7f292087b700 1 -- 172.21.15.78:0/3212920007 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f291c157430 con 0x7f291c15c880 2023-10-11T09:50:01.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.106+0000 7f292087b700 1 -- 172.21.15.78:0/3212920007 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f291c157950 con 0x7f291c15c880 2023-10-11T09:50:01.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.106+0000 7f29137fe700 1 -- 172.21.15.78:0/3212920007 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f29140050d0 con 0x7f291c15c880 2023-10-11T09:50:01.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.110+0000 7f29137fe700 1 --2- 172.21.15.78:0/3212920007 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f28fc065ed0 0x7f28fc068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:01.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.110+0000 7f29137fe700 1 -- 172.21.15.78:0/3212920007 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f2914083b00 con 0x7f291c15c880 2023-10-11T09:50:01.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.110+0000 7f291a59c700 1 --2- 172.21.15.78:0/3212920007 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f28fc065ed0 0x7f28fc068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:01.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.110+0000 7f291a59c700 1 --2- 172.21.15.78:0/3212920007 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f28fc065ed0 0x7f28fc068380 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f290c0011e0 tx=0x7f290c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:01.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.118+0000 7f29137fe700 1 -- 172.21.15.78:0/3212920007 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f291c157950 con 0x7f291c15c880 2023-10-11T09:50:01.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.326+0000 7f292087b700 1 -- 172.21.15.78:0/3212920007 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f291c157950 con 0x7f291c15c880 2023-10-11T09:50:01.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.326+0000 7f29137fe700 1 -- 172.21.15.78:0/3212920007 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4659 (secure 0 0 0) 0x7f291c157950 con 0x7f291c15c880 2023-10-11T09:50:01.335 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:01.336 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":80,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6812,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/3697741427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3697741427},{"type":"v1","addr":"172.21.15.133:6833","nonce":3697741427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":6815,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2640710924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2640710924},{"type":"v1","addr":"172.21.15.133:6835","nonce":2640710924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8028,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/1538320637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":1538320637},{"type":"v1","addr":"172.21.15.78:6835","nonce":1538320637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:49:50.956153+0000","modified":"2023-10-11T09:50:00.877567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8031},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8031":{"gid":8031,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6837/1286798022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1286798022},{"type":"v1","addr":"172.21.15.78:6837","nonce":1286798022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15,16],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-11T09:50:01.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.334+0000 7f292087b700 1 -- 172.21.15.78:0/3212920007 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f28fc065ed0 msgr2=0x7f28fc068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:01.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.334+0000 7f292087b700 1 --2- 172.21.15.78:0/3212920007 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f28fc065ed0 0x7f28fc068380 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f290c0011e0 tx=0x7f290c009040 comp rx=0 tx=0).stop 2023-10-11T09:50:01.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.334+0000 7f292087b700 1 -- 172.21.15.78:0/3212920007 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f291c15c880 msgr2=0x7f291c0b3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:01.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.334+0000 7f292087b700 1 --2- 172.21.15.78:0/3212920007 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f291c15c880 0x7f291c0b3380 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f291400cc80 tx=0x7f291400a340 comp rx=0 tx=0).stop 2023-10-11T09:50:01.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.334+0000 7f292087b700 1 -- 172.21.15.78:0/3212920007 shutdown_connections 2023-10-11T09:50:01.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.334+0000 7f292087b700 1 --2- 172.21.15.78:0/3212920007 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f28fc065ed0 0x7f28fc068380 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:01.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.334+0000 7f292087b700 1 --2- 172.21.15.78:0/3212920007 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f291c15c880 0x7f291c0b3380 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:01.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.334+0000 7f292087b700 1 --2- 172.21.15.78:0/3212920007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f291c0cd210 0x7f291c146c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:01.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.334+0000 7f292087b700 1 --2- 172.21.15.78:0/3212920007 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f291c0cbd30 0x7f291c153b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:01.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.334+0000 7f292087b700 1 -- 172.21.15.78:0/3212920007 >> 172.21.15.78:0/3212920007 conn(0x7f291c06aca0 msgr2=0x7f291c0c9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:01.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.334+0000 7f292087b700 1 -- 172.21.15.78:0/3212920007 shutdown_connections 2023-10-11T09:50:01.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.334+0000 7f292087b700 1 -- 172.21.15.78:0/3212920007 wait complete. 2023-10-11T09:50:01.343 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 80 2023-10-11T09:50:01.360 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:50:01.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.526+0000 7f4b4f1ef700 1 Processor -- start 2023-10-11T09:50:01.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.526+0000 7f4b4f1ef700 1 -- start start 2023-10-11T09:50:01.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.526+0000 7f4b4f1ef700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4b4815a940 0x7f4b480ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:01.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.526+0000 7f4b4f1ef700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b4815e010 con 0x7f4b4815b8e0 2023-10-11T09:50:01.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.526+0000 7f4b4f1ef700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b480cab80 con 0x7f4b480ca780 2023-10-11T09:50:01.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.526+0000 7f4b4f1ef700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4815e280 con 0x7f4b4815a940 2023-10-11T09:50:01.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.526+0000 7f4b4e1ed700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f4b4815b8e0 0x7f4b4806d0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:55026/0 (socket says 172.21.15.78:55026) 2023-10-11T09:50:01.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.526+0000 7f4b4e1ed700 1 -- 172.21.15.78:0/1311664370 learned_addr learned my addr 172.21.15.78:0/1311664370 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:50:01.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.526+0000 7f4b4d9ec700 1 --2- 172.21.15.78:0/1311664370 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4b4815a940 0x7f4b480ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:01.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.530+0000 7f4b4d1eb700 1 -- 172.21.15.78:0/1311664370 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1919950934 0 0) 0x7f4b4815e010 con 0x7f4b4815b8e0 2023-10-11T09:50:01.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.530+0000 7f4b4d1eb700 1 -- 172.21.15.78:0/1311664370 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b340035d0 con 0x7f4b4815b8e0 2023-10-11T09:50:01.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.530+0000 7f4b4d1eb700 1 -- 172.21.15.78:0/1311664370 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 645879517 0 0) 0x7f4b480cab80 con 0x7f4b480ca780 2023-10-11T09:50:01.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.530+0000 7f4b4d1eb700 1 -- 172.21.15.78:0/1311664370 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b4815e010 con 0x7f4b480ca780 2023-10-11T09:50:01.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.530+0000 7f4b4d9ec700 1 -- 172.21.15.78:0/1311664370 >> v1:172.21.15.133:6789/0 conn(0x7f4b480ca780 legacy=0x7f4b4813fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:01.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.530+0000 7f4b4d9ec700 1 -- 172.21.15.78:0/1311664370 >> v1:172.21.15.78:6789/0 conn(0x7f4b4815b8e0 legacy=0x7f4b4806d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:01.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.530+0000 7f4b4d1eb700 1 -- 172.21.15.78:0/1311664370 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1576439959 0 0) 0x7f4b340035d0 con 0x7f4b4815b8e0 2023-10-11T09:50:01.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.530+0000 7f4b4d9ec700 1 -- 172.21.15.78:0/1311664370 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b4815e560 con 0x7f4b4815a940 2023-10-11T09:50:01.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.530+0000 7f4b4d9ec700 1 --2- 172.21.15.78:0/1311664370 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4b4815a940 0x7f4b480ca240 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4b38009880 tx=0x7f4b38009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a53e1a358b5faaa7 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:01.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.530+0000 7f4b4d1eb700 1 -- 172.21.15.78:0/1311664370 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b44001670 con 0x7f4b4815a940 2023-10-11T09:50:01.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.530+0000 7f4b4d1eb700 1 -- 172.21.15.78:0/1311664370 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b38004030 con 0x7f4b4815a940 2023-10-11T09:50:01.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.530+0000 7f4b4d1eb700 1 -- 172.21.15.78:0/1311664370 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b380043c0 con 0x7f4b4815a940 2023-10-11T09:50:01.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.530+0000 7f4b4f1ef700 1 -- 172.21.15.78:0/1311664370 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4b4815a940 msgr2=0x7f4b480ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:01.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.530+0000 7f4b4f1ef700 1 --2- 172.21.15.78:0/1311664370 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4b4815a940 0x7f4b480ca240 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4b38009880 tx=0x7f4b38009b90 comp rx=0 tx=0).stop 2023-10-11T09:50:01.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.530+0000 7f4b4f1ef700 1 -- 172.21.15.78:0/1311664370 shutdown_connections 2023-10-11T09:50:01.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.530+0000 7f4b4f1ef700 1 --2- 172.21.15.78:0/1311664370 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4b4815a940 0x7f4b480ca240 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:01.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.530+0000 7f4b4f1ef700 1 -- 172.21.15.78:0/1311664370 >> 172.21.15.78:0/1311664370 conn(0x7f4b480cea60 msgr2=0x7f4b480cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:01.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.530+0000 7f4b4f1ef700 1 -- 172.21.15.78:0/1311664370 shutdown_connections 2023-10-11T09:50:01.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.530+0000 7f4b4f1ef700 1 -- 172.21.15.78:0/1311664370 wait complete. 2023-10-11T09:50:01.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.534+0000 7f4b4f1ef700 1 Processor -- start 2023-10-11T09:50:01.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.534+0000 7f4b4f1ef700 1 -- start start 2023-10-11T09:50:01.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.534+0000 7f4b4f1ef700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b480ca780 0x7f4b4813f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:01.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.534+0000 7f4b4f1ef700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4b4815a940 0x7f4b480afca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:01.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.534+0000 7f4b4f1ef700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b4815b8e0 0x7f4b480c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:01.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.534+0000 7f4b4f1ef700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b481f1b10 con 0x7f4b4815b8e0 2023-10-11T09:50:01.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.534+0000 7f4b4f1ef700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4813f9f0 con 0x7f4b480ca780 2023-10-11T09:50:01.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.534+0000 7f4b4f1ef700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4813fcd0 con 0x7f4b4815a940 2023-10-11T09:50:01.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.534+0000 7f4b4e9ee700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b4815b8e0 0x7f4b480c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:01.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.534+0000 7f4b4e9ee700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b4815b8e0 0x7f4b480c7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36244/0 (socket says 172.21.15.78:36244) 2023-10-11T09:50:01.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.534+0000 7f4b4e9ee700 1 -- 172.21.15.78:0/2849970154 learned_addr learned my addr 172.21.15.78:0/2849970154 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:01.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.534+0000 7f4b4e1ed700 1 --2- 172.21.15.78:0/2849970154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b480ca780 0x7f4b4813f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:01.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.534+0000 7f4b4d9ec700 1 --2- 172.21.15.78:0/2849970154 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4b4815a940 0x7f4b480afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:01.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.538+0000 7f4b4e9ee700 1 -- 172.21.15.78:0/2849970154 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4b4815a940 msgr2=0x7f4b480afca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:01.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.538+0000 7f4b4e9ee700 1 --2- 172.21.15.78:0/2849970154 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4b4815a940 0x7f4b480afca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:01.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.538+0000 7f4b4e9ee700 1 -- 172.21.15.78:0/2849970154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b480ca780 msgr2=0x7f4b4813f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:01.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.538+0000 7f4b4e9ee700 1 --2- 172.21.15.78:0/2849970154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b480ca780 0x7f4b4813f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:01.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.538+0000 7f4b4e9ee700 1 -- 172.21.15.78:0/2849970154 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b380094b0 con 0x7f4b4815b8e0 2023-10-11T09:50:01.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.538+0000 7f4b4d9ec700 1 --2- 172.21.15.78:0/2849970154 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4b4815a940 0x7f4b480afca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:50:01.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.538+0000 7f4b4e9ee700 1 --2- 172.21.15.78:0/2849970154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b4815b8e0 0x7f4b480c7d90 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000cbc0 tx=0x7f4b4000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:01.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.538+0000 7f4b3effd700 1 -- 172.21.15.78:0/2849970154 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b40003c20 con 0x7f4b4815b8e0 2023-10-11T09:50:01.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.538+0000 7f4b3effd700 1 -- 172.21.15.78:0/2849970154 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b40003d80 con 0x7f4b4815b8e0 2023-10-11T09:50:01.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.538+0000 7f4b3effd700 1 -- 172.21.15.78:0/2849970154 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4000ab00 con 0x7f4b4815b8e0 2023-10-11T09:50:01.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.538+0000 7f4b4f1ef700 1 -- 172.21.15.78:0/2849970154 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b4813ffb0 con 0x7f4b4815b8e0 2023-10-11T09:50:01.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.538+0000 7f4b4f1ef700 1 -- 172.21.15.78:0/2849970154 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b4813f630 con 0x7f4b4815b8e0 2023-10-11T09:50:01.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.538+0000 7f4b4f1ef700 1 -- 172.21.15.78:0/2849970154 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b4815e820 con 0x7f4b4815b8e0 2023-10-11T09:50:01.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.538+0000 7f4b3effd700 1 -- 172.21.15.78:0/2849970154 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f4b4000ac60 con 0x7f4b4815b8e0 2023-10-11T09:50:01.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.542+0000 7f4b3effd700 1 --2- 172.21.15.78:0/2849970154 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4b2c065ed0 0x7f4b2c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:01.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.542+0000 7f4b3effd700 1 -- 172.21.15.78:0/2849970154 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f4b40083a70 con 0x7f4b4815b8e0 2023-10-11T09:50:01.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.542+0000 7f4b4e1ed700 1 --2- 172.21.15.78:0/2849970154 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4b2c065ed0 0x7f4b2c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:01.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.542+0000 7f4b4e1ed700 1 --2- 172.21.15.78:0/2849970154 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4b2c065ed0 0x7f4b2c068380 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f4b44001380 tx=0x7f4b44001770 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:01.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.550+0000 7f4b3effd700 1 -- 172.21.15.78:0/2849970154 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f4b4815e820 con 0x7f4b4815b8e0 2023-10-11T09:50:01.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.726+0000 7f4b4f1ef700 1 -- 172.21.15.78:0/2849970154 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b4815e820 con 0x7f4b4815b8e0 2023-10-11T09:50:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.730+0000 7f4b3effd700 1 -- 172.21.15.78:0/2849970154 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+19636 (secure 0 0 0) 0x7f4b4815e820 con 0x7f4b4815b8e0 2023-10-11T09:50:01.737 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:01.737 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":107,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:50:00.876926+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:49:49.297597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-11T09:49:49.955104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data_extra","create_time":"2023-10-11T09:49:59.915561+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:50:01.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.730+0000 7f4b4f1ef700 1 -- 172.21.15.78:0/2849970154 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4b2c065ed0 msgr2=0x7f4b2c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:01.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.730+0000 7f4b4f1ef700 1 --2- 172.21.15.78:0/2849970154 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4b2c065ed0 0x7f4b2c068380 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f4b44001380 tx=0x7f4b44001770 comp rx=0 tx=0).stop 2023-10-11T09:50:01.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.730+0000 7f4b4f1ef700 1 -- 172.21.15.78:0/2849970154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b4815b8e0 msgr2=0x7f4b480c7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:01.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.730+0000 7f4b4f1ef700 1 --2- 172.21.15.78:0/2849970154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b4815b8e0 0x7f4b480c7d90 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000cbc0 tx=0x7f4b4000ced0 comp rx=0 tx=0).stop 2023-10-11T09:50:01.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.730+0000 7f4b4f1ef700 1 -- 172.21.15.78:0/2849970154 shutdown_connections 2023-10-11T09:50:01.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.730+0000 7f4b4f1ef700 1 --2- 172.21.15.78:0/2849970154 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4b2c065ed0 0x7f4b2c068380 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:01.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.730+0000 7f4b4f1ef700 1 --2- 172.21.15.78:0/2849970154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b4815b8e0 0x7f4b480c7d90 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:01.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.730+0000 7f4b4f1ef700 1 --2- 172.21.15.78:0/2849970154 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4b4815a940 0x7f4b480afca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:01.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.730+0000 7f4b4f1ef700 1 --2- 172.21.15.78:0/2849970154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b480ca780 0x7f4b4813f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:01.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.730+0000 7f4b4f1ef700 1 -- 172.21.15.78:0/2849970154 >> 172.21.15.78:0/2849970154 conn(0x7f4b480cea60 msgr2=0x7f4b4806d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:01.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.730+0000 7f4b4f1ef700 1 -- 172.21.15.78:0/2849970154 shutdown_connections 2023-10-11T09:50:01.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:01.730+0000 7f4b4f1ef700 1 -- 172.21.15.78:0/2849970154 wait complete. 2023-10-11T09:50:01.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:50:01.761 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir stripey) 2023-10-11T09:50:01.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:50:01.792 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout -v 'stripe_unit=65536 stripe_count=4 object_size=262144 pool=cephfs_data_extra' ./stripey) 2023-10-11T09:50:01.870 INFO:tasks.cephfs.mount:Writing 1048576 bytes to stripey/flushed_file_0 2023-10-11T09:50:01.871 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:50:01.871 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:50:01.871 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_0" 2023-10-11T09:50:01.872 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'w'"'"') as f: 2023-10-11T09:50:01.872 DEBUG:teuthology.orchestra.run.smithi078:> for i in range(0, 1048576): 2023-10-11T09:50:01.872 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:50:01.873 DEBUG:teuthology.orchestra.run.smithi078:> f.write(chr(val)) 2023-10-11T09:50:01.873 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:50:02.595 INFO:tasks.cephfs.mount:Validating 1048576 bytes from stripey/flushed_file_0 2023-10-11T09:50:02.596 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:50:02.596 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:50:02.597 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_0" 2023-10-11T09:50:02.597 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:50:02.597 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:50:02.598 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 1048576: 2023-10-11T09:50:02.598 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:50:02.598 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 1048576 2023-10-11T09:50:02.599 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:50:02.599 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:50:02.600 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:50:02.600 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:50:02.600 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:50:02.601 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:50:03.331 INFO:tasks.cephfs.mount:Writing 524288 bytes to stripey/flushed_file_1 2023-10-11T09:50:03.331 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:50:03.332 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:50:03.332 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_1" 2023-10-11T09:50:03.332 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'w'"'"') as f: 2023-10-11T09:50:03.333 DEBUG:teuthology.orchestra.run.smithi078:> for i in range(0, 524288): 2023-10-11T09:50:03.333 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:50:03.333 DEBUG:teuthology.orchestra.run.smithi078:> f.write(chr(val)) 2023-10-11T09:50:03.334 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:50:03.851 INFO:tasks.cephfs.mount:Validating 524288 bytes from stripey/flushed_file_1 2023-10-11T09:50:03.852 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:50:03.852 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:50:03.852 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_1" 2023-10-11T09:50:03.853 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:50:03.853 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:50:03.853 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 524288: 2023-10-11T09:50:03.854 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:50:03.854 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 524288 2023-10-11T09:50:03.854 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:50:03.855 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:50:03.855 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:50:03.855 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:50:03.856 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:50:03.856 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:50:04.390 INFO:tasks.cephfs.mount:Writing 917504 bytes to stripey/flushed_file_2 2023-10-11T09:50:04.391 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:50:04.391 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:50:04.391 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_2" 2023-10-11T09:50:04.392 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'w'"'"') as f: 2023-10-11T09:50:04.392 DEBUG:teuthology.orchestra.run.smithi078:> for i in range(0, 917504): 2023-10-11T09:50:04.392 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:50:04.393 DEBUG:teuthology.orchestra.run.smithi078:> f.write(chr(val)) 2023-10-11T09:50:04.393 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:50:05.108 INFO:tasks.cephfs.mount:Validating 917504 bytes from stripey/flushed_file_2 2023-10-11T09:50:05.109 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:50:05.109 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:50:05.109 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_2" 2023-10-11T09:50:05.110 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:50:05.110 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:50:05.110 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 917504: 2023-10-11T09:50:05.111 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:50:05.111 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 917504 2023-10-11T09:50:05.111 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:50:05.112 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:50:05.112 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:50:05.112 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:50:05.113 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:50:05.113 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:50:05.850 INFO:tasks.cephfs.mount:Writing 917503 bytes to stripey/flushed_file_3 2023-10-11T09:50:05.851 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:50:05.851 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:50:05.852 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_3" 2023-10-11T09:50:05.852 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'w'"'"') as f: 2023-10-11T09:50:05.852 DEBUG:teuthology.orchestra.run.smithi078:> for i in range(0, 917503): 2023-10-11T09:50:05.853 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:50:05.853 DEBUG:teuthology.orchestra.run.smithi078:> f.write(chr(val)) 2023-10-11T09:50:05.853 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:50:06.576 INFO:tasks.cephfs.mount:Validating 917503 bytes from stripey/flushed_file_3 2023-10-11T09:50:06.576 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:50:06.577 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:50:06.577 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_3" 2023-10-11T09:50:06.577 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:50:06.578 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:50:06.578 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 917503: 2023-10-11T09:50:06.579 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:50:06.579 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 917503 2023-10-11T09:50:06.579 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:50:06.579 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:50:06.580 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:50:06.580 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:50:06.580 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:50:06.581 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:50:07.321 INFO:tasks.cephfs.mount:Writing 1441792 bytes to stripey/flushed_file_4 2023-10-11T09:50:07.322 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:50:07.322 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:50:07.323 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_4" 2023-10-11T09:50:07.323 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'w'"'"') as f: 2023-10-11T09:50:07.323 DEBUG:teuthology.orchestra.run.smithi078:> for i in range(0, 1441792): 2023-10-11T09:50:07.323 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:50:07.324 DEBUG:teuthology.orchestra.run.smithi078:> f.write(chr(val)) 2023-10-11T09:50:07.324 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:50:08.241 INFO:tasks.cephfs.mount:Validating 1441792 bytes from stripey/flushed_file_4 2023-10-11T09:50:08.242 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:50:08.242 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:50:08.242 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_4" 2023-10-11T09:50:08.243 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:50:08.243 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:50:08.243 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 1441792: 2023-10-11T09:50:08.244 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:50:08.244 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 1441792 2023-10-11T09:50:08.244 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:50:08.245 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:50:08.245 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:50:08.245 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:50:08.246 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:50:08.246 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:50:09.179 INFO:tasks.cephfs.mount:Writing 1441793 bytes to stripey/flushed_file_5 2023-10-11T09:50:09.180 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:50:09.180 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:50:09.180 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_5" 2023-10-11T09:50:09.181 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'w'"'"') as f: 2023-10-11T09:50:09.181 DEBUG:teuthology.orchestra.run.smithi078:> for i in range(0, 1441793): 2023-10-11T09:50:09.181 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:50:09.182 DEBUG:teuthology.orchestra.run.smithi078:> f.write(chr(val)) 2023-10-11T09:50:09.182 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:50:10.103 INFO:tasks.cephfs.mount:Validating 1441793 bytes from stripey/flushed_file_5 2023-10-11T09:50:10.104 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:50:10.104 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:50:10.104 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_5" 2023-10-11T09:50:10.105 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:50:10.105 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:50:10.105 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 1441793: 2023-10-11T09:50:10.106 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:50:10.106 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 1441793 2023-10-11T09:50:10.106 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:50:10.107 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:50:10.107 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:50:10.108 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:50:10.108 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:50:10.108 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:50:11.039 INFO:tasks.cephfs.mount:Writing 1572864 bytes to stripey/flushed_file_6 2023-10-11T09:50:11.040 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:50:11.040 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:50:11.040 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_6" 2023-10-11T09:50:11.041 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'w'"'"') as f: 2023-10-11T09:50:11.041 DEBUG:teuthology.orchestra.run.smithi078:> for i in range(0, 1572864): 2023-10-11T09:50:11.041 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:50:11.042 DEBUG:teuthology.orchestra.run.smithi078:> f.write(chr(val)) 2023-10-11T09:50:11.042 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:50:12.166 INFO:tasks.cephfs.mount:Validating 1572864 bytes from stripey/flushed_file_6 2023-10-11T09:50:12.167 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:50:12.167 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:50:12.168 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_6" 2023-10-11T09:50:12.168 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:50:12.168 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:50:12.169 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 1572864: 2023-10-11T09:50:12.169 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:50:12.169 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 1572864 2023-10-11T09:50:12.170 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:50:12.170 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:50:12.170 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:50:12.171 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:50:12.171 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:50:12.171 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:50:13.110 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:50:13.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.318+0000 7f6231964700 1 Processor -- start 2023-10-11T09:50:13.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.318+0000 7f6231964700 1 -- start start 2023-10-11T09:50:13.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.318+0000 7f6231964700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f622c15c970 0x7f622c0b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:13.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.318+0000 7f6231964700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f622c06a7e0 0x7f622c160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:13.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.318+0000 7f6231964700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f622c06b680 con 0x7f622c161070 2023-10-11T09:50:13.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.318+0000 7f6231964700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f622c0b1580 con 0x7f622c06a7e0 2023-10-11T09:50:13.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.318+0000 7f6231964700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f622c0b1860 con 0x7f622c15c970 2023-10-11T09:50:13.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.318+0000 7f6223fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f622c06a7e0 0x7f622c160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:13.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.318+0000 7f6223fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f622c06a7e0 0x7f622c160b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36590/0 (socket says 172.21.15.78:36590) 2023-10-11T09:50:13.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.318+0000 7f6223fff700 1 -- 172.21.15.78:0/3358887303 learned_addr learned my addr 172.21.15.78:0/3358887303 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:13.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.318+0000 7f6230962700 1 --2- 172.21.15.78:0/3358887303 >> v2:172.21.15.133:3300/0 conn(0x7f622c15c970 0x7f622c0b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:13.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.322+0000 7f6223fff700 1 -- 172.21.15.78:0/3358887303 >> v2:172.21.15.133:3300/0 conn(0x7f622c15c970 msgr2=0x7f622c0b0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:13.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.322+0000 7f6223fff700 1 --2- 172.21.15.78:0/3358887303 >> v2:172.21.15.133:3300/0 conn(0x7f622c15c970 0x7f622c0b0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:13.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.322+0000 7f6223fff700 1 -- 172.21.15.78:0/3358887303 >> v1:172.21.15.133:6789/0 conn(0x7f622c161070 legacy=0x7f622c0cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:13.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.322+0000 7f6223fff700 1 -- 172.21.15.78:0/3358887303 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f622c0b1b40 con 0x7f622c06a7e0 2023-10-11T09:50:13.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.322+0000 7f6223fff700 1 --2- 172.21.15.78:0/3358887303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f622c06a7e0 0x7f622c160b30 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f622c06a740 tx=0x7f621c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=510dc131f7d7c831 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:13.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.322+0000 7f62237fe700 1 -- 172.21.15.78:0/3358887303 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c011830 con 0x7f622c06a7e0 2023-10-11T09:50:13.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.322+0000 7f62237fe700 1 -- 172.21.15.78:0/3358887303 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f621c011990 con 0x7f622c06a7e0 2023-10-11T09:50:13.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.322+0000 7f62237fe700 1 -- 172.21.15.78:0/3358887303 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c011d20 con 0x7f622c06a7e0 2023-10-11T09:50:13.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.322+0000 7f6231964700 1 -- 172.21.15.78:0/3358887303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f622c06a7e0 msgr2=0x7f622c160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:13.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.322+0000 7f6231964700 1 --2- 172.21.15.78:0/3358887303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f622c06a7e0 0x7f622c160b30 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f622c06a740 tx=0x7f621c009bf0 comp rx=0 tx=0).stop 2023-10-11T09:50:13.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.322+0000 7f6231964700 1 -- 172.21.15.78:0/3358887303 shutdown_connections 2023-10-11T09:50:13.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.322+0000 7f6231964700 1 --2- 172.21.15.78:0/3358887303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f622c06a7e0 0x7f622c160b30 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:13.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.322+0000 7f6231964700 1 --2- 172.21.15.78:0/3358887303 >> v2:172.21.15.133:3300/0 conn(0x7f622c15c970 0x7f622c0b0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:13.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.322+0000 7f6231964700 1 -- 172.21.15.78:0/3358887303 >> 172.21.15.78:0/3358887303 conn(0x7f622c06c590 msgr2=0x7f622c06c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:13.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.322+0000 7f6231964700 1 -- 172.21.15.78:0/3358887303 shutdown_connections 2023-10-11T09:50:13.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.322+0000 7f6231964700 1 -- 172.21.15.78:0/3358887303 wait complete. 2023-10-11T09:50:13.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.326+0000 7f6231964700 1 Processor -- start 2023-10-11T09:50:13.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.326+0000 7f6231964700 1 -- start start 2023-10-11T09:50:13.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.326+0000 7f6231964700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f622c06a7e0 0x7f622c143b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:13.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.326+0000 7f6231964700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f622c15c970 0x7f622c0c2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:13.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.326+0000 7f6231964700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f622c161070 0x7f622c0c34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:13.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.326+0000 7f6231964700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c1440b0 con 0x7f622c15c970 2023-10-11T09:50:13.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.326+0000 7f6231964700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c144330 con 0x7f622c06a7e0 2023-10-11T09:50:13.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.326+0000 7f6231964700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f622c1560d0 con 0x7f622c161070 2023-10-11T09:50:13.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.326+0000 7f6223fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f622c15c970 0x7f622c0c2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:13.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.326+0000 7f6223fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f622c15c970 0x7f622c0c2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36592/0 (socket says 172.21.15.78:36592) 2023-10-11T09:50:13.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.326+0000 7f6223fff700 1 -- 172.21.15.78:0/3817581488 learned_addr learned my addr 172.21.15.78:0/3817581488 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:13.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.330+0000 7f6231163700 1 --2- 172.21.15.78:0/3817581488 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f622c161070 0x7f622c0c34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:13.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.330+0000 7f6230962700 1 --2- 172.21.15.78:0/3817581488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f622c06a7e0 0x7f622c143b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:13.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.330+0000 7f6223fff700 1 -- 172.21.15.78:0/3817581488 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f622c161070 msgr2=0x7f622c0c34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:13.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.330+0000 7f6223fff700 1 --2- 172.21.15.78:0/3817581488 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f622c161070 0x7f622c0c34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:13.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.330+0000 7f6223fff700 1 -- 172.21.15.78:0/3817581488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f622c06a7e0 msgr2=0x7f622c143b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:13.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.330+0000 7f6223fff700 1 --2- 172.21.15.78:0/3817581488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f622c06a7e0 0x7f622c143b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:13.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.330+0000 7f6223fff700 1 -- 172.21.15.78:0/3817581488 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f621c009570 con 0x7f622c15c970 2023-10-11T09:50:13.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.330+0000 7f6231163700 1 --2- 172.21.15.78:0/3817581488 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f622c161070 0x7f622c0c34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:50:13.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.330+0000 7f6230962700 1 --2- 172.21.15.78:0/3817581488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f622c06a7e0 0x7f622c143b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:50:13.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.330+0000 7f6223fff700 1 --2- 172.21.15.78:0/3817581488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f622c15c970 0x7f622c0c2fa0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f621c000c00 tx=0x7f621c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:13.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.330+0000 7f62217fa700 1 -- 172.21.15.78:0/3817581488 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c004610 con 0x7f622c15c970 2023-10-11T09:50:13.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.330+0000 7f62217fa700 1 -- 172.21.15.78:0/3817581488 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f621c01e070 con 0x7f622c15c970 2023-10-11T09:50:13.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.330+0000 7f6231964700 1 -- 172.21.15.78:0/3817581488 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f622c1563b0 con 0x7f622c15c970 2023-10-11T09:50:13.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.330+0000 7f62217fa700 1 -- 172.21.15.78:0/3817581488 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c0038e0 con 0x7f622c15c970 2023-10-11T09:50:13.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.330+0000 7f6231964700 1 -- 172.21.15.78:0/3817581488 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f622c156970 con 0x7f622c15c970 2023-10-11T09:50:13.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.330+0000 7f6231964700 1 -- 172.21.15.78:0/3817581488 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f622c065d40 con 0x7f622c15c970 2023-10-11T09:50:13.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.334+0000 7f62217fa700 1 -- 172.21.15.78:0/3817581488 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f621c003a40 con 0x7f622c15c970 2023-10-11T09:50:13.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.334+0000 7f62217fa700 1 --2- 172.21.15.78:0/3817581488 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6218065ed0 0x7f6218068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:13.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.334+0000 7f62217fa700 1 -- 172.21.15.78:0/3817581488 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f621c084a70 con 0x7f622c15c970 2023-10-11T09:50:13.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.334+0000 7f6230962700 1 --2- 172.21.15.78:0/3817581488 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6218065ed0 0x7f6218068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:13.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.338+0000 7f6230962700 1 --2- 172.21.15.78:0/3817581488 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6218065ed0 0x7f6218068380 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f6228009fd0 tx=0x7f622800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:13.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.342+0000 7f62217fa700 1 -- 172.21.15.78:0/3817581488 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f621c052030 con 0x7f622c15c970 2023-10-11T09:50:13.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.546+0000 7f6231964700 1 -- 172.21.15.78:0/3817581488 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f622c0ce490 con 0x7f622c15c970 2023-10-11T09:50:13.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.550+0000 7f62217fa700 1 -- 172.21.15.78:0/3817581488 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4659 (secure 0 0 0) 0x7f621c015090 con 0x7f622c15c970 2023-10-11T09:50:13.556 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:13.557 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":80,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6812,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/3697741427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3697741427},{"type":"v1","addr":"172.21.15.133:6833","nonce":3697741427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":6815,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2640710924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2640710924},{"type":"v1","addr":"172.21.15.133:6835","nonce":2640710924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8028,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/1538320637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":1538320637},{"type":"v1","addr":"172.21.15.78:6835","nonce":1538320637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:49:50.956153+0000","modified":"2023-10-11T09:50:00.877567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8031},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8031":{"gid":8031,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6837/1286798022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1286798022},{"type":"v1","addr":"172.21.15.78:6837","nonce":1286798022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15,16],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-11T09:50:13.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.550+0000 7f6231964700 1 -- 172.21.15.78:0/3817581488 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6218065ed0 msgr2=0x7f6218068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:13.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.550+0000 7f6231964700 1 --2- 172.21.15.78:0/3817581488 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6218065ed0 0x7f6218068380 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f6228009fd0 tx=0x7f622800b040 comp rx=0 tx=0).stop 2023-10-11T09:50:13.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.550+0000 7f6231964700 1 -- 172.21.15.78:0/3817581488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f622c15c970 msgr2=0x7f622c0c2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:13.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.550+0000 7f6231964700 1 --2- 172.21.15.78:0/3817581488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f622c15c970 0x7f622c0c2fa0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f621c000c00 tx=0x7f621c004770 comp rx=0 tx=0).stop 2023-10-11T09:50:13.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.550+0000 7f6231964700 1 -- 172.21.15.78:0/3817581488 shutdown_connections 2023-10-11T09:50:13.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.550+0000 7f6231964700 1 --2- 172.21.15.78:0/3817581488 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6218065ed0 0x7f6218068380 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:13.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.550+0000 7f6231964700 1 --2- 172.21.15.78:0/3817581488 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f622c161070 0x7f622c0c34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:13.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.550+0000 7f6231964700 1 --2- 172.21.15.78:0/3817581488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f622c15c970 0x7f622c0c2fa0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:13.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.550+0000 7f6231964700 1 --2- 172.21.15.78:0/3817581488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f622c06a7e0 0x7f622c143b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:13.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.550+0000 7f6231964700 1 -- 172.21.15.78:0/3817581488 >> 172.21.15.78:0/3817581488 conn(0x7f622c06c590 msgr2=0x7f622c15bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:13.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.550+0000 7f6231964700 1 -- 172.21.15.78:0/3817581488 shutdown_connections 2023-10-11T09:50:13.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:13.550+0000 7f6231964700 1 -- 172.21.15.78:0/3817581488 wait complete. 2023-10-11T09:50:13.562 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 80 2023-10-11T09:50:13.579 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2023-10-11T09:50:13.781 INFO:teuthology.orchestra.run.smithi078.stdout:{"message":"","return_code":0} 2023-10-11T09:50:13.782 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-10-11T09:50:13.782 INFO:tasks.cephfs.mount:Writing 524288 bytes to stripey/unflushed_file 2023-10-11T09:50:13.783 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:50:13.783 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:50:13.783 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/stripey/unflushed_file" 2023-10-11T09:50:13.784 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'w'"'"') as f: 2023-10-11T09:50:13.784 DEBUG:teuthology.orchestra.run.smithi078:> for i in range(0, 524288): 2023-10-11T09:50:13.785 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:50:13.785 DEBUG:teuthology.orchestra.run.smithi078:> f.write(chr(val)) 2023-10-11T09:50:13.785 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:50:14.267 INFO:tasks.cephfs.mount:Validating 524288 bytes from stripey/unflushed_file 2023-10-11T09:50:14.267 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:50:14.268 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:50:14.268 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/stripey/unflushed_file" 2023-10-11T09:50:14.268 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:50:14.269 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:50:14.269 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 524288: 2023-10-11T09:50:14.269 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:50:14.270 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 524288 2023-10-11T09:50:14.270 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:50:14.270 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:50:14.271 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:50:14.271 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:50:14.271 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:50:14.272 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:50:14.811 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:50:14.811 DEBUG:teuthology.orchestra.run.smithi078:> import os 2023-10-11T09:50:14.812 DEBUG:teuthology.orchestra.run.smithi078:> import stat 2023-10-11T09:50:14.812 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:50:14.812 DEBUG:teuthology.orchestra.run.smithi078:> print(os.stat("/home/ubuntu/cephtest/mnt.0/stripey/unflushed_file").st_ino) 2023-10-11T09:50:14.813 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:50:14.948 INFO:teuthology.orchestra.run.smithi078.stdout:1099511628284 2023-10-11T09:50:15.139 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:50:15.140 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:50:15.198 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:50:15.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.410+0000 7f64cace7700 1 Processor -- start 2023-10-11T09:50:15.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.410+0000 7f64cace7700 1 -- start start 2023-10-11T09:50:15.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.410+0000 7f64cace7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64c406a770 0x7f64c41608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:15.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.410+0000 7f64cace7700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64c40ceab0 con 0x7f64c40af3e0 2023-10-11T09:50:15.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.410+0000 7f64cace7700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64c406d780 con 0x7f64c40af010 2023-10-11T09:50:15.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.410+0000 7f64cace7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64c40cfaf0 con 0x7f64c406a770 2023-10-11T09:50:15.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.414+0000 7f64c9284700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f64c40af3e0 0x7f64c4160df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:57404/0 (socket says 172.21.15.78:57404) 2023-10-11T09:50:15.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.414+0000 7f64c9284700 1 -- 172.21.15.78:0/1776351088 learned_addr learned my addr 172.21.15.78:0/1776351088 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:50:15.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.414+0000 7f64c3fff700 1 --2- 172.21.15.78:0/1776351088 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64c406a770 0x7f64c41608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:15.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.414+0000 7f64c37fe700 1 -- 172.21.15.78:0/1776351088 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1142106410 0 0) 0x7f64c406d780 con 0x7f64c40af010 2023-10-11T09:50:15.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.414+0000 7f64c37fe700 1 -- 172.21.15.78:0/1776351088 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64b40035d0 con 0x7f64c40af010 2023-10-11T09:50:15.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.414+0000 7f64c37fe700 1 -- 172.21.15.78:0/1776351088 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2263575370 0 0) 0x7f64c40ceab0 con 0x7f64c40af3e0 2023-10-11T09:50:15.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.414+0000 7f64c37fe700 1 -- 172.21.15.78:0/1776351088 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64c406d780 con 0x7f64c40af3e0 2023-10-11T09:50:15.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.414+0000 7f64c3fff700 1 -- 172.21.15.78:0/1776351088 >> v1:172.21.15.133:6789/0 conn(0x7f64c40af010 legacy=0x7f64c406a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:15.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.414+0000 7f64c3fff700 1 -- 172.21.15.78:0/1776351088 >> v1:172.21.15.78:6789/0 conn(0x7f64c40af3e0 legacy=0x7f64c4160df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:15.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.414+0000 7f64c3fff700 1 -- 172.21.15.78:0/1776351088 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c40ced30 con 0x7f64c406a770 2023-10-11T09:50:15.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.414+0000 7f64c3fff700 1 --2- 172.21.15.78:0/1776351088 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64c406a770 0x7f64c41608b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f64b0009880 tx=0x7f64b0009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d7fc5e1e932393d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:15.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.414+0000 7f64c37fe700 1 -- 172.21.15.78:0/1776351088 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b0011830 con 0x7f64c406a770 2023-10-11T09:50:15.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.414+0000 7f64c37fe700 1 -- 172.21.15.78:0/1776351088 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b0011990 con 0x7f64c406a770 2023-10-11T09:50:15.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.414+0000 7f64c37fe700 1 -- 172.21.15.78:0/1776351088 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b0011d00 con 0x7f64c406a770 2023-10-11T09:50:15.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.414+0000 7f64cace7700 1 -- 172.21.15.78:0/1776351088 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64c406a770 msgr2=0x7f64c41608b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:15.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.414+0000 7f64cace7700 1 --2- 172.21.15.78:0/1776351088 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64c406a770 0x7f64c41608b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f64b0009880 tx=0x7f64b0009b90 comp rx=0 tx=0).stop 2023-10-11T09:50:15.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.414+0000 7f64cace7700 1 -- 172.21.15.78:0/1776351088 shutdown_connections 2023-10-11T09:50:15.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.414+0000 7f64cace7700 1 --2- 172.21.15.78:0/1776351088 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64c406a770 0x7f64c41608b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:15.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.414+0000 7f64cace7700 1 -- 172.21.15.78:0/1776351088 >> 172.21.15.78:0/1776351088 conn(0x7f64c40cc820 msgr2=0x7f64c40ccc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:15.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.414+0000 7f64cace7700 1 -- 172.21.15.78:0/1776351088 shutdown_connections 2023-10-11T09:50:15.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.414+0000 7f64cace7700 1 -- 172.21.15.78:0/1776351088 wait complete. 2023-10-11T09:50:15.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.418+0000 7f64cace7700 1 Processor -- start 2023-10-11T09:50:15.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.418+0000 7f64cace7700 1 -- start start 2023-10-11T09:50:15.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.418+0000 7f64cace7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64c406a770 0x7f64c414a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:15.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.418+0000 7f64cace7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64c40af010 0x7f64c40c4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:15.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.418+0000 7f64cace7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64c40af3e0 0x7f64c40c4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:15.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.418+0000 7f64cace7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c40c54f0 con 0x7f64c40af010 2023-10-11T09:50:15.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.418+0000 7f64cace7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c4150610 con 0x7f64c40af3e0 2023-10-11T09:50:15.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.418+0000 7f64cace7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64c40c7010 con 0x7f64c406a770 2023-10-11T09:50:15.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.418+0000 7f64c3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64c40af010 0x7f64c40c4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:15.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.418+0000 7f64c3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64c40af010 0x7f64c40c4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42754/0 (socket says 172.21.15.78:42754) 2023-10-11T09:50:15.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.418+0000 7f64c3fff700 1 -- 172.21.15.78:0/3583134831 learned_addr learned my addr 172.21.15.78:0/3583134831 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:15.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.418+0000 7f64c9284700 1 --2- 172.21.15.78:0/3583134831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64c40af3e0 0x7f64c40c4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:15.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.418+0000 7f64c8a83700 1 --2- 172.21.15.78:0/3583134831 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64c406a770 0x7f64c414a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:15.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.418+0000 7f64c3fff700 1 -- 172.21.15.78:0/3583134831 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64c406a770 msgr2=0x7f64c414a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:15.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.418+0000 7f64c3fff700 1 --2- 172.21.15.78:0/3583134831 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64c406a770 0x7f64c414a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:15.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.418+0000 7f64c3fff700 1 -- 172.21.15.78:0/3583134831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64c40af3e0 msgr2=0x7f64c40c4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:15.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.418+0000 7f64c3fff700 1 --2- 172.21.15.78:0/3583134831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64c40af3e0 0x7f64c40c4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:15.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.418+0000 7f64c3fff700 1 -- 172.21.15.78:0/3583134831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b00094b0 con 0x7f64c40af010 2023-10-11T09:50:15.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.422+0000 7f64c3fff700 1 --2- 172.21.15.78:0/3583134831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64c40af010 0x7f64c40c4920 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f64b00090b0 tx=0x7f64b0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:15.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.422+0000 7f64c17fa700 1 -- 172.21.15.78:0/3583134831 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b0004610 con 0x7f64c40af010 2023-10-11T09:50:15.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.422+0000 7f64c17fa700 1 -- 172.21.15.78:0/3583134831 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b00036c0 con 0x7f64c40af010 2023-10-11T09:50:15.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.422+0000 7f64cace7700 1 -- 172.21.15.78:0/3583134831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64c40c72f0 con 0x7f64c40af010 2023-10-11T09:50:15.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.422+0000 7f64c17fa700 1 -- 172.21.15.78:0/3583134831 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b0003a70 con 0x7f64c40af010 2023-10-11T09:50:15.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.422+0000 7f64cace7700 1 -- 172.21.15.78:0/3583134831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64c414a760 con 0x7f64c40af010 2023-10-11T09:50:15.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.422+0000 7f64cace7700 1 -- 172.21.15.78:0/3583134831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64c40c5660 con 0x7f64c40af010 2023-10-11T09:50:15.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.426+0000 7f64c17fa700 1 -- 172.21.15.78:0/3583134831 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f64b0003bd0 con 0x7f64c40af010 2023-10-11T09:50:15.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.426+0000 7f64c17fa700 1 --2- 172.21.15.78:0/3583134831 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f64a8065ed0 0x7f64a8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:15.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.426+0000 7f64c17fa700 1 -- 172.21.15.78:0/3583134831 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f64b0084c40 con 0x7f64c40af010 2023-10-11T09:50:15.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.426+0000 7f64c8a83700 1 --2- 172.21.15.78:0/3583134831 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f64a8065ed0 0x7f64a8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:15.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.430+0000 7f64c8a83700 1 --2- 172.21.15.78:0/3583134831 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f64a8065ed0 0x7f64a8068380 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f64b8001560 tx=0x7f64b8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:15.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.434+0000 7f64c17fa700 1 -- 172.21.15.78:0/3583134831 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f64c40c5660 con 0x7f64c40af010 2023-10-11T09:50:15.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.606+0000 7f64cace7700 1 -- 172.21.15.78:0/3583134831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64c40c5660 con 0x7f64c40af010 2023-10-11T09:50:15.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.610+0000 7f64c17fa700 1 -- 172.21.15.78:0/3583134831 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+19623 (secure 0 0 0) 0x7f64c40c5660 con 0x7f64c40af010 2023-10-11T09:50:15.618 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:15.618 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":108,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:50:01.881632+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:49:49.297597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-11T09:49:49.955104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data_extra","create_time":"2023-10-11T09:49:59.915561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:50:15.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.614+0000 7f64cace7700 1 -- 172.21.15.78:0/3583134831 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f64a8065ed0 msgr2=0x7f64a8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:15.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.614+0000 7f64cace7700 1 --2- 172.21.15.78:0/3583134831 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f64a8065ed0 0x7f64a8068380 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f64b8001560 tx=0x7f64b8009040 comp rx=0 tx=0).stop 2023-10-11T09:50:15.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.614+0000 7f64cace7700 1 -- 172.21.15.78:0/3583134831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64c40af010 msgr2=0x7f64c40c4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:15.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.614+0000 7f64cace7700 1 --2- 172.21.15.78:0/3583134831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64c40af010 0x7f64c40c4920 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f64b00090b0 tx=0x7f64b0004770 comp rx=0 tx=0).stop 2023-10-11T09:50:15.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.614+0000 7f64cace7700 1 -- 172.21.15.78:0/3583134831 shutdown_connections 2023-10-11T09:50:15.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.614+0000 7f64cace7700 1 --2- 172.21.15.78:0/3583134831 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f64a8065ed0 0x7f64a8068380 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:15.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.614+0000 7f64cace7700 1 --2- 172.21.15.78:0/3583134831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64c40af3e0 0x7f64c40c4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:15.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.614+0000 7f64cace7700 1 --2- 172.21.15.78:0/3583134831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64c40af010 0x7f64c40c4920 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:15.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.614+0000 7f64cace7700 1 --2- 172.21.15.78:0/3583134831 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64c406a770 0x7f64c414a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:15.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.614+0000 7f64cace7700 1 -- 172.21.15.78:0/3583134831 >> 172.21.15.78:0/3583134831 conn(0x7f64c40cc820 msgr2=0x7f64c40ca3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:15.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.614+0000 7f64cace7700 1 -- 172.21.15.78:0/3583134831 shutdown_connections 2023-10-11T09:50:15.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.614+0000 7f64cace7700 1 -- 172.21.15.78:0/3583134831 wait complete. 2023-10-11T09:50:15.645 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T09:49:49.297597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T09:49:49.955104+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-10-11T09:49:59.915561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:50:15.646 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:50:15.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.810+0000 7f1604d70700 1 Processor -- start 2023-10-11T09:50:15.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.810+0000 7f1604d70700 1 -- start start 2023-10-11T09:50:15.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.814+0000 7f1604d70700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f16001533a0 0x7f1600159200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:15.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.814+0000 7f1604d70700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f160015a1b0 con 0x7f160015d270 2023-10-11T09:50:15.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.814+0000 7f1604d70700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f160006bd10 con 0x7f16000b2120 2023-10-11T09:50:15.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.814+0000 7f1604d70700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1600153770 con 0x7f16001533a0 2023-10-11T09:50:15.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.814+0000 7f15ff7fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f16001533a0 0x7f1600159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:15.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.814+0000 7f15feffd700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f160015d270 0x7f16000b1a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:57408/0 (socket says 172.21.15.78:57408) 2023-10-11T09:50:15.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.814+0000 7f15feffd700 1 -- 172.21.15.78:0/1425894820 learned_addr learned my addr 172.21.15.78:0/1425894820 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:50:15.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.814+0000 7f15ff7fe700 1 -- 172.21.15.78:0/1425894820 >> v1:172.21.15.133:6789/0 conn(0x7f16000b2120 legacy=0x7f160006c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:15.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.814+0000 7f15ff7fe700 1 -- 172.21.15.78:0/1425894820 >> v1:172.21.15.78:6789/0 conn(0x7f160015d270 legacy=0x7f16000b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:15.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.814+0000 7f15ff7fe700 1 -- 172.21.15.78:0/1425894820 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f160015d6f0 con 0x7f16001533a0 2023-10-11T09:50:15.824 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.814+0000 7f15ff7fe700 1 --2- 172.21.15.78:0/1425894820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16001533a0 0x7f1600159200 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f15f0009850 tx=0x7f15f0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c92cebe023badf1 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:15.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.814+0000 7f15fe7fc700 1 -- 172.21.15.78:0/1425894820 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f0011830 con 0x7f16001533a0 2023-10-11T09:50:15.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.814+0000 7f15fe7fc700 1 -- 172.21.15.78:0/1425894820 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f0011990 con 0x7f16001533a0 2023-10-11T09:50:15.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.814+0000 7f15fe7fc700 1 -- 172.21.15.78:0/1425894820 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f0011d20 con 0x7f16001533a0 2023-10-11T09:50:15.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.818+0000 7f1604d70700 1 -- 172.21.15.78:0/1425894820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16001533a0 msgr2=0x7f1600159200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:15.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.818+0000 7f1604d70700 1 --2- 172.21.15.78:0/1425894820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16001533a0 0x7f1600159200 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f15f0009850 tx=0x7f15f0009b60 comp rx=0 tx=0).stop 2023-10-11T09:50:15.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.818+0000 7f1604d70700 1 -- 172.21.15.78:0/1425894820 shutdown_connections 2023-10-11T09:50:15.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.818+0000 7f1604d70700 1 --2- 172.21.15.78:0/1425894820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16001533a0 0x7f1600159200 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:15.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.818+0000 7f1604d70700 1 -- 172.21.15.78:0/1425894820 >> 172.21.15.78:0/1425894820 conn(0x7f160006a060 msgr2=0x7f160006a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:15.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.818+0000 7f1604d70700 1 -- 172.21.15.78:0/1425894820 shutdown_connections 2023-10-11T09:50:15.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.818+0000 7f1604d70700 1 -- 172.21.15.78:0/1425894820 wait complete. 2023-10-11T09:50:15.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.818+0000 7f1604d70700 1 Processor -- start 2023-10-11T09:50:15.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.822+0000 7f1604d70700 1 -- start start 2023-10-11T09:50:15.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.822+0000 7f1604d70700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f16000b2120 0x7f1600136790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:15.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.822+0000 7f1604d70700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16001533a0 0x7f160014bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:15.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.822+0000 7f1604d70700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f160015d270 0x7f1600151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:15.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.822+0000 7f1604d70700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f160014f730 con 0x7f16001533a0 2023-10-11T09:50:15.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.822+0000 7f1604d70700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16001525a0 con 0x7f160015d270 2023-10-11T09:50:15.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.822+0000 7f1604d70700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f160014fc50 con 0x7f16000b2120 2023-10-11T09:50:15.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.822+0000 7f15feffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16001533a0 0x7f160014bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:15.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.822+0000 7f15feffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16001533a0 0x7f160014bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42772/0 (socket says 172.21.15.78:42772) 2023-10-11T09:50:15.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.822+0000 7f15feffd700 1 -- 172.21.15.78:0/1177355937 learned_addr learned my addr 172.21.15.78:0/1177355937 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:15.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.822+0000 7f15fffff700 1 --2- 172.21.15.78:0/1177355937 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f160015d270 0x7f1600151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:15.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.822+0000 7f15ff7fe700 1 --2- 172.21.15.78:0/1177355937 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f16000b2120 0x7f1600136790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:15.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.822+0000 7f15feffd700 1 -- 172.21.15.78:0/1177355937 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f16000b2120 msgr2=0x7f1600136790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:15.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.822+0000 7f15feffd700 1 --2- 172.21.15.78:0/1177355937 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f16000b2120 0x7f1600136790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:15.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.822+0000 7f15feffd700 1 -- 172.21.15.78:0/1177355937 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f160015d270 msgr2=0x7f1600151470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:15.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.822+0000 7f15feffd700 1 --2- 172.21.15.78:0/1177355937 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f160015d270 0x7f1600151470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:15.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.822+0000 7f15feffd700 1 -- 172.21.15.78:0/1177355937 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f00094b0 con 0x7f16001533a0 2023-10-11T09:50:15.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.822+0000 7f15feffd700 1 --2- 172.21.15.78:0/1177355937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16001533a0 0x7f160014bf30 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f15f400e910 tx=0x7f15f400ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:15.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.822+0000 7f15e7fff700 1 -- 172.21.15.78:0/1177355937 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f4005d90 con 0x7f16001533a0 2023-10-11T09:50:15.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.822+0000 7f15e7fff700 1 -- 172.21.15.78:0/1177355937 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f4007070 con 0x7f16001533a0 2023-10-11T09:50:15.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.822+0000 7f1604d70700 1 -- 172.21.15.78:0/1177355937 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f160014f970 con 0x7f16001533a0 2023-10-11T09:50:15.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.826+0000 7f15e7fff700 1 -- 172.21.15.78:0/1177355937 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f40073e0 con 0x7f16001533a0 2023-10-11T09:50:15.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.826+0000 7f1604d70700 1 -- 172.21.15.78:0/1177355937 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f160014f450 con 0x7f16001533a0 2023-10-11T09:50:15.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.826+0000 7f1604d70700 1 -- 172.21.15.78:0/1177355937 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16000b1a10 con 0x7f16001533a0 2023-10-11T09:50:15.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.826+0000 7f15e7fff700 1 -- 172.21.15.78:0/1177355937 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f15f4007540 con 0x7f16001533a0 2023-10-11T09:50:15.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.830+0000 7f15e7fff700 1 --2- 172.21.15.78:0/1177355937 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f15ec065ed0 0x7f15ec068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:15.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.830+0000 7f15e7fff700 1 -- 172.21.15.78:0/1177355937 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f15f4085bb0 con 0x7f16001533a0 2023-10-11T09:50:15.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.830+0000 7f15ff7fe700 1 --2- 172.21.15.78:0/1177355937 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f15ec065ed0 0x7f15ec068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:15.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.830+0000 7f15ff7fe700 1 --2- 172.21.15.78:0/1177355937 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f15ec065ed0 0x7f15ec068380 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f15f0014040 tx=0x7f15f001c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:15.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:15.838+0000 7f15e7fff700 1 -- 172.21.15.78:0/1177355937 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f15f4053050 con 0x7f16001533a0 2023-10-11T09:50:16.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.010+0000 7f1604d70700 1 -- 172.21.15.78:0/1177355937 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1600158ee0 con 0x7f16001533a0 2023-10-11T09:50:16.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.014+0000 7f15e7fff700 1 -- 172.21.15.78:0/1177355937 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v108) v1 ==== 93+0+10 (secure 0 0 0) 0x7f15f401f0a0 con 0x7f16001533a0 2023-10-11T09:50:16.020 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:50:16.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.014+0000 7f1604d70700 1 -- 172.21.15.78:0/1177355937 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f15ec065ed0 msgr2=0x7f15ec068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:16.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.014+0000 7f1604d70700 1 --2- 172.21.15.78:0/1177355937 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f15ec065ed0 0x7f15ec068380 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f15f0014040 tx=0x7f15f001c040 comp rx=0 tx=0).stop 2023-10-11T09:50:16.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.014+0000 7f1604d70700 1 -- 172.21.15.78:0/1177355937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16001533a0 msgr2=0x7f160014bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:16.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.014+0000 7f1604d70700 1 --2- 172.21.15.78:0/1177355937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16001533a0 0x7f160014bf30 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f15f400e910 tx=0x7f15f400ec20 comp rx=0 tx=0).stop 2023-10-11T09:50:16.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.014+0000 7f1604d70700 1 -- 172.21.15.78:0/1177355937 shutdown_connections 2023-10-11T09:50:16.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.014+0000 7f1604d70700 1 --2- 172.21.15.78:0/1177355937 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f15ec065ed0 0x7f15ec068380 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:16.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.014+0000 7f1604d70700 1 --2- 172.21.15.78:0/1177355937 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f160015d270 0x7f1600151470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:16.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.014+0000 7f1604d70700 1 --2- 172.21.15.78:0/1177355937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16001533a0 0x7f160014bf30 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:16.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.014+0000 7f1604d70700 1 --2- 172.21.15.78:0/1177355937 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f16000b2120 0x7f1600136790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:16.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.014+0000 7f1604d70700 1 -- 172.21.15.78:0/1177355937 >> 172.21.15.78:0/1177355937 conn(0x7f160006a060 msgr2=0x7f1600152940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:16.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.014+0000 7f1604d70700 1 -- 172.21.15.78:0/1177355937 shutdown_connections 2023-10-11T09:50:16.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.018+0000 7f1604d70700 1 -- 172.21.15.78:0/1177355937 wait complete. 2023-10-11T09:50:16.046 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:50:16.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.206+0000 7f9d3d0e4700 1 Processor -- start 2023-10-11T09:50:16.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.210+0000 7f9d3d0e4700 1 -- start start 2023-10-11T09:50:16.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.210+0000 7f9d3d0e4700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9d38146fe0 0x7f9d3806bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:16.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.210+0000 7f9d3d0e4700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f9d380b36f0 0x7f9d381629e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:16.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.210+0000 7f9d3d0e4700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d3806ccb0 con 0x7f9d38161d90 2023-10-11T09:50:16.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.210+0000 7f9d3d0e4700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d3806c410 con 0x7f9d380b36f0 2023-10-11T09:50:16.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.210+0000 7f9d3d0e4700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3806c6f0 con 0x7f9d38146fe0 2023-10-11T09:50:16.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.214+0000 7f9d2f7fe700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f9d38161d90 0x7f9d380b2fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:57418/0 (socket says 172.21.15.78:57418) 2023-10-11T09:50:16.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.214+0000 7f9d2f7fe700 1 -- 172.21.15.78:0/4203788637 learned_addr learned my addr 172.21.15.78:0/4203788637 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:50:16.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.214+0000 7f9d3c8e3700 1 --2- 172.21.15.78:0/4203788637 >> v2:172.21.15.78:3300/0 conn(0x7f9d380b36f0 0x7f9d381629e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:16.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.214+0000 7f9d37fff700 1 --2- 172.21.15.78:0/4203788637 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9d38146fe0 0x7f9d3806bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:16.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.214+0000 7f9d3c8e3700 1 -- 172.21.15.78:0/4203788637 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9d38146fe0 msgr2=0x7f9d3806bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:16.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.214+0000 7f9d3c8e3700 1 --2- 172.21.15.78:0/4203788637 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9d38146fe0 0x7f9d3806bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:16.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.214+0000 7f9d377fe700 1 -- 172.21.15.78:0/4203788637 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3149482450 0 0) 0x7f9d20001110 con 0x7f9d38161d90 2023-10-11T09:50:16.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.214+0000 7f9d3c8e3700 1 -- 172.21.15.78:0/4203788637 >> v1:172.21.15.78:6789/0 conn(0x7f9d38161d90 legacy=0x7f9d380b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:16.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.214+0000 7f9d3c8e3700 1 -- 172.21.15.78:0/4203788637 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d3806c9d0 con 0x7f9d380b36f0 2023-10-11T09:50:16.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.214+0000 7f9d3c8e3700 1 --2- 172.21.15.78:0/4203788637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d380b36f0 0x7f9d381629e0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f9d30009850 tx=0x7f9d30009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d14aaaa1b500b46 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:16.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.214+0000 7f9d377fe700 1 -- 172.21.15.78:0/4203788637 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d30011830 con 0x7f9d380b36f0 2023-10-11T09:50:16.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.214+0000 7f9d377fe700 1 -- 172.21.15.78:0/4203788637 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d30011990 con 0x7f9d380b36f0 2023-10-11T09:50:16.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.214+0000 7f9d377fe700 1 -- 172.21.15.78:0/4203788637 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d30011d40 con 0x7f9d380b36f0 2023-10-11T09:50:16.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.214+0000 7f9d3d0e4700 1 -- 172.21.15.78:0/4203788637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d380b36f0 msgr2=0x7f9d381629e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:16.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.214+0000 7f9d3d0e4700 1 --2- 172.21.15.78:0/4203788637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d380b36f0 0x7f9d381629e0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f9d30009850 tx=0x7f9d30009b60 comp rx=0 tx=0).stop 2023-10-11T09:50:16.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.214+0000 7f9d3d0e4700 1 -- 172.21.15.78:0/4203788637 shutdown_connections 2023-10-11T09:50:16.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.214+0000 7f9d3d0e4700 1 --2- 172.21.15.78:0/4203788637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d380b36f0 0x7f9d381629e0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:16.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.214+0000 7f9d3d0e4700 1 --2- 172.21.15.78:0/4203788637 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9d38146fe0 0x7f9d3806bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:16.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.214+0000 7f9d3d0e4700 1 -- 172.21.15.78:0/4203788637 >> 172.21.15.78:0/4203788637 conn(0x7f9d3806a060 msgr2=0x7f9d3806a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:16.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.214+0000 7f9d3d0e4700 1 -- 172.21.15.78:0/4203788637 shutdown_connections 2023-10-11T09:50:16.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.214+0000 7f9d3d0e4700 1 -- 172.21.15.78:0/4203788637 wait complete. 2023-10-11T09:50:16.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.218+0000 7f9d3d0e4700 1 Processor -- start 2023-10-11T09:50:16.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.218+0000 7f9d3d0e4700 1 -- start start 2023-10-11T09:50:16.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.218+0000 7f9d3d0e4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d380b36f0 0x7f9d3815d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:16.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.218+0000 7f9d3d0e4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d38146fe0 0x7f9d38145f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:16.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.218+0000 7f9d3d0e4700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9d38161d90 0x7f9d38158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:16.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.218+0000 7f9d3d0e4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d38146460 con 0x7f9d380b36f0 2023-10-11T09:50:16.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.218+0000 7f9d3d0e4700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d38159750 con 0x7f9d38146fe0 2023-10-11T09:50:16.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.218+0000 7f9d3d0e4700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d38159a30 con 0x7f9d38161d90 2023-10-11T09:50:16.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.218+0000 7f9d37fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d380b36f0 0x7f9d3815d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:16.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.218+0000 7f9d37fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d380b36f0 0x7f9d3815d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42784/0 (socket says 172.21.15.78:42784) 2023-10-11T09:50:16.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.218+0000 7f9d37fff700 1 -- 172.21.15.78:0/881583754 learned_addr learned my addr 172.21.15.78:0/881583754 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:16.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.218+0000 7f9d3c8e3700 1 --2- 172.21.15.78:0/881583754 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9d38161d90 0x7f9d38158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:16.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.218+0000 7f9d2f7fe700 1 --2- 172.21.15.78:0/881583754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d38146fe0 0x7f9d38145f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:16.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.218+0000 7f9d37fff700 1 -- 172.21.15.78:0/881583754 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9d38161d90 msgr2=0x7f9d38158eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:16.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.218+0000 7f9d37fff700 1 --2- 172.21.15.78:0/881583754 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9d38161d90 0x7f9d38158eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:16.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.218+0000 7f9d37fff700 1 -- 172.21.15.78:0/881583754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d38146fe0 msgr2=0x7f9d38145f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:16.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.218+0000 7f9d37fff700 1 --2- 172.21.15.78:0/881583754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d38146fe0 0x7f9d38145f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:16.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.218+0000 7f9d37fff700 1 -- 172.21.15.78:0/881583754 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d300094b0 con 0x7f9d380b36f0 2023-10-11T09:50:16.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.218+0000 7f9d3c8e3700 1 --2- 172.21.15.78:0/881583754 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9d38161d90 0x7f9d38158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:50:16.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.218+0000 7f9d2f7fe700 1 --2- 172.21.15.78:0/881583754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d38146fe0 0x7f9d38145f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:50:16.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.222+0000 7f9d37fff700 1 --2- 172.21.15.78:0/881583754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d380b36f0 0x7f9d3815d8a0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f9d2800efc0 tx=0x7f9d2800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:16.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.222+0000 7f9d357fa700 1 -- 172.21.15.78:0/881583754 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2800cdc0 con 0x7f9d380b36f0 2023-10-11T09:50:16.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.222+0000 7f9d357fa700 1 -- 172.21.15.78:0/881583754 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d2800f040 con 0x7f9d380b36f0 2023-10-11T09:50:16.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.222+0000 7f9d3d0e4700 1 -- 172.21.15.78:0/881583754 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d38159d10 con 0x7f9d380b36f0 2023-10-11T09:50:16.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.222+0000 7f9d357fa700 1 -- 172.21.15.78:0/881583754 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d280146a0 con 0x7f9d380b36f0 2023-10-11T09:50:16.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.222+0000 7f9d3d0e4700 1 -- 172.21.15.78:0/881583754 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d381593f0 con 0x7f9d380b36f0 2023-10-11T09:50:16.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.222+0000 7f9d3d0e4700 1 -- 172.21.15.78:0/881583754 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d380cc440 con 0x7f9d380b36f0 2023-10-11T09:50:16.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.222+0000 7f9d357fa700 1 -- 172.21.15.78:0/881583754 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f9d28014800 con 0x7f9d380b36f0 2023-10-11T09:50:16.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.226+0000 7f9d357fa700 1 --2- 172.21.15.78:0/881583754 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9d18065ed0 0x7f9d18068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:16.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.226+0000 7f9d357fa700 1 -- 172.21.15.78:0/881583754 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f9d280845d0 con 0x7f9d380b36f0 2023-10-11T09:50:16.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.226+0000 7f9d2f7fe700 1 --2- 172.21.15.78:0/881583754 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9d18065ed0 0x7f9d18068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:16.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.226+0000 7f9d2f7fe700 1 --2- 172.21.15.78:0/881583754 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9d18065ed0 0x7f9d18068380 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f9d20007fd0 tx=0x7f9d20009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:16.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.230+0000 7f9d357fa700 1 -- 172.21.15.78:0/881583754 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f9d380cc440 con 0x7f9d380b36f0 2023-10-11T09:50:16.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.406+0000 7f9d3d0e4700 1 -- 172.21.15.78:0/881583754 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9d380cc440 con 0x7f9d380b36f0 2023-10-11T09:50:16.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.406+0000 7f9d357fa700 1 -- 172.21.15.78:0/881583754 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v108) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9d380cc440 con 0x7f9d380b36f0 2023-10-11T09:50:16.414 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:50:16.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.410+0000 7f9d3d0e4700 1 -- 172.21.15.78:0/881583754 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9d18065ed0 msgr2=0x7f9d18068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:16.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.410+0000 7f9d3d0e4700 1 --2- 172.21.15.78:0/881583754 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9d18065ed0 0x7f9d18068380 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f9d20007fd0 tx=0x7f9d20009040 comp rx=0 tx=0).stop 2023-10-11T09:50:16.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.410+0000 7f9d3d0e4700 1 -- 172.21.15.78:0/881583754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d380b36f0 msgr2=0x7f9d3815d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:16.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.410+0000 7f9d3d0e4700 1 --2- 172.21.15.78:0/881583754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d380b36f0 0x7f9d3815d8a0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f9d2800efc0 tx=0x7f9d2800c3a0 comp rx=0 tx=0).stop 2023-10-11T09:50:16.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.410+0000 7f9d3d0e4700 1 -- 172.21.15.78:0/881583754 shutdown_connections 2023-10-11T09:50:16.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.410+0000 7f9d3d0e4700 1 --2- 172.21.15.78:0/881583754 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9d18065ed0 0x7f9d18068380 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:16.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.410+0000 7f9d3d0e4700 1 --2- 172.21.15.78:0/881583754 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9d38161d90 0x7f9d38158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:16.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.410+0000 7f9d3d0e4700 1 --2- 172.21.15.78:0/881583754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d38146fe0 0x7f9d38145f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:16.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.410+0000 7f9d3d0e4700 1 --2- 172.21.15.78:0/881583754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d380b36f0 0x7f9d3815d8a0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:16.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.410+0000 7f9d3d0e4700 1 -- 172.21.15.78:0/881583754 >> 172.21.15.78:0/881583754 conn(0x7f9d3806a060 msgr2=0x7f9d3806a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:16.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.410+0000 7f9d3d0e4700 1 -- 172.21.15.78:0/881583754 shutdown_connections 2023-10-11T09:50:16.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.410+0000 7f9d3d0e4700 1 -- 172.21.15.78:0/881583754 wait complete. 2023-10-11T09:50:16.436 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T09:50:16.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.598+0000 7f3cdc9b7700 1 Processor -- start 2023-10-11T09:50:16.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.598+0000 7f3cdc9b7700 1 -- start start 2023-10-11T09:50:16.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.598+0000 7f3cdc9b7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3cd8146fe0 0x7f3cd806bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:16.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.598+0000 7f3cdc9b7700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f3cd80b36f0 0x7f3cd81629e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:16.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.598+0000 7f3cdc9b7700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cd806ccb0 con 0x7f3cd8161d90 2023-10-11T09:50:16.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.598+0000 7f3cdc9b7700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cd806c410 con 0x7f3cd80b36f0 2023-10-11T09:50:16.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.598+0000 7f3cdc9b7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd806c6f0 con 0x7f3cd8146fe0 2023-10-11T09:50:16.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.602+0000 7f3ccffff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f3cd8161d90 0x7f3cd80b2fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:57426/0 (socket says 172.21.15.78:57426) 2023-10-11T09:50:16.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.602+0000 7f3ccffff700 1 -- 172.21.15.78:0/3302834032 learned_addr learned my addr 172.21.15.78:0/3302834032 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:50:16.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.602+0000 7f3cd7fff700 1 --2- 172.21.15.78:0/3302834032 >> v2:172.21.15.78:3300/0 conn(0x7f3cd80b36f0 0x7f3cd81629e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:16.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.602+0000 7f3cd77fe700 1 --2- 172.21.15.78:0/3302834032 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3cd8146fe0 0x7f3cd806bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:16.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.602+0000 7f3cd6ffd700 1 -- 172.21.15.78:0/3302834032 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 406131961 0 0) 0x7f3cc0001110 con 0x7f3cd8161d90 2023-10-11T09:50:16.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.602+0000 7f3cd6ffd700 1 -- 172.21.15.78:0/3302834032 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cc40035d0 con 0x7f3cd8161d90 2023-10-11T09:50:16.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.602+0000 7f3cd7fff700 1 -- 172.21.15.78:0/3302834032 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3cd8146fe0 msgr2=0x7f3cd806bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:16.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.602+0000 7f3cd7fff700 1 --2- 172.21.15.78:0/3302834032 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3cd8146fe0 0x7f3cd806bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:16.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.602+0000 7f3cd7fff700 1 -- 172.21.15.78:0/3302834032 >> v1:172.21.15.78:6789/0 conn(0x7f3cd8161d90 legacy=0x7f3cd80b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:16.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.602+0000 7f3cd7fff700 1 -- 172.21.15.78:0/3302834032 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd806c9d0 con 0x7f3cd80b36f0 2023-10-11T09:50:16.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.602+0000 7f3cd7fff700 1 --2- 172.21.15.78:0/3302834032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3cd80b36f0 0x7f3cd81629e0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0009d90 tx=0x7f3cd000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2daaa6fb86c974f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:16.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.602+0000 7f3cd6ffd700 1 -- 172.21.15.78:0/3302834032 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd000e070 con 0x7f3cd80b36f0 2023-10-11T09:50:16.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.602+0000 7f3cd6ffd700 1 -- 172.21.15.78:0/3302834032 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd000bb20 con 0x7f3cd80b36f0 2023-10-11T09:50:16.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.602+0000 7f3cd6ffd700 1 -- 172.21.15.78:0/3302834032 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd0012430 con 0x7f3cd80b36f0 2023-10-11T09:50:16.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.602+0000 7f3cdc9b7700 1 -- 172.21.15.78:0/3302834032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3cd80b36f0 msgr2=0x7f3cd81629e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:16.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.602+0000 7f3cdc9b7700 1 --2- 172.21.15.78:0/3302834032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3cd80b36f0 0x7f3cd81629e0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0009d90 tx=0x7f3cd000b330 comp rx=0 tx=0).stop 2023-10-11T09:50:16.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.602+0000 7f3cdc9b7700 1 -- 172.21.15.78:0/3302834032 shutdown_connections 2023-10-11T09:50:16.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.602+0000 7f3cdc9b7700 1 --2- 172.21.15.78:0/3302834032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3cd80b36f0 0x7f3cd81629e0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:16.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.602+0000 7f3cdc9b7700 1 --2- 172.21.15.78:0/3302834032 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3cd8146fe0 0x7f3cd806bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:16.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.602+0000 7f3cdc9b7700 1 -- 172.21.15.78:0/3302834032 >> 172.21.15.78:0/3302834032 conn(0x7f3cd806a060 msgr2=0x7f3cd806a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:16.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.602+0000 7f3cdc9b7700 1 -- 172.21.15.78:0/3302834032 shutdown_connections 2023-10-11T09:50:16.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.602+0000 7f3cdc9b7700 1 -- 172.21.15.78:0/3302834032 wait complete. 2023-10-11T09:50:16.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.606+0000 7f3cdc9b7700 1 Processor -- start 2023-10-11T09:50:16.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.606+0000 7f3cdc9b7700 1 -- start start 2023-10-11T09:50:16.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.606+0000 7f3cdc9b7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3cd80b36f0 0x7f3cd81f59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:16.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.606+0000 7f3cdc9b7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3cd8146fe0 0x7f3cd8159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:16.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.606+0000 7f3cdc9b7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cd8161d90 0x7f3cd815a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:16.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.606+0000 7f3cdc9b7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd8157e70 con 0x7f3cd80b36f0 2023-10-11T09:50:16.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.606+0000 7f3cdc9b7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd8157540 con 0x7f3cd8161d90 2023-10-11T09:50:16.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.606+0000 7f3cdc9b7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd81577b0 con 0x7f3cd8146fe0 2023-10-11T09:50:16.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.606+0000 7f3cd77fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3cd80b36f0 0x7f3cd81f59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:16.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.606+0000 7f3cd77fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3cd80b36f0 0x7f3cd81f59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42802/0 (socket says 172.21.15.78:42802) 2023-10-11T09:50:16.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.606+0000 7f3cd77fe700 1 -- 172.21.15.78:0/3008183910 learned_addr learned my addr 172.21.15.78:0/3008183910 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:16.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.606+0000 7f3ccffff700 1 --2- 172.21.15.78:0/3008183910 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3cd8146fe0 0x7f3cd8159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:16.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.606+0000 7f3cd7fff700 1 --2- 172.21.15.78:0/3008183910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cd8161d90 0x7f3cd815a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:16.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.606+0000 7f3cd77fe700 1 -- 172.21.15.78:0/3008183910 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3cd8146fe0 msgr2=0x7f3cd8159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:16.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.606+0000 7f3cd77fe700 1 --2- 172.21.15.78:0/3008183910 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3cd8146fe0 0x7f3cd8159cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:16.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.606+0000 7f3cd77fe700 1 -- 172.21.15.78:0/3008183910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cd8161d90 msgr2=0x7f3cd815a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:16.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.606+0000 7f3cd77fe700 1 --2- 172.21.15.78:0/3008183910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cd8161d90 0x7f3cd815a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:16.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.610+0000 7f3cd77fe700 1 -- 172.21.15.78:0/3008183910 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd0009a70 con 0x7f3cd80b36f0 2023-10-11T09:50:16.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.610+0000 7f3cd77fe700 1 --2- 172.21.15.78:0/3008183910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3cd80b36f0 0x7f3cd81f59c0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f3cc800ea00 tx=0x7f3cc800ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:16.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.610+0000 7f3cd4ff9700 1 -- 172.21.15.78:0/3008183910 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8009720 con 0x7f3cd80b36f0 2023-10-11T09:50:16.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.610+0000 7f3cd4ff9700 1 -- 172.21.15.78:0/3008183910 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc800eea0 con 0x7f3cd80b36f0 2023-10-11T09:50:16.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.610+0000 7f3cd4ff9700 1 -- 172.21.15.78:0/3008183910 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc800ca50 con 0x7f3cd80b36f0 2023-10-11T09:50:16.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.610+0000 7f3cdc9b7700 1 -- 172.21.15.78:0/3008183910 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cd8157a90 con 0x7f3cd80b36f0 2023-10-11T09:50:16.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.610+0000 7f3cdc9b7700 1 -- 172.21.15.78:0/3008183910 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cd81f5f00 con 0x7f3cd80b36f0 2023-10-11T09:50:16.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.610+0000 7f3cdc9b7700 1 -- 172.21.15.78:0/3008183910 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cd81580e0 con 0x7f3cd80b36f0 2023-10-11T09:50:16.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.614+0000 7f3cd4ff9700 1 -- 172.21.15.78:0/3008183910 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f3cc800cbb0 con 0x7f3cd80b36f0 2023-10-11T09:50:16.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.614+0000 7f3cd4ff9700 1 --2- 172.21.15.78:0/3008183910 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3cb8065ed0 0x7f3cb8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:16.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.614+0000 7f3cd4ff9700 1 -- 172.21.15.78:0/3008183910 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f3cc8083920 con 0x7f3cd80b36f0 2023-10-11T09:50:16.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.614+0000 7f3ccffff700 1 --2- 172.21.15.78:0/3008183910 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3cb8065ed0 0x7f3cb8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:16.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.614+0000 7f3ccffff700 1 --2- 172.21.15.78:0/3008183910 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3cb8065ed0 0x7f3cb8068380 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0007fd0 tx=0x7f3cc0007c60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:16.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.618+0000 7f3cd4ff9700 1 -- 172.21.15.78:0/3008183910 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f3cc8050dc0 con 0x7f3cd80b36f0 2023-10-11T09:50:16.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.790+0000 7f3cdc9b7700 1 -- 172.21.15.78:0/3008183910 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3cd8065d40 con 0x7f3cd80b36f0 2023-10-11T09:50:16.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.790+0000 7f3cd4ff9700 1 -- 172.21.15.78:0/3008183910 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v108) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3cc80543e0 con 0x7f3cd80b36f0 2023-10-11T09:50:16.799 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:50:16.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.794+0000 7f3cdc9b7700 1 -- 172.21.15.78:0/3008183910 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3cb8065ed0 msgr2=0x7f3cb8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:16.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.794+0000 7f3cdc9b7700 1 --2- 172.21.15.78:0/3008183910 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3cb8065ed0 0x7f3cb8068380 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0007fd0 tx=0x7f3cc0007c60 comp rx=0 tx=0).stop 2023-10-11T09:50:16.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.794+0000 7f3cdc9b7700 1 -- 172.21.15.78:0/3008183910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3cd80b36f0 msgr2=0x7f3cd81f59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:16.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.794+0000 7f3cdc9b7700 1 --2- 172.21.15.78:0/3008183910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3cd80b36f0 0x7f3cd81f59c0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f3cc800ea00 tx=0x7f3cc800ed10 comp rx=0 tx=0).stop 2023-10-11T09:50:16.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.794+0000 7f3cdc9b7700 1 -- 172.21.15.78:0/3008183910 shutdown_connections 2023-10-11T09:50:16.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.794+0000 7f3cdc9b7700 1 --2- 172.21.15.78:0/3008183910 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3cb8065ed0 0x7f3cb8068380 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:16.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.794+0000 7f3cdc9b7700 1 --2- 172.21.15.78:0/3008183910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cd8161d90 0x7f3cd815a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:16.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.794+0000 7f3cdc9b7700 1 --2- 172.21.15.78:0/3008183910 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3cd8146fe0 0x7f3cd8159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:16.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.794+0000 7f3cdc9b7700 1 --2- 172.21.15.78:0/3008183910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3cd80b36f0 0x7f3cd81f59c0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:16.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.794+0000 7f3cdc9b7700 1 -- 172.21.15.78:0/3008183910 >> 172.21.15.78:0/3008183910 conn(0x7f3cd806a060 msgr2=0x7f3cd806a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:16.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.794+0000 7f3cdc9b7700 1 -- 172.21.15.78:0/3008183910 shutdown_connections 2023-10-11T09:50:16.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.794+0000 7f3cdc9b7700 1 -- 172.21.15.78:0/3008183910 wait complete. 2023-10-11T09:50:16.823 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T09:50:17.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.998+0000 7f200db01700 1 Processor -- start 2023-10-11T09:50:17.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:16.998+0000 7f200db01700 1 -- start start 2023-10-11T09:50:17.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.002+0000 7f200db01700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f20080b8890 0x7f20080d2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:17.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.002+0000 7f200db01700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20080d3510 0x7f20080d3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:17.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.002+0000 7f200db01700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20080d3e80 con 0x7f20080b84c0 2023-10-11T09:50:17.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.002+0000 7f200db01700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20080cdea0 con 0x7f20080b8890 2023-10-11T09:50:17.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.002+0000 7f200db01700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20080d0c10 con 0x7f20080d3510 2023-10-11T09:50:17.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.002+0000 7f20077fe700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f20080b84c0 0x7f20080cfda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:57442/0 (socket says 172.21.15.78:57442) 2023-10-11T09:50:17.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.002+0000 7f20077fe700 1 -- 172.21.15.78:0/1361660053 learned_addr learned my addr 172.21.15.78:0/1361660053 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:50:17.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.002+0000 7f2006ffd700 1 --2- 172.21.15.78:0/1361660053 >> v2:172.21.15.78:3300/0 conn(0x7f20080b8890 0x7f20080d2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:17.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.002+0000 7f2007fff700 1 --2- 172.21.15.78:0/1361660053 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20080d3510 0x7f20080d3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:17.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.002+0000 7f20067fc700 1 -- 172.21.15.78:0/1361660053 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3709706920 0 0) 0x7f20080d3e80 con 0x7f20080b84c0 2023-10-11T09:50:17.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.002+0000 7f20067fc700 1 -- 172.21.15.78:0/1361660053 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ff00035d0 con 0x7f20080b84c0 2023-10-11T09:50:17.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.002+0000 7f2006ffd700 1 -- 172.21.15.78:0/1361660053 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20080d3510 msgr2=0x7f20080d3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:17.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.002+0000 7f2006ffd700 1 --2- 172.21.15.78:0/1361660053 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20080d3510 0x7f20080d3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:17.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.002+0000 7f2006ffd700 1 -- 172.21.15.78:0/1361660053 >> v1:172.21.15.78:6789/0 conn(0x7f20080b84c0 legacy=0x7f20080cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:17.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.002+0000 7f2006ffd700 1 -- 172.21.15.78:0/1361660053 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20080cd690 con 0x7f20080b8890 2023-10-11T09:50:17.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.002+0000 7f2006ffd700 1 --2- 172.21.15.78:0/1361660053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20080b8890 0x7f20080d2fd0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc009850 tx=0x7f1ffc009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75935e62e5e7b277 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:17.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.002+0000 7f20067fc700 1 -- 172.21.15.78:0/1361660053 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc011830 con 0x7f20080b8890 2023-10-11T09:50:17.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.002+0000 7f20067fc700 1 -- 172.21.15.78:0/1361660053 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ffc011990 con 0x7f20080b8890 2023-10-11T09:50:17.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.002+0000 7f20067fc700 1 -- 172.21.15.78:0/1361660053 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc011d40 con 0x7f20080b8890 2023-10-11T09:50:17.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.002+0000 7f200db01700 1 -- 172.21.15.78:0/1361660053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20080b8890 msgr2=0x7f20080d2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:17.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.002+0000 7f200db01700 1 --2- 172.21.15.78:0/1361660053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20080b8890 0x7f20080d2fd0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc009850 tx=0x7f1ffc009b60 comp rx=0 tx=0).stop 2023-10-11T09:50:17.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.006+0000 7f200db01700 1 -- 172.21.15.78:0/1361660053 shutdown_connections 2023-10-11T09:50:17.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.006+0000 7f200db01700 1 --2- 172.21.15.78:0/1361660053 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20080d3510 0x7f20080d3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:17.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.006+0000 7f200db01700 1 --2- 172.21.15.78:0/1361660053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20080b8890 0x7f20080d2fd0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:17.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.006+0000 7f200db01700 1 -- 172.21.15.78:0/1361660053 >> 172.21.15.78:0/1361660053 conn(0x7f200806cb50 msgr2=0x7f200806cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:17.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.006+0000 7f200db01700 1 -- 172.21.15.78:0/1361660053 shutdown_connections 2023-10-11T09:50:17.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.006+0000 7f200db01700 1 -- 172.21.15.78:0/1361660053 wait complete. 2023-10-11T09:50:17.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.006+0000 7f200db01700 1 Processor -- start 2023-10-11T09:50:17.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.006+0000 7f200db01700 1 -- start start 2023-10-11T09:50:17.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.006+0000 7f200db01700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20080b84c0 0x7f20080cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:17.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.006+0000 7f200db01700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20080b8890 0x7f20080c73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:17.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.006+0000 7f200db01700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20080d3510 0x7f20080ca110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:17.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.006+0000 7f200db01700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2008151e00 con 0x7f20080b8890 2023-10-11T09:50:17.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.006+0000 7f200db01700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20080c8cb0 con 0x7f20080b84c0 2023-10-11T09:50:17.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.006+0000 7f200db01700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20080c7c10 con 0x7f20080d3510 2023-10-11T09:50:17.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.010+0000 7f2006ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20080b8890 0x7f20080c73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:17.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.010+0000 7f2006ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20080b8890 0x7f20080c73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42812/0 (socket says 172.21.15.78:42812) 2023-10-11T09:50:17.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.010+0000 7f2006ffd700 1 -- 172.21.15.78:0/3736212167 learned_addr learned my addr 172.21.15.78:0/3736212167 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:17.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.010+0000 7f20077fe700 1 --2- 172.21.15.78:0/3736212167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20080b84c0 0x7f20080cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:17.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.010+0000 7f2007fff700 1 --2- 172.21.15.78:0/3736212167 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20080d3510 0x7f20080ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:17.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.010+0000 7f2006ffd700 1 -- 172.21.15.78:0/3736212167 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20080d3510 msgr2=0x7f20080ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:17.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.010+0000 7f2006ffd700 1 --2- 172.21.15.78:0/3736212167 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20080d3510 0x7f20080ca110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:17.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.010+0000 7f2006ffd700 1 -- 172.21.15.78:0/3736212167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20080b84c0 msgr2=0x7f20080cc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:17.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.010+0000 7f2006ffd700 1 --2- 172.21.15.78:0/3736212167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20080b84c0 0x7f20080cc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:17.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.010+0000 7f2006ffd700 1 -- 172.21.15.78:0/3736212167 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ffc0094b0 con 0x7f20080b8890 2023-10-11T09:50:17.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.010+0000 7f20077fe700 1 --2- 172.21.15.78:0/3736212167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20080b84c0 0x7f20080cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:50:17.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.010+0000 7f2007fff700 1 --2- 172.21.15.78:0/3736212167 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20080d3510 0x7f20080ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:50:17.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.010+0000 7f2006ffd700 1 --2- 172.21.15.78:0/3736212167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20080b8890 0x7f20080c73c0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f2008067640 tx=0x7f1ffc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:17.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.010+0000 7f1feffff700 1 -- 172.21.15.78:0/3736212167 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc004480 con 0x7f20080b8890 2023-10-11T09:50:17.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.010+0000 7f1feffff700 1 -- 172.21.15.78:0/3736212167 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ffc01f070 con 0x7f20080b8890 2023-10-11T09:50:17.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.010+0000 7f1feffff700 1 -- 172.21.15.78:0/3736212167 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc003900 con 0x7f20080b8890 2023-10-11T09:50:17.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.010+0000 7f200db01700 1 -- 172.21.15.78:0/3736212167 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20080c7ef0 con 0x7f20080b8890 2023-10-11T09:50:17.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.010+0000 7f200db01700 1 -- 172.21.15.78:0/3736212167 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20080c84b0 con 0x7f20080b8890 2023-10-11T09:50:17.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.010+0000 7f200db01700 1 -- 172.21.15.78:0/3736212167 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f200808bff0 con 0x7f20080b8890 2023-10-11T09:50:17.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.014+0000 7f1feffff700 1 -- 172.21.15.78:0/3736212167 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f1ffc021030 con 0x7f20080b8890 2023-10-11T09:50:17.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.014+0000 7f1feffff700 1 --2- 172.21.15.78:0/3736212167 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1ff4065ed0 0x7f1ff4068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:17.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.014+0000 7f1feffff700 1 -- 172.21.15.78:0/3736212167 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f1ffc084c90 con 0x7f20080b8890 2023-10-11T09:50:17.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.014+0000 7f20077fe700 1 --2- 172.21.15.78:0/3736212167 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1ff4065ed0 0x7f1ff4068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:17.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.014+0000 7f20077fe700 1 --2- 172.21.15.78:0/3736212167 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1ff4065ed0 0x7f1ff4068380 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f1ff80011e0 tx=0x7f1ff8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:17.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.022+0000 7f1feffff700 1 -- 172.21.15.78:0/3736212167 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f1ffc052130 con 0x7f20080b8890 2023-10-11T09:50:17.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.194+0000 7f200db01700 1 -- 172.21.15.78:0/3736212167 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2008065db0 con 0x7f20080b8890 2023-10-11T09:50:17.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.198+0000 7f1feffff700 1 -- 172.21.15.78:0/3736212167 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v108) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1ffc0150a0 con 0x7f20080b8890 2023-10-11T09:50:17.205 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2023-10-11T09:50:17.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.198+0000 7f200db01700 1 -- 172.21.15.78:0/3736212167 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1ff4065ed0 msgr2=0x7f1ff4068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:17.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.198+0000 7f200db01700 1 --2- 172.21.15.78:0/3736212167 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1ff4065ed0 0x7f1ff4068380 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f1ff80011e0 tx=0x7f1ff8009040 comp rx=0 tx=0).stop 2023-10-11T09:50:17.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.198+0000 7f200db01700 1 -- 172.21.15.78:0/3736212167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20080b8890 msgr2=0x7f20080c73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:17.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.198+0000 7f200db01700 1 --2- 172.21.15.78:0/3736212167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20080b8890 0x7f20080c73c0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f2008067640 tx=0x7f1ffc004770 comp rx=0 tx=0).stop 2023-10-11T09:50:17.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.198+0000 7f200db01700 1 -- 172.21.15.78:0/3736212167 shutdown_connections 2023-10-11T09:50:17.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.198+0000 7f200db01700 1 --2- 172.21.15.78:0/3736212167 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1ff4065ed0 0x7f1ff4068380 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:17.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.198+0000 7f200db01700 1 --2- 172.21.15.78:0/3736212167 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20080d3510 0x7f20080ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:17.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.198+0000 7f200db01700 1 --2- 172.21.15.78:0/3736212167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20080b8890 0x7f20080c73c0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:17.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.198+0000 7f200db01700 1 --2- 172.21.15.78:0/3736212167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20080b84c0 0x7f20080cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:17.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.198+0000 7f200db01700 1 -- 172.21.15.78:0/3736212167 >> 172.21.15.78:0/3736212167 conn(0x7f200806cb50 msgr2=0x7f20080b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:17.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.202+0000 7f200db01700 1 -- 172.21.15.78:0/3736212167 shutdown_connections 2023-10-11T09:50:17.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.202+0000 7f200db01700 1 -- 172.21.15.78:0/3736212167 wait complete. 2023-10-11T09:50:17.229 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-10-11T09:50:17.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.398+0000 7fdeb7516700 1 Processor -- start 2023-10-11T09:50:17.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.398+0000 7fdeb7516700 1 -- start start 2023-10-11T09:50:17.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.402+0000 7fdeb7516700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdeb00b3240 0x7fdeb00d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:17.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.402+0000 7fdeb7516700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fdeb00b3610 0x7fdeb00b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:17.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.402+0000 7fdeb7516700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdeb015d040 0x7fdeb015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:17.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.402+0000 7fdeb7516700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeb00b3fd0 con 0x7fdeb00b3610 2023-10-11T09:50:17.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.402+0000 7fdeb7516700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeb00b47f0 con 0x7fdeb015d040 2023-10-11T09:50:17.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.402+0000 7fdeb7516700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb00b8130 con 0x7fdeb00b3240 2023-10-11T09:50:17.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.402+0000 7fdeb5d13700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fdeb00b3610 0x7fdeb00b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:17.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.402+0000 7fdeb5d13700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fdeb00b3610 0x7fdeb00b3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42816/0 (socket says 172.21.15.78:42816) 2023-10-11T09:50:17.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.402+0000 7fdeb5d13700 1 -- 172.21.15.78:0/1896137581 learned_addr learned my addr 172.21.15.78:0/1896137581 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:17.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.402+0000 7fdeb6514700 1 --2- 172.21.15.78:0/1896137581 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdeb00b3240 0x7fdeb00d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:17.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.402+0000 7fdeb6d15700 1 --2- 172.21.15.78:0/1896137581 >> v2:172.21.15.133:3300/0 conn(0x7fdeb015d040 0x7fdeb015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:17.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.402+0000 7fdeb5d13700 1 -- 172.21.15.78:0/1896137581 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdeb00b3240 msgr2=0x7fdeb00d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:17.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.402+0000 7fdeb5d13700 1 --2- 172.21.15.78:0/1896137581 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdeb00b3240 0x7fdeb00d29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:17.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.402+0000 7fdeb5d13700 1 -- 172.21.15.78:0/1896137581 >> v2:172.21.15.133:3300/0 conn(0x7fdeb015d040 msgr2=0x7fdeb015d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:17.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.402+0000 7fdeb5d13700 1 --2- 172.21.15.78:0/1896137581 >> v2:172.21.15.133:3300/0 conn(0x7fdeb015d040 0x7fdeb015d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:17.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.402+0000 7fdeb5d13700 1 -- 172.21.15.78:0/1896137581 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb00b41e0 con 0x7fdeb00b3610 2023-10-11T09:50:17.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.406+0000 7fdeb5d13700 1 --2- 172.21.15.78:0/1896137581 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdeb00b3610 0x7fdeb00b3a90 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fdeb00672a0 tx=0x7fdea0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344d241b97c73b65 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:17.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.406+0000 7fdeb5512700 1 -- 172.21.15.78:0/1896137581 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea000b8a0 con 0x7fdeb00b3610 2023-10-11T09:50:17.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.406+0000 7fdeb5512700 1 -- 172.21.15.78:0/1896137581 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea000ba00 con 0x7fdeb00b3610 2023-10-11T09:50:17.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.406+0000 7fdeb5512700 1 -- 172.21.15.78:0/1896137581 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea000bd90 con 0x7fdeb00b3610 2023-10-11T09:50:17.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.406+0000 7fdeb7516700 1 -- 172.21.15.78:0/1896137581 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdeb00b3610 msgr2=0x7fdeb00b3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:17.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.406+0000 7fdeb7516700 1 --2- 172.21.15.78:0/1896137581 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdeb00b3610 0x7fdeb00b3a90 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fdeb00672a0 tx=0x7fdea0009bf0 comp rx=0 tx=0).stop 2023-10-11T09:50:17.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.406+0000 7fdeb7516700 1 -- 172.21.15.78:0/1896137581 shutdown_connections 2023-10-11T09:50:17.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.406+0000 7fdeb7516700 1 --2- 172.21.15.78:0/1896137581 >> v2:172.21.15.133:3300/0 conn(0x7fdeb015d040 0x7fdeb015d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:17.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.406+0000 7fdeb7516700 1 --2- 172.21.15.78:0/1896137581 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdeb00b3610 0x7fdeb00b3a90 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:17.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.406+0000 7fdeb7516700 1 --2- 172.21.15.78:0/1896137581 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdeb00b3240 0x7fdeb00d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:17.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.406+0000 7fdeb7516700 1 -- 172.21.15.78:0/1896137581 >> 172.21.15.78:0/1896137581 conn(0x7fdeb009c260 msgr2=0x7fdeb00d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:17.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.406+0000 7fdeb7516700 1 -- 172.21.15.78:0/1896137581 shutdown_connections 2023-10-11T09:50:17.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.406+0000 7fdeb7516700 1 -- 172.21.15.78:0/1896137581 wait complete. 2023-10-11T09:50:17.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.410+0000 7fdeb7516700 1 Processor -- start 2023-10-11T09:50:17.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.410+0000 7fdeb7516700 1 -- start start 2023-10-11T09:50:17.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.414+0000 7fdeb7516700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdeb00b3240 0x7fdeb015af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:17.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.414+0000 7fdeb7516700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdeb00b3610 0x7fdeb009f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:17.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.414+0000 7fdeb7516700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdeb015d040 0x7fdeb009fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:17.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.414+0000 7fdeb7516700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb00c8230 con 0x7fdeb015d040 2023-10-11T09:50:17.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.414+0000 7fdeb7516700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb00c8010 con 0x7fdeb00b3240 2023-10-11T09:50:17.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.414+0000 7fdeb7516700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb015b450 con 0x7fdeb00b3610 2023-10-11T09:50:17.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.414+0000 7fdeb6d15700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdeb015d040 0x7fdeb009fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:17.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.414+0000 7fdeb6d15700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdeb015d040 0x7fdeb009fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42820/0 (socket says 172.21.15.78:42820) 2023-10-11T09:50:17.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.414+0000 7fdeb6d15700 1 -- 172.21.15.78:0/2728417322 learned_addr learned my addr 172.21.15.78:0/2728417322 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:17.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.414+0000 7fdeb5d13700 1 --2- 172.21.15.78:0/2728417322 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdeb00b3610 0x7fdeb009f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:17.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.414+0000 7fdeb6514700 1 --2- 172.21.15.78:0/2728417322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdeb00b3240 0x7fdeb015af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:17.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.414+0000 7fdeb6d15700 1 -- 172.21.15.78:0/2728417322 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdeb00b3610 msgr2=0x7fdeb009f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:17.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.414+0000 7fdeb6d15700 1 --2- 172.21.15.78:0/2728417322 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdeb00b3610 0x7fdeb009f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:17.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.414+0000 7fdeb6d15700 1 -- 172.21.15.78:0/2728417322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdeb00b3240 msgr2=0x7fdeb015af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:17.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.414+0000 7fdeb6d15700 1 --2- 172.21.15.78:0/2728417322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdeb00b3240 0x7fdeb015af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:17.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.414+0000 7fdeb6d15700 1 -- 172.21.15.78:0/2728417322 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea0009570 con 0x7fdeb015d040 2023-10-11T09:50:17.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.414+0000 7fdeb6d15700 1 --2- 172.21.15.78:0/2728417322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdeb015d040 0x7fdeb009fa90 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fdea8007e60 tx=0x7fdea800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:17.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.414+0000 7fdea6ffd700 1 -- 172.21.15.78:0/2728417322 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea800ed30 con 0x7fdeb015d040 2023-10-11T09:50:17.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.414+0000 7fdeb7516700 1 -- 172.21.15.78:0/2728417322 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeb00a0110 con 0x7fdeb015d040 2023-10-11T09:50:17.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.414+0000 7fdea6ffd700 1 -- 172.21.15.78:0/2728417322 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea8014420 con 0x7fdeb015d040 2023-10-11T09:50:17.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.414+0000 7fdea6ffd700 1 -- 172.21.15.78:0/2728417322 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea80147d0 con 0x7fdeb015d040 2023-10-11T09:50:17.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.414+0000 7fdeb7516700 1 -- 172.21.15.78:0/2728417322 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeb00a06d0 con 0x7fdeb015d040 2023-10-11T09:50:17.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.418+0000 7fdeb7516700 1 -- 172.21.15.78:0/2728417322 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeb00c83a0 con 0x7fdeb015d040 2023-10-11T09:50:17.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.418+0000 7fdea6ffd700 1 -- 172.21.15.78:0/2728417322 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fdea8014930 con 0x7fdeb015d040 2023-10-11T09:50:17.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.418+0000 7fdea6ffd700 1 --2- 172.21.15.78:0/2728417322 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fde94065ed0 0x7fde94068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:17.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.418+0000 7fdea6ffd700 1 -- 172.21.15.78:0/2728417322 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6466+0+0 (secure 0 0 0) 0x7fdea8083f90 con 0x7fdeb015d040 2023-10-11T09:50:17.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.418+0000 7fdeb6514700 1 --2- 172.21.15.78:0/2728417322 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fde94065ed0 0x7fde94068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:17.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.422+0000 7fdeb6514700 1 --2- 172.21.15.78:0/2728417322 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fde94065ed0 0x7fde94068380 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fdeac0045f0 tx=0x7fdeac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:17.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.426+0000 7fdea6ffd700 1 -- 172.21.15.78:0/2728417322 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fdea80b4fa0 con 0x7fdeb015d040 2023-10-11T09:50:17.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.610+0000 7fdeb7516700 1 -- 172.21.15.78:0/2728417322 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7fdeb0065d40 con 0x7fdeb015d040 2023-10-11T09:50:17.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.610+0000 7fdea6ffd700 1 -- 172.21.15.78:0/2728417322 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v108) v1 ==== 106+0+11 (secure 0 0 0) 0x7fdea8051450 con 0x7fdeb015d040 2023-10-11T09:50:17.620 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:50:17.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.614+0000 7fdeb7516700 1 -- 172.21.15.78:0/2728417322 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fde94065ed0 msgr2=0x7fde94068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:17.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.614+0000 7fdeb7516700 1 --2- 172.21.15.78:0/2728417322 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fde94065ed0 0x7fde94068380 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fdeac0045f0 tx=0x7fdeac008040 comp rx=0 tx=0).stop 2023-10-11T09:50:17.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.614+0000 7fdeb7516700 1 -- 172.21.15.78:0/2728417322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdeb015d040 msgr2=0x7fdeb009fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.614+0000 7fdeb7516700 1 --2- 172.21.15.78:0/2728417322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdeb015d040 0x7fdeb009fa90 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fdea8007e60 tx=0x7fdea800e3c0 comp rx=0 tx=0).stop 2023-10-11T09:50:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.614+0000 7fdeb7516700 1 -- 172.21.15.78:0/2728417322 shutdown_connections 2023-10-11T09:50:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.614+0000 7fdeb7516700 1 --2- 172.21.15.78:0/2728417322 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fde94065ed0 0x7fde94068380 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.614+0000 7fdeb7516700 1 --2- 172.21.15.78:0/2728417322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdeb015d040 0x7fdeb009fa90 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.614+0000 7fdeb7516700 1 --2- 172.21.15.78:0/2728417322 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdeb00b3610 0x7fdeb009f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:17.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.614+0000 7fdeb7516700 1 --2- 172.21.15.78:0/2728417322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdeb00b3240 0x7fdeb015af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:17.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.614+0000 7fdeb7516700 1 -- 172.21.15.78:0/2728417322 >> 172.21.15.78:0/2728417322 conn(0x7fdeb009c260 msgr2=0x7fdeb00d0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:17.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.614+0000 7fdeb7516700 1 -- 172.21.15.78:0/2728417322 shutdown_connections 2023-10-11T09:50:17.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.614+0000 7fdeb7516700 1 -- 172.21.15.78:0/2728417322 wait complete. 2023-10-11T09:50:17.644 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T09:50:17.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.806+0000 7f447c9fc700 1 Processor -- start 2023-10-11T09:50:17.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.806+0000 7f447c9fc700 1 -- start start 2023-10-11T09:50:17.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.810+0000 7f447c9fc700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f447815b8e0 0x7f447806d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:17.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.810+0000 7f447c9fc700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f447815e010 con 0x7f44780ca950 2023-10-11T09:50:17.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.810+0000 7f447c9fc700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f447815e280 con 0x7f447806d610 2023-10-11T09:50:17.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.810+0000 7f447c9fc700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f447815e560 con 0x7f447815b8e0 2023-10-11T09:50:17.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.810+0000 7f4477fff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f44780ca950 0x7f447813fcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:57450/0 (socket says 172.21.15.78:57450) 2023-10-11T09:50:17.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.810+0000 7f4477fff700 1 -- 172.21.15.78:0/3944615811 learned_addr learned my addr 172.21.15.78:0/3944615811 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:50:17.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.810+0000 7f44777fe700 1 --2- 172.21.15.78:0/3944615811 >> v2:172.21.15.133:3300/0 conn(0x7f447815b8e0 0x7f447806d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:17.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.810+0000 7f44767fc700 1 -- 172.21.15.78:0/3944615811 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2592861418 0 0) 0x7f447815e010 con 0x7f44780ca950 2023-10-11T09:50:17.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.810+0000 7f44767fc700 1 -- 172.21.15.78:0/3944615811 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44600035d0 con 0x7f44780ca950 2023-10-11T09:50:17.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.810+0000 7f44767fc700 1 -- 172.21.15.78:0/3944615811 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 221065032 0 0) 0x7f447815e280 con 0x7f447806d610 2023-10-11T09:50:17.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.810+0000 7f44767fc700 1 -- 172.21.15.78:0/3944615811 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f447815e010 con 0x7f447806d610 2023-10-11T09:50:17.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.810+0000 7f44767fc700 1 -- 172.21.15.78:0/3944615811 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3291585951 0 0) 0x7f44600035d0 con 0x7f44780ca950 2023-10-11T09:50:17.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.810+0000 7f44767fc700 1 -- 172.21.15.78:0/3944615811 >> v2:172.21.15.133:3300/0 conn(0x7f447815b8e0 msgr2=0x7f447806d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:17.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.810+0000 7f44767fc700 1 --2- 172.21.15.78:0/3944615811 >> v2:172.21.15.133:3300/0 conn(0x7f447815b8e0 0x7f447806d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:17.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.810+0000 7f44767fc700 1 -- 172.21.15.78:0/3944615811 >> v1:172.21.15.133:6789/0 conn(0x7f447806d610 legacy=0x7f44780ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:17.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.810+0000 7f44767fc700 1 -- 172.21.15.78:0/3944615811 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447815e840 con 0x7f44780ca950 2023-10-11T09:50:17.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.810+0000 7f44767fc700 1 -- 172.21.15.78:0/3944615811 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f4470001a10 con 0x7f44780ca950 2023-10-11T09:50:17.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.810+0000 7f44777fe700 1 --2- 172.21.15.78:0/3944615811 >> v2:172.21.15.133:3300/0 conn(0x7f447815b8e0 0x7f447806d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:50:17.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.810+0000 7f44767fc700 1 -- 172.21.15.78:0/3944615811 >> v1:172.21.15.78:6789/0 conn(0x7f44780ca950 legacy=0x7f447813fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:17.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.810+0000 7f44767fc700 1 --2- 172.21.15.78:0/3944615811 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4460003bb0 0x7f4460003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:17.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.810+0000 7f44767fc700 1 --2- 172.21.15.78:0/3944615811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4460004870 0x7f4460004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:17.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.810+0000 7f44767fc700 1 --2- 172.21.15.78:0/3944615811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4460005220 0x7f446000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:17.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.810+0000 7f44767fc700 1 -- 172.21.15.78:0/3944615811 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447815e560 con 0x7f4460004870 2023-10-11T09:50:17.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.810+0000 7f44767fc700 1 -- 172.21.15.78:0/3944615811 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446000c080 con 0x7f4460005220 2023-10-11T09:50:17.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.810+0000 7f44767fc700 1 -- 172.21.15.78:0/3944615811 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f446000c240 con 0x7f4460003bb0 2023-10-11T09:50:17.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.810+0000 7f4476ffd700 1 --2- 172.21.15.78:0/3944615811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4460004870 0x7f4460004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:17.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.810+0000 7f4477fff700 1 --2- 172.21.15.78:0/3944615811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4460005220 0x7f446000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:17.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.810+0000 7f44777fe700 1 --2- 172.21.15.78:0/3944615811 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4460003bb0 0x7f4460003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:17.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.814+0000 7f4476ffd700 1 -- 172.21.15.78:0/3944615811 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4460003bb0 msgr2=0x7f4460003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:17.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.814+0000 7f4476ffd700 1 --2- 172.21.15.78:0/3944615811 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4460003bb0 0x7f4460003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:17.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.814+0000 7f4476ffd700 1 -- 172.21.15.78:0/3944615811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4460005220 msgr2=0x7f446000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:17.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.814+0000 7f4476ffd700 1 --2- 172.21.15.78:0/3944615811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4460005220 0x7f446000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:17.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.814+0000 7f4476ffd700 1 -- 172.21.15.78:0/3944615811 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44600035d0 con 0x7f4460004870 2023-10-11T09:50:17.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.814+0000 7f4477fff700 1 -- 172.21.15.78:0/3944615811 reap_dead start 2023-10-11T09:50:17.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.814+0000 7f4476ffd700 1 --2- 172.21.15.78:0/3944615811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4460004870 0x7f4460004ce0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f446c001960 tx=0x7f44600058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92c6d7d42e1a9e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:17.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.814+0000 7f44767fc700 1 -- 172.21.15.78:0/3944615811 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c001450 con 0x7f4460004870 2023-10-11T09:50:17.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.814+0000 7f44767fc700 1 -- 172.21.15.78:0/3944615811 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446c006aa0 con 0x7f4460004870 2023-10-11T09:50:17.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.814+0000 7f44767fc700 1 -- 172.21.15.78:0/3944615811 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c006e50 con 0x7f4460004870 2023-10-11T09:50:17.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.814+0000 7f447c9fc700 1 -- 172.21.15.78:0/3944615811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4460004870 msgr2=0x7f4460004ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:17.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.814+0000 7f447c9fc700 1 --2- 172.21.15.78:0/3944615811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4460004870 0x7f4460004ce0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f446c001960 tx=0x7f44600058a0 comp rx=0 tx=0).stop 2023-10-11T09:50:17.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.814+0000 7f447c9fc700 1 -- 172.21.15.78:0/3944615811 shutdown_connections 2023-10-11T09:50:17.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.814+0000 7f447c9fc700 1 --2- 172.21.15.78:0/3944615811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4460004870 0x7f4460004ce0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:17.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.814+0000 7f447c9fc700 1 -- 172.21.15.78:0/3944615811 >> 172.21.15.78:0/3944615811 conn(0x7f44780cea60 msgr2=0x7f44780cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:17.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.814+0000 7f447c9fc700 1 -- 172.21.15.78:0/3944615811 shutdown_connections 2023-10-11T09:50:17.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.814+0000 7f447c9fc700 1 -- 172.21.15.78:0/3944615811 wait complete. 2023-10-11T09:50:17.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.818+0000 7f447c9fc700 1 Processor -- start 2023-10-11T09:50:17.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.818+0000 7f447c9fc700 1 -- start start 2023-10-11T09:50:17.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.822+0000 7f447c9fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4460004870 0x7f44780b4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:17.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.822+0000 7f447c9fc700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4478070c60 0x7f44780710d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:17.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.822+0000 7f447c9fc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f447813cf30 0x7f447813d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:17.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.822+0000 7f447c9fc700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4478071610 con 0x7f4460004870 2023-10-11T09:50:17.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.822+0000 7f447c9fc700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44780b4ac0 con 0x7f447813cf30 2023-10-11T09:50:17.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.822+0000 7f447c9fc700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44780b2230 con 0x7f4478070c60 2023-10-11T09:50:17.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.822+0000 7f44777fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4460004870 0x7f44780b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:17.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.822+0000 7f44777fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4460004870 0x7f44780b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42838/0 (socket says 172.21.15.78:42838) 2023-10-11T09:50:17.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.822+0000 7f44777fe700 1 -- 172.21.15.78:0/1683317955 learned_addr learned my addr 172.21.15.78:0/1683317955 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:17.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.822+0000 7f4476ffd700 1 --2- 172.21.15.78:0/1683317955 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4478070c60 0x7f44780710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:17.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.822+0000 7f4477fff700 1 --2- 172.21.15.78:0/1683317955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f447813cf30 0x7f447813d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:17.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.822+0000 7f44777fe700 1 -- 172.21.15.78:0/1683317955 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4478070c60 msgr2=0x7f44780710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:17.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.822+0000 7f44777fe700 1 --2- 172.21.15.78:0/1683317955 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4478070c60 0x7f44780710d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:17.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.822+0000 7f44777fe700 1 -- 172.21.15.78:0/1683317955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f447813cf30 msgr2=0x7f447813d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:17.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.822+0000 7f44777fe700 1 --2- 172.21.15.78:0/1683317955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f447813cf30 0x7f447813d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:17.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.822+0000 7f44777fe700 1 -- 172.21.15.78:0/1683317955 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447813dd40 con 0x7f4460004870 2023-10-11T09:50:17.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.822+0000 7f4477fff700 1 --2- 172.21.15.78:0/1683317955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f447813cf30 0x7f447813d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:50:17.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.822+0000 7f44777fe700 1 --2- 172.21.15.78:0/1683317955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4460004870 0x7f44780b4580 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f4468011c30 tx=0x7f4468011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:17.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.822+0000 7f445ffff700 1 -- 172.21.15.78:0/1683317955 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446800fac0 con 0x7f4460004870 2023-10-11T09:50:17.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.822+0000 7f445ffff700 1 -- 172.21.15.78:0/1683317955 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446800fc20 con 0x7f4460004870 2023-10-11T09:50:17.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.822+0000 7f447c9fc700 1 -- 172.21.15.78:0/1683317955 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f447813e020 con 0x7f4460004870 2023-10-11T09:50:17.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.822+0000 7f445ffff700 1 -- 172.21.15.78:0/1683317955 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4468014440 con 0x7f4460004870 2023-10-11T09:50:17.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.822+0000 7f447c9fc700 1 -- 172.21.15.78:0/1683317955 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f447813e300 con 0x7f4460004870 2023-10-11T09:50:17.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.826+0000 7f447c9fc700 1 -- 172.21.15.78:0/1683317955 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4478065d40 con 0x7f4460004870 2023-10-11T09:50:17.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.826+0000 7f445ffff700 1 -- 172.21.15.78:0/1683317955 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f44680145a0 con 0x7f4460004870 2023-10-11T09:50:17.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.826+0000 7f445ffff700 1 --2- 172.21.15.78:0/1683317955 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4464065f00 0x7f44640683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:17.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.830+0000 7f445ffff700 1 -- 172.21.15.78:0/1683317955 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f4468084090 con 0x7f4460004870 2023-10-11T09:50:17.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.830+0000 7f4476ffd700 1 --2- 172.21.15.78:0/1683317955 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4464065f00 0x7f44640683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:17.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.830+0000 7f4476ffd700 1 --2- 172.21.15.78:0/1683317955 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4464065f00 0x7f44640683b0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f446c015040 tx=0x7f446c005040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:17.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:17.834+0000 7f445ffff700 1 -- 172.21.15.78:0/1683317955 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f44680514a0 con 0x7f4460004870 2023-10-11T09:50:18.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.022+0000 7f447c9fc700 1 -- 172.21.15.78:0/1683317955 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f44780cdbd0 con 0x7f4460004870 2023-10-11T09:50:18.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.022+0000 7f445ffff700 1 -- 172.21.15.78:0/1683317955 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v108) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4468054ac0 con 0x7f4460004870 2023-10-11T09:50:18.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.022+0000 7f447c9fc700 1 -- 172.21.15.78:0/1683317955 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4464065f00 msgr2=0x7f44640683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:18.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.022+0000 7f447c9fc700 1 --2- 172.21.15.78:0/1683317955 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4464065f00 0x7f44640683b0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f446c015040 tx=0x7f446c005040 comp rx=0 tx=0).stop 2023-10-11T09:50:18.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.022+0000 7f447c9fc700 1 -- 172.21.15.78:0/1683317955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4460004870 msgr2=0x7f44780b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:18.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.022+0000 7f447c9fc700 1 --2- 172.21.15.78:0/1683317955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4460004870 0x7f44780b4580 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f4468011c30 tx=0x7f4468011f40 comp rx=0 tx=0).stop 2023-10-11T09:50:18.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.022+0000 7f447c9fc700 1 -- 172.21.15.78:0/1683317955 shutdown_connections 2023-10-11T09:50:18.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.026+0000 7f447c9fc700 1 --2- 172.21.15.78:0/1683317955 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4464065f00 0x7f44640683b0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:18.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.026+0000 7f447c9fc700 1 --2- 172.21.15.78:0/1683317955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f447813cf30 0x7f447813d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:18.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.026+0000 7f447c9fc700 1 --2- 172.21.15.78:0/1683317955 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4478070c60 0x7f44780710d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:18.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.026+0000 7f447c9fc700 1 --2- 172.21.15.78:0/1683317955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4460004870 0x7f44780b4580 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:18.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.026+0000 7f447c9fc700 1 -- 172.21.15.78:0/1683317955 >> 172.21.15.78:0/1683317955 conn(0x7f44780cea60 msgr2=0x7f44780ce0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:18.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.026+0000 7f447c9fc700 1 -- 172.21.15.78:0/1683317955 shutdown_connections 2023-10-11T09:50:18.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.026+0000 7f447c9fc700 1 -- 172.21.15.78:0/1683317955 wait complete. 2023-10-11T09:50:18.035 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2023-10-11T09:50:18.052 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:50:18.053 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:50:18.070 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:50:18.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.278+0000 7f9c1a9d2700 1 Processor -- start 2023-10-11T09:50:18.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.282+0000 7f9c1a9d2700 1 -- start start 2023-10-11T09:50:18.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.282+0000 7f9c1a9d2700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9c1415a940 0x7f9c140ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:18.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.282+0000 7f9c1a9d2700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c1415e010 con 0x7f9c1415b8e0 2023-10-11T09:50:18.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.282+0000 7f9c1a9d2700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c140cab70 con 0x7f9c140ca780 2023-10-11T09:50:18.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.282+0000 7f9c1a9d2700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c1415e280 con 0x7f9c1415a940 2023-10-11T09:50:18.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.282+0000 7f9c199d0700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f9c1415b8e0 0x7f9c1406d0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:57466/0 (socket says 172.21.15.78:57466) 2023-10-11T09:50:18.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.282+0000 7f9c199d0700 1 -- 172.21.15.78:0/1745332756 learned_addr learned my addr 172.21.15.78:0/1745332756 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:50:18.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.286+0000 7f9c191cf700 1 --2- 172.21.15.78:0/1745332756 >> v2:172.21.15.133:3300/0 conn(0x7f9c1415a940 0x7f9c140ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:18.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.286+0000 7f9c189ce700 1 -- 172.21.15.78:0/1745332756 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 183276877 0 0) 0x7f9c1415e010 con 0x7f9c1415b8e0 2023-10-11T09:50:18.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.286+0000 7f9c189ce700 1 -- 172.21.15.78:0/1745332756 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c000035d0 con 0x7f9c1415b8e0 2023-10-11T09:50:18.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.286+0000 7f9c191cf700 1 -- 172.21.15.78:0/1745332756 >> v1:172.21.15.133:6789/0 conn(0x7f9c140ca780 legacy=0x7f9c1413fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:18.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.286+0000 7f9c191cf700 1 -- 172.21.15.78:0/1745332756 >> v1:172.21.15.78:6789/0 conn(0x7f9c1415b8e0 legacy=0x7f9c1406d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:18.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.286+0000 7f9c191cf700 1 -- 172.21.15.78:0/1745332756 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c1415e560 con 0x7f9c1415a940 2023-10-11T09:50:18.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.286+0000 7f9c191cf700 1 --2- 172.21.15.78:0/1745332756 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c1415a940 0x7f9c140ca240 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9c04009850 tx=0x7f9c04009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90000d0ca6b38a5d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:18.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.286+0000 7f9c189ce700 1 -- 172.21.15.78:0/1745332756 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c04011830 con 0x7f9c1415a940 2023-10-11T09:50:18.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.286+0000 7f9c189ce700 1 -- 172.21.15.78:0/1745332756 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c04011990 con 0x7f9c1415a940 2023-10-11T09:50:18.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.286+0000 7f9c189ce700 1 -- 172.21.15.78:0/1745332756 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c04011d40 con 0x7f9c1415a940 2023-10-11T09:50:18.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.286+0000 7f9c1a9d2700 1 -- 172.21.15.78:0/1745332756 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c1415a940 msgr2=0x7f9c140ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:18.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.286+0000 7f9c1a9d2700 1 --2- 172.21.15.78:0/1745332756 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c1415a940 0x7f9c140ca240 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9c04009850 tx=0x7f9c04009b60 comp rx=0 tx=0).stop 2023-10-11T09:50:18.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.286+0000 7f9c1a9d2700 1 -- 172.21.15.78:0/1745332756 shutdown_connections 2023-10-11T09:50:18.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.286+0000 7f9c1a9d2700 1 --2- 172.21.15.78:0/1745332756 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c1415a940 0x7f9c140ca240 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:18.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.286+0000 7f9c1a9d2700 1 -- 172.21.15.78:0/1745332756 >> 172.21.15.78:0/1745332756 conn(0x7f9c140cea60 msgr2=0x7f9c140cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:18.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.286+0000 7f9c1a9d2700 1 -- 172.21.15.78:0/1745332756 shutdown_connections 2023-10-11T09:50:18.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.286+0000 7f9c1a9d2700 1 -- 172.21.15.78:0/1745332756 wait complete. 2023-10-11T09:50:18.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.290+0000 7f9c1a9d2700 1 Processor -- start 2023-10-11T09:50:18.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.290+0000 7f9c1a9d2700 1 -- start start 2023-10-11T09:50:18.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.294+0000 7f9c1a9d2700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c140ca780 0x7f9c14141360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:18.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.294+0000 7f9c1a9d2700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c1415a940 0x7f9c1413d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:18.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.294+0000 7f9c1a9d2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c1415b8e0 0x7f9c1413fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:18.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.294+0000 7f9c1a9d2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c140ccc90 con 0x7f9c1415b8e0 2023-10-11T09:50:18.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.294+0000 7f9c1a9d2700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1413da20 con 0x7f9c1415a940 2023-10-11T09:50:18.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.294+0000 7f9c1a9d2700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c14140420 con 0x7f9c140ca780 2023-10-11T09:50:18.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.294+0000 7f9c1a1d1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c1415b8e0 0x7f9c1413fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:18.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.294+0000 7f9c1a1d1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c1415b8e0 0x7f9c1413fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42844/0 (socket says 172.21.15.78:42844) 2023-10-11T09:50:18.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.294+0000 7f9c1a1d1700 1 -- 172.21.15.78:0/3510013522 learned_addr learned my addr 172.21.15.78:0/3510013522 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:18.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.294+0000 7f9c199d0700 1 --2- 172.21.15.78:0/3510013522 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c140ca780 0x7f9c14141360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:18.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.294+0000 7f9c191cf700 1 --2- 172.21.15.78:0/3510013522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c1415a940 0x7f9c1413d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:18.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.294+0000 7f9c1a1d1700 1 -- 172.21.15.78:0/3510013522 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c140ca780 msgr2=0x7f9c14141360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:18.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.294+0000 7f9c1a1d1700 1 --2- 172.21.15.78:0/3510013522 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c140ca780 0x7f9c14141360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:18.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.294+0000 7f9c1a1d1700 1 -- 172.21.15.78:0/3510013522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c1415a940 msgr2=0x7f9c1413d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:18.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.294+0000 7f9c1a1d1700 1 --2- 172.21.15.78:0/3510013522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c1415a940 0x7f9c1413d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:18.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.294+0000 7f9c1a1d1700 1 -- 172.21.15.78:0/3510013522 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c040094b0 con 0x7f9c1415b8e0 2023-10-11T09:50:18.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.294+0000 7f9c199d0700 1 --2- 172.21.15.78:0/3510013522 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c140ca780 0x7f9c14141360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:50:18.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.294+0000 7f9c191cf700 1 --2- 172.21.15.78:0/3510013522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c1415a940 0x7f9c1413d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:50:18.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.294+0000 7f9c1a1d1700 1 --2- 172.21.15.78:0/3510013522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c1415b8e0 0x7f9c1413fee0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c00cb80 tx=0x7f9c0c00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:18.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.294+0000 7f9c0a7fc700 1 -- 172.21.15.78:0/3510013522 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c003c20 con 0x7f9c1415b8e0 2023-10-11T09:50:18.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.294+0000 7f9c0a7fc700 1 -- 172.21.15.78:0/3510013522 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c0c003d80 con 0x7f9c1415b8e0 2023-10-11T09:50:18.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.294+0000 7f9c0a7fc700 1 -- 172.21.15.78:0/3510013522 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c00abb0 con 0x7f9c1415b8e0 2023-10-11T09:50:18.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.294+0000 7f9c1a9d2700 1 -- 172.21.15.78:0/3510013522 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c141406c0 con 0x7f9c1415b8e0 2023-10-11T09:50:18.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.294+0000 7f9c1a9d2700 1 -- 172.21.15.78:0/3510013522 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c140cc640 con 0x7f9c1415b8e0 2023-10-11T09:50:18.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.298+0000 7f9c1a9d2700 1 -- 172.21.15.78:0/3510013522 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c140cce00 con 0x7f9c1415b8e0 2023-10-11T09:50:18.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.298+0000 7f9c0a7fc700 1 -- 172.21.15.78:0/3510013522 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f9c0c00ad10 con 0x7f9c1415b8e0 2023-10-11T09:50:18.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.302+0000 7f9c0a7fc700 1 --2- 172.21.15.78:0/3510013522 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9bf8065ed0 0x7f9bf8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:18.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.302+0000 7f9c0a7fc700 1 -- 172.21.15.78:0/3510013522 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f9c0c083a70 con 0x7f9c1415b8e0 2023-10-11T09:50:18.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.302+0000 7f9c199d0700 1 --2- 172.21.15.78:0/3510013522 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9bf8065ed0 0x7f9bf8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:18.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.302+0000 7f9c199d0700 1 --2- 172.21.15.78:0/3510013522 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9bf8065ed0 0x7f9bf8068380 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9c100011e0 tx=0x7f9c10009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:18.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.306+0000 7f9c0a7fc700 1 -- 172.21.15.78:0/3510013522 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f9c0c050e80 con 0x7f9c1415b8e0 2023-10-11T09:50:18.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.482+0000 7f9c1a9d2700 1 -- 172.21.15.78:0/3510013522 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c14065d40 con 0x7f9c1415b8e0 2023-10-11T09:50:18.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.482+0000 7f9c0a7fc700 1 -- 172.21.15.78:0/3510013522 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+19623 (secure 0 0 0) 0x7f9c0c0544a0 con 0x7f9c1415b8e0 2023-10-11T09:50:18.492 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:18.492 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":108,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:50:01.881632+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:49:49.297597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-11T09:49:49.955104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data_extra","create_time":"2023-10-11T09:49:59.915561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:50:18.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.490+0000 7f9c1a9d2700 1 -- 172.21.15.78:0/3510013522 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9bf8065ed0 msgr2=0x7f9bf8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:18.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.490+0000 7f9c1a9d2700 1 --2- 172.21.15.78:0/3510013522 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9bf8065ed0 0x7f9bf8068380 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9c100011e0 tx=0x7f9c10009040 comp rx=0 tx=0).stop 2023-10-11T09:50:18.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.490+0000 7f9c1a9d2700 1 -- 172.21.15.78:0/3510013522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c1415b8e0 msgr2=0x7f9c1413fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:18.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.490+0000 7f9c1a9d2700 1 --2- 172.21.15.78:0/3510013522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c1415b8e0 0x7f9c1413fee0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c00cb80 tx=0x7f9c0c00cf40 comp rx=0 tx=0).stop 2023-10-11T09:50:18.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.490+0000 7f9c1a9d2700 1 -- 172.21.15.78:0/3510013522 shutdown_connections 2023-10-11T09:50:18.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.490+0000 7f9c1a9d2700 1 --2- 172.21.15.78:0/3510013522 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9bf8065ed0 0x7f9bf8068380 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:18.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.490+0000 7f9c1a9d2700 1 --2- 172.21.15.78:0/3510013522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c1415b8e0 0x7f9c1413fee0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:18.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.490+0000 7f9c1a9d2700 1 --2- 172.21.15.78:0/3510013522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c1415a940 0x7f9c1413d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:18.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.490+0000 7f9c1a9d2700 1 --2- 172.21.15.78:0/3510013522 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c140ca780 0x7f9c14141360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:18.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.490+0000 7f9c1a9d2700 1 -- 172.21.15.78:0/3510013522 >> 172.21.15.78:0/3510013522 conn(0x7f9c140cea60 msgr2=0x7f9c1406d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:18.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.490+0000 7f9c1a9d2700 1 -- 172.21.15.78:0/3510013522 shutdown_connections 2023-10-11T09:50:18.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.490+0000 7f9c1a9d2700 1 -- 172.21.15.78:0/3510013522 wait complete. 2023-10-11T09:50:18.516 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T09:49:49.297597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T09:49:49.955104+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-10-11T09:49:59.915561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:50:18.517 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:50:18.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.686+0000 7f19e6151700 1 Processor -- start 2023-10-11T09:50:18.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.686+0000 7f19e6151700 1 -- start start 2023-10-11T09:50:18.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.686+0000 7f19e6151700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f19e00b3240 0x7f19e00d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:18.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.686+0000 7f19e6151700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f19e00b3610 0x7f19e00b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:18.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.686+0000 7f19e6151700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19e00b47f0 con 0x7f19e015d040 2023-10-11T09:50:18.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.686+0000 7f19e6151700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19e00b3fd0 con 0x7f19e00b3240 2023-10-11T09:50:18.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.686+0000 7f19e6151700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e00b8130 con 0x7f19e00b3610 2023-10-11T09:50:18.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.686+0000 7f19e514f700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f19e00b3240 0x7f19e00d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:18.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.686+0000 7f19e514f700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f19e00b3240 0x7f19e00d29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42852/0 (socket says 172.21.15.78:42852) 2023-10-11T09:50:18.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.686+0000 7f19e514f700 1 -- 172.21.15.78:0/1013404748 learned_addr learned my addr 172.21.15.78:0/1013404748 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:18.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.686+0000 7f19e494e700 1 --2- 172.21.15.78:0/1013404748 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f19e00b3610 0x7f19e00b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:18.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.686+0000 7f19e514f700 1 -- 172.21.15.78:0/1013404748 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f19e00b3610 msgr2=0x7f19e00b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:18.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.686+0000 7f19e514f700 1 --2- 172.21.15.78:0/1013404748 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f19e00b3610 0x7f19e00b3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:18.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.686+0000 7f19e514f700 1 -- 172.21.15.78:0/1013404748 >> v1:172.21.15.133:6789/0 conn(0x7f19e015d040 legacy=0x7f19e00b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:18.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.686+0000 7f19e514f700 1 -- 172.21.15.78:0/1013404748 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e00b41e0 con 0x7f19e00b3240 2023-10-11T09:50:18.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.686+0000 7f19e494e700 1 --2- 172.21.15.78:0/1013404748 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f19e00b3610 0x7f19e00b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:50:18.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.690+0000 7f19e514f700 1 --2- 172.21.15.78:0/1013404748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19e00b3240 0x7f19e00d29d0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f19dc009910 tx=0x7f19dc009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1c393e3d7653a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:18.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.690+0000 7f19d7fff700 1 -- 172.21.15.78:0/1013404748 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc00ba60 con 0x7f19e00b3240 2023-10-11T09:50:18.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.690+0000 7f19d7fff700 1 -- 172.21.15.78:0/1013404748 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19dc00bbc0 con 0x7f19e00b3240 2023-10-11T09:50:18.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.690+0000 7f19d7fff700 1 -- 172.21.15.78:0/1013404748 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc009e60 con 0x7f19e00b3240 2023-10-11T09:50:18.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.690+0000 7f19e6151700 1 -- 172.21.15.78:0/1013404748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19e00b3240 msgr2=0x7f19e00d29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:18.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.690+0000 7f19e6151700 1 --2- 172.21.15.78:0/1013404748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19e00b3240 0x7f19e00d29d0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f19dc009910 tx=0x7f19dc009c20 comp rx=0 tx=0).stop 2023-10-11T09:50:18.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.690+0000 7f19e6151700 1 -- 172.21.15.78:0/1013404748 shutdown_connections 2023-10-11T09:50:18.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.690+0000 7f19e6151700 1 --2- 172.21.15.78:0/1013404748 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f19e00b3610 0x7f19e00b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:18.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.690+0000 7f19e6151700 1 --2- 172.21.15.78:0/1013404748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19e00b3240 0x7f19e00d29d0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:18.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.690+0000 7f19e6151700 1 -- 172.21.15.78:0/1013404748 >> 172.21.15.78:0/1013404748 conn(0x7f19e009c260 msgr2=0x7f19e00d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:18.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.690+0000 7f19e6151700 1 -- 172.21.15.78:0/1013404748 shutdown_connections 2023-10-11T09:50:18.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.690+0000 7f19e6151700 1 -- 172.21.15.78:0/1013404748 wait complete. 2023-10-11T09:50:18.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.694+0000 7f19e6151700 1 Processor -- start 2023-10-11T09:50:18.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.694+0000 7f19e6151700 1 -- start start 2023-10-11T09:50:18.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.694+0000 7f19e6151700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f19e00b3240 0x7f19e0147cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:18.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.694+0000 7f19e6151700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19e00b3610 0x7f19e006c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:18.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.694+0000 7f19e6151700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19e015d040 0x7f19e006c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:18.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.694+0000 7f19e6151700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e014b260 con 0x7f19e00b3610 2023-10-11T09:50:18.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.694+0000 7f19e6151700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e014a670 con 0x7f19e015d040 2023-10-11T09:50:18.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.694+0000 7f19e6151700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e014a8c0 con 0x7f19e00b3240 2023-10-11T09:50:18.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.694+0000 7f19e494e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19e00b3610 0x7f19e006c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:18.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.694+0000 7f19e514f700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f19e00b3240 0x7f19e0147cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:18.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.694+0000 7f19e494e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19e00b3610 0x7f19e006c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42858/0 (socket says 172.21.15.78:42858) 2023-10-11T09:50:18.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.694+0000 7f19e494e700 1 -- 172.21.15.78:0/2669246724 learned_addr learned my addr 172.21.15.78:0/2669246724 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:18.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.694+0000 7f19e5950700 1 --2- 172.21.15.78:0/2669246724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19e015d040 0x7f19e006c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:18.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.694+0000 7f19e494e700 1 -- 172.21.15.78:0/2669246724 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f19e00b3240 msgr2=0x7f19e0147cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:18.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.694+0000 7f19e494e700 1 --2- 172.21.15.78:0/2669246724 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f19e00b3240 0x7f19e0147cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:18.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.694+0000 7f19e494e700 1 -- 172.21.15.78:0/2669246724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19e015d040 msgr2=0x7f19e006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:18.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.694+0000 7f19e494e700 1 --2- 172.21.15.78:0/2669246724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19e015d040 0x7f19e006c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:18.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.694+0000 7f19e494e700 1 -- 172.21.15.78:0/2669246724 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19dc009570 con 0x7f19e00b3610 2023-10-11T09:50:18.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.694+0000 7f19e494e700 1 --2- 172.21.15.78:0/2669246724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19e00b3610 0x7f19e006c130 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f19d000e7e0 tx=0x7f19d000eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:18.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.694+0000 7f19d5ffb700 1 -- 172.21.15.78:0/2669246724 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d000c020 con 0x7f19e00b3610 2023-10-11T09:50:18.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.698+0000 7f19d5ffb700 1 -- 172.21.15.78:0/2669246724 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d0004230 con 0x7f19e00b3610 2023-10-11T09:50:18.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.698+0000 7f19d5ffb700 1 -- 172.21.15.78:0/2669246724 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d00045c0 con 0x7f19e00b3610 2023-10-11T09:50:18.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.698+0000 7f19e6151700 1 -- 172.21.15.78:0/2669246724 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e014aba0 con 0x7f19e00b3610 2023-10-11T09:50:18.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.698+0000 7f19e6151700 1 -- 172.21.15.78:0/2669246724 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e0148200 con 0x7f19e00b3610 2023-10-11T09:50:18.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.698+0000 7f19e6151700 1 -- 172.21.15.78:0/2669246724 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e014b3a0 con 0x7f19e00b3610 2023-10-11T09:50:18.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.698+0000 7f19d5ffb700 1 -- 172.21.15.78:0/2669246724 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f19d00036c0 con 0x7f19e00b3610 2023-10-11T09:50:18.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.702+0000 7f19d5ffb700 1 --2- 172.21.15.78:0/2669246724 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f19c4065ed0 0x7f19c4068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:18.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.702+0000 7f19d5ffb700 1 -- 172.21.15.78:0/2669246724 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f19d00836e0 con 0x7f19e00b3610 2023-10-11T09:50:18.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.702+0000 7f19e514f700 1 --2- 172.21.15.78:0/2669246724 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f19c4065ed0 0x7f19c4068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:18.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.702+0000 7f19e514f700 1 --2- 172.21.15.78:0/2669246724 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f19c4065ed0 0x7f19c4068380 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f19dc000c00 tx=0x7f19dc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:18.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.706+0000 7f19d5ffb700 1 -- 172.21.15.78:0/2669246724 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f19e014b3a0 con 0x7f19e00b3610 2023-10-11T09:50:18.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.882+0000 7f19e6151700 1 -- 172.21.15.78:0/2669246724 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f19e014b3a0 con 0x7f19e00b3610 2023-10-11T09:50:18.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.882+0000 7f19d5ffb700 1 -- 172.21.15.78:0/2669246724 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v108) v1 ==== 93+0+10 (secure 0 0 0) 0x7f19e014b3a0 con 0x7f19e00b3610 2023-10-11T09:50:18.889 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:50:18.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.886+0000 7f19e6151700 1 -- 172.21.15.78:0/2669246724 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f19c4065ed0 msgr2=0x7f19c4068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:18.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.886+0000 7f19e6151700 1 --2- 172.21.15.78:0/2669246724 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f19c4065ed0 0x7f19c4068380 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f19dc000c00 tx=0x7f19dc008030 comp rx=0 tx=0).stop 2023-10-11T09:50:18.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.886+0000 7f19e6151700 1 -- 172.21.15.78:0/2669246724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19e00b3610 msgr2=0x7f19e006c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:18.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.886+0000 7f19e6151700 1 --2- 172.21.15.78:0/2669246724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19e00b3610 0x7f19e006c130 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f19d000e7e0 tx=0x7f19d000eaf0 comp rx=0 tx=0).stop 2023-10-11T09:50:18.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.886+0000 7f19e6151700 1 -- 172.21.15.78:0/2669246724 shutdown_connections 2023-10-11T09:50:18.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.886+0000 7f19e6151700 1 --2- 172.21.15.78:0/2669246724 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f19c4065ed0 0x7f19c4068380 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:18.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.886+0000 7f19e6151700 1 --2- 172.21.15.78:0/2669246724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19e015d040 0x7f19e006c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:18.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.886+0000 7f19e6151700 1 --2- 172.21.15.78:0/2669246724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19e00b3610 0x7f19e006c130 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:18.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.886+0000 7f19e6151700 1 --2- 172.21.15.78:0/2669246724 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f19e00b3240 0x7f19e0147cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:18.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.886+0000 7f19e6151700 1 -- 172.21.15.78:0/2669246724 >> 172.21.15.78:0/2669246724 conn(0x7f19e009c260 msgr2=0x7f19e015c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:18.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.886+0000 7f19e6151700 1 -- 172.21.15.78:0/2669246724 shutdown_connections 2023-10-11T09:50:18.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:18.886+0000 7f19e6151700 1 -- 172.21.15.78:0/2669246724 wait complete. 2023-10-11T09:50:18.913 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:50:19.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.094+0000 7fe2b1151700 1 Processor -- start 2023-10-11T09:50:19.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.094+0000 7fe2b1151700 1 -- start start 2023-10-11T09:50:19.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.098+0000 7fe2b1151700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe2ac0cc890 0x7fe2ac0ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:19.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.098+0000 7fe2b1151700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe2ac06cda0 0x7fe2ac0afb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:19.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.098+0000 7fe2b1151700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2ac161550 con 0x7fe2ac0cfc30 2023-10-11T09:50:19.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.098+0000 7fe2b1151700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2ac06d1f0 con 0x7fe2ac06cda0 2023-10-11T09:50:19.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.098+0000 7fe2b1151700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2ac0ce700 con 0x7fe2ac0cc890 2023-10-11T09:50:19.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.098+0000 7fe2b0950700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe2ac06cda0 0x7fe2ac0afb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:19.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.098+0000 7fe2b0950700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe2ac06cda0 0x7fe2ac0afb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42868/0 (socket says 172.21.15.78:42868) 2023-10-11T09:50:19.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.098+0000 7fe2b0950700 1 -- 172.21.15.78:0/4101434699 learned_addr learned my addr 172.21.15.78:0/4101434699 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:19.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.098+0000 7fe2abfff700 1 --2- 172.21.15.78:0/4101434699 >> v2:172.21.15.133:3300/0 conn(0x7fe2ac0cc890 0x7fe2ac0ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:19.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.098+0000 7fe2b0950700 1 -- 172.21.15.78:0/4101434699 >> v2:172.21.15.133:3300/0 conn(0x7fe2ac0cc890 msgr2=0x7fe2ac0ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:19.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.098+0000 7fe2b0950700 1 --2- 172.21.15.78:0/4101434699 >> v2:172.21.15.133:3300/0 conn(0x7fe2ac0cc890 0x7fe2ac0ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:19.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.098+0000 7fe2b0950700 1 -- 172.21.15.78:0/4101434699 >> v1:172.21.15.133:6789/0 conn(0x7fe2ac0cfc30 legacy=0x7fe2ac0b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:19.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.098+0000 7fe2b0950700 1 -- 172.21.15.78:0/4101434699 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ac160940 con 0x7fe2ac06cda0 2023-10-11T09:50:19.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.102+0000 7fe2b0950700 1 --2- 172.21.15.78:0/4101434699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2ac06cda0 0x7fe2ac0afb20 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac06d350 tx=0x7fe2a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f3eb6d7dfbd2d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:19.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.102+0000 7fe2ab7fe700 1 -- 172.21.15.78:0/4101434699 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a400e070 con 0x7fe2ac06cda0 2023-10-11T09:50:19.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.102+0000 7fe2ab7fe700 1 -- 172.21.15.78:0/4101434699 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a400b9c0 con 0x7fe2ac06cda0 2023-10-11T09:50:19.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.102+0000 7fe2ab7fe700 1 -- 172.21.15.78:0/4101434699 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a400bd70 con 0x7fe2ac06cda0 2023-10-11T09:50:19.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.102+0000 7fe2b1151700 1 -- 172.21.15.78:0/4101434699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2ac06cda0 msgr2=0x7fe2ac0afb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:19.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.102+0000 7fe2b1151700 1 --2- 172.21.15.78:0/4101434699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2ac06cda0 0x7fe2ac0afb20 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac06d350 tx=0x7fe2a400b330 comp rx=0 tx=0).stop 2023-10-11T09:50:19.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.102+0000 7fe2b1151700 1 -- 172.21.15.78:0/4101434699 shutdown_connections 2023-10-11T09:50:19.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.102+0000 7fe2b1151700 1 --2- 172.21.15.78:0/4101434699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2ac06cda0 0x7fe2ac0afb20 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:19.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.102+0000 7fe2b1151700 1 --2- 172.21.15.78:0/4101434699 >> v2:172.21.15.133:3300/0 conn(0x7fe2ac0cc890 0x7fe2ac0ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:19.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.102+0000 7fe2b1151700 1 -- 172.21.15.78:0/4101434699 >> 172.21.15.78:0/4101434699 conn(0x7fe2ac06a060 msgr2=0x7fe2ac06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:19.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.102+0000 7fe2b1151700 1 -- 172.21.15.78:0/4101434699 shutdown_connections 2023-10-11T09:50:19.114 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.102+0000 7fe2b1151700 1 -- 172.21.15.78:0/4101434699 wait complete. 2023-10-11T09:50:19.114 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.106+0000 7fe2b1151700 1 Processor -- start 2023-10-11T09:50:19.114 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.106+0000 7fe2b1151700 1 -- start start 2023-10-11T09:50:19.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.106+0000 7fe2b1151700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2ac06cda0 0x7fe2ac15a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:19.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.106+0000 7fe2b1151700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2ac0cc890 0x7fe2ac06e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:19.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.106+0000 7fe2b1151700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2ac0cfc30 0x7fe2ac06e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:19.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.106+0000 7fe2b1151700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac15ac80 con 0x7fe2ac0cc890 2023-10-11T09:50:19.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.106+0000 7fe2b1151700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac0b4c70 con 0x7fe2ac0cfc30 2023-10-11T09:50:19.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.106+0000 7fe2b1151700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac0b4f50 con 0x7fe2ac06cda0 2023-10-11T09:50:19.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.110+0000 7fe2a37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2ac0cc890 0x7fe2ac06e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:19.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.110+0000 7fe2a37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2ac0cc890 0x7fe2ac06e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42882/0 (socket says 172.21.15.78:42882) 2023-10-11T09:50:19.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.110+0000 7fe2a37fe700 1 -- 172.21.15.78:0/2474335385 learned_addr learned my addr 172.21.15.78:0/2474335385 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:19.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.110+0000 7fe2abfff700 1 --2- 172.21.15.78:0/2474335385 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2ac06cda0 0x7fe2ac15a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:19.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.110+0000 7fe2b0950700 1 --2- 172.21.15.78:0/2474335385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2ac0cfc30 0x7fe2ac06e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:19.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.110+0000 7fe2a37fe700 1 -- 172.21.15.78:0/2474335385 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2ac06cda0 msgr2=0x7fe2ac15a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:19.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.110+0000 7fe2a37fe700 1 --2- 172.21.15.78:0/2474335385 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2ac06cda0 0x7fe2ac15a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:19.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.110+0000 7fe2a37fe700 1 -- 172.21.15.78:0/2474335385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2ac0cfc30 msgr2=0x7fe2ac06e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:19.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.110+0000 7fe2a37fe700 1 --2- 172.21.15.78:0/2474335385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2ac0cfc30 0x7fe2ac06e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:19.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.110+0000 7fe2a37fe700 1 -- 172.21.15.78:0/2474335385 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a4009b30 con 0x7fe2ac0cc890 2023-10-11T09:50:19.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.110+0000 7fe2abfff700 1 --2- 172.21.15.78:0/2474335385 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2ac06cda0 0x7fe2ac15a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:50:19.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.110+0000 7fe2b0950700 1 --2- 172.21.15.78:0/2474335385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2ac0cfc30 0x7fe2ac06e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:50:19.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.110+0000 7fe2a37fe700 1 --2- 172.21.15.78:0/2474335385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2ac0cc890 0x7fe2ac06e250 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fe29400c400 tx=0x7fe29400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:19.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.110+0000 7fe2a97fa700 1 -- 172.21.15.78:0/2474335385 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2940052d0 con 0x7fe2ac0cc890 2023-10-11T09:50:19.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.110+0000 7fe2a97fa700 1 -- 172.21.15.78:0/2474335385 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe294005450 con 0x7fe2ac0cc890 2023-10-11T09:50:19.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.110+0000 7fe2b1151700 1 -- 172.21.15.78:0/2474335385 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2ac0b5230 con 0x7fe2ac0cc890 2023-10-11T09:50:19.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.110+0000 7fe2a97fa700 1 -- 172.21.15.78:0/2474335385 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe294003930 con 0x7fe2ac0cc890 2023-10-11T09:50:19.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.110+0000 7fe2b1151700 1 -- 172.21.15.78:0/2474335385 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2ac0b57f0 con 0x7fe2ac0cc890 2023-10-11T09:50:19.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.110+0000 7fe2b1151700 1 -- 172.21.15.78:0/2474335385 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2ac065d40 con 0x7fe2ac0cc890 2023-10-11T09:50:19.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.114+0000 7fe2a97fa700 1 -- 172.21.15.78:0/2474335385 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fe294003a90 con 0x7fe2ac0cc890 2023-10-11T09:50:19.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.114+0000 7fe2a97fa700 1 --2- 172.21.15.78:0/2474335385 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe28c065ed0 0x7fe28c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:19.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.114+0000 7fe2a97fa700 1 -- 172.21.15.78:0/2474335385 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6466+0+0 (secure 0 0 0) 0x7fe294082d30 con 0x7fe2ac0cc890 2023-10-11T09:50:19.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.114+0000 7fe2abfff700 1 --2- 172.21.15.78:0/2474335385 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe28c065ed0 0x7fe28c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:19.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.118+0000 7fe2abfff700 1 --2- 172.21.15.78:0/2474335385 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe28c065ed0 0x7fe28c068380 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe29c009a10 tx=0x7fe29c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:19.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.122+0000 7fe2a97fa700 1 -- 172.21.15.78:0/2474335385 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fe2940501d0 con 0x7fe2ac0cc890 2023-10-11T09:50:19.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.298+0000 7fe2b1151700 1 -- 172.21.15.78:0/2474335385 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe2ac0b6df0 con 0x7fe2ac0cc890 2023-10-11T09:50:19.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.298+0000 7fe2a97fa700 1 -- 172.21.15.78:0/2474335385 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v108) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe2940170a0 con 0x7fe2ac0cc890 2023-10-11T09:50:19.306 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:50:19.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.302+0000 7fe2b1151700 1 -- 172.21.15.78:0/2474335385 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe28c065ed0 msgr2=0x7fe28c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:19.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.302+0000 7fe2b1151700 1 --2- 172.21.15.78:0/2474335385 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe28c065ed0 0x7fe28c068380 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe29c009a10 tx=0x7fe29c008040 comp rx=0 tx=0).stop 2023-10-11T09:50:19.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.302+0000 7fe2b1151700 1 -- 172.21.15.78:0/2474335385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2ac0cc890 msgr2=0x7fe2ac06e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:19.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.302+0000 7fe2b1151700 1 --2- 172.21.15.78:0/2474335385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2ac0cc890 0x7fe2ac06e250 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fe29400c400 tx=0x7fe29400c7c0 comp rx=0 tx=0).stop 2023-10-11T09:50:19.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.302+0000 7fe2b1151700 1 -- 172.21.15.78:0/2474335385 shutdown_connections 2023-10-11T09:50:19.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.302+0000 7fe2b1151700 1 --2- 172.21.15.78:0/2474335385 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe28c065ed0 0x7fe28c068380 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:19.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.302+0000 7fe2b1151700 1 --2- 172.21.15.78:0/2474335385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2ac0cfc30 0x7fe2ac06e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:19.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.302+0000 7fe2b1151700 1 --2- 172.21.15.78:0/2474335385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2ac0cc890 0x7fe2ac06e250 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:19.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.302+0000 7fe2b1151700 1 --2- 172.21.15.78:0/2474335385 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2ac06cda0 0x7fe2ac15a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:19.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.302+0000 7fe2b1151700 1 -- 172.21.15.78:0/2474335385 >> 172.21.15.78:0/2474335385 conn(0x7fe2ac06a060 msgr2=0x7fe2ac15b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:19.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.302+0000 7fe2b1151700 1 -- 172.21.15.78:0/2474335385 shutdown_connections 2023-10-11T09:50:19.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.302+0000 7fe2b1151700 1 -- 172.21.15.78:0/2474335385 wait complete. 2023-10-11T09:50:19.332 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T09:50:19.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.502+0000 7f3fb88f9700 1 Processor -- start 2023-10-11T09:50:19.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.502+0000 7f3fb88f9700 1 -- start start 2023-10-11T09:50:19.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.502+0000 7f3fb88f9700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f3fb40b3240 0x7f3fb40d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:19.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.502+0000 7f3fb88f9700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3fb40b3610 0x7f3fb40b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:19.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.502+0000 7f3fb88f9700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3fb415d040 0x7f3fb415d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:19.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.502+0000 7f3fb88f9700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fb40b3fd0 con 0x7f3fb40b3240 2023-10-11T09:50:19.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.506+0000 7f3fb88f9700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fb40b47f0 con 0x7f3fb40b3610 2023-10-11T09:50:19.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.506+0000 7f3fb88f9700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb40b8130 con 0x7f3fb415d040 2023-10-11T09:50:19.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.506+0000 7f3fb2d9d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f3fb40b3240 0x7f3fb40d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:19.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.506+0000 7f3fb2d9d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f3fb40b3240 0x7f3fb40d29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42886/0 (socket says 172.21.15.78:42886) 2023-10-11T09:50:19.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.506+0000 7f3fb2d9d700 1 -- 172.21.15.78:0/2267029977 learned_addr learned my addr 172.21.15.78:0/2267029977 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:19.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.506+0000 7f3fb359e700 1 --2- 172.21.15.78:0/2267029977 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3fb415d040 0x7f3fb415d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:19.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.506+0000 7f3fb259c700 1 --2- 172.21.15.78:0/2267029977 >> v2:172.21.15.133:3300/0 conn(0x7f3fb40b3610 0x7f3fb40b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:19.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.506+0000 7f3fb2d9d700 1 -- 172.21.15.78:0/2267029977 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3fb415d040 msgr2=0x7f3fb415d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:19.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.506+0000 7f3fb2d9d700 1 --2- 172.21.15.78:0/2267029977 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3fb415d040 0x7f3fb415d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:19.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.506+0000 7f3fb2d9d700 1 -- 172.21.15.78:0/2267029977 >> v2:172.21.15.133:3300/0 conn(0x7f3fb40b3610 msgr2=0x7f3fb40b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:19.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.506+0000 7f3fb2d9d700 1 --2- 172.21.15.78:0/2267029977 >> v2:172.21.15.133:3300/0 conn(0x7f3fb40b3610 0x7f3fb40b3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:19.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.506+0000 7f3fb2d9d700 1 -- 172.21.15.78:0/2267029977 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb40b41e0 con 0x7f3fb40b3240 2023-10-11T09:50:19.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.506+0000 7f3fb2d9d700 1 --2- 172.21.15.78:0/2267029977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fb40b3240 0x7f3fb40d29d0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f3fb40d0a90 tx=0x7f3fa4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa3aa95939d9288 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:19.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.506+0000 7f3fb1d9b700 1 -- 172.21.15.78:0/2267029977 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4011830 con 0x7f3fb40b3240 2023-10-11T09:50:19.517 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.506+0000 7f3fb1d9b700 1 -- 172.21.15.78:0/2267029977 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa4011990 con 0x7f3fb40b3240 2023-10-11T09:50:19.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.506+0000 7f3fb1d9b700 1 -- 172.21.15.78:0/2267029977 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4011d20 con 0x7f3fb40b3240 2023-10-11T09:50:19.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.506+0000 7f3fb88f9700 1 -- 172.21.15.78:0/2267029977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fb40b3240 msgr2=0x7f3fb40d29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:19.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.506+0000 7f3fb88f9700 1 --2- 172.21.15.78:0/2267029977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fb40b3240 0x7f3fb40d29d0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f3fb40d0a90 tx=0x7f3fa4009bf0 comp rx=0 tx=0).stop 2023-10-11T09:50:19.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.506+0000 7f3fb88f9700 1 -- 172.21.15.78:0/2267029977 shutdown_connections 2023-10-11T09:50:19.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.506+0000 7f3fb88f9700 1 --2- 172.21.15.78:0/2267029977 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3fb415d040 0x7f3fb415d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:19.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.506+0000 7f3fb88f9700 1 --2- 172.21.15.78:0/2267029977 >> v2:172.21.15.133:3300/0 conn(0x7f3fb40b3610 0x7f3fb40b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:19.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.506+0000 7f3fb88f9700 1 --2- 172.21.15.78:0/2267029977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fb40b3240 0x7f3fb40d29d0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:19.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.506+0000 7f3fb88f9700 1 -- 172.21.15.78:0/2267029977 >> 172.21.15.78:0/2267029977 conn(0x7f3fb409c260 msgr2=0x7f3fb40d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:19.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.506+0000 7f3fb88f9700 1 -- 172.21.15.78:0/2267029977 shutdown_connections 2023-10-11T09:50:19.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.506+0000 7f3fb88f9700 1 -- 172.21.15.78:0/2267029977 wait complete. 2023-10-11T09:50:19.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.510+0000 7f3fb88f9700 1 Processor -- start 2023-10-11T09:50:19.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.510+0000 7f3fb88f9700 1 -- start start 2023-10-11T09:50:19.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.510+0000 7f3fb88f9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fb40b3240 0x7f3fb4159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:19.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.510+0000 7f3fb88f9700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fb40b3610 0x7f3fb40c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:19.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.510+0000 7f3fb88f9700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3fb415d040 0x7f3fb415aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:19.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.510+0000 7f3fb88f9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb40c7b10 con 0x7f3fb40b3240 2023-10-11T09:50:19.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.510+0000 7f3fb88f9700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb40c6f00 con 0x7f3fb40b3610 2023-10-11T09:50:19.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.510+0000 7f3fb88f9700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb40c71e0 con 0x7f3fb415d040 2023-10-11T09:50:19.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.510+0000 7f3fb2d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fb40b3240 0x7f3fb4159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:19.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.510+0000 7f3fb2d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fb40b3240 0x7f3fb4159e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42888/0 (socket says 172.21.15.78:42888) 2023-10-11T09:50:19.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.510+0000 7f3fb2d9d700 1 -- 172.21.15.78:0/1472471827 learned_addr learned my addr 172.21.15.78:0/1472471827 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:19.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.510+0000 7f3fb359e700 1 --2- 172.21.15.78:0/1472471827 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3fb415d040 0x7f3fb415aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:19.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.510+0000 7f3fb259c700 1 --2- 172.21.15.78:0/1472471827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fb40b3610 0x7f3fb40c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:19.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.514+0000 7f3fb2d9d700 1 -- 172.21.15.78:0/1472471827 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3fb415d040 msgr2=0x7f3fb415aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:19.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.514+0000 7f3fb2d9d700 1 --2- 172.21.15.78:0/1472471827 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3fb415d040 0x7f3fb415aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:19.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.514+0000 7f3fb2d9d700 1 -- 172.21.15.78:0/1472471827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fb40b3610 msgr2=0x7f3fb40c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:19.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.514+0000 7f3fb2d9d700 1 --2- 172.21.15.78:0/1472471827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fb40b3610 0x7f3fb40c7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:19.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.514+0000 7f3fb2d9d700 1 -- 172.21.15.78:0/1472471827 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa4009570 con 0x7f3fb40b3240 2023-10-11T09:50:19.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.514+0000 7f3fb259c700 1 --2- 172.21.15.78:0/1472471827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fb40b3610 0x7f3fb40c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:50:19.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.514+0000 7f3fb2d9d700 1 --2- 172.21.15.78:0/1472471827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fb40b3240 0x7f3fb4159e20 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4000c00 tx=0x7f3fa4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:19.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.514+0000 7f3f9b7fe700 1 -- 172.21.15.78:0/1472471827 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4004450 con 0x7f3fb40b3240 2023-10-11T09:50:19.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.514+0000 7f3f9b7fe700 1 -- 172.21.15.78:0/1472471827 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa401e070 con 0x7f3fb40b3240 2023-10-11T09:50:19.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.514+0000 7f3fb88f9700 1 -- 172.21.15.78:0/1472471827 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fb40c74c0 con 0x7f3fb40b3240 2023-10-11T09:50:19.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.514+0000 7f3f9b7fe700 1 -- 172.21.15.78:0/1472471827 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4003900 con 0x7f3fb40b3240 2023-10-11T09:50:19.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.514+0000 7f3fb88f9700 1 -- 172.21.15.78:0/1472471827 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fb415a360 con 0x7f3fb40b3240 2023-10-11T09:50:19.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.514+0000 7f3fb88f9700 1 -- 172.21.15.78:0/1472471827 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fb4065d40 con 0x7f3fb40b3240 2023-10-11T09:50:19.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.514+0000 7f3f9b7fe700 1 -- 172.21.15.78:0/1472471827 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f3fa4003a60 con 0x7f3fb40b3240 2023-10-11T09:50:19.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.518+0000 7f3f9b7fe700 1 --2- 172.21.15.78:0/1472471827 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3fa0065ed0 0x7f3fa0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:19.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.518+0000 7f3f9b7fe700 1 -- 172.21.15.78:0/1472471827 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f3fa4084530 con 0x7f3fb40b3240 2023-10-11T09:50:19.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.518+0000 7f3fb259c700 1 --2- 172.21.15.78:0/1472471827 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3fa0065ed0 0x7f3fa0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:19.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.518+0000 7f3fb259c700 1 --2- 172.21.15.78:0/1472471827 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3fa0065ed0 0x7f3fa0068380 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f3fb40d1650 tx=0x7f3fa8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:19.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.522+0000 7f3f9b7fe700 1 -- 172.21.15.78:0/1472471827 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f3fa4051a70 con 0x7f3fb40b3240 2023-10-11T09:50:19.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.698+0000 7f3fb88f9700 1 -- 172.21.15.78:0/1472471827 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3fb40d3660 con 0x7f3fb40b3240 2023-10-11T09:50:19.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.702+0000 7f3f9b7fe700 1 -- 172.21.15.78:0/1472471827 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v108) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3fa40045d0 con 0x7f3fb40b3240 2023-10-11T09:50:19.709 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:50:19.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.702+0000 7f3fb88f9700 1 -- 172.21.15.78:0/1472471827 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3fa0065ed0 msgr2=0x7f3fa0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:19.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.702+0000 7f3fb88f9700 1 --2- 172.21.15.78:0/1472471827 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3fa0065ed0 0x7f3fa0068380 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f3fb40d1650 tx=0x7f3fa8008040 comp rx=0 tx=0).stop 2023-10-11T09:50:19.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.702+0000 7f3fb88f9700 1 -- 172.21.15.78:0/1472471827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fb40b3240 msgr2=0x7f3fb4159e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:19.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.702+0000 7f3fb88f9700 1 --2- 172.21.15.78:0/1472471827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fb40b3240 0x7f3fb4159e20 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4000c00 tx=0x7f3fa4004770 comp rx=0 tx=0).stop 2023-10-11T09:50:19.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.702+0000 7f3fb88f9700 1 -- 172.21.15.78:0/1472471827 shutdown_connections 2023-10-11T09:50:19.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.702+0000 7f3fb88f9700 1 --2- 172.21.15.78:0/1472471827 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3fa0065ed0 0x7f3fa0068380 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:19.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.702+0000 7f3fb88f9700 1 --2- 172.21.15.78:0/1472471827 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3fb415d040 0x7f3fb415aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:19.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.702+0000 7f3fb88f9700 1 --2- 172.21.15.78:0/1472471827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fb40b3610 0x7f3fb40c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:19.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.702+0000 7f3fb88f9700 1 --2- 172.21.15.78:0/1472471827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fb40b3240 0x7f3fb4159e20 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:19.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.702+0000 7f3fb88f9700 1 -- 172.21.15.78:0/1472471827 >> 172.21.15.78:0/1472471827 conn(0x7f3fb409c260 msgr2=0x7f3fb415c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:19.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.702+0000 7f3fb88f9700 1 -- 172.21.15.78:0/1472471827 shutdown_connections 2023-10-11T09:50:19.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.702+0000 7f3fb88f9700 1 -- 172.21.15.78:0/1472471827 wait complete. 2023-10-11T09:50:19.732 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T09:50:19.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.898+0000 7fdd01616700 1 Processor -- start 2023-10-11T09:50:19.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.898+0000 7fdd01616700 1 -- start start 2023-10-11T09:50:19.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.902+0000 7fdd01616700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdcfc158430 0x7fdcfc0710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:19.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.902+0000 7fdd01616700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fdcfc0caf90 0x7fdcfc15afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:19.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.902+0000 7fdd01616700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcfc071610 con 0x7fdcfc06d340 2023-10-11T09:50:19.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.902+0000 7fdd01616700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcfc070c10 con 0x7fdcfc0caf90 2023-10-11T09:50:19.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.902+0000 7fdd01616700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc15e280 con 0x7fdcfc158430 2023-10-11T09:50:19.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.902+0000 7fdcfa7fc700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fdcfc06d340 0x7fdcfc0ca880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:57470/0 (socket says 172.21.15.78:57470) 2023-10-11T09:50:19.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.902+0000 7fdcfa7fc700 1 -- 172.21.15.78:0/380351667 learned_addr learned my addr 172.21.15.78:0/380351667 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:50:19.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.902+0000 7fdcfb7fe700 1 --2- 172.21.15.78:0/380351667 >> v2:172.21.15.78:3300/0 conn(0x7fdcfc0caf90 0x7fdcfc15afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:19.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.902+0000 7fdcfaffd700 1 --2- 172.21.15.78:0/380351667 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdcfc158430 0x7fdcfc0710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:19.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.902+0000 7fdcf9ffb700 1 -- 172.21.15.78:0/380351667 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2437569149 0 0) 0x7fdcfc071610 con 0x7fdcfc06d340 2023-10-11T09:50:19.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.902+0000 7fdcf9ffb700 1 -- 172.21.15.78:0/380351667 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdce40035d0 con 0x7fdcfc06d340 2023-10-11T09:50:19.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.902+0000 7fdcfb7fe700 1 -- 172.21.15.78:0/380351667 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdcfc158430 msgr2=0x7fdcfc0710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:19.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.902+0000 7fdcfb7fe700 1 --2- 172.21.15.78:0/380351667 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdcfc158430 0x7fdcfc0710d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:19.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.902+0000 7fdcfb7fe700 1 -- 172.21.15.78:0/380351667 >> v1:172.21.15.78:6789/0 conn(0x7fdcfc06d340 legacy=0x7fdcfc0ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:19.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.902+0000 7fdcfb7fe700 1 -- 172.21.15.78:0/380351667 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcfc15e560 con 0x7fdcfc0caf90 2023-10-11T09:50:19.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.906+0000 7fdcfb7fe700 1 --2- 172.21.15.78:0/380351667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcfc0caf90 0x7fdcfc15afe0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4009d90 tx=0x7fdcf400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17cf181bdb618d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:19.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.906+0000 7fdcf9ffb700 1 -- 172.21.15.78:0/380351667 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf400e070 con 0x7fdcfc0caf90 2023-10-11T09:50:19.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.906+0000 7fdcf9ffb700 1 -- 172.21.15.78:0/380351667 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf400bb20 con 0x7fdcfc0caf90 2023-10-11T09:50:19.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.906+0000 7fdcf9ffb700 1 -- 172.21.15.78:0/380351667 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf4012430 con 0x7fdcfc0caf90 2023-10-11T09:50:19.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.906+0000 7fdd01616700 1 -- 172.21.15.78:0/380351667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcfc0caf90 msgr2=0x7fdcfc15afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:19.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.906+0000 7fdd01616700 1 --2- 172.21.15.78:0/380351667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcfc0caf90 0x7fdcfc15afe0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4009d90 tx=0x7fdcf400b330 comp rx=0 tx=0).stop 2023-10-11T09:50:19.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.906+0000 7fdd01616700 1 -- 172.21.15.78:0/380351667 shutdown_connections 2023-10-11T09:50:19.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.906+0000 7fdd01616700 1 --2- 172.21.15.78:0/380351667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcfc0caf90 0x7fdcfc15afe0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:19.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.906+0000 7fdd01616700 1 --2- 172.21.15.78:0/380351667 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdcfc158430 0x7fdcfc0710d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:19.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.906+0000 7fdd01616700 1 -- 172.21.15.78:0/380351667 >> 172.21.15.78:0/380351667 conn(0x7fdcfc0cecc0 msgr2=0x7fdcfc0cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:19.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.906+0000 7fdd01616700 1 -- 172.21.15.78:0/380351667 shutdown_connections 2023-10-11T09:50:19.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.906+0000 7fdd01616700 1 -- 172.21.15.78:0/380351667 wait complete. 2023-10-11T09:50:19.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.910+0000 7fdd01616700 1 Processor -- start 2023-10-11T09:50:19.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.910+0000 7fdd01616700 1 -- start start 2023-10-11T09:50:19.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.910+0000 7fdd01616700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcfc06d340 0x7fdcfc156c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:19.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.910+0000 7fdd01616700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcfc0caf90 0x7fdcfc0cda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:19.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.914+0000 7fdd01616700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdcfc158430 0x7fdcfc0b0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:19.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.914+0000 7fdd01616700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc15cf40 con 0x7fdcfc06d340 2023-10-11T09:50:19.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.914+0000 7fdd01616700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc15c3c0 con 0x7fdcfc0caf90 2023-10-11T09:50:19.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.914+0000 7fdd01616700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc15c6a0 con 0x7fdcfc158430 2023-10-11T09:50:19.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.914+0000 7fdcfaffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcfc06d340 0x7fdcfc156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:19.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.914+0000 7fdcfaffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcfc06d340 0x7fdcfc156c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42900/0 (socket says 172.21.15.78:42900) 2023-10-11T09:50:19.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.914+0000 7fdcfaffd700 1 -- 172.21.15.78:0/3317928726 learned_addr learned my addr 172.21.15.78:0/3317928726 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:19.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.914+0000 7fdcfb7fe700 1 --2- 172.21.15.78:0/3317928726 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdcfc158430 0x7fdcfc0b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:19.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.914+0000 7fdcfa7fc700 1 --2- 172.21.15.78:0/3317928726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcfc0caf90 0x7fdcfc0cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:19.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.914+0000 7fdcfaffd700 1 -- 172.21.15.78:0/3317928726 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdcfc158430 msgr2=0x7fdcfc0b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:19.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.914+0000 7fdcfaffd700 1 --2- 172.21.15.78:0/3317928726 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdcfc158430 0x7fdcfc0b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:19.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.914+0000 7fdcfaffd700 1 -- 172.21.15.78:0/3317928726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcfc0caf90 msgr2=0x7fdcfc0cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:19.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.914+0000 7fdcfaffd700 1 --2- 172.21.15.78:0/3317928726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcfc0caf90 0x7fdcfc0cda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:19.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.914+0000 7fdcfaffd700 1 -- 172.21.15.78:0/3317928726 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf4009a70 con 0x7fdcfc06d340 2023-10-11T09:50:19.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.914+0000 7fdcfa7fc700 1 --2- 172.21.15.78:0/3317928726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcfc0caf90 0x7fdcfc0cda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:50:19.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.914+0000 7fdcfaffd700 1 --2- 172.21.15.78:0/3317928726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcfc06d340 0x7fdcfc156c60 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fdcec00e840 tx=0x7fdcec00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:19.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.914+0000 7fdce37fe700 1 -- 172.21.15.78:0/3317928726 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec0095a0 con 0x7fdcfc06d340 2023-10-11T09:50:19.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.914+0000 7fdce37fe700 1 -- 172.21.15.78:0/3317928726 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcec00c690 con 0x7fdcfc06d340 2023-10-11T09:50:19.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.914+0000 7fdd01616700 1 -- 172.21.15.78:0/3317928726 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcfc15c980 con 0x7fdcfc06d340 2023-10-11T09:50:19.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.914+0000 7fdce37fe700 1 -- 172.21.15.78:0/3317928726 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec00ca00 con 0x7fdcfc06d340 2023-10-11T09:50:19.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.914+0000 7fdd01616700 1 -- 172.21.15.78:0/3317928726 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcfc1571a0 con 0x7fdcfc06d340 2023-10-11T09:50:19.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.914+0000 7fdd01616700 1 -- 172.21.15.78:0/3317928726 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcfc06eb80 con 0x7fdcfc06d340 2023-10-11T09:50:19.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.918+0000 7fdce37fe700 1 -- 172.21.15.78:0/3317928726 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fdcec00cb60 con 0x7fdcfc06d340 2023-10-11T09:50:19.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.918+0000 7fdce37fe700 1 --2- 172.21.15.78:0/3317928726 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdce8065f00 0x7fdce80683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:19.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.918+0000 7fdce37fe700 1 -- 172.21.15.78:0/3317928726 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6466+0+0 (secure 0 0 0) 0x7fdcec0841b0 con 0x7fdcfc06d340 2023-10-11T09:50:19.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.918+0000 7fdcfa7fc700 1 --2- 172.21.15.78:0/3317928726 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdce8065f00 0x7fdce80683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:19.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.922+0000 7fdcfa7fc700 1 --2- 172.21.15.78:0/3317928726 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdce8065f00 0x7fdce80683b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc0cce40 tx=0x7fdcf0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:19.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:19.926+0000 7fdce37fe700 1 -- 172.21.15.78:0/3317928726 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fdcec0515b0 con 0x7fdcfc06d340 2023-10-11T09:50:20.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.098+0000 7fdd01616700 1 -- 172.21.15.78:0/3317928726 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdcfc065db0 con 0x7fdcfc06d340 2023-10-11T09:50:20.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.098+0000 7fdce37fe700 1 -- 172.21.15.78:0/3317928726 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v108) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdcec0170a0 con 0x7fdcfc06d340 2023-10-11T09:50:20.106 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2023-10-11T09:50:20.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.102+0000 7fdd01616700 1 -- 172.21.15.78:0/3317928726 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdce8065f00 msgr2=0x7fdce80683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:20.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.102+0000 7fdd01616700 1 --2- 172.21.15.78:0/3317928726 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdce8065f00 0x7fdce80683b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc0cce40 tx=0x7fdcf0009040 comp rx=0 tx=0).stop 2023-10-11T09:50:20.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.102+0000 7fdd01616700 1 -- 172.21.15.78:0/3317928726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcfc06d340 msgr2=0x7fdcfc156c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:20.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.102+0000 7fdd01616700 1 --2- 172.21.15.78:0/3317928726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcfc06d340 0x7fdcfc156c60 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fdcec00e840 tx=0x7fdcec00ec00 comp rx=0 tx=0).stop 2023-10-11T09:50:20.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.102+0000 7fdd01616700 1 -- 172.21.15.78:0/3317928726 shutdown_connections 2023-10-11T09:50:20.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.102+0000 7fdd01616700 1 --2- 172.21.15.78:0/3317928726 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fdce8065f00 0x7fdce80683b0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:20.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.102+0000 7fdd01616700 1 --2- 172.21.15.78:0/3317928726 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdcfc158430 0x7fdcfc0b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:20.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.102+0000 7fdd01616700 1 --2- 172.21.15.78:0/3317928726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcfc0caf90 0x7fdcfc0cda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:20.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.102+0000 7fdd01616700 1 --2- 172.21.15.78:0/3317928726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcfc06d340 0x7fdcfc156c60 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:20.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.102+0000 7fdd01616700 1 -- 172.21.15.78:0/3317928726 >> 172.21.15.78:0/3317928726 conn(0x7fdcfc0cecc0 msgr2=0x7fdcfc0ca880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:20.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.102+0000 7fdd01616700 1 -- 172.21.15.78:0/3317928726 shutdown_connections 2023-10-11T09:50:20.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.102+0000 7fdd01616700 1 -- 172.21.15.78:0/3317928726 wait complete. 2023-10-11T09:50:20.127 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-10-11T09:50:20.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.298+0000 7f1ca92cf700 1 Processor -- start 2023-10-11T09:50:20.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.298+0000 7f1ca92cf700 1 -- start start 2023-10-11T09:50:20.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.298+0000 7f1ca92cf700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1ca4158430 0x7f1ca40710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:20.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.298+0000 7f1ca92cf700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1ca40caf90 0x7f1ca415afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:20.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.298+0000 7f1ca92cf700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ca4071610 con 0x7f1ca406d340 2023-10-11T09:50:20.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.298+0000 7f1ca92cf700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ca4070c10 con 0x7f1ca40caf90 2023-10-11T09:50:20.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.298+0000 7f1ca92cf700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca415e280 con 0x7f1ca4158430 2023-10-11T09:50:20.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.298+0000 7f1ca27fc700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f1ca406d340 0x7f1ca40ca880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:57484/0 (socket says 172.21.15.78:57484) 2023-10-11T09:50:20.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.298+0000 7f1ca27fc700 1 -- 172.21.15.78:0/2445408373 learned_addr learned my addr 172.21.15.78:0/2445408373 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:50:20.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.302+0000 7f1ca2ffd700 1 --2- 172.21.15.78:0/2445408373 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1ca4158430 0x7f1ca40710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:20.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.302+0000 7f1ca37fe700 1 --2- 172.21.15.78:0/2445408373 >> v2:172.21.15.133:3300/0 conn(0x7f1ca40caf90 0x7f1ca415afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:20.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.302+0000 7f1ca1ffb700 1 -- 172.21.15.78:0/2445408373 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2492050371 0 0) 0x7f1ca4071610 con 0x7f1ca406d340 2023-10-11T09:50:20.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.302+0000 7f1ca1ffb700 1 -- 172.21.15.78:0/2445408373 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c8c0035d0 con 0x7f1ca406d340 2023-10-11T09:50:20.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.302+0000 7f1ca37fe700 1 -- 172.21.15.78:0/2445408373 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1ca4158430 msgr2=0x7f1ca40710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:20.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.302+0000 7f1ca37fe700 1 --2- 172.21.15.78:0/2445408373 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1ca4158430 0x7f1ca40710d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:20.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.302+0000 7f1ca37fe700 1 -- 172.21.15.78:0/2445408373 >> v1:172.21.15.78:6789/0 conn(0x7f1ca406d340 legacy=0x7f1ca40ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:20.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.302+0000 7f1ca37fe700 1 -- 172.21.15.78:0/2445408373 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca415e560 con 0x7f1ca40caf90 2023-10-11T09:50:20.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.302+0000 7f1ca37fe700 1 --2- 172.21.15.78:0/2445408373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ca40caf90 0x7f1ca415afe0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c009d90 tx=0x7f1c9c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=151364e9843eff44 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:20.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.302+0000 7f1ca1ffb700 1 -- 172.21.15.78:0/2445408373 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c00e070 con 0x7f1ca40caf90 2023-10-11T09:50:20.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.302+0000 7f1ca1ffb700 1 -- 172.21.15.78:0/2445408373 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9c00bb20 con 0x7f1ca40caf90 2023-10-11T09:50:20.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.302+0000 7f1ca1ffb700 1 -- 172.21.15.78:0/2445408373 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c012430 con 0x7f1ca40caf90 2023-10-11T09:50:20.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.302+0000 7f1ca92cf700 1 -- 172.21.15.78:0/2445408373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ca40caf90 msgr2=0x7f1ca415afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:20.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.302+0000 7f1ca92cf700 1 --2- 172.21.15.78:0/2445408373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ca40caf90 0x7f1ca415afe0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c009d90 tx=0x7f1c9c00b330 comp rx=0 tx=0).stop 2023-10-11T09:50:20.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.302+0000 7f1ca92cf700 1 -- 172.21.15.78:0/2445408373 shutdown_connections 2023-10-11T09:50:20.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.302+0000 7f1ca92cf700 1 --2- 172.21.15.78:0/2445408373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ca40caf90 0x7f1ca415afe0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:20.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.302+0000 7f1ca92cf700 1 --2- 172.21.15.78:0/2445408373 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1ca4158430 0x7f1ca40710d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:20.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.306+0000 7f1ca92cf700 1 -- 172.21.15.78:0/2445408373 >> 172.21.15.78:0/2445408373 conn(0x7f1ca40cecc0 msgr2=0x7f1ca40cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:20.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.306+0000 7f1ca92cf700 1 -- 172.21.15.78:0/2445408373 shutdown_connections 2023-10-11T09:50:20.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.306+0000 7f1ca92cf700 1 -- 172.21.15.78:0/2445408373 wait complete. 2023-10-11T09:50:20.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.310+0000 7f1ca92cf700 1 Processor -- start 2023-10-11T09:50:20.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.310+0000 7f1ca92cf700 1 -- start start 2023-10-11T09:50:20.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.310+0000 7f1ca92cf700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1ca406d340 0x7f1ca415bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:20.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.310+0000 7f1ca92cf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ca40caf90 0x7f1ca415aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:20.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.310+0000 7f1ca92cf700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ca4158430 0x7f1ca4159650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:20.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.310+0000 7f1ca92cf700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca414e180 con 0x7f1ca40caf90 2023-10-11T09:50:20.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.310+0000 7f1ca92cf700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca40cc800 con 0x7f1ca4158430 2023-10-11T09:50:20.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.310+0000 7f1ca92cf700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca40b0730 con 0x7f1ca406d340 2023-10-11T09:50:20.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.310+0000 7f1ca27fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ca40caf90 0x7f1ca415aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:20.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.310+0000 7f1ca27fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ca40caf90 0x7f1ca415aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42906/0 (socket says 172.21.15.78:42906) 2023-10-11T09:50:20.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.310+0000 7f1ca27fc700 1 -- 172.21.15.78:0/3217847070 learned_addr learned my addr 172.21.15.78:0/3217847070 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:20.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.310+0000 7f1ca37fe700 1 --2- 172.21.15.78:0/3217847070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ca4158430 0x7f1ca4159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:20.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.310+0000 7f1ca2ffd700 1 --2- 172.21.15.78:0/3217847070 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1ca406d340 0x7f1ca415bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:20.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.310+0000 7f1ca27fc700 1 -- 172.21.15.78:0/3217847070 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1ca406d340 msgr2=0x7f1ca415bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:20.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.310+0000 7f1ca27fc700 1 --2- 172.21.15.78:0/3217847070 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1ca406d340 0x7f1ca415bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:20.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.310+0000 7f1ca27fc700 1 -- 172.21.15.78:0/3217847070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ca4158430 msgr2=0x7f1ca4159650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:20.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.310+0000 7f1ca27fc700 1 --2- 172.21.15.78:0/3217847070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ca4158430 0x7f1ca4159650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:20.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.310+0000 7f1ca27fc700 1 -- 172.21.15.78:0/3217847070 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c9c009a70 con 0x7f1ca40caf90 2023-10-11T09:50:20.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.310+0000 7f1ca2ffd700 1 --2- 172.21.15.78:0/3217847070 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1ca406d340 0x7f1ca415bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:50:20.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.310+0000 7f1ca27fc700 1 --2- 172.21.15.78:0/3217847070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ca40caf90 0x7f1ca415aee0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f1c9800c4f0 tx=0x7f1c9800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:20.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.310+0000 7f1c8b7fe700 1 -- 172.21.15.78:0/3217847070 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c98005d90 con 0x7f1ca40caf90 2023-10-11T09:50:20.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.310+0000 7f1c8b7fe700 1 -- 172.21.15.78:0/3217847070 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c980070d0 con 0x7f1ca40caf90 2023-10-11T09:50:20.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.310+0000 7f1ca92cf700 1 -- 172.21.15.78:0/3217847070 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ca40b0a10 con 0x7f1ca40caf90 2023-10-11T09:50:20.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.310+0000 7f1c8b7fe700 1 -- 172.21.15.78:0/3217847070 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c98007440 con 0x7f1ca40caf90 2023-10-11T09:50:20.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.310+0000 7f1ca92cf700 1 -- 172.21.15.78:0/3217847070 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ca40b0fd0 con 0x7f1ca40caf90 2023-10-11T09:50:20.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.314+0000 7f1ca92cf700 1 -- 172.21.15.78:0/3217847070 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ca40b1160 con 0x7f1ca40caf90 2023-10-11T09:50:20.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.314+0000 7f1c8b7fe700 1 -- 172.21.15.78:0/3217847070 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f1c980075a0 con 0x7f1ca40caf90 2023-10-11T09:50:20.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.318+0000 7f1c8b7fe700 1 --2- 172.21.15.78:0/3217847070 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1c90065ed0 0x7f1c90068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:20.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.318+0000 7f1c8b7fe700 1 -- 172.21.15.78:0/3217847070 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f1c98082e50 con 0x7f1ca40caf90 2023-10-11T09:50:20.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.318+0000 7f1ca2ffd700 1 --2- 172.21.15.78:0/3217847070 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1c90065ed0 0x7f1c90068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:20.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.318+0000 7f1ca2ffd700 1 --2- 172.21.15.78:0/3217847070 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1c90065ed0 0x7f1c90068380 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f1c94009ad0 tx=0x7f1c94008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:20.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.326+0000 7f1c8b7fe700 1 -- 172.21.15.78:0/3217847070 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f1c98050320 con 0x7f1ca40caf90 2023-10-11T09:50:20.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.498+0000 7f1ca92cf700 1 -- 172.21.15.78:0/3217847070 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7f1ca40cc9c0 con 0x7f1ca40caf90 2023-10-11T09:50:20.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.498+0000 7f1c8b7fe700 1 -- 172.21.15.78:0/3217847070 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v108) v1 ==== 106+0+11 (secure 0 0 0) 0x7f1c980170b0 con 0x7f1ca40caf90 2023-10-11T09:50:20.508 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:50:20.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.502+0000 7f1ca92cf700 1 -- 172.21.15.78:0/3217847070 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1c90065ed0 msgr2=0x7f1c90068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:20.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.502+0000 7f1ca92cf700 1 --2- 172.21.15.78:0/3217847070 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1c90065ed0 0x7f1c90068380 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f1c94009ad0 tx=0x7f1c94008040 comp rx=0 tx=0).stop 2023-10-11T09:50:20.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.502+0000 7f1ca92cf700 1 -- 172.21.15.78:0/3217847070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ca40caf90 msgr2=0x7f1ca415aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:20.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.502+0000 7f1ca92cf700 1 --2- 172.21.15.78:0/3217847070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ca40caf90 0x7f1ca415aee0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f1c9800c4f0 tx=0x7f1c9800c8b0 comp rx=0 tx=0).stop 2023-10-11T09:50:20.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.502+0000 7f1ca92cf700 1 -- 172.21.15.78:0/3217847070 shutdown_connections 2023-10-11T09:50:20.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.502+0000 7f1ca92cf700 1 --2- 172.21.15.78:0/3217847070 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1c90065ed0 0x7f1c90068380 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:20.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.502+0000 7f1ca92cf700 1 --2- 172.21.15.78:0/3217847070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ca4158430 0x7f1ca4159650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:20.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.502+0000 7f1ca92cf700 1 --2- 172.21.15.78:0/3217847070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ca40caf90 0x7f1ca415aee0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:20.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.502+0000 7f1ca92cf700 1 --2- 172.21.15.78:0/3217847070 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1ca406d340 0x7f1ca415bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:20.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.502+0000 7f1ca92cf700 1 -- 172.21.15.78:0/3217847070 >> 172.21.15.78:0/3217847070 conn(0x7f1ca40cecc0 msgr2=0x7f1ca40ca880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:20.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.502+0000 7f1ca92cf700 1 -- 172.21.15.78:0/3217847070 shutdown_connections 2023-10-11T09:50:20.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.502+0000 7f1ca92cf700 1 -- 172.21.15.78:0/3217847070 wait complete. 2023-10-11T09:50:20.531 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T09:50:20.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.702+0000 7f5539819700 1 Processor -- start 2023-10-11T09:50:20.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.702+0000 7f5539819700 1 -- start start 2023-10-11T09:50:20.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.702+0000 7f5539819700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f553415b8e0 0x7f553406d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:20.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.702+0000 7f5539819700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f553406d610 0x7f55340ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:20.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.702+0000 7f5539819700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f553415a940 0x7f55340ca780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:20.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.702+0000 7f5539819700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f553415e010 con 0x7f553415a940 2023-10-11T09:50:20.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.702+0000 7f5539819700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f553415e280 con 0x7f553415b8e0 2023-10-11T09:50:20.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.702+0000 7f5539819700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f553415e560 con 0x7f553406d610 2023-10-11T09:50:20.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.702+0000 7f55337fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f553415a940 0x7f55340ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:20.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.702+0000 7f55337fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f553415a940 0x7f55340ca780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42920/0 (socket says 172.21.15.78:42920) 2023-10-11T09:50:20.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.702+0000 7f55337fe700 1 -- 172.21.15.78:0/1519798703 learned_addr learned my addr 172.21.15.78:0/1519798703 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:20.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.702+0000 7f5532ffd700 1 --2- 172.21.15.78:0/1519798703 >> v2:172.21.15.133:3300/0 conn(0x7f553415b8e0 0x7f553406d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:20.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.702+0000 7f552a7fc700 1 --2- 172.21.15.78:0/1519798703 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f553406d610 0x7f55340ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:20.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.702+0000 7f55337fe700 1 -- 172.21.15.78:0/1519798703 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f553406d610 msgr2=0x7f55340ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:20.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.702+0000 7f55337fe700 1 --2- 172.21.15.78:0/1519798703 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f553406d610 0x7f55340ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:20.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.702+0000 7f55337fe700 1 -- 172.21.15.78:0/1519798703 >> v2:172.21.15.133:3300/0 conn(0x7f553415b8e0 msgr2=0x7f553406d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:20.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.702+0000 7f55337fe700 1 --2- 172.21.15.78:0/1519798703 >> v2:172.21.15.133:3300/0 conn(0x7f553415b8e0 0x7f553406d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:20.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.702+0000 7f55337fe700 1 -- 172.21.15.78:0/1519798703 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f553415e840 con 0x7f553415a940 2023-10-11T09:50:20.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.706+0000 7f55337fe700 1 --2- 172.21.15.78:0/1519798703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f553415a940 0x7f55340ca780 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f55340b4cd0 tx=0x7f552c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1cc077c4cd2a9a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:20.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.706+0000 7f55327fc700 1 -- 172.21.15.78:0/1519798703 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552c00e070 con 0x7f553415a940 2023-10-11T09:50:20.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.706+0000 7f55327fc700 1 -- 172.21.15.78:0/1519798703 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f552c00b910 con 0x7f553415a940 2023-10-11T09:50:20.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.706+0000 7f55327fc700 1 -- 172.21.15.78:0/1519798703 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552c00bcc0 con 0x7f553415a940 2023-10-11T09:50:20.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.706+0000 7f5539819700 1 -- 172.21.15.78:0/1519798703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f553415a940 msgr2=0x7f55340ca780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:20.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.706+0000 7f5539819700 1 --2- 172.21.15.78:0/1519798703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f553415a940 0x7f55340ca780 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f55340b4cd0 tx=0x7f552c00b330 comp rx=0 tx=0).stop 2023-10-11T09:50:20.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.706+0000 7f5539819700 1 -- 172.21.15.78:0/1519798703 shutdown_connections 2023-10-11T09:50:20.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.706+0000 7f5539819700 1 --2- 172.21.15.78:0/1519798703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f553415a940 0x7f55340ca780 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:20.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.706+0000 7f5539819700 1 --2- 172.21.15.78:0/1519798703 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f553406d610 0x7f55340ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:20.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.706+0000 7f5539819700 1 --2- 172.21.15.78:0/1519798703 >> v2:172.21.15.133:3300/0 conn(0x7f553415b8e0 0x7f553406d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:20.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.706+0000 7f5539819700 1 -- 172.21.15.78:0/1519798703 >> 172.21.15.78:0/1519798703 conn(0x7f55340cea60 msgr2=0x7f55340cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:20.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.706+0000 7f5539819700 1 -- 172.21.15.78:0/1519798703 shutdown_connections 2023-10-11T09:50:20.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.706+0000 7f5539819700 1 -- 172.21.15.78:0/1519798703 wait complete. 2023-10-11T09:50:20.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.710+0000 7f5539819700 1 Processor -- start 2023-10-11T09:50:20.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.710+0000 7f5539819700 1 -- start start 2023-10-11T09:50:20.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.710+0000 7f5539819700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f553406d610 0x7f553413f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:20.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.710+0000 7f5539819700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f553415a940 0x7f55340b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:20.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.710+0000 7f5539819700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f553415b8e0 0x7f55340b4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:20.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.710+0000 7f5539819700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553413faf0 con 0x7f553415a940 2023-10-11T09:50:20.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.710+0000 7f5539819700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553413fdd0 con 0x7f553415b8e0 2023-10-11T09:50:20.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.710+0000 7f5539819700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55341400b0 con 0x7f553406d610 2023-10-11T09:50:20.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.710+0000 7f552a7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f553415a940 0x7f55340b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:20.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.710+0000 7f5532ffd700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f553406d610 0x7f553413f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:20.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.710+0000 7f552a7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f553415a940 0x7f55340b3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42924/0 (socket says 172.21.15.78:42924) 2023-10-11T09:50:20.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.710+0000 7f55337fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f553415b8e0 0x7f55340b4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:20.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.710+0000 7f552a7fc700 1 -- 172.21.15.78:0/205847895 learned_addr learned my addr 172.21.15.78:0/205847895 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:20.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.710+0000 7f552a7fc700 1 -- 172.21.15.78:0/205847895 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f553406d610 msgr2=0x7f553413f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:20.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.710+0000 7f552a7fc700 1 --2- 172.21.15.78:0/205847895 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f553406d610 0x7f553413f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:20.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.710+0000 7f552a7fc700 1 -- 172.21.15.78:0/205847895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f553415b8e0 msgr2=0x7f55340b4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:20.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.710+0000 7f552a7fc700 1 --2- 172.21.15.78:0/205847895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f553415b8e0 0x7f55340b4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:20.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.710+0000 7f552a7fc700 1 -- 172.21.15.78:0/205847895 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f552c009b30 con 0x7f553415a940 2023-10-11T09:50:20.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.710+0000 7f552a7fc700 1 --2- 172.21.15.78:0/205847895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f553415a940 0x7f55340b3be0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f551c00e870 tx=0x7f551c00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:20.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.714+0000 7f5530ff9700 1 -- 172.21.15.78:0/205847895 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551c0095c0 con 0x7f553415a940 2023-10-11T09:50:20.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.714+0000 7f5530ff9700 1 -- 172.21.15.78:0/205847895 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551c013680 con 0x7f553415a940 2023-10-11T09:50:20.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.714+0000 7f5530ff9700 1 -- 172.21.15.78:0/205847895 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551c013a10 con 0x7f553415a940 2023-10-11T09:50:20.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.714+0000 7f5539819700 1 -- 172.21.15.78:0/205847895 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5534140390 con 0x7f553415a940 2023-10-11T09:50:20.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.714+0000 7f5539819700 1 -- 172.21.15.78:0/205847895 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f553413f750 con 0x7f553415a940 2023-10-11T09:50:20.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.714+0000 7f5539819700 1 -- 172.21.15.78:0/205847895 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5534065d40 con 0x7f553415a940 2023-10-11T09:50:20.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.714+0000 7f5530ff9700 1 -- 172.21.15.78:0/205847895 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f551c013b90 con 0x7f553415a940 2023-10-11T09:50:20.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.714+0000 7f5530ff9700 1 --2- 172.21.15.78:0/205847895 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5514065ed0 0x7f5514068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:20.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.714+0000 7f5530ff9700 1 -- 172.21.15.78:0/205847895 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f551c0838a0 con 0x7f553415a940 2023-10-11T09:50:20.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.718+0000 7f5532ffd700 1 --2- 172.21.15.78:0/205847895 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5514065ed0 0x7f5514068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:20.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.718+0000 7f5532ffd700 1 --2- 172.21.15.78:0/205847895 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5514065ed0 0x7f5514068380 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5524009980 tx=0x7f5524008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:20.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.722+0000 7f5530ff9700 1 -- 172.21.15.78:0/205847895 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f551c050cb0 con 0x7f553415a940 2023-10-11T09:50:20.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.906+0000 7f5539819700 1 -- 172.21.15.78:0/205847895 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f55340c8d70 con 0x7f553415a940 2023-10-11T09:50:20.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.906+0000 7f5530ff9700 1 -- 172.21.15.78:0/205847895 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v108) v1 ==== 80+0+0 (secure 0 0 0) 0x7f551c0542d0 con 0x7f553415a940 2023-10-11T09:50:20.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.910+0000 7f5539819700 1 -- 172.21.15.78:0/205847895 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5514065ed0 msgr2=0x7f5514068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:20.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.910+0000 7f5539819700 1 --2- 172.21.15.78:0/205847895 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5514065ed0 0x7f5514068380 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5524009980 tx=0x7f5524008040 comp rx=0 tx=0).stop 2023-10-11T09:50:20.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.910+0000 7f5539819700 1 -- 172.21.15.78:0/205847895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f553415a940 msgr2=0x7f55340b3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:20.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.910+0000 7f5539819700 1 --2- 172.21.15.78:0/205847895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f553415a940 0x7f55340b3be0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f551c00e870 tx=0x7f551c00eb80 comp rx=0 tx=0).stop 2023-10-11T09:50:20.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.910+0000 7f5539819700 1 -- 172.21.15.78:0/205847895 shutdown_connections 2023-10-11T09:50:20.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.910+0000 7f5539819700 1 --2- 172.21.15.78:0/205847895 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5514065ed0 0x7f5514068380 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:20.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.910+0000 7f5539819700 1 --2- 172.21.15.78:0/205847895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f553415b8e0 0x7f55340b4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:20.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.910+0000 7f5539819700 1 --2- 172.21.15.78:0/205847895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f553415a940 0x7f55340b3be0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:20.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.910+0000 7f5539819700 1 --2- 172.21.15.78:0/205847895 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f553406d610 0x7f553413f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:20.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.910+0000 7f5539819700 1 -- 172.21.15.78:0/205847895 >> 172.21.15.78:0/205847895 conn(0x7f55340cea60 msgr2=0x7f553415d450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:20.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.914+0000 7f5539819700 1 -- 172.21.15.78:0/205847895 shutdown_connections 2023-10-11T09:50:20.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:20.914+0000 7f5539819700 1 -- 172.21.15.78:0/205847895 wait complete. 2023-10-11T09:50:20.923 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2023-10-11T09:50:20.940 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi078.front.sepia.ceph.com... 2023-10-11T09:50:20.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:50:20.941 DEBUG:teuthology.orchestra.run.smithi078:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-11T09:50:20.972 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:ceph-fuse[56273]: fuse finished with error 0 and tester_r 0 2023-10-11T09:50:20.979 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:50:27.083 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2023-10-11T09:50:27.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:50:27.084 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-11T09:50:27.105 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-11T09:50:27.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.318+0000 7f74c8866700 1 Processor -- start 2023-10-11T09:50:27.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.318+0000 7f74c8866700 1 -- start start 2023-10-11T09:50:27.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.318+0000 7f74c8866700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74c415b8e0 0x7f74c406d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:27.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.318+0000 7f74c8866700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f74c40ca950 0x7f74c413fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:27.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.318+0000 7f74c8866700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74c415e010 con 0x7f74c406d610 2023-10-11T09:50:27.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.318+0000 7f74c8866700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74c415e280 con 0x7f74c40ca950 2023-10-11T09:50:27.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.318+0000 7f74c8866700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c415e560 con 0x7f74c415b8e0 2023-10-11T09:50:27.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.318+0000 7f74c37fe700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74c415b8e0 0x7f74c406d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:27.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.322+0000 7f74c37fe700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74c415b8e0 0x7f74c406d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:55792/0 (socket says 172.21.15.78:55792) 2023-10-11T09:50:27.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.322+0000 7f74c2ffd700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f74c406d610 0x7f74c40ca240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:60078/0 (socket says 172.21.15.78:60078) 2023-10-11T09:50:27.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.322+0000 7f74c37fe700 1 -- 172.21.15.78:0/1420318402 learned_addr learned my addr 172.21.15.78:0/1420318402 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:27.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.322+0000 7f74c3fff700 1 --2- 172.21.15.78:0/1420318402 >> v2:172.21.15.133:3300/0 conn(0x7f74c40ca950 0x7f74c413fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:27.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.322+0000 7f74c37fe700 1 -- 172.21.15.78:0/1420318402 >> v2:172.21.15.133:3300/0 conn(0x7f74c40ca950 msgr2=0x7f74c413fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:27.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.322+0000 7f74c37fe700 1 --2- 172.21.15.78:0/1420318402 >> v2:172.21.15.133:3300/0 conn(0x7f74c40ca950 0x7f74c413fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:27.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.322+0000 7f74c37fe700 1 -- 172.21.15.78:0/1420318402 >> v1:172.21.15.133:6789/0 conn(0x7f74c406d610 legacy=0x7f74c40ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:27.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.322+0000 7f74c37fe700 1 -- 172.21.15.78:0/1420318402 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c415e840 con 0x7f74c415b8e0 2023-10-11T09:50:27.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.322+0000 7f74c27fc700 1 -- 172.21.15.78:0/1420318402 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 755233813 0 0) 0x7f74c415e010 con 0x7f74c406d610 2023-10-11T09:50:27.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.322+0000 7f74c37fe700 1 --2- 172.21.15.78:0/1420318402 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74c415b8e0 0x7f74c406d0d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f74b4009910 tx=0x7f74b4009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=818d1c8528649de0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:27.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.322+0000 7f74c27fc700 1 -- 172.21.15.78:0/1420318402 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b4011830 con 0x7f74c415b8e0 2023-10-11T09:50:27.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.322+0000 7f74c27fc700 1 -- 172.21.15.78:0/1420318402 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b4011990 con 0x7f74c415b8e0 2023-10-11T09:50:27.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.322+0000 7f74c27fc700 1 -- 172.21.15.78:0/1420318402 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b4011d40 con 0x7f74c415b8e0 2023-10-11T09:50:27.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.322+0000 7f74c8866700 1 -- 172.21.15.78:0/1420318402 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74c415b8e0 msgr2=0x7f74c406d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:27.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.322+0000 7f74c8866700 1 --2- 172.21.15.78:0/1420318402 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74c415b8e0 0x7f74c406d0d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f74b4009910 tx=0x7f74b4009c20 comp rx=0 tx=0).stop 2023-10-11T09:50:27.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.322+0000 7f74c8866700 1 -- 172.21.15.78:0/1420318402 shutdown_connections 2023-10-11T09:50:27.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.322+0000 7f74c8866700 1 --2- 172.21.15.78:0/1420318402 >> v2:172.21.15.133:3300/0 conn(0x7f74c40ca950 0x7f74c413fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:27.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.322+0000 7f74c8866700 1 --2- 172.21.15.78:0/1420318402 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74c415b8e0 0x7f74c406d0d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:27.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.322+0000 7f74c8866700 1 -- 172.21.15.78:0/1420318402 >> 172.21.15.78:0/1420318402 conn(0x7f74c40cea60 msgr2=0x7f74c40cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:27.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.322+0000 7f74c8866700 1 -- 172.21.15.78:0/1420318402 shutdown_connections 2023-10-11T09:50:27.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.322+0000 7f74c8866700 1 -- 172.21.15.78:0/1420318402 wait complete. 2023-10-11T09:50:27.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.326+0000 7f74c8866700 1 Processor -- start 2023-10-11T09:50:27.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.326+0000 7f74c8866700 1 -- start start 2023-10-11T09:50:27.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.330+0000 7f74c8866700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74c406d610 0x7f74c40c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:27.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.330+0000 7f74c8866700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74c40ca950 0x7f74c40b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:27.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.330+0000 7f74c8866700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f74c415b8e0 0x7f74c414b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:27.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.330+0000 7f74c8866700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c40b3ec0 con 0x7f74c40ca950 2023-10-11T09:50:27.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.330+0000 7f74c8866700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c40b41a0 con 0x7f74c415b8e0 2023-10-11T09:50:27.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.330+0000 7f74c8866700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c40b4480 con 0x7f74c406d610 2023-10-11T09:50:27.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.330+0000 7f74c2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74c40ca950 0x7f74c40b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:27.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.330+0000 7f74c2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74c40ca950 0x7f74c40b3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35718/0 (socket says 172.21.15.78:35718) 2023-10-11T09:50:27.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.330+0000 7f74c2ffd700 1 -- 172.21.15.78:0/2543200927 learned_addr learned my addr 172.21.15.78:0/2543200927 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:27.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.330+0000 7f74c37fe700 1 --2- 172.21.15.78:0/2543200927 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74c406d610 0x7f74c40c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:27.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.330+0000 7f74c3fff700 1 --2- 172.21.15.78:0/2543200927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f74c415b8e0 0x7f74c414b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:27.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.330+0000 7f74c2ffd700 1 -- 172.21.15.78:0/2543200927 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74c406d610 msgr2=0x7f74c40c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:27.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.330+0000 7f74c2ffd700 1 --2- 172.21.15.78:0/2543200927 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74c406d610 0x7f74c40c8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:27.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.330+0000 7f74c2ffd700 1 -- 172.21.15.78:0/2543200927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f74c415b8e0 msgr2=0x7f74c414b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:27.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.330+0000 7f74c2ffd700 1 --2- 172.21.15.78:0/2543200927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f74c415b8e0 0x7f74c414b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:27.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.330+0000 7f74c2ffd700 1 -- 172.21.15.78:0/2543200927 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b4009570 con 0x7f74c40ca950 2023-10-11T09:50:27.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.330+0000 7f74c2ffd700 1 --2- 172.21.15.78:0/2543200927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74c40ca950 0x7f74c40b3470 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f74b800c580 tx=0x7f74b800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:27.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.330+0000 7f74abfff700 1 -- 172.21.15.78:0/2543200927 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b8004ca0 con 0x7f74c40ca950 2023-10-11T09:50:27.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.330+0000 7f74abfff700 1 -- 172.21.15.78:0/2543200927 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b8004e20 con 0x7f74c40ca950 2023-10-11T09:50:27.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.330+0000 7f74abfff700 1 -- 172.21.15.78:0/2543200927 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b8007310 con 0x7f74c40ca950 2023-10-11T09:50:27.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.330+0000 7f74c8866700 1 -- 172.21.15.78:0/2543200927 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c40b4760 con 0x7f74c40ca950 2023-10-11T09:50:27.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.330+0000 7f74c8866700 1 -- 172.21.15.78:0/2543200927 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c414f880 con 0x7f74c40ca950 2023-10-11T09:50:27.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.334+0000 7f74c8866700 1 -- 172.21.15.78:0/2543200927 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74c4149900 con 0x7f74c40ca950 2023-10-11T09:50:27.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.334+0000 7f74abfff700 1 -- 172.21.15.78:0/2543200927 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f74b8007470 con 0x7f74c40ca950 2023-10-11T09:50:27.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.334+0000 7f74abfff700 1 --2- 172.21.15.78:0/2543200927 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f74b0065ed0 0x7f74b0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:27.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.334+0000 7f74abfff700 1 -- 172.21.15.78:0/2543200927 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6466+0+0 (secure 0 0 0) 0x7f74b8083720 con 0x7f74c40ca950 2023-10-11T09:50:27.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.334+0000 7f74c37fe700 1 --2- 172.21.15.78:0/2543200927 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f74b0065ed0 0x7f74b0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:27.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.334+0000 7f74c37fe700 1 --2- 172.21.15.78:0/2543200927 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f74b0065ed0 0x7f74b0068380 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f74b4014040 tx=0x7f74b4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:27.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.342+0000 7f74abfff700 1 -- 172.21.15.78:0/2543200927 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f74c4149900 con 0x7f74c40ca950 2023-10-11T09:50:27.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.554+0000 7f74c8866700 1 -- 172.21.15.78:0/2543200927 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f74c4149900 con 0x7f74c40ca950 2023-10-11T09:50:27.565 INFO:tasks.ceph.mon.a.smithi078.stderr:2023-10-11T09:50:27.558+0000 7f90131f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-11T09:50:27.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.574+0000 7f74abfff700 1 -- 172.21.15.78:0/2543200927 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v81) v1 ==== 161+0+0 (secure 0 0 0) 0x7f74c4149900 con 0x7f74c40ca950 2023-10-11T09:50:27.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.578+0000 7f74c8866700 1 -- 172.21.15.78:0/2543200927 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f74b0065ed0 msgr2=0x7f74b0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:27.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.578+0000 7f74c8866700 1 --2- 172.21.15.78:0/2543200927 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f74b0065ed0 0x7f74b0068380 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f74b4014040 tx=0x7f74b4008030 comp rx=0 tx=0).stop 2023-10-11T09:50:27.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.578+0000 7f74c8866700 1 -- 172.21.15.78:0/2543200927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74c40ca950 msgr2=0x7f74c40b3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:27.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.578+0000 7f74c8866700 1 --2- 172.21.15.78:0/2543200927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74c40ca950 0x7f74c40b3470 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f74b800c580 tx=0x7f74b800c890 comp rx=0 tx=0).stop 2023-10-11T09:50:27.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.578+0000 7f74c8866700 1 -- 172.21.15.78:0/2543200927 shutdown_connections 2023-10-11T09:50:27.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.578+0000 7f74c8866700 1 --2- 172.21.15.78:0/2543200927 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f74b0065ed0 0x7f74b0068380 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:27.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.578+0000 7f74c8866700 1 --2- 172.21.15.78:0/2543200927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f74c415b8e0 0x7f74c414b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:27.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.578+0000 7f74c8866700 1 --2- 172.21.15.78:0/2543200927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74c40ca950 0x7f74c40b3470 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:27.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.578+0000 7f74c8866700 1 --2- 172.21.15.78:0/2543200927 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74c406d610 0x7f74c40c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:27.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.578+0000 7f74c8866700 1 -- 172.21.15.78:0/2543200927 >> 172.21.15.78:0/2543200927 conn(0x7f74c40cea60 msgr2=0x7f74c40c9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:27.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.578+0000 7f74c8866700 1 -- 172.21.15.78:0/2543200927 shutdown_connections 2023-10-11T09:50:27.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.578+0000 7f74c8866700 1 -- 172.21.15.78:0/2543200927 wait complete. 2023-10-11T09:50:27.589 INFO:teuthology.orchestra.run.smithi078.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-11T09:50:27.594 INFO:tasks.ceph.mds.c.smithi078.stderr: -9860> 2023-10-11T09:49:56.434+0000 7fecdb37f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-11T09:50:27.609 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-11T09:50:27.614 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-11T09:50:27.661 INFO:tasks.ceph.mds.c.smithi078.stdout:starting mds.c at 2023-10-11T09:50:27.673 INFO:teuthology.orchestra.run.smithi078.stdout:[global] 2023-10-11T09:50:27.673 INFO:teuthology.orchestra.run.smithi078.stdout: chdir = "" 2023-10-11T09:50:27.674 INFO:teuthology.orchestra.run.smithi078.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-11T09:50:27.674 INFO:teuthology.orchestra.run.smithi078.stdout: auth supported = cephx 2023-10-11T09:50:27.674 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.674 INFO:teuthology.orchestra.run.smithi078.stdout: filestore xattr use omap = true 2023-10-11T09:50:27.674 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.675 INFO:teuthology.orchestra.run.smithi078.stdout: mon clock drift allowed = 1.000 2023-10-11T09:50:27.675 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.675 INFO:teuthology.orchestra.run.smithi078.stdout: osd crush chooseleaf type = 0 2023-10-11T09:50:27.675 INFO:teuthology.orchestra.run.smithi078.stdout: auth debug = true 2023-10-11T09:50:27.675 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.676 INFO:teuthology.orchestra.run.smithi078.stdout: ms die on old message = true 2023-10-11T09:50:27.676 INFO:teuthology.orchestra.run.smithi078.stdout: ms die on bug = true 2023-10-11T09:50:27.676 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.676 INFO:teuthology.orchestra.run.smithi078.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-11T09:50:27.677 INFO:teuthology.orchestra.run.smithi078.stdout: mon pg warn max object skew = 0 2023-10-11T09:50:27.677 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.677 INFO:teuthology.orchestra.run.smithi078.stdout: # disable pg_autoscaler by default for new pools 2023-10-11T09:50:27.677 INFO:teuthology.orchestra.run.smithi078.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-11T09:50:27.678 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.678 INFO:teuthology.orchestra.run.smithi078.stdout: osd pool default size = 2 2023-10-11T09:50:27.678 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.678 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd allow primary affinity = true 2023-10-11T09:50:27.678 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd allow pg remap = true 2023-10-11T09:50:27.679 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on legacy crush tunables = false 2023-10-11T09:50:27.679 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on crush straw calc version zero = false 2023-10-11T09:50:27.679 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on no sortbitwise = false 2023-10-11T09:50:27.679 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on osd down out interval zero = false 2023-10-11T09:50:27.679 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on too few osds = false 2023-10-11T09:50:27.680 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-11T09:50:27.680 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-11T09:50:27.680 INFO:teuthology.orchestra.run.smithi078.stdout: mon_allow_pool_size_one = true 2023-10-11T09:50:27.680 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.680 INFO:teuthology.orchestra.run.smithi078.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-11T09:50:27.680 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.681 INFO:teuthology.orchestra.run.smithi078.stdout: osd default data pool replay window = 5 2023-10-11T09:50:27.681 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.681 INFO:teuthology.orchestra.run.smithi078.stdout: mon allow pool delete = true 2023-10-11T09:50:27.681 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.681 INFO:teuthology.orchestra.run.smithi078.stdout: mon cluster log file level = debug 2023-10-11T09:50:27.682 INFO:teuthology.orchestra.run.smithi078.stdout: debug asserts on shutdown = true 2023-10-11T09:50:27.682 INFO:teuthology.orchestra.run.smithi078.stdout: mon health detail to clog = false 2023-10-11T09:50:27.682 INFO:teuthology.orchestra.run.smithi078.stdout: mon host = "172.21.15.78,172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790]" 2023-10-11T09:50:27.682 INFO:teuthology.orchestra.run.smithi078.stdout: fsid = dee99db9-7a72-406f-a1e2-917943f2b13d 2023-10-11T09:50:27.682 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.683 INFO:teuthology.orchestra.run.smithi078.stdout:[osd] 2023-10-11T09:50:27.683 INFO:teuthology.orchestra.run.smithi078.stdout: osd journal size = 100 2023-10-11T09:50:27.683 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.683 INFO:teuthology.orchestra.run.smithi078.stdout: osd scrub load threshold = 5.0 2023-10-11T09:50:27.683 INFO:teuthology.orchestra.run.smithi078.stdout: osd scrub max interval = 600 2023-10-11T09:50:27.683 INFO:teuthology.orchestra.run.smithi078.stdout: osd mclock profile = high_recovery_ops 2023-10-11T09:50:27.684 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.684 INFO:teuthology.orchestra.run.smithi078.stdout: osd recover clone overlap = true 2023-10-11T09:50:27.684 INFO:teuthology.orchestra.run.smithi078.stdout: osd recovery max chunk = 1048576 2023-10-11T09:50:27.684 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.684 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug shutdown = true 2023-10-11T09:50:27.685 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug op order = true 2023-10-11T09:50:27.685 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug verify stray on activate = true 2023-10-11T09:50:27.685 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.685 INFO:teuthology.orchestra.run.smithi078.stdout: osd open classes on start = true 2023-10-11T09:50:27.685 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug pg log writeout = true 2023-10-11T09:50:27.686 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.686 INFO:teuthology.orchestra.run.smithi078.stdout: osd deep scrub update digest min age = 30 2023-10-11T09:50:27.686 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.686 INFO:teuthology.orchestra.run.smithi078.stdout: osd map max advance = 10 2023-10-11T09:50:27.686 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.686 INFO:teuthology.orchestra.run.smithi078.stdout: journal zero on create = true 2023-10-11T09:50:27.687 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.687 INFO:teuthology.orchestra.run.smithi078.stdout: filestore ondisk finisher threads = 3 2023-10-11T09:50:27.687 INFO:teuthology.orchestra.run.smithi078.stdout: filestore apply finisher threads = 3 2023-10-11T09:50:27.687 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.687 INFO:teuthology.orchestra.run.smithi078.stdout: bdev debug aio = true 2023-10-11T09:50:27.688 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug misdirected ops = true 2023-10-11T09:50:27.688 INFO:teuthology.orchestra.run.smithi078.stdout: bdev async discard = True 2023-10-11T09:50:27.688 INFO:teuthology.orchestra.run.smithi078.stdout: bdev enable discard = True 2023-10-11T09:50:27.688 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore allocator = bitmap 2023-10-11T09:50:27.688 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore block size = 96636764160 2023-10-11T09:50:27.688 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore fsck on mount = True 2023-10-11T09:50:27.689 INFO:teuthology.orchestra.run.smithi078.stdout: debug bluefs = 1/20 2023-10-11T09:50:27.689 INFO:teuthology.orchestra.run.smithi078.stdout: debug bluestore = 1/20 2023-10-11T09:50:27.689 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:50:27.689 INFO:teuthology.orchestra.run.smithi078.stdout: debug osd = 20 2023-10-11T09:50:27.689 INFO:teuthology.orchestra.run.smithi078.stdout: debug rocksdb = 4/10 2023-10-11T09:50:27.690 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd backfillfull_ratio = 0.85 2023-10-11T09:50:27.690 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd full ratio = 0.9 2023-10-11T09:50:27.690 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd nearfull ratio = 0.8 2023-10-11T09:50:27.690 INFO:teuthology.orchestra.run.smithi078.stdout: osd failsafe full ratio = 0.95 2023-10-11T09:50:27.690 INFO:teuthology.orchestra.run.smithi078.stdout: osd objectstore = bluestore 2023-10-11T09:50:27.690 INFO:teuthology.orchestra.run.smithi078.stdout: osd op complaint time = 180 2023-10-11T09:50:27.691 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.691 INFO:teuthology.orchestra.run.smithi078.stdout:[mgr] 2023-10-11T09:50:27.691 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:50:27.691 INFO:teuthology.orchestra.run.smithi078.stdout: debug mgr = 20 2023-10-11T09:50:27.691 INFO:teuthology.orchestra.run.smithi078.stdout: debug mon = 20 2023-10-11T09:50:27.692 INFO:teuthology.orchestra.run.smithi078.stdout: debug auth = 20 2023-10-11T09:50:27.692 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:50:27.692 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:50:27.692 INFO:teuthology.orchestra.run.smithi078.stdout: mgr/telemetry/nag = false 2023-10-11T09:50:27.692 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.692 INFO:teuthology.orchestra.run.smithi078.stdout:[mon] 2023-10-11T09:50:27.693 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:50:27.693 INFO:teuthology.orchestra.run.smithi078.stdout: debug mon = 20 2023-10-11T09:50:27.693 INFO:teuthology.orchestra.run.smithi078.stdout: debug paxos = 20 2023-10-11T09:50:27.693 INFO:teuthology.orchestra.run.smithi078.stdout: debug auth = 20 2023-10-11T09:50:27.693 INFO:teuthology.orchestra.run.smithi078.stdout: mon data avail warn = 5 2023-10-11T09:50:27.694 INFO:teuthology.orchestra.run.smithi078.stdout: mon mgr mkfs grace = 240 2023-10-11T09:50:27.694 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:50:27.694 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd reporter subtree level = osd 2023-10-11T09:50:27.694 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd prime pg temp = true 2023-10-11T09:50:27.694 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:50:27.695 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.695 INFO:teuthology.orchestra.run.smithi078.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-11T09:50:27.695 INFO:teuthology.orchestra.run.smithi078.stdout: auth mon ticket ttl = 660 # 11m 2023-10-11T09:50:27.695 INFO:teuthology.orchestra.run.smithi078.stdout: auth service ticket ttl = 240 # 4m 2023-10-11T09:50:27.695 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.695 INFO:teuthology.orchestra.run.smithi078.stdout: # don't complain about insecure global_id in the test suite 2023-10-11T09:50:27.696 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-11T09:50:27.696 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-11T09:50:27.696 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.696 INFO:teuthology.orchestra.run.smithi078.stdout: # 1m isn't quite enough 2023-10-11T09:50:27.696 INFO:teuthology.orchestra.run.smithi078.stdout: mon_down_mkfs_grace = 2m 2023-10-11T09:50:27.697 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.697 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_filestore_osds = false 2023-10-11T09:50:27.697 INFO:teuthology.orchestra.run.smithi078.stdout: mon op complaint time = 120 2023-10-11T09:50:27.697 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.698 INFO:teuthology.orchestra.run.smithi078.stdout:[client] 2023-10-11T09:50:27.698 INFO:teuthology.orchestra.run.smithi078.stdout: rgw cache enabled = true 2023-10-11T09:50:27.698 INFO:teuthology.orchestra.run.smithi078.stdout: rgw enable ops log = true 2023-10-11T09:50:27.698 INFO:teuthology.orchestra.run.smithi078.stdout: rgw enable usage log = true 2023-10-11T09:50:27.698 INFO:teuthology.orchestra.run.smithi078.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-11T09:50:27.699 INFO:teuthology.orchestra.run.smithi078.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-11T09:50:27.699 INFO:teuthology.orchestra.run.smithi078.stdout: client mount timeout = 600 2023-10-11T09:50:27.699 INFO:teuthology.orchestra.run.smithi078.stdout: debug client = 20 2023-10-11T09:50:27.699 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:50:27.699 INFO:teuthology.orchestra.run.smithi078.stdout: pid file = "" 2023-10-11T09:50:27.700 INFO:teuthology.orchestra.run.smithi078.stdout: rados mon op timeout = 900 2023-10-11T09:50:27.700 INFO:teuthology.orchestra.run.smithi078.stdout: rados osd op timeout = 900 2023-10-11T09:50:27.700 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.a] 2023-10-11T09:50:27.700 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.b] 2023-10-11T09:50:27.700 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.c] 2023-10-11T09:50:27.701 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.a] 2023-10-11T09:50:27.701 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.c] 2023-10-11T09:50:27.701 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.b] 2023-10-11T09:50:27.701 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.d] 2023-10-11T09:50:27.702 INFO:teuthology.orchestra.run.smithi078.stdout:[mds] 2023-10-11T09:50:27.702 INFO:teuthology.orchestra.run.smithi078.stdout: debug mds = 20 2023-10-11T09:50:27.702 INFO:teuthology.orchestra.run.smithi078.stdout: debug mds balancer = 20 2023-10-11T09:50:27.702 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:50:27.702 INFO:teuthology.orchestra.run.smithi078.stdout: mds debug frag = True 2023-10-11T09:50:27.703 INFO:teuthology.orchestra.run.smithi078.stdout: mds debug scatterstat = False 2023-10-11T09:50:27.703 INFO:teuthology.orchestra.run.smithi078.stdout: mds op complaint time = 180 2023-10-11T09:50:27.703 INFO:teuthology.orchestra.run.smithi078.stdout: mds verify scatter = False 2023-10-11T09:50:27.703 INFO:teuthology.orchestra.run.smithi078.stdout: osd op complaint time = 180 2023-10-11T09:50:27.703 INFO:teuthology.orchestra.run.smithi078.stdout: rados mon op timeout = 900 2023-10-11T09:50:27.704 INFO:teuthology.orchestra.run.smithi078.stdout: rados osd op timeout = 900 2023-10-11T09:50:27.704 INFO:teuthology.orchestra.run.smithi133.stdout:[global] 2023-10-11T09:50:27.705 INFO:teuthology.orchestra.run.smithi133.stdout: chdir = "" 2023-10-11T09:50:27.705 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-11T09:50:27.705 INFO:teuthology.orchestra.run.smithi133.stdout: auth supported = cephx 2023-10-11T09:50:27.705 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.706 INFO:teuthology.orchestra.run.smithi133.stdout: filestore xattr use omap = true 2023-10-11T09:50:27.706 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.706 INFO:teuthology.orchestra.run.smithi133.stdout: mon clock drift allowed = 1.000 2023-10-11T09:50:27.706 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.707 INFO:teuthology.orchestra.run.smithi133.stdout: osd crush chooseleaf type = 0 2023-10-11T09:50:27.707 INFO:teuthology.orchestra.run.smithi133.stdout: auth debug = true 2023-10-11T09:50:27.707 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.707 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on old message = true 2023-10-11T09:50:27.708 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on bug = true 2023-10-11T09:50:27.708 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.708 INFO:teuthology.orchestra.run.smithi133.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-11T09:50:27.708 INFO:teuthology.orchestra.run.smithi133.stdout: mon pg warn max object skew = 0 2023-10-11T09:50:27.708 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.709 INFO:teuthology.orchestra.run.smithi133.stdout: # disable pg_autoscaler by default for new pools 2023-10-11T09:50:27.709 INFO:teuthology.orchestra.run.smithi133.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-11T09:50:27.709 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.710 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default size = 2 2023-10-11T09:50:27.710 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.710 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow primary affinity = true 2023-10-11T09:50:27.710 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow pg remap = true 2023-10-11T09:50:27.710 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on legacy crush tunables = false 2023-10-11T09:50:27.711 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on crush straw calc version zero = false 2023-10-11T09:50:27.711 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on no sortbitwise = false 2023-10-11T09:50:27.711 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on osd down out interval zero = false 2023-10-11T09:50:27.711 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on too few osds = false 2023-10-11T09:50:27.711 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-11T09:50:27.711 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-11T09:50:27.712 INFO:teuthology.orchestra.run.smithi133.stdout: mon_allow_pool_size_one = true 2023-10-11T09:50:27.712 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.712 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-11T09:50:27.712 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.712 INFO:teuthology.orchestra.run.smithi133.stdout: osd default data pool replay window = 5 2023-10-11T09:50:27.713 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.713 INFO:teuthology.orchestra.run.smithi133.stdout: mon allow pool delete = true 2023-10-11T09:50:27.713 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.713 INFO:teuthology.orchestra.run.smithi133.stdout: mon cluster log file level = debug 2023-10-11T09:50:27.713 INFO:teuthology.orchestra.run.smithi133.stdout: debug asserts on shutdown = true 2023-10-11T09:50:27.713 INFO:teuthology.orchestra.run.smithi133.stdout: mon health detail to clog = false 2023-10-11T09:50:27.714 INFO:teuthology.orchestra.run.smithi133.stdout: mon host = "172.21.15.78,172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790]" 2023-10-11T09:50:27.714 INFO:teuthology.orchestra.run.smithi133.stdout: fsid = dee99db9-7a72-406f-a1e2-917943f2b13d 2023-10-11T09:50:27.714 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.714 INFO:teuthology.orchestra.run.smithi133.stdout:[osd] 2023-10-11T09:50:27.714 INFO:teuthology.orchestra.run.smithi133.stdout: osd journal size = 100 2023-10-11T09:50:27.715 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.715 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub load threshold = 5.0 2023-10-11T09:50:27.715 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub max interval = 600 2023-10-11T09:50:27.715 INFO:teuthology.orchestra.run.smithi133.stdout: osd mclock profile = high_recovery_ops 2023-10-11T09:50:27.715 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.716 INFO:teuthology.orchestra.run.smithi133.stdout: osd recover clone overlap = true 2023-10-11T09:50:27.716 INFO:teuthology.orchestra.run.smithi133.stdout: osd recovery max chunk = 1048576 2023-10-11T09:50:27.716 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.716 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug shutdown = true 2023-10-11T09:50:27.716 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug op order = true 2023-10-11T09:50:27.716 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug verify stray on activate = true 2023-10-11T09:50:27.717 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.717 INFO:teuthology.orchestra.run.smithi133.stdout: osd open classes on start = true 2023-10-11T09:50:27.717 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug pg log writeout = true 2023-10-11T09:50:27.717 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.717 INFO:teuthology.orchestra.run.smithi133.stdout: osd deep scrub update digest min age = 30 2023-10-11T09:50:27.718 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.718 INFO:teuthology.orchestra.run.smithi133.stdout: osd map max advance = 10 2023-10-11T09:50:27.718 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.718 INFO:teuthology.orchestra.run.smithi133.stdout: journal zero on create = true 2023-10-11T09:50:27.718 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.719 INFO:teuthology.orchestra.run.smithi133.stdout: filestore ondisk finisher threads = 3 2023-10-11T09:50:27.719 INFO:teuthology.orchestra.run.smithi133.stdout: filestore apply finisher threads = 3 2023-10-11T09:50:27.719 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.719 INFO:teuthology.orchestra.run.smithi133.stdout: bdev debug aio = true 2023-10-11T09:50:27.719 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug misdirected ops = true 2023-10-11T09:50:27.719 INFO:teuthology.orchestra.run.smithi133.stdout: bdev async discard = True 2023-10-11T09:50:27.720 INFO:teuthology.orchestra.run.smithi133.stdout: bdev enable discard = True 2023-10-11T09:50:27.720 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore allocator = bitmap 2023-10-11T09:50:27.720 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore block size = 96636764160 2023-10-11T09:50:27.720 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore fsck on mount = True 2023-10-11T09:50:27.720 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluefs = 1/20 2023-10-11T09:50:27.721 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluestore = 1/20 2023-10-11T09:50:27.721 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:50:27.721 INFO:teuthology.orchestra.run.smithi133.stdout: debug osd = 20 2023-10-11T09:50:27.721 INFO:teuthology.orchestra.run.smithi133.stdout: debug rocksdb = 4/10 2023-10-11T09:50:27.721 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd backfillfull_ratio = 0.85 2023-10-11T09:50:27.721 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd full ratio = 0.9 2023-10-11T09:50:27.722 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd nearfull ratio = 0.8 2023-10-11T09:50:27.722 INFO:teuthology.orchestra.run.smithi133.stdout: osd failsafe full ratio = 0.95 2023-10-11T09:50:27.722 INFO:teuthology.orchestra.run.smithi133.stdout: osd objectstore = bluestore 2023-10-11T09:50:27.722 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2023-10-11T09:50:27.722 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.723 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr] 2023-10-11T09:50:27.723 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:50:27.723 INFO:teuthology.orchestra.run.smithi133.stdout: debug mgr = 20 2023-10-11T09:50:27.723 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2023-10-11T09:50:27.723 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2023-10-11T09:50:27.723 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:50:27.724 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:50:27.724 INFO:teuthology.orchestra.run.smithi133.stdout: mgr/telemetry/nag = false 2023-10-11T09:50:27.724 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.724 INFO:teuthology.orchestra.run.smithi133.stdout:[mon] 2023-10-11T09:50:27.724 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:50:27.725 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2023-10-11T09:50:27.725 INFO:teuthology.orchestra.run.smithi133.stdout: debug paxos = 20 2023-10-11T09:50:27.725 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2023-10-11T09:50:27.725 INFO:teuthology.orchestra.run.smithi133.stdout: mon data avail warn = 5 2023-10-11T09:50:27.725 INFO:teuthology.orchestra.run.smithi133.stdout: mon mgr mkfs grace = 240 2023-10-11T09:50:27.725 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:50:27.726 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd reporter subtree level = osd 2023-10-11T09:50:27.726 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd prime pg temp = true 2023-10-11T09:50:27.726 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:50:27.726 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.726 INFO:teuthology.orchestra.run.smithi133.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-11T09:50:27.727 INFO:teuthology.orchestra.run.smithi133.stdout: auth mon ticket ttl = 660 # 11m 2023-10-11T09:50:27.727 INFO:teuthology.orchestra.run.smithi133.stdout: auth service ticket ttl = 240 # 4m 2023-10-11T09:50:27.727 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.727 INFO:teuthology.orchestra.run.smithi133.stdout: # don't complain about insecure global_id in the test suite 2023-10-11T09:50:27.727 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-11T09:50:27.728 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-11T09:50:27.728 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.728 INFO:teuthology.orchestra.run.smithi133.stdout: # 1m isn't quite enough 2023-10-11T09:50:27.728 INFO:teuthology.orchestra.run.smithi133.stdout: mon_down_mkfs_grace = 2m 2023-10-11T09:50:27.728 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.729 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_filestore_osds = false 2023-10-11T09:50:27.729 INFO:teuthology.orchestra.run.smithi133.stdout: mon op complaint time = 120 2023-10-11T09:50:27.729 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.729 INFO:teuthology.orchestra.run.smithi133.stdout:[client] 2023-10-11T09:50:27.729 INFO:teuthology.orchestra.run.smithi133.stdout: rgw cache enabled = true 2023-10-11T09:50:27.729 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable ops log = true 2023-10-11T09:50:27.730 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable usage log = true 2023-10-11T09:50:27.730 INFO:teuthology.orchestra.run.smithi133.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-11T09:50:27.730 INFO:teuthology.orchestra.run.smithi133.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-11T09:50:27.730 INFO:teuthology.orchestra.run.smithi133.stdout: client mount timeout = 600 2023-10-11T09:50:27.730 INFO:teuthology.orchestra.run.smithi133.stdout: debug client = 20 2023-10-11T09:50:27.731 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:50:27.731 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = "" 2023-10-11T09:50:27.731 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2023-10-11T09:50:27.731 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2023-10-11T09:50:27.732 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.a] 2023-10-11T09:50:27.732 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.b] 2023-10-11T09:50:27.732 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.c] 2023-10-11T09:50:27.732 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.a] 2023-10-11T09:50:27.732 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.c] 2023-10-11T09:50:27.733 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.b] 2023-10-11T09:50:27.733 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.d] 2023-10-11T09:50:27.733 INFO:teuthology.orchestra.run.smithi133.stdout:[mds] 2023-10-11T09:50:27.733 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds = 20 2023-10-11T09:50:27.733 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds balancer = 20 2023-10-11T09:50:27.734 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:50:27.734 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug frag = True 2023-10-11T09:50:27.734 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug scatterstat = False 2023-10-11T09:50:27.734 INFO:teuthology.orchestra.run.smithi133.stdout: mds op complaint time = 180 2023-10-11T09:50:27.735 INFO:teuthology.orchestra.run.smithi133.stdout: mds verify scatter = False 2023-10-11T09:50:27.735 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2023-10-11T09:50:27.735 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2023-10-11T09:50:27.735 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2023-10-11T09:50:27.738 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-11T09:50:27.786 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-11T09:50:27.854 INFO:teuthology.orchestra.run.smithi078.stdout:[global] 2023-10-11T09:50:27.855 INFO:teuthology.orchestra.run.smithi078.stdout: chdir = "" 2023-10-11T09:50:27.855 INFO:teuthology.orchestra.run.smithi078.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-11T09:50:27.855 INFO:teuthology.orchestra.run.smithi078.stdout: auth supported = cephx 2023-10-11T09:50:27.855 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.856 INFO:teuthology.orchestra.run.smithi078.stdout: filestore xattr use omap = true 2023-10-11T09:50:27.856 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.856 INFO:teuthology.orchestra.run.smithi078.stdout: mon clock drift allowed = 1.000 2023-10-11T09:50:27.856 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.856 INFO:teuthology.orchestra.run.smithi078.stdout: osd crush chooseleaf type = 0 2023-10-11T09:50:27.857 INFO:teuthology.orchestra.run.smithi078.stdout: auth debug = true 2023-10-11T09:50:27.857 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.857 INFO:teuthology.orchestra.run.smithi078.stdout: ms die on old message = true 2023-10-11T09:50:27.857 INFO:teuthology.orchestra.run.smithi078.stdout: ms die on bug = true 2023-10-11T09:50:27.858 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.858 INFO:teuthology.orchestra.run.smithi078.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-11T09:50:27.858 INFO:teuthology.orchestra.run.smithi078.stdout: mon pg warn max object skew = 0 2023-10-11T09:50:27.858 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.858 INFO:teuthology.orchestra.run.smithi078.stdout: # disable pg_autoscaler by default for new pools 2023-10-11T09:50:27.859 INFO:teuthology.orchestra.run.smithi078.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-11T09:50:27.859 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.859 INFO:teuthology.orchestra.run.smithi078.stdout: osd pool default size = 2 2023-10-11T09:50:27.859 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.859 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd allow primary affinity = true 2023-10-11T09:50:27.860 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd allow pg remap = true 2023-10-11T09:50:27.860 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on legacy crush tunables = false 2023-10-11T09:50:27.860 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on crush straw calc version zero = false 2023-10-11T09:50:27.860 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on no sortbitwise = false 2023-10-11T09:50:27.860 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on osd down out interval zero = false 2023-10-11T09:50:27.861 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on too few osds = false 2023-10-11T09:50:27.861 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-11T09:50:27.861 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-11T09:50:27.861 INFO:teuthology.orchestra.run.smithi078.stdout: mon_allow_pool_size_one = true 2023-10-11T09:50:27.861 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.862 INFO:teuthology.orchestra.run.smithi078.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-11T09:50:27.862 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.862 INFO:teuthology.orchestra.run.smithi078.stdout: osd default data pool replay window = 5 2023-10-11T09:50:27.863 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.863 INFO:teuthology.orchestra.run.smithi078.stdout: mon allow pool delete = true 2023-10-11T09:50:27.863 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.863 INFO:teuthology.orchestra.run.smithi078.stdout: mon cluster log file level = debug 2023-10-11T09:50:27.863 INFO:teuthology.orchestra.run.smithi078.stdout: debug asserts on shutdown = true 2023-10-11T09:50:27.864 INFO:teuthology.orchestra.run.smithi078.stdout: mon health detail to clog = false 2023-10-11T09:50:27.864 INFO:teuthology.orchestra.run.smithi078.stdout: mon host = "172.21.15.78,172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790]" 2023-10-11T09:50:27.864 INFO:teuthology.orchestra.run.smithi078.stdout: fsid = dee99db9-7a72-406f-a1e2-917943f2b13d 2023-10-11T09:50:27.864 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.864 INFO:teuthology.orchestra.run.smithi078.stdout:[osd] 2023-10-11T09:50:27.864 INFO:teuthology.orchestra.run.smithi078.stdout: osd journal size = 100 2023-10-11T09:50:27.865 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.865 INFO:teuthology.orchestra.run.smithi078.stdout: osd scrub load threshold = 5.0 2023-10-11T09:50:27.865 INFO:teuthology.orchestra.run.smithi078.stdout: osd scrub max interval = 600 2023-10-11T09:50:27.865 INFO:teuthology.orchestra.run.smithi078.stdout: osd mclock profile = high_recovery_ops 2023-10-11T09:50:27.865 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.866 INFO:teuthology.orchestra.run.smithi078.stdout: osd recover clone overlap = true 2023-10-11T09:50:27.866 INFO:teuthology.orchestra.run.smithi078.stdout: osd recovery max chunk = 1048576 2023-10-11T09:50:27.866 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.866 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug shutdown = true 2023-10-11T09:50:27.866 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug op order = true 2023-10-11T09:50:27.867 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug verify stray on activate = true 2023-10-11T09:50:27.867 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.867 INFO:teuthology.orchestra.run.smithi078.stdout: osd open classes on start = true 2023-10-11T09:50:27.867 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug pg log writeout = true 2023-10-11T09:50:27.867 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.867 INFO:teuthology.orchestra.run.smithi078.stdout: osd deep scrub update digest min age = 30 2023-10-11T09:50:27.868 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.868 INFO:teuthology.orchestra.run.smithi078.stdout: osd map max advance = 10 2023-10-11T09:50:27.868 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.868 INFO:teuthology.orchestra.run.smithi078.stdout: journal zero on create = true 2023-10-11T09:50:27.868 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.869 INFO:teuthology.orchestra.run.smithi078.stdout: filestore ondisk finisher threads = 3 2023-10-11T09:50:27.869 INFO:teuthology.orchestra.run.smithi078.stdout: filestore apply finisher threads = 3 2023-10-11T09:50:27.869 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.869 INFO:teuthology.orchestra.run.smithi078.stdout: bdev debug aio = true 2023-10-11T09:50:27.869 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug misdirected ops = true 2023-10-11T09:50:27.870 INFO:teuthology.orchestra.run.smithi078.stdout: bdev async discard = True 2023-10-11T09:50:27.870 INFO:teuthology.orchestra.run.smithi078.stdout: bdev enable discard = True 2023-10-11T09:50:27.870 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore allocator = bitmap 2023-10-11T09:50:27.870 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore block size = 96636764160 2023-10-11T09:50:27.870 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore fsck on mount = True 2023-10-11T09:50:27.871 INFO:teuthology.orchestra.run.smithi078.stdout: debug bluefs = 1/20 2023-10-11T09:50:27.871 INFO:teuthology.orchestra.run.smithi078.stdout: debug bluestore = 1/20 2023-10-11T09:50:27.871 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:50:27.871 INFO:teuthology.orchestra.run.smithi078.stdout: debug osd = 20 2023-10-11T09:50:27.871 INFO:teuthology.orchestra.run.smithi078.stdout: debug rocksdb = 4/10 2023-10-11T09:50:27.871 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd backfillfull_ratio = 0.85 2023-10-11T09:50:27.872 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd full ratio = 0.9 2023-10-11T09:50:27.872 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd nearfull ratio = 0.8 2023-10-11T09:50:27.872 INFO:teuthology.orchestra.run.smithi078.stdout: osd failsafe full ratio = 0.95 2023-10-11T09:50:27.872 INFO:teuthology.orchestra.run.smithi078.stdout: osd objectstore = bluestore 2023-10-11T09:50:27.872 INFO:teuthology.orchestra.run.smithi078.stdout: osd op complaint time = 180 2023-10-11T09:50:27.873 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.873 INFO:teuthology.orchestra.run.smithi078.stdout:[mgr] 2023-10-11T09:50:27.873 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:50:27.873 INFO:teuthology.orchestra.run.smithi078.stdout: debug mgr = 20 2023-10-11T09:50:27.873 INFO:teuthology.orchestra.run.smithi078.stdout: debug mon = 20 2023-10-11T09:50:27.874 INFO:teuthology.orchestra.run.smithi078.stdout: debug auth = 20 2023-10-11T09:50:27.874 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:50:27.874 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:50:27.874 INFO:teuthology.orchestra.run.smithi078.stdout: mgr/telemetry/nag = false 2023-10-11T09:50:27.874 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.875 INFO:teuthology.orchestra.run.smithi078.stdout:[mon] 2023-10-11T09:50:27.875 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:50:27.875 INFO:teuthology.orchestra.run.smithi078.stdout: debug mon = 20 2023-10-11T09:50:27.875 INFO:teuthology.orchestra.run.smithi078.stdout: debug paxos = 20 2023-10-11T09:50:27.875 INFO:teuthology.orchestra.run.smithi078.stdout: debug auth = 20 2023-10-11T09:50:27.876 INFO:teuthology.orchestra.run.smithi078.stdout: mon data avail warn = 5 2023-10-11T09:50:27.876 INFO:teuthology.orchestra.run.smithi078.stdout: mon mgr mkfs grace = 240 2023-10-11T09:50:27.876 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:50:27.876 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd reporter subtree level = osd 2023-10-11T09:50:27.876 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd prime pg temp = true 2023-10-11T09:50:27.876 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:50:27.877 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.877 INFO:teuthology.orchestra.run.smithi078.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-11T09:50:27.877 INFO:teuthology.orchestra.run.smithi078.stdout: auth mon ticket ttl = 660 # 11m 2023-10-11T09:50:27.877 INFO:teuthology.orchestra.run.smithi078.stdout: auth service ticket ttl = 240 # 4m 2023-10-11T09:50:27.877 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.878 INFO:teuthology.orchestra.run.smithi078.stdout: # don't complain about insecure global_id in the test suite 2023-10-11T09:50:27.878 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-11T09:50:27.878 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-11T09:50:27.878 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.878 INFO:teuthology.orchestra.run.smithi078.stdout: # 1m isn't quite enough 2023-10-11T09:50:27.879 INFO:teuthology.orchestra.run.smithi078.stdout: mon_down_mkfs_grace = 2m 2023-10-11T09:50:27.879 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.879 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_filestore_osds = false 2023-10-11T09:50:27.879 INFO:teuthology.orchestra.run.smithi078.stdout: mon op complaint time = 120 2023-10-11T09:50:27.879 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:27.879 INFO:teuthology.orchestra.run.smithi078.stdout:[client] 2023-10-11T09:50:27.880 INFO:teuthology.orchestra.run.smithi078.stdout: rgw cache enabled = true 2023-10-11T09:50:27.880 INFO:teuthology.orchestra.run.smithi078.stdout: rgw enable ops log = true 2023-10-11T09:50:27.880 INFO:teuthology.orchestra.run.smithi078.stdout: rgw enable usage log = true 2023-10-11T09:50:27.880 INFO:teuthology.orchestra.run.smithi078.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-11T09:50:27.880 INFO:teuthology.orchestra.run.smithi078.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-11T09:50:27.881 INFO:teuthology.orchestra.run.smithi078.stdout: client mount timeout = 600 2023-10-11T09:50:27.881 INFO:teuthology.orchestra.run.smithi078.stdout: debug client = 20 2023-10-11T09:50:27.881 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:50:27.881 INFO:teuthology.orchestra.run.smithi078.stdout: pid file = "" 2023-10-11T09:50:27.881 INFO:teuthology.orchestra.run.smithi078.stdout: rados mon op timeout = 900 2023-10-11T09:50:27.882 INFO:teuthology.orchestra.run.smithi078.stdout: rados osd op timeout = 900 2023-10-11T09:50:27.882 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.a] 2023-10-11T09:50:27.882 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.b] 2023-10-11T09:50:27.882 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.c] 2023-10-11T09:50:27.882 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.a] 2023-10-11T09:50:27.883 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.c] 2023-10-11T09:50:27.883 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.b] 2023-10-11T09:50:27.883 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.d] 2023-10-11T09:50:27.883 INFO:teuthology.orchestra.run.smithi078.stdout:[mds] 2023-10-11T09:50:27.883 INFO:teuthology.orchestra.run.smithi078.stdout: debug mds = 20 2023-10-11T09:50:27.883 INFO:teuthology.orchestra.run.smithi078.stdout: debug mds balancer = 20 2023-10-11T09:50:27.884 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:50:27.884 INFO:teuthology.orchestra.run.smithi078.stdout: mds debug frag = True 2023-10-11T09:50:27.884 INFO:teuthology.orchestra.run.smithi078.stdout: mds debug scatterstat = False 2023-10-11T09:50:27.884 INFO:teuthology.orchestra.run.smithi078.stdout: mds op complaint time = 180 2023-10-11T09:50:27.884 INFO:teuthology.orchestra.run.smithi078.stdout: mds verify scatter = False 2023-10-11T09:50:27.885 INFO:teuthology.orchestra.run.smithi078.stdout: osd op complaint time = 180 2023-10-11T09:50:27.885 INFO:teuthology.orchestra.run.smithi078.stdout: rados mon op timeout = 900 2023-10-11T09:50:27.885 INFO:teuthology.orchestra.run.smithi078.stdout: rados osd op timeout = 900 2023-10-11T09:50:27.886 INFO:teuthology.orchestra.run.smithi133.stdout:[global] 2023-10-11T09:50:27.886 INFO:teuthology.orchestra.run.smithi133.stdout: chdir = "" 2023-10-11T09:50:27.886 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-11T09:50:27.886 INFO:teuthology.orchestra.run.smithi133.stdout: auth supported = cephx 2023-10-11T09:50:27.887 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.887 INFO:teuthology.orchestra.run.smithi133.stdout: filestore xattr use omap = true 2023-10-11T09:50:27.887 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.887 INFO:teuthology.orchestra.run.smithi133.stdout: mon clock drift allowed = 1.000 2023-10-11T09:50:27.887 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.888 INFO:teuthology.orchestra.run.smithi133.stdout: osd crush chooseleaf type = 0 2023-10-11T09:50:27.888 INFO:teuthology.orchestra.run.smithi133.stdout: auth debug = true 2023-10-11T09:50:27.888 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.888 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on old message = true 2023-10-11T09:50:27.888 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on bug = true 2023-10-11T09:50:27.889 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.889 INFO:teuthology.orchestra.run.smithi133.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-11T09:50:27.889 INFO:teuthology.orchestra.run.smithi133.stdout: mon pg warn max object skew = 0 2023-10-11T09:50:27.889 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.889 INFO:teuthology.orchestra.run.smithi133.stdout: # disable pg_autoscaler by default for new pools 2023-10-11T09:50:27.890 INFO:teuthology.orchestra.run.smithi133.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-11T09:50:27.890 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.890 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default size = 2 2023-10-11T09:50:27.890 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.890 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow primary affinity = true 2023-10-11T09:50:27.890 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow pg remap = true 2023-10-11T09:50:27.891 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on legacy crush tunables = false 2023-10-11T09:50:27.891 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on crush straw calc version zero = false 2023-10-11T09:50:27.891 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on no sortbitwise = false 2023-10-11T09:50:27.891 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on osd down out interval zero = false 2023-10-11T09:50:27.891 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on too few osds = false 2023-10-11T09:50:27.892 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-11T09:50:27.892 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-11T09:50:27.892 INFO:teuthology.orchestra.run.smithi133.stdout: mon_allow_pool_size_one = true 2023-10-11T09:50:27.892 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.892 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-11T09:50:27.893 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.893 INFO:teuthology.orchestra.run.smithi133.stdout: osd default data pool replay window = 5 2023-10-11T09:50:27.893 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.893 INFO:teuthology.orchestra.run.smithi133.stdout: mon allow pool delete = true 2023-10-11T09:50:27.893 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.893 INFO:teuthology.orchestra.run.smithi133.stdout: mon cluster log file level = debug 2023-10-11T09:50:27.894 INFO:teuthology.orchestra.run.smithi133.stdout: debug asserts on shutdown = true 2023-10-11T09:50:27.894 INFO:teuthology.orchestra.run.smithi133.stdout: mon health detail to clog = false 2023-10-11T09:50:27.894 INFO:teuthology.orchestra.run.smithi133.stdout: mon host = "172.21.15.78,172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790]" 2023-10-11T09:50:27.894 INFO:teuthology.orchestra.run.smithi133.stdout: fsid = dee99db9-7a72-406f-a1e2-917943f2b13d 2023-10-11T09:50:27.894 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.895 INFO:teuthology.orchestra.run.smithi133.stdout:[osd] 2023-10-11T09:50:27.895 INFO:teuthology.orchestra.run.smithi133.stdout: osd journal size = 100 2023-10-11T09:50:27.895 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.895 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub load threshold = 5.0 2023-10-11T09:50:27.895 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub max interval = 600 2023-10-11T09:50:27.896 INFO:teuthology.orchestra.run.smithi133.stdout: osd mclock profile = high_recovery_ops 2023-10-11T09:50:27.896 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.896 INFO:teuthology.orchestra.run.smithi133.stdout: osd recover clone overlap = true 2023-10-11T09:50:27.896 INFO:teuthology.orchestra.run.smithi133.stdout: osd recovery max chunk = 1048576 2023-10-11T09:50:27.896 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.896 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug shutdown = true 2023-10-11T09:50:27.897 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug op order = true 2023-10-11T09:50:27.897 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug verify stray on activate = true 2023-10-11T09:50:27.897 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.897 INFO:teuthology.orchestra.run.smithi133.stdout: osd open classes on start = true 2023-10-11T09:50:27.897 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug pg log writeout = true 2023-10-11T09:50:27.898 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.898 INFO:teuthology.orchestra.run.smithi133.stdout: osd deep scrub update digest min age = 30 2023-10-11T09:50:27.898 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.898 INFO:teuthology.orchestra.run.smithi133.stdout: osd map max advance = 10 2023-10-11T09:50:27.898 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.899 INFO:teuthology.orchestra.run.smithi133.stdout: journal zero on create = true 2023-10-11T09:50:27.899 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.899 INFO:teuthology.orchestra.run.smithi133.stdout: filestore ondisk finisher threads = 3 2023-10-11T09:50:27.899 INFO:teuthology.orchestra.run.smithi133.stdout: filestore apply finisher threads = 3 2023-10-11T09:50:27.899 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.900 INFO:teuthology.orchestra.run.smithi133.stdout: bdev debug aio = true 2023-10-11T09:50:27.900 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug misdirected ops = true 2023-10-11T09:50:27.900 INFO:teuthology.orchestra.run.smithi133.stdout: bdev async discard = True 2023-10-11T09:50:27.900 INFO:teuthology.orchestra.run.smithi133.stdout: bdev enable discard = True 2023-10-11T09:50:27.900 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore allocator = bitmap 2023-10-11T09:50:27.900 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore block size = 96636764160 2023-10-11T09:50:27.901 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore fsck on mount = True 2023-10-11T09:50:27.901 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluefs = 1/20 2023-10-11T09:50:27.901 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluestore = 1/20 2023-10-11T09:50:27.901 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:50:27.901 INFO:teuthology.orchestra.run.smithi133.stdout: debug osd = 20 2023-10-11T09:50:27.902 INFO:teuthology.orchestra.run.smithi133.stdout: debug rocksdb = 4/10 2023-10-11T09:50:27.902 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd backfillfull_ratio = 0.85 2023-10-11T09:50:27.902 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd full ratio = 0.9 2023-10-11T09:50:27.902 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd nearfull ratio = 0.8 2023-10-11T09:50:27.902 INFO:teuthology.orchestra.run.smithi133.stdout: osd failsafe full ratio = 0.95 2023-10-11T09:50:27.903 INFO:teuthology.orchestra.run.smithi133.stdout: osd objectstore = bluestore 2023-10-11T09:50:27.903 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2023-10-11T09:50:27.903 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.903 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr] 2023-10-11T09:50:27.903 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:50:27.904 INFO:teuthology.orchestra.run.smithi133.stdout: debug mgr = 20 2023-10-11T09:50:27.904 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2023-10-11T09:50:27.904 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2023-10-11T09:50:27.904 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:50:27.904 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:50:27.904 INFO:teuthology.orchestra.run.smithi133.stdout: mgr/telemetry/nag = false 2023-10-11T09:50:27.905 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.905 INFO:teuthology.orchestra.run.smithi133.stdout:[mon] 2023-10-11T09:50:27.905 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:50:27.905 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2023-10-11T09:50:27.905 INFO:teuthology.orchestra.run.smithi133.stdout: debug paxos = 20 2023-10-11T09:50:27.906 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2023-10-11T09:50:27.906 INFO:teuthology.orchestra.run.smithi133.stdout: mon data avail warn = 5 2023-10-11T09:50:27.906 INFO:teuthology.orchestra.run.smithi133.stdout: mon mgr mkfs grace = 240 2023-10-11T09:50:27.906 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:50:27.906 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd reporter subtree level = osd 2023-10-11T09:50:27.907 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd prime pg temp = true 2023-10-11T09:50:27.907 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:50:27.907 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.907 INFO:teuthology.orchestra.run.smithi133.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-11T09:50:27.907 INFO:teuthology.orchestra.run.smithi133.stdout: auth mon ticket ttl = 660 # 11m 2023-10-11T09:50:27.908 INFO:teuthology.orchestra.run.smithi133.stdout: auth service ticket ttl = 240 # 4m 2023-10-11T09:50:27.908 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.908 INFO:teuthology.orchestra.run.smithi133.stdout: # don't complain about insecure global_id in the test suite 2023-10-11T09:50:27.908 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-11T09:50:27.908 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-11T09:50:27.908 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.909 INFO:teuthology.orchestra.run.smithi133.stdout: # 1m isn't quite enough 2023-10-11T09:50:27.909 INFO:teuthology.orchestra.run.smithi133.stdout: mon_down_mkfs_grace = 2m 2023-10-11T09:50:27.909 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.909 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_filestore_osds = false 2023-10-11T09:50:27.909 INFO:teuthology.orchestra.run.smithi133.stdout: mon op complaint time = 120 2023-10-11T09:50:27.910 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:50:27.910 INFO:teuthology.orchestra.run.smithi133.stdout:[client] 2023-10-11T09:50:27.910 INFO:teuthology.orchestra.run.smithi133.stdout: rgw cache enabled = true 2023-10-11T09:50:27.910 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable ops log = true 2023-10-11T09:50:27.910 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable usage log = true 2023-10-11T09:50:27.911 INFO:teuthology.orchestra.run.smithi133.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-11T09:50:27.911 INFO:teuthology.orchestra.run.smithi133.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-11T09:50:27.911 INFO:teuthology.orchestra.run.smithi133.stdout: client mount timeout = 600 2023-10-11T09:50:27.911 INFO:teuthology.orchestra.run.smithi133.stdout: debug client = 20 2023-10-11T09:50:27.911 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:50:27.911 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = "" 2023-10-11T09:50:27.912 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2023-10-11T09:50:27.912 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2023-10-11T09:50:27.912 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.a] 2023-10-11T09:50:27.912 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.b] 2023-10-11T09:50:27.912 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.c] 2023-10-11T09:50:27.913 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.a] 2023-10-11T09:50:27.913 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.c] 2023-10-11T09:50:27.913 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.b] 2023-10-11T09:50:27.913 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.d] 2023-10-11T09:50:27.913 INFO:teuthology.orchestra.run.smithi133.stdout:[mds] 2023-10-11T09:50:27.914 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds = 20 2023-10-11T09:50:27.914 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds balancer = 20 2023-10-11T09:50:27.914 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:50:27.914 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug frag = True 2023-10-11T09:50:27.914 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug scatterstat = False 2023-10-11T09:50:27.915 INFO:teuthology.orchestra.run.smithi133.stdout: mds op complaint time = 180 2023-10-11T09:50:27.915 INFO:teuthology.orchestra.run.smithi133.stdout: mds verify scatter = False 2023-10-11T09:50:27.915 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2023-10-11T09:50:27.915 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2023-10-11T09:50:27.915 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2023-10-11T09:50:27.917 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph purge cephfs_metadata --yes-i-really-really-mean-it 2023-10-11T09:50:27.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.982+0000 7fc2c05ee140 1 Processor -- start 2023-10-11T09:50:27.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.982+0000 7fc2c05ee140 1 -- start start 2023-10-11T09:50:27.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.982+0000 7fc2c05ee140 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2a80039b0 0x55ab93146180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:27.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.982+0000 7fc2c05ee140 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab931cef70 0x55ab931cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:27.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.982+0000 7fc2c05ee140 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab9313e9a0 0x55ab9313eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:27.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.982+0000 7fc2c05ee140 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab9308ece0 con 0x55ab931cef70 2023-10-11T09:50:27.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.982+0000 7fc2c05ee140 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab9304b250 con 0x55ab9313e9a0 2023-10-11T09:50:27.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.982+0000 7fc2c05ee140 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ab9304aef0 con 0x7fc2a80039b0 2023-10-11T09:50:27.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.986+0000 7fc2bd922700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab931cef70 0x55ab931cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:27.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.986+0000 7fc2bd922700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab931cef70 0x55ab931cf340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35752/0 (socket says 172.21.15.78:35752) 2023-10-11T09:50:27.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.986+0000 7fc2bd922700 1 -- 172.21.15.78:0/834354702 learned_addr learned my addr 172.21.15.78:0/834354702 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:27.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.986+0000 7fc2be123700 1 --2- 172.21.15.78:0/834354702 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2a80039b0 0x55ab93146180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:27.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.986+0000 7fc2be924700 1 --2- 172.21.15.78:0/834354702 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab9313e9a0 0x55ab9313eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:27.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.986+0000 7fc2bd922700 1 -- 172.21.15.78:0/834354702 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2a80039b0 msgr2=0x55ab93146180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:27.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.986+0000 7fc2bd922700 1 --2- 172.21.15.78:0/834354702 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2a80039b0 0x55ab93146180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:27.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.986+0000 7fc2bd922700 1 -- 172.21.15.78:0/834354702 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab9313e9a0 msgr2=0x55ab9313eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:27.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.986+0000 7fc2bd922700 1 --2- 172.21.15.78:0/834354702 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab9313e9a0 0x55ab9313eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:27.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.986+0000 7fc2bd922700 1 -- 172.21.15.78:0/834354702 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab9302c8b0 con 0x55ab931cef70 2023-10-11T09:50:27.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.986+0000 7fc2be123700 1 --2- 172.21.15.78:0/834354702 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2a80039b0 0x55ab93146180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:50:27.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.986+0000 7fc2be924700 1 --2- 172.21.15.78:0/834354702 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab9313e9a0 0x55ab9313eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:50:27.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.986+0000 7fc2bd922700 1 --2- 172.21.15.78:0/834354702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab931cef70 0x55ab931cf340 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4002fb0 tx=0x7fc2b400e5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=164160f36382f95e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:27.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.986+0000 7fc2bc920700 1 -- 172.21.15.78:0/834354702 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b400ece0 con 0x55ab931cef70 2023-10-11T09:50:27.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.986+0000 7fc2bc920700 1 -- 172.21.15.78:0/834354702 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b4008030 con 0x55ab931cef70 2023-10-11T09:50:27.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.986+0000 7fc2bc920700 1 -- 172.21.15.78:0/834354702 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b4011450 con 0x55ab931cef70 2023-10-11T09:50:27.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.986+0000 7fc2c05ee140 1 -- 172.21.15.78:0/834354702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab931cef70 msgr2=0x55ab931cf340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:28.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.986+0000 7fc2c05ee140 1 --2- 172.21.15.78:0/834354702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab931cef70 0x55ab931cf340 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4002fb0 tx=0x7fc2b400e5b0 comp rx=0 tx=0).stop 2023-10-11T09:50:28.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.986+0000 7fc2c05ee140 1 -- 172.21.15.78:0/834354702 shutdown_connections 2023-10-11T09:50:28.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.986+0000 7fc2c05ee140 1 --2- 172.21.15.78:0/834354702 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab9313e9a0 0x55ab9313eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:28.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.986+0000 7fc2c05ee140 1 --2- 172.21.15.78:0/834354702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab931cef70 0x55ab931cf340 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:28.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.986+0000 7fc2c05ee140 1 --2- 172.21.15.78:0/834354702 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2a80039b0 0x55ab93146180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:28.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.986+0000 7fc2c05ee140 1 -- 172.21.15.78:0/834354702 >> 172.21.15.78:0/834354702 conn(0x55ab93100be0 msgr2=0x55ab931d2160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:28.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.986+0000 7fc2c05ee140 1 -- 172.21.15.78:0/834354702 shutdown_connections 2023-10-11T09:50:28.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.986+0000 7fc2c05ee140 1 -- 172.21.15.78:0/834354702 wait complete. 2023-10-11T09:50:28.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.990+0000 7fc2c05ee140 1 Processor -- start 2023-10-11T09:50:28.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.990+0000 7fc2c05ee140 1 -- start start 2023-10-11T09:50:28.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.990+0000 7fc2c05ee140 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab9313e9a0 0x55ab931bc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:28.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.990+0000 7fc2c05ee140 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab931cef70 0x55ab9313ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:28.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.990+0000 7fc2c05ee140 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2a80039b0 0x55ab9313f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:28.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.990+0000 7fc2c05ee140 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab93021240 con 0x55ab931cef70 2023-10-11T09:50:28.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.990+0000 7fc2c05ee140 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab9326a5e0 con 0x55ab9313e9a0 2023-10-11T09:50:28.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.990+0000 7fc2c05ee140 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ab9326a880 con 0x7fc2a80039b0 2023-10-11T09:50:28.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.990+0000 7fc2bd922700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab931cef70 0x55ab9313ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:28.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.990+0000 7fc2bd922700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab931cef70 0x55ab9313ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35758/0 (socket says 172.21.15.78:35758) 2023-10-11T09:50:28.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.990+0000 7fc2bd922700 1 -- 172.21.15.78:0/3498686889 learned_addr learned my addr 172.21.15.78:0/3498686889 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:28.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.990+0000 7fc2be123700 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab9313e9a0 0x55ab931bc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:28.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.990+0000 7fc2be924700 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2a80039b0 0x55ab9313f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:28.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.990+0000 7fc2bd922700 1 -- 172.21.15.78:0/3498686889 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2a80039b0 msgr2=0x55ab9313f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:28.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.990+0000 7fc2bd922700 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2a80039b0 0x55ab9313f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:28.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.990+0000 7fc2bd922700 1 -- 172.21.15.78:0/3498686889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab9313e9a0 msgr2=0x55ab931bc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:28.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.990+0000 7fc2bd922700 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab9313e9a0 0x55ab931bc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:28.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.990+0000 7fc2bd922700 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b400a040 con 0x55ab931cef70 2023-10-11T09:50:28.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.990+0000 7fc2bd922700 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab931cef70 0x55ab9313ed70 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fc2b400e9b0 tx=0x7fc2b4001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:28.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.990+0000 7fc2af7fe700 1 -- 172.21.15.78:0/3498686889 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b4021070 con 0x55ab931cef70 2023-10-11T09:50:28.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.990+0000 7fc2af7fe700 1 -- 172.21.15.78:0/3498686889 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b401a420 con 0x55ab931cef70 2023-10-11T09:50:28.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.990+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ab9302c310 con 0x55ab931cef70 2023-10-11T09:50:28.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.990+0000 7fc2af7fe700 1 -- 172.21.15.78:0/3498686889 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b401a7d0 con 0x55ab931cef70 2023-10-11T09:50:28.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.990+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ab9326adc0 con 0x55ab931cef70 2023-10-11T09:50:28.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.994+0000 7fc2af7fe700 1 -- 172.21.15.78:0/3498686889 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fc2b401a930 con 0x55ab931cef70 2023-10-11T09:50:28.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.994+0000 7fc2af7fe700 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc2a0064830 0x7fc2a0066ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:28.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.994+0000 7fc2af7fe700 1 -- 172.21.15.78:0/3498686889 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6552+0+0 (secure 0 0 0) 0x7fc2b4083580 con 0x55ab931cef70 2023-10-11T09:50:28.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.994+0000 7fc2be123700 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc2a0064830 0x7fc2a0066ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:28.007 INFO:teuthology.orchestra.run.smithi078.stdout:Warning: using slow linear search 2023-10-11T09:50:28.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.994+0000 7fc2c05ee140 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55ab932727e0 0x55ab93274bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:28.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.994+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:1 14.0 14:00000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ab93275100 con 0x55ab932727e0 2023-10-11T09:50:28.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.998+0000 7fc2be924700 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55ab932727e0 0x55ab93274bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:28.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.998+0000 7fc2be123700 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc2a0064830 0x7fc2a0066ce0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0000c00 tx=0x7fc2b0000fb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:28.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.998+0000 7fc2be924700 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55ab932727e0 0x55ab93274bc0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:28.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.998+0000 7fc2be924700 1 -- 172.21.15.78:0/3498686889 <== osd.2 v2:172.21.15.78:6808/35732 1 ==== osd_op_reply(1 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc2b8018070 con 0x55ab932727e0 2023-10-11T09:50:28.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.998+0000 7fc2be924700 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55ab930fa6b0 0x7fc2b801c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:28.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.998+0000 7fc2be924700 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:2 14.10 14:08000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ab93275100 con 0x55ab930fa6b0 2023-10-11T09:50:28.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.998+0000 7fc2be123700 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55ab930fa6b0 0x7fc2b801c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:28.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:27.998+0000 7fc2be123700 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55ab930fa6b0 0x7fc2b801c940 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:28.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.006+0000 7fc2be123700 1 -- 172.21.15.78:0/3498686889 <== osd.1 v2:172.21.15.78:6809/35734 1 ==== osd_op_reply(2 [pgnls start_epoch 109 out=177b] v108'10 uv10 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x7fc2b000e510 con 0x55ab930fa6b0 2023-10-11T09:50:28.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.006+0000 7fc2be123700 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x7fc2b0010970 0x7fc2b001f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:28.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.006+0000 7fc2be123700 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:3 14.8 14:10000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ab93275100 con 0x7fc2b0010970 2023-10-11T09:50:28.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.006+0000 7fc2bd922700 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x7fc2b0010970 0x7fc2b001f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:28.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.006+0000 7fc2bd922700 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x7fc2b0010970 0x7fc2b001f440 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:28.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.006+0000 7fc2bd922700 1 -- 172.21.15.78:0/3498686889 <== osd.0 v2:172.21.15.78:6800/35735 1 ==== osd_op_reply(3 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc2b4017070 con 0x7fc2b0010970 2023-10-11T09:50:28.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.006+0000 7fc2bd922700 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:4 14.18 14:18000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ab93275100 con 0x55ab930fa6b0 2023-10-11T09:50:28.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.006+0000 7fc2be123700 1 -- 172.21.15.78:0/3498686889 <== osd.1 v2:172.21.15.78:6809/35734 2 ==== osd_op_reply(4 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc2b000e9e0 con 0x55ab930fa6b0 2023-10-11T09:50:28.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.006+0000 7fc2be123700 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x7fc2b001d930 0x7fc2b000d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:28.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.006+0000 7fc2be123700 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:5 14.4 14:20000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ab93275100 con 0x7fc2b001d930 2023-10-11T09:50:28.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.006+0000 7fc2be924700 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x7fc2b001d930 0x7fc2b000d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:28.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.010+0000 7fc2be924700 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x7fc2b001d930 0x7fc2b000d3d0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:28.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.010+0000 7fc2be924700 1 -- 172.21.15.78:0/3498686889 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== osd_op_reply(5 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc2b8026040 con 0x7fc2b001d930 2023-10-11T09:50:28.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.010+0000 7fc2be924700 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x7fc2b802c040 0x7fc2b801b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:28.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.010+0000 7fc2be924700 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:6 14.14 14:28000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ab93275100 con 0x7fc2b802c040 2023-10-11T09:50:28.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.010+0000 7fc2be123700 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x7fc2b802c040 0x7fc2b801b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:28.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.010+0000 7fc2be123700 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x7fc2b802c040 0x7fc2b801b930 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:28.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.018+0000 7fc2be123700 1 -- 172.21.15.78:0/3498686889 <== osd.4 v2:172.21.15.133:6808/35898 1 ==== osd_op_reply(6 [pgnls start_epoch 109 out=149b] v108'44 uv44 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x7fc2b000e9e0 con 0x7fc2b802c040 2023-10-11T09:50:28.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.018+0000 7fc2be123700 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:7 14.c 14:30000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ab93275100 con 0x7fc2b0010970 2023-10-11T09:50:28.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.018+0000 7fc2bd922700 1 -- 172.21.15.78:0/3498686889 <== osd.0 v2:172.21.15.78:6800/35735 2 ==== osd_op_reply(7 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc2b4017070 con 0x7fc2b0010970 2023-10-11T09:50:28.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.018+0000 7fc2bd922700 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:8 14.1c 14:38000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ab93275100 con 0x7fc2b0010970 2023-10-11T09:50:28.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.018+0000 7fc2bd922700 1 -- 172.21.15.78:0/3498686889 <== osd.0 v2:172.21.15.78:6800/35735 3 ==== osd_op_reply(8 [pgnls start_epoch 109 out=73b] v104'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fc2b4017070 con 0x7fc2b0010970 2023-10-11T09:50:28.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.018+0000 7fc2bd922700 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:9 14.2 14:40000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ab93275100 con 0x7fc2b001d930 2023-10-11T09:50:28.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.022+0000 7fc2be924700 1 -- 172.21.15.78:0/3498686889 <== osd.6 v2:172.21.15.133:6800/35900 2 ==== osd_op_reply(9 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc2b8026040 con 0x7fc2b001d930 2023-10-11T09:50:28.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.022+0000 7fc2be924700 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:10 14.12 14:48000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ab93275100 con 0x55ab932727e0 2023-10-11T09:50:28.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.022+0000 7fc2be924700 1 -- 172.21.15.78:0/3498686889 <== osd.2 v2:172.21.15.78:6808/35732 2 ==== osd_op_reply(10 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc2b8026040 con 0x55ab932727e0 2023-10-11T09:50:28.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.022+0000 7fc2be924700 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55ab930faa80 0x7fc2b802c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:28.030 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.022+0000 7fc2be924700 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:11 14.a 14:50000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ab93275100 con 0x55ab930faa80 2023-10-11T09:50:28.030 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.022+0000 7fc2bd922700 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55ab930faa80 0x7fc2b802c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:28.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.022+0000 7fc2bd922700 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55ab930faa80 0x7fc2b802c920 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:28.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.026+0000 7fc2bd922700 1 -- 172.21.15.78:0/3498686889 <== osd.5 v2:172.21.15.133:6816/35899 1 ==== osd_op_reply(11 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc2b4017070 con 0x55ab930faa80 2023-10-11T09:50:28.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.026+0000 7fc2bd922700 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:12 14.1a 14:58000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ab93275100 con 0x7fc2b0010970 2023-10-11T09:50:28.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.030+0000 7fc2bd922700 1 -- 172.21.15.78:0/3498686889 <== osd.0 v2:172.21.15.78:6800/35735 4 ==== osd_op_reply(12 [pgnls start_epoch 109 out=49b] v104'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc2b4017070 con 0x7fc2b0010970 2023-10-11T09:50:28.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.030+0000 7fc2bd922700 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:13 14.6 14:60000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ab93275100 con 0x7fc2b0010970 2023-10-11T09:50:28.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.030+0000 7fc2bd922700 1 -- 172.21.15.78:0/3498686889 <== osd.0 v2:172.21.15.78:6800/35735 5 ==== osd_op_reply(13 [pgnls start_epoch 109 out=77b] v108'7 uv7 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x7fc2b4017070 con 0x7fc2b0010970 2023-10-11T09:50:28.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.030+0000 7fc2bd922700 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x7fc2b401abe0 0x7fc2b40535f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:28.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.030+0000 7fc2bd922700 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:14 14.16 14:68000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ab93275100 con 0x7fc2b401abe0 2023-10-11T09:50:28.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.030+0000 7fc2be924700 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x7fc2b401abe0 0x7fc2b40535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:28.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.030+0000 7fc2be924700 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x7fc2b401abe0 0x7fc2b40535f0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:28.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.038+0000 7fc2be924700 1 -- 172.21.15.78:0/3498686889 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(14 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc2b8026040 con 0x7fc2b401abe0 2023-10-11T09:50:28.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.038+0000 7fc2be924700 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:15 14.e 14:70000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ab93275100 con 0x7fc2b802c040 2023-10-11T09:50:28.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.038+0000 7fc2be123700 1 -- 172.21.15.78:0/3498686889 <== osd.4 v2:172.21.15.133:6808/35898 2 ==== osd_op_reply(15 [pgnls start_epoch 109 out=73b] v104'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fc2b000e9e0 con 0x7fc2b802c040 2023-10-11T09:50:28.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.038+0000 7fc2be123700 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x7fc2b0032970 0x7fc2b00313d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:28.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.038+0000 7fc2be123700 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:16 14.1e 14:78000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ab93275100 con 0x7fc2b0032970 2023-10-11T09:50:28.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.038+0000 7fc2be123700 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x7fc2b0032970 0x7fc2b00313d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:28.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.038+0000 7fc2be123700 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x7fc2b0032970 0x7fc2b00313d0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:28.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.042+0000 7fc2be123700 1 -- 172.21.15.78:0/3498686889 <== osd.7 v2:172.21.15.133:6824/35901 1 ==== osd_op_reply(16 [pgnls start_epoch 109 out=49b] v104'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc2b000e9e0 con 0x7fc2b0032970 2023-10-11T09:50:28.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.042+0000 7fc2be123700 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:17 14.1 14:80000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ab93275100 con 0x7fc2b401abe0 2023-10-11T09:50:28.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.042+0000 7fc2be924700 1 -- 172.21.15.78:0/3498686889 <== osd.3 v2:172.21.15.78:6824/35738 2 ==== osd_op_reply(17 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc2b8026040 con 0x7fc2b401abe0 2023-10-11T09:50:28.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.042+0000 7fc2be924700 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:18 14.11 14:88000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ab93275100 con 0x7fc2b802c040 2023-10-11T09:50:28.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.042+0000 7fc2be123700 1 -- 172.21.15.78:0/3498686889 <== osd.4 v2:172.21.15.133:6808/35898 3 ==== osd_op_reply(18 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc2b000e9e0 con 0x7fc2b802c040 2023-10-11T09:50:28.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.042+0000 7fc2be123700 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:19 14.9 14:90000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ab93275100 con 0x55ab930faa80 2023-10-11T09:50:28.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.042+0000 7fc2bd922700 1 -- 172.21.15.78:0/3498686889 <== osd.5 v2:172.21.15.133:6816/35899 2 ==== osd_op_reply(19 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc2b4081040 con 0x55ab930faa80 2023-10-11T09:50:28.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.042+0000 7fc2bd922700 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:20 14.19 14:98000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ab93275100 con 0x7fc2b001d930 2023-10-11T09:50:28.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.046+0000 7fc2be924700 1 -- 172.21.15.78:0/3498686889 <== osd.6 v2:172.21.15.133:6800/35900 3 ==== osd_op_reply(20 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc2b8026040 con 0x7fc2b001d930 2023-10-11T09:50:28.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.046+0000 7fc2be924700 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:21 14.5 14:a0000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ab93275100 con 0x7fc2b0010970 2023-10-11T09:50:28.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.046+0000 7fc2bd922700 1 -- 172.21.15.78:0/3498686889 <== osd.0 v2:172.21.15.78:6800/35735 6 ==== osd_op_reply(21 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc2b4081040 con 0x7fc2b0010970 2023-10-11T09:50:28.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.046+0000 7fc2bd922700 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:22 14.15 14:a8000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ab93275100 con 0x7fc2b0032970 2023-10-11T09:50:28.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.046+0000 7fc2be123700 1 -- 172.21.15.78:0/3498686889 <== osd.7 v2:172.21.15.133:6824/35901 2 ==== osd_op_reply(22 [pgnls start_epoch 109 out=97b] v104'3 uv3 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x7fc2b000e9e0 con 0x7fc2b0032970 2023-10-11T09:50:28.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.046+0000 7fc2be123700 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:23 14.d 14:b0000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ab93275100 con 0x7fc2b001d930 2023-10-11T09:50:28.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.046+0000 7fc2be924700 1 -- 172.21.15.78:0/3498686889 <== osd.6 v2:172.21.15.133:6800/35900 4 ==== osd_op_reply(23 [pgnls start_epoch 109 out=74b] v104'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fc2b8026040 con 0x7fc2b001d930 2023-10-11T09:50:28.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.046+0000 7fc2be924700 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:24 14.1d 14:b8000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ab93275100 con 0x7fc2b0010970 2023-10-11T09:50:28.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.050+0000 7fc2bd922700 1 -- 172.21.15.78:0/3498686889 <== osd.0 v2:172.21.15.78:6800/35735 7 ==== osd_op_reply(24 [pgnls start_epoch 109 out=73b] v104'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fc2b4081040 con 0x7fc2b0010970 2023-10-11T09:50:28.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.050+0000 7fc2bd922700 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:25 14.3 14:c0000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ab93275100 con 0x7fc2b0010970 2023-10-11T09:50:28.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.050+0000 7fc2bd922700 1 -- 172.21.15.78:0/3498686889 <== osd.0 v2:172.21.15.78:6800/35735 8 ==== osd_op_reply(25 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc2b4081040 con 0x7fc2b0010970 2023-10-11T09:50:28.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.050+0000 7fc2bd922700 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:26 14.13 14:c8000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ab93275100 con 0x55ab932727e0 2023-10-11T09:50:28.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.050+0000 7fc2be924700 1 -- 172.21.15.78:0/3498686889 <== osd.2 v2:172.21.15.78:6808/35732 3 ==== osd_op_reply(26 [pgnls start_epoch 109 out=98b] v108'3 uv3 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x7fc2b8026040 con 0x55ab932727e0 2023-10-11T09:50:28.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.050+0000 7fc2be924700 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:27 14.b 14:d0000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ab93275100 con 0x7fc2b0010970 2023-10-11T09:50:28.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.050+0000 7fc2bd922700 1 -- 172.21.15.78:0/3498686889 <== osd.0 v2:172.21.15.78:6800/35735 9 ==== osd_op_reply(27 [pgnls start_epoch 109 out=76b] v108'3 uv3 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x7fc2b4081040 con 0x7fc2b0010970 2023-10-11T09:50:28.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.050+0000 7fc2bd922700 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:28 14.1b 14:d8000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ab93275100 con 0x55ab930fa6b0 2023-10-11T09:50:28.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.050+0000 7fc2be123700 1 -- 172.21.15.78:0/3498686889 <== osd.1 v2:172.21.15.78:6809/35734 3 ==== osd_op_reply(28 [pgnls start_epoch 109 out=49b] v104'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc2b000e9e0 con 0x55ab930fa6b0 2023-10-11T09:50:28.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.050+0000 7fc2be123700 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:29 14.7 14:e0000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ab93275100 con 0x55ab930faa80 2023-10-11T09:50:28.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.054+0000 7fc2bd922700 1 -- 172.21.15.78:0/3498686889 <== osd.5 v2:172.21.15.133:6816/35899 3 ==== osd_op_reply(29 [pgnls start_epoch 109 out=73b] v104'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fc2b4081040 con 0x55ab930faa80 2023-10-11T09:50:28.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.054+0000 7fc2bd922700 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:30 14.17 14:e8000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ab93275100 con 0x55ab930fa6b0 2023-10-11T09:50:28.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.054+0000 7fc2be123700 1 -- 172.21.15.78:0/3498686889 <== osd.1 v2:172.21.15.78:6809/35734 4 ==== osd_op_reply(30 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc2b000e9e0 con 0x55ab930fa6b0 2023-10-11T09:50:28.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.054+0000 7fc2be123700 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:31 14.f 14:f0000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ab93275100 con 0x7fc2b0032970 2023-10-11T09:50:28.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.054+0000 7fc2be123700 1 -- 172.21.15.78:0/3498686889 <== osd.7 v2:172.21.15.133:6824/35901 3 ==== osd_op_reply(31 [pgnls start_epoch 109 out=73b] v104'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fc2b000e9e0 con 0x7fc2b0032970 2023-10-11T09:50:28.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.054+0000 7fc2be123700 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:32 14.1f 14:f8000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55ab93275100 con 0x7fc2b0010970 2023-10-11T09:50:28.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.058+0000 7fc2bd922700 1 -- 172.21.15.78:0/3498686889 <== osd.0 v2:172.21.15.78:6800/35735 10 ==== osd_op_reply(32 [pgnls start_epoch 109 out=101b] v108'6 uv6 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x7fc2b4081040 con 0x7fc2b0010970 2023-10-11T09:50:28.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.058+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:33 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55ab931b9de0 con 0x55ab930fa6b0 2023-10-11T09:50:28.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.058+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:34 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55ab931ba6b0 con 0x55ab930fa6b0 2023-10-11T09:50:28.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.058+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:35 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55ab932759a0 con 0x55ab930fa6b0 2023-10-11T09:50:28.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.058+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:36 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55ab93276240 con 0x55ab930fa6b0 2023-10-11T09:50:28.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.058+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:37 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55ab93276ae0 con 0x55ab930fa6b0 2023-10-11T09:50:28.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.058+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:38 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55ab93277380 con 0x7fc2b802c040 2023-10-11T09:50:28.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.058+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55ab93277c20 con 0x7fc2b802c040 2023-10-11T09:50:28.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.058+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:40 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55ab932784c0 con 0x7fc2b802c040 2023-10-11T09:50:28.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.058+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:41 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55ab93278d60 con 0x7fc2b802c040 2023-10-11T09:50:28.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.058+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:42 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55ab93279600 con 0x7fc2b0010970 2023-10-11T09:50:28.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.058+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:43 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55ab93279ea0 con 0x7fc2b0010970 2023-10-11T09:50:28.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.058+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:44 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55ab9327a8b0 con 0x7fc2b802c040 2023-10-11T09:50:28.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.058+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:45 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55ab9327b280 con 0x7fc2b0032970 2023-10-11T09:50:28.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.058+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:46 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55ab93278d60 con 0x7fc2b0032970 2023-10-11T09:50:28.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.058+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:47 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55ab9327c290 con 0x7fc2b001d930 2023-10-11T09:50:28.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.058+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:48 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55ab9327b280 con 0x7fc2b0010970 2023-10-11T09:50:28.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.062+0000 7fc2bd922700 1 -- 172.21.15.78:0/3498686889 <== osd.0 v2:172.21.15.78:6800/35735 11 ==== osd_op_reply(43 mds0_openfiles.0 [delete] v109'8 uv8 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fc2b4081040 con 0x7fc2b0010970 2023-10-11T09:50:28.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.062+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:49 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55ab93278d60 con 0x55ab932727e0 2023-10-11T09:50:28.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.062+0000 7fc2bd922700 1 -- 172.21.15.78:0/3498686889 <== osd.0 v2:172.21.15.78:6800/35735 12 ==== osd_op_reply(48 605.00000000 [delete] v109'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fc2b4081040 con 0x7fc2b0010970 2023-10-11T09:50:28.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.062+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:50 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55ab9327cb80 con 0x55ab932727e0 2023-10-11T09:50:28.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.062+0000 7fc2be123700 1 -- 172.21.15.78:0/3498686889 <== osd.4 v2:172.21.15.133:6808/35898 4 ==== osd_op_reply(44 608.00000000 [delete] v109'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fc2b000e9e0 con 0x7fc2b802c040 2023-10-11T09:50:28.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.062+0000 7fc2be123700 1 -- 172.21.15.78:0/3498686889 <== osd.1 v2:172.21.15.78:6809/35734 5 ==== osd_op_reply(33 601.00000000 [delete] v109'11 uv11 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fc2b000e9e0 con 0x55ab930fa6b0 2023-10-11T09:50:28.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.062+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:51 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55ab93269900 con 0x7fc2b0010970 2023-10-11T09:50:28.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.062+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:52 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55ab9327d470 con 0x55ab930faa80 2023-10-11T09:50:28.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.062+0000 7fc2be123700 1 -- 172.21.15.78:0/3498686889 <== osd.4 v2:172.21.15.133:6808/35898 5 ==== osd_op_reply(38 1.00000000.inode [delete] v109'45 uv45 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fc2b000e9e0 con 0x7fc2b802c040 2023-10-11T09:50:28.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.062+0000 7fc2be924700 1 -- 172.21.15.78:0/3498686889 <== osd.6 v2:172.21.15.133:6800/35900 5 ==== osd_op_reply(47 mds_snaptable [delete] v109'2 uv2 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc2b8026040 con 0x7fc2b001d930 2023-10-11T09:50:28.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.062+0000 7fc2be123700 1 -- 172.21.15.78:0/3498686889 <== osd.4 v2:172.21.15.133:6808/35898 6 ==== osd_op_reply(39 200.00000000 [delete] v109'46 uv46 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fc2b000e9e0 con 0x7fc2b802c040 2023-10-11T09:50:28.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.062+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:53 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55ab93276ef0 con 0x7fc2b0032970 2023-10-11T09:50:28.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.062+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:54 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55ab9327d8c0 con 0x7fc2b0010970 2023-10-11T09:50:28.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.062+0000 7fc2bd922700 1 -- 172.21.15.78:0/3498686889 <== osd.0 v2:172.21.15.78:6800/35735 13 ==== osd_op_reply(42 607.00000000 [delete] v109'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fc2b4081040 con 0x7fc2b0010970 2023-10-11T09:50:28.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.062+0000 7fc2be123700 1 -- 172.21.15.78:0/3498686889 <== osd.7 v2:172.21.15.133:6824/35901 4 ==== osd_op_reply(45 604.00000000 [delete] v109'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fc2b000e9e0 con 0x7fc2b0032970 2023-10-11T09:50:28.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.062+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:55 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55ab9327add0 con 0x7fc2b0010970 2023-10-11T09:50:28.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.062+0000 7fc2be123700 1 -- 172.21.15.78:0/3498686889 <== osd.1 v2:172.21.15.78:6809/35734 6 ==== osd_op_reply(34 602.00000000 [delete] v109'12 uv12 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fc2b000e9e0 con 0x55ab930fa6b0 2023-10-11T09:50:28.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.062+0000 7fc2be123700 1 -- 172.21.15.78:0/3498686889 <== osd.1 v2:172.21.15.78:6809/35734 7 ==== osd_op_reply(35 10000000000.00000000 [delete] v109'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fc2b000e9e0 con 0x55ab930fa6b0 2023-10-11T09:50:28.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.062+0000 7fc2be123700 1 -- 172.21.15.78:0/3498686889 <== osd.4 v2:172.21.15.133:6808/35898 7 ==== osd_op_reply(40 200.00000001 [delete] v109'47 uv47 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fc2b000e9e0 con 0x7fc2b802c040 2023-10-11T09:50:28.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.066+0000 7fc2be123700 1 -- 172.21.15.78:0/3498686889 <== osd.1 v2:172.21.15.78:6809/35734 8 ==== osd_op_reply(36 600.00000000 [delete] v109'14 uv14 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fc2b000e9e0 con 0x55ab930fa6b0 2023-10-11T09:50:28.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.066+0000 7fc2be123700 1 -- 172.21.15.78:0/3498686889 <== osd.7 v2:172.21.15.133:6824/35901 5 ==== osd_op_reply(46 500.00000000 [delete] v109'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fc2b000e9e0 con 0x7fc2b0032970 2023-10-11T09:50:28.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.066+0000 7fc2be123700 1 -- 172.21.15.78:0/3498686889 <== osd.4 v2:172.21.15.133:6808/35898 8 ==== osd_op_reply(41 606.00000000 [delete] v109'48 uv48 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fc2b000e9e0 con 0x7fc2b802c040 2023-10-11T09:50:28.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.066+0000 7fc2be924700 1 -- 172.21.15.78:0/3498686889 <== osd.2 v2:172.21.15.78:6808/35732 4 ==== osd_op_reply(49 mds0_inotable [delete] v109'4 uv4 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc2b8026040 con 0x55ab932727e0 2023-10-11T09:50:28.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.066+0000 7fc2be123700 1 -- 172.21.15.78:0/3498686889 <== osd.1 v2:172.21.15.78:6809/35734 9 ==== osd_op_reply(37 603.00000000 [delete] v109'15 uv15 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fc2b000e9e0 con 0x55ab930fa6b0 2023-10-11T09:50:28.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.066+0000 7fc2bd922700 1 -- 172.21.15.78:0/3498686889 <== osd.0 v2:172.21.15.78:6800/35735 14 ==== osd_op_reply(51 mds0_sessionmap [delete] v109'4 uv4 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fc2b4081040 con 0x7fc2b0010970 2023-10-11T09:50:28.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.066+0000 7fc2be924700 1 -- 172.21.15.78:0/3498686889 <== osd.2 v2:172.21.15.78:6808/35732 5 ==== osd_op_reply(50 100.00000000 [delete] v109'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fc2b8026040 con 0x55ab932727e0 2023-10-11T09:50:28.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.066+0000 7fc2be123700 1 -- 172.21.15.78:0/3498686889 <== osd.7 v2:172.21.15.133:6824/35901 6 ==== osd_op_reply(53 400.00000000 [delete] v109'2 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fc2b000e9e0 con 0x7fc2b0032970 2023-10-11T09:50:28.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.066+0000 7fc2bd922700 1 -- 172.21.15.78:0/3498686889 <== osd.0 v2:172.21.15.78:6800/35735 15 ==== osd_op_reply(54 100.00000000.inode [delete] v109'7 uv7 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fc2b4081040 con 0x7fc2b0010970 2023-10-11T09:50:28.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.066+0000 7fc2bd922700 1 -- 172.21.15.78:0/3498686889 <== osd.5 v2:172.21.15.133:6816/35899 4 ==== osd_op_reply(52 609.00000000 [delete] v109'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fc2b4081040 con 0x55ab930faa80 2023-10-11T09:50:28.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.066+0000 7fc2bd922700 1 -- 172.21.15.78:0/3498686889 <== osd.0 v2:172.21.15.78:6800/35735 16 ==== osd_op_reply(55 1.00000000 [delete] v109'8 uv8 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fc2b4081040 con 0x7fc2b0010970 2023-10-11T09:50:28.078 INFO:teuthology.orchestra.run.smithi078.stdout:Removed 23 objects 2023-10-11T09:50:28.078 INFO:teuthology.orchestra.run.smithi078.stdout:successfully purged pool cephfs_metadata 2023-10-11T09:50:28.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.066+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x7fc2b0010970 msgr2=0x7fc2b001f440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:28.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.066+0000 7fc2c05ee140 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x7fc2b0010970 0x7fc2b001f440 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:28.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.066+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55ab930fa6b0 msgr2=0x7fc2b801c940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:28.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.066+0000 7fc2c05ee140 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55ab930fa6b0 0x7fc2b801c940 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:28.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.066+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55ab932727e0 msgr2=0x55ab93274bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:28.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.066+0000 7fc2c05ee140 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55ab932727e0 0x55ab93274bc0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:28.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.066+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x7fc2b401abe0 msgr2=0x7fc2b40535f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:28.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.066+0000 7fc2c05ee140 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x7fc2b401abe0 0x7fc2b40535f0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:28.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.066+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x7fc2b802c040 msgr2=0x7fc2b801b930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:28.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.066+0000 7fc2c05ee140 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x7fc2b802c040 0x7fc2b801b930 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:28.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.066+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55ab930faa80 msgr2=0x7fc2b802c920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:28.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.066+0000 7fc2be924700 1 -- 172.21.15.78:0/3498686889 reap_dead start 2023-10-11T09:50:28.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.066+0000 7fc2c05ee140 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55ab930faa80 0x7fc2b802c920 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:28.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.066+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x7fc2b001d930 msgr2=0x7fc2b000d3d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:28.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.066+0000 7fc2c05ee140 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x7fc2b001d930 0x7fc2b000d3d0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:28.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.066+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x7fc2b0032970 msgr2=0x7fc2b00313d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:28.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.066+0000 7fc2c05ee140 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x7fc2b0032970 0x7fc2b00313d0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:28.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.066+0000 7fc2be924700 1 -- 172.21.15.78:0/3498686889 reap_dead start 2023-10-11T09:50:28.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.066+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc2a0064830 msgr2=0x7fc2a0066ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:28.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.066+0000 7fc2c05ee140 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc2a0064830 0x7fc2a0066ce0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0000c00 tx=0x7fc2b0000fb0 comp rx=0 tx=0).stop 2023-10-11T09:50:28.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.066+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab931cef70 msgr2=0x55ab9313ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:28.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.066+0000 7fc2c05ee140 1 --2- 172.21.15.78:0/3498686889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ab931cef70 0x55ab9313ed70 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fc2b400e9b0 tx=0x7fc2b4001230 comp rx=0 tx=0).stop 2023-10-11T09:50:28.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.066+0000 7fc2be924700 1 -- 172.21.15.78:0/3498686889 reap_dead start 2023-10-11T09:50:28.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.066+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 shutdown_connections 2023-10-11T09:50:28.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.066+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 >> 172.21.15.78:0/3498686889 conn(0x55ab93100be0 msgr2=0x55ab931d2160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:28.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.066+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 shutdown_connections 2023-10-11T09:50:28.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.066+0000 7fc2c05ee140 1 -- 172.21.15.78:0/3498686889 wait complete. 2023-10-11T09:50:28.086 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs reset cephfs --yes-i-really-mean-it 2023-10-11T09:50:28.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.258+0000 7f844400a700 1 Processor -- start 2023-10-11T09:50:28.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.258+0000 7f844400a700 1 -- start start 2023-10-11T09:50:28.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.262+0000 7f844400a700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f843c160cb0 0x7f843c0cb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:28.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.262+0000 7f844400a700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f843c0b1030 0x7f843c06ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:28.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.262+0000 7f844400a700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f843c165570 con 0x7f843c06d080 2023-10-11T09:50:28.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.262+0000 7f844400a700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f843c161080 con 0x7f843c160cb0 2023-10-11T09:50:28.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.262+0000 7f844400a700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f843c164750 con 0x7f843c0b1030 2023-10-11T09:50:28.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.262+0000 7f8441da6700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f843c160cb0 0x7f843c0cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:28.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.262+0000 7f8441da6700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f843c160cb0 0x7f843c0cb100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35766/0 (socket says 172.21.15.78:35766) 2023-10-11T09:50:28.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.262+0000 7f8441da6700 1 -- 172.21.15.78:0/2004375415 learned_addr learned my addr 172.21.15.78:0/2004375415 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:28.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.262+0000 7f84425a7700 1 --2- 172.21.15.78:0/2004375415 >> v2:172.21.15.133:3300/0 conn(0x7f843c0b1030 0x7f843c06ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:28.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.262+0000 7f8441da6700 1 -- 172.21.15.78:0/2004375415 >> v2:172.21.15.133:3300/0 conn(0x7f843c0b1030 msgr2=0x7f843c06ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:28.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.262+0000 7f8441da6700 1 --2- 172.21.15.78:0/2004375415 >> v2:172.21.15.133:3300/0 conn(0x7f843c0b1030 0x7f843c06ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:28.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.262+0000 7f8441da6700 1 -- 172.21.15.78:0/2004375415 >> v1:172.21.15.133:6789/0 conn(0x7f843c06d080 legacy=0x7f843c0b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:28.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.262+0000 7f8441da6700 1 -- 172.21.15.78:0/2004375415 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843c164960 con 0x7f843c160cb0 2023-10-11T09:50:28.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.262+0000 7f8441da6700 1 --2- 172.21.15.78:0/2004375415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f843c160cb0 0x7f843c0cb100 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f843c0cfcd0 tx=0x7f8438009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a28502b90d7c98 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:28.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.262+0000 7f8440da4700 1 -- 172.21.15.78:0/2004375415 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8438011830 con 0x7f843c160cb0 2023-10-11T09:50:28.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.262+0000 7f8440da4700 1 -- 172.21.15.78:0/2004375415 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8438011990 con 0x7f843c160cb0 2023-10-11T09:50:28.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.262+0000 7f8440da4700 1 -- 172.21.15.78:0/2004375415 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8438011d20 con 0x7f843c160cb0 2023-10-11T09:50:28.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.266+0000 7f844400a700 1 -- 172.21.15.78:0/2004375415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f843c160cb0 msgr2=0x7f843c0cb100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:28.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.266+0000 7f844400a700 1 --2- 172.21.15.78:0/2004375415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f843c160cb0 0x7f843c0cb100 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f843c0cfcd0 tx=0x7f8438009bf0 comp rx=0 tx=0).stop 2023-10-11T09:50:28.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.266+0000 7f844400a700 1 -- 172.21.15.78:0/2004375415 shutdown_connections 2023-10-11T09:50:28.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.266+0000 7f844400a700 1 --2- 172.21.15.78:0/2004375415 >> v2:172.21.15.133:3300/0 conn(0x7f843c0b1030 0x7f843c06ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:28.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.266+0000 7f844400a700 1 --2- 172.21.15.78:0/2004375415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f843c160cb0 0x7f843c0cb100 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:28.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.266+0000 7f844400a700 1 -- 172.21.15.78:0/2004375415 >> 172.21.15.78:0/2004375415 conn(0x7f843c0cecc0 msgr2=0x7f843c0cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:28.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.266+0000 7f844400a700 1 -- 172.21.15.78:0/2004375415 shutdown_connections 2023-10-11T09:50:28.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.266+0000 7f844400a700 1 -- 172.21.15.78:0/2004375415 wait complete. 2023-10-11T09:50:28.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.266+0000 7f844400a700 1 Processor -- start 2023-10-11T09:50:28.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.270+0000 7f844400a700 1 -- start start 2023-10-11T09:50:28.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.270+0000 7f844400a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f843c06d080 0x7f843c151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:28.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.270+0000 7f844400a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f843c0b1030 0x7f843c163610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:28.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.270+0000 7f844400a700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f843c160cb0 0x7f843c162770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:28.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.270+0000 7f844400a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843c152090 con 0x7f843c0b1030 2023-10-11T09:50:28.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.270+0000 7f844400a700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843c0b21e0 con 0x7f843c06d080 2023-10-11T09:50:28.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.270+0000 7f844400a700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f843c0b24c0 con 0x7f843c160cb0 2023-10-11T09:50:28.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.270+0000 7f84415a5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f843c0b1030 0x7f843c163610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:28.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.270+0000 7f84415a5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f843c0b1030 0x7f843c163610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35778/0 (socket says 172.21.15.78:35778) 2023-10-11T09:50:28.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.270+0000 7f84415a5700 1 -- 172.21.15.78:0/3706515121 learned_addr learned my addr 172.21.15.78:0/3706515121 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:28.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.270+0000 7f8441da6700 1 --2- 172.21.15.78:0/3706515121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f843c06d080 0x7f843c151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:28.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.270+0000 7f84425a7700 1 --2- 172.21.15.78:0/3706515121 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f843c160cb0 0x7f843c162770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:28.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.270+0000 7f84415a5700 1 -- 172.21.15.78:0/3706515121 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f843c160cb0 msgr2=0x7f843c162770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:28.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.270+0000 7f84415a5700 1 --2- 172.21.15.78:0/3706515121 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f843c160cb0 0x7f843c162770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:28.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.270+0000 7f84415a5700 1 -- 172.21.15.78:0/3706515121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f843c06d080 msgr2=0x7f843c151b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:28.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.270+0000 7f84415a5700 1 --2- 172.21.15.78:0/3706515121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f843c06d080 0x7f843c151b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:28.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.270+0000 7f84415a5700 1 -- 172.21.15.78:0/3706515121 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8438009570 con 0x7f843c0b1030 2023-10-11T09:50:28.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.270+0000 7f84415a5700 1 --2- 172.21.15.78:0/3706515121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f843c0b1030 0x7f843c163610 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f842c00c500 tx=0x7f842c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:28.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.270+0000 7f84327fc700 1 -- 172.21.15.78:0/3706515121 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c0052d0 con 0x7f843c0b1030 2023-10-11T09:50:28.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.270+0000 7f84327fc700 1 -- 172.21.15.78:0/3706515121 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f842c005430 con 0x7f843c0b1030 2023-10-11T09:50:28.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.274+0000 7f844400a700 1 -- 172.21.15.78:0/3706515121 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f843c0b27a0 con 0x7f843c0b1030 2023-10-11T09:50:28.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.274+0000 7f84327fc700 1 -- 172.21.15.78:0/3706515121 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c0055a0 con 0x7f843c0b1030 2023-10-11T09:50:28.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.274+0000 7f844400a700 1 -- 172.21.15.78:0/3706515121 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f843c0b2d60 con 0x7f843c0b1030 2023-10-11T09:50:28.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.274+0000 7f844400a700 1 -- 172.21.15.78:0/3706515121 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f843c06eb80 con 0x7f843c0b1030 2023-10-11T09:50:28.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.274+0000 7f84327fc700 1 -- 172.21.15.78:0/3706515121 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f842c0038e0 con 0x7f843c0b1030 2023-10-11T09:50:28.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.274+0000 7f84327fc700 1 --2- 172.21.15.78:0/3706515121 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8420065f00 0x7f84200683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:28.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.278+0000 7f84327fc700 1 -- 172.21.15.78:0/3706515121 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6552+0+0 (secure 0 0 0) 0x7f842c082e50 con 0x7f843c0b1030 2023-10-11T09:50:28.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.278+0000 7f8441da6700 1 --2- 172.21.15.78:0/3706515121 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8420065f00 0x7f84200683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:28.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.278+0000 7f8441da6700 1 --2- 172.21.15.78:0/3706515121 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8420065f00 0x7f84200683b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f8438000c00 tx=0x7f8438008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:28.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.282+0000 7f84327fc700 1 -- 172.21.15.78:0/3706515121 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f842c050210 con 0x7f843c0b1030 2023-10-11T09:50:28.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.498+0000 7f844400a700 1 -- 172.21.15.78:0/3706515121 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs reset", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f843c065db0 con 0x7f843c0b1030 2023-10-11T09:50:28.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.582+0000 7f84327fc700 1 -- 172.21.15.78:0/3706515121 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs reset", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v82) v1 ==== 107+0+0 (secure 0 0 0) 0x7f842c053830 con 0x7f843c0b1030 2023-10-11T09:50:28.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.586+0000 7f844400a700 1 -- 172.21.15.78:0/3706515121 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8420065f00 msgr2=0x7f84200683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:28.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.586+0000 7f844400a700 1 --2- 172.21.15.78:0/3706515121 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8420065f00 0x7f84200683b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f8438000c00 tx=0x7f8438008030 comp rx=0 tx=0).stop 2023-10-11T09:50:28.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.586+0000 7f844400a700 1 -- 172.21.15.78:0/3706515121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f843c0b1030 msgr2=0x7f843c163610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:28.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.586+0000 7f844400a700 1 --2- 172.21.15.78:0/3706515121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f843c0b1030 0x7f843c163610 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f842c00c500 tx=0x7f842c00c810 comp rx=0 tx=0).stop 2023-10-11T09:50:28.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.586+0000 7f844400a700 1 -- 172.21.15.78:0/3706515121 shutdown_connections 2023-10-11T09:50:28.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.586+0000 7f844400a700 1 --2- 172.21.15.78:0/3706515121 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8420065f00 0x7f84200683b0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:28.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.586+0000 7f844400a700 1 --2- 172.21.15.78:0/3706515121 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f843c160cb0 0x7f843c162770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:28.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.586+0000 7f844400a700 1 --2- 172.21.15.78:0/3706515121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f843c0b1030 0x7f843c163610 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:28.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.586+0000 7f844400a700 1 --2- 172.21.15.78:0/3706515121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f843c06d080 0x7f843c151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:28.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.586+0000 7f844400a700 1 -- 172.21.15.78:0/3706515121 >> 172.21.15.78:0/3706515121 conn(0x7f843c0cecc0 msgr2=0x7f843c071ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:28.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.586+0000 7f844400a700 1 -- 172.21.15.78:0/3706515121 shutdown_connections 2023-10-11T09:50:28.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.586+0000 7f844400a700 1 -- 172.21.15.78:0/3706515121 wait complete. 2023-10-11T09:50:28.602 INFO:tasks.ceph.mds.c.smithi078.stderr:2023-10-11T09:50:28.594+0000 7f0c11ee4700 -1 mds.0.inotable: load_2 could not read table: -2 2023-10-11T09:50:28.603 INFO:tasks.ceph.mds.c.smithi078.stderr:2023-10-11T09:50:28.594+0000 7f0c11ee4700 -1 log_channel(cluster) log [ERR] : error reading table object 'mds0_inotable' -2 ((2) No such file or directory) 2023-10-11T09:50:28.604 INFO:tasks.ceph.mon.a.smithi078.stderr:2023-10-11T09:50:28.598+0000 7f90131f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-11T09:50:28.604 INFO:tasks.ceph.mon.a.smithi078.stderr:2023-10-11T09:50:28.598+0000 7f90131f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-10-11T09:50:28.617 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-11T09:50:28.623 INFO:tasks.ceph.mds.c.smithi078.stderr: -32> 2023-10-11T09:50:28.594+0000 7f0c11ee4700 -1 mds.0.inotable: load_2 could not read table: -2 2023-10-11T09:50:28.623 INFO:tasks.ceph.mds.c.smithi078.stderr: -30> 2023-10-11T09:50:28.594+0000 7f0c11ee4700 -1 log_channel(cluster) log [ERR] : error reading table object 'mds0_inotable' -2 ((2) No such file or directory) 2023-10-11T09:50:28.672 INFO:tasks.ceph.mds.c.smithi078.stdout:starting mds.c at 2023-10-11T09:50:28.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.782+0000 7f95e126d700 1 Processor -- start 2023-10-11T09:50:28.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.782+0000 7f95e126d700 1 -- start start 2023-10-11T09:50:28.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.786+0000 7f95e126d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f95dc0b3610 0x7f95dc0d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:28.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.786+0000 7f95e126d700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95dc0b47f0 con 0x7f95dc0b3240 2023-10-11T09:50:28.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.786+0000 7f95e126d700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95dc0b8130 con 0x7f95dc0b3a90 2023-10-11T09:50:28.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.786+0000 7f95e126d700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95dc0b41e0 con 0x7f95dc0b3610 2023-10-11T09:50:28.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.786+0000 7f95d37fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f95dc0b3610 0x7f95dc0d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:28.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.786+0000 7f95dbfff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f95dc0b3240 0x7f95dc15d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:39186/0 (socket says 172.21.15.78:39186) 2023-10-11T09:50:28.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.786+0000 7f95d37fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f95dc0b3610 0x7f95dc0d29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35786/0 (socket says 172.21.15.78:35786) 2023-10-11T09:50:28.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.786+0000 7f95dbfff700 1 -- 172.21.15.78:0/1501466836 learned_addr learned my addr 172.21.15.78:0/1501466836 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:50:28.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.786+0000 7f95d37fe700 1 -- 172.21.15.78:0/1501466836 >> v1:172.21.15.133:6789/0 conn(0x7f95dc0b3a90 legacy=0x7f95dc0b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:28.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.786+0000 7f95d37fe700 1 -- 172.21.15.78:0/1501466836 >> v1:172.21.15.78:6789/0 conn(0x7f95dc0b3240 legacy=0x7f95dc15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:28.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.786+0000 7f95d37fe700 1 -- 172.21.15.78:0/1501466836 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95dc0b7800 con 0x7f95dc0b3610 2023-10-11T09:50:28.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.786+0000 7f95d37fe700 1 --2- 172.21.15.78:0/1501466836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95dc0b3610 0x7f95dc0d29d0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f95c4009850 tx=0x7f95c4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5de6791e14d748a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:28.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.786+0000 7f95db7fe700 1 -- 172.21.15.78:0/1501466836 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c4011830 con 0x7f95dc0b3610 2023-10-11T09:50:28.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.786+0000 7f95db7fe700 1 -- 172.21.15.78:0/1501466836 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c4011990 con 0x7f95dc0b3610 2023-10-11T09:50:28.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.786+0000 7f95db7fe700 1 -- 172.21.15.78:0/1501466836 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c4011d20 con 0x7f95dc0b3610 2023-10-11T09:50:28.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.790+0000 7f95e126d700 1 -- 172.21.15.78:0/1501466836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95dc0b3610 msgr2=0x7f95dc0d29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:28.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.790+0000 7f95e126d700 1 --2- 172.21.15.78:0/1501466836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95dc0b3610 0x7f95dc0d29d0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f95c4009850 tx=0x7f95c4009b60 comp rx=0 tx=0).stop 2023-10-11T09:50:28.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.790+0000 7f95e126d700 1 -- 172.21.15.78:0/1501466836 shutdown_connections 2023-10-11T09:50:28.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.790+0000 7f95e126d700 1 --2- 172.21.15.78:0/1501466836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95dc0b3610 0x7f95dc0d29d0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:28.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.790+0000 7f95e126d700 1 -- 172.21.15.78:0/1501466836 >> 172.21.15.78:0/1501466836 conn(0x7f95dc09c260 msgr2=0x7f95dc0d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:28.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.790+0000 7f95e126d700 1 -- 172.21.15.78:0/1501466836 shutdown_connections 2023-10-11T09:50:28.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.790+0000 7f95e126d700 1 -- 172.21.15.78:0/1501466836 wait complete. 2023-10-11T09:50:28.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.790+0000 7f95e126d700 1 Processor -- start 2023-10-11T09:50:28.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.790+0000 7f95e126d700 1 -- start start 2023-10-11T09:50:28.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.790+0000 7f95e126d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95dc0b3240 0x7f95dc15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:28.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.790+0000 7f95e126d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95dc0b3610 0x7f95dc09ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:28.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.790+0000 7f95e126d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95dc0b3a90 0x7f95dc0b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:28.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.790+0000 7f95e126d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95dc0a0810 con 0x7f95dc0b3240 2023-10-11T09:50:28.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.790+0000 7f95e126d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95dc0a0ca0 con 0x7f95dc0b3610 2023-10-11T09:50:28.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.790+0000 7f95e126d700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95dc09fc00 con 0x7f95dc0b3a90 2023-10-11T09:50:28.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.790+0000 7f95dbfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95dc0b3240 0x7f95dc15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:28.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.790+0000 7f95dbfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95dc0b3240 0x7f95dc15aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35802/0 (socket says 172.21.15.78:35802) 2023-10-11T09:50:28.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.794+0000 7f95dbfff700 1 -- 172.21.15.78:0/1723275334 learned_addr learned my addr 172.21.15.78:0/1723275334 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:28.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.794+0000 7f95d37fe700 1 --2- 172.21.15.78:0/1723275334 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95dc0b3610 0x7f95dc09ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:28.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.794+0000 7f95e0a6c700 1 --2- 172.21.15.78:0/1723275334 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95dc0b3a90 0x7f95dc0b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:28.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.794+0000 7f95dbfff700 1 -- 172.21.15.78:0/1723275334 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95dc0b3a90 msgr2=0x7f95dc0b0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:28.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.794+0000 7f95dbfff700 1 --2- 172.21.15.78:0/1723275334 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95dc0b3a90 0x7f95dc0b0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:28.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.794+0000 7f95dbfff700 1 -- 172.21.15.78:0/1723275334 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95dc0b3610 msgr2=0x7f95dc09ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:28.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.794+0000 7f95dbfff700 1 --2- 172.21.15.78:0/1723275334 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95dc0b3610 0x7f95dc09ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:28.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.794+0000 7f95dbfff700 1 -- 172.21.15.78:0/1723275334 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c40094b0 con 0x7f95dc0b3240 2023-10-11T09:50:28.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.794+0000 7f95dbfff700 1 --2- 172.21.15.78:0/1723275334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95dc0b3240 0x7f95dc15aa80 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f95cc00cbc0 tx=0x7f95cc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:28.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.794+0000 7f95d97fa700 1 -- 172.21.15.78:0/1723275334 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95cc007480 con 0x7f95dc0b3240 2023-10-11T09:50:28.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.794+0000 7f95d97fa700 1 -- 172.21.15.78:0/1723275334 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95cc0075e0 con 0x7f95dc0b3240 2023-10-11T09:50:28.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.794+0000 7f95d97fa700 1 -- 172.21.15.78:0/1723275334 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95cc007950 con 0x7f95dc0b3240 2023-10-11T09:50:28.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.794+0000 7f95e126d700 1 -- 172.21.15.78:0/1723275334 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95dc09fee0 con 0x7f95dc0b3240 2023-10-11T09:50:28.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.794+0000 7f95e126d700 1 -- 172.21.15.78:0/1723275334 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95dc0a04a0 con 0x7f95dc0b3240 2023-10-11T09:50:28.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.794+0000 7f95e126d700 1 -- 172.21.15.78:0/1723275334 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95dc065d40 con 0x7f95dc0b3240 2023-10-11T09:50:28.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.798+0000 7f95d97fa700 1 -- 172.21.15.78:0/1723275334 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f95cc01b3b0 con 0x7f95dc0b3240 2023-10-11T09:50:28.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.798+0000 7f95d97fa700 1 --2- 172.21.15.78:0/1723275334 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f95bc065ed0 0x7f95bc068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:28.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.798+0000 7f95d97fa700 1 -- 172.21.15.78:0/1723275334 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6638+0+0 (secure 0 0 0) 0x7f95cc0831c0 con 0x7f95dc0b3240 2023-10-11T09:50:28.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.798+0000 7f95d37fe700 1 --2- 172.21.15.78:0/1723275334 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f95bc065ed0 0x7f95bc068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:28.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.802+0000 7f95d37fe700 1 --2- 172.21.15.78:0/1723275334 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f95bc065ed0 0x7f95bc068380 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f95c4014040 tx=0x7f95c401c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:28.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:28.806+0000 7f95d97fa700 1 -- 172.21.15.78:0/1723275334 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f95cc0505e0 con 0x7f95dc0b3240 2023-10-11T09:50:29.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.014+0000 7f95e126d700 1 -- 172.21.15.78:0/1723275334 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f95dc0b47f0 con 0x7f95dc0b3240 2023-10-11T09:50:29.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.618+0000 7f95d97fa700 1 -- 172.21.15.78:0/1723275334 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v85) v1 ==== 162+0+0 (secure 0 0 0) 0x7f95cc053c00 con 0x7f95dc0b3240 2023-10-11T09:50:29.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.626+0000 7f95e126d700 1 -- 172.21.15.78:0/1723275334 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f95bc065ed0 msgr2=0x7f95bc068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:29.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.626+0000 7f95e126d700 1 --2- 172.21.15.78:0/1723275334 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f95bc065ed0 0x7f95bc068380 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f95c4014040 tx=0x7f95c401c040 comp rx=0 tx=0).stop 2023-10-11T09:50:29.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.626+0000 7f95e126d700 1 -- 172.21.15.78:0/1723275334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95dc0b3240 msgr2=0x7f95dc15aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:29.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.626+0000 7f95e126d700 1 --2- 172.21.15.78:0/1723275334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95dc0b3240 0x7f95dc15aa80 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f95cc00cbc0 tx=0x7f95cc00cbf0 comp rx=0 tx=0).stop 2023-10-11T09:50:29.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.626+0000 7f95e126d700 1 -- 172.21.15.78:0/1723275334 shutdown_connections 2023-10-11T09:50:29.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.626+0000 7f95e126d700 1 --2- 172.21.15.78:0/1723275334 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f95bc065ed0 0x7f95bc068380 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:29.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.626+0000 7f95e126d700 1 --2- 172.21.15.78:0/1723275334 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95dc0b3a90 0x7f95dc0b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:29.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.626+0000 7f95e126d700 1 --2- 172.21.15.78:0/1723275334 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95dc0b3610 0x7f95dc09ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:29.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.626+0000 7f95e126d700 1 --2- 172.21.15.78:0/1723275334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95dc0b3240 0x7f95dc15aa80 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:29.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.626+0000 7f95e126d700 1 -- 172.21.15.78:0/1723275334 >> 172.21.15.78:0/1723275334 conn(0x7f95dc09c260 msgr2=0x7f95dc0afc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:29.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.626+0000 7f95e126d700 1 -- 172.21.15.78:0/1723275334 shutdown_connections 2023-10-11T09:50:29.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.626+0000 7f95e126d700 1 -- 172.21.15.78:0/1723275334 wait complete. 2023-10-11T09:50:29.636 INFO:teuthology.orchestra.run.smithi078.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-11T09:50:29.656 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:50:29.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.870+0000 7f9902eb5700 1 Processor -- start 2023-10-11T09:50:29.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.870+0000 7f9902eb5700 1 -- start start 2023-10-11T09:50:29.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.870+0000 7f9902eb5700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f98fc15c880 0x7f98fc06be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:29.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.874+0000 7f9902eb5700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f98fc0cbd30 0x7f98fc0c9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:29.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.874+0000 7f9902eb5700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98fc0c93a0 con 0x7f98fc0cd210 2023-10-11T09:50:29.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.874+0000 7f9902eb5700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98fc161420 con 0x7f98fc15c880 2023-10-11T09:50:29.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.874+0000 7f9902eb5700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98fc0cca10 con 0x7f98fc0cbd30 2023-10-11T09:50:29.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.874+0000 7f9900c51700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f98fc15c880 0x7f98fc06be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:29.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.874+0000 7f98f3fff700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f98fc0cd210 0x7f98fc0cb620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:60116/0 (socket says 172.21.15.78:60116) 2023-10-11T09:50:29.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.874+0000 7f98f3fff700 1 -- 172.21.15.78:0/220056222 learned_addr learned my addr 172.21.15.78:0/220056222 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:50:29.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.874+0000 7f9900c51700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f98fc15c880 0x7f98fc06be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35804/0 (socket says 172.21.15.78:35804) 2023-10-11T09:50:29.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.874+0000 7f9901452700 1 --2- 172.21.15.78:0/220056222 >> v2:172.21.15.133:3300/0 conn(0x7f98fc0cbd30 0x7f98fc0c9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:29.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.874+0000 7f9900c51700 1 -- 172.21.15.78:0/220056222 >> v2:172.21.15.133:3300/0 conn(0x7f98fc0cbd30 msgr2=0x7f98fc0c9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:29.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.874+0000 7f9900c51700 1 --2- 172.21.15.78:0/220056222 >> v2:172.21.15.133:3300/0 conn(0x7f98fc0cbd30 0x7f98fc0c9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:29.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.874+0000 7f9900c51700 1 -- 172.21.15.78:0/220056222 >> v1:172.21.15.133:6789/0 conn(0x7f98fc0cd210 legacy=0x7f98fc0cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:29.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.874+0000 7f9900c51700 1 -- 172.21.15.78:0/220056222 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98fc0cccb0 con 0x7f98fc15c880 2023-10-11T09:50:29.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.874+0000 7f98f37fe700 1 -- 172.21.15.78:0/220056222 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2432508050 0 0) 0x7f98fc0c93a0 con 0x7f98fc0cd210 2023-10-11T09:50:29.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.874+0000 7f9900c51700 1 --2- 172.21.15.78:0/220056222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98fc15c880 0x7f98fc06be60 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f98fc0cc640 tx=0x7f98f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d375bb6c60bfe8 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:29.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.874+0000 7f98f37fe700 1 -- 172.21.15.78:0/220056222 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f800e070 con 0x7f98fc15c880 2023-10-11T09:50:29.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.874+0000 7f98f37fe700 1 -- 172.21.15.78:0/220056222 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f800bbd0 con 0x7f98fc15c880 2023-10-11T09:50:29.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.874+0000 7f98f37fe700 1 -- 172.21.15.78:0/220056222 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f8012430 con 0x7f98fc15c880 2023-10-11T09:50:29.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.878+0000 7f9902eb5700 1 -- 172.21.15.78:0/220056222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98fc15c880 msgr2=0x7f98fc06be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:29.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.878+0000 7f9902eb5700 1 --2- 172.21.15.78:0/220056222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98fc15c880 0x7f98fc06be60 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f98fc0cc640 tx=0x7f98f800b330 comp rx=0 tx=0).stop 2023-10-11T09:50:29.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.878+0000 7f9902eb5700 1 -- 172.21.15.78:0/220056222 shutdown_connections 2023-10-11T09:50:29.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.878+0000 7f9902eb5700 1 --2- 172.21.15.78:0/220056222 >> v2:172.21.15.133:3300/0 conn(0x7f98fc0cbd30 0x7f98fc0c9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:29.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.878+0000 7f9902eb5700 1 --2- 172.21.15.78:0/220056222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98fc15c880 0x7f98fc06be60 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:29.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.878+0000 7f9902eb5700 1 -- 172.21.15.78:0/220056222 >> 172.21.15.78:0/220056222 conn(0x7f98fc06aca0 msgr2=0x7f98fc0b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:29.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.878+0000 7f9902eb5700 1 -- 172.21.15.78:0/220056222 shutdown_connections 2023-10-11T09:50:29.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.878+0000 7f9902eb5700 1 -- 172.21.15.78:0/220056222 wait complete. 2023-10-11T09:50:29.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.882+0000 7f9902eb5700 1 Processor -- start 2023-10-11T09:50:29.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.882+0000 7f9902eb5700 1 -- start start 2023-10-11T09:50:29.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.882+0000 7f9902eb5700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f98fc0cbd30 0x7f98fc0afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:29.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.882+0000 7f9902eb5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98fc0cd210 0x7f98fc13d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:29.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.882+0000 7f9902eb5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98fc15c880 0x7f98fc0ae850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:29.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.882+0000 7f9902eb5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc160100 con 0x7f98fc15c880 2023-10-11T09:50:29.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.882+0000 7f9902eb5700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc15f4f0 con 0x7f98fc0cd210 2023-10-11T09:50:29.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.882+0000 7f9902eb5700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc15f7d0 con 0x7f98fc0cbd30 2023-10-11T09:50:29.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.882+0000 7f9901452700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98fc15c880 0x7f98fc0ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:29.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.882+0000 7f9901452700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98fc15c880 0x7f98fc0ae850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35810/0 (socket says 172.21.15.78:35810) 2023-10-11T09:50:29.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.882+0000 7f9901452700 1 -- 172.21.15.78:0/1556676239 learned_addr learned my addr 172.21.15.78:0/1556676239 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:29.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.882+0000 7f98f3fff700 1 --2- 172.21.15.78:0/1556676239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98fc0cd210 0x7f98fc13d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:29.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.882+0000 7f9900c51700 1 --2- 172.21.15.78:0/1556676239 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f98fc0cbd30 0x7f98fc0afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:29.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.882+0000 7f9901452700 1 -- 172.21.15.78:0/1556676239 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f98fc0cbd30 msgr2=0x7f98fc0afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:29.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.882+0000 7f9901452700 1 --2- 172.21.15.78:0/1556676239 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f98fc0cbd30 0x7f98fc0afbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:29.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.882+0000 7f9901452700 1 -- 172.21.15.78:0/1556676239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98fc0cd210 msgr2=0x7f98fc13d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:29.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.882+0000 7f9901452700 1 --2- 172.21.15.78:0/1556676239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98fc0cd210 0x7f98fc13d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:29.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.882+0000 7f9901452700 1 -- 172.21.15.78:0/1556676239 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f8009a70 con 0x7f98fc15c880 2023-10-11T09:50:29.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.882+0000 7f9901452700 1 --2- 172.21.15.78:0/1556676239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98fc15c880 0x7f98fc0ae850 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f98f400e7e0 tx=0x7f98f400eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:29.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.886+0000 7f98f17fa700 1 -- 172.21.15.78:0/1556676239 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f400c070 con 0x7f98fc15c880 2023-10-11T09:50:29.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.886+0000 7f98f17fa700 1 -- 172.21.15.78:0/1556676239 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f4013630 con 0x7f98fc15c880 2023-10-11T09:50:29.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.886+0000 7f98f17fa700 1 -- 172.21.15.78:0/1556676239 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f40139e0 con 0x7f98fc15c880 2023-10-11T09:50:29.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.886+0000 7f9902eb5700 1 -- 172.21.15.78:0/1556676239 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98fc15fab0 con 0x7f98fc15c880 2023-10-11T09:50:29.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.886+0000 7f9902eb5700 1 -- 172.21.15.78:0/1556676239 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98fc13dac0 con 0x7f98fc15c880 2023-10-11T09:50:29.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.886+0000 7f9902eb5700 1 -- 172.21.15.78:0/1556676239 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98fc0cfa50 con 0x7f98fc15c880 2023-10-11T09:50:29.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.890+0000 7f98f17fa700 1 -- 172.21.15.78:0/1556676239 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f98f4013b40 con 0x7f98fc15c880 2023-10-11T09:50:29.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.890+0000 7f98f17fa700 1 --2- 172.21.15.78:0/1556676239 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f98e8065ed0 0x7f98e8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:29.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.890+0000 7f98f17fa700 1 -- 172.21.15.78:0/1556676239 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6638+0+0 (secure 0 0 0) 0x7f98f4083750 con 0x7f98fc15c880 2023-10-11T09:50:29.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.890+0000 7f9900c51700 1 --2- 172.21.15.78:0/1556676239 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f98e8065ed0 0x7f98e8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:29.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.890+0000 7f9900c51700 1 --2- 172.21.15.78:0/1556676239 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f98e8065ed0 0x7f98e8068380 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f98f8014040 tx=0x7f98f8011000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:29.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:29.894+0000 7f98f17fa700 1 -- 172.21.15.78:0/1556676239 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f98f4050b70 con 0x7f98fc15c880 2023-10-11T09:50:30.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.110+0000 7f9902eb5700 1 -- 172.21.15.78:0/1556676239 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98fc04e530 con 0x7f98fc15c880 2023-10-11T09:50:30.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.110+0000 7f98f17fa700 1 -- 172.21.15.78:0/1556676239 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4652 (secure 0 0 0) 0x7f98f4054190 con 0x7f98fc15c880 2023-10-11T09:50:30.118 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:30.118 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":85,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6812,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/3697741427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3697741427},{"type":"v1","addr":"172.21.15.133:6833","nonce":3697741427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":6815,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2640710924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2640710924},{"type":"v1","addr":"172.21.15.133:6835","nonce":2640710924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8028,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/1538320637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":1538320637},{"type":"v1","addr":"172.21.15.78:6835","nonce":1538320637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8358,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/2876755099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2876755099},{"type":"v1","addr":"172.21.15.78:6837","nonce":2876755099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:50:28.502984+0000","modified":"2023-10-11T09:50:29.613806+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15,16],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-11T09:50:30.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.114+0000 7f9902eb5700 1 -- 172.21.15.78:0/1556676239 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f98e8065ed0 msgr2=0x7f98e8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:30.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.114+0000 7f9902eb5700 1 --2- 172.21.15.78:0/1556676239 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f98e8065ed0 0x7f98e8068380 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f98f8014040 tx=0x7f98f8011000 comp rx=0 tx=0).stop 2023-10-11T09:50:30.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.114+0000 7f9902eb5700 1 -- 172.21.15.78:0/1556676239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98fc15c880 msgr2=0x7f98fc0ae850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:30.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.114+0000 7f9902eb5700 1 --2- 172.21.15.78:0/1556676239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98fc15c880 0x7f98fc0ae850 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f98f400e7e0 tx=0x7f98f400eaf0 comp rx=0 tx=0).stop 2023-10-11T09:50:30.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.114+0000 7f9902eb5700 1 -- 172.21.15.78:0/1556676239 shutdown_connections 2023-10-11T09:50:30.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.114+0000 7f9902eb5700 1 --2- 172.21.15.78:0/1556676239 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f98e8065ed0 0x7f98e8068380 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:30.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.114+0000 7f9902eb5700 1 --2- 172.21.15.78:0/1556676239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98fc15c880 0x7f98fc0ae850 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:30.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.114+0000 7f9902eb5700 1 --2- 172.21.15.78:0/1556676239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98fc0cd210 0x7f98fc13d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:30.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.114+0000 7f9902eb5700 1 --2- 172.21.15.78:0/1556676239 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f98fc0cbd30 0x7f98fc0afbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:30.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.114+0000 7f9902eb5700 1 -- 172.21.15.78:0/1556676239 >> 172.21.15.78:0/1556676239 conn(0x7f98fc06aca0 msgr2=0x7f98fc151aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:30.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.114+0000 7f9902eb5700 1 -- 172.21.15.78:0/1556676239 shutdown_connections 2023-10-11T09:50:30.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.114+0000 7f9902eb5700 1 -- 172.21.15.78:0/1556676239 wait complete. 2023-10-11T09:50:30.123 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 85 2023-10-11T09:50:30.140 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-11T09:50:30.141 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:50:30.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.306+0000 7feb5b07b700 1 Processor -- start 2023-10-11T09:50:30.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.306+0000 7feb5b07b700 1 -- start start 2023-10-11T09:50:30.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.306+0000 7feb5b07b700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb540d2fb0 0x7feb540d3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:30.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.306+0000 7feb5b07b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7feb540b2600 0x7feb540d38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:30.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.306+0000 7feb5b07b700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb540cdcc0 con 0x7feb54160350 2023-10-11T09:50:30.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.306+0000 7feb5b07b700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb540d2d20 con 0x7feb540b2600 2023-10-11T09:50:30.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.306+0000 7feb5b07b700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb540cfd30 con 0x7feb540d2fb0 2023-10-11T09:50:30.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.310+0000 7feb53fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7feb540b2600 0x7feb540d38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:30.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.310+0000 7feb53fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7feb540b2600 0x7feb540d38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35818/0 (socket says 172.21.15.78:35818) 2023-10-11T09:50:30.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.310+0000 7feb53fff700 1 -- 172.21.15.78:0/2143363319 learned_addr learned my addr 172.21.15.78:0/2143363319 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:30.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.310+0000 7feb58e17700 1 --2- 172.21.15.78:0/2143363319 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb540d2fb0 0x7feb540d3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:30.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.310+0000 7feb53fff700 1 -- 172.21.15.78:0/2143363319 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb540d2fb0 msgr2=0x7feb540d3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:30.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.310+0000 7feb53fff700 1 --2- 172.21.15.78:0/2143363319 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb540d2fb0 0x7feb540d3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:30.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.310+0000 7feb53fff700 1 -- 172.21.15.78:0/2143363319 >> v1:172.21.15.78:6789/0 conn(0x7feb54160350 legacy=0x7feb541607d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:30.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.310+0000 7feb53fff700 1 -- 172.21.15.78:0/2143363319 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb540d3e00 con 0x7feb540b2600 2023-10-11T09:50:30.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.310+0000 7feb537fe700 1 -- 172.21.15.78:0/2143363319 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4142274422 0 0) 0x7feb540cdcc0 con 0x7feb54160350 2023-10-11T09:50:30.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.310+0000 7feb53fff700 1 --2- 172.21.15.78:0/2143363319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb540b2600 0x7feb540d38c0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7feb40009910 tx=0x7feb40009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a2c1bb1e311823 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:30.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.310+0000 7feb537fe700 1 -- 172.21.15.78:0/2143363319 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb40011830 con 0x7feb540b2600 2023-10-11T09:50:30.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.310+0000 7feb537fe700 1 -- 172.21.15.78:0/2143363319 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb40011990 con 0x7feb540b2600 2023-10-11T09:50:30.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.310+0000 7feb537fe700 1 -- 172.21.15.78:0/2143363319 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb40011d40 con 0x7feb540b2600 2023-10-11T09:50:30.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.310+0000 7feb5b07b700 1 -- 172.21.15.78:0/2143363319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb540b2600 msgr2=0x7feb540d38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:30.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.310+0000 7feb5b07b700 1 --2- 172.21.15.78:0/2143363319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb540b2600 0x7feb540d38c0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7feb40009910 tx=0x7feb40009c20 comp rx=0 tx=0).stop 2023-10-11T09:50:30.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.310+0000 7feb5b07b700 1 -- 172.21.15.78:0/2143363319 shutdown_connections 2023-10-11T09:50:30.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.310+0000 7feb5b07b700 1 --2- 172.21.15.78:0/2143363319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb540b2600 0x7feb540d38c0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:30.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.310+0000 7feb5b07b700 1 --2- 172.21.15.78:0/2143363319 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb540d2fb0 0x7feb540d3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:30.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.310+0000 7feb5b07b700 1 -- 172.21.15.78:0/2143363319 >> 172.21.15.78:0/2143363319 conn(0x7feb5406cb50 msgr2=0x7feb5406cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:30.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.310+0000 7feb5b07b700 1 -- 172.21.15.78:0/2143363319 shutdown_connections 2023-10-11T09:50:30.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.310+0000 7feb5b07b700 1 -- 172.21.15.78:0/2143363319 wait complete. 2023-10-11T09:50:30.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.314+0000 7feb5b07b700 1 Processor -- start 2023-10-11T09:50:30.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.314+0000 7feb5b07b700 1 -- start start 2023-10-11T09:50:30.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.314+0000 7feb5b07b700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb540b2600 0x7feb540cb140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:30.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.314+0000 7feb5b07b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb540d2fb0 0x7feb540a9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:30.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.314+0000 7feb5b07b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb54160350 0x7feb541f4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:30.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.314+0000 7feb5b07b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb540cbb00 con 0x7feb54160350 2023-10-11T09:50:30.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.314+0000 7feb5b07b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb540cbde0 con 0x7feb540d2fb0 2023-10-11T09:50:30.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.314+0000 7feb5b07b700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb540cc0c0 con 0x7feb540b2600 2023-10-11T09:50:30.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.314+0000 7feb59618700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb54160350 0x7feb541f4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:30.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.314+0000 7feb59618700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb54160350 0x7feb541f4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35826/0 (socket says 172.21.15.78:35826) 2023-10-11T09:50:30.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.314+0000 7feb59618700 1 -- 172.21.15.78:0/1932587008 learned_addr learned my addr 172.21.15.78:0/1932587008 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:30.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.314+0000 7feb58e17700 1 --2- 172.21.15.78:0/1932587008 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb540b2600 0x7feb540cb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:30.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.314+0000 7feb53fff700 1 --2- 172.21.15.78:0/1932587008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb540d2fb0 0x7feb540a9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:30.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.314+0000 7feb59618700 1 -- 172.21.15.78:0/1932587008 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb540b2600 msgr2=0x7feb540cb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:30.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.314+0000 7feb59618700 1 --2- 172.21.15.78:0/1932587008 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb540b2600 0x7feb540cb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:30.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.314+0000 7feb59618700 1 -- 172.21.15.78:0/1932587008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb540d2fb0 msgr2=0x7feb540a9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:30.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.314+0000 7feb59618700 1 --2- 172.21.15.78:0/1932587008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb540d2fb0 0x7feb540a9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:30.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.314+0000 7feb59618700 1 -- 172.21.15.78:0/1932587008 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb40009570 con 0x7feb54160350 2023-10-11T09:50:30.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.314+0000 7feb53fff700 1 --2- 172.21.15.78:0/1932587008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb540d2fb0 0x7feb540a9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:50:30.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.318+0000 7feb59618700 1 --2- 172.21.15.78:0/1932587008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb54160350 0x7feb541f4910 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7feb4c00cc20 tx=0x7feb4c00cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:30.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.318+0000 7feb517fa700 1 -- 172.21.15.78:0/1932587008 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4c003e50 con 0x7feb54160350 2023-10-11T09:50:30.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.318+0000 7feb517fa700 1 -- 172.21.15.78:0/1932587008 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb4c00a740 con 0x7feb54160350 2023-10-11T09:50:30.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.318+0000 7feb517fa700 1 -- 172.21.15.78:0/1932587008 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4c00aaf0 con 0x7feb54160350 2023-10-11T09:50:30.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.318+0000 7feb5b07b700 1 -- 172.21.15.78:0/1932587008 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb540cc3a0 con 0x7feb54160350 2023-10-11T09:50:30.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.318+0000 7feb5b07b700 1 -- 172.21.15.78:0/1932587008 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb540cb680 con 0x7feb54160350 2023-10-11T09:50:30.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.318+0000 7feb5b07b700 1 -- 172.21.15.78:0/1932587008 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb540cb870 con 0x7feb54160350 2023-10-11T09:50:30.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.322+0000 7feb517fa700 1 -- 172.21.15.78:0/1932587008 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7feb4c00ac50 con 0x7feb54160350 2023-10-11T09:50:30.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.322+0000 7feb517fa700 1 --2- 172.21.15.78:0/1932587008 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7feb38065ed0 0x7feb38068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:30.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.322+0000 7feb517fa700 1 -- 172.21.15.78:0/1932587008 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6638+0+0 (secure 0 0 0) 0x7feb4c083ac0 con 0x7feb54160350 2023-10-11T09:50:30.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.322+0000 7feb58e17700 1 --2- 172.21.15.78:0/1932587008 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7feb38065ed0 0x7feb38068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:30.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.322+0000 7feb58e17700 1 --2- 172.21.15.78:0/1932587008 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7feb38065ed0 0x7feb38068380 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7feb48009b50 tx=0x7feb48009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:30.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.330+0000 7feb517fa700 1 -- 172.21.15.78:0/1932587008 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7feb4c050ec0 con 0x7feb54160350 2023-10-11T09:50:30.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.542+0000 7feb5b07b700 1 -- 172.21.15.78:0/1932587008 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb5404e530 con 0x7feb54160350 2023-10-11T09:50:30.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.542+0000 7feb517fa700 1 -- 172.21.15.78:0/1932587008 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4652 (secure 0 0 0) 0x7feb4c0544e0 con 0x7feb54160350 2023-10-11T09:50:30.551 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:30.551 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":85,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6812,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/3697741427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3697741427},{"type":"v1","addr":"172.21.15.133:6833","nonce":3697741427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":6815,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2640710924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2640710924},{"type":"v1","addr":"172.21.15.133:6835","nonce":2640710924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8028,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/1538320637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":1538320637},{"type":"v1","addr":"172.21.15.78:6835","nonce":1538320637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8358,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/2876755099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2876755099},{"type":"v1","addr":"172.21.15.78:6837","nonce":2876755099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:50:28.502984+0000","modified":"2023-10-11T09:50:29.613806+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15,16],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-11T09:50:30.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.546+0000 7feb5b07b700 1 -- 172.21.15.78:0/1932587008 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7feb38065ed0 msgr2=0x7feb38068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:30.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.546+0000 7feb5b07b700 1 --2- 172.21.15.78:0/1932587008 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7feb38065ed0 0x7feb38068380 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7feb48009b50 tx=0x7feb48009240 comp rx=0 tx=0).stop 2023-10-11T09:50:30.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.546+0000 7feb5b07b700 1 -- 172.21.15.78:0/1932587008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb54160350 msgr2=0x7feb541f4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:30.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.546+0000 7feb5b07b700 1 --2- 172.21.15.78:0/1932587008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb54160350 0x7feb541f4910 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7feb4c00cc20 tx=0x7feb4c00cf30 comp rx=0 tx=0).stop 2023-10-11T09:50:30.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.546+0000 7feb5b07b700 1 -- 172.21.15.78:0/1932587008 shutdown_connections 2023-10-11T09:50:30.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.546+0000 7feb5b07b700 1 --2- 172.21.15.78:0/1932587008 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7feb38065ed0 0x7feb38068380 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:30.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.546+0000 7feb5b07b700 1 --2- 172.21.15.78:0/1932587008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb54160350 0x7feb541f4910 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:30.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.546+0000 7feb5b07b700 1 --2- 172.21.15.78:0/1932587008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb540d2fb0 0x7feb540a9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:30.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.546+0000 7feb5b07b700 1 --2- 172.21.15.78:0/1932587008 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb540b2600 0x7feb540cb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:30.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.546+0000 7feb5b07b700 1 -- 172.21.15.78:0/1932587008 >> 172.21.15.78:0/1932587008 conn(0x7feb5406cb50 msgr2=0x7feb5415f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:30.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.546+0000 7feb5b07b700 1 -- 172.21.15.78:0/1932587008 shutdown_connections 2023-10-11T09:50:30.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.546+0000 7feb5b07b700 1 -- 172.21.15.78:0/1932587008 wait complete. 2023-10-11T09:50:30.556 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 85 2023-10-11T09:50:30.573 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:50:30.692 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:50:30.698 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:50:30.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.738+0000 7fbf40bf9700 1 Processor -- start 2023-10-11T09:50:30.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.738+0000 7fbf40bf9700 1 -- start start 2023-10-11T09:50:30.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.742+0000 7fbf40bf9700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbf3c159490 0x7fbf3c0ce500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:30.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.742+0000 7fbf40bf9700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fbf3c15d030 0x7fbf3c15d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:30.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.742+0000 7fbf40bf9700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf3c159c50 con 0x7fbf3c15d9c0 2023-10-11T09:50:30.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.742+0000 7fbf40bf9700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf3c0cea40 con 0x7fbf3c15d030 2023-10-11T09:50:30.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.742+0000 7fbf40bf9700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf3c15de20 con 0x7fbf3c159490 2023-10-11T09:50:30.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.742+0000 7fbf39d9b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fbf3c15d030 0x7fbf3c15d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:30.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.742+0000 7fbf39d9b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fbf3c15d030 0x7fbf3c15d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35842/0 (socket says 172.21.15.78:35842) 2023-10-11T09:50:30.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.742+0000 7fbf39d9b700 1 -- 172.21.15.78:0/3405227340 learned_addr learned my addr 172.21.15.78:0/3405227340 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:30.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.742+0000 7fbf3a59c700 1 --2- 172.21.15.78:0/3405227340 >> v2:172.21.15.133:3300/0 conn(0x7fbf3c159490 0x7fbf3c0ce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:30.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.742+0000 7fbf39d9b700 1 -- 172.21.15.78:0/3405227340 >> v2:172.21.15.133:3300/0 conn(0x7fbf3c159490 msgr2=0x7fbf3c0ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:30.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.742+0000 7fbf39d9b700 1 --2- 172.21.15.78:0/3405227340 >> v2:172.21.15.133:3300/0 conn(0x7fbf3c159490 0x7fbf3c0ce500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:30.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.742+0000 7fbf39d9b700 1 -- 172.21.15.78:0/3405227340 >> v1:172.21.15.133:6789/0 conn(0x7fbf3c15d9c0 legacy=0x7fbf3c0c92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:30.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.742+0000 7fbf39d9b700 1 -- 172.21.15.78:0/3405227340 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf3c159ef0 con 0x7fbf3c15d030 2023-10-11T09:50:30.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.742+0000 7fbf39d9b700 1 --2- 172.21.15.78:0/3405227340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf3c15d030 0x7fbf3c15d480 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c06f760 tx=0x7fbf2400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b316d4be745501 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:30.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.742+0000 7fbf3959a700 1 -- 172.21.15.78:0/3405227340 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2400dd90 con 0x7fbf3c15d030 2023-10-11T09:50:30.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.742+0000 7fbf3959a700 1 -- 172.21.15.78:0/3405227340 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf24010470 con 0x7fbf3c15d030 2023-10-11T09:50:30.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.742+0000 7fbf3959a700 1 -- 172.21.15.78:0/3405227340 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf24010800 con 0x7fbf3c15d030 2023-10-11T09:50:30.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.742+0000 7fbf40bf9700 1 -- 172.21.15.78:0/3405227340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf3c15d030 msgr2=0x7fbf3c15d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:30.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.742+0000 7fbf40bf9700 1 --2- 172.21.15.78:0/3405227340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf3c15d030 0x7fbf3c15d480 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c06f760 tx=0x7fbf2400be90 comp rx=0 tx=0).stop 2023-10-11T09:50:30.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.742+0000 7fbf40bf9700 1 -- 172.21.15.78:0/3405227340 shutdown_connections 2023-10-11T09:50:30.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.742+0000 7fbf40bf9700 1 --2- 172.21.15.78:0/3405227340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf3c15d030 0x7fbf3c15d480 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:30.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.742+0000 7fbf40bf9700 1 --2- 172.21.15.78:0/3405227340 >> v2:172.21.15.133:3300/0 conn(0x7fbf3c159490 0x7fbf3c0ce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:30.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.742+0000 7fbf40bf9700 1 -- 172.21.15.78:0/3405227340 >> 172.21.15.78:0/3405227340 conn(0x7fbf3c06aca0 msgr2=0x7fbf3c0b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:30.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.742+0000 7fbf40bf9700 1 -- 172.21.15.78:0/3405227340 shutdown_connections 2023-10-11T09:50:30.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.742+0000 7fbf40bf9700 1 -- 172.21.15.78:0/3405227340 wait complete. 2023-10-11T09:50:30.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.746+0000 7fbf40bf9700 1 Processor -- start 2023-10-11T09:50:30.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.746+0000 7fbf40bf9700 1 -- start start 2023-10-11T09:50:30.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.746+0000 7fbf40bf9700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf3c159490 0x7fbf3c13c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:30.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.746+0000 7fbf40bf9700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbf3c15d030 0x7fbf3c0b1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:30.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.746+0000 7fbf40bf9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf3c15d9c0 0x7fbf3c0b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:30.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.746+0000 7fbf40bf9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c13c690 con 0x7fbf3c15d9c0 2023-10-11T09:50:30.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.746+0000 7fbf40bf9700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c151ec0 con 0x7fbf3c159490 2023-10-11T09:50:30.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.746+0000 7fbf40bf9700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c1521a0 con 0x7fbf3c15d030 2023-10-11T09:50:30.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.746+0000 7fbf3ad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf3c15d9c0 0x7fbf3c0b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:30.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.746+0000 7fbf3ad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf3c15d9c0 0x7fbf3c0b21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35850/0 (socket says 172.21.15.78:35850) 2023-10-11T09:50:30.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.746+0000 7fbf3ad9d700 1 -- 172.21.15.78:0/1834818676 learned_addr learned my addr 172.21.15.78:0/1834818676 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:30.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.750+0000 7fbf3a59c700 1 --2- 172.21.15.78:0/1834818676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf3c159490 0x7fbf3c13c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:30.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.750+0000 7fbf39d9b700 1 --2- 172.21.15.78:0/1834818676 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbf3c15d030 0x7fbf3c0b1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:30.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.750+0000 7fbf3ad9d700 1 -- 172.21.15.78:0/1834818676 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbf3c15d030 msgr2=0x7fbf3c0b1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:30.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.750+0000 7fbf3ad9d700 1 --2- 172.21.15.78:0/1834818676 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbf3c15d030 0x7fbf3c0b1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:30.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.750+0000 7fbf3ad9d700 1 -- 172.21.15.78:0/1834818676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf3c159490 msgr2=0x7fbf3c13c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:30.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.750+0000 7fbf3ad9d700 1 --2- 172.21.15.78:0/1834818676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf3c159490 0x7fbf3c13c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:30.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.750+0000 7fbf3ad9d700 1 -- 172.21.15.78:0/1834818676 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf2400b810 con 0x7fbf3c15d9c0 2023-10-11T09:50:30.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.750+0000 7fbf3ad9d700 1 --2- 172.21.15.78:0/1834818676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf3c15d9c0 0x7fbf3c0b21d0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fbf3400cb50 tx=0x7fbf3400cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:30.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.750+0000 7fbf32ffd700 1 -- 172.21.15.78:0/1834818676 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf34003c20 con 0x7fbf3c15d9c0 2023-10-11T09:50:30.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.750+0000 7fbf32ffd700 1 -- 172.21.15.78:0/1834818676 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf34003d80 con 0x7fbf3c15d9c0 2023-10-11T09:50:30.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.750+0000 7fbf40bf9700 1 -- 172.21.15.78:0/1834818676 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf3c152480 con 0x7fbf3c15d9c0 2023-10-11T09:50:30.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.750+0000 7fbf32ffd700 1 -- 172.21.15.78:0/1834818676 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3400ab70 con 0x7fbf3c15d9c0 2023-10-11T09:50:30.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.750+0000 7fbf40bf9700 1 -- 172.21.15.78:0/1834818676 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf3c151160 con 0x7fbf3c15d9c0 2023-10-11T09:50:30.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.750+0000 7fbf40bf9700 1 -- 172.21.15.78:0/1834818676 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf3c1529a0 con 0x7fbf3c15d9c0 2023-10-11T09:50:30.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.753+0000 7fbf32ffd700 1 -- 172.21.15.78:0/1834818676 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fbf3400acd0 con 0x7fbf3c15d9c0 2023-10-11T09:50:30.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.753+0000 7fbf32ffd700 1 --2- 172.21.15.78:0/1834818676 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbf1c065ed0 0x7fbf1c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:30.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.753+0000 7fbf32ffd700 1 -- 172.21.15.78:0/1834818676 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6638+0+0 (secure 0 0 0) 0x7fbf34083ac0 con 0x7fbf3c15d9c0 2023-10-11T09:50:30.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.753+0000 7fbf3a59c700 1 --2- 172.21.15.78:0/1834818676 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbf1c065ed0 0x7fbf1c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:30.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.753+0000 7fbf3a59c700 1 --2- 172.21.15.78:0/1834818676 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbf1c065ed0 0x7fbf1c068380 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c1f5390 tx=0x7fbf2c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:30.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.761+0000 7fbf32ffd700 1 -- 172.21.15.78:0/1834818676 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fbf34050ed0 con 0x7fbf3c15d9c0 2023-10-11T09:50:30.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.969+0000 7fbf40bf9700 1 -- 172.21.15.78:0/1834818676 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf3c04e530 con 0x7fbf3c15d9c0 2023-10-11T09:50:30.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.969+0000 7fbf32ffd700 1 -- 172.21.15.78:0/1834818676 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4652 (secure 0 0 0) 0x7fbf340544f0 con 0x7fbf3c15d9c0 2023-10-11T09:50:30.980 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:30.980 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":85,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6812,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/3697741427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3697741427},{"type":"v1","addr":"172.21.15.133:6833","nonce":3697741427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":6815,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2640710924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2640710924},{"type":"v1","addr":"172.21.15.133:6835","nonce":2640710924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8028,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/1538320637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":1538320637},{"type":"v1","addr":"172.21.15.78:6835","nonce":1538320637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8358,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/2876755099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2876755099},{"type":"v1","addr":"172.21.15.78:6837","nonce":2876755099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:50:28.502984+0000","modified":"2023-10-11T09:50:29.613806+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15,16],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-11T09:50:30.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.973+0000 7fbf40bf9700 1 -- 172.21.15.78:0/1834818676 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbf1c065ed0 msgr2=0x7fbf1c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:30.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.973+0000 7fbf40bf9700 1 --2- 172.21.15.78:0/1834818676 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbf1c065ed0 0x7fbf1c068380 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c1f5390 tx=0x7fbf2c008040 comp rx=0 tx=0).stop 2023-10-11T09:50:30.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.973+0000 7fbf40bf9700 1 -- 172.21.15.78:0/1834818676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf3c15d9c0 msgr2=0x7fbf3c0b21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:30.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.973+0000 7fbf40bf9700 1 --2- 172.21.15.78:0/1834818676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf3c15d9c0 0x7fbf3c0b21d0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fbf3400cb50 tx=0x7fbf3400cf10 comp rx=0 tx=0).stop 2023-10-11T09:50:30.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.973+0000 7fbf40bf9700 1 -- 172.21.15.78:0/1834818676 shutdown_connections 2023-10-11T09:50:30.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.973+0000 7fbf40bf9700 1 --2- 172.21.15.78:0/1834818676 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbf1c065ed0 0x7fbf1c068380 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:30.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.973+0000 7fbf40bf9700 1 --2- 172.21.15.78:0/1834818676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf3c15d9c0 0x7fbf3c0b21d0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:30.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.973+0000 7fbf40bf9700 1 --2- 172.21.15.78:0/1834818676 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbf3c15d030 0x7fbf3c0b1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:30.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.973+0000 7fbf40bf9700 1 --2- 172.21.15.78:0/1834818676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf3c159490 0x7fbf3c13c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:30.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.973+0000 7fbf40bf9700 1 -- 172.21.15.78:0/1834818676 >> 172.21.15.78:0/1834818676 conn(0x7fbf3c06aca0 msgr2=0x7fbf3c0c8010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:30.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.973+0000 7fbf40bf9700 1 -- 172.21.15.78:0/1834818676 shutdown_connections 2023-10-11T09:50:30.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:30.973+0000 7fbf40bf9700 1 -- 172.21.15.78:0/1834818676 wait complete. 2023-10-11T09:50:30.985 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 85 2023-10-11T09:50:31.003 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:50:31.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.213+0000 7f0fdf59e700 1 Processor -- start 2023-10-11T09:50:31.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.217+0000 7f0fdf59e700 1 -- start start 2023-10-11T09:50:31.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.217+0000 7f0fdf59e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0fe015b8e0 0x7f0fe006d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:31.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.217+0000 7f0fdf59e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0fe006d610 0x7f0fe00ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:31.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.217+0000 7f0fdf59e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fe015e010 con 0x7f0fe015a940 2023-10-11T09:50:31.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.217+0000 7f0fdf59e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fe015e280 con 0x7f0fe006d610 2023-10-11T09:50:31.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.217+0000 7f0fdf59e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fe015e560 con 0x7f0fe015b8e0 2023-10-11T09:50:31.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.217+0000 7f0fd5d9b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0fe006d610 0x7f0fe00ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:31.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.217+0000 7f0fd5d9b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0fe006d610 0x7f0fe00ca240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35858/0 (socket says 172.21.15.78:35858) 2023-10-11T09:50:31.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.217+0000 7f0fd5d9b700 1 -- 172.21.15.78:0/3474081438 learned_addr learned my addr 172.21.15.78:0/3474081438 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:31.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.217+0000 7f0fde59c700 1 --2- 172.21.15.78:0/3474081438 >> v2:172.21.15.133:3300/0 conn(0x7f0fe015b8e0 0x7f0fe006d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:31.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.217+0000 7f0fd5d9b700 1 -- 172.21.15.78:0/3474081438 >> v2:172.21.15.133:3300/0 conn(0x7f0fe015b8e0 msgr2=0x7f0fe006d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:31.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.217+0000 7f0fd5d9b700 1 --2- 172.21.15.78:0/3474081438 >> v2:172.21.15.133:3300/0 conn(0x7f0fe015b8e0 0x7f0fe006d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:31.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.217+0000 7f0fd5d9b700 1 -- 172.21.15.78:0/3474081438 >> v1:172.21.15.133:6789/0 conn(0x7f0fe015a940 legacy=0x7f0fe00c9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:31.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.217+0000 7f0fd5d9b700 1 -- 172.21.15.78:0/3474081438 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe015e840 con 0x7f0fe006d610 2023-10-11T09:50:31.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.217+0000 7f0fd5d9b700 1 --2- 172.21.15.78:0/3474081438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0fe006d610 0x7f0fe00ca240 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f0fe00b4cd0 tx=0x7f0fc8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76845f15feed802c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:31.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.221+0000 7f0fddd9b700 1 -- 172.21.15.78:0/3474081438 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc800b8a0 con 0x7f0fe006d610 2023-10-11T09:50:31.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.221+0000 7f0fddd9b700 1 -- 172.21.15.78:0/3474081438 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc800ba00 con 0x7f0fe006d610 2023-10-11T09:50:31.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.221+0000 7f0fddd9b700 1 -- 172.21.15.78:0/3474081438 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc800bd90 con 0x7f0fe006d610 2023-10-11T09:50:31.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.221+0000 7f0fdf59e700 1 -- 172.21.15.78:0/3474081438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0fe006d610 msgr2=0x7f0fe00ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:31.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.221+0000 7f0fdf59e700 1 --2- 172.21.15.78:0/3474081438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0fe006d610 0x7f0fe00ca240 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f0fe00b4cd0 tx=0x7f0fc8009bf0 comp rx=0 tx=0).stop 2023-10-11T09:50:31.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.221+0000 7f0fdf59e700 1 -- 172.21.15.78:0/3474081438 shutdown_connections 2023-10-11T09:50:31.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.221+0000 7f0fdf59e700 1 --2- 172.21.15.78:0/3474081438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0fe006d610 0x7f0fe00ca240 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:31.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.221+0000 7f0fdf59e700 1 --2- 172.21.15.78:0/3474081438 >> v2:172.21.15.133:3300/0 conn(0x7f0fe015b8e0 0x7f0fe006d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:31.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.221+0000 7f0fdf59e700 1 -- 172.21.15.78:0/3474081438 >> 172.21.15.78:0/3474081438 conn(0x7f0fe00cea60 msgr2=0x7f0fe00cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:31.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.221+0000 7f0fdf59e700 1 -- 172.21.15.78:0/3474081438 shutdown_connections 2023-10-11T09:50:31.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.221+0000 7f0fdf59e700 1 -- 172.21.15.78:0/3474081438 wait complete. 2023-10-11T09:50:31.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.221+0000 7f0fdf59e700 1 Processor -- start 2023-10-11T09:50:31.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.225+0000 7f0fdf59e700 1 -- start start 2023-10-11T09:50:31.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.225+0000 7f0fdf59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0fe006d610 0x7f0fe01f5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:31.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.225+0000 7f0fdf59e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fe015a940 0x7f0fe013d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:31.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.225+0000 7f0fdf59e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0fe015b8e0 0x7f0fe01f1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:31.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.225+0000 7f0fdf59e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe01f2cc0 con 0x7f0fe006d610 2023-10-11T09:50:31.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.225+0000 7f0fdf59e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe01f5790 con 0x7f0fe015a940 2023-10-11T09:50:31.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.225+0000 7f0fdf59e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe01f2700 con 0x7f0fe015b8e0 2023-10-11T09:50:31.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.225+0000 7f0fde59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0fe006d610 0x7f0fe01f5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:31.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.225+0000 7f0fde59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0fe006d610 0x7f0fe01f5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35862/0 (socket says 172.21.15.78:35862) 2023-10-11T09:50:31.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.225+0000 7f0fde59c700 1 -- 172.21.15.78:0/2056633329 learned_addr learned my addr 172.21.15.78:0/2056633329 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:31.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.225+0000 7f0fded9d700 1 --2- 172.21.15.78:0/2056633329 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0fe015b8e0 0x7f0fe01f1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:31.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.225+0000 7f0fd5d9b700 1 --2- 172.21.15.78:0/2056633329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fe015a940 0x7f0fe013d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:31.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.225+0000 7f0fde59c700 1 -- 172.21.15.78:0/2056633329 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0fe015b8e0 msgr2=0x7f0fe01f1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:31.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.225+0000 7f0fde59c700 1 --2- 172.21.15.78:0/2056633329 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0fe015b8e0 0x7f0fe01f1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:31.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.225+0000 7f0fde59c700 1 -- 172.21.15.78:0/2056633329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fe015a940 msgr2=0x7f0fe013d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:31.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.225+0000 7f0fde59c700 1 --2- 172.21.15.78:0/2056633329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fe015a940 0x7f0fe013d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:31.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.225+0000 7f0fde59c700 1 -- 172.21.15.78:0/2056633329 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc8009570 con 0x7f0fe006d610 2023-10-11T09:50:31.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.225+0000 7f0fded9d700 1 --2- 172.21.15.78:0/2056633329 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0fe015b8e0 0x7f0fe01f1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:50:31.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.225+0000 7f0fd5d9b700 1 --2- 172.21.15.78:0/2056633329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fe015a940 0x7f0fe013d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:50:31.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.225+0000 7f0fde59c700 1 --2- 172.21.15.78:0/2056633329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0fe006d610 0x7f0fe01f5c20 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000d800 tx=0x7f0fd000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:31.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.225+0000 7f0fd77fe700 1 -- 172.21.15.78:0/2056633329 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd00125f0 con 0x7f0fe006d610 2023-10-11T09:50:31.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.225+0000 7f0fd77fe700 1 -- 172.21.15.78:0/2056633329 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd0012750 con 0x7f0fe006d610 2023-10-11T09:50:31.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.225+0000 7f0fd77fe700 1 -- 172.21.15.78:0/2056633329 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd0012b00 con 0x7f0fe006d610 2023-10-11T09:50:31.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.225+0000 7f0fdf59e700 1 -- 172.21.15.78:0/2056633329 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fe01f29e0 con 0x7f0fe006d610 2023-10-11T09:50:31.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.225+0000 7f0fdf59e700 1 -- 172.21.15.78:0/2056633329 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fe01f6160 con 0x7f0fe006d610 2023-10-11T09:50:31.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.229+0000 7f0fdf59e700 1 -- 172.21.15.78:0/2056633329 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fe01f5280 con 0x7f0fe006d610 2023-10-11T09:50:31.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.229+0000 7f0fd77fe700 1 -- 172.21.15.78:0/2056633329 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f0fd0012c60 con 0x7f0fe006d610 2023-10-11T09:50:31.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.229+0000 7f0fd77fe700 1 --2- 172.21.15.78:0/2056633329 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0fc0065ed0 0x7f0fc0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:31.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.229+0000 7f0fd77fe700 1 -- 172.21.15.78:0/2056633329 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6638+0+0 (secure 0 0 0) 0x7f0fd0083ec0 con 0x7f0fe006d610 2023-10-11T09:50:31.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.229+0000 7f0fd5d9b700 1 --2- 172.21.15.78:0/2056633329 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0fc0065ed0 0x7f0fc0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:31.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.229+0000 7f0fd5d9b700 1 --2- 172.21.15.78:0/2056633329 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0fc0065ed0 0x7f0fc0068380 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8000c00 tx=0x7f0fc8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:31.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.237+0000 7f0fd77fe700 1 -- 172.21.15.78:0/2056633329 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f0fe01f5280 con 0x7f0fe006d610 2023-10-11T09:50:31.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.445+0000 7f0fdf59e700 1 -- 172.21.15.78:0/2056633329 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0fe01f5280 con 0x7f0fe006d610 2023-10-11T09:50:31.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.445+0000 7f0fd77fe700 1 -- 172.21.15.78:0/2056633329 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4652 (secure 0 0 0) 0x7f0fe01f5280 con 0x7f0fe006d610 2023-10-11T09:50:31.455 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:31.455 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":85,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6812,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/3697741427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3697741427},{"type":"v1","addr":"172.21.15.133:6833","nonce":3697741427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":6815,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2640710924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2640710924},{"type":"v1","addr":"172.21.15.133:6835","nonce":2640710924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8028,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/1538320637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":1538320637},{"type":"v1","addr":"172.21.15.78:6835","nonce":1538320637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8358,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/2876755099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2876755099},{"type":"v1","addr":"172.21.15.78:6837","nonce":2876755099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:50:28.502984+0000","modified":"2023-10-11T09:50:29.613806+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15,16],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-11T09:50:31.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.449+0000 7f0fdf59e700 1 -- 172.21.15.78:0/2056633329 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0fc0065ed0 msgr2=0x7f0fc0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:31.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.449+0000 7f0fdf59e700 1 --2- 172.21.15.78:0/2056633329 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0fc0065ed0 0x7f0fc0068380 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8000c00 tx=0x7f0fc8008030 comp rx=0 tx=0).stop 2023-10-11T09:50:31.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.449+0000 7f0fdf59e700 1 -- 172.21.15.78:0/2056633329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0fe006d610 msgr2=0x7f0fe01f5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:31.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.449+0000 7f0fdf59e700 1 --2- 172.21.15.78:0/2056633329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0fe006d610 0x7f0fe01f5c20 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000d800 tx=0x7f0fd000dbc0 comp rx=0 tx=0).stop 2023-10-11T09:50:31.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.449+0000 7f0fdf59e700 1 -- 172.21.15.78:0/2056633329 shutdown_connections 2023-10-11T09:50:31.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.449+0000 7f0fdf59e700 1 --2- 172.21.15.78:0/2056633329 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0fc0065ed0 0x7f0fc0068380 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:31.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.449+0000 7f0fdf59e700 1 --2- 172.21.15.78:0/2056633329 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0fe015b8e0 0x7f0fe01f1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:31.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.449+0000 7f0fdf59e700 1 --2- 172.21.15.78:0/2056633329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fe015a940 0x7f0fe013d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:31.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.449+0000 7f0fdf59e700 1 --2- 172.21.15.78:0/2056633329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0fe006d610 0x7f0fe01f5c20 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:31.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.449+0000 7f0fdf59e700 1 -- 172.21.15.78:0/2056633329 >> 172.21.15.78:0/2056633329 conn(0x7f0fe00cea60 msgr2=0x7f0fe00cee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:31.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.449+0000 7f0fdf59e700 1 -- 172.21.15.78:0/2056633329 shutdown_connections 2023-10-11T09:50:31.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.449+0000 7f0fdf59e700 1 -- 172.21.15.78:0/2056633329 wait complete. 2023-10-11T09:50:31.461 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 85 2023-10-11T09:50:31.482 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:50:31.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.649+0000 7fb558997700 1 Processor -- start 2023-10-11T09:50:31.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.653+0000 7fb558997700 1 -- start start 2023-10-11T09:50:31.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.653+0000 7fb558997700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fb55415b8e0 0x7fb55406d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:31.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.653+0000 7fb558997700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb55406d610 0x7fb5540ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:31.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.653+0000 7fb558997700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb55415e010 con 0x7fb55415a940 2023-10-11T09:50:31.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.653+0000 7fb558997700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb55415e280 con 0x7fb55415b8e0 2023-10-11T09:50:31.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.653+0000 7fb558997700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb55415e560 con 0x7fb55406d610 2023-10-11T09:50:31.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.653+0000 7fb5537fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fb55415b8e0 0x7fb55406d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:31.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.653+0000 7fb553fff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fb55415a940 0x7fb5540c9020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:39218/0 (socket says 172.21.15.78:39218) 2023-10-11T09:50:31.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.653+0000 7fb553fff700 1 -- 172.21.15.78:0/3394464600 learned_addr learned my addr 172.21.15.78:0/3394464600 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:50:31.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.653+0000 7fb54affd700 1 --2- 172.21.15.78:0/3394464600 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb55406d610 0x7fb5540ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:31.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.653+0000 7fb552ffd700 1 -- 172.21.15.78:0/3394464600 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2477598050 0 0) 0x7fb55415e010 con 0x7fb55415a940 2023-10-11T09:50:31.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.653+0000 7fb552ffd700 1 -- 172.21.15.78:0/3394464600 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5400035d0 con 0x7fb55415a940 2023-10-11T09:50:31.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.653+0000 7fb5537fe700 1 -- 172.21.15.78:0/3394464600 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb55406d610 msgr2=0x7fb5540ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:31.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.653+0000 7fb5537fe700 1 --2- 172.21.15.78:0/3394464600 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb55406d610 0x7fb5540ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:31.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.653+0000 7fb5537fe700 1 -- 172.21.15.78:0/3394464600 >> v1:172.21.15.78:6789/0 conn(0x7fb55415a940 legacy=0x7fb5540c9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:31.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.653+0000 7fb5537fe700 1 -- 172.21.15.78:0/3394464600 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55415e840 con 0x7fb55415b8e0 2023-10-11T09:50:31.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.657+0000 7fb5537fe700 1 --2- 172.21.15.78:0/3394464600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb55415b8e0 0x7fb55406d0d0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fb544009850 tx=0x7fb544009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e31feac4fc60670e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:31.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.657+0000 7fb552ffd700 1 -- 172.21.15.78:0/3394464600 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb544011830 con 0x7fb55415b8e0 2023-10-11T09:50:31.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.657+0000 7fb552ffd700 1 -- 172.21.15.78:0/3394464600 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb544011990 con 0x7fb55415b8e0 2023-10-11T09:50:31.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.657+0000 7fb552ffd700 1 -- 172.21.15.78:0/3394464600 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb544011d40 con 0x7fb55415b8e0 2023-10-11T09:50:31.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.657+0000 7fb558997700 1 -- 172.21.15.78:0/3394464600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb55415b8e0 msgr2=0x7fb55406d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:31.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.657+0000 7fb558997700 1 --2- 172.21.15.78:0/3394464600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb55415b8e0 0x7fb55406d0d0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fb544009850 tx=0x7fb544009b60 comp rx=0 tx=0).stop 2023-10-11T09:50:31.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.657+0000 7fb558997700 1 -- 172.21.15.78:0/3394464600 shutdown_connections 2023-10-11T09:50:31.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.657+0000 7fb558997700 1 --2- 172.21.15.78:0/3394464600 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb55406d610 0x7fb5540ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:31.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.657+0000 7fb558997700 1 --2- 172.21.15.78:0/3394464600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb55415b8e0 0x7fb55406d0d0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:31.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.657+0000 7fb558997700 1 -- 172.21.15.78:0/3394464600 >> 172.21.15.78:0/3394464600 conn(0x7fb5540cea60 msgr2=0x7fb5540cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:31.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.657+0000 7fb558997700 1 -- 172.21.15.78:0/3394464600 shutdown_connections 2023-10-11T09:50:31.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.657+0000 7fb558997700 1 -- 172.21.15.78:0/3394464600 wait complete. 2023-10-11T09:50:31.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.661+0000 7fb558997700 1 Processor -- start 2023-10-11T09:50:31.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.661+0000 7fb558997700 1 -- start start 2023-10-11T09:50:31.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.661+0000 7fb558997700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb55406d610 0x7fb554142830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:31.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.661+0000 7fb558997700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb55415a940 0x7fb554151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:31.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.661+0000 7fb558997700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb55415b8e0 0x7fb5540b2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:31.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.661+0000 7fb558997700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb554142d70 con 0x7fb55406d610 2023-10-11T09:50:31.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.661+0000 7fb558997700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb554141790 con 0x7fb55415b8e0 2023-10-11T09:50:31.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.661+0000 7fb558997700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb554141a70 con 0x7fb55415a940 2023-10-11T09:50:31.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.661+0000 7fb5537fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb55406d610 0x7fb554142830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:31.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.661+0000 7fb5537fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb55406d610 0x7fb554142830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35868/0 (socket says 172.21.15.78:35868) 2023-10-11T09:50:31.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.661+0000 7fb5537fe700 1 -- 172.21.15.78:0/4104116624 learned_addr learned my addr 172.21.15.78:0/4104116624 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:31.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.661+0000 7fb54affd700 1 --2- 172.21.15.78:0/4104116624 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb55415a940 0x7fb554151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:31.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.661+0000 7fb553fff700 1 --2- 172.21.15.78:0/4104116624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb55415b8e0 0x7fb5540b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:31.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.661+0000 7fb5537fe700 1 -- 172.21.15.78:0/4104116624 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb55415a940 msgr2=0x7fb554151a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:31.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.661+0000 7fb5537fe700 1 --2- 172.21.15.78:0/4104116624 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb55415a940 0x7fb554151a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:31.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.661+0000 7fb5537fe700 1 -- 172.21.15.78:0/4104116624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb55415b8e0 msgr2=0x7fb5540b2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:31.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.661+0000 7fb5537fe700 1 --2- 172.21.15.78:0/4104116624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb55415b8e0 0x7fb5540b2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:31.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.661+0000 7fb5537fe700 1 -- 172.21.15.78:0/4104116624 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5440094b0 con 0x7fb55406d610 2023-10-11T09:50:31.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.665+0000 7fb5537fe700 1 --2- 172.21.15.78:0/4104116624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb55406d610 0x7fb554142830 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fb5440090b0 tx=0x7fb544004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:31.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.665+0000 7fb550ff9700 1 -- 172.21.15.78:0/4104116624 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb544004480 con 0x7fb55406d610 2023-10-11T09:50:31.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.665+0000 7fb550ff9700 1 -- 172.21.15.78:0/4104116624 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54401f070 con 0x7fb55406d610 2023-10-11T09:50:31.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.665+0000 7fb550ff9700 1 -- 172.21.15.78:0/4104116624 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb544003900 con 0x7fb55406d610 2023-10-11T09:50:31.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.665+0000 7fb558997700 1 -- 172.21.15.78:0/4104116624 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb554141d50 con 0x7fb55406d610 2023-10-11T09:50:31.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.665+0000 7fb558997700 1 -- 172.21.15.78:0/4104116624 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb554142310 con 0x7fb55406d610 2023-10-11T09:50:31.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.665+0000 7fb558997700 1 -- 172.21.15.78:0/4104116624 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb554142500 con 0x7fb55406d610 2023-10-11T09:50:31.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.665+0000 7fb550ff9700 1 -- 172.21.15.78:0/4104116624 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fb544021030 con 0x7fb55406d610 2023-10-11T09:50:31.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.669+0000 7fb550ff9700 1 --2- 172.21.15.78:0/4104116624 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb534065ed0 0x7fb534068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:31.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.669+0000 7fb550ff9700 1 -- 172.21.15.78:0/4104116624 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6638+0+0 (secure 0 0 0) 0x7fb544084ee0 con 0x7fb55406d610 2023-10-11T09:50:31.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.669+0000 7fb54affd700 1 --2- 172.21.15.78:0/4104116624 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb534065ed0 0x7fb534068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:31.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.669+0000 7fb54affd700 1 --2- 172.21.15.78:0/4104116624 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb534065ed0 0x7fb534068380 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb5540cc2a0 tx=0x7fb53c00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:31.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.673+0000 7fb550ff9700 1 -- 172.21.15.78:0/4104116624 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fb544052400 con 0x7fb55406d610 2023-10-11T09:50:31.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.881+0000 7fb558997700 1 -- 172.21.15.78:0/4104116624 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb554065d40 con 0x7fb55406d610 2023-10-11T09:50:31.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.881+0000 7fb550ff9700 1 -- 172.21.15.78:0/4104116624 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4652 (secure 0 0 0) 0x7fb544006ce0 con 0x7fb55406d610 2023-10-11T09:50:31.890 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:31.890 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":85,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6812,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/3697741427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3697741427},{"type":"v1","addr":"172.21.15.133:6833","nonce":3697741427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":6815,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2640710924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2640710924},{"type":"v1","addr":"172.21.15.133:6835","nonce":2640710924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8028,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/1538320637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":1538320637},{"type":"v1","addr":"172.21.15.78:6835","nonce":1538320637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8358,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/2876755099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2876755099},{"type":"v1","addr":"172.21.15.78:6837","nonce":2876755099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:50:28.502984+0000","modified":"2023-10-11T09:50:29.613806+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15,16],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-11T09:50:31.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.885+0000 7fb558997700 1 -- 172.21.15.78:0/4104116624 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb534065ed0 msgr2=0x7fb534068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:31.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.885+0000 7fb558997700 1 --2- 172.21.15.78:0/4104116624 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb534065ed0 0x7fb534068380 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb5540cc2a0 tx=0x7fb53c00b040 comp rx=0 tx=0).stop 2023-10-11T09:50:31.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.885+0000 7fb558997700 1 -- 172.21.15.78:0/4104116624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb55406d610 msgr2=0x7fb554142830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:31.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.885+0000 7fb558997700 1 --2- 172.21.15.78:0/4104116624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb55406d610 0x7fb554142830 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fb5440090b0 tx=0x7fb544004770 comp rx=0 tx=0).stop 2023-10-11T09:50:31.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.885+0000 7fb558997700 1 -- 172.21.15.78:0/4104116624 shutdown_connections 2023-10-11T09:50:31.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.885+0000 7fb558997700 1 --2- 172.21.15.78:0/4104116624 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb534065ed0 0x7fb534068380 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:31.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.885+0000 7fb558997700 1 --2- 172.21.15.78:0/4104116624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb55415b8e0 0x7fb5540b2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:31.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.885+0000 7fb558997700 1 --2- 172.21.15.78:0/4104116624 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb55415a940 0x7fb554151a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:31.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.885+0000 7fb558997700 1 --2- 172.21.15.78:0/4104116624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb55406d610 0x7fb554142830 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:31.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.885+0000 7fb558997700 1 -- 172.21.15.78:0/4104116624 >> 172.21.15.78:0/4104116624 conn(0x7fb5540cea60 msgr2=0x7fb5540717f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:31.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.885+0000 7fb558997700 1 -- 172.21.15.78:0/4104116624 shutdown_connections 2023-10-11T09:50:31.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.885+0000 7fb558997700 1 -- 172.21.15.78:0/4104116624 wait complete. 2023-10-11T09:50:31.898 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 85 2023-10-11T09:50:31.915 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset session 2023-10-11T09:50:31.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.969+0000 7fa640600380 1 -- start start 2023-10-11T09:50:31.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.969+0000 7fa640600380 1 Processor -- start 2023-10-11T09:50:31.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.969+0000 7fa640600380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a48b9f9800 0x55a48ba9e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:31.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.969+0000 7fa640600380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a48b9f9400 0x55a48ba9eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:31.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.969+0000 7fa640600380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a48b9f9000 0x55a48ba9f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:31.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.969+0000 7fa640600380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a48ba78780 con 0x55a48b9f9800 2023-10-11T09:50:31.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.969+0000 7fa640600380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a48ba78a00 con 0x55a48b9f9400 2023-10-11T09:50:31.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.969+0000 7fa640600380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a48ba78640 con 0x55a48b9f9000 2023-10-11T09:50:31.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.969+0000 7fa63edba700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a48b9f9800 0x55a48ba9e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:31.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.969+0000 7fa63edba700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a48b9f9800 0x55a48ba9e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35880/0 (socket says 172.21.15.78:35880) 2023-10-11T09:50:31.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.969+0000 7fa63edba700 1 -- 172.21.15.78:0/2830676755 learned_addr learned my addr 172.21.15.78:0/2830676755 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:31.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.969+0000 7fa63f5bb700 1 --2- 172.21.15.78:0/2830676755 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a48b9f9000 0x55a48ba9f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:31.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.969+0000 7fa63e5b9700 1 --2- 172.21.15.78:0/2830676755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a48b9f9400 0x55a48ba9eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:31.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.969+0000 7fa63edba700 1 -- 172.21.15.78:0/2830676755 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a48b9f9000 msgr2=0x55a48ba9f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:31.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.969+0000 7fa63edba700 1 --2- 172.21.15.78:0/2830676755 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a48b9f9000 0x55a48ba9f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:31.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.969+0000 7fa63edba700 1 -- 172.21.15.78:0/2830676755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a48b9f9400 msgr2=0x55a48ba9eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:31.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.969+0000 7fa63edba700 1 --2- 172.21.15.78:0/2830676755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a48b9f9400 0x55a48ba9eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:31.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.969+0000 7fa63edba700 1 -- 172.21.15.78:0/2830676755 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a48ba1ab60 con 0x55a48b9f9800 2023-10-11T09:50:31.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.969+0000 7fa63edba700 1 --2- 172.21.15.78:0/2830676755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a48b9f9800 0x55a48ba9e580 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x55a48c7d2c30 tx=0x55a48c7dcfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20bc81216c91db6a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:31.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.973+0000 7fa63c5b5700 1 -- 172.21.15.78:0/2830676755 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a48ba29600 con 0x55a48b9f9800 2023-10-11T09:50:31.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.973+0000 7fa63c5b5700 1 -- 172.21.15.78:0/2830676755 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a48ba44480 con 0x55a48b9f9800 2023-10-11T09:50:31.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.973+0000 7fa63c5b5700 1 -- 172.21.15.78:0/2830676755 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a48ba291e0 con 0x55a48b9f9800 2023-10-11T09:50:31.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.973+0000 7fa640600380 1 -- 172.21.15.78:0/2830676755 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a48ba1a340 con 0x55a48b9f9800 2023-10-11T09:50:31.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.973+0000 7fa640600380 1 -- 172.21.15.78:0/2830676755 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55a48ba1a000 con 0x55a48b9f9800 2023-10-11T09:50:31.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.973+0000 7fa640600380 4 waiting for MDS map... 2023-10-11T09:50:31.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.973+0000 7fa63c5b5700 1 -- 172.21.15.78:0/2830676755 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(110..110 src has 1..110) v4 ==== 6638+0+0 (secure 0 0 0) 0x55a48ba32c40 con 0x55a48b9f9800 2023-10-11T09:50:31.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.973+0000 7fa63c5b5700 1 -- 172.21.15.78:0/2830676755 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 85) v1 ==== 3118+0+0 (secure 0 0 0) 0x55a48bce6000 con 0x55a48b9f9800 2023-10-11T09:50:31.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.973+0000 7fa640600380 4 Got MDS map 85 2023-10-11T09:50:31.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.973+0000 7fa640600380 10 main: main 2023-10-11T09:50:31.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.973+0000 7fa640600380 4 main: connecting to RADOS... 2023-10-11T09:50:31.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.973+0000 7fa640600380 1 Processor -- start 2023-10-11T09:50:31.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.973+0000 7fa640600380 1 -- start start 2023-10-11T09:50:31.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.973+0000 7fa640600380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a48c770400 0x55a48ba9f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:31.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.973+0000 7fa640600380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a48c770800 0x55a48ba9fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:31.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.973+0000 7fa640600380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a48c770c00 0x55a48baa0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:31.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.973+0000 7fa640600380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a48ba2fe00 con 0x55a48c770400 2023-10-11T09:50:31.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.973+0000 7fa640600380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a48ba78a00 con 0x55a48c770c00 2023-10-11T09:50:31.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.973+0000 7fa640600380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a48ba78500 con 0x55a48c770800 2023-10-11T09:50:31.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.973+0000 7fa63f5bb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a48c770400 0x55a48ba9f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:31.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.973+0000 7fa63f5bb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a48c770400 0x55a48ba9f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35886/0 (socket says 172.21.15.78:35886) 2023-10-11T09:50:31.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.973+0000 7fa63f5bb700 1 -- 172.21.15.78:0/2486112400 learned_addr learned my addr 172.21.15.78:0/2486112400 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:31.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.977+0000 7fa63edba700 1 --2- 172.21.15.78:0/2486112400 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a48c770800 0x55a48ba9fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:31.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.977+0000 7fa63e5b9700 1 --2- 172.21.15.78:0/2486112400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a48c770c00 0x55a48baa0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:31.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.977+0000 7fa63f5bb700 1 -- 172.21.15.78:0/2486112400 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a48c770800 msgr2=0x55a48ba9fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:31.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.977+0000 7fa63f5bb700 1 --2- 172.21.15.78:0/2486112400 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a48c770800 0x55a48ba9fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:31.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.977+0000 7fa63f5bb700 1 -- 172.21.15.78:0/2486112400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a48c770c00 msgr2=0x55a48baa0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:31.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.977+0000 7fa63f5bb700 1 --2- 172.21.15.78:0/2486112400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a48c770c00 0x55a48baa0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:31.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.977+0000 7fa63f5bb700 1 -- 172.21.15.78:0/2486112400 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a48ba1aea0 con 0x55a48c770400 2023-10-11T09:50:31.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.977+0000 7fa63f5bb700 1 --2- 172.21.15.78:0/2486112400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a48c770400 0x55a48ba9f600 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x55a48bd440f0 tx=0x55a48bcf0c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f2471170668cd6d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:31.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.977+0000 7fa63a5b1700 1 -- 172.21.15.78:0/2486112400 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a48ba29600 con 0x55a48c770400 2023-10-11T09:50:31.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.977+0000 7fa63a5b1700 1 -- 172.21.15.78:0/2486112400 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a48ba44480 con 0x55a48c770400 2023-10-11T09:50:31.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.977+0000 7fa63a5b1700 1 -- 172.21.15.78:0/2486112400 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a48ba291e0 con 0x55a48c770400 2023-10-11T09:50:31.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.977+0000 7fa640600380 1 -- 172.21.15.78:0/2486112400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a48c770400 msgr2=0x55a48ba9f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:31.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.977+0000 7fa640600380 1 --2- 172.21.15.78:0/2486112400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a48c770400 0x55a48ba9f600 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x55a48bd440f0 tx=0x55a48bcf0c00 comp rx=0 tx=0).stop 2023-10-11T09:50:31.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.977+0000 7fa640600380 1 -- 172.21.15.78:0/2486112400 shutdown_connections 2023-10-11T09:50:31.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.977+0000 7fa640600380 1 --2- 172.21.15.78:0/2486112400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a48c770c00 0x55a48baa0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:31.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.977+0000 7fa640600380 1 --2- 172.21.15.78:0/2486112400 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a48c770800 0x55a48ba9fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:31.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.977+0000 7fa640600380 1 --2- 172.21.15.78:0/2486112400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a48c770400 0x55a48ba9f600 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:31.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.977+0000 7fa640600380 1 -- 172.21.15.78:0/2486112400 >> 172.21.15.78:0/2486112400 conn(0x55a48b9f8c00 msgr2=0x55a48bae5800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:31.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.977+0000 7fa640600380 1 -- 172.21.15.78:0/2486112400 shutdown_connections 2023-10-11T09:50:31.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.977+0000 7fa640600380 1 -- 172.21.15.78:0/2486112400 wait complete. 2023-10-11T09:50:31.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.977+0000 7fa640600380 1 Processor -- start 2023-10-11T09:50:31.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.977+0000 7fa640600380 1 -- start start 2023-10-11T09:50:32.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.981+0000 7fa640600380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a48c770400 0x55a48baa1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.981+0000 7fa640600380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a48c770800 0x55a48baa0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.981+0000 7fa640600380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a48c770c00 0x55a48baa0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.981+0000 7fa640600380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a48ba783c0 con 0x55a48c770c00 2023-10-11T09:50:32.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.981+0000 7fa640600380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a48ba78280 con 0x55a48c770800 2023-10-11T09:50:32.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.981+0000 7fa640600380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a48ba78140 con 0x55a48c770400 2023-10-11T09:50:32.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.981+0000 7fa63e5b9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a48c770c00 0x55a48baa0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.981+0000 7fa63e5b9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a48c770c00 0x55a48baa0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35900/0 (socket says 172.21.15.78:35900) 2023-10-11T09:50:32.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.981+0000 7fa63e5b9700 1 -- 172.21.15.78:0/3754341209 learned_addr learned my addr 172.21.15.78:0/3754341209 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:32.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.981+0000 7fa63f5bb700 1 --2- 172.21.15.78:0/3754341209 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a48c770400 0x55a48baa1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.981+0000 7fa63edba700 1 --2- 172.21.15.78:0/3754341209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a48c770800 0x55a48baa0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.981+0000 7fa63e5b9700 1 -- 172.21.15.78:0/3754341209 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a48c770400 msgr2=0x55a48baa1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.981+0000 7fa63e5b9700 1 --2- 172.21.15.78:0/3754341209 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a48c770400 0x55a48baa1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.981+0000 7fa63e5b9700 1 -- 172.21.15.78:0/3754341209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a48c770800 msgr2=0x55a48baa0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.981+0000 7fa63e5b9700 1 --2- 172.21.15.78:0/3754341209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a48c770800 0x55a48baa0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.981+0000 7fa63e5b9700 1 -- 172.21.15.78:0/3754341209 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a48ba1ab60 con 0x55a48c770c00 2023-10-11T09:50:32.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.981+0000 7fa63f5bb700 1 --2- 172.21.15.78:0/3754341209 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a48c770400 0x55a48baa1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:50:32.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.981+0000 7fa63edba700 1 --2- 172.21.15.78:0/3754341209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a48c770800 0x55a48baa0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:50:32.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.981+0000 7fa63e5b9700 1 --2- 172.21.15.78:0/3754341209 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a48c770c00 0x55a48baa0680 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x55a48bd44e40 tx=0x55a48c7dc060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.981+0000 7fa638dae700 1 -- 172.21.15.78:0/3754341209 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a48ba29080 con 0x55a48c770c00 2023-10-11T09:50:32.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.981+0000 7fa638dae700 1 -- 172.21.15.78:0/3754341209 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a48c7ea480 con 0x55a48c770c00 2023-10-11T09:50:32.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.981+0000 7fa640600380 1 -- 172.21.15.78:0/3754341209 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a48ba1a340 con 0x55a48c770c00 2023-10-11T09:50:32.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.981+0000 7fa638dae700 1 -- 172.21.15.78:0/3754341209 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a48ba29600 con 0x55a48c770c00 2023-10-11T09:50:32.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.981+0000 7fa640600380 1 -- 172.21.15.78:0/3754341209 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a48ba1b520 con 0x55a48c770c00 2023-10-11T09:50:32.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.981+0000 7fa640600380 4 main: resolving pool 14 2023-10-11T09:50:32.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.985+0000 7fa638dae700 1 -- 172.21.15.78:0/3754341209 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x55a48ba76840 con 0x55a48c770c00 2023-10-11T09:50:32.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.985+0000 7fa638dae700 1 --2- 172.21.15.78:0/3754341209 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55a48be40000 0x55a48baa1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.985+0000 7fa638dae700 1 -- 172.21.15.78:0/3754341209 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6638+0+0 (secure 0 0 0) 0x55a48ba32a80 con 0x55a48c770c00 2023-10-11T09:50:32.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.985+0000 7fa640600380 4 main: creating IoCtx.. 2023-10-11T09:50:32.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.985+0000 7fa640600380 1 --2- 172.21.15.78:0/3754341209 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55a48c771000 0x55a48baa0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.985+0000 7fa63f5bb700 1 --2- 172.21.15.78:0/3754341209 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55a48be40000 0x55a48baa1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.985+0000 7fa640600380 1 -- 172.21.15.78:0/3754341209 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:1 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-clear,omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55a48c771400 con 0x55a48c771000 2023-10-11T09:50:32.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.985+0000 7fa63edba700 1 --2- 172.21.15.78:0/3754341209 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55a48c771000 0x55a48baa0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.985+0000 7fa63f5bb700 1 --2- 172.21.15.78:0/3754341209 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55a48be40000 0x55a48baa1700 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x55a48bd440f0 tx=0x55a48c7dcf60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.985+0000 7fa63edba700 1 --2- 172.21.15.78:0/3754341209 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55a48c771000 0x55a48baa0100 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.989+0000 7fa63edba700 1 -- 172.21.15.78:0/3754341209 <== osd.0 v2:172.21.15.78:6800/35735 1 ==== osd_op_reply(1 mds0_sessionmap [omap-clear,omap-set-header] v110'5 uv5 ondisk = 0) v8 ==== 201+0+0 (crc 0 0 0) 0x55a48c7d8fc0 con 0x55a48c771000 2023-10-11T09:50:32.007 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:50:32.007 INFO:teuthology.orchestra.run.smithi078.stdout: "0": { 2023-10-11T09:50:32.007 INFO:teuthology.orchestra.run.smithi078.stdout: "data": {}, 2023-10-11T09:50:32.007 INFO:teuthology.orchestra.run.smithi078.stdout: "result": 0 2023-10-11T09:50:32.008 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:50:32.008 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:50:32.008 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:32.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.989+0000 7fa640600380 1 -- 172.21.15.78:0/3754341209 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55a48c771000 msgr2=0x55a48baa0100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.989+0000 7fa640600380 1 --2- 172.21.15.78:0/3754341209 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55a48c771000 0x55a48baa0100 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.989+0000 7fa640600380 1 -- 172.21.15.78:0/3754341209 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55a48be40000 msgr2=0x55a48baa1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.989+0000 7fa640600380 1 --2- 172.21.15.78:0/3754341209 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55a48be40000 0x55a48baa1700 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x55a48bd440f0 tx=0x55a48c7dcf60 comp rx=0 tx=0).stop 2023-10-11T09:50:32.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.989+0000 7fa640600380 1 -- 172.21.15.78:0/3754341209 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a48c770c00 msgr2=0x55a48baa0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.989+0000 7fa640600380 1 --2- 172.21.15.78:0/3754341209 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a48c770c00 0x55a48baa0680 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x55a48bd44e40 tx=0x55a48c7dc060 comp rx=0 tx=0).stop 2023-10-11T09:50:32.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.989+0000 7fa63e5b9700 1 -- 172.21.15.78:0/3754341209 reap_dead start 2023-10-11T09:50:32.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.989+0000 7fa640600380 1 -- 172.21.15.78:0/3754341209 shutdown_connections 2023-10-11T09:50:32.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.989+0000 7fa640600380 1 -- 172.21.15.78:0/3754341209 >> 172.21.15.78:0/3754341209 conn(0x55a48b9f8c00 msgr2=0x55a48bae5800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:32.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.989+0000 7fa640600380 1 -- 172.21.15.78:0/3754341209 shutdown_connections 2023-10-11T09:50:32.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.989+0000 7fa640600380 1 -- 172.21.15.78:0/3754341209 wait complete. 2023-10-11T09:50:32.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.989+0000 7fa640600380 1 -- 172.21.15.78:0/2830676755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a48b9f9800 msgr2=0x55a48ba9e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.989+0000 7fa640600380 1 --2- 172.21.15.78:0/2830676755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a48b9f9800 0x55a48ba9e580 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x55a48c7d2c30 tx=0x55a48c7dcfc0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.989+0000 7fa640600380 1 -- 172.21.15.78:0/2830676755 shutdown_connections 2023-10-11T09:50:32.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.989+0000 7fa640600380 1 --2- 172.21.15.78:0/2830676755 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a48b9f9000 0x55a48ba9f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.989+0000 7fa640600380 1 --2- 172.21.15.78:0/2830676755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a48b9f9400 0x55a48ba9eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.989+0000 7fa640600380 1 --2- 172.21.15.78:0/2830676755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a48b9f9800 0x55a48ba9e580 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.989+0000 7fa640600380 1 -- 172.21.15.78:0/2830676755 >> 172.21.15.78:0/2830676755 conn(0x55a48b9f9c00 msgr2=0x55a48bae5000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:32.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.989+0000 7fa640600380 1 -- 172.21.15.78:0/2830676755 shutdown_connections 2023-10-11T09:50:32.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:31.989+0000 7fa640600380 1 -- 172.21.15.78:0/2830676755 wait complete. 2023-10-11T09:50:32.014 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'session'] in time 0:00:00.098706, result: { "0": { "data": {}, "result": 0 } } 2023-10-11T09:50:32.014 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset snap 2023-10-11T09:50:32.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.069+0000 7ff036562380 1 -- start start 2023-10-11T09:50:32.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.069+0000 7ff036562380 1 Processor -- start 2023-10-11T09:50:32.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.073+0000 7ff036562380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b1ea059800 0x55b1ea0fe580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.073+0000 7ff036562380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1ea059400 0x55b1ea0feb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.073+0000 7ff036562380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1ea059000 0x55b1ea0ff080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.073+0000 7ff036562380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ea08fe00 con 0x55b1ea059400 2023-10-11T09:50:32.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.073+0000 7ff036562380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ea0d8a00 con 0x55b1ea059000 2023-10-11T09:50:32.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.073+0000 7ff036562380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b1ea0d8640 con 0x55b1ea059800 2023-10-11T09:50:32.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.073+0000 7ff03451b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1ea059400 0x55b1ea0feb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.073+0000 7ff03451b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1ea059400 0x55b1ea0feb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35908/0 (socket says 172.21.15.78:35908) 2023-10-11T09:50:32.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.073+0000 7ff03451b700 1 -- 172.21.15.78:0/2480623592 learned_addr learned my addr 172.21.15.78:0/2480623592 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:32.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.073+0000 7ff034d1c700 1 --2- 172.21.15.78:0/2480623592 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b1ea059800 0x55b1ea0fe580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.073+0000 7ff03551d700 1 --2- 172.21.15.78:0/2480623592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1ea059000 0x55b1ea0ff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.073+0000 7ff03451b700 1 -- 172.21.15.78:0/2480623592 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b1ea059800 msgr2=0x55b1ea0fe580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:32.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.073+0000 7ff03451b700 1 --2- 172.21.15.78:0/2480623592 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b1ea059800 0x55b1ea0fe580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.073+0000 7ff03451b700 1 -- 172.21.15.78:0/2480623592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1ea059000 msgr2=0x55b1ea0ff080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:32.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.073+0000 7ff03451b700 1 --2- 172.21.15.78:0/2480623592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1ea059000 0x55b1ea0ff080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.073+0000 7ff03451b700 1 -- 172.21.15.78:0/2480623592 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1ea07ab60 con 0x55b1ea059400 2023-10-11T09:50:32.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.073+0000 7ff03451b700 1 --2- 172.21.15.78:0/2480623592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1ea059400 0x55b1ea0feb00 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x55b1ea1baba0 tx=0x55b1eae32fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de4bf1e41ebf4e10 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.073+0000 7ff032517700 1 -- 172.21.15.78:0/2480623592 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b1ea089600 con 0x55b1ea059400 2023-10-11T09:50:32.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.073+0000 7ff032517700 1 -- 172.21.15.78:0/2480623592 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b1eae3a900 con 0x55b1ea059400 2023-10-11T09:50:32.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.073+0000 7ff032517700 1 -- 172.21.15.78:0/2480623592 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b1ea0891e0 con 0x55b1ea059400 2023-10-11T09:50:32.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.073+0000 7ff036562380 1 -- 172.21.15.78:0/2480623592 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b1ea07a340 con 0x55b1ea059400 2023-10-11T09:50:32.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.073+0000 7ff036562380 1 -- 172.21.15.78:0/2480623592 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55b1ea07a000 con 0x55b1ea059400 2023-10-11T09:50:32.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.073+0000 7ff036562380 4 waiting for MDS map... 2023-10-11T09:50:32.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.073+0000 7ff032517700 1 -- 172.21.15.78:0/2480623592 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(110..110 src has 1..110) v4 ==== 6638+0+0 (secure 0 0 0) 0x55b1ea092c40 con 0x55b1ea059400 2023-10-11T09:50:32.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.073+0000 7ff032517700 1 -- 172.21.15.78:0/2480623592 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 85) v1 ==== 3118+0+0 (secure 0 0 0) 0x55b1eaed4000 con 0x55b1ea059400 2023-10-11T09:50:32.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.073+0000 7ff036562380 4 Got MDS map 85 2023-10-11T09:50:32.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.073+0000 7ff036562380 10 main: main 2023-10-11T09:50:32.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.073+0000 7ff036562380 4 main: connecting to RADOS... 2023-10-11T09:50:32.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.073+0000 7ff036562380 1 Processor -- start 2023-10-11T09:50:32.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.077+0000 7ff036562380 1 -- start start 2023-10-11T09:50:32.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.077+0000 7ff036562380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b1eaee2400 0x55b1ea0ff600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.077+0000 7ff036562380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1eaee2800 0x55b1ea0ffb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.077+0000 7ff036562380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1eaee2c00 0x55b1ea100100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.077+0000 7ff036562380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ea0d8500 con 0x55b1eaee2c00 2023-10-11T09:50:32.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.077+0000 7ff036562380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ea0d83c0 con 0x55b1eaee2800 2023-10-11T09:50:32.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.077+0000 7ff036562380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b1ea0d8a00 con 0x55b1eaee2400 2023-10-11T09:50:32.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.077+0000 7ff03451b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1eaee2c00 0x55b1ea100100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.077+0000 7ff03451b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1eaee2c00 0x55b1ea100100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35924/0 (socket says 172.21.15.78:35924) 2023-10-11T09:50:32.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.077+0000 7ff03451b700 1 -- 172.21.15.78:0/881535138 learned_addr learned my addr 172.21.15.78:0/881535138 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:32.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.077+0000 7ff03551d700 1 --2- 172.21.15.78:0/881535138 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b1eaee2400 0x55b1ea0ff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.077+0000 7ff034d1c700 1 --2- 172.21.15.78:0/881535138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1eaee2800 0x55b1ea0ffb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.077+0000 7ff03451b700 1 -- 172.21.15.78:0/881535138 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b1eaee2400 msgr2=0x55b1ea0ff600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:32.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.077+0000 7ff03451b700 1 --2- 172.21.15.78:0/881535138 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b1eaee2400 0x55b1ea0ff600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.077+0000 7ff03451b700 1 -- 172.21.15.78:0/881535138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1eaee2800 msgr2=0x55b1ea0ffb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:32.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.077+0000 7ff03451b700 1 --2- 172.21.15.78:0/881535138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1eaee2800 0x55b1ea0ffb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.077+0000 7ff03451b700 1 -- 172.21.15.78:0/881535138 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1ea07aea0 con 0x55b1eaee2c00 2023-10-11T09:50:32.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.077+0000 7ff034d1c700 1 --2- 172.21.15.78:0/881535138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1eaee2800 0x55b1ea0ffb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:50:32.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.077+0000 7ff03451b700 1 --2- 172.21.15.78:0/881535138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1eaee2c00 0x55b1ea100100 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x55b1ea39a090 tx=0x55b1eae32de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a386e8ed51fb1e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.077+0000 7ff030513700 1 -- 172.21.15.78:0/881535138 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b1ea088f20 con 0x55b1eaee2c00 2023-10-11T09:50:32.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.077+0000 7ff030513700 1 -- 172.21.15.78:0/881535138 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b1eae3ac00 con 0x55b1eaee2c00 2023-10-11T09:50:32.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.077+0000 7ff030513700 1 -- 172.21.15.78:0/881535138 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b1ea089600 con 0x55b1eaee2c00 2023-10-11T09:50:32.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.077+0000 7ff036562380 1 -- 172.21.15.78:0/881535138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1eaee2c00 msgr2=0x55b1ea100100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.077+0000 7ff036562380 1 --2- 172.21.15.78:0/881535138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1eaee2c00 0x55b1ea100100 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x55b1ea39a090 tx=0x55b1eae32de0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.077+0000 7ff036562380 1 -- 172.21.15.78:0/881535138 shutdown_connections 2023-10-11T09:50:32.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.077+0000 7ff036562380 1 --2- 172.21.15.78:0/881535138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1eaee2c00 0x55b1ea100100 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.077+0000 7ff036562380 1 --2- 172.21.15.78:0/881535138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1eaee2800 0x55b1ea0ffb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.077+0000 7ff036562380 1 --2- 172.21.15.78:0/881535138 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b1eaee2400 0x55b1ea0ff600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.077+0000 7ff036562380 1 -- 172.21.15.78:0/881535138 >> 172.21.15.78:0/881535138 conn(0x55b1ea058c00 msgr2=0x55b1ea145800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:32.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.077+0000 7ff036562380 1 -- 172.21.15.78:0/881535138 shutdown_connections 2023-10-11T09:50:32.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.077+0000 7ff036562380 1 -- 172.21.15.78:0/881535138 wait complete. 2023-10-11T09:50:32.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.081+0000 7ff036562380 1 Processor -- start 2023-10-11T09:50:32.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.081+0000 7ff036562380 1 -- start start 2023-10-11T09:50:32.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.081+0000 7ff036562380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1eaee2400 0x55b1ea101180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.081+0000 7ff036562380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b1eaee2800 0x55b1ea100c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.081+0000 7ff036562380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1eaee2c00 0x55b1ea100680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.081+0000 7ff036562380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ea0d8280 con 0x55b1eaee2c00 2023-10-11T09:50:32.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.081+0000 7ff036562380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1ea0d8140 con 0x55b1eaee2400 2023-10-11T09:50:32.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.081+0000 7ff036562380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b1ea0d8000 con 0x55b1eaee2800 2023-10-11T09:50:32.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.081+0000 7ff03451b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1eaee2c00 0x55b1ea100680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.081+0000 7ff03451b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1eaee2c00 0x55b1ea100680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35930/0 (socket says 172.21.15.78:35930) 2023-10-11T09:50:32.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.081+0000 7ff03451b700 1 -- 172.21.15.78:0/3571425382 learned_addr learned my addr 172.21.15.78:0/3571425382 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:32.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.081+0000 7ff03551d700 1 --2- 172.21.15.78:0/3571425382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1eaee2400 0x55b1ea101180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.081+0000 7ff034d1c700 1 --2- 172.21.15.78:0/3571425382 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b1eaee2800 0x55b1ea100c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.081+0000 7ff03451b700 1 -- 172.21.15.78:0/3571425382 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b1eaee2800 msgr2=0x55b1ea100c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.081+0000 7ff03451b700 1 --2- 172.21.15.78:0/3571425382 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b1eaee2800 0x55b1ea100c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.081+0000 7ff03451b700 1 -- 172.21.15.78:0/3571425382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1eaee2400 msgr2=0x55b1ea101180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.081+0000 7ff03451b700 1 --2- 172.21.15.78:0/3571425382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1eaee2400 0x55b1ea101180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.081+0000 7ff03451b700 1 -- 172.21.15.78:0/3571425382 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1ea07ab60 con 0x55b1eaee2c00 2023-10-11T09:50:32.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.081+0000 7ff03451b700 1 --2- 172.21.15.78:0/3571425382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1eaee2c00 0x55b1ea100680 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x55b1ea39a300 tx=0x55b1ea352c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.085+0000 7ff02ed10700 1 -- 172.21.15.78:0/3571425382 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b1ea0891e0 con 0x55b1eaee2c00 2023-10-11T09:50:32.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.085+0000 7ff02ed10700 1 -- 172.21.15.78:0/3571425382 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b1eae3a600 con 0x55b1eaee2c00 2023-10-11T09:50:32.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.085+0000 7ff036562380 1 -- 172.21.15.78:0/3571425382 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b1ea07a340 con 0x55b1eaee2c00 2023-10-11T09:50:32.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.085+0000 7ff02ed10700 1 -- 172.21.15.78:0/3571425382 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b1ea088dc0 con 0x55b1eaee2c00 2023-10-11T09:50:32.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.085+0000 7ff036562380 1 -- 172.21.15.78:0/3571425382 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b1ea07a000 con 0x55b1eaee2c00 2023-10-11T09:50:32.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.085+0000 7ff036562380 4 main: resolving pool 14 2023-10-11T09:50:32.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.085+0000 7ff02ed10700 1 -- 172.21.15.78:0/3571425382 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x55b1ea0d6840 con 0x55b1eaee2c00 2023-10-11T09:50:32.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.085+0000 7ff02ed10700 1 --2- 172.21.15.78:0/3571425382 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55b1ea49e000 0x55b1ea101700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.085+0000 7ff02ed10700 1 -- 172.21.15.78:0/3571425382 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6638+0+0 (secure 0 0 0) 0x55b1ea092a80 con 0x55b1eaee2c00 2023-10-11T09:50:32.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.085+0000 7ff036562380 4 main: creating IoCtx.. 2023-10-11T09:50:32.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.085+0000 7ff03551d700 1 --2- 172.21.15.78:0/3571425382 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55b1ea49e000 0x55b1ea101700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.085+0000 7ff036562380 1 --2- 172.21.15.78:0/3571425382 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55b1eaee3000 0x55b1ea100100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.085+0000 7ff036562380 1 -- 172.21.15.78:0/3571425382 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:1 14.d 14:b50e409b:::mds_snaptable:head [writefull 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b1eaee3400 con 0x55b1eaee3000 2023-10-11T09:50:32.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.089+0000 7ff034d1c700 1 --2- 172.21.15.78:0/3571425382 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55b1eaee3000 0x55b1ea100100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.089+0000 7ff03551d700 1 --2- 172.21.15.78:0/3571425382 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55b1ea49e000 0x55b1ea101700 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x55b1ea1ba1e0 tx=0x55b1ea352cc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.089+0000 7ff034d1c700 1 --2- 172.21.15.78:0/3571425382 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55b1eaee3000 0x55b1ea100100 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.093+0000 7ff034d1c700 1 -- 172.21.15.78:0/3571425382 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== osd_op_reply(1 mds_snaptable [writefull 0~70] v110'3 uv3 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x55b1eaed0fc0 con 0x55b1eaee3000 2023-10-11T09:50:32.108 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:50:32.109 INFO:teuthology.orchestra.run.smithi078.stdout: "result": 0 2023-10-11T09:50:32.109 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:50:32.109 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:32.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.093+0000 7ff036562380 1 -- 172.21.15.78:0/3571425382 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55b1eaee3000 msgr2=0x55b1ea100100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.093+0000 7ff036562380 1 --2- 172.21.15.78:0/3571425382 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55b1eaee3000 0x55b1ea100100 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.093+0000 7ff036562380 1 -- 172.21.15.78:0/3571425382 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55b1ea49e000 msgr2=0x55b1ea101700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.093+0000 7ff036562380 1 --2- 172.21.15.78:0/3571425382 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55b1ea49e000 0x55b1ea101700 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x55b1ea1ba1e0 tx=0x55b1ea352cc0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.093+0000 7ff036562380 1 -- 172.21.15.78:0/3571425382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1eaee2c00 msgr2=0x55b1ea100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.093+0000 7ff036562380 1 --2- 172.21.15.78:0/3571425382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1eaee2c00 0x55b1ea100680 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x55b1ea39a300 tx=0x55b1ea352c00 comp rx=0 tx=0).stop 2023-10-11T09:50:32.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.093+0000 7ff034d1c700 1 -- 172.21.15.78:0/3571425382 reap_dead start 2023-10-11T09:50:32.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.093+0000 7ff036562380 1 -- 172.21.15.78:0/3571425382 shutdown_connections 2023-10-11T09:50:32.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.093+0000 7ff036562380 1 -- 172.21.15.78:0/3571425382 >> 172.21.15.78:0/3571425382 conn(0x55b1ea058c00 msgr2=0x55b1ea145800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:32.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.093+0000 7ff036562380 1 -- 172.21.15.78:0/3571425382 shutdown_connections 2023-10-11T09:50:32.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.093+0000 7ff036562380 1 -- 172.21.15.78:0/3571425382 wait complete. 2023-10-11T09:50:32.114 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.093+0000 7ff036562380 1 -- 172.21.15.78:0/2480623592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1ea059400 msgr2=0x55b1ea0feb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.114 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.093+0000 7ff036562380 1 --2- 172.21.15.78:0/2480623592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1ea059400 0x55b1ea0feb00 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x55b1ea1baba0 tx=0x55b1eae32fc0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.114 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.093+0000 7ff036562380 1 -- 172.21.15.78:0/2480623592 shutdown_connections 2023-10-11T09:50:32.114 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.093+0000 7ff036562380 1 --2- 172.21.15.78:0/2480623592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1ea059000 0x55b1ea0ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.093+0000 7ff036562380 1 --2- 172.21.15.78:0/2480623592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1ea059400 0x55b1ea0feb00 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.093+0000 7ff036562380 1 --2- 172.21.15.78:0/2480623592 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b1ea059800 0x55b1ea0fe580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.093+0000 7ff036562380 1 -- 172.21.15.78:0/2480623592 >> 172.21.15.78:0/2480623592 conn(0x55b1ea059c00 msgr2=0x55b1ea145000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:32.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.093+0000 7ff036562380 1 -- 172.21.15.78:0/2480623592 shutdown_connections 2023-10-11T09:50:32.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.093+0000 7ff036562380 1 -- 172.21.15.78:0/2480623592 wait complete. 2023-10-11T09:50:32.116 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'snap'] in time 0:00:00.101318, result: { "result": 0 } 2023-10-11T09:50:32.116 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset inode 2023-10-11T09:50:32.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.173+0000 7fe0c8d8d380 1 -- start start 2023-10-11T09:50:32.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.173+0000 7fe0c8d8d380 1 Processor -- start 2023-10-11T09:50:32.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.173+0000 7fe0c8d8d380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55718aabf800 0x55718ab64580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.173+0000 7fe0c8d8d380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55718aabf400 0x55718ab64b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.173+0000 7fe0c8d8d380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55718aabf000 0x55718ab65080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.173+0000 7fe0c8d8d380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718ab3ea00 con 0x55718aabf800 2023-10-11T09:50:32.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.173+0000 7fe0c8d8d380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718ab3e780 con 0x55718aabf000 2023-10-11T09:50:32.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.173+0000 7fe0c8d8d380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55718ab3e640 con 0x55718aabf400 2023-10-11T09:50:32.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.173+0000 7fe0c7547700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55718aabf800 0x55718ab64580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.173+0000 7fe0c7547700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55718aabf800 0x55718ab64580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35960/0 (socket says 172.21.15.78:35960) 2023-10-11T09:50:32.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.173+0000 7fe0c7547700 1 -- 172.21.15.78:0/4121982977 learned_addr learned my addr 172.21.15.78:0/4121982977 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:32.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.173+0000 7fe0c6d46700 1 --2- 172.21.15.78:0/4121982977 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55718aabf400 0x55718ab64b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.173+0000 7fe0c7d48700 1 --2- 172.21.15.78:0/4121982977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55718aabf000 0x55718ab65080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.173+0000 7fe0c7547700 1 -- 172.21.15.78:0/4121982977 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55718aabf400 msgr2=0x55718ab64b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:32.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.173+0000 7fe0c7547700 1 --2- 172.21.15.78:0/4121982977 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55718aabf400 0x55718ab64b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.173+0000 7fe0c7547700 1 -- 172.21.15.78:0/4121982977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55718aabf000 msgr2=0x55718ab65080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:32.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.173+0000 7fe0c7547700 1 --2- 172.21.15.78:0/4121982977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55718aabf000 0x55718ab65080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.173+0000 7fe0c7547700 1 -- 172.21.15.78:0/4121982977 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55718aae0b60 con 0x55718aabf800 2023-10-11T09:50:32.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.173+0000 7fe0c6d46700 1 --2- 172.21.15.78:0/4121982977 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55718aabf400 0x55718ab64b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:50:32.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.177+0000 7fe0c7547700 1 --2- 172.21.15.78:0/4121982977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55718aabf800 0x55718ab64580 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x55718b89ac30 tx=0x55718b8a6fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb46561120a0702 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.177+0000 7fe0c4d42700 1 -- 172.21.15.78:0/4121982977 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55718aaef600 con 0x55718aabf800 2023-10-11T09:50:32.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.177+0000 7fe0c4d42700 1 -- 172.21.15.78:0/4121982977 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55718ab0a480 con 0x55718aabf800 2023-10-11T09:50:32.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.177+0000 7fe0c4d42700 1 -- 172.21.15.78:0/4121982977 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55718aaef1e0 con 0x55718aabf800 2023-10-11T09:50:32.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.177+0000 7fe0c8d8d380 1 -- 172.21.15.78:0/4121982977 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55718aae0340 con 0x55718aabf800 2023-10-11T09:50:32.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.177+0000 7fe0c8d8d380 1 -- 172.21.15.78:0/4121982977 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55718aae0000 con 0x55718aabf800 2023-10-11T09:50:32.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.177+0000 7fe0c8d8d380 4 waiting for MDS map... 2023-10-11T09:50:32.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.177+0000 7fe0c4d42700 1 -- 172.21.15.78:0/4121982977 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(110..110 src has 1..110) v4 ==== 6638+0+0 (secure 0 0 0) 0x55718aaf8c40 con 0x55718aabf800 2023-10-11T09:50:32.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.177+0000 7fe0c4d42700 1 -- 172.21.15.78:0/4121982977 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 85) v1 ==== 3118+0+0 (secure 0 0 0) 0x55718b8ae000 con 0x55718aabf800 2023-10-11T09:50:32.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.177+0000 7fe0c8d8d380 4 Got MDS map 85 2023-10-11T09:50:32.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.177+0000 7fe0c8d8d380 10 main: main 2023-10-11T09:50:32.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.177+0000 7fe0c8d8d380 4 main: connecting to RADOS... 2023-10-11T09:50:32.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.177+0000 7fe0c8d8d380 1 Processor -- start 2023-10-11T09:50:32.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.177+0000 7fe0c8d8d380 1 -- start start 2023-10-11T09:50:32.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.177+0000 7fe0c8d8d380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55718b82c400 0x55718ab65600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.177+0000 7fe0c8d8d380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55718b82c800 0x55718ab65b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.177+0000 7fe0c8d8d380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55718b82cc00 0x55718ab66100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.177+0000 7fe0c8d8d380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718aaf5e00 con 0x55718b82cc00 2023-10-11T09:50:32.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.177+0000 7fe0c8d8d380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718ab3e780 con 0x55718b82c800 2023-10-11T09:50:32.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.177+0000 7fe0c8d8d380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55718ab3e500 con 0x55718b82c400 2023-10-11T09:50:32.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.177+0000 7fe0c6d46700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55718b82cc00 0x55718ab66100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.177+0000 7fe0c6d46700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55718b82cc00 0x55718ab66100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35970/0 (socket says 172.21.15.78:35970) 2023-10-11T09:50:32.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.177+0000 7fe0c6d46700 1 -- 172.21.15.78:0/181349428 learned_addr learned my addr 172.21.15.78:0/181349428 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:32.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.177+0000 7fe0c7d48700 1 --2- 172.21.15.78:0/181349428 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55718b82c400 0x55718ab65600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.181+0000 7fe0c7547700 1 --2- 172.21.15.78:0/181349428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55718b82c800 0x55718ab65b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.181+0000 7fe0c6d46700 1 -- 172.21.15.78:0/181349428 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55718b82c400 msgr2=0x55718ab65600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:32.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.181+0000 7fe0c6d46700 1 --2- 172.21.15.78:0/181349428 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55718b82c400 0x55718ab65600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.181+0000 7fe0c6d46700 1 -- 172.21.15.78:0/181349428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55718b82c800 msgr2=0x55718ab65b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:32.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.181+0000 7fe0c6d46700 1 --2- 172.21.15.78:0/181349428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55718b82c800 0x55718ab65b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.181+0000 7fe0c6d46700 1 -- 172.21.15.78:0/181349428 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55718aae0ea0 con 0x55718b82cc00 2023-10-11T09:50:32.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.181+0000 7fe0c6d46700 1 --2- 172.21.15.78:0/181349428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55718b82cc00 0x55718ab66100 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x55718b89c150 tx=0x55718b8a6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e58c720738f6489e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.181+0000 7fe0c2d3e700 1 -- 172.21.15.78:0/181349428 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55718aaef080 con 0x55718b82cc00 2023-10-11T09:50:32.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.181+0000 7fe0c2d3e700 1 -- 172.21.15.78:0/181349428 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55718b8bb080 con 0x55718b82cc00 2023-10-11T09:50:32.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.181+0000 7fe0c2d3e700 1 -- 172.21.15.78:0/181349428 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55718aaef600 con 0x55718b82cc00 2023-10-11T09:50:32.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.181+0000 7fe0c8d8d380 1 -- 172.21.15.78:0/181349428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55718b82cc00 msgr2=0x55718ab66100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.181+0000 7fe0c8d8d380 1 --2- 172.21.15.78:0/181349428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55718b82cc00 0x55718ab66100 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x55718b89c150 tx=0x55718b8a6000 comp rx=0 tx=0).stop 2023-10-11T09:50:32.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.181+0000 7fe0c8d8d380 1 -- 172.21.15.78:0/181349428 shutdown_connections 2023-10-11T09:50:32.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.181+0000 7fe0c8d8d380 1 --2- 172.21.15.78:0/181349428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55718b82cc00 0x55718ab66100 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.181+0000 7fe0c8d8d380 1 --2- 172.21.15.78:0/181349428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55718b82c800 0x55718ab65b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.181+0000 7fe0c8d8d380 1 --2- 172.21.15.78:0/181349428 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55718b82c400 0x55718ab65600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.181+0000 7fe0c8d8d380 1 -- 172.21.15.78:0/181349428 >> 172.21.15.78:0/181349428 conn(0x55718aabec00 msgr2=0x55718ab7b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:32.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.181+0000 7fe0c8d8d380 1 -- 172.21.15.78:0/181349428 shutdown_connections 2023-10-11T09:50:32.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.181+0000 7fe0c8d8d380 1 -- 172.21.15.78:0/181349428 wait complete. 2023-10-11T09:50:32.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.181+0000 7fe0c8d8d380 1 Processor -- start 2023-10-11T09:50:32.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.181+0000 7fe0c8d8d380 1 -- start start 2023-10-11T09:50:32.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.185+0000 7fe0c8d8d380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55718b82c400 0x55718ab67180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.185+0000 7fe0c8d8d380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55718b82c800 0x55718ab66c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.185+0000 7fe0c8d8d380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55718b82cc00 0x55718ab66680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.185+0000 7fe0c8d8d380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718ab3e3c0 con 0x55718b82c800 2023-10-11T09:50:32.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.185+0000 7fe0c8d8d380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55718ab3e280 con 0x55718b82c400 2023-10-11T09:50:32.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.185+0000 7fe0c8d8d380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55718ab3e140 con 0x55718b82cc00 2023-10-11T09:50:32.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.185+0000 7fe0c7547700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55718b82c800 0x55718ab66c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.185+0000 7fe0c7547700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55718b82c800 0x55718ab66c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35976/0 (socket says 172.21.15.78:35976) 2023-10-11T09:50:32.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.185+0000 7fe0c7547700 1 -- 172.21.15.78:0/897158026 learned_addr learned my addr 172.21.15.78:0/897158026 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:32.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.185+0000 7fe0c6d46700 1 --2- 172.21.15.78:0/897158026 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55718b82cc00 0x55718ab66680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.185+0000 7fe0c7d48700 1 --2- 172.21.15.78:0/897158026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55718b82c400 0x55718ab67180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.185+0000 7fe0c7547700 1 -- 172.21.15.78:0/897158026 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55718b82cc00 msgr2=0x55718ab66680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.185+0000 7fe0c7547700 1 --2- 172.21.15.78:0/897158026 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55718b82cc00 0x55718ab66680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.185+0000 7fe0c7547700 1 -- 172.21.15.78:0/897158026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55718b82c400 msgr2=0x55718ab67180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.185+0000 7fe0c7547700 1 --2- 172.21.15.78:0/897158026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55718b82c400 0x55718ab67180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.185+0000 7fe0c7547700 1 -- 172.21.15.78:0/897158026 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55718aae11e0 con 0x55718b82c800 2023-10-11T09:50:32.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.185+0000 7fe0c7547700 1 --2- 172.21.15.78:0/897158026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55718b82c800 0x55718ab66c00 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x55718b89cdb0 tx=0x55718b8a6180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.185+0000 7fe0c153b700 1 -- 172.21.15.78:0/897158026 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55718aaeef20 con 0x55718b82c800 2023-10-11T09:50:32.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.185+0000 7fe0c153b700 1 -- 172.21.15.78:0/897158026 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55718b8bad80 con 0x55718b82c800 2023-10-11T09:50:32.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.185+0000 7fe0c153b700 1 -- 172.21.15.78:0/897158026 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55718aaef080 con 0x55718b82c800 2023-10-11T09:50:32.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.185+0000 7fe0c8d8d380 1 -- 172.21.15.78:0/897158026 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55718aae0ea0 con 0x55718b82c800 2023-10-11T09:50:32.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.185+0000 7fe0c8d8d380 1 -- 172.21.15.78:0/897158026 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55718aae0b60 con 0x55718b82c800 2023-10-11T09:50:32.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.185+0000 7fe0c8d8d380 4 main: resolving pool 14 2023-10-11T09:50:32.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.189+0000 7fe0c153b700 1 -- 172.21.15.78:0/897158026 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x55718ab3c840 con 0x55718b82c800 2023-10-11T09:50:32.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.189+0000 7fe0c153b700 1 --2- 172.21.15.78:0/897158026 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55718af02000 0x55718ab67700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.189+0000 7fe0c153b700 1 -- 172.21.15.78:0/897158026 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6638+0+0 (secure 0 0 0) 0x55718aaf8a80 con 0x55718b82c800 2023-10-11T09:50:32.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.189+0000 7fe0c8d8d380 4 main: creating IoCtx.. 2023-10-11T09:50:32.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.189+0000 7fe0c8d8d380 1 --2- 172.21.15.78:0/897158026 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55718b82d000 0x55718ab66100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.189+0000 7fe0c8d8d380 1 -- 172.21.15.78:0/897158026 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55718b82d400 con 0x55718b82d000 2023-10-11T09:50:32.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.189+0000 7fe0c7d48700 1 --2- 172.21.15.78:0/897158026 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55718af02000 0x55718ab67700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.189+0000 7fe0c6d46700 1 --2- 172.21.15.78:0/897158026 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55718b82d000 0x55718ab66100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.189+0000 7fe0c7d48700 1 --2- 172.21.15.78:0/897158026 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55718af02000 0x55718ab67700 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x55718b89a1b0 tx=0x55718adb6d20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.189+0000 7fe0c6d46700 1 --2- 172.21.15.78:0/897158026 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55718b82d000 0x55718ab66100 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.193+0000 7fe0c6d46700 1 -- 172.21.15.78:0/897158026 <== osd.2 v2:172.21.15.78:6808/35732 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v110'6 uv6 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x55718b948fc0 con 0x55718b82d000 2023-10-11T09:50:32.210 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:50:32.210 INFO:teuthology.orchestra.run.smithi078.stdout: "0": { 2023-10-11T09:50:32.210 INFO:teuthology.orchestra.run.smithi078.stdout: "data": {}, 2023-10-11T09:50:32.210 INFO:teuthology.orchestra.run.smithi078.stdout: "result": 0 2023-10-11T09:50:32.211 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:50:32.211 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:50:32.211 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:32.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.193+0000 7fe0c8d8d380 1 -- 172.21.15.78:0/897158026 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55718b82d000 msgr2=0x55718ab66100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.193+0000 7fe0c8d8d380 1 --2- 172.21.15.78:0/897158026 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55718b82d000 0x55718ab66100 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.193+0000 7fe0c8d8d380 1 -- 172.21.15.78:0/897158026 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55718af02000 msgr2=0x55718ab67700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.193+0000 7fe0c8d8d380 1 --2- 172.21.15.78:0/897158026 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55718af02000 0x55718ab67700 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x55718b89a1b0 tx=0x55718adb6d20 comp rx=0 tx=0).stop 2023-10-11T09:50:32.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.193+0000 7fe0c8d8d380 1 -- 172.21.15.78:0/897158026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55718b82c800 msgr2=0x55718ab66c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.193+0000 7fe0c8d8d380 1 --2- 172.21.15.78:0/897158026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55718b82c800 0x55718ab66c00 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x55718b89cdb0 tx=0x55718b8a6180 comp rx=0 tx=0).stop 2023-10-11T09:50:32.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.193+0000 7fe0c6d46700 1 -- 172.21.15.78:0/897158026 reap_dead start 2023-10-11T09:50:32.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.193+0000 7fe0c8d8d380 1 -- 172.21.15.78:0/897158026 shutdown_connections 2023-10-11T09:50:32.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.193+0000 7fe0c8d8d380 1 -- 172.21.15.78:0/897158026 >> 172.21.15.78:0/897158026 conn(0x55718aabec00 msgr2=0x55718ab7b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:32.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.193+0000 7fe0c8d8d380 1 -- 172.21.15.78:0/897158026 shutdown_connections 2023-10-11T09:50:32.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.193+0000 7fe0c8d8d380 1 -- 172.21.15.78:0/897158026 wait complete. 2023-10-11T09:50:32.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.193+0000 7fe0c8d8d380 1 -- 172.21.15.78:0/4121982977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55718aabf800 msgr2=0x55718ab64580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.193+0000 7fe0c8d8d380 1 --2- 172.21.15.78:0/4121982977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55718aabf800 0x55718ab64580 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x55718b89ac30 tx=0x55718b8a6fc0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.197+0000 7fe0c8d8d380 1 -- 172.21.15.78:0/4121982977 shutdown_connections 2023-10-11T09:50:32.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.197+0000 7fe0c8d8d380 1 --2- 172.21.15.78:0/4121982977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55718aabf000 0x55718ab65080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.197+0000 7fe0c8d8d380 1 --2- 172.21.15.78:0/4121982977 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55718aabf400 0x55718ab64b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.197+0000 7fe0c8d8d380 1 --2- 172.21.15.78:0/4121982977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55718aabf800 0x55718ab64580 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.197+0000 7fe0c8d8d380 1 -- 172.21.15.78:0/4121982977 >> 172.21.15.78:0/4121982977 conn(0x55718aabfc00 msgr2=0x55718ab7b000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:32.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.197+0000 7fe0c8d8d380 1 -- 172.21.15.78:0/4121982977 shutdown_connections 2023-10-11T09:50:32.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.197+0000 7fe0c8d8d380 1 -- 172.21.15.78:0/4121982977 wait complete. 2023-10-11T09:50:32.217 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'inode'] in time 0:00:00.101051, result: { "0": { "data": {}, "result": 0 } } 2023-10-11T09:50:32.218 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2023-10-11T09:50:32.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.273+0000 7facfe48d380 1 -- start start 2023-10-11T09:50:32.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.273+0000 7facfe48d380 1 Processor -- start 2023-10-11T09:50:32.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.273+0000 7facfe48d380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56180d7ef800 0x56180d894580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.273+0000 7facfe48d380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56180d7ef400 0x56180d894b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.273+0000 7facfe48d380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56180d7ef000 0x56180d895080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.273+0000 7facfe48d380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56180d86ea00 con 0x56180d7ef400 2023-10-11T09:50:32.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.273+0000 7facfe48d380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56180d86e780 con 0x56180d7ef800 2023-10-11T09:50:32.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.273+0000 7facfe48d380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x56180d86e640 con 0x56180d7ef000 2023-10-11T09:50:32.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.273+0000 7facfc446700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56180d7ef400 0x56180d894b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.273+0000 7facfc446700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56180d7ef400 0x56180d894b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36002/0 (socket says 172.21.15.78:36002) 2023-10-11T09:50:32.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.273+0000 7facfc446700 1 -- 172.21.15.78:0/616166957 learned_addr learned my addr 172.21.15.78:0/616166957 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:32.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.273+0000 7facfc446700 1 -- 172.21.15.78:0/616166957 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56180d7ef000 msgr2=0x56180d895080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:32.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.273+0000 7facfc446700 1 --2- 172.21.15.78:0/616166957 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56180d7ef000 0x56180d895080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.273+0000 7facfc446700 1 -- 172.21.15.78:0/616166957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56180d7ef800 msgr2=0x56180d894580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:32.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.273+0000 7facfc446700 1 --2- 172.21.15.78:0/616166957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56180d7ef800 0x56180d894580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.273+0000 7facfc446700 1 -- 172.21.15.78:0/616166957 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56180d810b60 con 0x56180d7ef400 2023-10-11T09:50:32.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.273+0000 7facfc446700 1 --2- 172.21.15.78:0/616166957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56180d7ef400 0x56180d894b00 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x56180d950e40 tx=0x56180e66ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2104d8df8661b446 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.273+0000 7facfa442700 1 -- 172.21.15.78:0/616166957 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56180d81f080 con 0x56180d7ef400 2023-10-11T09:50:32.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.273+0000 7facfe48d380 1 -- 172.21.15.78:0/616166957 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56180d8111e0 con 0x56180d7ef400 2023-10-11T09:50:32.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.273+0000 7facfe48d380 1 -- 172.21.15.78:0/616166957 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x56180d810b60 con 0x56180d7ef400 2023-10-11T09:50:32.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.273+0000 7facfa442700 1 -- 172.21.15.78:0/616166957 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56180e5d4300 con 0x56180d7ef400 2023-10-11T09:50:32.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.273+0000 7facfe48d380 4 waiting for MDS map... 2023-10-11T09:50:32.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.273+0000 7facfa442700 1 -- 172.21.15.78:0/616166957 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56180d81f600 con 0x56180d7ef400 2023-10-11T09:50:32.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.277+0000 7facfa442700 1 -- 172.21.15.78:0/616166957 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(110..110 src has 1..110) v4 ==== 6638+0+0 (secure 0 0 0) 0x56180d828c40 con 0x56180d7ef400 2023-10-11T09:50:32.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.277+0000 7facfa442700 1 -- 172.21.15.78:0/616166957 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 85) v1 ==== 3118+0+0 (secure 0 0 0) 0x56180d830f00 con 0x56180d7ef400 2023-10-11T09:50:32.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.277+0000 7facfe48d380 4 Got MDS map 85 2023-10-11T09:50:32.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.277+0000 7facfe48d380 10 main: JournalTool::main 2023-10-11T09:50:32.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.277+0000 7facfe48d380 4 main: JournalTool: connecting to RADOS... 2023-10-11T09:50:32.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.277+0000 7facfe48d380 1 Processor -- start 2023-10-11T09:50:32.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.277+0000 7facfe48d380 1 -- start start 2023-10-11T09:50:32.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.277+0000 7facfe48d380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56180e672400 0x56180d895600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.277+0000 7facfe48d380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56180e672800 0x56180d895b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.277+0000 7facfe48d380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56180e672c00 0x56180d896100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.277+0000 7facfe48d380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56180d825e00 con 0x56180e672800 2023-10-11T09:50:32.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.277+0000 7facfe48d380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56180d86e780 con 0x56180e672c00 2023-10-11T09:50:32.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.277+0000 7facfe48d380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x56180d86ea00 con 0x56180e672400 2023-10-11T09:50:32.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.277+0000 7facfcc47700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56180e672800 0x56180d895b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.277+0000 7facfcc47700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56180e672800 0x56180d895b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36014/0 (socket says 172.21.15.78:36014) 2023-10-11T09:50:32.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.277+0000 7facfcc47700 1 -- 172.21.15.78:0/1103516605 learned_addr learned my addr 172.21.15.78:0/1103516605 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:32.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.281+0000 7facfc446700 1 --2- 172.21.15.78:0/1103516605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56180e672c00 0x56180d896100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.281+0000 7facfd448700 1 --2- 172.21.15.78:0/1103516605 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56180e672400 0x56180d895600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.281+0000 7facfcc47700 1 -- 172.21.15.78:0/1103516605 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56180e672400 msgr2=0x56180d895600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:32.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.281+0000 7facfcc47700 1 --2- 172.21.15.78:0/1103516605 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56180e672400 0x56180d895600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.281+0000 7facfcc47700 1 -- 172.21.15.78:0/1103516605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56180e672c00 msgr2=0x56180d896100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:32.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.281+0000 7facfcc47700 1 --2- 172.21.15.78:0/1103516605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56180e672c00 0x56180d896100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.281+0000 7facfcc47700 1 -- 172.21.15.78:0/1103516605 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56180d810340 con 0x56180e672800 2023-10-11T09:50:32.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.281+0000 7facfd448700 1 --2- 172.21.15.78:0/1103516605 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56180e672400 0x56180d895600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:50:32.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.281+0000 7facfcc47700 1 --2- 172.21.15.78:0/1103516605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56180e672800 0x56180d895b80 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x56180db340c0 tx=0x56180dae4de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2ffe24b4ec72330 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.281+0000 7facf843e700 1 -- 172.21.15.78:0/1103516605 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56180d81f080 con 0x56180e672800 2023-10-11T09:50:32.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.281+0000 7facf843e700 1 -- 172.21.15.78:0/1103516605 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56180e5d4a80 con 0x56180e672800 2023-10-11T09:50:32.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.281+0000 7facf843e700 1 -- 172.21.15.78:0/1103516605 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56180d81f600 con 0x56180e672800 2023-10-11T09:50:32.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.281+0000 7facfe48d380 1 -- 172.21.15.78:0/1103516605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56180e672800 msgr2=0x56180d895b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.281+0000 7facfe48d380 1 --2- 172.21.15.78:0/1103516605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56180e672800 0x56180d895b80 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x56180db340c0 tx=0x56180dae4de0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.281+0000 7facfe48d380 1 -- 172.21.15.78:0/1103516605 shutdown_connections 2023-10-11T09:50:32.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.281+0000 7facfe48d380 1 --2- 172.21.15.78:0/1103516605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56180e672c00 0x56180d896100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.281+0000 7facfe48d380 1 --2- 172.21.15.78:0/1103516605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56180e672800 0x56180d895b80 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.281+0000 7facfe48d380 1 --2- 172.21.15.78:0/1103516605 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56180e672400 0x56180d895600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.281+0000 7facfe48d380 1 -- 172.21.15.78:0/1103516605 >> 172.21.15.78:0/1103516605 conn(0x56180d7eec00 msgr2=0x56180d8db800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:32.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.281+0000 7facfe48d380 1 -- 172.21.15.78:0/1103516605 shutdown_connections 2023-10-11T09:50:32.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.281+0000 7facfe48d380 1 -- 172.21.15.78:0/1103516605 wait complete. 2023-10-11T09:50:32.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.281+0000 7facfe48d380 1 Processor -- start 2023-10-11T09:50:32.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.281+0000 7facfe48d380 1 -- start start 2023-10-11T09:50:32.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.285+0000 7facfe48d380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56180e672400 0x56180d897180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.285+0000 7facfe48d380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56180e672800 0x56180d896c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.285+0000 7facfe48d380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56180e672c00 0x56180d896680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.285+0000 7facfe48d380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56180d86e500 con 0x56180e672400 2023-10-11T09:50:32.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.285+0000 7facfe48d380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56180d86e3c0 con 0x56180e672c00 2023-10-11T09:50:32.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.285+0000 7facfe48d380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x56180d86e280 con 0x56180e672800 2023-10-11T09:50:32.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.285+0000 7facfd448700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56180e672400 0x56180d897180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.285+0000 7facfd448700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56180e672400 0x56180d897180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36030/0 (socket says 172.21.15.78:36030) 2023-10-11T09:50:32.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.285+0000 7facfd448700 1 -- 172.21.15.78:0/2914959022 learned_addr learned my addr 172.21.15.78:0/2914959022 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:32.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.285+0000 7facfc446700 1 --2- 172.21.15.78:0/2914959022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56180e672c00 0x56180d896680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.285+0000 7facfcc47700 1 --2- 172.21.15.78:0/2914959022 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56180e672800 0x56180d896c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.285+0000 7facfd448700 1 -- 172.21.15.78:0/2914959022 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56180e672800 msgr2=0x56180d896c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.285+0000 7facfd448700 1 --2- 172.21.15.78:0/2914959022 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56180e672800 0x56180d896c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.285+0000 7facfd448700 1 -- 172.21.15.78:0/2914959022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56180e672c00 msgr2=0x56180d896680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.285+0000 7facfd448700 1 --2- 172.21.15.78:0/2914959022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56180e672c00 0x56180d896680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.285+0000 7facfd448700 1 -- 172.21.15.78:0/2914959022 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56180d810000 con 0x56180e672400 2023-10-11T09:50:32.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.285+0000 7facfd448700 1 --2- 172.21.15.78:0/2914959022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56180e672400 0x56180d897180 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x56180db353b0 tx=0x56180dae4f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.285+0000 7facf6c3b700 1 -- 172.21.15.78:0/2914959022 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56180d81f080 con 0x56180e672400 2023-10-11T09:50:32.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.285+0000 7facf6c3b700 1 -- 172.21.15.78:0/2914959022 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56180d83a180 con 0x56180e672400 2023-10-11T09:50:32.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.285+0000 7facfe48d380 1 -- 172.21.15.78:0/2914959022 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56180d810ea0 con 0x56180e672400 2023-10-11T09:50:32.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.285+0000 7facf6c3b700 1 -- 172.21.15.78:0/2914959022 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56180d81f600 con 0x56180e672400 2023-10-11T09:50:32.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.285+0000 7facfe48d380 1 -- 172.21.15.78:0/2914959022 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56180d811520 con 0x56180e672400 2023-10-11T09:50:32.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.285+0000 7facfe48d380 4 main: JournalTool: resolving pool 14 2023-10-11T09:50:32.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.289+0000 7facf6c3b700 1 -- 172.21.15.78:0/2914959022 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x56180d86c840 con 0x56180e672400 2023-10-11T09:50:32.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.289+0000 7facf6c3b700 1 --2- 172.21.15.78:0/2914959022 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x56180dc30000 0x56180d897700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.289+0000 7facf6c3b700 1 -- 172.21.15.78:0/2914959022 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6638+0+0 (secure 0 0 0) 0x56180d828a80 con 0x56180e672400 2023-10-11T09:50:32.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.289+0000 7facfe48d380 4 main: JournalTool: creating IoCtx.. 2023-10-11T09:50:32.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.289+0000 7facfe48d380 4 main: Executing for rank 0 2023-10-11T09:50:32.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.289+0000 7facfcc47700 1 --2- 172.21.15.78:0/2914959022 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x56180dc30000 0x56180d897700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.289+0000 7facfcc47700 1 --2- 172.21.15.78:0/2914959022 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x56180dc30000 0x56180d897700 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x56180db340c0 tx=0x56180dae4c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.293+0000 7facfe48d380 1 -- start start 2023-10-11T09:50:32.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.293+0000 7facfe48d380 1 Processor -- start 2023-10-11T09:50:32.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.293+0000 7facfe48d380 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56180e673400 0x56180d896100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.293+0000 7facfe48d380 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56180e673800 0x56180d895b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.293+0000 7facfe48d380 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56180dc30400 0x56180d895600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.293+0000 7facfe48d380 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56180d86e3c0 con 0x56180dc30400 2023-10-11T09:50:32.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.293+0000 7facfe48d380 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56180d86ea00 con 0x56180e673800 2023-10-11T09:50:32.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.293+0000 7facfe48d380 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x56180d86e780 con 0x56180e673400 2023-10-11T09:50:32.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.293+0000 7facfc446700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56180dc30400 0x56180d895600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.293+0000 7facfc446700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56180dc30400 0x56180d895600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36044/0 (socket says 172.21.15.78:36044) 2023-10-11T09:50:32.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.293+0000 7facfc446700 1 -- 172.21.15.78:0/1057460259 learned_addr learned my addr 172.21.15.78:0/1057460259 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:32.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.293+0000 7facfcc47700 1 --2- 172.21.15.78:0/1057460259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56180e673800 0x56180d895b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.293+0000 7facfd448700 1 --2- 172.21.15.78:0/1057460259 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56180e673400 0x56180d896100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.293+0000 7facfc446700 1 -- 172.21.15.78:0/1057460259 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56180e673400 msgr2=0x56180d896100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:32.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.293+0000 7facfc446700 1 --2- 172.21.15.78:0/1057460259 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56180e673400 0x56180d896100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.293+0000 7facfc446700 1 -- 172.21.15.78:0/1057460259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56180e673800 msgr2=0x56180d895b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:32.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.293+0000 7facfc446700 1 --2- 172.21.15.78:0/1057460259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56180e673800 0x56180d895b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.293+0000 7facfc446700 1 -- 172.21.15.78:0/1057460259 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56180db7f040 con 0x56180dc30400 2023-10-11T09:50:32.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.293+0000 7facfcc47700 1 --2- 172.21.15.78:0/1057460259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56180e673800 0x56180d895b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:50:32.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.293+0000 7facfd448700 1 --2- 172.21.15.78:0/1057460259 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56180e673400 0x56180d896100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:50:32.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.297+0000 7facfc446700 1 --2- 172.21.15.78:0/1057460259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56180dc30400 0x56180d895600 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x56180dbfd200 tx=0x56180e66ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98552c9c1e7520e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.297+0000 7facf3c35700 1 -- 172.21.15.78:0/1057460259 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56180d81f1e0 con 0x56180dc30400 2023-10-11T09:50:32.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.297+0000 7facf3c35700 1 -- 172.21.15.78:0/1057460259 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56180e5d4c00 con 0x56180dc30400 2023-10-11T09:50:32.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.297+0000 7facf3c35700 1 -- 172.21.15.78:0/1057460259 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56180d81ef20 con 0x56180dc30400 2023-10-11T09:50:32.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.297+0000 7facfe48d380 1 -- 172.21.15.78:0/1057460259 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56180db7eea0 con 0x56180dc30400 2023-10-11T09:50:32.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.297+0000 7facfe48d380 1 -- 172.21.15.78:0/1057460259 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x56180db7ed00 con 0x56180dc30400 2023-10-11T09:50:32.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.297+0000 7facfe48d380 4 waiting for MDS map... 2023-10-11T09:50:32.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.297+0000 7facf3c35700 1 -- 172.21.15.78:0/1057460259 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(110..110 src has 1..110) v4 ==== 6638+0+0 (secure 0 0 0) 0x56180d8288c0 con 0x56180dc30400 2023-10-11T09:50:32.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.297+0000 7facf3c35700 1 -- 172.21.15.78:0/1057460259 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 85) v1 ==== 3118+0+0 (secure 0 0 0) 0x56180dc85500 con 0x56180dc30400 2023-10-11T09:50:32.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.297+0000 7facfe48d380 4 Got MDS map 85 2023-10-11T09:50:32.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.297+0000 7facfe48d380 1 --2- 172.21.15.78:0/1057460259 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x56180dc30800 0x56180dcbc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.297+0000 7facfe48d380 1 -- 172.21.15.78:0/1057460259 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:1 14.f 14:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x56180dc30c00 con 0x56180dc30800 2023-10-11T09:50:32.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.297+0000 7facfd448700 1 --2- 172.21.15.78:0/1057460259 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x56180dc30800 0x56180dcbc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.297+0000 7facfd448700 1 --2- 172.21.15.78:0/1057460259 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x56180dc30800 0x56180dcbc000 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=f19d8254f13cf58b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.301+0000 7facfd448700 1 -- 172.21.15.78:0/1057460259 <== osd.7 v2:172.21.15.133:6824/35901 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v110'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x56180e66d200 con 0x56180dc30800 2023-10-11T09:50:32.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.301+0000 7facfe48d380 1 --2- 172.21.15.78:0/1057460259 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x56180dc31000 0x56180dcbc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.301+0000 7facfe48d380 1 -- 172.21.15.78:0/1057460259 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:2 14.14 14:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x56180dc31400 con 0x56180dc31000 2023-10-11T09:50:32.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.301+0000 7facfcc47700 1 --2- 172.21.15.78:0/1057460259 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x56180dc31000 0x56180dcbc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.301+0000 7facfcc47700 1 --2- 172.21.15.78:0/1057460259 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x56180dc31000 0x56180dcbc580 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=f73359453b8a2e4b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.305+0000 7facfcc47700 1 -- 172.21.15.78:0/1057460259 <== osd.4 v2:172.21.15.133:6808/35898 1 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v110'49 uv49 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x56180e66d440 con 0x56180dc31000 2023-10-11T09:50:32.318 INFO:teuthology.orchestra.run.smithi078.stdout:writing EResetJournal entry 2023-10-11T09:50:32.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.305+0000 7facfe48d380 1 -- 172.21.15.78:0/1057460259 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:3 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x56180dc31800 con 0x56180dc31000 2023-10-11T09:50:32.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.305+0000 7facfe48d380 1 --2- 172.21.15.78:0/1057460259 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x56180dc31c00 0x56180dcbcb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.305+0000 7facfe48d380 1 -- 172.21.15.78:0/1057460259 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:4 14.1b 14:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x56180dc9a800 con 0x56180dc31c00 2023-10-11T09:50:32.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.305+0000 7facfe48d380 1 --2- 172.21.15.78:0/1057460259 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x56180dc9ac00 0x56180dcbd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.305+0000 7facfe48d380 1 -- 172.21.15.78:0/1057460259 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:5 14.b 14:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x56180dc9b000 con 0x56180dc9ac00 2023-10-11T09:50:32.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.305+0000 7facfe48d380 1 -- 172.21.15.78:0/1057460259 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:6 14.1f 14:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x56180dc9b400 con 0x56180dc9ac00 2023-10-11T09:50:32.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.305+0000 7facfe48d380 1 -- 172.21.15.78:0/1057460259 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:7 14.1a 14:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x56180dc9b800 con 0x56180dc9ac00 2023-10-11T09:50:32.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.305+0000 7facfe48d380 1 -- 172.21.15.78:0/1057460259 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:8 14.1e 14:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x56180dc9bc00 con 0x56180dc30800 2023-10-11T09:50:32.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.305+0000 7facfd448700 1 --2- 172.21.15.78:0/1057460259 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x56180dc31c00 0x56180dcbcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.305+0000 7facfcc47700 1 --2- 172.21.15.78:0/1057460259 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x56180dc9ac00 0x56180dcbd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.309+0000 7facfcc47700 1 --2- 172.21.15.78:0/1057460259 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x56180dc9ac00 0x56180dcbd080 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=79eb50c2a85ae6ac server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.309+0000 7facfd448700 1 --2- 172.21.15.78:0/1057460259 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x56180dc31c00 0x56180dcbcb00 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=b75aa3ebe217ef7c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.309+0000 7facfcc47700 1 -- 172.21.15.78:0/1057460259 <== osd.4 v2:172.21.15.133:6808/35898 2 ==== osd_op_reply(3 200.00000001 [delete] v110'50 uv49 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x56180e66d440 con 0x56180dc31000 2023-10-11T09:50:32.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.309+0000 7facfd448700 1 -- 172.21.15.78:0/1057460259 <== osd.7 v2:172.21.15.133:6824/35901 2 ==== osd_op_reply(8 200.00000006 [delete] v110'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x56180e66d200 con 0x56180dc30800 2023-10-11T09:50:32.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.309+0000 7facfcc47700 1 -- 172.21.15.78:0/1057460259 <== osd.0 v2:172.21.15.78:6800/35735 1 ==== osd_op_reply(5 200.00000003 [delete] v110'6 uv5 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x56180e66d440 con 0x56180dc9ac00 2023-10-11T09:50:32.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.309+0000 7facfcc47700 1 -- 172.21.15.78:0/1057460259 <== osd.0 v2:172.21.15.78:6800/35735 2 ==== osd_op_reply(6 200.00000004 [delete] v110'9 uv8 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x56180e66d440 con 0x56180dc9ac00 2023-10-11T09:50:32.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.313+0000 7facfd448700 1 -- 172.21.15.78:0/1057460259 <== osd.1 v2:172.21.15.78:6809/35734 1 ==== osd_op_reply(4 200.00000002 [delete] v110'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x56180e66d200 con 0x56180dc31c00 2023-10-11T09:50:32.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.313+0000 7facf2432700 1 -- 172.21.15.78:0/1057460259 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:9 14.14 14:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x56180dc30c00 con 0x56180dc31000 2023-10-11T09:50:32.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.313+0000 7facfcc47700 1 -- 172.21.15.78:0/1057460259 <== osd.0 v2:172.21.15.78:6800/35735 3 ==== osd_op_reply(7 200.00000005 [delete] v110'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x56180e66d440 con 0x56180dc9ac00 2023-10-11T09:50:32.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.313+0000 7facfcc47700 1 -- 172.21.15.78:0/1057460259 <== osd.4 v2:172.21.15.133:6808/35898 3 ==== osd_op_reply(9 200.00000001 [write 0~48] v110'51 uv51 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x56180e66d440 con 0x56180dc31000 2023-10-11T09:50:32.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.313+0000 7facfe48d380 4 Successfully wrote new journal pointer and header for rank 5:0 2023-10-11T09:50:32.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.313+0000 7facfe48d380 1 -- 172.21.15.78:0/1057460259 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x56180dc9ac00 msgr2=0x56180dcbd080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.313+0000 7facfe48d380 1 --2- 172.21.15.78:0/1057460259 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x56180dc9ac00 0x56180dcbd080 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.313+0000 7facfe48d380 1 -- 172.21.15.78:0/1057460259 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x56180dc31c00 msgr2=0x56180dcbcb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.313+0000 7facfe48d380 1 --2- 172.21.15.78:0/1057460259 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x56180dc31c00 0x56180dcbcb00 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.313+0000 7facfe48d380 1 -- 172.21.15.78:0/1057460259 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x56180dc31000 msgr2=0x56180dcbc580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.313+0000 7facfe48d380 1 --2- 172.21.15.78:0/1057460259 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x56180dc31000 0x56180dcbc580 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.313+0000 7facfe48d380 1 -- 172.21.15.78:0/1057460259 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x56180dc30800 msgr2=0x56180dcbc000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.313+0000 7facfe48d380 1 --2- 172.21.15.78:0/1057460259 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x56180dc30800 0x56180dcbc000 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.313+0000 7facfc446700 1 -- 172.21.15.78:0/1057460259 reap_dead start 2023-10-11T09:50:32.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.313+0000 7facfe48d380 1 -- 172.21.15.78:0/1057460259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56180dc30400 msgr2=0x56180d895600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.313+0000 7facfe48d380 1 --2- 172.21.15.78:0/1057460259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56180dc30400 0x56180d895600 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x56180dbfd200 tx=0x56180e66ed20 comp rx=0 tx=0).stop 2023-10-11T09:50:32.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.313+0000 7facfe48d380 1 -- 172.21.15.78:0/1057460259 shutdown_connections 2023-10-11T09:50:32.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.313+0000 7facfe48d380 1 --2- 172.21.15.78:0/1057460259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56180dc30400 0x56180d895600 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.313+0000 7facfe48d380 1 -- 172.21.15.78:0/1057460259 >> 172.21.15.78:0/1057460259 conn(0x56180e673000 msgr2=0x56180d8dc000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:32.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.313+0000 7facfe48d380 1 -- 172.21.15.78:0/1057460259 shutdown_connections 2023-10-11T09:50:32.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.313+0000 7facfe48d380 1 -- 172.21.15.78:0/1057460259 wait complete. 2023-10-11T09:50:32.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.313+0000 7facfe48d380 1 -- 172.21.15.78:0/2914959022 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x56180dc30000 msgr2=0x56180d897700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.313+0000 7facfe48d380 1 --2- 172.21.15.78:0/2914959022 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x56180dc30000 0x56180d897700 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x56180db340c0 tx=0x56180dae4c00 comp rx=0 tx=0).stop 2023-10-11T09:50:32.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.313+0000 7facfe48d380 1 -- 172.21.15.78:0/2914959022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56180e672400 msgr2=0x56180d897180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.313+0000 7facfe48d380 1 --2- 172.21.15.78:0/2914959022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56180e672400 0x56180d897180 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x56180db353b0 tx=0x56180dae4f00 comp rx=0 tx=0).stop 2023-10-11T09:50:32.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.317+0000 7facfe48d380 1 -- 172.21.15.78:0/2914959022 shutdown_connections 2023-10-11T09:50:32.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.317+0000 7facfe48d380 1 --2- 172.21.15.78:0/2914959022 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x56180dc30000 0x56180d897700 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.317+0000 7facfe48d380 1 --2- 172.21.15.78:0/2914959022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56180e672c00 0x56180d896680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.317+0000 7facfe48d380 1 --2- 172.21.15.78:0/2914959022 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56180e672800 0x56180d896c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.317+0000 7facfe48d380 1 --2- 172.21.15.78:0/2914959022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56180e672400 0x56180d897180 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.317+0000 7facfe48d380 1 -- 172.21.15.78:0/2914959022 >> 172.21.15.78:0/2914959022 conn(0x56180d7eec00 msgr2=0x56180d8db800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:32.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.317+0000 7facfe48d380 1 -- 172.21.15.78:0/2914959022 shutdown_connections 2023-10-11T09:50:32.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.317+0000 7facfe48d380 1 -- 172.21.15.78:0/2914959022 wait complete. 2023-10-11T09:50:32.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.317+0000 7facfe48d380 1 -- 172.21.15.78:0/616166957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56180d7ef400 msgr2=0x56180d894b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.317+0000 7facfe48d380 1 --2- 172.21.15.78:0/616166957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56180d7ef400 0x56180d894b00 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x56180d950e40 tx=0x56180e66ec60 comp rx=0 tx=0).stop 2023-10-11T09:50:32.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.317+0000 7facfe48d380 1 -- 172.21.15.78:0/616166957 shutdown_connections 2023-10-11T09:50:32.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.317+0000 7facfe48d380 1 --2- 172.21.15.78:0/616166957 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56180d7ef000 0x56180d895080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.317+0000 7facfe48d380 1 --2- 172.21.15.78:0/616166957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56180d7ef400 0x56180d894b00 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.317+0000 7facfe48d380 1 --2- 172.21.15.78:0/616166957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56180d7ef800 0x56180d894580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.317+0000 7facfe48d380 1 -- 172.21.15.78:0/616166957 >> 172.21.15.78:0/616166957 conn(0x56180d7efc00 msgr2=0x56180d8db000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:32.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.317+0000 7facfe48d380 1 -- 172.21.15.78:0/616166957 shutdown_connections 2023-10-11T09:50:32.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.317+0000 7facfe48d380 1 -- 172.21.15.78:0/616166957 wait complete. 2023-10-11T09:50:32.336 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.118208, result: writing EResetJournal entry 2023-10-11T09:50:32.336 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 init 2023-10-11T09:50:32.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.393+0000 7fc405592840 1 -- start start 2023-10-11T09:50:32.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.393+0000 7fc405592840 1 Processor -- start 2023-10-11T09:50:32.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.393+0000 7fc405592840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f6fa5800 0x55b2f704a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.393+0000 7fc405592840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b2f6fa5400 0x55b2f704ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.393+0000 7fc405592840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b2f6fa5000 0x55b2f704b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.393+0000 7fc405592840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f7024a00 con 0x55b2f6fa5800 2023-10-11T09:50:32.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.393+0000 7fc405592840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f7024780 con 0x55b2f6fa5000 2023-10-11T09:50:32.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.393+0000 7fc405592840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b2f7024640 con 0x55b2f6fa5400 2023-10-11T09:50:32.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.393+0000 7fc403d4e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f6fa5800 0x55b2f704a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.393+0000 7fc403d4e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f6fa5800 0x55b2f704a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36062/0 (socket says 172.21.15.78:36062) 2023-10-11T09:50:32.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.393+0000 7fc403d4e700 1 -- 172.21.15.78:0/2942445135 learned_addr learned my addr 172.21.15.78:0/2942445135 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:32.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.393+0000 7fc40454f700 1 --2- 172.21.15.78:0/2942445135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b2f6fa5000 0x55b2f704b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.393+0000 7fc40354d700 1 --2- 172.21.15.78:0/2942445135 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b2f6fa5400 0x55b2f704ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.393+0000 7fc403d4e700 1 -- 172.21.15.78:0/2942445135 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b2f6fa5400 msgr2=0x55b2f704ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:32.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.393+0000 7fc403d4e700 1 --2- 172.21.15.78:0/2942445135 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b2f6fa5400 0x55b2f704ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.393+0000 7fc403d4e700 1 -- 172.21.15.78:0/2942445135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b2f6fa5000 msgr2=0x55b2f704b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:32.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.393+0000 7fc403d4e700 1 --2- 172.21.15.78:0/2942445135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b2f6fa5000 0x55b2f704b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.393+0000 7fc403d4e700 1 -- 172.21.15.78:0/2942445135 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2f6fc6b60 con 0x55b2f6fa5800 2023-10-11T09:50:32.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.393+0000 7fc40354d700 1 --2- 172.21.15.78:0/2942445135 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b2f6fa5400 0x55b2f704ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:50:32.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.393+0000 7fc403d4e700 1 --2- 172.21.15.78:0/2942445135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f6fa5800 0x55b2f704a580 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x55b2f70733e0 tx=0x55b2f7d9cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=372623ab7547cdf1 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.397+0000 7fc401549700 1 -- 172.21.15.78:0/2942445135 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b2f6fd5600 con 0x55b2f6fa5800 2023-10-11T09:50:32.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.397+0000 7fc401549700 1 -- 172.21.15.78:0/2942445135 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b2f6ff0480 con 0x55b2f6fa5800 2023-10-11T09:50:32.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.397+0000 7fc401549700 1 -- 172.21.15.78:0/2942445135 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b2f6fd51e0 con 0x55b2f6fa5800 2023-10-11T09:50:32.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.397+0000 7fc405592840 1 -- 172.21.15.78:0/2942445135 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b2f6fc6340 con 0x55b2f6fa5800 2023-10-11T09:50:32.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.397+0000 7fc405592840 1 -- 172.21.15.78:0/2942445135 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55b2f6fc6000 con 0x55b2f6fa5800 2023-10-11T09:50:32.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.397+0000 7fc405592840 4 waiting for MDS map... 2023-10-11T09:50:32.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.397+0000 7fc401549700 1 -- 172.21.15.78:0/2942445135 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(110..110 src has 1..110) v4 ==== 6638+0+0 (secure 0 0 0) 0x55b2f6fdec40 con 0x55b2f6fa5800 2023-10-11T09:50:32.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.397+0000 7fc401549700 1 -- 172.21.15.78:0/2942445135 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 85) v1 ==== 3118+0+0 (secure 0 0 0) 0x55b2f7295200 con 0x55b2f6fa5800 2023-10-11T09:50:32.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.397+0000 7fc405592840 4 Got MDS map 85 2023-10-11T09:50:32.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.397+0000 7fc405592840 4 datascan.main: Using metadata pool output 2023-10-11T09:50:32.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.397+0000 7fc405592840 4 datascan.main: connecting to RADOS... 2023-10-11T09:50:32.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.397+0000 7fc405592840 1 Processor -- start 2023-10-11T09:50:32.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.397+0000 7fc405592840 1 -- start start 2023-10-11T09:50:32.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.401+0000 7fc405592840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b2f7e2e400 0x55b2f704b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.401+0000 7fc405592840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b2f7e2e800 0x55b2f704bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.401+0000 7fc405592840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f7e2ec00 0x55b2f704c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.401+0000 7fc405592840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f6fdbe00 con 0x55b2f7e2ec00 2023-10-11T09:50:32.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.401+0000 7fc405592840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f7024780 con 0x55b2f7e2e400 2023-10-11T09:50:32.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.401+0000 7fc405592840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b2f7024500 con 0x55b2f7e2e800 2023-10-11T09:50:32.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.401+0000 7fc40354d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f7e2ec00 0x55b2f704c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.401+0000 7fc40354d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f7e2ec00 0x55b2f704c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36064/0 (socket says 172.21.15.78:36064) 2023-10-11T09:50:32.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.401+0000 7fc40354d700 1 -- 172.21.15.78:0/13028951 learned_addr learned my addr 172.21.15.78:0/13028951 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:32.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.401+0000 7fc403d4e700 1 --2- 172.21.15.78:0/13028951 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b2f7e2e800 0x55b2f704bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.401+0000 7fc40454f700 1 --2- 172.21.15.78:0/13028951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b2f7e2e400 0x55b2f704b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.401+0000 7fc40354d700 1 -- 172.21.15.78:0/13028951 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b2f7e2e800 msgr2=0x55b2f704bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:32.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.401+0000 7fc40354d700 1 --2- 172.21.15.78:0/13028951 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b2f7e2e800 0x55b2f704bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.401+0000 7fc40354d700 1 -- 172.21.15.78:0/13028951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b2f7e2e400 msgr2=0x55b2f704b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:32.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.401+0000 7fc40354d700 1 --2- 172.21.15.78:0/13028951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b2f7e2e400 0x55b2f704b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.401+0000 7fc40354d700 1 -- 172.21.15.78:0/13028951 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2f6fc6ea0 con 0x55b2f7e2ec00 2023-10-11T09:50:32.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.401+0000 7fc40454f700 1 --2- 172.21.15.78:0/13028951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b2f7e2e400 0x55b2f704b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:50:32.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.401+0000 7fc40354d700 1 --2- 172.21.15.78:0/13028951 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f7e2ec00 0x55b2f704c100 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x55b2f729e8d0 tx=0x55b2f72a2c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d46dcf6e18e6aa2f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.401+0000 7fc3ff545700 1 -- 172.21.15.78:0/13028951 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b2f6fd5600 con 0x55b2f7e2ec00 2023-10-11T09:50:32.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.401+0000 7fc3ff545700 1 -- 172.21.15.78:0/13028951 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b2f6ff0900 con 0x55b2f7e2ec00 2023-10-11T09:50:32.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.401+0000 7fc3ff545700 1 -- 172.21.15.78:0/13028951 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b2f6fd51e0 con 0x55b2f7e2ec00 2023-10-11T09:50:32.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.401+0000 7fc405592840 1 -- 172.21.15.78:0/13028951 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f7e2ec00 msgr2=0x55b2f704c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.401+0000 7fc405592840 1 --2- 172.21.15.78:0/13028951 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f7e2ec00 0x55b2f704c100 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x55b2f729e8d0 tx=0x55b2f72a2c00 comp rx=0 tx=0).stop 2023-10-11T09:50:32.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.401+0000 7fc405592840 1 -- 172.21.15.78:0/13028951 shutdown_connections 2023-10-11T09:50:32.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.401+0000 7fc405592840 1 --2- 172.21.15.78:0/13028951 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f7e2ec00 0x55b2f704c100 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.401+0000 7fc405592840 1 --2- 172.21.15.78:0/13028951 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b2f7e2e800 0x55b2f704bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.401+0000 7fc405592840 1 --2- 172.21.15.78:0/13028951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b2f7e2e400 0x55b2f704b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.401+0000 7fc405592840 1 -- 172.21.15.78:0/13028951 >> 172.21.15.78:0/13028951 conn(0x55b2f6fa4c00 msgr2=0x55b2f7095800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:32.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.401+0000 7fc405592840 1 -- 172.21.15.78:0/13028951 shutdown_connections 2023-10-11T09:50:32.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.401+0000 7fc405592840 1 -- 172.21.15.78:0/13028951 wait complete. 2023-10-11T09:50:32.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.405+0000 7fc405592840 1 Processor -- start 2023-10-11T09:50:32.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.405+0000 7fc405592840 1 -- start start 2023-10-11T09:50:32.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.405+0000 7fc405592840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b2f7e2e400 0x55b2f704d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.405+0000 7fc405592840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b2f7e2e800 0x55b2f704cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.405+0000 7fc405592840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f7e2ec00 0x55b2f704c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.405+0000 7fc405592840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f70243c0 con 0x55b2f7e2ec00 2023-10-11T09:50:32.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.405+0000 7fc405592840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f7024280 con 0x55b2f7e2e400 2023-10-11T09:50:32.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.405+0000 7fc405592840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b2f7024140 con 0x55b2f7e2e800 2023-10-11T09:50:32.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.405+0000 7fc40354d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f7e2ec00 0x55b2f704c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.405+0000 7fc40354d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f7e2ec00 0x55b2f704c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36066/0 (socket says 172.21.15.78:36066) 2023-10-11T09:50:32.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.405+0000 7fc40354d700 1 -- 172.21.15.78:0/4118380585 learned_addr learned my addr 172.21.15.78:0/4118380585 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:32.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.405+0000 7fc403d4e700 1 --2- 172.21.15.78:0/4118380585 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b2f7e2e800 0x55b2f704cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.405+0000 7fc40454f700 1 --2- 172.21.15.78:0/4118380585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b2f7e2e400 0x55b2f704d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.409+0000 7fc40354d700 1 -- 172.21.15.78:0/4118380585 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b2f7e2e800 msgr2=0x55b2f704cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.409+0000 7fc40354d700 1 --2- 172.21.15.78:0/4118380585 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b2f7e2e800 0x55b2f704cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.409+0000 7fc40354d700 1 -- 172.21.15.78:0/4118380585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b2f7e2e400 msgr2=0x55b2f704d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.409+0000 7fc40354d700 1 --2- 172.21.15.78:0/4118380585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b2f7e2e400 0x55b2f704d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.409+0000 7fc40354d700 1 -- 172.21.15.78:0/4118380585 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2f6fc6b60 con 0x55b2f7e2ec00 2023-10-11T09:50:32.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.409+0000 7fc403d4e700 1 --2- 172.21.15.78:0/4118380585 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b2f7e2e800 0x55b2f704cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:50:32.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.409+0000 7fc40354d700 1 --2- 172.21.15.78:0/4118380585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f7e2ec00 0x55b2f704c680 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x55b2f729f590 tx=0x55b2f7d9cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.409+0000 7fc3fdd42700 1 -- 172.21.15.78:0/4118380585 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b2f6fd4dc0 con 0x55b2f7e2ec00 2023-10-11T09:50:32.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.409+0000 7fc3fdd42700 1 -- 172.21.15.78:0/4118380585 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b2f6ff0480 con 0x55b2f7e2ec00 2023-10-11T09:50:32.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.409+0000 7fc3fdd42700 1 -- 172.21.15.78:0/4118380585 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b2f6fd5600 con 0x55b2f7e2ec00 2023-10-11T09:50:32.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.409+0000 7fc405592840 1 -- 172.21.15.78:0/4118380585 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b2f6fc6340 con 0x55b2f7e2ec00 2023-10-11T09:50:32.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.409+0000 7fc405592840 1 -- 172.21.15.78:0/4118380585 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b2f6fc7520 con 0x55b2f7e2ec00 2023-10-11T09:50:32.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.409+0000 7fc405592840 4 datascan.init: resolving metadata pool 14 2023-10-11T09:50:32.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.413+0000 7fc3fdd42700 1 -- 172.21.15.78:0/4118380585 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x55b2f7002840 con 0x55b2f7e2ec00 2023-10-11T09:50:32.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.413+0000 7fc3fdd42700 1 --2- 172.21.15.78:0/4118380585 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55b2f730e800 0x55b2f704d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.413+0000 7fc3fdd42700 1 -- 172.21.15.78:0/4118380585 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6638+0+0 (secure 0 0 0) 0x55b2f6fdea80 con 0x55b2f7e2ec00 2023-10-11T09:50:32.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.413+0000 7fc405592840 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-10-11T09:50:32.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.413+0000 7fc405592840 1 --2- 172.21.15.78:0/4118380585 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55b2f7e2f000 0x55b2f704c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.413+0000 7fc40454f700 1 --2- 172.21.15.78:0/4118380585 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55b2f730e800 0x55b2f704d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.413+0000 7fc405592840 1 -- 172.21.15.78:0/4118380585 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:1 14.14 14:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b2f7e2f400 con 0x55b2f7e2f000 2023-10-11T09:50:32.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.413+0000 7fc403d4e700 1 --2- 172.21.15.78:0/4118380585 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55b2f7e2f000 0x55b2f704c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.413+0000 7fc40454f700 1 --2- 172.21.15.78:0/4118380585 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55b2f730e800 0x55b2f704d700 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x55b2f6ffbc20 tx=0x55b2f72a3da0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.413+0000 7fc403d4e700 1 --2- 172.21.15.78:0/4118380585 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55b2f7e2f000 0x55b2f704c100 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.417+0000 7fc403d4e700 1 -- 172.21.15.78:0/4118380585 <== osd.4 v2:172.21.15.133:6808/35898 1 ==== osd_op_reply(1 1.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 160+0+0 (crc 0 0 0) 0x55b2f7d8efc0 con 0x55b2f7e2f000 2023-10-11T09:50:32.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.417+0000 7fc405592840 1 -- 172.21.15.78:0/4118380585 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:2 14.14 14:287034c4:::1.00000000.inode:head [writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b2f7e2f800 con 0x55b2f7e2f000 2023-10-11T09:50:32.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.417+0000 7fc403d4e700 1 -- 172.21.15.78:0/4118380585 <== osd.4 v2:172.21.15.133:6808/35898 2 ==== osd_op_reply(2 1.00000000.inode [writefull 0~582] v110'52 uv52 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x55b2f7d8efc0 con 0x55b2f7e2f000 2023-10-11T09:50:32.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.417+0000 7fc405592840 1 --2- 172.21.15.78:0/4118380585 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55b2f730ec00 0x55b2f704bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.417+0000 7fc405592840 1 -- 172.21.15.78:0/4118380585 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:3 14.1f 14:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b2f730f000 con 0x55b2f730ec00 2023-10-11T09:50:32.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.417+0000 7fc40454f700 1 --2- 172.21.15.78:0/4118380585 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55b2f730ec00 0x55b2f704bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.421+0000 7fc40454f700 1 --2- 172.21.15.78:0/4118380585 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55b2f730ec00 0x55b2f704bb80 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.421+0000 7fc40454f700 1 -- 172.21.15.78:0/4118380585 <== osd.0 v2:172.21.15.78:6800/35735 1 ==== osd_op_reply(3 100.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 162+0+0 (crc 0 0 0) 0x55b2f7d8f680 con 0x55b2f730ec00 2023-10-11T09:50:32.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.421+0000 7fc405592840 1 -- 172.21.15.78:0/4118380585 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:4 14.1f 14:fe07bba1:::100.00000000.inode:head [writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b2f730f400 con 0x55b2f730ec00 2023-10-11T09:50:32.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.421+0000 7fc40454f700 1 -- 172.21.15.78:0/4118380585 <== osd.0 v2:172.21.15.78:6800/35735 2 ==== osd_op_reply(4 100.00000000.inode [writefull 0~582] v110'10 uv10 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x55b2f7d8f680 con 0x55b2f730ec00 2023-10-11T09:50:32.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.425+0000 7fc405592840 1 --2- 172.21.15.78:0/4118380585 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55b2f730f800 0x55b2f704b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.425+0000 7fc405592840 1 -- 172.21.15.78:0/4118380585 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:5 14.13 14:cd5a64a3:::100.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b2f730fc00 con 0x55b2f730f800 2023-10-11T09:50:32.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.425+0000 7fc403d4e700 1 --2- 172.21.15.78:0/4118380585 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55b2f730f800 0x55b2f704b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.425+0000 7fc403d4e700 1 --2- 172.21.15.78:0/4118380585 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55b2f730f800 0x55b2f704b600 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.425+0000 7fc403d4e700 1 -- 172.21.15.78:0/4118380585 <== osd.2 v2:172.21.15.78:6808/35732 1 ==== osd_op_reply(5 100.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55b2f7d8efc0 con 0x55b2f730f800 2023-10-11T09:50:32.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.425+0000 7fc405592840 10 datascan.find_or_create_dirfrag: read_version = 0 2023-10-11T09:50:32.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.425+0000 7fc405592840 1 -- 172.21.15.78:0/4118380585 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:6 14.13 14:cd5a64a3:::100.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b2f730e400 con 0x55b2f730f800 2023-10-11T09:50:32.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.425+0000 7fc403d4e700 1 -- 172.21.15.78:0/4118380585 <== osd.2 v2:172.21.15.78:6808/35732 2 ==== osd_op_reply(6 100.00000000 [create,omap-set-header] v110'7 uv7 ondisk = 0) v8 ==== 198+0+0 (crc 0 0 0) 0x55b2f7d8efc0 con 0x55b2f730f800 2023-10-11T09:50:32.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.425+0000 7fc405592840 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x100 2023-10-11T09:50:32.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.425+0000 7fc405592840 1 -- 172.21.15.78:0/4118380585 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55b2f730ec00 msgr2=0x55b2f704bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.425+0000 7fc405592840 1 --2- 172.21.15.78:0/4118380585 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55b2f730ec00 0x55b2f704bb80 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.425+0000 7fc405592840 1 -- 172.21.15.78:0/4118380585 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55b2f730f800 msgr2=0x55b2f704b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.425+0000 7fc405592840 1 --2- 172.21.15.78:0/4118380585 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55b2f730f800 0x55b2f704b600 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.425+0000 7fc405592840 1 -- 172.21.15.78:0/4118380585 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55b2f7e2f000 msgr2=0x55b2f704c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.425+0000 7fc405592840 1 --2- 172.21.15.78:0/4118380585 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55b2f7e2f000 0x55b2f704c100 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.425+0000 7fc40354d700 1 -- 172.21.15.78:0/4118380585 reap_dead start 2023-10-11T09:50:32.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.425+0000 7fc405592840 1 -- 172.21.15.78:0/4118380585 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55b2f730e800 msgr2=0x55b2f704d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.425+0000 7fc405592840 1 --2- 172.21.15.78:0/4118380585 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55b2f730e800 0x55b2f704d700 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x55b2f6ffbc20 tx=0x55b2f72a3da0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.425+0000 7fc405592840 1 -- 172.21.15.78:0/4118380585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f7e2ec00 msgr2=0x55b2f704c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.425+0000 7fc405592840 1 --2- 172.21.15.78:0/4118380585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f7e2ec00 0x55b2f704c680 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x55b2f729f590 tx=0x55b2f7d9cd80 comp rx=0 tx=0).stop 2023-10-11T09:50:32.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.429+0000 7fc405592840 1 -- 172.21.15.78:0/4118380585 shutdown_connections 2023-10-11T09:50:32.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.429+0000 7fc405592840 1 --2- 172.21.15.78:0/4118380585 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55b2f730e800 0x55b2f704d700 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.429+0000 7fc405592840 1 --2- 172.21.15.78:0/4118380585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f7e2ec00 0x55b2f704c680 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.429+0000 7fc405592840 1 -- 172.21.15.78:0/4118380585 >> 172.21.15.78:0/4118380585 conn(0x55b2f6fa4c00 msgr2=0x55b2f7095800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:32.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.429+0000 7fc405592840 1 -- 172.21.15.78:0/4118380585 shutdown_connections 2023-10-11T09:50:32.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.429+0000 7fc405592840 1 -- 172.21.15.78:0/4118380585 wait complete. 2023-10-11T09:50:32.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.429+0000 7fc405592840 1 -- 172.21.15.78:0/2942445135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f6fa5800 msgr2=0x55b2f704a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.429+0000 7fc405592840 1 --2- 172.21.15.78:0/2942445135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f6fa5800 0x55b2f704a580 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x55b2f70733e0 tx=0x55b2f7d9cc60 comp rx=0 tx=0).stop 2023-10-11T09:50:32.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.429+0000 7fc405592840 1 -- 172.21.15.78:0/2942445135 shutdown_connections 2023-10-11T09:50:32.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.429+0000 7fc405592840 1 --2- 172.21.15.78:0/2942445135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b2f6fa5000 0x55b2f704b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.429+0000 7fc405592840 1 --2- 172.21.15.78:0/2942445135 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b2f6fa5400 0x55b2f704ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.429+0000 7fc405592840 1 --2- 172.21.15.78:0/2942445135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2f6fa5800 0x55b2f704a580 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.429+0000 7fc405592840 1 -- 172.21.15.78:0/2942445135 >> 172.21.15.78:0/2942445135 conn(0x55b2f6fa5c00 msgr2=0x55b2f7095000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:32.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.429+0000 7fc405592840 1 -- 172.21.15.78:0/2942445135 shutdown_connections 2023-10-11T09:50:32.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.429+0000 7fc405592840 1 -- 172.21.15.78:0/2942445135 wait complete. 2023-10-11T09:50:32.444 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'init'] in time 0:00:00.108149, result: 2023-10-11T09:50:32.445 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents 2023-10-11T09:50:32.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.497+0000 7f9ffbb09840 1 -- start start 2023-10-11T09:50:32.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.497+0000 7f9ffbb09840 1 Processor -- start 2023-10-11T09:50:32.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.497+0000 7f9ffbb09840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d3d213800 0x557d3d2b8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.497+0000 7f9ffbb09840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d3d213400 0x557d3d2b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.497+0000 7f9ffbb09840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x557d3d213000 0x557d3d2b9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.497+0000 7f9ffbb09840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d3d292a00 con 0x557d3d213800 2023-10-11T09:50:32.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.497+0000 7f9ffbb09840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d3d292780 con 0x557d3d213400 2023-10-11T09:50:32.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.497+0000 7f9ffbb09840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x557d3d292640 con 0x557d3d213000 2023-10-11T09:50:32.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.497+0000 7f9ffa2c5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d3d213800 0x557d3d2b8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.497+0000 7f9ffa2c5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d3d213800 0x557d3d2b8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36084/0 (socket says 172.21.15.78:36084) 2023-10-11T09:50:32.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.497+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/488040104 learned_addr learned my addr 172.21.15.78:0/488040104 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:32.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.497+0000 7f9ffaac6700 1 --2- 172.21.15.78:0/488040104 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x557d3d213000 0x557d3d2b9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.497+0000 7f9ff9ac4700 1 --2- 172.21.15.78:0/488040104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d3d213400 0x557d3d2b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.497+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/488040104 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x557d3d213000 msgr2=0x557d3d2b9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:32.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.497+0000 7f9ffa2c5700 1 --2- 172.21.15.78:0/488040104 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x557d3d213000 0x557d3d2b9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.497+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/488040104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d3d213400 msgr2=0x557d3d2b8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:32.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.497+0000 7f9ffa2c5700 1 --2- 172.21.15.78:0/488040104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d3d213400 0x557d3d2b8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.497+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/488040104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d3d234b60 con 0x557d3d213800 2023-10-11T09:50:32.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.497+0000 7f9ffaac6700 1 --2- 172.21.15.78:0/488040104 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x557d3d213000 0x557d3d2b9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:50:32.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.501+0000 7f9ffa2c5700 1 --2- 172.21.15.78:0/488040104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d3d213800 0x557d3d2b8580 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x557d3df7b3e0 tx=0x557d3e004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c92546abb7f38ee3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.501+0000 7f9ff7ac0700 1 -- 172.21.15.78:0/488040104 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557d3d243600 con 0x557d3d213800 2023-10-11T09:50:32.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.501+0000 7f9ff7ac0700 1 -- 172.21.15.78:0/488040104 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557d3d25e480 con 0x557d3d213800 2023-10-11T09:50:32.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.501+0000 7f9ff7ac0700 1 -- 172.21.15.78:0/488040104 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557d3d2431e0 con 0x557d3d213800 2023-10-11T09:50:32.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.501+0000 7f9ffbb09840 1 -- 172.21.15.78:0/488040104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d3d234340 con 0x557d3d213800 2023-10-11T09:50:32.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.501+0000 7f9ffbb09840 1 -- 172.21.15.78:0/488040104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x557d3d234000 con 0x557d3d213800 2023-10-11T09:50:32.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.501+0000 7f9ffbb09840 4 waiting for MDS map... 2023-10-11T09:50:32.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.501+0000 7f9ff7ac0700 1 -- 172.21.15.78:0/488040104 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(110..110 src has 1..110) v4 ==== 6638+0+0 (secure 0 0 0) 0x557d3d24cc40 con 0x557d3d213800 2023-10-11T09:50:32.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.501+0000 7f9ff7ac0700 1 -- 172.21.15.78:0/488040104 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 85) v1 ==== 3118+0+0 (secure 0 0 0) 0x557d3dff2000 con 0x557d3d213800 2023-10-11T09:50:32.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.501+0000 7f9ffbb09840 4 Got MDS map 85 2023-10-11T09:50:32.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.501+0000 7f9ffbb09840 4 datascan.main: Using metadata pool output 2023-10-11T09:50:32.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.501+0000 7f9ffbb09840 4 datascan.main: connecting to RADOS... 2023-10-11T09:50:32.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.501+0000 7f9ffbb09840 1 Processor -- start 2023-10-11T09:50:32.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.501+0000 7f9ffbb09840 1 -- start start 2023-10-11T09:50:32.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.501+0000 7f9ffbb09840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d3d508800 0x557d3d2b9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.501+0000 7f9ffbb09840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x557d3d508c00 0x557d3d2b9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.501+0000 7f9ffbb09840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d3d509000 0x557d3d2ba100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.501+0000 7f9ffbb09840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d3d249e00 con 0x557d3d508800 2023-10-11T09:50:32.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.501+0000 7f9ffbb09840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d3d292780 con 0x557d3d509000 2023-10-11T09:50:32.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.505+0000 7f9ffbb09840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x557d3d292500 con 0x557d3d508c00 2023-10-11T09:50:32.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.505+0000 7f9ffaac6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d3d508800 0x557d3d2b9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.505+0000 7f9ffaac6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d3d508800 0x557d3d2b9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36094/0 (socket says 172.21.15.78:36094) 2023-10-11T09:50:32.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.505+0000 7f9ffaac6700 1 -- 172.21.15.78:0/2634038214 learned_addr learned my addr 172.21.15.78:0/2634038214 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:32.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.505+0000 7f9ff9ac4700 1 --2- 172.21.15.78:0/2634038214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d3d509000 0x557d3d2ba100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.505+0000 7f9ffa2c5700 1 --2- 172.21.15.78:0/2634038214 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x557d3d508c00 0x557d3d2b9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.505+0000 7f9ffaac6700 1 -- 172.21.15.78:0/2634038214 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x557d3d508c00 msgr2=0x557d3d2b9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:32.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.505+0000 7f9ffaac6700 1 --2- 172.21.15.78:0/2634038214 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x557d3d508c00 0x557d3d2b9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.505+0000 7f9ffaac6700 1 -- 172.21.15.78:0/2634038214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d3d509000 msgr2=0x557d3d2ba100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:32.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.505+0000 7f9ffaac6700 1 --2- 172.21.15.78:0/2634038214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d3d509000 0x557d3d2ba100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.505+0000 7f9ffaac6700 1 -- 172.21.15.78:0/2634038214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d3d234ea0 con 0x557d3d508800 2023-10-11T09:50:32.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.505+0000 7f9ffaac6700 1 --2- 172.21.15.78:0/2634038214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d3d508800 0x557d3d2b9600 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x557d3dff68a0 tx=0x557d3dffac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5db4e210a62e464 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.505+0000 7f9ff5abc700 1 -- 172.21.15.78:0/2634038214 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557d3d243600 con 0x557d3d508800 2023-10-11T09:50:32.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.505+0000 7f9ff5abc700 1 -- 172.21.15.78:0/2634038214 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557d3d25e900 con 0x557d3d508800 2023-10-11T09:50:32.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.505+0000 7f9ff5abc700 1 -- 172.21.15.78:0/2634038214 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557d3d2431e0 con 0x557d3d508800 2023-10-11T09:50:32.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.505+0000 7f9ffbb09840 1 -- 172.21.15.78:0/2634038214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d3d508800 msgr2=0x557d3d2b9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.505+0000 7f9ffbb09840 1 --2- 172.21.15.78:0/2634038214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d3d508800 0x557d3d2b9600 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x557d3dff68a0 tx=0x557d3dffac00 comp rx=0 tx=0).stop 2023-10-11T09:50:32.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.505+0000 7f9ffbb09840 1 -- 172.21.15.78:0/2634038214 shutdown_connections 2023-10-11T09:50:32.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.505+0000 7f9ffbb09840 1 --2- 172.21.15.78:0/2634038214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d3d509000 0x557d3d2ba100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.505+0000 7f9ffbb09840 1 --2- 172.21.15.78:0/2634038214 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x557d3d508c00 0x557d3d2b9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.505+0000 7f9ffbb09840 1 --2- 172.21.15.78:0/2634038214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d3d508800 0x557d3d2b9600 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.505+0000 7f9ffbb09840 1 -- 172.21.15.78:0/2634038214 >> 172.21.15.78:0/2634038214 conn(0x557d3d212c00 msgr2=0x557d3d2e1800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:32.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.505+0000 7f9ffbb09840 1 -- 172.21.15.78:0/2634038214 shutdown_connections 2023-10-11T09:50:32.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.505+0000 7f9ffbb09840 1 -- 172.21.15.78:0/2634038214 wait complete. 2023-10-11T09:50:32.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.505+0000 7f9ffbb09840 1 Processor -- start 2023-10-11T09:50:32.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.505+0000 7f9ffbb09840 1 -- start start 2023-10-11T09:50:32.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.509+0000 7f9ffbb09840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x557d3d508800 0x557d3d2bb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.509+0000 7f9ffbb09840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d3d508c00 0x557d3d2bac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.509+0000 7f9ffbb09840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d3d509000 0x557d3d2ba680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.509+0000 7f9ffbb09840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d3d2923c0 con 0x557d3d508c00 2023-10-11T09:50:32.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.509+0000 7f9ffbb09840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d3d292280 con 0x557d3d509000 2023-10-11T09:50:32.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.509+0000 7f9ffbb09840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x557d3d292140 con 0x557d3d508800 2023-10-11T09:50:32.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.509+0000 7f9ffa2c5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d3d508c00 0x557d3d2bac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.509+0000 7f9ffa2c5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d3d508c00 0x557d3d2bac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36104/0 (socket says 172.21.15.78:36104) 2023-10-11T09:50:32.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.509+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 learned_addr learned my addr 172.21.15.78:0/1569352335 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:32.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.509+0000 7f9ffaac6700 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x557d3d508800 0x557d3d2bb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.509+0000 7f9ff9ac4700 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d3d509000 0x557d3d2ba680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.509+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x557d3d508800 msgr2=0x557d3d2bb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.509+0000 7f9ffa2c5700 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x557d3d508800 0x557d3d2bb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.509+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d3d509000 msgr2=0x557d3d2ba680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.509+0000 7f9ffa2c5700 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d3d509000 0x557d3d2ba680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.509+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d3d234b60 con 0x557d3d508c00 2023-10-11T09:50:32.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.509+0000 7f9ff9ac4700 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d3d509000 0x557d3d2ba680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:50:32.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.509+0000 7f9ffa2c5700 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d3d508c00 0x557d3d2bac00 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x557d3dff74a0 tx=0x557d3e004ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.509+0000 7f9ff42b9700 1 -- 172.21.15.78:0/1569352335 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557d3d242f20 con 0x557d3d508c00 2023-10-11T09:50:32.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.509+0000 7f9ff42b9700 1 -- 172.21.15.78:0/1569352335 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557d3d308d80 con 0x557d3d508c00 2023-10-11T09:50:32.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.509+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557d3d234340 con 0x557d3d508c00 2023-10-11T09:50:32.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.509+0000 7f9ff42b9700 1 -- 172.21.15.78:0/1569352335 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557d3d243600 con 0x557d3d508c00 2023-10-11T09:50:32.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.509+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d3d235520 con 0x557d3d508c00 2023-10-11T09:50:32.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.509+0000 7f9ffbb09840 4 datascan.init: resolving metadata pool 14 2023-10-11T09:50:32.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.513+0000 7f9ff42b9700 1 -- 172.21.15.78:0/1569352335 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x557d3d270840 con 0x557d3d508c00 2023-10-11T09:50:32.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.513+0000 7f9ff42b9700 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x557d3d576800 0x557d3d2bb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.513+0000 7f9ff42b9700 1 -- 172.21.15.78:0/1569352335 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6638+0+0 (secure 0 0 0) 0x557d3d24ca80 con 0x557d3d508c00 2023-10-11T09:50:32.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.513+0000 7f9ffaac6700 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x557d3d576800 0x557d3d2bb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.513+0000 7f9ffbb09840 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-10-11T09:50:32.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.513+0000 7f9ffbb09840 4 datascan.main: data pool 'cephfs_data' has ID 15 2023-10-11T09:50:32.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.513+0000 7f9ffbb09840 4 datascan.main: opening data pool 'cephfs_data' 2023-10-11T09:50:32.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.513+0000 7f9ffbb09840 4 datascan.main: data pool 'cephfs_data_extra' has ID 16 2023-10-11T09:50:32.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.513+0000 7f9ffbb09840 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-10-11T09:50:32.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.513+0000 7f9ffbb09840 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x557d3d509400 0x557d3d2ba100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.513+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:1 15.0 15:00000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d509800 con 0x557d3d509400 2023-10-11T09:50:32.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.513+0000 7f9ff9ac4700 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x557d3d509400 0x557d3d2ba100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.513+0000 7f9ffaac6700 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x557d3d576800 0x557d3d2bb700 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x557d3dff68a0 tx=0x557d3e004de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.517+0000 7f9ff9ac4700 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x557d3d509400 0x557d3d2ba100 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.517+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.2 v2:172.21.15.78:6808/35732 1 ==== osd_op_reply(1 [pgnls start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d509400 2023-10-11T09:50:32.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.517+0000 7f9ff52bb700 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x557d3d576c00 0x557d3d676000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.517+0000 7f9ff52bb700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:2 15.20 15:04000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d577000 con 0x557d3d576c00 2023-10-11T09:50:32.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.517+0000 7f9ffaac6700 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x557d3d576c00 0x557d3d676000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.517+0000 7f9ffaac6700 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x557d3d576c00 0x557d3d676000 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.517+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 1 ==== osd_op_reply(2 [pgnls start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.517+0000 7f9ff5abc700 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x557d3d577800 0x557d3d676580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.517+0000 7f9ff5abc700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:3 15.10 15:08000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d577c00 con 0x557d3d577800 2023-10-11T09:50:32.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.521+0000 7f9ffa2c5700 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x557d3d577800 0x557d3d676580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.521+0000 7f9ffa2c5700 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x557d3d577800 0x557d3d676580 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.521+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.4 v2:172.21.15.133:6808/35898 1 ==== osd_op_reply(3 [pgnls start_epoch 110 out=49b] v108'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e093440 con 0x557d3d577800 2023-10-11T09:50:32.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.521+0000 7f9ff52bb700 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x557d3d577400 0x557d3d676b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.521+0000 7f9ff52bb700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:4 15.30 15:0c000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68c000 con 0x557d3d577400 2023-10-11T09:50:32.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.521+0000 7f9ff9ac4700 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x557d3d577400 0x557d3d676b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.525+0000 7f9ff9ac4700 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x557d3d577400 0x557d3d676b00 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.525+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.7 v2:172.21.15.133:6824/35901 1 ==== osd_op_reply(4 [pgnls start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d577400 2023-10-11T09:50:32.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.525+0000 7f9ff5abc700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:5 15.8 15:10000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68c000 con 0x557d3d509400 2023-10-11T09:50:32.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.525+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.2 v2:172.21.15.78:6808/35732 2 ==== osd_op_reply(5 [pgnls start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d509400 2023-10-11T09:50:32.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.525+0000 7f9ff52bb700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:6 15.28 15:14000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68c400 con 0x557d3d577400 2023-10-11T09:50:32.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.525+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.7 v2:172.21.15.133:6824/35901 2 ==== osd_op_reply(6 [pgnls start_epoch 110 out=49b] v108'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d577400 2023-10-11T09:50:32.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.525+0000 7f9ff5abc700 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x557d3d68cc00 0x557d3d677600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.525+0000 7f9ff5abc700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:7 15.18 15:18000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d509800 con 0x557d3d68cc00 2023-10-11T09:50:32.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.525+0000 7f9ffaac6700 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x557d3d68cc00 0x557d3d677600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.529+0000 7f9ffaac6700 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x557d3d68cc00 0x557d3d677600 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.529+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(7 [pgnls start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e093200 con 0x557d3d68cc00 2023-10-11T09:50:32.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.529+0000 7f9ff52bb700 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x557d3d68c800 0x557d3d677080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.529+0000 7f9ff52bb700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:8 15.38 15:1c000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68d000 con 0x557d3d68c800 2023-10-11T09:50:32.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.529+0000 7f9ffa2c5700 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x557d3d68c800 0x557d3d677080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.529+0000 7f9ffa2c5700 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x557d3d68c800 0x557d3d677080 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.529+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.0 v2:172.21.15.78:6800/35735 1 ==== osd_op_reply(8 [pgnls start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e093440 con 0x557d3d68c800 2023-10-11T09:50:32.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.529+0000 7f9ff5abc700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:9 15.4 15:20000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69a000 con 0x557d3d576c00 2023-10-11T09:50:32.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.529+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 2 ==== osd_op_reply(9 [pgnls start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.533+0000 7f9ff52bb700 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x557d3d68d400 0x557d3d678100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.533+0000 7f9ff52bb700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:10 15.24 15:24000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68d800 con 0x557d3d68d400 2023-10-11T09:50:32.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.533+0000 7f9ff9ac4700 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x557d3d68d400 0x557d3d678100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.533+0000 7f9ff9ac4700 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x557d3d68d400 0x557d3d678100 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.533+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== osd_op_reply(10 [pgnls start_epoch 110 out=49b] v108'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d68d400 2023-10-11T09:50:32.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.533+0000 7f9ff5abc700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:11 15.14 15:28000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69a400 con 0x557d3d68d400 2023-10-11T09:50:32.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.533+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.6 v2:172.21.15.133:6800/35900 2 ==== osd_op_reply(11 [pgnls start_epoch 110 out=49b] v108'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d68d400 2023-10-11T09:50:32.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.533+0000 7f9ff52bb700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:12 15.34 15:2c000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68dc00 con 0x557d3d577800 2023-10-11T09:50:32.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.537+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.4 v2:172.21.15.133:6808/35898 2 ==== osd_op_reply(12 [pgnls start_epoch 110 out=49b] v108'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e093440 con 0x557d3d577800 2023-10-11T09:50:32.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.537+0000 7f9ff5abc700 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x557d3d69a800 0x557d3d677b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.537+0000 7f9ff5abc700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:13 15.c 15:30000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69ac00 con 0x557d3d69a800 2023-10-11T09:50:32.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.537+0000 7f9ffaac6700 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x557d3d69a800 0x557d3d677b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.537+0000 7f9ffaac6700 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x557d3d69a800 0x557d3d677b80 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.537+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.5 v2:172.21.15.133:6816/35899 1 ==== osd_op_reply(13 [pgnls start_epoch 110 out=49b] v108'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e093200 con 0x557d3d69a800 2023-10-11T09:50:32.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.537+0000 7f9ff52bb700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:14 15.2c 15:34000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68d000 con 0x557d3d577800 2023-10-11T09:50:32.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.541+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.4 v2:172.21.15.133:6808/35898 3 ==== osd_op_reply(14 [pgnls start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e093440 con 0x557d3d577800 2023-10-11T09:50:32.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.541+0000 7f9ff5abc700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:15 15.1c 15:38000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69a000 con 0x557d3d576c00 2023-10-11T09:50:32.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.541+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 3 ==== osd_op_reply(15 [pgnls start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.541+0000 7f9ff52bb700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:16 15.3c 15:3c000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68dc00 con 0x557d3d68cc00 2023-10-11T09:50:32.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.541+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.3 v2:172.21.15.78:6824/35738 2 ==== osd_op_reply(16 [pgnls start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e093200 con 0x557d3d68cc00 2023-10-11T09:50:32.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.541+0000 7f9ff5abc700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:17 15.2 15:40000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d509800 con 0x557d3d509400 2023-10-11T09:50:32.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.541+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.2 v2:172.21.15.78:6808/35732 3 ==== osd_op_reply(17 [pgnls start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d509400 2023-10-11T09:50:32.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.541+0000 7f9ff52bb700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:18 15.22 15:44000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68c000 con 0x557d3d509400 2023-10-11T09:50:32.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.541+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.2 v2:172.21.15.78:6808/35732 4 ==== osd_op_reply(18 [pgnls start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d509400 2023-10-11T09:50:32.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.541+0000 7f9ff5abc700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:19 15.12 15:48000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d576400 con 0x557d3d68cc00 2023-10-11T09:50:32.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.541+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.3 v2:172.21.15.78:6824/35738 3 ==== osd_op_reply(19 [pgnls start_epoch 110 out=49b] v108'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e093200 con 0x557d3d68cc00 2023-10-11T09:50:32.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.541+0000 7f9ff52bb700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:20 15.32 15:4c000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68c400 con 0x557d3d509400 2023-10-11T09:50:32.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.545+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.2 v2:172.21.15.78:6808/35732 5 ==== osd_op_reply(20 [pgnls start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d509400 2023-10-11T09:50:32.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.545+0000 7f9ff5abc700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:21 15.a 15:50000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d577c00 con 0x557d3d509400 2023-10-11T09:50:32.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.545+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.2 v2:172.21.15.78:6808/35732 6 ==== osd_op_reply(21 [pgnls start_epoch 110 out=49b] v108'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d509400 2023-10-11T09:50:32.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.545+0000 7f9ff52bb700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:22 15.2a 15:54000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68d800 con 0x557d3d68c800 2023-10-11T09:50:32.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.545+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.0 v2:172.21.15.78:6800/35735 2 ==== osd_op_reply(22 [pgnls start_epoch 110 out=81b] v108'10 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557d3e093440 con 0x557d3d68c800 2023-10-11T09:50:32.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.545+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:23 15.2a 15:57fe7535:::100000001f6.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d509c00 con 0x557d3d68c800 2023-10-11T09:50:32.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.545+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.0 v2:172.21.15.78:6800/35735 3 ==== osd_op_reply(23 100000001f6.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e093440 con 0x557d3d68c800 2023-10-11T09:50:32.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.545+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628278.0 2023-10-11T09:50:32.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.545+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:24 15.2a 15:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69a400 con 0x557d3d68c800 2023-10-11T09:50:32.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.549+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.0 v2:172.21.15.78:6800/35735 4 ==== osd_op_reply(24 100000001f6.00000000 [call] v110'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x557d3e093440 con 0x557d3d68c800 2023-10-11T09:50:32.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.549+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:25 15.1a 15:58000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69ac00 con 0x557d3d69a800 2023-10-11T09:50:32.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.549+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.5 v2:172.21.15.133:6816/35899 2 ==== osd_op_reply(25 [pgnls start_epoch 110 out=49b] v108'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e093200 con 0x557d3d69a800 2023-10-11T09:50:32.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.549+0000 7f9ff52bb700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:26 15.3a 15:5c000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69a000 con 0x557d3d577400 2023-10-11T09:50:32.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.549+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.7 v2:172.21.15.133:6824/35901 3 ==== osd_op_reply(26 [pgnls start_epoch 110 out=49b] v108'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d577400 2023-10-11T09:50:32.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.553+0000 7f9ff5abc700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:27 15.6 15:60000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d577000 con 0x557d3d68c800 2023-10-11T09:50:32.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.553+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.0 v2:172.21.15.78:6800/35735 5 ==== osd_op_reply(27 [pgnls start_epoch 110 out=49b] v108'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e093440 con 0x557d3d68c800 2023-10-11T09:50:32.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.553+0000 7f9ff52bb700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:28 15.26 15:64000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69b000 con 0x557d3d68c800 2023-10-11T09:50:32.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.553+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.0 v2:172.21.15.78:6800/35735 6 ==== osd_op_reply(28 [pgnls start_epoch 110 out=49b] v108'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e093440 con 0x557d3d68c800 2023-10-11T09:50:32.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.553+0000 7f9ff5abc700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:29 15.16 15:68000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d577000 con 0x557d3d68c800 2023-10-11T09:50:32.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.553+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.0 v2:172.21.15.78:6800/35735 7 ==== osd_op_reply(29 [pgnls start_epoch 110 out=49b] v108'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e093440 con 0x557d3d68c800 2023-10-11T09:50:32.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.553+0000 7f9ff52bb700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:30 15.36 15:6c000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69b400 con 0x557d3d68d400 2023-10-11T09:50:32.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.553+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.6 v2:172.21.15.133:6800/35900 3 ==== osd_op_reply(30 [pgnls start_epoch 110 out=101b] v108'17 uv2 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d68d400 2023-10-11T09:50:32.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.553+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:31 15.36 15:6ee5e2c4:::100000001f9.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d576400 con 0x557d3d68d400 2023-10-11T09:50:32.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.557+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.6 v2:172.21.15.133:6800/35900 4 ==== osd_op_reply(31 100000001f9.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d68d400 2023-10-11T09:50:32.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.557+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628281.0 2023-10-11T09:50:32.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.557+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:32 15.36 15:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68c000 con 0x557d3d68d400 2023-10-11T09:50:32.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.557+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.6 v2:172.21.15.133:6800/35900 5 ==== osd_op_reply(32 100000001f9.00000000 [call] v110'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d68d400 2023-10-11T09:50:32.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.557+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:33 15.36 15:6fe38dc2::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68c400 con 0x557d3d68d400 2023-10-11T09:50:32.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.561+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.6 v2:172.21.15.133:6800/35900 6 ==== osd_op_reply(33 [pgnls start_epoch 110 out=81b] v110'18 uv18 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d68d400 2023-10-11T09:50:32.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.561+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:34 15.36 15:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68dc00 con 0x557d3d68d400 2023-10-11T09:50:32.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.561+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.6 v2:172.21.15.133:6800/35900 7 ==== osd_op_reply(34 100000001f7.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d68d400 2023-10-11T09:50:32.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.561+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628279.0 2023-10-11T09:50:32.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.561+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:35 15.36 15:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d509800 con 0x557d3d68d400 2023-10-11T09:50:32.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.565+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.6 v2:172.21.15.133:6800/35900 8 ==== osd_op_reply(35 100000001f7.00000000 [call] v110'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d68d400 2023-10-11T09:50:32.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.565+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:36 15.e 15:70000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68c000 con 0x557d3d509400 2023-10-11T09:50:32.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.565+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.2 v2:172.21.15.78:6808/35732 7 ==== osd_op_reply(36 [pgnls start_epoch 110 out=49b] v108'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d509400 2023-10-11T09:50:32.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.565+0000 7f9ff5abc700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:37 15.2e 15:74000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69a400 con 0x557d3d68cc00 2023-10-11T09:50:32.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.565+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.3 v2:172.21.15.78:6824/35738 4 ==== osd_op_reply(37 [pgnls start_epoch 110 out=49b] v108'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e093200 con 0x557d3d68cc00 2023-10-11T09:50:32.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.565+0000 7f9ff52bb700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:38 15.1e 15:78000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69b800 con 0x557d3d509400 2023-10-11T09:50:32.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.565+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.2 v2:172.21.15.78:6808/35732 8 ==== osd_op_reply(38 [pgnls start_epoch 110 out=49b] v108'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d509400 2023-10-11T09:50:32.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.565+0000 7f9ff5abc700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:39 15.3e 15:7c000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d509c00 con 0x557d3d68c800 2023-10-11T09:50:32.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.569+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.0 v2:172.21.15.78:6800/35735 8 ==== osd_op_reply(39 [pgnls start_epoch 110 out=49b] v108'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e093440 con 0x557d3d68c800 2023-10-11T09:50:32.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.569+0000 7f9ff52bb700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:40 15.1 15:80000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69bc00 con 0x557d3d68d400 2023-10-11T09:50:32.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.569+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.6 v2:172.21.15.133:6800/35900 9 ==== osd_op_reply(40 [pgnls start_epoch 110 out=49b] v108'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d68d400 2023-10-11T09:50:32.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.569+0000 7f9ff5abc700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:41 15.21 15:84000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68d000 con 0x557d3d68cc00 2023-10-11T09:50:32.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.569+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.3 v2:172.21.15.78:6824/35738 5 ==== osd_op_reply(41 [pgnls start_epoch 110 out=49b] v108'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e093200 con 0x557d3d68cc00 2023-10-11T09:50:32.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.569+0000 7f9ff52bb700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:42 15.11 15:88000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69b800 con 0x557d3d509400 2023-10-11T09:50:32.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.569+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.2 v2:172.21.15.78:6808/35732 9 ==== osd_op_reply(42 [pgnls start_epoch 110 out=49b] v108'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d509400 2023-10-11T09:50:32.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.569+0000 7f9ff5abc700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:43 15.31 15:8c000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68d800 con 0x557d3d68cc00 2023-10-11T09:50:32.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.569+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.3 v2:172.21.15.78:6824/35738 6 ==== osd_op_reply(43 [pgnls start_epoch 110 out=49b] v108'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e093200 con 0x557d3d68cc00 2023-10-11T09:50:32.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.569+0000 7f9ff52bb700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:44 15.9 15:90000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69bc00 con 0x557d3d68d400 2023-10-11T09:50:32.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.573+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.6 v2:172.21.15.133:6800/35900 10 ==== osd_op_reply(44 [pgnls start_epoch 110 out=49b] v108'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d68d400 2023-10-11T09:50:32.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.573+0000 7f9ff5abc700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:45 15.29 15:94000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ac000 con 0x557d3d68c800 2023-10-11T09:50:32.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.573+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.0 v2:172.21.15.78:6800/35735 9 ==== osd_op_reply(45 [pgnls start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e093440 con 0x557d3d68c800 2023-10-11T09:50:32.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.573+0000 7f9ff52bb700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:46 15.19 15:98000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68c400 con 0x557d3d577800 2023-10-11T09:50:32.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.573+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.4 v2:172.21.15.133:6808/35898 4 ==== osd_op_reply(46 [pgnls start_epoch 110 out=81b] v108'11 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557d3e093440 con 0x557d3d577800 2023-10-11T09:50:32.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.573+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:47 15.19 15:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d577c00 con 0x557d3d577800 2023-10-11T09:50:32.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.573+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.4 v2:172.21.15.133:6808/35898 5 ==== osd_op_reply(47 100000001f8.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e093440 con 0x557d3d577800 2023-10-11T09:50:32.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.577+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628280.0 2023-10-11T09:50:32.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.577+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:48 15.19 15:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d576400 con 0x557d3d577800 2023-10-11T09:50:32.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.577+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.4 v2:172.21.15.133:6808/35898 6 ==== osd_op_reply(48 100000001f8.00000000 [call] v110'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x557d3e093440 con 0x557d3d577800 2023-10-11T09:50:32.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.577+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:49 15.39 15:9c000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69a000 con 0x557d3d577400 2023-10-11T09:50:32.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.581+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.7 v2:172.21.15.133:6824/35901 4 ==== osd_op_reply(49 [pgnls start_epoch 110 out=49b] v108'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d577400 2023-10-11T09:50:32.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.581+0000 7f9ff52bb700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:50 15.5 15:a0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68dc00 con 0x557d3d68d400 2023-10-11T09:50:32.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.581+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.6 v2:172.21.15.133:6800/35900 11 ==== osd_op_reply(50 [pgnls start_epoch 110 out=49b] v108'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d68d400 2023-10-11T09:50:32.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.581+0000 7f9ff5abc700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:51 15.25 15:a4000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ac400 con 0x557d3d577400 2023-10-11T09:50:32.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.581+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.7 v2:172.21.15.133:6824/35901 5 ==== osd_op_reply(51 [pgnls start_epoch 110 out=49b] v108'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d577400 2023-10-11T09:50:32.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.581+0000 7f9ff52bb700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:52 15.15 15:a8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68c000 con 0x557d3d577400 2023-10-11T09:50:32.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.581+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.7 v2:172.21.15.133:6824/35901 6 ==== osd_op_reply(52 [pgnls start_epoch 110 out=49b] v108'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d577400 2023-10-11T09:50:32.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.585+0000 7f9ff5abc700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:53 15.35 15:ac000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ac800 con 0x557d3d577400 2023-10-11T09:50:32.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.585+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.7 v2:172.21.15.133:6824/35901 7 ==== osd_op_reply(53 [pgnls start_epoch 110 out=49b] v108'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d577400 2023-10-11T09:50:32.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.585+0000 7f9ff52bb700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:54 15.d 15:b0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d509800 con 0x557d3d577400 2023-10-11T09:50:32.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.585+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.7 v2:172.21.15.133:6824/35901 8 ==== osd_op_reply(54 [pgnls start_epoch 110 out=49b] v108'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d577400 2023-10-11T09:50:32.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.585+0000 7f9ff5abc700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:55 15.2d 15:b4000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6acc00 con 0x557d3d509400 2023-10-11T09:50:32.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.585+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.2 v2:172.21.15.78:6808/35732 10 ==== osd_op_reply(55 [pgnls start_epoch 110 out=49b] v108'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d509400 2023-10-11T09:50:32.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.585+0000 7f9ff52bb700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:56 15.1d 15:b8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68dc00 con 0x557d3d577800 2023-10-11T09:50:32.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.585+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.4 v2:172.21.15.133:6808/35898 7 ==== osd_op_reply(56 [pgnls start_epoch 110 out=49b] v108'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e093440 con 0x557d3d577800 2023-10-11T09:50:32.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.585+0000 7f9ff5abc700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:57 15.3d 15:bc000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ad000 con 0x557d3d576c00 2023-10-11T09:50:32.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.589+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 4 ==== osd_op_reply(57 [pgnls start_epoch 110 out=49b] v108'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.589+0000 7f9ff52bb700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:58 15.3 15:c0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68c000 con 0x557d3d577800 2023-10-11T09:50:32.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.589+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.4 v2:172.21.15.133:6808/35898 8 ==== osd_op_reply(58 [pgnls start_epoch 110 out=49b] v108'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e093440 con 0x557d3d577800 2023-10-11T09:50:32.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.589+0000 7f9ff5abc700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:59 15.23 15:c4000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ad400 con 0x557d3d68c800 2023-10-11T09:50:32.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.589+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.0 v2:172.21.15.78:6800/35735 10 ==== osd_op_reply(59 [pgnls start_epoch 110 out=49b] v108'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e093440 con 0x557d3d68c800 2023-10-11T09:50:32.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.589+0000 7f9ff52bb700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:60 15.13 15:c8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68c400 con 0x557d3d576c00 2023-10-11T09:50:32.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.589+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 5 ==== osd_op_reply(60 [pgnls start_epoch 110 out=81b] v108'16 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.589+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:61 15.13 15:ca8fdbba:::100000001fb.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69b400 con 0x557d3d576c00 2023-10-11T09:50:32.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.593+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 6 ==== osd_op_reply(61 100000001fb.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.593+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628283.0 2023-10-11T09:50:32.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.593+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:62 15.13 15:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ad000 con 0x557d3d576c00 2023-10-11T09:50:32.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.593+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 7 ==== osd_op_reply(62 100000001fb.00000000 [call] v110'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.593+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:63 15.33 15:cc000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ac400 con 0x557d3d68cc00 2023-10-11T09:50:32.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.593+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.3 v2:172.21.15.78:6824/35738 7 ==== osd_op_reply(63 [pgnls start_epoch 110 out=49b] v108'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e093200 con 0x557d3d68cc00 2023-10-11T09:50:32.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.593+0000 7f9ff52bb700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:64 15.b 15:d0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d509c00 con 0x557d3d509400 2023-10-11T09:50:32.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.597+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.2 v2:172.21.15.78:6808/35732 11 ==== osd_op_reply(64 [pgnls start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d509400 2023-10-11T09:50:32.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.597+0000 7f9ff5abc700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:65 15.2b 15:d4000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ad800 con 0x557d3d509400 2023-10-11T09:50:32.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.597+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.2 v2:172.21.15.78:6808/35732 12 ==== osd_op_reply(65 [pgnls start_epoch 110 out=49b] v108'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d509400 2023-10-11T09:50:32.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.597+0000 7f9ff52bb700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:66 15.1b 15:d8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69b800 con 0x557d3d577400 2023-10-11T09:50:32.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.597+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.7 v2:172.21.15.133:6824/35901 9 ==== osd_op_reply(66 [pgnls start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d577400 2023-10-11T09:50:32.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.597+0000 7f9ff5abc700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:67 15.3b 15:dc000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ac000 con 0x557d3d68d400 2023-10-11T09:50:32.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.597+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.6 v2:172.21.15.133:6800/35900 12 ==== osd_op_reply(67 [pgnls start_epoch 110 out=49b] v108'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d68d400 2023-10-11T09:50:32.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.597+0000 7f9ff52bb700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:68 15.7 15:e0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69bc00 con 0x557d3d576c00 2023-10-11T09:50:32.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.597+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 8 ==== osd_op_reply(68 [pgnls start_epoch 110 out=81b] v108'6 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.601+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:69 15.7 15:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ac800 con 0x557d3d576c00 2023-10-11T09:50:32.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.601+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 9 ==== osd_op_reply(69 10000000001.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.601+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511627777.0 2023-10-11T09:50:32.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.601+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:70 15.7 15:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d577000 con 0x557d3d576c00 2023-10-11T09:50:32.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.601+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 10 ==== osd_op_reply(70 10000000001.00000000 [call] v110'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.601+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:71 15.27 15:e4000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d577c00 con 0x557d3d68d400 2023-10-11T09:50:32.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.605+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.6 v2:172.21.15.133:6800/35900 13 ==== osd_op_reply(71 [pgnls start_epoch 110 out=49b] v108'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d68d400 2023-10-11T09:50:32.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.605+0000 7f9ff52bb700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:72 15.17 15:e8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69a000 con 0x557d3d577800 2023-10-11T09:50:32.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.605+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.4 v2:172.21.15.133:6808/35898 9 ==== osd_op_reply(72 [pgnls start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e093440 con 0x557d3d577800 2023-10-11T09:50:32.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.605+0000 7f9ff5abc700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:73 15.37 15:ec000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6adc00 con 0x557d3d577800 2023-10-11T09:50:32.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.605+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.4 v2:172.21.15.133:6808/35898 10 ==== osd_op_reply(73 [pgnls start_epoch 110 out=49b] v108'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e093440 con 0x557d3d577800 2023-10-11T09:50:32.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.605+0000 7f9ff52bb700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:74 15.f 15:f0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69b000 con 0x557d3d68d400 2023-10-11T09:50:32.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.605+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.6 v2:172.21.15.133:6800/35900 14 ==== osd_op_reply(74 [pgnls start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d68d400 2023-10-11T09:50:32.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.605+0000 7f9ff5abc700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:75 15.2f 15:f4000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6acc00 con 0x557d3d68c800 2023-10-11T09:50:32.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.609+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.0 v2:172.21.15.78:6800/35735 11 ==== osd_op_reply(75 [pgnls start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e093440 con 0x557d3d68c800 2023-10-11T09:50:32.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.609+0000 7f9ff52bb700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:76 15.1f 15:f8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69a000 con 0x557d3d68cc00 2023-10-11T09:50:32.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.609+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.3 v2:172.21.15.78:6824/35738 8 ==== osd_op_reply(76 [pgnls start_epoch 110 out=49b] v108'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e093200 con 0x557d3d68cc00 2023-10-11T09:50:32.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.609+0000 7f9ff5abc700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:77 15.3f 15:fc000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69b800 con 0x557d3d576c00 2023-10-11T09:50:32.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.609+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 11 ==== osd_op_reply(77 [pgnls start_epoch 110 out=81b] v108'9 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.609+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:78 15.3f 15:feb1c803:::100000001fa.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68d000 con 0x557d3d576c00 2023-10-11T09:50:32.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.609+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 12 ==== osd_op_reply(78 100000001fa.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.609+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628282.0 2023-10-11T09:50:32.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.609+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:79 15.3f 15:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68d800 con 0x557d3d576c00 2023-10-11T09:50:32.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.613+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 13 ==== osd_op_reply(79 100000001fa.00000000 [call] v110'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.613+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:80 16.0 16:00000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69ac00 con 0x557d3d68c800 2023-10-11T09:50:32.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.613+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.0 v2:172.21.15.78:6800/35735 12 ==== osd_op_reply(80 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e093440 con 0x557d3d68c800 2023-10-11T09:50:32.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.613+0000 7f9ff5abc700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:81 16.10 16:08000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69b000 con 0x557d3d509400 2023-10-11T09:50:32.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.617+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.2 v2:172.21.15.78:6808/35732 13 ==== osd_op_reply(81 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d509400 2023-10-11T09:50:32.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.617+0000 7f9ff52bb700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:82 16.8 16:10000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68dc00 con 0x557d3d68c800 2023-10-11T09:50:32.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.617+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.0 v2:172.21.15.78:6800/35735 13 ==== osd_op_reply(82 [pgnls start_epoch 110 out=81b] v108'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557d3e093440 con 0x557d3d68c800 2023-10-11T09:50:32.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.617+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:83 16.8 16:1247e94b:::100000001f9.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69a400 con 0x557d3d68c800 2023-10-11T09:50:32.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.617+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.0 v2:172.21.15.78:6800/35735 14 ==== osd_op_reply(83 100000001f9.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e093440 con 0x557d3d68c800 2023-10-11T09:50:32.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.617+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628281.1 2023-10-11T09:50:32.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.617+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:84 15.36 15:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ac000 con 0x557d3d68d400 2023-10-11T09:50:32.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.621+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.6 v2:172.21.15.133:6800/35900 15 ==== osd_op_reply(84 100000001f9.00000000 [call,setxattr (8)] v110'20 uv20 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d68d400 2023-10-11T09:50:32.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.621+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:85 16.18 16:18000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ac400 con 0x557d3d68c800 2023-10-11T09:50:32.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.621+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.0 v2:172.21.15.78:6800/35735 15 ==== osd_op_reply(85 [pgnls start_epoch 110 out=81b] v108'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557d3e093440 con 0x557d3d68c800 2023-10-11T09:50:32.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.621+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:86 16.18 16:1cd44203:::100000001fb.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ac800 con 0x557d3d68c800 2023-10-11T09:50:32.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.621+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.0 v2:172.21.15.78:6800/35735 16 ==== osd_op_reply(86 100000001fb.00000002 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e093440 con 0x557d3d68c800 2023-10-11T09:50:32.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.621+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628283.2 2023-10-11T09:50:32.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.621+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:87 15.13 15:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6acc00 con 0x557d3d576c00 2023-10-11T09:50:32.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.625+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 14 ==== osd_op_reply(87 100000001fb.00000000 [call,setxattr (8)] v110'18 uv18 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.625+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:88 16.4 16:20000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ad000 con 0x557d3d68cc00 2023-10-11T09:50:32.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.625+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.3 v2:172.21.15.78:6824/35738 9 ==== osd_op_reply(88 [pgnls start_epoch 110 out=81b] v108'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557d3e093200 con 0x557d3d68cc00 2023-10-11T09:50:32.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.625+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:89 16.4 16:24ae92ec:::100000001fb.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ad400 con 0x557d3d68cc00 2023-10-11T09:50:32.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.625+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.3 v2:172.21.15.78:6824/35738 10 ==== osd_op_reply(89 100000001fb.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e093200 con 0x557d3d68cc00 2023-10-11T09:50:32.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.625+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628283.1 2023-10-11T09:50:32.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.625+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:90 15.13 15:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ad800 con 0x557d3d576c00 2023-10-11T09:50:32.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.629+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 15 ==== osd_op_reply(90 100000001fb.00000000 [call,setxattr (8)] v110'19 uv19 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.629+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:91 16.14 16:28000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6adc00 con 0x557d3d68c800 2023-10-11T09:50:32.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.629+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.0 v2:172.21.15.78:6800/35735 17 ==== osd_op_reply(91 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e093440 con 0x557d3d68c800 2023-10-11T09:50:32.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.629+0000 7f9ff52bb700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:92 16.c 16:30000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68c000 con 0x557d3d69a800 2023-10-11T09:50:32.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.629+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.5 v2:172.21.15.133:6816/35899 3 ==== osd_op_reply(92 [pgnls start_epoch 110 out=81b] v108'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557d3e093200 con 0x557d3d69a800 2023-10-11T09:50:32.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.629+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:93 16.c 16:30604137:::100000001fc.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6acc00 con 0x557d3d69a800 2023-10-11T09:50:32.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.629+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.5 v2:172.21.15.133:6816/35899 4 ==== osd_op_reply(93 100000001fc.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e093200 con 0x557d3d69a800 2023-10-11T09:50:32.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.629+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628284.0 2023-10-11T09:50:32.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.629+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:94 15.c 15:30604137:::100000001fc.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ad000 con 0x557d3d69a800 2023-10-11T09:50:32.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.633+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.5 v2:172.21.15.133:6816/35899 5 ==== osd_op_reply(94 100000001fc.00000000 [call,setxattr (8)] v110'7 uv7 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e093200 con 0x557d3d69a800 2023-10-11T09:50:32.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.633+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:95 16.1c 16:38000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68d000 con 0x557d3d509400 2023-10-11T09:50:32.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.637+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.2 v2:172.21.15.78:6808/35732 14 ==== osd_op_reply(95 [pgnls start_epoch 110 out=101b] v108'3 uv3 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d509400 2023-10-11T09:50:32.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.637+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:96 16.1c 16:398da958:::100000001f7.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68d800 con 0x557d3d509400 2023-10-11T09:50:32.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.637+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.2 v2:172.21.15.78:6808/35732 15 ==== osd_op_reply(96 100000001f7.00000002 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d509400 2023-10-11T09:50:32.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.637+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628279.2 2023-10-11T09:50:32.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.637+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:97 15.36 15:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69a000 con 0x557d3d68d400 2023-10-11T09:50:32.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.637+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.6 v2:172.21.15.133:6800/35900 16 ==== osd_op_reply(97 100000001f7.00000000 [call,setxattr (8)] v110'21 uv21 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d68d400 2023-10-11T09:50:32.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.637+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:98 16.1c 16:3c0114ac::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69b800 con 0x557d3d509400 2023-10-11T09:50:32.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.641+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.2 v2:172.21.15.78:6808/35732 16 ==== osd_op_reply(98 [pgnls start_epoch 110 out=101b] v108'3 uv3 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d509400 2023-10-11T09:50:32.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.641+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:99 16.1c 16:3c0114ac:::100000001fa.00000004:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d577000 con 0x557d3d509400 2023-10-11T09:50:32.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.641+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.2 v2:172.21.15.78:6808/35732 17 ==== osd_op_reply(99 100000001fa.00000004 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d509400 2023-10-11T09:50:32.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.641+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628282.4 2023-10-11T09:50:32.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.641+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:100 15.3f 15:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6c6000 con 0x557d3d576c00 2023-10-11T09:50:32.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.645+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 16 ==== osd_op_reply(100 100000001fa.00000000 [call,setxattr (8)] v110'11 uv11 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.645+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:101 16.1c 16:3d6db04b::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6c6400 con 0x557d3d509400 2023-10-11T09:50:32.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.645+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.2 v2:172.21.15.78:6808/35732 18 ==== osd_op_reply(101 [pgnls start_epoch 110 out=81b] v108'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d509400 2023-10-11T09:50:32.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.645+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:102 16.1c 16:3d6db04b:::100000001f6.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6c6800 con 0x557d3d509400 2023-10-11T09:50:32.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.645+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.2 v2:172.21.15.78:6808/35732 19 ==== osd_op_reply(102 100000001f6.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d509400 2023-10-11T09:50:32.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.645+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628278.1 2023-10-11T09:50:32.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.645+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:103 15.2a 15:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6c6c00 con 0x557d3d68c800 2023-10-11T09:50:32.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.649+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.0 v2:172.21.15.78:6800/35735 18 ==== osd_op_reply(103 100000001f6.00000000 [call,setxattr (8)] v110'12 uv12 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e093440 con 0x557d3d68c800 2023-10-11T09:50:32.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.649+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:104 16.2 16:40000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6c7000 con 0x557d3d577800 2023-10-11T09:50:32.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.649+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.4 v2:172.21.15.133:6808/35898 11 ==== osd_op_reply(104 [pgnls start_epoch 110 out=101b] v108'2 uv2 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x557d3e093440 con 0x557d3d577800 2023-10-11T09:50:32.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.649+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:105 16.2 16:40d56c48:::100000001fc.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6c7400 con 0x557d3d577800 2023-10-11T09:50:32.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.649+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.4 v2:172.21.15.133:6808/35898 12 ==== osd_op_reply(105 100000001fc.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e093440 con 0x557d3d577800 2023-10-11T09:50:32.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.649+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628284.1 2023-10-11T09:50:32.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.649+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:106 15.c 15:30604137:::100000001fc.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6c7800 con 0x557d3d69a800 2023-10-11T09:50:32.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.653+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.5 v2:172.21.15.133:6816/35899 6 ==== osd_op_reply(106 100000001fc.00000000 [call,setxattr (8)] v110'8 uv8 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e093200 con 0x557d3d69a800 2023-10-11T09:50:32.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.653+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:107 16.2 16:46f89f8a::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6c7c00 con 0x557d3d577800 2023-10-11T09:50:32.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.653+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.4 v2:172.21.15.133:6808/35898 13 ==== osd_op_reply(107 [pgnls start_epoch 110 out=81b] v108'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557d3e093440 con 0x557d3d577800 2023-10-11T09:50:32.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.653+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:108 16.2 16:46f89f8a:::100000001f9.00000005:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d577000 con 0x557d3d577800 2023-10-11T09:50:32.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.657+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.4 v2:172.21.15.133:6808/35898 14 ==== osd_op_reply(108 100000001f9.00000005 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e093440 con 0x557d3d577800 2023-10-11T09:50:32.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.657+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628281.5 2023-10-11T09:50:32.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.657+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:109 15.36 15:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69ac00 con 0x557d3d68d400 2023-10-11T09:50:32.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.661+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.6 v2:172.21.15.133:6800/35900 17 ==== osd_op_reply(109 100000001f9.00000000 [call,setxattr (8)] v110'22 uv22 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d68d400 2023-10-11T09:50:32.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.661+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:110 16.12 16:48000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69a400 con 0x557d3d68d400 2023-10-11T09:50:32.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.661+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.6 v2:172.21.15.133:6800/35900 18 ==== osd_op_reply(110 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d68d400 2023-10-11T09:50:32.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.661+0000 7f9ff5abc700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:111 16.a 16:50000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69b400 con 0x557d3d509400 2023-10-11T09:50:32.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.661+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.2 v2:172.21.15.78:6808/35732 20 ==== osd_op_reply(111 [pgnls start_epoch 110 out=101b] v108'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d509400 2023-10-11T09:50:32.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.661+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:112 16.a 16:544c5ef0:::100000001fc.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69b000 con 0x557d3d509400 2023-10-11T09:50:32.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.661+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.2 v2:172.21.15.78:6808/35732 21 ==== osd_op_reply(112 100000001fc.00000003 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d509400 2023-10-11T09:50:32.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.661+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628284.3 2023-10-11T09:50:32.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.661+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:113 15.c 15:30604137:::100000001fc.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69a000 con 0x557d3d69a800 2023-10-11T09:50:32.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.665+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.5 v2:172.21.15.133:6816/35899 7 ==== osd_op_reply(113 100000001fc.00000000 [call,setxattr (8)] v110'9 uv9 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e093200 con 0x557d3d69a800 2023-10-11T09:50:32.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.665+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:114 16.a 16:56c65d28::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69b800 con 0x557d3d509400 2023-10-11T09:50:32.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.665+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.2 v2:172.21.15.78:6808/35732 22 ==== osd_op_reply(114 [pgnls start_epoch 110 out=101b] v108'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d509400 2023-10-11T09:50:32.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.665+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:115 16.a 16:56c65d28:::100000001fa.00000006:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68dc00 con 0x557d3d509400 2023-10-11T09:50:32.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.665+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.2 v2:172.21.15.78:6808/35732 23 ==== osd_op_reply(115 100000001fa.00000006 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d509400 2023-10-11T09:50:32.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.665+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628282.6 2023-10-11T09:50:32.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.669+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:116 15.3f 15:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68d000 con 0x557d3d576c00 2023-10-11T09:50:32.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.669+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 17 ==== osd_op_reply(116 100000001fa.00000000 [call,setxattr (8)] v110'12 uv12 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.669+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:117 16.a 16:57fe7535::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68d800 con 0x557d3d509400 2023-10-11T09:50:32.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.669+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.2 v2:172.21.15.78:6808/35732 24 ==== osd_op_reply(117 [pgnls start_epoch 110 out=81b] v108'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d509400 2023-10-11T09:50:32.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.669+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:118 16.a 16:57fe7535:::100000001f6.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ac400 con 0x557d3d509400 2023-10-11T09:50:32.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.673+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.2 v2:172.21.15.78:6808/35732 25 ==== osd_op_reply(118 100000001f6.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d509400 2023-10-11T09:50:32.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.673+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628278.0 2023-10-11T09:50:32.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.673+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:119 15.2a 15:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ac800 con 0x557d3d68c800 2023-10-11T09:50:32.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.673+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.0 v2:172.21.15.78:6800/35735 19 ==== osd_op_reply(119 100000001f6.00000000 [call,setxattr (8)] v110'13 uv13 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e093440 con 0x557d3d68c800 2023-10-11T09:50:32.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.673+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:120 16.1a 16:58000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6adc00 con 0x557d3d68cc00 2023-10-11T09:50:32.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.677+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.3 v2:172.21.15.78:6824/35738 11 ==== osd_op_reply(120 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e093200 con 0x557d3d68cc00 2023-10-11T09:50:32.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.677+0000 7f9ff5abc700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:121 16.6 16:60000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69bc00 con 0x557d3d68d400 2023-10-11T09:50:32.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.677+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.6 v2:172.21.15.133:6800/35900 19 ==== osd_op_reply(121 [pgnls start_epoch 110 out=81b] v108'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d68d400 2023-10-11T09:50:32.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.677+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:122 16.6 16:613bf181:::100000001f7.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ac000 con 0x557d3d68d400 2023-10-11T09:50:32.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.677+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.6 v2:172.21.15.133:6800/35900 20 ==== osd_op_reply(122 100000001f7.00000003 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d68d400 2023-10-11T09:50:32.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.677+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628279.3 2023-10-11T09:50:32.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.677+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:123 15.36 15:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69ac00 con 0x557d3d68d400 2023-10-11T09:50:32.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.681+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.6 v2:172.21.15.133:6800/35900 21 ==== osd_op_reply(123 100000001f7.00000000 [call,setxattr (8)] v110'23 uv23 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d68d400 2023-10-11T09:50:32.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.681+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:124 16.16 16:68000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69a400 con 0x557d3d68cc00 2023-10-11T09:50:32.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.681+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.3 v2:172.21.15.78:6824/35738 12 ==== osd_op_reply(124 [pgnls start_epoch 110 out=101b] v108'5 uv5 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x557d3e093200 con 0x557d3d68cc00 2023-10-11T09:50:32.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.681+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:125 16.16 16:6df493a7:::100000001f9.00000006:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69b400 con 0x557d3d68cc00 2023-10-11T09:50:32.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.681+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.3 v2:172.21.15.78:6824/35738 13 ==== osd_op_reply(125 100000001f9.00000006 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e093200 con 0x557d3d68cc00 2023-10-11T09:50:32.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.681+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628281.6 2023-10-11T09:50:32.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.681+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:126 15.36 15:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69b000 con 0x557d3d68d400 2023-10-11T09:50:32.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.685+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.6 v2:172.21.15.133:6800/35900 22 ==== osd_op_reply(126 100000001f9.00000000 [call,setxattr (8)] v110'24 uv24 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d68d400 2023-10-11T09:50:32.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.685+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:127 16.16 16:6ee5e2c4::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69b800 con 0x557d3d68cc00 2023-10-11T09:50:32.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.685+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.3 v2:172.21.15.78:6824/35738 14 ==== osd_op_reply(127 [pgnls start_epoch 110 out=101b] v108'5 uv5 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x557d3e093200 con 0x557d3d68cc00 2023-10-11T09:50:32.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.685+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:128 16.16 16:6ee5e2c4:::100000001f9.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ac400 con 0x557d3d68cc00 2023-10-11T09:50:32.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.685+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.3 v2:172.21.15.78:6824/35738 15 ==== osd_op_reply(128 100000001f9.00000000 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e093200 con 0x557d3d68cc00 2023-10-11T09:50:32.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.685+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628281.0 2023-10-11T09:50:32.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.685+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:129 15.36 15:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ac000 con 0x557d3d68d400 2023-10-11T09:50:32.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.689+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.6 v2:172.21.15.133:6800/35900 23 ==== osd_op_reply(129 100000001f9.00000000 [call,setxattr (8)] v110'25 uv25 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d68d400 2023-10-11T09:50:32.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.689+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:130 16.16 16:6fe38dc2::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ad400 con 0x557d3d68cc00 2023-10-11T09:50:32.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.689+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.3 v2:172.21.15.78:6824/35738 16 ==== osd_op_reply(130 [pgnls start_epoch 110 out=81b] v108'5 uv5 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557d3e093200 con 0x557d3d68cc00 2023-10-11T09:50:32.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.693+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:131 16.16 16:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ad800 con 0x557d3d68cc00 2023-10-11T09:50:32.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.693+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.3 v2:172.21.15.78:6824/35738 17 ==== osd_op_reply(131 100000001f7.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e093200 con 0x557d3d68cc00 2023-10-11T09:50:32.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.693+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628279.0 2023-10-11T09:50:32.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.693+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:132 15.36 15:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6acc00 con 0x557d3d68d400 2023-10-11T09:50:32.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.693+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.6 v2:172.21.15.133:6800/35900 24 ==== osd_op_reply(132 100000001f7.00000000 [call,setxattr (8)] v110'26 uv26 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d68d400 2023-10-11T09:50:32.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.693+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:133 16.e 16:70000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ad000 con 0x557d3d509400 2023-10-11T09:50:32.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.697+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.2 v2:172.21.15.78:6808/35732 26 ==== osd_op_reply(133 [pgnls start_epoch 110 out=101b] v108'3 uv3 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d509400 2023-10-11T09:50:32.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.697+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:134 16.e 16:72395991:::10000000001.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68dc00 con 0x557d3d509400 2023-10-11T09:50:32.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.697+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.2 v2:172.21.15.78:6808/35732 27 ==== osd_op_reply(134 10000000001.00000003 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d509400 2023-10-11T09:50:32.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.697+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511627777.3 2023-10-11T09:50:32.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.697+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:135 15.7 15:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68d800 con 0x557d3d576c00 2023-10-11T09:50:32.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.697+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 18 ==== osd_op_reply(135 10000000001.00000000 [call,setxattr (8)] v110'8 uv8 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.697+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:136 16.e 16:728aa123::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68c000 con 0x557d3d509400 2023-10-11T09:50:32.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.701+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.2 v2:172.21.15.78:6808/35732 28 ==== osd_op_reply(136 [pgnls start_epoch 110 out=101b] v108'3 uv3 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d509400 2023-10-11T09:50:32.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.701+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:137 16.e 16:728aa123:::100000001f6.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6c6000 con 0x557d3d509400 2023-10-11T09:50:32.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.701+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.2 v2:172.21.15.78:6808/35732 29 ==== osd_op_reply(137 100000001f6.00000003 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d509400 2023-10-11T09:50:32.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.701+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628278.3 2023-10-11T09:50:32.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.701+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:138 15.2a 15:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6c7800 con 0x557d3d68c800 2023-10-11T09:50:32.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.701+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.0 v2:172.21.15.78:6800/35735 20 ==== osd_op_reply(138 100000001f6.00000000 [call,setxattr (8)] v110'14 uv14 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e093440 con 0x557d3d68c800 2023-10-11T09:50:32.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.701+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:139 16.e 16:7780a862::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6c6c00 con 0x557d3d509400 2023-10-11T09:50:32.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.705+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.2 v2:172.21.15.78:6808/35732 30 ==== osd_op_reply(139 [pgnls start_epoch 110 out=81b] v108'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d509400 2023-10-11T09:50:32.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.705+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:140 16.e 16:7780a862:::100000001fa.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6c7000 con 0x557d3d509400 2023-10-11T09:50:32.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.705+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.2 v2:172.21.15.78:6808/35732 31 ==== osd_op_reply(140 100000001fa.00000003 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d509400 2023-10-11T09:50:32.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.705+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628282.3 2023-10-11T09:50:32.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.705+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:141 15.3f 15:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69a400 con 0x557d3d576c00 2023-10-11T09:50:32.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.705+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 19 ==== osd_op_reply(141 100000001fa.00000000 [call,setxattr (8)] v110'13 uv13 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.705+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:142 16.1e 16:78000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69b400 con 0x557d3d577400 2023-10-11T09:50:32.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.709+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.7 v2:172.21.15.133:6824/35901 10 ==== osd_op_reply(142 [pgnls start_epoch 110 out=101b] v108'2 uv2 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d577400 2023-10-11T09:50:32.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.709+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:143 16.1e 16:7e4d8c81:::10000000001.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69b800 con 0x557d3d577400 2023-10-11T09:50:32.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.709+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.7 v2:172.21.15.133:6824/35901 11 ==== osd_op_reply(143 10000000001.00000002 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d577400 2023-10-11T09:50:32.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.709+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511627777.2 2023-10-11T09:50:32.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.709+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:144 15.7 15:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69bc00 con 0x557d3d576c00 2023-10-11T09:50:32.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.709+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 20 ==== osd_op_reply(144 10000000001.00000000 [call,setxattr (8)] v110'9 uv9 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.709+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:145 16.1e 16:7e8133fb::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69a000 con 0x557d3d577400 2023-10-11T09:50:32.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.713+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.7 v2:172.21.15.133:6824/35901 12 ==== osd_op_reply(145 [pgnls start_epoch 110 out=81b] v108'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d577400 2023-10-11T09:50:32.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.713+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:146 16.1e 16:7e8133fb:::100000001fb.00000006:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68d000 con 0x557d3d577400 2023-10-11T09:50:32.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.713+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.7 v2:172.21.15.133:6824/35901 13 ==== osd_op_reply(146 100000001fb.00000006 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d577400 2023-10-11T09:50:32.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.713+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628283.6 2023-10-11T09:50:32.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.713+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:147 15.13 15:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68d800 con 0x557d3d576c00 2023-10-11T09:50:32.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.713+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 21 ==== osd_op_reply(147 100000001fb.00000000 [call,setxattr (8)] v110'20 uv20 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.713+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:148 16.1 16:80000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6adc00 con 0x557d3d509400 2023-10-11T09:50:32.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.717+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.2 v2:172.21.15.78:6808/35732 32 ==== osd_op_reply(148 [pgnls start_epoch 110 out=81b] v108'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d509400 2023-10-11T09:50:32.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.717+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:149 16.1 16:848ebb0b:::100000001f7.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ac400 con 0x557d3d509400 2023-10-11T09:50:32.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.717+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.2 v2:172.21.15.78:6808/35732 33 ==== osd_op_reply(149 100000001f7.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d509400 2023-10-11T09:50:32.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.717+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628279.1 2023-10-11T09:50:32.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.717+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:150 15.36 15:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ad400 con 0x557d3d68d400 2023-10-11T09:50:32.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.721+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.6 v2:172.21.15.133:6800/35900 25 ==== osd_op_reply(150 100000001f7.00000000 [call,setxattr (8)] v110'27 uv27 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d68d400 2023-10-11T09:50:32.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.721+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:151 16.11 16:88000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ad800 con 0x557d3d69a800 2023-10-11T09:50:32.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.721+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.5 v2:172.21.15.133:6816/35899 8 ==== osd_op_reply(151 [pgnls start_epoch 110 out=81b] v108'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557d3e093200 con 0x557d3d69a800 2023-10-11T09:50:32.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.721+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:152 16.11 16:893be98d:::100000001f8.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6c6000 con 0x557d3d69a800 2023-10-11T09:50:32.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.721+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.5 v2:172.21.15.133:6816/35899 9 ==== osd_op_reply(152 100000001f8.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e093200 con 0x557d3d69a800 2023-10-11T09:50:32.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.721+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628280.1 2023-10-11T09:50:32.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.721+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:153 15.19 15:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6c6c00 con 0x557d3d577800 2023-10-11T09:50:32.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.725+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.4 v2:172.21.15.133:6808/35898 15 ==== osd_op_reply(153 100000001f8.00000000 [call,setxattr (8)] v110'13 uv13 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e093440 con 0x557d3d577800 2023-10-11T09:50:32.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.725+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:154 16.9 16:90000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6c7000 con 0x557d3d576c00 2023-10-11T09:50:32.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.725+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 22 ==== osd_op_reply(154 [pgnls start_epoch 110 out=81b] v108'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.725+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:155 16.9 16:94f3e7b4:::100000001fa.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6c6400 con 0x557d3d576c00 2023-10-11T09:50:32.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.729+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 23 ==== osd_op_reply(155 100000001fa.00000002 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.729+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628282.2 2023-10-11T09:50:32.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.729+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:156 15.3f 15:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6c6800 con 0x557d3d576c00 2023-10-11T09:50:32.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.729+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 24 ==== osd_op_reply(156 100000001fa.00000000 [call,setxattr (8)] v110'14 uv14 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.729+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:157 16.19 16:98000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69ac00 con 0x557d3d577800 2023-10-11T09:50:32.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.733+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.4 v2:172.21.15.133:6808/35898 16 ==== osd_op_reply(157 [pgnls start_epoch 110 out=81b] v108'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557d3e093440 con 0x557d3d577800 2023-10-11T09:50:32.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.733+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:158 16.19 16:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69b000 con 0x557d3d577800 2023-10-11T09:50:32.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.733+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.4 v2:172.21.15.133:6808/35898 17 ==== osd_op_reply(158 100000001f8.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e093440 con 0x557d3d577800 2023-10-11T09:50:32.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.733+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628280.0 2023-10-11T09:50:32.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.733+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:159 15.19 15:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69b400 con 0x557d3d577800 2023-10-11T09:50:32.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.733+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.4 v2:172.21.15.133:6808/35898 18 ==== osd_op_reply(159 100000001f8.00000000 [call,setxattr (8)] v110'14 uv14 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e093440 con 0x557d3d577800 2023-10-11T09:50:32.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.737+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:160 16.5 16:a0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68dc00 con 0x557d3d68c800 2023-10-11T09:50:32.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.737+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.0 v2:172.21.15.78:6800/35735 21 ==== osd_op_reply(160 [pgnls start_epoch 110 out=81b] v108'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557d3e093440 con 0x557d3d68c800 2023-10-11T09:50:32.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.737+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:161 16.5 16:a238cdfc:::100000001f9.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68c000 con 0x557d3d68c800 2023-10-11T09:50:32.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.737+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.0 v2:172.21.15.78:6800/35735 22 ==== osd_op_reply(161 100000001f9.00000002 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e093440 con 0x557d3d68c800 2023-10-11T09:50:32.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.737+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628281.2 2023-10-11T09:50:32.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.737+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:162 15.36 15:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ac000 con 0x557d3d68d400 2023-10-11T09:50:32.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.741+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.6 v2:172.21.15.133:6800/35900 26 ==== osd_op_reply(162 100000001f9.00000000 [call,setxattr (8)] v110'28 uv28 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d68d400 2023-10-11T09:50:32.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.741+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:163 16.15 16:a8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6acc00 con 0x557d3d68c800 2023-10-11T09:50:32.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.741+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.0 v2:172.21.15.78:6800/35735 23 ==== osd_op_reply(163 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e093440 con 0x557d3d68c800 2023-10-11T09:50:32.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.741+0000 7f9ff52bb700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:164 16.d 16:b0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d576400 con 0x557d3d68d400 2023-10-11T09:50:32.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.741+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.6 v2:172.21.15.133:6800/35900 27 ==== osd_op_reply(164 [pgnls start_epoch 110 out=81b] v108'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d68d400 2023-10-11T09:50:32.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.741+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:165 16.d 16:b0a6c804:::100000001fc.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ad000 con 0x557d3d68d400 2023-10-11T09:50:32.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.741+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.6 v2:172.21.15.133:6800/35900 28 ==== osd_op_reply(165 100000001fc.00000002 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d68d400 2023-10-11T09:50:32.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.741+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628284.2 2023-10-11T09:50:32.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.741+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:166 15.c 15:30604137:::100000001fc.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69ac00 con 0x557d3d69a800 2023-10-11T09:50:32.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.745+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.5 v2:172.21.15.133:6816/35899 10 ==== osd_op_reply(166 100000001fc.00000000 [call,setxattr (8)] v110'10 uv10 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e093200 con 0x557d3d69a800 2023-10-11T09:50:32.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.745+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:167 16.1d 16:b8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69b000 con 0x557d3d509400 2023-10-11T09:50:32.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.745+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.2 v2:172.21.15.78:6808/35732 34 ==== osd_op_reply(167 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d509400 2023-10-11T09:50:32.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.745+0000 7f9ff52bb700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:168 16.3 16:c0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6c8000 con 0x557d3d69a800 2023-10-11T09:50:32.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.749+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.5 v2:172.21.15.133:6816/35899 11 ==== osd_op_reply(168 [pgnls start_epoch 110 out=101b] v108'2 uv2 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x557d3e093200 con 0x557d3d69a800 2023-10-11T09:50:32.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.749+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:169 16.3 16:c0957981:::100000001fa.00000005:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69b400 con 0x557d3d69a800 2023-10-11T09:50:32.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.749+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.5 v2:172.21.15.133:6816/35899 12 ==== osd_op_reply(169 100000001fa.00000005 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e093200 con 0x557d3d69a800 2023-10-11T09:50:32.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.749+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628282.5 2023-10-11T09:50:32.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.749+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:170 15.3f 15:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6c7400 con 0x557d3d576c00 2023-10-11T09:50:32.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.753+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 25 ==== osd_op_reply(170 100000001fa.00000000 [call,setxattr (8)] v110'15 uv15 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.753+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:171 16.3 16:c5d8c51d::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6c7c00 con 0x557d3d69a800 2023-10-11T09:50:32.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.753+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.5 v2:172.21.15.133:6816/35899 13 ==== osd_op_reply(171 [pgnls start_epoch 110 out=81b] v108'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557d3e093200 con 0x557d3d69a800 2023-10-11T09:50:32.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.753+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:172 16.3 16:c5d8c51d:::100000001fb.00000004:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6c7800 con 0x557d3d69a800 2023-10-11T09:50:32.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.753+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.5 v2:172.21.15.133:6816/35899 14 ==== osd_op_reply(172 100000001fb.00000004 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e093200 con 0x557d3d69a800 2023-10-11T09:50:32.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.753+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628283.4 2023-10-11T09:50:32.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.753+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:173 15.13 15:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6c6c00 con 0x557d3d576c00 2023-10-11T09:50:32.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.757+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 26 ==== osd_op_reply(173 100000001fb.00000000 [call,setxattr (8)] v110'21 uv21 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.757+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:174 16.13 16:c8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ac800 con 0x557d3d576c00 2023-10-11T09:50:32.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.757+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 27 ==== osd_op_reply(174 [pgnls start_epoch 110 out=101b] v108'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.757+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:175 16.13 16:c9e7fe79:::100000001f9.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d577000 con 0x557d3d576c00 2023-10-11T09:50:32.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.757+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 28 ==== osd_op_reply(175 100000001f9.00000003 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.757+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628281.3 2023-10-11T09:50:32.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.757+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:176 15.36 15:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6c8000 con 0x557d3d68d400 2023-10-11T09:50:32.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.761+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.6 v2:172.21.15.133:6800/35900 29 ==== osd_op_reply(176 100000001f9.00000000 [call,setxattr (8)] v110'29 uv29 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d68d400 2023-10-11T09:50:32.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.761+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:177 16.13 16:ca8fdbba::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69a400 con 0x557d3d576c00 2023-10-11T09:50:32.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.761+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 29 ==== osd_op_reply(177 [pgnls start_epoch 110 out=101b] v108'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.761+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:178 16.13 16:ca8fdbba:::100000001fb.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69bc00 con 0x557d3d576c00 2023-10-11T09:50:32.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.761+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 30 ==== osd_op_reply(178 100000001fb.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.761+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628283.0 2023-10-11T09:50:32.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.761+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:179 15.13 15:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69ac00 con 0x557d3d576c00 2023-10-11T09:50:32.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.765+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 31 ==== osd_op_reply(179 100000001fb.00000000 [call,setxattr (8)] v110'22 uv22 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.765+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:180 16.13 16:cb24bee2::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6c6000 con 0x557d3d576c00 2023-10-11T09:50:32.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.765+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 32 ==== osd_op_reply(180 [pgnls start_epoch 110 out=81b] v108'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.765+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:181 16.13 16:cb24bee2:::100000001fa.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6c7000 con 0x557d3d576c00 2023-10-11T09:50:32.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.765+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 33 ==== osd_op_reply(181 100000001fa.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.765+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628282.1 2023-10-11T09:50:32.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.765+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:182 15.3f 15:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6c6400 con 0x557d3d576c00 2023-10-11T09:50:32.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.769+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 34 ==== osd_op_reply(182 100000001fa.00000000 [call,setxattr (8)] v110'16 uv16 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.769+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:183 16.b 16:d0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6c6800 con 0x557d3d576c00 2023-10-11T09:50:32.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.769+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 35 ==== osd_op_reply(183 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.769+0000 7f9ff52bb700 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:184 16.1b 16:d8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6c8400 con 0x557d3d69a800 2023-10-11T09:50:32.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.769+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.5 v2:172.21.15.133:6816/35899 15 ==== osd_op_reply(184 [pgnls start_epoch 110 out=81b] v108'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557d3e093200 con 0x557d3d69a800 2023-10-11T09:50:32.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.769+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:185 16.1b 16:de1cf3c0:::100000001fb.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ad800 con 0x557d3d69a800 2023-10-11T09:50:32.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.769+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.5 v2:172.21.15.133:6816/35899 16 ==== osd_op_reply(185 100000001fb.00000003 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e093200 con 0x557d3d69a800 2023-10-11T09:50:32.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.769+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628283.3 2023-10-11T09:50:32.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.769+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:186 15.13 15:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68d800 con 0x557d3d576c00 2023-10-11T09:50:32.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.773+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 36 ==== osd_op_reply(186 100000001fb.00000000 [call,setxattr (8)] v110'23 uv23 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.773+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:187 16.7 16:e0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6c7400 con 0x557d3d68cc00 2023-10-11T09:50:32.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.773+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.3 v2:172.21.15.78:6824/35738 18 ==== osd_op_reply(187 [pgnls start_epoch 110 out=101b] v108'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x557d3e093200 con 0x557d3d68cc00 2023-10-11T09:50:32.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.773+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:188 16.7 16:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6c7c00 con 0x557d3d68cc00 2023-10-11T09:50:32.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.773+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.3 v2:172.21.15.78:6824/35738 19 ==== osd_op_reply(188 10000000001.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e093200 con 0x557d3d68cc00 2023-10-11T09:50:32.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.773+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511627777.0 2023-10-11T09:50:32.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.773+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:189 15.7 15:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6c7800 con 0x557d3d576c00 2023-10-11T09:50:32.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.777+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 37 ==== osd_op_reply(189 10000000001.00000000 [call,setxattr (8)] v110'10 uv10 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.777+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:190 16.7 16:e7ecf421::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6c6c00 con 0x557d3d68cc00 2023-10-11T09:50:32.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.777+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.3 v2:172.21.15.78:6824/35738 20 ==== osd_op_reply(190 [pgnls start_epoch 110 out=81b] v108'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557d3e093200 con 0x557d3d68cc00 2023-10-11T09:50:32.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.777+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:191 16.7 16:e7ecf421:::100000001f9.00000004:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6c6000 con 0x557d3d68cc00 2023-10-11T09:50:32.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.777+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.3 v2:172.21.15.78:6824/35738 21 ==== osd_op_reply(191 100000001f9.00000004 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e093200 con 0x557d3d68cc00 2023-10-11T09:50:32.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.777+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628281.4 2023-10-11T09:50:32.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.777+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:192 15.36 15:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69b400 con 0x557d3d68d400 2023-10-11T09:50:32.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.781+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.6 v2:172.21.15.133:6800/35900 30 ==== osd_op_reply(192 100000001f9.00000000 [call,setxattr (8)] v110'30 uv30 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d68d400 2023-10-11T09:50:32.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.781+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:193 16.17 16:e8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69a400 con 0x557d3d509400 2023-10-11T09:50:32.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.781+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.2 v2:172.21.15.78:6808/35732 35 ==== osd_op_reply(193 [pgnls start_epoch 110 out=101b] v108'2 uv2 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d509400 2023-10-11T09:50:32.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.781+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:194 16.17 16:ee7e3069:::100000001f9.00000007:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69bc00 con 0x557d3d509400 2023-10-11T09:50:32.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.781+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.2 v2:172.21.15.78:6808/35732 36 ==== osd_op_reply(194 100000001f9.00000007 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d509400 2023-10-11T09:50:32.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.781+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628281.7 2023-10-11T09:50:32.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.781+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:195 15.36 15:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69ac00 con 0x557d3d68d400 2023-10-11T09:50:32.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.785+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.6 v2:172.21.15.133:6800/35900 31 ==== osd_op_reply(195 100000001f9.00000000 [call,setxattr (8)] v110'31 uv31 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d68d400 2023-10-11T09:50:32.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.785+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:196 16.17 16:eff9002a::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d577000 con 0x557d3d509400 2023-10-11T09:50:32.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.785+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.2 v2:172.21.15.78:6808/35732 37 ==== osd_op_reply(196 [pgnls start_epoch 110 out=81b] v108'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d509400 2023-10-11T09:50:32.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.785+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:197 16.17 16:eff9002a:::100000001f8.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ac800 con 0x557d3d509400 2023-10-11T09:50:32.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.785+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 <== osd.2 v2:172.21.15.78:6808/35732 38 ==== osd_op_reply(197 100000001f8.00000003 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e092fc0 con 0x557d3d509400 2023-10-11T09:50:32.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.785+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628280.3 2023-10-11T09:50:32.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.785+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:198 15.19 15:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69b800 con 0x557d3d577800 2023-10-11T09:50:32.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.789+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.4 v2:172.21.15.133:6808/35898 19 ==== osd_op_reply(198 100000001f8.00000000 [call,setxattr (8)] v110'15 uv15 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e093440 con 0x557d3d577800 2023-10-11T09:50:32.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.789+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:199 16.f 16:f0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69a000 con 0x557d3d68c800 2023-10-11T09:50:32.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.789+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.0 v2:172.21.15.78:6800/35735 24 ==== osd_op_reply(199 [pgnls start_epoch 110 out=101b] v108'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x557d3e093440 con 0x557d3d68c800 2023-10-11T09:50:32.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.789+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:200 16.f 16:f26fdfa9:::100000001f8.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69b000 con 0x557d3d68c800 2023-10-11T09:50:32.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.793+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.0 v2:172.21.15.78:6800/35735 25 ==== osd_op_reply(200 100000001f8.00000002 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e093440 con 0x557d3d68c800 2023-10-11T09:50:32.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.793+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628280.2 2023-10-11T09:50:32.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.793+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:201 15.19 15:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d69b400 con 0x557d3d577800 2023-10-11T09:50:32.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.793+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.4 v2:172.21.15.133:6808/35898 20 ==== osd_op_reply(201 100000001f8.00000000 [call,setxattr (8)] v110'16 uv16 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e093440 con 0x557d3d577800 2023-10-11T09:50:32.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.797+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:202 16.f 16:f376f0dc::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6c8000 con 0x557d3d68c800 2023-10-11T09:50:32.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.797+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.0 v2:172.21.15.78:6800/35735 26 ==== osd_op_reply(202 [pgnls start_epoch 110 out=101b] v108'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x557d3e093440 con 0x557d3d68c800 2023-10-11T09:50:32.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.797+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:203 16.f 16:f376f0dc:::100000001fb.00000005:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6adc00 con 0x557d3d68c800 2023-10-11T09:50:32.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.797+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.0 v2:172.21.15.78:6800/35735 27 ==== osd_op_reply(203 100000001fb.00000005 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e093440 con 0x557d3d68c800 2023-10-11T09:50:32.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.797+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628283.5 2023-10-11T09:50:32.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.797+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:204 15.13 15:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ac400 con 0x557d3d576c00 2023-10-11T09:50:32.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.797+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 38 ==== osd_op_reply(204 100000001fb.00000000 [call,setxattr (8)] v110'24 uv24 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.801+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:205 16.f 16:f526278d::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ad400 con 0x557d3d68c800 2023-10-11T09:50:32.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.801+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.0 v2:172.21.15.78:6800/35735 28 ==== osd_op_reply(205 [pgnls start_epoch 110 out=101b] v108'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x557d3e093440 con 0x557d3d68c800 2023-10-11T09:50:32.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.801+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:206 16.f 16:f526278d:::10000000001.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ac000 con 0x557d3d68c800 2023-10-11T09:50:32.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.801+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.0 v2:172.21.15.78:6800/35735 29 ==== osd_op_reply(206 10000000001.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e093440 con 0x557d3d68c800 2023-10-11T09:50:32.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.801+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511627777.1 2023-10-11T09:50:32.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.801+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:207 15.7 15:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ad000 con 0x557d3d576c00 2023-10-11T09:50:32.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.801+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 39 ==== osd_op_reply(207 10000000001.00000000 [call,setxattr (8)] v110'11 uv11 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.801+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:208 16.f 16:f776b802::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d576400 con 0x557d3d68c800 2023-10-11T09:50:32.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.805+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.0 v2:172.21.15.78:6800/35735 30 ==== osd_op_reply(208 [pgnls start_epoch 110 out=81b] v108'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557d3e093440 con 0x557d3d68c800 2023-10-11T09:50:32.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.805+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:209 16.f 16:f776b802:::100000001fb.00000007:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68d000 con 0x557d3d68c800 2023-10-11T09:50:32.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.805+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.0 v2:172.21.15.78:6800/35735 31 ==== osd_op_reply(209 100000001fb.00000007 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e093440 con 0x557d3d68c800 2023-10-11T09:50:32.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.805+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628283.7 2023-10-11T09:50:32.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.805+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:210 15.13 15:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6acc00 con 0x557d3d576c00 2023-10-11T09:50:32.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.805+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 40 ==== osd_op_reply(210 100000001fb.00000000 [call,setxattr (8)] v110'25 uv25 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.805+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:211 16.1f 16:f8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6adc00 con 0x557d3d69a800 2023-10-11T09:50:32.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.809+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.5 v2:172.21.15.133:6816/35899 17 ==== osd_op_reply(211 [pgnls start_epoch 110 out=101b] v108'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x557d3e093200 con 0x557d3d69a800 2023-10-11T09:50:32.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.809+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:212 16.1f 16:faedc8eb:::100000001f6.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ad400 con 0x557d3d69a800 2023-10-11T09:50:32.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.809+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.5 v2:172.21.15.133:6816/35899 18 ==== osd_op_reply(212 100000001f6.00000002 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e093200 con 0x557d3d69a800 2023-10-11T09:50:32.818 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.809+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628278.2 2023-10-11T09:50:32.818 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.809+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:213 15.2a 15:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ad800 con 0x557d3d68c800 2023-10-11T09:50:32.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.813+0000 7f9ffa2c5700 1 -- 172.21.15.78:0/1569352335 <== osd.0 v2:172.21.15.78:6800/35735 32 ==== osd_op_reply(213 100000001f6.00000000 [call,setxattr (8)] v110'15 uv15 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e093440 con 0x557d3d68c800 2023-10-11T09:50:32.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.813+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:214 16.1f 16:fb032386::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6ac400 con 0x557d3d69a800 2023-10-11T09:50:32.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.813+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.5 v2:172.21.15.133:6816/35899 19 ==== osd_op_reply(214 [pgnls start_epoch 110 out=101b] v108'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x557d3e093200 con 0x557d3d69a800 2023-10-11T09:50:32.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.813+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:215 16.1f 16:fb032386:::100000001fa.00000007:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68dc00 con 0x557d3d69a800 2023-10-11T09:50:32.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.813+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.5 v2:172.21.15.133:6816/35899 20 ==== osd_op_reply(215 100000001fa.00000007 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e093200 con 0x557d3d69a800 2023-10-11T09:50:32.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.813+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628282.7 2023-10-11T09:50:32.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.813+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:216 15.3f 15:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68c000 con 0x557d3d576c00 2023-10-11T09:50:32.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.817+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 41 ==== osd_op_reply(216 100000001fa.00000000 [call,setxattr (8)] v110'17 uv17 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.817+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:217 16.1f 16:feb1c803::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d68d800 con 0x557d3d69a800 2023-10-11T09:50:32.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.817+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.5 v2:172.21.15.133:6816/35899 21 ==== osd_op_reply(217 [pgnls start_epoch 110 out=81b] v108'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557d3e093200 con 0x557d3d69a800 2023-10-11T09:50:32.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.817+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:218 16.1f 16:feb1c803:::100000001fa.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6c8000 con 0x557d3d69a800 2023-10-11T09:50:32.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.817+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.5 v2:172.21.15.133:6816/35899 22 ==== osd_op_reply(218 100000001fa.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x557d3e093200 con 0x557d3d69a800 2023-10-11T09:50:32.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.817+0000 7f9ffbb09840 10 datascan.operator(): handling object 1099511628282.0 2023-10-11T09:50:32.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.817+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:219 15.3f 15:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x557d3d6c8400 con 0x557d3d576c00 2023-10-11T09:50:32.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.821+0000 7f9ffaac6700 1 -- 172.21.15.78:0/1569352335 <== osd.1 v2:172.21.15.78:6809/35734 42 ==== osd_op_reply(219 100000001fa.00000000 [call,setxattr (8)] v110'18 uv18 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557d3e093200 con 0x557d3d576c00 2023-10-11T09:50:32.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.821+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x557d3d68c800 msgr2=0x557d3d677080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.821+0000 7f9ffbb09840 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x557d3d68c800 0x557d3d677080 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.821+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x557d3d576c00 msgr2=0x557d3d676000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.821+0000 7f9ffbb09840 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x557d3d576c00 0x557d3d676000 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.821+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x557d3d509400 msgr2=0x557d3d2ba100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.821+0000 7f9ffbb09840 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x557d3d509400 0x557d3d2ba100 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.821+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x557d3d68cc00 msgr2=0x557d3d677600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.821+0000 7f9ffbb09840 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x557d3d68cc00 0x557d3d677600 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.821+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x557d3d577800 msgr2=0x557d3d676580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.821+0000 7f9ffbb09840 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x557d3d577800 0x557d3d676580 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.821+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x557d3d69a800 msgr2=0x557d3d677b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.821+0000 7f9ffbb09840 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x557d3d69a800 0x557d3d677b80 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.821+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 reap_dead start 2023-10-11T09:50:32.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.821+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x557d3d68d400 msgr2=0x557d3d678100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.821+0000 7f9ffbb09840 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x557d3d68d400 0x557d3d678100 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.821+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x557d3d577400 msgr2=0x557d3d676b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.821+0000 7f9ffbb09840 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x557d3d577400 0x557d3d676b00 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.821+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 reap_dead start 2023-10-11T09:50:32.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.821+0000 7f9ff9ac4700 1 -- 172.21.15.78:0/1569352335 reap_dead start 2023-10-11T09:50:32.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.821+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x557d3d576800 msgr2=0x557d3d2bb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.821+0000 7f9ffbb09840 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x557d3d576800 0x557d3d2bb700 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x557d3dff68a0 tx=0x557d3e004de0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.821+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d3d508c00 msgr2=0x557d3d2bac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.821+0000 7f9ffbb09840 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d3d508c00 0x557d3d2bac00 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x557d3dff74a0 tx=0x557d3e004ea0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.821+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 shutdown_connections 2023-10-11T09:50:32.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.821+0000 7f9ffbb09840 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x557d3d576800 0x557d3d2bb700 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.821+0000 7f9ffbb09840 1 --2- 172.21.15.78:0/1569352335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d3d508c00 0x557d3d2bac00 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.821+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 >> 172.21.15.78:0/1569352335 conn(0x557d3d212c00 msgr2=0x557d3d2e1800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:32.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.825+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 shutdown_connections 2023-10-11T09:50:32.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.825+0000 7f9ffbb09840 1 -- 172.21.15.78:0/1569352335 wait complete. 2023-10-11T09:50:32.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.825+0000 7f9ffbb09840 1 -- 172.21.15.78:0/488040104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d3d213800 msgr2=0x557d3d2b8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.825+0000 7f9ffbb09840 1 --2- 172.21.15.78:0/488040104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d3d213800 0x557d3d2b8580 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x557d3df7b3e0 tx=0x557d3e004c60 comp rx=0 tx=0).stop 2023-10-11T09:50:32.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.825+0000 7f9ffbb09840 1 -- 172.21.15.78:0/488040104 shutdown_connections 2023-10-11T09:50:32.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.825+0000 7f9ffbb09840 1 --2- 172.21.15.78:0/488040104 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x557d3d213000 0x557d3d2b9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.825+0000 7f9ffbb09840 1 --2- 172.21.15.78:0/488040104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d3d213400 0x557d3d2b8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.825+0000 7f9ffbb09840 1 --2- 172.21.15.78:0/488040104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d3d213800 0x557d3d2b8580 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.825+0000 7f9ffbb09840 1 -- 172.21.15.78:0/488040104 >> 172.21.15.78:0/488040104 conn(0x557d3d213c00 msgr2=0x557d3d2e1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:32.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.825+0000 7f9ffbb09840 1 -- 172.21.15.78:0/488040104 shutdown_connections 2023-10-11T09:50:32.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.825+0000 7f9ffbb09840 1 -- 172.21.15.78:0/488040104 wait complete. 2023-10-11T09:50:32.841 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents'] in time 0:00:00.395898, result: 2023-10-11T09:50:32.841 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes 2023-10-11T09:50:32.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.889+0000 7f9b5d790840 1 -- start start 2023-10-11T09:50:32.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.889+0000 7f9b5d790840 1 Processor -- start 2023-10-11T09:50:32.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.893+0000 7f9b5d790840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55bcdb669800 0x55bcdb70e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.893+0000 7f9b5d790840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcdb669400 0x55bcdb70eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.893+0000 7f9b5d790840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcdb669000 0x55bcdb70f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.893+0000 7f9b5d790840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdb6e8a00 con 0x55bcdb669000 2023-10-11T09:50:32.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.893+0000 7f9b5d790840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdb6e8780 con 0x55bcdb669400 2023-10-11T09:50:32.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.893+0000 7f9b5d790840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bcdb6e8640 con 0x55bcdb669800 2023-10-11T09:50:32.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.893+0000 7f9b5c74d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcdb669000 0x55bcdb70f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.893+0000 7f9b5c74d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcdb669000 0x55bcdb70f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36120/0 (socket says 172.21.15.78:36120) 2023-10-11T09:50:32.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.893+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2902393064 learned_addr learned my addr 172.21.15.78:0/2902393064 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:32.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.893+0000 7f9b5b74b700 1 --2- 172.21.15.78:0/2902393064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcdb669400 0x55bcdb70eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.893+0000 7f9b5bf4c700 1 --2- 172.21.15.78:0/2902393064 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55bcdb669800 0x55bcdb70e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.893+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2902393064 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55bcdb669800 msgr2=0x55bcdb70e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:32.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.893+0000 7f9b5c74d700 1 --2- 172.21.15.78:0/2902393064 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55bcdb669800 0x55bcdb70e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.893+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2902393064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcdb669400 msgr2=0x55bcdb70eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:32.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.893+0000 7f9b5c74d700 1 --2- 172.21.15.78:0/2902393064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcdb669400 0x55bcdb70eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.893+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2902393064 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bcdb68ab60 con 0x55bcdb669000 2023-10-11T09:50:32.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.893+0000 7f9b5bf4c700 1 --2- 172.21.15.78:0/2902393064 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55bcdb669800 0x55bcdb70e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:50:32.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.893+0000 7f9b5c74d700 1 --2- 172.21.15.78:0/2902393064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcdb669000 0x55bcdb70f080 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x55bcdc3d1410 tx=0x55bcdc4e10e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7714bfad8a16f30b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.893+0000 7f9b59747700 1 -- 172.21.15.78:0/2902393064 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55bcdb699600 con 0x55bcdb669000 2023-10-11T09:50:32.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.893+0000 7f9b5d790840 1 -- 172.21.15.78:0/2902393064 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bcdb68a340 con 0x55bcdb669000 2023-10-11T09:50:32.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.893+0000 7f9b59747700 1 -- 172.21.15.78:0/2902393064 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bcdb6b4480 con 0x55bcdb669000 2023-10-11T09:50:32.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.893+0000 7f9b5d790840 1 -- 172.21.15.78:0/2902393064 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55bcdb68a000 con 0x55bcdb669000 2023-10-11T09:50:32.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.893+0000 7f9b5d790840 4 waiting for MDS map... 2023-10-11T09:50:32.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.893+0000 7f9b59747700 1 -- 172.21.15.78:0/2902393064 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55bcdb6991e0 con 0x55bcdb669000 2023-10-11T09:50:32.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.893+0000 7f9b59747700 1 -- 172.21.15.78:0/2902393064 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(110..110 src has 1..110) v4 ==== 6638+0+0 (secure 0 0 0) 0x55bcdb6a2c40 con 0x55bcdb669000 2023-10-11T09:50:32.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.893+0000 7f9b59747700 1 -- 172.21.15.78:0/2902393064 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 85) v1 ==== 3118+0+0 (secure 0 0 0) 0x55bcdb95b200 con 0x55bcdb669000 2023-10-11T09:50:32.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.893+0000 7f9b5d790840 4 Got MDS map 85 2023-10-11T09:50:32.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.893+0000 7f9b5d790840 4 datascan.main: Using metadata pool output 2023-10-11T09:50:32.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.893+0000 7f9b5d790840 4 datascan.main: connecting to RADOS... 2023-10-11T09:50:32.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.893+0000 7f9b5d790840 1 Processor -- start 2023-10-11T09:50:32.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.897+0000 7f9b5d790840 1 -- start start 2023-10-11T09:50:32.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.897+0000 7f9b5d790840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcdb95e400 0x55bcdb70f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.897+0000 7f9b5d790840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55bcdb95e800 0x55bcdb70fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.897+0000 7f9b5d790840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcdb95ec00 0x55bcdb710100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.897+0000 7f9b5d790840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdb69fe00 con 0x55bcdb95ec00 2023-10-11T09:50:32.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.897+0000 7f9b5d790840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdb6e8780 con 0x55bcdb95e400 2023-10-11T09:50:32.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.897+0000 7f9b5d790840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bcdb6e8500 con 0x55bcdb95e800 2023-10-11T09:50:32.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.897+0000 7f9b5b74b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcdb95ec00 0x55bcdb710100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.897+0000 7f9b5b74b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcdb95ec00 0x55bcdb710100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36132/0 (socket says 172.21.15.78:36132) 2023-10-11T09:50:32.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.897+0000 7f9b5b74b700 1 -- 172.21.15.78:0/3756247521 learned_addr learned my addr 172.21.15.78:0/3756247521 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:32.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.897+0000 7f9b5b74b700 1 --2- 172.21.15.78:0/3756247521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcdb95e400 0x55bcdb70f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.897+0000 7f9b5bf4c700 1 --2- 172.21.15.78:0/3756247521 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55bcdb95e800 0x55bcdb70fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.897+0000 7f9b5b74b700 1 -- 172.21.15.78:0/3756247521 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55bcdb95e800 msgr2=0x55bcdb70fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:32.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.897+0000 7f9b5b74b700 1 --2- 172.21.15.78:0/3756247521 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55bcdb95e800 0x55bcdb70fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.897+0000 7f9b5b74b700 1 -- 172.21.15.78:0/3756247521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcdb95ec00 msgr2=0x55bcdb710100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:32.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.897+0000 7f9b5b74b700 1 --2- 172.21.15.78:0/3756247521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcdb95ec00 0x55bcdb710100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.897+0000 7f9b5b74b700 1 -- 172.21.15.78:0/3756247521 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bcdb68aea0 con 0x55bcdb95e400 2023-10-11T09:50:32.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.901+0000 7f9b5b74b700 1 --2- 172.21.15.78:0/3756247521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcdb95e400 0x55bcdb70f600 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x55bcdb95c840 tx=0x55bcdb956de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cea4c9a60ed5a070 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.901+0000 7f9b57743700 1 -- 172.21.15.78:0/3756247521 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55bcdb699080 con 0x55bcdb95e400 2023-10-11T09:50:32.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.901+0000 7f9b57743700 1 -- 172.21.15.78:0/3756247521 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bcdb6b4480 con 0x55bcdb95e400 2023-10-11T09:50:32.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.901+0000 7f9b57743700 1 -- 172.21.15.78:0/3756247521 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55bcdb699600 con 0x55bcdb95e400 2023-10-11T09:50:32.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.901+0000 7f9b5d790840 1 -- 172.21.15.78:0/3756247521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcdb95e400 msgr2=0x55bcdb70f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.901+0000 7f9b5d790840 1 --2- 172.21.15.78:0/3756247521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcdb95e400 0x55bcdb70f600 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x55bcdb95c840 tx=0x55bcdb956de0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.901+0000 7f9b5d790840 1 -- 172.21.15.78:0/3756247521 shutdown_connections 2023-10-11T09:50:32.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.901+0000 7f9b5d790840 1 --2- 172.21.15.78:0/3756247521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcdb95ec00 0x55bcdb710100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.901+0000 7f9b5d790840 1 --2- 172.21.15.78:0/3756247521 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55bcdb95e800 0x55bcdb70fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.901+0000 7f9b5d790840 1 --2- 172.21.15.78:0/3756247521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcdb95e400 0x55bcdb70f600 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.901+0000 7f9b5d790840 1 -- 172.21.15.78:0/3756247521 >> 172.21.15.78:0/3756247521 conn(0x55bcdb668c00 msgr2=0x55bcdb737800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:32.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.901+0000 7f9b5d790840 1 -- 172.21.15.78:0/3756247521 shutdown_connections 2023-10-11T09:50:32.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.901+0000 7f9b5d790840 1 -- 172.21.15.78:0/3756247521 wait complete. 2023-10-11T09:50:32.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.901+0000 7f9b5d790840 1 Processor -- start 2023-10-11T09:50:32.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.901+0000 7f9b5d790840 1 -- start start 2023-10-11T09:50:32.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.905+0000 7f9b5d790840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcdb95e400 0x55bcdb711180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.905+0000 7f9b5d790840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcdb95e800 0x55bcdb710c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.905+0000 7f9b5d790840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55bcdb95ec00 0x55bcdb710680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.905+0000 7f9b5d790840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdb6e83c0 con 0x55bcdb95e400 2023-10-11T09:50:32.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.905+0000 7f9b5d790840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdb6e8280 con 0x55bcdb95e800 2023-10-11T09:50:32.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.905+0000 7f9b5d790840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bcdb6e8140 con 0x55bcdb95ec00 2023-10-11T09:50:32.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.905+0000 7f9b5b74b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcdb95e400 0x55bcdb711180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.905+0000 7f9b5b74b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcdb95e400 0x55bcdb711180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36144/0 (socket says 172.21.15.78:36144) 2023-10-11T09:50:32.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.905+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 learned_addr learned my addr 172.21.15.78:0/2738478221 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:32.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.905+0000 7f9b5bf4c700 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcdb95e800 0x55bcdb710c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.905+0000 7f9b5b74b700 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55bcdb95ec00 0x55bcdb710680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.905+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55bcdb95ec00 msgr2=0x55bcdb710680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.905+0000 7f9b5b74b700 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55bcdb95ec00 0x55bcdb710680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.905+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcdb95e800 msgr2=0x55bcdb710c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:32.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.905+0000 7f9b5b74b700 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcdb95e800 0x55bcdb710c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:32.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.905+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bcdb68b1e0 con 0x55bcdb95e400 2023-10-11T09:50:32.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.905+0000 7f9b5b74b700 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcdb95e400 0x55bcdb711180 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x55bcdb95d500 tx=0x55bcdb956c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.905+0000 7f9b55f40700 1 -- 172.21.15.78:0/2738478221 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55bcdb6991e0 con 0x55bcdb95e400 2023-10-11T09:50:32.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.905+0000 7f9b55f40700 1 -- 172.21.15.78:0/2738478221 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bcdb75e300 con 0x55bcdb95e400 2023-10-11T09:50:32.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.905+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bcdb68aea0 con 0x55bcdb95e400 2023-10-11T09:50:32.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.905+0000 7f9b55f40700 1 -- 172.21.15.78:0/2738478221 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55bcdb698dc0 con 0x55bcdb95e400 2023-10-11T09:50:32.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.905+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bcdb68ab60 con 0x55bcdb95e400 2023-10-11T09:50:32.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.905+0000 7f9b5d790840 4 datascan.init: resolving metadata pool 14 2023-10-11T09:50:32.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.909+0000 7f9b55f40700 1 -- 172.21.15.78:0/2738478221 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x55bcdb6c6840 con 0x55bcdb95e400 2023-10-11T09:50:32.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.909+0000 7f9b55f40700 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55bcdbabe000 0x55bcdb711700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.909+0000 7f9b55f40700 1 -- 172.21.15.78:0/2738478221 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6638+0+0 (secure 0 0 0) 0x55bcdb6a2a80 con 0x55bcdb95e400 2023-10-11T09:50:32.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.909+0000 7f9b5d790840 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-10-11T09:50:32.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.909+0000 7f9b5d790840 4 datascan.main: data pool 'cephfs_data' has ID 15 2023-10-11T09:50:32.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.909+0000 7f9b5d790840 4 datascan.main: opening data pool 'cephfs_data' 2023-10-11T09:50:32.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.909+0000 7f9b5d790840 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55bcdb95f000 0x55bcdb710100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.909+0000 7f9b5bf4c700 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55bcdbabe000 0x55bcdb711700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.909+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:1 14.14 14:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdb95f400 con 0x55bcdb95f000 2023-10-11T09:50:32.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.909+0000 7f9b5b74b700 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55bcdb95f000 0x55bcdb710100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.909+0000 7f9b5bf4c700 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55bcdbabe000 0x55bcdb711700 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x55bcdc3d06f0 tx=0x55bcdb957d40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.909+0000 7f9b5b74b700 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55bcdb95f000 0x55bcdb710100 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.909+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.4 v2:172.21.15.133:6808/35898 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv52 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdb95f000 2023-10-11T09:50:32.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.909+0000 7f9b5d790840 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55bcdb95f800 0x55bcdb70fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.909+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:2 14.1f 14:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabe400 con 0x55bcdb95f800 2023-10-11T09:50:32.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.909+0000 7f9b5c74d700 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55bcdb95f800 0x55bcdb70fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.913+0000 7f9b5c74d700 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55bcdb95f800 0x55bcdb70fb80 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.913+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.0 v2:172.21.15.78:6800/35735 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdb95f800 2023-10-11T09:50:32.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.913+0000 7f9b5d790840 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55bcdbabe800 0x55bcdb70f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.913+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:3 15.0 15:00000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabec00 con 0x55bcdbabe800 2023-10-11T09:50:32.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.913+0000 7f9b5bf4c700 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55bcdbabe800 0x55bcdb70f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.913+0000 7f9b5bf4c700 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55bcdbabe800 0x55bcdb70f600 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.913+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.2 v2:172.21.15.78:6808/35732 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbabe800 2023-10-11T09:50:32.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.913+0000 7f9b56f42700 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55bcdbaec000 0x55bcdbaf2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.913+0000 7f9b56f42700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:4 15.20 15:04000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbaec400 con 0x55bcdbaec000 2023-10-11T09:50:32.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.913+0000 7f9b5b74b700 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55bcdbaec000 0x55bcdbaf2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.913+0000 7f9b5b74b700 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55bcdbaec000 0x55bcdbaf2000 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.917+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:32.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.917+0000 7f9b57743700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:5 15.10 15:08000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbaecc00 con 0x55bcdb95f000 2023-10-11T09:50:32.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.917+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.4 v2:172.21.15.133:6808/35898 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 110 out=49b] v108'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdb95f000 2023-10-11T09:50:32.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.917+0000 7f9b56f42700 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55bcdbaec800 0x55bcdbaf2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.917+0000 7f9b56f42700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:6 15.30 15:0c000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbaec400 con 0x55bcdbaec800 2023-10-11T09:50:32.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.917+0000 7f9b5c74d700 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55bcdbaec800 0x55bcdbaf2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.917+0000 7f9b5c74d700 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55bcdbaec800 0x55bcdbaf2580 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.917+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.7 v2:172.21.15.133:6824/35901 1 ==== osd_op_reply(6 [pgnls-filter start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdbaec800 2023-10-11T09:50:32.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.917+0000 7f9b57743700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:7 15.8 15:10000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbaed000 con 0x55bcdbabe800 2023-10-11T09:50:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.921+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.2 v2:172.21.15.78:6808/35732 2 ==== osd_op_reply(7 [pgnls-filter start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbabe800 2023-10-11T09:50:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.921+0000 7f9b56f42700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:8 15.28 15:14000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbaecc00 con 0x55bcdbaec800 2023-10-11T09:50:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.921+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.7 v2:172.21.15.133:6824/35901 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 110 out=49b] v108'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdbaec800 2023-10-11T09:50:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.921+0000 7f9b57743700 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55bcdbaed400 0x55bcdbaf3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.921+0000 7f9b57743700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:9 15.18 15:18000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbaec400 con 0x55bcdbaed400 2023-10-11T09:50:32.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.921+0000 7f9b5bf4c700 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55bcdbaed400 0x55bcdbaf3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.921+0000 7f9b5bf4c700 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55bcdbaed400 0x55bcdbaf3080 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.921+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(9 [pgnls-filter start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbaed400 2023-10-11T09:50:32.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.921+0000 7f9b56f42700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:10 15.38 15:1c000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdb95f400 con 0x55bcdb95f800 2023-10-11T09:50:32.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.921+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.0 v2:172.21.15.78:6800/35735 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdb95f800 2023-10-11T09:50:32.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.921+0000 7f9b57743700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:11 15.4 15:20000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabe400 con 0x55bcdbaec000 2023-10-11T09:50:32.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.925+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 2 ==== osd_op_reply(11 [pgnls-filter start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:32.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.925+0000 7f9b56f42700 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55bcdbaecc00 0x55bcdbaf2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.925+0000 7f9b56f42700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:12 15.24 15:24000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbaed000 con 0x55bcdbaecc00 2023-10-11T09:50:32.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.925+0000 7f9b5b74b700 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55bcdbaecc00 0x55bcdbaf2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.925+0000 7f9b5b74b700 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55bcdbaecc00 0x55bcdbaf2b00 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.925+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== osd_op_reply(12 [pgnls-filter start_epoch 110 out=49b] v108'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaecc00 2023-10-11T09:50:32.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.925+0000 7f9b57743700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:13 15.14 15:28000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabec00 con 0x55bcdbaecc00 2023-10-11T09:50:32.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.929+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.6 v2:172.21.15.133:6800/35900 2 ==== osd_op_reply(13 [pgnls-filter start_epoch 110 out=49b] v108'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaecc00 2023-10-11T09:50:32.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.929+0000 7f9b56f42700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:14 15.34 15:2c000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbaed800 con 0x55bcdb95f000 2023-10-11T09:50:32.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.929+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.4 v2:172.21.15.133:6808/35898 3 ==== osd_op_reply(14 [pgnls-filter start_epoch 110 out=49b] v108'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdb95f000 2023-10-11T09:50:32.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.929+0000 7f9b57743700 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55bcdbaed000 0x55bcdbaf3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:32.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.929+0000 7f9b57743700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:15 15.c 15:30000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbaed800 con 0x55bcdbaed000 2023-10-11T09:50:32.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.929+0000 7f9b5c74d700 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55bcdbaed000 0x55bcdbaf3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:32.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.929+0000 7f9b5c74d700 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55bcdbaed000 0x55bcdbaf3600 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:32.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.933+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.5 v2:172.21.15.133:6816/35899 1 ==== osd_op_reply(15 [pgnls-filter start_epoch 110 out=81b] v110'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdbaed000 2023-10-11T09:50:32.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.933+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:16 15.0 15:00000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabf000 con 0x55bcdbabe800 2023-10-11T09:50:32.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.933+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.2 v2:172.21.15.78:6808/35732 3 ==== osd_op_reply(16 [pgnls-filter start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbabe800 2023-10-11T09:50:32.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.933+0000 7f9b57743700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:17 15.20 15:04000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabe400 con 0x55bcdbaec000 2023-10-11T09:50:32.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.933+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 3 ==== osd_op_reply(17 [pgnls-filter start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:32.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.933+0000 7f9b56f42700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:18 15.10 15:08000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbaedc00 con 0x55bcdb95f000 2023-10-11T09:50:32.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.933+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.4 v2:172.21.15.133:6808/35898 4 ==== osd_op_reply(18 [pgnls-filter start_epoch 110 out=49b] v108'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdb95f000 2023-10-11T09:50:32.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.933+0000 7f9b57743700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:19 15.30 15:0c000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabec00 con 0x55bcdbaec800 2023-10-11T09:50:32.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.933+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.7 v2:172.21.15.133:6824/35901 3 ==== osd_op_reply(19 [pgnls-filter start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdbaec800 2023-10-11T09:50:32.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.937+0000 7f9b56f42700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:20 15.8 15:10000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb06000 con 0x55bcdbabe800 2023-10-11T09:50:32.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.937+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.2 v2:172.21.15.78:6808/35732 4 ==== osd_op_reply(20 [pgnls-filter start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbabe800 2023-10-11T09:50:32.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.937+0000 7f9b57743700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:21 15.28 15:14000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb06000 con 0x55bcdbaec800 2023-10-11T09:50:32.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.937+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.7 v2:172.21.15.133:6824/35901 4 ==== osd_op_reply(21 [pgnls-filter start_epoch 110 out=49b] v108'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdbaec800 2023-10-11T09:50:32.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.937+0000 7f9b56f42700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:22 15.18 15:18000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb06400 con 0x55bcdbaed400 2023-10-11T09:50:32.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.937+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.3 v2:172.21.15.78:6824/35738 2 ==== osd_op_reply(22 [pgnls-filter start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbaed400 2023-10-11T09:50:32.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.937+0000 7f9b57743700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:23 15.38 15:1c000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb07400 con 0x55bcdb95f800 2023-10-11T09:50:32.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.937+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.0 v2:172.21.15.78:6800/35735 3 ==== osd_op_reply(23 [pgnls-filter start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdb95f800 2023-10-11T09:50:32.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.937+0000 7f9b56f42700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:24 15.4 15:20000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb06800 con 0x55bcdbaec000 2023-10-11T09:50:32.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.937+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:32.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.937+0000 7f9b57743700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:25 15.24 15:24000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb07800 con 0x55bcdbaecc00 2023-10-11T09:50:32.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.941+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.6 v2:172.21.15.133:6800/35900 3 ==== osd_op_reply(25 [pgnls-filter start_epoch 110 out=49b] v108'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaecc00 2023-10-11T09:50:32.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.941+0000 7f9b56f42700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:26 15.14 15:28000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb06c00 con 0x55bcdbaecc00 2023-10-11T09:50:32.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.941+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.6 v2:172.21.15.133:6800/35900 4 ==== osd_op_reply(26 [pgnls-filter start_epoch 110 out=49b] v108'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaecc00 2023-10-11T09:50:32.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.941+0000 7f9b57743700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:27 15.34 15:2c000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabf000 con 0x55bcdb95f000 2023-10-11T09:50:32.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.941+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.4 v2:172.21.15.133:6808/35898 5 ==== osd_op_reply(27 [pgnls-filter start_epoch 110 out=49b] v108'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdb95f000 2023-10-11T09:50:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.941+0000 7f9b56f42700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:28 15.c 15:30000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb07000 con 0x55bcdbaed000 2023-10-11T09:50:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.941+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.5 v2:172.21.15.133:6816/35899 2 ==== osd_op_reply(28 [pgnls-filter start_epoch 110 out=81b] v110'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdbaed000 2023-10-11T09:50:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.945+0000 7f9b5d790840 20 datascan.forall_objects: OSD matched oid 100000001fc.00000000 2023-10-11T09:50:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.945+0000 7f9b5d790840 10 datascan.operator(): handling object 100000001fc.0 2023-10-11T09:50:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.945+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:29 15.c 15:30604137:::100000001fc.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabf400 con 0x55bcdbaed000 2023-10-11T09:50:32.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.945+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.5 v2:172.21.15.133:6816/35899 3 ==== osd_op_reply(29 100000001fc.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr,getxattr] v0'0 uv10 ondisk = 0) v8 ==== 416+0+46 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdbaed000 2023-10-11T09:50:32.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.945+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:30 16.c 16:30604137:::100000001fc.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabf800 con 0x55bcdbaed000 2023-10-11T09:50:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.945+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.5 v2:172.21.15.133:6816/35899 4 ==== osd_op_reply(30 100000001fc.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdbaed000 2023-10-11T09:50:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.945+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:31 14.1f 14:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabfc00 con 0x55bcdb95f800 2023-10-11T09:50:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.945+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.0 v2:172.21.15.78:6800/35735 4 ==== osd_op_reply(31 1.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdb95f800 2023-10-11T09:50:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.945+0000 7f9b5d790840 10 datascan.find_or_create_dirfrag: read_version = 0 2023-10-11T09:50:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.945+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:32 14.1f 14:ff5b34d6:::1.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabf400 con 0x55bcdb95f800 2023-10-11T09:50:32.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.949+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.0 v2:172.21.15.78:6800/35735 5 ==== osd_op_reply(32 1.00000000 [create,omap-set-header] v110'11 uv11 ondisk = 0) v8 ==== 196+0+0 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdb95f800 2023-10-11T09:50:32.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.949+0000 7f9b5d790840 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x1 2023-10-11T09:50:32.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.949+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:33 14.1f 14:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=23b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabf800 con 0x55bcdb95f800 2023-10-11T09:50:32.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.949+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.0 v2:172.21.15.78:6800/35735 6 ==== osd_op_reply(33 1.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv11 ondisk = 0) v8 ==== 154+0+4 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdb95f800 2023-10-11T09:50:32.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.949+0000 7f9b5d790840 20 datascan.read_dentry: oid=1.00000000 dname=lost+found frag=*, r=0 2023-10-11T09:50:32.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.949+0000 7f9b5d790840 20 datascan.read_dentry: lost+found_head not found in result 2023-10-11T09:50:32.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.949+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:34 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabfc00 con 0x55bcdb95f800 2023-10-11T09:50:32.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.953+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.0 v2:172.21.15.78:6800/35735 7 ==== osd_op_reply(34 1.00000000 [omap-set-vals] v110'12 uv12 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdb95f800 2023-10-11T09:50:32.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.953+0000 7f9b5d790840 20 datascan.inject_linkage: Injected dentry 0x0x1/lost+found pointing to 0x0x4 2023-10-11T09:50:32.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.953+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:35 14.3 14:c0ce80d4:::4.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabe400 con 0x55bcdb95f800 2023-10-11T09:50:32.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.953+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.0 v2:172.21.15.78:6800/35735 8 ==== osd_op_reply(35 4.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdb95f800 2023-10-11T09:50:32.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.953+0000 7f9b5d790840 10 datascan.find_or_create_dirfrag: read_version = 0 2023-10-11T09:50:32.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.953+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:36 14.3 14:c0ce80d4:::4.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabec00 con 0x55bcdb95f800 2023-10-11T09:50:32.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.957+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.0 v2:172.21.15.78:6800/35735 9 ==== osd_op_reply(36 4.00000000 [create,omap-set-header] v110'1 uv1 ondisk = 0) v8 ==== 196+0+0 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdb95f800 2023-10-11T09:50:32.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.957+0000 7f9b5d790840 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x4 2023-10-11T09:50:32.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.957+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:37 14.3 14:c0ce80d4:::4.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbaedc00 con 0x55bcdb95f800 2023-10-11T09:50:32.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.957+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.0 v2:172.21.15.78:6800/35735 10 ==== osd_op_reply(37 4.00000000 [omap-set-vals] v110'2 uv2 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdb95f800 2023-10-11T09:50:32.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.957+0000 7f9b5d790840 20 datascan.inject_linkage: Injected dentry 0x0x4/100000001fc pointing to 0x0x100000001fc 2023-10-11T09:50:32.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.957+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:38 15.2c 15:34000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbaed800 con 0x55bcdb95f000 2023-10-11T09:50:32.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.957+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.4 v2:172.21.15.133:6808/35898 6 ==== osd_op_reply(38 [pgnls-filter start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdb95f000 2023-10-11T09:50:32.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.957+0000 7f9b56f42700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:39 15.1c 15:38000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbaedc00 con 0x55bcdbaec000 2023-10-11T09:50:32.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.961+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 5 ==== osd_op_reply(39 [pgnls-filter start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:32.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.961+0000 7f9b57743700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:40 15.3c 15:3c000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbaec400 con 0x55bcdbaed400 2023-10-11T09:50:32.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.961+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.3 v2:172.21.15.78:6824/35738 3 ==== osd_op_reply(40 [pgnls-filter start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbaed400 2023-10-11T09:50:32.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.961+0000 7f9b56f42700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:41 15.2 15:40000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabf400 con 0x55bcdbabe800 2023-10-11T09:50:32.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.961+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.2 v2:172.21.15.78:6808/35732 5 ==== osd_op_reply(41 [pgnls-filter start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbabe800 2023-10-11T09:50:32.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.961+0000 7f9b57743700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:42 15.22 15:44000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb07400 con 0x55bcdbabe800 2023-10-11T09:50:32.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.961+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.2 v2:172.21.15.78:6808/35732 6 ==== osd_op_reply(42 [pgnls-filter start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbabe800 2023-10-11T09:50:32.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.961+0000 7f9b56f42700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:43 15.12 15:48000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabf800 con 0x55bcdbaed400 2023-10-11T09:50:32.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.961+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.3 v2:172.21.15.78:6824/35738 4 ==== osd_op_reply(43 [pgnls-filter start_epoch 110 out=49b] v108'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbaed400 2023-10-11T09:50:32.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.961+0000 7f9b57743700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:44 15.32 15:4c000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb07000 con 0x55bcdbabe800 2023-10-11T09:50:32.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.961+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.2 v2:172.21.15.78:6808/35732 7 ==== osd_op_reply(44 [pgnls-filter start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbabe800 2023-10-11T09:50:32.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.961+0000 7f9b56f42700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:45 15.a 15:50000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabfc00 con 0x55bcdbabe800 2023-10-11T09:50:32.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.965+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.2 v2:172.21.15.78:6808/35732 8 ==== osd_op_reply(45 [pgnls-filter start_epoch 110 out=49b] v108'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbabe800 2023-10-11T09:50:32.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.965+0000 7f9b57743700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:46 15.2a 15:54000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb06800 con 0x55bcdb95f800 2023-10-11T09:50:32.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.965+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.0 v2:172.21.15.78:6800/35735 11 ==== osd_op_reply(46 [pgnls-filter start_epoch 110 out=81b] v110'15 uv15 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdb95f800 2023-10-11T09:50:32.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.965+0000 7f9b5d790840 20 datascan.forall_objects: OSD matched oid 100000001f6.00000000 2023-10-11T09:50:32.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.965+0000 7f9b5d790840 10 datascan.operator(): handling object 100000001f6.0 2023-10-11T09:50:32.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.965+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:47 15.2a 15:57fe7535:::100000001f6.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdb95f400 con 0x55bcdb95f800 2023-10-11T09:50:32.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.965+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.0 v2:172.21.15.78:6800/35735 12 ==== osd_op_reply(47 100000001f6.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (111) out=111b,getxattr,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdb95f800 2023-10-11T09:50:32.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.965+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:48 16.a 16:57fe7535:::100000001f6.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb06400 con 0x55bcdbabe800 2023-10-11T09:50:32.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.965+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.2 v2:172.21.15.78:6808/35732 9 ==== osd_op_reply(48 100000001f6.00000000 [getxattr (30) out=30b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbabe800 2023-10-11T09:50:32.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.965+0000 7f9b5d790840 20 datascan.operator(): loaded layout from xattr: pi: 16 os: 262144 sc: 4 su: 65536 2023-10-11T09:50:32.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.965+0000 7f9b5d790840 10 datascan.operator(): calculating striped size from complete objs: 0, partial objs: 4 2023-10-11T09:50:32.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.965+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:49 16.a 16:57fe7535:::100000001f6.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb07400 con 0x55bcdbabe800 2023-10-11T09:50:32.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.965+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.2 v2:172.21.15.78:6808/35732 10 ==== osd_op_reply(49 100000001f6.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbabe800 2023-10-11T09:50:32.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.965+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:50 16.1c 16:3d6db04b:::100000001f6.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb07c00 con 0x55bcdbabe800 2023-10-11T09:50:32.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.965+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.2 v2:172.21.15.78:6808/35732 11 ==== osd_op_reply(50 100000001f6.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbabe800 2023-10-11T09:50:32.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.965+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:51 16.1f 16:faedc8eb:::100000001f6.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabf400 con 0x55bcdbaed000 2023-10-11T09:50:32.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.969+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.5 v2:172.21.15.133:6816/35899 5 ==== osd_op_reply(51 100000001f6.00000002 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdbaed000 2023-10-11T09:50:32.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.969+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:52 16.e 16:728aa123:::100000001f6.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbaec400 con 0x55bcdbabe800 2023-10-11T09:50:32.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.969+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.2 v2:172.21.15.78:6808/35732 12 ==== osd_op_reply(52 100000001f6.00000003 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbabe800 2023-10-11T09:50:32.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.969+0000 7f9b5d790840 10 datascan.inject_with_backtrace: inode: 0x0x100000001f6 2023-10-11T09:50:32.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.969+0000 7f9b5d790840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/flushed_file_1 2023-10-11T09:50:32.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.969+0000 7f9b5d790840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=flushed_file_1 2023-10-11T09:50:32.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.969+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:53 14.10 14:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb07000 con 0x55bcdbaec000 2023-10-11T09:50:32.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.969+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 6 ==== osd_op_reply(53 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:32.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.969+0000 7f9b5d790840 4 datascan.get_frag_of: Unexpected error on '10000000000.00000000': (2) No such file or directory 2023-10-11T09:50:32.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.969+0000 7f9b5d790840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/flushed_file_1, will insert to root 2023-10-11T09:50:32.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.969+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:54 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb06400 con 0x55bcdbaec000 2023-10-11T09:50:32.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.969+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 7 ==== osd_op_reply(54 10000000000.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:32.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.969+0000 7f9b5d790840 10 datascan.find_or_create_dirfrag: read_version = 0 2023-10-11T09:50:32.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.969+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:55 14.10 14:0cf6ad0f:::10000000000.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb07400 con 0x55bcdbaec000 2023-10-11T09:50:32.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.973+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 8 ==== osd_op_reply(55 10000000000.00000000 [create,omap-set-header] v110'16 uv16 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:32.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.973+0000 7f9b5d790840 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x10000000000 2023-10-11T09:50:32.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.973+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:56 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabf800 con 0x55bcdbaec000 2023-10-11T09:50:32.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.973+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 9 ==== osd_op_reply(56 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:32.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.973+0000 7f9b5d790840 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_1 frag=*, r=0 2023-10-11T09:50:32.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.973+0000 7f9b5d790840 20 datascan.read_dentry: flushed_file_1_head not found in result 2023-10-11T09:50:32.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.973+0000 7f9b5d790840 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f6 at 0x0x10000000000/flushed_file_1 with size=524288 bytes 2023-10-11T09:50:32.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.973+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:57 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabfc00 con 0x55bcdbaec000 2023-10-11T09:50:32.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.977+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 10 ==== osd_op_reply(57 10000000000.00000000 [omap-set-vals] v110'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:32.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.977+0000 7f9b5d790840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_1 pointing to 0x0x100000001f6 2023-10-11T09:50:32.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.977+0000 7f9b5d790840 10 datascan.inject_with_backtrace: backptr: 0x0x1/stripey 2023-10-11T09:50:32.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.977+0000 7f9b5d790840 20 datascan.get_frag_of: dirino=0x1 target_dname=stripey 2023-10-11T09:50:32.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.977+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:58 14.1f 14:ff5b34d6:::1.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb07000 con 0x55bcdb95f800 2023-10-11T09:50:32.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.977+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.0 v2:172.21.15.78:6800/35735 13 ==== osd_op_reply(58 1.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 154+0+0 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdb95f800 2023-10-11T09:50:32.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.977+0000 7f9b5d790840 10 datascan.get_frag_of: No backtrace on '1.00000000' 2023-10-11T09:50:32.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.977+0000 7f9b5d790840 4 datascan.get_frag_of: No backtrace on '1.00000000': cannot determine fragtree 2023-10-11T09:50:32.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.977+0000 7f9b5d790840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x1/stripey, will insert to root 2023-10-11T09:50:32.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.977+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:59 14.1f 14:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb06400 con 0x55bcdb95f800 2023-10-11T09:50:32.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.977+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.0 v2:172.21.15.78:6800/35735 14 ==== osd_op_reply(59 1.00000000 [omap-get-header out=274b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+274 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdb95f800 2023-10-11T09:50:32.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.977+0000 7f9b5d790840 10 datascan.find_or_create_dirfrag: read_version = 12 2023-10-11T09:50:32.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.977+0000 7f9b5d790840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x1 * 2023-10-11T09:50:32.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.977+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:60 14.1f 14:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb07400 con 0x55bcdb95f800 2023-10-11T09:50:32.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.977+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.0 v2:172.21.15.78:6800/35735 15 ==== osd_op_reply(60 1.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+4 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdb95f800 2023-10-11T09:50:32.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.977+0000 7f9b5d790840 20 datascan.read_dentry: oid=1.00000000 dname=stripey frag=*, r=0 2023-10-11T09:50:32.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.977+0000 7f9b5d790840 20 datascan.read_dentry: stripey_head not found in result 2023-10-11T09:50:32.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.977+0000 7f9b5d790840 10 datascan.inject_with_backtrace: Linking ancestor directory of inode 0x0x10000000000 at 0x0x1:stripey 2023-10-11T09:50:32.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.977+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:61 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=508b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbaed800 con 0x55bcdb95f800 2023-10-11T09:50:32.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.981+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.0 v2:172.21.15.78:6800/35735 16 ==== osd_op_reply(61 1.00000000 [omap-set-vals] v110'13 uv13 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdb95f800 2023-10-11T09:50:32.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.981+0000 7f9b5d790840 20 datascan.inject_linkage: Injected dentry 0x0x1/stripey pointing to 0x0x10000000000 2023-10-11T09:50:32.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.981+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:62 15.1a 15:58000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbaedc00 con 0x55bcdbaed000 2023-10-11T09:50:32.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.981+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.5 v2:172.21.15.133:6816/35899 6 ==== osd_op_reply(62 [pgnls-filter start_epoch 110 out=49b] v108'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdbaed000 2023-10-11T09:50:32.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.981+0000 7f9b57743700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:63 15.3a 15:5c000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb07800 con 0x55bcdbaec800 2023-10-11T09:50:32.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.981+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.7 v2:172.21.15.133:6824/35901 5 ==== osd_op_reply(63 [pgnls-filter start_epoch 110 out=49b] v108'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdbaec800 2023-10-11T09:50:32.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.981+0000 7f9b56f42700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:64 15.6 15:60000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabe400 con 0x55bcdb95f800 2023-10-11T09:50:32.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.985+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.0 v2:172.21.15.78:6800/35735 17 ==== osd_op_reply(64 [pgnls-filter start_epoch 110 out=49b] v108'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdb95f800 2023-10-11T09:50:32.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.985+0000 7f9b57743700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:65 15.26 15:64000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb06c00 con 0x55bcdb95f800 2023-10-11T09:50:32.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.985+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.0 v2:172.21.15.78:6800/35735 18 ==== osd_op_reply(65 [pgnls-filter start_epoch 110 out=49b] v108'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdb95f800 2023-10-11T09:50:32.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.985+0000 7f9b56f42700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:66 15.16 15:68000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabec00 con 0x55bcdb95f800 2023-10-11T09:50:32.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.985+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.0 v2:172.21.15.78:6800/35735 19 ==== osd_op_reply(66 [pgnls-filter start_epoch 110 out=49b] v108'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdb95f800 2023-10-11T09:50:32.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.985+0000 7f9b57743700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:67 15.36 15:6c000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb06000 con 0x55bcdbaecc00 2023-10-11T09:50:32.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.985+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.6 v2:172.21.15.133:6800/35900 5 ==== osd_op_reply(67 [pgnls-filter start_epoch 110 out=101b] v110'31 uv31 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaecc00 2023-10-11T09:50:32.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.985+0000 7f9b5d790840 20 datascan.forall_objects: OSD matched oid 100000001f9.00000000 2023-10-11T09:50:32.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.985+0000 7f9b5d790840 10 datascan.operator(): handling object 100000001f9.0 2023-10-11T09:50:32.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.985+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:68 15.36 15:6ee5e2c4:::100000001f9.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabf000 con 0x55bcdbaecc00 2023-10-11T09:50:32.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.989+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.6 v2:172.21.15.133:6800/35900 6 ==== osd_op_reply(68 100000001f9.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (111) out=111b,getxattr,getxattr] v0'0 uv31 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaecc00 2023-10-11T09:50:32.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.989+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:69 16.16 16:6ee5e2c4:::100000001f9.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbaed800 con 0x55bcdbaed400 2023-10-11T09:50:32.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.989+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.3 v2:172.21.15.78:6824/35738 5 ==== osd_op_reply(69 100000001f9.00000000 [getxattr (30) out=30b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbaed400 2023-10-11T09:50:32.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.989+0000 7f9b5d790840 20 datascan.operator(): loaded layout from xattr: pi: 16 os: 262144 sc: 4 su: 65536 2023-10-11T09:50:32.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.989+0000 7f9b5d790840 10 datascan.operator(): calculating striped size from complete objs: 4, partial objs: 4 2023-10-11T09:50:32.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.989+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:70 16.7 16:e7ecf421:::100000001f9.00000004:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb06800 con 0x55bcdbaed400 2023-10-11T09:50:32.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.989+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.3 v2:172.21.15.78:6824/35738 6 ==== osd_op_reply(70 100000001f9.00000004 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbaed400 2023-10-11T09:50:32.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.989+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:71 16.2 16:46f89f8a:::100000001f9.00000005:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb07000 con 0x55bcdb95f000 2023-10-11T09:50:33.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.989+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.4 v2:172.21.15.133:6808/35898 7 ==== osd_op_reply(71 100000001f9.00000005 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdb95f000 2023-10-11T09:50:33.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.989+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:72 16.16 16:6df493a7:::100000001f9.00000006:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb06400 con 0x55bcdbaed400 2023-10-11T09:50:33.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.989+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.3 v2:172.21.15.78:6824/35738 7 ==== osd_op_reply(72 100000001f9.00000006 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbaed400 2023-10-11T09:50:33.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.993+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:73 16.17 16:ee7e3069:::100000001f9.00000007:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb07400 con 0x55bcdbabe800 2023-10-11T09:50:33.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.993+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.2 v2:172.21.15.78:6808/35732 13 ==== osd_op_reply(73 100000001f9.00000007 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbabe800 2023-10-11T09:50:33.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.993+0000 7f9b5d790840 10 datascan.inject_with_backtrace: inode: 0x0x100000001f9 2023-10-11T09:50:33.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.993+0000 7f9b5d790840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/flushed_file_4 2023-10-11T09:50:33.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.993+0000 7f9b5d790840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=flushed_file_4 2023-10-11T09:50:33.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.993+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:74 14.10 14:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabf400 con 0x55bcdbaec000 2023-10-11T09:50:33.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.993+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 11 ==== osd_op_reply(74 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:33.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.993+0000 7f9b5d790840 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-10-11T09:50:33.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.993+0000 7f9b5d790840 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-10-11T09:50:33.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.993+0000 7f9b5d790840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/flushed_file_4, will insert to root 2023-10-11T09:50:33.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.993+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:75 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdb95f400 con 0x55bcdbaec000 2023-10-11T09:50:33.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.993+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 12 ==== osd_op_reply(75 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:33.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.993+0000 7f9b5d790840 10 datascan.find_or_create_dirfrag: read_version = 17 2023-10-11T09:50:33.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.993+0000 7f9b5d790840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-10-11T09:50:33.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.993+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:76 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb06000 con 0x55bcdbaec000 2023-10-11T09:50:33.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.993+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 13 ==== osd_op_reply(76 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:33.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.993+0000 7f9b5d790840 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_4 frag=*, r=0 2023-10-11T09:50:33.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.993+0000 7f9b5d790840 20 datascan.read_dentry: flushed_file_4_head not found in result 2023-10-11T09:50:33.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.993+0000 7f9b5d790840 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f9 at 0x0x10000000000/flushed_file_4 with size=1441792 bytes 2023-10-11T09:50:33.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.993+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:77 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb07000 con 0x55bcdbaec000 2023-10-11T09:50:33.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.997+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 14 ==== osd_op_reply(77 10000000000.00000000 [omap-set-vals] v110'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:33.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.997+0000 7f9b5d790840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_4 pointing to 0x0x100000001f9 2023-10-11T09:50:33.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.997+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:78 15.36 15:6fe38dc2::::head [pgnls-filter start_epoch 110 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb07c00 con 0x55bcdbaecc00 2023-10-11T09:50:33.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.997+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.6 v2:172.21.15.133:6800/35900 7 ==== osd_op_reply(78 [pgnls-filter start_epoch 110 out=81b] v110'31 uv31 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaecc00 2023-10-11T09:50:33.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.997+0000 7f9b5d790840 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2023-10-11T09:50:33.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.997+0000 7f9b5d790840 10 datascan.operator(): handling object 100000001f7.0 2023-10-11T09:50:33.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.997+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:79 15.36 15:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb06800 con 0x55bcdbaecc00 2023-10-11T09:50:33.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.997+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.6 v2:172.21.15.133:6800/35900 8 ==== osd_op_reply(79 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (111) out=111b,getxattr,getxattr] v0'0 uv27 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaecc00 2023-10-11T09:50:33.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:32.997+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:80 16.16 16:6fe38dc2:::100000001f7.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb06400 con 0x55bcdbaed400 2023-10-11T09:50:33.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.001+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.3 v2:172.21.15.78:6824/35738 8 ==== osd_op_reply(80 100000001f7.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbaed400 2023-10-11T09:50:33.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.001+0000 7f9b5d790840 20 datascan.operator(): loaded layout from xattr: pi: 16 os: 262144 sc: 4 su: 65536 2023-10-11T09:50:33.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.001+0000 7f9b5d790840 10 datascan.operator(): calculating striped size from complete objs: 0, partial objs: 4 2023-10-11T09:50:33.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.001+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:81 16.16 16:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb07400 con 0x55bcdbaed400 2023-10-11T09:50:33.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.001+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.3 v2:172.21.15.78:6824/35738 9 ==== osd_op_reply(81 100000001f7.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbaed400 2023-10-11T09:50:33.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.001+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:82 16.1 16:848ebb0b:::100000001f7.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbaec400 con 0x55bcdbabe800 2023-10-11T09:50:33.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.001+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.2 v2:172.21.15.78:6808/35732 14 ==== osd_op_reply(82 100000001f7.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbabe800 2023-10-11T09:50:33.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.001+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:83 16.1c 16:398da958:::100000001f7.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbaed800 con 0x55bcdbabe800 2023-10-11T09:50:33.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.001+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.2 v2:172.21.15.78:6808/35732 15 ==== osd_op_reply(83 100000001f7.00000002 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbabe800 2023-10-11T09:50:33.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.001+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:84 16.6 16:613bf181:::100000001f7.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdb95f400 con 0x55bcdbaecc00 2023-10-11T09:50:33.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.001+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.6 v2:172.21.15.133:6800/35900 9 ==== osd_op_reply(84 100000001f7.00000003 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaecc00 2023-10-11T09:50:33.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.001+0000 7f9b5d790840 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2023-10-11T09:50:33.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.001+0000 7f9b5d790840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/flushed_file_2 2023-10-11T09:50:33.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.001+0000 7f9b5d790840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=flushed_file_2 2023-10-11T09:50:33.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.001+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:85 14.10 14:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabf800 con 0x55bcdbaec000 2023-10-11T09:50:33.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.001+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 15 ==== osd_op_reply(85 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:33.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.001+0000 7f9b5d790840 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-10-11T09:50:33.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.001+0000 7f9b5d790840 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-10-11T09:50:33.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.001+0000 7f9b5d790840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/flushed_file_2, will insert to root 2023-10-11T09:50:33.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.001+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:86 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabfc00 con 0x55bcdbaec000 2023-10-11T09:50:33.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.001+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 16 ==== osd_op_reply(86 10000000000.00000000 [omap-get-header out=274b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:33.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.001+0000 7f9b5d790840 10 datascan.find_or_create_dirfrag: read_version = 18 2023-10-11T09:50:33.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.001+0000 7f9b5d790840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-10-11T09:50:33.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.001+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:87 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabf000 con 0x55bcdbaec000 2023-10-11T09:50:33.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.005+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 17 ==== osd_op_reply(87 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:33.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.005+0000 7f9b5d790840 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_2 frag=*, r=0 2023-10-11T09:50:33.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.005+0000 7f9b5d790840 20 datascan.read_dentry: flushed_file_2_head not found in result 2023-10-11T09:50:33.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.005+0000 7f9b5d790840 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000000/flushed_file_2 with size=917504 bytes 2023-10-11T09:50:33.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.005+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:88 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabf400 con 0x55bcdbaec000 2023-10-11T09:50:33.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.005+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 18 ==== osd_op_reply(88 10000000000.00000000 [omap-set-vals] v110'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:33.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.005+0000 7f9b5d790840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_2 pointing to 0x0x100000001f7 2023-10-11T09:50:33.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.005+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:89 15.e 15:70000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabf800 con 0x55bcdbabe800 2023-10-11T09:50:33.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.005+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.2 v2:172.21.15.78:6808/35732 16 ==== osd_op_reply(89 [pgnls-filter start_epoch 110 out=49b] v108'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbabe800 2023-10-11T09:50:33.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.005+0000 7f9b56f42700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:90 15.2e 15:74000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb28000 con 0x55bcdbaed400 2023-10-11T09:50:33.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.009+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.3 v2:172.21.15.78:6824/35738 10 ==== osd_op_reply(90 [pgnls-filter start_epoch 110 out=49b] v108'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbaed400 2023-10-11T09:50:33.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.009+0000 7f9b57743700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:91 15.1e 15:78000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb29c00 con 0x55bcdbabe800 2023-10-11T09:50:33.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.009+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.2 v2:172.21.15.78:6808/35732 17 ==== osd_op_reply(91 [pgnls-filter start_epoch 110 out=49b] v108'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbabe800 2023-10-11T09:50:33.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.009+0000 7f9b56f42700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:92 15.3e 15:7c000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb28400 con 0x55bcdb95f800 2023-10-11T09:50:33.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.009+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.0 v2:172.21.15.78:6800/35735 20 ==== osd_op_reply(92 [pgnls-filter start_epoch 110 out=49b] v108'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdb95f800 2023-10-11T09:50:33.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.009+0000 7f9b57743700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:93 15.1 15:80000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb28000 con 0x55bcdbaecc00 2023-10-11T09:50:33.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.009+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.6 v2:172.21.15.133:6800/35900 10 ==== osd_op_reply(93 [pgnls-filter start_epoch 110 out=49b] v108'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaecc00 2023-10-11T09:50:33.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.009+0000 7f9b56f42700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:94 15.21 15:84000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb28800 con 0x55bcdbaed400 2023-10-11T09:50:33.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.009+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.3 v2:172.21.15.78:6824/35738 11 ==== osd_op_reply(94 [pgnls-filter start_epoch 110 out=49b] v108'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbaed400 2023-10-11T09:50:33.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.009+0000 7f9b57743700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:95 15.11 15:88000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb29c00 con 0x55bcdbabe800 2023-10-11T09:50:33.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.013+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.2 v2:172.21.15.78:6808/35732 18 ==== osd_op_reply(95 [pgnls-filter start_epoch 110 out=49b] v108'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbabe800 2023-10-11T09:50:33.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.013+0000 7f9b56f42700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:96 15.31 15:8c000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb28c00 con 0x55bcdbaed400 2023-10-11T09:50:33.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.013+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.3 v2:172.21.15.78:6824/35738 12 ==== osd_op_reply(96 [pgnls-filter start_epoch 110 out=49b] v108'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbaed400 2023-10-11T09:50:33.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.013+0000 7f9b57743700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:97 15.9 15:90000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabf800 con 0x55bcdbaecc00 2023-10-11T09:50:33.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.013+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.6 v2:172.21.15.133:6800/35900 11 ==== osd_op_reply(97 [pgnls-filter start_epoch 110 out=49b] v108'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaecc00 2023-10-11T09:50:33.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.013+0000 7f9b56f42700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:98 15.29 15:94000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb29000 con 0x55bcdb95f800 2023-10-11T09:50:33.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.013+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.0 v2:172.21.15.78:6800/35735 21 ==== osd_op_reply(98 [pgnls-filter start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdb95f800 2023-10-11T09:50:33.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.013+0000 7f9b57743700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:99 15.19 15:98000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbaec400 con 0x55bcdb95f000 2023-10-11T09:50:33.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.017+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.4 v2:172.21.15.133:6808/35898 8 ==== osd_op_reply(99 [pgnls-filter start_epoch 110 out=81b] v110'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdb95f000 2023-10-11T09:50:33.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.017+0000 7f9b5d790840 20 datascan.forall_objects: OSD matched oid 100000001f8.00000000 2023-10-11T09:50:33.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.017+0000 7f9b5d790840 10 datascan.operator(): handling object 100000001f8.0 2023-10-11T09:50:33.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.017+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:100 15.19 15:997b7129:::100000001f8.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabfc00 con 0x55bcdb95f000 2023-10-11T09:50:33.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.017+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.4 v2:172.21.15.133:6808/35898 9 ==== osd_op_reply(100 100000001f8.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (111) out=111b,getxattr,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdb95f000 2023-10-11T09:50:33.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.017+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:101 16.19 16:997b7129:::100000001f8.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabf000 con 0x55bcdb95f000 2023-10-11T09:50:33.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.017+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.4 v2:172.21.15.133:6808/35898 10 ==== osd_op_reply(101 100000001f8.00000000 [getxattr (30) out=30b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdb95f000 2023-10-11T09:50:33.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.017+0000 7f9b5d790840 20 datascan.operator(): loaded layout from xattr: pi: 16 os: 262144 sc: 4 su: 65536 2023-10-11T09:50:33.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.017+0000 7f9b5d790840 10 datascan.operator(): calculating striped size from complete objs: 0, partial objs: 4 2023-10-11T09:50:33.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.017+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:102 16.19 16:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdb95f400 con 0x55bcdb95f000 2023-10-11T09:50:33.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.017+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.4 v2:172.21.15.133:6808/35898 11 ==== osd_op_reply(102 100000001f8.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdb95f000 2023-10-11T09:50:33.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.017+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:103 16.11 16:893be98d:::100000001f8.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb06000 con 0x55bcdbaed000 2023-10-11T09:50:33.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.021+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.5 v2:172.21.15.133:6816/35899 7 ==== osd_op_reply(103 100000001f8.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdbaed000 2023-10-11T09:50:33.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.021+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:104 16.f 16:f26fdfa9:::100000001f8.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb07000 con 0x55bcdb95f800 2023-10-11T09:50:33.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.021+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.0 v2:172.21.15.78:6800/35735 22 ==== osd_op_reply(104 100000001f8.00000002 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdb95f800 2023-10-11T09:50:33.030 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.021+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:105 16.17 16:eff9002a:::100000001f8.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb07c00 con 0x55bcdbabe800 2023-10-11T09:50:33.030 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.021+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.2 v2:172.21.15.78:6808/35732 19 ==== osd_op_reply(105 100000001f8.00000003 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbabe800 2023-10-11T09:50:33.030 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.021+0000 7f9b5d790840 10 datascan.inject_with_backtrace: inode: 0x0x100000001f8 2023-10-11T09:50:33.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.021+0000 7f9b5d790840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/flushed_file_3 2023-10-11T09:50:33.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.021+0000 7f9b5d790840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=flushed_file_3 2023-10-11T09:50:33.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.021+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:106 14.10 14:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb06800 con 0x55bcdbaec000 2023-10-11T09:50:33.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.021+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 19 ==== osd_op_reply(106 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:33.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.021+0000 7f9b5d790840 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-10-11T09:50:33.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.021+0000 7f9b5d790840 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-10-11T09:50:33.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.021+0000 7f9b5d790840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/flushed_file_3, will insert to root 2023-10-11T09:50:33.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.021+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:107 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb07800 con 0x55bcdbaec000 2023-10-11T09:50:33.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.021+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 20 ==== osd_op_reply(107 10000000000.00000000 [omap-get-header out=274b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:33.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.021+0000 7f9b5d790840 10 datascan.find_or_create_dirfrag: read_version = 19 2023-10-11T09:50:33.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.021+0000 7f9b5d790840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-10-11T09:50:33.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.021+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:108 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb06c00 con 0x55bcdbaec000 2023-10-11T09:50:33.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.025+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 21 ==== osd_op_reply(108 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:33.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.025+0000 7f9b5d790840 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_3 frag=*, r=0 2023-10-11T09:50:33.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.025+0000 7f9b5d790840 20 datascan.read_dentry: flushed_file_3_head not found in result 2023-10-11T09:50:33.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.025+0000 7f9b5d790840 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f8 at 0x0x10000000000/flushed_file_3 with size=917503 bytes 2023-10-11T09:50:33.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.025+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:109 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb06000 con 0x55bcdbaec000 2023-10-11T09:50:33.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.025+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 22 ==== osd_op_reply(109 10000000000.00000000 [omap-set-vals] v110'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:33.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.025+0000 7f9b5d790840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_3 pointing to 0x0x100000001f8 2023-10-11T09:50:33.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.025+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:110 15.39 15:9c000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb28400 con 0x55bcdbaec800 2023-10-11T09:50:33.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.029+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.7 v2:172.21.15.133:6824/35901 6 ==== osd_op_reply(110 [pgnls-filter start_epoch 110 out=49b] v108'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdbaec800 2023-10-11T09:50:33.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.029+0000 7f9b57743700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:111 15.5 15:a0000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbaed800 con 0x55bcdbaecc00 2023-10-11T09:50:33.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.029+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.6 v2:172.21.15.133:6800/35900 12 ==== osd_op_reply(111 [pgnls-filter start_epoch 110 out=49b] v108'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaecc00 2023-10-11T09:50:33.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.029+0000 7f9b56f42700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:112 15.25 15:a4000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb29400 con 0x55bcdbaec800 2023-10-11T09:50:33.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.029+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.7 v2:172.21.15.133:6824/35901 7 ==== osd_op_reply(112 [pgnls-filter start_epoch 110 out=49b] v108'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdbaec800 2023-10-11T09:50:33.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.029+0000 7f9b57743700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:113 15.15 15:a8000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb06400 con 0x55bcdbaec800 2023-10-11T09:50:33.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.029+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.7 v2:172.21.15.133:6824/35901 8 ==== osd_op_reply(113 [pgnls-filter start_epoch 110 out=49b] v108'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdbaec800 2023-10-11T09:50:33.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.029+0000 7f9b56f42700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:114 15.35 15:ac000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb29800 con 0x55bcdbaec800 2023-10-11T09:50:33.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.033+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.7 v2:172.21.15.133:6824/35901 9 ==== osd_op_reply(114 [pgnls-filter start_epoch 110 out=49b] v108'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdbaec800 2023-10-11T09:50:33.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.033+0000 7f9b57743700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:115 15.d 15:b0000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb07400 con 0x55bcdbaec800 2023-10-11T09:50:33.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.033+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.7 v2:172.21.15.133:6824/35901 10 ==== osd_op_reply(115 [pgnls-filter start_epoch 110 out=49b] v108'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdbaec800 2023-10-11T09:50:33.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.033+0000 7f9b56f42700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:116 15.2d 15:b4000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb06800 con 0x55bcdbabe800 2023-10-11T09:50:33.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.033+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.2 v2:172.21.15.78:6808/35732 20 ==== osd_op_reply(116 [pgnls-filter start_epoch 110 out=49b] v108'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbabe800 2023-10-11T09:50:33.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.033+0000 7f9b57743700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:117 15.1d 15:b8000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb07800 con 0x55bcdb95f000 2023-10-11T09:50:33.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.037+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.4 v2:172.21.15.133:6808/35898 12 ==== osd_op_reply(117 [pgnls-filter start_epoch 110 out=49b] v108'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdb95f000 2023-10-11T09:50:33.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.037+0000 7f9b56f42700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:118 15.3d 15:bc000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdb95f400 con 0x55bcdbaec000 2023-10-11T09:50:33.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.037+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 23 ==== osd_op_reply(118 [pgnls-filter start_epoch 110 out=49b] v108'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:33.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.037+0000 7f9b57743700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:119 15.3 15:c0000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb34000 con 0x55bcdb95f000 2023-10-11T09:50:33.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.037+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.4 v2:172.21.15.133:6808/35898 13 ==== osd_op_reply(119 [pgnls-filter start_epoch 110 out=49b] v108'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdb95f000 2023-10-11T09:50:33.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.037+0000 7f9b56f42700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:120 15.23 15:c4000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabf400 con 0x55bcdb95f800 2023-10-11T09:50:33.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.037+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.0 v2:172.21.15.78:6800/35735 23 ==== osd_op_reply(120 [pgnls-filter start_epoch 110 out=49b] v108'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdb95f800 2023-10-11T09:50:33.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.037+0000 7f9b57743700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:121 15.13 15:c8000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb34400 con 0x55bcdbaec000 2023-10-11T09:50:33.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.041+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 24 ==== osd_op_reply(121 [pgnls-filter start_epoch 110 out=81b] v110'25 uv25 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:33.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.041+0000 7f9b5d790840 20 datascan.forall_objects: OSD matched oid 100000001fb.00000000 2023-10-11T09:50:33.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.041+0000 7f9b5d790840 10 datascan.operator(): handling object 100000001fb.0 2023-10-11T09:50:33.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.041+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:122 15.13 15:ca8fdbba:::100000001fb.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb29000 con 0x55bcdbaec000 2023-10-11T09:50:33.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.041+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 25 ==== osd_op_reply(122 100000001fb.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (111) out=111b,getxattr,getxattr] v0'0 uv25 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:33.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.041+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:123 16.13 16:ca8fdbba:::100000001fb.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabe400 con 0x55bcdbaec000 2023-10-11T09:50:33.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.041+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 26 ==== osd_op_reply(123 100000001fb.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:33.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.041+0000 7f9b5d790840 20 datascan.operator(): loaded layout from xattr: pi: 16 os: 262144 sc: 4 su: 65536 2023-10-11T09:50:33.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.041+0000 7f9b5d790840 10 datascan.operator(): calculating striped size from complete objs: 4, partial objs: 4 2023-10-11T09:50:33.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.041+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:124 16.3 16:c5d8c51d:::100000001fb.00000004:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabec00 con 0x55bcdbaed000 2023-10-11T09:50:33.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.041+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.5 v2:172.21.15.133:6816/35899 8 ==== osd_op_reply(124 100000001fb.00000004 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdbaed000 2023-10-11T09:50:33.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.041+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:125 16.f 16:f376f0dc:::100000001fb.00000005:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbaedc00 con 0x55bcdb95f800 2023-10-11T09:50:33.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.041+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.0 v2:172.21.15.78:6800/35735 24 ==== osd_op_reply(125 100000001fb.00000005 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdb95f800 2023-10-11T09:50:33.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.041+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:126 16.1e 16:7e8133fb:::100000001fb.00000006:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbaedc00 con 0x55bcdbaec800 2023-10-11T09:50:33.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.045+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.7 v2:172.21.15.133:6824/35901 11 ==== osd_op_reply(126 100000001fb.00000006 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdbaec800 2023-10-11T09:50:33.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.045+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:127 16.f 16:f776b802:::100000001fb.00000007:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabf400 con 0x55bcdb95f800 2023-10-11T09:50:33.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.045+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.0 v2:172.21.15.78:6800/35735 25 ==== osd_op_reply(127 100000001fb.00000007 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdb95f800 2023-10-11T09:50:33.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.045+0000 7f9b5d790840 10 datascan.inject_with_backtrace: inode: 0x0x100000001fb 2023-10-11T09:50:33.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.045+0000 7f9b5d790840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/flushed_file_6 2023-10-11T09:50:33.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.045+0000 7f9b5d790840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=flushed_file_6 2023-10-11T09:50:33.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.045+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:128 14.10 14:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabec00 con 0x55bcdbaec000 2023-10-11T09:50:33.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.045+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 27 ==== osd_op_reply(128 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:33.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.045+0000 7f9b5d790840 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-10-11T09:50:33.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.045+0000 7f9b5d790840 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-10-11T09:50:33.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.045+0000 7f9b5d790840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/flushed_file_6, will insert to root 2023-10-11T09:50:33.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.045+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:129 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb07000 con 0x55bcdbaec000 2023-10-11T09:50:33.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.045+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 28 ==== osd_op_reply(129 10000000000.00000000 [omap-get-header out=274b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:33.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.045+0000 7f9b5d790840 10 datascan.find_or_create_dirfrag: read_version = 20 2023-10-11T09:50:33.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.045+0000 7f9b5d790840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-10-11T09:50:33.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.045+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:130 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb06400 con 0x55bcdbaec000 2023-10-11T09:50:33.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.045+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 29 ==== osd_op_reply(130 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:33.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.045+0000 7f9b5d790840 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_6 frag=*, r=0 2023-10-11T09:50:33.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.045+0000 7f9b5d790840 20 datascan.read_dentry: flushed_file_6_head not found in result 2023-10-11T09:50:33.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.045+0000 7f9b5d790840 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fb at 0x0x10000000000/flushed_file_6 with size=1572864 bytes 2023-10-11T09:50:33.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.045+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:131 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb07400 con 0x55bcdbaec000 2023-10-11T09:50:33.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.049+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 30 ==== osd_op_reply(131 10000000000.00000000 [omap-set-vals] v110'21 uv21 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:33.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.049+0000 7f9b5d790840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_6 pointing to 0x0x100000001fb 2023-10-11T09:50:33.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.049+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:132 15.33 15:cc000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb28400 con 0x55bcdbaed400 2023-10-11T09:50:33.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.049+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.3 v2:172.21.15.78:6824/35738 13 ==== osd_op_reply(132 [pgnls-filter start_epoch 110 out=49b] v108'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbaed400 2023-10-11T09:50:33.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.049+0000 7f9b57743700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:133 15.b 15:d0000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb34800 con 0x55bcdbabe800 2023-10-11T09:50:33.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.053+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.2 v2:172.21.15.78:6808/35732 21 ==== osd_op_reply(133 [pgnls-filter start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbabe800 2023-10-11T09:50:33.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.053+0000 7f9b56f42700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:134 15.2b 15:d4000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabf800 con 0x55bcdbabe800 2023-10-11T09:50:33.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.053+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.2 v2:172.21.15.78:6808/35732 22 ==== osd_op_reply(134 [pgnls-filter start_epoch 110 out=49b] v108'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbabe800 2023-10-11T09:50:33.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.053+0000 7f9b57743700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:135 15.1b 15:d8000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb34c00 con 0x55bcdbaec800 2023-10-11T09:50:33.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.053+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.7 v2:172.21.15.133:6824/35901 12 ==== osd_op_reply(135 [pgnls-filter start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdbaec800 2023-10-11T09:50:33.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.053+0000 7f9b56f42700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:136 15.3b 15:dc000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabfc00 con 0x55bcdbaecc00 2023-10-11T09:50:33.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.053+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.6 v2:172.21.15.133:6800/35900 13 ==== osd_op_reply(136 [pgnls-filter start_epoch 110 out=49b] v108'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaecc00 2023-10-11T09:50:33.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.053+0000 7f9b57743700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:137 15.7 15:e0000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb35000 con 0x55bcdbaec000 2023-10-11T09:50:33.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.053+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 31 ==== osd_op_reply(137 [pgnls-filter start_epoch 110 out=81b] v110'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:33.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.057+0000 7f9b5d790840 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2023-10-11T09:50:33.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.057+0000 7f9b5d790840 10 datascan.operator(): handling object 10000000001.0 2023-10-11T09:50:33.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.057+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:138 15.7 15:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb29400 con 0x55bcdbaec000 2023-10-11T09:50:33.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.057+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 32 ==== osd_op_reply(138 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (111) out=111b,getxattr,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:33.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.057+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:139 16.7 16:e2e07930:::10000000001.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb29800 con 0x55bcdbaed400 2023-10-11T09:50:33.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.057+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.3 v2:172.21.15.78:6824/35738 14 ==== osd_op_reply(139 10000000001.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbaed400 2023-10-11T09:50:33.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.057+0000 7f9b5d790840 20 datascan.operator(): loaded layout from xattr: pi: 16 os: 262144 sc: 4 su: 65536 2023-10-11T09:50:33.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.057+0000 7f9b5d790840 10 datascan.operator(): calculating striped size from complete objs: 0, partial objs: 4 2023-10-11T09:50:33.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.057+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:140 16.7 16:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabf800 con 0x55bcdbaed400 2023-10-11T09:50:33.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.057+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.3 v2:172.21.15.78:6824/35738 15 ==== osd_op_reply(140 10000000001.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbaed400 2023-10-11T09:50:33.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.057+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:141 16.f 16:f526278d:::10000000001.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabe400 con 0x55bcdb95f800 2023-10-11T09:50:33.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.057+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.0 v2:172.21.15.78:6800/35735 26 ==== osd_op_reply(141 10000000001.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdb95f800 2023-10-11T09:50:33.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.057+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:142 16.1e 16:7e4d8c81:::10000000001.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb28800 con 0x55bcdbaec800 2023-10-11T09:50:33.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.057+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.7 v2:172.21.15.133:6824/35901 13 ==== osd_op_reply(142 10000000001.00000002 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdbaec800 2023-10-11T09:50:33.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.057+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:143 16.e 16:72395991:::10000000001.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb29c00 con 0x55bcdbabe800 2023-10-11T09:50:33.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.061+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.2 v2:172.21.15.78:6808/35732 23 ==== osd_op_reply(143 10000000001.00000003 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbabe800 2023-10-11T09:50:33.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.061+0000 7f9b5d790840 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2023-10-11T09:50:33.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.061+0000 7f9b5d790840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/flushed_file_0 2023-10-11T09:50:33.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.061+0000 7f9b5d790840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=flushed_file_0 2023-10-11T09:50:33.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.061+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:144 14.10 14:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb28c00 con 0x55bcdbaec000 2023-10-11T09:50:33.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.061+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 33 ==== osd_op_reply(144 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:33.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.061+0000 7f9b5d790840 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-10-11T09:50:33.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.061+0000 7f9b5d790840 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-10-11T09:50:33.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.061+0000 7f9b5d790840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/flushed_file_0, will insert to root 2023-10-11T09:50:33.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.061+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:145 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb28400 con 0x55bcdbaec000 2023-10-11T09:50:33.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.061+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 34 ==== osd_op_reply(145 10000000000.00000000 [omap-get-header out=274b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:33.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.061+0000 7f9b5d790840 10 datascan.find_or_create_dirfrag: read_version = 21 2023-10-11T09:50:33.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.061+0000 7f9b5d790840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-10-11T09:50:33.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.061+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:146 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb29000 con 0x55bcdbaec000 2023-10-11T09:50:33.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.061+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 35 ==== osd_op_reply(146 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:33.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.061+0000 7f9b5d790840 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_0 frag=*, r=0 2023-10-11T09:50:33.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.061+0000 7f9b5d790840 20 datascan.read_dentry: flushed_file_0_head not found in result 2023-10-11T09:50:33.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.061+0000 7f9b5d790840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/flushed_file_0 with size=1048576 bytes 2023-10-11T09:50:33.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.061+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:147 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb34800 con 0x55bcdbaec000 2023-10-11T09:50:33.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.065+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 36 ==== osd_op_reply(147 10000000000.00000000 [omap-set-vals] v110'22 uv22 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:33.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.065+0000 7f9b5d790840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_0 pointing to 0x0x10000000001 2023-10-11T09:50:33.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.065+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:148 15.27 15:e4000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb34000 con 0x55bcdbaecc00 2023-10-11T09:50:33.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.065+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.6 v2:172.21.15.133:6800/35900 14 ==== osd_op_reply(148 [pgnls-filter start_epoch 110 out=49b] v108'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaecc00 2023-10-11T09:50:33.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.065+0000 7f9b57743700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:149 15.17 15:e8000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb35400 con 0x55bcdb95f000 2023-10-11T09:50:33.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.065+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.4 v2:172.21.15.133:6808/35898 14 ==== osd_op_reply(149 [pgnls-filter start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdb95f000 2023-10-11T09:50:33.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.065+0000 7f9b56f42700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:150 15.37 15:ec000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabf000 con 0x55bcdb95f000 2023-10-11T09:50:33.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.065+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.4 v2:172.21.15.133:6808/35898 15 ==== osd_op_reply(150 [pgnls-filter start_epoch 110 out=49b] v108'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdb95f000 2023-10-11T09:50:33.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.069+0000 7f9b57743700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:151 15.f 15:f0000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb35800 con 0x55bcdbaecc00 2023-10-11T09:50:33.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.069+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.6 v2:172.21.15.133:6800/35900 15 ==== osd_op_reply(151 [pgnls-filter start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaecc00 2023-10-11T09:50:33.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.069+0000 7f9b56f42700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:152 15.2f 15:f4000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbaec400 con 0x55bcdb95f800 2023-10-11T09:50:33.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.069+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.0 v2:172.21.15.78:6800/35735 27 ==== osd_op_reply(152 [pgnls-filter start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdb95f800 2023-10-11T09:50:33.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.069+0000 7f9b57743700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:153 15.1f 15:f8000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb35c00 con 0x55bcdbaed400 2023-10-11T09:50:33.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.069+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.3 v2:172.21.15.78:6824/35738 16 ==== osd_op_reply(153 [pgnls-filter start_epoch 110 out=49b] v108'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbaed400 2023-10-11T09:50:33.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.069+0000 7f9b56f42700 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:154 15.3f 15:fc000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb28000 con 0x55bcdbaec000 2023-10-11T09:50:33.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.069+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 37 ==== osd_op_reply(154 [pgnls-filter start_epoch 110 out=81b] v110'18 uv18 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:33.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.069+0000 7f9b5d790840 20 datascan.forall_objects: OSD matched oid 100000001fa.00000000 2023-10-11T09:50:33.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.069+0000 7f9b5d790840 10 datascan.operator(): handling object 100000001fa.0 2023-10-11T09:50:33.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.069+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:155 15.3f 15:feb1c803:::100000001fa.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb34400 con 0x55bcdbaec000 2023-10-11T09:50:33.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.073+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 38 ==== osd_op_reply(155 100000001fa.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (111) out=111b,getxattr,getxattr] v0'0 uv18 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:33.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.073+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:156 16.1f 16:feb1c803:::100000001fa.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdb95f400 con 0x55bcdbaed000 2023-10-11T09:50:33.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.073+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.5 v2:172.21.15.133:6816/35899 9 ==== osd_op_reply(156 100000001fa.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdbaed000 2023-10-11T09:50:33.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.073+0000 7f9b5d790840 20 datascan.operator(): loaded layout from xattr: pi: 16 os: 262144 sc: 4 su: 65536 2023-10-11T09:50:33.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.073+0000 7f9b5d790840 10 datascan.operator(): calculating striped size from complete objs: 4, partial objs: 4 2023-10-11T09:50:33.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.073+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:157 16.1c 16:3c0114ac:::100000001fa.00000004:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb07c00 con 0x55bcdbabe800 2023-10-11T09:50:33.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.073+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.2 v2:172.21.15.78:6808/35732 24 ==== osd_op_reply(157 100000001fa.00000004 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbabe800 2023-10-11T09:50:33.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.073+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:158 16.3 16:c0957981:::100000001fa.00000005:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb06800 con 0x55bcdbaed000 2023-10-11T09:50:33.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.073+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.5 v2:172.21.15.133:6816/35899 10 ==== osd_op_reply(158 100000001fa.00000005 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdbaed000 2023-10-11T09:50:33.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.073+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:159 16.a 16:56c65d28:::100000001fa.00000006:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb06c00 con 0x55bcdbabe800 2023-10-11T09:50:33.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.077+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 <== osd.2 v2:172.21.15.78:6808/35732 25 ==== osd_op_reply(159 100000001fa.00000006 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55bcdc44d440 con 0x55bcdbabe800 2023-10-11T09:50:33.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.077+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:160 16.1f 16:fb032386:::100000001fa.00000007:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb06000 con 0x55bcdbaed000 2023-10-11T09:50:33.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.077+0000 7f9b5c74d700 1 -- 172.21.15.78:0/2738478221 <== osd.5 v2:172.21.15.133:6816/35899 11 ==== osd_op_reply(160 100000001fa.00000007 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55bcdc44d200 con 0x55bcdbaed000 2023-10-11T09:50:33.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.077+0000 7f9b5d790840 10 datascan.inject_with_backtrace: inode: 0x0x100000001fa 2023-10-11T09:50:33.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.077+0000 7f9b5d790840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/flushed_file_5 2023-10-11T09:50:33.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.077+0000 7f9b5d790840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=flushed_file_5 2023-10-11T09:50:33.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.077+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:161 14.10 14:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbb07800 con 0x55bcdbaec000 2023-10-11T09:50:33.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.077+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 39 ==== osd_op_reply(161 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:33.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.077+0000 7f9b5d790840 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-10-11T09:50:33.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.077+0000 7f9b5d790840 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-10-11T09:50:33.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.077+0000 7f9b5d790840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/flushed_file_5, will insert to root 2023-10-11T09:50:33.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.077+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:162 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbaed800 con 0x55bcdbaec000 2023-10-11T09:50:33.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.077+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 40 ==== osd_op_reply(162 10000000000.00000000 [omap-get-header out=274b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:33.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.077+0000 7f9b5d790840 10 datascan.find_or_create_dirfrag: read_version = 22 2023-10-11T09:50:33.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.077+0000 7f9b5d790840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-10-11T09:50:33.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.077+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:163 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabfc00 con 0x55bcdbaec000 2023-10-11T09:50:33.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.077+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 41 ==== osd_op_reply(163 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:33.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.077+0000 7f9b5d790840 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_5 frag=*, r=0 2023-10-11T09:50:33.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.077+0000 7f9b5d790840 20 datascan.read_dentry: flushed_file_5_head not found in result 2023-10-11T09:50:33.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.077+0000 7f9b5d790840 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fa at 0x0x10000000000/flushed_file_5 with size=1441793 bytes 2023-10-11T09:50:33.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.077+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:164 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55bcdbabec00 con 0x55bcdbaec000 2023-10-11T09:50:33.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.081+0000 7f9b5b74b700 1 -- 172.21.15.78:0/2738478221 <== osd.1 v2:172.21.15.78:6809/35734 42 ==== osd_op_reply(164 10000000000.00000000 [omap-set-vals] v110'23 uv23 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55bcdc44cfc0 con 0x55bcdbaec000 2023-10-11T09:50:33.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.081+0000 7f9b5d790840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_5 pointing to 0x0x100000001fa 2023-10-11T09:50:33.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.081+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55bcdb95f800 msgr2=0x55bcdb70fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:33.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.081+0000 7f9b5d790840 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55bcdb95f800 0x55bcdb70fb80 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.081+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55bcdbaec000 msgr2=0x55bcdbaf2000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:33.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.081+0000 7f9b5d790840 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55bcdbaec000 0x55bcdbaf2000 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.081+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55bcdbabe800 msgr2=0x55bcdb70f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:33.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.081+0000 7f9b5d790840 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55bcdbabe800 0x55bcdb70f600 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.081+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55bcdbaed400 msgr2=0x55bcdbaf3080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:33.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.081+0000 7f9b5d790840 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55bcdbaed400 0x55bcdbaf3080 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.081+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55bcdb95f000 msgr2=0x55bcdb710100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:33.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.081+0000 7f9b5d790840 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55bcdb95f000 0x55bcdb710100 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.081+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55bcdbaed000 msgr2=0x55bcdbaf3600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:33.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.081+0000 7f9b5d790840 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55bcdbaed000 0x55bcdbaf3600 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.081+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 reap_dead start 2023-10-11T09:50:33.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.081+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55bcdbaecc00 msgr2=0x55bcdbaf2b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:33.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.081+0000 7f9b5d790840 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55bcdbaecc00 0x55bcdbaf2b00 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.081+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55bcdbaec800 msgr2=0x55bcdbaf2580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:33.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.081+0000 7f9b5d790840 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55bcdbaec800 0x55bcdbaf2580 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.081+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 reap_dead start 2023-10-11T09:50:33.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.081+0000 7f9b5bf4c700 1 -- 172.21.15.78:0/2738478221 reap_dead start 2023-10-11T09:50:33.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.081+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55bcdbabe000 msgr2=0x55bcdb711700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:33.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.081+0000 7f9b5d790840 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55bcdbabe000 0x55bcdb711700 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x55bcdc3d06f0 tx=0x55bcdb957d40 comp rx=0 tx=0).stop 2023-10-11T09:50:33.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.081+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcdb95e400 msgr2=0x55bcdb711180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:33.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.081+0000 7f9b5d790840 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcdb95e400 0x55bcdb711180 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x55bcdb95d500 tx=0x55bcdb956c00 comp rx=0 tx=0).stop 2023-10-11T09:50:33.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.081+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 shutdown_connections 2023-10-11T09:50:33.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.081+0000 7f9b5d790840 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55bcdbabe000 0x55bcdb711700 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.081+0000 7f9b5d790840 1 --2- 172.21.15.78:0/2738478221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcdb95e400 0x55bcdb711180 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.081+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 >> 172.21.15.78:0/2738478221 conn(0x55bcdb668c00 msgr2=0x55bcdb737800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:33.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.081+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 shutdown_connections 2023-10-11T09:50:33.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.081+0000 7f9b5d790840 1 -- 172.21.15.78:0/2738478221 wait complete. 2023-10-11T09:50:33.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.085+0000 7f9b5d790840 1 -- 172.21.15.78:0/2902393064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcdb669000 msgr2=0x55bcdb70f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:33.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.085+0000 7f9b5d790840 1 --2- 172.21.15.78:0/2902393064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcdb669000 0x55bcdb70f080 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x55bcdc3d1410 tx=0x55bcdc4e10e0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.085+0000 7f9b5d790840 1 -- 172.21.15.78:0/2902393064 shutdown_connections 2023-10-11T09:50:33.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.085+0000 7f9b5d790840 1 --2- 172.21.15.78:0/2902393064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcdb669000 0x55bcdb70f080 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.085+0000 7f9b5d790840 1 --2- 172.21.15.78:0/2902393064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bcdb669400 0x55bcdb70eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.085+0000 7f9b5d790840 1 --2- 172.21.15.78:0/2902393064 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55bcdb669800 0x55bcdb70e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.085+0000 7f9b5d790840 1 -- 172.21.15.78:0/2902393064 >> 172.21.15.78:0/2902393064 conn(0x55bcdb669c00 msgr2=0x55bcdb737000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:33.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.085+0000 7f9b5d790840 1 -- 172.21.15.78:0/2902393064 shutdown_connections 2023-10-11T09:50:33.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.085+0000 7f9b5d790840 1 -- 172.21.15.78:0/2902393064 wait complete. 2023-10-11T09:50:33.102 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes'] in time 0:00:00.260582, result: 2023-10-11T09:50:33.102 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2023-10-11T09:50:33.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.161+0000 7fb80f3dc840 1 -- start start 2023-10-11T09:50:33.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.161+0000 7fb80f3dc840 1 Processor -- start 2023-10-11T09:50:33.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.165+0000 7fb80f3dc840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d17c03800 0x564d17ca8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:33.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.165+0000 7fb80f3dc840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564d17c03400 0x564d17ca8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:33.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.165+0000 7fb80f3dc840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d17c03000 0x564d17ca9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:33.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.165+0000 7fb80f3dc840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d17c82a00 con 0x564d17c03800 2023-10-11T09:50:33.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.165+0000 7fb80f3dc840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d17c82780 con 0x564d17c03000 2023-10-11T09:50:33.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.165+0000 7fb80f3dc840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x564d17c82640 con 0x564d17c03400 2023-10-11T09:50:33.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.165+0000 7fb80db98700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d17c03800 0x564d17ca8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:33.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.165+0000 7fb80db98700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d17c03800 0x564d17ca8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36150/0 (socket says 172.21.15.78:36150) 2023-10-11T09:50:33.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.165+0000 7fb80db98700 1 -- 172.21.15.78:0/3189362571 learned_addr learned my addr 172.21.15.78:0/3189362571 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:33.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.165+0000 7fb80d397700 1 --2- 172.21.15.78:0/3189362571 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564d17c03400 0x564d17ca8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:33.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.165+0000 7fb80e399700 1 --2- 172.21.15.78:0/3189362571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d17c03000 0x564d17ca9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:33.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.165+0000 7fb80db98700 1 -- 172.21.15.78:0/3189362571 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564d17c03400 msgr2=0x564d17ca8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:33.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.165+0000 7fb80db98700 1 --2- 172.21.15.78:0/3189362571 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564d17c03400 0x564d17ca8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.165+0000 7fb80db98700 1 -- 172.21.15.78:0/3189362571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d17c03000 msgr2=0x564d17ca9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:33.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.165+0000 7fb80db98700 1 --2- 172.21.15.78:0/3189362571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d17c03000 0x564d17ca9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.165+0000 7fb80db98700 1 -- 172.21.15.78:0/3189362571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d17c24b60 con 0x564d17c03800 2023-10-11T09:50:33.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.165+0000 7fb80d397700 1 --2- 172.21.15.78:0/3189362571 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564d17c03400 0x564d17ca8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:50:33.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.165+0000 7fb80db98700 1 --2- 172.21.15.78:0/3189362571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d17c03800 0x564d17ca8580 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x564d17cd1380 tx=0x564d18a00d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d19001ab4493ee6 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:33.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.165+0000 7fb80b393700 1 -- 172.21.15.78:0/3189362571 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564d17c33600 con 0x564d17c03800 2023-10-11T09:50:33.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.165+0000 7fb80b393700 1 -- 172.21.15.78:0/3189362571 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564d17c4e480 con 0x564d17c03800 2023-10-11T09:50:33.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.165+0000 7fb80b393700 1 -- 172.21.15.78:0/3189362571 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564d17c331e0 con 0x564d17c03800 2023-10-11T09:50:33.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.165+0000 7fb80f3dc840 1 -- 172.21.15.78:0/3189362571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564d17c24340 con 0x564d17c03800 2023-10-11T09:50:33.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.165+0000 7fb80f3dc840 1 -- 172.21.15.78:0/3189362571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x564d17c24000 con 0x564d17c03800 2023-10-11T09:50:33.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.165+0000 7fb80f3dc840 4 waiting for MDS map... 2023-10-11T09:50:33.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.165+0000 7fb80b393700 1 -- 172.21.15.78:0/3189362571 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(110..110 src has 1..110) v4 ==== 6638+0+0 (secure 0 0 0) 0x564d17c3cc40 con 0x564d17c03800 2023-10-11T09:50:33.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.165+0000 7fb80b393700 1 -- 172.21.15.78:0/3189362571 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 85) v1 ==== 3118+0+0 (secure 0 0 0) 0x564d189f8000 con 0x564d17c03800 2023-10-11T09:50:33.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.165+0000 7fb80f3dc840 4 Got MDS map 85 2023-10-11T09:50:33.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.165+0000 7fb80f3dc840 4 datascan.main: Using metadata pool output 2023-10-11T09:50:33.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.165+0000 7fb80f3dc840 4 datascan.main: connecting to RADOS... 2023-10-11T09:50:33.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.169+0000 7fb80f3dc840 1 Processor -- start 2023-10-11T09:50:33.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.169+0000 7fb80f3dc840 1 -- start start 2023-10-11T09:50:33.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.169+0000 7fb80f3dc840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d189fe400 0x564d17ca9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:33.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.169+0000 7fb80f3dc840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d189fe800 0x564d17ca9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:33.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.169+0000 7fb80f3dc840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564d189fec00 0x564d17caa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:33.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.169+0000 7fb80f3dc840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d17c39e00 con 0x564d189fe800 2023-10-11T09:50:33.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.169+0000 7fb80f3dc840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d17c82780 con 0x564d189fe400 2023-10-11T09:50:33.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.169+0000 7fb80f3dc840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x564d17c82500 con 0x564d189fec00 2023-10-11T09:50:33.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.169+0000 7fb80db98700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d189fe800 0x564d17ca9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:33.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.169+0000 7fb80db98700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d189fe800 0x564d17ca9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36152/0 (socket says 172.21.15.78:36152) 2023-10-11T09:50:33.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.169+0000 7fb80db98700 1 -- 172.21.15.78:0/3748595906 learned_addr learned my addr 172.21.15.78:0/3748595906 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:33.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.169+0000 7fb80e399700 1 --2- 172.21.15.78:0/3748595906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d189fe400 0x564d17ca9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:33.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.169+0000 7fb80d397700 1 --2- 172.21.15.78:0/3748595906 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564d189fec00 0x564d17caa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:33.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.169+0000 7fb80db98700 1 -- 172.21.15.78:0/3748595906 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564d189fec00 msgr2=0x564d17caa100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:33.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.169+0000 7fb80db98700 1 --2- 172.21.15.78:0/3748595906 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564d189fec00 0x564d17caa100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.169+0000 7fb80db98700 1 -- 172.21.15.78:0/3748595906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d189fe400 msgr2=0x564d17ca9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:33.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.169+0000 7fb80db98700 1 --2- 172.21.15.78:0/3748595906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d189fe400 0x564d17ca9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.169+0000 7fb80db98700 1 -- 172.21.15.78:0/3748595906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d17c24ea0 con 0x564d189fe800 2023-10-11T09:50:33.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.169+0000 7fb80db98700 1 --2- 172.21.15.78:0/3748595906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d189fe800 0x564d17ca9b80 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x564d17ef6810 tx=0x564d18a00fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b13cf8216fd6cc server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:33.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.169+0000 7fb80938f700 1 -- 172.21.15.78:0/3748595906 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564d17c32f20 con 0x564d189fe800 2023-10-11T09:50:33.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.169+0000 7fb80938f700 1 -- 172.21.15.78:0/3748595906 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564d189e6f00 con 0x564d189fe800 2023-10-11T09:50:33.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.169+0000 7fb80938f700 1 -- 172.21.15.78:0/3748595906 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564d17c33600 con 0x564d189fe800 2023-10-11T09:50:33.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.169+0000 7fb80f3dc840 1 -- 172.21.15.78:0/3748595906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d189fe800 msgr2=0x564d17ca9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:33.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.169+0000 7fb80f3dc840 1 --2- 172.21.15.78:0/3748595906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d189fe800 0x564d17ca9b80 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x564d17ef6810 tx=0x564d18a00fc0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.169+0000 7fb80f3dc840 1 -- 172.21.15.78:0/3748595906 shutdown_connections 2023-10-11T09:50:33.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.169+0000 7fb80f3dc840 1 --2- 172.21.15.78:0/3748595906 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564d189fec00 0x564d17caa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.169+0000 7fb80f3dc840 1 --2- 172.21.15.78:0/3748595906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d189fe800 0x564d17ca9b80 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.169+0000 7fb80f3dc840 1 --2- 172.21.15.78:0/3748595906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d189fe400 0x564d17ca9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.169+0000 7fb80f3dc840 1 -- 172.21.15.78:0/3748595906 >> 172.21.15.78:0/3748595906 conn(0x564d17c02c00 msgr2=0x564d17cf3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:33.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.173+0000 7fb80f3dc840 1 -- 172.21.15.78:0/3748595906 shutdown_connections 2023-10-11T09:50:33.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.173+0000 7fb80f3dc840 1 -- 172.21.15.78:0/3748595906 wait complete. 2023-10-11T09:50:33.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.173+0000 7fb80f3dc840 1 Processor -- start 2023-10-11T09:50:33.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.173+0000 7fb80f3dc840 1 -- start start 2023-10-11T09:50:33.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.173+0000 7fb80f3dc840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d189fe400 0x564d17cab180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:33.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.173+0000 7fb80f3dc840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d189fe800 0x564d17caac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:33.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.173+0000 7fb80f3dc840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564d189fec00 0x564d17caa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:33.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.173+0000 7fb80f3dc840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d17c823c0 con 0x564d189fe400 2023-10-11T09:50:33.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.173+0000 7fb80f3dc840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d17c82280 con 0x564d189fe800 2023-10-11T09:50:33.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.173+0000 7fb80f3dc840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x564d17c82140 con 0x564d189fec00 2023-10-11T09:50:33.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.173+0000 7fb80e399700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d189fe400 0x564d17cab180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:33.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.173+0000 7fb80e399700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d189fe400 0x564d17cab180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36156/0 (socket says 172.21.15.78:36156) 2023-10-11T09:50:33.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.173+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 learned_addr learned my addr 172.21.15.78:0/2239557512 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:33.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.173+0000 7fb80db98700 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d189fe800 0x564d17caac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:33.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.173+0000 7fb80d397700 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564d189fec00 0x564d17caa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:33.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.173+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564d189fec00 msgr2=0x564d17caa680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:33.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.173+0000 7fb80e399700 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564d189fec00 0x564d17caa680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.173+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d189fe800 msgr2=0x564d17caac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:33.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.173+0000 7fb80e399700 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d189fe800 0x564d17caac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.173+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d17c24b60 con 0x564d189fe400 2023-10-11T09:50:33.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.173+0000 7fb80db98700 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d189fe800 0x564d17caac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:50:33.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.173+0000 7fb80e399700 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d189fe400 0x564d17cab180 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x564d17ef7680 tx=0x564d17cd9500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:33.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.177+0000 7fb807b8c700 1 -- 172.21.15.78:0/2239557512 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564d17c33080 con 0x564d189fe400 2023-10-11T09:50:33.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.177+0000 7fb807b8c700 1 -- 172.21.15.78:0/2239557512 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564d189e6180 con 0x564d189fe400 2023-10-11T09:50:33.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.177+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564d17c24340 con 0x564d189fe400 2023-10-11T09:50:33.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.177+0000 7fb807b8c700 1 -- 172.21.15.78:0/2239557512 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564d17c32f20 con 0x564d189fe400 2023-10-11T09:50:33.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.177+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564d17c25520 con 0x564d189fe400 2023-10-11T09:50:33.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.177+0000 7fb80f3dc840 4 datascan.init: resolving metadata pool 14 2023-10-11T09:50:33.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.177+0000 7fb807b8c700 1 -- 172.21.15.78:0/2239557512 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x564d17c60840 con 0x564d189fe400 2023-10-11T09:50:33.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.177+0000 7fb807b8c700 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x564d18044000 0x564d17cab700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:33.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.177+0000 7fb807b8c700 1 -- 172.21.15.78:0/2239557512 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6638+0+0 (secure 0 0 0) 0x564d17c3ca80 con 0x564d189fe400 2023-10-11T09:50:33.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.181+0000 7fb80f3dc840 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-10-11T09:50:33.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.181+0000 7fb80f3dc840 4 datascan.main: resolving metadata pool 14 2023-10-11T09:50:33.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.181+0000 7fb80f3dc840 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x564d189ff000 0x564d17caa100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:33.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.181+0000 7fb80db98700 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x564d18044000 0x564d17cab700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:33.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.181+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:1 14.0 14:00000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d189ff400 con 0x564d189ff000 2023-10-11T09:50:33.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.181+0000 7fb80d397700 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x564d189ff000 0x564d17caa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:33.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.181+0000 7fb80db98700 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x564d18044000 0x564d17cab700 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x564d17ef6810 tx=0x564d18a00f60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:33.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.181+0000 7fb80d397700 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x564d189ff000 0x564d17caa100 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:33.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.181+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 <== osd.2 v2:172.21.15.78:6808/35732 1 ==== osd_op_reply(1 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8cfc0 con 0x564d189ff000 2023-10-11T09:50:33.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.181+0000 7fb80d397700 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x564d189ff400 0x564d18056000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:33.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.181+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:2 14.10 14:08000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18044400 con 0x564d189ff400 2023-10-11T09:50:33.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.181+0000 7fb80e399700 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x564d189ff400 0x564d18056000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:33.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.181+0000 7fb80e399700 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x564d189ff400 0x564d18056000 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:33.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.185+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 <== osd.1 v2:172.21.15.78:6809/35734 1 ==== osd_op_reply(2 [pgnls start_epoch 110 out=81b] v110'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564d18a8d200 con 0x564d189ff400 2023-10-11T09:50:33.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.185+0000 7fb80e399700 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x564d18044400 0x564d18056580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:33.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.185+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:3 14.8 14:10000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18044800 con 0x564d18044400 2023-10-11T09:50:33.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.185+0000 7fb80db98700 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x564d18044400 0x564d18056580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:33.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.185+0000 7fb80db98700 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x564d18044400 0x564d18056580 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:33.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.185+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 <== osd.0 v2:172.21.15.78:6800/35735 1 ==== osd_op_reply(3 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8d440 con 0x564d18044400 2023-10-11T09:50:33.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.185+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:4 14.18 14:18000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18044800 con 0x564d189ff400 2023-10-11T09:50:33.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.185+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 <== osd.1 v2:172.21.15.78:6809/35734 2 ==== osd_op_reply(4 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8d200 con 0x564d189ff400 2023-10-11T09:50:33.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.185+0000 7fb80e399700 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x564d18044800 0x564d18056b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:33.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.185+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:5 14.4 14:20000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18044c00 con 0x564d18044800 2023-10-11T09:50:33.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.185+0000 7fb80d397700 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x564d18044800 0x564d18056b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:33.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.189+0000 7fb80d397700 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x564d18044800 0x564d18056b00 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:33.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.189+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== osd_op_reply(5 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8cfc0 con 0x564d18044800 2023-10-11T09:50:33.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.189+0000 7fb80d397700 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x564d18044c00 0x564d18057600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:33.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.189+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:6 14.14 14:28000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18045000 con 0x564d18044c00 2023-10-11T09:50:33.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.189+0000 7fb80e399700 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x564d18044c00 0x564d18057600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:33.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.189+0000 7fb80e399700 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x564d18044c00 0x564d18057600 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:33.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.189+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 <== osd.4 v2:172.21.15.133:6808/35898 1 ==== osd_op_reply(6 [pgnls start_epoch 110 out=125b] v110'52 uv52 ondisk = 1) v8 ==== 144+0+125 (crc 0 0 0) 0x564d18a8d200 con 0x564d18044c00 2023-10-11T09:50:33.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.193+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:7 14.c 14:30000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18045000 con 0x564d18044400 2023-10-11T09:50:33.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.193+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 <== osd.0 v2:172.21.15.78:6800/35735 2 ==== osd_op_reply(7 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8d440 con 0x564d18044400 2023-10-11T09:50:33.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.193+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:8 14.1c 14:38000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18045000 con 0x564d18044400 2023-10-11T09:50:33.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.193+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 <== osd.0 v2:172.21.15.78:6800/35735 3 ==== osd_op_reply(8 [pgnls start_epoch 110 out=49b] v109'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8d440 con 0x564d18044400 2023-10-11T09:50:33.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.193+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:9 14.2 14:40000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18045000 con 0x564d18044800 2023-10-11T09:50:33.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.193+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 <== osd.6 v2:172.21.15.133:6800/35900 2 ==== osd_op_reply(9 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8cfc0 con 0x564d18044800 2023-10-11T09:50:33.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.193+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:10 14.12 14:48000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18045000 con 0x564d189ff000 2023-10-11T09:50:33.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.193+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 <== osd.2 v2:172.21.15.78:6808/35732 2 ==== osd_op_reply(10 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8cfc0 con 0x564d189ff000 2023-10-11T09:50:33.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.193+0000 7fb80d397700 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x564d18045000 0x564d18057b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:33.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.193+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:11 14.a 14:50000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18045400 con 0x564d18045000 2023-10-11T09:50:33.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.197+0000 7fb80db98700 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x564d18045000 0x564d18057b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:33.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.197+0000 7fb80db98700 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x564d18045000 0x564d18057b80 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:33.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.197+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 <== osd.5 v2:172.21.15.133:6816/35899 1 ==== osd_op_reply(11 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8d440 con 0x564d18045000 2023-10-11T09:50:33.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.197+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:12 14.1a 14:58000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18045400 con 0x564d18044400 2023-10-11T09:50:33.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.197+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 <== osd.0 v2:172.21.15.78:6800/35735 4 ==== osd_op_reply(12 [pgnls start_epoch 110 out=49b] v110'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8d440 con 0x564d18044400 2023-10-11T09:50:33.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.197+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:13 14.6 14:60000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18045400 con 0x564d18044400 2023-10-11T09:50:33.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.197+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 <== osd.0 v2:172.21.15.78:6800/35735 5 ==== osd_op_reply(13 [pgnls start_epoch 110 out=49b] v109'8 uv8 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8d440 con 0x564d18044400 2023-10-11T09:50:33.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.197+0000 7fb80db98700 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x564d18045400 0x564d18058100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:33.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.197+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:14 14.16 14:68000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18045800 con 0x564d18045400 2023-10-11T09:50:33.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.197+0000 7fb80d397700 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x564d18045400 0x564d18058100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:33.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.201+0000 7fb80d397700 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x564d18045400 0x564d18058100 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:33.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.201+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(14 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8cfc0 con 0x564d18045400 2023-10-11T09:50:33.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.201+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:15 14.e 14:70000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18045800 con 0x564d18044c00 2023-10-11T09:50:33.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.201+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 <== osd.4 v2:172.21.15.133:6808/35898 2 ==== osd_op_reply(15 [pgnls start_epoch 110 out=49b] v109'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8d200 con 0x564d18044c00 2023-10-11T09:50:33.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.201+0000 7fb80e399700 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x564d18045800 0x564d18057080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:33.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.201+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:16 14.1e 14:78000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18080000 con 0x564d18045800 2023-10-11T09:50:33.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.201+0000 7fb80e399700 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x564d18045800 0x564d18057080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:33.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.201+0000 7fb80e399700 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x564d18045800 0x564d18057080 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:33.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.205+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 <== osd.7 v2:172.21.15.133:6824/35901 1 ==== osd_op_reply(16 [pgnls start_epoch 110 out=49b] v110'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8d200 con 0x564d18045800 2023-10-11T09:50:33.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.205+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:17 14.1 14:80000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18080000 con 0x564d18045400 2023-10-11T09:50:33.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.205+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 <== osd.3 v2:172.21.15.78:6824/35738 2 ==== osd_op_reply(17 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8cfc0 con 0x564d18045400 2023-10-11T09:50:33.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.205+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:18 14.11 14:88000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18080000 con 0x564d18044c00 2023-10-11T09:50:33.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.205+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 <== osd.4 v2:172.21.15.133:6808/35898 3 ==== osd_op_reply(18 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8d200 con 0x564d18044c00 2023-10-11T09:50:33.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.205+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:19 14.9 14:90000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18080000 con 0x564d18045000 2023-10-11T09:50:33.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.205+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 <== osd.5 v2:172.21.15.133:6816/35899 2 ==== osd_op_reply(19 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8d440 con 0x564d18045000 2023-10-11T09:50:33.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.205+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:20 14.19 14:98000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18080000 con 0x564d18044800 2023-10-11T09:50:33.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.209+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 <== osd.6 v2:172.21.15.133:6800/35900 3 ==== osd_op_reply(20 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8cfc0 con 0x564d18044800 2023-10-11T09:50:33.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.209+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:21 14.5 14:a0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18080000 con 0x564d18044400 2023-10-11T09:50:33.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.209+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 <== osd.0 v2:172.21.15.78:6800/35735 6 ==== osd_op_reply(21 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8d440 con 0x564d18044400 2023-10-11T09:50:33.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.209+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:22 14.15 14:a8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18080000 con 0x564d18045800 2023-10-11T09:50:33.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.209+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 <== osd.7 v2:172.21.15.133:6824/35901 2 ==== osd_op_reply(22 [pgnls start_epoch 110 out=73b] v109'6 uv6 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x564d18a8d200 con 0x564d18045800 2023-10-11T09:50:33.218 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.209+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:23 14.d 14:b0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18080000 con 0x564d18044800 2023-10-11T09:50:33.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.209+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 <== osd.6 v2:172.21.15.133:6800/35900 4 ==== osd_op_reply(23 [pgnls start_epoch 110 out=74b] v110'3 uv3 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x564d18a8cfc0 con 0x564d18044800 2023-10-11T09:50:33.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.209+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:24 14.1d 14:b8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18080000 con 0x564d18044400 2023-10-11T09:50:33.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.209+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 <== osd.0 v2:172.21.15.78:6800/35735 7 ==== osd_op_reply(24 [pgnls start_epoch 110 out=49b] v109'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8d440 con 0x564d18044400 2023-10-11T09:50:33.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.209+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:25 14.3 14:c0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18080000 con 0x564d18044400 2023-10-11T09:50:33.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.213+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 <== osd.0 v2:172.21.15.78:6800/35735 8 ==== osd_op_reply(25 [pgnls start_epoch 110 out=71b] v110'2 uv2 ondisk = 1) v8 ==== 144+0+71 (crc 0 0 0) 0x564d18a8d440 con 0x564d18044400 2023-10-11T09:50:33.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.213+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:26 14.13 14:c8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18080000 con 0x564d189ff000 2023-10-11T09:50:33.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.213+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 <== osd.2 v2:172.21.15.78:6808/35732 3 ==== osd_op_reply(26 [pgnls start_epoch 110 out=98b] v110'7 uv7 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x564d18a8cfc0 con 0x564d189ff000 2023-10-11T09:50:33.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.213+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:27 14.b 14:d0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18080000 con 0x564d18044400 2023-10-11T09:50:33.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.213+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 <== osd.0 v2:172.21.15.78:6800/35735 9 ==== osd_op_reply(27 [pgnls start_epoch 110 out=76b] v110'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x564d18a8d440 con 0x564d18044400 2023-10-11T09:50:33.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.213+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:28 14.1b 14:d8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18080000 con 0x564d189ff400 2023-10-11T09:50:33.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.213+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 <== osd.1 v2:172.21.15.78:6809/35734 3 ==== osd_op_reply(28 [pgnls start_epoch 110 out=49b] v110'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8d200 con 0x564d189ff400 2023-10-11T09:50:33.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.213+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:29 14.7 14:e0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18080000 con 0x564d18045000 2023-10-11T09:50:33.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.213+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 <== osd.5 v2:172.21.15.133:6816/35899 3 ==== osd_op_reply(29 [pgnls start_epoch 110 out=49b] v109'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8d440 con 0x564d18045000 2023-10-11T09:50:33.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.213+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:30 14.17 14:e8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18080000 con 0x564d189ff400 2023-10-11T09:50:33.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.217+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 <== osd.1 v2:172.21.15.78:6809/35734 4 ==== osd_op_reply(30 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8d200 con 0x564d189ff400 2023-10-11T09:50:33.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.217+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:31 14.f 14:f0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18080000 con 0x564d18045800 2023-10-11T09:50:33.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.217+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 <== osd.7 v2:172.21.15.133:6824/35901 3 ==== osd_op_reply(31 [pgnls start_epoch 110 out=73b] v110'4 uv4 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x564d18a8d200 con 0x564d18045800 2023-10-11T09:50:33.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.217+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:32 14.1f 14:f8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18080000 con 0x564d18044400 2023-10-11T09:50:33.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.217+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 <== osd.0 v2:172.21.15.78:6800/35735 10 ==== osd_op_reply(32 [pgnls start_epoch 110 out=101b] v110'13 uv13 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x564d18a8d440 con 0x564d18044400 2023-10-11T09:50:33.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.217+0000 7fb80f3dc840 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2023-10-11T09:50:33.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.217+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:33 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d189ff800 con 0x564d189ff400 2023-10-11T09:50:33.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.217+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 <== osd.1 v2:172.21.15.78:6809/35734 5 ==== osd_op_reply(33 10000000000.00000000 [omap-get-vals out=3582b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+3582 (crc 0 0 0) 0x564d18a8d200 con 0x564d189ff400 2023-10-11T09:50:33.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.217+0000 7fb80f3dc840 20 datascan.scan_links: injected ino detected 2023-10-11T09:50:33.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.217+0000 7fb80f3dc840 20 datascan.scan_links: injected ino detected 2023-10-11T09:50:33.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.217+0000 7fb80f3dc840 20 datascan.scan_links: injected ino detected 2023-10-11T09:50:33.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.217+0000 7fb80f3dc840 20 datascan.scan_links: injected ino detected 2023-10-11T09:50:33.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.217+0000 7fb80f3dc840 20 datascan.scan_links: injected ino detected 2023-10-11T09:50:33.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.217+0000 7fb80f3dc840 20 datascan.scan_links: injected ino detected 2023-10-11T09:50:33.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.217+0000 7fb80f3dc840 20 datascan.scan_links: injected ino detected 2023-10-11T09:50:33.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.217+0000 7fb80f3dc840 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2023-10-11T09:50:33.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.217+0000 7fb80f3dc840 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-10-11T09:50:33.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.217+0000 7fb80f3dc840 10 datascan.scan_links: step 1: handling object 200.00000000 2023-10-11T09:50:33.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.217+0000 7fb80f3dc840 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-10-11T09:50:33.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.217+0000 7fb80f3dc840 10 datascan.scan_links: step 1: handling object 200.00000001 2023-10-11T09:50:33.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.217+0000 7fb80f3dc840 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-10-11T09:50:33.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.217+0000 7fb80f3dc840 10 datascan.scan_links: step 1: handling object 500.00000000 2023-10-11T09:50:33.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.217+0000 7fb80f3dc840 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-10-11T09:50:33.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.217+0000 7fb80f3dc840 10 datascan.scan_links: step 1: handling object mds_snaptable 2023-10-11T09:50:33.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.217+0000 7fb80f3dc840 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-10-11T09:50:33.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.217+0000 7fb80f3dc840 10 datascan.scan_links: step 1: handling object 4.00000000 2023-10-11T09:50:33.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.217+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:34 14.3 14:c0ce80d4:::4.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18080c00 con 0x564d18044400 2023-10-11T09:50:33.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.217+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 <== osd.0 v2:172.21.15.78:6800/35735 11 ==== osd_op_reply(34 4.00000000 [omap-get-vals out=513b] v0'0 uv2 ondisk = 0) v8 ==== 154+0+513 (crc 0 0 0) 0x564d18a8d440 con 0x564d18044400 2023-10-11T09:50:33.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.221+0000 7fb80f3dc840 20 datascan.scan_links: injected ino detected 2023-10-11T09:50:33.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.221+0000 7fb80f3dc840 10 datascan.scan_links: step 1: handling object mds0_inotable 2023-10-11T09:50:33.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.221+0000 7fb80f3dc840 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-10-11T09:50:33.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.221+0000 7fb80f3dc840 10 datascan.scan_links: step 1: handling object 100.00000000 2023-10-11T09:50:33.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.221+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:35 14.13 14:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081000 con 0x564d189ff000 2023-10-11T09:50:33.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.221+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 <== osd.2 v2:172.21.15.78:6808/35732 4 ==== osd_op_reply(35 100.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x564d18a8cfc0 con 0x564d189ff000 2023-10-11T09:50:33.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.221+0000 7fb80f3dc840 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2023-10-11T09:50:33.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.221+0000 7fb80f3dc840 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-10-11T09:50:33.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.221+0000 7fb80f3dc840 10 datascan.scan_links: step 1: handling object 400.00000000 2023-10-11T09:50:33.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.221+0000 7fb80f3dc840 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-10-11T09:50:33.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.221+0000 7fb80f3dc840 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2023-10-11T09:50:33.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.221+0000 7fb80f3dc840 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-10-11T09:50:33.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.221+0000 7fb80f3dc840 10 datascan.scan_links: step 1: handling object 1.00000000 2023-10-11T09:50:33.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.221+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:36 14.1f 14:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081400 con 0x564d18044400 2023-10-11T09:50:33.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.221+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 <== osd.0 v2:172.21.15.78:6800/35735 12 ==== osd_op_reply(36 1.00000000 [omap-get-vals out=1016b] v0'0 uv13 ondisk = 0) v8 ==== 154+0+1016 (crc 0 0 0) 0x564d18a8d440 con 0x564d18044400 2023-10-11T09:50:33.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.221+0000 7fb80f3dc840 20 datascan.scan_links: injected ino detected 2023-10-11T09:50:33.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.221+0000 7fb80f3dc840 20 datascan.scan_links: injected ino detected 2023-10-11T09:50:33.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.221+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:37 14.0 14:00000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081800 con 0x564d189ff000 2023-10-11T09:50:33.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.221+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 <== osd.2 v2:172.21.15.78:6808/35732 5 ==== osd_op_reply(37 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8cfc0 con 0x564d189ff000 2023-10-11T09:50:33.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.221+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:38 14.10 14:08000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081800 con 0x564d189ff400 2023-10-11T09:50:33.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.221+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 <== osd.1 v2:172.21.15.78:6809/35734 6 ==== osd_op_reply(38 [pgnls start_epoch 110 out=81b] v110'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564d18a8d200 con 0x564d189ff400 2023-10-11T09:50:33.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.221+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:39 14.8 14:10000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081800 con 0x564d18044400 2023-10-11T09:50:33.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.225+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 <== osd.0 v2:172.21.15.78:6800/35735 13 ==== osd_op_reply(39 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8d440 con 0x564d18044400 2023-10-11T09:50:33.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.225+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:40 14.18 14:18000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081800 con 0x564d189ff400 2023-10-11T09:50:33.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.225+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 <== osd.1 v2:172.21.15.78:6809/35734 7 ==== osd_op_reply(40 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8d200 con 0x564d189ff400 2023-10-11T09:50:33.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.225+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:41 14.4 14:20000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081800 con 0x564d18044800 2023-10-11T09:50:33.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.225+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 <== osd.6 v2:172.21.15.133:6800/35900 5 ==== osd_op_reply(41 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8cfc0 con 0x564d18044800 2023-10-11T09:50:33.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.225+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:42 14.14 14:28000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081800 con 0x564d18044c00 2023-10-11T09:50:33.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.225+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 <== osd.4 v2:172.21.15.133:6808/35898 4 ==== osd_op_reply(42 [pgnls start_epoch 110 out=125b] v110'52 uv52 ondisk = 1) v8 ==== 144+0+125 (crc 0 0 0) 0x564d18a8d200 con 0x564d18044c00 2023-10-11T09:50:33.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.225+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:43 14.c 14:30000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081800 con 0x564d18044400 2023-10-11T09:50:33.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.225+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 <== osd.0 v2:172.21.15.78:6800/35735 14 ==== osd_op_reply(43 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8d440 con 0x564d18044400 2023-10-11T09:50:33.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.225+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:44 14.1c 14:38000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081800 con 0x564d18044400 2023-10-11T09:50:33.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.229+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 <== osd.0 v2:172.21.15.78:6800/35735 15 ==== osd_op_reply(44 [pgnls start_epoch 110 out=49b] v109'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8d440 con 0x564d18044400 2023-10-11T09:50:33.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.229+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:45 14.2 14:40000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081800 con 0x564d18044800 2023-10-11T09:50:33.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.229+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 <== osd.6 v2:172.21.15.133:6800/35900 6 ==== osd_op_reply(45 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8cfc0 con 0x564d18044800 2023-10-11T09:50:33.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.229+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:46 14.12 14:48000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081800 con 0x564d189ff000 2023-10-11T09:50:33.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.229+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 <== osd.2 v2:172.21.15.78:6808/35732 6 ==== osd_op_reply(46 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8cfc0 con 0x564d189ff000 2023-10-11T09:50:33.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.229+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:47 14.a 14:50000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081800 con 0x564d18045000 2023-10-11T09:50:33.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.229+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 <== osd.5 v2:172.21.15.133:6816/35899 4 ==== osd_op_reply(47 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8d440 con 0x564d18045000 2023-10-11T09:50:33.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.229+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:48 14.1a 14:58000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081800 con 0x564d18044400 2023-10-11T09:50:33.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.229+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 <== osd.0 v2:172.21.15.78:6800/35735 16 ==== osd_op_reply(48 [pgnls start_epoch 110 out=49b] v110'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8d440 con 0x564d18044400 2023-10-11T09:50:33.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.229+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:49 14.6 14:60000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081800 con 0x564d18044400 2023-10-11T09:50:33.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.233+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 <== osd.0 v2:172.21.15.78:6800/35735 17 ==== osd_op_reply(49 [pgnls start_epoch 110 out=49b] v109'8 uv8 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8d440 con 0x564d18044400 2023-10-11T09:50:33.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.233+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:50 14.16 14:68000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081800 con 0x564d18045400 2023-10-11T09:50:33.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.233+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 <== osd.3 v2:172.21.15.78:6824/35738 3 ==== osd_op_reply(50 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8cfc0 con 0x564d18045400 2023-10-11T09:50:33.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.233+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:51 14.e 14:70000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081800 con 0x564d18044c00 2023-10-11T09:50:33.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.233+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 <== osd.4 v2:172.21.15.133:6808/35898 5 ==== osd_op_reply(51 [pgnls start_epoch 110 out=49b] v109'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8d200 con 0x564d18044c00 2023-10-11T09:50:33.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.233+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:52 14.1e 14:78000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081800 con 0x564d18045800 2023-10-11T09:50:33.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.233+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 <== osd.7 v2:172.21.15.133:6824/35901 4 ==== osd_op_reply(52 [pgnls start_epoch 110 out=49b] v110'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8d200 con 0x564d18045800 2023-10-11T09:50:33.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.233+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:53 14.1 14:80000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081800 con 0x564d18045400 2023-10-11T09:50:33.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.237+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 <== osd.3 v2:172.21.15.78:6824/35738 4 ==== osd_op_reply(53 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8cfc0 con 0x564d18045400 2023-10-11T09:50:33.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.237+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:54 14.11 14:88000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081800 con 0x564d18044c00 2023-10-11T09:50:33.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.237+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 <== osd.4 v2:172.21.15.133:6808/35898 6 ==== osd_op_reply(54 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8d200 con 0x564d18044c00 2023-10-11T09:50:33.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.237+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:55 14.9 14:90000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081800 con 0x564d18045000 2023-10-11T09:50:33.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.237+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 <== osd.5 v2:172.21.15.133:6816/35899 5 ==== osd_op_reply(55 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8d440 con 0x564d18045000 2023-10-11T09:50:33.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.237+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:56 14.19 14:98000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081800 con 0x564d18044800 2023-10-11T09:50:33.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.237+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 <== osd.6 v2:172.21.15.133:6800/35900 7 ==== osd_op_reply(56 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8cfc0 con 0x564d18044800 2023-10-11T09:50:33.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.237+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:57 14.5 14:a0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081800 con 0x564d18044400 2023-10-11T09:50:33.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.241+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 <== osd.0 v2:172.21.15.78:6800/35735 18 ==== osd_op_reply(57 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8d440 con 0x564d18044400 2023-10-11T09:50:33.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.241+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:58 14.15 14:a8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081800 con 0x564d18045800 2023-10-11T09:50:33.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.241+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 <== osd.7 v2:172.21.15.133:6824/35901 5 ==== osd_op_reply(58 [pgnls start_epoch 110 out=73b] v109'6 uv6 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x564d18a8d200 con 0x564d18045800 2023-10-11T09:50:33.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.241+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:59 14.d 14:b0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081800 con 0x564d18044800 2023-10-11T09:50:33.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.241+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 <== osd.6 v2:172.21.15.133:6800/35900 8 ==== osd_op_reply(59 [pgnls start_epoch 110 out=74b] v110'3 uv3 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x564d18a8cfc0 con 0x564d18044800 2023-10-11T09:50:33.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.241+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:60 14.1d 14:b8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081800 con 0x564d18044400 2023-10-11T09:50:33.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.241+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 <== osd.0 v2:172.21.15.78:6800/35735 19 ==== osd_op_reply(60 [pgnls start_epoch 110 out=49b] v109'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8d440 con 0x564d18044400 2023-10-11T09:50:33.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.241+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:61 14.3 14:c0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081800 con 0x564d18044400 2023-10-11T09:50:33.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.245+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 <== osd.0 v2:172.21.15.78:6800/35735 20 ==== osd_op_reply(61 [pgnls start_epoch 110 out=71b] v110'2 uv2 ondisk = 1) v8 ==== 144+0+71 (crc 0 0 0) 0x564d18a8d440 con 0x564d18044400 2023-10-11T09:50:33.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.245+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:62 14.13 14:c8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081800 con 0x564d189ff000 2023-10-11T09:50:33.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.245+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 <== osd.2 v2:172.21.15.78:6808/35732 7 ==== osd_op_reply(62 [pgnls start_epoch 110 out=98b] v110'7 uv7 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x564d18a8cfc0 con 0x564d189ff000 2023-10-11T09:50:33.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.245+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:63 14.b 14:d0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081800 con 0x564d18044400 2023-10-11T09:50:33.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.245+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 <== osd.0 v2:172.21.15.78:6800/35735 21 ==== osd_op_reply(63 [pgnls start_epoch 110 out=76b] v110'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x564d18a8d440 con 0x564d18044400 2023-10-11T09:50:33.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.245+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:64 14.1b 14:d8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081800 con 0x564d189ff400 2023-10-11T09:50:33.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.245+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 <== osd.1 v2:172.21.15.78:6809/35734 8 ==== osd_op_reply(64 [pgnls start_epoch 110 out=49b] v110'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8d200 con 0x564d189ff400 2023-10-11T09:50:33.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.245+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:65 14.7 14:e0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081800 con 0x564d18045000 2023-10-11T09:50:33.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.249+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 <== osd.5 v2:172.21.15.133:6816/35899 6 ==== osd_op_reply(65 [pgnls start_epoch 110 out=49b] v109'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8d440 con 0x564d18045000 2023-10-11T09:50:33.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.249+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:66 14.17 14:e8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081800 con 0x564d189ff400 2023-10-11T09:50:33.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.249+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 <== osd.1 v2:172.21.15.78:6809/35734 9 ==== osd_op_reply(66 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564d18a8d200 con 0x564d189ff400 2023-10-11T09:50:33.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.249+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:67 14.f 14:f0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081800 con 0x564d18045800 2023-10-11T09:50:33.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.249+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 <== osd.7 v2:172.21.15.133:6824/35901 6 ==== osd_op_reply(67 [pgnls start_epoch 110 out=73b] v110'4 uv4 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x564d18a8d200 con 0x564d18045800 2023-10-11T09:50:33.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.249+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:68 14.1f 14:f8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081800 con 0x564d18044400 2023-10-11T09:50:33.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.249+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 <== osd.0 v2:172.21.15.78:6800/35735 22 ==== osd_op_reply(68 [pgnls start_epoch 110 out=101b] v110'13 uv13 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x564d18a8d440 con 0x564d18044400 2023-10-11T09:50:33.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.249+0000 7fb80f3dc840 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2023-10-11T09:50:33.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.249+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:69 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081c00 con 0x564d189ff400 2023-10-11T09:50:33.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 <== osd.1 v2:172.21.15.78:6809/35734 10 ==== osd_op_reply(69 10000000000.00000000 [omap-get-vals out=3582b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+3582 (crc 0 0 0) 0x564d18a8d200 con 0x564d189ff400 2023-10-11T09:50:33.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 20 datascan.scan_links: injected ino detected 2023-10-11T09:50:33.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2023-10-11T09:50:33.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 20 datascan.scan_links: injected ino detected 2023-10-11T09:50:33.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 20 datascan.scan_links: adding 0x100000001f6 for future processing to fix dnfirst 2023-10-11T09:50:33.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 20 datascan.scan_links: injected ino detected 2023-10-11T09:50:33.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2023-10-11T09:50:33.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 20 datascan.scan_links: injected ino detected 2023-10-11T09:50:33.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 20 datascan.scan_links: adding 0x100000001f8 for future processing to fix dnfirst 2023-10-11T09:50:33.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 20 datascan.scan_links: injected ino detected 2023-10-11T09:50:33.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 20 datascan.scan_links: adding 0x100000001f9 for future processing to fix dnfirst 2023-10-11T09:50:33.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 20 datascan.scan_links: injected ino detected 2023-10-11T09:50:33.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 20 datascan.scan_links: adding 0x100000001fa for future processing to fix dnfirst 2023-10-11T09:50:33.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 20 datascan.scan_links: injected ino detected 2023-10-11T09:50:33.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 20 datascan.scan_links: adding 0x100000001fb for future processing to fix dnfirst 2023-10-11T09:50:33.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2023-10-11T09:50:33.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-10-11T09:50:33.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 10 datascan.scan_links: step 2: handling object 200.00000000 2023-10-11T09:50:33.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-10-11T09:50:33.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 10 datascan.scan_links: step 2: handling object 200.00000001 2023-10-11T09:50:33.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-10-11T09:50:33.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 10 datascan.scan_links: step 2: handling object 500.00000000 2023-10-11T09:50:33.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-10-11T09:50:33.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 10 datascan.scan_links: step 2: handling object mds_snaptable 2023-10-11T09:50:33.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-10-11T09:50:33.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 10 datascan.scan_links: step 2: handling object 4.00000000 2023-10-11T09:50:33.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:70 14.3 14:c0ce80d4:::4.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18080000 con 0x564d18044400 2023-10-11T09:50:33.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 <== osd.0 v2:172.21.15.78:6800/35735 23 ==== osd_op_reply(70 4.00000000 [omap-get-vals out=513b] v0'0 uv2 ondisk = 0) v8 ==== 154+0+513 (crc 0 0 0) 0x564d18a8d440 con 0x564d18044400 2023-10-11T09:50:33.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 20 datascan.scan_links: injected ino detected 2023-10-11T09:50:33.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 20 datascan.scan_links: adding 0x100000001fc for future processing to fix dnfirst 2023-10-11T09:50:33.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 10 datascan.scan_links: step 2: handling object mds0_inotable 2023-10-11T09:50:33.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-10-11T09:50:33.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 10 datascan.scan_links: step 2: handling object 100.00000000 2023-10-11T09:50:33.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:71 14.13 14:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18080c00 con 0x564d189ff000 2023-10-11T09:50:33.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 <== osd.2 v2:172.21.15.78:6808/35732 8 ==== osd_op_reply(71 100.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x564d18a8cfc0 con 0x564d189ff000 2023-10-11T09:50:33.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2023-10-11T09:50:33.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-10-11T09:50:33.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 10 datascan.scan_links: step 2: handling object 400.00000000 2023-10-11T09:50:33.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-10-11T09:50:33.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2023-10-11T09:50:33.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-10-11T09:50:33.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 10 datascan.scan_links: step 2: handling object 1.00000000 2023-10-11T09:50:33.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:72 14.1f 14:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081400 con 0x564d18044400 2023-10-11T09:50:33.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 <== osd.0 v2:172.21.15.78:6800/35735 24 ==== osd_op_reply(72 1.00000000 [omap-get-vals out=1016b] v0'0 uv13 ondisk = 0) v8 ==== 154+0+1016 (crc 0 0 0) 0x564d18a8d440 con 0x564d18044400 2023-10-11T09:50:33.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 20 datascan.scan_links: injected ino detected 2023-10-11T09:50:33.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 20 datascan.scan_links: adding 0x4 for future processing to fix dnfirst 2023-10-11T09:50:33.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 20 datascan.scan_links: injected ino detected 2023-10-11T09:50:33.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 20 datascan.scan_links: adding 0x10000000000 for future processing to fix dnfirst 2023-10-11T09:50:33.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2023-10-11T09:50:33.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 10 datascan.scan_links: removing dup dentries from 0 objects 2023-10-11T09:50:33.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 10 datascan.scan_links: processing 0 bad_nlink_inos 2023-10-11T09:50:33.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 10 datascan.scan_links: processing 10 injected_inos 2023-10-11T09:50:33.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 10 datascan.scan_links: handling injected_ino 0x4 2023-10-11T09:50:33.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x1:lost+found 2023-10-11T09:50:33.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:73 14.1f 14:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=23b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18045c00 con 0x564d18044400 2023-10-11T09:50:33.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 <== osd.0 v2:172.21.15.78:6800/35735 25 ==== osd_op_reply(73 1.00000000 [omap-get-vals-by-keys out=511b] v0'0 uv13 ondisk = 0) v8 ==== 154+0+511 (crc 0 0 0) 0x564d18a8d440 con 0x564d18044400 2023-10-11T09:50:33.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 20 datascan.read_dentry: oid=1.00000000 dname=lost+found frag=*, r=0 2023-10-11T09:50:33.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 20 datascan.scan_links: first is now 2 2023-10-11T09:50:33.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.253+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:74 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d1808e000 con 0x564d18044400 2023-10-11T09:50:33.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.257+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 <== osd.0 v2:172.21.15.78:6800/35735 26 ==== osd_op_reply(74 1.00000000 [omap-set-vals] v110'14 uv14 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x564d18a8d440 con 0x564d18044400 2023-10-11T09:50:33.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.257+0000 7fb80f3dc840 20 datascan.inject_linkage: Injected dentry 0x0x1/lost+found pointing to 0x0x4 2023-10-11T09:50:33.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.257+0000 7fb80f3dc840 10 datascan.scan_links: handling injected_ino 0x10000000000 2023-10-11T09:50:33.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.257+0000 7fb80f3dc840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x1:stripey 2023-10-11T09:50:33.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.257+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:75 14.1f 14:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d1808e400 con 0x564d18044400 2023-10-11T09:50:33.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.257+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 <== osd.0 v2:172.21.15.78:6800/35735 27 ==== osd_op_reply(75 1.00000000 [omap-get-vals-by-keys out=508b] v0'0 uv14 ondisk = 0) v8 ==== 154+0+508 (crc 0 0 0) 0x564d18a8d440 con 0x564d18044400 2023-10-11T09:50:33.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.257+0000 7fb80f3dc840 20 datascan.read_dentry: oid=1.00000000 dname=stripey frag=*, r=0 2023-10-11T09:50:33.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.257+0000 7fb80f3dc840 20 datascan.scan_links: first is now 2 2023-10-11T09:50:33.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.257+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:76 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=508b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d1808e800 con 0x564d18044400 2023-10-11T09:50:33.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.261+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 <== osd.0 v2:172.21.15.78:6800/35735 28 ==== osd_op_reply(76 1.00000000 [omap-set-vals] v110'15 uv15 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x564d18a8d440 con 0x564d18044400 2023-10-11T09:50:33.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.261+0000 7fb80f3dc840 20 datascan.inject_linkage: Injected dentry 0x0x1/stripey pointing to 0x0x10000000000 2023-10-11T09:50:33.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.261+0000 7fb80f3dc840 10 datascan.scan_links: handling injected_ino 0x10000000001 2023-10-11T09:50:33.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.261+0000 7fb80f3dc840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:flushed_file_0 2023-10-11T09:50:33.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.261+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:77 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d1808ec00 con 0x564d189ff400 2023-10-11T09:50:33.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.261+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 <== osd.1 v2:172.21.15.78:6809/35734 11 ==== osd_op_reply(77 10000000000.00000000 [omap-get-vals-by-keys out=515b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+515 (crc 0 0 0) 0x564d18a8d200 con 0x564d189ff400 2023-10-11T09:50:33.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.261+0000 7fb80f3dc840 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_0 frag=*, r=0 2023-10-11T09:50:33.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.261+0000 7fb80f3dc840 20 datascan.scan_links: first is now 2 2023-10-11T09:50:33.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.261+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:78 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d1808f000 con 0x564d189ff400 2023-10-11T09:50:33.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.265+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 <== osd.1 v2:172.21.15.78:6809/35734 12 ==== osd_op_reply(78 10000000000.00000000 [omap-set-vals] v110'24 uv24 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564d18a8d200 con 0x564d189ff400 2023-10-11T09:50:33.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.265+0000 7fb80f3dc840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_0 pointing to 0x0x10000000001 2023-10-11T09:50:33.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.265+0000 7fb80f3dc840 10 datascan.scan_links: handling injected_ino 0x100000001f6 2023-10-11T09:50:33.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.265+0000 7fb80f3dc840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:flushed_file_1 2023-10-11T09:50:33.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.265+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:79 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d1808f400 con 0x564d189ff400 2023-10-11T09:50:33.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.265+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 <== osd.1 v2:172.21.15.78:6809/35734 13 ==== osd_op_reply(79 10000000000.00000000 [omap-get-vals-by-keys out=515b] v0'0 uv24 ondisk = 0) v8 ==== 164+0+515 (crc 0 0 0) 0x564d18a8d200 con 0x564d189ff400 2023-10-11T09:50:33.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.265+0000 7fb80f3dc840 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_1 frag=*, r=0 2023-10-11T09:50:33.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.265+0000 7fb80f3dc840 20 datascan.scan_links: first is now 2 2023-10-11T09:50:33.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.265+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:80 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d1808f800 con 0x564d189ff400 2023-10-11T09:50:33.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.269+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 <== osd.1 v2:172.21.15.78:6809/35734 14 ==== osd_op_reply(80 10000000000.00000000 [omap-set-vals] v110'25 uv25 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564d18a8d200 con 0x564d189ff400 2023-10-11T09:50:33.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.269+0000 7fb80f3dc840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_1 pointing to 0x0x100000001f6 2023-10-11T09:50:33.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.269+0000 7fb80f3dc840 10 datascan.scan_links: handling injected_ino 0x100000001f7 2023-10-11T09:50:33.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.269+0000 7fb80f3dc840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:flushed_file_2 2023-10-11T09:50:33.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.269+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:81 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d1808fc00 con 0x564d189ff400 2023-10-11T09:50:33.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.269+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 <== osd.1 v2:172.21.15.78:6809/35734 15 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals-by-keys out=515b] v0'0 uv25 ondisk = 0) v8 ==== 164+0+515 (crc 0 0 0) 0x564d18a8d200 con 0x564d189ff400 2023-10-11T09:50:33.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.269+0000 7fb80f3dc840 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_2 frag=*, r=0 2023-10-11T09:50:33.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.269+0000 7fb80f3dc840 20 datascan.scan_links: first is now 2 2023-10-11T09:50:33.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.269+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:82 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d1808ec00 con 0x564d189ff400 2023-10-11T09:50:33.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.269+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 <== osd.1 v2:172.21.15.78:6809/35734 16 ==== osd_op_reply(82 10000000000.00000000 [omap-set-vals] v110'26 uv26 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564d18a8d200 con 0x564d189ff400 2023-10-11T09:50:33.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.269+0000 7fb80f3dc840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_2 pointing to 0x0x100000001f7 2023-10-11T09:50:33.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.269+0000 7fb80f3dc840 10 datascan.scan_links: handling injected_ino 0x100000001f8 2023-10-11T09:50:33.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.269+0000 7fb80f3dc840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:flushed_file_3 2023-10-11T09:50:33.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.269+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:83 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d1808e000 con 0x564d189ff400 2023-10-11T09:50:33.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.273+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 <== osd.1 v2:172.21.15.78:6809/35734 17 ==== osd_op_reply(83 10000000000.00000000 [omap-get-vals-by-keys out=515b] v0'0 uv26 ondisk = 0) v8 ==== 164+0+515 (crc 0 0 0) 0x564d18a8d200 con 0x564d189ff400 2023-10-11T09:50:33.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.273+0000 7fb80f3dc840 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_3 frag=*, r=0 2023-10-11T09:50:33.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.273+0000 7fb80f3dc840 20 datascan.scan_links: first is now 2 2023-10-11T09:50:33.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.273+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:84 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18045c00 con 0x564d189ff400 2023-10-11T09:50:33.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.273+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 <== osd.1 v2:172.21.15.78:6809/35734 18 ==== osd_op_reply(84 10000000000.00000000 [omap-set-vals] v110'27 uv27 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564d18a8d200 con 0x564d189ff400 2023-10-11T09:50:33.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.273+0000 7fb80f3dc840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_3 pointing to 0x0x100000001f8 2023-10-11T09:50:33.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.273+0000 7fb80f3dc840 10 datascan.scan_links: handling injected_ino 0x100000001f9 2023-10-11T09:50:33.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.273+0000 7fb80f3dc840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:flushed_file_4 2023-10-11T09:50:33.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.273+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:85 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18080800 con 0x564d189ff400 2023-10-11T09:50:33.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.273+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 <== osd.1 v2:172.21.15.78:6809/35734 19 ==== osd_op_reply(85 10000000000.00000000 [omap-get-vals-by-keys out=515b] v0'0 uv27 ondisk = 0) v8 ==== 164+0+515 (crc 0 0 0) 0x564d18a8d200 con 0x564d189ff400 2023-10-11T09:50:33.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.273+0000 7fb80f3dc840 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_4 frag=*, r=0 2023-10-11T09:50:33.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.273+0000 7fb80f3dc840 20 datascan.scan_links: first is now 2 2023-10-11T09:50:33.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.273+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:86 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18080400 con 0x564d189ff400 2023-10-11T09:50:33.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.277+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 <== osd.1 v2:172.21.15.78:6809/35734 20 ==== osd_op_reply(86 10000000000.00000000 [omap-set-vals] v110'28 uv28 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564d18a8d200 con 0x564d189ff400 2023-10-11T09:50:33.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.277+0000 7fb80f3dc840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_4 pointing to 0x0x100000001f9 2023-10-11T09:50:33.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.277+0000 7fb80f3dc840 10 datascan.scan_links: handling injected_ino 0x100000001fa 2023-10-11T09:50:33.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.277+0000 7fb80f3dc840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:flushed_file_5 2023-10-11T09:50:33.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.277+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:87 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081c00 con 0x564d189ff400 2023-10-11T09:50:33.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.277+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 <== osd.1 v2:172.21.15.78:6809/35734 21 ==== osd_op_reply(87 10000000000.00000000 [omap-get-vals-by-keys out=515b] v0'0 uv28 ondisk = 0) v8 ==== 164+0+515 (crc 0 0 0) 0x564d18a8d200 con 0x564d189ff400 2023-10-11T09:50:33.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.277+0000 7fb80f3dc840 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_5 frag=*, r=0 2023-10-11T09:50:33.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.277+0000 7fb80f3dc840 20 datascan.scan_links: first is now 2 2023-10-11T09:50:33.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.277+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:88 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081800 con 0x564d189ff400 2023-10-11T09:50:33.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.281+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 <== osd.1 v2:172.21.15.78:6809/35734 22 ==== osd_op_reply(88 10000000000.00000000 [omap-set-vals] v110'29 uv29 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564d18a8d200 con 0x564d189ff400 2023-10-11T09:50:33.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.281+0000 7fb80f3dc840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_5 pointing to 0x0x100000001fa 2023-10-11T09:50:33.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.281+0000 7fb80f3dc840 10 datascan.scan_links: handling injected_ino 0x100000001fb 2023-10-11T09:50:33.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.281+0000 7fb80f3dc840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:flushed_file_6 2023-10-11T09:50:33.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.281+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:89 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18080000 con 0x564d189ff400 2023-10-11T09:50:33.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.281+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 <== osd.1 v2:172.21.15.78:6809/35734 23 ==== osd_op_reply(89 10000000000.00000000 [omap-get-vals-by-keys out=515b] v0'0 uv29 ondisk = 0) v8 ==== 164+0+515 (crc 0 0 0) 0x564d18a8d200 con 0x564d189ff400 2023-10-11T09:50:33.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.281+0000 7fb80f3dc840 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_6 frag=*, r=0 2023-10-11T09:50:33.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.281+0000 7fb80f3dc840 20 datascan.scan_links: first is now 2 2023-10-11T09:50:33.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.281+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:90 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d18081400 con 0x564d189ff400 2023-10-11T09:50:33.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.285+0000 7fb80e399700 1 -- 172.21.15.78:0/2239557512 <== osd.1 v2:172.21.15.78:6809/35734 24 ==== osd_op_reply(90 10000000000.00000000 [omap-set-vals] v110'30 uv30 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564d18a8d200 con 0x564d189ff400 2023-10-11T09:50:33.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.285+0000 7fb80f3dc840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_6 pointing to 0x0x100000001fb 2023-10-11T09:50:33.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.285+0000 7fb80f3dc840 10 datascan.scan_links: handling injected_ino 0x100000001fc 2023-10-11T09:50:33.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.285+0000 7fb80f3dc840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x4:100000001fc 2023-10-11T09:50:33.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.285+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:91 14.3 14:c0ce80d4:::4.00000000:head [omap-get-vals-by-keys in=24b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d189ff800 con 0x564d18044400 2023-10-11T09:50:33.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.285+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 <== osd.0 v2:172.21.15.78:6800/35735 29 ==== osd_op_reply(91 4.00000000 [omap-get-vals-by-keys out=512b] v0'0 uv2 ondisk = 0) v8 ==== 154+0+512 (crc 0 0 0) 0x564d18a8d440 con 0x564d18044400 2023-10-11T09:50:33.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.285+0000 7fb80f3dc840 20 datascan.read_dentry: oid=4.00000000 dname=100000001fc frag=*, r=0 2023-10-11T09:50:33.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.285+0000 7fb80f3dc840 20 datascan.scan_links: first is now 2 2023-10-11T09:50:33.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.285+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:92 14.3 14:c0ce80d4:::4.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d1808f000 con 0x564d18044400 2023-10-11T09:50:33.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.285+0000 7fb80db98700 1 -- 172.21.15.78:0/2239557512 <== osd.0 v2:172.21.15.78:6800/35735 30 ==== osd_op_reply(92 4.00000000 [omap-set-vals] v110'3 uv3 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x564d18a8d440 con 0x564d18044400 2023-10-11T09:50:33.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.285+0000 7fb80f3dc840 20 datascan.inject_linkage: Injected dentry 0x0x4/100000001fc pointing to 0x0x100000001fc 2023-10-11T09:50:33.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.285+0000 7fb80f3dc840 10 datascan.scan_links: updating inotable 2023-10-11T09:50:33.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.285+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d1808f400 con 0x564d189ff000 2023-10-11T09:50:33.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.289+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 <== osd.2 v2:172.21.15.78:6808/35732 9 ==== osd_op_reply(93 mds0_inotable [read 0~34 out=34b] v0'0 uv6 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x564d18a8cfc0 con 0x564d189ff000 2023-10-11T09:50:33.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.289+0000 7fb80f3dc840 10 mds.0.inotable: skip_inos was [0x10000000000~0x10000000000] 2023-10-11T09:50:33.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.289+0000 7fb80f3dc840 10 mds.0.inotable: skip_inos now [0x100000001fd~0xfffffffe03] 2023-10-11T09:50:33.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.289+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:94 14.13 14:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d1808f800 con 0x564d189ff000 2023-10-11T09:50:33.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.289+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 <== osd.2 v2:172.21.15.78:6808/35732 10 ==== osd_op_reply(94 mds0_inotable [writefull 0~34] v110'8 uv8 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x564d18a8cfc0 con 0x564d189ff000 2023-10-11T09:50:33.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.289+0000 7fb80f3dc840 10 datascan.scan_links: updating snaptable 2023-10-11T09:50:33.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.289+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:95 14.d 14:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x564d1808fc00 con 0x564d18044800 2023-10-11T09:50:33.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 <== osd.6 v2:172.21.15.133:6800/35900 9 ==== osd_op_reply(95 mds_snaptable [read 0~70 out=70b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x564d18a8cfc0 con 0x564d18044800 2023-10-11T09:50:33.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x564d18044400 msgr2=0x564d18056580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:33.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80f3dc840 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x564d18044400 0x564d18056580 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x564d189ff400 msgr2=0x564d18056000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:33.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80f3dc840 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x564d189ff400 0x564d18056000 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x564d189ff000 msgr2=0x564d17caa100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:33.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80f3dc840 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x564d189ff000 0x564d17caa100 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x564d18045400 msgr2=0x564d18058100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:33.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80f3dc840 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x564d18045400 0x564d18058100 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x564d18044c00 msgr2=0x564d18057600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:33.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80f3dc840 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x564d18044c00 0x564d18057600 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 reap_dead start 2023-10-11T09:50:33.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x564d18045000 msgr2=0x564d18057b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:33.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80f3dc840 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x564d18045000 0x564d18057b80 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x564d18044800 msgr2=0x564d18056b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:33.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80f3dc840 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x564d18044800 0x564d18056b00 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x564d18045800 msgr2=0x564d18057080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:33.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80f3dc840 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x564d18045800 0x564d18057080 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80d397700 1 -- 172.21.15.78:0/2239557512 reap_dead start 2023-10-11T09:50:33.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x564d18044000 msgr2=0x564d17cab700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:33.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80f3dc840 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x564d18044000 0x564d17cab700 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x564d17ef6810 tx=0x564d18a00f60 comp rx=0 tx=0).stop 2023-10-11T09:50:33.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d189fe400 msgr2=0x564d17cab180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:33.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80f3dc840 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d189fe400 0x564d17cab180 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x564d17ef7680 tx=0x564d17cd9500 comp rx=0 tx=0).stop 2023-10-11T09:50:33.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 shutdown_connections 2023-10-11T09:50:33.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80f3dc840 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x564d18045800 0x564d18057080 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80f3dc840 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x564d18044000 0x564d17cab700 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80f3dc840 1 --2- 172.21.15.78:0/2239557512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d189fe400 0x564d17cab180 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 >> 172.21.15.78:0/2239557512 conn(0x564d17c02c00 msgr2=0x564d17cf3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:33.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 shutdown_connections 2023-10-11T09:50:33.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80f3dc840 1 -- 172.21.15.78:0/2239557512 wait complete. 2023-10-11T09:50:33.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80f3dc840 1 -- 172.21.15.78:0/3189362571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d17c03800 msgr2=0x564d17ca8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:33.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80f3dc840 1 --2- 172.21.15.78:0/3189362571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d17c03800 0x564d17ca8580 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x564d17cd1380 tx=0x564d18a00d80 comp rx=0 tx=0).stop 2023-10-11T09:50:33.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80f3dc840 1 -- 172.21.15.78:0/3189362571 shutdown_connections 2023-10-11T09:50:33.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80f3dc840 1 --2- 172.21.15.78:0/3189362571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d17c03000 0x564d17ca9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80f3dc840 1 --2- 172.21.15.78:0/3189362571 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564d17c03400 0x564d17ca8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80f3dc840 1 --2- 172.21.15.78:0/3189362571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d17c03800 0x564d17ca8580 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80f3dc840 1 -- 172.21.15.78:0/3189362571 >> 172.21.15.78:0/3189362571 conn(0x564d17c03c00 msgr2=0x564d17cf3000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:33.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80f3dc840 1 -- 172.21.15.78:0/3189362571 shutdown_connections 2023-10-11T09:50:33.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.293+0000 7fb80f3dc840 1 -- 172.21.15.78:0/3189362571 wait complete. 2023-10-11T09:50:33.310 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.207736, result: 2023-10-11T09:50:33.310 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-10-11T09:50:33.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.473+0000 7fc28f466700 1 Processor -- start 2023-10-11T09:50:33.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.473+0000 7fc28f466700 1 -- start start 2023-10-11T09:50:33.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.477+0000 7fc28f466700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fc28806d180 0x7fc288070380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:33.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.477+0000 7fc28f466700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2880cd1c0 con 0x7fc28813f740 2023-10-11T09:50:33.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.477+0000 7fc28f466700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2880ce4c0 con 0x7fc28806cd00 2023-10-11T09:50:33.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.477+0000 7fc28f466700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc28806d600 con 0x7fc28806d180 2023-10-11T09:50:33.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.477+0000 7fc28e464700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fc28813f740 0x7fc2880ccab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:39274/0 (socket says 172.21.15.78:39274) 2023-10-11T09:50:33.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.477+0000 7fc28e464700 1 -- 172.21.15.78:0/647906829 learned_addr learned my addr 172.21.15.78:0/647906829 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:50:33.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.477+0000 7fc28ec65700 1 --2- 172.21.15.78:0/647906829 >> v2:172.21.15.78:3300/0 conn(0x7fc28806d180 0x7fc288070380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:33.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.477+0000 7fc28d462700 1 -- 172.21.15.78:0/647906829 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2084602492 0 0) 0x7fc2880cd1c0 con 0x7fc28813f740 2023-10-11T09:50:33.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.477+0000 7fc28d462700 1 -- 172.21.15.78:0/647906829 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2740035d0 con 0x7fc28813f740 2023-10-11T09:50:33.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.477+0000 7fc28ec65700 1 -- 172.21.15.78:0/647906829 >> v1:172.21.15.133:6789/0 conn(0x7fc28806cd00 legacy=0x7fc28806fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:33.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.477+0000 7fc28ec65700 1 -- 172.21.15.78:0/647906829 >> v1:172.21.15.78:6789/0 conn(0x7fc28813f740 legacy=0x7fc2880ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:33.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.477+0000 7fc28ec65700 1 -- 172.21.15.78:0/647906829 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2880cd420 con 0x7fc28806d180 2023-10-11T09:50:33.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.481+0000 7fc28ec65700 1 --2- 172.21.15.78:0/647906829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc28806d180 0x7fc288070380 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fc280009850 tx=0x7fc280009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef9efccb9f64e38 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:33.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.481+0000 7fc28d462700 1 -- 172.21.15.78:0/647906829 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc280011830 con 0x7fc28806d180 2023-10-11T09:50:33.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.481+0000 7fc28d462700 1 -- 172.21.15.78:0/647906829 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc280011990 con 0x7fc28806d180 2023-10-11T09:50:33.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.481+0000 7fc28d462700 1 -- 172.21.15.78:0/647906829 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc280011d40 con 0x7fc28806d180 2023-10-11T09:50:33.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.481+0000 7fc28f466700 1 -- 172.21.15.78:0/647906829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc28806d180 msgr2=0x7fc288070380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:33.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.481+0000 7fc28f466700 1 --2- 172.21.15.78:0/647906829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc28806d180 0x7fc288070380 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fc280009850 tx=0x7fc280009b60 comp rx=0 tx=0).stop 2023-10-11T09:50:33.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.481+0000 7fc28f466700 1 -- 172.21.15.78:0/647906829 shutdown_connections 2023-10-11T09:50:33.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.481+0000 7fc28f466700 1 --2- 172.21.15.78:0/647906829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc28806d180 0x7fc288070380 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.481+0000 7fc28f466700 1 -- 172.21.15.78:0/647906829 >> 172.21.15.78:0/647906829 conn(0x7fc2880cea60 msgr2=0x7fc2880cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:33.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.481+0000 7fc28f466700 1 -- 172.21.15.78:0/647906829 shutdown_connections 2023-10-11T09:50:33.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.481+0000 7fc28f466700 1 -- 172.21.15.78:0/647906829 wait complete. 2023-10-11T09:50:33.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.485+0000 7fc28f466700 1 Processor -- start 2023-10-11T09:50:33.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.485+0000 7fc28f466700 1 -- start start 2023-10-11T09:50:33.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.485+0000 7fc28f466700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc28806cd00 0x7fc2880b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:33.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.485+0000 7fc28f466700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc28806d180 0x7fc288153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:33.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.485+0000 7fc28f466700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc28813f740 0x7fc2881536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:33.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.485+0000 7fc28f466700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc288154d50 con 0x7fc28806cd00 2023-10-11T09:50:33.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.485+0000 7fc28f466700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc288155030 con 0x7fc28806d180 2023-10-11T09:50:33.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.485+0000 7fc28f466700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc288155310 con 0x7fc28813f740 2023-10-11T09:50:33.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.485+0000 7fc28e464700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc28806cd00 0x7fc2880b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:33.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.485+0000 7fc28e464700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc28806cd00 0x7fc2880b42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36174/0 (socket says 172.21.15.78:36174) 2023-10-11T09:50:33.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.485+0000 7fc28e464700 1 -- 172.21.15.78:0/25965941 learned_addr learned my addr 172.21.15.78:0/25965941 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:33.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.485+0000 7fc28dc63700 1 --2- 172.21.15.78:0/25965941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc28806d180 0x7fc288153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:33.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.485+0000 7fc28ec65700 1 --2- 172.21.15.78:0/25965941 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc28813f740 0x7fc2881536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:33.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.485+0000 7fc28e464700 1 -- 172.21.15.78:0/25965941 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc28813f740 msgr2=0x7fc2881536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:33.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.485+0000 7fc28e464700 1 --2- 172.21.15.78:0/25965941 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc28813f740 0x7fc2881536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.485+0000 7fc28e464700 1 -- 172.21.15.78:0/25965941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc28806d180 msgr2=0x7fc288153190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:33.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.485+0000 7fc28e464700 1 --2- 172.21.15.78:0/25965941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc28806d180 0x7fc288153190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:33.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.485+0000 7fc28e464700 1 -- 172.21.15.78:0/25965941 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2800094b0 con 0x7fc28806cd00 2023-10-11T09:50:33.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.489+0000 7fc28e464700 1 --2- 172.21.15.78:0/25965941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc28806cd00 0x7fc2880b42b0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fc28400cbc0 tx=0x7fc28400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:33.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.489+0000 7fc27effd700 1 -- 172.21.15.78:0/25965941 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28400a9e0 con 0x7fc28806cd00 2023-10-11T09:50:33.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.489+0000 7fc27effd700 1 -- 172.21.15.78:0/25965941 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc28400ab40 con 0x7fc28806cd00 2023-10-11T09:50:33.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.489+0000 7fc28f466700 1 -- 172.21.15.78:0/25965941 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2881555f0 con 0x7fc28806cd00 2023-10-11T09:50:33.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.489+0000 7fc27effd700 1 -- 172.21.15.78:0/25965941 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc284012400 con 0x7fc28806cd00 2023-10-11T09:50:33.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.489+0000 7fc28f466700 1 -- 172.21.15.78:0/25965941 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2880b47f0 con 0x7fc28806cd00 2023-10-11T09:50:33.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.489+0000 7fc28f466700 1 -- 172.21.15.78:0/25965941 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2880b4a40 con 0x7fc28806cd00 2023-10-11T09:50:33.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.493+0000 7fc27effd700 1 -- 172.21.15.78:0/25965941 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fc28401b3b0 con 0x7fc28806cd00 2023-10-11T09:50:33.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.493+0000 7fc27effd700 1 --2- 172.21.15.78:0/25965941 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc26c065ed0 0x7fc26c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:33.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.493+0000 7fc27effd700 1 -- 172.21.15.78:0/25965941 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6638+0+0 (secure 0 0 0) 0x7fc284083590 con 0x7fc28806cd00 2023-10-11T09:50:33.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.493+0000 7fc28dc63700 1 --2- 172.21.15.78:0/25965941 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc26c065ed0 0x7fc26c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:33.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.493+0000 7fc28dc63700 1 --2- 172.21.15.78:0/25965941 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc26c065ed0 0x7fc26c068380 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc2880b16f0 tx=0x7fc278009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:33.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.501+0000 7fc27effd700 1 -- 172.21.15.78:0/25965941 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fc284050990 con 0x7fc28806cd00 2023-10-11T09:50:33.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:33.745+0000 7fc28f466700 1 -- 172.21.15.78:0/25965941 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7fc288155b10 con 0x7fc28806cd00 2023-10-11T09:50:34.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.385+0000 7fc27effd700 1 -- 172.21.15.78:0/25965941 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 5:0 v86) v1 ==== 101+0+0 (secure 0 0 0) 0x7fc288155b10 con 0x7fc28806cd00 2023-10-11T09:50:34.397 INFO:tasks.ceph.mds.c.smithi078.stderr:2023-10-11T09:50:34.389+0000 7fca31cdc700 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-10-11T09:50:34.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.389+0000 7fc28f466700 1 -- 172.21.15.78:0/25965941 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc26c065ed0 msgr2=0x7fc26c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:34.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.389+0000 7fc28f466700 1 --2- 172.21.15.78:0/25965941 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc26c065ed0 0x7fc26c068380 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc2880b16f0 tx=0x7fc278009040 comp rx=0 tx=0).stop 2023-10-11T09:50:34.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.389+0000 7fc28f466700 1 -- 172.21.15.78:0/25965941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc28806cd00 msgr2=0x7fc2880b42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:34.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.389+0000 7fc28f466700 1 --2- 172.21.15.78:0/25965941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc28806cd00 0x7fc2880b42b0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fc28400cbc0 tx=0x7fc28400cca0 comp rx=0 tx=0).stop 2023-10-11T09:50:34.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.389+0000 7fc28f466700 1 -- 172.21.15.78:0/25965941 shutdown_connections 2023-10-11T09:50:34.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.389+0000 7fc28f466700 1 --2- 172.21.15.78:0/25965941 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc26c065ed0 0x7fc26c068380 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:34.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.389+0000 7fc28f466700 1 --2- 172.21.15.78:0/25965941 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc28813f740 0x7fc2881536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:34.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.389+0000 7fc28f466700 1 --2- 172.21.15.78:0/25965941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc28806d180 0x7fc288153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:34.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.389+0000 7fc28f466700 1 --2- 172.21.15.78:0/25965941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc28806cd00 0x7fc2880b42b0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:34.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.389+0000 7fc28f466700 1 -- 172.21.15.78:0/25965941 >> 172.21.15.78:0/25965941 conn(0x7fc2880cea60 msgr2=0x7fc2880b1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:34.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.393+0000 7fc28f466700 1 -- 172.21.15.78:0/25965941 shutdown_connections 2023-10-11T09:50:34.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.393+0000 7fc28f466700 1 -- 172.21.15.78:0/25965941 wait complete. 2023-10-11T09:50:34.402 INFO:teuthology.orchestra.run.smithi078.stderr:repaired: restoring rank 5:0 2023-10-11T09:50:34.422 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-11T09:50:34.422 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-11T09:50:34.423 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-11T09:50:34.423 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:50:34.423 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-11T09:50:34.423 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-11T09:50:34.424 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-11T09:50:34.424 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:50:34.424 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-11T09:50:34.424 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-11T09:50:34.425 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-11T09:50:34.425 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:50:34.425 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-11T09:50:34.426 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-11T09:50:34.426 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-11T09:50:34.426 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:50:34.446 INFO:tasks.ceph.mds.a:Stopped 2023-10-11T09:50:34.447 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-11T09:50:34.447 INFO:tasks.ceph.mds.d:Stopped 2023-10-11T09:50:34.447 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-11T09:50:34.449 INFO:tasks.ceph.mds.c:Stopped 2023-10-11T09:50:34.450 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-11T09:50:34.450 INFO:tasks.ceph.mds.b:Stopped 2023-10-11T09:50:34.450 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-11T09:50:34.495 INFO:tasks.ceph.mds.d:Started 2023-10-11T09:50:34.496 DEBUG:teuthology.parallel:result is None 2023-10-11T09:50:34.497 INFO:tasks.ceph.mds.a:Started 2023-10-11T09:50:34.498 DEBUG:teuthology.parallel:result is None 2023-10-11T09:50:34.500 INFO:tasks.ceph.mds.b:Started 2023-10-11T09:50:34.500 DEBUG:teuthology.parallel:result is None 2023-10-11T09:50:34.501 INFO:tasks.ceph.mds.c:Started 2023-10-11T09:50:34.502 DEBUG:teuthology.parallel:result is None 2023-10-11T09:50:34.502 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:50:34.609 INFO:tasks.ceph.mds.a.smithi078.stdout:starting mds.a at 2023-10-11T09:50:34.614 INFO:tasks.ceph.mds.b.smithi133.stdout:starting mds.b at 2023-10-11T09:50:34.615 INFO:tasks.ceph.mds.c.smithi078.stdout:starting mds.c at 2023-10-11T09:50:34.615 INFO:tasks.ceph.mds.d.smithi133.stdout:starting mds.d at 2023-10-11T09:50:34.627 INFO:tasks.ceph.mon.a.smithi078.stderr:2023-10-11T09:50:34.617+0000 7f90131f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-11T09:50:34.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.661+0000 7f989171f700 1 Processor -- start 2023-10-11T09:50:34.672 INFO:tasks.ceph.mds.c.smithi078.stderr:2023-10-11T09:50:34.661+0000 7f608874c700 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-10-11T09:50:34.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.661+0000 7f989171f700 1 -- start start 2023-10-11T09:50:34.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.665+0000 7f989171f700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f988c0d1660 0x7f988c0d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:34.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.665+0000 7f989171f700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f988c15dc10 con 0x7f988c156ce0 2023-10-11T09:50:34.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.665+0000 7f989171f700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f988c160380 con 0x7f988c0d28e0 2023-10-11T09:50:34.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.665+0000 7f989171f700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f988c15d930 con 0x7f988c0d1660 2023-10-11T09:50:34.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.665+0000 7f988affd700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f988c156ce0 0x7f988c160b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:41574/0 (socket says 172.21.15.78:41574) 2023-10-11T09:50:34.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.665+0000 7f988affd700 1 -- 172.21.15.78:0/2703098090 learned_addr learned my addr 172.21.15.78:0/2703098090 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:50:34.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.665+0000 7f988a7fc700 1 --2- 172.21.15.78:0/2703098090 >> v2:172.21.15.133:3300/0 conn(0x7f988c0d1660 0x7f988c0d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:34.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.665+0000 7f988a7fc700 1 -- 172.21.15.78:0/2703098090 >> v1:172.21.15.133:6789/0 conn(0x7f988c0d28e0 legacy=0x7f988c0cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:34.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.665+0000 7f988a7fc700 1 -- 172.21.15.78:0/2703098090 >> v1:172.21.15.78:6789/0 conn(0x7f988c156ce0 legacy=0x7f988c160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:34.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.665+0000 7f988a7fc700 1 -- 172.21.15.78:0/2703098090 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988c160620 con 0x7f988c0d1660 2023-10-11T09:50:34.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.665+0000 7f9889ffb700 1 -- 172.21.15.78:0/2703098090 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1915203531 0 0) 0x7f988c15dc10 con 0x7f988c156ce0 2023-10-11T09:50:34.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.665+0000 7f988a7fc700 1 --2- 172.21.15.78:0/2703098090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f988c0d1660 0x7f988c0d0a50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9880009850 tx=0x7f9880009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a1860a76482f985 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:34.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.665+0000 7f9889ffb700 1 -- 172.21.15.78:0/2703098090 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9880011830 con 0x7f988c0d1660 2023-10-11T09:50:34.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.665+0000 7f9889ffb700 1 -- 172.21.15.78:0/2703098090 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9880011990 con 0x7f988c0d1660 2023-10-11T09:50:34.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.665+0000 7f9889ffb700 1 -- 172.21.15.78:0/2703098090 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9880011d20 con 0x7f988c0d1660 2023-10-11T09:50:34.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.665+0000 7f989171f700 1 -- 172.21.15.78:0/2703098090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f988c0d1660 msgr2=0x7f988c0d0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:34.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.665+0000 7f989171f700 1 --2- 172.21.15.78:0/2703098090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f988c0d1660 0x7f988c0d0a50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9880009850 tx=0x7f9880009b60 comp rx=0 tx=0).stop 2023-10-11T09:50:34.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.669+0000 7f989171f700 1 -- 172.21.15.78:0/2703098090 shutdown_connections 2023-10-11T09:50:34.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.669+0000 7f989171f700 1 --2- 172.21.15.78:0/2703098090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f988c0d1660 0x7f988c0d0a50 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:34.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.669+0000 7f989171f700 1 -- 172.21.15.78:0/2703098090 >> 172.21.15.78:0/2703098090 conn(0x7f988c06afb0 msgr2=0x7f988c06b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:34.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.669+0000 7f989171f700 1 -- 172.21.15.78:0/2703098090 shutdown_connections 2023-10-11T09:50:34.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.669+0000 7f989171f700 1 -- 172.21.15.78:0/2703098090 wait complete. 2023-10-11T09:50:34.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.669+0000 7f989171f700 1 Processor -- start 2023-10-11T09:50:34.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.669+0000 7f989171f700 1 -- start start 2023-10-11T09:50:34.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.673+0000 7f989171f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f988c0d1660 0x7f988c06b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:34.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.673+0000 7f989171f700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f988c0d28e0 0x7f988c143f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:34.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.673+0000 7f989171f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f988c156ce0 0x7f988c155060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:34.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.673+0000 7f989171f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988c145320 con 0x7f988c0d1660 2023-10-11T09:50:34.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.673+0000 7f989171f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988c144470 con 0x7f988c156ce0 2023-10-11T09:50:34.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.673+0000 7f989171f700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988c0c69b0 con 0x7f988c0d28e0 2023-10-11T09:50:34.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.673+0000 7f988affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f988c0d1660 0x7f988c06b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:34.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.673+0000 7f988affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f988c0d1660 0x7f988c06b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40876/0 (socket says 172.21.15.78:40876) 2023-10-11T09:50:34.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.673+0000 7f988affd700 1 -- 172.21.15.78:0/3571642633 learned_addr learned my addr 172.21.15.78:0/3571642633 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:34.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.673+0000 7f988a7fc700 1 --2- 172.21.15.78:0/3571642633 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f988c0d28e0 0x7f988c143f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:34.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.673+0000 7f988b7fe700 1 --2- 172.21.15.78:0/3571642633 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f988c156ce0 0x7f988c155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:34.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.673+0000 7f988affd700 1 -- 172.21.15.78:0/3571642633 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f988c0d28e0 msgr2=0x7f988c143f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:34.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.673+0000 7f988affd700 1 --2- 172.21.15.78:0/3571642633 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f988c0d28e0 0x7f988c143f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:34.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.673+0000 7f988affd700 1 -- 172.21.15.78:0/3571642633 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f988c156ce0 msgr2=0x7f988c155060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:34.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.673+0000 7f988affd700 1 --2- 172.21.15.78:0/3571642633 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f988c156ce0 0x7f988c155060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:34.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.673+0000 7f988affd700 1 -- 172.21.15.78:0/3571642633 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98800094b0 con 0x7f988c0d1660 2023-10-11T09:50:34.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.673+0000 7f988affd700 1 --2- 172.21.15.78:0/3571642633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f988c0d1660 0x7f988c06b940 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f987c00c4a0 tx=0x7f987c00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:34.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.673+0000 7f98737fe700 1 -- 172.21.15.78:0/3571642633 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c004ca0 con 0x7f988c0d1660 2023-10-11T09:50:34.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.673+0000 7f98737fe700 1 -- 172.21.15.78:0/3571642633 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f987c004e00 con 0x7f988c0d1660 2023-10-11T09:50:34.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.673+0000 7f98737fe700 1 -- 172.21.15.78:0/3571642633 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c007310 con 0x7f988c0d1660 2023-10-11T09:50:34.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.673+0000 7f989171f700 1 -- 172.21.15.78:0/3571642633 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f988c144710 con 0x7f988c0d1660 2023-10-11T09:50:34.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.673+0000 7f989171f700 1 -- 172.21.15.78:0/3571642633 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f988c144cd0 con 0x7f988c0d1660 2023-10-11T09:50:34.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.677+0000 7f989171f700 1 -- 172.21.15.78:0/3571642633 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f988c065d40 con 0x7f988c0d1660 2023-10-11T09:50:34.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.677+0000 7f98737fe700 1 -- 172.21.15.78:0/3571642633 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f987c007470 con 0x7f988c0d1660 2023-10-11T09:50:34.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.677+0000 7f98737fe700 1 --2- 172.21.15.78:0/3571642633 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9878065f00 0x7f98780683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:34.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.677+0000 7f98737fe700 1 -- 172.21.15.78:0/3571642633 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f987c0836e0 con 0x7f988c0d1660 2023-10-11T09:50:34.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.677+0000 7f988a7fc700 1 --2- 172.21.15.78:0/3571642633 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9878065f00 0x7f98780683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:34.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.681+0000 7f988a7fc700 1 --2- 172.21.15.78:0/3571642633 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9878065f00 0x7f98780683b0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f988c08c200 tx=0x7f9880008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:34.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.685+0000 7f98737fe700 1 -- 172.21.15.78:0/3571642633 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f987c050960 con 0x7f988c0d1660 2023-10-11T09:50:34.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.897+0000 7f989171f700 1 -- 172.21.15.78:0/3571642633 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f988c06d310 con 0x7f988c0d1660 2023-10-11T09:50:34.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.897+0000 7f98737fe700 1 -- 172.21.15.78:0/3571642633 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+4655 (secure 0 0 0) 0x7f987c007720 con 0x7f988c0d1660 2023-10-11T09:50:34.908 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:34.908 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":88,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6815,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2640710924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2640710924},{"type":"v1","addr":"172.21.15.133:6835","nonce":2640710924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":7148,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/331520590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":331520590},{"type":"v1","addr":"172.21.15.78:6835","nonce":331520590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":7151,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/934539531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":934539531},{"type":"v1","addr":"172.21.15.133:6834","nonce":934539531}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:50:28.502984+0000","modified":"2023-10-11T09:50:34.641888+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8550":{"gid":8550,"name":"c","rank":0,"incarnation":88,"state":"up:replay","state_seq":1,"addr":"172.21.15.78:6837/2869034036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2869034036},{"type":"v1","addr":"172.21.15.78:6837","nonce":2869034036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15,16],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-11T09:50:34.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.901+0000 7f989171f700 1 -- 172.21.15.78:0/3571642633 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9878065f00 msgr2=0x7f98780683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:34.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.901+0000 7f989171f700 1 --2- 172.21.15.78:0/3571642633 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9878065f00 0x7f98780683b0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f988c08c200 tx=0x7f9880008030 comp rx=0 tx=0).stop 2023-10-11T09:50:34.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.901+0000 7f989171f700 1 -- 172.21.15.78:0/3571642633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f988c0d1660 msgr2=0x7f988c06b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:34.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.901+0000 7f989171f700 1 --2- 172.21.15.78:0/3571642633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f988c0d1660 0x7f988c06b940 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f987c00c4a0 tx=0x7f987c00c7b0 comp rx=0 tx=0).stop 2023-10-11T09:50:34.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.901+0000 7f989171f700 1 -- 172.21.15.78:0/3571642633 shutdown_connections 2023-10-11T09:50:34.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.901+0000 7f989171f700 1 --2- 172.21.15.78:0/3571642633 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9878065f00 0x7f98780683b0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:34.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.901+0000 7f989171f700 1 --2- 172.21.15.78:0/3571642633 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f988c156ce0 0x7f988c155060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:34.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.901+0000 7f989171f700 1 --2- 172.21.15.78:0/3571642633 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f988c0d28e0 0x7f988c143f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:34.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.901+0000 7f989171f700 1 --2- 172.21.15.78:0/3571642633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f988c0d1660 0x7f988c06b940 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:34.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.901+0000 7f989171f700 1 -- 172.21.15.78:0/3571642633 >> 172.21.15.78:0/3571642633 conn(0x7f988c06afb0 msgr2=0x7f988c0cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:34.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.901+0000 7f989171f700 1 -- 172.21.15.78:0/3571642633 shutdown_connections 2023-10-11T09:50:34.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:34.901+0000 7f989171f700 1 -- 172.21.15.78:0/3571642633 wait complete. 2023-10-11T09:50:34.913 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 88 2023-10-11T09:50:35.332 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 88, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-11T09:50:28.502984+0000', 'modified': '2023-10-11T09:50:34.641888+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 112, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8550}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8550': {'gid': 8550, 'name': 'c', 'rank': 0, 'incarnation': 88, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.78:6837/2869034036', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6836', 'nonce': 2869034036}, {'type': 'v1', 'addr': '172.21.15.78:6837', 'nonce': 2869034036}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15, 16], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-11T09:50:35.332 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8550:up:replay 2023-10-11T09:50:36.333 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:50:36.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.501+0000 7ffb04f00700 1 Processor -- start 2023-10-11T09:50:36.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.501+0000 7ffb04f00700 1 -- start start 2023-10-11T09:50:36.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.505+0000 7ffb04f00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ffb0015a940 0x7ffb000ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:36.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.505+0000 7ffb04f00700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb0015e010 con 0x7ffb000ca780 2023-10-11T09:50:36.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.505+0000 7ffb04f00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb000cab70 con 0x7ffb0015b8e0 2023-10-11T09:50:36.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.505+0000 7ffb04f00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb0015e280 con 0x7ffb0015a940 2023-10-11T09:50:36.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.505+0000 7ffaff7fe700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ffb0015b8e0 0x7ffb0006d0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:45938/0 (socket says 172.21.15.78:45938) 2023-10-11T09:50:36.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.505+0000 7ffaff7fe700 1 -- 172.21.15.78:0/763899978 learned_addr learned my addr 172.21.15.78:0/763899978 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:50:36.515 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.505+0000 7ffafffff700 1 --1- 172.21.15.78:0/763899978 >> v1:172.21.15.78:6789/0 conn(0x7ffb000ca780 0x7ffb0013fcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:41580/0 (socket says 172.21.15.78:41580) 2023-10-11T09:50:36.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.505+0000 7ffafeffd700 1 --2- 172.21.15.78:0/763899978 >> v2:172.21.15.133:3300/0 conn(0x7ffb0015a940 0x7ffb000ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:36.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.505+0000 7ffafe7fc700 1 -- 172.21.15.78:0/763899978 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3978491827 0 0) 0x7ffb000cab70 con 0x7ffb0015b8e0 2023-10-11T09:50:36.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.505+0000 7ffafe7fc700 1 -- 172.21.15.78:0/763899978 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffae80035d0 con 0x7ffb0015b8e0 2023-10-11T09:50:36.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.505+0000 7ffafe7fc700 1 -- 172.21.15.78:0/763899978 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2948998219 0 0) 0x7ffb0015e010 con 0x7ffb000ca780 2023-10-11T09:50:36.516 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.505+0000 7ffafe7fc700 1 -- 172.21.15.78:0/763899978 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb000cab70 con 0x7ffb000ca780 2023-10-11T09:50:36.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.509+0000 7ffafeffd700 1 -- 172.21.15.78:0/763899978 >> v1:172.21.15.133:6789/0 conn(0x7ffb0015b8e0 legacy=0x7ffb0006d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:36.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.509+0000 7ffafeffd700 1 -- 172.21.15.78:0/763899978 >> v1:172.21.15.78:6789/0 conn(0x7ffb000ca780 legacy=0x7ffb0013fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:36.518 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.509+0000 7ffafeffd700 1 -- 172.21.15.78:0/763899978 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb0015e560 con 0x7ffb0015a940 2023-10-11T09:50:36.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.509+0000 7ffafeffd700 1 --2- 172.21.15.78:0/763899978 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb0015a940 0x7ffb000ca240 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4009850 tx=0x7ffaf4009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bbf6563b6374216 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:36.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.509+0000 7ffafe7fc700 1 -- 172.21.15.78:0/763899978 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4011830 con 0x7ffb0015a940 2023-10-11T09:50:36.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.509+0000 7ffafe7fc700 1 -- 172.21.15.78:0/763899978 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf4011990 con 0x7ffb0015a940 2023-10-11T09:50:36.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.509+0000 7ffafe7fc700 1 -- 172.21.15.78:0/763899978 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4011d40 con 0x7ffb0015a940 2023-10-11T09:50:36.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.509+0000 7ffb04f00700 1 -- 172.21.15.78:0/763899978 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb0015a940 msgr2=0x7ffb000ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:36.519 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.509+0000 7ffb04f00700 1 --2- 172.21.15.78:0/763899978 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb0015a940 0x7ffb000ca240 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4009850 tx=0x7ffaf4009b60 comp rx=0 tx=0).stop 2023-10-11T09:50:36.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.509+0000 7ffb04f00700 1 -- 172.21.15.78:0/763899978 shutdown_connections 2023-10-11T09:50:36.520 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.509+0000 7ffb04f00700 1 --2- 172.21.15.78:0/763899978 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb0015a940 0x7ffb000ca240 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:36.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.509+0000 7ffb04f00700 1 -- 172.21.15.78:0/763899978 >> 172.21.15.78:0/763899978 conn(0x7ffb000cea60 msgr2=0x7ffb000cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:36.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.509+0000 7ffb04f00700 1 -- 172.21.15.78:0/763899978 shutdown_connections 2023-10-11T09:50:36.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.509+0000 7ffb04f00700 1 -- 172.21.15.78:0/763899978 wait complete. 2023-10-11T09:50:36.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.513+0000 7ffb04f00700 1 Processor -- start 2023-10-11T09:50:36.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.513+0000 7ffb04f00700 1 -- start start 2023-10-11T09:50:36.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.513+0000 7ffb04f00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb000ca780 0x7ffb0013f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:36.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.513+0000 7ffb04f00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb0015a940 0x7ffb00140f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:36.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.513+0000 7ffb04f00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffb0015b8e0 0x7ffb00141f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:36.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.513+0000 7ffb04f00700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb00140b60 con 0x7ffb0015a940 2023-10-11T09:50:36.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.513+0000 7ffb04f00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0013ec10 con 0x7ffb000ca780 2023-10-11T09:50:36.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.513+0000 7ffb04f00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0013fee0 con 0x7ffb0015b8e0 2023-10-11T09:50:36.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.513+0000 7ffafeffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb0015a940 0x7ffb00140f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:36.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.513+0000 7ffafeffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb0015a940 0x7ffb00140f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40890/0 (socket says 172.21.15.78:40890) 2023-10-11T09:50:36.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.513+0000 7ffafeffd700 1 -- 172.21.15.78:0/2531023390 learned_addr learned my addr 172.21.15.78:0/2531023390 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:36.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.513+0000 7ffaff7fe700 1 --2- 172.21.15.78:0/2531023390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb000ca780 0x7ffb0013f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:36.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.513+0000 7ffafffff700 1 --2- 172.21.15.78:0/2531023390 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffb0015b8e0 0x7ffb00141f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:36.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.513+0000 7ffafeffd700 1 -- 172.21.15.78:0/2531023390 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffb0015b8e0 msgr2=0x7ffb00141f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:36.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.513+0000 7ffafeffd700 1 --2- 172.21.15.78:0/2531023390 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffb0015b8e0 0x7ffb00141f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:36.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.513+0000 7ffafeffd700 1 -- 172.21.15.78:0/2531023390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb000ca780 msgr2=0x7ffb0013f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:36.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.513+0000 7ffafeffd700 1 --2- 172.21.15.78:0/2531023390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb000ca780 0x7ffb0013f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:36.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.513+0000 7ffafeffd700 1 -- 172.21.15.78:0/2531023390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf40094b0 con 0x7ffb0015a940 2023-10-11T09:50:36.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.513+0000 7ffafffff700 1 --2- 172.21.15.78:0/2531023390 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffb0015b8e0 0x7ffb00141f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:50:36.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.513+0000 7ffafeffd700 1 --2- 172.21.15.78:0/2531023390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb0015a940 0x7ffb00140f80 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7ffaf40090b0 tx=0x7ffaf4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:36.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.513+0000 7ffae7fff700 1 -- 172.21.15.78:0/2531023390 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4004590 con 0x7ffb0015a940 2023-10-11T09:50:36.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.513+0000 7ffae7fff700 1 -- 172.21.15.78:0/2531023390 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf401f070 con 0x7ffb0015a940 2023-10-11T09:50:36.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.513+0000 7ffae7fff700 1 -- 172.21.15.78:0/2531023390 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4003900 con 0x7ffb0015a940 2023-10-11T09:50:36.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.513+0000 7ffb04f00700 1 -- 172.21.15.78:0/2531023390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb001401c0 con 0x7ffb0015a940 2023-10-11T09:50:36.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.517+0000 7ffb04f00700 1 -- 172.21.15.78:0/2531023390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb00140780 con 0x7ffb0015a940 2023-10-11T09:50:36.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.517+0000 7ffb04f00700 1 -- 172.21.15.78:0/2531023390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb00140ca0 con 0x7ffb0015a940 2023-10-11T09:50:36.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.517+0000 7ffae7fff700 1 -- 172.21.15.78:0/2531023390 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7ffaf4003a60 con 0x7ffb0015a940 2023-10-11T09:50:36.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.521+0000 7ffae7fff700 1 --2- 172.21.15.78:0/2531023390 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffaec065ed0 0x7ffaec068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:36.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.521+0000 7ffae7fff700 1 -- 172.21.15.78:0/2531023390 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6724+0+0 (secure 0 0 0) 0x7ffaf4021070 con 0x7ffb0015a940 2023-10-11T09:50:36.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.521+0000 7ffaff7fe700 1 --2- 172.21.15.78:0/2531023390 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffaec065ed0 0x7ffaec068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:36.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.521+0000 7ffaff7fe700 1 --2- 172.21.15.78:0/2531023390 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffaec065ed0 0x7ffaec068380 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ffaf00012e0 tx=0x7ffaf0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:36.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.529+0000 7ffae7fff700 1 -- 172.21.15.78:0/2531023390 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7ffb00140ca0 con 0x7ffb0015a940 2023-10-11T09:50:36.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.733+0000 7ffb04f00700 1 -- 172.21.15.78:0/2531023390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb00140ca0 con 0x7ffb0015a940 2023-10-11T09:50:36.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.737+0000 7ffae7fff700 1 -- 172.21.15.78:0/2531023390 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4655 (secure 0 0 0) 0x7ffb00140ca0 con 0x7ffb0015a940 2023-10-11T09:50:36.744 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:36.745 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":90,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7148,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/331520590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":331520590},{"type":"v1","addr":"172.21.15.78:6835","nonce":331520590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":7151,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/934539531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":934539531},{"type":"v1","addr":"172.21.15.133:6834","nonce":934539531}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":7154,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3894002128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":3894002128},{"type":"v1","addr":"172.21.15.133:6835","nonce":3894002128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:50:28.502984+0000","modified":"2023-10-11T09:50:36.654959+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8550":{"gid":8550,"name":"c","rank":0,"incarnation":88,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.78:6837/2869034036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2869034036},{"type":"v1","addr":"172.21.15.78:6837","nonce":2869034036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15,16],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-11T09:50:36.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.737+0000 7ffb04f00700 1 -- 172.21.15.78:0/2531023390 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffaec065ed0 msgr2=0x7ffaec068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:36.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.737+0000 7ffb04f00700 1 --2- 172.21.15.78:0/2531023390 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffaec065ed0 0x7ffaec068380 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ffaf00012e0 tx=0x7ffaf0009040 comp rx=0 tx=0).stop 2023-10-11T09:50:36.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.737+0000 7ffb04f00700 1 -- 172.21.15.78:0/2531023390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb0015a940 msgr2=0x7ffb00140f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:36.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.737+0000 7ffb04f00700 1 --2- 172.21.15.78:0/2531023390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb0015a940 0x7ffb00140f80 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7ffaf40090b0 tx=0x7ffaf4004770 comp rx=0 tx=0).stop 2023-10-11T09:50:36.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.737+0000 7ffb04f00700 1 -- 172.21.15.78:0/2531023390 shutdown_connections 2023-10-11T09:50:36.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.737+0000 7ffb04f00700 1 --2- 172.21.15.78:0/2531023390 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffaec065ed0 0x7ffaec068380 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:36.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.737+0000 7ffb04f00700 1 --2- 172.21.15.78:0/2531023390 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffb0015b8e0 0x7ffb00141f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:36.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.737+0000 7ffb04f00700 1 --2- 172.21.15.78:0/2531023390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb0015a940 0x7ffb00140f80 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:36.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.737+0000 7ffb04f00700 1 --2- 172.21.15.78:0/2531023390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb000ca780 0x7ffb0013f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:36.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.737+0000 7ffb04f00700 1 -- 172.21.15.78:0/2531023390 >> 172.21.15.78:0/2531023390 conn(0x7ffb000cea60 msgr2=0x7ffb000c9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:36.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.737+0000 7ffb04f00700 1 -- 172.21.15.78:0/2531023390 shutdown_connections 2023-10-11T09:50:36.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:36.737+0000 7ffb04f00700 1 -- 172.21.15.78:0/2531023390 wait complete. 2023-10-11T09:50:36.750 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 90 2023-10-11T09:50:37.170 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-11T09:50:28.502984+0000', 'modified': '2023-10-11T09:50:36.654959+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 112, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8550}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8550': {'gid': 8550, 'name': 'c', 'rank': 0, 'incarnation': 88, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.78:6837/2869034036', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6836', 'nonce': 2869034036}, {'type': 'v1', 'addr': '172.21.15.78:6837', 'nonce': 2869034036}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15, 16], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-11T09:50:37.170 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8550:up:rejoin 2023-10-11T09:50:38.171 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:50:38.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.341+0000 7f3823f5e700 1 Processor -- start 2023-10-11T09:50:38.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.341+0000 7f3823f5e700 1 -- start start 2023-10-11T09:50:38.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.345+0000 7f3823f5e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f381c160d40 0x7f381c0cd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:38.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.345+0000 7f3823f5e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f381c0ce500 0x7f381c06be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:38.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.345+0000 7f3823f5e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f381c06ffa0 con 0x7f381c071040 2023-10-11T09:50:38.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.345+0000 7f3823f5e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f381c070c20 con 0x7f381c0ce500 2023-10-11T09:50:38.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.345+0000 7f3823f5e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f381c161110 con 0x7f381c160d40 2023-10-11T09:50:38.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.345+0000 7f38214f9700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f381c071040 0x7f381c0cddf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:41582/0 (socket says 172.21.15.78:41582) 2023-10-11T09:50:38.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.345+0000 7f38214f9700 1 -- 172.21.15.78:0/3581399120 learned_addr learned my addr 172.21.15.78:0/3581399120 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:50:38.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.345+0000 7f38224fb700 1 --2- 172.21.15.78:0/3581399120 >> v2:172.21.15.78:3300/0 conn(0x7f381c0ce500 0x7f381c06be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:38.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.345+0000 7f3821cfa700 1 --2- 172.21.15.78:0/3581399120 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f381c160d40 0x7f381c0cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:38.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.345+0000 7f38224fb700 1 -- 172.21.15.78:0/3581399120 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f381c160d40 msgr2=0x7f381c0cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:38.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.345+0000 7f38224fb700 1 --2- 172.21.15.78:0/3581399120 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f381c160d40 0x7f381c0cd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:38.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.345+0000 7f38224fb700 1 -- 172.21.15.78:0/3581399120 >> v1:172.21.15.78:6789/0 conn(0x7f381c071040 legacy=0x7f381c0cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:38.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.345+0000 7f38224fb700 1 -- 172.21.15.78:0/3581399120 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f381c070280 con 0x7f381c0ce500 2023-10-11T09:50:38.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.345+0000 7f3821cfa700 1 --2- 172.21.15.78:0/3581399120 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f381c160d40 0x7f381c0cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:50:38.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.349+0000 7f38224fb700 1 --2- 172.21.15.78:0/3581399120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f381c0ce500 0x7f381c06be60 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f3814009850 tx=0x7f3814009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7da33de4da0a398 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:38.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.349+0000 7f3820cf8700 1 -- 172.21.15.78:0/3581399120 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3814011830 con 0x7f381c0ce500 2023-10-11T09:50:38.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.349+0000 7f3820cf8700 1 -- 172.21.15.78:0/3581399120 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3814011990 con 0x7f381c0ce500 2023-10-11T09:50:38.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.349+0000 7f3820cf8700 1 -- 172.21.15.78:0/3581399120 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3814011d40 con 0x7f381c0ce500 2023-10-11T09:50:38.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.349+0000 7f3823f5e700 1 -- 172.21.15.78:0/3581399120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f381c0ce500 msgr2=0x7f381c06be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:38.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.349+0000 7f3823f5e700 1 --2- 172.21.15.78:0/3581399120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f381c0ce500 0x7f381c06be60 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f3814009850 tx=0x7f3814009b60 comp rx=0 tx=0).stop 2023-10-11T09:50:38.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.349+0000 7f3823f5e700 1 -- 172.21.15.78:0/3581399120 shutdown_connections 2023-10-11T09:50:38.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.349+0000 7f3823f5e700 1 --2- 172.21.15.78:0/3581399120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f381c0ce500 0x7f381c06be60 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:38.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.349+0000 7f3823f5e700 1 --2- 172.21.15.78:0/3581399120 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f381c160d40 0x7f381c0cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:38.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.349+0000 7f3823f5e700 1 -- 172.21.15.78:0/3581399120 >> 172.21.15.78:0/3581399120 conn(0x7f381c06aca0 msgr2=0x7f381c0b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:38.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.349+0000 7f3823f5e700 1 -- 172.21.15.78:0/3581399120 shutdown_connections 2023-10-11T09:50:38.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.349+0000 7f3823f5e700 1 -- 172.21.15.78:0/3581399120 wait complete. 2023-10-11T09:50:38.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.353+0000 7f3823f5e700 1 Processor -- start 2023-10-11T09:50:38.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.353+0000 7f3823f5e700 1 -- start start 2023-10-11T09:50:38.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.353+0000 7f3823f5e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f381c071040 0x7f381c0cef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:38.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.353+0000 7f3823f5e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f381c0ce500 0x7f381c15bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:38.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.353+0000 7f3823f5e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f381c160d40 0x7f381c15c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:38.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.353+0000 7f3823f5e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381c0b1c90 con 0x7f381c0ce500 2023-10-11T09:50:38.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.353+0000 7f3823f5e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381c0b2c60 con 0x7f381c071040 2023-10-11T09:50:38.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.353+0000 7f3823f5e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f381c0b27d0 con 0x7f381c160d40 2023-10-11T09:50:38.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.353+0000 7f38214f9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f381c0ce500 0x7f381c15bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:38.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.353+0000 7f38214f9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f381c0ce500 0x7f381c15bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40902/0 (socket says 172.21.15.78:40902) 2023-10-11T09:50:38.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.353+0000 7f38214f9700 1 -- 172.21.15.78:0/3050619164 learned_addr learned my addr 172.21.15.78:0/3050619164 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:38.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.357+0000 7f3821cfa700 1 --2- 172.21.15.78:0/3050619164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f381c071040 0x7f381c0cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:38.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.357+0000 7f38224fb700 1 --2- 172.21.15.78:0/3050619164 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f381c160d40 0x7f381c15c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:38.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.357+0000 7f38214f9700 1 -- 172.21.15.78:0/3050619164 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f381c160d40 msgr2=0x7f381c15c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:38.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.357+0000 7f38214f9700 1 --2- 172.21.15.78:0/3050619164 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f381c160d40 0x7f381c15c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:38.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.357+0000 7f38214f9700 1 -- 172.21.15.78:0/3050619164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f381c071040 msgr2=0x7f381c0cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:38.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.357+0000 7f38214f9700 1 --2- 172.21.15.78:0/3050619164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f381c071040 0x7f381c0cef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:38.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.357+0000 7f38214f9700 1 -- 172.21.15.78:0/3050619164 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3818009a70 con 0x7f381c0ce500 2023-10-11T09:50:38.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.357+0000 7f38214f9700 1 --2- 172.21.15.78:0/3050619164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f381c0ce500 0x7f381c15bc80 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f380c00c520 tx=0x7f380c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:38.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.357+0000 7f38127fc700 1 -- 172.21.15.78:0/3050619164 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c005d90 con 0x7f381c0ce500 2023-10-11T09:50:38.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.357+0000 7f38127fc700 1 -- 172.21.15.78:0/3050619164 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380c0070d0 con 0x7f381c0ce500 2023-10-11T09:50:38.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.357+0000 7f3823f5e700 1 -- 172.21.15.78:0/3050619164 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38140094b0 con 0x7f381c0ce500 2023-10-11T09:50:38.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.357+0000 7f38127fc700 1 -- 172.21.15.78:0/3050619164 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c007440 con 0x7f381c0ce500 2023-10-11T09:50:38.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.357+0000 7f3823f5e700 1 -- 172.21.15.78:0/3050619164 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f381c0b1ea0 con 0x7f381c0ce500 2023-10-11T09:50:38.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.357+0000 7f3823f5e700 1 -- 172.21.15.78:0/3050619164 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f381c072250 con 0x7f381c0ce500 2023-10-11T09:50:38.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.361+0000 7f38127fc700 1 -- 172.21.15.78:0/3050619164 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f380c0075a0 con 0x7f381c0ce500 2023-10-11T09:50:38.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.361+0000 7f38127fc700 1 --2- 172.21.15.78:0/3050619164 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3800065ed0 0x7f3800068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:38.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.361+0000 7f38127fc700 1 -- 172.21.15.78:0/3050619164 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f380c082f90 con 0x7f381c0ce500 2023-10-11T09:50:38.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.361+0000 7f3821cfa700 1 --2- 172.21.15.78:0/3050619164 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3800065ed0 0x7f3800068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:38.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.365+0000 7f3821cfa700 1 --2- 172.21.15.78:0/3050619164 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3800065ed0 0x7f3800068380 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f381c0c9980 tx=0x7f381800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:38.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.369+0000 7f38127fc700 1 -- 172.21.15.78:0/3050619164 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f380c0502d0 con 0x7f381c0ce500 2023-10-11T09:50:38.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.581+0000 7f3823f5e700 1 -- 172.21.15.78:0/3050619164 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f381c070510 con 0x7f381c0ce500 2023-10-11T09:50:38.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.581+0000 7f38127fc700 1 -- 172.21.15.78:0/3050619164 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4655 (secure 0 0 0) 0x7f380c0538f0 con 0x7f381c0ce500 2023-10-11T09:50:38.591 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:38.591 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":91,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7148,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/331520590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":331520590},{"type":"v1","addr":"172.21.15.78:6835","nonce":331520590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":7151,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/934539531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":934539531},{"type":"v1","addr":"172.21.15.133:6834","nonce":934539531}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":7154,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3894002128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":3894002128},{"type":"v1","addr":"172.21.15.133:6835","nonce":3894002128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:50:28.502984+0000","modified":"2023-10-11T09:50:37.659950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8550":{"gid":8550,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":4,"addr":"172.21.15.78:6837/2869034036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2869034036},{"type":"v1","addr":"172.21.15.78:6837","nonce":2869034036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15,16],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-11T09:50:38.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.585+0000 7f3823f5e700 1 -- 172.21.15.78:0/3050619164 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3800065ed0 msgr2=0x7f3800068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:38.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.585+0000 7f3823f5e700 1 --2- 172.21.15.78:0/3050619164 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3800065ed0 0x7f3800068380 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f381c0c9980 tx=0x7f381800b040 comp rx=0 tx=0).stop 2023-10-11T09:50:38.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.585+0000 7f3823f5e700 1 -- 172.21.15.78:0/3050619164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f381c0ce500 msgr2=0x7f381c15bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:38.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.585+0000 7f3823f5e700 1 --2- 172.21.15.78:0/3050619164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f381c0ce500 0x7f381c15bc80 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f380c00c520 tx=0x7f380c00c8e0 comp rx=0 tx=0).stop 2023-10-11T09:50:38.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.585+0000 7f3823f5e700 1 -- 172.21.15.78:0/3050619164 shutdown_connections 2023-10-11T09:50:38.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.585+0000 7f3823f5e700 1 --2- 172.21.15.78:0/3050619164 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3800065ed0 0x7f3800068380 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:38.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.585+0000 7f3823f5e700 1 --2- 172.21.15.78:0/3050619164 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f381c160d40 0x7f381c15c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:38.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.585+0000 7f3823f5e700 1 --2- 172.21.15.78:0/3050619164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f381c0ce500 0x7f381c15bc80 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:38.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.585+0000 7f3823f5e700 1 --2- 172.21.15.78:0/3050619164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f381c071040 0x7f381c0cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:38.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.585+0000 7f3823f5e700 1 -- 172.21.15.78:0/3050619164 >> 172.21.15.78:0/3050619164 conn(0x7f381c06aca0 msgr2=0x7f381c06d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:38.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.585+0000 7f3823f5e700 1 -- 172.21.15.78:0/3050619164 shutdown_connections 2023-10-11T09:50:38.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:38.585+0000 7f3823f5e700 1 -- 172.21.15.78:0/3050619164 wait complete. 2023-10-11T09:50:38.597 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 91 2023-10-11T09:50:39.018 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 91, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-11T09:50:28.502984+0000', 'modified': '2023-10-11T09:50:37.659950+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 112, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8550}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8550': {'gid': 8550, 'name': 'c', 'rank': 0, 'incarnation': 88, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.78:6837/2869034036', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6836', 'nonce': 2869034036}, {'type': 'v1', 'addr': '172.21.15.78:6837', 'nonce': 2869034036}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15, 16], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-11T09:50:39.019 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-11T09:50:39.019 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-11T09:50:39.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.185+0000 7f63d1cc4700 1 Processor -- start 2023-10-11T09:50:39.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.185+0000 7f63d1cc4700 1 -- start start 2023-10-11T09:50:39.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.189+0000 7f63d1cc4700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f63cc13f740 0x7f63cc0ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:39.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.189+0000 7f63d1cc4700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63cc06d180 0x7f63cc0b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:39.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.189+0000 7f63d1cc4700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63cc070c80 con 0x7f63cc06cd00 2023-10-11T09:50:39.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.189+0000 7f63d1cc4700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63cc0ce4c0 con 0x7f63cc13f740 2023-10-11T09:50:39.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.189+0000 7f63d1cc4700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc06d600 con 0x7f63cc06d180 2023-10-11T09:50:39.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.189+0000 7f63cbfff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f63cc06cd00 0x7f63cc0b06a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:41584/0 (socket says 172.21.15.78:41584) 2023-10-11T09:50:39.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.189+0000 7f63cbfff700 1 -- 172.21.15.78:0/1699484866 learned_addr learned my addr 172.21.15.78:0/1699484866 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:50:39.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.189+0000 7f63d0cc2700 1 --2- 172.21.15.78:0/1699484866 >> v2:172.21.15.133:3300/0 conn(0x7f63cc13f740 0x7f63cc0ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:39.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.189+0000 7f63d14c3700 1 --2- 172.21.15.78:0/1699484866 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63cc06d180 0x7f63cc0b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:39.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.189+0000 7f63cb7fe700 1 -- 172.21.15.78:0/1699484866 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 61725110 0 0) 0x7f63cc070c80 con 0x7f63cc06cd00 2023-10-11T09:50:39.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.189+0000 7f63cb7fe700 1 -- 172.21.15.78:0/1699484866 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63bc0035d0 con 0x7f63cc06cd00 2023-10-11T09:50:39.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.189+0000 7f63cb7fe700 1 -- 172.21.15.78:0/1699484866 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1381582680 0 0) 0x7f63bc0035d0 con 0x7f63cc06cd00 2023-10-11T09:50:39.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.189+0000 7f63cb7fe700 1 -- 172.21.15.78:0/1699484866 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63cc06d180 msgr2=0x7f63cc0b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:39.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.189+0000 7f63cb7fe700 1 --2- 172.21.15.78:0/1699484866 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63cc06d180 0x7f63cc0b0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.189+0000 7f63cb7fe700 1 -- 172.21.15.78:0/1699484866 >> v2:172.21.15.133:3300/0 conn(0x7f63cc13f740 msgr2=0x7f63cc0ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:39.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.189+0000 7f63cb7fe700 1 --2- 172.21.15.78:0/1699484866 >> v2:172.21.15.133:3300/0 conn(0x7f63cc13f740 0x7f63cc0ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.189+0000 7f63cb7fe700 1 -- 172.21.15.78:0/1699484866 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63cc0cd420 con 0x7f63cc06cd00 2023-10-11T09:50:39.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.189+0000 7f63cb7fe700 1 -- 172.21.15.78:0/1699484866 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f63b8001670 con 0x7f63cc06cd00 2023-10-11T09:50:39.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.189+0000 7f63cb7fe700 1 -- 172.21.15.78:0/1699484866 >> v1:172.21.15.78:6789/0 conn(0x7f63cc06cd00 legacy=0x7f63cc0b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:39.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.189+0000 7f63cb7fe700 1 --2- 172.21.15.78:0/1699484866 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63bc003a20 0x7f63bc003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:39.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.189+0000 7f63cb7fe700 1 --2- 172.21.15.78:0/1699484866 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63cc06d180 0x7f63bc004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:39.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.189+0000 7f63cb7fe700 1 --2- 172.21.15.78:0/1699484866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63bc004db0 0x7f63bc005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:39.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.189+0000 7f63cb7fe700 1 -- 172.21.15.78:0/1699484866 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc0ce4c0 con 0x7f63bc004db0 2023-10-11T09:50:39.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.189+0000 7f63cb7fe700 1 -- 172.21.15.78:0/1699484866 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc06d600 con 0x7f63bc003a20 2023-10-11T09:50:39.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.189+0000 7f63cb7fe700 1 -- 172.21.15.78:0/1699484866 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc00ba30 con 0x7f63cc06d180 2023-10-11T09:50:39.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.189+0000 7f63d14c3700 1 --2- 172.21.15.78:0/1699484866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63bc004db0 0x7f63bc005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:39.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.189+0000 7f63cbfff700 1 --2- 172.21.15.78:0/1699484866 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63cc06d180 0x7f63bc004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:39.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.189+0000 7f63d0cc2700 1 --2- 172.21.15.78:0/1699484866 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63bc003a20 0x7f63bc003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:39.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.189+0000 7f63d14c3700 1 -- 172.21.15.78:0/1699484866 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63cc06d180 msgr2=0x7f63bc004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:39.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.189+0000 7f63d14c3700 1 --2- 172.21.15.78:0/1699484866 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63cc06d180 0x7f63bc004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.193+0000 7f63d14c3700 1 -- 172.21.15.78:0/1699484866 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63bc003a20 msgr2=0x7f63bc003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:39.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.193+0000 7f63d14c3700 1 --2- 172.21.15.78:0/1699484866 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63bc003a20 0x7f63bc003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.193+0000 7f63d14c3700 1 -- 172.21.15.78:0/1699484866 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63bc0035d0 con 0x7f63bc004db0 2023-10-11T09:50:39.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.193+0000 7f63d0cc2700 1 --2- 172.21.15.78:0/1699484866 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63bc003a20 0x7f63bc003df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:50:39.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.193+0000 7f63d14c3700 1 --2- 172.21.15.78:0/1699484866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63bc004db0 0x7f63bc005220 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f63c4009fc0 tx=0x7f63bc0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad70d666e080f8f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:39.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.193+0000 7f63cb7fe700 1 -- 172.21.15.78:0/1699484866 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c400ebd0 con 0x7f63bc004db0 2023-10-11T09:50:39.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.193+0000 7f63cb7fe700 1 -- 172.21.15.78:0/1699484866 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c400f040 con 0x7f63bc004db0 2023-10-11T09:50:39.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.193+0000 7f63cb7fe700 1 -- 172.21.15.78:0/1699484866 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c4014480 con 0x7f63bc004db0 2023-10-11T09:50:39.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.193+0000 7f63d1cc4700 1 -- 172.21.15.78:0/1699484866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63bc004db0 msgr2=0x7f63bc005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:39.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.193+0000 7f63d1cc4700 1 --2- 172.21.15.78:0/1699484866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63bc004db0 0x7f63bc005220 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f63c4009fc0 tx=0x7f63bc0058a0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.193+0000 7f63d14c3700 1 -- 172.21.15.78:0/1699484866 reap_dead start 2023-10-11T09:50:39.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.193+0000 7f63d1cc4700 1 -- 172.21.15.78:0/1699484866 shutdown_connections 2023-10-11T09:50:39.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.193+0000 7f63d1cc4700 1 -- 172.21.15.78:0/1699484866 >> 172.21.15.78:0/1699484866 conn(0x7f63cc0cea60 msgr2=0x7f63cc0cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:39.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.193+0000 7f63d1cc4700 1 -- 172.21.15.78:0/1699484866 shutdown_connections 2023-10-11T09:50:39.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.193+0000 7f63d1cc4700 1 -- 172.21.15.78:0/1699484866 wait complete. 2023-10-11T09:50:39.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.197+0000 7f63d1cc4700 1 Processor -- start 2023-10-11T09:50:39.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.197+0000 7f63d1cc4700 1 -- start start 2023-10-11T09:50:39.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.197+0000 7f63d1cc4700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63bc004db0 0x7f63cc150b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:39.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.197+0000 7f63d1cc4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63cc1ed990 0x7f63cc0b4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:39.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.197+0000 7f63d1cc4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63cc13e340 0x7f63cc13e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:39.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.197+0000 7f63d1cc4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc159870 con 0x7f63cc1ed990 2023-10-11T09:50:39.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.197+0000 7f63d1cc4700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc13ecf0 con 0x7f63cc13e340 2023-10-11T09:50:39.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.197+0000 7f63d1cc4700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc1edf00 con 0x7f63bc004db0 2023-10-11T09:50:39.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.197+0000 7f63cbfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63cc1ed990 0x7f63cc0b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:39.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.197+0000 7f63cbfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63cc1ed990 0x7f63cc0b4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40930/0 (socket says 172.21.15.78:40930) 2023-10-11T09:50:39.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.197+0000 7f63cbfff700 1 -- 172.21.15.78:0/3018144467 learned_addr learned my addr 172.21.15.78:0/3018144467 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:39.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.201+0000 7f63d0cc2700 1 --2- 172.21.15.78:0/3018144467 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63bc004db0 0x7f63cc150b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:39.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.201+0000 7f63d14c3700 1 --2- 172.21.15.78:0/3018144467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63cc13e340 0x7f63cc13e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:39.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.201+0000 7f63cbfff700 1 -- 172.21.15.78:0/3018144467 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63bc004db0 msgr2=0x7f63cc150b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:39.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.201+0000 7f63cbfff700 1 --2- 172.21.15.78:0/3018144467 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63bc004db0 0x7f63cc150b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.201+0000 7f63cbfff700 1 -- 172.21.15.78:0/3018144467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63cc13e340 msgr2=0x7f63cc13e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:39.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.201+0000 7f63cbfff700 1 --2- 172.21.15.78:0/3018144467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63cc13e340 0x7f63cc13e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.201+0000 7f63cbfff700 1 -- 172.21.15.78:0/3018144467 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63cc1ee1e0 con 0x7f63cc1ed990 2023-10-11T09:50:39.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.201+0000 7f63cbfff700 1 --2- 172.21.15.78:0/3018144467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63cc1ed990 0x7f63cc0b4400 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f63b80016e0 tx=0x7f63b80117d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:39.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.201+0000 7f63c97fa700 1 -- 172.21.15.78:0/3018144467 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b8016070 con 0x7f63cc1ed990 2023-10-11T09:50:39.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.201+0000 7f63c97fa700 1 -- 172.21.15.78:0/3018144467 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63b801b5a0 con 0x7f63cc1ed990 2023-10-11T09:50:39.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.201+0000 7f63d1cc4700 1 -- 172.21.15.78:0/3018144467 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63cc1ee4c0 con 0x7f63cc1ed990 2023-10-11T09:50:39.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.201+0000 7f63c97fa700 1 -- 172.21.15.78:0/3018144467 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b801b950 con 0x7f63cc1ed990 2023-10-11T09:50:39.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.201+0000 7f63d1cc4700 1 -- 172.21.15.78:0/3018144467 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63cc1510a0 con 0x7f63cc1ed990 2023-10-11T09:50:39.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.205+0000 7f63c97fa700 1 -- 172.21.15.78:0/3018144467 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f63b801bab0 con 0x7f63cc1ed990 2023-10-11T09:50:39.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.205+0000 7f63c97fa700 1 --2- 172.21.15.78:0/3018144467 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f63b0065f00 0x7f63b00683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:39.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.205+0000 7f63c97fa700 1 -- 172.21.15.78:0/3018144467 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f63b8087bb0 con 0x7f63cc1ed990 2023-10-11T09:50:39.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.205+0000 7f63d0cc2700 1 --2- 172.21.15.78:0/3018144467 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f63b0065f00 0x7f63b00683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:39.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.209+0000 7f63d61b6740 1 Processor -- start 2023-10-11T09:50:39.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.209+0000 7f63d0cc2700 1 --2- 172.21.15.78:0/3018144467 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f63b0065f00 0x7f63b00683b0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f63cc0c9730 tx=0x7f63c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:39.218 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.209+0000 7f63d61b6740 1 -- start start 2023-10-11T09:50:39.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.209+0000 7f63d61b6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1387eb0 0x1640020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:39.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.209+0000 7f63d61b6740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1387ae0 0x1640560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:39.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.209+0000 7f63d61b6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1387710 0x1607640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:39.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.209+0000 7f63d61b6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x15674a0 con 0x1387eb0 2023-10-11T09:50:39.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.209+0000 7f63d61b6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x1564c30 con 0x1387710 2023-10-11T09:50:39.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.209+0000 7f63d61b6740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x1560520 con 0x1387ae0 2023-10-11T09:50:39.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.209+0000 7f63d0cc2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1387eb0 0x1640020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:39.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.209+0000 7f63d0cc2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1387eb0 0x1640020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40942/0 (socket says 172.21.15.78:40942) 2023-10-11T09:50:39.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.209+0000 7f63d0cc2700 1 -- 172.21.15.78:0/1518971427 learned_addr learned my addr 172.21.15.78:0/1518971427 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:39.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.209+0000 7f63cbfff700 1 --2- 172.21.15.78:0/1518971427 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1387ae0 0x1640560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:39.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.209+0000 7f63d14c3700 1 --2- 172.21.15.78:0/1518971427 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1387710 0x1607640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:39.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.209+0000 7f63d0cc2700 1 -- 172.21.15.78:0/1518971427 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1387ae0 msgr2=0x1640560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:39.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.209+0000 7f63d0cc2700 1 --2- 172.21.15.78:0/1518971427 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1387ae0 0x1640560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.209+0000 7f63d0cc2700 1 -- 172.21.15.78:0/1518971427 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1387710 msgr2=0x1607640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:39.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.209+0000 7f63d0cc2700 1 --2- 172.21.15.78:0/1518971427 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1387710 0x1607640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.209+0000 7f63d0cc2700 1 -- 172.21.15.78:0/1518971427 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x13d39e0 con 0x1387eb0 2023-10-11T09:50:39.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.209+0000 7f63d14c3700 1 --2- 172.21.15.78:0/1518971427 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1387710 0x1607640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:50:39.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.209+0000 7f63d0cc2700 1 --2- 172.21.15.78:0/1518971427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1387eb0 0x1640020 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f63c0013cb0 tx=0x7f63c0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76cf2c13db72bbe5 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:39.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.213+0000 7f63b67fc700 1 -- 172.21.15.78:0/1518971427 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c0015bb0 con 0x1387eb0 2023-10-11T09:50:39.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.213+0000 7f63b67fc700 1 -- 172.21.15.78:0/1518971427 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c0015d10 con 0x1387eb0 2023-10-11T09:50:39.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.213+0000 7f63b67fc700 1 -- 172.21.15.78:0/1518971427 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c0022650 con 0x1387eb0 2023-10-11T09:50:39.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.213+0000 7f63d61b6740 1 -- 172.21.15.78:0/1518971427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1387eb0 msgr2=0x1640020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:39.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.213+0000 7f63d61b6740 1 --2- 172.21.15.78:0/1518971427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1387eb0 0x1640020 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f63c0013cb0 tx=0x7f63c0016000 comp rx=0 tx=0).stop 2023-10-11T09:50:39.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.213+0000 7f63d61b6740 1 -- 172.21.15.78:0/1518971427 shutdown_connections 2023-10-11T09:50:39.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.213+0000 7f63d61b6740 1 --2- 172.21.15.78:0/1518971427 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1387710 0x1607640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.213+0000 7f63d61b6740 1 --2- 172.21.15.78:0/1518971427 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1387ae0 0x1640560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.213+0000 7f63d61b6740 1 --2- 172.21.15.78:0/1518971427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1387eb0 0x1640020 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.213+0000 7f63d61b6740 1 -- 172.21.15.78:0/1518971427 >> 172.21.15.78:0/1518971427 conn(0x15effa0 msgr2=0x1621420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:39.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.213+0000 7f63d61b6740 1 -- 172.21.15.78:0/1518971427 shutdown_connections 2023-10-11T09:50:39.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.213+0000 7f63d61b6740 1 -- 172.21.15.78:0/1518971427 wait complete. 2023-10-11T09:50:39.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.213+0000 7f63d61b6740 1 -- start start 2023-10-11T09:50:39.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.213+0000 7f63d61b6740 1 Processor -- start 2023-10-11T09:50:39.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.217+0000 7f63d61b6740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1387710 0x16486f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:39.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.217+0000 7f63d61b6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1387ae0 0x1617c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:39.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.217+0000 7f63d61b6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1387eb0 0x1618710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:39.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.217+0000 7f63d61b6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x1523b00 con 0x1387ae0 2023-10-11T09:50:39.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.217+0000 7f63d61b6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x15237a0 con 0x1387eb0 2023-10-11T09:50:39.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.217+0000 7f63d61b6740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x140cb20 con 0x1387710 2023-10-11T09:50:39.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.217+0000 7f63d61b6740 10 client.0 _ll_register_callbacks cb 0x1559e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:50:39.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.217+0000 7f63cbfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1387ae0 0x1617c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:39.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.217+0000 7f63cbfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1387ae0 0x1617c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40954/0 (socket says 172.21.15.78:40954) 2023-10-11T09:50:39.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.217+0000 7f63cbfff700 1 -- 172.21.15.78:0/2082527552 learned_addr learned my addr 172.21.15.78:0/2082527552 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:39.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.217+0000 7f63d0cc2700 1 --2- 172.21.15.78:0/2082527552 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1387710 0x16486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:39.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.217+0000 7f63d14c3700 1 --2- 172.21.15.78:0/2082527552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1387eb0 0x1618710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:39.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.217+0000 7f63cbfff700 1 -- 172.21.15.78:0/2082527552 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1387710 msgr2=0x16486f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:39.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.217+0000 7f63cbfff700 1 --2- 172.21.15.78:0/2082527552 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1387710 0x16486f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.217+0000 7f63cbfff700 1 -- 172.21.15.78:0/2082527552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1387eb0 msgr2=0x1618710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:39.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.217+0000 7f63cbfff700 1 --2- 172.21.15.78:0/2082527552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1387eb0 0x1618710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.217+0000 7f63cbfff700 1 -- 172.21.15.78:0/2082527552 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63c0013960 con 0x1387ae0 2023-10-11T09:50:39.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.217+0000 7f63d14c3700 1 --2- 172.21.15.78:0/2082527552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1387eb0 0x1618710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:50:39.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.217+0000 7f63cbfff700 1 --2- 172.21.15.78:0/2082527552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1387ae0 0x1617c40 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f63b8085c40 tx=0x7f63b80593c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d799cd26068c4e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:39.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.217+0000 7f63ab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:50:39.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.217+0000 7f63ab7fe700 1 -- 172.21.15.78:0/2082527552 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b8070170 con 0x1387ae0 2023-10-11T09:50:39.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.217+0000 7f63ab7fe700 1 -- 172.21.15.78:0/2082527552 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f63b8060b60 con 0x1387ae0 2023-10-11T09:50:39.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.217+0000 7f63d61b6740 1 -- 172.21.15.78:0/2082527552 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x15595a0 con 0x1387ae0 2023-10-11T09:50:39.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.217+0000 7f63ab7fe700 1 -- 172.21.15.78:0/2082527552 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63b8056f20 con 0x1387ae0 2023-10-11T09:50:39.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.217+0000 7f63d61b6740 10 client.8583 fetch_fsmap learned FSMap version 91 2023-10-11T09:50:39.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.217+0000 7f63d61b6740 1 -- 172.21.15.78:0/2082527552 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x1526750 con 0x1387ae0 2023-10-11T09:50:39.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.217+0000 7f63ab7fe700 1 -- 172.21.15.78:0/2082527552 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x15595a0 con 0x1387ae0 2023-10-11T09:50:39.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.217+0000 7f63ab7fe700 1 -- 172.21.15.78:0/2082527552 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 91) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f63b8058070 con 0x1387ae0 2023-10-11T09:50:39.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.217+0000 7f63d61b6740 10 client.8583 fetch_fsmap finished waiting for FSMap version 91 2023-10-11T09:50:39.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.217+0000 7f63d61b6740 10 client.8583 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-11T09:50:39.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.217+0000 7f63d61b6740 20 client.8583 populate_metadata read hostname 'smithi078' 2023-10-11T09:50:39.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.217+0000 7f63d61b6740 1 --2- 172.21.15.78:0/2082527552 >> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] conn(0x13d2790 0x1653000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:39.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.217+0000 7f63d61b6740 4 client.8583 mds_command: new command op to 8550 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-11T09:50:39.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.217+0000 7f63d61b6740 1 -- 172.21.15.78:0/2082527552 --> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x15592c0 con 0x13d2790 2023-10-11T09:50:39.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.217+0000 7f63d0cc2700 1 --2- 172.21.15.78:0/2082527552 >> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] conn(0x13d2790 0x1653000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:39.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.217+0000 7f63d0cc2700 1 --2- 172.21.15.78:0/2082527552 >> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] conn(0x13d2790 0x1653000 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b516dabf8a8e00cc server_cookie=d2aaa5cbb82eb98d in_seq=0 out_seq=0 2023-10-11T09:50:39.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.217+0000 7f63ab7fe700 10 client.8583 ms_handle_connect on v2:172.21.15.78:6836/2869034036 2023-10-11T09:50:39.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.221+0000 7f63ab7fe700 1 -- 172.21.15.78:0/2082527552 <== mds.0 v2:172.21.15.78:6836/2869034036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x15592c0 con 0x13d2790 2023-10-11T09:50:39.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.221+0000 7f63ab7fe700 10 client.8583 handle_command_reply: tid=0 2023-10-11T09:50:39.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.221+0000 7f63d61b6740 1 client.8583 shutdown 2023-10-11T09:50:39.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.221+0000 7f63d61b6740 1 -- 172.21.15.78:0/2082527552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1387ae0 msgr2=0x1617c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:39.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.221+0000 7f63d61b6740 1 --2- 172.21.15.78:0/2082527552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1387ae0 0x1617c40 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f63b8085c40 tx=0x7f63b80593c0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.221+0000 7f63d61b6740 1 -- 172.21.15.78:0/2082527552 shutdown_connections 2023-10-11T09:50:39.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.221+0000 7f63d61b6740 1 --2- 172.21.15.78:0/2082527552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1387eb0 0x1618710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.221+0000 7f63d61b6740 1 --2- 172.21.15.78:0/2082527552 >> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] conn(0x13d2790 0x1653000 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.221+0000 7f63d61b6740 1 --2- 172.21.15.78:0/2082527552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1387ae0 0x1617c40 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.221+0000 7f63ab7fe700 0 client.8583 ms_handle_reset on v2:172.21.15.78:6836/2869034036 2023-10-11T09:50:39.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.221+0000 7f63d61b6740 1 --2- 172.21.15.78:0/2082527552 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1387710 0x16486f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.221+0000 7f63d61b6740 1 -- 172.21.15.78:0/2082527552 >> 172.21.15.78:0/2082527552 conn(0x15effa0 msgr2=0x161fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:39.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.221+0000 7f63d61b6740 1 -- 172.21.15.78:0/2082527552 shutdown_connections 2023-10-11T09:50:39.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.221+0000 7f63d61b6740 1 -- 172.21.15.78:0/2082527552 wait complete. 2023-10-11T09:50:39.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.221+0000 7f63d61b6740 20 client.8583 trim_cache size 0 max 16384 2023-10-11T09:50:39.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.237+0000 7f63d61b6740 1 Processor -- start 2023-10-11T09:50:39.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.237+0000 7f63d61b6740 1 -- start start 2023-10-11T09:50:39.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.241+0000 7f63d61b6740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1387710 0x16175d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:39.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.241+0000 7f63d61b6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1387ae0 0x1617b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:39.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.241+0000 7f63d61b6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1387eb0 0x1644cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:39.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.241+0000 7f63d61b6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x1505760 con 0x1387ae0 2023-10-11T09:50:39.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.241+0000 7f63d61b6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x151a760 con 0x1387eb0 2023-10-11T09:50:39.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.241+0000 7f63d61b6740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x15255c0 con 0x1387710 2023-10-11T09:50:39.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.241+0000 7f63cbfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1387ae0 0x1617b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:39.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.241+0000 7f63cbfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1387ae0 0x1617b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40968/0 (socket says 172.21.15.78:40968) 2023-10-11T09:50:39.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.241+0000 7f63cbfff700 1 -- 172.21.15.78:0/3539553650 learned_addr learned my addr 172.21.15.78:0/3539553650 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:39.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.241+0000 7f63d0cc2700 1 --2- 172.21.15.78:0/3539553650 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1387710 0x16175d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:39.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.241+0000 7f63d14c3700 1 --2- 172.21.15.78:0/3539553650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1387eb0 0x1644cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:39.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.241+0000 7f63cbfff700 1 -- 172.21.15.78:0/3539553650 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1387710 msgr2=0x16175d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:39.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.241+0000 7f63cbfff700 1 --2- 172.21.15.78:0/3539553650 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1387710 0x16175d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.241+0000 7f63cbfff700 1 -- 172.21.15.78:0/3539553650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1387eb0 msgr2=0x1644cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:39.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.241+0000 7f63cbfff700 1 --2- 172.21.15.78:0/3539553650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1387eb0 0x1644cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.241+0000 7f63cbfff700 1 -- 172.21.15.78:0/3539553650 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63cc1577a0 con 0x1387ae0 2023-10-11T09:50:39.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.241+0000 7f63d14c3700 1 --2- 172.21.15.78:0/3539553650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1387eb0 0x1644cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:50:39.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.241+0000 7f63cbfff700 1 --2- 172.21.15.78:0/3539553650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1387ae0 0x1617b10 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f63b8085c90 tx=0x7f63b805c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7865f832fba5de5 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:39.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.241+0000 7f63b6ffd700 1 -- 172.21.15.78:0/3539553650 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b8056f20 con 0x1387ae0 2023-10-11T09:50:39.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.241+0000 7f63b6ffd700 1 -- 172.21.15.78:0/3539553650 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63b80586c0 con 0x1387ae0 2023-10-11T09:50:39.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.241+0000 7f63b6ffd700 1 -- 172.21.15.78:0/3539553650 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b8070170 con 0x1387ae0 2023-10-11T09:50:39.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.245+0000 7f63d61b6740 1 -- 172.21.15.78:0/3539553650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1387ae0 msgr2=0x1617b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:39.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.245+0000 7f63d61b6740 1 --2- 172.21.15.78:0/3539553650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1387ae0 0x1617b10 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f63b8085c90 tx=0x7f63b805c000 comp rx=0 tx=0).stop 2023-10-11T09:50:39.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.245+0000 7f63d61b6740 1 -- 172.21.15.78:0/3539553650 shutdown_connections 2023-10-11T09:50:39.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.245+0000 7f63d61b6740 1 --2- 172.21.15.78:0/3539553650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x1387eb0 0x1644cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.245+0000 7f63d61b6740 1 --2- 172.21.15.78:0/3539553650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1387ae0 0x1617b10 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.245+0000 7f63d61b6740 1 --2- 172.21.15.78:0/3539553650 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1387710 0x16175d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.245+0000 7f63d61b6740 1 -- 172.21.15.78:0/3539553650 >> 172.21.15.78:0/3539553650 conn(0x15effa0 msgr2=0x15f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:39.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.245+0000 7f63d61b6740 1 -- 172.21.15.78:0/3539553650 shutdown_connections 2023-10-11T09:50:39.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.245+0000 7f63d61b6740 1 -- 172.21.15.78:0/3539553650 wait complete. 2023-10-11T09:50:39.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.245+0000 7f63d61b6740 1 -- start start 2023-10-11T09:50:39.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.245+0000 7f63d61b6740 1 Processor -- start 2023-10-11T09:50:39.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.249+0000 7f63d61b6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1387710 0x169b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:39.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.249+0000 7f63d61b6740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1387eb0 0x1617ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:39.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.249+0000 7f63d61b6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x164fa20 0x1643cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:39.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.249+0000 7f63d61b6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x1561ab0 con 0x1387710 2023-10-11T09:50:39.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.249+0000 7f63d61b6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x1637000 con 0x164fa20 2023-10-11T09:50:39.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.249+0000 7f63d61b6740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x1647f90 con 0x1387eb0 2023-10-11T09:50:39.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.249+0000 7f63d61b6740 10 client.0 _ll_register_callbacks cb 0x155ec80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:50:39.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.249+0000 7f63d0cc2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1387710 0x169b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:39.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.249+0000 7f63d0cc2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1387710 0x169b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40980/0 (socket says 172.21.15.78:40980) 2023-10-11T09:50:39.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.249+0000 7f63d0cc2700 1 -- 172.21.15.78:0/3438602238 learned_addr learned my addr 172.21.15.78:0/3438602238 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:39.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.249+0000 7f63d14c3700 1 --2- 172.21.15.78:0/3438602238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x164fa20 0x1643cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:39.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.249+0000 7f63cbfff700 1 --2- 172.21.15.78:0/3438602238 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1387eb0 0x1617ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:39.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.249+0000 7f63d0cc2700 1 -- 172.21.15.78:0/3438602238 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1387eb0 msgr2=0x1617ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:39.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.249+0000 7f63d0cc2700 1 --2- 172.21.15.78:0/3438602238 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1387eb0 0x1617ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.249+0000 7f63d0cc2700 1 -- 172.21.15.78:0/3438602238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x164fa20 msgr2=0x1643cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:39.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.249+0000 7f63d0cc2700 1 --2- 172.21.15.78:0/3438602238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x164fa20 0x1643cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.249+0000 7f63d0cc2700 1 -- 172.21.15.78:0/3438602238 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1526750 con 0x1387710 2023-10-11T09:50:39.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.249+0000 7f63d0cc2700 1 --2- 172.21.15.78:0/3438602238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1387710 0x169b630 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f63c00136d0 tx=0x7f63c00047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1535afa29a6543e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:39.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.249+0000 7f63ab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:50:39.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.249+0000 7f63ab7fe700 1 -- 172.21.15.78:0/3438602238 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c0031040 con 0x1387710 2023-10-11T09:50:39.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.249+0000 7f63ab7fe700 1 -- 172.21.15.78:0/3438602238 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f63c0038ba0 con 0x1387710 2023-10-11T09:50:39.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.249+0000 7f63d61b6740 1 -- 172.21.15.78:0/3438602238 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63c0015d10 con 0x1387710 2023-10-11T09:50:39.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.249+0000 7f63ab7fe700 1 -- 172.21.15.78:0/3438602238 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63c0038430 con 0x1387710 2023-10-11T09:50:39.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.249+0000 7f63d61b6740 10 client.8589 fetch_fsmap learned FSMap version 91 2023-10-11T09:50:39.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.249+0000 7f63d61b6740 1 -- 172.21.15.78:0/3438602238 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x150c980 con 0x1387710 2023-10-11T09:50:39.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.249+0000 7f63ab7fe700 1 -- 172.21.15.78:0/3438602238 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c0015d10 con 0x1387710 2023-10-11T09:50:39.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.249+0000 7f63ab7fe700 1 -- 172.21.15.78:0/3438602238 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 91) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f63c002c640 con 0x1387710 2023-10-11T09:50:39.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.253+0000 7f63d61b6740 10 client.8589 fetch_fsmap finished waiting for FSMap version 91 2023-10-11T09:50:39.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.253+0000 7f63d61b6740 10 client.8589 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-11T09:50:39.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.253+0000 7f63d61b6740 20 client.8589 populate_metadata read hostname 'smithi078' 2023-10-11T09:50:39.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.253+0000 7f63d61b6740 1 --2- 172.21.15.78:0/3438602238 >> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] conn(0x161d750 0x16165c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:39.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.253+0000 7f63d61b6740 4 client.8589 mds_command: new command op to 8550 tid=0[{"prefix": "status"}] 2023-10-11T09:50:39.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.253+0000 7f63d61b6740 1 -- 172.21.15.78:0/3438602238 --> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] -- command(tid 0: {"prefix": "status"}) v1 -- 0x13df4e0 con 0x161d750 2023-10-11T09:50:39.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.253+0000 7f63cbfff700 1 --2- 172.21.15.78:0/3438602238 >> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] conn(0x161d750 0x16165c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:39.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.253+0000 7f63cbfff700 1 --2- 172.21.15.78:0/3438602238 >> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] conn(0x161d750 0x16165c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=312bd96926ee4916 server_cookie=1a442fa07c18d9a in_seq=0 out_seq=0 2023-10-11T09:50:39.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.253+0000 7f63ab7fe700 10 client.8589 ms_handle_connect on v2:172.21.15.78:6836/2869034036 2023-10-11T09:50:39.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.253+0000 7f63ab7fe700 1 -- 172.21.15.78:0/3438602238 <== mds.0 v2:172.21.15.78:6836/2869034036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x13df4e0 con 0x161d750 2023-10-11T09:50:39.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.253+0000 7f63ab7fe700 10 client.8589 handle_command_reply: tid=0 2023-10-11T09:50:39.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.253+0000 7f63d61b6740 1 client.8589 shutdown 2023-10-11T09:50:39.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.253+0000 7f63d61b6740 1 -- 172.21.15.78:0/3438602238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1387710 msgr2=0x169b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:39.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.253+0000 7f63d61b6740 1 --2- 172.21.15.78:0/3438602238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1387710 0x169b630 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f63c00136d0 tx=0x7f63c00047e0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.253+0000 7f63d61b6740 1 -- 172.21.15.78:0/3438602238 shutdown_connections 2023-10-11T09:50:39.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.253+0000 7f63d61b6740 1 --2- 172.21.15.78:0/3438602238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x164fa20 0x1643cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.253+0000 7f63d61b6740 1 --2- 172.21.15.78:0/3438602238 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1387eb0 0x1617ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.253+0000 7f63d61b6740 1 --2- 172.21.15.78:0/3438602238 >> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] conn(0x161d750 0x16165c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.253+0000 7f63d61b6740 1 --2- 172.21.15.78:0/3438602238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1387710 0x169b630 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.253+0000 7f63d61b6740 1 -- 172.21.15.78:0/3438602238 >> 172.21.15.78:0/3438602238 conn(0x15effa0 msgr2=0x15fa190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:39.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.253+0000 7f63ab7fe700 0 client.8589 ms_handle_reset on v2:172.21.15.78:6836/2869034036 2023-10-11T09:50:39.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.253+0000 7f63d61b6740 1 -- 172.21.15.78:0/3438602238 shutdown_connections 2023-10-11T09:50:39.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.253+0000 7f63d61b6740 1 -- 172.21.15.78:0/3438602238 wait complete. 2023-10-11T09:50:39.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.253+0000 7f63d61b6740 20 client.8589 trim_cache size 0 max 16384 2023-10-11T09:50:39.272 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:50:39.272 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "dee99db9-7a72-406f-a1e2-917943f2b13d", 2023-10-11T09:50:39.272 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2023-10-11T09:50:39.273 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 8550, 2023-10-11T09:50:39.273 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2023-10-11T09:50:39.273 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2023-10-11T09:50:39.273 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2023-10-11T09:50:39.273 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 4.6064786350000002, 2023-10-11T09:50:39.273 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 91, 2023-10-11T09:50:39.274 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 114, 2023-10-11T09:50:39.274 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 114, 2023-10-11T09:50:39.274 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 4.6427373660000004 2023-10-11T09:50:39.274 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:50:39.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.253+0000 7f63d1cc4700 1 -- 172.21.15.78:0/3018144467 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f63b0065f00 msgr2=0x7f63b00683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:39.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.253+0000 7f63d1cc4700 1 --2- 172.21.15.78:0/3018144467 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f63b0065f00 0x7f63b00683b0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f63cc0c9730 tx=0x7f63c0008040 comp rx=0 tx=0).stop 2023-10-11T09:50:39.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.253+0000 7f63d1cc4700 1 -- 172.21.15.78:0/3018144467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63cc1ed990 msgr2=0x7f63cc0b4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:39.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.253+0000 7f63d1cc4700 1 --2- 172.21.15.78:0/3018144467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63cc1ed990 0x7f63cc0b4400 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f63b80016e0 tx=0x7f63b80117d0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.257+0000 7f63d1cc4700 1 -- 172.21.15.78:0/3018144467 shutdown_connections 2023-10-11T09:50:39.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.257+0000 7f63d1cc4700 1 --2- 172.21.15.78:0/3018144467 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f63b0065f00 0x7f63b00683b0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.257+0000 7f63d1cc4700 1 --2- 172.21.15.78:0/3018144467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63cc13e340 0x7f63cc13e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.257+0000 7f63d1cc4700 1 --2- 172.21.15.78:0/3018144467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63cc1ed990 0x7f63cc0b4400 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.257+0000 7f63d1cc4700 1 --2- 172.21.15.78:0/3018144467 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63bc004db0 0x7f63cc150b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.257+0000 7f63d1cc4700 1 -- 172.21.15.78:0/3018144467 >> 172.21.15.78:0/3018144467 conn(0x7f63cc0cea60 msgr2=0x7f63cc13d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:39.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.257+0000 7f63d1cc4700 1 -- 172.21.15.78:0/3018144467 shutdown_connections 2023-10-11T09:50:39.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.257+0000 7f63d1cc4700 1 -- 172.21.15.78:0/3018144467 wait complete. 2023-10-11T09:50:39.287 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:50:39.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.449+0000 7f891509f700 1 Processor -- start 2023-10-11T09:50:39.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.449+0000 7f891509f700 1 -- start start 2023-10-11T09:50:39.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.453+0000 7f891509f700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f891015b8e0 0x7f891006d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:39.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.453+0000 7f891509f700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f891006d610 0x7f89100ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:39.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.453+0000 7f891509f700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f891015e010 con 0x7f891015a940 2023-10-11T09:50:39.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.453+0000 7f891509f700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f891015e280 con 0x7f891015b8e0 2023-10-11T09:50:39.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.453+0000 7f891509f700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f891015e560 con 0x7f891006d610 2023-10-11T09:50:39.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.453+0000 7f89077fe700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f891006d610 0x7f89100ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:39.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.453+0000 7f89077fe700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f891006d610 0x7f89100ca240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:56570/0 (socket says 172.21.15.78:56570) 2023-10-11T09:50:39.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.453+0000 7f89077fe700 1 -- 172.21.15.78:0/846911330 learned_addr learned my addr 172.21.15.78:0/846911330 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:39.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.453+0000 7f890ffff700 1 --2- 172.21.15.78:0/846911330 >> v2:172.21.15.133:3300/0 conn(0x7f891015b8e0 0x7f891006d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:39.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.453+0000 7f89077fe700 1 -- 172.21.15.78:0/846911330 >> v2:172.21.15.133:3300/0 conn(0x7f891015b8e0 msgr2=0x7f891006d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:39.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.453+0000 7f89077fe700 1 --2- 172.21.15.78:0/846911330 >> v2:172.21.15.133:3300/0 conn(0x7f891015b8e0 0x7f891006d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.453+0000 7f89077fe700 1 -- 172.21.15.78:0/846911330 >> v1:172.21.15.133:6789/0 conn(0x7f891015a940 legacy=0x7f89100c9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:39.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.453+0000 7f89077fe700 1 -- 172.21.15.78:0/846911330 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f891015e840 con 0x7f891006d610 2023-10-11T09:50:39.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.453+0000 7f89077fe700 1 --2- 172.21.15.78:0/846911330 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f891006d610 0x7f89100ca240 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f88f8009910 tx=0x7f88f8009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff88bfabd7a043f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:39.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.453+0000 7f890f7fe700 1 -- 172.21.15.78:0/846911330 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f800ba60 con 0x7f891006d610 2023-10-11T09:50:39.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.453+0000 7f890f7fe700 1 -- 172.21.15.78:0/846911330 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f800bbc0 con 0x7f891006d610 2023-10-11T09:50:39.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.453+0000 7f890f7fe700 1 -- 172.21.15.78:0/846911330 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f8009e60 con 0x7f891006d610 2023-10-11T09:50:39.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.457+0000 7f891509f700 1 -- 172.21.15.78:0/846911330 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f891006d610 msgr2=0x7f89100ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:39.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.457+0000 7f891509f700 1 --2- 172.21.15.78:0/846911330 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f891006d610 0x7f89100ca240 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f88f8009910 tx=0x7f88f8009c20 comp rx=0 tx=0).stop 2023-10-11T09:50:39.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.457+0000 7f891509f700 1 -- 172.21.15.78:0/846911330 shutdown_connections 2023-10-11T09:50:39.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.457+0000 7f891509f700 1 --2- 172.21.15.78:0/846911330 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f891006d610 0x7f89100ca240 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.457+0000 7f891509f700 1 --2- 172.21.15.78:0/846911330 >> v2:172.21.15.133:3300/0 conn(0x7f891015b8e0 0x7f891006d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.457+0000 7f891509f700 1 -- 172.21.15.78:0/846911330 >> 172.21.15.78:0/846911330 conn(0x7f89100cea60 msgr2=0x7f89100cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:39.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.457+0000 7f891509f700 1 -- 172.21.15.78:0/846911330 shutdown_connections 2023-10-11T09:50:39.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.457+0000 7f891509f700 1 -- 172.21.15.78:0/846911330 wait complete. 2023-10-11T09:50:39.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.461+0000 7f891509f700 1 Processor -- start 2023-10-11T09:50:39.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.461+0000 7f891509f700 1 -- start start 2023-10-11T09:50:39.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.461+0000 7f891509f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f891006d610 0x7f891013d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:39.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.461+0000 7f891509f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f891015a940 0x7f891013dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:39.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.461+0000 7f891509f700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f891015b8e0 0x7f891013e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:39.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.461+0000 7f891509f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89100ccf70 con 0x7f891006d610 2023-10-11T09:50:39.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.461+0000 7f891509f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89100cc640 con 0x7f891015a940 2023-10-11T09:50:39.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.461+0000 7f891509f700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89100cc920 con 0x7f891015b8e0 2023-10-11T09:50:39.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.461+0000 7f890ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f891006d610 0x7f891013d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:39.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.461+0000 7f890ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f891006d610 0x7f891013d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40992/0 (socket says 172.21.15.78:40992) 2023-10-11T09:50:39.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.461+0000 7f890ffff700 1 -- 172.21.15.78:0/3927587356 learned_addr learned my addr 172.21.15.78:0/3927587356 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:39.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.461+0000 7f89077fe700 1 --2- 172.21.15.78:0/3927587356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f891015a940 0x7f891013dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:39.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.461+0000 7f891489e700 1 --2- 172.21.15.78:0/3927587356 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f891015b8e0 0x7f891013e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:39.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.461+0000 7f890ffff700 1 -- 172.21.15.78:0/3927587356 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f891015b8e0 msgr2=0x7f891013e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:39.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.461+0000 7f890ffff700 1 --2- 172.21.15.78:0/3927587356 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f891015b8e0 0x7f891013e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.461+0000 7f890ffff700 1 -- 172.21.15.78:0/3927587356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f891015a940 msgr2=0x7f891013dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:39.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.461+0000 7f890ffff700 1 --2- 172.21.15.78:0/3927587356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f891015a940 0x7f891013dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.461+0000 7f890ffff700 1 -- 172.21.15.78:0/3927587356 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f8009570 con 0x7f891006d610 2023-10-11T09:50:39.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.465+0000 7f890ffff700 1 --2- 172.21.15.78:0/3927587356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f891006d610 0x7f891013d4c0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f890000e760 tx=0x7f890000ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:39.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.465+0000 7f890d7fa700 1 -- 172.21.15.78:0/3927587356 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890000c070 con 0x7f891006d610 2023-10-11T09:50:39.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.465+0000 7f890d7fa700 1 -- 172.21.15.78:0/3927587356 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8900013610 con 0x7f891006d610 2023-10-11T09:50:39.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.465+0000 7f890d7fa700 1 -- 172.21.15.78:0/3927587356 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8900013980 con 0x7f891006d610 2023-10-11T09:50:39.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.465+0000 7f891509f700 1 -- 172.21.15.78:0/3927587356 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89100ccc00 con 0x7f891006d610 2023-10-11T09:50:39.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.465+0000 7f891509f700 1 -- 172.21.15.78:0/3927587356 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89100ce4c0 con 0x7f891006d610 2023-10-11T09:50:39.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.465+0000 7f891509f700 1 -- 172.21.15.78:0/3927587356 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8910065d40 con 0x7f891006d610 2023-10-11T09:50:39.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.465+0000 7f890d7fa700 1 -- 172.21.15.78:0/3927587356 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f8900013ae0 con 0x7f891006d610 2023-10-11T09:50:39.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.469+0000 7f890d7fa700 1 --2- 172.21.15.78:0/3927587356 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f88f0065ed0 0x7f88f0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:39.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.469+0000 7f890d7fa700 1 -- 172.21.15.78:0/3927587356 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f89000838e0 con 0x7f891006d610 2023-10-11T09:50:39.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.469+0000 7f89077fe700 1 --2- 172.21.15.78:0/3927587356 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f88f0065ed0 0x7f88f0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:39.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.469+0000 7f89077fe700 1 --2- 172.21.15.78:0/3927587356 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f88f0065ed0 0x7f88f0068380 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f88f8000c00 tx=0x7f88f8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:39.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.473+0000 7f890d7fa700 1 -- 172.21.15.78:0/3927587356 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f8900050c00 con 0x7f891006d610 2023-10-11T09:50:39.652 INFO:tasks.ceph.mds.c.smithi078.stderr:2023-10-11T09:50:39.641+0000 7f608bf53700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-11T09:50:39.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.681+0000 7f891509f700 1 -- 172.21.15.78:0/3927587356 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8910070280 con 0x7f891006d610 2023-10-11T09:50:39.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.681+0000 7f890d7fa700 1 -- 172.21.15.78:0/3927587356 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4655 (secure 0 0 0) 0x7f8900054220 con 0x7f891006d610 2023-10-11T09:50:39.691 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:50:39.691 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":91,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7148,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/331520590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":331520590},{"type":"v1","addr":"172.21.15.78:6835","nonce":331520590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":7151,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/934539531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":934539531},{"type":"v1","addr":"172.21.15.133:6834","nonce":934539531}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":7154,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3894002128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":3894002128},{"type":"v1","addr":"172.21.15.133:6835","nonce":3894002128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:50:28.502984+0000","modified":"2023-10-11T09:50:37.659950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8550":{"gid":8550,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":4,"addr":"172.21.15.78:6837/2869034036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2869034036},{"type":"v1","addr":"172.21.15.78:6837","nonce":2869034036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15,16],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-11T09:50:39.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.685+0000 7f891509f700 1 -- 172.21.15.78:0/3927587356 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f88f0065ed0 msgr2=0x7f88f0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:39.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.685+0000 7f891509f700 1 --2- 172.21.15.78:0/3927587356 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f88f0065ed0 0x7f88f0068380 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f88f8000c00 tx=0x7f88f8008030 comp rx=0 tx=0).stop 2023-10-11T09:50:39.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.685+0000 7f891509f700 1 -- 172.21.15.78:0/3927587356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f891006d610 msgr2=0x7f891013d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:39.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.685+0000 7f891509f700 1 --2- 172.21.15.78:0/3927587356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f891006d610 0x7f891013d4c0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f890000e760 tx=0x7f890000ea70 comp rx=0 tx=0).stop 2023-10-11T09:50:39.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.685+0000 7f891509f700 1 -- 172.21.15.78:0/3927587356 shutdown_connections 2023-10-11T09:50:39.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.685+0000 7f891509f700 1 --2- 172.21.15.78:0/3927587356 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f88f0065ed0 0x7f88f0068380 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.685+0000 7f891509f700 1 --2- 172.21.15.78:0/3927587356 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f891015b8e0 0x7f891013e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.685+0000 7f891509f700 1 --2- 172.21.15.78:0/3927587356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f891015a940 0x7f891013dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.685+0000 7f891509f700 1 --2- 172.21.15.78:0/3927587356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f891006d610 0x7f891013d4c0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.685+0000 7f891509f700 1 -- 172.21.15.78:0/3927587356 >> 172.21.15.78:0/3927587356 conn(0x7f89100cea60 msgr2=0x7f89100714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:39.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.685+0000 7f891509f700 1 -- 172.21.15.78:0/3927587356 shutdown_connections 2023-10-11T09:50:39.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.685+0000 7f891509f700 1 -- 172.21.15.78:0/3927587356 wait complete. 2023-10-11T09:50:39.696 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 91 2023-10-11T09:50:39.714 INFO:tasks.cephfs.test_data_scan:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 5, "epoch": 91, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 5, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-11T09:50:28.502984+0000", "damaged": [], "data_pools": [ 15, 16 ], "enabled": true, "epoch": 91, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_8550": { "addr": "172.21.15.78:6837/2869034036", "addrs": { "addrvec": [ { "addr": "172.21.15.78:6836", "nonce": 2869034036, "type": "v2" }, { "addr": "172.21.15.78:6837", "nonce": 2869034036, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 8550, "incarnation": 88, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 4 } }, "last_failure": 0, "last_failure_osd_epoch": 112, "max_file_size": 1099511627776, "max_mds": 1, "metadata_pool": 14, "modified": "2023-10-11T09:50:37.659950+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 8550 } } } ], "standbys": [ { "addr": "172.21.15.78:6835/331520590", "addrs": { "addrvec": [ { "addr": "172.21.15.78:6834", "nonce": 331520590, "type": "v2" }, { "addr": "172.21.15.78:6835", "nonce": 331520590, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 87, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7148, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.133:6834/934539531", "addrs": { "addrvec": [ { "addr": "172.21.15.133:6832", "nonce": 934539531, "type": "v2" }, { "addr": "172.21.15.133:6834", "nonce": 934539531, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 88, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7151, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.133:6835/3894002128", "addrs": { "addrvec": [ { "addr": "172.21.15.133:6833", "nonce": 3894002128, "type": "v2" }, { "addr": "172.21.15.133:6835", "nonce": 3894002128, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 89, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7154, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-11T09:50:39.714 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-11T09:50:39.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.869+0000 7fa70c0c8700 1 Processor -- start 2023-10-11T09:50:39.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.873+0000 7fa70c0c8700 1 -- start start 2023-10-11T09:50:39.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.873+0000 7fa70c0c8700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa704154e00 0x7fa70415b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:39.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.873+0000 7fa70c0c8700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7040720f0 con 0x7fa7040cbbd0 2023-10-11T09:50:39.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.873+0000 7fa70c0c8700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa70415bb90 con 0x7fa704071040 2023-10-11T09:50:39.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.873+0000 7fa70c0c8700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7040714e0 con 0x7fa704154e00 2023-10-11T09:50:39.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.873+0000 7fa70a665700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fa7040cbbd0 0x7fa70406c010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:41594/0 (socket says 172.21.15.78:41594) 2023-10-11T09:50:39.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.873+0000 7fa70a665700 1 -- 172.21.15.78:0/949274914 learned_addr learned my addr 172.21.15.78:0/949274914 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:50:39.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.877+0000 7fa709e64700 1 --2- 172.21.15.78:0/949274914 >> v2:172.21.15.133:3300/0 conn(0x7fa704154e00 0x7fa70415b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:39.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.877+0000 7fa708e62700 1 -- 172.21.15.78:0/949274914 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2081985760 0 0) 0x7fa7040720f0 con 0x7fa7040cbbd0 2023-10-11T09:50:39.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.877+0000 7fa708e62700 1 -- 172.21.15.78:0/949274914 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6f00035d0 con 0x7fa7040cbbd0 2023-10-11T09:50:39.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.877+0000 7fa708e62700 1 -- 172.21.15.78:0/949274914 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 432533530 0 0) 0x7fa6f00035d0 con 0x7fa7040cbbd0 2023-10-11T09:50:39.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.877+0000 7fa708e62700 1 -- 172.21.15.78:0/949274914 >> v2:172.21.15.133:3300/0 conn(0x7fa704154e00 msgr2=0x7fa70415b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:39.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.877+0000 7fa708e62700 1 --2- 172.21.15.78:0/949274914 >> v2:172.21.15.133:3300/0 conn(0x7fa704154e00 0x7fa70415b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.877+0000 7fa708e62700 1 -- 172.21.15.78:0/949274914 >> v1:172.21.15.133:6789/0 conn(0x7fa704071040 legacy=0x7fa7040cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:39.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.877+0000 7fa708e62700 1 -- 172.21.15.78:0/949274914 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7040717c0 con 0x7fa7040cbbd0 2023-10-11T09:50:39.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.877+0000 7fa708e62700 1 -- 172.21.15.78:0/949274914 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7fa6fc0019f0 con 0x7fa7040cbbd0 2023-10-11T09:50:39.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.877+0000 7fa708e62700 1 -- 172.21.15.78:0/949274914 >> v1:172.21.15.78:6789/0 conn(0x7fa7040cbbd0 legacy=0x7fa70406c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:39.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.877+0000 7fa708e62700 1 --2- 172.21.15.78:0/949274914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6f0003a20 0x7fa6f0003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:39.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.877+0000 7fa708e62700 1 --2- 172.21.15.78:0/949274914 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa6f0004410 0x7fa6f00048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:39.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.877+0000 7fa708e62700 1 --2- 172.21.15.78:0/949274914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6f00050d0 0x7fa6f000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:39.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.877+0000 7fa708e62700 1 -- 172.21.15.78:0/949274914 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7040714e0 con 0x7fa6f0003a20 2023-10-11T09:50:39.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.877+0000 7fa708e62700 1 -- 172.21.15.78:0/949274914 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f000c020 con 0x7fa6f00050d0 2023-10-11T09:50:39.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.877+0000 7fa708e62700 1 -- 172.21.15.78:0/949274914 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f000c1e0 con 0x7fa6f0004410 2023-10-11T09:50:39.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.877+0000 7fa709e64700 1 --2- 172.21.15.78:0/949274914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6f0003a20 0x7fa6f0003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:39.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.877+0000 7fa709663700 1 --2- 172.21.15.78:0/949274914 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa6f0004410 0x7fa6f00048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:39.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.877+0000 7fa70a665700 1 --2- 172.21.15.78:0/949274914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6f00050d0 0x7fa6f000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:39.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.877+0000 7fa709e64700 1 -- 172.21.15.78:0/949274914 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa6f0004410 msgr2=0x7fa6f00048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:39.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.877+0000 7fa709e64700 1 --2- 172.21.15.78:0/949274914 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa6f0004410 0x7fa6f00048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.877+0000 7fa709e64700 1 -- 172.21.15.78:0/949274914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6f00050d0 msgr2=0x7fa6f000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:39.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.877+0000 7fa709e64700 1 --2- 172.21.15.78:0/949274914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6f00050d0 0x7fa6f000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.877+0000 7fa709e64700 1 -- 172.21.15.78:0/949274914 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f00035d0 con 0x7fa6f0003a20 2023-10-11T09:50:39.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.877+0000 7fa70a665700 1 --2- 172.21.15.78:0/949274914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6f00050d0 0x7fa6f000ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:50:39.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.877+0000 7fa70a665700 1 -- 172.21.15.78:0/949274914 reap_dead start 2023-10-11T09:50:39.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.877+0000 7fa709e64700 1 --2- 172.21.15.78:0/949274914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6f0003a20 0x7fa6f0003df0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fa700009b70 tx=0x7fa6f00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87642488e1a6bf60 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:39.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.877+0000 7fa708e62700 1 -- 172.21.15.78:0/949274914 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70000e6c0 con 0x7fa6f0003a20 2023-10-11T09:50:39.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.877+0000 7fa708e62700 1 -- 172.21.15.78:0/949274914 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa70000e820 con 0x7fa6f0003a20 2023-10-11T09:50:39.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.877+0000 7fa708e62700 1 -- 172.21.15.78:0/949274914 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70000eb90 con 0x7fa6f0003a20 2023-10-11T09:50:39.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.877+0000 7fa70c0c8700 1 -- 172.21.15.78:0/949274914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6f0003a20 msgr2=0x7fa6f0003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:39.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.877+0000 7fa70c0c8700 1 --2- 172.21.15.78:0/949274914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6f0003a20 0x7fa6f0003df0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fa700009b70 tx=0x7fa6f00058a0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.881+0000 7fa70c0c8700 1 -- 172.21.15.78:0/949274914 shutdown_connections 2023-10-11T09:50:39.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.881+0000 7fa70c0c8700 1 --2- 172.21.15.78:0/949274914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6f0003a20 0x7fa6f0003df0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.881+0000 7fa70c0c8700 1 -- 172.21.15.78:0/949274914 >> 172.21.15.78:0/949274914 conn(0x7fa70406aca0 msgr2=0x7fa7040b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:39.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.881+0000 7fa70c0c8700 1 -- 172.21.15.78:0/949274914 shutdown_connections 2023-10-11T09:50:39.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.881+0000 7fa70c0c8700 1 -- 172.21.15.78:0/949274914 wait complete. 2023-10-11T09:50:39.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.881+0000 7fa70c0c8700 1 Processor -- start 2023-10-11T09:50:39.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.885+0000 7fa70c0c8700 1 -- start start 2023-10-11T09:50:39.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.885+0000 7fa70c0c8700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa6f0003a20 0x7fa7041474d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:39.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.885+0000 7fa70c0c8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa70415be30 0x7fa70415d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:39.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.885+0000 7fa70c0c8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa70415ac40 0x7fa70406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:39.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.885+0000 7fa70c0c8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa704149dc0 con 0x7fa70415ac40 2023-10-11T09:50:39.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.885+0000 7fa70c0c8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa704147d90 con 0x7fa70415be30 2023-10-11T09:50:39.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.885+0000 7fa70c0c8700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa704147ab0 con 0x7fa6f0003a20 2023-10-11T09:50:39.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.885+0000 7fa70a665700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa70415ac40 0x7fa70406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:39.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.885+0000 7fa70a665700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa70415ac40 0x7fa70406c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41010/0 (socket says 172.21.15.78:41010) 2023-10-11T09:50:39.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.885+0000 7fa70a665700 1 -- 172.21.15.78:0/4217925692 learned_addr learned my addr 172.21.15.78:0/4217925692 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:39.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.885+0000 7fa709e64700 1 --2- 172.21.15.78:0/4217925692 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa6f0003a20 0x7fa7041474d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:39.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.885+0000 7fa709663700 1 --2- 172.21.15.78:0/4217925692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa70415be30 0x7fa70415d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:39.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.885+0000 7fa70a665700 1 -- 172.21.15.78:0/4217925692 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa6f0003a20 msgr2=0x7fa7041474d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:39.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.885+0000 7fa70a665700 1 --2- 172.21.15.78:0/4217925692 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa6f0003a20 0x7fa7041474d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.885+0000 7fa70a665700 1 -- 172.21.15.78:0/4217925692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa70415be30 msgr2=0x7fa70415d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:39.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.885+0000 7fa70a665700 1 --2- 172.21.15.78:0/4217925692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa70415be30 0x7fa70415d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:39.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.885+0000 7fa70a665700 1 -- 172.21.15.78:0/4217925692 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa704149ff0 con 0x7fa70415ac40 2023-10-11T09:50:39.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.885+0000 7fa70a665700 1 --2- 172.21.15.78:0/4217925692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa70415ac40 0x7fa70406c920 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fa7040c92a0 tx=0x7fa6fc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:39.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.885+0000 7fa6fa7fc700 1 -- 172.21.15.78:0/4217925692 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc007500 con 0x7fa70415ac40 2023-10-11T09:50:39.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.889+0000 7fa6fa7fc700 1 -- 172.21.15.78:0/4217925692 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6fc007660 con 0x7fa70415ac40 2023-10-11T09:50:39.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.889+0000 7fa70c0c8700 1 -- 172.21.15.78:0/4217925692 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa70414a2d0 con 0x7fa70415ac40 2023-10-11T09:50:39.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.889+0000 7fa6fa7fc700 1 -- 172.21.15.78:0/4217925692 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc007a10 con 0x7fa70415ac40 2023-10-11T09:50:39.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.889+0000 7fa70c0c8700 1 -- 172.21.15.78:0/4217925692 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa70415db30 con 0x7fa70415ac40 2023-10-11T09:50:39.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.889+0000 7fa70c0c8700 1 -- 172.21.15.78:0/4217925692 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa70404e530 con 0x7fa70415ac40 2023-10-11T09:50:39.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.889+0000 7fa6fa7fc700 1 -- 172.21.15.78:0/4217925692 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fa6fc0123d0 con 0x7fa70415ac40 2023-10-11T09:50:39.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.893+0000 7fa6fa7fc700 1 --2- 172.21.15.78:0/4217925692 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa6e8065ed0 0x7fa6e8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:39.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.893+0000 7fa6fa7fc700 1 -- 172.21.15.78:0/4217925692 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6724+0+0 (secure 0 0 0) 0x7fa6fc0835a0 con 0x7fa70415ac40 2023-10-11T09:50:39.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.893+0000 7fa709e64700 1 --2- 172.21.15.78:0/4217925692 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa6e8065ed0 0x7fa6e8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:39.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.893+0000 7fa709e64700 1 --2- 172.21.15.78:0/4217925692 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa6e8065ed0 0x7fa6e8068380 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fa7000090b0 tx=0x7fa70001e040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:39.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:39.897+0000 7fa6fa7fc700 1 -- 172.21.15.78:0/4217925692 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fa6fc0508b0 con 0x7fa70415ac40 2023-10-11T09:50:40.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:40.121+0000 7fa70c0c8700 1 -- 172.21.15.78:0/4217925692 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa7040c9080 con 0x7fa70415ac40 2023-10-11T09:50:40.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:40.121+0000 7fa6fa7fc700 1 -- 172.21.15.78:0/4217925692 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v91) v1 ==== 53+0+91 (secure 0 0 0) 0x7fa6fc019070 con 0x7fa70415ac40 2023-10-11T09:50:40.129 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_extra ] 2023-10-11T09:50:40.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:40.125+0000 7fa70c0c8700 1 -- 172.21.15.78:0/4217925692 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa6e8065ed0 msgr2=0x7fa6e8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:40.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:40.125+0000 7fa70c0c8700 1 --2- 172.21.15.78:0/4217925692 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa6e8065ed0 0x7fa6e8068380 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fa7000090b0 tx=0x7fa70001e040 comp rx=0 tx=0).stop 2023-10-11T09:50:40.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:40.125+0000 7fa70c0c8700 1 -- 172.21.15.78:0/4217925692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa70415ac40 msgr2=0x7fa70406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:40.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:40.125+0000 7fa70c0c8700 1 --2- 172.21.15.78:0/4217925692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa70415ac40 0x7fa70406c920 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fa7040c92a0 tx=0x7fa6fc00dc10 comp rx=0 tx=0).stop 2023-10-11T09:50:40.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:40.125+0000 7fa70c0c8700 1 -- 172.21.15.78:0/4217925692 shutdown_connections 2023-10-11T09:50:40.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:40.125+0000 7fa70c0c8700 1 --2- 172.21.15.78:0/4217925692 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa6e8065ed0 0x7fa6e8068380 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:40.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:40.125+0000 7fa70c0c8700 1 --2- 172.21.15.78:0/4217925692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa70415ac40 0x7fa70406c920 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:40.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:40.125+0000 7fa70c0c8700 1 --2- 172.21.15.78:0/4217925692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa70415be30 0x7fa70415d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:40.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:40.125+0000 7fa70c0c8700 1 --2- 172.21.15.78:0/4217925692 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa6f0003a20 0x7fa7041474d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:40.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:40.125+0000 7fa70c0c8700 1 -- 172.21.15.78:0/4217925692 >> 172.21.15.78:0/4217925692 conn(0x7fa70406aca0 msgr2=0x7fa7040af620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:40.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:40.125+0000 7fa70c0c8700 1 -- 172.21.15.78:0/4217925692 shutdown_connections 2023-10-11T09:50:40.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:40.125+0000 7fa70c0c8700 1 -- 172.21.15.78:0/4217925692 wait complete. 2023-10-11T09:50:40.156 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-11T09:50:40.156 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-11T09:50:40.157 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi078.front.sepia.ceph.com 2023-10-11T09:50:40.157 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-11T09:50:40.157 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-11T09:50:40.158 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-11T09:50:40.158 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-11T09:50:40.158 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-11T09:50:40.158 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-11T09:50:40.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:50:40.159 DEBUG:teuthology.orchestra.run.smithi078:> ip addr 2023-10-11T09:50:40.176 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-11T09:50:40.177 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-11T09:50:40.177 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2023-10-11T09:50:40.177 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:50:40.177 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2023-10-11T09:50:40.177 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:50:40.177 INFO:teuthology.orchestra.run.smithi078.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:50:40.178 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:22 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:50:40.178 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f0 2023-10-11T09:50:40.178 INFO:teuthology.orchestra.run.smithi078.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:50:40.178 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:23 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:50:40.178 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f1 2023-10-11T09:50:40.179 INFO:teuthology.orchestra.run.smithi078.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:50:40.179 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dc brd ff:ff:ff:ff:ff:ff 2023-10-11T09:50:40.179 INFO:teuthology.orchestra.run.smithi078.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-11T09:50:40.179 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dd brd ff:ff:ff:ff:ff:ff 2023-10-11T09:50:40.179 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.21.15.78/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-11T09:50:40.180 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft 41824sec preferred_lft 41824sec 2023-10-11T09:50:40.180 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::ec4:7aff:fe8f:d4dd/64 scope link 2023-10-11T09:50:40.180 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:50:40.180 INFO:teuthology.orchestra.run.smithi078.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-11T09:50:40.180 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 02:42:b6:e4:f8:0d brd ff:ff:ff:ff:ff:ff 2023-10-11T09:50:40.180 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-11T09:50:40.181 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:50:40.181 INFO:teuthology.orchestra.run.smithi078.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-11T09:50:40.181 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 7a:89:8a:a9:e1:23 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:50:40.181 INFO:teuthology.orchestra.run.smithi078.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-11T09:50:40.181 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:50:40.181 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::8ca:ebff:fea9:13f1/64 scope link 2023-10-11T09:50:40.182 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:50:40.182 INFO:teuthology.orchestra.run.smithi078.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-11T09:50:40.182 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 7a:89:8a:a9:e1:23 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-11T09:50:40.182 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::7889:8aff:fea9:e123/64 scope link 2023-10-11T09:50:40.182 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:50:40.183 INFO:teuthology.orchestra.run.smithi078.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-11T09:50:40.183 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether ca:ac:e8:27:51:0d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-11T09:50:40.183 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::c8ac:e8ff:fe27:510d/64 scope link 2023-10-11T09:50:40.183 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:50:40.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:50:40.184 DEBUG:teuthology.orchestra.run.smithi078:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-11T09:50:40.263 INFO:teuthology.orchestra.run.smithi078.stdout:1 2023-10-11T09:50:40.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:50:40.266 DEBUG:teuthology.orchestra.run.smithi078:> route 2023-10-11T09:50:40.327 INFO:teuthology.orchestra.run.smithi078.stdout:Kernel IP routing table 2023-10-11T09:50:40.327 INFO:teuthology.orchestra.run.smithi078.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-11T09:50:40.327 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-11T09:50:40.327 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-11T09:50:40.328 INFO:teuthology.orchestra.run.smithi078.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-11T09:50:40.328 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-10-11T09:50:40.328 INFO:teuthology.orchestra.run.smithi078.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-11T09:50:40.328 INFO:teuthology.orchestra.run.smithi078.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-11T09:50:40.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:50:40.329 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec bash -c ' 2023-10-11T09:50:40.329 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2023-10-11T09:50:40.330 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-11T09:50:40.330 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-11T09:50:40.331 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-11T09:50:40.331 DEBUG:teuthology.orchestra.run.smithi078:> ') 2023-10-11T09:50:40.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:50:40.458 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2023-10-11T09:50:40.473 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-11T09:50:40.473 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-11T09:50:40.474 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-11T09:50:40.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:50:40.475 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set brx.0 up 2023-10-11T09:50:40.554 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-11T09:50:40.555 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T09:50:40.555 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-11T09:50:40.613 INFO:teuthology.orchestra.run.smithi078.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-11T09:50:40.614 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T09:50:40.614 DEBUG:teuthology.orchestra.run.smithi078:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-11T09:50:40.676 DEBUG:teuthology.orchestra.run.smithi078:> sudo modprobe fuse 2023-10-11T09:50:40.756 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T09:50:40.757 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T09:50:40.833 INFO:teuthology.orchestra.run.smithi078.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T09:50:40.835 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T09:50:40.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:50:40.836 DEBUG:teuthology.orchestra.run.smithi078:> ls /sys/fs/fuse/connections 2023-10-11T09:50:40.903 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-11T09:50:40.903 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-11T09:50:40.954 DEBUG:teuthology.orchestra.run.smithi078:> sudo modprobe fuse 2023-10-11T09:50:40.987 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T09:50:40.988 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T09:50:41.063 INFO:teuthology.orchestra.run.smithi078.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T09:50:41.065 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T09:50:41.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:50:41.066 DEBUG:teuthology.orchestra.run.smithi078:> ls /sys/fs/fuse/connections 2023-10-11T09:50:41.092 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:2023-10-11T09:50:41.081+0000 7fa2fc9d3640 -1 init, newargv = 0x561f843179a0 newargc=15 2023-10-11T09:50:41.092 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:ceph-fuse[57751]: starting ceph client 2023-10-11T09:50:41.103 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:ceph-fuse[57751]: starting fuse 2023-10-11T09:50:41.106 INFO:teuthology.orchestra.run.smithi078.stdout:49 2023-10-11T09:50:41.106 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2023-10-11T09:50:41.107 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:50:41.107 DEBUG:teuthology.orchestra.run.smithi078:> import glob 2023-10-11T09:50:41.107 DEBUG:teuthology.orchestra.run.smithi078:> import re 2023-10-11T09:50:41.108 DEBUG:teuthology.orchestra.run.smithi078:> import os 2023-10-11T09:50:41.108 DEBUG:teuthology.orchestra.run.smithi078:> import subprocess 2023-10-11T09:50:41.109 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:50:41.109 DEBUG:teuthology.orchestra.run.smithi078:> def _find_admin_socket(client_name): 2023-10-11T09:50:41.109 DEBUG:teuthology.orchestra.run.smithi078:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-11T09:50:41.110 DEBUG:teuthology.orchestra.run.smithi078:> files = glob.glob(asok_path) 2023-10-11T09:50:41.110 DEBUG:teuthology.orchestra.run.smithi078:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-11T09:50:41.110 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:50:41.111 DEBUG:teuthology.orchestra.run.smithi078:> # Given a non-glob path, it better be there 2023-10-11T09:50:41.111 DEBUG:teuthology.orchestra.run.smithi078:> if "*" not in asok_path: 2023-10-11T09:50:41.112 DEBUG:teuthology.orchestra.run.smithi078:> assert(len(files) == 1) 2023-10-11T09:50:41.112 DEBUG:teuthology.orchestra.run.smithi078:> return files[0] 2023-10-11T09:50:41.112 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:50:41.113 DEBUG:teuthology.orchestra.run.smithi078:> for f in files: 2023-10-11T09:50:41.113 DEBUG:teuthology.orchestra.run.smithi078:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-11T09:50:41.113 DEBUG:teuthology.orchestra.run.smithi078:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-11T09:50:41.114 DEBUG:teuthology.orchestra.run.smithi078:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-11T09:50:41.114 DEBUG:teuthology.orchestra.run.smithi078:> contents = proc_f.read() 2023-10-11T09:50:41.115 DEBUG:teuthology.orchestra.run.smithi078:> if mountpoint in contents: 2023-10-11T09:50:41.115 DEBUG:teuthology.orchestra.run.smithi078:> return f 2023-10-11T09:50:41.115 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-11T09:50:41.116 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:50:41.116 DEBUG:teuthology.orchestra.run.smithi078:> print(_find_admin_socket("client.0")) 2023-10-11T09:50:41.117 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:50:41.257 INFO:teuthology.orchestra.run.smithi078.stdout:/var/run/ceph/ceph-client.0.57751.asok 2023-10-11T09:50:41.447 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.57751.asok 2023-10-11T09:50:41.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:50:41.448 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.57751.asok status 2023-10-11T09:50:41.642 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:50:41.642 INFO:teuthology.orchestra.run.smithi078.stdout: "metadata": { 2023-10-11T09:50:41.643 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_sha1": "9a774796f0ecdd11a4b771de396e627aa1a187af", 2023-10-11T09:50:41.643 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_version": "ceph version 18.2.0-662-g9a774796 (9a774796f0ecdd11a4b771de396e627aa1a187af) reef (stable)", 2023-10-11T09:50:41.643 INFO:teuthology.orchestra.run.smithi078.stdout: "entity_id": "0", 2023-10-11T09:50:41.643 INFO:teuthology.orchestra.run.smithi078.stdout: "hostname": "smithi078", 2023-10-11T09:50:41.644 INFO:teuthology.orchestra.run.smithi078.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-11T09:50:41.644 INFO:teuthology.orchestra.run.smithi078.stdout: "pid": "57751", 2023-10-11T09:50:41.644 INFO:teuthology.orchestra.run.smithi078.stdout: "root": "/" 2023-10-11T09:50:41.644 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:50:41.645 INFO:teuthology.orchestra.run.smithi078.stdout: "dentry_count": 0, 2023-10-11T09:50:41.645 INFO:teuthology.orchestra.run.smithi078.stdout: "dentry_pinned_count": 0, 2023-10-11T09:50:41.645 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 8607, 2023-10-11T09:50:41.646 INFO:teuthology.orchestra.run.smithi078.stdout: "inst": { 2023-10-11T09:50:41.646 INFO:teuthology.orchestra.run.smithi078.stdout: "name": { 2023-10-11T09:50:41.646 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "client", 2023-10-11T09:50:41.647 INFO:teuthology.orchestra.run.smithi078.stdout: "num": 8607 2023-10-11T09:50:41.647 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:50:41.647 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": { 2023-10-11T09:50:41.648 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2023-10-11T09:50:41.648 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "192.168.0.1:0", 2023-10-11T09:50:41.648 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 561660033 2023-10-11T09:50:41.648 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:50:41.649 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:50:41.649 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": { 2023-10-11T09:50:41.649 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2023-10-11T09:50:41.650 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "192.168.0.1:0", 2023-10-11T09:50:41.650 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 561660033 2023-10-11T09:50:41.650 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:50:41.651 INFO:teuthology.orchestra.run.smithi078.stdout: "inst_str": "client.8607 192.168.0.1:0/561660033", 2023-10-11T09:50:41.651 INFO:teuthology.orchestra.run.smithi078.stdout: "addr_str": "192.168.0.1:0/561660033", 2023-10-11T09:50:41.651 INFO:teuthology.orchestra.run.smithi078.stdout: "inode_count": 1, 2023-10-11T09:50:41.651 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_epoch": 91, 2023-10-11T09:50:41.652 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_epoch": 114, 2023-10-11T09:50:41.652 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_epoch_barrier": 0, 2023-10-11T09:50:41.652 INFO:teuthology.orchestra.run.smithi078.stdout: "blocklisted": false, 2023-10-11T09:50:41.653 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs" 2023-10-11T09:50:41.653 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:50:41.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:50:41.656 DEBUG:teuthology.orchestra.run.smithi078:> stat --file-system '--printf=%T 2023-10-11T09:50:41.657 DEBUG:teuthology.orchestra.run.smithi078:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-11T09:50:41.676 INFO:teuthology.orchestra.run.smithi078.stdout:fuseblk 2023-10-11T09:50:41.677 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-11T09:50:41.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:50:41.678 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-11T09:50:41.758 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 scrub start / repair,recursive 2023-10-11T09:50:41.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.969+0000 7f573191f700 1 Processor -- start 2023-10-11T09:50:41.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.969+0000 7f573191f700 1 -- start start 2023-10-11T09:50:41.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.973+0000 7f573191f700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f572c06c630 0x7f572c0c2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:41.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.973+0000 7f573191f700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f572c1649c0 0x7f572c164dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:41.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.973+0000 7f573191f700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f572c0cd3e0 con 0x7f572c160d90 2023-10-11T09:50:41.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.973+0000 7f573191f700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f572c0710f0 con 0x7f572c06c630 2023-10-11T09:50:41.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.973+0000 7f573191f700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f572c0cb8f0 con 0x7f572c1649c0 2023-10-11T09:50:41.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.973+0000 7f572a7fc700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f572c06c630 0x7f572c0c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:41.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.973+0000 7f572a7fc700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f572c06c630 0x7f572c0c2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41020/0 (socket says 172.21.15.78:41020) 2023-10-11T09:50:41.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.973+0000 7f572a7fc700 1 -- 172.21.15.78:0/459796316 learned_addr learned my addr 172.21.15.78:0/459796316 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:41.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.973+0000 7f572a7fc700 1 -- 172.21.15.78:0/459796316 >> v2:172.21.15.133:3300/0 conn(0x7f572c1649c0 msgr2=0x7f572c164dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:41.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.973+0000 7f572b7fe700 1 --2- 172.21.15.78:0/459796316 >> v2:172.21.15.133:3300/0 conn(0x7f572c1649c0 0x7f572c164dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:41.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.973+0000 7f572a7fc700 1 --2- 172.21.15.78:0/459796316 >> v2:172.21.15.133:3300/0 conn(0x7f572c1649c0 0x7f572c164dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:41.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.973+0000 7f572a7fc700 1 -- 172.21.15.78:0/459796316 >> v1:172.21.15.78:6789/0 conn(0x7f572c160d90 legacy=0x7f572c0c1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:41.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.973+0000 7f572a7fc700 1 -- 172.21.15.78:0/459796316 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f572c072300 con 0x7f572c06c630 2023-10-11T09:50:41.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.973+0000 7f572b7fe700 1 --2- 172.21.15.78:0/459796316 >> v2:172.21.15.133:3300/0 conn(0x7f572c1649c0 0x7f572c164dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:50:41.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.973+0000 7f572a7fc700 1 --2- 172.21.15.78:0/459796316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f572c06c630 0x7f572c0c2180 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f572c06e970 tx=0x7f5720009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e64e8996431f45bb server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:41.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.977+0000 7f5729ffb700 1 -- 172.21.15.78:0/459796316 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5720005e50 con 0x7f572c06c630 2023-10-11T09:50:41.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.977+0000 7f5729ffb700 1 -- 172.21.15.78:0/459796316 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5720006030 con 0x7f572c06c630 2023-10-11T09:50:41.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.977+0000 7f5729ffb700 1 -- 172.21.15.78:0/459796316 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57200063c0 con 0x7f572c06c630 2023-10-11T09:50:41.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.977+0000 7f573191f700 1 -- 172.21.15.78:0/459796316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f572c06c630 msgr2=0x7f572c0c2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:41.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.977+0000 7f573191f700 1 --2- 172.21.15.78:0/459796316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f572c06c630 0x7f572c0c2180 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f572c06e970 tx=0x7f5720009bf0 comp rx=0 tx=0).stop 2023-10-11T09:50:41.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.977+0000 7f573191f700 1 -- 172.21.15.78:0/459796316 shutdown_connections 2023-10-11T09:50:41.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.977+0000 7f573191f700 1 --2- 172.21.15.78:0/459796316 >> v2:172.21.15.133:3300/0 conn(0x7f572c1649c0 0x7f572c164dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:41.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.977+0000 7f573191f700 1 --2- 172.21.15.78:0/459796316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f572c06c630 0x7f572c0c2180 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:41.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.977+0000 7f573191f700 1 -- 172.21.15.78:0/459796316 >> 172.21.15.78:0/459796316 conn(0x7f572c06a060 msgr2=0x7f572c06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:41.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.977+0000 7f573191f700 1 -- 172.21.15.78:0/459796316 shutdown_connections 2023-10-11T09:50:41.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.977+0000 7f573191f700 1 -- 172.21.15.78:0/459796316 wait complete. 2023-10-11T09:50:41.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.981+0000 7f573191f700 1 Processor -- start 2023-10-11T09:50:41.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.981+0000 7f573191f700 1 -- start start 2023-10-11T09:50:41.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.981+0000 7f573191f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f572c06c630 0x7f572c161990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:41.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.981+0000 7f573191f700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f572c160d90 0x7f572c1f5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:41.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.981+0000 7f573191f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f572c1649c0 0x7f572c0c7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:41.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.981+0000 7f573191f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572c0c6610 con 0x7f572c06c630 2023-10-11T09:50:41.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.981+0000 7f573191f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572c0b36c0 con 0x7f572c1649c0 2023-10-11T09:50:41.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.981+0000 7f573191f700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f572c1fa900 con 0x7f572c160d90 2023-10-11T09:50:41.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.981+0000 7f572affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f572c06c630 0x7f572c161990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:41.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.981+0000 7f572affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f572c06c630 0x7f572c161990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41030/0 (socket says 172.21.15.78:41030) 2023-10-11T09:50:41.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.981+0000 7f572affd700 1 -- 172.21.15.78:0/824153961 learned_addr learned my addr 172.21.15.78:0/824153961 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:41.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.981+0000 7f572a7fc700 1 --2- 172.21.15.78:0/824153961 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f572c160d90 0x7f572c1f5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:41.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.981+0000 7f572b7fe700 1 --2- 172.21.15.78:0/824153961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f572c1649c0 0x7f572c0c7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:41.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.985+0000 7f572affd700 1 -- 172.21.15.78:0/824153961 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f572c160d90 msgr2=0x7f572c1f5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:41.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.985+0000 7f572affd700 1 --2- 172.21.15.78:0/824153961 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f572c160d90 0x7f572c1f5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:41.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.985+0000 7f572affd700 1 -- 172.21.15.78:0/824153961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f572c1649c0 msgr2=0x7f572c0c7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:41.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.985+0000 7f572affd700 1 --2- 172.21.15.78:0/824153961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f572c1649c0 0x7f572c0c7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:41.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.985+0000 7f572affd700 1 -- 172.21.15.78:0/824153961 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5720009520 con 0x7f572c06c630 2023-10-11T09:50:41.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.985+0000 7f572b7fe700 1 --2- 172.21.15.78:0/824153961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f572c1649c0 0x7f572c0c7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:50:41.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.985+0000 7f572affd700 1 --2- 172.21.15.78:0/824153961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f572c06c630 0x7f572c161990 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f571c00c530 tx=0x7f571c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:41.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.985+0000 7f5713fff700 1 -- 172.21.15.78:0/824153961 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571c0052d0 con 0x7f572c06c630 2023-10-11T09:50:41.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.985+0000 7f5713fff700 1 -- 172.21.15.78:0/824153961 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f571c005430 con 0x7f572c06c630 2023-10-11T09:50:41.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.985+0000 7f573191f700 1 -- 172.21.15.78:0/824153961 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f572c1fabe0 con 0x7f572c06c630 2023-10-11T09:50:41.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.985+0000 7f5713fff700 1 -- 172.21.15.78:0/824153961 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571c0055a0 con 0x7f572c06c630 2023-10-11T09:50:41.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.985+0000 7f573191f700 1 -- 172.21.15.78:0/824153961 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f572c0c62a0 con 0x7f572c06c630 2023-10-11T09:50:41.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.989+0000 7f5713fff700 1 -- 172.21.15.78:0/824153961 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f571c0038e0 con 0x7f572c06c630 2023-10-11T09:50:41.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.989+0000 7f5734bb0740 1 Processor -- start 2023-10-11T09:50:41.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.989+0000 7f5734bb0740 1 -- start start 2023-10-11T09:50:41.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.989+0000 7f5713fff700 1 --2- 172.21.15.78:0/824153961 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5718065ed0 0x7f5718068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:41.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.989+0000 7f5713fff700 1 -- 172.21.15.78:0/824153961 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f571c083040 con 0x7f572c06c630 2023-10-11T09:50:42.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.989+0000 7f572b7fe700 1 --2- 172.21.15.78:0/824153961 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5718065ed0 0x7f5718068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:42.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.993+0000 7f572b7fe700 1 --2- 172.21.15.78:0/824153961 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5718065ed0 0x7f5718068380 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f5724007f40 tx=0x7f5724007490 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:42.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.993+0000 7f5734bb0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x14e7d80 0x1782e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:42.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.993+0000 7f5734bb0740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x14e79b0 0x179cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:42.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.993+0000 7f5734bb0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1532a30 0x179d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:42.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.993+0000 7f5734bb0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x167a2f0 con 0x1532a30 2023-10-11T09:50:42.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.993+0000 7f5734bb0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x16c7610 con 0x14e7d80 2023-10-11T09:50:42.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.993+0000 7f5734bb0740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x16c4da0 con 0x14e79b0 2023-10-11T09:50:42.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.993+0000 7f572b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1532a30 0x179d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:42.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.993+0000 7f572affd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x14e7d80 0x1782e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:42.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.993+0000 7f572b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1532a30 0x179d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41038/0 (socket says 172.21.15.78:41038) 2023-10-11T09:50:42.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.993+0000 7f572b7fe700 1 -- 172.21.15.78:0/1261106452 learned_addr learned my addr 172.21.15.78:0/1261106452 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:42.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.993+0000 7f572a7fc700 1 --2- 172.21.15.78:0/1261106452 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x14e79b0 0x179cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:42.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.993+0000 7f572b7fe700 1 -- 172.21.15.78:0/1261106452 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x14e79b0 msgr2=0x179cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:42.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.993+0000 7f572b7fe700 1 --2- 172.21.15.78:0/1261106452 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x14e79b0 0x179cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.993+0000 7f572b7fe700 1 -- 172.21.15.78:0/1261106452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x14e7d80 msgr2=0x1782e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:42.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.993+0000 7f572b7fe700 1 --2- 172.21.15.78:0/1261106452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x14e7d80 0x1782e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.993+0000 7f572b7fe700 1 -- 172.21.15.78:0/1261106452 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x16e7b50 con 0x1532a30 2023-10-11T09:50:42.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.993+0000 7f572affd700 1 --2- 172.21.15.78:0/1261106452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x14e7d80 0x1782e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:50:42.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.993+0000 7f572b7fe700 1 --2- 172.21.15.78:0/1261106452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1532a30 0x179d400 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f572400de00 tx=0x7f5724010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de22e62191dd602d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:42.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.993+0000 7f57117fa700 1 -- 172.21.15.78:0/1261106452 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572400de30 con 0x1532a30 2023-10-11T09:50:42.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.993+0000 7f57117fa700 1 -- 172.21.15.78:0/1261106452 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f572401a420 con 0x1532a30 2023-10-11T09:50:42.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.993+0000 7f57117fa700 1 -- 172.21.15.78:0/1261106452 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572401a790 con 0x1532a30 2023-10-11T09:50:42.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.993+0000 7f5734bb0740 1 -- 172.21.15.78:0/1261106452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1532a30 msgr2=0x179d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:42.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.993+0000 7f5734bb0740 1 --2- 172.21.15.78:0/1261106452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1532a30 0x179d400 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f572400de00 tx=0x7f5724010000 comp rx=0 tx=0).stop 2023-10-11T09:50:42.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.997+0000 7f5734bb0740 1 -- 172.21.15.78:0/1261106452 shutdown_connections 2023-10-11T09:50:42.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.997+0000 7f5734bb0740 1 --2- 172.21.15.78:0/1261106452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x1532a30 0x179d400 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.997+0000 7f5734bb0740 1 --2- 172.21.15.78:0/1261106452 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x14e79b0 0x179cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.997+0000 7f5734bb0740 1 --2- 172.21.15.78:0/1261106452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x14e7d80 0x1782e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.997+0000 7f5734bb0740 1 -- 172.21.15.78:0/1261106452 >> 172.21.15.78:0/1261106452 conn(0x14e8150 msgr2=0x1757860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:42.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.997+0000 7f5734bb0740 1 -- 172.21.15.78:0/1261106452 shutdown_connections 2023-10-11T09:50:42.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.997+0000 7f5734bb0740 1 -- 172.21.15.78:0/1261106452 wait complete. 2023-10-11T09:50:42.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.997+0000 7f5734bb0740 1 -- start start 2023-10-11T09:50:42.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.997+0000 7f5734bb0740 1 Processor -- start 2023-10-11T09:50:42.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.997+0000 7f5734bb0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x14e79b0 0x179f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:42.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.997+0000 7f5734bb0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x14e7d80 0x17a67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:42.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.997+0000 7f5734bb0740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1532a30 0x177bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:42.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.997+0000 7f5734bb0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x16c0690 con 0x14e79b0 2023-10-11T09:50:42.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.997+0000 7f5734bb0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x1683c50 con 0x14e7d80 2023-10-11T09:50:42.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.997+0000 7f5734bb0740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x16838f0 con 0x1532a30 2023-10-11T09:50:42.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.997+0000 7f5734bb0740 10 client.0 _ll_register_callbacks cb 0x1713a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:50:42.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.997+0000 7f572affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x14e79b0 0x179f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:42.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.997+0000 7f572affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x14e79b0 0x179f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41046/0 (socket says 172.21.15.78:41046) 2023-10-11T09:50:42.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.997+0000 7f572affd700 1 -- 172.21.15.78:0/1545752840 learned_addr learned my addr 172.21.15.78:0/1545752840 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:42.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.997+0000 7f572b7fe700 1 --2- 172.21.15.78:0/1545752840 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1532a30 0x177bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:42.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.997+0000 7f572a7fc700 1 --2- 172.21.15.78:0/1545752840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x14e7d80 0x17a67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:42.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.997+0000 7f572affd700 1 -- 172.21.15.78:0/1545752840 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1532a30 msgr2=0x177bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:42.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.997+0000 7f572affd700 1 --2- 172.21.15.78:0/1545752840 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1532a30 0x177bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.997+0000 7f572affd700 1 -- 172.21.15.78:0/1545752840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x14e7d80 msgr2=0x17a67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:42.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.997+0000 7f572affd700 1 --2- 172.21.15.78:0/1545752840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x14e7d80 0x17a67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.997+0000 7f572affd700 1 -- 172.21.15.78:0/1545752840 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f572c0c62a0 con 0x14e79b0 2023-10-11T09:50:42.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.997+0000 7f572b7fe700 1 --2- 172.21.15.78:0/1545752840 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1532a30 0x177bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:50:42.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:41.997+0000 7f572a7fc700 1 --2- 172.21.15.78:0/1545752840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x14e7d80 0x17a67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:50:42.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.001+0000 7f572affd700 1 --2- 172.21.15.78:0/1545752840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x14e79b0 0x179f520 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f571c00ccf0 tx=0x7f571c0540c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a632fa9a2118aed server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:42.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.001+0000 7f570a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:50:42.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.001+0000 7f570a7fc700 1 -- 172.21.15.78:0/1545752840 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571c053700 con 0x14e79b0 2023-10-11T09:50:42.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.001+0000 7f570a7fc700 1 -- 172.21.15.78:0/1545752840 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f571c05bff0 con 0x14e79b0 2023-10-11T09:50:42.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.001+0000 7f5734bb0740 1 -- 172.21.15.78:0/1545752840 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x16b9710 con 0x14e79b0 2023-10-11T09:50:42.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.001+0000 7f570a7fc700 1 -- 172.21.15.78:0/1545752840 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f571c05bb60 con 0x14e79b0 2023-10-11T09:50:42.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.001+0000 7f5734bb0740 10 client.8622 fetch_fsmap learned FSMap version 91 2023-10-11T09:50:42.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.001+0000 7f5734bb0740 1 -- 172.21.15.78:0/1545752840 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x7f5724013070 con 0x14e79b0 2023-10-11T09:50:42.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.001+0000 7f570a7fc700 1 -- 172.21.15.78:0/1545752840 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x16b9710 con 0x14e79b0 2023-10-11T09:50:42.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.001+0000 7f570a7fc700 1 -- 172.21.15.78:0/1545752840 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 91) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f571c07d020 con 0x14e79b0 2023-10-11T09:50:42.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.001+0000 7f5734bb0740 10 client.8622 fetch_fsmap finished waiting for FSMap version 91 2023-10-11T09:50:42.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.001+0000 7f5734bb0740 10 client.8622 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2023-10-11T09:50:42.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.001+0000 7f5734bb0740 20 client.8622 populate_metadata read hostname 'smithi078' 2023-10-11T09:50:42.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.001+0000 7f5734bb0740 1 --2- 172.21.15.78:0/1545752840 >> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] conn(0x175b630 0x17598f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:42.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.001+0000 7f5734bb0740 4 client.8622 mds_command: new command op to 8550 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-11T09:50:42.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.001+0000 7f5734bb0740 1 -- 172.21.15.78:0/1545752840 --> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x16b9430 con 0x175b630 2023-10-11T09:50:42.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.001+0000 7f572a7fc700 1 --2- 172.21.15.78:0/1545752840 >> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] conn(0x175b630 0x17598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:42.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.001+0000 7f572a7fc700 1 --2- 172.21.15.78:0/1545752840 >> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] conn(0x175b630 0x17598f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36355eb9fae74aab server_cookie=e31a9f4cc539ecb3 in_seq=0 out_seq=0 2023-10-11T09:50:42.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.001+0000 7f570a7fc700 10 client.8622 ms_handle_connect on v2:172.21.15.78:6836/2869034036 2023-10-11T09:50:42.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.001+0000 7f570a7fc700 1 -- 172.21.15.78:0/1545752840 <== mds.0 v2:172.21.15.78:6836/2869034036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x16b9430 con 0x175b630 2023-10-11T09:50:42.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.001+0000 7f570a7fc700 10 client.8622 handle_command_reply: tid=0 2023-10-11T09:50:42.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.001+0000 7f5734bb0740 1 client.8622 shutdown 2023-10-11T09:50:42.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.001+0000 7f5734bb0740 1 -- 172.21.15.78:0/1545752840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x14e79b0 msgr2=0x179f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:42.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.001+0000 7f5734bb0740 1 --2- 172.21.15.78:0/1545752840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x14e79b0 0x179f520 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f571c00ccf0 tx=0x7f571c0540c0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.001+0000 7f5734bb0740 1 -- 172.21.15.78:0/1545752840 shutdown_connections 2023-10-11T09:50:42.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.001+0000 7f5734bb0740 1 --2- 172.21.15.78:0/1545752840 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x1532a30 0x177bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.001+0000 7f5734bb0740 1 --2- 172.21.15.78:0/1545752840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x14e7d80 0x17a67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.001+0000 7f5734bb0740 1 --2- 172.21.15.78:0/1545752840 >> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] conn(0x175b630 0x17598f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.001+0000 7f5734bb0740 1 --2- 172.21.15.78:0/1545752840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x14e79b0 0x179f520 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.001+0000 7f5734bb0740 1 -- 172.21.15.78:0/1545752840 >> 172.21.15.78:0/1545752840 conn(0x14e8150 msgr2=0x177fc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:42.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.001+0000 7f570a7fc700 0 client.8622 ms_handle_reset on v2:172.21.15.78:6836/2869034036 2023-10-11T09:50:42.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.005+0000 7f5734bb0740 1 -- 172.21.15.78:0/1545752840 shutdown_connections 2023-10-11T09:50:42.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.005+0000 7f5734bb0740 1 -- 172.21.15.78:0/1545752840 wait complete. 2023-10-11T09:50:42.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.005+0000 7f5734bb0740 20 client.8622 trim_cache size 0 max 16384 2023-10-11T09:50:42.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.025+0000 7f5734bb0740 1 Processor -- start 2023-10-11T09:50:42.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.025+0000 7f5734bb0740 1 -- start start 2023-10-11T09:50:42.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.025+0000 7f5734bb0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x14e8150 0x17598f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:42.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.025+0000 7f5734bb0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x14e79b0 0x1759e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:42.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.025+0000 7f5734bb0740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x14e7d80 0x175a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:42.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.025+0000 7f5734bb0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x156cdc0 con 0x14e79b0 2023-10-11T09:50:42.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.025+0000 7f5734bb0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x1680c90 con 0x14e8150 2023-10-11T09:50:42.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.025+0000 7f5734bb0740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x16bda90 con 0x14e7d80 2023-10-11T09:50:42.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.025+0000 7f572a7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x14e79b0 0x1759e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:42.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.025+0000 7f572a7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x14e79b0 0x1759e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41056/0 (socket says 172.21.15.78:41056) 2023-10-11T09:50:42.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.025+0000 7f572a7fc700 1 -- 172.21.15.78:0/790290773 learned_addr learned my addr 172.21.15.78:0/790290773 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:42.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.025+0000 7f572affd700 1 --2- 172.21.15.78:0/790290773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x14e8150 0x17598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:42.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.025+0000 7f572b7fe700 1 --2- 172.21.15.78:0/790290773 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x14e7d80 0x175a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:42.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.025+0000 7f572a7fc700 1 -- 172.21.15.78:0/790290773 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x14e7d80 msgr2=0x175a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:42.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.025+0000 7f572a7fc700 1 --2- 172.21.15.78:0/790290773 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x14e7d80 0x175a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.025+0000 7f572a7fc700 1 -- 172.21.15.78:0/790290773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x14e8150 msgr2=0x17598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:42.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.025+0000 7f572a7fc700 1 --2- 172.21.15.78:0/790290773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x14e8150 0x17598f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.025+0000 7f572a7fc700 1 -- 172.21.15.78:0/790290773 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f572c0c87a0 con 0x14e79b0 2023-10-11T09:50:42.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.025+0000 7f572affd700 1 --2- 172.21.15.78:0/790290773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x14e8150 0x17598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:50:42.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.025+0000 7f572a7fc700 1 --2- 172.21.15.78:0/790290773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x14e79b0 0x1759e30 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f572000b0e0 tx=0x7f5720010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=449dd3f69917b5bc server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:42.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.029+0000 7f5711ffb700 1 -- 172.21.15.78:0/790290773 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5720029430 con 0x14e79b0 2023-10-11T09:50:42.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.029+0000 7f5711ffb700 1 -- 172.21.15.78:0/790290773 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5720023040 con 0x14e79b0 2023-10-11T09:50:42.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.029+0000 7f5711ffb700 1 -- 172.21.15.78:0/790290773 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5720029a20 con 0x14e79b0 2023-10-11T09:50:42.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.029+0000 7f5734bb0740 1 -- 172.21.15.78:0/790290773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x14e79b0 msgr2=0x1759e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:42.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.029+0000 7f5734bb0740 1 --2- 172.21.15.78:0/790290773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x14e79b0 0x1759e30 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f572000b0e0 tx=0x7f5720010000 comp rx=0 tx=0).stop 2023-10-11T09:50:42.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.029+0000 7f5734bb0740 1 -- 172.21.15.78:0/790290773 shutdown_connections 2023-10-11T09:50:42.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.029+0000 7f5734bb0740 1 --2- 172.21.15.78:0/790290773 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x14e7d80 0x175a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.029+0000 7f5734bb0740 1 --2- 172.21.15.78:0/790290773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x14e79b0 0x1759e30 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.029+0000 7f5734bb0740 1 --2- 172.21.15.78:0/790290773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x14e8150 0x17598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.029+0000 7f5734bb0740 1 -- 172.21.15.78:0/790290773 >> 172.21.15.78:0/790290773 conn(0x17832b0 msgr2=0x17b0590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:42.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.029+0000 7f5734bb0740 1 -- 172.21.15.78:0/790290773 shutdown_connections 2023-10-11T09:50:42.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.029+0000 7f5734bb0740 1 -- 172.21.15.78:0/790290773 wait complete. 2023-10-11T09:50:42.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.029+0000 7f5734bb0740 1 -- start start 2023-10-11T09:50:42.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.033+0000 7f5734bb0740 1 Processor -- start 2023-10-11T09:50:42.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.033+0000 7f5734bb0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x14e79b0 0x17fc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:42.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.033+0000 7f5734bb0740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x14e7d80 0x177af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:42.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.033+0000 7f5734bb0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x14e8150 0x177a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:42.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.033+0000 7f5734bb0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x1780bc0 con 0x14e79b0 2023-10-11T09:50:42.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.033+0000 7f5734bb0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x179e8f0 con 0x14e8150 2023-10-11T09:50:42.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.033+0000 7f5734bb0740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x179e690 con 0x14e7d80 2023-10-11T09:50:42.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.033+0000 7f5734bb0740 10 client.0 _ll_register_callbacks cb 0x1678d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:50:42.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.033+0000 7f572affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x14e79b0 0x17fc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:42.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.033+0000 7f572a7fc700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x14e7d80 0x177af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:42.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.033+0000 7f572affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x14e79b0 0x17fc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41064/0 (socket says 172.21.15.78:41064) 2023-10-11T09:50:42.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.033+0000 7f572affd700 1 -- 172.21.15.78:0/3922050391 learned_addr learned my addr 172.21.15.78:0/3922050391 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:42.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.033+0000 7f572b7fe700 1 --2- 172.21.15.78:0/3922050391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x14e8150 0x177a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:42.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.033+0000 7f572affd700 1 -- 172.21.15.78:0/3922050391 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x14e7d80 msgr2=0x177af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:42.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.033+0000 7f572affd700 1 --2- 172.21.15.78:0/3922050391 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x14e7d80 0x177af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.033+0000 7f572affd700 1 -- 172.21.15.78:0/3922050391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x14e8150 msgr2=0x177a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:42.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.033+0000 7f572affd700 1 --2- 172.21.15.78:0/3922050391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x14e8150 0x177a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.033+0000 7f572affd700 1 -- 172.21.15.78:0/3922050391 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f572c072300 con 0x14e79b0 2023-10-11T09:50:42.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.033+0000 7f572affd700 1 --2- 172.21.15.78:0/3922050391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x14e79b0 0x17fc500 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f571c053d10 tx=0x7f571c01e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aed6406e2cf2e5b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:42.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.033+0000 7f570a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:50:42.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.033+0000 7f570a7fc700 1 -- 172.21.15.78:0/3922050391 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571c00ce60 con 0x14e79b0 2023-10-11T09:50:42.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.033+0000 7f570a7fc700 1 -- 172.21.15.78:0/3922050391 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f571c05bff0 con 0x14e79b0 2023-10-11T09:50:42.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.033+0000 7f5734bb0740 1 -- 172.21.15.78:0/3922050391 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f572401a420 con 0x14e79b0 2023-10-11T09:50:42.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.037+0000 7f570a7fc700 1 -- 172.21.15.78:0/3922050391 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f571c079ea0 con 0x14e79b0 2023-10-11T09:50:42.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.037+0000 7f5734bb0740 10 client.8628 fetch_fsmap learned FSMap version 91 2023-10-11T09:50:42.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.037+0000 7f5734bb0740 1 -- 172.21.15.78:0/3922050391 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x1533c80 con 0x14e79b0 2023-10-11T09:50:42.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.037+0000 7f570a7fc700 1 -- 172.21.15.78:0/3922050391 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f572401a420 con 0x14e79b0 2023-10-11T09:50:42.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.037+0000 7f570a7fc700 1 -- 172.21.15.78:0/3922050391 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 91) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f571c07d070 con 0x14e79b0 2023-10-11T09:50:42.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.037+0000 7f5734bb0740 10 client.8628 fetch_fsmap finished waiting for FSMap version 91 2023-10-11T09:50:42.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.037+0000 7f5734bb0740 10 client.8628 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2023-10-11T09:50:42.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.037+0000 7f5734bb0740 20 client.8628 populate_metadata read hostname 'smithi078' 2023-10-11T09:50:42.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.037+0000 7f5734bb0740 1 --2- 172.21.15.78:0/3922050391 >> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] conn(0x1532a30 0x177a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:42.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.037+0000 7f5734bb0740 4 client.8628 mds_command: new command op to 8550 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2023-10-11T09:50:42.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.037+0000 7f5734bb0740 1 -- 172.21.15.78:0/3922050391 --> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x153f780 con 0x1532a30 2023-10-11T09:50:42.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.037+0000 7f572a7fc700 1 --2- 172.21.15.78:0/3922050391 >> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] conn(0x1532a30 0x177a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:42.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.037+0000 7f572a7fc700 1 --2- 172.21.15.78:0/3922050391 >> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] conn(0x1532a30 0x177a5f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6a86a2afb0579fb server_cookie=33d3febbf6a28ece in_seq=0 out_seq=0 2023-10-11T09:50:42.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.037+0000 7f570a7fc700 10 client.8628 ms_handle_connect on v2:172.21.15.78:6836/2869034036 2023-10-11T09:50:42.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.037+0000 7f570a7fc700 1 -- 172.21.15.78:0/3922050391 <== mds.0 v2:172.21.15.78:6836/2869034036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x153f780 con 0x1532a30 2023-10-11T09:50:42.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.037+0000 7f570a7fc700 10 client.8628 handle_command_reply: tid=0 2023-10-11T09:50:42.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.037+0000 7f5734bb0740 1 client.8628 shutdown 2023-10-11T09:50:42.053 INFO:tasks.ceph.mds.c.smithi078.stderr:2023-10-11T09:50:42.037+0000 7f608874c700 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2023-10-11T09:50:42.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.037+0000 7f5734bb0740 1 -- 172.21.15.78:0/3922050391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x14e79b0 msgr2=0x17fc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:42.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.037+0000 7f5734bb0740 1 --2- 172.21.15.78:0/3922050391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x14e79b0 0x17fc500 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f571c053d10 tx=0x7f571c01e070 comp rx=0 tx=0).stop 2023-10-11T09:50:42.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.037+0000 7f5734bb0740 1 -- 172.21.15.78:0/3922050391 shutdown_connections 2023-10-11T09:50:42.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.037+0000 7f5734bb0740 1 --2- 172.21.15.78:0/3922050391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x14e8150 0x177a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.037+0000 7f5734bb0740 1 --2- 172.21.15.78:0/3922050391 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x14e7d80 0x177af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.037+0000 7f5734bb0740 1 --2- 172.21.15.78:0/3922050391 >> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] conn(0x1532a30 0x177a5f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.037+0000 7f5734bb0740 1 --2- 172.21.15.78:0/3922050391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x14e79b0 0x17fc500 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.041+0000 7f5734bb0740 1 -- 172.21.15.78:0/3922050391 >> 172.21.15.78:0/3922050391 conn(0x17832b0 msgr2=0x179d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:42.055 INFO:tasks.ceph.mds.c.smithi078.stderr:2023-10-11T09:50:42.041+0000 7f608874c700 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x4 2023-10-11T09:50:42.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.041+0000 7f570a7fc700 0 client.8628 ms_handle_reset on v2:172.21.15.78:6836/2869034036 2023-10-11T09:50:42.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.041+0000 7f5734bb0740 1 -- 172.21.15.78:0/3922050391 shutdown_connections 2023-10-11T09:50:42.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.041+0000 7f5734bb0740 1 -- 172.21.15.78:0/3922050391 wait complete. 2023-10-11T09:50:42.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.041+0000 7f5734bb0740 20 client.8628 trim_cache size 0 max 16384 2023-10-11T09:50:42.057 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:50:42.057 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2023-10-11T09:50:42.057 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "5fff9c3f-bfeb-41e7-9d9e-0d94063a73bc", 2023-10-11T09:50:42.058 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2023-10-11T09:50:42.058 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:50:42.058 INFO:tasks.ceph.mds.c.smithi078.stderr:2023-10-11T09:50:42.041+0000 7f608874c700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /stripey/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55e666158100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000000:[<0x1/stripey v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-10-11T09:50:42.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.041+0000 7f573191f700 1 -- 172.21.15.78:0/824153961 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5718065ed0 msgr2=0x7f5718068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:42.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.041+0000 7f573191f700 1 --2- 172.21.15.78:0/824153961 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5718065ed0 0x7f5718068380 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f5724007f40 tx=0x7f5724007490 comp rx=0 tx=0).stop 2023-10-11T09:50:42.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.041+0000 7f573191f700 1 -- 172.21.15.78:0/824153961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f572c06c630 msgr2=0x7f572c161990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:42.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.041+0000 7f573191f700 1 --2- 172.21.15.78:0/824153961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f572c06c630 0x7f572c161990 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f571c00c530 tx=0x7f571c00c8f0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.041+0000 7f573191f700 1 -- 172.21.15.78:0/824153961 shutdown_connections 2023-10-11T09:50:42.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.041+0000 7f573191f700 1 --2- 172.21.15.78:0/824153961 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5718065ed0 0x7f5718068380 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.041+0000 7f573191f700 1 --2- 172.21.15.78:0/824153961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f572c1649c0 0x7f572c0c7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.041+0000 7f573191f700 1 --2- 172.21.15.78:0/824153961 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f572c160d90 0x7f572c1f5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.041+0000 7f573191f700 1 --2- 172.21.15.78:0/824153961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f572c06c630 0x7f572c161990 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.061 INFO:tasks.ceph.mds.c.smithi078.stderr:2023-10-11T09:50:42.041+0000 7f608874c700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x4 [...2,head] /lost+found/ auth v1 pv6 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55e666158680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x4:[<0x1/lost+found v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-10-11T09:50:42.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.041+0000 7f573191f700 1 -- 172.21.15.78:0/824153961 >> 172.21.15.78:0/824153961 conn(0x7f572c06a060 msgr2=0x7f572c15eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:42.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.041+0000 7f573191f700 1 -- 172.21.15.78:0/824153961 shutdown_connections 2023-10-11T09:50:42.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.041+0000 7f573191f700 1 -- 172.21.15.78:0/824153961 wait complete. 2023-10-11T09:50:42.063 INFO:tasks.ceph.mds.c.smithi078.stderr:2023-10-11T09:50:42.045+0000 7f608874c700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fb [2,head] /stripey/flushed_file_6 auth v1 ap=1 DIRTYPARENT s=1572864 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e6661c5180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(16)0x100000001fb:[<0x10000000000/flushed_file_6 v68>,<0x1/stripey v73>]//[15]","memoryvalue":"(16)0x100000001fb:[<0x10000000000/flushed_file_6 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:50:42.063 INFO:tasks.ceph.mds.c.smithi078.stderr:2023-10-11T09:50:42.045+0000 7f608874c700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f9 [2,head] /stripey/flushed_file_4 auth v1 ap=1 DIRTYPARENT s=1441792 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e666215600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(16)0x100000001f9:[<0x10000000000/flushed_file_4 v42>,<0x1/stripey v73>]//[15]","memoryvalue":"(16)0x100000001f9:[<0x10000000000/flushed_file_4 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:50:42.064 INFO:tasks.ceph.mds.c.smithi078.stderr:2023-10-11T09:50:42.045+0000 7f608874c700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f8 [2,head] /stripey/flushed_file_3 auth v1 ap=1 DIRTYPARENT s=917503 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e666215080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(16)0x100000001f8:[<0x10000000000/flushed_file_3 v64>,<0x1/stripey v73>]//[15]","memoryvalue":"(16)0x100000001f8:[<0x10000000000/flushed_file_3 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:50:42.064 INFO:tasks.ceph.mds.c.smithi078.stderr:2023-10-11T09:50:42.045+0000 7f608874c700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f7 [2,head] /stripey/flushed_file_2 auth v1 ap=1 DIRTYPARENT s=917504 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e666214000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(16)0x100000001f7:[<0x10000000000/flushed_file_2 v56>,<0x1/stripey v73>]//[15]","memoryvalue":"(16)0x100000001f7:[<0x10000000000/flushed_file_2 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:50:42.064 INFO:tasks.ceph.mds.c.smithi078.stderr:2023-10-11T09:50:42.045+0000 7f608874c700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f6 [2,head] /stripey/flushed_file_1 auth v1 ap=1 DIRTYPARENT s=524288 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e666215b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(16)0x100000001f6:[<0x10000000000/flushed_file_1 v50>,<0x1/stripey v73>]//[15]","memoryvalue":"(16)0x100000001f6:[<0x10000000000/flushed_file_1 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:50:42.064 INFO:tasks.ceph.mds.c.smithi078.stderr:2023-10-11T09:50:42.045+0000 7f608874c700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /stripey/flushed_file_0 auth v1 ap=1 DIRTYPARENT s=1048576 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e6652e1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(16)0x10000000001:[<0x10000000000/flushed_file_0 v36>,<0x1/stripey v73>]//[15]","memoryvalue":"(16)0x10000000001:[<0x10000000000/flushed_file_0 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:50:42.064 INFO:tasks.ceph.mds.c.smithi078.stderr:2023-10-11T09:50:42.045+0000 7f608874c700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fa [2,head] /stripey/flushed_file_5 auth v1 ap=1 DIRTYPARENT s=1441793 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e6661c5700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(16)0x100000001fa:[<0x10000000000/flushed_file_5 v54>,<0x1/stripey v73>]//[15]","memoryvalue":"(16)0x100000001fa:[<0x10000000000/flushed_file_5 v1>,<0x1/stripey v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:50:42.065 INFO:tasks.ceph.mds.c.smithi078.stderr:2023-10-11T09:50:42.045+0000 7f608874c700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fc [2,head] /lost+found/100000001fc auth v1 ap=1 DIRTYPARENT s=524288 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e6652e1b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(16)0x100000001fc:[<0x4/100000001fc v1>,<0x1/lost+found v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2023-10-11T09:50:42.065 INFO:tasks.ceph.mds.c.smithi078.stderr:2023-10-11T09:50:42.053+0000 7f608874c700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v7 ap=5 snaprealm=0x55e666142fc0 DIRTYPARENT f(v1 2=0+2) n(v0 rc2023-10-11T09:50:41.751037+0000 2=0+2)/n() (idft sync r=1) (inest sync r=1) (ifile sync->mix r=1) (iversion lock) caps={8607=pLsXs/-@3} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x55e666157600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 2=0+2)","ondisk_value.rstat":"n(v0 3=0+3)","memory_value.dirstat":"f(v1 2=0+2)","memory_value.rstat":"n(v0 rc2023-10-11T09:50:41.751037+0000 2=0+2)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-10-11T09:50:42.072 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 scrub status 2023-10-11T09:50:42.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.237+0000 7fb9a1f97700 1 Processor -- start 2023-10-11T09:50:42.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.237+0000 7fb9a1f97700 1 -- start start 2023-10-11T09:50:42.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.241+0000 7fb9a1f97700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb99c06d400 0x7fb99c15a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:42.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.241+0000 7fb9a1f97700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb99c157500 con 0x7fb99c0ce240 2023-10-11T09:50:42.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.241+0000 7fb9a1f97700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb99c070d20 con 0x7fb99c157130 2023-10-11T09:50:42.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.241+0000 7fb9a1f97700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c0b1aa0 con 0x7fb99c06d400 2023-10-11T09:50:42.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.241+0000 7fb9a1796700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fb99c0ce240 0x7fb99c0b1ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:41618/0 (socket says 172.21.15.78:41618) 2023-10-11T09:50:42.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.241+0000 7fb9a1796700 1 -- 172.21.15.78:0/351997259 learned_addr learned my addr 172.21.15.78:0/351997259 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:50:42.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.241+0000 7fb993fff700 1 --2- 172.21.15.78:0/351997259 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb99c06d400 0x7fb99c15a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:42.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.241+0000 7fb9937fe700 1 -- 172.21.15.78:0/351997259 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2560075846 0 0) 0x7fb99c157500 con 0x7fb99c0ce240 2023-10-11T09:50:42.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.241+0000 7fb9937fe700 1 -- 172.21.15.78:0/351997259 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9840035d0 con 0x7fb99c0ce240 2023-10-11T09:50:42.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.241+0000 7fb9937fe700 1 -- 172.21.15.78:0/351997259 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 906483206 0 0) 0x7fb99c070d20 con 0x7fb99c157130 2023-10-11T09:50:42.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.241+0000 7fb9937fe700 1 -- 172.21.15.78:0/351997259 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb99c157500 con 0x7fb99c157130 2023-10-11T09:50:42.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.241+0000 7fb993fff700 1 -- 172.21.15.78:0/351997259 >> v1:172.21.15.133:6789/0 conn(0x7fb99c157130 legacy=0x7fb99c06ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:42.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.241+0000 7fb993fff700 1 -- 172.21.15.78:0/351997259 >> v1:172.21.15.78:6789/0 conn(0x7fb99c0ce240 legacy=0x7fb99c0b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:42.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.241+0000 7fb993fff700 1 -- 172.21.15.78:0/351997259 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb99c071000 con 0x7fb99c06d400 2023-10-11T09:50:42.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.245+0000 7fb993fff700 1 --2- 172.21.15.78:0/351997259 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb99c06d400 0x7fb99c15a680 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb98c009880 tx=0x7fb98c009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da59211db9b95d22 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:42.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.245+0000 7fb9937fe700 1 -- 172.21.15.78:0/351997259 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98c011830 con 0x7fb99c06d400 2023-10-11T09:50:42.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.245+0000 7fb9937fe700 1 -- 172.21.15.78:0/351997259 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb98c011990 con 0x7fb99c06d400 2023-10-11T09:50:42.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.245+0000 7fb9937fe700 1 -- 172.21.15.78:0/351997259 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98c011d20 con 0x7fb99c06d400 2023-10-11T09:50:42.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.245+0000 7fb9a1f97700 1 -- 172.21.15.78:0/351997259 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb99c06d400 msgr2=0x7fb99c15a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:42.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.245+0000 7fb9a1f97700 1 --2- 172.21.15.78:0/351997259 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb99c06d400 0x7fb99c15a680 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb98c009880 tx=0x7fb98c009b90 comp rx=0 tx=0).stop 2023-10-11T09:50:42.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.245+0000 7fb9a1f97700 1 -- 172.21.15.78:0/351997259 shutdown_connections 2023-10-11T09:50:42.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.245+0000 7fb9a1f97700 1 --2- 172.21.15.78:0/351997259 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb99c06d400 0x7fb99c15a680 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.245+0000 7fb9a1f97700 1 -- 172.21.15.78:0/351997259 >> 172.21.15.78:0/351997259 conn(0x7fb99c06a060 msgr2=0x7fb99c06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:42.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.245+0000 7fb9a1f97700 1 -- 172.21.15.78:0/351997259 shutdown_connections 2023-10-11T09:50:42.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.245+0000 7fb9a1f97700 1 -- 172.21.15.78:0/351997259 wait complete. 2023-10-11T09:50:42.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.245+0000 7fb9a1f97700 1 Processor -- start 2023-10-11T09:50:42.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.249+0000 7fb9a1f97700 1 -- start start 2023-10-11T09:50:42.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.249+0000 7fb9a1f97700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb99c06d400 0x7fb99c1f3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:42.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.249+0000 7fb9a1f97700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb99c0ce240 0x7fb99c1518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:42.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.249+0000 7fb9a1f97700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb99c157130 0x7fb99c15be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:42.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.249+0000 7fb9a1f97700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c1f3ad0 con 0x7fb99c0ce240 2023-10-11T09:50:42.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.249+0000 7fb9a1f97700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c0b05c0 con 0x7fb99c06d400 2023-10-11T09:50:42.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.249+0000 7fb9a1f97700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c0b02e0 con 0x7fb99c157130 2023-10-11T09:50:42.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.249+0000 7fb993fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb99c0ce240 0x7fb99c1518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:42.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.249+0000 7fb993fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb99c0ce240 0x7fb99c1518c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41072/0 (socket says 172.21.15.78:41072) 2023-10-11T09:50:42.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.249+0000 7fb993fff700 1 -- 172.21.15.78:0/1642463424 learned_addr learned my addr 172.21.15.78:0/1642463424 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:42.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.249+0000 7fb9a1796700 1 --2- 172.21.15.78:0/1642463424 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb99c157130 0x7fb99c15be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:42.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.249+0000 7fb9a0f95700 1 --2- 172.21.15.78:0/1642463424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb99c06d400 0x7fb99c1f3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:42.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.249+0000 7fb993fff700 1 -- 172.21.15.78:0/1642463424 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb99c157130 msgr2=0x7fb99c15be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:42.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.249+0000 7fb993fff700 1 --2- 172.21.15.78:0/1642463424 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb99c157130 0x7fb99c15be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.249+0000 7fb993fff700 1 -- 172.21.15.78:0/1642463424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb99c06d400 msgr2=0x7fb99c1f3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:42.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.249+0000 7fb993fff700 1 --2- 172.21.15.78:0/1642463424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb99c06d400 0x7fb99c1f3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.249+0000 7fb993fff700 1 -- 172.21.15.78:0/1642463424 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb98c0094b0 con 0x7fb99c0ce240 2023-10-11T09:50:42.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.249+0000 7fb993fff700 1 --2- 172.21.15.78:0/1642463424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb99c0ce240 0x7fb99c1518c0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fb98c013040 tx=0x7fb98c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:42.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.249+0000 7fb9917fa700 1 -- 172.21.15.78:0/1642463424 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98c01a8c0 con 0x7fb99c0ce240 2023-10-11T09:50:42.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.249+0000 7fb9917fa700 1 -- 172.21.15.78:0/1642463424 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb98c01e070 con 0x7fb99c0ce240 2023-10-11T09:50:42.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.249+0000 7fb9917fa700 1 -- 172.21.15.78:0/1642463424 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98c01ac40 con 0x7fb99c0ce240 2023-10-11T09:50:42.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.249+0000 7fb9a1f97700 1 -- 172.21.15.78:0/1642463424 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb99c153140 con 0x7fb99c0ce240 2023-10-11T09:50:42.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.253+0000 7fb9a1f97700 1 -- 172.21.15.78:0/1642463424 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb99c153700 con 0x7fb99c0ce240 2023-10-11T09:50:42.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.257+0000 7fb9917fa700 1 -- 172.21.15.78:0/1642463424 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fb98c02d450 con 0x7fb99c0ce240 2023-10-11T09:50:42.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.257+0000 7fb9917fa700 1 --2- 172.21.15.78:0/1642463424 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb988065ed0 0x7fb988068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:42.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.257+0000 7fb9917fa700 1 -- 172.21.15.78:0/1642463424 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6724+0+0 (secure 0 0 0) 0x7fb98c084950 con 0x7fb99c0ce240 2023-10-11T09:50:42.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.257+0000 7fb9a0f95700 1 --2- 172.21.15.78:0/1642463424 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb988065ed0 0x7fb988068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:42.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.257+0000 7fb9a6490740 1 Processor -- start 2023-10-11T09:50:42.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.261+0000 7fb9a0f95700 1 --2- 172.21.15.78:0/1642463424 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb988065ed0 0x7fb988068380 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb9980011e0 tx=0x7fb998009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:42.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.261+0000 7fb9a6490740 1 -- start start 2023-10-11T09:50:42.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.261+0000 7fb9a6490740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x22fec10 0x25945a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:42.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.261+0000 7fb9a6490740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x22fe840 0x25b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:42.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.261+0000 7fb9a6490740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x23498c0 0x25b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:42.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.261+0000 7fb9a6490740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x24de630 con 0x22fe840 2023-10-11T09:50:42.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.261+0000 7fb9a6490740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x24dbdc0 con 0x22fec10 2023-10-11T09:50:42.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.261+0000 7fb9a6490740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x24d76b0 con 0x23498c0 2023-10-11T09:50:42.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.261+0000 7fb993fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x22fe840 0x25b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:42.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.261+0000 7fb993fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x22fe840 0x25b5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41082/0 (socket says 172.21.15.78:41082) 2023-10-11T09:50:42.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.261+0000 7fb993fff700 1 -- 172.21.15.78:0/2728487919 learned_addr learned my addr 172.21.15.78:0/2728487919 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:42.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.261+0000 7fb9a1796700 1 --2- 172.21.15.78:0/2728487919 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x23498c0 0x25b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:42.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.261+0000 7fb9a0f95700 1 --2- 172.21.15.78:0/2728487919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x22fec10 0x25945a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:42.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.261+0000 7fb993fff700 1 -- 172.21.15.78:0/2728487919 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x23498c0 msgr2=0x25b62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:42.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.261+0000 7fb993fff700 1 --2- 172.21.15.78:0/2728487919 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x23498c0 0x25b62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.261+0000 7fb993fff700 1 -- 172.21.15.78:0/2728487919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x22fec10 msgr2=0x25945a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:42.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.261+0000 7fb993fff700 1 --2- 172.21.15.78:0/2728487919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x22fec10 0x25945a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.261+0000 7fb993fff700 1 -- 172.21.15.78:0/2728487919 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x24fd5b0 con 0x22fe840 2023-10-11T09:50:42.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.261+0000 7fb993fff700 1 --2- 172.21.15.78:0/2728487919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x22fe840 0x25b5d90 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fb98c05d460 tx=0x7fb98c055310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0571f118e17b1a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:42.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.261+0000 7fb982ffd700 1 -- 172.21.15.78:0/2728487919 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98c0559b0 con 0x22fe840 2023-10-11T09:50:42.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.261+0000 7fb982ffd700 1 -- 172.21.15.78:0/2728487919 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb98c0597e0 con 0x22fe840 2023-10-11T09:50:42.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.261+0000 7fb982ffd700 1 -- 172.21.15.78:0/2728487919 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98c05d990 con 0x22fe840 2023-10-11T09:50:42.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.265+0000 7fb9a6490740 1 -- 172.21.15.78:0/2728487919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x22fe840 msgr2=0x25b5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:42.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.265+0000 7fb9a6490740 1 --2- 172.21.15.78:0/2728487919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x22fe840 0x25b5d90 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fb98c05d460 tx=0x7fb98c055310 comp rx=0 tx=0).stop 2023-10-11T09:50:42.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.265+0000 7fb9a6490740 1 -- 172.21.15.78:0/2728487919 shutdown_connections 2023-10-11T09:50:42.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.265+0000 7fb9a6490740 1 --2- 172.21.15.78:0/2728487919 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x23498c0 0x25b62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.265+0000 7fb9a6490740 1 --2- 172.21.15.78:0/2728487919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x22fe840 0x25b5d90 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.265+0000 7fb9a6490740 1 --2- 172.21.15.78:0/2728487919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x22fec10 0x25945a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.265+0000 7fb9a6490740 1 -- 172.21.15.78:0/2728487919 >> 172.21.15.78:0/2728487919 conn(0x22fefe0 msgr2=0x2597350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:42.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.265+0000 7fb9a6490740 1 -- 172.21.15.78:0/2728487919 shutdown_connections 2023-10-11T09:50:42.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.265+0000 7fb9a6490740 1 -- 172.21.15.78:0/2728487919 wait complete. 2023-10-11T09:50:42.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.265+0000 7fb9a6490740 1 -- start start 2023-10-11T09:50:42.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.265+0000 7fb9a6490740 1 Processor -- start 2023-10-11T09:50:42.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.269+0000 7fb9a6490740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x22fec10 0x25904e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:42.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.269+0000 7fb9a6490740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x23498c0 0x2593660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:42.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.269+0000 7fb9a6490740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2593ba0 0x259f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:42.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.269+0000 7fb9a6490740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x249ac10 con 0x2593ba0 2023-10-11T09:50:42.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.269+0000 7fb9a6490740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x249a8b0 con 0x23498c0 2023-10-11T09:50:42.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.269+0000 7fb9a6490740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x2383c50 con 0x22fec10 2023-10-11T09:50:42.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.269+0000 7fb9a6490740 10 client.0 _ll_register_callbacks cb 0x24d0ff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:50:42.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.269+0000 7fb9a1796700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2593ba0 0x259f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:42.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.269+0000 7fb9a1796700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2593ba0 0x259f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41086/0 (socket says 172.21.15.78:41086) 2023-10-11T09:50:42.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.269+0000 7fb9a1796700 1 -- 172.21.15.78:0/3404185715 learned_addr learned my addr 172.21.15.78:0/3404185715 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:42.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.269+0000 7fb993fff700 1 --2- 172.21.15.78:0/3404185715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x23498c0 0x2593660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:42.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.269+0000 7fb9a0f95700 1 --2- 172.21.15.78:0/3404185715 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x22fec10 0x25904e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:42.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.269+0000 7fb9a1796700 1 -- 172.21.15.78:0/3404185715 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x22fec10 msgr2=0x25904e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:42.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.269+0000 7fb9a1796700 1 --2- 172.21.15.78:0/3404185715 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x22fec10 0x25904e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.269+0000 7fb9a1796700 1 -- 172.21.15.78:0/3404185715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x23498c0 msgr2=0x2593660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:42.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.269+0000 7fb9a1796700 1 --2- 172.21.15.78:0/3404185715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x23498c0 0x2593660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.269+0000 7fb9a1796700 1 -- 172.21.15.78:0/3404185715 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb99c153700 con 0x2593ba0 2023-10-11T09:50:42.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.269+0000 7fb9a1796700 1 --2- 172.21.15.78:0/3404185715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2593ba0 0x259f7a0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fb994007b00 tx=0x7fb994007b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1871983431b93767 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:42.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.269+0000 7fb97bfff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:50:42.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.269+0000 7fb97bfff700 1 -- 172.21.15.78:0/3404185715 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb994007ba0 con 0x2593ba0 2023-10-11T09:50:42.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.269+0000 7fb9a6490740 1 -- 172.21.15.78:0/3404185715 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x24d0730 con 0x2593ba0 2023-10-11T09:50:42.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.269+0000 7fb97bfff700 1 -- 172.21.15.78:0/3404185715 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 6724+0+0 (secure 0 0 0) 0x7fb994012960 con 0x2593ba0 2023-10-11T09:50:42.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.269+0000 7fb97bfff700 1 -- 172.21.15.78:0/3404185715 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb994012b20 con 0x2593ba0 2023-10-11T09:50:42.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.269+0000 7fb9a6490740 10 client.8640 fetch_fsmap learned FSMap version 91 2023-10-11T09:50:42.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.269+0000 7fb9a6490740 1 -- 172.21.15.78:0/3404185715 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x234ab10 con 0x2593ba0 2023-10-11T09:50:42.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.269+0000 7fb97bfff700 1 -- 172.21.15.78:0/3404185715 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x24d0730 con 0x2593ba0 2023-10-11T09:50:42.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.269+0000 7fb97bfff700 1 -- 172.21.15.78:0/3404185715 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 91) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fb994012c80 con 0x2593ba0 2023-10-11T09:50:42.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.269+0000 7fb9a6490740 10 client.8640 fetch_fsmap finished waiting for FSMap version 91 2023-10-11T09:50:42.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.269+0000 7fb9a6490740 10 client.8640 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2023-10-11T09:50:42.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.269+0000 7fb9a6490740 20 client.8640 populate_metadata read hostname 'smithi078' 2023-10-11T09:50:42.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.269+0000 7fb9a6490740 1 --2- 172.21.15.78:0/3404185715 >> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] conn(0x2574810 0x25c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:42.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.269+0000 7fb9a6490740 4 client.8640 mds_command: new command op to 8550 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-11T09:50:42.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.269+0000 7fb9a6490740 1 -- 172.21.15.78:0/3404185715 --> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x24d0450 con 0x2574810 2023-10-11T09:50:42.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.273+0000 7fb9a0f95700 1 --2- 172.21.15.78:0/3404185715 >> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] conn(0x2574810 0x25c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:42.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.273+0000 7fb9a0f95700 1 --2- 172.21.15.78:0/3404185715 >> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] conn(0x2574810 0x25c8d70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35b1021b193f272d server_cookie=635f0777da4bfd37 in_seq=0 out_seq=0 2023-10-11T09:50:42.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.273+0000 7fb97bfff700 10 client.8640 ms_handle_connect on v2:172.21.15.78:6836/2869034036 2023-10-11T09:50:42.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.273+0000 7fb97bfff700 1 -- 172.21.15.78:0/3404185715 <== mds.0 v2:172.21.15.78:6836/2869034036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x24d0450 con 0x2574810 2023-10-11T09:50:42.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.273+0000 7fb97bfff700 10 client.8640 handle_command_reply: tid=0 2023-10-11T09:50:42.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.273+0000 7fb9a6490740 1 client.8640 shutdown 2023-10-11T09:50:42.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.273+0000 7fb9a6490740 1 -- 172.21.15.78:0/3404185715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2593ba0 msgr2=0x259f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:42.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.273+0000 7fb9a6490740 1 --2- 172.21.15.78:0/3404185715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2593ba0 0x259f7a0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fb994007b00 tx=0x7fb994007b30 comp rx=0 tx=0).stop 2023-10-11T09:50:42.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.273+0000 7fb9a6490740 1 -- 172.21.15.78:0/3404185715 shutdown_connections 2023-10-11T09:50:42.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.273+0000 7fb9a6490740 1 --2- 172.21.15.78:0/3404185715 >> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] conn(0x2574810 0x25c8d70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.273+0000 7fb9a6490740 1 --2- 172.21.15.78:0/3404185715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2593ba0 0x259f7a0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.273+0000 7fb9a6490740 1 --2- 172.21.15.78:0/3404185715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x23498c0 0x2593660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.273+0000 7fb97bfff700 0 client.8640 ms_handle_reset on v2:172.21.15.78:6836/2869034036 2023-10-11T09:50:42.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.273+0000 7fb9a6490740 1 --2- 172.21.15.78:0/3404185715 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x22fec10 0x25904e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.273+0000 7fb9a6490740 1 -- 172.21.15.78:0/3404185715 >> 172.21.15.78:0/3404185715 conn(0x22fefe0 msgr2=0x2597750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:42.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.273+0000 7fb9a6490740 1 -- 172.21.15.78:0/3404185715 shutdown_connections 2023-10-11T09:50:42.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.273+0000 7fb9a6490740 1 -- 172.21.15.78:0/3404185715 wait complete. 2023-10-11T09:50:42.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.273+0000 7fb9a6490740 20 client.8640 trim_cache size 0 max 16384 2023-10-11T09:50:42.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.293+0000 7fb9a6490740 1 Processor -- start 2023-10-11T09:50:42.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.293+0000 7fb9a6490740 1 -- start start 2023-10-11T09:50:42.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.293+0000 7fb9a6490740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x22fec10 0x256fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:42.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.293+0000 7fb9a6490740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x23498c0 0x256ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:42.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.293+0000 7fb9a6490740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2593ba0 0x2596460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:42.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.293+0000 7fb9a6490740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x247f480 con 0x23498c0 2023-10-11T09:50:42.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.293+0000 7fb9a6490740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x2487dc0 con 0x22fec10 2023-10-11T09:50:42.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.293+0000 7fb9a6490740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x2565e70 con 0x2593ba0 2023-10-11T09:50:42.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.293+0000 7fb993fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x23498c0 0x256ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:42.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.293+0000 7fb993fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x23498c0 0x256ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41102/0 (socket says 172.21.15.78:41102) 2023-10-11T09:50:42.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.293+0000 7fb993fff700 1 -- 172.21.15.78:0/3136770587 learned_addr learned my addr 172.21.15.78:0/3136770587 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:42.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.297+0000 7fb9a1796700 1 --2- 172.21.15.78:0/3136770587 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2593ba0 0x2596460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:42.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.297+0000 7fb9a0f95700 1 --2- 172.21.15.78:0/3136770587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x22fec10 0x256fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:42.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.297+0000 7fb993fff700 1 -- 172.21.15.78:0/3136770587 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2593ba0 msgr2=0x2596460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:42.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.297+0000 7fb993fff700 1 --2- 172.21.15.78:0/3136770587 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2593ba0 0x2596460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.297+0000 7fb993fff700 1 -- 172.21.15.78:0/3136770587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x22fec10 msgr2=0x256fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:42.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.297+0000 7fb993fff700 1 --2- 172.21.15.78:0/3136770587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x22fec10 0x256fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.297+0000 7fb993fff700 1 -- 172.21.15.78:0/3136770587 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb99c15c6a0 con 0x23498c0 2023-10-11T09:50:42.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.297+0000 7fb993fff700 1 --2- 172.21.15.78:0/3136770587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x23498c0 0x256ff70 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fb98c009ee0 tx=0x7fb98c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b4248488c2b5c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:42.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.297+0000 7fb9837fe700 1 -- 172.21.15.78:0/3136770587 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98c051cf0 con 0x23498c0 2023-10-11T09:50:42.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.297+0000 7fb9837fe700 1 -- 172.21.15.78:0/3136770587 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb98c05d990 con 0x23498c0 2023-10-11T09:50:42.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.297+0000 7fb9837fe700 1 -- 172.21.15.78:0/3136770587 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98c0559b0 con 0x23498c0 2023-10-11T09:50:42.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.297+0000 7fb9a6490740 1 -- 172.21.15.78:0/3136770587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x23498c0 msgr2=0x256ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:42.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.297+0000 7fb9a6490740 1 --2- 172.21.15.78:0/3136770587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x23498c0 0x256ff70 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fb98c009ee0 tx=0x7fb98c01c000 comp rx=0 tx=0).stop 2023-10-11T09:50:42.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.297+0000 7fb9a6490740 1 -- 172.21.15.78:0/3136770587 shutdown_connections 2023-10-11T09:50:42.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.297+0000 7fb9a6490740 1 --2- 172.21.15.78:0/3136770587 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2593ba0 0x2596460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.297+0000 7fb9a6490740 1 --2- 172.21.15.78:0/3136770587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x23498c0 0x256ff70 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.297+0000 7fb9a6490740 1 --2- 172.21.15.78:0/3136770587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x22fec10 0x256fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.297+0000 7fb9a6490740 1 -- 172.21.15.78:0/3136770587 >> 172.21.15.78:0/3136770587 conn(0x22fefe0 msgr2=0x25a1230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:42.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.297+0000 7fb9a6490740 1 -- 172.21.15.78:0/3136770587 shutdown_connections 2023-10-11T09:50:42.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.297+0000 7fb9a6490740 1 -- 172.21.15.78:0/3136770587 wait complete. 2023-10-11T09:50:42.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.297+0000 7fb9a6490740 1 -- start start 2023-10-11T09:50:42.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.301+0000 7fb9a6490740 1 Processor -- start 2023-10-11T09:50:42.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.301+0000 7fb9a6490740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x22fec10 0x2612090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:42.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.301+0000 7fb9a6490740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x23498c0 0x25927b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:42.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.301+0000 7fb9a6490740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2593ba0 0x2592cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:42.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.301+0000 7fb9a6490740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x259eaa0 con 0x22fec10 2023-10-11T09:50:42.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.301+0000 7fb9a6490740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x259e560 con 0x23498c0 2023-10-11T09:50:42.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.301+0000 7fb9a6490740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x259e800 con 0x2593ba0 2023-10-11T09:50:42.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.301+0000 7fb9a6490740 10 client.0 _ll_register_callbacks cb 0x24d5e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:50:42.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.301+0000 7fb9a0f95700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x22fec10 0x2612090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:42.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.301+0000 7fb9a0f95700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x22fec10 0x2612090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41116/0 (socket says 172.21.15.78:41116) 2023-10-11T09:50:42.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.301+0000 7fb9a0f95700 1 -- 172.21.15.78:0/1089767655 learned_addr learned my addr 172.21.15.78:0/1089767655 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:50:42.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.301+0000 7fb9a1796700 1 --2- 172.21.15.78:0/1089767655 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2593ba0 0x2592cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:42.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.301+0000 7fb993fff700 1 --2- 172.21.15.78:0/1089767655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x23498c0 0x25927b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:42.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.301+0000 7fb9a0f95700 1 -- 172.21.15.78:0/1089767655 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2593ba0 msgr2=0x2592cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:42.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.301+0000 7fb9a0f95700 1 --2- 172.21.15.78:0/1089767655 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2593ba0 0x2592cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.301+0000 7fb9a0f95700 1 -- 172.21.15.78:0/1089767655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x23498c0 msgr2=0x25927b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:50:42.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.301+0000 7fb9a0f95700 1 --2- 172.21.15.78:0/1089767655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x23498c0 0x25927b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.301+0000 7fb9a0f95700 1 -- 172.21.15.78:0/1089767655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x24fd5b0 con 0x22fec10 2023-10-11T09:50:42.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.301+0000 7fb9a0f95700 1 --2- 172.21.15.78:0/1089767655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x22fec10 0x2612090 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fb998007ec0 tx=0x7fb998000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167f166709338780 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:50:42.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.301+0000 7fb97bfff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:50:42.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.305+0000 7fb97bfff700 1 -- 172.21.15.78:0/1089767655 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb998025430 con 0x22fec10 2023-10-11T09:50:42.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.305+0000 7fb9a6490740 1 -- 172.21.15.78:0/1089767655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb98c0597e0 con 0x22fec10 2023-10-11T09:50:42.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.305+0000 7fb97bfff700 1 -- 172.21.15.78:0/1089767655 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 6724+0+0 (secure 0 0 0) 0x7fb998003e10 con 0x22fec10 2023-10-11T09:50:42.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.305+0000 7fb97bfff700 1 -- 172.21.15.78:0/1089767655 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb998025ab0 con 0x22fec10 2023-10-11T09:50:42.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.305+0000 7fb9a6490740 10 client.8646 fetch_fsmap learned FSMap version 91 2023-10-11T09:50:42.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.305+0000 7fb9a6490740 1 -- 172.21.15.78:0/1089767655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x249d820 con 0x22fec10 2023-10-11T09:50:42.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.305+0000 7fb97bfff700 1 -- 172.21.15.78:0/1089767655 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb98c0597e0 con 0x22fec10 2023-10-11T09:50:42.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.305+0000 7fb97bfff700 1 -- 172.21.15.78:0/1089767655 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 91) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fb998025c10 con 0x22fec10 2023-10-11T09:50:42.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.305+0000 7fb9a6490740 10 client.8646 fetch_fsmap finished waiting for FSMap version 91 2023-10-11T09:50:42.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.305+0000 7fb9a6490740 10 client.8646 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2023-10-11T09:50:42.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.305+0000 7fb9a6490740 20 client.8646 populate_metadata read hostname 'smithi078' 2023-10-11T09:50:42.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.305+0000 7fb9a6490740 1 --2- 172.21.15.78:0/1089767655 >> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] conn(0x25936e0 0x259afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:50:42.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.305+0000 7fb9a6490740 4 client.8646 mds_command: new command op to 8550 tid=0[{"prefix": "scrub status"}] 2023-10-11T09:50:42.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.305+0000 7fb9a6490740 1 -- 172.21.15.78:0/1089767655 --> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x2356610 con 0x25936e0 2023-10-11T09:50:42.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.305+0000 7fb993fff700 1 --2- 172.21.15.78:0/1089767655 >> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] conn(0x25936e0 0x259afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:50:42.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.305+0000 7fb993fff700 1 --2- 172.21.15.78:0/1089767655 >> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] conn(0x25936e0 0x259afa0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dab86def99ad233 server_cookie=37df2c115a106e2b in_seq=0 out_seq=0 2023-10-11T09:50:42.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.305+0000 7fb97bfff700 10 client.8646 ms_handle_connect on v2:172.21.15.78:6836/2869034036 2023-10-11T09:50:42.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.305+0000 7fb97bfff700 1 -- 172.21.15.78:0/1089767655 <== mds.0 v2:172.21.15.78:6836/2869034036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x2356610 con 0x25936e0 2023-10-11T09:50:42.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.305+0000 7fb97bfff700 10 client.8646 handle_command_reply: tid=0 2023-10-11T09:50:42.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.305+0000 7fb9a6490740 1 client.8646 shutdown 2023-10-11T09:50:42.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.305+0000 7fb9a6490740 1 -- 172.21.15.78:0/1089767655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x22fec10 msgr2=0x2612090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:42.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.305+0000 7fb9a6490740 1 --2- 172.21.15.78:0/1089767655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x22fec10 0x2612090 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fb998007ec0 tx=0x7fb998000ed0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.305+0000 7fb9a6490740 1 -- 172.21.15.78:0/1089767655 shutdown_connections 2023-10-11T09:50:42.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.305+0000 7fb9a6490740 1 --2- 172.21.15.78:0/1089767655 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2593ba0 0x2592cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.305+0000 7fb9a6490740 1 --2- 172.21.15.78:0/1089767655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x23498c0 0x25927b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.305+0000 7fb9a6490740 1 --2- 172.21.15.78:0/1089767655 >> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] conn(0x25936e0 0x259afa0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.305+0000 7fb9a6490740 1 --2- 172.21.15.78:0/1089767655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x22fec10 0x2612090 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.305+0000 7fb97bfff700 0 client.8646 ms_handle_reset on v2:172.21.15.78:6836/2869034036 2023-10-11T09:50:42.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.305+0000 7fb9a6490740 1 -- 172.21.15.78:0/1089767655 >> 172.21.15.78:0/1089767655 conn(0x22fefe0 msgr2=0x25c4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:42.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.305+0000 7fb9a6490740 1 -- 172.21.15.78:0/1089767655 shutdown_connections 2023-10-11T09:50:42.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.305+0000 7fb9a6490740 1 -- 172.21.15.78:0/1089767655 wait complete. 2023-10-11T09:50:42.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.309+0000 7fb9a6490740 20 client.8646 trim_cache size 0 max 16384 2023-10-11T09:50:42.324 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:50:42.324 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-11T09:50:42.324 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2023-10-11T09:50:42.325 INFO:teuthology.orchestra.run.smithi078.stdout: "5fff9c3f-bfeb-41e7-9d9e-0d94063a73bc": { 2023-10-11T09:50:42.325 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2023-10-11T09:50:42.325 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "5fff9c3f-bfeb-41e7-9d9e-0d94063a73bc", 2023-10-11T09:50:42.325 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,repair" 2023-10-11T09:50:42.325 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:50:42.326 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:50:42.326 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:50:42.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.309+0000 7fb9a1f97700 1 -- 172.21.15.78:0/1642463424 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb988065ed0 msgr2=0x7fb988068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:42.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.309+0000 7fb9a1f97700 1 --2- 172.21.15.78:0/1642463424 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb988065ed0 0x7fb988068380 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb9980011e0 tx=0x7fb998009040 comp rx=0 tx=0).stop 2023-10-11T09:50:42.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.309+0000 7fb9a1f97700 1 -- 172.21.15.78:0/1642463424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb99c0ce240 msgr2=0x7fb99c1518c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:50:42.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.309+0000 7fb9a1f97700 1 --2- 172.21.15.78:0/1642463424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb99c0ce240 0x7fb99c1518c0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fb98c013040 tx=0x7fb98c004770 comp rx=0 tx=0).stop 2023-10-11T09:50:42.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.309+0000 7fb9a1f97700 1 -- 172.21.15.78:0/1642463424 shutdown_connections 2023-10-11T09:50:42.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.309+0000 7fb9a1f97700 1 --2- 172.21.15.78:0/1642463424 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb988065ed0 0x7fb988068380 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.309+0000 7fb9a1f97700 1 --2- 172.21.15.78:0/1642463424 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb99c157130 0x7fb99c15be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.309+0000 7fb9a1f97700 1 --2- 172.21.15.78:0/1642463424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb99c0ce240 0x7fb99c1518c0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.309+0000 7fb9a1f97700 1 --2- 172.21.15.78:0/1642463424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb99c06d400 0x7fb99c1f3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:50:42.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.309+0000 7fb9a1f97700 1 -- 172.21.15.78:0/1642463424 >> 172.21.15.78:0/1642463424 conn(0x7fb99c06a060 msgr2=0x7fb99c0acdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:50:42.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.309+0000 7fb9a1f97700 1 -- 172.21.15.78:0/1642463424 shutdown_connections 2023-10-11T09:50:42.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:50:42.309+0000 7fb9a1f97700 1 -- 172.21.15.78:0/1642463424 wait complete. 2023-10-11T09:50:42.339 INFO:tasks.cephfs.filesystem:scrub status for tag:5fff9c3f-bfeb-41e7-9d9e-0d94063a73bc - {'path': '/', 'tag': '5fff9c3f-bfeb-41e7-9d9e-0d94063a73bc', 'options': 'recursive,repair'} 2023-10-11T09:51:00.765 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:51:00.774 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:51:12.340 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 scrub status 2023-10-11T09:51:12.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.513+0000 7fed2afd5700 1 Processor -- start 2023-10-11T09:51:12.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.513+0000 7fed2afd5700 1 -- start start 2023-10-11T09:51:12.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.517+0000 7fed2afd5700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed2415c900 0x7fed240b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:12.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.517+0000 7fed2afd5700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fed240ce7f0 0x7fed240cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:12.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.517+0000 7fed2afd5700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed2406fde0 con 0x7fed240ad290 2023-10-11T09:51:12.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.517+0000 7fed2afd5700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed240705f0 con 0x7fed240ce7f0 2023-10-11T09:51:12.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.517+0000 7fed2afd5700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed240cf1b0 con 0x7fed2415c900 2023-10-11T09:51:12.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.517+0000 7fed297d2700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fed240ad290 0x7fed24071a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:44402/0 (socket says 172.21.15.78:44402) 2023-10-11T09:51:12.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.517+0000 7fed297d2700 1 -- 172.21.15.78:0/1626470988 learned_addr learned my addr 172.21.15.78:0/1626470988 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:51:12.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.517+0000 7fed29fd3700 1 --2- 172.21.15.78:0/1626470988 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed2415c900 0x7fed240b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:12.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.517+0000 7fed2a7d4700 1 --2- 172.21.15.78:0/1626470988 >> v2:172.21.15.133:3300/0 conn(0x7fed240ce7f0 0x7fed240cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:12.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.517+0000 7fed28fd1700 1 -- 172.21.15.78:0/1626470988 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1474879892 0 0) 0x7fed2406fde0 con 0x7fed240ad290 2023-10-11T09:51:12.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.517+0000 7fed28fd1700 1 -- 172.21.15.78:0/1626470988 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed0c0035d0 con 0x7fed240ad290 2023-10-11T09:51:12.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.517+0000 7fed29fd3700 1 -- 172.21.15.78:0/1626470988 >> v2:172.21.15.133:3300/0 conn(0x7fed240ce7f0 msgr2=0x7fed240cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:12.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.517+0000 7fed29fd3700 1 --2- 172.21.15.78:0/1626470988 >> v2:172.21.15.133:3300/0 conn(0x7fed240ce7f0 0x7fed240cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:12.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.517+0000 7fed29fd3700 1 -- 172.21.15.78:0/1626470988 >> v1:172.21.15.78:6789/0 conn(0x7fed240ad290 legacy=0x7fed24071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:12.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.517+0000 7fed29fd3700 1 -- 172.21.15.78:0/1626470988 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed240ae000 con 0x7fed2415c900 2023-10-11T09:51:12.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.517+0000 7fed29fd3700 1 --2- 172.21.15.78:0/1626470988 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed2415c900 0x7fed240b1a10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fed14009880 tx=0x7fed14009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf3481db43a440fb server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:12.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.521+0000 7fed28fd1700 1 -- 172.21.15.78:0/1626470988 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed14011830 con 0x7fed2415c900 2023-10-11T09:51:12.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.521+0000 7fed28fd1700 1 -- 172.21.15.78:0/1626470988 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed14011990 con 0x7fed2415c900 2023-10-11T09:51:12.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.521+0000 7fed28fd1700 1 -- 172.21.15.78:0/1626470988 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed14011d20 con 0x7fed2415c900 2023-10-11T09:51:12.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.521+0000 7fed2afd5700 1 -- 172.21.15.78:0/1626470988 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed2415c900 msgr2=0x7fed240b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:12.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.521+0000 7fed2afd5700 1 --2- 172.21.15.78:0/1626470988 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed2415c900 0x7fed240b1a10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fed14009880 tx=0x7fed14009b90 comp rx=0 tx=0).stop 2023-10-11T09:51:12.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.521+0000 7fed2afd5700 1 -- 172.21.15.78:0/1626470988 shutdown_connections 2023-10-11T09:51:12.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.521+0000 7fed2afd5700 1 --2- 172.21.15.78:0/1626470988 >> v2:172.21.15.133:3300/0 conn(0x7fed240ce7f0 0x7fed240cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:12.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.521+0000 7fed2afd5700 1 --2- 172.21.15.78:0/1626470988 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed2415c900 0x7fed240b1a10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:12.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.521+0000 7fed2afd5700 1 -- 172.21.15.78:0/1626470988 >> 172.21.15.78:0/1626470988 conn(0x7fed2406a060 msgr2=0x7fed2406a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:12.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.521+0000 7fed2afd5700 1 -- 172.21.15.78:0/1626470988 shutdown_connections 2023-10-11T09:51:12.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.521+0000 7fed2afd5700 1 -- 172.21.15.78:0/1626470988 wait complete. 2023-10-11T09:51:12.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.525+0000 7fed2afd5700 1 Processor -- start 2023-10-11T09:51:12.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.525+0000 7fed2afd5700 1 -- start start 2023-10-11T09:51:12.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.525+0000 7fed2afd5700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed240ad290 0x7fed2414d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:12.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.525+0000 7fed2afd5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed240ce7f0 0x7fed2415e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:12.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.525+0000 7fed2afd5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed2415c900 0x7fed240afc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:12.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.525+0000 7fed2afd5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2415f0f0 con 0x7fed2415c900 2023-10-11T09:51:12.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.525+0000 7fed2afd5700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2415f3d0 con 0x7fed240ce7f0 2023-10-11T09:51:12.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.525+0000 7fed2afd5700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed2415f6b0 con 0x7fed240ad290 2023-10-11T09:51:12.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.525+0000 7fed2a7d4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed2415c900 0x7fed240afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:12.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.525+0000 7fed29fd3700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed240ad290 0x7fed2414d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:12.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.525+0000 7fed297d2700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed240ce7f0 0x7fed2415e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:12.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.525+0000 7fed29fd3700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed240ad290 0x7fed2414d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:58282/0 (socket says 172.21.15.78:58282) 2023-10-11T09:51:12.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.525+0000 7fed2a7d4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed2415c900 0x7fed240afc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43812/0 (socket says 172.21.15.78:43812) 2023-10-11T09:51:12.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.525+0000 7fed29fd3700 1 -- 172.21.15.78:0/3568684162 learned_addr learned my addr 172.21.15.78:0/3568684162 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:12.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.525+0000 7fed29fd3700 1 -- 172.21.15.78:0/3568684162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed240ce7f0 msgr2=0x7fed2415e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:12.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.525+0000 7fed29fd3700 1 --2- 172.21.15.78:0/3568684162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed240ce7f0 0x7fed2415e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:12.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.525+0000 7fed29fd3700 1 -- 172.21.15.78:0/3568684162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed2415c900 msgr2=0x7fed240afc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:12.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.525+0000 7fed29fd3700 1 --2- 172.21.15.78:0/3568684162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed2415c900 0x7fed240afc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:12.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.525+0000 7fed29fd3700 1 -- 172.21.15.78:0/3568684162 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed140094b0 con 0x7fed240ad290 2023-10-11T09:51:12.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.529+0000 7fed29fd3700 1 --2- 172.21.15.78:0/3568684162 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed240ad290 0x7fed2414d830 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fed14009bd0 tx=0x7fed14004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:12.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.529+0000 7fed227fc700 1 -- 172.21.15.78:0/3568684162 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed14004520 con 0x7fed240ad290 2023-10-11T09:51:12.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.529+0000 7fed227fc700 1 -- 172.21.15.78:0/3568684162 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1401f070 con 0x7fed240ad290 2023-10-11T09:51:12.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.529+0000 7fed227fc700 1 -- 172.21.15.78:0/3568684162 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed14003900 con 0x7fed240ad290 2023-10-11T09:51:12.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.529+0000 7fed2afd5700 1 -- 172.21.15.78:0/3568684162 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed2415f990 con 0x7fed240ad290 2023-10-11T09:51:12.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.529+0000 7fed2afd5700 1 -- 172.21.15.78:0/3568684162 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed2414dd70 con 0x7fed240ad290 2023-10-11T09:51:12.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.533+0000 7fed227fc700 1 -- 172.21.15.78:0/3568684162 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fed14003a60 con 0x7fed240ad290 2023-10-11T09:51:12.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.533+0000 7fed227fc700 1 --2- 172.21.15.78:0/3568684162 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fed10065f20 0x7fed100683d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:12.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.533+0000 7fed227fc700 1 -- 172.21.15.78:0/3568684162 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6724+0+0 (secure 0 0 0) 0x7fed14021070 con 0x7fed240ad290 2023-10-11T09:51:12.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.533+0000 7fed297d2700 1 --2- 172.21.15.78:0/3568684162 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fed10065f20 0x7fed100683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:12.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.533+0000 7fed297d2700 1 --2- 172.21.15.78:0/3568684162 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fed10065f20 0x7fed100683d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fed24157870 tx=0x7fed18007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:12.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.537+0000 7fed2f4c9740 1 Processor -- start 2023-10-11T09:51:12.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.537+0000 7fed2f4c9740 1 -- start start 2023-10-11T09:51:12.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.537+0000 7fed2f4c9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb0c10 0xf465a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:12.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.537+0000 7fed2f4c9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xcb0840 0xf67d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:12.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.537+0000 7fed2f4c9740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xcfb8c0 0xf682d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:12.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.537+0000 7fed2f4c9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0xe90630 con 0xcb0c10 2023-10-11T09:51:12.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.537+0000 7fed2f4c9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0xe8ddc0 con 0xcb0840 2023-10-11T09:51:12.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.537+0000 7fed2f4c9740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0xe896b0 con 0xcfb8c0 2023-10-11T09:51:12.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.537+0000 7fed29fd3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb0c10 0xf465a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:12.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.537+0000 7fed29fd3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb0c10 0xf465a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43814/0 (socket says 172.21.15.78:43814) 2023-10-11T09:51:12.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.537+0000 7fed29fd3700 1 -- 172.21.15.78:0/2361368321 learned_addr learned my addr 172.21.15.78:0/2361368321 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:12.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.537+0000 7fed297d2700 1 --2- 172.21.15.78:0/2361368321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xcb0840 0xf67d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:12.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.537+0000 7fed2a7d4700 1 --2- 172.21.15.78:0/2361368321 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xcfb8c0 0xf682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:12.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.537+0000 7fed29fd3700 1 -- 172.21.15.78:0/2361368321 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xcfb8c0 msgr2=0xf682d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:12.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.537+0000 7fed29fd3700 1 --2- 172.21.15.78:0/2361368321 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xcfb8c0 0xf682d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:12.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.537+0000 7fed29fd3700 1 -- 172.21.15.78:0/2361368321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xcb0840 msgr2=0xf67d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:12.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.537+0000 7fed29fd3700 1 --2- 172.21.15.78:0/2361368321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xcb0840 0xf67d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:12.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.537+0000 7fed29fd3700 1 -- 172.21.15.78:0/2361368321 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xeaf5b0 con 0xcb0c10 2023-10-11T09:51:12.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.537+0000 7fed297d2700 1 --2- 172.21.15.78:0/2361368321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xcb0840 0xf67d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:51:12.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.537+0000 7fed29fd3700 1 --2- 172.21.15.78:0/2361368321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb0c10 0xf465a0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fed14052440 tx=0x7fed14009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e70a6b2f1df3036 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:12.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.537+0000 7fed0b7fe700 1 -- 172.21.15.78:0/2361368321 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed14059d70 con 0xcb0c10 2023-10-11T09:51:12.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.537+0000 7fed0b7fe700 1 -- 172.21.15.78:0/2361368321 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1407e030 con 0xcb0c10 2023-10-11T09:51:12.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.537+0000 7fed0b7fe700 1 -- 172.21.15.78:0/2361368321 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1405df20 con 0xcb0c10 2023-10-11T09:51:12.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.537+0000 7fed2f4c9740 1 -- 172.21.15.78:0/2361368321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb0c10 msgr2=0xf465a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:12.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.537+0000 7fed2f4c9740 1 --2- 172.21.15.78:0/2361368321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb0c10 0xf465a0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fed14052440 tx=0x7fed14009240 comp rx=0 tx=0).stop 2023-10-11T09:51:12.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.541+0000 7fed2f4c9740 1 -- 172.21.15.78:0/2361368321 shutdown_connections 2023-10-11T09:51:12.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.541+0000 7fed2f4c9740 1 --2- 172.21.15.78:0/2361368321 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xcfb8c0 0xf682d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:12.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.541+0000 7fed2f4c9740 1 --2- 172.21.15.78:0/2361368321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xcb0840 0xf67d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:12.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.541+0000 7fed2f4c9740 1 --2- 172.21.15.78:0/2361368321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb0c10 0xf465a0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:12.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.541+0000 7fed2f4c9740 1 -- 172.21.15.78:0/2361368321 >> 172.21.15.78:0/2361368321 conn(0xcb0fe0 msgr2=0xf49350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:12.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.541+0000 7fed2f4c9740 1 -- 172.21.15.78:0/2361368321 shutdown_connections 2023-10-11T09:51:12.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.541+0000 7fed2f4c9740 1 -- 172.21.15.78:0/2361368321 wait complete. 2023-10-11T09:51:12.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.541+0000 7fed2f4c9740 1 -- start start 2023-10-11T09:51:12.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.541+0000 7fed2f4c9740 1 Processor -- start 2023-10-11T09:51:12.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.545+0000 7fed2f4c9740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xcb0840 0xf3efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:12.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.545+0000 7fed2f4c9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb0c10 0xf448a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:12.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.545+0000 7fed2f4c9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xcfb8c0 0xf52880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:12.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.545+0000 7fed2f4c9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0xe4cc10 con 0xcb0c10 2023-10-11T09:51:12.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.545+0000 7fed2f4c9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0xe4c8b0 con 0xcfb8c0 2023-10-11T09:51:12.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.545+0000 7fed2f4c9740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0xd35c50 con 0xcb0840 2023-10-11T09:51:12.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.545+0000 7fed2f4c9740 10 client.0 _ll_register_callbacks cb 0xe82ff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:51:12.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.545+0000 7fed297d2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb0c10 0xf448a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:12.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.545+0000 7fed297d2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb0c10 0xf448a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43820/0 (socket says 172.21.15.78:43820) 2023-10-11T09:51:12.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.545+0000 7fed297d2700 1 -- 172.21.15.78:0/3647911374 learned_addr learned my addr 172.21.15.78:0/3647911374 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:12.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.545+0000 7fed29fd3700 1 --2- 172.21.15.78:0/3647911374 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xcb0840 0xf3efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:12.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.545+0000 7fed2a7d4700 1 --2- 172.21.15.78:0/3647911374 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xcfb8c0 0xf52880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:12.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.545+0000 7fed297d2700 1 -- 172.21.15.78:0/3647911374 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xcb0840 msgr2=0xf3efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:12.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.545+0000 7fed297d2700 1 --2- 172.21.15.78:0/3647911374 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xcb0840 0xf3efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:12.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.545+0000 7fed297d2700 1 -- 172.21.15.78:0/3647911374 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xcfb8c0 msgr2=0xf52880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:12.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.545+0000 7fed297d2700 1 --2- 172.21.15.78:0/3647911374 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xcfb8c0 0xf52880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:12.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.545+0000 7fed297d2700 1 -- 172.21.15.78:0/3647911374 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed2414dd70 con 0xcb0c10 2023-10-11T09:51:12.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.545+0000 7fed297d2700 1 --2- 172.21.15.78:0/3647911374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb0c10 0xf448a0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fed18000f60 tx=0x7fed18001150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50fa7728936184de server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:12.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.545+0000 7fed08ff9700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:51:12.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.545+0000 7fed08ff9700 1 -- 172.21.15.78:0/3647911374 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed18025070 con 0xcb0c10 2023-10-11T09:51:12.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.545+0000 7fed08ff9700 1 -- 172.21.15.78:0/3647911374 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 6724+0+0 (secure 0 0 0) 0x7fed1801b040 con 0xcb0c10 2023-10-11T09:51:12.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.545+0000 7fed2f4c9740 1 -- 172.21.15.78:0/3647911374 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xe82730 con 0xcb0c10 2023-10-11T09:51:12.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.545+0000 7fed08ff9700 1 -- 172.21.15.78:0/3647911374 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed18001500 con 0xcb0c10 2023-10-11T09:51:12.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.545+0000 7fed2f4c9740 10 client.8658 fetch_fsmap learned FSMap version 91 2023-10-11T09:51:12.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.545+0000 7fed2f4c9740 1 -- 172.21.15.78:0/3647911374 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0xcfcb10 con 0xcb0c10 2023-10-11T09:51:12.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.545+0000 7fed08ff9700 1 -- 172.21.15.78:0/3647911374 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1802f020 con 0xcb0c10 2023-10-11T09:51:12.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.545+0000 7fed08ff9700 1 -- 172.21.15.78:0/3647911374 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 91) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fed18028020 con 0xcb0c10 2023-10-11T09:51:12.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.545+0000 7fed2f4c9740 10 client.8658 fetch_fsmap finished waiting for FSMap version 91 2023-10-11T09:51:12.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.545+0000 7fed2f4c9740 10 client.8658 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2023-10-11T09:51:12.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.545+0000 7fed2f4c9740 20 client.8658 populate_metadata read hostname 'smithi078' 2023-10-11T09:51:12.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.545+0000 7fed2f4c9740 1 --2- 172.21.15.78:0/3647911374 >> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] conn(0xf52dc0 0xf251a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:12.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.545+0000 7fed2f4c9740 4 client.8658 mds_command: new command op to 8550 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-11T09:51:12.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.545+0000 7fed2f4c9740 1 -- 172.21.15.78:0/3647911374 --> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0xe82450 con 0xf52dc0 2023-10-11T09:51:12.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.549+0000 7fed29fd3700 1 --2- 172.21.15.78:0/3647911374 >> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] conn(0xf52dc0 0xf251a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:12.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.549+0000 7fed08ff9700 10 client.8658 ms_handle_connect on v2:172.21.15.78:6836/2869034036 2023-10-11T09:51:12.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.549+0000 7fed29fd3700 1 --2- 172.21.15.78:0/3647911374 >> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] conn(0xf52dc0 0xf251a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fee4128b636e966a server_cookie=e939d73903de22d8 in_seq=0 out_seq=0 2023-10-11T09:51:12.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.549+0000 7fed08ff9700 1 -- 172.21.15.78:0/3647911374 <== mds.0 v2:172.21.15.78:6836/2869034036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0xe82450 con 0xf52dc0 2023-10-11T09:51:12.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.549+0000 7fed08ff9700 10 client.8658 handle_command_reply: tid=0 2023-10-11T09:51:12.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.549+0000 7fed2f4c9740 1 client.8658 shutdown 2023-10-11T09:51:12.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.549+0000 7fed2f4c9740 1 -- 172.21.15.78:0/3647911374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb0c10 msgr2=0xf448a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:12.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.549+0000 7fed2f4c9740 1 --2- 172.21.15.78:0/3647911374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb0c10 0xf448a0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fed18000f60 tx=0x7fed18001150 comp rx=0 tx=0).stop 2023-10-11T09:51:12.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.549+0000 7fed2f4c9740 1 -- 172.21.15.78:0/3647911374 shutdown_connections 2023-10-11T09:51:12.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.549+0000 7fed2f4c9740 1 --2- 172.21.15.78:0/3647911374 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xcfb8c0 0xf52880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:12.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.549+0000 7fed2f4c9740 1 --2- 172.21.15.78:0/3647911374 >> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] conn(0xf52dc0 0xf251a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:12.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.549+0000 7fed2f4c9740 1 --2- 172.21.15.78:0/3647911374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb0c10 0xf448a0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:12.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.549+0000 7fed2f4c9740 1 --2- 172.21.15.78:0/3647911374 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xcb0840 0xf3efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:12.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.549+0000 7fed08ff9700 0 client.8658 ms_handle_reset on v2:172.21.15.78:6836/2869034036 2023-10-11T09:51:12.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.549+0000 7fed2f4c9740 1 -- 172.21.15.78:0/3647911374 >> 172.21.15.78:0/3647911374 conn(0xcb0fe0 msgr2=0xf49750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:12.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.549+0000 7fed2f4c9740 1 -- 172.21.15.78:0/3647911374 shutdown_connections 2023-10-11T09:51:12.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.549+0000 7fed2f4c9740 1 -- 172.21.15.78:0/3647911374 wait complete. 2023-10-11T09:51:12.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.549+0000 7fed2f4c9740 20 client.8658 trim_cache size 0 max 16384 2023-10-11T09:51:12.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.569+0000 7fed2f4c9740 1 Processor -- start 2023-10-11T09:51:12.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.569+0000 7fed2f4c9740 1 -- start start 2023-10-11T09:51:12.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.569+0000 7fed2f4c9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xcb0840 0xf23aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:12.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.569+0000 7fed2f4c9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb0c10 0xf23fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:12.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.569+0000 7fed2f4c9740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xcfb8c0 0xf66a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:12.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.569+0000 7fed2f4c9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0xf43ab0 con 0xcb0c10 2023-10-11T09:51:12.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.569+0000 7fed2f4c9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0xe86ab0 con 0xcb0840 2023-10-11T09:51:12.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.569+0000 7fed2f4c9740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0xf6d470 con 0xcfb8c0 2023-10-11T09:51:12.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.569+0000 7fed297d2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb0c10 0xf23fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:12.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.569+0000 7fed297d2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb0c10 0xf23fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43826/0 (socket says 172.21.15.78:43826) 2023-10-11T09:51:12.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.569+0000 7fed297d2700 1 -- 172.21.15.78:0/3819380873 learned_addr learned my addr 172.21.15.78:0/3819380873 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:12.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.569+0000 7fed29fd3700 1 --2- 172.21.15.78:0/3819380873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xcb0840 0xf23aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:12.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.569+0000 7fed297d2700 1 -- 172.21.15.78:0/3819380873 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xcfb8c0 msgr2=0xf66a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:12.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.569+0000 7fed297d2700 1 --2- 172.21.15.78:0/3819380873 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xcfb8c0 0xf66a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:12.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.569+0000 7fed297d2700 1 -- 172.21.15.78:0/3819380873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xcb0840 msgr2=0xf23aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:12.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.569+0000 7fed297d2700 1 --2- 172.21.15.78:0/3819380873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xcb0840 0xf23aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:12.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.569+0000 7fed297d2700 1 -- 172.21.15.78:0/3819380873 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed240b0500 con 0xcb0c10 2023-10-11T09:51:12.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.569+0000 7fed29fd3700 1 --2- 172.21.15.78:0/3819380873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xcb0840 0xf23aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:51:12.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.573+0000 7fed297d2700 1 --2- 172.21.15.78:0/3819380873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb0c10 0xf23fe0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fed18005fd0 tx=0x7fed18015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe16a65709e1790 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:12.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.573+0000 7fed0bfff700 1 -- 172.21.15.78:0/3819380873 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed18025040 con 0xcb0c10 2023-10-11T09:51:12.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.573+0000 7fed0bfff700 1 -- 172.21.15.78:0/3819380873 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1802f070 con 0xcb0c10 2023-10-11T09:51:12.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.573+0000 7fed0bfff700 1 -- 172.21.15.78:0/3819380873 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed18001910 con 0xcb0c10 2023-10-11T09:51:12.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.573+0000 7fed2f4c9740 1 -- 172.21.15.78:0/3819380873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb0c10 msgr2=0xf23fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:12.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.573+0000 7fed2f4c9740 1 --2- 172.21.15.78:0/3819380873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb0c10 0xf23fe0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fed18005fd0 tx=0x7fed18015000 comp rx=0 tx=0).stop 2023-10-11T09:51:12.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.573+0000 7fed2f4c9740 1 -- 172.21.15.78:0/3819380873 shutdown_connections 2023-10-11T09:51:12.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.573+0000 7fed2f4c9740 1 --2- 172.21.15.78:0/3819380873 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xcfb8c0 0xf66a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:12.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.573+0000 7fed2f4c9740 1 --2- 172.21.15.78:0/3819380873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcb0c10 0xf23fe0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:12.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.573+0000 7fed2f4c9740 1 --2- 172.21.15.78:0/3819380873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xcb0840 0xf23aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:12.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.573+0000 7fed2f4c9740 1 -- 172.21.15.78:0/3819380873 >> 172.21.15.78:0/3819380873 conn(0xcb0fe0 msgr2=0xf25710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:12.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.573+0000 7fed2f4c9740 1 -- 172.21.15.78:0/3819380873 shutdown_connections 2023-10-11T09:51:12.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.573+0000 7fed2f4c9740 1 -- 172.21.15.78:0/3819380873 wait complete. 2023-10-11T09:51:12.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.573+0000 7fed2f4c9740 1 -- start start 2023-10-11T09:51:12.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.573+0000 7fed2f4c9740 1 Processor -- start 2023-10-11T09:51:12.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.577+0000 7fed2f4c9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xcb0840 0xf6b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:12.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.577+0000 7fed2f4c9740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xcb0c10 0xfc3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:12.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.577+0000 7fed2f4c9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcfb8c0 0xf68e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:12.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.577+0000 7fed2f4c9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0xf6a3e0 con 0xcfb8c0 2023-10-11T09:51:12.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.577+0000 7fed2f4c9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0xf6a680 con 0xcb0840 2023-10-11T09:51:12.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.577+0000 7fed2f4c9740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0xf6b760 con 0xcb0c10 2023-10-11T09:51:12.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.577+0000 7fed2f4c9740 10 client.0 _ll_register_callbacks cb 0xe87e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:51:12.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.577+0000 7fed2a7d4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcfb8c0 0xf68e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:12.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.577+0000 7fed2a7d4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcfb8c0 0xf68e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43840/0 (socket says 172.21.15.78:43840) 2023-10-11T09:51:12.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.577+0000 7fed2a7d4700 1 -- 172.21.15.78:0/3950242558 learned_addr learned my addr 172.21.15.78:0/3950242558 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:12.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.577+0000 7fed2a7d4700 1 -- 172.21.15.78:0/3950242558 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xcb0c10 msgr2=0xfc3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:12.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.577+0000 7fed2a7d4700 1 --2- 172.21.15.78:0/3950242558 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xcb0c10 0xfc3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:12.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.577+0000 7fed2a7d4700 1 -- 172.21.15.78:0/3950242558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xcb0840 msgr2=0xf6b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:12.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.577+0000 7fed2a7d4700 1 --2- 172.21.15.78:0/3950242558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xcb0840 0xf6b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:12.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.577+0000 7fed2a7d4700 1 -- 172.21.15.78:0/3950242558 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0xcfcb10 con 0xcfb8c0 2023-10-11T09:51:12.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.577+0000 7fed2a7d4700 1 --2- 172.21.15.78:0/3950242558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcfb8c0 0xf68e30 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fed1c004330 tx=0x7fed1c004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893be2cbd06009a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:12.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.577+0000 7fed08ff9700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:51:12.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.577+0000 7fed08ff9700 1 -- 172.21.15.78:0/3950242558 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1c00e040 con 0xcfb8c0 2023-10-11T09:51:12.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.577+0000 7fed08ff9700 1 -- 172.21.15.78:0/3950242558 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 6724+0+0 (secure 0 0 0) 0x7fed1c0178e0 con 0xcfb8c0 2023-10-11T09:51:12.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.577+0000 7fed2f4c9740 1 -- 172.21.15.78:0/3950242558 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed1407e030 con 0xcfb8c0 2023-10-11T09:51:12.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.577+0000 7fed08ff9700 1 -- 172.21.15.78:0/3950242558 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed1c017aa0 con 0xcfb8c0 2023-10-11T09:51:12.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.577+0000 7fed2f4c9740 10 client.8664 fetch_fsmap learned FSMap version 91 2023-10-11T09:51:12.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.577+0000 7fed2f4c9740 1 -- 172.21.15.78:0/3950242558 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0xe4f820 con 0xcfb8c0 2023-10-11T09:51:12.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.577+0000 7fed08ff9700 1 -- 172.21.15.78:0/3950242558 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1407e030 con 0xcfb8c0 2023-10-11T09:51:12.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.577+0000 7fed08ff9700 1 -- 172.21.15.78:0/3950242558 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 91) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fed1c01d020 con 0xcfb8c0 2023-10-11T09:51:12.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.577+0000 7fed2f4c9740 10 client.8664 fetch_fsmap finished waiting for FSMap version 91 2023-10-11T09:51:12.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.577+0000 7fed2f4c9740 10 client.8664 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2023-10-11T09:51:12.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.577+0000 7fed2f4c9740 20 client.8664 populate_metadata read hostname 'smithi078' 2023-10-11T09:51:12.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.577+0000 7fed2f4c9740 1 --2- 172.21.15.78:0/3950242558 >> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] conn(0xf69370 0xf66ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:12.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.577+0000 7fed2f4c9740 4 client.8664 mds_command: new command op to 8550 tid=0[{"prefix": "scrub status"}] 2023-10-11T09:51:12.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.577+0000 7fed2f4c9740 1 -- 172.21.15.78:0/3950242558 --> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0xd08610 con 0xf69370 2023-10-11T09:51:12.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.577+0000 7fed29fd3700 1 --2- 172.21.15.78:0/3950242558 >> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] conn(0xf69370 0xf66ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:12.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.577+0000 7fed29fd3700 1 --2- 172.21.15.78:0/3950242558 >> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] conn(0xf69370 0xf66ac0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=720cf9191b398975 server_cookie=8c8cff094745af50 in_seq=0 out_seq=0 2023-10-11T09:51:12.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.577+0000 7fed08ff9700 10 client.8664 ms_handle_connect on v2:172.21.15.78:6836/2869034036 2023-10-11T09:51:12.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.581+0000 7fed08ff9700 1 -- 172.21.15.78:0/3950242558 <== mds.0 v2:172.21.15.78:6836/2869034036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0xd08610 con 0xf69370 2023-10-11T09:51:12.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.581+0000 7fed08ff9700 10 client.8664 handle_command_reply: tid=0 2023-10-11T09:51:12.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.581+0000 7fed2f4c9740 1 client.8664 shutdown 2023-10-11T09:51:12.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.581+0000 7fed2f4c9740 1 -- 172.21.15.78:0/3950242558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcfb8c0 msgr2=0xf68e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:12.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.581+0000 7fed2f4c9740 1 --2- 172.21.15.78:0/3950242558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcfb8c0 0xf68e30 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fed1c004330 tx=0x7fed1c004410 comp rx=0 tx=0).stop 2023-10-11T09:51:12.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.581+0000 7fed2f4c9740 1 -- 172.21.15.78:0/3950242558 shutdown_connections 2023-10-11T09:51:12.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.581+0000 7fed2f4c9740 1 --2- 172.21.15.78:0/3950242558 >> [v2:172.21.15.78:6836/2869034036,v1:172.21.15.78:6837/2869034036] conn(0xf69370 0xf66ac0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:12.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.581+0000 7fed2f4c9740 1 --2- 172.21.15.78:0/3950242558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0xcfb8c0 0xf68e30 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:12.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.581+0000 7fed2f4c9740 1 --2- 172.21.15.78:0/3950242558 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0xcb0c10 0xfc3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:12.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.581+0000 7fed2f4c9740 1 --2- 172.21.15.78:0/3950242558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0xcb0840 0xf6b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:12.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.581+0000 7fed2f4c9740 1 -- 172.21.15.78:0/3950242558 >> 172.21.15.78:0/3950242558 conn(0xcb0fe0 msgr2=0xf23aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:12.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.581+0000 7fed08ff9700 0 client.8664 ms_handle_reset on v2:172.21.15.78:6836/2869034036 2023-10-11T09:51:12.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.581+0000 7fed2f4c9740 1 -- 172.21.15.78:0/3950242558 shutdown_connections 2023-10-11T09:51:12.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.581+0000 7fed2f4c9740 1 -- 172.21.15.78:0/3950242558 wait complete. 2023-10-11T09:51:12.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.581+0000 7fed2f4c9740 20 client.8664 trim_cache size 0 max 16384 2023-10-11T09:51:12.599 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:51:12.600 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2023-10-11T09:51:12.600 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2023-10-11T09:51:12.600 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:51:12.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.581+0000 7fed2afd5700 1 -- 172.21.15.78:0/3568684162 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fed10065f20 msgr2=0x7fed100683d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:12.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.581+0000 7fed2afd5700 1 --2- 172.21.15.78:0/3568684162 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fed10065f20 0x7fed100683d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fed24157870 tx=0x7fed18007de0 comp rx=0 tx=0).stop 2023-10-11T09:51:12.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.581+0000 7fed2afd5700 1 -- 172.21.15.78:0/3568684162 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed240ad290 msgr2=0x7fed2414d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:12.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.581+0000 7fed2afd5700 1 --2- 172.21.15.78:0/3568684162 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed240ad290 0x7fed2414d830 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fed14009bd0 tx=0x7fed14004770 comp rx=0 tx=0).stop 2023-10-11T09:51:12.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.581+0000 7fed2afd5700 1 -- 172.21.15.78:0/3568684162 shutdown_connections 2023-10-11T09:51:12.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.581+0000 7fed2afd5700 1 --2- 172.21.15.78:0/3568684162 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fed10065f20 0x7fed100683d0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:12.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.581+0000 7fed2afd5700 1 --2- 172.21.15.78:0/3568684162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed2415c900 0x7fed240afc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:12.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.581+0000 7fed2afd5700 1 --2- 172.21.15.78:0/3568684162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed240ce7f0 0x7fed2415e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:12.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.581+0000 7fed2afd5700 1 --2- 172.21.15.78:0/3568684162 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed240ad290 0x7fed2414d830 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:12.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.581+0000 7fed2afd5700 1 -- 172.21.15.78:0/3568684162 >> 172.21.15.78:0/3568684162 conn(0x7fed2406a060 msgr2=0x7fed240aca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:12.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.581+0000 7fed2afd5700 1 -- 172.21.15.78:0/3568684162 shutdown_connections 2023-10-11T09:51:12.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:12.581+0000 7fed2afd5700 1 -- 172.21.15.78:0/3568684162 wait complete. 2023-10-11T09:51:12.611 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-11T09:51:12.611 INFO:tasks.cephfs.mount:Validating 1048576 bytes from stripey/flushed_file_0 2023-10-11T09:51:12.612 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:51:12.612 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:51:12.613 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_0" 2023-10-11T09:51:12.613 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:51:12.613 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:51:12.614 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 1048576: 2023-10-11T09:51:12.616 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:51:12.616 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 1048576 2023-10-11T09:51:12.616 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:51:12.617 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:51:12.617 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:51:12.618 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:51:12.618 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:51:12.618 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:51:13.318 INFO:tasks.cephfs.mount:Validating 524288 bytes from stripey/flushed_file_1 2023-10-11T09:51:13.319 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:51:13.319 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:51:13.320 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_1" 2023-10-11T09:51:13.320 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:51:13.321 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:51:13.321 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 524288: 2023-10-11T09:51:13.322 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:51:13.322 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 524288 2023-10-11T09:51:13.322 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:51:13.323 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:51:13.323 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:51:13.323 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:51:13.324 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:51:13.324 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:51:13.858 INFO:tasks.cephfs.mount:Validating 917504 bytes from stripey/flushed_file_2 2023-10-11T09:51:13.859 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:51:13.860 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:51:13.860 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_2" 2023-10-11T09:51:13.860 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:51:13.861 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:51:13.861 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 917504: 2023-10-11T09:51:13.861 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:51:13.862 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 917504 2023-10-11T09:51:13.862 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:51:13.863 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:51:13.863 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:51:13.863 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:51:13.864 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:51:13.864 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:51:14.603 INFO:tasks.cephfs.mount:Validating 917503 bytes from stripey/flushed_file_3 2023-10-11T09:51:14.603 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:51:14.604 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:51:14.604 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_3" 2023-10-11T09:51:14.605 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:51:14.605 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:51:14.605 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 917503: 2023-10-11T09:51:14.606 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:51:14.606 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 917503 2023-10-11T09:51:14.606 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:51:14.607 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:51:14.607 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:51:14.607 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:51:14.608 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:51:14.608 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:51:15.342 INFO:tasks.cephfs.mount:Validating 1441792 bytes from stripey/flushed_file_4 2023-10-11T09:51:15.342 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:51:15.343 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:51:15.343 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_4" 2023-10-11T09:51:15.344 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:51:15.344 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:51:15.344 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 1441792: 2023-10-11T09:51:15.345 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:51:15.345 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 1441792 2023-10-11T09:51:15.345 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:51:15.346 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:51:15.346 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:51:15.346 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:51:15.347 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:51:15.347 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:51:16.279 INFO:tasks.cephfs.mount:Validating 1441793 bytes from stripey/flushed_file_5 2023-10-11T09:51:16.280 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:51:16.280 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:51:16.280 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_5" 2023-10-11T09:51:16.281 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:51:16.281 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:51:16.282 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 1441793: 2023-10-11T09:51:16.282 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:51:16.282 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 1441793 2023-10-11T09:51:16.283 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:51:16.283 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:51:16.284 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:51:16.284 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:51:16.284 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:51:16.285 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:51:17.219 INFO:tasks.cephfs.mount:Validating 1572864 bytes from stripey/flushed_file_6 2023-10-11T09:51:17.220 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:51:17.220 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:51:17.221 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_6" 2023-10-11T09:51:17.221 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:51:17.222 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:51:17.222 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 1572864: 2023-10-11T09:51:17.222 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:51:17.223 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 1572864 2023-10-11T09:51:17.223 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:51:17.224 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:51:17.224 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:51:17.224 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:51:17.225 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:51:17.225 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:51:18.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:51:18.161 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo ls lost+found) 2023-10-11T09:51:18.249 INFO:teuthology.orchestra.run.smithi078.stdout:100000001fc 2023-10-11T09:51:18.252 INFO:tasks.cephfs.mount:Validating 524288 bytes from lost+found/100000001fc 2023-10-11T09:51:18.253 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:51:18.253 DEBUG:teuthology.orchestra.run.smithi078:> import zlib 2023-10-11T09:51:18.254 DEBUG:teuthology.orchestra.run.smithi078:> path = "/home/ubuntu/cephtest/mnt.0/lost+found/100000001fc" 2023-10-11T09:51:18.254 DEBUG:teuthology.orchestra.run.smithi078:> with open(path, '"'"'r'"'"') as f: 2023-10-11T09:51:18.254 DEBUG:teuthology.orchestra.run.smithi078:> bytes = f.read() 2023-10-11T09:51:18.255 DEBUG:teuthology.orchestra.run.smithi078:> if len(bytes) != 524288: 2023-10-11T09:51:18.255 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-10-11T09:51:18.255 DEBUG:teuthology.orchestra.run.smithi078:> len(bytes), 524288 2023-10-11T09:51:18.256 DEBUG:teuthology.orchestra.run.smithi078:> )) 2023-10-11T09:51:18.256 DEBUG:teuthology.orchestra.run.smithi078:> for i, b in enumerate(bytes): 2023-10-11T09:51:18.256 DEBUG:teuthology.orchestra.run.smithi078:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-10-11T09:51:18.257 DEBUG:teuthology.orchestra.run.smithi078:> if b != chr(val): 2023-10-11T09:51:18.257 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-10-11T09:51:18.258 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:51:18.454 INFO:teuthology.orchestra.run.smithi078.stderr:Traceback (most recent call last): 2023-10-11T09:51:18.454 INFO:teuthology.orchestra.run.smithi078.stderr: File "", line 13, in 2023-10-11T09:51:18.454 INFO:teuthology.orchestra.run.smithi078.stderr:RuntimeError: Bad data at offset 65536 2023-10-11T09:51:18.582 INFO:teuthology.orchestra.run.smithi078.stderr:daemon-helper: command failed with exit status 1 2023-10-11T09:51:18.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T09:51:18.599 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-11T09:51:18.600 DEBUG:teuthology.orchestra.run.smithi078:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-11T09:51:18.689 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-11T09:51:18.733 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-11T09:51:18.733 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:51:18.734 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:51:18.754 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:51:18.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.956+0000 7f8e696c8700 1 Processor -- start 2023-10-11T09:51:18.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.956+0000 7f8e696c8700 1 -- start start 2023-10-11T09:51:18.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.960+0000 7f8e696c8700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f8e640ca940 0x7f8e64165370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:18.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.960+0000 7f8e696c8700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e640ae0d0 con 0x7f8e640add00 2023-10-11T09:51:18.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.960+0000 7f8e696c8700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e64071f20 con 0x7f8e6415e120 2023-10-11T09:51:18.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.960+0000 7f8e696c8700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e6415a9f0 con 0x7f8e640ca940 2023-10-11T09:51:18.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.960+0000 7f8e637fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f8e640ca940 0x7f8e64165370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:18.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.960+0000 7f8e62ffd700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f8e640add00 0x7f8e64070c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33616/0 (socket says 172.21.15.78:33616) 2023-10-11T09:51:18.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.960+0000 7f8e62ffd700 1 -- 172.21.15.78:0/2061694494 learned_addr learned my addr 172.21.15.78:0/2061694494 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:51:18.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.960+0000 7f8e627fc700 1 -- 172.21.15.78:0/2061694494 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2422306771 0 0) 0x7f8e640ae0d0 con 0x7f8e640add00 2023-10-11T09:51:18.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.960+0000 7f8e627fc700 1 -- 172.21.15.78:0/2061694494 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e500035d0 con 0x7f8e640add00 2023-10-11T09:51:18.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.960+0000 7f8e637fe700 1 -- 172.21.15.78:0/2061694494 >> v1:172.21.15.133:6789/0 conn(0x7f8e6415e120 legacy=0x7f8e640ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:18.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.960+0000 7f8e637fe700 1 -- 172.21.15.78:0/2061694494 >> v1:172.21.15.78:6789/0 conn(0x7f8e640add00 legacy=0x7f8e64070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:18.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.960+0000 7f8e637fe700 1 -- 172.21.15.78:0/2061694494 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e64071680 con 0x7f8e640ca940 2023-10-11T09:51:18.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.960+0000 7f8e637fe700 1 --2- 172.21.15.78:0/2061694494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e640ca940 0x7f8e64165370 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c009d90 tx=0x7f8e5c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d7c5f850ae6ee6 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:18.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.960+0000 7f8e627fc700 1 -- 172.21.15.78:0/2061694494 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c00e070 con 0x7f8e640ca940 2023-10-11T09:51:18.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.960+0000 7f8e627fc700 1 -- 172.21.15.78:0/2061694494 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e5c00bb20 con 0x7f8e640ca940 2023-10-11T09:51:18.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.960+0000 7f8e627fc700 1 -- 172.21.15.78:0/2061694494 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c012430 con 0x7f8e640ca940 2023-10-11T09:51:18.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.960+0000 7f8e696c8700 1 -- 172.21.15.78:0/2061694494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e640ca940 msgr2=0x7f8e64165370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:18.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.960+0000 7f8e696c8700 1 --2- 172.21.15.78:0/2061694494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e640ca940 0x7f8e64165370 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c009d90 tx=0x7f8e5c00b330 comp rx=0 tx=0).stop 2023-10-11T09:51:18.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.960+0000 7f8e696c8700 1 -- 172.21.15.78:0/2061694494 shutdown_connections 2023-10-11T09:51:18.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.960+0000 7f8e696c8700 1 --2- 172.21.15.78:0/2061694494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e640ca940 0x7f8e64165370 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:18.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.960+0000 7f8e696c8700 1 -- 172.21.15.78:0/2061694494 >> 172.21.15.78:0/2061694494 conn(0x7f8e640cc070 msgr2=0x7f8e640ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:18.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.960+0000 7f8e696c8700 1 -- 172.21.15.78:0/2061694494 shutdown_connections 2023-10-11T09:51:18.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.960+0000 7f8e696c8700 1 -- 172.21.15.78:0/2061694494 wait complete. 2023-10-11T09:51:18.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.964+0000 7f8e696c8700 1 Processor -- start 2023-10-11T09:51:18.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.964+0000 7f8e696c8700 1 -- start start 2023-10-11T09:51:18.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.968+0000 7f8e696c8700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8e640add00 0x7f8e64147ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:18.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.968+0000 7f8e696c8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e640ca940 0x7f8e640b3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:18.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.968+0000 7f8e696c8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e6415e120 0x7f8e640b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:18.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.968+0000 7f8e696c8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e64148430 con 0x7f8e6415e120 2023-10-11T09:51:18.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.968+0000 7f8e696c8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e641eea20 con 0x7f8e640ca940 2023-10-11T09:51:18.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.968+0000 7f8e696c8700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e641ee740 con 0x7f8e640add00 2023-10-11T09:51:18.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.968+0000 7f8e637fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e6415e120 0x7f8e640b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:18.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.968+0000 7f8e637fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e6415e120 0x7f8e640b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45462/0 (socket says 172.21.15.78:45462) 2023-10-11T09:51:18.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.968+0000 7f8e637fe700 1 -- 172.21.15.78:0/1874676069 learned_addr learned my addr 172.21.15.78:0/1874676069 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:18.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.968+0000 7f8e62ffd700 1 --2- 172.21.15.78:0/1874676069 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8e640add00 0x7f8e64147ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:18.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.968+0000 7f8e5a7fc700 1 --2- 172.21.15.78:0/1874676069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e640ca940 0x7f8e640b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:18.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.968+0000 7f8e637fe700 1 -- 172.21.15.78:0/1874676069 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8e640add00 msgr2=0x7f8e64147ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:18.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.968+0000 7f8e637fe700 1 --2- 172.21.15.78:0/1874676069 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8e640add00 0x7f8e64147ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:18.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.968+0000 7f8e637fe700 1 -- 172.21.15.78:0/1874676069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e640ca940 msgr2=0x7f8e640b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:18.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.968+0000 7f8e637fe700 1 --2- 172.21.15.78:0/1874676069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e640ca940 0x7f8e640b3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:18.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.968+0000 7f8e637fe700 1 -- 172.21.15.78:0/1874676069 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e5c009a70 con 0x7f8e6415e120 2023-10-11T09:51:18.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.968+0000 7f8e637fe700 1 --2- 172.21.15.78:0/1874676069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e6415e120 0x7f8e640b3d40 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f8e640669c0 tx=0x7f8e5c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:18.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.968+0000 7f8e5bfff700 1 -- 172.21.15.78:0/1874676069 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c012430 con 0x7f8e6415e120 2023-10-11T09:51:18.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.968+0000 7f8e5bfff700 1 -- 172.21.15.78:0/1874676069 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e5c020070 con 0x7f8e6415e120 2023-10-11T09:51:18.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.968+0000 7f8e696c8700 1 -- 172.21.15.78:0/1874676069 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e640c3270 con 0x7f8e6415e120 2023-10-11T09:51:18.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.968+0000 7f8e5bfff700 1 -- 172.21.15.78:0/1874676069 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c004280 con 0x7f8e6415e120 2023-10-11T09:51:18.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.968+0000 7f8e696c8700 1 -- 172.21.15.78:0/1874676069 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e640c3830 con 0x7f8e6415e120 2023-10-11T09:51:18.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.972+0000 7f8e696c8700 1 -- 172.21.15.78:0/1874676069 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e64070000 con 0x7f8e6415e120 2023-10-11T09:51:18.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.972+0000 7f8e5bfff700 1 -- 172.21.15.78:0/1874676069 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f8e5c0043e0 con 0x7f8e6415e120 2023-10-11T09:51:18.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.972+0000 7f8e5bfff700 1 --2- 172.21.15.78:0/1874676069 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8e44065ed0 0x7f8e44068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:18.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.972+0000 7f8e5bfff700 1 -- 172.21.15.78:0/1874676069 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f8e5c022070 con 0x7f8e6415e120 2023-10-11T09:51:18.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.972+0000 7f8e62ffd700 1 --2- 172.21.15.78:0/1874676069 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8e44065ed0 0x7f8e44068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:18.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.976+0000 7f8e62ffd700 1 --2- 172.21.15.78:0/1874676069 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8e44065ed0 0x7f8e44068380 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f8e54007fd0 tx=0x7f8e54007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:18.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:18.976+0000 7f8e5bfff700 1 -- 172.21.15.78:0/1874676069 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f8e5c052a60 con 0x7f8e6415e120 2023-10-11T09:51:19.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.152+0000 7f8e696c8700 1 -- 172.21.15.78:0/1874676069 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8e640c3a20 con 0x7f8e6415e120 2023-10-11T09:51:19.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.156+0000 7f8e5bfff700 1 -- 172.21.15.78:0/1874676069 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+20012 (secure 0 0 0) 0x7f8e5c056080 con 0x7f8e6415e120 2023-10-11T09:51:19.166 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:19.166 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":114,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:50:34.654747+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:49:49.297597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-11T09:49:49.955104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data_extra","create_time":"2023-10-11T09:49:59.915561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6836/2876755099":"2023-10-12T09:50:34.623416+0000","172.21.15.78:6837/1390949105":"2023-10-12T09:50:28.599447+0000","172.21.15.78:6836/1390949105":"2023-10-12T09:50:28.599447+0000","172.21.15.78:6837/2876755099":"2023-10-12T09:50:34.623416+0000","172.21.15.78:6837/1286798022":"2023-10-12T09:50:27.559697+0000","172.21.15.78:6836/1286798022":"2023-10-12T09:50:27.559697+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:51:19.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.160+0000 7f8e696c8700 1 -- 172.21.15.78:0/1874676069 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8e44065ed0 msgr2=0x7f8e44068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:19.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.160+0000 7f8e696c8700 1 --2- 172.21.15.78:0/1874676069 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8e44065ed0 0x7f8e44068380 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f8e54007fd0 tx=0x7f8e54007de0 comp rx=0 tx=0).stop 2023-10-11T09:51:19.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.160+0000 7f8e696c8700 1 -- 172.21.15.78:0/1874676069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e6415e120 msgr2=0x7f8e640b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:19.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.160+0000 7f8e696c8700 1 --2- 172.21.15.78:0/1874676069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e6415e120 0x7f8e640b3d40 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f8e640669c0 tx=0x7f8e5c004af0 comp rx=0 tx=0).stop 2023-10-11T09:51:19.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.160+0000 7f8e696c8700 1 -- 172.21.15.78:0/1874676069 shutdown_connections 2023-10-11T09:51:19.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.160+0000 7f8e696c8700 1 --2- 172.21.15.78:0/1874676069 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8e44065ed0 0x7f8e44068380 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:19.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.160+0000 7f8e696c8700 1 --2- 172.21.15.78:0/1874676069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e6415e120 0x7f8e640b3d40 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:19.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.160+0000 7f8e696c8700 1 --2- 172.21.15.78:0/1874676069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e640ca940 0x7f8e640b3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:19.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.160+0000 7f8e696c8700 1 --2- 172.21.15.78:0/1874676069 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8e640add00 0x7f8e64147ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:19.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.160+0000 7f8e696c8700 1 -- 172.21.15.78:0/1874676069 >> 172.21.15.78:0/1874676069 conn(0x7f8e640cc070 msgr2=0x7f8e64070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:19.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.160+0000 7f8e696c8700 1 -- 172.21.15.78:0/1874676069 shutdown_connections 2023-10-11T09:51:19.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.160+0000 7f8e696c8700 1 -- 172.21.15.78:0/1874676069 wait complete. 2023-10-11T09:51:19.189 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T09:49:49.297597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T09:49:49.955104+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-10-11T09:49:59.915561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:51:19.190 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:51:19.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.348+0000 7f435d2db700 1 Processor -- start 2023-10-11T09:51:19.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.348+0000 7f435d2db700 1 -- start start 2023-10-11T09:51:19.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.352+0000 7f435d2db700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f43580ad290 0x7f43580b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:19.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.352+0000 7f435d2db700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f43580ce7f0 0x7f43580cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:19.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.352+0000 7f435d2db700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f435806fde0 con 0x7f435815c900 2023-10-11T09:51:19.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.352+0000 7f435d2db700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43580705f0 con 0x7f43580ad290 2023-10-11T09:51:19.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.352+0000 7f435d2db700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43580cf1b0 con 0x7f43580ce7f0 2023-10-11T09:51:19.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.352+0000 7f43577fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f43580ad290 0x7f43580b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:19.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.352+0000 7f43577fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f43580ad290 0x7f43580b1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45476/0 (socket says 172.21.15.78:45476) 2023-10-11T09:51:19.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.352+0000 7f43577fe700 1 -- 172.21.15.78:0/4140993141 learned_addr learned my addr 172.21.15.78:0/4140993141 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:19.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.352+0000 7f43577fe700 1 -- 172.21.15.78:0/4140993141 >> v2:172.21.15.133:3300/0 conn(0x7f43580ce7f0 msgr2=0x7f43580cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:19.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.352+0000 7f43577fe700 1 --2- 172.21.15.78:0/4140993141 >> v2:172.21.15.133:3300/0 conn(0x7f43580ce7f0 0x7f43580cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:19.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.352+0000 7f43577fe700 1 -- 172.21.15.78:0/4140993141 >> v1:172.21.15.78:6789/0 conn(0x7f435815c900 legacy=0x7f4358071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:19.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.352+0000 7f43577fe700 1 -- 172.21.15.78:0/4140993141 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43580ae000 con 0x7f43580ad290 2023-10-11T09:51:19.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.352+0000 7f43577fe700 1 --2- 172.21.15.78:0/4140993141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43580ad290 0x7f43580b1a10 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f43580721c0 tx=0x7f434c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31fe231fff703dd4 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:19.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.352+0000 7f4356ffd700 1 -- 172.21.15.78:0/4140993141 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434c004230 con 0x7f43580ad290 2023-10-11T09:51:19.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.352+0000 7f4356ffd700 1 -- 172.21.15.78:0/4140993141 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f434c004390 con 0x7f43580ad290 2023-10-11T09:51:19.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.352+0000 7f4356ffd700 1 -- 172.21.15.78:0/4140993141 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434c0036c0 con 0x7f43580ad290 2023-10-11T09:51:19.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.352+0000 7f435d2db700 1 -- 172.21.15.78:0/4140993141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43580ad290 msgr2=0x7f43580b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:19.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.352+0000 7f435d2db700 1 --2- 172.21.15.78:0/4140993141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43580ad290 0x7f43580b1a10 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f43580721c0 tx=0x7f434c009bf0 comp rx=0 tx=0).stop 2023-10-11T09:51:19.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.356+0000 7f435d2db700 1 -- 172.21.15.78:0/4140993141 shutdown_connections 2023-10-11T09:51:19.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.356+0000 7f435d2db700 1 --2- 172.21.15.78:0/4140993141 >> v2:172.21.15.133:3300/0 conn(0x7f43580ce7f0 0x7f43580cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:19.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.356+0000 7f435d2db700 1 --2- 172.21.15.78:0/4140993141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43580ad290 0x7f43580b1a10 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:19.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.356+0000 7f435d2db700 1 -- 172.21.15.78:0/4140993141 >> 172.21.15.78:0/4140993141 conn(0x7f435806a060 msgr2=0x7f435806a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:19.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.356+0000 7f435d2db700 1 -- 172.21.15.78:0/4140993141 shutdown_connections 2023-10-11T09:51:19.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.356+0000 7f435d2db700 1 -- 172.21.15.78:0/4140993141 wait complete. 2023-10-11T09:51:19.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.356+0000 7f435d2db700 1 Processor -- start 2023-10-11T09:51:19.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.360+0000 7f435d2db700 1 -- start start 2023-10-11T09:51:19.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.360+0000 7f435d2db700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43580ad290 0x7f435813e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:19.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.360+0000 7f435d2db700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43580ce7f0 0x7f435813e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:19.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.360+0000 7f435d2db700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f435815c900 0x7f4358155d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:19.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.360+0000 7f435d2db700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43580b02c0 con 0x7f435815c900 2023-10-11T09:51:19.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.360+0000 7f435d2db700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435813e590 con 0x7f43580ad290 2023-10-11T09:51:19.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.360+0000 7f435d2db700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43580af6d0 con 0x7f43580ce7f0 2023-10-11T09:51:19.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.360+0000 7f435cada700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f435815c900 0x7f4358155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:19.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.360+0000 7f435cada700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f435815c900 0x7f4358155d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45482/0 (socket says 172.21.15.78:45482) 2023-10-11T09:51:19.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.360+0000 7f435cada700 1 -- 172.21.15.78:0/479941998 learned_addr learned my addr 172.21.15.78:0/479941998 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:19.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.360+0000 7f43577fe700 1 --2- 172.21.15.78:0/479941998 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43580ce7f0 0x7f435813e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:19.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.360+0000 7f4357fff700 1 --2- 172.21.15.78:0/479941998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43580ad290 0x7f435813e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:19.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.360+0000 7f435cada700 1 -- 172.21.15.78:0/479941998 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43580ce7f0 msgr2=0x7f435813e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:19.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.360+0000 7f435cada700 1 --2- 172.21.15.78:0/479941998 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43580ce7f0 0x7f435813e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:19.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.360+0000 7f435cada700 1 -- 172.21.15.78:0/479941998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43580ad290 msgr2=0x7f435813e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:19.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.360+0000 7f435cada700 1 --2- 172.21.15.78:0/479941998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43580ad290 0x7f435813e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:19.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.360+0000 7f435cada700 1 -- 172.21.15.78:0/479941998 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f434c009570 con 0x7f435815c900 2023-10-11T09:51:19.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.360+0000 7f4357fff700 1 --2- 172.21.15.78:0/479941998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43580ad290 0x7f435813e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:51:19.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.360+0000 7f435cada700 1 --2- 172.21.15.78:0/479941998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f435815c900 0x7f4358155d50 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f435000b9a0 tx=0x7f435000bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:19.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.360+0000 7f4354ff9700 1 -- 172.21.15.78:0/479941998 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435000d8e0 con 0x7f435815c900 2023-10-11T09:51:19.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.360+0000 7f4354ff9700 1 -- 172.21.15.78:0/479941998 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f435000da60 con 0x7f435815c900 2023-10-11T09:51:19.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.360+0000 7f435d2db700 1 -- 172.21.15.78:0/479941998 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f435814dc00 con 0x7f435815c900 2023-10-11T09:51:19.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.360+0000 7f4354ff9700 1 -- 172.21.15.78:0/479941998 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435000de10 con 0x7f435815c900 2023-10-11T09:51:19.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.364+0000 7f435d2db700 1 -- 172.21.15.78:0/479941998 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43580afc00 con 0x7f435815c900 2023-10-11T09:51:19.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.364+0000 7f435d2db700 1 -- 172.21.15.78:0/479941998 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43580b0400 con 0x7f435815c900 2023-10-11T09:51:19.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.364+0000 7f4354ff9700 1 -- 172.21.15.78:0/479941998 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f4350015420 con 0x7f435815c900 2023-10-11T09:51:19.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.364+0000 7f4354ff9700 1 --2- 172.21.15.78:0/479941998 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4344065ed0 0x7f4344068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:19.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.368+0000 7f4354ff9700 1 -- 172.21.15.78:0/479941998 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f4350011070 con 0x7f435815c900 2023-10-11T09:51:19.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.368+0000 7f4357fff700 1 --2- 172.21.15.78:0/479941998 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4344065ed0 0x7f4344068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:19.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.368+0000 7f4357fff700 1 --2- 172.21.15.78:0/479941998 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4344065ed0 0x7f4344068380 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f4348004000 tx=0x7f4348009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:19.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.368+0000 7f4354ff9700 1 -- 172.21.15.78:0/479941998 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f435004cfa0 con 0x7f435815c900 2023-10-11T09:51:19.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.540+0000 7f435d2db700 1 -- 172.21.15.78:0/479941998 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4358065d40 con 0x7f435815c900 2023-10-11T09:51:19.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.540+0000 7f4354ff9700 1 -- 172.21.15.78:0/479941998 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v114) v1 ==== 93+0+10 (secure 0 0 0) 0x7f43580b0400 con 0x7f435815c900 2023-10-11T09:51:19.549 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:51:19.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.540+0000 7f435d2db700 1 -- 172.21.15.78:0/479941998 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4344065ed0 msgr2=0x7f4344068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:19.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.540+0000 7f435d2db700 1 --2- 172.21.15.78:0/479941998 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4344065ed0 0x7f4344068380 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f4348004000 tx=0x7f4348009040 comp rx=0 tx=0).stop 2023-10-11T09:51:19.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.540+0000 7f435d2db700 1 -- 172.21.15.78:0/479941998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f435815c900 msgr2=0x7f4358155d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:19.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.540+0000 7f435d2db700 1 --2- 172.21.15.78:0/479941998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f435815c900 0x7f4358155d50 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f435000b9a0 tx=0x7f435000bcb0 comp rx=0 tx=0).stop 2023-10-11T09:51:19.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.540+0000 7f435d2db700 1 -- 172.21.15.78:0/479941998 shutdown_connections 2023-10-11T09:51:19.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.540+0000 7f435d2db700 1 --2- 172.21.15.78:0/479941998 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4344065ed0 0x7f4344068380 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:19.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.540+0000 7f435d2db700 1 --2- 172.21.15.78:0/479941998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f435815c900 0x7f4358155d50 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:19.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.540+0000 7f435d2db700 1 --2- 172.21.15.78:0/479941998 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43580ce7f0 0x7f435813e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:19.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.540+0000 7f435d2db700 1 --2- 172.21.15.78:0/479941998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43580ad290 0x7f435813e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:19.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.540+0000 7f435d2db700 1 -- 172.21.15.78:0/479941998 >> 172.21.15.78:0/479941998 conn(0x7f435806a060 msgr2=0x7f435815a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:19.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.544+0000 7f435d2db700 1 -- 172.21.15.78:0/479941998 shutdown_connections 2023-10-11T09:51:19.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.544+0000 7f435d2db700 1 -- 172.21.15.78:0/479941998 wait complete. 2023-10-11T09:51:19.572 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:51:19.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.740+0000 7f1dba984700 1 Processor -- start 2023-10-11T09:51:19.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.740+0000 7f1dba984700 1 -- start start 2023-10-11T09:51:19.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.744+0000 7f1dba984700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f1db4146fe0 0x7f1db406bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:19.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.744+0000 7f1dba984700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1db40b36f0 0x7f1db41629e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:19.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.744+0000 7f1dba984700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1db406ccb0 con 0x7f1db4161d90 2023-10-11T09:51:19.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.744+0000 7f1dba984700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1db406c410 con 0x7f1db4146fe0 2023-10-11T09:51:19.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.744+0000 7f1dba984700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1db406c6f0 con 0x7f1db40b36f0 2023-10-11T09:51:19.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.744+0000 7f1db9181700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f1db4161d90 0x7f1db40b2fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33636/0 (socket says 172.21.15.78:33636) 2023-10-11T09:51:19.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.744+0000 7f1db9181700 1 -- 172.21.15.78:0/1153379848 learned_addr learned my addr 172.21.15.78:0/1153379848 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:51:19.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.744+0000 7f1db9982700 1 --2- 172.21.15.78:0/1153379848 >> v2:172.21.15.78:3300/0 conn(0x7f1db4146fe0 0x7f1db406bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:19.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.744+0000 7f1dba183700 1 --2- 172.21.15.78:0/1153379848 >> v2:172.21.15.133:3300/0 conn(0x7f1db40b36f0 0x7f1db41629e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:19.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.744+0000 7f1db8980700 1 -- 172.21.15.78:0/1153379848 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 481119923 0 0) 0x7f1da4001110 con 0x7f1db4161d90 2023-10-11T09:51:19.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.744+0000 7f1db8980700 1 -- 172.21.15.78:0/1153379848 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1da00035d0 con 0x7f1db4161d90 2023-10-11T09:51:19.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.744+0000 7f1db9982700 1 -- 172.21.15.78:0/1153379848 >> v2:172.21.15.133:3300/0 conn(0x7f1db40b36f0 msgr2=0x7f1db41629e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:19.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.744+0000 7f1db9982700 1 --2- 172.21.15.78:0/1153379848 >> v2:172.21.15.133:3300/0 conn(0x7f1db40b36f0 0x7f1db41629e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:19.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.744+0000 7f1db9982700 1 -- 172.21.15.78:0/1153379848 >> v1:172.21.15.78:6789/0 conn(0x7f1db4161d90 legacy=0x7f1db40b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:19.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.744+0000 7f1db9982700 1 -- 172.21.15.78:0/1153379848 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db406c9d0 con 0x7f1db4146fe0 2023-10-11T09:51:19.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.744+0000 7f1db9982700 1 --2- 172.21.15.78:0/1153379848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1db4146fe0 0x7f1db406bbe0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f1db40b40c0 tx=0x7f1db0009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5442d97490c7b6b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:19.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.744+0000 7f1db8980700 1 -- 172.21.15.78:0/1153379848 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db0011830 con 0x7f1db4146fe0 2023-10-11T09:51:19.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.744+0000 7f1db8980700 1 -- 172.21.15.78:0/1153379848 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1db0011990 con 0x7f1db4146fe0 2023-10-11T09:51:19.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.744+0000 7f1db8980700 1 -- 172.21.15.78:0/1153379848 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db0011d00 con 0x7f1db4146fe0 2023-10-11T09:51:19.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.744+0000 7f1dba984700 1 -- 172.21.15.78:0/1153379848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1db4146fe0 msgr2=0x7f1db406bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:19.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.744+0000 7f1dba984700 1 --2- 172.21.15.78:0/1153379848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1db4146fe0 0x7f1db406bbe0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f1db40b40c0 tx=0x7f1db0009b30 comp rx=0 tx=0).stop 2023-10-11T09:51:19.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.744+0000 7f1dba984700 1 -- 172.21.15.78:0/1153379848 shutdown_connections 2023-10-11T09:51:19.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.748+0000 7f1dba984700 1 --2- 172.21.15.78:0/1153379848 >> v2:172.21.15.133:3300/0 conn(0x7f1db40b36f0 0x7f1db41629e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:19.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.748+0000 7f1dba984700 1 --2- 172.21.15.78:0/1153379848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1db4146fe0 0x7f1db406bbe0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:19.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.748+0000 7f1dba984700 1 -- 172.21.15.78:0/1153379848 >> 172.21.15.78:0/1153379848 conn(0x7f1db406a060 msgr2=0x7f1db406a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:19.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.748+0000 7f1dba984700 1 -- 172.21.15.78:0/1153379848 shutdown_connections 2023-10-11T09:51:19.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.748+0000 7f1dba984700 1 -- 172.21.15.78:0/1153379848 wait complete. 2023-10-11T09:51:19.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.752+0000 7f1dba984700 1 Processor -- start 2023-10-11T09:51:19.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.752+0000 7f1dba984700 1 -- start start 2023-10-11T09:51:19.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.752+0000 7f1dba984700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1db40b36f0 0x7f1db40ab850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:19.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.752+0000 7f1dba984700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1db4146fe0 0x7f1db40cec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:19.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.752+0000 7f1dba984700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1db4161d90 0x7f1db4145e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:19.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.752+0000 7f1dba984700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db4155340 con 0x7f1db4161d90 2023-10-11T09:51:19.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.752+0000 7f1dba984700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db40abe30 con 0x7f1db4146fe0 2023-10-11T09:51:19.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.752+0000 7f1dba984700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1db40ac110 con 0x7f1db40b36f0 2023-10-11T09:51:19.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.752+0000 7f1dba183700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1db4161d90 0x7f1db4145e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:19.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.752+0000 7f1dba183700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1db4161d90 0x7f1db4145e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45492/0 (socket says 172.21.15.78:45492) 2023-10-11T09:51:19.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.752+0000 7f1dba183700 1 -- 172.21.15.78:0/2421645407 learned_addr learned my addr 172.21.15.78:0/2421645407 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:19.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.752+0000 7f1db9982700 1 --2- 172.21.15.78:0/2421645407 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1db40b36f0 0x7f1db40ab850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:19.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.752+0000 7f1db9181700 1 --2- 172.21.15.78:0/2421645407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1db4146fe0 0x7f1db40cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:19.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.752+0000 7f1dba183700 1 -- 172.21.15.78:0/2421645407 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1db40b36f0 msgr2=0x7f1db40ab850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:19.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.752+0000 7f1dba183700 1 --2- 172.21.15.78:0/2421645407 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1db40b36f0 0x7f1db40ab850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:19.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.752+0000 7f1dba183700 1 -- 172.21.15.78:0/2421645407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1db4146fe0 msgr2=0x7f1db40cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:19.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.752+0000 7f1dba183700 1 --2- 172.21.15.78:0/2421645407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1db4146fe0 0x7f1db40cec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:19.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.752+0000 7f1dba183700 1 -- 172.21.15.78:0/2421645407 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db00094b0 con 0x7f1db4161d90 2023-10-11T09:51:19.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.752+0000 7f1db9181700 1 --2- 172.21.15.78:0/2421645407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1db4146fe0 0x7f1db40cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:51:19.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.752+0000 7f1dba183700 1 --2- 172.21.15.78:0/2421645407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1db4161d90 0x7f1db4145e40 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f1dac007e40 tx=0x7f1dac00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:19.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.752+0000 7f1daa7fc700 1 -- 172.21.15.78:0/2421645407 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dac00eea0 con 0x7f1db4161d90 2023-10-11T09:51:19.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.752+0000 7f1daa7fc700 1 -- 172.21.15.78:0/2421645407 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dac014420 con 0x7f1db4161d90 2023-10-11T09:51:19.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.752+0000 7f1dba984700 1 -- 172.21.15.78:0/2421645407 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1db40a8560 con 0x7f1db4161d90 2023-10-11T09:51:19.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.752+0000 7f1daa7fc700 1 -- 172.21.15.78:0/2421645407 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dac0147b0 con 0x7f1db4161d90 2023-10-11T09:51:19.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.752+0000 7f1dba984700 1 -- 172.21.15.78:0/2421645407 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1db4146380 con 0x7f1db4161d90 2023-10-11T09:51:19.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.756+0000 7f1dba984700 1 -- 172.21.15.78:0/2421645407 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1db40cb050 con 0x7f1db4161d90 2023-10-11T09:51:19.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.756+0000 7f1daa7fc700 1 -- 172.21.15.78:0/2421645407 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f1dac014910 con 0x7f1db4161d90 2023-10-11T09:51:19.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.760+0000 7f1daa7fc700 1 --2- 172.21.15.78:0/2421645407 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1d98065ed0 0x7f1d98068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:19.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.760+0000 7f1daa7fc700 1 -- 172.21.15.78:0/2421645407 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f1dac0843f0 con 0x7f1db4161d90 2023-10-11T09:51:19.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.760+0000 7f1db9982700 1 --2- 172.21.15.78:0/2421645407 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1d98065ed0 0x7f1d98068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:19.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.760+0000 7f1db9982700 1 --2- 172.21.15.78:0/2421645407 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1d98065ed0 0x7f1d98068380 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1db0000c00 tx=0x7f1db0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:19.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.764+0000 7f1daa7fc700 1 -- 172.21.15.78:0/2421645407 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f1dac051680 con 0x7f1db4161d90 2023-10-11T09:51:19.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.936+0000 7f1dba984700 1 -- 172.21.15.78:0/2421645407 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1db40a8a80 con 0x7f1db4161d90 2023-10-11T09:51:19.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.936+0000 7f1daa7fc700 1 -- 172.21.15.78:0/2421645407 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v114) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1db40a8a80 con 0x7f1db4161d90 2023-10-11T09:51:19.948 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:51:19.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.940+0000 7f1dba984700 1 -- 172.21.15.78:0/2421645407 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1d98065ed0 msgr2=0x7f1d98068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:19.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.940+0000 7f1dba984700 1 --2- 172.21.15.78:0/2421645407 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1d98065ed0 0x7f1d98068380 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1db0000c00 tx=0x7f1db0008030 comp rx=0 tx=0).stop 2023-10-11T09:51:19.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.940+0000 7f1dba984700 1 -- 172.21.15.78:0/2421645407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1db4161d90 msgr2=0x7f1db4145e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:19.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.940+0000 7f1dba984700 1 --2- 172.21.15.78:0/2421645407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1db4161d90 0x7f1db4145e40 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f1dac007e40 tx=0x7f1dac00e3c0 comp rx=0 tx=0).stop 2023-10-11T09:51:19.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.940+0000 7f1dba984700 1 -- 172.21.15.78:0/2421645407 shutdown_connections 2023-10-11T09:51:19.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.940+0000 7f1dba984700 1 --2- 172.21.15.78:0/2421645407 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1d98065ed0 0x7f1d98068380 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:19.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.940+0000 7f1dba984700 1 --2- 172.21.15.78:0/2421645407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1db4161d90 0x7f1db4145e40 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:19.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.940+0000 7f1dba984700 1 --2- 172.21.15.78:0/2421645407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1db4146fe0 0x7f1db40cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:19.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.940+0000 7f1dba984700 1 --2- 172.21.15.78:0/2421645407 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1db40b36f0 0x7f1db40ab850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:19.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.940+0000 7f1dba984700 1 -- 172.21.15.78:0/2421645407 >> 172.21.15.78:0/2421645407 conn(0x7f1db406a060 msgr2=0x7f1db40ae6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:19.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.940+0000 7f1dba984700 1 -- 172.21.15.78:0/2421645407 shutdown_connections 2023-10-11T09:51:19.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:19.940+0000 7f1dba984700 1 -- 172.21.15.78:0/2421645407 wait complete. 2023-10-11T09:51:19.971 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T09:51:20.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.144+0000 7fe323b8b700 1 Processor -- start 2023-10-11T09:51:20.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.148+0000 7fe323b8b700 1 -- start start 2023-10-11T09:51:20.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.148+0000 7fe323b8b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe31c0b1030 0x7fe31c0cb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:20.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.148+0000 7fe323b8b700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe31c06d080 0x7fe31c06ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:20.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.148+0000 7fe323b8b700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe31c165570 con 0x7fe31c160cb0 2023-10-11T09:51:20.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.148+0000 7fe323b8b700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe31c161080 con 0x7fe31c0b1030 2023-10-11T09:51:20.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.148+0000 7fe323b8b700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe31c164750 con 0x7fe31c06d080 2023-10-11T09:51:20.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.148+0000 7fe321126700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe31c0b1030 0x7fe31c0cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:20.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.148+0000 7fe321126700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe31c0b1030 0x7fe31c0cb100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45506/0 (socket says 172.21.15.78:45506) 2023-10-11T09:51:20.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.148+0000 7fe321126700 1 -- 172.21.15.78:0/2074022746 learned_addr learned my addr 172.21.15.78:0/2074022746 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:20.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.148+0000 7fe322128700 1 --2- 172.21.15.78:0/2074022746 >> v2:172.21.15.133:3300/0 conn(0x7fe31c06d080 0x7fe31c06ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:20.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.148+0000 7fe321126700 1 -- 172.21.15.78:0/2074022746 >> v2:172.21.15.133:3300/0 conn(0x7fe31c06d080 msgr2=0x7fe31c06ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:20.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.148+0000 7fe321126700 1 --2- 172.21.15.78:0/2074022746 >> v2:172.21.15.133:3300/0 conn(0x7fe31c06d080 0x7fe31c06ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:20.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.148+0000 7fe321126700 1 -- 172.21.15.78:0/2074022746 >> v1:172.21.15.133:6789/0 conn(0x7fe31c160cb0 legacy=0x7fe31c0b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:20.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.148+0000 7fe321126700 1 -- 172.21.15.78:0/2074022746 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe31c164960 con 0x7fe31c0b1030 2023-10-11T09:51:20.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.148+0000 7fe322128700 1 --2- 172.21.15.78:0/2074022746 >> v2:172.21.15.133:3300/0 conn(0x7fe31c06d080 0x7fe31c06ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:51:20.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.152+0000 7fe321126700 1 --2- 172.21.15.78:0/2074022746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe31c0b1030 0x7fe31c0cb100 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fe31c0726a0 tx=0x7fe30c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d64ae622456d99c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:20.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.152+0000 7fe320925700 1 -- 172.21.15.78:0/2074022746 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30c00b8a0 con 0x7fe31c0b1030 2023-10-11T09:51:20.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.152+0000 7fe320925700 1 -- 172.21.15.78:0/2074022746 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe30c00ba00 con 0x7fe31c0b1030 2023-10-11T09:51:20.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.152+0000 7fe320925700 1 -- 172.21.15.78:0/2074022746 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30c00bd90 con 0x7fe31c0b1030 2023-10-11T09:51:20.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.152+0000 7fe323b8b700 1 -- 172.21.15.78:0/2074022746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe31c0b1030 msgr2=0x7fe31c0cb100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:20.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.152+0000 7fe323b8b700 1 --2- 172.21.15.78:0/2074022746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe31c0b1030 0x7fe31c0cb100 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fe31c0726a0 tx=0x7fe30c009bf0 comp rx=0 tx=0).stop 2023-10-11T09:51:20.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.152+0000 7fe323b8b700 1 -- 172.21.15.78:0/2074022746 shutdown_connections 2023-10-11T09:51:20.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.152+0000 7fe323b8b700 1 --2- 172.21.15.78:0/2074022746 >> v2:172.21.15.133:3300/0 conn(0x7fe31c06d080 0x7fe31c06ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:20.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.152+0000 7fe323b8b700 1 --2- 172.21.15.78:0/2074022746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe31c0b1030 0x7fe31c0cb100 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:20.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.152+0000 7fe323b8b700 1 -- 172.21.15.78:0/2074022746 >> 172.21.15.78:0/2074022746 conn(0x7fe31c0cecc0 msgr2=0x7fe31c0cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:20.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.152+0000 7fe323b8b700 1 -- 172.21.15.78:0/2074022746 shutdown_connections 2023-10-11T09:51:20.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.152+0000 7fe323b8b700 1 -- 172.21.15.78:0/2074022746 wait complete. 2023-10-11T09:51:20.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.152+0000 7fe323b8b700 1 Processor -- start 2023-10-11T09:51:20.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.156+0000 7fe323b8b700 1 -- start start 2023-10-11T09:51:20.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.156+0000 7fe323b8b700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe31c06d080 0x7fe31c151920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:20.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.156+0000 7fe323b8b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe31c0b1030 0x7fe31c15ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:20.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.156+0000 7fe323b8b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe31c160cb0 0x7fe31c15f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:20.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.156+0000 7fe323b8b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c155920 con 0x7fe31c0b1030 2023-10-11T09:51:20.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.156+0000 7fe323b8b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c152750 con 0x7fe31c160cb0 2023-10-11T09:51:20.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.156+0000 7fe323b8b700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c1535b0 con 0x7fe31c06d080 2023-10-11T09:51:20.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.156+0000 7fe321126700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe31c0b1030 0x7fe31c15ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:20.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.156+0000 7fe321126700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe31c0b1030 0x7fe31c15ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45508/0 (socket says 172.21.15.78:45508) 2023-10-11T09:51:20.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.156+0000 7fe321126700 1 -- 172.21.15.78:0/1546074236 learned_addr learned my addr 172.21.15.78:0/1546074236 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:20.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.156+0000 7fe322128700 1 --2- 172.21.15.78:0/1546074236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe31c160cb0 0x7fe31c15f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:20.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.156+0000 7fe321927700 1 --2- 172.21.15.78:0/1546074236 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe31c06d080 0x7fe31c151920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:20.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.156+0000 7fe321126700 1 -- 172.21.15.78:0/1546074236 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe31c06d080 msgr2=0x7fe31c151920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:20.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.156+0000 7fe321126700 1 --2- 172.21.15.78:0/1546074236 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe31c06d080 0x7fe31c151920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:20.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.156+0000 7fe321126700 1 -- 172.21.15.78:0/1546074236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe31c160cb0 msgr2=0x7fe31c15f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:20.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.156+0000 7fe321126700 1 --2- 172.21.15.78:0/1546074236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe31c160cb0 0x7fe31c15f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:20.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.156+0000 7fe321126700 1 -- 172.21.15.78:0/1546074236 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe30c009570 con 0x7fe31c0b1030 2023-10-11T09:51:20.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.156+0000 7fe321927700 1 --2- 172.21.15.78:0/1546074236 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe31c06d080 0x7fe31c151920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:51:20.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.156+0000 7fe321126700 1 --2- 172.21.15.78:0/1546074236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe31c0b1030 0x7fe31c15ed10 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fe30c000c00 tx=0x7fe30c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:20.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.156+0000 7fe3127fc700 1 -- 172.21.15.78:0/1546074236 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30c0038c0 con 0x7fe31c0b1030 2023-10-11T09:51:20.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.156+0000 7fe3127fc700 1 -- 172.21.15.78:0/1546074236 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe30c01e070 con 0x7fe31c0b1030 2023-10-11T09:51:20.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.156+0000 7fe3127fc700 1 -- 172.21.15.78:0/1546074236 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30c003c80 con 0x7fe31c0b1030 2023-10-11T09:51:20.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.156+0000 7fe323b8b700 1 -- 172.21.15.78:0/1546074236 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe31c1529a0 con 0x7fe31c0b1030 2023-10-11T09:51:20.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.156+0000 7fe323b8b700 1 -- 172.21.15.78:0/1546074236 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe31c152f60 con 0x7fe31c0b1030 2023-10-11T09:51:20.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.160+0000 7fe323b8b700 1 -- 172.21.15.78:0/1546074236 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe31c153740 con 0x7fe31c0b1030 2023-10-11T09:51:20.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.160+0000 7fe3127fc700 1 -- 172.21.15.78:0/1546074236 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fe30c011420 con 0x7fe31c0b1030 2023-10-11T09:51:20.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.160+0000 7fe3127fc700 1 --2- 172.21.15.78:0/1546074236 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe300065ed0 0x7fe300068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:20.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.164+0000 7fe3127fc700 1 -- 172.21.15.78:0/1546074236 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6724+0+0 (secure 0 0 0) 0x7fe30c020070 con 0x7fe31c0b1030 2023-10-11T09:51:20.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.164+0000 7fe321927700 1 --2- 172.21.15.78:0/1546074236 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe300065ed0 0x7fe300068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:20.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.164+0000 7fe321927700 1 --2- 172.21.15.78:0/1546074236 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe300065ed0 0x7fe300068380 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fe318001700 tx=0x7fe318009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:20.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.168+0000 7fe3127fc700 1 -- 172.21.15.78:0/1546074236 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fe30c052580 con 0x7fe31c0b1030 2023-10-11T09:51:20.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.344+0000 7fe323b8b700 1 -- 172.21.15.78:0/1546074236 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe31c0ccff0 con 0x7fe31c0b1030 2023-10-11T09:51:20.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.344+0000 7fe3127fc700 1 -- 172.21.15.78:0/1546074236 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v114) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe30c055ba0 con 0x7fe31c0b1030 2023-10-11T09:51:20.355 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:51:20.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.348+0000 7fe323b8b700 1 -- 172.21.15.78:0/1546074236 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe300065ed0 msgr2=0x7fe300068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:20.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.348+0000 7fe323b8b700 1 --2- 172.21.15.78:0/1546074236 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe300065ed0 0x7fe300068380 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fe318001700 tx=0x7fe318009040 comp rx=0 tx=0).stop 2023-10-11T09:51:20.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.348+0000 7fe323b8b700 1 -- 172.21.15.78:0/1546074236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe31c0b1030 msgr2=0x7fe31c15ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:20.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.348+0000 7fe323b8b700 1 --2- 172.21.15.78:0/1546074236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe31c0b1030 0x7fe31c15ed10 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fe30c000c00 tx=0x7fe30c004770 comp rx=0 tx=0).stop 2023-10-11T09:51:20.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.348+0000 7fe323b8b700 1 -- 172.21.15.78:0/1546074236 shutdown_connections 2023-10-11T09:51:20.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.348+0000 7fe323b8b700 1 --2- 172.21.15.78:0/1546074236 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe300065ed0 0x7fe300068380 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:20.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.348+0000 7fe323b8b700 1 --2- 172.21.15.78:0/1546074236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe31c160cb0 0x7fe31c15f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:20.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.348+0000 7fe323b8b700 1 --2- 172.21.15.78:0/1546074236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe31c0b1030 0x7fe31c15ed10 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:20.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.348+0000 7fe323b8b700 1 --2- 172.21.15.78:0/1546074236 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe31c06d080 0x7fe31c151920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:20.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.348+0000 7fe323b8b700 1 -- 172.21.15.78:0/1546074236 >> 172.21.15.78:0/1546074236 conn(0x7fe31c0cecc0 msgr2=0x7fe31c0cdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:20.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.348+0000 7fe323b8b700 1 -- 172.21.15.78:0/1546074236 shutdown_connections 2023-10-11T09:51:20.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.348+0000 7fe323b8b700 1 -- 172.21.15.78:0/1546074236 wait complete. 2023-10-11T09:51:20.381 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T09:51:20.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.548+0000 7f0400a39700 1 Processor -- start 2023-10-11T09:51:20.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.548+0000 7f0400a39700 1 -- start start 2023-10-11T09:51:20.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.548+0000 7f0400a39700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f03fc161d90 0x7f03fc0b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:20.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.548+0000 7f0400a39700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f03fc0b3520 0x7f03fc1629e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:20.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.548+0000 7f0400a39700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03fc06ccb0 con 0x7f03fc146fe0 2023-10-11T09:51:20.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.548+0000 7f0400a39700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03fc06c410 con 0x7f03fc161d90 2023-10-11T09:51:20.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.548+0000 7f0400a39700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc06c6f0 con 0x7f03fc0b3520 2023-10-11T09:51:20.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.548+0000 7f03faffd700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f03fc161d90 0x7f03fc0b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:20.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.548+0000 7f03faffd700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f03fc161d90 0x7f03fc0b2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45514/0 (socket says 172.21.15.78:45514) 2023-10-11T09:51:20.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.548+0000 7f03faffd700 1 -- 172.21.15.78:0/2532176062 learned_addr learned my addr 172.21.15.78:0/2532176062 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:20.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.548+0000 7f03fbfff700 1 --2- 172.21.15.78:0/2532176062 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f03fc0b3520 0x7f03fc1629e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:20.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.548+0000 7f03faffd700 1 -- 172.21.15.78:0/2532176062 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f03fc0b3520 msgr2=0x7f03fc1629e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:20.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.548+0000 7f03faffd700 1 --2- 172.21.15.78:0/2532176062 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f03fc0b3520 0x7f03fc1629e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:20.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.548+0000 7f03faffd700 1 -- 172.21.15.78:0/2532176062 >> v1:172.21.15.133:6789/0 conn(0x7f03fc146fe0 legacy=0x7f03fc06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:20.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.548+0000 7f03faffd700 1 -- 172.21.15.78:0/2532176062 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03fc06c9d0 con 0x7f03fc161d90 2023-10-11T09:51:20.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.548+0000 7f03fbfff700 1 --2- 172.21.15.78:0/2532176062 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f03fc0b3520 0x7f03fc1629e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:51:20.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.548+0000 7f03faffd700 1 --2- 172.21.15.78:0/2532176062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03fc161d90 0x7f03fc0b2fe0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009910 tx=0x7f03f0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea1abbf4d9ec7617 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:20.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.552+0000 7f03fa7fc700 1 -- 172.21.15.78:0/2532176062 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f000ba60 con 0x7f03fc161d90 2023-10-11T09:51:20.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.552+0000 7f03fa7fc700 1 -- 172.21.15.78:0/2532176062 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f000bbc0 con 0x7f03fc161d90 2023-10-11T09:51:20.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.552+0000 7f03fa7fc700 1 -- 172.21.15.78:0/2532176062 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f0009e60 con 0x7f03fc161d90 2023-10-11T09:51:20.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.552+0000 7f0400a39700 1 -- 172.21.15.78:0/2532176062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03fc161d90 msgr2=0x7f03fc0b2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:20.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.552+0000 7f0400a39700 1 --2- 172.21.15.78:0/2532176062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03fc161d90 0x7f03fc0b2fe0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009910 tx=0x7f03f0009c20 comp rx=0 tx=0).stop 2023-10-11T09:51:20.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.552+0000 7f0400a39700 1 -- 172.21.15.78:0/2532176062 shutdown_connections 2023-10-11T09:51:20.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.552+0000 7f0400a39700 1 --2- 172.21.15.78:0/2532176062 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f03fc0b3520 0x7f03fc1629e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:20.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.552+0000 7f0400a39700 1 --2- 172.21.15.78:0/2532176062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03fc161d90 0x7f03fc0b2fe0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:20.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.552+0000 7f0400a39700 1 -- 172.21.15.78:0/2532176062 >> 172.21.15.78:0/2532176062 conn(0x7f03fc06a060 msgr2=0x7f03fc06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:20.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.552+0000 7f0400a39700 1 -- 172.21.15.78:0/2532176062 shutdown_connections 2023-10-11T09:51:20.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.552+0000 7f0400a39700 1 -- 172.21.15.78:0/2532176062 wait complete. 2023-10-11T09:51:20.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.552+0000 7f0400a39700 1 Processor -- start 2023-10-11T09:51:20.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.556+0000 7f0400a39700 1 -- start start 2023-10-11T09:51:20.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.556+0000 7f0400a39700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03fc0b3520 0x7f03fc145fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:20.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.556+0000 7f0400a39700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f03fc146fe0 0x7f03fc0cb710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:20.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.556+0000 7f0400a39700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03fc161d90 0x7f03fc0cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:20.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.556+0000 7f0400a39700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc1f7000 con 0x7f03fc0b3520 2023-10-11T09:51:20.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.556+0000 7f0400a39700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc1f72e0 con 0x7f03fc161d90 2023-10-11T09:51:20.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.556+0000 7f0400a39700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc1f75c0 con 0x7f03fc146fe0 2023-10-11T09:51:20.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.556+0000 7f03fb7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03fc0b3520 0x7f03fc145fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:20.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.556+0000 7f03fb7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03fc0b3520 0x7f03fc145fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45518/0 (socket says 172.21.15.78:45518) 2023-10-11T09:51:20.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.556+0000 7f03fb7fe700 1 -- 172.21.15.78:0/1392489606 learned_addr learned my addr 172.21.15.78:0/1392489606 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:20.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.556+0000 7f03fbfff700 1 --2- 172.21.15.78:0/1392489606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03fc161d90 0x7f03fc0cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:20.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.556+0000 7f03faffd700 1 --2- 172.21.15.78:0/1392489606 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f03fc146fe0 0x7f03fc0cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:20.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.556+0000 7f03fb7fe700 1 -- 172.21.15.78:0/1392489606 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f03fc146fe0 msgr2=0x7f03fc0cb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:20.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.556+0000 7f03fb7fe700 1 --2- 172.21.15.78:0/1392489606 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f03fc146fe0 0x7f03fc0cb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:20.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.556+0000 7f03fb7fe700 1 -- 172.21.15.78:0/1392489606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03fc161d90 msgr2=0x7f03fc0cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:20.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.556+0000 7f03fb7fe700 1 --2- 172.21.15.78:0/1392489606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03fc161d90 0x7f03fc0cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:20.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.556+0000 7f03fb7fe700 1 -- 172.21.15.78:0/1392489606 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f0009570 con 0x7f03fc0b3520 2023-10-11T09:51:20.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.556+0000 7f03fb7fe700 1 --2- 172.21.15.78:0/1392489606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03fc0b3520 0x7f03fc145fd0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f03ec00c500 tx=0x7f03ec00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:20.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.556+0000 7f03e3fff700 1 -- 172.21.15.78:0/1392489606 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ec0052d0 con 0x7f03fc0b3520 2023-10-11T09:51:20.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.556+0000 7f0400a39700 1 -- 172.21.15.78:0/1392489606 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03fc1f78a0 con 0x7f03fc0b3520 2023-10-11T09:51:20.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.556+0000 7f03e3fff700 1 -- 172.21.15.78:0/1392489606 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03ec005430 con 0x7f03fc0b3520 2023-10-11T09:51:20.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.556+0000 7f03e3fff700 1 -- 172.21.15.78:0/1392489606 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ec0055a0 con 0x7f03fc0b3520 2023-10-11T09:51:20.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.556+0000 7f0400a39700 1 -- 172.21.15.78:0/1392489606 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03fc071660 con 0x7f03fc0b3520 2023-10-11T09:51:20.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.560+0000 7f0400a39700 1 -- 172.21.15.78:0/1392489606 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03fc160fb0 con 0x7f03fc0b3520 2023-10-11T09:51:20.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.560+0000 7f03e3fff700 1 -- 172.21.15.78:0/1392489606 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f03ec0038e0 con 0x7f03fc0b3520 2023-10-11T09:51:20.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.560+0000 7f03e3fff700 1 --2- 172.21.15.78:0/1392489606 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f03e8065ed0 0x7f03e8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:20.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.564+0000 7f03e3fff700 1 -- 172.21.15.78:0/1392489606 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f03ec082c90 con 0x7f03fc0b3520 2023-10-11T09:51:20.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.564+0000 7f03faffd700 1 --2- 172.21.15.78:0/1392489606 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f03e8065ed0 0x7f03e8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:20.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.564+0000 7f03faffd700 1 --2- 172.21.15.78:0/1392489606 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f03e8065ed0 0x7f03e8068380 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f03f0000c00 tx=0x7f03f0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:20.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.568+0000 7f03e3fff700 1 -- 172.21.15.78:0/1392489606 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f03ec04ffc0 con 0x7f03fc0b3520 2023-10-11T09:51:20.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.752+0000 7f0400a39700 1 -- 172.21.15.78:0/1392489606 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f03fc065d40 con 0x7f03fc0b3520 2023-10-11T09:51:20.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.752+0000 7f03e3fff700 1 -- 172.21.15.78:0/1392489606 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v114) v1 ==== 100+0+11 (secure 0 0 0) 0x7f03ec0535e0 con 0x7f03fc0b3520 2023-10-11T09:51:20.762 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2023-10-11T09:51:20.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.756+0000 7f0400a39700 1 -- 172.21.15.78:0/1392489606 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f03e8065ed0 msgr2=0x7f03e8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:20.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.756+0000 7f0400a39700 1 --2- 172.21.15.78:0/1392489606 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f03e8065ed0 0x7f03e8068380 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f03f0000c00 tx=0x7f03f0008030 comp rx=0 tx=0).stop 2023-10-11T09:51:20.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.756+0000 7f0400a39700 1 -- 172.21.15.78:0/1392489606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03fc0b3520 msgr2=0x7f03fc145fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:20.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.756+0000 7f0400a39700 1 --2- 172.21.15.78:0/1392489606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03fc0b3520 0x7f03fc145fd0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f03ec00c500 tx=0x7f03ec00c810 comp rx=0 tx=0).stop 2023-10-11T09:51:20.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.756+0000 7f0400a39700 1 -- 172.21.15.78:0/1392489606 shutdown_connections 2023-10-11T09:51:20.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.756+0000 7f0400a39700 1 --2- 172.21.15.78:0/1392489606 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f03e8065ed0 0x7f03e8068380 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:20.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.756+0000 7f0400a39700 1 --2- 172.21.15.78:0/1392489606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03fc161d90 0x7f03fc0cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:20.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.756+0000 7f0400a39700 1 --2- 172.21.15.78:0/1392489606 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f03fc146fe0 0x7f03fc0cb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:20.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.756+0000 7f0400a39700 1 --2- 172.21.15.78:0/1392489606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03fc0b3520 0x7f03fc145fd0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:20.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.756+0000 7f0400a39700 1 -- 172.21.15.78:0/1392489606 >> 172.21.15.78:0/1392489606 conn(0x7f03fc06a060 msgr2=0x7f03fc0af3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:20.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.756+0000 7f0400a39700 1 -- 172.21.15.78:0/1392489606 shutdown_connections 2023-10-11T09:51:20.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.756+0000 7f0400a39700 1 -- 172.21.15.78:0/1392489606 wait complete. 2023-10-11T09:51:20.788 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-10-11T09:51:20.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.952+0000 7f071b59e700 1 Processor -- start 2023-10-11T09:51:20.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.952+0000 7f071b59e700 1 -- start start 2023-10-11T09:51:20.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.952+0000 7f071b59e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f071c15c900 0x7f071c0b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:20.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.952+0000 7f071b59e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f071c0ce7f0 0x7f071c0cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:20.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.952+0000 7f071b59e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f071c06fde0 con 0x7f071c0ad290 2023-10-11T09:51:20.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.952+0000 7f071b59e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f071c0705f0 con 0x7f071c15c900 2023-10-11T09:51:20.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.952+0000 7f071b59e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f071c0cf1b0 con 0x7f071c0ce7f0 2023-10-11T09:51:20.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.956+0000 7f0711d9b700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f071c0ad290 0x7f071c071a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33642/0 (socket says 172.21.15.78:33642) 2023-10-11T09:51:20.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.956+0000 7f0711d9b700 1 -- 172.21.15.78:0/3718157967 learned_addr learned my addr 172.21.15.78:0/3718157967 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:51:20.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.956+0000 7f071a59c700 1 --2- 172.21.15.78:0/3718157967 >> v2:172.21.15.133:3300/0 conn(0x7f071c15c900 0x7f071c0b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:20.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.956+0000 7f071ad9d700 1 --2- 172.21.15.78:0/3718157967 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f071c0ce7f0 0x7f071c0cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:20.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.956+0000 7f0719d9b700 1 -- 172.21.15.78:0/3718157967 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4230158183 0 0) 0x7f071c06fde0 con 0x7f071c0ad290 2023-10-11T09:51:20.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.956+0000 7f0719d9b700 1 -- 172.21.15.78:0/3718157967 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07080035d0 con 0x7f071c0ad290 2023-10-11T09:51:20.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.956+0000 7f071ad9d700 1 -- 172.21.15.78:0/3718157967 >> v2:172.21.15.133:3300/0 conn(0x7f071c15c900 msgr2=0x7f071c0b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:20.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.956+0000 7f071ad9d700 1 --2- 172.21.15.78:0/3718157967 >> v2:172.21.15.133:3300/0 conn(0x7f071c15c900 0x7f071c0b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:20.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.956+0000 7f071ad9d700 1 -- 172.21.15.78:0/3718157967 >> v1:172.21.15.78:6789/0 conn(0x7f071c0ad290 legacy=0x7f071c071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:20.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.956+0000 7f071ad9d700 1 -- 172.21.15.78:0/3718157967 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f071c0ae000 con 0x7f071c0ce7f0 2023-10-11T09:51:20.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.956+0000 7f071a59c700 1 --2- 172.21.15.78:0/3718157967 >> v2:172.21.15.133:3300/0 conn(0x7f071c15c900 0x7f071c0b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:51:20.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.956+0000 7f071ad9d700 1 --2- 172.21.15.78:0/3718157967 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f071c0ce7f0 0x7f071c0cec70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0714009850 tx=0x7f0714009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18882385c188318d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:20.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.956+0000 7f0719d9b700 1 -- 172.21.15.78:0/3718157967 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0714011830 con 0x7f071c0ce7f0 2023-10-11T09:51:20.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.956+0000 7f0719d9b700 1 -- 172.21.15.78:0/3718157967 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0714011990 con 0x7f071c0ce7f0 2023-10-11T09:51:20.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.956+0000 7f0719d9b700 1 -- 172.21.15.78:0/3718157967 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0714011d40 con 0x7f071c0ce7f0 2023-10-11T09:51:20.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.956+0000 7f071b59e700 1 -- 172.21.15.78:0/3718157967 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f071c0ce7f0 msgr2=0x7f071c0cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:20.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.956+0000 7f071b59e700 1 --2- 172.21.15.78:0/3718157967 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f071c0ce7f0 0x7f071c0cec70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0714009850 tx=0x7f0714009b60 comp rx=0 tx=0).stop 2023-10-11T09:51:20.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.956+0000 7f071b59e700 1 -- 172.21.15.78:0/3718157967 shutdown_connections 2023-10-11T09:51:20.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.956+0000 7f071b59e700 1 --2- 172.21.15.78:0/3718157967 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f071c0ce7f0 0x7f071c0cec70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:20.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.956+0000 7f071b59e700 1 --2- 172.21.15.78:0/3718157967 >> v2:172.21.15.133:3300/0 conn(0x7f071c15c900 0x7f071c0b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:20.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.956+0000 7f071b59e700 1 -- 172.21.15.78:0/3718157967 >> 172.21.15.78:0/3718157967 conn(0x7f071c06a060 msgr2=0x7f071c06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:20.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.956+0000 7f071b59e700 1 -- 172.21.15.78:0/3718157967 shutdown_connections 2023-10-11T09:51:20.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.956+0000 7f071b59e700 1 -- 172.21.15.78:0/3718157967 wait complete. 2023-10-11T09:51:20.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.960+0000 7f071b59e700 1 Processor -- start 2023-10-11T09:51:20.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.960+0000 7f071b59e700 1 -- start start 2023-10-11T09:51:20.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.964+0000 7f071b59e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f071c0ad290 0x7f071c0cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:20.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.964+0000 7f071b59e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f071c0ce7f0 0x7f071c144170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:20.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.964+0000 7f071b59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f071c15c900 0x7f071c1434b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:20.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.964+0000 7f071b59e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071c1458f0 con 0x7f071c15c900 2023-10-11T09:51:20.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.964+0000 7f071b59e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071c144c70 con 0x7f071c0ce7f0 2023-10-11T09:51:20.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.964+0000 7f071b59e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f071c144f50 con 0x7f071c0ad290 2023-10-11T09:51:20.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.964+0000 7f071ad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f071c15c900 0x7f071c1434b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:20.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.964+0000 7f071ad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f071c15c900 0x7f071c1434b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45524/0 (socket says 172.21.15.78:45524) 2023-10-11T09:51:20.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.964+0000 7f071ad9d700 1 -- 172.21.15.78:0/1312685433 learned_addr learned my addr 172.21.15.78:0/1312685433 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:20.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.964+0000 7f0711d9b700 1 --2- 172.21.15.78:0/1312685433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f071c0ce7f0 0x7f071c144170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:20.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.964+0000 7f071a59c700 1 --2- 172.21.15.78:0/1312685433 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f071c0ad290 0x7f071c0cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:20.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.964+0000 7f071ad9d700 1 -- 172.21.15.78:0/1312685433 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f071c0ad290 msgr2=0x7f071c0cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:20.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.964+0000 7f071ad9d700 1 --2- 172.21.15.78:0/1312685433 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f071c0ad290 0x7f071c0cbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:20.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.964+0000 7f071ad9d700 1 -- 172.21.15.78:0/1312685433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f071c0ce7f0 msgr2=0x7f071c144170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:20.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.964+0000 7f071ad9d700 1 --2- 172.21.15.78:0/1312685433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f071c0ce7f0 0x7f071c144170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:20.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.964+0000 7f071ad9d700 1 -- 172.21.15.78:0/1312685433 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f070c009a70 con 0x7f071c15c900 2023-10-11T09:51:20.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.964+0000 7f071ad9d700 1 --2- 172.21.15.78:0/1312685433 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f071c15c900 0x7f071c1434b0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f07140090b0 tx=0x7f0714004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:20.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.964+0000 7f07137fe700 1 -- 172.21.15.78:0/1312685433 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0714004510 con 0x7f071c15c900 2023-10-11T09:51:20.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.964+0000 7f07137fe700 1 -- 172.21.15.78:0/1312685433 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f071401e070 con 0x7f071c15c900 2023-10-11T09:51:20.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.964+0000 7f07137fe700 1 -- 172.21.15.78:0/1312685433 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07140038e0 con 0x7f071c15c900 2023-10-11T09:51:20.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.964+0000 7f071b59e700 1 -- 172.21.15.78:0/1312685433 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07140094b0 con 0x7f071c15c900 2023-10-11T09:51:20.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.964+0000 7f071b59e700 1 -- 172.21.15.78:0/1312685433 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f071c145510 con 0x7f071c15c900 2023-10-11T09:51:20.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.968+0000 7f071b59e700 1 -- 172.21.15.78:0/1312685433 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f071c065d40 con 0x7f071c15c900 2023-10-11T09:51:20.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.968+0000 7f07137fe700 1 -- 172.21.15.78:0/1312685433 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f0714003a40 con 0x7f071c15c900 2023-10-11T09:51:20.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.972+0000 7f07137fe700 1 --2- 172.21.15.78:0/1312685433 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f06fc065ed0 0x7f06fc068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:20.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.972+0000 7f07137fe700 1 -- 172.21.15.78:0/1312685433 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f0714020070 con 0x7f071c15c900 2023-10-11T09:51:20.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.972+0000 7f071a59c700 1 --2- 172.21.15.78:0/1312685433 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f06fc065ed0 0x7f06fc068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:20.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.972+0000 7f071a59c700 1 --2- 172.21.15.78:0/1312685433 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f06fc065ed0 0x7f06fc068380 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f070c009fd0 tx=0x7f070c00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:20.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:20.980+0000 7f07137fe700 1 -- 172.21.15.78:0/1312685433 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f0714052170 con 0x7f071c15c900 2023-10-11T09:51:21.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.156+0000 7f071b59e700 1 -- 172.21.15.78:0/1312685433 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7f071c0cf5b0 con 0x7f071c15c900 2023-10-11T09:51:21.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.156+0000 7f07137fe700 1 -- 172.21.15.78:0/1312685433 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v114) v1 ==== 106+0+11 (secure 0 0 0) 0x7f0714015020 con 0x7f071c15c900 2023-10-11T09:51:21.167 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:51:21.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.160+0000 7f071b59e700 1 -- 172.21.15.78:0/1312685433 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f06fc065ed0 msgr2=0x7f06fc068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:21.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.160+0000 7f071b59e700 1 --2- 172.21.15.78:0/1312685433 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f06fc065ed0 0x7f06fc068380 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f070c009fd0 tx=0x7f070c00b040 comp rx=0 tx=0).stop 2023-10-11T09:51:21.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.160+0000 7f071b59e700 1 -- 172.21.15.78:0/1312685433 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f071c15c900 msgr2=0x7f071c1434b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:21.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.160+0000 7f071b59e700 1 --2- 172.21.15.78:0/1312685433 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f071c15c900 0x7f071c1434b0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f07140090b0 tx=0x7f0714004770 comp rx=0 tx=0).stop 2023-10-11T09:51:21.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.160+0000 7f071b59e700 1 -- 172.21.15.78:0/1312685433 shutdown_connections 2023-10-11T09:51:21.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.160+0000 7f071b59e700 1 --2- 172.21.15.78:0/1312685433 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f06fc065ed0 0x7f06fc068380 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:21.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.160+0000 7f071b59e700 1 --2- 172.21.15.78:0/1312685433 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f071c15c900 0x7f071c1434b0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:21.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.160+0000 7f071b59e700 1 --2- 172.21.15.78:0/1312685433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f071c0ce7f0 0x7f071c144170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:21.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.160+0000 7f071b59e700 1 --2- 172.21.15.78:0/1312685433 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f071c0ad290 0x7f071c0cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:21.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.160+0000 7f071b59e700 1 -- 172.21.15.78:0/1312685433 >> 172.21.15.78:0/1312685433 conn(0x7f071c06a060 msgr2=0x7f071c0ccf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:21.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.160+0000 7f071b59e700 1 -- 172.21.15.78:0/1312685433 shutdown_connections 2023-10-11T09:51:21.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.160+0000 7f071b59e700 1 -- 172.21.15.78:0/1312685433 wait complete. 2023-10-11T09:51:21.191 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T09:51:21.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.356+0000 7f6d266b0700 1 Processor -- start 2023-10-11T09:51:21.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.356+0000 7f6d266b0700 1 -- start start 2023-10-11T09:51:21.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.356+0000 7f6d266b0700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f6d2013f740 0x7f6d200ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:21.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.356+0000 7f6d266b0700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6d2006d180 0x7f6d200b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:21.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.356+0000 7f6d266b0700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d20070c80 con 0x7f6d2006cd00 2023-10-11T09:51:21.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.356+0000 7f6d266b0700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d200ce4c0 con 0x7f6d2013f740 2023-10-11T09:51:21.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.356+0000 7f6d266b0700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d2006d600 con 0x7f6d2006d180 2023-10-11T09:51:21.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.356+0000 7f6d256ae700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f6d2013f740 0x7f6d200ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:21.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.356+0000 7f6d256ae700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f6d2013f740 0x7f6d200ccab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45534/0 (socket says 172.21.15.78:45534) 2023-10-11T09:51:21.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.356+0000 7f6d256ae700 1 -- 172.21.15.78:0/2332765668 learned_addr learned my addr 172.21.15.78:0/2332765668 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:21.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.356+0000 7f6d25eaf700 1 --2- 172.21.15.78:0/2332765668 >> v2:172.21.15.133:3300/0 conn(0x7f6d2006d180 0x7f6d200b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:21.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.360+0000 7f6d256ae700 1 -- 172.21.15.78:0/2332765668 >> v2:172.21.15.133:3300/0 conn(0x7f6d2006d180 msgr2=0x7f6d200b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:21.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.360+0000 7f6d256ae700 1 --2- 172.21.15.78:0/2332765668 >> v2:172.21.15.133:3300/0 conn(0x7f6d2006d180 0x7f6d200b0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:21.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.360+0000 7f6d256ae700 1 -- 172.21.15.78:0/2332765668 >> v1:172.21.15.133:6789/0 conn(0x7f6d2006cd00 legacy=0x7f6d200b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:21.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.360+0000 7f6d256ae700 1 -- 172.21.15.78:0/2332765668 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d200cd420 con 0x7f6d2013f740 2023-10-11T09:51:21.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.360+0000 7f6d256ae700 1 --2- 172.21.15.78:0/2332765668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d2013f740 0x7f6d200ccab0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f6d200b1780 tx=0x7f6d1c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=228a201baa5c012d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:21.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.360+0000 7f6d17fff700 1 -- 172.21.15.78:0/2332765668 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c00b8a0 con 0x7f6d2013f740 2023-10-11T09:51:21.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.360+0000 7f6d17fff700 1 -- 172.21.15.78:0/2332765668 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d1c00ba00 con 0x7f6d2013f740 2023-10-11T09:51:21.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.360+0000 7f6d17fff700 1 -- 172.21.15.78:0/2332765668 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c00bd90 con 0x7f6d2013f740 2023-10-11T09:51:21.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.360+0000 7f6d266b0700 1 -- 172.21.15.78:0/2332765668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d2013f740 msgr2=0x7f6d200ccab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:21.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.360+0000 7f6d266b0700 1 --2- 172.21.15.78:0/2332765668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d2013f740 0x7f6d200ccab0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f6d200b1780 tx=0x7f6d1c009bf0 comp rx=0 tx=0).stop 2023-10-11T09:51:21.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.360+0000 7f6d266b0700 1 -- 172.21.15.78:0/2332765668 shutdown_connections 2023-10-11T09:51:21.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.360+0000 7f6d266b0700 1 --2- 172.21.15.78:0/2332765668 >> v2:172.21.15.133:3300/0 conn(0x7f6d2006d180 0x7f6d200b0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:21.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.360+0000 7f6d266b0700 1 --2- 172.21.15.78:0/2332765668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d2013f740 0x7f6d200ccab0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:21.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.360+0000 7f6d266b0700 1 -- 172.21.15.78:0/2332765668 >> 172.21.15.78:0/2332765668 conn(0x7f6d200cea60 msgr2=0x7f6d200cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:21.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.360+0000 7f6d266b0700 1 -- 172.21.15.78:0/2332765668 shutdown_connections 2023-10-11T09:51:21.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.360+0000 7f6d266b0700 1 -- 172.21.15.78:0/2332765668 wait complete. 2023-10-11T09:51:21.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.364+0000 7f6d266b0700 1 Processor -- start 2023-10-11T09:51:21.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.364+0000 7f6d266b0700 1 -- start start 2023-10-11T09:51:21.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.364+0000 7f6d266b0700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d2006cd00 0x7f6d200b4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:21.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.364+0000 7f6d266b0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d2006d180 0x7f6d2013dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:21.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.364+0000 7f6d266b0700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d2013f740 0x7f6d200aa510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:21.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.364+0000 7f6d266b0700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d200a8dd0 con 0x7f6d2006d180 2023-10-11T09:51:21.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.364+0000 7f6d266b0700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d200a84c0 con 0x7f6d2013f740 2023-10-11T09:51:21.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.364+0000 7f6d266b0700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d200b4960 con 0x7f6d2006cd00 2023-10-11T09:51:21.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.364+0000 7f6d24ead700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d2006d180 0x7f6d2013dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:21.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.364+0000 7f6d24ead700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d2006d180 0x7f6d2013dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45536/0 (socket says 172.21.15.78:45536) 2023-10-11T09:51:21.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.364+0000 7f6d24ead700 1 -- 172.21.15.78:0/3572212933 learned_addr learned my addr 172.21.15.78:0/3572212933 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:21.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.364+0000 7f6d256ae700 1 --2- 172.21.15.78:0/3572212933 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d2006cd00 0x7f6d200b4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:21.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.364+0000 7f6d25eaf700 1 --2- 172.21.15.78:0/3572212933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d2013f740 0x7f6d200aa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:21.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.364+0000 7f6d24ead700 1 -- 172.21.15.78:0/3572212933 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d2006cd00 msgr2=0x7f6d200b4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:21.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.364+0000 7f6d24ead700 1 --2- 172.21.15.78:0/3572212933 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d2006cd00 0x7f6d200b4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:21.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.364+0000 7f6d24ead700 1 -- 172.21.15.78:0/3572212933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d2013f740 msgr2=0x7f6d200aa510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:21.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.364+0000 7f6d24ead700 1 --2- 172.21.15.78:0/3572212933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d2013f740 0x7f6d200aa510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:21.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.364+0000 7f6d24ead700 1 -- 172.21.15.78:0/3572212933 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d1c009570 con 0x7f6d2006d180 2023-10-11T09:51:21.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.364+0000 7f6d256ae700 1 --2- 172.21.15.78:0/3572212933 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d2006cd00 0x7f6d200b4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:51:21.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.364+0000 7f6d25eaf700 1 --2- 172.21.15.78:0/3572212933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d2013f740 0x7f6d200aa510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:51:21.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.364+0000 7f6d24ead700 1 --2- 172.21.15.78:0/3572212933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d2006d180 0x7f6d2013dbc0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f6d1000c500 tx=0x7f6d1000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:21.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.364+0000 7f6d15ffb700 1 -- 172.21.15.78:0/3572212933 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d100052d0 con 0x7f6d2006d180 2023-10-11T09:51:21.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.368+0000 7f6d15ffb700 1 -- 172.21.15.78:0/3572212933 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d10005430 con 0x7f6d2006d180 2023-10-11T09:51:21.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.368+0000 7f6d266b0700 1 -- 172.21.15.78:0/3572212933 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d200a8710 con 0x7f6d2006d180 2023-10-11T09:51:21.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.368+0000 7f6d15ffb700 1 -- 172.21.15.78:0/3572212933 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d100055a0 con 0x7f6d2006d180 2023-10-11T09:51:21.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.368+0000 7f6d266b0700 1 -- 172.21.15.78:0/3572212933 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d2013e100 con 0x7f6d2006d180 2023-10-11T09:51:21.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.368+0000 7f6d266b0700 1 -- 172.21.15.78:0/3572212933 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d20065d40 con 0x7f6d2006d180 2023-10-11T09:51:21.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.368+0000 7f6d15ffb700 1 -- 172.21.15.78:0/3572212933 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f6d100036a0 con 0x7f6d2006d180 2023-10-11T09:51:21.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.372+0000 7f6d15ffb700 1 --2- 172.21.15.78:0/3572212933 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6d04065ed0 0x7f6d04068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:21.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.372+0000 7f6d15ffb700 1 -- 172.21.15.78:0/3572212933 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f6d10082df0 con 0x7f6d2006d180 2023-10-11T09:51:21.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.372+0000 7f6d256ae700 1 --2- 172.21.15.78:0/3572212933 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6d04065ed0 0x7f6d04068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:21.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.372+0000 7f6d256ae700 1 --2- 172.21.15.78:0/3572212933 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6d04065ed0 0x7f6d04068380 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c000c00 tx=0x7f6d1c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:21.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.376+0000 7f6d15ffb700 1 -- 172.21.15.78:0/3572212933 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f6d10050190 con 0x7f6d2006d180 2023-10-11T09:51:21.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.564+0000 7f6d266b0700 1 -- 172.21.15.78:0/3572212933 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6d20071a50 con 0x7f6d2006d180 2023-10-11T09:51:21.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.568+0000 7f6d15ffb700 1 -- 172.21.15.78:0/3572212933 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v114) v1 ==== 80+0+366 (secure 0 0 0) 0x7f6d100537b0 con 0x7f6d2006d180 2023-10-11T09:51:21.577 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6836/2876755099 2023-10-12T09:50:34.623416+0000 2023-10-11T09:51:21.577 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6837/1390949105 2023-10-12T09:50:28.599447+0000 2023-10-11T09:51:21.577 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6836/1390949105 2023-10-12T09:50:28.599447+0000 2023-10-11T09:51:21.578 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6837/2876755099 2023-10-12T09:50:34.623416+0000 2023-10-11T09:51:21.578 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6837/1286798022 2023-10-12T09:50:27.559697+0000 2023-10-11T09:51:21.578 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6836/1286798022 2023-10-12T09:50:27.559697+0000 2023-10-11T09:51:21.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.568+0000 7f6d266b0700 1 -- 172.21.15.78:0/3572212933 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6d04065ed0 msgr2=0x7f6d04068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:21.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.568+0000 7f6d266b0700 1 --2- 172.21.15.78:0/3572212933 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6d04065ed0 0x7f6d04068380 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c000c00 tx=0x7f6d1c008030 comp rx=0 tx=0).stop 2023-10-11T09:51:21.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.568+0000 7f6d266b0700 1 -- 172.21.15.78:0/3572212933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d2006d180 msgr2=0x7f6d2013dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:21.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.568+0000 7f6d266b0700 1 --2- 172.21.15.78:0/3572212933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d2006d180 0x7f6d2013dbc0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f6d1000c500 tx=0x7f6d1000c8c0 comp rx=0 tx=0).stop 2023-10-11T09:51:21.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.568+0000 7f6d266b0700 1 -- 172.21.15.78:0/3572212933 shutdown_connections 2023-10-11T09:51:21.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.568+0000 7f6d266b0700 1 --2- 172.21.15.78:0/3572212933 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6d04065ed0 0x7f6d04068380 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:21.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.572+0000 7f6d266b0700 1 --2- 172.21.15.78:0/3572212933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d2013f740 0x7f6d200aa510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:21.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.572+0000 7f6d266b0700 1 --2- 172.21.15.78:0/3572212933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d2006d180 0x7f6d2013dbc0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:21.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.572+0000 7f6d266b0700 1 --2- 172.21.15.78:0/3572212933 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d2006cd00 0x7f6d200b4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:21.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.572+0000 7f6d266b0700 1 -- 172.21.15.78:0/3572212933 >> 172.21.15.78:0/3572212933 conn(0x7f6d200cea60 msgr2=0x7f6d200c7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:21.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.572+0000 7f6d266b0700 1 -- 172.21.15.78:0/3572212933 shutdown_connections 2023-10-11T09:51:21.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:21.572+0000 7f6d266b0700 1 -- 172.21.15.78:0/3572212933 wait complete. 2023-10-11T09:51:21.583 INFO:teuthology.orchestra.run.smithi078.stderr:listed 6 entries 2023-10-11T09:51:21.601 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi078.front.sepia.ceph.com... 2023-10-11T09:51:21.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:51:21.603 DEBUG:teuthology.orchestra.run.smithi078:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-11T09:51:21.632 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:ceph-fuse[57751]: fuse finished with error 0 and tester_r 0 2023-10-11T09:51:21.640 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2023-10-11T09:51:21.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:51:21.641 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-11T09:51:21.737 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:51:21.738 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:51:21.757 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352292k,nr_inodes=4088073,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1430 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275376k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-11T09:51:21.757 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2023-10-11T09:51:21.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:51:21.758 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-11T09:51:21.817 INFO:teuthology.orchestra.run.smithi133.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-10-11T09:51:21.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T09:51:21.819 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:51:21.820 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:51:21.838 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352272k,nr_inodes=4088068,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17487 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-11T09:51:21.838 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2023-10-11T09:51:21.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:51:21.839 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-11T09:51:21.899 INFO:teuthology.orchestra.run.smithi078.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-11T09:51:21.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T09:51:21.901 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:51:21.901 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:51:21.920 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352292k,nr_inodes=4088073,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1430 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275376k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-11T09:51:21.921 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2023-10-11T09:51:21.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:51:21.921 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-11T09:51:21.980 INFO:teuthology.orchestra.run.smithi133.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-11T09:51:21.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T09:51:21.982 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:51:22.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.144+0000 7f95a4dba700 1 Processor -- start 2023-10-11T09:51:22.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.144+0000 7f95a4dba700 1 -- start start 2023-10-11T09:51:22.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.148+0000 7f95a4dba700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f95a0071040 0x7f95a00cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:22.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.148+0000 7f95a4dba700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95a00720f0 con 0x7f95a00cba00 2023-10-11T09:51:22.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.148+0000 7f95a4dba700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95a00714e0 con 0x7f95a0154e00 2023-10-11T09:51:22.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.148+0000 7f95a4dba700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95a00717c0 con 0x7f95a0071040 2023-10-11T09:51:22.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.148+0000 7f959ed9d700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f95a00cba00 0x7f95a006c010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33648/0 (socket says 172.21.15.78:33648) 2023-10-11T09:51:22.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.148+0000 7f959ed9d700 1 -- 172.21.15.78:0/3125194254 learned_addr learned my addr 172.21.15.78:0/3125194254 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:51:22.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.148+0000 7f9595d9b700 1 --2- 172.21.15.78:0/3125194254 >> v2:172.21.15.133:3300/0 conn(0x7f95a0071040 0x7f95a00cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:22.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.148+0000 7f959dd9b700 1 -- 172.21.15.78:0/3125194254 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1316437978 0 0) 0x7f95a00720f0 con 0x7f95a00cba00 2023-10-11T09:51:22.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.148+0000 7f959dd9b700 1 -- 172.21.15.78:0/3125194254 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f958c0035d0 con 0x7f95a00cba00 2023-10-11T09:51:22.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.148+0000 7f9595d9b700 1 -- 172.21.15.78:0/3125194254 >> v1:172.21.15.133:6789/0 conn(0x7f95a0154e00 legacy=0x7f95a015b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:22.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.148+0000 7f9595d9b700 1 -- 172.21.15.78:0/3125194254 >> v1:172.21.15.78:6789/0 conn(0x7f95a00cba00 legacy=0x7f95a006c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:22.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.148+0000 7f9595d9b700 1 -- 172.21.15.78:0/3125194254 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a0071aa0 con 0x7f95a0071040 2023-10-11T09:51:22.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.148+0000 7f9595d9b700 1 --2- 172.21.15.78:0/3125194254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95a0071040 0x7f95a00cb4c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9588009850 tx=0x7f9588009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3eb0e9462683457c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:22.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.148+0000 7f959dd9b700 1 -- 172.21.15.78:0/3125194254 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9588011830 con 0x7f95a0071040 2023-10-11T09:51:22.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.148+0000 7f959dd9b700 1 -- 172.21.15.78:0/3125194254 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9588011990 con 0x7f95a0071040 2023-10-11T09:51:22.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.148+0000 7f959dd9b700 1 -- 172.21.15.78:0/3125194254 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9588011d40 con 0x7f95a0071040 2023-10-11T09:51:22.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.152+0000 7f95a4dba700 1 -- 172.21.15.78:0/3125194254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95a0071040 msgr2=0x7f95a00cb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:22.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.152+0000 7f95a4dba700 1 --2- 172.21.15.78:0/3125194254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95a0071040 0x7f95a00cb4c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9588009850 tx=0x7f9588009b60 comp rx=0 tx=0).stop 2023-10-11T09:51:22.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.152+0000 7f95a4dba700 1 -- 172.21.15.78:0/3125194254 shutdown_connections 2023-10-11T09:51:22.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.152+0000 7f95a4dba700 1 --2- 172.21.15.78:0/3125194254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95a0071040 0x7f95a00cb4c0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:22.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.152+0000 7f95a4dba700 1 -- 172.21.15.78:0/3125194254 >> 172.21.15.78:0/3125194254 conn(0x7f95a006aca0 msgr2=0x7f95a00b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:22.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.152+0000 7f95a4dba700 1 -- 172.21.15.78:0/3125194254 shutdown_connections 2023-10-11T09:51:22.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.152+0000 7f95a4dba700 1 -- 172.21.15.78:0/3125194254 wait complete. 2023-10-11T09:51:22.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.156+0000 7f95a4dba700 1 Processor -- start 2023-10-11T09:51:22.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.156+0000 7f95a4dba700 1 -- start start 2023-10-11T09:51:22.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.156+0000 7f95a4dba700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95a0071040 0x7f95a0143080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:22.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.156+0000 7f95a4dba700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95a00cba00 0x7f95a01420a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:22.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.156+0000 7f95a4dba700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95a0154e00 0x7f95a00ca060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:22.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.156+0000 7f95a4dba700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a015aa40 con 0x7f95a0071040 2023-10-11T09:51:22.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.156+0000 7f95a4dba700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a00ca5a0 con 0x7f95a00cba00 2023-10-11T09:51:22.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.156+0000 7f95a4dba700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95a0153200 con 0x7f95a0154e00 2023-10-11T09:51:22.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.156+0000 7f959e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95a0071040 0x7f95a0143080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:22.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.156+0000 7f959e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95a0071040 0x7f95a0143080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45550/0 (socket says 172.21.15.78:45550) 2023-10-11T09:51:22.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.156+0000 7f959e59c700 1 -- 172.21.15.78:0/1941492652 learned_addr learned my addr 172.21.15.78:0/1941492652 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:22.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.156+0000 7f9595d9b700 1 --2- 172.21.15.78:0/1941492652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95a00cba00 0x7f95a01420a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:22.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.156+0000 7f959ed9d700 1 --2- 172.21.15.78:0/1941492652 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95a0154e00 0x7f95a00ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:22.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.156+0000 7f959e59c700 1 -- 172.21.15.78:0/1941492652 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95a0154e00 msgr2=0x7f95a00ca060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:22.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.156+0000 7f959e59c700 1 --2- 172.21.15.78:0/1941492652 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95a0154e00 0x7f95a00ca060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:22.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.156+0000 7f959e59c700 1 -- 172.21.15.78:0/1941492652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95a00cba00 msgr2=0x7f95a01420a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:22.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.156+0000 7f959e59c700 1 --2- 172.21.15.78:0/1941492652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95a00cba00 0x7f95a01420a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:22.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.156+0000 7f959e59c700 1 -- 172.21.15.78:0/1941492652 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95880094b0 con 0x7f95a0071040 2023-10-11T09:51:22.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.156+0000 7f959ed9d700 1 --2- 172.21.15.78:0/1941492652 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95a0154e00 0x7f95a00ca060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:51:22.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.156+0000 7f959e59c700 1 --2- 172.21.15.78:0/1941492652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95a0071040 0x7f95a0143080 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f959000cc00 tx=0x7f959000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:22.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.156+0000 7f95977fe700 1 -- 172.21.15.78:0/1941492652 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9590003c20 con 0x7f95a0071040 2023-10-11T09:51:22.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.156+0000 7f95977fe700 1 -- 172.21.15.78:0/1941492652 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9590003d80 con 0x7f95a0071040 2023-10-11T09:51:22.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.156+0000 7f95977fe700 1 -- 172.21.15.78:0/1941492652 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959000abe0 con 0x7f95a0071040 2023-10-11T09:51:22.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.156+0000 7f95a4dba700 1 -- 172.21.15.78:0/1941492652 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95a01534e0 con 0x7f95a0071040 2023-10-11T09:51:22.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.156+0000 7f95a4dba700 1 -- 172.21.15.78:0/1941492652 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95a0153aa0 con 0x7f95a0071040 2023-10-11T09:51:22.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.160+0000 7f95a4dba700 1 -- 172.21.15.78:0/1941492652 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95a01425e0 con 0x7f95a0071040 2023-10-11T09:51:22.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.160+0000 7f95977fe700 1 -- 172.21.15.78:0/1941492652 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f959000ad40 con 0x7f95a0071040 2023-10-11T09:51:22.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.160+0000 7f95977fe700 1 --2- 172.21.15.78:0/1941492652 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9580065ed0 0x7f9580068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:22.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.160+0000 7f95977fe700 1 -- 172.21.15.78:0/1941492652 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f9590083b70 con 0x7f95a0071040 2023-10-11T09:51:22.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.164+0000 7f9595d9b700 1 --2- 172.21.15.78:0/1941492652 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9580065ed0 0x7f9580068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:22.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.164+0000 7f9595d9b700 1 --2- 172.21.15.78:0/1941492652 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9580065ed0 0x7f9580068380 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f95a0071410 tx=0x7f9588008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:22.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.168+0000 7f95977fe700 1 -- 172.21.15.78:0/1941492652 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f9590050f30 con 0x7f95a0071040 2023-10-11T09:51:22.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.376+0000 7f95a4dba700 1 -- 172.21.15.78:0/1941492652 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95a0155260 con 0x7f95a0071040 2023-10-11T09:51:22.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.376+0000 7f95977fe700 1 -- 172.21.15.78:0/1941492652 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4655 (secure 0 0 0) 0x7f9590054550 con 0x7f95a0071040 2023-10-11T09:51:22.387 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:22.387 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":91,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7148,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/331520590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":331520590},{"type":"v1","addr":"172.21.15.78:6835","nonce":331520590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":7151,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/934539531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":934539531},{"type":"v1","addr":"172.21.15.133:6834","nonce":934539531}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":7154,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3894002128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":3894002128},{"type":"v1","addr":"172.21.15.133:6835","nonce":3894002128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:50:28.502984+0000","modified":"2023-10-11T09:50:37.659950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8550":{"gid":8550,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":4,"addr":"172.21.15.78:6837/2869034036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2869034036},{"type":"v1","addr":"172.21.15.78:6837","nonce":2869034036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15,16],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-11T09:51:22.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.380+0000 7f95a4dba700 1 -- 172.21.15.78:0/1941492652 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9580065ed0 msgr2=0x7f9580068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:22.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.380+0000 7f95a4dba700 1 --2- 172.21.15.78:0/1941492652 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9580065ed0 0x7f9580068380 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f95a0071410 tx=0x7f9588008030 comp rx=0 tx=0).stop 2023-10-11T09:51:22.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.380+0000 7f95a4dba700 1 -- 172.21.15.78:0/1941492652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95a0071040 msgr2=0x7f95a0143080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:22.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.380+0000 7f95a4dba700 1 --2- 172.21.15.78:0/1941492652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95a0071040 0x7f95a0143080 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f959000cc00 tx=0x7f959000a060 comp rx=0 tx=0).stop 2023-10-11T09:51:22.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.380+0000 7f95a4dba700 1 -- 172.21.15.78:0/1941492652 shutdown_connections 2023-10-11T09:51:22.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.380+0000 7f95a4dba700 1 --2- 172.21.15.78:0/1941492652 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9580065ed0 0x7f9580068380 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:22.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.380+0000 7f95a4dba700 1 --2- 172.21.15.78:0/1941492652 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95a0154e00 0x7f95a00ca060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:22.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.380+0000 7f95a4dba700 1 --2- 172.21.15.78:0/1941492652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95a00cba00 0x7f95a01420a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:22.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.380+0000 7f95a4dba700 1 --2- 172.21.15.78:0/1941492652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95a0071040 0x7f95a0143080 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:22.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.380+0000 7f95a4dba700 1 -- 172.21.15.78:0/1941492652 >> 172.21.15.78:0/1941492652 conn(0x7f95a006aca0 msgr2=0x7f95a014a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:22.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.380+0000 7f95a4dba700 1 -- 172.21.15.78:0/1941492652 shutdown_connections 2023-10-11T09:51:22.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.380+0000 7f95a4dba700 1 -- 172.21.15.78:0/1941492652 wait complete. 2023-10-11T09:51:22.394 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 91 2023-10-11T09:51:22.413 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:51:22.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.568+0000 7fd62d8a7700 1 Processor -- start 2023-10-11T09:51:22.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.568+0000 7fd62d8a7700 1 -- start start 2023-10-11T09:51:22.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.572+0000 7fd62d8a7700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd62815c730 0x7fd6280c6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:22.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.572+0000 7fd62d8a7700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd628160300 con 0x7fd6280cd140 2023-10-11T09:51:22.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.572+0000 7fd62d8a7700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd628160e80 con 0x7fd6280cccf0 2023-10-11T09:51:22.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.572+0000 7fd62d8a7700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6280b2e70 con 0x7fd62815c730 2023-10-11T09:51:22.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.572+0000 7fd627fff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fd6280cd140 0x7fd6280b2760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33652/0 (socket says 172.21.15.78:33652) 2023-10-11T09:51:22.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.572+0000 7fd627fff700 1 -- 172.21.15.78:0/2624011418 learned_addr learned my addr 172.21.15.78:0/2624011418 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:51:22.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.572+0000 7fd626ffd700 1 --2- 172.21.15.78:0/2624011418 >> v2:172.21.15.133:3300/0 conn(0x7fd62815c730 0x7fd6280c6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:22.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.572+0000 7fd6267fc700 1 -- 172.21.15.78:0/2624011418 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1507733136 0 0) 0x7fd628160300 con 0x7fd6280cd140 2023-10-11T09:51:22.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.572+0000 7fd6267fc700 1 -- 172.21.15.78:0/2624011418 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6140035d0 con 0x7fd6280cd140 2023-10-11T09:51:22.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.572+0000 7fd626ffd700 1 -- 172.21.15.78:0/2624011418 >> v1:172.21.15.133:6789/0 conn(0x7fd6280cccf0 legacy=0x7fd6280b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:22.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.572+0000 7fd6267fc700 1 -- 172.21.15.78:0/2624011418 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2801306390 0 0) 0x7fd628160e80 con 0x7fd6280cccf0 2023-10-11T09:51:22.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.572+0000 7fd626ffd700 1 -- 172.21.15.78:0/2624011418 >> v1:172.21.15.78:6789/0 conn(0x7fd6280cd140 legacy=0x7fd6280b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:22.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.572+0000 7fd626ffd700 1 -- 172.21.15.78:0/2624011418 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd628160550 con 0x7fd62815c730 2023-10-11T09:51:22.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.576+0000 7fd626ffd700 1 --2- 172.21.15.78:0/2624011418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd62815c730 0x7fd6280c6a30 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd618009d90 tx=0x7fd61800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7cfce54b8aa916e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:22.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.576+0000 7fd6267fc700 1 -- 172.21.15.78:0/2624011418 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61800e070 con 0x7fd62815c730 2023-10-11T09:51:22.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.576+0000 7fd6267fc700 1 -- 172.21.15.78:0/2624011418 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd61800bb20 con 0x7fd62815c730 2023-10-11T09:51:22.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.576+0000 7fd6267fc700 1 -- 172.21.15.78:0/2624011418 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61800bed0 con 0x7fd62815c730 2023-10-11T09:51:22.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.576+0000 7fd62d8a7700 1 -- 172.21.15.78:0/2624011418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd62815c730 msgr2=0x7fd6280c6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:22.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.576+0000 7fd62d8a7700 1 --2- 172.21.15.78:0/2624011418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd62815c730 0x7fd6280c6a30 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd618009d90 tx=0x7fd61800b330 comp rx=0 tx=0).stop 2023-10-11T09:51:22.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.576+0000 7fd62d8a7700 1 -- 172.21.15.78:0/2624011418 shutdown_connections 2023-10-11T09:51:22.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.576+0000 7fd62d8a7700 1 --2- 172.21.15.78:0/2624011418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd62815c730 0x7fd6280c6a30 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:22.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.576+0000 7fd62d8a7700 1 -- 172.21.15.78:0/2624011418 >> 172.21.15.78:0/2624011418 conn(0x7fd62806dd80 msgr2=0x7fd62806e150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:22.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.576+0000 7fd62d8a7700 1 -- 172.21.15.78:0/2624011418 shutdown_connections 2023-10-11T09:51:22.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.576+0000 7fd62d8a7700 1 -- 172.21.15.78:0/2624011418 wait complete. 2023-10-11T09:51:22.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.580+0000 7fd62d8a7700 1 Processor -- start 2023-10-11T09:51:22.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.580+0000 7fd62d8a7700 1 -- start start 2023-10-11T09:51:22.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.580+0000 7fd62d8a7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6280cccf0 0x7fd6280af270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:22.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.580+0000 7fd62d8a7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd6280cd140 0x7fd6280c74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:22.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.580+0000 7fd62d8a7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd62815c730 0x7fd6281f2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:22.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.580+0000 7fd62d8a7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6280c8b60 con 0x7fd62815c730 2023-10-11T09:51:22.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.580+0000 7fd62d8a7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6280c8880 con 0x7fd6280cccf0 2023-10-11T09:51:22.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.580+0000 7fd62d8a7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6280c7ce0 con 0x7fd6280cd140 2023-10-11T09:51:22.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.580+0000 7fd627fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd62815c730 0x7fd6281f2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:22.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.580+0000 7fd627fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd62815c730 0x7fd6281f2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45562/0 (socket says 172.21.15.78:45562) 2023-10-11T09:51:22.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.580+0000 7fd627fff700 1 -- 172.21.15.78:0/2974872143 learned_addr learned my addr 172.21.15.78:0/2974872143 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:22.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.580+0000 7fd61e7fc700 1 --2- 172.21.15.78:0/2974872143 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd6280cd140 0x7fd6280c74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:22.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.580+0000 7fd627fff700 1 -- 172.21.15.78:0/2974872143 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd6280cd140 msgr2=0x7fd6280c74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:22.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.580+0000 7fd626ffd700 1 --2- 172.21.15.78:0/2974872143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6280cccf0 0x7fd6280af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:22.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.580+0000 7fd627fff700 1 --2- 172.21.15.78:0/2974872143 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd6280cd140 0x7fd6280c74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:22.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.580+0000 7fd627fff700 1 -- 172.21.15.78:0/2974872143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6280cccf0 msgr2=0x7fd6280af270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:22.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.580+0000 7fd627fff700 1 --2- 172.21.15.78:0/2974872143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6280cccf0 0x7fd6280af270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:22.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.580+0000 7fd627fff700 1 -- 172.21.15.78:0/2974872143 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd618009a70 con 0x7fd62815c730 2023-10-11T09:51:22.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.580+0000 7fd61e7fc700 1 --2- 172.21.15.78:0/2974872143 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd6280cd140 0x7fd6280c74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:51:22.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.580+0000 7fd627fff700 1 --2- 172.21.15.78:0/2974872143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd62815c730 0x7fd6281f2f00 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fd62000c4f0 tx=0x7fd62000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:22.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.580+0000 7fd624ff9700 1 -- 172.21.15.78:0/2974872143 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd620005d90 con 0x7fd62815c730 2023-10-11T09:51:22.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.580+0000 7fd624ff9700 1 -- 172.21.15.78:0/2974872143 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6200070d0 con 0x7fd62815c730 2023-10-11T09:51:22.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.580+0000 7fd62d8a7700 1 -- 172.21.15.78:0/2974872143 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6280c7fc0 con 0x7fd62815c730 2023-10-11T09:51:22.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.580+0000 7fd624ff9700 1 -- 172.21.15.78:0/2974872143 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd620007460 con 0x7fd62815c730 2023-10-11T09:51:22.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.580+0000 7fd62d8a7700 1 -- 172.21.15.78:0/2974872143 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6280c79f0 con 0x7fd62815c730 2023-10-11T09:51:22.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.584+0000 7fd62d8a7700 1 -- 172.21.15.78:0/2974872143 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6280c84e0 con 0x7fd62815c730 2023-10-11T09:51:22.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.584+0000 7fd624ff9700 1 -- 172.21.15.78:0/2974872143 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fd6200075c0 con 0x7fd62815c730 2023-10-11T09:51:22.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.588+0000 7fd624ff9700 1 --2- 172.21.15.78:0/2974872143 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd608065ed0 0x7fd608068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:22.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.588+0000 7fd624ff9700 1 -- 172.21.15.78:0/2974872143 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6724+0+0 (secure 0 0 0) 0x7fd6200830b0 con 0x7fd62815c730 2023-10-11T09:51:22.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.588+0000 7fd626ffd700 1 --2- 172.21.15.78:0/2974872143 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd608065ed0 0x7fd608068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:22.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.588+0000 7fd626ffd700 1 --2- 172.21.15.78:0/2974872143 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd608065ed0 0x7fd608068380 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fd618009fd0 tx=0x7fd618011000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:22.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.596+0000 7fd624ff9700 1 -- 172.21.15.78:0/2974872143 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fd6200503c0 con 0x7fd62815c730 2023-10-11T09:51:22.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.772+0000 7fd62d8a7700 1 -- 172.21.15.78:0/2974872143 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6280c8d10 con 0x7fd62815c730 2023-10-11T09:51:22.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.776+0000 7fd624ff9700 1 -- 172.21.15.78:0/2974872143 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+20012 (secure 0 0 0) 0x7fd6280c8d10 con 0x7fd62815c730 2023-10-11T09:51:22.787 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:22.787 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":114,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:50:34.654747+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:49:49.297597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-11T09:49:49.955104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data_extra","create_time":"2023-10-11T09:49:59.915561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6836/2876755099":"2023-10-12T09:50:34.623416+0000","172.21.15.78:6837/1390949105":"2023-10-12T09:50:28.599447+0000","172.21.15.78:6836/1390949105":"2023-10-12T09:50:28.599447+0000","172.21.15.78:6837/2876755099":"2023-10-12T09:50:34.623416+0000","172.21.15.78:6837/1286798022":"2023-10-12T09:50:27.559697+0000","172.21.15.78:6836/1286798022":"2023-10-12T09:50:27.559697+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:51:22.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.780+0000 7fd62d8a7700 1 -- 172.21.15.78:0/2974872143 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd608065ed0 msgr2=0x7fd608068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:22.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.780+0000 7fd62d8a7700 1 --2- 172.21.15.78:0/2974872143 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd608065ed0 0x7fd608068380 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fd618009fd0 tx=0x7fd618011000 comp rx=0 tx=0).stop 2023-10-11T09:51:22.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.780+0000 7fd62d8a7700 1 -- 172.21.15.78:0/2974872143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd62815c730 msgr2=0x7fd6281f2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:22.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.780+0000 7fd62d8a7700 1 --2- 172.21.15.78:0/2974872143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd62815c730 0x7fd6281f2f00 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fd62000c4f0 tx=0x7fd62000c8b0 comp rx=0 tx=0).stop 2023-10-11T09:51:22.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.780+0000 7fd62d8a7700 1 -- 172.21.15.78:0/2974872143 shutdown_connections 2023-10-11T09:51:22.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.780+0000 7fd62d8a7700 1 --2- 172.21.15.78:0/2974872143 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd608065ed0 0x7fd608068380 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:22.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.780+0000 7fd62d8a7700 1 --2- 172.21.15.78:0/2974872143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd62815c730 0x7fd6281f2f00 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:22.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.780+0000 7fd62d8a7700 1 --2- 172.21.15.78:0/2974872143 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd6280cd140 0x7fd6280c74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:22.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.780+0000 7fd62d8a7700 1 --2- 172.21.15.78:0/2974872143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6280cccf0 0x7fd6280af270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:22.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.780+0000 7fd62d8a7700 1 -- 172.21.15.78:0/2974872143 >> 172.21.15.78:0/2974872143 conn(0x7fd62806dd80 msgr2=0x7fd628156b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:22.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.780+0000 7fd62d8a7700 1 -- 172.21.15.78:0/2974872143 shutdown_connections 2023-10-11T09:51:22.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.780+0000 7fd62d8a7700 1 -- 172.21.15.78:0/2974872143 wait complete. 2023-10-11T09:51:22.809 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T09:49:49.297597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T09:49:49.955104+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-10-11T09:49:59.915561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:51:22.810 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:51:22.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.980+0000 7ff624851700 1 Processor -- start 2023-10-11T09:51:22.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.980+0000 7ff624851700 1 -- start start 2023-10-11T09:51:22.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.980+0000 7ff624851700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ff620158430 0x7ff6200710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:22.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.980+0000 7ff624851700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff6200caf90 0x7ff62015afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:22.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.980+0000 7ff624851700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff620071610 con 0x7ff62006d340 2023-10-11T09:51:22.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.980+0000 7ff624851700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff620070c10 con 0x7ff620158430 2023-10-11T09:51:22.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.980+0000 7ff624851700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff62015e280 con 0x7ff6200caf90 2023-10-11T09:51:22.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.980+0000 7ff61e59c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ff620158430 0x7ff6200710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:22.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.980+0000 7ff61e59c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ff620158430 0x7ff6200710d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45576/0 (socket says 172.21.15.78:45576) 2023-10-11T09:51:22.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.980+0000 7ff61e59c700 1 -- 172.21.15.78:0/340111142 learned_addr learned my addr 172.21.15.78:0/340111142 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:22.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.980+0000 7ff61ed9d700 1 --2- 172.21.15.78:0/340111142 >> v2:172.21.15.133:3300/0 conn(0x7ff6200caf90 0x7ff62015afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:22.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.980+0000 7ff61e59c700 1 -- 172.21.15.78:0/340111142 >> v2:172.21.15.133:3300/0 conn(0x7ff6200caf90 msgr2=0x7ff62015afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:22.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.980+0000 7ff61e59c700 1 --2- 172.21.15.78:0/340111142 >> v2:172.21.15.133:3300/0 conn(0x7ff6200caf90 0x7ff62015afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:22.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.984+0000 7ff61e59c700 1 -- 172.21.15.78:0/340111142 >> v1:172.21.15.133:6789/0 conn(0x7ff62006d340 legacy=0x7ff6200ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:22.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.984+0000 7ff61e59c700 1 -- 172.21.15.78:0/340111142 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff62015e560 con 0x7ff620158430 2023-10-11T09:51:22.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.984+0000 7ff61e59c700 1 --2- 172.21.15.78:0/340111142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff620158430 0x7ff6200710d0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7ff62006e310 tx=0x7ff61000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c0f6d49c1984b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:22.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.984+0000 7ff61d59a700 1 -- 172.21.15.78:0/340111142 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61000e070 con 0x7ff620158430 2023-10-11T09:51:22.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.984+0000 7ff61d59a700 1 -- 172.21.15.78:0/340111142 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff61000b9c0 con 0x7ff620158430 2023-10-11T09:51:22.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.984+0000 7ff61d59a700 1 -- 172.21.15.78:0/340111142 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61000bd70 con 0x7ff620158430 2023-10-11T09:51:22.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.984+0000 7ff624851700 1 -- 172.21.15.78:0/340111142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff620158430 msgr2=0x7ff6200710d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:22.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.984+0000 7ff624851700 1 --2- 172.21.15.78:0/340111142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff620158430 0x7ff6200710d0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7ff62006e310 tx=0x7ff61000b330 comp rx=0 tx=0).stop 2023-10-11T09:51:22.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.984+0000 7ff624851700 1 -- 172.21.15.78:0/340111142 shutdown_connections 2023-10-11T09:51:22.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.984+0000 7ff624851700 1 --2- 172.21.15.78:0/340111142 >> v2:172.21.15.133:3300/0 conn(0x7ff6200caf90 0x7ff62015afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:22.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.984+0000 7ff624851700 1 --2- 172.21.15.78:0/340111142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff620158430 0x7ff6200710d0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:22.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.984+0000 7ff624851700 1 -- 172.21.15.78:0/340111142 >> 172.21.15.78:0/340111142 conn(0x7ff6200cecc0 msgr2=0x7ff6200cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:22.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.984+0000 7ff624851700 1 -- 172.21.15.78:0/340111142 shutdown_connections 2023-10-11T09:51:22.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.984+0000 7ff624851700 1 -- 172.21.15.78:0/340111142 wait complete. 2023-10-11T09:51:22.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.988+0000 7ff624851700 1 Processor -- start 2023-10-11T09:51:22.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.988+0000 7ff624851700 1 -- start start 2023-10-11T09:51:22.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.988+0000 7ff624851700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff62006d340 0x7ff6201538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:22.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.988+0000 7ff624851700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6200caf90 0x7ff620154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:22.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.988+0000 7ff624851700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff620158430 0x7ff620155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:23.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.988+0000 7ff624851700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff620153e00 con 0x7ff6200caf90 2023-10-11T09:51:23.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.988+0000 7ff624851700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff620155b60 con 0x7ff62006d340 2023-10-11T09:51:23.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.988+0000 7ff624851700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff620155e40 con 0x7ff620158430 2023-10-11T09:51:23.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.988+0000 7ff61dd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6200caf90 0x7ff620154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:23.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.988+0000 7ff61dd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6200caf90 0x7ff620154fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45592/0 (socket says 172.21.15.78:45592) 2023-10-11T09:51:23.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.988+0000 7ff61e59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff62006d340 0x7ff6201538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:23.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.988+0000 7ff61dd9b700 1 -- 172.21.15.78:0/1072450695 learned_addr learned my addr 172.21.15.78:0/1072450695 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:23.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.988+0000 7ff61ed9d700 1 --2- 172.21.15.78:0/1072450695 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff620158430 0x7ff620155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:23.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.992+0000 7ff61dd9b700 1 -- 172.21.15.78:0/1072450695 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff620158430 msgr2=0x7ff620155520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:23.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.992+0000 7ff61dd9b700 1 --2- 172.21.15.78:0/1072450695 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff620158430 0x7ff620155520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:23.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.992+0000 7ff61dd9b700 1 -- 172.21.15.78:0/1072450695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff62006d340 msgr2=0x7ff6201538c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:23.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.992+0000 7ff61dd9b700 1 --2- 172.21.15.78:0/1072450695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff62006d340 0x7ff6201538c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:23.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.992+0000 7ff61dd9b700 1 -- 172.21.15.78:0/1072450695 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff610009b30 con 0x7ff6200caf90 2023-10-11T09:51:23.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.992+0000 7ff61dd9b700 1 --2- 172.21.15.78:0/1072450695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6200caf90 0x7ff620154fe0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7ff61400c500 tx=0x7ff61400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:23.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.992+0000 7ff60effd700 1 -- 172.21.15.78:0/1072450695 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6140052d0 con 0x7ff6200caf90 2023-10-11T09:51:23.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.992+0000 7ff60effd700 1 -- 172.21.15.78:0/1072450695 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff614005430 con 0x7ff6200caf90 2023-10-11T09:51:23.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.992+0000 7ff60effd700 1 -- 172.21.15.78:0/1072450695 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6140055a0 con 0x7ff6200caf90 2023-10-11T09:51:23.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.992+0000 7ff624851700 1 -- 172.21.15.78:0/1072450695 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff620156120 con 0x7ff6200caf90 2023-10-11T09:51:23.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.992+0000 7ff624851700 1 -- 172.21.15.78:0/1072450695 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6201f22f0 con 0x7ff6200caf90 2023-10-11T09:51:23.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.992+0000 7ff624851700 1 -- 172.21.15.78:0/1072450695 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff620156640 con 0x7ff6200caf90 2023-10-11T09:51:23.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.996+0000 7ff60effd700 1 -- 172.21.15.78:0/1072450695 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7ff6140038e0 con 0x7ff6200caf90 2023-10-11T09:51:23.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.996+0000 7ff60effd700 1 --2- 172.21.15.78:0/1072450695 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff600065ed0 0x7ff600068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:23.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.996+0000 7ff60effd700 1 -- 172.21.15.78:0/1072450695 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6724+0+0 (secure 0 0 0) 0x7ff614082ec0 con 0x7ff6200caf90 2023-10-11T09:51:23.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.996+0000 7ff61e59c700 1 --2- 172.21.15.78:0/1072450695 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff600065ed0 0x7ff600068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:23.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:22.996+0000 7ff61e59c700 1 --2- 172.21.15.78:0/1072450695 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff600065ed0 0x7ff600068380 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff610000f80 tx=0x7ff610009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:23.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.004+0000 7ff60effd700 1 -- 172.21.15.78:0/1072450695 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7ff620156640 con 0x7ff6200caf90 2023-10-11T09:51:23.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.176+0000 7ff624851700 1 -- 172.21.15.78:0/1072450695 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff620156640 con 0x7ff6200caf90 2023-10-11T09:51:23.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.176+0000 7ff60effd700 1 -- 172.21.15.78:0/1072450695 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v114) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff620156640 con 0x7ff6200caf90 2023-10-11T09:51:23.188 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:51:23.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.180+0000 7ff624851700 1 -- 172.21.15.78:0/1072450695 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff600065ed0 msgr2=0x7ff600068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:23.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.180+0000 7ff624851700 1 --2- 172.21.15.78:0/1072450695 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff600065ed0 0x7ff600068380 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff610000f80 tx=0x7ff610009000 comp rx=0 tx=0).stop 2023-10-11T09:51:23.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.180+0000 7ff624851700 1 -- 172.21.15.78:0/1072450695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6200caf90 msgr2=0x7ff620154fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:23.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.180+0000 7ff624851700 1 --2- 172.21.15.78:0/1072450695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6200caf90 0x7ff620154fe0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7ff61400c500 tx=0x7ff61400c810 comp rx=0 tx=0).stop 2023-10-11T09:51:23.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.180+0000 7ff624851700 1 -- 172.21.15.78:0/1072450695 shutdown_connections 2023-10-11T09:51:23.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.180+0000 7ff624851700 1 --2- 172.21.15.78:0/1072450695 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff600065ed0 0x7ff600068380 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:23.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.180+0000 7ff624851700 1 --2- 172.21.15.78:0/1072450695 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff620158430 0x7ff620155520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:23.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.180+0000 7ff624851700 1 --2- 172.21.15.78:0/1072450695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6200caf90 0x7ff620154fe0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:23.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.180+0000 7ff624851700 1 --2- 172.21.15.78:0/1072450695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff62006d340 0x7ff6201538c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:23.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.180+0000 7ff624851700 1 -- 172.21.15.78:0/1072450695 >> 172.21.15.78:0/1072450695 conn(0x7ff6200cecc0 msgr2=0x7ff62015a170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:23.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.180+0000 7ff624851700 1 -- 172.21.15.78:0/1072450695 shutdown_connections 2023-10-11T09:51:23.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.180+0000 7ff624851700 1 -- 172.21.15.78:0/1072450695 wait complete. 2023-10-11T09:51:23.212 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:51:23.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.376+0000 7f488587b700 1 Processor -- start 2023-10-11T09:51:23.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.376+0000 7f488587b700 1 -- start start 2023-10-11T09:51:23.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.380+0000 7f488587b700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f488015c900 0x7f48800b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:23.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.380+0000 7f488587b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f48800ce7f0 0x7f48800cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:23.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.380+0000 7f488587b700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f488006fde0 con 0x7f48800ad290 2023-10-11T09:51:23.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.380+0000 7f488587b700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48800705f0 con 0x7f48800ce7f0 2023-10-11T09:51:23.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.380+0000 7f488587b700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48800cf1b0 con 0x7f488015c900 2023-10-11T09:51:23.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.380+0000 7f487f7fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f48800ce7f0 0x7f48800cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:23.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.380+0000 7f48767fc700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f48800ad290 0x7f4880071a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33660/0 (socket says 172.21.15.78:33660) 2023-10-11T09:51:23.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.380+0000 7f48767fc700 1 -- 172.21.15.78:0/948056444 learned_addr learned my addr 172.21.15.78:0/948056444 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:51:23.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.380+0000 7f487effd700 1 --2- 172.21.15.78:0/948056444 >> v2:172.21.15.133:3300/0 conn(0x7f488015c900 0x7f48800b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:23.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.380+0000 7f487f7fe700 1 -- 172.21.15.78:0/948056444 >> v2:172.21.15.133:3300/0 conn(0x7f488015c900 msgr2=0x7f48800b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:23.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.380+0000 7f487f7fe700 1 --2- 172.21.15.78:0/948056444 >> v2:172.21.15.133:3300/0 conn(0x7f488015c900 0x7f48800b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:23.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.380+0000 7f487f7fe700 1 -- 172.21.15.78:0/948056444 >> v1:172.21.15.78:6789/0 conn(0x7f48800ad290 legacy=0x7f4880071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:23.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.380+0000 7f487f7fe700 1 -- 172.21.15.78:0/948056444 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48800ae000 con 0x7f48800ce7f0 2023-10-11T09:51:23.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.380+0000 7f487e7fc700 1 -- 172.21.15.78:0/948056444 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2956375159 0 0) 0x7f488006fde0 con 0x7f48800ad290 2023-10-11T09:51:23.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.380+0000 7f487f7fe700 1 --2- 172.21.15.78:0/948056444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48800ce7f0 0x7f48800cec70 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f48800724b0 tx=0x7f487800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97878912a854bf78 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:23.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.380+0000 7f487e7fc700 1 -- 172.21.15.78:0/948056444 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487800e070 con 0x7f48800ce7f0 2023-10-11T09:51:23.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.384+0000 7f487e7fc700 1 -- 172.21.15.78:0/948056444 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f487800bb20 con 0x7f48800ce7f0 2023-10-11T09:51:23.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.384+0000 7f487e7fc700 1 -- 172.21.15.78:0/948056444 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4878012430 con 0x7f48800ce7f0 2023-10-11T09:51:23.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.384+0000 7f488587b700 1 -- 172.21.15.78:0/948056444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48800ce7f0 msgr2=0x7f48800cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:23.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.384+0000 7f488587b700 1 --2- 172.21.15.78:0/948056444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48800ce7f0 0x7f48800cec70 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f48800724b0 tx=0x7f487800b330 comp rx=0 tx=0).stop 2023-10-11T09:51:23.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.384+0000 7f488587b700 1 -- 172.21.15.78:0/948056444 shutdown_connections 2023-10-11T09:51:23.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.384+0000 7f488587b700 1 --2- 172.21.15.78:0/948056444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48800ce7f0 0x7f48800cec70 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:23.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.384+0000 7f488587b700 1 --2- 172.21.15.78:0/948056444 >> v2:172.21.15.133:3300/0 conn(0x7f488015c900 0x7f48800b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:23.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.384+0000 7f488587b700 1 -- 172.21.15.78:0/948056444 >> 172.21.15.78:0/948056444 conn(0x7f488006a060 msgr2=0x7f488006a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:23.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.384+0000 7f488587b700 1 -- 172.21.15.78:0/948056444 shutdown_connections 2023-10-11T09:51:23.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.384+0000 7f488587b700 1 -- 172.21.15.78:0/948056444 wait complete. 2023-10-11T09:51:23.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.388+0000 7f488587b700 1 Processor -- start 2023-10-11T09:51:23.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.388+0000 7f488587b700 1 -- start start 2023-10-11T09:51:23.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.388+0000 7f488587b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48800ad290 0x7f488015ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:23.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.388+0000 7f488587b700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48800ce7f0 0x7f488015f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:23.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.388+0000 7f488587b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f488015c900 0x7f488006c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:23.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.388+0000 7f488587b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488015d3a0 con 0x7f488015c900 2023-10-11T09:51:23.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.388+0000 7f488587b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48800aee90 con 0x7f48800ad290 2023-10-11T09:51:23.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.388+0000 7f488587b700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f488015fcc0 con 0x7f48800ce7f0 2023-10-11T09:51:23.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.388+0000 7f487f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f488015c900 0x7f488006c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:23.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.388+0000 7f487f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f488015c900 0x7f488006c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45600/0 (socket says 172.21.15.78:45600) 2023-10-11T09:51:23.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.388+0000 7f487f7fe700 1 -- 172.21.15.78:0/2462037027 learned_addr learned my addr 172.21.15.78:0/2462037027 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:23.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.388+0000 7f487effd700 1 --2- 172.21.15.78:0/2462037027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48800ad290 0x7f488015ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:23.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.388+0000 7f48767fc700 1 --2- 172.21.15.78:0/2462037027 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48800ce7f0 0x7f488015f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:23.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.388+0000 7f487f7fe700 1 -- 172.21.15.78:0/2462037027 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48800ce7f0 msgr2=0x7f488015f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:23.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.388+0000 7f487f7fe700 1 --2- 172.21.15.78:0/2462037027 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48800ce7f0 0x7f488015f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:23.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.388+0000 7f487f7fe700 1 -- 172.21.15.78:0/2462037027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48800ad290 msgr2=0x7f488015ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:23.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.388+0000 7f487f7fe700 1 --2- 172.21.15.78:0/2462037027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48800ad290 0x7f488015ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:23.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.388+0000 7f487f7fe700 1 -- 172.21.15.78:0/2462037027 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4878009a70 con 0x7f488015c900 2023-10-11T09:51:23.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.392+0000 7f487f7fe700 1 --2- 172.21.15.78:0/2462037027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f488015c900 0x7f488006c130 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f4878014040 tx=0x7f4878004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:23.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.392+0000 7f487cff9700 1 -- 172.21.15.78:0/2462037027 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4878012430 con 0x7f488015c900 2023-10-11T09:51:23.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.392+0000 7f487cff9700 1 -- 172.21.15.78:0/2462037027 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4878020070 con 0x7f488015c900 2023-10-11T09:51:23.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.392+0000 7f488587b700 1 -- 172.21.15.78:0/2462037027 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f488015ff10 con 0x7f488015c900 2023-10-11T09:51:23.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.392+0000 7f487cff9700 1 -- 172.21.15.78:0/2462037027 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4878004280 con 0x7f488015c900 2023-10-11T09:51:23.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.392+0000 7f488587b700 1 -- 172.21.15.78:0/2462037027 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48800ae840 con 0x7f488015c900 2023-10-11T09:51:23.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.392+0000 7f488587b700 1 -- 172.21.15.78:0/2462037027 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4880065d40 con 0x7f488015c900 2023-10-11T09:51:23.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.396+0000 7f487cff9700 1 -- 172.21.15.78:0/2462037027 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f48780043e0 con 0x7f488015c900 2023-10-11T09:51:23.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.396+0000 7f487cff9700 1 --2- 172.21.15.78:0/2462037027 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4860065ed0 0x7f4860068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:23.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.396+0000 7f487cff9700 1 -- 172.21.15.78:0/2462037027 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f4878022070 con 0x7f488015c900 2023-10-11T09:51:23.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.396+0000 7f487effd700 1 --2- 172.21.15.78:0/2462037027 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4860065ed0 0x7f4860068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:23.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.400+0000 7f487effd700 1 --2- 172.21.15.78:0/2462037027 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4860065ed0 0x7f4860068380 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f4870009bf0 tx=0x7f4870009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:23.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.404+0000 7f487cff9700 1 -- 172.21.15.78:0/2462037027 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f4878052ac0 con 0x7f488015c900 2023-10-11T09:51:23.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.576+0000 7f488587b700 1 -- 172.21.15.78:0/2462037027 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f48800aefd0 con 0x7f488015c900 2023-10-11T09:51:23.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.576+0000 7f487cff9700 1 -- 172.21.15.78:0/2462037027 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v114) v1 ==== 92+0+10 (secure 0 0 0) 0x7f48780560e0 con 0x7f488015c900 2023-10-11T09:51:23.586 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:51:23.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.580+0000 7f488587b700 1 -- 172.21.15.78:0/2462037027 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4860065ed0 msgr2=0x7f4860068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:23.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.580+0000 7f488587b700 1 --2- 172.21.15.78:0/2462037027 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4860065ed0 0x7f4860068380 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f4870009bf0 tx=0x7f4870009240 comp rx=0 tx=0).stop 2023-10-11T09:51:23.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.580+0000 7f488587b700 1 -- 172.21.15.78:0/2462037027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f488015c900 msgr2=0x7f488006c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:23.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.580+0000 7f488587b700 1 --2- 172.21.15.78:0/2462037027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f488015c900 0x7f488006c130 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f4878014040 tx=0x7f4878004af0 comp rx=0 tx=0).stop 2023-10-11T09:51:23.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.580+0000 7f488587b700 1 -- 172.21.15.78:0/2462037027 shutdown_connections 2023-10-11T09:51:23.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.580+0000 7f488587b700 1 --2- 172.21.15.78:0/2462037027 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4860065ed0 0x7f4860068380 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:23.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.580+0000 7f488587b700 1 --2- 172.21.15.78:0/2462037027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f488015c900 0x7f488006c130 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:23.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.580+0000 7f488587b700 1 --2- 172.21.15.78:0/2462037027 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48800ce7f0 0x7f488015f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:23.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.580+0000 7f488587b700 1 --2- 172.21.15.78:0/2462037027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48800ad290 0x7f488015ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:23.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.580+0000 7f488587b700 1 -- 172.21.15.78:0/2462037027 >> 172.21.15.78:0/2462037027 conn(0x7f488006a060 msgr2=0x7f48800b3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:23.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.580+0000 7f488587b700 1 -- 172.21.15.78:0/2462037027 shutdown_connections 2023-10-11T09:51:23.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.580+0000 7f488587b700 1 -- 172.21.15.78:0/2462037027 wait complete. 2023-10-11T09:51:23.608 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T09:51:23.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.772+0000 7f520b71f700 1 Processor -- start 2023-10-11T09:51:23.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.776+0000 7f520b71f700 1 -- start start 2023-10-11T09:51:23.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.776+0000 7f520b71f700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f52040727d0 0x7f520406b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:23.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.776+0000 7f520b71f700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f520415ce80 0x7f520415d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:23.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.776+0000 7f520b71f700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52040cb320 con 0x7f520406c6a0 2023-10-11T09:51:23.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.776+0000 7f520b71f700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5204160500 con 0x7f52040727d0 2023-10-11T09:51:23.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.776+0000 7f520b71f700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52040cbea0 con 0x7f520415ce80 2023-10-11T09:51:23.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.776+0000 7f5208cba700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f52040727d0 0x7f520406b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:23.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.776+0000 7f5208cba700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f52040727d0 0x7f520406b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45604/0 (socket says 172.21.15.78:45604) 2023-10-11T09:51:23.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.776+0000 7f5208cba700 1 -- 172.21.15.78:0/742626014 learned_addr learned my addr 172.21.15.78:0/742626014 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:23.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.776+0000 7f5209cbc700 1 --2- 172.21.15.78:0/742626014 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f520415ce80 0x7f520415d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:23.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.776+0000 7f5208cba700 1 -- 172.21.15.78:0/742626014 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f520415ce80 msgr2=0x7f520415d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:23.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.776+0000 7f5208cba700 1 --2- 172.21.15.78:0/742626014 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f520415ce80 0x7f520415d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:23.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.776+0000 7f5208cba700 1 -- 172.21.15.78:0/742626014 >> v1:172.21.15.78:6789/0 conn(0x7f520406c6a0 legacy=0x7f520406ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:23.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.780+0000 7f5208cba700 1 -- 172.21.15.78:0/742626014 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52040cb600 con 0x7f52040727d0 2023-10-11T09:51:23.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.780+0000 7f5209cbc700 1 --2- 172.21.15.78:0/742626014 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f520415ce80 0x7f520415d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:51:23.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.780+0000 7f5208cba700 1 --2- 172.21.15.78:0/742626014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52040727d0 0x7f520406b540 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f51f4009910 tx=0x7f51f4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb83d8a4ac0079f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:23.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.780+0000 7f51fbfff700 1 -- 172.21.15.78:0/742626014 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f400ba60 con 0x7f52040727d0 2023-10-11T09:51:23.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.780+0000 7f51fbfff700 1 -- 172.21.15.78:0/742626014 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f400bbc0 con 0x7f52040727d0 2023-10-11T09:51:23.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.780+0000 7f51fbfff700 1 -- 172.21.15.78:0/742626014 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f4009e60 con 0x7f52040727d0 2023-10-11T09:51:23.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.780+0000 7f520b71f700 1 -- 172.21.15.78:0/742626014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52040727d0 msgr2=0x7f520406b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:23.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.780+0000 7f520b71f700 1 --2- 172.21.15.78:0/742626014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52040727d0 0x7f520406b540 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f51f4009910 tx=0x7f51f4009c20 comp rx=0 tx=0).stop 2023-10-11T09:51:23.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.780+0000 7f520b71f700 1 -- 172.21.15.78:0/742626014 shutdown_connections 2023-10-11T09:51:23.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.780+0000 7f520b71f700 1 --2- 172.21.15.78:0/742626014 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f520415ce80 0x7f520415d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:23.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.780+0000 7f520b71f700 1 --2- 172.21.15.78:0/742626014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52040727d0 0x7f520406b540 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:23.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.780+0000 7f520b71f700 1 -- 172.21.15.78:0/742626014 >> 172.21.15.78:0/742626014 conn(0x7f52040b1d60 msgr2=0x7f52040cf5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:23.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.780+0000 7f520b71f700 1 -- 172.21.15.78:0/742626014 shutdown_connections 2023-10-11T09:51:23.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.780+0000 7f520b71f700 1 -- 172.21.15.78:0/742626014 wait complete. 2023-10-11T09:51:23.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.784+0000 7f520b71f700 1 Processor -- start 2023-10-11T09:51:23.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.784+0000 7f520b71f700 1 -- start start 2023-10-11T09:51:23.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.784+0000 7f520b71f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f520406c6a0 0x7f5204158960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:23.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.784+0000 7f520b71f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52040727d0 0x7f52040c38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:23.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.784+0000 7f520b71f700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f520415ce80 0x7f520415ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:23.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.784+0000 7f520b71f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52040cca50 con 0x7f520406c6a0 2023-10-11T09:51:23.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.784+0000 7f520b71f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520415bd30 con 0x7f52040727d0 2023-10-11T09:51:23.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.784+0000 7f520b71f700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f520415c010 con 0x7f520415ce80 2023-10-11T09:51:23.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.784+0000 7f52094bb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f520406c6a0 0x7f5204158960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:23.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.784+0000 7f52094bb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f520406c6a0 0x7f5204158960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45606/0 (socket says 172.21.15.78:45606) 2023-10-11T09:51:23.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.784+0000 7f52094bb700 1 -- 172.21.15.78:0/3771011501 learned_addr learned my addr 172.21.15.78:0/3771011501 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:23.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.784+0000 7f5208cba700 1 --2- 172.21.15.78:0/3771011501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52040727d0 0x7f52040c38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:23.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.784+0000 7f5209cbc700 1 --2- 172.21.15.78:0/3771011501 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f520415ce80 0x7f520415ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:23.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.784+0000 7f52094bb700 1 -- 172.21.15.78:0/3771011501 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f520415ce80 msgr2=0x7f520415ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:23.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.784+0000 7f52094bb700 1 --2- 172.21.15.78:0/3771011501 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f520415ce80 0x7f520415ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:23.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.784+0000 7f52094bb700 1 -- 172.21.15.78:0/3771011501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52040727d0 msgr2=0x7f52040c38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:23.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.784+0000 7f52094bb700 1 --2- 172.21.15.78:0/3771011501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52040727d0 0x7f52040c38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:23.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.784+0000 7f52094bb700 1 -- 172.21.15.78:0/3771011501 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f4009570 con 0x7f520406c6a0 2023-10-11T09:51:23.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.784+0000 7f5209cbc700 1 --2- 172.21.15.78:0/3771011501 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f520415ce80 0x7f520415ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:51:23.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.784+0000 7f52094bb700 1 --2- 172.21.15.78:0/3771011501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f520406c6a0 0x7f5204158960 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f520000c500 tx=0x7f520000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:23.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.784+0000 7f51f9ffb700 1 -- 172.21.15.78:0/3771011501 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52000052d0 con 0x7f520406c6a0 2023-10-11T09:51:23.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.784+0000 7f51f9ffb700 1 -- 172.21.15.78:0/3771011501 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5200005430 con 0x7f520406c6a0 2023-10-11T09:51:23.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.784+0000 7f51f9ffb700 1 -- 172.21.15.78:0/3771011501 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52000055a0 con 0x7f520406c6a0 2023-10-11T09:51:23.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.784+0000 7f520b71f700 1 -- 172.21.15.78:0/3771011501 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f520415c2f0 con 0x7f520406c6a0 2023-10-11T09:51:23.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.788+0000 7f520b71f700 1 -- 172.21.15.78:0/3771011501 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52040c3de0 con 0x7f520406c6a0 2023-10-11T09:51:23.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.788+0000 7f520b71f700 1 -- 172.21.15.78:0/3771011501 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52041591a0 con 0x7f520406c6a0 2023-10-11T09:51:23.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.788+0000 7f51f9ffb700 1 -- 172.21.15.78:0/3771011501 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f52000038e0 con 0x7f520406c6a0 2023-10-11T09:51:23.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.792+0000 7f51f9ffb700 1 --2- 172.21.15.78:0/3771011501 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f51e8065ed0 0x7f51e8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:23.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.792+0000 7f51f9ffb700 1 -- 172.21.15.78:0/3771011501 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f5200082f80 con 0x7f520406c6a0 2023-10-11T09:51:23.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.792+0000 7f5208cba700 1 --2- 172.21.15.78:0/3771011501 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f51e8065ed0 0x7f51e8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:23.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.792+0000 7f5208cba700 1 --2- 172.21.15.78:0/3771011501 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f51e8065ed0 0x7f51e8068380 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f51f4000c00 tx=0x7f51f4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:23.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.796+0000 7f51f9ffb700 1 -- 172.21.15.78:0/3771011501 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f52000b3fa0 con 0x7f520406c6a0 2023-10-11T09:51:23.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.972+0000 7f520b71f700 1 -- 172.21.15.78:0/3771011501 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5204065db0 con 0x7f520406c6a0 2023-10-11T09:51:23.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.972+0000 7f51f9ffb700 1 -- 172.21.15.78:0/3771011501 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v114) v1 ==== 104+0+11 (secure 0 0 0) 0x7f52041591a0 con 0x7f520406c6a0 2023-10-11T09:51:23.984 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:51:23.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.976+0000 7f520b71f700 1 -- 172.21.15.78:0/3771011501 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f51e8065ed0 msgr2=0x7f51e8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:23.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.976+0000 7f520b71f700 1 --2- 172.21.15.78:0/3771011501 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f51e8065ed0 0x7f51e8068380 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f51f4000c00 tx=0x7f51f4008030 comp rx=0 tx=0).stop 2023-10-11T09:51:23.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.976+0000 7f520b71f700 1 -- 172.21.15.78:0/3771011501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f520406c6a0 msgr2=0x7f5204158960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:23.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.976+0000 7f520b71f700 1 --2- 172.21.15.78:0/3771011501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f520406c6a0 0x7f5204158960 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f520000c500 tx=0x7f520000c810 comp rx=0 tx=0).stop 2023-10-11T09:51:23.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.976+0000 7f520b71f700 1 -- 172.21.15.78:0/3771011501 shutdown_connections 2023-10-11T09:51:23.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.976+0000 7f520b71f700 1 --2- 172.21.15.78:0/3771011501 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f51e8065ed0 0x7f51e8068380 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:23.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.976+0000 7f520b71f700 1 --2- 172.21.15.78:0/3771011501 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f520415ce80 0x7f520415ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:23.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.976+0000 7f520b71f700 1 --2- 172.21.15.78:0/3771011501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52040727d0 0x7f52040c38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:23.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.976+0000 7f520b71f700 1 --2- 172.21.15.78:0/3771011501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f520406c6a0 0x7f5204158960 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:23.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.976+0000 7f520b71f700 1 -- 172.21.15.78:0/3771011501 >> 172.21.15.78:0/3771011501 conn(0x7f52040b1d60 msgr2=0x7f52041598b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:23.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.976+0000 7f520b71f700 1 -- 172.21.15.78:0/3771011501 shutdown_connections 2023-10-11T09:51:23.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:23.976+0000 7f520b71f700 1 -- 172.21.15.78:0/3771011501 wait complete. 2023-10-11T09:51:24.010 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T09:51:24.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.172+0000 7f140aa39700 1 Processor -- start 2023-10-11T09:51:24.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.172+0000 7f140aa39700 1 -- start start 2023-10-11T09:51:24.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.176+0000 7f140aa39700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f14040ad290 0x7f14040b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:24.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.176+0000 7f140aa39700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f14040ce7f0 0x7f14040cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:24.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.176+0000 7f140aa39700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f140406fde0 con 0x7f140415c900 2023-10-11T09:51:24.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.176+0000 7f140aa39700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14040705f0 con 0x7f14040ad290 2023-10-11T09:51:24.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.176+0000 7f140aa39700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14040cf1b0 con 0x7f14040ce7f0 2023-10-11T09:51:24.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.176+0000 7f1409236700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f14040ad290 0x7f14040b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:24.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.176+0000 7f1409236700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f14040ad290 0x7f14040b1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45614/0 (socket says 172.21.15.78:45614) 2023-10-11T09:51:24.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.176+0000 7f1409236700 1 -- 172.21.15.78:0/999372229 learned_addr learned my addr 172.21.15.78:0/999372229 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:24.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.176+0000 7f1409236700 1 -- 172.21.15.78:0/999372229 >> v2:172.21.15.133:3300/0 conn(0x7f14040ce7f0 msgr2=0x7f14040cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:24.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.176+0000 7f140a238700 1 --2- 172.21.15.78:0/999372229 >> v2:172.21.15.133:3300/0 conn(0x7f14040ce7f0 0x7f14040cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:24.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.176+0000 7f1409236700 1 --2- 172.21.15.78:0/999372229 >> v2:172.21.15.133:3300/0 conn(0x7f14040ce7f0 0x7f14040cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:24.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.176+0000 7f1409236700 1 -- 172.21.15.78:0/999372229 >> v1:172.21.15.133:6789/0 conn(0x7f140415c900 legacy=0x7f1404071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:24.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.176+0000 7f1409236700 1 -- 172.21.15.78:0/999372229 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14040ae000 con 0x7f14040ad290 2023-10-11T09:51:24.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.176+0000 7f140a238700 1 --2- 172.21.15.78:0/999372229 >> v2:172.21.15.133:3300/0 conn(0x7f14040ce7f0 0x7f14040cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:51:24.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.180+0000 7f1409236700 1 --2- 172.21.15.78:0/999372229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14040ad290 0x7f14040b1a10 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f14040721c0 tx=0x7f13f4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9165adcf0f578b85 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:24.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.180+0000 7f1408a35700 1 -- 172.21.15.78:0/999372229 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f400b8a0 con 0x7f14040ad290 2023-10-11T09:51:24.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.180+0000 7f1408a35700 1 -- 172.21.15.78:0/999372229 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f400ba00 con 0x7f14040ad290 2023-10-11T09:51:24.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.180+0000 7f1408a35700 1 -- 172.21.15.78:0/999372229 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f400bd90 con 0x7f14040ad290 2023-10-11T09:51:24.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.180+0000 7f140aa39700 1 -- 172.21.15.78:0/999372229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14040ad290 msgr2=0x7f14040b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:24.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.180+0000 7f140aa39700 1 --2- 172.21.15.78:0/999372229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14040ad290 0x7f14040b1a10 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f14040721c0 tx=0x7f13f4009bf0 comp rx=0 tx=0).stop 2023-10-11T09:51:24.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.180+0000 7f140aa39700 1 -- 172.21.15.78:0/999372229 shutdown_connections 2023-10-11T09:51:24.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.180+0000 7f140aa39700 1 --2- 172.21.15.78:0/999372229 >> v2:172.21.15.133:3300/0 conn(0x7f14040ce7f0 0x7f14040cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:24.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.180+0000 7f140aa39700 1 --2- 172.21.15.78:0/999372229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14040ad290 0x7f14040b1a10 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:24.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.180+0000 7f140aa39700 1 -- 172.21.15.78:0/999372229 >> 172.21.15.78:0/999372229 conn(0x7f140406a060 msgr2=0x7f140406a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:24.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.180+0000 7f140aa39700 1 -- 172.21.15.78:0/999372229 shutdown_connections 2023-10-11T09:51:24.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.180+0000 7f140aa39700 1 -- 172.21.15.78:0/999372229 wait complete. 2023-10-11T09:51:24.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.184+0000 7f140aa39700 1 Processor -- start 2023-10-11T09:51:24.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.184+0000 7f140aa39700 1 -- start start 2023-10-11T09:51:24.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.184+0000 7f140aa39700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14040ad290 0x7f14041f16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:24.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.184+0000 7f140aa39700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14040ce7f0 0x7f140415f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:24.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.184+0000 7f140aa39700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f140415c900 0x7f140415f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:24.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.184+0000 7f140aa39700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14041f22b0 con 0x7f14040ce7f0 2023-10-11T09:51:24.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.184+0000 7f140aa39700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14041f3030 con 0x7f14040ad290 2023-10-11T09:51:24.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.184+0000 7f140aa39700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f140415ccd0 con 0x7f140415c900 2023-10-11T09:51:24.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.184+0000 7f1409236700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14040ce7f0 0x7f140415f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:24.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.188+0000 7f1409236700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14040ce7f0 0x7f140415f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45628/0 (socket says 172.21.15.78:45628) 2023-10-11T09:51:24.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.188+0000 7f1409236700 1 -- 172.21.15.78:0/3859191912 learned_addr learned my addr 172.21.15.78:0/3859191912 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:24.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.188+0000 7f1409a37700 1 --2- 172.21.15.78:0/3859191912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14040ad290 0x7f14041f16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:24.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.188+0000 7f140a238700 1 --2- 172.21.15.78:0/3859191912 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f140415c900 0x7f140415f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:24.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.188+0000 7f1409236700 1 -- 172.21.15.78:0/3859191912 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f140415c900 msgr2=0x7f140415f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:24.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.188+0000 7f1409236700 1 --2- 172.21.15.78:0/3859191912 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f140415c900 0x7f140415f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:24.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.188+0000 7f1409236700 1 -- 172.21.15.78:0/3859191912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14040ad290 msgr2=0x7f14041f16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:24.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.188+0000 7f1409236700 1 --2- 172.21.15.78:0/3859191912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14040ad290 0x7f14041f16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:24.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.188+0000 7f1409236700 1 -- 172.21.15.78:0/3859191912 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f4009570 con 0x7f14040ce7f0 2023-10-11T09:51:24.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.188+0000 7f1409a37700 1 --2- 172.21.15.78:0/3859191912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14040ad290 0x7f14041f16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:51:24.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.188+0000 7f1409236700 1 --2- 172.21.15.78:0/3859191912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14040ce7f0 0x7f140415f0f0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f13f4000c00 tx=0x7f13f4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:24.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.188+0000 7f13fa7fc700 1 -- 172.21.15.78:0/3859191912 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f40038c0 con 0x7f14040ce7f0 2023-10-11T09:51:24.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.188+0000 7f13fa7fc700 1 -- 172.21.15.78:0/3859191912 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f4003a40 con 0x7f14040ce7f0 2023-10-11T09:51:24.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.188+0000 7f13fa7fc700 1 -- 172.21.15.78:0/3859191912 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f4003df0 con 0x7f14040ce7f0 2023-10-11T09:51:24.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.188+0000 7f140aa39700 1 -- 172.21.15.78:0/3859191912 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14040cd600 con 0x7f14040ce7f0 2023-10-11T09:51:24.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.188+0000 7f140aa39700 1 -- 172.21.15.78:0/3859191912 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f140415cf20 con 0x7f14040ce7f0 2023-10-11T09:51:24.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.188+0000 7f140aa39700 1 -- 172.21.15.78:0/3859191912 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1404065d40 con 0x7f14040ce7f0 2023-10-11T09:51:24.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.192+0000 7f13fa7fc700 1 -- 172.21.15.78:0/3859191912 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f13f4011420 con 0x7f14040ce7f0 2023-10-11T09:51:24.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.192+0000 7f13fa7fc700 1 --2- 172.21.15.78:0/3859191912 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f13e8065ed0 0x7f13e8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:24.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.196+0000 7f13fa7fc700 1 -- 172.21.15.78:0/3859191912 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f13f4020030 con 0x7f14040ce7f0 2023-10-11T09:51:24.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.196+0000 7f1409a37700 1 --2- 172.21.15.78:0/3859191912 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f13e8065ed0 0x7f13e8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:24.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.196+0000 7f1409a37700 1 --2- 172.21.15.78:0/3859191912 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f13e8065ed0 0x7f13e8068380 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f14000012e0 tx=0x7f1400009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:24.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.200+0000 7f13fa7fc700 1 -- 172.21.15.78:0/3859191912 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f13f40524c0 con 0x7f14040ce7f0 2023-10-11T09:51:24.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.376+0000 7f140aa39700 1 -- 172.21.15.78:0/3859191912 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f14040cf5b0 con 0x7f14040ce7f0 2023-10-11T09:51:24.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.380+0000 7f13fa7fc700 1 -- 172.21.15.78:0/3859191912 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v114) v1 ==== 100+0+11 (secure 0 0 0) 0x7f13f4055ae0 con 0x7f14040ce7f0 2023-10-11T09:51:24.388 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2023-10-11T09:51:24.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.384+0000 7f140aa39700 1 -- 172.21.15.78:0/3859191912 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f13e8065ed0 msgr2=0x7f13e8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:24.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.384+0000 7f140aa39700 1 --2- 172.21.15.78:0/3859191912 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f13e8065ed0 0x7f13e8068380 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f14000012e0 tx=0x7f1400009040 comp rx=0 tx=0).stop 2023-10-11T09:51:24.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.384+0000 7f140aa39700 1 -- 172.21.15.78:0/3859191912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14040ce7f0 msgr2=0x7f140415f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:24.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.384+0000 7f140aa39700 1 --2- 172.21.15.78:0/3859191912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14040ce7f0 0x7f140415f0f0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f13f4000c00 tx=0x7f13f4004770 comp rx=0 tx=0).stop 2023-10-11T09:51:24.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.384+0000 7f140aa39700 1 -- 172.21.15.78:0/3859191912 shutdown_connections 2023-10-11T09:51:24.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.384+0000 7f140aa39700 1 --2- 172.21.15.78:0/3859191912 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f13e8065ed0 0x7f13e8068380 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:24.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.384+0000 7f140aa39700 1 --2- 172.21.15.78:0/3859191912 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f140415c900 0x7f140415f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:24.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.384+0000 7f140aa39700 1 --2- 172.21.15.78:0/3859191912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14040ce7f0 0x7f140415f0f0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:24.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.384+0000 7f140aa39700 1 --2- 172.21.15.78:0/3859191912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14040ad290 0x7f14041f16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:24.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.384+0000 7f140aa39700 1 -- 172.21.15.78:0/3859191912 >> 172.21.15.78:0/3859191912 conn(0x7f140406a060 msgr2=0x7f14040b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:24.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.384+0000 7f140aa39700 1 -- 172.21.15.78:0/3859191912 shutdown_connections 2023-10-11T09:51:24.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.384+0000 7f140aa39700 1 -- 172.21.15.78:0/3859191912 wait complete. 2023-10-11T09:51:24.418 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-10-11T09:51:24.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.580+0000 7f0a70d29700 1 Processor -- start 2023-10-11T09:51:24.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.580+0000 7f0a70d29700 1 -- start start 2023-10-11T09:51:24.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.584+0000 7f0a70d29700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0a6c0b3a60 0x7f0a6c06d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:24.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.584+0000 7f0a70d29700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a6c06d560 0x7f0a6c06bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:24.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.584+0000 7f0a70d29700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a6c0b2d30 con 0x7f0a6c15a6b0 2023-10-11T09:51:24.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.584+0000 7f0a70d29700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a6c06d960 con 0x7f0a6c0b3a60 2023-10-11T09:51:24.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.584+0000 7f0a70d29700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c0cf360 con 0x7f0a6c06d560 2023-10-11T09:51:24.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.584+0000 7f0a6affd700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0a6c0b3a60 0x7f0a6c06d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:24.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.584+0000 7f0a6affd700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0a6c0b3a60 0x7f0a6c06d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34450/0 (socket says 172.21.15.78:34450) 2023-10-11T09:51:24.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.584+0000 7f0a6affd700 1 -- 172.21.15.78:0/1139926740 learned_addr learned my addr 172.21.15.78:0/1139926740 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:24.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.584+0000 7f0a6bfff700 1 --2- 172.21.15.78:0/1139926740 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a6c06d560 0x7f0a6c06bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:24.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.584+0000 7f0a6affd700 1 -- 172.21.15.78:0/1139926740 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a6c06d560 msgr2=0x7f0a6c06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:24.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.584+0000 7f0a6affd700 1 --2- 172.21.15.78:0/1139926740 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a6c06d560 0x7f0a6c06bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:24.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.584+0000 7f0a6affd700 1 -- 172.21.15.78:0/1139926740 >> v1:172.21.15.133:6789/0 conn(0x7f0a6c15a6b0 legacy=0x7f0a6c0ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:24.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.584+0000 7f0a6affd700 1 -- 172.21.15.78:0/1139926740 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a6c0b2f80 con 0x7f0a6c0b3a60 2023-10-11T09:51:24.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.584+0000 7f0a6a7fc700 1 -- 172.21.15.78:0/1139926740 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3524779907 0 0) 0x7f0a6c0b2d30 con 0x7f0a6c15a6b0 2023-10-11T09:51:24.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.584+0000 7f0a6affd700 1 --2- 172.21.15.78:0/1139926740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a6c0b3a60 0x7f0a6c06d020 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f0a60009910 tx=0x7f0a60009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743c362fe923f959 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:24.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.584+0000 7f0a6a7fc700 1 -- 172.21.15.78:0/1139926740 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6000ba60 con 0x7f0a6c0b3a60 2023-10-11T09:51:24.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.584+0000 7f0a6a7fc700 1 -- 172.21.15.78:0/1139926740 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a6000bbc0 con 0x7f0a6c0b3a60 2023-10-11T09:51:24.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.584+0000 7f0a6a7fc700 1 -- 172.21.15.78:0/1139926740 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a60009e60 con 0x7f0a6c0b3a60 2023-10-11T09:51:24.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.588+0000 7f0a70d29700 1 -- 172.21.15.78:0/1139926740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a6c0b3a60 msgr2=0x7f0a6c06d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:24.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.588+0000 7f0a70d29700 1 --2- 172.21.15.78:0/1139926740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a6c0b3a60 0x7f0a6c06d020 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f0a60009910 tx=0x7f0a60009c20 comp rx=0 tx=0).stop 2023-10-11T09:51:24.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.588+0000 7f0a70d29700 1 -- 172.21.15.78:0/1139926740 shutdown_connections 2023-10-11T09:51:24.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.588+0000 7f0a70d29700 1 --2- 172.21.15.78:0/1139926740 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a6c06d560 0x7f0a6c06bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:24.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.588+0000 7f0a70d29700 1 --2- 172.21.15.78:0/1139926740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a6c0b3a60 0x7f0a6c06d020 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:24.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.588+0000 7f0a70d29700 1 -- 172.21.15.78:0/1139926740 >> 172.21.15.78:0/1139926740 conn(0x7f0a6c06a060 msgr2=0x7f0a6c06a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:24.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.588+0000 7f0a70d29700 1 -- 172.21.15.78:0/1139926740 shutdown_connections 2023-10-11T09:51:24.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.588+0000 7f0a70d29700 1 -- 172.21.15.78:0/1139926740 wait complete. 2023-10-11T09:51:24.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.592+0000 7f0a70d29700 1 Processor -- start 2023-10-11T09:51:24.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.592+0000 7f0a70d29700 1 -- start start 2023-10-11T09:51:24.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.592+0000 7f0a70d29700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a6c06d560 0x7f0a6c1ef700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:24.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.592+0000 7f0a70d29700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a6c0b3a60 0x7f0a6c1588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:24.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.592+0000 7f0a70d29700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a6c15a6b0 0x7f0a6c158e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:24.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.592+0000 7f0a70d29700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c1f0cc0 con 0x7f0a6c0b3a60 2023-10-11T09:51:24.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.592+0000 7f0a70d29700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c1f4270 con 0x7f0a6c06d560 2023-10-11T09:51:24.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.592+0000 7f0a70d29700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c1efc40 con 0x7f0a6c15a6b0 2023-10-11T09:51:24.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.596+0000 7f0a6affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a6c0b3a60 0x7f0a6c1588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:24.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.596+0000 7f0a6affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a6c0b3a60 0x7f0a6c1588e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34460/0 (socket says 172.21.15.78:34460) 2023-10-11T09:51:24.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.596+0000 7f0a6affd700 1 -- 172.21.15.78:0/3937550174 learned_addr learned my addr 172.21.15.78:0/3937550174 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:24.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.596+0000 7f0a6b7fe700 1 --2- 172.21.15.78:0/3937550174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a6c06d560 0x7f0a6c1ef700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:24.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.596+0000 7f0a6bfff700 1 --2- 172.21.15.78:0/3937550174 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a6c15a6b0 0x7f0a6c158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:24.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.596+0000 7f0a6affd700 1 -- 172.21.15.78:0/3937550174 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a6c15a6b0 msgr2=0x7f0a6c158e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:24.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.596+0000 7f0a6affd700 1 --2- 172.21.15.78:0/3937550174 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a6c15a6b0 0x7f0a6c158e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:24.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.596+0000 7f0a6affd700 1 -- 172.21.15.78:0/3937550174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a6c06d560 msgr2=0x7f0a6c1ef700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:24.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.596+0000 7f0a6affd700 1 --2- 172.21.15.78:0/3937550174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a6c06d560 0x7f0a6c1ef700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:24.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.596+0000 7f0a6affd700 1 -- 172.21.15.78:0/3937550174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a60009570 con 0x7f0a6c0b3a60 2023-10-11T09:51:24.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.596+0000 7f0a6affd700 1 --2- 172.21.15.78:0/3937550174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a6c0b3a60 0x7f0a6c1588e0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f0a60000c00 tx=0x7f0a60004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:24.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.596+0000 7f0a53fff700 1 -- 172.21.15.78:0/3937550174 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a60004610 con 0x7f0a6c0b3a60 2023-10-11T09:51:24.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.596+0000 7f0a53fff700 1 -- 172.21.15.78:0/3937550174 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a6001e070 con 0x7f0a6c0b3a60 2023-10-11T09:51:24.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.596+0000 7f0a70d29700 1 -- 172.21.15.78:0/3937550174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a6c1f3de0 con 0x7f0a6c0b3a60 2023-10-11T09:51:24.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.596+0000 7f0a53fff700 1 -- 172.21.15.78:0/3937550174 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a600038e0 con 0x7f0a6c0b3a60 2023-10-11T09:51:24.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.596+0000 7f0a70d29700 1 -- 172.21.15.78:0/3937550174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a6c1f3790 con 0x7f0a6c0b3a60 2023-10-11T09:51:24.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.596+0000 7f0a70d29700 1 -- 172.21.15.78:0/3937550174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a6c06c820 con 0x7f0a6c0b3a60 2023-10-11T09:51:24.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.600+0000 7f0a53fff700 1 -- 172.21.15.78:0/3937550174 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f0a60003a40 con 0x7f0a6c0b3a60 2023-10-11T09:51:24.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.600+0000 7f0a53fff700 1 --2- 172.21.15.78:0/3937550174 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0a58065f00 0x7f0a580683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:24.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.600+0000 7f0a53fff700 1 -- 172.21.15.78:0/3937550174 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f0a60020070 con 0x7f0a6c0b3a60 2023-10-11T09:51:24.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.600+0000 7f0a6b7fe700 1 --2- 172.21.15.78:0/3937550174 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0a58065f00 0x7f0a580683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:24.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.600+0000 7f0a6b7fe700 1 --2- 172.21.15.78:0/3937550174 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0a58065f00 0x7f0a580683b0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c000f60 tx=0x7f0a5c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:24.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.608+0000 7f0a53fff700 1 -- 172.21.15.78:0/3937550174 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f0a600522e0 con 0x7f0a6c0b3a60 2023-10-11T09:51:24.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.780+0000 7f0a70d29700 1 -- 172.21.15.78:0/3937550174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7f0a6c15eec0 con 0x7f0a6c0b3a60 2023-10-11T09:51:24.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.780+0000 7f0a53fff700 1 -- 172.21.15.78:0/3937550174 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v114) v1 ==== 106+0+11 (secure 0 0 0) 0x7f0a600150b0 con 0x7f0a6c0b3a60 2023-10-11T09:51:24.790 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:51:24.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.784+0000 7f0a70d29700 1 -- 172.21.15.78:0/3937550174 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0a58065f00 msgr2=0x7f0a580683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:24.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.784+0000 7f0a70d29700 1 --2- 172.21.15.78:0/3937550174 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0a58065f00 0x7f0a580683b0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c000f60 tx=0x7f0a5c009040 comp rx=0 tx=0).stop 2023-10-11T09:51:24.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.784+0000 7f0a70d29700 1 -- 172.21.15.78:0/3937550174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a6c0b3a60 msgr2=0x7f0a6c1588e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:24.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.784+0000 7f0a70d29700 1 --2- 172.21.15.78:0/3937550174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a6c0b3a60 0x7f0a6c1588e0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f0a60000c00 tx=0x7f0a60004770 comp rx=0 tx=0).stop 2023-10-11T09:51:24.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.784+0000 7f0a70d29700 1 -- 172.21.15.78:0/3937550174 shutdown_connections 2023-10-11T09:51:24.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.784+0000 7f0a70d29700 1 --2- 172.21.15.78:0/3937550174 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0a58065f00 0x7f0a580683b0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:24.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.784+0000 7f0a70d29700 1 --2- 172.21.15.78:0/3937550174 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a6c15a6b0 0x7f0a6c158e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:24.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.784+0000 7f0a70d29700 1 --2- 172.21.15.78:0/3937550174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a6c0b3a60 0x7f0a6c1588e0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:24.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.784+0000 7f0a70d29700 1 --2- 172.21.15.78:0/3937550174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a6c06d560 0x7f0a6c1ef700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:24.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.784+0000 7f0a70d29700 1 -- 172.21.15.78:0/3937550174 >> 172.21.15.78:0/3937550174 conn(0x7f0a6c06a060 msgr2=0x7f0a6c06bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:24.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.784+0000 7f0a70d29700 1 -- 172.21.15.78:0/3937550174 shutdown_connections 2023-10-11T09:51:24.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.784+0000 7f0a70d29700 1 -- 172.21.15.78:0/3937550174 wait complete. 2023-10-11T09:51:24.819 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:51:24.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.976+0000 7f1401891700 1 Processor -- start 2023-10-11T09:51:24.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.980+0000 7f1401891700 1 -- start start 2023-10-11T09:51:24.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.980+0000 7f1401891700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f13fc15c730 0x7f13fc0c6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:24.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.980+0000 7f1401891700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13fc160300 con 0x7f13fc0cccf0 2023-10-11T09:51:24.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.980+0000 7f1401891700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13fc160e80 con 0x7f13fc0cd140 2023-10-11T09:51:24.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.980+0000 7f1401891700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13fc0b2e70 con 0x7f13fc15c730 2023-10-11T09:51:24.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.980+0000 7f13faffd700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f13fc15c730 0x7f13fc0c6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:24.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.980+0000 7f13fa7fc700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f13fc0cccf0 0x7f13fc0b2050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33778/0 (socket says 172.21.15.78:33778) 2023-10-11T09:51:24.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.980+0000 7f13faffd700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f13fc15c730 0x7f13fc0c6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34464/0 (socket says 172.21.15.78:34464) 2023-10-11T09:51:24.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.980+0000 7f13fa7fc700 1 -- 172.21.15.78:0/3361794720 learned_addr learned my addr 172.21.15.78:0/3361794720 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:51:24.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.984+0000 7f13faffd700 1 -- 172.21.15.78:0/3361794720 >> v1:172.21.15.133:6789/0 conn(0x7f13fc0cd140 legacy=0x7f13fc0b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:24.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.984+0000 7f13faffd700 1 -- 172.21.15.78:0/3361794720 >> v1:172.21.15.78:6789/0 conn(0x7f13fc0cccf0 legacy=0x7f13fc0b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:24.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.984+0000 7f13faffd700 1 -- 172.21.15.78:0/3361794720 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13fc160550 con 0x7f13fc15c730 2023-10-11T09:51:24.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.984+0000 7f13f9ffb700 1 -- 172.21.15.78:0/3361794720 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 124029179 0 0) 0x7f13fc160300 con 0x7f13fc0cccf0 2023-10-11T09:51:24.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.984+0000 7f13faffd700 1 --2- 172.21.15.78:0/3361794720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13fc15c730 0x7f13fc0c6a30 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f13ec009850 tx=0x7f13ec009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f272c7e0d2f0e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:24.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.984+0000 7f13f9ffb700 1 -- 172.21.15.78:0/3361794720 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ec011830 con 0x7f13fc15c730 2023-10-11T09:51:24.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.984+0000 7f13f9ffb700 1 -- 172.21.15.78:0/3361794720 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13ec011990 con 0x7f13fc15c730 2023-10-11T09:51:24.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.984+0000 7f13f9ffb700 1 -- 172.21.15.78:0/3361794720 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ec011d20 con 0x7f13fc15c730 2023-10-11T09:51:24.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.984+0000 7f1401891700 1 -- 172.21.15.78:0/3361794720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13fc15c730 msgr2=0x7f13fc0c6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:24.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.984+0000 7f1401891700 1 --2- 172.21.15.78:0/3361794720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13fc15c730 0x7f13fc0c6a30 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f13ec009850 tx=0x7f13ec009b60 comp rx=0 tx=0).stop 2023-10-11T09:51:24.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.984+0000 7f1401891700 1 -- 172.21.15.78:0/3361794720 shutdown_connections 2023-10-11T09:51:24.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.984+0000 7f1401891700 1 --2- 172.21.15.78:0/3361794720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13fc15c730 0x7f13fc0c6a30 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:24.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.984+0000 7f1401891700 1 -- 172.21.15.78:0/3361794720 >> 172.21.15.78:0/3361794720 conn(0x7f13fc06dd80 msgr2=0x7f13fc06e150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:24.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.984+0000 7f1401891700 1 -- 172.21.15.78:0/3361794720 shutdown_connections 2023-10-11T09:51:24.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.984+0000 7f1401891700 1 -- 172.21.15.78:0/3361794720 wait complete. 2023-10-11T09:51:24.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.988+0000 7f1401891700 1 Processor -- start 2023-10-11T09:51:24.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.988+0000 7f1401891700 1 -- start start 2023-10-11T09:51:24.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.988+0000 7f1401891700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13fc0cccf0 0x7f13fc0c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:24.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.988+0000 7f1401891700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f13fc0cd140 0x7f13fc0b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:24.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.988+0000 7f1401891700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13fc15c730 0x7f13fc0b5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:24.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.988+0000 7f1401891700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13fc0b3ee0 con 0x7f13fc0cccf0 2023-10-11T09:51:25.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.988+0000 7f1401891700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13fc0b41c0 con 0x7f13fc15c730 2023-10-11T09:51:25.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.988+0000 7f1401891700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13fc0b44a0 con 0x7f13fc0cd140 2023-10-11T09:51:25.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.988+0000 7f13faffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13fc0cccf0 0x7f13fc0c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:25.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.988+0000 7f13faffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13fc0cccf0 0x7f13fc0c9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34478/0 (socket says 172.21.15.78:34478) 2023-10-11T09:51:25.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.988+0000 7f13faffd700 1 -- 172.21.15.78:0/2727141583 learned_addr learned my addr 172.21.15.78:0/2727141583 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:25.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.988+0000 7f13fa7fc700 1 --2- 172.21.15.78:0/2727141583 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f13fc0cd140 0x7f13fc0b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:25.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.988+0000 7f13faffd700 1 -- 172.21.15.78:0/2727141583 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f13fc0cd140 msgr2=0x7f13fc0b53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:25.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.988+0000 7f13fbfff700 1 --2- 172.21.15.78:0/2727141583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13fc15c730 0x7f13fc0b5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:25.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.988+0000 7f13faffd700 1 --2- 172.21.15.78:0/2727141583 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f13fc0cd140 0x7f13fc0b53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:25.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.988+0000 7f13faffd700 1 -- 172.21.15.78:0/2727141583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13fc15c730 msgr2=0x7f13fc0b5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:25.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.988+0000 7f13faffd700 1 --2- 172.21.15.78:0/2727141583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13fc15c730 0x7f13fc0b5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:25.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.988+0000 7f13faffd700 1 -- 172.21.15.78:0/2727141583 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ec0094b0 con 0x7f13fc0cccf0 2023-10-11T09:51:25.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.988+0000 7f13fbfff700 1 --2- 172.21.15.78:0/2727141583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13fc15c730 0x7f13fc0b5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:51:25.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.988+0000 7f13fa7fc700 1 --2- 172.21.15.78:0/2727141583 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f13fc0cd140 0x7f13fc0b53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:51:25.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.992+0000 7f13faffd700 1 --2- 172.21.15.78:0/2727141583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13fc0cccf0 0x7f13fc0c9ba0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f13ec014040 tx=0x7f13ec004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:25.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.992+0000 7f13e3fff700 1 -- 172.21.15.78:0/2727141583 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ec00bc10 con 0x7f13fc0cccf0 2023-10-11T09:51:25.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.992+0000 7f13e3fff700 1 -- 172.21.15.78:0/2727141583 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13ec01e070 con 0x7f13fc0cccf0 2023-10-11T09:51:25.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.992+0000 7f1401891700 1 -- 172.21.15.78:0/2727141583 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13fc0b4780 con 0x7f13fc0cccf0 2023-10-11T09:51:25.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.992+0000 7f13e3fff700 1 -- 172.21.15.78:0/2727141583 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ec01a420 con 0x7f13fc0cccf0 2023-10-11T09:51:25.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.992+0000 7f1401891700 1 -- 172.21.15.78:0/2727141583 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13fc0b4e80 con 0x7f13fc0cccf0 2023-10-11T09:51:25.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.992+0000 7f1401891700 1 -- 172.21.15.78:0/2727141583 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13fc0b5010 con 0x7f13fc0cccf0 2023-10-11T09:51:25.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.996+0000 7f13e3fff700 1 -- 172.21.15.78:0/2727141583 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f13ec01a580 con 0x7f13fc0cccf0 2023-10-11T09:51:25.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.996+0000 7f13e3fff700 1 --2- 172.21.15.78:0/2727141583 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f13e8065f20 0x7f13e80683d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:25.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.996+0000 7f13e3fff700 1 -- 172.21.15.78:0/2727141583 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f13ec084e50 con 0x7f13fc0cccf0 2023-10-11T09:51:25.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.996+0000 7f13fa7fc700 1 --2- 172.21.15.78:0/2727141583 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f13e8065f20 0x7f13e80683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:25.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:24.996+0000 7f13fa7fc700 1 --2- 172.21.15.78:0/2727141583 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f13e8065f20 0x7f13e80683d0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f13fc0b1f40 tx=0x7f13f0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:25.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.000+0000 7f13e3fff700 1 -- 172.21.15.78:0/2727141583 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f13fc0b5010 con 0x7f13fc0cccf0 2023-10-11T09:51:25.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.208+0000 7f1401891700 1 -- 172.21.15.78:0/2727141583 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f13fc0c76f0 con 0x7f13fc0cccf0 2023-10-11T09:51:25.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.208+0000 7f13e3fff700 1 -- 172.21.15.78:0/2727141583 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4655 (secure 0 0 0) 0x7f13fc0c76f0 con 0x7f13fc0cccf0 2023-10-11T09:51:25.220 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:25.221 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":91,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7148,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/331520590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":331520590},{"type":"v1","addr":"172.21.15.78:6835","nonce":331520590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":7151,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/934539531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":934539531},{"type":"v1","addr":"172.21.15.133:6834","nonce":934539531}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":7154,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3894002128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":3894002128},{"type":"v1","addr":"172.21.15.133:6835","nonce":3894002128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:50:28.502984+0000","modified":"2023-10-11T09:50:37.659950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8550":{"gid":8550,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":4,"addr":"172.21.15.78:6837/2869034036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2869034036},{"type":"v1","addr":"172.21.15.78:6837","nonce":2869034036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15,16],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-11T09:51:25.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.212+0000 7f1401891700 1 -- 172.21.15.78:0/2727141583 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f13e8065f20 msgr2=0x7f13e80683d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:25.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.212+0000 7f1401891700 1 --2- 172.21.15.78:0/2727141583 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f13e8065f20 0x7f13e80683d0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f13fc0b1f40 tx=0x7f13f0009040 comp rx=0 tx=0).stop 2023-10-11T09:51:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.212+0000 7f1401891700 1 -- 172.21.15.78:0/2727141583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13fc0cccf0 msgr2=0x7f13fc0c9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.212+0000 7f1401891700 1 --2- 172.21.15.78:0/2727141583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13fc0cccf0 0x7f13fc0c9ba0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f13ec014040 tx=0x7f13ec004770 comp rx=0 tx=0).stop 2023-10-11T09:51:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.212+0000 7f1401891700 1 -- 172.21.15.78:0/2727141583 shutdown_connections 2023-10-11T09:51:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.212+0000 7f1401891700 1 --2- 172.21.15.78:0/2727141583 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f13e8065f20 0x7f13e80683d0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:25.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.212+0000 7f1401891700 1 --2- 172.21.15.78:0/2727141583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13fc15c730 0x7f13fc0b5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:25.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.212+0000 7f1401891700 1 --2- 172.21.15.78:0/2727141583 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f13fc0cd140 0x7f13fc0b53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:25.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.212+0000 7f1401891700 1 --2- 172.21.15.78:0/2727141583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13fc0cccf0 0x7f13fc0c9ba0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:25.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.212+0000 7f1401891700 1 -- 172.21.15.78:0/2727141583 >> 172.21.15.78:0/2727141583 conn(0x7f13fc06dd80 msgr2=0x7f13fc06aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:25.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.212+0000 7f1401891700 1 -- 172.21.15.78:0/2727141583 shutdown_connections 2023-10-11T09:51:25.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.212+0000 7f1401891700 1 -- 172.21.15.78:0/2727141583 wait complete. 2023-10-11T09:51:25.227 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 91 2023-10-11T09:51:25.244 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:51:25.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.408+0000 7f422f42f700 1 Processor -- start 2023-10-11T09:51:25.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.408+0000 7f422f42f700 1 -- start start 2023-10-11T09:51:25.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.412+0000 7f422f42f700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42280d1660 0x7f42280d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:25.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.412+0000 7f422f42f700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f42280d28e0 0x7f42280d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:25.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.412+0000 7f422f42f700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f422815dc10 con 0x7f4228156ce0 2023-10-11T09:51:25.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.412+0000 7f422f42f700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4228160380 con 0x7f42280d28e0 2023-10-11T09:51:25.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.412+0000 7f422f42f700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f422815d930 con 0x7f42280d1660 2023-10-11T09:51:25.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.412+0000 7f422e42d700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f4228156ce0 0x7f4228160b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33780/0 (socket says 172.21.15.78:33780) 2023-10-11T09:51:25.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.412+0000 7f422e42d700 1 -- 172.21.15.78:0/3008259013 learned_addr learned my addr 172.21.15.78:0/3008259013 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:51:25.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.412+0000 7f422dc2c700 1 --2- 172.21.15.78:0/3008259013 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42280d1660 0x7f42280d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:25.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.412+0000 7f422ec2e700 1 --2- 172.21.15.78:0/3008259013 >> v2:172.21.15.133:3300/0 conn(0x7f42280d28e0 0x7f42280d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:25.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.412+0000 7f422d42b700 1 -- 172.21.15.78:0/3008259013 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1804239025 0 0) 0x7f422815dc10 con 0x7f4228156ce0 2023-10-11T09:51:25.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.412+0000 7f422d42b700 1 -- 172.21.15.78:0/3008259013 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42140035d0 con 0x7f4228156ce0 2023-10-11T09:51:25.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.412+0000 7f422dc2c700 1 -- 172.21.15.78:0/3008259013 >> v2:172.21.15.133:3300/0 conn(0x7f42280d28e0 msgr2=0x7f42280d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:25.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.412+0000 7f422dc2c700 1 --2- 172.21.15.78:0/3008259013 >> v2:172.21.15.133:3300/0 conn(0x7f42280d28e0 0x7f42280d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:25.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.412+0000 7f422dc2c700 1 -- 172.21.15.78:0/3008259013 >> v1:172.21.15.78:6789/0 conn(0x7f4228156ce0 legacy=0x7f4228160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:25.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.412+0000 7f422dc2c700 1 -- 172.21.15.78:0/3008259013 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4228160620 con 0x7f42280d1660 2023-10-11T09:51:25.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.416+0000 7f422dc2c700 1 --2- 172.21.15.78:0/3008259013 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42280d1660 0x7f42280d0a50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4218009850 tx=0x7f4218009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf70bd93879c5a58 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:25.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.416+0000 7f422d42b700 1 -- 172.21.15.78:0/3008259013 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4218011830 con 0x7f42280d1660 2023-10-11T09:51:25.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.416+0000 7f422d42b700 1 -- 172.21.15.78:0/3008259013 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4218011990 con 0x7f42280d1660 2023-10-11T09:51:25.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.416+0000 7f422d42b700 1 -- 172.21.15.78:0/3008259013 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4218011d40 con 0x7f42280d1660 2023-10-11T09:51:25.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.416+0000 7f422f42f700 1 -- 172.21.15.78:0/3008259013 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42280d1660 msgr2=0x7f42280d0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:25.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.416+0000 7f422f42f700 1 --2- 172.21.15.78:0/3008259013 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42280d1660 0x7f42280d0a50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4218009850 tx=0x7f4218009b60 comp rx=0 tx=0).stop 2023-10-11T09:51:25.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.416+0000 7f422f42f700 1 -- 172.21.15.78:0/3008259013 shutdown_connections 2023-10-11T09:51:25.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.416+0000 7f422f42f700 1 --2- 172.21.15.78:0/3008259013 >> v2:172.21.15.133:3300/0 conn(0x7f42280d28e0 0x7f42280d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:25.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.416+0000 7f422f42f700 1 --2- 172.21.15.78:0/3008259013 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42280d1660 0x7f42280d0a50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:25.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.416+0000 7f422f42f700 1 -- 172.21.15.78:0/3008259013 >> 172.21.15.78:0/3008259013 conn(0x7f422806afb0 msgr2=0x7f422806b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:25.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.416+0000 7f422f42f700 1 -- 172.21.15.78:0/3008259013 shutdown_connections 2023-10-11T09:51:25.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.416+0000 7f422f42f700 1 -- 172.21.15.78:0/3008259013 wait complete. 2023-10-11T09:51:25.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.420+0000 7f422f42f700 1 Processor -- start 2023-10-11T09:51:25.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.420+0000 7f422f42f700 1 -- start start 2023-10-11T09:51:25.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.420+0000 7f422f42f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42280d1660 0x7f4228149f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:25.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.420+0000 7f422f42f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42280d28e0 0x7f42281572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:25.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.420+0000 7f422f42f700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4228156ce0 0x7f42280cc920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:25.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.420+0000 7f422f42f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4228157800 con 0x7f42280d1660 2023-10-11T09:51:25.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.420+0000 7f422f42f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42280cb880 con 0x7f42280d28e0 2023-10-11T09:51:25.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.420+0000 7f422f42f700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42280cbb60 con 0x7f4228156ce0 2023-10-11T09:51:25.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.420+0000 7f422e42d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42280d1660 0x7f4228149f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:25.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.420+0000 7f422e42d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42280d1660 0x7f4228149f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34488/0 (socket says 172.21.15.78:34488) 2023-10-11T09:51:25.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.420+0000 7f422e42d700 1 -- 172.21.15.78:0/3691814458 learned_addr learned my addr 172.21.15.78:0/3691814458 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:25.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.420+0000 7f422dc2c700 1 --2- 172.21.15.78:0/3691814458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42280d28e0 0x7f42281572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:25.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.420+0000 7f422ec2e700 1 --2- 172.21.15.78:0/3691814458 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4228156ce0 0x7f42280cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:25.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.420+0000 7f422e42d700 1 -- 172.21.15.78:0/3691814458 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4228156ce0 msgr2=0x7f42280cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:25.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.420+0000 7f422e42d700 1 --2- 172.21.15.78:0/3691814458 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4228156ce0 0x7f42280cc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:25.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.420+0000 7f422e42d700 1 -- 172.21.15.78:0/3691814458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42280d28e0 msgr2=0x7f42281572c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:25.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.420+0000 7f422e42d700 1 --2- 172.21.15.78:0/3691814458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42280d28e0 0x7f42281572c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:25.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.420+0000 7f422e42d700 1 -- 172.21.15.78:0/3691814458 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42180094b0 con 0x7f42280d1660 2023-10-11T09:51:25.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.420+0000 7f422ec2e700 1 --2- 172.21.15.78:0/3691814458 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4228156ce0 0x7f42280cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:51:25.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.424+0000 7f422e42d700 1 --2- 172.21.15.78:0/3691814458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42280d1660 0x7f4228149f70 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f422400c4d0 tx=0x7f422400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:25.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.424+0000 7f421effd700 1 -- 172.21.15.78:0/3691814458 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4224005d90 con 0x7f42280d1660 2023-10-11T09:51:25.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.424+0000 7f421effd700 1 -- 172.21.15.78:0/3691814458 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42240070d0 con 0x7f42280d1660 2023-10-11T09:51:25.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.424+0000 7f422f42f700 1 -- 172.21.15.78:0/3691814458 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42280cbe40 con 0x7f42280d1660 2023-10-11T09:51:25.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.424+0000 7f421effd700 1 -- 172.21.15.78:0/3691814458 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4224007480 con 0x7f42280d1660 2023-10-11T09:51:25.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.424+0000 7f422f42f700 1 -- 172.21.15.78:0/3691814458 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42280cc400 con 0x7f42280d1660 2023-10-11T09:51:25.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.424+0000 7f422f42f700 1 -- 172.21.15.78:0/3691814458 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4228065d40 con 0x7f42280d1660 2023-10-11T09:51:25.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.428+0000 7f421effd700 1 -- 172.21.15.78:0/3691814458 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f42240075e0 con 0x7f42280d1660 2023-10-11T09:51:25.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.428+0000 7f421effd700 1 --2- 172.21.15.78:0/3691814458 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f420c065ed0 0x7f420c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:25.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.428+0000 7f421effd700 1 -- 172.21.15.78:0/3691814458 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f4224083090 con 0x7f42280d1660 2023-10-11T09:51:25.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.428+0000 7f422dc2c700 1 --2- 172.21.15.78:0/3691814458 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f420c065ed0 0x7f420c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:25.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.428+0000 7f422dc2c700 1 --2- 172.21.15.78:0/3691814458 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f420c065ed0 0x7f420c068380 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f42280d1a30 tx=0x7f4218008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:25.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.436+0000 7f421effd700 1 -- 172.21.15.78:0/3691814458 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f4224050310 con 0x7f42280d1660 2023-10-11T09:51:25.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.608+0000 7f422f42f700 1 -- 172.21.15.78:0/3691814458 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42280cc5f0 con 0x7f42280d1660 2023-10-11T09:51:25.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.612+0000 7f421effd700 1 -- 172.21.15.78:0/3691814458 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+20012 (secure 0 0 0) 0x7f4224017090 con 0x7f42280d1660 2023-10-11T09:51:25.622 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:25.622 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":114,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:50:34.654747+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:49:49.297597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-11T09:49:49.955104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data_extra","create_time":"2023-10-11T09:49:59.915561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6836/2876755099":"2023-10-12T09:50:34.623416+0000","172.21.15.78:6837/1390949105":"2023-10-12T09:50:28.599447+0000","172.21.15.78:6836/1390949105":"2023-10-12T09:50:28.599447+0000","172.21.15.78:6837/2876755099":"2023-10-12T09:50:34.623416+0000","172.21.15.78:6837/1286798022":"2023-10-12T09:50:27.559697+0000","172.21.15.78:6836/1286798022":"2023-10-12T09:50:27.559697+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:51:25.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.612+0000 7f422f42f700 1 -- 172.21.15.78:0/3691814458 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f420c065ed0 msgr2=0x7f420c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:25.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.612+0000 7f422f42f700 1 --2- 172.21.15.78:0/3691814458 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f420c065ed0 0x7f420c068380 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f42280d1a30 tx=0x7f4218008030 comp rx=0 tx=0).stop 2023-10-11T09:51:25.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.612+0000 7f422f42f700 1 -- 172.21.15.78:0/3691814458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42280d1660 msgr2=0x7f4228149f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:25.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.612+0000 7f422f42f700 1 --2- 172.21.15.78:0/3691814458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42280d1660 0x7f4228149f70 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f422400c4d0 tx=0x7f422400c7e0 comp rx=0 tx=0).stop 2023-10-11T09:51:25.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.612+0000 7f422f42f700 1 -- 172.21.15.78:0/3691814458 shutdown_connections 2023-10-11T09:51:25.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.612+0000 7f422f42f700 1 --2- 172.21.15.78:0/3691814458 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f420c065ed0 0x7f420c068380 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:25.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.612+0000 7f422f42f700 1 --2- 172.21.15.78:0/3691814458 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4228156ce0 0x7f42280cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:25.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.612+0000 7f422f42f700 1 --2- 172.21.15.78:0/3691814458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42280d28e0 0x7f42281572c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:25.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.612+0000 7f422f42f700 1 --2- 172.21.15.78:0/3691814458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42280d1660 0x7f4228149f70 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:25.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.612+0000 7f422f42f700 1 -- 172.21.15.78:0/3691814458 >> 172.21.15.78:0/3691814458 conn(0x7f422806afb0 msgr2=0x7f422806b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:25.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.616+0000 7f422f42f700 1 -- 172.21.15.78:0/3691814458 shutdown_connections 2023-10-11T09:51:25.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.616+0000 7f422f42f700 1 -- 172.21.15.78:0/3691814458 wait complete. 2023-10-11T09:51:25.645 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-11T09:51:25.646 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:51:25.818 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.808+0000 7f00da578700 1 Processor -- start 2023-10-11T09:51:25.819 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.808+0000 7f00da578700 1 -- start start 2023-10-11T09:51:25.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.812+0000 7f00da578700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00d40ce500 0x7f00d415d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:25.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.812+0000 7f00da578700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f00d4070100 0x7f00d4070580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:25.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.812+0000 7f00da578700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d4159c50 con 0x7f00d4159490 2023-10-11T09:51:25.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.812+0000 7f00da578700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00d4159ef0 con 0x7f00d4070100 2023-10-11T09:51:25.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.812+0000 7f00da578700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d415a1d0 con 0x7f00d40ce500 2023-10-11T09:51:25.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.812+0000 7f00d8b15700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f00d4070100 0x7f00d4070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:25.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.812+0000 7f00d8b15700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f00d4070100 0x7f00d4070580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34494/0 (socket says 172.21.15.78:34494) 2023-10-11T09:51:25.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.812+0000 7f00d8b15700 1 -- 172.21.15.78:0/130561505 learned_addr learned my addr 172.21.15.78:0/130561505 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:25.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.812+0000 7f00cbfff700 1 --2- 172.21.15.78:0/130561505 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00d40ce500 0x7f00d415d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:25.824 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.812+0000 7f00d8b15700 1 -- 172.21.15.78:0/130561505 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00d40ce500 msgr2=0x7f00d415d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:25.824 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.812+0000 7f00d8b15700 1 --2- 172.21.15.78:0/130561505 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00d40ce500 0x7f00d415d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:25.824 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.812+0000 7f00d8b15700 1 -- 172.21.15.78:0/130561505 >> v1:172.21.15.133:6789/0 conn(0x7f00d4159490 legacy=0x7f00d415d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:25.824 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.812+0000 7f00d8b15700 1 -- 172.21.15.78:0/130561505 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d415a4b0 con 0x7f00d4070100 2023-10-11T09:51:25.824 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.812+0000 7f00d8b15700 1 --2- 172.21.15.78:0/130561505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00d4070100 0x7f00d4070580 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f00cc009e50 tx=0x7f00cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e297795482e30a7d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:25.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.812+0000 7f00d37fe700 1 -- 172.21.15.78:0/130561505 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00cc00e070 con 0x7f00d4070100 2023-10-11T09:51:25.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.812+0000 7f00d37fe700 1 -- 172.21.15.78:0/130561505 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00cc00b910 con 0x7f00d4070100 2023-10-11T09:51:25.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.812+0000 7f00d37fe700 1 -- 172.21.15.78:0/130561505 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00cc00bcc0 con 0x7f00d4070100 2023-10-11T09:51:25.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.812+0000 7f00da578700 1 -- 172.21.15.78:0/130561505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00d4070100 msgr2=0x7f00d4070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:25.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.812+0000 7f00da578700 1 --2- 172.21.15.78:0/130561505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00d4070100 0x7f00d4070580 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f00cc009e50 tx=0x7f00cc00b330 comp rx=0 tx=0).stop 2023-10-11T09:51:25.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.816+0000 7f00da578700 1 -- 172.21.15.78:0/130561505 shutdown_connections 2023-10-11T09:51:25.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.816+0000 7f00da578700 1 --2- 172.21.15.78:0/130561505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00d4070100 0x7f00d4070580 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:25.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.816+0000 7f00da578700 1 --2- 172.21.15.78:0/130561505 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00d40ce500 0x7f00d415d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:25.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.816+0000 7f00da578700 1 -- 172.21.15.78:0/130561505 >> 172.21.15.78:0/130561505 conn(0x7f00d406aca0 msgr2=0x7f00d40b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:25.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.816+0000 7f00da578700 1 -- 172.21.15.78:0/130561505 shutdown_connections 2023-10-11T09:51:25.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.816+0000 7f00da578700 1 -- 172.21.15.78:0/130561505 wait complete. 2023-10-11T09:51:25.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.816+0000 7f00da578700 1 Processor -- start 2023-10-11T09:51:25.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.820+0000 7f00da578700 1 -- start start 2023-10-11T09:51:25.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.820+0000 7f00da578700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00d4070100 0x7f00d4147590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:25.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.820+0000 7f00da578700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00d40ce500 0x7f00d40b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:25.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.820+0000 7f00da578700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00d4159490 0x7f00d41f23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:25.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.820+0000 7f00da578700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d4157c00 con 0x7f00d4159490 2023-10-11T09:51:25.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.820+0000 7f00da578700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d414e950 con 0x7f00d4070100 2023-10-11T09:51:25.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.820+0000 7f00da578700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d4156f80 con 0x7f00d40ce500 2023-10-11T09:51:25.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.820+0000 7f00d8b15700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00d4159490 0x7f00d41f23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:25.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.820+0000 7f00cbfff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00d40ce500 0x7f00d40b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:25.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.820+0000 7f00d8b15700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00d4159490 0x7f00d41f23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34510/0 (socket says 172.21.15.78:34510) 2023-10-11T09:51:25.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.820+0000 7f00d8b15700 1 -- 172.21.15.78:0/1160892719 learned_addr learned my addr 172.21.15.78:0/1160892719 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:25.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.820+0000 7f00d3fff700 1 --2- 172.21.15.78:0/1160892719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00d4070100 0x7f00d4147590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:25.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.820+0000 7f00d8b15700 1 -- 172.21.15.78:0/1160892719 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00d40ce500 msgr2=0x7f00d40b44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:25.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.820+0000 7f00d8b15700 1 --2- 172.21.15.78:0/1160892719 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00d40ce500 0x7f00d40b44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:25.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.820+0000 7f00d8b15700 1 -- 172.21.15.78:0/1160892719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00d4070100 msgr2=0x7f00d4147590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:25.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.820+0000 7f00d8b15700 1 --2- 172.21.15.78:0/1160892719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00d4070100 0x7f00d4147590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:25.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.820+0000 7f00d8b15700 1 -- 172.21.15.78:0/1160892719 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00cc009b30 con 0x7f00d4159490 2023-10-11T09:51:25.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.820+0000 7f00d8b15700 1 --2- 172.21.15.78:0/1160892719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00d4159490 0x7f00d41f23b0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f00d4072540 tx=0x7f00cc004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:25.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.820+0000 7f00d17fa700 1 -- 172.21.15.78:0/1160892719 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00cc004800 con 0x7f00d4159490 2023-10-11T09:51:25.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.820+0000 7f00d17fa700 1 -- 172.21.15.78:0/1160892719 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00cc0099b0 con 0x7f00d4159490 2023-10-11T09:51:25.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.820+0000 7f00da578700 1 -- 172.21.15.78:0/1160892719 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00d4157260 con 0x7f00d4159490 2023-10-11T09:51:25.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.820+0000 7f00d17fa700 1 -- 172.21.15.78:0/1160892719 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00cc01c680 con 0x7f00d4159490 2023-10-11T09:51:25.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.820+0000 7f00da578700 1 -- 172.21.15.78:0/1160892719 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d4157820 con 0x7f00d4159490 2023-10-11T09:51:25.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.824+0000 7f00da578700 1 -- 172.21.15.78:0/1160892719 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00d40721a0 con 0x7f00d4159490 2023-10-11T09:51:25.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.824+0000 7f00d17fa700 1 -- 172.21.15.78:0/1160892719 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f00cc01c7e0 con 0x7f00d4159490 2023-10-11T09:51:25.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.828+0000 7f00d17fa700 1 --2- 172.21.15.78:0/1160892719 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f00b4065ed0 0x7f00b4068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:25.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.828+0000 7f00d17fa700 1 -- 172.21.15.78:0/1160892719 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f00cc0856b0 con 0x7f00d4159490 2023-10-11T09:51:25.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.828+0000 7f00d3fff700 1 --2- 172.21.15.78:0/1160892719 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f00b4065ed0 0x7f00b4068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:25.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.828+0000 7f00d3fff700 1 --2- 172.21.15.78:0/1160892719 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f00b4065ed0 0x7f00b4068380 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f00c4000f60 tx=0x7f00c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:25.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:25.832+0000 7f00d17fa700 1 -- 172.21.15.78:0/1160892719 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f00cc052a50 con 0x7f00d4159490 2023-10-11T09:51:26.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.040+0000 7f00da578700 1 -- 172.21.15.78:0/1160892719 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00d4147b90 con 0x7f00d4159490 2023-10-11T09:51:26.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.040+0000 7f00d17fa700 1 -- 172.21.15.78:0/1160892719 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4655 (secure 0 0 0) 0x7f00cc056070 con 0x7f00d4159490 2023-10-11T09:51:26.051 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:26.051 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":91,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7148,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/331520590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":331520590},{"type":"v1","addr":"172.21.15.78:6835","nonce":331520590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":7151,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/934539531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":934539531},{"type":"v1","addr":"172.21.15.133:6834","nonce":934539531}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":7154,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3894002128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":3894002128},{"type":"v1","addr":"172.21.15.133:6835","nonce":3894002128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:50:28.502984+0000","modified":"2023-10-11T09:50:37.659950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8550":{"gid":8550,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":4,"addr":"172.21.15.78:6837/2869034036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2869034036},{"type":"v1","addr":"172.21.15.78:6837","nonce":2869034036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15,16],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-11T09:51:26.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.044+0000 7f00da578700 1 -- 172.21.15.78:0/1160892719 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f00b4065ed0 msgr2=0x7f00b4068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:26.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.044+0000 7f00da578700 1 --2- 172.21.15.78:0/1160892719 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f00b4065ed0 0x7f00b4068380 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f00c4000f60 tx=0x7f00c4009040 comp rx=0 tx=0).stop 2023-10-11T09:51:26.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.044+0000 7f00da578700 1 -- 172.21.15.78:0/1160892719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00d4159490 msgr2=0x7f00d41f23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:26.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.044+0000 7f00da578700 1 --2- 172.21.15.78:0/1160892719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00d4159490 0x7f00d41f23b0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f00d4072540 tx=0x7f00cc004af0 comp rx=0 tx=0).stop 2023-10-11T09:51:26.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.044+0000 7f00da578700 1 -- 172.21.15.78:0/1160892719 shutdown_connections 2023-10-11T09:51:26.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.044+0000 7f00da578700 1 --2- 172.21.15.78:0/1160892719 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f00b4065ed0 0x7f00b4068380 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:26.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.044+0000 7f00da578700 1 --2- 172.21.15.78:0/1160892719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00d4159490 0x7f00d41f23b0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:26.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.044+0000 7f00da578700 1 --2- 172.21.15.78:0/1160892719 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00d40ce500 0x7f00d40b44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:26.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.044+0000 7f00da578700 1 --2- 172.21.15.78:0/1160892719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00d4070100 0x7f00d4147590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:26.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.044+0000 7f00da578700 1 -- 172.21.15.78:0/1160892719 >> 172.21.15.78:0/1160892719 conn(0x7f00d406aca0 msgr2=0x7f00d415cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:26.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.044+0000 7f00da578700 1 -- 172.21.15.78:0/1160892719 shutdown_connections 2023-10-11T09:51:26.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.044+0000 7f00da578700 1 -- 172.21.15.78:0/1160892719 wait complete. 2023-10-11T09:51:26.056 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 91 2023-10-11T09:51:26.075 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:51:26.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.228+0000 7fae12516700 1 Processor -- start 2023-10-11T09:51:26.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.228+0000 7fae12516700 1 -- start start 2023-10-11T09:51:26.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.228+0000 7fae12516700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fae0c15d050 0x7fae0c0cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:26.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.228+0000 7fae12516700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae0c160c60 0x7fae0c15d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:26.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.228+0000 7fae12516700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae0c160210 con 0x7fae0c15d960 2023-10-11T09:51:26.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.228+0000 7fae12516700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae0c0ccd60 con 0x7fae0c15d050 2023-10-11T09:51:26.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.228+0000 7fae12516700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c0ccfc0 con 0x7fae0c160c60 2023-10-11T09:51:26.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.228+0000 7fae11514700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fae0c15d050 0x7fae0c0cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:26.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.228+0000 7fae11514700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fae0c15d050 0x7fae0c0cc820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34514/0 (socket says 172.21.15.78:34514) 2023-10-11T09:51:26.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.228+0000 7fae11514700 1 -- 172.21.15.78:0/3692678375 learned_addr learned my addr 172.21.15.78:0/3692678375 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:26.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.232+0000 7fae10d13700 1 --2- 172.21.15.78:0/3692678375 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae0c160c60 0x7fae0c15d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:26.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.232+0000 7fae11514700 1 -- 172.21.15.78:0/3692678375 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae0c160c60 msgr2=0x7fae0c15d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:26.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.232+0000 7fae11514700 1 --2- 172.21.15.78:0/3692678375 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae0c160c60 0x7fae0c15d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:26.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.232+0000 7fae11514700 1 -- 172.21.15.78:0/3692678375 >> v1:172.21.15.133:6789/0 conn(0x7fae0c15d960 legacy=0x7fae0c0b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:26.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.232+0000 7fae11514700 1 -- 172.21.15.78:0/3692678375 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0c0cd2a0 con 0x7fae0c15d050 2023-10-11T09:51:26.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.232+0000 7fae03fff700 1 -- 172.21.15.78:0/3692678375 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3366250270 0 0) 0x7fae0c160210 con 0x7fae0c15d960 2023-10-11T09:51:26.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.232+0000 7fae11514700 1 --2- 172.21.15.78:0/3692678375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae0c15d050 0x7fae0c0cc820 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fae08009910 tx=0x7fae08009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ceb8814c2b928ec server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:26.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.232+0000 7fae03fff700 1 -- 172.21.15.78:0/3692678375 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae08011830 con 0x7fae0c15d050 2023-10-11T09:51:26.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.232+0000 7fae03fff700 1 -- 172.21.15.78:0/3692678375 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae08011990 con 0x7fae0c15d050 2023-10-11T09:51:26.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.232+0000 7fae03fff700 1 -- 172.21.15.78:0/3692678375 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae08011d40 con 0x7fae0c15d050 2023-10-11T09:51:26.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.232+0000 7fae12516700 1 -- 172.21.15.78:0/3692678375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae0c15d050 msgr2=0x7fae0c0cc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:26.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.232+0000 7fae12516700 1 --2- 172.21.15.78:0/3692678375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae0c15d050 0x7fae0c0cc820 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fae08009910 tx=0x7fae08009c20 comp rx=0 tx=0).stop 2023-10-11T09:51:26.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.232+0000 7fae12516700 1 -- 172.21.15.78:0/3692678375 shutdown_connections 2023-10-11T09:51:26.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.232+0000 7fae12516700 1 --2- 172.21.15.78:0/3692678375 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae0c160c60 0x7fae0c15d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:26.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.232+0000 7fae12516700 1 --2- 172.21.15.78:0/3692678375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae0c15d050 0x7fae0c0cc820 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:26.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.232+0000 7fae12516700 1 -- 172.21.15.78:0/3692678375 >> 172.21.15.78:0/3692678375 conn(0x7fae0c0cea60 msgr2=0x7fae0c0cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:26.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.232+0000 7fae12516700 1 -- 172.21.15.78:0/3692678375 shutdown_connections 2023-10-11T09:51:26.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.232+0000 7fae12516700 1 -- 172.21.15.78:0/3692678375 wait complete. 2023-10-11T09:51:26.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.236+0000 7fae12516700 1 Processor -- start 2023-10-11T09:51:26.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.236+0000 7fae12516700 1 -- start start 2023-10-11T09:51:26.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.236+0000 7fae12516700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae0c15d050 0x7fae0c0b2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:26.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.236+0000 7fae12516700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae0c15d960 0x7fae0c0b3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:26.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.236+0000 7fae12516700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae0c160c60 0x7fae0c0b4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:26.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.236+0000 7fae12516700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c0b3230 con 0x7fae0c15d050 2023-10-11T09:51:26.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.236+0000 7fae12516700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c06fda0 con 0x7fae0c160c60 2023-10-11T09:51:26.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.236+0000 7fae12516700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c0b2c70 con 0x7fae0c15d960 2023-10-11T09:51:26.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.236+0000 7fae11514700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae0c15d050 0x7fae0c0b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:26.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.236+0000 7fae11514700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae0c15d050 0x7fae0c0b2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34520/0 (socket says 172.21.15.78:34520) 2023-10-11T09:51:26.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.236+0000 7fae11514700 1 -- 172.21.15.78:0/32790953 learned_addr learned my addr 172.21.15.78:0/32790953 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:26.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.236+0000 7fae11d15700 1 --2- 172.21.15.78:0/32790953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae0c160c60 0x7fae0c0b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:26.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.240+0000 7fae10d13700 1 --2- 172.21.15.78:0/32790953 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae0c15d960 0x7fae0c0b3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:26.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.240+0000 7fae11514700 1 -- 172.21.15.78:0/32790953 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae0c15d960 msgr2=0x7fae0c0b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:26.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.240+0000 7fae11514700 1 --2- 172.21.15.78:0/32790953 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae0c15d960 0x7fae0c0b3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:26.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.240+0000 7fae11514700 1 -- 172.21.15.78:0/32790953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae0c160c60 msgr2=0x7fae0c0b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:26.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.240+0000 7fae11514700 1 --2- 172.21.15.78:0/32790953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae0c160c60 0x7fae0c0b4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:26.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.240+0000 7fae11514700 1 -- 172.21.15.78:0/32790953 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae08009570 con 0x7fae0c15d050 2023-10-11T09:51:26.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.240+0000 7fae10d13700 1 --2- 172.21.15.78:0/32790953 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae0c15d960 0x7fae0c0b3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:51:26.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.240+0000 7fae11514700 1 --2- 172.21.15.78:0/32790953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae0c15d050 0x7fae0c0b2420 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fae08014040 tx=0x7fae08004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:26.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.240+0000 7fae01ffb700 1 -- 172.21.15.78:0/32790953 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0800b970 con 0x7fae0c15d050 2023-10-11T09:51:26.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.240+0000 7fae01ffb700 1 -- 172.21.15.78:0/32790953 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0801e070 con 0x7fae0c15d050 2023-10-11T09:51:26.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.240+0000 7fae01ffb700 1 -- 172.21.15.78:0/32790953 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0800bd30 con 0x7fae0c15d050 2023-10-11T09:51:26.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.240+0000 7fae12516700 1 -- 172.21.15.78:0/32790953 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae0c0b2f50 con 0x7fae0c15d050 2023-10-11T09:51:26.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.240+0000 7fae12516700 1 -- 172.21.15.78:0/32790953 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae0c0b2960 con 0x7fae0c15d050 2023-10-11T09:51:26.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.240+0000 7fae12516700 1 -- 172.21.15.78:0/32790953 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae0c070280 con 0x7fae0c15d050 2023-10-11T09:51:26.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.244+0000 7fae01ffb700 1 -- 172.21.15.78:0/32790953 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fae0802d450 con 0x7fae0c15d050 2023-10-11T09:51:26.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.244+0000 7fae01ffb700 1 --2- 172.21.15.78:0/32790953 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fadf0065ed0 0x7fadf0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:26.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.244+0000 7fae01ffb700 1 -- 172.21.15.78:0/32790953 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6724+0+0 (secure 0 0 0) 0x7fae08084a70 con 0x7fae0c15d050 2023-10-11T09:51:26.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.244+0000 7fae10d13700 1 --2- 172.21.15.78:0/32790953 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fadf0065ed0 0x7fadf0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:26.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.244+0000 7fae10d13700 1 --2- 172.21.15.78:0/32790953 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fadf0065ed0 0x7fadf0068380 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fae0c0cd500 tx=0x7fadfc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:26.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.248+0000 7fae01ffb700 1 -- 172.21.15.78:0/32790953 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fae0c070280 con 0x7fae0c15d050 2023-10-11T09:51:26.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.456+0000 7fae12516700 1 -- 172.21.15.78:0/32790953 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae0c070280 con 0x7fae0c15d050 2023-10-11T09:51:26.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.460+0000 7fae01ffb700 1 -- 172.21.15.78:0/32790953 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4655 (secure 0 0 0) 0x7fae0c070280 con 0x7fae0c15d050 2023-10-11T09:51:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:26.470 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":91,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7148,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/331520590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":331520590},{"type":"v1","addr":"172.21.15.78:6835","nonce":331520590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":7151,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/934539531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":934539531},{"type":"v1","addr":"172.21.15.133:6834","nonce":934539531}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":7154,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3894002128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":3894002128},{"type":"v1","addr":"172.21.15.133:6835","nonce":3894002128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:50:28.502984+0000","modified":"2023-10-11T09:50:37.659950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8550":{"gid":8550,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":4,"addr":"172.21.15.78:6837/2869034036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":2869034036},{"type":"v1","addr":"172.21.15.78:6837","nonce":2869034036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15,16],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-11T09:51:26.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.460+0000 7fae12516700 1 -- 172.21.15.78:0/32790953 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fadf0065ed0 msgr2=0x7fadf0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:26.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.460+0000 7fae12516700 1 --2- 172.21.15.78:0/32790953 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fadf0065ed0 0x7fadf0068380 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fae0c0cd500 tx=0x7fadfc008040 comp rx=0 tx=0).stop 2023-10-11T09:51:26.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.460+0000 7fae12516700 1 -- 172.21.15.78:0/32790953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae0c15d050 msgr2=0x7fae0c0b2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:26.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.460+0000 7fae12516700 1 --2- 172.21.15.78:0/32790953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae0c15d050 0x7fae0c0b2420 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fae08014040 tx=0x7fae08004770 comp rx=0 tx=0).stop 2023-10-11T09:51:26.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.460+0000 7fae12516700 1 -- 172.21.15.78:0/32790953 shutdown_connections 2023-10-11T09:51:26.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.460+0000 7fae12516700 1 --2- 172.21.15.78:0/32790953 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fadf0065ed0 0x7fadf0068380 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:26.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.460+0000 7fae12516700 1 --2- 172.21.15.78:0/32790953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae0c160c60 0x7fae0c0b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:26.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.460+0000 7fae12516700 1 --2- 172.21.15.78:0/32790953 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae0c15d960 0x7fae0c0b3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:26.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.460+0000 7fae12516700 1 --2- 172.21.15.78:0/32790953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae0c15d050 0x7fae0c0b2420 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:26.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.460+0000 7fae12516700 1 -- 172.21.15.78:0/32790953 >> 172.21.15.78:0/32790953 conn(0x7fae0c0cea60 msgr2=0x7fae0c06c950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:26.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.460+0000 7fae12516700 1 -- 172.21.15.78:0/32790953 shutdown_connections 2023-10-11T09:51:26.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.464+0000 7fae12516700 1 -- 172.21.15.78:0/32790953 wait complete. 2023-10-11T09:51:26.475 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 91 2023-10-11T09:51:26.492 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:51:26.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.648+0000 7f4e53351700 1 Processor -- start 2023-10-11T09:51:26.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.648+0000 7f4e53351700 1 -- start start 2023-10-11T09:51:26.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.652+0000 7f4e53351700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4e4c15d050 0x7f4e4c0cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:26.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.652+0000 7f4e53351700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e4c160c60 0x7f4e4c15d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:26.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.652+0000 7f4e53351700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f4e4c15d960 0x7f4e4c0b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:26.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.652+0000 7f4e53351700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e4c160210 con 0x7f4e4c15d960 2023-10-11T09:51:26.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.652+0000 7f4e53351700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e4c0ccd60 con 0x7f4e4c15d050 2023-10-11T09:51:26.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.652+0000 7f4e53351700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c0ccfc0 con 0x7f4e4c160c60 2023-10-11T09:51:26.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.652+0000 7f4e52b50700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f4e4c15d960 0x7f4e4c0b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:26.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.652+0000 7f4e5234f700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4e4c15d050 0x7f4e4c0cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:26.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.652+0000 7f4e52b50700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f4e4c15d960 0x7f4e4c0b06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34534/0 (socket says 172.21.15.78:34534) 2023-10-11T09:51:26.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.652+0000 7f4e51b4e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e4c160c60 0x7f4e4c15d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:26.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.652+0000 7f4e52b50700 1 -- 172.21.15.78:0/4028746326 learned_addr learned my addr 172.21.15.78:0/4028746326 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:26.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.652+0000 7f4e5234f700 1 -- 172.21.15.78:0/4028746326 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e4c160c60 msgr2=0x7f4e4c15d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:26.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.652+0000 7f4e5234f700 1 --2- 172.21.15.78:0/4028746326 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e4c160c60 0x7f4e4c15d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:26.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.652+0000 7f4e5234f700 1 -- 172.21.15.78:0/4028746326 >> v2:172.21.15.78:3300/0 conn(0x7f4e4c15d960 msgr2=0x7f4e4c0b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:26.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.652+0000 7f4e5234f700 1 --2- 172.21.15.78:0/4028746326 >> v2:172.21.15.78:3300/0 conn(0x7f4e4c15d960 0x7f4e4c0b06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:26.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.652+0000 7f4e5234f700 1 -- 172.21.15.78:0/4028746326 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e4c0cd2a0 con 0x7f4e4c15d050 2023-10-11T09:51:26.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.652+0000 7f4e52b50700 1 --2- 172.21.15.78:0/4028746326 >> v2:172.21.15.78:3300/0 conn(0x7f4e4c15d960 0x7f4e4c0b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:51:26.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.652+0000 7f4e5234f700 1 --2- 172.21.15.78:0/4028746326 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e4c15d050 0x7f4e4c0cc820 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c0cf8b0 tx=0x7f4e48009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d32e682acabb868a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:26.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.652+0000 7f4e5134d700 1 -- 172.21.15.78:0/4028746326 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e48011830 con 0x7f4e4c15d050 2023-10-11T09:51:26.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.652+0000 7f4e5134d700 1 -- 172.21.15.78:0/4028746326 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e48011990 con 0x7f4e4c15d050 2023-10-11T09:51:26.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.652+0000 7f4e5134d700 1 -- 172.21.15.78:0/4028746326 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e48011d20 con 0x7f4e4c15d050 2023-10-11T09:51:26.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.656+0000 7f4e53351700 1 -- 172.21.15.78:0/4028746326 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e4c15d050 msgr2=0x7f4e4c0cc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:26.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.656+0000 7f4e53351700 1 --2- 172.21.15.78:0/4028746326 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e4c15d050 0x7f4e4c0cc820 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c0cf8b0 tx=0x7f4e48009b30 comp rx=0 tx=0).stop 2023-10-11T09:51:26.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.656+0000 7f4e53351700 1 -- 172.21.15.78:0/4028746326 shutdown_connections 2023-10-11T09:51:26.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.656+0000 7f4e53351700 1 --2- 172.21.15.78:0/4028746326 >> v2:172.21.15.78:3300/0 conn(0x7f4e4c15d960 0x7f4e4c0b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:26.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.656+0000 7f4e53351700 1 --2- 172.21.15.78:0/4028746326 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e4c160c60 0x7f4e4c15d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:26.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.656+0000 7f4e53351700 1 --2- 172.21.15.78:0/4028746326 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e4c15d050 0x7f4e4c0cc820 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:26.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.656+0000 7f4e53351700 1 -- 172.21.15.78:0/4028746326 >> 172.21.15.78:0/4028746326 conn(0x7f4e4c0cea60 msgr2=0x7f4e4c0cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:26.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.656+0000 7f4e53351700 1 -- 172.21.15.78:0/4028746326 shutdown_connections 2023-10-11T09:51:26.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.656+0000 7f4e53351700 1 -- 172.21.15.78:0/4028746326 wait complete. 2023-10-11T09:51:26.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.656+0000 7f4e53351700 1 Processor -- start 2023-10-11T09:51:26.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.656+0000 7f4e53351700 1 -- start start 2023-10-11T09:51:26.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.660+0000 7f4e53351700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e4c15d050 0x7f4e4c1f1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:26.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.660+0000 7f4e53351700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e4c15d960 0x7f4e4c151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:26.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.660+0000 7f4e53351700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e4c160c60 0x7f4e4c0b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:26.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.660+0000 7f4e53351700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c1f20b0 con 0x7f4e4c15d960 2023-10-11T09:51:26.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.660+0000 7f4e53351700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c152440 con 0x7f4e4c160c60 2023-10-11T09:51:26.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.660+0000 7f4e53351700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c0700d0 con 0x7f4e4c15d050 2023-10-11T09:51:26.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.660+0000 7f4e51b4e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e4c15d960 0x7f4e4c151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:26.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.660+0000 7f4e51b4e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e4c15d960 0x7f4e4c151f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34536/0 (socket says 172.21.15.78:34536) 2023-10-11T09:51:26.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.660+0000 7f4e51b4e700 1 -- 172.21.15.78:0/1453556694 learned_addr learned my addr 172.21.15.78:0/1453556694 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:26.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.660+0000 7f4e52b50700 1 --2- 172.21.15.78:0/1453556694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e4c160c60 0x7f4e4c0b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:26.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.660+0000 7f4e5234f700 1 --2- 172.21.15.78:0/1453556694 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e4c15d050 0x7f4e4c1f1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:26.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.660+0000 7f4e51b4e700 1 -- 172.21.15.78:0/1453556694 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e4c15d050 msgr2=0x7f4e4c1f1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:26.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.660+0000 7f4e51b4e700 1 --2- 172.21.15.78:0/1453556694 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e4c15d050 0x7f4e4c1f1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:26.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.660+0000 7f4e51b4e700 1 -- 172.21.15.78:0/1453556694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e4c160c60 msgr2=0x7f4e4c0b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:26.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.660+0000 7f4e51b4e700 1 --2- 172.21.15.78:0/1453556694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e4c160c60 0x7f4e4c0b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:26.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.660+0000 7f4e51b4e700 1 -- 172.21.15.78:0/1453556694 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e44009b30 con 0x7f4e4c15d960 2023-10-11T09:51:26.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.660+0000 7f4e52b50700 1 --2- 172.21.15.78:0/1453556694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e4c160c60 0x7f4e4c0b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:51:26.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.660+0000 7f4e51b4e700 1 --2- 172.21.15.78:0/1453556694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e4c15d960 0x7f4e4c151f00 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c00ebe0 tx=0x7f4e3c009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:26.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.660+0000 7f4e42ffd700 1 -- 172.21.15.78:0/1453556694 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c009740 con 0x7f4e4c15d960 2023-10-11T09:51:26.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.660+0000 7f4e42ffd700 1 -- 172.21.15.78:0/1453556694 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e3c00c7f0 con 0x7f4e4c15d960 2023-10-11T09:51:26.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.660+0000 7f4e53351700 1 -- 172.21.15.78:0/1453556694 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e480094b0 con 0x7f4e4c15d960 2023-10-11T09:51:26.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.660+0000 7f4e42ffd700 1 -- 172.21.15.78:0/1453556694 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c00cba0 con 0x7f4e4c15d960 2023-10-11T09:51:26.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.660+0000 7f4e53351700 1 -- 172.21.15.78:0/1453556694 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e4c070320 con 0x7f4e4c15d960 2023-10-11T09:51:26.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.664+0000 7f4e53351700 1 -- 172.21.15.78:0/1453556694 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e4c065d40 con 0x7f4e4c15d960 2023-10-11T09:51:26.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.664+0000 7f4e42ffd700 1 -- 172.21.15.78:0/1453556694 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f4e3c00cd00 con 0x7f4e4c15d960 2023-10-11T09:51:26.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.664+0000 7f4e42ffd700 1 --2- 172.21.15.78:0/1453556694 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4e30065ed0 0x7f4e30068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:26.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.668+0000 7f4e42ffd700 1 -- 172.21.15.78:0/1453556694 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6724+0+0 (secure 0 0 0) 0x7f4e3c083b20 con 0x7f4e4c15d960 2023-10-11T09:51:26.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.668+0000 7f4e5234f700 1 --2- 172.21.15.78:0/1453556694 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4e30065ed0 0x7f4e30068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:26.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.668+0000 7f4e5234f700 1 --2- 172.21.15.78:0/1453556694 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4e30065ed0 0x7f4e30068380 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4e48000c00 tx=0x7f4e48008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:26.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.672+0000 7f4e42ffd700 1 -- 172.21.15.78:0/1453556694 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f4e3c050ec0 con 0x7f4e4c15d960 2023-10-11T09:51:26.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.848+0000 7f4e53351700 1 -- 172.21.15.78:0/1453556694 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e4c0b1310 con 0x7f4e4c15d960 2023-10-11T09:51:26.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.852+0000 7f4e42ffd700 1 -- 172.21.15.78:0/1453556694 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+20012 (secure 0 0 0) 0x7f4e3c0544e0 con 0x7f4e4c15d960 2023-10-11T09:51:26.863 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:26.864 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":114,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:50:34.654747+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:49:49.297597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-11T09:49:49.955104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data_extra","create_time":"2023-10-11T09:49:59.915561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6836/2876755099":"2023-10-12T09:50:34.623416+0000","172.21.15.78:6837/1390949105":"2023-10-12T09:50:28.599447+0000","172.21.15.78:6836/1390949105":"2023-10-12T09:50:28.599447+0000","172.21.15.78:6837/2876755099":"2023-10-12T09:50:34.623416+0000","172.21.15.78:6837/1286798022":"2023-10-12T09:50:27.559697+0000","172.21.15.78:6836/1286798022":"2023-10-12T09:50:27.559697+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:51:26.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.856+0000 7f4e53351700 1 -- 172.21.15.78:0/1453556694 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4e30065ed0 msgr2=0x7f4e30068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:26.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.856+0000 7f4e53351700 1 --2- 172.21.15.78:0/1453556694 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4e30065ed0 0x7f4e30068380 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4e48000c00 tx=0x7f4e48008030 comp rx=0 tx=0).stop 2023-10-11T09:51:26.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.856+0000 7f4e53351700 1 -- 172.21.15.78:0/1453556694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e4c15d960 msgr2=0x7f4e4c151f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:26.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.856+0000 7f4e53351700 1 --2- 172.21.15.78:0/1453556694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e4c15d960 0x7f4e4c151f00 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c00ebe0 tx=0x7f4e3c009e00 comp rx=0 tx=0).stop 2023-10-11T09:51:26.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.856+0000 7f4e53351700 1 -- 172.21.15.78:0/1453556694 shutdown_connections 2023-10-11T09:51:26.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.856+0000 7f4e53351700 1 --2- 172.21.15.78:0/1453556694 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4e30065ed0 0x7f4e30068380 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:26.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.856+0000 7f4e53351700 1 --2- 172.21.15.78:0/1453556694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e4c160c60 0x7f4e4c0b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:26.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.856+0000 7f4e53351700 1 --2- 172.21.15.78:0/1453556694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e4c15d960 0x7f4e4c151f00 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:26.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.856+0000 7f4e53351700 1 --2- 172.21.15.78:0/1453556694 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e4c15d050 0x7f4e4c1f1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:26.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.856+0000 7f4e53351700 1 -- 172.21.15.78:0/1453556694 >> 172.21.15.78:0/1453556694 conn(0x7f4e4c0cea60 msgr2=0x7f4e4c06c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:26.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.856+0000 7f4e53351700 1 -- 172.21.15.78:0/1453556694 shutdown_connections 2023-10-11T09:51:26.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:26.856+0000 7f4e53351700 1 -- 172.21.15.78:0/1453556694 wait complete. 2023-10-11T09:51:26.886 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-11T09:51:27.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.056+0000 7effeef0e700 1 Processor -- start 2023-10-11T09:51:27.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.060+0000 7effeef0e700 1 -- start start 2023-10-11T09:51:27.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.060+0000 7effeef0e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7effe815b8e0 0x7effe806d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:27.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.060+0000 7effeef0e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7effe806d610 0x7effe80ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:27.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.060+0000 7effeef0e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effe815e010 con 0x7effe815a940 2023-10-11T09:51:27.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.060+0000 7effeef0e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effe815e280 con 0x7effe806d610 2023-10-11T09:51:27.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.060+0000 7effeef0e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe815e560 con 0x7effe815b8e0 2023-10-11T09:51:27.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.060+0000 7effed70b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7effe806d610 0x7effe80ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:27.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.060+0000 7effed70b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7effe806d610 0x7effe80ca240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34546/0 (socket says 172.21.15.78:34546) 2023-10-11T09:51:27.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.060+0000 7effed70b700 1 -- 172.21.15.78:0/1545130097 learned_addr learned my addr 172.21.15.78:0/1545130097 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:27.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.060+0000 7effee70d700 1 --1- 172.21.15.78:0/1545130097 >> v1:172.21.15.133:6789/0 conn(0x7effe815a940 0x7effe80c9020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:57270/0 (socket says 172.21.15.78:57270) 2023-10-11T09:51:27.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.060+0000 7effedf0c700 1 --2- 172.21.15.78:0/1545130097 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7effe815b8e0 0x7effe806d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:27.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.064+0000 7effed70b700 1 -- 172.21.15.78:0/1545130097 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7effe815b8e0 msgr2=0x7effe806d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:27.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.064+0000 7effed70b700 1 --2- 172.21.15.78:0/1545130097 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7effe815b8e0 0x7effe806d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:27.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.064+0000 7effed70b700 1 -- 172.21.15.78:0/1545130097 >> v1:172.21.15.133:6789/0 conn(0x7effe815a940 legacy=0x7effe80c9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:27.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.064+0000 7effed70b700 1 -- 172.21.15.78:0/1545130097 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe815e840 con 0x7effe806d610 2023-10-11T09:51:27.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.064+0000 7effed70b700 1 --2- 172.21.15.78:0/1545130097 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effe806d610 0x7effe80ca240 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7effd8009910 tx=0x7effd8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb4146ad680cfa1a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:27.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.064+0000 7effecf0a700 1 -- 172.21.15.78:0/1545130097 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd8011830 con 0x7effe806d610 2023-10-11T09:51:27.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.064+0000 7effecf0a700 1 -- 172.21.15.78:0/1545130097 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd8011990 con 0x7effe806d610 2023-10-11T09:51:27.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.064+0000 7effecf0a700 1 -- 172.21.15.78:0/1545130097 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd8011d40 con 0x7effe806d610 2023-10-11T09:51:27.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.064+0000 7effeef0e700 1 -- 172.21.15.78:0/1545130097 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effe806d610 msgr2=0x7effe80ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:27.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.064+0000 7effeef0e700 1 --2- 172.21.15.78:0/1545130097 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effe806d610 0x7effe80ca240 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7effd8009910 tx=0x7effd8009c20 comp rx=0 tx=0).stop 2023-10-11T09:51:27.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.064+0000 7effeef0e700 1 -- 172.21.15.78:0/1545130097 shutdown_connections 2023-10-11T09:51:27.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.064+0000 7effeef0e700 1 --2- 172.21.15.78:0/1545130097 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effe806d610 0x7effe80ca240 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:27.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.064+0000 7effeef0e700 1 --2- 172.21.15.78:0/1545130097 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7effe815b8e0 0x7effe806d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:27.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.064+0000 7effeef0e700 1 -- 172.21.15.78:0/1545130097 >> 172.21.15.78:0/1545130097 conn(0x7effe80cea60 msgr2=0x7effe80cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:27.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.064+0000 7effeef0e700 1 -- 172.21.15.78:0/1545130097 shutdown_connections 2023-10-11T09:51:27.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.064+0000 7effeef0e700 1 -- 172.21.15.78:0/1545130097 wait complete. 2023-10-11T09:51:27.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.068+0000 7effeef0e700 1 Processor -- start 2023-10-11T09:51:27.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.068+0000 7effeef0e700 1 -- start start 2023-10-11T09:51:27.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.072+0000 7effeef0e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effe806d610 0x7effe80c6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:27.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.072+0000 7effeef0e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effe815a940 0x7effe813fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:27.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.072+0000 7effeef0e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7effe815b8e0 0x7effe80c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:27.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.072+0000 7effeef0e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe80c6a80 con 0x7effe815a940 2023-10-11T09:51:27.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.072+0000 7effeef0e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe80b1a10 con 0x7effe806d610 2023-10-11T09:51:27.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.072+0000 7effeef0e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe80b1cb0 con 0x7effe815b8e0 2023-10-11T09:51:27.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.072+0000 7effed70b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effe815a940 0x7effe813fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:27.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.072+0000 7effed70b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effe815a940 0x7effe813fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34548/0 (socket says 172.21.15.78:34548) 2023-10-11T09:51:27.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.072+0000 7effed70b700 1 -- 172.21.15.78:0/2669957856 learned_addr learned my addr 172.21.15.78:0/2669957856 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:27.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.072+0000 7effedf0c700 1 --2- 172.21.15.78:0/2669957856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effe806d610 0x7effe80c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:27.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.072+0000 7effee70d700 1 --2- 172.21.15.78:0/2669957856 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7effe815b8e0 0x7effe80c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:27.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.072+0000 7effed70b700 1 -- 172.21.15.78:0/2669957856 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7effe815b8e0 msgr2=0x7effe80c8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:27.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.072+0000 7effed70b700 1 --2- 172.21.15.78:0/2669957856 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7effe815b8e0 0x7effe80c8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:27.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.072+0000 7effed70b700 1 -- 172.21.15.78:0/2669957856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effe806d610 msgr2=0x7effe80c6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:27.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.072+0000 7effed70b700 1 --2- 172.21.15.78:0/2669957856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effe806d610 0x7effe80c6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:27.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.072+0000 7effed70b700 1 -- 172.21.15.78:0/2669957856 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd8009570 con 0x7effe815a940 2023-10-11T09:51:27.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.072+0000 7effee70d700 1 --2- 172.21.15.78:0/2669957856 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7effe815b8e0 0x7effe80c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:51:27.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.072+0000 7effed70b700 1 --2- 172.21.15.78:0/2669957856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effe815a940 0x7effe813fc00 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7effd8000c00 tx=0x7effd8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:27.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.072+0000 7effde7fc700 1 -- 172.21.15.78:0/2669957856 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd8004590 con 0x7effe815a940 2023-10-11T09:51:27.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.072+0000 7effde7fc700 1 -- 172.21.15.78:0/2669957856 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd80036c0 con 0x7effe815a940 2023-10-11T09:51:27.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.072+0000 7effeef0e700 1 -- 172.21.15.78:0/2669957856 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effe80b1f90 con 0x7effe815a940 2023-10-11T09:51:27.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.072+0000 7effde7fc700 1 -- 172.21.15.78:0/2669957856 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd8003a70 con 0x7effe815a940 2023-10-11T09:51:27.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.072+0000 7effeef0e700 1 -- 172.21.15.78:0/2669957856 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effe80b2550 con 0x7effe815a940 2023-10-11T09:51:27.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.076+0000 7effeef0e700 1 -- 172.21.15.78:0/2669957856 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effe80b0e30 con 0x7effe815a940 2023-10-11T09:51:27.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.080+0000 7effde7fc700 1 -- 172.21.15.78:0/2669957856 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7effd8003bd0 con 0x7effe815a940 2023-10-11T09:51:27.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.080+0000 7effde7fc700 1 --2- 172.21.15.78:0/2669957856 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7effcc065f00 0x7effcc0683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:27.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.080+0000 7effde7fc700 1 -- 172.21.15.78:0/2669957856 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6724+0+0 (secure 0 0 0) 0x7effd8021030 con 0x7effe815a940 2023-10-11T09:51:27.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.080+0000 7effedf0c700 1 --2- 172.21.15.78:0/2669957856 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7effcc065f00 0x7effcc0683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:27.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.080+0000 7effedf0c700 1 --2- 172.21.15.78:0/2669957856 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7effcc065f00 0x7effcc0683b0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7effe4009b50 tx=0x7effe4009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:27.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.084+0000 7effde7fc700 1 -- 172.21.15.78:0/2669957856 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7effd80529f0 con 0x7effe815a940 2023-10-11T09:51:27.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.296+0000 7effeef0e700 1 -- 172.21.15.78:0/2669957856 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7effe8065d40 con 0x7effe815a940 2023-10-11T09:51:27.309 INFO:tasks.ceph.mon.a.smithi078.stderr:2023-10-11T09:51:27.300+0000 7f90131f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-11T09:51:27.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.316+0000 7effde7fc700 1 -- 172.21.15.78:0/2669957856 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v92) v1 ==== 161+0+0 (secure 0 0 0) 0x7effd8015090 con 0x7effe815a940 2023-10-11T09:51:27.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.320+0000 7effeef0e700 1 -- 172.21.15.78:0/2669957856 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7effcc065f00 msgr2=0x7effcc0683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:27.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.320+0000 7effeef0e700 1 --2- 172.21.15.78:0/2669957856 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7effcc065f00 0x7effcc0683b0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7effe4009b50 tx=0x7effe4009240 comp rx=0 tx=0).stop 2023-10-11T09:51:27.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.320+0000 7effeef0e700 1 -- 172.21.15.78:0/2669957856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effe815a940 msgr2=0x7effe813fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:27.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.320+0000 7effeef0e700 1 --2- 172.21.15.78:0/2669957856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effe815a940 0x7effe813fc00 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7effd8000c00 tx=0x7effd8004770 comp rx=0 tx=0).stop 2023-10-11T09:51:27.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.320+0000 7effeef0e700 1 -- 172.21.15.78:0/2669957856 shutdown_connections 2023-10-11T09:51:27.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.320+0000 7effeef0e700 1 --2- 172.21.15.78:0/2669957856 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7effcc065f00 0x7effcc0683b0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:27.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.320+0000 7effeef0e700 1 --2- 172.21.15.78:0/2669957856 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7effe815b8e0 0x7effe80c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:27.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.320+0000 7effeef0e700 1 --2- 172.21.15.78:0/2669957856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effe815a940 0x7effe813fc00 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:27.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.320+0000 7effeef0e700 1 --2- 172.21.15.78:0/2669957856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effe806d610 0x7effe80c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:27.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.320+0000 7effeef0e700 1 -- 172.21.15.78:0/2669957856 >> 172.21.15.78:0/2669957856 conn(0x7effe80cea60 msgr2=0x7effe8143640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:27.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.320+0000 7effeef0e700 1 -- 172.21.15.78:0/2669957856 shutdown_connections 2023-10-11T09:51:27.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.320+0000 7effeef0e700 1 -- 172.21.15.78:0/2669957856 wait complete. 2023-10-11T09:51:27.333 INFO:teuthology.orchestra.run.smithi078.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-11T09:51:27.337 INFO:tasks.ceph.mds.c.smithi078.stderr: -6387> 2023-10-11T09:50:34.661+0000 7f608874c700 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-10-11T09:51:27.341 INFO:tasks.ceph.mds.c.smithi078.stderr: -4946> 2023-10-11T09:50:39.641+0000 7f608bf53700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-11T09:51:27.342 INFO:tasks.ceph.mds.c.smithi078.stderr: -4545> 2023-10-11T09:50:42.037+0000 7f608874c700 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2023-10-11T09:51:27.342 INFO:tasks.ceph.mds.c.smithi078.stderr: -4508> 2023-10-11T09:50:42.041+0000 7f608874c700 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x4 2023-10-11T09:51:27.343 INFO:tasks.ceph.mds.c.smithi078.stderr: -4361> 2023-10-11T09:50:42.041+0000 7f608874c700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /stripey/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55e666158100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000000:[<0x1/stripey v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-10-11T09:51:27.343 INFO:tasks.ceph.mds.c.smithi078.stderr: -4229> 2023-10-11T09:50:42.041+0000 7f608874c700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x4 [...2,head] /lost+found/ auth v1 pv6 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55e666158680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x4:[<0x1/lost+found v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-10-11T09:51:27.343 INFO:tasks.ceph.mds.c.smithi078.stderr: -4040> 2023-10-11T09:50:42.045+0000 7f608874c700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fb [2,head] /stripey/flushed_file_6 auth v1 ap=1 DIRTYPARENT s=1572864 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e6661c5180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(16)0x100000001fb:[<0x10000000000/flushed_file_6 v68>,<0x1/stripey v73>]//[15]","memoryvalue":"(16)0x100000001fb:[<0x10000000000/flushed_file_6 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:51:27.344 INFO:tasks.ceph.mds.c.smithi078.stderr: -4015> 2023-10-11T09:50:42.045+0000 7f608874c700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f9 [2,head] /stripey/flushed_file_4 auth v1 ap=1 DIRTYPARENT s=1441792 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e666215600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(16)0x100000001f9:[<0x10000000000/flushed_file_4 v42>,<0x1/stripey v73>]//[15]","memoryvalue":"(16)0x100000001f9:[<0x10000000000/flushed_file_4 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:51:27.345 INFO:tasks.ceph.mds.c.smithi078.stderr: -3991> 2023-10-11T09:50:42.045+0000 7f608874c700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f8 [2,head] /stripey/flushed_file_3 auth v1 ap=1 DIRTYPARENT s=917503 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e666215080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(16)0x100000001f8:[<0x10000000000/flushed_file_3 v64>,<0x1/stripey v73>]//[15]","memoryvalue":"(16)0x100000001f8:[<0x10000000000/flushed_file_3 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:51:27.345 INFO:tasks.ceph.mds.c.smithi078.stderr: -3968> 2023-10-11T09:50:42.045+0000 7f608874c700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f7 [2,head] /stripey/flushed_file_2 auth v1 ap=1 DIRTYPARENT s=917504 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e666214000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(16)0x100000001f7:[<0x10000000000/flushed_file_2 v56>,<0x1/stripey v73>]//[15]","memoryvalue":"(16)0x100000001f7:[<0x10000000000/flushed_file_2 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:51:27.345 INFO:tasks.ceph.mds.c.smithi078.stderr: -3945> 2023-10-11T09:50:42.045+0000 7f608874c700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f6 [2,head] /stripey/flushed_file_1 auth v1 ap=1 DIRTYPARENT s=524288 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e666215b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(16)0x100000001f6:[<0x10000000000/flushed_file_1 v50>,<0x1/stripey v73>]//[15]","memoryvalue":"(16)0x100000001f6:[<0x10000000000/flushed_file_1 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:51:27.345 INFO:tasks.ceph.mds.c.smithi078.stderr: -3870> 2023-10-11T09:50:42.045+0000 7f608874c700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /stripey/flushed_file_0 auth v1 ap=1 DIRTYPARENT s=1048576 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e6652e1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(16)0x10000000001:[<0x10000000000/flushed_file_0 v36>,<0x1/stripey v73>]//[15]","memoryvalue":"(16)0x10000000001:[<0x10000000000/flushed_file_0 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:51:27.346 INFO:tasks.ceph.mds.c.smithi078.stderr: -3727> 2023-10-11T09:50:42.045+0000 7f608874c700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fa [2,head] /stripey/flushed_file_5 auth v1 ap=1 DIRTYPARENT s=1441793 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e6661c5700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(16)0x100000001fa:[<0x10000000000/flushed_file_5 v54>,<0x1/stripey v73>]//[15]","memoryvalue":"(16)0x100000001fa:[<0x10000000000/flushed_file_5 v1>,<0x1/stripey v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-11T09:51:27.346 INFO:tasks.ceph.mds.c.smithi078.stderr: -3712> 2023-10-11T09:50:42.045+0000 7f608874c700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fc [2,head] /lost+found/100000001fc auth v1 ap=1 DIRTYPARENT s=524288 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e6652e1b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(16)0x100000001fc:[<0x4/100000001fc v1>,<0x1/lost+found v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2023-10-11T09:51:27.346 INFO:tasks.ceph.mds.c.smithi078.stderr: -3409> 2023-10-11T09:50:42.053+0000 7f608874c700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v7 ap=5 snaprealm=0x55e666142fc0 DIRTYPARENT f(v1 2=0+2) n(v0 rc2023-10-11T09:50:41.751037+0000 2=0+2)/n() (idft sync r=1) (inest sync r=1) (ifile sync->mix r=1) (iversion lock) caps={8607=pLsXs/-@3} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x55e666157600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 2=0+2)","ondisk_value.rstat":"n(v0 3=0+3)","memory_value.dirstat":"f(v1 2=0+2)","memory_value.rstat":"n(v0 rc2023-10-11T09:50:41.751037+0000 2=0+2)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-10-11T09:51:27.352 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-11T09:51:27.405 INFO:tasks.ceph.mds.c.smithi078.stdout:starting mds.c at 2023-10-11T09:51:27.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.512+0000 7f3a9164a700 1 Processor -- start 2023-10-11T09:51:27.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.516+0000 7f3a9164a700 1 -- start start 2023-10-11T09:51:27.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.516+0000 7f3a9164a700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a8c158430 0x7f3a8c0710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:27.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.516+0000 7f3a9164a700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f3a8c0caf90 0x7f3a8c15afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:27.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.516+0000 7f3a9164a700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a8c071610 con 0x7f3a8c06d340 2023-10-11T09:51:27.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.516+0000 7f3a9164a700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a8c070c10 con 0x7f3a8c0caf90 2023-10-11T09:51:27.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.516+0000 7f3a9164a700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c15e280 con 0x7f3a8c158430 2023-10-11T09:51:27.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.516+0000 7f3a8b7fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f3a8c0caf90 0x7f3a8c15afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:27.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.516+0000 7f3a8b7fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f3a8c0caf90 0x7f3a8c15afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34572/0 (socket says 172.21.15.78:34572) 2023-10-11T09:51:27.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.516+0000 7f3a8b7fe700 1 -- 172.21.15.78:0/1541313824 learned_addr learned my addr 172.21.15.78:0/1541313824 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:27.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.516+0000 7f3a8affd700 1 --2- 172.21.15.78:0/1541313824 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a8c158430 0x7f3a8c0710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:27.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.516+0000 7f3a8b7fe700 1 -- 172.21.15.78:0/1541313824 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a8c158430 msgr2=0x7f3a8c0710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:27.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.516+0000 7f3a8b7fe700 1 --2- 172.21.15.78:0/1541313824 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a8c158430 0x7f3a8c0710d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:27.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.516+0000 7f3a8b7fe700 1 -- 172.21.15.78:0/1541313824 >> v1:172.21.15.78:6789/0 conn(0x7f3a8c06d340 legacy=0x7f3a8c0ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:27.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.516+0000 7f3a8b7fe700 1 -- 172.21.15.78:0/1541313824 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a8c15e560 con 0x7f3a8c0caf90 2023-10-11T09:51:27.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.516+0000 7f3a8affd700 1 --2- 172.21.15.78:0/1541313824 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a8c158430 0x7f3a8c0710d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:51:27.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.520+0000 7f3a8b7fe700 1 --2- 172.21.15.78:0/1541313824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a8c0caf90 0x7f3a8c15afe0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009e50 tx=0x7f3a8400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c4e6ab0af5e224 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:27.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.520+0000 7f3a8a7fc700 1 -- 172.21.15.78:0/1541313824 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8400e070 con 0x7f3a8c0caf90 2023-10-11T09:51:27.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.520+0000 7f3a8a7fc700 1 -- 172.21.15.78:0/1541313824 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a8400b9c0 con 0x7f3a8c0caf90 2023-10-11T09:51:27.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.520+0000 7f3a8a7fc700 1 -- 172.21.15.78:0/1541313824 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8400bd70 con 0x7f3a8c0caf90 2023-10-11T09:51:27.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.520+0000 7f3a9164a700 1 -- 172.21.15.78:0/1541313824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a8c0caf90 msgr2=0x7f3a8c15afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:27.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.520+0000 7f3a9164a700 1 --2- 172.21.15.78:0/1541313824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a8c0caf90 0x7f3a8c15afe0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009e50 tx=0x7f3a8400b330 comp rx=0 tx=0).stop 2023-10-11T09:51:27.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.520+0000 7f3a9164a700 1 -- 172.21.15.78:0/1541313824 shutdown_connections 2023-10-11T09:51:27.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.520+0000 7f3a9164a700 1 --2- 172.21.15.78:0/1541313824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a8c0caf90 0x7f3a8c15afe0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:27.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.520+0000 7f3a9164a700 1 --2- 172.21.15.78:0/1541313824 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a8c158430 0x7f3a8c0710d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:27.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.520+0000 7f3a9164a700 1 -- 172.21.15.78:0/1541313824 >> 172.21.15.78:0/1541313824 conn(0x7f3a8c0cecc0 msgr2=0x7f3a8c0cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:27.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.520+0000 7f3a9164a700 1 -- 172.21.15.78:0/1541313824 shutdown_connections 2023-10-11T09:51:27.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.520+0000 7f3a9164a700 1 -- 172.21.15.78:0/1541313824 wait complete. 2023-10-11T09:51:27.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.524+0000 7f3a9164a700 1 Processor -- start 2023-10-11T09:51:27.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.524+0000 7f3a9164a700 1 -- start start 2023-10-11T09:51:27.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.524+0000 7f3a9164a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a8c06d340 0x7f3a8c0b0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:27.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.524+0000 7f3a9164a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a8c0caf90 0x7f3a8c14a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:27.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.524+0000 7f3a9164a700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a8c158430 0x7f3a8c0ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:27.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.524+0000 7f3a9164a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c14e590 con 0x7f3a8c0caf90 2023-10-11T09:51:27.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.524+0000 7f3a9164a700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c0b1050 con 0x7f3a8c06d340 2023-10-11T09:51:27.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.524+0000 7f3a9164a700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c0b0a90 con 0x7f3a8c158430 2023-10-11T09:51:27.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.528+0000 7f3a83fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a8c0caf90 0x7f3a8c14a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:27.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.528+0000 7f3a83fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a8c0caf90 0x7f3a8c14a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34580/0 (socket says 172.21.15.78:34580) 2023-10-11T09:51:27.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.528+0000 7f3a83fff700 1 -- 172.21.15.78:0/591675110 learned_addr learned my addr 172.21.15.78:0/591675110 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:27.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.528+0000 7f3a8b7fe700 1 --2- 172.21.15.78:0/591675110 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a8c158430 0x7f3a8c0ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:27.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.528+0000 7f3a8affd700 1 --2- 172.21.15.78:0/591675110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a8c06d340 0x7f3a8c0b0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:27.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.528+0000 7f3a83fff700 1 -- 172.21.15.78:0/591675110 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a8c158430 msgr2=0x7f3a8c0ae900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:27.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.528+0000 7f3a83fff700 1 --2- 172.21.15.78:0/591675110 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a8c158430 0x7f3a8c0ae900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:27.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.528+0000 7f3a83fff700 1 -- 172.21.15.78:0/591675110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a8c06d340 msgr2=0x7f3a8c0b0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:27.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.528+0000 7f3a83fff700 1 --2- 172.21.15.78:0/591675110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a8c06d340 0x7f3a8c0b0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:27.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.528+0000 7f3a83fff700 1 -- 172.21.15.78:0/591675110 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a84009b30 con 0x7f3a8c0caf90 2023-10-11T09:51:27.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.528+0000 7f3a83fff700 1 --2- 172.21.15.78:0/591675110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a8c0caf90 0x7f3a8c14a120 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f3a7400c500 tx=0x7f3a7400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:27.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.528+0000 7f3a837fe700 1 -- 172.21.15.78:0/591675110 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a740052d0 con 0x7f3a8c0caf90 2023-10-11T09:51:27.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.528+0000 7f3a837fe700 1 -- 172.21.15.78:0/591675110 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a74005430 con 0x7f3a8c0caf90 2023-10-11T09:51:27.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.528+0000 7f3a837fe700 1 -- 172.21.15.78:0/591675110 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a740055a0 con 0x7f3a8c0caf90 2023-10-11T09:51:27.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.528+0000 7f3a9164a700 1 -- 172.21.15.78:0/591675110 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a8c0b0d70 con 0x7f3a8c0caf90 2023-10-11T09:51:27.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.528+0000 7f3a9164a700 1 -- 172.21.15.78:0/591675110 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a8c0aee40 con 0x7f3a8c0caf90 2023-10-11T09:51:27.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.528+0000 7f3a9164a700 1 -- 172.21.15.78:0/591675110 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a8c14e6d0 con 0x7f3a8c0caf90 2023-10-11T09:51:27.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.532+0000 7f3a837fe700 1 -- 172.21.15.78:0/591675110 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f3a740038e0 con 0x7f3a8c0caf90 2023-10-11T09:51:27.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.532+0000 7f3a837fe700 1 --2- 172.21.15.78:0/591675110 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3a6c065ed0 0x7f3a6c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:27.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.532+0000 7f3a837fe700 1 -- 172.21.15.78:0/591675110 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6810+0+0 (secure 0 0 0) 0x7f3a74082f00 con 0x7f3a8c0caf90 2023-10-11T09:51:27.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.532+0000 7f3a8affd700 1 --2- 172.21.15.78:0/591675110 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3a6c065ed0 0x7f3a6c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:27.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.536+0000 7f3a8affd700 1 --2- 172.21.15.78:0/591675110 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3a6c065ed0 0x7f3a6c068380 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c00c960 tx=0x7f3a7c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:27.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.540+0000 7f3a837fe700 1 -- 172.21.15.78:0/591675110 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f3a8c14e6d0 con 0x7f3a8c0caf90 2023-10-11T09:51:27.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:27.760+0000 7f3a9164a700 1 -- 172.21.15.78:0/591675110 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3a8c14e6d0 con 0x7f3a8c0caf90 2023-10-11T09:51:28.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.324+0000 7f3a837fe700 1 -- 172.21.15.78:0/591675110 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v93) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3a8c14e6d0 con 0x7f3a8c0caf90 2023-10-11T09:51:28.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.328+0000 7f3a9164a700 1 -- 172.21.15.78:0/591675110 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3a6c065ed0 msgr2=0x7f3a6c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:28.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.328+0000 7f3a9164a700 1 --2- 172.21.15.78:0/591675110 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3a6c065ed0 0x7f3a6c068380 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c00c960 tx=0x7f3a7c008040 comp rx=0 tx=0).stop 2023-10-11T09:51:28.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.328+0000 7f3a9164a700 1 -- 172.21.15.78:0/591675110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a8c0caf90 msgr2=0x7f3a8c14a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:28.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.328+0000 7f3a9164a700 1 --2- 172.21.15.78:0/591675110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a8c0caf90 0x7f3a8c14a120 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f3a7400c500 tx=0x7f3a7400c810 comp rx=0 tx=0).stop 2023-10-11T09:51:28.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.328+0000 7f3a9164a700 1 -- 172.21.15.78:0/591675110 shutdown_connections 2023-10-11T09:51:28.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.328+0000 7f3a9164a700 1 --2- 172.21.15.78:0/591675110 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3a6c065ed0 0x7f3a6c068380 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:28.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.328+0000 7f3a9164a700 1 --2- 172.21.15.78:0/591675110 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a8c158430 0x7f3a8c0ae900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:28.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.328+0000 7f3a9164a700 1 --2- 172.21.15.78:0/591675110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a8c0caf90 0x7f3a8c14a120 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:28.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.328+0000 7f3a9164a700 1 --2- 172.21.15.78:0/591675110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a8c06d340 0x7f3a8c0b0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:28.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.328+0000 7f3a9164a700 1 -- 172.21.15.78:0/591675110 >> 172.21.15.78:0/591675110 conn(0x7f3a8c0cecc0 msgr2=0x7f3a8c150560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:28.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.328+0000 7f3a9164a700 1 -- 172.21.15.78:0/591675110 shutdown_connections 2023-10-11T09:51:28.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.328+0000 7f3a9164a700 1 -- 172.21.15.78:0/591675110 wait complete. 2023-10-11T09:51:28.362 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-11T09:51:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.532+0000 7ff041498700 1 Processor -- start 2023-10-11T09:51:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.536+0000 7ff041498700 1 -- start start 2023-10-11T09:51:28.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.536+0000 7ff041498700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff03c0cefc0 0x7ff03c0b52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:28.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.536+0000 7ff041498700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff03c0d2bd0 con 0x7ff03c15cbf0 2023-10-11T09:51:28.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.536+0000 7ff041498700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff03c0cf630 con 0x7ff03c15a0f0 2023-10-11T09:51:28.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.536+0000 7ff041498700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff03c0cf430 con 0x7ff03c0cefc0 2023-10-11T09:51:28.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.536+0000 7ff03bfff700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff03c15a0f0 0x7ff03c1608a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:57300/0 (socket says 172.21.15.78:57300) 2023-10-11T09:51:28.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.536+0000 7ff03bfff700 1 -- 172.21.15.78:0/2196504193 learned_addr learned my addr 172.21.15.78:0/2196504193 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:51:28.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.536+0000 7ff0337fe700 1 --2- 172.21.15.78:0/2196504193 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff03c0cefc0 0x7ff03c0b52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:28.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.540+0000 7ff03b7fe700 1 -- 172.21.15.78:0/2196504193 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3655616707 0 0) 0x7ff03c0d2bd0 con 0x7ff03c15cbf0 2023-10-11T09:51:28.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.540+0000 7ff03b7fe700 1 -- 172.21.15.78:0/2196504193 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0280035d0 con 0x7ff03c15cbf0 2023-10-11T09:51:28.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.540+0000 7ff03b7fe700 1 -- 172.21.15.78:0/2196504193 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1218386373 0 0) 0x7ff03c0cf630 con 0x7ff03c15a0f0 2023-10-11T09:51:28.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.540+0000 7ff03b7fe700 1 -- 172.21.15.78:0/2196504193 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff03c0d2bd0 con 0x7ff03c15a0f0 2023-10-11T09:51:28.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.540+0000 7ff03b7fe700 1 -- 172.21.15.78:0/2196504193 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3486077491 0 0) 0x7ff0280035d0 con 0x7ff03c15cbf0 2023-10-11T09:51:28.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.540+0000 7ff03b7fe700 1 -- 172.21.15.78:0/2196504193 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff03c0cefc0 msgr2=0x7ff03c0b52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:28.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.540+0000 7ff03b7fe700 1 --2- 172.21.15.78:0/2196504193 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff03c0cefc0 0x7ff03c0b52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:28.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.540+0000 7ff03b7fe700 1 -- 172.21.15.78:0/2196504193 >> v1:172.21.15.133:6789/0 conn(0x7ff03c15a0f0 legacy=0x7ff03c1608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:28.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.540+0000 7ff03b7fe700 1 -- 172.21.15.78:0/2196504193 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff03c160410 con 0x7ff03c15cbf0 2023-10-11T09:51:28.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.540+0000 7ff03b7fe700 1 -- 172.21.15.78:0/2196504193 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7ff034001670 con 0x7ff03c15cbf0 2023-10-11T09:51:28.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.540+0000 7ff03b7fe700 1 -- 172.21.15.78:0/2196504193 >> v1:172.21.15.78:6789/0 conn(0x7ff03c15cbf0 legacy=0x7ff03c0cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:28.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.540+0000 7ff03b7fe700 1 --2- 172.21.15.78:0/2196504193 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff028004c40 0x7ff028005030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:28.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.540+0000 7ff03b7fe700 1 --2- 172.21.15.78:0/2196504193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff028003810 0x7ff028003c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:28.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.540+0000 7ff03b7fe700 1 --2- 172.21.15.78:0/2196504193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0280041c0 0x7ff02800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:28.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.540+0000 7ff03b7fe700 1 -- 172.21.15.78:0/2196504193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03c0cf430 con 0x7ff028003810 2023-10-11T09:51:28.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.540+0000 7ff03b7fe700 1 -- 172.21.15.78:0/2196504193 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02800c050 con 0x7ff0280041c0 2023-10-11T09:51:28.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.540+0000 7ff03b7fe700 1 -- 172.21.15.78:0/2196504193 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff02800c210 con 0x7ff028004c40 2023-10-11T09:51:28.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.540+0000 7ff03bfff700 1 --2- 172.21.15.78:0/2196504193 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff028004c40 0x7ff028005030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:28.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.540+0000 7ff0337fe700 1 --2- 172.21.15.78:0/2196504193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff028003810 0x7ff028003c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:28.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.540+0000 7ff040c97700 1 --2- 172.21.15.78:0/2196504193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0280041c0 0x7ff02800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:28.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.540+0000 7ff03bfff700 1 -- 172.21.15.78:0/2196504193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0280041c0 msgr2=0x7ff02800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:28.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.540+0000 7ff03bfff700 1 --2- 172.21.15.78:0/2196504193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0280041c0 0x7ff02800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:28.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.540+0000 7ff03bfff700 1 -- 172.21.15.78:0/2196504193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff028003810 msgr2=0x7ff028003c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:28.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.540+0000 7ff03bfff700 1 --2- 172.21.15.78:0/2196504193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff028003810 0x7ff028003c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:28.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.540+0000 7ff03bfff700 1 -- 172.21.15.78:0/2196504193 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0280035d0 con 0x7ff028004c40 2023-10-11T09:51:28.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.540+0000 7ff03bfff700 1 --2- 172.21.15.78:0/2196504193 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff028004c40 0x7ff028005030 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff02c00c320 tx=0x7ff0280058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c583a059f31dc31b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:28.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.540+0000 7ff03b7fe700 1 -- 172.21.15.78:0/2196504193 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c001f00 con 0x7ff028004c40 2023-10-11T09:51:28.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.540+0000 7ff03b7fe700 1 -- 172.21.15.78:0/2196504193 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff02c00cb80 con 0x7ff028004c40 2023-10-11T09:51:28.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.540+0000 7ff03b7fe700 1 -- 172.21.15.78:0/2196504193 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c012400 con 0x7ff028004c40 2023-10-11T09:51:28.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.540+0000 7ff041498700 1 -- 172.21.15.78:0/2196504193 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff028004c40 msgr2=0x7ff028005030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:28.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.540+0000 7ff041498700 1 --2- 172.21.15.78:0/2196504193 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff028004c40 0x7ff028005030 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff02c00c320 tx=0x7ff0280058a0 comp rx=0 tx=0).stop 2023-10-11T09:51:28.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.540+0000 7ff040c97700 1 -- 172.21.15.78:0/2196504193 reap_dead start 2023-10-11T09:51:28.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.540+0000 7ff041498700 1 -- 172.21.15.78:0/2196504193 shutdown_connections 2023-10-11T09:51:28.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.540+0000 7ff041498700 1 -- 172.21.15.78:0/2196504193 >> 172.21.15.78:0/2196504193 conn(0x7ff03c06afd0 msgr2=0x7ff03c06b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:28.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.544+0000 7ff041498700 1 -- 172.21.15.78:0/2196504193 shutdown_connections 2023-10-11T09:51:28.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.544+0000 7ff041498700 1 -- 172.21.15.78:0/2196504193 wait complete. 2023-10-11T09:51:28.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.544+0000 7ff041498700 1 Processor -- start 2023-10-11T09:51:28.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.544+0000 7ff041498700 1 -- start start 2023-10-11T09:51:28.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.548+0000 7ff041498700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff028004c40 0x7ff03c0c69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:28.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.548+0000 7ff041498700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff03c0cc680 0x7ff03c151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:28.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.548+0000 7ff041498700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff03c0ccaf0 0x7ff03c152420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:28.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.548+0000 7ff041498700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03c15d9f0 con 0x7ff03c0ccaf0 2023-10-11T09:51:28.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.548+0000 7ff041498700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03c15e810 con 0x7ff028004c40 2023-10-11T09:51:28.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.548+0000 7ff041498700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff03c15dc00 con 0x7ff03c0cc680 2023-10-11T09:51:28.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.548+0000 7ff040c97700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff03c0ccaf0 0x7ff03c152420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:28.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.548+0000 7ff03bfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff028004c40 0x7ff03c0c69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:28.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.548+0000 7ff040c97700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff03c0ccaf0 0x7ff03c152420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34600/0 (socket says 172.21.15.78:34600) 2023-10-11T09:51:28.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.548+0000 7ff040c97700 1 -- 172.21.15.78:0/1977562967 learned_addr learned my addr 172.21.15.78:0/1977562967 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:28.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.548+0000 7ff0337fe700 1 --2- 172.21.15.78:0/1977562967 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff03c0cc680 0x7ff03c151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:28.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.548+0000 7ff040c97700 1 -- 172.21.15.78:0/1977562967 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff03c0cc680 msgr2=0x7ff03c151ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:28.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.548+0000 7ff040c97700 1 --2- 172.21.15.78:0/1977562967 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff03c0cc680 0x7ff03c151ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:28.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.548+0000 7ff040c97700 1 -- 172.21.15.78:0/1977562967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff028004c40 msgr2=0x7ff03c0c69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:28.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.548+0000 7ff040c97700 1 --2- 172.21.15.78:0/1977562967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff028004c40 0x7ff03c0c69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:28.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.548+0000 7ff040c97700 1 -- 172.21.15.78:0/1977562967 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff03c15dee0 con 0x7ff03c0ccaf0 2023-10-11T09:51:28.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.548+0000 7ff0337fe700 1 --2- 172.21.15.78:0/1977562967 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff03c0cc680 0x7ff03c151ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:51:28.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.548+0000 7ff040c97700 1 --2- 172.21.15.78:0/1977562967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff03c0ccaf0 0x7ff03c152420 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7ff03c06cb60 tx=0x7ff03400fa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:28.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.548+0000 7ff0397fa700 1 -- 172.21.15.78:0/1977562967 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0340052e0 con 0x7ff03c0ccaf0 2023-10-11T09:51:28.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.548+0000 7ff0397fa700 1 -- 172.21.15.78:0/1977562967 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff034005460 con 0x7ff03c0ccaf0 2023-10-11T09:51:28.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.548+0000 7ff041498700 1 -- 172.21.15.78:0/1977562967 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff03c15e1c0 con 0x7ff03c0ccaf0 2023-10-11T09:51:28.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.548+0000 7ff0397fa700 1 -- 172.21.15.78:0/1977562967 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0340038e0 con 0x7ff03c0ccaf0 2023-10-11T09:51:28.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.548+0000 7ff041498700 1 -- 172.21.15.78:0/1977562967 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff03c0c6ef0 con 0x7ff03c0ccaf0 2023-10-11T09:51:28.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.552+0000 7ff041498700 1 -- 172.21.15.78:0/1977562967 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff03c0c70b0 con 0x7ff03c0ccaf0 2023-10-11T09:51:28.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.552+0000 7ff0397fa700 1 -- 172.21.15.78:0/1977562967 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7ff034003a40 con 0x7ff03c0ccaf0 2023-10-11T09:51:28.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.552+0000 7ff0397fa700 1 --2- 172.21.15.78:0/1977562967 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff01c065ed0 0x7ff01c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:28.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.552+0000 7ff0397fa700 1 -- 172.21.15.78:0/1977562967 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6810+0+0 (secure 0 0 0) 0x7ff034083860 con 0x7ff03c0ccaf0 2023-10-11T09:51:28.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.552+0000 7ff03bfff700 1 --2- 172.21.15.78:0/1977562967 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff01c065ed0 0x7ff01c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:28.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.556+0000 7ff03bfff700 1 --2- 172.21.15.78:0/1977562967 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff01c065ed0 0x7ff01c068380 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff02c015040 tx=0x7ff02c007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:28.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.560+0000 7ff0397fa700 1 -- 172.21.15.78:0/1977562967 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7ff034050b20 con 0x7ff03c0ccaf0 2023-10-11T09:51:28.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:28.736+0000 7ff041498700 1 -- 172.21.15.78:0/1977562967 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff03c08bb90 con 0x7ff03c0ccaf0 2023-10-11T09:51:29.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.652+0000 7ff0397fa700 1 -- 172.21.15.78:0/1977562967 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v116) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff034054140 con 0x7ff03c0ccaf0 2023-10-11T09:51:29.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.660+0000 7ff041498700 1 -- 172.21.15.78:0/1977562967 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff01c065ed0 msgr2=0x7ff01c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:29.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.660+0000 7ff041498700 1 --2- 172.21.15.78:0/1977562967 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff01c065ed0 0x7ff01c068380 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff02c015040 tx=0x7ff02c007000 comp rx=0 tx=0).stop 2023-10-11T09:51:29.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.660+0000 7ff041498700 1 -- 172.21.15.78:0/1977562967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff03c0ccaf0 msgr2=0x7ff03c152420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:29.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.660+0000 7ff041498700 1 --2- 172.21.15.78:0/1977562967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff03c0ccaf0 0x7ff03c152420 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7ff03c06cb60 tx=0x7ff03400fa50 comp rx=0 tx=0).stop 2023-10-11T09:51:29.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.660+0000 7ff041498700 1 -- 172.21.15.78:0/1977562967 shutdown_connections 2023-10-11T09:51:29.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.660+0000 7ff041498700 1 --2- 172.21.15.78:0/1977562967 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff01c065ed0 0x7ff01c068380 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:29.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.660+0000 7ff041498700 1 --2- 172.21.15.78:0/1977562967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff03c0ccaf0 0x7ff03c152420 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:29.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.660+0000 7ff041498700 1 --2- 172.21.15.78:0/1977562967 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff03c0cc680 0x7ff03c151ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:29.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.660+0000 7ff041498700 1 --2- 172.21.15.78:0/1977562967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff028004c40 0x7ff03c0c69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:29.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.660+0000 7ff041498700 1 -- 172.21.15.78:0/1977562967 >> 172.21.15.78:0/1977562967 conn(0x7ff03c06afd0 msgr2=0x7ff03c1e70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:29.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.660+0000 7ff041498700 1 -- 172.21.15.78:0/1977562967 shutdown_connections 2023-10-11T09:51:29.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.660+0000 7ff041498700 1 -- 172.21.15.78:0/1977562967 wait complete. 2023-10-11T09:51:29.672 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' removed 2023-10-11T09:51:29.692 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-11T09:51:29.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.868+0000 7f7de359e700 1 Processor -- start 2023-10-11T09:51:29.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.868+0000 7f7de359e700 1 -- start start 2023-10-11T09:51:29.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.872+0000 7f7de359e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7de40cb200 0x7f7de40cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:29.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.872+0000 7f7de359e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f7de415cc30 0x7f7de40addc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:29.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.872+0000 7f7de359e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7de4160830 con 0x7f7de406d160 2023-10-11T09:51:29.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.872+0000 7f7de359e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7de4160500 con 0x7f7de415cc30 2023-10-11T09:51:29.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.872+0000 7f7de359e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7de40cbb10 con 0x7f7de40cb200 2023-10-11T09:51:29.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.872+0000 7f7de1d9b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f7de415cc30 0x7f7de40addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:29.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.872+0000 7f7de1d9b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f7de415cc30 0x7f7de40addc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34608/0 (socket says 172.21.15.78:34608) 2023-10-11T09:51:29.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.872+0000 7f7de1d9b700 1 -- 172.21.15.78:0/3043077412 learned_addr learned my addr 172.21.15.78:0/3043077412 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:29.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.872+0000 7f7de259c700 1 --2- 172.21.15.78:0/3043077412 >> v2:172.21.15.133:3300/0 conn(0x7f7de40cb200 0x7f7de40cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:29.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.872+0000 7f7de1d9b700 1 -- 172.21.15.78:0/3043077412 >> v2:172.21.15.133:3300/0 conn(0x7f7de40cb200 msgr2=0x7f7de40cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:29.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.872+0000 7f7de1d9b700 1 --2- 172.21.15.78:0/3043077412 >> v2:172.21.15.133:3300/0 conn(0x7f7de40cb200 0x7f7de40cb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:29.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.872+0000 7f7de1d9b700 1 -- 172.21.15.78:0/3043077412 >> v1:172.21.15.133:6789/0 conn(0x7f7de406d160 legacy=0x7f7de4160cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:29.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.872+0000 7f7de1d9b700 1 -- 172.21.15.78:0/3043077412 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de41613d0 con 0x7f7de415cc30 2023-10-11T09:51:29.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.872+0000 7f7de1d9b700 1 --2- 172.21.15.78:0/3043077412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7de415cc30 0x7f7de40addc0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f7de406fa30 tx=0x7f7dd8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d87e639b4184cc02 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:29.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.872+0000 7f7de159a700 1 -- 172.21.15.78:0/3043077412 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd800b8a0 con 0x7f7de415cc30 2023-10-11T09:51:29.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.872+0000 7f7de159a700 1 -- 172.21.15.78:0/3043077412 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd800ba00 con 0x7f7de415cc30 2023-10-11T09:51:29.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.872+0000 7f7de159a700 1 -- 172.21.15.78:0/3043077412 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd800bd90 con 0x7f7de415cc30 2023-10-11T09:51:29.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.876+0000 7f7de359e700 1 -- 172.21.15.78:0/3043077412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7de415cc30 msgr2=0x7f7de40addc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:29.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.876+0000 7f7de359e700 1 --2- 172.21.15.78:0/3043077412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7de415cc30 0x7f7de40addc0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f7de406fa30 tx=0x7f7dd8009bf0 comp rx=0 tx=0).stop 2023-10-11T09:51:29.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.876+0000 7f7de359e700 1 -- 172.21.15.78:0/3043077412 shutdown_connections 2023-10-11T09:51:29.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.876+0000 7f7de359e700 1 --2- 172.21.15.78:0/3043077412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7de415cc30 0x7f7de40addc0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:29.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.876+0000 7f7de359e700 1 --2- 172.21.15.78:0/3043077412 >> v2:172.21.15.133:3300/0 conn(0x7f7de40cb200 0x7f7de40cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:29.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.876+0000 7f7de359e700 1 -- 172.21.15.78:0/3043077412 >> 172.21.15.78:0/3043077412 conn(0x7f7de406acf0 msgr2=0x7f7de40b0710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:29.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.876+0000 7f7de359e700 1 -- 172.21.15.78:0/3043077412 shutdown_connections 2023-10-11T09:51:29.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.876+0000 7f7de359e700 1 -- 172.21.15.78:0/3043077412 wait complete. 2023-10-11T09:51:29.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.876+0000 7f7de359e700 1 Processor -- start 2023-10-11T09:51:29.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.876+0000 7f7de359e700 1 -- start start 2023-10-11T09:51:29.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.880+0000 7f7de359e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7de406d160 0x7f7de415d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:29.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.880+0000 7f7de359e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7de40cb200 0x7f7de4156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:29.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.880+0000 7f7de359e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7de415cc30 0x7f7de40c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:29.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.880+0000 7f7de359e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de4156980 con 0x7f7de415cc30 2023-10-11T09:51:29.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.880+0000 7f7de359e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de40c8150 con 0x7f7de406d160 2023-10-11T09:51:29.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.880+0000 7f7de359e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7de415d700 con 0x7f7de40cb200 2023-10-11T09:51:29.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.880+0000 7f7de2d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7de415cc30 0x7f7de40c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:29.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.880+0000 7f7de2d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7de415cc30 0x7f7de40c7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34610/0 (socket says 172.21.15.78:34610) 2023-10-11T09:51:29.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.880+0000 7f7de2d9d700 1 -- 172.21.15.78:0/2200329016 learned_addr learned my addr 172.21.15.78:0/2200329016 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:29.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.880+0000 7f7de259c700 1 --2- 172.21.15.78:0/2200329016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7de406d160 0x7f7de415d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:29.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.880+0000 7f7de1d9b700 1 --2- 172.21.15.78:0/2200329016 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7de40cb200 0x7f7de4156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:29.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.880+0000 7f7de2d9d700 1 -- 172.21.15.78:0/2200329016 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7de40cb200 msgr2=0x7f7de4156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:29.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.880+0000 7f7de2d9d700 1 --2- 172.21.15.78:0/2200329016 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7de40cb200 0x7f7de4156440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:29.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.880+0000 7f7de2d9d700 1 -- 172.21.15.78:0/2200329016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7de406d160 msgr2=0x7f7de415d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:29.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.880+0000 7f7de2d9d700 1 --2- 172.21.15.78:0/2200329016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7de406d160 0x7f7de415d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:29.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.880+0000 7f7de2d9d700 1 -- 172.21.15.78:0/2200329016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd8009570 con 0x7f7de415cc30 2023-10-11T09:51:29.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.880+0000 7f7de1d9b700 1 --2- 172.21.15.78:0/2200329016 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7de40cb200 0x7f7de4156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:51:29.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.880+0000 7f7de2d9d700 1 --2- 172.21.15.78:0/2200329016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7de415cc30 0x7f7de40c7c10 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc00cb80 tx=0x7f7ddc00ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:29.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.880+0000 7f7dd2ffd700 1 -- 172.21.15.78:0/2200329016 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc003c20 con 0x7f7de415cc30 2023-10-11T09:51:29.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.880+0000 7f7dd2ffd700 1 -- 172.21.15.78:0/2200329016 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ddc003d80 con 0x7f7de415cc30 2023-10-11T09:51:29.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.880+0000 7f7de359e700 1 -- 172.21.15.78:0/2200329016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7de415d9e0 con 0x7f7de415cc30 2023-10-11T09:51:29.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.880+0000 7f7dd2ffd700 1 -- 172.21.15.78:0/2200329016 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc00ab00 con 0x7f7de415cc30 2023-10-11T09:51:29.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.880+0000 7f7de359e700 1 -- 172.21.15.78:0/2200329016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7de415dfa0 con 0x7f7de415cc30 2023-10-11T09:51:29.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.884+0000 7f7de359e700 1 -- 172.21.15.78:0/2200329016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7de406eca0 con 0x7f7de415cc30 2023-10-11T09:51:29.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.884+0000 7f7dd2ffd700 1 -- 172.21.15.78:0/2200329016 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f7ddc00ac60 con 0x7f7de415cc30 2023-10-11T09:51:29.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.884+0000 7f7dd2ffd700 1 --2- 172.21.15.78:0/2200329016 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7dc4065ed0 0x7f7dc4068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:29.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.888+0000 7f7dd2ffd700 1 -- 172.21.15.78:0/2200329016 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6372+0+0 (secure 0 0 0) 0x7f7ddc083910 con 0x7f7de415cc30 2023-10-11T09:51:29.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.888+0000 7f7de259c700 1 --2- 172.21.15.78:0/2200329016 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7dc4065ed0 0x7f7dc4068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:29.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.888+0000 7f7de259c700 1 --2- 172.21.15.78:0/2200329016 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7dc4065ed0 0x7f7dc4068380 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4006b40 tx=0x7f7dd4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:29.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:29.892+0000 7f7dd2ffd700 1 -- 172.21.15.78:0/2200329016 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f7ddc050e10 con 0x7f7de415cc30 2023-10-11T09:51:30.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.064+0000 7f7de359e700 1 -- 172.21.15.78:0/2200329016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7de4065db0 con 0x7f7de415cc30 2023-10-11T09:51:30.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.664+0000 7f7dd2ffd700 1 -- 172.21.15.78:0/2200329016 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v117) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7ddc054430 con 0x7f7de415cc30 2023-10-11T09:51:30.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.668+0000 7f7de359e700 1 -- 172.21.15.78:0/2200329016 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7dc4065ed0 msgr2=0x7f7dc4068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:30.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.668+0000 7f7de359e700 1 --2- 172.21.15.78:0/2200329016 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7dc4065ed0 0x7f7dc4068380 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4006b40 tx=0x7f7dd4008040 comp rx=0 tx=0).stop 2023-10-11T09:51:30.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.668+0000 7f7de359e700 1 -- 172.21.15.78:0/2200329016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7de415cc30 msgr2=0x7f7de40c7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:30.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.668+0000 7f7de359e700 1 --2- 172.21.15.78:0/2200329016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7de415cc30 0x7f7de40c7c10 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc00cb80 tx=0x7f7ddc00ce90 comp rx=0 tx=0).stop 2023-10-11T09:51:30.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.668+0000 7f7de359e700 1 -- 172.21.15.78:0/2200329016 shutdown_connections 2023-10-11T09:51:30.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.668+0000 7f7de359e700 1 --2- 172.21.15.78:0/2200329016 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7dc4065ed0 0x7f7dc4068380 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:30.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.668+0000 7f7de359e700 1 --2- 172.21.15.78:0/2200329016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7de415cc30 0x7f7de40c7c10 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:30.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.668+0000 7f7de359e700 1 --2- 172.21.15.78:0/2200329016 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7de40cb200 0x7f7de4156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:30.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.668+0000 7f7de359e700 1 --2- 172.21.15.78:0/2200329016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7de406d160 0x7f7de415d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:30.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.668+0000 7f7de359e700 1 -- 172.21.15.78:0/2200329016 >> 172.21.15.78:0/2200329016 conn(0x7f7de406acf0 msgr2=0x7f7de40ad3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:30.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.668+0000 7f7de359e700 1 -- 172.21.15.78:0/2200329016 shutdown_connections 2023-10-11T09:51:30.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.668+0000 7f7de359e700 1 -- 172.21.15.78:0/2200329016 wait complete. 2023-10-11T09:51:30.682 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' removed 2023-10-11T09:51:30.701 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_extra cephfs_data_extra --yes-i-really-really-mean-it 2023-10-11T09:51:30.840 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:51:30.847 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:51:30.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.872+0000 7f8458b8b700 1 Processor -- start 2023-10-11T09:51:30.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.872+0000 7f8458b8b700 1 -- start start 2023-10-11T09:51:30.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.872+0000 7f8458b8b700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f84540cfbe0 0x7f84540d0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:30.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.872+0000 7f8458b8b700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f845415df90 0x7f845415e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:30.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.872+0000 7f8458b8b700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84540cd720 con 0x7f84540b4ab0 2023-10-11T09:51:30.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.872+0000 7f8458b8b700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f845415e950 con 0x7f84540cfbe0 2023-10-11T09:51:30.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.872+0000 7f8458b8b700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84540cda00 con 0x7f845415df90 2023-10-11T09:51:30.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.872+0000 7f8452d9d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f845415df90 0x7f845415e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:30.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.872+0000 7f8451d9b700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f84540cfbe0 0x7f84540d0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:30.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.872+0000 7f845259c700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f84540b4ab0 0x7f8454160dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:57322/0 (socket says 172.21.15.78:57322) 2023-10-11T09:51:30.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.872+0000 7f8452d9d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f845415df90 0x7f845415e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:36508/0 (socket says 172.21.15.78:36508) 2023-10-11T09:51:30.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.872+0000 7f845259c700 1 -- 172.21.15.78:0/1337286995 learned_addr learned my addr 172.21.15.78:0/1337286995 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:51:30.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.876+0000 7f8452d9d700 1 -- 172.21.15.78:0/1337286995 >> v2:172.21.15.133:3300/0 conn(0x7f84540cfbe0 msgr2=0x7f84540d0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:30.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.876+0000 7f8452d9d700 1 --2- 172.21.15.78:0/1337286995 >> v2:172.21.15.133:3300/0 conn(0x7f84540cfbe0 0x7f84540d0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:30.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.876+0000 7f8452d9d700 1 -- 172.21.15.78:0/1337286995 >> v1:172.21.15.133:6789/0 conn(0x7f84540b4ab0 legacy=0x7f8454160dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:30.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.876+0000 7f8452d9d700 1 -- 172.21.15.78:0/1337286995 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84540cf3e0 con 0x7f845415df90 2023-10-11T09:51:30.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.876+0000 7f8452d9d700 1 --2- 172.21.15.78:0/1337286995 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f845415df90 0x7f845415e410 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f844c009d90 tx=0x7f844c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cc0ce5cf957aa37 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:30.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.876+0000 7f845159a700 1 -- 172.21.15.78:0/1337286995 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c00e070 con 0x7f845415df90 2023-10-11T09:51:30.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.876+0000 7f845159a700 1 -- 172.21.15.78:0/1337286995 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f844c00bb20 con 0x7f845415df90 2023-10-11T09:51:30.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.876+0000 7f845159a700 1 -- 172.21.15.78:0/1337286995 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c012430 con 0x7f845415df90 2023-10-11T09:51:30.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.876+0000 7f8458b8b700 1 -- 172.21.15.78:0/1337286995 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f845415df90 msgr2=0x7f845415e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:30.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.876+0000 7f8458b8b700 1 --2- 172.21.15.78:0/1337286995 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f845415df90 0x7f845415e410 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f844c009d90 tx=0x7f844c00b330 comp rx=0 tx=0).stop 2023-10-11T09:51:30.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.876+0000 7f8458b8b700 1 -- 172.21.15.78:0/1337286995 shutdown_connections 2023-10-11T09:51:30.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.876+0000 7f8458b8b700 1 --2- 172.21.15.78:0/1337286995 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f845415df90 0x7f845415e410 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:30.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.876+0000 7f8458b8b700 1 --2- 172.21.15.78:0/1337286995 >> v2:172.21.15.133:3300/0 conn(0x7f84540cfbe0 0x7f84540d0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:30.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.876+0000 7f8458b8b700 1 -- 172.21.15.78:0/1337286995 >> 172.21.15.78:0/1337286995 conn(0x7f845406b1d0 msgr2=0x7f845406b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:30.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.876+0000 7f8458b8b700 1 -- 172.21.15.78:0/1337286995 shutdown_connections 2023-10-11T09:51:30.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.876+0000 7f8458b8b700 1 -- 172.21.15.78:0/1337286995 wait complete. 2023-10-11T09:51:30.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.880+0000 7f8458b8b700 1 Processor -- start 2023-10-11T09:51:30.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.880+0000 7f8458b8b700 1 -- start start 2023-10-11T09:51:30.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.880+0000 7f8458b8b700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84540b4ab0 0x7f8454149650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:30.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.880+0000 7f8458b8b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84540cfbe0 0x7f84540b7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:30.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.880+0000 7f8458b8b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f845415df90 0x7f84540b64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:30.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.880+0000 7f8458b8b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84540d1b90 con 0x7f845415df90 2023-10-11T09:51:30.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.880+0000 7f8458b8b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84540d0f10 con 0x7f84540cfbe0 2023-10-11T09:51:30.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.880+0000 7f8458b8b700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84540d11f0 con 0x7f84540b4ab0 2023-10-11T09:51:30.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.880+0000 7f8452d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f845415df90 0x7f84540b64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:30.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.880+0000 7f8451d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84540cfbe0 0x7f84540b7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:30.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.880+0000 7f8451d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84540cfbe0 0x7f84540b7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:51200/0 (socket says 172.21.15.78:51200) 2023-10-11T09:51:30.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.880+0000 7f8451d9b700 1 -- 172.21.15.78:0/3491161981 learned_addr learned my addr 172.21.15.78:0/3491161981 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:30.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.880+0000 7f845259c700 1 --2- 172.21.15.78:0/3491161981 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84540b4ab0 0x7f8454149650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:30.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.884+0000 7f8451d9b700 1 -- 172.21.15.78:0/3491161981 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84540b4ab0 msgr2=0x7f8454149650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:30.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.884+0000 7f8451d9b700 1 --2- 172.21.15.78:0/3491161981 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84540b4ab0 0x7f8454149650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:30.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.884+0000 7f8451d9b700 1 -- 172.21.15.78:0/3491161981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f845415df90 msgr2=0x7f84540b64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:30.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.884+0000 7f8451d9b700 1 --2- 172.21.15.78:0/3491161981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f845415df90 0x7f84540b64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:30.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.884+0000 7f8451d9b700 1 -- 172.21.15.78:0/3491161981 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f844c009a70 con 0x7f84540cfbe0 2023-10-11T09:51:30.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.884+0000 7f8451d9b700 1 --2- 172.21.15.78:0/3491161981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84540cfbe0 0x7f84540b7860 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f844800e9d0 tx=0x7f844800ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:30.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.884+0000 7f8442ffd700 1 -- 172.21.15.78:0/3491161981 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8448009720 con 0x7f84540cfbe0 2023-10-11T09:51:30.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.884+0000 7f8442ffd700 1 -- 172.21.15.78:0/3491161981 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f844800ee70 con 0x7f84540cfbe0 2023-10-11T09:51:30.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.884+0000 7f8458b8b700 1 -- 172.21.15.78:0/3491161981 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84540d14d0 con 0x7f84540cfbe0 2023-10-11T09:51:30.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.884+0000 7f8442ffd700 1 -- 172.21.15.78:0/3491161981 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844800ca50 con 0x7f84540cfbe0 2023-10-11T09:51:30.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.884+0000 7f8458b8b700 1 -- 172.21.15.78:0/3491161981 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84540d0be0 con 0x7f84540cfbe0 2023-10-11T09:51:30.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.884+0000 7f8458b8b700 1 -- 172.21.15.78:0/3491161981 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84540d1cd0 con 0x7f84540cfbe0 2023-10-11T09:51:30.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.888+0000 7f8442ffd700 1 -- 172.21.15.78:0/3491161981 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f844800cbb0 con 0x7f84540cfbe0 2023-10-11T09:51:30.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.888+0000 7f8442ffd700 1 --2- 172.21.15.78:0/3491161981 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8434065ed0 0x7f8434068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:30.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.888+0000 7f8442ffd700 1 -- 172.21.15.78:0/3491161981 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f8448083810 con 0x7f84540cfbe0 2023-10-11T09:51:30.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.888+0000 7f845259c700 1 --2- 172.21.15.78:0/3491161981 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8434065ed0 0x7f8434068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:30.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.888+0000 7f845259c700 1 --2- 172.21.15.78:0/3491161981 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8434065ed0 0x7f8434068380 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f8444005930 tx=0x7f8444009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:30.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:30.896+0000 7f8442ffd700 1 -- 172.21.15.78:0/3491161981 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f84540d1cd0 con 0x7f84540cfbe0 2023-10-11T09:51:31.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.068+0000 7f8458b8b700 1 -- 172.21.15.78:0/3491161981 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_extra", "pool2": "cephfs_data_extra", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f84540d1cd0 con 0x7f84540cfbe0 2023-10-11T09:51:31.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.684+0000 7f8442ffd700 1 -- 172.21.15.78:0/3491161981 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_extra", "pool2": "cephfs_data_extra", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_extra' removed v118) v1 ==== 187+0+0 (secure 0 0 0) 0x7f84540d1cd0 con 0x7f84540cfbe0 2023-10-11T09:51:31.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.688+0000 7f8458b8b700 1 -- 172.21.15.78:0/3491161981 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8434065ed0 msgr2=0x7f8434068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:31.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.688+0000 7f8458b8b700 1 --2- 172.21.15.78:0/3491161981 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8434065ed0 0x7f8434068380 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f8444005930 tx=0x7f8444009040 comp rx=0 tx=0).stop 2023-10-11T09:51:31.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.688+0000 7f8458b8b700 1 -- 172.21.15.78:0/3491161981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84540cfbe0 msgr2=0x7f84540b7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:31.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.688+0000 7f8458b8b700 1 --2- 172.21.15.78:0/3491161981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84540cfbe0 0x7f84540b7860 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f844800e9d0 tx=0x7f844800ece0 comp rx=0 tx=0).stop 2023-10-11T09:51:31.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.688+0000 7f8458b8b700 1 -- 172.21.15.78:0/3491161981 shutdown_connections 2023-10-11T09:51:31.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.688+0000 7f8458b8b700 1 --2- 172.21.15.78:0/3491161981 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8434065ed0 0x7f8434068380 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:31.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.688+0000 7f8458b8b700 1 --2- 172.21.15.78:0/3491161981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f845415df90 0x7f84540b64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:31.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.688+0000 7f8458b8b700 1 --2- 172.21.15.78:0/3491161981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84540cfbe0 0x7f84540b7860 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:31.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.688+0000 7f8458b8b700 1 --2- 172.21.15.78:0/3491161981 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84540b4ab0 0x7f8454149650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:31.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.688+0000 7f8458b8b700 1 -- 172.21.15.78:0/3491161981 >> 172.21.15.78:0/3491161981 conn(0x7f845406b1d0 msgr2=0x7f845415c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:31.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.692+0000 7f8458b8b700 1 -- 172.21.15.78:0/3491161981 shutdown_connections 2023-10-11T09:51:31.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.692+0000 7f8458b8b700 1 -- 172.21.15.78:0/3491161981 wait complete. 2023-10-11T09:51:31.703 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data_extra' removed 2023-10-11T09:51:31.723 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-11T09:51:31.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.892+0000 7f27fb29c700 1 Processor -- start 2023-10-11T09:51:31.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.896+0000 7f27fb29c700 1 -- start start 2023-10-11T09:51:31.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.896+0000 7f27fb29c700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f27f4146fe0 0x7f27f406bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:31.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.896+0000 7f27fb29c700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27f406ccb0 con 0x7f27f40b36f0 2023-10-11T09:51:31.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.896+0000 7f27fb29c700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27f406c410 con 0x7f27f4161d90 2023-10-11T09:51:31.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.896+0000 7f27fb29c700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27f406c6f0 con 0x7f27f4146fe0 2023-10-11T09:51:31.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.896+0000 7f27faa9b700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f27f40b36f0 0x7f27f41629e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33816/0 (socket says 172.21.15.78:33816) 2023-10-11T09:51:31.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.900+0000 7f27faa9b700 1 -- 172.21.15.78:0/1447337732 learned_addr learned my addr 172.21.15.78:0/1447337732 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:51:31.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.900+0000 7f27fa29a700 1 --2- 172.21.15.78:0/1447337732 >> v2:172.21.15.133:3300/0 conn(0x7f27f4146fe0 0x7f27f406bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:31.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.900+0000 7f27f9298700 1 -- 172.21.15.78:0/1447337732 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1478946046 0 0) 0x7f27f0001110 con 0x7f27f40b36f0 2023-10-11T09:51:31.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.900+0000 7f27f9298700 1 -- 172.21.15.78:0/1447337732 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27e00035d0 con 0x7f27f40b36f0 2023-10-11T09:51:31.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.900+0000 7f27f9298700 1 -- 172.21.15.78:0/1447337732 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3241793464 0 0) 0x7f27e00035d0 con 0x7f27f40b36f0 2023-10-11T09:51:31.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.900+0000 7f27f9298700 1 -- 172.21.15.78:0/1447337732 >> v2:172.21.15.133:3300/0 conn(0x7f27f4146fe0 msgr2=0x7f27f406bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:31.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.900+0000 7f27f9298700 1 --2- 172.21.15.78:0/1447337732 >> v2:172.21.15.133:3300/0 conn(0x7f27f4146fe0 0x7f27f406bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:31.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.900+0000 7f27f9298700 1 -- 172.21.15.78:0/1447337732 >> v1:172.21.15.133:6789/0 conn(0x7f27f4161d90 legacy=0x7f27f40b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:31.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.900+0000 7f27f9298700 1 -- 172.21.15.78:0/1447337732 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f406c9d0 con 0x7f27f40b36f0 2023-10-11T09:51:31.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.900+0000 7f27f9298700 1 -- 172.21.15.78:0/1447337732 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f27f0001800 con 0x7f27f40b36f0 2023-10-11T09:51:31.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.900+0000 7f27f9298700 1 -- 172.21.15.78:0/1447337732 >> v1:172.21.15.78:6789/0 conn(0x7f27f40b36f0 legacy=0x7f27f41629e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:31.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.900+0000 7f27f9298700 1 --2- 172.21.15.78:0/1447337732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27e0003a20 0x7f27e0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:31.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.900+0000 7f27f9298700 1 --2- 172.21.15.78:0/1447337732 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f27e0004870 0x7f27e0004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:31.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.900+0000 7f27f9298700 1 --2- 172.21.15.78:0/1447337732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27e0005340 0x7f27e000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:31.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.900+0000 7f27f9298700 1 -- 172.21.15.78:0/1447337732 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f406c6f0 con 0x7f27e0003a20 2023-10-11T09:51:31.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.900+0000 7f27f9298700 1 -- 172.21.15.78:0/1447337732 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e000c200 con 0x7f27e0005340 2023-10-11T09:51:31.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.900+0000 7f27f9298700 1 -- 172.21.15.78:0/1447337732 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e000c3c0 con 0x7f27e0004870 2023-10-11T09:51:31.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.900+0000 7f27fa29a700 1 --2- 172.21.15.78:0/1447337732 >> v2:172.21.15.133:3300/0 conn(0x7f27f4146fe0 0x7f27f406bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:51:31.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.900+0000 7f27fa29a700 1 --2- 172.21.15.78:0/1447337732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27e0003a20 0x7f27e0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:31.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.900+0000 7f27f9a99700 1 --2- 172.21.15.78:0/1447337732 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f27e0004870 0x7f27e0004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:31.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.900+0000 7f27fa29a700 1 -- 172.21.15.78:0/1447337732 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f27e0004870 msgr2=0x7f27e0004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:31.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.900+0000 7f27fa29a700 1 --2- 172.21.15.78:0/1447337732 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f27e0004870 0x7f27e0004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:31.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.900+0000 7f27fa29a700 1 -- 172.21.15.78:0/1447337732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27e0005340 msgr2=0x7f27e000ba30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-11T09:51:31.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.900+0000 7f27fa29a700 1 --2- 172.21.15.78:0/1447337732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27e0005340 0x7f27e000ba30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:31.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.900+0000 7f27fa29a700 1 -- 172.21.15.78:0/1447337732 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f406c9d0 con 0x7f27e0003a20 2023-10-11T09:51:31.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.900+0000 7f27faa9b700 1 -- 172.21.15.78:0/1447337732 reap_dead start 2023-10-11T09:51:31.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.904+0000 7f27fa29a700 1 --2- 172.21.15.78:0/1447337732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27e0003a20 0x7f27e0003e30 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f27ec0073e0 tx=0x7f27e00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7529bf643fc9c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:31.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.904+0000 7f27f9298700 1 -- 172.21.15.78:0/1447337732 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ec007ea0 con 0x7f27e0003a20 2023-10-11T09:51:31.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.904+0000 7f27f9298700 1 -- 172.21.15.78:0/1447337732 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27ec014400 con 0x7f27e0003a20 2023-10-11T09:51:31.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.904+0000 7f27f9298700 1 -- 172.21.15.78:0/1447337732 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ec014790 con 0x7f27e0003a20 2023-10-11T09:51:31.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.904+0000 7f27fb29c700 1 -- 172.21.15.78:0/1447337732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27e0003a20 msgr2=0x7f27e0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:31.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.904+0000 7f27fb29c700 1 --2- 172.21.15.78:0/1447337732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27e0003a20 0x7f27e0003e30 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f27ec0073e0 tx=0x7f27e00058a0 comp rx=0 tx=0).stop 2023-10-11T09:51:31.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.908+0000 7f27fb29c700 1 -- 172.21.15.78:0/1447337732 shutdown_connections 2023-10-11T09:51:31.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.908+0000 7f27fb29c700 1 --2- 172.21.15.78:0/1447337732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27e0003a20 0x7f27e0003e30 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:31.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.908+0000 7f27fb29c700 1 -- 172.21.15.78:0/1447337732 >> 172.21.15.78:0/1447337732 conn(0x7f27f406a060 msgr2=0x7f27f406a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:31.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.908+0000 7f27fb29c700 1 -- 172.21.15.78:0/1447337732 shutdown_connections 2023-10-11T09:51:31.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.912+0000 7f27fb29c700 1 -- 172.21.15.78:0/1447337732 wait complete. 2023-10-11T09:51:31.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.912+0000 7f27fb29c700 1 Processor -- start 2023-10-11T09:51:31.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.912+0000 7f27fb29c700 1 -- start start 2023-10-11T09:51:31.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.916+0000 7f27fb29c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f27f4144f30 0x7f27f40cde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:31.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.916+0000 7f27fb29c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27f40ce380 0x7f27f41f6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:31.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.916+0000 7f27fb29c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27f41f7020 0x7f27f41f74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:31.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.916+0000 7f27fb29c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f4148e60 con 0x7f27f41f7020 2023-10-11T09:51:31.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.916+0000 7f27fb29c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f4147f40 con 0x7f27f40ce380 2023-10-11T09:51:31.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.916+0000 7f27fb29c700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27f41481e0 con 0x7f27f4144f30 2023-10-11T09:51:31.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.916+0000 7f27f9a99700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27f40ce380 0x7f27f41f6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:31.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.916+0000 7f27f9a99700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27f40ce380 0x7f27f41f6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:51230/0 (socket says 172.21.15.78:51230) 2023-10-11T09:51:31.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.916+0000 7f27f9a99700 1 -- 172.21.15.78:0/2198274638 learned_addr learned my addr 172.21.15.78:0/2198274638 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:31.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.916+0000 7f27faa9b700 1 --2- 172.21.15.78:0/2198274638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27f41f7020 0x7f27f41f74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:31.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.916+0000 7f27fa29a700 1 --2- 172.21.15.78:0/2198274638 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f27f4144f30 0x7f27f40cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:31.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.916+0000 7f27f9a99700 1 -- 172.21.15.78:0/2198274638 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f27f4144f30 msgr2=0x7f27f40cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:31.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.916+0000 7f27f9a99700 1 --2- 172.21.15.78:0/2198274638 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f27f4144f30 0x7f27f40cde40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:31.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.916+0000 7f27f9a99700 1 -- 172.21.15.78:0/2198274638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27f41f7020 msgr2=0x7f27f41f74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:31.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.916+0000 7f27f9a99700 1 --2- 172.21.15.78:0/2198274638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27f41f7020 0x7f27f41f74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:31.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.916+0000 7f27f9a99700 1 -- 172.21.15.78:0/2198274638 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f41484c0 con 0x7f27f40ce380 2023-10-11T09:51:31.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.916+0000 7f27faa9b700 1 --2- 172.21.15.78:0/2198274638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27f41f7020 0x7f27f41f74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:51:31.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.916+0000 7f27f9a99700 1 --2- 172.21.15.78:0/2198274638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27f40ce380 0x7f27f41f6ae0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f27e400f7d0 tx=0x7f27e400fae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:31.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.916+0000 7f27eaffd700 1 -- 172.21.15.78:0/2198274638 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e40052e0 con 0x7f27f40ce380 2023-10-11T09:51:31.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.920+0000 7f27eaffd700 1 -- 172.21.15.78:0/2198274638 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e4005440 con 0x7f27f40ce380 2023-10-11T09:51:31.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.920+0000 7f27eaffd700 1 -- 172.21.15.78:0/2198274638 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e40038c0 con 0x7f27f40ce380 2023-10-11T09:51:31.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.920+0000 7f27fb29c700 1 -- 172.21.15.78:0/2198274638 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27f41487a0 con 0x7f27f40ce380 2023-10-11T09:51:31.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.920+0000 7f27fb29c700 1 -- 172.21.15.78:0/2198274638 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27f4145300 con 0x7f27f40ce380 2023-10-11T09:51:31.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.920+0000 7f27fb29c700 1 -- 172.21.15.78:0/2198274638 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27f41f67e0 con 0x7f27f40ce380 2023-10-11T09:51:31.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.924+0000 7f27eaffd700 1 -- 172.21.15.78:0/2198274638 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f27e4003a20 con 0x7f27f40ce380 2023-10-11T09:51:31.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.924+0000 7f27eaffd700 1 --2- 172.21.15.78:0/2198274638 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f27d8065fd0 0x7f27d8068480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:31.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.924+0000 7f27eaffd700 1 -- 172.21.15.78:0/2198274638 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f27e4083260 con 0x7f27f40ce380 2023-10-11T09:51:31.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.924+0000 7f27fa29a700 1 --2- 172.21.15.78:0/2198274638 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f27d8065fd0 0x7f27d8068480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:31.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.924+0000 7f27fa29a700 1 --2- 172.21.15.78:0/2198274638 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f27d8065fd0 0x7f27d8068480 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f27f40724b0 tx=0x7f27ec00b030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:31.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:31.932+0000 7f27eaffd700 1 -- 172.21.15.78:0/2198274638 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f27e4050b10 con 0x7f27f40ce380 2023-10-11T09:51:32.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.100+0000 7f27fb29c700 1 -- 172.21.15.78:0/2198274638 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f27f4065d40 con 0x7f27f40ce380 2023-10-11T09:51:32.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.112+0000 7f27eaffd700 1 -- 172.21.15.78:0/2198274638 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f27e4054130 con 0x7f27f40ce380 2023-10-11T09:51:32.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.116+0000 7f27fb29c700 1 -- 172.21.15.78:0/2198274638 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f27d8065fd0 msgr2=0x7f27d8068480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:32.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.116+0000 7f27fb29c700 1 --2- 172.21.15.78:0/2198274638 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f27d8065fd0 0x7f27d8068480 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f27f40724b0 tx=0x7f27ec00b030 comp rx=0 tx=0).stop 2023-10-11T09:51:32.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.116+0000 7f27fb29c700 1 -- 172.21.15.78:0/2198274638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27f40ce380 msgr2=0x7f27f41f6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:32.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.116+0000 7f27fb29c700 1 --2- 172.21.15.78:0/2198274638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27f40ce380 0x7f27f41f6ae0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f27e400f7d0 tx=0x7f27e400fae0 comp rx=0 tx=0).stop 2023-10-11T09:51:32.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.116+0000 7f27fb29c700 1 -- 172.21.15.78:0/2198274638 shutdown_connections 2023-10-11T09:51:32.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.116+0000 7f27fb29c700 1 --2- 172.21.15.78:0/2198274638 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f27d8065fd0 0x7f27d8068480 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:32.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.116+0000 7f27fb29c700 1 --2- 172.21.15.78:0/2198274638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27f41f7020 0x7f27f41f74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:32.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.116+0000 7f27fb29c700 1 --2- 172.21.15.78:0/2198274638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27f40ce380 0x7f27f41f6ae0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:32.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.116+0000 7f27fb29c700 1 --2- 172.21.15.78:0/2198274638 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f27f4144f30 0x7f27f40cde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:32.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.116+0000 7f27fb29c700 1 -- 172.21.15.78:0/2198274638 >> 172.21.15.78:0/2198274638 conn(0x7f27f406a060 msgr2=0x7f27f41614f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:32.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.116+0000 7f27fb29c700 1 -- 172.21.15.78:0/2198274638 shutdown_connections 2023-10-11T09:51:32.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.116+0000 7f27fb29c700 1 -- 172.21.15.78:0/2198274638 wait complete. 2023-10-11T09:51:32.151 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_stashed_layout' 2023-10-11T09:51:32.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.312+0000 7f13a2485700 1 Processor -- start 2023-10-11T09:51:32.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.312+0000 7f13a2485700 1 -- start start 2023-10-11T09:51:32.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.316+0000 7f13a2485700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f139c15b8e0 0x7f139c06d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:32.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.316+0000 7f13a2485700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f139c0ca950 0x7f139c13fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:32.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.316+0000 7f13a2485700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f139c15e010 con 0x7f139c06d610 2023-10-11T09:51:32.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.316+0000 7f13a2485700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f139c15e280 con 0x7f139c15b8e0 2023-10-11T09:51:32.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.316+0000 7f13a2485700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f139c15e560 con 0x7f139c0ca950 2023-10-11T09:51:32.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.316+0000 7f13a0c82700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f139c06d610 0x7f139c0ca240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33826/0 (socket says 172.21.15.78:33826) 2023-10-11T09:51:32.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.316+0000 7f13a0c82700 1 -- 172.21.15.78:0/3711193641 learned_addr learned my addr 172.21.15.78:0/3711193641 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:51:32.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.316+0000 7f13a1483700 1 --2- 172.21.15.78:0/3711193641 >> v2:172.21.15.133:3300/0 conn(0x7f139c15b8e0 0x7f139c06d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:32.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.316+0000 7f13a1c84700 1 --2- 172.21.15.78:0/3711193641 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f139c0ca950 0x7f139c13fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:32.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.316+0000 7f1393fff700 1 -- 172.21.15.78:0/3711193641 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2069920343 0 0) 0x7f139c15e010 con 0x7f139c06d610 2023-10-11T09:51:32.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.316+0000 7f1393fff700 1 -- 172.21.15.78:0/3711193641 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13880035d0 con 0x7f139c06d610 2023-10-11T09:51:32.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.316+0000 7f13a1c84700 1 -- 172.21.15.78:0/3711193641 >> v2:172.21.15.133:3300/0 conn(0x7f139c15b8e0 msgr2=0x7f139c06d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:32.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.316+0000 7f13a1c84700 1 --2- 172.21.15.78:0/3711193641 >> v2:172.21.15.133:3300/0 conn(0x7f139c15b8e0 0x7f139c06d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:32.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.316+0000 7f13a1c84700 1 -- 172.21.15.78:0/3711193641 >> v1:172.21.15.78:6789/0 conn(0x7f139c06d610 legacy=0x7f139c0ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:32.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.316+0000 7f1393fff700 1 -- 172.21.15.78:0/3711193641 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3146748099 0 0) 0x7f13880035d0 con 0x7f139c06d610 2023-10-11T09:51:32.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.316+0000 7f13a1c84700 1 -- 172.21.15.78:0/3711193641 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f139c15e840 con 0x7f139c0ca950 2023-10-11T09:51:32.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.316+0000 7f13a1c84700 1 --2- 172.21.15.78:0/3711193641 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f139c0ca950 0x7f139c13fcb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1394009d90 tx=0x7f139400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0af9ba84f6daf2b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:32.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.320+0000 7f1393fff700 1 -- 172.21.15.78:0/3711193641 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139400e070 con 0x7f139c0ca950 2023-10-11T09:51:32.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.320+0000 7f1393fff700 1 -- 172.21.15.78:0/3711193641 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1394003a40 con 0x7f139c0ca950 2023-10-11T09:51:32.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.320+0000 7f1393fff700 1 -- 172.21.15.78:0/3711193641 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138c001670 con 0x7f139c0ca950 2023-10-11T09:51:32.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.320+0000 7f13a2485700 1 -- 172.21.15.78:0/3711193641 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f139c0ca950 msgr2=0x7f139c13fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:32.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.320+0000 7f13a2485700 1 --2- 172.21.15.78:0/3711193641 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f139c0ca950 0x7f139c13fcb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1394009d90 tx=0x7f139400b330 comp rx=0 tx=0).stop 2023-10-11T09:51:32.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.320+0000 7f13a2485700 1 -- 172.21.15.78:0/3711193641 shutdown_connections 2023-10-11T09:51:32.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.320+0000 7f13a2485700 1 --2- 172.21.15.78:0/3711193641 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f139c0ca950 0x7f139c13fcb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:32.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.320+0000 7f13a2485700 1 --2- 172.21.15.78:0/3711193641 >> v2:172.21.15.133:3300/0 conn(0x7f139c15b8e0 0x7f139c06d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:32.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.320+0000 7f13a2485700 1 -- 172.21.15.78:0/3711193641 >> 172.21.15.78:0/3711193641 conn(0x7f139c0cea60 msgr2=0x7f139c0cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:32.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.320+0000 7f13a2485700 1 -- 172.21.15.78:0/3711193641 shutdown_connections 2023-10-11T09:51:32.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.320+0000 7f13a2485700 1 -- 172.21.15.78:0/3711193641 wait complete. 2023-10-11T09:51:32.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.320+0000 7f13a2485700 1 Processor -- start 2023-10-11T09:51:32.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.320+0000 7f13a2485700 1 -- start start 2023-10-11T09:51:32.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.324+0000 7f13a2485700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f139c06d610 0x7f139c159be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:32.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.324+0000 7f13a2485700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f139c0ca950 0x7f139c1f5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:32.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.324+0000 7f13a2485700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f139c15b8e0 0x7f139c0b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:32.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.324+0000 7f13a2485700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f139c13f6c0 con 0x7f139c0ca950 2023-10-11T09:51:32.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.324+0000 7f13a2485700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f139c13f960 con 0x7f139c06d610 2023-10-11T09:51:32.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.324+0000 7f13a2485700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f139c13fc40 con 0x7f139c15b8e0 2023-10-11T09:51:32.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.324+0000 7f13a0c82700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f139c0ca950 0x7f139c1f5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:32.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.324+0000 7f13a0c82700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f139c0ca950 0x7f139c1f5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34638/0 (socket says 172.21.15.78:34638) 2023-10-11T09:51:32.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.324+0000 7f13a0c82700 1 -- 172.21.15.78:0/3332132774 learned_addr learned my addr 172.21.15.78:0/3332132774 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:32.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.324+0000 7f13a1483700 1 --2- 172.21.15.78:0/3332132774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f139c06d610 0x7f139c159be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:32.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.324+0000 7f13a1c84700 1 --2- 172.21.15.78:0/3332132774 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f139c15b8e0 0x7f139c0b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:32.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.324+0000 7f13a0c82700 1 -- 172.21.15.78:0/3332132774 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f139c15b8e0 msgr2=0x7f139c0b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:32.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.324+0000 7f13a0c82700 1 --2- 172.21.15.78:0/3332132774 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f139c15b8e0 0x7f139c0b3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:32.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.324+0000 7f13a0c82700 1 -- 172.21.15.78:0/3332132774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f139c06d610 msgr2=0x7f139c159be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:32.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.324+0000 7f13a0c82700 1 --2- 172.21.15.78:0/3332132774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f139c06d610 0x7f139c159be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:32.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.324+0000 7f13a0c82700 1 -- 172.21.15.78:0/3332132774 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1394009a70 con 0x7f139c0ca950 2023-10-11T09:51:32.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.324+0000 7f13a1483700 1 --2- 172.21.15.78:0/3332132774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f139c06d610 0x7f139c159be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:51:32.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.324+0000 7f13a0c82700 1 --2- 172.21.15.78:0/3332132774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f139c0ca950 0x7f139c1f5d70 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f138c00c520 tx=0x7f138c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:32.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.324+0000 7f1391ffb700 1 -- 172.21.15.78:0/3332132774 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138c004ca0 con 0x7f139c0ca950 2023-10-11T09:51:32.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.324+0000 7f1391ffb700 1 -- 172.21.15.78:0/3332132774 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f138c004e20 con 0x7f139c0ca950 2023-10-11T09:51:32.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.324+0000 7f1391ffb700 1 -- 172.21.15.78:0/3332132774 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138c007310 con 0x7f139c0ca950 2023-10-11T09:51:32.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.324+0000 7f13a2485700 1 -- 172.21.15.78:0/3332132774 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f139c13ff20 con 0x7f139c0ca950 2023-10-11T09:51:32.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.324+0000 7f13a2485700 1 -- 172.21.15.78:0/3332132774 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f139c15a120 con 0x7f139c0ca950 2023-10-11T09:51:32.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.328+0000 7f13a2485700 1 -- 172.21.15.78:0/3332132774 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f139c070280 con 0x7f139c0ca950 2023-10-11T09:51:32.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.328+0000 7f1391ffb700 1 -- 172.21.15.78:0/3332132774 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f138c007470 con 0x7f139c0ca950 2023-10-11T09:51:32.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.328+0000 7f1391ffb700 1 --2- 172.21.15.78:0/3332132774 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1380065ed0 0x7f1380068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:32.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.328+0000 7f1391ffb700 1 -- 172.21.15.78:0/3332132774 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f138c083340 con 0x7f139c0ca950 2023-10-11T09:51:32.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.332+0000 7f13a1483700 1 --2- 172.21.15.78:0/3332132774 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1380065ed0 0x7f1380068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:32.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.332+0000 7f13a1483700 1 --2- 172.21.15.78:0/3332132774 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1380065ed0 0x7f1380068380 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f1398009c40 tx=0x7f1398008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:32.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.336+0000 7f1391ffb700 1 -- 172.21.15.78:0/3332132774 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f138c050b80 con 0x7f139c0ca950 2023-10-11T09:51:32.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.576+0000 7f13a2485700 1 -- 172.21.15.78:0/3332132774 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_stashed_layout"]} v 0) v1 -- 0x7f139c065d40 con 0x7f139c0ca950 2023-10-11T09:51:32.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.688+0000 7f1391ffb700 1 -- 172.21.15.78:0/3332132774 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_stashed_layout"]}]=0 v483) v1 ==== 155+0+0 (secure 0 0 0) 0x7f138c007720 con 0x7f139c0ca950 2023-10-11T09:51:32.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.696+0000 7f13a2485700 1 -- 172.21.15.78:0/3332132774 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1380065ed0 msgr2=0x7f1380068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:32.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.696+0000 7f13a2485700 1 --2- 172.21.15.78:0/3332132774 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1380065ed0 0x7f1380068380 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f1398009c40 tx=0x7f1398008040 comp rx=0 tx=0).stop 2023-10-11T09:51:32.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.696+0000 7f13a2485700 1 -- 172.21.15.78:0/3332132774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f139c0ca950 msgr2=0x7f139c1f5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:32.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.696+0000 7f13a2485700 1 --2- 172.21.15.78:0/3332132774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f139c0ca950 0x7f139c1f5d70 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f138c00c520 tx=0x7f138c00c8e0 comp rx=0 tx=0).stop 2023-10-11T09:51:32.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.696+0000 7f13a2485700 1 -- 172.21.15.78:0/3332132774 shutdown_connections 2023-10-11T09:51:32.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.696+0000 7f13a2485700 1 --2- 172.21.15.78:0/3332132774 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1380065ed0 0x7f1380068380 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:32.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.696+0000 7f13a2485700 1 --2- 172.21.15.78:0/3332132774 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f139c15b8e0 0x7f139c0b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:32.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.696+0000 7f13a2485700 1 --2- 172.21.15.78:0/3332132774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f139c0ca950 0x7f139c1f5d70 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:32.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.696+0000 7f13a2485700 1 --2- 172.21.15.78:0/3332132774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f139c06d610 0x7f139c159be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:32.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.696+0000 7f13a2485700 1 -- 172.21.15.78:0/3332132774 >> 172.21.15.78:0/3332132774 conn(0x7f139c0cea60 msgr2=0x7f139c0c9300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:32.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.696+0000 7f13a2485700 1 -- 172.21.15.78:0/3332132774 shutdown_connections 2023-10-11T09:51:32.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.696+0000 7f13a2485700 1 -- 172.21.15.78:0/3332132774 wait complete. 2023-10-11T09:51:32.729 INFO:tasks.cephfs_test_runner:test_extra_data_pool_stashed_layout (tasks.cephfs.test_data_scan.TestDataScan) ... ok 2023-10-11T09:51:32.729 INFO:tasks.cephfs_test_runner:Starting test: test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan) That when injecting a dentry into a fragmented directory, we put it in the right fragment. 2023-10-11T09:51:32.730 INFO:tasks.cephfs_test_runner:test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan) 2023-10-11T09:51:32.730 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_data_scan.TestDataScan.test_fragmented_injection' 2023-10-11T09:51:32.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.900+0000 7f7a51526700 1 Processor -- start 2023-10-11T09:51:32.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.900+0000 7f7a51526700 1 -- start start 2023-10-11T09:51:32.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.900+0000 7f7a51526700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7a4c161150 0x7f7a4c0d1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:32.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.900+0000 7f7a51526700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f7a4c0d21e0 0x7f7a4c0b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:32.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.900+0000 7f7a51526700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a4c0d11e0 con 0x7f7a4c0b2310 2023-10-11T09:51:32.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.900+0000 7f7a51526700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a4c1615b0 con 0x7f7a4c0d21e0 2023-10-11T09:51:32.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.900+0000 7f7a51526700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a4c0b5370 con 0x7f7a4c161150 2023-10-11T09:51:32.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.900+0000 7f7a4b7fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f7a4c0d21e0 0x7f7a4c0b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:32.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.900+0000 7f7a4b7fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f7a4c0d21e0 0x7f7a4c0b5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34640/0 (socket says 172.21.15.78:34640) 2023-10-11T09:51:32.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.904+0000 7f7a4b7fe700 1 -- 172.21.15.78:0/2086228718 learned_addr learned my addr 172.21.15.78:0/2086228718 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:32.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.904+0000 7f7a4a7fc700 1 --2- 172.21.15.78:0/2086228718 >> v2:172.21.15.133:3300/0 conn(0x7f7a4c161150 0x7f7a4c0d1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:32.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.904+0000 7f7a4b7fe700 1 -- 172.21.15.78:0/2086228718 >> v2:172.21.15.133:3300/0 conn(0x7f7a4c161150 msgr2=0x7f7a4c0d1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:32.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.904+0000 7f7a4b7fe700 1 --2- 172.21.15.78:0/2086228718 >> v2:172.21.15.133:3300/0 conn(0x7f7a4c161150 0x7f7a4c0d1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:32.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.904+0000 7f7a4b7fe700 1 -- 172.21.15.78:0/2086228718 >> v1:172.21.15.78:6789/0 conn(0x7f7a4c0b2310 legacy=0x7f7a4c0b4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:32.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.904+0000 7f7a4b7fe700 1 -- 172.21.15.78:0/2086228718 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4c0d14a0 con 0x7f7a4c0d21e0 2023-10-11T09:51:32.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.904+0000 7f7a49ffb700 1 -- 172.21.15.78:0/2086228718 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3695381882 0 0) 0x7f7a4c0d11e0 con 0x7f7a4c0b2310 2023-10-11T09:51:32.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.904+0000 7f7a4a7fc700 1 --2- 172.21.15.78:0/2086228718 >> v2:172.21.15.133:3300/0 conn(0x7f7a4c161150 0x7f7a4c0d1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:51:32.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.904+0000 7f7a4b7fe700 1 --2- 172.21.15.78:0/2086228718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a4c0d21e0 0x7f7a4c0b5f30 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c0b5ba0 tx=0x7f7a4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef18ad0c3db2f0db server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:32.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.904+0000 7f7a49ffb700 1 -- 172.21.15.78:0/2086228718 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4400e070 con 0x7f7a4c0d21e0 2023-10-11T09:51:32.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.904+0000 7f7a49ffb700 1 -- 172.21.15.78:0/2086228718 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4400b9c0 con 0x7f7a4c0d21e0 2023-10-11T09:51:32.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.904+0000 7f7a49ffb700 1 -- 172.21.15.78:0/2086228718 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4400bd70 con 0x7f7a4c0d21e0 2023-10-11T09:51:32.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.904+0000 7f7a51526700 1 -- 172.21.15.78:0/2086228718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a4c0d21e0 msgr2=0x7f7a4c0b5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:32.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.904+0000 7f7a51526700 1 --2- 172.21.15.78:0/2086228718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a4c0d21e0 0x7f7a4c0b5f30 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c0b5ba0 tx=0x7f7a4400b330 comp rx=0 tx=0).stop 2023-10-11T09:51:32.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.904+0000 7f7a51526700 1 -- 172.21.15.78:0/2086228718 shutdown_connections 2023-10-11T09:51:32.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.904+0000 7f7a51526700 1 --2- 172.21.15.78:0/2086228718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a4c0d21e0 0x7f7a4c0b5f30 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:32.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.904+0000 7f7a51526700 1 --2- 172.21.15.78:0/2086228718 >> v2:172.21.15.133:3300/0 conn(0x7f7a4c161150 0x7f7a4c0d1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:32.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.904+0000 7f7a51526700 1 -- 172.21.15.78:0/2086228718 >> 172.21.15.78:0/2086228718 conn(0x7f7a4c0cfb60 msgr2=0x7f7a4c0cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:32.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.904+0000 7f7a51526700 1 -- 172.21.15.78:0/2086228718 shutdown_connections 2023-10-11T09:51:32.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.904+0000 7f7a51526700 1 -- 172.21.15.78:0/2086228718 wait complete. 2023-10-11T09:51:32.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.908+0000 7f7a51526700 1 Processor -- start 2023-10-11T09:51:32.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.908+0000 7f7a51526700 1 -- start start 2023-10-11T09:51:32.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.908+0000 7f7a51526700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a4c0b2310 0x7f7a4c151310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:32.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.908+0000 7f7a51526700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a4c0d21e0 0x7f7a4c06b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:32.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.908+0000 7f7a51526700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a4c161150 0x7f7a4c06bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:32.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.908+0000 7f7a51526700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c0c7750 con 0x7f7a4c161150 2023-10-11T09:51:32.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.912+0000 7f7a51526700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c0c6db0 con 0x7f7a4c0b2310 2023-10-11T09:51:32.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.912+0000 7f7a51526700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c0c7090 con 0x7f7a4c0d21e0 2023-10-11T09:51:32.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.912+0000 7f7a4b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a4c161150 0x7f7a4c06bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:32.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.912+0000 7f7a4b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a4c161150 0x7f7a4c06bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34642/0 (socket says 172.21.15.78:34642) 2023-10-11T09:51:32.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.912+0000 7f7a4b7fe700 1 -- 172.21.15.78:0/650346708 learned_addr learned my addr 172.21.15.78:0/650346708 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:32.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.912+0000 7f7a4affd700 1 --2- 172.21.15.78:0/650346708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a4c0b2310 0x7f7a4c151310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:32.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.912+0000 7f7a4a7fc700 1 --2- 172.21.15.78:0/650346708 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a4c0d21e0 0x7f7a4c06b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:32.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.912+0000 7f7a4b7fe700 1 -- 172.21.15.78:0/650346708 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a4c0d21e0 msgr2=0x7f7a4c06b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:32.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.912+0000 7f7a4b7fe700 1 --2- 172.21.15.78:0/650346708 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a4c0d21e0 0x7f7a4c06b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:32.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.912+0000 7f7a4b7fe700 1 -- 172.21.15.78:0/650346708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a4c0b2310 msgr2=0x7f7a4c151310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:32.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.912+0000 7f7a4b7fe700 1 --2- 172.21.15.78:0/650346708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a4c0b2310 0x7f7a4c151310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:32.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.912+0000 7f7a4b7fe700 1 -- 172.21.15.78:0/650346708 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a44009b30 con 0x7f7a4c161150 2023-10-11T09:51:32.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.912+0000 7f7a4b7fe700 1 --2- 172.21.15.78:0/650346708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a4c161150 0x7f7a4c06bc60 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c067480 tx=0x7f7a44004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:32.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.912+0000 7f7a337fe700 1 -- 172.21.15.78:0/650346708 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a44004840 con 0x7f7a4c161150 2023-10-11T09:51:32.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.912+0000 7f7a337fe700 1 -- 172.21.15.78:0/650346708 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a440099b0 con 0x7f7a4c161150 2023-10-11T09:51:32.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.912+0000 7f7a51526700 1 -- 172.21.15.78:0/650346708 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a4c0c7370 con 0x7f7a4c161150 2023-10-11T09:51:32.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.912+0000 7f7a337fe700 1 -- 172.21.15.78:0/650346708 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4401c680 con 0x7f7a4c161150 2023-10-11T09:51:32.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.912+0000 7f7a51526700 1 -- 172.21.15.78:0/650346708 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a4c0b7910 con 0x7f7a4c161150 2023-10-11T09:51:32.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.916+0000 7f7a51526700 1 -- 172.21.15.78:0/650346708 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a4c04e530 con 0x7f7a4c161150 2023-10-11T09:51:32.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.916+0000 7f7a337fe700 1 -- 172.21.15.78:0/650346708 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f7a4401c7e0 con 0x7f7a4c161150 2023-10-11T09:51:32.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.916+0000 7f7a337fe700 1 --2- 172.21.15.78:0/650346708 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7a38065ed0 0x7f7a38068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:32.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.916+0000 7f7a337fe700 1 -- 172.21.15.78:0/650346708 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f7a44085270 con 0x7f7a4c161150 2023-10-11T09:51:32.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.920+0000 7f7a4affd700 1 --2- 172.21.15.78:0/650346708 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7a38065ed0 0x7f7a38068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:32.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.920+0000 7f7a4affd700 1 --2- 172.21.15.78:0/650346708 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7a38065ed0 0x7f7a38068380 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c007fd0 tx=0x7f7a3c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:32.924+0000 7f7a337fe700 1 -- 172.21.15.78:0/650346708 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f7a44052a20 con 0x7f7a4c161150 2023-10-11T09:51:33.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.164+0000 7f7a51526700 1 -- 172.21.15.78:0/650346708 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_fragmented_injection"]} v 0) v1 -- 0x7f7a4c0cf1e0 con 0x7f7a4c161150 2023-10-11T09:51:33.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.692+0000 7f7a337fe700 1 -- 172.21.15.78:0/650346708 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_fragmented_injection"]}]=0 v484) v1 ==== 148+0+0 (secure 0 0 0) 0x7f7a44056040 con 0x7f7a4c161150 2023-10-11T09:51:33.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.696+0000 7f7a51526700 1 -- 172.21.15.78:0/650346708 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7a38065ed0 msgr2=0x7f7a38068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:33.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.696+0000 7f7a51526700 1 --2- 172.21.15.78:0/650346708 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7a38065ed0 0x7f7a38068380 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c007fd0 tx=0x7f7a3c009040 comp rx=0 tx=0).stop 2023-10-11T09:51:33.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.696+0000 7f7a51526700 1 -- 172.21.15.78:0/650346708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a4c161150 msgr2=0x7f7a4c06bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:33.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.696+0000 7f7a51526700 1 --2- 172.21.15.78:0/650346708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a4c161150 0x7f7a4c06bc60 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c067480 tx=0x7f7a44004af0 comp rx=0 tx=0).stop 2023-10-11T09:51:33.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.696+0000 7f7a51526700 1 -- 172.21.15.78:0/650346708 shutdown_connections 2023-10-11T09:51:33.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.696+0000 7f7a51526700 1 --2- 172.21.15.78:0/650346708 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7a38065ed0 0x7f7a38068380 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:33.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.696+0000 7f7a51526700 1 --2- 172.21.15.78:0/650346708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a4c161150 0x7f7a4c06bc60 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:33.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.696+0000 7f7a51526700 1 --2- 172.21.15.78:0/650346708 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a4c0d21e0 0x7f7a4c06b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:33.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.696+0000 7f7a51526700 1 --2- 172.21.15.78:0/650346708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a4c0b2310 0x7f7a4c151310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:33.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.696+0000 7f7a51526700 1 -- 172.21.15.78:0/650346708 >> 172.21.15.78:0/650346708 conn(0x7f7a4c0cfb60 msgr2=0x7f7a4c15d110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:33.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.696+0000 7f7a51526700 1 -- 172.21.15.78:0/650346708 shutdown_connections 2023-10-11T09:51:33.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.700+0000 7f7a51526700 1 -- 172.21.15.78:0/650346708 wait complete. 2023-10-11T09:51:33.732 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-11T09:51:33.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.908+0000 7f10b1f5d700 1 Processor -- start 2023-10-11T09:51:33.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.908+0000 7f10b1f5d700 1 -- start start 2023-10-11T09:51:33.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.908+0000 7f10b1f5d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10ac0b3240 0x7f10ac0d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:33.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.908+0000 7f10b1f5d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f10ac0b3610 0x7f10ac0b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:33.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.908+0000 7f10b1f5d700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10ac0b47f0 con 0x7f10ac15d040 2023-10-11T09:51:33.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.908+0000 7f10b1f5d700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10ac0b3fd0 con 0x7f10ac0b3610 2023-10-11T09:51:33.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.908+0000 7f10b1f5d700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac0b8130 con 0x7f10ac0b3240 2023-10-11T09:51:33.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.908+0000 7f10abfff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f10ac0b3610 0x7f10ac0b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:33.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.908+0000 7f10abfff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f10ac0b3610 0x7f10ac0b3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34652/0 (socket says 172.21.15.78:34652) 2023-10-11T09:51:33.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.908+0000 7f10abfff700 1 -- 172.21.15.78:0/3660730121 learned_addr learned my addr 172.21.15.78:0/3660730121 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:33.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.908+0000 7f10b0f5b700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10ac0b3240 0x7f10ac0d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:33.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.912+0000 7f10abfff700 1 -- 172.21.15.78:0/3660730121 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10ac0b3240 msgr2=0x7f10ac0d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:33.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.912+0000 7f10abfff700 1 --2- 172.21.15.78:0/3660730121 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10ac0b3240 0x7f10ac0d29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:33.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.912+0000 7f10abfff700 1 -- 172.21.15.78:0/3660730121 >> v1:172.21.15.133:6789/0 conn(0x7f10ac15d040 legacy=0x7f10ac0b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:33.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.912+0000 7f10abfff700 1 -- 172.21.15.78:0/3660730121 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10ac0b41e0 con 0x7f10ac0b3610 2023-10-11T09:51:33.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.912+0000 7f10abfff700 1 --2- 172.21.15.78:0/3660730121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10ac0b3610 0x7f10ac0b3a90 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f1098009910 tx=0x7f1098009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2d28d0acf3291b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:33.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.912+0000 7f10ab7fe700 1 -- 172.21.15.78:0/3660730121 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1098011830 con 0x7f10ac0b3610 2023-10-11T09:51:33.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.912+0000 7f10ab7fe700 1 -- 172.21.15.78:0/3660730121 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1098011990 con 0x7f10ac0b3610 2023-10-11T09:51:33.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.912+0000 7f10ab7fe700 1 -- 172.21.15.78:0/3660730121 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1098011d40 con 0x7f10ac0b3610 2023-10-11T09:51:33.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.912+0000 7f10b1f5d700 1 -- 172.21.15.78:0/3660730121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10ac0b3610 msgr2=0x7f10ac0b3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:33.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.912+0000 7f10b1f5d700 1 --2- 172.21.15.78:0/3660730121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10ac0b3610 0x7f10ac0b3a90 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f1098009910 tx=0x7f1098009c20 comp rx=0 tx=0).stop 2023-10-11T09:51:33.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.912+0000 7f10b1f5d700 1 -- 172.21.15.78:0/3660730121 shutdown_connections 2023-10-11T09:51:33.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.912+0000 7f10b1f5d700 1 --2- 172.21.15.78:0/3660730121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10ac0b3610 0x7f10ac0b3a90 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:33.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.912+0000 7f10b1f5d700 1 --2- 172.21.15.78:0/3660730121 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10ac0b3240 0x7f10ac0d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:33.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.912+0000 7f10b1f5d700 1 -- 172.21.15.78:0/3660730121 >> 172.21.15.78:0/3660730121 conn(0x7f10ac09c260 msgr2=0x7f10ac0d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:33.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.912+0000 7f10b1f5d700 1 -- 172.21.15.78:0/3660730121 shutdown_connections 2023-10-11T09:51:33.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.912+0000 7f10b1f5d700 1 -- 172.21.15.78:0/3660730121 wait complete. 2023-10-11T09:51:33.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.916+0000 7f10b1f5d700 1 Processor -- start 2023-10-11T09:51:33.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.916+0000 7f10b1f5d700 1 -- start start 2023-10-11T09:51:33.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.920+0000 7f10b1f5d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10ac0b3240 0x7f10ac0c8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:33.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.920+0000 7f10b1f5d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10ac0b3610 0x7f10ac0c9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:33.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.920+0000 7f10b1f5d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10ac15d040 0x7f10ac09fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:33.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.920+0000 7f10b1f5d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac0c78a0 con 0x7f10ac0b3610 2023-10-11T09:51:33.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.920+0000 7f10b1f5d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac0c7b80 con 0x7f10ac0b3240 2023-10-11T09:51:33.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.920+0000 7f10b1f5d700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac0c7e60 con 0x7f10ac15d040 2023-10-11T09:51:33.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.920+0000 7f10abfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10ac0b3610 0x7f10ac0c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:33.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.920+0000 7f10abfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10ac0b3610 0x7f10ac0c9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34664/0 (socket says 172.21.15.78:34664) 2023-10-11T09:51:33.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.920+0000 7f10abfff700 1 -- 172.21.15.78:0/3752331026 learned_addr learned my addr 172.21.15.78:0/3752331026 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:33.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.920+0000 7f10b0f5b700 1 --2- 172.21.15.78:0/3752331026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10ac0b3240 0x7f10ac0c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:33.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.920+0000 7f10b175c700 1 --2- 172.21.15.78:0/3752331026 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10ac15d040 0x7f10ac09fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:33.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.920+0000 7f10abfff700 1 -- 172.21.15.78:0/3752331026 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10ac15d040 msgr2=0x7f10ac09fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:33.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.920+0000 7f10abfff700 1 --2- 172.21.15.78:0/3752331026 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10ac15d040 0x7f10ac09fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:33.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.920+0000 7f10abfff700 1 -- 172.21.15.78:0/3752331026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10ac0b3240 msgr2=0x7f10ac0c8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:33.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.920+0000 7f10abfff700 1 --2- 172.21.15.78:0/3752331026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10ac0b3240 0x7f10ac0c8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:33.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.920+0000 7f10abfff700 1 -- 172.21.15.78:0/3752331026 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1098009570 con 0x7f10ac0b3610 2023-10-11T09:51:33.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.920+0000 7f10b0f5b700 1 --2- 172.21.15.78:0/3752331026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10ac0b3240 0x7f10ac0c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:51:33.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.920+0000 7f10abfff700 1 --2- 172.21.15.78:0/3752331026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10ac0b3610 0x7f10ac0c9380 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f1098013000 tx=0x7f1098004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:33.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.920+0000 7f10a97fa700 1 -- 172.21.15.78:0/3752331026 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1098004590 con 0x7f10ac0b3610 2023-10-11T09:51:33.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.920+0000 7f10a97fa700 1 -- 172.21.15.78:0/3752331026 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109801e070 con 0x7f10ac0b3610 2023-10-11T09:51:33.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.920+0000 7f10a97fa700 1 -- 172.21.15.78:0/3752331026 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1098003900 con 0x7f10ac0b3610 2023-10-11T09:51:33.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.920+0000 7f10b1f5d700 1 -- 172.21.15.78:0/3752331026 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10ac0c8140 con 0x7f10ac0b3610 2023-10-11T09:51:33.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.920+0000 7f10b1f5d700 1 -- 172.21.15.78:0/3752331026 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10ac0c8e60 con 0x7f10ac0b3610 2023-10-11T09:51:33.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.924+0000 7f10b1f5d700 1 -- 172.21.15.78:0/3752331026 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10ac065d40 con 0x7f10ac0b3610 2023-10-11T09:51:33.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.924+0000 7f10a97fa700 1 -- 172.21.15.78:0/3752331026 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f1098003a60 con 0x7f10ac0b3610 2023-10-11T09:51:33.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.924+0000 7f10a97fa700 1 --2- 172.21.15.78:0/3752331026 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1090065ed0 0x7f1090068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:33.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.924+0000 7f10a97fa700 1 -- 172.21.15.78:0/3752331026 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f1098084310 con 0x7f10ac0b3610 2023-10-11T09:51:33.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.924+0000 7f10b0f5b700 1 --2- 172.21.15.78:0/3752331026 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1090065ed0 0x7f1090068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:33.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.924+0000 7f10b0f5b700 1 --2- 172.21.15.78:0/3752331026 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1090065ed0 0x7f1090068380 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f10a0009ba0 tx=0x7f10a0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:33.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:33.932+0000 7f10a97fa700 1 -- 172.21.15.78:0/3752331026 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f1098051b40 con 0x7f10ac0b3610 2023-10-11T09:51:34.114 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.104+0000 7f10b1f5d700 1 -- 172.21.15.78:0/3752331026 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f10ac06ba00 con 0x7f10ac0b3610 2023-10-11T09:51:34.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.112+0000 7f10a97fa700 1 -- 172.21.15.78:0/3752331026 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1098055160 con 0x7f10ac0b3610 2023-10-11T09:51:34.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.116+0000 7f10b1f5d700 1 -- 172.21.15.78:0/3752331026 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1090065ed0 msgr2=0x7f1090068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:34.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.116+0000 7f10b1f5d700 1 --2- 172.21.15.78:0/3752331026 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1090065ed0 0x7f1090068380 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f10a0009ba0 tx=0x7f10a0008040 comp rx=0 tx=0).stop 2023-10-11T09:51:34.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.116+0000 7f10b1f5d700 1 -- 172.21.15.78:0/3752331026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10ac0b3610 msgr2=0x7f10ac0c9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:34.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.116+0000 7f10b1f5d700 1 --2- 172.21.15.78:0/3752331026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10ac0b3610 0x7f10ac0c9380 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f1098013000 tx=0x7f1098004770 comp rx=0 tx=0).stop 2023-10-11T09:51:34.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.116+0000 7f10b1f5d700 1 -- 172.21.15.78:0/3752331026 shutdown_connections 2023-10-11T09:51:34.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.116+0000 7f10b1f5d700 1 --2- 172.21.15.78:0/3752331026 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1090065ed0 0x7f1090068380 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:34.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.116+0000 7f10b1f5d700 1 --2- 172.21.15.78:0/3752331026 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10ac15d040 0x7f10ac09fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:34.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.116+0000 7f10b1f5d700 1 --2- 172.21.15.78:0/3752331026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10ac0b3610 0x7f10ac0c9380 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:34.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.116+0000 7f10b1f5d700 1 --2- 172.21.15.78:0/3752331026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10ac0b3240 0x7f10ac0c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:34.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.116+0000 7f10b1f5d700 1 -- 172.21.15.78:0/3752331026 >> 172.21.15.78:0/3752331026 conn(0x7f10ac09c260 msgr2=0x7f10ac15fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:34.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.116+0000 7f10b1f5d700 1 -- 172.21.15.78:0/3752331026 shutdown_connections 2023-10-11T09:51:34.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.116+0000 7f10b1f5d700 1 -- 172.21.15.78:0/3752331026 wait complete. 2023-10-11T09:51:34.148 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-11T09:51:34.149 DEBUG:teuthology.orchestra.run.smithi078:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-11T09:51:34.189 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-11T09:51:34.233 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-11T09:51:34.233 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:51:34.234 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:51:34.253 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352272k,nr_inodes=4088068,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17487 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-11T09:51:34.254 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:51:34.254 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:51:34.296 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352292k,nr_inodes=4088073,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1430 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275376k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-11T09:51:34.297 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:51:34.297 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:51:34.314 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352272k,nr_inodes=4088068,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17487 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-11T09:51:34.314 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:51:34.315 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:51:34.359 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352292k,nr_inodes=4088073,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1430 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275376k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-11T09:51:34.360 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-11T09:51:34.360 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:51:34.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.532+0000 7f095d2e0700 1 Processor -- start 2023-10-11T09:51:34.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.532+0000 7f095d2e0700 1 -- start start 2023-10-11T09:51:34.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.536+0000 7f095d2e0700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f095815dba0 0x7f09580af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:34.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.536+0000 7f095d2e0700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f09580b2760 0x7f09580ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:34.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.536+0000 7f095d2e0700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09580caef0 con 0x7f0958159500 2023-10-11T09:51:34.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.536+0000 7f095d2e0700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09580afd10 con 0x7f095815dba0 2023-10-11T09:51:34.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.536+0000 7f095d2e0700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09580cccd0 con 0x7f09580b2760 2023-10-11T09:51:34.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.536+0000 7f094e7fc700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f095815dba0 0x7f09580af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:34.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.536+0000 7f0956ffd700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f0958159500 0x7f09580b2050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:33858/0 (socket says 172.21.15.78:33858) 2023-10-11T09:51:34.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.536+0000 7f0956ffd700 1 -- 172.21.15.78:0/751319722 learned_addr learned my addr 172.21.15.78:0/751319722 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:51:34.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.536+0000 7f0957fff700 1 --2- 172.21.15.78:0/751319722 >> v2:172.21.15.133:3300/0 conn(0x7f09580b2760 0x7f09580ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:34.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.536+0000 7f094e7fc700 1 -- 172.21.15.78:0/751319722 >> v2:172.21.15.133:3300/0 conn(0x7f09580b2760 msgr2=0x7f09580ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:34.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.536+0000 7f094e7fc700 1 --2- 172.21.15.78:0/751319722 >> v2:172.21.15.133:3300/0 conn(0x7f09580b2760 0x7f09580ce1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:34.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.536+0000 7f094e7fc700 1 -- 172.21.15.78:0/751319722 >> v1:172.21.15.78:6789/0 conn(0x7f0958159500 legacy=0x7f09580b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:34.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.536+0000 7f094e7fc700 1 -- 172.21.15.78:0/751319722 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09580ccfb0 con 0x7f095815dba0 2023-10-11T09:51:34.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.536+0000 7f09567fc700 1 -- 172.21.15.78:0/751319722 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2283263932 0 0) 0x7f09580caef0 con 0x7f0958159500 2023-10-11T09:51:34.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.536+0000 7f094e7fc700 1 --2- 172.21.15.78:0/751319722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f095815dba0 0x7f09580af7d0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f09580d0400 tx=0x7f0940009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc6177ced4a2027 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:34.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.536+0000 7f09567fc700 1 -- 172.21.15.78:0/751319722 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0940011830 con 0x7f095815dba0 2023-10-11T09:51:34.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.536+0000 7f09567fc700 1 -- 172.21.15.78:0/751319722 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0940011990 con 0x7f095815dba0 2023-10-11T09:51:34.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.536+0000 7f09567fc700 1 -- 172.21.15.78:0/751319722 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0940011d40 con 0x7f095815dba0 2023-10-11T09:51:34.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.540+0000 7f095d2e0700 1 -- 172.21.15.78:0/751319722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f095815dba0 msgr2=0x7f09580af7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:34.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.540+0000 7f095d2e0700 1 --2- 172.21.15.78:0/751319722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f095815dba0 0x7f09580af7d0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f09580d0400 tx=0x7f0940009b30 comp rx=0 tx=0).stop 2023-10-11T09:51:34.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.540+0000 7f095d2e0700 1 -- 172.21.15.78:0/751319722 shutdown_connections 2023-10-11T09:51:34.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.540+0000 7f095d2e0700 1 --2- 172.21.15.78:0/751319722 >> v2:172.21.15.133:3300/0 conn(0x7f09580b2760 0x7f09580ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:34.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.540+0000 7f095d2e0700 1 --2- 172.21.15.78:0/751319722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f095815dba0 0x7f09580af7d0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:34.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.540+0000 7f095d2e0700 1 -- 172.21.15.78:0/751319722 >> 172.21.15.78:0/751319722 conn(0x7f095806dd80 msgr2=0x7f095806e150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:34.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.540+0000 7f095d2e0700 1 -- 172.21.15.78:0/751319722 shutdown_connections 2023-10-11T09:51:34.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.540+0000 7f095d2e0700 1 -- 172.21.15.78:0/751319722 wait complete. 2023-10-11T09:51:34.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.544+0000 7f095d2e0700 1 Processor -- start 2023-10-11T09:51:34.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.544+0000 7f095d2e0700 1 -- start start 2023-10-11T09:51:34.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.544+0000 7f095d2e0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09580b2760 0x7f0958147c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:34.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.544+0000 7f095d2e0700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0958159500 0x7f09580b1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:34.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.544+0000 7f095d2e0700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f095815dba0 0x7f095815ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:34.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.544+0000 7f095d2e0700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09580c81d0 con 0x7f09580b2760 2023-10-11T09:51:34.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.544+0000 7f095d2e0700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09580c7550 con 0x7f095815dba0 2023-10-11T09:51:34.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.544+0000 7f095d2e0700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09580c7830 con 0x7f0958159500 2023-10-11T09:51:34.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.544+0000 7f0956ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09580b2760 0x7f0958147c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:34.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.544+0000 7f0956ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09580b2760 0x7f0958147c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34670/0 (socket says 172.21.15.78:34670) 2023-10-11T09:51:34.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.544+0000 7f0956ffd700 1 -- 172.21.15.78:0/2891536982 learned_addr learned my addr 172.21.15.78:0/2891536982 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:34.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.544+0000 7f094e7fc700 1 --2- 172.21.15.78:0/2891536982 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0958159500 0x7f09580b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:34.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.544+0000 7f0957fff700 1 --2- 172.21.15.78:0/2891536982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f095815dba0 0x7f095815ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:34.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.544+0000 7f0956ffd700 1 -- 172.21.15.78:0/2891536982 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0958159500 msgr2=0x7f09580b1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:34.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.544+0000 7f0956ffd700 1 --2- 172.21.15.78:0/2891536982 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0958159500 0x7f09580b1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:34.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.544+0000 7f0956ffd700 1 -- 172.21.15.78:0/2891536982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f095815dba0 msgr2=0x7f095815ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:34.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.544+0000 7f0956ffd700 1 --2- 172.21.15.78:0/2891536982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f095815dba0 0x7f095815ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:34.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.544+0000 7f0956ffd700 1 -- 172.21.15.78:0/2891536982 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09400094b0 con 0x7f09580b2760 2023-10-11T09:51:34.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.544+0000 7f0956ffd700 1 --2- 172.21.15.78:0/2891536982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09580b2760 0x7f0958147c50 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f0948001f80 tx=0x7f094800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:34.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.544+0000 7f094ffff700 1 -- 172.21.15.78:0/2891536982 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094800ad80 con 0x7f09580b2760 2023-10-11T09:51:34.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.544+0000 7f094ffff700 1 -- 172.21.15.78:0/2891536982 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09480070d0 con 0x7f09580b2760 2023-10-11T09:51:34.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.544+0000 7f094ffff700 1 -- 172.21.15.78:0/2891536982 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0948007480 con 0x7f09580b2760 2023-10-11T09:51:34.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.544+0000 7f095d2e0700 1 -- 172.21.15.78:0/2891536982 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09580c7b10 con 0x7f09580b2760 2023-10-11T09:51:34.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.544+0000 7f095d2e0700 1 -- 172.21.15.78:0/2891536982 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0958154f70 con 0x7f09580b2760 2023-10-11T09:51:34.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.548+0000 7f095d2e0700 1 -- 172.21.15.78:0/2891536982 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f095804e530 con 0x7f09580b2760 2023-10-11T09:51:34.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.548+0000 7f094ffff700 1 -- 172.21.15.78:0/2891536982 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f09480075e0 con 0x7f09580b2760 2023-10-11T09:51:34.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.548+0000 7f094ffff700 1 --2- 172.21.15.78:0/2891536982 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0938065ed0 0x7f0938068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:34.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.548+0000 7f094ffff700 1 -- 172.21.15.78:0/2891536982 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f0948083240 con 0x7f09580b2760 2023-10-11T09:51:34.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.552+0000 7f094e7fc700 1 --2- 172.21.15.78:0/2891536982 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0938065ed0 0x7f0938068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:34.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.552+0000 7f094e7fc700 1 --2- 172.21.15.78:0/2891536982 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0938065ed0 0x7f0938068380 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f09580d0400 tx=0x7f0940008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:34.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.556+0000 7f094ffff700 1 -- 172.21.15.78:0/2891536982 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f0948050a00 con 0x7f09580b2760 2023-10-11T09:51:34.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.764+0000 7f095d2e0700 1 -- 172.21.15.78:0/2891536982 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09580ca7f0 con 0x7f09580b2760 2023-10-11T09:51:34.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.764+0000 7f094ffff700 1 -- 172.21.15.78:0/2891536982 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+3526 (secure 0 0 0) 0x7f0948018de0 con 0x7f09580b2760 2023-10-11T09:51:34.775 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:34.775 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":93,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7148,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/331520590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":331520590},{"type":"v1","addr":"172.21.15.78:6835","nonce":331520590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87},{"gid":7151,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/934539531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":934539531},{"type":"v1","addr":"172.21.15.133:6834","nonce":934539531}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":7154,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3894002128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":3894002128},{"type":"v1","addr":"172.21.15.133:6835","nonce":3894002128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":8805,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/800535024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":800535024},{"type":"v1","addr":"172.21.15.78:6837","nonce":800535024}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93}],"filesystems":[]} 2023-10-11T09:51:34.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.768+0000 7f095d2e0700 1 -- 172.21.15.78:0/2891536982 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0938065ed0 msgr2=0x7f0938068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:34.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.768+0000 7f095d2e0700 1 --2- 172.21.15.78:0/2891536982 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0938065ed0 0x7f0938068380 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f09580d0400 tx=0x7f0940008030 comp rx=0 tx=0).stop 2023-10-11T09:51:34.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.768+0000 7f095d2e0700 1 -- 172.21.15.78:0/2891536982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09580b2760 msgr2=0x7f0958147c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:34.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.768+0000 7f095d2e0700 1 --2- 172.21.15.78:0/2891536982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09580b2760 0x7f0958147c50 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f0948001f80 tx=0x7f094800a340 comp rx=0 tx=0).stop 2023-10-11T09:51:34.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.768+0000 7f095d2e0700 1 -- 172.21.15.78:0/2891536982 shutdown_connections 2023-10-11T09:51:34.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.768+0000 7f095d2e0700 1 --2- 172.21.15.78:0/2891536982 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0938065ed0 0x7f0938068380 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:34.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.768+0000 7f095d2e0700 1 --2- 172.21.15.78:0/2891536982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f095815dba0 0x7f095815ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:34.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.768+0000 7f095d2e0700 1 --2- 172.21.15.78:0/2891536982 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0958159500 0x7f09580b1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:34.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.768+0000 7f095d2e0700 1 --2- 172.21.15.78:0/2891536982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09580b2760 0x7f0958147c50 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:34.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.768+0000 7f095d2e0700 1 -- 172.21.15.78:0/2891536982 >> 172.21.15.78:0/2891536982 conn(0x7f095806dd80 msgr2=0x7f09580c64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:34.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.768+0000 7f095d2e0700 1 -- 172.21.15.78:0/2891536982 shutdown_connections 2023-10-11T09:51:34.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:34.768+0000 7f095d2e0700 1 -- 172.21.15.78:0/2891536982 wait complete. 2023-10-11T09:51:34.781 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 93 2023-10-11T09:51:34.799 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-11T09:51:34.816 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-11T09:51:34.817 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-11T09:51:34.818 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:51:34.818 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-11T09:51:34.819 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-11T09:51:34.819 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-11T09:51:34.819 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:51:34.820 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-11T09:51:34.820 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-11T09:51:34.820 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-11T09:51:34.821 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:51:34.821 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-11T09:51:34.821 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-11T09:51:34.821 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-11T09:51:34.822 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:51:34.844 INFO:tasks.ceph.mds.c:Stopped 2023-10-11T09:51:34.845 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-11T09:51:34.847 INFO:tasks.ceph.mds.a:Stopped 2023-10-11T09:51:34.847 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-11T09:51:34.849 INFO:tasks.ceph.mds.d:Stopped 2023-10-11T09:51:34.850 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-11T09:51:34.853 INFO:tasks.ceph.mds.b:Stopped 2023-10-11T09:51:34.854 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-11T09:51:34.899 INFO:tasks.ceph.mds.c:Started 2023-10-11T09:51:34.900 DEBUG:teuthology.parallel:result is None 2023-10-11T09:51:34.903 INFO:tasks.ceph.mds.d:Started 2023-10-11T09:51:34.903 DEBUG:teuthology.parallel:result is None 2023-10-11T09:51:34.904 INFO:tasks.ceph.mds.a:Started 2023-10-11T09:51:34.905 DEBUG:teuthology.parallel:result is None 2023-10-11T09:51:34.906 INFO:tasks.ceph.mds.b:Started 2023-10-11T09:51:34.906 DEBUG:teuthology.parallel:result is None 2023-10-11T09:51:34.907 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-11T09:51:35.014 INFO:tasks.ceph.mds.a.smithi078.stdout:starting mds.a at 2023-10-11T09:51:35.016 INFO:tasks.ceph.mds.c.smithi078.stdout:starting mds.c at 2023-10-11T09:51:35.025 INFO:tasks.ceph.mds.d.smithi133.stdout:starting mds.d at 2023-10-11T09:51:35.038 INFO:tasks.ceph.mds.b.smithi133.stdout:starting mds.b at 2023-10-11T09:51:35.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.076+0000 7fa98b59e700 1 Processor -- start 2023-10-11T09:51:35.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.076+0000 7fa98b59e700 1 -- start start 2023-10-11T09:51:35.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.080+0000 7fa98b59e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fa98c0cda10 0x7fa98c0caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:35.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.080+0000 7fa98b59e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa98c0cdde0 0x7fa98c164610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:35.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.080+0000 7fa98b59e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa98c164b50 0x7fa98c06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:35.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.080+0000 7fa98b59e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa98c165390 con 0x7fa98c0cda10 2023-10-11T09:51:35.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.080+0000 7fa98b59e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa98c0ca440 con 0x7fa98c164b50 2023-10-11T09:51:35.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.080+0000 7fa98b59e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c0ca720 con 0x7fa98c0cdde0 2023-10-11T09:51:35.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.080+0000 7fa98a59c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fa98c0cda10 0x7fa98c0caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:35.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.080+0000 7fa98a59c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fa98c0cda10 0x7fa98c0caf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43268/0 (socket says 172.21.15.78:43268) 2023-10-11T09:51:35.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.080+0000 7fa98a59c700 1 -- 172.21.15.78:0/3163767111 learned_addr learned my addr 172.21.15.78:0/3163767111 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:35.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.080+0000 7fa989d9b700 1 --2- 172.21.15.78:0/3163767111 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa98c0cdde0 0x7fa98c164610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:35.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.080+0000 7fa98ad9d700 1 --2- 172.21.15.78:0/3163767111 >> v2:172.21.15.133:3300/0 conn(0x7fa98c164b50 0x7fa98c06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:35.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.080+0000 7fa98a59c700 1 -- 172.21.15.78:0/3163767111 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa98c0cdde0 msgr2=0x7fa98c164610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:35.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.080+0000 7fa98a59c700 1 --2- 172.21.15.78:0/3163767111 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa98c0cdde0 0x7fa98c164610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:35.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.080+0000 7fa98a59c700 1 -- 172.21.15.78:0/3163767111 >> v2:172.21.15.133:3300/0 conn(0x7fa98c164b50 msgr2=0x7fa98c06bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:35.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.080+0000 7fa98a59c700 1 --2- 172.21.15.78:0/3163767111 >> v2:172.21.15.133:3300/0 conn(0x7fa98c164b50 0x7fa98c06bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:35.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.080+0000 7fa98a59c700 1 -- 172.21.15.78:0/3163767111 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa98c0caa00 con 0x7fa98c0cda10 2023-10-11T09:51:35.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.080+0000 7fa98a59c700 1 --2- 172.21.15.78:0/3163767111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa98c0cda10 0x7fa98c0caf20 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0b1730 tx=0x7fa97c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ef7be3cda1f2af server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:35.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.080+0000 7fa98959a700 1 -- 172.21.15.78:0/3163767111 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c011830 con 0x7fa98c0cda10 2023-10-11T09:51:35.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.080+0000 7fa98959a700 1 -- 172.21.15.78:0/3163767111 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa97c011990 con 0x7fa98c0cda10 2023-10-11T09:51:35.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.084+0000 7fa98959a700 1 -- 172.21.15.78:0/3163767111 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c011d20 con 0x7fa98c0cda10 2023-10-11T09:51:35.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.084+0000 7fa98b59e700 1 -- 172.21.15.78:0/3163767111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa98c0cda10 msgr2=0x7fa98c0caf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:35.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.084+0000 7fa98b59e700 1 --2- 172.21.15.78:0/3163767111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa98c0cda10 0x7fa98c0caf20 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0b1730 tx=0x7fa97c009bf0 comp rx=0 tx=0).stop 2023-10-11T09:51:35.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.084+0000 7fa98b59e700 1 -- 172.21.15.78:0/3163767111 shutdown_connections 2023-10-11T09:51:35.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.084+0000 7fa98b59e700 1 --2- 172.21.15.78:0/3163767111 >> v2:172.21.15.133:3300/0 conn(0x7fa98c164b50 0x7fa98c06bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:35.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.084+0000 7fa98b59e700 1 --2- 172.21.15.78:0/3163767111 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa98c0cdde0 0x7fa98c164610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:35.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.084+0000 7fa98b59e700 1 --2- 172.21.15.78:0/3163767111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa98c0cda10 0x7fa98c0caf20 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:35.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.084+0000 7fa98b59e700 1 -- 172.21.15.78:0/3163767111 >> 172.21.15.78:0/3163767111 conn(0x7fa98c0cea60 msgr2=0x7fa98c0cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:35.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.084+0000 7fa98b59e700 1 -- 172.21.15.78:0/3163767111 shutdown_connections 2023-10-11T09:51:35.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.084+0000 7fa98b59e700 1 -- 172.21.15.78:0/3163767111 wait complete. 2023-10-11T09:51:35.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.088+0000 7fa98b59e700 1 Processor -- start 2023-10-11T09:51:35.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.088+0000 7fa98b59e700 1 -- start start 2023-10-11T09:51:35.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.088+0000 7fa98b59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa98c0cda10 0x7fa98c1449b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:35.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.088+0000 7fa98b59e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa98c0cdde0 0x7fa98c152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:35.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.088+0000 7fa98b59e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa98c164b50 0x7fa98c0cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:35.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.088+0000 7fa98b59e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c144ef0 con 0x7fa98c0cda10 2023-10-11T09:51:35.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.088+0000 7fa98b59e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c146cd0 con 0x7fa98c164b50 2023-10-11T09:51:35.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.088+0000 7fa98b59e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c147b30 con 0x7fa98c0cdde0 2023-10-11T09:51:35.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.088+0000 7fa98a59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa98c0cda10 0x7fa98c1449b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:35.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.088+0000 7fa98a59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa98c0cda10 0x7fa98c1449b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43280/0 (socket says 172.21.15.78:43280) 2023-10-11T09:51:35.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.088+0000 7fa98a59c700 1 -- 172.21.15.78:0/4129878685 learned_addr learned my addr 172.21.15.78:0/4129878685 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:35.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.088+0000 7fa98ad9d700 1 --2- 172.21.15.78:0/4129878685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa98c164b50 0x7fa98c0cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:35.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.088+0000 7fa98a59c700 1 -- 172.21.15.78:0/4129878685 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa98c0cdde0 msgr2=0x7fa98c152bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:35.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.088+0000 7fa98a59c700 1 --2- 172.21.15.78:0/4129878685 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa98c0cdde0 0x7fa98c152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:35.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.088+0000 7fa98a59c700 1 -- 172.21.15.78:0/4129878685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa98c164b50 msgr2=0x7fa98c0cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:35.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.088+0000 7fa98a59c700 1 --2- 172.21.15.78:0/4129878685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa98c164b50 0x7fa98c0cb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:35.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.088+0000 7fa98a59c700 1 -- 172.21.15.78:0/4129878685 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa97c009570 con 0x7fa98c0cda10 2023-10-11T09:51:35.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.092+0000 7fa98a59c700 1 --2- 172.21.15.78:0/4129878685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa98c0cda10 0x7fa98c1449b0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fa98c072690 tx=0x7fa97c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:35.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.092+0000 7fa97affd700 1 -- 172.21.15.78:0/4129878685 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c004510 con 0x7fa98c0cda10 2023-10-11T09:51:35.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.092+0000 7fa97affd700 1 -- 172.21.15.78:0/4129878685 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa97c01e070 con 0x7fa98c0cda10 2023-10-11T09:51:35.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.092+0000 7fa98b59e700 1 -- 172.21.15.78:0/4129878685 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa98c0ce3b0 con 0x7fa98c0cda10 2023-10-11T09:51:35.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.092+0000 7fa97affd700 1 -- 172.21.15.78:0/4129878685 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c003900 con 0x7fa98c0cda10 2023-10-11T09:51:35.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.092+0000 7fa98b59e700 1 -- 172.21.15.78:0/4129878685 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa98c147200 con 0x7fa98c0cda10 2023-10-11T09:51:35.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.092+0000 7fa98b59e700 1 -- 172.21.15.78:0/4129878685 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa98c0b01a0 con 0x7fa98c0cda10 2023-10-11T09:51:35.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.092+0000 7fa97affd700 1 -- 172.21.15.78:0/4129878685 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fa97c003a60 con 0x7fa98c0cda10 2023-10-11T09:51:35.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.096+0000 7fa97affd700 1 --2- 172.21.15.78:0/4129878685 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa96c065ed0 0x7fa96c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:35.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.096+0000 7fa97affd700 1 -- 172.21.15.78:0/4129878685 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fa97c0847c0 con 0x7fa98c0cda10 2023-10-11T09:51:35.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.096+0000 7fa989d9b700 1 --2- 172.21.15.78:0/4129878685 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa96c065ed0 0x7fa96c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:35.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.096+0000 7fa989d9b700 1 --2- 172.21.15.78:0/4129878685 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa96c065ed0 0x7fa96c068380 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa98c1f9a90 tx=0x7fa9800095b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:35.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.100+0000 7fa97affd700 1 -- 172.21.15.78:0/4129878685 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fa97c051ff0 con 0x7fa98c0cda10 2023-10-11T09:51:35.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:35.288+0000 7fa98b59e700 1 -- 172.21.15.78:0/4129878685 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa98c0c8cf0 con 0x7fa98c0cda10 2023-10-11T09:51:36.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.060+0000 7fa97affd700 1 -- 172.21.15.78:0/4129878685 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v123) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa97c020080 con 0x7fa98c0cda10 2023-10-11T09:51:36.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.064+0000 7fa98b59e700 1 -- 172.21.15.78:0/4129878685 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa96c065ed0 msgr2=0x7fa96c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:36.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.064+0000 7fa98b59e700 1 --2- 172.21.15.78:0/4129878685 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa96c065ed0 0x7fa96c068380 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa98c1f9a90 tx=0x7fa9800095b0 comp rx=0 tx=0).stop 2023-10-11T09:51:36.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.064+0000 7fa98b59e700 1 -- 172.21.15.78:0/4129878685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa98c0cda10 msgr2=0x7fa98c1449b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:36.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.064+0000 7fa98b59e700 1 --2- 172.21.15.78:0/4129878685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa98c0cda10 0x7fa98c1449b0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fa98c072690 tx=0x7fa97c004770 comp rx=0 tx=0).stop 2023-10-11T09:51:36.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.068+0000 7fa98b59e700 1 -- 172.21.15.78:0/4129878685 shutdown_connections 2023-10-11T09:51:36.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.068+0000 7fa98b59e700 1 --2- 172.21.15.78:0/4129878685 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa96c065ed0 0x7fa96c068380 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:36.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.068+0000 7fa98b59e700 1 --2- 172.21.15.78:0/4129878685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa98c164b50 0x7fa98c0cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:36.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.068+0000 7fa98b59e700 1 --2- 172.21.15.78:0/4129878685 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa98c0cdde0 0x7fa98c152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:36.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.068+0000 7fa98b59e700 1 --2- 172.21.15.78:0/4129878685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa98c0cda10 0x7fa98c1449b0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:36.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.068+0000 7fa98b59e700 1 -- 172.21.15.78:0/4129878685 >> 172.21.15.78:0/4129878685 conn(0x7fa98c0cea60 msgr2=0x7fa98c071100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:36.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.068+0000 7fa98b59e700 1 -- 172.21.15.78:0/4129878685 shutdown_connections 2023-10-11T09:51:36.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.068+0000 7fa98b59e700 1 -- 172.21.15.78:0/4129878685 wait complete. 2023-10-11T09:51:36.079 INFO:teuthology.orchestra.run.smithi078.stderr: removed all blocklist entries 2023-10-11T09:51:36.098 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-11T09:51:36.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.264+0000 7fd150a76700 1 Processor -- start 2023-10-11T09:51:36.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.264+0000 7fd150a76700 1 -- start start 2023-10-11T09:51:36.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.264+0000 7fd150a76700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd14c15b8e0 0x7fd14c06d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:36.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.264+0000 7fd150a76700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd14c0ca950 0x7fd14c13fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:36.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.264+0000 7fd150a76700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd14c15e010 con 0x7fd14c06d610 2023-10-11T09:51:36.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.264+0000 7fd150a76700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd14c15e280 con 0x7fd14c15b8e0 2023-10-11T09:51:36.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.264+0000 7fd150a76700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c15e560 con 0x7fd14c0ca950 2023-10-11T09:51:36.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.264+0000 7fd14b7fe700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd14c15b8e0 0x7fd14c06d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:36.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.264+0000 7fd14bfff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd14c0ca950 0x7fd14c13fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:36.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.268+0000 7fd14bfff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd14c0ca950 0x7fd14c13fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:36664/0 (socket says 172.21.15.78:36664) 2023-10-11T09:51:36.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.268+0000 7fd14bfff700 1 -- 172.21.15.78:0/2999492405 learned_addr learned my addr 172.21.15.78:0/2999492405 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:36.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.268+0000 7fd14b7fe700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd14c15b8e0 0x7fd14c06d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:60834/0 (socket says 172.21.15.78:60834) 2023-10-11T09:51:36.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.268+0000 7fd142ffd700 1 --1- 172.21.15.78:0/2999492405 >> v1:172.21.15.133:6789/0 conn(0x7fd14c06d610 0x7fd14c0ca240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:43648/0 (socket says 172.21.15.78:43648) 2023-10-11T09:51:36.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.268+0000 7fd14bfff700 1 -- 172.21.15.78:0/2999492405 >> v2:172.21.15.133:3300/0 conn(0x7fd14c15b8e0 msgr2=0x7fd14c06d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:36.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.268+0000 7fd14bfff700 1 --2- 172.21.15.78:0/2999492405 >> v2:172.21.15.133:3300/0 conn(0x7fd14c15b8e0 0x7fd14c06d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:36.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.268+0000 7fd14bfff700 1 -- 172.21.15.78:0/2999492405 >> v1:172.21.15.133:6789/0 conn(0x7fd14c06d610 legacy=0x7fd14c0ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:36.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.268+0000 7fd14bfff700 1 -- 172.21.15.78:0/2999492405 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14c15e840 con 0x7fd14c0ca950 2023-10-11T09:51:36.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.268+0000 7fd14bfff700 1 --2- 172.21.15.78:0/2999492405 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd14c0ca950 0x7fd14c13fcb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd144009e50 tx=0x7fd14400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0c5f613a62882c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:36.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.268+0000 7fd14affd700 1 -- 172.21.15.78:0/2999492405 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14400e070 con 0x7fd14c0ca950 2023-10-11T09:51:36.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.268+0000 7fd14affd700 1 -- 172.21.15.78:0/2999492405 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd14400b910 con 0x7fd14c0ca950 2023-10-11T09:51:36.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.268+0000 7fd14affd700 1 -- 172.21.15.78:0/2999492405 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14400bcc0 con 0x7fd14c0ca950 2023-10-11T09:51:36.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.268+0000 7fd150a76700 1 -- 172.21.15.78:0/2999492405 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd14c0ca950 msgr2=0x7fd14c13fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:36.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.268+0000 7fd150a76700 1 --2- 172.21.15.78:0/2999492405 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd14c0ca950 0x7fd14c13fcb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd144009e50 tx=0x7fd14400b330 comp rx=0 tx=0).stop 2023-10-11T09:51:36.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.268+0000 7fd150a76700 1 -- 172.21.15.78:0/2999492405 shutdown_connections 2023-10-11T09:51:36.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.268+0000 7fd150a76700 1 --2- 172.21.15.78:0/2999492405 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd14c0ca950 0x7fd14c13fcb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:36.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.268+0000 7fd150a76700 1 --2- 172.21.15.78:0/2999492405 >> v2:172.21.15.133:3300/0 conn(0x7fd14c15b8e0 0x7fd14c06d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:36.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.268+0000 7fd150a76700 1 -- 172.21.15.78:0/2999492405 >> 172.21.15.78:0/2999492405 conn(0x7fd14c0cea60 msgr2=0x7fd14c0cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:36.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.268+0000 7fd150a76700 1 -- 172.21.15.78:0/2999492405 shutdown_connections 2023-10-11T09:51:36.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.268+0000 7fd150a76700 1 -- 172.21.15.78:0/2999492405 wait complete. 2023-10-11T09:51:36.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.272+0000 7fd150a76700 1 Processor -- start 2023-10-11T09:51:36.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.272+0000 7fd150a76700 1 -- start start 2023-10-11T09:51:36.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.272+0000 7fd150a76700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd14c06d610 0x7fd14c140320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:36.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.272+0000 7fd150a76700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd14c0ca950 0x7fd14c0c4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:36.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.272+0000 7fd150a76700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd14c15b8e0 0x7fd14c13e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:36.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.272+0000 7fd150a76700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c0b24e0 con 0x7fd14c15b8e0 2023-10-11T09:51:36.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.272+0000 7fd150a76700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c13eaa0 con 0x7fd14c0ca950 2023-10-11T09:51:36.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.272+0000 7fd150a76700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c0b1bb0 con 0x7fd14c06d610 2023-10-11T09:51:36.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.272+0000 7fd14bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd14c15b8e0 0x7fd14c13e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:36.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.272+0000 7fd14bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd14c15b8e0 0x7fd14c13e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43284/0 (socket says 172.21.15.78:43284) 2023-10-11T09:51:36.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.272+0000 7fd14bfff700 1 -- 172.21.15.78:0/837934857 learned_addr learned my addr 172.21.15.78:0/837934857 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:36.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.276+0000 7fd14b7fe700 1 --2- 172.21.15.78:0/837934857 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd14c06d610 0x7fd14c140320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:36.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.276+0000 7fd142ffd700 1 --2- 172.21.15.78:0/837934857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd14c0ca950 0x7fd14c0c4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:36.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.276+0000 7fd14bfff700 1 -- 172.21.15.78:0/837934857 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd14c06d610 msgr2=0x7fd14c140320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:36.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.276+0000 7fd14bfff700 1 --2- 172.21.15.78:0/837934857 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd14c06d610 0x7fd14c140320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:36.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.276+0000 7fd14bfff700 1 -- 172.21.15.78:0/837934857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd14c0ca950 msgr2=0x7fd14c0c4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:36.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.276+0000 7fd14bfff700 1 --2- 172.21.15.78:0/837934857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd14c0ca950 0x7fd14c0c4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:36.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.276+0000 7fd14bfff700 1 -- 172.21.15.78:0/837934857 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd144009b30 con 0x7fd14c15b8e0 2023-10-11T09:51:36.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.276+0000 7fd14bfff700 1 --2- 172.21.15.78:0/837934857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd14c15b8e0 0x7fd14c13e560 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fd144012000 tx=0x7fd144004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:36.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.276+0000 7fd148ff9700 1 -- 172.21.15.78:0/837934857 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd144004960 con 0x7fd14c15b8e0 2023-10-11T09:51:36.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.276+0000 7fd148ff9700 1 -- 172.21.15.78:0/837934857 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1440099b0 con 0x7fd14c15b8e0 2023-10-11T09:51:36.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.276+0000 7fd148ff9700 1 -- 172.21.15.78:0/837934857 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14401b670 con 0x7fd14c15b8e0 2023-10-11T09:51:36.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.276+0000 7fd150a76700 1 -- 172.21.15.78:0/837934857 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd14c0b1e90 con 0x7fd14c15b8e0 2023-10-11T09:51:36.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.276+0000 7fd150a76700 1 -- 172.21.15.78:0/837934857 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd14c0b42c0 con 0x7fd14c15b8e0 2023-10-11T09:51:36.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.276+0000 7fd150a76700 1 -- 172.21.15.78:0/837934857 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd14c0b2690 con 0x7fd14c15b8e0 2023-10-11T09:51:36.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.280+0000 7fd148ff9700 1 -- 172.21.15.78:0/837934857 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fd14401b7d0 con 0x7fd14c15b8e0 2023-10-11T09:51:36.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.280+0000 7fd148ff9700 1 --2- 172.21.15.78:0/837934857 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd12c065ed0 0x7fd12c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:36.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.280+0000 7fd148ff9700 1 -- 172.21.15.78:0/837934857 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fd144084dc0 con 0x7fd14c15b8e0 2023-10-11T09:51:36.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.280+0000 7fd14b7fe700 1 --2- 172.21.15.78:0/837934857 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd12c065ed0 0x7fd12c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:36.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.280+0000 7fd14b7fe700 1 --2- 172.21.15.78:0/837934857 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd12c065ed0 0x7fd12c068380 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fd14c0ce380 tx=0x7fd13c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:36.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.288+0000 7fd148ff9700 1 -- 172.21.15.78:0/837934857 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fd14c0b2690 con 0x7fd14c15b8e0 2023-10-11T09:51:36.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.516+0000 7fd150a76700 1 -- 172.21.15.78:0/837934857 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd14c065d40 con 0x7fd14c15b8e0 2023-10-11T09:51:36.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.520+0000 7fd148ff9700 1 -- 172.21.15.78:0/837934857 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fd1440527f0 con 0x7fd14c15b8e0 2023-10-11T09:51:36.529 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:36.530 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:51:36.530 INFO:teuthology.orchestra.run.smithi078.stdout: "auth_dump": [ 2023-10-11T09:51:36.530 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:51:36.530 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.a", 2023-10-11T09:51:36.531 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZlL58cAxAALe+WtkV/aLopy1w9i6OVRg==", 2023-10-11T09:51:36.531 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:51:36.531 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:51:36.531 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2023-10-11T09:51:36.531 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2023-10-11T09:51:36.532 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:51:36.532 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:51:36.532 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:51:36.532 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:51:36.532 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.b", 2023-10-11T09:51:36.533 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZliPLmFhAA+6wegaD6yTUsttFzo9OrIg==", 2023-10-11T09:51:36.533 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:51:36.533 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:51:36.533 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2023-10-11T09:51:36.533 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2023-10-11T09:51:36.534 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:51:36.534 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:51:36.534 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:51:36.534 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:51:36.534 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.c", 2023-10-11T09:51:36.535 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZl9jJBDhAAjos7/02OqF0WldFm70/fVA==", 2023-10-11T09:51:36.535 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:51:36.535 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:51:36.535 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2023-10-11T09:51:36.535 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2023-10-11T09:51:36.535 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:51:36.536 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:51:36.536 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:51:36.536 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:51:36.536 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.d", 2023-10-11T09:51:36.536 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZlTb/4IRAAeXtcbWVB5TnSoYch+uMZWw==", 2023-10-11T09:51:36.537 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:51:36.537 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:51:36.537 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2023-10-11T09:51:36.537 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2023-10-11T09:51:36.537 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:51:36.538 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:51:36.538 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:51:36.538 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:51:36.538 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.0", 2023-10-11T09:51:36.538 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBYbSZldbrVFxAA6ogUpCcFO8LhHn4zdkudNQ==", 2023-10-11T09:51:36.538 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:51:36.539 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:51:36.539 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:51:36.539 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:51:36.539 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:51:36.539 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:51:36.540 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:51:36.540 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.1", 2023-10-11T09:51:36.540 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBbbSZl+mkPJxAA4Rrt0j5lzXapYjXhdulhNQ==", 2023-10-11T09:51:36.540 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:51:36.540 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:51:36.541 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:51:36.541 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:51:36.541 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:51:36.541 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:51:36.541 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:51:36.541 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.2", 2023-10-11T09:51:36.542 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBebSZlZ6lSNRAA3ltw4ZTkFNpudCPc2zTYzQ==", 2023-10-11T09:51:36.542 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:51:36.542 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:51:36.542 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:51:36.542 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:51:36.543 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:51:36.543 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:51:36.543 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:51:36.543 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.3", 2023-10-11T09:51:36.543 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBibSZlsE+jBxAA2Ix2JAFTUxljpHAwPpJeUA==", 2023-10-11T09:51:36.544 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:51:36.544 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:51:36.544 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:51:36.544 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:51:36.544 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:51:36.545 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:51:36.545 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:51:36.545 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.4", 2023-10-11T09:51:36.545 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBobSZlII4OGxAA0Z5wGuu4wOs/14tQRnHB1Q==", 2023-10-11T09:51:36.545 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:51:36.546 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:51:36.546 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:51:36.546 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:51:36.546 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:51:36.546 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:51:36.546 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:51:36.547 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.5", 2023-10-11T09:51:36.547 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBrbSZlG6aSKhAAZq179SfpkelcJJLaUcLDCg==", 2023-10-11T09:51:36.547 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:51:36.547 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:51:36.547 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:51:36.548 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:51:36.548 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:51:36.548 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:51:36.548 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:51:36.548 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.6", 2023-10-11T09:51:36.549 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBubSZl3W89NxAAgUrd0/yMOWlrhAOU1BpM9A==", 2023-10-11T09:51:36.549 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:51:36.549 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:51:36.549 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:51:36.549 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:51:36.550 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:51:36.550 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:51:36.550 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:51:36.550 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.7", 2023-10-11T09:51:36.550 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBybSZloIjyCRAAjmvyK9zMSEsV8bVebwzPxg==", 2023-10-11T09:51:36.550 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:51:36.551 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2023-10-11T09:51:36.551 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2023-10-11T09:51:36.551 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:51:36.551 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:51:36.551 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:51:36.552 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:51:36.552 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.0", 2023-10-11T09:51:36.552 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZlmGmCKBAAOI8EvyRwg8HZt+q/vUy5Ow==", 2023-10-11T09:51:36.552 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:51:36.552 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:51:36.553 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow r", 2023-10-11T09:51:36.553 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-11T09:51:36.553 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:51:36.553 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:51:36.553 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:51:36.554 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.1", 2023-10-11T09:51:36.554 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZlYvfHNBAAReum0efx4MmqpLMxFnTROQ==", 2023-10-11T09:51:36.554 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:51:36.554 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:51:36.554 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2023-10-11T09:51:36.554 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2023-10-11T09:51:36.555 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2023-10-11T09:51:36.555 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:51:36.555 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:51:36.555 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:51:36.555 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.2", 2023-10-11T09:51:36.556 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZlbd9lLhAA9LAjreTcxwAjhiNlKndl+Q==", 2023-10-11T09:51:36.556 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:51:36.556 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:51:36.556 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2023-10-11T09:51:36.556 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2023-10-11T09:51:36.557 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2023-10-11T09:51:36.557 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:51:36.557 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:51:36.557 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:51:36.557 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.3", 2023-10-11T09:51:36.557 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBUbSZlHUjeOhAASPKTRvsillob4gjdW0akhQ==", 2023-10-11T09:51:36.558 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:51:36.558 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2023-10-11T09:51:36.558 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2023-10-11T09:51:36.558 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2023-10-11T09:51:36.558 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2023-10-11T09:51:36.559 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:51:36.559 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:51:36.559 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:51:36.559 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.admin", 2023-10-11T09:51:36.559 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBTbSZlhqcOFRAA/lkGYL03++trOuIqNLS0kQ==", 2023-10-11T09:51:36.560 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:51:36.560 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2023-10-11T09:51:36.560 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2023-10-11T09:51:36.560 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2023-10-11T09:51:36.560 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:51:36.560 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:51:36.561 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:51:36.561 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:51:36.561 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mgr.x", 2023-10-11T09:51:36.561 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBTbSZlS/TjNxAAMkq9xRMwgsbLlT4FdobATw==", 2023-10-11T09:51:36.561 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:51:36.562 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2023-10-11T09:51:36.562 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile mgr", 2023-10-11T09:51:36.562 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:51:36.562 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:51:36.562 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:51:36.563 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:51:36.563 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mgr.y", 2023-10-11T09:51:36.563 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQBTbSZlnv4yMRAAQh8w18k76KRjdCvSDJ4QbA==", 2023-10-11T09:51:36.563 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2023-10-11T09:51:36.563 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2023-10-11T09:51:36.564 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile mgr", 2023-10-11T09:51:36.564 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2023-10-11T09:51:36.564 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:51:36.564 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:51:36.564 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2023-10-11T09:51:36.564 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:51:36.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.520+0000 7fd150a76700 1 -- 172.21.15.78:0/837934857 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd12c065ed0 msgr2=0x7fd12c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:36.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.520+0000 7fd150a76700 1 --2- 172.21.15.78:0/837934857 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd12c065ed0 0x7fd12c068380 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fd14c0ce380 tx=0x7fd13c008040 comp rx=0 tx=0).stop 2023-10-11T09:51:36.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.520+0000 7fd150a76700 1 -- 172.21.15.78:0/837934857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd14c15b8e0 msgr2=0x7fd14c13e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:36.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.520+0000 7fd150a76700 1 --2- 172.21.15.78:0/837934857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd14c15b8e0 0x7fd14c13e560 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fd144012000 tx=0x7fd144004af0 comp rx=0 tx=0).stop 2023-10-11T09:51:36.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.524+0000 7fd150a76700 1 -- 172.21.15.78:0/837934857 shutdown_connections 2023-10-11T09:51:36.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.524+0000 7fd150a76700 1 --2- 172.21.15.78:0/837934857 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd12c065ed0 0x7fd12c068380 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:36.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.524+0000 7fd150a76700 1 --2- 172.21.15.78:0/837934857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd14c15b8e0 0x7fd14c13e560 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:36.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.524+0000 7fd150a76700 1 --2- 172.21.15.78:0/837934857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd14c0ca950 0x7fd14c0c4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:36.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.524+0000 7fd150a76700 1 --2- 172.21.15.78:0/837934857 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd14c06d610 0x7fd14c140320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:36.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.524+0000 7fd150a76700 1 -- 172.21.15.78:0/837934857 >> 172.21.15.78:0/837934857 conn(0x7fd14c0cea60 msgr2=0x7fd14c142e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:36.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.524+0000 7fd150a76700 1 -- 172.21.15.78:0/837934857 shutdown_connections 2023-10-11T09:51:36.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.524+0000 7fd150a76700 1 -- 172.21.15.78:0/837934857 wait complete. 2023-10-11T09:51:36.569 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:51:36.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.732+0000 7ffabd597700 1 Processor -- start 2023-10-11T09:51:36.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.732+0000 7ffabd597700 1 -- start start 2023-10-11T09:51:36.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.732+0000 7ffabd597700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ffab8158be0 0x7ffab80ce260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:36.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.732+0000 7ffabd597700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffab80c9020 0x7ffab806fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:36.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.736+0000 7ffabd597700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffab80ce840 con 0x7ffab815c630 2023-10-11T09:51:36.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.736+0000 7ffabd597700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffab8158fb0 con 0x7ffab8158be0 2023-10-11T09:51:36.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.736+0000 7ffabd597700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffab80c9450 con 0x7ffab80c9020 2023-10-11T09:51:36.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.736+0000 7ffabcd96700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7ffab815c630 0x7ffab806ca50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:46214/0 (socket says 172.21.15.78:46214) 2023-10-11T09:51:36.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.736+0000 7ffabcd96700 1 -- 172.21.15.78:0/2598121935 learned_addr learned my addr 172.21.15.78:0/2598121935 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:51:36.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.736+0000 7ffaaffff700 1 --2- 172.21.15.78:0/2598121935 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffab80c9020 0x7ffab806fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:36.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.736+0000 7ffab7fff700 1 --2- 172.21.15.78:0/2598121935 >> v2:172.21.15.133:3300/0 conn(0x7ffab8158be0 0x7ffab80ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:36.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.736+0000 7ffab77fe700 1 -- 172.21.15.78:0/2598121935 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 793702664 0 0) 0x7ffab80ce840 con 0x7ffab815c630 2023-10-11T09:51:36.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.736+0000 7ffaaffff700 1 -- 172.21.15.78:0/2598121935 >> v2:172.21.15.133:3300/0 conn(0x7ffab8158be0 msgr2=0x7ffab80ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:36.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.736+0000 7ffaaffff700 1 --2- 172.21.15.78:0/2598121935 >> v2:172.21.15.133:3300/0 conn(0x7ffab8158be0 0x7ffab80ce260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:36.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.736+0000 7ffaaffff700 1 -- 172.21.15.78:0/2598121935 >> v1:172.21.15.78:6789/0 conn(0x7ffab815c630 legacy=0x7ffab806ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:36.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.736+0000 7ffaaffff700 1 -- 172.21.15.78:0/2598121935 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab806bc90 con 0x7ffab80c9020 2023-10-11T09:51:36.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.736+0000 7ffaaffff700 1 --2- 172.21.15.78:0/2598121935 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffab80c9020 0x7ffab806fc70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0009850 tx=0x7ffaa0009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=827d4db4d6fe2511 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:36.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.736+0000 7ffab77fe700 1 -- 172.21.15.78:0/2598121935 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa0011830 con 0x7ffab80c9020 2023-10-11T09:51:36.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.736+0000 7ffab77fe700 1 -- 172.21.15.78:0/2598121935 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa0011990 con 0x7ffab80c9020 2023-10-11T09:51:36.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.736+0000 7ffab77fe700 1 -- 172.21.15.78:0/2598121935 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa0011d40 con 0x7ffab80c9020 2023-10-11T09:51:36.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.736+0000 7ffabd597700 1 -- 172.21.15.78:0/2598121935 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffab80c9020 msgr2=0x7ffab806fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:36.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.736+0000 7ffabd597700 1 --2- 172.21.15.78:0/2598121935 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffab80c9020 0x7ffab806fc70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0009850 tx=0x7ffaa0009b60 comp rx=0 tx=0).stop 2023-10-11T09:51:36.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.736+0000 7ffabd597700 1 -- 172.21.15.78:0/2598121935 shutdown_connections 2023-10-11T09:51:36.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.736+0000 7ffabd597700 1 --2- 172.21.15.78:0/2598121935 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffab80c9020 0x7ffab806fc70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:36.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.736+0000 7ffabd597700 1 --2- 172.21.15.78:0/2598121935 >> v2:172.21.15.133:3300/0 conn(0x7ffab8158be0 0x7ffab80ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:36.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.740+0000 7ffabd597700 1 -- 172.21.15.78:0/2598121935 >> 172.21.15.78:0/2598121935 conn(0x7ffab80cea60 msgr2=0x7ffab80cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:36.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.740+0000 7ffabd597700 1 -- 172.21.15.78:0/2598121935 shutdown_connections 2023-10-11T09:51:36.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.740+0000 7ffabd597700 1 -- 172.21.15.78:0/2598121935 wait complete. 2023-10-11T09:51:36.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.740+0000 7ffabd597700 1 Processor -- start 2023-10-11T09:51:36.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.740+0000 7ffabd597700 1 -- start start 2023-10-11T09:51:36.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.740+0000 7ffabd597700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffab80c9020 0x7ffab80c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:36.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.740+0000 7ffabd597700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffab8158be0 0x7ffab81461b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:36.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.740+0000 7ffabd597700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffab815c630 0x7ffab81466f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:36.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.740+0000 7ffabd597700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab8147050 con 0x7ffab80c9020 2023-10-11T09:51:36.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.740+0000 7ffabd597700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab8159da0 con 0x7ffab8158be0 2023-10-11T09:51:36.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.740+0000 7ffabd597700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffab8159190 con 0x7ffab815c630 2023-10-11T09:51:36.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.744+0000 7ffab7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffab80c9020 0x7ffab80c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:36.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.744+0000 7ffab7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffab80c9020 0x7ffab80c8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43298/0 (socket says 172.21.15.78:43298) 2023-10-11T09:51:36.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.744+0000 7ffab7fff700 1 -- 172.21.15.78:0/1671603264 learned_addr learned my addr 172.21.15.78:0/1671603264 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:36.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.744+0000 7ffabcd96700 1 --2- 172.21.15.78:0/1671603264 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffab815c630 0x7ffab81466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:36.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.744+0000 7ffaaffff700 1 --2- 172.21.15.78:0/1671603264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffab8158be0 0x7ffab81461b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:36.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.744+0000 7ffab7fff700 1 -- 172.21.15.78:0/1671603264 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffab815c630 msgr2=0x7ffab81466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:36.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.744+0000 7ffab7fff700 1 --2- 172.21.15.78:0/1671603264 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffab815c630 0x7ffab81466f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:36.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.744+0000 7ffab7fff700 1 -- 172.21.15.78:0/1671603264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffab8158be0 msgr2=0x7ffab81461b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:36.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.744+0000 7ffab7fff700 1 --2- 172.21.15.78:0/1671603264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffab8158be0 0x7ffab81461b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:36.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.744+0000 7ffab7fff700 1 -- 172.21.15.78:0/1671603264 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa00094b0 con 0x7ffab80c9020 2023-10-11T09:51:36.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.744+0000 7ffab7fff700 1 --2- 172.21.15.78:0/1671603264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffab80c9020 0x7ffab80c8000 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7ffaa800e8c0 tx=0x7ffaa800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:36.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.744+0000 7ffab57fa700 1 -- 172.21.15.78:0/1671603264 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa800c960 con 0x7ffab80c9020 2023-10-11T09:51:36.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.744+0000 7ffab57fa700 1 -- 172.21.15.78:0/1671603264 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa800cac0 con 0x7ffab80c9020 2023-10-11T09:51:36.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.744+0000 7ffabd597700 1 -- 172.21.15.78:0/1671603264 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffab8159470 con 0x7ffab80c9020 2023-10-11T09:51:36.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.744+0000 7ffab57fa700 1 -- 172.21.15.78:0/1671603264 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa800ce50 con 0x7ffab80c9020 2023-10-11T09:51:36.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.744+0000 7ffabd597700 1 -- 172.21.15.78:0/1671603264 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffab8159a30 con 0x7ffab80c9020 2023-10-11T09:51:36.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.744+0000 7ffabd597700 1 -- 172.21.15.78:0/1671603264 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffab806c510 con 0x7ffab80c9020 2023-10-11T09:51:36.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.748+0000 7ffab57fa700 1 -- 172.21.15.78:0/1671603264 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7ffaa80113f0 con 0x7ffab80c9020 2023-10-11T09:51:36.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.748+0000 7ffab57fa700 1 --2- 172.21.15.78:0/1671603264 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffa98065ed0 0x7ffa98068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:36.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.748+0000 7ffab57fa700 1 -- 172.21.15.78:0/1671603264 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ffaa8083180 con 0x7ffab80c9020 2023-10-11T09:51:36.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.748+0000 7ffaaffff700 1 --2- 172.21.15.78:0/1671603264 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffa98065ed0 0x7ffa98068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:36.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.748+0000 7ffaaffff700 1 --2- 172.21.15.78:0/1671603264 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffa98065ed0 0x7ffa98068380 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ffaa00097a0 tx=0x7ffaa0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:36.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.752+0000 7ffab57fa700 1 -- 172.21.15.78:0/1671603264 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7ffab806c510 con 0x7ffab80c9020 2023-10-11T09:51:36.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.932+0000 7ffabd597700 1 -- 172.21.15.78:0/1671603264 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffab806c510 con 0x7ffab80c9020 2023-10-11T09:51:36.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.932+0000 7ffab57fa700 1 -- 172.21.15.78:0/1671603264 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+14181 (secure 0 0 0) 0x7ffab806c510 con 0x7ffab80c9020 2023-10-11T09:51:36.943 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:36.943 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":123,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:51:36.057133+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:51:36.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.936+0000 7ffabd597700 1 -- 172.21.15.78:0/1671603264 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffa98065ed0 msgr2=0x7ffa98068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:36.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.936+0000 7ffabd597700 1 --2- 172.21.15.78:0/1671603264 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffa98065ed0 0x7ffa98068380 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ffaa00097a0 tx=0x7ffaa0008030 comp rx=0 tx=0).stop 2023-10-11T09:51:36.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.936+0000 7ffabd597700 1 -- 172.21.15.78:0/1671603264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffab80c9020 msgr2=0x7ffab80c8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:36.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.936+0000 7ffabd597700 1 --2- 172.21.15.78:0/1671603264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffab80c9020 0x7ffab80c8000 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7ffaa800e8c0 tx=0x7ffaa800ebd0 comp rx=0 tx=0).stop 2023-10-11T09:51:36.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.936+0000 7ffabd597700 1 -- 172.21.15.78:0/1671603264 shutdown_connections 2023-10-11T09:51:36.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.936+0000 7ffabd597700 1 --2- 172.21.15.78:0/1671603264 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffa98065ed0 0x7ffa98068380 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:36.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.936+0000 7ffabd597700 1 --2- 172.21.15.78:0/1671603264 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffab815c630 0x7ffab81466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:36.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.936+0000 7ffabd597700 1 --2- 172.21.15.78:0/1671603264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffab8158be0 0x7ffab81461b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:36.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.936+0000 7ffabd597700 1 --2- 172.21.15.78:0/1671603264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffab80c9020 0x7ffab80c8000 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:36.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.936+0000 7ffabd597700 1 -- 172.21.15.78:0/1671603264 >> 172.21.15.78:0/1671603264 conn(0x7ffab80cea60 msgr2=0x7ffab80cc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:36.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.936+0000 7ffabd597700 1 -- 172.21.15.78:0/1671603264 shutdown_connections 2023-10-11T09:51:36.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:36.936+0000 7ffabd597700 1 -- 172.21.15.78:0/1671603264 wait complete. 2023-10-11T09:51:36.968 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:51:36.969 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:51:37.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.128+0000 7f57ba7fc700 1 Processor -- start 2023-10-11T09:51:37.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.132+0000 7f57ba7fc700 1 -- start start 2023-10-11T09:51:37.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.132+0000 7f57ba7fc700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f57b4161d90 0x7f57b40b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:37.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.132+0000 7f57ba7fc700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57b406ccb0 con 0x7f57b40b3520 2023-10-11T09:51:37.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.132+0000 7f57ba7fc700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57b406c410 con 0x7f57b4146fe0 2023-10-11T09:51:37.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.132+0000 7f57ba7fc700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b406c6f0 con 0x7f57b4161d90 2023-10-11T09:51:37.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.132+0000 7f57b9ffb700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f57b40b3520 0x7f57b41629e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:46222/0 (socket says 172.21.15.78:46222) 2023-10-11T09:51:37.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.132+0000 7f57b9ffb700 1 -- 172.21.15.78:0/2346687967 learned_addr learned my addr 172.21.15.78:0/2346687967 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:51:37.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.132+0000 7f57b8ff9700 1 --2- 172.21.15.78:0/2346687967 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f57b4161d90 0x7f57b40b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:37.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.132+0000 7f57abfff700 1 -- 172.21.15.78:0/2346687967 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3790185885 0 0) 0x7f57ac001490 con 0x7f57b40b3520 2023-10-11T09:51:37.145 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.132+0000 7f57abfff700 1 -- 172.21.15.78:0/2346687967 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57a00035d0 con 0x7f57b40b3520 2023-10-11T09:51:37.145 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.132+0000 7f57abfff700 1 -- 172.21.15.78:0/2346687967 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2054763844 0 0) 0x7f57b406c410 con 0x7f57b4146fe0 2023-10-11T09:51:37.145 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.132+0000 7f57abfff700 1 -- 172.21.15.78:0/2346687967 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57ac001490 con 0x7f57b4146fe0 2023-10-11T09:51:37.145 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.132+0000 7f57b8ff9700 1 -- 172.21.15.78:0/2346687967 >> v1:172.21.15.133:6789/0 conn(0x7f57b4146fe0 legacy=0x7f57b406bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:37.145 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.132+0000 7f57b8ff9700 1 -- 172.21.15.78:0/2346687967 >> v1:172.21.15.78:6789/0 conn(0x7f57b40b3520 legacy=0x7f57b41629e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:37.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.132+0000 7f57abfff700 1 -- 172.21.15.78:0/2346687967 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 783580634 0 0) 0x7f57a00035d0 con 0x7f57b40b3520 2023-10-11T09:51:37.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.132+0000 7f57b8ff9700 1 -- 172.21.15.78:0/2346687967 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b406c9d0 con 0x7f57b4161d90 2023-10-11T09:51:37.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.136+0000 7f57b8ff9700 1 --2- 172.21.15.78:0/2346687967 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f57b4161d90 0x7f57b40b2fe0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f57a4009880 tx=0x7f57a4009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7df0ad4e10430d9f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:37.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.136+0000 7f57abfff700 1 -- 172.21.15.78:0/2346687967 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac001b80 con 0x7f57b4161d90 2023-10-11T09:51:37.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.136+0000 7f57abfff700 1 -- 172.21.15.78:0/2346687967 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57a4004030 con 0x7f57b4161d90 2023-10-11T09:51:37.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.136+0000 7f57abfff700 1 -- 172.21.15.78:0/2346687967 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a40043c0 con 0x7f57b4161d90 2023-10-11T09:51:37.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.136+0000 7f57ba7fc700 1 -- 172.21.15.78:0/2346687967 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f57b4161d90 msgr2=0x7f57b40b2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:37.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.136+0000 7f57ba7fc700 1 --2- 172.21.15.78:0/2346687967 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f57b4161d90 0x7f57b40b2fe0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f57a4009880 tx=0x7f57a4009b90 comp rx=0 tx=0).stop 2023-10-11T09:51:37.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.136+0000 7f57ba7fc700 1 -- 172.21.15.78:0/2346687967 shutdown_connections 2023-10-11T09:51:37.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.136+0000 7f57ba7fc700 1 --2- 172.21.15.78:0/2346687967 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f57b4161d90 0x7f57b40b2fe0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:37.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.136+0000 7f57ba7fc700 1 -- 172.21.15.78:0/2346687967 >> 172.21.15.78:0/2346687967 conn(0x7f57b406a060 msgr2=0x7f57b406a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:37.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.136+0000 7f57ba7fc700 1 -- 172.21.15.78:0/2346687967 shutdown_connections 2023-10-11T09:51:37.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.136+0000 7f57ba7fc700 1 -- 172.21.15.78:0/2346687967 wait complete. 2023-10-11T09:51:37.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.140+0000 7f57ba7fc700 1 Processor -- start 2023-10-11T09:51:37.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.140+0000 7f57ba7fc700 1 -- start start 2023-10-11T09:51:37.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.140+0000 7f57ba7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57b40b3520 0x7f57b4154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:37.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.140+0000 7f57ba7fc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57b4146fe0 0x7f57b40ab810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:37.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.140+0000 7f57ba7fc700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f57b4161d90 0x7f57b40abd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:37.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.140+0000 7f57ba7fc700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b40a84c0 con 0x7f57b40b3520 2023-10-11T09:51:37.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.140+0000 7f57ba7fc700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b40a9040 con 0x7f57b4146fe0 2023-10-11T09:51:37.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.140+0000 7f57ba7fc700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b40a8710 con 0x7f57b4161d90 2023-10-11T09:51:37.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.140+0000 7f57b97fa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57b40b3520 0x7f57b4154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:37.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.140+0000 7f57b97fa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57b40b3520 0x7f57b4154ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43300/0 (socket says 172.21.15.78:43300) 2023-10-11T09:51:37.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.140+0000 7f57b97fa700 1 -- 172.21.15.78:0/3951776396 learned_addr learned my addr 172.21.15.78:0/3951776396 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:37.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.140+0000 7f57b97fa700 1 -- 172.21.15.78:0/3951776396 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f57b4161d90 msgr2=0x7f57b40abd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:37.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.140+0000 7f57b9ffb700 1 --2- 172.21.15.78:0/3951776396 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f57b4161d90 0x7f57b40abd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:37.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.140+0000 7f57b8ff9700 1 --2- 172.21.15.78:0/3951776396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57b4146fe0 0x7f57b40ab810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:37.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.140+0000 7f57b97fa700 1 --2- 172.21.15.78:0/3951776396 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f57b4161d90 0x7f57b40abd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:37.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.140+0000 7f57b97fa700 1 -- 172.21.15.78:0/3951776396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57b4146fe0 msgr2=0x7f57b40ab810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:37.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.140+0000 7f57b97fa700 1 --2- 172.21.15.78:0/3951776396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57b4146fe0 0x7f57b40ab810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:37.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.140+0000 7f57b97fa700 1 -- 172.21.15.78:0/3951776396 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a40094b0 con 0x7f57b40b3520 2023-10-11T09:51:37.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.140+0000 7f57b9ffb700 1 --2- 172.21.15.78:0/3951776396 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f57b4161d90 0x7f57b40abd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:51:37.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.140+0000 7f57b8ff9700 1 --2- 172.21.15.78:0/3951776396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57b4146fe0 0x7f57b40ab810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:51:37.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.140+0000 7f57b97fa700 1 --2- 172.21.15.78:0/3951776396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57b40b3520 0x7f57b4154ff0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f57b000c730 tx=0x7f57b000caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:37.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.140+0000 7f57a9ffb700 1 -- 172.21.15.78:0/3951776396 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b00052d0 con 0x7f57b40b3520 2023-10-11T09:51:37.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.140+0000 7f57a9ffb700 1 -- 172.21.15.78:0/3951776396 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b0005430 con 0x7f57b40b3520 2023-10-11T09:51:37.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.144+0000 7f57ba7fc700 1 -- 172.21.15.78:0/3951776396 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b40a89f0 con 0x7f57b40b3520 2023-10-11T09:51:37.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.144+0000 7f57a9ffb700 1 -- 172.21.15.78:0/3951776396 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b00055a0 con 0x7f57b40b3520 2023-10-11T09:51:37.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.144+0000 7f57ba7fc700 1 -- 172.21.15.78:0/3951776396 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b40ce240 con 0x7f57b40b3520 2023-10-11T09:51:37.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.144+0000 7f57ba7fc700 1 -- 172.21.15.78:0/3951776396 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57b4065d40 con 0x7f57b40b3520 2023-10-11T09:51:37.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.144+0000 7f57a9ffb700 1 -- 172.21.15.78:0/3951776396 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f57b00038e0 con 0x7f57b40b3520 2023-10-11T09:51:37.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.148+0000 7f57a9ffb700 1 --2- 172.21.15.78:0/3951776396 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5798065ed0 0x7f5798068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:37.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.148+0000 7f57a9ffb700 1 -- 172.21.15.78:0/3951776396 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f57b00829a0 con 0x7f57b40b3520 2023-10-11T09:51:37.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.148+0000 7f57b8ff9700 1 --2- 172.21.15.78:0/3951776396 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5798065ed0 0x7f5798068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:37.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.148+0000 7f57b8ff9700 1 --2- 172.21.15.78:0/3951776396 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5798065ed0 0x7f5798068380 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f57b40ab0f0 tx=0x7f57a4010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:37.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.152+0000 7f57a9ffb700 1 -- 172.21.15.78:0/3951776396 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f57b00503d0 con 0x7f57b40b3520 2023-10-11T09:51:37.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.320+0000 7f57ba7fc700 1 -- 172.21.15.78:0/3951776396 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f57b406ce50 con 0x7f57b40b3520 2023-10-11T09:51:37.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.324+0000 7f57a9ffb700 1 -- 172.21.15.78:0/3951776396 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) v1 ==== 93+0+10 (secure 0 0 0) 0x7f57b00539f0 con 0x7f57b40b3520 2023-10-11T09:51:37.333 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:51:37.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.324+0000 7f57ba7fc700 1 -- 172.21.15.78:0/3951776396 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5798065ed0 msgr2=0x7f5798068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:37.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.324+0000 7f57ba7fc700 1 --2- 172.21.15.78:0/3951776396 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5798065ed0 0x7f5798068380 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f57b40ab0f0 tx=0x7f57a4010000 comp rx=0 tx=0).stop 2023-10-11T09:51:37.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.324+0000 7f57ba7fc700 1 -- 172.21.15.78:0/3951776396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57b40b3520 msgr2=0x7f57b4154ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:37.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.324+0000 7f57ba7fc700 1 --2- 172.21.15.78:0/3951776396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57b40b3520 0x7f57b4154ff0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f57b000c730 tx=0x7f57b000caf0 comp rx=0 tx=0).stop 2023-10-11T09:51:37.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.324+0000 7f57ba7fc700 1 -- 172.21.15.78:0/3951776396 shutdown_connections 2023-10-11T09:51:37.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.324+0000 7f57ba7fc700 1 --2- 172.21.15.78:0/3951776396 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5798065ed0 0x7f5798068380 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:37.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.324+0000 7f57ba7fc700 1 --2- 172.21.15.78:0/3951776396 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f57b4161d90 0x7f57b40abd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:37.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.324+0000 7f57ba7fc700 1 --2- 172.21.15.78:0/3951776396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57b4146fe0 0x7f57b40ab810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:37.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.324+0000 7f57ba7fc700 1 --2- 172.21.15.78:0/3951776396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57b40b3520 0x7f57b4154ff0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:37.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.328+0000 7f57ba7fc700 1 -- 172.21.15.78:0/3951776396 >> 172.21.15.78:0/3951776396 conn(0x7f57b406a060 msgr2=0x7f57b40ae660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:37.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.328+0000 7f57ba7fc700 1 -- 172.21.15.78:0/3951776396 shutdown_connections 2023-10-11T09:51:37.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.328+0000 7f57ba7fc700 1 -- 172.21.15.78:0/3951776396 wait complete. 2023-10-11T09:51:37.356 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:51:37.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.528+0000 7f30bab37700 1 Processor -- start 2023-10-11T09:51:37.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.528+0000 7f30bab37700 1 -- start start 2023-10-11T09:51:37.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.532+0000 7f30bab37700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f30b40b1030 0x7f30b40cc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:37.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.532+0000 7f30bab37700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30b40ca890 con 0x7f30b40ca4c0 2023-10-11T09:51:37.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.532+0000 7f30bab37700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30b40cd300 con 0x7f30b40c9aa0 2023-10-11T09:51:37.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.532+0000 7f30bab37700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30b40cce80 con 0x7f30b40b1030 2023-10-11T09:51:37.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.532+0000 7f30b88d3700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f30b40ca4c0 0x7f30b40c9390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:46234/0 (socket says 172.21.15.78:46234) 2023-10-11T09:51:37.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.532+0000 7f30b88d3700 1 -- 172.21.15.78:0/389420381 learned_addr learned my addr 172.21.15.78:0/389420381 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:51:37.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.532+0000 7f30b90d4700 1 --2- 172.21.15.78:0/389420381 >> v2:172.21.15.78:3300/0 conn(0x7f30b40b1030 0x7f30b40cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:37.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.532+0000 7f30b37fe700 1 -- 172.21.15.78:0/389420381 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 229177584 0 0) 0x7f30b40ca890 con 0x7f30b40ca4c0 2023-10-11T09:51:37.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.532+0000 7f30b37fe700 1 -- 172.21.15.78:0/389420381 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f309c0035d0 con 0x7f30b40ca4c0 2023-10-11T09:51:37.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.532+0000 7f30b90d4700 1 -- 172.21.15.78:0/389420381 >> v1:172.21.15.133:6789/0 conn(0x7f30b40c9aa0 legacy=0x7f30b40b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:37.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.532+0000 7f30b90d4700 1 -- 172.21.15.78:0/389420381 >> v1:172.21.15.78:6789/0 conn(0x7f30b40ca4c0 legacy=0x7f30b40c9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:37.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.532+0000 7f30b90d4700 1 -- 172.21.15.78:0/389420381 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b40cd5e0 con 0x7f30b40b1030 2023-10-11T09:51:37.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.532+0000 7f30b90d4700 1 --2- 172.21.15.78:0/389420381 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30b40b1030 0x7f30b40cc800 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f30ac009d90 tx=0x7f30ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632f7d9df903961 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:37.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.532+0000 7f30b37fe700 1 -- 172.21.15.78:0/389420381 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac00e070 con 0x7f30b40b1030 2023-10-11T09:51:37.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.532+0000 7f30b37fe700 1 -- 172.21.15.78:0/389420381 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30ac00bb20 con 0x7f30b40b1030 2023-10-11T09:51:37.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.532+0000 7f30b37fe700 1 -- 172.21.15.78:0/389420381 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac012430 con 0x7f30b40b1030 2023-10-11T09:51:37.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.536+0000 7f30bab37700 1 -- 172.21.15.78:0/389420381 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30b40b1030 msgr2=0x7f30b40cc800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:37.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.536+0000 7f30bab37700 1 --2- 172.21.15.78:0/389420381 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30b40b1030 0x7f30b40cc800 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f30ac009d90 tx=0x7f30ac00b330 comp rx=0 tx=0).stop 2023-10-11T09:51:37.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.536+0000 7f30bab37700 1 -- 172.21.15.78:0/389420381 shutdown_connections 2023-10-11T09:51:37.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.536+0000 7f30bab37700 1 --2- 172.21.15.78:0/389420381 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30b40b1030 0x7f30b40cc800 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:37.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.536+0000 7f30bab37700 1 -- 172.21.15.78:0/389420381 >> 172.21.15.78:0/389420381 conn(0x7f30b40cecc0 msgr2=0x7f30b40cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:37.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.536+0000 7f30bab37700 1 -- 172.21.15.78:0/389420381 shutdown_connections 2023-10-11T09:51:37.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.536+0000 7f30bab37700 1 -- 172.21.15.78:0/389420381 wait complete. 2023-10-11T09:51:37.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.540+0000 7f30bab37700 1 Processor -- start 2023-10-11T09:51:37.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.540+0000 7f30bab37700 1 -- start start 2023-10-11T09:51:37.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.540+0000 7f30bab37700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30b40b1030 0x7f30b41f2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:37.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.540+0000 7f30bab37700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30b40c9aa0 0x7f30b41422d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:37.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.540+0000 7f30bab37700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f30b40ca4c0 0x7f30b40714b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:37.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.540+0000 7f30bab37700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b41f32b0 con 0x7f30b40c9aa0 2023-10-11T09:51:37.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.540+0000 7f30bab37700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b4142810 con 0x7f30b40b1030 2023-10-11T09:51:37.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.540+0000 7f30bab37700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b40afc40 con 0x7f30b40ca4c0 2023-10-11T09:51:37.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.540+0000 7f30b3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30b40c9aa0 0x7f30b41422d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:37.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.540+0000 7f30b88d3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30b40b1030 0x7f30b41f2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:37.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.540+0000 7f30b3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30b40c9aa0 0x7f30b41422d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43318/0 (socket says 172.21.15.78:43318) 2023-10-11T09:51:37.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.540+0000 7f30b3fff700 1 -- 172.21.15.78:0/1453397583 learned_addr learned my addr 172.21.15.78:0/1453397583 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:37.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.540+0000 7f30b90d4700 1 --2- 172.21.15.78:0/1453397583 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f30b40ca4c0 0x7f30b40714b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:37.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.540+0000 7f30b3fff700 1 -- 172.21.15.78:0/1453397583 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f30b40ca4c0 msgr2=0x7f30b40714b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:37.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.540+0000 7f30b3fff700 1 --2- 172.21.15.78:0/1453397583 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f30b40ca4c0 0x7f30b40714b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:37.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.540+0000 7f30b3fff700 1 -- 172.21.15.78:0/1453397583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30b40b1030 msgr2=0x7f30b41f2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:37.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.540+0000 7f30b3fff700 1 --2- 172.21.15.78:0/1453397583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30b40b1030 0x7f30b41f2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:37.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.540+0000 7f30b3fff700 1 -- 172.21.15.78:0/1453397583 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30ac009a70 con 0x7f30b40c9aa0 2023-10-11T09:51:37.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.540+0000 7f30b3fff700 1 --2- 172.21.15.78:0/1453397583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30b40c9aa0 0x7f30b41422d0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f30a400c500 tx=0x7f30a400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:37.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.540+0000 7f30b17fa700 1 -- 172.21.15.78:0/1453397583 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a40052d0 con 0x7f30b40c9aa0 2023-10-11T09:51:37.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.540+0000 7f30b17fa700 1 -- 172.21.15.78:0/1453397583 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a4005430 con 0x7f30b40c9aa0 2023-10-11T09:51:37.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.540+0000 7f30bab37700 1 -- 172.21.15.78:0/1453397583 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30b40aff20 con 0x7f30b40c9aa0 2023-10-11T09:51:37.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.540+0000 7f30b17fa700 1 -- 172.21.15.78:0/1453397583 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a40055a0 con 0x7f30b40c9aa0 2023-10-11T09:51:37.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.544+0000 7f30bab37700 1 -- 172.21.15.78:0/1453397583 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30b40b04e0 con 0x7f30b40c9aa0 2023-10-11T09:51:37.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.544+0000 7f30bab37700 1 -- 172.21.15.78:0/1453397583 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30b406eb80 con 0x7f30b40c9aa0 2023-10-11T09:51:37.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.544+0000 7f30b17fa700 1 -- 172.21.15.78:0/1453397583 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f30a40038e0 con 0x7f30b40c9aa0 2023-10-11T09:51:37.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.548+0000 7f30b17fa700 1 --2- 172.21.15.78:0/1453397583 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f30a0065f00 0x7f30a00683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:37.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.548+0000 7f30b17fa700 1 -- 172.21.15.78:0/1453397583 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f30a4082880 con 0x7f30b40c9aa0 2023-10-11T09:51:37.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.548+0000 7f30b88d3700 1 --2- 172.21.15.78:0/1453397583 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f30a0065f00 0x7f30a00683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:37.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.548+0000 7f30b88d3700 1 --2- 172.21.15.78:0/1453397583 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f30a0065f00 0x7f30a00683b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f30a8007fd0 tx=0x7f30a8007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:37.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.552+0000 7f30b17fa700 1 -- 172.21.15.78:0/1453397583 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f30a4050230 con 0x7f30b40c9aa0 2023-10-11T09:51:37.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.728+0000 7f30bab37700 1 -- 172.21.15.78:0/1453397583 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f30b4065db0 con 0x7f30b40c9aa0 2023-10-11T09:51:37.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.728+0000 7f30b17fa700 1 -- 172.21.15.78:0/1453397583 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v123) v1 ==== 92+0+10 (secure 0 0 0) 0x7f30a4017020 con 0x7f30b40c9aa0 2023-10-11T09:51:37.738 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:51:37.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.728+0000 7f30bab37700 1 -- 172.21.15.78:0/1453397583 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f30a0065f00 msgr2=0x7f30a00683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:37.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.728+0000 7f30bab37700 1 --2- 172.21.15.78:0/1453397583 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f30a0065f00 0x7f30a00683b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f30a8007fd0 tx=0x7f30a8007de0 comp rx=0 tx=0).stop 2023-10-11T09:51:37.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.728+0000 7f30bab37700 1 -- 172.21.15.78:0/1453397583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30b40c9aa0 msgr2=0x7f30b41422d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:37.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.728+0000 7f30bab37700 1 --2- 172.21.15.78:0/1453397583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30b40c9aa0 0x7f30b41422d0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f30a400c500 tx=0x7f30a400c810 comp rx=0 tx=0).stop 2023-10-11T09:51:37.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.732+0000 7f30bab37700 1 -- 172.21.15.78:0/1453397583 shutdown_connections 2023-10-11T09:51:37.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.732+0000 7f30bab37700 1 --2- 172.21.15.78:0/1453397583 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f30a0065f00 0x7f30a00683b0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:37.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.732+0000 7f30bab37700 1 --2- 172.21.15.78:0/1453397583 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f30b40ca4c0 0x7f30b40714b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:37.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.732+0000 7f30bab37700 1 --2- 172.21.15.78:0/1453397583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30b40c9aa0 0x7f30b41422d0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:37.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.732+0000 7f30bab37700 1 --2- 172.21.15.78:0/1453397583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30b40b1030 0x7f30b41f2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:37.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.732+0000 7f30bab37700 1 -- 172.21.15.78:0/1453397583 >> 172.21.15.78:0/1453397583 conn(0x7f30b40cecc0 msgr2=0x7f30b415f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:37.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.732+0000 7f30bab37700 1 -- 172.21.15.78:0/1453397583 shutdown_connections 2023-10-11T09:51:37.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.732+0000 7f30bab37700 1 -- 172.21.15.78:0/1453397583 wait complete. 2023-10-11T09:51:37.760 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-11T09:51:37.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.916+0000 7f463ee7e700 1 Processor -- start 2023-10-11T09:51:37.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.916+0000 7f463ee7e700 1 -- start start 2023-10-11T09:51:37.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.920+0000 7f463ee7e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4638164d20 0x7f463806bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:37.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.920+0000 7f463ee7e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4638165390 con 0x7f46380caf20 2023-10-11T09:51:37.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.920+0000 7f463ee7e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46380ca440 con 0x7f46380cda10 2023-10-11T09:51:37.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.920+0000 7f463ee7e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46380cb3a0 con 0x7f4638164d20 2023-10-11T09:51:37.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.920+0000 7f463d67b700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f46380caf20 0x7f4638164610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:46248/0 (socket says 172.21.15.78:46248) 2023-10-11T09:51:37.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.920+0000 7f463d67b700 1 -- 172.21.15.78:0/3208607015 learned_addr learned my addr 172.21.15.78:0/3208607015 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:51:37.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.920+0000 7f463e67d700 1 --2- 172.21.15.78:0/3208607015 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4638164d20 0x7f463806bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:37.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.920+0000 7f463ce7a700 1 -- 172.21.15.78:0/3208607015 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1711711525 0 0) 0x7f4638165390 con 0x7f46380caf20 2023-10-11T09:51:37.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.920+0000 7f463ce7a700 1 -- 172.21.15.78:0/3208607015 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46240035d0 con 0x7f46380caf20 2023-10-11T09:51:37.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.920+0000 7f463ce7a700 1 -- 172.21.15.78:0/3208607015 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1067473893 0 0) 0x7f46380ca440 con 0x7f46380cda10 2023-10-11T09:51:37.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.920+0000 7f463ce7a700 1 -- 172.21.15.78:0/3208607015 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4638165390 con 0x7f46380cda10 2023-10-11T09:51:37.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.920+0000 7f463ce7a700 1 -- 172.21.15.78:0/3208607015 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1770460380 0 0) 0x7f46240035d0 con 0x7f46380caf20 2023-10-11T09:51:37.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.920+0000 7f463ce7a700 1 -- 172.21.15.78:0/3208607015 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4638164d20 msgr2=0x7f463806bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:37.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.920+0000 7f463ce7a700 1 --2- 172.21.15.78:0/3208607015 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4638164d20 0x7f463806bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:37.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.920+0000 7f463ce7a700 1 -- 172.21.15.78:0/3208607015 >> v1:172.21.15.133:6789/0 conn(0x7f46380cda10 legacy=0x7f46380cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:37.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.920+0000 7f463ce7a700 1 -- 172.21.15.78:0/3208607015 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46380ca720 con 0x7f46380caf20 2023-10-11T09:51:37.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.920+0000 7f463ce7a700 1 -- 172.21.15.78:0/3208607015 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f4628001670 con 0x7f46380caf20 2023-10-11T09:51:37.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.920+0000 7f463ce7a700 1 -- 172.21.15.78:0/3208607015 >> v1:172.21.15.78:6789/0 conn(0x7f46380caf20 legacy=0x7f4638164610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:37.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.920+0000 7f463ce7a700 1 --2- 172.21.15.78:0/3208607015 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4624003be0 0x7f4624003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:37.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.920+0000 7f463ce7a700 1 --2- 172.21.15.78:0/3208607015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4624004870 0x7f4624004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:37.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.920+0000 7f463ce7a700 1 --2- 172.21.15.78:0/3208607015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4624005220 0x7f462400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:37.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.920+0000 7f463ce7a700 1 -- 172.21.15.78:0/3208607015 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46380cb3a0 con 0x7f4624005220 2023-10-11T09:51:37.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.920+0000 7f463ce7a700 1 -- 172.21.15.78:0/3208607015 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462400c050 con 0x7f4624004870 2023-10-11T09:51:37.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.920+0000 7f463ce7a700 1 -- 172.21.15.78:0/3208607015 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f462400c210 con 0x7f4624003be0 2023-10-11T09:51:37.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.920+0000 7f463e67d700 1 --2- 172.21.15.78:0/3208607015 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4638164d20 0x7f463806bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:51:37.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.920+0000 7f463de7c700 1 --2- 172.21.15.78:0/3208607015 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4624003be0 0x7f4624003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:37.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.920+0000 7f463e67d700 1 --2- 172.21.15.78:0/3208607015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4624005220 0x7f462400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:37.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.920+0000 7f463d67b700 1 --2- 172.21.15.78:0/3208607015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4624004870 0x7f4624004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:37.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.920+0000 7f463de7c700 1 -- 172.21.15.78:0/3208607015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4624004870 msgr2=0x7f4624004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:37.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.920+0000 7f463de7c700 1 --2- 172.21.15.78:0/3208607015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4624004870 0x7f4624004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:37.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.920+0000 7f463de7c700 1 -- 172.21.15.78:0/3208607015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4624005220 msgr2=0x7f462400ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:37.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.920+0000 7f463de7c700 1 --2- 172.21.15.78:0/3208607015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4624005220 0x7f462400ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:37.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.920+0000 7f463de7c700 1 -- 172.21.15.78:0/3208607015 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46240035d0 con 0x7f4624003be0 2023-10-11T09:51:37.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.924+0000 7f463de7c700 1 --2- 172.21.15.78:0/3208607015 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4624003be0 0x7f4624003fd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4634001950 tx=0x7f46240058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f74c8aa7a717dc5e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:37.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.924+0000 7f463ce7a700 1 -- 172.21.15.78:0/3208607015 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4634001d80 con 0x7f4624003be0 2023-10-11T09:51:37.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.924+0000 7f463ce7a700 1 -- 172.21.15.78:0/3208607015 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46340052d0 con 0x7f4624003be0 2023-10-11T09:51:37.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.924+0000 7f463ce7a700 1 -- 172.21.15.78:0/3208607015 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4634001440 con 0x7f4624003be0 2023-10-11T09:51:37.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.924+0000 7f463ee7e700 1 -- 172.21.15.78:0/3208607015 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4624003be0 msgr2=0x7f4624003fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:37.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.924+0000 7f463ee7e700 1 --2- 172.21.15.78:0/3208607015 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4624003be0 0x7f4624003fd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4634001950 tx=0x7f46240058a0 comp rx=0 tx=0).stop 2023-10-11T09:51:37.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.924+0000 7f463e67d700 1 -- 172.21.15.78:0/3208607015 reap_dead start 2023-10-11T09:51:37.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.924+0000 7f463ee7e700 1 -- 172.21.15.78:0/3208607015 shutdown_connections 2023-10-11T09:51:37.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.924+0000 7f463ee7e700 1 -- 172.21.15.78:0/3208607015 >> 172.21.15.78:0/3208607015 conn(0x7f46380cea60 msgr2=0x7f46380cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:37.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.924+0000 7f463ee7e700 1 -- 172.21.15.78:0/3208607015 shutdown_connections 2023-10-11T09:51:37.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.924+0000 7f463ee7e700 1 -- 172.21.15.78:0/3208607015 wait complete. 2023-10-11T09:51:37.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.928+0000 7f463ee7e700 1 Processor -- start 2023-10-11T09:51:37.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.928+0000 7f463ee7e700 1 -- start start 2023-10-11T09:51:37.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.928+0000 7f463ee7e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4624003be0 0x7f46380b0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:37.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.928+0000 7f463ee7e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f463815ec20 0x7f463815f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:37.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.928+0000 7f463ee7e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4638148970 0x7f4638148e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:37.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.928+0000 7f463ee7e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46380b08b0 con 0x7f4638148970 2023-10-11T09:51:37.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.928+0000 7f463ee7e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463815f5d0 con 0x7f4624003be0 2023-10-11T09:51:37.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.928+0000 7f463ee7e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46380ae6e0 con 0x7f463815ec20 2023-10-11T09:51:37.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.928+0000 7f463e67d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4638148970 0x7f4638148e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:37.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.928+0000 7f463e67d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4638148970 0x7f4638148e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43334/0 (socket says 172.21.15.78:43334) 2023-10-11T09:51:37.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.928+0000 7f463e67d700 1 -- 172.21.15.78:0/1740032109 learned_addr learned my addr 172.21.15.78:0/1740032109 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:37.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.928+0000 7f463d67b700 1 --2- 172.21.15.78:0/1740032109 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f463815ec20 0x7f463815f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:37.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.928+0000 7f463de7c700 1 --2- 172.21.15.78:0/1740032109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4624003be0 0x7f46380b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:37.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.932+0000 7f463e67d700 1 -- 172.21.15.78:0/1740032109 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f463815ec20 msgr2=0x7f463815f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:37.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.932+0000 7f463e67d700 1 --2- 172.21.15.78:0/1740032109 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f463815ec20 0x7f463815f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:37.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.932+0000 7f463e67d700 1 -- 172.21.15.78:0/1740032109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4624003be0 msgr2=0x7f46380b0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:37.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.932+0000 7f463e67d700 1 --2- 172.21.15.78:0/1740032109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4624003be0 0x7f46380b0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:37.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.932+0000 7f463e67d700 1 -- 172.21.15.78:0/1740032109 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46380ae9c0 con 0x7f4638148970 2023-10-11T09:51:37.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.932+0000 7f463d67b700 1 --2- 172.21.15.78:0/1740032109 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f463815ec20 0x7f463815f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:51:37.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.932+0000 7f463e67d700 1 --2- 172.21.15.78:0/1740032109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4638148970 0x7f4638148e10 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f46381650f0 tx=0x7f463000f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:37.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.932+0000 7f462e7fc700 1 -- 172.21.15.78:0/1740032109 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463000e040 con 0x7f4638148970 2023-10-11T09:51:37.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.932+0000 7f462e7fc700 1 -- 172.21.15.78:0/1740032109 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f463000f040 con 0x7f4638148970 2023-10-11T09:51:37.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.932+0000 7f462e7fc700 1 -- 172.21.15.78:0/1740032109 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4630009410 con 0x7f4638148970 2023-10-11T09:51:37.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.932+0000 7f463ee7e700 1 -- 172.21.15.78:0/1740032109 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46380aeca0 con 0x7f4638148970 2023-10-11T09:51:37.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.932+0000 7f463ee7e700 1 -- 172.21.15.78:0/1740032109 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46380af4a0 con 0x7f4638148970 2023-10-11T09:51:37.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.932+0000 7f463ee7e700 1 -- 172.21.15.78:0/1740032109 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46380af1c0 con 0x7f4638148970 2023-10-11T09:51:37.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.936+0000 7f462e7fc700 1 -- 172.21.15.78:0/1740032109 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f4630015670 con 0x7f4638148970 2023-10-11T09:51:37.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.936+0000 7f462e7fc700 1 --2- 172.21.15.78:0/1740032109 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f461c065f00 0x7f461c0683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:37.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.936+0000 7f462e7fc700 1 -- 172.21.15.78:0/1740032109 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f4630083c90 con 0x7f4638148970 2023-10-11T09:51:37.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.936+0000 7f463de7c700 1 --2- 172.21.15.78:0/1740032109 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f461c065f00 0x7f461c0683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:37.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.940+0000 7f463de7c700 1 --2- 172.21.15.78:0/1740032109 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f461c065f00 0x7f461c0683b0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f4634015040 tx=0x7f4634009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:37.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:37.944+0000 7f462e7fc700 1 -- 172.21.15.78:0/1740032109 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f46380af1c0 con 0x7f4638148970 2023-10-11T09:51:38.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.116+0000 7f463ee7e700 1 -- 172.21.15.78:0/1740032109 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f46380af1c0 con 0x7f4638148970 2023-10-11T09:51:38.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.120+0000 7f462e7fc700 1 -- 172.21.15.78:0/1740032109 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v123) v1 ==== 84+0+126 (secure 0 0 0) 0x7f46380af1c0 con 0x7f4638148970 2023-10-11T09:51:38.130 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:38.130 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2023-10-11T09:51:38.130 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:51:38.131 INFO:teuthology.orchestra.run.smithi078.stdout: "poolnum": 1, 2023-10-11T09:51:38.131 INFO:teuthology.orchestra.run.smithi078.stdout: "poolname": ".mgr" 2023-10-11T09:51:38.131 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:51:38.131 INFO:teuthology.orchestra.run.smithi078.stdout: { 2023-10-11T09:51:38.131 INFO:teuthology.orchestra.run.smithi078.stdout: "poolnum": 2, 2023-10-11T09:51:38.132 INFO:teuthology.orchestra.run.smithi078.stdout: "poolname": "rbd" 2023-10-11T09:51:38.132 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:51:38.132 INFO:teuthology.orchestra.run.smithi078.stdout:] 2023-10-11T09:51:38.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.120+0000 7f463ee7e700 1 -- 172.21.15.78:0/1740032109 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f461c065f00 msgr2=0x7f461c0683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:38.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.120+0000 7f463ee7e700 1 --2- 172.21.15.78:0/1740032109 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f461c065f00 0x7f461c0683b0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f4634015040 tx=0x7f4634009030 comp rx=0 tx=0).stop 2023-10-11T09:51:38.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.120+0000 7f463ee7e700 1 -- 172.21.15.78:0/1740032109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4638148970 msgr2=0x7f4638148e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:38.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.120+0000 7f463ee7e700 1 --2- 172.21.15.78:0/1740032109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4638148970 0x7f4638148e10 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f46381650f0 tx=0x7f463000f770 comp rx=0 tx=0).stop 2023-10-11T09:51:38.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.124+0000 7f463ee7e700 1 -- 172.21.15.78:0/1740032109 shutdown_connections 2023-10-11T09:51:38.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.124+0000 7f463ee7e700 1 --2- 172.21.15.78:0/1740032109 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f461c065f00 0x7f461c0683b0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:38.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.124+0000 7f463ee7e700 1 --2- 172.21.15.78:0/1740032109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4638148970 0x7f4638148e10 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:38.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.124+0000 7f463ee7e700 1 --2- 172.21.15.78:0/1740032109 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f463815ec20 0x7f463815f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:38.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.124+0000 7f463ee7e700 1 --2- 172.21.15.78:0/1740032109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4624003be0 0x7f46380b0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:38.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.124+0000 7f463ee7e700 1 -- 172.21.15.78:0/1740032109 >> 172.21.15.78:0/1740032109 conn(0x7f46380cea60 msgr2=0x7f46380cb4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:38.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.124+0000 7f463ee7e700 1 -- 172.21.15.78:0/1740032109 shutdown_connections 2023-10-11T09:51:38.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.124+0000 7f463ee7e700 1 -- 172.21.15.78:0/1740032109 wait complete. 2023-10-11T09:51:38.152 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-11T09:51:38.153 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-11T09:51:38.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.320+0000 7f797a8db700 1 Processor -- start 2023-10-11T09:51:38.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.320+0000 7f797a8db700 1 -- start start 2023-10-11T09:51:38.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.320+0000 7f797a8db700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f797415a7b0 0x7f79740cb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:38.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.320+0000 7f797a8db700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f797406c840 0x7f7974071810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:38.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.324+0000 7f797a8db700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79740cbba0 con 0x7f7974071d50 2023-10-11T09:51:38.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.324+0000 7f797a8db700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79740cb9b0 con 0x7f797415a7b0 2023-10-11T09:51:38.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.324+0000 7f797a8db700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79740cbe80 con 0x7f797406c840 2023-10-11T09:51:38.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.324+0000 7f79798d9700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f797415a7b0 0x7f79740cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:38.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.324+0000 7f79790d8700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f797406c840 0x7f7974071810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:38.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.324+0000 7f797a0da700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f7974071d50 0x7f79740b1a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:43678/0 (socket says 172.21.15.78:43678) 2023-10-11T09:51:38.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.324+0000 7f797a0da700 1 -- 172.21.15.78:0/1627288553 learned_addr learned my addr 172.21.15.78:0/1627288553 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:51:38.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.324+0000 7f79790d8700 1 -- 172.21.15.78:0/1627288553 >> v2:172.21.15.78:3300/0 conn(0x7f797415a7b0 msgr2=0x7f79740cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:38.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.324+0000 7f79790d8700 1 --2- 172.21.15.78:0/1627288553 >> v2:172.21.15.78:3300/0 conn(0x7f797415a7b0 0x7f79740cb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:38.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.324+0000 7f79788d7700 1 -- 172.21.15.78:0/1627288553 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1354587342 0 0) 0x7f79740cbba0 con 0x7f7974071d50 2023-10-11T09:51:38.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.324+0000 7f79790d8700 1 -- 172.21.15.78:0/1627288553 >> v1:172.21.15.133:6789/0 conn(0x7f7974071d50 legacy=0x7f79740b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:38.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.324+0000 7f79790d8700 1 -- 172.21.15.78:0/1627288553 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7974071010 con 0x7f797406c840 2023-10-11T09:51:38.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.324+0000 7f79798d9700 1 --2- 172.21.15.78:0/1627288553 >> v2:172.21.15.78:3300/0 conn(0x7f797415a7b0 0x7f79740cb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:51:38.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.324+0000 7f79790d8700 1 --2- 172.21.15.78:0/1627288553 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f797406c840 0x7f7974071810 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7964009850 tx=0x7f7964009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=465d1fcf7e33af41 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:38.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.324+0000 7f79788d7700 1 -- 172.21.15.78:0/1627288553 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7964011830 con 0x7f797406c840 2023-10-11T09:51:38.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.324+0000 7f79788d7700 1 -- 172.21.15.78:0/1627288553 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7964011990 con 0x7f797406c840 2023-10-11T09:51:38.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.324+0000 7f79788d7700 1 -- 172.21.15.78:0/1627288553 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7964011d40 con 0x7f797406c840 2023-10-11T09:51:38.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.324+0000 7f797a8db700 1 -- 172.21.15.78:0/1627288553 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f797406c840 msgr2=0x7f7974071810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:38.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.324+0000 7f797a8db700 1 --2- 172.21.15.78:0/1627288553 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f797406c840 0x7f7974071810 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7964009850 tx=0x7f7964009b60 comp rx=0 tx=0).stop 2023-10-11T09:51:38.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.328+0000 7f797a8db700 1 -- 172.21.15.78:0/1627288553 shutdown_connections 2023-10-11T09:51:38.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.328+0000 7f797a8db700 1 --2- 172.21.15.78:0/1627288553 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f797406c840 0x7f7974071810 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:38.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.328+0000 7f797a8db700 1 --2- 172.21.15.78:0/1627288553 >> v2:172.21.15.78:3300/0 conn(0x7f797415a7b0 0x7f79740cb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:38.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.328+0000 7f797a8db700 1 -- 172.21.15.78:0/1627288553 >> 172.21.15.78:0/1627288553 conn(0x7f797406acf0 msgr2=0x7f79740b0710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:38.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.328+0000 7f797a8db700 1 -- 172.21.15.78:0/1627288553 shutdown_connections 2023-10-11T09:51:38.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.328+0000 7f797a8db700 1 -- 172.21.15.78:0/1627288553 wait complete. 2023-10-11T09:51:38.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.328+0000 7f797a8db700 1 Processor -- start 2023-10-11T09:51:38.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.332+0000 7f797a8db700 1 -- start start 2023-10-11T09:51:38.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.332+0000 7f797a8db700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f797406c840 0x7f7974156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:38.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.332+0000 7f797a8db700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7974071d50 0x7f79741f5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:38.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.332+0000 7f797a8db700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f797415a7b0 0x7f7974155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:38.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.332+0000 7f797a8db700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79741f5cb0 con 0x7f797415a7b0 2023-10-11T09:51:38.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.332+0000 7f797a8db700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7974155a70 con 0x7f797406c840 2023-10-11T09:51:38.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.332+0000 7f797a8db700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79740cc6f0 con 0x7f7974071d50 2023-10-11T09:51:38.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.332+0000 7f797a0da700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f797415a7b0 0x7f7974155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:38.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.332+0000 7f797a0da700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f797415a7b0 0x7f7974155530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43348/0 (socket says 172.21.15.78:43348) 2023-10-11T09:51:38.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.332+0000 7f797a0da700 1 -- 172.21.15.78:0/2323997919 learned_addr learned my addr 172.21.15.78:0/2323997919 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:38.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.332+0000 7f79798d9700 1 --2- 172.21.15.78:0/2323997919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f797406c840 0x7f7974156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:38.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.332+0000 7f79790d8700 1 --2- 172.21.15.78:0/2323997919 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7974071d50 0x7f79741f5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:38.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.332+0000 7f797a0da700 1 -- 172.21.15.78:0/2323997919 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7974071d50 msgr2=0x7f79741f5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:38.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.332+0000 7f797a0da700 1 --2- 172.21.15.78:0/2323997919 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7974071d50 0x7f79741f5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:38.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.332+0000 7f797a0da700 1 -- 172.21.15.78:0/2323997919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f797406c840 msgr2=0x7f7974156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:38.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.332+0000 7f797a0da700 1 --2- 172.21.15.78:0/2323997919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f797406c840 0x7f7974156680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:38.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.332+0000 7f797a0da700 1 -- 172.21.15.78:0/2323997919 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79700094b0 con 0x7f797415a7b0 2023-10-11T09:51:38.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.336+0000 7f797a0da700 1 --2- 172.21.15.78:0/2323997919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f797415a7b0 0x7f7974155530 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f796c00cca0 tx=0x7f796c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:38.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.336+0000 7f796a7fc700 1 -- 172.21.15.78:0/2323997919 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796c00ab70 con 0x7f797415a7b0 2023-10-11T09:51:38.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.336+0000 7f796a7fc700 1 -- 172.21.15.78:0/2323997919 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796c00acd0 con 0x7f797415a7b0 2023-10-11T09:51:38.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.336+0000 7f796a7fc700 1 -- 172.21.15.78:0/2323997919 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796c007310 con 0x7f797415a7b0 2023-10-11T09:51:38.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.336+0000 7f797a8db700 1 -- 172.21.15.78:0/2323997919 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79640094b0 con 0x7f797415a7b0 2023-10-11T09:51:38.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.336+0000 7f797a8db700 1 -- 172.21.15.78:0/2323997919 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79740cc9d0 con 0x7f797415a7b0 2023-10-11T09:51:38.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.336+0000 7f797a8db700 1 -- 172.21.15.78:0/2323997919 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79740ccb60 con 0x7f797415a7b0 2023-10-11T09:51:38.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.340+0000 7f796a7fc700 1 -- 172.21.15.78:0/2323997919 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f796c007470 con 0x7f797415a7b0 2023-10-11T09:51:38.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.340+0000 7f796a7fc700 1 --2- 172.21.15.78:0/2323997919 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7958065f00 0x7f79580683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:38.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.340+0000 7f796a7fc700 1 -- 172.21.15.78:0/2323997919 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f796c082ed0 con 0x7f797415a7b0 2023-10-11T09:51:38.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.340+0000 7f79798d9700 1 --2- 172.21.15.78:0/2323997919 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7958065f00 0x7f79580683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:38.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.340+0000 7f79798d9700 1 --2- 172.21.15.78:0/2323997919 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7958065f00 0x7f79580683b0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f7970009d80 tx=0x7f7970009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:38.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.348+0000 7f796a7fc700 1 -- 172.21.15.78:0/2323997919 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f796c050880 con 0x7f797415a7b0 2023-10-11T09:51:38.531 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.520+0000 7f797a8db700 1 -- 172.21.15.78:0/2323997919 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7974065db0 con 0x7f797415a7b0 2023-10-11T09:51:38.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.728+0000 7f796a7fc700 1 -- 172.21.15.78:0/2323997919 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v124) v1 ==== 138+0+0 (secure 0 0 0) 0x7f796c053ea0 con 0x7f797415a7b0 2023-10-11T09:51:38.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.736+0000 7f797a8db700 1 -- 172.21.15.78:0/2323997919 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7958065f00 msgr2=0x7f79580683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:38.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.736+0000 7f797a8db700 1 --2- 172.21.15.78:0/2323997919 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7958065f00 0x7f79580683b0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f7970009d80 tx=0x7f7970009240 comp rx=0 tx=0).stop 2023-10-11T09:51:38.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.736+0000 7f797a8db700 1 -- 172.21.15.78:0/2323997919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f797415a7b0 msgr2=0x7f7974155530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:38.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.736+0000 7f797a8db700 1 --2- 172.21.15.78:0/2323997919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f797415a7b0 0x7f7974155530 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f796c00cca0 tx=0x7f796c00a060 comp rx=0 tx=0).stop 2023-10-11T09:51:38.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.736+0000 7f797a8db700 1 -- 172.21.15.78:0/2323997919 shutdown_connections 2023-10-11T09:51:38.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.736+0000 7f797a8db700 1 --2- 172.21.15.78:0/2323997919 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7958065f00 0x7f79580683b0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:38.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.736+0000 7f797a8db700 1 --2- 172.21.15.78:0/2323997919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f797415a7b0 0x7f7974155530 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:38.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.736+0000 7f797a8db700 1 --2- 172.21.15.78:0/2323997919 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7974071d50 0x7f79741f5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:38.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.736+0000 7f797a8db700 1 --2- 172.21.15.78:0/2323997919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f797406c840 0x7f7974156680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:38.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.736+0000 7f797a8db700 1 -- 172.21.15.78:0/2323997919 >> 172.21.15.78:0/2323997919 conn(0x7f797406acf0 msgr2=0x7f79741597e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:38.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.736+0000 7f797a8db700 1 -- 172.21.15.78:0/2323997919 shutdown_connections 2023-10-11T09:51:38.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.736+0000 7f797a8db700 1 -- 172.21.15.78:0/2323997919 wait complete. 2023-10-11T09:51:38.750 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' created 2023-10-11T09:51:38.767 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-11T09:51:38.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.936+0000 7ff975682700 1 Processor -- start 2023-10-11T09:51:38.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.936+0000 7ff975682700 1 -- start start 2023-10-11T09:51:38.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.936+0000 7ff975682700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ff9700cf8d0 0x7ff9700b4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:38.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.936+0000 7ff975682700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff9700cfca0 0x7ff9700cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:38.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.936+0000 7ff975682700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9700b4fd0 con 0x7ff970160870 2023-10-11T09:51:38.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.940+0000 7ff975682700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9700b5870 con 0x7ff9700cf8d0 2023-10-11T09:51:38.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.940+0000 7ff975682700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9700b52b0 con 0x7ff9700cfca0 2023-10-11T09:51:38.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.940+0000 7ff96f7fe700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7ff970160870 0x7ff97015cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:46264/0 (socket says 172.21.15.78:46264) 2023-10-11T09:51:38.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.940+0000 7ff96f7fe700 1 -- 172.21.15.78:0/1438404482 learned_addr learned my addr 172.21.15.78:0/1438404482 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:51:38.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.940+0000 7ff96effd700 1 --2- 172.21.15.78:0/1438404482 >> v2:172.21.15.78:3300/0 conn(0x7ff9700cf8d0 0x7ff9700b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:38.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.940+0000 7ff96e7fc700 1 --2- 172.21.15.78:0/1438404482 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff9700cfca0 0x7ff9700cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:38.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.940+0000 7ff96dffb700 1 -- 172.21.15.78:0/1438404482 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1907497838 0 0) 0x7ff9700b4fd0 con 0x7ff970160870 2023-10-11T09:51:38.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.940+0000 7ff96dffb700 1 -- 172.21.15.78:0/1438404482 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9580035d0 con 0x7ff970160870 2023-10-11T09:51:38.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.940+0000 7ff96effd700 1 -- 172.21.15.78:0/1438404482 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff9700cfca0 msgr2=0x7ff9700cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:38.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.940+0000 7ff96effd700 1 --2- 172.21.15.78:0/1438404482 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff9700cfca0 0x7ff9700cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:38.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.940+0000 7ff96effd700 1 -- 172.21.15.78:0/1438404482 >> v1:172.21.15.78:6789/0 conn(0x7ff970160870 legacy=0x7ff97015cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:38.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.940+0000 7ff96effd700 1 -- 172.21.15.78:0/1438404482 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9700b5590 con 0x7ff9700cf8d0 2023-10-11T09:51:38.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.940+0000 7ff96effd700 1 --2- 172.21.15.78:0/1438404482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9700cf8d0 0x7ff9700b4940 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7ff960009850 tx=0x7ff960009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e67426640ea243 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:38.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.940+0000 7ff96dffb700 1 -- 172.21.15.78:0/1438404482 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff960011830 con 0x7ff9700cf8d0 2023-10-11T09:51:38.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.940+0000 7ff96dffb700 1 -- 172.21.15.78:0/1438404482 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff960011990 con 0x7ff9700cf8d0 2023-10-11T09:51:38.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.940+0000 7ff96dffb700 1 -- 172.21.15.78:0/1438404482 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff960011d00 con 0x7ff9700cf8d0 2023-10-11T09:51:38.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.940+0000 7ff975682700 1 -- 172.21.15.78:0/1438404482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9700cf8d0 msgr2=0x7ff9700b4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:38.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.940+0000 7ff975682700 1 --2- 172.21.15.78:0/1438404482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9700cf8d0 0x7ff9700b4940 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7ff960009850 tx=0x7ff960009b60 comp rx=0 tx=0).stop 2023-10-11T09:51:38.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.940+0000 7ff975682700 1 -- 172.21.15.78:0/1438404482 shutdown_connections 2023-10-11T09:51:38.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.940+0000 7ff975682700 1 --2- 172.21.15.78:0/1438404482 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff9700cfca0 0x7ff9700cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:38.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.940+0000 7ff975682700 1 --2- 172.21.15.78:0/1438404482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9700cf8d0 0x7ff9700b4940 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:38.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.940+0000 7ff975682700 1 -- 172.21.15.78:0/1438404482 >> 172.21.15.78:0/1438404482 conn(0x7ff97006b010 msgr2=0x7ff97006b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:38.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.940+0000 7ff975682700 1 -- 172.21.15.78:0/1438404482 shutdown_connections 2023-10-11T09:51:38.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.940+0000 7ff975682700 1 -- 172.21.15.78:0/1438404482 wait complete. 2023-10-11T09:51:38.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.944+0000 7ff975682700 1 Processor -- start 2023-10-11T09:51:38.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.944+0000 7ff975682700 1 -- start start 2023-10-11T09:51:38.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.948+0000 7ff975682700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff9700cf8d0 0x7ff97015f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:38.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.948+0000 7ff975682700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9700cfca0 0x7ff970154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:38.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.948+0000 7ff975682700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff970160870 0x7ff970141260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:38.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.948+0000 7ff975682700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9701551c0 con 0x7ff9700cfca0 2023-10-11T09:51:38.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.948+0000 7ff975682700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9701e5d10 con 0x7ff970160870 2023-10-11T09:51:38.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.948+0000 7ff975682700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9701e5ff0 con 0x7ff9700cf8d0 2023-10-11T09:51:38.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.948+0000 7ff96e7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9700cfca0 0x7ff970154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:38.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.948+0000 7ff96e7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9700cfca0 0x7ff970154c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43370/0 (socket says 172.21.15.78:43370) 2023-10-11T09:51:38.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.948+0000 7ff96e7fc700 1 -- 172.21.15.78:0/2722884653 learned_addr learned my addr 172.21.15.78:0/2722884653 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:38.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.948+0000 7ff96effd700 1 --2- 172.21.15.78:0/2722884653 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff9700cf8d0 0x7ff97015f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:38.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.948+0000 7ff96f7fe700 1 --2- 172.21.15.78:0/2722884653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff970160870 0x7ff970141260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:38.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.948+0000 7ff96e7fc700 1 -- 172.21.15.78:0/2722884653 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff9700cf8d0 msgr2=0x7ff97015f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:38.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.948+0000 7ff96e7fc700 1 --2- 172.21.15.78:0/2722884653 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff9700cf8d0 0x7ff97015f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:38.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.948+0000 7ff96e7fc700 1 -- 172.21.15.78:0/2722884653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff970160870 msgr2=0x7ff970141260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:38.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.948+0000 7ff96e7fc700 1 --2- 172.21.15.78:0/2722884653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff970160870 0x7ff970141260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:38.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.948+0000 7ff96e7fc700 1 -- 172.21.15.78:0/2722884653 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9600094b0 con 0x7ff9700cfca0 2023-10-11T09:51:38.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.948+0000 7ff96f7fe700 1 --2- 172.21.15.78:0/2722884653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff970160870 0x7ff970141260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:51:38.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.948+0000 7ff96e7fc700 1 --2- 172.21.15.78:0/2722884653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9700cfca0 0x7ff970154c80 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7ff96400e840 tx=0x7ff96400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:38.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.948+0000 7ff9577fe700 1 -- 172.21.15.78:0/2722884653 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9640095a0 con 0x7ff9700cfca0 2023-10-11T09:51:38.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.948+0000 7ff9577fe700 1 -- 172.21.15.78:0/2722884653 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff96400c690 con 0x7ff9700cfca0 2023-10-11T09:51:38.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.948+0000 7ff975682700 1 -- 172.21.15.78:0/2722884653 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff97015e550 con 0x7ff9700cfca0 2023-10-11T09:51:38.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.948+0000 7ff9577fe700 1 -- 172.21.15.78:0/2722884653 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96400ca00 con 0x7ff9700cfca0 2023-10-11T09:51:38.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.948+0000 7ff975682700 1 -- 172.21.15.78:0/2722884653 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff97015eb10 con 0x7ff9700cfca0 2023-10-11T09:51:38.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.952+0000 7ff975682700 1 -- 172.21.15.78:0/2722884653 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff970160350 con 0x7ff9700cfca0 2023-10-11T09:51:38.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.952+0000 7ff9577fe700 1 -- 172.21.15.78:0/2722884653 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7ff96400cb60 con 0x7ff9700cfca0 2023-10-11T09:51:38.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.956+0000 7ff9577fe700 1 --2- 172.21.15.78:0/2722884653 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff95c065ed0 0x7ff95c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:38.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.956+0000 7ff9577fe700 1 -- 172.21.15.78:0/2722884653 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5580+0+0 (secure 0 0 0) 0x7ff964083a00 con 0x7ff9700cfca0 2023-10-11T09:51:38.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.956+0000 7ff96effd700 1 --2- 172.21.15.78:0/2722884653 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff95c065ed0 0x7ff95c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:38.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.956+0000 7ff96effd700 1 --2- 172.21.15.78:0/2722884653 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff95c065ed0 0x7ff95c068380 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff9600090b0 tx=0x7ff960008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:38.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:38.960+0000 7ff9577fe700 1 -- 172.21.15.78:0/2722884653 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7ff970160350 con 0x7ff9700cfca0 2023-10-11T09:51:39.145 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.136+0000 7ff975682700 1 -- 172.21.15.78:0/2722884653 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff970160350 con 0x7ff9700cfca0 2023-10-11T09:51:39.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.736+0000 7ff9577fe700 1 -- 172.21.15.78:0/2722884653 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v125) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff970160350 con 0x7ff9700cfca0 2023-10-11T09:51:39.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.748+0000 7ff975682700 1 -- 172.21.15.78:0/2722884653 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff95c065ed0 msgr2=0x7ff95c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:39.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.748+0000 7ff975682700 1 --2- 172.21.15.78:0/2722884653 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff95c065ed0 0x7ff95c068380 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff9600090b0 tx=0x7ff960008030 comp rx=0 tx=0).stop 2023-10-11T09:51:39.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.748+0000 7ff975682700 1 -- 172.21.15.78:0/2722884653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9700cfca0 msgr2=0x7ff970154c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:39.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.748+0000 7ff975682700 1 --2- 172.21.15.78:0/2722884653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9700cfca0 0x7ff970154c80 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7ff96400e840 tx=0x7ff96400ec00 comp rx=0 tx=0).stop 2023-10-11T09:51:39.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.752+0000 7ff975682700 1 -- 172.21.15.78:0/2722884653 shutdown_connections 2023-10-11T09:51:39.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.752+0000 7ff975682700 1 --2- 172.21.15.78:0/2722884653 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff95c065ed0 0x7ff95c068380 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:39.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.752+0000 7ff975682700 1 --2- 172.21.15.78:0/2722884653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff970160870 0x7ff970141260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:39.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.752+0000 7ff975682700 1 --2- 172.21.15.78:0/2722884653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9700cfca0 0x7ff970154c80 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:39.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.752+0000 7ff975682700 1 --2- 172.21.15.78:0/2722884653 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff9700cf8d0 0x7ff97015f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:39.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.752+0000 7ff975682700 1 -- 172.21.15.78:0/2722884653 >> 172.21.15.78:0/2722884653 conn(0x7ff97006b010 msgr2=0x7ff9700b5cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:39.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.752+0000 7ff975682700 1 -- 172.21.15.78:0/2722884653 shutdown_connections 2023-10-11T09:51:39.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.752+0000 7ff975682700 1 -- 172.21.15.78:0/2722884653 wait complete. 2023-10-11T09:51:39.764 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' created 2023-10-11T09:51:39.786 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-11T09:51:39.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.956+0000 7f6296929700 1 Processor -- start 2023-10-11T09:51:39.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.956+0000 7f6296929700 1 -- start start 2023-10-11T09:51:39.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.960+0000 7f6296929700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62900b3240 0x7f62900d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:39.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.960+0000 7f6296929700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f62900b3610 0x7f62900b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:39.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.960+0000 7f6296929700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f629015d040 0x7f629015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:39.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.960+0000 7f6296929700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62900b3fd0 con 0x7f629015d040 2023-10-11T09:51:39.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.960+0000 7f6296929700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62900b47f0 con 0x7f62900b3610 2023-10-11T09:51:39.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.960+0000 7f6296929700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62900b8130 con 0x7f62900b3240 2023-10-11T09:51:39.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.960+0000 7f6296128700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f629015d040 0x7f629015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:39.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.960+0000 7f6296128700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f629015d040 0x7f629015d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43376/0 (socket says 172.21.15.78:43376) 2023-10-11T09:51:39.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.960+0000 7f6296128700 1 -- 172.21.15.78:0/2534329473 learned_addr learned my addr 172.21.15.78:0/2534329473 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:39.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.960+0000 7f6295126700 1 --2- 172.21.15.78:0/2534329473 >> v2:172.21.15.133:3300/0 conn(0x7f62900b3610 0x7f62900b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:39.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.960+0000 7f6295927700 1 --2- 172.21.15.78:0/2534329473 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62900b3240 0x7f62900d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:39.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.960+0000 7f6296128700 1 -- 172.21.15.78:0/2534329473 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62900b3240 msgr2=0x7f62900d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:39.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.960+0000 7f6296128700 1 --2- 172.21.15.78:0/2534329473 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62900b3240 0x7f62900d29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:39.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.960+0000 7f6296128700 1 -- 172.21.15.78:0/2534329473 >> v2:172.21.15.133:3300/0 conn(0x7f62900b3610 msgr2=0x7f62900b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:39.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.960+0000 7f6296128700 1 --2- 172.21.15.78:0/2534329473 >> v2:172.21.15.133:3300/0 conn(0x7f62900b3610 0x7f62900b3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:39.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.960+0000 7f6296128700 1 -- 172.21.15.78:0/2534329473 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62900b41e0 con 0x7f629015d040 2023-10-11T09:51:39.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.960+0000 7f6295126700 1 --2- 172.21.15.78:0/2534329473 >> v2:172.21.15.133:3300/0 conn(0x7f62900b3610 0x7f62900b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:51:39.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.960+0000 7f6296128700 1 --2- 172.21.15.78:0/2534329473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f629015d040 0x7f629015d4c0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f629008bb30 tx=0x7f628800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=350c6dff31e24412 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:39.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.960+0000 7f6294925700 1 -- 172.21.15.78:0/2534329473 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628800e070 con 0x7f629015d040 2023-10-11T09:51:39.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.960+0000 7f6294925700 1 -- 172.21.15.78:0/2534329473 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628800b9c0 con 0x7f629015d040 2023-10-11T09:51:39.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.960+0000 7f6294925700 1 -- 172.21.15.78:0/2534329473 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628800bd70 con 0x7f629015d040 2023-10-11T09:51:39.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.964+0000 7f6296929700 1 -- 172.21.15.78:0/2534329473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f629015d040 msgr2=0x7f629015d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:39.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.964+0000 7f6296929700 1 --2- 172.21.15.78:0/2534329473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f629015d040 0x7f629015d4c0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f629008bb30 tx=0x7f628800b330 comp rx=0 tx=0).stop 2023-10-11T09:51:39.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.964+0000 7f6296929700 1 -- 172.21.15.78:0/2534329473 shutdown_connections 2023-10-11T09:51:39.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.964+0000 7f6296929700 1 --2- 172.21.15.78:0/2534329473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f629015d040 0x7f629015d4c0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:39.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.964+0000 7f6296929700 1 --2- 172.21.15.78:0/2534329473 >> v2:172.21.15.133:3300/0 conn(0x7f62900b3610 0x7f62900b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:39.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.964+0000 7f6296929700 1 --2- 172.21.15.78:0/2534329473 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62900b3240 0x7f62900d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:39.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.964+0000 7f6296929700 1 -- 172.21.15.78:0/2534329473 >> 172.21.15.78:0/2534329473 conn(0x7f629009c260 msgr2=0x7f62900d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:39.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.964+0000 7f6296929700 1 -- 172.21.15.78:0/2534329473 shutdown_connections 2023-10-11T09:51:39.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.964+0000 7f6296929700 1 -- 172.21.15.78:0/2534329473 wait complete. 2023-10-11T09:51:39.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.968+0000 7f6296929700 1 Processor -- start 2023-10-11T09:51:39.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.968+0000 7f6296929700 1 -- start start 2023-10-11T09:51:39.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.968+0000 7f6296929700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62900b3240 0x7f629015a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:39.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.968+0000 7f6296929700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62900b3610 0x7f62900b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:39.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.968+0000 7f6296929700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f629015d040 0x7f62900c7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:39.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.968+0000 7f6296929700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62900b1a50 con 0x7f62900b3610 2023-10-11T09:51:39.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.968+0000 7f6296929700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62900b09b0 con 0x7f62900b3240 2023-10-11T09:51:39.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.968+0000 7f6296929700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62900b0c90 con 0x7f629015d040 2023-10-11T09:51:39.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.968+0000 7f6295126700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62900b3610 0x7f62900b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:39.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.968+0000 7f6295126700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62900b3610 0x7f62900b22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43384/0 (socket says 172.21.15.78:43384) 2023-10-11T09:51:39.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.968+0000 7f6295126700 1 -- 172.21.15.78:0/989779146 learned_addr learned my addr 172.21.15.78:0/989779146 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:39.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.968+0000 7f6296128700 1 --2- 172.21.15.78:0/989779146 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f629015d040 0x7f62900c7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:39.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.968+0000 7f6295927700 1 --2- 172.21.15.78:0/989779146 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62900b3240 0x7f629015a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:39.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.968+0000 7f6295126700 1 -- 172.21.15.78:0/989779146 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f629015d040 msgr2=0x7f62900c7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:39.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.968+0000 7f6295126700 1 --2- 172.21.15.78:0/989779146 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f629015d040 0x7f62900c7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:39.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.968+0000 7f6295126700 1 -- 172.21.15.78:0/989779146 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62900b3240 msgr2=0x7f629015a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:39.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.968+0000 7f6295126700 1 --2- 172.21.15.78:0/989779146 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62900b3240 0x7f629015a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:40.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.968+0000 7f6295126700 1 -- 172.21.15.78:0/989779146 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6288009b30 con 0x7f62900b3610 2023-10-11T09:51:40.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.968+0000 7f6295126700 1 --2- 172.21.15.78:0/989779146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62900b3610 0x7f62900b22a0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f628000e7e0 tx=0x7f628000eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:40.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.968+0000 7f62867fc700 1 -- 172.21.15.78:0/989779146 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628000c020 con 0x7f62900b3610 2023-10-11T09:51:40.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.968+0000 7f62867fc700 1 -- 172.21.15.78:0/989779146 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6280004230 con 0x7f62900b3610 2023-10-11T09:51:40.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.968+0000 7f62867fc700 1 -- 172.21.15.78:0/989779146 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62800045c0 con 0x7f62900b3610 2023-10-11T09:51:40.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.968+0000 7f6296929700 1 -- 172.21.15.78:0/989779146 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62900b0f70 con 0x7f62900b3610 2023-10-11T09:51:40.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.972+0000 7f6296929700 1 -- 172.21.15.78:0/989779146 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62900b1530 con 0x7f62900b3610 2023-10-11T09:51:40.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.972+0000 7f6296929700 1 -- 172.21.15.78:0/989779146 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62900b16c0 con 0x7f62900b3610 2023-10-11T09:51:40.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.972+0000 7f62867fc700 1 -- 172.21.15.78:0/989779146 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f62800036c0 con 0x7f62900b3610 2023-10-11T09:51:40.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.976+0000 7f62867fc700 1 --2- 172.21.15.78:0/989779146 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6274065ed0 0x7f6274068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:40.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.976+0000 7f62867fc700 1 -- 172.21.15.78:0/989779146 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f62800835d0 con 0x7f62900b3610 2023-10-11T09:51:40.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.976+0000 7f6295927700 1 --2- 172.21.15.78:0/989779146 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6274065ed0 0x7f6274068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:40.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.976+0000 7f6295927700 1 --2- 172.21.15.78:0/989779146 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6274065ed0 0x7f6274068380 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f628c00ba30 tx=0x7f628c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:40.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:39.980+0000 7f62867fc700 1 -- 172.21.15.78:0/989779146 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f6280050bf0 con 0x7f62900b3610 2023-10-11T09:51:40.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.204+0000 7f6296929700 1 -- 172.21.15.78:0/989779146 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f62900d0990 con 0x7f62900b3610 2023-10-11T09:51:40.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.216+0000 7f62867fc700 1 -- 172.21.15.78:0/989779146 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v95) v1 ==== 174+0+0 (secure 0 0 0) 0x7f6280054210 con 0x7f62900b3610 2023-10-11T09:51:40.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.228+0000 7f6296929700 1 -- 172.21.15.78:0/989779146 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6274065ed0 msgr2=0x7f6274068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:40.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.228+0000 7f6296929700 1 --2- 172.21.15.78:0/989779146 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6274065ed0 0x7f6274068380 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f628c00ba30 tx=0x7f628c008040 comp rx=0 tx=0).stop 2023-10-11T09:51:40.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.228+0000 7f6296929700 1 -- 172.21.15.78:0/989779146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62900b3610 msgr2=0x7f62900b22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:40.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.228+0000 7f6296929700 1 --2- 172.21.15.78:0/989779146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62900b3610 0x7f62900b22a0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f628000e7e0 tx=0x7f628000eba0 comp rx=0 tx=0).stop 2023-10-11T09:51:40.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.228+0000 7f6296929700 1 -- 172.21.15.78:0/989779146 shutdown_connections 2023-10-11T09:51:40.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.228+0000 7f6296929700 1 --2- 172.21.15.78:0/989779146 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6274065ed0 0x7f6274068380 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:40.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.228+0000 7f6296929700 1 --2- 172.21.15.78:0/989779146 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f629015d040 0x7f62900c7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:40.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.228+0000 7f6296929700 1 --2- 172.21.15.78:0/989779146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62900b3610 0x7f62900b22a0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:40.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.228+0000 7f6296929700 1 --2- 172.21.15.78:0/989779146 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62900b3240 0x7f629015a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:40.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.228+0000 7f6296929700 1 -- 172.21.15.78:0/989779146 >> 172.21.15.78:0/989779146 conn(0x7f629009c260 msgr2=0x7f629006b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:40.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.232+0000 7f6296929700 1 -- 172.21.15.78:0/989779146 shutdown_connections 2023-10-11T09:51:40.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.232+0000 7f6296929700 1 -- 172.21.15.78:0/989779146 wait complete. 2023-10-11T09:51:40.250 INFO:teuthology.orchestra.run.smithi078.stderr:new fs with metadata pool 17 and data pool 18 2023-10-11T09:51:40.273 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:51:40.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.432+0000 7f31d18a7700 1 Processor -- start 2023-10-11T09:51:40.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.432+0000 7f31d18a7700 1 -- start start 2023-10-11T09:51:40.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.432+0000 7f31d18a7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f31cc06cd00 0x7f31cc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:40.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.432+0000 7f31d18a7700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f31cc06fc70 0x7f31cc0700f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:40.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.432+0000 7f31d18a7700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31cc06d690 con 0x7f31cc13f740 2023-10-11T09:51:40.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.432+0000 7f31d18a7700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31cc0cd1c0 con 0x7f31cc06fc70 2023-10-11T09:51:40.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.432+0000 7f31d18a7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31cc0ce4c0 con 0x7f31cc06cd00 2023-10-11T09:51:40.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.432+0000 7f31caffd700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f31cc13f740 0x7f31cc0ccab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:46270/0 (socket says 172.21.15.78:46270) 2023-10-11T09:51:40.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.432+0000 7f31caffd700 1 -- 172.21.15.78:0/4228400311 learned_addr learned my addr 172.21.15.78:0/4228400311 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:51:40.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.432+0000 7f31cb7fe700 1 --2- 172.21.15.78:0/4228400311 >> v2:172.21.15.78:3300/0 conn(0x7f31cc06fc70 0x7f31cc0700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:40.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.432+0000 7f31c3fff700 1 --2- 172.21.15.78:0/4228400311 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f31cc06cd00 0x7f31cc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:40.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.432+0000 7f31ca7fc700 1 -- 172.21.15.78:0/4228400311 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1569665884 0 0) 0x7f31bc001110 con 0x7f31cc13f740 2023-10-11T09:51:40.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.432+0000 7f31ca7fc700 1 -- 172.21.15.78:0/4228400311 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31b80035d0 con 0x7f31cc13f740 2023-10-11T09:51:40.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.432+0000 7f31cb7fe700 1 -- 172.21.15.78:0/4228400311 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f31cc06cd00 msgr2=0x7f31cc06d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:40.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.432+0000 7f31cb7fe700 1 --2- 172.21.15.78:0/4228400311 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f31cc06cd00 0x7f31cc06d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:40.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.432+0000 7f31cb7fe700 1 -- 172.21.15.78:0/4228400311 >> v1:172.21.15.78:6789/0 conn(0x7f31cc13f740 legacy=0x7f31cc0ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:40.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.432+0000 7f31cb7fe700 1 -- 172.21.15.78:0/4228400311 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31cc0cd420 con 0x7f31cc06fc70 2023-10-11T09:51:40.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.436+0000 7f31cb7fe700 1 --2- 172.21.15.78:0/4228400311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31cc06fc70 0x7f31cc0700f0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f31c4009d90 tx=0x7f31c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4644fcf19d958c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:40.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.436+0000 7f31ca7fc700 1 -- 172.21.15.78:0/4228400311 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c400e070 con 0x7f31cc06fc70 2023-10-11T09:51:40.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.436+0000 7f31ca7fc700 1 -- 172.21.15.78:0/4228400311 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c400bb20 con 0x7f31cc06fc70 2023-10-11T09:51:40.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.436+0000 7f31ca7fc700 1 -- 172.21.15.78:0/4228400311 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c4012430 con 0x7f31cc06fc70 2023-10-11T09:51:40.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.436+0000 7f31d18a7700 1 -- 172.21.15.78:0/4228400311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31cc06fc70 msgr2=0x7f31cc0700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:40.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.436+0000 7f31d18a7700 1 --2- 172.21.15.78:0/4228400311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31cc06fc70 0x7f31cc0700f0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f31c4009d90 tx=0x7f31c400b330 comp rx=0 tx=0).stop 2023-10-11T09:51:40.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.436+0000 7f31d18a7700 1 -- 172.21.15.78:0/4228400311 shutdown_connections 2023-10-11T09:51:40.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.436+0000 7f31d18a7700 1 --2- 172.21.15.78:0/4228400311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31cc06fc70 0x7f31cc0700f0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:40.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.436+0000 7f31d18a7700 1 --2- 172.21.15.78:0/4228400311 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f31cc06cd00 0x7f31cc06d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:40.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.436+0000 7f31d18a7700 1 -- 172.21.15.78:0/4228400311 >> 172.21.15.78:0/4228400311 conn(0x7f31cc0cea60 msgr2=0x7f31cc0cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:40.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.436+0000 7f31d18a7700 1 -- 172.21.15.78:0/4228400311 shutdown_connections 2023-10-11T09:51:40.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.436+0000 7f31d18a7700 1 -- 172.21.15.78:0/4228400311 wait complete. 2023-10-11T09:51:40.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.436+0000 7f31d18a7700 1 Processor -- start 2023-10-11T09:51:40.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.440+0000 7f31d18a7700 1 -- start start 2023-10-11T09:51:40.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.440+0000 7f31d18a7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31cc06cd00 0x7f31cc13d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:40.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.440+0000 7f31d18a7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31cc06fc70 0x7f31cc0c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:40.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.440+0000 7f31d18a7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f31cc13f740 0x7f31cc071240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:40.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.440+0000 7f31d18a7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31cc13da40 con 0x7f31cc06cd00 2023-10-11T09:51:40.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.440+0000 7f31d18a7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31cc1f5470 con 0x7f31cc06fc70 2023-10-11T09:51:40.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.440+0000 7f31d18a7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31cc1f5710 con 0x7f31cc13f740 2023-10-11T09:51:40.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.440+0000 7f31caffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31cc06cd00 0x7f31cc13d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:40.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.440+0000 7f31caffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31cc06cd00 0x7f31cc13d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43404/0 (socket says 172.21.15.78:43404) 2023-10-11T09:51:40.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.440+0000 7f31caffd700 1 -- 172.21.15.78:0/205981771 learned_addr learned my addr 172.21.15.78:0/205981771 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:40.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.440+0000 7f31caffd700 1 -- 172.21.15.78:0/205981771 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f31cc13f740 msgr2=0x7f31cc071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:40.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.440+0000 7f31caffd700 1 --2- 172.21.15.78:0/205981771 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f31cc13f740 0x7f31cc071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:40.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.440+0000 7f31caffd700 1 -- 172.21.15.78:0/205981771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31cc06fc70 msgr2=0x7f31cc0c82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:40.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.440+0000 7f31c3fff700 1 --2- 172.21.15.78:0/205981771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31cc06fc70 0x7f31cc0c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:40.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.440+0000 7f31caffd700 1 --2- 172.21.15.78:0/205981771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31cc06fc70 0x7f31cc0c82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:40.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.440+0000 7f31caffd700 1 -- 172.21.15.78:0/205981771 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c4009a70 con 0x7f31cc06cd00 2023-10-11T09:51:40.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.440+0000 7f31caffd700 1 --2- 172.21.15.78:0/205981771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31cc06cd00 0x7f31cc13d500 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f31bc001270 tx=0x7f31bc00ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:40.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.440+0000 7f31c8ff9700 1 -- 172.21.15.78:0/205981771 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31bc005d90 con 0x7f31cc06cd00 2023-10-11T09:51:40.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.440+0000 7f31c8ff9700 1 -- 172.21.15.78:0/205981771 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31bc0070d0 con 0x7f31cc06cd00 2023-10-11T09:51:40.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.440+0000 7f31c8ff9700 1 -- 172.21.15.78:0/205981771 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31bc007480 con 0x7f31cc06cd00 2023-10-11T09:51:40.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.440+0000 7f31d18a7700 1 -- 172.21.15.78:0/205981771 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31cc1f59f0 con 0x7f31cc06cd00 2023-10-11T09:51:40.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.440+0000 7f31d18a7700 1 -- 172.21.15.78:0/205981771 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31cc1f5fb0 con 0x7f31cc06cd00 2023-10-11T09:51:40.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.444+0000 7f31d18a7700 1 -- 172.21.15.78:0/205981771 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31cc065d40 con 0x7f31cc06cd00 2023-10-11T09:51:40.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.444+0000 7f31c8ff9700 1 -- 172.21.15.78:0/205981771 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f31bc0075e0 con 0x7f31cc06cd00 2023-10-11T09:51:40.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.448+0000 7f31c8ff9700 1 --2- 172.21.15.78:0/205981771 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f31ac065ed0 0x7f31ac068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:40.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.448+0000 7f31c8ff9700 1 -- 172.21.15.78:0/205981771 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f31bc082b70 con 0x7f31cc06cd00 2023-10-11T09:51:40.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.448+0000 7f31c3fff700 1 --2- 172.21.15.78:0/205981771 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f31ac065ed0 0x7f31ac068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:40.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.448+0000 7f31c3fff700 1 --2- 172.21.15.78:0/205981771 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f31ac065ed0 0x7f31ac068380 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f31cc151990 tx=0x7f31b4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:40.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.452+0000 7f31c8ff9700 1 -- 172.21.15.78:0/205981771 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f31bc050250 con 0x7f31cc06cd00 2023-10-11T09:51:40.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.628+0000 7f31d18a7700 1 -- 172.21.15.78:0/205981771 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f31cc06d690 con 0x7f31cc06cd00 2023-10-11T09:51:40.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.628+0000 7f31c8ff9700 1 -- 172.21.15.78:0/205981771 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+17906 (secure 0 0 0) 0x7f31cc06d690 con 0x7f31cc06cd00 2023-10-11T09:51:40.639 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:40.639 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":126,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:51:40.211059+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:51:38.592892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-11T09:51:39.204588+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:51:40.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.628+0000 7f31d18a7700 1 -- 172.21.15.78:0/205981771 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f31ac065ed0 msgr2=0x7f31ac068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:40.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.628+0000 7f31d18a7700 1 --2- 172.21.15.78:0/205981771 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f31ac065ed0 0x7f31ac068380 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f31cc151990 tx=0x7f31b4008040 comp rx=0 tx=0).stop 2023-10-11T09:51:40.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.628+0000 7f31d18a7700 1 -- 172.21.15.78:0/205981771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31cc06cd00 msgr2=0x7f31cc13d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:40.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.632+0000 7f31d18a7700 1 --2- 172.21.15.78:0/205981771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31cc06cd00 0x7f31cc13d500 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f31bc001270 tx=0x7f31bc00ca90 comp rx=0 tx=0).stop 2023-10-11T09:51:40.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.632+0000 7f31d18a7700 1 -- 172.21.15.78:0/205981771 shutdown_connections 2023-10-11T09:51:40.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.632+0000 7f31d18a7700 1 --2- 172.21.15.78:0/205981771 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f31ac065ed0 0x7f31ac068380 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:40.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.632+0000 7f31d18a7700 1 --2- 172.21.15.78:0/205981771 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f31cc13f740 0x7f31cc071240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:40.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.632+0000 7f31d18a7700 1 --2- 172.21.15.78:0/205981771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31cc06fc70 0x7f31cc0c82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:40.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.632+0000 7f31d18a7700 1 --2- 172.21.15.78:0/205981771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31cc06cd00 0x7f31cc13d500 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:40.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.632+0000 7f31d18a7700 1 -- 172.21.15.78:0/205981771 >> 172.21.15.78:0/205981771 conn(0x7f31cc0cea60 msgr2=0x7f31cc0703e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:40.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.632+0000 7f31d18a7700 1 -- 172.21.15.78:0/205981771 shutdown_connections 2023-10-11T09:51:40.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.632+0000 7f31d18a7700 1 -- 172.21.15.78:0/205981771 wait complete. 2023-10-11T09:51:40.662 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:51:40.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.824+0000 7fb5680c6700 1 Processor -- start 2023-10-11T09:51:40.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.824+0000 7fb5680c6700 1 -- start start 2023-10-11T09:51:40.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.824+0000 7fb5680c6700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fb560071040 0x7fb5600cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:40.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.824+0000 7fb5680c6700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb5600ce330 0x7fb56006be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:40.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.824+0000 7fb5680c6700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb56006ffa0 con 0x7fb560160d40 2023-10-11T09:51:40.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.824+0000 7fb5680c6700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb560070c20 con 0x7fb560071040 2023-10-11T09:51:40.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.824+0000 7fb5680c6700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb560161110 con 0x7fb5600ce330 2023-10-11T09:51:40.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.824+0000 7fb565661700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fb560071040 0x7fb5600cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:40.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.828+0000 7fb565661700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fb560071040 0x7fb5600cddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43412/0 (socket says 172.21.15.78:43412) 2023-10-11T09:51:40.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.828+0000 7fb565661700 1 -- 172.21.15.78:0/596262621 learned_addr learned my addr 172.21.15.78:0/596262621 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:40.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.828+0000 7fb566663700 1 --2- 172.21.15.78:0/596262621 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb5600ce330 0x7fb56006be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:40.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.828+0000 7fb565661700 1 -- 172.21.15.78:0/596262621 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb5600ce330 msgr2=0x7fb56006be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:40.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.828+0000 7fb565661700 1 --2- 172.21.15.78:0/596262621 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb5600ce330 0x7fb56006be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:40.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.828+0000 7fb565661700 1 -- 172.21.15.78:0/596262621 >> v1:172.21.15.133:6789/0 conn(0x7fb560160d40 legacy=0x7fb5600cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:40.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.828+0000 7fb565661700 1 -- 172.21.15.78:0/596262621 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb560070280 con 0x7fb560071040 2023-10-11T09:51:40.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.828+0000 7fb565661700 1 --2- 172.21.15.78:0/596262621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb560071040 0x7fb5600cddf0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fb550009910 tx=0x7fb550009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1c1310d1e0ff84 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:40.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.828+0000 7fb564e60700 1 -- 172.21.15.78:0/596262621 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55000ba60 con 0x7fb560071040 2023-10-11T09:51:40.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.828+0000 7fb564e60700 1 -- 172.21.15.78:0/596262621 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb55000bbc0 con 0x7fb560071040 2023-10-11T09:51:40.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.828+0000 7fb564e60700 1 -- 172.21.15.78:0/596262621 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb550009e60 con 0x7fb560071040 2023-10-11T09:51:40.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.828+0000 7fb5680c6700 1 -- 172.21.15.78:0/596262621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb560071040 msgr2=0x7fb5600cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:40.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.828+0000 7fb5680c6700 1 --2- 172.21.15.78:0/596262621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb560071040 0x7fb5600cddf0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fb550009910 tx=0x7fb550009c20 comp rx=0 tx=0).stop 2023-10-11T09:51:40.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.828+0000 7fb5680c6700 1 -- 172.21.15.78:0/596262621 shutdown_connections 2023-10-11T09:51:40.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.828+0000 7fb5680c6700 1 --2- 172.21.15.78:0/596262621 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb5600ce330 0x7fb56006be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:40.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.828+0000 7fb5680c6700 1 --2- 172.21.15.78:0/596262621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb560071040 0x7fb5600cddf0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:40.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.828+0000 7fb5680c6700 1 -- 172.21.15.78:0/596262621 >> 172.21.15.78:0/596262621 conn(0x7fb56006aca0 msgr2=0x7fb5600b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:40.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.828+0000 7fb5680c6700 1 -- 172.21.15.78:0/596262621 shutdown_connections 2023-10-11T09:51:40.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.828+0000 7fb5680c6700 1 -- 172.21.15.78:0/596262621 wait complete. 2023-10-11T09:51:40.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.832+0000 7fb5680c6700 1 Processor -- start 2023-10-11T09:51:40.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.832+0000 7fb5680c6700 1 -- start start 2023-10-11T09:51:40.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.832+0000 7fb5680c6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb560071040 0x7fb5601506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:40.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.832+0000 7fb5680c6700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5600ce330 0x7fb56014c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:40.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.832+0000 7fb5680c6700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb560160d40 0x7fb5600b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:40.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.832+0000 7fb5680c6700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56014d1b0 con 0x7fb560071040 2023-10-11T09:51:40.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.832+0000 7fb5680c6700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56014d450 con 0x7fb5600ce330 2023-10-11T09:51:40.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.832+0000 7fb5680c6700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb56014d730 con 0x7fb560160d40 2023-10-11T09:51:40.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.832+0000 7fb565e62700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb560071040 0x7fb5601506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:40.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.832+0000 7fb565661700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5600ce330 0x7fb56014c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:40.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.832+0000 7fb565e62700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb560071040 0x7fb5601506a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43428/0 (socket says 172.21.15.78:43428) 2023-10-11T09:51:40.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.832+0000 7fb565e62700 1 -- 172.21.15.78:0/2834523918 learned_addr learned my addr 172.21.15.78:0/2834523918 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:40.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.832+0000 7fb566663700 1 --2- 172.21.15.78:0/2834523918 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb560160d40 0x7fb5600b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:40.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.836+0000 7fb565e62700 1 -- 172.21.15.78:0/2834523918 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb560160d40 msgr2=0x7fb5600b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:40.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.836+0000 7fb565e62700 1 --2- 172.21.15.78:0/2834523918 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb560160d40 0x7fb5600b3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:40.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.836+0000 7fb565e62700 1 -- 172.21.15.78:0/2834523918 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5600ce330 msgr2=0x7fb56014c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:40.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.836+0000 7fb565e62700 1 --2- 172.21.15.78:0/2834523918 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5600ce330 0x7fb56014c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:40.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.836+0000 7fb565e62700 1 -- 172.21.15.78:0/2834523918 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb550009570 con 0x7fb560071040 2023-10-11T09:51:40.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.836+0000 7fb565e62700 1 --2- 172.21.15.78:0/2834523918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb560071040 0x7fb5601506a0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fb55c00c500 tx=0x7fb55c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:40.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.836+0000 7fb5567fc700 1 -- 172.21.15.78:0/2834523918 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c0052d0 con 0x7fb560071040 2023-10-11T09:51:40.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.836+0000 7fb5567fc700 1 -- 172.21.15.78:0/2834523918 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb55c005430 con 0x7fb560071040 2023-10-11T09:51:40.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.836+0000 7fb5680c6700 1 -- 172.21.15.78:0/2834523918 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb56014da10 con 0x7fb560071040 2023-10-11T09:51:40.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.836+0000 7fb5567fc700 1 -- 172.21.15.78:0/2834523918 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c0055a0 con 0x7fb560071040 2023-10-11T09:51:40.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.836+0000 7fb5680c6700 1 -- 172.21.15.78:0/2834523918 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb56014dfd0 con 0x7fb560071040 2023-10-11T09:51:40.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.836+0000 7fb5680c6700 1 -- 172.21.15.78:0/2834523918 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb560072250 con 0x7fb560071040 2023-10-11T09:51:40.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.840+0000 7fb5567fc700 1 -- 172.21.15.78:0/2834523918 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fb55c0038e0 con 0x7fb560071040 2023-10-11T09:51:40.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.840+0000 7fb5567fc700 1 --2- 172.21.15.78:0/2834523918 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb544065f00 0x7fb5440683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:40.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.844+0000 7fb5567fc700 1 -- 172.21.15.78:0/2834523918 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb55c082b30 con 0x7fb560071040 2023-10-11T09:51:40.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.844+0000 7fb565661700 1 --2- 172.21.15.78:0/2834523918 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb544065f00 0x7fb5440683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:40.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.844+0000 7fb565661700 1 --2- 172.21.15.78:0/2834523918 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb544065f00 0x7fb5440683b0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb560071430 tx=0x7fb550008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:40.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:40.848+0000 7fb5567fc700 1 -- 172.21.15.78:0/2834523918 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fb55c050170 con 0x7fb560071040 2023-10-11T09:51:41.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.020+0000 7fb5680c6700 1 -- 172.21.15.78:0/2834523918 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb56004e530 con 0x7fb560071040 2023-10-11T09:51:41.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.024+0000 7fb5567fc700 1 -- 172.21.15.78:0/2834523918 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+17906 (secure 0 0 0) 0x7fb55c053790 con 0x7fb560071040 2023-10-11T09:51:41.035 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:41.036 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":126,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:51:40.211059+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:51:38.592892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-11T09:51:39.204588+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:51:41.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.028+0000 7fb5680c6700 1 -- 172.21.15.78:0/2834523918 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb544065f00 msgr2=0x7fb5440683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:41.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.028+0000 7fb5680c6700 1 --2- 172.21.15.78:0/2834523918 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb544065f00 0x7fb5440683b0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb560071430 tx=0x7fb550008030 comp rx=0 tx=0).stop 2023-10-11T09:51:41.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.028+0000 7fb5680c6700 1 -- 172.21.15.78:0/2834523918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb560071040 msgr2=0x7fb5601506a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:41.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.028+0000 7fb5680c6700 1 --2- 172.21.15.78:0/2834523918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb560071040 0x7fb5601506a0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fb55c00c500 tx=0x7fb55c00c810 comp rx=0 tx=0).stop 2023-10-11T09:51:41.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.028+0000 7fb5680c6700 1 -- 172.21.15.78:0/2834523918 shutdown_connections 2023-10-11T09:51:41.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.028+0000 7fb5680c6700 1 --2- 172.21.15.78:0/2834523918 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb544065f00 0x7fb5440683b0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:41.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.028+0000 7fb5680c6700 1 --2- 172.21.15.78:0/2834523918 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb560160d40 0x7fb5600b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:41.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.028+0000 7fb5680c6700 1 --2- 172.21.15.78:0/2834523918 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5600ce330 0x7fb56014c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:41.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.028+0000 7fb5680c6700 1 --2- 172.21.15.78:0/2834523918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb560071040 0x7fb5601506a0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:41.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.028+0000 7fb5680c6700 1 -- 172.21.15.78:0/2834523918 >> 172.21.15.78:0/2834523918 conn(0x7fb56006aca0 msgr2=0x7fb560161350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:41.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.028+0000 7fb5680c6700 1 -- 172.21.15.78:0/2834523918 shutdown_connections 2023-10-11T09:51:41.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.028+0000 7fb5680c6700 1 -- 172.21.15.78:0/2834523918 wait complete. 2023-10-11T09:51:41.065 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-11T09:51:41.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.232+0000 7f7012d7d700 1 Processor -- start 2023-10-11T09:51:41.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.232+0000 7f7012d7d700 1 -- start start 2023-10-11T09:51:41.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.236+0000 7f7012d7d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f700c0b3610 0x7f700c0d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:41.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.236+0000 7f7012d7d700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f700c0b47f0 con 0x7f700c0b3240 2023-10-11T09:51:41.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.236+0000 7f7012d7d700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f700c0b8130 con 0x7f700c0b3a90 2023-10-11T09:51:41.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.236+0000 7f7012d7d700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f700c0b41e0 con 0x7f700c0b3610 2023-10-11T09:51:41.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.236+0000 7f701157a700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f700c0b3610 0x7f700c0d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:41.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.236+0000 7f701157a700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f700c0b3610 0x7f700c0d29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43438/0 (socket says 172.21.15.78:43438) 2023-10-11T09:51:41.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.236+0000 7f701157a700 1 -- 172.21.15.78:0/1202328634 learned_addr learned my addr 172.21.15.78:0/1202328634 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:41.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.236+0000 7f7011d7b700 1 --1- 172.21.15.78:0/1202328634 >> v1:172.21.15.78:6789/0 conn(0x7f700c0b3240 0x7f700c15d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:46280/0 (socket says 172.21.15.78:46280) 2023-10-11T09:51:41.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.236+0000 7f701157a700 1 -- 172.21.15.78:0/1202328634 >> v1:172.21.15.133:6789/0 conn(0x7f700c0b3a90 legacy=0x7f700c0b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:41.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.236+0000 7f701157a700 1 -- 172.21.15.78:0/1202328634 >> v1:172.21.15.78:6789/0 conn(0x7f700c0b3240 legacy=0x7f700c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:41.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.236+0000 7f701157a700 1 -- 172.21.15.78:0/1202328634 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700c0b7800 con 0x7f700c0b3610 2023-10-11T09:51:41.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.236+0000 7f7010d79700 1 -- 172.21.15.78:0/1202328634 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3280425891 0 0) 0x7f7008001110 con 0x7f700c0b3240 2023-10-11T09:51:41.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.236+0000 7f701157a700 1 --2- 172.21.15.78:0/1202328634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f700c0b3610 0x7f700c0d29d0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc009910 tx=0x7f6ffc009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b31b2b59ad0c1947 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:41.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.236+0000 7f7010d79700 1 -- 172.21.15.78:0/1202328634 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc00ba60 con 0x7f700c0b3610 2023-10-11T09:51:41.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.236+0000 7f7010d79700 1 -- 172.21.15.78:0/1202328634 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ffc00bbc0 con 0x7f700c0b3610 2023-10-11T09:51:41.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.236+0000 7f7010d79700 1 -- 172.21.15.78:0/1202328634 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc009e60 con 0x7f700c0b3610 2023-10-11T09:51:41.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.236+0000 7f7012d7d700 1 -- 172.21.15.78:0/1202328634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f700c0b3610 msgr2=0x7f700c0d29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:41.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.236+0000 7f7012d7d700 1 --2- 172.21.15.78:0/1202328634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f700c0b3610 0x7f700c0d29d0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc009910 tx=0x7f6ffc009c20 comp rx=0 tx=0).stop 2023-10-11T09:51:41.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.240+0000 7f7012d7d700 1 -- 172.21.15.78:0/1202328634 shutdown_connections 2023-10-11T09:51:41.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.240+0000 7f7012d7d700 1 --2- 172.21.15.78:0/1202328634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f700c0b3610 0x7f700c0d29d0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:41.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.240+0000 7f7012d7d700 1 -- 172.21.15.78:0/1202328634 >> 172.21.15.78:0/1202328634 conn(0x7f700c09c260 msgr2=0x7f700c0d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:41.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.240+0000 7f7012d7d700 1 -- 172.21.15.78:0/1202328634 shutdown_connections 2023-10-11T09:51:41.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.240+0000 7f7012d7d700 1 -- 172.21.15.78:0/1202328634 wait complete. 2023-10-11T09:51:41.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.240+0000 7f7012d7d700 1 Processor -- start 2023-10-11T09:51:41.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.240+0000 7f7012d7d700 1 -- start start 2023-10-11T09:51:41.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.244+0000 7f7012d7d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f700c0b3240 0x7f700c1f61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:41.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.244+0000 7f7012d7d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f700c0b3610 0x7f700c0c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:41.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.244+0000 7f7012d7d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f700c0b3a90 0x7f700c1f1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:41.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.244+0000 7f7012d7d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c0c9a60 con 0x7f700c0b3a90 2023-10-11T09:51:41.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.244+0000 7f7012d7d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c0c9830 con 0x7f700c0b3610 2023-10-11T09:51:41.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.244+0000 7f7012d7d700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f700c09db50 con 0x7f700c0b3240 2023-10-11T09:51:41.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.244+0000 7f701257c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f700c0b3a90 0x7f700c1f1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:41.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.244+0000 7f7011d7b700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f700c0b3240 0x7f700c1f61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:41.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.244+0000 7f7011d7b700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f700c0b3240 0x7f700c1f61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:36840/0 (socket says 172.21.15.78:36840) 2023-10-11T09:51:41.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.244+0000 7f7011d7b700 1 -- 172.21.15.78:0/118143807 learned_addr learned my addr 172.21.15.78:0/118143807 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:41.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.244+0000 7f701157a700 1 --2- 172.21.15.78:0/118143807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f700c0b3610 0x7f700c0c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:41.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.244+0000 7f7011d7b700 1 -- 172.21.15.78:0/118143807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f700c0b3610 msgr2=0x7f700c0c92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:41.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.244+0000 7f7011d7b700 1 --2- 172.21.15.78:0/118143807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f700c0b3610 0x7f700c0c92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:41.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.244+0000 7f7011d7b700 1 -- 172.21.15.78:0/118143807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f700c0b3a90 msgr2=0x7f700c1f1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:41.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.244+0000 7f7011d7b700 1 --2- 172.21.15.78:0/118143807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f700c0b3a90 0x7f700c1f1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:41.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.244+0000 7f7011d7b700 1 -- 172.21.15.78:0/118143807 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ffc009570 con 0x7f700c0b3240 2023-10-11T09:51:41.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.244+0000 7f701157a700 1 --2- 172.21.15.78:0/118143807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f700c0b3610 0x7f700c0c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:51:41.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.244+0000 7f7011d7b700 1 --2- 172.21.15.78:0/118143807 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f700c0b3240 0x7f700c1f61c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f700800c4d0 tx=0x7f700800c7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:41.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.244+0000 7f70027fc700 1 -- 172.21.15.78:0/118143807 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700800cea0 con 0x7f700c0b3240 2023-10-11T09:51:41.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.248+0000 7f70027fc700 1 -- 172.21.15.78:0/118143807 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7008004ca0 con 0x7f700c0b3240 2023-10-11T09:51:41.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.248+0000 7f7012d7d700 1 -- 172.21.15.78:0/118143807 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f700c09ddf0 con 0x7f700c0b3240 2023-10-11T09:51:41.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.248+0000 7f70027fc700 1 -- 172.21.15.78:0/118143807 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7008007310 con 0x7f700c0b3240 2023-10-11T09:51:41.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.248+0000 7f7012d7d700 1 -- 172.21.15.78:0/118143807 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f700c09e3b0 con 0x7f700c0b3240 2023-10-11T09:51:41.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.248+0000 7f7012d7d700 1 -- 172.21.15.78:0/118143807 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f700c065d40 con 0x7f700c0b3240 2023-10-11T09:51:41.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.248+0000 7f70027fc700 1 -- 172.21.15.78:0/118143807 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f7008007470 con 0x7f700c0b3240 2023-10-11T09:51:41.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.252+0000 7f70027fc700 1 --2- 172.21.15.78:0/118143807 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6ff0065ed0 0x7f6ff0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:41.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.252+0000 7f70027fc700 1 -- 172.21.15.78:0/118143807 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7008083290 con 0x7f700c0b3240 2023-10-11T09:51:41.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.252+0000 7f701157a700 1 --2- 172.21.15.78:0/118143807 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6ff0065ed0 0x7f6ff0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:41.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.252+0000 7f701157a700 1 --2- 172.21.15.78:0/118143807 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6ff0065ed0 0x7f6ff0068380 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc014040 tx=0x7f6ffc01c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:41.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.260+0000 7f70027fc700 1 -- 172.21.15.78:0/118143807 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f7008050a50 con 0x7f700c0b3240 2023-10-11T09:51:41.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:41.472+0000 7f7012d7d700 1 -- 172.21.15.78:0/118143807 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f700c0b47f0 con 0x7f700c0b3240 2023-10-11T09:51:42.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.236+0000 7f70027fc700 1 -- 172.21.15.78:0/118143807 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v98) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7008007720 con 0x7f700c0b3240 2023-10-11T09:51:42.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.240+0000 7f7012d7d700 1 -- 172.21.15.78:0/118143807 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6ff0065ed0 msgr2=0x7f6ff0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:42.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.240+0000 7f7012d7d700 1 --2- 172.21.15.78:0/118143807 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6ff0065ed0 0x7f6ff0068380 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc014040 tx=0x7f6ffc01c040 comp rx=0 tx=0).stop 2023-10-11T09:51:42.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.240+0000 7f7012d7d700 1 -- 172.21.15.78:0/118143807 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f700c0b3240 msgr2=0x7f700c1f61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:42.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.240+0000 7f7012d7d700 1 --2- 172.21.15.78:0/118143807 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f700c0b3240 0x7f700c1f61c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f700800c4d0 tx=0x7f700800c7e0 comp rx=0 tx=0).stop 2023-10-11T09:51:42.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.240+0000 7f7012d7d700 1 -- 172.21.15.78:0/118143807 shutdown_connections 2023-10-11T09:51:42.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.240+0000 7f7012d7d700 1 --2- 172.21.15.78:0/118143807 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6ff0065ed0 0x7f6ff0068380 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:42.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.240+0000 7f7012d7d700 1 --2- 172.21.15.78:0/118143807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f700c0b3a90 0x7f700c1f1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:42.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.240+0000 7f7012d7d700 1 --2- 172.21.15.78:0/118143807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f700c0b3610 0x7f700c0c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:42.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.240+0000 7f7012d7d700 1 --2- 172.21.15.78:0/118143807 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f700c0b3240 0x7f700c1f61c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:42.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.240+0000 7f7012d7d700 1 -- 172.21.15.78:0/118143807 >> 172.21.15.78:0/118143807 conn(0x7f700c09c260 msgr2=0x7f700c0d0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:42.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.240+0000 7f7012d7d700 1 -- 172.21.15.78:0/118143807 shutdown_connections 2023-10-11T09:51:42.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.240+0000 7f7012d7d700 1 -- 172.21.15.78:0/118143807 wait complete. 2023-10-11T09:51:42.274 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-11T09:51:42.274 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-11T09:51:42.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.440+0000 7fe4812d8700 1 Processor -- start 2023-10-11T09:51:42.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.440+0000 7fe4812d8700 1 -- start start 2023-10-11T09:51:42.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.444+0000 7fe4812d8700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe47c0b3240 0x7fe47c0d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:42.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.444+0000 7fe4812d8700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe47c0b47f0 con 0x7fe47c0b3610 2023-10-11T09:51:42.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.444+0000 7fe4812d8700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe47c0b8130 con 0x7fe47c0b3a90 2023-10-11T09:51:42.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.444+0000 7fe4812d8700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe47c0b41e0 con 0x7fe47c0b3240 2023-10-11T09:51:42.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.444+0000 7fe47a7fc700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fe47c0b3610 0x7fe47c15d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:46290/0 (socket says 172.21.15.78:46290) 2023-10-11T09:51:42.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.444+0000 7fe47a7fc700 1 -- 172.21.15.78:0/1267183394 learned_addr learned my addr 172.21.15.78:0/1267183394 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:51:42.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.444+0000 7fe47affd700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe47c0b3240 0x7fe47c0d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:42.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.448+0000 7fe479ffb700 1 -- 172.21.15.78:0/1267183394 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2613113259 0 0) 0x7fe470001110 con 0x7fe47c0b3610 2023-10-11T09:51:42.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.448+0000 7fe479ffb700 1 -- 172.21.15.78:0/1267183394 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4640035d0 con 0x7fe47c0b3610 2023-10-11T09:51:42.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.448+0000 7fe479ffb700 1 -- 172.21.15.78:0/1267183394 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1348797618 0 0) 0x7fe47c0b8130 con 0x7fe47c0b3a90 2023-10-11T09:51:42.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.448+0000 7fe479ffb700 1 -- 172.21.15.78:0/1267183394 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe470001110 con 0x7fe47c0b3a90 2023-10-11T09:51:42.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.448+0000 7fe47affd700 1 -- 172.21.15.78:0/1267183394 >> v1:172.21.15.133:6789/0 conn(0x7fe47c0b3a90 legacy=0x7fe47c0b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:42.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.448+0000 7fe479ffb700 1 -- 172.21.15.78:0/1267183394 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2628795657 0 0) 0x7fe4640035d0 con 0x7fe47c0b3610 2023-10-11T09:51:42.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.448+0000 7fe47affd700 1 -- 172.21.15.78:0/1267183394 >> v1:172.21.15.78:6789/0 conn(0x7fe47c0b3610 legacy=0x7fe47c15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:42.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.448+0000 7fe47affd700 1 -- 172.21.15.78:0/1267183394 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe47c0b7800 con 0x7fe47c0b3240 2023-10-11T09:51:42.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.448+0000 7fe47affd700 1 --2- 172.21.15.78:0/1267183394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe47c0b3240 0x7fe47c0d29d0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fe46c009850 tx=0x7fe46c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af91a99a49e7d771 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:42.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.448+0000 7fe479ffb700 1 -- 172.21.15.78:0/1267183394 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe470001800 con 0x7fe47c0b3240 2023-10-11T09:51:42.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.448+0000 7fe479ffb700 1 -- 172.21.15.78:0/1267183394 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe46c004030 con 0x7fe47c0b3240 2023-10-11T09:51:42.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.448+0000 7fe479ffb700 1 -- 172.21.15.78:0/1267183394 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c0043c0 con 0x7fe47c0b3240 2023-10-11T09:51:42.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.448+0000 7fe4812d8700 1 -- 172.21.15.78:0/1267183394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe47c0b3240 msgr2=0x7fe47c0d29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:42.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.448+0000 7fe4812d8700 1 --2- 172.21.15.78:0/1267183394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe47c0b3240 0x7fe47c0d29d0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fe46c009850 tx=0x7fe46c009b60 comp rx=0 tx=0).stop 2023-10-11T09:51:42.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.448+0000 7fe4812d8700 1 -- 172.21.15.78:0/1267183394 shutdown_connections 2023-10-11T09:51:42.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.448+0000 7fe4812d8700 1 --2- 172.21.15.78:0/1267183394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe47c0b3240 0x7fe47c0d29d0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:42.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.448+0000 7fe4812d8700 1 -- 172.21.15.78:0/1267183394 >> 172.21.15.78:0/1267183394 conn(0x7fe47c09c260 msgr2=0x7fe47c0d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:42.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.448+0000 7fe4812d8700 1 -- 172.21.15.78:0/1267183394 shutdown_connections 2023-10-11T09:51:42.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.448+0000 7fe4812d8700 1 -- 172.21.15.78:0/1267183394 wait complete. 2023-10-11T09:51:42.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.452+0000 7fe4812d8700 1 Processor -- start 2023-10-11T09:51:42.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.452+0000 7fe4812d8700 1 -- start start 2023-10-11T09:51:42.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.452+0000 7fe4812d8700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe47c0b3240 0x7fe47c0c7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:42.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.452+0000 7fe4812d8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe47c0b3610 0x7fe47c15daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:42.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.452+0000 7fe4812d8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe47c0b3a90 0x7fe47c0c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:42.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.452+0000 7fe4812d8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47c0c7b80 con 0x7fe47c0b3610 2023-10-11T09:51:42.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.452+0000 7fe4812d8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47c0c71b0 con 0x7fe47c0b3a90 2023-10-11T09:51:42.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.452+0000 7fe4812d8700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe47c15e2f0 con 0x7fe47c0b3240 2023-10-11T09:51:42.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.456+0000 7fe47a7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe47c0b3610 0x7fe47c15daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:42.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.456+0000 7fe47a7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe47c0b3610 0x7fe47c15daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43458/0 (socket says 172.21.15.78:43458) 2023-10-11T09:51:42.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.456+0000 7fe47a7fc700 1 -- 172.21.15.78:0/2989097945 learned_addr learned my addr 172.21.15.78:0/2989097945 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:42.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.456+0000 7fe47affd700 1 --2- 172.21.15.78:0/2989097945 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe47c0b3240 0x7fe47c0c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:42.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.456+0000 7fe47b7fe700 1 --2- 172.21.15.78:0/2989097945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe47c0b3a90 0x7fe47c0c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:42.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.456+0000 7fe47a7fc700 1 -- 172.21.15.78:0/2989097945 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe47c0b3240 msgr2=0x7fe47c0c7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:42.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.456+0000 7fe47a7fc700 1 --2- 172.21.15.78:0/2989097945 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe47c0b3240 0x7fe47c0c7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:42.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.456+0000 7fe47a7fc700 1 -- 172.21.15.78:0/2989097945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe47c0b3a90 msgr2=0x7fe47c0c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:42.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.456+0000 7fe47a7fc700 1 --2- 172.21.15.78:0/2989097945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe47c0b3a90 0x7fe47c0c91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:42.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.456+0000 7fe47a7fc700 1 -- 172.21.15.78:0/2989097945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe46c0094b0 con 0x7fe47c0b3610 2023-10-11T09:51:42.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.456+0000 7fe47b7fe700 1 --2- 172.21.15.78:0/2989097945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe47c0b3a90 0x7fe47c0c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:51:42.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.456+0000 7fe47a7fc700 1 --2- 172.21.15.78:0/2989097945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe47c0b3610 0x7fe47c15daa0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fe47000c7b0 tx=0x7fe47000cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:42.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.456+0000 7fe4637fe700 1 -- 172.21.15.78:0/2989097945 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe470004ca0 con 0x7fe47c0b3610 2023-10-11T09:51:42.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.456+0000 7fe4637fe700 1 -- 172.21.15.78:0/2989097945 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe470004e00 con 0x7fe47c0b3610 2023-10-11T09:51:42.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.456+0000 7fe4637fe700 1 -- 172.21.15.78:0/2989097945 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4700072d0 con 0x7fe47c0b3610 2023-10-11T09:51:42.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.456+0000 7fe4812d8700 1 -- 172.21.15.78:0/2989097945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe47c15e5d0 con 0x7fe47c0b3610 2023-10-11T09:51:42.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.456+0000 7fe4812d8700 1 -- 172.21.15.78:0/2989097945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe47c0c6e40 con 0x7fe47c0b3610 2023-10-11T09:51:42.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.456+0000 7fe4812d8700 1 -- 172.21.15.78:0/2989097945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe47c065d40 con 0x7fe47c0b3610 2023-10-11T09:51:42.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.460+0000 7fe4637fe700 1 -- 172.21.15.78:0/2989097945 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fe470007430 con 0x7fe47c0b3610 2023-10-11T09:51:42.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.460+0000 7fe4637fe700 1 --2- 172.21.15.78:0/2989097945 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe468065ed0 0x7fe468068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:42.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.460+0000 7fe4637fe700 1 -- 172.21.15.78:0/2989097945 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe4700827f0 con 0x7fe47c0b3610 2023-10-11T09:51:42.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.460+0000 7fe47affd700 1 --2- 172.21.15.78:0/2989097945 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe468065ed0 0x7fe468068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:42.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.460+0000 7fe47affd700 1 --2- 172.21.15.78:0/2989097945 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe468065ed0 0x7fe468068380 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fe46c0090b0 tx=0x7fe46c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:42.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.468+0000 7fe4637fe700 1 -- 172.21.15.78:0/2989097945 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fe47004fe50 con 0x7fe47c0b3610 2023-10-11T09:51:42.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:42.680+0000 7fe4812d8700 1 -- 172.21.15.78:0/2989097945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe47c0b47f0 con 0x7fe47c0b3610 2023-10-11T09:51:43.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.244+0000 7fe4637fe700 1 -- 172.21.15.78:0/2989097945 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v99) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe470053470 con 0x7fe47c0b3610 2023-10-11T09:51:43.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.248+0000 7fe4812d8700 1 -- 172.21.15.78:0/2989097945 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe468065ed0 msgr2=0x7fe468068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:43.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.248+0000 7fe4812d8700 1 --2- 172.21.15.78:0/2989097945 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe468065ed0 0x7fe468068380 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fe46c0090b0 tx=0x7fe46c008030 comp rx=0 tx=0).stop 2023-10-11T09:51:43.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.248+0000 7fe4812d8700 1 -- 172.21.15.78:0/2989097945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe47c0b3610 msgr2=0x7fe47c15daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:43.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.248+0000 7fe4812d8700 1 --2- 172.21.15.78:0/2989097945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe47c0b3610 0x7fe47c15daa0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fe47000c7b0 tx=0x7fe47000cac0 comp rx=0 tx=0).stop 2023-10-11T09:51:43.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.248+0000 7fe4812d8700 1 -- 172.21.15.78:0/2989097945 shutdown_connections 2023-10-11T09:51:43.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.248+0000 7fe4812d8700 1 --2- 172.21.15.78:0/2989097945 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe468065ed0 0x7fe468068380 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:43.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.248+0000 7fe4812d8700 1 --2- 172.21.15.78:0/2989097945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe47c0b3a90 0x7fe47c0c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:43.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.248+0000 7fe4812d8700 1 --2- 172.21.15.78:0/2989097945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe47c0b3610 0x7fe47c15daa0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:43.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.248+0000 7fe4812d8700 1 --2- 172.21.15.78:0/2989097945 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe47c0b3240 0x7fe47c0c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:43.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.248+0000 7fe4812d8700 1 -- 172.21.15.78:0/2989097945 >> 172.21.15.78:0/2989097945 conn(0x7fe47c09c260 msgr2=0x7fe47c15c570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:43.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.248+0000 7fe4812d8700 1 -- 172.21.15.78:0/2989097945 shutdown_connections 2023-10-11T09:51:43.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.248+0000 7fe4812d8700 1 -- 172.21.15.78:0/2989097945 wait complete. 2023-10-11T09:51:43.283 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-11T09:51:43.284 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:51:43.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.452+0000 7fd2a5245700 1 Processor -- start 2023-10-11T09:51:43.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.452+0000 7fd2a5245700 1 -- start start 2023-10-11T09:51:43.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.452+0000 7fd2a5245700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd2a013f740 0x7fd2a00ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:43.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.452+0000 7fd2a5245700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2a00b14c0 con 0x7fd2a006d180 2023-10-11T09:51:43.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.452+0000 7fd2a5245700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2a0070c80 con 0x7fd2a006cd00 2023-10-11T09:51:43.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.452+0000 7fd2a5245700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2a00ce4c0 con 0x7fd2a013f740 2023-10-11T09:51:43.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.452+0000 7fd2a4a44700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fd2a006d180 0x7fd2a00b0db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:46306/0 (socket says 172.21.15.78:46306) 2023-10-11T09:51:43.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.452+0000 7fd2a4a44700 1 -- 172.21.15.78:0/3660260825 learned_addr learned my addr 172.21.15.78:0/3660260825 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:51:43.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.452+0000 7fd29ffff700 1 --2- 172.21.15.78:0/3660260825 >> v2:172.21.15.133:3300/0 conn(0x7fd2a013f740 0x7fd2a00ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:43.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.452+0000 7fd29f7fe700 1 -- 172.21.15.78:0/3660260825 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2037551869 0 0) 0x7fd2a00b14c0 con 0x7fd2a006d180 2023-10-11T09:51:43.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.452+0000 7fd29f7fe700 1 -- 172.21.15.78:0/3660260825 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd28c0057e0 con 0x7fd2a006d180 2023-10-11T09:51:43.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.452+0000 7fd29ffff700 1 -- 172.21.15.78:0/3660260825 >> v1:172.21.15.133:6789/0 conn(0x7fd2a006cd00 legacy=0x7fd2a00b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:43.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.452+0000 7fd29ffff700 1 -- 172.21.15.78:0/3660260825 >> v1:172.21.15.78:6789/0 conn(0x7fd2a006d180 legacy=0x7fd2a00b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:43.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.452+0000 7fd29ffff700 1 -- 172.21.15.78:0/3660260825 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a006d600 con 0x7fd2a013f740 2023-10-11T09:51:43.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.452+0000 7fd29ffff700 1 --2- 172.21.15.78:0/3660260825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2a013f740 0x7fd2a00ccab0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd290009850 tx=0x7fd290009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=393a74cdf4abeb8c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:43.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.452+0000 7fd29f7fe700 1 -- 172.21.15.78:0/3660260825 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd290011830 con 0x7fd2a013f740 2023-10-11T09:51:43.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.452+0000 7fd29f7fe700 1 -- 172.21.15.78:0/3660260825 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd290011990 con 0x7fd2a013f740 2023-10-11T09:51:43.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.452+0000 7fd29f7fe700 1 -- 172.21.15.78:0/3660260825 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd290011d40 con 0x7fd2a013f740 2023-10-11T09:51:43.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.452+0000 7fd2a5245700 1 -- 172.21.15.78:0/3660260825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2a013f740 msgr2=0x7fd2a00ccab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:43.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.452+0000 7fd2a5245700 1 --2- 172.21.15.78:0/3660260825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2a013f740 0x7fd2a00ccab0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd290009850 tx=0x7fd290009b60 comp rx=0 tx=0).stop 2023-10-11T09:51:43.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.456+0000 7fd2a5245700 1 -- 172.21.15.78:0/3660260825 shutdown_connections 2023-10-11T09:51:43.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.456+0000 7fd2a5245700 1 --2- 172.21.15.78:0/3660260825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2a013f740 0x7fd2a00ccab0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:43.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.456+0000 7fd2a5245700 1 -- 172.21.15.78:0/3660260825 >> 172.21.15.78:0/3660260825 conn(0x7fd2a00cea60 msgr2=0x7fd2a00cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:43.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.456+0000 7fd2a5245700 1 -- 172.21.15.78:0/3660260825 shutdown_connections 2023-10-11T09:51:43.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.456+0000 7fd2a5245700 1 -- 172.21.15.78:0/3660260825 wait complete. 2023-10-11T09:51:43.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.456+0000 7fd2a5245700 1 Processor -- start 2023-10-11T09:51:43.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.456+0000 7fd2a5245700 1 -- start start 2023-10-11T09:51:43.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.460+0000 7fd2a5245700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd2a006cd00 0x7fd2a013ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:43.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.460+0000 7fd2a5245700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2a006d180 0x7fd2a00b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:43.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.460+0000 7fd2a5245700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2a013f740 0x7fd2a015b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:43.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.460+0000 7fd2a5245700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a00b1140 con 0x7fd2a013f740 2023-10-11T09:51:43.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.460+0000 7fd2a5245700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a013cf80 con 0x7fd2a006d180 2023-10-11T09:51:43.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.460+0000 7fd2a5245700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a00b2770 con 0x7fd2a006cd00 2023-10-11T09:51:43.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.460+0000 7fd2a4a44700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2a013f740 0x7fd2a015b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:43.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.460+0000 7fd2a4a44700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2a013f740 0x7fd2a015b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43462/0 (socket says 172.21.15.78:43462) 2023-10-11T09:51:43.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.460+0000 7fd2a4a44700 1 -- 172.21.15.78:0/1557284638 learned_addr learned my addr 172.21.15.78:0/1557284638 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:43.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.460+0000 7fd29ffff700 1 --2- 172.21.15.78:0/1557284638 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd2a006cd00 0x7fd2a013ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:43.472 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.460+0000 7fd2977fe700 1 --2- 172.21.15.78:0/1557284638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2a006d180 0x7fd2a00b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:43.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.460+0000 7fd2a4a44700 1 -- 172.21.15.78:0/1557284638 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd2a006cd00 msgr2=0x7fd2a013ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:43.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.460+0000 7fd2a4a44700 1 --2- 172.21.15.78:0/1557284638 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd2a006cd00 0x7fd2a013ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:43.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.460+0000 7fd2a4a44700 1 -- 172.21.15.78:0/1557284638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2a006d180 msgr2=0x7fd2a00b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:43.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.460+0000 7fd2a4a44700 1 --2- 172.21.15.78:0/1557284638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2a006d180 0x7fd2a00b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:43.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.460+0000 7fd2a4a44700 1 -- 172.21.15.78:0/1557284638 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2900094b0 con 0x7fd2a013f740 2023-10-11T09:51:43.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.460+0000 7fd2977fe700 1 --2- 172.21.15.78:0/1557284638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2a006d180 0x7fd2a00b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:51:43.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.460+0000 7fd2a4a44700 1 --2- 172.21.15.78:0/1557284638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2a013f740 0x7fd2a015b970 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fd29800caa0 tx=0x7fd29800cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:43.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.460+0000 7fd29d7fa700 1 -- 172.21.15.78:0/1557284638 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd298007480 con 0x7fd2a013f740 2023-10-11T09:51:43.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.460+0000 7fd29d7fa700 1 -- 172.21.15.78:0/1557284638 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2980075e0 con 0x7fd2a013f740 2023-10-11T09:51:43.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.460+0000 7fd29d7fa700 1 -- 172.21.15.78:0/1557284638 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd298007970 con 0x7fd2a013f740 2023-10-11T09:51:43.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.460+0000 7fd2a5245700 1 -- 172.21.15.78:0/1557284638 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a00b2a50 con 0x7fd2a013f740 2023-10-11T09:51:43.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.460+0000 7fd2a5245700 1 -- 172.21.15.78:0/1557284638 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a00b3010 con 0x7fd2a013f740 2023-10-11T09:51:43.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.464+0000 7fd2a5245700 1 -- 172.21.15.78:0/1557284638 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2a0065d40 con 0x7fd2a013f740 2023-10-11T09:51:43.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.464+0000 7fd29d7fa700 1 -- 172.21.15.78:0/1557284638 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fd29801b3b0 con 0x7fd2a013f740 2023-10-11T09:51:43.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.464+0000 7fd29d7fa700 1 --2- 172.21.15.78:0/1557284638 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd280065f00 0x7fd2800683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:43.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.464+0000 7fd29d7fa700 1 -- 172.21.15.78:0/1557284638 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd298083160 con 0x7fd2a013f740 2023-10-11T09:51:43.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.468+0000 7fd29ffff700 1 --2- 172.21.15.78:0/1557284638 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd280065f00 0x7fd2800683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:43.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.468+0000 7fd29ffff700 1 --2- 172.21.15.78:0/1557284638 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd280065f00 0x7fd2800683b0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd2900090b0 tx=0x7fd290008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:43.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.472+0000 7fd29d7fa700 1 -- 172.21.15.78:0/1557284638 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fd2980507a0 con 0x7fd2a013f740 2023-10-11T09:51:43.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.680+0000 7fd2a5245700 1 -- 172.21.15.78:0/1557284638 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2a0159870 con 0x7fd2a013f740 2023-10-11T09:51:43.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.684+0000 7fd29d7fa700 1 -- 172.21.15.78:0/1557284638 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4656 (secure 0 0 0) 0x7fd298053dc0 con 0x7fd2a013f740 2023-10-11T09:51:43.693 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:43.694 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":99,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8452,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/2481257990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2481257990},{"type":"v1","addr":"172.21.15.78:6836","nonce":2481257990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":8464,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1035227282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1035227282},{"type":"v1","addr":"172.21.15.133:6835","nonce":1035227282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":8880,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/4189411531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":4189411531},{"type":"v1","addr":"172.21.15.78:6837","nonce":4189411531}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:51:40.211859+0000","modified":"2023-10-11T09:51:43.234832+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8883},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8883":{"gid":8883,"name":"d","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6833/1433079498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1433079498},{"type":"v1","addr":"172.21.15.133:6833","nonce":1433079498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-11T09:51:43.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.684+0000 7fd2a5245700 1 -- 172.21.15.78:0/1557284638 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd280065f00 msgr2=0x7fd2800683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:43.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.684+0000 7fd2a5245700 1 --2- 172.21.15.78:0/1557284638 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd280065f00 0x7fd2800683b0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd2900090b0 tx=0x7fd290008030 comp rx=0 tx=0).stop 2023-10-11T09:51:43.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.684+0000 7fd2a5245700 1 -- 172.21.15.78:0/1557284638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2a013f740 msgr2=0x7fd2a015b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:43.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.684+0000 7fd2a5245700 1 --2- 172.21.15.78:0/1557284638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2a013f740 0x7fd2a015b970 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fd29800caa0 tx=0x7fd29800cad0 comp rx=0 tx=0).stop 2023-10-11T09:51:43.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.688+0000 7fd2a5245700 1 -- 172.21.15.78:0/1557284638 shutdown_connections 2023-10-11T09:51:43.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.688+0000 7fd2a5245700 1 --2- 172.21.15.78:0/1557284638 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd280065f00 0x7fd2800683b0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:43.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.688+0000 7fd2a5245700 1 --2- 172.21.15.78:0/1557284638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2a013f740 0x7fd2a015b970 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:43.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.688+0000 7fd2a5245700 1 --2- 172.21.15.78:0/1557284638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2a006d180 0x7fd2a00b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:43.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.688+0000 7fd2a5245700 1 --2- 172.21.15.78:0/1557284638 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd2a006cd00 0x7fd2a013ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:43.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.688+0000 7fd2a5245700 1 -- 172.21.15.78:0/1557284638 >> 172.21.15.78:0/1557284638 conn(0x7fd2a00cea60 msgr2=0x7fd2a006bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:43.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.688+0000 7fd2a5245700 1 -- 172.21.15.78:0/1557284638 shutdown_connections 2023-10-11T09:51:43.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.688+0000 7fd2a5245700 1 -- 172.21.15.78:0/1557284638 wait complete. 2023-10-11T09:51:43.699 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 99 2023-10-11T09:51:43.720 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:51:43.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.884+0000 7fa006112700 1 Processor -- start 2023-10-11T09:51:43.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.884+0000 7fa006112700 1 -- start start 2023-10-11T09:51:43.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.884+0000 7fa006112700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa000142770 0x7fa00006c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:43.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.884+0000 7fa006112700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa00006ff00 0x7fa0000cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:43.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.884+0000 7fa006112700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fa000070350 0x7fa0000cd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:43.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.884+0000 7fa006112700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0000ce3b0 con 0x7fa000070350 2023-10-11T09:51:43.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.884+0000 7fa006112700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0000cc870 con 0x7fa000142770 2023-10-11T09:51:43.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.884+0000 7fa006112700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0000cce30 con 0x7fa00006ff00 2023-10-11T09:51:43.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.884+0000 7f9ffffff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fa000070350 0x7fa0000cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:43.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.884+0000 7f9ffffff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fa000070350 0x7fa0000cd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43468/0 (socket says 172.21.15.78:43468) 2023-10-11T09:51:43.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.884+0000 7f9ffffff700 1 -- 172.21.15.78:0/2664223470 learned_addr learned my addr 172.21.15.78:0/2664223470 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:43.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.884+0000 7f9ff7fff700 1 --2- 172.21.15.78:0/2664223470 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa00006ff00 0x7fa0000cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:43.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.884+0000 7f9fff7fe700 1 --2- 172.21.15.78:0/2664223470 >> v2:172.21.15.133:3300/0 conn(0x7fa000142770 0x7fa00006c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:43.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.888+0000 7f9ffffff700 1 -- 172.21.15.78:0/2664223470 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa00006ff00 msgr2=0x7fa0000cd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:43.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.888+0000 7f9ffffff700 1 --2- 172.21.15.78:0/2664223470 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa00006ff00 0x7fa0000cd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:43.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.888+0000 7f9ffffff700 1 -- 172.21.15.78:0/2664223470 >> v2:172.21.15.133:3300/0 conn(0x7fa000142770 msgr2=0x7fa00006c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:43.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.888+0000 7f9ffffff700 1 --2- 172.21.15.78:0/2664223470 >> v2:172.21.15.133:3300/0 conn(0x7fa000142770 0x7fa00006c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:43.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.888+0000 7f9ffffff700 1 -- 172.21.15.78:0/2664223470 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0000ce600 con 0x7fa000070350 2023-10-11T09:51:43.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.888+0000 7f9fff7fe700 1 --2- 172.21.15.78:0/2664223470 >> v2:172.21.15.133:3300/0 conn(0x7fa000142770 0x7fa00006c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:51:43.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.888+0000 7f9ffffff700 1 --2- 172.21.15.78:0/2664223470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa000070350 0x7fa0000cd800 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fa0000b19c0 tx=0x7f9ff800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e5347ae41c2193c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:43.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.888+0000 7f9ffeffd700 1 -- 172.21.15.78:0/2664223470 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff800e070 con 0x7fa000070350 2023-10-11T09:51:43.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.888+0000 7f9ffeffd700 1 -- 172.21.15.78:0/2664223470 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff800b910 con 0x7fa000070350 2023-10-11T09:51:43.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.888+0000 7f9ffeffd700 1 -- 172.21.15.78:0/2664223470 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff800bcc0 con 0x7fa000070350 2023-10-11T09:51:43.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.888+0000 7fa006112700 1 -- 172.21.15.78:0/2664223470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa000070350 msgr2=0x7fa0000cd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:43.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.888+0000 7fa006112700 1 --2- 172.21.15.78:0/2664223470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa000070350 0x7fa0000cd800 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fa0000b19c0 tx=0x7f9ff800b330 comp rx=0 tx=0).stop 2023-10-11T09:51:43.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.888+0000 7fa006112700 1 -- 172.21.15.78:0/2664223470 shutdown_connections 2023-10-11T09:51:43.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.888+0000 7fa006112700 1 --2- 172.21.15.78:0/2664223470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa000070350 0x7fa0000cd800 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:43.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.888+0000 7fa006112700 1 --2- 172.21.15.78:0/2664223470 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa00006ff00 0x7fa0000cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:43.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.888+0000 7fa006112700 1 --2- 172.21.15.78:0/2664223470 >> v2:172.21.15.133:3300/0 conn(0x7fa000142770 0x7fa00006c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:43.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.888+0000 7fa006112700 1 -- 172.21.15.78:0/2664223470 >> 172.21.15.78:0/2664223470 conn(0x7fa00006aca0 msgr2=0x7fa0000b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:43.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.888+0000 7fa006112700 1 -- 172.21.15.78:0/2664223470 shutdown_connections 2023-10-11T09:51:43.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.888+0000 7fa006112700 1 -- 172.21.15.78:0/2664223470 wait complete. 2023-10-11T09:51:43.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.892+0000 7fa006112700 1 Processor -- start 2023-10-11T09:51:43.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.892+0000 7fa006112700 1 -- start start 2023-10-11T09:51:43.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.892+0000 7fa006112700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa00006ff00 0x7fa0001f2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:43.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.892+0000 7fa006112700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa000070350 0x7fa00013f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:43.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.892+0000 7fa006112700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa000142770 0x7fa0001f5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:43.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.892+0000 7fa006112700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0001f2210 con 0x7fa000070350 2023-10-11T09:51:43.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.892+0000 7fa006112700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0001f3160 con 0x7fa00006ff00 2023-10-11T09:51:43.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.892+0000 7fa006112700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0001f6250 con 0x7fa000142770 2023-10-11T09:51:43.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.892+0000 7f9ff7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa000070350 0x7fa00013f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:43.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.892+0000 7f9ff7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa000070350 0x7fa00013f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43472/0 (socket says 172.21.15.78:43472) 2023-10-11T09:51:43.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.892+0000 7f9ff7fff700 1 -- 172.21.15.78:0/2490398544 learned_addr learned my addr 172.21.15.78:0/2490398544 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:43.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.892+0000 7f9ffffff700 1 --2- 172.21.15.78:0/2490398544 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa000142770 0x7fa0001f5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:43.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.892+0000 7f9fff7fe700 1 --2- 172.21.15.78:0/2490398544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa00006ff00 0x7fa0001f2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:43.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.892+0000 7f9ffffff700 1 -- 172.21.15.78:0/2490398544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa00006ff00 msgr2=0x7fa0001f2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:43.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.892+0000 7f9ffffff700 1 --2- 172.21.15.78:0/2490398544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa00006ff00 0x7fa0001f2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:43.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.892+0000 7f9ffffff700 1 -- 172.21.15.78:0/2490398544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa000070350 msgr2=0x7fa00013f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:43.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.892+0000 7f9ffffff700 1 --2- 172.21.15.78:0/2490398544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa000070350 0x7fa00013f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:43.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.892+0000 7f9ffffff700 1 -- 172.21.15.78:0/2490398544 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff8009b30 con 0x7fa000142770 2023-10-11T09:51:43.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.892+0000 7f9ff7fff700 1 --2- 172.21.15.78:0/2490398544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa000070350 0x7fa00013f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:51:43.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.896+0000 7f9ffffff700 1 --2- 172.21.15.78:0/2490398544 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa000142770 0x7fa0001f5790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8015000 tx=0x7f9ff8004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:43.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.896+0000 7f9ffcff9700 1 -- 172.21.15.78:0/2490398544 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff80099b0 con 0x7fa000142770 2023-10-11T09:51:43.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.896+0000 7f9ffcff9700 1 -- 172.21.15.78:0/2490398544 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff80092e0 con 0x7fa000142770 2023-10-11T09:51:43.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.896+0000 7f9ffcff9700 1 -- 172.21.15.78:0/2490398544 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff8004470 con 0x7fa000142770 2023-10-11T09:51:43.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.896+0000 7fa006112700 1 -- 172.21.15.78:0/2490398544 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0001f6530 con 0x7fa000142770 2023-10-11T09:51:43.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.896+0000 7fa006112700 1 -- 172.21.15.78:0/2490398544 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0001f2700 con 0x7fa000142770 2023-10-11T09:51:43.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.896+0000 7fa006112700 1 -- 172.21.15.78:0/2490398544 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa00004e530 con 0x7fa000142770 2023-10-11T09:51:43.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.900+0000 7f9ffcff9700 1 -- 172.21.15.78:0/2490398544 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f9ff80045d0 con 0x7fa000142770 2023-10-11T09:51:43.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.900+0000 7f9ffcff9700 1 --2- 172.21.15.78:0/2490398544 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9fe0065ed0 0x7f9fe0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:43.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.900+0000 7f9ffcff9700 1 -- 172.21.15.78:0/2490398544 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9ff8084c10 con 0x7fa000142770 2023-10-11T09:51:43.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.900+0000 7f9fff7fe700 1 --2- 172.21.15.78:0/2490398544 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9fe0065ed0 0x7f9fe0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:43.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.900+0000 7f9fff7fe700 1 --2- 172.21.15.78:0/2490398544 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9fe0065ed0 0x7f9fe0068380 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fa0000ce300 tx=0x7f9ff0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:43.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:43.912+0000 7f9ffcff9700 1 -- 172.21.15.78:0/2490398544 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f9ff80521d0 con 0x7fa000142770 2023-10-11T09:51:44.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.084+0000 7fa006112700 1 -- 172.21.15.78:0/2490398544 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa00006d050 con 0x7fa000142770 2023-10-11T09:51:44.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.092+0000 7f9ffcff9700 1 -- 172.21.15.78:0/2490398544 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+17893 (secure 0 0 0) 0x7f9ff8020090 con 0x7fa000142770 2023-10-11T09:51:44.099 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:44.100 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":127,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:51:41.218018+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:51:38.592892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-11T09:51:39.204588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:51:44.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.092+0000 7fa006112700 1 -- 172.21.15.78:0/2490398544 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9fe0065ed0 msgr2=0x7f9fe0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:44.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.092+0000 7fa006112700 1 --2- 172.21.15.78:0/2490398544 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9fe0065ed0 0x7f9fe0068380 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fa0000ce300 tx=0x7f9ff0008040 comp rx=0 tx=0).stop 2023-10-11T09:51:44.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.092+0000 7fa006112700 1 -- 172.21.15.78:0/2490398544 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa000142770 msgr2=0x7fa0001f5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:44.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.092+0000 7fa006112700 1 --2- 172.21.15.78:0/2490398544 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa000142770 0x7fa0001f5790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8015000 tx=0x7f9ff8004af0 comp rx=0 tx=0).stop 2023-10-11T09:51:44.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.092+0000 7fa006112700 1 -- 172.21.15.78:0/2490398544 shutdown_connections 2023-10-11T09:51:44.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.092+0000 7fa006112700 1 --2- 172.21.15.78:0/2490398544 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f9fe0065ed0 0x7f9fe0068380 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:44.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.092+0000 7fa006112700 1 --2- 172.21.15.78:0/2490398544 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa000142770 0x7fa0001f5790 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:44.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.092+0000 7fa006112700 1 --2- 172.21.15.78:0/2490398544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa000070350 0x7fa00013f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:44.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.092+0000 7fa006112700 1 --2- 172.21.15.78:0/2490398544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa00006ff00 0x7fa0001f2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:44.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.092+0000 7fa006112700 1 -- 172.21.15.78:0/2490398544 >> 172.21.15.78:0/2490398544 conn(0x7fa00006aca0 msgr2=0x7fa0000b1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:44.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.092+0000 7fa006112700 1 -- 172.21.15.78:0/2490398544 shutdown_connections 2023-10-11T09:51:44.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.092+0000 7fa006112700 1 -- 172.21.15.78:0/2490398544 wait complete. 2023-10-11T09:51:44.124 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-11T09:51:44.125 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-11T09:51:44.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.284+0000 7f1e60a50700 1 Processor -- start 2023-10-11T09:51:44.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.284+0000 7f1e60a50700 1 -- start start 2023-10-11T09:51:44.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.288+0000 7f1e60a50700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1e5c0cda10 0x7f1e5c0caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.288+0000 7f1e60a50700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f1e5c164d20 0x7f1e5c06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.288+0000 7f1e60a50700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e5c165390 con 0x7f1e5c0cdde0 2023-10-11T09:51:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.288+0000 7f1e60a50700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e5c1651a0 con 0x7f1e5c164d20 2023-10-11T09:51:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.288+0000 7f1e60a50700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e5c0ca440 con 0x7f1e5c0cda10 2023-10-11T09:51:44.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.288+0000 7f1e53fff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f1e5c0cdde0 0x7f1e5c164610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:46318/0 (socket says 172.21.15.78:46318) 2023-10-11T09:51:44.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.288+0000 7f1e53fff700 1 -- 172.21.15.78:0/259612276 learned_addr learned my addr 172.21.15.78:0/259612276 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:51:44.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.288+0000 7f1e5b59e700 1 --2- 172.21.15.78:0/259612276 >> v2:172.21.15.78:3300/0 conn(0x7f1e5c164d20 0x7f1e5c06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:44.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.288+0000 7f1e5ad9d700 1 --2- 172.21.15.78:0/259612276 >> v2:172.21.15.133:3300/0 conn(0x7f1e5c0cda10 0x7f1e5c0caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:44.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.288+0000 7f1e5b59e700 1 -- 172.21.15.78:0/259612276 >> v2:172.21.15.133:3300/0 conn(0x7f1e5c0cda10 msgr2=0x7f1e5c0caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:44.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.288+0000 7f1e5b59e700 1 --2- 172.21.15.78:0/259612276 >> v2:172.21.15.133:3300/0 conn(0x7f1e5c0cda10 0x7f1e5c0caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:44.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.288+0000 7f1e5b59e700 1 -- 172.21.15.78:0/259612276 >> v1:172.21.15.78:6789/0 conn(0x7f1e5c0cdde0 legacy=0x7f1e5c164610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:44.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.288+0000 7f1e5b59e700 1 -- 172.21.15.78:0/259612276 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e5c0ca720 con 0x7f1e5c164d20 2023-10-11T09:51:44.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.288+0000 7f1e5a59c700 1 -- 172.21.15.78:0/259612276 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1412350767 0 0) 0x7f1e5c165390 con 0x7f1e5c0cdde0 2023-10-11T09:51:44.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.288+0000 7f1e5b59e700 1 --2- 172.21.15.78:0/259612276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e5c164d20 0x7f1e5c06bbf0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c06afb0 tx=0x7f1e5400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7548ccaa27c7e7b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:44.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.288+0000 7f1e5a59c700 1 -- 172.21.15.78:0/259612276 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5400e070 con 0x7f1e5c164d20 2023-10-11T09:51:44.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.288+0000 7f1e5a59c700 1 -- 172.21.15.78:0/259612276 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e5400bbd0 con 0x7f1e5c164d20 2023-10-11T09:51:44.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.288+0000 7f1e5a59c700 1 -- 172.21.15.78:0/259612276 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e54012430 con 0x7f1e5c164d20 2023-10-11T09:51:44.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.292+0000 7f1e60a50700 1 -- 172.21.15.78:0/259612276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e5c164d20 msgr2=0x7f1e5c06bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:44.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.292+0000 7f1e60a50700 1 --2- 172.21.15.78:0/259612276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e5c164d20 0x7f1e5c06bbf0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c06afb0 tx=0x7f1e5400b330 comp rx=0 tx=0).stop 2023-10-11T09:51:44.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.292+0000 7f1e60a50700 1 -- 172.21.15.78:0/259612276 shutdown_connections 2023-10-11T09:51:44.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.292+0000 7f1e60a50700 1 --2- 172.21.15.78:0/259612276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e5c164d20 0x7f1e5c06bbf0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:44.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.292+0000 7f1e60a50700 1 --2- 172.21.15.78:0/259612276 >> v2:172.21.15.133:3300/0 conn(0x7f1e5c0cda10 0x7f1e5c0caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:44.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.292+0000 7f1e60a50700 1 -- 172.21.15.78:0/259612276 >> 172.21.15.78:0/259612276 conn(0x7f1e5c0cea60 msgr2=0x7f1e5c0cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:44.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.292+0000 7f1e60a50700 1 -- 172.21.15.78:0/259612276 shutdown_connections 2023-10-11T09:51:44.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.292+0000 7f1e60a50700 1 -- 172.21.15.78:0/259612276 wait complete. 2023-10-11T09:51:44.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.292+0000 7f1e60a50700 1 Processor -- start 2023-10-11T09:51:44.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.292+0000 7f1e60a50700 1 -- start start 2023-10-11T09:51:44.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.296+0000 7f1e60a50700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e5c0cda10 0x7f1e5c163730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:44.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.296+0000 7f1e60a50700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e5c0cdde0 0x7f1e5c06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:44.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.296+0000 7f1e60a50700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e5c164d20 0x7f1e5c06c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:44.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.296+0000 7f1e60a50700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c1f7b90 con 0x7f1e5c0cdde0 2023-10-11T09:51:44.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.296+0000 7f1e60a50700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c0b38b0 con 0x7f1e5c164d20 2023-10-11T09:51:44.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.296+0000 7f1e60a50700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c1f7540 con 0x7f1e5c0cda10 2023-10-11T09:51:44.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.296+0000 7f1e53fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e5c0cdde0 0x7f1e5c06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:44.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.296+0000 7f1e53fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e5c0cdde0 0x7f1e5c06bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43478/0 (socket says 172.21.15.78:43478) 2023-10-11T09:51:44.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.296+0000 7f1e53fff700 1 -- 172.21.15.78:0/1925675805 learned_addr learned my addr 172.21.15.78:0/1925675805 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:44.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.296+0000 7f1e5ad9d700 1 --2- 172.21.15.78:0/1925675805 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e5c0cda10 0x7f1e5c163730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:44.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.296+0000 7f1e5b59e700 1 --2- 172.21.15.78:0/1925675805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e5c164d20 0x7f1e5c06c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:44.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.296+0000 7f1e53fff700 1 -- 172.21.15.78:0/1925675805 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e5c0cda10 msgr2=0x7f1e5c163730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:44.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.296+0000 7f1e53fff700 1 --2- 172.21.15.78:0/1925675805 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e5c0cda10 0x7f1e5c163730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:44.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.296+0000 7f1e53fff700 1 -- 172.21.15.78:0/1925675805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e5c164d20 msgr2=0x7f1e5c06c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:44.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.296+0000 7f1e53fff700 1 --2- 172.21.15.78:0/1925675805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e5c164d20 0x7f1e5c06c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:44.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.296+0000 7f1e53fff700 1 -- 172.21.15.78:0/1925675805 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e54009a70 con 0x7f1e5c0cdde0 2023-10-11T09:51:44.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.296+0000 7f1e53fff700 1 --2- 172.21.15.78:0/1925675805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e5c0cdde0 0x7f1e5c06bbf0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f1e4400c4e0 tx=0x7f1e4400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:44.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.296+0000 7f1e537fe700 1 -- 172.21.15.78:0/1925675805 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e44004ca0 con 0x7f1e5c0cdde0 2023-10-11T09:51:44.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.296+0000 7f1e537fe700 1 -- 172.21.15.78:0/1925675805 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e44004e20 con 0x7f1e5c0cdde0 2023-10-11T09:51:44.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.296+0000 7f1e60a50700 1 -- 172.21.15.78:0/1925675805 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e5c1f7820 con 0x7f1e5c0cdde0 2023-10-11T09:51:44.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.296+0000 7f1e537fe700 1 -- 172.21.15.78:0/1925675805 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e44007310 con 0x7f1e5c0cdde0 2023-10-11T09:51:44.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.296+0000 7f1e60a50700 1 -- 172.21.15.78:0/1925675805 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e5c0b2b40 con 0x7f1e5c0cdde0 2023-10-11T09:51:44.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.300+0000 7f1e60a50700 1 -- 172.21.15.78:0/1925675805 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e5c1f7d40 con 0x7f1e5c0cdde0 2023-10-11T09:51:44.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.300+0000 7f1e537fe700 1 -- 172.21.15.78:0/1925675805 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f1e44007470 con 0x7f1e5c0cdde0 2023-10-11T09:51:44.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.300+0000 7f1e537fe700 1 --2- 172.21.15.78:0/1925675805 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1e3c065ed0 0x7f1e3c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:44.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.300+0000 7f1e537fe700 1 -- 172.21.15.78:0/1925675805 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1e440834e0 con 0x7f1e5c0cdde0 2023-10-11T09:51:44.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.304+0000 7f1e5ad9d700 1 --2- 172.21.15.78:0/1925675805 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1e3c065ed0 0x7f1e3c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:44.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.304+0000 7f1e5ad9d700 1 --2- 172.21.15.78:0/1925675805 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1e3c065ed0 0x7f1e3c068380 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c152b20 tx=0x7f1e4c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:44.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.308+0000 7f1e537fe700 1 -- 172.21.15.78:0/1925675805 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f1e5c1f7d40 con 0x7f1e5c0cdde0 2023-10-11T09:51:44.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.480+0000 7f1e60a50700 1 -- 172.21.15.78:0/1925675805 --> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1e5c0caa00 con 0x7f1e3c065ed0 2023-10-11T09:51:44.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.492+0000 7f1e537fe700 1 -- 172.21.15.78:0/1925675805 <== mgr.4101 v2:172.21.15.78:6832/35606 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+294131 (secure 0 0 0) 0x7f1e4c010740 con 0x7f1e3c065ed0 2023-10-11T09:51:44.504 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:44.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.500+0000 7f1e60a50700 1 -- 172.21.15.78:0/1925675805 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1e3c065ed0 msgr2=0x7f1e3c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:44.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.500+0000 7f1e60a50700 1 --2- 172.21.15.78:0/1925675805 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1e3c065ed0 0x7f1e3c068380 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c152b20 tx=0x7f1e4c008040 comp rx=0 tx=0).stop 2023-10-11T09:51:44.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.500+0000 7f1e60a50700 1 -- 172.21.15.78:0/1925675805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e5c0cdde0 msgr2=0x7f1e5c06bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:44.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.500+0000 7f1e60a50700 1 --2- 172.21.15.78:0/1925675805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e5c0cdde0 0x7f1e5c06bbf0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f1e4400c4e0 tx=0x7f1e4400c7f0 comp rx=0 tx=0).stop 2023-10-11T09:51:44.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.500+0000 7f1e60a50700 1 -- 172.21.15.78:0/1925675805 shutdown_connections 2023-10-11T09:51:44.512 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.500+0000 7f1e60a50700 1 --2- 172.21.15.78:0/1925675805 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1e3c065ed0 0x7f1e3c068380 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:44.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.500+0000 7f1e60a50700 1 --2- 172.21.15.78:0/1925675805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e5c164d20 0x7f1e5c06c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:44.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.500+0000 7f1e60a50700 1 --2- 172.21.15.78:0/1925675805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e5c0cdde0 0x7f1e5c06bbf0 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:44.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.500+0000 7f1e60a50700 1 --2- 172.21.15.78:0/1925675805 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e5c0cda10 0x7f1e5c163730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:44.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.500+0000 7f1e60a50700 1 -- 172.21.15.78:0/1925675805 >> 172.21.15.78:0/1925675805 conn(0x7f1e5c0cea60 msgr2=0x7f1e5c0705a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:44.513 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.500+0000 7f1e60a50700 1 -- 172.21.15.78:0/1925675805 shutdown_connections 2023-10-11T09:51:44.514 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.500+0000 7f1e60a50700 1 -- 172.21.15.78:0/1925675805 wait complete. 2023-10-11T09:51:44.514 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2023-10-11T09:51:44.532 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":457,"stamp":"2023-10-11T09:51:42.562021+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":261480,"kb_used_data":13096,"kb_used_omap":0,"kb_used_meta":248320,"kb_avail":754713240,"statfs":{"total":773094113280,"available":772826357760,"internally_reserved":0,"allocated":13410304,"data_stored":7113376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":254279680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001092"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:38.716631+0000","last_change":"2023-10-11T09:51:38.716631+0000","last_active":"2023-10-11T09:51:38.716631+0000","last_peered":"2023-10-11T09:51:38.716631+0000","last_clean":"2023-10-11T09:51:38.716631+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:38.716631+0000","last_undegraded":"2023-10-11T09:51:38.716631+0000","last_fullsized":"2023-10-11T09:51:38.716631+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:38.716631+0000","last_change":"2023-10-11T09:51:38.716631+0000","last_active":"2023-10-11T09:51:38.716631+0000","last_peered":"2023-10-11T09:51:38.716631+0000","last_clean":"2023-10-11T09:51:38.716631+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:38.716631+0000","last_undegraded":"2023-10-11T09:51:38.716631+0000","last_fullsized":"2023-10-11T09:51:38.716631+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:38.716631+0000","last_change":"2023-10-11T09:51:38.716631+0000","last_active":"2023-10-11T09:51:38.716631+0000","last_peered":"2023-10-11T09:51:38.716631+0000","last_clean":"2023-10-11T09:51:38.716631+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:38.716631+0000","last_undegraded":"2023-10-11T09:51:38.716631+0000","last_fullsized":"2023-10-11T09:51:38.716631+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:38.716631+0000","last_change":"2023-10-11T09:51:38.716631+0000","last_active":"2023-10-11T09:51:38.716631+0000","last_peered":"2023-10-11T09:51:38.716631+0000","last_clean":"2023-10-11T09:51:38.716631+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:38.716631+0000","last_undegraded":"2023-10-11T09:51:38.716631+0000","last_fullsized":"2023-10-11T09:51:38.716631+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:38.716631+0000","last_change":"2023-10-11T09:51:38.716631+0000","last_active":"2023-10-11T09:51:38.716631+0000","last_peered":"2023-10-11T09:51:38.716631+0000","last_clean":"2023-10-11T09:51:38.716631+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:38.716631+0000","last_undegraded":"2023-10-11T09:51:38.716631+0000","last_fullsized":"2023-10-11T09:51:38.716631+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:38.716631+0000","last_change":"2023-10-11T09:51:38.716631+0000","last_active":"2023-10-11T09:51:38.716631+0000","last_peered":"2023-10-11T09:51:38.716631+0000","last_clean":"2023-10-11T09:51:38.716631+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:38.716631+0000","last_undegraded":"2023-10-11T09:51:38.716631+0000","last_fullsized":"2023-10-11T09:51:38.716631+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:38.716631+0000","last_change":"2023-10-11T09:51:38.716631+0000","last_active":"2023-10-11T09:51:38.716631+0000","last_peered":"2023-10-11T09:51:38.716631+0000","last_clean":"2023-10-11T09:51:38.716631+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:38.716631+0000","last_undegraded":"2023-10-11T09:51:38.716631+0000","last_fullsized":"2023-10-11T09:51:38.716631+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:38.716631+0000","last_change":"2023-10-11T09:51:38.716631+0000","last_active":"2023-10-11T09:51:38.716631+0000","last_peered":"2023-10-11T09:51:38.716631+0000","last_clean":"2023-10-11T09:51:38.716631+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:38.716631+0000","last_undegraded":"2023-10-11T09:51:38.716631+0000","last_fullsized":"2023-10-11T09:51:38.716631+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:38.716631+0000","last_change":"2023-10-11T09:51:38.716631+0000","last_active":"2023-10-11T09:51:38.716631+0000","last_peered":"2023-10-11T09:51:38.716631+0000","last_clean":"2023-10-11T09:51:38.716631+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:38.716631+0000","last_undegraded":"2023-10-11T09:51:38.716631+0000","last_fullsized":"2023-10-11T09:51:38.716631+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:38.716631+0000","last_change":"2023-10-11T09:51:38.716631+0000","last_active":"2023-10-11T09:51:38.716631+0000","last_peered":"2023-10-11T09:51:38.716631+0000","last_clean":"2023-10-11T09:51:38.716631+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:38.716631+0000","last_undegraded":"2023-10-11T09:51:38.716631+0000","last_fullsized":"2023-10-11T09:51:38.716631+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:38.716631+0000","last_change":"2023-10-11T09:51:38.716631+0000","last_active":"2023-10-11T09:51:38.716631+0000","last_peered":"2023-10-11T09:51:38.716631+0000","last_clean":"2023-10-11T09:51:38.716631+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:38.716631+0000","last_undegraded":"2023-10-11T09:51:38.716631+0000","last_fullsized":"2023-10-11T09:51:38.716631+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:38.716631+0000","last_change":"2023-10-11T09:51:38.716631+0000","last_active":"2023-10-11T09:51:38.716631+0000","last_peered":"2023-10-11T09:51:38.716631+0000","last_clean":"2023-10-11T09:51:38.716631+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:38.716631+0000","last_undegraded":"2023-10-11T09:51:38.716631+0000","last_fullsized":"2023-10-11T09:51:38.716631+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:38.716631+0000","last_change":"2023-10-11T09:51:38.716631+0000","last_active":"2023-10-11T09:51:38.716631+0000","last_peered":"2023-10-11T09:51:38.716631+0000","last_clean":"2023-10-11T09:51:38.716631+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:38.716631+0000","last_undegraded":"2023-10-11T09:51:38.716631+0000","last_fullsized":"2023-10-11T09:51:38.716631+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:38.716631+0000","last_change":"2023-10-11T09:51:38.716631+0000","last_active":"2023-10-11T09:51:38.716631+0000","last_peered":"2023-10-11T09:51:38.716631+0000","last_clean":"2023-10-11T09:51:38.716631+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:38.716631+0000","last_undegraded":"2023-10-11T09:51:38.716631+0000","last_fullsized":"2023-10-11T09:51:38.716631+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:38.716631+0000","last_change":"2023-10-11T09:51:38.716631+0000","last_active":"2023-10-11T09:51:38.716631+0000","last_peered":"2023-10-11T09:51:38.716631+0000","last_clean":"2023-10-11T09:51:38.716631+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:38.716631+0000","last_undegraded":"2023-10-11T09:51:38.716631+0000","last_fullsized":"2023-10-11T09:51:38.716631+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:38.716631+0000","last_change":"2023-10-11T09:51:38.716631+0000","last_active":"2023-10-11T09:51:38.716631+0000","last_peered":"2023-10-11T09:51:38.716631+0000","last_clean":"2023-10-11T09:51:38.716631+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:38.716631+0000","last_undegraded":"2023-10-11T09:51:38.716631+0000","last_fullsized":"2023-10-11T09:51:38.716631+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:38.716631+0000","last_change":"2023-10-11T09:51:38.716631+0000","last_active":"2023-10-11T09:51:38.716631+0000","last_peered":"2023-10-11T09:51:38.716631+0000","last_clean":"2023-10-11T09:51:38.716631+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:38.716631+0000","last_undegraded":"2023-10-11T09:51:38.716631+0000","last_fullsized":"2023-10-11T09:51:38.716631+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:38.716631+0000","last_change":"2023-10-11T09:51:38.716631+0000","last_active":"2023-10-11T09:51:38.716631+0000","last_peered":"2023-10-11T09:51:38.716631+0000","last_clean":"2023-10-11T09:51:38.716631+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:38.716631+0000","last_undegraded":"2023-10-11T09:51:38.716631+0000","last_fullsized":"2023-10-11T09:51:38.716631+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:38.716631+0000","last_change":"2023-10-11T09:51:38.716631+0000","last_active":"2023-10-11T09:51:38.716631+0000","last_peered":"2023-10-11T09:51:38.716631+0000","last_clean":"2023-10-11T09:51:38.716631+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:38.716631+0000","last_undegraded":"2023-10-11T09:51:38.716631+0000","last_fullsized":"2023-10-11T09:51:38.716631+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:38.716631+0000","last_change":"2023-10-11T09:51:38.716631+0000","last_active":"2023-10-11T09:51:38.716631+0000","last_peered":"2023-10-11T09:51:38.716631+0000","last_clean":"2023-10-11T09:51:38.716631+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:38.716631+0000","last_undegraded":"2023-10-11T09:51:38.716631+0000","last_fullsized":"2023-10-11T09:51:38.716631+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:38.716631+0000","last_change":"2023-10-11T09:51:38.716631+0000","last_active":"2023-10-11T09:51:38.716631+0000","last_peered":"2023-10-11T09:51:38.716631+0000","last_clean":"2023-10-11T09:51:38.716631+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:38.716631+0000","last_undegraded":"2023-10-11T09:51:38.716631+0000","last_fullsized":"2023-10-11T09:51:38.716631+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:38.716631+0000","last_change":"2023-10-11T09:51:38.716631+0000","last_active":"2023-10-11T09:51:38.716631+0000","last_peered":"2023-10-11T09:51:38.716631+0000","last_clean":"2023-10-11T09:51:38.716631+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:38.716631+0000","last_undegraded":"2023-10-11T09:51:38.716631+0000","last_fullsized":"2023-10-11T09:51:38.716631+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:38.716631+0000","last_change":"2023-10-11T09:51:38.716631+0000","last_active":"2023-10-11T09:51:38.716631+0000","last_peered":"2023-10-11T09:51:38.716631+0000","last_clean":"2023-10-11T09:51:38.716631+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:38.716631+0000","last_undegraded":"2023-10-11T09:51:38.716631+0000","last_fullsized":"2023-10-11T09:51:38.716631+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:38.716631+0000","last_change":"2023-10-11T09:51:38.716631+0000","last_active":"2023-10-11T09:51:38.716631+0000","last_peered":"2023-10-11T09:51:38.716631+0000","last_clean":"2023-10-11T09:51:38.716631+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:38.716631+0000","last_undegraded":"2023-10-11T09:51:38.716631+0000","last_fullsized":"2023-10-11T09:51:38.716631+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:38.716631+0000","last_change":"2023-10-11T09:51:38.716631+0000","last_active":"2023-10-11T09:51:38.716631+0000","last_peered":"2023-10-11T09:51:38.716631+0000","last_clean":"2023-10-11T09:51:38.716631+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:38.716631+0000","last_undegraded":"2023-10-11T09:51:38.716631+0000","last_fullsized":"2023-10-11T09:51:38.716631+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":90,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-11T09:51:35.066974+0000","last_change":"2023-10-11T09:40:36.151839+0000","last_active":"2023-10-11T09:51:35.066974+0000","last_peered":"2023-10-11T09:51:35.066974+0000","last_clean":"2023-10-11T09:51:35.066974+0000","last_became_active":"2023-10-11T09:40:36.151179+0000","last_became_peered":"2023-10-11T09:40:36.151179+0000","last_unstale":"2023-10-11T09:51:35.066974+0000","last_undegraded":"2023-10-11T09:51:35.066974+0000","last_fullsized":"2023-10-11T09:51:35.066974+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:38.716631+0000","last_change":"2023-10-11T09:51:38.716631+0000","last_active":"2023-10-11T09:51:38.716631+0000","last_peered":"2023-10-11T09:51:38.716631+0000","last_clean":"2023-10-11T09:51:38.716631+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:38.716631+0000","last_undegraded":"2023-10-11T09:51:38.716631+0000","last_fullsized":"2023-10-11T09:51:38.716631+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":97,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-11T09:51:35.817793+0000","last_change":"2023-10-11T09:40:36.150224+0000","last_active":"2023-10-11T09:51:35.817793+0000","last_peered":"2023-10-11T09:51:35.817793+0000","last_clean":"2023-10-11T09:51:35.817793+0000","last_became_active":"2023-10-11T09:40:36.149919+0000","last_became_peered":"2023-10-11T09:40:36.149919+0000","last_unstale":"2023-10-11T09:51:35.817793+0000","last_undegraded":"2023-10-11T09:51:35.817793+0000","last_fullsized":"2023-10-11T09:51:35.817793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:38.716631+0000","last_change":"2023-10-11T09:51:38.716631+0000","last_active":"2023-10-11T09:51:38.716631+0000","last_peered":"2023-10-11T09:51:38.716631+0000","last_clean":"2023-10-11T09:51:38.716631+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:38.716631+0000","last_undegraded":"2023-10-11T09:51:38.716631+0000","last_fullsized":"2023-10-11T09:51:38.716631+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":119,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-11T09:51:35.063051+0000","last_change":"2023-10-11T09:40:36.162174+0000","last_active":"2023-10-11T09:51:35.063051+0000","last_peered":"2023-10-11T09:51:35.063051+0000","last_clean":"2023-10-11T09:51:35.063051+0000","last_became_active":"2023-10-11T09:40:36.161710+0000","last_became_peered":"2023-10-11T09:40:36.161710+0000","last_unstale":"2023-10-11T09:51:35.063051+0000","last_undegraded":"2023-10-11T09:51:35.063051+0000","last_fullsized":"2023-10-11T09:51:35.063051+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:57:03.630878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:38.716631+0000","last_change":"2023-10-11T09:51:38.716631+0000","last_active":"2023-10-11T09:51:38.716631+0000","last_peered":"2023-10-11T09:51:38.716631+0000","last_clean":"2023-10-11T09:51:38.716631+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:38.716631+0000","last_undegraded":"2023-10-11T09:51:38.716631+0000","last_fullsized":"2023-10-11T09:51:38.716631+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":97,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-11T09:51:35.817786+0000","last_change":"2023-10-11T09:40:36.160620+0000","last_active":"2023-10-11T09:51:35.817786+0000","last_peered":"2023-10-11T09:51:35.817786+0000","last_clean":"2023-10-11T09:51:35.817786+0000","last_became_active":"2023-10-11T09:40:36.160171+0000","last_became_peered":"2023-10-11T09:40:36.160171+0000","last_unstale":"2023-10-11T09:51:35.817786+0000","last_undegraded":"2023-10-11T09:51:35.817786+0000","last_fullsized":"2023-10-11T09:51:35.817786+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:38.716631+0000","last_change":"2023-10-11T09:51:38.716631+0000","last_active":"2023-10-11T09:51:38.716631+0000","last_peered":"2023-10-11T09:51:38.716631+0000","last_clean":"2023-10-11T09:51:38.716631+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:38.716631+0000","last_undegraded":"2023-10-11T09:51:38.716631+0000","last_fullsized":"2023-10-11T09:51:38.716631+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":95,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-11T09:51:35.321695+0000","last_change":"2023-10-11T09:40:36.151710+0000","last_active":"2023-10-11T09:51:35.321695+0000","last_peered":"2023-10-11T09:51:35.321695+0000","last_clean":"2023-10-11T09:51:35.321695+0000","last_became_active":"2023-10-11T09:40:36.151219+0000","last_became_peered":"2023-10-11T09:40:36.151219+0000","last_unstale":"2023-10-11T09:51:35.321695+0000","last_undegraded":"2023-10-11T09:51:35.321695+0000","last_fullsized":"2023-10-11T09:51:35.321695+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:38.716631+0000","last_change":"2023-10-11T09:51:38.716631+0000","last_active":"2023-10-11T09:51:38.716631+0000","last_peered":"2023-10-11T09:51:38.716631+0000","last_clean":"2023-10-11T09:51:38.716631+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:38.716631+0000","last_undegraded":"2023-10-11T09:51:38.716631+0000","last_fullsized":"2023-10-11T09:51:38.716631+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":97,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-11T09:51:35.689308+0000","last_change":"2023-10-11T09:40:36.216248+0000","last_active":"2023-10-11T09:51:35.689308+0000","last_peered":"2023-10-11T09:51:35.689308+0000","last_clean":"2023-10-11T09:51:35.689308+0000","last_became_active":"2023-10-11T09:40:36.215890+0000","last_became_peered":"2023-10-11T09:40:36.215890+0000","last_unstale":"2023-10-11T09:51:35.689308+0000","last_undegraded":"2023-10-11T09:51:35.689308+0000","last_fullsized":"2023-10-11T09:51:35.689308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:38.716631+0000","last_change":"2023-10-11T09:51:38.716631+0000","last_active":"2023-10-11T09:51:38.716631+0000","last_peered":"2023-10-11T09:51:38.716631+0000","last_clean":"2023-10-11T09:51:38.716631+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:38.716631+0000","last_undegraded":"2023-10-11T09:51:38.716631+0000","last_fullsized":"2023-10-11T09:51:38.716631+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":119,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-11T09:51:35.063051+0000","last_change":"2023-10-11T09:40:36.149739+0000","last_active":"2023-10-11T09:51:35.063051+0000","last_peered":"2023-10-11T09:51:35.063051+0000","last_clean":"2023-10-11T09:51:35.063051+0000","last_became_active":"2023-10-11T09:40:36.149419+0000","last_became_peered":"2023-10-11T09:40:36.149419+0000","last_unstale":"2023-10-11T09:51:35.063051+0000","last_undegraded":"2023-10-11T09:51:35.063051+0000","last_fullsized":"2023-10-11T09:51:35.063051+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:54:50.640362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:39.729189+0000","last_change":"2023-10-11T09:51:39.729189+0000","last_active":"2023-10-11T09:51:39.729189+0000","last_peered":"2023-10-11T09:51:39.729189+0000","last_clean":"2023-10-11T09:51:39.729189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:39.729189+0000","last_undegraded":"2023-10-11T09:51:39.729189+0000","last_fullsized":"2023-10-11T09:51:39.729189+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T09:51:38.716631+0000","last_change":"2023-10-11T09:51:38.716631+0000","last_active":"2023-10-11T09:51:38.716631+0000","last_peered":"2023-10-11T09:51:38.716631+0000","last_clean":"2023-10-11T09:51:38.716631+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T09:51:38.716631+0000","last_undegraded":"2023-10-11T09:51:38.716631+0000","last_fullsized":"2023-10-11T09:51:38.716631+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":95,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-11T09:51:35.321649+0000","last_change":"2023-10-11T09:40:36.148773+0000","last_active":"2023-10-11T09:51:35.321649+0000","last_peered":"2023-10-11T09:51:35.321649+0000","last_clean":"2023-10-11T09:51:35.321649+0000","last_became_active":"2023-10-11T09:40:36.148489+0000","last_became_peered":"2023-10-11T09:40:36.148489+0000","last_unstale":"2023-10-11T09:51:35.321649+0000","last_undegraded":"2023-10-11T09:51:35.321649+0000","last_fullsized":"2023-10-11T09:51:35.321649+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":272,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-11T09:51:35.063090+0000","last_change":"2023-10-11T09:40:29.874372+0000","last_active":"2023-10-11T09:51:35.063090+0000","last_peered":"2023-10-11T09:51:35.063090+0000","last_clean":"2023-10-11T09:51:35.063090+0000","last_became_active":"2023-10-11T09:40:29.873621+0000","last_became_peered":"2023-10-11T09:40:29.873621+0000","last_unstale":"2023-10-11T09:51:35.063090+0000","last_undegraded":"2023-10-11T09:51:35.063090+0000","last_fullsized":"2023-10-11T09:51:35.063090+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_clean_scrub_stamp":"2023-10-11T09:40:28.742276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:22:07.985311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574986,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33120,"kb_used_data":2072,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338720,"statfs":{"total":96636764160,"available":96602849280,"internally_reserved":0,"allocated":2121728,"data_stored":1331948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:50:51 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.0369999999999999,"15min":1.0960000000000001},"min":{"1min":0.78000000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.554,"5min":1.7390000000000001,"15min":1.903},"last":0.68999999999999995},{"interface":"front","average":{"1min":1.117,"5min":1.0429999999999999,"15min":1.095},"min":{"1min":0.873,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.554,"5min":1.5760000000000001,"15min":1.696},"last":0.54700000000000004}]},{"osd":1,"last update":"Wed Oct 11 09:50:51 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.097,"15min":1.1200000000000001},"min":{"1min":0.72799999999999998,"5min":0.66800000000000004,"15min":0.63100000000000001},"max":{"1min":1.524,"5min":1.645,"15min":2.25},"last":1.143},{"interface":"front","average":{"1min":1.079,"5min":1.038,"15min":1.091},"min":{"1min":0.754,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.3300000000000001,"5min":1.587,"15min":1.696},"last":1.113}]},{"osd":2,"last update":"Wed Oct 11 09:50:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1000000000000001,"15min":1.1100000000000001},"min":{"1min":0.72099999999999997,"5min":0.59499999999999997,"15min":0.58999999999999997},"max":{"1min":1.486,"5min":1.6339999999999999,"15min":1.873},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.236,"5min":1.1679999999999999,"15min":1.1539999999999999},"min":{"1min":1.0129999999999999,"5min":0.66400000000000003,"15min":0.65500000000000003},"max":{"1min":1.4610000000000001,"5min":1.78,"15min":1.78},"last":0.76900000000000002}]},{"osd":3,"last update":"Wed Oct 11 09:50:51 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.2050000000000001,"15min":1.2230000000000001},"min":{"1min":0.94499999999999995,"5min":0.70499999999999996,"15min":0.64900000000000002},"max":{"1min":1.6220000000000001,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.046},{"interface":"front","average":{"1min":1.337,"5min":1.2789999999999999,"15min":1.282},"min":{"1min":0.752,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.744,"5min":1.8720000000000001,"15min":2.1579999999999999},"last":0.96399999999999997}]},{"osd":4,"last update":"Wed Oct 11 09:50:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0449999999999999,"15min":1.1240000000000001},"min":{"1min":0.72899999999999998,"5min":0.64900000000000002,"15min":0.53300000000000003},"max":{"1min":1.323,"5min":1.661,"15min":2.2010000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":1.232,"5min":1.208,"15min":1.2330000000000001},"min":{"1min":0.873,"5min":0.67200000000000004,"15min":0.63500000000000001},"max":{"1min":1.706,"5min":1.768,"15min":1.9339999999999999},"last":1.254}]},{"osd":5,"last update":"Wed Oct 11 09:50:51 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.272,"15min":1.2809999999999999},"min":{"1min":0.90100000000000002,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.196,"15min":1.284},"min":{"1min":0.83899999999999997,"5min":0.78300000000000003,"15min":0.65900000000000003},"max":{"1min":1.718,"5min":1.9450000000000001,"15min":3.0750000000000002},"last":0.82999999999999996}]},{"osd":6,"last update":"Wed Oct 11 09:50:51 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.323,"15min":1.397},"min":{"1min":1.034,"5min":0.77000000000000002,"15min":0.70999999999999996},"max":{"1min":1.6399999999999999,"5min":1.835,"15min":3.137},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3580000000000001,"15min":1.387},"min":{"1min":0.82799999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.696,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.1839999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574985,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32540,"kb_used_data":1492,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339300,"statfs":{"total":96636764160,"available":96603443200,"internally_reserved":0,"allocated":1527808,"data_stored":741580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:50:43 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.127,"15min":1.038},"min":{"1min":0.65700000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.8400000000000001,"5min":7.0730000000000004,"15min":7.0730000000000004},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.177,"15min":1.073},"min":{"1min":0.60399999999999998,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.5229999999999999,"5min":8.8450000000000006,"15min":8.8450000000000006},"last":0.92000000000000004}]},{"osd":1,"last update":"Wed Oct 11 09:50:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.3049999999999999,"15min":1.226},"min":{"1min":0.75800000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.4770000000000001,"5min":15.523,"15min":15.523},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.302,"15min":1.2450000000000001},"min":{"1min":0.79000000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.657,"5min":15.253,"15min":15.253},"last":0.73299999999999998}]},{"osd":2,"last update":"Wed Oct 11 09:50:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.111,"15min":1.137},"min":{"1min":0.873,"5min":0.434,"15min":0.434},"max":{"1min":1.4750000000000001,"5min":2.7919999999999998,"15min":2.7919999999999998},"last":1.147},{"interface":"front","average":{"1min":1.107,"5min":1.151,"15min":1.202},"min":{"1min":0.77800000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.7929999999999999,"5min":5.0990000000000002,"15min":5.0990000000000002},"last":1.1910000000000001}]},{"osd":3,"last update":"Wed Oct 11 09:50:43 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.194,"15min":1.1659999999999999},"min":{"1min":0.79600000000000004,"5min":0.70499999999999996,"15min":0.627},"max":{"1min":1.754,"5min":11.945,"15min":11.945},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.214,"5min":1.1279999999999999,"15min":1.2150000000000001},"min":{"1min":0.84499999999999997,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.5329999999999999,"5min":1.5980000000000001,"15min":3.0819999999999999},"last":0.98599999999999999}]},{"osd":4,"last update":"Wed Oct 11 09:50:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.274,"15min":1.2849999999999999},"min":{"1min":0.58399999999999996,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.542,"5min":7.5599999999999996,"15min":7.5599999999999996},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.202,"5min":1.3149999999999999,"15min":1.3280000000000001},"min":{"1min":0.75900000000000001,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.839,"5min":8.7520000000000007,"15min":8.7520000000000007},"last":1.3080000000000001}]},{"osd":5,"last update":"Wed Oct 11 09:50:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.1830000000000001,"15min":1.2370000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.573,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.26,"15min":1.339},"min":{"1min":0.72099999999999997,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.595,"5min":2.4390000000000001,"15min":2.7290000000000001},"last":1.339}]},{"osd":7,"last update":"Wed Oct 11 09:50:43 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.319,"15min":1.359},"min":{"1min":0.92700000000000005,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.931,"5min":8.7970000000000006,"15min":8.7970000000000006},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.3620000000000001,"15min":1.429},"min":{"1min":0.82099999999999995,"5min":0.625,"15min":0.625},"max":{"1min":1.7430000000000001,"5min":2.641,"15min":2.641},"last":1.375}]}]},{"osd":5,"up_from":13,"seq":55834574985,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32540,"kb_used_data":1492,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339300,"statfs":{"total":96636764160,"available":96603443200,"internally_reserved":0,"allocated":1527808,"data_stored":741580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:50:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":0.999,"15min":1},"min":{"1min":0.70099999999999996,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":1.514},{"interface":"front","average":{"1min":1.044,"5min":1.0469999999999999,"15min":1.008},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.46500000000000002},"max":{"1min":1.536,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":1.1990000000000001}]},{"osd":1,"last update":"Wed Oct 11 09:50:46 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.0780000000000001,"15min":1.008},"min":{"1min":0.70999999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.48,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.5680000000000001},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.111,"15min":1.0780000000000001},"min":{"1min":0.68000000000000005,"5min":0.61599999999999999,"15min":0.437},"max":{"1min":1.5880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.3939999999999999}]},{"osd":2,"last update":"Wed Oct 11 09:50:46 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.171,"15min":1.1819999999999999},"min":{"1min":0.746,"5min":0.746,"15min":0.73899999999999999},"max":{"1min":1.669,"5min":2.133,"15min":2.133},"last":1.488},{"interface":"front","average":{"1min":1.298,"5min":1.204,"15min":1.2150000000000001},"min":{"1min":0.878,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.599}]},{"osd":3,"last update":"Wed Oct 11 09:50:46 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.1299999999999999,"15min":1.127},"min":{"1min":0.752,"5min":0.59799999999999998,"15min":0.56999999999999995},"max":{"1min":1.53,"5min":1.8859999999999999,"15min":1.921},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.1719999999999999,"15min":1.163},"min":{"1min":0.82299999999999995,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.9690000000000001,"5min":1.9690000000000001,"15min":2.0209999999999999},"last":1.3200000000000001}]},{"osd":4,"last update":"Wed Oct 11 09:50:46 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.2090000000000001,"15min":1.1899999999999999},"min":{"1min":0.94199999999999995,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.0169999999999999,"5min":2.117,"15min":2.117},"last":1.26},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.1850000000000001,"15min":1.1870000000000001},"min":{"1min":0.82999999999999996,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.887,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1}]},{"osd":6,"last update":"Wed Oct 11 09:50:46 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.2609999999999999,"15min":1.2649999999999999},"min":{"1min":0.90100000000000002,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.8859999999999999,"5min":1.9239999999999999,"15min":2.4529999999999998},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.403,"5min":1.3240000000000001,"15min":1.3160000000000001},"min":{"1min":0.91400000000000003,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.7350000000000001}]},{"osd":7,"last update":"Wed Oct 11 09:50:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.3380000000000001,"15min":1.3600000000000001},"min":{"1min":1.071,"5min":0.61899999999999999,"15min":0.57699999999999996},"max":{"1min":2.004,"5min":2.347,"15min":3.6120000000000001},"last":1.306},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.423,"15min":1.4410000000000001},"min":{"1min":1.1240000000000001,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":2.016,"5min":2.2679999999999998,"15min":3.2109999999999999},"last":1.6259999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574985,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32540,"kb_used_data":1492,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339300,"statfs":{"total":96636764160,"available":96603443200,"internally_reserved":0,"allocated":1527808,"data_stored":741580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:50:50 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.056,"15min":1.0580000000000001},"min":{"1min":0.77500000000000002,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.3979999999999999,"5min":1.579,"15min":1.7170000000000001},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0720000000000001,"15min":1.0860000000000001},"min":{"1min":0.61899999999999999,"5min":0.60199999999999998,"15min":0.54700000000000004},"max":{"1min":1.4079999999999999,"5min":1.669,"15min":2.1160000000000001},"last":1.5129999999999999}]},{"osd":1,"last update":"Wed Oct 11 09:50:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.091,"15min":1.0860000000000001},"min":{"1min":0.747,"5min":0.66500000000000004,"15min":0.60699999999999998},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.95},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.141,"15min":1.1439999999999999},"min":{"1min":0.90700000000000003,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.486,"5min":1.7849999999999999,"15min":1.8220000000000001},"last":2.02}]},{"osd":2,"last update":"Wed Oct 11 09:50:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.206,"15min":1.194},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.52,"5min":1.9159999999999999,"15min":1.919},"last":2.5859999999999999},{"interface":"front","average":{"1min":1.286,"5min":1.232,"15min":1.218},"min":{"1min":0.78800000000000003,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.5409999999999999,"5min":1.8300000000000001,"15min":1.8859999999999999},"last":2.5350000000000001}]},{"osd":3,"last update":"Wed Oct 11 09:50:50 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.1259999999999999,"15min":1.129},"min":{"1min":0.82799999999999996,"5min":0.72599999999999998,"15min":0.60199999999999998},"max":{"1min":1.5760000000000001,"5min":1.6839999999999999,"15min":1.6930000000000001},"last":1.117},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.1659999999999999,"15min":1.2410000000000001},"min":{"1min":0.89300000000000002,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.4470000000000001,"5min":1.5329999999999999,"15min":2.3340000000000001},"last":2.5030000000000001}]},{"osd":5,"last update":"Wed Oct 11 09:50:50 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.175,"15min":1.1890000000000001},"min":{"1min":0.82299999999999995,"5min":0.753,"15min":0.55700000000000005},"max":{"1min":1.5760000000000001,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":1.595},{"interface":"front","average":{"1min":1.159,"5min":1.155,"15min":1.1850000000000001},"min":{"1min":0.81999999999999995,"5min":0.63300000000000001,"15min":0.61199999999999999},"max":{"1min":1.51,"5min":1.732,"15min":1.8520000000000001},"last":1.6539999999999999}]},{"osd":6,"last update":"Wed Oct 11 09:50:50 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.226,"15min":1.252},"min":{"1min":0.89800000000000002,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.798,"5min":1.798,"15min":1.8},"last":2.0699999999999998},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.3080000000000001,"15min":1.319},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.63500000000000001},"max":{"1min":1.7490000000000001,"5min":1.9399999999999999,"15min":2.1120000000000001},"last":1.3360000000000001}]},{"osd":7,"last update":"Wed Oct 11 09:50:50 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.276,"15min":1.3049999999999999},"min":{"1min":0.89000000000000001,"5min":0.78100000000000003,"15min":0.60899999999999999},"max":{"1min":1.6020000000000001,"5min":1.794,"15min":1.9470000000000001},"last":1.3799999999999999},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.363,"15min":1.417},"min":{"1min":0.98999999999999999,"5min":0.875,"15min":0.75900000000000001},"max":{"1min":1.6080000000000001,"5min":1.962,"15min":1.9990000000000001},"last":2.1669999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574984,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32540,"kb_used_data":1492,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339300,"statfs":{"total":96636764160,"available":96603443200,"internally_reserved":0,"allocated":1527808,"data_stored":741580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.76100000000000001,"15min":0.80400000000000005},"min":{"1min":0.53500000000000003,"5min":0.442,"15min":0.33000000000000002},"max":{"1min":1.399,"5min":1.399,"15min":1.462},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.874,"5min":0.81899999999999995,"15min":0.83699999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.3320000000000001,"5min":1.363,"15min":1.4950000000000001},"last":0.77400000000000002}]},{"osd":1,"last update":"Wed Oct 11 09:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.79200000000000004,"15min":0.80700000000000005},"min":{"1min":0.53400000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.2150000000000001,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.90400000000000003,"15min":0.92200000000000004},"min":{"1min":0.624,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.3049999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":1.228}]},{"osd":2,"last update":"Wed Oct 11 09:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.95099999999999996,"15min":0.96599999999999997},"min":{"1min":0.57699999999999996,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.4690000000000001,"5min":1.601,"15min":1.601},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.028,"5min":1.012,"15min":1.0580000000000001},"min":{"1min":0.80700000000000005,"5min":0.58499999999999996,"15min":0.432},"max":{"1min":1.3009999999999999,"5min":1.607,"15min":2.5870000000000002},"last":1.353}]},{"osd":4,"last update":"Wed Oct 11 09:50:44 2023","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.115,"15min":1.1639999999999999},"min":{"1min":0.84799999999999998,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.5549999999999999,"5min":1.7210000000000001,"15min":2.395},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.173,"15min":1.2430000000000001},"min":{"1min":0.86499999999999999,"5min":0.78100000000000003,"15min":0.73599999999999999},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":3.1579999999999999},"last":1.167}]},{"osd":5,"last update":"Wed Oct 11 09:50:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.23,"15min":1.28},"min":{"1min":0.95699999999999996,"5min":0.84199999999999997,"15min":0.76500000000000001},"max":{"1min":1.724,"5min":1.724,"15min":3.0009999999999999},"last":1.383},{"interface":"front","average":{"1min":1.252,"5min":1.236,"15min":1.3380000000000001},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.5800000000000001,"5min":1.5840000000000001,"15min":3.8050000000000002},"last":1.651}]},{"osd":6,"last update":"Wed Oct 11 09:50:44 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.226,"15min":1.258},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":2.1949999999999998},"last":1.514},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2709999999999999,"15min":1.339},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.84499999999999997},"max":{"1min":1.782,"5min":1.782,"15min":3.2549999999999999},"last":1.556}]},{"osd":7,"last update":"Wed Oct 11 09:50:44 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.2849999999999999,"15min":1.383},"min":{"1min":0.98399999999999999,"5min":0.95199999999999996,"15min":0.76700000000000002},"max":{"1min":1.669,"5min":1.7150000000000001,"15min":3.7490000000000001},"last":1.587},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.272,"15min":1.3320000000000001},"min":{"1min":0.83199999999999996,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.7969999999999999,"5min":1.7969999999999999,"15min":2.625},"last":1.427}]}]},{"osd":2,"up_from":13,"seq":55834574986,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32540,"kb_used_data":1492,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339300,"statfs":{"total":96636764160,"available":96603443200,"internally_reserved":0,"allocated":1527808,"data_stored":741580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:50:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.78900000000000003,"15min":0.80400000000000005},"min":{"1min":0.57499999999999996,"5min":0.40600000000000003,"15min":0.38500000000000001},"max":{"1min":1.26,"5min":1.5920000000000001,"15min":2.0600000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.83199999999999996,"15min":0.89400000000000002},"min":{"1min":0.57199999999999995,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.3300000000000001,"5min":1.5329999999999999,"15min":2.8919999999999999},"last":0.69499999999999995}]},{"osd":1,"last update":"Wed Oct 11 09:50:46 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.84599999999999997,"15min":0.94699999999999995},"min":{"1min":0.41999999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.4810000000000001,"5min":1.8069999999999999,"15min":3.7650000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.96399999999999997,"15min":0.97199999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.40300000000000002},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":2.5339999999999998},"last":0.79100000000000004}]},{"osd":3,"last update":"Wed Oct 11 09:50:46 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.91500000000000004,"15min":1.018},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.2769999999999999,"5min":1.591,"15min":2.403},"last":0.72899999999999998},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0229999999999999,"15min":1.0449999999999999},"min":{"1min":0.68200000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.754,"5min":1.754,"15min":2.7589999999999999},"last":0.88400000000000001}]},{"osd":4,"last update":"Wed Oct 11 09:50:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.141,"15min":1.1819999999999999},"min":{"1min":0.71699999999999997,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.629,"5min":1.7809999999999999,"15min":2.2549999999999999},"last":1.21},{"interface":"front","average":{"1min":1.204,"5min":1.2210000000000001,"15min":1.3300000000000001},"min":{"1min":0.751,"5min":0.751,"15min":0.73399999999999999},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":2.6859999999999999},"last":1.244}]},{"osd":5,"last update":"Wed Oct 11 09:50:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.2330000000000001,"15min":1.3},"min":{"1min":0.80500000000000005,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.72,"5min":1.76,"15min":3.044},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1890000000000001,"15min":1.2569999999999999},"min":{"1min":0.83099999999999996,"5min":0.72799999999999998,"15min":0.69599999999999995},"max":{"1min":1.4259999999999999,"5min":1.837,"15min":3.5790000000000002},"last":1.292}]},{"osd":6,"last update":"Wed Oct 11 09:50:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.2050000000000001,"15min":1.319},"min":{"1min":0.82299999999999995,"5min":0.75,"15min":0.75},"max":{"1min":1.585,"5min":1.6659999999999999,"15min":3.1600000000000001},"last":1.141},{"interface":"front","average":{"1min":1.337,"5min":1.3080000000000001,"15min":1.333},"min":{"1min":1.0269999999999999,"5min":0.83499999999999996,"15min":0.623},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":2.839},"last":1.054}]},{"osd":7,"last update":"Wed Oct 11 09:50:46 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.2729999999999999,"15min":1.3500000000000001},"min":{"1min":0.86399999999999999,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.663,"5min":1.774,"15min":3.1230000000000002},"last":0.999},{"interface":"front","average":{"1min":1.28,"5min":1.325,"15min":1.431},"min":{"1min":0.874,"5min":0.84899999999999998,"15min":0.80000000000000004},"max":{"1min":1.704,"5min":1.7909999999999999,"15min":2.988},"last":1.359}]}]},{"osd":1,"up_from":13,"seq":55834574985,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32540,"kb_used_data":1492,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339300,"statfs":{"total":96636764160,"available":96603443200,"internally_reserved":0,"allocated":1527808,"data_stored":741580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.78700000000000003,"15min":0.78300000000000003},"min":{"1min":0.51700000000000002,"5min":0.375,"15min":0.375},"max":{"1min":1.014,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.79700000000000004,"15min":0.81799999999999995},"min":{"1min":0.48099999999999998,"5min":0.40100000000000002,"15min":0.36699999999999999},"max":{"1min":1.1699999999999999,"5min":1.593,"15min":1.593},"last":0.66400000000000003}]},{"osd":2,"last update":"Wed Oct 11 09:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.81299999999999994,"15min":0.83799999999999997},"min":{"1min":0.53600000000000003,"5min":0.437,"15min":0.36199999999999999},"max":{"1min":1.004,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.91400000000000003,"15min":0.95199999999999996},"min":{"1min":0.51700000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.3160000000000001,"5min":1.3380000000000001,"15min":1.6619999999999999},"last":0.72899999999999998}]},{"osd":3,"last update":"Wed Oct 11 09:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":1.01,"15min":1.0089999999999999},"min":{"1min":0.61599999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.373,"5min":1.714,"15min":1.714},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.94599999999999995,"15min":0.96999999999999997},"min":{"1min":0.59299999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.3240000000000001,"5min":1.611,"15min":1.611},"last":0.77300000000000002}]},{"osd":4,"last update":"Wed Oct 11 09:50:44 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1180000000000001,"15min":1.149},"min":{"1min":0.81699999999999995,"5min":0.77400000000000002,"15min":0.73599999999999999},"max":{"1min":1.46,"5min":1.5980000000000001,"15min":2.016},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.214,"15min":1.266},"min":{"1min":0.91500000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.409,"5min":1.665,"15min":2.323},"last":1.2130000000000001}]},{"osd":5,"last update":"Wed Oct 11 09:50:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.198,"15min":1.2669999999999999},"min":{"1min":0.84099999999999997,"5min":0.79300000000000004,"15min":0.65800000000000003},"max":{"1min":1.335,"5min":1.722,"15min":2.4079999999999999},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.1850000000000001,"15min":1.2290000000000001},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.4419999999999999,"5min":1.677,"15min":2.1349999999999998},"last":1.0840000000000001}]},{"osd":6,"last update":"Wed Oct 11 09:50:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.244,"15min":1.3169999999999999},"min":{"1min":0.90800000000000003,"5min":0.84599999999999997,"15min":0.71499999999999997},"max":{"1min":1.746,"5min":2.4620000000000002,"15min":2.8940000000000001},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.21,"5min":1.2450000000000001,"15min":1.3149999999999999},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.79200000000000004},"max":{"1min":1.615,"5min":1.8129999999999999,"15min":3.3719999999999999},"last":1.1759999999999999}]},{"osd":7,"last update":"Wed Oct 11 09:50:44 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.3069999999999999,"15min":1.367},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86099999999999999},"max":{"1min":1.5089999999999999,"5min":1.835,"15min":2.194},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.286,"5min":1.3080000000000001,"15min":1.3839999999999999},"min":{"1min":0.96199999999999997,"5min":0.93300000000000005,"15min":0.89600000000000002},"max":{"1min":1.8120000000000001,"5min":1.857,"15min":1.968},"last":1.258}]}]},{"osd":0,"up_from":13,"seq":55834574984,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33120,"kb_used_data":2072,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338720,"statfs":{"total":96636764160,"available":96602849280,"internally_reserved":0,"allocated":2121728,"data_stored":1331948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 11 09:50:39 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.80600000000000005,"15min":0.80000000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.38500000000000001},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.874,"15min":0.90000000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.04,"5min":1.5249999999999999,"15min":2.3100000000000001},"last":0.92300000000000004}]},{"osd":2,"last update":"Wed Oct 11 09:50:39 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.89800000000000002,"15min":0.93100000000000005},"min":{"1min":0.56000000000000005,"5min":0.53100000000000003,"15min":0.50600000000000001},"max":{"1min":1.522,"5min":1.522,"15min":1.7509999999999999},"last":1.8919999999999999},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.93799999999999994,"15min":0.95499999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.569,"5min":1.5820000000000001,"15min":1.9450000000000001},"last":1.5529999999999999}]},{"osd":3,"last update":"Wed Oct 11 09:50:39 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.78900000000000003,"15min":0.84299999999999997},"min":{"1min":0.50800000000000001,"5min":0.49299999999999999,"15min":0.45600000000000002},"max":{"1min":1.1359999999999999,"5min":1.22,"15min":2.7639999999999998},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.996,"5min":0.93200000000000005,"15min":0.995},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.48999999999999999},"max":{"1min":1.3200000000000001,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.482}]},{"osd":4,"last update":"Wed Oct 11 09:50:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.1399999999999999,"15min":1.173},"min":{"1min":0.82199999999999995,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.3700000000000001,"5min":1.792,"15min":1.792},"last":1.448},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.111,"15min":1.173},"min":{"1min":0.755,"5min":0.755,"15min":0.69799999999999995},"max":{"1min":1.466,"5min":1.466,"15min":2.6629999999999998},"last":1.9399999999999999}]},{"osd":5,"last update":"Wed Oct 11 09:50:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.206,"15min":1.302},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.829,"5min":1.829,"15min":3.0150000000000001},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.173,"5min":1.1850000000000001,"15min":1.2749999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.452,"5min":1.7070000000000001,"15min":3.601},"last":1.1240000000000001}]},{"osd":6,"last update":"Wed Oct 11 09:50:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.1419999999999999,"15min":1.2010000000000001},"min":{"1min":0.85299999999999998,"5min":0.79100000000000004,"15min":0.78000000000000003},"max":{"1min":1.4099999999999999,"5min":1.502,"15min":1.825},"last":1.8360000000000001},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.254,"15min":1.3069999999999999},"min":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":1.732,"5min":1.732,"15min":2.1440000000000001},"last":1.7430000000000001}]},{"osd":7,"last update":"Wed Oct 11 09:50:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.25,"15min":1.298},"min":{"1min":0.96199999999999997,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.4219999999999999,"5min":1.75,"15min":1.8759999999999999},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.2370000000000001,"15min":1.302},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.73899999999999999},"max":{"1min":1.379,"5min":1.5900000000000001,"15min":2.5419999999999998},"last":2.016}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-11T09:51:44.539 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-11T09:51:44.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.744+0000 7f7061bfc700 1 Processor -- start 2023-10-11T09:51:44.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.744+0000 7f7061bfc700 1 -- start start 2023-10-11T09:51:44.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.744+0000 7f7061bfc700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f705c15b8e0 0x7f705c06d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:44.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.744+0000 7f7061bfc700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f705c06d610 0x7f705c0ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:44.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.744+0000 7f7061bfc700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f705c15e010 con 0x7f705c15a940 2023-10-11T09:51:44.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.744+0000 7f7061bfc700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f705c15e280 con 0x7f705c15b8e0 2023-10-11T09:51:44.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.744+0000 7f7061bfc700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f705c15e560 con 0x7f705c06d610 2023-10-11T09:51:44.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.744+0000 7f7060bfa700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f705c15b8e0 0x7f705c06d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:44.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.744+0000 7f70613fb700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f705c15a940 0x7f705c0c9020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:50230/0 (socket says 172.21.15.78:50230) 2023-10-11T09:51:44.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.744+0000 7f70613fb700 1 -- 172.21.15.78:0/3582607961 learned_addr learned my addr 172.21.15.78:0/3582607961 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:51:44.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.748+0000 7f705bfff700 1 --2- 172.21.15.78:0/3582607961 >> v2:172.21.15.133:3300/0 conn(0x7f705c06d610 0x7f705c0ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:44.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.748+0000 7f705b7fe700 1 -- 172.21.15.78:0/3582607961 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3208466818 0 0) 0x7f705c15e010 con 0x7f705c15a940 2023-10-11T09:51:44.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.748+0000 7f7060bfa700 1 -- 172.21.15.78:0/3582607961 >> v2:172.21.15.133:3300/0 conn(0x7f705c06d610 msgr2=0x7f705c0ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:44.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.748+0000 7f7060bfa700 1 --2- 172.21.15.78:0/3582607961 >> v2:172.21.15.133:3300/0 conn(0x7f705c06d610 0x7f705c0ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:44.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.748+0000 7f7060bfa700 1 -- 172.21.15.78:0/3582607961 >> v1:172.21.15.78:6789/0 conn(0x7f705c15a940 legacy=0x7f705c0c9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:44.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.748+0000 7f7060bfa700 1 -- 172.21.15.78:0/3582607961 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f705c15e840 con 0x7f705c15b8e0 2023-10-11T09:51:44.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.748+0000 7f705bfff700 1 --2- 172.21.15.78:0/3582607961 >> v2:172.21.15.133:3300/0 conn(0x7f705c06d610 0x7f705c0ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:51:44.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.748+0000 7f7060bfa700 1 --2- 172.21.15.78:0/3582607961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f705c15b8e0 0x7f705c06d0d0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f705c06f800 tx=0x7f7050009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60975649dce83911 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:44.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.748+0000 7f705b7fe700 1 -- 172.21.15.78:0/3582607961 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7050011830 con 0x7f705c15b8e0 2023-10-11T09:51:44.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.748+0000 7f705b7fe700 1 -- 172.21.15.78:0/3582607961 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7050011990 con 0x7f705c15b8e0 2023-10-11T09:51:44.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.748+0000 7f705b7fe700 1 -- 172.21.15.78:0/3582607961 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7050011d40 con 0x7f705c15b8e0 2023-10-11T09:51:44.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.748+0000 7f7061bfc700 1 -- 172.21.15.78:0/3582607961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f705c15b8e0 msgr2=0x7f705c06d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:44.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.748+0000 7f7061bfc700 1 --2- 172.21.15.78:0/3582607961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f705c15b8e0 0x7f705c06d0d0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f705c06f800 tx=0x7f7050009b30 comp rx=0 tx=0).stop 2023-10-11T09:51:44.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.748+0000 7f7061bfc700 1 -- 172.21.15.78:0/3582607961 shutdown_connections 2023-10-11T09:51:44.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.748+0000 7f7061bfc700 1 --2- 172.21.15.78:0/3582607961 >> v2:172.21.15.133:3300/0 conn(0x7f705c06d610 0x7f705c0ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:44.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.748+0000 7f7061bfc700 1 --2- 172.21.15.78:0/3582607961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f705c15b8e0 0x7f705c06d0d0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:44.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.748+0000 7f7061bfc700 1 -- 172.21.15.78:0/3582607961 >> 172.21.15.78:0/3582607961 conn(0x7f705c0cea60 msgr2=0x7f705c0cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:44.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.748+0000 7f7061bfc700 1 -- 172.21.15.78:0/3582607961 shutdown_connections 2023-10-11T09:51:44.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.748+0000 7f7061bfc700 1 -- 172.21.15.78:0/3582607961 wait complete. 2023-10-11T09:51:44.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.752+0000 7f7061bfc700 1 Processor -- start 2023-10-11T09:51:44.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.752+0000 7f7061bfc700 1 -- start start 2023-10-11T09:51:44.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.756+0000 7f7061bfc700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f705c06d610 0x7f705c159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:44.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.756+0000 7f7061bfc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f705c15a940 0x7f705c13ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:44.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.756+0000 7f7061bfc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f705c15b8e0 0x7f705c0b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:44.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.756+0000 7f7061bfc700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705c0c8a60 con 0x7f705c15a940 2023-10-11T09:51:44.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.756+0000 7f7061bfc700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705c0c8d40 con 0x7f705c15b8e0 2023-10-11T09:51:44.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.756+0000 7f7061bfc700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f705c0c9020 con 0x7f705c06d610 2023-10-11T09:51:44.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.756+0000 7f705bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f705c15a940 0x7f705c13ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:44.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.756+0000 7f705bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f705c15a940 0x7f705c13ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43996/0 (socket says 172.21.15.78:43996) 2023-10-11T09:51:44.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.756+0000 7f705bfff700 1 -- 172.21.15.78:0/2868643455 learned_addr learned my addr 172.21.15.78:0/2868643455 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:44.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.756+0000 7f70613fb700 1 --2- 172.21.15.78:0/2868643455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f705c15b8e0 0x7f705c0b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:44.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.756+0000 7f7060bfa700 1 --2- 172.21.15.78:0/2868643455 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f705c06d610 0x7f705c159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:44.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.756+0000 7f705bfff700 1 -- 172.21.15.78:0/2868643455 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f705c06d610 msgr2=0x7f705c159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:44.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.756+0000 7f705bfff700 1 --2- 172.21.15.78:0/2868643455 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f705c06d610 0x7f705c159060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:44.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.756+0000 7f705bfff700 1 -- 172.21.15.78:0/2868643455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f705c15b8e0 msgr2=0x7f705c0b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:44.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.756+0000 7f705bfff700 1 --2- 172.21.15.78:0/2868643455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f705c15b8e0 0x7f705c0b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:44.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.756+0000 7f705bfff700 1 -- 172.21.15.78:0/2868643455 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70500094b0 con 0x7f705c15a940 2023-10-11T09:51:44.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.756+0000 7f705bfff700 1 --2- 172.21.15.78:0/2868643455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f705c15a940 0x7f705c13ce20 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f704800e880 tx=0x7f704800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:44.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.756+0000 7f70597fa700 1 -- 172.21.15.78:0/2868643455 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704800c780 con 0x7f705c15a940 2023-10-11T09:51:44.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.756+0000 7f70597fa700 1 -- 172.21.15.78:0/2868643455 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704800c8e0 con 0x7f705c15a940 2023-10-11T09:51:44.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.756+0000 7f70597fa700 1 -- 172.21.15.78:0/2868643455 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704800cc70 con 0x7f705c15a940 2023-10-11T09:51:44.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.756+0000 7f7061bfc700 1 -- 172.21.15.78:0/2868643455 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f705c0c9300 con 0x7f705c15a940 2023-10-11T09:51:44.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.756+0000 7f7061bfc700 1 -- 172.21.15.78:0/2868643455 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f705c1595a0 con 0x7f705c15a940 2023-10-11T09:51:44.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.760+0000 7f7061bfc700 1 -- 172.21.15.78:0/2868643455 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f705c065d40 con 0x7f705c15a940 2023-10-11T09:51:44.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.760+0000 7f70597fa700 1 -- 172.21.15.78:0/2868643455 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f7048013410 con 0x7f705c15a940 2023-10-11T09:51:44.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.760+0000 7f70597fa700 1 --2- 172.21.15.78:0/2868643455 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7040065ed0 0x7f7040068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:44.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.760+0000 7f70597fa700 1 -- 172.21.15.78:0/2868643455 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f70480833f0 con 0x7f705c15a940 2023-10-11T09:51:44.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.760+0000 7f7060bfa700 1 --2- 172.21.15.78:0/2868643455 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7040065ed0 0x7f7040068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:44.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.760+0000 7f7060bfa700 1 --2- 172.21.15.78:0/2868643455 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7040065ed0 0x7f7040068380 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f70500090b0 tx=0x7f7050008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:44.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.768+0000 7f70597fa700 1 -- 172.21.15.78:0/2868643455 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f7048050ad0 con 0x7f705c15a940 2023-10-11T09:51:44.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.936+0000 7f7061bfc700 1 -- 172.21.15.78:0/2868643455 --> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f705c15eb20 con 0x7f7040065ed0 2023-10-11T09:51:44.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.948+0000 7f70597fa700 1 -- 172.21.15.78:0/2868643455 <== mgr.4101 v2:172.21.15.78:6832/35606 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309310 (secure 0 0 0) 0x7f7050011d30 con 0x7f7040065ed0 2023-10-11T09:51:44.956 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:44.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.956+0000 7f7061bfc700 1 -- 172.21.15.78:0/2868643455 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7040065ed0 msgr2=0x7f7040068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:44.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.956+0000 7f7061bfc700 1 --2- 172.21.15.78:0/2868643455 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7040065ed0 0x7f7040068380 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f70500090b0 tx=0x7f7050008030 comp rx=0 tx=0).stop 2023-10-11T09:51:44.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.956+0000 7f7061bfc700 1 -- 172.21.15.78:0/2868643455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f705c15a940 msgr2=0x7f705c13ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:44.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.956+0000 7f7061bfc700 1 --2- 172.21.15.78:0/2868643455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f705c15a940 0x7f705c13ce20 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f704800e880 tx=0x7f704800ec40 comp rx=0 tx=0).stop 2023-10-11T09:51:44.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.956+0000 7f7061bfc700 1 -- 172.21.15.78:0/2868643455 shutdown_connections 2023-10-11T09:51:44.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.956+0000 7f7061bfc700 1 --2- 172.21.15.78:0/2868643455 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7040065ed0 0x7f7040068380 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:44.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.956+0000 7f7061bfc700 1 --2- 172.21.15.78:0/2868643455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f705c15b8e0 0x7f705c0b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:44.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.956+0000 7f7061bfc700 1 --2- 172.21.15.78:0/2868643455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f705c15a940 0x7f705c13ce20 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:44.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.956+0000 7f7061bfc700 1 --2- 172.21.15.78:0/2868643455 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f705c06d610 0x7f705c159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:44.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.956+0000 7f7061bfc700 1 -- 172.21.15.78:0/2868643455 >> 172.21.15.78:0/2868643455 conn(0x7f705c0cea60 msgr2=0x7f705c13f160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:44.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.956+0000 7f7061bfc700 1 -- 172.21.15.78:0/2868643455 shutdown_connections 2023-10-11T09:51:44.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:44.956+0000 7f7061bfc700 1 -- 172.21.15.78:0/2868643455 wait complete. 2023-10-11T09:51:44.965 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2023-10-11T09:51:44.985 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":458,"stamp":"2023-10-11T09:51:44.562491+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261928,"kb_used_data":13544,"kb_used_omap":0,"kb_used_meta":248320,"kb_avail":754712792,"statfs":{"total":773094113280,"available":772825899008,"internally_reserved":0,"allocated":13869056,"data_stored":7330732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":254279680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.831030"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.490548+0000","last_change":"2023-10-11T09:51:40.247514+0000","last_active":"2023-10-11T09:51:41.490548+0000","last_peered":"2023-10-11T09:51:41.490548+0000","last_clean":"2023-10-11T09:51:41.490548+0000","last_became_active":"2023-10-11T09:51:40.247269+0000","last_became_peered":"2023-10-11T09:51:40.247269+0000","last_unstale":"2023-10-11T09:51:41.490548+0000","last_undegraded":"2023-10-11T09:51:41.490548+0000","last_fullsized":"2023-10-11T09:51:41.490548+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:28:07.140682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.890400+0000","last_change":"2023-10-11T09:51:40.256193+0000","last_active":"2023-10-11T09:51:41.890400+0000","last_peered":"2023-10-11T09:51:41.890400+0000","last_clean":"2023-10-11T09:51:41.890400+0000","last_became_active":"2023-10-11T09:51:40.255854+0000","last_became_peered":"2023-10-11T09:51:40.255854+0000","last_unstale":"2023-10-11T09:51:41.890400+0000","last_undegraded":"2023-10-11T09:51:41.890400+0000","last_fullsized":"2023-10-11T09:51:41.890400+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:15:04.560718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.890803+0000","last_change":"2023-10-11T09:51:40.248064+0000","last_active":"2023-10-11T09:51:41.890803+0000","last_peered":"2023-10-11T09:51:41.890803+0000","last_clean":"2023-10-11T09:51:41.890803+0000","last_became_active":"2023-10-11T09:51:40.247759+0000","last_became_peered":"2023-10-11T09:51:40.247759+0000","last_unstale":"2023-10-11T09:51:41.890803+0000","last_undegraded":"2023-10-11T09:51:41.890803+0000","last_fullsized":"2023-10-11T09:51:41.890803+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:32:08.922116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.237961+0000","last_change":"2023-10-11T09:51:40.253373+0000","last_active":"2023-10-11T09:51:41.237961+0000","last_peered":"2023-10-11T09:51:41.237961+0000","last_clean":"2023-10-11T09:51:41.237961+0000","last_became_active":"2023-10-11T09:51:40.253023+0000","last_became_peered":"2023-10-11T09:51:40.253023+0000","last_unstale":"2023-10-11T09:51:41.237961+0000","last_undegraded":"2023-10-11T09:51:41.237961+0000","last_fullsized":"2023-10-11T09:51:41.237961+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:15:04.560718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.489883+0000","last_change":"2023-10-11T09:51:40.249116+0000","last_active":"2023-10-11T09:51:41.489883+0000","last_peered":"2023-10-11T09:51:41.489883+0000","last_clean":"2023-10-11T09:51:41.489883+0000","last_became_active":"2023-10-11T09:51:40.248839+0000","last_became_peered":"2023-10-11T09:51:40.248839+0000","last_unstale":"2023-10-11T09:51:41.489883+0000","last_undegraded":"2023-10-11T09:51:41.489883+0000","last_fullsized":"2023-10-11T09:51:41.489883+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:41:00.113202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.306441+0000","last_change":"2023-10-11T09:51:40.236785+0000","last_active":"2023-10-11T09:51:41.306441+0000","last_peered":"2023-10-11T09:51:41.306441+0000","last_clean":"2023-10-11T09:51:41.306441+0000","last_became_active":"2023-10-11T09:51:40.236485+0000","last_became_peered":"2023-10-11T09:51:40.236485+0000","last_unstale":"2023-10-11T09:51:41.306441+0000","last_undegraded":"2023-10-11T09:51:41.306441+0000","last_fullsized":"2023-10-11T09:51:41.306441+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:45:34.496784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.306443+0000","last_change":"2023-10-11T09:51:40.253012+0000","last_active":"2023-10-11T09:51:41.306443+0000","last_peered":"2023-10-11T09:51:41.306443+0000","last_clean":"2023-10-11T09:51:41.306443+0000","last_became_active":"2023-10-11T09:51:40.252767+0000","last_became_peered":"2023-10-11T09:51:40.252767+0000","last_unstale":"2023-10-11T09:51:41.306443+0000","last_undegraded":"2023-10-11T09:51:41.306443+0000","last_fullsized":"2023-10-11T09:51:41.306443+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:55:15.498498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.889678+0000","last_change":"2023-10-11T09:51:40.249096+0000","last_active":"2023-10-11T09:51:41.889678+0000","last_peered":"2023-10-11T09:51:41.889678+0000","last_clean":"2023-10-11T09:51:41.889678+0000","last_became_active":"2023-10-11T09:51:40.248762+0000","last_became_peered":"2023-10-11T09:51:40.248762+0000","last_unstale":"2023-10-11T09:51:41.889678+0000","last_undegraded":"2023-10-11T09:51:41.889678+0000","last_fullsized":"2023-10-11T09:51:41.889678+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:09:33.360022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.238007+0000","last_change":"2023-10-11T09:51:40.258057+0000","last_active":"2023-10-11T09:51:41.238007+0000","last_peered":"2023-10-11T09:51:41.238007+0000","last_clean":"2023-10-11T09:51:41.238007+0000","last_became_active":"2023-10-11T09:51:40.257744+0000","last_became_peered":"2023-10-11T09:51:40.257744+0000","last_unstale":"2023-10-11T09:51:41.238007+0000","last_undegraded":"2023-10-11T09:51:41.238007+0000","last_fullsized":"2023-10-11T09:51:41.238007+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:27:20.415765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.232989+0000","last_change":"2023-10-11T09:51:40.237992+0000","last_active":"2023-10-11T09:51:41.232989+0000","last_peered":"2023-10-11T09:51:41.232989+0000","last_clean":"2023-10-11T09:51:41.232989+0000","last_became_active":"2023-10-11T09:51:40.237754+0000","last_became_peered":"2023-10-11T09:51:40.237754+0000","last_unstale":"2023-10-11T09:51:41.232989+0000","last_undegraded":"2023-10-11T09:51:41.232989+0000","last_fullsized":"2023-10-11T09:51:41.232989+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T21:34:21.525626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.490084+0000","last_change":"2023-10-11T09:51:40.246796+0000","last_active":"2023-10-11T09:51:41.490084+0000","last_peered":"2023-10-11T09:51:41.490084+0000","last_clean":"2023-10-11T09:51:41.490084+0000","last_became_active":"2023-10-11T09:51:40.246525+0000","last_became_peered":"2023-10-11T09:51:40.246525+0000","last_unstale":"2023-10-11T09:51:41.490084+0000","last_undegraded":"2023-10-11T09:51:41.490084+0000","last_fullsized":"2023-10-11T09:51:41.490084+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:26:52.594103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.307114+0000","last_change":"2023-10-11T09:51:40.244474+0000","last_active":"2023-10-11T09:51:41.307114+0000","last_peered":"2023-10-11T09:51:41.307114+0000","last_clean":"2023-10-11T09:51:41.307114+0000","last_became_active":"2023-10-11T09:51:40.244185+0000","last_became_peered":"2023-10-11T09:51:40.244185+0000","last_unstale":"2023-10-11T09:51:41.307114+0000","last_undegraded":"2023-10-11T09:51:41.307114+0000","last_fullsized":"2023-10-11T09:51:41.307114+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:08:45.290040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.232887+0000","last_change":"2023-10-11T09:51:40.250195+0000","last_active":"2023-10-11T09:51:41.232887+0000","last_peered":"2023-10-11T09:51:41.232887+0000","last_clean":"2023-10-11T09:51:41.232887+0000","last_became_active":"2023-10-11T09:51:40.249860+0000","last_became_peered":"2023-10-11T09:51:40.249860+0000","last_unstale":"2023-10-11T09:51:41.232887+0000","last_undegraded":"2023-10-11T09:51:41.232887+0000","last_fullsized":"2023-10-11T09:51:41.232887+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:55:29.533086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.890466+0000","last_change":"2023-10-11T09:51:40.248951+0000","last_active":"2023-10-11T09:51:41.890466+0000","last_peered":"2023-10-11T09:51:41.890466+0000","last_clean":"2023-10-11T09:51:41.890466+0000","last_became_active":"2023-10-11T09:51:40.248651+0000","last_became_peered":"2023-10-11T09:51:40.248651+0000","last_unstale":"2023-10-11T09:51:41.890466+0000","last_undegraded":"2023-10-11T09:51:41.890466+0000","last_fullsized":"2023-10-11T09:51:41.890466+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:01:19.716835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.489683+0000","last_change":"2023-10-11T09:51:40.249604+0000","last_active":"2023-10-11T09:51:41.489683+0000","last_peered":"2023-10-11T09:51:41.489683+0000","last_clean":"2023-10-11T09:51:41.489683+0000","last_became_active":"2023-10-11T09:51:40.249291+0000","last_became_peered":"2023-10-11T09:51:40.249291+0000","last_unstale":"2023-10-11T09:51:41.489683+0000","last_undegraded":"2023-10-11T09:51:41.489683+0000","last_fullsized":"2023-10-11T09:51:41.489683+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:15:04.560718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.306472+0000","last_change":"2023-10-11T09:51:40.235557+0000","last_active":"2023-10-11T09:51:41.306472+0000","last_peered":"2023-10-11T09:51:41.306472+0000","last_clean":"2023-10-11T09:51:41.306472+0000","last_became_active":"2023-10-11T09:51:40.235179+0000","last_became_peered":"2023-10-11T09:51:40.235179+0000","last_unstale":"2023-10-11T09:51:41.306472+0000","last_undegraded":"2023-10-11T09:51:41.306472+0000","last_fullsized":"2023-10-11T09:51:41.306472+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:04:19.588737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.233265+0000","last_change":"2023-10-11T09:51:40.241475+0000","last_active":"2023-10-11T09:51:41.233265+0000","last_peered":"2023-10-11T09:51:41.233265+0000","last_clean":"2023-10-11T09:51:41.233265+0000","last_became_active":"2023-10-11T09:51:40.241229+0000","last_became_peered":"2023-10-11T09:51:40.241229+0000","last_unstale":"2023-10-11T09:51:41.233265+0000","last_undegraded":"2023-10-11T09:51:41.233265+0000","last_fullsized":"2023-10-11T09:51:41.233265+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:29:04.167095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.890645+0000","last_change":"2023-10-11T09:51:40.256126+0000","last_active":"2023-10-11T09:51:41.890645+0000","last_peered":"2023-10-11T09:51:41.890645+0000","last_clean":"2023-10-11T09:51:41.890645+0000","last_became_active":"2023-10-11T09:51:40.255843+0000","last_became_peered":"2023-10-11T09:51:40.255843+0000","last_unstale":"2023-10-11T09:51:41.890645+0000","last_undegraded":"2023-10-11T09:51:41.890645+0000","last_fullsized":"2023-10-11T09:51:41.890645+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:22:04.778725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.237427+0000","last_change":"2023-10-11T09:51:40.258358+0000","last_active":"2023-10-11T09:51:41.237427+0000","last_peered":"2023-10-11T09:51:41.237427+0000","last_clean":"2023-10-11T09:51:41.237427+0000","last_became_active":"2023-10-11T09:51:40.257706+0000","last_became_peered":"2023-10-11T09:51:40.257706+0000","last_unstale":"2023-10-11T09:51:41.237427+0000","last_undegraded":"2023-10-11T09:51:41.237427+0000","last_fullsized":"2023-10-11T09:51:41.237427+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:03:55.584256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.486155+0000","last_change":"2023-10-11T09:51:40.238529+0000","last_active":"2023-10-11T09:51:41.486155+0000","last_peered":"2023-10-11T09:51:41.486155+0000","last_clean":"2023-10-11T09:51:41.486155+0000","last_became_active":"2023-10-11T09:51:40.238224+0000","last_became_peered":"2023-10-11T09:51:40.238224+0000","last_unstale":"2023-10-11T09:51:41.486155+0000","last_undegraded":"2023-10-11T09:51:41.486155+0000","last_fullsized":"2023-10-11T09:51:41.486155+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:56:27.544566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.237568+0000","last_change":"2023-10-11T09:51:40.256698+0000","last_active":"2023-10-11T09:51:41.237568+0000","last_peered":"2023-10-11T09:51:41.237568+0000","last_clean":"2023-10-11T09:51:41.237568+0000","last_became_active":"2023-10-11T09:51:40.252203+0000","last_became_peered":"2023-10-11T09:51:40.252203+0000","last_unstale":"2023-10-11T09:51:41.237568+0000","last_undegraded":"2023-10-11T09:51:41.237568+0000","last_fullsized":"2023-10-11T09:51:41.237568+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:43:05.751413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.490080+0000","last_change":"2023-10-11T09:51:40.257996+0000","last_active":"2023-10-11T09:51:41.490080+0000","last_peered":"2023-10-11T09:51:41.490080+0000","last_clean":"2023-10-11T09:51:41.490080+0000","last_became_active":"2023-10-11T09:51:40.257716+0000","last_became_peered":"2023-10-11T09:51:40.257716+0000","last_unstale":"2023-10-11T09:51:41.490080+0000","last_undegraded":"2023-10-11T09:51:41.490080+0000","last_fullsized":"2023-10-11T09:51:41.490080+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:27:20.415765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.237326+0000","last_change":"2023-10-11T09:51:40.258048+0000","last_active":"2023-10-11T09:51:41.237326+0000","last_peered":"2023-10-11T09:51:41.237326+0000","last_clean":"2023-10-11T09:51:41.237326+0000","last_became_active":"2023-10-11T09:51:40.257774+0000","last_became_peered":"2023-10-11T09:51:40.257774+0000","last_unstale":"2023-10-11T09:51:41.237326+0000","last_undegraded":"2023-10-11T09:51:41.237326+0000","last_fullsized":"2023-10-11T09:51:41.237326+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:41:00.113202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.719646+0000","last_change":"2023-10-11T09:51:40.249290+0000","last_active":"2023-10-11T09:51:41.719646+0000","last_peered":"2023-10-11T09:51:41.719646+0000","last_clean":"2023-10-11T09:51:41.719646+0000","last_became_active":"2023-10-11T09:51:40.248857+0000","last_became_peered":"2023-10-11T09:51:40.248857+0000","last_unstale":"2023-10-11T09:51:41.719646+0000","last_undegraded":"2023-10-11T09:51:41.719646+0000","last_fullsized":"2023-10-11T09:51:41.719646+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:26:52.594103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.720674+0000","last_change":"2023-10-11T09:51:40.246982+0000","last_active":"2023-10-11T09:51:41.720674+0000","last_peered":"2023-10-11T09:51:41.720674+0000","last_clean":"2023-10-11T09:51:41.720674+0000","last_became_active":"2023-10-11T09:51:40.245825+0000","last_became_peered":"2023-10-11T09:51:40.245825+0000","last_unstale":"2023-10-11T09:51:41.720674+0000","last_undegraded":"2023-10-11T09:51:41.720674+0000","last_fullsized":"2023-10-11T09:51:41.720674+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:15:04.560718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.486251+0000","last_change":"2023-10-11T09:51:40.239012+0000","last_active":"2023-10-11T09:51:41.486251+0000","last_peered":"2023-10-11T09:51:41.486251+0000","last_clean":"2023-10-11T09:51:41.486251+0000","last_became_active":"2023-10-11T09:51:40.238760+0000","last_became_peered":"2023-10-11T09:51:40.238760+0000","last_unstale":"2023-10-11T09:51:41.486251+0000","last_undegraded":"2023-10-11T09:51:41.486251+0000","last_fullsized":"2023-10-11T09:51:41.486251+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T21:09:28.097416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.237456+0000","last_change":"2023-10-11T09:51:40.258591+0000","last_active":"2023-10-11T09:51:41.237456+0000","last_peered":"2023-10-11T09:51:41.237456+0000","last_clean":"2023-10-11T09:51:41.237456+0000","last_became_active":"2023-10-11T09:51:40.258062+0000","last_became_peered":"2023-10-11T09:51:40.258062+0000","last_unstale":"2023-10-11T09:51:41.237456+0000","last_undegraded":"2023-10-11T09:51:41.237456+0000","last_fullsized":"2023-10-11T09:51:41.237456+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:56:27.544566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.307259+0000","last_change":"2023-10-11T09:51:40.248823+0000","last_active":"2023-10-11T09:51:41.307259+0000","last_peered":"2023-10-11T09:51:41.307259+0000","last_clean":"2023-10-11T09:51:41.307259+0000","last_became_active":"2023-10-11T09:51:40.248556+0000","last_became_peered":"2023-10-11T09:51:40.248556+0000","last_unstale":"2023-10-11T09:51:41.307259+0000","last_undegraded":"2023-10-11T09:51:41.307259+0000","last_fullsized":"2023-10-11T09:51:41.307259+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:22:05.628952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.485937+0000","last_change":"2023-10-11T09:51:40.237569+0000","last_active":"2023-10-11T09:51:41.485937+0000","last_peered":"2023-10-11T09:51:41.485937+0000","last_clean":"2023-10-11T09:51:41.485937+0000","last_became_active":"2023-10-11T09:51:40.237294+0000","last_became_peered":"2023-10-11T09:51:40.237294+0000","last_unstale":"2023-10-11T09:51:41.485937+0000","last_undegraded":"2023-10-11T09:51:41.485937+0000","last_fullsized":"2023-10-11T09:51:41.485937+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:53:28.732197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.890487+0000","last_change":"2023-10-11T09:51:40.247632+0000","last_active":"2023-10-11T09:51:41.890487+0000","last_peered":"2023-10-11T09:51:41.890487+0000","last_clean":"2023-10-11T09:51:41.890487+0000","last_became_active":"2023-10-11T09:51:40.247165+0000","last_became_peered":"2023-10-11T09:51:40.247165+0000","last_unstale":"2023-10-11T09:51:41.890487+0000","last_undegraded":"2023-10-11T09:51:41.890487+0000","last_fullsized":"2023-10-11T09:51:41.890487+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:28:07.140682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.719926+0000","last_change":"2023-10-11T09:51:40.242039+0000","last_active":"2023-10-11T09:51:41.719926+0000","last_peered":"2023-10-11T09:51:41.719926+0000","last_clean":"2023-10-11T09:51:41.719926+0000","last_became_active":"2023-10-11T09:51:40.241630+0000","last_became_peered":"2023-10-11T09:51:40.241630+0000","last_unstale":"2023-10-11T09:51:41.719926+0000","last_undegraded":"2023-10-11T09:51:41.719926+0000","last_fullsized":"2023-10-11T09:51:41.719926+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:57:16.192512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.719670+0000","last_change":"2023-10-11T09:51:40.249112+0000","last_active":"2023-10-11T09:51:41.719670+0000","last_peered":"2023-10-11T09:51:41.719670+0000","last_clean":"2023-10-11T09:51:41.719670+0000","last_became_active":"2023-10-11T09:51:40.248415+0000","last_became_peered":"2023-10-11T09:51:40.248415+0000","last_unstale":"2023-10-11T09:51:41.719670+0000","last_undegraded":"2023-10-11T09:51:41.719670+0000","last_fullsized":"2023-10-11T09:51:41.719670+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:26:18.137695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.233000+0000","last_change":"2023-10-11T09:51:40.237390+0000","last_active":"2023-10-11T09:51:41.233000+0000","last_peered":"2023-10-11T09:51:41.233000+0000","last_clean":"2023-10-11T09:51:41.233000+0000","last_became_active":"2023-10-11T09:51:40.237151+0000","last_became_peered":"2023-10-11T09:51:40.237151+0000","last_unstale":"2023-10-11T09:51:41.233000+0000","last_undegraded":"2023-10-11T09:51:41.233000+0000","last_fullsized":"2023-10-11T09:51:41.233000+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:16:14.644388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"126'4","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.237400+0000","last_change":"2023-10-11T09:51:39.766505+0000","last_active":"2023-10-11T09:51:41.237400+0000","last_peered":"2023-10-11T09:51:41.237400+0000","last_clean":"2023-10-11T09:51:41.237400+0000","last_became_active":"2023-10-11T09:51:39.766228+0000","last_became_peered":"2023-10-11T09:51:39.766228+0000","last_unstale":"2023-10-11T09:51:41.237400+0000","last_undegraded":"2023-10-11T09:51:41.237400+0000","last_fullsized":"2023-10-11T09:51:41.237400+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:17:59.103457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.306722+0000","last_change":"2023-10-11T09:51:40.235919+0000","last_active":"2023-10-11T09:51:41.306722+0000","last_peered":"2023-10-11T09:51:41.306722+0000","last_clean":"2023-10-11T09:51:41.306722+0000","last_became_active":"2023-10-11T09:51:40.235616+0000","last_became_peered":"2023-10-11T09:51:40.235616+0000","last_unstale":"2023-10-11T09:51:41.306722+0000","last_undegraded":"2023-10-11T09:51:41.306722+0000","last_fullsized":"2023-10-11T09:51:41.306722+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:54:27.140116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"126'1","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.233193+0000","last_change":"2023-10-11T09:51:39.759688+0000","last_active":"2023-10-11T09:51:41.233193+0000","last_peered":"2023-10-11T09:51:41.233193+0000","last_clean":"2023-10-11T09:51:41.233193+0000","last_became_active":"2023-10-11T09:51:39.759315+0000","last_became_peered":"2023-10-11T09:51:39.759315+0000","last_unstale":"2023-10-11T09:51:41.233193+0000","last_undegraded":"2023-10-11T09:51:41.233193+0000","last_fullsized":"2023-10-11T09:51:41.233193+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:53:16.999254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.720386+0000","last_change":"2023-10-11T09:51:40.241943+0000","last_active":"2023-10-11T09:51:41.720386+0000","last_peered":"2023-10-11T09:51:41.720386+0000","last_clean":"2023-10-11T09:51:41.720386+0000","last_became_active":"2023-10-11T09:51:40.241611+0000","last_became_peered":"2023-10-11T09:51:40.241611+0000","last_unstale":"2023-10-11T09:51:41.720386+0000","last_undegraded":"2023-10-11T09:51:41.720386+0000","last_fullsized":"2023-10-11T09:51:41.720386+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T21:07:33.774700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"126'2","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.238131+0000","last_change":"2023-10-11T09:51:39.758774+0000","last_active":"2023-10-11T09:51:41.238131+0000","last_peered":"2023-10-11T09:51:41.238131+0000","last_clean":"2023-10-11T09:51:41.238131+0000","last_became_active":"2023-10-11T09:51:39.758473+0000","last_became_peered":"2023-10-11T09:51:39.758473+0000","last_unstale":"2023-10-11T09:51:41.238131+0000","last_undegraded":"2023-10-11T09:51:41.238131+0000","last_fullsized":"2023-10-11T09:51:41.238131+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:08:30.049861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.238190+0000","last_change":"2023-10-11T09:51:40.253057+0000","last_active":"2023-10-11T09:51:41.238190+0000","last_peered":"2023-10-11T09:51:41.238190+0000","last_clean":"2023-10-11T09:51:41.238190+0000","last_became_active":"2023-10-11T09:51:40.252422+0000","last_became_peered":"2023-10-11T09:51:40.252422+0000","last_unstale":"2023-10-11T09:51:41.238190+0000","last_undegraded":"2023-10-11T09:51:41.238190+0000","last_fullsized":"2023-10-11T09:51:41.238190+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:28:07.140682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"126'2","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.233121+0000","last_change":"2023-10-11T09:51:39.761478+0000","last_active":"2023-10-11T09:51:41.233121+0000","last_peered":"2023-10-11T09:51:41.233121+0000","last_clean":"2023-10-11T09:51:41.233121+0000","last_became_active":"2023-10-11T09:51:39.761141+0000","last_became_peered":"2023-10-11T09:51:39.761141+0000","last_unstale":"2023-10-11T09:51:41.233121+0000","last_undegraded":"2023-10-11T09:51:41.233121+0000","last_fullsized":"2023-10-11T09:51:41.233121+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:56:09.703238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.232686+0000","last_change":"2023-10-11T09:51:40.245620+0000","last_active":"2023-10-11T09:51:41.232686+0000","last_peered":"2023-10-11T09:51:41.232686+0000","last_clean":"2023-10-11T09:51:41.232686+0000","last_became_active":"2023-10-11T09:51:40.245345+0000","last_became_peered":"2023-10-11T09:51:40.245345+0000","last_unstale":"2023-10-11T09:51:41.232686+0000","last_undegraded":"2023-10-11T09:51:41.232686+0000","last_fullsized":"2023-10-11T09:51:41.232686+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:50:32.342142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"126'1","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.489893+0000","last_change":"2023-10-11T09:51:39.769545+0000","last_active":"2023-10-11T09:51:41.489893+0000","last_peered":"2023-10-11T09:51:41.489893+0000","last_clean":"2023-10-11T09:51:41.489893+0000","last_became_active":"2023-10-11T09:51:39.769239+0000","last_became_peered":"2023-10-11T09:51:39.769239+0000","last_unstale":"2023-10-11T09:51:41.489893+0000","last_undegraded":"2023-10-11T09:51:41.489893+0000","last_fullsized":"2023-10-11T09:51:41.489893+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:34:52.926023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.489838+0000","last_change":"2023-10-11T09:51:40.255038+0000","last_active":"2023-10-11T09:51:41.489838+0000","last_peered":"2023-10-11T09:51:41.489838+0000","last_clean":"2023-10-11T09:51:41.489838+0000","last_became_active":"2023-10-11T09:51:40.254779+0000","last_became_peered":"2023-10-11T09:51:40.254779+0000","last_unstale":"2023-10-11T09:51:41.489838+0000","last_undegraded":"2023-10-11T09:51:41.489838+0000","last_fullsized":"2023-10-11T09:51:41.489838+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:22:04.778725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"126'1","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.306368+0000","last_change":"2023-10-11T09:51:39.760507+0000","last_active":"2023-10-11T09:51:41.306368+0000","last_peered":"2023-10-11T09:51:41.306368+0000","last_clean":"2023-10-11T09:51:41.306368+0000","last_became_active":"2023-10-11T09:51:39.760186+0000","last_became_peered":"2023-10-11T09:51:39.760186+0000","last_unstale":"2023-10-11T09:51:41.306368+0000","last_undegraded":"2023-10-11T09:51:41.306368+0000","last_fullsized":"2023-10-11T09:51:41.306368+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T21:46:14.948200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.890870+0000","last_change":"2023-10-11T09:51:40.245724+0000","last_active":"2023-10-11T09:51:41.890870+0000","last_peered":"2023-10-11T09:51:41.890870+0000","last_clean":"2023-10-11T09:51:41.890870+0000","last_became_active":"2023-10-11T09:51:40.241919+0000","last_became_peered":"2023-10-11T09:51:40.241919+0000","last_unstale":"2023-10-11T09:51:41.890870+0000","last_undegraded":"2023-10-11T09:51:41.890870+0000","last_fullsized":"2023-10-11T09:51:41.890870+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:02:00.332753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.719953+0000","last_change":"2023-10-11T09:51:39.766976+0000","last_active":"2023-10-11T09:51:41.719953+0000","last_peered":"2023-10-11T09:51:41.719953+0000","last_clean":"2023-10-11T09:51:41.719953+0000","last_became_active":"2023-10-11T09:51:39.766308+0000","last_became_peered":"2023-10-11T09:51:39.766308+0000","last_unstale":"2023-10-11T09:51:41.719953+0000","last_undegraded":"2023-10-11T09:51:41.719953+0000","last_fullsized":"2023-10-11T09:51:41.719953+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:55:28.520528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.232977+0000","last_change":"2023-10-11T09:51:40.236412+0000","last_active":"2023-10-11T09:51:41.232977+0000","last_peered":"2023-10-11T09:51:41.232977+0000","last_clean":"2023-10-11T09:51:41.232977+0000","last_became_active":"2023-10-11T09:51:40.236121+0000","last_became_peered":"2023-10-11T09:51:40.236121+0000","last_unstale":"2023-10-11T09:51:41.232977+0000","last_undegraded":"2023-10-11T09:51:41.232977+0000","last_fullsized":"2023-10-11T09:51:41.232977+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:05:47.248308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.306571+0000","last_change":"2023-10-11T09:51:39.765589+0000","last_active":"2023-10-11T09:51:41.306571+0000","last_peered":"2023-10-11T09:51:41.306571+0000","last_clean":"2023-10-11T09:51:41.306571+0000","last_became_active":"2023-10-11T09:51:39.765312+0000","last_became_peered":"2023-10-11T09:51:39.765312+0000","last_unstale":"2023-10-11T09:51:41.306571+0000","last_undegraded":"2023-10-11T09:51:41.306571+0000","last_fullsized":"2023-10-11T09:51:41.306571+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:34:12.805264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.720208+0000","last_change":"2023-10-11T09:51:40.243504+0000","last_active":"2023-10-11T09:51:41.720208+0000","last_peered":"2023-10-11T09:51:41.720208+0000","last_clean":"2023-10-11T09:51:41.720208+0000","last_became_active":"2023-10-11T09:51:40.243158+0000","last_became_peered":"2023-10-11T09:51:40.243158+0000","last_unstale":"2023-10-11T09:51:41.720208+0000","last_undegraded":"2023-10-11T09:51:41.720208+0000","last_fullsized":"2023-10-11T09:51:41.720208+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:05:45.311991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"126'1","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.233305+0000","last_change":"2023-10-11T09:51:39.757450+0000","last_active":"2023-10-11T09:51:41.233305+0000","last_peered":"2023-10-11T09:51:41.233305+0000","last_clean":"2023-10-11T09:51:41.233305+0000","last_became_active":"2023-10-11T09:51:39.757157+0000","last_became_peered":"2023-10-11T09:51:39.757157+0000","last_unstale":"2023-10-11T09:51:41.233305+0000","last_undegraded":"2023-10-11T09:51:41.233305+0000","last_fullsized":"2023-10-11T09:51:41.233305+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:55:49.324206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.490205+0000","last_change":"2023-10-11T09:51:40.258759+0000","last_active":"2023-10-11T09:51:41.490205+0000","last_peered":"2023-10-11T09:51:41.490205+0000","last_clean":"2023-10-11T09:51:41.490205+0000","last_became_active":"2023-10-11T09:51:40.258406+0000","last_became_peered":"2023-10-11T09:51:40.258406+0000","last_unstale":"2023-10-11T09:51:41.490205+0000","last_undegraded":"2023-10-11T09:51:41.490205+0000","last_fullsized":"2023-10-11T09:51:41.490205+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:26:18.137695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"126'2","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.486398+0000","last_change":"2023-10-11T09:51:39.759996+0000","last_active":"2023-10-11T09:51:41.486398+0000","last_peered":"2023-10-11T09:51:41.486398+0000","last_clean":"2023-10-11T09:51:41.486398+0000","last_became_active":"2023-10-11T09:51:39.759752+0000","last_became_peered":"2023-10-11T09:51:39.759752+0000","last_unstale":"2023-10-11T09:51:41.486398+0000","last_undegraded":"2023-10-11T09:51:41.486398+0000","last_fullsized":"2023-10-11T09:51:41.486398+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:55:49.324206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.307324+0000","last_change":"2023-10-11T09:51:40.244807+0000","last_active":"2023-10-11T09:51:41.307324+0000","last_peered":"2023-10-11T09:51:41.307324+0000","last_clean":"2023-10-11T09:51:41.307324+0000","last_became_active":"2023-10-11T09:51:40.244486+0000","last_became_peered":"2023-10-11T09:51:40.244486+0000","last_unstale":"2023-10-11T09:51:41.307324+0000","last_undegraded":"2023-10-11T09:51:41.307324+0000","last_fullsized":"2023-10-11T09:51:41.307324+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:13:10.336993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"126'1","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.238257+0000","last_change":"2023-10-11T09:51:39.762960+0000","last_active":"2023-10-11T09:51:41.238257+0000","last_peered":"2023-10-11T09:51:41.238257+0000","last_clean":"2023-10-11T09:51:41.238257+0000","last_became_active":"2023-10-11T09:51:39.762632+0000","last_became_peered":"2023-10-11T09:51:39.762632+0000","last_unstale":"2023-10-11T09:51:41.238257+0000","last_undegraded":"2023-10-11T09:51:41.238257+0000","last_fullsized":"2023-10-11T09:51:41.238257+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:34:52.926023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.720893+0000","last_change":"2023-10-11T09:51:40.254679+0000","last_active":"2023-10-11T09:51:41.720893+0000","last_peered":"2023-10-11T09:51:41.720893+0000","last_clean":"2023-10-11T09:51:41.720893+0000","last_became_active":"2023-10-11T09:51:40.254360+0000","last_became_peered":"2023-10-11T09:51:40.254360+0000","last_unstale":"2023-10-11T09:51:41.720893+0000","last_undegraded":"2023-10-11T09:51:41.720893+0000","last_fullsized":"2023-10-11T09:51:41.720893+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:08:40.751877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.238071+0000","last_change":"2023-10-11T09:51:39.762288+0000","last_active":"2023-10-11T09:51:41.238071+0000","last_peered":"2023-10-11T09:51:41.238071+0000","last_clean":"2023-10-11T09:51:41.238071+0000","last_became_active":"2023-10-11T09:51:39.761990+0000","last_became_peered":"2023-10-11T09:51:39.761990+0000","last_unstale":"2023-10-11T09:51:41.238071+0000","last_undegraded":"2023-10-11T09:51:41.238071+0000","last_fullsized":"2023-10-11T09:51:41.238071+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:14:51.518896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.306342+0000","last_change":"2023-10-11T09:51:40.239135+0000","last_active":"2023-10-11T09:51:41.306342+0000","last_peered":"2023-10-11T09:51:41.306342+0000","last_clean":"2023-10-11T09:51:41.306342+0000","last_became_active":"2023-10-11T09:51:40.238854+0000","last_became_peered":"2023-10-11T09:51:40.238854+0000","last_unstale":"2023-10-11T09:51:41.306342+0000","last_undegraded":"2023-10-11T09:51:41.306342+0000","last_fullsized":"2023-10-11T09:51:41.306342+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:21:25.508173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"126'2","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.306454+0000","last_change":"2023-10-11T09:51:39.750952+0000","last_active":"2023-10-11T09:51:41.306454+0000","last_peered":"2023-10-11T09:51:41.306454+0000","last_clean":"2023-10-11T09:51:41.306454+0000","last_became_active":"2023-10-11T09:51:39.750660+0000","last_became_peered":"2023-10-11T09:51:39.750660+0000","last_unstale":"2023-10-11T09:51:41.306454+0000","last_undegraded":"2023-10-11T09:51:41.306454+0000","last_fullsized":"2023-10-11T09:51:41.306454+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:49:26.494642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.889788+0000","last_change":"2023-10-11T09:51:40.249160+0000","last_active":"2023-10-11T09:51:41.889788+0000","last_peered":"2023-10-11T09:51:41.889788+0000","last_clean":"2023-10-11T09:51:41.889788+0000","last_became_active":"2023-10-11T09:51:40.248865+0000","last_became_peered":"2023-10-11T09:51:40.248865+0000","last_unstale":"2023-10-11T09:51:41.889788+0000","last_undegraded":"2023-10-11T09:51:41.889788+0000","last_fullsized":"2023-10-11T09:51:41.889788+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:26:18.137695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.237403+0000","last_change":"2023-10-11T09:51:39.755430+0000","last_active":"2023-10-11T09:51:41.237403+0000","last_peered":"2023-10-11T09:51:41.237403+0000","last_clean":"2023-10-11T09:51:41.237403+0000","last_became_active":"2023-10-11T09:51:39.755138+0000","last_became_peered":"2023-10-11T09:51:39.755138+0000","last_unstale":"2023-10-11T09:51:41.237403+0000","last_undegraded":"2023-10-11T09:51:41.237403+0000","last_fullsized":"2023-10-11T09:51:41.237403+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:33:43.223786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.890774+0000","last_change":"2023-10-11T09:51:40.247463+0000","last_active":"2023-10-11T09:51:41.890774+0000","last_peered":"2023-10-11T09:51:41.890774+0000","last_clean":"2023-10-11T09:51:41.890774+0000","last_became_active":"2023-10-11T09:51:40.246842+0000","last_became_peered":"2023-10-11T09:51:40.246842+0000","last_unstale":"2023-10-11T09:51:41.890774+0000","last_undegraded":"2023-10-11T09:51:41.890774+0000","last_fullsized":"2023-10-11T09:51:41.890774+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:03:07.719820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.890418+0000","last_change":"2023-10-11T09:51:39.761386+0000","last_active":"2023-10-11T09:51:41.890418+0000","last_peered":"2023-10-11T09:51:41.890418+0000","last_clean":"2023-10-11T09:51:41.890418+0000","last_became_active":"2023-10-11T09:51:39.761141+0000","last_became_peered":"2023-10-11T09:51:39.761141+0000","last_unstale":"2023-10-11T09:51:41.890418+0000","last_undegraded":"2023-10-11T09:51:41.890418+0000","last_fullsized":"2023-10-11T09:51:41.890418+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:16:13.631830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.720077+0000","last_change":"2023-10-11T09:51:40.248437+0000","last_active":"2023-10-11T09:51:41.720077+0000","last_peered":"2023-10-11T09:51:41.720077+0000","last_clean":"2023-10-11T09:51:41.720077+0000","last_became_active":"2023-10-11T09:51:40.246812+0000","last_became_peered":"2023-10-11T09:51:40.246812+0000","last_unstale":"2023-10-11T09:51:41.720077+0000","last_undegraded":"2023-10-11T09:51:41.720077+0000","last_fullsized":"2023-10-11T09:51:41.720077+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:15:18.458331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.236708+0000","last_change":"2023-10-11T09:51:39.764803+0000","last_active":"2023-10-11T09:51:41.236708+0000","last_peered":"2023-10-11T09:51:41.236708+0000","last_clean":"2023-10-11T09:51:41.236708+0000","last_became_active":"2023-10-11T09:51:39.764523+0000","last_became_peered":"2023-10-11T09:51:39.764523+0000","last_unstale":"2023-10-11T09:51:41.236708+0000","last_undegraded":"2023-10-11T09:51:41.236708+0000","last_fullsized":"2023-10-11T09:51:41.236708+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:04:55.166171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.307213+0000","last_change":"2023-10-11T09:51:40.239668+0000","last_active":"2023-10-11T09:51:41.307213+0000","last_peered":"2023-10-11T09:51:41.307213+0000","last_clean":"2023-10-11T09:51:41.307213+0000","last_became_active":"2023-10-11T09:51:40.239299+0000","last_became_peered":"2023-10-11T09:51:40.239299+0000","last_unstale":"2023-10-11T09:51:41.307213+0000","last_undegraded":"2023-10-11T09:51:41.307213+0000","last_fullsized":"2023-10-11T09:51:41.307213+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:47:29.540098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"126'2","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.307140+0000","last_change":"2023-10-11T09:51:39.754135+0000","last_active":"2023-10-11T09:51:41.307140+0000","last_peered":"2023-10-11T09:51:41.307140+0000","last_clean":"2023-10-11T09:51:41.307140+0000","last_became_active":"2023-10-11T09:51:39.753607+0000","last_became_peered":"2023-10-11T09:51:39.753607+0000","last_unstale":"2023-10-11T09:51:41.307140+0000","last_undegraded":"2023-10-11T09:51:41.307140+0000","last_fullsized":"2023-10-11T09:51:41.307140+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:30:20.902689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.233093+0000","last_change":"2023-10-11T09:51:40.243718+0000","last_active":"2023-10-11T09:51:41.233093+0000","last_peered":"2023-10-11T09:51:41.233093+0000","last_clean":"2023-10-11T09:51:41.233093+0000","last_became_active":"2023-10-11T09:51:40.243408+0000","last_became_peered":"2023-10-11T09:51:40.243408+0000","last_unstale":"2023-10-11T09:51:41.233093+0000","last_undegraded":"2023-10-11T09:51:41.233093+0000","last_fullsized":"2023-10-11T09:51:41.233093+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:09:53.694631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.237666+0000","last_change":"2023-10-11T09:51:39.770306+0000","last_active":"2023-10-11T09:51:41.237666+0000","last_peered":"2023-10-11T09:51:41.237666+0000","last_clean":"2023-10-11T09:51:41.237666+0000","last_became_active":"2023-10-11T09:51:39.770006+0000","last_became_peered":"2023-10-11T09:51:39.770006+0000","last_unstale":"2023-10-11T09:51:41.237666+0000","last_undegraded":"2023-10-11T09:51:41.237666+0000","last_fullsized":"2023-10-11T09:51:41.237666+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:56:09.703238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.238071+0000","last_change":"2023-10-11T09:51:40.255927+0000","last_active":"2023-10-11T09:51:41.238071+0000","last_peered":"2023-10-11T09:51:41.238071+0000","last_clean":"2023-10-11T09:51:41.238071+0000","last_became_active":"2023-10-11T09:51:40.253476+0000","last_became_peered":"2023-10-11T09:51:40.253476+0000","last_unstale":"2023-10-11T09:51:41.238071+0000","last_undegraded":"2023-10-11T09:51:41.238071+0000","last_fullsized":"2023-10-11T09:51:41.238071+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:26:18.137695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.237710+0000","last_change":"2023-10-11T09:51:39.769691+0000","last_active":"2023-10-11T09:51:41.237710+0000","last_peered":"2023-10-11T09:51:41.237710+0000","last_clean":"2023-10-11T09:51:41.237710+0000","last_became_active":"2023-10-11T09:51:39.769404+0000","last_became_peered":"2023-10-11T09:51:39.769404+0000","last_unstale":"2023-10-11T09:51:41.237710+0000","last_undegraded":"2023-10-11T09:51:41.237710+0000","last_fullsized":"2023-10-11T09:51:41.237710+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:55:49.324206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.486327+0000","last_change":"2023-10-11T09:51:40.241206+0000","last_active":"2023-10-11T09:51:41.486327+0000","last_peered":"2023-10-11T09:51:41.486327+0000","last_clean":"2023-10-11T09:51:41.486327+0000","last_became_active":"2023-10-11T09:51:40.240879+0000","last_became_peered":"2023-10-11T09:51:40.240879+0000","last_unstale":"2023-10-11T09:51:41.486327+0000","last_undegraded":"2023-10-11T09:51:41.486327+0000","last_fullsized":"2023-10-11T09:51:41.486327+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:03:55.584256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.307294+0000","last_change":"2023-10-11T09:51:39.764863+0000","last_active":"2023-10-11T09:51:41.307294+0000","last_peered":"2023-10-11T09:51:41.307294+0000","last_clean":"2023-10-11T09:51:41.307294+0000","last_became_active":"2023-10-11T09:51:39.764591+0000","last_became_peered":"2023-10-11T09:51:39.764591+0000","last_unstale":"2023-10-11T09:51:41.307294+0000","last_undegraded":"2023-10-11T09:51:41.307294+0000","last_fullsized":"2023-10-11T09:51:41.307294+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:57:52.984262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.306609+0000","last_change":"2023-10-11T09:51:40.244133+0000","last_active":"2023-10-11T09:51:41.306609+0000","last_peered":"2023-10-11T09:51:41.306609+0000","last_clean":"2023-10-11T09:51:41.306609+0000","last_became_active":"2023-10-11T09:51:40.243867+0000","last_became_peered":"2023-10-11T09:51:40.243867+0000","last_unstale":"2023-10-11T09:51:41.306609+0000","last_undegraded":"2023-10-11T09:51:41.306609+0000","last_fullsized":"2023-10-11T09:51:41.306609+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:39:37.082941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.490133+0000","last_change":"2023-10-11T09:51:39.761598+0000","last_active":"2023-10-11T09:51:41.490133+0000","last_peered":"2023-10-11T09:51:41.490133+0000","last_clean":"2023-10-11T09:51:41.490133+0000","last_became_active":"2023-10-11T09:51:39.761341+0000","last_became_peered":"2023-10-11T09:51:39.761341+0000","last_unstale":"2023-10-11T09:51:41.490133+0000","last_undegraded":"2023-10-11T09:51:41.490133+0000","last_fullsized":"2023-10-11T09:51:41.490133+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:14:51.518896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.890083+0000","last_change":"2023-10-11T09:51:40.246165+0000","last_active":"2023-10-11T09:51:41.890083+0000","last_peered":"2023-10-11T09:51:41.890083+0000","last_clean":"2023-10-11T09:51:41.890083+0000","last_became_active":"2023-10-11T09:51:40.245341+0000","last_became_peered":"2023-10-11T09:51:40.245341+0000","last_unstale":"2023-10-11T09:51:41.890083+0000","last_undegraded":"2023-10-11T09:51:41.890083+0000","last_fullsized":"2023-10-11T09:51:41.890083+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:41:00.113202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.490063+0000","last_change":"2023-10-11T09:51:39.761270+0000","last_active":"2023-10-11T09:51:41.490063+0000","last_peered":"2023-10-11T09:51:41.490063+0000","last_clean":"2023-10-11T09:51:41.490063+0000","last_became_active":"2023-10-11T09:51:39.760897+0000","last_became_peered":"2023-10-11T09:51:39.760897+0000","last_unstale":"2023-10-11T09:51:41.490063+0000","last_undegraded":"2023-10-11T09:51:41.490063+0000","last_fullsized":"2023-10-11T09:51:41.490063+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:04:29.131745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.237241+0000","last_change":"2023-10-11T09:51:40.255584+0000","last_active":"2023-10-11T09:51:41.237241+0000","last_peered":"2023-10-11T09:51:41.237241+0000","last_clean":"2023-10-11T09:51:41.237241+0000","last_became_active":"2023-10-11T09:51:40.252476+0000","last_became_peered":"2023-10-11T09:51:40.252476+0000","last_unstale":"2023-10-11T09:51:41.237241+0000","last_undegraded":"2023-10-11T09:51:41.237241+0000","last_fullsized":"2023-10-11T09:51:41.237241+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:22:04.778725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.306671+0000","last_change":"2023-10-11T09:51:39.764105+0000","last_active":"2023-10-11T09:51:41.306671+0000","last_peered":"2023-10-11T09:51:41.306671+0000","last_clean":"2023-10-11T09:51:41.306671+0000","last_became_active":"2023-10-11T09:51:39.762581+0000","last_became_peered":"2023-10-11T09:51:39.762581+0000","last_unstale":"2023-10-11T09:51:41.306671+0000","last_undegraded":"2023-10-11T09:51:41.306671+0000","last_fullsized":"2023-10-11T09:51:41.306671+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:43:47.401460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.232983+0000","last_change":"2023-10-11T09:51:40.258104+0000","last_active":"2023-10-11T09:51:41.232983+0000","last_peered":"2023-10-11T09:51:41.232983+0000","last_clean":"2023-10-11T09:51:41.232983+0000","last_became_active":"2023-10-11T09:51:40.257764+0000","last_became_peered":"2023-10-11T09:51:40.257764+0000","last_unstale":"2023-10-11T09:51:41.232983+0000","last_undegraded":"2023-10-11T09:51:41.232983+0000","last_fullsized":"2023-10-11T09:51:41.232983+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:26:40.308306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.236725+0000","last_change":"2023-10-11T09:51:39.766696+0000","last_active":"2023-10-11T09:51:41.236725+0000","last_peered":"2023-10-11T09:51:41.236725+0000","last_clean":"2023-10-11T09:51:41.236725+0000","last_became_active":"2023-10-11T09:51:39.766421+0000","last_became_peered":"2023-10-11T09:51:39.766421+0000","last_unstale":"2023-10-11T09:51:41.236725+0000","last_undegraded":"2023-10-11T09:51:41.236725+0000","last_fullsized":"2023-10-11T09:51:41.236725+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:56:41.273315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.720850+0000","last_change":"2023-10-11T09:51:40.242365+0000","last_active":"2023-10-11T09:51:41.720850+0000","last_peered":"2023-10-11T09:51:41.720850+0000","last_clean":"2023-10-11T09:51:41.720850+0000","last_became_active":"2023-10-11T09:51:40.241684+0000","last_became_peered":"2023-10-11T09:51:40.241684+0000","last_unstale":"2023-10-11T09:51:41.720850+0000","last_undegraded":"2023-10-11T09:51:41.720850+0000","last_fullsized":"2023-10-11T09:51:41.720850+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:22:04.778725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"126'6","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.890457+0000","last_change":"2023-10-11T09:51:39.758419+0000","last_active":"2023-10-11T09:51:41.890457+0000","last_peered":"2023-10-11T09:51:41.890457+0000","last_clean":"2023-10-11T09:51:41.890457+0000","last_became_active":"2023-10-11T09:51:39.758166+0000","last_became_peered":"2023-10-11T09:51:39.758166+0000","last_unstale":"2023-10-11T09:51:41.890457+0000","last_undegraded":"2023-10-11T09:51:41.890457+0000","last_fullsized":"2023-10-11T09:51:41.890457+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:08:30.049861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":94,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-11T09:51:40.229255+0000","last_change":"2023-10-11T09:40:36.151839+0000","last_active":"2023-10-11T09:51:40.229255+0000","last_peered":"2023-10-11T09:51:40.229255+0000","last_clean":"2023-10-11T09:51:40.229255+0000","last_became_active":"2023-10-11T09:40:36.151179+0000","last_became_peered":"2023-10-11T09:40:36.151179+0000","last_unstale":"2023-10-11T09:51:40.229255+0000","last_undegraded":"2023-10-11T09:51:40.229255+0000","last_fullsized":"2023-10-11T09:51:40.229255+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.890734+0000","last_change":"2023-10-11T09:51:40.247578+0000","last_active":"2023-10-11T09:51:41.890734+0000","last_peered":"2023-10-11T09:51:41.890734+0000","last_clean":"2023-10-11T09:51:41.890734+0000","last_became_active":"2023-10-11T09:51:40.247306+0000","last_became_peered":"2023-10-11T09:51:40.247306+0000","last_unstale":"2023-10-11T09:51:41.890734+0000","last_undegraded":"2023-10-11T09:51:41.890734+0000","last_fullsized":"2023-10-11T09:51:41.890734+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:43:05.751413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"126'3","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.720326+0000","last_change":"2023-10-11T09:51:39.763613+0000","last_active":"2023-10-11T09:51:41.720326+0000","last_peered":"2023-10-11T09:51:41.720326+0000","last_clean":"2023-10-11T09:51:41.720326+0000","last_became_active":"2023-10-11T09:51:39.763112+0000","last_became_peered":"2023-10-11T09:51:39.763112+0000","last_unstale":"2023-10-11T09:51:41.720326+0000","last_undegraded":"2023-10-11T09:51:41.720326+0000","last_fullsized":"2023-10-11T09:51:41.720326+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:29:03.154537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":101,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-11T09:51:40.232622+0000","last_change":"2023-10-11T09:40:36.150224+0000","last_active":"2023-10-11T09:51:40.232622+0000","last_peered":"2023-10-11T09:51:40.232622+0000","last_clean":"2023-10-11T09:51:40.232622+0000","last_became_active":"2023-10-11T09:40:36.149919+0000","last_became_peered":"2023-10-11T09:40:36.149919+0000","last_unstale":"2023-10-11T09:51:40.232622+0000","last_undegraded":"2023-10-11T09:51:40.232622+0000","last_fullsized":"2023-10-11T09:51:40.232622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.720121+0000","last_change":"2023-10-11T09:51:40.254443+0000","last_active":"2023-10-11T09:51:41.720121+0000","last_peered":"2023-10-11T09:51:41.720121+0000","last_clean":"2023-10-11T09:51:41.720121+0000","last_became_active":"2023-10-11T09:51:40.254121+0000","last_became_peered":"2023-10-11T09:51:40.254121+0000","last_unstale":"2023-10-11T09:51:41.720121+0000","last_undegraded":"2023-10-11T09:51:41.720121+0000","last_fullsized":"2023-10-11T09:51:41.720121+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T21:40:43.880040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.720419+0000","last_change":"2023-10-11T09:51:39.762434+0000","last_active":"2023-10-11T09:51:41.720419+0000","last_peered":"2023-10-11T09:51:41.720419+0000","last_clean":"2023-10-11T09:51:41.720419+0000","last_became_active":"2023-10-11T09:51:39.762154+0000","last_became_peered":"2023-10-11T09:51:39.762154+0000","last_unstale":"2023-10-11T09:51:41.720419+0000","last_undegraded":"2023-10-11T09:51:41.720419+0000","last_fullsized":"2023-10-11T09:51:41.720419+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:50:31.329585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":123,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-11T09:51:40.234279+0000","last_change":"2023-10-11T09:40:36.162174+0000","last_active":"2023-10-11T09:51:40.234279+0000","last_peered":"2023-10-11T09:51:40.234279+0000","last_clean":"2023-10-11T09:51:40.234279+0000","last_became_active":"2023-10-11T09:40:36.161710+0000","last_became_peered":"2023-10-11T09:40:36.161710+0000","last_unstale":"2023-10-11T09:51:40.234279+0000","last_undegraded":"2023-10-11T09:51:40.234279+0000","last_fullsized":"2023-10-11T09:51:40.234279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:57:03.630878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.237264+0000","last_change":"2023-10-11T09:51:40.258732+0000","last_active":"2023-10-11T09:51:41.237264+0000","last_peered":"2023-10-11T09:51:41.237264+0000","last_clean":"2023-10-11T09:51:41.237264+0000","last_became_active":"2023-10-11T09:51:40.257428+0000","last_became_peered":"2023-10-11T09:51:40.257428+0000","last_unstale":"2023-10-11T09:51:41.237264+0000","last_undegraded":"2023-10-11T09:51:41.237264+0000","last_fullsized":"2023-10-11T09:51:41.237264+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:02:39.408532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.237462+0000","last_change":"2023-10-11T09:51:39.769877+0000","last_active":"2023-10-11T09:51:41.237462+0000","last_peered":"2023-10-11T09:51:41.237462+0000","last_clean":"2023-10-11T09:51:41.237462+0000","last_became_active":"2023-10-11T09:51:39.769567+0000","last_became_peered":"2023-10-11T09:51:39.769567+0000","last_unstale":"2023-10-11T09:51:41.237462+0000","last_undegraded":"2023-10-11T09:51:41.237462+0000","last_fullsized":"2023-10-11T09:51:41.237462+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:53:16.999254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":101,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-11T09:51:40.231423+0000","last_change":"2023-10-11T09:40:36.160620+0000","last_active":"2023-10-11T09:51:40.231423+0000","last_peered":"2023-10-11T09:51:40.231423+0000","last_clean":"2023-10-11T09:51:40.231423+0000","last_became_active":"2023-10-11T09:40:36.160171+0000","last_became_peered":"2023-10-11T09:40:36.160171+0000","last_unstale":"2023-10-11T09:51:40.231423+0000","last_undegraded":"2023-10-11T09:51:40.231423+0000","last_fullsized":"2023-10-11T09:51:40.231423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.485870+0000","last_change":"2023-10-11T09:51:40.243183+0000","last_active":"2023-10-11T09:51:41.485870+0000","last_peered":"2023-10-11T09:51:41.485870+0000","last_clean":"2023-10-11T09:51:41.485870+0000","last_became_active":"2023-10-11T09:51:40.242927+0000","last_became_peered":"2023-10-11T09:51:40.242927+0000","last_unstale":"2023-10-11T09:51:41.485870+0000","last_undegraded":"2023-10-11T09:51:41.485870+0000","last_fullsized":"2023-10-11T09:51:41.485870+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:06:10.733214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.890049+0000","last_change":"2023-10-11T09:51:39.760904+0000","last_active":"2023-10-11T09:51:41.890049+0000","last_peered":"2023-10-11T09:51:41.890049+0000","last_clean":"2023-10-11T09:51:41.890049+0000","last_became_active":"2023-10-11T09:51:39.759110+0000","last_became_peered":"2023-10-11T09:51:39.759110+0000","last_unstale":"2023-10-11T09:51:41.890049+0000","last_undegraded":"2023-10-11T09:51:41.890049+0000","last_fullsized":"2023-10-11T09:51:41.890049+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:04:55.166171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":99,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-11T09:51:40.235717+0000","last_change":"2023-10-11T09:40:36.151710+0000","last_active":"2023-10-11T09:51:40.235717+0000","last_peered":"2023-10-11T09:51:40.235717+0000","last_clean":"2023-10-11T09:51:40.235717+0000","last_became_active":"2023-10-11T09:40:36.151219+0000","last_became_peered":"2023-10-11T09:40:36.151219+0000","last_unstale":"2023-10-11T09:51:40.235717+0000","last_undegraded":"2023-10-11T09:51:40.235717+0000","last_fullsized":"2023-10-11T09:51:40.235717+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.232642+0000","last_change":"2023-10-11T09:51:40.252680+0000","last_active":"2023-10-11T09:51:41.232642+0000","last_peered":"2023-10-11T09:51:41.232642+0000","last_clean":"2023-10-11T09:51:41.232642+0000","last_became_active":"2023-10-11T09:51:40.252262+0000","last_became_peered":"2023-10-11T09:51:40.252262+0000","last_unstale":"2023-10-11T09:51:41.232642+0000","last_undegraded":"2023-10-11T09:51:41.232642+0000","last_fullsized":"2023-10-11T09:51:41.232642+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T21:09:28.097416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":11,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.890974+0000","last_change":"2023-10-11T09:51:39.763005+0000","last_active":"2023-10-11T09:51:41.890974+0000","last_peered":"2023-10-11T09:51:41.890974+0000","last_clean":"2023-10-11T09:51:41.890974+0000","last_became_active":"2023-10-11T09:51:39.762755+0000","last_became_peered":"2023-10-11T09:51:39.762755+0000","last_unstale":"2023-10-11T09:51:41.890974+0000","last_undegraded":"2023-10-11T09:51:41.890974+0000","last_fullsized":"2023-10-11T09:51:41.890974+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T21:34:20.513069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":101,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-11T09:51:40.221500+0000","last_change":"2023-10-11T09:40:36.216248+0000","last_active":"2023-10-11T09:51:40.221500+0000","last_peered":"2023-10-11T09:51:40.221500+0000","last_clean":"2023-10-11T09:51:40.221500+0000","last_became_active":"2023-10-11T09:40:36.215890+0000","last_became_peered":"2023-10-11T09:40:36.215890+0000","last_unstale":"2023-10-11T09:51:40.221500+0000","last_undegraded":"2023-10-11T09:51:40.221500+0000","last_fullsized":"2023-10-11T09:51:40.221500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.237582+0000","last_change":"2023-10-11T09:51:40.239441+0000","last_active":"2023-10-11T09:51:41.237582+0000","last_peered":"2023-10-11T09:51:41.237582+0000","last_clean":"2023-10-11T09:51:41.237582+0000","last_became_active":"2023-10-11T09:51:40.239142+0000","last_became_peered":"2023-10-11T09:51:40.239142+0000","last_unstale":"2023-10-11T09:51:41.237582+0000","last_undegraded":"2023-10-11T09:51:41.237582+0000","last_fullsized":"2023-10-11T09:51:41.237582+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:53:28.732197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"126'2","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.890564+0000","last_change":"2023-10-11T09:51:39.755912+0000","last_active":"2023-10-11T09:51:41.890564+0000","last_peered":"2023-10-11T09:51:41.890564+0000","last_clean":"2023-10-11T09:51:41.890564+0000","last_became_active":"2023-10-11T09:51:39.755662+0000","last_became_peered":"2023-10-11T09:51:39.755662+0000","last_unstale":"2023-10-11T09:51:41.890564+0000","last_undegraded":"2023-10-11T09:51:41.890564+0000","last_fullsized":"2023-10-11T09:51:41.890564+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:33:43.223786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":123,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-11T09:51:40.240434+0000","last_change":"2023-10-11T09:40:36.149739+0000","last_active":"2023-10-11T09:51:40.240434+0000","last_peered":"2023-10-11T09:51:40.240434+0000","last_clean":"2023-10-11T09:51:40.240434+0000","last_became_active":"2023-10-11T09:40:36.149419+0000","last_became_peered":"2023-10-11T09:40:36.149419+0000","last_unstale":"2023-10-11T09:51:40.240434+0000","last_undegraded":"2023-10-11T09:51:40.240434+0000","last_fullsized":"2023-10-11T09:51:40.240434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:54:50.640362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":10,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.237570+0000","last_change":"2023-10-11T09:51:40.256975+0000","last_active":"2023-10-11T09:51:41.237570+0000","last_peered":"2023-10-11T09:51:41.237570+0000","last_clean":"2023-10-11T09:51:41.237570+0000","last_became_active":"2023-10-11T09:51:40.253250+0000","last_became_peered":"2023-10-11T09:51:40.253250+0000","last_unstale":"2023-10-11T09:51:41.237570+0000","last_undegraded":"2023-10-11T09:51:41.237570+0000","last_fullsized":"2023-10-11T09:51:41.237570+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:39.729189+0000","last_clean_scrub_stamp":"2023-10-11T09:51:39.729189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:26:52.594103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"126'8","reported_seq":19,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-11T09:51:41.889940+0000","last_change":"2023-10-11T09:51:39.758542+0000","last_active":"2023-10-11T09:51:41.889940+0000","last_peered":"2023-10-11T09:51:41.889940+0000","last_clean":"2023-10-11T09:51:41.889940+0000","last_became_active":"2023-10-11T09:51:39.758290+0000","last_became_peered":"2023-10-11T09:51:39.758290+0000","last_unstale":"2023-10-11T09:51:41.889940+0000","last_undegraded":"2023-10-11T09:51:41.889940+0000","last_fullsized":"2023-10-11T09:51:41.889940+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:51:38.716631+0000","last_clean_scrub_stamp":"2023-10-11T09:51:38.716631+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:56:41.273315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":99,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-11T09:51:40.236335+0000","last_change":"2023-10-11T09:40:36.148773+0000","last_active":"2023-10-11T09:51:40.236335+0000","last_peered":"2023-10-11T09:51:40.236335+0000","last_clean":"2023-10-11T09:51:40.236335+0000","last_became_active":"2023-10-11T09:40:36.148489+0000","last_became_peered":"2023-10-11T09:40:36.148489+0000","last_unstale":"2023-10-11T09:51:40.236335+0000","last_undegraded":"2023-10-11T09:51:40.236335+0000","last_fullsized":"2023-10-11T09:51:40.236335+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":276,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-11T09:51:40.239874+0000","last_change":"2023-10-11T09:40:29.874372+0000","last_active":"2023-10-11T09:51:40.239874+0000","last_peered":"2023-10-11T09:51:40.239874+0000","last_clean":"2023-10-11T09:51:40.239874+0000","last_became_active":"2023-10-11T09:40:29.873621+0000","last_became_peered":"2023-10-11T09:40:29.873621+0000","last_unstale":"2023-10-11T09:51:40.239874+0000","last_undegraded":"2023-10-11T09:51:40.239874+0000","last_fullsized":"2023-10-11T09:51:40.239874+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_clean_scrub_stamp":"2023-10-11T09:40:28.742276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:22:07.985311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574987,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33184,"kb_used_data":2136,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338656,"statfs":{"total":96636764160,"available":96602783744,"internally_reserved":0,"allocated":2187264,"data_stored":1359303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:50:51 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.0369999999999999,"15min":1.0960000000000001},"min":{"1min":0.78000000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.554,"5min":1.7390000000000001,"15min":1.903},"last":1.008},{"interface":"front","average":{"1min":1.117,"5min":1.0429999999999999,"15min":1.095},"min":{"1min":0.873,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.554,"5min":1.5760000000000001,"15min":1.696},"last":1.155}]},{"osd":1,"last update":"Wed Oct 11 09:50:51 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.097,"15min":1.1200000000000001},"min":{"1min":0.72799999999999998,"5min":0.66800000000000004,"15min":0.63100000000000001},"max":{"1min":1.524,"5min":1.645,"15min":2.25},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.079,"5min":1.038,"15min":1.091},"min":{"1min":0.754,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.3300000000000001,"5min":1.587,"15min":1.696},"last":0.97299999999999998}]},{"osd":2,"last update":"Wed Oct 11 09:50:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1000000000000001,"15min":1.1100000000000001},"min":{"1min":0.72099999999999997,"5min":0.59499999999999997,"15min":0.58999999999999997},"max":{"1min":1.486,"5min":1.6339999999999999,"15min":1.873},"last":1.111},{"interface":"front","average":{"1min":1.236,"5min":1.1679999999999999,"15min":1.1539999999999999},"min":{"1min":1.0129999999999999,"5min":0.66400000000000003,"15min":0.65500000000000003},"max":{"1min":1.4610000000000001,"5min":1.78,"15min":1.78},"last":1.579}]},{"osd":3,"last update":"Wed Oct 11 09:50:51 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.2050000000000001,"15min":1.2230000000000001},"min":{"1min":0.94499999999999995,"5min":0.70499999999999996,"15min":0.64900000000000002},"max":{"1min":1.6220000000000001,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.425},{"interface":"front","average":{"1min":1.337,"5min":1.2789999999999999,"15min":1.282},"min":{"1min":0.752,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.744,"5min":1.8720000000000001,"15min":2.1579999999999999},"last":1.514}]},{"osd":4,"last update":"Wed Oct 11 09:50:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0449999999999999,"15min":1.1240000000000001},"min":{"1min":0.72899999999999998,"5min":0.64900000000000002,"15min":0.53300000000000003},"max":{"1min":1.323,"5min":1.661,"15min":2.2010000000000001},"last":0.87},{"interface":"front","average":{"1min":1.232,"5min":1.208,"15min":1.2330000000000001},"min":{"1min":0.873,"5min":0.67200000000000004,"15min":0.63500000000000001},"max":{"1min":1.706,"5min":1.768,"15min":1.9339999999999999},"last":1.28}]},{"osd":5,"last update":"Wed Oct 11 09:50:51 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.272,"15min":1.2809999999999999},"min":{"1min":0.90100000000000002,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.252},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.196,"15min":1.284},"min":{"1min":0.83899999999999997,"5min":0.78300000000000003,"15min":0.65900000000000003},"max":{"1min":1.718,"5min":1.9450000000000001,"15min":3.0750000000000002},"last":1.3580000000000001}]},{"osd":6,"last update":"Wed Oct 11 09:50:51 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.323,"15min":1.397},"min":{"1min":1.034,"5min":0.77000000000000002,"15min":0.70999999999999996},"max":{"1min":1.6399999999999999,"5min":1.835,"15min":3.137},"last":1.552},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3580000000000001,"15min":1.387},"min":{"1min":0.82799999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.696,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.2090000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574986,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32592,"kb_used_data":1544,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1581056,"data_stored":768741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:50:43 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.127,"15min":1.038},"min":{"1min":0.65700000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.8400000000000001,"5min":7.0730000000000004,"15min":7.0730000000000004},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.177,"15min":1.073},"min":{"1min":0.60399999999999998,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.5229999999999999,"5min":8.8450000000000006,"15min":8.8450000000000006},"last":1.371}]},{"osd":1,"last update":"Wed Oct 11 09:50:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.3049999999999999,"15min":1.226},"min":{"1min":0.75800000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.4770000000000001,"5min":15.523,"15min":15.523},"last":1.6439999999999999},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.302,"15min":1.2450000000000001},"min":{"1min":0.79000000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.657,"5min":15.253,"15min":15.253},"last":1.171}]},{"osd":2,"last update":"Wed Oct 11 09:50:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.111,"15min":1.137},"min":{"1min":0.873,"5min":0.434,"15min":0.434},"max":{"1min":1.4750000000000001,"5min":2.7919999999999998,"15min":2.7919999999999998},"last":1.321},{"interface":"front","average":{"1min":1.107,"5min":1.151,"15min":1.202},"min":{"1min":0.77800000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.7929999999999999,"5min":5.0990000000000002,"15min":5.0990000000000002},"last":1.296}]},{"osd":3,"last update":"Wed Oct 11 09:50:43 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.194,"15min":1.1659999999999999},"min":{"1min":0.79600000000000004,"5min":0.70499999999999996,"15min":0.627},"max":{"1min":1.754,"5min":11.945,"15min":11.945},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.214,"5min":1.1279999999999999,"15min":1.2150000000000001},"min":{"1min":0.84499999999999997,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.5329999999999999,"5min":1.5980000000000001,"15min":3.0819999999999999},"last":1.502}]},{"osd":4,"last update":"Wed Oct 11 09:50:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.274,"15min":1.2849999999999999},"min":{"1min":0.58399999999999996,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.542,"5min":7.5599999999999996,"15min":7.5599999999999996},"last":1.54},{"interface":"front","average":{"1min":1.202,"5min":1.3149999999999999,"15min":1.3280000000000001},"min":{"1min":0.75900000000000001,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.839,"5min":8.7520000000000007,"15min":8.7520000000000007},"last":1.23}]},{"osd":5,"last update":"Wed Oct 11 09:50:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.1830000000000001,"15min":1.2370000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.573,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.26,"15min":1.339},"min":{"1min":0.72099999999999997,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.595,"5min":2.4390000000000001,"15min":2.7290000000000001},"last":1.4590000000000001}]},{"osd":7,"last update":"Wed Oct 11 09:50:43 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.319,"15min":1.359},"min":{"1min":0.92700000000000005,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.931,"5min":8.7970000000000006,"15min":8.7970000000000006},"last":1.446},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.3620000000000001,"15min":1.429},"min":{"1min":0.82099999999999995,"5min":0.625,"15min":0.625},"max":{"1min":1.7430000000000001,"5min":2.641,"15min":2.641},"last":1.5760000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574986,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32596,"kb_used_data":1548,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1585152,"data_stored":768319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:50:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":0.999,"15min":1},"min":{"1min":0.70099999999999996,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":1.7030000000000001},{"interface":"front","average":{"1min":1.044,"5min":1.0469999999999999,"15min":1.008},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.46500000000000002},"max":{"1min":1.536,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":1.177}]},{"osd":1,"last update":"Wed Oct 11 09:50:46 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.0780000000000001,"15min":1.008},"min":{"1min":0.70999999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.48,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.7190000000000001},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.111,"15min":1.0780000000000001},"min":{"1min":0.68000000000000005,"5min":0.61599999999999999,"15min":0.437},"max":{"1min":1.5880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.4359999999999999}]},{"osd":2,"last update":"Wed Oct 11 09:50:46 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.171,"15min":1.1819999999999999},"min":{"1min":0.746,"5min":0.746,"15min":0.73899999999999999},"max":{"1min":1.669,"5min":2.133,"15min":2.133},"last":1.5},{"interface":"front","average":{"1min":1.298,"5min":1.204,"15min":1.2150000000000001},"min":{"1min":0.878,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.665}]},{"osd":3,"last update":"Wed Oct 11 09:50:46 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.1299999999999999,"15min":1.127},"min":{"1min":0.752,"5min":0.59799999999999998,"15min":0.56999999999999995},"max":{"1min":1.53,"5min":1.8859999999999999,"15min":1.921},"last":1.264},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.1719999999999999,"15min":1.163},"min":{"1min":0.82299999999999995,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.9690000000000001,"5min":1.9690000000000001,"15min":2.0209999999999999},"last":1.298}]},{"osd":4,"last update":"Wed Oct 11 09:50:46 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.2090000000000001,"15min":1.1899999999999999},"min":{"1min":0.94199999999999995,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.0169999999999999,"5min":2.117,"15min":2.117},"last":1.518},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.1850000000000001,"15min":1.1870000000000001},"min":{"1min":0.82999999999999996,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.887,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.0529999999999999}]},{"osd":6,"last update":"Wed Oct 11 09:50:46 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.2609999999999999,"15min":1.2649999999999999},"min":{"1min":0.90100000000000002,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.8859999999999999,"5min":1.9239999999999999,"15min":2.4529999999999998},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.403,"5min":1.3240000000000001,"15min":1.3160000000000001},"min":{"1min":0.91400000000000003,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.621}]},{"osd":7,"last update":"Wed Oct 11 09:50:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.3380000000000001,"15min":1.3600000000000001},"min":{"1min":1.071,"5min":0.61899999999999999,"15min":0.57699999999999996},"max":{"1min":2.004,"5min":2.347,"15min":3.6120000000000001},"last":1.48},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.423,"15min":1.4410000000000001},"min":{"1min":1.1240000000000001,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":2.016,"5min":2.2679999999999998,"15min":3.2109999999999999},"last":1.6399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574986,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1552,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1589248,"data_stored":769723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:50:50 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.056,"15min":1.0580000000000001},"min":{"1min":0.77500000000000002,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.3979999999999999,"5min":1.579,"15min":1.7170000000000001},"last":1.3200000000000001},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0720000000000001,"15min":1.0860000000000001},"min":{"1min":0.61899999999999999,"5min":0.60199999999999998,"15min":0.54700000000000004},"max":{"1min":1.4079999999999999,"5min":1.669,"15min":2.1160000000000001},"last":1.292}]},{"osd":1,"last update":"Wed Oct 11 09:50:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.091,"15min":1.0860000000000001},"min":{"1min":0.747,"5min":0.66500000000000004,"15min":0.60699999999999998},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.141,"15min":1.1439999999999999},"min":{"1min":0.90700000000000003,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.486,"5min":1.7849999999999999,"15min":1.8220000000000001},"last":1.5549999999999999}]},{"osd":2,"last update":"Wed Oct 11 09:50:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.206,"15min":1.194},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.52,"5min":1.9159999999999999,"15min":1.919},"last":2.0710000000000002},{"interface":"front","average":{"1min":1.286,"5min":1.232,"15min":1.218},"min":{"1min":0.78800000000000003,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.5409999999999999,"5min":1.8300000000000001,"15min":1.8859999999999999},"last":2.1709999999999998}]},{"osd":3,"last update":"Wed Oct 11 09:50:50 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.1259999999999999,"15min":1.129},"min":{"1min":0.82799999999999996,"5min":0.72599999999999998,"15min":0.60199999999999998},"max":{"1min":1.5760000000000001,"5min":1.6839999999999999,"15min":1.6930000000000001},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.1659999999999999,"15min":1.2410000000000001},"min":{"1min":0.89300000000000002,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.4470000000000001,"5min":1.5329999999999999,"15min":2.3340000000000001},"last":1.145}]},{"osd":5,"last update":"Wed Oct 11 09:50:50 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.175,"15min":1.1890000000000001},"min":{"1min":0.82299999999999995,"5min":0.753,"15min":0.55700000000000005},"max":{"1min":1.5760000000000001,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":1.23},{"interface":"front","average":{"1min":1.159,"5min":1.155,"15min":1.1850000000000001},"min":{"1min":0.81999999999999995,"5min":0.63300000000000001,"15min":0.61199999999999999},"max":{"1min":1.51,"5min":1.732,"15min":1.8520000000000001},"last":1.198}]},{"osd":6,"last update":"Wed Oct 11 09:50:50 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.226,"15min":1.252},"min":{"1min":0.89800000000000002,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.798,"5min":1.798,"15min":1.8},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.3080000000000001,"15min":1.319},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.63500000000000001},"max":{"1min":1.7490000000000001,"5min":1.9399999999999999,"15min":2.1120000000000001},"last":1.3919999999999999}]},{"osd":7,"last update":"Wed Oct 11 09:50:50 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.276,"15min":1.3049999999999999},"min":{"1min":0.89000000000000001,"5min":0.78100000000000003,"15min":0.60899999999999999},"max":{"1min":1.6020000000000001,"5min":1.794,"15min":1.9470000000000001},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.363,"15min":1.417},"min":{"1min":0.98999999999999999,"5min":0.875,"15min":0.75900000000000001},"max":{"1min":1.6080000000000001,"5min":1.962,"15min":1.9990000000000001},"last":1.425}]}]},{"osd":3,"up_from":13,"seq":55834574985,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1556,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1593344,"data_stored":769757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.76100000000000001,"15min":0.80400000000000005},"min":{"1min":0.53500000000000003,"5min":0.442,"15min":0.33000000000000002},"max":{"1min":1.399,"5min":1.399,"15min":1.462},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.874,"5min":0.81899999999999995,"15min":0.83699999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.3320000000000001,"5min":1.363,"15min":1.4950000000000001},"last":0.502}]},{"osd":1,"last update":"Wed Oct 11 09:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.79200000000000004,"15min":0.80700000000000005},"min":{"1min":0.53400000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.2150000000000001,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.90400000000000003,"15min":0.92200000000000004},"min":{"1min":0.624,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.3049999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":1.3069999999999999}]},{"osd":2,"last update":"Wed Oct 11 09:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.95099999999999996,"15min":0.96599999999999997},"min":{"1min":0.57699999999999996,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.4690000000000001,"5min":1.601,"15min":1.601},"last":1.258},{"interface":"front","average":{"1min":1.028,"5min":1.012,"15min":1.0580000000000001},"min":{"1min":0.80700000000000005,"5min":0.58499999999999996,"15min":0.432},"max":{"1min":1.3009999999999999,"5min":1.607,"15min":2.5870000000000002},"last":1.0089999999999999}]},{"osd":4,"last update":"Wed Oct 11 09:50:44 2023","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.115,"15min":1.1639999999999999},"min":{"1min":0.84799999999999998,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.5549999999999999,"5min":1.7210000000000001,"15min":2.395},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.173,"15min":1.2430000000000001},"min":{"1min":0.86499999999999999,"5min":0.78100000000000003,"15min":0.73599999999999999},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":3.1579999999999999},"last":1.1919999999999999}]},{"osd":5,"last update":"Wed Oct 11 09:50:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.23,"15min":1.28},"min":{"1min":0.95699999999999996,"5min":0.84199999999999997,"15min":0.76500000000000001},"max":{"1min":1.724,"5min":1.724,"15min":3.0009999999999999},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.252,"5min":1.236,"15min":1.3380000000000001},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.5800000000000001,"5min":1.5840000000000001,"15min":3.8050000000000002},"last":1.4279999999999999}]},{"osd":6,"last update":"Wed Oct 11 09:50:44 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.226,"15min":1.258},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":2.1949999999999998},"last":1.218},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2709999999999999,"15min":1.339},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.84499999999999997},"max":{"1min":1.782,"5min":1.782,"15min":3.2549999999999999},"last":1.343}]},{"osd":7,"last update":"Wed Oct 11 09:50:44 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.2849999999999999,"15min":1.383},"min":{"1min":0.98399999999999999,"5min":0.95199999999999996,"15min":0.76700000000000002},"max":{"1min":1.669,"5min":1.7150000000000001,"15min":3.7490000000000001},"last":1.5009999999999999},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.272,"15min":1.3320000000000001},"min":{"1min":0.83199999999999996,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.7969999999999999,"5min":1.7969999999999999,"15min":2.625},"last":1.7130000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574987,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1540,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1576960,"data_stored":768159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:50:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.78900000000000003,"15min":0.80400000000000005},"min":{"1min":0.57499999999999996,"5min":0.40600000000000003,"15min":0.38500000000000001},"max":{"1min":1.26,"5min":1.5920000000000001,"15min":2.0600000000000001},"last":0.876},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.83199999999999996,"15min":0.89400000000000002},"min":{"1min":0.57199999999999995,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.3300000000000001,"5min":1.5329999999999999,"15min":2.8919999999999999},"last":1.6220000000000001}]},{"osd":1,"last update":"Wed Oct 11 09:50:46 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.84599999999999997,"15min":0.94699999999999995},"min":{"1min":0.41999999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.4810000000000001,"5min":1.8069999999999999,"15min":3.7650000000000001},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.96399999999999997,"15min":0.97199999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.40300000000000002},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":2.5339999999999998},"last":1.0940000000000001}]},{"osd":3,"last update":"Wed Oct 11 09:50:46 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.91500000000000004,"15min":1.018},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.2769999999999999,"5min":1.591,"15min":2.403},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0229999999999999,"15min":1.0449999999999999},"min":{"1min":0.68200000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.754,"5min":1.754,"15min":2.7589999999999999},"last":1.1870000000000001}]},{"osd":4,"last update":"Wed Oct 11 09:50:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.141,"15min":1.1819999999999999},"min":{"1min":0.71699999999999997,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.629,"5min":1.7809999999999999,"15min":2.2549999999999999},"last":1.8759999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.2210000000000001,"15min":1.3300000000000001},"min":{"1min":0.751,"5min":0.751,"15min":0.73399999999999999},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":2.6859999999999999},"last":1.9059999999999999}]},{"osd":5,"last update":"Wed Oct 11 09:50:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.2330000000000001,"15min":1.3},"min":{"1min":0.80500000000000005,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.72,"5min":1.76,"15min":3.044},"last":1.5009999999999999},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1890000000000001,"15min":1.2569999999999999},"min":{"1min":0.83099999999999996,"5min":0.72799999999999998,"15min":0.69599999999999995},"max":{"1min":1.4259999999999999,"5min":1.837,"15min":3.5790000000000002},"last":1.7190000000000001}]},{"osd":6,"last update":"Wed Oct 11 09:50:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.2050000000000001,"15min":1.319},"min":{"1min":0.82299999999999995,"5min":0.75,"15min":0.75},"max":{"1min":1.585,"5min":1.6659999999999999,"15min":3.1600000000000001},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.337,"5min":1.3080000000000001,"15min":1.333},"min":{"1min":1.0269999999999999,"5min":0.83499999999999996,"15min":0.623},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":2.839},"last":1.5820000000000001}]},{"osd":7,"last update":"Wed Oct 11 09:50:46 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.2729999999999999,"15min":1.3500000000000001},"min":{"1min":0.86399999999999999,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.663,"5min":1.774,"15min":3.1230000000000002},"last":1.788},{"interface":"front","average":{"1min":1.28,"5min":1.325,"15min":1.431},"min":{"1min":0.874,"5min":0.84899999999999998,"15min":0.80000000000000004},"max":{"1min":1.704,"5min":1.7909999999999999,"15min":2.988},"last":1.444}]}]},{"osd":1,"up_from":13,"seq":55834574986,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32592,"kb_used_data":1544,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1581056,"data_stored":768181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.78700000000000003,"15min":0.78300000000000003},"min":{"1min":0.51700000000000002,"5min":0.375,"15min":0.375},"max":{"1min":1.014,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.79700000000000004,"15min":0.81799999999999995},"min":{"1min":0.48099999999999998,"5min":0.40100000000000002,"15min":0.36699999999999999},"max":{"1min":1.1699999999999999,"5min":1.593,"15min":1.593},"last":0.81799999999999995}]},{"osd":2,"last update":"Wed Oct 11 09:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.81299999999999994,"15min":0.83799999999999997},"min":{"1min":0.53600000000000003,"5min":0.437,"15min":0.36199999999999999},"max":{"1min":1.004,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.91400000000000003,"15min":0.95199999999999996},"min":{"1min":0.51700000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.3160000000000001,"5min":1.3380000000000001,"15min":1.6619999999999999},"last":1.4159999999999999}]},{"osd":3,"last update":"Wed Oct 11 09:50:44 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":1.01,"15min":1.0089999999999999},"min":{"1min":0.61599999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.373,"5min":1.714,"15min":1.714},"last":1.2230000000000001},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.94599999999999995,"15min":0.96999999999999997},"min":{"1min":0.59299999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.3240000000000001,"5min":1.611,"15min":1.611},"last":1.1890000000000001}]},{"osd":4,"last update":"Wed Oct 11 09:50:44 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1180000000000001,"15min":1.149},"min":{"1min":0.81699999999999995,"5min":0.77400000000000002,"15min":0.73599999999999999},"max":{"1min":1.46,"5min":1.5980000000000001,"15min":2.016},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.214,"15min":1.266},"min":{"1min":0.91500000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.409,"5min":1.665,"15min":2.323},"last":1.5589999999999999}]},{"osd":5,"last update":"Wed Oct 11 09:50:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.198,"15min":1.2669999999999999},"min":{"1min":0.84099999999999997,"5min":0.79300000000000004,"15min":0.65800000000000003},"max":{"1min":1.335,"5min":1.722,"15min":2.4079999999999999},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.1850000000000001,"15min":1.2290000000000001},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.4419999999999999,"5min":1.677,"15min":2.1349999999999998},"last":1.3149999999999999}]},{"osd":6,"last update":"Wed Oct 11 09:50:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.244,"15min":1.3169999999999999},"min":{"1min":0.90800000000000003,"5min":0.84599999999999997,"15min":0.71499999999999997},"max":{"1min":1.746,"5min":2.4620000000000002,"15min":2.8940000000000001},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.21,"5min":1.2450000000000001,"15min":1.3149999999999999},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.79200000000000004},"max":{"1min":1.615,"5min":1.8129999999999999,"15min":3.3719999999999999},"last":1.159}]},{"osd":7,"last update":"Wed Oct 11 09:50:44 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.3069999999999999,"15min":1.367},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86099999999999999},"max":{"1min":1.5089999999999999,"5min":1.835,"15min":2.194},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.286,"5min":1.3080000000000001,"15min":1.3839999999999999},"min":{"1min":0.96199999999999997,"5min":0.93300000000000005,"15min":0.89600000000000002},"max":{"1min":1.8120000000000001,"5min":1.857,"15min":1.968},"last":1.4470000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574985,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33172,"kb_used_data":2124,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338668,"statfs":{"total":96636764160,"available":96602796032,"internally_reserved":0,"allocated":2174976,"data_stored":1358549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 11 09:51:41 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.81000000000000005,"15min":0.80300000000000005},"min":{"1min":0.60399999999999998,"5min":0.45300000000000001,"15min":0.38500000000000001},"max":{"1min":1.0660000000000001,"5min":1.611,"15min":1.611},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.87,"15min":0.89500000000000002},"min":{"1min":0.56200000000000006,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1220000000000001,"5min":1.5249999999999999,"15min":2.3100000000000001},"last":1.1220000000000001}]},{"osd":2,"last update":"Wed Oct 11 09:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":0.92800000000000005,"15min":0.93799999999999994},"min":{"1min":0.59599999999999997,"5min":0.53100000000000003,"15min":0.50600000000000001},"max":{"1min":1.8919999999999999,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.998,"5min":0.93100000000000005,"15min":0.95599999999999996},"min":{"1min":0.63100000000000001,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.5529999999999999,"5min":1.5820000000000001,"15min":1.9450000000000001},"last":1.0649999999999999}]},{"osd":3,"last update":"Wed Oct 11 09:51:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.79000000000000004,"15min":0.83799999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.45600000000000002},"max":{"1min":1.1759999999999999,"5min":1.22,"15min":2.7639999999999998},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.002,"5min":0.93100000000000005,"15min":0.98799999999999999},"min":{"1min":0.53300000000000003,"5min":0.51400000000000001,"15min":0.48999999999999999},"max":{"1min":1.482,"5min":1.482,"15min":2.1789999999999998},"last":1.018}]},{"osd":4,"last update":"Wed Oct 11 09:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.125,"15min":1.169},"min":{"1min":0.94599999999999995,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.6659999999999999,"5min":1.792,"15min":1.792},"last":1.6659999999999999},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.125,"15min":1.169},"min":{"1min":0.82399999999999995,"5min":0.755,"15min":0.69799999999999995},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":2.6629999999999998},"last":1.365}]},{"osd":5,"last update":"Wed Oct 11 09:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.2210000000000001,"15min":1.292},"min":{"1min":1.1240000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.619,"5min":1.829,"15min":3.0150000000000001},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1599999999999999,"15min":1.256},"min":{"1min":0.81200000000000006,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.399,"5min":1.7070000000000001,"15min":3.601},"last":1.2350000000000001}]},{"osd":6,"last update":"Wed Oct 11 09:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.1399999999999999,"15min":1.1950000000000001},"min":{"1min":0.98499999999999999,"5min":0.79100000000000004,"15min":0.78000000000000003},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.272,"15min":1.3049999999999999},"min":{"1min":1.028,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":1.7430000000000001,"5min":1.7430000000000001,"15min":2.1440000000000001},"last":1.5660000000000001}]},{"osd":7,"last update":"Wed Oct 11 09:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.2410000000000001,"15min":1.2909999999999999},"min":{"1min":0.84899999999999998,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.607,"5min":1.75,"15min":1.8759999999999999},"last":1.607},{"interface":"front","average":{"1min":1.24,"5min":1.2290000000000001,"15min":1.2909999999999999},"min":{"1min":0.95099999999999996,"5min":0.88800000000000001,"15min":0.73899999999999999},"max":{"1min":2.016,"5min":2.016,"15min":2.5419999999999998},"last":1.2889999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-11T09:51:44.991 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-11T09:51:44.992 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-11T09:51:45.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.208+0000 7ff1cf59e700 1 Processor -- start 2023-10-11T09:51:45.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.208+0000 7ff1cf59e700 1 -- start start 2023-10-11T09:51:45.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.212+0000 7ff1cf59e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ff1d00cf8d0 0x7ff1d00b4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:45.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.212+0000 7ff1cf59e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1d00cfca0 0x7ff1d00cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:45.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.212+0000 7ff1cf59e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1d00b4fd0 con 0x7ff1d0160870 2023-10-11T09:51:45.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.212+0000 7ff1cf59e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1d00b5870 con 0x7ff1d00cf8d0 2023-10-11T09:51:45.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.212+0000 7ff1cf59e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d00b52b0 con 0x7ff1d00cfca0 2023-10-11T09:51:45.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.212+0000 7ff1ce59c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ff1d00cf8d0 0x7ff1d00b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:45.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.212+0000 7ff1ce59c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ff1d00cf8d0 0x7ff1d00b4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44006/0 (socket says 172.21.15.78:44006) 2023-10-11T09:51:45.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.212+0000 7ff1ce59c700 1 -- 172.21.15.78:0/1713186318 learned_addr learned my addr 172.21.15.78:0/1713186318 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:45.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.212+0000 7ff1cdd9b700 1 --2- 172.21.15.78:0/1713186318 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1d00cfca0 0x7ff1d00cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:45.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.216+0000 7ff1cd59a700 1 -- 172.21.15.78:0/1713186318 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1912367986 0 0) 0x7ff1d00b4fd0 con 0x7ff1d0160870 2023-10-11T09:51:45.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.216+0000 7ff1ce59c700 1 -- 172.21.15.78:0/1713186318 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1d00cfca0 msgr2=0x7ff1d00cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:45.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.216+0000 7ff1ce59c700 1 --2- 172.21.15.78:0/1713186318 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1d00cfca0 0x7ff1d00cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:45.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.216+0000 7ff1ce59c700 1 -- 172.21.15.78:0/1713186318 >> v1:172.21.15.133:6789/0 conn(0x7ff1d0160870 legacy=0x7ff1d015cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:45.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.216+0000 7ff1ce59c700 1 -- 172.21.15.78:0/1713186318 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d00b5590 con 0x7ff1d00cf8d0 2023-10-11T09:51:45.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.216+0000 7ff1ce59c700 1 --2- 172.21.15.78:0/1713186318 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1d00cf8d0 0x7ff1d00b4940 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0009910 tx=0x7ff1c0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25e670c9e089763 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:45.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.216+0000 7ff1cd59a700 1 -- 172.21.15.78:0/1713186318 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0011830 con 0x7ff1d00cf8d0 2023-10-11T09:51:45.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.216+0000 7ff1cd59a700 1 -- 172.21.15.78:0/1713186318 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c0011990 con 0x7ff1d00cf8d0 2023-10-11T09:51:45.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.216+0000 7ff1cd59a700 1 -- 172.21.15.78:0/1713186318 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0011d40 con 0x7ff1d00cf8d0 2023-10-11T09:51:45.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.216+0000 7ff1cf59e700 1 -- 172.21.15.78:0/1713186318 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1d00cf8d0 msgr2=0x7ff1d00b4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:45.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.216+0000 7ff1cf59e700 1 --2- 172.21.15.78:0/1713186318 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1d00cf8d0 0x7ff1d00b4940 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0009910 tx=0x7ff1c0009c20 comp rx=0 tx=0).stop 2023-10-11T09:51:45.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.216+0000 7ff1cf59e700 1 -- 172.21.15.78:0/1713186318 shutdown_connections 2023-10-11T09:51:45.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.216+0000 7ff1cf59e700 1 --2- 172.21.15.78:0/1713186318 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1d00cfca0 0x7ff1d00cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:45.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.216+0000 7ff1cf59e700 1 --2- 172.21.15.78:0/1713186318 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1d00cf8d0 0x7ff1d00b4940 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:45.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.216+0000 7ff1cf59e700 1 -- 172.21.15.78:0/1713186318 >> 172.21.15.78:0/1713186318 conn(0x7ff1d006b010 msgr2=0x7ff1d006b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:45.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.216+0000 7ff1cf59e700 1 -- 172.21.15.78:0/1713186318 shutdown_connections 2023-10-11T09:51:45.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.216+0000 7ff1cf59e700 1 -- 172.21.15.78:0/1713186318 wait complete. 2023-10-11T09:51:45.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.220+0000 7ff1cf59e700 1 Processor -- start 2023-10-11T09:51:45.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.220+0000 7ff1cf59e700 1 -- start start 2023-10-11T09:51:45.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.224+0000 7ff1cf59e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1d00cf8d0 0x7ff1d0156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:45.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.224+0000 7ff1cf59e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1d00cfca0 0x7ff1d01e8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:45.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.224+0000 7ff1cf59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1d0160870 0x7ff1d01e8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:45.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.224+0000 7ff1cf59e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d01f1950 con 0x7ff1d0160870 2023-10-11T09:51:45.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.224+0000 7ff1cf59e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d015ef60 con 0x7ff1d00cf8d0 2023-10-11T09:51:45.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.224+0000 7ff1cf59e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d015e9a0 con 0x7ff1d00cfca0 2023-10-11T09:51:45.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.224+0000 7ff1ced9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1d0160870 0x7ff1d01e8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:45.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.224+0000 7ff1ced9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1d0160870 0x7ff1d01e8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44012/0 (socket says 172.21.15.78:44012) 2023-10-11T09:51:45.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.224+0000 7ff1ced9d700 1 -- 172.21.15.78:0/1128925046 learned_addr learned my addr 172.21.15.78:0/1128925046 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:45.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.224+0000 7ff1cdd9b700 1 --2- 172.21.15.78:0/1128925046 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1d00cfca0 0x7ff1d01e8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:45.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.224+0000 7ff1ce59c700 1 --2- 172.21.15.78:0/1128925046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1d00cf8d0 0x7ff1d0156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:45.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.224+0000 7ff1ced9d700 1 -- 172.21.15.78:0/1128925046 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1d00cfca0 msgr2=0x7ff1d01e8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:45.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.224+0000 7ff1ced9d700 1 --2- 172.21.15.78:0/1128925046 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1d00cfca0 0x7ff1d01e8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:45.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.224+0000 7ff1ced9d700 1 -- 172.21.15.78:0/1128925046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1d00cf8d0 msgr2=0x7ff1d0156300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:45.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.224+0000 7ff1ced9d700 1 --2- 172.21.15.78:0/1128925046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1d00cf8d0 0x7ff1d0156300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:45.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.224+0000 7ff1ced9d700 1 -- 172.21.15.78:0/1128925046 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c0009570 con 0x7ff1d0160870 2023-10-11T09:51:45.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.224+0000 7ff1ced9d700 1 --2- 172.21.15.78:0/1128925046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1d0160870 0x7ff1d01e8570 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7ff1c800cc10 tx=0x7ff1c800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:45.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.224+0000 7ff1beffd700 1 -- 172.21.15.78:0/1128925046 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c8003e50 con 0x7ff1d0160870 2023-10-11T09:51:45.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.224+0000 7ff1beffd700 1 -- 172.21.15.78:0/1128925046 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c800a650 con 0x7ff1d0160870 2023-10-11T09:51:45.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.224+0000 7ff1cf59e700 1 -- 172.21.15.78:0/1128925046 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1d015ec80 con 0x7ff1d0160870 2023-10-11T09:51:45.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.224+0000 7ff1beffd700 1 -- 172.21.15.78:0/1128925046 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c800aa00 con 0x7ff1d0160870 2023-10-11T09:51:45.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.224+0000 7ff1cf59e700 1 -- 172.21.15.78:0/1128925046 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1d01e5e20 con 0x7ff1d0160870 2023-10-11T09:51:45.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.228+0000 7ff1cf59e700 1 -- 172.21.15.78:0/1128925046 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1d01e59a0 con 0x7ff1d0160870 2023-10-11T09:51:45.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.228+0000 7ff1beffd700 1 -- 172.21.15.78:0/1128925046 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7ff1c800ab60 con 0x7ff1d0160870 2023-10-11T09:51:45.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.232+0000 7ff1beffd700 1 --2- 172.21.15.78:0/1128925046 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff1b0065ed0 0x7ff1b0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:45.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.232+0000 7ff1beffd700 1 -- 172.21.15.78:0/1128925046 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff1c8083250 con 0x7ff1d0160870 2023-10-11T09:51:45.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.232+0000 7ff1ce59c700 1 --2- 172.21.15.78:0/1128925046 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff1b0065ed0 0x7ff1b0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:45.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.232+0000 7ff1ce59c700 1 --2- 172.21.15.78:0/1128925046 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff1b0065ed0 0x7ff1b0068380 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0000c00 tx=0x7ff1c0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:45.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.236+0000 7ff1beffd700 1 -- 172.21.15.78:0/1128925046 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7ff1d01e59a0 con 0x7ff1d0160870 2023-10-11T09:51:45.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.472+0000 7ff1cf59e700 1 -- 172.21.15.78:0/1128925046 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff1d01e59a0 con 0x7ff1d0160870 2023-10-11T09:51:45.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.484+0000 7ff1beffd700 1 -- 172.21.15.78:0/1128925046 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+0 (secure 0 0 0) 0x7ff1d01e59a0 con 0x7ff1d0160870 2023-10-11T09:51:45.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.488+0000 7ff1cf59e700 1 -- 172.21.15.78:0/1128925046 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff1b0065ed0 msgr2=0x7ff1b0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:45.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.488+0000 7ff1cf59e700 1 --2- 172.21.15.78:0/1128925046 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff1b0065ed0 0x7ff1b0068380 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0000c00 tx=0x7ff1c0008030 comp rx=0 tx=0).stop 2023-10-11T09:51:45.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.488+0000 7ff1cf59e700 1 -- 172.21.15.78:0/1128925046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1d0160870 msgr2=0x7ff1d01e8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:45.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.488+0000 7ff1cf59e700 1 --2- 172.21.15.78:0/1128925046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1d0160870 0x7ff1d01e8570 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7ff1c800cc10 tx=0x7ff1c800cf20 comp rx=0 tx=0).stop 2023-10-11T09:51:45.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.488+0000 7ff1cf59e700 1 -- 172.21.15.78:0/1128925046 shutdown_connections 2023-10-11T09:51:45.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.488+0000 7ff1cf59e700 1 --2- 172.21.15.78:0/1128925046 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff1b0065ed0 0x7ff1b0068380 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:45.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.488+0000 7ff1cf59e700 1 --2- 172.21.15.78:0/1128925046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1d0160870 0x7ff1d01e8570 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:45.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.488+0000 7ff1cf59e700 1 --2- 172.21.15.78:0/1128925046 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1d00cfca0 0x7ff1d01e8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:45.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.488+0000 7ff1cf59e700 1 --2- 172.21.15.78:0/1128925046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1d00cf8d0 0x7ff1d0156300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:45.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.488+0000 7ff1cf59e700 1 -- 172.21.15.78:0/1128925046 >> 172.21.15.78:0/1128925046 conn(0x7ff1d006b010 msgr2=0x7ff1d00b72e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:45.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.488+0000 7ff1cf59e700 1 -- 172.21.15.78:0/1128925046 shutdown_connections 2023-10-11T09:51:45.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.488+0000 7ff1cf59e700 1 -- 172.21.15.78:0/1128925046 wait complete. 2023-10-11T09:51:45.497 INFO:teuthology.orchestra.run.smithi078.stderr:updated caps for client.0 2023-10-11T09:51:45.516 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:51:45.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.684+0000 7f43e36cf700 1 Processor -- start 2023-10-11T09:51:45.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.684+0000 7f43e36cf700 1 -- start start 2023-10-11T09:51:45.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.688+0000 7f43e36cf700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43dc071040 0x7f43dc0cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:45.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.688+0000 7f43e36cf700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f43dc0ce330 0x7f43dc06be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:45.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.688+0000 7f43e36cf700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43dc06ffa0 con 0x7f43dc160d40 2023-10-11T09:51:45.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.688+0000 7f43e36cf700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43dc070c20 con 0x7f43dc0ce330 2023-10-11T09:51:45.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.688+0000 7f43e36cf700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc161110 con 0x7f43dc071040 2023-10-11T09:51:45.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.688+0000 7f43e1c6c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f43dc0ce330 0x7f43dc06be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:45.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.688+0000 7f43e0c6a700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43dc071040 0x7f43dc0cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:45.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.688+0000 7f43e1c6c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f43dc0ce330 0x7f43dc06be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44020/0 (socket says 172.21.15.78:44020) 2023-10-11T09:51:45.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.688+0000 7f43e146b700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f43dc160d40 0x7f43dc0cd280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:51186/0 (socket says 172.21.15.78:51186) 2023-10-11T09:51:45.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.688+0000 7f43e1c6c700 1 -- 172.21.15.78:0/2942882679 learned_addr learned my addr 172.21.15.78:0/2942882679 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:45.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.688+0000 7f43e1c6c700 1 -- 172.21.15.78:0/2942882679 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43dc071040 msgr2=0x7f43dc0cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:45.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.688+0000 7f43e1c6c700 1 --2- 172.21.15.78:0/2942882679 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43dc071040 0x7f43dc0cddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:45.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.688+0000 7f43e1c6c700 1 -- 172.21.15.78:0/2942882679 >> v1:172.21.15.133:6789/0 conn(0x7f43dc160d40 legacy=0x7f43dc0cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:45.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.688+0000 7f43e1c6c700 1 -- 172.21.15.78:0/2942882679 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43dc070280 con 0x7f43dc0ce330 2023-10-11T09:51:45.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.688+0000 7f43d3fff700 1 -- 172.21.15.78:0/2942882679 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2111710281 0 0) 0x7f43dc06ffa0 con 0x7f43dc160d40 2023-10-11T09:51:45.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.688+0000 7f43e0c6a700 1 --2- 172.21.15.78:0/2942882679 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43dc071040 0x7f43dc0cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:51:45.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.688+0000 7f43e1c6c700 1 --2- 172.21.15.78:0/2942882679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43dc0ce330 0x7f43dc06be60 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f43d4009e50 tx=0x7f43d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ed2dff491a6b25 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:45.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.688+0000 7f43d3fff700 1 -- 172.21.15.78:0/2942882679 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d400e070 con 0x7f43dc0ce330 2023-10-11T09:51:45.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.692+0000 7f43d3fff700 1 -- 172.21.15.78:0/2942882679 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d400b910 con 0x7f43dc0ce330 2023-10-11T09:51:45.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.692+0000 7f43d3fff700 1 -- 172.21.15.78:0/2942882679 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d400bcc0 con 0x7f43dc0ce330 2023-10-11T09:51:45.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.692+0000 7f43e36cf700 1 -- 172.21.15.78:0/2942882679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43dc0ce330 msgr2=0x7f43dc06be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:45.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.692+0000 7f43e36cf700 1 --2- 172.21.15.78:0/2942882679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43dc0ce330 0x7f43dc06be60 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f43d4009e50 tx=0x7f43d400b330 comp rx=0 tx=0).stop 2023-10-11T09:51:45.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.692+0000 7f43e36cf700 1 -- 172.21.15.78:0/2942882679 shutdown_connections 2023-10-11T09:51:45.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.692+0000 7f43e36cf700 1 --2- 172.21.15.78:0/2942882679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43dc0ce330 0x7f43dc06be60 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:45.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.692+0000 7f43e36cf700 1 --2- 172.21.15.78:0/2942882679 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43dc071040 0x7f43dc0cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:45.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.692+0000 7f43e36cf700 1 -- 172.21.15.78:0/2942882679 >> 172.21.15.78:0/2942882679 conn(0x7f43dc06aca0 msgr2=0x7f43dc0b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:45.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.692+0000 7f43e36cf700 1 -- 172.21.15.78:0/2942882679 shutdown_connections 2023-10-11T09:51:45.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.692+0000 7f43e36cf700 1 -- 172.21.15.78:0/2942882679 wait complete. 2023-10-11T09:51:45.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.696+0000 7f43e36cf700 1 Processor -- start 2023-10-11T09:51:45.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.696+0000 7f43e36cf700 1 -- start start 2023-10-11T09:51:45.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.696+0000 7f43e36cf700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43dc071040 0x7f43dc1f63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:45.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.696+0000 7f43e36cf700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43dc0ce330 0x7f43dc0b1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:45.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.696+0000 7f43e36cf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43dc160d40 0x7f43dc0ca690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:45.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.696+0000 7f43e36cf700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc155b20 con 0x7f43dc160d40 2023-10-11T09:51:45.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.696+0000 7f43e36cf700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc0b2ca0 con 0x7f43dc0ce330 2023-10-11T09:51:45.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.696+0000 7f43e36cf700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc0b26e0 con 0x7f43dc071040 2023-10-11T09:51:45.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.696+0000 7f43e1c6c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43dc160d40 0x7f43dc0ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:45.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.696+0000 7f43e1c6c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43dc160d40 0x7f43dc0ca690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44036/0 (socket says 172.21.15.78:44036) 2023-10-11T09:51:45.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.696+0000 7f43e1c6c700 1 -- 172.21.15.78:0/3417250378 learned_addr learned my addr 172.21.15.78:0/3417250378 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:45.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.696+0000 7f43e146b700 1 --2- 172.21.15.78:0/3417250378 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43dc071040 0x7f43dc1f63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:45.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.696+0000 7f43e0c6a700 1 --2- 172.21.15.78:0/3417250378 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43dc0ce330 0x7f43dc0b1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:45.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.700+0000 7f43e1c6c700 1 -- 172.21.15.78:0/3417250378 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43dc071040 msgr2=0x7f43dc1f63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:45.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.700+0000 7f43e1c6c700 1 --2- 172.21.15.78:0/3417250378 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43dc071040 0x7f43dc1f63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:45.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.700+0000 7f43e1c6c700 1 -- 172.21.15.78:0/3417250378 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43dc0ce330 msgr2=0x7f43dc0b1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:45.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.700+0000 7f43e1c6c700 1 --2- 172.21.15.78:0/3417250378 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43dc0ce330 0x7f43dc0b1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:45.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.700+0000 7f43e1c6c700 1 -- 172.21.15.78:0/3417250378 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43cc0094b0 con 0x7f43dc160d40 2023-10-11T09:51:45.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.700+0000 7f43e146b700 1 --2- 172.21.15.78:0/3417250378 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43dc071040 0x7f43dc1f63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:51:45.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.700+0000 7f43e0c6a700 1 --2- 172.21.15.78:0/3417250378 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43dc0ce330 0x7f43dc0b1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:51:45.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.700+0000 7f43e1c6c700 1 --2- 172.21.15.78:0/3417250378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43dc160d40 0x7f43dc0ca690 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f43dc06fbc0 tx=0x7f43d4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:45.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.700+0000 7f43d1ffb700 1 -- 172.21.15.78:0/3417250378 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d40099b0 con 0x7f43dc160d40 2023-10-11T09:51:45.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.700+0000 7f43d1ffb700 1 -- 172.21.15.78:0/3417250378 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d40092e0 con 0x7f43dc160d40 2023-10-11T09:51:45.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.700+0000 7f43d1ffb700 1 -- 172.21.15.78:0/3417250378 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d401c660 con 0x7f43dc160d40 2023-10-11T09:51:45.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.700+0000 7f43e36cf700 1 -- 172.21.15.78:0/3417250378 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43d4009b30 con 0x7f43dc160d40 2023-10-11T09:51:45.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.700+0000 7f43e36cf700 1 -- 172.21.15.78:0/3417250378 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43dc15d630 con 0x7f43dc160d40 2023-10-11T09:51:45.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.700+0000 7f43e36cf700 1 -- 172.21.15.78:0/3417250378 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43dc072250 con 0x7f43dc160d40 2023-10-11T09:51:45.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.704+0000 7f43d1ffb700 1 -- 172.21.15.78:0/3417250378 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f43d401c7c0 con 0x7f43dc160d40 2023-10-11T09:51:45.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.704+0000 7f43d1ffb700 1 --2- 172.21.15.78:0/3417250378 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f43c0065f00 0x7f43c00683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:45.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.704+0000 7f43d1ffb700 1 -- 172.21.15.78:0/3417250378 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f43d4085540 con 0x7f43dc160d40 2023-10-11T09:51:45.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.704+0000 7f43e146b700 1 --2- 172.21.15.78:0/3417250378 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f43c0065f00 0x7f43c00683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:45.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.704+0000 7f43e146b700 1 --2- 172.21.15.78:0/3417250378 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f43c0065f00 0x7f43c00683b0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f43d8000f60 tx=0x7f43d8007d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:45.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.712+0000 7f43d1ffb700 1 -- 172.21.15.78:0/3417250378 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f43d4052b00 con 0x7f43dc160d40 2023-10-11T09:51:45.724 INFO:tasks.ceph.mds.d.smithi133.stderr:2023-10-11T09:51:45.717+0000 7f2c47266700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-11T09:51:45.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.924+0000 7f43e36cf700 1 -- 172.21.15.78:0/3417250378 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43dc15d490 con 0x7f43dc160d40 2023-10-11T09:51:45.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.924+0000 7f43d1ffb700 1 -- 172.21.15.78:0/3417250378 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4656 (secure 0 0 0) 0x7f43dc15d490 con 0x7f43dc160d40 2023-10-11T09:51:45.931 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:45.932 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":99,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8452,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/2481257990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2481257990},{"type":"v1","addr":"172.21.15.78:6836","nonce":2481257990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":8464,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1035227282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1035227282},{"type":"v1","addr":"172.21.15.133:6835","nonce":1035227282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":8880,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/4189411531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":4189411531},{"type":"v1","addr":"172.21.15.78:6837","nonce":4189411531}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:51:40.211859+0000","modified":"2023-10-11T09:51:43.234832+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8883},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8883":{"gid":8883,"name":"d","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6833/1433079498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1433079498},{"type":"v1","addr":"172.21.15.133:6833","nonce":1433079498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-11T09:51:45.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.928+0000 7f43e36cf700 1 -- 172.21.15.78:0/3417250378 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f43c0065f00 msgr2=0x7f43c00683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:45.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.928+0000 7f43e36cf700 1 --2- 172.21.15.78:0/3417250378 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f43c0065f00 0x7f43c00683b0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f43d8000f60 tx=0x7f43d8007d50 comp rx=0 tx=0).stop 2023-10-11T09:51:45.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.928+0000 7f43e36cf700 1 -- 172.21.15.78:0/3417250378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43dc160d40 msgr2=0x7f43dc0ca690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:45.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.928+0000 7f43e36cf700 1 --2- 172.21.15.78:0/3417250378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43dc160d40 0x7f43dc0ca690 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f43dc06fbc0 tx=0x7f43d4004af0 comp rx=0 tx=0).stop 2023-10-11T09:51:45.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.928+0000 7f43e36cf700 1 -- 172.21.15.78:0/3417250378 shutdown_connections 2023-10-11T09:51:45.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.928+0000 7f43e36cf700 1 --2- 172.21.15.78:0/3417250378 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f43c0065f00 0x7f43c00683b0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:45.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.928+0000 7f43e36cf700 1 --2- 172.21.15.78:0/3417250378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43dc160d40 0x7f43dc0ca690 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:45.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.928+0000 7f43e36cf700 1 --2- 172.21.15.78:0/3417250378 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43dc0ce330 0x7f43dc0b1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:45.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.928+0000 7f43e36cf700 1 --2- 172.21.15.78:0/3417250378 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43dc071040 0x7f43dc1f63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:45.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.928+0000 7f43e36cf700 1 -- 172.21.15.78:0/3417250378 >> 172.21.15.78:0/3417250378 conn(0x7f43dc06aca0 msgr2=0x7f43dc0cd280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:45.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.928+0000 7f43e36cf700 1 -- 172.21.15.78:0/3417250378 shutdown_connections 2023-10-11T09:51:45.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:45.928+0000 7f43e36cf700 1 -- 172.21.15.78:0/3417250378 wait complete. 2023-10-11T09:51:45.937 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 99 2023-10-11T09:51:46.358 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 99, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-11T09:51:40.211859+0000', 'modified': '2023-10-11T09:51:43.234832+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8883}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8883': {'gid': 8883, 'name': 'd', 'rank': 0, 'incarnation': 95, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.133:6833/1433079498', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6832', 'nonce': 1433079498}, {'type': 'v1', 'addr': '172.21.15.133:6833', 'nonce': 1433079498}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-11T09:51:46.359 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-11T09:51:46.359 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-11T09:51:46.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.532+0000 7f083e9b3700 1 Processor -- start 2023-10-11T09:51:46.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.532+0000 7f083e9b3700 1 -- start start 2023-10-11T09:51:46.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.536+0000 7f083e9b3700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0838154e00 0x7f083815b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:46.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.536+0000 7f083e9b3700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08380720f0 con 0x7f0838071040 2023-10-11T09:51:46.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.536+0000 7f083e9b3700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f083815bb90 con 0x7f08380cbbd0 2023-10-11T09:51:46.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.536+0000 7f083e9b3700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08380714e0 con 0x7f0838154e00 2023-10-11T09:51:46.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.536+0000 7f082f7fe700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f0838071040 0x7f08380cb4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:50234/0 (socket says 172.21.15.78:50234) 2023-10-11T09:51:46.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.536+0000 7f082f7fe700 1 -- 172.21.15.78:0/3592571356 learned_addr learned my addr 172.21.15.78:0/3592571356 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:51:46.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.536+0000 7f0837fff700 1 --2- 172.21.15.78:0/3592571356 >> v2:172.21.15.133:3300/0 conn(0x7f0838154e00 0x7f083815b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:46.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.536+0000 7f08377fe700 1 -- 172.21.15.78:0/3592571356 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 740287981 0 0) 0x7f08380720f0 con 0x7f0838071040 2023-10-11T09:51:46.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.536+0000 7f08377fe700 1 -- 172.21.15.78:0/3592571356 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08240035d0 con 0x7f0838071040 2023-10-11T09:51:46.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.536+0000 7f08377fe700 1 -- 172.21.15.78:0/3592571356 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 842104616 0 0) 0x7f08240035d0 con 0x7f0838071040 2023-10-11T09:51:46.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.536+0000 7f08377fe700 1 -- 172.21.15.78:0/3592571356 >> v2:172.21.15.133:3300/0 conn(0x7f0838154e00 msgr2=0x7f083815b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:46.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.536+0000 7f08377fe700 1 --2- 172.21.15.78:0/3592571356 >> v2:172.21.15.133:3300/0 conn(0x7f0838154e00 0x7f083815b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:46.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.536+0000 7f08377fe700 1 -- 172.21.15.78:0/3592571356 >> v1:172.21.15.133:6789/0 conn(0x7f08380cbbd0 legacy=0x7f083806c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:46.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.536+0000 7f08377fe700 1 -- 172.21.15.78:0/3592571356 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08380717c0 con 0x7f0838071040 2023-10-11T09:51:46.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.536+0000 7f08377fe700 1 -- 172.21.15.78:0/3592571356 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f08200019f0 con 0x7f0838071040 2023-10-11T09:51:46.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.536+0000 7f08377fe700 1 -- 172.21.15.78:0/3592571356 >> v1:172.21.15.78:6789/0 conn(0x7f0838071040 legacy=0x7f08380cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:46.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.536+0000 7f08377fe700 1 --2- 172.21.15.78:0/3592571356 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0824003a70 0x7f0824003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:46.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.536+0000 7f08377fe700 1 --2- 172.21.15.78:0/3592571356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0824004870 0x7f0824004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:46.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.536+0000 7f08377fe700 1 --2- 172.21.15.78:0/3592571356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0824005220 0x7f082400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:46.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.536+0000 7f08377fe700 1 -- 172.21.15.78:0/3592571356 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08380714e0 con 0x7f0824005220 2023-10-11T09:51:46.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.536+0000 7f08377fe700 1 -- 172.21.15.78:0/3592571356 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082400c100 con 0x7f0824004870 2023-10-11T09:51:46.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.536+0000 7f08377fe700 1 -- 172.21.15.78:0/3592571356 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f082400c2c0 con 0x7f0824003a70 2023-10-11T09:51:46.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.536+0000 7f083cf50700 1 --2- 172.21.15.78:0/3592571356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0824005220 0x7f082400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:46.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.536+0000 7f082f7fe700 1 --2- 172.21.15.78:0/3592571356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0824004870 0x7f0824004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:46.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.536+0000 7f0837fff700 1 --2- 172.21.15.78:0/3592571356 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0824003a70 0x7f0824003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:46.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.540+0000 7f083cf50700 1 -- 172.21.15.78:0/3592571356 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0824003a70 msgr2=0x7f0824003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:46.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.540+0000 7f083cf50700 1 --2- 172.21.15.78:0/3592571356 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0824003a70 0x7f0824003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:46.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.540+0000 7f083cf50700 1 -- 172.21.15.78:0/3592571356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0824004870 msgr2=0x7f0824004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:46.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.540+0000 7f083cf50700 1 --2- 172.21.15.78:0/3592571356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0824004870 0x7f0824004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:46.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.540+0000 7f083cf50700 1 -- 172.21.15.78:0/3592571356 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08240035d0 con 0x7f0824005220 2023-10-11T09:51:46.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.540+0000 7f0837fff700 1 --2- 172.21.15.78:0/3592571356 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0824003a70 0x7f0824003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:51:46.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.540+0000 7f083cf50700 1 -- 172.21.15.78:0/3592571356 reap_dead start 2023-10-11T09:51:46.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.540+0000 7f083cf50700 1 --2- 172.21.15.78:0/3592571356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0824005220 0x7f082400ba30 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f0830007fd0 tx=0x7f08240058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3061882de4e19c39 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:46.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.540+0000 7f08377fe700 1 -- 172.21.15.78:0/3592571356 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08300052d0 con 0x7f0824005220 2023-10-11T09:51:46.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.540+0000 7f08377fe700 1 -- 172.21.15.78:0/3592571356 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0830005430 con 0x7f0824005220 2023-10-11T09:51:46.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.540+0000 7f08377fe700 1 -- 172.21.15.78:0/3592571356 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08300055a0 con 0x7f0824005220 2023-10-11T09:51:46.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.540+0000 7f083e9b3700 1 -- 172.21.15.78:0/3592571356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0824005220 msgr2=0x7f082400ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:46.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.540+0000 7f083e9b3700 1 --2- 172.21.15.78:0/3592571356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0824005220 0x7f082400ba30 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f0830007fd0 tx=0x7f08240058a0 comp rx=0 tx=0).stop 2023-10-11T09:51:46.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.540+0000 7f083e9b3700 1 -- 172.21.15.78:0/3592571356 shutdown_connections 2023-10-11T09:51:46.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.540+0000 7f083e9b3700 1 --2- 172.21.15.78:0/3592571356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0824005220 0x7f082400ba30 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:46.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.540+0000 7f083e9b3700 1 -- 172.21.15.78:0/3592571356 >> 172.21.15.78:0/3592571356 conn(0x7f083806aca0 msgr2=0x7f08380b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:46.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.540+0000 7f083e9b3700 1 -- 172.21.15.78:0/3592571356 shutdown_connections 2023-10-11T09:51:46.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.540+0000 7f083e9b3700 1 -- 172.21.15.78:0/3592571356 wait complete. 2023-10-11T09:51:46.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.544+0000 7f083e9b3700 1 Processor -- start 2023-10-11T09:51:46.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.544+0000 7f083e9b3700 1 -- start start 2023-10-11T09:51:46.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.544+0000 7f083e9b3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0824005220 0x7f0838150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:46.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.544+0000 7f083e9b3700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08380b3590 0x7f0838153da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:46.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.544+0000 7f083e9b3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0838139ca0 0x7f08380aeea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:46.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.544+0000 7f083e9b3700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083813b430 con 0x7f0838139ca0 2023-10-11T09:51:46.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.544+0000 7f083e9b3700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083813a5a0 con 0x7f0824005220 2023-10-11T09:51:46.555 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.544+0000 7f083e9b3700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0838151420 con 0x7f08380b3590 2023-10-11T09:51:46.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.548+0000 7f083cf50700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0838139ca0 0x7f08380aeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:46.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.548+0000 7f083cf50700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0838139ca0 0x7f08380aeea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44058/0 (socket says 172.21.15.78:44058) 2023-10-11T09:51:46.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.548+0000 7f083cf50700 1 -- 172.21.15.78:0/1182623860 learned_addr learned my addr 172.21.15.78:0/1182623860 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:46.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.548+0000 7f082f7fe700 1 --2- 172.21.15.78:0/1182623860 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08380b3590 0x7f0838153da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:46.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.548+0000 7f0837fff700 1 --2- 172.21.15.78:0/1182623860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0824005220 0x7f0838150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:46.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.548+0000 7f083cf50700 1 -- 172.21.15.78:0/1182623860 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08380b3590 msgr2=0x7f0838153da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:46.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.548+0000 7f083cf50700 1 --2- 172.21.15.78:0/1182623860 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08380b3590 0x7f0838153da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:46.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.548+0000 7f083cf50700 1 -- 172.21.15.78:0/1182623860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0824005220 msgr2=0x7f0838150ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:46.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.548+0000 7f083cf50700 1 --2- 172.21.15.78:0/1182623860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0824005220 0x7f0838150ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:46.557 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.548+0000 7f083cf50700 1 -- 172.21.15.78:0/1182623860 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f083813a7b0 con 0x7f0838139ca0 2023-10-11T09:51:46.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.548+0000 7f083cf50700 1 --2- 172.21.15.78:0/1182623860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0838139ca0 0x7f08380aeea0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f0830015040 tx=0x7f0830001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:46.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.548+0000 7f08357fa700 1 -- 172.21.15.78:0/1182623860 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0830001290 con 0x7f0838139ca0 2023-10-11T09:51:46.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.548+0000 7f08357fa700 1 -- 172.21.15.78:0/1182623860 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0830012420 con 0x7f0838139ca0 2023-10-11T09:51:46.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.548+0000 7f083e9b3700 1 -- 172.21.15.78:0/1182623860 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f083813aa90 con 0x7f0838139ca0 2023-10-11T09:51:46.558 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.548+0000 7f08357fa700 1 -- 172.21.15.78:0/1182623860 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08300127d0 con 0x7f0838139ca0 2023-10-11T09:51:46.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.548+0000 7f083e9b3700 1 -- 172.21.15.78:0/1182623860 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f083813b050 con 0x7f0838139ca0 2023-10-11T09:51:46.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.552+0000 7f08357fa700 1 -- 172.21.15.78:0/1182623860 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f0830019050 con 0x7f0838139ca0 2023-10-11T09:51:46.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.552+0000 7f08357fa700 1 --2- 172.21.15.78:0/1182623860 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0818065f00 0x7f08180683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:46.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.552+0000 7f08357fa700 1 -- 172.21.15.78:0/1182623860 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0830083c20 con 0x7f0838139ca0 2023-10-11T09:51:46.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.552+0000 7f0837fff700 1 --2- 172.21.15.78:0/1182623860 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0818065f00 0x7f08180683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:46.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.552+0000 7f0841c48740 1 Processor -- start 2023-10-11T09:51:46.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.552+0000 7f0841c48740 1 -- start start 2023-10-11T09:51:46.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.552+0000 7f0837fff700 1 --2- 172.21.15.78:0/1182623860 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0818065f00 0x7f08180683b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f0828009aa0 tx=0x7f0828008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:46.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.556+0000 7f0841c48740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2009eb0 0x22a3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:46.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.556+0000 7f0841c48740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2009ae0 0x22b9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:46.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.556+0000 7f0841c48740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2009710 0x227a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:46.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.556+0000 7f0841c48740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x21e94a0 con 0x2009eb0 2023-10-11T09:51:46.563 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.556+0000 7f0841c48740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x21e6c30 con 0x2009710 2023-10-11T09:51:46.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.556+0000 7f0841c48740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x21e2520 con 0x2009ae0 2023-10-11T09:51:46.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.556+0000 7f0837fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2009eb0 0x22a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:46.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.556+0000 7f0837fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2009eb0 0x22a3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44062/0 (socket says 172.21.15.78:44062) 2023-10-11T09:51:46.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.556+0000 7f0837fff700 1 -- 172.21.15.78:0/940210835 learned_addr learned my addr 172.21.15.78:0/940210835 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:46.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.556+0000 7f082f7fe700 1 --2- 172.21.15.78:0/940210835 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2009ae0 0x22b9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:46.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.556+0000 7f083cf50700 1 --2- 172.21.15.78:0/940210835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2009710 0x227a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:46.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.556+0000 7f0837fff700 1 -- 172.21.15.78:0/940210835 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2009ae0 msgr2=0x22b9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:46.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.556+0000 7f0837fff700 1 --2- 172.21.15.78:0/940210835 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2009ae0 0x22b9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:46.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.556+0000 7f0837fff700 1 -- 172.21.15.78:0/940210835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2009710 msgr2=0x227a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:46.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.556+0000 7f0837fff700 1 --2- 172.21.15.78:0/940210835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2009710 0x227a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:46.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.556+0000 7f0837fff700 1 -- 172.21.15.78:0/940210835 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x20559e0 con 0x2009eb0 2023-10-11T09:51:46.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.556+0000 7f082f7fe700 1 --2- 172.21.15.78:0/940210835 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2009ae0 0x22b9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:51:46.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.556+0000 7f0837fff700 1 --2- 172.21.15.78:0/940210835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2009eb0 0x22a3920 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f0828009480 tx=0x7f0828015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9f284a619fefbae server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:46.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.556+0000 7f082dffb700 1 -- 172.21.15.78:0/940210835 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08280158b0 con 0x2009eb0 2023-10-11T09:51:46.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.556+0000 7f082dffb700 1 -- 172.21.15.78:0/940210835 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0828015a10 con 0x2009eb0 2023-10-11T09:51:46.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.556+0000 7f082dffb700 1 -- 172.21.15.78:0/940210835 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0828015d80 con 0x2009eb0 2023-10-11T09:51:46.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.556+0000 7f0841c48740 1 -- 172.21.15.78:0/940210835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2009eb0 msgr2=0x22a3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:46.569 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.556+0000 7f0841c48740 1 --2- 172.21.15.78:0/940210835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2009eb0 0x22a3920 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f0828009480 tx=0x7f0828015000 comp rx=0 tx=0).stop 2023-10-11T09:51:46.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.556+0000 7f0841c48740 1 -- 172.21.15.78:0/940210835 shutdown_connections 2023-10-11T09:51:46.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.556+0000 7f0841c48740 1 --2- 172.21.15.78:0/940210835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2009710 0x227a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:46.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.556+0000 7f0841c48740 1 --2- 172.21.15.78:0/940210835 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2009ae0 0x22b9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:46.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.556+0000 7f0841c48740 1 --2- 172.21.15.78:0/940210835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2009eb0 0x22a3920 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:46.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.556+0000 7f0841c48740 1 -- 172.21.15.78:0/940210835 >> 172.21.15.78:0/940210835 conn(0x2271fa0 msgr2=0x22a3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:46.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.556+0000 7f0841c48740 1 -- 172.21.15.78:0/940210835 shutdown_connections 2023-10-11T09:51:46.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.556+0000 7f0841c48740 1 -- 172.21.15.78:0/940210835 wait complete. 2023-10-11T09:51:46.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.560+0000 7f0841c48740 1 -- start start 2023-10-11T09:51:46.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.560+0000 7f0841c48740 1 Processor -- start 2023-10-11T09:51:46.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.560+0000 7f0841c48740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2009710 0x22b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:46.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.560+0000 7f0841c48740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2009ae0 0x22a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:46.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.560+0000 7f0841c48740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2009eb0 0x22a2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:46.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.560+0000 7f0841c48740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x21a5b00 con 0x2009eb0 2023-10-11T09:51:46.572 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.560+0000 7f0841c48740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x21a57a0 con 0x2009710 2023-10-11T09:51:46.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.560+0000 7f0841c48740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x208eb20 con 0x2009ae0 2023-10-11T09:51:46.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.560+0000 7f0841c48740 10 client.0 _ll_register_callbacks cb 0x21dbe60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:51:46.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.560+0000 7f083cf50700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2009eb0 0x22a2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:46.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.560+0000 7f083cf50700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2009eb0 0x22a2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44076/0 (socket says 172.21.15.78:44076) 2023-10-11T09:51:46.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.560+0000 7f083cf50700 1 -- 172.21.15.78:0/2958812197 learned_addr learned my addr 172.21.15.78:0/2958812197 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:46.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.560+0000 7f0837fff700 1 --2- 172.21.15.78:0/2958812197 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2009710 0x22b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:46.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.560+0000 7f082f7fe700 1 --2- 172.21.15.78:0/2958812197 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2009ae0 0x22a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:46.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.564+0000 7f083cf50700 1 -- 172.21.15.78:0/2958812197 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2009ae0 msgr2=0x22a1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:46.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.564+0000 7f083cf50700 1 --2- 172.21.15.78:0/2958812197 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2009ae0 0x22a1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:46.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.564+0000 7f083cf50700 1 -- 172.21.15.78:0/2958812197 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2009710 msgr2=0x22b5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:46.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.564+0000 7f083cf50700 1 --2- 172.21.15.78:0/2958812197 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2009710 0x22b5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:46.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.564+0000 7f083cf50700 1 -- 172.21.15.78:0/2958812197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0828012920 con 0x2009eb0 2023-10-11T09:51:46.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.564+0000 7f082f7fe700 1 --2- 172.21.15.78:0/2958812197 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2009ae0 0x22a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:51:46.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.564+0000 7f083cf50700 1 --2- 172.21.15.78:0/2958812197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2009eb0 0x22a2480 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f0830054c20 tx=0x7f0830003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c101b7cc2b8fdb6f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:46.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.564+0000 7f0816ffd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:51:46.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.564+0000 7f0816ffd700 1 -- 172.21.15.78:0/2958812197 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0830055620 con 0x2009eb0 2023-10-11T09:51:46.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.564+0000 7f0816ffd700 1 -- 172.21.15.78:0/2958812197 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0830054f20 con 0x2009eb0 2023-10-11T09:51:46.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.564+0000 7f0841c48740 1 -- 172.21.15.78:0/2958812197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x21db5a0 con 0x2009eb0 2023-10-11T09:51:46.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.564+0000 7f0816ffd700 1 -- 172.21.15.78:0/2958812197 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0830058d50 con 0x2009eb0 2023-10-11T09:51:46.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.564+0000 7f0841c48740 10 client.9030 fetch_fsmap learned FSMap version 99 2023-10-11T09:51:46.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.564+0000 7f0841c48740 1 -- 172.21.15.78:0/2958812197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x21a8750 con 0x2009eb0 2023-10-11T09:51:46.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.564+0000 7f0816ffd700 1 -- 172.21.15.78:0/2958812197 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x21db5a0 con 0x2009eb0 2023-10-11T09:51:46.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.564+0000 7f0816ffd700 1 -- 172.21.15.78:0/2958812197 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 99) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f0830054410 con 0x2009eb0 2023-10-11T09:51:46.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.564+0000 7f0841c48740 10 client.9030 fetch_fsmap finished waiting for FSMap version 99 2023-10-11T09:51:46.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.564+0000 7f0841c48740 10 client.9030 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-11T09:51:46.579 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.564+0000 7f0841c48740 20 client.9030 populate_metadata read hostname 'smithi078' 2023-10-11T09:51:46.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.564+0000 7f0841c48740 1 --2- 172.21.15.78:0/2958812197 >> [v2:172.21.15.133:6832/1433079498,v1:172.21.15.133:6833/1433079498] conn(0x2054790 0x22c3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:46.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.564+0000 7f0841c48740 4 client.9030 mds_command: new command op to 8883 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-11T09:51:46.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.564+0000 7f0841c48740 1 -- 172.21.15.78:0/2958812197 --> [v2:172.21.15.133:6832/1433079498,v1:172.21.15.133:6833/1433079498] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x21db2c0 con 0x2054790 2023-10-11T09:51:46.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.564+0000 7f0837fff700 1 --2- 172.21.15.78:0/2958812197 >> [v2:172.21.15.133:6832/1433079498,v1:172.21.15.133:6833/1433079498] conn(0x2054790 0x22c3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:46.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.568+0000 7f0837fff700 1 --2- 172.21.15.78:0/2958812197 >> [v2:172.21.15.133:6832/1433079498,v1:172.21.15.133:6833/1433079498] conn(0x2054790 0x22c3a30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bafcd1e4ec7fa830 server_cookie=264d461b9d370109 in_seq=0 out_seq=0 2023-10-11T09:51:46.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.568+0000 7f0816ffd700 10 client.9030 ms_handle_connect on v2:172.21.15.133:6832/1433079498 2023-10-11T09:51:46.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.568+0000 7f0816ffd700 1 -- 172.21.15.78:0/2958812197 <== mds.0 v2:172.21.15.133:6832/1433079498 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x21db2c0 con 0x2054790 2023-10-11T09:51:46.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.568+0000 7f0816ffd700 10 client.9030 handle_command_reply: tid=0 2023-10-11T09:51:46.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.568+0000 7f0841c48740 1 client.9030 shutdown 2023-10-11T09:51:46.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.568+0000 7f0841c48740 1 -- 172.21.15.78:0/2958812197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2009eb0 msgr2=0x22a2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:46.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.568+0000 7f0841c48740 1 --2- 172.21.15.78:0/2958812197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2009eb0 0x22a2480 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f0830054c20 tx=0x7f0830003f70 comp rx=0 tx=0).stop 2023-10-11T09:51:46.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.568+0000 7f0841c48740 1 -- 172.21.15.78:0/2958812197 shutdown_connections 2023-10-11T09:51:46.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.568+0000 7f0841c48740 1 --2- 172.21.15.78:0/2958812197 >> [v2:172.21.15.133:6832/1433079498,v1:172.21.15.133:6833/1433079498] conn(0x2054790 0x22c3a30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:46.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.568+0000 7f0841c48740 1 --2- 172.21.15.78:0/2958812197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2009eb0 0x22a2480 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:46.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.568+0000 7f0841c48740 1 --2- 172.21.15.78:0/2958812197 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2009ae0 0x22a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:46.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.568+0000 7f0841c48740 1 --2- 172.21.15.78:0/2958812197 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2009710 0x22b5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:46.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.568+0000 7f0816ffd700 0 client.9030 ms_handle_reset on v2:172.21.15.133:6832/1433079498 2023-10-11T09:51:46.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.568+0000 7f0841c48740 1 -- 172.21.15.78:0/2958812197 >> 172.21.15.78:0/2958812197 conn(0x2271fa0 msgr2=0x229db60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:46.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.568+0000 7f0841c48740 1 -- 172.21.15.78:0/2958812197 shutdown_connections 2023-10-11T09:51:46.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.568+0000 7f0841c48740 1 -- 172.21.15.78:0/2958812197 wait complete. 2023-10-11T09:51:46.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.568+0000 7f0841c48740 20 client.9030 trim_cache size 0 max 16384 2023-10-11T09:51:46.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.588+0000 7f0841c48740 1 Processor -- start 2023-10-11T09:51:46.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.588+0000 7f0841c48740 1 -- start start 2023-10-11T09:51:46.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.592+0000 7f0841c48740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2009710 0x22d5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:46.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.592+0000 7f0841c48740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2009ae0 0x227b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:46.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.592+0000 7f0841c48740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2009eb0 0x227bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:46.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.592+0000 7f0841c48740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x22a37b0 con 0x2009ae0 2023-10-11T09:51:46.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.592+0000 7f0841c48740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x218a550 con 0x2009710 2023-10-11T09:51:46.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.592+0000 7f0841c48740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x2185930 con 0x2009eb0 2023-10-11T09:51:46.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.592+0000 7f083cf50700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2009eb0 0x227bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:46.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.592+0000 7f083cf50700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2009eb0 0x227bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:41772/0 (socket says 172.21.15.78:41772) 2023-10-11T09:51:46.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.592+0000 7f083cf50700 1 -- 172.21.15.78:0/484499046 learned_addr learned my addr 172.21.15.78:0/484499046 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:46.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.592+0000 7f082f7fe700 1 --2- 172.21.15.78:0/484499046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2009ae0 0x227b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:46.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.592+0000 7f0837fff700 1 --2- 172.21.15.78:0/484499046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2009710 0x22d5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:46.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.592+0000 7f083cf50700 1 -- 172.21.15.78:0/484499046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2009710 msgr2=0x22d5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:46.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.592+0000 7f083cf50700 1 --2- 172.21.15.78:0/484499046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2009710 0x22d5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:46.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.592+0000 7f083cf50700 1 -- 172.21.15.78:0/484499046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2009ae0 msgr2=0x227b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:46.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.592+0000 7f083cf50700 1 --2- 172.21.15.78:0/484499046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2009ae0 0x227b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:46.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.592+0000 7f083cf50700 1 -- 172.21.15.78:0/484499046 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08380af740 con 0x2009eb0 2023-10-11T09:51:46.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.592+0000 7f083cf50700 1 --2- 172.21.15.78:0/484499046 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2009eb0 0x227bac0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f083005c9d0 tx=0x7f083002b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55084b745303caff server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:46.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.592+0000 7f082e7fc700 1 -- 172.21.15.78:0/484499046 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0830058d50 con 0x2009eb0 2023-10-11T09:51:46.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.592+0000 7f082e7fc700 1 -- 172.21.15.78:0/484499046 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0830051260 con 0x2009eb0 2023-10-11T09:51:46.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.592+0000 7f082e7fc700 1 -- 172.21.15.78:0/484499046 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0830054cf0 con 0x2009eb0 2023-10-11T09:51:46.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.592+0000 7f0841c48740 1 -- 172.21.15.78:0/484499046 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2009eb0 msgr2=0x227bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:46.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.592+0000 7f0841c48740 1 --2- 172.21.15.78:0/484499046 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2009eb0 0x227bac0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f083005c9d0 tx=0x7f083002b000 comp rx=0 tx=0).stop 2023-10-11T09:51:46.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.592+0000 7f0841c48740 1 -- 172.21.15.78:0/484499046 shutdown_connections 2023-10-11T09:51:46.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.592+0000 7f0841c48740 1 --2- 172.21.15.78:0/484499046 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2009eb0 0x227bac0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:46.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.592+0000 7f0841c48740 1 --2- 172.21.15.78:0/484499046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2009ae0 0x227b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:46.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.592+0000 7f0841c48740 1 --2- 172.21.15.78:0/484499046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2009710 0x22d5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:46.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.592+0000 7f0841c48740 1 -- 172.21.15.78:0/484499046 >> 172.21.15.78:0/484499046 conn(0x2271fa0 msgr2=0x22b85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:46.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.592+0000 7f0841c48740 1 -- 172.21.15.78:0/484499046 shutdown_connections 2023-10-11T09:51:46.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.592+0000 7f0841c48740 1 -- 172.21.15.78:0/484499046 wait complete. 2023-10-11T09:51:46.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.596+0000 7f0841c48740 1 -- start start 2023-10-11T09:51:46.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.596+0000 7f0841c48740 1 Processor -- start 2023-10-11T09:51:46.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.596+0000 7f0841c48740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2009710 0x2321060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.596+0000 7f0841c48740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2009ae0 0x22a1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.596+0000 7f0841c48740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2009eb0 0x22a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.596+0000 7f0841c48740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x22d9ed0 con 0x2009eb0 2023-10-11T09:51:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.596+0000 7f0841c48740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x22a1ff0 con 0x2009ae0 2023-10-11T09:51:46.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.596+0000 7f0841c48740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x22b8b80 con 0x2009710 2023-10-11T09:51:46.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.596+0000 7f0841c48740 10 client.0 _ll_register_callbacks cb 0x211be90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:51:46.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.596+0000 7f083cf50700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2009eb0 0x22a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:46.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.596+0000 7f083cf50700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2009eb0 0x22a19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44106/0 (socket says 172.21.15.78:44106) 2023-10-11T09:51:46.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.596+0000 7f083cf50700 1 -- 172.21.15.78:0/4163066967 learned_addr learned my addr 172.21.15.78:0/4163066967 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:46.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.596+0000 7f082f7fe700 1 --2- 172.21.15.78:0/4163066967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2009ae0 0x22a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:46.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.596+0000 7f0837fff700 1 --2- 172.21.15.78:0/4163066967 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2009710 0x2321060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:46.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.596+0000 7f083cf50700 1 -- 172.21.15.78:0/4163066967 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2009710 msgr2=0x2321060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:46.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.596+0000 7f083cf50700 1 --2- 172.21.15.78:0/4163066967 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2009710 0x2321060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:46.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.596+0000 7f083cf50700 1 -- 172.21.15.78:0/4163066967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2009ae0 msgr2=0x22a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:46.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.596+0000 7f083cf50700 1 --2- 172.21.15.78:0/4163066967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2009ae0 0x22a1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:46.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.596+0000 7f083cf50700 1 -- 172.21.15.78:0/4163066967 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0828012920 con 0x2009eb0 2023-10-11T09:51:46.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.596+0000 7f083cf50700 1 --2- 172.21.15.78:0/4163066967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2009eb0 0x22a19c0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f08300050a0 tx=0x7f0830069000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b29c443c181931be server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:46.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.596+0000 7f0816ffd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:51:46.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.596+0000 7f0816ffd700 1 -- 172.21.15.78:0/4163066967 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0830058d50 con 0x2009eb0 2023-10-11T09:51:46.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.596+0000 7f0841c48740 1 -- 172.21.15.78:0/4163066967 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0828015a10 con 0x2009eb0 2023-10-11T09:51:46.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.600+0000 7f0816ffd700 1 -- 172.21.15.78:0/4163066967 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0830055620 con 0x2009eb0 2023-10-11T09:51:46.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.600+0000 7f0816ffd700 1 -- 172.21.15.78:0/4163066967 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0830054cf0 con 0x2009eb0 2023-10-11T09:51:46.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.600+0000 7f0841c48740 10 client.9036 fetch_fsmap learned FSMap version 99 2023-10-11T09:51:46.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.600+0000 7f0841c48740 1 -- 172.21.15.78:0/4163066967 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x7f08380ce8f0 con 0x2009eb0 2023-10-11T09:51:46.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.600+0000 7f0816ffd700 1 -- 172.21.15.78:0/4163066967 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0828015a10 con 0x2009eb0 2023-10-11T09:51:46.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.600+0000 7f0816ffd700 1 -- 172.21.15.78:0/4163066967 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 99) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f0830079030 con 0x2009eb0 2023-10-11T09:51:46.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.600+0000 7f0841c48740 10 client.9036 fetch_fsmap finished waiting for FSMap version 99 2023-10-11T09:51:46.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.600+0000 7f0841c48740 10 client.9036 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-11T09:51:46.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.600+0000 7f0841c48740 20 client.9036 populate_metadata read hostname 'smithi078' 2023-10-11T09:51:46.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.600+0000 7f0841c48740 1 --2- 172.21.15.78:0/4163066967 >> [v2:172.21.15.133:6832/1433079498,v1:172.21.15.133:6833/1433079498] conn(0x227a700 0x227aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:46.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.600+0000 7f0841c48740 4 client.9036 mds_command: new command op to 8883 tid=0[{"prefix": "status"}] 2023-10-11T09:51:46.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.600+0000 7f0841c48740 1 -- 172.21.15.78:0/4163066967 --> [v2:172.21.15.133:6832/1433079498,v1:172.21.15.133:6833/1433079498] -- command(tid 0: {"prefix": "status"}) v1 -- 0x20614e0 con 0x227a700 2023-10-11T09:51:46.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.600+0000 7f0837fff700 1 --2- 172.21.15.78:0/4163066967 >> [v2:172.21.15.133:6832/1433079498,v1:172.21.15.133:6833/1433079498] conn(0x227a700 0x227aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:46.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.600+0000 7f0837fff700 1 --2- 172.21.15.78:0/4163066967 >> [v2:172.21.15.133:6832/1433079498,v1:172.21.15.133:6833/1433079498] conn(0x227a700 0x227aad0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0abb9627183c062 server_cookie=23867b60160608a in_seq=0 out_seq=0 2023-10-11T09:51:46.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.600+0000 7f0816ffd700 10 client.9036 ms_handle_connect on v2:172.21.15.133:6832/1433079498 2023-10-11T09:51:46.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.600+0000 7f0816ffd700 1 -- 172.21.15.78:0/4163066967 <== mds.0 v2:172.21.15.133:6832/1433079498 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x20614e0 con 0x227a700 2023-10-11T09:51:46.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.600+0000 7f0816ffd700 10 client.9036 handle_command_reply: tid=0 2023-10-11T09:51:46.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.600+0000 7f0841c48740 1 client.9036 shutdown 2023-10-11T09:51:46.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.600+0000 7f0841c48740 1 -- 172.21.15.78:0/4163066967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2009eb0 msgr2=0x22a19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:46.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.600+0000 7f0841c48740 1 --2- 172.21.15.78:0/4163066967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2009eb0 0x22a19c0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f08300050a0 tx=0x7f0830069000 comp rx=0 tx=0).stop 2023-10-11T09:51:46.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.600+0000 7f0841c48740 1 -- 172.21.15.78:0/4163066967 shutdown_connections 2023-10-11T09:51:46.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.600+0000 7f0841c48740 1 --2- 172.21.15.78:0/4163066967 >> [v2:172.21.15.133:6832/1433079498,v1:172.21.15.133:6833/1433079498] conn(0x227a700 0x227aad0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:46.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.600+0000 7f0841c48740 1 --2- 172.21.15.78:0/4163066967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x2009eb0 0x22a19c0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:46.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.600+0000 7f0841c48740 1 --2- 172.21.15.78:0/4163066967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x2009ae0 0x22a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:46.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.600+0000 7f0816ffd700 0 client.9036 ms_handle_reset on v2:172.21.15.133:6832/1433079498 2023-10-11T09:51:46.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.600+0000 7f0841c48740 1 --2- 172.21.15.78:0/4163066967 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x2009710 0x2321060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:46.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.600+0000 7f0841c48740 1 -- 172.21.15.78:0/4163066967 >> 172.21.15.78:0/4163066967 conn(0x2271fa0 msgr2=0x22b96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:46.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.600+0000 7f0841c48740 1 -- 172.21.15.78:0/4163066967 shutdown_connections 2023-10-11T09:51:46.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.600+0000 7f0841c48740 1 -- 172.21.15.78:0/4163066967 wait complete. 2023-10-11T09:51:46.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.604+0000 7f0841c48740 20 client.9036 trim_cache size 0 max 16384 2023-10-11T09:51:46.618 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:51:46.618 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "dee99db9-7a72-406f-a1e2-917943f2b13d", 2023-10-11T09:51:46.618 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2023-10-11T09:51:46.619 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 8883, 2023-10-11T09:51:46.619 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2023-10-11T09:51:46.619 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2023-10-11T09:51:46.619 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2023-10-11T09:51:46.619 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 6.3844264910000001, 2023-10-11T09:51:46.620 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 99, 2023-10-11T09:51:46.620 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 127, 2023-10-11T09:51:46.620 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 127, 2023-10-11T09:51:46.620 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 11.577005045 2023-10-11T09:51:46.620 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:51:46.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.604+0000 7f083e9b3700 1 -- 172.21.15.78:0/1182623860 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0818065f00 msgr2=0x7f08180683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:46.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.604+0000 7f083e9b3700 1 --2- 172.21.15.78:0/1182623860 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0818065f00 0x7f08180683b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f0828009aa0 tx=0x7f0828008040 comp rx=0 tx=0).stop 2023-10-11T09:51:46.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.604+0000 7f083e9b3700 1 -- 172.21.15.78:0/1182623860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0838139ca0 msgr2=0x7f08380aeea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:46.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.604+0000 7f083e9b3700 1 --2- 172.21.15.78:0/1182623860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0838139ca0 0x7f08380aeea0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f0830015040 tx=0x7f0830001120 comp rx=0 tx=0).stop 2023-10-11T09:51:46.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.604+0000 7f083e9b3700 1 -- 172.21.15.78:0/1182623860 shutdown_connections 2023-10-11T09:51:46.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.604+0000 7f083e9b3700 1 --2- 172.21.15.78:0/1182623860 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0818065f00 0x7f08180683b0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:46.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.604+0000 7f083e9b3700 1 --2- 172.21.15.78:0/1182623860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0838139ca0 0x7f08380aeea0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:46.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.604+0000 7f083e9b3700 1 --2- 172.21.15.78:0/1182623860 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08380b3590 0x7f0838153da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:46.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.604+0000 7f083e9b3700 1 --2- 172.21.15.78:0/1182623860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0824005220 0x7f0838150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:46.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.604+0000 7f083e9b3700 1 -- 172.21.15.78:0/1182623860 >> 172.21.15.78:0/1182623860 conn(0x7f083806aca0 msgr2=0x7f08380b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:46.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.604+0000 7f083e9b3700 1 -- 172.21.15.78:0/1182623860 shutdown_connections 2023-10-11T09:51:46.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.604+0000 7f083e9b3700 1 -- 172.21.15.78:0/1182623860 wait complete. 2023-10-11T09:51:46.632 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-11T09:51:46.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.796+0000 7fb6e759e700 1 Processor -- start 2023-10-11T09:51:46.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.796+0000 7fb6e759e700 1 -- start start 2023-10-11T09:51:46.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.800+0000 7fb6e759e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb6e813f740 0x7fb6e80ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:46.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.800+0000 7fb6e759e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fb6e806cd00 0x7fb6e806d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:46.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.800+0000 7fb6e759e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6e8070c80 con 0x7fb6e80b06a0 2023-10-11T09:51:46.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.800+0000 7fb6e759e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6e806d6c0 con 0x7fb6e806cd00 2023-10-11T09:51:46.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.800+0000 7fb6e759e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6e80ce4c0 con 0x7fb6e813f740 2023-10-11T09:51:46.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.800+0000 7fb6e659c700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb6e813f740 0x7fb6e80ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:46.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.800+0000 7fb6e5d9b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fb6e806cd00 0x7fb6e806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:46.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.800+0000 7fb6e659c700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb6e813f740 0x7fb6e80ccab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:52978/0 (socket says 172.21.15.78:52978) 2023-10-11T09:51:46.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.800+0000 7fb6e659c700 1 -- 172.21.15.78:0/1183039192 learned_addr learned my addr 172.21.15.78:0/1183039192 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:46.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.800+0000 7fb6e659c700 1 -- 172.21.15.78:0/1183039192 >> v2:172.21.15.78:3300/0 conn(0x7fb6e806cd00 msgr2=0x7fb6e806d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:46.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.800+0000 7fb6e659c700 1 --2- 172.21.15.78:0/1183039192 >> v2:172.21.15.78:3300/0 conn(0x7fb6e806cd00 0x7fb6e806d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:46.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.800+0000 7fb6e659c700 1 -- 172.21.15.78:0/1183039192 >> v1:172.21.15.78:6789/0 conn(0x7fb6e80b06a0 legacy=0x7fb6e80b0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:46.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.800+0000 7fb6e659c700 1 -- 172.21.15.78:0/1183039192 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e80cd420 con 0x7fb6e813f740 2023-10-11T09:51:46.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.800+0000 7fb6e659c700 1 --2- 172.21.15.78:0/1183039192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6e813f740 0x7fb6e80ccab0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb6e806fb70 tx=0x7fb6d800be90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3de44dddb8099ed server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:46.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.804+0000 7fb6e559a700 1 -- 172.21.15.78:0/1183039192 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d800dd30 con 0x7fb6e813f740 2023-10-11T09:51:46.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.804+0000 7fb6e559a700 1 -- 172.21.15.78:0/1183039192 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d800de90 con 0x7fb6e813f740 2023-10-11T09:51:46.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.804+0000 7fb6e559a700 1 -- 172.21.15.78:0/1183039192 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8010690 con 0x7fb6e813f740 2023-10-11T09:51:46.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.804+0000 7fb6e759e700 1 -- 172.21.15.78:0/1183039192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6e813f740 msgr2=0x7fb6e80ccab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:46.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.804+0000 7fb6e759e700 1 --2- 172.21.15.78:0/1183039192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6e813f740 0x7fb6e80ccab0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb6e806fb70 tx=0x7fb6d800be90 comp rx=0 tx=0).stop 2023-10-11T09:51:46.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.804+0000 7fb6e759e700 1 -- 172.21.15.78:0/1183039192 shutdown_connections 2023-10-11T09:51:46.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.804+0000 7fb6e759e700 1 --2- 172.21.15.78:0/1183039192 >> v2:172.21.15.78:3300/0 conn(0x7fb6e806cd00 0x7fb6e806d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:46.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.804+0000 7fb6e759e700 1 --2- 172.21.15.78:0/1183039192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6e813f740 0x7fb6e80ccab0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:46.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.804+0000 7fb6e759e700 1 -- 172.21.15.78:0/1183039192 >> 172.21.15.78:0/1183039192 conn(0x7fb6e80cea60 msgr2=0x7fb6e80cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:46.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.804+0000 7fb6e759e700 1 -- 172.21.15.78:0/1183039192 shutdown_connections 2023-10-11T09:51:46.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.804+0000 7fb6e759e700 1 -- 172.21.15.78:0/1183039192 wait complete. 2023-10-11T09:51:46.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.808+0000 7fb6e759e700 1 Processor -- start 2023-10-11T09:51:46.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.808+0000 7fb6e759e700 1 -- start start 2023-10-11T09:51:46.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.808+0000 7fb6e759e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6e806cd00 0x7fb6e80b3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:46.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.808+0000 7fb6e759e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6e80b06a0 0x7fb6e8157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:46.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.808+0000 7fb6e759e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6e813f740 0x7fb6e80c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:46.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.808+0000 7fb6e759e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e80b3330 con 0x7fb6e80b06a0 2023-10-11T09:51:46.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.808+0000 7fb6e759e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e80b2a60 con 0x7fb6e806cd00 2023-10-11T09:51:46.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.808+0000 7fb6e759e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e81f5b00 con 0x7fb6e813f740 2023-10-11T09:51:46.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.812+0000 7fb6e5d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6e80b06a0 0x7fb6e8157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:46.818 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.812+0000 7fb6e5d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6e80b06a0 0x7fb6e8157cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44126/0 (socket says 172.21.15.78:44126) 2023-10-11T09:51:46.818 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.812+0000 7fb6e5d9b700 1 -- 172.21.15.78:0/736144909 learned_addr learned my addr 172.21.15.78:0/736144909 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:46.819 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.812+0000 7fb6e5d9b700 1 -- 172.21.15.78:0/736144909 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6e813f740 msgr2=0x7fb6e80c7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:46.819 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.812+0000 7fb6e5d9b700 1 --2- 172.21.15.78:0/736144909 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6e813f740 0x7fb6e80c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:46.819 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.812+0000 7fb6e5d9b700 1 -- 172.21.15.78:0/736144909 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6e806cd00 msgr2=0x7fb6e80b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:46.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.812+0000 7fb6e5d9b700 1 --2- 172.21.15.78:0/736144909 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6e806cd00 0x7fb6e80b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:46.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.812+0000 7fb6e5d9b700 1 -- 172.21.15.78:0/736144909 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d800b810 con 0x7fb6e80b06a0 2023-10-11T09:51:46.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.812+0000 7fb6e5d9b700 1 --2- 172.21.15.78:0/736144909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6e80b06a0 0x7fb6e8157cf0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc00e9d0 tx=0x7fb6dc00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:46.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.812+0000 7fb6d6ffd700 1 -- 172.21.15.78:0/736144909 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc009720 con 0x7fb6e80b06a0 2023-10-11T09:51:46.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.812+0000 7fb6d6ffd700 1 -- 172.21.15.78:0/736144909 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6dc00c850 con 0x7fb6e80b06a0 2023-10-11T09:51:46.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.812+0000 7fb6e759e700 1 -- 172.21.15.78:0/736144909 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e80b2c70 con 0x7fb6e80b06a0 2023-10-11T09:51:46.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.812+0000 7fb6d6ffd700 1 -- 172.21.15.78:0/736144909 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc00cbc0 con 0x7fb6e80b06a0 2023-10-11T09:51:46.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.812+0000 7fb6e759e700 1 -- 172.21.15.78:0/736144909 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e8158230 con 0x7fb6e80b06a0 2023-10-11T09:51:46.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.812+0000 7fb6e759e700 1 -- 172.21.15.78:0/736144909 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6e80b35a0 con 0x7fb6e80b06a0 2023-10-11T09:51:46.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.816+0000 7fb6d6ffd700 1 -- 172.21.15.78:0/736144909 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fb6dc00cd20 con 0x7fb6e80b06a0 2023-10-11T09:51:46.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.816+0000 7fb6d6ffd700 1 --2- 172.21.15.78:0/736144909 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb6c8065ed0 0x7fb6c8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:46.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.816+0000 7fb6d6ffd700 1 -- 172.21.15.78:0/736144909 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb6dc0839a0 con 0x7fb6e80b06a0 2023-10-11T09:51:46.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.816+0000 7fb6e659c700 1 --2- 172.21.15.78:0/736144909 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb6c8065ed0 0x7fb6c8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:46.824 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.820+0000 7fb6e659c700 1 --2- 172.21.15.78:0/736144909 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb6c8065ed0 0x7fb6c8068380 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb6d800bed0 tx=0x7fb6d8012000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:46.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:46.824+0000 7fb6d6ffd700 1 -- 172.21.15.78:0/736144909 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fb6dc050f50 con 0x7fb6e80b06a0 2023-10-11T09:51:47.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:47.036+0000 7fb6e759e700 1 -- 172.21.15.78:0/736144909 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb6e8065d40 con 0x7fb6e80b06a0 2023-10-11T09:51:47.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:47.036+0000 7fb6d6ffd700 1 -- 172.21.15.78:0/736144909 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v99) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb6dc017070 con 0x7fb6e80b06a0 2023-10-11T09:51:47.045 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-11T09:51:47.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:47.040+0000 7fb6e759e700 1 -- 172.21.15.78:0/736144909 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb6c8065ed0 msgr2=0x7fb6c8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:47.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:47.040+0000 7fb6e759e700 1 --2- 172.21.15.78:0/736144909 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb6c8065ed0 0x7fb6c8068380 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb6d800bed0 tx=0x7fb6d8012000 comp rx=0 tx=0).stop 2023-10-11T09:51:47.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:47.040+0000 7fb6e759e700 1 -- 172.21.15.78:0/736144909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6e80b06a0 msgr2=0x7fb6e8157cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:47.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:47.040+0000 7fb6e759e700 1 --2- 172.21.15.78:0/736144909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6e80b06a0 0x7fb6e8157cf0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc00e9d0 tx=0x7fb6dc00ed90 comp rx=0 tx=0).stop 2023-10-11T09:51:47.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:47.040+0000 7fb6e759e700 1 -- 172.21.15.78:0/736144909 shutdown_connections 2023-10-11T09:51:47.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:47.040+0000 7fb6e759e700 1 --2- 172.21.15.78:0/736144909 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb6c8065ed0 0x7fb6c8068380 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:47.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:47.040+0000 7fb6e759e700 1 --2- 172.21.15.78:0/736144909 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6e813f740 0x7fb6e80c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:47.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:47.040+0000 7fb6e759e700 1 --2- 172.21.15.78:0/736144909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6e80b06a0 0x7fb6e8157cf0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:47.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:47.040+0000 7fb6e759e700 1 --2- 172.21.15.78:0/736144909 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6e806cd00 0x7fb6e80b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:47.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:47.040+0000 7fb6e759e700 1 -- 172.21.15.78:0/736144909 >> 172.21.15.78:0/736144909 conn(0x7fb6e80cea60 msgr2=0x7fb6e806bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:47.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:47.040+0000 7fb6e759e700 1 -- 172.21.15.78:0/736144909 shutdown_connections 2023-10-11T09:51:47.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:47.040+0000 7fb6e759e700 1 -- 172.21.15.78:0/736144909 wait complete. 2023-10-11T09:51:47.067 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-11T09:51:47.067 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-11T09:51:47.067 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi078.front.sepia.ceph.com 2023-10-11T09:51:47.068 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-11T09:51:47.068 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-11T09:51:47.068 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-11T09:51:47.069 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-11T09:51:47.069 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-11T09:51:47.069 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-11T09:51:47.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:51:47.070 DEBUG:teuthology.orchestra.run.smithi078:> ip addr 2023-10-11T09:51:47.089 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-11T09:51:47.089 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-11T09:51:47.090 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2023-10-11T09:51:47.090 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:51:47.090 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2023-10-11T09:51:47.090 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:51:47.090 INFO:teuthology.orchestra.run.smithi078.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:51:47.091 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:22 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:51:47.091 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f0 2023-10-11T09:51:47.091 INFO:teuthology.orchestra.run.smithi078.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:51:47.091 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:23 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:51:47.091 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f1 2023-10-11T09:51:47.092 INFO:teuthology.orchestra.run.smithi078.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:51:47.092 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dc brd ff:ff:ff:ff:ff:ff 2023-10-11T09:51:47.092 INFO:teuthology.orchestra.run.smithi078.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-11T09:51:47.092 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dd brd ff:ff:ff:ff:ff:ff 2023-10-11T09:51:47.092 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.21.15.78/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-11T09:51:47.093 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft 41757sec preferred_lft 41757sec 2023-10-11T09:51:47.093 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::ec4:7aff:fe8f:d4dd/64 scope link 2023-10-11T09:51:47.093 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:51:47.093 INFO:teuthology.orchestra.run.smithi078.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-11T09:51:47.093 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 02:42:b6:e4:f8:0d brd ff:ff:ff:ff:ff:ff 2023-10-11T09:51:47.094 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-11T09:51:47.094 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:51:47.094 INFO:teuthology.orchestra.run.smithi078.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-11T09:51:47.094 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 7a:89:8a:a9:e1:23 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:51:47.094 INFO:teuthology.orchestra.run.smithi078.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-11T09:51:47.095 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:51:47.095 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::8ca:ebff:fea9:13f1/64 scope link 2023-10-11T09:51:47.095 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:51:47.095 INFO:teuthology.orchestra.run.smithi078.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-11T09:51:47.095 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 7a:89:8a:a9:e1:23 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-11T09:51:47.095 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::7889:8aff:fea9:e123/64 scope link 2023-10-11T09:51:47.096 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:51:47.096 INFO:teuthology.orchestra.run.smithi078.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-11T09:51:47.096 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether ca:ac:e8:27:51:0d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-11T09:51:47.096 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::c8ac:e8ff:fe27:510d/64 scope link 2023-10-11T09:51:47.096 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:51:47.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:51:47.098 DEBUG:teuthology.orchestra.run.smithi078:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-11T09:51:47.176 INFO:teuthology.orchestra.run.smithi078.stdout:1 2023-10-11T09:51:47.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:51:47.180 DEBUG:teuthology.orchestra.run.smithi078:> route 2023-10-11T09:51:47.244 INFO:teuthology.orchestra.run.smithi078.stdout:Kernel IP routing table 2023-10-11T09:51:47.245 INFO:teuthology.orchestra.run.smithi078.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-11T09:51:47.245 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-11T09:51:47.245 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-11T09:51:47.245 INFO:teuthology.orchestra.run.smithi078.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-11T09:51:47.245 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-10-11T09:51:47.246 INFO:teuthology.orchestra.run.smithi078.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-11T09:51:47.246 INFO:teuthology.orchestra.run.smithi078.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-11T09:51:47.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:51:47.247 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec bash -c ' 2023-10-11T09:51:47.247 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2023-10-11T09:51:47.247 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-11T09:51:47.248 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-11T09:51:47.248 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-11T09:51:47.249 DEBUG:teuthology.orchestra.run.smithi078:> ') 2023-10-11T09:51:47.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:51:47.385 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2023-10-11T09:51:47.403 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-11T09:51:47.403 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-11T09:51:47.404 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-11T09:51:47.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:51:47.405 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set brx.0 up 2023-10-11T09:51:47.486 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-11T09:51:47.486 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T09:51:47.487 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-11T09:51:47.551 INFO:teuthology.orchestra.run.smithi078.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-11T09:51:47.551 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T09:51:47.551 DEBUG:teuthology.orchestra.run.smithi078:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-11T09:51:47.612 DEBUG:teuthology.orchestra.run.smithi078:> sudo modprobe fuse 2023-10-11T09:51:47.693 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T09:51:47.694 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T09:51:47.768 INFO:teuthology.orchestra.run.smithi078.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T09:51:47.770 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T09:51:47.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:51:47.771 DEBUG:teuthology.orchestra.run.smithi078:> ls /sys/fs/fuse/connections 2023-10-11T09:51:47.830 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-11T09:51:47.830 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-11T09:51:47.879 DEBUG:teuthology.orchestra.run.smithi078:> sudo modprobe fuse 2023-10-11T09:51:47.912 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T09:51:47.912 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T09:51:47.989 INFO:teuthology.orchestra.run.smithi078.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T09:51:47.992 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T09:51:47.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:51:47.993 DEBUG:teuthology.orchestra.run.smithi078:> ls /sys/fs/fuse/connections 2023-10-11T09:51:48.025 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:2023-10-11T09:51:48.020+0000 7fa220c1f640 -1 init, newargv = 0x556ae91578f0 newargc=15 2023-10-11T09:51:48.025 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:ceph-fuse[59487]: starting ceph client 2023-10-11T09:51:48.037 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:ceph-fuse[59487]: starting fuse 2023-10-11T09:51:48.038 INFO:teuthology.orchestra.run.smithi078.stdout:49 2023-10-11T09:51:48.038 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2023-10-11T09:51:48.039 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:51:48.039 DEBUG:teuthology.orchestra.run.smithi078:> import glob 2023-10-11T09:51:48.040 DEBUG:teuthology.orchestra.run.smithi078:> import re 2023-10-11T09:51:48.040 DEBUG:teuthology.orchestra.run.smithi078:> import os 2023-10-11T09:51:48.040 DEBUG:teuthology.orchestra.run.smithi078:> import subprocess 2023-10-11T09:51:48.041 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:51:48.041 DEBUG:teuthology.orchestra.run.smithi078:> def _find_admin_socket(client_name): 2023-10-11T09:51:48.042 DEBUG:teuthology.orchestra.run.smithi078:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-11T09:51:48.042 DEBUG:teuthology.orchestra.run.smithi078:> files = glob.glob(asok_path) 2023-10-11T09:51:48.042 DEBUG:teuthology.orchestra.run.smithi078:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-11T09:51:48.043 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:51:48.043 DEBUG:teuthology.orchestra.run.smithi078:> # Given a non-glob path, it better be there 2023-10-11T09:51:48.044 DEBUG:teuthology.orchestra.run.smithi078:> if "*" not in asok_path: 2023-10-11T09:51:48.044 DEBUG:teuthology.orchestra.run.smithi078:> assert(len(files) == 1) 2023-10-11T09:51:48.044 DEBUG:teuthology.orchestra.run.smithi078:> return files[0] 2023-10-11T09:51:48.045 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:51:48.045 DEBUG:teuthology.orchestra.run.smithi078:> for f in files: 2023-10-11T09:51:48.045 DEBUG:teuthology.orchestra.run.smithi078:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-11T09:51:48.046 DEBUG:teuthology.orchestra.run.smithi078:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-11T09:51:48.046 DEBUG:teuthology.orchestra.run.smithi078:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-11T09:51:48.047 DEBUG:teuthology.orchestra.run.smithi078:> contents = proc_f.read() 2023-10-11T09:51:48.047 DEBUG:teuthology.orchestra.run.smithi078:> if mountpoint in contents: 2023-10-11T09:51:48.047 DEBUG:teuthology.orchestra.run.smithi078:> return f 2023-10-11T09:51:48.048 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-11T09:51:48.048 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:51:48.048 DEBUG:teuthology.orchestra.run.smithi078:> print(_find_admin_socket("client.0")) 2023-10-11T09:51:48.049 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:51:48.192 INFO:teuthology.orchestra.run.smithi078.stdout:/var/run/ceph/ceph-client.0.59487.asok 2023-10-11T09:51:48.378 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.59487.asok 2023-10-11T09:51:48.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:51:48.380 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.59487.asok status 2023-10-11T09:51:48.579 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:51:48.580 INFO:teuthology.orchestra.run.smithi078.stdout: "metadata": { 2023-10-11T09:51:48.580 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_sha1": "9a774796f0ecdd11a4b771de396e627aa1a187af", 2023-10-11T09:51:48.580 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_version": "ceph version 18.2.0-662-g9a774796 (9a774796f0ecdd11a4b771de396e627aa1a187af) reef (stable)", 2023-10-11T09:51:48.580 INFO:teuthology.orchestra.run.smithi078.stdout: "entity_id": "0", 2023-10-11T09:51:48.580 INFO:teuthology.orchestra.run.smithi078.stdout: "hostname": "smithi078", 2023-10-11T09:51:48.581 INFO:teuthology.orchestra.run.smithi078.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-11T09:51:48.581 INFO:teuthology.orchestra.run.smithi078.stdout: "pid": "59487", 2023-10-11T09:51:48.581 INFO:teuthology.orchestra.run.smithi078.stdout: "root": "/" 2023-10-11T09:51:48.581 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:51:48.581 INFO:teuthology.orchestra.run.smithi078.stdout: "dentry_count": 0, 2023-10-11T09:51:48.582 INFO:teuthology.orchestra.run.smithi078.stdout: "dentry_pinned_count": 0, 2023-10-11T09:51:48.582 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 9051, 2023-10-11T09:51:48.582 INFO:teuthology.orchestra.run.smithi078.stdout: "inst": { 2023-10-11T09:51:48.582 INFO:teuthology.orchestra.run.smithi078.stdout: "name": { 2023-10-11T09:51:48.583 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "client", 2023-10-11T09:51:48.583 INFO:teuthology.orchestra.run.smithi078.stdout: "num": 9051 2023-10-11T09:51:48.583 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:51:48.583 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": { 2023-10-11T09:51:48.583 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2023-10-11T09:51:48.583 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "192.168.0.1:0", 2023-10-11T09:51:48.584 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 1941460496 2023-10-11T09:51:48.584 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:51:48.584 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:51:48.584 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": { 2023-10-11T09:51:48.584 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2023-10-11T09:51:48.585 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "192.168.0.1:0", 2023-10-11T09:51:48.585 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 1941460496 2023-10-11T09:51:48.585 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:51:48.585 INFO:teuthology.orchestra.run.smithi078.stdout: "inst_str": "client.9051 192.168.0.1:0/1941460496", 2023-10-11T09:51:48.585 INFO:teuthology.orchestra.run.smithi078.stdout: "addr_str": "192.168.0.1:0/1941460496", 2023-10-11T09:51:48.585 INFO:teuthology.orchestra.run.smithi078.stdout: "inode_count": 1, 2023-10-11T09:51:48.586 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_epoch": 99, 2023-10-11T09:51:48.586 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_epoch": 127, 2023-10-11T09:51:48.586 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_epoch_barrier": 0, 2023-10-11T09:51:48.586 INFO:teuthology.orchestra.run.smithi078.stdout: "blocklisted": false, 2023-10-11T09:51:48.586 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs" 2023-10-11T09:51:48.587 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:51:48.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:51:48.596 DEBUG:teuthology.orchestra.run.smithi078:> stat --file-system '--printf=%T 2023-10-11T09:51:48.596 DEBUG:teuthology.orchestra.run.smithi078:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-11T09:51:48.618 INFO:teuthology.orchestra.run.smithi078.stdout:fuseblk 2023-10-11T09:51:48.618 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-11T09:51:48.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:51:48.619 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-11T09:51:48.695 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-11T09:51:48.742 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-11T09:51:48.812 INFO:teuthology.orchestra.run.smithi078.stdout:[global] 2023-10-11T09:51:48.813 INFO:teuthology.orchestra.run.smithi078.stdout: chdir = "" 2023-10-11T09:51:48.813 INFO:teuthology.orchestra.run.smithi078.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-11T09:51:48.813 INFO:teuthology.orchestra.run.smithi078.stdout: auth supported = cephx 2023-10-11T09:51:48.814 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.814 INFO:teuthology.orchestra.run.smithi078.stdout: filestore xattr use omap = true 2023-10-11T09:51:48.814 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.815 INFO:teuthology.orchestra.run.smithi078.stdout: mon clock drift allowed = 1.000 2023-10-11T09:51:48.815 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.815 INFO:teuthology.orchestra.run.smithi078.stdout: osd crush chooseleaf type = 0 2023-10-11T09:51:48.816 INFO:teuthology.orchestra.run.smithi078.stdout: auth debug = true 2023-10-11T09:51:48.816 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.816 INFO:teuthology.orchestra.run.smithi078.stdout: ms die on old message = true 2023-10-11T09:51:48.816 INFO:teuthology.orchestra.run.smithi078.stdout: ms die on bug = true 2023-10-11T09:51:48.817 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.817 INFO:teuthology.orchestra.run.smithi078.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-11T09:51:48.817 INFO:teuthology.orchestra.run.smithi078.stdout: mon pg warn max object skew = 0 2023-10-11T09:51:48.818 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.818 INFO:teuthology.orchestra.run.smithi078.stdout: # disable pg_autoscaler by default for new pools 2023-10-11T09:51:48.819 INFO:teuthology.orchestra.run.smithi078.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-11T09:51:48.819 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.820 INFO:teuthology.orchestra.run.smithi078.stdout: osd pool default size = 2 2023-10-11T09:51:48.820 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.820 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd allow primary affinity = true 2023-10-11T09:51:48.821 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd allow pg remap = true 2023-10-11T09:51:48.821 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on legacy crush tunables = false 2023-10-11T09:51:48.821 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on crush straw calc version zero = false 2023-10-11T09:51:48.822 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on no sortbitwise = false 2023-10-11T09:51:48.822 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on osd down out interval zero = false 2023-10-11T09:51:48.822 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on too few osds = false 2023-10-11T09:51:48.823 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-11T09:51:48.823 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-11T09:51:48.823 INFO:teuthology.orchestra.run.smithi078.stdout: mon_allow_pool_size_one = true 2023-10-11T09:51:48.824 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.824 INFO:teuthology.orchestra.run.smithi078.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-11T09:51:48.824 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.825 INFO:teuthology.orchestra.run.smithi078.stdout: osd default data pool replay window = 5 2023-10-11T09:51:48.825 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.825 INFO:teuthology.orchestra.run.smithi078.stdout: mon allow pool delete = true 2023-10-11T09:51:48.825 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.826 INFO:teuthology.orchestra.run.smithi078.stdout: mon cluster log file level = debug 2023-10-11T09:51:48.826 INFO:teuthology.orchestra.run.smithi078.stdout: debug asserts on shutdown = true 2023-10-11T09:51:48.827 INFO:teuthology.orchestra.run.smithi078.stdout: mon health detail to clog = false 2023-10-11T09:51:48.827 INFO:teuthology.orchestra.run.smithi078.stdout: mon host = "172.21.15.78,172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790]" 2023-10-11T09:51:48.827 INFO:teuthology.orchestra.run.smithi078.stdout: fsid = dee99db9-7a72-406f-a1e2-917943f2b13d 2023-10-11T09:51:48.828 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.828 INFO:teuthology.orchestra.run.smithi078.stdout:[osd] 2023-10-11T09:51:48.828 INFO:teuthology.orchestra.run.smithi078.stdout: osd journal size = 100 2023-10-11T09:51:48.829 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.829 INFO:teuthology.orchestra.run.smithi078.stdout: osd scrub load threshold = 5.0 2023-10-11T09:51:48.829 INFO:teuthology.orchestra.run.smithi078.stdout: osd scrub max interval = 600 2023-10-11T09:51:48.830 INFO:teuthology.orchestra.run.smithi078.stdout: osd mclock profile = high_recovery_ops 2023-10-11T09:51:48.830 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.830 INFO:teuthology.orchestra.run.smithi078.stdout: osd recover clone overlap = true 2023-10-11T09:51:48.831 INFO:teuthology.orchestra.run.smithi078.stdout: osd recovery max chunk = 1048576 2023-10-11T09:51:48.831 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.831 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug shutdown = true 2023-10-11T09:51:48.832 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug op order = true 2023-10-11T09:51:48.832 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug verify stray on activate = true 2023-10-11T09:51:48.832 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.833 INFO:teuthology.orchestra.run.smithi078.stdout: osd open classes on start = true 2023-10-11T09:51:48.833 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug pg log writeout = true 2023-10-11T09:51:48.833 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.834 INFO:teuthology.orchestra.run.smithi078.stdout: osd deep scrub update digest min age = 30 2023-10-11T09:51:48.834 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.834 INFO:teuthology.orchestra.run.smithi078.stdout: osd map max advance = 10 2023-10-11T09:51:48.835 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.835 INFO:teuthology.orchestra.run.smithi078.stdout: journal zero on create = true 2023-10-11T09:51:48.835 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.835 INFO:teuthology.orchestra.run.smithi078.stdout: filestore ondisk finisher threads = 3 2023-10-11T09:51:48.836 INFO:teuthology.orchestra.run.smithi078.stdout: filestore apply finisher threads = 3 2023-10-11T09:51:48.836 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.836 INFO:teuthology.orchestra.run.smithi078.stdout: bdev debug aio = true 2023-10-11T09:51:48.836 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug misdirected ops = true 2023-10-11T09:51:48.837 INFO:teuthology.orchestra.run.smithi078.stdout: bdev async discard = True 2023-10-11T09:51:48.837 INFO:teuthology.orchestra.run.smithi078.stdout: bdev enable discard = True 2023-10-11T09:51:48.837 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore allocator = bitmap 2023-10-11T09:51:48.837 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore block size = 96636764160 2023-10-11T09:51:48.838 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore fsck on mount = True 2023-10-11T09:51:48.838 INFO:teuthology.orchestra.run.smithi078.stdout: debug bluefs = 1/20 2023-10-11T09:51:48.838 INFO:teuthology.orchestra.run.smithi078.stdout: debug bluestore = 1/20 2023-10-11T09:51:48.839 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:51:48.839 INFO:teuthology.orchestra.run.smithi078.stdout: debug osd = 20 2023-10-11T09:51:48.839 INFO:teuthology.orchestra.run.smithi078.stdout: debug rocksdb = 4/10 2023-10-11T09:51:48.839 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd backfillfull_ratio = 0.85 2023-10-11T09:51:48.839 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd full ratio = 0.9 2023-10-11T09:51:48.840 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd nearfull ratio = 0.8 2023-10-11T09:51:48.840 INFO:teuthology.orchestra.run.smithi078.stdout: osd failsafe full ratio = 0.95 2023-10-11T09:51:48.840 INFO:teuthology.orchestra.run.smithi078.stdout: osd objectstore = bluestore 2023-10-11T09:51:48.840 INFO:teuthology.orchestra.run.smithi078.stdout: osd op complaint time = 180 2023-10-11T09:51:48.840 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.840 INFO:teuthology.orchestra.run.smithi078.stdout:[mgr] 2023-10-11T09:51:48.841 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:51:48.841 INFO:teuthology.orchestra.run.smithi078.stdout: debug mgr = 20 2023-10-11T09:51:48.841 INFO:teuthology.orchestra.run.smithi078.stdout: debug mon = 20 2023-10-11T09:51:48.841 INFO:teuthology.orchestra.run.smithi078.stdout: debug auth = 20 2023-10-11T09:51:48.842 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:51:48.842 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:51:48.842 INFO:teuthology.orchestra.run.smithi078.stdout: mgr/telemetry/nag = false 2023-10-11T09:51:48.842 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.842 INFO:teuthology.orchestra.run.smithi078.stdout:[mon] 2023-10-11T09:51:48.842 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:51:48.843 INFO:teuthology.orchestra.run.smithi078.stdout: debug mon = 20 2023-10-11T09:51:48.843 INFO:teuthology.orchestra.run.smithi078.stdout: debug paxos = 20 2023-10-11T09:51:48.843 INFO:teuthology.orchestra.run.smithi078.stdout: debug auth = 20 2023-10-11T09:51:48.843 INFO:teuthology.orchestra.run.smithi078.stdout: mon data avail warn = 5 2023-10-11T09:51:48.844 INFO:teuthology.orchestra.run.smithi078.stdout: mon mgr mkfs grace = 240 2023-10-11T09:51:48.844 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:51:48.844 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd reporter subtree level = osd 2023-10-11T09:51:48.844 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd prime pg temp = true 2023-10-11T09:51:48.844 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:51:48.845 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.845 INFO:teuthology.orchestra.run.smithi078.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-11T09:51:48.845 INFO:teuthology.orchestra.run.smithi078.stdout: auth mon ticket ttl = 660 # 11m 2023-10-11T09:51:48.845 INFO:teuthology.orchestra.run.smithi078.stdout: auth service ticket ttl = 240 # 4m 2023-10-11T09:51:48.845 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.846 INFO:teuthology.orchestra.run.smithi078.stdout: # don't complain about insecure global_id in the test suite 2023-10-11T09:51:48.846 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-11T09:51:48.846 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-11T09:51:48.846 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.846 INFO:teuthology.orchestra.run.smithi078.stdout: # 1m isn't quite enough 2023-10-11T09:51:48.846 INFO:teuthology.orchestra.run.smithi078.stdout: mon_down_mkfs_grace = 2m 2023-10-11T09:51:48.847 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.847 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_filestore_osds = false 2023-10-11T09:51:48.847 INFO:teuthology.orchestra.run.smithi078.stdout: mon op complaint time = 120 2023-10-11T09:51:48.847 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.847 INFO:teuthology.orchestra.run.smithi078.stdout:[client] 2023-10-11T09:51:48.848 INFO:teuthology.orchestra.run.smithi078.stdout: rgw cache enabled = true 2023-10-11T09:51:48.848 INFO:teuthology.orchestra.run.smithi078.stdout: rgw enable ops log = true 2023-10-11T09:51:48.848 INFO:teuthology.orchestra.run.smithi078.stdout: rgw enable usage log = true 2023-10-11T09:51:48.848 INFO:teuthology.orchestra.run.smithi078.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-11T09:51:48.848 INFO:teuthology.orchestra.run.smithi078.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-11T09:51:48.848 INFO:teuthology.orchestra.run.smithi078.stdout: client mount timeout = 600 2023-10-11T09:51:48.849 INFO:teuthology.orchestra.run.smithi078.stdout: debug client = 20 2023-10-11T09:51:48.849 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:51:48.849 INFO:teuthology.orchestra.run.smithi078.stdout: pid file = "" 2023-10-11T09:51:48.849 INFO:teuthology.orchestra.run.smithi078.stdout: rados mon op timeout = 900 2023-10-11T09:51:48.849 INFO:teuthology.orchestra.run.smithi078.stdout: rados osd op timeout = 900 2023-10-11T09:51:48.849 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.a] 2023-10-11T09:51:48.850 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.b] 2023-10-11T09:51:48.850 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.c] 2023-10-11T09:51:48.850 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.a] 2023-10-11T09:51:48.850 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.c] 2023-10-11T09:51:48.851 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.b] 2023-10-11T09:51:48.851 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.d] 2023-10-11T09:51:48.851 INFO:teuthology.orchestra.run.smithi078.stdout:[mds] 2023-10-11T09:51:48.851 INFO:teuthology.orchestra.run.smithi078.stdout: debug mds = 20 2023-10-11T09:51:48.851 INFO:teuthology.orchestra.run.smithi078.stdout: debug mds balancer = 20 2023-10-11T09:51:48.851 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:51:48.852 INFO:teuthology.orchestra.run.smithi078.stdout: mds debug frag = True 2023-10-11T09:51:48.852 INFO:teuthology.orchestra.run.smithi078.stdout: mds debug scatterstat = False 2023-10-11T09:51:48.852 INFO:teuthology.orchestra.run.smithi078.stdout: mds op complaint time = 180 2023-10-11T09:51:48.852 INFO:teuthology.orchestra.run.smithi078.stdout: mds verify scatter = False 2023-10-11T09:51:48.852 INFO:teuthology.orchestra.run.smithi078.stdout: osd op complaint time = 180 2023-10-11T09:51:48.852 INFO:teuthology.orchestra.run.smithi078.stdout: rados mon op timeout = 900 2023-10-11T09:51:48.853 INFO:teuthology.orchestra.run.smithi078.stdout: rados osd op timeout = 900 2023-10-11T09:51:48.853 INFO:teuthology.orchestra.run.smithi078.stdout: mds bal merge size = 0 2023-10-11T09:51:48.854 INFO:teuthology.orchestra.run.smithi133.stdout:[global] 2023-10-11T09:51:48.855 INFO:teuthology.orchestra.run.smithi133.stdout: chdir = "" 2023-10-11T09:51:48.855 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-11T09:51:48.855 INFO:teuthology.orchestra.run.smithi133.stdout: auth supported = cephx 2023-10-11T09:51:48.855 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:48.855 INFO:teuthology.orchestra.run.smithi133.stdout: filestore xattr use omap = true 2023-10-11T09:51:48.856 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:48.856 INFO:teuthology.orchestra.run.smithi133.stdout: mon clock drift allowed = 1.000 2023-10-11T09:51:48.856 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:48.856 INFO:teuthology.orchestra.run.smithi133.stdout: osd crush chooseleaf type = 0 2023-10-11T09:51:48.856 INFO:teuthology.orchestra.run.smithi133.stdout: auth debug = true 2023-10-11T09:51:48.857 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:48.857 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on old message = true 2023-10-11T09:51:48.857 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on bug = true 2023-10-11T09:51:48.857 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:48.857 INFO:teuthology.orchestra.run.smithi133.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-11T09:51:48.858 INFO:teuthology.orchestra.run.smithi133.stdout: mon pg warn max object skew = 0 2023-10-11T09:51:48.858 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:48.858 INFO:teuthology.orchestra.run.smithi133.stdout: # disable pg_autoscaler by default for new pools 2023-10-11T09:51:48.858 INFO:teuthology.orchestra.run.smithi133.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-11T09:51:48.858 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:48.859 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default size = 2 2023-10-11T09:51:48.859 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:48.859 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow primary affinity = true 2023-10-11T09:51:48.859 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow pg remap = true 2023-10-11T09:51:48.859 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on legacy crush tunables = false 2023-10-11T09:51:48.860 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on crush straw calc version zero = false 2023-10-11T09:51:48.860 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on no sortbitwise = false 2023-10-11T09:51:48.860 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on osd down out interval zero = false 2023-10-11T09:51:48.860 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on too few osds = false 2023-10-11T09:51:48.860 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-11T09:51:48.860 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-11T09:51:48.861 INFO:teuthology.orchestra.run.smithi133.stdout: mon_allow_pool_size_one = true 2023-10-11T09:51:48.861 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:48.861 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-11T09:51:48.861 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:48.861 INFO:teuthology.orchestra.run.smithi133.stdout: osd default data pool replay window = 5 2023-10-11T09:51:48.862 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:48.862 INFO:teuthology.orchestra.run.smithi133.stdout: mon allow pool delete = true 2023-10-11T09:51:48.862 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:48.862 INFO:teuthology.orchestra.run.smithi133.stdout: mon cluster log file level = debug 2023-10-11T09:51:48.862 INFO:teuthology.orchestra.run.smithi133.stdout: debug asserts on shutdown = true 2023-10-11T09:51:48.863 INFO:teuthology.orchestra.run.smithi133.stdout: mon health detail to clog = false 2023-10-11T09:51:48.863 INFO:teuthology.orchestra.run.smithi133.stdout: mon host = "172.21.15.78,172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790]" 2023-10-11T09:51:48.863 INFO:teuthology.orchestra.run.smithi133.stdout: fsid = dee99db9-7a72-406f-a1e2-917943f2b13d 2023-10-11T09:51:48.863 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:48.863 INFO:teuthology.orchestra.run.smithi133.stdout:[osd] 2023-10-11T09:51:48.864 INFO:teuthology.orchestra.run.smithi133.stdout: osd journal size = 100 2023-10-11T09:51:48.864 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:48.864 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub load threshold = 5.0 2023-10-11T09:51:48.864 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub max interval = 600 2023-10-11T09:51:48.865 INFO:teuthology.orchestra.run.smithi133.stdout: osd mclock profile = high_recovery_ops 2023-10-11T09:51:48.865 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:48.865 INFO:teuthology.orchestra.run.smithi133.stdout: osd recover clone overlap = true 2023-10-11T09:51:48.865 INFO:teuthology.orchestra.run.smithi133.stdout: osd recovery max chunk = 1048576 2023-10-11T09:51:48.865 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:48.866 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug shutdown = true 2023-10-11T09:51:48.866 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug op order = true 2023-10-11T09:51:48.866 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug verify stray on activate = true 2023-10-11T09:51:48.866 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:48.866 INFO:teuthology.orchestra.run.smithi133.stdout: osd open classes on start = true 2023-10-11T09:51:48.867 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug pg log writeout = true 2023-10-11T09:51:48.867 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:48.867 INFO:teuthology.orchestra.run.smithi133.stdout: osd deep scrub update digest min age = 30 2023-10-11T09:51:48.867 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:48.867 INFO:teuthology.orchestra.run.smithi133.stdout: osd map max advance = 10 2023-10-11T09:51:48.868 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:48.868 INFO:teuthology.orchestra.run.smithi133.stdout: journal zero on create = true 2023-10-11T09:51:48.868 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:48.868 INFO:teuthology.orchestra.run.smithi133.stdout: filestore ondisk finisher threads = 3 2023-10-11T09:51:48.868 INFO:teuthology.orchestra.run.smithi133.stdout: filestore apply finisher threads = 3 2023-10-11T09:51:48.868 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:48.869 INFO:teuthology.orchestra.run.smithi133.stdout: bdev debug aio = true 2023-10-11T09:51:48.869 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug misdirected ops = true 2023-10-11T09:51:48.869 INFO:teuthology.orchestra.run.smithi133.stdout: bdev async discard = True 2023-10-11T09:51:48.869 INFO:teuthology.orchestra.run.smithi133.stdout: bdev enable discard = True 2023-10-11T09:51:48.869 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore allocator = bitmap 2023-10-11T09:51:48.870 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore block size = 96636764160 2023-10-11T09:51:48.870 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore fsck on mount = True 2023-10-11T09:51:48.870 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluefs = 1/20 2023-10-11T09:51:48.870 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluestore = 1/20 2023-10-11T09:51:48.870 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:51:48.871 INFO:teuthology.orchestra.run.smithi133.stdout: debug osd = 20 2023-10-11T09:51:48.871 INFO:teuthology.orchestra.run.smithi133.stdout: debug rocksdb = 4/10 2023-10-11T09:51:48.871 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd backfillfull_ratio = 0.85 2023-10-11T09:51:48.871 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd full ratio = 0.9 2023-10-11T09:51:48.871 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd nearfull ratio = 0.8 2023-10-11T09:51:48.872 INFO:teuthology.orchestra.run.smithi133.stdout: osd failsafe full ratio = 0.95 2023-10-11T09:51:48.872 INFO:teuthology.orchestra.run.smithi133.stdout: osd objectstore = bluestore 2023-10-11T09:51:48.872 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2023-10-11T09:51:48.872 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:48.872 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr] 2023-10-11T09:51:48.873 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:51:48.873 INFO:teuthology.orchestra.run.smithi133.stdout: debug mgr = 20 2023-10-11T09:51:48.873 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2023-10-11T09:51:48.873 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2023-10-11T09:51:48.873 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:51:48.874 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:51:48.874 INFO:teuthology.orchestra.run.smithi133.stdout: mgr/telemetry/nag = false 2023-10-11T09:51:48.874 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:48.874 INFO:teuthology.orchestra.run.smithi133.stdout:[mon] 2023-10-11T09:51:48.874 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:51:48.875 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2023-10-11T09:51:48.875 INFO:teuthology.orchestra.run.smithi133.stdout: debug paxos = 20 2023-10-11T09:51:48.875 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2023-10-11T09:51:48.875 INFO:teuthology.orchestra.run.smithi133.stdout: mon data avail warn = 5 2023-10-11T09:51:48.875 INFO:teuthology.orchestra.run.smithi133.stdout: mon mgr mkfs grace = 240 2023-10-11T09:51:48.876 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:51:48.876 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd reporter subtree level = osd 2023-10-11T09:51:48.876 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd prime pg temp = true 2023-10-11T09:51:48.876 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:51:48.877 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:48.877 INFO:teuthology.orchestra.run.smithi133.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-11T09:51:48.877 INFO:teuthology.orchestra.run.smithi133.stdout: auth mon ticket ttl = 660 # 11m 2023-10-11T09:51:48.877 INFO:teuthology.orchestra.run.smithi133.stdout: auth service ticket ttl = 240 # 4m 2023-10-11T09:51:48.877 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:48.878 INFO:teuthology.orchestra.run.smithi133.stdout: # don't complain about insecure global_id in the test suite 2023-10-11T09:51:48.878 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-11T09:51:48.878 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-11T09:51:48.878 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:48.879 INFO:teuthology.orchestra.run.smithi133.stdout: # 1m isn't quite enough 2023-10-11T09:51:48.879 INFO:teuthology.orchestra.run.smithi133.stdout: mon_down_mkfs_grace = 2m 2023-10-11T09:51:48.879 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:48.879 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_filestore_osds = false 2023-10-11T09:51:48.879 INFO:teuthology.orchestra.run.smithi133.stdout: mon op complaint time = 120 2023-10-11T09:51:48.879 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:48.880 INFO:teuthology.orchestra.run.smithi133.stdout:[client] 2023-10-11T09:51:48.880 INFO:teuthology.orchestra.run.smithi133.stdout: rgw cache enabled = true 2023-10-11T09:51:48.880 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable ops log = true 2023-10-11T09:51:48.880 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable usage log = true 2023-10-11T09:51:48.880 INFO:teuthology.orchestra.run.smithi133.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-11T09:51:48.881 INFO:teuthology.orchestra.run.smithi133.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-11T09:51:48.881 INFO:teuthology.orchestra.run.smithi133.stdout: client mount timeout = 600 2023-10-11T09:51:48.881 INFO:teuthology.orchestra.run.smithi133.stdout: debug client = 20 2023-10-11T09:51:48.881 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:51:48.881 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = "" 2023-10-11T09:51:48.881 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2023-10-11T09:51:48.882 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2023-10-11T09:51:48.882 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.a] 2023-10-11T09:51:48.882 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.b] 2023-10-11T09:51:48.882 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.c] 2023-10-11T09:51:48.883 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.a] 2023-10-11T09:51:48.883 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.c] 2023-10-11T09:51:48.883 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.b] 2023-10-11T09:51:48.883 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.d] 2023-10-11T09:51:48.884 INFO:teuthology.orchestra.run.smithi133.stdout:[mds] 2023-10-11T09:51:48.884 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds = 20 2023-10-11T09:51:48.884 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds balancer = 20 2023-10-11T09:51:48.884 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:51:48.884 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug frag = True 2023-10-11T09:51:48.885 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug scatterstat = False 2023-10-11T09:51:48.885 INFO:teuthology.orchestra.run.smithi133.stdout: mds op complaint time = 180 2023-10-11T09:51:48.885 INFO:teuthology.orchestra.run.smithi133.stdout: mds verify scatter = False 2023-10-11T09:51:48.885 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2023-10-11T09:51:48.885 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2023-10-11T09:51:48.886 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2023-10-11T09:51:48.886 INFO:teuthology.orchestra.run.smithi133.stdout: mds bal merge size = 0 2023-10-11T09:51:48.887 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-11T09:51:48.894 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-11T09:51:48.964 INFO:teuthology.orchestra.run.smithi078.stdout:[global] 2023-10-11T09:51:48.965 INFO:teuthology.orchestra.run.smithi078.stdout: chdir = "" 2023-10-11T09:51:48.965 INFO:teuthology.orchestra.run.smithi078.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-11T09:51:48.965 INFO:teuthology.orchestra.run.smithi078.stdout: auth supported = cephx 2023-10-11T09:51:48.966 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.966 INFO:teuthology.orchestra.run.smithi078.stdout: filestore xattr use omap = true 2023-10-11T09:51:48.967 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.967 INFO:teuthology.orchestra.run.smithi078.stdout: mon clock drift allowed = 1.000 2023-10-11T09:51:48.967 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.967 INFO:teuthology.orchestra.run.smithi078.stdout: osd crush chooseleaf type = 0 2023-10-11T09:51:48.968 INFO:teuthology.orchestra.run.smithi078.stdout: auth debug = true 2023-10-11T09:51:48.968 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.968 INFO:teuthology.orchestra.run.smithi078.stdout: ms die on old message = true 2023-10-11T09:51:48.968 INFO:teuthology.orchestra.run.smithi078.stdout: ms die on bug = true 2023-10-11T09:51:48.969 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.969 INFO:teuthology.orchestra.run.smithi078.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-11T09:51:48.969 INFO:teuthology.orchestra.run.smithi078.stdout: mon pg warn max object skew = 0 2023-10-11T09:51:48.969 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.970 INFO:teuthology.orchestra.run.smithi078.stdout: # disable pg_autoscaler by default for new pools 2023-10-11T09:51:48.970 INFO:teuthology.orchestra.run.smithi078.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-11T09:51:48.970 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.971 INFO:teuthology.orchestra.run.smithi078.stdout: osd pool default size = 2 2023-10-11T09:51:48.971 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.971 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd allow primary affinity = true 2023-10-11T09:51:48.971 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd allow pg remap = true 2023-10-11T09:51:48.971 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on legacy crush tunables = false 2023-10-11T09:51:48.972 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on crush straw calc version zero = false 2023-10-11T09:51:48.972 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on no sortbitwise = false 2023-10-11T09:51:48.972 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on osd down out interval zero = false 2023-10-11T09:51:48.972 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on too few osds = false 2023-10-11T09:51:48.972 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-11T09:51:48.972 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-11T09:51:48.973 INFO:teuthology.orchestra.run.smithi078.stdout: mon_allow_pool_size_one = true 2023-10-11T09:51:48.973 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.973 INFO:teuthology.orchestra.run.smithi078.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-11T09:51:48.973 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.973 INFO:teuthology.orchestra.run.smithi078.stdout: osd default data pool replay window = 5 2023-10-11T09:51:48.974 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.974 INFO:teuthology.orchestra.run.smithi078.stdout: mon allow pool delete = true 2023-10-11T09:51:48.974 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.974 INFO:teuthology.orchestra.run.smithi078.stdout: mon cluster log file level = debug 2023-10-11T09:51:48.974 INFO:teuthology.orchestra.run.smithi078.stdout: debug asserts on shutdown = true 2023-10-11T09:51:48.975 INFO:teuthology.orchestra.run.smithi078.stdout: mon health detail to clog = false 2023-10-11T09:51:48.975 INFO:teuthology.orchestra.run.smithi078.stdout: mon host = "172.21.15.78,172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790]" 2023-10-11T09:51:48.975 INFO:teuthology.orchestra.run.smithi078.stdout: fsid = dee99db9-7a72-406f-a1e2-917943f2b13d 2023-10-11T09:51:48.975 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.975 INFO:teuthology.orchestra.run.smithi078.stdout:[osd] 2023-10-11T09:51:48.976 INFO:teuthology.orchestra.run.smithi078.stdout: osd journal size = 100 2023-10-11T09:51:48.976 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.976 INFO:teuthology.orchestra.run.smithi078.stdout: osd scrub load threshold = 5.0 2023-10-11T09:51:48.976 INFO:teuthology.orchestra.run.smithi078.stdout: osd scrub max interval = 600 2023-10-11T09:51:48.976 INFO:teuthology.orchestra.run.smithi078.stdout: osd mclock profile = high_recovery_ops 2023-10-11T09:51:48.977 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.977 INFO:teuthology.orchestra.run.smithi078.stdout: osd recover clone overlap = true 2023-10-11T09:51:48.977 INFO:teuthology.orchestra.run.smithi078.stdout: osd recovery max chunk = 1048576 2023-10-11T09:51:48.977 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.977 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug shutdown = true 2023-10-11T09:51:48.978 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug op order = true 2023-10-11T09:51:48.978 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug verify stray on activate = true 2023-10-11T09:51:48.978 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.978 INFO:teuthology.orchestra.run.smithi078.stdout: osd open classes on start = true 2023-10-11T09:51:48.978 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug pg log writeout = true 2023-10-11T09:51:48.979 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.979 INFO:teuthology.orchestra.run.smithi078.stdout: osd deep scrub update digest min age = 30 2023-10-11T09:51:48.979 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.979 INFO:teuthology.orchestra.run.smithi078.stdout: osd map max advance = 10 2023-10-11T09:51:48.979 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.980 INFO:teuthology.orchestra.run.smithi078.stdout: journal zero on create = true 2023-10-11T09:51:48.980 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.980 INFO:teuthology.orchestra.run.smithi078.stdout: filestore ondisk finisher threads = 3 2023-10-11T09:51:48.980 INFO:teuthology.orchestra.run.smithi078.stdout: filestore apply finisher threads = 3 2023-10-11T09:51:48.980 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.981 INFO:teuthology.orchestra.run.smithi078.stdout: bdev debug aio = true 2023-10-11T09:51:48.981 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug misdirected ops = true 2023-10-11T09:51:48.981 INFO:teuthology.orchestra.run.smithi078.stdout: bdev async discard = True 2023-10-11T09:51:48.981 INFO:teuthology.orchestra.run.smithi078.stdout: bdev enable discard = True 2023-10-11T09:51:48.981 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore allocator = bitmap 2023-10-11T09:51:48.982 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore block size = 96636764160 2023-10-11T09:51:48.982 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore fsck on mount = True 2023-10-11T09:51:48.982 INFO:teuthology.orchestra.run.smithi078.stdout: debug bluefs = 1/20 2023-10-11T09:51:48.982 INFO:teuthology.orchestra.run.smithi078.stdout: debug bluestore = 1/20 2023-10-11T09:51:48.982 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:51:48.983 INFO:teuthology.orchestra.run.smithi078.stdout: debug osd = 20 2023-10-11T09:51:48.983 INFO:teuthology.orchestra.run.smithi078.stdout: debug rocksdb = 4/10 2023-10-11T09:51:48.983 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd backfillfull_ratio = 0.85 2023-10-11T09:51:48.983 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd full ratio = 0.9 2023-10-11T09:51:48.983 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd nearfull ratio = 0.8 2023-10-11T09:51:48.983 INFO:teuthology.orchestra.run.smithi078.stdout: osd failsafe full ratio = 0.95 2023-10-11T09:51:48.984 INFO:teuthology.orchestra.run.smithi078.stdout: osd objectstore = bluestore 2023-10-11T09:51:48.984 INFO:teuthology.orchestra.run.smithi078.stdout: osd op complaint time = 180 2023-10-11T09:51:48.984 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.984 INFO:teuthology.orchestra.run.smithi078.stdout:[mgr] 2023-10-11T09:51:48.985 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:51:48.985 INFO:teuthology.orchestra.run.smithi078.stdout: debug mgr = 20 2023-10-11T09:51:48.985 INFO:teuthology.orchestra.run.smithi078.stdout: debug mon = 20 2023-10-11T09:51:48.985 INFO:teuthology.orchestra.run.smithi078.stdout: debug auth = 20 2023-10-11T09:51:48.985 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:51:48.986 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:51:48.986 INFO:teuthology.orchestra.run.smithi078.stdout: mgr/telemetry/nag = false 2023-10-11T09:51:48.986 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.986 INFO:teuthology.orchestra.run.smithi078.stdout:[mon] 2023-10-11T09:51:48.986 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:51:48.987 INFO:teuthology.orchestra.run.smithi078.stdout: debug mon = 20 2023-10-11T09:51:48.987 INFO:teuthology.orchestra.run.smithi078.stdout: debug paxos = 20 2023-10-11T09:51:48.987 INFO:teuthology.orchestra.run.smithi078.stdout: debug auth = 20 2023-10-11T09:51:48.987 INFO:teuthology.orchestra.run.smithi078.stdout: mon data avail warn = 5 2023-10-11T09:51:48.987 INFO:teuthology.orchestra.run.smithi078.stdout: mon mgr mkfs grace = 240 2023-10-11T09:51:48.987 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:51:48.988 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd reporter subtree level = osd 2023-10-11T09:51:48.988 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd prime pg temp = true 2023-10-11T09:51:48.988 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:51:48.988 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.988 INFO:teuthology.orchestra.run.smithi078.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-11T09:51:48.989 INFO:teuthology.orchestra.run.smithi078.stdout: auth mon ticket ttl = 660 # 11m 2023-10-11T09:51:48.989 INFO:teuthology.orchestra.run.smithi078.stdout: auth service ticket ttl = 240 # 4m 2023-10-11T09:51:48.989 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.989 INFO:teuthology.orchestra.run.smithi078.stdout: # don't complain about insecure global_id in the test suite 2023-10-11T09:51:48.989 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-11T09:51:48.989 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-11T09:51:48.990 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.990 INFO:teuthology.orchestra.run.smithi078.stdout: # 1m isn't quite enough 2023-10-11T09:51:48.990 INFO:teuthology.orchestra.run.smithi078.stdout: mon_down_mkfs_grace = 2m 2023-10-11T09:51:48.990 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.990 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_filestore_osds = false 2023-10-11T09:51:48.991 INFO:teuthology.orchestra.run.smithi078.stdout: mon op complaint time = 120 2023-10-11T09:51:48.991 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:48.991 INFO:teuthology.orchestra.run.smithi078.stdout:[client] 2023-10-11T09:51:48.991 INFO:teuthology.orchestra.run.smithi078.stdout: rgw cache enabled = true 2023-10-11T09:51:48.991 INFO:teuthology.orchestra.run.smithi078.stdout: rgw enable ops log = true 2023-10-11T09:51:48.991 INFO:teuthology.orchestra.run.smithi078.stdout: rgw enable usage log = true 2023-10-11T09:51:48.992 INFO:teuthology.orchestra.run.smithi078.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-11T09:51:48.992 INFO:teuthology.orchestra.run.smithi078.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-11T09:51:48.992 INFO:teuthology.orchestra.run.smithi078.stdout: client mount timeout = 600 2023-10-11T09:51:48.992 INFO:teuthology.orchestra.run.smithi078.stdout: debug client = 20 2023-10-11T09:51:48.993 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:51:48.993 INFO:teuthology.orchestra.run.smithi078.stdout: pid file = "" 2023-10-11T09:51:48.993 INFO:teuthology.orchestra.run.smithi078.stdout: rados mon op timeout = 900 2023-10-11T09:51:48.993 INFO:teuthology.orchestra.run.smithi078.stdout: rados osd op timeout = 900 2023-10-11T09:51:48.993 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.a] 2023-10-11T09:51:48.994 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.b] 2023-10-11T09:51:48.994 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.c] 2023-10-11T09:51:48.994 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.a] 2023-10-11T09:51:48.994 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.c] 2023-10-11T09:51:48.994 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.b] 2023-10-11T09:51:48.995 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.d] 2023-10-11T09:51:48.995 INFO:teuthology.orchestra.run.smithi078.stdout:[mds] 2023-10-11T09:51:48.995 INFO:teuthology.orchestra.run.smithi078.stdout: debug mds = 20 2023-10-11T09:51:48.995 INFO:teuthology.orchestra.run.smithi078.stdout: debug mds balancer = 20 2023-10-11T09:51:48.996 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-10-11T09:51:48.996 INFO:teuthology.orchestra.run.smithi078.stdout: mds debug frag = True 2023-10-11T09:51:48.996 INFO:teuthology.orchestra.run.smithi078.stdout: mds debug scatterstat = False 2023-10-11T09:51:48.996 INFO:teuthology.orchestra.run.smithi078.stdout: mds op complaint time = 180 2023-10-11T09:51:48.996 INFO:teuthology.orchestra.run.smithi078.stdout: mds verify scatter = False 2023-10-11T09:51:48.996 INFO:teuthology.orchestra.run.smithi078.stdout: osd op complaint time = 180 2023-10-11T09:51:48.997 INFO:teuthology.orchestra.run.smithi078.stdout: rados mon op timeout = 900 2023-10-11T09:51:48.997 INFO:teuthology.orchestra.run.smithi078.stdout: rados osd op timeout = 900 2023-10-11T09:51:48.997 INFO:teuthology.orchestra.run.smithi078.stdout: mds bal merge size = 0 2023-10-11T09:51:48.997 INFO:teuthology.orchestra.run.smithi078.stdout: mds bal split size = 10000 2023-10-11T09:51:48.999 INFO:teuthology.orchestra.run.smithi133.stdout:[global] 2023-10-11T09:51:48.999 INFO:teuthology.orchestra.run.smithi133.stdout: chdir = "" 2023-10-11T09:51:48.999 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-11T09:51:49.000 INFO:teuthology.orchestra.run.smithi133.stdout: auth supported = cephx 2023-10-11T09:51:49.000 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:49.000 INFO:teuthology.orchestra.run.smithi133.stdout: filestore xattr use omap = true 2023-10-11T09:51:49.000 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:49.000 INFO:teuthology.orchestra.run.smithi133.stdout: mon clock drift allowed = 1.000 2023-10-11T09:51:49.001 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:49.001 INFO:teuthology.orchestra.run.smithi133.stdout: osd crush chooseleaf type = 0 2023-10-11T09:51:49.001 INFO:teuthology.orchestra.run.smithi133.stdout: auth debug = true 2023-10-11T09:51:49.001 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:49.002 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on old message = true 2023-10-11T09:51:49.002 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on bug = true 2023-10-11T09:51:49.002 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:49.002 INFO:teuthology.orchestra.run.smithi133.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-11T09:51:49.002 INFO:teuthology.orchestra.run.smithi133.stdout: mon pg warn max object skew = 0 2023-10-11T09:51:49.002 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:49.003 INFO:teuthology.orchestra.run.smithi133.stdout: # disable pg_autoscaler by default for new pools 2023-10-11T09:51:49.003 INFO:teuthology.orchestra.run.smithi133.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-11T09:51:49.003 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:49.003 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default size = 2 2023-10-11T09:51:49.003 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:49.003 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow primary affinity = true 2023-10-11T09:51:49.004 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow pg remap = true 2023-10-11T09:51:49.004 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on legacy crush tunables = false 2023-10-11T09:51:49.004 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on crush straw calc version zero = false 2023-10-11T09:51:49.004 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on no sortbitwise = false 2023-10-11T09:51:49.004 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on osd down out interval zero = false 2023-10-11T09:51:49.005 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on too few osds = false 2023-10-11T09:51:49.005 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-11T09:51:49.005 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-11T09:51:49.005 INFO:teuthology.orchestra.run.smithi133.stdout: mon_allow_pool_size_one = true 2023-10-11T09:51:49.005 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:49.006 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-11T09:51:49.006 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:49.006 INFO:teuthology.orchestra.run.smithi133.stdout: osd default data pool replay window = 5 2023-10-11T09:51:49.006 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:49.006 INFO:teuthology.orchestra.run.smithi133.stdout: mon allow pool delete = true 2023-10-11T09:51:49.007 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:49.007 INFO:teuthology.orchestra.run.smithi133.stdout: mon cluster log file level = debug 2023-10-11T09:51:49.007 INFO:teuthology.orchestra.run.smithi133.stdout: debug asserts on shutdown = true 2023-10-11T09:51:49.007 INFO:teuthology.orchestra.run.smithi133.stdout: mon health detail to clog = false 2023-10-11T09:51:49.007 INFO:teuthology.orchestra.run.smithi133.stdout: mon host = "172.21.15.78,172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790]" 2023-10-11T09:51:49.007 INFO:teuthology.orchestra.run.smithi133.stdout: fsid = dee99db9-7a72-406f-a1e2-917943f2b13d 2023-10-11T09:51:49.008 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:49.008 INFO:teuthology.orchestra.run.smithi133.stdout:[osd] 2023-10-11T09:51:49.008 INFO:teuthology.orchestra.run.smithi133.stdout: osd journal size = 100 2023-10-11T09:51:49.008 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:49.008 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub load threshold = 5.0 2023-10-11T09:51:49.008 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub max interval = 600 2023-10-11T09:51:49.009 INFO:teuthology.orchestra.run.smithi133.stdout: osd mclock profile = high_recovery_ops 2023-10-11T09:51:49.009 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:49.009 INFO:teuthology.orchestra.run.smithi133.stdout: osd recover clone overlap = true 2023-10-11T09:51:49.009 INFO:teuthology.orchestra.run.smithi133.stdout: osd recovery max chunk = 1048576 2023-10-11T09:51:49.009 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:49.010 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug shutdown = true 2023-10-11T09:51:49.010 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug op order = true 2023-10-11T09:51:49.010 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug verify stray on activate = true 2023-10-11T09:51:49.010 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:49.010 INFO:teuthology.orchestra.run.smithi133.stdout: osd open classes on start = true 2023-10-11T09:51:49.011 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug pg log writeout = true 2023-10-11T09:51:49.011 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:49.011 INFO:teuthology.orchestra.run.smithi133.stdout: osd deep scrub update digest min age = 30 2023-10-11T09:51:49.011 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:49.011 INFO:teuthology.orchestra.run.smithi133.stdout: osd map max advance = 10 2023-10-11T09:51:49.012 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:49.012 INFO:teuthology.orchestra.run.smithi133.stdout: journal zero on create = true 2023-10-11T09:51:49.012 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:49.012 INFO:teuthology.orchestra.run.smithi133.stdout: filestore ondisk finisher threads = 3 2023-10-11T09:51:49.012 INFO:teuthology.orchestra.run.smithi133.stdout: filestore apply finisher threads = 3 2023-10-11T09:51:49.012 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:49.013 INFO:teuthology.orchestra.run.smithi133.stdout: bdev debug aio = true 2023-10-11T09:51:49.013 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug misdirected ops = true 2023-10-11T09:51:49.013 INFO:teuthology.orchestra.run.smithi133.stdout: bdev async discard = True 2023-10-11T09:51:49.013 INFO:teuthology.orchestra.run.smithi133.stdout: bdev enable discard = True 2023-10-11T09:51:49.013 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore allocator = bitmap 2023-10-11T09:51:49.014 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore block size = 96636764160 2023-10-11T09:51:49.014 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore fsck on mount = True 2023-10-11T09:51:49.014 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluefs = 1/20 2023-10-11T09:51:49.014 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluestore = 1/20 2023-10-11T09:51:49.014 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:51:49.015 INFO:teuthology.orchestra.run.smithi133.stdout: debug osd = 20 2023-10-11T09:51:49.015 INFO:teuthology.orchestra.run.smithi133.stdout: debug rocksdb = 4/10 2023-10-11T09:51:49.015 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd backfillfull_ratio = 0.85 2023-10-11T09:51:49.015 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd full ratio = 0.9 2023-10-11T09:51:49.015 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd nearfull ratio = 0.8 2023-10-11T09:51:49.016 INFO:teuthology.orchestra.run.smithi133.stdout: osd failsafe full ratio = 0.95 2023-10-11T09:51:49.016 INFO:teuthology.orchestra.run.smithi133.stdout: osd objectstore = bluestore 2023-10-11T09:51:49.016 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2023-10-11T09:51:49.016 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:49.016 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr] 2023-10-11T09:51:49.016 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:51:49.017 INFO:teuthology.orchestra.run.smithi133.stdout: debug mgr = 20 2023-10-11T09:51:49.017 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2023-10-11T09:51:49.017 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2023-10-11T09:51:49.017 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:51:49.018 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:51:49.018 INFO:teuthology.orchestra.run.smithi133.stdout: mgr/telemetry/nag = false 2023-10-11T09:51:49.018 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:49.018 INFO:teuthology.orchestra.run.smithi133.stdout:[mon] 2023-10-11T09:51:49.018 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:51:49.019 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2023-10-11T09:51:49.019 INFO:teuthology.orchestra.run.smithi133.stdout: debug paxos = 20 2023-10-11T09:51:49.019 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2023-10-11T09:51:49.019 INFO:teuthology.orchestra.run.smithi133.stdout: mon data avail warn = 5 2023-10-11T09:51:49.019 INFO:teuthology.orchestra.run.smithi133.stdout: mon mgr mkfs grace = 240 2023-10-11T09:51:49.020 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2023-10-11T09:51:49.020 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd reporter subtree level = osd 2023-10-11T09:51:49.020 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd prime pg temp = true 2023-10-11T09:51:49.020 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2023-10-11T09:51:49.020 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:49.020 INFO:teuthology.orchestra.run.smithi133.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-11T09:51:49.021 INFO:teuthology.orchestra.run.smithi133.stdout: auth mon ticket ttl = 660 # 11m 2023-10-11T09:51:49.021 INFO:teuthology.orchestra.run.smithi133.stdout: auth service ticket ttl = 240 # 4m 2023-10-11T09:51:49.021 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:49.021 INFO:teuthology.orchestra.run.smithi133.stdout: # don't complain about insecure global_id in the test suite 2023-10-11T09:51:49.021 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-11T09:51:49.022 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-11T09:51:49.022 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:49.022 INFO:teuthology.orchestra.run.smithi133.stdout: # 1m isn't quite enough 2023-10-11T09:51:49.022 INFO:teuthology.orchestra.run.smithi133.stdout: mon_down_mkfs_grace = 2m 2023-10-11T09:51:49.023 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:49.023 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_filestore_osds = false 2023-10-11T09:51:49.023 INFO:teuthology.orchestra.run.smithi133.stdout: mon op complaint time = 120 2023-10-11T09:51:49.023 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-10-11T09:51:49.023 INFO:teuthology.orchestra.run.smithi133.stdout:[client] 2023-10-11T09:51:49.023 INFO:teuthology.orchestra.run.smithi133.stdout: rgw cache enabled = true 2023-10-11T09:51:49.024 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable ops log = true 2023-10-11T09:51:49.024 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable usage log = true 2023-10-11T09:51:49.024 INFO:teuthology.orchestra.run.smithi133.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-11T09:51:49.024 INFO:teuthology.orchestra.run.smithi133.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-11T09:51:49.024 INFO:teuthology.orchestra.run.smithi133.stdout: client mount timeout = 600 2023-10-11T09:51:49.024 INFO:teuthology.orchestra.run.smithi133.stdout: debug client = 20 2023-10-11T09:51:49.025 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:51:49.025 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = "" 2023-10-11T09:51:49.025 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2023-10-11T09:51:49.025 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2023-10-11T09:51:49.025 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.a] 2023-10-11T09:51:49.026 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.b] 2023-10-11T09:51:49.026 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.c] 2023-10-11T09:51:49.026 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.a] 2023-10-11T09:51:49.026 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.c] 2023-10-11T09:51:49.026 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.b] 2023-10-11T09:51:49.027 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.d] 2023-10-11T09:51:49.027 INFO:teuthology.orchestra.run.smithi133.stdout:[mds] 2023-10-11T09:51:49.027 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds = 20 2023-10-11T09:51:49.027 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds balancer = 20 2023-10-11T09:51:49.027 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2023-10-11T09:51:49.028 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug frag = True 2023-10-11T09:51:49.028 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug scatterstat = False 2023-10-11T09:51:49.028 INFO:teuthology.orchestra.run.smithi133.stdout: mds op complaint time = 180 2023-10-11T09:51:49.028 INFO:teuthology.orchestra.run.smithi133.stdout: mds verify scatter = False 2023-10-11T09:51:49.028 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2023-10-11T09:51:49.029 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2023-10-11T09:51:49.029 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2023-10-11T09:51:49.029 INFO:teuthology.orchestra.run.smithi133.stdout: mds bal merge size = 0 2023-10-11T09:51:49.029 INFO:teuthology.orchestra.run.smithi133.stdout: mds bal split size = 10000 2023-10-11T09:51:49.030 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:51:49.030 DEBUG:teuthology.orchestra.run.smithi078:> import os 2023-10-11T09:51:49.030 DEBUG:teuthology.orchestra.run.smithi078:> path = os.path.join("/home/ubuntu/cephtest/mnt.0", "subdir") 2023-10-11T09:51:49.031 DEBUG:teuthology.orchestra.run.smithi078:> os.mkdir(path) 2023-10-11T09:51:49.031 DEBUG:teuthology.orchestra.run.smithi078:> for n in range(0, 100): 2023-10-11T09:51:49.031 DEBUG:teuthology.orchestra.run.smithi078:> open(os.path.join(path, "%s" % n), '"'"'w'"'"').write("%s" % n) 2023-10-11T09:51:49.032 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:51:49.717 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:51:49.717 DEBUG:teuthology.orchestra.run.smithi078:> import os 2023-10-11T09:51:49.718 DEBUG:teuthology.orchestra.run.smithi078:> import stat 2023-10-11T09:51:49.718 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:51:49.718 DEBUG:teuthology.orchestra.run.smithi078:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2023-10-11T09:51:49.719 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:51:49.863 INFO:teuthology.orchestra.run.smithi078.stdout:1099511627776 2023-10-11T09:51:50.051 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:51:50.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.256+0000 7f3abcd37700 1 Processor -- start 2023-10-11T09:51:50.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.256+0000 7f3abcd37700 1 -- start start 2023-10-11T09:51:50.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.256+0000 7f3abcd37700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f3ab815a800 0x7f3ab80b4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:50.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.256+0000 7f3abcd37700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3ab806bb60 0x7f3ab80b0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:50.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.256+0000 7f3abcd37700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ab806d060 0x7f3ab80b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:50.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.256+0000 7f3abcd37700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ab80b46a0 con 0x7f3ab815a800 2023-10-11T09:51:50.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.256+0000 7f3abcd37700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ab8070dc0 con 0x7f3ab806bb60 2023-10-11T09:51:50.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.256+0000 7f3abcd37700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab8070ff0 con 0x7f3ab806d060 2023-10-11T09:51:50.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.260+0000 7f3ab77fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f3ab815a800 0x7f3ab80b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:50.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.260+0000 7f3ab77fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f3ab815a800 0x7f3ab80b4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44130/0 (socket says 172.21.15.78:44130) 2023-10-11T09:51:50.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.260+0000 7f3ab77fe700 1 -- 172.21.15.78:0/345542492 learned_addr learned my addr 172.21.15.78:0/345542492 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:50.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.260+0000 7f3aaeffd700 1 --2- 172.21.15.78:0/345542492 >> v2:172.21.15.133:3300/0 conn(0x7f3ab806bb60 0x7f3ab80b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:50.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.260+0000 7f3ab7fff700 1 --2- 172.21.15.78:0/345542492 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ab806d060 0x7f3ab80b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:50.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.260+0000 7f3ab77fe700 1 -- 172.21.15.78:0/345542492 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ab806d060 msgr2=0x7f3ab80b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:50.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.260+0000 7f3ab77fe700 1 --2- 172.21.15.78:0/345542492 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ab806d060 0x7f3ab80b0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:50.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.260+0000 7f3ab77fe700 1 -- 172.21.15.78:0/345542492 >> v2:172.21.15.133:3300/0 conn(0x7f3ab806bb60 msgr2=0x7f3ab80b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:50.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.260+0000 7f3ab77fe700 1 --2- 172.21.15.78:0/345542492 >> v2:172.21.15.133:3300/0 conn(0x7f3ab806bb60 0x7f3ab80b0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:50.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.260+0000 7f3ab77fe700 1 -- 172.21.15.78:0/345542492 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab80712d0 con 0x7f3ab815a800 2023-10-11T09:51:50.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.260+0000 7f3ab77fe700 1 --2- 172.21.15.78:0/345542492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ab815a800 0x7f3ab80b4160 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f3ab806b7a0 tx=0x7f3aa8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1da4f501fb0f38a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:50.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.260+0000 7f3ab6ffd700 1 -- 172.21.15.78:0/345542492 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa8011830 con 0x7f3ab815a800 2023-10-11T09:51:50.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.260+0000 7f3ab6ffd700 1 -- 172.21.15.78:0/345542492 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa8011990 con 0x7f3ab815a800 2023-10-11T09:51:50.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.260+0000 7f3ab6ffd700 1 -- 172.21.15.78:0/345542492 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa8011d20 con 0x7f3ab815a800 2023-10-11T09:51:50.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.260+0000 7f3abcd37700 1 -- 172.21.15.78:0/345542492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ab815a800 msgr2=0x7f3ab80b4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:50.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.260+0000 7f3abcd37700 1 --2- 172.21.15.78:0/345542492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ab815a800 0x7f3ab80b4160 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f3ab806b7a0 tx=0x7f3aa8009bf0 comp rx=0 tx=0).stop 2023-10-11T09:51:50.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.260+0000 7f3abcd37700 1 -- 172.21.15.78:0/345542492 shutdown_connections 2023-10-11T09:51:50.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.260+0000 7f3abcd37700 1 --2- 172.21.15.78:0/345542492 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ab806d060 0x7f3ab80b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:50.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.260+0000 7f3abcd37700 1 --2- 172.21.15.78:0/345542492 >> v2:172.21.15.133:3300/0 conn(0x7f3ab806bb60 0x7f3ab80b0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:50.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.260+0000 7f3abcd37700 1 --2- 172.21.15.78:0/345542492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ab815a800 0x7f3ab80b4160 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:50.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.260+0000 7f3abcd37700 1 -- 172.21.15.78:0/345542492 >> 172.21.15.78:0/345542492 conn(0x7f3ab80cf820 msgr2=0x7f3ab80cb610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:50.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.260+0000 7f3abcd37700 1 -- 172.21.15.78:0/345542492 shutdown_connections 2023-10-11T09:51:50.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.260+0000 7f3abcd37700 1 -- 172.21.15.78:0/345542492 wait complete. 2023-10-11T09:51:50.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.264+0000 7f3abcd37700 1 Processor -- start 2023-10-11T09:51:50.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.264+0000 7f3abcd37700 1 -- start start 2023-10-11T09:51:50.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.264+0000 7f3abcd37700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ab806bb60 0x7f3ab8136830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:50.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.264+0000 7f3abcd37700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ab806d060 0x7f3ab8136d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:50.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.264+0000 7f3abcd37700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ab815a800 0x7f3ab814dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:50.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.268+0000 7f3abcd37700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab815db50 con 0x7f3ab806bb60 2023-10-11T09:51:50.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.268+0000 7f3abcd37700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab814a2f0 con 0x7f3ab806d060 2023-10-11T09:51:50.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.268+0000 7f3abcd37700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab815ced0 con 0x7f3ab815a800 2023-10-11T09:51:50.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.268+0000 7f3ab77fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ab806bb60 0x7f3ab8136830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:50.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.268+0000 7f3ab77fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ab806bb60 0x7f3ab8136830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44136/0 (socket says 172.21.15.78:44136) 2023-10-11T09:51:50.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.268+0000 7f3ab77fe700 1 -- 172.21.15.78:0/427965140 learned_addr learned my addr 172.21.15.78:0/427965140 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:50.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.268+0000 7f3aaeffd700 1 --2- 172.21.15.78:0/427965140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ab806d060 0x7f3ab8136d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:50.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.268+0000 7f3ab7fff700 1 --2- 172.21.15.78:0/427965140 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ab815a800 0x7f3ab814dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:50.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.268+0000 7f3ab77fe700 1 -- 172.21.15.78:0/427965140 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ab815a800 msgr2=0x7f3ab814dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:50.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.268+0000 7f3ab77fe700 1 --2- 172.21.15.78:0/427965140 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ab815a800 0x7f3ab814dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:50.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.268+0000 7f3ab77fe700 1 -- 172.21.15.78:0/427965140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ab806d060 msgr2=0x7f3ab8136d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:50.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.268+0000 7f3ab77fe700 1 --2- 172.21.15.78:0/427965140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ab806d060 0x7f3ab8136d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:50.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.268+0000 7f3ab77fe700 1 -- 172.21.15.78:0/427965140 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa8009570 con 0x7f3ab806bb60 2023-10-11T09:51:50.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.268+0000 7f3ab77fe700 1 --2- 172.21.15.78:0/427965140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ab806bb60 0x7f3ab8136830 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f3ab806f5f0 tx=0x7f3aa8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:50.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.268+0000 7f3ab4ff9700 1 -- 172.21.15.78:0/427965140 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa8004510 con 0x7f3ab806bb60 2023-10-11T09:51:50.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.268+0000 7f3ab4ff9700 1 -- 172.21.15.78:0/427965140 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa801e070 con 0x7f3ab806bb60 2023-10-11T09:51:50.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.268+0000 7f3abcd37700 1 -- 172.21.15.78:0/427965140 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab815d1b0 con 0x7f3ab806bb60 2023-10-11T09:51:50.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.268+0000 7f3ab4ff9700 1 -- 172.21.15.78:0/427965140 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa80038c0 con 0x7f3ab806bb60 2023-10-11T09:51:50.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.268+0000 7f3abcd37700 1 -- 172.21.15.78:0/427965140 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab815d770 con 0x7f3ab806bb60 2023-10-11T09:51:50.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.268+0000 7f3abcd37700 1 -- 172.21.15.78:0/427965140 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ab8067d00 con 0x7f3ab806bb60 2023-10-11T09:51:50.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.272+0000 7f3ab4ff9700 1 -- 172.21.15.78:0/427965140 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f3aa8003a20 con 0x7f3ab806bb60 2023-10-11T09:51:50.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.272+0000 7f3ab4ff9700 1 --2- 172.21.15.78:0/427965140 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3a98065ed0 0x7f3a98068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:50.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.272+0000 7f3ab4ff9700 1 -- 172.21.15.78:0/427965140 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3aa8020070 con 0x7f3ab806bb60 2023-10-11T09:51:50.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.272+0000 7f3aaeffd700 1 --2- 172.21.15.78:0/427965140 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3a98065ed0 0x7f3a98068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:50.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.276+0000 7f3aaeffd700 1 --2- 172.21.15.78:0/427965140 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3a98065ed0 0x7f3a98068380 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8145070 tx=0x7f3aa000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:50.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.280+0000 7f3ab4ff9700 1 -- 172.21.15.78:0/427965140 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f3aa8052220 con 0x7f3ab806bb60 2023-10-11T09:51:50.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.496+0000 7f3abcd37700 1 -- 172.21.15.78:0/427965140 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ab815d900 con 0x7f3ab806bb60 2023-10-11T09:51:50.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.496+0000 7f3ab4ff9700 1 -- 172.21.15.78:0/427965140 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4656 (secure 0 0 0) 0x7f3ab815d900 con 0x7f3ab806bb60 2023-10-11T09:51:50.503 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:50.503 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":99,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8452,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/2481257990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2481257990},{"type":"v1","addr":"172.21.15.78:6836","nonce":2481257990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":8464,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1035227282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1035227282},{"type":"v1","addr":"172.21.15.133:6835","nonce":1035227282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":8880,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/4189411531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":4189411531},{"type":"v1","addr":"172.21.15.78:6837","nonce":4189411531}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:51:40.211859+0000","modified":"2023-10-11T09:51:43.234832+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8883},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8883":{"gid":8883,"name":"d","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6833/1433079498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1433079498},{"type":"v1","addr":"172.21.15.133:6833","nonce":1433079498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-11T09:51:50.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.500+0000 7f3abcd37700 1 -- 172.21.15.78:0/427965140 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3a98065ed0 msgr2=0x7f3a98068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:50.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.500+0000 7f3abcd37700 1 --2- 172.21.15.78:0/427965140 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3a98065ed0 0x7f3a98068380 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8145070 tx=0x7f3aa000b040 comp rx=0 tx=0).stop 2023-10-11T09:51:50.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.500+0000 7f3abcd37700 1 -- 172.21.15.78:0/427965140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ab806bb60 msgr2=0x7f3ab8136830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:50.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.500+0000 7f3abcd37700 1 --2- 172.21.15.78:0/427965140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ab806bb60 0x7f3ab8136830 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f3ab806f5f0 tx=0x7f3aa8004770 comp rx=0 tx=0).stop 2023-10-11T09:51:50.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.500+0000 7f3abcd37700 1 -- 172.21.15.78:0/427965140 shutdown_connections 2023-10-11T09:51:50.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.500+0000 7f3abcd37700 1 --2- 172.21.15.78:0/427965140 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3a98065ed0 0x7f3a98068380 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:50.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.500+0000 7f3abcd37700 1 --2- 172.21.15.78:0/427965140 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ab815a800 0x7f3ab814dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:50.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.500+0000 7f3abcd37700 1 --2- 172.21.15.78:0/427965140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ab806d060 0x7f3ab8136d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:50.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.500+0000 7f3abcd37700 1 --2- 172.21.15.78:0/427965140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ab806bb60 0x7f3ab8136830 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:50.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.500+0000 7f3abcd37700 1 -- 172.21.15.78:0/427965140 >> 172.21.15.78:0/427965140 conn(0x7f3ab80cf820 msgr2=0x7f3ab80b2ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:50.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.500+0000 7f3abcd37700 1 -- 172.21.15.78:0/427965140 shutdown_connections 2023-10-11T09:51:50.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.500+0000 7f3abcd37700 1 -- 172.21.15.78:0/427965140 wait complete. 2023-10-11T09:51:50.508 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 99 2023-10-11T09:51:50.526 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:51:50.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.696+0000 7f89cdfba700 1 Processor -- start 2023-10-11T09:51:50.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.696+0000 7f89cdfba700 1 -- start start 2023-10-11T09:51:50.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.700+0000 7f89cdfba700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f89c815ddb0 0x7f89c8070790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:50.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.700+0000 7f89cdfba700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f89c815e2d0 0x7f89c815e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:50.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.700+0000 7f89cdfba700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f89c80b0920 0x7f89c80b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:50.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.700+0000 7f89cdfba700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89c806fd60 con 0x7f89c815ddb0 2023-10-11T09:51:50.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.700+0000 7f89cdfba700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89c8070cd0 con 0x7f89c815e2d0 2023-10-11T09:51:50.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.700+0000 7f89cdfba700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89c80b1270 con 0x7f89c80b0920 2023-10-11T09:51:50.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.700+0000 7f89c77fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f89c815ddb0 0x7f89c8070790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:50.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.700+0000 7f89c77fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f89c815ddb0 0x7f89c8070790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44148/0 (socket says 172.21.15.78:44148) 2023-10-11T09:51:50.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.700+0000 7f89c77fe700 1 -- 172.21.15.78:0/599863680 learned_addr learned my addr 172.21.15.78:0/599863680 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:50.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.700+0000 7f89c7fff700 1 --2- 172.21.15.78:0/599863680 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f89c80b0920 0x7f89c80b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:50.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.700+0000 7f89c6ffd700 1 --2- 172.21.15.78:0/599863680 >> v2:172.21.15.133:3300/0 conn(0x7f89c815e2d0 0x7f89c815e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:50.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.700+0000 7f89c77fe700 1 -- 172.21.15.78:0/599863680 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f89c80b0920 msgr2=0x7f89c80b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:50.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.700+0000 7f89c77fe700 1 --2- 172.21.15.78:0/599863680 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f89c80b0920 0x7f89c80b0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:50.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.700+0000 7f89c77fe700 1 -- 172.21.15.78:0/599863680 >> v2:172.21.15.133:3300/0 conn(0x7f89c815e2d0 msgr2=0x7f89c815e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:50.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.700+0000 7f89c77fe700 1 --2- 172.21.15.78:0/599863680 >> v2:172.21.15.133:3300/0 conn(0x7f89c815e2d0 0x7f89c815e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:50.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.700+0000 7f89c77fe700 1 -- 172.21.15.78:0/599863680 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c815ed50 con 0x7f89c815ddb0 2023-10-11T09:51:50.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.700+0000 7f89c77fe700 1 --2- 172.21.15.78:0/599863680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89c815ddb0 0x7f89c8070790 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f89c806e8f0 tx=0x7f89b8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eea91c01063a623b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:50.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.700+0000 7f89c67fc700 1 -- 172.21.15.78:0/599863680 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b8011830 con 0x7f89c815ddb0 2023-10-11T09:51:50.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.700+0000 7f89c67fc700 1 -- 172.21.15.78:0/599863680 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89b8011990 con 0x7f89c815ddb0 2023-10-11T09:51:50.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.700+0000 7f89c67fc700 1 -- 172.21.15.78:0/599863680 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b8011d20 con 0x7f89c815ddb0 2023-10-11T09:51:50.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.704+0000 7f89cdfba700 1 -- 172.21.15.78:0/599863680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89c815ddb0 msgr2=0x7f89c8070790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:50.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.704+0000 7f89cdfba700 1 --2- 172.21.15.78:0/599863680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89c815ddb0 0x7f89c8070790 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f89c806e8f0 tx=0x7f89b8009bf0 comp rx=0 tx=0).stop 2023-10-11T09:51:50.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.704+0000 7f89cdfba700 1 -- 172.21.15.78:0/599863680 shutdown_connections 2023-10-11T09:51:50.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.704+0000 7f89cdfba700 1 --2- 172.21.15.78:0/599863680 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f89c80b0920 0x7f89c80b0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:50.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.704+0000 7f89cdfba700 1 --2- 172.21.15.78:0/599863680 >> v2:172.21.15.133:3300/0 conn(0x7f89c815e2d0 0x7f89c815e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:50.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.704+0000 7f89cdfba700 1 --2- 172.21.15.78:0/599863680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89c815ddb0 0x7f89c8070790 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:50.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.704+0000 7f89cdfba700 1 -- 172.21.15.78:0/599863680 >> 172.21.15.78:0/599863680 conn(0x7f89c80cb920 msgr2=0x7f89c80cbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:50.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.704+0000 7f89cdfba700 1 -- 172.21.15.78:0/599863680 shutdown_connections 2023-10-11T09:51:50.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.704+0000 7f89cdfba700 1 -- 172.21.15.78:0/599863680 wait complete. 2023-10-11T09:51:50.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.704+0000 7f89cdfba700 1 Processor -- start 2023-10-11T09:51:50.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.704+0000 7f89cdfba700 1 -- start start 2023-10-11T09:51:50.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.708+0000 7f89cdfba700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f89c80b0920 0x7f89c815ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:50.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.708+0000 7f89cdfba700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89c815ddb0 0x7f89c80c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:50.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.708+0000 7f89cdfba700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89c815e2d0 0x7f89c80aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:50.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.708+0000 7f89cdfba700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c80cb040 con 0x7f89c815e2d0 2023-10-11T09:51:50.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.708+0000 7f89cdfba700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c815b360 con 0x7f89c815ddb0 2023-10-11T09:51:50.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.708+0000 7f89cdfba700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89c80ca3c0 con 0x7f89c80b0920 2023-10-11T09:51:50.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.708+0000 7f89c7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89c815e2d0 0x7f89c80aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:50.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.708+0000 7f89c7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89c815e2d0 0x7f89c80aeac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44160/0 (socket says 172.21.15.78:44160) 2023-10-11T09:51:50.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.708+0000 7f89c7fff700 1 -- 172.21.15.78:0/1333638461 learned_addr learned my addr 172.21.15.78:0/1333638461 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:50.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.708+0000 7f89c6ffd700 1 --2- 172.21.15.78:0/1333638461 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89c815ddb0 0x7f89c80c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:50.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.708+0000 7f89c77fe700 1 --2- 172.21.15.78:0/1333638461 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f89c80b0920 0x7f89c815ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:50.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.708+0000 7f89c7fff700 1 -- 172.21.15.78:0/1333638461 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f89c80b0920 msgr2=0x7f89c815ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:50.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.708+0000 7f89c7fff700 1 --2- 172.21.15.78:0/1333638461 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f89c80b0920 0x7f89c815ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:50.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.708+0000 7f89c7fff700 1 -- 172.21.15.78:0/1333638461 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89c815ddb0 msgr2=0x7f89c80c9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:50.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.708+0000 7f89c7fff700 1 --2- 172.21.15.78:0/1333638461 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89c815ddb0 0x7f89c80c9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:50.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.708+0000 7f89c7fff700 1 -- 172.21.15.78:0/1333638461 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b8009570 con 0x7f89c815e2d0 2023-10-11T09:51:50.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.708+0000 7f89c7fff700 1 --2- 172.21.15.78:0/1333638461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89c815e2d0 0x7f89c80aeac0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f89c000eed0 tx=0x7f89c000ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:50.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.708+0000 7f89affff700 1 -- 172.21.15.78:0/1333638461 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c0010040 con 0x7f89c815e2d0 2023-10-11T09:51:50.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.708+0000 7f89affff700 1 -- 172.21.15.78:0/1333638461 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c000b9f0 con 0x7f89c815e2d0 2023-10-11T09:51:50.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.708+0000 7f89cdfba700 1 -- 172.21.15.78:0/1333638461 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89c80ca6a0 con 0x7f89c815e2d0 2023-10-11T09:51:50.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.708+0000 7f89affff700 1 -- 172.21.15.78:0/1333638461 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c000bda0 con 0x7f89c815e2d0 2023-10-11T09:51:50.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.708+0000 7f89cdfba700 1 -- 172.21.15.78:0/1333638461 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89c80cac60 con 0x7f89c815e2d0 2023-10-11T09:51:50.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.712+0000 7f89cdfba700 1 -- 172.21.15.78:0/1333638461 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89c80cb180 con 0x7f89c815e2d0 2023-10-11T09:51:50.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.712+0000 7f89affff700 1 -- 172.21.15.78:0/1333638461 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f89c001d400 con 0x7f89c815e2d0 2023-10-11T09:51:50.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.712+0000 7f89affff700 1 --2- 172.21.15.78:0/1333638461 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f89b4065ed0 0x7f89b4068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:50.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.712+0000 7f89affff700 1 -- 172.21.15.78:0/1333638461 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f89c0083dc0 con 0x7f89c815e2d0 2023-10-11T09:51:50.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.716+0000 7f89c77fe700 1 --2- 172.21.15.78:0/1333638461 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f89b4065ed0 0x7f89b4068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:50.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.716+0000 7f89c77fe700 1 --2- 172.21.15.78:0/1333638461 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f89b4065ed0 0x7f89b4068380 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f89b8000c00 tx=0x7f89b801d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:50.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.720+0000 7f89affff700 1 -- 172.21.15.78:0/1333638461 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f89c80cb180 con 0x7f89c815e2d0 2023-10-11T09:51:50.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.928+0000 7f89cdfba700 1 -- 172.21.15.78:0/1333638461 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89c80cb180 con 0x7f89c815e2d0 2023-10-11T09:51:50.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.928+0000 7f89affff700 1 -- 172.21.15.78:0/1333638461 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4656 (secure 0 0 0) 0x7f89c80cb180 con 0x7f89c815e2d0 2023-10-11T09:51:50.936 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:50.936 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":99,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8452,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/2481257990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2481257990},{"type":"v1","addr":"172.21.15.78:6836","nonce":2481257990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":8464,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1035227282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1035227282},{"type":"v1","addr":"172.21.15.133:6835","nonce":1035227282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":8880,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/4189411531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":4189411531},{"type":"v1","addr":"172.21.15.78:6837","nonce":4189411531}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:51:40.211859+0000","modified":"2023-10-11T09:51:43.234832+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8883},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8883":{"gid":8883,"name":"d","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6833/1433079498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1433079498},{"type":"v1","addr":"172.21.15.133:6833","nonce":1433079498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-11T09:51:50.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.932+0000 7f89cdfba700 1 -- 172.21.15.78:0/1333638461 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f89b4065ed0 msgr2=0x7f89b4068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:50.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.932+0000 7f89cdfba700 1 --2- 172.21.15.78:0/1333638461 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f89b4065ed0 0x7f89b4068380 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f89b8000c00 tx=0x7f89b801d040 comp rx=0 tx=0).stop 2023-10-11T09:51:50.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.932+0000 7f89cdfba700 1 -- 172.21.15.78:0/1333638461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89c815e2d0 msgr2=0x7f89c80aeac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:50.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.932+0000 7f89cdfba700 1 --2- 172.21.15.78:0/1333638461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89c815e2d0 0x7f89c80aeac0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f89c000eed0 tx=0x7f89c000ef00 comp rx=0 tx=0).stop 2023-10-11T09:51:50.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.932+0000 7f89cdfba700 1 -- 172.21.15.78:0/1333638461 shutdown_connections 2023-10-11T09:51:50.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.932+0000 7f89cdfba700 1 --2- 172.21.15.78:0/1333638461 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f89b4065ed0 0x7f89b4068380 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:50.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.932+0000 7f89cdfba700 1 --2- 172.21.15.78:0/1333638461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89c815e2d0 0x7f89c80aeac0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:50.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.932+0000 7f89cdfba700 1 --2- 172.21.15.78:0/1333638461 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89c815ddb0 0x7f89c80c9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:50.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.932+0000 7f89cdfba700 1 --2- 172.21.15.78:0/1333638461 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f89c80b0920 0x7f89c815ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:50.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.932+0000 7f89cdfba700 1 -- 172.21.15.78:0/1333638461 >> 172.21.15.78:0/1333638461 conn(0x7f89c80cb920 msgr2=0x7f89c806cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:50.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.932+0000 7f89cdfba700 1 -- 172.21.15.78:0/1333638461 shutdown_connections 2023-10-11T09:51:50.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:50.932+0000 7f89cdfba700 1 -- 172.21.15.78:0/1333638461 wait complete. 2023-10-11T09:51:50.942 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 99 2023-10-11T09:51:50.960 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json dirfrag split /subdir 0/0 1 2023-10-11T09:51:51.144 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2023-10-11T09:51:51.145 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:51:51.145 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:51:51.162 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:51:51.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.376+0000 7fec9e434700 1 Processor -- start 2023-10-11T09:51:51.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.380+0000 7fec9e434700 1 -- start start 2023-10-11T09:51:51.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.380+0000 7fec9e434700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fec980b4be0 0x7fec9806b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:51.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.380+0000 7fec9e434700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec9815de60 con 0x7fec980d0d30 2023-10-11T09:51:51.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.380+0000 7fec9e434700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec980cd850 con 0x7fec9815d6a0 2023-10-11T09:51:51.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.380+0000 7fec9e434700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec980cdb30 con 0x7fec980b4be0 2023-10-11T09:51:51.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.380+0000 7fec9c9d1700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fec980d0d30 0x7fec980cfa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:50252/0 (socket says 172.21.15.78:50252) 2023-10-11T09:51:51.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.384+0000 7fec9c9d1700 1 -- 172.21.15.78:0/1342214291 learned_addr learned my addr 172.21.15.78:0/1342214291 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:51:51.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.384+0000 7fec97fff700 1 --2- 172.21.15.78:0/1342214291 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fec980b4be0 0x7fec9806b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:51.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.384+0000 7fec977fe700 1 -- 172.21.15.78:0/1342214291 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 608093293 0 0) 0x7fec9815de60 con 0x7fec980d0d30 2023-10-11T09:51:51.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.384+0000 7fec977fe700 1 -- 172.21.15.78:0/1342214291 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec840035d0 con 0x7fec980d0d30 2023-10-11T09:51:51.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.384+0000 7fec977fe700 1 -- 172.21.15.78:0/1342214291 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2137405956 0 0) 0x7fec980cd850 con 0x7fec9815d6a0 2023-10-11T09:51:51.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.384+0000 7fec977fe700 1 -- 172.21.15.78:0/1342214291 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec9815de60 con 0x7fec9815d6a0 2023-10-11T09:51:51.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.384+0000 7fec97fff700 1 -- 172.21.15.78:0/1342214291 >> v1:172.21.15.133:6789/0 conn(0x7fec9815d6a0 legacy=0x7fec980b4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:51.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.384+0000 7fec97fff700 1 -- 172.21.15.78:0/1342214291 >> v1:172.21.15.78:6789/0 conn(0x7fec980d0d30 legacy=0x7fec980cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:51.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.384+0000 7fec977fe700 1 -- 172.21.15.78:0/1342214291 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 757678923 0 0) 0x7fec840035d0 con 0x7fec980d0d30 2023-10-11T09:51:51.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.384+0000 7fec97fff700 1 -- 172.21.15.78:0/1342214291 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec980cde10 con 0x7fec980b4be0 2023-10-11T09:51:51.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.384+0000 7fec97fff700 1 --2- 172.21.15.78:0/1342214291 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fec980b4be0 0x7fec9806b700 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fec88009880 tx=0x7fec88009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb85a4c317f4788e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:51.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.384+0000 7fec977fe700 1 -- 172.21.15.78:0/1342214291 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec90001670 con 0x7fec980b4be0 2023-10-11T09:51:51.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.384+0000 7fec977fe700 1 -- 172.21.15.78:0/1342214291 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec88004030 con 0x7fec980b4be0 2023-10-11T09:51:51.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.384+0000 7fec977fe700 1 -- 172.21.15.78:0/1342214291 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec880043c0 con 0x7fec980b4be0 2023-10-11T09:51:51.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.384+0000 7fec9e434700 1 -- 172.21.15.78:0/1342214291 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fec980b4be0 msgr2=0x7fec9806b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:51.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.384+0000 7fec9e434700 1 --2- 172.21.15.78:0/1342214291 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fec980b4be0 0x7fec9806b700 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fec88009880 tx=0x7fec88009b90 comp rx=0 tx=0).stop 2023-10-11T09:51:51.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.384+0000 7fec9e434700 1 -- 172.21.15.78:0/1342214291 shutdown_connections 2023-10-11T09:51:51.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.384+0000 7fec9e434700 1 --2- 172.21.15.78:0/1342214291 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fec980b4be0 0x7fec9806b700 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:51.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.384+0000 7fec9e434700 1 -- 172.21.15.78:0/1342214291 >> 172.21.15.78:0/1342214291 conn(0x7fec9806abf0 msgr2=0x7fec9806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:51.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.384+0000 7fec9e434700 1 -- 172.21.15.78:0/1342214291 shutdown_connections 2023-10-11T09:51:51.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.384+0000 7fec9e434700 1 -- 172.21.15.78:0/1342214291 wait complete. 2023-10-11T09:51:51.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.388+0000 7fec9e434700 1 Processor -- start 2023-10-11T09:51:51.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.388+0000 7fec9e434700 1 -- start start 2023-10-11T09:51:51.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.388+0000 7fec9e434700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fec980b4be0 0x7fec9814d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:51.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.388+0000 7fec9e434700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec980d0d30 0x7fec980c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:51.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.388+0000 7fec9e434700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec9815d6a0 0x7fec980c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:51.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.388+0000 7fec9e434700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec980b67f0 con 0x7fec980d0d30 2023-10-11T09:51:51.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.388+0000 7fec9e434700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec981f6a10 con 0x7fec9815d6a0 2023-10-11T09:51:51.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.388+0000 7fec9e434700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec980c9af0 con 0x7fec980b4be0 2023-10-11T09:51:51.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.388+0000 7fec8f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec980d0d30 0x7fec980c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:51.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.388+0000 7fec8f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec980d0d30 0x7fec980c9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44174/0 (socket says 172.21.15.78:44174) 2023-10-11T09:51:51.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.388+0000 7fec8f7fe700 1 -- 172.21.15.78:0/2084699344 learned_addr learned my addr 172.21.15.78:0/2084699344 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:51.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.392+0000 7fec97fff700 1 --2- 172.21.15.78:0/2084699344 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fec980b4be0 0x7fec9814d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:51.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.392+0000 7fec9c9d1700 1 --2- 172.21.15.78:0/2084699344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec9815d6a0 0x7fec980c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:51.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.392+0000 7fec8f7fe700 1 -- 172.21.15.78:0/2084699344 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fec980b4be0 msgr2=0x7fec9814d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:51.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.392+0000 7fec8f7fe700 1 --2- 172.21.15.78:0/2084699344 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fec980b4be0 0x7fec9814d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:51.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.392+0000 7fec8f7fe700 1 -- 172.21.15.78:0/2084699344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec9815d6a0 msgr2=0x7fec980c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:51.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.392+0000 7fec8f7fe700 1 --2- 172.21.15.78:0/2084699344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec9815d6a0 0x7fec980c95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:51.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.392+0000 7fec8f7fe700 1 -- 172.21.15.78:0/2084699344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec880094b0 con 0x7fec980d0d30 2023-10-11T09:51:51.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.392+0000 7fec8f7fe700 1 --2- 172.21.15.78:0/2084699344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec980d0d30 0x7fec980c9070 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fec80001f00 tx=0x7fec8000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:51.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.392+0000 7fec957fa700 1 -- 172.21.15.78:0/2084699344 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec80002100 con 0x7fec980d0d30 2023-10-11T09:51:51.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.392+0000 7fec957fa700 1 -- 172.21.15.78:0/2084699344 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec8000ade0 con 0x7fec980d0d30 2023-10-11T09:51:51.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.392+0000 7fec957fa700 1 -- 172.21.15.78:0/2084699344 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec80003c60 con 0x7fec980d0d30 2023-10-11T09:51:51.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.392+0000 7fec9e434700 1 -- 172.21.15.78:0/2084699344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec9814da50 con 0x7fec980d0d30 2023-10-11T09:51:51.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.392+0000 7fec9e434700 1 -- 172.21.15.78:0/2084699344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec980b6130 con 0x7fec980d0d30 2023-10-11T09:51:51.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.392+0000 7fec9e434700 1 -- 172.21.15.78:0/2084699344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec980b5f90 con 0x7fec980d0d30 2023-10-11T09:51:51.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.396+0000 7fec957fa700 1 -- 172.21.15.78:0/2084699344 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fec800050d0 con 0x7fec980d0d30 2023-10-11T09:51:51.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.396+0000 7fec957fa700 1 --2- 172.21.15.78:0/2084699344 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fec78065ed0 0x7fec78068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:51.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.396+0000 7fec957fa700 1 -- 172.21.15.78:0/2084699344 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fec80083a40 con 0x7fec980d0d30 2023-10-11T09:51:51.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.396+0000 7fec97fff700 1 --2- 172.21.15.78:0/2084699344 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fec78065ed0 0x7fec78068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:51.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.396+0000 7fec97fff700 1 --2- 172.21.15.78:0/2084699344 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fec78065ed0 0x7fec78068380 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fec880090b0 tx=0x7fec88010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:51.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.404+0000 7fec957fa700 1 -- 172.21.15.78:0/2084699344 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fec980b5f90 con 0x7fec980d0d30 2023-10-11T09:51:51.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.580+0000 7fec9e434700 1 -- 172.21.15.78:0/2084699344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec980b6930 con 0x7fec980d0d30 2023-10-11T09:51:51.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.580+0000 7fec957fa700 1 -- 172.21.15.78:0/2084699344 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+17893 (secure 0 0 0) 0x7fec980b6930 con 0x7fec980d0d30 2023-10-11T09:51:51.587 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:51.587 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":127,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:51:41.218018+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:51:38.592892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-11T09:51:39.204588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:51:51.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.584+0000 7fec9e434700 1 -- 172.21.15.78:0/2084699344 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fec78065ed0 msgr2=0x7fec78068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:51.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.584+0000 7fec9e434700 1 --2- 172.21.15.78:0/2084699344 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fec78065ed0 0x7fec78068380 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fec880090b0 tx=0x7fec88010000 comp rx=0 tx=0).stop 2023-10-11T09:51:51.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.584+0000 7fec9e434700 1 -- 172.21.15.78:0/2084699344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec980d0d30 msgr2=0x7fec980c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:51.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.584+0000 7fec9e434700 1 --2- 172.21.15.78:0/2084699344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec980d0d30 0x7fec980c9070 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fec80001f00 tx=0x7fec8000a340 comp rx=0 tx=0).stop 2023-10-11T09:51:51.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.584+0000 7fec9e434700 1 -- 172.21.15.78:0/2084699344 shutdown_connections 2023-10-11T09:51:51.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.584+0000 7fec9e434700 1 --2- 172.21.15.78:0/2084699344 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fec78065ed0 0x7fec78068380 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:51.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.584+0000 7fec9e434700 1 --2- 172.21.15.78:0/2084699344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec9815d6a0 0x7fec980c95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:51.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.584+0000 7fec9e434700 1 --2- 172.21.15.78:0/2084699344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec980d0d30 0x7fec980c9070 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:51.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.584+0000 7fec9e434700 1 --2- 172.21.15.78:0/2084699344 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fec980b4be0 0x7fec9814d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:51.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.584+0000 7fec9e434700 1 -- 172.21.15.78:0/2084699344 >> 172.21.15.78:0/2084699344 conn(0x7fec9806abf0 msgr2=0x7fec98160e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:51.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.584+0000 7fec9e434700 1 -- 172.21.15.78:0/2084699344 shutdown_connections 2023-10-11T09:51:51.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.584+0000 7fec9e434700 1 -- 172.21.15.78:0/2084699344 wait complete. 2023-10-11T09:51:51.614 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T09:51:38.592892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T09:51:39.204588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '127', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:51:51.615 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:51:51.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.792+0000 7fbbce2b1700 1 Processor -- start 2023-10-11T09:51:51.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.792+0000 7fbbce2b1700 1 -- start start 2023-10-11T09:51:51.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.796+0000 7fbbce2b1700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fbbc80d16f0 0x7fbbc80cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:51.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.796+0000 7fbbce2b1700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbbc80b74c0 0x7fbbc80cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:51.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.796+0000 7fbbce2b1700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbc809c960 con 0x7fbbc80b4530 2023-10-11T09:51:51.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.796+0000 7fbbce2b1700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbc815d180 con 0x7fbbc80d16f0 2023-10-11T09:51:51.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.796+0000 7fbbce2b1700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc80b5fd0 con 0x7fbbc80b74c0 2023-10-11T09:51:51.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.796+0000 7fbbccaae700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fbbc80d16f0 0x7fbbc80cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:51.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.796+0000 7fbbccaae700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fbbc80d16f0 0x7fbbc80cd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44186/0 (socket says 172.21.15.78:44186) 2023-10-11T09:51:51.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.796+0000 7fbbccaae700 1 -- 172.21.15.78:0/2768636795 learned_addr learned my addr 172.21.15.78:0/2768636795 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:51.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.796+0000 7fbbccaae700 1 -- 172.21.15.78:0/2768636795 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbbc80b74c0 msgr2=0x7fbbc80cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:51.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.796+0000 7fbbccaae700 1 --2- 172.21.15.78:0/2768636795 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbbc80b74c0 0x7fbbc80cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:51.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.796+0000 7fbbccaae700 1 -- 172.21.15.78:0/2768636795 >> v1:172.21.15.133:6789/0 conn(0x7fbbc80b4530 legacy=0x7fbbc80d0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:51.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.796+0000 7fbbccaae700 1 -- 172.21.15.78:0/2768636795 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc80cfb70 con 0x7fbbc80d16f0 2023-10-11T09:51:51.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.796+0000 7fbbccaae700 1 --2- 172.21.15.78:0/2768636795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbc80d16f0 0x7fbbc80cd730 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8009910 tx=0x7fbbb8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c7be30da1167a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:51.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.796+0000 7fbbbffff700 1 -- 172.21.15.78:0/2768636795 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8004230 con 0x7fbbc80d16f0 2023-10-11T09:51:51.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.796+0000 7fbbbffff700 1 -- 172.21.15.78:0/2768636795 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb8004390 con 0x7fbbc80d16f0 2023-10-11T09:51:51.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.796+0000 7fbbbffff700 1 -- 172.21.15.78:0/2768636795 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8009e60 con 0x7fbbc80d16f0 2023-10-11T09:51:51.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.796+0000 7fbbce2b1700 1 -- 172.21.15.78:0/2768636795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbc80d16f0 msgr2=0x7fbbc80cd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:51.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.796+0000 7fbbce2b1700 1 --2- 172.21.15.78:0/2768636795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbc80d16f0 0x7fbbc80cd730 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8009910 tx=0x7fbbb8009c20 comp rx=0 tx=0).stop 2023-10-11T09:51:51.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.800+0000 7fbbce2b1700 1 -- 172.21.15.78:0/2768636795 shutdown_connections 2023-10-11T09:51:51.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.800+0000 7fbbce2b1700 1 --2- 172.21.15.78:0/2768636795 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbbc80b74c0 0x7fbbc80cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:51.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.800+0000 7fbbce2b1700 1 --2- 172.21.15.78:0/2768636795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbc80d16f0 0x7fbbc80cd730 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:51.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.800+0000 7fbbce2b1700 1 -- 172.21.15.78:0/2768636795 >> 172.21.15.78:0/2768636795 conn(0x7fbbc809c3d0 msgr2=0x7fbbc80d30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:51.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.800+0000 7fbbce2b1700 1 -- 172.21.15.78:0/2768636795 shutdown_connections 2023-10-11T09:51:51.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.800+0000 7fbbce2b1700 1 -- 172.21.15.78:0/2768636795 wait complete. 2023-10-11T09:51:51.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.800+0000 7fbbce2b1700 1 Processor -- start 2023-10-11T09:51:51.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.800+0000 7fbbce2b1700 1 -- start start 2023-10-11T09:51:51.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.804+0000 7fbbce2b1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbc80b4530 0x7fbbc81f5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:51.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.804+0000 7fbbce2b1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbc80b74c0 0x7fbbc80c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:51.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.804+0000 7fbbce2b1700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbbc80d16f0 0x7fbbc80b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:51.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.804+0000 7fbbce2b1700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc81f6110 con 0x7fbbc80b74c0 2023-10-11T09:51:51.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.804+0000 7fbbce2b1700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc81573a0 con 0x7fbbc80b4530 2023-10-11T09:51:51.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.804+0000 7fbbce2b1700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc8153170 con 0x7fbbc80d16f0 2023-10-11T09:51:51.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.804+0000 7fbbccaae700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbc80b74c0 0x7fbbc80c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:51.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.804+0000 7fbbccaae700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbc80b74c0 0x7fbbc80c85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44188/0 (socket says 172.21.15.78:44188) 2023-10-11T09:51:51.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.804+0000 7fbbccaae700 1 -- 172.21.15.78:0/4232574184 learned_addr learned my addr 172.21.15.78:0/4232574184 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:51.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.804+0000 7fbbcd2af700 1 --2- 172.21.15.78:0/4232574184 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbc80b4530 0x7fbbc81f5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:51.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.804+0000 7fbbccaae700 1 -- 172.21.15.78:0/4232574184 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbbc80d16f0 msgr2=0x7fbbc80b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:51.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.804+0000 7fbbcdab0700 1 --2- 172.21.15.78:0/4232574184 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbbc80d16f0 0x7fbbc80b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:51.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.804+0000 7fbbccaae700 1 --2- 172.21.15.78:0/4232574184 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbbc80d16f0 0x7fbbc80b4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:51.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.804+0000 7fbbccaae700 1 -- 172.21.15.78:0/4232574184 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbc80b4530 msgr2=0x7fbbc81f5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:51.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.804+0000 7fbbccaae700 1 --2- 172.21.15.78:0/4232574184 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbc80b4530 0x7fbbc81f5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:51.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.804+0000 7fbbccaae700 1 -- 172.21.15.78:0/4232574184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb8009570 con 0x7fbbc80b74c0 2023-10-11T09:51:51.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.804+0000 7fbbcd2af700 1 --2- 172.21.15.78:0/4232574184 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbc80b4530 0x7fbbc81f5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:51:51.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.804+0000 7fbbcdab0700 1 --2- 172.21.15.78:0/4232574184 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbbc80d16f0 0x7fbbc80b4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:51:51.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.804+0000 7fbbccaae700 1 --2- 172.21.15.78:0/4232574184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbc80b74c0 0x7fbbc80c85f0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fbbc80b5cd0 tx=0x7fbbb8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:51.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.804+0000 7fbbbdffb700 1 -- 172.21.15.78:0/4232574184 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb800bd30 con 0x7fbbc80b74c0 2023-10-11T09:51:51.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.804+0000 7fbbbdffb700 1 -- 172.21.15.78:0/4232574184 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb801f070 con 0x7fbbc80b74c0 2023-10-11T09:51:51.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.804+0000 7fbbbdffb700 1 -- 172.21.15.78:0/4232574184 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb80038e0 con 0x7fbbc80b74c0 2023-10-11T09:51:51.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.804+0000 7fbbce2b1700 1 -- 172.21.15.78:0/4232574184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbc81f2190 con 0x7fbbc80b74c0 2023-10-11T09:51:51.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.804+0000 7fbbce2b1700 1 -- 172.21.15.78:0/4232574184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbc81f2750 con 0x7fbbc80b74c0 2023-10-11T09:51:51.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.808+0000 7fbbce2b1700 1 -- 172.21.15.78:0/4232574184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbc8157570 con 0x7fbbc80b74c0 2023-10-11T09:51:51.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.808+0000 7fbbbdffb700 1 -- 172.21.15.78:0/4232574184 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fbbb8021030 con 0x7fbbc80b74c0 2023-10-11T09:51:51.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.812+0000 7fbbbdffb700 1 --2- 172.21.15.78:0/4232574184 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbbac065ed0 0x7fbbac068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:51.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.812+0000 7fbbbdffb700 1 -- 172.21.15.78:0/4232574184 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbbb8084ea0 con 0x7fbbc80b74c0 2023-10-11T09:51:51.818 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.812+0000 7fbbcd2af700 1 --2- 172.21.15.78:0/4232574184 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbbac065ed0 0x7fbbac068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:51.818 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.812+0000 7fbbcd2af700 1 --2- 172.21.15.78:0/4232574184 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbbac065ed0 0x7fbbac068380 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fbbc80a1080 tx=0x7fbbc4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:51.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.816+0000 7fbbbdffb700 1 -- 172.21.15.78:0/4232574184 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fbbc8157570 con 0x7fbbc80b74c0 2023-10-11T09:51:51.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.992+0000 7fbbce2b1700 1 -- 172.21.15.78:0/4232574184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbbc8157570 con 0x7fbbc80b74c0 2023-10-11T09:51:51.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.992+0000 7fbbbdffb700 1 -- 172.21.15.78:0/4232574184 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbbc8157570 con 0x7fbbc80b74c0 2023-10-11T09:51:51.998 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:51:52.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.992+0000 7fbbce2b1700 1 -- 172.21.15.78:0/4232574184 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbbac065ed0 msgr2=0x7fbbac068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:52.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.992+0000 7fbbce2b1700 1 --2- 172.21.15.78:0/4232574184 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbbac065ed0 0x7fbbac068380 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fbbc80a1080 tx=0x7fbbc4008040 comp rx=0 tx=0).stop 2023-10-11T09:51:52.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.992+0000 7fbbce2b1700 1 -- 172.21.15.78:0/4232574184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbc80b74c0 msgr2=0x7fbbc80c85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:52.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.992+0000 7fbbce2b1700 1 --2- 172.21.15.78:0/4232574184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbc80b74c0 0x7fbbc80c85f0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fbbc80b5cd0 tx=0x7fbbb8004770 comp rx=0 tx=0).stop 2023-10-11T09:51:52.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.992+0000 7fbbce2b1700 1 -- 172.21.15.78:0/4232574184 shutdown_connections 2023-10-11T09:51:52.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.996+0000 7fbbce2b1700 1 --2- 172.21.15.78:0/4232574184 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbbac065ed0 0x7fbbac068380 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:52.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.996+0000 7fbbce2b1700 1 --2- 172.21.15.78:0/4232574184 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbbc80d16f0 0x7fbbc80b4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:52.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.996+0000 7fbbce2b1700 1 --2- 172.21.15.78:0/4232574184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbc80b74c0 0x7fbbc80c85f0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:52.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.996+0000 7fbbce2b1700 1 --2- 172.21.15.78:0/4232574184 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbc80b4530 0x7fbbc81f5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:52.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.996+0000 7fbbce2b1700 1 -- 172.21.15.78:0/4232574184 >> 172.21.15.78:0/4232574184 conn(0x7fbbc809c3d0 msgr2=0x7fbbc80d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:52.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.996+0000 7fbbce2b1700 1 -- 172.21.15.78:0/4232574184 shutdown_connections 2023-10-11T09:51:52.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:51.996+0000 7fbbce2b1700 1 -- 172.21.15.78:0/4232574184 wait complete. 2023-10-11T09:51:52.022 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:51:52.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.188+0000 7fccadedb700 1 Processor -- start 2023-10-11T09:51:52.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.188+0000 7fccadedb700 1 -- start start 2023-10-11T09:51:52.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.188+0000 7fccadedb700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcca80b0d20 0x7fcca80b4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:52.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.188+0000 7fccadedb700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcca806bb60 0x7fcca815e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:52.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.188+0000 7fccadedb700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcca8070dc0 con 0x7fcca815a800 2023-10-11T09:51:52.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.188+0000 7fccadedb700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcca80b46a0 con 0x7fcca80b0d20 2023-10-11T09:51:52.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.188+0000 7fccadedb700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcca8070ff0 con 0x7fcca806bb60 2023-10-11T09:51:52.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.192+0000 7fccad6da700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcca806bb60 0x7fcca815e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:52.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.192+0000 7fcca7fff700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcca80b0d20 0x7fcca80b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:52.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.192+0000 7fccaced9700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fcca815a800 0x7fcca80b0610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:51206/0 (socket says 172.21.15.78:51206) 2023-10-11T09:51:52.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.192+0000 7fccad6da700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcca806bb60 0x7fcca815e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:41844/0 (socket says 172.21.15.78:41844) 2023-10-11T09:51:52.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.192+0000 7fccaced9700 1 -- 172.21.15.78:0/1540560884 learned_addr learned my addr 172.21.15.78:0/1540560884 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:51:52.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.192+0000 7fccad6da700 1 -- 172.21.15.78:0/1540560884 >> v2:172.21.15.133:3300/0 conn(0x7fcca80b0d20 msgr2=0x7fcca80b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:52.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.192+0000 7fccad6da700 1 --2- 172.21.15.78:0/1540560884 >> v2:172.21.15.133:3300/0 conn(0x7fcca80b0d20 0x7fcca80b4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:52.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.192+0000 7fccad6da700 1 -- 172.21.15.78:0/1540560884 >> v1:172.21.15.133:6789/0 conn(0x7fcca815a800 legacy=0x7fcca80b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:52.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.192+0000 7fccad6da700 1 -- 172.21.15.78:0/1540560884 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca80712d0 con 0x7fcca806bb60 2023-10-11T09:51:52.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.192+0000 7fcca77fe700 1 -- 172.21.15.78:0/1540560884 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3076794939 0 0) 0x7fcca8070dc0 con 0x7fcca815a800 2023-10-11T09:51:52.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.192+0000 7fccad6da700 1 --2- 172.21.15.78:0/1540560884 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcca806bb60 0x7fcca815e590 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcca0009850 tx=0x7fcca0009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47fff42fc9f234ae server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:52.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.192+0000 7fcca77fe700 1 -- 172.21.15.78:0/1540560884 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca0011830 con 0x7fcca806bb60 2023-10-11T09:51:52.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.192+0000 7fcca77fe700 1 -- 172.21.15.78:0/1540560884 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca0011990 con 0x7fcca806bb60 2023-10-11T09:51:52.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.192+0000 7fcca77fe700 1 -- 172.21.15.78:0/1540560884 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca0011d40 con 0x7fcca806bb60 2023-10-11T09:51:52.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.192+0000 7fccadedb700 1 -- 172.21.15.78:0/1540560884 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcca806bb60 msgr2=0x7fcca815e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:52.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.192+0000 7fccadedb700 1 --2- 172.21.15.78:0/1540560884 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcca806bb60 0x7fcca815e590 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcca0009850 tx=0x7fcca0009b60 comp rx=0 tx=0).stop 2023-10-11T09:51:52.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.192+0000 7fccadedb700 1 -- 172.21.15.78:0/1540560884 shutdown_connections 2023-10-11T09:51:52.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.192+0000 7fccadedb700 1 --2- 172.21.15.78:0/1540560884 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcca806bb60 0x7fcca815e590 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:52.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.192+0000 7fccadedb700 1 --2- 172.21.15.78:0/1540560884 >> v2:172.21.15.133:3300/0 conn(0x7fcca80b0d20 0x7fcca80b4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:52.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.192+0000 7fccadedb700 1 -- 172.21.15.78:0/1540560884 >> 172.21.15.78:0/1540560884 conn(0x7fcca80cf820 msgr2=0x7fcca80cb610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:52.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.192+0000 7fccadedb700 1 -- 172.21.15.78:0/1540560884 shutdown_connections 2023-10-11T09:51:52.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.192+0000 7fccadedb700 1 -- 172.21.15.78:0/1540560884 wait complete. 2023-10-11T09:51:52.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.196+0000 7fccadedb700 1 Processor -- start 2023-10-11T09:51:52.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.196+0000 7fccadedb700 1 -- start start 2023-10-11T09:51:52.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.200+0000 7fccadedb700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcca806bb60 0x7fcca81e5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:52.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.200+0000 7fccadedb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcca80b0d20 0x7fcca80c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:52.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.200+0000 7fccadedb700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcca815a800 0x7fcca814ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:52.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.200+0000 7fccadedb700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca80c75d0 con 0x7fcca80b0d20 2023-10-11T09:51:52.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.200+0000 7fccadedb700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca81e5d60 con 0x7fcca815a800 2023-10-11T09:51:52.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.200+0000 7fccadedb700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcca80c6b20 con 0x7fcca806bb60 2023-10-11T09:51:52.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.200+0000 7fcca7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcca80b0d20 0x7fcca80c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:52.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.200+0000 7fcca7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcca80b0d20 0x7fcca80c65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44198/0 (socket says 172.21.15.78:44198) 2023-10-11T09:51:52.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.200+0000 7fcca7fff700 1 -- 172.21.15.78:0/3371749271 learned_addr learned my addr 172.21.15.78:0/3371749271 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:52.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.200+0000 7fccaced9700 1 --2- 172.21.15.78:0/3371749271 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcca806bb60 0x7fcca81e5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:52.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.200+0000 7fccad6da700 1 --2- 172.21.15.78:0/3371749271 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcca815a800 0x7fcca814ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:52.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.200+0000 7fcca7fff700 1 -- 172.21.15.78:0/3371749271 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcca806bb60 msgr2=0x7fcca81e5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:52.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.200+0000 7fcca7fff700 1 --2- 172.21.15.78:0/3371749271 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcca806bb60 0x7fcca81e5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:52.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.200+0000 7fcca7fff700 1 -- 172.21.15.78:0/3371749271 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcca815a800 msgr2=0x7fcca814ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:52.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.200+0000 7fcca7fff700 1 --2- 172.21.15.78:0/3371749271 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcca815a800 0x7fcca814ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:52.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.200+0000 7fcca7fff700 1 -- 172.21.15.78:0/3371749271 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca00094b0 con 0x7fcca80b0d20 2023-10-11T09:51:52.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.200+0000 7fcca7fff700 1 --2- 172.21.15.78:0/3371749271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcca80b0d20 0x7fcca80c65e0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fcc9000e980 tx=0x7fcc9000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:52.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.200+0000 7fcca57fa700 1 -- 172.21.15.78:0/3371749271 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc90009710 con 0x7fcca80b0d20 2023-10-11T09:51:52.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.200+0000 7fcca57fa700 1 -- 172.21.15.78:0/3371749271 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc9000c690 con 0x7fcca80b0d20 2023-10-11T09:51:52.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.200+0000 7fccadedb700 1 -- 172.21.15.78:0/3371749271 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcca814d3b0 con 0x7fcca80b0d20 2023-10-11T09:51:52.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.200+0000 7fcca57fa700 1 -- 172.21.15.78:0/3371749271 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9000ca40 con 0x7fcca80b0d20 2023-10-11T09:51:52.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.204+0000 7fccadedb700 1 -- 172.21.15.78:0/3371749271 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcca80c7010 con 0x7fcca80b0d20 2023-10-11T09:51:52.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.204+0000 7fccadedb700 1 -- 172.21.15.78:0/3371749271 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcca8067d00 con 0x7fcca80b0d20 2023-10-11T09:51:52.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.204+0000 7fcca57fa700 1 -- 172.21.15.78:0/3371749271 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fcc9000cba0 con 0x7fcca80b0d20 2023-10-11T09:51:52.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.208+0000 7fcca57fa700 1 --2- 172.21.15.78:0/3371749271 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fcc88065ed0 0x7fcc88068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:52.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.208+0000 7fcca57fa700 1 -- 172.21.15.78:0/3371749271 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcc90083bf0 con 0x7fcca80b0d20 2023-10-11T09:51:52.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.208+0000 7fccaced9700 1 --2- 172.21.15.78:0/3371749271 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fcc88065ed0 0x7fcc88068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:52.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.208+0000 7fccaced9700 1 --2- 172.21.15.78:0/3371749271 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fcc88065ed0 0x7fcc88068380 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fcc980012e0 tx=0x7fcc98001aa0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:52.218 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.212+0000 7fcca57fa700 1 -- 172.21.15.78:0/3371749271 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fcc900511a0 con 0x7fcca80b0d20 2023-10-11T09:51:52.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.384+0000 7fccadedb700 1 -- 172.21.15.78:0/3371749271 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fcca8065d40 con 0x7fcca80b0d20 2023-10-11T09:51:52.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.384+0000 7fcca57fa700 1 -- 172.21.15.78:0/3371749271 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v127) v1 ==== 92+0+10 (secure 0 0 0) 0x7fcc90021020 con 0x7fcca80b0d20 2023-10-11T09:51:52.392 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:51:52.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.388+0000 7fccadedb700 1 -- 172.21.15.78:0/3371749271 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fcc88065ed0 msgr2=0x7fcc88068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:52.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.388+0000 7fccadedb700 1 --2- 172.21.15.78:0/3371749271 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fcc88065ed0 0x7fcc88068380 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fcc980012e0 tx=0x7fcc98001aa0 comp rx=0 tx=0).stop 2023-10-11T09:51:52.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.388+0000 7fccadedb700 1 -- 172.21.15.78:0/3371749271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcca80b0d20 msgr2=0x7fcca80c65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:52.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.388+0000 7fccadedb700 1 --2- 172.21.15.78:0/3371749271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcca80b0d20 0x7fcca80c65e0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fcc9000e980 tx=0x7fcc9000ec90 comp rx=0 tx=0).stop 2023-10-11T09:51:52.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.388+0000 7fccadedb700 1 -- 172.21.15.78:0/3371749271 shutdown_connections 2023-10-11T09:51:52.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.388+0000 7fccadedb700 1 --2- 172.21.15.78:0/3371749271 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fcc88065ed0 0x7fcc88068380 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:52.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.388+0000 7fccadedb700 1 --2- 172.21.15.78:0/3371749271 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcca815a800 0x7fcca814ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:52.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.388+0000 7fccadedb700 1 --2- 172.21.15.78:0/3371749271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcca80b0d20 0x7fcca80c65e0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:52.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.388+0000 7fccadedb700 1 --2- 172.21.15.78:0/3371749271 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcca806bb60 0x7fcca81e5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:52.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.388+0000 7fccadedb700 1 -- 172.21.15.78:0/3371749271 >> 172.21.15.78:0/3371749271 conn(0x7fcca80cf820 msgr2=0x7fcca80cdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:52.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.388+0000 7fccadedb700 1 -- 172.21.15.78:0/3371749271 shutdown_connections 2023-10-11T09:51:52.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.388+0000 7fccadedb700 1 -- 172.21.15.78:0/3371749271 wait complete. 2023-10-11T09:51:52.417 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T09:51:52.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.588+0000 7fc61eeaa700 1 Processor -- start 2023-10-11T09:51:52.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.588+0000 7fc61eeaa700 1 -- start start 2023-10-11T09:51:52.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.592+0000 7fc61eeaa700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc61806c8a0 0x7fc6180cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:52.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.592+0000 7fc61eeaa700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fc6180cc230 0x7fc6180cc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:52.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.592+0000 7fc61eeaa700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6181648a0 con 0x7fc618160de0 2023-10-11T09:51:52.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.592+0000 7fc61eeaa700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6180cc6d0 con 0x7fc6180cc230 2023-10-11T09:51:52.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.592+0000 7fc61eeaa700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc618164b80 con 0x7fc61806c8a0 2023-10-11T09:51:52.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.592+0000 7fc61e6a9700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fc6180cc230 0x7fc6180cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:52.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.592+0000 7fc61e6a9700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fc6180cc230 0x7fc6180cc940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44200/0 (socket says 172.21.15.78:44200) 2023-10-11T09:51:52.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.592+0000 7fc61dea8700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fc618160de0 0x7fc61806c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:50254/0 (socket says 172.21.15.78:50254) 2023-10-11T09:51:52.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.592+0000 7fc61e6a9700 1 -- 172.21.15.78:0/1823671576 learned_addr learned my addr 172.21.15.78:0/1823671576 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:52.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.592+0000 7fc61d6a7700 1 --2- 172.21.15.78:0/1823671576 >> v2:172.21.15.133:3300/0 conn(0x7fc61806c8a0 0x7fc6180cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:52.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.592+0000 7fc61e6a9700 1 -- 172.21.15.78:0/1823671576 >> v2:172.21.15.133:3300/0 conn(0x7fc61806c8a0 msgr2=0x7fc6180cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:52.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.592+0000 7fc61e6a9700 1 --2- 172.21.15.78:0/1823671576 >> v2:172.21.15.133:3300/0 conn(0x7fc61806c8a0 0x7fc6180cbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:52.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.592+0000 7fc61cea6700 1 -- 172.21.15.78:0/1823671576 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3672389489 0 0) 0x7fc6181648a0 con 0x7fc618160de0 2023-10-11T09:51:52.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.592+0000 7fc61e6a9700 1 -- 172.21.15.78:0/1823671576 >> v1:172.21.15.78:6789/0 conn(0x7fc618160de0 legacy=0x7fc61806c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:52.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.592+0000 7fc61e6a9700 1 -- 172.21.15.78:0/1823671576 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc618164e60 con 0x7fc6180cc230 2023-10-11T09:51:52.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.592+0000 7fc61e6a9700 1 --2- 172.21.15.78:0/1823671576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6180cc230 0x7fc6180cc940 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fc61806dcb0 tx=0x7fc61000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d191e9f1bee7dd69 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:52.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.592+0000 7fc61cea6700 1 -- 172.21.15.78:0/1823671576 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61000e070 con 0x7fc6180cc230 2023-10-11T09:51:52.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.592+0000 7fc61cea6700 1 -- 172.21.15.78:0/1823671576 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc61000b910 con 0x7fc6180cc230 2023-10-11T09:51:52.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.592+0000 7fc61cea6700 1 -- 172.21.15.78:0/1823671576 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61000bcc0 con 0x7fc6180cc230 2023-10-11T09:51:52.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.592+0000 7fc61eeaa700 1 -- 172.21.15.78:0/1823671576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6180cc230 msgr2=0x7fc6180cc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:52.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.592+0000 7fc61eeaa700 1 --2- 172.21.15.78:0/1823671576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6180cc230 0x7fc6180cc940 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fc61806dcb0 tx=0x7fc61000b330 comp rx=0 tx=0).stop 2023-10-11T09:51:52.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.596+0000 7fc61eeaa700 1 -- 172.21.15.78:0/1823671576 shutdown_connections 2023-10-11T09:51:52.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.596+0000 7fc61eeaa700 1 --2- 172.21.15.78:0/1823671576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6180cc230 0x7fc6180cc940 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:52.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.596+0000 7fc61eeaa700 1 --2- 172.21.15.78:0/1823671576 >> v2:172.21.15.133:3300/0 conn(0x7fc61806c8a0 0x7fc6180cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:52.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.596+0000 7fc61eeaa700 1 -- 172.21.15.78:0/1823671576 >> 172.21.15.78:0/1823671576 conn(0x7fc618073100 msgr2=0x7fc6180734d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:52.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.596+0000 7fc61eeaa700 1 -- 172.21.15.78:0/1823671576 shutdown_connections 2023-10-11T09:51:52.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.596+0000 7fc61eeaa700 1 -- 172.21.15.78:0/1823671576 wait complete. 2023-10-11T09:51:52.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.596+0000 7fc61eeaa700 1 Processor -- start 2023-10-11T09:51:52.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.596+0000 7fc61eeaa700 1 -- start start 2023-10-11T09:51:52.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.600+0000 7fc61eeaa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc61806c8a0 0x7fc61815d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:52.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.600+0000 7fc61eeaa700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6180cc230 0x7fc6180c5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:52.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.600+0000 7fc61eeaa700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc618160de0 0x7fc618070050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:52.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.600+0000 7fc61eeaa700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6180c3d60 con 0x7fc61806c8a0 2023-10-11T09:51:52.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.600+0000 7fc61eeaa700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc61815d830 con 0x7fc6180cc230 2023-10-11T09:51:52.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.600+0000 7fc61eeaa700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6180c3fc0 con 0x7fc618160de0 2023-10-11T09:51:52.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.600+0000 7fc61dea8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc61806c8a0 0x7fc61815d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:52.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.600+0000 7fc61dea8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc61806c8a0 0x7fc61815d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44208/0 (socket says 172.21.15.78:44208) 2023-10-11T09:51:52.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.600+0000 7fc61dea8700 1 -- 172.21.15.78:0/489136844 learned_addr learned my addr 172.21.15.78:0/489136844 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:52.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.600+0000 7fc61d6a7700 1 --2- 172.21.15.78:0/489136844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6180cc230 0x7fc6180c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:52.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.600+0000 7fc61e6a9700 1 --2- 172.21.15.78:0/489136844 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc618160de0 0x7fc618070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:52.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.600+0000 7fc61dea8700 1 -- 172.21.15.78:0/489136844 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc618160de0 msgr2=0x7fc618070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:52.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.600+0000 7fc61dea8700 1 --2- 172.21.15.78:0/489136844 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc618160de0 0x7fc618070050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:52.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.600+0000 7fc61dea8700 1 -- 172.21.15.78:0/489136844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6180cc230 msgr2=0x7fc6180c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:52.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.600+0000 7fc61dea8700 1 --2- 172.21.15.78:0/489136844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6180cc230 0x7fc6180c5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:52.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.600+0000 7fc61dea8700 1 -- 172.21.15.78:0/489136844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc610009b30 con 0x7fc61806c8a0 2023-10-11T09:51:52.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.600+0000 7fc61e6a9700 1 --2- 172.21.15.78:0/489136844 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc618160de0 0x7fc618070050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:51:52.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.600+0000 7fc61dea8700 1 --2- 172.21.15.78:0/489136844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc61806c8a0 0x7fc61815d2f0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fc61400c580 tx=0x7fc61400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:52.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.600+0000 7fc60e7fc700 1 -- 172.21.15.78:0/489136844 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc614004ca0 con 0x7fc61806c8a0 2023-10-11T09:51:52.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.600+0000 7fc60e7fc700 1 -- 172.21.15.78:0/489136844 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc614004e20 con 0x7fc61806c8a0 2023-10-11T09:51:52.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.600+0000 7fc61eeaa700 1 -- 172.21.15.78:0/489136844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6180c42a0 con 0x7fc61806c8a0 2023-10-11T09:51:52.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.600+0000 7fc60e7fc700 1 -- 172.21.15.78:0/489136844 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc614007310 con 0x7fc61806c8a0 2023-10-11T09:51:52.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.600+0000 7fc61eeaa700 1 -- 172.21.15.78:0/489136844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6180c4860 con 0x7fc61806c8a0 2023-10-11T09:51:52.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.604+0000 7fc61eeaa700 1 -- 172.21.15.78:0/489136844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6180cb3e0 con 0x7fc61806c8a0 2023-10-11T09:51:52.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.604+0000 7fc60e7fc700 1 -- 172.21.15.78:0/489136844 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fc614007470 con 0x7fc61806c8a0 2023-10-11T09:51:52.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.604+0000 7fc60e7fc700 1 --2- 172.21.15.78:0/489136844 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc5fc065f50 0x7fc5fc068400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:52.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.604+0000 7fc60e7fc700 1 -- 172.21.15.78:0/489136844 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc614083520 con 0x7fc61806c8a0 2023-10-11T09:51:52.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.604+0000 7fc61d6a7700 1 --2- 172.21.15.78:0/489136844 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc5fc065f50 0x7fc5fc068400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:52.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.608+0000 7fc61d6a7700 1 --2- 172.21.15.78:0/489136844 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc5fc065f50 0x7fc5fc068400 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc6180b4b50 tx=0x7fc608008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:52.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.612+0000 7fc60e7fc700 1 -- 172.21.15.78:0/489136844 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fc6180cb3e0 con 0x7fc61806c8a0 2023-10-11T09:51:52.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.784+0000 7fc61eeaa700 1 -- 172.21.15.78:0/489136844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc6180cb3e0 con 0x7fc61806c8a0 2023-10-11T09:51:52.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.788+0000 7fc60e7fc700 1 -- 172.21.15.78:0/489136844 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v127) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc6180cb3e0 con 0x7fc61806c8a0 2023-10-11T09:51:52.793 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:51:52.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.788+0000 7fc61eeaa700 1 -- 172.21.15.78:0/489136844 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc5fc065f50 msgr2=0x7fc5fc068400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:52.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.788+0000 7fc61eeaa700 1 --2- 172.21.15.78:0/489136844 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc5fc065f50 0x7fc5fc068400 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc6180b4b50 tx=0x7fc608008040 comp rx=0 tx=0).stop 2023-10-11T09:51:52.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.788+0000 7fc61eeaa700 1 -- 172.21.15.78:0/489136844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc61806c8a0 msgr2=0x7fc61815d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:52.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.788+0000 7fc61eeaa700 1 --2- 172.21.15.78:0/489136844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc61806c8a0 0x7fc61815d2f0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fc61400c580 tx=0x7fc61400c940 comp rx=0 tx=0).stop 2023-10-11T09:51:52.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.788+0000 7fc61eeaa700 1 -- 172.21.15.78:0/489136844 shutdown_connections 2023-10-11T09:51:52.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.788+0000 7fc61eeaa700 1 --2- 172.21.15.78:0/489136844 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc5fc065f50 0x7fc5fc068400 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:52.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.788+0000 7fc61eeaa700 1 --2- 172.21.15.78:0/489136844 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc618160de0 0x7fc618070050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:52.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.788+0000 7fc61eeaa700 1 --2- 172.21.15.78:0/489136844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6180cc230 0x7fc6180c5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:52.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.788+0000 7fc61eeaa700 1 --2- 172.21.15.78:0/489136844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc61806c8a0 0x7fc61815d2f0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:52.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.788+0000 7fc61eeaa700 1 -- 172.21.15.78:0/489136844 >> 172.21.15.78:0/489136844 conn(0x7fc618073100 msgr2=0x7fc6180cd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:52.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.788+0000 7fc61eeaa700 1 -- 172.21.15.78:0/489136844 shutdown_connections 2023-10-11T09:51:52.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.788+0000 7fc61eeaa700 1 -- 172.21.15.78:0/489136844 wait complete. 2023-10-11T09:51:52.815 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T09:51:52.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.984+0000 7f4fc9b30700 1 Processor -- start 2023-10-11T09:51:52.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.984+0000 7f4fc9b30700 1 -- start start 2023-10-11T09:51:52.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.988+0000 7f4fc9b30700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4fc40b4530 0x7f4fc40d0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:52.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.988+0000 7f4fc9b30700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f4fc40d1520 0x7f4fc40b64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:52.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.988+0000 7f4fc9b30700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fc409c960 con 0x7f4fc40b69f0 2023-10-11T09:51:52.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.988+0000 7f4fc9b30700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fc40d19a0 con 0x7f4fc40d1520 2023-10-11T09:51:52.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.988+0000 7f4fc9b30700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fc415d180 con 0x7f4fc40b4530 2023-10-11T09:51:52.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.988+0000 7f4fc3fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f4fc40d1520 0x7f4fc40b64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:52.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.988+0000 7f4fc932f700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f4fc40b69f0 0x7f4fc40b6e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:50270/0 (socket says 172.21.15.78:50270) 2023-10-11T09:51:52.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.988+0000 7f4fc932f700 1 -- 172.21.15.78:0/4223614125 learned_addr learned my addr 172.21.15.78:0/4223614125 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:51:52.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.988+0000 7f4fc8b2e700 1 --2- 172.21.15.78:0/4223614125 >> v2:172.21.15.133:3300/0 conn(0x7f4fc40b4530 0x7f4fc40d0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:52.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.988+0000 7f4fc3fff700 1 -- 172.21.15.78:0/4223614125 >> v2:172.21.15.133:3300/0 conn(0x7f4fc40b4530 msgr2=0x7f4fc40d0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:52.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.988+0000 7f4fc3fff700 1 --2- 172.21.15.78:0/4223614125 >> v2:172.21.15.133:3300/0 conn(0x7f4fc40b4530 0x7f4fc40d0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:52.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.988+0000 7f4fc3fff700 1 -- 172.21.15.78:0/4223614125 >> v1:172.21.15.78:6789/0 conn(0x7f4fc40b69f0 legacy=0x7f4fc40b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:52.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.988+0000 7f4fc37fe700 1 -- 172.21.15.78:0/4223614125 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 456295321 0 0) 0x7f4fc409c960 con 0x7f4fc40b69f0 2023-10-11T09:51:52.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.988+0000 7f4fc3fff700 1 -- 172.21.15.78:0/4223614125 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc40b5fd0 con 0x7f4fc40d1520 2023-10-11T09:51:52.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.992+0000 7f4fc3fff700 1 --2- 172.21.15.78:0/4223614125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fc40d1520 0x7f4fc40b64b0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4066040 tx=0x7f4fb0009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e63b6adb74def43f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:52.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.992+0000 7f4fc37fe700 1 -- 172.21.15.78:0/4223614125 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0011830 con 0x7f4fc40d1520 2023-10-11T09:51:52.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.992+0000 7f4fc37fe700 1 -- 172.21.15.78:0/4223614125 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb0011990 con 0x7f4fc40d1520 2023-10-11T09:51:53.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.992+0000 7f4fc37fe700 1 -- 172.21.15.78:0/4223614125 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0011d20 con 0x7f4fc40d1520 2023-10-11T09:51:53.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.992+0000 7f4fc9b30700 1 -- 172.21.15.78:0/4223614125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fc40d1520 msgr2=0x7f4fc40b64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:53.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.992+0000 7f4fc9b30700 1 --2- 172.21.15.78:0/4223614125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fc40d1520 0x7f4fc40b64b0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4066040 tx=0x7f4fb0009b30 comp rx=0 tx=0).stop 2023-10-11T09:51:53.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.992+0000 7f4fc9b30700 1 -- 172.21.15.78:0/4223614125 shutdown_connections 2023-10-11T09:51:53.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.992+0000 7f4fc9b30700 1 --2- 172.21.15.78:0/4223614125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fc40d1520 0x7f4fc40b64b0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:53.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.992+0000 7f4fc9b30700 1 --2- 172.21.15.78:0/4223614125 >> v2:172.21.15.133:3300/0 conn(0x7f4fc40b4530 0x7f4fc40d0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:53.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.992+0000 7f4fc9b30700 1 -- 172.21.15.78:0/4223614125 >> 172.21.15.78:0/4223614125 conn(0x7f4fc409c3d0 msgr2=0x7f4fc40d30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:53.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.992+0000 7f4fc9b30700 1 -- 172.21.15.78:0/4223614125 shutdown_connections 2023-10-11T09:51:53.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.992+0000 7f4fc9b30700 1 -- 172.21.15.78:0/4223614125 wait complete. 2023-10-11T09:51:53.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.996+0000 7f4fc9b30700 1 Processor -- start 2023-10-11T09:51:53.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.996+0000 7f4fc9b30700 1 -- start start 2023-10-11T09:51:53.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.996+0000 7f4fc9b30700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fc40b4530 0x7f4fc40a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:53.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.996+0000 7f4fc9b30700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fc40b69f0 0x7f4fc4150520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:53.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.996+0000 7f4fc9b30700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fc40d1520 0x7f4fc40d30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:53.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.996+0000 7f4fc9b30700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc4150100 con 0x7f4fc40d1520 2023-10-11T09:51:53.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.996+0000 7f4fc9b30700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc414f480 con 0x7f4fc40b69f0 2023-10-11T09:51:53.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.996+0000 7f4fc9b30700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc414f760 con 0x7f4fc40b4530 2023-10-11T09:51:53.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.996+0000 7f4fc932f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fc40d1520 0x7f4fc40d30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:53.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.996+0000 7f4fc932f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fc40d1520 0x7f4fc40d30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44228/0 (socket says 172.21.15.78:44228) 2023-10-11T09:51:53.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:52.996+0000 7f4fc932f700 1 -- 172.21.15.78:0/1033588996 learned_addr learned my addr 172.21.15.78:0/1033588996 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:53.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.000+0000 7f4fc8b2e700 1 --2- 172.21.15.78:0/1033588996 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fc40b4530 0x7f4fc40a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:53.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.000+0000 7f4fc3fff700 1 --2- 172.21.15.78:0/1033588996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fc40b69f0 0x7f4fc4150520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:53.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.000+0000 7f4fc932f700 1 -- 172.21.15.78:0/1033588996 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fc40b4530 msgr2=0x7f4fc40a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:53.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.000+0000 7f4fc932f700 1 --2- 172.21.15.78:0/1033588996 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fc40b4530 0x7f4fc40a0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:53.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.000+0000 7f4fc932f700 1 -- 172.21.15.78:0/1033588996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fc40b69f0 msgr2=0x7f4fc4150520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:53.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.000+0000 7f4fc932f700 1 --2- 172.21.15.78:0/1033588996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fc40b69f0 0x7f4fc4150520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:53.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.000+0000 7f4fc932f700 1 -- 172.21.15.78:0/1033588996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb00094b0 con 0x7f4fc40d1520 2023-10-11T09:51:53.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.000+0000 7f4fc932f700 1 --2- 172.21.15.78:0/1033588996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fc40d1520 0x7f4fc40d30c0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc00cca0 tx=0x7f4fbc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:53.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.000+0000 7f4fc17fa700 1 -- 172.21.15.78:0/1033588996 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc00cea0 con 0x7f4fc40d1520 2023-10-11T09:51:53.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.000+0000 7f4fc17fa700 1 -- 172.21.15.78:0/1033588996 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fbc00ade0 con 0x7f4fc40d1520 2023-10-11T09:51:53.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.000+0000 7f4fc9b30700 1 -- 172.21.15.78:0/1033588996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc414fa40 con 0x7f4fc40d1520 2023-10-11T09:51:53.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.000+0000 7f4fc17fa700 1 -- 172.21.15.78:0/1033588996 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc007310 con 0x7f4fc40d1520 2023-10-11T09:51:53.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.000+0000 7f4fc9b30700 1 -- 172.21.15.78:0/1033588996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc40a0d70 con 0x7f4fc40d1520 2023-10-11T09:51:53.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.000+0000 7f4fc9b30700 1 -- 172.21.15.78:0/1033588996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc40d0b00 con 0x7f4fc40d1520 2023-10-11T09:51:53.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.004+0000 7f4fc17fa700 1 -- 172.21.15.78:0/1033588996 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f4fbc007470 con 0x7f4fc40d1520 2023-10-11T09:51:53.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.004+0000 7f4fc17fa700 1 --2- 172.21.15.78:0/1033588996 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4fa8065ed0 0x7f4fa8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:53.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.004+0000 7f4fc17fa700 1 -- 172.21.15.78:0/1033588996 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4fbc083430 con 0x7f4fc40d1520 2023-10-11T09:51:53.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.004+0000 7f4fc8b2e700 1 --2- 172.21.15.78:0/1033588996 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4fa8065ed0 0x7f4fa8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:53.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.004+0000 7f4fc8b2e700 1 --2- 172.21.15.78:0/1033588996 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4fa8065ed0 0x7f4fa8068380 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8009b70 tx=0x7f4fb8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:53.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.008+0000 7f4fc17fa700 1 -- 172.21.15.78:0/1033588996 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f4fbc050af0 con 0x7f4fc40d1520 2023-10-11T09:51:53.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.180+0000 7f4fc9b30700 1 -- 172.21.15.78:0/1033588996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4fc4150240 con 0x7f4fc40d1520 2023-10-11T09:51:53.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.180+0000 7f4fc17fa700 1 -- 172.21.15.78:0/1033588996 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v127) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4fc4150240 con 0x7f4fc40d1520 2023-10-11T09:51:53.187 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2023-10-11T09:51:53.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.184+0000 7f4fc9b30700 1 -- 172.21.15.78:0/1033588996 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4fa8065ed0 msgr2=0x7f4fa8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:53.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.184+0000 7f4fc9b30700 1 --2- 172.21.15.78:0/1033588996 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4fa8065ed0 0x7f4fa8068380 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8009b70 tx=0x7f4fb8008040 comp rx=0 tx=0).stop 2023-10-11T09:51:53.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.184+0000 7f4fc9b30700 1 -- 172.21.15.78:0/1033588996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fc40d1520 msgr2=0x7f4fc40d30c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:53.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.184+0000 7f4fc9b30700 1 --2- 172.21.15.78:0/1033588996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fc40d1520 0x7f4fc40d30c0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc00cca0 tx=0x7f4fbc00a340 comp rx=0 tx=0).stop 2023-10-11T09:51:53.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.184+0000 7f4fc9b30700 1 -- 172.21.15.78:0/1033588996 shutdown_connections 2023-10-11T09:51:53.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.184+0000 7f4fc9b30700 1 --2- 172.21.15.78:0/1033588996 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4fa8065ed0 0x7f4fa8068380 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:53.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.184+0000 7f4fc9b30700 1 --2- 172.21.15.78:0/1033588996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fc40d1520 0x7f4fc40d30c0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:53.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.184+0000 7f4fc9b30700 1 --2- 172.21.15.78:0/1033588996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fc40b69f0 0x7f4fc4150520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:53.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.184+0000 7f4fc9b30700 1 --2- 172.21.15.78:0/1033588996 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fc40b4530 0x7f4fc40a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:53.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.184+0000 7f4fc9b30700 1 -- 172.21.15.78:0/1033588996 >> 172.21.15.78:0/1033588996 conn(0x7f4fc409c3d0 msgr2=0x7f4fc415f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:53.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.184+0000 7f4fc9b30700 1 -- 172.21.15.78:0/1033588996 shutdown_connections 2023-10-11T09:51:53.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.184+0000 7f4fc9b30700 1 -- 172.21.15.78:0/1033588996 wait complete. 2023-10-11T09:51:53.213 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T09:51:53.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.372+0000 7f427ebfb700 1 Processor -- start 2023-10-11T09:51:53.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.376+0000 7f427ebfb700 1 -- start start 2023-10-11T09:51:53.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.376+0000 7f427ebfb700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f42780b0d20 0x7f42780b4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:53.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.376+0000 7f427ebfb700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f427806bb60 0x7f427815e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:53.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.376+0000 7f427ebfb700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4278070dc0 con 0x7f427815a800 2023-10-11T09:51:53.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.376+0000 7f427ebfb700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42780b46a0 con 0x7f42780b0d20 2023-10-11T09:51:53.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.376+0000 7f427ebfb700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4278070ff0 con 0x7f427806bb60 2023-10-11T09:51:53.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.376+0000 7f427d3f8700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f42780b0d20 0x7f42780b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:53.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.376+0000 7f427d3f8700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f42780b0d20 0x7f42780b4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44238/0 (socket says 172.21.15.78:44238) 2023-10-11T09:51:53.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.376+0000 7f427d3f8700 1 -- 172.21.15.78:0/2940136866 learned_addr learned my addr 172.21.15.78:0/2940136866 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:53.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.376+0000 7f427e3fa700 1 --2- 172.21.15.78:0/2940136866 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f427806bb60 0x7f427815e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:53.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.376+0000 7f427d3f8700 1 -- 172.21.15.78:0/2940136866 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f427806bb60 msgr2=0x7f427815e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:53.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.376+0000 7f427d3f8700 1 --2- 172.21.15.78:0/2940136866 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f427806bb60 0x7f427815e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:53.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.376+0000 7f427d3f8700 1 -- 172.21.15.78:0/2940136866 >> v1:172.21.15.133:6789/0 conn(0x7f427815a800 legacy=0x7f42780b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:53.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.376+0000 7f427d3f8700 1 -- 172.21.15.78:0/2940136866 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42780712d0 con 0x7f42780b0d20 2023-10-11T09:51:53.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.380+0000 7f427d3f8700 1 --2- 172.21.15.78:0/2940136866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42780b0d20 0x7f42780b4160 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f4268009900 tx=0x7f4268009c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a21342d1b4c5387f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:53.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.380+0000 7f427cbf7700 1 -- 172.21.15.78:0/2940136866 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426800ba60 con 0x7f42780b0d20 2023-10-11T09:51:53.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.380+0000 7f427cbf7700 1 -- 172.21.15.78:0/2940136866 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f426800bbc0 con 0x7f42780b0d20 2023-10-11T09:51:53.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.380+0000 7f427cbf7700 1 -- 172.21.15.78:0/2940136866 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4268009e50 con 0x7f42780b0d20 2023-10-11T09:51:53.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.380+0000 7f427ebfb700 1 -- 172.21.15.78:0/2940136866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42780b0d20 msgr2=0x7f42780b4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:53.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.380+0000 7f427ebfb700 1 --2- 172.21.15.78:0/2940136866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42780b0d20 0x7f42780b4160 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f4268009900 tx=0x7f4268009c10 comp rx=0 tx=0).stop 2023-10-11T09:51:53.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.380+0000 7f427ebfb700 1 -- 172.21.15.78:0/2940136866 shutdown_connections 2023-10-11T09:51:53.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.380+0000 7f427ebfb700 1 --2- 172.21.15.78:0/2940136866 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f427806bb60 0x7f427815e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:53.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.380+0000 7f427ebfb700 1 --2- 172.21.15.78:0/2940136866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42780b0d20 0x7f42780b4160 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:53.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.380+0000 7f427ebfb700 1 -- 172.21.15.78:0/2940136866 >> 172.21.15.78:0/2940136866 conn(0x7f42780cf820 msgr2=0x7f42780cb610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:53.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.380+0000 7f427ebfb700 1 -- 172.21.15.78:0/2940136866 shutdown_connections 2023-10-11T09:51:53.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.380+0000 7f427ebfb700 1 -- 172.21.15.78:0/2940136866 wait complete. 2023-10-11T09:51:53.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.384+0000 7f427ebfb700 1 Processor -- start 2023-10-11T09:51:53.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.384+0000 7f427ebfb700 1 -- start start 2023-10-11T09:51:53.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.384+0000 7f427ebfb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f427806bb60 0x7f42780c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:53.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.384+0000 7f427ebfb700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42780b0d20 0x7f42781ef500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:53.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.384+0000 7f427ebfb700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f427815a800 0x7f42781efa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:53.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.384+0000 7f427ebfb700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42780c9370 con 0x7f427806bb60 2023-10-11T09:51:53.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.384+0000 7f427ebfb700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42780c9650 con 0x7f427815a800 2023-10-11T09:51:53.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.384+0000 7f427ebfb700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42780c9930 con 0x7f42780b0d20 2023-10-11T09:51:53.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.384+0000 7f427dbf9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f427806bb60 0x7f42780c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:53.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.384+0000 7f427dbf9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f427806bb60 0x7f42780c4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44246/0 (socket says 172.21.15.78:44246) 2023-10-11T09:51:53.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.384+0000 7f427dbf9700 1 -- 172.21.15.78:0/967130566 learned_addr learned my addr 172.21.15.78:0/967130566 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.384+0000 7f427d3f8700 1 --2- 172.21.15.78:0/967130566 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42780b0d20 0x7f42781ef500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.384+0000 7f427e3fa700 1 --2- 172.21.15.78:0/967130566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f427815a800 0x7f42781efa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.384+0000 7f427dbf9700 1 -- 172.21.15.78:0/967130566 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42780b0d20 msgr2=0x7f42781ef500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.384+0000 7f427dbf9700 1 --2- 172.21.15.78:0/967130566 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42780b0d20 0x7f42781ef500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.384+0000 7f427dbf9700 1 -- 172.21.15.78:0/967130566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f427815a800 msgr2=0x7f42781efa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.384+0000 7f427dbf9700 1 --2- 172.21.15.78:0/967130566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f427815a800 0x7f42781efa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.384+0000 7f427dbf9700 1 -- 172.21.15.78:0/967130566 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4268009510 con 0x7f427806bb60 2023-10-11T09:51:53.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.388+0000 7f427dbf9700 1 --2- 172.21.15.78:0/967130566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f427806bb60 0x7f42780c4f60 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f427400c500 tx=0x7f427400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:53.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.388+0000 7f426e7fc700 1 -- 172.21.15.78:0/967130566 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42740052d0 con 0x7f427806bb60 2023-10-11T09:51:53.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.388+0000 7f426e7fc700 1 -- 172.21.15.78:0/967130566 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4274005430 con 0x7f427806bb60 2023-10-11T09:51:53.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.388+0000 7f427ebfb700 1 -- 172.21.15.78:0/967130566 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42780c9c10 con 0x7f427806bb60 2023-10-11T09:51:53.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.388+0000 7f426e7fc700 1 -- 172.21.15.78:0/967130566 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42740055a0 con 0x7f427806bb60 2023-10-11T09:51:53.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.388+0000 7f427ebfb700 1 -- 172.21.15.78:0/967130566 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42780c5680 con 0x7f427806bb60 2023-10-11T09:51:53.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.388+0000 7f427ebfb700 1 -- 172.21.15.78:0/967130566 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4278067d00 con 0x7f427806bb60 2023-10-11T09:51:53.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.392+0000 7f426e7fc700 1 -- 172.21.15.78:0/967130566 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f42740038e0 con 0x7f427806bb60 2023-10-11T09:51:53.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.392+0000 7f426e7fc700 1 --2- 172.21.15.78:0/967130566 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f425c065ed0 0x7f425c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:53.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.392+0000 7f426e7fc700 1 -- 172.21.15.78:0/967130566 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4274082bd0 con 0x7f427806bb60 2023-10-11T09:51:53.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.392+0000 7f427d3f8700 1 --2- 172.21.15.78:0/967130566 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f425c065ed0 0x7f425c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:53.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.392+0000 7f427d3f8700 1 --2- 172.21.15.78:0/967130566 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f425c065ed0 0x7f425c068380 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f42780b1700 tx=0x7f4268008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:53.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.400+0000 7f426e7fc700 1 -- 172.21.15.78:0/967130566 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f4274050100 con 0x7f427806bb60 2023-10-11T09:51:53.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.588+0000 7f427ebfb700 1 -- 172.21.15.78:0/967130566 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4278065d40 con 0x7f427806bb60 2023-10-11T09:51:53.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.588+0000 7f426e7fc700 1 -- 172.21.15.78:0/967130566 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v127) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4274053720 con 0x7f427806bb60 2023-10-11T09:51:53.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.592+0000 7f427ebfb700 1 -- 172.21.15.78:0/967130566 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f425c065ed0 msgr2=0x7f425c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:53.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.592+0000 7f427ebfb700 1 --2- 172.21.15.78:0/967130566 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f425c065ed0 0x7f425c068380 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f42780b1700 tx=0x7f4268008030 comp rx=0 tx=0).stop 2023-10-11T09:51:53.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.592+0000 7f427ebfb700 1 -- 172.21.15.78:0/967130566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f427806bb60 msgr2=0x7f42780c4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:53.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.592+0000 7f427ebfb700 1 --2- 172.21.15.78:0/967130566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f427806bb60 0x7f42780c4f60 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f427400c500 tx=0x7f427400c8c0 comp rx=0 tx=0).stop 2023-10-11T09:51:53.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.592+0000 7f427ebfb700 1 -- 172.21.15.78:0/967130566 shutdown_connections 2023-10-11T09:51:53.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.592+0000 7f427ebfb700 1 --2- 172.21.15.78:0/967130566 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f425c065ed0 0x7f425c068380 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:53.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.592+0000 7f427ebfb700 1 --2- 172.21.15.78:0/967130566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f427815a800 0x7f42781efa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:53.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.592+0000 7f427ebfb700 1 --2- 172.21.15.78:0/967130566 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42780b0d20 0x7f42781ef500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:53.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.592+0000 7f427ebfb700 1 --2- 172.21.15.78:0/967130566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f427806bb60 0x7f42780c4f60 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:53.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.592+0000 7f427ebfb700 1 -- 172.21.15.78:0/967130566 >> 172.21.15.78:0/967130566 conn(0x7f42780cf820 msgr2=0x7f427815b510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:53.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.592+0000 7f427ebfb700 1 -- 172.21.15.78:0/967130566 shutdown_connections 2023-10-11T09:51:53.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.592+0000 7f427ebfb700 1 -- 172.21.15.78:0/967130566 wait complete. 2023-10-11T09:51:53.600 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2023-10-11T09:51:53.619 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:51:53.619 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:51:53.637 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:51:53.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.844+0000 7fae5759e700 1 Processor -- start 2023-10-11T09:51:53.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.844+0000 7fae5759e700 1 -- start start 2023-10-11T09:51:53.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.848+0000 7fae5759e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae5806bb60 0x7fae580b4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:53.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.848+0000 7fae5759e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae58070dc0 con 0x7fae5815a800 2023-10-11T09:51:53.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.848+0000 7fae5759e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae58070ff0 con 0x7fae580b0d20 2023-10-11T09:51:53.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.848+0000 7fae5759e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae580b46a0 con 0x7fae5806bb60 2023-10-11T09:51:53.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.848+0000 7fae5659c700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fae5815a800 0x7fae580b0610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:50286/0 (socket says 172.21.15.78:50286) 2023-10-11T09:51:53.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.848+0000 7fae5659c700 1 -- 172.21.15.78:0/3277171807 learned_addr learned my addr 172.21.15.78:0/3277171807 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:51:53.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.848+0000 7fae56d9d700 1 --2- 172.21.15.78:0/3277171807 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae5806bb60 0x7fae580b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:53.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.848+0000 7fae55d9b700 1 -- 172.21.15.78:0/3277171807 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3132514962 0 0) 0x7fae58070dc0 con 0x7fae5815a800 2023-10-11T09:51:53.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.848+0000 7fae55d9b700 1 -- 172.21.15.78:0/3277171807 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae440035d0 con 0x7fae5815a800 2023-10-11T09:51:53.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.848+0000 7fae55d9b700 1 -- 172.21.15.78:0/3277171807 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3862486336 0 0) 0x7fae58070ff0 con 0x7fae580b0d20 2023-10-11T09:51:53.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.848+0000 7fae55d9b700 1 -- 172.21.15.78:0/3277171807 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae58070dc0 con 0x7fae580b0d20 2023-10-11T09:51:53.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.848+0000 7fae56d9d700 1 -- 172.21.15.78:0/3277171807 >> v1:172.21.15.133:6789/0 conn(0x7fae580b0d20 legacy=0x7fae5815e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:53.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.848+0000 7fae56d9d700 1 -- 172.21.15.78:0/3277171807 >> v1:172.21.15.78:6789/0 conn(0x7fae5815a800 legacy=0x7fae580b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:53.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.848+0000 7fae56d9d700 1 -- 172.21.15.78:0/3277171807 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae580712d0 con 0x7fae5806bb60 2023-10-11T09:51:53.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.848+0000 7fae56d9d700 1 --2- 172.21.15.78:0/3277171807 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae5806bb60 0x7fae580b4160 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fae50009dc0 tx=0x7fae5000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6035de26e041958f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:53.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.852+0000 7fae55d9b700 1 -- 172.21.15.78:0/3277171807 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5000e070 con 0x7fae5806bb60 2023-10-11T09:51:53.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.852+0000 7fae55d9b700 1 -- 172.21.15.78:0/3277171807 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae5000bb20 con 0x7fae5806bb60 2023-10-11T09:51:53.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.852+0000 7fae55d9b700 1 -- 172.21.15.78:0/3277171807 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae50012430 con 0x7fae5806bb60 2023-10-11T09:51:53.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.852+0000 7fae5759e700 1 -- 172.21.15.78:0/3277171807 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae5806bb60 msgr2=0x7fae580b4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:53.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.852+0000 7fae5759e700 1 --2- 172.21.15.78:0/3277171807 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae5806bb60 0x7fae580b4160 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fae50009dc0 tx=0x7fae5000b330 comp rx=0 tx=0).stop 2023-10-11T09:51:53.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.852+0000 7fae5759e700 1 -- 172.21.15.78:0/3277171807 shutdown_connections 2023-10-11T09:51:53.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.852+0000 7fae5759e700 1 --2- 172.21.15.78:0/3277171807 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae5806bb60 0x7fae580b4160 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:53.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.852+0000 7fae5759e700 1 -- 172.21.15.78:0/3277171807 >> 172.21.15.78:0/3277171807 conn(0x7fae580cf820 msgr2=0x7fae580cb610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:53.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.852+0000 7fae5759e700 1 -- 172.21.15.78:0/3277171807 shutdown_connections 2023-10-11T09:51:53.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.852+0000 7fae5759e700 1 -- 172.21.15.78:0/3277171807 wait complete. 2023-10-11T09:51:53.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.856+0000 7fae5759e700 1 Processor -- start 2023-10-11T09:51:53.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.856+0000 7fae5759e700 1 -- start start 2023-10-11T09:51:53.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.856+0000 7fae5759e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae5806bb60 0x7fae58153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:53.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.856+0000 7fae5759e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae580b0d20 0x7fae581f5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:53.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.856+0000 7fae5759e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae5815a800 0x7fae580cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:53.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.856+0000 7fae5759e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae580cc6d0 con 0x7fae5806bb60 2023-10-11T09:51:53.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.856+0000 7fae5759e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae581f6160 con 0x7fae580b0d20 2023-10-11T09:51:53.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.856+0000 7fae5759e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae580cc920 con 0x7fae5815a800 2023-10-11T09:51:53.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.856+0000 7fae5659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae5806bb60 0x7fae58153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:53.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.856+0000 7fae5659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae5806bb60 0x7fae58153e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44260/0 (socket says 172.21.15.78:44260) 2023-10-11T09:51:53.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.856+0000 7fae5659c700 1 -- 172.21.15.78:0/1728843965 learned_addr learned my addr 172.21.15.78:0/1728843965 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:53.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.856+0000 7fae56d9d700 1 --2- 172.21.15.78:0/1728843965 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae5815a800 0x7fae580cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:53.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.856+0000 7fae4dd9b700 1 --2- 172.21.15.78:0/1728843965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae580b0d20 0x7fae581f5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:53.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.860+0000 7fae5659c700 1 -- 172.21.15.78:0/1728843965 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae5815a800 msgr2=0x7fae580cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:53.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.860+0000 7fae5659c700 1 --2- 172.21.15.78:0/1728843965 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae5815a800 0x7fae580cd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:53.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.860+0000 7fae5659c700 1 -- 172.21.15.78:0/1728843965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae580b0d20 msgr2=0x7fae581f5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:53.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.860+0000 7fae5659c700 1 --2- 172.21.15.78:0/1728843965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae580b0d20 0x7fae581f5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:53.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.860+0000 7fae5659c700 1 -- 172.21.15.78:0/1728843965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae50009a70 con 0x7fae5806bb60 2023-10-11T09:51:53.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.860+0000 7fae4dd9b700 1 --2- 172.21.15.78:0/1728843965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae580b0d20 0x7fae581f5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:51:53.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.860+0000 7fae5659c700 1 --2- 172.21.15.78:0/1728843965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae5806bb60 0x7fae58153e00 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fae4800c560 tx=0x7fae4800c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:53.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.860+0000 7fae4f7fe700 1 -- 172.21.15.78:0/1728843965 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae48005d90 con 0x7fae5806bb60 2023-10-11T09:51:53.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.860+0000 7fae5759e700 1 -- 172.21.15.78:0/1728843965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae580ccc00 con 0x7fae5806bb60 2023-10-11T09:51:53.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.860+0000 7fae4f7fe700 1 -- 172.21.15.78:0/1728843965 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae480070d0 con 0x7fae5806bb60 2023-10-11T09:51:53.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.860+0000 7fae4f7fe700 1 -- 172.21.15.78:0/1728843965 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae48007460 con 0x7fae5806bb60 2023-10-11T09:51:53.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.860+0000 7fae5759e700 1 -- 172.21.15.78:0/1728843965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae580cd1c0 con 0x7fae5806bb60 2023-10-11T09:51:53.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.860+0000 7fae5759e700 1 -- 172.21.15.78:0/1728843965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae58067d00 con 0x7fae5806bb60 2023-10-11T09:51:53.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.864+0000 7fae4f7fe700 1 -- 172.21.15.78:0/1728843965 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fae480075c0 con 0x7fae5806bb60 2023-10-11T09:51:53.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.864+0000 7fae4f7fe700 1 --2- 172.21.15.78:0/1728843965 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fae38065ed0 0x7fae38068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:53.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.864+0000 7fae4f7fe700 1 -- 172.21.15.78:0/1728843965 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fae48082e40 con 0x7fae5806bb60 2023-10-11T09:51:53.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.864+0000 7fae4dd9b700 1 --2- 172.21.15.78:0/1728843965 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fae38065ed0 0x7fae38068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:53.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.864+0000 7fae4dd9b700 1 --2- 172.21.15.78:0/1728843965 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fae38065ed0 0x7fae38068380 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fae40001230 tx=0x7fae40009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:53.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:53.868+0000 7fae4f7fe700 1 -- 172.21.15.78:0/1728843965 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fae48050370 con 0x7fae5806bb60 2023-10-11T09:51:54.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.040+0000 7fae5759e700 1 -- 172.21.15.78:0/1728843965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae58065d40 con 0x7fae5806bb60 2023-10-11T09:51:54.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.044+0000 7fae4f7fe700 1 -- 172.21.15.78:0/1728843965 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+17893 (secure 0 0 0) 0x7fae48053990 con 0x7fae5806bb60 2023-10-11T09:51:54.050 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:51:54.051 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":127,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:51:41.218018+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:51:38.592892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-11T09:51:39.204588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:51:54.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.048+0000 7fae5759e700 1 -- 172.21.15.78:0/1728843965 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fae38065ed0 msgr2=0x7fae38068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:54.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.048+0000 7fae5759e700 1 --2- 172.21.15.78:0/1728843965 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fae38065ed0 0x7fae38068380 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fae40001230 tx=0x7fae40009040 comp rx=0 tx=0).stop 2023-10-11T09:51:54.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.048+0000 7fae5759e700 1 -- 172.21.15.78:0/1728843965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae5806bb60 msgr2=0x7fae58153e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:54.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.048+0000 7fae5759e700 1 --2- 172.21.15.78:0/1728843965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae5806bb60 0x7fae58153e00 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fae4800c560 tx=0x7fae4800c870 comp rx=0 tx=0).stop 2023-10-11T09:51:54.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.048+0000 7fae5759e700 1 -- 172.21.15.78:0/1728843965 shutdown_connections 2023-10-11T09:51:54.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.048+0000 7fae5759e700 1 --2- 172.21.15.78:0/1728843965 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fae38065ed0 0x7fae38068380 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:54.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.048+0000 7fae5759e700 1 --2- 172.21.15.78:0/1728843965 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae5815a800 0x7fae580cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:54.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.048+0000 7fae5759e700 1 --2- 172.21.15.78:0/1728843965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae580b0d20 0x7fae581f5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:54.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.048+0000 7fae5759e700 1 --2- 172.21.15.78:0/1728843965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae5806bb60 0x7fae58153e00 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:54.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.048+0000 7fae5759e700 1 -- 172.21.15.78:0/1728843965 >> 172.21.15.78:0/1728843965 conn(0x7fae580cf820 msgr2=0x7fae5806bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:54.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.048+0000 7fae5759e700 1 -- 172.21.15.78:0/1728843965 shutdown_connections 2023-10-11T09:51:54.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.048+0000 7fae5759e700 1 -- 172.21.15.78:0/1728843965 wait complete. 2023-10-11T09:51:54.074 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T09:51:38.592892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T09:51:39.204588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '127', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:51:54.075 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:51:54.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.244+0000 7fb00caca700 1 Processor -- start 2023-10-11T09:51:54.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.244+0000 7fb00caca700 1 -- start start 2023-10-11T09:51:54.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.248+0000 7fb00caca700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb008160de0 0x7fb00806c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:54.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.248+0000 7fb00caca700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0081648a0 con 0x7fb00806c6d0 2023-10-11T09:51:54.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.248+0000 7fb00caca700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb008164b80 con 0x7fb0080cc400 2023-10-11T09:51:54.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.248+0000 7fb00caca700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb00806cb20 con 0x7fb008160de0 2023-10-11T09:51:54.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.248+0000 7fb006ffd700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fb00806c6d0 0x7fb0080cbcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:50302/0 (socket says 172.21.15.78:50302) 2023-10-11T09:51:54.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.248+0000 7fb006ffd700 1 -- 172.21.15.78:0/721128538 learned_addr learned my addr 172.21.15.78:0/721128538 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:51:54.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.248+0000 7fb0077fe700 1 --2- 172.21.15.78:0/721128538 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb008160de0 0x7fb00806c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:54.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.248+0000 7fb0067fc700 1 -- 172.21.15.78:0/721128538 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 685342936 0 0) 0x7fb0081648a0 con 0x7fb00806c6d0 2023-10-11T09:51:54.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.248+0000 7fb0067fc700 1 -- 172.21.15.78:0/721128538 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faff00035d0 con 0x7fb00806c6d0 2023-10-11T09:51:54.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.248+0000 7fb0067fc700 1 -- 172.21.15.78:0/721128538 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 770748626 0 0) 0x7fb008164b80 con 0x7fb0080cc400 2023-10-11T09:51:54.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.248+0000 7fb0067fc700 1 -- 172.21.15.78:0/721128538 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0081648a0 con 0x7fb0080cc400 2023-10-11T09:51:54.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.248+0000 7fb0077fe700 1 -- 172.21.15.78:0/721128538 >> v1:172.21.15.133:6789/0 conn(0x7fb0080cc400 legacy=0x7fb0080cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:54.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.248+0000 7fb0077fe700 1 -- 172.21.15.78:0/721128538 >> v1:172.21.15.78:6789/0 conn(0x7fb00806c6d0 legacy=0x7fb0080cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:54.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.248+0000 7fb0077fe700 1 -- 172.21.15.78:0/721128538 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb008164e60 con 0x7fb008160de0 2023-10-11T09:51:54.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.252+0000 7fb0077fe700 1 --2- 172.21.15.78:0/721128538 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb008160de0 0x7fb00806c190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7faff8009880 tx=0x7faff8009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b2267fa9328f40c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:54.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.252+0000 7fb0067fc700 1 -- 172.21.15.78:0/721128538 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff8011830 con 0x7fb008160de0 2023-10-11T09:51:54.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.252+0000 7fb0067fc700 1 -- 172.21.15.78:0/721128538 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff8011990 con 0x7fb008160de0 2023-10-11T09:51:54.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.252+0000 7fb0067fc700 1 -- 172.21.15.78:0/721128538 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff8011d20 con 0x7fb008160de0 2023-10-11T09:51:54.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.252+0000 7fb00caca700 1 -- 172.21.15.78:0/721128538 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb008160de0 msgr2=0x7fb00806c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:54.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.252+0000 7fb00caca700 1 --2- 172.21.15.78:0/721128538 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb008160de0 0x7fb00806c190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7faff8009880 tx=0x7faff8009b90 comp rx=0 tx=0).stop 2023-10-11T09:51:54.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.252+0000 7fb00caca700 1 -- 172.21.15.78:0/721128538 shutdown_connections 2023-10-11T09:51:54.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.252+0000 7fb00caca700 1 --2- 172.21.15.78:0/721128538 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb008160de0 0x7fb00806c190 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:54.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.252+0000 7fb00caca700 1 -- 172.21.15.78:0/721128538 >> 172.21.15.78:0/721128538 conn(0x7fb008073100 msgr2=0x7fb0080734d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:54.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.252+0000 7fb00caca700 1 -- 172.21.15.78:0/721128538 shutdown_connections 2023-10-11T09:51:54.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.252+0000 7fb00caca700 1 -- 172.21.15.78:0/721128538 wait complete. 2023-10-11T09:51:54.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.256+0000 7fb00caca700 1 Processor -- start 2023-10-11T09:51:54.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.256+0000 7fb00caca700 1 -- start start 2023-10-11T09:51:54.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.256+0000 7fb00caca700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb00806c6d0 0x7fb0081530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:54.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.256+0000 7fb00caca700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0080cc400 0x7fb0080c2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:54.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.256+0000 7fb00caca700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb008160de0 0x7fb0080c2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:54.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.256+0000 7fb00caca700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00815db90 con 0x7fb0080cc400 2023-10-11T09:51:54.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.256+0000 7fb00caca700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0081622c0 con 0x7fb00806c6d0 2023-10-11T09:51:54.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.256+0000 7fb00caca700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0080b2710 con 0x7fb008160de0 2023-10-11T09:51:54.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.256+0000 7fb006ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0080cc400 0x7fb0080c2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:54.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.256+0000 7fb006ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0080cc400 0x7fb0080c2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44264/0 (socket says 172.21.15.78:44264) 2023-10-11T09:51:54.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.256+0000 7fb006ffd700 1 -- 172.21.15.78:0/1596589759 learned_addr learned my addr 172.21.15.78:0/1596589759 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:54.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.256+0000 7fb007fff700 1 --2- 172.21.15.78:0/1596589759 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb008160de0 0x7fb0080c2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:54.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.256+0000 7fb0077fe700 1 --2- 172.21.15.78:0/1596589759 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb00806c6d0 0x7fb0081530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:54.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.260+0000 7fb006ffd700 1 -- 172.21.15.78:0/1596589759 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb008160de0 msgr2=0x7fb0080c2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:54.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.260+0000 7fb006ffd700 1 --2- 172.21.15.78:0/1596589759 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb008160de0 0x7fb0080c2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:54.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.260+0000 7fb006ffd700 1 -- 172.21.15.78:0/1596589759 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb00806c6d0 msgr2=0x7fb0081530c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:54.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.260+0000 7fb006ffd700 1 --2- 172.21.15.78:0/1596589759 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb00806c6d0 0x7fb0081530c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:54.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.260+0000 7fb006ffd700 1 -- 172.21.15.78:0/1596589759 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff80094b0 con 0x7fb0080cc400 2023-10-11T09:51:54.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.260+0000 7fb006ffd700 1 --2- 172.21.15.78:0/1596589759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0080cc400 0x7fb0080c2780 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7faffc00c560 tx=0x7faffc00c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:54.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.260+0000 7fafeffff700 1 -- 172.21.15.78:0/1596589759 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc005d90 con 0x7fb0080cc400 2023-10-11T09:51:54.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.260+0000 7fafeffff700 1 -- 172.21.15.78:0/1596589759 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faffc0070d0 con 0x7fb0080cc400 2023-10-11T09:51:54.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.260+0000 7fb00caca700 1 -- 172.21.15.78:0/1596589759 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0080b29f0 con 0x7fb0080cc400 2023-10-11T09:51:54.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.260+0000 7fafeffff700 1 -- 172.21.15.78:0/1596589759 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc007460 con 0x7fb0080cc400 2023-10-11T09:51:54.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.260+0000 7fb00caca700 1 -- 172.21.15.78:0/1596589759 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb008162840 con 0x7fb0080cc400 2023-10-11T09:51:54.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.260+0000 7fb00caca700 1 -- 172.21.15.78:0/1596589759 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0080744c0 con 0x7fb0080cc400 2023-10-11T09:51:54.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.264+0000 7fafeffff700 1 -- 172.21.15.78:0/1596589759 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7faffc0075c0 con 0x7fb0080cc400 2023-10-11T09:51:54.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.264+0000 7fafeffff700 1 --2- 172.21.15.78:0/1596589759 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7faff4065ed0 0x7faff4068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:54.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.264+0000 7fafeffff700 1 -- 172.21.15.78:0/1596589759 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faffc082d30 con 0x7fb0080cc400 2023-10-11T09:51:54.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.264+0000 7fb0077fe700 1 --2- 172.21.15.78:0/1596589759 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7faff4065ed0 0x7faff4068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:54.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.264+0000 7fb0077fe700 1 --2- 172.21.15.78:0/1596589759 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7faff4065ed0 0x7faff4068380 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7faff80090b0 tx=0x7faff801c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:54.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.268+0000 7fafeffff700 1 -- 172.21.15.78:0/1596589759 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7faffc0502e0 con 0x7fb0080cc400 2023-10-11T09:51:54.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.440+0000 7fb00caca700 1 -- 172.21.15.78:0/1596589759 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb0080cb3e0 con 0x7fb0080cc400 2023-10-11T09:51:54.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.440+0000 7fafeffff700 1 -- 172.21.15.78:0/1596589759 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb0080cb3e0 con 0x7fb0080cc400 2023-10-11T09:51:54.449 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:51:54.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.444+0000 7fb00caca700 1 -- 172.21.15.78:0/1596589759 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7faff4065ed0 msgr2=0x7faff4068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:54.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.444+0000 7fb00caca700 1 --2- 172.21.15.78:0/1596589759 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7faff4065ed0 0x7faff4068380 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7faff80090b0 tx=0x7faff801c040 comp rx=0 tx=0).stop 2023-10-11T09:51:54.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.444+0000 7fb00caca700 1 -- 172.21.15.78:0/1596589759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0080cc400 msgr2=0x7fb0080c2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:54.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.444+0000 7fb00caca700 1 --2- 172.21.15.78:0/1596589759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0080cc400 0x7fb0080c2780 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7faffc00c560 tx=0x7faffc00c870 comp rx=0 tx=0).stop 2023-10-11T09:51:54.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.444+0000 7fb00caca700 1 -- 172.21.15.78:0/1596589759 shutdown_connections 2023-10-11T09:51:54.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.444+0000 7fb00caca700 1 --2- 172.21.15.78:0/1596589759 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7faff4065ed0 0x7faff4068380 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:54.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.444+0000 7fb00caca700 1 --2- 172.21.15.78:0/1596589759 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb008160de0 0x7fb0080c2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:54.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.444+0000 7fb00caca700 1 --2- 172.21.15.78:0/1596589759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0080cc400 0x7fb0080c2780 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:54.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.444+0000 7fb00caca700 1 --2- 172.21.15.78:0/1596589759 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb00806c6d0 0x7fb0081530c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:54.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.444+0000 7fb00caca700 1 -- 172.21.15.78:0/1596589759 >> 172.21.15.78:0/1596589759 conn(0x7fb008073100 msgr2=0x7fb0080cec00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:54.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.444+0000 7fb00caca700 1 -- 172.21.15.78:0/1596589759 shutdown_connections 2023-10-11T09:51:54.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.444+0000 7fb00caca700 1 -- 172.21.15.78:0/1596589759 wait complete. 2023-10-11T09:51:54.472 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:51:54.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.644+0000 7fae326b0700 1 Processor -- start 2023-10-11T09:51:54.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.644+0000 7fae326b0700 1 -- start start 2023-10-11T09:51:54.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.648+0000 7fae326b0700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fae2c0d16f0 0x7fae2c0cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:54.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.648+0000 7fae326b0700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae2c0b74c0 0x7fae2c0cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:54.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.648+0000 7fae326b0700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae2c09c960 con 0x7fae2c0b4530 2023-10-11T09:51:54.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.648+0000 7fae326b0700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae2c15d180 con 0x7fae2c0d16f0 2023-10-11T09:51:54.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.648+0000 7fae326b0700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c0b5fd0 con 0x7fae2c0b74c0 2023-10-11T09:51:54.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.648+0000 7fae316ae700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fae2c0b4530 0x7fae2c0d0fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:34552/0 (socket says 172.21.15.78:34552) 2023-10-11T09:51:54.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.648+0000 7fae316ae700 1 -- 172.21.15.78:0/1903325077 learned_addr learned my addr 172.21.15.78:0/1903325077 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:51:54.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.648+0000 7fae31eaf700 1 --2- 172.21.15.78:0/1903325077 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae2c0b74c0 0x7fae2c0cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:54.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.648+0000 7fae30ead700 1 --2- 172.21.15.78:0/1903325077 >> v2:172.21.15.133:3300/0 conn(0x7fae2c0d16f0 0x7fae2c0cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:54.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.648+0000 7fae23fff700 1 -- 172.21.15.78:0/1903325077 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1933636091 0 0) 0x7fae2c09c960 con 0x7fae2c0b4530 2023-10-11T09:51:54.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.648+0000 7fae23fff700 1 -- 172.21.15.78:0/1903325077 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae180035d0 con 0x7fae2c0b4530 2023-10-11T09:51:54.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.648+0000 7fae31eaf700 1 -- 172.21.15.78:0/1903325077 >> v2:172.21.15.133:3300/0 conn(0x7fae2c0d16f0 msgr2=0x7fae2c0cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:54.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.648+0000 7fae31eaf700 1 --2- 172.21.15.78:0/1903325077 >> v2:172.21.15.133:3300/0 conn(0x7fae2c0d16f0 0x7fae2c0cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:54.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.648+0000 7fae31eaf700 1 -- 172.21.15.78:0/1903325077 >> v1:172.21.15.78:6789/0 conn(0x7fae2c0b4530 legacy=0x7fae2c0d0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:54.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.648+0000 7fae31eaf700 1 -- 172.21.15.78:0/1903325077 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae2c0cfb70 con 0x7fae2c0b74c0 2023-10-11T09:51:54.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.648+0000 7fae31eaf700 1 --2- 172.21.15.78:0/1903325077 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae2c0b74c0 0x7fae2c0cdc70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fae24009d90 tx=0x7fae2400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1eb52324915878d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:54.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.648+0000 7fae23fff700 1 -- 172.21.15.78:0/1903325077 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2400e070 con 0x7fae2c0b74c0 2023-10-11T09:51:54.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.648+0000 7fae23fff700 1 -- 172.21.15.78:0/1903325077 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2400bb20 con 0x7fae2c0b74c0 2023-10-11T09:51:54.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.648+0000 7fae23fff700 1 -- 172.21.15.78:0/1903325077 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae24012430 con 0x7fae2c0b74c0 2023-10-11T09:51:54.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.652+0000 7fae326b0700 1 -- 172.21.15.78:0/1903325077 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae2c0b74c0 msgr2=0x7fae2c0cdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:54.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.652+0000 7fae326b0700 1 --2- 172.21.15.78:0/1903325077 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae2c0b74c0 0x7fae2c0cdc70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fae24009d90 tx=0x7fae2400b330 comp rx=0 tx=0).stop 2023-10-11T09:51:54.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.652+0000 7fae326b0700 1 -- 172.21.15.78:0/1903325077 shutdown_connections 2023-10-11T09:51:54.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.652+0000 7fae326b0700 1 --2- 172.21.15.78:0/1903325077 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae2c0b74c0 0x7fae2c0cdc70 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:54.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.652+0000 7fae326b0700 1 --2- 172.21.15.78:0/1903325077 >> v2:172.21.15.133:3300/0 conn(0x7fae2c0d16f0 0x7fae2c0cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:54.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.652+0000 7fae326b0700 1 -- 172.21.15.78:0/1903325077 >> 172.21.15.78:0/1903325077 conn(0x7fae2c09c3d0 msgr2=0x7fae2c0d30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:54.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.652+0000 7fae326b0700 1 -- 172.21.15.78:0/1903325077 shutdown_connections 2023-10-11T09:51:54.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.652+0000 7fae326b0700 1 -- 172.21.15.78:0/1903325077 wait complete. 2023-10-11T09:51:54.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.652+0000 7fae326b0700 1 Processor -- start 2023-10-11T09:51:54.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.652+0000 7fae326b0700 1 -- start start 2023-10-11T09:51:54.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.652+0000 7fae326b0700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae2c0b4530 0x7fae2c09de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:54.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.652+0000 7fae326b0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae2c0b74c0 0x7fae2c0b7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:54.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.652+0000 7fae326b0700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae2c0d16f0 0x7fae2c0a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:54.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.652+0000 7fae326b0700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c09e3c0 con 0x7fae2c0b74c0 2023-10-11T09:51:54.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.652+0000 7fae326b0700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c09cde0 con 0x7fae2c0d16f0 2023-10-11T09:51:54.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.652+0000 7fae326b0700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c09d0c0 con 0x7fae2c0b4530 2023-10-11T09:51:54.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.652+0000 7fae30ead700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae2c0b74c0 0x7fae2c0b7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:54.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.652+0000 7fae30ead700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae2c0b74c0 0x7fae2c0b7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49572/0 (socket says 172.21.15.78:49572) 2023-10-11T09:51:54.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.652+0000 7fae30ead700 1 -- 172.21.15.78:0/1009310696 learned_addr learned my addr 172.21.15.78:0/1009310696 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:54.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.656+0000 7fae31eaf700 1 --2- 172.21.15.78:0/1009310696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae2c0d16f0 0x7fae2c0a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:54.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.656+0000 7fae316ae700 1 --2- 172.21.15.78:0/1009310696 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae2c0b4530 0x7fae2c09de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:54.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.656+0000 7fae30ead700 1 -- 172.21.15.78:0/1009310696 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae2c0b4530 msgr2=0x7fae2c09de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:54.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.656+0000 7fae30ead700 1 --2- 172.21.15.78:0/1009310696 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae2c0b4530 0x7fae2c09de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:54.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.656+0000 7fae30ead700 1 -- 172.21.15.78:0/1009310696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae2c0d16f0 msgr2=0x7fae2c0a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:54.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.656+0000 7fae30ead700 1 --2- 172.21.15.78:0/1009310696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae2c0d16f0 0x7fae2c0a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:54.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.656+0000 7fae30ead700 1 -- 172.21.15.78:0/1009310696 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae24009a70 con 0x7fae2c0b74c0 2023-10-11T09:51:54.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.656+0000 7fae316ae700 1 --2- 172.21.15.78:0/1009310696 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae2c0b4530 0x7fae2c09de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:51:54.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.656+0000 7fae31eaf700 1 --2- 172.21.15.78:0/1009310696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae2c0d16f0 0x7fae2c0a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:51:54.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.656+0000 7fae30ead700 1 --2- 172.21.15.78:0/1009310696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae2c0b74c0 0x7fae2c0b7bf0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fae1c00e9d0 tx=0x7fae1c00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:54.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.656+0000 7fae21ffb700 1 -- 172.21.15.78:0/1009310696 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1c009720 con 0x7fae2c0b74c0 2023-10-11T09:51:54.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.656+0000 7fae21ffb700 1 -- 172.21.15.78:0/1009310696 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae1c00c850 con 0x7fae2c0b74c0 2023-10-11T09:51:54.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.656+0000 7fae326b0700 1 -- 172.21.15.78:0/1009310696 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae2c09d3a0 con 0x7fae2c0b74c0 2023-10-11T09:51:54.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.656+0000 7fae21ffb700 1 -- 172.21.15.78:0/1009310696 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1c00cbc0 con 0x7fae2c0b74c0 2023-10-11T09:51:54.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.656+0000 7fae326b0700 1 -- 172.21.15.78:0/1009310696 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae2c09d960 con 0x7fae2c0b74c0 2023-10-11T09:51:54.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.656+0000 7fae326b0700 1 -- 172.21.15.78:0/1009310696 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae2c0744c0 con 0x7fae2c0b74c0 2023-10-11T09:51:54.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.660+0000 7fae21ffb700 1 -- 172.21.15.78:0/1009310696 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fae1c00cd20 con 0x7fae2c0b74c0 2023-10-11T09:51:54.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.660+0000 7fae21ffb700 1 --2- 172.21.15.78:0/1009310696 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fae10065f00 0x7fae100683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:54.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.660+0000 7fae21ffb700 1 -- 172.21.15.78:0/1009310696 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fae1c083a00 con 0x7fae2c0b74c0 2023-10-11T09:51:54.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.660+0000 7fae316ae700 1 --2- 172.21.15.78:0/1009310696 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fae10065f00 0x7fae100683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:54.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.660+0000 7fae316ae700 1 --2- 172.21.15.78:0/1009310696 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fae10065f00 0x7fae100683b0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fae280010e0 tx=0x7fae28009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:54.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.664+0000 7fae21ffb700 1 -- 172.21.15.78:0/1009310696 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fae1c050f20 con 0x7fae2c0b74c0 2023-10-11T09:51:54.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.836+0000 7fae326b0700 1 -- 172.21.15.78:0/1009310696 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fae2c08ba90 con 0x7fae2c0b74c0 2023-10-11T09:51:54.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.836+0000 7fae21ffb700 1 -- 172.21.15.78:0/1009310696 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v127) v1 ==== 92+0+10 (secure 0 0 0) 0x7fae1c0210a0 con 0x7fae2c0b74c0 2023-10-11T09:51:54.843 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:51:54.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.840+0000 7fae326b0700 1 -- 172.21.15.78:0/1009310696 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fae10065f00 msgr2=0x7fae100683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:54.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.840+0000 7fae326b0700 1 --2- 172.21.15.78:0/1009310696 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fae10065f00 0x7fae100683b0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fae280010e0 tx=0x7fae28009040 comp rx=0 tx=0).stop 2023-10-11T09:51:54.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.840+0000 7fae326b0700 1 -- 172.21.15.78:0/1009310696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae2c0b74c0 msgr2=0x7fae2c0b7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:54.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.840+0000 7fae326b0700 1 --2- 172.21.15.78:0/1009310696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae2c0b74c0 0x7fae2c0b7bf0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fae1c00e9d0 tx=0x7fae1c00ed90 comp rx=0 tx=0).stop 2023-10-11T09:51:54.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.840+0000 7fae326b0700 1 -- 172.21.15.78:0/1009310696 shutdown_connections 2023-10-11T09:51:54.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.840+0000 7fae326b0700 1 --2- 172.21.15.78:0/1009310696 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fae10065f00 0x7fae100683b0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:54.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.840+0000 7fae326b0700 1 --2- 172.21.15.78:0/1009310696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae2c0d16f0 0x7fae2c0a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:54.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.840+0000 7fae326b0700 1 --2- 172.21.15.78:0/1009310696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae2c0b74c0 0x7fae2c0b7bf0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:54.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.840+0000 7fae326b0700 1 --2- 172.21.15.78:0/1009310696 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae2c0b4530 0x7fae2c09de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:54.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.840+0000 7fae326b0700 1 -- 172.21.15.78:0/1009310696 >> 172.21.15.78:0/1009310696 conn(0x7fae2c09c3d0 msgr2=0x7fae2c15f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:54.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.840+0000 7fae326b0700 1 -- 172.21.15.78:0/1009310696 shutdown_connections 2023-10-11T09:51:54.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:54.840+0000 7fae326b0700 1 -- 172.21.15.78:0/1009310696 wait complete. 2023-10-11T09:51:54.864 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T09:51:55.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.036+0000 7fc06f159700 1 Processor -- start 2023-10-11T09:51:55.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.036+0000 7fc06f159700 1 -- start start 2023-10-11T09:51:55.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.040+0000 7fc06f159700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc0680b2640 0x7fc0680b2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:55.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.040+0000 7fc06f159700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0680b30f0 con 0x7fc0680d0e20 2023-10-11T09:51:55.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.040+0000 7fc06f159700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc06815d050 con 0x7fc0680cfc60 2023-10-11T09:51:55.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.040+0000 7fc06f159700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0680b3320 con 0x7fc0680b2640 2023-10-11T09:51:55.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.040+0000 7fc06d6f6700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fc0680d0e20 0x7fc0680d1220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:34556/0 (socket says 172.21.15.78:34556) 2023-10-11T09:51:55.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.040+0000 7fc06d6f6700 1 -- 172.21.15.78:0/3319222551 learned_addr learned my addr 172.21.15.78:0/3319222551 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:51:55.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.040+0000 7fc06cef5700 1 --2- 172.21.15.78:0/3319222551 >> v2:172.21.15.133:3300/0 conn(0x7fc0680b2640 0x7fc0680b2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:55.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.040+0000 7fc0677fe700 1 -- 172.21.15.78:0/3319222551 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 322583453 0 0) 0x7fc0680b30f0 con 0x7fc0680d0e20 2023-10-11T09:51:55.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.040+0000 7fc0677fe700 1 -- 172.21.15.78:0/3319222551 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0580035d0 con 0x7fc0680d0e20 2023-10-11T09:51:55.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.040+0000 7fc06cef5700 1 -- 172.21.15.78:0/3319222551 >> v1:172.21.15.133:6789/0 conn(0x7fc0680cfc60 legacy=0x7fc0680b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:55.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.040+0000 7fc0677fe700 1 -- 172.21.15.78:0/3319222551 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1179385363 0 0) 0x7fc0580035d0 con 0x7fc0680d0e20 2023-10-11T09:51:55.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.040+0000 7fc06cef5700 1 -- 172.21.15.78:0/3319222551 >> v1:172.21.15.78:6789/0 conn(0x7fc0680d0e20 legacy=0x7fc0680d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:55.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.040+0000 7fc06cef5700 1 -- 172.21.15.78:0/3319222551 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0680b3600 con 0x7fc0680b2640 2023-10-11T09:51:55.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.040+0000 7fc06cef5700 1 --2- 172.21.15.78:0/3319222551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0680b2640 0x7fc0680b2bb0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc05c009850 tx=0x7fc05c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=309eecee612f3d9d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:55.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.040+0000 7fc0677fe700 1 -- 172.21.15.78:0/3319222551 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0600019f0 con 0x7fc0680b2640 2023-10-11T09:51:55.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.040+0000 7fc0677fe700 1 -- 172.21.15.78:0/3319222551 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc05c004030 con 0x7fc0680b2640 2023-10-11T09:51:55.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.040+0000 7fc0677fe700 1 -- 172.21.15.78:0/3319222551 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05c0043a0 con 0x7fc0680b2640 2023-10-11T09:51:55.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.044+0000 7fc06f159700 1 -- 172.21.15.78:0/3319222551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0680b2640 msgr2=0x7fc0680b2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:55.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.044+0000 7fc06f159700 1 --2- 172.21.15.78:0/3319222551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0680b2640 0x7fc0680b2bb0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc05c009850 tx=0x7fc05c009b60 comp rx=0 tx=0).stop 2023-10-11T09:51:55.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.044+0000 7fc06f159700 1 -- 172.21.15.78:0/3319222551 shutdown_connections 2023-10-11T09:51:55.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.044+0000 7fc06f159700 1 --2- 172.21.15.78:0/3319222551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0680b2640 0x7fc0680b2bb0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:55.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.044+0000 7fc06f159700 1 -- 172.21.15.78:0/3319222551 >> 172.21.15.78:0/3319222551 conn(0x7fc06809c560 msgr2=0x7fc0680d3250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:55.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.044+0000 7fc06f159700 1 -- 172.21.15.78:0/3319222551 shutdown_connections 2023-10-11T09:51:55.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.044+0000 7fc06f159700 1 -- 172.21.15.78:0/3319222551 wait complete. 2023-10-11T09:51:55.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.044+0000 7fc06f159700 1 Processor -- start 2023-10-11T09:51:55.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.044+0000 7fc06f159700 1 -- start start 2023-10-11T09:51:55.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.048+0000 7fc06f159700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0680b2640 0x7fc06815d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:55.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.048+0000 7fc06f159700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0680cfc60 0x7fc0681f5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:55.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.048+0000 7fc06f159700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc0680d0e20 0x7fc0680b8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:55.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.048+0000 7fc06f159700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0680c7260 con 0x7fc0680b2640 2023-10-11T09:51:55.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.048+0000 7fc06f159700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0680b7ae0 con 0x7fc0680cfc60 2023-10-11T09:51:55.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.048+0000 7fc06f159700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0681f5540 con 0x7fc0680d0e20 2023-10-11T09:51:55.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.048+0000 7fc06cef5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0680b2640 0x7fc06815d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:55.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.048+0000 7fc067fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0680cfc60 0x7fc0681f5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:55.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.048+0000 7fc06cef5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0680b2640 0x7fc06815d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49574/0 (socket says 172.21.15.78:49574) 2023-10-11T09:51:55.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.048+0000 7fc06cef5700 1 -- 172.21.15.78:0/3905326028 learned_addr learned my addr 172.21.15.78:0/3905326028 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:55.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.048+0000 7fc06d6f6700 1 --2- 172.21.15.78:0/3905326028 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc0680d0e20 0x7fc0680b8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:55.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.048+0000 7fc06cef5700 1 -- 172.21.15.78:0/3905326028 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc0680d0e20 msgr2=0x7fc0680b8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:55.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.048+0000 7fc06cef5700 1 --2- 172.21.15.78:0/3905326028 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc0680d0e20 0x7fc0680b8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:55.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.048+0000 7fc06cef5700 1 -- 172.21.15.78:0/3905326028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0680cfc60 msgr2=0x7fc0681f5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:55.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.048+0000 7fc06cef5700 1 --2- 172.21.15.78:0/3905326028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0680cfc60 0x7fc0681f5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:55.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.048+0000 7fc06cef5700 1 -- 172.21.15.78:0/3905326028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc05c0094b0 con 0x7fc0680b2640 2023-10-11T09:51:55.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.048+0000 7fc067fff700 1 --2- 172.21.15.78:0/3905326028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0680cfc60 0x7fc0681f5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:51:55.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.048+0000 7fc06cef5700 1 --2- 172.21.15.78:0/3905326028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0680b2640 0x7fc06815d240 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fc0680b5e40 tx=0x7fc05c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:55.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.048+0000 7fc0657fa700 1 -- 172.21.15.78:0/3905326028 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05c011630 con 0x7fc0680b2640 2023-10-11T09:51:55.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.048+0000 7fc0657fa700 1 -- 172.21.15.78:0/3905326028 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc05c01f070 con 0x7fc0680b2640 2023-10-11T09:51:55.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.048+0000 7fc06f159700 1 -- 172.21.15.78:0/3905326028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0680b8a70 con 0x7fc0680b2640 2023-10-11T09:51:55.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.048+0000 7fc0657fa700 1 -- 172.21.15.78:0/3905326028 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05c0119d0 con 0x7fc0680b2640 2023-10-11T09:51:55.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.048+0000 7fc06f159700 1 -- 172.21.15.78:0/3905326028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0680c9cb0 con 0x7fc0680b2640 2023-10-11T09:51:55.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.052+0000 7fc06f159700 1 -- 172.21.15.78:0/3905326028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc06808bc20 con 0x7fc0680b2640 2023-10-11T09:51:55.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.052+0000 7fc0657fa700 1 -- 172.21.15.78:0/3905326028 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fc05c011b30 con 0x7fc0680b2640 2023-10-11T09:51:55.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.052+0000 7fc0657fa700 1 --2- 172.21.15.78:0/3905326028 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc04c065ed0 0x7fc04c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:55.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.052+0000 7fc0657fa700 1 -- 172.21.15.78:0/3905326028 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc05c021070 con 0x7fc0680b2640 2023-10-11T09:51:55.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.052+0000 7fc067fff700 1 --2- 172.21.15.78:0/3905326028 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc04c065ed0 0x7fc04c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:55.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.056+0000 7fc067fff700 1 --2- 172.21.15.78:0/3905326028 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc04c065ed0 0x7fc04c068380 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc068147f50 tx=0x7fc054009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:55.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.060+0000 7fc0657fa700 1 -- 172.21.15.78:0/3905326028 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fc05c052760 con 0x7fc0680b2640 2023-10-11T09:51:55.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.232+0000 7fc06f159700 1 -- 172.21.15.78:0/3905326028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc068074530 con 0x7fc0680b2640 2023-10-11T09:51:55.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.236+0000 7fc0657fa700 1 -- 172.21.15.78:0/3905326028 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v127) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc05c055d80 con 0x7fc0680b2640 2023-10-11T09:51:55.241 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:51:55.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.236+0000 7fc06f159700 1 -- 172.21.15.78:0/3905326028 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc04c065ed0 msgr2=0x7fc04c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:55.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.236+0000 7fc06f159700 1 --2- 172.21.15.78:0/3905326028 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc04c065ed0 0x7fc04c068380 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc068147f50 tx=0x7fc054009040 comp rx=0 tx=0).stop 2023-10-11T09:51:55.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.236+0000 7fc06f159700 1 -- 172.21.15.78:0/3905326028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0680b2640 msgr2=0x7fc06815d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:55.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.236+0000 7fc06f159700 1 --2- 172.21.15.78:0/3905326028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0680b2640 0x7fc06815d240 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fc0680b5e40 tx=0x7fc05c004770 comp rx=0 tx=0).stop 2023-10-11T09:51:55.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.236+0000 7fc06f159700 1 -- 172.21.15.78:0/3905326028 shutdown_connections 2023-10-11T09:51:55.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.236+0000 7fc06f159700 1 --2- 172.21.15.78:0/3905326028 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc04c065ed0 0x7fc04c068380 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:55.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.236+0000 7fc06f159700 1 --2- 172.21.15.78:0/3905326028 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc0680d0e20 0x7fc0680b8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:55.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.236+0000 7fc06f159700 1 --2- 172.21.15.78:0/3905326028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0680cfc60 0x7fc0681f5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:55.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.236+0000 7fc06f159700 1 --2- 172.21.15.78:0/3905326028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0680b2640 0x7fc06815d240 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:55.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.236+0000 7fc06f159700 1 -- 172.21.15.78:0/3905326028 >> 172.21.15.78:0/3905326028 conn(0x7fc06809c560 msgr2=0x7fc06806c190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:55.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.236+0000 7fc06f159700 1 -- 172.21.15.78:0/3905326028 shutdown_connections 2023-10-11T09:51:55.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.236+0000 7fc06f159700 1 -- 172.21.15.78:0/3905326028 wait complete. 2023-10-11T09:51:55.265 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T09:51:55.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.428+0000 7fc41d225700 1 Processor -- start 2023-10-11T09:51:55.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.432+0000 7fc41d225700 1 -- start start 2023-10-11T09:51:55.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.432+0000 7fc41d225700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc4180caf80 0x7fc4180711c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:55.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.432+0000 7fc41d225700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fc41815f090 0x7fc418071700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:55.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.432+0000 7fc41d225700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc418072040 con 0x7fc4180cb350 2023-10-11T09:51:55.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.432+0000 7fc41d225700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc41815ec00 con 0x7fc41815f090 2023-10-11T09:51:55.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.432+0000 7fc41d225700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc418071d60 con 0x7fc4180caf80 2023-10-11T09:51:55.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.432+0000 7fc40ffff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fc41815f090 0x7fc418071700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:55.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.432+0000 7fc40ffff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fc41815f090 0x7fc418071700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49582/0 (socket says 172.21.15.78:49582) 2023-10-11T09:51:55.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.432+0000 7fc40ffff700 1 -- 172.21.15.78:0/1108669567 learned_addr learned my addr 172.21.15.78:0/1108669567 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:55.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.432+0000 7fc40ffff700 1 -- 172.21.15.78:0/1108669567 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc4180caf80 msgr2=0x7fc4180711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:55.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.432+0000 7fc416d9d700 1 --2- 172.21.15.78:0/1108669567 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc4180caf80 0x7fc4180711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:55.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.432+0000 7fc40ffff700 1 --2- 172.21.15.78:0/1108669567 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc4180caf80 0x7fc4180711c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:55.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.432+0000 7fc40ffff700 1 -- 172.21.15.78:0/1108669567 >> v1:172.21.15.133:6789/0 conn(0x7fc4180cb350 legacy=0x7fc41806bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:51:55.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.432+0000 7fc40ffff700 1 -- 172.21.15.78:0/1108669567 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc41815e5b0 con 0x7fc41815f090 2023-10-11T09:51:55.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.432+0000 7fc416d9d700 1 --2- 172.21.15.78:0/1108669567 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc4180caf80 0x7fc4180711c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:51:55.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.432+0000 7fc40ffff700 1 --2- 172.21.15.78:0/1108669567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc41815f090 0x7fc418071700 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fc400009910 tx=0x7fc400009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb2204c2605f3c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:55.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.436+0000 7fc41659c700 1 -- 172.21.15.78:0/1108669567 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc400004230 con 0x7fc41815f090 2023-10-11T09:51:55.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.436+0000 7fc41659c700 1 -- 172.21.15.78:0/1108669567 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc400004390 con 0x7fc41815f090 2023-10-11T09:51:55.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.436+0000 7fc41659c700 1 -- 172.21.15.78:0/1108669567 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc400009e60 con 0x7fc41815f090 2023-10-11T09:51:55.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.436+0000 7fc41d225700 1 -- 172.21.15.78:0/1108669567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc41815f090 msgr2=0x7fc418071700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:55.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.436+0000 7fc41d225700 1 --2- 172.21.15.78:0/1108669567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc41815f090 0x7fc418071700 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fc400009910 tx=0x7fc400009c20 comp rx=0 tx=0).stop 2023-10-11T09:51:55.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.436+0000 7fc41d225700 1 -- 172.21.15.78:0/1108669567 shutdown_connections 2023-10-11T09:51:55.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.436+0000 7fc41d225700 1 --2- 172.21.15.78:0/1108669567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc41815f090 0x7fc418071700 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:55.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.436+0000 7fc41d225700 1 --2- 172.21.15.78:0/1108669567 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc4180caf80 0x7fc4180711c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:55.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.436+0000 7fc41d225700 1 -- 172.21.15.78:0/1108669567 >> 172.21.15.78:0/1108669567 conn(0x7fc41806ae10 msgr2=0x7fc4180b0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:55.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.436+0000 7fc41d225700 1 -- 172.21.15.78:0/1108669567 shutdown_connections 2023-10-11T09:51:55.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.436+0000 7fc41d225700 1 -- 172.21.15.78:0/1108669567 wait complete. 2023-10-11T09:51:55.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.440+0000 7fc41d225700 1 Processor -- start 2023-10-11T09:51:55.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.440+0000 7fc41d225700 1 -- start start 2023-10-11T09:51:55.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.440+0000 7fc41d225700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc4180caf80 0x7fc4180af3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:55.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.440+0000 7fc41d225700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4180cb350 0x7fc4180b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:55.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.440+0000 7fc41d225700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc41815f090 0x7fc418151cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:55.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.440+0000 7fc41d225700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4180b0a70 con 0x7fc4180cb350 2023-10-11T09:51:55.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.440+0000 7fc41d225700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4180b0070 con 0x7fc41815f090 2023-10-11T09:51:55.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.440+0000 7fc41d225700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc418152230 con 0x7fc4180caf80 2023-10-11T09:51:55.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.440+0000 7fc40ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4180cb350 0x7fc4180b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:55.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.440+0000 7fc40ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4180cb350 0x7fc4180b0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49590/0 (socket says 172.21.15.78:49590) 2023-10-11T09:51:55.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.440+0000 7fc40ffff700 1 -- 172.21.15.78:0/3966898464 learned_addr learned my addr 172.21.15.78:0/3966898464 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:55.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.440+0000 7fc416d9d700 1 --2- 172.21.15.78:0/3966898464 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc4180caf80 0x7fc4180af3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:55.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.440+0000 7fc41759e700 1 --2- 172.21.15.78:0/3966898464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc41815f090 0x7fc418151cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:55.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.444+0000 7fc40ffff700 1 -- 172.21.15.78:0/3966898464 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc4180caf80 msgr2=0x7fc4180af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:55.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.444+0000 7fc40ffff700 1 --2- 172.21.15.78:0/3966898464 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc4180caf80 0x7fc4180af3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:55.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.444+0000 7fc40ffff700 1 -- 172.21.15.78:0/3966898464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc41815f090 msgr2=0x7fc418151cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:55.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.444+0000 7fc40ffff700 1 --2- 172.21.15.78:0/3966898464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc41815f090 0x7fc418151cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:55.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.444+0000 7fc40ffff700 1 -- 172.21.15.78:0/3966898464 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc400009570 con 0x7fc4180cb350 2023-10-11T09:51:55.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.444+0000 7fc40ffff700 1 --2- 172.21.15.78:0/3966898464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4180cb350 0x7fc4180b0530 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fc41806bd10 tx=0x7fc400004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:55.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.444+0000 7fc40f7fe700 1 -- 172.21.15.78:0/3966898464 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4000036c0 con 0x7fc4180cb350 2023-10-11T09:51:55.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.444+0000 7fc40f7fe700 1 -- 172.21.15.78:0/3966898464 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc400003820 con 0x7fc4180cb350 2023-10-11T09:51:55.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.444+0000 7fc40f7fe700 1 -- 172.21.15.78:0/3966898464 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc400003bb0 con 0x7fc4180cb350 2023-10-11T09:51:55.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.444+0000 7fc41d225700 1 -- 172.21.15.78:0/3966898464 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4181efca0 con 0x7fc4180cb350 2023-10-11T09:51:55.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.444+0000 7fc41d225700 1 -- 172.21.15.78:0/3966898464 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4180afd90 con 0x7fc4180cb350 2023-10-11T09:51:55.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.444+0000 7fc41d225700 1 -- 172.21.15.78:0/3966898464 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc41806f4f0 con 0x7fc4180cb350 2023-10-11T09:51:55.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.448+0000 7fc40f7fe700 1 -- 172.21.15.78:0/3966898464 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fc400003d10 con 0x7fc4180cb350 2023-10-11T09:51:55.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.448+0000 7fc40f7fe700 1 --2- 172.21.15.78:0/3966898464 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc3f8065ed0 0x7fc3f8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:55.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.448+0000 7fc40f7fe700 1 -- 172.21.15.78:0/3966898464 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc400084fe0 con 0x7fc4180cb350 2023-10-11T09:51:55.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.448+0000 7fc416d9d700 1 --2- 172.21.15.78:0/3966898464 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc3f8065ed0 0x7fc3f8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:55.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.452+0000 7fc416d9d700 1 --2- 172.21.15.78:0/3966898464 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc3f8065ed0 0x7fc3f8068380 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc408006fd0 tx=0x7fc408008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:55.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.452+0000 7fc40f7fe700 1 -- 172.21.15.78:0/3966898464 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fc400052660 con 0x7fc4180cb350 2023-10-11T09:51:55.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.632+0000 7fc41d225700 1 -- 172.21.15.78:0/3966898464 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc418074530 con 0x7fc4180cb350 2023-10-11T09:51:55.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.632+0000 7fc40f7fe700 1 -- 172.21.15.78:0/3966898464 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v127) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc40001b440 con 0x7fc4180cb350 2023-10-11T09:51:55.638 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2023-10-11T09:51:55.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.636+0000 7fc41d225700 1 -- 172.21.15.78:0/3966898464 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc3f8065ed0 msgr2=0x7fc3f8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:55.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.636+0000 7fc41d225700 1 --2- 172.21.15.78:0/3966898464 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc3f8065ed0 0x7fc3f8068380 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc408006fd0 tx=0x7fc408008040 comp rx=0 tx=0).stop 2023-10-11T09:51:55.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.636+0000 7fc41d225700 1 -- 172.21.15.78:0/3966898464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4180cb350 msgr2=0x7fc4180b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:55.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.636+0000 7fc41d225700 1 --2- 172.21.15.78:0/3966898464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4180cb350 0x7fc4180b0530 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fc41806bd10 tx=0x7fc400004770 comp rx=0 tx=0).stop 2023-10-11T09:51:55.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.636+0000 7fc41d225700 1 -- 172.21.15.78:0/3966898464 shutdown_connections 2023-10-11T09:51:55.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.636+0000 7fc41d225700 1 --2- 172.21.15.78:0/3966898464 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc3f8065ed0 0x7fc3f8068380 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:55.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.636+0000 7fc41d225700 1 --2- 172.21.15.78:0/3966898464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc41815f090 0x7fc418151cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:55.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.636+0000 7fc41d225700 1 --2- 172.21.15.78:0/3966898464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4180cb350 0x7fc4180b0530 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:55.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.636+0000 7fc41d225700 1 --2- 172.21.15.78:0/3966898464 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc4180caf80 0x7fc4180af3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:55.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.636+0000 7fc41d225700 1 -- 172.21.15.78:0/3966898464 >> 172.21.15.78:0/3966898464 conn(0x7fc41806ae10 msgr2=0x7fc41815ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:55.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.636+0000 7fc41d225700 1 -- 172.21.15.78:0/3966898464 shutdown_connections 2023-10-11T09:51:55.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.636+0000 7fc41d225700 1 -- 172.21.15.78:0/3966898464 wait complete. 2023-10-11T09:51:55.666 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T09:51:55.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.840+0000 7f2930043700 1 Processor -- start 2023-10-11T09:51:55.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.840+0000 7f2930043700 1 -- start start 2023-10-11T09:51:55.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.844+0000 7f2930043700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29280ae7e0 0x7f29280b2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:55.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.844+0000 7f2930043700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29280b3b80 con 0x7f29280cc1b0 2023-10-11T09:51:55.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.844+0000 7f2930043700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29280c60c0 con 0x7f29280ae410 2023-10-11T09:51:55.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.844+0000 7f2930043700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29280b3000 con 0x7f29280ae7e0 2023-10-11T09:51:55.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.844+0000 7f292d5de700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f29280cc1b0 0x7f292806bfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:34570/0 (socket says 172.21.15.78:34570) 2023-10-11T09:51:55.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.844+0000 7f292d5de700 1 -- 172.21.15.78:0/1527130185 learned_addr learned my addr 172.21.15.78:0/1527130185 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:51:55.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.844+0000 7f292e5e0700 1 --2- 172.21.15.78:0/1527130185 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29280ae7e0 0x7f29280b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:55.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.844+0000 7f292e5e0700 1 -- 172.21.15.78:0/1527130185 >> v1:172.21.15.133:6789/0 conn(0x7f29280ae410 legacy=0x7f29280cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:55.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.844+0000 7f292e5e0700 1 -- 172.21.15.78:0/1527130185 >> v1:172.21.15.78:6789/0 conn(0x7f29280cc1b0 legacy=0x7f292806bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:55.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.844+0000 7f292e5e0700 1 -- 172.21.15.78:0/1527130185 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29280b32e0 con 0x7f29280ae7e0 2023-10-11T09:51:55.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.844+0000 7f292cddd700 1 -- 172.21.15.78:0/1527130185 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2685396565 0 0) 0x7f29280b3b80 con 0x7f29280cc1b0 2023-10-11T09:51:55.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.844+0000 7f292e5e0700 1 --2- 172.21.15.78:0/1527130185 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29280ae7e0 0x7f29280b2900 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2920009880 tx=0x7f2920009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70e726ef6bbf8bc9 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:55.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.844+0000 7f292cddd700 1 -- 172.21.15.78:0/1527130185 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292000f940 con 0x7f29280ae7e0 2023-10-11T09:51:55.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.844+0000 7f292cddd700 1 -- 172.21.15.78:0/1527130185 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f292000faa0 con 0x7f29280ae7e0 2023-10-11T09:51:55.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.844+0000 7f292cddd700 1 -- 172.21.15.78:0/1527130185 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292000fe30 con 0x7f29280ae7e0 2023-10-11T09:51:55.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.844+0000 7f2930043700 1 -- 172.21.15.78:0/1527130185 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29280ae7e0 msgr2=0x7f29280b2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:55.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.844+0000 7f2930043700 1 --2- 172.21.15.78:0/1527130185 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29280ae7e0 0x7f29280b2900 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2920009880 tx=0x7f2920009b90 comp rx=0 tx=0).stop 2023-10-11T09:51:55.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.848+0000 7f2930043700 1 -- 172.21.15.78:0/1527130185 shutdown_connections 2023-10-11T09:51:55.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.848+0000 7f2930043700 1 --2- 172.21.15.78:0/1527130185 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29280ae7e0 0x7f29280b2900 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:55.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.848+0000 7f2930043700 1 -- 172.21.15.78:0/1527130185 >> 172.21.15.78:0/1527130185 conn(0x7f292806a180 msgr2=0x7f292806a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:55.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.848+0000 7f2930043700 1 -- 172.21.15.78:0/1527130185 shutdown_connections 2023-10-11T09:51:55.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.848+0000 7f2930043700 1 -- 172.21.15.78:0/1527130185 wait complete. 2023-10-11T09:51:55.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.852+0000 7f2930043700 1 Processor -- start 2023-10-11T09:51:55.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.852+0000 7f2930043700 1 -- start start 2023-10-11T09:51:55.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.852+0000 7f2930043700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29280ae410 0x7f29280b1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:55.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.852+0000 7f2930043700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29280ae7e0 0x7f2928156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:55.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.852+0000 7f2930043700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29280cc1b0 0x7f292815a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:55.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.852+0000 7f2930043700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29280b1b80 con 0x7f29280ae7e0 2023-10-11T09:51:55.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.852+0000 7f2930043700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29281568b0 con 0x7f29280cc1b0 2023-10-11T09:51:55.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.852+0000 7f2930043700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f292815a7b0 con 0x7f29280ae410 2023-10-11T09:51:55.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.852+0000 7f292d5de700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29280ae7e0 0x7f2928156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:55.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.852+0000 7f292d5de700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29280ae7e0 0x7f2928156370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49594/0 (socket says 172.21.15.78:49594) 2023-10-11T09:51:55.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.852+0000 7f292d5de700 1 -- 172.21.15.78:0/3791705436 learned_addr learned my addr 172.21.15.78:0/3791705436 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:51:55.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.852+0000 7f292dddf700 1 --2- 172.21.15.78:0/3791705436 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29280ae410 0x7f29280b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:55.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.852+0000 7f292e5e0700 1 --2- 172.21.15.78:0/3791705436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29280cc1b0 0x7f292815a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:55.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.852+0000 7f292d5de700 1 -- 172.21.15.78:0/3791705436 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29280ae410 msgr2=0x7f29280b1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:55.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.852+0000 7f292d5de700 1 --2- 172.21.15.78:0/3791705436 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29280ae410 0x7f29280b1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:55.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.852+0000 7f292d5de700 1 -- 172.21.15.78:0/3791705436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29280cc1b0 msgr2=0x7f292815a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:55.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.852+0000 7f292d5de700 1 --2- 172.21.15.78:0/3791705436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29280cc1b0 0x7f292815a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:55.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.852+0000 7f292d5de700 1 -- 172.21.15.78:0/3791705436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29200094b0 con 0x7f29280ae7e0 2023-10-11T09:51:55.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.856+0000 7f292d5de700 1 --2- 172.21.15.78:0/3791705436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29280ae7e0 0x7f2928156370 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f291800c550 tx=0x7f291800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:55.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.856+0000 7f291e7fc700 1 -- 172.21.15.78:0/3791705436 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2918004ca0 con 0x7f29280ae7e0 2023-10-11T09:51:55.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.856+0000 7f2930043700 1 -- 172.21.15.78:0/3791705436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29280aff50 con 0x7f29280ae7e0 2023-10-11T09:51:55.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.856+0000 7f291e7fc700 1 -- 172.21.15.78:0/3791705436 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2918004e00 con 0x7f29280ae7e0 2023-10-11T09:51:55.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.856+0000 7f291e7fc700 1 -- 172.21.15.78:0/3791705436 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2918007310 con 0x7f29280ae7e0 2023-10-11T09:51:55.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.856+0000 7f2930043700 1 -- 172.21.15.78:0/3791705436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29280b0510 con 0x7f29280ae7e0 2023-10-11T09:51:55.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.856+0000 7f2930043700 1 -- 172.21.15.78:0/3791705436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29280cc990 con 0x7f29280ae7e0 2023-10-11T09:51:55.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.860+0000 7f291e7fc700 1 -- 172.21.15.78:0/3791705436 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f2918007470 con 0x7f29280ae7e0 2023-10-11T09:51:55.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.860+0000 7f291e7fc700 1 --2- 172.21.15.78:0/3791705436 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f290c065ed0 0x7f290c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:51:55.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.860+0000 7f291e7fc700 1 -- 172.21.15.78:0/3791705436 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2918083500 con 0x7f29280ae7e0 2023-10-11T09:51:55.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.860+0000 7f292dddf700 1 --2- 172.21.15.78:0/3791705436 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f290c065ed0 0x7f290c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:51:55.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.860+0000 7f292dddf700 1 --2- 172.21.15.78:0/3791705436 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f290c065ed0 0x7f290c068380 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f29240012e0 tx=0x7f2924009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:51:55.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:55.868+0000 7f291e7fc700 1 -- 172.21.15.78:0/3791705436 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f2918050ac0 con 0x7f29280ae7e0 2023-10-11T09:51:56.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:56.052+0000 7f2930043700 1 -- 172.21.15.78:0/3791705436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f29280b3870 con 0x7f29280ae7e0 2023-10-11T09:51:56.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:56.052+0000 7f291e7fc700 1 -- 172.21.15.78:0/3791705436 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v127) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2918007720 con 0x7f29280ae7e0 2023-10-11T09:51:56.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:56.056+0000 7f2930043700 1 -- 172.21.15.78:0/3791705436 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f290c065ed0 msgr2=0x7f290c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:56.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:56.056+0000 7f2930043700 1 --2- 172.21.15.78:0/3791705436 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f290c065ed0 0x7f290c068380 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f29240012e0 tx=0x7f2924009040 comp rx=0 tx=0).stop 2023-10-11T09:51:56.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:56.056+0000 7f2930043700 1 -- 172.21.15.78:0/3791705436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29280ae7e0 msgr2=0x7f2928156370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:51:56.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:56.056+0000 7f2930043700 1 --2- 172.21.15.78:0/3791705436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29280ae7e0 0x7f2928156370 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f291800c550 tx=0x7f291800c860 comp rx=0 tx=0).stop 2023-10-11T09:51:56.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:56.056+0000 7f2930043700 1 -- 172.21.15.78:0/3791705436 shutdown_connections 2023-10-11T09:51:56.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:56.056+0000 7f2930043700 1 --2- 172.21.15.78:0/3791705436 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f290c065ed0 0x7f290c068380 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:56.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:56.056+0000 7f2930043700 1 --2- 172.21.15.78:0/3791705436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29280cc1b0 0x7f292815a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:56.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:56.056+0000 7f2930043700 1 --2- 172.21.15.78:0/3791705436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29280ae7e0 0x7f2928156370 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:56.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:56.056+0000 7f2930043700 1 --2- 172.21.15.78:0/3791705436 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29280ae410 0x7f29280b1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:51:56.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:56.056+0000 7f2930043700 1 -- 172.21.15.78:0/3791705436 >> 172.21.15.78:0/3791705436 conn(0x7f292806a180 msgr2=0x7f2928158660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:51:56.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:56.056+0000 7f2930043700 1 -- 172.21.15.78:0/3791705436 shutdown_connections 2023-10-11T09:51:56.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:51:56.056+0000 7f2930043700 1 -- 172.21.15.78:0/3791705436 wait complete. 2023-10-11T09:51:56.066 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2023-10-11T09:51:56.081 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi078.front.sepia.ceph.com... 2023-10-11T09:51:56.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:51:56.082 DEBUG:teuthology.orchestra.run.smithi078:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-11T09:51:56.114 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:ceph-fuse[59487]: fuse finished with error 0 and tester_r 0 2023-10-11T09:51:56.119 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:52:00.913 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:52:00.920 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:52:02.222 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2023-10-11T09:52:02.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:02.223 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-11T09:52:02.247 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2023-10-11T09:52:02.464 INFO:teuthology.orchestra.run.smithi133.stdout:{"message":"","return_code":0} 2023-10-11T09:52:02.464 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-10-11T09:52:02.465 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-11T09:52:02.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.635+0000 7fa1c5e41700 1 Processor -- start 2023-10-11T09:52:02.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.635+0000 7fa1c5e41700 1 -- start start 2023-10-11T09:52:02.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.639+0000 7fa1c5e41700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fa1c00cd680 0x7fa1c00cda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:02.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.639+0000 7fa1c5e41700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa1c00cdf90 0x7fa1c00ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:02.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.639+0000 7fa1c5e41700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1c00ced00 con 0x7fa1c006b290 2023-10-11T09:52:02.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.639+0000 7fa1c5e41700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1c00cf510 con 0x7fa1c00cd680 2023-10-11T09:52:02.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.639+0000 7fa1c5e41700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c015cec0 con 0x7fa1c00cdf90 2023-10-11T09:52:02.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.639+0000 7fa1c4e3f700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fa1c00cd680 0x7fa1c00cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:02.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.639+0000 7fa1c4e3f700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fa1c00cd680 0x7fa1c00cda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49602/0 (socket says 172.21.15.78:49602) 2023-10-11T09:52:02.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.639+0000 7fa1c4e3f700 1 -- 172.21.15.78:0/1057625849 learned_addr learned my addr 172.21.15.78:0/1057625849 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:02.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.639+0000 7fa1bffff700 1 --2- 172.21.15.78:0/1057625849 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa1c00cdf90 0x7fa1c00ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:02.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.643+0000 7fa1c4e3f700 1 -- 172.21.15.78:0/1057625849 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa1c00cdf90 msgr2=0x7fa1c00ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:02.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.643+0000 7fa1c4e3f700 1 --2- 172.21.15.78:0/1057625849 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa1c00cdf90 0x7fa1c00ce3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:02.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.643+0000 7fa1c4e3f700 1 -- 172.21.15.78:0/1057625849 >> v1:172.21.15.133:6789/0 conn(0x7fa1c006b290 legacy=0x7fa1c006b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:02.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.643+0000 7fa1c4e3f700 1 -- 172.21.15.78:0/1057625849 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c00cef50 con 0x7fa1c00cd680 2023-10-11T09:52:02.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.643+0000 7fa1c4e3f700 1 --2- 172.21.15.78:0/1057625849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1c00cd680 0x7fa1c00cda50 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4009900 tx=0x7fa1b4009c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e155a0def4161516 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:02.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.643+0000 7fa1bf7fe700 1 -- 172.21.15.78:0/1057625849 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b4011830 con 0x7fa1c00cd680 2023-10-11T09:52:02.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.643+0000 7fa1bf7fe700 1 -- 172.21.15.78:0/1057625849 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1b4011990 con 0x7fa1c00cd680 2023-10-11T09:52:02.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.643+0000 7fa1bf7fe700 1 -- 172.21.15.78:0/1057625849 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b4011d40 con 0x7fa1c00cd680 2023-10-11T09:52:02.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.643+0000 7fa1c5e41700 1 -- 172.21.15.78:0/1057625849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1c00cd680 msgr2=0x7fa1c00cda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:02.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.643+0000 7fa1c5e41700 1 --2- 172.21.15.78:0/1057625849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1c00cd680 0x7fa1c00cda50 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4009900 tx=0x7fa1b4009c10 comp rx=0 tx=0).stop 2023-10-11T09:52:02.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.643+0000 7fa1c5e41700 1 -- 172.21.15.78:0/1057625849 shutdown_connections 2023-10-11T09:52:02.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.643+0000 7fa1c5e41700 1 --2- 172.21.15.78:0/1057625849 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa1c00cdf90 0x7fa1c00ce3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:02.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.643+0000 7fa1c5e41700 1 --2- 172.21.15.78:0/1057625849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1c00cd680 0x7fa1c00cda50 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:02.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.643+0000 7fa1c5e41700 1 -- 172.21.15.78:0/1057625849 >> 172.21.15.78:0/1057625849 conn(0x7fa1c006aaa0 msgr2=0x7fa1c009a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:02.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.643+0000 7fa1c5e41700 1 -- 172.21.15.78:0/1057625849 shutdown_connections 2023-10-11T09:52:02.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.643+0000 7fa1c5e41700 1 -- 172.21.15.78:0/1057625849 wait complete. 2023-10-11T09:52:02.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.647+0000 7fa1c5e41700 1 Processor -- start 2023-10-11T09:52:02.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.647+0000 7fa1c5e41700 1 -- start start 2023-10-11T09:52:02.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.647+0000 7fa1c5e41700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1c006b290 0x7fa1c015fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:02.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.647+0000 7fa1c5e41700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1c00cd680 0x7fa1c015a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:02.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.647+0000 7fa1c5e41700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa1c00cdf90 0x7fa1c015daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:02.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.647+0000 7fa1c5e41700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c015ab10 con 0x7fa1c00cd680 2023-10-11T09:52:02.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.647+0000 7fa1c5e41700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c015ee00 con 0x7fa1c006b290 2023-10-11T09:52:02.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.647+0000 7fa1c5e41700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c015dfe0 con 0x7fa1c00cdf90 2023-10-11T09:52:02.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.651+0000 7fa1bffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1c00cd680 0x7fa1c015a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:02.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.651+0000 7fa1bffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1c00cd680 0x7fa1c015a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49604/0 (socket says 172.21.15.78:49604) 2023-10-11T09:52:02.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.651+0000 7fa1bffff700 1 -- 172.21.15.78:0/2423702874 learned_addr learned my addr 172.21.15.78:0/2423702874 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:02.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.651+0000 7fa1c4e3f700 1 --2- 172.21.15.78:0/2423702874 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1c006b290 0x7fa1c015fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:02.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.651+0000 7fa1c5640700 1 --2- 172.21.15.78:0/2423702874 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa1c00cdf90 0x7fa1c015daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:02.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.651+0000 7fa1bffff700 1 -- 172.21.15.78:0/2423702874 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa1c00cdf90 msgr2=0x7fa1c015daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:02.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.651+0000 7fa1bffff700 1 --2- 172.21.15.78:0/2423702874 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa1c00cdf90 0x7fa1c015daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:02.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.651+0000 7fa1bffff700 1 -- 172.21.15.78:0/2423702874 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1c006b290 msgr2=0x7fa1c015fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:02.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.651+0000 7fa1bffff700 1 --2- 172.21.15.78:0/2423702874 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1c006b290 0x7fa1c015fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:02.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.651+0000 7fa1bffff700 1 -- 172.21.15.78:0/2423702874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b4009510 con 0x7fa1c00cd680 2023-10-11T09:52:02.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.651+0000 7fa1bffff700 1 --2- 172.21.15.78:0/2423702874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1c00cd680 0x7fa1c015a5d0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00e840 tx=0x7fa1ac00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:02.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.651+0000 7fa1bd7fa700 1 -- 172.21.15.78:0/2423702874 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac0095a0 con 0x7fa1c00cd680 2023-10-11T09:52:02.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.651+0000 7fa1bd7fa700 1 -- 172.21.15.78:0/2423702874 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1ac013680 con 0x7fa1c00cd680 2023-10-11T09:52:02.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.651+0000 7fa1c5e41700 1 -- 172.21.15.78:0/2423702874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1c015e280 con 0x7fa1c00cd680 2023-10-11T09:52:02.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.651+0000 7fa1bd7fa700 1 -- 172.21.15.78:0/2423702874 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac0139f0 con 0x7fa1c00cd680 2023-10-11T09:52:02.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.651+0000 7fa1c5e41700 1 -- 172.21.15.78:0/2423702874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1c015e840 con 0x7fa1c00cd680 2023-10-11T09:52:02.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.655+0000 7fa1c5e41700 1 -- 172.21.15.78:0/2423702874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1c00cf210 con 0x7fa1c00cd680 2023-10-11T09:52:02.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.655+0000 7fa1bd7fa700 1 -- 172.21.15.78:0/2423702874 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fa1ac013b50 con 0x7fa1c00cd680 2023-10-11T09:52:02.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.655+0000 7fa1bd7fa700 1 --2- 172.21.15.78:0/2423702874 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa1a4065ed0 0x7fa1a4068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:02.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.659+0000 7fa1bd7fa700 1 -- 172.21.15.78:0/2423702874 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa1ac083600 con 0x7fa1c00cd680 2023-10-11T09:52:02.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.659+0000 7fa1c4e3f700 1 --2- 172.21.15.78:0/2423702874 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa1a4065ed0 0x7fa1a4068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:02.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.659+0000 7fa1c4e3f700 1 --2- 172.21.15.78:0/2423702874 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa1a4065ed0 0x7fa1a4068380 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4009c50 tx=0x7fa1b4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:02.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.663+0000 7fa1bd7fa700 1 -- 172.21.15.78:0/2423702874 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fa1c00cf210 con 0x7fa1c00cd680 2023-10-11T09:52:02.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.875+0000 7fa1c5e41700 1 -- 172.21.15.78:0/2423702874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa1c00cf210 con 0x7fa1c00cd680 2023-10-11T09:52:02.883 INFO:tasks.ceph.mon.a.smithi078.stderr:2023-10-11T09:52:02.875+0000 7f90131f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-11T09:52:02.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.891+0000 7fa1bd7fa700 1 -- 172.21.15.78:0/2423702874 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v100) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa1c00cf210 con 0x7fa1c00cd680 2023-10-11T09:52:02.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.895+0000 7fa1c5e41700 1 -- 172.21.15.78:0/2423702874 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa1a4065ed0 msgr2=0x7fa1a4068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:02.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.895+0000 7fa1c5e41700 1 --2- 172.21.15.78:0/2423702874 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa1a4065ed0 0x7fa1a4068380 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4009c50 tx=0x7fa1b4008030 comp rx=0 tx=0).stop 2023-10-11T09:52:02.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.895+0000 7fa1c5e41700 1 -- 172.21.15.78:0/2423702874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1c00cd680 msgr2=0x7fa1c015a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:02.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.895+0000 7fa1c5e41700 1 --2- 172.21.15.78:0/2423702874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1c00cd680 0x7fa1c015a5d0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00e840 tx=0x7fa1ac00eb50 comp rx=0 tx=0).stop 2023-10-11T09:52:02.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.895+0000 7fa1c5e41700 1 -- 172.21.15.78:0/2423702874 shutdown_connections 2023-10-11T09:52:02.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.895+0000 7fa1c5e41700 1 --2- 172.21.15.78:0/2423702874 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa1a4065ed0 0x7fa1a4068380 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:02.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.895+0000 7fa1c5e41700 1 --2- 172.21.15.78:0/2423702874 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa1c00cdf90 0x7fa1c015daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:02.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.895+0000 7fa1c5e41700 1 --2- 172.21.15.78:0/2423702874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1c00cd680 0x7fa1c015a5d0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:02.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.895+0000 7fa1c5e41700 1 --2- 172.21.15.78:0/2423702874 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1c006b290 0x7fa1c015fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:02.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.895+0000 7fa1c5e41700 1 -- 172.21.15.78:0/2423702874 >> 172.21.15.78:0/2423702874 conn(0x7fa1c006aaa0 msgr2=0x7fa1c00d0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:02.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.895+0000 7fa1c5e41700 1 -- 172.21.15.78:0/2423702874 shutdown_connections 2023-10-11T09:52:02.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.895+0000 7fa1c5e41700 1 -- 172.21.15.78:0/2423702874 wait complete. 2023-10-11T09:52:02.906 INFO:teuthology.orchestra.run.smithi078.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-11T09:52:02.924 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 10000000000.01000000 2023-10-11T09:52:02.976 INFO:tasks.ceph.mds.d.smithi133.stdout:starting mds.d at 2023-10-11T09:52:02.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.983+0000 7f11c5153140 1 Processor -- start 2023-10-11T09:52:02.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.983+0000 7f11c5153140 1 -- start start 2023-10-11T09:52:02.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.987+0000 7f11c5153140 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fb0972f960 0x55fb096ffc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:02.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.987+0000 7f11c5153140 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb0974a7a0 0x55fb097333a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:02.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.987+0000 7f11c5153140 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb097d7820 0x55fb097d6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:02.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.987+0000 7f11c5153140 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb09693ce0 con 0x55fb097d7820 2023-10-11T09:52:02.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.987+0000 7f11c5153140 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb09650250 con 0x55fb0974a7a0 2023-10-11T09:52:02.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.987+0000 7f11c5153140 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb0964fef0 con 0x55fb0972f960 2023-10-11T09:52:02.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.987+0000 7f11c3489700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb097d7820 0x55fb097d6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:02.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.987+0000 7f11c3489700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb097d7820 0x55fb097d6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49612/0 (socket says 172.21.15.78:49612) 2023-10-11T09:52:02.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.987+0000 7f11c3489700 1 -- 172.21.15.78:0/1578294810 learned_addr learned my addr 172.21.15.78:0/1578294810 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:02.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.987+0000 7f11c2c88700 1 --2- 172.21.15.78:0/1578294810 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fb0972f960 0x55fb096ffc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:02.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.987+0000 7f11ba487700 1 --2- 172.21.15.78:0/1578294810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb0974a7a0 0x55fb097333a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:02.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.987+0000 7f11c3489700 1 -- 172.21.15.78:0/1578294810 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fb0972f960 msgr2=0x55fb096ffc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:02.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.987+0000 7f11c3489700 1 --2- 172.21.15.78:0/1578294810 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fb0972f960 0x55fb096ffc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:02.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.987+0000 7f11c3489700 1 -- 172.21.15.78:0/1578294810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb0974a7a0 msgr2=0x55fb097333a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:02.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.987+0000 7f11c3489700 1 --2- 172.21.15.78:0/1578294810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb0974a7a0 0x55fb097333a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:02.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.987+0000 7f11c3489700 1 -- 172.21.15.78:0/1578294810 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ac005de0 con 0x55fb097d7820 2023-10-11T09:52:02.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.987+0000 7f11c3489700 1 --2- 172.21.15.78:0/1578294810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb097d7820 0x55fb097d6b00 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f11bc006fb0 tx=0x7f11bc001820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a267d9fc1a50b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:03.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.987+0000 7f11c1c86700 1 -- 172.21.15.78:0/1578294810 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11bc012600 con 0x55fb097d7820 2023-10-11T09:52:03.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.987+0000 7f11c1c86700 1 -- 172.21.15.78:0/1578294810 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11bc012760 con 0x55fb097d7820 2023-10-11T09:52:03.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.987+0000 7f11c1c86700 1 -- 172.21.15.78:0/1578294810 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11bc012af0 con 0x55fb097d7820 2023-10-11T09:52:03.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.987+0000 7f11c5153140 1 -- 172.21.15.78:0/1578294810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb097d7820 msgr2=0x55fb097d6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:03.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.987+0000 7f11c5153140 1 --2- 172.21.15.78:0/1578294810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb097d7820 0x55fb097d6b00 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f11bc006fb0 tx=0x7f11bc001820 comp rx=0 tx=0).stop 2023-10-11T09:52:03.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.987+0000 7f11c5153140 1 -- 172.21.15.78:0/1578294810 shutdown_connections 2023-10-11T09:52:03.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.987+0000 7f11c5153140 1 --2- 172.21.15.78:0/1578294810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb097d7820 0x55fb097d6b00 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:03.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.987+0000 7f11c5153140 1 --2- 172.21.15.78:0/1578294810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb0974a7a0 0x55fb097333a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:03.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.987+0000 7f11c5153140 1 --2- 172.21.15.78:0/1578294810 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fb0972f960 0x55fb096ffc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:03.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.987+0000 7f11c5153140 1 -- 172.21.15.78:0/1578294810 >> 172.21.15.78:0/1578294810 conn(0x55fb0972e550 msgr2=0x55fb097d80f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:03.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.987+0000 7f11c5153140 1 -- 172.21.15.78:0/1578294810 shutdown_connections 2023-10-11T09:52:03.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.987+0000 7f11c5153140 1 -- 172.21.15.78:0/1578294810 wait complete. 2023-10-11T09:52:03.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.991+0000 7f11c5153140 1 Processor -- start 2023-10-11T09:52:03.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.991+0000 7f11c5153140 1 -- start start 2023-10-11T09:52:03.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.991+0000 7f11c5153140 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb0972f960 0x55fb098616a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:03.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.991+0000 7f11c5153140 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb0974a7a0 0x55fb097d5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:03.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.991+0000 7f11c5153140 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fb097d7820 0x55fb097d6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:03.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.991+0000 7f11c5153140 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb09626240 con 0x55fb0974a7a0 2023-10-11T09:52:03.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.991+0000 7f11c5153140 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb09861be0 con 0x55fb0972f960 2023-10-11T09:52:03.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.991+0000 7f11c5153140 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb09733b60 con 0x55fb097d7820 2023-10-11T09:52:03.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.991+0000 7f11ba487700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb0974a7a0 0x55fb097d5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:03.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.991+0000 7f11ba487700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb0974a7a0 0x55fb097d5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49620/0 (socket says 172.21.15.78:49620) 2023-10-11T09:52:03.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.991+0000 7f11ba487700 1 -- 172.21.15.78:0/4021690400 learned_addr learned my addr 172.21.15.78:0/4021690400 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:03.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.991+0000 7f11c3489700 1 --2- 172.21.15.78:0/4021690400 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fb097d7820 0x55fb097d6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:03.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.991+0000 7f11ba487700 1 -- 172.21.15.78:0/4021690400 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fb097d7820 msgr2=0x55fb097d6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:03.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.991+0000 7f11c2c88700 1 --2- 172.21.15.78:0/4021690400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb0972f960 0x55fb098616a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:03.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.991+0000 7f11ba487700 1 --2- 172.21.15.78:0/4021690400 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fb097d7820 0x55fb097d6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:03.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.991+0000 7f11ba487700 1 -- 172.21.15.78:0/4021690400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb0972f960 msgr2=0x55fb098616a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:03.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.991+0000 7f11ba487700 1 --2- 172.21.15.78:0/4021690400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb0972f960 0x55fb098616a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:03.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.991+0000 7f11ba487700 1 -- 172.21.15.78:0/4021690400 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb09631520 con 0x55fb0974a7a0 2023-10-11T09:52:03.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.991+0000 7f11ba487700 1 --2- 172.21.15.78:0/4021690400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb0974a7a0 0x55fb097d5e50 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f11ac002000 tx=0x7f11ac002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:03.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.991+0000 7f11c0c84700 1 -- 172.21.15.78:0/4021690400 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ac012860 con 0x55fb0974a7a0 2023-10-11T09:52:03.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.991+0000 7f11c0c84700 1 -- 172.21.15.78:0/4021690400 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11ac012a00 con 0x55fb0974a7a0 2023-10-11T09:52:03.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.991+0000 7f11c5153140 1 -- 172.21.15.78:0/4021690400 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fb096318b0 con 0x55fb0974a7a0 2023-10-11T09:52:03.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.991+0000 7f11c0c84700 1 -- 172.21.15.78:0/4021690400 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ac012db0 con 0x55fb0974a7a0 2023-10-11T09:52:03.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.991+0000 7f11c5153140 1 -- 172.21.15.78:0/4021690400 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb09631310 con 0x55fb0974a7a0 2023-10-11T09:52:03.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.995+0000 7f11c0c84700 1 -- 172.21.15.78:0/4021690400 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f11ac019030 con 0x55fb0974a7a0 2023-10-11T09:52:03.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.995+0000 7f11c0c84700 1 --2- 172.21.15.78:0/4021690400 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f11a4064800 0x7f11a4066cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:03.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.995+0000 7f11c0c84700 1 -- 172.21.15.78:0/4021690400 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f11ac083c20 con 0x55fb0974a7a0 2023-10-11T09:52:03.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.995+0000 7f11c2c88700 1 --2- 172.21.15.78:0/4021690400 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f11a4064800 0x7f11a4066cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:03.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.995+0000 7f11c5153140 1 --2- 172.21.15.78:0/4021690400 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55fb09747f60 0x55fb09748360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:03.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.995+0000 7f11c5153140 1 -- 172.21.15.78:0/4021690400 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:1 17.5 17:a40620de:::10000000000.01000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55fb097488a0 con 0x55fb09747f60 2023-10-11T09:52:03.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.995+0000 7f11c2c88700 1 --2- 172.21.15.78:0/4021690400 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f11a4064800 0x7f11a4066cb0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f11b400b500 tx=0x7f11b4005620 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:03.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.995+0000 7f11c3489700 1 --2- 172.21.15.78:0/4021690400 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55fb09747f60 0x55fb09748360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:03.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:02.999+0000 7f11c3489700 1 --2- 172.21.15.78:0/4021690400 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55fb09747f60 0x55fb09748360 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:03.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.003+0000 7f11c3489700 1 -- 172.21.15.78:0/4021690400 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== osd_op_reply(1 10000000000.01000000 [omap-get-keys out=560b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+560 (crc 0 0 0) 0x7f11bc0128d0 con 0x55fb09747f60 2023-10-11T09:52:03.014 INFO:teuthology.orchestra.run.smithi078.stdout:10_head 2023-10-11T09:52:03.014 INFO:teuthology.orchestra.run.smithi078.stdout:11_head 2023-10-11T09:52:03.014 INFO:teuthology.orchestra.run.smithi078.stdout:12_head 2023-10-11T09:52:03.014 INFO:teuthology.orchestra.run.smithi078.stdout:15_head 2023-10-11T09:52:03.014 INFO:teuthology.orchestra.run.smithi078.stdout:19_head 2023-10-11T09:52:03.015 INFO:teuthology.orchestra.run.smithi078.stdout:1_head 2023-10-11T09:52:03.015 INFO:teuthology.orchestra.run.smithi078.stdout:20_head 2023-10-11T09:52:03.015 INFO:teuthology.orchestra.run.smithi078.stdout:21_head 2023-10-11T09:52:03.015 INFO:teuthology.orchestra.run.smithi078.stdout:27_head 2023-10-11T09:52:03.015 INFO:teuthology.orchestra.run.smithi078.stdout:28_head 2023-10-11T09:52:03.016 INFO:teuthology.orchestra.run.smithi078.stdout:2_head 2023-10-11T09:52:03.016 INFO:teuthology.orchestra.run.smithi078.stdout:31_head 2023-10-11T09:52:03.016 INFO:teuthology.orchestra.run.smithi078.stdout:34_head 2023-10-11T09:52:03.016 INFO:teuthology.orchestra.run.smithi078.stdout:36_head 2023-10-11T09:52:03.016 INFO:teuthology.orchestra.run.smithi078.stdout:38_head 2023-10-11T09:52:03.017 INFO:teuthology.orchestra.run.smithi078.stdout:39_head 2023-10-11T09:52:03.017 INFO:teuthology.orchestra.run.smithi078.stdout:3_head 2023-10-11T09:52:03.017 INFO:teuthology.orchestra.run.smithi078.stdout:42_head 2023-10-11T09:52:03.017 INFO:teuthology.orchestra.run.smithi078.stdout:43_head 2023-10-11T09:52:03.017 INFO:teuthology.orchestra.run.smithi078.stdout:46_head 2023-10-11T09:52:03.017 INFO:teuthology.orchestra.run.smithi078.stdout:4_head 2023-10-11T09:52:03.018 INFO:teuthology.orchestra.run.smithi078.stdout:50_head 2023-10-11T09:52:03.018 INFO:teuthology.orchestra.run.smithi078.stdout:52_head 2023-10-11T09:52:03.018 INFO:teuthology.orchestra.run.smithi078.stdout:53_head 2023-10-11T09:52:03.018 INFO:teuthology.orchestra.run.smithi078.stdout:55_head 2023-10-11T09:52:03.018 INFO:teuthology.orchestra.run.smithi078.stdout:59_head 2023-10-11T09:52:03.019 INFO:teuthology.orchestra.run.smithi078.stdout:61_head 2023-10-11T09:52:03.019 INFO:teuthology.orchestra.run.smithi078.stdout:63_head 2023-10-11T09:52:03.019 INFO:teuthology.orchestra.run.smithi078.stdout:64_head 2023-10-11T09:52:03.019 INFO:teuthology.orchestra.run.smithi078.stdout:65_head 2023-10-11T09:52:03.019 INFO:teuthology.orchestra.run.smithi078.stdout:66_head 2023-10-11T09:52:03.020 INFO:teuthology.orchestra.run.smithi078.stdout:69_head 2023-10-11T09:52:03.020 INFO:teuthology.orchestra.run.smithi078.stdout:70_head 2023-10-11T09:52:03.020 INFO:teuthology.orchestra.run.smithi078.stdout:73_head 2023-10-11T09:52:03.020 INFO:teuthology.orchestra.run.smithi078.stdout:74_head 2023-10-11T09:52:03.020 INFO:teuthology.orchestra.run.smithi078.stdout:75_head 2023-10-11T09:52:03.021 INFO:teuthology.orchestra.run.smithi078.stdout:77_head 2023-10-11T09:52:03.021 INFO:teuthology.orchestra.run.smithi078.stdout:80_head 2023-10-11T09:52:03.021 INFO:teuthology.orchestra.run.smithi078.stdout:83_head 2023-10-11T09:52:03.021 INFO:teuthology.orchestra.run.smithi078.stdout:84_head 2023-10-11T09:52:03.021 INFO:teuthology.orchestra.run.smithi078.stdout:85_head 2023-10-11T09:52:03.022 INFO:teuthology.orchestra.run.smithi078.stdout:86_head 2023-10-11T09:52:03.022 INFO:teuthology.orchestra.run.smithi078.stdout:87_head 2023-10-11T09:52:03.022 INFO:teuthology.orchestra.run.smithi078.stdout:88_head 2023-10-11T09:52:03.022 INFO:teuthology.orchestra.run.smithi078.stdout:8_head 2023-10-11T09:52:03.022 INFO:teuthology.orchestra.run.smithi078.stdout:90_head 2023-10-11T09:52:03.023 INFO:teuthology.orchestra.run.smithi078.stdout:92_head 2023-10-11T09:52:03.023 INFO:teuthology.orchestra.run.smithi078.stdout:95_head 2023-10-11T09:52:03.023 INFO:teuthology.orchestra.run.smithi078.stdout:96_head 2023-10-11T09:52:03.023 INFO:teuthology.orchestra.run.smithi078.stdout:97_head 2023-10-11T09:52:03.023 INFO:teuthology.orchestra.run.smithi078.stdout:9_head 2023-10-11T09:52:03.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.003+0000 7f11c5153140 1 -- 172.21.15.78:0/4021690400 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55fb09747f60 msgr2=0x55fb09748360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:03.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.003+0000 7f11c5153140 1 --2- 172.21.15.78:0/4021690400 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55fb09747f60 0x55fb09748360 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:03.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.003+0000 7f11c5153140 1 -- 172.21.15.78:0/4021690400 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f11a4064800 msgr2=0x7f11a4066cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:03.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.003+0000 7f11c5153140 1 --2- 172.21.15.78:0/4021690400 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f11a4064800 0x7f11a4066cb0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f11b400b500 tx=0x7f11b4005620 comp rx=0 tx=0).stop 2023-10-11T09:52:03.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.003+0000 7f11c5153140 1 -- 172.21.15.78:0/4021690400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb0974a7a0 msgr2=0x55fb097d5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:03.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.003+0000 7f11c5153140 1 --2- 172.21.15.78:0/4021690400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb0974a7a0 0x55fb097d5e50 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f11ac002000 tx=0x7f11ac002fb0 comp rx=0 tx=0).stop 2023-10-11T09:52:03.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.003+0000 7f11c3489700 1 -- 172.21.15.78:0/4021690400 reap_dead start 2023-10-11T09:52:03.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.003+0000 7f11c5153140 1 -- 172.21.15.78:0/4021690400 shutdown_connections 2023-10-11T09:52:03.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.003+0000 7f11c5153140 1 -- 172.21.15.78:0/4021690400 >> 172.21.15.78:0/4021690400 conn(0x55fb0972e550 msgr2=0x55fb097da850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:03.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.003+0000 7f11c5153140 1 -- 172.21.15.78:0/4021690400 shutdown_connections 2023-10-11T09:52:03.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.003+0000 7f11c5153140 1 -- 172.21.15.78:0/4021690400 wait complete. 2023-10-11T09:52:03.027 INFO:tasks.cephfs.test_data_scan:victim_key=21_head 2023-10-11T09:52:03.028 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000000.01000000 21_head 2023-10-11T09:52:03.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.091+0000 7fb274f76140 1 Processor -- start 2023-10-11T09:52:03.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.091+0000 7fb274f76140 1 -- start start 2023-10-11T09:52:03.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.091+0000 7fb274f76140 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564dce2e56e0 0x564dce3c0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:03.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.091+0000 7fb274f76140 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564dce319af0 0x564dce3ba4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:03.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.091+0000 7fb274f76140 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564dce3bd7e0 0x564dce44f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:03.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.091+0000 7fb274f76140 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dce236250 con 0x564dce319af0 2023-10-11T09:52:03.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.091+0000 7fb274f76140 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dce235ef0 con 0x564dce2e56e0 2023-10-11T09:52:03.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.091+0000 7fb274f76140 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x564dce20c240 con 0x564dce3bd7e0 2023-10-11T09:52:03.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.091+0000 7fb2722aa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564dce319af0 0x564dce3ba4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:03.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.091+0000 7fb2722aa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564dce319af0 0x564dce3ba4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49646/0 (socket says 172.21.15.78:49646) 2023-10-11T09:52:03.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.091+0000 7fb2722aa700 1 -- 172.21.15.78:0/804212286 learned_addr learned my addr 172.21.15.78:0/804212286 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:03.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.091+0000 7fb272aab700 1 --2- 172.21.15.78:0/804212286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564dce2e56e0 0x564dce3c0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:03.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.091+0000 7fb2732ac700 1 --2- 172.21.15.78:0/804212286 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564dce3bd7e0 0x564dce44f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:03.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.091+0000 7fb2722aa700 1 -- 172.21.15.78:0/804212286 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564dce3bd7e0 msgr2=0x564dce44f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:03.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.091+0000 7fb2722aa700 1 --2- 172.21.15.78:0/804212286 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564dce3bd7e0 0x564dce44f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:03.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.091+0000 7fb2722aa700 1 -- 172.21.15.78:0/804212286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564dce2e56e0 msgr2=0x564dce3c0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:03.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.091+0000 7fb2722aa700 1 --2- 172.21.15.78:0/804212286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564dce2e56e0 0x564dce3c0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:03.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.091+0000 7fb2722aa700 1 -- 172.21.15.78:0/804212286 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb26400a1c0 con 0x564dce319af0 2023-10-11T09:52:03.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.091+0000 7fb272aab700 1 --2- 172.21.15.78:0/804212286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564dce2e56e0 0x564dce3c0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:52:03.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.091+0000 7fb2732ac700 1 --2- 172.21.15.78:0/804212286 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564dce3bd7e0 0x564dce44f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:52:03.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.091+0000 7fb2722aa700 1 --2- 172.21.15.78:0/804212286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564dce319af0 0x564dce3ba4e0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fb26800a7d0 tx=0x7fb26800aae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb7f800e9165d0eb server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:03.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.091+0000 7fb2712a8700 1 -- 172.21.15.78:0/804212286 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26800f070 con 0x564dce319af0 2023-10-11T09:52:03.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.091+0000 7fb2712a8700 1 -- 172.21.15.78:0/804212286 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2680087b0 con 0x564dce319af0 2023-10-11T09:52:03.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.091+0000 7fb2712a8700 1 -- 172.21.15.78:0/804212286 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb268008b20 con 0x564dce319af0 2023-10-11T09:52:03.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.095+0000 7fb274f76140 1 -- 172.21.15.78:0/804212286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564dce319af0 msgr2=0x564dce3ba4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:03.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.095+0000 7fb274f76140 1 --2- 172.21.15.78:0/804212286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564dce319af0 0x564dce3ba4e0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fb26800a7d0 tx=0x7fb26800aae0 comp rx=0 tx=0).stop 2023-10-11T09:52:03.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.095+0000 7fb274f76140 1 -- 172.21.15.78:0/804212286 shutdown_connections 2023-10-11T09:52:03.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.095+0000 7fb274f76140 1 --2- 172.21.15.78:0/804212286 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564dce3bd7e0 0x564dce44f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:03.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.095+0000 7fb274f76140 1 --2- 172.21.15.78:0/804212286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564dce319af0 0x564dce3ba4e0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:03.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.095+0000 7fb274f76140 1 --2- 172.21.15.78:0/804212286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564dce2e56e0 0x564dce3c0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:03.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.095+0000 7fb274f76140 1 -- 172.21.15.78:0/804212286 >> 172.21.15.78:0/804212286 conn(0x564dce2ebe00 msgr2=0x564dce3bb7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:03.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.095+0000 7fb274f76140 1 -- 172.21.15.78:0/804212286 shutdown_connections 2023-10-11T09:52:03.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.095+0000 7fb274f76140 1 -- 172.21.15.78:0/804212286 wait complete. 2023-10-11T09:52:03.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.095+0000 7fb274f76140 1 Processor -- start 2023-10-11T09:52:03.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.095+0000 7fb274f76140 1 -- start start 2023-10-11T09:52:03.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.095+0000 7fb274f76140 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564dce2e56e0 0x564dce3191b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:03.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.095+0000 7fb274f76140 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564dce319af0 0x564dce315800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:03.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.095+0000 7fb274f76140 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564dce3bd7e0 0x564dce456550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:03.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.095+0000 7fb274f76140 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dce3b83e0 con 0x564dce2e56e0 2023-10-11T09:52:03.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.095+0000 7fb274f76140 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dce3b8680 con 0x564dce319af0 2023-10-11T09:52:03.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.095+0000 7fb274f76140 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x564dce3b8920 con 0x564dce3bd7e0 2023-10-11T09:52:03.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.095+0000 7fb272aab700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564dce2e56e0 0x564dce3191b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:03.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.095+0000 7fb272aab700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564dce2e56e0 0x564dce3191b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49652/0 (socket says 172.21.15.78:49652) 2023-10-11T09:52:03.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.095+0000 7fb272aab700 1 -- 172.21.15.78:0/2797741564 learned_addr learned my addr 172.21.15.78:0/2797741564 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:03.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.095+0000 7fb2732ac700 1 --2- 172.21.15.78:0/2797741564 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564dce3bd7e0 0x564dce456550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:03.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.095+0000 7fb2722aa700 1 --2- 172.21.15.78:0/2797741564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564dce319af0 0x564dce315800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:03.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.095+0000 7fb272aab700 1 -- 172.21.15.78:0/2797741564 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564dce3bd7e0 msgr2=0x564dce456550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:03.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.095+0000 7fb272aab700 1 --2- 172.21.15.78:0/2797741564 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564dce3bd7e0 0x564dce456550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:03.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.095+0000 7fb272aab700 1 -- 172.21.15.78:0/2797741564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564dce319af0 msgr2=0x564dce315800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:03.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.095+0000 7fb272aab700 1 --2- 172.21.15.78:0/2797741564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564dce319af0 0x564dce315800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:03.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.095+0000 7fb272aab700 1 -- 172.21.15.78:0/2797741564 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb26800a420 con 0x564dce2e56e0 2023-10-11T09:52:03.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.095+0000 7fb2722aa700 1 --2- 172.21.15.78:0/2797741564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564dce319af0 0x564dce315800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:03.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.095+0000 7fb272aab700 1 --2- 172.21.15.78:0/2797741564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564dce2e56e0 0x564dce3191b0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fb26400a880 tx=0x7fb264003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:03.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.099+0000 7fb25bfff700 1 -- 172.21.15.78:0/2797741564 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb264010910 con 0x564dce2e56e0 2023-10-11T09:52:03.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.099+0000 7fb25bfff700 1 -- 172.21.15.78:0/2797741564 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb264010a70 con 0x564dce2e56e0 2023-10-11T09:52:03.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.099+0000 7fb25bfff700 1 -- 172.21.15.78:0/2797741564 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb264010e20 con 0x564dce2e56e0 2023-10-11T09:52:03.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.099+0000 7fb274f76140 1 -- 172.21.15.78:0/2797741564 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564dce224470 con 0x564dce2e56e0 2023-10-11T09:52:03.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.099+0000 7fb274f76140 1 -- 172.21.15.78:0/2797741564 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564dce2178b0 con 0x564dce2e56e0 2023-10-11T09:52:03.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.099+0000 7fb25bfff700 1 -- 172.21.15.78:0/2797741564 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fb264017070 con 0x564dce2e56e0 2023-10-11T09:52:03.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.099+0000 7fb25bfff700 1 --2- 172.21.15.78:0/2797741564 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb260064860 0x7fb260066d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:03.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.099+0000 7fb25bfff700 1 -- 172.21.15.78:0/2797741564 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fb264083ef0 con 0x564dce2e56e0 2023-10-11T09:52:03.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.099+0000 7fb2722aa700 1 --2- 172.21.15.78:0/2797741564 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb260064860 0x7fb260066d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:03.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.099+0000 7fb274f76140 1 --2- 172.21.15.78:0/2797741564 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x564dce3196f0 0x564dce3b7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:03.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.103+0000 7fb274f76140 1 -- 172.21.15.78:0/2797741564 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:1 17.5 17:a40620de:::10000000000.01000000:head [omap-rm-keys in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x564dce3b7590 con 0x564dce3196f0 2023-10-11T09:52:03.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.103+0000 7fb2722aa700 1 --2- 172.21.15.78:0/2797741564 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb260064860 0x7fb260066d10 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fb268000c00 tx=0x7fb268011000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:03.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.103+0000 7fb2732ac700 1 --2- 172.21.15.78:0/2797741564 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x564dce3196f0 0x564dce3b7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:03.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.103+0000 7fb2732ac700 1 --2- 172.21.15.78:0/2797741564 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x564dce3196f0 0x564dce3b7050 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:03.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.107+0000 7fb2732ac700 1 -- 172.21.15.78:0/2797741564 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== osd_op_reply(1 10000000000.01000000 [omap-rm-keys] v128'3 uv3 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fb26c00a040 con 0x564dce3196f0 2023-10-11T09:52:03.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.107+0000 7fb274f76140 1 -- 172.21.15.78:0/2797741564 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x564dce3196f0 msgr2=0x564dce3b7050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:03.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.107+0000 7fb274f76140 1 --2- 172.21.15.78:0/2797741564 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x564dce3196f0 0x564dce3b7050 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:03.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.107+0000 7fb274f76140 1 -- 172.21.15.78:0/2797741564 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb260064860 msgr2=0x7fb260066d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:03.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.107+0000 7fb274f76140 1 --2- 172.21.15.78:0/2797741564 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb260064860 0x7fb260066d10 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fb268000c00 tx=0x7fb268011000 comp rx=0 tx=0).stop 2023-10-11T09:52:03.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.107+0000 7fb274f76140 1 -- 172.21.15.78:0/2797741564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564dce2e56e0 msgr2=0x564dce3191b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:03.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.107+0000 7fb274f76140 1 --2- 172.21.15.78:0/2797741564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564dce2e56e0 0x564dce3191b0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fb26400a880 tx=0x7fb264003330 comp rx=0 tx=0).stop 2023-10-11T09:52:03.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.107+0000 7fb2732ac700 1 -- 172.21.15.78:0/2797741564 reap_dead start 2023-10-11T09:52:03.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.107+0000 7fb274f76140 1 -- 172.21.15.78:0/2797741564 shutdown_connections 2023-10-11T09:52:03.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.107+0000 7fb274f76140 1 -- 172.21.15.78:0/2797741564 >> 172.21.15.78:0/2797741564 conn(0x564dce2ebe00 msgr2=0x564dce3bfc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:03.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.107+0000 7fb274f76140 1 -- 172.21.15.78:0/2797741564 shutdown_connections 2023-10-11T09:52:03.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.107+0000 7fb274f76140 1 -- 172.21.15.78:0/2797741564 wait complete. 2023-10-11T09:52:03.120 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-11T09:52:03.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.291+0000 7ff0749be700 1 Processor -- start 2023-10-11T09:52:03.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.291+0000 7ff0749be700 1 -- start start 2023-10-11T09:52:03.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.291+0000 7ff0749be700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ff07006c8a0 0x7ff0700cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:03.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.291+0000 7ff0749be700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff0700cc230 0x7ff0700cc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:03.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.291+0000 7ff0749be700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0701648a0 con 0x7ff070160de0 2023-10-11T09:52:03.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.291+0000 7ff0749be700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0700cc6d0 con 0x7ff07006c8a0 2023-10-11T09:52:03.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.291+0000 7ff0749be700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff070164b80 con 0x7ff0700cc230 2023-10-11T09:52:03.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.291+0000 7ff06e59c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ff07006c8a0 0x7ff0700cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:03.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.295+0000 7ff06e59c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ff07006c8a0 0x7ff0700cbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49660/0 (socket says 172.21.15.78:49660) 2023-10-11T09:52:03.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.295+0000 7ff06e59c700 1 -- 172.21.15.78:0/619599469 learned_addr learned my addr 172.21.15.78:0/619599469 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:03.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.295+0000 7ff06f59e700 1 --2- 172.21.15.78:0/619599469 >> v2:172.21.15.133:3300/0 conn(0x7ff0700cc230 0x7ff0700cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:03.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.295+0000 7ff06e59c700 1 -- 172.21.15.78:0/619599469 >> v2:172.21.15.133:3300/0 conn(0x7ff0700cc230 msgr2=0x7ff0700cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:03.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.295+0000 7ff06e59c700 1 --2- 172.21.15.78:0/619599469 >> v2:172.21.15.133:3300/0 conn(0x7ff0700cc230 0x7ff0700cc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:03.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.295+0000 7ff06e59c700 1 -- 172.21.15.78:0/619599469 >> v1:172.21.15.133:6789/0 conn(0x7ff070160de0 legacy=0x7ff07006c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:03.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.295+0000 7ff06e59c700 1 -- 172.21.15.78:0/619599469 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff070164e60 con 0x7ff07006c8a0 2023-10-11T09:52:03.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.295+0000 7ff06e59c700 1 --2- 172.21.15.78:0/619599469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff07006c8a0 0x7ff0700cbcf0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7ff07006f900 tx=0x7ff064009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755ae152ed945b9d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:03.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.295+0000 7ff06dd9b700 1 -- 172.21.15.78:0/619599469 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff064011830 con 0x7ff07006c8a0 2023-10-11T09:52:03.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.295+0000 7ff06dd9b700 1 -- 172.21.15.78:0/619599469 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff064011990 con 0x7ff07006c8a0 2023-10-11T09:52:03.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.295+0000 7ff06dd9b700 1 -- 172.21.15.78:0/619599469 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff064011d20 con 0x7ff07006c8a0 2023-10-11T09:52:03.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.295+0000 7ff0749be700 1 -- 172.21.15.78:0/619599469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff07006c8a0 msgr2=0x7ff0700cbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:03.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.295+0000 7ff0749be700 1 --2- 172.21.15.78:0/619599469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff07006c8a0 0x7ff0700cbcf0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7ff07006f900 tx=0x7ff064009bf0 comp rx=0 tx=0).stop 2023-10-11T09:52:03.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.295+0000 7ff0749be700 1 -- 172.21.15.78:0/619599469 shutdown_connections 2023-10-11T09:52:03.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.295+0000 7ff0749be700 1 --2- 172.21.15.78:0/619599469 >> v2:172.21.15.133:3300/0 conn(0x7ff0700cc230 0x7ff0700cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:03.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.295+0000 7ff0749be700 1 --2- 172.21.15.78:0/619599469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff07006c8a0 0x7ff0700cbcf0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:03.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.295+0000 7ff0749be700 1 -- 172.21.15.78:0/619599469 >> 172.21.15.78:0/619599469 conn(0x7ff070073100 msgr2=0x7ff0700734d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:03.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.295+0000 7ff0749be700 1 -- 172.21.15.78:0/619599469 shutdown_connections 2023-10-11T09:52:03.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.295+0000 7ff0749be700 1 -- 172.21.15.78:0/619599469 wait complete. 2023-10-11T09:52:03.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.299+0000 7ff0749be700 1 Processor -- start 2023-10-11T09:52:03.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.299+0000 7ff0749be700 1 -- start start 2023-10-11T09:52:03.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.303+0000 7ff0749be700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff07006c8a0 0x7ff07015a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:03.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.303+0000 7ff0749be700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff0700cc230 0x7ff0700b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:03.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.303+0000 7ff0749be700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff070160de0 0x7ff0700b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:03.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.303+0000 7ff0749be700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0700c4950 con 0x7ff07006c8a0 2023-10-11T09:52:03.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.303+0000 7ff0749be700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0700c4c30 con 0x7ff070160de0 2023-10-11T09:52:03.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.303+0000 7ff0749be700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0700c4f10 con 0x7ff0700cc230 2023-10-11T09:52:03.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.303+0000 7ff06ed9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff07006c8a0 0x7ff07015a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:03.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.303+0000 7ff06ed9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff07006c8a0 0x7ff07015a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49670/0 (socket says 172.21.15.78:49670) 2023-10-11T09:52:03.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.303+0000 7ff06ed9d700 1 -- 172.21.15.78:0/3543372042 learned_addr learned my addr 172.21.15.78:0/3543372042 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:03.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.303+0000 7ff06e59c700 1 --2- 172.21.15.78:0/3543372042 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff0700cc230 0x7ff0700b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:03.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.303+0000 7ff06ed9d700 1 -- 172.21.15.78:0/3543372042 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff0700cc230 msgr2=0x7ff0700b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:03.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.303+0000 7ff06ed9d700 1 --2- 172.21.15.78:0/3543372042 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff0700cc230 0x7ff0700b36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:03.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.303+0000 7ff06ed9d700 1 -- 172.21.15.78:0/3543372042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff070160de0 msgr2=0x7ff0700b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:03.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.303+0000 7ff06f59e700 1 --2- 172.21.15.78:0/3543372042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff070160de0 0x7ff0700b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:03.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.303+0000 7ff06ed9d700 1 --2- 172.21.15.78:0/3543372042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff070160de0 0x7ff0700b3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:03.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.303+0000 7ff06ed9d700 1 -- 172.21.15.78:0/3543372042 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff064009570 con 0x7ff07006c8a0 2023-10-11T09:52:03.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.303+0000 7ff06e59c700 1 --2- 172.21.15.78:0/3543372042 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff0700cc230 0x7ff0700b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:52:03.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.303+0000 7ff06f59e700 1 --2- 172.21.15.78:0/3543372042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff070160de0 0x7ff0700b3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:52:03.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.303+0000 7ff06ed9d700 1 --2- 172.21.15.78:0/3543372042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff07006c8a0 0x7ff07015a020 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7ff06000c500 tx=0x7ff06000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:03.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.303+0000 7ff0577fe700 1 -- 172.21.15.78:0/3543372042 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0600052d0 con 0x7ff07006c8a0 2023-10-11T09:52:03.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.303+0000 7ff0577fe700 1 -- 172.21.15.78:0/3543372042 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff060005430 con 0x7ff07006c8a0 2023-10-11T09:52:03.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.303+0000 7ff0749be700 1 -- 172.21.15.78:0/3543372042 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0700c51f0 con 0x7ff07006c8a0 2023-10-11T09:52:03.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.303+0000 7ff0577fe700 1 -- 172.21.15.78:0/3543372042 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0600055a0 con 0x7ff07006c8a0 2023-10-11T09:52:03.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.303+0000 7ff0749be700 1 -- 172.21.15.78:0/3543372042 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0700b1c80 con 0x7ff07006c8a0 2023-10-11T09:52:03.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.303+0000 7ff0749be700 1 -- 172.21.15.78:0/3543372042 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0700744c0 con 0x7ff07006c8a0 2023-10-11T09:52:03.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.307+0000 7ff0577fe700 1 -- 172.21.15.78:0/3543372042 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7ff0600038e0 con 0x7ff07006c8a0 2023-10-11T09:52:03.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.307+0000 7ff0577fe700 1 --2- 172.21.15.78:0/3543372042 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff05c065f00 0x7ff05c0683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:03.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.307+0000 7ff0577fe700 1 -- 172.21.15.78:0/3543372042 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6148+0+0 (secure 0 0 0) 0x7ff060082c90 con 0x7ff07006c8a0 2023-10-11T09:52:03.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.307+0000 7ff06e59c700 1 --2- 172.21.15.78:0/3543372042 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff05c065f00 0x7ff05c0683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:03.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.311+0000 7ff06e59c700 1 --2- 172.21.15.78:0/3543372042 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff05c065f00 0x7ff05c0683b0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ff07006f900 tx=0x7ff064008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:03.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.315+0000 7ff0577fe700 1 -- 172.21.15.78:0/3543372042 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7ff060050310 con 0x7ff07006c8a0 2023-10-11T09:52:03.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.527+0000 7ff0749be700 1 -- 172.21.15.78:0/3543372042 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7ff0701622c0 con 0x7ff07006c8a0 2023-10-11T09:52:03.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.899+0000 7ff0577fe700 1 -- 172.21.15.78:0/3543372042 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v101) v1 ==== 162+0+0 (secure 0 0 0) 0x7ff0600170b0 con 0x7ff07006c8a0 2023-10-11T09:52:03.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.903+0000 7ff0749be700 1 -- 172.21.15.78:0/3543372042 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff05c065f00 msgr2=0x7ff05c0683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:03.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.903+0000 7ff0749be700 1 --2- 172.21.15.78:0/3543372042 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff05c065f00 0x7ff05c0683b0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ff07006f900 tx=0x7ff064008030 comp rx=0 tx=0).stop 2023-10-11T09:52:03.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.903+0000 7ff0749be700 1 -- 172.21.15.78:0/3543372042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff07006c8a0 msgr2=0x7ff07015a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:03.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.903+0000 7ff0749be700 1 --2- 172.21.15.78:0/3543372042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff07006c8a0 0x7ff07015a020 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7ff06000c500 tx=0x7ff06000c8c0 comp rx=0 tx=0).stop 2023-10-11T09:52:03.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.907+0000 7ff0749be700 1 -- 172.21.15.78:0/3543372042 shutdown_connections 2023-10-11T09:52:03.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.907+0000 7ff0749be700 1 --2- 172.21.15.78:0/3543372042 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff05c065f00 0x7ff05c0683b0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:03.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.907+0000 7ff0749be700 1 --2- 172.21.15.78:0/3543372042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff070160de0 0x7ff0700b3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:03.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.907+0000 7ff0749be700 1 --2- 172.21.15.78:0/3543372042 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff0700cc230 0x7ff0700b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:03.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.907+0000 7ff0749be700 1 --2- 172.21.15.78:0/3543372042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff07006c8a0 0x7ff07015a020 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:03.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.907+0000 7ff0749be700 1 -- 172.21.15.78:0/3543372042 >> 172.21.15.78:0/3543372042 conn(0x7ff070073100 msgr2=0x7ff0700cd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:03.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.907+0000 7ff0749be700 1 -- 172.21.15.78:0/3543372042 shutdown_connections 2023-10-11T09:52:03.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:03.907+0000 7ff0749be700 1 -- 172.21.15.78:0/3543372042 wait complete. 2023-10-11T09:52:03.917 INFO:teuthology.orchestra.run.smithi078.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-11T09:52:03.940 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:52:04.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.115+0000 7fb4c15f9700 1 Processor -- start 2023-10-11T09:52:04.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.115+0000 7fb4c15f9700 1 -- start start 2023-10-11T09:52:04.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.115+0000 7fb4c15f9700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb4bc15cc20 0x7fb4bc0d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:04.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.115+0000 7fb4c15f9700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4bc0d1820 con 0x7fb4bc0cd7f0 2023-10-11T09:52:04.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.115+0000 7fb4c15f9700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4bc0d1b00 con 0x7fb4bc0cfbb0 2023-10-11T09:52:04.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.115+0000 7fb4c15f9700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4bc0d1de0 con 0x7fb4bc15cc20 2023-10-11T09:52:04.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.115+0000 7fb4bb7fe700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fb4bc0cd7f0 0x7fb4bc0b5ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:34596/0 (socket says 172.21.15.78:34596) 2023-10-11T09:52:04.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.115+0000 7fb4bb7fe700 1 -- 172.21.15.78:0/1629239987 learned_addr learned my addr 172.21.15.78:0/1629239987 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:04.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.115+0000 7fb4ba7fc700 1 --2- 172.21.15.78:0/1629239987 >> v2:172.21.15.133:3300/0 conn(0x7fb4bc15cc20 0x7fb4bc0d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:04.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.119+0000 7fb4b9ffb700 1 -- 172.21.15.78:0/1629239987 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 893757790 0 0) 0x7fb4bc0d1820 con 0x7fb4bc0cd7f0 2023-10-11T09:52:04.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.119+0000 7fb4b9ffb700 1 -- 172.21.15.78:0/1629239987 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4a40035d0 con 0x7fb4bc0cd7f0 2023-10-11T09:52:04.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.119+0000 7fb4ba7fc700 1 -- 172.21.15.78:0/1629239987 >> v1:172.21.15.133:6789/0 conn(0x7fb4bc0cfbb0 legacy=0x7fb4bc0cff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:04.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.119+0000 7fb4ba7fc700 1 -- 172.21.15.78:0/1629239987 >> v1:172.21.15.78:6789/0 conn(0x7fb4bc0cd7f0 legacy=0x7fb4bc0b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:04.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.119+0000 7fb4ba7fc700 1 -- 172.21.15.78:0/1629239987 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4bc0d20c0 con 0x7fb4bc15cc20 2023-10-11T09:52:04.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.119+0000 7fb4ba7fc700 1 --2- 172.21.15.78:0/1629239987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4bc15cc20 0x7fb4bc0d28c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0009850 tx=0x7fb4b0009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a42e125308b4fdf server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:04.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.119+0000 7fb4b9ffb700 1 -- 172.21.15.78:0/1629239987 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0011830 con 0x7fb4bc15cc20 2023-10-11T09:52:04.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.119+0000 7fb4b9ffb700 1 -- 172.21.15.78:0/1629239987 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b0011990 con 0x7fb4bc15cc20 2023-10-11T09:52:04.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.119+0000 7fb4b9ffb700 1 -- 172.21.15.78:0/1629239987 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0011d40 con 0x7fb4bc15cc20 2023-10-11T09:52:04.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.119+0000 7fb4c15f9700 1 -- 172.21.15.78:0/1629239987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4bc15cc20 msgr2=0x7fb4bc0d28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:04.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.119+0000 7fb4c15f9700 1 --2- 172.21.15.78:0/1629239987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4bc15cc20 0x7fb4bc0d28c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0009850 tx=0x7fb4b0009b60 comp rx=0 tx=0).stop 2023-10-11T09:52:04.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.119+0000 7fb4c15f9700 1 -- 172.21.15.78:0/1629239987 shutdown_connections 2023-10-11T09:52:04.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.119+0000 7fb4c15f9700 1 --2- 172.21.15.78:0/1629239987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4bc15cc20 0x7fb4bc0d28c0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:04.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.119+0000 7fb4c15f9700 1 -- 172.21.15.78:0/1629239987 >> 172.21.15.78:0/1629239987 conn(0x7fb4bc06abf0 msgr2=0x7fb4bc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:04.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.119+0000 7fb4c15f9700 1 -- 172.21.15.78:0/1629239987 shutdown_connections 2023-10-11T09:52:04.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.119+0000 7fb4c15f9700 1 -- 172.21.15.78:0/1629239987 wait complete. 2023-10-11T09:52:04.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.123+0000 7fb4c15f9700 1 Processor -- start 2023-10-11T09:52:04.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.123+0000 7fb4c15f9700 1 -- start start 2023-10-11T09:52:04.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.123+0000 7fb4c15f9700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4bc0cd7f0 0x7fb4bc0b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:04.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.123+0000 7fb4c15f9700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb4bc0cfbb0 0x7fb4bc0b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:04.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.123+0000 7fb4c15f9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4bc15cc20 0x7fb4bc1f1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:04.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.123+0000 7fb4c15f9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc0b4ce0 con 0x7fb4bc15cc20 2023-10-11T09:52:04.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.123+0000 7fb4c15f9700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc0b4310 con 0x7fb4bc0cd7f0 2023-10-11T09:52:04.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.123+0000 7fb4c15f9700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc14bda0 con 0x7fb4bc0cfbb0 2023-10-11T09:52:04.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.123+0000 7fb4bb7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4bc15cc20 0x7fb4bc1f1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:04.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.123+0000 7fb4bb7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4bc15cc20 0x7fb4bc1f1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49680/0 (socket says 172.21.15.78:49680) 2023-10-11T09:52:04.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.123+0000 7fb4bb7fe700 1 -- 172.21.15.78:0/2144280043 learned_addr learned my addr 172.21.15.78:0/2144280043 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:04.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.123+0000 7fb4baffd700 1 --2- 172.21.15.78:0/2144280043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4bc0cd7f0 0x7fb4bc0b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:04.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.123+0000 7fb4ba7fc700 1 --2- 172.21.15.78:0/2144280043 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb4bc0cfbb0 0x7fb4bc0b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:04.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.127+0000 7fb4bb7fe700 1 -- 172.21.15.78:0/2144280043 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb4bc0cfbb0 msgr2=0x7fb4bc0b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:04.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.127+0000 7fb4bb7fe700 1 --2- 172.21.15.78:0/2144280043 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb4bc0cfbb0 0x7fb4bc0b4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:04.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.127+0000 7fb4bb7fe700 1 -- 172.21.15.78:0/2144280043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4bc0cd7f0 msgr2=0x7fb4bc0b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:04.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.127+0000 7fb4bb7fe700 1 --2- 172.21.15.78:0/2144280043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4bc0cd7f0 0x7fb4bc0b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:04.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.127+0000 7fb4bb7fe700 1 -- 172.21.15.78:0/2144280043 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b00094b0 con 0x7fb4bc15cc20 2023-10-11T09:52:04.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.127+0000 7fb4bb7fe700 1 --2- 172.21.15.78:0/2144280043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4bc15cc20 0x7fb4bc1f1520 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400cb40 tx=0x7fb4b400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:04.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.127+0000 7fb4a37fe700 1 -- 172.21.15.78:0/2144280043 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b400a7b0 con 0x7fb4bc15cc20 2023-10-11T09:52:04.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.127+0000 7fb4a37fe700 1 -- 172.21.15.78:0/2144280043 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b400a910 con 0x7fb4bc15cc20 2023-10-11T09:52:04.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.127+0000 7fb4c15f9700 1 -- 172.21.15.78:0/2144280043 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4bc0d2f80 con 0x7fb4bc15cc20 2023-10-11T09:52:04.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.127+0000 7fb4a37fe700 1 -- 172.21.15.78:0/2144280043 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b400ac80 con 0x7fb4bc15cc20 2023-10-11T09:52:04.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.127+0000 7fb4c15f9700 1 -- 172.21.15.78:0/2144280043 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4bc0b3700 con 0x7fb4bc15cc20 2023-10-11T09:52:04.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.127+0000 7fb4c15f9700 1 -- 172.21.15.78:0/2144280043 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4bc067d00 con 0x7fb4bc15cc20 2023-10-11T09:52:04.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.131+0000 7fb4a37fe700 1 -- 172.21.15.78:0/2144280043 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fb4b401b3b0 con 0x7fb4bc15cc20 2023-10-11T09:52:04.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.131+0000 7fb4a37fe700 1 --2- 172.21.15.78:0/2144280043 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb4a8065ed0 0x7fb4a8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:04.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.131+0000 7fb4a37fe700 1 -- 172.21.15.78:0/2144280043 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fb4b40831e0 con 0x7fb4bc15cc20 2023-10-11T09:52:04.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.131+0000 7fb4baffd700 1 --2- 172.21.15.78:0/2144280043 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb4a8065ed0 0x7fb4a8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:04.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.131+0000 7fb4baffd700 1 --2- 172.21.15.78:0/2144280043 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb4a8065ed0 0x7fb4a8068380 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac000f60 tx=0x7fb4ac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:04.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.139+0000 7fb4a37fe700 1 -- 172.21.15.78:0/2144280043 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fb4b40507e0 con 0x7fb4bc15cc20 2023-10-11T09:52:04.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.359+0000 7fb4c15f9700 1 -- 172.21.15.78:0/2144280043 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4bc0b4450 con 0x7fb4bc15cc20 2023-10-11T09:52:04.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.359+0000 7fb4a37fe700 1 -- 172.21.15.78:0/2144280043 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+4656 (secure 0 0 0) 0x7fb4b4053e00 con 0x7fb4bc15cc20 2023-10-11T09:52:04.367 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:52:04.367 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":102,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8452,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/2481257990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2481257990},{"type":"v1","addr":"172.21.15.78:6836","nonce":2481257990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":8464,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1035227282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1035227282},{"type":"v1","addr":"172.21.15.133:6835","nonce":1035227282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":8680,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/36737634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":36737634},{"type":"v1","addr":"172.21.15.133:6833","nonce":36737634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":101}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:51:40.211859+0000","modified":"2023-10-11T09:52:03.902696+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8880},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8880":{"gid":8880,"name":"c","rank":0,"incarnation":102,"state":"up:replay","state_seq":1,"addr":"172.21.15.78:6837/4189411531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":4189411531},{"type":"v1","addr":"172.21.15.78:6837","nonce":4189411531}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-11T09:52:04.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.363+0000 7fb4c15f9700 1 -- 172.21.15.78:0/2144280043 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb4a8065ed0 msgr2=0x7fb4a8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:04.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.363+0000 7fb4c15f9700 1 --2- 172.21.15.78:0/2144280043 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb4a8065ed0 0x7fb4a8068380 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac000f60 tx=0x7fb4ac009040 comp rx=0 tx=0).stop 2023-10-11T09:52:04.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.363+0000 7fb4c15f9700 1 -- 172.21.15.78:0/2144280043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4bc15cc20 msgr2=0x7fb4bc1f1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:04.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.363+0000 7fb4c15f9700 1 --2- 172.21.15.78:0/2144280043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4bc15cc20 0x7fb4bc1f1520 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400cb40 tx=0x7fb4b400cb70 comp rx=0 tx=0).stop 2023-10-11T09:52:04.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.363+0000 7fb4c15f9700 1 -- 172.21.15.78:0/2144280043 shutdown_connections 2023-10-11T09:52:04.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.363+0000 7fb4c15f9700 1 --2- 172.21.15.78:0/2144280043 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb4a8065ed0 0x7fb4a8068380 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:04.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.363+0000 7fb4c15f9700 1 --2- 172.21.15.78:0/2144280043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4bc15cc20 0x7fb4bc1f1520 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:04.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.363+0000 7fb4c15f9700 1 --2- 172.21.15.78:0/2144280043 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb4bc0cfbb0 0x7fb4bc0b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:04.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.363+0000 7fb4c15f9700 1 --2- 172.21.15.78:0/2144280043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4bc0cd7f0 0x7fb4bc0b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:04.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.363+0000 7fb4c15f9700 1 -- 172.21.15.78:0/2144280043 >> 172.21.15.78:0/2144280043 conn(0x7fb4bc06abf0 msgr2=0x7fb4bc0cff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:04.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.363+0000 7fb4c15f9700 1 -- 172.21.15.78:0/2144280043 shutdown_connections 2023-10-11T09:52:04.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:04.363+0000 7fb4c15f9700 1 -- 172.21.15.78:0/2144280043 wait complete. 2023-10-11T09:52:04.372 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 102 2023-10-11T09:52:04.794 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 102, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-11T09:51:40.211859+0000', 'modified': '2023-10-11T09:52:03.902696+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 128, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8880}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8880': {'gid': 8880, 'name': 'c', 'rank': 0, 'incarnation': 102, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.78:6837/4189411531', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6835', 'nonce': 4189411531}, {'type': 'v1', 'addr': '172.21.15.78:6837', 'nonce': 4189411531}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-11T09:52:04.794 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8880:up:replay 2023-10-11T09:52:05.796 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:52:06.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.231+0000 7fbe0927e700 1 Processor -- start 2023-10-11T09:52:06.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.235+0000 7fbe0927e700 1 -- start start 2023-10-11T09:52:06.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.235+0000 7fbe0927e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe040b0d20 0x7fbe040b4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:06.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.235+0000 7fbe0927e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fbe0406bb60 0x7fbe0415e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:06.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.235+0000 7fbe0927e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe04070dc0 con 0x7fbe0415a800 2023-10-11T09:52:06.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.235+0000 7fbe0927e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe040b46a0 con 0x7fbe0406bb60 2023-10-11T09:52:06.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.235+0000 7fbe0927e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe04070ff0 con 0x7fbe040b0d20 2023-10-11T09:52:06.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.235+0000 7fbe03fff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fbe0415a800 0x7fbe040b0610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:60470/0 (socket says 172.21.15.78:60470) 2023-10-11T09:52:06.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.235+0000 7fbe03fff700 1 -- 172.21.15.78:0/2504601480 learned_addr learned my addr 172.21.15.78:0/2504601480 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:06.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.235+0000 7fbe08a7d700 1 --2- 172.21.15.78:0/2504601480 >> v2:172.21.15.78:3300/0 conn(0x7fbe0406bb60 0x7fbe0415e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:06.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.235+0000 7fbdfb7fe700 1 --2- 172.21.15.78:0/2504601480 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe040b0d20 0x7fbe040b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:06.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.235+0000 7fbe037fe700 1 -- 172.21.15.78:0/2504601480 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3510721989 0 0) 0x7fbe04070dc0 con 0x7fbe0415a800 2023-10-11T09:52:06.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.235+0000 7fbe037fe700 1 -- 172.21.15.78:0/2504601480 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdf00035d0 con 0x7fbe0415a800 2023-10-11T09:52:06.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.239+0000 7fbe08a7d700 1 -- 172.21.15.78:0/2504601480 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe040b0d20 msgr2=0x7fbe040b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:06.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.239+0000 7fbe08a7d700 1 --2- 172.21.15.78:0/2504601480 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe040b0d20 0x7fbe040b4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:06.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.239+0000 7fbe08a7d700 1 -- 172.21.15.78:0/2504601480 >> v1:172.21.15.78:6789/0 conn(0x7fbe0415a800 legacy=0x7fbe040b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:06.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.239+0000 7fbe08a7d700 1 -- 172.21.15.78:0/2504601480 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe040712d0 con 0x7fbe0406bb60 2023-10-11T09:52:06.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.239+0000 7fbe08a7d700 1 --2- 172.21.15.78:0/2504601480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe0406bb60 0x7fbe0415e590 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc009d90 tx=0x7fbdfc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b51df691afd8c32d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:06.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.239+0000 7fbe037fe700 1 -- 172.21.15.78:0/2504601480 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc00e070 con 0x7fbe0406bb60 2023-10-11T09:52:06.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.239+0000 7fbe037fe700 1 -- 172.21.15.78:0/2504601480 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdfc00bb20 con 0x7fbe0406bb60 2023-10-11T09:52:06.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.239+0000 7fbe037fe700 1 -- 172.21.15.78:0/2504601480 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc00be90 con 0x7fbe0406bb60 2023-10-11T09:52:06.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.239+0000 7fbe0927e700 1 -- 172.21.15.78:0/2504601480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe0406bb60 msgr2=0x7fbe0415e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:06.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.239+0000 7fbe0927e700 1 --2- 172.21.15.78:0/2504601480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe0406bb60 0x7fbe0415e590 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc009d90 tx=0x7fbdfc00b330 comp rx=0 tx=0).stop 2023-10-11T09:52:06.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.239+0000 7fbe0927e700 1 -- 172.21.15.78:0/2504601480 shutdown_connections 2023-10-11T09:52:06.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.239+0000 7fbe0927e700 1 --2- 172.21.15.78:0/2504601480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe0406bb60 0x7fbe0415e590 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:06.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.239+0000 7fbe0927e700 1 --2- 172.21.15.78:0/2504601480 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe040b0d20 0x7fbe040b4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:06.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.239+0000 7fbe0927e700 1 -- 172.21.15.78:0/2504601480 >> 172.21.15.78:0/2504601480 conn(0x7fbe040cf820 msgr2=0x7fbe040cb610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:06.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.239+0000 7fbe0927e700 1 -- 172.21.15.78:0/2504601480 shutdown_connections 2023-10-11T09:52:06.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.239+0000 7fbe0927e700 1 -- 172.21.15.78:0/2504601480 wait complete. 2023-10-11T09:52:06.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.243+0000 7fbe0927e700 1 Processor -- start 2023-10-11T09:52:06.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.243+0000 7fbe0927e700 1 -- start start 2023-10-11T09:52:06.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.243+0000 7fbe0927e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe0406bb60 0x7fbe0415b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:06.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.243+0000 7fbe0927e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe040b0d20 0x7fbe040a9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:06.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.243+0000 7fbe0927e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe0415a800 0x7fbe040a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:06.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.243+0000 7fbe0927e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe040c40a0 con 0x7fbe0415a800 2023-10-11T09:52:06.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.243+0000 7fbe0927e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe041f2c80 con 0x7fbe040b0d20 2023-10-11T09:52:06.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.243+0000 7fbe0927e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe040c3700 con 0x7fbe0406bb60 2023-10-11T09:52:06.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.243+0000 7fbe08a7d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe0415a800 0x7fbe040a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:06.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.243+0000 7fbe08a7d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe0415a800 0x7fbe040a9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39066/0 (socket says 172.21.15.78:39066) 2023-10-11T09:52:06.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.243+0000 7fbe08a7d700 1 -- 172.21.15.78:0/4244114485 learned_addr learned my addr 172.21.15.78:0/4244114485 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:06.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.243+0000 7fbe03fff700 1 --2- 172.21.15.78:0/4244114485 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe0406bb60 0x7fbe0415b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:06.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.243+0000 7fbdfb7fe700 1 --2- 172.21.15.78:0/4244114485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe040b0d20 0x7fbe040a9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:06.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.243+0000 7fbe08a7d700 1 -- 172.21.15.78:0/4244114485 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe0406bb60 msgr2=0x7fbe0415b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:06.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.243+0000 7fbe08a7d700 1 --2- 172.21.15.78:0/4244114485 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe0406bb60 0x7fbe0415b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:06.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.243+0000 7fbe08a7d700 1 -- 172.21.15.78:0/4244114485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe040b0d20 msgr2=0x7fbe040a9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:06.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.243+0000 7fbe08a7d700 1 --2- 172.21.15.78:0/4244114485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe040b0d20 0x7fbe040a9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:06.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.243+0000 7fbe08a7d700 1 -- 172.21.15.78:0/4244114485 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdfc009a70 con 0x7fbe0415a800 2023-10-11T09:52:06.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.243+0000 7fbdfb7fe700 1 --2- 172.21.15.78:0/4244114485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe040b0d20 0x7fbe040a9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:06.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.243+0000 7fbe08a7d700 1 --2- 172.21.15.78:0/4244114485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe0415a800 0x7fbe040a9c60 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc014040 tx=0x7fbdfc004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:06.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.243+0000 7fbe017fa700 1 -- 172.21.15.78:0/4244114485 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc007000 con 0x7fbe0415a800 2023-10-11T09:52:06.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.243+0000 7fbe017fa700 1 -- 172.21.15.78:0/4244114485 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdfc012650 con 0x7fbe0415a800 2023-10-11T09:52:06.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.243+0000 7fbe0927e700 1 -- 172.21.15.78:0/4244114485 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe040c39e0 con 0x7fbe0415a800 2023-10-11T09:52:06.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.247+0000 7fbe017fa700 1 -- 172.21.15.78:0/4244114485 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc012a00 con 0x7fbe0415a800 2023-10-11T09:52:06.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.247+0000 7fbe0927e700 1 -- 172.21.15.78:0/4244114485 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe0415bb50 con 0x7fbe0415a800 2023-10-11T09:52:06.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.247+0000 7fbe0927e700 1 -- 172.21.15.78:0/4244114485 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe04067d00 con 0x7fbe0415a800 2023-10-11T09:52:06.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.247+0000 7fbe017fa700 1 -- 172.21.15.78:0/4244114485 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fbdfc012b60 con 0x7fbe0415a800 2023-10-11T09:52:06.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.251+0000 7fbe017fa700 1 --2- 172.21.15.78:0/4244114485 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbde4065ed0 0x7fbde4068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:06.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.251+0000 7fbe017fa700 1 -- 172.21.15.78:0/4244114485 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fbdfc085660 con 0x7fbe0415a800 2023-10-11T09:52:06.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.251+0000 7fbe03fff700 1 --2- 172.21.15.78:0/4244114485 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbde4065ed0 0x7fbde4068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:06.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.251+0000 7fbe03fff700 1 --2- 172.21.15.78:0/4244114485 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbde4065ed0 0x7fbde4068380 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4007fd0 tx=0x7fbdf4007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:06.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.255+0000 7fbe017fa700 1 -- 172.21.15.78:0/4244114485 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fbdfc052e10 con 0x7fbe0415a800 2023-10-11T09:52:06.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.467+0000 7fbe0927e700 1 -- 172.21.15.78:0/4244114485 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe040c41e0 con 0x7fbe0415a800 2023-10-11T09:52:06.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.467+0000 7fbe017fa700 1 -- 172.21.15.78:0/4244114485 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4657 (secure 0 0 0) 0x7fbe040c41e0 con 0x7fbe0415a800 2023-10-11T09:52:06.475 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:52:06.476 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":104,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8452,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/2481257990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2481257990},{"type":"v1","addr":"172.21.15.78:6836","nonce":2481257990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":8464,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1035227282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1035227282},{"type":"v1","addr":"172.21.15.133:6835","nonce":1035227282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":8680,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/36737634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":36737634},{"type":"v1","addr":"172.21.15.133:6833","nonce":36737634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":101}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:51:40.211859+0000","modified":"2023-10-11T09:52:05.913333+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8880},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8880":{"gid":8880,"name":"c","rank":0,"incarnation":102,"state":"up:rejoin","state_seq":10,"addr":"172.21.15.78:6837/4189411531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":4189411531},{"type":"v1","addr":"172.21.15.78:6837","nonce":4189411531}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-11T09:52:06.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.471+0000 7fbe0927e700 1 -- 172.21.15.78:0/4244114485 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbde4065ed0 msgr2=0x7fbde4068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:06.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.471+0000 7fbe0927e700 1 --2- 172.21.15.78:0/4244114485 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbde4065ed0 0x7fbde4068380 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4007fd0 tx=0x7fbdf4007de0 comp rx=0 tx=0).stop 2023-10-11T09:52:06.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.471+0000 7fbe0927e700 1 -- 172.21.15.78:0/4244114485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe0415a800 msgr2=0x7fbe040a9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:06.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.471+0000 7fbe0927e700 1 --2- 172.21.15.78:0/4244114485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe0415a800 0x7fbe040a9c60 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc014040 tx=0x7fbdfc004af0 comp rx=0 tx=0).stop 2023-10-11T09:52:06.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.471+0000 7fbe0927e700 1 -- 172.21.15.78:0/4244114485 shutdown_connections 2023-10-11T09:52:06.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.471+0000 7fbe0927e700 1 --2- 172.21.15.78:0/4244114485 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbde4065ed0 0x7fbde4068380 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:06.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.471+0000 7fbe0927e700 1 --2- 172.21.15.78:0/4244114485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe0415a800 0x7fbe040a9c60 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:06.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.471+0000 7fbe0927e700 1 --2- 172.21.15.78:0/4244114485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe040b0d20 0x7fbe040a9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:06.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.471+0000 7fbe0927e700 1 --2- 172.21.15.78:0/4244114485 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe0406bb60 0x7fbe0415b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:06.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.471+0000 7fbe0927e700 1 -- 172.21.15.78:0/4244114485 >> 172.21.15.78:0/4244114485 conn(0x7fbe040cf820 msgr2=0x7fbe0406bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:06.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.471+0000 7fbe0927e700 1 -- 172.21.15.78:0/4244114485 shutdown_connections 2023-10-11T09:52:06.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:06.471+0000 7fbe0927e700 1 -- 172.21.15.78:0/4244114485 wait complete. 2023-10-11T09:52:06.481 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 104 2023-10-11T09:52:06.903 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 104, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-11T09:51:40.211859+0000', 'modified': '2023-10-11T09:52:05.913333+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 128, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8880}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8880': {'gid': 8880, 'name': 'c', 'rank': 0, 'incarnation': 102, 'state': 'up:rejoin', 'state_seq': 10, 'addr': '172.21.15.78:6837/4189411531', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6835', 'nonce': 4189411531}, {'type': 'v1', 'addr': '172.21.15.78:6837', 'nonce': 4189411531}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-11T09:52:06.904 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8880:up:rejoin 2023-10-11T09:52:07.905 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:52:08.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.071+0000 7f7f19d07700 1 Processor -- start 2023-10-11T09:52:08.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.071+0000 7f7f19d07700 1 -- start start 2023-10-11T09:52:08.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.071+0000 7f7f19d07700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7f1415d760 0x7f7f140cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:08.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.071+0000 7f7f19d07700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f1415dc40 con 0x7f7f1415a230 2023-10-11T09:52:08.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.071+0000 7f7f19d07700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f140cf3b0 con 0x7f7f140cfdd0 2023-10-11T09:52:08.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.071+0000 7f7f19d07700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f140cf1a0 con 0x7f7f1415d760 2023-10-11T09:52:08.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.071+0000 7f7f18d05700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f7f1415a230 0x7f7f140d0b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:60480/0 (socket says 172.21.15.78:60480) 2023-10-11T09:52:08.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.071+0000 7f7f18d05700 1 -- 172.21.15.78:0/1651034320 learned_addr learned my addr 172.21.15.78:0/1651034320 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:08.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.075+0000 7f7f137fe700 1 -- 172.21.15.78:0/1651034320 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 570709626 0 0) 0x7f7f1415dc40 con 0x7f7f1415a230 2023-10-11T09:52:08.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.075+0000 7f7f137fe700 1 -- 172.21.15.78:0/1651034320 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f040035d0 con 0x7f7f1415a230 2023-10-11T09:52:08.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.075+0000 7f7f13fff700 1 --2- 172.21.15.78:0/1651034320 >> v2:172.21.15.133:3300/0 conn(0x7f7f1415d760 0x7f7f140cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:08.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.075+0000 7f7f137fe700 1 -- 172.21.15.78:0/1651034320 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1818123097 0 0) 0x7f7f040035d0 con 0x7f7f1415a230 2023-10-11T09:52:08.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.075+0000 7f7f137fe700 1 -- 172.21.15.78:0/1651034320 >> v2:172.21.15.133:3300/0 conn(0x7f7f1415d760 msgr2=0x7f7f140cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:08.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.075+0000 7f7f137fe700 1 --2- 172.21.15.78:0/1651034320 >> v2:172.21.15.133:3300/0 conn(0x7f7f1415d760 0x7f7f140cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:08.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.075+0000 7f7f137fe700 1 -- 172.21.15.78:0/1651034320 >> v1:172.21.15.133:6789/0 conn(0x7f7f140cfdd0 legacy=0x7f7f140b7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:08.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.075+0000 7f7f137fe700 1 -- 172.21.15.78:0/1651034320 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f140cd6d0 con 0x7f7f1415a230 2023-10-11T09:52:08.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.075+0000 7f7f137fe700 1 -- 172.21.15.78:0/1651034320 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f7f08001670 con 0x7f7f1415a230 2023-10-11T09:52:08.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.075+0000 7f7f137fe700 1 -- 172.21.15.78:0/1651034320 >> v1:172.21.15.78:6789/0 conn(0x7f7f1415a230 legacy=0x7f7f140d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:08.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.075+0000 7f7f137fe700 1 --2- 172.21.15.78:0/1651034320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f04003a20 0x7f7f04003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:08.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.075+0000 7f7f137fe700 1 --2- 172.21.15.78:0/1651034320 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f04004870 0x7f7f04004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:08.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.075+0000 7f7f137fe700 1 --2- 172.21.15.78:0/1651034320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f04005220 0x7f7f0400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:08.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.075+0000 7f7f137fe700 1 -- 172.21.15.78:0/1651034320 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f140cf1a0 con 0x7f7f04005220 2023-10-11T09:52:08.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.075+0000 7f7f137fe700 1 -- 172.21.15.78:0/1651034320 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0400c130 con 0x7f7f04003a20 2023-10-11T09:52:08.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.075+0000 7f7f137fe700 1 -- 172.21.15.78:0/1651034320 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0400c2f0 con 0x7f7f04004870 2023-10-11T09:52:08.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.075+0000 7f7f18d05700 1 --2- 172.21.15.78:0/1651034320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f04003a20 0x7f7f04003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:08.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.075+0000 7f7f19506700 1 --2- 172.21.15.78:0/1651034320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f04005220 0x7f7f0400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:08.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.075+0000 7f7f13fff700 1 --2- 172.21.15.78:0/1651034320 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f04004870 0x7f7f04004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:08.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.075+0000 7f7f19506700 1 -- 172.21.15.78:0/1651034320 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f04004870 msgr2=0x7f7f04004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:08.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.075+0000 7f7f19506700 1 --2- 172.21.15.78:0/1651034320 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f04004870 0x7f7f04004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:08.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.075+0000 7f7f19506700 1 -- 172.21.15.78:0/1651034320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f04003a20 msgr2=0x7f7f04003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:08.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.075+0000 7f7f19506700 1 --2- 172.21.15.78:0/1651034320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f04003a20 0x7f7f04003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:08.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.075+0000 7f7f19506700 1 -- 172.21.15.78:0/1651034320 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f040035d0 con 0x7f7f04005220 2023-10-11T09:52:08.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.075+0000 7f7f18d05700 1 --2- 172.21.15.78:0/1651034320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f04003a20 0x7f7f04003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:52:08.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.075+0000 7f7f19506700 1 -- 172.21.15.78:0/1651034320 reap_dead start 2023-10-11T09:52:08.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.075+0000 7f7f19506700 1 --2- 172.21.15.78:0/1651034320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f04005220 0x7f7f0400ba30 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c00c190 tx=0x7f7f040058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11960a755cb97a28 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:08.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.075+0000 7f7f137fe700 1 -- 172.21.15.78:0/1651034320 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c001db0 con 0x7f7f04005220 2023-10-11T09:52:08.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.075+0000 7f7f137fe700 1 -- 172.21.15.78:0/1651034320 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f0c00ccf0 con 0x7f7f04005220 2023-10-11T09:52:08.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.075+0000 7f7f137fe700 1 -- 172.21.15.78:0/1651034320 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c003c60 con 0x7f7f04005220 2023-10-11T09:52:08.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.079+0000 7f7f19d07700 1 -- 172.21.15.78:0/1651034320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f04005220 msgr2=0x7f7f0400ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:08.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.079+0000 7f7f19d07700 1 --2- 172.21.15.78:0/1651034320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f04005220 0x7f7f0400ba30 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c00c190 tx=0x7f7f040058a0 comp rx=0 tx=0).stop 2023-10-11T09:52:08.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.079+0000 7f7f19d07700 1 -- 172.21.15.78:0/1651034320 shutdown_connections 2023-10-11T09:52:08.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.079+0000 7f7f19d07700 1 --2- 172.21.15.78:0/1651034320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f04005220 0x7f7f0400ba30 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:08.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.079+0000 7f7f19d07700 1 -- 172.21.15.78:0/1651034320 >> 172.21.15.78:0/1651034320 conn(0x7f7f1406aa20 msgr2=0x7f7f1406adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:08.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.079+0000 7f7f19d07700 1 -- 172.21.15.78:0/1651034320 shutdown_connections 2023-10-11T09:52:08.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.079+0000 7f7f19d07700 1 -- 172.21.15.78:0/1651034320 wait complete. 2023-10-11T09:52:08.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.079+0000 7f7f19d07700 1 Processor -- start 2023-10-11T09:52:08.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.079+0000 7f7f19d07700 1 -- start start 2023-10-11T09:52:08.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.079+0000 7f7f19d07700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f04005220 0x7f7f140b7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:08.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.079+0000 7f7f19d07700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f140b5d80 0x7f7f140c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:08.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.079+0000 7f7f19d07700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f1414fd00 0x7f7f141f5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:08.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.079+0000 7f7f19d07700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f140b62b0 con 0x7f7f1414fd00 2023-10-11T09:52:08.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.079+0000 7f7f19d07700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f140b6590 con 0x7f7f04005220 2023-10-11T09:52:08.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.079+0000 7f7f19d07700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f140b6870 con 0x7f7f140b5d80 2023-10-11T09:52:08.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.083+0000 7f7f19506700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f1414fd00 0x7f7f141f5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:08.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.083+0000 7f7f19506700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f1414fd00 0x7f7f141f5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39096/0 (socket says 172.21.15.78:39096) 2023-10-11T09:52:08.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.083+0000 7f7f19506700 1 -- 172.21.15.78:0/4261848298 learned_addr learned my addr 172.21.15.78:0/4261848298 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:08.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.083+0000 7f7f13fff700 1 --2- 172.21.15.78:0/4261848298 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f140b5d80 0x7f7f140c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:08.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.083+0000 7f7f18d05700 1 --2- 172.21.15.78:0/4261848298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f04005220 0x7f7f140b7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:08.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.083+0000 7f7f19506700 1 -- 172.21.15.78:0/4261848298 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f140b5d80 msgr2=0x7f7f140c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:08.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.083+0000 7f7f19506700 1 --2- 172.21.15.78:0/4261848298 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f140b5d80 0x7f7f140c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:08.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.083+0000 7f7f19506700 1 -- 172.21.15.78:0/4261848298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f04005220 msgr2=0x7f7f140b7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:08.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.083+0000 7f7f19506700 1 --2- 172.21.15.78:0/4261848298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f04005220 0x7f7f140b7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:08.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.083+0000 7f7f19506700 1 -- 172.21.15.78:0/4261848298 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f140b6b50 con 0x7f7f1414fd00 2023-10-11T09:52:08.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.083+0000 7f7f19506700 1 --2- 172.21.15.78:0/4261848298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f1414fd00 0x7f7f141f5580 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f7f140c6a50 tx=0x7f7f0c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:08.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.083+0000 7f7f117fa700 1 -- 172.21.15.78:0/4261848298 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c005070 con 0x7f7f1414fd00 2023-10-11T09:52:08.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.083+0000 7f7f117fa700 1 -- 172.21.15.78:0/4261848298 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f0c0051d0 con 0x7f7f1414fd00 2023-10-11T09:52:08.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.083+0000 7f7f19d07700 1 -- 172.21.15.78:0/4261848298 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f140b6e30 con 0x7f7f1414fd00 2023-10-11T09:52:08.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.083+0000 7f7f117fa700 1 -- 172.21.15.78:0/4261848298 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c005560 con 0x7f7f1414fd00 2023-10-11T09:52:08.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.083+0000 7f7f19d07700 1 -- 172.21.15.78:0/4261848298 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f141f5ac0 con 0x7f7f1414fd00 2023-10-11T09:52:08.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.083+0000 7f7f19d07700 1 -- 172.21.15.78:0/4261848298 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f140b7070 con 0x7f7f1414fd00 2023-10-11T09:52:08.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.087+0000 7f7f117fa700 1 -- 172.21.15.78:0/4261848298 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f7f0c0056c0 con 0x7f7f1414fd00 2023-10-11T09:52:08.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.087+0000 7f7f117fa700 1 --2- 172.21.15.78:0/4261848298 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7ef8065ed0 0x7f7ef8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:08.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.087+0000 7f7f117fa700 1 -- 172.21.15.78:0/4261848298 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f7f0c084a00 con 0x7f7f1414fd00 2023-10-11T09:52:08.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.087+0000 7f7f18d05700 1 --2- 172.21.15.78:0/4261848298 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7ef8065ed0 0x7f7ef8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:08.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.087+0000 7f7f18d05700 1 --2- 172.21.15.78:0/4261848298 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7ef8065ed0 0x7f7ef8068380 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f7f08001c90 tx=0x7f7f08001b80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:08.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.091+0000 7f7f117fa700 1 -- 172.21.15.78:0/4261848298 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f7f140b7070 con 0x7f7f1414fd00 2023-10-11T09:52:08.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.299+0000 7f7f19d07700 1 -- 172.21.15.78:0/4261848298 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f140b7070 con 0x7f7f1414fd00 2023-10-11T09:52:08.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.299+0000 7f7f117fa700 1 -- 172.21.15.78:0/4261848298 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4657 (secure 0 0 0) 0x7f7f140b7070 con 0x7f7f1414fd00 2023-10-11T09:52:08.309 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:52:08.309 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":105,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8452,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/2481257990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2481257990},{"type":"v1","addr":"172.21.15.78:6836","nonce":2481257990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":8464,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1035227282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1035227282},{"type":"v1","addr":"172.21.15.133:6835","nonce":1035227282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":8680,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/36737634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":36737634},{"type":"v1","addr":"172.21.15.133:6833","nonce":36737634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":101}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:51:40.211859+0000","modified":"2023-10-11T09:52:06.926846+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8880},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8880":{"gid":8880,"name":"c","rank":0,"incarnation":102,"state":"up:active","state_seq":11,"addr":"172.21.15.78:6837/4189411531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":4189411531},{"type":"v1","addr":"172.21.15.78:6837","nonce":4189411531}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-11T09:52:08.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.303+0000 7f7f19d07700 1 -- 172.21.15.78:0/4261848298 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7ef8065ed0 msgr2=0x7f7ef8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:08.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.303+0000 7f7f19d07700 1 --2- 172.21.15.78:0/4261848298 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7ef8065ed0 0x7f7ef8068380 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f7f08001c90 tx=0x7f7f08001b80 comp rx=0 tx=0).stop 2023-10-11T09:52:08.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.303+0000 7f7f19d07700 1 -- 172.21.15.78:0/4261848298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f1414fd00 msgr2=0x7f7f141f5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:08.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.303+0000 7f7f19d07700 1 --2- 172.21.15.78:0/4261848298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f1414fd00 0x7f7f141f5580 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f7f140c6a50 tx=0x7f7f0c0014a0 comp rx=0 tx=0).stop 2023-10-11T09:52:08.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.303+0000 7f7f19d07700 1 -- 172.21.15.78:0/4261848298 shutdown_connections 2023-10-11T09:52:08.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.303+0000 7f7f19d07700 1 --2- 172.21.15.78:0/4261848298 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7ef8065ed0 0x7f7ef8068380 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:08.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.303+0000 7f7f19d07700 1 --2- 172.21.15.78:0/4261848298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f1414fd00 0x7f7f141f5580 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:08.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.303+0000 7f7f19d07700 1 --2- 172.21.15.78:0/4261848298 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f140b5d80 0x7f7f140c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:08.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.303+0000 7f7f19d07700 1 --2- 172.21.15.78:0/4261848298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f04005220 0x7f7f140b7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:08.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.303+0000 7f7f19d07700 1 -- 172.21.15.78:0/4261848298 >> 172.21.15.78:0/4261848298 conn(0x7f7f1406aa20 msgr2=0x7f7f1415f540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:08.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.303+0000 7f7f19d07700 1 -- 172.21.15.78:0/4261848298 shutdown_connections 2023-10-11T09:52:08.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.303+0000 7f7f19d07700 1 -- 172.21.15.78:0/4261848298 wait complete. 2023-10-11T09:52:08.315 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 105 2023-10-11T09:52:08.735 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 105, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-11T09:51:40.211859+0000', 'modified': '2023-10-11T09:52:06.926846+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 128, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8880}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8880': {'gid': 8880, 'name': 'c', 'rank': 0, 'incarnation': 102, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.78:6837/4189411531', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6835', 'nonce': 4189411531}, {'type': 'v1', 'addr': '172.21.15.78:6837', 'nonce': 4189411531}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-11T09:52:08.735 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-11T09:52:08.736 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-11T09:52:08.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.911+0000 7f757dde1700 1 Processor -- start 2023-10-11T09:52:08.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.911+0000 7f757dde1700 1 -- start start 2023-10-11T09:52:08.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.915+0000 7f757dde1700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f757815a230 0x7f75780d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:08.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.915+0000 7f757dde1700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f757815dc40 con 0x7f757815d760 2023-10-11T09:52:08.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.915+0000 7f757dde1700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75780cf3b0 con 0x7f75780cffa0 2023-10-11T09:52:08.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.915+0000 7f757dde1700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75780cf1a0 con 0x7f757815a230 2023-10-11T09:52:08.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.915+0000 7f7577fff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f757815d760 0x7f75780cf890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:60494/0 (socket says 172.21.15.78:60494) 2023-10-11T09:52:08.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.915+0000 7f7577fff700 1 -- 172.21.15.78:0/1893120424 learned_addr learned my addr 172.21.15.78:0/1893120424 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:08.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.915+0000 7f757cddf700 1 --2- 172.21.15.78:0/1893120424 >> v2:172.21.15.78:3300/0 conn(0x7f757815a230 0x7f75780d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:08.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.915+0000 7f757cddf700 1 -- 172.21.15.78:0/1893120424 >> v1:172.21.15.133:6789/0 conn(0x7f75780cffa0 legacy=0x7f75780b7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:08.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.915+0000 7f757cddf700 1 -- 172.21.15.78:0/1893120424 >> v1:172.21.15.78:6789/0 conn(0x7f757815d760 legacy=0x7f75780cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:08.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.915+0000 7f75777fe700 1 -- 172.21.15.78:0/1893120424 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1773338288 0 0) 0x7f757815dc40 con 0x7f757815d760 2023-10-11T09:52:08.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.915+0000 7f757cddf700 1 -- 172.21.15.78:0/1893120424 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75780cd6d0 con 0x7f757815a230 2023-10-11T09:52:08.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.915+0000 7f757cddf700 1 --2- 172.21.15.78:0/1893120424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f757815a230 0x7f75780d0b20 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f756c009850 tx=0x7f756c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eecc2963803b4ca server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:08.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.915+0000 7f75777fe700 1 -- 172.21.15.78:0/1893120424 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c011830 con 0x7f757815a230 2023-10-11T09:52:08.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.919+0000 7f75777fe700 1 -- 172.21.15.78:0/1893120424 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756c011990 con 0x7f757815a230 2023-10-11T09:52:08.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.919+0000 7f75777fe700 1 -- 172.21.15.78:0/1893120424 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c011d20 con 0x7f757815a230 2023-10-11T09:52:08.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.919+0000 7f757dde1700 1 -- 172.21.15.78:0/1893120424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f757815a230 msgr2=0x7f75780d0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:08.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.919+0000 7f757dde1700 1 --2- 172.21.15.78:0/1893120424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f757815a230 0x7f75780d0b20 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f756c009850 tx=0x7f756c009b60 comp rx=0 tx=0).stop 2023-10-11T09:52:08.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.919+0000 7f757dde1700 1 -- 172.21.15.78:0/1893120424 shutdown_connections 2023-10-11T09:52:08.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.919+0000 7f757dde1700 1 --2- 172.21.15.78:0/1893120424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f757815a230 0x7f75780d0b20 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:08.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.919+0000 7f757dde1700 1 -- 172.21.15.78:0/1893120424 >> 172.21.15.78:0/1893120424 conn(0x7f757806aa20 msgr2=0x7f757806adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:08.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.919+0000 7f757dde1700 1 -- 172.21.15.78:0/1893120424 shutdown_connections 2023-10-11T09:52:08.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.919+0000 7f757dde1700 1 -- 172.21.15.78:0/1893120424 wait complete. 2023-10-11T09:52:08.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.919+0000 7f757dde1700 1 Processor -- start 2023-10-11T09:52:08.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.923+0000 7f757dde1700 1 -- start start 2023-10-11T09:52:08.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.923+0000 7f757dde1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75780cffa0 0x7f75780c6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:08.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.923+0000 7f757dde1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f757815a230 0x7f75781589e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:08.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.923+0000 7f757dde1700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f757815d760 0x7f75780b8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:08.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.923+0000 7f757dde1700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75781f64a0 con 0x7f757815a230 2023-10-11T09:52:08.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.923+0000 7f757dde1700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75781f5f20 con 0x7f75780cffa0 2023-10-11T09:52:08.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.923+0000 7f757dde1700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75780c8450 con 0x7f757815d760 2023-10-11T09:52:08.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.923+0000 7f7577fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f757815a230 0x7f75781589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:08.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.923+0000 7f7577fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f757815a230 0x7f75781589e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39108/0 (socket says 172.21.15.78:39108) 2023-10-11T09:52:08.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.923+0000 7f7577fff700 1 -- 172.21.15.78:0/1425289084 learned_addr learned my addr 172.21.15.78:0/1425289084 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:08.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.923+0000 7f757cddf700 1 --2- 172.21.15.78:0/1425289084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75780cffa0 0x7f75780c6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:08.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.923+0000 7f757d5e0700 1 --2- 172.21.15.78:0/1425289084 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f757815d760 0x7f75780b8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:08.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.923+0000 7f7577fff700 1 -- 172.21.15.78:0/1425289084 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f757815d760 msgr2=0x7f75780b8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:08.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.923+0000 7f7577fff700 1 --2- 172.21.15.78:0/1425289084 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f757815d760 0x7f75780b8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:08.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.923+0000 7f7577fff700 1 -- 172.21.15.78:0/1425289084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75780cffa0 msgr2=0x7f75780c6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:08.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.923+0000 7f7577fff700 1 --2- 172.21.15.78:0/1425289084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75780cffa0 0x7f75780c6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:08.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.923+0000 7f7577fff700 1 -- 172.21.15.78:0/1425289084 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756c0094b0 con 0x7f757815a230 2023-10-11T09:52:08.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.923+0000 7f7577fff700 1 --2- 172.21.15.78:0/1425289084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f757815a230 0x7f75781589e0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f756400c450 tx=0x7f756400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:08.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.923+0000 7f75757fa700 1 -- 172.21.15.78:0/1425289084 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7564004ca0 con 0x7f757815a230 2023-10-11T09:52:08.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.923+0000 7f75757fa700 1 -- 172.21.15.78:0/1425289084 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7564004e00 con 0x7f757815a230 2023-10-11T09:52:08.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.923+0000 7f75757fa700 1 -- 172.21.15.78:0/1425289084 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7564007310 con 0x7f757815a230 2023-10-11T09:52:08.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.923+0000 7f757dde1700 1 -- 172.21.15.78:0/1425289084 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75781f61c0 con 0x7f757815a230 2023-10-11T09:52:08.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.923+0000 7f757dde1700 1 -- 172.21.15.78:0/1425289084 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75780c89d0 con 0x7f757815a230 2023-10-11T09:52:08.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.927+0000 7f75757fa700 1 -- 172.21.15.78:0/1425289084 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f7564007470 con 0x7f757815a230 2023-10-11T09:52:08.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.931+0000 7f75757fa700 1 --2- 172.21.15.78:0/1425289084 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f755c065ed0 0x7f755c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:08.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.931+0000 7f75757fa700 1 -- 172.21.15.78:0/1425289084 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f7564083540 con 0x7f757815a230 2023-10-11T09:52:08.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.931+0000 7f757cddf700 1 --2- 172.21.15.78:0/1425289084 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f755c065ed0 0x7f755c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:08.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.931+0000 7f75822d8740 1 Processor -- start 2023-10-11T09:52:08.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.931+0000 7f75822d8740 1 -- start start 2023-10-11T09:52:08.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.931+0000 7f757cddf700 1 --2- 172.21.15.78:0/1425289084 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f755c065ed0 0x7f755c068380 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f756c0090b0 tx=0x7f756c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:08.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.935+0000 7f75822d8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x26e5eb0 0x299e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:08.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.935+0000 7f75822d8740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x26e5ae0 0x299e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:08.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.935+0000 7f75822d8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26e5710 0x2965640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.935+0000 7f75822d8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x28c54a0 con 0x26e5710 2023-10-11T09:52:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.935+0000 7f75822d8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x28c2c30 con 0x26e5eb0 2023-10-11T09:52:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.935+0000 7f75822d8740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x28be520 con 0x26e5ae0 2023-10-11T09:52:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.935+0000 7f757d5e0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26e5710 0x2965640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.935+0000 7f757d5e0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26e5710 0x2965640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39122/0 (socket says 172.21.15.78:39122) 2023-10-11T09:52:08.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.935+0000 7f757d5e0700 1 -- 172.21.15.78:0/727857947 learned_addr learned my addr 172.21.15.78:0/727857947 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:08.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.935+0000 7f757cddf700 1 --2- 172.21.15.78:0/727857947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x26e5eb0 0x299e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:08.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.935+0000 7f7577fff700 1 --2- 172.21.15.78:0/727857947 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x26e5ae0 0x299e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:08.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.935+0000 7f757d5e0700 1 -- 172.21.15.78:0/727857947 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x26e5ae0 msgr2=0x299e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:08.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.935+0000 7f757d5e0700 1 --2- 172.21.15.78:0/727857947 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x26e5ae0 0x299e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:08.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.935+0000 7f757d5e0700 1 -- 172.21.15.78:0/727857947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x26e5eb0 msgr2=0x299e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:08.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.935+0000 7f757d5e0700 1 --2- 172.21.15.78:0/727857947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x26e5eb0 0x299e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:08.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.935+0000 7f757d5e0700 1 -- 172.21.15.78:0/727857947 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x27319e0 con 0x26e5710 2023-10-11T09:52:08.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.935+0000 7f757cddf700 1 --2- 172.21.15.78:0/727857947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x26e5eb0 0x299e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:08.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.935+0000 7f757d5e0700 1 --2- 172.21.15.78:0/727857947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26e5710 0x2965640 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f7570002110 tx=0x7f757000e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2f962e0b9ca876 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:08.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.935+0000 7f75627fc700 1 -- 172.21.15.78:0/727857947 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757000ec60 con 0x26e5710 2023-10-11T09:52:08.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.935+0000 7f75627fc700 1 -- 172.21.15.78:0/727857947 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f757000edc0 con 0x26e5710 2023-10-11T09:52:08.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.935+0000 7f75627fc700 1 -- 172.21.15.78:0/727857947 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7570003c40 con 0x26e5710 2023-10-11T09:52:08.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.935+0000 7f75822d8740 1 -- 172.21.15.78:0/727857947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26e5710 msgr2=0x2965640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:08.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.935+0000 7f75822d8740 1 --2- 172.21.15.78:0/727857947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26e5710 0x2965640 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f7570002110 tx=0x7f757000e440 comp rx=0 tx=0).stop 2023-10-11T09:52:08.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.935+0000 7f75822d8740 1 -- 172.21.15.78:0/727857947 shutdown_connections 2023-10-11T09:52:08.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.935+0000 7f75822d8740 1 --2- 172.21.15.78:0/727857947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26e5710 0x2965640 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:08.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.935+0000 7f75822d8740 1 --2- 172.21.15.78:0/727857947 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x26e5ae0 0x299e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:08.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.935+0000 7f75822d8740 1 --2- 172.21.15.78:0/727857947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x26e5eb0 0x299e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:08.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.935+0000 7f75822d8740 1 -- 172.21.15.78:0/727857947 >> 172.21.15.78:0/727857947 conn(0x294dfa0 msgr2=0x297f420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:08.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.935+0000 7f75822d8740 1 -- 172.21.15.78:0/727857947 shutdown_connections 2023-10-11T09:52:08.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.935+0000 7f75822d8740 1 -- 172.21.15.78:0/727857947 wait complete. 2023-10-11T09:52:08.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.939+0000 7f75822d8740 1 -- start start 2023-10-11T09:52:08.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.939+0000 7f75822d8740 1 Processor -- start 2023-10-11T09:52:08.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.939+0000 7f75822d8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x26e5710 0x29587a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:08.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.939+0000 7f75822d8740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x26e5ae0 0x2993b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:08.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.939+0000 7f75822d8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26e5eb0 0x29940d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:08.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.939+0000 7f75822d8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x2881b00 con 0x26e5eb0 2023-10-11T09:52:08.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.939+0000 7f75822d8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x28817a0 con 0x26e5710 2023-10-11T09:52:08.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.939+0000 7f75822d8740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x276ab20 con 0x26e5ae0 2023-10-11T09:52:08.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.939+0000 7f75822d8740 10 client.0 _ll_register_callbacks cb 0x28b7e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:52:08.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.939+0000 7f757d5e0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26e5eb0 0x29940d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:08.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.939+0000 7f757d5e0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26e5eb0 0x29940d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39138/0 (socket says 172.21.15.78:39138) 2023-10-11T09:52:08.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.939+0000 7f757d5e0700 1 -- 172.21.15.78:0/2561822924 learned_addr learned my addr 172.21.15.78:0/2561822924 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:08.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.939+0000 7f757cddf700 1 --2- 172.21.15.78:0/2561822924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x26e5710 0x29587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:08.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.939+0000 7f7577fff700 1 --2- 172.21.15.78:0/2561822924 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x26e5ae0 0x2993b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:08.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.939+0000 7f757d5e0700 1 -- 172.21.15.78:0/2561822924 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x26e5ae0 msgr2=0x2993b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:08.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.939+0000 7f757d5e0700 1 --2- 172.21.15.78:0/2561822924 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x26e5ae0 0x2993b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:08.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.939+0000 7f757d5e0700 1 -- 172.21.15.78:0/2561822924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x26e5710 msgr2=0x29587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:08.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.939+0000 7f757d5e0700 1 --2- 172.21.15.78:0/2561822924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x26e5710 0x29587a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:08.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.939+0000 7f757d5e0700 1 -- 172.21.15.78:0/2561822924 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f757000f040 con 0x26e5eb0 2023-10-11T09:52:08.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.939+0000 7f7577fff700 1 --2- 172.21.15.78:0/2561822924 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x26e5ae0 0x2993b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:08.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.943+0000 7f757d5e0700 1 --2- 172.21.15.78:0/2561822924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26e5eb0 0x29940d0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f7570009010 tx=0x7f7570001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2575bb7792f700c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:08.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.943+0000 7f75577fe700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:52:08.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.943+0000 7f75577fe700 1 -- 172.21.15.78:0/2561822924 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757000ec60 con 0x26e5eb0 2023-10-11T09:52:08.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.943+0000 7f75577fe700 1 -- 172.21.15.78:0/2561822924 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f7570001870 con 0x26e5eb0 2023-10-11T09:52:08.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.943+0000 7f75822d8740 1 -- 172.21.15.78:0/2561822924 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x28b75a0 con 0x26e5eb0 2023-10-11T09:52:08.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.947+0000 7f75577fe700 1 -- 172.21.15.78:0/2561822924 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7570003e80 con 0x26e5eb0 2023-10-11T09:52:08.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.947+0000 7f75822d8740 10 client.9216 fetch_fsmap learned FSMap version 105 2023-10-11T09:52:08.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.947+0000 7f75822d8740 1 -- 172.21.15.78:0/2561822924 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x2884750 con 0x26e5eb0 2023-10-11T09:52:08.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.947+0000 7f75577fe700 1 -- 172.21.15.78:0/2561822924 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x28b75a0 con 0x26e5eb0 2023-10-11T09:52:08.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.947+0000 7f75577fe700 1 -- 172.21.15.78:0/2561822924 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 105) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f75700052e0 con 0x26e5eb0 2023-10-11T09:52:08.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.947+0000 7f75822d8740 10 client.9216 fetch_fsmap finished waiting for FSMap version 105 2023-10-11T09:52:08.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.947+0000 7f75822d8740 10 client.9216 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-11T09:52:08.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.947+0000 7f75822d8740 20 client.9216 populate_metadata read hostname 'smithi078' 2023-10-11T09:52:08.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.947+0000 7f75822d8740 1 --2- 172.21.15.78:0/2561822924 >> [v2:172.21.15.78:6835/4189411531,v1:172.21.15.78:6837/4189411531] conn(0x2730790 0x29b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:08.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.947+0000 7f75822d8740 4 client.9216 mds_command: new command op to 8880 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-11T09:52:08.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.947+0000 7f75822d8740 1 -- 172.21.15.78:0/2561822924 --> [v2:172.21.15.78:6835/4189411531,v1:172.21.15.78:6837/4189411531] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x28b72c0 con 0x2730790 2023-10-11T09:52:08.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.947+0000 7f757cddf700 1 --2- 172.21.15.78:0/2561822924 >> [v2:172.21.15.78:6835/4189411531,v1:172.21.15.78:6837/4189411531] conn(0x2730790 0x29b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:08.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.951+0000 7f757cddf700 1 --2- 172.21.15.78:0/2561822924 >> [v2:172.21.15.78:6835/4189411531,v1:172.21.15.78:6837/4189411531] conn(0x2730790 0x29b0f00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=247ce9d577c480f1 server_cookie=4f02e1cb5061bef8 in_seq=0 out_seq=0 2023-10-11T09:52:08.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.951+0000 7f75577fe700 10 client.9216 ms_handle_connect on v2:172.21.15.78:6835/4189411531 2023-10-11T09:52:08.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.951+0000 7f75577fe700 1 -- 172.21.15.78:0/2561822924 <== mds.0 v2:172.21.15.78:6835/4189411531 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x28b72c0 con 0x2730790 2023-10-11T09:52:08.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.951+0000 7f75577fe700 10 client.9216 handle_command_reply: tid=0 2023-10-11T09:52:08.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.951+0000 7f75822d8740 1 client.9216 shutdown 2023-10-11T09:52:08.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.951+0000 7f75822d8740 1 -- 172.21.15.78:0/2561822924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26e5eb0 msgr2=0x29940d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:08.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.951+0000 7f75822d8740 1 --2- 172.21.15.78:0/2561822924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26e5eb0 0x29940d0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f7570009010 tx=0x7f7570001250 comp rx=0 tx=0).stop 2023-10-11T09:52:08.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.951+0000 7f75822d8740 1 -- 172.21.15.78:0/2561822924 shutdown_connections 2023-10-11T09:52:08.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.951+0000 7f75822d8740 1 --2- 172.21.15.78:0/2561822924 >> [v2:172.21.15.78:6835/4189411531,v1:172.21.15.78:6837/4189411531] conn(0x2730790 0x29b0f00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:08.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.951+0000 7f75822d8740 1 --2- 172.21.15.78:0/2561822924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26e5eb0 0x29940d0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:08.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.951+0000 7f75822d8740 1 --2- 172.21.15.78:0/2561822924 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x26e5ae0 0x2993b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:08.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.951+0000 7f75577fe700 0 client.9216 ms_handle_reset on v2:172.21.15.78:6835/4189411531 2023-10-11T09:52:08.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.951+0000 7f75822d8740 1 --2- 172.21.15.78:0/2561822924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x26e5710 0x29587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:08.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.951+0000 7f75822d8740 1 -- 172.21.15.78:0/2561822924 >> 172.21.15.78:0/2561822924 conn(0x294dfa0 msgr2=0x2972b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:08.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.951+0000 7f75822d8740 1 -- 172.21.15.78:0/2561822924 shutdown_connections 2023-10-11T09:52:08.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.951+0000 7f75822d8740 1 -- 172.21.15.78:0/2561822924 wait complete. 2023-10-11T09:52:08.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.951+0000 7f75822d8740 20 client.9216 trim_cache size 0 max 16384 2023-10-11T09:52:08.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.983+0000 7f75822d8740 1 Processor -- start 2023-10-11T09:52:08.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.983+0000 7f75822d8740 1 -- start start 2023-10-11T09:52:08.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.987+0000 7f75822d8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26e5710 0x29926e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:08.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.987+0000 7f75822d8740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x26e5ae0 0x2992c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:08.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.987+0000 7f75822d8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x26e5eb0 0x2993160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:08.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.987+0000 7f75822d8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x2864d90 con 0x26e5710 2023-10-11T09:52:08.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.987+0000 7f75822d8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x286ecf0 con 0x26e5eb0 2023-10-11T09:52:08.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.987+0000 7f75822d8740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x287d400 con 0x26e5ae0 2023-10-11T09:52:08.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.987+0000 7f757cddf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26e5710 0x29926e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:08.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.987+0000 7f757cddf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26e5710 0x29926e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39144/0 (socket says 172.21.15.78:39144) 2023-10-11T09:52:08.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.987+0000 7f757cddf700 1 -- 172.21.15.78:0/3449728185 learned_addr learned my addr 172.21.15.78:0/3449728185 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:08.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.987+0000 7f7577fff700 1 --2- 172.21.15.78:0/3449728185 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x26e5ae0 0x2992c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:08.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.987+0000 7f757d5e0700 1 --2- 172.21.15.78:0/3449728185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x26e5eb0 0x2993160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:08.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.987+0000 7f757cddf700 1 -- 172.21.15.78:0/3449728185 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x26e5ae0 msgr2=0x2992c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:08.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.987+0000 7f757cddf700 1 --2- 172.21.15.78:0/3449728185 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x26e5ae0 0x2992c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:08.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.987+0000 7f757cddf700 1 -- 172.21.15.78:0/3449728185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x26e5eb0 msgr2=0x2993160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:08.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.987+0000 7f757cddf700 1 --2- 172.21.15.78:0/3449728185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x26e5eb0 0x2993160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:08.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.987+0000 7f757cddf700 1 -- 172.21.15.78:0/3449728185 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757815cbc0 con 0x26e5710 2023-10-11T09:52:08.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.987+0000 7f757d5e0700 1 --2- 172.21.15.78:0/3449728185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x26e5eb0 0x2993160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:52:08.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.987+0000 7f757cddf700 1 --2- 172.21.15.78:0/3449728185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26e5710 0x29926e0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f757806c750 tx=0x7f756c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99bb9feef315f152 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:08.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.987+0000 7f7562ffd700 1 -- 172.21.15.78:0/3449728185 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c01f070 con 0x26e5710 2023-10-11T09:52:09.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.987+0000 7f7562ffd700 1 -- 172.21.15.78:0/3449728185 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756c02c040 con 0x26e5710 2023-10-11T09:52:09.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.987+0000 7f7562ffd700 1 -- 172.21.15.78:0/3449728185 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c025c60 con 0x26e5710 2023-10-11T09:52:09.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.991+0000 7f75822d8740 1 -- 172.21.15.78:0/3449728185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26e5710 msgr2=0x29926e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:09.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.991+0000 7f75822d8740 1 --2- 172.21.15.78:0/3449728185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26e5710 0x29926e0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f757806c750 tx=0x7f756c010000 comp rx=0 tx=0).stop 2023-10-11T09:52:09.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.991+0000 7f75822d8740 1 -- 172.21.15.78:0/3449728185 shutdown_connections 2023-10-11T09:52:09.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.991+0000 7f75822d8740 1 --2- 172.21.15.78:0/3449728185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x26e5eb0 0x2993160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:09.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.991+0000 7f75822d8740 1 --2- 172.21.15.78:0/3449728185 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x26e5ae0 0x2992c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:09.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.991+0000 7f75822d8740 1 --2- 172.21.15.78:0/3449728185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26e5710 0x29926e0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:09.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.991+0000 7f75822d8740 1 -- 172.21.15.78:0/3449728185 >> 172.21.15.78:0/3449728185 conn(0x294dfa0 msgr2=0x2994c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:09.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.991+0000 7f75822d8740 1 -- 172.21.15.78:0/3449728185 shutdown_connections 2023-10-11T09:52:09.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.991+0000 7f75822d8740 1 -- 172.21.15.78:0/3449728185 wait complete. 2023-10-11T09:52:09.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.991+0000 7f75822d8740 1 -- start start 2023-10-11T09:52:09.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.991+0000 7f75822d8740 1 Processor -- start 2023-10-11T09:52:09.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.995+0000 7f75822d8740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x26e5710 0x29fae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:09.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.995+0000 7f75822d8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26e5ae0 0x29b2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:09.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.995+0000 7f75822d8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x26e5eb0 0x29b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:09.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.995+0000 7f75822d8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x2977210 con 0x26e5ae0 2023-10-11T09:52:09.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.995+0000 7f75822d8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x29b3240 con 0x26e5eb0 2023-10-11T09:52:09.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.995+0000 7f75822d8740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x28c0470 con 0x26e5710 2023-10-11T09:52:09.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.995+0000 7f75822d8740 10 client.0 _ll_register_callbacks cb 0x28bcc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:52:09.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.995+0000 7f7577fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26e5ae0 0x29b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:09.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.995+0000 7f7577fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26e5ae0 0x29b2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39150/0 (socket says 172.21.15.78:39150) 2023-10-11T09:52:09.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.995+0000 7f7577fff700 1 -- 172.21.15.78:0/908331763 learned_addr learned my addr 172.21.15.78:0/908331763 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:09.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.995+0000 7f757d5e0700 1 --2- 172.21.15.78:0/908331763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x26e5eb0 0x29b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:09.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.995+0000 7f757cddf700 1 --2- 172.21.15.78:0/908331763 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x26e5710 0x29fae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:09.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.995+0000 7f7577fff700 1 -- 172.21.15.78:0/908331763 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x26e5710 msgr2=0x29fae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:09.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.995+0000 7f7577fff700 1 --2- 172.21.15.78:0/908331763 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x26e5710 0x29fae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:09.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.995+0000 7f7577fff700 1 -- 172.21.15.78:0/908331763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x26e5eb0 msgr2=0x29b26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:09.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.995+0000 7f7577fff700 1 --2- 172.21.15.78:0/908331763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x26e5eb0 0x29b26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:09.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.995+0000 7f7577fff700 1 -- 172.21.15.78:0/908331763 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75780cd6d0 con 0x26e5ae0 2023-10-11T09:52:09.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.995+0000 7f7577fff700 1 --2- 172.21.15.78:0/908331763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26e5ae0 0x29b2180 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f7564004fd0 tx=0x7f7564007820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c137c501675597 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:09.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.995+0000 7f75577fe700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:52:09.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.995+0000 7f75577fe700 1 -- 172.21.15.78:0/908331763 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7564053ed0 con 0x26e5ae0 2023-10-11T09:52:09.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.995+0000 7f75822d8740 1 -- 172.21.15.78:0/908331763 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f757000edc0 con 0x26e5ae0 2023-10-11T09:52:09.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.995+0000 7f75577fe700 1 -- 172.21.15.78:0/908331763 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f75640547e0 con 0x26e5ae0 2023-10-11T09:52:09.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.995+0000 7f75577fe700 1 -- 172.21.15.78:0/908331763 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7564058610 con 0x26e5ae0 2023-10-11T09:52:09.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.995+0000 7f75822d8740 10 client.9222 fetch_fsmap learned FSMap version 105 2023-10-11T09:52:09.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.995+0000 7f75822d8740 1 -- 172.21.15.78:0/908331763 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x286a980 con 0x26e5ae0 2023-10-11T09:52:09.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.995+0000 7f75577fe700 1 -- 172.21.15.78:0/908331763 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f757000edc0 con 0x26e5ae0 2023-10-11T09:52:09.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.995+0000 7f75577fe700 1 -- 172.21.15.78:0/908331763 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 105) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f756405c330 con 0x26e5ae0 2023-10-11T09:52:09.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.995+0000 7f75822d8740 10 client.9222 fetch_fsmap finished waiting for FSMap version 105 2023-10-11T09:52:09.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.995+0000 7f75822d8740 10 client.9222 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-11T09:52:09.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.995+0000 7f75822d8740 20 client.9222 populate_metadata read hostname 'smithi078' 2023-10-11T09:52:09.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.995+0000 7f75822d8740 1 --2- 172.21.15.78:0/908331763 >> [v2:172.21.15.78:6835/4189411531,v1:172.21.15.78:6837/4189411531] conn(0x297b150 0x297b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:09.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.995+0000 7f75822d8740 4 client.9222 mds_command: new command op to 8880 tid=0[{"prefix": "status"}] 2023-10-11T09:52:09.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.995+0000 7f75822d8740 1 -- 172.21.15.78:0/908331763 --> [v2:172.21.15.78:6835/4189411531,v1:172.21.15.78:6837/4189411531] -- command(tid 0: {"prefix": "status"}) v1 -- 0x273d4e0 con 0x297b150 2023-10-11T09:52:09.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.995+0000 7f757cddf700 1 --2- 172.21.15.78:0/908331763 >> [v2:172.21.15.78:6835/4189411531,v1:172.21.15.78:6837/4189411531] conn(0x297b150 0x297b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:09.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.999+0000 7f757cddf700 1 --2- 172.21.15.78:0/908331763 >> [v2:172.21.15.78:6835/4189411531,v1:172.21.15.78:6837/4189411531] conn(0x297b150 0x297b520 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da045455bc6bd2fb server_cookie=dec18355ea286dd7 in_seq=0 out_seq=0 2023-10-11T09:52:09.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.999+0000 7f75577fe700 10 client.9222 ms_handle_connect on v2:172.21.15.78:6835/4189411531 2023-10-11T09:52:09.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.999+0000 7f75577fe700 1 -- 172.21.15.78:0/908331763 <== mds.0 v2:172.21.15.78:6835/4189411531 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x273d4e0 con 0x297b150 2023-10-11T09:52:09.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.999+0000 7f75577fe700 10 client.9222 handle_command_reply: tid=0 2023-10-11T09:52:09.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.999+0000 7f75822d8740 1 client.9222 shutdown 2023-10-11T09:52:09.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.999+0000 7f75822d8740 1 -- 172.21.15.78:0/908331763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26e5ae0 msgr2=0x29b2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:09.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.999+0000 7f75822d8740 1 --2- 172.21.15.78:0/908331763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26e5ae0 0x29b2180 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f7564004fd0 tx=0x7f7564007820 comp rx=0 tx=0).stop 2023-10-11T09:52:09.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.999+0000 7f75822d8740 1 -- 172.21.15.78:0/908331763 shutdown_connections 2023-10-11T09:52:09.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.999+0000 7f75822d8740 1 --2- 172.21.15.78:0/908331763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x26e5eb0 0x29b26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:09.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.999+0000 7f75822d8740 1 --2- 172.21.15.78:0/908331763 >> [v2:172.21.15.78:6835/4189411531,v1:172.21.15.78:6837/4189411531] conn(0x297b150 0x297b520 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:09.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.999+0000 7f75822d8740 1 --2- 172.21.15.78:0/908331763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x26e5ae0 0x29b2180 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:09.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.999+0000 7f75822d8740 1 --2- 172.21.15.78:0/908331763 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x26e5710 0x29fae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:09.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.999+0000 7f75822d8740 1 -- 172.21.15.78:0/908331763 >> 172.21.15.78:0/908331763 conn(0x294dfa0 msgr2=0x29937d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:09.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.999+0000 7f75577fe700 0 client.9222 ms_handle_reset on v2:172.21.15.78:6835/4189411531 2023-10-11T09:52:09.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.999+0000 7f75822d8740 1 -- 172.21.15.78:0/908331763 shutdown_connections 2023-10-11T09:52:09.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.999+0000 7f75822d8740 1 -- 172.21.15.78:0/908331763 wait complete. 2023-10-11T09:52:09.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.999+0000 7f75822d8740 20 client.9222 trim_cache size 0 max 16384 2023-10-11T09:52:09.018 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:52:09.018 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "dee99db9-7a72-406f-a1e2-917943f2b13d", 2023-10-11T09:52:09.018 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2023-10-11T09:52:09.018 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 8880, 2023-10-11T09:52:09.019 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2023-10-11T09:52:09.019 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2023-10-11T09:52:09.019 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2023-10-11T09:52:09.019 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 5.0916280230000002, 2023-10-11T09:52:09.019 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 105, 2023-10-11T09:52:09.019 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 128, 2023-10-11T09:52:09.020 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 128, 2023-10-11T09:52:09.020 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 33.983328147000002 2023-10-11T09:52:09.020 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:52:09.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.999+0000 7f757dde1700 1 -- 172.21.15.78:0/1425289084 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f755c065ed0 msgr2=0x7f755c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:09.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.999+0000 7f757dde1700 1 --2- 172.21.15.78:0/1425289084 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f755c065ed0 0x7f755c068380 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f756c0090b0 tx=0x7f756c008030 comp rx=0 tx=0).stop 2023-10-11T09:52:09.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.999+0000 7f757dde1700 1 -- 172.21.15.78:0/1425289084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f757815a230 msgr2=0x7f75781589e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:09.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.999+0000 7f757dde1700 1 --2- 172.21.15.78:0/1425289084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f757815a230 0x7f75781589e0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f756400c450 tx=0x7f756400c810 comp rx=0 tx=0).stop 2023-10-11T09:52:09.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.999+0000 7f757dde1700 1 -- 172.21.15.78:0/1425289084 shutdown_connections 2023-10-11T09:52:09.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.999+0000 7f757dde1700 1 --2- 172.21.15.78:0/1425289084 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f755c065ed0 0x7f755c068380 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:09.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.999+0000 7f757dde1700 1 --2- 172.21.15.78:0/1425289084 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f757815d760 0x7f75780b8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:09.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.999+0000 7f757dde1700 1 --2- 172.21.15.78:0/1425289084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f757815a230 0x7f75781589e0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:09.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.999+0000 7f757dde1700 1 --2- 172.21.15.78:0/1425289084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75780cffa0 0x7f75780c6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:09.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.999+0000 7f757dde1700 1 -- 172.21.15.78:0/1425289084 >> 172.21.15.78:0/1425289084 conn(0x7f757806aa20 msgr2=0x7f75780d0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:09.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.999+0000 7f757dde1700 1 -- 172.21.15.78:0/1425289084 shutdown_connections 2023-10-11T09:52:09.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:08.999+0000 7f757dde1700 1 -- 172.21.15.78:0/1425289084 wait complete. 2023-10-11T09:52:09.029 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-11T09:52:09.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.191+0000 7f0b2de0f700 1 Processor -- start 2023-10-11T09:52:09.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.191+0000 7f0b2de0f700 1 -- start start 2023-10-11T09:52:09.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.191+0000 7f0b2de0f700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0b28071720 0x7f0b280c20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:09.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.191+0000 7f0b2de0f700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b280cb220 con 0x7f0b28161020 2023-10-11T09:52:09.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.191+0000 7f0b2de0f700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b281613f0 con 0x7f0b280ce700 2023-10-11T09:52:09.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.191+0000 7f0b2de0f700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b28071c70 con 0x7f0b28071720 2023-10-11T09:52:09.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.191+0000 7f0b2ce0d700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f0b28161020 0x7f0b280cdff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:60498/0 (socket says 172.21.15.78:60498) 2023-10-11T09:52:09.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.191+0000 7f0b2ce0d700 1 -- 172.21.15.78:0/2071295888 learned_addr learned my addr 172.21.15.78:0/2071295888 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:09.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.191+0000 7f0b2d60e700 1 --2- 172.21.15.78:0/2071295888 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0b28071720 0x7f0b280c20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:09.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.195+0000 7f0b1f7fe700 1 -- 172.21.15.78:0/2071295888 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2806437510 0 0) 0x7f0b281613f0 con 0x7f0b280ce700 2023-10-11T09:52:09.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.195+0000 7f0b1f7fe700 1 -- 172.21.15.78:0/2071295888 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b100035d0 con 0x7f0b280ce700 2023-10-11T09:52:09.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.195+0000 7f0b1f7fe700 1 -- 172.21.15.78:0/2071295888 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2627591383 0 0) 0x7f0b280cb220 con 0x7f0b28161020 2023-10-11T09:52:09.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.195+0000 7f0b1f7fe700 1 -- 172.21.15.78:0/2071295888 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b281613f0 con 0x7f0b28161020 2023-10-11T09:52:09.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.195+0000 7f0b2d60e700 1 -- 172.21.15.78:0/2071295888 >> v1:172.21.15.133:6789/0 conn(0x7f0b280ce700 legacy=0x7f0b2806bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:09.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.195+0000 7f0b2d60e700 1 -- 172.21.15.78:0/2071295888 >> v1:172.21.15.78:6789/0 conn(0x7f0b28161020 legacy=0x7f0b280cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:09.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.195+0000 7f0b2d60e700 1 -- 172.21.15.78:0/2071295888 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b280ca650 con 0x7f0b28071720 2023-10-11T09:52:09.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.195+0000 7f0b2d60e700 1 --2- 172.21.15.78:0/2071295888 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0b28071720 0x7f0b280c20c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0b20009dc0 tx=0x7f0b2000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=560b7d0ac050250a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:09.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.195+0000 7f0b1f7fe700 1 -- 172.21.15.78:0/2071295888 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2000e070 con 0x7f0b28071720 2023-10-11T09:52:09.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.195+0000 7f0b1f7fe700 1 -- 172.21.15.78:0/2071295888 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b2000bb20 con 0x7f0b28071720 2023-10-11T09:52:09.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.195+0000 7f0b1f7fe700 1 -- 172.21.15.78:0/2071295888 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b20012430 con 0x7f0b28071720 2023-10-11T09:52:09.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.195+0000 7f0b2de0f700 1 -- 172.21.15.78:0/2071295888 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0b28071720 msgr2=0x7f0b280c20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:09.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.195+0000 7f0b2de0f700 1 --2- 172.21.15.78:0/2071295888 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0b28071720 0x7f0b280c20c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0b20009dc0 tx=0x7f0b2000b330 comp rx=0 tx=0).stop 2023-10-11T09:52:09.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.195+0000 7f0b2de0f700 1 -- 172.21.15.78:0/2071295888 shutdown_connections 2023-10-11T09:52:09.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.195+0000 7f0b2de0f700 1 --2- 172.21.15.78:0/2071295888 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0b28071720 0x7f0b280c20c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:09.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.195+0000 7f0b2de0f700 1 -- 172.21.15.78:0/2071295888 >> 172.21.15.78:0/2071295888 conn(0x7f0b280cf820 msgr2=0x7f0b280cb610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:09.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.195+0000 7f0b2de0f700 1 -- 172.21.15.78:0/2071295888 shutdown_connections 2023-10-11T09:52:09.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.195+0000 7f0b2de0f700 1 -- 172.21.15.78:0/2071295888 wait complete. 2023-10-11T09:52:09.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.199+0000 7f0b2de0f700 1 Processor -- start 2023-10-11T09:52:09.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.199+0000 7f0b2de0f700 1 -- start start 2023-10-11T09:52:09.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.199+0000 7f0b2de0f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b28071720 0x7f0b281f79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:09.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.199+0000 7f0b2de0f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b280ce700 0x7f0b280c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:09.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.199+0000 7f0b2de0f700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0b28161020 0x7f0b281f68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:09.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.199+0000 7f0b2de0f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b280c3fa0 con 0x7f0b28071720 2023-10-11T09:52:09.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.199+0000 7f0b2de0f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b281f7ef0 con 0x7f0b280ce700 2023-10-11T09:52:09.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.199+0000 7f0b2de0f700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b280c8a10 con 0x7f0b28161020 2023-10-11T09:52:09.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.199+0000 7f0b2ce0d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b28071720 0x7f0b281f79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:09.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.199+0000 7f0b2ce0d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b28071720 0x7f0b281f79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39160/0 (socket says 172.21.15.78:39160) 2023-10-11T09:52:09.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.199+0000 7f0b2ce0d700 1 -- 172.21.15.78:0/28002251 learned_addr learned my addr 172.21.15.78:0/28002251 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:09.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.199+0000 7f0b1ffff700 1 --2- 172.21.15.78:0/28002251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b280ce700 0x7f0b280c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:09.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.199+0000 7f0b2d60e700 1 --2- 172.21.15.78:0/28002251 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0b28161020 0x7f0b281f68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:09.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.203+0000 7f0b2ce0d700 1 -- 172.21.15.78:0/28002251 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0b28161020 msgr2=0x7f0b281f68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:09.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.203+0000 7f0b2ce0d700 1 --2- 172.21.15.78:0/28002251 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0b28161020 0x7f0b281f68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:09.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.203+0000 7f0b2ce0d700 1 -- 172.21.15.78:0/28002251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b280ce700 msgr2=0x7f0b280c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:09.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.203+0000 7f0b2ce0d700 1 --2- 172.21.15.78:0/28002251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b280ce700 0x7f0b280c84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:09.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.203+0000 7f0b2ce0d700 1 -- 172.21.15.78:0/28002251 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b20009a70 con 0x7f0b28071720 2023-10-11T09:52:09.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.203+0000 7f0b2ce0d700 1 --2- 172.21.15.78:0/28002251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b28071720 0x7f0b281f79b0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f0b2400c4d0 tx=0x7f0b2400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:09.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.203+0000 7f0b1d7fa700 1 -- 172.21.15.78:0/28002251 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b24005d90 con 0x7f0b28071720 2023-10-11T09:52:09.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.203+0000 7f0b1d7fa700 1 -- 172.21.15.78:0/28002251 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b240070d0 con 0x7f0b28071720 2023-10-11T09:52:09.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.203+0000 7f0b2de0f700 1 -- 172.21.15.78:0/28002251 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b281f7160 con 0x7f0b28071720 2023-10-11T09:52:09.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.203+0000 7f0b1d7fa700 1 -- 172.21.15.78:0/28002251 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b24007480 con 0x7f0b28071720 2023-10-11T09:52:09.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.203+0000 7f0b2de0f700 1 -- 172.21.15.78:0/28002251 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b281f7720 con 0x7f0b28071720 2023-10-11T09:52:09.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.203+0000 7f0b2de0f700 1 -- 172.21.15.78:0/28002251 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b28067d00 con 0x7f0b28071720 2023-10-11T09:52:09.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.207+0000 7f0b1d7fa700 1 -- 172.21.15.78:0/28002251 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f0b240075e0 con 0x7f0b28071720 2023-10-11T09:52:09.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.207+0000 7f0b1d7fa700 1 --2- 172.21.15.78:0/28002251 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0b14065ed0 0x7f0b14068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:09.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.207+0000 7f0b1d7fa700 1 -- 172.21.15.78:0/28002251 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f0b24082dc0 con 0x7f0b28071720 2023-10-11T09:52:09.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.207+0000 7f0b1ffff700 1 --2- 172.21.15.78:0/28002251 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0b14065ed0 0x7f0b14068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:09.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.207+0000 7f0b1ffff700 1 --2- 172.21.15.78:0/28002251 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0b14065ed0 0x7f0b14068380 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0b2815ffe0 tx=0x7f0b18009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:09.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.211+0000 7f0b1d7fa700 1 -- 172.21.15.78:0/28002251 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f0b240503a0 con 0x7f0b28071720 2023-10-11T09:52:09.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.423+0000 7f0b2de0f700 1 -- 172.21.15.78:0/28002251 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0b280c19c0 con 0x7f0b28071720 2023-10-11T09:52:09.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.423+0000 7f0b1d7fa700 1 -- 172.21.15.78:0/28002251 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v105) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0b24017060 con 0x7f0b28071720 2023-10-11T09:52:09.432 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-11T09:52:09.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.427+0000 7f0b2de0f700 1 -- 172.21.15.78:0/28002251 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0b14065ed0 msgr2=0x7f0b14068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:09.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.427+0000 7f0b2de0f700 1 --2- 172.21.15.78:0/28002251 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0b14065ed0 0x7f0b14068380 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0b2815ffe0 tx=0x7f0b18009040 comp rx=0 tx=0).stop 2023-10-11T09:52:09.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.427+0000 7f0b2de0f700 1 -- 172.21.15.78:0/28002251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b28071720 msgr2=0x7f0b281f79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:09.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.427+0000 7f0b2de0f700 1 --2- 172.21.15.78:0/28002251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b28071720 0x7f0b281f79b0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f0b2400c4d0 tx=0x7f0b2400c890 comp rx=0 tx=0).stop 2023-10-11T09:52:09.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.427+0000 7f0b2de0f700 1 -- 172.21.15.78:0/28002251 shutdown_connections 2023-10-11T09:52:09.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.427+0000 7f0b2de0f700 1 --2- 172.21.15.78:0/28002251 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0b14065ed0 0x7f0b14068380 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:09.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.427+0000 7f0b2de0f700 1 --2- 172.21.15.78:0/28002251 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0b28161020 0x7f0b281f68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:09.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.427+0000 7f0b2de0f700 1 --2- 172.21.15.78:0/28002251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b280ce700 0x7f0b280c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:09.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.427+0000 7f0b2de0f700 1 --2- 172.21.15.78:0/28002251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b28071720 0x7f0b281f79b0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:09.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.427+0000 7f0b2de0f700 1 -- 172.21.15.78:0/28002251 >> 172.21.15.78:0/28002251 conn(0x7f0b280cf820 msgr2=0x7f0b2806f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:09.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.427+0000 7f0b2de0f700 1 -- 172.21.15.78:0/28002251 shutdown_connections 2023-10-11T09:52:09.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:09.427+0000 7f0b2de0f700 1 -- 172.21.15.78:0/28002251 wait complete. 2023-10-11T09:52:09.456 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-11T09:52:09.457 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-11T09:52:09.457 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi078.front.sepia.ceph.com 2023-10-11T09:52:09.458 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-11T09:52:09.458 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-11T09:52:09.458 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-11T09:52:09.459 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-11T09:52:09.459 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-11T09:52:09.459 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-11T09:52:09.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:09.460 DEBUG:teuthology.orchestra.run.smithi078:> ip addr 2023-10-11T09:52:09.477 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-11T09:52:09.478 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-11T09:52:09.478 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2023-10-11T09:52:09.478 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:52:09.478 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2023-10-11T09:52:09.479 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:52:09.479 INFO:teuthology.orchestra.run.smithi078.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:52:09.479 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:22 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:52:09.479 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f0 2023-10-11T09:52:09.479 INFO:teuthology.orchestra.run.smithi078.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:52:09.480 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:23 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:52:09.480 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f1 2023-10-11T09:52:09.480 INFO:teuthology.orchestra.run.smithi078.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:52:09.480 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dc brd ff:ff:ff:ff:ff:ff 2023-10-11T09:52:09.481 INFO:teuthology.orchestra.run.smithi078.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-11T09:52:09.481 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dd brd ff:ff:ff:ff:ff:ff 2023-10-11T09:52:09.481 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.21.15.78/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-11T09:52:09.481 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft 41735sec preferred_lft 41735sec 2023-10-11T09:52:09.482 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::ec4:7aff:fe8f:d4dd/64 scope link 2023-10-11T09:52:09.482 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:52:09.482 INFO:teuthology.orchestra.run.smithi078.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-11T09:52:09.482 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 02:42:b6:e4:f8:0d brd ff:ff:ff:ff:ff:ff 2023-10-11T09:52:09.483 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-11T09:52:09.483 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:52:09.483 INFO:teuthology.orchestra.run.smithi078.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-11T09:52:09.483 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 7a:89:8a:a9:e1:23 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:52:09.483 INFO:teuthology.orchestra.run.smithi078.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-11T09:52:09.484 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:52:09.484 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::8ca:ebff:fea9:13f1/64 scope link 2023-10-11T09:52:09.484 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:52:09.484 INFO:teuthology.orchestra.run.smithi078.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-11T09:52:09.484 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 7a:89:8a:a9:e1:23 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-11T09:52:09.485 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::7889:8aff:fea9:e123/64 scope link 2023-10-11T09:52:09.485 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:52:09.485 INFO:teuthology.orchestra.run.smithi078.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-11T09:52:09.485 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether ca:ac:e8:27:51:0d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-11T09:52:09.485 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::c8ac:e8ff:fe27:510d/64 scope link 2023-10-11T09:52:09.485 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:52:09.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:09.487 DEBUG:teuthology.orchestra.run.smithi078:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-11T09:52:09.566 INFO:teuthology.orchestra.run.smithi078.stdout:1 2023-10-11T09:52:09.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:09.569 DEBUG:teuthology.orchestra.run.smithi078:> route 2023-10-11T09:52:09.633 INFO:teuthology.orchestra.run.smithi078.stdout:Kernel IP routing table 2023-10-11T09:52:09.634 INFO:teuthology.orchestra.run.smithi078.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-11T09:52:09.634 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-11T09:52:09.634 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-11T09:52:09.634 INFO:teuthology.orchestra.run.smithi078.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-11T09:52:09.634 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-10-11T09:52:09.635 INFO:teuthology.orchestra.run.smithi078.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-11T09:52:09.635 INFO:teuthology.orchestra.run.smithi078.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-11T09:52:09.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:09.636 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec bash -c ' 2023-10-11T09:52:09.636 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2023-10-11T09:52:09.636 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-11T09:52:09.637 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-11T09:52:09.637 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-11T09:52:09.637 DEBUG:teuthology.orchestra.run.smithi078:> ') 2023-10-11T09:52:09.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:09.775 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2023-10-11T09:52:09.793 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-11T09:52:09.793 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-11T09:52:09.794 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-11T09:52:09.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:09.795 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set brx.0 up 2023-10-11T09:52:09.871 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-11T09:52:09.871 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T09:52:09.872 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-11T09:52:09.930 INFO:teuthology.orchestra.run.smithi078.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-11T09:52:09.930 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T09:52:09.931 DEBUG:teuthology.orchestra.run.smithi078:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-11T09:52:09.990 DEBUG:teuthology.orchestra.run.smithi078:> sudo modprobe fuse 2023-10-11T09:52:10.073 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T09:52:10.074 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T09:52:10.150 INFO:teuthology.orchestra.run.smithi078.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T09:52:10.153 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T09:52:10.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:10.154 DEBUG:teuthology.orchestra.run.smithi078:> ls /sys/fs/fuse/connections 2023-10-11T09:52:10.212 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-11T09:52:10.213 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-11T09:52:10.266 DEBUG:teuthology.orchestra.run.smithi078:> sudo modprobe fuse 2023-10-11T09:52:10.304 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T09:52:10.305 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T09:52:10.381 INFO:teuthology.orchestra.run.smithi078.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T09:52:10.384 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T09:52:10.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:10.385 DEBUG:teuthology.orchestra.run.smithi078:> ls /sys/fs/fuse/connections 2023-10-11T09:52:10.406 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:2023-10-11T09:52:10.399+0000 7f6d32f8c640 -1 init, newargv = 0x56182d8aea60 newargc=15 2023-10-11T09:52:10.406 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:ceph-fuse[60247]: starting ceph client 2023-10-11T09:52:10.416 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:ceph-fuse[60247]: starting fuse 2023-10-11T09:52:10.420 INFO:teuthology.orchestra.run.smithi078.stdout:49 2023-10-11T09:52:10.420 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2023-10-11T09:52:10.420 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:52:10.421 DEBUG:teuthology.orchestra.run.smithi078:> import glob 2023-10-11T09:52:10.421 DEBUG:teuthology.orchestra.run.smithi078:> import re 2023-10-11T09:52:10.422 DEBUG:teuthology.orchestra.run.smithi078:> import os 2023-10-11T09:52:10.422 DEBUG:teuthology.orchestra.run.smithi078:> import subprocess 2023-10-11T09:52:10.422 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:52:10.423 DEBUG:teuthology.orchestra.run.smithi078:> def _find_admin_socket(client_name): 2023-10-11T09:52:10.423 DEBUG:teuthology.orchestra.run.smithi078:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-11T09:52:10.424 DEBUG:teuthology.orchestra.run.smithi078:> files = glob.glob(asok_path) 2023-10-11T09:52:10.424 DEBUG:teuthology.orchestra.run.smithi078:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-11T09:52:10.424 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:52:10.425 DEBUG:teuthology.orchestra.run.smithi078:> # Given a non-glob path, it better be there 2023-10-11T09:52:10.425 DEBUG:teuthology.orchestra.run.smithi078:> if "*" not in asok_path: 2023-10-11T09:52:10.425 DEBUG:teuthology.orchestra.run.smithi078:> assert(len(files) == 1) 2023-10-11T09:52:10.426 DEBUG:teuthology.orchestra.run.smithi078:> return files[0] 2023-10-11T09:52:10.430 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:52:10.431 DEBUG:teuthology.orchestra.run.smithi078:> for f in files: 2023-10-11T09:52:10.432 DEBUG:teuthology.orchestra.run.smithi078:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-11T09:52:10.432 DEBUG:teuthology.orchestra.run.smithi078:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-11T09:52:10.432 DEBUG:teuthology.orchestra.run.smithi078:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-11T09:52:10.433 DEBUG:teuthology.orchestra.run.smithi078:> contents = proc_f.read() 2023-10-11T09:52:10.433 DEBUG:teuthology.orchestra.run.smithi078:> if mountpoint in contents: 2023-10-11T09:52:10.433 DEBUG:teuthology.orchestra.run.smithi078:> return f 2023-10-11T09:52:10.434 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-11T09:52:10.434 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:52:10.435 DEBUG:teuthology.orchestra.run.smithi078:> print(_find_admin_socket("client.0")) 2023-10-11T09:52:10.435 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:52:10.573 INFO:teuthology.orchestra.run.smithi078.stdout:/var/run/ceph/ceph-client.0.60247.asok 2023-10-11T09:52:10.726 INFO:tasks.ceph.mds.c.smithi078.stderr:2023-10-11T09:52:10.719+0000 7f3086953700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-11T09:52:10.759 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.60247.asok 2023-10-11T09:52:10.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:10.760 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.60247.asok status 2023-10-11T09:52:10.957 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:52:10.957 INFO:teuthology.orchestra.run.smithi078.stdout: "metadata": { 2023-10-11T09:52:10.958 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_sha1": "9a774796f0ecdd11a4b771de396e627aa1a187af", 2023-10-11T09:52:10.958 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_version": "ceph version 18.2.0-662-g9a774796 (9a774796f0ecdd11a4b771de396e627aa1a187af) reef (stable)", 2023-10-11T09:52:10.958 INFO:teuthology.orchestra.run.smithi078.stdout: "entity_id": "0", 2023-10-11T09:52:10.958 INFO:teuthology.orchestra.run.smithi078.stdout: "hostname": "smithi078", 2023-10-11T09:52:10.958 INFO:teuthology.orchestra.run.smithi078.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-11T09:52:10.959 INFO:teuthology.orchestra.run.smithi078.stdout: "pid": "60247", 2023-10-11T09:52:10.959 INFO:teuthology.orchestra.run.smithi078.stdout: "root": "/" 2023-10-11T09:52:10.959 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:52:10.959 INFO:teuthology.orchestra.run.smithi078.stdout: "dentry_count": 0, 2023-10-11T09:52:10.959 INFO:teuthology.orchestra.run.smithi078.stdout: "dentry_pinned_count": 0, 2023-10-11T09:52:10.960 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 9237, 2023-10-11T09:52:10.960 INFO:teuthology.orchestra.run.smithi078.stdout: "inst": { 2023-10-11T09:52:10.960 INFO:teuthology.orchestra.run.smithi078.stdout: "name": { 2023-10-11T09:52:10.960 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "client", 2023-10-11T09:52:10.961 INFO:teuthology.orchestra.run.smithi078.stdout: "num": 9237 2023-10-11T09:52:10.961 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:52:10.961 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": { 2023-10-11T09:52:10.961 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2023-10-11T09:52:10.961 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "192.168.0.1:0", 2023-10-11T09:52:10.961 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 114506255 2023-10-11T09:52:10.962 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:52:10.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:52:10.962 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": { 2023-10-11T09:52:10.962 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2023-10-11T09:52:10.963 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "192.168.0.1:0", 2023-10-11T09:52:10.963 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 114506255 2023-10-11T09:52:10.963 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:52:10.963 INFO:teuthology.orchestra.run.smithi078.stdout: "inst_str": "client.9237 192.168.0.1:0/114506255", 2023-10-11T09:52:10.963 INFO:teuthology.orchestra.run.smithi078.stdout: "addr_str": "192.168.0.1:0/114506255", 2023-10-11T09:52:10.964 INFO:teuthology.orchestra.run.smithi078.stdout: "inode_count": 1, 2023-10-11T09:52:10.964 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_epoch": 105, 2023-10-11T09:52:10.964 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_epoch": 128, 2023-10-11T09:52:10.964 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_epoch_barrier": 0, 2023-10-11T09:52:10.964 INFO:teuthology.orchestra.run.smithi078.stdout: "blocklisted": false, 2023-10-11T09:52:10.964 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs" 2023-10-11T09:52:10.965 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:52:10.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:10.973 DEBUG:teuthology.orchestra.run.smithi078:> stat --file-system '--printf=%T 2023-10-11T09:52:10.973 DEBUG:teuthology.orchestra.run.smithi078:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-11T09:52:10.992 INFO:teuthology.orchestra.run.smithi078.stdout:fuseblk 2023-10-11T09:52:10.992 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-11T09:52:10.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:10.993 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-11T09:52:11.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:11.069 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2023-10-11T09:52:11.165 INFO:teuthology.orchestra.run.smithi078.stdout:0 2023-10-11T09:52:11.165 INFO:teuthology.orchestra.run.smithi078.stdout:1 2023-10-11T09:52:11.165 INFO:teuthology.orchestra.run.smithi078.stdout:10 2023-10-11T09:52:11.165 INFO:teuthology.orchestra.run.smithi078.stdout:11 2023-10-11T09:52:11.166 INFO:teuthology.orchestra.run.smithi078.stdout:12 2023-10-11T09:52:11.166 INFO:teuthology.orchestra.run.smithi078.stdout:13 2023-10-11T09:52:11.166 INFO:teuthology.orchestra.run.smithi078.stdout:14 2023-10-11T09:52:11.166 INFO:teuthology.orchestra.run.smithi078.stdout:15 2023-10-11T09:52:11.166 INFO:teuthology.orchestra.run.smithi078.stdout:16 2023-10-11T09:52:11.167 INFO:teuthology.orchestra.run.smithi078.stdout:17 2023-10-11T09:52:11.167 INFO:teuthology.orchestra.run.smithi078.stdout:18 2023-10-11T09:52:11.167 INFO:teuthology.orchestra.run.smithi078.stdout:19 2023-10-11T09:52:11.167 INFO:teuthology.orchestra.run.smithi078.stdout:2 2023-10-11T09:52:11.167 INFO:teuthology.orchestra.run.smithi078.stdout:20 2023-10-11T09:52:11.168 INFO:teuthology.orchestra.run.smithi078.stdout:22 2023-10-11T09:52:11.168 INFO:teuthology.orchestra.run.smithi078.stdout:23 2023-10-11T09:52:11.168 INFO:teuthology.orchestra.run.smithi078.stdout:24 2023-10-11T09:52:11.168 INFO:teuthology.orchestra.run.smithi078.stdout:25 2023-10-11T09:52:11.168 INFO:teuthology.orchestra.run.smithi078.stdout:26 2023-10-11T09:52:11.169 INFO:teuthology.orchestra.run.smithi078.stdout:27 2023-10-11T09:52:11.169 INFO:teuthology.orchestra.run.smithi078.stdout:28 2023-10-11T09:52:11.169 INFO:teuthology.orchestra.run.smithi078.stdout:29 2023-10-11T09:52:11.170 INFO:teuthology.orchestra.run.smithi078.stdout:3 2023-10-11T09:52:11.170 INFO:teuthology.orchestra.run.smithi078.stdout:30 2023-10-11T09:52:11.170 INFO:teuthology.orchestra.run.smithi078.stdout:31 2023-10-11T09:52:11.171 INFO:teuthology.orchestra.run.smithi078.stdout:32 2023-10-11T09:52:11.171 INFO:teuthology.orchestra.run.smithi078.stdout:33 2023-10-11T09:52:11.171 INFO:teuthology.orchestra.run.smithi078.stdout:34 2023-10-11T09:52:11.171 INFO:teuthology.orchestra.run.smithi078.stdout:35 2023-10-11T09:52:11.171 INFO:teuthology.orchestra.run.smithi078.stdout:36 2023-10-11T09:52:11.172 INFO:teuthology.orchestra.run.smithi078.stdout:37 2023-10-11T09:52:11.172 INFO:teuthology.orchestra.run.smithi078.stdout:38 2023-10-11T09:52:11.172 INFO:teuthology.orchestra.run.smithi078.stdout:39 2023-10-11T09:52:11.172 INFO:teuthology.orchestra.run.smithi078.stdout:4 2023-10-11T09:52:11.172 INFO:teuthology.orchestra.run.smithi078.stdout:40 2023-10-11T09:52:11.173 INFO:teuthology.orchestra.run.smithi078.stdout:41 2023-10-11T09:52:11.173 INFO:teuthology.orchestra.run.smithi078.stdout:42 2023-10-11T09:52:11.173 INFO:teuthology.orchestra.run.smithi078.stdout:43 2023-10-11T09:52:11.173 INFO:teuthology.orchestra.run.smithi078.stdout:44 2023-10-11T09:52:11.173 INFO:teuthology.orchestra.run.smithi078.stdout:45 2023-10-11T09:52:11.173 INFO:teuthology.orchestra.run.smithi078.stdout:46 2023-10-11T09:52:11.174 INFO:teuthology.orchestra.run.smithi078.stdout:47 2023-10-11T09:52:11.174 INFO:teuthology.orchestra.run.smithi078.stdout:48 2023-10-11T09:52:11.174 INFO:teuthology.orchestra.run.smithi078.stdout:49 2023-10-11T09:52:11.174 INFO:teuthology.orchestra.run.smithi078.stdout:5 2023-10-11T09:52:11.174 INFO:teuthology.orchestra.run.smithi078.stdout:50 2023-10-11T09:52:11.175 INFO:teuthology.orchestra.run.smithi078.stdout:51 2023-10-11T09:52:11.175 INFO:teuthology.orchestra.run.smithi078.stdout:52 2023-10-11T09:52:11.175 INFO:teuthology.orchestra.run.smithi078.stdout:53 2023-10-11T09:52:11.175 INFO:teuthology.orchestra.run.smithi078.stdout:54 2023-10-11T09:52:11.175 INFO:teuthology.orchestra.run.smithi078.stdout:55 2023-10-11T09:52:11.176 INFO:teuthology.orchestra.run.smithi078.stdout:56 2023-10-11T09:52:11.176 INFO:teuthology.orchestra.run.smithi078.stdout:57 2023-10-11T09:52:11.176 INFO:teuthology.orchestra.run.smithi078.stdout:58 2023-10-11T09:52:11.176 INFO:teuthology.orchestra.run.smithi078.stdout:59 2023-10-11T09:52:11.176 INFO:teuthology.orchestra.run.smithi078.stdout:6 2023-10-11T09:52:11.177 INFO:teuthology.orchestra.run.smithi078.stdout:60 2023-10-11T09:52:11.177 INFO:teuthology.orchestra.run.smithi078.stdout:61 2023-10-11T09:52:11.177 INFO:teuthology.orchestra.run.smithi078.stdout:62 2023-10-11T09:52:11.177 INFO:teuthology.orchestra.run.smithi078.stdout:63 2023-10-11T09:52:11.177 INFO:teuthology.orchestra.run.smithi078.stdout:64 2023-10-11T09:52:11.178 INFO:teuthology.orchestra.run.smithi078.stdout:65 2023-10-11T09:52:11.178 INFO:teuthology.orchestra.run.smithi078.stdout:66 2023-10-11T09:52:11.178 INFO:teuthology.orchestra.run.smithi078.stdout:67 2023-10-11T09:52:11.178 INFO:teuthology.orchestra.run.smithi078.stdout:68 2023-10-11T09:52:11.178 INFO:teuthology.orchestra.run.smithi078.stdout:69 2023-10-11T09:52:11.179 INFO:teuthology.orchestra.run.smithi078.stdout:7 2023-10-11T09:52:11.179 INFO:teuthology.orchestra.run.smithi078.stdout:70 2023-10-11T09:52:11.179 INFO:teuthology.orchestra.run.smithi078.stdout:71 2023-10-11T09:52:11.179 INFO:teuthology.orchestra.run.smithi078.stdout:72 2023-10-11T09:52:11.179 INFO:teuthology.orchestra.run.smithi078.stdout:73 2023-10-11T09:52:11.179 INFO:teuthology.orchestra.run.smithi078.stdout:74 2023-10-11T09:52:11.180 INFO:teuthology.orchestra.run.smithi078.stdout:75 2023-10-11T09:52:11.180 INFO:teuthology.orchestra.run.smithi078.stdout:76 2023-10-11T09:52:11.180 INFO:teuthology.orchestra.run.smithi078.stdout:77 2023-10-11T09:52:11.180 INFO:teuthology.orchestra.run.smithi078.stdout:78 2023-10-11T09:52:11.180 INFO:teuthology.orchestra.run.smithi078.stdout:79 2023-10-11T09:52:11.181 INFO:teuthology.orchestra.run.smithi078.stdout:8 2023-10-11T09:52:11.181 INFO:teuthology.orchestra.run.smithi078.stdout:80 2023-10-11T09:52:11.181 INFO:teuthology.orchestra.run.smithi078.stdout:81 2023-10-11T09:52:11.181 INFO:teuthology.orchestra.run.smithi078.stdout:82 2023-10-11T09:52:11.181 INFO:teuthology.orchestra.run.smithi078.stdout:83 2023-10-11T09:52:11.182 INFO:teuthology.orchestra.run.smithi078.stdout:84 2023-10-11T09:52:11.182 INFO:teuthology.orchestra.run.smithi078.stdout:85 2023-10-11T09:52:11.182 INFO:teuthology.orchestra.run.smithi078.stdout:86 2023-10-11T09:52:11.182 INFO:teuthology.orchestra.run.smithi078.stdout:87 2023-10-11T09:52:11.182 INFO:teuthology.orchestra.run.smithi078.stdout:88 2023-10-11T09:52:11.183 INFO:teuthology.orchestra.run.smithi078.stdout:89 2023-10-11T09:52:11.183 INFO:teuthology.orchestra.run.smithi078.stdout:9 2023-10-11T09:52:11.183 INFO:teuthology.orchestra.run.smithi078.stdout:90 2023-10-11T09:52:11.183 INFO:teuthology.orchestra.run.smithi078.stdout:91 2023-10-11T09:52:11.183 INFO:teuthology.orchestra.run.smithi078.stdout:92 2023-10-11T09:52:11.183 INFO:teuthology.orchestra.run.smithi078.stdout:93 2023-10-11T09:52:11.184 INFO:teuthology.orchestra.run.smithi078.stdout:94 2023-10-11T09:52:11.184 INFO:teuthology.orchestra.run.smithi078.stdout:95 2023-10-11T09:52:11.184 INFO:teuthology.orchestra.run.smithi078.stdout:96 2023-10-11T09:52:11.184 INFO:teuthology.orchestra.run.smithi078.stdout:97 2023-10-11T09:52:11.185 INFO:teuthology.orchestra.run.smithi078.stdout:98 2023-10-11T09:52:11.185 INFO:teuthology.orchestra.run.smithi078.stdout:99 2023-10-11T09:52:11.186 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:52:11.186 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:52:11.249 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:52:11.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.459+0000 7fee958fa700 1 Processor -- start 2023-10-11T09:52:11.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.459+0000 7fee958fa700 1 -- start start 2023-10-11T09:52:11.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.463+0000 7fee958fa700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fee9015a720 0x7fee9006a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:11.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.463+0000 7fee958fa700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fee9006a6c0 0x7fee900cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:11.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.463+0000 7fee958fa700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee900cb280 con 0x7fee900cef70 2023-10-11T09:52:11.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.463+0000 7fee958fa700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee900cf410 con 0x7fee9006a6c0 2023-10-11T09:52:11.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.463+0000 7fee958fa700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee9006abd0 con 0x7fee9015a720 2023-10-11T09:52:11.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.463+0000 7fee8e7fc700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fee9006a6c0 0x7fee900cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:11.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.463+0000 7fee8e7fc700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fee9006a6c0 0x7fee900cea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39168/0 (socket says 172.21.15.78:39168) 2023-10-11T09:52:11.473 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.463+0000 7fee8e7fc700 1 -- 172.21.15.78:0/3931746775 learned_addr learned my addr 172.21.15.78:0/3931746775 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:11.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.463+0000 7fee8effd700 1 --2- 172.21.15.78:0/3931746775 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fee9015a720 0x7fee9006a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:11.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.463+0000 7fee8dffb700 1 -- 172.21.15.78:0/3931746775 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3548858803 0 0) 0x7fee900cb280 con 0x7fee900cef70 2023-10-11T09:52:11.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.463+0000 7fee8e7fc700 1 -- 172.21.15.78:0/3931746775 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fee9015a720 msgr2=0x7fee9006a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:11.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.463+0000 7fee8e7fc700 1 --2- 172.21.15.78:0/3931746775 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fee9015a720 0x7fee9006a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:11.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.463+0000 7fee8e7fc700 1 -- 172.21.15.78:0/3931746775 >> v1:172.21.15.78:6789/0 conn(0x7fee900cef70 legacy=0x7fee90071520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:11.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.463+0000 7fee8e7fc700 1 -- 172.21.15.78:0/3931746775 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee9006aeb0 con 0x7fee9006a6c0 2023-10-11T09:52:11.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.467+0000 7fee8e7fc700 1 --2- 172.21.15.78:0/3931746775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee9006a6c0 0x7fee900cea30 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fee84009910 tx=0x7fee84009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c9c55ccc5364f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:11.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.467+0000 7fee8dffb700 1 -- 172.21.15.78:0/3931746775 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8400ba60 con 0x7fee9006a6c0 2023-10-11T09:52:11.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.467+0000 7fee8dffb700 1 -- 172.21.15.78:0/3931746775 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee8400bbc0 con 0x7fee9006a6c0 2023-10-11T09:52:11.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.467+0000 7fee8dffb700 1 -- 172.21.15.78:0/3931746775 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee84009e60 con 0x7fee9006a6c0 2023-10-11T09:52:11.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.467+0000 7fee958fa700 1 -- 172.21.15.78:0/3931746775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee9006a6c0 msgr2=0x7fee900cea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:11.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.467+0000 7fee958fa700 1 --2- 172.21.15.78:0/3931746775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee9006a6c0 0x7fee900cea30 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fee84009910 tx=0x7fee84009c20 comp rx=0 tx=0).stop 2023-10-11T09:52:11.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.467+0000 7fee958fa700 1 -- 172.21.15.78:0/3931746775 shutdown_connections 2023-10-11T09:52:11.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.467+0000 7fee958fa700 1 --2- 172.21.15.78:0/3931746775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee9006a6c0 0x7fee900cea30 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:11.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.467+0000 7fee958fa700 1 --2- 172.21.15.78:0/3931746775 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fee9015a720 0x7fee9006a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:11.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.467+0000 7fee958fa700 1 -- 172.21.15.78:0/3931746775 >> 172.21.15.78:0/3931746775 conn(0x7fee9006c5a0 msgr2=0x7fee900d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:11.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.467+0000 7fee958fa700 1 -- 172.21.15.78:0/3931746775 shutdown_connections 2023-10-11T09:52:11.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.467+0000 7fee958fa700 1 -- 172.21.15.78:0/3931746775 wait complete. 2023-10-11T09:52:11.478 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.471+0000 7fee958fa700 1 Processor -- start 2023-10-11T09:52:11.479 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.471+0000 7fee958fa700 1 -- start start 2023-10-11T09:52:11.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.475+0000 7fee958fa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee9006a6c0 0x7fee900c6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:11.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.475+0000 7fee958fa700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fee900cef70 0x7fee901ef410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:11.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.475+0000 7fee958fa700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee9015a720 0x7fee900c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:11.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.475+0000 7fee958fa700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9014b360 con 0x7fee9006a6c0 2023-10-11T09:52:11.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.475+0000 7fee958fa700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee900c7280 con 0x7fee9015a720 2023-10-11T09:52:11.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.475+0000 7fee958fa700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee9015ade0 con 0x7fee900cef70 2023-10-11T09:52:11.482 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.475+0000 7fee8effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee9006a6c0 0x7fee900c6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:11.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.475+0000 7fee8effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee9006a6c0 0x7fee900c6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39170/0 (socket says 172.21.15.78:39170) 2023-10-11T09:52:11.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.475+0000 7fee8effd700 1 -- 172.21.15.78:0/555806963 learned_addr learned my addr 172.21.15.78:0/555806963 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:11.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.475+0000 7fee8effd700 1 -- 172.21.15.78:0/555806963 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fee900cef70 msgr2=0x7fee901ef410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:11.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.475+0000 7fee8effd700 1 --2- 172.21.15.78:0/555806963 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fee900cef70 0x7fee901ef410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:11.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.475+0000 7fee8ffff700 1 --2- 172.21.15.78:0/555806963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee9015a720 0x7fee900c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:11.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.475+0000 7fee8effd700 1 -- 172.21.15.78:0/555806963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee9015a720 msgr2=0x7fee900c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:11.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.475+0000 7fee8effd700 1 --2- 172.21.15.78:0/555806963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee9015a720 0x7fee900c98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:11.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.475+0000 7fee8effd700 1 -- 172.21.15.78:0/555806963 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee84009570 con 0x7fee9006a6c0 2023-10-11T09:52:11.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.475+0000 7fee8ffff700 1 --2- 172.21.15.78:0/555806963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee9015a720 0x7fee900c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:52:11.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.475+0000 7fee8effd700 1 --2- 172.21.15.78:0/555806963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee9006a6c0 0x7fee900c6d40 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fee8000d800 tx=0x7fee8000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:11.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.475+0000 7fee77fff700 1 -- 172.21.15.78:0/555806963 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee800125f0 con 0x7fee9006a6c0 2023-10-11T09:52:11.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.475+0000 7fee77fff700 1 -- 172.21.15.78:0/555806963 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee80012750 con 0x7fee9006a6c0 2023-10-11T09:52:11.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.475+0000 7fee958fa700 1 -- 172.21.15.78:0/555806963 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee9015b6f0 con 0x7fee9006a6c0 2023-10-11T09:52:11.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.475+0000 7fee77fff700 1 -- 172.21.15.78:0/555806963 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee80012ac0 con 0x7fee9006a6c0 2023-10-11T09:52:11.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.475+0000 7fee958fa700 1 -- 172.21.15.78:0/555806963 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee9015b030 con 0x7fee9006a6c0 2023-10-11T09:52:11.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.479+0000 7fee958fa700 1 -- 172.21.15.78:0/555806963 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee900744c0 con 0x7fee9006a6c0 2023-10-11T09:52:11.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.479+0000 7fee77fff700 1 -- 172.21.15.78:0/555806963 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fee80012c20 con 0x7fee9006a6c0 2023-10-11T09:52:11.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.479+0000 7fee77fff700 1 --2- 172.21.15.78:0/555806963 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fee7c065ed0 0x7fee7c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:11.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.479+0000 7fee77fff700 1 -- 172.21.15.78:0/555806963 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fee80083b30 con 0x7fee9006a6c0 2023-10-11T09:52:11.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.483+0000 7fee8e7fc700 1 --2- 172.21.15.78:0/555806963 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fee7c065ed0 0x7fee7c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:11.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.483+0000 7fee8e7fc700 1 --2- 172.21.15.78:0/555806963 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fee7c065ed0 0x7fee7c068380 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fee901f30a0 tx=0x7fee84010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:11.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.487+0000 7fee77fff700 1 -- 172.21.15.78:0/555806963 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fee800511b0 con 0x7fee9006a6c0 2023-10-11T09:52:11.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.663+0000 7fee958fa700 1 -- 172.21.15.78:0/555806963 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee9006b440 con 0x7fee9006a6c0 2023-10-11T09:52:11.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.663+0000 7fee77fff700 1 -- 172.21.15.78:0/555806963 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+18024 (secure 0 0 0) 0x7fee80012ed0 con 0x7fee9006a6c0 2023-10-11T09:52:11.671 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:52:11.671 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":128,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:52:02.877803+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:51:38.592892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-11T09:51:39.204588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6833/1433079498":"2023-10-12T09:52:02.877755+0000","172.21.15.133:6832/1433079498":"2023-10-12T09:52:02.877755+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:52:11.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.667+0000 7fee958fa700 1 -- 172.21.15.78:0/555806963 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fee7c065ed0 msgr2=0x7fee7c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:11.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.667+0000 7fee958fa700 1 --2- 172.21.15.78:0/555806963 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fee7c065ed0 0x7fee7c068380 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fee901f30a0 tx=0x7fee84010000 comp rx=0 tx=0).stop 2023-10-11T09:52:11.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.667+0000 7fee958fa700 1 -- 172.21.15.78:0/555806963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee9006a6c0 msgr2=0x7fee900c6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:11.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.667+0000 7fee958fa700 1 --2- 172.21.15.78:0/555806963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee9006a6c0 0x7fee900c6d40 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fee8000d800 tx=0x7fee8000dbc0 comp rx=0 tx=0).stop 2023-10-11T09:52:11.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.667+0000 7fee958fa700 1 -- 172.21.15.78:0/555806963 shutdown_connections 2023-10-11T09:52:11.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.667+0000 7fee958fa700 1 --2- 172.21.15.78:0/555806963 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fee7c065ed0 0x7fee7c068380 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:11.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.667+0000 7fee958fa700 1 --2- 172.21.15.78:0/555806963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee9015a720 0x7fee900c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:11.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.667+0000 7fee958fa700 1 --2- 172.21.15.78:0/555806963 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fee900cef70 0x7fee901ef410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:11.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.667+0000 7fee958fa700 1 --2- 172.21.15.78:0/555806963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee9006a6c0 0x7fee900c6d40 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:11.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.667+0000 7fee958fa700 1 -- 172.21.15.78:0/555806963 >> 172.21.15.78:0/555806963 conn(0x7fee9006c5a0 msgr2=0x7fee901552b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:11.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.667+0000 7fee958fa700 1 -- 172.21.15.78:0/555806963 shutdown_connections 2023-10-11T09:52:11.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.667+0000 7fee958fa700 1 -- 172.21.15.78:0/555806963 wait complete. 2023-10-11T09:52:11.697 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T09:51:38.592892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T09:51:39.204588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '127', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:52:11.698 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:52:11.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.867+0000 7fca73344700 1 Processor -- start 2023-10-11T09:52:11.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.867+0000 7fca73344700 1 -- start start 2023-10-11T09:52:11.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.867+0000 7fca73344700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fca6c0d0360 0x7fca6c0b2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:11.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.867+0000 7fca73344700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca6c0b21b0 0x7fca6c0d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:11.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.867+0000 7fca73344700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca6c161640 con 0x7fca6c15cd50 2023-10-11T09:52:11.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.867+0000 7fca73344700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca6c160750 con 0x7fca6c0d0360 2023-10-11T09:52:11.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.867+0000 7fca73344700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca6c160a30 con 0x7fca6c0b21b0 2023-10-11T09:52:11.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.871+0000 7fca708df700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fca6c0d0360 0x7fca6c0b2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:11.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.871+0000 7fca708df700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fca6c0d0360 0x7fca6c0b2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39184/0 (socket says 172.21.15.78:39184) 2023-10-11T09:52:11.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.871+0000 7fca708df700 1 -- 172.21.15.78:0/2217457373 learned_addr learned my addr 172.21.15.78:0/2217457373 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:11.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.871+0000 7fca718e1700 1 --2- 172.21.15.78:0/2217457373 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca6c0b21b0 0x7fca6c0d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:11.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.871+0000 7fca708df700 1 -- 172.21.15.78:0/2217457373 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca6c0b21b0 msgr2=0x7fca6c0d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:11.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.871+0000 7fca708df700 1 --2- 172.21.15.78:0/2217457373 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca6c0b21b0 0x7fca6c0d10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:11.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.871+0000 7fca708df700 1 -- 172.21.15.78:0/2217457373 >> v1:172.21.15.133:6789/0 conn(0x7fca6c15cd50 legacy=0x7fca6c0cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:11.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.871+0000 7fca708df700 1 -- 172.21.15.78:0/2217457373 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca6c160d10 con 0x7fca6c0d0360 2023-10-11T09:52:11.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.871+0000 7fca708df700 1 --2- 172.21.15.78:0/2217457373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca6c0d0360 0x7fca6c0b2700 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fca5c009910 tx=0x7fca5c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=169d430a82c41962 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:11.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.871+0000 7fca63fff700 1 -- 172.21.15.78:0/2217457373 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5c00ba60 con 0x7fca6c0d0360 2023-10-11T09:52:11.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.871+0000 7fca63fff700 1 -- 172.21.15.78:0/2217457373 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca5c00bbc0 con 0x7fca6c0d0360 2023-10-11T09:52:11.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.871+0000 7fca63fff700 1 -- 172.21.15.78:0/2217457373 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5c009e60 con 0x7fca6c0d0360 2023-10-11T09:52:11.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.871+0000 7fca73344700 1 -- 172.21.15.78:0/2217457373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca6c0d0360 msgr2=0x7fca6c0b2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:11.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.871+0000 7fca73344700 1 --2- 172.21.15.78:0/2217457373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca6c0d0360 0x7fca6c0b2700 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fca5c009910 tx=0x7fca5c009c20 comp rx=0 tx=0).stop 2023-10-11T09:52:11.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.871+0000 7fca73344700 1 -- 172.21.15.78:0/2217457373 shutdown_connections 2023-10-11T09:52:11.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.871+0000 7fca73344700 1 --2- 172.21.15.78:0/2217457373 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca6c0b21b0 0x7fca6c0d10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:11.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.871+0000 7fca73344700 1 --2- 172.21.15.78:0/2217457373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca6c0d0360 0x7fca6c0b2700 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:11.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.871+0000 7fca73344700 1 -- 172.21.15.78:0/2217457373 >> 172.21.15.78:0/2217457373 conn(0x7fca6c0718b0 msgr2=0x7fca6c071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:11.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.871+0000 7fca73344700 1 -- 172.21.15.78:0/2217457373 shutdown_connections 2023-10-11T09:52:11.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.871+0000 7fca73344700 1 -- 172.21.15.78:0/2217457373 wait complete. 2023-10-11T09:52:11.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.875+0000 7fca73344700 1 Processor -- start 2023-10-11T09:52:11.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.875+0000 7fca73344700 1 -- start start 2023-10-11T09:52:11.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.879+0000 7fca73344700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca6c0b21b0 0x7fca6c15f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:11.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.879+0000 7fca73344700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca6c0d0360 0x7fca6c0ce280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:11.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.879+0000 7fca73344700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca6c15cd50 0x7fca6c0b8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:11.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.879+0000 7fca73344700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca6c1f6670 con 0x7fca6c15cd50 2023-10-11T09:52:11.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.879+0000 7fca73344700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca6c1f57c0 con 0x7fca6c0d0360 2023-10-11T09:52:11.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.879+0000 7fca73344700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca6c15fc80 con 0x7fca6c0b21b0 2023-10-11T09:52:11.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.879+0000 7fca718e1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca6c15cd50 0x7fca6c0b8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:11.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.879+0000 7fca718e1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca6c15cd50 0x7fca6c0b8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39194/0 (socket says 172.21.15.78:39194) 2023-10-11T09:52:11.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.879+0000 7fca718e1700 1 -- 172.21.15.78:0/1346303897 learned_addr learned my addr 172.21.15.78:0/1346303897 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:11.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.879+0000 7fca708df700 1 --2- 172.21.15.78:0/1346303897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca6c0d0360 0x7fca6c0ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:11.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.879+0000 7fca710e0700 1 --2- 172.21.15.78:0/1346303897 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca6c0b21b0 0x7fca6c15f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:11.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.879+0000 7fca718e1700 1 -- 172.21.15.78:0/1346303897 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca6c0b21b0 msgr2=0x7fca6c15f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:11.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.879+0000 7fca718e1700 1 --2- 172.21.15.78:0/1346303897 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca6c0b21b0 0x7fca6c15f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:11.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.879+0000 7fca718e1700 1 -- 172.21.15.78:0/1346303897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca6c0d0360 msgr2=0x7fca6c0ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:11.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.879+0000 7fca718e1700 1 --2- 172.21.15.78:0/1346303897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca6c0d0360 0x7fca6c0ce280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:11.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.879+0000 7fca718e1700 1 -- 172.21.15.78:0/1346303897 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca5c009570 con 0x7fca6c15cd50 2023-10-11T09:52:11.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.879+0000 7fca718e1700 1 --2- 172.21.15.78:0/1346303897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca6c15cd50 0x7fca6c0b8410 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fca64007e40 tx=0x7fca6400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:11.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.879+0000 7fca61ffb700 1 -- 172.21.15.78:0/1346303897 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6400eea0 con 0x7fca6c15cd50 2023-10-11T09:52:11.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.879+0000 7fca61ffb700 1 -- 172.21.15.78:0/1346303897 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca64014420 con 0x7fca6c15cd50 2023-10-11T09:52:11.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.879+0000 7fca73344700 1 -- 172.21.15.78:0/1346303897 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca6c1f5a60 con 0x7fca6c15cd50 2023-10-11T09:52:11.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.879+0000 7fca61ffb700 1 -- 172.21.15.78:0/1346303897 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca640147b0 con 0x7fca6c15cd50 2023-10-11T09:52:11.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.879+0000 7fca73344700 1 -- 172.21.15.78:0/1346303897 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca6c1f6020 con 0x7fca6c15cd50 2023-10-11T09:52:11.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.883+0000 7fca73344700 1 -- 172.21.15.78:0/1346303897 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca6c08c200 con 0x7fca6c15cd50 2023-10-11T09:52:11.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.883+0000 7fca61ffb700 1 -- 172.21.15.78:0/1346303897 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fca64014910 con 0x7fca6c15cd50 2023-10-11T09:52:11.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.883+0000 7fca61ffb700 1 --2- 172.21.15.78:0/1346303897 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fca50065ed0 0x7fca50068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:11.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.883+0000 7fca61ffb700 1 -- 172.21.15.78:0/1346303897 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fca640841b0 con 0x7fca6c15cd50 2023-10-11T09:52:11.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.887+0000 7fca710e0700 1 --2- 172.21.15.78:0/1346303897 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fca50065ed0 0x7fca50068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:11.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.887+0000 7fca710e0700 1 --2- 172.21.15.78:0/1346303897 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fca50065ed0 0x7fca50068380 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fca6c15e980 tx=0x7fca68009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:11.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:11.891+0000 7fca61ffb700 1 -- 172.21.15.78:0/1346303897 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fca64051680 con 0x7fca6c15cd50 2023-10-11T09:52:12.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.067+0000 7fca73344700 1 -- 172.21.15.78:0/1346303897 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fca6c1604c0 con 0x7fca6c15cd50 2023-10-11T09:52:12.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.067+0000 7fca61ffb700 1 -- 172.21.15.78:0/1346303897 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v128) v1 ==== 93+0+10 (secure 0 0 0) 0x7fca64054ca0 con 0x7fca6c15cd50 2023-10-11T09:52:12.077 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:52:12.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.071+0000 7fca73344700 1 -- 172.21.15.78:0/1346303897 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fca50065ed0 msgr2=0x7fca50068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:12.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.071+0000 7fca73344700 1 --2- 172.21.15.78:0/1346303897 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fca50065ed0 0x7fca50068380 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fca6c15e980 tx=0x7fca68009040 comp rx=0 tx=0).stop 2023-10-11T09:52:12.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.071+0000 7fca73344700 1 -- 172.21.15.78:0/1346303897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca6c15cd50 msgr2=0x7fca6c0b8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:12.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.071+0000 7fca73344700 1 --2- 172.21.15.78:0/1346303897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca6c15cd50 0x7fca6c0b8410 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fca64007e40 tx=0x7fca6400e3c0 comp rx=0 tx=0).stop 2023-10-11T09:52:12.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.071+0000 7fca73344700 1 -- 172.21.15.78:0/1346303897 shutdown_connections 2023-10-11T09:52:12.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.071+0000 7fca73344700 1 --2- 172.21.15.78:0/1346303897 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fca50065ed0 0x7fca50068380 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:12.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.071+0000 7fca73344700 1 --2- 172.21.15.78:0/1346303897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca6c15cd50 0x7fca6c0b8410 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:12.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.071+0000 7fca73344700 1 --2- 172.21.15.78:0/1346303897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca6c0d0360 0x7fca6c0ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:12.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.071+0000 7fca73344700 1 --2- 172.21.15.78:0/1346303897 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca6c0b21b0 0x7fca6c15f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:12.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.071+0000 7fca73344700 1 -- 172.21.15.78:0/1346303897 >> 172.21.15.78:0/1346303897 conn(0x7fca6c0718b0 msgr2=0x7fca6c15bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:12.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.071+0000 7fca73344700 1 -- 172.21.15.78:0/1346303897 shutdown_connections 2023-10-11T09:52:12.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.071+0000 7fca73344700 1 -- 172.21.15.78:0/1346303897 wait complete. 2023-10-11T09:52:12.100 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:52:12.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.263+0000 7f22a65a8700 1 Processor -- start 2023-10-11T09:52:12.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.263+0000 7f22a65a8700 1 -- start start 2023-10-11T09:52:12.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.263+0000 7f22a65a8700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f22a00b4530 0x7f22a00d0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:12.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.267+0000 7f22a65a8700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22a009c960 con 0x7f22a00ce740 2023-10-11T09:52:12.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.267+0000 7f22a65a8700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22a015d180 con 0x7f22a00d1520 2023-10-11T09:52:12.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.267+0000 7f22a65a8700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22a00d19a0 con 0x7f22a00b4530 2023-10-11T09:52:12.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.267+0000 7f22a5da7700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f22a00ce740 0x7f22a00b6bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:60506/0 (socket says 172.21.15.78:60506) 2023-10-11T09:52:12.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.267+0000 7f22a5da7700 1 -- 172.21.15.78:0/32120800 learned_addr learned my addr 172.21.15.78:0/32120800 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:12.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.267+0000 7f22a55a6700 1 --2- 172.21.15.78:0/32120800 >> v2:172.21.15.133:3300/0 conn(0x7f22a00b4530 0x7f22a00d0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:12.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.267+0000 7f2297fff700 1 -- 172.21.15.78:0/32120800 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1362114309 0 0) 0x7f22a009c960 con 0x7f22a00ce740 2023-10-11T09:52:12.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.267+0000 7f2297fff700 1 -- 172.21.15.78:0/32120800 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f228c0035d0 con 0x7f22a00ce740 2023-10-11T09:52:12.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.267+0000 7f2297fff700 1 -- 172.21.15.78:0/32120800 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2956665840 0 0) 0x7f22a015d180 con 0x7f22a00d1520 2023-10-11T09:52:12.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.267+0000 7f2297fff700 1 -- 172.21.15.78:0/32120800 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22a009c960 con 0x7f22a00d1520 2023-10-11T09:52:12.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.267+0000 7f22a55a6700 1 -- 172.21.15.78:0/32120800 >> v1:172.21.15.133:6789/0 conn(0x7f22a00d1520 legacy=0x7f22a00b64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:12.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.267+0000 7f22a55a6700 1 -- 172.21.15.78:0/32120800 >> v1:172.21.15.78:6789/0 conn(0x7f22a00ce740 legacy=0x7f22a00b6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:12.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.267+0000 7f2297fff700 1 -- 172.21.15.78:0/32120800 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 585422784 0 0) 0x7f228c0035d0 con 0x7f22a00ce740 2023-10-11T09:52:12.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.267+0000 7f22a55a6700 1 -- 172.21.15.78:0/32120800 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a00b5fd0 con 0x7f22a00b4530 2023-10-11T09:52:12.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.267+0000 7f22a55a6700 1 --2- 172.21.15.78:0/32120800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22a00b4530 0x7f22a00d0fe0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f229c009840 tx=0x7f229c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7eb5b5991c75d517 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:12.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.267+0000 7f2297fff700 1 -- 172.21.15.78:0/32120800 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22980019f0 con 0x7f22a00b4530 2023-10-11T09:52:12.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.267+0000 7f2297fff700 1 -- 172.21.15.78:0/32120800 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f229c004030 con 0x7f22a00b4530 2023-10-11T09:52:12.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.267+0000 7f2297fff700 1 -- 172.21.15.78:0/32120800 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c0043c0 con 0x7f22a00b4530 2023-10-11T09:52:12.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.271+0000 7f22a65a8700 1 -- 172.21.15.78:0/32120800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22a00b4530 msgr2=0x7f22a00d0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:12.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.271+0000 7f22a65a8700 1 --2- 172.21.15.78:0/32120800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22a00b4530 0x7f22a00d0fe0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f229c009840 tx=0x7f229c009b50 comp rx=0 tx=0).stop 2023-10-11T09:52:12.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.271+0000 7f22a65a8700 1 -- 172.21.15.78:0/32120800 shutdown_connections 2023-10-11T09:52:12.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.271+0000 7f22a65a8700 1 --2- 172.21.15.78:0/32120800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22a00b4530 0x7f22a00d0fe0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:12.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.271+0000 7f22a65a8700 1 -- 172.21.15.78:0/32120800 >> 172.21.15.78:0/32120800 conn(0x7f22a009c3d0 msgr2=0x7f22a00d30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:12.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.271+0000 7f22a65a8700 1 -- 172.21.15.78:0/32120800 shutdown_connections 2023-10-11T09:52:12.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.271+0000 7f22a65a8700 1 -- 172.21.15.78:0/32120800 wait complete. 2023-10-11T09:52:12.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.271+0000 7f22a65a8700 1 Processor -- start 2023-10-11T09:52:12.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.271+0000 7f22a65a8700 1 -- start start 2023-10-11T09:52:12.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.271+0000 7f22a65a8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22a00b4530 0x7f22a01538a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:12.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.271+0000 7f22a65a8700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22a00ce740 0x7f22a015e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:12.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.271+0000 7f22a65a8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22a00d1520 0x7f22a01522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:12.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.271+0000 7f22a65a8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a00c8500 con 0x7f22a00b4530 2023-10-11T09:52:12.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.271+0000 7f22a65a8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a0154990 con 0x7f22a00d1520 2023-10-11T09:52:12.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.271+0000 7f22a65a8700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a01540f0 con 0x7f22a00ce740 2023-10-11T09:52:12.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.275+0000 7f22a55a6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22a00b4530 0x7f22a01538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:12.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.275+0000 7f22a55a6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22a00b4530 0x7f22a01538a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39208/0 (socket says 172.21.15.78:39208) 2023-10-11T09:52:12.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.275+0000 7f22a55a6700 1 -- 172.21.15.78:0/4031060900 learned_addr learned my addr 172.21.15.78:0/4031060900 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:12.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.275+0000 7f22a5da7700 1 --2- 172.21.15.78:0/4031060900 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22a00d1520 0x7f22a01522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:12.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.275+0000 7f22a4da5700 1 --2- 172.21.15.78:0/4031060900 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22a00ce740 0x7f22a015e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:12.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.275+0000 7f22a55a6700 1 -- 172.21.15.78:0/4031060900 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22a00ce740 msgr2=0x7f22a015e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:12.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.275+0000 7f22a55a6700 1 --2- 172.21.15.78:0/4031060900 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22a00ce740 0x7f22a015e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:12.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.275+0000 7f22a55a6700 1 -- 172.21.15.78:0/4031060900 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22a00d1520 msgr2=0x7f22a01522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:12.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.275+0000 7f22a55a6700 1 --2- 172.21.15.78:0/4031060900 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22a00d1520 0x7f22a01522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:12.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.275+0000 7f22a55a6700 1 -- 172.21.15.78:0/4031060900 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f229c009450 con 0x7f22a00b4530 2023-10-11T09:52:12.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.275+0000 7f22a55a6700 1 --2- 172.21.15.78:0/4031060900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22a00b4530 0x7f22a01538a0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f22a00b5cd0 tx=0x7f229c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:12.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.275+0000 7f2295ffb700 1 -- 172.21.15.78:0/4031060900 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c006e20 con 0x7f22a00b4530 2023-10-11T09:52:12.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.275+0000 7f2295ffb700 1 -- 172.21.15.78:0/4031060900 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f229c020070 con 0x7f22a00b4530 2023-10-11T09:52:12.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.275+0000 7f22a65a8700 1 -- 172.21.15.78:0/4031060900 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a01543d0 con 0x7f22a00b4530 2023-10-11T09:52:12.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.275+0000 7f2295ffb700 1 -- 172.21.15.78:0/4031060900 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c011870 con 0x7f22a00b4530 2023-10-11T09:52:12.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.275+0000 7f22a65a8700 1 -- 172.21.15.78:0/4031060900 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a0154760 con 0x7f22a00b4530 2023-10-11T09:52:12.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.275+0000 7f22a65a8700 1 -- 172.21.15.78:0/4031060900 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22a0153ea0 con 0x7f22a00b4530 2023-10-11T09:52:12.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.279+0000 7f2295ffb700 1 -- 172.21.15.78:0/4031060900 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f229c022030 con 0x7f22a00b4530 2023-10-11T09:52:12.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.279+0000 7f2295ffb700 1 --2- 172.21.15.78:0/4031060900 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2284065ed0 0x7f2284068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:12.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.279+0000 7f2295ffb700 1 -- 172.21.15.78:0/4031060900 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f229c084ff0 con 0x7f22a00b4530 2023-10-11T09:52:12.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.279+0000 7f22a4da5700 1 --2- 172.21.15.78:0/4031060900 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2284065ed0 0x7f2284068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:12.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.283+0000 7f22a4da5700 1 --2- 172.21.15.78:0/4031060900 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2284065ed0 0x7f2284068380 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f2290007fd0 tx=0x7f2290009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:12.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.287+0000 7f2295ffb700 1 -- 172.21.15.78:0/4031060900 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f229c0b5fa0 con 0x7f22a00b4530 2023-10-11T09:52:12.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.463+0000 7f22a65a8700 1 -- 172.21.15.78:0/4031060900 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f22a00d0100 con 0x7f22a00b4530 2023-10-11T09:52:12.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.463+0000 7f2295ffb700 1 -- 172.21.15.78:0/4031060900 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v128) v1 ==== 92+0+10 (secure 0 0 0) 0x7f229c052730 con 0x7f22a00b4530 2023-10-11T09:52:12.471 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:52:12.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.467+0000 7f22a65a8700 1 -- 172.21.15.78:0/4031060900 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2284065ed0 msgr2=0x7f2284068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:12.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.467+0000 7f22a65a8700 1 --2- 172.21.15.78:0/4031060900 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2284065ed0 0x7f2284068380 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f2290007fd0 tx=0x7f2290009040 comp rx=0 tx=0).stop 2023-10-11T09:52:12.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.467+0000 7f22a65a8700 1 -- 172.21.15.78:0/4031060900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22a00b4530 msgr2=0x7f22a01538a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:12.474 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.467+0000 7f22a65a8700 1 --2- 172.21.15.78:0/4031060900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22a00b4530 0x7f22a01538a0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f22a00b5cd0 tx=0x7f229c004770 comp rx=0 tx=0).stop 2023-10-11T09:52:12.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.467+0000 7f22a65a8700 1 -- 172.21.15.78:0/4031060900 shutdown_connections 2023-10-11T09:52:12.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.467+0000 7f22a65a8700 1 --2- 172.21.15.78:0/4031060900 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2284065ed0 0x7f2284068380 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:12.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.467+0000 7f22a65a8700 1 --2- 172.21.15.78:0/4031060900 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22a00d1520 0x7f22a01522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:12.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.467+0000 7f22a65a8700 1 --2- 172.21.15.78:0/4031060900 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22a00ce740 0x7f22a015e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:12.475 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.467+0000 7f22a65a8700 1 --2- 172.21.15.78:0/4031060900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22a00b4530 0x7f22a01538a0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:12.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.467+0000 7f22a65a8700 1 -- 172.21.15.78:0/4031060900 >> 172.21.15.78:0/4031060900 conn(0x7f22a009c3d0 msgr2=0x7f22a00b6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:12.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.467+0000 7f22a65a8700 1 -- 172.21.15.78:0/4031060900 shutdown_connections 2023-10-11T09:52:12.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.467+0000 7f22a65a8700 1 -- 172.21.15.78:0/4031060900 wait complete. 2023-10-11T09:52:12.493 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T09:52:12.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.663+0000 7ffbe5a43700 1 Processor -- start 2023-10-11T09:52:12.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.663+0000 7ffbe5a43700 1 -- start start 2023-10-11T09:52:12.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.663+0000 7ffbe5a43700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ffbe00ce700 0x7ffbe00c20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:12.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.663+0000 7ffbe5a43700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffbe0071720 0x7ffbe006bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:12.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.663+0000 7ffbe5a43700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbe00cb220 con 0x7ffbe0161020 2023-10-11T09:52:12.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.663+0000 7ffbe5a43700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbe0071c70 con 0x7ffbe00ce700 2023-10-11T09:52:12.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.663+0000 7ffbe5a43700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe01613f0 con 0x7ffbe0071720 2023-10-11T09:52:12.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.663+0000 7ffbdeffd700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7ffbe0161020 0x7ffbe00cdff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:60520/0 (socket says 172.21.15.78:60520) 2023-10-11T09:52:12.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.663+0000 7ffbdeffd700 1 -- 172.21.15.78:0/616868634 learned_addr learned my addr 172.21.15.78:0/616868634 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:12.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.663+0000 7ffbd7fff700 1 --2- 172.21.15.78:0/616868634 >> v2:172.21.15.78:3300/0 conn(0x7ffbe00ce700 0x7ffbe00c20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:12.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.663+0000 7ffbdf7fe700 1 --2- 172.21.15.78:0/616868634 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffbe0071720 0x7ffbe006bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:12.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.667+0000 7ffbd7fff700 1 -- 172.21.15.78:0/616868634 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffbe0071720 msgr2=0x7ffbe006bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:12.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.667+0000 7ffbd7fff700 1 --2- 172.21.15.78:0/616868634 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffbe0071720 0x7ffbe006bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:12.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.667+0000 7ffbd7fff700 1 -- 172.21.15.78:0/616868634 >> v1:172.21.15.78:6789/0 conn(0x7ffbe0161020 legacy=0x7ffbe00cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:12.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.663+0000 7ffbde7fc700 1 -- 172.21.15.78:0/616868634 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3057234411 0 0) 0x7ffbe00cb220 con 0x7ffbe0161020 2023-10-11T09:52:12.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.667+0000 7ffbd7fff700 1 -- 172.21.15.78:0/616868634 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe00ca650 con 0x7ffbe00ce700 2023-10-11T09:52:12.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.667+0000 7ffbdf7fe700 1 --2- 172.21.15.78:0/616868634 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffbe0071720 0x7ffbe006bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:52:12.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.667+0000 7ffbd7fff700 1 --2- 172.21.15.78:0/616868634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbe00ce700 0x7ffbe00c20c0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8009840 tx=0x7ffbc8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d74f0a342f153e05 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:12.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.667+0000 7ffbde7fc700 1 -- 172.21.15.78:0/616868634 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8005e50 con 0x7ffbe00ce700 2023-10-11T09:52:12.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.667+0000 7ffbde7fc700 1 -- 172.21.15.78:0/616868634 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc8006030 con 0x7ffbe00ce700 2023-10-11T09:52:12.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.667+0000 7ffbde7fc700 1 -- 172.21.15.78:0/616868634 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc80063c0 con 0x7ffbe00ce700 2023-10-11T09:52:12.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.667+0000 7ffbe5a43700 1 -- 172.21.15.78:0/616868634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbe00ce700 msgr2=0x7ffbe00c20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:12.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.667+0000 7ffbe5a43700 1 --2- 172.21.15.78:0/616868634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbe00ce700 0x7ffbe00c20c0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8009840 tx=0x7ffbc8009b50 comp rx=0 tx=0).stop 2023-10-11T09:52:12.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.667+0000 7ffbe5a43700 1 -- 172.21.15.78:0/616868634 shutdown_connections 2023-10-11T09:52:12.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.667+0000 7ffbe5a43700 1 --2- 172.21.15.78:0/616868634 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffbe0071720 0x7ffbe006bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:12.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.667+0000 7ffbe5a43700 1 --2- 172.21.15.78:0/616868634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbe00ce700 0x7ffbe00c20c0 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:12.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.667+0000 7ffbe5a43700 1 -- 172.21.15.78:0/616868634 >> 172.21.15.78:0/616868634 conn(0x7ffbe00cf820 msgr2=0x7ffbe00cb610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:12.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.667+0000 7ffbe5a43700 1 -- 172.21.15.78:0/616868634 shutdown_connections 2023-10-11T09:52:12.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.667+0000 7ffbe5a43700 1 -- 172.21.15.78:0/616868634 wait complete. 2023-10-11T09:52:12.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.671+0000 7ffbe5a43700 1 Processor -- start 2023-10-11T09:52:12.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.671+0000 7ffbe5a43700 1 -- start start 2023-10-11T09:52:12.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.675+0000 7ffbe5a43700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbe0071720 0x7ffbe015e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:12.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.675+0000 7ffbe5a43700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbe00ce700 0x7ffbe006bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:12.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.675+0000 7ffbe5a43700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffbe0161020 0x7ffbe006c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:12.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.675+0000 7ffbe5a43700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe00c2630 con 0x7ffbe0071720 2023-10-11T09:52:12.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.675+0000 7ffbe5a43700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe00c2880 con 0x7ffbe00ce700 2023-10-11T09:52:12.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.675+0000 7ffbe5a43700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe00c2b60 con 0x7ffbe0161020 2023-10-11T09:52:12.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.675+0000 7ffbdeffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbe0071720 0x7ffbe015e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:12.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.675+0000 7ffbdeffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbe0071720 0x7ffbe015e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39226/0 (socket says 172.21.15.78:39226) 2023-10-11T09:52:12.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.675+0000 7ffbdeffd700 1 -- 172.21.15.78:0/3113059493 learned_addr learned my addr 172.21.15.78:0/3113059493 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:12.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.675+0000 7ffbdeffd700 1 -- 172.21.15.78:0/3113059493 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffbe0161020 msgr2=0x7ffbe006c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:12.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.675+0000 7ffbdeffd700 1 --2- 172.21.15.78:0/3113059493 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffbe0161020 0x7ffbe006c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:12.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.675+0000 7ffbdeffd700 1 -- 172.21.15.78:0/3113059493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbe00ce700 msgr2=0x7ffbe006bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:12.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.675+0000 7ffbdeffd700 1 --2- 172.21.15.78:0/3113059493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbe00ce700 0x7ffbe006bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:12.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.675+0000 7ffbdeffd700 1 -- 172.21.15.78:0/3113059493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc8009450 con 0x7ffbe0071720 2023-10-11T09:52:12.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.675+0000 7ffbdeffd700 1 --2- 172.21.15.78:0/3113059493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbe0071720 0x7ffbe015e540 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7ffbd000c580 tx=0x7ffbd000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:12.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.675+0000 7ffbdcff9700 1 -- 172.21.15.78:0/3113059493 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd0004ca0 con 0x7ffbe0071720 2023-10-11T09:52:12.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.675+0000 7ffbdcff9700 1 -- 172.21.15.78:0/3113059493 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbd0004e20 con 0x7ffbe0071720 2023-10-11T09:52:12.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.675+0000 7ffbdcff9700 1 -- 172.21.15.78:0/3113059493 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd0007310 con 0x7ffbe0071720 2023-10-11T09:52:12.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.675+0000 7ffbe5a43700 1 -- 172.21.15.78:0/3113059493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbe00c2e40 con 0x7ffbe0071720 2023-10-11T09:52:12.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.675+0000 7ffbe5a43700 1 -- 172.21.15.78:0/3113059493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbe00c3400 con 0x7ffbe0071720 2023-10-11T09:52:12.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.675+0000 7ffbe5a43700 1 -- 172.21.15.78:0/3113059493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbe0067d00 con 0x7ffbe0071720 2023-10-11T09:52:12.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.679+0000 7ffbdcff9700 1 -- 172.21.15.78:0/3113059493 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7ffbd0007470 con 0x7ffbe0071720 2023-10-11T09:52:12.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.679+0000 7ffbdcff9700 1 --2- 172.21.15.78:0/3113059493 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffbc0065ed0 0x7ffbc0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:12.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.679+0000 7ffbdcff9700 1 -- 172.21.15.78:0/3113059493 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6148+0+0 (secure 0 0 0) 0x7ffbd00835f0 con 0x7ffbe0071720 2023-10-11T09:52:12.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.679+0000 7ffbd7fff700 1 --2- 172.21.15.78:0/3113059493 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffbc0065ed0 0x7ffbc0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:12.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.683+0000 7ffbd7fff700 1 --2- 172.21.15.78:0/3113059493 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffbc0065ed0 0x7ffbc0068380 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8009840 tx=0x7ffbc8010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:12.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.683+0000 7ffbdcff9700 1 -- 172.21.15.78:0/3113059493 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7ffbd0050bd0 con 0x7ffbe0071720 2023-10-11T09:52:12.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.859+0000 7ffbe5a43700 1 -- 172.21.15.78:0/3113059493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffbe00b19c0 con 0x7ffbe0071720 2023-10-11T09:52:12.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.859+0000 7ffbdcff9700 1 -- 172.21.15.78:0/3113059493 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v128) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffbd0007720 con 0x7ffbe0071720 2023-10-11T09:52:12.868 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:52:12.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.863+0000 7ffbe5a43700 1 -- 172.21.15.78:0/3113059493 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffbc0065ed0 msgr2=0x7ffbc0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:12.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.863+0000 7ffbe5a43700 1 --2- 172.21.15.78:0/3113059493 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffbc0065ed0 0x7ffbc0068380 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8009840 tx=0x7ffbc8010000 comp rx=0 tx=0).stop 2023-10-11T09:52:12.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.863+0000 7ffbe5a43700 1 -- 172.21.15.78:0/3113059493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbe0071720 msgr2=0x7ffbe015e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:12.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.863+0000 7ffbe5a43700 1 --2- 172.21.15.78:0/3113059493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbe0071720 0x7ffbe015e540 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7ffbd000c580 tx=0x7ffbd000c940 comp rx=0 tx=0).stop 2023-10-11T09:52:12.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.863+0000 7ffbe5a43700 1 -- 172.21.15.78:0/3113059493 shutdown_connections 2023-10-11T09:52:12.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.863+0000 7ffbe5a43700 1 --2- 172.21.15.78:0/3113059493 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ffbc0065ed0 0x7ffbc0068380 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:12.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.863+0000 7ffbe5a43700 1 --2- 172.21.15.78:0/3113059493 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffbe0161020 0x7ffbe006c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:12.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.863+0000 7ffbe5a43700 1 --2- 172.21.15.78:0/3113059493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbe00ce700 0x7ffbe006bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:12.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.863+0000 7ffbe5a43700 1 --2- 172.21.15.78:0/3113059493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbe0071720 0x7ffbe015e540 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:12.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.863+0000 7ffbe5a43700 1 -- 172.21.15.78:0/3113059493 >> 172.21.15.78:0/3113059493 conn(0x7ffbe00cf820 msgr2=0x7ffbe00ae5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:12.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.863+0000 7ffbe5a43700 1 -- 172.21.15.78:0/3113059493 shutdown_connections 2023-10-11T09:52:12.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:12.863+0000 7ffbe5a43700 1 -- 172.21.15.78:0/3113059493 wait complete. 2023-10-11T09:52:12.892 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T09:52:13.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.071+0000 7fbd97b44700 1 Processor -- start 2023-10-11T09:52:13.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.071+0000 7fbd97b44700 1 -- start start 2023-10-11T09:52:13.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.071+0000 7fbd97b44700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbd90161010 0x7fbd9006d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:13.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.071+0000 7fbd97b44700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd900cc2c0 con 0x7fbd900c23c0 2023-10-11T09:52:13.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.071+0000 7fbd97b44700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd90071840 con 0x7fbd900cea30 2023-10-11T09:52:13.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.071+0000 7fbd97b44700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd90072050 con 0x7fbd90161010 2023-10-11T09:52:13.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.075+0000 7fbd950df700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fbd900c23c0 0x7fbd900ce320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:60532/0 (socket says 172.21.15.78:60532) 2023-10-11T09:52:13.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.075+0000 7fbd950df700 1 -- 172.21.15.78:0/1625966735 learned_addr learned my addr 172.21.15.78:0/1625966735 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:13.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.075+0000 7fbd958e0700 1 --2- 172.21.15.78:0/1625966735 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbd90161010 0x7fbd9006d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:13.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.075+0000 7fbd948de700 1 -- 172.21.15.78:0/1625966735 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2406761721 0 0) 0x7fbd90071840 con 0x7fbd900cea30 2023-10-11T09:52:13.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.075+0000 7fbd948de700 1 -- 172.21.15.78:0/1625966735 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd7c0035d0 con 0x7fbd900cea30 2023-10-11T09:52:13.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.075+0000 7fbd948de700 1 -- 172.21.15.78:0/1625966735 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1785306803 0 0) 0x7fbd900cc2c0 con 0x7fbd900c23c0 2023-10-11T09:52:13.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.075+0000 7fbd948de700 1 -- 172.21.15.78:0/1625966735 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd90071840 con 0x7fbd900c23c0 2023-10-11T09:52:13.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.075+0000 7fbd958e0700 1 -- 172.21.15.78:0/1625966735 >> v1:172.21.15.133:6789/0 conn(0x7fbd900cea30 legacy=0x7fbd90164c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:13.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.075+0000 7fbd958e0700 1 -- 172.21.15.78:0/1625966735 >> v1:172.21.15.78:6789/0 conn(0x7fbd900c23c0 legacy=0x7fbd900ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:13.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.075+0000 7fbd958e0700 1 -- 172.21.15.78:0/1625966735 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd900caa30 con 0x7fbd90161010 2023-10-11T09:52:13.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.075+0000 7fbd958e0700 1 --2- 172.21.15.78:0/1625966735 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbd90161010 0x7fbd9006d560 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c009880 tx=0x7fbd8c009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4346ff16c70f8b9c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:13.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.075+0000 7fbd948de700 1 -- 172.21.15.78:0/1625966735 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c011830 con 0x7fbd90161010 2023-10-11T09:52:13.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.075+0000 7fbd948de700 1 -- 172.21.15.78:0/1625966735 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd8c011990 con 0x7fbd90161010 2023-10-11T09:52:13.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.075+0000 7fbd948de700 1 -- 172.21.15.78:0/1625966735 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c011d20 con 0x7fbd90161010 2023-10-11T09:52:13.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.079+0000 7fbd97b44700 1 -- 172.21.15.78:0/1625966735 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbd90161010 msgr2=0x7fbd9006d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:13.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.079+0000 7fbd97b44700 1 --2- 172.21.15.78:0/1625966735 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbd90161010 0x7fbd9006d560 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c009880 tx=0x7fbd8c009b90 comp rx=0 tx=0).stop 2023-10-11T09:52:13.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.079+0000 7fbd97b44700 1 -- 172.21.15.78:0/1625966735 shutdown_connections 2023-10-11T09:52:13.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.079+0000 7fbd97b44700 1 --2- 172.21.15.78:0/1625966735 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbd90161010 0x7fbd9006d560 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:13.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.079+0000 7fbd97b44700 1 -- 172.21.15.78:0/1625966735 >> 172.21.15.78:0/1625966735 conn(0x7fbd9006ae10 msgr2=0x7fbd900b0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:13.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.079+0000 7fbd97b44700 1 -- 172.21.15.78:0/1625966735 shutdown_connections 2023-10-11T09:52:13.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.079+0000 7fbd97b44700 1 -- 172.21.15.78:0/1625966735 wait complete. 2023-10-11T09:52:13.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.079+0000 7fbd97b44700 1 Processor -- start 2023-10-11T09:52:13.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.079+0000 7fbd97b44700 1 -- start start 2023-10-11T09:52:13.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.083+0000 7fbd97b44700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbd900cea30 0x7fbd9015a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:13.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.083+0000 7fbd97b44700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd90161010 0x7fbd90153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:13.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.083+0000 7fbd97b44700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd900cd310 0x7fbd900cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:13.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.083+0000 7fbd97b44700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd900cdcf0 con 0x7fbd900cd310 2023-10-11T09:52:13.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.083+0000 7fbd97b44700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9015d580 con 0x7fbd90161010 2023-10-11T09:52:13.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.083+0000 7fbd97b44700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9015cd40 con 0x7fbd900cea30 2023-10-11T09:52:13.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.083+0000 7fbd960e1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd900cd310 0x7fbd900cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:13.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.083+0000 7fbd960e1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd900cd310 0x7fbd900cd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39240/0 (socket says 172.21.15.78:39240) 2023-10-11T09:52:13.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.083+0000 7fbd960e1700 1 -- 172.21.15.78:0/1981001886 learned_addr learned my addr 172.21.15.78:0/1981001886 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:13.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.083+0000 7fbd958e0700 1 --2- 172.21.15.78:0/1981001886 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbd900cea30 0x7fbd9015a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:13.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.083+0000 7fbd950df700 1 --2- 172.21.15.78:0/1981001886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd90161010 0x7fbd90153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:13.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.083+0000 7fbd960e1700 1 -- 172.21.15.78:0/1981001886 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbd900cea30 msgr2=0x7fbd9015a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:13.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.083+0000 7fbd960e1700 1 --2- 172.21.15.78:0/1981001886 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbd900cea30 0x7fbd9015a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:13.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.083+0000 7fbd960e1700 1 -- 172.21.15.78:0/1981001886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd90161010 msgr2=0x7fbd90153c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:13.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.083+0000 7fbd960e1700 1 --2- 172.21.15.78:0/1981001886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd90161010 0x7fbd90153c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:13.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.083+0000 7fbd960e1700 1 -- 172.21.15.78:0/1981001886 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8c0094b0 con 0x7fbd900cd310 2023-10-11T09:52:13.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.083+0000 7fbd960e1700 1 --2- 172.21.15.78:0/1981001886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd900cd310 0x7fbd900cd7b0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fbd8800cc50 tx=0x7fbd8800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:13.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.083+0000 7fbd867fc700 1 -- 172.21.15.78:0/1981001886 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8800ad40 con 0x7fbd900cd310 2023-10-11T09:52:13.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.083+0000 7fbd867fc700 1 -- 172.21.15.78:0/1981001886 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd8800ce50 con 0x7fbd900cd310 2023-10-11T09:52:13.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.083+0000 7fbd97b44700 1 -- 172.21.15.78:0/1981001886 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd9015b260 con 0x7fbd900cd310 2023-10-11T09:52:13.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.083+0000 7fbd867fc700 1 -- 172.21.15.78:0/1981001886 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8800aea0 con 0x7fbd900cd310 2023-10-11T09:52:13.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.083+0000 7fbd97b44700 1 -- 172.21.15.78:0/1981001886 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd9015d2a0 con 0x7fbd900cd310 2023-10-11T09:52:13.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.087+0000 7fbd97b44700 1 -- 172.21.15.78:0/1981001886 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd9015b0c0 con 0x7fbd900cd310 2023-10-11T09:52:13.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.087+0000 7fbd867fc700 1 -- 172.21.15.78:0/1981001886 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fbd88003c60 con 0x7fbd900cd310 2023-10-11T09:52:13.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.087+0000 7fbd867fc700 1 --2- 172.21.15.78:0/1981001886 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbd74065ed0 0x7fbd74068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:13.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.091+0000 7fbd867fc700 1 -- 172.21.15.78:0/1981001886 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fbd88083a50 con 0x7fbd900cd310 2023-10-11T09:52:13.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.091+0000 7fbd958e0700 1 --2- 172.21.15.78:0/1981001886 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbd74065ed0 0x7fbd74068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:13.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.091+0000 7fbd958e0700 1 --2- 172.21.15.78:0/1981001886 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbd74065ed0 0x7fbd74068380 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c0090b0 tx=0x7fbd8c01c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:13.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.095+0000 7fbd867fc700 1 -- 172.21.15.78:0/1981001886 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fbd9015b0c0 con 0x7fbd900cd310 2023-10-11T09:52:13.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.267+0000 7fbd97b44700 1 -- 172.21.15.78:0/1981001886 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbd9015b0c0 con 0x7fbd900cd310 2023-10-11T09:52:13.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.267+0000 7fbd867fc700 1 -- 172.21.15.78:0/1981001886 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v128) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbd9015b0c0 con 0x7fbd900cd310 2023-10-11T09:52:13.274 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2023-10-11T09:52:13.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.267+0000 7fbd97b44700 1 -- 172.21.15.78:0/1981001886 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbd74065ed0 msgr2=0x7fbd74068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:13.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.267+0000 7fbd97b44700 1 --2- 172.21.15.78:0/1981001886 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbd74065ed0 0x7fbd74068380 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c0090b0 tx=0x7fbd8c01c040 comp rx=0 tx=0).stop 2023-10-11T09:52:13.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.267+0000 7fbd97b44700 1 -- 172.21.15.78:0/1981001886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd900cd310 msgr2=0x7fbd900cd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:13.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.267+0000 7fbd97b44700 1 --2- 172.21.15.78:0/1981001886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd900cd310 0x7fbd900cd7b0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fbd8800cc50 tx=0x7fbd8800a340 comp rx=0 tx=0).stop 2023-10-11T09:52:13.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.267+0000 7fbd97b44700 1 -- 172.21.15.78:0/1981001886 shutdown_connections 2023-10-11T09:52:13.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.267+0000 7fbd97b44700 1 --2- 172.21.15.78:0/1981001886 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbd74065ed0 0x7fbd74068380 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:13.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.267+0000 7fbd97b44700 1 --2- 172.21.15.78:0/1981001886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd900cd310 0x7fbd900cd7b0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:13.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.267+0000 7fbd97b44700 1 --2- 172.21.15.78:0/1981001886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd90161010 0x7fbd90153c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:13.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.267+0000 7fbd97b44700 1 --2- 172.21.15.78:0/1981001886 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbd900cea30 0x7fbd9015a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:13.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.267+0000 7fbd97b44700 1 -- 172.21.15.78:0/1981001886 >> 172.21.15.78:0/1981001886 conn(0x7fbd9006ae10 msgr2=0x7fbd900ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:13.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.271+0000 7fbd97b44700 1 -- 172.21.15.78:0/1981001886 shutdown_connections 2023-10-11T09:52:13.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.271+0000 7fbd97b44700 1 -- 172.21.15.78:0/1981001886 wait complete. 2023-10-11T09:52:13.297 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T09:52:13.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.467+0000 7f07089a6700 1 Processor -- start 2023-10-11T09:52:13.477 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.471+0000 7f07089a6700 1 -- start start 2023-10-11T09:52:13.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.471+0000 7f07089a6700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f070415d760 0x7f07040cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:13.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.471+0000 7f07089a6700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f07040cfdd0 0x7f07040b7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:13.480 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.471+0000 7f07089a6700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f070415dc40 con 0x7f070415a230 2023-10-11T09:52:13.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.471+0000 7f07089a6700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07040cf1a0 con 0x7f07040cfdd0 2023-10-11T09:52:13.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.471+0000 7f07089a6700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07040d1230 con 0x7f070415d760 2023-10-11T09:52:13.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.475+0000 7f0702d9d700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f070415a230 0x7f07040d0b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:60534/0 (socket says 172.21.15.78:60534) 2023-10-11T09:52:13.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.475+0000 7f070359e700 1 --2- 172.21.15.78:0/1717888831 >> v2:172.21.15.78:3300/0 conn(0x7f07040cfdd0 0x7f07040b7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:13.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.475+0000 7f0702d9d700 1 -- 172.21.15.78:0/1717888831 learned_addr learned my addr 172.21.15.78:0/1717888831 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:13.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.475+0000 7f070259c700 1 --2- 172.21.15.78:0/1717888831 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f070415d760 0x7f07040cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:13.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.475+0000 7f070359e700 1 -- 172.21.15.78:0/1717888831 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f070415d760 msgr2=0x7f07040cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:13.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.475+0000 7f070359e700 1 --2- 172.21.15.78:0/1717888831 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f070415d760 0x7f07040cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:13.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.475+0000 7f070359e700 1 -- 172.21.15.78:0/1717888831 >> v1:172.21.15.78:6789/0 conn(0x7f070415a230 legacy=0x7f07040d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:13.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.475+0000 7f070359e700 1 -- 172.21.15.78:0/1717888831 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07040cf3b0 con 0x7f07040cfdd0 2023-10-11T09:52:13.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.475+0000 7f0701d9b700 1 -- 172.21.15.78:0/1717888831 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1416400002 0 0) 0x7f070415dc40 con 0x7f070415a230 2023-10-11T09:52:13.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.475+0000 7f070359e700 1 --2- 172.21.15.78:0/1717888831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07040cfdd0 0x7f07040b7ec0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f06fc009d90 tx=0x7f06fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf526183b5f1128 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:13.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.475+0000 7f0701d9b700 1 -- 172.21.15.78:0/1717888831 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06fc00e070 con 0x7f07040cfdd0 2023-10-11T09:52:13.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.475+0000 7f0701d9b700 1 -- 172.21.15.78:0/1717888831 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06fc00bb20 con 0x7f07040cfdd0 2023-10-11T09:52:13.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.475+0000 7f0701d9b700 1 -- 172.21.15.78:0/1717888831 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06fc012430 con 0x7f07040cfdd0 2023-10-11T09:52:13.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.475+0000 7f07089a6700 1 -- 172.21.15.78:0/1717888831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07040cfdd0 msgr2=0x7f07040b7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:13.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.475+0000 7f07089a6700 1 --2- 172.21.15.78:0/1717888831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07040cfdd0 0x7f07040b7ec0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f06fc009d90 tx=0x7f06fc00b330 comp rx=0 tx=0).stop 2023-10-11T09:52:13.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.475+0000 7f07089a6700 1 -- 172.21.15.78:0/1717888831 shutdown_connections 2023-10-11T09:52:13.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.475+0000 7f07089a6700 1 --2- 172.21.15.78:0/1717888831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07040cfdd0 0x7f07040b7ec0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:13.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.475+0000 7f07089a6700 1 --2- 172.21.15.78:0/1717888831 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f070415d760 0x7f07040cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:13.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.475+0000 7f07089a6700 1 -- 172.21.15.78:0/1717888831 >> 172.21.15.78:0/1717888831 conn(0x7f070406aa20 msgr2=0x7f070406adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:13.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.475+0000 7f07089a6700 1 -- 172.21.15.78:0/1717888831 shutdown_connections 2023-10-11T09:52:13.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.475+0000 7f07089a6700 1 -- 172.21.15.78:0/1717888831 wait complete. 2023-10-11T09:52:13.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.479+0000 7f07089a6700 1 Processor -- start 2023-10-11T09:52:13.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.479+0000 7f07089a6700 1 -- start start 2023-10-11T09:52:13.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.483+0000 7f07089a6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07040cfdd0 0x7f0704156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:13.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.483+0000 7f07089a6700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f070415a230 0x7f070415de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:13.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.483+0000 7f07089a6700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f070415d760 0x7f070415e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:13.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.483+0000 7f07089a6700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07040b6a40 con 0x7f07040cfdd0 2023-10-11T09:52:13.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.483+0000 7f07089a6700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070415f2b0 con 0x7f070415d760 2023-10-11T09:52:13.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.483+0000 7f07089a6700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f070415f590 con 0x7f070415a230 2023-10-11T09:52:13.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.483+0000 7f0702d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07040cfdd0 0x7f0704156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:13.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.483+0000 7f0702d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07040cfdd0 0x7f0704156490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39262/0 (socket says 172.21.15.78:39262) 2023-10-11T09:52:13.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.483+0000 7f0702d9d700 1 -- 172.21.15.78:0/4033373104 learned_addr learned my addr 172.21.15.78:0/4033373104 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:13.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.483+0000 7f070259c700 1 --2- 172.21.15.78:0/4033373104 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f070415a230 0x7f070415de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:13.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.483+0000 7f070359e700 1 --2- 172.21.15.78:0/4033373104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f070415d760 0x7f070415e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:13.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.483+0000 7f0702d9d700 1 -- 172.21.15.78:0/4033373104 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f070415a230 msgr2=0x7f070415de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:13.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.483+0000 7f0702d9d700 1 --2- 172.21.15.78:0/4033373104 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f070415a230 0x7f070415de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:13.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.483+0000 7f0702d9d700 1 -- 172.21.15.78:0/4033373104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f070415d760 msgr2=0x7f070415e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:13.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.483+0000 7f0702d9d700 1 --2- 172.21.15.78:0/4033373104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f070415d760 0x7f070415e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:13.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.483+0000 7f0702d9d700 1 -- 172.21.15.78:0/4033373104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06fc009a70 con 0x7f07040cfdd0 2023-10-11T09:52:13.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.483+0000 7f0702d9d700 1 --2- 172.21.15.78:0/4033373104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07040cfdd0 0x7f0704156490 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f06f400c580 tx=0x7f06f400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:13.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.483+0000 7f06eb7fe700 1 -- 172.21.15.78:0/4033373104 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f4004ca0 con 0x7f07040cfdd0 2023-10-11T09:52:13.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.483+0000 7f06eb7fe700 1 -- 172.21.15.78:0/4033373104 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f4004e20 con 0x7f07040cfdd0 2023-10-11T09:52:13.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.483+0000 7f07089a6700 1 -- 172.21.15.78:0/4033373104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f070415f870 con 0x7f07040cfdd0 2023-10-11T09:52:13.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.483+0000 7f06eb7fe700 1 -- 172.21.15.78:0/4033373104 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f4007310 con 0x7f07040cfdd0 2023-10-11T09:52:13.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.483+0000 7f07089a6700 1 -- 172.21.15.78:0/4033373104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07041569d0 con 0x7f07040cfdd0 2023-10-11T09:52:13.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.487+0000 7f07089a6700 1 -- 172.21.15.78:0/4033373104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0704065d40 con 0x7f07040cfdd0 2023-10-11T09:52:13.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.487+0000 7f06eb7fe700 1 -- 172.21.15.78:0/4033373104 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f06f4007470 con 0x7f07040cfdd0 2023-10-11T09:52:13.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.491+0000 7f06eb7fe700 1 --2- 172.21.15.78:0/4033373104 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f06f0065ed0 0x7f06f0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:13.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.491+0000 7f06eb7fe700 1 -- 172.21.15.78:0/4033373104 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f06f40834f0 con 0x7f07040cfdd0 2023-10-11T09:52:13.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.491+0000 7f070259c700 1 --2- 172.21.15.78:0/4033373104 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f06f0065ed0 0x7f06f0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:13.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.491+0000 7f070259c700 1 --2- 172.21.15.78:0/4033373104 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f06f0065ed0 0x7f06f0068380 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f07040c9f50 tx=0x7f06f8009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:13.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.495+0000 7f06eb7fe700 1 -- 172.21.15.78:0/4033373104 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f06f4050ad0 con 0x7f07040cfdd0 2023-10-11T09:52:13.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.695+0000 7f07089a6700 1 -- 172.21.15.78:0/4033373104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f07040b6b80 con 0x7f07040cfdd0 2023-10-11T09:52:13.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.695+0000 7f06eb7fe700 1 -- 172.21.15.78:0/4033373104 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v128) v1 ==== 80+0+124 (secure 0 0 0) 0x7f06f4007720 con 0x7f07040cfdd0 2023-10-11T09:52:13.704 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.133:6833/1433079498 2023-10-12T09:52:02.877755+0000 2023-10-11T09:52:13.704 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.133:6832/1433079498 2023-10-12T09:52:02.877755+0000 2023-10-11T09:52:13.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.699+0000 7f07089a6700 1 -- 172.21.15.78:0/4033373104 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f06f0065ed0 msgr2=0x7f06f0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:13.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.699+0000 7f07089a6700 1 --2- 172.21.15.78:0/4033373104 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f06f0065ed0 0x7f06f0068380 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f07040c9f50 tx=0x7f06f8009240 comp rx=0 tx=0).stop 2023-10-11T09:52:13.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.699+0000 7f07089a6700 1 -- 172.21.15.78:0/4033373104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07040cfdd0 msgr2=0x7f0704156490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:13.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.699+0000 7f07089a6700 1 --2- 172.21.15.78:0/4033373104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07040cfdd0 0x7f0704156490 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f06f400c580 tx=0x7f06f400c890 comp rx=0 tx=0).stop 2023-10-11T09:52:13.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.699+0000 7f07089a6700 1 -- 172.21.15.78:0/4033373104 shutdown_connections 2023-10-11T09:52:13.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.699+0000 7f07089a6700 1 --2- 172.21.15.78:0/4033373104 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f06f0065ed0 0x7f06f0068380 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:13.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.699+0000 7f07089a6700 1 --2- 172.21.15.78:0/4033373104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f070415d760 0x7f070415e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:13.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.699+0000 7f07089a6700 1 --2- 172.21.15.78:0/4033373104 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f070415a230 0x7f070415de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:13.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.699+0000 7f07089a6700 1 --2- 172.21.15.78:0/4033373104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07040cfdd0 0x7f0704156490 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:13.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.699+0000 7f07089a6700 1 -- 172.21.15.78:0/4033373104 >> 172.21.15.78:0/4033373104 conn(0x7f070406aa20 msgr2=0x7f07040b5230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:13.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.699+0000 7f07089a6700 1 -- 172.21.15.78:0/4033373104 shutdown_connections 2023-10-11T09:52:13.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.699+0000 7f07089a6700 1 -- 172.21.15.78:0/4033373104 wait complete. 2023-10-11T09:52:13.709 INFO:teuthology.orchestra.run.smithi078.stderr:listed 2 entries 2023-10-11T09:52:13.727 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:52:13.727 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:52:13.741 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:52:13.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.947+0000 7ff7362aa700 1 Processor -- start 2023-10-11T09:52:13.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.947+0000 7ff7362aa700 1 -- start start 2023-10-11T09:52:13.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.951+0000 7ff7362aa700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff73015a800 0x7ff7300b4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:13.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.951+0000 7ff7362aa700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ff73006bb60 0x7ff7300b0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:13.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.951+0000 7ff7362aa700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff730070dc0 con 0x7ff73006d060 2023-10-11T09:52:13.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.951+0000 7ff7362aa700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7300b46a0 con 0x7ff73006bb60 2023-10-11T09:52:13.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.951+0000 7ff7362aa700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff730070ff0 con 0x7ff73015a800 2023-10-11T09:52:13.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.951+0000 7ff734aa7700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ff73006bb60 0x7ff7300b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:13.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.951+0000 7ff734aa7700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7ff73006bb60 0x7ff7300b0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39276/0 (socket says 172.21.15.78:39276) 2023-10-11T09:52:13.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.951+0000 7ff734aa7700 1 -- 172.21.15.78:0/2182090890 learned_addr learned my addr 172.21.15.78:0/2182090890 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:13.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.951+0000 7ff735aa9700 1 --1- 172.21.15.78:0/2182090890 >> v1:172.21.15.78:6789/0 conn(0x7ff73006d060 0x7ff7300c9150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:60542/0 (socket says 172.21.15.78:60542) 2023-10-11T09:52:13.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.951+0000 7ff7352a8700 1 --2- 172.21.15.78:0/2182090890 >> v2:172.21.15.133:3300/0 conn(0x7ff73015a800 0x7ff7300b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:13.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.951+0000 7ff734aa7700 1 -- 172.21.15.78:0/2182090890 >> v2:172.21.15.133:3300/0 conn(0x7ff73015a800 msgr2=0x7ff7300b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:13.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.951+0000 7ff734aa7700 1 --2- 172.21.15.78:0/2182090890 >> v2:172.21.15.133:3300/0 conn(0x7ff73015a800 0x7ff7300b4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:13.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.951+0000 7ff734aa7700 1 -- 172.21.15.78:0/2182090890 >> v1:172.21.15.78:6789/0 conn(0x7ff73006d060 legacy=0x7ff7300c9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:13.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.951+0000 7ff734aa7700 1 -- 172.21.15.78:0/2182090890 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7300712d0 con 0x7ff73006bb60 2023-10-11T09:52:13.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.951+0000 7ff727fff700 1 -- 172.21.15.78:0/2182090890 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2842004165 0 0) 0x7ff730070dc0 con 0x7ff73006d060 2023-10-11T09:52:13.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.951+0000 7ff734aa7700 1 --2- 172.21.15.78:0/2182090890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff73006bb60 0x7ff7300b0610 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7ff7300cbea0 tx=0x7ff720009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b1dc0f8905b5bfc server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:13.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.955+0000 7ff727fff700 1 -- 172.21.15.78:0/2182090890 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff720005e50 con 0x7ff73006bb60 2023-10-11T09:52:13.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.955+0000 7ff727fff700 1 -- 172.21.15.78:0/2182090890 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff720006030 con 0x7ff73006bb60 2023-10-11T09:52:13.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.955+0000 7ff727fff700 1 -- 172.21.15.78:0/2182090890 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7200063c0 con 0x7ff73006bb60 2023-10-11T09:52:13.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.955+0000 7ff7362aa700 1 -- 172.21.15.78:0/2182090890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff73006bb60 msgr2=0x7ff7300b0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:13.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.955+0000 7ff7362aa700 1 --2- 172.21.15.78:0/2182090890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff73006bb60 0x7ff7300b0610 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7ff7300cbea0 tx=0x7ff720009bf0 comp rx=0 tx=0).stop 2023-10-11T09:52:13.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.955+0000 7ff7362aa700 1 -- 172.21.15.78:0/2182090890 shutdown_connections 2023-10-11T09:52:13.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.955+0000 7ff7362aa700 1 --2- 172.21.15.78:0/2182090890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff73006bb60 0x7ff7300b0610 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:13.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.955+0000 7ff7362aa700 1 --2- 172.21.15.78:0/2182090890 >> v2:172.21.15.133:3300/0 conn(0x7ff73015a800 0x7ff7300b4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:13.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.955+0000 7ff7362aa700 1 -- 172.21.15.78:0/2182090890 >> 172.21.15.78:0/2182090890 conn(0x7ff7300cf820 msgr2=0x7ff7300cb610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:13.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.955+0000 7ff7362aa700 1 -- 172.21.15.78:0/2182090890 shutdown_connections 2023-10-11T09:52:13.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.955+0000 7ff7362aa700 1 -- 172.21.15.78:0/2182090890 wait complete. 2023-10-11T09:52:13.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.959+0000 7ff7362aa700 1 Processor -- start 2023-10-11T09:52:13.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.959+0000 7ff7362aa700 1 -- start start 2023-10-11T09:52:13.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.959+0000 7ff7362aa700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff73006bb60 0x7ff7300c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:13.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.959+0000 7ff7362aa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff73006d060 0x7ff7301ed710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:13.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.959+0000 7ff7362aa700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff73015a800 0x7ff7301f1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:13.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.959+0000 7ff7362aa700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7300c9230 con 0x7ff73006d060 2023-10-11T09:52:13.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.959+0000 7ff7362aa700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7300c6b20 con 0x7ff73015a800 2023-10-11T09:52:13.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.959+0000 7ff7362aa700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7300c5bd0 con 0x7ff73006bb60 2023-10-11T09:52:13.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.959+0000 7ff734aa7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff73006d060 0x7ff7301ed710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:13.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.963+0000 7ff734aa7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff73006d060 0x7ff7301ed710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39290/0 (socket says 172.21.15.78:39290) 2023-10-11T09:52:13.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.963+0000 7ff734aa7700 1 -- 172.21.15.78:0/1394275073 learned_addr learned my addr 172.21.15.78:0/1394275073 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:13.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.963+0000 7ff735aa9700 1 --2- 172.21.15.78:0/1394275073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff73015a800 0x7ff7301f1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:13.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.963+0000 7ff7352a8700 1 --2- 172.21.15.78:0/1394275073 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff73006bb60 0x7ff7300c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:13.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.963+0000 7ff734aa7700 1 -- 172.21.15.78:0/1394275073 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff73006bb60 msgr2=0x7ff7300c65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:13.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.963+0000 7ff734aa7700 1 --2- 172.21.15.78:0/1394275073 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff73006bb60 0x7ff7300c65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:13.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.963+0000 7ff734aa7700 1 -- 172.21.15.78:0/1394275073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff73015a800 msgr2=0x7ff7301f1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:13.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.963+0000 7ff734aa7700 1 --2- 172.21.15.78:0/1394275073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff73015a800 0x7ff7301f1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:13.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.963+0000 7ff734aa7700 1 -- 172.21.15.78:0/1394275073 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff720009520 con 0x7ff73006d060 2023-10-11T09:52:13.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.963+0000 7ff734aa7700 1 --2- 172.21.15.78:0/1394275073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff73006d060 0x7ff7301ed710 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7ff720009c30 tx=0x7ff720004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:13.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.963+0000 7ff725ffb700 1 -- 172.21.15.78:0/1394275073 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff720004250 con 0x7ff73006d060 2023-10-11T09:52:13.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.963+0000 7ff725ffb700 1 -- 172.21.15.78:0/1394275073 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7200043d0 con 0x7ff73006d060 2023-10-11T09:52:13.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.963+0000 7ff725ffb700 1 -- 172.21.15.78:0/1394275073 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7200036c0 con 0x7ff73006d060 2023-10-11T09:52:13.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.963+0000 7ff7362aa700 1 -- 172.21.15.78:0/1394275073 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7300c5eb0 con 0x7ff73006d060 2023-10-11T09:52:13.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.963+0000 7ff7362aa700 1 -- 172.21.15.78:0/1394275073 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7300c8ec0 con 0x7ff73006d060 2023-10-11T09:52:13.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.963+0000 7ff7362aa700 1 -- 172.21.15.78:0/1394275073 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff730067d00 con 0x7ff73006d060 2023-10-11T09:52:13.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.967+0000 7ff725ffb700 1 -- 172.21.15.78:0/1394275073 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7ff720022070 con 0x7ff73006d060 2023-10-11T09:52:13.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.967+0000 7ff725ffb700 1 --2- 172.21.15.78:0/1394275073 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff714065ed0 0x7ff714068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:13.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.967+0000 7ff725ffb700 1 -- 172.21.15.78:0/1394275073 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6148+0+0 (secure 0 0 0) 0x7ff720085100 con 0x7ff73006d060 2023-10-11T09:52:13.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.967+0000 7ff7352a8700 1 --2- 172.21.15.78:0/1394275073 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff714065ed0 0x7ff714068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:13.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.967+0000 7ff7352a8700 1 --2- 172.21.15.78:0/1394275073 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff714065ed0 0x7ff714068380 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ff72c009fd0 tx=0x7ff72c011040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:13.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:13.975+0000 7ff725ffb700 1 -- 172.21.15.78:0/1394275073 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7ff720052780 con 0x7ff73006d060 2023-10-11T09:52:14.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.155+0000 7ff7362aa700 1 -- 172.21.15.78:0/1394275073 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff730065d40 con 0x7ff73006d060 2023-10-11T09:52:14.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.155+0000 7ff725ffb700 1 -- 172.21.15.78:0/1394275073 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+18024 (secure 0 0 0) 0x7ff720055da0 con 0x7ff73006d060 2023-10-11T09:52:14.164 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:52:14.164 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":128,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:52:02.877803+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:51:38.592892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-11T09:51:39.204588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6833/1433079498":"2023-10-12T09:52:02.877755+0000","172.21.15.133:6832/1433079498":"2023-10-12T09:52:02.877755+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:52:14.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.159+0000 7ff7362aa700 1 -- 172.21.15.78:0/1394275073 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff714065ed0 msgr2=0x7ff714068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:14.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.159+0000 7ff7362aa700 1 --2- 172.21.15.78:0/1394275073 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff714065ed0 0x7ff714068380 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ff72c009fd0 tx=0x7ff72c011040 comp rx=0 tx=0).stop 2023-10-11T09:52:14.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.159+0000 7ff7362aa700 1 -- 172.21.15.78:0/1394275073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff73006d060 msgr2=0x7ff7301ed710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:14.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.159+0000 7ff7362aa700 1 --2- 172.21.15.78:0/1394275073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff73006d060 0x7ff7301ed710 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7ff720009c30 tx=0x7ff720004770 comp rx=0 tx=0).stop 2023-10-11T09:52:14.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.159+0000 7ff7362aa700 1 -- 172.21.15.78:0/1394275073 shutdown_connections 2023-10-11T09:52:14.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.159+0000 7ff7362aa700 1 --2- 172.21.15.78:0/1394275073 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff714065ed0 0x7ff714068380 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:14.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.159+0000 7ff7362aa700 1 --2- 172.21.15.78:0/1394275073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff73015a800 0x7ff7301f1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:14.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.159+0000 7ff7362aa700 1 --2- 172.21.15.78:0/1394275073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff73006d060 0x7ff7301ed710 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:14.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.159+0000 7ff7362aa700 1 --2- 172.21.15.78:0/1394275073 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff73006bb60 0x7ff7300c65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:14.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.159+0000 7ff7362aa700 1 -- 172.21.15.78:0/1394275073 >> 172.21.15.78:0/1394275073 conn(0x7ff7300cf820 msgr2=0x7ff730159d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:14.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.159+0000 7ff7362aa700 1 -- 172.21.15.78:0/1394275073 shutdown_connections 2023-10-11T09:52:14.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.159+0000 7ff7362aa700 1 -- 172.21.15.78:0/1394275073 wait complete. 2023-10-11T09:52:14.189 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T09:51:38.592892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T09:51:39.204588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '127', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:52:14.190 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:52:14.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.355+0000 7f67c759e700 1 Processor -- start 2023-10-11T09:52:14.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.355+0000 7f67c759e700 1 -- start start 2023-10-11T09:52:14.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.355+0000 7f67c759e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67c80cdcd0 0x7f67c806a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:14.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.355+0000 7f67c759e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f67c806a6c0 0x7f67c806ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:14.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.355+0000 7f67c759e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67c80ced50 con 0x7f67c80ce980 2023-10-11T09:52:14.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.355+0000 7f67c759e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67c806ab40 con 0x7f67c806a6c0 2023-10-11T09:52:14.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.355+0000 7f67c759e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c815ec90 con 0x7f67c80cdcd0 2023-10-11T09:52:14.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.359+0000 7f67c6d9d700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f67c806a6c0 0x7f67c806ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:14.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.359+0000 7f67bdd9b700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67c80cdcd0 0x7f67c806a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:14.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.359+0000 7f67bdd9b700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67c80cdcd0 0x7f67c806a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:41700/0 (socket says 172.21.15.78:41700) 2023-10-11T09:52:14.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.359+0000 7f67c6d9d700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f67c806a6c0 0x7f67c806ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:53772/0 (socket says 172.21.15.78:53772) 2023-10-11T09:52:14.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.359+0000 7f67bdd9b700 1 -- 172.21.15.78:0/2966554844 learned_addr learned my addr 172.21.15.78:0/2966554844 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:14.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.359+0000 7f67c659c700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f67c80ce980 0x7f67c80ca420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:50918/0 (socket says 172.21.15.78:50918) 2023-10-11T09:52:14.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.359+0000 7f67bdd9b700 1 -- 172.21.15.78:0/2966554844 >> v2:172.21.15.133:3300/0 conn(0x7f67c806a6c0 msgr2=0x7f67c806ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:14.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.359+0000 7f67bdd9b700 1 --2- 172.21.15.78:0/2966554844 >> v2:172.21.15.133:3300/0 conn(0x7f67c806a6c0 0x7f67c806ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:14.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.359+0000 7f67bdd9b700 1 -- 172.21.15.78:0/2966554844 >> v1:172.21.15.133:6789/0 conn(0x7f67c80ce980 legacy=0x7f67c80ca420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:14.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.359+0000 7f67bdd9b700 1 -- 172.21.15.78:0/2966554844 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c81611c0 con 0x7f67c80cdcd0 2023-10-11T09:52:14.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.359+0000 7f67c5d9b700 1 -- 172.21.15.78:0/2966554844 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2898023938 0 0) 0x7f67c80ced50 con 0x7f67c80ce980 2023-10-11T09:52:14.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.359+0000 7f67bdd9b700 1 --2- 172.21.15.78:0/2966554844 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67c80cdcd0 0x7f67c806a180 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f67b0009e50 tx=0x7f67b000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43a9b9cee925ad31 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:14.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.359+0000 7f67c5d9b700 1 -- 172.21.15.78:0/2966554844 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b000e070 con 0x7f67c80cdcd0 2023-10-11T09:52:14.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.359+0000 7f67c5d9b700 1 -- 172.21.15.78:0/2966554844 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b000b910 con 0x7f67c80cdcd0 2023-10-11T09:52:14.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.359+0000 7f67c5d9b700 1 -- 172.21.15.78:0/2966554844 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b000bcc0 con 0x7f67c80cdcd0 2023-10-11T09:52:14.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.359+0000 7f67c759e700 1 -- 172.21.15.78:0/2966554844 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67c80cdcd0 msgr2=0x7f67c806a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:14.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.359+0000 7f67c759e700 1 --2- 172.21.15.78:0/2966554844 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67c80cdcd0 0x7f67c806a180 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f67b0009e50 tx=0x7f67b000b330 comp rx=0 tx=0).stop 2023-10-11T09:52:14.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.359+0000 7f67c759e700 1 -- 172.21.15.78:0/2966554844 shutdown_connections 2023-10-11T09:52:14.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.359+0000 7f67c759e700 1 --2- 172.21.15.78:0/2966554844 >> v2:172.21.15.133:3300/0 conn(0x7f67c806a6c0 0x7f67c806ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:14.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.359+0000 7f67c759e700 1 --2- 172.21.15.78:0/2966554844 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67c80cdcd0 0x7f67c806a180 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:14.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.359+0000 7f67c759e700 1 -- 172.21.15.78:0/2966554844 >> 172.21.15.78:0/2966554844 conn(0x7f67c80cc850 msgr2=0x7f67c80ccc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:14.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.359+0000 7f67c759e700 1 -- 172.21.15.78:0/2966554844 shutdown_connections 2023-10-11T09:52:14.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.359+0000 7f67c759e700 1 -- 172.21.15.78:0/2966554844 wait complete. 2023-10-11T09:52:14.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.363+0000 7f67c759e700 1 Processor -- start 2023-10-11T09:52:14.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.363+0000 7f67c759e700 1 -- start start 2023-10-11T09:52:14.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.363+0000 7f67c759e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67c806a6c0 0x7f67c80b4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:14.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.363+0000 7f67c759e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67c80cdcd0 0x7f67c80c2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:14.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.363+0000 7f67c759e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67c80ce980 0x7f67c80c4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:14.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.363+0000 7f67c759e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c80b2c00 con 0x7f67c80ce980 2023-10-11T09:52:14.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.363+0000 7f67c759e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c80b2ee0 con 0x7f67c806a6c0 2023-10-11T09:52:14.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.363+0000 7f67c759e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c80b31c0 con 0x7f67c80cdcd0 2023-10-11T09:52:14.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.363+0000 7f67c6d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67c80ce980 0x7f67c80c4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:14.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.367+0000 7f67c6d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67c80ce980 0x7f67c80c4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39304/0 (socket says 172.21.15.78:39304) 2023-10-11T09:52:14.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.367+0000 7f67c6d9d700 1 -- 172.21.15.78:0/3002607324 learned_addr learned my addr 172.21.15.78:0/3002607324 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:14.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.367+0000 7f67bdd9b700 1 --2- 172.21.15.78:0/3002607324 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67c80cdcd0 0x7f67c80c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:14.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.367+0000 7f67c659c700 1 --2- 172.21.15.78:0/3002607324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67c806a6c0 0x7f67c80b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:14.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.367+0000 7f67c6d9d700 1 -- 172.21.15.78:0/3002607324 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67c80cdcd0 msgr2=0x7f67c80c2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:14.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.367+0000 7f67c6d9d700 1 --2- 172.21.15.78:0/3002607324 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67c80cdcd0 0x7f67c80c2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:14.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.367+0000 7f67c6d9d700 1 -- 172.21.15.78:0/3002607324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67c806a6c0 msgr2=0x7f67c80b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:14.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.367+0000 7f67c6d9d700 1 --2- 172.21.15.78:0/3002607324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67c806a6c0 0x7f67c80b4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:14.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.367+0000 7f67c6d9d700 1 -- 172.21.15.78:0/3002607324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b0009b30 con 0x7f67c80ce980 2023-10-11T09:52:14.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.367+0000 7f67c6d9d700 1 --2- 172.21.15.78:0/3002607324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67c80ce980 0x7f67c80c4650 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f67c000e7f0 tx=0x7f67c000eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:14.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.367+0000 7f67bf7fe700 1 -- 172.21.15.78:0/3002607324 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c00095c0 con 0x7f67c80ce980 2023-10-11T09:52:14.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.367+0000 7f67bf7fe700 1 -- 172.21.15.78:0/3002607324 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c0013680 con 0x7f67c80ce980 2023-10-11T09:52:14.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.367+0000 7f67c759e700 1 -- 172.21.15.78:0/3002607324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67c80b34a0 con 0x7f67c80ce980 2023-10-11T09:52:14.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.367+0000 7f67bf7fe700 1 -- 172.21.15.78:0/3002607324 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c0013a10 con 0x7f67c80ce980 2023-10-11T09:52:14.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.367+0000 7f67c759e700 1 -- 172.21.15.78:0/3002607324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c80b3a60 con 0x7f67c80ce980 2023-10-11T09:52:14.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.371+0000 7f67c759e700 1 -- 172.21.15.78:0/3002607324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67c80744c0 con 0x7f67c80ce980 2023-10-11T09:52:14.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.371+0000 7f67bf7fe700 1 -- 172.21.15.78:0/3002607324 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f67c0013b90 con 0x7f67c80ce980 2023-10-11T09:52:14.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.371+0000 7f67bf7fe700 1 --2- 172.21.15.78:0/3002607324 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f67a8065ed0 0x7f67a8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:14.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.371+0000 7f67bf7fe700 1 -- 172.21.15.78:0/3002607324 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f67c00836a0 con 0x7f67c80ce980 2023-10-11T09:52:14.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.371+0000 7f67c659c700 1 --2- 172.21.15.78:0/3002607324 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f67a8065ed0 0x7f67a8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:14.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.375+0000 7f67c659c700 1 --2- 172.21.15.78:0/3002607324 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f67a8065ed0 0x7f67a8068380 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f67b8000f60 tx=0x7f67b8007e00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:14.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.379+0000 7f67bf7fe700 1 -- 172.21.15.78:0/3002607324 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f67c0050bf0 con 0x7f67c80ce980 2023-10-11T09:52:14.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.555+0000 7f67c759e700 1 -- 172.21.15.78:0/3002607324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f67c80af4b0 con 0x7f67c80ce980 2023-10-11T09:52:14.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.555+0000 7f67bf7fe700 1 -- 172.21.15.78:0/3002607324 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v128) v1 ==== 93+0+10 (secure 0 0 0) 0x7f67c0054210 con 0x7f67c80ce980 2023-10-11T09:52:14.562 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:52:14.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.555+0000 7f67c759e700 1 -- 172.21.15.78:0/3002607324 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f67a8065ed0 msgr2=0x7f67a8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:14.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.555+0000 7f67c759e700 1 --2- 172.21.15.78:0/3002607324 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f67a8065ed0 0x7f67a8068380 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f67b8000f60 tx=0x7f67b8007e00 comp rx=0 tx=0).stop 2023-10-11T09:52:14.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.555+0000 7f67c759e700 1 -- 172.21.15.78:0/3002607324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67c80ce980 msgr2=0x7f67c80c4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:14.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.555+0000 7f67c759e700 1 --2- 172.21.15.78:0/3002607324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67c80ce980 0x7f67c80c4650 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f67c000e7f0 tx=0x7f67c000eb00 comp rx=0 tx=0).stop 2023-10-11T09:52:14.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.559+0000 7f67c759e700 1 -- 172.21.15.78:0/3002607324 shutdown_connections 2023-10-11T09:52:14.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.559+0000 7f67c759e700 1 --2- 172.21.15.78:0/3002607324 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f67a8065ed0 0x7f67a8068380 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:14.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.559+0000 7f67c759e700 1 --2- 172.21.15.78:0/3002607324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67c80ce980 0x7f67c80c4650 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:14.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.559+0000 7f67c759e700 1 --2- 172.21.15.78:0/3002607324 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67c80cdcd0 0x7f67c80c2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:14.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.559+0000 7f67c759e700 1 --2- 172.21.15.78:0/3002607324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67c806a6c0 0x7f67c80b4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:14.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.559+0000 7f67c759e700 1 -- 172.21.15.78:0/3002607324 >> 172.21.15.78:0/3002607324 conn(0x7f67c80cc850 msgr2=0x7f67c80ccc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:14.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.559+0000 7f67c759e700 1 -- 172.21.15.78:0/3002607324 shutdown_connections 2023-10-11T09:52:14.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.559+0000 7f67c759e700 1 -- 172.21.15.78:0/3002607324 wait complete. 2023-10-11T09:52:14.588 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:52:14.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.755+0000 7f174b47e700 1 Processor -- start 2023-10-11T09:52:14.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.755+0000 7f174b47e700 1 -- start start 2023-10-11T09:52:14.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.759+0000 7f174b47e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f17440d16f0 0x7f17440cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:14.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.759+0000 7f174b47e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f174409c960 con 0x7f17440b74c0 2023-10-11T09:52:14.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.759+0000 7f174b47e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f174415d180 con 0x7f17440b4530 2023-10-11T09:52:14.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.759+0000 7f174b47e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17440b5fd0 con 0x7f17440d16f0 2023-10-11T09:52:14.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.759+0000 7f174ac7d700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f17440b74c0 0x7f17440cdc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:48042/0 (socket says 172.21.15.78:48042) 2023-10-11T09:52:14.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.759+0000 7f174ac7d700 1 -- 172.21.15.78:0/3686725320 learned_addr learned my addr 172.21.15.78:0/3686725320 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:14.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.759+0000 7f1749c7b700 1 --2- 172.21.15.78:0/3686725320 >> v2:172.21.15.133:3300/0 conn(0x7f17440d16f0 0x7f17440cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:14.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.759+0000 7f174947a700 1 -- 172.21.15.78:0/3686725320 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3332614810 0 0) 0x7f174409c960 con 0x7f17440b74c0 2023-10-11T09:52:14.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.759+0000 7f174947a700 1 -- 172.21.15.78:0/3686725320 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17300035d0 con 0x7f17440b74c0 2023-10-11T09:52:14.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.763+0000 7f174947a700 1 -- 172.21.15.78:0/3686725320 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 180410638 0 0) 0x7f17300035d0 con 0x7f17440b74c0 2023-10-11T09:52:14.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.763+0000 7f174947a700 1 -- 172.21.15.78:0/3686725320 >> v2:172.21.15.133:3300/0 conn(0x7f17440d16f0 msgr2=0x7f17440cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:14.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.763+0000 7f174947a700 1 --2- 172.21.15.78:0/3686725320 >> v2:172.21.15.133:3300/0 conn(0x7f17440d16f0 0x7f17440cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:14.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.763+0000 7f174947a700 1 -- 172.21.15.78:0/3686725320 >> v1:172.21.15.133:6789/0 conn(0x7f17440b4530 legacy=0x7f17440d0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:14.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.763+0000 7f174947a700 1 -- 172.21.15.78:0/3686725320 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17440cfb70 con 0x7f17440b74c0 2023-10-11T09:52:14.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.763+0000 7f174947a700 1 -- 172.21.15.78:0/3686725320 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f173c0019f0 con 0x7f17440b74c0 2023-10-11T09:52:14.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.763+0000 7f174947a700 1 -- 172.21.15.78:0/3686725320 >> v1:172.21.15.78:6789/0 conn(0x7f17440b74c0 legacy=0x7f17440cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:14.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.763+0000 7f174947a700 1 --2- 172.21.15.78:0/3686725320 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1730003a20 0x7f1730003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:14.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.763+0000 7f174947a700 1 --2- 172.21.15.78:0/3686725320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1730004870 0x7f1730004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:14.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.763+0000 7f174947a700 1 --2- 172.21.15.78:0/3686725320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1730005220 0x7f173000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:14.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.763+0000 7f174947a700 1 -- 172.21.15.78:0/3686725320 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17440b5fd0 con 0x7f1730004870 2023-10-11T09:52:14.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.763+0000 7f174947a700 1 -- 172.21.15.78:0/3686725320 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173000c080 con 0x7f1730005220 2023-10-11T09:52:14.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.763+0000 7f174947a700 1 -- 172.21.15.78:0/3686725320 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f173000c240 con 0x7f1730003a20 2023-10-11T09:52:14.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.763+0000 7f1749c7b700 1 --2- 172.21.15.78:0/3686725320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1730004870 0x7f1730004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:14.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.763+0000 7f174a47c700 1 --2- 172.21.15.78:0/3686725320 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1730003a20 0x7f1730003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:14.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.763+0000 7f174ac7d700 1 --2- 172.21.15.78:0/3686725320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1730005220 0x7f173000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:14.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.763+0000 7f1749c7b700 1 -- 172.21.15.78:0/3686725320 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1730003a20 msgr2=0x7f1730003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:14.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.763+0000 7f1749c7b700 1 --2- 172.21.15.78:0/3686725320 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1730003a20 0x7f1730003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:14.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.763+0000 7f1749c7b700 1 -- 172.21.15.78:0/3686725320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1730005220 msgr2=0x7f173000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:14.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.763+0000 7f1749c7b700 1 --2- 172.21.15.78:0/3686725320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1730005220 0x7f173000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:14.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.763+0000 7f1749c7b700 1 -- 172.21.15.78:0/3686725320 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17300035d0 con 0x7f1730004870 2023-10-11T09:52:14.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.763+0000 7f174ac7d700 1 -- 172.21.15.78:0/3686725320 reap_dead start 2023-10-11T09:52:14.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.763+0000 7f1749c7b700 1 --2- 172.21.15.78:0/3686725320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1730004870 0x7f1730004ce0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f1734009b70 tx=0x7f17300058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5684db3afed0dd6 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:14.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.763+0000 7f174947a700 1 -- 172.21.15.78:0/3686725320 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173400e6c0 con 0x7f1730004870 2023-10-11T09:52:14.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.763+0000 7f174947a700 1 -- 172.21.15.78:0/3686725320 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f173400e820 con 0x7f1730004870 2023-10-11T09:52:14.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.763+0000 7f174947a700 1 -- 172.21.15.78:0/3686725320 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173400eb90 con 0x7f1730004870 2023-10-11T09:52:14.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.763+0000 7f174b47e700 1 -- 172.21.15.78:0/3686725320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1730004870 msgr2=0x7f1730004ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:14.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.763+0000 7f174b47e700 1 --2- 172.21.15.78:0/3686725320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1730004870 0x7f1730004ce0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f1734009b70 tx=0x7f17300058a0 comp rx=0 tx=0).stop 2023-10-11T09:52:14.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.763+0000 7f174b47e700 1 -- 172.21.15.78:0/3686725320 shutdown_connections 2023-10-11T09:52:14.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.763+0000 7f174b47e700 1 --2- 172.21.15.78:0/3686725320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1730004870 0x7f1730004ce0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:14.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.763+0000 7f174b47e700 1 -- 172.21.15.78:0/3686725320 >> 172.21.15.78:0/3686725320 conn(0x7f174409c3d0 msgr2=0x7f17440d30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:14.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.763+0000 7f174b47e700 1 -- 172.21.15.78:0/3686725320 shutdown_connections 2023-10-11T09:52:14.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.763+0000 7f174b47e700 1 -- 172.21.15.78:0/3686725320 wait complete. 2023-10-11T09:52:14.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.767+0000 7f174b47e700 1 Processor -- start 2023-10-11T09:52:14.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.767+0000 7f174b47e700 1 -- start start 2023-10-11T09:52:14.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.767+0000 7f174b47e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1730004870 0x7f174415c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:14.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.767+0000 7f174b47e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f17440b2700 0x7f17440c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:14.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.767+0000 7f174b47e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17440afd30 0x7f17440b01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:14.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.767+0000 7f174b47e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17440b0710 con 0x7f17440afd30 2023-10-11T09:52:14.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.767+0000 7f174b47e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17440c91c0 con 0x7f1730004870 2023-10-11T09:52:14.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.767+0000 7f174b47e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17440c85b0 con 0x7f17440b2700 2023-10-11T09:52:14.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.771+0000 7f174ac7d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17440afd30 0x7f17440b01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:14.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.771+0000 7f174ac7d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17440afd30 0x7f17440b01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47566/0 (socket says 172.21.15.78:47566) 2023-10-11T09:52:14.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.771+0000 7f174ac7d700 1 -- 172.21.15.78:0/1995242408 learned_addr learned my addr 172.21.15.78:0/1995242408 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:14.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.771+0000 7f1749c7b700 1 --2- 172.21.15.78:0/1995242408 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f17440b2700 0x7f17440c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:14.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.771+0000 7f174a47c700 1 --2- 172.21.15.78:0/1995242408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1730004870 0x7f174415c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:14.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.771+0000 7f174ac7d700 1 -- 172.21.15.78:0/1995242408 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f17440b2700 msgr2=0x7f17440c7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:14.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.771+0000 7f174ac7d700 1 --2- 172.21.15.78:0/1995242408 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f17440b2700 0x7f17440c7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:14.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.771+0000 7f174ac7d700 1 -- 172.21.15.78:0/1995242408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1730004870 msgr2=0x7f174415c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:14.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.771+0000 7f174ac7d700 1 --2- 172.21.15.78:0/1995242408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1730004870 0x7f174415c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:14.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.771+0000 7f174ac7d700 1 -- 172.21.15.78:0/1995242408 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17440c8890 con 0x7f17440afd30 2023-10-11T09:52:14.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.771+0000 7f174a47c700 1 --2- 172.21.15.78:0/1995242408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1730004870 0x7f174415c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:14.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.771+0000 7f1749c7b700 1 --2- 172.21.15.78:0/1995242408 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f17440b2700 0x7f17440c7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:14.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.771+0000 7f174ac7d700 1 --2- 172.21.15.78:0/1995242408 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17440afd30 0x7f17440b01d0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f17440d41e0 tx=0x7f173c007c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:14.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.771+0000 7f173affd700 1 -- 172.21.15.78:0/1995242408 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c00a9b0 con 0x7f17440afd30 2023-10-11T09:52:14.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.771+0000 7f173affd700 1 -- 172.21.15.78:0/1995242408 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f173c00ab10 con 0x7f17440afd30 2023-10-11T09:52:14.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.771+0000 7f174b47e700 1 -- 172.21.15.78:0/1995242408 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17440c8b70 con 0x7f17440afd30 2023-10-11T09:52:14.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.771+0000 7f173affd700 1 -- 172.21.15.78:0/1995242408 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c0133f0 con 0x7f17440afd30 2023-10-11T09:52:14.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.771+0000 7f174b47e700 1 -- 172.21.15.78:0/1995242408 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17440c8180 con 0x7f17440afd30 2023-10-11T09:52:14.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.771+0000 7f174b47e700 1 -- 172.21.15.78:0/1995242408 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17440744c0 con 0x7f17440afd30 2023-10-11T09:52:14.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.775+0000 7f173affd700 1 -- 172.21.15.78:0/1995242408 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f173c013550 con 0x7f17440afd30 2023-10-11T09:52:14.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.775+0000 7f173affd700 1 --2- 172.21.15.78:0/1995242408 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1728065f00 0x7f17280683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:14.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.775+0000 7f173affd700 1 -- 172.21.15.78:0/1995242408 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f173c083dc0 con 0x7f17440afd30 2023-10-11T09:52:14.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.775+0000 7f174a47c700 1 --2- 172.21.15.78:0/1995242408 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1728065f00 0x7f17280683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:14.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.775+0000 7f174a47c700 1 --2- 172.21.15.78:0/1995242408 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1728065f00 0x7f17280683b0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f1740000c00 tx=0x7f1740007cd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:14.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.783+0000 7f173affd700 1 -- 172.21.15.78:0/1995242408 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f173c0513a0 con 0x7f17440afd30 2023-10-11T09:52:14.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.955+0000 7f174b47e700 1 -- 172.21.15.78:0/1995242408 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f17440c9300 con 0x7f17440afd30 2023-10-11T09:52:14.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.955+0000 7f173affd700 1 -- 172.21.15.78:0/1995242408 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v128) v1 ==== 92+0+10 (secure 0 0 0) 0x7f173c0549c0 con 0x7f17440afd30 2023-10-11T09:52:14.963 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:52:14.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.959+0000 7f174b47e700 1 -- 172.21.15.78:0/1995242408 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1728065f00 msgr2=0x7f17280683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:14.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.959+0000 7f174b47e700 1 --2- 172.21.15.78:0/1995242408 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1728065f00 0x7f17280683b0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f1740000c00 tx=0x7f1740007cd0 comp rx=0 tx=0).stop 2023-10-11T09:52:14.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.959+0000 7f174b47e700 1 -- 172.21.15.78:0/1995242408 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17440afd30 msgr2=0x7f17440b01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:14.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.959+0000 7f174b47e700 1 --2- 172.21.15.78:0/1995242408 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17440afd30 0x7f17440b01d0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f17440d41e0 tx=0x7f173c007c30 comp rx=0 tx=0).stop 2023-10-11T09:52:14.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.959+0000 7f174b47e700 1 -- 172.21.15.78:0/1995242408 shutdown_connections 2023-10-11T09:52:14.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.959+0000 7f174b47e700 1 --2- 172.21.15.78:0/1995242408 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1728065f00 0x7f17280683b0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:14.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.959+0000 7f174b47e700 1 --2- 172.21.15.78:0/1995242408 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17440afd30 0x7f17440b01d0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:14.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.959+0000 7f174b47e700 1 --2- 172.21.15.78:0/1995242408 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f17440b2700 0x7f17440c7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:14.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.959+0000 7f174b47e700 1 --2- 172.21.15.78:0/1995242408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1730004870 0x7f174415c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:14.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.959+0000 7f174b47e700 1 -- 172.21.15.78:0/1995242408 >> 172.21.15.78:0/1995242408 conn(0x7f174409c3d0 msgr2=0x7f17440d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:14.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.959+0000 7f174b47e700 1 -- 172.21.15.78:0/1995242408 shutdown_connections 2023-10-11T09:52:14.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:14.959+0000 7f174b47e700 1 -- 172.21.15.78:0/1995242408 wait complete. 2023-10-11T09:52:14.987 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T09:52:15.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.167+0000 7faddeb00700 1 Processor -- start 2023-10-11T09:52:15.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.167+0000 7faddeb00700 1 -- start start 2023-10-11T09:52:15.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.171+0000 7faddeb00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fadd8160de0 0x7fadd806c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:15.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.171+0000 7faddeb00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fadd806c6d0 0x7fadd80cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:15.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.171+0000 7faddeb00700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadd80cc680 con 0x7fadd80cc230 2023-10-11T09:52:15.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.171+0000 7faddeb00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadd806cb20 con 0x7fadd8160de0 2023-10-11T09:52:15.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.171+0000 7faddeb00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd81648a0 con 0x7fadd806c6d0 2023-10-11T09:52:15.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.171+0000 7fadde2ff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fadd80cc230 0x7fadd80cec00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:48052/0 (socket says 172.21.15.78:48052) 2023-10-11T09:52:15.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.171+0000 7fadde2ff700 1 -- 172.21.15.78:0/4090989255 learned_addr learned my addr 172.21.15.78:0/4090989255 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:15.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.171+0000 7faddd2fd700 1 --2- 172.21.15.78:0/4090989255 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fadd806c6d0 0x7fadd80cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:15.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.171+0000 7fadddafe700 1 --2- 172.21.15.78:0/4090989255 >> v2:172.21.15.133:3300/0 conn(0x7fadd8160de0 0x7fadd806c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:15.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.171+0000 7faddcafc700 1 -- 172.21.15.78:0/4090989255 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3523373186 0 0) 0x7fadd0001490 con 0x7fadd80cc230 2023-10-11T09:52:15.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.171+0000 7faddcafc700 1 -- 172.21.15.78:0/4090989255 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadc40035d0 con 0x7fadd80cc230 2023-10-11T09:52:15.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.171+0000 7faddd2fd700 1 -- 172.21.15.78:0/4090989255 >> v2:172.21.15.133:3300/0 conn(0x7fadd8160de0 msgr2=0x7fadd806c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:15.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.171+0000 7faddd2fd700 1 --2- 172.21.15.78:0/4090989255 >> v2:172.21.15.133:3300/0 conn(0x7fadd8160de0 0x7fadd806c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:15.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.171+0000 7faddd2fd700 1 -- 172.21.15.78:0/4090989255 >> v1:172.21.15.78:6789/0 conn(0x7fadd80cc230 legacy=0x7fadd80cec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:15.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.171+0000 7faddd2fd700 1 -- 172.21.15.78:0/4090989255 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd8164b80 con 0x7fadd806c6d0 2023-10-11T09:52:15.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.171+0000 7faddd2fd700 1 --2- 172.21.15.78:0/4090989255 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fadd806c6d0 0x7fadd80cbcf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fadc8009850 tx=0x7fadc8009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50532572c434053c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:15.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.171+0000 7faddcafc700 1 -- 172.21.15.78:0/4090989255 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc8011830 con 0x7fadd806c6d0 2023-10-11T09:52:15.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.171+0000 7faddcafc700 1 -- 172.21.15.78:0/4090989255 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc8011990 con 0x7fadd806c6d0 2023-10-11T09:52:15.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.171+0000 7faddcafc700 1 -- 172.21.15.78:0/4090989255 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc8011d40 con 0x7fadd806c6d0 2023-10-11T09:52:15.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.175+0000 7faddeb00700 1 -- 172.21.15.78:0/4090989255 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fadd806c6d0 msgr2=0x7fadd80cbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:15.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.175+0000 7faddeb00700 1 --2- 172.21.15.78:0/4090989255 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fadd806c6d0 0x7fadd80cbcf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fadc8009850 tx=0x7fadc8009b60 comp rx=0 tx=0).stop 2023-10-11T09:52:15.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.175+0000 7faddeb00700 1 -- 172.21.15.78:0/4090989255 shutdown_connections 2023-10-11T09:52:15.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.175+0000 7faddeb00700 1 --2- 172.21.15.78:0/4090989255 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fadd806c6d0 0x7fadd80cbcf0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:15.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.175+0000 7faddeb00700 1 --2- 172.21.15.78:0/4090989255 >> v2:172.21.15.133:3300/0 conn(0x7fadd8160de0 0x7fadd806c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:15.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.175+0000 7faddeb00700 1 -- 172.21.15.78:0/4090989255 >> 172.21.15.78:0/4090989255 conn(0x7fadd8073100 msgr2=0x7fadd80734d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:15.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.175+0000 7faddeb00700 1 -- 172.21.15.78:0/4090989255 shutdown_connections 2023-10-11T09:52:15.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.175+0000 7faddeb00700 1 -- 172.21.15.78:0/4090989255 wait complete. 2023-10-11T09:52:15.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.179+0000 7faddeb00700 1 Processor -- start 2023-10-11T09:52:15.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.179+0000 7faddeb00700 1 -- start start 2023-10-11T09:52:15.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.179+0000 7faddeb00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fadd806c6d0 0x7fadd814c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:15.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.179+0000 7faddeb00700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fadd80cc230 0x7fadd80afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:15.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.179+0000 7faddeb00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fadd8160de0 0x7fadd80b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:15.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.179+0000 7faddeb00700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd814cbe0 con 0x7fadd80cc230 2023-10-11T09:52:15.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.179+0000 7faddeb00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd80c5720 con 0x7fadd806c6d0 2023-10-11T09:52:15.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.179+0000 7faddeb00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd80c4ba0 con 0x7fadd8160de0 2023-10-11T09:52:15.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.179+0000 7faddd2fd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fadd80cc230 0x7fadd80afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:15.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.179+0000 7faddd2fd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fadd80cc230 0x7fadd80afdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47570/0 (socket says 172.21.15.78:47570) 2023-10-11T09:52:15.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.179+0000 7faddd2fd700 1 -- 172.21.15.78:0/2214074534 learned_addr learned my addr 172.21.15.78:0/2214074534 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:15.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.179+0000 7fadddafe700 1 --2- 172.21.15.78:0/2214074534 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fadd806c6d0 0x7fadd814c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:15.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.179+0000 7fadde2ff700 1 --2- 172.21.15.78:0/2214074534 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fadd8160de0 0x7fadd80b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:15.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.179+0000 7faddd2fd700 1 -- 172.21.15.78:0/2214074534 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fadd8160de0 msgr2=0x7fadd80b0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:15.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.179+0000 7faddd2fd700 1 --2- 172.21.15.78:0/2214074534 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fadd8160de0 0x7fadd80b0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:15.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.179+0000 7faddd2fd700 1 -- 172.21.15.78:0/2214074534 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fadd806c6d0 msgr2=0x7fadd814c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:15.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.179+0000 7faddd2fd700 1 --2- 172.21.15.78:0/2214074534 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fadd806c6d0 0x7fadd814c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:15.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.179+0000 7faddd2fd700 1 -- 172.21.15.78:0/2214074534 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc80094b0 con 0x7fadd80cc230 2023-10-11T09:52:15.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.183+0000 7faddd2fd700 1 --2- 172.21.15.78:0/2214074534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fadd80cc230 0x7fadd80afdf0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fadc80090b0 tx=0x7fadc8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:15.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.183+0000 7fadce7fc700 1 -- 172.21.15.78:0/2214074534 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc8004380 con 0x7fadd80cc230 2023-10-11T09:52:15.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.183+0000 7fadce7fc700 1 -- 172.21.15.78:0/2214074534 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc801f070 con 0x7fadd80cc230 2023-10-11T09:52:15.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.183+0000 7fadce7fc700 1 -- 172.21.15.78:0/2214074534 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc80036c0 con 0x7fadd80cc230 2023-10-11T09:52:15.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.183+0000 7faddeb00700 1 -- 172.21.15.78:0/2214074534 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadd80c4e80 con 0x7fadd80cc230 2023-10-11T09:52:15.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.183+0000 7faddeb00700 1 -- 172.21.15.78:0/2214074534 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd80c5440 con 0x7fadd80cc230 2023-10-11T09:52:15.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.183+0000 7faddeb00700 1 -- 172.21.15.78:0/2214074534 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadd80744c0 con 0x7fadd80cc230 2023-10-11T09:52:15.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.187+0000 7fadce7fc700 1 -- 172.21.15.78:0/2214074534 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fadc8021030 con 0x7fadd80cc230 2023-10-11T09:52:15.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.187+0000 7fadce7fc700 1 --2- 172.21.15.78:0/2214074534 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fadbc065ed0 0x7fadbc068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:15.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.187+0000 7fadce7fc700 1 -- 172.21.15.78:0/2214074534 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fadc8084ca0 con 0x7fadd80cc230 2023-10-11T09:52:15.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.187+0000 7fadddafe700 1 --2- 172.21.15.78:0/2214074534 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fadbc065ed0 0x7fadbc068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:15.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.187+0000 7fadddafe700 1 --2- 172.21.15.78:0/2214074534 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fadbc065ed0 0x7fadbc068380 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fadd4009b50 tx=0x7fadd4009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:15.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.191+0000 7fadce7fc700 1 -- 172.21.15.78:0/2214074534 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fadc80523a0 con 0x7fadd80cc230 2023-10-11T09:52:15.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.367+0000 7faddeb00700 1 -- 172.21.15.78:0/2214074534 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fadd814cf00 con 0x7fadd80cc230 2023-10-11T09:52:15.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.367+0000 7fadce7fc700 1 -- 172.21.15.78:0/2214074534 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v128) v1 ==== 104+0+11 (secure 0 0 0) 0x7fadc8006ce0 con 0x7fadd80cc230 2023-10-11T09:52:15.375 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:52:15.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.371+0000 7faddeb00700 1 -- 172.21.15.78:0/2214074534 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fadbc065ed0 msgr2=0x7fadbc068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:15.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.371+0000 7faddeb00700 1 --2- 172.21.15.78:0/2214074534 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fadbc065ed0 0x7fadbc068380 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fadd4009b50 tx=0x7fadd4009240 comp rx=0 tx=0).stop 2023-10-11T09:52:15.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.371+0000 7faddeb00700 1 -- 172.21.15.78:0/2214074534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fadd80cc230 msgr2=0x7fadd80afdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:15.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.371+0000 7faddeb00700 1 --2- 172.21.15.78:0/2214074534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fadd80cc230 0x7fadd80afdf0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fadc80090b0 tx=0x7fadc8004770 comp rx=0 tx=0).stop 2023-10-11T09:52:15.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.371+0000 7faddeb00700 1 -- 172.21.15.78:0/2214074534 shutdown_connections 2023-10-11T09:52:15.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.371+0000 7faddeb00700 1 --2- 172.21.15.78:0/2214074534 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fadbc065ed0 0x7fadbc068380 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:15.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.371+0000 7faddeb00700 1 --2- 172.21.15.78:0/2214074534 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fadd8160de0 0x7fadd80b0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:15.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.371+0000 7faddeb00700 1 --2- 172.21.15.78:0/2214074534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fadd80cc230 0x7fadd80afdf0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:15.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.371+0000 7faddeb00700 1 --2- 172.21.15.78:0/2214074534 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fadd806c6d0 0x7fadd814c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:15.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.371+0000 7faddeb00700 1 -- 172.21.15.78:0/2214074534 >> 172.21.15.78:0/2214074534 conn(0x7fadd8073100 msgr2=0x7fadd8067f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:15.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.371+0000 7faddeb00700 1 -- 172.21.15.78:0/2214074534 shutdown_connections 2023-10-11T09:52:15.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.371+0000 7faddeb00700 1 -- 172.21.15.78:0/2214074534 wait complete. 2023-10-11T09:52:15.399 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T09:52:15.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.579+0000 7fe23c9e6700 1 Processor -- start 2023-10-11T09:52:15.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.579+0000 7fe23c9e6700 1 -- start start 2023-10-11T09:52:15.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.583+0000 7fe23c9e6700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe2380718d0 0x7fe23815a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:15.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.583+0000 7fe23c9e6700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe23815f090 0x7fe23815aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:15.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.583+0000 7fe23c9e6700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe238072040 con 0x7fe2380caf80 2023-10-11T09:52:15.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.583+0000 7fe23c9e6700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe23815ec00 con 0x7fe23815f090 2023-10-11T09:52:15.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.583+0000 7fe23c9e6700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe238071d60 con 0x7fe2380718d0 2023-10-11T09:52:15.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.583+0000 7fe23659c700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fe2380caf80 0x7fe2380711c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:48064/0 (socket says 172.21.15.78:48064) 2023-10-11T09:52:15.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.583+0000 7fe23659c700 1 -- 172.21.15.78:0/2809976813 learned_addr learned my addr 172.21.15.78:0/2809976813 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:15.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.583+0000 7fe235d9b700 1 --2- 172.21.15.78:0/2809976813 >> v2:172.21.15.133:3300/0 conn(0x7fe2380718d0 0x7fe23815a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:15.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.583+0000 7fe236d9d700 1 --2- 172.21.15.78:0/2809976813 >> v2:172.21.15.78:3300/0 conn(0x7fe23815f090 0x7fe23815aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:15.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.583+0000 7fe23559a700 1 -- 172.21.15.78:0/2809976813 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1967279110 0 0) 0x7fe238072040 con 0x7fe2380caf80 2023-10-11T09:52:15.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.583+0000 7fe23559a700 1 -- 172.21.15.78:0/2809976813 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2200035d0 con 0x7fe2380caf80 2023-10-11T09:52:15.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.583+0000 7fe236d9d700 1 -- 172.21.15.78:0/2809976813 >> v2:172.21.15.133:3300/0 conn(0x7fe2380718d0 msgr2=0x7fe23815a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:15.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.583+0000 7fe236d9d700 1 --2- 172.21.15.78:0/2809976813 >> v2:172.21.15.133:3300/0 conn(0x7fe2380718d0 0x7fe23815a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:15.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.583+0000 7fe236d9d700 1 -- 172.21.15.78:0/2809976813 >> v1:172.21.15.78:6789/0 conn(0x7fe2380caf80 legacy=0x7fe2380711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:15.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.583+0000 7fe236d9d700 1 -- 172.21.15.78:0/2809976813 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe23815e5b0 con 0x7fe23815f090 2023-10-11T09:52:15.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.583+0000 7fe235d9b700 1 --2- 172.21.15.78:0/2809976813 >> v2:172.21.15.133:3300/0 conn(0x7fe2380718d0 0x7fe23815a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:52:15.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.583+0000 7fe236d9d700 1 --2- 172.21.15.78:0/2809976813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe23815f090 0x7fe23815aba0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fe2380c5f00 tx=0x7fe23000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55914b38957f0633 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:15.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.583+0000 7fe23559a700 1 -- 172.21.15.78:0/2809976813 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23000e070 con 0x7fe23815f090 2023-10-11T09:52:15.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.583+0000 7fe23559a700 1 -- 172.21.15.78:0/2809976813 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe23000bb20 con 0x7fe23815f090 2023-10-11T09:52:15.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.583+0000 7fe23559a700 1 -- 172.21.15.78:0/2809976813 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe230012430 con 0x7fe23815f090 2023-10-11T09:52:15.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.587+0000 7fe23c9e6700 1 -- 172.21.15.78:0/2809976813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe23815f090 msgr2=0x7fe23815aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:15.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.587+0000 7fe23c9e6700 1 --2- 172.21.15.78:0/2809976813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe23815f090 0x7fe23815aba0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fe2380c5f00 tx=0x7fe23000b330 comp rx=0 tx=0).stop 2023-10-11T09:52:15.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.587+0000 7fe23c9e6700 1 -- 172.21.15.78:0/2809976813 shutdown_connections 2023-10-11T09:52:15.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.587+0000 7fe23c9e6700 1 --2- 172.21.15.78:0/2809976813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe23815f090 0x7fe23815aba0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:15.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.587+0000 7fe23c9e6700 1 --2- 172.21.15.78:0/2809976813 >> v2:172.21.15.133:3300/0 conn(0x7fe2380718d0 0x7fe23815a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:15.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.587+0000 7fe23c9e6700 1 -- 172.21.15.78:0/2809976813 >> 172.21.15.78:0/2809976813 conn(0x7fe23806ae10 msgr2=0x7fe2380b0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:15.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.587+0000 7fe23c9e6700 1 -- 172.21.15.78:0/2809976813 shutdown_connections 2023-10-11T09:52:15.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.587+0000 7fe23c9e6700 1 -- 172.21.15.78:0/2809976813 wait complete. 2023-10-11T09:52:15.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.587+0000 7fe23c9e6700 1 Processor -- start 2023-10-11T09:52:15.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.587+0000 7fe23c9e6700 1 -- start start 2023-10-11T09:52:15.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.591+0000 7fe23c9e6700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2380718d0 0x7fe2380cca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:15.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.591+0000 7fe23c9e6700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2380caf80 0x7fe238151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:15.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.591+0000 7fe23c9e6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe23815f090 0x7fe23815c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:15.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.591+0000 7fe23c9e6700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2380cda40 con 0x7fe23815f090 2023-10-11T09:52:15.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.591+0000 7fe23c9e6700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe238152800 con 0x7fe2380718d0 2023-10-11T09:52:15.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.591+0000 7fe23c9e6700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe238153d50 con 0x7fe2380caf80 2023-10-11T09:52:15.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.591+0000 7fe236d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe23815f090 0x7fe23815c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:15.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.591+0000 7fe236d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe23815f090 0x7fe23815c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47590/0 (socket says 172.21.15.78:47590) 2023-10-11T09:52:15.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.591+0000 7fe236d9d700 1 -- 172.21.15.78:0/1842971277 learned_addr learned my addr 172.21.15.78:0/1842971277 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:15.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.591+0000 7fe23659c700 1 --2- 172.21.15.78:0/1842971277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2380718d0 0x7fe2380cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:15.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.591+0000 7fe235d9b700 1 --2- 172.21.15.78:0/1842971277 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2380caf80 0x7fe238151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:15.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.591+0000 7fe236d9d700 1 -- 172.21.15.78:0/1842971277 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2380caf80 msgr2=0x7fe238151340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:15.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.591+0000 7fe236d9d700 1 --2- 172.21.15.78:0/1842971277 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2380caf80 0x7fe238151340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:15.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.591+0000 7fe236d9d700 1 -- 172.21.15.78:0/1842971277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2380718d0 msgr2=0x7fe2380cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:15.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.591+0000 7fe236d9d700 1 --2- 172.21.15.78:0/1842971277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2380718d0 0x7fe2380cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:15.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.591+0000 7fe236d9d700 1 -- 172.21.15.78:0/1842971277 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22c0094b0 con 0x7fe23815f090 2023-10-11T09:52:15.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.591+0000 7fe23659c700 1 --2- 172.21.15.78:0/1842971277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2380718d0 0x7fe2380cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:15.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.591+0000 7fe236d9d700 1 --2- 172.21.15.78:0/1842971277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe23815f090 0x7fe23815c070 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fe230014040 tx=0x7fe230004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:15.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.591+0000 7fe226ffd700 1 -- 172.21.15.78:0/1842971277 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe230012430 con 0x7fe23815f090 2023-10-11T09:52:15.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.591+0000 7fe226ffd700 1 -- 172.21.15.78:0/1842971277 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe230020070 con 0x7fe23815f090 2023-10-11T09:52:15.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.591+0000 7fe226ffd700 1 -- 172.21.15.78:0/1842971277 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe230004280 con 0x7fe23815f090 2023-10-11T09:52:15.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.591+0000 7fe23c9e6700 1 -- 172.21.15.78:0/1842971277 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe230009a70 con 0x7fe23815f090 2023-10-11T09:52:15.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.591+0000 7fe23c9e6700 1 -- 172.21.15.78:0/1842971277 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe238152a50 con 0x7fe23815f090 2023-10-11T09:52:15.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.595+0000 7fe23c9e6700 1 -- 172.21.15.78:0/1842971277 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe23806f4f0 con 0x7fe23815f090 2023-10-11T09:52:15.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.595+0000 7fe226ffd700 1 -- 172.21.15.78:0/1842971277 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fe2300043e0 con 0x7fe23815f090 2023-10-11T09:52:15.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.599+0000 7fe226ffd700 1 --2- 172.21.15.78:0/1842971277 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe218065ed0 0x7fe218068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:15.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.599+0000 7fe226ffd700 1 -- 172.21.15.78:0/1842971277 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fe230022070 con 0x7fe23815f090 2023-10-11T09:52:15.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.599+0000 7fe23659c700 1 --2- 172.21.15.78:0/1842971277 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe218065ed0 0x7fe218068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:15.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.599+0000 7fe23659c700 1 --2- 172.21.15.78:0/1842971277 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe218065ed0 0x7fe218068380 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe2280011e0 tx=0x7fe228009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:15.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.603+0000 7fe226ffd700 1 -- 172.21.15.78:0/1842971277 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fe230052ae0 con 0x7fe23815f090 2023-10-11T09:52:15.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.775+0000 7fe23c9e6700 1 -- 172.21.15.78:0/1842971277 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe238074530 con 0x7fe23815f090 2023-10-11T09:52:15.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.775+0000 7fe226ffd700 1 -- 172.21.15.78:0/1842971277 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v128) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe23002c020 con 0x7fe23815f090 2023-10-11T09:52:15.783 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2023-10-11T09:52:15.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.779+0000 7fe23c9e6700 1 -- 172.21.15.78:0/1842971277 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe218065ed0 msgr2=0x7fe218068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:15.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.779+0000 7fe23c9e6700 1 --2- 172.21.15.78:0/1842971277 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe218065ed0 0x7fe218068380 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe2280011e0 tx=0x7fe228009040 comp rx=0 tx=0).stop 2023-10-11T09:52:15.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.779+0000 7fe23c9e6700 1 -- 172.21.15.78:0/1842971277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe23815f090 msgr2=0x7fe23815c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:15.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.779+0000 7fe23c9e6700 1 --2- 172.21.15.78:0/1842971277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe23815f090 0x7fe23815c070 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fe230014040 tx=0x7fe230004af0 comp rx=0 tx=0).stop 2023-10-11T09:52:15.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.779+0000 7fe23c9e6700 1 -- 172.21.15.78:0/1842971277 shutdown_connections 2023-10-11T09:52:15.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.779+0000 7fe23c9e6700 1 --2- 172.21.15.78:0/1842971277 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe218065ed0 0x7fe218068380 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:15.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.779+0000 7fe23c9e6700 1 --2- 172.21.15.78:0/1842971277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe23815f090 0x7fe23815c070 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:15.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.779+0000 7fe23c9e6700 1 --2- 172.21.15.78:0/1842971277 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2380caf80 0x7fe238151340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:15.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.779+0000 7fe23c9e6700 1 --2- 172.21.15.78:0/1842971277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2380718d0 0x7fe2380cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:15.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.779+0000 7fe23c9e6700 1 -- 172.21.15.78:0/1842971277 >> 172.21.15.78:0/1842971277 conn(0x7fe23806ae10 msgr2=0x7fe23815d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:15.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.779+0000 7fe23c9e6700 1 -- 172.21.15.78:0/1842971277 shutdown_connections 2023-10-11T09:52:15.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.779+0000 7fe23c9e6700 1 -- 172.21.15.78:0/1842971277 wait complete. 2023-10-11T09:52:15.806 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T09:52:15.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.979+0000 7f8288acf700 1 Processor -- start 2023-10-11T09:52:15.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.979+0000 7f8288acf700 1 -- start start 2023-10-11T09:52:15.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.979+0000 7f8288acf700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f8284070590 0x7f828406e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:15.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.983+0000 7f8288acf700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f828406c890 0x7f828406cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:15.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.983+0000 7f8288acf700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82840d07d0 con 0x7f82840cff60 2023-10-11T09:52:15.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.983+0000 7f8288acf700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f828415d030 con 0x7f8284070590 2023-10-11T09:52:15.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.983+0000 7f8288acf700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f828406d250 con 0x7f828406c890 2023-10-11T09:52:15.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.983+0000 7f828259c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f8284070590 0x7f828406e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:15.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.983+0000 7f828259c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f8284070590 0x7f828406e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47604/0 (socket says 172.21.15.78:47604) 2023-10-11T09:52:15.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.983+0000 7f828259c700 1 -- 172.21.15.78:0/1972743735 learned_addr learned my addr 172.21.15.78:0/1972743735 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:15.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.983+0000 7f8282d9d700 1 --2- 172.21.15.78:0/1972743735 >> v2:172.21.15.133:3300/0 conn(0x7f828406c890 0x7f828406cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:15.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.983+0000 7f828259c700 1 -- 172.21.15.78:0/1972743735 >> v2:172.21.15.133:3300/0 conn(0x7f828406c890 msgr2=0x7f828406cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:15.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.983+0000 7f828259c700 1 --2- 172.21.15.78:0/1972743735 >> v2:172.21.15.133:3300/0 conn(0x7f828406c890 0x7f828406cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:15.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.983+0000 7f828259c700 1 -- 172.21.15.78:0/1972743735 >> v1:172.21.15.133:6789/0 conn(0x7f82840cff60 legacy=0x7f8284160f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:15.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.983+0000 7f828259c700 1 -- 172.21.15.78:0/1972743735 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82840d0ab0 con 0x7f8284070590 2023-10-11T09:52:15.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.983+0000 7f828259c700 1 --2- 172.21.15.78:0/1972743735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8284070590 0x7f828406e480 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f8284071c80 tx=0x7f8274009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf2bb0a3fe33d7d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:15.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.983+0000 7f828159a700 1 -- 172.21.15.78:0/1972743735 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827400b8a0 con 0x7f8284070590 2023-10-11T09:52:15.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.983+0000 7f828159a700 1 -- 172.21.15.78:0/1972743735 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827400ba00 con 0x7f8284070590 2023-10-11T09:52:15.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.983+0000 7f828159a700 1 -- 172.21.15.78:0/1972743735 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827400bd90 con 0x7f8284070590 2023-10-11T09:52:15.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.983+0000 7f8288acf700 1 -- 172.21.15.78:0/1972743735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8284070590 msgr2=0x7f828406e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:15.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.983+0000 7f8288acf700 1 --2- 172.21.15.78:0/1972743735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8284070590 0x7f828406e480 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f8284071c80 tx=0x7f8274009bf0 comp rx=0 tx=0).stop 2023-10-11T09:52:15.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.983+0000 7f8288acf700 1 -- 172.21.15.78:0/1972743735 shutdown_connections 2023-10-11T09:52:15.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.983+0000 7f8288acf700 1 --2- 172.21.15.78:0/1972743735 >> v2:172.21.15.133:3300/0 conn(0x7f828406c890 0x7f828406cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:15.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.983+0000 7f8288acf700 1 --2- 172.21.15.78:0/1972743735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8284070590 0x7f828406e480 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:15.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.987+0000 7f8288acf700 1 -- 172.21.15.78:0/1972743735 >> 172.21.15.78:0/1972743735 conn(0x7f82840d1000 msgr2=0x7f82840d1400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:15.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.987+0000 7f8288acf700 1 -- 172.21.15.78:0/1972743735 shutdown_connections 2023-10-11T09:52:15.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.987+0000 7f8288acf700 1 -- 172.21.15.78:0/1972743735 wait complete. 2023-10-11T09:52:15.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.987+0000 7f8288acf700 1 Processor -- start 2023-10-11T09:52:15.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.987+0000 7f8288acf700 1 -- start start 2023-10-11T09:52:15.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.991+0000 7f8288acf700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f828406c890 0x7f82840ac850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:15.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.991+0000 7f8288acf700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8284070590 0x7f82841f2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:15.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.991+0000 7f8288acf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82840cff60 0x7f8284149b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:15.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.991+0000 7f8288acf700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8284150870 con 0x7f82840cff60 2023-10-11T09:52:15.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.991+0000 7f8288acf700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82840c6aa0 con 0x7f828406c890 2023-10-11T09:52:15.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.991+0000 7f8288acf700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82840c6d80 con 0x7f8284070590 2023-10-11T09:52:15.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.991+0000 7f8282d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82840cff60 0x7f8284149b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:16.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.991+0000 7f8282d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82840cff60 0x7f8284149b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47606/0 (socket says 172.21.15.78:47606) 2023-10-11T09:52:16.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.991+0000 7f8282d9d700 1 -- 172.21.15.78:0/2488565079 learned_addr learned my addr 172.21.15.78:0/2488565079 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:16.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.991+0000 7f828259c700 1 --2- 172.21.15.78:0/2488565079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f828406c890 0x7f82840ac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:16.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.991+0000 7f8281d9b700 1 --2- 172.21.15.78:0/2488565079 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8284070590 0x7f82841f2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:16.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.991+0000 7f8282d9d700 1 -- 172.21.15.78:0/2488565079 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8284070590 msgr2=0x7f82841f2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:16.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.991+0000 7f8282d9d700 1 --2- 172.21.15.78:0/2488565079 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8284070590 0x7f82841f2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:16.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.991+0000 7f8282d9d700 1 -- 172.21.15.78:0/2488565079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f828406c890 msgr2=0x7f82840ac850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:16.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.991+0000 7f8282d9d700 1 --2- 172.21.15.78:0/2488565079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f828406c890 0x7f82840ac850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:16.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.991+0000 7f8282d9d700 1 -- 172.21.15.78:0/2488565079 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8274009570 con 0x7f82840cff60 2023-10-11T09:52:16.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.991+0000 7f8282d9d700 1 --2- 172.21.15.78:0/2488565079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82840cff60 0x7f8284149b40 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f827c007e40 tx=0x7f827c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:16.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.991+0000 7f8272ffd700 1 -- 172.21.15.78:0/2488565079 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c00ede0 con 0x7f82840cff60 2023-10-11T09:52:16.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.991+0000 7f8272ffd700 1 -- 172.21.15.78:0/2488565079 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827c014420 con 0x7f82840cff60 2023-10-11T09:52:16.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.991+0000 7f8288acf700 1 -- 172.21.15.78:0/2488565079 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82840c7060 con 0x7f82840cff60 2023-10-11T09:52:16.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.991+0000 7f8272ffd700 1 -- 172.21.15.78:0/2488565079 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c0147d0 con 0x7f82840cff60 2023-10-11T09:52:16.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.991+0000 7f8288acf700 1 -- 172.21.15.78:0/2488565079 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f828414a930 con 0x7f82840cff60 2023-10-11T09:52:16.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.995+0000 7f8288acf700 1 -- 172.21.15.78:0/2488565079 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82840744c0 con 0x7f82840cff60 2023-10-11T09:52:16.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.995+0000 7f8272ffd700 1 -- 172.21.15.78:0/2488565079 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f827c014930 con 0x7f82840cff60 2023-10-11T09:52:16.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.995+0000 7f8272ffd700 1 --2- 172.21.15.78:0/2488565079 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8264065ed0 0x7f8264068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:16.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.995+0000 7f8272ffd700 1 -- 172.21.15.78:0/2488565079 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f827c083fe0 con 0x7f82840cff60 2023-10-11T09:52:16.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.995+0000 7f828259c700 1 --2- 172.21.15.78:0/2488565079 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8264065ed0 0x7f8264068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:16.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:15.995+0000 7f828259c700 1 --2- 172.21.15.78:0/2488565079 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8264065ed0 0x7f8264068380 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f8274000c00 tx=0x7f8274008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:16.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:16.003+0000 7f8272ffd700 1 -- 172.21.15.78:0/2488565079 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f827c0515e0 con 0x7f82840cff60 2023-10-11T09:52:16.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:16.187+0000 7f8288acf700 1 -- 172.21.15.78:0/2488565079 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f82841601b0 con 0x7f82840cff60 2023-10-11T09:52:16.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:16.187+0000 7f8272ffd700 1 -- 172.21.15.78:0/2488565079 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v128) v1 ==== 80+0+124 (secure 0 0 0) 0x7f827c054c00 con 0x7f82840cff60 2023-10-11T09:52:16.194 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.133:6833/1433079498 2023-10-12T09:52:02.877755+0000 2023-10-11T09:52:16.194 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.133:6832/1433079498 2023-10-12T09:52:02.877755+0000 2023-10-11T09:52:16.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:16.191+0000 7f8288acf700 1 -- 172.21.15.78:0/2488565079 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8264065ed0 msgr2=0x7f8264068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:16.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:16.191+0000 7f8288acf700 1 --2- 172.21.15.78:0/2488565079 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8264065ed0 0x7f8264068380 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f8274000c00 tx=0x7f8274008030 comp rx=0 tx=0).stop 2023-10-11T09:52:16.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:16.191+0000 7f8288acf700 1 -- 172.21.15.78:0/2488565079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82840cff60 msgr2=0x7f8284149b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:16.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:16.191+0000 7f8288acf700 1 --2- 172.21.15.78:0/2488565079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82840cff60 0x7f8284149b40 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f827c007e40 tx=0x7f827c00e3c0 comp rx=0 tx=0).stop 2023-10-11T09:52:16.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:16.191+0000 7f8288acf700 1 -- 172.21.15.78:0/2488565079 shutdown_connections 2023-10-11T09:52:16.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:16.191+0000 7f8288acf700 1 --2- 172.21.15.78:0/2488565079 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8264065ed0 0x7f8264068380 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:16.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:16.191+0000 7f8288acf700 1 --2- 172.21.15.78:0/2488565079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82840cff60 0x7f8284149b40 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:16.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:16.191+0000 7f8288acf700 1 --2- 172.21.15.78:0/2488565079 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8284070590 0x7f82841f2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:16.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:16.191+0000 7f8288acf700 1 --2- 172.21.15.78:0/2488565079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f828406c890 0x7f82840ac850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:16.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:16.191+0000 7f8288acf700 1 -- 172.21.15.78:0/2488565079 >> 172.21.15.78:0/2488565079 conn(0x7f82840d1000 msgr2=0x7f82840d13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:16.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:16.191+0000 7f8288acf700 1 -- 172.21.15.78:0/2488565079 shutdown_connections 2023-10-11T09:52:16.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:16.191+0000 7f8288acf700 1 -- 172.21.15.78:0/2488565079 wait complete. 2023-10-11T09:52:16.201 INFO:teuthology.orchestra.run.smithi078.stderr:listed 2 entries 2023-10-11T09:52:16.220 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi078.front.sepia.ceph.com... 2023-10-11T09:52:16.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:16.221 DEBUG:teuthology.orchestra.run.smithi078:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-11T09:52:16.250 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:ceph-fuse[60247]: fuse finished with error 0 and tester_r 0 2023-10-11T09:52:16.254 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:52:22.356 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2023-10-11T09:52:22.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:22.357 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-11T09:52:22.380 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-11T09:52:22.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.595+0000 7f1975434700 1 Processor -- start 2023-10-11T09:52:22.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.595+0000 7f1975434700 1 -- start start 2023-10-11T09:52:22.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.599+0000 7f1975434700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f19700afbe0 0x7f197006ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:22.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.599+0000 7f1975434700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f197006f450 con 0x7f19700cd900 2023-10-11T09:52:22.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.599+0000 7f1975434700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f197006a210 con 0x7f19700af810 2023-10-11T09:52:22.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.599+0000 7f1975434700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f197006d340 con 0x7f19700afbe0 2023-10-11T09:52:22.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.599+0000 7f196ffff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f19700cd900 0x7f19700c3640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:48068/0 (socket says 172.21.15.78:48068) 2023-10-11T09:52:22.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.599+0000 7f196ffff700 1 -- 172.21.15.78:0/3446479794 learned_addr learned my addr 172.21.15.78:0/3446479794 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:22.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.599+0000 7f196e7fc700 1 --2- 172.21.15.78:0/3446479794 >> v2:172.21.15.133:3300/0 conn(0x7f19700afbe0 0x7f197006ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:22.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.599+0000 7f196dffb700 1 -- 172.21.15.78:0/3446479794 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2071765268 0 0) 0x7f197006f450 con 0x7f19700cd900 2023-10-11T09:52:22.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.599+0000 7f196dffb700 1 -- 172.21.15.78:0/3446479794 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19580035d0 con 0x7f19700cd900 2023-10-11T09:52:22.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.603+0000 7f196dffb700 1 -- 172.21.15.78:0/3446479794 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3784999466 0 0) 0x7f197006a210 con 0x7f19700af810 2023-10-11T09:52:22.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.603+0000 7f196dffb700 1 -- 172.21.15.78:0/3446479794 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f197006f450 con 0x7f19700af810 2023-10-11T09:52:22.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.603+0000 7f196e7fc700 1 -- 172.21.15.78:0/3446479794 >> v1:172.21.15.133:6789/0 conn(0x7f19700af810 legacy=0x7f19700cacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:22.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.603+0000 7f196dffb700 1 -- 172.21.15.78:0/3446479794 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3787002167 0 0) 0x7f19580035d0 con 0x7f19700cd900 2023-10-11T09:52:22.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.603+0000 7f196e7fc700 1 -- 172.21.15.78:0/3446479794 >> v1:172.21.15.78:6789/0 conn(0x7f19700cd900 legacy=0x7f19700c3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:22.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.603+0000 7f196e7fc700 1 -- 172.21.15.78:0/3446479794 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1970071aa0 con 0x7f19700afbe0 2023-10-11T09:52:22.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.603+0000 7f196e7fc700 1 --2- 172.21.15.78:0/3446479794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19700afbe0 0x7f197006ef10 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1964009850 tx=0x7f1964009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f98e485cf22c5400 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:22.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.603+0000 7f196dffb700 1 -- 172.21.15.78:0/3446479794 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19680019f0 con 0x7f19700afbe0 2023-10-11T09:52:22.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.603+0000 7f196dffb700 1 -- 172.21.15.78:0/3446479794 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1964004030 con 0x7f19700afbe0 2023-10-11T09:52:22.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.603+0000 7f196dffb700 1 -- 172.21.15.78:0/3446479794 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19640043c0 con 0x7f19700afbe0 2023-10-11T09:52:22.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.603+0000 7f1975434700 1 -- 172.21.15.78:0/3446479794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19700afbe0 msgr2=0x7f197006ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:22.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.603+0000 7f1975434700 1 --2- 172.21.15.78:0/3446479794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19700afbe0 0x7f197006ef10 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1964009850 tx=0x7f1964009b60 comp rx=0 tx=0).stop 2023-10-11T09:52:22.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.603+0000 7f1975434700 1 -- 172.21.15.78:0/3446479794 shutdown_connections 2023-10-11T09:52:22.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.603+0000 7f1975434700 1 --2- 172.21.15.78:0/3446479794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19700afbe0 0x7f197006ef10 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:22.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.603+0000 7f1975434700 1 -- 172.21.15.78:0/3446479794 >> 172.21.15.78:0/3446479794 conn(0x7f197006c5a0 msgr2=0x7f19700d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:22.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.603+0000 7f1975434700 1 -- 172.21.15.78:0/3446479794 shutdown_connections 2023-10-11T09:52:22.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.603+0000 7f1975434700 1 -- 172.21.15.78:0/3446479794 wait complete. 2023-10-11T09:52:22.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.607+0000 7f1975434700 1 Processor -- start 2023-10-11T09:52:22.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.607+0000 7f1975434700 1 -- start start 2023-10-11T09:52:22.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.607+0000 7f1975434700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19700af810 0x7f19700c7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:22.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.607+0000 7f1975434700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f19700afbe0 0x7f19700c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:22.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.607+0000 7f1975434700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19700cd900 0x7f1970162020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:22.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.607+0000 7f1975434700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19700ca400 con 0x7f19700af810 2023-10-11T09:52:22.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.607+0000 7f1975434700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1970162560 con 0x7f19700cd900 2023-10-11T09:52:22.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.607+0000 7f1975434700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19700c7dd0 con 0x7f19700afbe0 2023-10-11T09:52:22.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.607+0000 7f196effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19700af810 0x7f19700c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:22.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.607+0000 7f196effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19700af810 0x7f19700c7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47620/0 (socket says 172.21.15.78:47620) 2023-10-11T09:52:22.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.607+0000 7f196effd700 1 -- 172.21.15.78:0/1969727362 learned_addr learned my addr 172.21.15.78:0/1969727362 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:22.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.607+0000 7f196ffff700 1 --2- 172.21.15.78:0/1969727362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19700cd900 0x7f1970162020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:22.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.607+0000 7f196e7fc700 1 --2- 172.21.15.78:0/1969727362 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f19700afbe0 0x7f19700c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:22.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.607+0000 7f196effd700 1 -- 172.21.15.78:0/1969727362 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f19700afbe0 msgr2=0x7f19700c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:22.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.607+0000 7f196effd700 1 --2- 172.21.15.78:0/1969727362 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f19700afbe0 0x7f19700c9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:22.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.607+0000 7f196effd700 1 -- 172.21.15.78:0/1969727362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19700cd900 msgr2=0x7f1970162020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:22.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.607+0000 7f196effd700 1 --2- 172.21.15.78:0/1969727362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19700cd900 0x7f1970162020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:22.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.607+0000 7f196effd700 1 -- 172.21.15.78:0/1969727362 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19640094b0 con 0x7f19700af810 2023-10-11T09:52:22.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.607+0000 7f196effd700 1 --2- 172.21.15.78:0/1969727362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19700af810 0x7f19700c7740 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f196000c490 tx=0x7f196000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:22.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.611+0000 7f19577fe700 1 -- 172.21.15.78:0/1969727362 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19600052d0 con 0x7f19700af810 2023-10-11T09:52:22.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.611+0000 7f19577fe700 1 -- 172.21.15.78:0/1969727362 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1960005450 con 0x7f19700af810 2023-10-11T09:52:22.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.611+0000 7f1975434700 1 -- 172.21.15.78:0/1969727362 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19700c80b0 con 0x7f19700af810 2023-10-11T09:52:22.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.611+0000 7f19577fe700 1 -- 172.21.15.78:0/1969727362 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19600038e0 con 0x7f19700af810 2023-10-11T09:52:22.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.611+0000 7f1975434700 1 -- 172.21.15.78:0/1969727362 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19700c8670 con 0x7f19700af810 2023-10-11T09:52:22.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.611+0000 7f1975434700 1 -- 172.21.15.78:0/1969727362 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f197006df90 con 0x7f19700af810 2023-10-11T09:52:22.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.611+0000 7f19577fe700 1 -- 172.21.15.78:0/1969727362 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f1960003a40 con 0x7f19700af810 2023-10-11T09:52:22.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.615+0000 7f19577fe700 1 --2- 172.21.15.78:0/1969727362 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f195c065f00 0x7f195c0683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:22.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.615+0000 7f19577fe700 1 -- 172.21.15.78:0/1969727362 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f1960082de0 con 0x7f19700af810 2023-10-11T09:52:22.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.615+0000 7f196e7fc700 1 --2- 172.21.15.78:0/1969727362 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f195c065f00 0x7f195c0683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:22.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.615+0000 7f196e7fc700 1 --2- 172.21.15.78:0/1969727362 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f195c065f00 0x7f195c0683b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f197006b5e0 tx=0x7f1964008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:22.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.619+0000 7f19577fe700 1 -- 172.21.15.78:0/1969727362 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f19600502b0 con 0x7f19700af810 2023-10-11T09:52:22.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.831+0000 7f1975434700 1 -- 172.21.15.78:0/1969727362 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f19700c7480 con 0x7f19700af810 2023-10-11T09:52:22.841 INFO:tasks.ceph.mon.a.smithi078.stderr:2023-10-11T09:52:22.835+0000 7f90131f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-11T09:52:22.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.851+0000 7f19577fe700 1 -- 172.21.15.78:0/1969727362 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v106) v1 ==== 161+0+0 (secure 0 0 0) 0x7f19600538d0 con 0x7f19700af810 2023-10-11T09:52:22.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.855+0000 7f1975434700 1 -- 172.21.15.78:0/1969727362 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f195c065f00 msgr2=0x7f195c0683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:22.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.855+0000 7f1975434700 1 --2- 172.21.15.78:0/1969727362 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f195c065f00 0x7f195c0683b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f197006b5e0 tx=0x7f1964008030 comp rx=0 tx=0).stop 2023-10-11T09:52:22.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.855+0000 7f1975434700 1 -- 172.21.15.78:0/1969727362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19700af810 msgr2=0x7f19700c7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:22.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.855+0000 7f1975434700 1 --2- 172.21.15.78:0/1969727362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19700af810 0x7f19700c7740 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f196000c490 tx=0x7f196000c850 comp rx=0 tx=0).stop 2023-10-11T09:52:22.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.855+0000 7f1975434700 1 -- 172.21.15.78:0/1969727362 shutdown_connections 2023-10-11T09:52:22.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.855+0000 7f1975434700 1 --2- 172.21.15.78:0/1969727362 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f195c065f00 0x7f195c0683b0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:22.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.855+0000 7f1975434700 1 --2- 172.21.15.78:0/1969727362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19700cd900 0x7f1970162020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:22.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.855+0000 7f1975434700 1 --2- 172.21.15.78:0/1969727362 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f19700afbe0 0x7f19700c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:22.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.855+0000 7f1975434700 1 --2- 172.21.15.78:0/1969727362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19700af810 0x7f19700c7740 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:22.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.855+0000 7f1975434700 1 -- 172.21.15.78:0/1969727362 >> 172.21.15.78:0/1969727362 conn(0x7f197006c5a0 msgr2=0x7f19700affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:22.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.855+0000 7f1975434700 1 -- 172.21.15.78:0/1969727362 shutdown_connections 2023-10-11T09:52:22.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.855+0000 7f1975434700 1 -- 172.21.15.78:0/1969727362 wait complete. 2023-10-11T09:52:22.864 INFO:teuthology.orchestra.run.smithi078.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-11T09:52:22.870 INFO:tasks.ceph.mds.c.smithi078.stderr: -3095> 2023-10-11T09:52:10.719+0000 7f3086953700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-11T09:52:22.883 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents 2023-10-11T09:52:22.932 INFO:tasks.ceph.mds.c.smithi078.stdout:starting mds.c at 2023-10-11T09:52:22.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.935+0000 7f776aadc840 1 -- start start 2023-10-11T09:52:22.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.935+0000 7f776aadc840 1 Processor -- start 2023-10-11T09:52:22.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.935+0000 7f776aadc840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c4bba13800 0x55c4bbab8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:22.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.935+0000 7f776aadc840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4bba13400 0x55c4bbab8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:22.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.935+0000 7f776aadc840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c4bba13000 0x55c4bbab9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:22.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.935+0000 7f776aadc840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4bba92a00 con 0x55c4bba13400 2023-10-11T09:52:22.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.935+0000 7f776aadc840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4bba92780 con 0x55c4bba13000 2023-10-11T09:52:22.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.935+0000 7f776aadc840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c4bba92640 con 0x55c4bba13800 2023-10-11T09:52:22.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.935+0000 7f7768a97700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4bba13400 0x55c4bbab8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:22.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.935+0000 7f7768a97700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4bba13400 0x55c4bbab8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47666/0 (socket says 172.21.15.78:47666) 2023-10-11T09:52:22.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.935+0000 7f7768a97700 1 -- 172.21.15.78:0/2866718319 learned_addr learned my addr 172.21.15.78:0/2866718319 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:22.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.935+0000 7f7769298700 1 --2- 172.21.15.78:0/2866718319 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c4bba13800 0x55c4bbab8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:22.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.935+0000 7f7769a99700 1 --2- 172.21.15.78:0/2866718319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c4bba13000 0x55c4bbab9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:22.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.935+0000 7f7768a97700 1 -- 172.21.15.78:0/2866718319 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c4bba13800 msgr2=0x55c4bbab8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:22.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.935+0000 7f7768a97700 1 --2- 172.21.15.78:0/2866718319 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c4bba13800 0x55c4bbab8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:22.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.935+0000 7f7768a97700 1 -- 172.21.15.78:0/2866718319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c4bba13000 msgr2=0x55c4bbab9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:22.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.935+0000 7f7768a97700 1 --2- 172.21.15.78:0/2866718319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c4bba13000 0x55c4bbab9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:22.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.935+0000 7f7768a97700 1 -- 172.21.15.78:0/2866718319 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4bba34b60 con 0x55c4bba13400 2023-10-11T09:52:22.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.935+0000 7f7769a99700 1 --2- 172.21.15.78:0/2866718319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c4bba13000 0x55c4bbab9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:52:22.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.939+0000 7f7768a97700 1 --2- 172.21.15.78:0/2866718319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4bba13400 0x55c4bbab8b00 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x55c4bc77b410 tx=0x55c4bbb08f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f2357975636764 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:22.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.939+0000 7f7766a93700 1 -- 172.21.15.78:0/2866718319 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c4bba43600 con 0x55c4bba13400 2023-10-11T09:52:22.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.939+0000 7f7766a93700 1 -- 172.21.15.78:0/2866718319 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c4bba5e480 con 0x55c4bba13400 2023-10-11T09:52:22.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.939+0000 7f7766a93700 1 -- 172.21.15.78:0/2866718319 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c4bba431e0 con 0x55c4bba13400 2023-10-11T09:52:22.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.939+0000 7f776aadc840 1 -- 172.21.15.78:0/2866718319 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4bba34340 con 0x55c4bba13400 2023-10-11T09:52:22.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.939+0000 7f776aadc840 1 -- 172.21.15.78:0/2866718319 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55c4bba34000 con 0x55c4bba13400 2023-10-11T09:52:22.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.939+0000 7f776aadc840 4 waiting for MDS map... 2023-10-11T09:52:22.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.939+0000 7f7766a93700 1 -- 172.21.15.78:0/2866718319 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(129..129 src has 1..129) v4 ==== 6234+0+0 (secure 0 0 0) 0x55c4bba4cc40 con 0x55c4bba13400 2023-10-11T09:52:22.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.939+0000 7f7766a93700 1 -- 172.21.15.78:0/2866718319 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 106) v1 ==== 2565+0+0 (secure 0 0 0) 0x55c4bc88c300 con 0x55c4bba13400 2023-10-11T09:52:22.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.939+0000 7f776aadc840 4 Got MDS map 106 2023-10-11T09:52:22.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.939+0000 7f776aadc840 4 datascan.main: Using metadata pool output 2023-10-11T09:52:22.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.939+0000 7f776aadc840 4 datascan.main: connecting to RADOS... 2023-10-11T09:52:22.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.939+0000 7f776aadc840 1 Processor -- start 2023-10-11T09:52:22.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.939+0000 7f776aadc840 1 -- start start 2023-10-11T09:52:22.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.943+0000 7f776aadc840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c4bbcfe400 0x55c4bbab9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:22.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.943+0000 7f776aadc840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c4bbcfe800 0x55c4bbab9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:22.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.943+0000 7f776aadc840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4bbcfec00 0x55c4bbaba100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:22.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.943+0000 7f776aadc840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4bba49e00 con 0x55c4bbcfec00 2023-10-11T09:52:22.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.943+0000 7f776aadc840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4bba92780 con 0x55c4bbcfe800 2023-10-11T09:52:22.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.943+0000 7f776aadc840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c4bba92500 con 0x55c4bbcfe400 2023-10-11T09:52:22.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.943+0000 7f7768a97700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4bbcfec00 0x55c4bbaba100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:22.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.943+0000 7f7768a97700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4bbcfec00 0x55c4bbaba100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47674/0 (socket says 172.21.15.78:47674) 2023-10-11T09:52:22.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.943+0000 7f7768a97700 1 -- 172.21.15.78:0/3776866477 learned_addr learned my addr 172.21.15.78:0/3776866477 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:22.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.943+0000 7f7769a99700 1 --2- 172.21.15.78:0/3776866477 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c4bbcfe400 0x55c4bbab9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:22.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.943+0000 7f7769298700 1 --2- 172.21.15.78:0/3776866477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c4bbcfe800 0x55c4bbab9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:22.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.943+0000 7f7768a97700 1 -- 172.21.15.78:0/3776866477 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c4bbcfe400 msgr2=0x55c4bbab9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:22.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.943+0000 7f7768a97700 1 --2- 172.21.15.78:0/3776866477 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c4bbcfe400 0x55c4bbab9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:22.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.943+0000 7f7768a97700 1 -- 172.21.15.78:0/3776866477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c4bbcfe800 msgr2=0x55c4bbab9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:22.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.943+0000 7f7768a97700 1 --2- 172.21.15.78:0/3776866477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c4bbcfe800 0x55c4bbab9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:22.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.943+0000 7f7768a97700 1 -- 172.21.15.78:0/3776866477 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4bba34ea0 con 0x55c4bbcfec00 2023-10-11T09:52:22.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.943+0000 7f7768a97700 1 --2- 172.21.15.78:0/3776866477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4bbcfec00 0x55c4bbaba100 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x55c4bc77b800 tx=0x55c4bbb08e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9eb54fc8ab77ce server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:22.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.943+0000 7f7764a8f700 1 -- 172.21.15.78:0/3776866477 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c4bba42f20 con 0x55c4bbcfec00 2023-10-11T09:52:22.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.943+0000 7f7764a8f700 1 -- 172.21.15.78:0/3776866477 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c4bc7fca80 con 0x55c4bbcfec00 2023-10-11T09:52:22.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.943+0000 7f7764a8f700 1 -- 172.21.15.78:0/3776866477 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c4bba43600 con 0x55c4bbcfec00 2023-10-11T09:52:22.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.943+0000 7f776aadc840 1 -- 172.21.15.78:0/3776866477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4bbcfec00 msgr2=0x55c4bbaba100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:22.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.943+0000 7f776aadc840 1 --2- 172.21.15.78:0/3776866477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4bbcfec00 0x55c4bbaba100 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x55c4bc77b800 tx=0x55c4bbb08e40 comp rx=0 tx=0).stop 2023-10-11T09:52:22.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.943+0000 7f776aadc840 1 -- 172.21.15.78:0/3776866477 shutdown_connections 2023-10-11T09:52:22.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.943+0000 7f776aadc840 1 --2- 172.21.15.78:0/3776866477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4bbcfec00 0x55c4bbaba100 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:22.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.943+0000 7f776aadc840 1 --2- 172.21.15.78:0/3776866477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c4bbcfe800 0x55c4bbab9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:22.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.943+0000 7f776aadc840 1 --2- 172.21.15.78:0/3776866477 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c4bbcfe400 0x55c4bbab9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:22.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.943+0000 7f776aadc840 1 -- 172.21.15.78:0/3776866477 >> 172.21.15.78:0/3776866477 conn(0x55c4bba12c00 msgr2=0x55c4bbae1800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:22.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.943+0000 7f776aadc840 1 -- 172.21.15.78:0/3776866477 shutdown_connections 2023-10-11T09:52:22.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.943+0000 7f776aadc840 1 -- 172.21.15.78:0/3776866477 wait complete. 2023-10-11T09:52:22.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.943+0000 7f776aadc840 1 Processor -- start 2023-10-11T09:52:22.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.947+0000 7f776aadc840 1 -- start start 2023-10-11T09:52:22.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.947+0000 7f776aadc840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c4bbcfe400 0x55c4bbabb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:22.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.947+0000 7f776aadc840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c4bbcfe800 0x55c4bbabac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:22.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.947+0000 7f776aadc840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4bbcfec00 0x55c4bbaba680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:22.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.947+0000 7f776aadc840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4bba923c0 con 0x55c4bbcfec00 2023-10-11T09:52:22.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.947+0000 7f776aadc840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4bba92280 con 0x55c4bbcfe800 2023-10-11T09:52:22.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.947+0000 7f776aadc840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c4bba92140 con 0x55c4bbcfe400 2023-10-11T09:52:22.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.947+0000 7f7768a97700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4bbcfec00 0x55c4bbaba680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:22.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.947+0000 7f7768a97700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4bbcfec00 0x55c4bbaba680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47684/0 (socket says 172.21.15.78:47684) 2023-10-11T09:52:22.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.947+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 learned_addr learned my addr 172.21.15.78:0/2472870157 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:22.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.947+0000 7f7769298700 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c4bbcfe800 0x55c4bbabac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:22.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.947+0000 7f7769a99700 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c4bbcfe400 0x55c4bbabb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:22.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.947+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c4bbcfe400 msgr2=0x55c4bbabb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:22.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.947+0000 7f7768a97700 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c4bbcfe400 0x55c4bbabb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:22.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.947+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c4bbcfe800 msgr2=0x55c4bbabac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:22.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.947+0000 7f7768a97700 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c4bbcfe800 0x55c4bbabac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:22.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.947+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4bba34b60 con 0x55c4bbcfec00 2023-10-11T09:52:22.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.947+0000 7f7768a97700 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4bbcfec00 0x55c4bbaba680 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x55c4bc7f1470 tx=0x55c4bbd12cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:22.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.947+0000 7f776328c700 1 -- 172.21.15.78:0/2472870157 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c4bba431e0 con 0x55c4bbcfec00 2023-10-11T09:52:22.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.947+0000 7f776328c700 1 -- 172.21.15.78:0/2472870157 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c4bc7fd380 con 0x55c4bbcfec00 2023-10-11T09:52:22.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.947+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c4bba34340 con 0x55c4bbcfec00 2023-10-11T09:52:22.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.947+0000 7f776328c700 1 -- 172.21.15.78:0/2472870157 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c4bba42dc0 con 0x55c4bbcfec00 2023-10-11T09:52:22.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.947+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4bba35520 con 0x55c4bbcfec00 2023-10-11T09:52:22.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.947+0000 7f776aadc840 4 datascan.init: resolving metadata pool 17 2023-10-11T09:52:22.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.951+0000 7f776328c700 1 -- 172.21.15.78:0/2472870157 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x55c4bba70840 con 0x55c4bbcfec00 2023-10-11T09:52:22.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.951+0000 7f776328c700 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55c4bbe60000 0x55c4bbabb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:22.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.951+0000 7f776328c700 1 -- 172.21.15.78:0/2472870157 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6234+0+0 (secure 0 0 0) 0x55c4bba4ca80 con 0x55c4bbcfec00 2023-10-11T09:52:22.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.951+0000 7f776aadc840 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-10-11T09:52:22.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.951+0000 7f776aadc840 4 datascan.main: data pool 'cephfs_data' has ID 18 2023-10-11T09:52:22.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.951+0000 7f776aadc840 4 datascan.main: opening data pool 'cephfs_data' 2023-10-11T09:52:22.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.951+0000 7f776aadc840 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55c4bbcff000 0x55c4bbaba100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:22.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.951+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:1 18.0 18:00000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbcff400 con 0x55c4bbcff000 2023-10-11T09:52:22.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.951+0000 7f7769a99700 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55c4bbe60000 0x55c4bbabb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:22.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.951+0000 7f7769298700 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55c4bbcff000 0x55c4bbaba100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:22.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.951+0000 7f7769a99700 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55c4bbe60000 0x55c4bbabb700 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x55c4bc77b2f0 tx=0x55c4bbd12d80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:22.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.955+0000 7f7769298700 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55c4bbcff000 0x55c4bbaba100 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:22.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.959+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.0 v2:172.21.15.78:6800/35735 1 ==== osd_op_reply(1 [pgnls start_epoch 129 out=81b] v127'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff000 2023-10-11T09:52:22.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.959+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:2 18.0 18:012cdbc7:::10000000206.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbcff800 con 0x55c4bbcff000 2023-10-11T09:52:22.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.959+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.0 v2:172.21.15.78:6800/35735 2 ==== osd_op_reply(2 10000000206.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff000 2023-10-11T09:52:22.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.959+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628294.0 2023-10-11T09:52:22.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.959+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:3 18.0 18:012cdbc7:::10000000206.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe60400 con 0x55c4bbcff000 2023-10-11T09:52:22.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.963+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.0 v2:172.21.15.78:6800/35735 3 ==== osd_op_reply(3 10000000206.00000000 [call] v129'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff000 2023-10-11T09:52:22.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.963+0000 7f776aadc840 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55c4bbe60800 0x55c4bbab9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:22.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.963+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:4 18.20 18:04000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe60c00 con 0x55c4bbe60800 2023-10-11T09:52:22.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.963+0000 7f7769a99700 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55c4bbe60800 0x55c4bbab9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:22.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.963+0000 7f7769a99700 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55c4bbe60800 0x55c4bbab9b80 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:22.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.971+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 1 ==== osd_op_reply(4 [pgnls start_epoch 129 out=81b] v127'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:22.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.971+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:5 18.20 18:05552577:::1000000020b.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61000 con 0x55c4bbe60800 2023-10-11T09:52:22.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.971+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 2 ==== osd_op_reply(5 1000000020b.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:22.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.971+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628299.0 2023-10-11T09:52:22.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.971+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:6 18.20 18:05552577:::1000000020b.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61400 con 0x55c4bbe60800 2023-10-11T09:52:22.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.971+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 3 ==== osd_op_reply(6 1000000020b.00000000 [call] v129'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:22.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.971+0000 7f776aadc840 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55c4bbcff400 0x55c4bbab9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:22.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.971+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:7 18.10 18:08000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbcff800 con 0x55c4bbcff400 2023-10-11T09:52:22.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.975+0000 7f7769298700 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55c4bbcff400 0x55c4bbab9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:22.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.975+0000 7f7769298700 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55c4bbcff400 0x55c4bbab9600 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:22.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.975+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== osd_op_reply(7 [pgnls start_epoch 129 out=101b] v127'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff400 2023-10-11T09:52:22.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.975+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:8 18.10 18:083d5826:::1000000022a.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe60c00 con 0x55c4bbcff400 2023-10-11T09:52:22.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.975+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.6 v2:172.21.15.133:6800/35900 2 ==== osd_op_reply(8 1000000022a.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff400 2023-10-11T09:52:22.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.975+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628330.0 2023-10-11T09:52:22.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.975+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:9 18.10 18:083d5826:::1000000022a.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61000 con 0x55c4bbcff400 2023-10-11T09:52:22.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.979+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.6 v2:172.21.15.133:6800/35900 3 ==== osd_op_reply(9 1000000022a.00000000 [call] v129'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff400 2023-10-11T09:52:22.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.979+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:10 18.10 18:08500202::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61800 con 0x55c4bbcff400 2023-10-11T09:52:22.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.979+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.6 v2:172.21.15.133:6800/35900 4 ==== osd_op_reply(10 [pgnls start_epoch 129 out=101b] v129'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff400 2023-10-11T09:52:22.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.979+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:11 18.10 18:08500202:::1000000023e.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61c00 con 0x55c4bbcff400 2023-10-11T09:52:22.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.979+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.6 v2:172.21.15.133:6800/35900 5 ==== osd_op_reply(11 1000000023e.00000000 [stat out=16b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff400 2023-10-11T09:52:22.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.983+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628350.0 2023-10-11T09:52:22.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.983+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:12 18.10 18:08500202:::1000000023e.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe60400 con 0x55c4bbcff400 2023-10-11T09:52:22.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.983+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.6 v2:172.21.15.133:6800/35900 6 ==== osd_op_reply(12 1000000023e.00000000 [call] v129'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff400 2023-10-11T09:52:22.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.983+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:13 18.10 18:08b99fea::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe60c00 con 0x55c4bbcff400 2023-10-11T09:52:22.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.983+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.6 v2:172.21.15.133:6800/35900 7 ==== osd_op_reply(13 [pgnls start_epoch 129 out=101b] v129'16 uv16 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff400 2023-10-11T09:52:22.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.983+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:14 18.10 18:08b99fea:::100000001fd.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbcff800 con 0x55c4bbcff400 2023-10-11T09:52:22.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.987+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.6 v2:172.21.15.133:6800/35900 8 ==== osd_op_reply(14 100000001fd.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff400 2023-10-11T09:52:22.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.987+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628285.0 2023-10-11T09:52:22.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.987+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:15 18.10 18:08b99fea:::100000001fd.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61000 con 0x55c4bbcff400 2023-10-11T09:52:22.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.987+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.6 v2:172.21.15.133:6800/35900 9 ==== osd_op_reply(15 100000001fd.00000000 [call] v129'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff400 2023-10-11T09:52:22.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.987+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:16 18.10 18:08e6f8ca::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61800 con 0x55c4bbcff400 2023-10-11T09:52:22.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.987+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.6 v2:172.21.15.133:6800/35900 10 ==== osd_op_reply(16 [pgnls start_epoch 129 out=81b] v129'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff400 2023-10-11T09:52:22.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.987+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:17 18.10 18:08e6f8ca:::10000000257.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61c00 con 0x55c4bbcff400 2023-10-11T09:52:22.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.987+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.6 v2:172.21.15.133:6800/35900 11 ==== osd_op_reply(17 10000000257.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff400 2023-10-11T09:52:22.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.987+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628375.0 2023-10-11T09:52:22.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.987+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:18 18.10 18:08e6f8ca:::10000000257.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe60400 con 0x55c4bbcff400 2023-10-11T09:52:22.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.991+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.6 v2:172.21.15.133:6800/35900 12 ==== osd_op_reply(18 10000000257.00000000 [call] v129'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff400 2023-10-11T09:52:22.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.991+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:19 18.30 18:0c000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe60c00 con 0x55c4bbcff000 2023-10-11T09:52:23.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.991+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.0 v2:172.21.15.78:6800/35735 4 ==== osd_op_reply(19 [pgnls start_epoch 129 out=101b] v127'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff000 2023-10-11T09:52:23.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.991+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:20 18.30 18:0c00cb85:::1000000021d.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61000 con 0x55c4bbcff000 2023-10-11T09:52:23.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.991+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.0 v2:172.21.15.78:6800/35735 5 ==== osd_op_reply(20 1000000021d.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff000 2023-10-11T09:52:23.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.991+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628317.0 2023-10-11T09:52:23.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.991+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:21 18.30 18:0c00cb85:::1000000021d.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61800 con 0x55c4bbcff000 2023-10-11T09:52:23.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.995+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.0 v2:172.21.15.78:6800/35735 6 ==== osd_op_reply(21 1000000021d.00000000 [call] v129'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff000 2023-10-11T09:52:23.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.995+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:22 18.30 18:0c28cce1::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61c00 con 0x55c4bbcff000 2023-10-11T09:52:23.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.995+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.0 v2:172.21.15.78:6800/35735 7 ==== osd_op_reply(22 [pgnls start_epoch 129 out=101b] v129'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff000 2023-10-11T09:52:23.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.995+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:23 18.30 18:0c28cce1:::10000000244.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbcff800 con 0x55c4bbcff000 2023-10-11T09:52:23.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.999+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.0 v2:172.21.15.78:6800/35735 8 ==== osd_op_reply(23 10000000244.00000000 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff000 2023-10-11T09:52:23.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.999+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628356.0 2023-10-11T09:52:23.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.999+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:24 18.30 18:0c28cce1:::10000000244.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbcff800 con 0x55c4bbcff000 2023-10-11T09:52:23.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.999+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.0 v2:172.21.15.78:6800/35735 9 ==== osd_op_reply(24 10000000244.00000000 [call] v129'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff000 2023-10-11T09:52:23.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.999+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:25 18.30 18:0cc9e906::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe60400 con 0x55c4bbcff000 2023-10-11T09:52:23.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.999+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.0 v2:172.21.15.78:6800/35735 10 ==== osd_op_reply(25 [pgnls start_epoch 129 out=101b] v129'19 uv19 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff000 2023-10-11T09:52:23.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.999+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:26 18.30 18:0cc9e906:::10000000227.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe60c00 con 0x55c4bbcff000 2023-10-11T09:52:23.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.999+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.0 v2:172.21.15.78:6800/35735 11 ==== osd_op_reply(26 10000000227.00000000 [stat out=16b] v0'0 uv15 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff000 2023-10-11T09:52:23.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.999+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628327.0 2023-10-11T09:52:23.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:22.999+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:27 18.30 18:0cc9e906:::10000000227.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61000 con 0x55c4bbcff000 2023-10-11T09:52:23.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.003+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.0 v2:172.21.15.78:6800/35735 12 ==== osd_op_reply(27 10000000227.00000000 [call] v129'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff000 2023-10-11T09:52:23.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.003+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:28 18.30 18:0e3d5a27::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61800 con 0x55c4bbcff000 2023-10-11T09:52:23.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.003+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.0 v2:172.21.15.78:6800/35735 13 ==== osd_op_reply(28 [pgnls start_epoch 129 out=81b] v129'20 uv20 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff000 2023-10-11T09:52:23.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.003+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:29 18.30 18:0e3d5a27:::10000000250.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61c00 con 0x55c4bbcff000 2023-10-11T09:52:23.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.003+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.0 v2:172.21.15.78:6800/35735 14 ==== osd_op_reply(29 10000000250.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff000 2023-10-11T09:52:23.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.003+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628368.0 2023-10-11T09:52:23.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.003+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:30 18.30 18:0e3d5a27:::10000000250.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8e000 con 0x55c4bbcff000 2023-10-11T09:52:23.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.007+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.0 v2:172.21.15.78:6800/35735 15 ==== osd_op_reply(30 10000000250.00000000 [call] v129'21 uv21 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff000 2023-10-11T09:52:23.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.007+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:31 18.8 18:10000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8e400 con 0x55c4bbcff000 2023-10-11T09:52:23.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.007+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.0 v2:172.21.15.78:6800/35735 16 ==== osd_op_reply(31 [pgnls start_epoch 129 out=49b] v127'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff000 2023-10-11T09:52:23.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.007+0000 7f776428e700 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:32 18.28 18:14000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbcff800 con 0x55c4bbe60800 2023-10-11T09:52:23.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.007+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 4 ==== osd_op_reply(32 [pgnls start_epoch 129 out=101b] v127'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.007+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:33 18.28 18:14ce3b86:::10000000233.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8e800 con 0x55c4bbe60800 2023-10-11T09:52:23.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.007+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 5 ==== osd_op_reply(33 10000000233.00000000 [stat out=16b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.007+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628339.0 2023-10-11T09:52:23.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.007+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:34 18.28 18:14ce3b86:::10000000233.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8ec00 con 0x55c4bbe60800 2023-10-11T09:52:23.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.011+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 6 ==== osd_op_reply(34 10000000233.00000000 [call] v129'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.011+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:35 18.28 18:15246602::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8f000 con 0x55c4bbe60800 2023-10-11T09:52:23.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.011+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 7 ==== osd_op_reply(35 [pgnls start_epoch 129 out=101b] v129'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.011+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:36 18.28 18:15246602:::10000000202.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8f400 con 0x55c4bbe60800 2023-10-11T09:52:23.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.015+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 8 ==== osd_op_reply(36 10000000202.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.015+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628290.0 2023-10-11T09:52:23.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.015+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:37 18.28 18:15246602:::10000000202.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8f800 con 0x55c4bbe60800 2023-10-11T09:52:23.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.015+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 9 ==== osd_op_reply(37 10000000202.00000000 [call] v129'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.015+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:38 18.28 18:1537fa38::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8fc00 con 0x55c4bbe60800 2023-10-11T09:52:23.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.019+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 10 ==== osd_op_reply(38 [pgnls start_epoch 129 out=101b] v129'16 uv16 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.019+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:39 18.28 18:1537fa38:::10000000236.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8ec00 con 0x55c4bbe60800 2023-10-11T09:52:23.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.019+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 11 ==== osd_op_reply(39 10000000236.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.019+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628342.0 2023-10-11T09:52:23.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.019+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:40 18.28 18:1537fa38:::10000000236.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8f000 con 0x55c4bbe60800 2023-10-11T09:52:23.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.019+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 12 ==== osd_op_reply(40 10000000236.00000000 [call] v129'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.019+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:41 18.28 18:177a762d::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8f400 con 0x55c4bbe60800 2023-10-11T09:52:23.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.023+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 13 ==== osd_op_reply(41 [pgnls start_epoch 129 out=81b] v129'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.030 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.023+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:42 18.28 18:177a762d:::10000000201.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8f800 con 0x55c4bbe60800 2023-10-11T09:52:23.030 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.023+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 14 ==== osd_op_reply(42 10000000201.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.023+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628289.0 2023-10-11T09:52:23.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.023+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:43 18.28 18:177a762d:::10000000201.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8e800 con 0x55c4bbe60800 2023-10-11T09:52:23.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.027+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 15 ==== osd_op_reply(43 10000000201.00000000 [call] v129'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.027+0000 7f776aadc840 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55c4bbe8e000 0x55c4bbe94000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:23.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.027+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:44 18.18 18:18000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61400 con 0x55c4bbe8e000 2023-10-11T09:52:23.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.027+0000 7f7768a97700 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55c4bbe8e000 0x55c4bbe94000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:23.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.027+0000 7f7768a97700 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55c4bbe8e000 0x55c4bbe94000 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:23.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.027+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.1 v2:172.21.15.78:6809/35734 1 ==== osd_op_reply(44 [pgnls start_epoch 129 out=49b] v127'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbe8e000 2023-10-11T09:52:23.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.027+0000 7f7764a8f700 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55c4bbe8fc00 0x55c4bbe96680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:23.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.027+0000 7f7764a8f700 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:45 18.38 18:1c000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8ec00 con 0x55c4bbe8fc00 2023-10-11T09:52:23.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.027+0000 7f7769a99700 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55c4bbe8fc00 0x55c4bbe96680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:23.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.031+0000 7f7769a99700 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55c4bbe8fc00 0x55c4bbe96680 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:23.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.031+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(45 [pgnls start_epoch 129 out=49b] v127'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.031+0000 7f776428e700 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:46 18.4 18:20000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8f400 con 0x55c4bbcff000 2023-10-11T09:52:23.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.031+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.0 v2:172.21.15.78:6800/35735 17 ==== osd_op_reply(46 [pgnls start_epoch 129 out=81b] v127'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff000 2023-10-11T09:52:23.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.031+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:47 18.4 18:2231e764:::10000000220.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe60400 con 0x55c4bbcff000 2023-10-11T09:52:23.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.031+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.0 v2:172.21.15.78:6800/35735 18 ==== osd_op_reply(47 10000000220.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff000 2023-10-11T09:52:23.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.031+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628320.0 2023-10-11T09:52:23.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.031+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:48 18.4 18:2231e764:::10000000220.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe60c00 con 0x55c4bbcff000 2023-10-11T09:52:23.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.035+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.0 v2:172.21.15.78:6800/35735 19 ==== osd_op_reply(48 10000000220.00000000 [call] v129'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff000 2023-10-11T09:52:23.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.035+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:49 18.24 18:24000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61000 con 0x55c4bbcff000 2023-10-11T09:52:23.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.035+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.0 v2:172.21.15.78:6800/35735 20 ==== osd_op_reply(49 [pgnls start_epoch 129 out=101b] v127'8 uv8 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff000 2023-10-11T09:52:23.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.035+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:50 18.24 18:242ba651:::10000000234.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61800 con 0x55c4bbcff000 2023-10-11T09:52:23.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.035+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.0 v2:172.21.15.78:6800/35735 21 ==== osd_op_reply(50 10000000234.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff000 2023-10-11T09:52:23.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.035+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628340.0 2023-10-11T09:52:23.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.035+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:51 18.24 18:242ba651:::10000000234.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61c00 con 0x55c4bbcff000 2023-10-11T09:52:23.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.039+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.0 v2:172.21.15.78:6800/35735 22 ==== osd_op_reply(51 10000000234.00000000 [call] v129'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff000 2023-10-11T09:52:23.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.039+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:52 18.24 18:25b5fce8::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbcff800 con 0x55c4bbcff000 2023-10-11T09:52:23.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.039+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.0 v2:172.21.15.78:6800/35735 23 ==== osd_op_reply(52 [pgnls start_epoch 129 out=81b] v129'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff000 2023-10-11T09:52:23.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.039+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:53 18.24 18:25b5fce8:::10000000231.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbcff800 con 0x55c4bbcff000 2023-10-11T09:52:23.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.039+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.0 v2:172.21.15.78:6800/35735 24 ==== osd_op_reply(53 10000000231.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff000 2023-10-11T09:52:23.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.039+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628337.0 2023-10-11T09:52:23.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.039+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:54 18.24 18:25b5fce8:::10000000231.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe60400 con 0x55c4bbcff000 2023-10-11T09:52:23.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.043+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.0 v2:172.21.15.78:6800/35735 25 ==== osd_op_reply(54 10000000231.00000000 [call] v129'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff000 2023-10-11T09:52:23.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.043+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:55 18.14 18:28000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe60c00 con 0x55c4bbcff400 2023-10-11T09:52:23.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.043+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.6 v2:172.21.15.133:6800/35900 13 ==== osd_op_reply(55 [pgnls start_epoch 129 out=101b] v127'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff400 2023-10-11T09:52:23.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.043+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:56 18.14 18:29971f97:::1000000020e.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61000 con 0x55c4bbcff400 2023-10-11T09:52:23.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.043+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.6 v2:172.21.15.133:6800/35900 14 ==== osd_op_reply(56 1000000020e.00000000 [stat out=16b] v0'0 uv15 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff400 2023-10-11T09:52:23.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.043+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628302.0 2023-10-11T09:52:23.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.047+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:57 18.14 18:29971f97:::1000000020e.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61800 con 0x55c4bbcff400 2023-10-11T09:52:23.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.047+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.6 v2:172.21.15.133:6800/35900 15 ==== osd_op_reply(57 1000000020e.00000000 [call] v129'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff400 2023-10-11T09:52:23.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.047+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:58 18.14 18:2ab28be7::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61c00 con 0x55c4bbcff400 2023-10-11T09:52:23.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.051+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.6 v2:172.21.15.133:6800/35900 16 ==== osd_op_reply(58 [pgnls start_epoch 129 out=101b] v129'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff400 2023-10-11T09:52:23.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.051+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:59 18.14 18:2ab28be7:::10000000211.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8e400 con 0x55c4bbcff400 2023-10-11T09:52:23.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.051+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.6 v2:172.21.15.133:6800/35900 17 ==== osd_op_reply(59 10000000211.00000000 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff400 2023-10-11T09:52:23.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.051+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628305.0 2023-10-11T09:52:23.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.051+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:60 18.14 18:2ab28be7:::10000000211.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8f400 con 0x55c4bbcff400 2023-10-11T09:52:23.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.055+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.6 v2:172.21.15.133:6800/35900 18 ==== osd_op_reply(60 10000000211.00000000 [call] v129'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff400 2023-10-11T09:52:23.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.055+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:61 18.14 18:2b89f3ba::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbea2000 con 0x55c4bbcff400 2023-10-11T09:52:23.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.055+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.6 v2:172.21.15.133:6800/35900 19 ==== osd_op_reply(61 [pgnls start_epoch 129 out=81b] v129'19 uv19 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff400 2023-10-11T09:52:23.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.055+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:62 18.14 18:2b89f3ba:::10000000221.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbea2400 con 0x55c4bbcff400 2023-10-11T09:52:23.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.055+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.6 v2:172.21.15.133:6800/35900 20 ==== osd_op_reply(62 10000000221.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff400 2023-10-11T09:52:23.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.055+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628321.0 2023-10-11T09:52:23.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.055+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:63 18.14 18:2b89f3ba:::10000000221.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbea2800 con 0x55c4bbcff400 2023-10-11T09:52:23.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.059+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.6 v2:172.21.15.133:6800/35900 21 ==== osd_op_reply(63 10000000221.00000000 [call] v129'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff400 2023-10-11T09:52:23.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.059+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:64 18.34 18:2c000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbea2c00 con 0x55c4bbcff000 2023-10-11T09:52:23.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.059+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.0 v2:172.21.15.78:6800/35735 26 ==== osd_op_reply(64 [pgnls start_epoch 129 out=49b] v127'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff000 2023-10-11T09:52:23.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.059+0000 7f7764a8f700 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:65 18.c 18:30000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8f000 con 0x55c4bbe60800 2023-10-11T09:52:23.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.059+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 16 ==== osd_op_reply(65 [pgnls start_epoch 129 out=81b] v127'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.059+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:66 18.c 18:30604137:::100000001fc.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbea3000 con 0x55c4bbe60800 2023-10-11T09:52:23.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.059+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 17 ==== osd_op_reply(66 100000001fc.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.059+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628284.0 2023-10-11T09:52:23.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.059+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:67 18.c 18:30604137:::100000001fc.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbea3400 con 0x55c4bbe60800 2023-10-11T09:52:23.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.063+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 18 ==== osd_op_reply(67 100000001fc.00000000 [call] v129'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.063+0000 7f776aadc840 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55c4bbea3800 0x55c4bbe94580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:23.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.063+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:68 18.2c 18:34000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbea3c00 con 0x55c4bbea3800 2023-10-11T09:52:23.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.063+0000 7f7769298700 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55c4bbea3800 0x55c4bbe94580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:23.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.063+0000 7f7769298700 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55c4bbea3800 0x55c4bbe94580 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:23.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.067+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.2 v2:172.21.15.78:6808/35732 1 ==== osd_op_reply(68 [pgnls start_epoch 129 out=101b] v127'12 uv12 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbea3800 2023-10-11T09:52:23.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.067+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:69 18.2c 18:35964134:::10000000223.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbea2000 con 0x55c4bbea3800 2023-10-11T09:52:23.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.067+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.2 v2:172.21.15.78:6808/35732 2 ==== osd_op_reply(69 10000000223.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbea3800 2023-10-11T09:52:23.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.067+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628323.0 2023-10-11T09:52:23.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.067+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:70 18.2c 18:35964134:::10000000223.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8e400 con 0x55c4bbea3800 2023-10-11T09:52:23.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.067+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.2 v2:172.21.15.78:6808/35732 3 ==== osd_op_reply(70 10000000223.00000000 [call] v129'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbea3800 2023-10-11T09:52:23.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.067+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:71 18.2c 18:37389ade::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8f400 con 0x55c4bbea3800 2023-10-11T09:52:23.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.071+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.2 v2:172.21.15.78:6808/35732 4 ==== osd_op_reply(71 [pgnls start_epoch 129 out=81b] v129'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbea3800 2023-10-11T09:52:23.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.071+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:72 18.2c 18:37389ade:::10000000209.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe60400 con 0x55c4bbea3800 2023-10-11T09:52:23.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.071+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.2 v2:172.21.15.78:6808/35732 5 ==== osd_op_reply(72 10000000209.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbea3800 2023-10-11T09:52:23.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.071+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628297.0 2023-10-11T09:52:23.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.071+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:73 18.2c 18:37389ade:::10000000209.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe60c00 con 0x55c4bbea3800 2023-10-11T09:52:23.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.071+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.2 v2:172.21.15.78:6808/35732 6 ==== osd_op_reply(73 10000000209.00000000 [call] v129'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbea3800 2023-10-11T09:52:23.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.071+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:74 18.1c 18:38000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61000 con 0x55c4bbe8e000 2023-10-11T09:52:23.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.075+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.1 v2:172.21.15.78:6809/35734 2 ==== osd_op_reply(74 [pgnls start_epoch 129 out=49b] v127'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbe8e000 2023-10-11T09:52:23.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.075+0000 7f7764a8f700 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55c4bbeac000 0x55c4bbe96c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:23.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.075+0000 7f7764a8f700 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:75 18.3c 18:3c000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeac400 con 0x55c4bbeac000 2023-10-11T09:52:23.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.075+0000 7f7768a97700 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55c4bbeac000 0x55c4bbe96c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:23.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.075+0000 7f7768a97700 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55c4bbeac000 0x55c4bbe96c00 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:23.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.075+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 1 ==== osd_op_reply(75 [pgnls start_epoch 129 out=49b] v127'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.075+0000 7f776428e700 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:76 18.2 18:40000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8f800 con 0x55c4bbeac000 2023-10-11T09:52:23.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.075+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 2 ==== osd_op_reply(76 [pgnls start_epoch 129 out=49b] v127'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.079+0000 7f7764a8f700 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:77 18.22 18:44000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeac800 con 0x55c4bbe8fc00 2023-10-11T09:52:23.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.079+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 2 ==== osd_op_reply(77 [pgnls start_epoch 129 out=101b] v127'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.079+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:78 18.22 18:450d4ca5:::1000000024c.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61800 con 0x55c4bbe8fc00 2023-10-11T09:52:23.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.079+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 3 ==== osd_op_reply(78 1000000024c.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.079+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628364.0 2023-10-11T09:52:23.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.079+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:79 18.22 18:450d4ca5:::1000000024c.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61c00 con 0x55c4bbe8fc00 2023-10-11T09:52:23.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.083+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 4 ==== osd_op_reply(79 1000000024c.00000000 [call] v129'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.083+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:80 18.22 18:45b8cac8::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbcff800 con 0x55c4bbe8fc00 2023-10-11T09:52:23.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.083+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 5 ==== osd_op_reply(80 [pgnls start_epoch 129 out=101b] v129'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.083+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:81 18.22 18:45b8cac8:::1000000023d.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeac800 con 0x55c4bbe8fc00 2023-10-11T09:52:23.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.083+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 6 ==== osd_op_reply(81 1000000023d.00000000 [stat out=16b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.083+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628349.0 2023-10-11T09:52:23.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.083+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:82 18.22 18:45b8cac8:::1000000023d.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeac400 con 0x55c4bbe8fc00 2023-10-11T09:52:23.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.083+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 7 ==== osd_op_reply(82 1000000023d.00000000 [call] v129'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.083+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:83 18.22 18:47efb22f::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeacc00 con 0x55c4bbe8fc00 2023-10-11T09:52:23.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.087+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 8 ==== osd_op_reply(83 [pgnls start_epoch 129 out=81b] v129'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.087+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:84 18.22 18:47efb22f:::10000000235.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbead000 con 0x55c4bbe8fc00 2023-10-11T09:52:23.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.087+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 9 ==== osd_op_reply(84 10000000235.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.087+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628341.0 2023-10-11T09:52:23.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.087+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:85 18.22 18:47efb22f:::10000000235.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbead400 con 0x55c4bbe8fc00 2023-10-11T09:52:23.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.087+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 10 ==== osd_op_reply(85 10000000235.00000000 [call] v129'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.087+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:86 18.12 18:48000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbead800 con 0x55c4bbea3800 2023-10-11T09:52:23.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.091+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.2 v2:172.21.15.78:6808/35732 7 ==== osd_op_reply(86 [pgnls start_epoch 129 out=49b] v127'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbea3800 2023-10-11T09:52:23.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.091+0000 7f7764a8f700 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:87 18.32 18:4c000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbea2400 con 0x55c4bbe8fc00 2023-10-11T09:52:23.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.091+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 11 ==== osd_op_reply(87 [pgnls start_epoch 129 out=81b] v127'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.091+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:88 18.32 18:4e8c6251:::10000000252.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeadc00 con 0x55c4bbe8fc00 2023-10-11T09:52:23.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.091+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 12 ==== osd_op_reply(88 10000000252.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.091+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628370.0 2023-10-11T09:52:23.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.091+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:89 18.32 18:4e8c6251:::10000000252.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8e800 con 0x55c4bbe8fc00 2023-10-11T09:52:23.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.095+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 13 ==== osd_op_reply(89 10000000252.00000000 [call] v129'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.095+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:90 18.a 18:50000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8ec00 con 0x55c4bbe8fc00 2023-10-11T09:52:23.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.095+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 14 ==== osd_op_reply(90 [pgnls start_epoch 129 out=101b] v127'9 uv9 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.095+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:91 18.a 18:52dcfedc:::1000000023f.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8f000 con 0x55c4bbe8fc00 2023-10-11T09:52:23.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.095+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 15 ==== osd_op_reply(91 1000000023f.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.095+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628351.0 2023-10-11T09:52:23.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.095+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:92 18.a 18:52dcfedc:::1000000023f.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8f800 con 0x55c4bbe8fc00 2023-10-11T09:52:23.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.099+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 16 ==== osd_op_reply(92 1000000023f.00000000 [call] v129'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.099+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:93 18.a 18:53d16f76::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61400 con 0x55c4bbe8fc00 2023-10-11T09:52:23.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.099+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 17 ==== osd_op_reply(93 [pgnls start_epoch 129 out=81b] v129'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.099+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:94 18.a 18:53d16f76:::1000000023b.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61000 con 0x55c4bbe8fc00 2023-10-11T09:52:23.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.099+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 18 ==== osd_op_reply(94 1000000023b.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.099+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628347.0 2023-10-11T09:52:23.107 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.099+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:95 18.a 18:53d16f76:::1000000023b.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbea3000 con 0x55c4bbe8fc00 2023-10-11T09:52:23.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.103+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 19 ==== osd_op_reply(95 1000000023b.00000000 [call] v129'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.103+0000 7f776aadc840 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55c4bbea3400 0x55c4bbe94b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:23.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.103+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:96 18.2a 18:54000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbead400 con 0x55c4bbea3400 2023-10-11T09:52:23.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.103+0000 7f7769a99700 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55c4bbea3400 0x55c4bbe94b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:23.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.103+0000 7f7769a99700 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55c4bbea3400 0x55c4bbe94b00 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:23.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.103+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 1 ==== osd_op_reply(96 [pgnls start_epoch 129 out=101b] v127'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.103+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:97 18.2a 18:55a6cd83:::1000000020a.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeadc00 con 0x55c4bbea3400 2023-10-11T09:52:23.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.103+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 2 ==== osd_op_reply(97 1000000020a.00000000 [stat out=16b] v0'0 uv15 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.103+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628298.0 2023-10-11T09:52:23.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.103+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:98 18.2a 18:55a6cd83:::1000000020a.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbead800 con 0x55c4bbea3400 2023-10-11T09:52:23.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.107+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 3 ==== osd_op_reply(98 1000000020a.00000000 [call] v129'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.107+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:99 18.2a 18:56889a45::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeac800 con 0x55c4bbea3400 2023-10-11T09:52:23.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.107+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 4 ==== osd_op_reply(99 [pgnls start_epoch 129 out=101b] v129'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.107+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:100 18.2a 18:56889a45:::10000000213.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeac400 con 0x55c4bbea3400 2023-10-11T09:52:23.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.111+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 5 ==== osd_op_reply(100 10000000213.00000000 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.111+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628307.0 2023-10-11T09:52:23.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.111+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:101 18.2a 18:56889a45:::10000000213.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbcff800 con 0x55c4bbea3400 2023-10-11T09:52:23.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.111+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 6 ==== osd_op_reply(101 10000000213.00000000 [call] v129'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.111+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:102 18.2a 18:576ad4f4::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61400 con 0x55c4bbea3400 2023-10-11T09:52:23.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.111+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 7 ==== osd_op_reply(102 [pgnls start_epoch 129 out=101b] v129'19 uv19 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.111+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:103 18.2a 18:576ad4f4:::10000000245.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe60400 con 0x55c4bbea3400 2023-10-11T09:52:23.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.115+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 8 ==== osd_op_reply(103 10000000245.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.115+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628357.0 2023-10-11T09:52:23.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.115+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:104 18.2a 18:576ad4f4:::10000000245.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe60c00 con 0x55c4bbea3400 2023-10-11T09:52:23.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.115+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 9 ==== osd_op_reply(104 10000000245.00000000 [call] v129'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.115+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:105 18.2a 18:57fe7535::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61800 con 0x55c4bbea3400 2023-10-11T09:52:23.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.115+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 10 ==== osd_op_reply(105 [pgnls start_epoch 129 out=81b] v129'20 uv20 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.115+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:106 18.2a 18:57fe7535:::100000001f6.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61c00 con 0x55c4bbea3400 2023-10-11T09:52:23.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.119+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 11 ==== osd_op_reply(106 100000001f6.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.119+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628278.0 2023-10-11T09:52:23.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.119+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:107 18.2a 18:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8e800 con 0x55c4bbea3400 2023-10-11T09:52:23.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.119+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 12 ==== osd_op_reply(107 100000001f6.00000000 [call] v129'21 uv21 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.119+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:108 18.1a 18:58000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8ec00 con 0x55c4bbe8fc00 2023-10-11T09:52:23.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.119+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 20 ==== osd_op_reply(108 [pgnls start_epoch 129 out=101b] v127'11 uv11 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.123+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:109 18.1a 18:582f7b2d:::10000000226.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8f000 con 0x55c4bbe8fc00 2023-10-11T09:52:23.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.123+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 21 ==== osd_op_reply(109 10000000226.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.123+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628326.0 2023-10-11T09:52:23.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.123+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:110 18.1a 18:582f7b2d:::10000000226.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8f800 con 0x55c4bbe8fc00 2023-10-11T09:52:23.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.123+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 22 ==== osd_op_reply(110 10000000226.00000000 [call] v129'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.123+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:111 18.1a 18:5a2a1294::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8e400 con 0x55c4bbe8fc00 2023-10-11T09:52:23.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.123+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 23 ==== osd_op_reply(111 [pgnls start_epoch 129 out=81b] v129'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.127+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:112 18.1a 18:5a2a1294:::1000000023a.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8f400 con 0x55c4bbe8fc00 2023-10-11T09:52:23.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.127+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 24 ==== osd_op_reply(112 1000000023a.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.127+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628346.0 2023-10-11T09:52:23.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.127+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:113 18.1a 18:5a2a1294:::1000000023a.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbea2400 con 0x55c4bbe8fc00 2023-10-11T09:52:23.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.127+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 25 ==== osd_op_reply(113 1000000023a.00000000 [call] v129'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.127+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:114 18.3a 18:5c000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbea2000 con 0x55c4bbcff000 2023-10-11T09:52:23.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.127+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.0 v2:172.21.15.78:6800/35735 27 ==== osd_op_reply(114 [pgnls start_epoch 129 out=49b] v127'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff000 2023-10-11T09:52:23.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.127+0000 7f7764a8f700 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:115 18.6 18:60000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbea2800 con 0x55c4bbeac000 2023-10-11T09:52:23.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.131+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 3 ==== osd_op_reply(115 [pgnls start_epoch 129 out=81b] v127'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.131+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:116 18.6 18:614797eb:::10000000228.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbead400 con 0x55c4bbeac000 2023-10-11T09:52:23.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.131+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 4 ==== osd_op_reply(116 10000000228.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.131+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628328.0 2023-10-11T09:52:23.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.131+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:117 18.6 18:614797eb:::10000000228.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeadc00 con 0x55c4bbeac000 2023-10-11T09:52:23.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.135+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 5 ==== osd_op_reply(117 10000000228.00000000 [call] v129'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.135+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:118 18.26 18:64000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbead800 con 0x55c4bbea3800 2023-10-11T09:52:23.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.135+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.2 v2:172.21.15.78:6808/35732 8 ==== osd_op_reply(118 [pgnls start_epoch 129 out=101b] v127'21 uv21 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbea3800 2023-10-11T09:52:23.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.135+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:119 18.26 18:658f2455:::100000001ff.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeac800 con 0x55c4bbea3800 2023-10-11T09:52:23.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.135+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.2 v2:172.21.15.78:6808/35732 9 ==== osd_op_reply(119 100000001ff.00000000 [stat out=16b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbea3800 2023-10-11T09:52:23.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.135+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628287.0 2023-10-11T09:52:23.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.135+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:120 18.26 18:658f2455:::100000001ff.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeac400 con 0x55c4bbea3800 2023-10-11T09:52:23.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.139+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.2 v2:172.21.15.78:6808/35732 10 ==== osd_op_reply(120 100000001ff.00000000 [call] v129'22 uv22 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbea3800 2023-10-11T09:52:23.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.139+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:121 18.26 18:65d05a5c::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeacc00 con 0x55c4bbea3800 2023-10-11T09:52:23.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.139+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.2 v2:172.21.15.78:6808/35732 11 ==== osd_op_reply(121 [pgnls start_epoch 129 out=101b] v129'22 uv22 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbea3800 2023-10-11T09:52:23.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.139+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:122 18.26 18:65d05a5c:::10000000258.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe60400 con 0x55c4bbea3800 2023-10-11T09:52:23.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.139+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.2 v2:172.21.15.78:6808/35732 12 ==== osd_op_reply(122 10000000258.00000000 [stat out=16b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbea3800 2023-10-11T09:52:23.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.139+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628376.0 2023-10-11T09:52:23.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.139+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:123 18.26 18:65d05a5c:::10000000258.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe60c00 con 0x55c4bbea3800 2023-10-11T09:52:23.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.143+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.2 v2:172.21.15.78:6808/35732 13 ==== osd_op_reply(123 10000000258.00000000 [call] v129'23 uv23 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbea3800 2023-10-11T09:52:23.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.143+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:124 18.26 18:65e03fee::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61800 con 0x55c4bbea3800 2023-10-11T09:52:23.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.143+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.2 v2:172.21.15.78:6808/35732 14 ==== osd_op_reply(124 [pgnls start_epoch 129 out=81b] v129'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbea3800 2023-10-11T09:52:23.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.143+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:125 18.26 18:65e03fee:::10000000240.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61c00 con 0x55c4bbea3800 2023-10-11T09:52:23.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.143+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.2 v2:172.21.15.78:6808/35732 15 ==== osd_op_reply(125 10000000240.00000000 [stat out=16b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbea3800 2023-10-11T09:52:23.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.143+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628352.0 2023-10-11T09:52:23.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.143+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:126 18.26 18:65e03fee:::10000000240.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61000 con 0x55c4bbea3800 2023-10-11T09:52:23.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.147+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.2 v2:172.21.15.78:6808/35732 16 ==== osd_op_reply(126 10000000240.00000000 [call] v129'24 uv24 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbea3800 2023-10-11T09:52:23.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.147+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:127 18.16 18:68000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61400 con 0x55c4bbe8fc00 2023-10-11T09:52:23.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.147+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 26 ==== osd_op_reply(127 [pgnls start_epoch 129 out=49b] v127'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.147+0000 7f776428e700 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:128 18.36 18:6c000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbead000 con 0x55c4bbe8e000 2023-10-11T09:52:23.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.147+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.1 v2:172.21.15.78:6809/35734 3 ==== osd_op_reply(128 [pgnls start_epoch 129 out=101b] v127'23 uv23 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbe8e000 2023-10-11T09:52:23.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.147+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:129 18.36 18:6cdfec98:::10000000255.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbcff800 con 0x55c4bbe8e000 2023-10-11T09:52:23.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.147+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.1 v2:172.21.15.78:6809/35734 4 ==== osd_op_reply(129 10000000255.00000000 [stat out=16b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbe8e000 2023-10-11T09:52:23.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.147+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628373.0 2023-10-11T09:52:23.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.147+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:130 18.36 18:6cdfec98:::10000000255.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbea3000 con 0x55c4bbe8e000 2023-10-11T09:52:23.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.151+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.1 v2:172.21.15.78:6809/35734 5 ==== osd_op_reply(130 10000000255.00000000 [call] v129'24 uv24 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbe8e000 2023-10-11T09:52:23.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.151+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:131 18.36 18:6ee5e2c4::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8e800 con 0x55c4bbe8e000 2023-10-11T09:52:23.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.151+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.1 v2:172.21.15.78:6809/35734 6 ==== osd_op_reply(131 [pgnls start_epoch 129 out=101b] v129'24 uv24 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbe8e000 2023-10-11T09:52:23.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.151+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:132 18.36 18:6ee5e2c4:::100000001f9.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8ec00 con 0x55c4bbe8e000 2023-10-11T09:52:23.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.151+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.1 v2:172.21.15.78:6809/35734 7 ==== osd_op_reply(132 100000001f9.00000000 [stat out=16b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbe8e000 2023-10-11T09:52:23.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.151+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628281.0 2023-10-11T09:52:23.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.151+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:133 18.36 18:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8f000 con 0x55c4bbe8e000 2023-10-11T09:52:23.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.155+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.1 v2:172.21.15.78:6809/35734 8 ==== osd_op_reply(133 100000001f9.00000000 [call] v129'25 uv25 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbe8e000 2023-10-11T09:52:23.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.155+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:134 18.36 18:6f746908::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8f800 con 0x55c4bbe8e000 2023-10-11T09:52:23.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.155+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.1 v2:172.21.15.78:6809/35734 9 ==== osd_op_reply(134 [pgnls start_epoch 129 out=101b] v129'25 uv25 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbe8e000 2023-10-11T09:52:23.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.155+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:135 18.36 18:6f746908:::1000000022c.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8e400 con 0x55c4bbe8e000 2023-10-11T09:52:23.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.155+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.1 v2:172.21.15.78:6809/35734 10 ==== osd_op_reply(135 1000000022c.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbe8e000 2023-10-11T09:52:23.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.155+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628332.0 2023-10-11T09:52:23.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.155+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:136 18.36 18:6f746908:::1000000022c.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8f400 con 0x55c4bbe8e000 2023-10-11T09:52:23.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.159+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.1 v2:172.21.15.78:6809/35734 11 ==== osd_op_reply(136 1000000022c.00000000 [call] v129'26 uv26 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbe8e000 2023-10-11T09:52:23.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.159+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:137 18.36 18:6fe38dc2::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbcff800 con 0x55c4bbe8e000 2023-10-11T09:52:23.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.159+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.1 v2:172.21.15.78:6809/35734 12 ==== osd_op_reply(137 [pgnls start_epoch 129 out=81b] v129'26 uv26 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbe8e000 2023-10-11T09:52:23.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.159+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:138 18.36 18:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbead400 con 0x55c4bbe8e000 2023-10-11T09:52:23.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.159+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.1 v2:172.21.15.78:6809/35734 13 ==== osd_op_reply(138 100000001f7.00000000 [stat out=16b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbe8e000 2023-10-11T09:52:23.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.159+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628279.0 2023-10-11T09:52:23.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.159+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:139 18.36 18:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeadc00 con 0x55c4bbe8e000 2023-10-11T09:52:23.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.159+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.1 v2:172.21.15.78:6809/35734 14 ==== osd_op_reply(139 100000001f7.00000000 [call] v129'27 uv27 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbe8e000 2023-10-11T09:52:23.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.159+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:140 18.e 18:70000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbead000 con 0x55c4bbcff000 2023-10-11T09:52:23.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.159+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.0 v2:172.21.15.78:6800/35735 28 ==== osd_op_reply(140 [pgnls start_epoch 129 out=81b] v127'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff000 2023-10-11T09:52:23.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.163+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:141 18.e 18:71c13145:::1000000022f.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbea3000 con 0x55c4bbcff000 2023-10-11T09:52:23.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.163+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.0 v2:172.21.15.78:6800/35735 29 ==== osd_op_reply(141 1000000022f.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff000 2023-10-11T09:52:23.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.163+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628335.0 2023-10-11T09:52:23.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.163+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:142 18.e 18:71c13145:::1000000022f.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbea2800 con 0x55c4bbcff000 2023-10-11T09:52:23.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.163+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.0 v2:172.21.15.78:6800/35735 30 ==== osd_op_reply(142 1000000022f.00000000 [call] v129'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff000 2023-10-11T09:52:23.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.163+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:143 18.2e 18:74000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8e800 con 0x55c4bbe8fc00 2023-10-11T09:52:23.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.167+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 27 ==== osd_op_reply(143 [pgnls start_epoch 129 out=101b] v127'16 uv16 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.167+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:144 18.2e 18:74d13cfc:::1000000021c.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8ec00 con 0x55c4bbe8fc00 2023-10-11T09:52:23.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.167+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 28 ==== osd_op_reply(144 1000000021c.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.167+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628316.0 2023-10-11T09:52:23.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.167+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:145 18.2e 18:74d13cfc:::1000000021c.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8f000 con 0x55c4bbe8fc00 2023-10-11T09:52:23.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.167+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 29 ==== osd_op_reply(145 1000000021c.00000000 [call] v129'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.167+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:146 18.2e 18:7528f44f::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8f800 con 0x55c4bbe8fc00 2023-10-11T09:52:23.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.167+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 30 ==== osd_op_reply(146 [pgnls start_epoch 129 out=101b] v129'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.167+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:147 18.2e 18:7528f44f:::10000000242.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8e400 con 0x55c4bbe8fc00 2023-10-11T09:52:23.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.171+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 31 ==== osd_op_reply(147 10000000242.00000000 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.171+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628354.0 2023-10-11T09:52:23.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.171+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:148 18.2e 18:7528f44f:::10000000242.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbead800 con 0x55c4bbe8fc00 2023-10-11T09:52:23.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.171+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 32 ==== osd_op_reply(148 10000000242.00000000 [call] v129'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.171+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:149 18.2e 18:7611a124::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeac800 con 0x55c4bbe8fc00 2023-10-11T09:52:23.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.171+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 33 ==== osd_op_reply(149 [pgnls start_epoch 129 out=81b] v129'18 uv18 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.171+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:150 18.2e 18:7611a124:::1000000021e.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeac400 con 0x55c4bbe8fc00 2023-10-11T09:52:23.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.175+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 34 ==== osd_op_reply(150 1000000021e.00000000 [stat out=16b] v0'0 uv15 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.175+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628318.0 2023-10-11T09:52:23.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.175+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:151 18.2e 18:7611a124:::1000000021e.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeacc00 con 0x55c4bbe8fc00 2023-10-11T09:52:23.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.175+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 35 ==== osd_op_reply(151 1000000021e.00000000 [call] v129'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.175+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:152 18.1e 18:78000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbead000 con 0x55c4bbe60800 2023-10-11T09:52:23.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.179+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 19 ==== osd_op_reply(152 [pgnls start_epoch 129 out=101b] v127'11 uv11 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.179+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:153 18.1e 18:789b801b:::1000000020c.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe60400 con 0x55c4bbe60800 2023-10-11T09:52:23.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.179+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 20 ==== osd_op_reply(153 1000000020c.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.179+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628300.0 2023-10-11T09:52:23.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.179+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:154 18.1e 18:789b801b:::1000000020c.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe60c00 con 0x55c4bbe60800 2023-10-11T09:52:23.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.179+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 21 ==== osd_op_reply(154 1000000020c.00000000 [call] v129'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.179+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:155 18.1e 18:7a9d9b47::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61800 con 0x55c4bbe60800 2023-10-11T09:52:23.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.183+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 22 ==== osd_op_reply(155 [pgnls start_epoch 129 out=101b] v129'12 uv12 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.189 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.183+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:156 18.1e 18:7a9d9b47:::1000000022b.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61c00 con 0x55c4bbe60800 2023-10-11T09:52:23.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.183+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 23 ==== osd_op_reply(156 1000000022b.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.183+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628331.0 2023-10-11T09:52:23.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.183+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:157 18.1e 18:7a9d9b47:::1000000022b.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61000 con 0x55c4bbe60800 2023-10-11T09:52:23.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.187+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 24 ==== osd_op_reply(157 1000000022b.00000000 [call] v129'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.187+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:158 18.1e 18:7ade2fd8::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbea2000 con 0x55c4bbe60800 2023-10-11T09:52:23.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.187+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 25 ==== osd_op_reply(158 [pgnls start_epoch 129 out=81b] v129'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.187+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:159 18.1e 18:7ade2fd8:::1000000024e.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbead800 con 0x55c4bbe60800 2023-10-11T09:52:23.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.187+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 26 ==== osd_op_reply(159 1000000024e.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.187+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628366.0 2023-10-11T09:52:23.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.187+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:160 18.1e 18:7ade2fd8:::1000000024e.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeac800 con 0x55c4bbe60800 2023-10-11T09:52:23.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.191+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 27 ==== osd_op_reply(160 1000000024e.00000000 [call] v129'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.191+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:161 18.3e 18:7c000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeac400 con 0x55c4bbe8fc00 2023-10-11T09:52:23.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.191+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 36 ==== osd_op_reply(161 [pgnls start_epoch 129 out=81b] v127'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.191+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:162 18.3e 18:7f918703:::10000000229.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeacc00 con 0x55c4bbe8fc00 2023-10-11T09:52:23.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.191+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 37 ==== osd_op_reply(162 10000000229.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.191+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628329.0 2023-10-11T09:52:23.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.191+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:163 18.3e 18:7f918703:::10000000229.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8e800 con 0x55c4bbe8fc00 2023-10-11T09:52:23.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.195+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 38 ==== osd_op_reply(163 10000000229.00000000 [call] v129'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.195+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:164 18.1 18:80000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8ec00 con 0x55c4bbe8fc00 2023-10-11T09:52:23.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.195+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 39 ==== osd_op_reply(164 [pgnls start_epoch 129 out=81b] v127'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.195+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:165 18.1 18:805bc955:::10000000224.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8f000 con 0x55c4bbe8fc00 2023-10-11T09:52:23.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.195+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 40 ==== osd_op_reply(165 10000000224.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.195+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628324.0 2023-10-11T09:52:23.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.195+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:166 18.1 18:805bc955:::10000000224.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8f800 con 0x55c4bbe8fc00 2023-10-11T09:52:23.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.199+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 41 ==== osd_op_reply(166 10000000224.00000000 [call] v129'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.199+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:167 18.21 18:84000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8e400 con 0x55c4bbe60800 2023-10-11T09:52:23.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.199+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 28 ==== osd_op_reply(167 [pgnls start_epoch 129 out=49b] v127'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.199+0000 7f776428e700 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:168 18.11 18:88000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe60400 con 0x55c4bbe8e000 2023-10-11T09:52:23.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.199+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.1 v2:172.21.15.78:6809/35734 15 ==== osd_op_reply(168 [pgnls start_epoch 129 out=49b] v127'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbe8e000 2023-10-11T09:52:23.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.199+0000 7f7764a8f700 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:169 18.31 18:8c000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbea2c00 con 0x55c4bbea3400 2023-10-11T09:52:23.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.203+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 13 ==== osd_op_reply(169 [pgnls start_epoch 129 out=101b] v127'11 uv11 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.203+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:170 18.31 18:8c51d9b0:::10000000216.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61400 con 0x55c4bbea3400 2023-10-11T09:52:23.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.203+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 14 ==== osd_op_reply(170 10000000216.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.203+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628310.0 2023-10-11T09:52:23.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.203+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:171 18.31 18:8c51d9b0:::10000000216.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbea2400 con 0x55c4bbea3400 2023-10-11T09:52:23.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.207+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 15 ==== osd_op_reply(171 10000000216.00000000 [call] v129'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.207+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:172 18.31 18:8fb7f7b7::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8e800 con 0x55c4bbea3400 2023-10-11T09:52:23.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.207+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 16 ==== osd_op_reply(172 [pgnls start_epoch 129 out=81b] v129'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.207+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:173 18.31 18:8fb7f7b7:::10000000251.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbead000 con 0x55c4bbea3400 2023-10-11T09:52:23.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.207+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 17 ==== osd_op_reply(173 10000000251.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.207+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628369.0 2023-10-11T09:52:23.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.207+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:174 18.31 18:8fb7f7b7:::10000000251.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbead800 con 0x55c4bbea3400 2023-10-11T09:52:23.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.211+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 18 ==== osd_op_reply(174 10000000251.00000000 [call] v129'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.211+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:175 18.9 18:90000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeac800 con 0x55c4bbe8fc00 2023-10-11T09:52:23.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.211+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 42 ==== osd_op_reply(175 [pgnls start_epoch 129 out=101b] v127'6 uv6 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.211+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:176 18.9 18:915bff82:::1000000024d.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeac400 con 0x55c4bbe8fc00 2023-10-11T09:52:23.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.211+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 43 ==== osd_op_reply(176 1000000024d.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.215+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628365.0 2023-10-11T09:52:23.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.215+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:177 18.9 18:915bff82:::1000000024d.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeacc00 con 0x55c4bbe8fc00 2023-10-11T09:52:23.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.215+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 44 ==== osd_op_reply(177 1000000024d.00000000 [call] v129'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.215+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:178 18.9 18:91e366d7::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbea2000 con 0x55c4bbe8fc00 2023-10-11T09:52:23.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.215+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 45 ==== osd_op_reply(178 [pgnls start_epoch 129 out=81b] v129'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.215+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:179 18.9 18:91e366d7:::10000000230.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61000 con 0x55c4bbe8fc00 2023-10-11T09:52:23.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.219+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 46 ==== osd_op_reply(179 10000000230.00000000 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.219+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628336.0 2023-10-11T09:52:23.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.219+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:180 18.9 18:91e366d7:::10000000230.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbead000 con 0x55c4bbe8fc00 2023-10-11T09:52:23.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.219+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 47 ==== osd_op_reply(180 10000000230.00000000 [call] v129'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.219+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:181 18.29 18:94000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbead800 con 0x55c4bbeac000 2023-10-11T09:52:23.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.219+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 6 ==== osd_op_reply(181 [pgnls start_epoch 129 out=101b] v127'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.223+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:182 18.29 18:94c02207:::1000000024a.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeac800 con 0x55c4bbeac000 2023-10-11T09:52:23.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.223+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 7 ==== osd_op_reply(182 1000000024a.00000000 [stat out=16b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.223+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628362.0 2023-10-11T09:52:23.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.223+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:183 18.29 18:94c02207:::1000000024a.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeac400 con 0x55c4bbeac000 2023-10-11T09:52:23.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.227+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 8 ==== osd_op_reply(183 1000000024a.00000000 [call] v129'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.227+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:184 18.29 18:95e8e66d::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeacc00 con 0x55c4bbeac000 2023-10-11T09:52:23.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.227+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 9 ==== osd_op_reply(184 [pgnls start_epoch 129 out=101b] v129'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.227+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:185 18.29 18:95e8e66d:::10000000256.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8ec00 con 0x55c4bbeac000 2023-10-11T09:52:23.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.227+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 10 ==== osd_op_reply(185 10000000256.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.227+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628374.0 2023-10-11T09:52:23.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.227+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:186 18.29 18:95e8e66d:::10000000256.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8f000 con 0x55c4bbeac000 2023-10-11T09:52:23.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.231+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 11 ==== osd_op_reply(186 10000000256.00000000 [call] v129'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.231+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:187 18.29 18:97401066::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8f800 con 0x55c4bbeac000 2023-10-11T09:52:23.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.231+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 12 ==== osd_op_reply(187 [pgnls start_epoch 129 out=81b] v129'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.231+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:188 18.29 18:97401066:::10000000247.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8e400 con 0x55c4bbeac000 2023-10-11T09:52:23.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.231+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 13 ==== osd_op_reply(188 10000000247.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.231+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628359.0 2023-10-11T09:52:23.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.231+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:189 18.29 18:97401066:::10000000247.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8e800 con 0x55c4bbeac000 2023-10-11T09:52:23.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.235+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 14 ==== osd_op_reply(189 10000000247.00000000 [call] v129'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.235+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:190 18.19 18:98000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbea2c00 con 0x55c4bbea3400 2023-10-11T09:52:23.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.235+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 19 ==== osd_op_reply(190 [pgnls start_epoch 129 out=101b] v127'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.235+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:191 18.19 18:992a6246:::10000000200.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbea2400 con 0x55c4bbea3400 2023-10-11T09:52:23.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.239+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 20 ==== osd_op_reply(191 10000000200.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.239+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628288.0 2023-10-11T09:52:23.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.239+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:192 18.19 18:992a6246:::10000000200.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61400 con 0x55c4bbea3400 2023-10-11T09:52:23.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.239+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 21 ==== osd_op_reply(192 10000000200.00000000 [call] v129'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.239+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:193 18.19 18:997b7129::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbead400 con 0x55c4bbea3400 2023-10-11T09:52:23.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.243+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 22 ==== osd_op_reply(193 [pgnls start_epoch 129 out=81b] v129'15 uv15 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.243+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:194 18.19 18:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeadc00 con 0x55c4bbea3400 2023-10-11T09:52:23.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.243+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 23 ==== osd_op_reply(194 100000001f8.00000000 [stat out=16b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.243+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628280.0 2023-10-11T09:52:23.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.243+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:195 18.19 18:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbead800 con 0x55c4bbea3400 2023-10-11T09:52:23.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.247+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 24 ==== osd_op_reply(195 100000001f8.00000000 [call] v129'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.247+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:196 18.39 18:9c000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeac800 con 0x55c4bbcff000 2023-10-11T09:52:23.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.247+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.0 v2:172.21.15.78:6800/35735 31 ==== osd_op_reply(196 [pgnls start_epoch 129 out=81b] v127'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff000 2023-10-11T09:52:23.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.247+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:197 18.39 18:9f1351c3:::10000000222.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe60400 con 0x55c4bbcff000 2023-10-11T09:52:23.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.247+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.0 v2:172.21.15.78:6800/35735 32 ==== osd_op_reply(197 10000000222.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff000 2023-10-11T09:52:23.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.247+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628322.0 2023-10-11T09:52:23.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.247+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:198 18.39 18:9f1351c3:::10000000222.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbcff800 con 0x55c4bbcff000 2023-10-11T09:52:23.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.251+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.0 v2:172.21.15.78:6800/35735 33 ==== osd_op_reply(198 10000000222.00000000 [call] v129'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff000 2023-10-11T09:52:23.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.251+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:199 18.5 18:a0000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8f400 con 0x55c4bbe8e000 2023-10-11T09:52:23.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.251+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.1 v2:172.21.15.78:6809/35734 16 ==== osd_op_reply(199 [pgnls start_epoch 129 out=101b] v127'9 uv9 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbe8e000 2023-10-11T09:52:23.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.251+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:200 18.5 18:a0beda95:::10000000204.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8ec00 con 0x55c4bbe8e000 2023-10-11T09:52:23.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.251+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.1 v2:172.21.15.78:6809/35734 17 ==== osd_op_reply(200 10000000204.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbe8e000 2023-10-11T09:52:23.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.251+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628292.0 2023-10-11T09:52:23.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.251+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:201 18.5 18:a0beda95:::10000000204.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8f000 con 0x55c4bbe8e000 2023-10-11T09:52:23.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.255+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.1 v2:172.21.15.78:6809/35734 18 ==== osd_op_reply(201 10000000204.00000000 [call] v129'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbe8e000 2023-10-11T09:52:23.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.255+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:202 18.5 18:a3a50212::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8f800 con 0x55c4bbe8e000 2023-10-11T09:52:23.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.255+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.1 v2:172.21.15.78:6809/35734 19 ==== osd_op_reply(202 [pgnls start_epoch 129 out=81b] v129'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbe8e000 2023-10-11T09:52:23.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.255+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:203 18.5 18:a3a50212:::10000000254.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8e400 con 0x55c4bbe8e000 2023-10-11T09:52:23.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.255+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.1 v2:172.21.15.78:6809/35734 20 ==== osd_op_reply(203 10000000254.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbe8e000 2023-10-11T09:52:23.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.255+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628372.0 2023-10-11T09:52:23.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.255+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:204 18.5 18:a3a50212:::10000000254.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8e800 con 0x55c4bbe8e000 2023-10-11T09:52:23.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.259+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.1 v2:172.21.15.78:6809/35734 21 ==== osd_op_reply(204 10000000254.00000000 [call] v129'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbe8e000 2023-10-11T09:52:23.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.259+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:205 18.25 18:a4000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8f400 con 0x55c4bbcff400 2023-10-11T09:52:23.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.259+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.6 v2:172.21.15.133:6800/35900 22 ==== osd_op_reply(205 [pgnls start_epoch 129 out=101b] v127'10 uv10 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff400 2023-10-11T09:52:23.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.259+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:206 18.25 18:a53878a9:::10000000237.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeac400 con 0x55c4bbcff400 2023-10-11T09:52:23.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.259+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.6 v2:172.21.15.133:6800/35900 23 ==== osd_op_reply(206 10000000237.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff400 2023-10-11T09:52:23.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.259+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628343.0 2023-10-11T09:52:23.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.259+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:207 18.25 18:a53878a9:::10000000237.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeacc00 con 0x55c4bbcff400 2023-10-11T09:52:23.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.263+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.6 v2:172.21.15.133:6800/35900 24 ==== osd_op_reply(207 10000000237.00000000 [call] v129'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff400 2023-10-11T09:52:23.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.263+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:208 18.25 18:a7ae8a0f::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbee4000 con 0x55c4bbcff400 2023-10-11T09:52:23.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.263+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.6 v2:172.21.15.133:6800/35900 25 ==== osd_op_reply(208 [pgnls start_epoch 129 out=81b] v129'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff400 2023-10-11T09:52:23.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.263+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:209 18.25 18:a7ae8a0f:::10000000243.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbee4400 con 0x55c4bbcff400 2023-10-11T09:52:23.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.263+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.6 v2:172.21.15.133:6800/35900 26 ==== osd_op_reply(209 10000000243.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff400 2023-10-11T09:52:23.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.263+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628355.0 2023-10-11T09:52:23.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.263+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:210 18.25 18:a7ae8a0f:::10000000243.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbee4800 con 0x55c4bbcff400 2023-10-11T09:52:23.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.267+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.6 v2:172.21.15.133:6800/35900 27 ==== osd_op_reply(210 10000000243.00000000 [call] v129'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff400 2023-10-11T09:52:23.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.267+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:211 18.15 18:a8000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbee4c00 con 0x55c4bbe60800 2023-10-11T09:52:23.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.267+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 29 ==== osd_op_reply(211 [pgnls start_epoch 129 out=81b] v127'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.271+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:212 18.15 18:a8e69ea8:::1000000020d.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbee5000 con 0x55c4bbe60800 2023-10-11T09:52:23.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.271+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 30 ==== osd_op_reply(212 1000000020d.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.271+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628301.0 2023-10-11T09:52:23.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.271+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:213 18.15 18:a8e69ea8:::1000000020d.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbee5400 con 0x55c4bbe60800 2023-10-11T09:52:23.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.275+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 31 ==== osd_op_reply(213 1000000020d.00000000 [call] v129'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.275+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:214 18.35 18:ac000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbee5800 con 0x55c4bbea3400 2023-10-11T09:52:23.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.275+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 25 ==== osd_op_reply(214 [pgnls start_epoch 129 out=49b] v127'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.275+0000 7f7764a8f700 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:215 18.d 18:b0000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbea3c00 con 0x55c4bbea3400 2023-10-11T09:52:23.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.275+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 26 ==== osd_op_reply(215 [pgnls start_epoch 129 out=49b] v127'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.275+0000 7f776428e700 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:216 18.2d 18:b4000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe60c00 con 0x55c4bbcff400 2023-10-11T09:52:23.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.275+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.6 v2:172.21.15.133:6800/35900 28 ==== osd_op_reply(216 [pgnls start_epoch 129 out=81b] v127'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff400 2023-10-11T09:52:23.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.275+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:217 18.2d 18:b79a54f6:::10000000218.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbee5c00 con 0x55c4bbcff400 2023-10-11T09:52:23.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.275+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.6 v2:172.21.15.133:6800/35900 29 ==== osd_op_reply(217 10000000218.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff400 2023-10-11T09:52:23.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.279+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628312.0 2023-10-11T09:52:23.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.279+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:218 18.2d 18:b79a54f6:::10000000218.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbea3000 con 0x55c4bbcff400 2023-10-11T09:52:23.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.279+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.6 v2:172.21.15.133:6800/35900 30 ==== osd_op_reply(218 10000000218.00000000 [call] v129'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff400 2023-10-11T09:52:23.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.279+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:219 18.1d 18:b8000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbea2800 con 0x55c4bbcff000 2023-10-11T09:52:23.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.283+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.0 v2:172.21.15.78:6800/35735 34 ==== osd_op_reply(219 [pgnls start_epoch 129 out=81b] v127'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff000 2023-10-11T09:52:23.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.283+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:220 18.1d 18:bab97ecf:::10000000246.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbea2000 con 0x55c4bbcff000 2023-10-11T09:52:23.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.283+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.0 v2:172.21.15.78:6800/35735 35 ==== osd_op_reply(220 10000000246.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff000 2023-10-11T09:52:23.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.283+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628358.0 2023-10-11T09:52:23.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.283+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:221 18.1d 18:bab97ecf:::10000000246.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbea2c00 con 0x55c4bbcff000 2023-10-11T09:52:23.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.287+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.0 v2:172.21.15.78:6800/35735 36 ==== osd_op_reply(221 10000000246.00000000 [call] v129'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbcff000 2023-10-11T09:52:23.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.287+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:222 18.3d 18:bc000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbea2400 con 0x55c4bbe8fc00 2023-10-11T09:52:23.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.287+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 48 ==== osd_op_reply(222 [pgnls start_epoch 129 out=101b] v127'11 uv11 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.287+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:223 18.3d 18:bc0c3701:::10000000219.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbea3c00 con 0x55c4bbe8fc00 2023-10-11T09:52:23.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.287+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 49 ==== osd_op_reply(223 10000000219.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.287+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628313.0 2023-10-11T09:52:23.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.287+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:224 18.3d 18:bc0c3701:::10000000219.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbee4000 con 0x55c4bbe8fc00 2023-10-11T09:52:23.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.291+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 50 ==== osd_op_reply(224 10000000219.00000000 [call] v129'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.291+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:225 18.3d 18:bfcd3380::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbee4400 con 0x55c4bbe8fc00 2023-10-11T09:52:23.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.291+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 51 ==== osd_op_reply(225 [pgnls start_epoch 129 out=81b] v129'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.291+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:226 18.3d 18:bfcd3380:::10000000205.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbee4800 con 0x55c4bbe8fc00 2023-10-11T09:52:23.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.291+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 52 ==== osd_op_reply(226 10000000205.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.291+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628293.0 2023-10-11T09:52:23.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.291+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:227 18.3d 18:bfcd3380:::10000000205.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbee4c00 con 0x55c4bbe8fc00 2023-10-11T09:52:23.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.295+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.3 v2:172.21.15.78:6824/35738 53 ==== osd_op_reply(227 10000000205.00000000 [call] v129'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe8fc00 2023-10-11T09:52:23.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.295+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:228 18.3 18:c0000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbee5000 con 0x55c4bbe8e000 2023-10-11T09:52:23.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.295+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.1 v2:172.21.15.78:6809/35734 22 ==== osd_op_reply(228 [pgnls start_epoch 129 out=49b] v127'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbe8e000 2023-10-11T09:52:23.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.295+0000 7f7764a8f700 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:229 18.23 18:c4000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeea000 con 0x55c4bbea3800 2023-10-11T09:52:23.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.295+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.2 v2:172.21.15.78:6808/35732 17 ==== osd_op_reply(229 [pgnls start_epoch 129 out=49b] v127'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbea3800 2023-10-11T09:52:23.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.295+0000 7f776428e700 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:230 18.13 18:c8000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61800 con 0x55c4bbeac000 2023-10-11T09:52:23.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.299+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 15 ==== osd_op_reply(230 [pgnls start_epoch 129 out=101b] v127'25 uv25 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.299+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:231 18.13 18:c8893608:::1000000023c.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbee5400 con 0x55c4bbeac000 2023-10-11T09:52:23.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.299+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 16 ==== osd_op_reply(231 1000000023c.00000000 [stat out=16b] v0'0 uv24 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.299+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628348.0 2023-10-11T09:52:23.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.299+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:232 18.13 18:c8893608:::1000000023c.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbee5800 con 0x55c4bbeac000 2023-10-11T09:52:23.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.303+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 17 ==== osd_op_reply(232 1000000023c.00000000 [call] v129'26 uv26 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.303+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:233 18.13 18:c933d8fa::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeac800 con 0x55c4bbeac000 2023-10-11T09:52:23.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.303+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 18 ==== osd_op_reply(233 [pgnls start_epoch 129 out=101b] v129'26 uv26 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.303+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:234 18.13 18:c933d8fa:::10000000241.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeac400 con 0x55c4bbeac000 2023-10-11T09:52:23.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.303+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 19 ==== osd_op_reply(234 10000000241.00000000 [stat out=16b] v0'0 uv25 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.303+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628353.0 2023-10-11T09:52:23.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.303+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:235 18.13 18:c933d8fa:::10000000241.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeacc00 con 0x55c4bbeac000 2023-10-11T09:52:23.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.307+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 20 ==== osd_op_reply(235 10000000241.00000000 [call] v129'27 uv27 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.307+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:236 18.13 18:c9855970::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbead000 con 0x55c4bbeac000 2023-10-11T09:52:23.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.307+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 21 ==== osd_op_reply(236 [pgnls start_epoch 129 out=101b] v129'27 uv27 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.307+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:237 18.13 18:c9855970:::10000000239.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbead400 con 0x55c4bbeac000 2023-10-11T09:52:23.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.307+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 22 ==== osd_op_reply(237 10000000239.00000000 [stat out=16b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.307+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628345.0 2023-10-11T09:52:23.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.307+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:238 18.13 18:c9855970:::10000000239.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeadc00 con 0x55c4bbeac000 2023-10-11T09:52:23.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.311+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 23 ==== osd_op_reply(238 10000000239.00000000 [call] v129'28 uv28 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.311+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:239 18.13 18:ca8fdbba::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbead800 con 0x55c4bbeac000 2023-10-11T09:52:23.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.311+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 24 ==== osd_op_reply(239 [pgnls start_epoch 129 out=101b] v129'28 uv28 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.311+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:240 18.13 18:ca8fdbba:::100000001fb.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe60400 con 0x55c4bbeac000 2023-10-11T09:52:23.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.311+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 25 ==== osd_op_reply(240 100000001fb.00000000 [stat out=16b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.311+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628283.0 2023-10-11T09:52:23.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.311+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:241 18.13 18:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe60c00 con 0x55c4bbeac000 2023-10-11T09:52:23.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.315+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 26 ==== osd_op_reply(241 100000001fb.00000000 [call] v129'29 uv29 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.315+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:242 18.13 18:cbcb6224::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61000 con 0x55c4bbeac000 2023-10-11T09:52:23.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.315+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 27 ==== osd_op_reply(242 [pgnls start_epoch 129 out=81b] v129'29 uv29 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.315+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:243 18.13 18:cbcb6224:::10000000212.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61400 con 0x55c4bbeac000 2023-10-11T09:52:23.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.319+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 28 ==== osd_op_reply(243 10000000212.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.319+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628306.0 2023-10-11T09:52:23.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.319+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:244 18.13 18:cbcb6224:::10000000212.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8f400 con 0x55c4bbeac000 2023-10-11T09:52:23.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.319+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 29 ==== osd_op_reply(244 10000000212.00000000 [call] v129'30 uv30 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.319+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:245 18.33 18:cc000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbcff800 con 0x55c4bbe8e000 2023-10-11T09:52:23.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.323+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.1 v2:172.21.15.78:6809/35734 23 ==== osd_op_reply(245 [pgnls start_epoch 129 out=81b] v127'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbe8e000 2023-10-11T09:52:23.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.323+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:246 18.33 18:cf9e7a4f:::10000000248.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeeb400 con 0x55c4bbe8e000 2023-10-11T09:52:23.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.323+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.1 v2:172.21.15.78:6809/35734 24 ==== osd_op_reply(246 10000000248.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbe8e000 2023-10-11T09:52:23.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.323+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628360.0 2023-10-11T09:52:23.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.323+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:247 18.33 18:cf9e7a4f:::10000000248.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeeb800 con 0x55c4bbe8e000 2023-10-11T09:52:23.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.327+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.1 v2:172.21.15.78:6809/35734 25 ==== osd_op_reply(247 10000000248.00000000 [call] v129'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbe8e000 2023-10-11T09:52:23.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.327+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:248 18.b 18:d0000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeebc00 con 0x55c4bbe60800 2023-10-11T09:52:23.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.327+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 32 ==== osd_op_reply(248 [pgnls start_epoch 129 out=101b] v127'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.327+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:249 18.b 18:d0d26e50:::10000000225.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbee5000 con 0x55c4bbe60800 2023-10-11T09:52:23.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.327+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 33 ==== osd_op_reply(249 10000000225.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.327+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628325.0 2023-10-11T09:52:23.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.327+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:250 18.b 18:d0d26e50:::10000000225.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbee5400 con 0x55c4bbe60800 2023-10-11T09:52:23.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.331+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 34 ==== osd_op_reply(250 10000000225.00000000 [call] v129'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.331+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:251 18.b 18:d2dbd4ac::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbee5800 con 0x55c4bbe60800 2023-10-11T09:52:23.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.331+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 35 ==== osd_op_reply(251 [pgnls start_epoch 129 out=101b] v129'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.331+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:252 18.b 18:d2dbd4ac:::10000000217.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe60400 con 0x55c4bbe60800 2023-10-11T09:52:23.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.331+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 36 ==== osd_op_reply(252 10000000217.00000000 [stat out=16b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.331+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628311.0 2023-10-11T09:52:23.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.331+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:253 18.b 18:d2dbd4ac:::10000000217.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe60c00 con 0x55c4bbe60800 2023-10-11T09:52:23.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.335+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 37 ==== osd_op_reply(253 10000000217.00000000 [call] v129'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.335+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:254 18.b 18:d3eda40d::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61000 con 0x55c4bbe60800 2023-10-11T09:52:23.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.335+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 38 ==== osd_op_reply(254 [pgnls start_epoch 129 out=81b] v129'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.335+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:255 18.b 18:d3eda40d:::1000000020f.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61800 con 0x55c4bbe60800 2023-10-11T09:52:23.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.335+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 39 ==== osd_op_reply(255 1000000020f.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.335+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628303.0 2023-10-11T09:52:23.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.335+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:256 18.b 18:d3eda40d:::1000000020f.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8ec00 con 0x55c4bbe60800 2023-10-11T09:52:23.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.339+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 40 ==== osd_op_reply(256 1000000020f.00000000 [call] v129'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.339+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:257 18.2b 18:d4000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8f000 con 0x55c4bbeac000 2023-10-11T09:52:23.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.339+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 30 ==== osd_op_reply(257 [pgnls start_epoch 129 out=101b] v127'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.339+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:258 18.2b 18:d4a58307:::10000000203.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8f800 con 0x55c4bbeac000 2023-10-11T09:52:23.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.343+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 31 ==== osd_op_reply(258 10000000203.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.343+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628291.0 2023-10-11T09:52:23.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.343+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:259 18.2b 18:d4a58307:::10000000203.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8e400 con 0x55c4bbeac000 2023-10-11T09:52:23.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.343+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 32 ==== osd_op_reply(259 10000000203.00000000 [call] v129'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.343+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:260 18.2b 18:d5ee56ee::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8e800 con 0x55c4bbeac000 2023-10-11T09:52:23.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.347+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 33 ==== osd_op_reply(260 [pgnls start_epoch 129 out=81b] v129'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.347+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:261 18.2b 18:d5ee56ee:::1000000022d.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeac800 con 0x55c4bbeac000 2023-10-11T09:52:23.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.347+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 34 ==== osd_op_reply(261 1000000022d.00000000 [stat out=16b] v0'0 uv15 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.347+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628333.0 2023-10-11T09:52:23.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.347+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:262 18.2b 18:d5ee56ee:::1000000022d.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeac400 con 0x55c4bbeac000 2023-10-11T09:52:23.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.347+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 35 ==== osd_op_reply(262 1000000022d.00000000 [call] v129'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.347+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:263 18.1b 18:d8000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeacc00 con 0x55c4bbe8e000 2023-10-11T09:52:23.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.351+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.1 v2:172.21.15.78:6809/35734 26 ==== osd_op_reply(263 [pgnls start_epoch 129 out=49b] v127'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbe8e000 2023-10-11T09:52:23.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.351+0000 7f776428e700 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:264 18.3b 18:dc000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61c00 con 0x55c4bbea3400 2023-10-11T09:52:23.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.351+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 27 ==== osd_op_reply(264 [pgnls start_epoch 129 out=101b] v127'10 uv10 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.351+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:265 18.3b 18:dd50df43:::10000000249.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbead000 con 0x55c4bbea3400 2023-10-11T09:52:23.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.351+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 28 ==== osd_op_reply(265 10000000249.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.351+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628361.0 2023-10-11T09:52:23.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.351+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:266 18.3b 18:dd50df43:::10000000249.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbead400 con 0x55c4bbea3400 2023-10-11T09:52:23.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.355+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 29 ==== osd_op_reply(266 10000000249.00000000 [call] v129'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.355+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:267 18.3b 18:deccbd37::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeadc00 con 0x55c4bbea3400 2023-10-11T09:52:23.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.355+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 30 ==== osd_op_reply(267 [pgnls start_epoch 129 out=101b] v129'11 uv11 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.355+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:268 18.3b 18:deccbd37:::100000001fe.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbead800 con 0x55c4bbea3400 2023-10-11T09:52:23.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.359+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 31 ==== osd_op_reply(268 100000001fe.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.359+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628286.0 2023-10-11T09:52:23.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.359+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:269 18.3b 18:deccbd37:::100000001fe.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8f400 con 0x55c4bbea3400 2023-10-11T09:52:23.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.359+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 32 ==== osd_op_reply(269 100000001fe.00000000 [call] v129'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.359+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:270 18.3b 18:ded14b84::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8f000 con 0x55c4bbea3400 2023-10-11T09:52:23.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.363+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 33 ==== osd_op_reply(270 [pgnls start_epoch 129 out=81b] v129'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.363+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:271 18.3b 18:ded14b84:::1000000024b.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8f800 con 0x55c4bbea3400 2023-10-11T09:52:23.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.363+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 34 ==== osd_op_reply(271 1000000024b.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.363+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628363.0 2023-10-11T09:52:23.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.363+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:272 18.3b 18:ded14b84:::1000000024b.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8e400 con 0x55c4bbea3400 2023-10-11T09:52:23.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.367+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 35 ==== osd_op_reply(272 1000000024b.00000000 [call] v129'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.367+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:273 18.7 18:e0000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8e800 con 0x55c4bbea3800 2023-10-11T09:52:23.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.367+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.2 v2:172.21.15.78:6808/35732 18 ==== osd_op_reply(273 [pgnls start_epoch 129 out=101b] v127'12 uv12 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbea3800 2023-10-11T09:52:23.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.367+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:274 18.7 18:e15a914f:::10000000207.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8ec00 con 0x55c4bbea3800 2023-10-11T09:52:23.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.367+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.2 v2:172.21.15.78:6808/35732 19 ==== osd_op_reply(274 10000000207.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbea3800 2023-10-11T09:52:23.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.367+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628295.0 2023-10-11T09:52:23.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.367+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:275 18.7 18:e15a914f:::10000000207.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61400 con 0x55c4bbea3800 2023-10-11T09:52:23.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.371+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.2 v2:172.21.15.78:6808/35732 20 ==== osd_op_reply(275 10000000207.00000000 [call] v129'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbea3800 2023-10-11T09:52:23.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.371+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:276 18.7 18:e2e07930::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe60400 con 0x55c4bbea3800 2023-10-11T09:52:23.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.371+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.2 v2:172.21.15.78:6808/35732 21 ==== osd_op_reply(276 [pgnls start_epoch 129 out=101b] v129'13 uv13 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbea3800 2023-10-11T09:52:23.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.371+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:277 18.7 18:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe60c00 con 0x55c4bbea3800 2023-10-11T09:52:23.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.371+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.2 v2:172.21.15.78:6808/35732 22 ==== osd_op_reply(277 10000000001.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbea3800 2023-10-11T09:52:23.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.371+0000 7f776aadc840 10 datascan.operator(): handling object 1099511627777.0 2023-10-11T09:52:23.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.371+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:278 18.7 18:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61000 con 0x55c4bbea3800 2023-10-11T09:52:23.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.375+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.2 v2:172.21.15.78:6808/35732 23 ==== osd_op_reply(278 10000000001.00000000 [call] v129'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbea3800 2023-10-11T09:52:23.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.375+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:279 18.7 18:e37752f2::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61800 con 0x55c4bbea3800 2023-10-11T09:52:23.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.375+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.2 v2:172.21.15.78:6808/35732 24 ==== osd_op_reply(279 [pgnls start_epoch 129 out=81b] v129'14 uv14 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbea3800 2023-10-11T09:52:23.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.375+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:280 18.7 18:e37752f2:::1000000021f.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbee4000 con 0x55c4bbea3800 2023-10-11T09:52:23.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.375+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.2 v2:172.21.15.78:6808/35732 25 ==== osd_op_reply(280 1000000021f.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbea3800 2023-10-11T09:52:23.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.375+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628319.0 2023-10-11T09:52:23.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.375+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:281 18.7 18:e37752f2:::1000000021f.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbee4400 con 0x55c4bbea3800 2023-10-11T09:52:23.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.379+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 <== osd.2 v2:172.21.15.78:6808/35732 26 ==== osd_op_reply(281 1000000021f.00000000 [call] v129'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eafc0 con 0x55c4bbea3800 2023-10-11T09:52:23.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.379+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:282 18.27 18:e4000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbee4800 con 0x55c4bbe60800 2023-10-11T09:52:23.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.379+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 41 ==== osd_op_reply(282 [pgnls start_epoch 129 out=101b] v127'12 uv12 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.379+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:283 18.27 18:e40c9c52:::10000000210.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbee4c00 con 0x55c4bbe60800 2023-10-11T09:52:23.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.379+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 42 ==== osd_op_reply(283 10000000210.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.379+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628304.0 2023-10-11T09:52:23.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.379+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:284 18.27 18:e40c9c52:::10000000210.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbee5000 con 0x55c4bbe60800 2023-10-11T09:52:23.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.383+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 43 ==== osd_op_reply(284 10000000210.00000000 [call] v129'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.383+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:285 18.27 18:e5671758::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbee5400 con 0x55c4bbe60800 2023-10-11T09:52:23.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.383+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 44 ==== osd_op_reply(285 [pgnls start_epoch 129 out=101b] v129'13 uv13 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.383+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:286 18.27 18:e5671758:::1000000024f.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbee5800 con 0x55c4bbe60800 2023-10-11T09:52:23.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.383+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 45 ==== osd_op_reply(286 1000000024f.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.387+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628367.0 2023-10-11T09:52:23.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.387+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:287 18.27 18:e5671758:::1000000024f.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeeb400 con 0x55c4bbe60800 2023-10-11T09:52:23.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.387+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 46 ==== osd_op_reply(287 1000000024f.00000000 [call] v129'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.387+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:288 18.27 18:e5993619::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeeb800 con 0x55c4bbe60800 2023-10-11T09:52:23.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.387+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 47 ==== osd_op_reply(288 [pgnls start_epoch 129 out=101b] v129'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.391+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:289 18.27 18:e5993619:::10000000208.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeebc00 con 0x55c4bbe60800 2023-10-11T09:52:23.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.391+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 48 ==== osd_op_reply(289 10000000208.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.391+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628296.0 2023-10-11T09:52:23.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.391+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:290 18.27 18:e5993619:::10000000208.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbea2400 con 0x55c4bbe60800 2023-10-11T09:52:23.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.391+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 49 ==== osd_op_reply(290 10000000208.00000000 [call] v129'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.391+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:291 18.27 18:e5a5749e::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbea3c00 con 0x55c4bbe60800 2023-10-11T09:52:23.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.395+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 50 ==== osd_op_reply(291 [pgnls start_epoch 129 out=101b] v129'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.395+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:292 18.27 18:e5a5749e:::1000000021a.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeac800 con 0x55c4bbe60800 2023-10-11T09:52:23.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.395+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 51 ==== osd_op_reply(292 1000000021a.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.395+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628314.0 2023-10-11T09:52:23.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.395+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:293 18.27 18:e5a5749e:::1000000021a.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeac400 con 0x55c4bbe60800 2023-10-11T09:52:23.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.395+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 52 ==== osd_op_reply(293 1000000021a.00000000 [call] v129'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.399+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:294 18.27 18:e692fc64::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbcff800 con 0x55c4bbe60800 2023-10-11T09:52:23.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.399+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 53 ==== osd_op_reply(294 [pgnls start_epoch 129 out=81b] v129'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.399+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:295 18.27 18:e692fc64:::1000000021b.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61c00 con 0x55c4bbe60800 2023-10-11T09:52:23.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.399+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 54 ==== osd_op_reply(295 1000000021b.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.399+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628315.0 2023-10-11T09:52:23.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.399+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:296 18.27 18:e692fc64:::1000000021b.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61400 con 0x55c4bbe60800 2023-10-11T09:52:23.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.403+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 55 ==== osd_op_reply(296 1000000021b.00000000 [call] v129'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.403+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:297 18.17 18:e8000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe60400 con 0x55c4bbe60800 2023-10-11T09:52:23.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.403+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 56 ==== osd_op_reply(297 [pgnls start_epoch 129 out=101b] v127'13 uv13 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.403+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:298 18.17 18:eace0b58:::10000000215.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe60c00 con 0x55c4bbe60800 2023-10-11T09:52:23.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.403+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 57 ==== osd_op_reply(298 10000000215.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.403+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628309.0 2023-10-11T09:52:23.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.403+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:299 18.17 18:eace0b58:::10000000215.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61000 con 0x55c4bbe60800 2023-10-11T09:52:23.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.407+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 58 ==== osd_op_reply(299 10000000215.00000000 [call] v129'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.407+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:300 18.17 18:eb1b33a4::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe61800 con 0x55c4bbe60800 2023-10-11T09:52:23.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.407+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 59 ==== osd_op_reply(300 [pgnls start_epoch 129 out=81b] v129'14 uv14 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.407+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:301 18.17 18:eb1b33a4:::10000000253.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8f400 con 0x55c4bbe60800 2023-10-11T09:52:23.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.407+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 60 ==== osd_op_reply(301 10000000253.00000000 [stat out=16b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.407+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628371.0 2023-10-11T09:52:23.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.407+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:302 18.17 18:eb1b33a4:::10000000253.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8f000 con 0x55c4bbe60800 2023-10-11T09:52:23.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.411+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 61 ==== osd_op_reply(302 10000000253.00000000 [call] v129'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.411+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:303 18.37 18:ec000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8f800 con 0x55c4bbeac000 2023-10-11T09:52:23.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.411+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 36 ==== osd_op_reply(303 [pgnls start_epoch 129 out=49b] v127'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.411+0000 7f776428e700 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:304 18.f 18:f0000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbf04000 con 0x55c4bbe60800 2023-10-11T09:52:23.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.415+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 62 ==== osd_op_reply(304 [pgnls start_epoch 129 out=81b] v127'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.415+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:305 18.f 18:f25dd5df:::1000000022e.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8e400 con 0x55c4bbe60800 2023-10-11T09:52:23.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.415+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 63 ==== osd_op_reply(305 1000000022e.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.415+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628334.0 2023-10-11T09:52:23.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.415+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:306 18.f 18:f25dd5df:::1000000022e.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8e800 con 0x55c4bbe60800 2023-10-11T09:52:23.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.419+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.5 v2:172.21.15.133:6816/35899 64 ==== osd_op_reply(306 1000000022e.00000000 [call] v129'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbe60800 2023-10-11T09:52:23.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.419+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:307 18.2f 18:f4000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbe8ec00 con 0x55c4bbe8e000 2023-10-11T09:52:23.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.419+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.1 v2:172.21.15.78:6809/35734 27 ==== osd_op_reply(307 [pgnls start_epoch 129 out=49b] v127'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbe8e000 2023-10-11T09:52:23.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.419+0000 7f776428e700 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:308 18.1f 18:f8000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbf04400 con 0x55c4bbeac000 2023-10-11T09:52:23.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.419+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 37 ==== osd_op_reply(308 [pgnls start_epoch 129 out=101b] v127'10 uv10 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.419+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:309 18.1f 18:f87ea8f1:::10000000238.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeeb400 con 0x55c4bbeac000 2023-10-11T09:52:23.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.419+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 38 ==== osd_op_reply(309 10000000238.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.419+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628344.0 2023-10-11T09:52:23.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.419+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:310 18.1f 18:f87ea8f1:::10000000238.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeeb800 con 0x55c4bbeac000 2023-10-11T09:52:23.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.423+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 39 ==== osd_op_reply(310 10000000238.00000000 [call] v129'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.423+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:311 18.1f 18:f8e2d5cc::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbeea000 con 0x55c4bbeac000 2023-10-11T09:52:23.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.427+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 40 ==== osd_op_reply(311 [pgnls start_epoch 129 out=81b] v129'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.427+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:312 18.1f 18:f8e2d5cc:::10000000214.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbea3000 con 0x55c4bbeac000 2023-10-11T09:52:23.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.427+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 41 ==== osd_op_reply(312 10000000214.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.427+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628308.0 2023-10-11T09:52:23.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.427+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:313 18.1f 18:f8e2d5cc:::10000000214.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbea2800 con 0x55c4bbeac000 2023-10-11T09:52:23.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.431+0000 7f7768a97700 1 -- 172.21.15.78:0/2472870157 <== osd.7 v2:172.21.15.133:6824/35901 42 ==== osd_op_reply(313 10000000214.00000000 [call] v129'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb440 con 0x55c4bbeac000 2023-10-11T09:52:23.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.431+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:314 18.3f 18:fc000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbea2000 con 0x55c4bbea3400 2023-10-11T09:52:23.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.431+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 36 ==== osd_op_reply(314 [pgnls start_epoch 129 out=101b] v127'12 uv12 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.431+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:315 18.3f 18:fd83cc12:::10000000232.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbea2c00 con 0x55c4bbea3400 2023-10-11T09:52:23.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.431+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 37 ==== osd_op_reply(315 10000000232.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.431+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628338.0 2023-10-11T09:52:23.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.431+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:316 18.3f 18:fd83cc12:::10000000232.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbee4800 con 0x55c4bbea3400 2023-10-11T09:52:23.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.435+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 38 ==== osd_op_reply(316 10000000232.00000000 [call] v129'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.435+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:317 18.3f 18:feb1c803::::head [pgnls start_epoch 129 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbee4c00 con 0x55c4bbea3400 2023-10-11T09:52:23.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.435+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 39 ==== osd_op_reply(317 [pgnls start_epoch 129 out=81b] v129'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.435+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:318 18.3f 18:feb1c803:::100000001fa.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbee5000 con 0x55c4bbea3400 2023-10-11T09:52:23.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.435+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 40 ==== osd_op_reply(318 100000001fa.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.435+0000 7f776aadc840 10 datascan.operator(): handling object 1099511628282.0 2023-10-11T09:52:23.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.435+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:319 18.3f 18:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x55c4bbee5400 con 0x55c4bbea3400 2023-10-11T09:52:23.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.439+0000 7f7769a99700 1 -- 172.21.15.78:0/2472870157 <== osd.4 v2:172.21.15.133:6808/35898 41 ==== osd_op_reply(319 100000001fa.00000000 [call] v129'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4bc7eb200 con 0x55c4bbea3400 2023-10-11T09:52:23.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.439+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55c4bbcff000 msgr2=0x55c4bbaba100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:23.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.439+0000 7f776aadc840 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x55c4bbcff000 0x55c4bbaba100 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:23.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.439+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55c4bbe8e000 msgr2=0x55c4bbe94000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:23.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.439+0000 7f776aadc840 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x55c4bbe8e000 0x55c4bbe94000 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:23.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.439+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55c4bbea3800 msgr2=0x55c4bbe94580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:23.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.439+0000 7f776aadc840 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x55c4bbea3800 0x55c4bbe94580 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:23.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.439+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55c4bbe8fc00 msgr2=0x55c4bbe96680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:23.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.439+0000 7f776aadc840 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x55c4bbe8fc00 0x55c4bbe96680 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:23.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.439+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55c4bbea3400 msgr2=0x55c4bbe94b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:23.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.439+0000 7f776aadc840 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x55c4bbea3400 0x55c4bbe94b00 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:23.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.439+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55c4bbe60800 msgr2=0x55c4bbab9b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:23.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.439+0000 7f776aadc840 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x55c4bbe60800 0x55c4bbab9b80 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:23.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.439+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55c4bbcff400 msgr2=0x55c4bbab9600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:23.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.439+0000 7f776aadc840 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55c4bbcff400 0x55c4bbab9600 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:23.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.439+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55c4bbeac000 msgr2=0x55c4bbe96c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:23.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.439+0000 7f776aadc840 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x55c4bbeac000 0x55c4bbe96c00 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:23.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.439+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 reap_dead start 2023-10-11T09:52:23.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.439+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55c4bbe60000 msgr2=0x55c4bbabb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:23.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.439+0000 7f776aadc840 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x55c4bbe60000 0x55c4bbabb700 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x55c4bc77b2f0 tx=0x55c4bbd12d80 comp rx=0 tx=0).stop 2023-10-11T09:52:23.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.439+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 reap_dead start 2023-10-11T09:52:23.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.439+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 reap_dead start 2023-10-11T09:52:23.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.439+0000 7f7769298700 1 -- 172.21.15.78:0/2472870157 reap_dead start 2023-10-11T09:52:23.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.439+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4bbcfec00 msgr2=0x55c4bbaba680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:23.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.439+0000 7f776aadc840 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4bbcfec00 0x55c4bbaba680 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x55c4bc7f1470 tx=0x55c4bbd12cc0 comp rx=0 tx=0).stop 2023-10-11T09:52:23.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.439+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 shutdown_connections 2023-10-11T09:52:23.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.439+0000 7f776aadc840 1 --2- 172.21.15.78:0/2472870157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4bbcfec00 0x55c4bbaba680 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:23.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.443+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 >> 172.21.15.78:0/2472870157 conn(0x55c4bba12c00 msgr2=0x55c4bbae1800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:23.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.443+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 shutdown_connections 2023-10-11T09:52:23.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.443+0000 7f776aadc840 1 -- 172.21.15.78:0/2472870157 wait complete. 2023-10-11T09:52:23.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.443+0000 7f776aadc840 1 -- 172.21.15.78:0/2866718319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4bba13400 msgr2=0x55c4bbab8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:23.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.443+0000 7f776aadc840 1 --2- 172.21.15.78:0/2866718319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4bba13400 0x55c4bbab8b00 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x55c4bc77b410 tx=0x55c4bbb08f60 comp rx=0 tx=0).stop 2023-10-11T09:52:23.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.443+0000 7f776aadc840 1 -- 172.21.15.78:0/2866718319 shutdown_connections 2023-10-11T09:52:23.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.443+0000 7f776aadc840 1 --2- 172.21.15.78:0/2866718319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c4bba13000 0x55c4bbab9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:23.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.443+0000 7f776aadc840 1 --2- 172.21.15.78:0/2866718319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4bba13400 0x55c4bbab8b00 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:23.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.443+0000 7f776aadc840 1 --2- 172.21.15.78:0/2866718319 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c4bba13800 0x55c4bbab8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:23.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.443+0000 7f776aadc840 1 -- 172.21.15.78:0/2866718319 >> 172.21.15.78:0/2866718319 conn(0x55c4bba13c00 msgr2=0x55c4bbae1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:23.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.443+0000 7f776aadc840 1 -- 172.21.15.78:0/2866718319 shutdown_connections 2023-10-11T09:52:23.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.443+0000 7f776aadc840 1 -- 172.21.15.78:0/2866718319 wait complete. 2023-10-11T09:52:23.461 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents'] in time 0:00:00.578523, result: 2023-10-11T09:52:23.462 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes 2023-10-11T09:52:23.570 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.563+0000 7f239a687840 1 -- start start 2023-10-11T09:52:23.571 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.563+0000 7f239a687840 1 Processor -- start 2023-10-11T09:52:23.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.563+0000 7f239a687840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e8ad27800 0x561e8adcc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:23.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.563+0000 7f239a687840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8ad27400 0x561e8adccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:23.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.563+0000 7f239a687840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x561e8ad27000 0x561e8adcd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:23.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.563+0000 7f239a687840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e8ada6a00 con 0x561e8ad27800 2023-10-11T09:52:23.573 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.563+0000 7f239a687840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e8ada6780 con 0x561e8ad27400 2023-10-11T09:52:23.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.563+0000 7f239a687840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e8ada6640 con 0x561e8ad27000 2023-10-11T09:52:23.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.563+0000 7f2398e43700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e8ad27800 0x561e8adcc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:23.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.563+0000 7f2398e43700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e8ad27800 0x561e8adcc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47700/0 (socket says 172.21.15.78:47700) 2023-10-11T09:52:23.574 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.563+0000 7f2398e43700 1 -- 172.21.15.78:0/2275839196 learned_addr learned my addr 172.21.15.78:0/2275839196 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:23.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.563+0000 7f2399644700 1 --2- 172.21.15.78:0/2275839196 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x561e8ad27000 0x561e8adcd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:23.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.563+0000 7f2398642700 1 --2- 172.21.15.78:0/2275839196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8ad27400 0x561e8adccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:23.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.563+0000 7f2398e43700 1 -- 172.21.15.78:0/2275839196 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x561e8ad27000 msgr2=0x561e8adcd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:23.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.563+0000 7f2398e43700 1 --2- 172.21.15.78:0/2275839196 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x561e8ad27000 0x561e8adcd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:23.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.563+0000 7f2398e43700 1 -- 172.21.15.78:0/2275839196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8ad27400 msgr2=0x561e8adccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:23.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.563+0000 7f2398e43700 1 --2- 172.21.15.78:0/2275839196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8ad27400 0x561e8adccb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:23.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.563+0000 7f2398e43700 1 -- 172.21.15.78:0/2275839196 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e8ad48b60 con 0x561e8ad27800 2023-10-11T09:52:23.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.563+0000 7f2399644700 1 --2- 172.21.15.78:0/2275839196 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x561e8ad27000 0x561e8adcd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:23.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.567+0000 7f2398e43700 1 --2- 172.21.15.78:0/2275839196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e8ad27800 0x561e8adcc580 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x561e8adf5320 tx=0x561e8ae8c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d5e16d4c30fae3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:23.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.567+0000 7f239663e700 1 -- 172.21.15.78:0/2275839196 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561e8ad57080 con 0x561e8ad27800 2023-10-11T09:52:23.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.567+0000 7f239663e700 1 -- 172.21.15.78:0/2275839196 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561e8bb14300 con 0x561e8ad27800 2023-10-11T09:52:23.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.567+0000 7f239663e700 1 -- 172.21.15.78:0/2275839196 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561e8ad57600 con 0x561e8ad27800 2023-10-11T09:52:23.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.567+0000 7f239a687840 1 -- 172.21.15.78:0/2275839196 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e8ad491e0 con 0x561e8ad27800 2023-10-11T09:52:23.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.567+0000 7f239a687840 1 -- 172.21.15.78:0/2275839196 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x561e8ad48b60 con 0x561e8ad27800 2023-10-11T09:52:23.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.567+0000 7f239a687840 4 waiting for MDS map... 2023-10-11T09:52:23.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.567+0000 7f239663e700 1 -- 172.21.15.78:0/2275839196 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(129..129 src has 1..129) v4 ==== 6234+0+0 (secure 0 0 0) 0x561e8ad60c40 con 0x561e8ad27800 2023-10-11T09:52:23.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.567+0000 7f239663e700 1 -- 172.21.15.78:0/2275839196 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 106) v1 ==== 2565+0+0 (secure 0 0 0) 0x561e8ad68f00 con 0x561e8ad27800 2023-10-11T09:52:23.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.567+0000 7f239a687840 4 Got MDS map 106 2023-10-11T09:52:23.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.567+0000 7f239a687840 4 datascan.main: Using metadata pool output 2023-10-11T09:52:23.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.567+0000 7f239a687840 4 datascan.main: connecting to RADOS... 2023-10-11T09:52:23.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.567+0000 7f239a687840 1 Processor -- start 2023-10-11T09:52:23.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.567+0000 7f239a687840 1 -- start start 2023-10-11T09:52:23.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.567+0000 7f239a687840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e8b012400 0x561e8adcd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:23.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.567+0000 7f239a687840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x561e8b012800 0x561e8adcdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:23.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.567+0000 7f239a687840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8b012c00 0x561e8adce100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:23.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.571+0000 7f239a687840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e8ad5de00 con 0x561e8b012400 2023-10-11T09:52:23.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.571+0000 7f239a687840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e8ada6780 con 0x561e8b012c00 2023-10-11T09:52:23.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.571+0000 7f239a687840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e8ada6a00 con 0x561e8b012800 2023-10-11T09:52:23.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.571+0000 7f2399644700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e8b012400 0x561e8adcd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:23.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.571+0000 7f2399644700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e8b012400 0x561e8adcd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47706/0 (socket says 172.21.15.78:47706) 2023-10-11T09:52:23.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.571+0000 7f2399644700 1 -- 172.21.15.78:0/3486623404 learned_addr learned my addr 172.21.15.78:0/3486623404 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:23.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.571+0000 7f2398e43700 1 --2- 172.21.15.78:0/3486623404 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x561e8b012800 0x561e8adcdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:23.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.571+0000 7f2398642700 1 --2- 172.21.15.78:0/3486623404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8b012c00 0x561e8adce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:23.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.571+0000 7f2399644700 1 -- 172.21.15.78:0/3486623404 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x561e8b012800 msgr2=0x561e8adcdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:23.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.571+0000 7f2399644700 1 --2- 172.21.15.78:0/3486623404 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x561e8b012800 0x561e8adcdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:23.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.571+0000 7f2399644700 1 -- 172.21.15.78:0/3486623404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8b012c00 msgr2=0x561e8adce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:23.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.571+0000 7f2399644700 1 --2- 172.21.15.78:0/3486623404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8b012c00 0x561e8adce100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:23.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.571+0000 7f2399644700 1 -- 172.21.15.78:0/3486623404 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e8ad48340 con 0x561e8b012400 2023-10-11T09:52:23.585 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.571+0000 7f2398e43700 1 --2- 172.21.15.78:0/3486623404 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x561e8b012800 0x561e8adcdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:52:23.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.571+0000 7f2398642700 1 --2- 172.21.15.78:0/3486623404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8b012c00 0x561e8adce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:52:23.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.571+0000 7f2399644700 1 --2- 172.21.15.78:0/3486623404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e8b012400 0x561e8adcd600 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x561e8b02c690 tx=0x561e8b02ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c432d7803afc24bd server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:23.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.571+0000 7f239463a700 1 -- 172.21.15.78:0/3486623404 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561e8ad57080 con 0x561e8b012400 2023-10-11T09:52:23.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.571+0000 7f239463a700 1 -- 172.21.15.78:0/3486623404 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561e8ad72900 con 0x561e8b012400 2023-10-11T09:52:23.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.571+0000 7f239463a700 1 -- 172.21.15.78:0/3486623404 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561e8ad57600 con 0x561e8b012400 2023-10-11T09:52:23.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.571+0000 7f239a687840 1 -- 172.21.15.78:0/3486623404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e8b012400 msgr2=0x561e8adcd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:23.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.571+0000 7f239a687840 1 --2- 172.21.15.78:0/3486623404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e8b012400 0x561e8adcd600 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x561e8b02c690 tx=0x561e8b02ec00 comp rx=0 tx=0).stop 2023-10-11T09:52:23.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.571+0000 7f239a687840 1 -- 172.21.15.78:0/3486623404 shutdown_connections 2023-10-11T09:52:23.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.571+0000 7f239a687840 1 --2- 172.21.15.78:0/3486623404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8b012c00 0x561e8adce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:23.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.571+0000 7f239a687840 1 --2- 172.21.15.78:0/3486623404 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x561e8b012800 0x561e8adcdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:23.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.571+0000 7f239a687840 1 --2- 172.21.15.78:0/3486623404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e8b012400 0x561e8adcd600 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:23.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.571+0000 7f239a687840 1 -- 172.21.15.78:0/3486623404 >> 172.21.15.78:0/3486623404 conn(0x561e8ad26c00 msgr2=0x561e8ae17800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:23.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.571+0000 7f239a687840 1 -- 172.21.15.78:0/3486623404 shutdown_connections 2023-10-11T09:52:23.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.571+0000 7f239a687840 1 -- 172.21.15.78:0/3486623404 wait complete. 2023-10-11T09:52:23.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.571+0000 7f239a687840 1 Processor -- start 2023-10-11T09:52:23.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.571+0000 7f239a687840 1 -- start start 2023-10-11T09:52:23.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.575+0000 7f239a687840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x561e8b012400 0x561e8adcf180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:23.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.575+0000 7f239a687840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8b012800 0x561e8adcec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:23.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.575+0000 7f239a687840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e8b012c00 0x561e8adce680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:23.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.575+0000 7f239a687840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e8ada6500 con 0x561e8b012c00 2023-10-11T09:52:23.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.575+0000 7f239a687840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e8ada63c0 con 0x561e8b012800 2023-10-11T09:52:23.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.575+0000 7f239a687840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e8ada6280 con 0x561e8b012400 2023-10-11T09:52:23.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.575+0000 7f2398642700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e8b012c00 0x561e8adce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:23.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.575+0000 7f2398642700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e8b012c00 0x561e8adce680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47720/0 (socket says 172.21.15.78:47720) 2023-10-11T09:52:23.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.575+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 learned_addr learned my addr 172.21.15.78:0/612779174 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:23.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.575+0000 7f2399644700 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x561e8b012400 0x561e8adcf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:23.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.575+0000 7f2398e43700 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8b012800 0x561e8adcec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:23.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.575+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x561e8b012400 msgr2=0x561e8adcf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:23.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.575+0000 7f2398642700 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x561e8b012400 0x561e8adcf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:23.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.575+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8b012800 msgr2=0x561e8adcec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:23.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.575+0000 7f2398642700 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8b012800 0x561e8adcec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:23.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.575+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e8ad48000 con 0x561e8b012c00 2023-10-11T09:52:23.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.575+0000 7f2398642700 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e8b012c00 0x561e8adce680 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x561e8b02d350 tx=0x561e8b02ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:23.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.575+0000 7f2392e37700 1 -- 172.21.15.78:0/612779174 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561e8ad57080 con 0x561e8b012c00 2023-10-11T09:52:23.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.575+0000 7f2392e37700 1 -- 172.21.15.78:0/612779174 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561e8ad72480 con 0x561e8b012c00 2023-10-11T09:52:23.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.575+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561e8ad48ea0 con 0x561e8b012c00 2023-10-11T09:52:23.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.575+0000 7f2392e37700 1 -- 172.21.15.78:0/612779174 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561e8ad57600 con 0x561e8b012c00 2023-10-11T09:52:23.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.575+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e8ad49520 con 0x561e8b012c00 2023-10-11T09:52:23.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.575+0000 7f239a687840 4 datascan.init: resolving metadata pool 17 2023-10-11T09:52:23.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.579+0000 7f2392e37700 1 -- 172.21.15.78:0/612779174 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x561e8ad84840 con 0x561e8b012c00 2023-10-11T09:52:23.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.579+0000 7f2392e37700 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x561e8b182000 0x561e8adcf700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:23.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.579+0000 7f2392e37700 1 -- 172.21.15.78:0/612779174 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6234+0+0 (secure 0 0 0) 0x561e8ad60a80 con 0x561e8b012c00 2023-10-11T09:52:23.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.579+0000 7f239a687840 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-10-11T09:52:23.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.579+0000 7f239a687840 4 datascan.main: data pool 'cephfs_data' has ID 18 2023-10-11T09:52:23.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.579+0000 7f239a687840 4 datascan.main: opening data pool 'cephfs_data' 2023-10-11T09:52:23.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.579+0000 7f239a687840 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x561e8b013000 0x561e8adce100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:23.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.579+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:1 17.14 17:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b013400 con 0x561e8b013000 2023-10-11T09:52:23.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.579+0000 7f2399644700 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x561e8b182000 0x561e8adcf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:23.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.579+0000 7f2398e43700 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x561e8b013000 0x561e8adce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:23.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.583+0000 7f2399644700 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x561e8b182000 0x561e8adcf700 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x561e8b02c690 tx=0x561e8ae8cf60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:23.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.583+0000 7f2398e43700 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x561e8b013000 0x561e8adce100 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:23.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.583+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv19 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.583+0000 7f239a687840 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x561e8b013800 0x561e8adcdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:23.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.583+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:2 17.1f 17:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b182400 con 0x561e8b013800 2023-10-11T09:52:23.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.583+0000 7f2399644700 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x561e8b013800 0x561e8adcdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:23.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.583+0000 7f2399644700 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x561e8b013800 0x561e8adcdb80 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:23.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.583+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.583+0000 7f239a687840 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x561e8b182800 0x561e8adcd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:23.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.583+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:3 18.0 18:00000000::::head [pgnls-filter start_epoch 129 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b182c00 con 0x561e8b182800 2023-10-11T09:52:23.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.583+0000 7f2398e43700 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x561e8b182800 0x561e8adcd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:23.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.587+0000 7f2398e43700 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x561e8b182800 0x561e8adcd600 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:23.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.587+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.0 v2:172.21.15.78:6800/35735 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 129 out=81b] v129'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b182800 2023-10-11T09:52:23.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.587+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:4 18.0 18:00000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183000 con 0x561e8b182800 2023-10-11T09:52:23.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.587+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.0 v2:172.21.15.78:6800/35735 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 129 out=81b] v129'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b182800 2023-10-11T09:52:23.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.587+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000206.00000000 2023-10-11T09:52:23.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.587+0000 7f239a687840 10 datascan.operator(): handling object 10000000206.0 2023-10-11T09:52:23.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.587+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:5 18.0 18:012cdbc7:::10000000206.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183400 con 0x561e8b182800 2023-10-11T09:52:23.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.591+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.0 v2:172.21.15.78:6800/35735 3 ==== osd_op_reply(5 10000000206.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b182800 2023-10-11T09:52:23.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.591+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000206 2023-10-11T09:52:23.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.591+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/17 2023-10-11T09:52:23.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.591+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=17 2023-10-11T09:52:23.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.591+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:6 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183800 con 0x561e8b013000 2023-10-11T09:52:23.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.591+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 2 ==== osd_op_reply(6 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.591+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.591+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:7 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183c00 con 0x561e8b013800 2023-10-11T09:52:23.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.591+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 2 ==== osd_op_reply(7 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.591+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.591+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.591+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:23.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.591+0000 7f239a687840 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x561e8b182400 0x561e8b19c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:23.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.591+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:8 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183c00 con 0x561e8b182400 2023-10-11T09:52:23.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.591+0000 7f2398642700 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x561e8b182400 0x561e8b19c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:23.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.591+0000 7f2398642700 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x561e8b182400 0x561e8b19c580 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:23.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.595+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 1 ==== osd_op_reply(8 10000000000.01800000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.595+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 0 2023-10-11T09:52:23.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.595+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:9 17.1b 17:dcc776e0:::10000000000.01800000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b182c00 con 0x561e8b182400 2023-10-11T09:52:23.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.595+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 2 ==== osd_op_reply(9 10000000000.01800000 [create,omap-set-header] v129'6 uv6 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.595+0000 7f239a687840 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x10000000000 2023-10-11T09:52:23.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.595+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:10 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183000 con 0x561e8b182400 2023-10-11T09:52:23.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.599+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 3 ==== osd_op_reply(10 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.599+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=17 frag=1*, r=0 2023-10-11T09:52:23.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.599+0000 7f239a687840 20 datascan.read_dentry: 17_head not found in result 2023-10-11T09:52:23.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.599+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000206 at 0x0x10000000000/17 with size=2 bytes 2023-10-11T09:52:23.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.599+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:11 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b013400 con 0x561e8b182400 2023-10-11T09:52:23.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.599+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 4 ==== osd_op_reply(11 10000000000.01800000 [omap-set-vals] v129'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.599+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/17 pointing to 0x0x10000000206 2023-10-11T09:52:23.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.599+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x1/subdir 2023-10-11T09:52:23.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.599+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x1 target_dname=subdir 2023-10-11T09:52:23.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.599+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:12 17.1f 17:ff5b34d6:::1.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183c00 con 0x561e8b013800 2023-10-11T09:52:23.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.599+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 3 ==== osd_op_reply(12 1.00000000 [getxattr (30) out=30b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+30 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.599+0000 7f239a687840 4 datascan.get_frag_of: No backtrace on '1.00000000': cannot determine fragtree 2023-10-11T09:52:23.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.599+0000 7f239a687840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x1/subdir, will insert to root 2023-10-11T09:52:23.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.599+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:13 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b182c00 con 0x561e8b013800 2023-10-11T09:52:23.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.603+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 4 ==== osd_op_reply(13 1.00000000 [omap-get-header out=274b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.603+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 6 2023-10-11T09:52:23.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.603+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x1 * 2023-10-11T09:52:23.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.603+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:14 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183000 con 0x561e8b013800 2023-10-11T09:52:23.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.603+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 5 ==== osd_op_reply(14 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.603+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.603+0000 7f239a687840 20 datascan.inject_with_backtrace: Dentry 0x0x1/subdir already exists and points to me 2023-10-11T09:52:23.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.603+0000 7f239a687840 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x561e8b183c00 0x561e8b19cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:23.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.603+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:15 18.20 18:04000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b182c00 con 0x561e8b183c00 2023-10-11T09:52:23.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.603+0000 7f2399644700 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x561e8b183c00 0x561e8b19cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:23.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.603+0000 7f2399644700 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x561e8b183c00 0x561e8b19cb00 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:23.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.607+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 1 ==== osd_op_reply(15 [pgnls-filter start_epoch 129 out=81b] v129'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:23.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.607+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 1000000020b.00000000 2023-10-11T09:52:23.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.607+0000 7f239a687840 10 datascan.operator(): handling object 1000000020b.0 2023-10-11T09:52:23.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.607+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:16 18.20 18:05552577:::1000000020b.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183000 con 0x561e8b183c00 2023-10-11T09:52:23.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.607+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 2 ==== osd_op_reply(16 1000000020b.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:23.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.607+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x1000000020b 2023-10-11T09:52:23.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.607+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/22 2023-10-11T09:52:23.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.607+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=22 2023-10-11T09:52:23.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.607+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:17 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b2000 con 0x561e8b013000 2023-10-11T09:52:23.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.607+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 3 ==== osd_op_reply(17 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.607+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.607+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:18 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b2400 con 0x561e8b013800 2023-10-11T09:52:23.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.607+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 6 ==== osd_op_reply(18 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.607+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.607+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.607+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:23.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.607+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:19 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b2800 con 0x561e8b182400 2023-10-11T09:52:23.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.611+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 5 ==== osd_op_reply(19 10000000000.01800000 [omap-get-header out=274b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.611+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 7 2023-10-11T09:52:23.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.611+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:23.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.611+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:20 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b2c00 con 0x561e8b182400 2023-10-11T09:52:23.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.611+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 6 ==== osd_op_reply(20 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.611+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=22 frag=1*, r=0 2023-10-11T09:52:23.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.611+0000 7f239a687840 20 datascan.read_dentry: 22_head not found in result 2023-10-11T09:52:23.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.611+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000020b at 0x0x10000000000/22 with size=2 bytes 2023-10-11T09:52:23.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.611+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:21 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b3000 con 0x561e8b182400 2023-10-11T09:52:23.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.611+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 7 ==== osd_op_reply(21 10000000000.01800000 [omap-set-vals] v129'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.611+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/22 pointing to 0x0x1000000020b 2023-10-11T09:52:23.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.611+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:22 18.10 18:08000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b3400 con 0x561e8b013800 2023-10-11T09:52:23.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.615+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 7 ==== osd_op_reply(22 [pgnls-filter start_epoch 129 out=101b] v129'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.615+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 1000000022a.00000000 2023-10-11T09:52:23.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.615+0000 7f239a687840 10 datascan.operator(): handling object 1000000022a.0 2023-10-11T09:52:23.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.615+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:23 18.10 18:083d5826:::1000000022a.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b3800 con 0x561e8b013800 2023-10-11T09:52:23.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.615+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 8 ==== osd_op_reply(23 1000000022a.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.615+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x1000000022a 2023-10-11T09:52:23.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.615+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/53 2023-10-11T09:52:23.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.615+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=53 2023-10-11T09:52:23.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.615+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:24 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b3c00 con 0x561e8b013000 2023-10-11T09:52:23.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.615+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 4 ==== osd_op_reply(24 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.615+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.615+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:25 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b2000 con 0x561e8b013800 2023-10-11T09:52:23.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.615+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 9 ==== osd_op_reply(25 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.615+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.615+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.615+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:23.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.615+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:26 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b3c00 con 0x561e8b013800 2023-10-11T09:52:23.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.619+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 10 ==== osd_op_reply(26 10000000000.01000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.619+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 0 2023-10-11T09:52:23.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.619+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:27 17.5 17:a40620de:::10000000000.01000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b2400 con 0x561e8b013800 2023-10-11T09:52:23.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.619+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 11 ==== osd_op_reply(27 10000000000.01000000 [create,omap-set-header] v129'5 uv5 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.619+0000 7f239a687840 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x10000000000 2023-10-11T09:52:23.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.619+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:28 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b3400 con 0x561e8b013800 2023-10-11T09:52:23.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.623+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 12 ==== osd_op_reply(28 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.623+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=53 frag=0*, r=0 2023-10-11T09:52:23.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.623+0000 7f239a687840 20 datascan.read_dentry: 53_head not found in result 2023-10-11T09:52:23.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.623+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000022a at 0x0x10000000000/53 with size=2 bytes 2023-10-11T09:52:23.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.623+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:29 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b2800 con 0x561e8b013800 2023-10-11T09:52:23.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.623+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 13 ==== osd_op_reply(29 10000000000.01000000 [omap-set-vals] v129'6 uv6 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.623+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/53 pointing to 0x0x1000000022a 2023-10-11T09:52:23.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.623+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x1/subdir 2023-10-11T09:52:23.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.623+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x1 target_dname=subdir 2023-10-11T09:52:23.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.623+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:30 17.1f 17:ff5b34d6:::1.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b2c00 con 0x561e8b013800 2023-10-11T09:52:23.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.623+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 14 ==== osd_op_reply(30 1.00000000 [getxattr (30) out=30b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+30 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.623+0000 7f239a687840 4 datascan.get_frag_of: No backtrace on '1.00000000': cannot determine fragtree 2023-10-11T09:52:23.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.623+0000 7f239a687840 20 datascan.inject_with_backtrace: don't know fragment for 0x0x1/subdir, will insert to root 2023-10-11T09:52:23.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.623+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:31 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b3000 con 0x561e8b013800 2023-10-11T09:52:23.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.627+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 15 ==== osd_op_reply(31 1.00000000 [omap-get-header out=274b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.627+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 6 2023-10-11T09:52:23.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.627+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x1 * 2023-10-11T09:52:23.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.627+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:32 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b013400 con 0x561e8b013800 2023-10-11T09:52:23.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.627+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 16 ==== osd_op_reply(32 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.627+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.627+0000 7f239a687840 20 datascan.inject_with_backtrace: Dentry 0x0x1/subdir already exists and points to me 2023-10-11T09:52:23.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.627+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:33 18.10 18:08500202::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183400 con 0x561e8b013800 2023-10-11T09:52:23.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.627+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 17 ==== osd_op_reply(33 [pgnls-filter start_epoch 129 out=101b] v129'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.627+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 1000000023e.00000000 2023-10-11T09:52:23.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.627+0000 7f239a687840 10 datascan.operator(): handling object 1000000023e.0 2023-10-11T09:52:23.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.627+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:34 18.10 18:08500202:::1000000023e.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183800 con 0x561e8b013800 2023-10-11T09:52:23.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.627+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 18 ==== osd_op_reply(34 1000000023e.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.627+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x1000000023e 2023-10-11T09:52:23.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.627+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/73 2023-10-11T09:52:23.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.627+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=73 2023-10-11T09:52:23.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.627+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:35 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b182c00 con 0x561e8b013000 2023-10-11T09:52:23.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.631+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 5 ==== osd_op_reply(35 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.631+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.631+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:36 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183000 con 0x561e8b013800 2023-10-11T09:52:23.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.631+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 19 ==== osd_op_reply(36 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.631+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.631+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.631+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:23.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.631+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:37 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183400 con 0x561e8b013800 2023-10-11T09:52:23.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.631+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 20 ==== osd_op_reply(37 10000000000.01000000 [omap-get-header out=274b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.631+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 6 2023-10-11T09:52:23.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.631+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:23.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.631+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:38 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183800 con 0x561e8b013800 2023-10-11T09:52:23.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.631+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 21 ==== osd_op_reply(38 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.631+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=73 frag=0*, r=0 2023-10-11T09:52:23.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.631+0000 7f239a687840 20 datascan.read_dentry: 73_head not found in result 2023-10-11T09:52:23.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.631+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000023e at 0x0x10000000000/73 with size=2 bytes 2023-10-11T09:52:23.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.631+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:39 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b013400 con 0x561e8b013800 2023-10-11T09:52:23.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.635+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 22 ==== osd_op_reply(39 10000000000.01000000 [omap-set-vals] v129'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.635+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/73 pointing to 0x0x1000000023e 2023-10-11T09:52:23.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.635+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:40 18.10 18:08b99fea::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34000 con 0x561e8b013800 2023-10-11T09:52:23.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.635+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 23 ==== osd_op_reply(40 [pgnls-filter start_epoch 129 out=101b] v129'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.635+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 100000001fd.00000000 2023-10-11T09:52:23.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.635+0000 7f239a687840 10 datascan.operator(): handling object 100000001fd.0 2023-10-11T09:52:23.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.635+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:41 18.10 18:08b99fea:::100000001fd.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34400 con 0x561e8b013800 2023-10-11T09:52:23.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.635+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 24 ==== osd_op_reply(41 100000001fd.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.635+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x100000001fd 2023-10-11T09:52:23.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.635+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/8 2023-10-11T09:52:23.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.635+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=8 2023-10-11T09:52:23.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.635+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:42 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34800 con 0x561e8b013000 2023-10-11T09:52:23.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.635+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 6 ==== osd_op_reply(42 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.635+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.635+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:43 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34c00 con 0x561e8b013800 2023-10-11T09:52:23.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.639+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 25 ==== osd_op_reply(43 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.639+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.639+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.639+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:23.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.639+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:44 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35000 con 0x561e8b013800 2023-10-11T09:52:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.639+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 26 ==== osd_op_reply(44 10000000000.01000000 [omap-get-header out=274b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.639+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 7 2023-10-11T09:52:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.639+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.639+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:45 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35400 con 0x561e8b013800 2023-10-11T09:52:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.639+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 27 ==== osd_op_reply(45 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.639+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=8 frag=0*, r=0 2023-10-11T09:52:23.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.639+0000 7f239a687840 20 datascan.read_dentry: 8_head not found in result 2023-10-11T09:52:23.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.639+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fd at 0x0x10000000000/8 with size=1 bytes 2023-10-11T09:52:23.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.639+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:46 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35800 con 0x561e8b013800 2023-10-11T09:52:23.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.643+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 28 ==== osd_op_reply(46 10000000000.01000000 [omap-set-vals] v129'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.643+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/8 pointing to 0x0x100000001fd 2023-10-11T09:52:23.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.643+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:47 18.10 18:08e6f8ca::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35c00 con 0x561e8b013800 2023-10-11T09:52:23.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.643+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 29 ==== osd_op_reply(47 [pgnls-filter start_epoch 129 out=81b] v129'18 uv18 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.643+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000257.00000000 2023-10-11T09:52:23.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.643+0000 7f239a687840 10 datascan.operator(): handling object 10000000257.0 2023-10-11T09:52:23.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.643+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:48 18.10 18:08e6f8ca:::10000000257.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34000 con 0x561e8b013800 2023-10-11T09:52:23.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.643+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 30 ==== osd_op_reply(48 10000000257.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv18 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.643+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000257 2023-10-11T09:52:23.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.643+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/98 2023-10-11T09:52:23.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.643+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=98 2023-10-11T09:52:23.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.643+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:49 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34400 con 0x561e8b013000 2023-10-11T09:52:23.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.643+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 7 ==== osd_op_reply(49 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.643+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.643+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:50 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34c00 con 0x561e8b013800 2023-10-11T09:52:23.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.647+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 31 ==== osd_op_reply(50 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.647+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.647+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.647+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:23.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.647+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:51 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b013400 con 0x561e8b182400 2023-10-11T09:52:23.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.647+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 8 ==== osd_op_reply(51 10000000000.01800000 [omap-get-header out=274b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.647+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 8 2023-10-11T09:52:23.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.647+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:23.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.647+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:52 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183000 con 0x561e8b182400 2023-10-11T09:52:23.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.647+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 9 ==== osd_op_reply(52 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.647+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=98 frag=1*, r=0 2023-10-11T09:52:23.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.647+0000 7f239a687840 20 datascan.read_dentry: 98_head not found in result 2023-10-11T09:52:23.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.647+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000257 at 0x0x10000000000/98 with size=2 bytes 2023-10-11T09:52:23.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.647+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:53 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183400 con 0x561e8b182400 2023-10-11T09:52:23.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.651+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 10 ==== osd_op_reply(53 10000000000.01800000 [omap-set-vals] v129'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.651+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/98 pointing to 0x0x10000000257 2023-10-11T09:52:23.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.651+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:54 18.30 18:0c000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183800 con 0x561e8b182800 2023-10-11T09:52:23.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.651+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.0 v2:172.21.15.78:6800/35735 4 ==== osd_op_reply(54 [pgnls-filter start_epoch 129 out=101b] v129'21 uv21 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b182800 2023-10-11T09:52:23.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.651+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 1000000021d.00000000 2023-10-11T09:52:23.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.651+0000 7f239a687840 10 datascan.operator(): handling object 1000000021d.0 2023-10-11T09:52:23.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.651+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:55 18.30 18:0c00cb85:::1000000021d.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1c2000 con 0x561e8b182800 2023-10-11T09:52:23.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.651+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.0 v2:172.21.15.78:6800/35735 5 ==== osd_op_reply(55 1000000021d.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv18 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b182800 2023-10-11T09:52:23.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.651+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x1000000021d 2023-10-11T09:52:23.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.651+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/40 2023-10-11T09:52:23.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.651+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=40 2023-10-11T09:52:23.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.651+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:56 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1c2400 con 0x561e8b013000 2023-10-11T09:52:23.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.651+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 8 ==== osd_op_reply(56 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.651+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.651+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:57 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1c2800 con 0x561e8b013800 2023-10-11T09:52:23.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.655+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 32 ==== osd_op_reply(57 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.655+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.655+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.655+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:23.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.655+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:58 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1c2c00 con 0x561e8b182400 2023-10-11T09:52:23.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.655+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 11 ==== osd_op_reply(58 10000000000.01800000 [omap-get-header out=274b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.655+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 9 2023-10-11T09:52:23.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.655+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:23.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.655+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:59 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1c3000 con 0x561e8b182400 2023-10-11T09:52:23.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.655+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 12 ==== osd_op_reply(59 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.655+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=40 frag=1*, r=0 2023-10-11T09:52:23.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.655+0000 7f239a687840 20 datascan.read_dentry: 40_head not found in result 2023-10-11T09:52:23.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.655+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000021d at 0x0x10000000000/40 with size=2 bytes 2023-10-11T09:52:23.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.655+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:60 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1c3400 con 0x561e8b182400 2023-10-11T09:52:23.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.659+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 13 ==== osd_op_reply(60 10000000000.01800000 [omap-set-vals] v129'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.659+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/40 pointing to 0x0x1000000021d 2023-10-11T09:52:23.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.659+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:61 18.30 18:0c28cce1::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1c3800 con 0x561e8b182800 2023-10-11T09:52:23.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.659+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.0 v2:172.21.15.78:6800/35735 6 ==== osd_op_reply(61 [pgnls-filter start_epoch 129 out=101b] v129'21 uv21 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b182800 2023-10-11T09:52:23.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.659+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000244.00000000 2023-10-11T09:52:23.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.659+0000 7f239a687840 10 datascan.operator(): handling object 10000000244.0 2023-10-11T09:52:23.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.659+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:62 18.30 18:0c28cce1:::10000000244.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1c3c00 con 0x561e8b182800 2023-10-11T09:52:23.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.659+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.0 v2:172.21.15.78:6800/35735 7 ==== osd_op_reply(62 10000000244.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv19 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b182800 2023-10-11T09:52:23.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.659+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000244 2023-10-11T09:52:23.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.659+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/79 2023-10-11T09:52:23.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.659+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=79 2023-10-11T09:52:23.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.659+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:63 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1c2c00 con 0x561e8b013000 2023-10-11T09:52:23.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.663+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 9 ==== osd_op_reply(63 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.663+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.663+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:64 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1c3000 con 0x561e8b013800 2023-10-11T09:52:23.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.663+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 33 ==== osd_op_reply(64 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.663+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.663+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.663+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:23.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.663+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:65 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1c2000 con 0x561e8b182400 2023-10-11T09:52:23.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.663+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 14 ==== osd_op_reply(65 10000000000.01800000 [omap-get-header out=274b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.663+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 10 2023-10-11T09:52:23.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.663+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:23.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.663+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:66 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183000 con 0x561e8b182400 2023-10-11T09:52:23.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.663+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 15 ==== osd_op_reply(66 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.663+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=79 frag=1*, r=0 2023-10-11T09:52:23.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.663+0000 7f239a687840 20 datascan.read_dentry: 79_head not found in result 2023-10-11T09:52:23.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.663+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000244 at 0x0x10000000000/79 with size=2 bytes 2023-10-11T09:52:23.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.663+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:67 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183400 con 0x561e8b182400 2023-10-11T09:52:23.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.667+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 16 ==== osd_op_reply(67 10000000000.01800000 [omap-set-vals] v129'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.667+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/79 pointing to 0x0x10000000244 2023-10-11T09:52:23.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.667+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:68 18.30 18:0cc9e906::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b182c00 con 0x561e8b182800 2023-10-11T09:52:23.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.667+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.0 v2:172.21.15.78:6800/35735 8 ==== osd_op_reply(68 [pgnls-filter start_epoch 129 out=101b] v129'21 uv21 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b182800 2023-10-11T09:52:23.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.667+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000227.00000000 2023-10-11T09:52:23.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.667+0000 7f239a687840 10 datascan.operator(): handling object 10000000227.0 2023-10-11T09:52:23.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.667+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:69 18.30 18:0cc9e906:::10000000227.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183800 con 0x561e8b182800 2023-10-11T09:52:23.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.667+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.0 v2:172.21.15.78:6800/35735 9 ==== osd_op_reply(69 10000000227.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv20 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b182800 2023-10-11T09:52:23.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.667+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000227 2023-10-11T09:52:23.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.667+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/50 2023-10-11T09:52:23.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.667+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=50 2023-10-11T09:52:23.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.667+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:70 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34800 con 0x561e8b013000 2023-10-11T09:52:23.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.671+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 10 ==== osd_op_reply(70 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.671+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.671+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:71 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34400 con 0x561e8b013800 2023-10-11T09:52:23.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.671+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 34 ==== osd_op_reply(71 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.671+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.671+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.680 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.671+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:23.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.671+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:72 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b013400 con 0x561e8b013800 2023-10-11T09:52:23.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.671+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 35 ==== osd_op_reply(72 10000000000.01000000 [omap-get-header out=274b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.671+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 8 2023-10-11T09:52:23.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.671+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:23.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.671+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:73 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1c2400 con 0x561e8b013800 2023-10-11T09:52:23.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.671+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 36 ==== osd_op_reply(73 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.671+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=50 frag=0*, r=0 2023-10-11T09:52:23.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.671+0000 7f239a687840 20 datascan.read_dentry: 50_head not found in result 2023-10-11T09:52:23.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.671+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000227 at 0x0x10000000000/50 with size=2 bytes 2023-10-11T09:52:23.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.671+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:74 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1c3800 con 0x561e8b013800 2023-10-11T09:52:23.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.675+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 37 ==== osd_op_reply(74 10000000000.01000000 [omap-set-vals] v129'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.675+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/50 pointing to 0x0x10000000227 2023-10-11T09:52:23.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.675+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:75 18.30 18:0e3d5a27::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1c3c00 con 0x561e8b182800 2023-10-11T09:52:23.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.675+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.0 v2:172.21.15.78:6800/35735 10 ==== osd_op_reply(75 [pgnls-filter start_epoch 129 out=81b] v129'21 uv21 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b182800 2023-10-11T09:52:23.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.675+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000250.00000000 2023-10-11T09:52:23.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.675+0000 7f239a687840 10 datascan.operator(): handling object 10000000250.0 2023-10-11T09:52:23.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.675+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:76 18.30 18:0e3d5a27:::10000000250.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1c2c00 con 0x561e8b182800 2023-10-11T09:52:23.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.679+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.0 v2:172.21.15.78:6800/35735 11 ==== osd_op_reply(76 10000000250.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv21 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b182800 2023-10-11T09:52:23.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.679+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000250 2023-10-11T09:52:23.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.679+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/91 2023-10-11T09:52:23.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.679+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=91 2023-10-11T09:52:23.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.679+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:77 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1c2800 con 0x561e8b013000 2023-10-11T09:52:23.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.679+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 11 ==== osd_op_reply(77 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.679+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.679+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:78 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1c3000 con 0x561e8b013800 2023-10-11T09:52:23.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.679+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 38 ==== osd_op_reply(78 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.679+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.679+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.679+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:23.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.679+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:79 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35000 con 0x561e8b182400 2023-10-11T09:52:23.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.679+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 17 ==== osd_op_reply(79 10000000000.01800000 [omap-get-header out=274b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.679+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 11 2023-10-11T09:52:23.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.679+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:23.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.679+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:80 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35400 con 0x561e8b182400 2023-10-11T09:52:23.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.683+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 18 ==== osd_op_reply(80 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.683+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=91 frag=1*, r=0 2023-10-11T09:52:23.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.683+0000 7f239a687840 20 datascan.read_dentry: 91_head not found in result 2023-10-11T09:52:23.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.683+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000250 at 0x0x10000000000/91 with size=2 bytes 2023-10-11T09:52:23.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.683+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:81 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35800 con 0x561e8b182400 2023-10-11T09:52:23.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.683+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 19 ==== osd_op_reply(81 10000000000.01800000 [omap-set-vals] v129'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.683+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/91 pointing to 0x0x10000000250 2023-10-11T09:52:23.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.683+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:82 18.8 18:10000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35c00 con 0x561e8b182800 2023-10-11T09:52:23.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.687+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.0 v2:172.21.15.78:6800/35735 12 ==== osd_op_reply(82 [pgnls-filter start_epoch 129 out=49b] v127'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b182800 2023-10-11T09:52:23.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.687+0000 7f239463a700 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:83 18.28 18:14000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1c3400 con 0x561e8b183c00 2023-10-11T09:52:23.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.687+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 3 ==== osd_op_reply(83 [pgnls-filter start_epoch 129 out=101b] v129'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:23.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.687+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000233.00000000 2023-10-11T09:52:23.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.687+0000 7f239a687840 10 datascan.operator(): handling object 10000000233.0 2023-10-11T09:52:23.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.687+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:84 18.28 18:14ce3b86:::10000000233.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34000 con 0x561e8b183c00 2023-10-11T09:52:23.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.687+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 4 ==== osd_op_reply(84 10000000233.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:23.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.687+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000233 2023-10-11T09:52:23.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.687+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/62 2023-10-11T09:52:23.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.687+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=62 2023-10-11T09:52:23.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.687+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:85 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34c00 con 0x561e8b013000 2023-10-11T09:52:23.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.687+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 12 ==== osd_op_reply(85 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.687+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.687+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:86 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b182c00 con 0x561e8b013800 2023-10-11T09:52:23.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.691+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 39 ==== osd_op_reply(86 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.691+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.691+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.691+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:23.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.691+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:87 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183800 con 0x561e8b182400 2023-10-11T09:52:23.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.691+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 20 ==== osd_op_reply(87 10000000000.01800000 [omap-get-header out=274b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.691+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 12 2023-10-11T09:52:23.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.691+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:23.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.691+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:88 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35000 con 0x561e8b182400 2023-10-11T09:52:23.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.691+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 21 ==== osd_op_reply(88 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.691+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=62 frag=1*, r=0 2023-10-11T09:52:23.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.691+0000 7f239a687840 20 datascan.read_dentry: 62_head not found in result 2023-10-11T09:52:23.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.691+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000233 at 0x0x10000000000/62 with size=2 bytes 2023-10-11T09:52:23.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.691+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:89 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35400 con 0x561e8b182400 2023-10-11T09:52:23.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.695+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 22 ==== osd_op_reply(89 10000000000.01800000 [omap-set-vals] v129'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.695+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/62 pointing to 0x0x10000000233 2023-10-11T09:52:23.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.695+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:90 18.28 18:15246602::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183000 con 0x561e8b183c00 2023-10-11T09:52:23.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.695+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 5 ==== osd_op_reply(90 [pgnls-filter start_epoch 129 out=101b] v129'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:23.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.695+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000202.00000000 2023-10-11T09:52:23.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.695+0000 7f239a687840 10 datascan.operator(): handling object 10000000202.0 2023-10-11T09:52:23.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.695+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:91 18.28 18:15246602:::10000000202.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183400 con 0x561e8b183c00 2023-10-11T09:52:23.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.695+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 6 ==== osd_op_reply(91 10000000202.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:23.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.695+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000202 2023-10-11T09:52:23.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.695+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/13 2023-10-11T09:52:23.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.695+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=13 2023-10-11T09:52:23.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.695+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:92 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1c2000 con 0x561e8b013000 2023-10-11T09:52:23.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.699+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 13 ==== osd_op_reply(92 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.699+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.699+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:93 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b182c00 con 0x561e8b013800 2023-10-11T09:52:23.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.699+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 40 ==== osd_op_reply(93 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.699+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.699+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.699+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:23.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.699+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:94 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183000 con 0x561e8b182400 2023-10-11T09:52:23.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.699+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 23 ==== osd_op_reply(94 10000000000.01800000 [omap-get-header out=274b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.699+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 13 2023-10-11T09:52:23.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.699+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:23.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.699+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:95 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34800 con 0x561e8b182400 2023-10-11T09:52:23.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.699+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 24 ==== osd_op_reply(95 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.699+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=13 frag=1*, r=0 2023-10-11T09:52:23.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.699+0000 7f239a687840 20 datascan.read_dentry: 13_head not found in result 2023-10-11T09:52:23.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.699+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000202 at 0x0x10000000000/13 with size=2 bytes 2023-10-11T09:52:23.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.699+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:96 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35c00 con 0x561e8b182400 2023-10-11T09:52:23.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.703+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 25 ==== osd_op_reply(96 10000000000.01800000 [omap-set-vals] v129'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.703+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/13 pointing to 0x0x10000000202 2023-10-11T09:52:23.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.703+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:97 18.28 18:1537fa38::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34c00 con 0x561e8b183c00 2023-10-11T09:52:23.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.703+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 7 ==== osd_op_reply(97 [pgnls-filter start_epoch 129 out=101b] v129'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:23.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.703+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000236.00000000 2023-10-11T09:52:23.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.703+0000 7f239a687840 10 datascan.operator(): handling object 10000000236.0 2023-10-11T09:52:23.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.703+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:98 18.28 18:1537fa38:::10000000236.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34400 con 0x561e8b183c00 2023-10-11T09:52:23.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.707+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 8 ==== osd_op_reply(98 10000000236.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:23.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.707+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000236 2023-10-11T09:52:23.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.707+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/65 2023-10-11T09:52:23.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.707+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=65 2023-10-11T09:52:23.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.707+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:99 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34000 con 0x561e8b013000 2023-10-11T09:52:23.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.707+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 14 ==== osd_op_reply(99 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.707+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.707+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:100 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b013400 con 0x561e8b013800 2023-10-11T09:52:23.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.707+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 41 ==== osd_op_reply(100 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.707+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.707+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.707+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:23.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.707+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:101 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35800 con 0x561e8b013800 2023-10-11T09:52:23.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.707+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 42 ==== osd_op_reply(101 10000000000.01000000 [omap-get-header out=274b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.707+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 9 2023-10-11T09:52:23.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.707+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:23.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.707+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:102 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35000 con 0x561e8b013800 2023-10-11T09:52:23.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.711+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 43 ==== osd_op_reply(102 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.711+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=65 frag=0*, r=0 2023-10-11T09:52:23.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.711+0000 7f239a687840 20 datascan.read_dentry: 65_head not found in result 2023-10-11T09:52:23.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.711+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000236 at 0x0x10000000000/65 with size=2 bytes 2023-10-11T09:52:23.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.711+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:103 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35400 con 0x561e8b013800 2023-10-11T09:52:23.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.715+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 44 ==== osd_op_reply(103 10000000000.01000000 [omap-set-vals] v129'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.715+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/65 pointing to 0x0x10000000236 2023-10-11T09:52:23.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.715+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:104 18.28 18:177a762d::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34800 con 0x561e8b183c00 2023-10-11T09:52:23.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.715+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 9 ==== osd_op_reply(104 [pgnls-filter start_epoch 129 out=81b] v129'18 uv18 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:23.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.715+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000201.00000000 2023-10-11T09:52:23.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.715+0000 7f239a687840 10 datascan.operator(): handling object 10000000201.0 2023-10-11T09:52:23.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.715+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:105 18.28 18:177a762d:::10000000201.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35c00 con 0x561e8b183c00 2023-10-11T09:52:23.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.715+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 10 ==== osd_op_reply(105 10000000201.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv18 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:23.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.715+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000201 2023-10-11T09:52:23.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.715+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/12 2023-10-11T09:52:23.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.715+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=12 2023-10-11T09:52:23.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.715+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:106 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183800 con 0x561e8b013000 2023-10-11T09:52:23.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.715+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 15 ==== osd_op_reply(106 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.715+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.715+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:107 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183000 con 0x561e8b013800 2023-10-11T09:52:23.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.715+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 45 ==== osd_op_reply(107 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.715+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.715+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.715+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:23.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.715+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:108 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34c00 con 0x561e8b013800 2023-10-11T09:52:23.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.719+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 46 ==== osd_op_reply(108 10000000000.01000000 [omap-get-header out=274b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.719+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 10 2023-10-11T09:52:23.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.719+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:23.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.719+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:109 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34400 con 0x561e8b013800 2023-10-11T09:52:23.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.719+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 47 ==== osd_op_reply(109 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.719+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=12 frag=0*, r=0 2023-10-11T09:52:23.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.719+0000 7f239a687840 20 datascan.read_dentry: 12_head not found in result 2023-10-11T09:52:23.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.719+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000201 at 0x0x10000000000/12 with size=2 bytes 2023-10-11T09:52:23.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.719+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:110 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35800 con 0x561e8b013800 2023-10-11T09:52:23.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.723+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 48 ==== osd_op_reply(110 10000000000.01000000 [omap-set-vals] v129'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.723+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/12 pointing to 0x0x10000000201 2023-10-11T09:52:23.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.723+0000 7f239a687840 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x561e8ae35000 0x561e8b19c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:23.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.723+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:111 18.18 18:18000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35400 con 0x561e8ae35000 2023-10-11T09:52:23.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.723+0000 7f2398e43700 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x561e8ae35000 0x561e8b19c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:23.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.723+0000 7f2398e43700 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x561e8ae35000 0x561e8b19c000 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:23.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.723+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.1 v2:172.21.15.78:6809/35734 1 ==== osd_op_reply(111 [pgnls-filter start_epoch 129 out=49b] v127'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8ae35000 2023-10-11T09:52:23.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.723+0000 7f2393e39700 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:112 18.38 18:1c000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2000 con 0x561e8b013000 2023-10-11T09:52:23.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.727+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 16 ==== osd_op_reply(112 [pgnls-filter start_epoch 129 out=49b] v127'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.727+0000 7f239463a700 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:113 18.4 18:20000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2400 con 0x561e8b182800 2023-10-11T09:52:23.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.727+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.0 v2:172.21.15.78:6800/35735 13 ==== osd_op_reply(113 [pgnls-filter start_epoch 129 out=81b] v129'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b182800 2023-10-11T09:52:23.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.727+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000220.00000000 2023-10-11T09:52:23.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.727+0000 7f239a687840 10 datascan.operator(): handling object 10000000220.0 2023-10-11T09:52:23.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.727+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:114 18.4 18:2231e764:::10000000220.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34800 con 0x561e8b182800 2023-10-11T09:52:23.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.727+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.0 v2:172.21.15.78:6800/35735 14 ==== osd_op_reply(114 10000000220.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b182800 2023-10-11T09:52:23.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.727+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000220 2023-10-11T09:52:23.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.727+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/43 2023-10-11T09:52:23.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.727+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=43 2023-10-11T09:52:23.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.727+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:115 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35c00 con 0x561e8b013000 2023-10-11T09:52:23.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.727+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 17 ==== osd_op_reply(115 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.727+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.727+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:116 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b013400 con 0x561e8b013800 2023-10-11T09:52:23.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.727+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 49 ==== osd_op_reply(116 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.727+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.727+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.727+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:23.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.727+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:117 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183400 con 0x561e8b013800 2023-10-11T09:52:23.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.731+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 50 ==== osd_op_reply(117 10000000000.01000000 [omap-get-header out=274b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.731+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 11 2023-10-11T09:52:23.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.731+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:23.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.731+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:118 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b182c00 con 0x561e8b013800 2023-10-11T09:52:23.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.731+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 51 ==== osd_op_reply(118 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.731+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=43 frag=0*, r=0 2023-10-11T09:52:23.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.731+0000 7f239a687840 20 datascan.read_dentry: 43_head not found in result 2023-10-11T09:52:23.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.731+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000220 at 0x0x10000000000/43 with size=2 bytes 2023-10-11T09:52:23.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.731+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:119 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2c00 con 0x561e8b013800 2023-10-11T09:52:23.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.735+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 52 ==== osd_op_reply(119 10000000000.01000000 [omap-set-vals] v129'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.735+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/43 pointing to 0x0x10000000220 2023-10-11T09:52:23.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.735+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:120 18.24 18:24000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3000 con 0x561e8b182800 2023-10-11T09:52:23.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.735+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.0 v2:172.21.15.78:6800/35735 15 ==== osd_op_reply(120 [pgnls-filter start_epoch 129 out=101b] v129'10 uv10 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b182800 2023-10-11T09:52:23.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.735+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000234.00000000 2023-10-11T09:52:23.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.735+0000 7f239a687840 10 datascan.operator(): handling object 10000000234.0 2023-10-11T09:52:23.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.735+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:121 18.24 18:242ba651:::10000000234.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3400 con 0x561e8b182800 2023-10-11T09:52:23.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.735+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.0 v2:172.21.15.78:6800/35735 16 ==== osd_op_reply(121 10000000234.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv9 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b182800 2023-10-11T09:52:23.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.735+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000234 2023-10-11T09:52:23.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.735+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/63 2023-10-11T09:52:23.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.735+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=63 2023-10-11T09:52:23.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.735+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:122 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3800 con 0x561e8b013000 2023-10-11T09:52:23.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.735+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 18 ==== osd_op_reply(122 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.735+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.735+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:123 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3c00 con 0x561e8b013800 2023-10-11T09:52:23.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.735+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 53 ==== osd_op_reply(123 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.735+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.735+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.735+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:23.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.735+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:124 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2000 con 0x561e8b013800 2023-10-11T09:52:23.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.739+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 54 ==== osd_op_reply(124 10000000000.01000000 [omap-get-header out=274b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.739+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 12 2023-10-11T09:52:23.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.739+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:23.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.739+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:125 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2400 con 0x561e8b013800 2023-10-11T09:52:23.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.739+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 55 ==== osd_op_reply(125 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.739+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=63 frag=0*, r=0 2023-10-11T09:52:23.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.739+0000 7f239a687840 20 datascan.read_dentry: 63_head not found in result 2023-10-11T09:52:23.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.739+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000234 at 0x0x10000000000/63 with size=2 bytes 2023-10-11T09:52:23.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.739+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:126 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3000 con 0x561e8b013800 2023-10-11T09:52:23.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.743+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 56 ==== osd_op_reply(126 10000000000.01000000 [omap-set-vals] v129'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.743+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/63 pointing to 0x0x10000000234 2023-10-11T09:52:23.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.743+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:127 18.24 18:25b5fce8::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34000 con 0x561e8b182800 2023-10-11T09:52:23.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.743+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.0 v2:172.21.15.78:6800/35735 17 ==== osd_op_reply(127 [pgnls-filter start_epoch 129 out=81b] v129'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b182800 2023-10-11T09:52:23.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.743+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000231.00000000 2023-10-11T09:52:23.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.743+0000 7f239a687840 10 datascan.operator(): handling object 10000000231.0 2023-10-11T09:52:23.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.743+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:128 18.24 18:25b5fce8:::10000000231.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35400 con 0x561e8b182800 2023-10-11T09:52:23.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.743+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.0 v2:172.21.15.78:6800/35735 18 ==== osd_op_reply(128 10000000231.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv10 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b182800 2023-10-11T09:52:23.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.743+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000231 2023-10-11T09:52:23.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.743+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/60 2023-10-11T09:52:23.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.743+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=60 2023-10-11T09:52:23.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.743+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:129 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34800 con 0x561e8b013000 2023-10-11T09:52:23.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.743+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 19 ==== osd_op_reply(129 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.743+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.743+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:130 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35c00 con 0x561e8b013800 2023-10-11T09:52:23.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.743+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 57 ==== osd_op_reply(130 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.747+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.747+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.747+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:23.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.747+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:131 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183800 con 0x561e8b182400 2023-10-11T09:52:23.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.747+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 26 ==== osd_op_reply(131 10000000000.01800000 [omap-get-header out=274b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.747+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 14 2023-10-11T09:52:23.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.747+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:23.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.747+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:132 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2c00 con 0x561e8b182400 2023-10-11T09:52:23.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.747+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 27 ==== osd_op_reply(132 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.747+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=60 frag=1*, r=0 2023-10-11T09:52:23.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.747+0000 7f239a687840 20 datascan.read_dentry: 60_head not found in result 2023-10-11T09:52:23.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.747+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000231 at 0x0x10000000000/60 with size=2 bytes 2023-10-11T09:52:23.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.747+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:133 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3c00 con 0x561e8b182400 2023-10-11T09:52:23.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.751+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 28 ==== osd_op_reply(133 10000000000.01800000 [omap-set-vals] v129'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.751+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/60 pointing to 0x0x10000000231 2023-10-11T09:52:23.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.751+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:134 18.14 18:28000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2000 con 0x561e8b013800 2023-10-11T09:52:23.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.751+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 58 ==== osd_op_reply(134 [pgnls-filter start_epoch 129 out=101b] v129'20 uv20 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.751+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 1000000020e.00000000 2023-10-11T09:52:23.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.751+0000 7f239a687840 10 datascan.operator(): handling object 1000000020e.0 2023-10-11T09:52:23.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.751+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:135 18.14 18:29971f97:::1000000020e.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2400 con 0x561e8b013800 2023-10-11T09:52:23.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.751+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 59 ==== osd_op_reply(135 1000000020e.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv18 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.751+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x1000000020e 2023-10-11T09:52:23.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.751+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/25 2023-10-11T09:52:23.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.751+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=25 2023-10-11T09:52:23.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.751+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:136 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183000 con 0x561e8b013000 2023-10-11T09:52:23.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.755+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 20 ==== osd_op_reply(136 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.755+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.755+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:137 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183400 con 0x561e8b013800 2023-10-11T09:52:23.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.755+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 60 ==== osd_op_reply(137 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.755+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.755+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.755+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:23.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.755+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:138 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b182c00 con 0x561e8b182400 2023-10-11T09:52:23.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.755+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 29 ==== osd_op_reply(138 10000000000.01800000 [omap-get-header out=274b] v0'0 uv15 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.755+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 15 2023-10-11T09:52:23.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.755+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:23.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.755+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:139 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b013400 con 0x561e8b182400 2023-10-11T09:52:23.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.755+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 30 ==== osd_op_reply(139 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv15 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.755+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=25 frag=1*, r=0 2023-10-11T09:52:23.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.755+0000 7f239a687840 20 datascan.read_dentry: 25_head not found in result 2023-10-11T09:52:23.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.755+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000020e at 0x0x10000000000/25 with size=2 bytes 2023-10-11T09:52:23.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.755+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:140 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34c00 con 0x561e8b182400 2023-10-11T09:52:23.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.759+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 31 ==== osd_op_reply(140 10000000000.01800000 [omap-set-vals] v129'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.759+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/25 pointing to 0x0x1000000020e 2023-10-11T09:52:23.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.759+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:141 18.14 18:2ab28be7::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34400 con 0x561e8b013800 2023-10-11T09:52:23.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.759+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 61 ==== osd_op_reply(141 [pgnls-filter start_epoch 129 out=101b] v129'20 uv20 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.759+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000211.00000000 2023-10-11T09:52:23.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.759+0000 7f239a687840 10 datascan.operator(): handling object 10000000211.0 2023-10-11T09:52:23.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.759+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:142 18.14 18:2ab28be7:::10000000211.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35800 con 0x561e8b013800 2023-10-11T09:52:23.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.759+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 62 ==== osd_op_reply(142 10000000211.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv19 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.759+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000211 2023-10-11T09:52:23.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.759+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/28 2023-10-11T09:52:23.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.759+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=28 2023-10-11T09:52:23.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.763+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:143 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ec000 con 0x561e8b013000 2023-10-11T09:52:23.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.763+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 21 ==== osd_op_reply(143 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.763+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.763+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:144 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ec400 con 0x561e8b013800 2023-10-11T09:52:23.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.763+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 63 ==== osd_op_reply(144 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.763+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.763+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.763+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:23.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.763+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:145 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ec800 con 0x561e8b013800 2023-10-11T09:52:23.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.763+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 64 ==== osd_op_reply(145 10000000000.01000000 [omap-get-header out=274b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.763+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 13 2023-10-11T09:52:23.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.763+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:23.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.763+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:146 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ecc00 con 0x561e8b013800 2023-10-11T09:52:23.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.763+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 65 ==== osd_op_reply(146 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.763+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=28 frag=0*, r=0 2023-10-11T09:52:23.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.763+0000 7f239a687840 20 datascan.read_dentry: 28_head not found in result 2023-10-11T09:52:23.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.763+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000211 at 0x0x10000000000/28 with size=2 bytes 2023-10-11T09:52:23.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.763+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:147 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ed000 con 0x561e8b013800 2023-10-11T09:52:23.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.767+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 66 ==== osd_op_reply(147 10000000000.01000000 [omap-set-vals] v129'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.767+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/28 pointing to 0x0x10000000211 2023-10-11T09:52:23.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.767+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:148 18.14 18:2b89f3ba::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ed400 con 0x561e8b013800 2023-10-11T09:52:23.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.767+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 67 ==== osd_op_reply(148 [pgnls-filter start_epoch 129 out=81b] v129'20 uv20 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.767+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000221.00000000 2023-10-11T09:52:23.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.767+0000 7f239a687840 10 datascan.operator(): handling object 10000000221.0 2023-10-11T09:52:23.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.767+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:149 18.14 18:2b89f3ba:::10000000221.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ed800 con 0x561e8b013800 2023-10-11T09:52:23.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.767+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 68 ==== osd_op_reply(149 10000000221.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv20 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.771+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000221 2023-10-11T09:52:23.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.771+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/44 2023-10-11T09:52:23.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.771+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=44 2023-10-11T09:52:23.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.771+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:150 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1edc00 con 0x561e8b013000 2023-10-11T09:52:23.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.771+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 22 ==== osd_op_reply(150 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.771+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.771+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:151 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ec400 con 0x561e8b013800 2023-10-11T09:52:23.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.771+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 69 ==== osd_op_reply(151 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.771+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.771+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.771+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:23.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.771+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:152 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ec800 con 0x561e8b182400 2023-10-11T09:52:23.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.771+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 32 ==== osd_op_reply(152 10000000000.01800000 [omap-get-header out=274b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.771+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 16 2023-10-11T09:52:23.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.771+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:23.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.771+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:153 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ecc00 con 0x561e8b182400 2023-10-11T09:52:23.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.771+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 33 ==== osd_op_reply(153 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.771+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=44 frag=1*, r=0 2023-10-11T09:52:23.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.771+0000 7f239a687840 20 datascan.read_dentry: 44_head not found in result 2023-10-11T09:52:23.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.771+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000221 at 0x0x10000000000/44 with size=2 bytes 2023-10-11T09:52:23.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.771+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:154 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ed000 con 0x561e8b182400 2023-10-11T09:52:23.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.775+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 34 ==== osd_op_reply(154 10000000000.01800000 [omap-set-vals] v129'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.775+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/44 pointing to 0x0x10000000221 2023-10-11T09:52:23.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.775+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:155 18.34 18:2c000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ed400 con 0x561e8b182800 2023-10-11T09:52:23.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.775+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.0 v2:172.21.15.78:6800/35735 19 ==== osd_op_reply(155 [pgnls-filter start_epoch 129 out=49b] v127'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b182800 2023-10-11T09:52:23.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.775+0000 7f2393e39700 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:156 18.c 18:30000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3400 con 0x561e8b183c00 2023-10-11T09:52:23.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.779+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 11 ==== osd_op_reply(156 [pgnls-filter start_epoch 129 out=81b] v129'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:23.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.779+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 100000001fc.00000000 2023-10-11T09:52:23.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.779+0000 7f239a687840 10 datascan.operator(): handling object 100000001fc.0 2023-10-11T09:52:23.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.779+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:157 18.c 18:30604137:::100000001fc.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35c00 con 0x561e8b183c00 2023-10-11T09:52:23.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.779+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 12 ==== osd_op_reply(157 100000001fc.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv9 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:23.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.779+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x100000001fc 2023-10-11T09:52:23.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.779+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/7 2023-10-11T09:52:23.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.779+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=7 2023-10-11T09:52:23.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.779+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:158 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34400 con 0x561e8b013000 2023-10-11T09:52:23.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.779+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 23 ==== osd_op_reply(158 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.779+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.779+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:159 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35800 con 0x561e8b013800 2023-10-11T09:52:23.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.779+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 70 ==== osd_op_reply(159 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.779+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.779+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.779+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:23.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.779+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:160 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183400 con 0x561e8b182400 2023-10-11T09:52:23.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.783+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 35 ==== osd_op_reply(160 10000000000.01800000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.783+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 17 2023-10-11T09:52:23.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.783+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:23.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.783+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:161 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3000 con 0x561e8b182400 2023-10-11T09:52:23.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.783+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 36 ==== osd_op_reply(161 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.783+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=7 frag=1*, r=0 2023-10-11T09:52:23.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.783+0000 7f239a687840 20 datascan.read_dentry: 7_head not found in result 2023-10-11T09:52:23.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.783+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fc at 0x0x10000000000/7 with size=1 bytes 2023-10-11T09:52:23.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.783+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:162 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2000 con 0x561e8b182400 2023-10-11T09:52:23.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.787+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 37 ==== osd_op_reply(162 10000000000.01800000 [omap-set-vals] v129'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.787+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/7 pointing to 0x0x100000001fc 2023-10-11T09:52:23.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.787+0000 7f239a687840 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x561e8b1e2400 0x561e8b19f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:23.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.787+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:163 18.2c 18:34000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ec000 con 0x561e8b1e2400 2023-10-11T09:52:23.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.787+0000 7f2398642700 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x561e8b1e2400 0x561e8b19f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:23.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.787+0000 7f2398642700 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x561e8b1e2400 0x561e8b19f180 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:23.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.787+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.2 v2:172.21.15.78:6808/35732 1 ==== osd_op_reply(163 [pgnls-filter start_epoch 129 out=101b] v129'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b1e2400 2023-10-11T09:52:23.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.787+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000223.00000000 2023-10-11T09:52:23.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.787+0000 7f239a687840 10 datascan.operator(): handling object 10000000223.0 2023-10-11T09:52:23.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.787+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:164 18.2c 18:35964134:::10000000223.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1edc00 con 0x561e8b1e2400 2023-10-11T09:52:23.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.787+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.2 v2:172.21.15.78:6808/35732 2 ==== osd_op_reply(164 10000000223.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b1e2400 2023-10-11T09:52:23.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.787+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000223 2023-10-11T09:52:23.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.787+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/46 2023-10-11T09:52:23.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.787+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=46 2023-10-11T09:52:23.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.787+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:165 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ed400 con 0x561e8b013000 2023-10-11T09:52:23.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.787+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 24 ==== osd_op_reply(165 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.787+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.791+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:166 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ec800 con 0x561e8b013800 2023-10-11T09:52:23.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.791+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 71 ==== osd_op_reply(166 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.791+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.791+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.791+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:23.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.791+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:167 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ecc00 con 0x561e8b013800 2023-10-11T09:52:23.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.791+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 72 ==== osd_op_reply(167 10000000000.01000000 [omap-get-header out=274b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.791+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 14 2023-10-11T09:52:23.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.791+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:23.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.791+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:168 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34000 con 0x561e8b013800 2023-10-11T09:52:23.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.791+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 73 ==== osd_op_reply(168 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.791+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=46 frag=0*, r=0 2023-10-11T09:52:23.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.791+0000 7f239a687840 20 datascan.read_dentry: 46_head not found in result 2023-10-11T09:52:23.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.791+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000223 at 0x0x10000000000/46 with size=2 bytes 2023-10-11T09:52:23.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.791+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:169 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35400 con 0x561e8b013800 2023-10-11T09:52:23.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.795+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 74 ==== osd_op_reply(169 10000000000.01000000 [omap-set-vals] v129'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.795+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/46 pointing to 0x0x10000000223 2023-10-11T09:52:23.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.795+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:170 18.2c 18:37389ade::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34800 con 0x561e8b1e2400 2023-10-11T09:52:23.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.795+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.2 v2:172.21.15.78:6808/35732 3 ==== osd_op_reply(170 [pgnls-filter start_epoch 129 out=81b] v129'14 uv14 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b1e2400 2023-10-11T09:52:23.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.795+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000209.00000000 2023-10-11T09:52:23.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.795+0000 7f239a687840 10 datascan.operator(): handling object 10000000209.0 2023-10-11T09:52:23.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.795+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:171 18.2c 18:37389ade:::10000000209.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34c00 con 0x561e8b1e2400 2023-10-11T09:52:23.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.795+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.2 v2:172.21.15.78:6808/35732 4 ==== osd_op_reply(171 10000000209.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv14 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b1e2400 2023-10-11T09:52:23.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.795+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000209 2023-10-11T09:52:23.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.795+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/20 2023-10-11T09:52:23.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.795+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=20 2023-10-11T09:52:23.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.795+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:172 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b013400 con 0x561e8b013000 2023-10-11T09:52:23.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.795+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 25 ==== osd_op_reply(172 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.795+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.795+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:173 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183000 con 0x561e8b013800 2023-10-11T09:52:23.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.799+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 75 ==== osd_op_reply(173 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.799+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.799+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.799+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:23.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.799+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:174 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183800 con 0x561e8b013800 2023-10-11T09:52:23.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.799+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 76 ==== osd_op_reply(174 10000000000.01000000 [omap-get-header out=274b] v0'0 uv15 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.799+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 15 2023-10-11T09:52:23.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.799+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:23.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.799+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:175 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b182c00 con 0x561e8b013800 2023-10-11T09:52:23.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.799+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 77 ==== osd_op_reply(175 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv15 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.799+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=20 frag=0*, r=0 2023-10-11T09:52:23.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.799+0000 7f239a687840 20 datascan.read_dentry: 20_head not found in result 2023-10-11T09:52:23.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.799+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000209 at 0x0x10000000000/20 with size=2 bytes 2023-10-11T09:52:23.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.799+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:176 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2c00 con 0x561e8b013800 2023-10-11T09:52:23.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.803+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 78 ==== osd_op_reply(176 10000000000.01000000 [omap-set-vals] v129'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.803+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/20 pointing to 0x0x10000000209 2023-10-11T09:52:23.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.803+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:177 18.1c 18:38000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3c00 con 0x561e8ae35000 2023-10-11T09:52:23.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.803+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.1 v2:172.21.15.78:6809/35734 2 ==== osd_op_reply(177 [pgnls-filter start_epoch 129 out=49b] v127'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8ae35000 2023-10-11T09:52:23.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.803+0000 7f2393e39700 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x561e8b1e3800 0x561e8b1f2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:23.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.803+0000 7f2393e39700 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:178 18.3c 18:3c000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ed800 con 0x561e8b1e3800 2023-10-11T09:52:23.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.803+0000 7f2399644700 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x561e8b1e3800 0x561e8b1f2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:23.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.803+0000 7f2399644700 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x561e8b1e3800 0x561e8b1f2000 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:23.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.803+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.7 v2:172.21.15.133:6824/35901 1 ==== osd_op_reply(178 [pgnls-filter start_epoch 129 out=49b] v127'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b1e3800 2023-10-11T09:52:23.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.807+0000 7f239463a700 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:179 18.2 18:40000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2800 con 0x561e8b1e3800 2023-10-11T09:52:23.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.807+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.7 v2:172.21.15.133:6824/35901 2 ==== osd_op_reply(179 [pgnls-filter start_epoch 129 out=49b] v127'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b1e3800 2023-10-11T09:52:23.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.807+0000 7f2393e39700 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:180 18.22 18:44000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ec400 con 0x561e8b013000 2023-10-11T09:52:23.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.807+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 26 ==== osd_op_reply(180 [pgnls-filter start_epoch 129 out=101b] v129'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.807+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 1000000024c.00000000 2023-10-11T09:52:23.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.807+0000 7f239a687840 10 datascan.operator(): handling object 1000000024c.0 2023-10-11T09:52:23.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.807+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:181 18.22 18:450d4ca5:::1000000024c.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3400 con 0x561e8b013000 2023-10-11T09:52:23.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.807+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 27 ==== osd_op_reply(181 1000000024c.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.807+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x1000000024c 2023-10-11T09:52:23.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.807+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/87 2023-10-11T09:52:23.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.807+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=87 2023-10-11T09:52:23.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.807+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:182 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2c00 con 0x561e8b013000 2023-10-11T09:52:23.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.807+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 28 ==== osd_op_reply(182 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.807+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.807+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:183 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183000 con 0x561e8b013800 2023-10-11T09:52:23.818 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.811+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 79 ==== osd_op_reply(183 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.819 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.811+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.819 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.811+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.819 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.811+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:23.819 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.811+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:184 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183800 con 0x561e8b013800 2023-10-11T09:52:23.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.811+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 80 ==== osd_op_reply(184 10000000000.01000000 [omap-get-header out=274b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.811+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 16 2023-10-11T09:52:23.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.811+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:23.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.811+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:185 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b182c00 con 0x561e8b013800 2023-10-11T09:52:23.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.811+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 81 ==== osd_op_reply(185 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.811+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=87 frag=0*, r=0 2023-10-11T09:52:23.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.811+0000 7f239a687840 20 datascan.read_dentry: 87_head not found in result 2023-10-11T09:52:23.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.811+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000024c at 0x0x10000000000/87 with size=2 bytes 2023-10-11T09:52:23.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.811+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:186 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35c00 con 0x561e8b013800 2023-10-11T09:52:23.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.815+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 82 ==== osd_op_reply(186 10000000000.01000000 [omap-set-vals] v129'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.815+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/87 pointing to 0x0x1000000024c 2023-10-11T09:52:23.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.815+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:187 18.22 18:45b8cac8::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35800 con 0x561e8b013000 2023-10-11T09:52:23.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.815+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 29 ==== osd_op_reply(187 [pgnls-filter start_epoch 129 out=101b] v129'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.815+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 1000000023d.00000000 2023-10-11T09:52:23.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.815+0000 7f239a687840 10 datascan.operator(): handling object 1000000023d.0 2023-10-11T09:52:23.824 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.815+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:188 18.22 18:45b8cac8:::1000000023d.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34000 con 0x561e8b013000 2023-10-11T09:52:23.824 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.815+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 30 ==== osd_op_reply(188 1000000023d.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.824 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.815+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x1000000023d 2023-10-11T09:52:23.824 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.815+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/72 2023-10-11T09:52:23.824 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.815+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=72 2023-10-11T09:52:23.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.815+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:189 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35400 con 0x561e8b013000 2023-10-11T09:52:23.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.815+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 31 ==== osd_op_reply(189 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.815+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.815+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:190 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ecc00 con 0x561e8b013800 2023-10-11T09:52:23.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.815+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 83 ==== osd_op_reply(190 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.815+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.815+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.815+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:23.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.815+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:191 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34400 con 0x561e8b182400 2023-10-11T09:52:23.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.819+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 38 ==== osd_op_reply(191 10000000000.01800000 [omap-get-header out=274b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.819+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 18 2023-10-11T09:52:23.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.819+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:23.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.819+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:192 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35800 con 0x561e8b182400 2023-10-11T09:52:23.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.819+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 39 ==== osd_op_reply(192 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.819+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=72 frag=1*, r=0 2023-10-11T09:52:23.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.819+0000 7f239a687840 20 datascan.read_dentry: 72_head not found in result 2023-10-11T09:52:23.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.819+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000023d at 0x0x10000000000/72 with size=2 bytes 2023-10-11T09:52:23.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.819+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:193 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34000 con 0x561e8b182400 2023-10-11T09:52:23.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.823+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 40 ==== osd_op_reply(193 10000000000.01800000 [omap-set-vals] v129'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.823+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/72 pointing to 0x0x1000000023d 2023-10-11T09:52:23.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.823+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:194 18.22 18:47efb22f::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35400 con 0x561e8b013000 2023-10-11T09:52:23.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.823+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 32 ==== osd_op_reply(194 [pgnls-filter start_epoch 129 out=81b] v129'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.823+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000235.00000000 2023-10-11T09:52:23.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.823+0000 7f239a687840 10 datascan.operator(): handling object 10000000235.0 2023-10-11T09:52:23.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.823+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:195 18.22 18:47efb22f:::10000000235.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3c00 con 0x561e8b013000 2023-10-11T09:52:23.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.823+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 33 ==== osd_op_reply(195 10000000235.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.823+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000235 2023-10-11T09:52:23.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.823+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/64 2023-10-11T09:52:23.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.823+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=64 2023-10-11T09:52:23.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.823+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:196 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3400 con 0x561e8b013000 2023-10-11T09:52:23.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.823+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 34 ==== osd_op_reply(196 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.823+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.823+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:197 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2c00 con 0x561e8b013800 2023-10-11T09:52:23.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.827+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 84 ==== osd_op_reply(197 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.827+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.827+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.827+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:23.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.827+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:198 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ed400 con 0x561e8b013800 2023-10-11T09:52:23.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.827+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 85 ==== osd_op_reply(198 10000000000.01000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.827+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 17 2023-10-11T09:52:23.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.827+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:23.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.827+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:199 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ec400 con 0x561e8b013800 2023-10-11T09:52:23.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.827+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 86 ==== osd_op_reply(199 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.827+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=64 frag=0*, r=0 2023-10-11T09:52:23.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.827+0000 7f239a687840 20 datascan.read_dentry: 64_head not found in result 2023-10-11T09:52:23.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.827+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000235 at 0x0x10000000000/64 with size=2 bytes 2023-10-11T09:52:23.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.827+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:200 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b013400 con 0x561e8b013800 2023-10-11T09:52:23.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.831+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 87 ==== osd_op_reply(200 10000000000.01000000 [omap-set-vals] v129'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.831+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/64 pointing to 0x0x10000000235 2023-10-11T09:52:23.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.831+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:201 18.12 18:48000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34800 con 0x561e8b1e2400 2023-10-11T09:52:23.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.831+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.2 v2:172.21.15.78:6808/35732 5 ==== osd_op_reply(201 [pgnls-filter start_epoch 129 out=49b] v127'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b1e2400 2023-10-11T09:52:23.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.831+0000 7f2393e39700 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:202 18.32 18:4c000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ec800 con 0x561e8b013000 2023-10-11T09:52:23.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.831+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 35 ==== osd_op_reply(202 [pgnls-filter start_epoch 129 out=81b] v129'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.831+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000252.00000000 2023-10-11T09:52:23.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.831+0000 7f239a687840 10 datascan.operator(): handling object 10000000252.0 2023-10-11T09:52:23.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.831+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:203 18.32 18:4e8c6251:::10000000252.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34c00 con 0x561e8b013000 2023-10-11T09:52:23.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.835+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 36 ==== osd_op_reply(203 10000000252.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.835+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000252 2023-10-11T09:52:23.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.835+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/93 2023-10-11T09:52:23.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.835+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=93 2023-10-11T09:52:23.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.835+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:204 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34400 con 0x561e8b013000 2023-10-11T09:52:23.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.835+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 37 ==== osd_op_reply(204 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.835+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.835+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:205 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ed000 con 0x561e8b013800 2023-10-11T09:52:23.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.835+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 88 ==== osd_op_reply(205 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.835+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.835+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.835+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:23.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.835+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:206 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ec000 con 0x561e8b182400 2023-10-11T09:52:23.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.835+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 41 ==== osd_op_reply(206 10000000000.01800000 [omap-get-header out=274b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.835+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 19 2023-10-11T09:52:23.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.835+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:23.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.835+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:207 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1edc00 con 0x561e8b182400 2023-10-11T09:52:23.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.839+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 42 ==== osd_op_reply(207 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.839+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=93 frag=1*, r=0 2023-10-11T09:52:23.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.839+0000 7f239a687840 20 datascan.read_dentry: 93_head not found in result 2023-10-11T09:52:23.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.839+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000252 at 0x0x10000000000/93 with size=2 bytes 2023-10-11T09:52:23.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.839+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:208 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3000 con 0x561e8b182400 2023-10-11T09:52:23.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.839+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 43 ==== osd_op_reply(208 10000000000.01800000 [omap-set-vals] v129'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.839+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/93 pointing to 0x0x10000000252 2023-10-11T09:52:23.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.839+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:209 18.a 18:50000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2000 con 0x561e8b013000 2023-10-11T09:52:23.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.843+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 38 ==== osd_op_reply(209 [pgnls-filter start_epoch 129 out=101b] v129'11 uv11 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.843+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 1000000023f.00000000 2023-10-11T09:52:23.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.843+0000 7f239a687840 10 datascan.operator(): handling object 1000000023f.0 2023-10-11T09:52:23.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.843+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:210 18.a 18:52dcfedc:::1000000023f.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183400 con 0x561e8b013000 2023-10-11T09:52:23.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.843+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 39 ==== osd_op_reply(210 1000000023f.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv10 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.843+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x1000000023f 2023-10-11T09:52:23.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.843+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/74 2023-10-11T09:52:23.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.843+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=74 2023-10-11T09:52:23.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.843+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:211 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b210000 con 0x561e8b013000 2023-10-11T09:52:23.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.843+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 40 ==== osd_op_reply(211 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.843+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.843+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:212 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b210400 con 0x561e8b013800 2023-10-11T09:52:23.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.843+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 89 ==== osd_op_reply(212 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.843+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.843+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.843+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:23.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.843+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:213 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b210800 con 0x561e8b013800 2023-10-11T09:52:23.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.843+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 90 ==== osd_op_reply(213 10000000000.01000000 [omap-get-header out=274b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.843+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 18 2023-10-11T09:52:23.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.843+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:23.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.843+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:214 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b210c00 con 0x561e8b013800 2023-10-11T09:52:23.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.847+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 91 ==== osd_op_reply(214 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.847+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=74 frag=0*, r=0 2023-10-11T09:52:23.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.847+0000 7f239a687840 20 datascan.read_dentry: 74_head not found in result 2023-10-11T09:52:23.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.847+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000023f at 0x0x10000000000/74 with size=2 bytes 2023-10-11T09:52:23.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.847+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:215 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b211000 con 0x561e8b013800 2023-10-11T09:52:23.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.847+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 92 ==== osd_op_reply(215 10000000000.01000000 [omap-set-vals] v129'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.847+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/74 pointing to 0x0x1000000023f 2023-10-11T09:52:23.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.847+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:216 18.a 18:53d16f76::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b211400 con 0x561e8b013000 2023-10-11T09:52:23.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.847+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 41 ==== osd_op_reply(216 [pgnls-filter start_epoch 129 out=81b] v129'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.847+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 1000000023b.00000000 2023-10-11T09:52:23.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.847+0000 7f239a687840 10 datascan.operator(): handling object 1000000023b.0 2023-10-11T09:52:23.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.847+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:217 18.a 18:53d16f76:::1000000023b.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b211800 con 0x561e8b013000 2023-10-11T09:52:23.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.851+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 42 ==== osd_op_reply(217 1000000023b.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.851+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x1000000023b 2023-10-11T09:52:23.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.851+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/70 2023-10-11T09:52:23.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.851+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=70 2023-10-11T09:52:23.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.851+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:218 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b211c00 con 0x561e8b013000 2023-10-11T09:52:23.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.851+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 43 ==== osd_op_reply(218 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.851+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.851+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:219 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b210000 con 0x561e8b013800 2023-10-11T09:52:23.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.851+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 93 ==== osd_op_reply(219 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.851+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.851+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.851+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:23.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.851+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:220 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b211400 con 0x561e8b013800 2023-10-11T09:52:23.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.851+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 94 ==== osd_op_reply(220 10000000000.01000000 [omap-get-header out=274b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.851+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 19 2023-10-11T09:52:23.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.851+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:23.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.851+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:221 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b211800 con 0x561e8b013800 2023-10-11T09:52:23.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.855+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 95 ==== osd_op_reply(221 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.855+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=70 frag=0*, r=0 2023-10-11T09:52:23.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.855+0000 7f239a687840 20 datascan.read_dentry: 70_head not found in result 2023-10-11T09:52:23.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.855+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000023b at 0x0x10000000000/70 with size=2 bytes 2023-10-11T09:52:23.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.855+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:222 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b210400 con 0x561e8b013800 2023-10-11T09:52:23.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.855+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 96 ==== osd_op_reply(222 10000000000.01000000 [omap-set-vals] v129'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.855+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/70 pointing to 0x0x1000000023b 2023-10-11T09:52:23.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.855+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:223 18.2a 18:54000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b210800 con 0x561e8b182400 2023-10-11T09:52:23.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.859+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 44 ==== osd_op_reply(223 [pgnls-filter start_epoch 129 out=101b] v129'21 uv21 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.859+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 1000000020a.00000000 2023-10-11T09:52:23.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.859+0000 7f239a687840 10 datascan.operator(): handling object 1000000020a.0 2023-10-11T09:52:23.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.859+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:224 18.2a 18:55a6cd83:::1000000020a.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ec800 con 0x561e8b182400 2023-10-11T09:52:23.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.859+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 45 ==== osd_op_reply(224 1000000020a.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv18 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.859+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x1000000020a 2023-10-11T09:52:23.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.859+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/21 2023-10-11T09:52:23.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.859+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=21 2023-10-11T09:52:23.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.859+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:225 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ed800 con 0x561e8b013000 2023-10-11T09:52:23.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.859+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 44 ==== osd_op_reply(225 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.859+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.859+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:226 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ecc00 con 0x561e8b013800 2023-10-11T09:52:23.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.859+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 97 ==== osd_op_reply(226 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.859+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.859+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.859+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:23.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.859+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:227 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ed400 con 0x561e8b013800 2023-10-11T09:52:23.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.859+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 98 ==== osd_op_reply(227 10000000000.01000000 [omap-get-header out=274b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.859+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 20 2023-10-11T09:52:23.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.859+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:23.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.859+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:228 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ec400 con 0x561e8b013800 2023-10-11T09:52:23.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.859+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 99 ==== osd_op_reply(228 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.859+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=21 frag=0*, r=0 2023-10-11T09:52:23.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.859+0000 7f239a687840 20 datascan.read_dentry: 21_head not found in result 2023-10-11T09:52:23.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.859+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000020a at 0x0x10000000000/21 with size=2 bytes 2023-10-11T09:52:23.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.863+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:229 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ed000 con 0x561e8b013800 2023-10-11T09:52:23.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.863+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 100 ==== osd_op_reply(229 10000000000.01000000 [omap-set-vals] v129'21 uv21 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.863+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/21 pointing to 0x0x1000000020a 2023-10-11T09:52:23.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.863+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:230 18.2a 18:56889a45::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b013400 con 0x561e8b182400 2023-10-11T09:52:23.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.863+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 46 ==== osd_op_reply(230 [pgnls-filter start_epoch 129 out=101b] v129'21 uv21 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.863+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000213.00000000 2023-10-11T09:52:23.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.863+0000 7f239a687840 10 datascan.operator(): handling object 10000000213.0 2023-10-11T09:52:23.873 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.863+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:231 18.2a 18:56889a45:::10000000213.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3c00 con 0x561e8b182400 2023-10-11T09:52:23.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.867+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 47 ==== osd_op_reply(231 10000000213.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv19 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.867+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000213 2023-10-11T09:52:23.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.867+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/30 2023-10-11T09:52:23.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.867+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=30 2023-10-11T09:52:23.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.867+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:232 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3400 con 0x561e8b013000 2023-10-11T09:52:23.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.867+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 45 ==== osd_op_reply(232 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.867+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.867+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:233 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2000 con 0x561e8b013800 2023-10-11T09:52:23.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.867+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 101 ==== osd_op_reply(233 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.867+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.867+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.867+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:23.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.867+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:234 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2800 con 0x561e8b182400 2023-10-11T09:52:23.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.867+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 48 ==== osd_op_reply(234 10000000000.01800000 [omap-get-header out=274b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.867+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 20 2023-10-11T09:52:23.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.867+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:23.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.867+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:235 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2c00 con 0x561e8b182400 2023-10-11T09:52:23.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.867+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 49 ==== osd_op_reply(235 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.867+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=30 frag=1*, r=0 2023-10-11T09:52:23.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.867+0000 7f239a687840 20 datascan.read_dentry: 30_head not found in result 2023-10-11T09:52:23.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.867+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000213 at 0x0x10000000000/30 with size=2 bytes 2023-10-11T09:52:23.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.867+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:236 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35400 con 0x561e8b182400 2023-10-11T09:52:23.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.871+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 50 ==== osd_op_reply(236 10000000000.01800000 [omap-set-vals] v129'21 uv21 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.871+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/30 pointing to 0x0x10000000213 2023-10-11T09:52:23.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.871+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:237 18.2a 18:576ad4f4::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34c00 con 0x561e8b182400 2023-10-11T09:52:23.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.871+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 51 ==== osd_op_reply(237 [pgnls-filter start_epoch 129 out=101b] v129'21 uv21 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.871+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000245.00000000 2023-10-11T09:52:23.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.871+0000 7f239a687840 10 datascan.operator(): handling object 10000000245.0 2023-10-11T09:52:23.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.871+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:238 18.2a 18:576ad4f4:::10000000245.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34400 con 0x561e8b182400 2023-10-11T09:52:23.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.875+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 52 ==== osd_op_reply(238 10000000245.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv20 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.875+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000245 2023-10-11T09:52:23.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.875+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/80 2023-10-11T09:52:23.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.875+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=80 2023-10-11T09:52:23.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.875+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:239 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35c00 con 0x561e8b013000 2023-10-11T09:52:23.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.875+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 46 ==== osd_op_reply(239 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.875+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.875+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:240 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183400 con 0x561e8b013800 2023-10-11T09:52:23.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.875+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 102 ==== osd_op_reply(240 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.875+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.875+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.875+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:23.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.875+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:241 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183000 con 0x561e8b013800 2023-10-11T09:52:23.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.875+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 103 ==== osd_op_reply(241 10000000000.01000000 [omap-get-header out=274b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.875+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 21 2023-10-11T09:52:23.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.875+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:23.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.875+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:242 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183800 con 0x561e8b013800 2023-10-11T09:52:23.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.875+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 104 ==== osd_op_reply(242 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.875+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=80 frag=0*, r=0 2023-10-11T09:52:23.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.875+0000 7f239a687840 20 datascan.read_dentry: 80_head not found in result 2023-10-11T09:52:23.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.875+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000245 at 0x0x10000000000/80 with size=2 bytes 2023-10-11T09:52:23.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.875+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:243 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b182c00 con 0x561e8b013800 2023-10-11T09:52:23.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.879+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 105 ==== osd_op_reply(243 10000000000.01000000 [omap-set-vals] v129'22 uv22 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.879+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/80 pointing to 0x0x10000000245 2023-10-11T09:52:23.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.879+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:244 18.2a 18:57fe7535::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2000 con 0x561e8b182400 2023-10-11T09:52:23.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.879+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 53 ==== osd_op_reply(244 [pgnls-filter start_epoch 129 out=81b] v129'21 uv21 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.879+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 100000001f6.00000000 2023-10-11T09:52:23.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.879+0000 7f239a687840 10 datascan.operator(): handling object 100000001f6.0 2023-10-11T09:52:23.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.879+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:245 18.2a 18:57fe7535:::100000001f6.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3000 con 0x561e8b182400 2023-10-11T09:52:23.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.883+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 54 ==== osd_op_reply(245 100000001f6.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv21 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.883+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x100000001f6 2023-10-11T09:52:23.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.883+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/1 2023-10-11T09:52:23.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.883+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=1 2023-10-11T09:52:23.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.883+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:246 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3c00 con 0x561e8b013000 2023-10-11T09:52:23.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.883+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 47 ==== osd_op_reply(246 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.883+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.883+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:247 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b013400 con 0x561e8b013800 2023-10-11T09:52:23.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.883+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 106 ==== osd_op_reply(247 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.883+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.883+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.883+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:23.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.883+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:248 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ecc00 con 0x561e8b013800 2023-10-11T09:52:23.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.883+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 107 ==== osd_op_reply(248 10000000000.01000000 [omap-get-header out=274b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.883+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 22 2023-10-11T09:52:23.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.883+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:23.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.883+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:249 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ed400 con 0x561e8b013800 2023-10-11T09:52:23.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.883+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 108 ==== osd_op_reply(249 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.883+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=1 frag=0*, r=0 2023-10-11T09:52:23.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.883+0000 7f239a687840 20 datascan.read_dentry: 1_head not found in result 2023-10-11T09:52:23.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.883+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f6 at 0x0x10000000000/1 with size=1 bytes 2023-10-11T09:52:23.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.883+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:250 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ec400 con 0x561e8b013800 2023-10-11T09:52:23.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.887+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 109 ==== osd_op_reply(250 10000000000.01000000 [omap-set-vals] v129'23 uv23 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.887+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/1 pointing to 0x0x100000001f6 2023-10-11T09:52:23.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.887+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:251 18.1a 18:58000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ed000 con 0x561e8b013000 2023-10-11T09:52:23.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.887+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 48 ==== osd_op_reply(251 [pgnls-filter start_epoch 129 out=101b] v129'13 uv13 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.891+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000226.00000000 2023-10-11T09:52:23.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.891+0000 7f239a687840 10 datascan.operator(): handling object 10000000226.0 2023-10-11T09:52:23.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.891+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:252 18.1a 18:582f7b2d:::10000000226.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ec000 con 0x561e8b013000 2023-10-11T09:52:23.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.891+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 49 ==== osd_op_reply(252 10000000226.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.891+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000226 2023-10-11T09:52:23.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.891+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/49 2023-10-11T09:52:23.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.891+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=49 2023-10-11T09:52:23.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.891+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:253 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1edc00 con 0x561e8b013000 2023-10-11T09:52:23.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.891+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 50 ==== osd_op_reply(253 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.891+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.891+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:254 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ec800 con 0x561e8b013800 2023-10-11T09:52:23.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.891+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 110 ==== osd_op_reply(254 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.891+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.891+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.891+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:23.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.891+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:255 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b210c00 con 0x561e8b182400 2023-10-11T09:52:23.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.891+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 55 ==== osd_op_reply(255 10000000000.01800000 [omap-get-header out=274b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.891+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 21 2023-10-11T09:52:23.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.891+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:23.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.891+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:256 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b211000 con 0x561e8b182400 2023-10-11T09:52:23.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.891+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 56 ==== osd_op_reply(256 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.891+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=49 frag=1*, r=0 2023-10-11T09:52:23.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.891+0000 7f239a687840 20 datascan.read_dentry: 49_head not found in result 2023-10-11T09:52:23.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.891+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000226 at 0x0x10000000000/49 with size=2 bytes 2023-10-11T09:52:23.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.891+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:257 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b210000 con 0x561e8b182400 2023-10-11T09:52:23.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.895+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 57 ==== osd_op_reply(257 10000000000.01800000 [omap-set-vals] v129'22 uv22 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.895+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/49 pointing to 0x0x10000000226 2023-10-11T09:52:23.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.895+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:258 18.1a 18:5a2a1294::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b211400 con 0x561e8b013000 2023-10-11T09:52:23.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.895+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 51 ==== osd_op_reply(258 [pgnls-filter start_epoch 129 out=81b] v129'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.895+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 1000000023a.00000000 2023-10-11T09:52:23.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.895+0000 7f239a687840 10 datascan.operator(): handling object 1000000023a.0 2023-10-11T09:52:23.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.895+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:259 18.1a 18:5a2a1294:::1000000023a.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b211800 con 0x561e8b013000 2023-10-11T09:52:23.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.899+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 52 ==== osd_op_reply(259 1000000023a.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.899+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x1000000023a 2023-10-11T09:52:23.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.899+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/69 2023-10-11T09:52:23.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.899+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=69 2023-10-11T09:52:23.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.899+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:260 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b210400 con 0x561e8b013000 2023-10-11T09:52:23.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.899+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 53 ==== osd_op_reply(260 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.899+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.899+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:261 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b210800 con 0x561e8b013800 2023-10-11T09:52:23.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.899+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 111 ==== osd_op_reply(261 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.899+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.899+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.899+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:23.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.899+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:262 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35800 con 0x561e8b013800 2023-10-11T09:52:23.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.899+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 112 ==== osd_op_reply(262 10000000000.01000000 [omap-get-header out=274b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.899+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 23 2023-10-11T09:52:23.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.899+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:23.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.899+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:263 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34000 con 0x561e8b013800 2023-10-11T09:52:23.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.899+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 113 ==== osd_op_reply(263 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.899+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=69 frag=0*, r=0 2023-10-11T09:52:23.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.899+0000 7f239a687840 20 datascan.read_dentry: 69_head not found in result 2023-10-11T09:52:23.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.899+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000023a at 0x0x10000000000/69 with size=2 bytes 2023-10-11T09:52:23.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.899+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:264 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34800 con 0x561e8b013800 2023-10-11T09:52:23.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.903+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 114 ==== osd_op_reply(264 10000000000.01000000 [omap-set-vals] v129'24 uv24 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.903+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/69 pointing to 0x0x1000000023a 2023-10-11T09:52:23.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.903+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:265 18.3a 18:5c000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183400 con 0x561e8b182800 2023-10-11T09:52:23.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.903+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.0 v2:172.21.15.78:6800/35735 20 ==== osd_op_reply(265 [pgnls-filter start_epoch 129 out=49b] v127'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b182800 2023-10-11T09:52:23.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.903+0000 7f2393e39700 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:266 18.6 18:60000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ed800 con 0x561e8b1e3800 2023-10-11T09:52:23.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.903+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.7 v2:172.21.15.133:6824/35901 3 ==== osd_op_reply(266 [pgnls-filter start_epoch 129 out=81b] v129'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b1e3800 2023-10-11T09:52:23.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.903+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000228.00000000 2023-10-11T09:52:23.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.903+0000 7f239a687840 10 datascan.operator(): handling object 10000000228.0 2023-10-11T09:52:23.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.903+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:267 18.6 18:614797eb:::10000000228.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183000 con 0x561e8b1e3800 2023-10-11T09:52:23.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.907+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.7 v2:172.21.15.133:6824/35901 4 ==== osd_op_reply(267 10000000228.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv10 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b1e3800 2023-10-11T09:52:23.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.907+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000228 2023-10-11T09:52:23.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.907+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/51 2023-10-11T09:52:23.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.907+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=51 2023-10-11T09:52:23.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.907+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:268 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183800 con 0x561e8b013000 2023-10-11T09:52:23.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.907+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 54 ==== osd_op_reply(268 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.907+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.907+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:269 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b182c00 con 0x561e8b013800 2023-10-11T09:52:23.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.907+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 115 ==== osd_op_reply(269 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.907+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.907+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.907+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:23.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.907+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:270 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183400 con 0x561e8b182400 2023-10-11T09:52:23.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.907+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 58 ==== osd_op_reply(270 10000000000.01800000 [omap-get-header out=274b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.907+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 22 2023-10-11T09:52:23.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.907+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:23.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.907+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:271 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b211c00 con 0x561e8b182400 2023-10-11T09:52:23.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.911+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 59 ==== osd_op_reply(271 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.911+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=51 frag=1*, r=0 2023-10-11T09:52:23.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.911+0000 7f239a687840 20 datascan.read_dentry: 51_head not found in result 2023-10-11T09:52:23.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.911+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000228 at 0x0x10000000000/51 with size=2 bytes 2023-10-11T09:52:23.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.911+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:272 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b211400 con 0x561e8b182400 2023-10-11T09:52:23.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.911+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 60 ==== osd_op_reply(272 10000000000.01800000 [omap-set-vals] v129'23 uv23 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.911+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/51 pointing to 0x0x10000000228 2023-10-11T09:52:23.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.911+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:273 18.26 18:64000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b211800 con 0x561e8b1e2400 2023-10-11T09:52:23.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.915+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.2 v2:172.21.15.78:6808/35732 6 ==== osd_op_reply(273 [pgnls-filter start_epoch 129 out=101b] v129'24 uv24 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b1e2400 2023-10-11T09:52:23.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.915+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 100000001ff.00000000 2023-10-11T09:52:23.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.915+0000 7f239a687840 10 datascan.operator(): handling object 100000001ff.0 2023-10-11T09:52:23.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.915+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:274 18.26 18:658f2455:::100000001ff.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b210400 con 0x561e8b1e2400 2023-10-11T09:52:23.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.915+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.2 v2:172.21.15.78:6808/35732 7 ==== osd_op_reply(274 100000001ff.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv22 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b1e2400 2023-10-11T09:52:23.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.915+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x100000001ff 2023-10-11T09:52:23.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.915+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/10 2023-10-11T09:52:23.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.915+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=10 2023-10-11T09:52:23.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.915+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:275 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ed000 con 0x561e8b013000 2023-10-11T09:52:23.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.915+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 55 ==== osd_op_reply(275 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.915+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.915+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:276 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ec000 con 0x561e8b013800 2023-10-11T09:52:23.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.915+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 116 ==== osd_op_reply(276 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.915+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.915+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.915+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:23.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.915+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:277 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1edc00 con 0x561e8b013800 2023-10-11T09:52:23.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.919+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 117 ==== osd_op_reply(277 10000000000.01000000 [omap-get-header out=274b] v0'0 uv24 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.919+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 24 2023-10-11T09:52:23.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.919+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:23.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.919+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:278 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183400 con 0x561e8b013800 2023-10-11T09:52:23.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.919+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 118 ==== osd_op_reply(278 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv24 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.919+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=10 frag=0*, r=0 2023-10-11T09:52:23.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.919+0000 7f239a687840 20 datascan.read_dentry: 10_head not found in result 2023-10-11T09:52:23.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.919+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001ff at 0x0x10000000000/10 with size=2 bytes 2023-10-11T09:52:23.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.919+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:279 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b210c00 con 0x561e8b013800 2023-10-11T09:52:23.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.919+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 119 ==== osd_op_reply(279 10000000000.01000000 [omap-set-vals] v129'25 uv25 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.919+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/10 pointing to 0x0x100000001ff 2023-10-11T09:52:23.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.919+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:280 18.26 18:65d05a5c::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b211000 con 0x561e8b1e2400 2023-10-11T09:52:23.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.923+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.2 v2:172.21.15.78:6808/35732 8 ==== osd_op_reply(280 [pgnls-filter start_epoch 129 out=101b] v129'24 uv24 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b1e2400 2023-10-11T09:52:23.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.923+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000258.00000000 2023-10-11T09:52:23.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.923+0000 7f239a687840 10 datascan.operator(): handling object 10000000258.0 2023-10-11T09:52:23.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.923+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:281 18.26 18:65d05a5c:::10000000258.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b210000 con 0x561e8b1e2400 2023-10-11T09:52:23.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.923+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.2 v2:172.21.15.78:6808/35732 9 ==== osd_op_reply(281 10000000258.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv23 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b1e2400 2023-10-11T09:52:23.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.923+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000258 2023-10-11T09:52:23.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.923+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/99 2023-10-11T09:52:23.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.923+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=99 2023-10-11T09:52:23.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.923+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:282 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2800 con 0x561e8b013000 2023-10-11T09:52:23.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.923+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 56 ==== osd_op_reply(282 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.923+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.923+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:283 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2c00 con 0x561e8b013800 2023-10-11T09:52:23.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.923+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 120 ==== osd_op_reply(283 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.923+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.923+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.923+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:23.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.923+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:284 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2000 con 0x561e8b182400 2023-10-11T09:52:23.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.927+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 61 ==== osd_op_reply(284 10000000000.01800000 [omap-get-header out=274b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.927+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 23 2023-10-11T09:52:23.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.927+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:23.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.927+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:285 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3000 con 0x561e8b182400 2023-10-11T09:52:23.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.927+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 62 ==== osd_op_reply(285 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.927+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=99 frag=1*, r=0 2023-10-11T09:52:23.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.927+0000 7f239a687840 20 datascan.read_dentry: 99_head not found in result 2023-10-11T09:52:23.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.927+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000258 at 0x0x10000000000/99 with size=2 bytes 2023-10-11T09:52:23.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.927+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:286 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35400 con 0x561e8b182400 2023-10-11T09:52:23.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.927+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 63 ==== osd_op_reply(286 10000000000.01800000 [omap-set-vals] v129'24 uv24 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.927+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/99 pointing to 0x0x10000000258 2023-10-11T09:52:23.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.927+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:287 18.26 18:65e03fee::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34c00 con 0x561e8b1e2400 2023-10-11T09:52:23.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.931+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.2 v2:172.21.15.78:6808/35732 10 ==== osd_op_reply(287 [pgnls-filter start_epoch 129 out=81b] v129'24 uv24 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b1e2400 2023-10-11T09:52:23.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.931+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000240.00000000 2023-10-11T09:52:23.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.931+0000 7f239a687840 10 datascan.operator(): handling object 10000000240.0 2023-10-11T09:52:23.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.931+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:288 18.26 18:65e03fee:::10000000240.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34400 con 0x561e8b1e2400 2023-10-11T09:52:23.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.931+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.2 v2:172.21.15.78:6808/35732 11 ==== osd_op_reply(288 10000000240.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv24 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b1e2400 2023-10-11T09:52:23.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.931+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000240 2023-10-11T09:52:23.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.931+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/75 2023-10-11T09:52:23.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.931+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=75 2023-10-11T09:52:23.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.931+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:289 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b210800 con 0x561e8b013000 2023-10-11T09:52:23.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.931+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 57 ==== osd_op_reply(289 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.931+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.931+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:290 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b210c00 con 0x561e8b013800 2023-10-11T09:52:23.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.931+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 121 ==== osd_op_reply(290 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.931+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.931+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.931+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:23.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.931+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:291 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183000 con 0x561e8b013800 2023-10-11T09:52:23.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.931+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 122 ==== osd_op_reply(291 10000000000.01000000 [omap-get-header out=274b] v0'0 uv25 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.931+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 25 2023-10-11T09:52:23.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.931+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:23.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.931+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:292 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b182c00 con 0x561e8b013800 2023-10-11T09:52:23.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.935+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 123 ==== osd_op_reply(292 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv25 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.935+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=75 frag=0*, r=0 2023-10-11T09:52:23.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.935+0000 7f239a687840 20 datascan.read_dentry: 75_head not found in result 2023-10-11T09:52:23.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.935+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000240 at 0x0x10000000000/75 with size=2 bytes 2023-10-11T09:52:23.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.935+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:293 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183400 con 0x561e8b013800 2023-10-11T09:52:23.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.935+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 124 ==== osd_op_reply(293 10000000000.01000000 [omap-set-vals] v129'26 uv26 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.935+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/75 pointing to 0x0x10000000240 2023-10-11T09:52:23.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.935+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:294 18.16 18:68000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ecc00 con 0x561e8b013000 2023-10-11T09:52:23.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.935+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 58 ==== osd_op_reply(294 [pgnls-filter start_epoch 129 out=49b] v127'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.939+0000 7f239463a700 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:295 18.36 18:6c000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b220000 con 0x561e8ae35000 2023-10-11T09:52:23.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.939+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.1 v2:172.21.15.78:6809/35734 3 ==== osd_op_reply(295 [pgnls-filter start_epoch 129 out=101b] v129'27 uv27 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8ae35000 2023-10-11T09:52:23.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.939+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000255.00000000 2023-10-11T09:52:23.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.939+0000 7f239a687840 10 datascan.operator(): handling object 10000000255.0 2023-10-11T09:52:23.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.939+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:296 18.36 18:6cdfec98:::10000000255.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ed400 con 0x561e8ae35000 2023-10-11T09:52:23.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.939+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.1 v2:172.21.15.78:6809/35734 4 ==== osd_op_reply(296 10000000255.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv24 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8ae35000 2023-10-11T09:52:23.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.939+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000255 2023-10-11T09:52:23.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.939+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/96 2023-10-11T09:52:23.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.939+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=96 2023-10-11T09:52:23.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.939+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:297 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ec400 con 0x561e8b013000 2023-10-11T09:52:23.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.939+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 59 ==== osd_op_reply(297 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.939+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.939+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:298 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ec800 con 0x561e8b013800 2023-10-11T09:52:23.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.939+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 125 ==== osd_op_reply(298 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.939+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.939+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.939+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:23.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.939+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:299 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ed800 con 0x561e8b013800 2023-10-11T09:52:23.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.943+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 126 ==== osd_op_reply(299 10000000000.01000000 [omap-get-header out=274b] v0'0 uv26 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.943+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 26 2023-10-11T09:52:23.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.943+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:23.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.943+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:300 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ec000 con 0x561e8b013800 2023-10-11T09:52:23.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.943+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 127 ==== osd_op_reply(300 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv26 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.943+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=96 frag=0*, r=0 2023-10-11T09:52:23.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.943+0000 7f239a687840 20 datascan.read_dentry: 96_head not found in result 2023-10-11T09:52:23.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.943+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000255 at 0x0x10000000000/96 with size=2 bytes 2023-10-11T09:52:23.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.943+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:301 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1edc00 con 0x561e8b013800 2023-10-11T09:52:23.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.947+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 128 ==== osd_op_reply(301 10000000000.01000000 [omap-set-vals] v129'27 uv27 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.947+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/96 pointing to 0x0x10000000255 2023-10-11T09:52:23.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.947+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:302 18.36 18:6ee5e2c4::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35800 con 0x561e8ae35000 2023-10-11T09:52:23.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.947+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.1 v2:172.21.15.78:6809/35734 5 ==== osd_op_reply(302 [pgnls-filter start_epoch 129 out=101b] v129'27 uv27 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8ae35000 2023-10-11T09:52:23.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.947+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 100000001f9.00000000 2023-10-11T09:52:23.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.947+0000 7f239a687840 10 datascan.operator(): handling object 100000001f9.0 2023-10-11T09:52:23.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.947+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:303 18.36 18:6ee5e2c4:::100000001f9.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34000 con 0x561e8ae35000 2023-10-11T09:52:23.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.947+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.1 v2:172.21.15.78:6809/35734 6 ==== osd_op_reply(303 100000001f9.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv25 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8ae35000 2023-10-11T09:52:23.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.947+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x100000001f9 2023-10-11T09:52:23.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.947+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/4 2023-10-11T09:52:23.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.947+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=4 2023-10-11T09:52:23.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.947+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:304 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34800 con 0x561e8b013000 2023-10-11T09:52:23.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.947+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 60 ==== osd_op_reply(304 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.947+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.947+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:305 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b013400 con 0x561e8b013800 2023-10-11T09:52:23.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.947+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 129 ==== osd_op_reply(305 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.947+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.947+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.947+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:23.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.947+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:306 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b220c00 con 0x561e8b013800 2023-10-11T09:52:23.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.951+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 130 ==== osd_op_reply(306 10000000000.01000000 [omap-get-header out=274b] v0'0 uv27 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.951+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 27 2023-10-11T09:52:23.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.951+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:23.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.951+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:307 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221000 con 0x561e8b013800 2023-10-11T09:52:23.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.951+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 131 ==== osd_op_reply(307 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv27 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.951+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=4 frag=0*, r=0 2023-10-11T09:52:23.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.951+0000 7f239a687840 20 datascan.read_dentry: 4_head not found in result 2023-10-11T09:52:23.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.951+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f9 at 0x0x10000000000/4 with size=1 bytes 2023-10-11T09:52:23.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.951+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:308 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221400 con 0x561e8b013800 2023-10-11T09:52:23.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.955+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 132 ==== osd_op_reply(308 10000000000.01000000 [omap-set-vals] v129'28 uv28 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.955+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/4 pointing to 0x0x100000001f9 2023-10-11T09:52:23.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.955+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:309 18.36 18:6f746908::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221800 con 0x561e8ae35000 2023-10-11T09:52:23.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.955+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.1 v2:172.21.15.78:6809/35734 7 ==== osd_op_reply(309 [pgnls-filter start_epoch 129 out=101b] v129'27 uv27 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8ae35000 2023-10-11T09:52:23.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.955+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 1000000022c.00000000 2023-10-11T09:52:23.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.955+0000 7f239a687840 10 datascan.operator(): handling object 1000000022c.0 2023-10-11T09:52:23.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.955+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:310 18.36 18:6f746908:::1000000022c.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221c00 con 0x561e8ae35000 2023-10-11T09:52:23.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.955+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.1 v2:172.21.15.78:6809/35734 8 ==== osd_op_reply(310 1000000022c.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv26 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8ae35000 2023-10-11T09:52:23.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.955+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x1000000022c 2023-10-11T09:52:23.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.955+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/55 2023-10-11T09:52:23.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.955+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=55 2023-10-11T09:52:23.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.955+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:311 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b220000 con 0x561e8b013000 2023-10-11T09:52:23.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.955+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 61 ==== osd_op_reply(311 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.955+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.955+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:312 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221800 con 0x561e8b013800 2023-10-11T09:52:23.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.955+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 133 ==== osd_op_reply(312 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.955+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.955+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.955+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:23.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.955+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:313 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221c00 con 0x561e8b013800 2023-10-11T09:52:23.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.959+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 134 ==== osd_op_reply(313 10000000000.01000000 [omap-get-header out=274b] v0'0 uv28 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.959+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 28 2023-10-11T09:52:23.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.959+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:23.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.959+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:314 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35800 con 0x561e8b013800 2023-10-11T09:52:23.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.959+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 135 ==== osd_op_reply(314 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv28 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.959+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=55 frag=0*, r=0 2023-10-11T09:52:23.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.959+0000 7f239a687840 20 datascan.read_dentry: 55_head not found in result 2023-10-11T09:52:23.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.959+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000022c at 0x0x10000000000/55 with size=2 bytes 2023-10-11T09:52:23.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.959+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:315 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34000 con 0x561e8b013800 2023-10-11T09:52:23.970 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.959+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 136 ==== osd_op_reply(315 10000000000.01000000 [omap-set-vals] v129'29 uv29 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.959+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/55 pointing to 0x0x1000000022c 2023-10-11T09:52:23.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.963+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:316 18.36 18:6fe38dc2::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34800 con 0x561e8ae35000 2023-10-11T09:52:23.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.963+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.1 v2:172.21.15.78:6809/35734 9 ==== osd_op_reply(316 [pgnls-filter start_epoch 129 out=81b] v129'27 uv27 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8ae35000 2023-10-11T09:52:23.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.963+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2023-10-11T09:52:23.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.963+0000 7f239a687840 10 datascan.operator(): handling object 100000001f7.0 2023-10-11T09:52:23.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.963+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:317 18.36 18:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ed000 con 0x561e8ae35000 2023-10-11T09:52:23.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.963+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.1 v2:172.21.15.78:6809/35734 10 ==== osd_op_reply(317 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv27 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8ae35000 2023-10-11T09:52:23.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.963+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2023-10-11T09:52:23.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.963+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/2 2023-10-11T09:52:23.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.963+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=2 2023-10-11T09:52:23.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.963+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:318 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ecc00 con 0x561e8b013000 2023-10-11T09:52:23.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.963+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 62 ==== osd_op_reply(318 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.963+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.963+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:319 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ed400 con 0x561e8b013800 2023-10-11T09:52:23.974 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.963+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 137 ==== osd_op_reply(319 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.963+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.963+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.963+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:23.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.963+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:320 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ec400 con 0x561e8b013800 2023-10-11T09:52:23.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.967+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 138 ==== osd_op_reply(320 10000000000.01000000 [omap-get-header out=274b] v0'0 uv29 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.967+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 29 2023-10-11T09:52:23.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.967+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:23.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.967+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:321 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b210800 con 0x561e8b013800 2023-10-11T09:52:23.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.967+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 139 ==== osd_op_reply(321 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv29 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.967+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=2 frag=0*, r=0 2023-10-11T09:52:23.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.967+0000 7f239a687840 20 datascan.read_dentry: 2_head not found in result 2023-10-11T09:52:23.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.967+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000000/2 with size=1 bytes 2023-10-11T09:52:23.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.967+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:322 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2800 con 0x561e8b013800 2023-10-11T09:52:23.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.971+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 140 ==== osd_op_reply(322 10000000000.01000000 [omap-set-vals] v129'30 uv30 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.971+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/2 pointing to 0x0x100000001f7 2023-10-11T09:52:23.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.971+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:323 18.e 18:70000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183800 con 0x561e8b182800 2023-10-11T09:52:23.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.971+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.0 v2:172.21.15.78:6800/35735 21 ==== osd_op_reply(323 [pgnls-filter start_epoch 129 out=81b] v129'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b182800 2023-10-11T09:52:23.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.971+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 1000000022f.00000000 2023-10-11T09:52:23.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.971+0000 7f239a687840 10 datascan.operator(): handling object 1000000022f.0 2023-10-11T09:52:23.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.971+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:324 18.e 18:71c13145:::1000000022f.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35800 con 0x561e8b182800 2023-10-11T09:52:23.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.971+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.0 v2:172.21.15.78:6800/35735 22 ==== osd_op_reply(324 1000000022f.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv8 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b182800 2023-10-11T09:52:23.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.971+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x1000000022f 2023-10-11T09:52:23.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.971+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/58 2023-10-11T09:52:23.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.971+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=58 2023-10-11T09:52:23.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.971+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:325 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34000 con 0x561e8b013000 2023-10-11T09:52:23.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.971+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 63 ==== osd_op_reply(325 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.971+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.971+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:326 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b220c00 con 0x561e8b013800 2023-10-11T09:52:23.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.971+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 141 ==== osd_op_reply(326 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.971+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.971+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.971+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:23.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.971+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:327 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221000 con 0x561e8b182400 2023-10-11T09:52:23.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.975+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 64 ==== osd_op_reply(327 10000000000.01800000 [omap-get-header out=274b] v0'0 uv24 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.975+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 24 2023-10-11T09:52:23.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.975+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:23.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.975+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:328 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221400 con 0x561e8b182400 2023-10-11T09:52:23.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.975+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 65 ==== osd_op_reply(328 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv24 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.975+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=58 frag=1*, r=0 2023-10-11T09:52:23.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.975+0000 7f239a687840 20 datascan.read_dentry: 58_head not found in result 2023-10-11T09:52:23.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.975+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000022f at 0x0x10000000000/58 with size=2 bytes 2023-10-11T09:52:23.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.975+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:329 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221800 con 0x561e8b182400 2023-10-11T09:52:23.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.979+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 66 ==== osd_op_reply(329 10000000000.01800000 [omap-set-vals] v129'25 uv25 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:23.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.979+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/58 pointing to 0x0x1000000022f 2023-10-11T09:52:23.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.979+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:330 18.2e 18:74000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221c00 con 0x561e8b013000 2023-10-11T09:52:23.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.979+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 64 ==== osd_op_reply(330 [pgnls-filter start_epoch 129 out=101b] v129'19 uv19 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.979+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 1000000021c.00000000 2023-10-11T09:52:23.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.979+0000 7f239a687840 10 datascan.operator(): handling object 1000000021c.0 2023-10-11T09:52:23.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.979+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:331 18.2e 18:74d13cfc:::1000000021c.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b013400 con 0x561e8b013000 2023-10-11T09:52:23.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.979+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 65 ==== osd_op_reply(331 1000000021c.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.979+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x1000000021c 2023-10-11T09:52:23.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.979+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/39 2023-10-11T09:52:23.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.979+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=39 2023-10-11T09:52:23.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.979+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:332 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ec800 con 0x561e8b013000 2023-10-11T09:52:23.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.979+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 66 ==== osd_op_reply(332 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.979+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.979+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:333 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ed800 con 0x561e8b013800 2023-10-11T09:52:23.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.983+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 142 ==== osd_op_reply(333 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.983+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.983+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:23.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.983+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:23.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.983+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:334 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ec000 con 0x561e8b013800 2023-10-11T09:52:23.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.983+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 143 ==== osd_op_reply(334 10000000000.01000000 [omap-get-header out=274b] v0'0 uv30 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.983+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 30 2023-10-11T09:52:23.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.983+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:23.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.983+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:335 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1edc00 con 0x561e8b013800 2023-10-11T09:52:23.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.983+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 144 ==== osd_op_reply(335 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv30 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.983+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=39 frag=0*, r=0 2023-10-11T09:52:23.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.983+0000 7f239a687840 20 datascan.read_dentry: 39_head not found in result 2023-10-11T09:52:23.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.983+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000021c at 0x0x10000000000/39 with size=2 bytes 2023-10-11T09:52:23.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.983+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:336 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183000 con 0x561e8b013800 2023-10-11T09:52:23.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.987+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 145 ==== osd_op_reply(336 10000000000.01000000 [omap-set-vals] v129'31 uv31 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.987+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/39 pointing to 0x0x1000000021c 2023-10-11T09:52:23.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.987+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:337 18.2e 18:7528f44f::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b182c00 con 0x561e8b013000 2023-10-11T09:52:23.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.987+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 67 ==== osd_op_reply(337 [pgnls-filter start_epoch 129 out=101b] v129'19 uv19 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.987+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000242.00000000 2023-10-11T09:52:23.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.987+0000 7f239a687840 10 datascan.operator(): handling object 10000000242.0 2023-10-11T09:52:23.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.987+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:338 18.2e 18:7528f44f:::10000000242.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183400 con 0x561e8b013000 2023-10-11T09:52:23.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.987+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 68 ==== osd_op_reply(338 10000000242.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv18 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.987+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000242 2023-10-11T09:52:23.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.987+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/77 2023-10-11T09:52:23.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.987+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=77 2023-10-11T09:52:23.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.987+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:339 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b210c00 con 0x561e8b013000 2023-10-11T09:52:23.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.991+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 69 ==== osd_op_reply(339 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:23.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.991+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:23.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.991+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:340 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2c00 con 0x561e8b013800 2023-10-11T09:52:23.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.991+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 146 ==== osd_op_reply(340 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:23.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.991+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:23.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.991+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.991+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:24.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.991+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:341 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221000 con 0x561e8b013800 2023-10-11T09:52:24.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.991+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 147 ==== osd_op_reply(341 10000000000.01000000 [omap-get-header out=274b] v0'0 uv31 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.991+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 31 2023-10-11T09:52:24.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.991+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:24.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.991+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:342 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35400 con 0x561e8b013800 2023-10-11T09:52:24.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.991+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 148 ==== osd_op_reply(342 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv31 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.991+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=77 frag=0*, r=0 2023-10-11T09:52:24.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.991+0000 7f239a687840 20 datascan.read_dentry: 77_head not found in result 2023-10-11T09:52:24.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.991+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000242 at 0x0x10000000000/77 with size=2 bytes 2023-10-11T09:52:24.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.991+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:343 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34c00 con 0x561e8b013800 2023-10-11T09:52:24.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.995+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 149 ==== osd_op_reply(343 10000000000.01000000 [omap-set-vals] v129'32 uv32 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.995+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/77 pointing to 0x0x10000000242 2023-10-11T09:52:24.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.995+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:344 18.2e 18:7611a124::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34400 con 0x561e8b013000 2023-10-11T09:52:24.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.995+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 70 ==== osd_op_reply(344 [pgnls-filter start_epoch 129 out=81b] v129'19 uv19 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.995+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 1000000021e.00000000 2023-10-11T09:52:24.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.995+0000 7f239a687840 10 datascan.operator(): handling object 1000000021e.0 2023-10-11T09:52:24.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.995+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:345 18.2e 18:7611a124:::1000000021e.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2000 con 0x561e8b013000 2023-10-11T09:52:24.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.999+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 71 ==== osd_op_reply(345 1000000021e.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv19 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.999+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x1000000021e 2023-10-11T09:52:24.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.999+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/41 2023-10-11T09:52:24.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.999+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=41 2023-10-11T09:52:24.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.999+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:346 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3000 con 0x561e8b013000 2023-10-11T09:52:24.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.999+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 72 ==== osd_op_reply(346 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.999+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.999+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:347 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b211c00 con 0x561e8b013800 2023-10-11T09:52:24.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.999+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 150 ==== osd_op_reply(347 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.999+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.999+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.999+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:24.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.999+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:348 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b211400 con 0x561e8b182400 2023-10-11T09:52:24.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.999+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 67 ==== osd_op_reply(348 10000000000.01800000 [omap-get-header out=274b] v0'0 uv25 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.999+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 25 2023-10-11T09:52:24.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.999+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:24.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:23.999+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:349 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b211800 con 0x561e8b182400 2023-10-11T09:52:24.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.003+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 68 ==== osd_op_reply(349 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv25 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.003+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=41 frag=1*, r=0 2023-10-11T09:52:24.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.003+0000 7f239a687840 20 datascan.read_dentry: 41_head not found in result 2023-10-11T09:52:24.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.003+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000021e at 0x0x10000000000/41 with size=2 bytes 2023-10-11T09:52:24.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.003+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:350 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b210400 con 0x561e8b182400 2023-10-11T09:52:24.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.003+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 69 ==== osd_op_reply(350 10000000000.01800000 [omap-set-vals] v129'26 uv26 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.003+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/41 pointing to 0x0x1000000021e 2023-10-11T09:52:24.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.003+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:351 18.1e 18:78000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b211000 con 0x561e8b183c00 2023-10-11T09:52:24.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.007+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 13 ==== osd_op_reply(351 [pgnls-filter start_epoch 129 out=101b] v129'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:24.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.007+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 1000000020c.00000000 2023-10-11T09:52:24.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.007+0000 7f239a687840 10 datascan.operator(): handling object 1000000020c.0 2023-10-11T09:52:24.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.007+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:352 18.1e 18:789b801b:::1000000020c.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b210000 con 0x561e8b183c00 2023-10-11T09:52:24.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.007+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 14 ==== osd_op_reply(352 1000000020c.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:24.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.007+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x1000000020c 2023-10-11T09:52:24.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.007+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/23 2023-10-11T09:52:24.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.007+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=23 2023-10-11T09:52:24.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.007+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:353 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34800 con 0x561e8b013000 2023-10-11T09:52:24.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.007+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 73 ==== osd_op_reply(353 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.007+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.007+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:354 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35800 con 0x561e8b013800 2023-10-11T09:52:24.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.007+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 151 ==== osd_op_reply(354 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.007+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.007+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.007+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:24.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.007+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:355 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34000 con 0x561e8b182400 2023-10-11T09:52:24.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.011+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 70 ==== osd_op_reply(355 10000000000.01800000 [omap-get-header out=274b] v0'0 uv26 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.011+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 26 2023-10-11T09:52:24.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.011+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:24.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.011+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:356 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34400 con 0x561e8b182400 2023-10-11T09:52:24.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.011+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 71 ==== osd_op_reply(356 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv26 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.011+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=23 frag=1*, r=0 2023-10-11T09:52:24.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.011+0000 7f239a687840 20 datascan.read_dentry: 23_head not found in result 2023-10-11T09:52:24.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.011+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000020c at 0x0x10000000000/23 with size=2 bytes 2023-10-11T09:52:24.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.011+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:357 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b210c00 con 0x561e8b182400 2023-10-11T09:52:24.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.015+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 72 ==== osd_op_reply(357 10000000000.01800000 [omap-set-vals] v129'27 uv27 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.015+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/23 pointing to 0x0x1000000020c 2023-10-11T09:52:24.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.015+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:358 18.1e 18:7a9d9b47::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183800 con 0x561e8b183c00 2023-10-11T09:52:24.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.015+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 15 ==== osd_op_reply(358 [pgnls-filter start_epoch 129 out=101b] v129'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:24.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.015+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 1000000022b.00000000 2023-10-11T09:52:24.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.015+0000 7f239a687840 10 datascan.operator(): handling object 1000000022b.0 2023-10-11T09:52:24.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.015+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:359 18.1e 18:7a9d9b47:::1000000022b.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b182c00 con 0x561e8b183c00 2023-10-11T09:52:24.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.015+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 16 ==== osd_op_reply(359 1000000022b.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:24.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.015+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x1000000022b 2023-10-11T09:52:24.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.015+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/54 2023-10-11T09:52:24.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.015+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=54 2023-10-11T09:52:24.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.015+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:360 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183400 con 0x561e8b013000 2023-10-11T09:52:24.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.015+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 74 ==== osd_op_reply(360 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.015+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.015+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:361 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ed000 con 0x561e8b013800 2023-10-11T09:52:24.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.019+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 152 ==== osd_op_reply(361 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.019+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.019+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.019+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:24.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.019+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:362 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ecc00 con 0x561e8b182400 2023-10-11T09:52:24.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.019+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 73 ==== osd_op_reply(362 10000000000.01800000 [omap-get-header out=274b] v0'0 uv27 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.019+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 27 2023-10-11T09:52:24.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.019+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:24.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.019+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:363 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ec800 con 0x561e8b182400 2023-10-11T09:52:24.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.019+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 74 ==== osd_op_reply(363 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv27 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.019+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=54 frag=1*, r=0 2023-10-11T09:52:24.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.019+0000 7f239a687840 20 datascan.read_dentry: 54_head not found in result 2023-10-11T09:52:24.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.019+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000022b at 0x0x10000000000/54 with size=2 bytes 2023-10-11T09:52:24.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.019+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:364 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b013400 con 0x561e8b182400 2023-10-11T09:52:24.030 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.023+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 75 ==== osd_op_reply(364 10000000000.01800000 [omap-set-vals] v129'28 uv28 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.030 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.023+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/54 pointing to 0x0x1000000022b 2023-10-11T09:52:24.030 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.023+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:365 18.1e 18:7ade2fd8::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b220000 con 0x561e8b183c00 2023-10-11T09:52:24.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.023+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 17 ==== osd_op_reply(365 [pgnls-filter start_epoch 129 out=81b] v129'14 uv14 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:24.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.023+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 1000000024e.00000000 2023-10-11T09:52:24.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.023+0000 7f239a687840 10 datascan.operator(): handling object 1000000024e.0 2023-10-11T09:52:24.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.023+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:366 18.1e 18:7ade2fd8:::1000000024e.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221c00 con 0x561e8b183c00 2023-10-11T09:52:24.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.023+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 18 ==== osd_op_reply(366 1000000024e.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv14 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:24.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.023+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x1000000024e 2023-10-11T09:52:24.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.023+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/89 2023-10-11T09:52:24.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.023+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=89 2023-10-11T09:52:24.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.023+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:367 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183000 con 0x561e8b013000 2023-10-11T09:52:24.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.023+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 75 ==== osd_op_reply(367 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.027+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.027+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:368 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183800 con 0x561e8b013800 2023-10-11T09:52:24.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.027+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 153 ==== osd_op_reply(368 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.027+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.027+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.027+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:24.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.027+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:369 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35400 con 0x561e8b182400 2023-10-11T09:52:24.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.027+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 76 ==== osd_op_reply(369 10000000000.01800000 [omap-get-header out=274b] v0'0 uv28 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.027+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 28 2023-10-11T09:52:24.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.027+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:24.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.027+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:370 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34c00 con 0x561e8b182400 2023-10-11T09:52:24.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.027+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 77 ==== osd_op_reply(370 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv28 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.027+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=89 frag=1*, r=0 2023-10-11T09:52:24.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.027+0000 7f239a687840 20 datascan.read_dentry: 89_head not found in result 2023-10-11T09:52:24.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.027+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000024e at 0x0x10000000000/89 with size=2 bytes 2023-10-11T09:52:24.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.027+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:371 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35800 con 0x561e8b182400 2023-10-11T09:52:24.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.031+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 78 ==== osd_op_reply(371 10000000000.01800000 [omap-set-vals] v129'29 uv29 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.031+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/89 pointing to 0x0x1000000024e 2023-10-11T09:52:24.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.031+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:372 18.3e 18:7c000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b210800 con 0x561e8b013000 2023-10-11T09:52:24.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.031+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 76 ==== osd_op_reply(372 [pgnls-filter start_epoch 129 out=81b] v129'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.031+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000229.00000000 2023-10-11T09:52:24.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.031+0000 7f239a687840 10 datascan.operator(): handling object 10000000229.0 2023-10-11T09:52:24.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.031+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:373 18.3e 18:7f918703:::10000000229.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b211c00 con 0x561e8b013000 2023-10-11T09:52:24.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.031+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 77 ==== osd_op_reply(373 10000000229.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv7 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.031+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000229 2023-10-11T09:52:24.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.031+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/52 2023-10-11T09:52:24.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.031+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=52 2023-10-11T09:52:24.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.031+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:374 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b211000 con 0x561e8b013000 2023-10-11T09:52:24.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.035+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 78 ==== osd_op_reply(374 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.035+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.035+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:375 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b210000 con 0x561e8b013800 2023-10-11T09:52:24.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.035+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 154 ==== osd_op_reply(375 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.035+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.035+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.035+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:24.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.035+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:376 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b220c00 con 0x561e8b013800 2023-10-11T09:52:24.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.035+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 155 ==== osd_op_reply(376 10000000000.01000000 [omap-get-header out=274b] v0'0 uv32 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.035+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 32 2023-10-11T09:52:24.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.035+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:24.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.035+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:377 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221000 con 0x561e8b013800 2023-10-11T09:52:24.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.035+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 156 ==== osd_op_reply(377 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv32 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.035+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=52 frag=0*, r=0 2023-10-11T09:52:24.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.035+0000 7f239a687840 20 datascan.read_dentry: 52_head not found in result 2023-10-11T09:52:24.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.035+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000229 at 0x0x10000000000/52 with size=2 bytes 2023-10-11T09:52:24.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.035+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:378 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2800 con 0x561e8b013800 2023-10-11T09:52:24.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.039+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 157 ==== osd_op_reply(378 10000000000.01000000 [omap-set-vals] v129'33 uv33 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.039+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/52 pointing to 0x0x10000000229 2023-10-11T09:52:24.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.039+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:379 18.1 18:80000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2c00 con 0x561e8b013000 2023-10-11T09:52:24.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.039+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 79 ==== osd_op_reply(379 [pgnls-filter start_epoch 129 out=81b] v129'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.039+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000224.00000000 2023-10-11T09:52:24.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.039+0000 7f239a687840 10 datascan.operator(): handling object 10000000224.0 2023-10-11T09:52:24.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.039+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:380 18.1 18:805bc955:::10000000224.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ed400 con 0x561e8b013000 2023-10-11T09:52:24.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.039+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 80 ==== osd_op_reply(380 10000000224.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv8 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.039+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000224 2023-10-11T09:52:24.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.039+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/47 2023-10-11T09:52:24.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.039+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=47 2023-10-11T09:52:24.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.039+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:381 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ec400 con 0x561e8b013000 2023-10-11T09:52:24.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.043+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 81 ==== osd_op_reply(381 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.043+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.043+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:382 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ed800 con 0x561e8b013800 2023-10-11T09:52:24.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.043+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 158 ==== osd_op_reply(382 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.043+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.043+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.043+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:24.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.043+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:383 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ec000 con 0x561e8b182400 2023-10-11T09:52:24.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.043+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 79 ==== osd_op_reply(383 10000000000.01800000 [omap-get-header out=274b] v0'0 uv29 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.043+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 29 2023-10-11T09:52:24.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.043+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:24.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.043+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:384 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1edc00 con 0x561e8b182400 2023-10-11T09:52:24.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.043+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 80 ==== osd_op_reply(384 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv29 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.043+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=47 frag=1*, r=0 2023-10-11T09:52:24.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.043+0000 7f239a687840 20 datascan.read_dentry: 47_head not found in result 2023-10-11T09:52:24.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.043+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000224 at 0x0x10000000000/47 with size=2 bytes 2023-10-11T09:52:24.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.043+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:385 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34000 con 0x561e8b182400 2023-10-11T09:52:24.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.047+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 81 ==== osd_op_reply(385 10000000000.01800000 [omap-set-vals] v129'30 uv30 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.047+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/47 pointing to 0x0x10000000224 2023-10-11T09:52:24.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.047+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:386 18.21 18:84000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34400 con 0x561e8b183c00 2023-10-11T09:52:24.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.047+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 19 ==== osd_op_reply(386 [pgnls-filter start_epoch 129 out=49b] v127'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:24.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.047+0000 7f239463a700 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:387 18.11 18:88000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b220400 con 0x561e8ae35000 2023-10-11T09:52:24.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.047+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.1 v2:172.21.15.78:6809/35734 11 ==== osd_op_reply(387 [pgnls-filter start_epoch 129 out=49b] v127'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8ae35000 2023-10-11T09:52:24.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.047+0000 7f2393e39700 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:388 18.31 18:8c000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3400 con 0x561e8b182400 2023-10-11T09:52:24.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.051+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 82 ==== osd_op_reply(388 [pgnls-filter start_epoch 129 out=101b] v129'13 uv13 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.051+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000216.00000000 2023-10-11T09:52:24.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.051+0000 7f239a687840 10 datascan.operator(): handling object 10000000216.0 2023-10-11T09:52:24.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.051+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:389 18.31 18:8c51d9b0:::10000000216.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35400 con 0x561e8b182400 2023-10-11T09:52:24.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.051+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 83 ==== osd_op_reply(389 10000000216.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.051+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000216 2023-10-11T09:52:24.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.051+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/33 2023-10-11T09:52:24.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.051+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=33 2023-10-11T09:52:24.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.051+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:390 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34c00 con 0x561e8b013000 2023-10-11T09:52:24.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.051+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 82 ==== osd_op_reply(390 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.051+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.051+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:391 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b013400 con 0x561e8b013800 2023-10-11T09:52:24.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.051+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 159 ==== osd_op_reply(391 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.051+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.051+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.051+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:24.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.051+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:392 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ecc00 con 0x561e8b182400 2023-10-11T09:52:24.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.055+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 84 ==== osd_op_reply(392 10000000000.01800000 [omap-get-header out=274b] v0'0 uv30 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.055+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 30 2023-10-11T09:52:24.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.055+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:24.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.055+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:393 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ec800 con 0x561e8b182400 2023-10-11T09:52:24.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.055+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 85 ==== osd_op_reply(393 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv30 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.055+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=33 frag=1*, r=0 2023-10-11T09:52:24.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.055+0000 7f239a687840 20 datascan.read_dentry: 33_head not found in result 2023-10-11T09:52:24.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.055+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000216 at 0x0x10000000000/33 with size=2 bytes 2023-10-11T09:52:24.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.055+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:394 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b211400 con 0x561e8b182400 2023-10-11T09:52:24.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.059+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 86 ==== osd_op_reply(394 10000000000.01800000 [omap-set-vals] v129'31 uv31 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.059+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/33 pointing to 0x0x10000000216 2023-10-11T09:52:24.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.059+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:395 18.31 18:8fb7f7b7::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b211800 con 0x561e8b182400 2023-10-11T09:52:24.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.059+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 87 ==== osd_op_reply(395 [pgnls-filter start_epoch 129 out=81b] v129'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.059+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000251.00000000 2023-10-11T09:52:24.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.059+0000 7f239a687840 10 datascan.operator(): handling object 10000000251.0 2023-10-11T09:52:24.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.059+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:396 18.31 18:8fb7f7b7:::10000000251.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b210400 con 0x561e8b182400 2023-10-11T09:52:24.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.059+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 88 ==== osd_op_reply(396 10000000251.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.059+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000251 2023-10-11T09:52:24.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.059+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/92 2023-10-11T09:52:24.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.059+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=92 2023-10-11T09:52:24.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.059+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:397 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b210c00 con 0x561e8b013000 2023-10-11T09:52:24.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.059+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 83 ==== osd_op_reply(397 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.059+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.059+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:398 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221400 con 0x561e8b013800 2023-10-11T09:52:24.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.059+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 160 ==== osd_op_reply(398 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.059+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.059+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.059+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:24.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.059+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:399 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221800 con 0x561e8b013800 2023-10-11T09:52:24.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.063+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 161 ==== osd_op_reply(399 10000000000.01000000 [omap-get-header out=274b] v0'0 uv33 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.063+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 33 2023-10-11T09:52:24.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.063+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:24.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.063+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:400 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b228000 con 0x561e8b013800 2023-10-11T09:52:24.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.063+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 162 ==== osd_op_reply(400 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv33 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.063+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=92 frag=0*, r=0 2023-10-11T09:52:24.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.063+0000 7f239a687840 20 datascan.read_dentry: 92_head not found in result 2023-10-11T09:52:24.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.063+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000251 at 0x0x10000000000/92 with size=2 bytes 2023-10-11T09:52:24.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.063+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:401 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b228400 con 0x561e8b013800 2023-10-11T09:52:24.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.067+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 163 ==== osd_op_reply(401 10000000000.01000000 [omap-set-vals] v129'34 uv34 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.067+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/92 pointing to 0x0x10000000251 2023-10-11T09:52:24.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.067+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:402 18.9 18:90000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b228800 con 0x561e8b013000 2023-10-11T09:52:24.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.067+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 84 ==== osd_op_reply(402 [pgnls-filter start_epoch 129 out=101b] v129'8 uv8 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.067+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 1000000024d.00000000 2023-10-11T09:52:24.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.067+0000 7f239a687840 10 datascan.operator(): handling object 1000000024d.0 2023-10-11T09:52:24.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.067+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:403 18.9 18:915bff82:::1000000024d.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b228c00 con 0x561e8b013000 2023-10-11T09:52:24.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.067+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 85 ==== osd_op_reply(403 1000000024d.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv7 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.067+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x1000000024d 2023-10-11T09:52:24.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.067+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/88 2023-10-11T09:52:24.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.067+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=88 2023-10-11T09:52:24.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.067+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:404 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b229000 con 0x561e8b013000 2023-10-11T09:52:24.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.067+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 86 ==== osd_op_reply(404 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.067+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.067+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:405 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b229400 con 0x561e8b013800 2023-10-11T09:52:24.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.071+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 164 ==== osd_op_reply(405 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.071+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.071+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.071+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:24.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.071+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:406 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b229800 con 0x561e8b013800 2023-10-11T09:52:24.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.071+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 165 ==== osd_op_reply(406 10000000000.01000000 [omap-get-header out=274b] v0'0 uv34 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.071+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 34 2023-10-11T09:52:24.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.071+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:24.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.071+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:407 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b229c00 con 0x561e8b013800 2023-10-11T09:52:24.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.071+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 166 ==== osd_op_reply(407 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv34 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.071+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=88 frag=0*, r=0 2023-10-11T09:52:24.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.071+0000 7f239a687840 20 datascan.read_dentry: 88_head not found in result 2023-10-11T09:52:24.080 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.071+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000024d at 0x0x10000000000/88 with size=2 bytes 2023-10-11T09:52:24.081 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.071+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:408 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b228000 con 0x561e8b013800 2023-10-11T09:52:24.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.075+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 167 ==== osd_op_reply(408 10000000000.01000000 [omap-set-vals] v129'35 uv35 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.075+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/88 pointing to 0x0x1000000024d 2023-10-11T09:52:24.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.075+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:409 18.9 18:91e366d7::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b228400 con 0x561e8b013000 2023-10-11T09:52:24.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.075+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 87 ==== osd_op_reply(409 [pgnls-filter start_epoch 129 out=81b] v129'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.075+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000230.00000000 2023-10-11T09:52:24.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.075+0000 7f239a687840 10 datascan.operator(): handling object 10000000230.0 2023-10-11T09:52:24.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.075+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:410 18.9 18:91e366d7:::10000000230.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b229400 con 0x561e8b013000 2023-10-11T09:52:24.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.075+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 88 ==== osd_op_reply(410 10000000230.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv8 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.084 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.075+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000230 2023-10-11T09:52:24.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.075+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/59 2023-10-11T09:52:24.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.075+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=59 2023-10-11T09:52:24.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.075+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:411 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b229800 con 0x561e8b013000 2023-10-11T09:52:24.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.075+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 89 ==== osd_op_reply(411 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.075+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.075+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:412 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b210000 con 0x561e8b013800 2023-10-11T09:52:24.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.079+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 168 ==== osd_op_reply(412 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.079+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.079+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.079+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:24.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.079+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:413 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b210800 con 0x561e8b013800 2023-10-11T09:52:24.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.079+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 169 ==== osd_op_reply(413 10000000000.01000000 [omap-get-header out=274b] v0'0 uv35 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.079+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 35 2023-10-11T09:52:24.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.079+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:24.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.079+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:414 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b211c00 con 0x561e8b013800 2023-10-11T09:52:24.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.079+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 170 ==== osd_op_reply(414 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv35 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.079+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=59 frag=0*, r=0 2023-10-11T09:52:24.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.079+0000 7f239a687840 20 datascan.read_dentry: 59_head not found in result 2023-10-11T09:52:24.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.079+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000230 at 0x0x10000000000/59 with size=2 bytes 2023-10-11T09:52:24.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.079+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:415 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b211000 con 0x561e8b013800 2023-10-11T09:52:24.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.083+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 171 ==== osd_op_reply(415 10000000000.01000000 [omap-set-vals] v129'36 uv36 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.083+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/59 pointing to 0x0x10000000230 2023-10-11T09:52:24.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.083+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:416 18.29 18:94000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b210c00 con 0x561e8b1e3800 2023-10-11T09:52:24.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.083+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.7 v2:172.21.15.133:6824/35901 5 ==== osd_op_reply(416 [pgnls-filter start_epoch 129 out=101b] v129'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b1e3800 2023-10-11T09:52:24.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.083+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 1000000024a.00000000 2023-10-11T09:52:24.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.083+0000 7f239a687840 10 datascan.operator(): handling object 1000000024a.0 2023-10-11T09:52:24.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.083+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:417 18.29 18:94c02207:::1000000024a.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34400 con 0x561e8b1e3800 2023-10-11T09:52:24.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.083+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.7 v2:172.21.15.133:6824/35901 6 ==== osd_op_reply(417 1000000024a.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b1e3800 2023-10-11T09:52:24.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.083+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x1000000024a 2023-10-11T09:52:24.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.083+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/85 2023-10-11T09:52:24.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.083+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=85 2023-10-11T09:52:24.093 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.083+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:418 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34800 con 0x561e8b013000 2023-10-11T09:52:24.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.087+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 90 ==== osd_op_reply(418 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.087+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.087+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:419 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34c00 con 0x561e8b013800 2023-10-11T09:52:24.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.087+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 172 ==== osd_op_reply(419 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.087+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.087+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.087+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:24.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.087+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:420 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b220000 con 0x561e8b013800 2023-10-11T09:52:24.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.087+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 173 ==== osd_op_reply(420 10000000000.01000000 [omap-get-header out=274b] v0'0 uv36 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.087+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 36 2023-10-11T09:52:24.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.087+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:24.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.087+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:421 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221c00 con 0x561e8b013800 2023-10-11T09:52:24.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.087+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 174 ==== osd_op_reply(421 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv36 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.087+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=85 frag=0*, r=0 2023-10-11T09:52:24.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.087+0000 7f239a687840 20 datascan.read_dentry: 85_head not found in result 2023-10-11T09:52:24.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.087+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000024a at 0x0x10000000000/85 with size=2 bytes 2023-10-11T09:52:24.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.087+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:422 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b220c00 con 0x561e8b013800 2023-10-11T09:52:24.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.091+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 175 ==== osd_op_reply(422 10000000000.01000000 [omap-set-vals] v129'37 uv37 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.091+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/85 pointing to 0x0x1000000024a 2023-10-11T09:52:24.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.091+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:423 18.29 18:95e8e66d::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221000 con 0x561e8b1e3800 2023-10-11T09:52:24.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.091+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.7 v2:172.21.15.133:6824/35901 7 ==== osd_op_reply(423 [pgnls-filter start_epoch 129 out=101b] v129'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b1e3800 2023-10-11T09:52:24.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.091+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000256.00000000 2023-10-11T09:52:24.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.091+0000 7f239a687840 10 datascan.operator(): handling object 10000000256.0 2023-10-11T09:52:24.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.091+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:424 18.29 18:95e8e66d:::10000000256.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221400 con 0x561e8b1e3800 2023-10-11T09:52:24.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.095+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.7 v2:172.21.15.133:6824/35901 8 ==== osd_op_reply(424 10000000256.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b1e3800 2023-10-11T09:52:24.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.095+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000256 2023-10-11T09:52:24.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.095+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/97 2023-10-11T09:52:24.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.095+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=97 2023-10-11T09:52:24.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.095+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:425 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221800 con 0x561e8b013000 2023-10-11T09:52:24.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.095+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 91 ==== osd_op_reply(425 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.095+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.095+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:426 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b220400 con 0x561e8b013800 2023-10-11T09:52:24.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.095+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 176 ==== osd_op_reply(426 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.095+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.095+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.095+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:24.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.095+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:427 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ed000 con 0x561e8b013800 2023-10-11T09:52:24.104 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.095+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 177 ==== osd_op_reply(427 10000000000.01000000 [omap-get-header out=274b] v0'0 uv37 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.095+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 37 2023-10-11T09:52:24.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.095+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:24.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.095+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:428 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ed800 con 0x561e8b013800 2023-10-11T09:52:24.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.095+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 178 ==== osd_op_reply(428 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv37 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.095+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=97 frag=0*, r=0 2023-10-11T09:52:24.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.095+0000 7f239a687840 20 datascan.read_dentry: 97_head not found in result 2023-10-11T09:52:24.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.095+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000256 at 0x0x10000000000/97 with size=2 bytes 2023-10-11T09:52:24.106 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.095+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:429 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ed400 con 0x561e8b013800 2023-10-11T09:52:24.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.099+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 179 ==== osd_op_reply(429 10000000000.01000000 [omap-set-vals] v129'38 uv38 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.099+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/97 pointing to 0x0x10000000256 2023-10-11T09:52:24.108 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.099+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:430 18.29 18:97401066::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1ec400 con 0x561e8b1e3800 2023-10-11T09:52:24.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.103+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.7 v2:172.21.15.133:6824/35901 9 ==== osd_op_reply(430 [pgnls-filter start_epoch 129 out=81b] v129'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b1e3800 2023-10-11T09:52:24.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.103+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000247.00000000 2023-10-11T09:52:24.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.103+0000 7f239a687840 10 datascan.operator(): handling object 10000000247.0 2023-10-11T09:52:24.109 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.103+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:431 18.29 18:97401066:::10000000247.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2800 con 0x561e8b1e3800 2023-10-11T09:52:24.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.103+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.7 v2:172.21.15.133:6824/35901 10 ==== osd_op_reply(431 10000000247.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b1e3800 2023-10-11T09:52:24.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.103+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000247 2023-10-11T09:52:24.110 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.103+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/82 2023-10-11T09:52:24.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.103+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=82 2023-10-11T09:52:24.111 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.103+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:432 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2000 con 0x561e8b013000 2023-10-11T09:52:24.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.103+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 92 ==== osd_op_reply(432 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.103+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.103+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:433 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3000 con 0x561e8b013800 2023-10-11T09:52:24.112 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.103+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 180 ==== osd_op_reply(433 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.103+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.103+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.103+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:24.113 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.103+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:434 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2c00 con 0x561e8b182400 2023-10-11T09:52:24.114 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.107+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 89 ==== osd_op_reply(434 10000000000.01800000 [omap-get-header out=274b] v0'0 uv31 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.114 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.107+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 31 2023-10-11T09:52:24.114 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.107+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:24.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.107+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:435 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b182c00 con 0x561e8b182400 2023-10-11T09:52:24.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.107+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 90 ==== osd_op_reply(435 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv31 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.107+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=82 frag=1*, r=0 2023-10-11T09:52:24.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.107+0000 7f239a687840 20 datascan.read_dentry: 82_head not found in result 2023-10-11T09:52:24.115 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.107+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000247 at 0x0x10000000000/82 with size=2 bytes 2023-10-11T09:52:24.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.107+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:436 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183800 con 0x561e8b182400 2023-10-11T09:52:24.116 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.111+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 91 ==== osd_op_reply(436 10000000000.01800000 [omap-set-vals] v129'32 uv32 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.111+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/82 pointing to 0x0x10000000247 2023-10-11T09:52:24.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.111+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:437 18.19 18:98000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183400 con 0x561e8b182400 2023-10-11T09:52:24.117 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.111+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 92 ==== osd_op_reply(437 [pgnls-filter start_epoch 129 out=101b] v129'16 uv16 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.111+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000200.00000000 2023-10-11T09:52:24.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.111+0000 7f239a687840 10 datascan.operator(): handling object 10000000200.0 2023-10-11T09:52:24.118 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.111+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:438 18.19 18:992a6246:::10000000200.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183000 con 0x561e8b182400 2023-10-11T09:52:24.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.111+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 93 ==== osd_op_reply(438 10000000200.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.111+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000200 2023-10-11T09:52:24.119 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.111+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/11 2023-10-11T09:52:24.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.111+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=11 2023-10-11T09:52:24.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.111+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:439 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b013400 con 0x561e8b013000 2023-10-11T09:52:24.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.111+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 93 ==== osd_op_reply(439 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.111+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.111+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:440 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b228000 con 0x561e8b013800 2023-10-11T09:52:24.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.111+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 181 ==== osd_op_reply(440 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.111+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.111+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.111+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:24.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.111+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:441 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b229c00 con 0x561e8b013800 2023-10-11T09:52:24.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.115+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 182 ==== osd_op_reply(441 10000000000.01000000 [omap-get-header out=274b] v0'0 uv38 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.115+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 38 2023-10-11T09:52:24.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.115+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:24.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.115+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:442 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b228800 con 0x561e8b013800 2023-10-11T09:52:24.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.115+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 183 ==== osd_op_reply(442 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv38 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.115+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=11 frag=0*, r=0 2023-10-11T09:52:24.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.115+0000 7f239a687840 20 datascan.read_dentry: 11_head not found in result 2023-10-11T09:52:24.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.115+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000200 at 0x0x10000000000/11 with size=2 bytes 2023-10-11T09:52:24.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.115+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:443 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3400 con 0x561e8b013800 2023-10-11T09:52:24.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.115+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 184 ==== osd_op_reply(443 10000000000.01000000 [omap-set-vals] v129'39 uv39 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.115+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/11 pointing to 0x0x10000000200 2023-10-11T09:52:24.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.115+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:444 18.19 18:997b7129::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2c00 con 0x561e8b182400 2023-10-11T09:52:24.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.119+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 94 ==== osd_op_reply(444 [pgnls-filter start_epoch 129 out=81b] v129'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.119+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 100000001f8.00000000 2023-10-11T09:52:24.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.119+0000 7f239a687840 10 datascan.operator(): handling object 100000001f8.0 2023-10-11T09:52:24.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.119+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:445 18.19 18:997b7129:::100000001f8.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2800 con 0x561e8b182400 2023-10-11T09:52:24.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.119+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 95 ==== osd_op_reply(445 100000001f8.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.119+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x100000001f8 2023-10-11T09:52:24.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.119+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/3 2023-10-11T09:52:24.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.119+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=3 2023-10-11T09:52:24.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.119+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:446 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3000 con 0x561e8b013000 2023-10-11T09:52:24.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.119+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 94 ==== osd_op_reply(446 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.119+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.119+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:447 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b220000 con 0x561e8b013800 2023-10-11T09:52:24.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.119+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 185 ==== osd_op_reply(447 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.119+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.119+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.119+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:24.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.119+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:448 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221c00 con 0x561e8b013800 2023-10-11T09:52:24.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.123+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 186 ==== osd_op_reply(448 10000000000.01000000 [omap-get-header out=274b] v0'0 uv39 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.123+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 39 2023-10-11T09:52:24.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.123+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:24.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.123+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:449 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b220c00 con 0x561e8b013800 2023-10-11T09:52:24.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.123+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 187 ==== osd_op_reply(449 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv39 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.123+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=3 frag=0*, r=0 2023-10-11T09:52:24.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.123+0000 7f239a687840 20 datascan.read_dentry: 3_head not found in result 2023-10-11T09:52:24.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.123+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f8 at 0x0x10000000000/3 with size=1 bytes 2023-10-11T09:52:24.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.123+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:450 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221000 con 0x561e8b013800 2023-10-11T09:52:24.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.123+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 188 ==== osd_op_reply(450 10000000000.01000000 [omap-set-vals] v129'40 uv40 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.123+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/3 pointing to 0x0x100000001f8 2023-10-11T09:52:24.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.123+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:451 18.39 18:9c000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221400 con 0x561e8b182800 2023-10-11T09:52:24.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.127+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.0 v2:172.21.15.78:6800/35735 23 ==== osd_op_reply(451 [pgnls-filter start_epoch 129 out=81b] v129'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b182800 2023-10-11T09:52:24.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.127+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000222.00000000 2023-10-11T09:52:24.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.127+0000 7f239a687840 10 datascan.operator(): handling object 10000000222.0 2023-10-11T09:52:24.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.127+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:452 18.39 18:9f1351c3:::10000000222.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b220400 con 0x561e8b182800 2023-10-11T09:52:24.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.127+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.0 v2:172.21.15.78:6800/35735 24 ==== osd_op_reply(452 10000000222.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv8 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b182800 2023-10-11T09:52:24.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.127+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000222 2023-10-11T09:52:24.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.127+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/45 2023-10-11T09:52:24.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.127+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=45 2023-10-11T09:52:24.136 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.127+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:453 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35800 con 0x561e8b013000 2023-10-11T09:52:24.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.127+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 95 ==== osd_op_reply(453 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.127+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.127+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:454 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34000 con 0x561e8b013800 2023-10-11T09:52:24.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.127+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 189 ==== osd_op_reply(454 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.127+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.127+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.127+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:24.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.127+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:455 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35400 con 0x561e8b182400 2023-10-11T09:52:24.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.131+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 96 ==== osd_op_reply(455 10000000000.01800000 [omap-get-header out=274b] v0'0 uv32 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.131+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 32 2023-10-11T09:52:24.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.131+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:24.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.131+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:456 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34400 con 0x561e8b182400 2023-10-11T09:52:24.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.131+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 97 ==== osd_op_reply(456 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv32 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.131+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=45 frag=1*, r=0 2023-10-11T09:52:24.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.131+0000 7f239a687840 20 datascan.read_dentry: 45_head not found in result 2023-10-11T09:52:24.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.131+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000222 at 0x0x10000000000/45 with size=2 bytes 2023-10-11T09:52:24.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.131+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:457 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34c00 con 0x561e8b182400 2023-10-11T09:52:24.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.135+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 98 ==== osd_op_reply(457 10000000000.01800000 [omap-set-vals] v129'33 uv33 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.135+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/45 pointing to 0x0x10000000222 2023-10-11T09:52:24.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.135+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:458 18.5 18:a0000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b182c00 con 0x561e8ae35000 2023-10-11T09:52:24.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.135+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.1 v2:172.21.15.78:6809/35734 12 ==== osd_op_reply(458 [pgnls-filter start_epoch 129 out=101b] v129'11 uv11 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8ae35000 2023-10-11T09:52:24.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.135+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000204.00000000 2023-10-11T09:52:24.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.135+0000 7f239a687840 10 datascan.operator(): handling object 10000000204.0 2023-10-11T09:52:24.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.135+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:459 18.5 18:a0beda95:::10000000204.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183800 con 0x561e8ae35000 2023-10-11T09:52:24.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.135+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.1 v2:172.21.15.78:6809/35734 13 ==== osd_op_reply(459 10000000204.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv10 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8ae35000 2023-10-11T09:52:24.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.135+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000204 2023-10-11T09:52:24.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.135+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/15 2023-10-11T09:52:24.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.135+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=15 2023-10-11T09:52:24.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.135+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:460 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b2000 con 0x561e8b013000 2023-10-11T09:52:24.145 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.135+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 96 ==== osd_op_reply(460 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.145 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.135+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.135+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:461 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b2400 con 0x561e8b013800 2023-10-11T09:52:24.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.135+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 190 ==== osd_op_reply(461 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.135+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.135+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.135+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:24.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.135+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:462 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b2800 con 0x561e8b013800 2023-10-11T09:52:24.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.139+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 191 ==== osd_op_reply(462 10000000000.01000000 [omap-get-header out=274b] v0'0 uv40 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.139+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 40 2023-10-11T09:52:24.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.139+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:24.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.139+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:463 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b2c00 con 0x561e8b013800 2023-10-11T09:52:24.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.139+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 192 ==== osd_op_reply(463 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv40 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.139+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=15 frag=0*, r=0 2023-10-11T09:52:24.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.139+0000 7f239a687840 20 datascan.read_dentry: 15_head not found in result 2023-10-11T09:52:24.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.139+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000204 at 0x0x10000000000/15 with size=2 bytes 2023-10-11T09:52:24.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.139+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:464 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b3000 con 0x561e8b013800 2023-10-11T09:52:24.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.143+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 193 ==== osd_op_reply(464 10000000000.01000000 [omap-set-vals] v129'41 uv41 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.143+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/15 pointing to 0x0x10000000204 2023-10-11T09:52:24.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.143+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:465 18.5 18:a3a50212::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b3400 con 0x561e8ae35000 2023-10-11T09:52:24.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.143+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.1 v2:172.21.15.78:6809/35734 14 ==== osd_op_reply(465 [pgnls-filter start_epoch 129 out=81b] v129'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8ae35000 2023-10-11T09:52:24.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.143+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000254.00000000 2023-10-11T09:52:24.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.143+0000 7f239a687840 10 datascan.operator(): handling object 10000000254.0 2023-10-11T09:52:24.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.143+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:466 18.5 18:a3a50212:::10000000254.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b3800 con 0x561e8ae35000 2023-10-11T09:52:24.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.143+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.1 v2:172.21.15.78:6809/35734 15 ==== osd_op_reply(466 10000000254.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8ae35000 2023-10-11T09:52:24.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.143+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000254 2023-10-11T09:52:24.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.143+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/95 2023-10-11T09:52:24.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.143+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=95 2023-10-11T09:52:24.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.143+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:467 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b3c00 con 0x561e8b013000 2023-10-11T09:52:24.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.143+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 97 ==== osd_op_reply(467 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.143+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.143+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:468 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b2000 con 0x561e8b013800 2023-10-11T09:52:24.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.143+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 194 ==== osd_op_reply(468 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.143+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.143+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.154 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.143+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:24.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.143+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:469 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b182c00 con 0x561e8b013800 2023-10-11T09:52:24.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.147+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 195 ==== osd_op_reply(469 10000000000.01000000 [omap-get-header out=274b] v0'0 uv41 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.147+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 41 2023-10-11T09:52:24.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.147+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:24.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.147+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:470 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183800 con 0x561e8b013800 2023-10-11T09:52:24.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.147+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 196 ==== osd_op_reply(470 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv41 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.147+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=95 frag=0*, r=0 2023-10-11T09:52:24.156 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.147+0000 7f239a687840 20 datascan.read_dentry: 95_head not found in result 2023-10-11T09:52:24.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.147+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000254 at 0x0x10000000000/95 with size=2 bytes 2023-10-11T09:52:24.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.147+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:471 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34800 con 0x561e8b013800 2023-10-11T09:52:24.157 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.151+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 197 ==== osd_op_reply(471 10000000000.01000000 [omap-set-vals] v129'42 uv42 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.151+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/95 pointing to 0x0x10000000254 2023-10-11T09:52:24.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.151+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:472 18.25 18:a4000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35800 con 0x561e8b013800 2023-10-11T09:52:24.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.151+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 198 ==== osd_op_reply(472 [pgnls-filter start_epoch 129 out=101b] v129'12 uv12 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.151+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000237.00000000 2023-10-11T09:52:24.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.151+0000 7f239a687840 10 datascan.operator(): handling object 10000000237.0 2023-10-11T09:52:24.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.151+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:473 18.25 18:a53878a9:::10000000237.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221800 con 0x561e8b013800 2023-10-11T09:52:24.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.151+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 199 ==== osd_op_reply(473 10000000237.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.151+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000237 2023-10-11T09:52:24.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.151+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/66 2023-10-11T09:52:24.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.151+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=66 2023-10-11T09:52:24.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.151+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:474 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221400 con 0x561e8b013000 2023-10-11T09:52:24.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.151+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 98 ==== osd_op_reply(474 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.151+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.151+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:475 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b220400 con 0x561e8b013800 2023-10-11T09:52:24.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.155+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 200 ==== osd_op_reply(475 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.155+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.155+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.155+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:24.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.155+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:476 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2000 con 0x561e8b013800 2023-10-11T09:52:24.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.155+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 201 ==== osd_op_reply(476 10000000000.01000000 [omap-get-header out=274b] v0'0 uv42 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.155+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 42 2023-10-11T09:52:24.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.155+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:24.163 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.155+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:477 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3000 con 0x561e8b013800 2023-10-11T09:52:24.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.155+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 202 ==== osd_op_reply(477 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv42 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.155+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=66 frag=0*, r=0 2023-10-11T09:52:24.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.155+0000 7f239a687840 20 datascan.read_dentry: 66_head not found in result 2023-10-11T09:52:24.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.155+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000237 at 0x0x10000000000/66 with size=2 bytes 2023-10-11T09:52:24.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.155+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:478 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b013400 con 0x561e8b013800 2023-10-11T09:52:24.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.159+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 203 ==== osd_op_reply(478 10000000000.01000000 [omap-set-vals] v129'43 uv43 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.159+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/66 pointing to 0x0x10000000237 2023-10-11T09:52:24.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.159+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:479 18.25 18:a7ae8a0f::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b228c00 con 0x561e8b013800 2023-10-11T09:52:24.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.159+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 204 ==== osd_op_reply(479 [pgnls-filter start_epoch 129 out=81b] v129'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.159+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000243.00000000 2023-10-11T09:52:24.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.159+0000 7f239a687840 10 datascan.operator(): handling object 10000000243.0 2023-10-11T09:52:24.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.159+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:480 18.25 18:a7ae8a0f:::10000000243.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b229000 con 0x561e8b013800 2023-10-11T09:52:24.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.159+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 205 ==== osd_op_reply(480 10000000243.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.159+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000243 2023-10-11T09:52:24.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.159+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/78 2023-10-11T09:52:24.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.159+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=78 2023-10-11T09:52:24.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.159+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:481 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b228400 con 0x561e8b013000 2023-10-11T09:52:24.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.159+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 99 ==== osd_op_reply(481 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.159+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.159+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:482 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b229400 con 0x561e8b013800 2023-10-11T09:52:24.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.163+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 206 ==== osd_op_reply(482 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.163+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.163+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.163+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:24.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.163+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:483 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b229800 con 0x561e8b182400 2023-10-11T09:52:24.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.163+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 99 ==== osd_op_reply(483 10000000000.01800000 [omap-get-header out=274b] v0'0 uv33 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.163+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 33 2023-10-11T09:52:24.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.163+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:24.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.163+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:484 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3400 con 0x561e8b182400 2023-10-11T09:52:24.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.163+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 100 ==== osd_op_reply(484 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv33 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.163+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=78 frag=1*, r=0 2023-10-11T09:52:24.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.163+0000 7f239a687840 20 datascan.read_dentry: 78_head not found in result 2023-10-11T09:52:24.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.163+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000243 at 0x0x10000000000/78 with size=2 bytes 2023-10-11T09:52:24.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.163+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:485 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2000 con 0x561e8b182400 2023-10-11T09:52:24.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.167+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 101 ==== osd_op_reply(485 10000000000.01800000 [omap-set-vals] v129'34 uv34 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.167+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/78 pointing to 0x0x10000000243 2023-10-11T09:52:24.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.167+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:486 18.15 18:a8000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b220000 con 0x561e8b183c00 2023-10-11T09:52:24.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.167+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 20 ==== osd_op_reply(486 [pgnls-filter start_epoch 129 out=81b] v129'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:24.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.167+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 1000000020d.00000000 2023-10-11T09:52:24.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.167+0000 7f239a687840 10 datascan.operator(): handling object 1000000020d.0 2023-10-11T09:52:24.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.167+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:487 18.15 18:a8e69ea8:::1000000020d.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221c00 con 0x561e8b183c00 2023-10-11T09:52:24.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.167+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 21 ==== osd_op_reply(487 1000000020d.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv9 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:24.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.167+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x1000000020d 2023-10-11T09:52:24.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.167+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/24 2023-10-11T09:52:24.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.167+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=24 2023-10-11T09:52:24.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.167+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:488 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b220c00 con 0x561e8b013000 2023-10-11T09:52:24.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.171+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 100 ==== osd_op_reply(488 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.171+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.171+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:489 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221000 con 0x561e8b013800 2023-10-11T09:52:24.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.171+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 207 ==== osd_op_reply(489 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.171+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.171+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.171+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:24.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.171+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:490 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221800 con 0x561e8b182400 2023-10-11T09:52:24.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.171+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 102 ==== osd_op_reply(490 10000000000.01800000 [omap-get-header out=274b] v0'0 uv34 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.171+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 34 2023-10-11T09:52:24.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.171+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:24.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.171+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:491 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b220400 con 0x561e8b182400 2023-10-11T09:52:24.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.171+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 103 ==== osd_op_reply(491 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv34 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.181 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.171+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=24 frag=1*, r=0 2023-10-11T09:52:24.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.171+0000 7f239a687840 20 datascan.read_dentry: 24_head not found in result 2023-10-11T09:52:24.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.171+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000020d at 0x0x10000000000/24 with size=2 bytes 2023-10-11T09:52:24.182 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.171+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:492 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34000 con 0x561e8b182400 2023-10-11T09:52:24.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.175+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 104 ==== osd_op_reply(492 10000000000.01800000 [omap-set-vals] v129'35 uv35 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.175+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/24 pointing to 0x0x1000000020d 2023-10-11T09:52:24.183 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.175+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:493 18.35 18:ac000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34800 con 0x561e8b182400 2023-10-11T09:52:24.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.175+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 105 ==== osd_op_reply(493 [pgnls-filter start_epoch 129 out=49b] v127'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.175+0000 7f2393e39700 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:494 18.d 18:b0000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3c00 con 0x561e8b182400 2023-10-11T09:52:24.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.175+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 106 ==== osd_op_reply(494 [pgnls-filter start_epoch 129 out=49b] v127'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.175+0000 7f239463a700 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:495 18.2d 18:b4000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b220800 con 0x561e8b013800 2023-10-11T09:52:24.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.179+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 208 ==== osd_op_reply(495 [pgnls-filter start_epoch 129 out=81b] v129'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.179+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000218.00000000 2023-10-11T09:52:24.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.179+0000 7f239a687840 10 datascan.operator(): handling object 10000000218.0 2023-10-11T09:52:24.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.179+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:496 18.2d 18:b79a54f6:::10000000218.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35800 con 0x561e8b013800 2023-10-11T09:52:24.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.179+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 209 ==== osd_op_reply(496 10000000218.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv10 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.179+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000218 2023-10-11T09:52:24.187 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.179+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/35 2023-10-11T09:52:24.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.179+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=35 2023-10-11T09:52:24.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.179+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:497 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b182c00 con 0x561e8b013000 2023-10-11T09:52:24.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.179+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 101 ==== osd_op_reply(497 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.179+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.188 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.179+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:498 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183800 con 0x561e8b013800 2023-10-11T09:52:24.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.179+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 210 ==== osd_op_reply(498 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.179+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.179+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.190 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.179+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:24.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.179+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:499 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b2400 con 0x561e8b182400 2023-10-11T09:52:24.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.183+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 107 ==== osd_op_reply(499 10000000000.01800000 [omap-get-header out=274b] v0'0 uv35 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.183+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 35 2023-10-11T09:52:24.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.183+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:24.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.183+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:500 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b2800 con 0x561e8b182400 2023-10-11T09:52:24.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.183+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 108 ==== osd_op_reply(500 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv35 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.183+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=35 frag=1*, r=0 2023-10-11T09:52:24.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.183+0000 7f239a687840 20 datascan.read_dentry: 35_head not found in result 2023-10-11T09:52:24.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.183+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000218 at 0x0x10000000000/35 with size=2 bytes 2023-10-11T09:52:24.192 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.183+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:501 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b2c00 con 0x561e8b182400 2023-10-11T09:52:24.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.183+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 109 ==== osd_op_reply(501 10000000000.01800000 [omap-set-vals] v129'36 uv36 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.183+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/35 pointing to 0x0x10000000218 2023-10-11T09:52:24.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.183+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:502 18.1d 18:b8000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b3000 con 0x561e8b182800 2023-10-11T09:52:24.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.187+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.0 v2:172.21.15.78:6800/35735 25 ==== osd_op_reply(502 [pgnls-filter start_epoch 129 out=81b] v129'18 uv18 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b182800 2023-10-11T09:52:24.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.187+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000246.00000000 2023-10-11T09:52:24.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.187+0000 7f239a687840 10 datascan.operator(): handling object 10000000246.0 2023-10-11T09:52:24.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.187+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:503 18.1d 18:bab97ecf:::10000000246.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b2000 con 0x561e8b182800 2023-10-11T09:52:24.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.187+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.0 v2:172.21.15.78:6800/35735 26 ==== osd_op_reply(503 10000000246.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv18 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b182800 2023-10-11T09:52:24.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.187+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000246 2023-10-11T09:52:24.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.187+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/81 2023-10-11T09:52:24.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.187+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=81 2023-10-11T09:52:24.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.187+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:504 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b228000 con 0x561e8b013000 2023-10-11T09:52:24.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.187+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 102 ==== osd_op_reply(504 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.187+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.187+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:505 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b229c00 con 0x561e8b013800 2023-10-11T09:52:24.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.187+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 211 ==== osd_op_reply(505 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.187+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.187+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.187+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:24.198 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.187+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:506 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b228800 con 0x561e8b182400 2023-10-11T09:52:24.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.191+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 110 ==== osd_op_reply(506 10000000000.01800000 [omap-get-header out=274b] v0'0 uv36 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.191+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 36 2023-10-11T09:52:24.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.191+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:24.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.191+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:507 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2c00 con 0x561e8b182400 2023-10-11T09:52:24.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.191+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 111 ==== osd_op_reply(507 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv36 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.191+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=81 frag=1*, r=0 2023-10-11T09:52:24.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.191+0000 7f239a687840 20 datascan.read_dentry: 81_head not found in result 2023-10-11T09:52:24.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.191+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000246 at 0x0x10000000000/81 with size=2 bytes 2023-10-11T09:52:24.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.191+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:508 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2800 con 0x561e8b182400 2023-10-11T09:52:24.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.191+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 112 ==== osd_op_reply(508 10000000000.01800000 [omap-set-vals] v129'37 uv37 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.195+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/81 pointing to 0x0x10000000246 2023-10-11T09:52:24.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.195+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:509 18.3d 18:bc000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3400 con 0x561e8b013000 2023-10-11T09:52:24.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.195+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 103 ==== osd_op_reply(509 [pgnls-filter start_epoch 129 out=101b] v129'13 uv13 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.195+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000219.00000000 2023-10-11T09:52:24.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.195+0000 7f239a687840 10 datascan.operator(): handling object 10000000219.0 2023-10-11T09:52:24.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.195+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:510 18.3d 18:bc0c3701:::10000000219.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2000 con 0x561e8b013000 2023-10-11T09:52:24.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.195+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 104 ==== osd_op_reply(510 10000000219.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.195+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000219 2023-10-11T09:52:24.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.195+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/36 2023-10-11T09:52:24.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.195+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=36 2023-10-11T09:52:24.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.195+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:511 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3c00 con 0x561e8b013000 2023-10-11T09:52:24.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.195+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 105 ==== osd_op_reply(511 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.195+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.195+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:512 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35400 con 0x561e8b013800 2023-10-11T09:52:24.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.195+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 212 ==== osd_op_reply(512 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.205 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.195+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.195+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.195+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:24.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.195+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:513 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34400 con 0x561e8b013800 2023-10-11T09:52:24.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.195+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 213 ==== osd_op_reply(513 10000000000.01000000 [omap-get-header out=274b] v0'0 uv43 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.195+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 43 2023-10-11T09:52:24.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.195+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:24.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.195+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:514 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34c00 con 0x561e8b013800 2023-10-11T09:52:24.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.199+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 214 ==== osd_op_reply(514 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv43 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.199+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=36 frag=0*, r=0 2023-10-11T09:52:24.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.199+0000 7f239a687840 20 datascan.read_dentry: 36_head not found in result 2023-10-11T09:52:24.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.199+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000219 at 0x0x10000000000/36 with size=2 bytes 2023-10-11T09:52:24.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.199+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:515 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34000 con 0x561e8b013800 2023-10-11T09:52:24.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.199+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 215 ==== osd_op_reply(515 10000000000.01000000 [omap-set-vals] v129'44 uv44 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.199+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/36 pointing to 0x0x10000000219 2023-10-11T09:52:24.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.199+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:516 18.3d 18:bfcd3380::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34800 con 0x561e8b013000 2023-10-11T09:52:24.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.203+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 106 ==== osd_op_reply(516 [pgnls-filter start_epoch 129 out=81b] v129'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.203+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000205.00000000 2023-10-11T09:52:24.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.203+0000 7f239a687840 10 datascan.operator(): handling object 10000000205.0 2023-10-11T09:52:24.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.203+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:517 18.3d 18:bfcd3380:::10000000205.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221800 con 0x561e8b013000 2023-10-11T09:52:24.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.203+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 107 ==== osd_op_reply(517 10000000205.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.203+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000205 2023-10-11T09:52:24.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.203+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/16 2023-10-11T09:52:24.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.203+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=16 2023-10-11T09:52:24.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.203+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:518 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b220400 con 0x561e8b013000 2023-10-11T09:52:24.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.203+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 108 ==== osd_op_reply(518 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.203+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.203+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:519 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b229800 con 0x561e8b013800 2023-10-11T09:52:24.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.203+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 216 ==== osd_op_reply(519 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.203+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.203+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.203+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:24.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.203+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:520 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183400 con 0x561e8b182400 2023-10-11T09:52:24.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.203+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 113 ==== osd_op_reply(520 10000000000.01800000 [omap-get-header out=274b] v0'0 uv37 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.203+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 37 2023-10-11T09:52:24.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.203+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:24.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.203+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:521 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183000 con 0x561e8b182400 2023-10-11T09:52:24.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.207+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 114 ==== osd_op_reply(521 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv37 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.207+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=16 frag=1*, r=0 2023-10-11T09:52:24.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.207+0000 7f239a687840 20 datascan.read_dentry: 16_head not found in result 2023-10-11T09:52:24.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.207+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000205 at 0x0x10000000000/16 with size=2 bytes 2023-10-11T09:52:24.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.207+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:522 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b238000 con 0x561e8b182400 2023-10-11T09:52:24.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.207+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 115 ==== osd_op_reply(522 10000000000.01800000 [omap-set-vals] v129'38 uv38 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.207+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/16 pointing to 0x0x10000000205 2023-10-11T09:52:24.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.207+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:523 18.3 18:c0000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b238400 con 0x561e8ae35000 2023-10-11T09:52:24.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.211+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.1 v2:172.21.15.78:6809/35734 16 ==== osd_op_reply(523 [pgnls-filter start_epoch 129 out=49b] v127'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8ae35000 2023-10-11T09:52:24.218 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.211+0000 7f2393e39700 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:524 18.23 18:c4000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35c00 con 0x561e8b1e2400 2023-10-11T09:52:24.218 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.211+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.2 v2:172.21.15.78:6808/35732 12 ==== osd_op_reply(524 [pgnls-filter start_epoch 129 out=49b] v127'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b1e2400 2023-10-11T09:52:24.218 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.211+0000 7f239463a700 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:525 18.13 18:c8000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b3400 con 0x561e8b1e3800 2023-10-11T09:52:24.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.211+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.7 v2:172.21.15.133:6824/35901 11 ==== osd_op_reply(525 [pgnls-filter start_epoch 129 out=101b] v129'30 uv30 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b1e3800 2023-10-11T09:52:24.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.211+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 1000000023c.00000000 2023-10-11T09:52:24.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.211+0000 7f239a687840 10 datascan.operator(): handling object 1000000023c.0 2023-10-11T09:52:24.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.211+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:526 18.13 18:c8893608:::1000000023c.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b238800 con 0x561e8b1e3800 2023-10-11T09:52:24.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.211+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.7 v2:172.21.15.133:6824/35901 12 ==== osd_op_reply(526 1000000023c.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv26 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b1e3800 2023-10-11T09:52:24.220 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.211+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x1000000023c 2023-10-11T09:52:24.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.211+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/71 2023-10-11T09:52:24.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.211+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=71 2023-10-11T09:52:24.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.211+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:527 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b238c00 con 0x561e8b013000 2023-10-11T09:52:24.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.215+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 109 ==== osd_op_reply(527 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.215+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.215+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:528 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b239000 con 0x561e8b013800 2023-10-11T09:52:24.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.215+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 217 ==== osd_op_reply(528 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.215+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.215+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.215+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:24.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.215+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:529 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b239400 con 0x561e8b182400 2023-10-11T09:52:24.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.215+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 116 ==== osd_op_reply(529 10000000000.01800000 [omap-get-header out=274b] v0'0 uv38 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.215+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 38 2023-10-11T09:52:24.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.215+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:24.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.215+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:530 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b239800 con 0x561e8b182400 2023-10-11T09:52:24.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.215+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 117 ==== osd_op_reply(530 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv38 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.215+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=71 frag=1*, r=0 2023-10-11T09:52:24.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.215+0000 7f239a687840 20 datascan.read_dentry: 71_head not found in result 2023-10-11T09:52:24.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.215+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000023c at 0x0x10000000000/71 with size=2 bytes 2023-10-11T09:52:24.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.215+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:531 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b239c00 con 0x561e8b182400 2023-10-11T09:52:24.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.219+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 118 ==== osd_op_reply(531 10000000000.01800000 [omap-set-vals] v129'39 uv39 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.219+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/71 pointing to 0x0x1000000023c 2023-10-11T09:52:24.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.219+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:532 18.13 18:c933d8fa::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b238400 con 0x561e8b1e3800 2023-10-11T09:52:24.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.219+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.7 v2:172.21.15.133:6824/35901 13 ==== osd_op_reply(532 [pgnls-filter start_epoch 129 out=101b] v129'30 uv30 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b1e3800 2023-10-11T09:52:24.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.219+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000241.00000000 2023-10-11T09:52:24.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.219+0000 7f239a687840 10 datascan.operator(): handling object 10000000241.0 2023-10-11T09:52:24.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.219+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:533 18.13 18:c933d8fa:::10000000241.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221400 con 0x561e8b1e3800 2023-10-11T09:52:24.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.219+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.7 v2:172.21.15.133:6824/35901 14 ==== osd_op_reply(533 10000000241.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv27 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b1e3800 2023-10-11T09:52:24.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.219+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000241 2023-10-11T09:52:24.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.219+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/76 2023-10-11T09:52:24.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.219+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=76 2023-10-11T09:52:24.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.219+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:534 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b220c00 con 0x561e8b013000 2023-10-11T09:52:24.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.223+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 110 ==== osd_op_reply(534 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.223+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.223+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:535 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221800 con 0x561e8b013800 2023-10-11T09:52:24.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.223+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 218 ==== osd_op_reply(535 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.223+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.223+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.223+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:24.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.223+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:536 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b220400 con 0x561e8b182400 2023-10-11T09:52:24.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.223+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 119 ==== osd_op_reply(536 10000000000.01800000 [omap-get-header out=274b] v0'0 uv39 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.223+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 39 2023-10-11T09:52:24.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.223+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:24.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.223+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:537 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34800 con 0x561e8b182400 2023-10-11T09:52:24.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.223+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 120 ==== osd_op_reply(537 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv39 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.223+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=76 frag=1*, r=0 2023-10-11T09:52:24.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.223+0000 7f239a687840 20 datascan.read_dentry: 76_head not found in result 2023-10-11T09:52:24.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.223+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000241 at 0x0x10000000000/76 with size=2 bytes 2023-10-11T09:52:24.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.223+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:538 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3400 con 0x561e8b182400 2023-10-11T09:52:24.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.227+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 121 ==== osd_op_reply(538 10000000000.01800000 [omap-set-vals] v129'40 uv40 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.227+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/76 pointing to 0x0x10000000241 2023-10-11T09:52:24.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.227+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:539 18.13 18:c9855970::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2000 con 0x561e8b1e3800 2023-10-11T09:52:24.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.227+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.7 v2:172.21.15.133:6824/35901 15 ==== osd_op_reply(539 [pgnls-filter start_epoch 129 out=101b] v129'30 uv30 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b1e3800 2023-10-11T09:52:24.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.227+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000239.00000000 2023-10-11T09:52:24.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.227+0000 7f239a687840 10 datascan.operator(): handling object 10000000239.0 2023-10-11T09:52:24.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.227+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:540 18.13 18:c9855970:::10000000239.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3c00 con 0x561e8b1e3800 2023-10-11T09:52:24.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.227+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.7 v2:172.21.15.133:6824/35901 16 ==== osd_op_reply(540 10000000239.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv28 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b1e3800 2023-10-11T09:52:24.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.227+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000239 2023-10-11T09:52:24.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.227+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/68 2023-10-11T09:52:24.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.227+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=68 2023-10-11T09:52:24.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.227+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:541 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b228400 con 0x561e8b013000 2023-10-11T09:52:24.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.231+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 111 ==== osd_op_reply(541 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.231+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.231+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:542 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b228000 con 0x561e8b013800 2023-10-11T09:52:24.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.231+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 219 ==== osd_op_reply(542 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.231+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.231+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.231+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:24.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.231+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:543 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b3000 con 0x561e8b182400 2023-10-11T09:52:24.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.231+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 122 ==== osd_op_reply(543 10000000000.01800000 [omap-get-header out=274b] v0'0 uv40 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.231+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 40 2023-10-11T09:52:24.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.231+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:24.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.231+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:544 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b2000 con 0x561e8b182400 2023-10-11T09:52:24.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.231+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 123 ==== osd_op_reply(544 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv40 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.231+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=68 frag=1*, r=0 2023-10-11T09:52:24.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.231+0000 7f239a687840 20 datascan.read_dentry: 68_head not found in result 2023-10-11T09:52:24.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.231+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000239 at 0x0x10000000000/68 with size=2 bytes 2023-10-11T09:52:24.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.231+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:545 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b238000 con 0x561e8b182400 2023-10-11T09:52:24.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.235+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 124 ==== osd_op_reply(545 10000000000.01800000 [omap-set-vals] v129'41 uv41 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.235+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/68 pointing to 0x0x10000000239 2023-10-11T09:52:24.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.235+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:546 18.13 18:ca8fdbba::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b239400 con 0x561e8b1e3800 2023-10-11T09:52:24.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.235+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.7 v2:172.21.15.133:6824/35901 17 ==== osd_op_reply(546 [pgnls-filter start_epoch 129 out=101b] v129'30 uv30 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b1e3800 2023-10-11T09:52:24.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.235+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 100000001fb.00000000 2023-10-11T09:52:24.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.235+0000 7f239a687840 10 datascan.operator(): handling object 100000001fb.0 2023-10-11T09:52:24.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.235+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:547 18.13 18:ca8fdbba:::100000001fb.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b239800 con 0x561e8b1e3800 2023-10-11T09:52:24.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.239+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.7 v2:172.21.15.133:6824/35901 18 ==== osd_op_reply(547 100000001fb.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv29 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b1e3800 2023-10-11T09:52:24.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.239+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x100000001fb 2023-10-11T09:52:24.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.239+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/6 2023-10-11T09:52:24.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.239+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=6 2023-10-11T09:52:24.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.239+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:548 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b239c00 con 0x561e8b013000 2023-10-11T09:52:24.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.239+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 112 ==== osd_op_reply(548 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.239+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.239+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:549 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b238800 con 0x561e8b013800 2023-10-11T09:52:24.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.239+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 220 ==== osd_op_reply(549 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.239+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.239+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.239+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:24.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.239+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:550 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b239000 con 0x561e8b182400 2023-10-11T09:52:24.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.239+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 125 ==== osd_op_reply(550 10000000000.01800000 [omap-get-header out=274b] v0'0 uv41 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.239+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 41 2023-10-11T09:52:24.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.239+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:24.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.239+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:551 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b238400 con 0x561e8b182400 2023-10-11T09:52:24.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.239+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 126 ==== osd_op_reply(551 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv41 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.239+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=6 frag=1*, r=0 2023-10-11T09:52:24.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.239+0000 7f239a687840 20 datascan.read_dentry: 6_head not found in result 2023-10-11T09:52:24.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.239+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fb at 0x0x10000000000/6 with size=1 bytes 2023-10-11T09:52:24.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.239+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:552 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b2400 con 0x561e8b182400 2023-10-11T09:52:24.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.243+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 127 ==== osd_op_reply(552 10000000000.01800000 [omap-set-vals] v129'42 uv42 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.243+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/6 pointing to 0x0x100000001fb 2023-10-11T09:52:24.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.243+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:553 18.13 18:cbcb6224::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b2800 con 0x561e8b1e3800 2023-10-11T09:52:24.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.243+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.7 v2:172.21.15.133:6824/35901 19 ==== osd_op_reply(553 [pgnls-filter start_epoch 129 out=81b] v129'30 uv30 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b1e3800 2023-10-11T09:52:24.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.243+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000212.00000000 2023-10-11T09:52:24.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.243+0000 7f239a687840 10 datascan.operator(): handling object 10000000212.0 2023-10-11T09:52:24.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.243+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:554 18.13 18:cbcb6224:::10000000212.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b2c00 con 0x561e8b1e3800 2023-10-11T09:52:24.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.243+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.7 v2:172.21.15.133:6824/35901 20 ==== osd_op_reply(554 10000000212.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv30 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b1e3800 2023-10-11T09:52:24.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.243+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000212 2023-10-11T09:52:24.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.243+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/29 2023-10-11T09:52:24.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.243+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=29 2023-10-11T09:52:24.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.243+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:555 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b3400 con 0x561e8b013000 2023-10-11T09:52:24.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.247+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 113 ==== osd_op_reply(555 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.247+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.247+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:556 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b228800 con 0x561e8b013800 2023-10-11T09:52:24.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.247+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 221 ==== osd_op_reply(556 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.247+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.247+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.247+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:24.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.247+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:557 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b228c00 con 0x561e8b182400 2023-10-11T09:52:24.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.247+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 128 ==== osd_op_reply(557 10000000000.01800000 [omap-get-header out=274b] v0'0 uv42 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.247+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 42 2023-10-11T09:52:24.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.247+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:24.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.247+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:558 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b229000 con 0x561e8b182400 2023-10-11T09:52:24.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.247+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 129 ==== osd_op_reply(558 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv42 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.247+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=29 frag=1*, r=0 2023-10-11T09:52:24.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.247+0000 7f239a687840 20 datascan.read_dentry: 29_head not found in result 2023-10-11T09:52:24.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.247+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000212 at 0x0x10000000000/29 with size=2 bytes 2023-10-11T09:52:24.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.247+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:559 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b229400 con 0x561e8b182400 2023-10-11T09:52:24.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.251+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 130 ==== osd_op_reply(559 10000000000.01800000 [omap-set-vals] v129'43 uv43 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.251+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/29 pointing to 0x0x10000000212 2023-10-11T09:52:24.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.251+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:560 18.33 18:cc000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b229c00 con 0x561e8ae35000 2023-10-11T09:52:24.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.251+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.1 v2:172.21.15.78:6809/35734 17 ==== osd_op_reply(560 [pgnls-filter start_epoch 129 out=81b] v129'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8ae35000 2023-10-11T09:52:24.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.251+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000248.00000000 2023-10-11T09:52:24.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.251+0000 7f239a687840 10 datascan.operator(): handling object 10000000248.0 2023-10-11T09:52:24.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.251+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:561 18.33 18:cf9e7a4f:::10000000248.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b229800 con 0x561e8ae35000 2023-10-11T09:52:24.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.251+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.1 v2:172.21.15.78:6809/35734 18 ==== osd_op_reply(561 10000000248.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8ae35000 2023-10-11T09:52:24.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.251+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000248 2023-10-11T09:52:24.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.251+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/83 2023-10-11T09:52:24.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.251+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=83 2023-10-11T09:52:24.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.251+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:562 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35c00 con 0x561e8b013000 2023-10-11T09:52:24.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.255+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 114 ==== osd_op_reply(562 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.255+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.255+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:563 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34800 con 0x561e8b013800 2023-10-11T09:52:24.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.255+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 222 ==== osd_op_reply(563 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.255+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.255+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.255+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:24.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.255+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:564 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35800 con 0x561e8b013800 2023-10-11T09:52:24.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.255+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 223 ==== osd_op_reply(564 10000000000.01000000 [omap-get-header out=274b] v0'0 uv44 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.255+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 44 2023-10-11T09:52:24.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.255+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:24.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.255+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:565 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35400 con 0x561e8b013800 2023-10-11T09:52:24.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.255+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 224 ==== osd_op_reply(565 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv44 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.255+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=83 frag=0*, r=0 2023-10-11T09:52:24.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.255+0000 7f239a687840 20 datascan.read_dentry: 83_head not found in result 2023-10-11T09:52:24.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.255+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000248 at 0x0x10000000000/83 with size=2 bytes 2023-10-11T09:52:24.266 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.255+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:566 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34400 con 0x561e8b013800 2023-10-11T09:52:24.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.259+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 225 ==== osd_op_reply(566 10000000000.01000000 [omap-set-vals] v129'45 uv45 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.267 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.259+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/83 pointing to 0x0x10000000248 2023-10-11T09:52:24.268 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.259+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:567 18.b 18:d0000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34c00 con 0x561e8b183c00 2023-10-11T09:52:24.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.259+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 22 ==== osd_op_reply(567 [pgnls-filter start_epoch 129 out=101b] v129'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:24.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.259+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000225.00000000 2023-10-11T09:52:24.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.259+0000 7f239a687840 10 datascan.operator(): handling object 10000000225.0 2023-10-11T09:52:24.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.259+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:568 18.b 18:d0d26e50:::10000000225.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34000 con 0x561e8b183c00 2023-10-11T09:52:24.269 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.263+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 23 ==== osd_op_reply(568 10000000225.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:24.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.263+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000225 2023-10-11T09:52:24.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.263+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/48 2023-10-11T09:52:24.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.263+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=48 2023-10-11T09:52:24.270 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.263+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:569 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183400 con 0x561e8b013000 2023-10-11T09:52:24.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.263+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 115 ==== osd_op_reply(569 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.263+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.263+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:570 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183000 con 0x561e8b013800 2023-10-11T09:52:24.271 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.263+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 226 ==== osd_op_reply(570 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.263+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.263+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.263+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:24.272 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.263+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:571 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183800 con 0x561e8b182400 2023-10-11T09:52:24.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.263+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 131 ==== osd_op_reply(571 10000000000.01800000 [omap-get-header out=274b] v0'0 uv43 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.263+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 43 2023-10-11T09:52:24.273 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.263+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:24.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.263+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:572 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b013400 con 0x561e8b182400 2023-10-11T09:52:24.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.263+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 132 ==== osd_op_reply(572 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv43 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.263+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=48 frag=1*, r=0 2023-10-11T09:52:24.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.263+0000 7f239a687840 20 datascan.read_dentry: 48_head not found in result 2023-10-11T09:52:24.274 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.263+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000225 at 0x0x10000000000/48 with size=2 bytes 2023-10-11T09:52:24.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.267+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:573 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2c00 con 0x561e8b182400 2023-10-11T09:52:24.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.267+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 133 ==== osd_op_reply(573 10000000000.01800000 [omap-set-vals] v129'44 uv44 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.267+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/48 pointing to 0x0x10000000225 2023-10-11T09:52:24.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.267+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:574 18.b 18:d2dbd4ac::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2800 con 0x561e8b183c00 2023-10-11T09:52:24.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.271+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 24 ==== osd_op_reply(574 [pgnls-filter start_epoch 129 out=101b] v129'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:24.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.271+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000217.00000000 2023-10-11T09:52:24.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.271+0000 7f239a687840 10 datascan.operator(): handling object 10000000217.0 2023-10-11T09:52:24.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.271+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:575 18.b 18:d2dbd4ac:::10000000217.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3400 con 0x561e8b183c00 2023-10-11T09:52:24.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.271+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 25 ==== osd_op_reply(575 10000000217.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:24.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.271+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000217 2023-10-11T09:52:24.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.271+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/34 2023-10-11T09:52:24.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.271+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=34 2023-10-11T09:52:24.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.271+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:576 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3000 con 0x561e8b013000 2023-10-11T09:52:24.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.271+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 116 ==== osd_op_reply(576 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.271+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.271+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:577 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b220400 con 0x561e8b013800 2023-10-11T09:52:24.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.271+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 227 ==== osd_op_reply(577 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.271+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.271+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.271+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:24.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.271+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:578 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b220000 con 0x561e8b013800 2023-10-11T09:52:24.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.271+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 228 ==== osd_op_reply(578 10000000000.01000000 [omap-get-header out=274b] v0'0 uv45 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.271+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 45 2023-10-11T09:52:24.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.271+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:24.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.271+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:579 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221c00 con 0x561e8b013800 2023-10-11T09:52:24.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.275+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 229 ==== osd_op_reply(579 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv45 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.275+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=34 frag=0*, r=0 2023-10-11T09:52:24.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.275+0000 7f239a687840 20 datascan.read_dentry: 34_head not found in result 2023-10-11T09:52:24.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.275+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000217 at 0x0x10000000000/34 with size=2 bytes 2023-10-11T09:52:24.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.275+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:580 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221000 con 0x561e8b013800 2023-10-11T09:52:24.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.275+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 230 ==== osd_op_reply(580 10000000000.01000000 [omap-set-vals] v129'46 uv46 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.275+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/34 pointing to 0x0x10000000217 2023-10-11T09:52:24.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.275+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:581 18.b 18:d3eda40d::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b220800 con 0x561e8b183c00 2023-10-11T09:52:24.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.279+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 26 ==== osd_op_reply(581 [pgnls-filter start_epoch 129 out=81b] v129'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:24.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.279+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 1000000020f.00000000 2023-10-11T09:52:24.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.279+0000 7f239a687840 10 datascan.operator(): handling object 1000000020f.0 2023-10-11T09:52:24.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.279+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:582 18.b 18:d3eda40d:::1000000020f.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221400 con 0x561e8b183c00 2023-10-11T09:52:24.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.279+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 27 ==== osd_op_reply(582 1000000020f.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:24.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.279+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x1000000020f 2023-10-11T09:52:24.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.279+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/26 2023-10-11T09:52:24.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.279+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=26 2023-10-11T09:52:24.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.279+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:583 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221800 con 0x561e8b013000 2023-10-11T09:52:24.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.279+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 117 ==== osd_op_reply(583 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.279+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.279+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:584 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b220400 con 0x561e8b013800 2023-10-11T09:52:24.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.279+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 231 ==== osd_op_reply(584 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.279+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.279+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.279+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:24.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.279+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:585 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b220000 con 0x561e8b182400 2023-10-11T09:52:24.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.279+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 134 ==== osd_op_reply(585 10000000000.01800000 [omap-get-header out=274b] v0'0 uv44 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.283+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 44 2023-10-11T09:52:24.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.283+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:24.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.283+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:586 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221c00 con 0x561e8b182400 2023-10-11T09:52:24.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.283+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 135 ==== osd_op_reply(586 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv44 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.283+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=26 frag=1*, r=0 2023-10-11T09:52:24.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.283+0000 7f239a687840 20 datascan.read_dentry: 26_head not found in result 2023-10-11T09:52:24.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.283+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000020f at 0x0x10000000000/26 with size=2 bytes 2023-10-11T09:52:24.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.283+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:587 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221000 con 0x561e8b182400 2023-10-11T09:52:24.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.283+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 136 ==== osd_op_reply(587 10000000000.01800000 [omap-set-vals] v129'45 uv45 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.283+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/26 pointing to 0x0x1000000020f 2023-10-11T09:52:24.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.283+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:588 18.2b 18:d4000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2000 con 0x561e8b1e3800 2023-10-11T09:52:24.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.287+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.7 v2:172.21.15.133:6824/35901 21 ==== osd_op_reply(588 [pgnls-filter start_epoch 129 out=101b] v129'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b1e3800 2023-10-11T09:52:24.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.287+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000203.00000000 2023-10-11T09:52:24.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.287+0000 7f239a687840 10 datascan.operator(): handling object 10000000203.0 2023-10-11T09:52:24.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.287+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:589 18.2b 18:d4a58307:::10000000203.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3c00 con 0x561e8b1e3800 2023-10-11T09:52:24.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.287+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.7 v2:172.21.15.133:6824/35901 22 ==== osd_op_reply(589 10000000203.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b1e3800 2023-10-11T09:52:24.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.287+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000203 2023-10-11T09:52:24.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.287+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/14 2023-10-11T09:52:24.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.287+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=14 2023-10-11T09:52:24.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.287+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:590 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2800 con 0x561e8b013000 2023-10-11T09:52:24.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.287+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 118 ==== osd_op_reply(590 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.287+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.287+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:591 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3400 con 0x561e8b013800 2023-10-11T09:52:24.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.287+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 232 ==== osd_op_reply(591 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.287+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.287+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.287+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:24.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.287+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:592 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183000 con 0x561e8b182400 2023-10-11T09:52:24.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.291+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 137 ==== osd_op_reply(592 10000000000.01800000 [omap-get-header out=274b] v0'0 uv45 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.298 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.291+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 45 2023-10-11T09:52:24.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.291+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:24.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.291+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:593 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34800 con 0x561e8b182400 2023-10-11T09:52:24.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.291+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 138 ==== osd_op_reply(593 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv45 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.291+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=14 frag=1*, r=0 2023-10-11T09:52:24.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.291+0000 7f239a687840 20 datascan.read_dentry: 14_head not found in result 2023-10-11T09:52:24.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.291+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000203 at 0x0x10000000000/14 with size=2 bytes 2023-10-11T09:52:24.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.291+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:594 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35800 con 0x561e8b182400 2023-10-11T09:52:24.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.295+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 139 ==== osd_op_reply(594 10000000000.01800000 [omap-set-vals] v129'46 uv46 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.295+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/14 pointing to 0x0x10000000203 2023-10-11T09:52:24.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.295+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:595 18.2b 18:d5ee56ee::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35400 con 0x561e8b1e3800 2023-10-11T09:52:24.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.295+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.7 v2:172.21.15.133:6824/35901 23 ==== osd_op_reply(595 [pgnls-filter start_epoch 129 out=81b] v129'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b1e3800 2023-10-11T09:52:24.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.295+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 1000000022d.00000000 2023-10-11T09:52:24.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.295+0000 7f239a687840 10 datascan.operator(): handling object 1000000022d.0 2023-10-11T09:52:24.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.295+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:596 18.2b 18:d5ee56ee:::1000000022d.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34400 con 0x561e8b1e3800 2023-10-11T09:52:24.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.295+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.7 v2:172.21.15.133:6824/35901 24 ==== osd_op_reply(596 1000000022d.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b1e3800 2023-10-11T09:52:24.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.295+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x1000000022d 2023-10-11T09:52:24.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.295+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/56 2023-10-11T09:52:24.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.295+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=56 2023-10-11T09:52:24.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.295+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:597 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34c00 con 0x561e8b013000 2023-10-11T09:52:24.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.295+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 119 ==== osd_op_reply(597 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.295+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.305 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.295+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:598 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34000 con 0x561e8b013800 2023-10-11T09:52:24.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.295+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 233 ==== osd_op_reply(598 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.295+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.295+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.295+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:24.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.295+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:599 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b228000 con 0x561e8b182400 2023-10-11T09:52:24.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.299+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 140 ==== osd_op_reply(599 10000000000.01800000 [omap-get-header out=274b] v0'0 uv46 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.299+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 46 2023-10-11T09:52:24.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.299+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:24.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.299+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:600 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b228800 con 0x561e8b182400 2023-10-11T09:52:24.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.299+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 141 ==== osd_op_reply(600 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv46 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.299+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=56 frag=1*, r=0 2023-10-11T09:52:24.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.299+0000 7f239a687840 20 datascan.read_dentry: 56_head not found in result 2023-10-11T09:52:24.308 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.299+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000022d at 0x0x10000000000/56 with size=2 bytes 2023-10-11T09:52:24.309 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.299+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:601 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b2800 con 0x561e8b182400 2023-10-11T09:52:24.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.303+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 142 ==== osd_op_reply(601 10000000000.01800000 [omap-set-vals] v129'47 uv47 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.303+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/56 pointing to 0x0x1000000022d 2023-10-11T09:52:24.310 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.303+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:602 18.1b 18:d8000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b2c00 con 0x561e8ae35000 2023-10-11T09:52:24.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.303+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.1 v2:172.21.15.78:6809/35734 19 ==== osd_op_reply(602 [pgnls-filter start_epoch 129 out=49b] v127'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8ae35000 2023-10-11T09:52:24.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.303+0000 7f239463a700 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:603 18.3b 18:dc000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b3800 con 0x561e8b182400 2023-10-11T09:52:24.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.303+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 143 ==== osd_op_reply(603 [pgnls-filter start_epoch 129 out=101b] v129'13 uv13 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.303+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000249.00000000 2023-10-11T09:52:24.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.303+0000 7f239a687840 10 datascan.operator(): handling object 10000000249.0 2023-10-11T09:52:24.312 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.303+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:604 18.3b 18:dd50df43:::10000000249.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b239400 con 0x561e8b182400 2023-10-11T09:52:24.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.307+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 144 ==== osd_op_reply(604 10000000249.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.313 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.307+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000249 2023-10-11T09:52:24.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.307+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/84 2023-10-11T09:52:24.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.307+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=84 2023-10-11T09:52:24.314 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.307+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:605 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b239800 con 0x561e8b013000 2023-10-11T09:52:24.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.307+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 120 ==== osd_op_reply(605 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.307+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.307+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:606 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b238800 con 0x561e8b013800 2023-10-11T09:52:24.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.307+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 234 ==== osd_op_reply(606 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.315 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.307+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.307+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.307+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:24.316 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.307+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:607 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34800 con 0x561e8b013800 2023-10-11T09:52:24.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.307+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 235 ==== osd_op_reply(607 10000000000.01000000 [omap-get-header out=274b] v0'0 uv46 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.307+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 46 2023-10-11T09:52:24.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.307+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:24.317 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.307+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:608 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183800 con 0x561e8b013800 2023-10-11T09:52:24.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.307+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 236 ==== osd_op_reply(608 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv46 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.307+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=84 frag=0*, r=0 2023-10-11T09:52:24.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.307+0000 7f239a687840 20 datascan.read_dentry: 84_head not found in result 2023-10-11T09:52:24.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.307+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000249 at 0x0x10000000000/84 with size=2 bytes 2023-10-11T09:52:24.318 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.307+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:609 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183000 con 0x561e8b013800 2023-10-11T09:52:24.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.311+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 237 ==== osd_op_reply(609 10000000000.01000000 [omap-set-vals] v129'47 uv47 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.319 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.311+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/84 pointing to 0x0x10000000249 2023-10-11T09:52:24.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.311+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:610 18.3b 18:deccbd37::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b220000 con 0x561e8b182400 2023-10-11T09:52:24.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.311+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 145 ==== osd_op_reply(610 [pgnls-filter start_epoch 129 out=101b] v129'13 uv13 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.320 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.311+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 100000001fe.00000000 2023-10-11T09:52:24.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.311+0000 7f239a687840 10 datascan.operator(): handling object 100000001fe.0 2023-10-11T09:52:24.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.311+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:611 18.3b 18:deccbd37:::100000001fe.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221c00 con 0x561e8b182400 2023-10-11T09:52:24.321 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.315+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 146 ==== osd_op_reply(611 100000001fe.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.315+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x100000001fe 2023-10-11T09:52:24.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.315+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/9 2023-10-11T09:52:24.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.315+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=9 2023-10-11T09:52:24.322 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.315+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:612 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221000 con 0x561e8b013000 2023-10-11T09:52:24.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.315+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 121 ==== osd_op_reply(612 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.315+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.315+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:613 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2c00 con 0x561e8b013800 2023-10-11T09:52:24.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.315+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 238 ==== osd_op_reply(613 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.315+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.315+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.315+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:24.324 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.315+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:614 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b013400 con 0x561e8b013800 2023-10-11T09:52:24.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.315+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 239 ==== osd_op_reply(614 10000000000.01000000 [omap-get-header out=274b] v0'0 uv47 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.315+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 47 2023-10-11T09:52:24.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.315+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:24.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.315+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:615 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b228c00 con 0x561e8b013800 2023-10-11T09:52:24.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.319+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 240 ==== osd_op_reply(615 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv47 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.319+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=9 frag=0*, r=0 2023-10-11T09:52:24.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.319+0000 7f239a687840 20 datascan.read_dentry: 9_head not found in result 2023-10-11T09:52:24.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.319+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fe at 0x0x10000000000/9 with size=1 bytes 2023-10-11T09:52:24.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.319+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:616 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b229000 con 0x561e8b013800 2023-10-11T09:52:24.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.319+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 241 ==== osd_op_reply(616 10000000000.01000000 [omap-set-vals] v129'48 uv48 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.319+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/9 pointing to 0x0x100000001fe 2023-10-11T09:52:24.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.319+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:617 18.3b 18:ded14b84::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b229400 con 0x561e8b182400 2023-10-11T09:52:24.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.319+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 147 ==== osd_op_reply(617 [pgnls-filter start_epoch 129 out=81b] v129'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.319+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 1000000024b.00000000 2023-10-11T09:52:24.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.319+0000 7f239a687840 10 datascan.operator(): handling object 1000000024b.0 2023-10-11T09:52:24.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.323+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:618 18.3b 18:ded14b84:::1000000024b.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b3000 con 0x561e8b182400 2023-10-11T09:52:24.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.323+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 148 ==== osd_op_reply(618 1000000024b.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.323+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x1000000024b 2023-10-11T09:52:24.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.323+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/86 2023-10-11T09:52:24.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.323+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=86 2023-10-11T09:52:24.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.323+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:619 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b2000 con 0x561e8b013000 2023-10-11T09:52:24.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.323+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 122 ==== osd_op_reply(619 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.323+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.323+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:620 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b2400 con 0x561e8b013800 2023-10-11T09:52:24.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.323+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 242 ==== osd_op_reply(620 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.323+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.323+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.323+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:24.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.323+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:621 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b238000 con 0x561e8b013800 2023-10-11T09:52:24.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.323+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 243 ==== osd_op_reply(621 10000000000.01000000 [omap-get-header out=274b] v0'0 uv48 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.323+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 48 2023-10-11T09:52:24.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.323+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:24.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.323+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:622 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b239000 con 0x561e8b013800 2023-10-11T09:52:24.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.327+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 244 ==== osd_op_reply(622 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv48 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.327+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=86 frag=0*, r=0 2023-10-11T09:52:24.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.327+0000 7f239a687840 20 datascan.read_dentry: 86_head not found in result 2023-10-11T09:52:24.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.327+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000024b at 0x0x10000000000/86 with size=2 bytes 2023-10-11T09:52:24.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.327+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:623 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b238400 con 0x561e8b013800 2023-10-11T09:52:24.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.327+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 245 ==== osd_op_reply(623 10000000000.01000000 [omap-set-vals] v129'49 uv49 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.327+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/86 pointing to 0x0x1000000024b 2023-10-11T09:52:24.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.327+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:624 18.7 18:e0000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b250000 con 0x561e8b1e2400 2023-10-11T09:52:24.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.331+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.2 v2:172.21.15.78:6808/35732 13 ==== osd_op_reply(624 [pgnls-filter start_epoch 129 out=101b] v129'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b1e2400 2023-10-11T09:52:24.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.331+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000207.00000000 2023-10-11T09:52:24.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.331+0000 7f239a687840 10 datascan.operator(): handling object 10000000207.0 2023-10-11T09:52:24.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.331+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:625 18.7 18:e15a914f:::10000000207.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b250400 con 0x561e8b1e2400 2023-10-11T09:52:24.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.331+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.2 v2:172.21.15.78:6808/35732 14 ==== osd_op_reply(625 10000000207.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b1e2400 2023-10-11T09:52:24.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.331+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000207 2023-10-11T09:52:24.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.331+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/18 2023-10-11T09:52:24.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.331+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=18 2023-10-11T09:52:24.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.331+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:626 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b250800 con 0x561e8b013000 2023-10-11T09:52:24.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.331+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 123 ==== osd_op_reply(626 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.331+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.331+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:627 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b250c00 con 0x561e8b013800 2023-10-11T09:52:24.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.331+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 246 ==== osd_op_reply(627 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.331+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.331+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.331+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:24.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.331+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:628 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b251000 con 0x561e8b182400 2023-10-11T09:52:24.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.335+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 149 ==== osd_op_reply(628 10000000000.01800000 [omap-get-header out=274b] v0'0 uv47 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.335+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 47 2023-10-11T09:52:24.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.335+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:24.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.335+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:629 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b251400 con 0x561e8b182400 2023-10-11T09:52:24.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.335+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 150 ==== osd_op_reply(629 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv47 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.335+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=18 frag=1*, r=0 2023-10-11T09:52:24.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.335+0000 7f239a687840 20 datascan.read_dentry: 18_head not found in result 2023-10-11T09:52:24.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.335+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000207 at 0x0x10000000000/18 with size=2 bytes 2023-10-11T09:52:24.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.335+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:630 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b251800 con 0x561e8b182400 2023-10-11T09:52:24.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.339+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 151 ==== osd_op_reply(630 10000000000.01800000 [omap-set-vals] v129'48 uv48 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.339+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/18 pointing to 0x0x10000000207 2023-10-11T09:52:24.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.339+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:631 18.7 18:e2e07930::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b251c00 con 0x561e8b1e2400 2023-10-11T09:52:24.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.339+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.2 v2:172.21.15.78:6808/35732 15 ==== osd_op_reply(631 [pgnls-filter start_epoch 129 out=101b] v129'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b1e2400 2023-10-11T09:52:24.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.339+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2023-10-11T09:52:24.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.339+0000 7f239a687840 10 datascan.operator(): handling object 10000000001.0 2023-10-11T09:52:24.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.339+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:632 18.7 18:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b250800 con 0x561e8b1e2400 2023-10-11T09:52:24.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.339+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.2 v2:172.21.15.78:6808/35732 16 ==== osd_op_reply(632 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv14 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b1e2400 2023-10-11T09:52:24.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.339+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2023-10-11T09:52:24.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.339+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/0 2023-10-11T09:52:24.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.339+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=0 2023-10-11T09:52:24.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.339+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:633 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b3400 con 0x561e8b013000 2023-10-11T09:52:24.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.339+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 124 ==== osd_op_reply(633 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.339+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.339+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:634 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b2c00 con 0x561e8b013800 2023-10-11T09:52:24.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.339+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 247 ==== osd_op_reply(634 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.339+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.339+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.339+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:24.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.339+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:635 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b2000 con 0x561e8b182400 2023-10-11T09:52:24.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.343+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 152 ==== osd_op_reply(635 10000000000.01800000 [omap-get-header out=274b] v0'0 uv48 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.343+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 48 2023-10-11T09:52:24.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.343+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:24.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.343+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:636 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b220c00 con 0x561e8b182400 2023-10-11T09:52:24.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.343+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 153 ==== osd_op_reply(636 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv48 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.343+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=0 frag=1*, r=0 2023-10-11T09:52:24.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.343+0000 7f239a687840 20 datascan.read_dentry: 0_head not found in result 2023-10-11T09:52:24.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.343+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/0 with size=1 bytes 2023-10-11T09:52:24.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.343+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:637 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221800 con 0x561e8b182400 2023-10-11T09:52:24.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.343+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 154 ==== osd_op_reply(637 10000000000.01800000 [omap-set-vals] v129'49 uv49 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.343+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/0 pointing to 0x0x10000000001 2023-10-11T09:52:24.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.343+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:638 18.7 18:e37752f2::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221000 con 0x561e8b1e2400 2023-10-11T09:52:24.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.347+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.2 v2:172.21.15.78:6808/35732 17 ==== osd_op_reply(638 [pgnls-filter start_epoch 129 out=81b] v129'15 uv15 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b1e2400 2023-10-11T09:52:24.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.347+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 1000000021f.00000000 2023-10-11T09:52:24.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.347+0000 7f239a687840 10 datascan.operator(): handling object 1000000021f.0 2023-10-11T09:52:24.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.347+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:639 18.7 18:e37752f2:::1000000021f.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b238c00 con 0x561e8b1e2400 2023-10-11T09:52:24.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.347+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.2 v2:172.21.15.78:6808/35732 18 ==== osd_op_reply(639 1000000021f.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b1e2400 2023-10-11T09:52:24.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.347+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x1000000021f 2023-10-11T09:52:24.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.347+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/42 2023-10-11T09:52:24.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.347+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=42 2023-10-11T09:52:24.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.347+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:640 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b239c00 con 0x561e8b013000 2023-10-11T09:52:24.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.347+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 125 ==== osd_op_reply(640 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.347+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.347+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:641 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b239800 con 0x561e8b013800 2023-10-11T09:52:24.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.347+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 248 ==== osd_op_reply(641 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.347+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.347+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.347+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:24.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.347+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:642 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35c00 con 0x561e8b013800 2023-10-11T09:52:24.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.351+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 249 ==== osd_op_reply(642 10000000000.01000000 [omap-get-header out=274b] v0'0 uv49 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.351+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 49 2023-10-11T09:52:24.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.351+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:24.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.351+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:643 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34c00 con 0x561e8b013800 2023-10-11T09:52:24.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.351+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 250 ==== osd_op_reply(643 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv49 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.351+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=42 frag=0*, r=0 2023-10-11T09:52:24.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.351+0000 7f239a687840 20 datascan.read_dentry: 42_head not found in result 2023-10-11T09:52:24.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.351+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000021f at 0x0x10000000000/42 with size=2 bytes 2023-10-11T09:52:24.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.351+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:644 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3000 con 0x561e8b013800 2023-10-11T09:52:24.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.355+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 251 ==== osd_op_reply(644 10000000000.01000000 [omap-set-vals] v129'50 uv50 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.355+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/42 pointing to 0x0x1000000021f 2023-10-11T09:52:24.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.355+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:645 18.27 18:e4000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2800 con 0x561e8b183c00 2023-10-11T09:52:24.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.355+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 28 ==== osd_op_reply(645 [pgnls-filter start_epoch 129 out=101b] v129'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:24.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.355+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000210.00000000 2023-10-11T09:52:24.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.355+0000 7f239a687840 10 datascan.operator(): handling object 10000000210.0 2023-10-11T09:52:24.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.355+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:646 18.27 18:e40c9c52:::10000000210.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183400 con 0x561e8b183c00 2023-10-11T09:52:24.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.355+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 29 ==== osd_op_reply(646 10000000210.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:24.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.355+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000210 2023-10-11T09:52:24.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.355+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/27 2023-10-11T09:52:24.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.355+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=27 2023-10-11T09:52:24.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.355+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:647 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b228400 con 0x561e8b013000 2023-10-11T09:52:24.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.359+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 126 ==== osd_op_reply(647 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.359+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.359+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:648 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b229c00 con 0x561e8b013800 2023-10-11T09:52:24.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.359+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 252 ==== osd_op_reply(648 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.359+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.359+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.359+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:24.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.359+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:649 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b229800 con 0x561e8b013800 2023-10-11T09:52:24.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.359+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 253 ==== osd_op_reply(649 10000000000.01000000 [omap-get-header out=274b] v0'0 uv50 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.359+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 50 2023-10-11T09:52:24.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.359+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:24.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.359+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:650 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b2400 con 0x561e8b013800 2023-10-11T09:52:24.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.359+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 254 ==== osd_op_reply(650 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv50 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.359+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=27 frag=0*, r=0 2023-10-11T09:52:24.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.359+0000 7f239a687840 20 datascan.read_dentry: 27_head not found in result 2023-10-11T09:52:24.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.359+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000210 at 0x0x10000000000/27 with size=2 bytes 2023-10-11T09:52:24.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.359+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:651 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b2c00 con 0x561e8b013800 2023-10-11T09:52:24.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.363+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 255 ==== osd_op_reply(651 10000000000.01000000 [omap-set-vals] v129'51 uv51 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.363+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/27 pointing to 0x0x10000000210 2023-10-11T09:52:24.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.363+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:652 18.27 18:e5671758::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b2800 con 0x561e8b183c00 2023-10-11T09:52:24.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.363+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 30 ==== osd_op_reply(652 [pgnls-filter start_epoch 129 out=101b] v129'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:24.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.363+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 1000000024f.00000000 2023-10-11T09:52:24.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.363+0000 7f239a687840 10 datascan.operator(): handling object 1000000024f.0 2023-10-11T09:52:24.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.363+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:653 18.27 18:e5671758:::1000000024f.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b3800 con 0x561e8b183c00 2023-10-11T09:52:24.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.363+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 31 ==== osd_op_reply(653 1000000024f.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv14 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:24.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.367+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x1000000024f 2023-10-11T09:52:24.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.367+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/90 2023-10-11T09:52:24.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.367+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=90 2023-10-11T09:52:24.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.367+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:654 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b228c00 con 0x561e8b013000 2023-10-11T09:52:24.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.367+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 127 ==== osd_op_reply(654 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.367+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.367+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:655 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b229000 con 0x561e8b013800 2023-10-11T09:52:24.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.367+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 256 ==== osd_op_reply(655 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.367+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.367+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.367+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:24.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.367+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:656 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b228000 con 0x561e8b013800 2023-10-11T09:52:24.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.367+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 257 ==== osd_op_reply(656 10000000000.01000000 [omap-get-header out=274b] v0'0 uv51 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.367+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 51 2023-10-11T09:52:24.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.367+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:24.377 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.367+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:657 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b228800 con 0x561e8b013800 2023-10-11T09:52:24.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.367+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 258 ==== osd_op_reply(657 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv51 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.367+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=90 frag=0*, r=0 2023-10-11T09:52:24.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.367+0000 7f239a687840 20 datascan.read_dentry: 90_head not found in result 2023-10-11T09:52:24.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.367+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000024f at 0x0x10000000000/90 with size=2 bytes 2023-10-11T09:52:24.378 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.367+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:658 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b229400 con 0x561e8b013800 2023-10-11T09:52:24.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.371+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 259 ==== osd_op_reply(658 10000000000.01000000 [omap-set-vals] v129'52 uv52 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.371+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/90 pointing to 0x0x1000000024f 2023-10-11T09:52:24.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.371+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:659 18.27 18:e5993619::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b220800 con 0x561e8b183c00 2023-10-11T09:52:24.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.371+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 32 ==== osd_op_reply(659 [pgnls-filter start_epoch 129 out=101b] v129'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:24.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.371+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000208.00000000 2023-10-11T09:52:24.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.371+0000 7f239a687840 10 datascan.operator(): handling object 10000000208.0 2023-10-11T09:52:24.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.371+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:660 18.27 18:e5993619:::10000000208.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221400 con 0x561e8b183c00 2023-10-11T09:52:24.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.371+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 33 ==== osd_op_reply(660 10000000208.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:24.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.371+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000208 2023-10-11T09:52:24.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.371+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/19 2023-10-11T09:52:24.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.371+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=19 2023-10-11T09:52:24.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.371+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:661 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b220400 con 0x561e8b013000 2023-10-11T09:52:24.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.375+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 128 ==== osd_op_reply(661 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.375+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.375+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:662 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b220000 con 0x561e8b013800 2023-10-11T09:52:24.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.375+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 260 ==== osd_op_reply(662 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.375+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.375+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.375+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:24.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.375+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:663 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221c00 con 0x561e8b013800 2023-10-11T09:52:24.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.375+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 261 ==== osd_op_reply(663 10000000000.01000000 [omap-get-header out=274b] v0'0 uv52 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.375+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 52 2023-10-11T09:52:24.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.375+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:24.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.375+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:664 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b238800 con 0x561e8b013800 2023-10-11T09:52:24.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.375+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 262 ==== osd_op_reply(664 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv52 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.375+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=19 frag=0*, r=0 2023-10-11T09:52:24.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.375+0000 7f239a687840 20 datascan.read_dentry: 19_head not found in result 2023-10-11T09:52:24.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.375+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000208 at 0x0x10000000000/19 with size=2 bytes 2023-10-11T09:52:24.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.375+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:665 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b238000 con 0x561e8b013800 2023-10-11T09:52:24.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.379+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 263 ==== osd_op_reply(665 10000000000.01000000 [omap-set-vals] v129'53 uv53 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.379+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/19 pointing to 0x0x10000000208 2023-10-11T09:52:24.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.379+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:666 18.27 18:e5a5749e::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b239000 con 0x561e8b183c00 2023-10-11T09:52:24.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.379+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 34 ==== osd_op_reply(666 [pgnls-filter start_epoch 129 out=101b] v129'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:24.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.379+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 1000000021a.00000000 2023-10-11T09:52:24.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.379+0000 7f239a687840 10 datascan.operator(): handling object 1000000021a.0 2023-10-11T09:52:24.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.379+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:667 18.27 18:e5a5749e:::1000000021a.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b238400 con 0x561e8b183c00 2023-10-11T09:52:24.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.383+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 35 ==== osd_op_reply(667 1000000021a.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:24.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.383+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x1000000021a 2023-10-11T09:52:24.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.383+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/37 2023-10-11T09:52:24.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.383+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=37 2023-10-11T09:52:24.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.383+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:668 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b239800 con 0x561e8b013000 2023-10-11T09:52:24.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.383+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 129 ==== osd_op_reply(668 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.383+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.383+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:669 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b239400 con 0x561e8b013800 2023-10-11T09:52:24.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.383+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 264 ==== osd_op_reply(669 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.383+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.383+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.383+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:24.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.383+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:670 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35400 con 0x561e8b182400 2023-10-11T09:52:24.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.383+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 155 ==== osd_op_reply(670 10000000000.01800000 [omap-get-header out=274b] v0'0 uv49 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.383+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 49 2023-10-11T09:52:24.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.383+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:24.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.383+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:671 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34400 con 0x561e8b182400 2023-10-11T09:52:24.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.383+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 156 ==== osd_op_reply(671 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv49 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.383+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=37 frag=1*, r=0 2023-10-11T09:52:24.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.383+0000 7f239a687840 20 datascan.read_dentry: 37_head not found in result 2023-10-11T09:52:24.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.383+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000021a at 0x0x10000000000/37 with size=2 bytes 2023-10-11T09:52:24.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.387+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:672 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34000 con 0x561e8b182400 2023-10-11T09:52:24.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.387+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 157 ==== osd_op_reply(672 10000000000.01800000 [omap-set-vals] v129'50 uv50 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.387+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/37 pointing to 0x0x1000000021a 2023-10-11T09:52:24.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.387+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:673 18.27 18:e692fc64::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34800 con 0x561e8b183c00 2023-10-11T09:52:24.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.391+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 36 ==== osd_op_reply(673 [pgnls-filter start_epoch 129 out=81b] v129'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:24.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.391+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 1000000021b.00000000 2023-10-11T09:52:24.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.391+0000 7f239a687840 10 datascan.operator(): handling object 1000000021b.0 2023-10-11T09:52:24.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.391+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:674 18.27 18:e692fc64:::1000000021b.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35800 con 0x561e8b183c00 2023-10-11T09:52:24.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.391+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 37 ==== osd_op_reply(674 1000000021b.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:24.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.391+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x1000000021b 2023-10-11T09:52:24.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.391+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/38 2023-10-11T09:52:24.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.391+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=38 2023-10-11T09:52:24.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.391+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:675 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b013400 con 0x561e8b013000 2023-10-11T09:52:24.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.391+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 130 ==== osd_op_reply(675 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.391+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.391+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:676 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2000 con 0x561e8b013800 2023-10-11T09:52:24.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.391+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 265 ==== osd_op_reply(676 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.391+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.391+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.391+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:24.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.391+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:677 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3c00 con 0x561e8b013800 2023-10-11T09:52:24.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.391+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 266 ==== osd_op_reply(677 10000000000.01000000 [omap-get-header out=274b] v0'0 uv53 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.391+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 53 2023-10-11T09:52:24.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.391+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:24.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.391+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:678 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3400 con 0x561e8b013800 2023-10-11T09:52:24.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.395+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 267 ==== osd_op_reply(678 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv53 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.395+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=38 frag=0*, r=0 2023-10-11T09:52:24.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.395+0000 7f239a687840 20 datascan.read_dentry: 38_head not found in result 2023-10-11T09:52:24.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.395+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000021b at 0x0x10000000000/38 with size=2 bytes 2023-10-11T09:52:24.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.395+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:679 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2c00 con 0x561e8b013800 2023-10-11T09:52:24.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.395+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 268 ==== osd_op_reply(679 10000000000.01000000 [omap-set-vals] v129'54 uv54 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.395+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/38 pointing to 0x0x1000000021b 2023-10-11T09:52:24.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.395+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:680 18.17 18:e8000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183800 con 0x561e8b183c00 2023-10-11T09:52:24.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.399+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 38 ==== osd_op_reply(680 [pgnls-filter start_epoch 129 out=101b] v129'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:24.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.399+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000215.00000000 2023-10-11T09:52:24.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.399+0000 7f239a687840 10 datascan.operator(): handling object 10000000215.0 2023-10-11T09:52:24.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.399+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:681 18.17 18:eace0b58:::10000000215.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183000 con 0x561e8b183c00 2023-10-11T09:52:24.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.399+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 39 ==== osd_op_reply(681 10000000215.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv14 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:24.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.399+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000215 2023-10-11T09:52:24.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.399+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/32 2023-10-11T09:52:24.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.399+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=32 2023-10-11T09:52:24.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.399+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:682 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b2400 con 0x561e8b013000 2023-10-11T09:52:24.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.399+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 131 ==== osd_op_reply(682 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.399+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.399+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:683 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b2c00 con 0x561e8b013800 2023-10-11T09:52:24.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.399+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 269 ==== osd_op_reply(683 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.399+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.399+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.399+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:24.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.399+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:684 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b2800 con 0x561e8b182400 2023-10-11T09:52:24.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.399+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 158 ==== osd_op_reply(684 10000000000.01800000 [omap-get-header out=274b] v0'0 uv50 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.399+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 50 2023-10-11T09:52:24.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.399+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:24.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.399+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:685 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b3800 con 0x561e8b182400 2023-10-11T09:52:24.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.403+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 159 ==== osd_op_reply(685 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv50 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.403+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=32 frag=1*, r=0 2023-10-11T09:52:24.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.403+0000 7f239a687840 20 datascan.read_dentry: 32_head not found in result 2023-10-11T09:52:24.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.403+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000215 at 0x0x10000000000/32 with size=2 bytes 2023-10-11T09:52:24.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.403+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:686 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b3000 con 0x561e8b182400 2023-10-11T09:52:24.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.403+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 160 ==== osd_op_reply(686 10000000000.01800000 [omap-set-vals] v129'51 uv51 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.403+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/32 pointing to 0x0x10000000215 2023-10-11T09:52:24.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.403+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:687 18.17 18:eb1b33a4::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b2000 con 0x561e8b183c00 2023-10-11T09:52:24.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.407+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 40 ==== osd_op_reply(687 [pgnls-filter start_epoch 129 out=81b] v129'15 uv15 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:24.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.407+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000253.00000000 2023-10-11T09:52:24.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.407+0000 7f239a687840 10 datascan.operator(): handling object 10000000253.0 2023-10-11T09:52:24.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.407+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:688 18.17 18:eb1b33a4:::10000000253.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35c00 con 0x561e8b183c00 2023-10-11T09:52:24.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.407+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 41 ==== osd_op_reply(688 10000000253.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:24.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.407+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000253 2023-10-11T09:52:24.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.407+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/94 2023-10-11T09:52:24.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.407+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=94 2023-10-11T09:52:24.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.407+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:689 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34c00 con 0x561e8b013000 2023-10-11T09:52:24.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.407+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 132 ==== osd_op_reply(689 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.407+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.407+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:690 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34800 con 0x561e8b013800 2023-10-11T09:52:24.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.407+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 270 ==== osd_op_reply(690 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.407+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.407+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.407+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:24.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.407+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:691 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b238800 con 0x561e8b182400 2023-10-11T09:52:24.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.407+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 161 ==== osd_op_reply(691 10000000000.01800000 [omap-get-header out=274b] v0'0 uv51 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.407+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 51 2023-10-11T09:52:24.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.407+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:24.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.407+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:692 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b238000 con 0x561e8b182400 2023-10-11T09:52:24.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.411+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 162 ==== osd_op_reply(692 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv51 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.411+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=94 frag=1*, r=0 2023-10-11T09:52:24.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.411+0000 7f239a687840 20 datascan.read_dentry: 94_head not found in result 2023-10-11T09:52:24.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.411+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000253 at 0x0x10000000000/94 with size=2 bytes 2023-10-11T09:52:24.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.411+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:693 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b239000 con 0x561e8b182400 2023-10-11T09:52:24.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.411+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 163 ==== osd_op_reply(693 10000000000.01800000 [omap-set-vals] v129'52 uv52 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.411+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/94 pointing to 0x0x10000000253 2023-10-11T09:52:24.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.411+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:694 18.37 18:ec000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b238400 con 0x561e8b1e3800 2023-10-11T09:52:24.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.411+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.7 v2:172.21.15.133:6824/35901 25 ==== osd_op_reply(694 [pgnls-filter start_epoch 129 out=49b] v127'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b1e3800 2023-10-11T09:52:24.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.411+0000 7f239463a700 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:695 18.f 18:f0000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b3c00 con 0x561e8b183c00 2023-10-11T09:52:24.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.415+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 42 ==== osd_op_reply(695 [pgnls-filter start_epoch 129 out=81b] v129'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:24.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.415+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 1000000022e.00000000 2023-10-11T09:52:24.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.415+0000 7f239a687840 10 datascan.operator(): handling object 1000000022e.0 2023-10-11T09:52:24.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.415+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:696 18.f 18:f25dd5df:::1000000022e.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b239400 con 0x561e8b183c00 2023-10-11T09:52:24.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.415+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.5 v2:172.21.15.133:6816/35899 43 ==== osd_op_reply(696 1000000022e.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b183c00 2023-10-11T09:52:24.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.415+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x1000000022e 2023-10-11T09:52:24.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.415+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/57 2023-10-11T09:52:24.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.415+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=57 2023-10-11T09:52:24.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.415+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:697 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b220800 con 0x561e8b013000 2023-10-11T09:52:24.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.415+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 133 ==== osd_op_reply(697 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.415+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.415+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:698 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221400 con 0x561e8b013800 2023-10-11T09:52:24.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.415+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 271 ==== osd_op_reply(698 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.415+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.415+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.415+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:24.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.415+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:699 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b220000 con 0x561e8b182400 2023-10-11T09:52:24.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.419+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 164 ==== osd_op_reply(699 10000000000.01800000 [omap-get-header out=274b] v0'0 uv52 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.419+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 52 2023-10-11T09:52:24.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.419+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:24.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.419+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:700 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221c00 con 0x561e8b182400 2023-10-11T09:52:24.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.419+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 165 ==== osd_op_reply(700 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv52 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.419+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=57 frag=1*, r=0 2023-10-11T09:52:24.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.419+0000 7f239a687840 20 datascan.read_dentry: 57_head not found in result 2023-10-11T09:52:24.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.419+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000022e at 0x0x10000000000/57 with size=2 bytes 2023-10-11T09:52:24.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.419+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:701 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b229c00 con 0x561e8b182400 2023-10-11T09:52:24.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.419+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 166 ==== osd_op_reply(701 10000000000.01800000 [omap-set-vals] v129'53 uv53 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.423+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/57 pointing to 0x0x1000000022e 2023-10-11T09:52:24.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.423+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:702 18.2f 18:f4000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b229800 con 0x561e8ae35000 2023-10-11T09:52:24.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.423+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.1 v2:172.21.15.78:6809/35734 20 ==== osd_op_reply(702 [pgnls-filter start_epoch 129 out=49b] v127'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8ae35000 2023-10-11T09:52:24.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.423+0000 7f239463a700 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:703 18.1f 18:f8000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b182c00 con 0x561e8b1e3800 2023-10-11T09:52:24.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.423+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.7 v2:172.21.15.133:6824/35901 26 ==== osd_op_reply(703 [pgnls-filter start_epoch 129 out=101b] v129'12 uv12 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b1e3800 2023-10-11T09:52:24.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.423+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000238.00000000 2023-10-11T09:52:24.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.423+0000 7f239a687840 10 datascan.operator(): handling object 10000000238.0 2023-10-11T09:52:24.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.423+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:704 18.1f 18:f87ea8f1:::10000000238.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b229000 con 0x561e8b1e3800 2023-10-11T09:52:24.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.423+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.7 v2:172.21.15.133:6824/35901 27 ==== osd_op_reply(704 10000000238.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b1e3800 2023-10-11T09:52:24.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.423+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000238 2023-10-11T09:52:24.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.423+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/67 2023-10-11T09:52:24.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.423+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=67 2023-10-11T09:52:24.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.423+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:705 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b228000 con 0x561e8b013000 2023-10-11T09:52:24.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.423+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 134 ==== osd_op_reply(705 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.423+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.423+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:706 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b228800 con 0x561e8b013800 2023-10-11T09:52:24.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.427+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 272 ==== osd_op_reply(706 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.427+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.427+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.427+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:24.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.427+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:707 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b229400 con 0x561e8b182400 2023-10-11T09:52:24.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.427+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 167 ==== osd_op_reply(707 10000000000.01800000 [omap-get-header out=274b] v0'0 uv53 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.427+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 53 2023-10-11T09:52:24.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.427+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:24.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.427+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:708 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183400 con 0x561e8b182400 2023-10-11T09:52:24.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.427+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 168 ==== osd_op_reply(708 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv53 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.427+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=67 frag=1*, r=0 2023-10-11T09:52:24.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.427+0000 7f239a687840 20 datascan.read_dentry: 67_head not found in result 2023-10-11T09:52:24.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.427+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000238 at 0x0x10000000000/67 with size=2 bytes 2023-10-11T09:52:24.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.427+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:709 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3000 con 0x561e8b182400 2023-10-11T09:52:24.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.431+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 169 ==== osd_op_reply(709 10000000000.01800000 [omap-set-vals] v129'54 uv54 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.431+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/67 pointing to 0x0x10000000238 2023-10-11T09:52:24.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.431+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:710 18.1f 18:f8e2d5cc::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e2800 con 0x561e8b1e3800 2023-10-11T09:52:24.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.431+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.7 v2:172.21.15.133:6824/35901 28 ==== osd_op_reply(710 [pgnls-filter start_epoch 129 out=81b] v129'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b1e3800 2023-10-11T09:52:24.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.431+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000214.00000000 2023-10-11T09:52:24.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.431+0000 7f239a687840 10 datascan.operator(): handling object 10000000214.0 2023-10-11T09:52:24.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.431+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:711 18.1f 18:f8e2d5cc:::10000000214.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1e3000 con 0x561e8b1e3800 2023-10-11T09:52:24.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.431+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.7 v2:172.21.15.133:6824/35901 29 ==== osd_op_reply(711 10000000214.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b1e3800 2023-10-11T09:52:24.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.431+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000214 2023-10-11T09:52:24.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.431+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/31 2023-10-11T09:52:24.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.431+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=31 2023-10-11T09:52:24.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.431+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:712 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b183400 con 0x561e8b013000 2023-10-11T09:52:24.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.431+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 135 ==== osd_op_reply(712 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.431+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.435+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:713 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b229c00 con 0x561e8b013800 2023-10-11T09:52:24.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.435+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 273 ==== osd_op_reply(713 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.435+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.435+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.435+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:24.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.435+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:714 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b229400 con 0x561e8b013800 2023-10-11T09:52:24.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.435+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 274 ==== osd_op_reply(714 10000000000.01000000 [omap-get-header out=274b] v0'0 uv54 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.435+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 54 2023-10-11T09:52:24.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.435+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:24.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.435+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:715 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b220c00 con 0x561e8b013800 2023-10-11T09:52:24.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.435+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 275 ==== osd_op_reply(715 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv54 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.435+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=31 frag=0*, r=0 2023-10-11T09:52:24.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.435+0000 7f239a687840 20 datascan.read_dentry: 31_head not found in result 2023-10-11T09:52:24.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.435+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000214 at 0x0x10000000000/31 with size=2 bytes 2023-10-11T09:52:24.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.435+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:716 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221800 con 0x561e8b013800 2023-10-11T09:52:24.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.439+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 276 ==== osd_op_reply(716 10000000000.01000000 [omap-set-vals] v129'55 uv55 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.439+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/31 pointing to 0x0x10000000214 2023-10-11T09:52:24.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.439+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:717 18.3f 18:fc000000::::head [pgnls-filter start_epoch 129 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221000 con 0x561e8b182400 2023-10-11T09:52:24.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.439+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 170 ==== osd_op_reply(717 [pgnls-filter start_epoch 129 out=101b] v129'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.439+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 10000000232.00000000 2023-10-11T09:52:24.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.439+0000 7f239a687840 10 datascan.operator(): handling object 10000000232.0 2023-10-11T09:52:24.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.439+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:718 18.3f 18:fd83cc12:::10000000232.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b220000 con 0x561e8b182400 2023-10-11T09:52:24.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.439+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 171 ==== osd_op_reply(718 10000000232.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.439+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x10000000232 2023-10-11T09:52:24.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.439+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/61 2023-10-11T09:52:24.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.439+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=61 2023-10-11T09:52:24.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.439+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:719 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b221c00 con 0x561e8b013000 2023-10-11T09:52:24.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.439+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 136 ==== osd_op_reply(719 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.439+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.439+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:720 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b238c00 con 0x561e8b013800 2023-10-11T09:52:24.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.443+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 277 ==== osd_op_reply(720 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.443+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.443+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.443+0000 7f239a687840 20 datascan.get_frag_of: frag is 0* 2023-10-11T09:52:24.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.443+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:721 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b238800 con 0x561e8b013800 2023-10-11T09:52:24.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.443+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 278 ==== osd_op_reply(721 10000000000.01000000 [omap-get-header out=274b] v0'0 uv55 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.443+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 55 2023-10-11T09:52:24.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.443+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-10-11T09:52:24.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.443+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:722 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b238000 con 0x561e8b013800 2023-10-11T09:52:24.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.443+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 279 ==== osd_op_reply(722 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv55 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.443+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01000000 dname=61 frag=0*, r=0 2023-10-11T09:52:24.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.443+0000 7f239a687840 20 datascan.read_dentry: 61_head not found in result 2023-10-11T09:52:24.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.443+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000232 at 0x0x10000000000/61 with size=2 bytes 2023-10-11T09:52:24.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.443+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:723 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b239000 con 0x561e8b013800 2023-10-11T09:52:24.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.443+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 280 ==== osd_op_reply(723 10000000000.01000000 [omap-set-vals] v129'56 uv56 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.443+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/61 pointing to 0x0x10000000232 2023-10-11T09:52:24.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.443+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:724 18.3f 18:feb1c803::::head [pgnls-filter start_epoch 129 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b2800 con 0x561e8b182400 2023-10-11T09:52:24.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.447+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 172 ==== osd_op_reply(724 [pgnls-filter start_epoch 129 out=81b] v129'14 uv14 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.447+0000 7f239a687840 20 datascan.forall_objects: OSD matched oid 100000001fa.00000000 2023-10-11T09:52:24.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.447+0000 7f239a687840 10 datascan.operator(): handling object 100000001fa.0 2023-10-11T09:52:24.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.447+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:725 18.3f 18:feb1c803:::100000001fa.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b3800 con 0x561e8b182400 2023-10-11T09:52:24.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.447+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 173 ==== osd_op_reply(725 100000001fa.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv14 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.447+0000 7f239a687840 10 datascan.inject_with_backtrace: inode: 0x0x100000001fa 2023-10-11T09:52:24.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.447+0000 7f239a687840 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/5 2023-10-11T09:52:24.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.447+0000 7f239a687840 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=5 2023-10-11T09:52:24.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.447+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:726 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b1b3000 con 0x561e8b013000 2023-10-11T09:52:24.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.447+0000 7f2398e43700 1 -- 172.21.15.78:0/612779174 <== osd.3 v2:172.21.15.78:6824/35738 137 ==== osd_op_reply(726 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x561e8bb0afc0 con 0x561e8b013000 2023-10-11T09:52:24.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.447+0000 7f239a687840 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-10-11T09:52:24.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.447+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:727 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae35400 con 0x561e8b013800 2023-10-11T09:52:24.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.447+0000 7f2399644700 1 -- 172.21.15.78:0/612779174 <== osd.6 v2:172.21.15.133:6800/35900 281 ==== osd_op_reply(727 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x561e8bb0b200 con 0x561e8b013800 2023-10-11T09:52:24.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.447+0000 7f239a687840 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-11T09:52:24.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.447+0000 7f239a687840 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-10-11T09:52:24.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.447+0000 7f239a687840 20 datascan.get_frag_of: frag is 1* 2023-10-11T09:52:24.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.447+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:728 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34400 con 0x561e8b182400 2023-10-11T09:52:24.459 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.447+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 174 ==== osd_op_reply(728 10000000000.01800000 [omap-get-header out=274b] v0'0 uv54 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.447+0000 7f239a687840 10 datascan.find_or_create_dirfrag: read_version = 54 2023-10-11T09:52:24.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.447+0000 7f239a687840 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-10-11T09:52:24.460 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.447+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:729 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8ae34000 con 0x561e8b182400 2023-10-11T09:52:24.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.447+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 175 ==== osd_op_reply(729 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv54 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.447+0000 7f239a687840 20 datascan.read_dentry: oid=10000000000.01800000 dname=5 frag=1*, r=0 2023-10-11T09:52:24.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.447+0000 7f239a687840 20 datascan.read_dentry: 5_head not found in result 2023-10-11T09:52:24.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.447+0000 7f239a687840 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fa at 0x0x10000000000/5 with size=1 bytes 2023-10-11T09:52:24.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.447+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:730 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x561e8b238400 con 0x561e8b182400 2023-10-11T09:52:24.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.451+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 <== osd.4 v2:172.21.15.133:6808/35898 176 ==== osd_op_reply(730 10000000000.01800000 [omap-set-vals] v129'55 uv55 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e8bb0b440 con 0x561e8b182400 2023-10-11T09:52:24.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.451+0000 7f239a687840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/5 pointing to 0x0x100000001fa 2023-10-11T09:52:24.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.451+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x561e8b182800 msgr2=0x561e8adcd600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:24.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.451+0000 7f239a687840 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x561e8b182800 0x561e8adcd600 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:24.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.451+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x561e8ae35000 msgr2=0x561e8b19c000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:24.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.451+0000 7f239a687840 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x561e8ae35000 0x561e8b19c000 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:24.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.451+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x561e8b1e2400 msgr2=0x561e8b19f180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:24.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.451+0000 7f239a687840 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x561e8b1e2400 0x561e8b19f180 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:24.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.451+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x561e8b013000 msgr2=0x561e8adce100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:24.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.451+0000 7f239a687840 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x561e8b013000 0x561e8adce100 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:24.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.451+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 reap_dead start 2023-10-11T09:52:24.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.451+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x561e8b182400 msgr2=0x561e8b19c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:24.465 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.451+0000 7f239a687840 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x561e8b182400 0x561e8b19c580 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:24.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.451+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x561e8b183c00 msgr2=0x561e8b19cb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:24.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.451+0000 7f239a687840 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x561e8b183c00 0x561e8b19cb00 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:24.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.451+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x561e8b013800 msgr2=0x561e8adcdb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:24.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.451+0000 7f239a687840 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x561e8b013800 0x561e8adcdb80 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:24.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.451+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x561e8b1e3800 msgr2=0x561e8b1f2000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:24.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.451+0000 7f239a687840 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x561e8b1e3800 0x561e8b1f2000 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:24.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.451+0000 7f2398642700 1 -- 172.21.15.78:0/612779174 reap_dead start 2023-10-11T09:52:24.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.451+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x561e8b182000 msgr2=0x561e8adcf700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:24.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.451+0000 7f239a687840 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x561e8b182000 0x561e8adcf700 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x561e8b02c690 tx=0x561e8ae8cf60 comp rx=0 tx=0).stop 2023-10-11T09:52:24.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.451+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e8b012c00 msgr2=0x561e8adce680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:24.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.451+0000 7f239a687840 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e8b012c00 0x561e8adce680 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x561e8b02d350 tx=0x561e8b02ed20 comp rx=0 tx=0).stop 2023-10-11T09:52:24.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.451+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 shutdown_connections 2023-10-11T09:52:24.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.451+0000 7f239a687840 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x561e8b182000 0x561e8adcf700 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:24.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.451+0000 7f239a687840 1 --2- 172.21.15.78:0/612779174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e8b012c00 0x561e8adce680 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:24.468 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.451+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 >> 172.21.15.78:0/612779174 conn(0x561e8ad26c00 msgr2=0x561e8ae17800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:24.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.451+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 shutdown_connections 2023-10-11T09:52:24.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.451+0000 7f239a687840 1 -- 172.21.15.78:0/612779174 wait complete. 2023-10-11T09:52:24.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.451+0000 7f239a687840 1 -- 172.21.15.78:0/2275839196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e8ad27800 msgr2=0x561e8adcc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:24.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.451+0000 7f239a687840 1 --2- 172.21.15.78:0/2275839196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e8ad27800 0x561e8adcc580 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x561e8adf5320 tx=0x561e8ae8c000 comp rx=0 tx=0).stop 2023-10-11T09:52:24.469 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.451+0000 7f239a687840 1 -- 172.21.15.78:0/2275839196 shutdown_connections 2023-10-11T09:52:24.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.451+0000 7f239a687840 1 --2- 172.21.15.78:0/2275839196 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x561e8ad27000 0x561e8adcd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:24.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.451+0000 7f239a687840 1 --2- 172.21.15.78:0/2275839196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e8ad27400 0x561e8adccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:24.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.451+0000 7f239a687840 1 --2- 172.21.15.78:0/2275839196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e8ad27800 0x561e8adcc580 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:24.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.451+0000 7f239a687840 1 -- 172.21.15.78:0/2275839196 >> 172.21.15.78:0/2275839196 conn(0x561e8ad27c00 msgr2=0x561e8ae17000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:24.470 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.455+0000 7f239a687840 1 -- 172.21.15.78:0/2275839196 shutdown_connections 2023-10-11T09:52:24.471 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.455+0000 7f239a687840 1 -- 172.21.15.78:0/2275839196 wait complete. 2023-10-11T09:52:24.472 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes'] in time 0:00:01.009869, result: 2023-10-11T09:52:24.472 DEBUG:teuthology.orchestra.run.smithi078:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2023-10-11T09:52:24.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.567+0000 7f0731a3b840 1 -- start start 2023-10-11T09:52:24.575 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.567+0000 7f0731a3b840 1 Processor -- start 2023-10-11T09:52:24.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.567+0000 7f0731a3b840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649233ed800 0x564923492580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:24.576 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.567+0000 7f0731a3b840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5649233ed400 0x564923492b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:24.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.567+0000 7f0731a3b840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649233ed000 0x564923493080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:24.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.567+0000 7f0731a3b840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56492346c780 con 0x5649233ed000 2023-10-11T09:52:24.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.567+0000 7f0731a3b840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56492346ca00 con 0x5649233ed800 2023-10-11T09:52:24.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.567+0000 7f0731a3b840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x56492346c640 con 0x5649233ed400 2023-10-11T09:52:24.577 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.567+0000 7f07309f8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649233ed000 0x564923493080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:24.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.567+0000 7f07309f8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649233ed000 0x564923493080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50426/0 (socket says 172.21.15.78:50426) 2023-10-11T09:52:24.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.567+0000 7f07309f8700 1 -- 172.21.15.78:0/1333752241 learned_addr learned my addr 172.21.15.78:0/1333752241 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:24.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.567+0000 7f07301f7700 1 --2- 172.21.15.78:0/1333752241 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649233ed800 0x564923492580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:24.578 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.567+0000 7f072f9f6700 1 --2- 172.21.15.78:0/1333752241 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5649233ed400 0x564923492b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:24.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.571+0000 7f07309f8700 1 -- 172.21.15.78:0/1333752241 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5649233ed400 msgr2=0x564923492b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:24.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.571+0000 7f07309f8700 1 --2- 172.21.15.78:0/1333752241 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5649233ed400 0x564923492b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:24.580 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.571+0000 7f07309f8700 1 -- 172.21.15.78:0/1333752241 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649233ed800 msgr2=0x564923492580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:24.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.571+0000 7f07309f8700 1 --2- 172.21.15.78:0/1333752241 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649233ed800 0x564923492580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:24.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.571+0000 7f07309f8700 1 -- 172.21.15.78:0/1333752241 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56492340eb60 con 0x5649233ed000 2023-10-11T09:52:24.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.571+0000 7f07309f8700 1 --2- 172.21.15.78:0/1333752241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649233ed000 0x564923493080 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x5649241553e0 tx=0x5649234e3020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e46b30ace10bfaa server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:24.581 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.571+0000 7f072d9f2700 1 -- 172.21.15.78:0/1333752241 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56492341d600 con 0x5649233ed000 2023-10-11T09:52:24.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.571+0000 7f072d9f2700 1 -- 172.21.15.78:0/1333752241 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564923438480 con 0x5649233ed000 2023-10-11T09:52:24.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.571+0000 7f072d9f2700 1 -- 172.21.15.78:0/1333752241 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56492341d1e0 con 0x5649233ed000 2023-10-11T09:52:24.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.571+0000 7f0731a3b840 1 -- 172.21.15.78:0/1333752241 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56492340e340 con 0x5649233ed000 2023-10-11T09:52:24.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.571+0000 7f0731a3b840 1 -- 172.21.15.78:0/1333752241 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x56492340e000 con 0x5649233ed000 2023-10-11T09:52:24.582 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.571+0000 7f0731a3b840 4 waiting for MDS map... 2023-10-11T09:52:24.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.571+0000 7f072d9f2700 1 -- 172.21.15.78:0/1333752241 <== mon.0 v2:172.21.15.78:3300/0 4 ==== osd_map(129..129 src has 1..129) v4 ==== 6234+0+0 (secure 0 0 0) 0x564923426c40 con 0x5649233ed000 2023-10-11T09:52:24.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.571+0000 7f072d9f2700 1 -- 172.21.15.78:0/1333752241 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 107) v1 ==== 3110+0+0 (secure 0 0 0) 0x5649236ed200 con 0x5649233ed000 2023-10-11T09:52:24.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.571+0000 7f0731a3b840 4 Got MDS map 107 2023-10-11T09:52:24.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.571+0000 7f0731a3b840 4 datascan.main: Using metadata pool output 2023-10-11T09:52:24.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.571+0000 7f0731a3b840 4 datascan.main: connecting to RADOS... 2023-10-11T09:52:24.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.571+0000 7f0731a3b840 1 Processor -- start 2023-10-11T09:52:24.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.571+0000 7f0731a3b840 1 -- start start 2023-10-11T09:52:24.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.575+0000 7f0731a3b840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649236e0400 0x564923493600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:24.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.575+0000 7f0731a3b840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649236e0800 0x564923493b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:24.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.575+0000 7f0731a3b840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5649236e0c00 0x564923494100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:24.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.575+0000 7f0731a3b840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x564923423e00 con 0x5649236e0400 2023-10-11T09:52:24.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.575+0000 7f0731a3b840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56492346ca00 con 0x5649236e0800 2023-10-11T09:52:24.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.575+0000 7f0731a3b840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x56492346c500 con 0x5649236e0c00 2023-10-11T09:52:24.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.575+0000 7f072f9f6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649236e0400 0x564923493600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:24.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.575+0000 7f072f9f6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649236e0400 0x564923493600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50434/0 (socket says 172.21.15.78:50434) 2023-10-11T09:52:24.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.575+0000 7f072f9f6700 1 -- 172.21.15.78:0/2599911418 learned_addr learned my addr 172.21.15.78:0/2599911418 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:24.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.575+0000 7f07301f7700 1 --2- 172.21.15.78:0/2599911418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649236e0800 0x564923493b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:24.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.575+0000 7f072f9f6700 1 --2- 172.21.15.78:0/2599911418 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5649236e0c00 0x564923494100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:24.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.575+0000 7f072f9f6700 1 -- 172.21.15.78:0/2599911418 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5649236e0c00 msgr2=0x564923494100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:24.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.575+0000 7f072f9f6700 1 --2- 172.21.15.78:0/2599911418 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5649236e0c00 0x564923494100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:24.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.575+0000 7f072f9f6700 1 -- 172.21.15.78:0/2599911418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649236e0800 msgr2=0x564923493b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:24.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.575+0000 7f072f9f6700 1 --2- 172.21.15.78:0/2599911418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649236e0800 0x564923493b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:24.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.575+0000 7f072f9f6700 1 -- 172.21.15.78:0/2599911418 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56492340eea0 con 0x5649236e0400 2023-10-11T09:52:24.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.575+0000 7f072f9f6700 1 --2- 172.21.15.78:0/2599911418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649236e0400 0x564923493600 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x5649236f6930 tx=0x564923700c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb867c32068d02a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:24.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.575+0000 7f072b9ee700 1 -- 172.21.15.78:0/2599911418 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56492341d600 con 0x5649236e0400 2023-10-11T09:52:24.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.575+0000 7f072b9ee700 1 -- 172.21.15.78:0/2599911418 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564923438480 con 0x5649236e0400 2023-10-11T09:52:24.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.575+0000 7f072b9ee700 1 -- 172.21.15.78:0/2599911418 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56492341d1e0 con 0x5649236e0400 2023-10-11T09:52:24.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.575+0000 7f0731a3b840 1 -- 172.21.15.78:0/2599911418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649236e0400 msgr2=0x564923493600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:24.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.575+0000 7f0731a3b840 1 --2- 172.21.15.78:0/2599911418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649236e0400 0x564923493600 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x5649236f6930 tx=0x564923700c60 comp rx=0 tx=0).stop 2023-10-11T09:52:24.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.575+0000 7f0731a3b840 1 -- 172.21.15.78:0/2599911418 shutdown_connections 2023-10-11T09:52:24.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.575+0000 7f0731a3b840 1 --2- 172.21.15.78:0/2599911418 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5649236e0c00 0x564923494100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:24.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.575+0000 7f0731a3b840 1 --2- 172.21.15.78:0/2599911418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649236e0800 0x564923493b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:24.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.575+0000 7f0731a3b840 1 --2- 172.21.15.78:0/2599911418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649236e0400 0x564923493600 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:24.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.575+0000 7f0731a3b840 1 -- 172.21.15.78:0/2599911418 >> 172.21.15.78:0/2599911418 conn(0x5649233ecc00 msgr2=0x5649234bb800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:24.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.575+0000 7f0731a3b840 1 -- 172.21.15.78:0/2599911418 shutdown_connections 2023-10-11T09:52:24.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.575+0000 7f0731a3b840 1 -- 172.21.15.78:0/2599911418 wait complete. 2023-10-11T09:52:24.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.579+0000 7f0731a3b840 1 Processor -- start 2023-10-11T09:52:24.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.579+0000 7f0731a3b840 1 -- start start 2023-10-11T09:52:24.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.579+0000 7f0731a3b840 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649236e0400 0x564923495180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:24.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.579+0000 7f0731a3b840 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649236e0800 0x564923494c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:24.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.579+0000 7f0731a3b840 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5649236e0c00 0x564923494680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:24.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.579+0000 7f0731a3b840 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56492346c3c0 con 0x5649236e0800 2023-10-11T09:52:24.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.579+0000 7f0731a3b840 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56492346c280 con 0x5649236e0400 2023-10-11T09:52:24.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.579+0000 7f0731a3b840 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x56492346c140 con 0x5649236e0c00 2023-10-11T09:52:24.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.579+0000 7f07301f7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649236e0800 0x564923494c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:24.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.579+0000 7f07301f7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649236e0800 0x564923494c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50438/0 (socket says 172.21.15.78:50438) 2023-10-11T09:52:24.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.579+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 learned_addr learned my addr 172.21.15.78:0/4264677043 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:24.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.579+0000 7f072f9f6700 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649236e0400 0x564923495180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:24.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.579+0000 7f072f9f6700 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5649236e0c00 0x564923494680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:24.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.579+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5649236e0c00 msgr2=0x564923494680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:24.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.579+0000 7f07301f7700 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5649236e0c00 0x564923494680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:24.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.579+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649236e0400 msgr2=0x564923495180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:24.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.579+0000 7f07301f7700 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649236e0400 0x564923495180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:24.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.579+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56492340eb60 con 0x5649236e0800 2023-10-11T09:52:24.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.579+0000 7f07301f7700 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649236e0800 0x564923494c00 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x5649236f7650 tx=0x5649234e2000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:24.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.579+0000 7f072a1eb700 1 -- 172.21.15.78:0/4264677043 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56492341d080 con 0x5649236e0800 2023-10-11T09:52:24.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.579+0000 7f072a1eb700 1 -- 172.21.15.78:0/4264677043 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5649241dc300 con 0x5649236e0800 2023-10-11T09:52:24.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.579+0000 7f072a1eb700 1 -- 172.21.15.78:0/4264677043 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56492341d600 con 0x5649236e0800 2023-10-11T09:52:24.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.579+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56492340e340 con 0x5649236e0800 2023-10-11T09:52:24.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.579+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56492340f520 con 0x5649236e0800 2023-10-11T09:52:24.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.579+0000 7f0731a3b840 4 datascan.init: resolving metadata pool 17 2023-10-11T09:52:24.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.583+0000 7f072a1eb700 1 -- 172.21.15.78:0/4264677043 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x56492344a840 con 0x5649236e0800 2023-10-11T09:52:24.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.583+0000 7f072a1eb700 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x564923844000 0x564923495700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:24.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.583+0000 7f072a1eb700 1 -- 172.21.15.78:0/4264677043 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6234+0+0 (secure 0 0 0) 0x564923426a80 con 0x5649236e0800 2023-10-11T09:52:24.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.583+0000 7f0731a3b840 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-10-11T09:52:24.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.583+0000 7f0731a3b840 4 datascan.main: resolving metadata pool 17 2023-10-11T09:52:24.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.583+0000 7f0731a3b840 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x5649236e1000 0x564923494100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:24.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.583+0000 7f072f9f6700 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x564923844000 0x564923495700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:24.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.583+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:1 17.0 17:00000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649236e1400 con 0x5649236e1000 2023-10-11T09:52:24.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.587+0000 7f072f9f6700 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x5649236e1000 0x564923494100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:24.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.587+0000 7f072f9f6700 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x564923844000 0x564923495700 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x5649236f6a20 tx=0x5649234e2f60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:24.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.587+0000 7f072f9f6700 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x5649236e1000 0x564923494100 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:24.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.587+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 <== osd.7 v2:172.21.15.133:6824/35901 1 ==== osd_op_reply(1 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649241d8fc0 con 0x5649236e1000 2023-10-11T09:52:24.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.587+0000 7f072f9f6700 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5649236e1400 0x56492384e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:24.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.587+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:2 17.10 17:08000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923844400 con 0x5649236e1400 2023-10-11T09:52:24.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.587+0000 7f07309f8700 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5649236e1400 0x56492384e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:24.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.587+0000 7f07309f8700 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5649236e1400 0x56492384e000 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:24.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.591+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== osd_op_reply(2 [pgnls start_epoch 129 out=177b] v128'12 uv12 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x5649241d9200 con 0x5649236e1400 2023-10-11T09:52:24.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.591+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:3 17.8 17:10000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923844400 con 0x5649236e1000 2023-10-11T09:52:24.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.591+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 <== osd.7 v2:172.21.15.133:6824/35901 2 ==== osd_op_reply(3 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649241d8fc0 con 0x5649236e1000 2023-10-11T09:52:24.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.591+0000 7f072f9f6700 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x564923844400 0x56492384e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:24.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.591+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:4 17.18 17:18000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923844800 con 0x564923844400 2023-10-11T09:52:24.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.591+0000 7f07301f7700 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x564923844400 0x56492384e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:24.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.591+0000 7f07301f7700 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x564923844400 0x56492384e580 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:24.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.591+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.0 v2:172.21.15.78:6800/35735 1 ==== osd_op_reply(4 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649241d9440 con 0x564923844400 2023-10-11T09:52:24.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.591+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:5 17.4 17:20000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923844800 con 0x564923844400 2023-10-11T09:52:24.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.595+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.0 v2:172.21.15.78:6800/35735 2 ==== osd_op_reply(5 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649241d9440 con 0x564923844400 2023-10-11T09:52:24.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.595+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:6 17.14 17:28000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923844800 con 0x5649236e1400 2023-10-11T09:52:24.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.595+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.3 v2:172.21.15.78:6824/35738 2 ==== osd_op_reply(6 [pgnls start_epoch 129 out=149b] v128'28 uv28 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x5649241d9200 con 0x5649236e1400 2023-10-11T09:52:24.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.595+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:7 17.c 17:30000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923844800 con 0x5649236e1000 2023-10-11T09:52:24.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.595+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 <== osd.7 v2:172.21.15.133:6824/35901 3 ==== osd_op_reply(7 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649241d8fc0 con 0x5649236e1000 2023-10-11T09:52:24.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.595+0000 7f072f9f6700 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x564923844800 0x56492384eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:24.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.595+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:8 17.1c 17:38000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923844c00 con 0x564923844800 2023-10-11T09:52:24.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.595+0000 7f072f9f6700 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x564923844800 0x56492384eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:24.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.595+0000 7f072f9f6700 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x564923844800 0x56492384eb00 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:24.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.595+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 <== osd.1 v2:172.21.15.78:6809/35734 1 ==== osd_op_reply(8 [pgnls start_epoch 129 out=73b] v126'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5649241d8fc0 con 0x564923844800 2023-10-11T09:52:24.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.595+0000 7f072f9f6700 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x564923844c00 0x56492384f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:24.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.595+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:9 17.2 17:40000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845000 con 0x564923844c00 2023-10-11T09:52:24.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.599+0000 7f07309f8700 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x564923844c00 0x56492384f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:24.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.599+0000 7f07309f8700 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x564923844c00 0x56492384f080 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:24.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.599+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 1 ==== osd_op_reply(9 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.599+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:10 17.12 17:48000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845000 con 0x5649236e1400 2023-10-11T09:52:24.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.599+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.3 v2:172.21.15.78:6824/35738 3 ==== osd_op_reply(10 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649241d9200 con 0x5649236e1400 2023-10-11T09:52:24.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.599+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:11 17.a 17:50000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845000 con 0x5649236e1000 2023-10-11T09:52:24.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.599+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 <== osd.7 v2:172.21.15.133:6824/35901 4 ==== osd_op_reply(11 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649241d8fc0 con 0x5649236e1000 2023-10-11T09:52:24.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.599+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:12 17.1a 17:58000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845000 con 0x564923844400 2023-10-11T09:52:24.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.603+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.0 v2:172.21.15.78:6800/35735 3 ==== osd_op_reply(12 [pgnls start_epoch 129 out=49b] v128'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649241d9440 con 0x564923844400 2023-10-11T09:52:24.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.603+0000 7f07301f7700 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x564923845000 0x564923850100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:24.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.603+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:13 17.6 17:60000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845c00 con 0x564923845000 2023-10-11T09:52:24.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.603+0000 7f07301f7700 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x564923845000 0x564923850100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:24.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.603+0000 7f07301f7700 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x564923845000 0x564923850100 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:24.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.603+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== osd_op_reply(13 [pgnls start_epoch 129 out=77b] v128'8 uv8 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.603+0000 7f07301f7700 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x564923845c00 0x564923850680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:24.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.603+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:14 17.16 17:68000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874000 con 0x564923845c00 2023-10-11T09:52:24.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.607+0000 7f072f9f6700 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x564923845c00 0x564923850680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:24.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.607+0000 7f072f9f6700 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x564923845c00 0x564923850680 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:24.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.607+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 <== osd.5 v2:172.21.15.133:6816/35899 1 ==== osd_op_reply(14 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649241d8fc0 con 0x564923845c00 2023-10-11T09:52:24.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.607+0000 7f072f9f6700 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x564923874000 0x56492384f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:24.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.607+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:15 17.e 17:70000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845400 con 0x564923874000 2023-10-11T09:52:24.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.607+0000 7f07309f8700 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x564923874000 0x56492384f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:24.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.607+0000 7f07309f8700 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x564923874000 0x56492384f600 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:24.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.611+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.2 v2:172.21.15.78:6808/35732 1 ==== osd_op_reply(15 [pgnls start_epoch 129 out=73b] v126'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5649241d9200 con 0x564923874000 2023-10-11T09:52:24.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.611+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:16 17.1e 17:78000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845400 con 0x564923844800 2023-10-11T09:52:24.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.611+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 <== osd.1 v2:172.21.15.78:6809/35734 2 ==== osd_op_reply(16 [pgnls start_epoch 129 out=49b] v128'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649241d8fc0 con 0x564923844800 2023-10-11T09:52:24.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.611+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:17 17.1 17:80000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845400 con 0x564923844400 2023-10-11T09:52:24.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.611+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.0 v2:172.21.15.78:6800/35735 4 ==== osd_op_reply(17 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649241d9440 con 0x564923844400 2023-10-11T09:52:24.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.611+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:18 17.11 17:88000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845400 con 0x5649236e1400 2023-10-11T09:52:24.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.611+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.3 v2:172.21.15.78:6824/35738 4 ==== osd_op_reply(18 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649241d9200 con 0x5649236e1400 2023-10-11T09:52:24.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.611+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:19 17.9 17:90000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845400 con 0x5649236e1400 2023-10-11T09:52:24.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.611+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.3 v2:172.21.15.78:6824/35738 5 ==== osd_op_reply(19 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649241d9200 con 0x5649236e1400 2023-10-11T09:52:24.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.611+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:20 17.19 17:98000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845400 con 0x564923845c00 2023-10-11T09:52:24.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.615+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 <== osd.5 v2:172.21.15.133:6816/35899 2 ==== osd_op_reply(20 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649241d8fc0 con 0x564923845c00 2023-10-11T09:52:24.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.615+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:21 17.5 17:a0000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845400 con 0x564923845000 2023-10-11T09:52:24.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.615+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 2 ==== osd_op_reply(21 [pgnls start_epoch 129 out=81b] v129'56 uv56 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.615+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:22 17.15 17:a8000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845400 con 0x564923845c00 2023-10-11T09:52:24.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.615+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 <== osd.5 v2:172.21.15.133:6816/35899 3 ==== osd_op_reply(22 [pgnls start_epoch 129 out=97b] v126'3 uv3 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x5649241d8fc0 con 0x564923845c00 2023-10-11T09:52:24.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.615+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:23 17.d 17:b0000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845400 con 0x5649236e1000 2023-10-11T09:52:24.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.615+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 <== osd.7 v2:172.21.15.133:6824/35901 5 ==== osd_op_reply(23 [pgnls start_epoch 129 out=74b] v126'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x5649241d8fc0 con 0x5649236e1000 2023-10-11T09:52:24.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.615+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:24 17.1d 17:b8000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845400 con 0x5649236e1000 2023-10-11T09:52:24.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.619+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 <== osd.7 v2:172.21.15.133:6824/35901 6 ==== osd_op_reply(24 [pgnls start_epoch 129 out=73b] v126'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5649241d8fc0 con 0x5649236e1000 2023-10-11T09:52:24.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.619+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:25 17.3 17:c0000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845400 con 0x564923844c00 2023-10-11T09:52:24.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.619+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 2 ==== osd_op_reply(25 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.619+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:26 17.13 17:c8000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845400 con 0x5649236e1400 2023-10-11T09:52:24.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.619+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.3 v2:172.21.15.78:6824/35738 6 ==== osd_op_reply(26 [pgnls start_epoch 129 out=98b] v127'3 uv3 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x5649241d9200 con 0x5649236e1400 2023-10-11T09:52:24.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.619+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:27 17.b 17:d0000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845400 con 0x564923844400 2023-10-11T09:52:24.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.619+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.0 v2:172.21.15.78:6800/35735 5 ==== osd_op_reply(27 [pgnls start_epoch 129 out=76b] v128'4 uv3 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x5649241d9440 con 0x564923844400 2023-10-11T09:52:24.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.619+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:28 17.1b 17:d8000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845400 con 0x564923844c00 2023-10-11T09:52:24.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.623+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 3 ==== osd_op_reply(28 [pgnls start_epoch 129 out=81b] v129'55 uv55 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.623+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:29 17.7 17:e0000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845400 con 0x564923844400 2023-10-11T09:52:24.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.623+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.0 v2:172.21.15.78:6800/35735 6 ==== osd_op_reply(29 [pgnls start_epoch 129 out=73b] v126'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5649241d9440 con 0x564923844400 2023-10-11T09:52:24.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.623+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:30 17.17 17:e8000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845400 con 0x564923845000 2023-10-11T09:52:24.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.623+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 3 ==== osd_op_reply(30 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.623+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:31 17.f 17:f0000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845400 con 0x564923844800 2023-10-11T09:52:24.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.623+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 <== osd.1 v2:172.21.15.78:6809/35734 3 ==== osd_op_reply(31 [pgnls start_epoch 129 out=73b] v126'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5649241d8fc0 con 0x564923844800 2023-10-11T09:52:24.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.623+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:32 17.1f 17:f8000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845400 con 0x564923845000 2023-10-11T09:52:24.632 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.623+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 4 ==== osd_op_reply(32 [pgnls start_epoch 129 out=101b] v128'7 uv6 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.627+0000 7f0731a3b840 10 datascan.scan_links: step 1: handling object 601.00000000 2023-10-11T09:52:24.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.627+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:33 17.10 17:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649236e1800 con 0x5649236e1400 2023-10-11T09:52:24.633 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.627+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.3 v2:172.21.15.78:6824/35738 7 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5649241d9200 con 0x5649236e1400 2023-10-11T09:52:24.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.627+0000 7f0731a3b840 10 datascan.scan_links: step 1: handling object 602.00000000 2023-10-11T09:52:24.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.627+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:34 17.10 17:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x5649236e1400 2023-10-11T09:52:24.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.627+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.3 v2:172.21.15.78:6824/35738 8 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv6 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5649241d9200 con 0x5649236e1400 2023-10-11T09:52:24.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.627+0000 7f0731a3b840 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2023-10-11T09:52:24.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.627+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:35 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875000 con 0x5649236e1400 2023-10-11T09:52:24.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.627+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.3 v2:172.21.15.78:6824/35738 9 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=50980b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+50980 (crc 0 0 0) 0x5649241d9200 con 0x5649236e1400 2023-10-11T09:52:24.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.631+0000 7f0731a3b840 10 datascan.scan_links: step 1: handling object 600.00000000 2023-10-11T09:52:24.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.631+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:36 17.10 17:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875400 con 0x5649236e1400 2023-10-11T09:52:24.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.631+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.3 v2:172.21.15.78:6824/35738 10 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5649241d9200 con 0x5649236e1400 2023-10-11T09:52:24.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.631+0000 7f0731a3b840 10 datascan.scan_links: step 1: handling object 603.00000000 2023-10-11T09:52:24.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.631+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:37 17.10 17:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875800 con 0x5649236e1400 2023-10-11T09:52:24.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.631+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.3 v2:172.21.15.78:6824/35738 11 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv8 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5649241d9200 con 0x5649236e1400 2023-10-11T09:52:24.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.631+0000 7f0731a3b840 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2023-10-11T09:52:24.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.631+0000 7f0731a3b840 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-10-11T09:52:24.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.631+0000 7f0731a3b840 10 datascan.scan_links: step 1: handling object 200.00000000 2023-10-11T09:52:24.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.631+0000 7f0731a3b840 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-10-11T09:52:24.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.631+0000 7f0731a3b840 10 datascan.scan_links: step 1: handling object 200.00000001 2023-10-11T09:52:24.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.631+0000 7f0731a3b840 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-10-11T09:52:24.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.631+0000 7f0731a3b840 10 datascan.scan_links: step 1: handling object 606.00000000 2023-10-11T09:52:24.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.631+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:38 17.14 17:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875c00 con 0x5649236e1400 2023-10-11T09:52:24.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.631+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.3 v2:172.21.15.78:6824/35738 12 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5649241d9200 con 0x5649236e1400 2023-10-11T09:52:24.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.631+0000 7f0731a3b840 10 datascan.scan_links: step 1: handling object 607.00000000 2023-10-11T09:52:24.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.631+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:39 17.1c 17:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x564923844800 2023-10-11T09:52:24.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.631+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 <== osd.1 v2:172.21.15.78:6809/35734 4 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5649241d8fc0 con 0x564923844800 2023-10-11T09:52:24.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.631+0000 7f0731a3b840 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2023-10-11T09:52:24.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.631+0000 7f0731a3b840 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2023-10-11T09:52:24.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.631+0000 7f0731a3b840 10 datascan.scan_links: step 1: handling object 608.00000000 2023-10-11T09:52:24.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.631+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:40 17.e 17:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875000 con 0x564923874000 2023-10-11T09:52:24.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.631+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.2 v2:172.21.15.78:6808/35732 2 ==== osd_op_reply(40 608.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5649241d9200 con 0x564923874000 2023-10-11T09:52:24.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.631+0000 7f0731a3b840 10 datascan.scan_links: step 1: handling object 10000000000.01000000 2023-10-11T09:52:24.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.631+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:41 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649236e1800 con 0x564923845000 2023-10-11T09:52:24.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 5 ==== osd_op_reply(41 10000000000.01000000 [omap-get-vals out=25448b] v0'0 uv56 ondisk = 0) v8 ==== 164+0+25448 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 10 datascan.scan_links: step 1: handling object 604.00000000 2023-10-11T09:52:24.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:42 17.15 17:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875400 con 0x564923845c00 2023-10-11T09:52:24.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 <== osd.5 v2:172.21.15.133:6816/35899 4 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5649241d8fc0 con 0x564923845c00 2023-10-11T09:52:24.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 10 datascan.scan_links: step 1: handling object 500.00000000 2023-10-11T09:52:24.655 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-10-11T09:52:24.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 10 datascan.scan_links: step 1: handling object mds_snaptable 2023-10-11T09:52:24.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-10-11T09:52:24.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 10 datascan.scan_links: step 1: handling object 605.00000000 2023-10-11T09:52:24.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:43 17.1d 17:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875800 con 0x5649236e1000 2023-10-11T09:52:24.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 <== osd.7 v2:172.21.15.133:6824/35901 7 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5649241d8fc0 con 0x5649236e1000 2023-10-11T09:52:24.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 10 datascan.scan_links: step 1: handling object mds0_inotable 2023-10-11T09:52:24.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-10-11T09:52:24.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 10 datascan.scan_links: step 1: handling object 100.00000000 2023-10-11T09:52:24.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.635+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:44 17.13 17:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875c00 con 0x5649236e1400 2023-10-11T09:52:24.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.3 v2:172.21.15.78:6824/35738 13 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x5649241d9200 con 0x5649236e1400 2023-10-11T09:52:24.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2023-10-11T09:52:24.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-10-11T09:52:24.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 10 datascan.scan_links: step 1: handling object 10000000000.01800000 2023-10-11T09:52:24.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:45 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875000 con 0x564923844c00 2023-10-11T09:52:24.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 4 ==== osd_op_reply(45 10000000000.01800000 [omap-get-vals out=24452b] v0'0 uv55 ondisk = 0) v8 ==== 164+0+24452 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.666 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.668 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.669 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.670 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.671 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 10 datascan.scan_links: step 1: handling object 609.00000000 2023-10-11T09:52:24.672 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.639+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:46 17.7 17:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8000 con 0x564923844400 2023-10-11T09:52:24.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.643+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.0 v2:172.21.15.78:6800/35735 7 ==== osd_op_reply(46 609.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5649241d9440 con 0x564923844400 2023-10-11T09:52:24.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.643+0000 7f0731a3b840 10 datascan.scan_links: step 1: handling object 400.00000000 2023-10-11T09:52:24.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.643+0000 7f0731a3b840 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-10-11T09:52:24.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.643+0000 7f0731a3b840 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2023-10-11T09:52:24.673 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.643+0000 7f0731a3b840 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-10-11T09:52:24.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.643+0000 7f0731a3b840 10 datascan.scan_links: step 1: handling object 1.00000000 2023-10-11T09:52:24.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.643+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:47 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8400 con 0x564923845000 2023-10-11T09:52:24.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.643+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 6 ==== osd_op_reply(47 1.00000000 [omap-get-vals out=532b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+532 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.643+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:48 17.0 17:00000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8800 con 0x5649236e1000 2023-10-11T09:52:24.674 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.643+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 <== osd.7 v2:172.21.15.133:6824/35901 8 ==== osd_op_reply(48 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649241d8fc0 con 0x5649236e1000 2023-10-11T09:52:24.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.643+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:49 17.10 17:08000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x5649236e1400 2023-10-11T09:52:24.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.643+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.3 v2:172.21.15.78:6824/35738 14 ==== osd_op_reply(49 [pgnls start_epoch 129 out=177b] v128'12 uv12 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x5649241d9200 con 0x5649236e1400 2023-10-11T09:52:24.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.643+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:50 17.8 17:10000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x5649236e1000 2023-10-11T09:52:24.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.647+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 <== osd.7 v2:172.21.15.133:6824/35901 9 ==== osd_op_reply(50 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649241d8fc0 con 0x5649236e1000 2023-10-11T09:52:24.675 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.647+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:51 17.18 17:18000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x564923844400 2023-10-11T09:52:24.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.647+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.0 v2:172.21.15.78:6800/35735 8 ==== osd_op_reply(51 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649241d9440 con 0x564923844400 2023-10-11T09:52:24.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.647+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:52 17.4 17:20000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x564923844400 2023-10-11T09:52:24.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.647+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.0 v2:172.21.15.78:6800/35735 9 ==== osd_op_reply(52 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649241d9440 con 0x564923844400 2023-10-11T09:52:24.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.647+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:53 17.14 17:28000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x5649236e1400 2023-10-11T09:52:24.676 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.647+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.3 v2:172.21.15.78:6824/35738 15 ==== osd_op_reply(53 [pgnls start_epoch 129 out=149b] v128'28 uv28 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x5649241d9200 con 0x5649236e1400 2023-10-11T09:52:24.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.647+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:54 17.c 17:30000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x5649236e1000 2023-10-11T09:52:24.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.651+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 <== osd.7 v2:172.21.15.133:6824/35901 10 ==== osd_op_reply(54 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649241d8fc0 con 0x5649236e1000 2023-10-11T09:52:24.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.651+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:55 17.1c 17:38000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x564923844800 2023-10-11T09:52:24.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.651+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 <== osd.1 v2:172.21.15.78:6809/35734 5 ==== osd_op_reply(55 [pgnls start_epoch 129 out=73b] v126'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5649241d8fc0 con 0x564923844800 2023-10-11T09:52:24.677 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.651+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:56 17.2 17:40000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x564923844c00 2023-10-11T09:52:24.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.651+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 5 ==== osd_op_reply(56 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.651+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:57 17.12 17:48000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x5649236e1400 2023-10-11T09:52:24.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.651+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.3 v2:172.21.15.78:6824/35738 16 ==== osd_op_reply(57 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649241d9200 con 0x5649236e1400 2023-10-11T09:52:24.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.651+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:58 17.a 17:50000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x5649236e1000 2023-10-11T09:52:24.678 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.655+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 <== osd.7 v2:172.21.15.133:6824/35901 11 ==== osd_op_reply(58 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649241d8fc0 con 0x5649236e1000 2023-10-11T09:52:24.679 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.655+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:59 17.1a 17:58000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x564923844400 2023-10-11T09:52:24.681 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.655+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.0 v2:172.21.15.78:6800/35735 10 ==== osd_op_reply(59 [pgnls start_epoch 129 out=49b] v128'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649241d9440 con 0x564923844400 2023-10-11T09:52:24.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.655+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:60 17.6 17:60000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x564923845000 2023-10-11T09:52:24.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.655+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 7 ==== osd_op_reply(60 [pgnls start_epoch 129 out=77b] v128'8 uv8 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.655+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:61 17.16 17:68000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x564923845c00 2023-10-11T09:52:24.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.655+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 <== osd.5 v2:172.21.15.133:6816/35899 5 ==== osd_op_reply(61 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649241d8fc0 con 0x564923845c00 2023-10-11T09:52:24.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.655+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:62 17.e 17:70000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x564923874000 2023-10-11T09:52:24.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.659+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.2 v2:172.21.15.78:6808/35732 3 ==== osd_op_reply(62 [pgnls start_epoch 129 out=73b] v126'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5649241d9200 con 0x564923874000 2023-10-11T09:52:24.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.659+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:63 17.1e 17:78000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x564923844800 2023-10-11T09:52:24.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.659+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 <== osd.1 v2:172.21.15.78:6809/35734 6 ==== osd_op_reply(63 [pgnls start_epoch 129 out=49b] v128'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649241d8fc0 con 0x564923844800 2023-10-11T09:52:24.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.659+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:64 17.1 17:80000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x564923844400 2023-10-11T09:52:24.683 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.659+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.0 v2:172.21.15.78:6800/35735 11 ==== osd_op_reply(64 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649241d9440 con 0x564923844400 2023-10-11T09:52:24.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.659+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:65 17.11 17:88000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x5649236e1400 2023-10-11T09:52:24.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.659+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.3 v2:172.21.15.78:6824/35738 17 ==== osd_op_reply(65 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649241d9200 con 0x5649236e1400 2023-10-11T09:52:24.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.659+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:66 17.9 17:90000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x5649236e1400 2023-10-11T09:52:24.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.659+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.3 v2:172.21.15.78:6824/35738 18 ==== osd_op_reply(66 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649241d9200 con 0x5649236e1400 2023-10-11T09:52:24.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.659+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:67 17.19 17:98000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x564923845c00 2023-10-11T09:52:24.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.663+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 <== osd.5 v2:172.21.15.133:6816/35899 6 ==== osd_op_reply(67 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649241d8fc0 con 0x564923845c00 2023-10-11T09:52:24.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.663+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:68 17.5 17:a0000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x564923845000 2023-10-11T09:52:24.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.663+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 8 ==== osd_op_reply(68 [pgnls start_epoch 129 out=81b] v129'56 uv56 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.663+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:69 17.15 17:a8000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x564923845c00 2023-10-11T09:52:24.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.663+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 <== osd.5 v2:172.21.15.133:6816/35899 7 ==== osd_op_reply(69 [pgnls start_epoch 129 out=97b] v126'3 uv3 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x5649241d8fc0 con 0x564923845c00 2023-10-11T09:52:24.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.663+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:70 17.d 17:b0000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x5649236e1000 2023-10-11T09:52:24.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.663+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 <== osd.7 v2:172.21.15.133:6824/35901 12 ==== osd_op_reply(70 [pgnls start_epoch 129 out=74b] v126'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x5649241d8fc0 con 0x5649236e1000 2023-10-11T09:52:24.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.663+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:71 17.1d 17:b8000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x5649236e1000 2023-10-11T09:52:24.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.667+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 <== osd.7 v2:172.21.15.133:6824/35901 13 ==== osd_op_reply(71 [pgnls start_epoch 129 out=73b] v126'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5649241d8fc0 con 0x5649236e1000 2023-10-11T09:52:24.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.667+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:72 17.3 17:c0000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x564923844c00 2023-10-11T09:52:24.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.667+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 6 ==== osd_op_reply(72 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.667+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:73 17.13 17:c8000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x5649236e1400 2023-10-11T09:52:24.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.667+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.3 v2:172.21.15.78:6824/35738 19 ==== osd_op_reply(73 [pgnls start_epoch 129 out=98b] v127'3 uv3 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x5649241d9200 con 0x5649236e1400 2023-10-11T09:52:24.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.667+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:74 17.b 17:d0000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x564923844400 2023-10-11T09:52:24.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.667+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.0 v2:172.21.15.78:6800/35735 12 ==== osd_op_reply(74 [pgnls start_epoch 129 out=76b] v128'4 uv3 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x5649241d9440 con 0x564923844400 2023-10-11T09:52:24.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.667+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:75 17.1b 17:d8000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x564923844c00 2023-10-11T09:52:24.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.671+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 7 ==== osd_op_reply(75 [pgnls start_epoch 129 out=81b] v129'55 uv55 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.671+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:76 17.7 17:e0000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x564923844400 2023-10-11T09:52:24.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.671+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.0 v2:172.21.15.78:6800/35735 13 ==== osd_op_reply(76 [pgnls start_epoch 129 out=73b] v126'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5649241d9440 con 0x564923844400 2023-10-11T09:52:24.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.671+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:77 17.17 17:e8000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x564923845000 2023-10-11T09:52:24.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.671+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 9 ==== osd_op_reply(77 [pgnls start_epoch 129 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.671+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:78 17.f 17:f0000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x564923844800 2023-10-11T09:52:24.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.671+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 <== osd.1 v2:172.21.15.78:6809/35734 7 ==== osd_op_reply(78 [pgnls start_epoch 129 out=73b] v126'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5649241d8fc0 con 0x564923844800 2023-10-11T09:52:24.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.671+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:79 17.1f 17:f8000000::::head [pgnls start_epoch 129 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x564923845000 2023-10-11T09:52:24.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.675+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 10 ==== osd_op_reply(79 [pgnls start_epoch 129 out=101b] v128'7 uv6 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.675+0000 7f0731a3b840 10 datascan.scan_links: step 2: handling object 601.00000000 2023-10-11T09:52:24.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.675+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:80 17.10 17:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8c00 con 0x5649236e1400 2023-10-11T09:52:24.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.675+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.3 v2:172.21.15.78:6824/35738 20 ==== osd_op_reply(80 601.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5649241d9200 con 0x5649236e1400 2023-10-11T09:52:24.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.675+0000 7f0731a3b840 10 datascan.scan_links: step 2: handling object 602.00000000 2023-10-11T09:52:24.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.675+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:81 17.10 17:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9000 con 0x5649236e1400 2023-10-11T09:52:24.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.675+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.3 v2:172.21.15.78:6824/35738 21 ==== osd_op_reply(81 602.00000000 [omap-get-vals out=5b] v0'0 uv6 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5649241d9200 con 0x5649236e1400 2023-10-11T09:52:24.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.675+0000 7f0731a3b840 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2023-10-11T09:52:24.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.675+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:82 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9400 con 0x5649236e1400 2023-10-11T09:52:24.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.679+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.3 v2:172.21.15.78:6824/35738 22 ==== osd_op_reply(82 10000000000.00000000 [omap-get-vals out=50980b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+50980 (crc 0 0 0) 0x5649241d9200 con 0x5649236e1400 2023-10-11T09:52:24.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.679+0000 7f0731a3b840 10 datascan.scan_links: step 2: handling object 600.00000000 2023-10-11T09:52:24.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.679+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:83 17.10 17:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9800 con 0x5649236e1400 2023-10-11T09:52:24.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.679+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.3 v2:172.21.15.78:6824/35738 23 ==== osd_op_reply(83 600.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5649241d9200 con 0x5649236e1400 2023-10-11T09:52:24.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.679+0000 7f0731a3b840 10 datascan.scan_links: step 2: handling object 603.00000000 2023-10-11T09:52:24.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.679+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:84 17.10 17:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9c00 con 0x5649236e1400 2023-10-11T09:52:24.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.679+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.3 v2:172.21.15.78:6824/35738 24 ==== osd_op_reply(84 603.00000000 [omap-get-vals out=5b] v0'0 uv8 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5649241d9200 con 0x5649236e1400 2023-10-11T09:52:24.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.679+0000 7f0731a3b840 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2023-10-11T09:52:24.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.679+0000 7f0731a3b840 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-10-11T09:52:24.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.679+0000 7f0731a3b840 10 datascan.scan_links: step 2: handling object 200.00000000 2023-10-11T09:52:24.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.679+0000 7f0731a3b840 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-10-11T09:52:24.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.679+0000 7f0731a3b840 10 datascan.scan_links: step 2: handling object 200.00000001 2023-10-11T09:52:24.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.679+0000 7f0731a3b840 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-10-11T09:52:24.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.679+0000 7f0731a3b840 10 datascan.scan_links: step 2: handling object 606.00000000 2023-10-11T09:52:24.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.679+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:85 17.14 17:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8c00 con 0x5649236e1400 2023-10-11T09:52:24.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.679+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.3 v2:172.21.15.78:6824/35738 25 ==== osd_op_reply(85 606.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5649241d9200 con 0x5649236e1400 2023-10-11T09:52:24.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.679+0000 7f0731a3b840 10 datascan.scan_links: step 2: handling object 607.00000000 2023-10-11T09:52:24.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.679+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- osd_op(unknown.0.0:86 17.1c 17:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9000 con 0x564923844800 2023-10-11T09:52:24.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 <== osd.1 v2:172.21.15.78:6809/35734 8 ==== osd_op_reply(86 607.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5649241d8fc0 con 0x564923844800 2023-10-11T09:52:24.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2023-10-11T09:52:24.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2023-10-11T09:52:24.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 10 datascan.scan_links: step 2: handling object 608.00000000 2023-10-11T09:52:24.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- osd_op(unknown.0.0:87 17.e 17:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9400 con 0x564923874000 2023-10-11T09:52:24.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.2 v2:172.21.15.78:6808/35732 4 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5649241d9200 con 0x564923874000 2023-10-11T09:52:24.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 10 datascan.scan_links: step 2: handling object 10000000000.01000000 2023-10-11T09:52:24.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:88 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8800 con 0x564923845000 2023-10-11T09:52:24.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 11 ==== osd_op_reply(88 10000000000.01000000 [omap-get-vals out=25448b] v0'0 uv56 ondisk = 0) v8 ==== 164+0+25448 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x100000001ff for future processing to fix dnfirst 2023-10-11T09:52:24.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000200 for future processing to fix dnfirst 2023-10-11T09:52:24.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000201 for future processing to fix dnfirst 2023-10-11T09:52:24.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000204 for future processing to fix dnfirst 2023-10-11T09:52:24.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000208 for future processing to fix dnfirst 2023-10-11T09:52:24.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x100000001f6 for future processing to fix dnfirst 2023-10-11T09:52:24.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000209 for future processing to fix dnfirst 2023-10-11T09:52:24.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x1000000020a for future processing to fix dnfirst 2023-10-11T09:52:24.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000210 for future processing to fix dnfirst 2023-10-11T09:52:24.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000211 for future processing to fix dnfirst 2023-10-11T09:52:24.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2023-10-11T09:52:24.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000214 for future processing to fix dnfirst 2023-10-11T09:52:24.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000217 for future processing to fix dnfirst 2023-10-11T09:52:24.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000219 for future processing to fix dnfirst 2023-10-11T09:52:24.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x1000000021b for future processing to fix dnfirst 2023-10-11T09:52:24.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x1000000021c for future processing to fix dnfirst 2023-10-11T09:52:24.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x100000001f8 for future processing to fix dnfirst 2023-10-11T09:52:24.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x1000000021f for future processing to fix dnfirst 2023-10-11T09:52:24.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000220 for future processing to fix dnfirst 2023-10-11T09:52:24.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000223 for future processing to fix dnfirst 2023-10-11T09:52:24.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x100000001f9 for future processing to fix dnfirst 2023-10-11T09:52:24.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000227 for future processing to fix dnfirst 2023-10-11T09:52:24.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000229 for future processing to fix dnfirst 2023-10-11T09:52:24.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x1000000022a for future processing to fix dnfirst 2023-10-11T09:52:24.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x1000000022c for future processing to fix dnfirst 2023-10-11T09:52:24.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000230 for future processing to fix dnfirst 2023-10-11T09:52:24.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000232 for future processing to fix dnfirst 2023-10-11T09:52:24.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000234 for future processing to fix dnfirst 2023-10-11T09:52:24.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000235 for future processing to fix dnfirst 2023-10-11T09:52:24.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000236 for future processing to fix dnfirst 2023-10-11T09:52:24.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000237 for future processing to fix dnfirst 2023-10-11T09:52:24.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x1000000023a for future processing to fix dnfirst 2023-10-11T09:52:24.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x1000000023b for future processing to fix dnfirst 2023-10-11T09:52:24.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x1000000023e for future processing to fix dnfirst 2023-10-11T09:52:24.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x1000000023f for future processing to fix dnfirst 2023-10-11T09:52:24.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000240 for future processing to fix dnfirst 2023-10-11T09:52:24.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000242 for future processing to fix dnfirst 2023-10-11T09:52:24.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000245 for future processing to fix dnfirst 2023-10-11T09:52:24.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000248 for future processing to fix dnfirst 2023-10-11T09:52:24.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000249 for future processing to fix dnfirst 2023-10-11T09:52:24.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x1000000024a for future processing to fix dnfirst 2023-10-11T09:52:24.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x1000000024b for future processing to fix dnfirst 2023-10-11T09:52:24.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x1000000024c for future processing to fix dnfirst 2023-10-11T09:52:24.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x1000000024d for future processing to fix dnfirst 2023-10-11T09:52:24.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x100000001fd for future processing to fix dnfirst 2023-10-11T09:52:24.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x1000000024f for future processing to fix dnfirst 2023-10-11T09:52:24.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000251 for future processing to fix dnfirst 2023-10-11T09:52:24.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000254 for future processing to fix dnfirst 2023-10-11T09:52:24.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000255 for future processing to fix dnfirst 2023-10-11T09:52:24.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000256 for future processing to fix dnfirst 2023-10-11T09:52:24.718 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 20 datascan.scan_links: adding 0x100000001fe for future processing to fix dnfirst 2023-10-11T09:52:24.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 10 datascan.scan_links: step 2: handling object 604.00000000 2023-10-11T09:52:24.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.683+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- osd_op(unknown.0.0:89 17.15 17:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8000 con 0x564923845c00 2023-10-11T09:52:24.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.687+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 <== osd.5 v2:172.21.15.133:6816/35899 8 ==== osd_op_reply(89 604.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5649241d8fc0 con 0x564923845c00 2023-10-11T09:52:24.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.687+0000 7f0731a3b840 10 datascan.scan_links: step 2: handling object 500.00000000 2023-10-11T09:52:24.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.687+0000 7f0731a3b840 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-10-11T09:52:24.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.687+0000 7f0731a3b840 10 datascan.scan_links: step 2: handling object mds_snaptable 2023-10-11T09:52:24.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.687+0000 7f0731a3b840 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-10-11T09:52:24.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.687+0000 7f0731a3b840 10 datascan.scan_links: step 2: handling object 605.00000000 2023-10-11T09:52:24.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.687+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:90 17.1d 17:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649236e1800 con 0x5649236e1000 2023-10-11T09:52:24.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.687+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 <== osd.7 v2:172.21.15.133:6824/35901 14 ==== osd_op_reply(90 605.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5649241d8fc0 con 0x5649236e1000 2023-10-11T09:52:24.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.687+0000 7f0731a3b840 10 datascan.scan_links: step 2: handling object mds0_inotable 2023-10-11T09:52:24.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.687+0000 7f0731a3b840 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-10-11T09:52:24.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.687+0000 7f0731a3b840 10 datascan.scan_links: step 2: handling object 100.00000000 2023-10-11T09:52:24.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.687+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:91 17.13 17:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845800 con 0x5649236e1400 2023-10-11T09:52:24.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.687+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.3 v2:172.21.15.78:6824/35738 26 ==== osd_op_reply(91 100.00000000 [omap-get-vals out=5195b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x5649241d9200 con 0x5649236e1400 2023-10-11T09:52:24.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.687+0000 7f0731a3b840 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2023-10-11T09:52:24.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.687+0000 7f0731a3b840 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-10-11T09:52:24.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.687+0000 7f0731a3b840 10 datascan.scan_links: step 2: handling object 10000000000.01800000 2023-10-11T09:52:24.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.687+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:92 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845400 con 0x564923844c00 2023-10-11T09:52:24.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 8 ==== osd_op_reply(92 10000000000.01800000 [omap-get-vals out=24452b] v0'0 uv55 ondisk = 0) v8 ==== 164+0+24452 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2023-10-11T09:52:24.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000202 for future processing to fix dnfirst 2023-10-11T09:52:24.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000203 for future processing to fix dnfirst 2023-10-11T09:52:24.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000205 for future processing to fix dnfirst 2023-10-11T09:52:24.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000206 for future processing to fix dnfirst 2023-10-11T09:52:24.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000207 for future processing to fix dnfirst 2023-10-11T09:52:24.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x1000000020b for future processing to fix dnfirst 2023-10-11T09:52:24.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x1000000020c for future processing to fix dnfirst 2023-10-11T09:52:24.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x1000000020d for future processing to fix dnfirst 2023-10-11T09:52:24.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x1000000020e for future processing to fix dnfirst 2023-10-11T09:52:24.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x1000000020f for future processing to fix dnfirst 2023-10-11T09:52:24.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000212 for future processing to fix dnfirst 2023-10-11T09:52:24.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000213 for future processing to fix dnfirst 2023-10-11T09:52:24.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000215 for future processing to fix dnfirst 2023-10-11T09:52:24.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000216 for future processing to fix dnfirst 2023-10-11T09:52:24.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000218 for future processing to fix dnfirst 2023-10-11T09:52:24.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x1000000021a for future processing to fix dnfirst 2023-10-11T09:52:24.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x1000000021d for future processing to fix dnfirst 2023-10-11T09:52:24.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x1000000021e for future processing to fix dnfirst 2023-10-11T09:52:24.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000221 for future processing to fix dnfirst 2023-10-11T09:52:24.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000222 for future processing to fix dnfirst 2023-10-11T09:52:24.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000224 for future processing to fix dnfirst 2023-10-11T09:52:24.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000225 for future processing to fix dnfirst 2023-10-11T09:52:24.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000226 for future processing to fix dnfirst 2023-10-11T09:52:24.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000228 for future processing to fix dnfirst 2023-10-11T09:52:24.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x1000000022b for future processing to fix dnfirst 2023-10-11T09:52:24.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x1000000022d for future processing to fix dnfirst 2023-10-11T09:52:24.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x1000000022e for future processing to fix dnfirst 2023-10-11T09:52:24.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x1000000022f for future processing to fix dnfirst 2023-10-11T09:52:24.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x100000001fa for future processing to fix dnfirst 2023-10-11T09:52:24.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000231 for future processing to fix dnfirst 2023-10-11T09:52:24.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000233 for future processing to fix dnfirst 2023-10-11T09:52:24.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000238 for future processing to fix dnfirst 2023-10-11T09:52:24.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000239 for future processing to fix dnfirst 2023-10-11T09:52:24.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x100000001fb for future processing to fix dnfirst 2023-10-11T09:52:24.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x1000000023c for future processing to fix dnfirst 2023-10-11T09:52:24.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x1000000023d for future processing to fix dnfirst 2023-10-11T09:52:24.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000241 for future processing to fix dnfirst 2023-10-11T09:52:24.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.744 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000243 for future processing to fix dnfirst 2023-10-11T09:52:24.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000244 for future processing to fix dnfirst 2023-10-11T09:52:24.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x100000001fc for future processing to fix dnfirst 2023-10-11T09:52:24.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000246 for future processing to fix dnfirst 2023-10-11T09:52:24.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000247 for future processing to fix dnfirst 2023-10-11T09:52:24.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x1000000024e for future processing to fix dnfirst 2023-10-11T09:52:24.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000250 for future processing to fix dnfirst 2023-10-11T09:52:24.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000252 for future processing to fix dnfirst 2023-10-11T09:52:24.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000253 for future processing to fix dnfirst 2023-10-11T09:52:24.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000257 for future processing to fix dnfirst 2023-10-11T09:52:24.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: injected ino detected 2023-10-11T09:52:24.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: adding 0x10000000258 for future processing to fix dnfirst 2023-10-11T09:52:24.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: step 2: handling object 609.00000000 2023-10-11T09:52:24.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- osd_op(unknown.0.0:93 17.7 17:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875c00 con 0x564923844400 2023-10-11T09:52:24.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.0 v2:172.21.15.78:6800/35735 14 ==== osd_op_reply(93 609.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5649241d9440 con 0x564923844400 2023-10-11T09:52:24.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: step 2: handling object 400.00000000 2023-10-11T09:52:24.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-10-11T09:52:24.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2023-10-11T09:52:24.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-10-11T09:52:24.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: step 2: handling object 1.00000000 2023-10-11T09:52:24.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:94 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875000 con 0x564923845000 2023-10-11T09:52:24.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 12 ==== osd_op_reply(94 1.00000000 [omap-get-vals out=532b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+532 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: processing 99 dup_primaries, 0 remote_links 2023-10-11T09:52:24.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000001 2023-10-11T09:52:24.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x100000001f6 2023-10-11T09:52:24.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x100000001f7 2023-10-11T09:52:24.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x100000001f8 2023-10-11T09:52:24.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x100000001f9 2023-10-11T09:52:24.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x100000001fa 2023-10-11T09:52:24.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x100000001fb 2023-10-11T09:52:24.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x100000001fc 2023-10-11T09:52:24.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x100000001fd 2023-10-11T09:52:24.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x100000001fe 2023-10-11T09:52:24.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x100000001ff 2023-10-11T09:52:24.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000200 2023-10-11T09:52:24.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000201 2023-10-11T09:52:24.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000202 2023-10-11T09:52:24.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000203 2023-10-11T09:52:24.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000204 2023-10-11T09:52:24.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000205 2023-10-11T09:52:24.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000206 2023-10-11T09:52:24.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000207 2023-10-11T09:52:24.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000208 2023-10-11T09:52:24.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000209 2023-10-11T09:52:24.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x1000000020b 2023-10-11T09:52:24.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x1000000020c 2023-10-11T09:52:24.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x1000000020d 2023-10-11T09:52:24.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x1000000020e 2023-10-11T09:52:24.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x1000000020f 2023-10-11T09:52:24.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000210 2023-10-11T09:52:24.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000211 2023-10-11T09:52:24.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000212 2023-10-11T09:52:24.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000213 2023-10-11T09:52:24.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000214 2023-10-11T09:52:24.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000215 2023-10-11T09:52:24.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000216 2023-10-11T09:52:24.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000217 2023-10-11T09:52:24.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000218 2023-10-11T09:52:24.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000219 2023-10-11T09:52:24.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x1000000021a 2023-10-11T09:52:24.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x1000000021b 2023-10-11T09:52:24.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x1000000021c 2023-10-11T09:52:24.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x1000000021d 2023-10-11T09:52:24.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x1000000021e 2023-10-11T09:52:24.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x1000000021f 2023-10-11T09:52:24.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000220 2023-10-11T09:52:24.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000221 2023-10-11T09:52:24.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000222 2023-10-11T09:52:24.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000223 2023-10-11T09:52:24.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000224 2023-10-11T09:52:24.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000225 2023-10-11T09:52:24.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000226 2023-10-11T09:52:24.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000227 2023-10-11T09:52:24.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000228 2023-10-11T09:52:24.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000229 2023-10-11T09:52:24.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x1000000022a 2023-10-11T09:52:24.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x1000000022b 2023-10-11T09:52:24.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x1000000022c 2023-10-11T09:52:24.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x1000000022d 2023-10-11T09:52:24.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x1000000022e 2023-10-11T09:52:24.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x1000000022f 2023-10-11T09:52:24.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000230 2023-10-11T09:52:24.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000231 2023-10-11T09:52:24.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000232 2023-10-11T09:52:24.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000233 2023-10-11T09:52:24.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000234 2023-10-11T09:52:24.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000235 2023-10-11T09:52:24.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000236 2023-10-11T09:52:24.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000237 2023-10-11T09:52:24.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000238 2023-10-11T09:52:24.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000239 2023-10-11T09:52:24.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x1000000023a 2023-10-11T09:52:24.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x1000000023b 2023-10-11T09:52:24.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x1000000023c 2023-10-11T09:52:24.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x1000000023d 2023-10-11T09:52:24.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x1000000023e 2023-10-11T09:52:24.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x1000000023f 2023-10-11T09:52:24.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000240 2023-10-11T09:52:24.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000241 2023-10-11T09:52:24.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000242 2023-10-11T09:52:24.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000243 2023-10-11T09:52:24.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000244 2023-10-11T09:52:24.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000245 2023-10-11T09:52:24.765 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000246 2023-10-11T09:52:24.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000247 2023-10-11T09:52:24.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000248 2023-10-11T09:52:24.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000249 2023-10-11T09:52:24.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x1000000024a 2023-10-11T09:52:24.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x1000000024b 2023-10-11T09:52:24.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x1000000024c 2023-10-11T09:52:24.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x1000000024d 2023-10-11T09:52:24.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x1000000024e 2023-10-11T09:52:24.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x1000000024f 2023-10-11T09:52:24.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000250 2023-10-11T09:52:24.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000251 2023-10-11T09:52:24.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000252 2023-10-11T09:52:24.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000253 2023-10-11T09:52:24.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000254 2023-10-11T09:52:24.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000255 2023-10-11T09:52:24.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000256 2023-10-11T09:52:24.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000257 2023-10-11T09:52:24.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling dup 0x10000000258 2023-10-11T09:52:24.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: removing dup dentries from 0 objects 2023-10-11T09:52:24.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: processing 0 bad_nlink_inos 2023-10-11T09:52:24.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: processing 100 injected_inos 2023-10-11T09:52:24.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000001 2023-10-11T09:52:24.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:0 2023-10-11T09:52:24.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:95 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9800 con 0x564923844c00 2023-10-11T09:52:24.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 9 ==== osd_op_reply(95 10000000000.01800000 [omap-get-vals-by-keys out=502b] v0'0 uv55 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=0 frag=1*, r=0 2023-10-11T09:52:24.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.691+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.695+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:96 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9c00 con 0x564923844c00 2023-10-11T09:52:24.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.695+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 10 ==== osd_op_reply(96 10000000000.01800000 [omap-set-vals] v129'56 uv56 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.695+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/0 pointing to 0x0x10000000001 2023-10-11T09:52:24.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.695+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x100000001f6 2023-10-11T09:52:24.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.695+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:1 2023-10-11T09:52:24.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.695+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:97 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8c00 con 0x564923845000 2023-10-11T09:52:24.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.699+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 13 ==== osd_op_reply(97 10000000000.01000000 [omap-get-vals-by-keys out=502b] v0'0 uv56 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.699+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=1 frag=0*, r=0 2023-10-11T09:52:24.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.699+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.699+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:98 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9400 con 0x564923845000 2023-10-11T09:52:24.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.699+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 14 ==== osd_op_reply(98 10000000000.01000000 [omap-set-vals] v129'57 uv57 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.699+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/1 pointing to 0x0x100000001f6 2023-10-11T09:52:24.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.699+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x100000001f7 2023-10-11T09:52:24.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.699+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:2 2023-10-11T09:52:24.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.699+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:99 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9000 con 0x564923845000 2023-10-11T09:52:24.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.699+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 15 ==== osd_op_reply(99 10000000000.01000000 [omap-get-vals-by-keys out=502b] v0'0 uv57 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.703+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=2 frag=0*, r=0 2023-10-11T09:52:24.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.703+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.703+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:100 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8000 con 0x564923845000 2023-10-11T09:52:24.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.703+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 16 ==== osd_op_reply(100 10000000000.01000000 [omap-set-vals] v129'58 uv58 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.703+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/2 pointing to 0x0x100000001f7 2023-10-11T09:52:24.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.703+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x100000001f8 2023-10-11T09:52:24.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.703+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:3 2023-10-11T09:52:24.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.703+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:101 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874400 con 0x564923845000 2023-10-11T09:52:24.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.707+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 17 ==== osd_op_reply(101 10000000000.01000000 [omap-get-vals-by-keys out=502b] v0'0 uv58 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.707+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=3 frag=0*, r=0 2023-10-11T09:52:24.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.707+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.707+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:102 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874800 con 0x564923845000 2023-10-11T09:52:24.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.707+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 18 ==== osd_op_reply(102 10000000000.01000000 [omap-set-vals] v129'59 uv59 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.707+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/3 pointing to 0x0x100000001f8 2023-10-11T09:52:24.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.707+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x100000001f9 2023-10-11T09:52:24.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.707+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:4 2023-10-11T09:52:24.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.707+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:103 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875800 con 0x564923845000 2023-10-11T09:52:24.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.711+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 19 ==== osd_op_reply(103 10000000000.01000000 [omap-get-vals-by-keys out=502b] v0'0 uv59 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.711+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=4 frag=0*, r=0 2023-10-11T09:52:24.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.711+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.711+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:104 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875400 con 0x564923845000 2023-10-11T09:52:24.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.711+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 20 ==== osd_op_reply(104 10000000000.01000000 [omap-set-vals] v129'60 uv60 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.711+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/4 pointing to 0x0x100000001f9 2023-10-11T09:52:24.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.711+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x100000001fa 2023-10-11T09:52:24.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.711+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:5 2023-10-11T09:52:24.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.711+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:105 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845800 con 0x564923844c00 2023-10-11T09:52:24.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.715+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 11 ==== osd_op_reply(105 10000000000.01800000 [omap-get-vals-by-keys out=502b] v0'0 uv56 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.715+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=5 frag=1*, r=0 2023-10-11T09:52:24.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.715+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.715+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:106 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845400 con 0x564923844c00 2023-10-11T09:52:24.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.715+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 12 ==== osd_op_reply(106 10000000000.01800000 [omap-set-vals] v129'57 uv57 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.715+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/5 pointing to 0x0x100000001fa 2023-10-11T09:52:24.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.715+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x100000001fb 2023-10-11T09:52:24.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.715+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:6 2023-10-11T09:52:24.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.715+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:107 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9400 con 0x564923844c00 2023-10-11T09:52:24.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.719+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 13 ==== osd_op_reply(107 10000000000.01800000 [omap-get-vals-by-keys out=502b] v0'0 uv57 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.719+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=6 frag=1*, r=0 2023-10-11T09:52:24.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.719+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.719+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:108 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9000 con 0x564923844c00 2023-10-11T09:52:24.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.719+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 14 ==== osd_op_reply(108 10000000000.01800000 [omap-set-vals] v129'58 uv58 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.719+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/6 pointing to 0x0x100000001fb 2023-10-11T09:52:24.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.719+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x100000001fc 2023-10-11T09:52:24.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.719+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:7 2023-10-11T09:52:24.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.719+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:109 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8000 con 0x564923844c00 2023-10-11T09:52:24.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.719+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 15 ==== osd_op_reply(109 10000000000.01800000 [omap-get-vals-by-keys out=502b] v0'0 uv58 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.723+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=7 frag=1*, r=0 2023-10-11T09:52:24.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.723+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.723+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:110 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8400 con 0x564923844c00 2023-10-11T09:52:24.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.723+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 16 ==== osd_op_reply(110 10000000000.01800000 [omap-set-vals] v129'59 uv59 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.723+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/7 pointing to 0x0x100000001fc 2023-10-11T09:52:24.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.723+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x100000001fd 2023-10-11T09:52:24.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.723+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:8 2023-10-11T09:52:24.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.723+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:111 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8800 con 0x564923845000 2023-10-11T09:52:24.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.723+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 21 ==== osd_op_reply(111 10000000000.01000000 [omap-get-vals-by-keys out=502b] v0'0 uv60 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.723+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=8 frag=0*, r=0 2023-10-11T09:52:24.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.723+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.723+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:112 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8c00 con 0x564923845000 2023-10-11T09:52:24.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.727+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 22 ==== osd_op_reply(112 10000000000.01000000 [omap-set-vals] v129'61 uv61 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.727+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/8 pointing to 0x0x100000001fd 2023-10-11T09:52:24.793 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.727+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x100000001fe 2023-10-11T09:52:24.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.727+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:9 2023-10-11T09:52:24.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.727+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:113 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874400 con 0x564923845000 2023-10-11T09:52:24.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.727+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 23 ==== osd_op_reply(113 10000000000.01000000 [omap-get-vals-by-keys out=502b] v0'0 uv61 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.727+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=9 frag=0*, r=0 2023-10-11T09:52:24.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.727+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.727+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:114 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874800 con 0x564923845000 2023-10-11T09:52:24.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.731+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 24 ==== osd_op_reply(114 10000000000.01000000 [omap-set-vals] v129'62 uv62 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.731+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/9 pointing to 0x0x100000001fe 2023-10-11T09:52:24.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.731+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x100000001ff 2023-10-11T09:52:24.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.731+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:10 2023-10-11T09:52:24.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.731+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:115 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875800 con 0x564923845000 2023-10-11T09:52:24.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.731+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 25 ==== osd_op_reply(115 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv62 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.731+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=10 frag=0*, r=0 2023-10-11T09:52:24.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.731+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.731+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:116 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875400 con 0x564923845000 2023-10-11T09:52:24.796 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.735+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 26 ==== osd_op_reply(116 10000000000.01000000 [omap-set-vals] v129'63 uv63 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.735+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/10 pointing to 0x0x100000001ff 2023-10-11T09:52:24.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.735+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000200 2023-10-11T09:52:24.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.735+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:11 2023-10-11T09:52:24.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.735+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:117 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x564923845000 2023-10-11T09:52:24.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.735+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 27 ==== osd_op_reply(117 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv63 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.735+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=11 frag=0*, r=0 2023-10-11T09:52:24.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.735+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.735+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:118 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875c00 con 0x564923845000 2023-10-11T09:52:24.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.739+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 28 ==== osd_op_reply(118 10000000000.01000000 [omap-set-vals] v129'64 uv64 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.739+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/11 pointing to 0x0x10000000200 2023-10-11T09:52:24.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.739+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000201 2023-10-11T09:52:24.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.739+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:12 2023-10-11T09:52:24.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.739+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:119 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875000 con 0x564923845000 2023-10-11T09:52:24.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.739+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 29 ==== osd_op_reply(119 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv64 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.739+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=12 frag=0*, r=0 2023-10-11T09:52:24.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.739+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.739+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:120 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8800 con 0x564923845000 2023-10-11T09:52:24.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.743+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 30 ==== osd_op_reply(120 10000000000.01000000 [omap-set-vals] v129'65 uv65 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.743+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/12 pointing to 0x0x10000000201 2023-10-11T09:52:24.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.743+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000202 2023-10-11T09:52:24.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.743+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:13 2023-10-11T09:52:24.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.743+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:121 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8c00 con 0x564923844c00 2023-10-11T09:52:24.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.743+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 17 ==== osd_op_reply(121 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv59 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.743+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=13 frag=1*, r=0 2023-10-11T09:52:24.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.743+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.743+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:122 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9800 con 0x564923844c00 2023-10-11T09:52:24.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.747+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 18 ==== osd_op_reply(122 10000000000.01800000 [omap-set-vals] v129'60 uv60 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.747+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/13 pointing to 0x0x10000000202 2023-10-11T09:52:24.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.747+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000203 2023-10-11T09:52:24.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.747+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:14 2023-10-11T09:52:24.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.747+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:123 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9c00 con 0x564923844c00 2023-10-11T09:52:24.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.747+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 19 ==== osd_op_reply(123 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv60 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.747+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=14 frag=1*, r=0 2023-10-11T09:52:24.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.747+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.747+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:124 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9400 con 0x564923844c00 2023-10-11T09:52:24.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.751+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 20 ==== osd_op_reply(124 10000000000.01800000 [omap-set-vals] v129'61 uv61 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.751+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/14 pointing to 0x0x10000000203 2023-10-11T09:52:24.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.751+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000204 2023-10-11T09:52:24.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.751+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:15 2023-10-11T09:52:24.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.751+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:125 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9000 con 0x564923845000 2023-10-11T09:52:24.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.751+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 31 ==== osd_op_reply(125 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv65 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.751+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=15 frag=0*, r=0 2023-10-11T09:52:24.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.751+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.751+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:126 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845800 con 0x564923845000 2023-10-11T09:52:24.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.755+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 32 ==== osd_op_reply(126 10000000000.01000000 [omap-set-vals] v129'66 uv66 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.755+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/15 pointing to 0x0x10000000204 2023-10-11T09:52:24.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.755+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000205 2023-10-11T09:52:24.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.755+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:16 2023-10-11T09:52:24.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.755+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:127 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845400 con 0x564923844c00 2023-10-11T09:52:24.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.755+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 21 ==== osd_op_reply(127 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv61 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.755+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=16 frag=1*, r=0 2023-10-11T09:52:24.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.755+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.755+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:128 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874400 con 0x564923844c00 2023-10-11T09:52:24.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.759+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 22 ==== osd_op_reply(128 10000000000.01800000 [omap-set-vals] v129'62 uv62 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.759+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/16 pointing to 0x0x10000000205 2023-10-11T09:52:24.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.759+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000206 2023-10-11T09:52:24.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.759+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:17 2023-10-11T09:52:24.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.759+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:129 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874800 con 0x564923844c00 2023-10-11T09:52:24.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.759+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 23 ==== osd_op_reply(129 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv62 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.759+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=17 frag=1*, r=0 2023-10-11T09:52:24.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.759+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.759+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:130 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875800 con 0x564923844c00 2023-10-11T09:52:24.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.763+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 24 ==== osd_op_reply(130 10000000000.01800000 [omap-set-vals] v129'63 uv63 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.763+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/17 pointing to 0x0x10000000206 2023-10-11T09:52:24.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.763+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000207 2023-10-11T09:52:24.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.763+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:18 2023-10-11T09:52:24.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.763+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:131 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875400 con 0x564923844c00 2023-10-11T09:52:24.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.763+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 25 ==== osd_op_reply(131 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv63 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.763+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=18 frag=1*, r=0 2023-10-11T09:52:24.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.763+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.763+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:132 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x564923844c00 2023-10-11T09:52:24.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.767+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 26 ==== osd_op_reply(132 10000000000.01800000 [omap-set-vals] v129'64 uv64 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.767+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/18 pointing to 0x0x10000000207 2023-10-11T09:52:24.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.767+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000208 2023-10-11T09:52:24.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.767+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:19 2023-10-11T09:52:24.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.767+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:133 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875c00 con 0x564923845000 2023-10-11T09:52:24.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.767+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 33 ==== osd_op_reply(133 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv66 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.767+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=19 frag=0*, r=0 2023-10-11T09:52:24.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.767+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.767+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:134 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649236e1800 con 0x564923845000 2023-10-11T09:52:24.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.771+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 34 ==== osd_op_reply(134 10000000000.01000000 [omap-set-vals] v129'67 uv67 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.771+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/19 pointing to 0x0x10000000208 2023-10-11T09:52:24.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.771+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000209 2023-10-11T09:52:24.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.771+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:20 2023-10-11T09:52:24.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.771+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:135 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8400 con 0x564923845000 2023-10-11T09:52:24.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.771+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 35 ==== osd_op_reply(135 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv67 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.771+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=20 frag=0*, r=0 2023-10-11T09:52:24.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.771+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.771+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:136 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8c00 con 0x564923845000 2023-10-11T09:52:24.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.775+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 36 ==== osd_op_reply(136 10000000000.01000000 [omap-set-vals] v129'68 uv68 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.775+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/20 pointing to 0x0x10000000209 2023-10-11T09:52:24.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.775+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x1000000020a 2023-10-11T09:52:24.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.775+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:21 2023-10-11T09:52:24.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.775+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:137 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9800 con 0x564923845000 2023-10-11T09:52:24.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.775+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 37 ==== osd_op_reply(137 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv68 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.775+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=21 frag=0*, r=0 2023-10-11T09:52:24.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.775+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.775+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:138 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9c00 con 0x564923845000 2023-10-11T09:52:24.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.779+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 38 ==== osd_op_reply(138 10000000000.01000000 [omap-set-vals] v129'69 uv69 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.779+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/21 pointing to 0x0x1000000020a 2023-10-11T09:52:24.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.779+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x1000000020b 2023-10-11T09:52:24.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.779+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:22 2023-10-11T09:52:24.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.779+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:139 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9400 con 0x564923844c00 2023-10-11T09:52:24.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.779+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 27 ==== osd_op_reply(139 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv64 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.779+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=22 frag=1*, r=0 2023-10-11T09:52:24.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.779+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.779+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:140 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8000 con 0x564923844c00 2023-10-11T09:52:24.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.783+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 28 ==== osd_op_reply(140 10000000000.01800000 [omap-set-vals] v129'65 uv65 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.783+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/22 pointing to 0x0x1000000020b 2023-10-11T09:52:24.818 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.783+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x1000000020c 2023-10-11T09:52:24.818 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.783+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:23 2023-10-11T09:52:24.818 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.783+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:141 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874400 con 0x564923844c00 2023-10-11T09:52:24.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.783+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 29 ==== osd_op_reply(141 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv65 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.783+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=23 frag=1*, r=0 2023-10-11T09:52:24.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.783+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.783+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:142 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874800 con 0x564923844c00 2023-10-11T09:52:24.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.787+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 30 ==== osd_op_reply(142 10000000000.01800000 [omap-set-vals] v129'66 uv66 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.787+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/23 pointing to 0x0x1000000020c 2023-10-11T09:52:24.824 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.787+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x1000000020d 2023-10-11T09:52:24.824 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.787+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:24 2023-10-11T09:52:24.824 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.787+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:143 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845400 con 0x564923844c00 2023-10-11T09:52:24.824 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.787+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 31 ==== osd_op_reply(143 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv66 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.824 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.787+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=24 frag=1*, r=0 2023-10-11T09:52:24.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.787+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.787+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:144 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8800 con 0x564923844c00 2023-10-11T09:52:24.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.791+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 32 ==== osd_op_reply(144 10000000000.01800000 [omap-set-vals] v129'67 uv67 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.791+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/24 pointing to 0x0x1000000020d 2023-10-11T09:52:24.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.791+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x1000000020e 2023-10-11T09:52:24.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.791+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:25 2023-10-11T09:52:24.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.791+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:145 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9000 con 0x564923844c00 2023-10-11T09:52:24.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.791+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 33 ==== osd_op_reply(145 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv67 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.791+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=25 frag=1*, r=0 2023-10-11T09:52:24.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.791+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.791+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:146 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8400 con 0x564923844c00 2023-10-11T09:52:24.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.791+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 34 ==== osd_op_reply(146 10000000000.01800000 [omap-set-vals] v129'68 uv68 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.791+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/25 pointing to 0x0x1000000020e 2023-10-11T09:52:24.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.791+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x1000000020f 2023-10-11T09:52:24.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.791+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:26 2023-10-11T09:52:24.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.791+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:147 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8c00 con 0x564923844c00 2023-10-11T09:52:24.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.795+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 35 ==== osd_op_reply(147 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv68 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.795+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=26 frag=1*, r=0 2023-10-11T09:52:24.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.795+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.795+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:148 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9800 con 0x564923844c00 2023-10-11T09:52:24.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.795+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 36 ==== osd_op_reply(148 10000000000.01800000 [omap-set-vals] v129'69 uv69 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.795+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/26 pointing to 0x0x1000000020f 2023-10-11T09:52:24.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.795+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000210 2023-10-11T09:52:24.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.795+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:27 2023-10-11T09:52:24.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.795+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:149 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649236e1800 con 0x564923845000 2023-10-11T09:52:24.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.799+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 39 ==== osd_op_reply(149 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv69 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.799+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=27 frag=0*, r=0 2023-10-11T09:52:24.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.799+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.799+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:150 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875000 con 0x564923845000 2023-10-11T09:52:24.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.799+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 40 ==== osd_op_reply(150 10000000000.01000000 [omap-set-vals] v129'70 uv70 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.799+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/27 pointing to 0x0x10000000210 2023-10-11T09:52:24.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.799+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000211 2023-10-11T09:52:24.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.799+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:28 2023-10-11T09:52:24.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.799+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:151 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875c00 con 0x564923845000 2023-10-11T09:52:24.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.803+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 41 ==== osd_op_reply(151 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv70 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.803+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=28 frag=0*, r=0 2023-10-11T09:52:24.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.803+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.803+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:152 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845800 con 0x564923845000 2023-10-11T09:52:24.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.803+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 42 ==== osd_op_reply(152 10000000000.01000000 [omap-set-vals] v129'71 uv71 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.803+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/28 pointing to 0x0x10000000211 2023-10-11T09:52:24.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.803+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000212 2023-10-11T09:52:24.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.803+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:29 2023-10-11T09:52:24.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.803+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:153 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9400 con 0x564923844c00 2023-10-11T09:52:24.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.807+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 37 ==== osd_op_reply(153 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv69 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.807+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=29 frag=1*, r=0 2023-10-11T09:52:24.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.807+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.807+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:154 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8000 con 0x564923844c00 2023-10-11T09:52:24.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.807+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 38 ==== osd_op_reply(154 10000000000.01800000 [omap-set-vals] v129'70 uv70 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.807+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/29 pointing to 0x0x10000000212 2023-10-11T09:52:24.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.807+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000213 2023-10-11T09:52:24.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.807+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:30 2023-10-11T09:52:24.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.807+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:155 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8800 con 0x564923844c00 2023-10-11T09:52:24.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.811+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 39 ==== osd_op_reply(155 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv70 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.811+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=30 frag=1*, r=0 2023-10-11T09:52:24.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.811+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.811+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:156 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845400 con 0x564923844c00 2023-10-11T09:52:24.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.811+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 40 ==== osd_op_reply(156 10000000000.01800000 [omap-set-vals] v129'71 uv71 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.811+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/30 pointing to 0x0x10000000213 2023-10-11T09:52:24.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.811+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000214 2023-10-11T09:52:24.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.811+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:31 2023-10-11T09:52:24.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.811+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:157 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875800 con 0x564923845000 2023-10-11T09:52:24.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.815+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 43 ==== osd_op_reply(157 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv71 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.815+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=31 frag=0*, r=0 2023-10-11T09:52:24.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.815+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.815+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:158 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875400 con 0x564923845000 2023-10-11T09:52:24.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.815+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 44 ==== osd_op_reply(158 10000000000.01000000 [omap-set-vals] v129'72 uv72 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.815+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/31 pointing to 0x0x10000000214 2023-10-11T09:52:24.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.815+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000215 2023-10-11T09:52:24.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.815+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:32 2023-10-11T09:52:24.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.815+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:159 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x564923844c00 2023-10-11T09:52:24.840 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.815+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 41 ==== osd_op_reply(159 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv71 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.815+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=32 frag=1*, r=0 2023-10-11T09:52:24.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.815+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.815+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:160 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874400 con 0x564923844c00 2023-10-11T09:52:24.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.819+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 42 ==== osd_op_reply(160 10000000000.01800000 [omap-set-vals] v129'72 uv72 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.841 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.819+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/32 pointing to 0x0x10000000215 2023-10-11T09:52:24.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.819+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000216 2023-10-11T09:52:24.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.819+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:33 2023-10-11T09:52:24.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.819+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:161 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874800 con 0x564923844c00 2023-10-11T09:52:24.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.819+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 43 ==== osd_op_reply(161 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv72 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.842 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.819+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=33 frag=1*, r=0 2023-10-11T09:52:24.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.819+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.819+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:162 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x564923844c00 2023-10-11T09:52:24.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.823+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 44 ==== osd_op_reply(162 10000000000.01800000 [omap-set-vals] v129'73 uv73 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.823+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/33 pointing to 0x0x10000000216 2023-10-11T09:52:24.843 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.823+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000217 2023-10-11T09:52:24.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.823+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:34 2023-10-11T09:52:24.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.823+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:163 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874400 con 0x564923845000 2023-10-11T09:52:24.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.823+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 45 ==== osd_op_reply(163 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv72 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.823+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=34 frag=0*, r=0 2023-10-11T09:52:24.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.823+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.823+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:164 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845400 con 0x564923845000 2023-10-11T09:52:24.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.827+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 46 ==== osd_op_reply(164 10000000000.01000000 [omap-set-vals] v129'73 uv73 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.827+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/34 pointing to 0x0x10000000217 2023-10-11T09:52:24.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.827+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000218 2023-10-11T09:52:24.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.827+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:35 2023-10-11T09:52:24.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.827+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:165 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9000 con 0x564923844c00 2023-10-11T09:52:24.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.827+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 45 ==== osd_op_reply(165 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv73 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.827+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=35 frag=1*, r=0 2023-10-11T09:52:24.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.827+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.827+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:166 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8400 con 0x564923844c00 2023-10-11T09:52:24.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.831+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 46 ==== osd_op_reply(166 10000000000.01800000 [omap-set-vals] v129'74 uv74 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.831+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/35 pointing to 0x0x10000000218 2023-10-11T09:52:24.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.831+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000219 2023-10-11T09:52:24.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.831+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:36 2023-10-11T09:52:24.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.831+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:167 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8c00 con 0x564923845000 2023-10-11T09:52:24.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.831+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 47 ==== osd_op_reply(167 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv73 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.831+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=36 frag=0*, r=0 2023-10-11T09:52:24.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.831+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.831+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:168 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9800 con 0x564923845000 2023-10-11T09:52:24.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.835+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 48 ==== osd_op_reply(168 10000000000.01000000 [omap-set-vals] v129'74 uv74 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.835+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/36 pointing to 0x0x10000000219 2023-10-11T09:52:24.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.835+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x1000000021a 2023-10-11T09:52:24.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.835+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:37 2023-10-11T09:52:24.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.835+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:169 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9400 con 0x564923844c00 2023-10-11T09:52:24.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.835+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 47 ==== osd_op_reply(169 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv74 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.835+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=37 frag=1*, r=0 2023-10-11T09:52:24.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.835+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.835+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:170 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8000 con 0x564923844c00 2023-10-11T09:52:24.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.839+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 48 ==== osd_op_reply(170 10000000000.01800000 [omap-set-vals] v129'75 uv75 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.839+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/37 pointing to 0x0x1000000021a 2023-10-11T09:52:24.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.839+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x1000000021b 2023-10-11T09:52:24.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.839+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:38 2023-10-11T09:52:24.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.839+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:171 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8800 con 0x564923845000 2023-10-11T09:52:24.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.839+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 49 ==== osd_op_reply(171 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv74 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.839+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=38 frag=0*, r=0 2023-10-11T09:52:24.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.839+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.839+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:172 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9c00 con 0x564923845000 2023-10-11T09:52:24.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.843+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 50 ==== osd_op_reply(172 10000000000.01000000 [omap-set-vals] v129'75 uv75 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.843+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/38 pointing to 0x0x1000000021b 2023-10-11T09:52:24.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.843+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x1000000021c 2023-10-11T09:52:24.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.843+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:39 2023-10-11T09:52:24.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.843+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:173 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8c00 con 0x564923845000 2023-10-11T09:52:24.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.843+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 51 ==== osd_op_reply(173 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv75 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.843+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=39 frag=0*, r=0 2023-10-11T09:52:24.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.843+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.843+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:174 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845800 con 0x564923845000 2023-10-11T09:52:24.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.843+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 52 ==== osd_op_reply(174 10000000000.01000000 [omap-set-vals] v129'76 uv76 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.843+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/39 pointing to 0x0x1000000021c 2023-10-11T09:52:24.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.843+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x1000000021d 2023-10-11T09:52:24.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.843+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:40 2023-10-11T09:52:24.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.843+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:175 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845400 con 0x564923844c00 2023-10-11T09:52:24.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.847+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 49 ==== osd_op_reply(175 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv75 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.847+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=40 frag=1*, r=0 2023-10-11T09:52:24.857 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.847+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.847+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:176 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875000 con 0x564923844c00 2023-10-11T09:52:24.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.847+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 50 ==== osd_op_reply(176 10000000000.01800000 [omap-set-vals] v129'76 uv76 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.847+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/40 pointing to 0x0x1000000021d 2023-10-11T09:52:24.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.847+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x1000000021e 2023-10-11T09:52:24.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.847+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:41 2023-10-11T09:52:24.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.847+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:177 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875c00 con 0x564923844c00 2023-10-11T09:52:24.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.847+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 51 ==== osd_op_reply(177 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv76 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.847+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=41 frag=1*, r=0 2023-10-11T09:52:24.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.847+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.847+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:178 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875800 con 0x564923844c00 2023-10-11T09:52:24.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.851+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 52 ==== osd_op_reply(178 10000000000.01800000 [omap-set-vals] v129'77 uv77 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.851+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/41 pointing to 0x0x1000000021e 2023-10-11T09:52:24.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.851+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x1000000021f 2023-10-11T09:52:24.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.851+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:42 2023-10-11T09:52:24.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.851+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:179 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875400 con 0x564923845000 2023-10-11T09:52:24.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.851+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 53 ==== osd_op_reply(179 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv76 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.851+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=42 frag=0*, r=0 2023-10-11T09:52:24.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.851+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.851+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:180 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874400 con 0x564923845000 2023-10-11T09:52:24.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.855+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 54 ==== osd_op_reply(180 10000000000.01000000 [omap-set-vals] v129'77 uv77 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.855+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/42 pointing to 0x0x1000000021f 2023-10-11T09:52:24.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.855+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000220 2023-10-11T09:52:24.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.855+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:43 2023-10-11T09:52:24.863 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.855+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:181 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649236e1800 con 0x564923845000 2023-10-11T09:52:24.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.855+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 55 ==== osd_op_reply(181 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv77 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.855+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=43 frag=0*, r=0 2023-10-11T09:52:24.864 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.855+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.855+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:182 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923930000 con 0x564923845000 2023-10-11T09:52:24.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.859+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 56 ==== osd_op_reply(182 10000000000.01000000 [omap-set-vals] v129'78 uv78 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.859+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/43 pointing to 0x0x10000000220 2023-10-11T09:52:24.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.859+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000221 2023-10-11T09:52:24.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.859+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:44 2023-10-11T09:52:24.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.859+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:183 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923930400 con 0x564923844c00 2023-10-11T09:52:24.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.859+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 53 ==== osd_op_reply(183 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv77 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.859+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=44 frag=1*, r=0 2023-10-11T09:52:24.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.859+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.859+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:184 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923930800 con 0x564923844c00 2023-10-11T09:52:24.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.863+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 54 ==== osd_op_reply(184 10000000000.01800000 [omap-set-vals] v129'78 uv78 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.863+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/44 pointing to 0x0x10000000221 2023-10-11T09:52:24.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.863+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000222 2023-10-11T09:52:24.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.863+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:45 2023-10-11T09:52:24.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.863+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:185 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923930c00 con 0x564923844c00 2023-10-11T09:52:24.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.863+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 55 ==== osd_op_reply(185 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv78 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.863+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=45 frag=1*, r=0 2023-10-11T09:52:24.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.863+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.872 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.863+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:186 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923931000 con 0x564923844c00 2023-10-11T09:52:24.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.867+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 56 ==== osd_op_reply(186 10000000000.01800000 [omap-set-vals] v129'79 uv79 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.867+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/45 pointing to 0x0x10000000222 2023-10-11T09:52:24.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.867+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000223 2023-10-11T09:52:24.874 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.867+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:46 2023-10-11T09:52:24.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.867+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:187 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923931400 con 0x564923845000 2023-10-11T09:52:24.875 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.867+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 57 ==== osd_op_reply(187 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv78 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.867+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=46 frag=0*, r=0 2023-10-11T09:52:24.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.867+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.876 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.867+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:188 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923931800 con 0x564923845000 2023-10-11T09:52:24.877 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.871+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 58 ==== osd_op_reply(188 10000000000.01000000 [omap-set-vals] v129'79 uv79 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.871+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/46 pointing to 0x0x10000000223 2023-10-11T09:52:24.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.871+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000224 2023-10-11T09:52:24.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.871+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:47 2023-10-11T09:52:24.878 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.871+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:189 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923931c00 con 0x564923844c00 2023-10-11T09:52:24.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.871+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 57 ==== osd_op_reply(189 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv79 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.871+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=47 frag=1*, r=0 2023-10-11T09:52:24.879 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.871+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.880 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.871+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:190 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923930000 con 0x564923844c00 2023-10-11T09:52:24.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.875+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 58 ==== osd_op_reply(190 10000000000.01800000 [omap-set-vals] v129'80 uv80 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.875+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/47 pointing to 0x0x10000000224 2023-10-11T09:52:24.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.875+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000225 2023-10-11T09:52:24.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.875+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:48 2023-10-11T09:52:24.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.875+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:191 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923931400 con 0x564923844c00 2023-10-11T09:52:24.882 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.875+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 59 ==== osd_op_reply(191 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv80 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.875+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=48 frag=1*, r=0 2023-10-11T09:52:24.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.875+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.883 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.875+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:192 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923931800 con 0x564923844c00 2023-10-11T09:52:24.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.879+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 60 ==== osd_op_reply(192 10000000000.01800000 [omap-set-vals] v129'81 uv81 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.879+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/48 pointing to 0x0x10000000225 2023-10-11T09:52:24.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.879+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000226 2023-10-11T09:52:24.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.879+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:49 2023-10-11T09:52:24.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.879+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:193 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923930400 con 0x564923844c00 2023-10-11T09:52:24.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.879+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 61 ==== osd_op_reply(193 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv81 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.879+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=49 frag=1*, r=0 2023-10-11T09:52:24.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.879+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.879+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:194 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875400 con 0x564923844c00 2023-10-11T09:52:24.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.879+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 62 ==== osd_op_reply(194 10000000000.01800000 [omap-set-vals] v129'82 uv82 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.879+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/49 pointing to 0x0x10000000226 2023-10-11T09:52:24.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.879+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000227 2023-10-11T09:52:24.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.879+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:50 2023-10-11T09:52:24.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.879+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:195 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874400 con 0x564923845000 2023-10-11T09:52:24.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.883+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 59 ==== osd_op_reply(195 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv79 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.883+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=50 frag=0*, r=0 2023-10-11T09:52:24.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.883+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.883+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:196 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874800 con 0x564923845000 2023-10-11T09:52:24.893 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.883+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 60 ==== osd_op_reply(196 10000000000.01000000 [omap-set-vals] v129'80 uv80 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.883+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/50 pointing to 0x0x10000000227 2023-10-11T09:52:24.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.883+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000228 2023-10-11T09:52:24.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.883+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:51 2023-10-11T09:52:24.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.883+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:197 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x564923844c00 2023-10-11T09:52:24.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.887+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 63 ==== osd_op_reply(197 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv82 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.887+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=51 frag=1*, r=0 2023-10-11T09:52:24.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.887+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.887+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:198 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875000 con 0x564923844c00 2023-10-11T09:52:24.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.887+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 64 ==== osd_op_reply(198 10000000000.01800000 [omap-set-vals] v129'83 uv83 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.887+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/51 pointing to 0x0x10000000228 2023-10-11T09:52:24.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.887+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000229 2023-10-11T09:52:24.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.887+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:52 2023-10-11T09:52:24.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.887+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:199 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875c00 con 0x564923845000 2023-10-11T09:52:24.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.887+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 61 ==== osd_op_reply(199 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv80 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.891+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=52 frag=0*, r=0 2023-10-11T09:52:24.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.891+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.891+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:200 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875800 con 0x564923845000 2023-10-11T09:52:24.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.891+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 62 ==== osd_op_reply(200 10000000000.01000000 [omap-set-vals] v129'81 uv81 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.891+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/52 pointing to 0x0x10000000229 2023-10-11T09:52:24.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.891+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x1000000022a 2023-10-11T09:52:24.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.891+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:53 2023-10-11T09:52:24.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.891+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:201 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845800 con 0x564923845000 2023-10-11T09:52:24.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.891+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 63 ==== osd_op_reply(201 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv81 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.891+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=53 frag=0*, r=0 2023-10-11T09:52:24.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.891+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.891+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:202 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845400 con 0x564923845000 2023-10-11T09:52:24.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.895+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 64 ==== osd_op_reply(202 10000000000.01000000 [omap-set-vals] v129'82 uv82 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.895+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/53 pointing to 0x0x1000000022a 2023-10-11T09:52:24.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.895+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x1000000022b 2023-10-11T09:52:24.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.895+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:54 2023-10-11T09:52:24.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.895+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:203 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649236e1800 con 0x564923844c00 2023-10-11T09:52:24.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.895+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 65 ==== osd_op_reply(203 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv83 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.895+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=54 frag=1*, r=0 2023-10-11T09:52:24.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.895+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.895+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:204 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649236e1800 con 0x564923844c00 2023-10-11T09:52:24.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.899+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 66 ==== osd_op_reply(204 10000000000.01800000 [omap-set-vals] v129'84 uv84 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.899+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/54 pointing to 0x0x1000000022b 2023-10-11T09:52:24.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.899+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x1000000022c 2023-10-11T09:52:24.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.899+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:55 2023-10-11T09:52:24.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.899+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:205 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875400 con 0x564923845000 2023-10-11T09:52:24.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.899+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 65 ==== osd_op_reply(205 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv82 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.899+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=55 frag=0*, r=0 2023-10-11T09:52:24.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.899+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.899+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:206 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x564923845000 2023-10-11T09:52:24.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.903+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 66 ==== osd_op_reply(206 10000000000.01000000 [omap-set-vals] v129'83 uv83 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.903+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/55 pointing to 0x0x1000000022c 2023-10-11T09:52:24.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.903+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x1000000022d 2023-10-11T09:52:24.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.903+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:56 2023-10-11T09:52:24.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.903+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:207 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875000 con 0x564923844c00 2023-10-11T09:52:24.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.903+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 67 ==== osd_op_reply(207 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv84 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.903+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=56 frag=1*, r=0 2023-10-11T09:52:24.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.903+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.903+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:208 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923930000 con 0x564923844c00 2023-10-11T09:52:24.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.907+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 68 ==== osd_op_reply(208 10000000000.01800000 [omap-set-vals] v129'85 uv85 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.913 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.907+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/56 pointing to 0x0x1000000022d 2023-10-11T09:52:24.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.907+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x1000000022e 2023-10-11T09:52:24.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.907+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:57 2023-10-11T09:52:24.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.907+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:209 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923930400 con 0x564923844c00 2023-10-11T09:52:24.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.907+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 69 ==== osd_op_reply(209 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv85 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.907+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=57 frag=1*, r=0 2023-10-11T09:52:24.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.907+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.907+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:210 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923930800 con 0x564923844c00 2023-10-11T09:52:24.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.911+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 70 ==== osd_op_reply(210 10000000000.01800000 [omap-set-vals] v129'86 uv86 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.911+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/57 pointing to 0x0x1000000022e 2023-10-11T09:52:24.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.911+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x1000000022f 2023-10-11T09:52:24.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.911+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:58 2023-10-11T09:52:24.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.911+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:211 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923930c00 con 0x564923844c00 2023-10-11T09:52:24.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.911+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 71 ==== osd_op_reply(211 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv86 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.911+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=58 frag=1*, r=0 2023-10-11T09:52:24.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.911+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.911+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:212 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923931000 con 0x564923844c00 2023-10-11T09:52:24.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.915+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 72 ==== osd_op_reply(212 10000000000.01800000 [omap-set-vals] v129'87 uv87 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.915+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/58 pointing to 0x0x1000000022f 2023-10-11T09:52:24.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.915+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000230 2023-10-11T09:52:24.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.915+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:59 2023-10-11T09:52:24.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.915+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:213 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923931400 con 0x564923845000 2023-10-11T09:52:24.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.915+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 67 ==== osd_op_reply(213 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv83 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.915+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=59 frag=0*, r=0 2023-10-11T09:52:24.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.915+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.915+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:214 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923931800 con 0x564923845000 2023-10-11T09:52:24.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.915+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 68 ==== osd_op_reply(214 10000000000.01000000 [omap-set-vals] v129'84 uv84 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.919+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/59 pointing to 0x0x10000000230 2023-10-11T09:52:24.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.919+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000231 2023-10-11T09:52:24.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.919+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:60 2023-10-11T09:52:24.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.919+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:215 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923931c00 con 0x564923844c00 2023-10-11T09:52:24.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.919+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 73 ==== osd_op_reply(215 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv87 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.919+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=60 frag=1*, r=0 2023-10-11T09:52:24.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.919+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.919+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:216 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8000 con 0x564923844c00 2023-10-11T09:52:24.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.919+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 74 ==== osd_op_reply(216 10000000000.01800000 [omap-set-vals] v129'88 uv88 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.919+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/60 pointing to 0x0x10000000231 2023-10-11T09:52:24.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.919+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000232 2023-10-11T09:52:24.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.919+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:61 2023-10-11T09:52:24.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.919+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:217 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8400 con 0x564923845000 2023-10-11T09:52:24.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.919+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 69 ==== osd_op_reply(217 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv84 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.923+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=61 frag=0*, r=0 2023-10-11T09:52:24.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.923+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.923+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:218 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8800 con 0x564923845000 2023-10-11T09:52:24.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.923+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 70 ==== osd_op_reply(218 10000000000.01000000 [omap-set-vals] v129'85 uv85 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.923+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/61 pointing to 0x0x10000000232 2023-10-11T09:52:24.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.923+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000233 2023-10-11T09:52:24.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.923+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:62 2023-10-11T09:52:24.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.923+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:219 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8c00 con 0x564923844c00 2023-10-11T09:52:24.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.923+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 75 ==== osd_op_reply(219 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv88 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.923+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=62 frag=1*, r=0 2023-10-11T09:52:24.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.923+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.923+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:220 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9000 con 0x564923844c00 2023-10-11T09:52:24.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.927+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 76 ==== osd_op_reply(220 10000000000.01800000 [omap-set-vals] v129'89 uv89 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.927+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/62 pointing to 0x0x10000000233 2023-10-11T09:52:24.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.927+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000234 2023-10-11T09:52:24.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.927+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:63 2023-10-11T09:52:24.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.927+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:221 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9400 con 0x564923845000 2023-10-11T09:52:24.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.927+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 71 ==== osd_op_reply(221 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv85 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.927+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=63 frag=0*, r=0 2023-10-11T09:52:24.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.927+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.927+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:222 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9800 con 0x564923845000 2023-10-11T09:52:24.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.931+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 72 ==== osd_op_reply(222 10000000000.01000000 [omap-set-vals] v129'86 uv86 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.931+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/63 pointing to 0x0x10000000234 2023-10-11T09:52:24.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.931+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000235 2023-10-11T09:52:24.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.931+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:64 2023-10-11T09:52:24.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.931+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:223 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9c00 con 0x564923845000 2023-10-11T09:52:24.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.931+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 73 ==== osd_op_reply(223 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv86 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.931+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=64 frag=0*, r=0 2023-10-11T09:52:24.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.931+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.931+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:224 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8400 con 0x564923845000 2023-10-11T09:52:24.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.935+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 74 ==== osd_op_reply(224 10000000000.01000000 [omap-set-vals] v129'87 uv87 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.935+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/64 pointing to 0x0x10000000235 2023-10-11T09:52:24.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.935+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000236 2023-10-11T09:52:24.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.935+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:65 2023-10-11T09:52:24.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.935+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:225 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8800 con 0x564923845000 2023-10-11T09:52:24.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.935+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 75 ==== osd_op_reply(225 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv87 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.935+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=65 frag=0*, r=0 2023-10-11T09:52:24.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.935+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.935+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:226 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923931400 con 0x564923845000 2023-10-11T09:52:24.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.939+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 76 ==== osd_op_reply(226 10000000000.01000000 [omap-set-vals] v129'88 uv88 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.939+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/65 pointing to 0x0x10000000236 2023-10-11T09:52:24.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.939+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000237 2023-10-11T09:52:24.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.939+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:66 2023-10-11T09:52:24.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.939+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:227 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923931800 con 0x564923845000 2023-10-11T09:52:24.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.939+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 77 ==== osd_op_reply(227 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv88 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.939+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=66 frag=0*, r=0 2023-10-11T09:52:24.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.939+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.939+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:228 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874400 con 0x564923845000 2023-10-11T09:52:24.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.939+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 78 ==== osd_op_reply(228 10000000000.01000000 [omap-set-vals] v129'89 uv89 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.939+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/66 pointing to 0x0x10000000237 2023-10-11T09:52:24.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.939+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000238 2023-10-11T09:52:24.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.939+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:67 2023-10-11T09:52:24.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.939+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:229 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874800 con 0x564923844c00 2023-10-11T09:52:24.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.943+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 77 ==== osd_op_reply(229 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv89 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.943+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=67 frag=1*, r=0 2023-10-11T09:52:24.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.943+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.943+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:230 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875c00 con 0x564923844c00 2023-10-11T09:52:24.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.943+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 78 ==== osd_op_reply(230 10000000000.01800000 [omap-set-vals] v129'90 uv90 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.943+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/67 pointing to 0x0x10000000238 2023-10-11T09:52:24.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.943+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000239 2023-10-11T09:52:24.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.943+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:68 2023-10-11T09:52:24.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.943+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:231 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875800 con 0x564923844c00 2023-10-11T09:52:24.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.947+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 79 ==== osd_op_reply(231 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv90 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.947+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=68 frag=1*, r=0 2023-10-11T09:52:24.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.947+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.947+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:232 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875400 con 0x564923844c00 2023-10-11T09:52:24.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.947+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 80 ==== osd_op_reply(232 10000000000.01800000 [omap-set-vals] v129'91 uv91 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.947+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/68 pointing to 0x0x10000000239 2023-10-11T09:52:24.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.947+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x1000000023a 2023-10-11T09:52:24.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.947+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:69 2023-10-11T09:52:24.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.947+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:233 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x564923845000 2023-10-11T09:52:24.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.951+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 79 ==== osd_op_reply(233 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv89 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.951+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=69 frag=0*, r=0 2023-10-11T09:52:24.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.951+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.951+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:234 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845800 con 0x564923845000 2023-10-11T09:52:24.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.951+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 80 ==== osd_op_reply(234 10000000000.01000000 [omap-set-vals] v129'90 uv90 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.951+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/69 pointing to 0x0x1000000023a 2023-10-11T09:52:24.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.951+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x1000000023b 2023-10-11T09:52:24.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.951+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:70 2023-10-11T09:52:24.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.951+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:235 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845400 con 0x564923845000 2023-10-11T09:52:24.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.951+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 81 ==== osd_op_reply(235 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv90 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.951+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=70 frag=0*, r=0 2023-10-11T09:52:24.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.951+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.951+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:236 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875000 con 0x564923845000 2023-10-11T09:52:24.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.955+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 82 ==== osd_op_reply(236 10000000000.01000000 [omap-set-vals] v129'91 uv91 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.955+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/70 pointing to 0x0x1000000023b 2023-10-11T09:52:24.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.955+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x1000000023c 2023-10-11T09:52:24.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.955+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:71 2023-10-11T09:52:24.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.955+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:237 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874800 con 0x564923844c00 2023-10-11T09:52:24.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.955+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 81 ==== osd_op_reply(237 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv91 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.955+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=71 frag=1*, r=0 2023-10-11T09:52:24.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.955+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.955+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:238 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875c00 con 0x564923844c00 2023-10-11T09:52:24.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.959+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 82 ==== osd_op_reply(238 10000000000.01800000 [omap-set-vals] v129'92 uv92 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.959+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/71 pointing to 0x0x1000000023c 2023-10-11T09:52:24.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.959+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x1000000023d 2023-10-11T09:52:24.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.959+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:72 2023-10-11T09:52:24.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.959+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:239 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8000 con 0x564923844c00 2023-10-11T09:52:24.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.959+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 83 ==== osd_op_reply(239 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv92 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.968 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.959+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=72 frag=1*, r=0 2023-10-11T09:52:24.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.959+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.969 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.959+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:240 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8c00 con 0x564923844c00 2023-10-11T09:52:24.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.963+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 84 ==== osd_op_reply(240 10000000000.01800000 [omap-set-vals] v129'93 uv93 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.963+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/72 pointing to 0x0x1000000023d 2023-10-11T09:52:24.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.963+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x1000000023e 2023-10-11T09:52:24.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.963+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:73 2023-10-11T09:52:24.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.963+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:241 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9000 con 0x564923845000 2023-10-11T09:52:24.972 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.963+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 83 ==== osd_op_reply(241 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv91 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.963+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=73 frag=0*, r=0 2023-10-11T09:52:24.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.963+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.973 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.963+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:242 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923930000 con 0x564923845000 2023-10-11T09:52:24.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.967+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 84 ==== osd_op_reply(242 10000000000.01000000 [omap-set-vals] v129'92 uv92 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.967+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/73 pointing to 0x0x1000000023e 2023-10-11T09:52:24.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.967+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x1000000023f 2023-10-11T09:52:24.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.967+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:74 2023-10-11T09:52:24.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.967+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:243 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923930400 con 0x564923845000 2023-10-11T09:52:24.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.967+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 85 ==== osd_op_reply(243 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv92 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.967+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=74 frag=0*, r=0 2023-10-11T09:52:24.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.967+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.967+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:244 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923930800 con 0x564923845000 2023-10-11T09:52:24.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.971+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 86 ==== osd_op_reply(244 10000000000.01000000 [omap-set-vals] v129'93 uv93 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.971+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/74 pointing to 0x0x1000000023f 2023-10-11T09:52:24.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.971+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000240 2023-10-11T09:52:24.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.971+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:75 2023-10-11T09:52:24.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.971+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:245 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923930c00 con 0x564923845000 2023-10-11T09:52:24.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.971+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 87 ==== osd_op_reply(245 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv93 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.971+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=75 frag=0*, r=0 2023-10-11T09:52:24.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.971+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.971+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:246 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923931000 con 0x564923845000 2023-10-11T09:52:24.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.975+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 88 ==== osd_op_reply(246 10000000000.01000000 [omap-set-vals] v129'94 uv94 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.975+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/75 pointing to 0x0x10000000240 2023-10-11T09:52:24.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.975+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000241 2023-10-11T09:52:24.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.975+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:76 2023-10-11T09:52:24.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.975+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:247 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923931c00 con 0x564923844c00 2023-10-11T09:52:24.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.975+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 85 ==== osd_op_reply(247 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv93 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.975+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=76 frag=1*, r=0 2023-10-11T09:52:24.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.975+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.975+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:248 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649236e1800 con 0x564923844c00 2023-10-11T09:52:24.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.975+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 86 ==== osd_op_reply(248 10000000000.01800000 [omap-set-vals] v129'94 uv94 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.975+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/76 pointing to 0x0x10000000241 2023-10-11T09:52:24.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.975+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000242 2023-10-11T09:52:24.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.975+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:77 2023-10-11T09:52:24.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.975+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:249 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9400 con 0x564923845000 2023-10-11T09:52:24.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.979+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 89 ==== osd_op_reply(249 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv94 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.979+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=77 frag=0*, r=0 2023-10-11T09:52:24.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.979+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.979+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:250 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9800 con 0x564923845000 2023-10-11T09:52:24.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.979+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 90 ==== osd_op_reply(250 10000000000.01000000 [omap-set-vals] v129'95 uv95 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.979+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/77 pointing to 0x0x10000000242 2023-10-11T09:52:24.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.979+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000243 2023-10-11T09:52:24.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.979+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:78 2023-10-11T09:52:24.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.979+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:251 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9c00 con 0x564923844c00 2023-10-11T09:52:24.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.983+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 87 ==== osd_op_reply(251 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv94 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.983+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=78 frag=1*, r=0 2023-10-11T09:52:24.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.983+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.983+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:252 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8400 con 0x564923844c00 2023-10-11T09:52:24.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.983+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 88 ==== osd_op_reply(252 10000000000.01800000 [omap-set-vals] v129'95 uv95 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.983+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/78 pointing to 0x0x10000000243 2023-10-11T09:52:24.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.983+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000244 2023-10-11T09:52:24.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.983+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:79 2023-10-11T09:52:24.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.983+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:253 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8800 con 0x564923844c00 2023-10-11T09:52:24.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.983+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 89 ==== osd_op_reply(253 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv95 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.983+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=79 frag=1*, r=0 2023-10-11T09:52:24.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.983+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.983+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:254 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9000 con 0x564923844c00 2023-10-11T09:52:24.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.987+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 90 ==== osd_op_reply(254 10000000000.01800000 [omap-set-vals] v129'96 uv96 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:24.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.987+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/79 pointing to 0x0x10000000244 2023-10-11T09:52:24.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.987+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000245 2023-10-11T09:52:24.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.987+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:80 2023-10-11T09:52:24.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.987+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:255 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874400 con 0x564923845000 2023-10-11T09:52:24.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.987+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 91 ==== osd_op_reply(255 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv95 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.987+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=80 frag=0*, r=0 2023-10-11T09:52:24.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.987+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:24.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.987+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:256 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x564923845000 2023-10-11T09:52:24.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.991+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 92 ==== osd_op_reply(256 10000000000.01000000 [omap-set-vals] v129'96 uv96 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:24.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.991+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/80 pointing to 0x0x10000000245 2023-10-11T09:52:24.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.991+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000246 2023-10-11T09:52:24.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.991+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:81 2023-10-11T09:52:24.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.991+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:257 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875000 con 0x564923844c00 2023-10-11T09:52:25.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.991+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 91 ==== osd_op_reply(257 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv96 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:25.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.991+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=81 frag=1*, r=0 2023-10-11T09:52:25.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.991+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:25.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.991+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:258 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845800 con 0x564923844c00 2023-10-11T09:52:25.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.995+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 92 ==== osd_op_reply(258 10000000000.01800000 [omap-set-vals] v129'97 uv97 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:25.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.995+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/81 pointing to 0x0x10000000246 2023-10-11T09:52:25.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.995+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000247 2023-10-11T09:52:25.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.995+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:82 2023-10-11T09:52:25.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.995+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:259 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845400 con 0x564923844c00 2023-10-11T09:52:25.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.995+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 93 ==== osd_op_reply(259 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv97 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:25.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.995+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=82 frag=1*, r=0 2023-10-11T09:52:25.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.995+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:25.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.995+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:260 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923931400 con 0x564923844c00 2023-10-11T09:52:25.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.995+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 94 ==== osd_op_reply(260 10000000000.01800000 [omap-set-vals] v129'98 uv98 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:25.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.995+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/82 pointing to 0x0x10000000247 2023-10-11T09:52:25.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.995+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000248 2023-10-11T09:52:25.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.995+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:83 2023-10-11T09:52:25.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.995+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:261 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923931800 con 0x564923845000 2023-10-11T09:52:25.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.999+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 93 ==== osd_op_reply(261 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv96 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:25.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.999+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=83 frag=0*, r=0 2023-10-11T09:52:25.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.999+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:25.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.999+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:262 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845800 con 0x564923845000 2023-10-11T09:52:25.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.999+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 94 ==== osd_op_reply(262 10000000000.01000000 [omap-set-vals] v129'97 uv97 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:25.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.999+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/83 pointing to 0x0x10000000248 2023-10-11T09:52:25.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.999+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000249 2023-10-11T09:52:25.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.999+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:84 2023-10-11T09:52:25.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.999+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:263 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875800 con 0x564923845000 2023-10-11T09:52:25.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.999+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 95 ==== osd_op_reply(263 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv97 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:25.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.999+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=84 frag=0*, r=0 2023-10-11T09:52:25.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.999+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:25.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:24.999+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:264 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875400 con 0x564923845000 2023-10-11T09:52:25.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.003+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 96 ==== osd_op_reply(264 10000000000.01000000 [omap-set-vals] v129'98 uv98 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:25.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.003+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/84 pointing to 0x0x10000000249 2023-10-11T09:52:25.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.003+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x1000000024a 2023-10-11T09:52:25.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.003+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:85 2023-10-11T09:52:25.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.003+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:265 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874800 con 0x564923845000 2023-10-11T09:52:25.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.003+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 97 ==== osd_op_reply(265 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv98 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:25.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.003+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=85 frag=0*, r=0 2023-10-11T09:52:25.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.003+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:25.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.003+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:266 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875c00 con 0x564923845000 2023-10-11T09:52:25.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.007+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 98 ==== osd_op_reply(266 10000000000.01000000 [omap-set-vals] v129'99 uv99 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:25.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.007+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/85 pointing to 0x0x1000000024a 2023-10-11T09:52:25.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.007+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x1000000024b 2023-10-11T09:52:25.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.007+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:86 2023-10-11T09:52:25.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.007+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:267 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875000 con 0x564923845000 2023-10-11T09:52:25.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.007+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 99 ==== osd_op_reply(267 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv99 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:25.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.007+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=86 frag=0*, r=0 2023-10-11T09:52:25.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.007+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:25.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.007+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:268 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8000 con 0x564923845000 2023-10-11T09:52:25.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.011+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 100 ==== osd_op_reply(268 10000000000.01000000 [omap-set-vals] v129'100 uv100 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:25.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.011+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/86 pointing to 0x0x1000000024b 2023-10-11T09:52:25.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.011+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x1000000024c 2023-10-11T09:52:25.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.011+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:87 2023-10-11T09:52:25.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.011+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:269 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8c00 con 0x564923845000 2023-10-11T09:52:25.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.011+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 101 ==== osd_op_reply(269 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv100 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:25.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.011+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=87 frag=0*, r=0 2023-10-11T09:52:25.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.011+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:25.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.011+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:270 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9c00 con 0x564923845000 2023-10-11T09:52:25.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.011+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 102 ==== osd_op_reply(270 10000000000.01000000 [omap-set-vals] v129'101 uv101 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:25.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.011+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/87 pointing to 0x0x1000000024c 2023-10-11T09:52:25.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.011+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x1000000024d 2023-10-11T09:52:25.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.011+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:88 2023-10-11T09:52:25.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.011+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:271 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8400 con 0x564923845000 2023-10-11T09:52:25.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.015+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 103 ==== osd_op_reply(271 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv101 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:25.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.015+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=88 frag=0*, r=0 2023-10-11T09:52:25.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.015+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:25.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.015+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:272 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8800 con 0x564923845000 2023-10-11T09:52:25.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.015+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 104 ==== osd_op_reply(272 10000000000.01000000 [omap-set-vals] v129'102 uv102 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:25.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.015+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/88 pointing to 0x0x1000000024d 2023-10-11T09:52:25.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.015+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x1000000024e 2023-10-11T09:52:25.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.015+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:89 2023-10-11T09:52:25.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.015+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:273 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9000 con 0x564923844c00 2023-10-11T09:52:25.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.019+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 95 ==== osd_op_reply(273 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv98 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:25.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.019+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=89 frag=1*, r=0 2023-10-11T09:52:25.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.019+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:25.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.019+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:274 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649236e1800 con 0x564923844c00 2023-10-11T09:52:25.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.019+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 96 ==== osd_op_reply(274 10000000000.01800000 [omap-set-vals] v129'99 uv99 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:25.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.019+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/89 pointing to 0x0x1000000024e 2023-10-11T09:52:25.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.019+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x1000000024f 2023-10-11T09:52:25.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.019+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:90 2023-10-11T09:52:25.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.019+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:275 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923931c00 con 0x564923845000 2023-10-11T09:52:25.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.019+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 105 ==== osd_op_reply(275 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv102 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:25.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.019+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=90 frag=0*, r=0 2023-10-11T09:52:25.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.019+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:25.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.019+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:276 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874400 con 0x564923845000 2023-10-11T09:52:25.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.023+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 106 ==== osd_op_reply(276 10000000000.01000000 [omap-set-vals] v129'103 uv103 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:25.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.023+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/90 pointing to 0x0x1000000024f 2023-10-11T09:52:25.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.023+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000250 2023-10-11T09:52:25.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.023+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:91 2023-10-11T09:52:25.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.023+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:277 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874c00 con 0x564923844c00 2023-10-11T09:52:25.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.023+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 97 ==== osd_op_reply(277 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv99 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:25.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.023+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=91 frag=1*, r=0 2023-10-11T09:52:25.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.023+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:25.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.023+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:278 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875800 con 0x564923844c00 2023-10-11T09:52:25.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.027+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 98 ==== osd_op_reply(278 10000000000.01800000 [omap-set-vals] v129'100 uv100 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:25.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.027+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/91 pointing to 0x0x10000000250 2023-10-11T09:52:25.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.027+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000251 2023-10-11T09:52:25.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.027+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:92 2023-10-11T09:52:25.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.027+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:279 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923875400 con 0x564923845000 2023-10-11T09:52:25.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.027+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 107 ==== osd_op_reply(279 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv103 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:25.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.027+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=92 frag=0*, r=0 2023-10-11T09:52:25.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.027+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:25.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.027+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:280 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923874800 con 0x564923845000 2023-10-11T09:52:25.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.031+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 108 ==== osd_op_reply(280 10000000000.01000000 [omap-set-vals] v129'104 uv104 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:25.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.031+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/92 pointing to 0x0x10000000251 2023-10-11T09:52:25.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.031+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000252 2023-10-11T09:52:25.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.031+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:93 2023-10-11T09:52:25.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.031+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:281 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923845800 con 0x564923844c00 2023-10-11T09:52:25.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.031+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 99 ==== osd_op_reply(281 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv100 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:25.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.031+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=93 frag=1*, r=0 2023-10-11T09:52:25.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.031+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:25.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.031+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:282 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923930000 con 0x564923844c00 2023-10-11T09:52:25.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.035+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 100 ==== osd_op_reply(282 10000000000.01800000 [omap-set-vals] v129'101 uv101 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:25.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.035+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/93 pointing to 0x0x10000000252 2023-10-11T09:52:25.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.035+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000253 2023-10-11T09:52:25.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.035+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:94 2023-10-11T09:52:25.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.035+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:283 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923930400 con 0x564923844c00 2023-10-11T09:52:25.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.035+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 101 ==== osd_op_reply(283 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv101 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:25.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.035+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=94 frag=1*, r=0 2023-10-11T09:52:25.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.035+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:25.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.035+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:284 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923930800 con 0x564923844c00 2023-10-11T09:52:25.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.039+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 102 ==== osd_op_reply(284 10000000000.01800000 [omap-set-vals] v129'102 uv102 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:25.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.039+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/94 pointing to 0x0x10000000253 2023-10-11T09:52:25.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.039+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000254 2023-10-11T09:52:25.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.039+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:95 2023-10-11T09:52:25.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.039+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:285 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923930c00 con 0x564923845000 2023-10-11T09:52:25.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.039+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 109 ==== osd_op_reply(285 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv104 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:25.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.039+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=95 frag=0*, r=0 2023-10-11T09:52:25.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.039+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:25.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.039+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:286 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923931000 con 0x564923845000 2023-10-11T09:52:25.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.043+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 110 ==== osd_op_reply(286 10000000000.01000000 [omap-set-vals] v129'105 uv105 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:25.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.043+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/95 pointing to 0x0x10000000254 2023-10-11T09:52:25.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.043+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000255 2023-10-11T09:52:25.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.043+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:96 2023-10-11T09:52:25.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.043+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:287 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923931800 con 0x564923845000 2023-10-11T09:52:25.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.043+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 111 ==== osd_op_reply(287 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv105 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:25.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.043+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=96 frag=0*, r=0 2023-10-11T09:52:25.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.043+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:25.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.043+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:288 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9400 con 0x564923845000 2023-10-11T09:52:25.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.047+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 112 ==== osd_op_reply(288 10000000000.01000000 [omap-set-vals] v129'106 uv106 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:25.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.047+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/96 pointing to 0x0x10000000255 2023-10-11T09:52:25.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.047+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000256 2023-10-11T09:52:25.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.047+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:97 2023-10-11T09:52:25.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.047+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:289 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9800 con 0x564923845000 2023-10-11T09:52:25.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.047+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 113 ==== osd_op_reply(289 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv106 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:25.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.047+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01000000 dname=97 frag=0*, r=0 2023-10-11T09:52:25.054 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.047+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:25.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.047+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:290 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8000 con 0x564923845000 2023-10-11T09:52:25.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.051+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 <== osd.6 v2:172.21.15.133:6800/35900 114 ==== osd_op_reply(290 10000000000.01000000 [omap-set-vals] v129'107 uv107 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9440 con 0x564923845000 2023-10-11T09:52:25.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.051+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/97 pointing to 0x0x10000000256 2023-10-11T09:52:25.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.051+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000257 2023-10-11T09:52:25.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.051+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:98 2023-10-11T09:52:25.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.051+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:291 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8c00 con 0x564923844c00 2023-10-11T09:52:25.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.051+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 103 ==== osd_op_reply(291 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv102 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:25.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.051+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=98 frag=1*, r=0 2023-10-11T09:52:25.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.051+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:25.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.051+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:292 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9c00 con 0x564923844c00 2023-10-11T09:52:25.060 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.055+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 104 ==== osd_op_reply(292 10000000000.01800000 [omap-set-vals] v129'103 uv103 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:25.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.055+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/98 pointing to 0x0x10000000257 2023-10-11T09:52:25.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.055+0000 7f0731a3b840 10 datascan.scan_links: handling injected_ino 0x10000000258 2023-10-11T09:52:25.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.055+0000 7f0731a3b840 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:99 2023-10-11T09:52:25.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.055+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:293 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8400 con 0x564923844c00 2023-10-11T09:52:25.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.055+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 105 ==== osd_op_reply(293 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv103 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:25.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.055+0000 7f0731a3b840 20 datascan.read_dentry: oid=10000000000.01800000 dname=99 frag=1*, r=0 2023-10-11T09:52:25.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.055+0000 7f0731a3b840 20 datascan.scan_links: first is now 2 2023-10-11T09:52:25.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.055+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- osd_op(unknown.0.0:294 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a8800 con 0x564923844c00 2023-10-11T09:52:25.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.055+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.4 v2:172.21.15.133:6808/35898 106 ==== osd_op_reply(294 10000000000.01800000 [omap-set-vals] v129'104 uv104 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649241d9200 con 0x564923844c00 2023-10-11T09:52:25.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.055+0000 7f0731a3b840 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/99 pointing to 0x0x10000000258 2023-10-11T09:52:25.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.055+0000 7f0731a3b840 10 datascan.scan_links: updating inotable 2023-10-11T09:52:25.065 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.055+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:295 17.13 17:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x5649238a9400 con 0x5649236e1400 2023-10-11T09:52:25.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.059+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.3 v2:172.21.15.78:6824/35738 27 ==== osd_op_reply(295 mds0_inotable [read 0~50 out=50b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+50 (crc 0 0 0) 0x5649241d9200 con 0x5649236e1400 2023-10-11T09:52:25.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.059+0000 7f0731a3b840 10 mds.0.inotable: skip_inos was [0x10000000002~0x1f4,0x10000000259~0xfffffffda7] 2023-10-11T09:52:25.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.059+0000 7f0731a3b840 10 mds.0.inotable: skip_inos now [0x10000000259~0xfffffffda7] 2023-10-11T09:52:25.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.059+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- osd_op(unknown.0.0:296 17.13 17:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923931c00 con 0x5649236e1400 2023-10-11T09:52:25.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f07309f8700 1 -- 172.21.15.78:0/4264677043 <== osd.3 v2:172.21.15.78:6824/35738 28 ==== osd_op_reply(296 mds0_inotable [writefull 0~34] v129'4 uv4 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x5649241d9200 con 0x5649236e1400 2023-10-11T09:52:25.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f0731a3b840 10 datascan.scan_links: updating snaptable 2023-10-11T09:52:25.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- osd_op(unknown.0.0:297 17.d 17:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x564923930c00 con 0x5649236e1000 2023-10-11T09:52:25.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f072f9f6700 1 -- 172.21.15.78:0/4264677043 <== osd.7 v2:172.21.15.133:6824/35901 15 ==== osd_op_reply(297 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x5649241d8fc0 con 0x5649236e1000 2023-10-11T09:52:25.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x564923844400 msgr2=0x56492384e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:25.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f0731a3b840 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x564923844400 0x56492384e580 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:25.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x564923844800 msgr2=0x56492384eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:25.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f0731a3b840 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x564923844800 0x56492384eb00 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:25.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x564923874000 msgr2=0x56492384f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:25.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f0731a3b840 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x564923874000 0x56492384f600 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:25.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5649236e1400 msgr2=0x56492384e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:25.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f0731a3b840 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x5649236e1400 0x56492384e000 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:25.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x564923844c00 msgr2=0x56492384f080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:25.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f0731a3b840 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x564923844c00 0x56492384f080 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:25.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x564923845c00 msgr2=0x564923850680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:25.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f0731a3b840 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x564923845c00 0x564923850680 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:25.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x564923845000 msgr2=0x564923850100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:25.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f0731a3b840 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x564923845000 0x564923850100 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:25.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x5649236e1000 msgr2=0x564923494100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:25.073 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f0731a3b840 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x5649236e1000 0x564923494100 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:25.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 reap_dead start 2023-10-11T09:52:25.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 reap_dead start 2023-10-11T09:52:25.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f07301f7700 1 -- 172.21.15.78:0/4264677043 reap_dead start 2023-10-11T09:52:25.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x564923844000 msgr2=0x564923495700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:25.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f0731a3b840 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x564923844000 0x564923495700 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x5649236f6a20 tx=0x5649234e2f60 comp rx=0 tx=0).stop 2023-10-11T09:52:25.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649236e0800 msgr2=0x564923494c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:25.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f0731a3b840 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649236e0800 0x564923494c00 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x5649236f7650 tx=0x5649234e2000 comp rx=0 tx=0).stop 2023-10-11T09:52:25.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 shutdown_connections 2023-10-11T09:52:25.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f0731a3b840 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x564923844000 0x564923495700 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:25.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f0731a3b840 1 --2- 172.21.15.78:0/4264677043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649236e0800 0x564923494c00 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:25.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 >> 172.21.15.78:0/4264677043 conn(0x5649233ecc00 msgr2=0x5649234bb800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:25.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 shutdown_connections 2023-10-11T09:52:25.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f0731a3b840 1 -- 172.21.15.78:0/4264677043 wait complete. 2023-10-11T09:52:25.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f0731a3b840 1 -- 172.21.15.78:0/1333752241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649233ed000 msgr2=0x564923493080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:25.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f0731a3b840 1 --2- 172.21.15.78:0/1333752241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649233ed000 0x564923493080 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x5649241553e0 tx=0x5649234e3020 comp rx=0 tx=0).stop 2023-10-11T09:52:25.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f0731a3b840 1 -- 172.21.15.78:0/1333752241 shutdown_connections 2023-10-11T09:52:25.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f0731a3b840 1 --2- 172.21.15.78:0/1333752241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649233ed000 0x564923493080 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:25.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f0731a3b840 1 --2- 172.21.15.78:0/1333752241 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5649233ed400 0x564923492b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:25.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f0731a3b840 1 --2- 172.21.15.78:0/1333752241 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649233ed800 0x564923492580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:25.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f0731a3b840 1 -- 172.21.15.78:0/1333752241 >> 172.21.15.78:0/1333752241 conn(0x5649233edc00 msgr2=0x5649234bb000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:25.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f0731a3b840 1 -- 172.21.15.78:0/1333752241 shutdown_connections 2023-10-11T09:52:25.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.063+0000 7f0731a3b840 1 -- 172.21.15.78:0/1333752241 wait complete. 2023-10-11T09:52:25.080 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.607702, result: 2023-10-11T09:52:25.080 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 10000000000.01000000 2023-10-11T09:52:25.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.187+0000 7f1020655140 1 Processor -- start 2023-10-11T09:52:25.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.187+0000 7f1020655140 1 -- start start 2023-10-11T09:52:25.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.187+0000 7f1020655140 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55abf73ffdf0 0x55abf73ff1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:25.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.187+0000 7f1020655140 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf7433af0 0x55abf74d3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:25.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.187+0000 7f1020655140 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55abf74d77e0 0x55abf74d3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:25.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.187+0000 7f1020655140 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abf7350250 con 0x55abf7433af0 2023-10-11T09:52:25.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.187+0000 7f1020655140 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abf734fef0 con 0x55abf74d77e0 2023-10-11T09:52:25.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.187+0000 7f1020655140 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55abf7326240 con 0x55abf73ffdf0 2023-10-11T09:52:25.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.191+0000 7f101d989700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf7433af0 0x55abf74d3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:25.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.191+0000 7f101d989700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf7433af0 0x55abf74d3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50454/0 (socket says 172.21.15.78:50454) 2023-10-11T09:52:25.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.191+0000 7f101d989700 1 -- 172.21.15.78:0/3790643328 learned_addr learned my addr 172.21.15.78:0/3790643328 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:25.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.191+0000 7f101e18a700 1 --2- 172.21.15.78:0/3790643328 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55abf73ffdf0 0x55abf73ff1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:25.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.191+0000 7f101e98b700 1 --2- 172.21.15.78:0/3790643328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55abf74d77e0 0x55abf74d3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:25.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.191+0000 7f101d989700 1 -- 172.21.15.78:0/3790643328 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55abf73ffdf0 msgr2=0x55abf73ff1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:25.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.191+0000 7f101d989700 1 --2- 172.21.15.78:0/3790643328 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55abf73ffdf0 0x55abf73ff1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:25.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.191+0000 7f101d989700 1 -- 172.21.15.78:0/3790643328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55abf74d77e0 msgr2=0x55abf74d3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:25.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.191+0000 7f101d989700 1 --2- 172.21.15.78:0/3790643328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55abf74d77e0 0x55abf74d3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:25.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.191+0000 7f101d989700 1 -- 172.21.15.78:0/3790643328 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1010005d20 con 0x55abf7433af0 2023-10-11T09:52:25.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.191+0000 7f101d989700 1 --2- 172.21.15.78:0/3790643328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf7433af0 0x55abf74d3540 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f101400b2c0 tx=0x7f101400b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d3fa80287630a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:25.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.191+0000 7f101c987700 1 -- 172.21.15.78:0/3790643328 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1014014070 con 0x55abf7433af0 2023-10-11T09:52:25.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.191+0000 7f101c987700 1 -- 172.21.15.78:0/3790643328 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f101400f3f0 con 0x55abf7433af0 2023-10-11T09:52:25.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.191+0000 7f101c987700 1 -- 172.21.15.78:0/3790643328 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101400f7a0 con 0x55abf7433af0 2023-10-11T09:52:25.204 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.191+0000 7f1020655140 1 -- 172.21.15.78:0/3790643328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf7433af0 msgr2=0x55abf74d3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:25.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.191+0000 7f1020655140 1 --2- 172.21.15.78:0/3790643328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf7433af0 0x55abf74d3540 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f101400b2c0 tx=0x7f101400b5d0 comp rx=0 tx=0).stop 2023-10-11T09:52:25.206 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.191+0000 7f1020655140 1 -- 172.21.15.78:0/3790643328 shutdown_connections 2023-10-11T09:52:25.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.191+0000 7f1020655140 1 --2- 172.21.15.78:0/3790643328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55abf74d77e0 0x55abf74d3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:25.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.191+0000 7f1020655140 1 --2- 172.21.15.78:0/3790643328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf7433af0 0x55abf74d3540 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:25.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.191+0000 7f1020655140 1 --2- 172.21.15.78:0/3790643328 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55abf73ffdf0 0x55abf73ff1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:25.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.191+0000 7f1020655140 1 -- 172.21.15.78:0/3790643328 >> 172.21.15.78:0/3790643328 conn(0x55abf7405e00 msgr2=0x55abf744c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:25.207 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.191+0000 7f1020655140 1 -- 172.21.15.78:0/3790643328 shutdown_connections 2023-10-11T09:52:25.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.191+0000 7f1020655140 1 -- 172.21.15.78:0/3790643328 wait complete. 2023-10-11T09:52:25.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.195+0000 7f1020655140 1 Processor -- start 2023-10-11T09:52:25.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.195+0000 7f1020655140 1 -- start start 2023-10-11T09:52:25.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.195+0000 7f1020655140 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55abf73ffdf0 0x55abf74c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:25.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.195+0000 7f1020655140 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55abf7433af0 0x55abf7444850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:25.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.195+0000 7f1020655140 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf74d77e0 0x55abf7447060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:25.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.195+0000 7f1020655140 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abf74450f0 con 0x55abf74d77e0 2023-10-11T09:52:25.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.195+0000 7f1020655140 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abf7445390 con 0x55abf7433af0 2023-10-11T09:52:25.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.195+0000 7f1020655140 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55abf7445630 con 0x55abf73ffdf0 2023-10-11T09:52:25.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.195+0000 7f101e98b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf74d77e0 0x55abf7447060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:25.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.195+0000 7f101e98b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf74d77e0 0x55abf7447060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50456/0 (socket says 172.21.15.78:50456) 2023-10-11T09:52:25.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.195+0000 7f101e98b700 1 -- 172.21.15.78:0/4012002984 learned_addr learned my addr 172.21.15.78:0/4012002984 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:25.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.195+0000 7f101d989700 1 --2- 172.21.15.78:0/4012002984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55abf7433af0 0x55abf7444850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:25.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.195+0000 7f101e18a700 1 --2- 172.21.15.78:0/4012002984 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55abf73ffdf0 0x55abf74c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:25.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.195+0000 7f101d989700 1 -- 172.21.15.78:0/4012002984 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55abf73ffdf0 msgr2=0x55abf74c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:25.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.195+0000 7f101d989700 1 --2- 172.21.15.78:0/4012002984 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55abf73ffdf0 0x55abf74c8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:25.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.195+0000 7f101d989700 1 -- 172.21.15.78:0/4012002984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf74d77e0 msgr2=0x55abf7447060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:25.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.195+0000 7f101d989700 1 --2- 172.21.15.78:0/4012002984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf74d77e0 0x55abf7447060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:25.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.195+0000 7f101d989700 1 -- 172.21.15.78:0/4012002984 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1014001ba0 con 0x55abf7433af0 2023-10-11T09:52:25.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.195+0000 7f101e98b700 1 --2- 172.21.15.78:0/4012002984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf74d77e0 0x55abf7447060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:52:25.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.195+0000 7f101d989700 1 --2- 172.21.15.78:0/4012002984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55abf7433af0 0x55abf7444850 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f101400b290 tx=0x7f1014000ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:25.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.195+0000 7f100f7fe700 1 -- 172.21.15.78:0/4012002984 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1014014070 con 0x55abf7433af0 2023-10-11T09:52:25.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.195+0000 7f100f7fe700 1 -- 172.21.15.78:0/4012002984 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f101401c870 con 0x55abf7433af0 2023-10-11T09:52:25.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.195+0000 7f1020655140 1 -- 172.21.15.78:0/4012002984 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f101800a1c0 con 0x55abf7433af0 2023-10-11T09:52:25.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.195+0000 7f100f7fe700 1 -- 172.21.15.78:0/4012002984 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101400bee0 con 0x55abf7433af0 2023-10-11T09:52:25.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.199+0000 7f1020655140 1 -- 172.21.15.78:0/4012002984 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55abf733e470 con 0x55abf7433af0 2023-10-11T09:52:25.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.199+0000 7f100f7fe700 1 -- 172.21.15.78:0/4012002984 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f101400d040 con 0x55abf7433af0 2023-10-11T09:52:25.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.203+0000 7f100f7fe700 1 --2- 172.21.15.78:0/4012002984 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1000064850 0x7f1000066d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:25.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.203+0000 7f100f7fe700 1 -- 172.21.15.78:0/4012002984 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f101400a040 con 0x55abf7433af0 2023-10-11T09:52:25.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.203+0000 7f101e18a700 1 --2- 172.21.15.78:0/4012002984 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1000064850 0x7f1000066d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:25.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.203+0000 7f1020655140 1 --2- 172.21.15.78:0/4012002984 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55abf74439a0 0x55abf74cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:25.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.203+0000 7f1020655140 1 -- 172.21.15.78:0/4012002984 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- osd_op(unknown.0.0:1 17.5 17:a40620de:::10000000000.01000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e129) v8 -- 0x55abf74cb2f0 con 0x55abf74439a0 2023-10-11T09:52:25.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.203+0000 7f101e98b700 1 --2- 172.21.15.78:0/4012002984 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55abf74439a0 0x55abf74cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:25.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.203+0000 7f101e18a700 1 --2- 172.21.15.78:0/4012002984 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1000064850 0x7f1000066d00 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1010005400 tx=0x7f101000c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:25.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.203+0000 7f101e98b700 1 --2- 172.21.15.78:0/4012002984 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55abf74439a0 0x55abf74cadb0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:25.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.207+0000 7f101e98b700 1 -- 172.21.15.78:0/4012002984 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== osd_op_reply(1 10000000000.01000000 [omap-get-keys out=560b] v0'0 uv107 ondisk = 0) v8 ==== 164+0+560 (crc 0 0 0) 0x7f1018017040 con 0x55abf74439a0 2023-10-11T09:52:25.217 INFO:teuthology.orchestra.run.smithi078.stdout:10_head 2023-10-11T09:52:25.218 INFO:teuthology.orchestra.run.smithi078.stdout:11_head 2023-10-11T09:52:25.218 INFO:teuthology.orchestra.run.smithi078.stdout:12_head 2023-10-11T09:52:25.218 INFO:teuthology.orchestra.run.smithi078.stdout:15_head 2023-10-11T09:52:25.218 INFO:teuthology.orchestra.run.smithi078.stdout:19_head 2023-10-11T09:52:25.218 INFO:teuthology.orchestra.run.smithi078.stdout:1_head 2023-10-11T09:52:25.219 INFO:teuthology.orchestra.run.smithi078.stdout:20_head 2023-10-11T09:52:25.219 INFO:teuthology.orchestra.run.smithi078.stdout:21_head 2023-10-11T09:52:25.219 INFO:teuthology.orchestra.run.smithi078.stdout:27_head 2023-10-11T09:52:25.219 INFO:teuthology.orchestra.run.smithi078.stdout:28_head 2023-10-11T09:52:25.219 INFO:teuthology.orchestra.run.smithi078.stdout:2_head 2023-10-11T09:52:25.220 INFO:teuthology.orchestra.run.smithi078.stdout:31_head 2023-10-11T09:52:25.220 INFO:teuthology.orchestra.run.smithi078.stdout:34_head 2023-10-11T09:52:25.220 INFO:teuthology.orchestra.run.smithi078.stdout:36_head 2023-10-11T09:52:25.220 INFO:teuthology.orchestra.run.smithi078.stdout:38_head 2023-10-11T09:52:25.220 INFO:teuthology.orchestra.run.smithi078.stdout:39_head 2023-10-11T09:52:25.221 INFO:teuthology.orchestra.run.smithi078.stdout:3_head 2023-10-11T09:52:25.221 INFO:teuthology.orchestra.run.smithi078.stdout:42_head 2023-10-11T09:52:25.221 INFO:teuthology.orchestra.run.smithi078.stdout:43_head 2023-10-11T09:52:25.221 INFO:teuthology.orchestra.run.smithi078.stdout:46_head 2023-10-11T09:52:25.221 INFO:teuthology.orchestra.run.smithi078.stdout:4_head 2023-10-11T09:52:25.221 INFO:teuthology.orchestra.run.smithi078.stdout:50_head 2023-10-11T09:52:25.222 INFO:teuthology.orchestra.run.smithi078.stdout:52_head 2023-10-11T09:52:25.222 INFO:teuthology.orchestra.run.smithi078.stdout:53_head 2023-10-11T09:52:25.222 INFO:teuthology.orchestra.run.smithi078.stdout:55_head 2023-10-11T09:52:25.222 INFO:teuthology.orchestra.run.smithi078.stdout:59_head 2023-10-11T09:52:25.223 INFO:teuthology.orchestra.run.smithi078.stdout:61_head 2023-10-11T09:52:25.223 INFO:teuthology.orchestra.run.smithi078.stdout:63_head 2023-10-11T09:52:25.223 INFO:teuthology.orchestra.run.smithi078.stdout:64_head 2023-10-11T09:52:25.223 INFO:teuthology.orchestra.run.smithi078.stdout:65_head 2023-10-11T09:52:25.223 INFO:teuthology.orchestra.run.smithi078.stdout:66_head 2023-10-11T09:52:25.224 INFO:teuthology.orchestra.run.smithi078.stdout:69_head 2023-10-11T09:52:25.224 INFO:teuthology.orchestra.run.smithi078.stdout:70_head 2023-10-11T09:52:25.224 INFO:teuthology.orchestra.run.smithi078.stdout:73_head 2023-10-11T09:52:25.224 INFO:teuthology.orchestra.run.smithi078.stdout:74_head 2023-10-11T09:52:25.224 INFO:teuthology.orchestra.run.smithi078.stdout:75_head 2023-10-11T09:52:25.224 INFO:teuthology.orchestra.run.smithi078.stdout:77_head 2023-10-11T09:52:25.225 INFO:teuthology.orchestra.run.smithi078.stdout:80_head 2023-10-11T09:52:25.225 INFO:teuthology.orchestra.run.smithi078.stdout:83_head 2023-10-11T09:52:25.225 INFO:teuthology.orchestra.run.smithi078.stdout:84_head 2023-10-11T09:52:25.225 INFO:teuthology.orchestra.run.smithi078.stdout:85_head 2023-10-11T09:52:25.225 INFO:teuthology.orchestra.run.smithi078.stdout:86_head 2023-10-11T09:52:25.225 INFO:teuthology.orchestra.run.smithi078.stdout:87_head 2023-10-11T09:52:25.226 INFO:teuthology.orchestra.run.smithi078.stdout:88_head 2023-10-11T09:52:25.226 INFO:teuthology.orchestra.run.smithi078.stdout:8_head 2023-10-11T09:52:25.226 INFO:teuthology.orchestra.run.smithi078.stdout:90_head 2023-10-11T09:52:25.226 INFO:teuthology.orchestra.run.smithi078.stdout:92_head 2023-10-11T09:52:25.226 INFO:teuthology.orchestra.run.smithi078.stdout:95_head 2023-10-11T09:52:25.227 INFO:teuthology.orchestra.run.smithi078.stdout:96_head 2023-10-11T09:52:25.227 INFO:teuthology.orchestra.run.smithi078.stdout:97_head 2023-10-11T09:52:25.227 INFO:teuthology.orchestra.run.smithi078.stdout:9_head 2023-10-11T09:52:25.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.207+0000 7f1020655140 1 -- 172.21.15.78:0/4012002984 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55abf74439a0 msgr2=0x55abf74cadb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:25.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.207+0000 7f1020655140 1 --2- 172.21.15.78:0/4012002984 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x55abf74439a0 0x55abf74cadb0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:25.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.207+0000 7f1020655140 1 -- 172.21.15.78:0/4012002984 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1000064850 msgr2=0x7f1000066d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:25.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.207+0000 7f1020655140 1 --2- 172.21.15.78:0/4012002984 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1000064850 0x7f1000066d00 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1010005400 tx=0x7f101000c040 comp rx=0 tx=0).stop 2023-10-11T09:52:25.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.207+0000 7f1020655140 1 -- 172.21.15.78:0/4012002984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55abf7433af0 msgr2=0x55abf7444850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:25.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.207+0000 7f1020655140 1 --2- 172.21.15.78:0/4012002984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55abf7433af0 0x55abf7444850 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f101400b290 tx=0x7f1014000ed0 comp rx=0 tx=0).stop 2023-10-11T09:52:25.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.207+0000 7f101e98b700 1 -- 172.21.15.78:0/4012002984 reap_dead start 2023-10-11T09:52:25.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.207+0000 7f1020655140 1 -- 172.21.15.78:0/4012002984 shutdown_connections 2023-10-11T09:52:25.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.207+0000 7f1020655140 1 -- 172.21.15.78:0/4012002984 >> 172.21.15.78:0/4012002984 conn(0x55abf7405e00 msgr2=0x55abf73ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:25.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.207+0000 7f1020655140 1 -- 172.21.15.78:0/4012002984 shutdown_connections 2023-10-11T09:52:25.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.207+0000 7f1020655140 1 -- 172.21.15.78:0/4012002984 wait complete. 2023-10-11T09:52:25.231 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-10-11T09:52:25.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.411+0000 7f0c2812c700 1 Processor -- start 2023-10-11T09:52:25.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.411+0000 7f0c2812c700 1 -- start start 2023-10-11T09:52:25.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.411+0000 7f0c2812c700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0c20159800 0x7f0c2015dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:25.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.411+0000 7f0c2812c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0c200ce000 0x7f0c200ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:25.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.411+0000 7f0c2812c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c200b1af0 0x7f0c200b1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:25.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.411+0000 7f0c2812c700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c200b24b0 con 0x7f0c200ce000 2023-10-11T09:52:25.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.411+0000 7f0c2812c700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c2015e230 con 0x7f0c20159800 2023-10-11T09:52:25.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.411+0000 7f0c2812c700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c20159bd0 con 0x7f0c200b1af0 2023-10-11T09:52:25.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.415+0000 7f0c256c7700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0c200ce000 0x7f0c200ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:25.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.415+0000 7f0c256c7700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0c200ce000 0x7f0c200ca7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50472/0 (socket says 172.21.15.78:50472) 2023-10-11T09:52:25.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.415+0000 7f0c25ec8700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0c20159800 0x7f0c2015dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:25.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.415+0000 7f0c256c7700 1 -- 172.21.15.78:0/2808288662 learned_addr learned my addr 172.21.15.78:0/2808288662 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:25.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.415+0000 7f0c266c9700 1 --2- 172.21.15.78:0/2808288662 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c200b1af0 0x7f0c200b1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:25.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.415+0000 7f0c256c7700 1 -- 172.21.15.78:0/2808288662 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c200b1af0 msgr2=0x7f0c200b1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:25.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.415+0000 7f0c256c7700 1 --2- 172.21.15.78:0/2808288662 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c200b1af0 0x7f0c200b1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:25.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.415+0000 7f0c256c7700 1 -- 172.21.15.78:0/2808288662 >> v2:172.21.15.133:3300/0 conn(0x7f0c20159800 msgr2=0x7f0c2015dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:25.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.415+0000 7f0c256c7700 1 --2- 172.21.15.78:0/2808288662 >> v2:172.21.15.133:3300/0 conn(0x7f0c20159800 0x7f0c2015dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:25.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.415+0000 7f0c256c7700 1 -- 172.21.15.78:0/2808288662 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c200cad20 con 0x7f0c200ce000 2023-10-11T09:52:25.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.415+0000 7f0c256c7700 1 --2- 172.21.15.78:0/2808288662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c200ce000 0x7f0c200ca7e0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f0c200cff10 tx=0x7f0c10009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75597ed437c7278b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:25.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.415+0000 7f0c24ec6700 1 -- 172.21.15.78:0/2808288662 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c10011830 con 0x7f0c200ce000 2023-10-11T09:52:25.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.415+0000 7f0c24ec6700 1 -- 172.21.15.78:0/2808288662 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c10011990 con 0x7f0c200ce000 2023-10-11T09:52:25.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.415+0000 7f0c24ec6700 1 -- 172.21.15.78:0/2808288662 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c10011d20 con 0x7f0c200ce000 2023-10-11T09:52:25.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.415+0000 7f0c2812c700 1 -- 172.21.15.78:0/2808288662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c200ce000 msgr2=0x7f0c200ca7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:25.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.415+0000 7f0c2812c700 1 --2- 172.21.15.78:0/2808288662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c200ce000 0x7f0c200ca7e0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f0c200cff10 tx=0x7f0c10009bf0 comp rx=0 tx=0).stop 2023-10-11T09:52:25.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.415+0000 7f0c2812c700 1 -- 172.21.15.78:0/2808288662 shutdown_connections 2023-10-11T09:52:25.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.415+0000 7f0c2812c700 1 --2- 172.21.15.78:0/2808288662 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c200b1af0 0x7f0c200b1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:25.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.415+0000 7f0c2812c700 1 --2- 172.21.15.78:0/2808288662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c200ce000 0x7f0c200ca7e0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:25.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.415+0000 7f0c2812c700 1 --2- 172.21.15.78:0/2808288662 >> v2:172.21.15.133:3300/0 conn(0x7f0c20159800 0x7f0c2015dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:25.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.415+0000 7f0c2812c700 1 -- 172.21.15.78:0/2808288662 >> 172.21.15.78:0/2808288662 conn(0x7f0c2006b290 msgr2=0x7f0c2006b660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:25.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.415+0000 7f0c2812c700 1 -- 172.21.15.78:0/2808288662 shutdown_connections 2023-10-11T09:52:25.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.415+0000 7f0c2812c700 1 -- 172.21.15.78:0/2808288662 wait complete. 2023-10-11T09:52:25.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.419+0000 7f0c2812c700 1 Processor -- start 2023-10-11T09:52:25.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.419+0000 7f0c2812c700 1 -- start start 2023-10-11T09:52:25.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.423+0000 7f0c2812c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c200b1af0 0x7f0c20152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:25.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.423+0000 7f0c2812c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c200ce000 0x7f0c200b4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:25.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.423+0000 7f0c2812c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c20159800 0x7f0c20154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:25.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.423+0000 7f0c2812c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c200b4a60 con 0x7f0c20159800 2023-10-11T09:52:25.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.423+0000 7f0c2812c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c200b06a0 con 0x7f0c200ce000 2023-10-11T09:52:25.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.423+0000 7f0c2812c700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c200afb20 con 0x7f0c200b1af0 2023-10-11T09:52:25.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.423+0000 7f0c266c9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c20159800 0x7f0c20154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:25.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.423+0000 7f0c266c9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c20159800 0x7f0c20154c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50482/0 (socket says 172.21.15.78:50482) 2023-10-11T09:52:25.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.423+0000 7f0c266c9700 1 -- 172.21.15.78:0/3674592658 learned_addr learned my addr 172.21.15.78:0/3674592658 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:25.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.423+0000 7f0c256c7700 1 --2- 172.21.15.78:0/3674592658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c200ce000 0x7f0c200b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:25.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.423+0000 7f0c25ec8700 1 --2- 172.21.15.78:0/3674592658 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c200b1af0 0x7f0c20152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:25.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.423+0000 7f0c266c9700 1 -- 172.21.15.78:0/3674592658 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c200b1af0 msgr2=0x7f0c20152720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:25.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.423+0000 7f0c266c9700 1 --2- 172.21.15.78:0/3674592658 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c200b1af0 0x7f0c20152720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:25.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.423+0000 7f0c266c9700 1 -- 172.21.15.78:0/3674592658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c200ce000 msgr2=0x7f0c200b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:25.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.423+0000 7f0c266c9700 1 --2- 172.21.15.78:0/3674592658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c200ce000 0x7f0c200b4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:25.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.423+0000 7f0c266c9700 1 -- 172.21.15.78:0/3674592658 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c10009570 con 0x7f0c20159800 2023-10-11T09:52:25.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.423+0000 7f0c25ec8700 1 --2- 172.21.15.78:0/3674592658 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c200b1af0 0x7f0c20152720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:25.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.423+0000 7f0c266c9700 1 --2- 172.21.15.78:0/3674592658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c20159800 0x7f0c20154c70 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800ef00 tx=0x7f0c1800ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:25.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.423+0000 7f0c167fc700 1 -- 172.21.15.78:0/3674592658 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c18010040 con 0x7f0c20159800 2023-10-11T09:52:25.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.423+0000 7f0c167fc700 1 -- 172.21.15.78:0/3674592658 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1800baa0 con 0x7f0c20159800 2023-10-11T09:52:25.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.423+0000 7f0c2812c700 1 -- 172.21.15.78:0/3674592658 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c200afe00 con 0x7f0c20159800 2023-10-11T09:52:25.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.423+0000 7f0c167fc700 1 -- 172.21.15.78:0/3674592658 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1800be50 con 0x7f0c20159800 2023-10-11T09:52:25.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.423+0000 7f0c2812c700 1 -- 172.21.15.78:0/3674592658 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c200b03c0 con 0x7f0c20159800 2023-10-11T09:52:25.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.427+0000 7f0c2812c700 1 -- 172.21.15.78:0/3674592658 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c200708e0 con 0x7f0c20159800 2023-10-11T09:52:25.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.427+0000 7f0c167fc700 1 -- 172.21.15.78:0/3674592658 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f0c1801d400 con 0x7f0c20159800 2023-10-11T09:52:25.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.431+0000 7f0c167fc700 1 --2- 172.21.15.78:0/3674592658 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0c04065ed0 0x7f0c04068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:25.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.431+0000 7f0c167fc700 1 -- 172.21.15.78:0/3674592658 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f0c18083f20 con 0x7f0c20159800 2023-10-11T09:52:25.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.431+0000 7f0c25ec8700 1 --2- 172.21.15.78:0/3674592658 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0c04065ed0 0x7f0c04068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:25.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.431+0000 7f0c25ec8700 1 --2- 172.21.15.78:0/3674592658 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0c04065ed0 0x7f0c04068380 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00b240 tx=0x7f0c1c00b450 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:25.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.435+0000 7f0c167fc700 1 -- 172.21.15.78:0/3674592658 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f0c18051420 con 0x7f0c20159800 2023-10-11T09:52:25.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:25.647+0000 7f0c2812c700 1 -- 172.21.15.78:0/3674592658 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f0c200ce860 con 0x7f0c20159800 2023-10-11T09:52:26.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.475+0000 7f0c167fc700 1 -- 172.21.15.78:0/3674592658 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v108) v1 ==== 162+0+0 (secure 0 0 0) 0x7f0c18054a40 con 0x7f0c20159800 2023-10-11T09:52:26.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.479+0000 7f0c2812c700 1 -- 172.21.15.78:0/3674592658 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0c04065ed0 msgr2=0x7f0c04068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:26.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.479+0000 7f0c2812c700 1 --2- 172.21.15.78:0/3674592658 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0c04065ed0 0x7f0c04068380 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00b240 tx=0x7f0c1c00b450 comp rx=0 tx=0).stop 2023-10-11T09:52:26.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.479+0000 7f0c2812c700 1 -- 172.21.15.78:0/3674592658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c20159800 msgr2=0x7f0c20154c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:26.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.479+0000 7f0c2812c700 1 --2- 172.21.15.78:0/3674592658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c20159800 0x7f0c20154c70 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800ef00 tx=0x7f0c1800ef30 comp rx=0 tx=0).stop 2023-10-11T09:52:26.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.479+0000 7f0c2812c700 1 -- 172.21.15.78:0/3674592658 shutdown_connections 2023-10-11T09:52:26.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.479+0000 7f0c2812c700 1 --2- 172.21.15.78:0/3674592658 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0c04065ed0 0x7f0c04068380 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:26.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.479+0000 7f0c2812c700 1 --2- 172.21.15.78:0/3674592658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c20159800 0x7f0c20154c70 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:26.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.479+0000 7f0c2812c700 1 --2- 172.21.15.78:0/3674592658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c200ce000 0x7f0c200b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:26.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.479+0000 7f0c2812c700 1 --2- 172.21.15.78:0/3674592658 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c200b1af0 0x7f0c20152720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:26.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.479+0000 7f0c2812c700 1 -- 172.21.15.78:0/3674592658 >> 172.21.15.78:0/3674592658 conn(0x7f0c2006b290 msgr2=0x7f0c200680b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:26.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.479+0000 7f0c2812c700 1 -- 172.21.15.78:0/3674592658 shutdown_connections 2023-10-11T09:52:26.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.479+0000 7f0c2812c700 1 -- 172.21.15.78:0/3674592658 wait complete. 2023-10-11T09:52:26.490 INFO:teuthology.orchestra.run.smithi078.stderr:cephfs marked joinable; MDS may join as newly active. 2023-10-11T09:52:26.513 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:52:26.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.679+0000 7f7011c39700 1 Processor -- start 2023-10-11T09:52:26.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.679+0000 7f7011c39700 1 -- start start 2023-10-11T09:52:26.688 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.679+0000 7f7011c39700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f700c161020 0x7f700c0c20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:26.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.679+0000 7f7011c39700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f700c071720 0x7f700c0cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:26.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.679+0000 7f7011c39700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f700c0cb220 con 0x7f700c0ce530 2023-10-11T09:52:26.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.679+0000 7f7011c39700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f700c071c70 con 0x7f700c161020 2023-10-11T09:52:26.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.679+0000 7f7011c39700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f700c0ce9d0 con 0x7f700c071720 2023-10-11T09:52:26.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.679+0000 7f700bfff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f700c071720 0x7f700c0cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:26.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.679+0000 7f700bfff700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f700c071720 0x7f700c0cdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:60480/0 (socket says 172.21.15.78:60480) 2023-10-11T09:52:26.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.679+0000 7f700bfff700 1 -- 172.21.15.78:0/1925683295 learned_addr learned my addr 172.21.15.78:0/1925683295 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:26.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.679+0000 7f7010c37700 1 --2- 172.21.15.78:0/1925683295 >> v2:172.21.15.133:3300/0 conn(0x7f700c161020 0x7f700c0c20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:26.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.679+0000 7f700bfff700 1 -- 172.21.15.78:0/1925683295 >> v2:172.21.15.133:3300/0 conn(0x7f700c161020 msgr2=0x7f700c0c20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:26.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.679+0000 7f700bfff700 1 --2- 172.21.15.78:0/1925683295 >> v2:172.21.15.133:3300/0 conn(0x7f700c161020 0x7f700c0c20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:26.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.679+0000 7f700bfff700 1 -- 172.21.15.78:0/1925683295 >> v1:172.21.15.133:6789/0 conn(0x7f700c0ce530 legacy=0x7f700c164af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:26.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.679+0000 7f700bfff700 1 -- 172.21.15.78:0/1925683295 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700c1613f0 con 0x7f700c071720 2023-10-11T09:52:26.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.679+0000 7f7010c37700 1 --2- 172.21.15.78:0/1925683295 >> v2:172.21.15.133:3300/0 conn(0x7f700c161020 0x7f700c0c20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:26.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.683+0000 7f700bfff700 1 --2- 172.21.15.78:0/1925683295 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f700c071720 0x7f700c0cdff0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8009910 tx=0x7f6ff8009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d8d91597c4d3056 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:26.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.683+0000 7f700b7fe700 1 -- 172.21.15.78:0/1925683295 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8005e50 con 0x7f700c071720 2023-10-11T09:52:26.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.683+0000 7f700b7fe700 1 -- 172.21.15.78:0/1925683295 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff8009e60 con 0x7f700c071720 2023-10-11T09:52:26.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.683+0000 7f700b7fe700 1 -- 172.21.15.78:0/1925683295 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff800b870 con 0x7f700c071720 2023-10-11T09:52:26.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.683+0000 7f7011c39700 1 -- 172.21.15.78:0/1925683295 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f700c071720 msgr2=0x7f700c0cdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:26.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.683+0000 7f7011c39700 1 --2- 172.21.15.78:0/1925683295 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f700c071720 0x7f700c0cdff0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8009910 tx=0x7f6ff8009c20 comp rx=0 tx=0).stop 2023-10-11T09:52:26.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.683+0000 7f7011c39700 1 -- 172.21.15.78:0/1925683295 shutdown_connections 2023-10-11T09:52:26.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.683+0000 7f7011c39700 1 --2- 172.21.15.78:0/1925683295 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f700c071720 0x7f700c0cdff0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:26.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.683+0000 7f7011c39700 1 --2- 172.21.15.78:0/1925683295 >> v2:172.21.15.133:3300/0 conn(0x7f700c161020 0x7f700c0c20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:26.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.683+0000 7f7011c39700 1 -- 172.21.15.78:0/1925683295 >> 172.21.15.78:0/1925683295 conn(0x7f700c0cf820 msgr2=0x7f700c0cb610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:26.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.683+0000 7f7011c39700 1 -- 172.21.15.78:0/1925683295 shutdown_connections 2023-10-11T09:52:26.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.683+0000 7f7011c39700 1 -- 172.21.15.78:0/1925683295 wait complete. 2023-10-11T09:52:26.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.687+0000 7f7011c39700 1 Processor -- start 2023-10-11T09:52:26.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.687+0000 7f7011c39700 1 -- start start 2023-10-11T09:52:26.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.691+0000 7f7011c39700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f700c071720 0x7f700c163f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:26.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.691+0000 7f7011c39700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f700c0ce530 0x7f700c1f6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:26.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.691+0000 7f7011c39700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f700c161020 0x7f700c0c5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:26.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.691+0000 7f7011c39700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c1f6e50 con 0x7f700c071720 2023-10-11T09:52:26.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.691+0000 7f7011c39700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c0c5650 con 0x7f700c161020 2023-10-11T09:52:26.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.691+0000 7f7011c39700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f700c1f7050 con 0x7f700c0ce530 2023-10-11T09:52:26.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.691+0000 7f7010c37700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f700c071720 0x7f700c163f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:26.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.691+0000 7f7010c37700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f700c071720 0x7f700c163f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50496/0 (socket says 172.21.15.78:50496) 2023-10-11T09:52:26.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.691+0000 7f7010c37700 1 -- 172.21.15.78:0/2785828005 learned_addr learned my addr 172.21.15.78:0/2785828005 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:26.701 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.691+0000 7f700bfff700 1 --2- 172.21.15.78:0/2785828005 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f700c0ce530 0x7f700c1f6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:26.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.691+0000 7f7011438700 1 --2- 172.21.15.78:0/2785828005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f700c161020 0x7f700c0c5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:26.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.691+0000 7f7010c37700 1 -- 172.21.15.78:0/2785828005 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f700c0ce530 msgr2=0x7f700c1f6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:26.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.691+0000 7f7010c37700 1 --2- 172.21.15.78:0/2785828005 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f700c0ce530 0x7f700c1f6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:26.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.691+0000 7f7010c37700 1 -- 172.21.15.78:0/2785828005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f700c161020 msgr2=0x7f700c0c5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:26.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.691+0000 7f7010c37700 1 --2- 172.21.15.78:0/2785828005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f700c161020 0x7f700c0c5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:26.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.691+0000 7f7010c37700 1 -- 172.21.15.78:0/2785828005 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff8009520 con 0x7f700c071720 2023-10-11T09:52:26.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.691+0000 7f700bfff700 1 --2- 172.21.15.78:0/2785828005 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f700c0ce530 0x7f700c1f6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:26.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.691+0000 7f7010c37700 1 --2- 172.21.15.78:0/2785828005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f700c071720 0x7f700c163f70 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f700000e880 tx=0x7f700000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:26.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.691+0000 7f70097fa700 1 -- 172.21.15.78:0/2785828005 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700000c780 con 0x7f700c071720 2023-10-11T09:52:26.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.691+0000 7f70097fa700 1 -- 172.21.15.78:0/2785828005 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f700000c8e0 con 0x7f700c071720 2023-10-11T09:52:26.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.691+0000 7f70097fa700 1 -- 172.21.15.78:0/2785828005 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700000cc70 con 0x7f700c071720 2023-10-11T09:52:26.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.691+0000 7f7011c39700 1 -- 172.21.15.78:0/2785828005 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f700c1f7330 con 0x7f700c071720 2023-10-11T09:52:26.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.691+0000 7f7011c39700 1 -- 172.21.15.78:0/2785828005 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f700c1f78f0 con 0x7f700c071720 2023-10-11T09:52:26.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.695+0000 7f7011c39700 1 -- 172.21.15.78:0/2785828005 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f700c067d00 con 0x7f700c071720 2023-10-11T09:52:26.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.695+0000 7f70097fa700 1 -- 172.21.15.78:0/2785828005 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f7000013410 con 0x7f700c071720 2023-10-11T09:52:26.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.699+0000 7f70097fa700 1 --2- 172.21.15.78:0/2785828005 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6ff0065ed0 0x7f6ff0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:26.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.699+0000 7f70097fa700 1 -- 172.21.15.78:0/2785828005 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f70000835e0 con 0x7f700c071720 2023-10-11T09:52:26.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.699+0000 7f700bfff700 1 --2- 172.21.15.78:0/2785828005 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6ff0065ed0 0x7f6ff0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:26.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.699+0000 7f700bfff700 1 --2- 172.21.15.78:0/2785828005 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6ff0065ed0 0x7f6ff0068380 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8009c60 tx=0x7f6ff8008070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:26.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.703+0000 7f70097fa700 1 -- 172.21.15.78:0/2785828005 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f7000050ae0 con 0x7f700c071720 2023-10-11T09:52:26.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.911+0000 7f7011c39700 1 -- 172.21.15.78:0/2785828005 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f700c065d40 con 0x7f700c071720 2023-10-11T09:52:26.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.915+0000 7f70097fa700 1 -- 172.21.15.78:0/2785828005 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+4656 (secure 0 0 0) 0x7f7000054100 con 0x7f700c071720 2023-10-11T09:52:26.921 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:52:26.921 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":109,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8452,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/2481257990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2481257990},{"type":"v1","addr":"172.21.15.78:6836","nonce":2481257990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":8464,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1035227282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1035227282},{"type":"v1","addr":"172.21.15.133:6835","nonce":1035227282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":8680,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/36737634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":36737634},{"type":"v1","addr":"172.21.15.133:6833","nonce":36737634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":101}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:51:40.211859+0000","modified":"2023-10-11T09:52:26.478453+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9336},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9336":{"gid":9336,"name":"c","rank":0,"incarnation":109,"state":"up:replay","state_seq":1,"addr":"172.21.15.78:6837/3884456900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":3884456900},{"type":"v1","addr":"172.21.15.78:6837","nonce":3884456900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-11T09:52:26.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.919+0000 7f7011c39700 1 -- 172.21.15.78:0/2785828005 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6ff0065ed0 msgr2=0x7f6ff0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:26.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.919+0000 7f7011c39700 1 --2- 172.21.15.78:0/2785828005 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6ff0065ed0 0x7f6ff0068380 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8009c60 tx=0x7f6ff8008070 comp rx=0 tx=0).stop 2023-10-11T09:52:26.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.919+0000 7f7011c39700 1 -- 172.21.15.78:0/2785828005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f700c071720 msgr2=0x7f700c163f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:26.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.919+0000 7f7011c39700 1 --2- 172.21.15.78:0/2785828005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f700c071720 0x7f700c163f70 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f700000e880 tx=0x7f700000ec40 comp rx=0 tx=0).stop 2023-10-11T09:52:26.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.919+0000 7f7011c39700 1 -- 172.21.15.78:0/2785828005 shutdown_connections 2023-10-11T09:52:26.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.919+0000 7f7011c39700 1 --2- 172.21.15.78:0/2785828005 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6ff0065ed0 0x7f6ff0068380 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:26.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.919+0000 7f7011c39700 1 --2- 172.21.15.78:0/2785828005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f700c161020 0x7f700c0c5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:26.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.919+0000 7f7011c39700 1 --2- 172.21.15.78:0/2785828005 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f700c0ce530 0x7f700c1f6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:26.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.919+0000 7f7011c39700 1 --2- 172.21.15.78:0/2785828005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f700c071720 0x7f700c163f70 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:26.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.919+0000 7f7011c39700 1 -- 172.21.15.78:0/2785828005 >> 172.21.15.78:0/2785828005 conn(0x7f700c0cf820 msgr2=0x7f700c1602d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:26.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.919+0000 7f7011c39700 1 -- 172.21.15.78:0/2785828005 shutdown_connections 2023-10-11T09:52:26.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:26.919+0000 7f7011c39700 1 -- 172.21.15.78:0/2785828005 wait complete. 2023-10-11T09:52:26.928 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 109 2023-10-11T09:52:27.350 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 109, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-11T09:51:40.211859+0000', 'modified': '2023-10-11T09:52:26.478453+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 129, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9336}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9336': {'gid': 9336, 'name': 'c', 'rank': 0, 'incarnation': 109, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.78:6837/3884456900', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6835', 'nonce': 3884456900}, {'type': 'v1', 'addr': '172.21.15.78:6837', 'nonce': 3884456900}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-11T09:52:27.351 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9336:up:replay 2023-10-11T09:52:28.352 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:52:28.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.515+0000 7f42b759e700 1 Processor -- start 2023-10-11T09:52:28.521 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.515+0000 7f42b759e700 1 -- start start 2023-10-11T09:52:28.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.515+0000 7f42b759e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42b815a800 0x7f42b80b4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:28.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.515+0000 7f42b759e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f42b806bb60 0x7f42b80b0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:28.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.515+0000 7f42b759e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42b8070dc0 con 0x7f42b806d060 2023-10-11T09:52:28.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.515+0000 7f42b759e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42b80b46a0 con 0x7f42b806bb60 2023-10-11T09:52:28.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.515+0000 7f42b759e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42b8070ff0 con 0x7f42b815a800 2023-10-11T09:52:28.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.515+0000 7f42b6d9d700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f42b806d060 0x7f42b80c9150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:57698/0 (socket says 172.21.15.78:57698) 2023-10-11T09:52:28.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.515+0000 7f42b6d9d700 1 -- 172.21.15.78:0/765319681 learned_addr learned my addr 172.21.15.78:0/765319681 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:28.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.515+0000 7f42b659c700 1 --2- 172.21.15.78:0/765319681 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42b815a800 0x7f42b80b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:28.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.515+0000 7f42b5d9b700 1 --2- 172.21.15.78:0/765319681 >> v2:172.21.15.133:3300/0 conn(0x7f42b806bb60 0x7f42b80b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:28.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.515+0000 7f42b559a700 1 -- 172.21.15.78:0/765319681 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3902162277 0 0) 0x7f42b8070dc0 con 0x7f42b806d060 2023-10-11T09:52:28.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.515+0000 7f42b559a700 1 -- 172.21.15.78:0/765319681 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42a00035d0 con 0x7f42b806d060 2023-10-11T09:52:28.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.515+0000 7f42b659c700 1 -- 172.21.15.78:0/765319681 >> v2:172.21.15.133:3300/0 conn(0x7f42b806bb60 msgr2=0x7f42b80b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:28.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.515+0000 7f42b659c700 1 --2- 172.21.15.78:0/765319681 >> v2:172.21.15.133:3300/0 conn(0x7f42b806bb60 0x7f42b80b0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:28.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.515+0000 7f42b659c700 1 -- 172.21.15.78:0/765319681 >> v1:172.21.15.133:6789/0 conn(0x7f42b806d060 legacy=0x7f42b80c9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:28.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.515+0000 7f42b659c700 1 -- 172.21.15.78:0/765319681 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b80712d0 con 0x7f42b815a800 2023-10-11T09:52:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.515+0000 7f42b659c700 1 --2- 172.21.15.78:0/765319681 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42b815a800 0x7f42b80b4160 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f42a8009850 tx=0x7f42a8009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f84df771ac4279f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.519+0000 7f42b559a700 1 -- 172.21.15.78:0/765319681 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a8011830 con 0x7f42b815a800 2023-10-11T09:52:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.519+0000 7f42b559a700 1 -- 172.21.15.78:0/765319681 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a8011990 con 0x7f42b815a800 2023-10-11T09:52:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.519+0000 7f42b559a700 1 -- 172.21.15.78:0/765319681 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a8011d40 con 0x7f42b815a800 2023-10-11T09:52:28.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.519+0000 7f42b759e700 1 -- 172.21.15.78:0/765319681 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42b815a800 msgr2=0x7f42b80b4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:28.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.519+0000 7f42b759e700 1 --2- 172.21.15.78:0/765319681 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42b815a800 0x7f42b80b4160 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f42a8009850 tx=0x7f42a8009b60 comp rx=0 tx=0).stop 2023-10-11T09:52:28.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.519+0000 7f42b759e700 1 -- 172.21.15.78:0/765319681 shutdown_connections 2023-10-11T09:52:28.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.519+0000 7f42b759e700 1 --2- 172.21.15.78:0/765319681 >> v2:172.21.15.133:3300/0 conn(0x7f42b806bb60 0x7f42b80b0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:28.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.519+0000 7f42b759e700 1 --2- 172.21.15.78:0/765319681 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42b815a800 0x7f42b80b4160 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:28.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.519+0000 7f42b759e700 1 -- 172.21.15.78:0/765319681 >> 172.21.15.78:0/765319681 conn(0x7f42b80cf820 msgr2=0x7f42b80cb610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:28.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.519+0000 7f42b759e700 1 -- 172.21.15.78:0/765319681 shutdown_connections 2023-10-11T09:52:28.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.519+0000 7f42b759e700 1 -- 172.21.15.78:0/765319681 wait complete. 2023-10-11T09:52:28.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.519+0000 7f42b759e700 1 Processor -- start 2023-10-11T09:52:28.530 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.523+0000 7f42b759e700 1 -- start start 2023-10-11T09:52:28.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.523+0000 7f42b759e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42b806bb60 0x7f42b80b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:28.532 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.523+0000 7f42b759e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42b806d060 0x7f42b806a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:28.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.523+0000 7f42b759e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42b815a800 0x7f42b806a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:28.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.523+0000 7f42b759e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b80b1140 con 0x7f42b806d060 2023-10-11T09:52:28.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.523+0000 7f42b759e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b80c99c0 con 0x7f42b815a800 2023-10-11T09:52:28.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.523+0000 7f42b759e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42b80c26b0 con 0x7f42b806bb60 2023-10-11T09:52:28.533 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.523+0000 7f42b5d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42b806d060 0x7f42b806a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:28.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.523+0000 7f42b5d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42b806d060 0x7f42b806a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50512/0 (socket says 172.21.15.78:50512) 2023-10-11T09:52:28.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.523+0000 7f42b5d9b700 1 -- 172.21.15.78:0/3362380737 learned_addr learned my addr 172.21.15.78:0/3362380737 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:28.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.523+0000 7f42b659c700 1 --2- 172.21.15.78:0/3362380737 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42b806bb60 0x7f42b80b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:28.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.523+0000 7f42b6d9d700 1 --2- 172.21.15.78:0/3362380737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42b815a800 0x7f42b806a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:28.534 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.523+0000 7f42b5d9b700 1 -- 172.21.15.78:0/3362380737 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42b806bb60 msgr2=0x7f42b80b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:28.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.523+0000 7f42b5d9b700 1 --2- 172.21.15.78:0/3362380737 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42b806bb60 0x7f42b80b0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:28.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.523+0000 7f42b5d9b700 1 -- 172.21.15.78:0/3362380737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42b815a800 msgr2=0x7f42b806a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:28.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.523+0000 7f42b5d9b700 1 --2- 172.21.15.78:0/3362380737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42b815a800 0x7f42b806a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:28.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.523+0000 7f42b5d9b700 1 -- 172.21.15.78:0/3362380737 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a80094b0 con 0x7f42b806d060 2023-10-11T09:52:28.535 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.523+0000 7f42b5d9b700 1 --2- 172.21.15.78:0/3362380737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42b806d060 0x7f42b806a180 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f42ac00e9b0 tx=0x7f42ac00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:28.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.523+0000 7f42a6ffd700 1 -- 172.21.15.78:0/3362380737 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac009710 con 0x7f42b806d060 2023-10-11T09:52:28.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.523+0000 7f42a6ffd700 1 -- 172.21.15.78:0/3362380737 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42ac00c850 con 0x7f42b806d060 2023-10-11T09:52:28.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.523+0000 7f42a6ffd700 1 -- 172.21.15.78:0/3362380737 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac00cbc0 con 0x7f42b806d060 2023-10-11T09:52:28.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.523+0000 7f42b759e700 1 -- 172.21.15.78:0/3362380737 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42b80c2990 con 0x7f42b806d060 2023-10-11T09:52:28.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.523+0000 7f42b759e700 1 -- 172.21.15.78:0/3362380737 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42b80c2f50 con 0x7f42b806d060 2023-10-11T09:52:28.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.527+0000 7f42b759e700 1 -- 172.21.15.78:0/3362380737 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42b8067d00 con 0x7f42b806d060 2023-10-11T09:52:28.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.527+0000 7f42a6ffd700 1 -- 172.21.15.78:0/3362380737 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f42ac00cd20 con 0x7f42b806d060 2023-10-11T09:52:28.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.531+0000 7f42a6ffd700 1 --2- 172.21.15.78:0/3362380737 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4298065ed0 0x7f4298068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:28.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.531+0000 7f42a6ffd700 1 -- 172.21.15.78:0/3362380737 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f42ac083cf0 con 0x7f42b806d060 2023-10-11T09:52:28.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.531+0000 7f42b659c700 1 --2- 172.21.15.78:0/3362380737 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4298065ed0 0x7f4298068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.531+0000 7f42b659c700 1 --2- 172.21.15.78:0/3362380737 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4298065ed0 0x7f4298068380 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f42a80090b0 tx=0x7f42a801c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.535+0000 7f42a6ffd700 1 -- 172.21.15.78:0/3362380737 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f42ac088070 con 0x7f42b806d060 2023-10-11T09:52:28.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.743+0000 7f42b759e700 1 -- 172.21.15.78:0/3362380737 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42b8065d40 con 0x7f42b806d060 2023-10-11T09:52:28.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.743+0000 7f42a6ffd700 1 -- 172.21.15.78:0/3362380737 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+4656 (secure 0 0 0) 0x7f42ac021090 con 0x7f42b806d060 2023-10-11T09:52:28.753 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:52:28.753 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":111,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8452,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/2481257990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2481257990},{"type":"v1","addr":"172.21.15.78:6836","nonce":2481257990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":8464,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1035227282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1035227282},{"type":"v1","addr":"172.21.15.133:6835","nonce":1035227282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":8680,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/36737634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":36737634},{"type":"v1","addr":"172.21.15.133:6833","nonce":36737634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":101}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:51:40.211859+0000","modified":"2023-10-11T09:52:28.496625+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9336},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9336":{"gid":9336,"name":"c","rank":0,"incarnation":109,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.78:6837/3884456900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":3884456900},{"type":"v1","addr":"172.21.15.78:6837","nonce":3884456900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-11T09:52:28.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.747+0000 7f42b759e700 1 -- 172.21.15.78:0/3362380737 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4298065ed0 msgr2=0x7f4298068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:28.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.747+0000 7f42b759e700 1 --2- 172.21.15.78:0/3362380737 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4298065ed0 0x7f4298068380 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f42a80090b0 tx=0x7f42a801c040 comp rx=0 tx=0).stop 2023-10-11T09:52:28.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.747+0000 7f42b759e700 1 -- 172.21.15.78:0/3362380737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42b806d060 msgr2=0x7f42b806a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:28.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.747+0000 7f42b759e700 1 --2- 172.21.15.78:0/3362380737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42b806d060 0x7f42b806a180 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f42ac00e9b0 tx=0x7f42ac00ecc0 comp rx=0 tx=0).stop 2023-10-11T09:52:28.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.747+0000 7f42b759e700 1 -- 172.21.15.78:0/3362380737 shutdown_connections 2023-10-11T09:52:28.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.747+0000 7f42b759e700 1 --2- 172.21.15.78:0/3362380737 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4298065ed0 0x7f4298068380 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:28.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.747+0000 7f42b759e700 1 --2- 172.21.15.78:0/3362380737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42b815a800 0x7f42b806a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:28.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.747+0000 7f42b759e700 1 --2- 172.21.15.78:0/3362380737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42b806d060 0x7f42b806a180 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:28.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.747+0000 7f42b759e700 1 --2- 172.21.15.78:0/3362380737 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42b806bb60 0x7f42b80b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:28.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.747+0000 7f42b759e700 1 -- 172.21.15.78:0/3362380737 >> 172.21.15.78:0/3362380737 conn(0x7f42b80cf820 msgr2=0x7f42b81e5850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:28.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.747+0000 7f42b759e700 1 -- 172.21.15.78:0/3362380737 shutdown_connections 2023-10-11T09:52:28.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:28.747+0000 7f42b759e700 1 -- 172.21.15.78:0/3362380737 wait complete. 2023-10-11T09:52:28.759 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 111 2023-10-11T09:52:29.178 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 111, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-11T09:51:40.211859+0000', 'modified': '2023-10-11T09:52:28.496625+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 129, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9336}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9336': {'gid': 9336, 'name': 'c', 'rank': 0, 'incarnation': 109, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.78:6837/3884456900', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6835', 'nonce': 3884456900}, {'type': 'v1', 'addr': '172.21.15.78:6837', 'nonce': 3884456900}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-11T09:52:29.179 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9336:up:rejoin 2023-10-11T09:52:30.180 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:52:30.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.347+0000 7f6b4d4ee700 1 Processor -- start 2023-10-11T09:52:30.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.351+0000 7f6b4d4ee700 1 -- start start 2023-10-11T09:52:30.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.351+0000 7f6b4d4ee700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6b4815a800 0x7f6b480b4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:30.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.351+0000 7f6b4d4ee700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f6b4806bb60 0x7f6b480b0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:30.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.351+0000 7f6b4d4ee700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b48070dc0 con 0x7f6b4806d060 2023-10-11T09:52:30.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.351+0000 7f6b4d4ee700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b480b46a0 con 0x7f6b4806bb60 2023-10-11T09:52:30.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.351+0000 7f6b4d4ee700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b48070ff0 con 0x7f6b4815a800 2023-10-11T09:52:30.360 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.351+0000 7f6b467fc700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f6b4806bb60 0x7f6b480b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:30.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.351+0000 7f6b467fc700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f6b4806bb60 0x7f6b480b0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50528/0 (socket says 172.21.15.78:50528) 2023-10-11T09:52:30.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.351+0000 7f6b467fc700 1 -- 172.21.15.78:0/1970721409 learned_addr learned my addr 172.21.15.78:0/1970721409 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:30.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.351+0000 7f6b46ffd700 1 --2- 172.21.15.78:0/1970721409 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6b4815a800 0x7f6b480b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:30.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.351+0000 7f6b467fc700 1 -- 172.21.15.78:0/1970721409 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6b4815a800 msgr2=0x7f6b480b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:30.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.351+0000 7f6b467fc700 1 --2- 172.21.15.78:0/1970721409 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6b4815a800 0x7f6b480b4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:30.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.351+0000 7f6b467fc700 1 -- 172.21.15.78:0/1970721409 >> v1:172.21.15.133:6789/0 conn(0x7f6b4806d060 legacy=0x7f6b480c9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:30.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.351+0000 7f6b467fc700 1 -- 172.21.15.78:0/1970721409 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b480712d0 con 0x7f6b4806bb60 2023-10-11T09:52:30.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.351+0000 7f6b467fc700 1 --2- 172.21.15.78:0/1970721409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b4806bb60 0x7f6b480b0610 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c009910 tx=0x7f6b3c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf853fdd2c35f23c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:30.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.351+0000 7f6b45ffb700 1 -- 172.21.15.78:0/1970721409 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c005e50 con 0x7f6b4806bb60 2023-10-11T09:52:30.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.355+0000 7f6b45ffb700 1 -- 172.21.15.78:0/1970721409 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b3c009e60 con 0x7f6b4806bb60 2023-10-11T09:52:30.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.355+0000 7f6b45ffb700 1 -- 172.21.15.78:0/1970721409 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c00b870 con 0x7f6b4806bb60 2023-10-11T09:52:30.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.355+0000 7f6b4d4ee700 1 -- 172.21.15.78:0/1970721409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b4806bb60 msgr2=0x7f6b480b0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:30.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.355+0000 7f6b4d4ee700 1 --2- 172.21.15.78:0/1970721409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b4806bb60 0x7f6b480b0610 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c009910 tx=0x7f6b3c009c20 comp rx=0 tx=0).stop 2023-10-11T09:52:30.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.355+0000 7f6b4d4ee700 1 -- 172.21.15.78:0/1970721409 shutdown_connections 2023-10-11T09:52:30.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.355+0000 7f6b4d4ee700 1 --2- 172.21.15.78:0/1970721409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b4806bb60 0x7f6b480b0610 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:30.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.355+0000 7f6b4d4ee700 1 --2- 172.21.15.78:0/1970721409 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6b4815a800 0x7f6b480b4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:30.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.355+0000 7f6b4d4ee700 1 -- 172.21.15.78:0/1970721409 >> 172.21.15.78:0/1970721409 conn(0x7f6b480cf820 msgr2=0x7f6b480cb610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:30.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.355+0000 7f6b4d4ee700 1 -- 172.21.15.78:0/1970721409 shutdown_connections 2023-10-11T09:52:30.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.355+0000 7f6b4d4ee700 1 -- 172.21.15.78:0/1970721409 wait complete. 2023-10-11T09:52:30.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.355+0000 7f6b4d4ee700 1 Processor -- start 2023-10-11T09:52:30.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.359+0000 7f6b4d4ee700 1 -- start start 2023-10-11T09:52:30.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.359+0000 7f6b4d4ee700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6b4806bb60 0x7f6b48157c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:30.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.359+0000 7f6b4d4ee700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b4806d060 0x7f6b481510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:30.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.359+0000 7f6b4d4ee700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b4815a800 0x7f6b48155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:30.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.359+0000 7f6b4d4ee700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b48151600 con 0x7f6b4806d060 2023-10-11T09:52:30.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.359+0000 7f6b4d4ee700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b480c4530 con 0x7f6b4815a800 2023-10-11T09:52:30.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.359+0000 7f6b4d4ee700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b480c4810 con 0x7f6b4806bb60 2023-10-11T09:52:30.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.359+0000 7f6b467fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b4806d060 0x7f6b481510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:30.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.359+0000 7f6b467fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b4806d060 0x7f6b481510c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50530/0 (socket says 172.21.15.78:50530) 2023-10-11T09:52:30.368 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.359+0000 7f6b467fc700 1 -- 172.21.15.78:0/1868343038 learned_addr learned my addr 172.21.15.78:0/1868343038 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:30.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.359+0000 7f6b46ffd700 1 --2- 172.21.15.78:0/1868343038 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6b4806bb60 0x7f6b48157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:30.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.359+0000 7f6b477fe700 1 --2- 172.21.15.78:0/1868343038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b4815a800 0x7f6b48155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:30.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.359+0000 7f6b467fc700 1 -- 172.21.15.78:0/1868343038 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6b4806bb60 msgr2=0x7f6b48157c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:30.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.359+0000 7f6b467fc700 1 --2- 172.21.15.78:0/1868343038 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6b4806bb60 0x7f6b48157c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:30.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.359+0000 7f6b467fc700 1 -- 172.21.15.78:0/1868343038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b4815a800 msgr2=0x7f6b48155ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:30.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.359+0000 7f6b467fc700 1 --2- 172.21.15.78:0/1868343038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b4815a800 0x7f6b48155ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:30.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.359+0000 7f6b467fc700 1 -- 172.21.15.78:0/1868343038 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b3c009520 con 0x7f6b4806d060 2023-10-11T09:52:30.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.359+0000 7f6b477fe700 1 --2- 172.21.15.78:0/1868343038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b4815a800 0x7f6b48155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:52:30.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.359+0000 7f6b467fc700 1 --2- 172.21.15.78:0/1868343038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b4806d060 0x7f6b481510c0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c009c60 tx=0x7f6b3c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:30.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.359+0000 7f6b2f7fe700 1 -- 172.21.15.78:0/1868343038 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c00be20 con 0x7f6b4806d060 2023-10-11T09:52:30.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.359+0000 7f6b2f7fe700 1 -- 172.21.15.78:0/1868343038 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b3c020070 con 0x7f6b4806d060 2023-10-11T09:52:30.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.359+0000 7f6b4d4ee700 1 -- 172.21.15.78:0/1868343038 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b480c4af0 con 0x7f6b4806d060 2023-10-11T09:52:30.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.359+0000 7f6b2f7fe700 1 -- 172.21.15.78:0/1868343038 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c006220 con 0x7f6b4806d060 2023-10-11T09:52:30.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.359+0000 7f6b4d4ee700 1 -- 172.21.15.78:0/1868343038 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b480c50b0 con 0x7f6b4806d060 2023-10-11T09:52:30.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.363+0000 7f6b4d4ee700 1 -- 172.21.15.78:0/1868343038 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b48067d00 con 0x7f6b4806d060 2023-10-11T09:52:30.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.363+0000 7f6b2f7fe700 1 -- 172.21.15.78:0/1868343038 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f6b3c022030 con 0x7f6b4806d060 2023-10-11T09:52:30.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.367+0000 7f6b2f7fe700 1 --2- 172.21.15.78:0/1868343038 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6b34065ed0 0x7f6b34068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:30.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.367+0000 7f6b2f7fe700 1 -- 172.21.15.78:0/1868343038 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f6b3c084f10 con 0x7f6b4806d060 2023-10-11T09:52:30.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.367+0000 7f6b46ffd700 1 --2- 172.21.15.78:0/1868343038 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6b34065ed0 0x7f6b34068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:30.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.367+0000 7f6b46ffd700 1 --2- 172.21.15.78:0/1868343038 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6b34065ed0 0x7f6b34068380 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f6b480b3f50 tx=0x7f6b38008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:30.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.371+0000 7f6b2f7fe700 1 -- 172.21.15.78:0/1868343038 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f6b3c052520 con 0x7f6b4806d060 2023-10-11T09:52:30.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.587+0000 7f6b4d4ee700 1 -- 172.21.15.78:0/1868343038 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b48065d40 con 0x7f6b4806d060 2023-10-11T09:52:30.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.587+0000 7f6b2f7fe700 1 -- 172.21.15.78:0/1868343038 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4656 (secure 0 0 0) 0x7f6b3c055b40 con 0x7f6b4806d060 2023-10-11T09:52:30.594 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:52:30.594 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":112,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8452,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/2481257990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2481257990},{"type":"v1","addr":"172.21.15.78:6836","nonce":2481257990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":8464,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1035227282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1035227282},{"type":"v1","addr":"172.21.15.133:6835","nonce":1035227282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":8680,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/36737634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":36737634},{"type":"v1","addr":"172.21.15.133:6833","nonce":36737634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":101}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:51:40.211859+0000","modified":"2023-10-11T09:52:29.509275+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9336},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9336":{"gid":9336,"name":"c","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.78:6837/3884456900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":3884456900},{"type":"v1","addr":"172.21.15.78:6837","nonce":3884456900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-11T09:52:30.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.591+0000 7f6b4d4ee700 1 -- 172.21.15.78:0/1868343038 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6b34065ed0 msgr2=0x7f6b34068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:30.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.591+0000 7f6b4d4ee700 1 --2- 172.21.15.78:0/1868343038 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6b34065ed0 0x7f6b34068380 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f6b480b3f50 tx=0x7f6b38008040 comp rx=0 tx=0).stop 2023-10-11T09:52:30.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.591+0000 7f6b4d4ee700 1 -- 172.21.15.78:0/1868343038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b4806d060 msgr2=0x7f6b481510c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:30.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.591+0000 7f6b4d4ee700 1 --2- 172.21.15.78:0/1868343038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b4806d060 0x7f6b481510c0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c009c60 tx=0x7f6b3c004770 comp rx=0 tx=0).stop 2023-10-11T09:52:30.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.591+0000 7f6b4d4ee700 1 -- 172.21.15.78:0/1868343038 shutdown_connections 2023-10-11T09:52:30.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.591+0000 7f6b4d4ee700 1 --2- 172.21.15.78:0/1868343038 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6b34065ed0 0x7f6b34068380 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:30.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.591+0000 7f6b4d4ee700 1 --2- 172.21.15.78:0/1868343038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b4815a800 0x7f6b48155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:30.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.591+0000 7f6b4d4ee700 1 --2- 172.21.15.78:0/1868343038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b4806d060 0x7f6b481510c0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:30.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.591+0000 7f6b4d4ee700 1 --2- 172.21.15.78:0/1868343038 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6b4806bb60 0x7f6b48157c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:30.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.591+0000 7f6b4d4ee700 1 -- 172.21.15.78:0/1868343038 >> 172.21.15.78:0/1868343038 conn(0x7f6b480cf820 msgr2=0x7f6b480b35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:30.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.591+0000 7f6b4d4ee700 1 -- 172.21.15.78:0/1868343038 shutdown_connections 2023-10-11T09:52:30.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:30.591+0000 7f6b4d4ee700 1 -- 172.21.15.78:0/1868343038 wait complete. 2023-10-11T09:52:30.601 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 112 2023-10-11T09:52:30.992 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:52:30.999 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:52:31.023 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 112, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-11T09:51:40.211859+0000', 'modified': '2023-10-11T09:52:29.509275+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 129, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9336}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9336': {'gid': 9336, 'name': 'c', 'rank': 0, 'incarnation': 109, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.78:6837/3884456900', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6835', 'nonce': 3884456900}, {'type': 'v1', 'addr': '172.21.15.78:6837', 'nonce': 3884456900}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-11T09:52:31.024 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-11T09:52:31.024 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-11T09:52:31.218 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.211+0000 7fed9359e700 1 Processor -- start 2023-10-11T09:52:31.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.211+0000 7fed9359e700 1 -- start start 2023-10-11T09:52:31.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.215+0000 7fed9359e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed9406ca50 0x7fed940cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:31.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.215+0000 7fed9359e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fed940b0d20 0x7fed94159760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:31.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.215+0000 7fed9359e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed9415a4d0 con 0x7fed94156cf0 2023-10-11T09:52:31.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.215+0000 7fed9359e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed9406bc90 con 0x7fed940b0d20 2023-10-11T09:52:31.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.215+0000 7fed9359e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed9406bf70 con 0x7fed9406ca50 2023-10-11T09:52:31.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.215+0000 7fed91d9b700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fed94156cf0 0x7fed940b0610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:57016/0 (socket says 172.21.15.78:57016) 2023-10-11T09:52:31.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.215+0000 7fed91d9b700 1 -- 172.21.15.78:0/1611748764 learned_addr learned my addr 172.21.15.78:0/1611748764 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:31.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.215+0000 7fed92d9d700 1 --2- 172.21.15.78:0/1611748764 >> v2:172.21.15.133:3300/0 conn(0x7fed940b0d20 0x7fed94159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:31.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.215+0000 7fed9259c700 1 --2- 172.21.15.78:0/1611748764 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed9406ca50 0x7fed940cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:31.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.215+0000 7fed9159a700 1 -- 172.21.15.78:0/1611748764 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4039838660 0 0) 0x7fed9415a4d0 con 0x7fed94156cf0 2023-10-11T09:52:31.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.215+0000 7fed9159a700 1 -- 172.21.15.78:0/1611748764 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed7c0035d0 con 0x7fed94156cf0 2023-10-11T09:52:31.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.215+0000 7fed92d9d700 1 -- 172.21.15.78:0/1611748764 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed9406ca50 msgr2=0x7fed940cbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:31.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.215+0000 7fed92d9d700 1 --2- 172.21.15.78:0/1611748764 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed9406ca50 0x7fed940cbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.215+0000 7fed92d9d700 1 -- 172.21.15.78:0/1611748764 >> v1:172.21.15.78:6789/0 conn(0x7fed94156cf0 legacy=0x7fed940b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:31.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.215+0000 7fed92d9d700 1 -- 172.21.15.78:0/1611748764 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed9406c250 con 0x7fed940b0d20 2023-10-11T09:52:31.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.215+0000 7fed92d9d700 1 --2- 172.21.15.78:0/1611748764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed940b0d20 0x7fed94159760 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fed8c009d90 tx=0x7fed8c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9ae59cf445ea4ab server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:31.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.215+0000 7fed9159a700 1 -- 172.21.15.78:0/1611748764 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8c00bd20 con 0x7fed940b0d20 2023-10-11T09:52:31.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.215+0000 7fed9159a700 1 -- 172.21.15.78:0/1611748764 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed8c00be80 con 0x7fed940b0d20 2023-10-11T09:52:31.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.219+0000 7fed9159a700 1 -- 172.21.15.78:0/1611748764 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8c0116a0 con 0x7fed940b0d20 2023-10-11T09:52:31.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.219+0000 7fed9359e700 1 -- 172.21.15.78:0/1611748764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed940b0d20 msgr2=0x7fed94159760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:31.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.219+0000 7fed9359e700 1 --2- 172.21.15.78:0/1611748764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed940b0d20 0x7fed94159760 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fed8c009d90 tx=0x7fed8c00b330 comp rx=0 tx=0).stop 2023-10-11T09:52:31.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.219+0000 7fed9359e700 1 -- 172.21.15.78:0/1611748764 shutdown_connections 2023-10-11T09:52:31.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.219+0000 7fed9359e700 1 --2- 172.21.15.78:0/1611748764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed940b0d20 0x7fed94159760 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.219+0000 7fed9359e700 1 --2- 172.21.15.78:0/1611748764 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed9406ca50 0x7fed940cbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.219+0000 7fed9359e700 1 -- 172.21.15.78:0/1611748764 >> 172.21.15.78:0/1611748764 conn(0x7fed940cf820 msgr2=0x7fed940cb610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:31.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.219+0000 7fed9359e700 1 -- 172.21.15.78:0/1611748764 shutdown_connections 2023-10-11T09:52:31.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.219+0000 7fed9359e700 1 -- 172.21.15.78:0/1611748764 wait complete. 2023-10-11T09:52:31.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.219+0000 7fed9359e700 1 Processor -- start 2023-10-11T09:52:31.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.219+0000 7fed9359e700 1 -- start start 2023-10-11T09:52:31.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.223+0000 7fed9359e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed9406ca50 0x7fed94157310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:31.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.223+0000 7fed9359e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed940b0d20 0x7fed940710a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:31.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.223+0000 7fed9359e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed94156cf0 0x7fed94146f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:31.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.223+0000 7fed9359e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed940ae660 con 0x7fed940b0d20 2023-10-11T09:52:31.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.223+0000 7fed9359e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed94157850 con 0x7fed94156cf0 2023-10-11T09:52:31.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.223+0000 7fed9359e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed940715e0 con 0x7fed9406ca50 2023-10-11T09:52:31.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.223+0000 7fed91d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed940b0d20 0x7fed940710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:31.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.223+0000 7fed92d9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed94156cf0 0x7fed94146f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:31.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.223+0000 7fed91d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed940b0d20 0x7fed940710a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50532/0 (socket says 172.21.15.78:50532) 2023-10-11T09:52:31.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.223+0000 7fed91d9b700 1 -- 172.21.15.78:0/3584982661 learned_addr learned my addr 172.21.15.78:0/3584982661 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:31.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.223+0000 7fed9259c700 1 --2- 172.21.15.78:0/3584982661 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed9406ca50 0x7fed94157310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:31.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.223+0000 7fed91d9b700 1 -- 172.21.15.78:0/3584982661 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed9406ca50 msgr2=0x7fed94157310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:31.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.223+0000 7fed91d9b700 1 --2- 172.21.15.78:0/3584982661 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed9406ca50 0x7fed94157310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.223+0000 7fed91d9b700 1 -- 172.21.15.78:0/3584982661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed94156cf0 msgr2=0x7fed94146f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:31.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.223+0000 7fed91d9b700 1 --2- 172.21.15.78:0/3584982661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed94156cf0 0x7fed94146f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.223+0000 7fed91d9b700 1 -- 172.21.15.78:0/3584982661 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed8c009a70 con 0x7fed940b0d20 2023-10-11T09:52:31.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.223+0000 7fed92d9d700 1 --2- 172.21.15.78:0/3584982661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed94156cf0 0x7fed94146f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:52:31.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.223+0000 7fed91d9b700 1 --2- 172.21.15.78:0/3584982661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed940b0d20 0x7fed940710a0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fed8800c480 tx=0x7fed8800c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:31.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.223+0000 7fed82ffd700 1 -- 172.21.15.78:0/3584982661 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed88005d90 con 0x7fed940b0d20 2023-10-11T09:52:31.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.223+0000 7fed82ffd700 1 -- 172.21.15.78:0/3584982661 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed8800ce50 con 0x7fed940b0d20 2023-10-11T09:52:31.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.223+0000 7fed9359e700 1 -- 172.21.15.78:0/3584982661 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed94071810 con 0x7fed940b0d20 2023-10-11T09:52:31.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.223+0000 7fed82ffd700 1 -- 172.21.15.78:0/3584982661 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed88007310 con 0x7fed940b0d20 2023-10-11T09:52:31.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.223+0000 7fed9359e700 1 -- 172.21.15.78:0/3584982661 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed94159b20 con 0x7fed940b0d20 2023-10-11T09:52:31.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.227+0000 7fed82ffd700 1 -- 172.21.15.78:0/3584982661 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fed88007470 con 0x7fed940b0d20 2023-10-11T09:52:31.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.231+0000 7fed82ffd700 1 --2- 172.21.15.78:0/3584982661 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fed74065ed0 0x7fed74068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:31.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.231+0000 7fed82ffd700 1 -- 172.21.15.78:0/3584982661 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6234+0+0 (secure 0 0 0) 0x7fed88082990 con 0x7fed940b0d20 2023-10-11T09:52:31.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.231+0000 7fed9259c700 1 --2- 172.21.15.78:0/3584982661 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fed74065ed0 0x7fed74068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:31.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.231+0000 7fed9259c700 1 --2- 172.21.15.78:0/3584982661 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fed74065ed0 0x7fed74068380 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fed84009bf0 tx=0x7fed84009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:31.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.231+0000 7fed9bea3740 1 Processor -- start 2023-10-11T09:52:31.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.231+0000 7fed9bea3740 1 -- start start 2023-10-11T09:52:31.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.235+0000 7fed9bea3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x290aeb0 0x2bc3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:31.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.235+0000 7fed9bea3740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x290aae0 0x2bc3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:31.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.235+0000 7fed9bea3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x290a710 0x2b8a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:31.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.235+0000 7fed9bea3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x2aea4a0 con 0x290aeb0 2023-10-11T09:52:31.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.235+0000 7fed9bea3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x2ae7c30 con 0x290a710 2023-10-11T09:52:31.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.235+0000 7fed9bea3740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x2ae3520 con 0x290aae0 2023-10-11T09:52:31.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.235+0000 7fed9259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x290aeb0 0x2bc3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:31.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.235+0000 7fed9259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x290aeb0 0x2bc3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50534/0 (socket says 172.21.15.78:50534) 2023-10-11T09:52:31.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.235+0000 7fed9259c700 1 -- 172.21.15.78:0/3347085152 learned_addr learned my addr 172.21.15.78:0/3347085152 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:31.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.235+0000 7fed92d9d700 1 --2- 172.21.15.78:0/3347085152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x290a710 0x2b8a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:31.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.235+0000 7fed9259c700 1 -- 172.21.15.78:0/3347085152 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x290aae0 msgr2=0x2bc3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:31.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.235+0000 7fed91d9b700 1 --2- 172.21.15.78:0/3347085152 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x290aae0 0x2bc3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:31.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.235+0000 7fed9259c700 1 --2- 172.21.15.78:0/3347085152 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x290aae0 0x2bc3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.235+0000 7fed9259c700 1 -- 172.21.15.78:0/3347085152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x290a710 msgr2=0x2b8a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:31.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.235+0000 7fed9259c700 1 --2- 172.21.15.78:0/3347085152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x290a710 0x2b8a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.235+0000 7fed9259c700 1 -- 172.21.15.78:0/3347085152 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x29569e0 con 0x290aeb0 2023-10-11T09:52:31.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.235+0000 7fed92d9d700 1 --2- 172.21.15.78:0/3347085152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x290a710 0x2b8a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:52:31.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.235+0000 7fed91d9b700 1 --2- 172.21.15.78:0/3347085152 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x290aae0 0x2bc3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:52:31.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.235+0000 7fed9259c700 1 --2- 172.21.15.78:0/3347085152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x290aeb0 0x2bc3020 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fed840044f0 tx=0x7fed84014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1625ff8dd8c1e16b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:31.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.235+0000 7fed73fff700 1 -- 172.21.15.78:0/3347085152 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8401c040 con 0x290aeb0 2023-10-11T09:52:31.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.235+0000 7fed73fff700 1 -- 172.21.15.78:0/3347085152 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed840038d0 con 0x290aeb0 2023-10-11T09:52:31.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.235+0000 7fed73fff700 1 -- 172.21.15.78:0/3347085152 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed84003c40 con 0x290aeb0 2023-10-11T09:52:31.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.235+0000 7fed9bea3740 1 -- 172.21.15.78:0/3347085152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x290aeb0 msgr2=0x2bc3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:31.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.235+0000 7fed9bea3740 1 --2- 172.21.15.78:0/3347085152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x290aeb0 0x2bc3020 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fed840044f0 tx=0x7fed84014000 comp rx=0 tx=0).stop 2023-10-11T09:52:31.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.235+0000 7fed9bea3740 1 -- 172.21.15.78:0/3347085152 shutdown_connections 2023-10-11T09:52:31.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.235+0000 7fed9bea3740 1 --2- 172.21.15.78:0/3347085152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x290a710 0x2b8a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.235+0000 7fed9bea3740 1 --2- 172.21.15.78:0/3347085152 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x290aae0 0x2bc3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.235+0000 7fed9bea3740 1 --2- 172.21.15.78:0/3347085152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x290aeb0 0x2bc3020 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.235+0000 7fed9bea3740 1 -- 172.21.15.78:0/3347085152 >> 172.21.15.78:0/3347085152 conn(0x2b72fa0 msgr2=0x2ba4420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:31.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.235+0000 7fed9bea3740 1 -- 172.21.15.78:0/3347085152 shutdown_connections 2023-10-11T09:52:31.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.235+0000 7fed9bea3740 1 -- 172.21.15.78:0/3347085152 wait complete. 2023-10-11T09:52:31.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.239+0000 7fed9bea3740 1 -- start start 2023-10-11T09:52:31.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.239+0000 7fed9bea3740 1 Processor -- start 2023-10-11T09:52:31.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.239+0000 7fed9bea3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x290a710 0x2b78ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:31.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.239+0000 7fed9bea3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x290aae0 0x2bb4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:31.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.239+0000 7fed9bea3740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x290aeb0 0x2bb5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:31.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.239+0000 7fed9bea3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x2aa6b00 con 0x290aae0 2023-10-11T09:52:31.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.239+0000 7fed9bea3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x2aa67a0 con 0x290a710 2023-10-11T09:52:31.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.239+0000 7fed9bea3740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x298fb20 con 0x290aeb0 2023-10-11T09:52:31.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.239+0000 7fed9bea3740 10 client.0 _ll_register_callbacks cb 0x2adce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:52:31.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.239+0000 7fed91d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x290aae0 0x2bb4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:31.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.239+0000 7fed91d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x290aae0 0x2bb4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50538/0 (socket says 172.21.15.78:50538) 2023-10-11T09:52:31.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.239+0000 7fed91d9b700 1 -- 172.21.15.78:0/2329933518 learned_addr learned my addr 172.21.15.78:0/2329933518 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:31.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.239+0000 7fed9259c700 1 --2- 172.21.15.78:0/2329933518 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x290a710 0x2b78ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:31.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.239+0000 7fed92d9d700 1 --2- 172.21.15.78:0/2329933518 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x290aeb0 0x2bb5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:31.255 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.239+0000 7fed91d9b700 1 -- 172.21.15.78:0/2329933518 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x290aeb0 msgr2=0x2bb5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:31.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.239+0000 7fed91d9b700 1 --2- 172.21.15.78:0/2329933518 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x290aeb0 0x2bb5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.239+0000 7fed91d9b700 1 -- 172.21.15.78:0/2329933518 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x290a710 msgr2=0x2b78ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:31.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.239+0000 7fed91d9b700 1 --2- 172.21.15.78:0/2329933518 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x290a710 0x2b78ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.239+0000 7fed91d9b700 1 -- 172.21.15.78:0/2329933518 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed840041d0 con 0x290aae0 2023-10-11T09:52:31.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.239+0000 7fed9259c700 1 --2- 172.21.15.78:0/2329933518 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x290a710 0x2b78ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:52:31.256 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.239+0000 7fed91d9b700 1 --2- 172.21.15.78:0/2329933518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x290aae0 0x2bb4f00 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fed8805b580 tx=0x7fed88054010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b3e9655e70b372 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:31.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.239+0000 7fed717fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:52:31.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.239+0000 7fed717fa700 1 -- 172.21.15.78:0/2329933518 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed88057900 con 0x290aae0 2023-10-11T09:52:31.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.243+0000 7fed717fa700 1 -- 172.21.15.78:0/2329933518 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 6234+0+0 (secure 0 0 0) 0x7fed8805bab0 con 0x290aae0 2023-10-11T09:52:31.257 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.243+0000 7fed9bea3740 1 -- 172.21.15.78:0/2329933518 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x2adc5a0 con 0x290aae0 2023-10-11T09:52:31.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.243+0000 7fed717fa700 1 -- 172.21.15.78:0/2329933518 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed88070120 con 0x290aae0 2023-10-11T09:52:31.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.243+0000 7fed9bea3740 10 client.9402 fetch_fsmap learned FSMap version 112 2023-10-11T09:52:31.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.243+0000 7fed9bea3740 1 -- 172.21.15.78:0/2329933518 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) v3 -- 0x2aa9750 con 0x290aae0 2023-10-11T09:52:31.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.243+0000 7fed717fa700 1 -- 172.21.15.78:0/2329933518 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x2adc5a0 con 0x290aae0 2023-10-11T09:52:31.258 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.243+0000 7fed717fa700 1 -- 172.21.15.78:0/2329933518 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 112) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fed8805b620 con 0x290aae0 2023-10-11T09:52:31.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.243+0000 7fed9bea3740 10 client.9402 fetch_fsmap finished waiting for FSMap version 112 2023-10-11T09:52:31.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.243+0000 7fed9bea3740 10 client.9402 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-11T09:52:31.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.243+0000 7fed9bea3740 20 client.9402 populate_metadata read hostname 'smithi078' 2023-10-11T09:52:31.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.243+0000 7fed9bea3740 1 --2- 172.21.15.78:0/2329933518 >> [v2:172.21.15.78:6835/3884456900,v1:172.21.15.78:6837/3884456900] conn(0x2955790 0x2b78120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:31.259 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.243+0000 7fed9bea3740 4 client.9402 mds_command: new command op to 9336 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-11T09:52:31.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.243+0000 7fed9bea3740 1 -- 172.21.15.78:0/2329933518 --> [v2:172.21.15.78:6835/3884456900,v1:172.21.15.78:6837/3884456900] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x2adc2c0 con 0x2955790 2023-10-11T09:52:31.260 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.243+0000 7fed9259c700 1 --2- 172.21.15.78:0/2329933518 >> [v2:172.21.15.78:6835/3884456900,v1:172.21.15.78:6837/3884456900] conn(0x2955790 0x2b78120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:31.261 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.243+0000 7fed9259c700 1 --2- 172.21.15.78:0/2329933518 >> [v2:172.21.15.78:6835/3884456900,v1:172.21.15.78:6837/3884456900] conn(0x2955790 0x2b78120 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98313b47af936278 server_cookie=d38b6803fc8986a3 in_seq=0 out_seq=0 2023-10-11T09:52:31.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.243+0000 7fed717fa700 10 client.9402 ms_handle_connect on v2:172.21.15.78:6835/3884456900 2023-10-11T09:52:31.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.243+0000 7fed717fa700 1 -- 172.21.15.78:0/2329933518 <== mds.0 v2:172.21.15.78:6835/3884456900 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x2adc2c0 con 0x2955790 2023-10-11T09:52:31.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.243+0000 7fed717fa700 10 client.9402 handle_command_reply: tid=0 2023-10-11T09:52:31.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.243+0000 7fed9bea3740 1 client.9402 shutdown 2023-10-11T09:52:31.262 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.247+0000 7fed9bea3740 1 -- 172.21.15.78:0/2329933518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x290aae0 msgr2=0x2bb4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:31.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.247+0000 7fed9bea3740 1 --2- 172.21.15.78:0/2329933518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x290aae0 0x2bb4f00 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fed8805b580 tx=0x7fed88054010 comp rx=0 tx=0).stop 2023-10-11T09:52:31.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.247+0000 7fed9bea3740 1 -- 172.21.15.78:0/2329933518 shutdown_connections 2023-10-11T09:52:31.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.247+0000 7fed9bea3740 1 --2- 172.21.15.78:0/2329933518 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x290aeb0 0x2bb5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.247+0000 7fed9bea3740 1 --2- 172.21.15.78:0/2329933518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x290aae0 0x2bb4f00 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.263 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.247+0000 7fed9bea3740 1 --2- 172.21.15.78:0/2329933518 >> [v2:172.21.15.78:6835/3884456900,v1:172.21.15.78:6837/3884456900] conn(0x2955790 0x2b78120 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.247+0000 7fed9bea3740 1 --2- 172.21.15.78:0/2329933518 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x290a710 0x2b78ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.247+0000 7fed9bea3740 1 -- 172.21.15.78:0/2329933518 >> 172.21.15.78:0/2329933518 conn(0x2b72fa0 msgr2=0x2b7d010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:31.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.247+0000 7fed717fa700 0 client.9402 ms_handle_reset on v2:172.21.15.78:6835/3884456900 2023-10-11T09:52:31.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.247+0000 7fed9bea3740 1 -- 172.21.15.78:0/2329933518 shutdown_connections 2023-10-11T09:52:31.264 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.247+0000 7fed9bea3740 1 -- 172.21.15.78:0/2329933518 wait complete. 2023-10-11T09:52:31.265 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.247+0000 7fed9bea3740 20 client.9402 trim_cache size 0 max 16384 2023-10-11T09:52:31.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.267+0000 7fed9bea3740 1 Processor -- start 2023-10-11T09:52:31.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.267+0000 7fed9bea3740 1 -- start start 2023-10-11T09:52:31.275 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.267+0000 7fed9bea3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x290a710 0x2ba1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:31.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.267+0000 7fed9bea3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x290aae0 0x2bb6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:31.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.267+0000 7fed9bea3740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x290aeb0 0x2b77fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:31.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.267+0000 7fed9bea3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x2ae4ab0 con 0x290aae0 2023-10-11T09:52:31.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.267+0000 7fed9bea3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a9d760 con 0x290a710 2023-10-11T09:52:31.276 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.267+0000 7fed9bea3740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x2a8b550 con 0x290aeb0 2023-10-11T09:52:31.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.267+0000 7fed91d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x290aae0 0x2bb6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:31.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.267+0000 7fed91d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x290aae0 0x2bb6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50548/0 (socket says 172.21.15.78:50548) 2023-10-11T09:52:31.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.267+0000 7fed91d9b700 1 -- 172.21.15.78:0/632608493 learned_addr learned my addr 172.21.15.78:0/632608493 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:31.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.267+0000 7fed9259c700 1 --2- 172.21.15.78:0/632608493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x290a710 0x2ba1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:31.277 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.267+0000 7fed92d9d700 1 --2- 172.21.15.78:0/632608493 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x290aeb0 0x2b77fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:31.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.267+0000 7fed91d9b700 1 -- 172.21.15.78:0/632608493 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x290aeb0 msgr2=0x2b77fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:31.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.267+0000 7fed91d9b700 1 --2- 172.21.15.78:0/632608493 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x290aeb0 0x2b77fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.267+0000 7fed91d9b700 1 -- 172.21.15.78:0/632608493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x290a710 msgr2=0x2ba1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:31.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.267+0000 7fed91d9b700 1 --2- 172.21.15.78:0/632608493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x290a710 0x2ba1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.267+0000 7fed91d9b700 1 -- 172.21.15.78:0/632608493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed94154400 con 0x290aae0 2023-10-11T09:52:31.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.267+0000 7fed9259c700 1 --2- 172.21.15.78:0/632608493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x290a710 0x2ba1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:52:31.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.267+0000 7fed91d9b700 1 --2- 172.21.15.78:0/632608493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x290aae0 0x2bb6090 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fed880537d0 tx=0x7fed88071000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd313bfae206ab3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:31.280 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.271+0000 7fed80ff9700 1 -- 172.21.15.78:0/632608493 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8805bab0 con 0x290aae0 2023-10-11T09:52:31.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.271+0000 7fed80ff9700 1 -- 172.21.15.78:0/632608493 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed88057900 con 0x290aae0 2023-10-11T09:52:31.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.271+0000 7fed80ff9700 1 -- 172.21.15.78:0/632608493 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed88070120 con 0x290aae0 2023-10-11T09:52:31.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.271+0000 7fed9bea3740 1 -- 172.21.15.78:0/632608493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x290aae0 msgr2=0x2bb6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:31.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.271+0000 7fed9bea3740 1 --2- 172.21.15.78:0/632608493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x290aae0 0x2bb6090 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fed880537d0 tx=0x7fed88071000 comp rx=0 tx=0).stop 2023-10-11T09:52:31.281 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.271+0000 7fed9bea3740 1 -- 172.21.15.78:0/632608493 shutdown_connections 2023-10-11T09:52:31.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.271+0000 7fed9bea3740 1 --2- 172.21.15.78:0/632608493 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x290aeb0 0x2b77fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.271+0000 7fed9bea3740 1 --2- 172.21.15.78:0/632608493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x290aae0 0x2bb6090 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.271+0000 7fed9bea3740 1 --2- 172.21.15.78:0/632608493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x290a710 0x2ba1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.271+0000 7fed9bea3740 1 -- 172.21.15.78:0/632608493 >> 172.21.15.78:0/632608493 conn(0x2b72fa0 msgr2=0x2b77160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:31.282 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.271+0000 7fed9bea3740 1 -- 172.21.15.78:0/632608493 shutdown_connections 2023-10-11T09:52:31.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.271+0000 7fed9bea3740 1 -- 172.21.15.78:0/632608493 wait complete. 2023-10-11T09:52:31.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.271+0000 7fed9bea3740 1 -- start start 2023-10-11T09:52:31.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.271+0000 7fed9bea3740 1 Processor -- start 2023-10-11T09:52:31.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.271+0000 7fed9bea3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x290a710 0x2c26d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:31.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.271+0000 7fed9bea3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x290aae0 0x2bb89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:31.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.271+0000 7fed9bea3740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x290aeb0 0x2bb8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:31.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.271+0000 7fed9bea3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x2bb2840 con 0x290a710 2023-10-11T09:52:31.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.271+0000 7fed9bea3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x2c272c0 con 0x290aae0 2023-10-11T09:52:31.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.271+0000 7fed9bea3740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x2bc7340 con 0x290aeb0 2023-10-11T09:52:31.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.271+0000 7fed9bea3740 10 client.0 _ll_register_callbacks cb 0x2ae1c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T09:52:31.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.271+0000 7fed9259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x290a710 0x2c26d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:31.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.275+0000 7fed9259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x290a710 0x2c26d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50552/0 (socket says 172.21.15.78:50552) 2023-10-11T09:52:31.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.275+0000 7fed9259c700 1 -- 172.21.15.78:0/2109864387 learned_addr learned my addr 172.21.15.78:0/2109864387 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:31.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.275+0000 7fed9259c700 1 -- 172.21.15.78:0/2109864387 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x290aeb0 msgr2=0x2bb8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:31.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.275+0000 7fed9259c700 1 --2- 172.21.15.78:0/2109864387 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x290aeb0 0x2bb8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.275+0000 7fed9259c700 1 -- 172.21.15.78:0/2109864387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x290aae0 msgr2=0x2bb89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:31.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.275+0000 7fed91d9b700 1 --2- 172.21.15.78:0/2109864387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x290aae0 0x2bb89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:31.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.275+0000 7fed9259c700 1 --2- 172.21.15.78:0/2109864387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x290aae0 0x2bb89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.275+0000 7fed9259c700 1 -- 172.21.15.78:0/2109864387 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x29569e0 con 0x290a710 2023-10-11T09:52:31.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.275+0000 7fed9259c700 1 --2- 172.21.15.78:0/2109864387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x290a710 0x2c26d80 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fed84006b10 tx=0x7fed840047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bd21ac12e6d2f09 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:31.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.275+0000 7fed717fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-11T09:52:31.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.275+0000 7fed717fa700 1 -- 172.21.15.78:0/2109864387 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed84032040 con 0x290a710 2023-10-11T09:52:31.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.275+0000 7fed717fa700 1 -- 172.21.15.78:0/2109864387 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 6234+0+0 (secure 0 0 0) 0x7fed84042d00 con 0x290a710 2023-10-11T09:52:31.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.275+0000 7fed9bea3740 1 -- 172.21.15.78:0/2109864387 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed840038d0 con 0x290a710 2023-10-11T09:52:31.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.275+0000 7fed717fa700 1 -- 172.21.15.78:0/2109864387 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed840387f0 con 0x290a710 2023-10-11T09:52:31.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.275+0000 7fed9bea3740 10 client.9408 fetch_fsmap learned FSMap version 112 2023-10-11T09:52:31.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.275+0000 7fed9bea3740 1 -- 172.21.15.78:0/2109864387 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) v3 -- 0x2aa9750 con 0x290a710 2023-10-11T09:52:31.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.275+0000 7fed717fa700 1 -- 172.21.15.78:0/2109864387 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed840038d0 con 0x290a710 2023-10-11T09:52:31.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.275+0000 7fed717fa700 1 -- 172.21.15.78:0/2109864387 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 112) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fed84038950 con 0x290a710 2023-10-11T09:52:31.291 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.275+0000 7fed9bea3740 10 client.9408 fetch_fsmap finished waiting for FSMap version 112 2023-10-11T09:52:31.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.275+0000 7fed9bea3740 10 client.9408 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-11T09:52:31.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.275+0000 7fed9bea3740 20 client.9408 populate_metadata read hostname 'smithi078' 2023-10-11T09:52:31.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.275+0000 7fed9bea3740 1 --2- 172.21.15.78:0/2109864387 >> [v2:172.21.15.78:6835/3884456900,v1:172.21.15.78:6837/3884456900] conn(0x2bb1d10 0x2bb20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:31.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.275+0000 7fed9bea3740 4 client.9408 mds_command: new command op to 9336 tid=0[{"prefix": "status"}] 2023-10-11T09:52:31.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.275+0000 7fed9bea3740 1 -- 172.21.15.78:0/2109864387 --> [v2:172.21.15.78:6835/3884456900,v1:172.21.15.78:6837/3884456900] -- command(tid 0: {"prefix": "status"}) v1 -- 0x29624e0 con 0x2bb1d10 2023-10-11T09:52:31.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.275+0000 7fed91d9b700 1 --2- 172.21.15.78:0/2109864387 >> [v2:172.21.15.78:6835/3884456900,v1:172.21.15.78:6837/3884456900] conn(0x2bb1d10 0x2bb20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:31.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.275+0000 7fed717fa700 10 client.9408 ms_handle_connect on v2:172.21.15.78:6835/3884456900 2023-10-11T09:52:31.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.275+0000 7fed91d9b700 1 --2- 172.21.15.78:0/2109864387 >> [v2:172.21.15.78:6835/3884456900,v1:172.21.15.78:6837/3884456900] conn(0x2bb1d10 0x2bb20e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28591fa3c4f47c72 server_cookie=9958589d08e45d38 in_seq=0 out_seq=0 2023-10-11T09:52:31.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.279+0000 7fed717fa700 1 -- 172.21.15.78:0/2109864387 <== mds.0 v2:172.21.15.78:6835/3884456900 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x29624e0 con 0x2bb1d10 2023-10-11T09:52:31.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.279+0000 7fed717fa700 10 client.9408 handle_command_reply: tid=0 2023-10-11T09:52:31.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.279+0000 7fed9bea3740 1 client.9408 shutdown 2023-10-11T09:52:31.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.279+0000 7fed9bea3740 1 -- 172.21.15.78:0/2109864387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x290a710 msgr2=0x2c26d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:31.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.279+0000 7fed9bea3740 1 --2- 172.21.15.78:0/2109864387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x290a710 0x2c26d80 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fed84006b10 tx=0x7fed840047e0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.279+0000 7fed9bea3740 1 -- 172.21.15.78:0/2109864387 shutdown_connections 2023-10-11T09:52:31.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.279+0000 7fed9bea3740 1 --2- 172.21.15.78:0/2109864387 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x290aeb0 0x2bb8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.279+0000 7fed9bea3740 1 --2- 172.21.15.78:0/2109864387 >> [v2:172.21.15.78:6835/3884456900,v1:172.21.15.78:6837/3884456900] conn(0x2bb1d10 0x2bb20e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.279+0000 7fed9bea3740 1 --2- 172.21.15.78:0/2109864387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x290aae0 0x2bb89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.279+0000 7fed9bea3740 1 --2- 172.21.15.78:0/2109864387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x290a710 0x2c26d80 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.279+0000 7fed717fa700 0 client.9408 ms_handle_reset on v2:172.21.15.78:6835/3884456900 2023-10-11T09:52:31.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.279+0000 7fed9bea3740 1 -- 172.21.15.78:0/2109864387 >> 172.21.15.78:0/2109864387 conn(0x2b72fa0 msgr2=0x2b77e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:31.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.279+0000 7fed9bea3740 1 -- 172.21.15.78:0/2109864387 shutdown_connections 2023-10-11T09:52:31.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.279+0000 7fed9bea3740 1 -- 172.21.15.78:0/2109864387 wait complete. 2023-10-11T09:52:31.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.279+0000 7fed9bea3740 20 client.9408 trim_cache size 0 max 16384 2023-10-11T09:52:31.296 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:52:31.297 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "dee99db9-7a72-406f-a1e2-917943f2b13d", 2023-10-11T09:52:31.297 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2023-10-11T09:52:31.297 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 9336, 2023-10-11T09:52:31.297 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2023-10-11T09:52:31.297 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2023-10-11T09:52:31.298 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2023-10-11T09:52:31.298 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 4.7979213449999998, 2023-10-11T09:52:31.298 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 112, 2023-10-11T09:52:31.298 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 129, 2023-10-11T09:52:31.298 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 129, 2023-10-11T09:52:31.299 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 8.3465574609999997 2023-10-11T09:52:31.299 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:52:31.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.279+0000 7fed9359e700 1 -- 172.21.15.78:0/3584982661 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fed74065ed0 msgr2=0x7fed74068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:31.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.279+0000 7fed9359e700 1 --2- 172.21.15.78:0/3584982661 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fed74065ed0 0x7fed74068380 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fed84009bf0 tx=0x7fed84009240 comp rx=0 tx=0).stop 2023-10-11T09:52:31.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.279+0000 7fed9359e700 1 -- 172.21.15.78:0/3584982661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed940b0d20 msgr2=0x7fed940710a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:31.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.279+0000 7fed9359e700 1 --2- 172.21.15.78:0/3584982661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed940b0d20 0x7fed940710a0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fed8800c480 tx=0x7fed8800c790 comp rx=0 tx=0).stop 2023-10-11T09:52:31.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.279+0000 7fed9359e700 1 -- 172.21.15.78:0/3584982661 shutdown_connections 2023-10-11T09:52:31.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.279+0000 7fed9359e700 1 --2- 172.21.15.78:0/3584982661 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fed74065ed0 0x7fed74068380 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.279+0000 7fed9359e700 1 --2- 172.21.15.78:0/3584982661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed94156cf0 0x7fed94146f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.279+0000 7fed9359e700 1 --2- 172.21.15.78:0/3584982661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed940b0d20 0x7fed940710a0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.279+0000 7fed9359e700 1 --2- 172.21.15.78:0/3584982661 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed9406ca50 0x7fed94157310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.279+0000 7fed9359e700 1 -- 172.21.15.78:0/3584982661 >> 172.21.15.78:0/3584982661 conn(0x7fed940cf820 msgr2=0x7fed940adc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:31.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.279+0000 7fed9359e700 1 -- 172.21.15.78:0/3584982661 shutdown_connections 2023-10-11T09:52:31.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.279+0000 7fed9359e700 1 -- 172.21.15.78:0/3584982661 wait complete. 2023-10-11T09:52:31.308 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-11T09:52:31.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.471+0000 7f29f2c34700 1 Processor -- start 2023-10-11T09:52:31.481 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.475+0000 7f29f2c34700 1 -- start start 2023-10-11T09:52:31.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.475+0000 7f29f2c34700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f29ec0ce7c0 0x7f29ec160bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:31.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.475+0000 7f29f2c34700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29ec0c5b40 0x7f29ec0b0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:31.483 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.475+0000 7f29f2c34700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29ec0ceb90 con 0x7f29ec0713b0 2023-10-11T09:52:31.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.475+0000 7f29f2c34700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29ec15cef0 con 0x7f29ec0ce7c0 2023-10-11T09:52:31.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.475+0000 7f29f2c34700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec160770 con 0x7f29ec0c5b40 2023-10-11T09:52:31.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.475+0000 7f29e3fff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f29ec0713b0 0x7f29ec0c94f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:57032/0 (socket says 172.21.15.78:57032) 2023-10-11T09:52:31.484 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.475+0000 7f29e3fff700 1 -- 172.21.15.78:0/3855834891 learned_addr learned my addr 172.21.15.78:0/3855834891 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:31.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.475+0000 7f29f09d0700 1 --2- 172.21.15.78:0/3855834891 >> v2:172.21.15.78:3300/0 conn(0x7f29ec0ce7c0 0x7f29ec160bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:31.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.475+0000 7f29f11d1700 1 --2- 172.21.15.78:0/3855834891 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29ec0c5b40 0x7f29ec0b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:31.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.475+0000 7f29e37fe700 1 -- 172.21.15.78:0/3855834891 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2784046008 0 0) 0x7f29ec0ceb90 con 0x7f29ec0713b0 2023-10-11T09:52:31.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.475+0000 7f29e37fe700 1 -- 172.21.15.78:0/3855834891 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29d40035d0 con 0x7f29ec0713b0 2023-10-11T09:52:31.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.475+0000 7f29f09d0700 1 -- 172.21.15.78:0/3855834891 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29ec0c5b40 msgr2=0x7f29ec0b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:31.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.475+0000 7f29f09d0700 1 --2- 172.21.15.78:0/3855834891 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29ec0c5b40 0x7f29ec0b0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.475+0000 7f29f09d0700 1 -- 172.21.15.78:0/3855834891 >> v1:172.21.15.78:6789/0 conn(0x7f29ec0713b0 legacy=0x7f29ec0c94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:31.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.479+0000 7f29f09d0700 1 -- 172.21.15.78:0/3855834891 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29ec071820 con 0x7f29ec0ce7c0 2023-10-11T09:52:31.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.479+0000 7f29f09d0700 1 --2- 172.21.15.78:0/3855834891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29ec0ce7c0 0x7f29ec160bc0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f29e8009850 tx=0x7f29e8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a292bf06162dfdaf server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:31.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.479+0000 7f29e37fe700 1 -- 172.21.15.78:0/3855834891 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e8011830 con 0x7f29ec0ce7c0 2023-10-11T09:52:31.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.479+0000 7f29e37fe700 1 -- 172.21.15.78:0/3855834891 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29e8011990 con 0x7f29ec0ce7c0 2023-10-11T09:52:31.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.479+0000 7f29e37fe700 1 -- 172.21.15.78:0/3855834891 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e8011d40 con 0x7f29ec0ce7c0 2023-10-11T09:52:31.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.479+0000 7f29f2c34700 1 -- 172.21.15.78:0/3855834891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29ec0ce7c0 msgr2=0x7f29ec160bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:31.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.479+0000 7f29f2c34700 1 --2- 172.21.15.78:0/3855834891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29ec0ce7c0 0x7f29ec160bc0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f29e8009850 tx=0x7f29e8009b60 comp rx=0 tx=0).stop 2023-10-11T09:52:31.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.479+0000 7f29f2c34700 1 -- 172.21.15.78:0/3855834891 shutdown_connections 2023-10-11T09:52:31.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.479+0000 7f29f2c34700 1 --2- 172.21.15.78:0/3855834891 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29ec0c5b40 0x7f29ec0b0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.479+0000 7f29f2c34700 1 --2- 172.21.15.78:0/3855834891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29ec0ce7c0 0x7f29ec160bc0 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.479+0000 7f29f2c34700 1 -- 172.21.15.78:0/3855834891 >> 172.21.15.78:0/3855834891 conn(0x7f29ec0cb920 msgr2=0x7f29ec0cbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:31.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.479+0000 7f29f2c34700 1 -- 172.21.15.78:0/3855834891 shutdown_connections 2023-10-11T09:52:31.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.479+0000 7f29f2c34700 1 -- 172.21.15.78:0/3855834891 wait complete. 2023-10-11T09:52:31.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.479+0000 7f29f2c34700 1 Processor -- start 2023-10-11T09:52:31.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.479+0000 7f29f2c34700 1 -- start start 2023-10-11T09:52:31.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.483+0000 7f29f2c34700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29ec0713b0 0x7f29ec09c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:31.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.483+0000 7f29f2c34700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29ec0c5b40 0x7f29ec1562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:31.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.483+0000 7f29f2c34700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29ec0ce7c0 0x7f29ec155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:31.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.483+0000 7f29f2c34700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec09c710 con 0x7f29ec0c5b40 2023-10-11T09:52:31.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.483+0000 7f29f2c34700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec09a3e0 con 0x7f29ec0ce7c0 2023-10-11T09:52:31.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.483+0000 7f29f2c34700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec09a6c0 con 0x7f29ec0713b0 2023-10-11T09:52:31.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.483+0000 7f29e3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29ec0c5b40 0x7f29ec1562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:31.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.483+0000 7f29e3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29ec0c5b40 0x7f29ec1562c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50566/0 (socket says 172.21.15.78:50566) 2023-10-11T09:52:31.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.483+0000 7f29e3fff700 1 -- 172.21.15.78:0/1360850039 learned_addr learned my addr 172.21.15.78:0/1360850039 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:31.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.483+0000 7f29f09d0700 1 --2- 172.21.15.78:0/1360850039 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29ec0713b0 0x7f29ec09c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:31.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.483+0000 7f29f11d1700 1 --2- 172.21.15.78:0/1360850039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29ec0ce7c0 0x7f29ec155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:31.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.483+0000 7f29e3fff700 1 -- 172.21.15.78:0/1360850039 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29ec0713b0 msgr2=0x7f29ec09c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:31.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.483+0000 7f29e3fff700 1 --2- 172.21.15.78:0/1360850039 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29ec0713b0 0x7f29ec09c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.483+0000 7f29e3fff700 1 -- 172.21.15.78:0/1360850039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29ec0ce7c0 msgr2=0x7f29ec155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:31.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.483+0000 7f29e3fff700 1 --2- 172.21.15.78:0/1360850039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29ec0ce7c0 0x7f29ec155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.483+0000 7f29e3fff700 1 -- 172.21.15.78:0/1360850039 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e80094b0 con 0x7f29ec0c5b40 2023-10-11T09:52:31.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.483+0000 7f29f11d1700 1 --2- 172.21.15.78:0/1360850039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29ec0ce7c0 0x7f29ec155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:31.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.483+0000 7f29e3fff700 1 --2- 172.21.15.78:0/1360850039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29ec0c5b40 0x7f29ec1562c0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f29dc00c700 tx=0x7f29dc00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:31.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.483+0000 7f29e17fa700 1 -- 172.21.15.78:0/1360850039 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc005d90 con 0x7f29ec0c5b40 2023-10-11T09:52:31.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.483+0000 7f29e17fa700 1 -- 172.21.15.78:0/1360850039 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29dc0070d0 con 0x7f29ec0c5b40 2023-10-11T09:52:31.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.483+0000 7f29e17fa700 1 -- 172.21.15.78:0/1360850039 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc007480 con 0x7f29ec0c5b40 2023-10-11T09:52:31.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.483+0000 7f29f2c34700 1 -- 172.21.15.78:0/1360850039 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29ec09a9a0 con 0x7f29ec0c5b40 2023-10-11T09:52:31.499 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.483+0000 7f29f2c34700 1 -- 172.21.15.78:0/1360850039 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29ec156050 con 0x7f29ec0c5b40 2023-10-11T09:52:31.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.487+0000 7f29f2c34700 1 -- 172.21.15.78:0/1360850039 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29ec09aec0 con 0x7f29ec0c5b40 2023-10-11T09:52:31.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.487+0000 7f29e17fa700 1 -- 172.21.15.78:0/1360850039 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f29dc0075e0 con 0x7f29ec0c5b40 2023-10-11T09:52:31.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.491+0000 7f29e17fa700 1 --2- 172.21.15.78:0/1360850039 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f29d8065ed0 0x7f29d8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:31.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.491+0000 7f29e17fa700 1 -- 172.21.15.78:0/1360850039 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f29dc082dd0 con 0x7f29ec0c5b40 2023-10-11T09:52:31.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.491+0000 7f29f09d0700 1 --2- 172.21.15.78:0/1360850039 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f29d8065ed0 0x7f29d8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:31.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.491+0000 7f29f09d0700 1 --2- 172.21.15.78:0/1360850039 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f29d8065ed0 0x7f29d8068380 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f29e80090b0 tx=0x7f29e801c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:31.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.495+0000 7f29e17fa700 1 -- 172.21.15.78:0/1360850039 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f29ec09aec0 con 0x7f29ec0c5b40 2023-10-11T09:52:31.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.707+0000 7f29f2c34700 1 -- 172.21.15.78:0/1360850039 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f29ec09aec0 con 0x7f29ec0c5b40 2023-10-11T09:52:31.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.711+0000 7f29e17fa700 1 -- 172.21.15.78:0/1360850039 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v112) v1 ==== 53+0+73 (secure 0 0 0) 0x7f29dc017060 con 0x7f29ec0c5b40 2023-10-11T09:52:31.717 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-11T09:52:31.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.711+0000 7f29f2c34700 1 -- 172.21.15.78:0/1360850039 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f29d8065ed0 msgr2=0x7f29d8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:31.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.711+0000 7f29f2c34700 1 --2- 172.21.15.78:0/1360850039 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f29d8065ed0 0x7f29d8068380 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f29e80090b0 tx=0x7f29e801c040 comp rx=0 tx=0).stop 2023-10-11T09:52:31.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.711+0000 7f29f2c34700 1 -- 172.21.15.78:0/1360850039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29ec0c5b40 msgr2=0x7f29ec1562c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:31.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.711+0000 7f29f2c34700 1 --2- 172.21.15.78:0/1360850039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29ec0c5b40 0x7f29ec1562c0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f29dc00c700 tx=0x7f29dc00ca10 comp rx=0 tx=0).stop 2023-10-11T09:52:31.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.711+0000 7f29f2c34700 1 -- 172.21.15.78:0/1360850039 shutdown_connections 2023-10-11T09:52:31.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.711+0000 7f29f2c34700 1 --2- 172.21.15.78:0/1360850039 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f29d8065ed0 0x7f29d8068380 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.711+0000 7f29f2c34700 1 --2- 172.21.15.78:0/1360850039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29ec0ce7c0 0x7f29ec155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.711+0000 7f29f2c34700 1 --2- 172.21.15.78:0/1360850039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29ec0c5b40 0x7f29ec1562c0 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.711+0000 7f29f2c34700 1 --2- 172.21.15.78:0/1360850039 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29ec0713b0 0x7f29ec09c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:31.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.711+0000 7f29f2c34700 1 -- 172.21.15.78:0/1360850039 >> 172.21.15.78:0/1360850039 conn(0x7f29ec0cb920 msgr2=0x7f29ec0ad240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:31.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.711+0000 7f29f2c34700 1 -- 172.21.15.78:0/1360850039 shutdown_connections 2023-10-11T09:52:31.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:31.711+0000 7f29f2c34700 1 -- 172.21.15.78:0/1360850039 wait complete. 2023-10-11T09:52:31.739 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-11T09:52:31.739 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-11T09:52:31.739 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi078.front.sepia.ceph.com 2023-10-11T09:52:31.740 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-11T09:52:31.740 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-11T09:52:31.740 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-11T09:52:31.741 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-11T09:52:31.741 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-11T09:52:31.741 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-11T09:52:31.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:31.742 DEBUG:teuthology.orchestra.run.smithi078:> ip addr 2023-10-11T09:52:31.760 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-11T09:52:31.760 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-11T09:52:31.761 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2023-10-11T09:52:31.761 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:52:31.761 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2023-10-11T09:52:31.761 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:52:31.761 INFO:teuthology.orchestra.run.smithi078.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:52:31.762 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:22 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:52:31.762 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f0 2023-10-11T09:52:31.762 INFO:teuthology.orchestra.run.smithi078.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:52:31.762 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:23 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:52:31.762 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f1 2023-10-11T09:52:31.763 INFO:teuthology.orchestra.run.smithi078.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-11T09:52:31.763 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dc brd ff:ff:ff:ff:ff:ff 2023-10-11T09:52:31.763 INFO:teuthology.orchestra.run.smithi078.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-11T09:52:31.763 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dd brd ff:ff:ff:ff:ff:ff 2023-10-11T09:52:31.763 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.21.15.78/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-11T09:52:31.764 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft 41712sec preferred_lft 41712sec 2023-10-11T09:52:31.764 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::ec4:7aff:fe8f:d4dd/64 scope link 2023-10-11T09:52:31.764 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:52:31.764 INFO:teuthology.orchestra.run.smithi078.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-11T09:52:31.764 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 02:42:b6:e4:f8:0d brd ff:ff:ff:ff:ff:ff 2023-10-11T09:52:31.764 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-11T09:52:31.765 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:52:31.765 INFO:teuthology.orchestra.run.smithi078.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-11T09:52:31.765 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 7a:89:8a:a9:e1:23 brd ff:ff:ff:ff:ff:ff 2023-10-11T09:52:31.765 INFO:teuthology.orchestra.run.smithi078.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-11T09:52:31.765 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:52:31.766 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::8ca:ebff:fea9:13f1/64 scope link 2023-10-11T09:52:31.766 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:52:31.766 INFO:teuthology.orchestra.run.smithi078.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-11T09:52:31.766 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 7a:89:8a:a9:e1:23 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-11T09:52:31.766 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::7889:8aff:fea9:e123/64 scope link 2023-10-11T09:52:31.767 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:52:31.767 INFO:teuthology.orchestra.run.smithi078.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-11T09:52:31.767 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether ca:ac:e8:27:51:0d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-11T09:52:31.767 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::c8ac:e8ff:fe27:510d/64 scope link 2023-10-11T09:52:31.767 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2023-10-11T09:52:31.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:31.768 DEBUG:teuthology.orchestra.run.smithi078:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-11T09:52:31.838 INFO:teuthology.orchestra.run.smithi078.stdout:1 2023-10-11T09:52:31.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:31.841 DEBUG:teuthology.orchestra.run.smithi078:> route 2023-10-11T09:52:31.903 INFO:teuthology.orchestra.run.smithi078.stdout:Kernel IP routing table 2023-10-11T09:52:31.903 INFO:teuthology.orchestra.run.smithi078.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-11T09:52:31.903 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-11T09:52:31.904 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-11T09:52:31.904 INFO:teuthology.orchestra.run.smithi078.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-11T09:52:31.904 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-10-11T09:52:31.904 INFO:teuthology.orchestra.run.smithi078.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-11T09:52:31.904 INFO:teuthology.orchestra.run.smithi078.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-11T09:52:31.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:31.906 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec bash -c ' 2023-10-11T09:52:31.906 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2023-10-11T09:52:31.907 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-11T09:52:31.907 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-11T09:52:31.907 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-11T09:52:31.908 DEBUG:teuthology.orchestra.run.smithi078:> ') 2023-10-11T09:52:32.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:32.040 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2023-10-11T09:52:32.055 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-11T09:52:32.055 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-11T09:52:32.056 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-11T09:52:32.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:32.057 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set brx.0 up 2023-10-11T09:52:32.135 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-11T09:52:32.136 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T09:52:32.136 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-11T09:52:32.194 INFO:teuthology.orchestra.run.smithi078.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-11T09:52:32.195 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T09:52:32.195 DEBUG:teuthology.orchestra.run.smithi078:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-11T09:52:32.252 DEBUG:teuthology.orchestra.run.smithi078:> sudo modprobe fuse 2023-10-11T09:52:32.334 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T09:52:32.334 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T09:52:32.410 INFO:teuthology.orchestra.run.smithi078.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T09:52:32.412 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T09:52:32.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:32.414 DEBUG:teuthology.orchestra.run.smithi078:> ls /sys/fs/fuse/connections 2023-10-11T09:52:32.475 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-11T09:52:32.476 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-11T09:52:32.523 DEBUG:teuthology.orchestra.run.smithi078:> sudo modprobe fuse 2023-10-11T09:52:32.554 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T09:52:32.554 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T09:52:32.633 INFO:teuthology.orchestra.run.smithi078.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T09:52:32.636 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T09:52:32.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:32.637 DEBUG:teuthology.orchestra.run.smithi078:> ls /sys/fs/fuse/connections 2023-10-11T09:52:32.665 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:2023-10-11T09:52:32.655+0000 7f4c27cf0640 -1 init, newargv = 0x55c7671218f0 newargc=15 2023-10-11T09:52:32.665 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:ceph-fuse[61006]: starting ceph client 2023-10-11T09:52:32.676 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:ceph-fuse[61006]: starting fuse 2023-10-11T09:52:32.678 INFO:teuthology.orchestra.run.smithi078.stdout:49 2023-10-11T09:52:32.678 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2023-10-11T09:52:32.679 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T09:52:32.679 DEBUG:teuthology.orchestra.run.smithi078:> import glob 2023-10-11T09:52:32.679 DEBUG:teuthology.orchestra.run.smithi078:> import re 2023-10-11T09:52:32.680 DEBUG:teuthology.orchestra.run.smithi078:> import os 2023-10-11T09:52:32.680 DEBUG:teuthology.orchestra.run.smithi078:> import subprocess 2023-10-11T09:52:32.681 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:52:32.681 DEBUG:teuthology.orchestra.run.smithi078:> def _find_admin_socket(client_name): 2023-10-11T09:52:32.681 DEBUG:teuthology.orchestra.run.smithi078:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-11T09:52:32.682 DEBUG:teuthology.orchestra.run.smithi078:> files = glob.glob(asok_path) 2023-10-11T09:52:32.682 DEBUG:teuthology.orchestra.run.smithi078:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-11T09:52:32.682 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:52:32.683 DEBUG:teuthology.orchestra.run.smithi078:> # Given a non-glob path, it better be there 2023-10-11T09:52:32.683 DEBUG:teuthology.orchestra.run.smithi078:> if "*" not in asok_path: 2023-10-11T09:52:32.684 DEBUG:teuthology.orchestra.run.smithi078:> assert(len(files) == 1) 2023-10-11T09:52:32.684 DEBUG:teuthology.orchestra.run.smithi078:> return files[0] 2023-10-11T09:52:32.684 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:52:32.685 DEBUG:teuthology.orchestra.run.smithi078:> for f in files: 2023-10-11T09:52:32.685 DEBUG:teuthology.orchestra.run.smithi078:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-11T09:52:32.685 DEBUG:teuthology.orchestra.run.smithi078:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-11T09:52:32.686 DEBUG:teuthology.orchestra.run.smithi078:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-11T09:52:32.686 DEBUG:teuthology.orchestra.run.smithi078:> contents = proc_f.read() 2023-10-11T09:52:32.687 DEBUG:teuthology.orchestra.run.smithi078:> if mountpoint in contents: 2023-10-11T09:52:32.687 DEBUG:teuthology.orchestra.run.smithi078:> return f 2023-10-11T09:52:32.687 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-11T09:52:32.688 DEBUG:teuthology.orchestra.run.smithi078:> 2023-10-11T09:52:32.688 DEBUG:teuthology.orchestra.run.smithi078:> print(_find_admin_socket("client.0")) 2023-10-11T09:52:32.688 DEBUG:teuthology.orchestra.run.smithi078:> ' 2023-10-11T09:52:32.835 INFO:teuthology.orchestra.run.smithi078.stdout:/var/run/ceph/ceph-client.0.61006.asok 2023-10-11T09:52:33.020 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.61006.asok 2023-10-11T09:52:33.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:33.021 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.61006.asok status 2023-10-11T09:52:33.212 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:52:33.212 INFO:teuthology.orchestra.run.smithi078.stdout: "metadata": { 2023-10-11T09:52:33.212 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_sha1": "9a774796f0ecdd11a4b771de396e627aa1a187af", 2023-10-11T09:52:33.213 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_version": "ceph version 18.2.0-662-g9a774796 (9a774796f0ecdd11a4b771de396e627aa1a187af) reef (stable)", 2023-10-11T09:52:33.213 INFO:teuthology.orchestra.run.smithi078.stdout: "entity_id": "0", 2023-10-11T09:52:33.213 INFO:teuthology.orchestra.run.smithi078.stdout: "hostname": "smithi078", 2023-10-11T09:52:33.213 INFO:teuthology.orchestra.run.smithi078.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-11T09:52:33.213 INFO:teuthology.orchestra.run.smithi078.stdout: "pid": "61006", 2023-10-11T09:52:33.214 INFO:teuthology.orchestra.run.smithi078.stdout: "root": "/" 2023-10-11T09:52:33.214 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:52:33.214 INFO:teuthology.orchestra.run.smithi078.stdout: "dentry_count": 0, 2023-10-11T09:52:33.214 INFO:teuthology.orchestra.run.smithi078.stdout: "dentry_pinned_count": 0, 2023-10-11T09:52:33.214 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 9420, 2023-10-11T09:52:33.215 INFO:teuthology.orchestra.run.smithi078.stdout: "inst": { 2023-10-11T09:52:33.215 INFO:teuthology.orchestra.run.smithi078.stdout: "name": { 2023-10-11T09:52:33.215 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "client", 2023-10-11T09:52:33.215 INFO:teuthology.orchestra.run.smithi078.stdout: "num": 9420 2023-10-11T09:52:33.215 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:52:33.216 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": { 2023-10-11T09:52:33.216 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2023-10-11T09:52:33.216 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "192.168.0.1:0", 2023-10-11T09:52:33.216 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 4046432584 2023-10-11T09:52:33.216 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-11T09:52:33.217 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:52:33.217 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": { 2023-10-11T09:52:33.217 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2023-10-11T09:52:33.217 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "192.168.0.1:0", 2023-10-11T09:52:33.217 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 4046432584 2023-10-11T09:52:33.217 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-11T09:52:33.218 INFO:teuthology.orchestra.run.smithi078.stdout: "inst_str": "client.9420 192.168.0.1:0/4046432584", 2023-10-11T09:52:33.218 INFO:teuthology.orchestra.run.smithi078.stdout: "addr_str": "192.168.0.1:0/4046432584", 2023-10-11T09:52:33.218 INFO:teuthology.orchestra.run.smithi078.stdout: "inode_count": 1, 2023-10-11T09:52:33.218 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_epoch": 112, 2023-10-11T09:52:33.219 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_epoch": 129, 2023-10-11T09:52:33.219 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_epoch_barrier": 0, 2023-10-11T09:52:33.219 INFO:teuthology.orchestra.run.smithi078.stdout: "blocklisted": false, 2023-10-11T09:52:33.219 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs" 2023-10-11T09:52:33.219 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:52:33.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:33.227 DEBUG:teuthology.orchestra.run.smithi078:> stat --file-system '--printf=%T 2023-10-11T09:52:33.227 DEBUG:teuthology.orchestra.run.smithi078:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-11T09:52:33.243 INFO:teuthology.orchestra.run.smithi078.stdout:fuseblk 2023-10-11T09:52:33.243 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-11T09:52:33.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:33.244 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-11T09:52:33.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:33.322 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -l subdir/) 2023-10-11T09:52:33.550 INFO:teuthology.orchestra.run.smithi078.stdout:total 50 2023-10-11T09:52:33.551 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 1 Oct 11 09:51 0 2023-10-11T09:52:33.551 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 1 Oct 11 09:51 1 2023-10-11T09:52:33.551 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 10 2023-10-11T09:52:33.551 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 11 2023-10-11T09:52:33.551 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 12 2023-10-11T09:52:33.552 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 13 2023-10-11T09:52:33.552 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 14 2023-10-11T09:52:33.552 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 15 2023-10-11T09:52:33.552 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 16 2023-10-11T09:52:33.552 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 17 2023-10-11T09:52:33.553 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 18 2023-10-11T09:52:33.553 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 19 2023-10-11T09:52:33.553 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 1 Oct 11 09:51 2 2023-10-11T09:52:33.553 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 20 2023-10-11T09:52:33.553 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 22 2023-10-11T09:52:33.554 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 23 2023-10-11T09:52:33.554 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 24 2023-10-11T09:52:33.554 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 25 2023-10-11T09:52:33.554 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 26 2023-10-11T09:52:33.554 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 27 2023-10-11T09:52:33.554 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 28 2023-10-11T09:52:33.555 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 29 2023-10-11T09:52:33.555 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 1 Oct 11 09:51 3 2023-10-11T09:52:33.555 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 30 2023-10-11T09:52:33.555 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 31 2023-10-11T09:52:33.555 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 32 2023-10-11T09:52:33.556 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 33 2023-10-11T09:52:33.556 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 34 2023-10-11T09:52:33.556 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 35 2023-10-11T09:52:33.556 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 36 2023-10-11T09:52:33.556 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 37 2023-10-11T09:52:33.557 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 38 2023-10-11T09:52:33.557 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 39 2023-10-11T09:52:33.557 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 1 Oct 11 09:51 4 2023-10-11T09:52:33.557 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 40 2023-10-11T09:52:33.557 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 41 2023-10-11T09:52:33.557 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 42 2023-10-11T09:52:33.558 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 43 2023-10-11T09:52:33.558 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 44 2023-10-11T09:52:33.558 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 45 2023-10-11T09:52:33.558 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 46 2023-10-11T09:52:33.558 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 47 2023-10-11T09:52:33.559 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 48 2023-10-11T09:52:33.559 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 49 2023-10-11T09:52:33.559 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 1 Oct 11 09:51 5 2023-10-11T09:52:33.559 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 50 2023-10-11T09:52:33.560 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 51 2023-10-11T09:52:33.560 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 52 2023-10-11T09:52:33.560 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 53 2023-10-11T09:52:33.560 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 54 2023-10-11T09:52:33.560 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 55 2023-10-11T09:52:33.560 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 56 2023-10-11T09:52:33.561 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 57 2023-10-11T09:52:33.561 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 58 2023-10-11T09:52:33.561 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 59 2023-10-11T09:52:33.561 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 1 Oct 11 09:51 6 2023-10-11T09:52:33.561 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 60 2023-10-11T09:52:33.562 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 61 2023-10-11T09:52:33.562 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 62 2023-10-11T09:52:33.562 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 63 2023-10-11T09:52:33.562 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 64 2023-10-11T09:52:33.562 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 65 2023-10-11T09:52:33.563 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 66 2023-10-11T09:52:33.563 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 67 2023-10-11T09:52:33.563 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 68 2023-10-11T09:52:33.563 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 69 2023-10-11T09:52:33.563 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 1 Oct 11 09:51 7 2023-10-11T09:52:33.563 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 70 2023-10-11T09:52:33.564 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 71 2023-10-11T09:52:33.564 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 72 2023-10-11T09:52:33.564 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 73 2023-10-11T09:52:33.564 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 74 2023-10-11T09:52:33.564 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 75 2023-10-11T09:52:33.565 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 76 2023-10-11T09:52:33.565 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 77 2023-10-11T09:52:33.565 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 78 2023-10-11T09:52:33.565 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 79 2023-10-11T09:52:33.565 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 1 Oct 11 09:51 8 2023-10-11T09:52:33.566 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 80 2023-10-11T09:52:33.566 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 81 2023-10-11T09:52:33.566 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 82 2023-10-11T09:52:33.566 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 83 2023-10-11T09:52:33.566 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 84 2023-10-11T09:52:33.566 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 85 2023-10-11T09:52:33.567 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 86 2023-10-11T09:52:33.567 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 87 2023-10-11T09:52:33.567 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 88 2023-10-11T09:52:33.567 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 89 2023-10-11T09:52:33.567 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 1 Oct 11 09:51 9 2023-10-11T09:52:33.568 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 90 2023-10-11T09:52:33.568 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 91 2023-10-11T09:52:33.568 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 92 2023-10-11T09:52:33.568 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 93 2023-10-11T09:52:33.568 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 94 2023-10-11T09:52:33.568 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 95 2023-10-11T09:52:33.569 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 96 2023-10-11T09:52:33.569 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 97 2023-10-11T09:52:33.569 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 98 2023-10-11T09:52:33.569 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r-- 1 ubuntu ubuntu 2 Oct 11 09:51 99 2023-10-11T09:52:33.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:33.571 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo cat subdir/21') 2023-10-11T09:52:33.611 INFO:teuthology.orchestra.run.smithi078.stderr:cat: subdir/21: No such file or directory 2023-10-11T09:52:33.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T09:52:33.614 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-11T09:52:33.614 DEBUG:teuthology.orchestra.run.smithi078:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-11T09:52:33.697 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-11T09:52:33.740 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-11T09:52:33.740 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:52:33.741 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:52:33.759 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:52:33.870 INFO:tasks.ceph.mds.c.smithi078.stderr:2023-10-11T09:52:33.863+0000 7f7d8d381700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-11T09:52:33.976 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.967+0000 7fe3b657f700 1 Processor -- start 2023-10-11T09:52:33.977 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.967+0000 7fe3b657f700 1 -- start start 2023-10-11T09:52:33.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.971+0000 7fe3b657f700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe3b00cfbb0 0x7fe3b00cff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:33.978 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.971+0000 7fe3b657f700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe3b015cc20 0x7fe3b00d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:33.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.971+0000 7fe3b657f700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3b00cd7f0 0x7fe3b00b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:33.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.971+0000 7fe3b657f700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3b00d04c0 con 0x7fe3b00cfbb0 2023-10-11T09:52:33.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.971+0000 7fe3b657f700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3b00d1820 con 0x7fe3b015cc20 2023-10-11T09:52:33.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.971+0000 7fe3b657f700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b00d1b00 con 0x7fe3b00cd7f0 2023-10-11T09:52:33.979 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.971+0000 7fe3affff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe3b00cfbb0 0x7fe3b00cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:33.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.971+0000 7fe3affff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe3b00cfbb0 0x7fe3b00cff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50572/0 (socket says 172.21.15.78:50572) 2023-10-11T09:52:33.980 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.971+0000 7fe3affff700 1 -- 172.21.15.78:0/3517649962 learned_addr learned my addr 172.21.15.78:0/3517649962 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:33.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.971+0000 7fe3affff700 1 -- 172.21.15.78:0/3517649962 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3b00cd7f0 msgr2=0x7fe3b00b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:33.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.971+0000 7fe3a77fe700 1 --2- 172.21.15.78:0/3517649962 >> v2:172.21.15.133:3300/0 conn(0x7fe3b015cc20 0x7fe3b00d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:33.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.971+0000 7fe3b4b1c700 1 --2- 172.21.15.78:0/3517649962 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3b00cd7f0 0x7fe3b00b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:33.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.971+0000 7fe3affff700 1 --2- 172.21.15.78:0/3517649962 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3b00cd7f0 0x7fe3b00b5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:33.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.971+0000 7fe3affff700 1 -- 172.21.15.78:0/3517649962 >> v2:172.21.15.133:3300/0 conn(0x7fe3b015cc20 msgr2=0x7fe3b00d28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:33.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.971+0000 7fe3affff700 1 --2- 172.21.15.78:0/3517649962 >> v2:172.21.15.133:3300/0 conn(0x7fe3b015cc20 0x7fe3b00d28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:33.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.971+0000 7fe3affff700 1 -- 172.21.15.78:0/3517649962 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b00d1de0 con 0x7fe3b00cfbb0 2023-10-11T09:52:33.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.971+0000 7fe3a77fe700 1 --2- 172.21.15.78:0/3517649962 >> v2:172.21.15.133:3300/0 conn(0x7fe3b015cc20 0x7fe3b00d28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:52:33.983 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.971+0000 7fe3b4b1c700 1 --2- 172.21.15.78:0/3517649962 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3b00cd7f0 0x7fe3b00b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:52:33.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.971+0000 7fe3affff700 1 --2- 172.21.15.78:0/3517649962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3b00cfbb0 0x7fe3b00cff80 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fe3b00ce930 tx=0x7fe3a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ed034fc9380004 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:33.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.971+0000 7fe3af7fe700 1 -- 172.21.15.78:0/3517649962 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a000e070 con 0x7fe3b00cfbb0 2023-10-11T09:52:33.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.971+0000 7fe3af7fe700 1 -- 172.21.15.78:0/3517649962 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a000b9c0 con 0x7fe3b00cfbb0 2023-10-11T09:52:33.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.971+0000 7fe3af7fe700 1 -- 172.21.15.78:0/3517649962 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a000bd70 con 0x7fe3b00cfbb0 2023-10-11T09:52:33.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.975+0000 7fe3b657f700 1 -- 172.21.15.78:0/3517649962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3b00cfbb0 msgr2=0x7fe3b00cff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:33.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.975+0000 7fe3b657f700 1 --2- 172.21.15.78:0/3517649962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3b00cfbb0 0x7fe3b00cff80 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fe3b00ce930 tx=0x7fe3a000b330 comp rx=0 tx=0).stop 2023-10-11T09:52:33.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.975+0000 7fe3b657f700 1 -- 172.21.15.78:0/3517649962 shutdown_connections 2023-10-11T09:52:33.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.975+0000 7fe3b657f700 1 --2- 172.21.15.78:0/3517649962 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3b00cd7f0 0x7fe3b00b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:33.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.975+0000 7fe3b657f700 1 --2- 172.21.15.78:0/3517649962 >> v2:172.21.15.133:3300/0 conn(0x7fe3b015cc20 0x7fe3b00d28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:33.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.975+0000 7fe3b657f700 1 --2- 172.21.15.78:0/3517649962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3b00cfbb0 0x7fe3b00cff80 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:33.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.975+0000 7fe3b657f700 1 -- 172.21.15.78:0/3517649962 >> 172.21.15.78:0/3517649962 conn(0x7fe3b006abf0 msgr2=0x7fe3b006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:33.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.975+0000 7fe3b657f700 1 -- 172.21.15.78:0/3517649962 shutdown_connections 2023-10-11T09:52:33.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.975+0000 7fe3b657f700 1 -- 172.21.15.78:0/3517649962 wait complete. 2023-10-11T09:52:33.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.979+0000 7fe3b657f700 1 Processor -- start 2023-10-11T09:52:33.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.979+0000 7fe3b657f700 1 -- start start 2023-10-11T09:52:33.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.979+0000 7fe3b657f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3b00cd7f0 0x7fe3b00c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:33.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.979+0000 7fe3b657f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3b00cfbb0 0x7fe3b01546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:33.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.979+0000 7fe3b657f700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3b015cc20 0x7fe3b00b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:33.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.979+0000 7fe3b657f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b00b50f0 con 0x7fe3b00cd7f0 2023-10-11T09:52:33.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.979+0000 7fe3b657f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b00c7a80 con 0x7fe3b00cfbb0 2023-10-11T09:52:33.989 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.979+0000 7fe3b657f700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b00c7d60 con 0x7fe3b015cc20 2023-10-11T09:52:33.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.983+0000 7fe3affff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3b00cd7f0 0x7fe3b00c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:33.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.983+0000 7fe3affff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3b00cd7f0 0x7fe3b00c8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50586/0 (socket says 172.21.15.78:50586) 2023-10-11T09:52:33.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.983+0000 7fe3affff700 1 -- 172.21.15.78:0/3520886252 learned_addr learned my addr 172.21.15.78:0/3520886252 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:33.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.983+0000 7fe3b4b1c700 1 --2- 172.21.15.78:0/3520886252 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3b015cc20 0x7fe3b00b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:33.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.983+0000 7fe3a77fe700 1 --2- 172.21.15.78:0/3520886252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3b00cfbb0 0x7fe3b01546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:33.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.983+0000 7fe3affff700 1 -- 172.21.15.78:0/3520886252 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3b015cc20 msgr2=0x7fe3b00b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:33.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.983+0000 7fe3affff700 1 --2- 172.21.15.78:0/3520886252 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3b015cc20 0x7fe3b00b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:33.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.983+0000 7fe3affff700 1 -- 172.21.15.78:0/3520886252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3b00cfbb0 msgr2=0x7fe3b01546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:33.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.983+0000 7fe3affff700 1 --2- 172.21.15.78:0/3520886252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3b00cfbb0 0x7fe3b01546a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:33.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.983+0000 7fe3affff700 1 -- 172.21.15.78:0/3520886252 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a0009b30 con 0x7fe3b00cd7f0 2023-10-11T09:52:33.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.983+0000 7fe3a77fe700 1 --2- 172.21.15.78:0/3520886252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3b00cfbb0 0x7fe3b01546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:33.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.983+0000 7fe3affff700 1 --2- 172.21.15.78:0/3520886252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3b00cd7f0 0x7fe3b00c8b20 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fe3a000b370 tx=0x7fe3a0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:33.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.983+0000 7fe3ad7fa700 1 -- 172.21.15.78:0/3520886252 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a00099b0 con 0x7fe3b00cd7f0 2023-10-11T09:52:33.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.983+0000 7fe3ad7fa700 1 -- 172.21.15.78:0/3520886252 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a0004970 con 0x7fe3b00cd7f0 2023-10-11T09:52:33.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.983+0000 7fe3ad7fa700 1 -- 172.21.15.78:0/3520886252 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a001c660 con 0x7fe3b00cd7f0 2023-10-11T09:52:33.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.983+0000 7fe3b657f700 1 -- 172.21.15.78:0/3520886252 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3b00c8040 con 0x7fe3b00cd7f0 2023-10-11T09:52:33.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.983+0000 7fe3b657f700 1 -- 172.21.15.78:0/3520886252 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3b00c8600 con 0x7fe3b00cd7f0 2023-10-11T09:52:33.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.987+0000 7fe3b657f700 1 -- 172.21.15.78:0/3520886252 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3b0067d00 con 0x7fe3b00cd7f0 2023-10-11T09:52:33.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.987+0000 7fe3ad7fa700 1 -- 172.21.15.78:0/3520886252 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fe3a001c7c0 con 0x7fe3b00cd7f0 2023-10-11T09:52:33.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.987+0000 7fe3ad7fa700 1 --2- 172.21.15.78:0/3520886252 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe390065ed0 0x7fe390068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:33.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.987+0000 7fe3ad7fa700 1 -- 172.21.15.78:0/3520886252 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6234+0+0 (secure 0 0 0) 0x7fe3a0085060 con 0x7fe3b00cd7f0 2023-10-11T09:52:33.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.987+0000 7fe3a77fe700 1 --2- 172.21.15.78:0/3520886252 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe390065ed0 0x7fe390068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:33.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.991+0000 7fe3a77fe700 1 --2- 172.21.15.78:0/3520886252 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe390065ed0 0x7fe390068380 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0157750 tx=0x7fe398008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:34.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:33.995+0000 7fe3ad7fa700 1 -- 172.21.15.78:0/3520886252 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fe3a0052560 con 0x7fe3b00cd7f0 2023-10-11T09:52:34.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.171+0000 7fe3b657f700 1 -- 172.21.15.78:0/3520886252 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3b00744c0 con 0x7fe3b00cd7f0 2023-10-11T09:52:34.180 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.171+0000 7fe3ad7fa700 1 -- 172.21.15.78:0/3520886252 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+18154 (secure 0 0 0) 0x7fe3a0055b80 con 0x7fe3b00cd7f0 2023-10-11T09:52:34.181 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:52:34.181 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":129,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:52:22.836530+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:51:38.592892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-11T09:51:39.204588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6837/4189411531":"2023-10-12T09:52:22.836499+0000","172.21.15.78:6835/4189411531":"2023-10-12T09:52:22.836499+0000","172.21.15.133:6833/1433079498":"2023-10-12T09:52:02.877755+0000","172.21.15.133:6832/1433079498":"2023-10-12T09:52:02.877755+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:52:34.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.175+0000 7fe3b657f700 1 -- 172.21.15.78:0/3520886252 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe390065ed0 msgr2=0x7fe390068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:34.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.175+0000 7fe3b657f700 1 --2- 172.21.15.78:0/3520886252 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe390065ed0 0x7fe390068380 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0157750 tx=0x7fe398008040 comp rx=0 tx=0).stop 2023-10-11T09:52:34.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.175+0000 7fe3b657f700 1 -- 172.21.15.78:0/3520886252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3b00cd7f0 msgr2=0x7fe3b00c8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:34.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.175+0000 7fe3b657f700 1 --2- 172.21.15.78:0/3520886252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3b00cd7f0 0x7fe3b00c8b20 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fe3a000b370 tx=0x7fe3a0004af0 comp rx=0 tx=0).stop 2023-10-11T09:52:34.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.175+0000 7fe3b657f700 1 -- 172.21.15.78:0/3520886252 shutdown_connections 2023-10-11T09:52:34.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.175+0000 7fe3b657f700 1 --2- 172.21.15.78:0/3520886252 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe390065ed0 0x7fe390068380 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:34.185 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.175+0000 7fe3b657f700 1 --2- 172.21.15.78:0/3520886252 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3b015cc20 0x7fe3b00b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:34.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.175+0000 7fe3b657f700 1 --2- 172.21.15.78:0/3520886252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3b00cfbb0 0x7fe3b01546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:34.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.175+0000 7fe3b657f700 1 --2- 172.21.15.78:0/3520886252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3b00cd7f0 0x7fe3b00c8b20 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:34.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.175+0000 7fe3b657f700 1 -- 172.21.15.78:0/3520886252 >> 172.21.15.78:0/3520886252 conn(0x7fe3b006abf0 msgr2=0x7fe3b00b6640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:34.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.179+0000 7fe3b657f700 1 -- 172.21.15.78:0/3520886252 shutdown_connections 2023-10-11T09:52:34.186 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.179+0000 7fe3b657f700 1 -- 172.21.15.78:0/3520886252 wait complete. 2023-10-11T09:52:34.206 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T09:51:38.592892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T09:51:39.204588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '127', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:52:34.207 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:52:34.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.383+0000 7f292af29700 1 Processor -- start 2023-10-11T09:52:34.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.383+0000 7f292af29700 1 -- start start 2023-10-11T09:52:34.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.383+0000 7f292af29700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f29240b4530 0x7f29240d0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:34.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.387+0000 7f292af29700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f292409c960 con 0x7f29240d1520 2023-10-11T09:52:34.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.387+0000 7f292af29700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f292415d180 con 0x7f29240ce740 2023-10-11T09:52:34.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.387+0000 7f292af29700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29240d19a0 con 0x7f29240b4530 2023-10-11T09:52:34.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.387+0000 7f2929f27700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f29240b4530 0x7f29240d0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:34.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.387+0000 7f2929f27700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f29240b4530 0x7f29240d0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50588/0 (socket says 172.21.15.78:50588) 2023-10-11T09:52:34.394 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.387+0000 7f2929f27700 1 -- 172.21.15.78:0/3237197441 learned_addr learned my addr 172.21.15.78:0/3237197441 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:34.395 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.387+0000 7f2929f27700 1 -- 172.21.15.78:0/3237197441 >> v1:172.21.15.133:6789/0 conn(0x7f29240ce740 legacy=0x7f29240b6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:34.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.387+0000 7f2929f27700 1 -- 172.21.15.78:0/3237197441 >> v1:172.21.15.78:6789/0 conn(0x7f29240d1520 legacy=0x7f29240b64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:34.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.387+0000 7f2929f27700 1 -- 172.21.15.78:0/3237197441 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29240b5fd0 con 0x7f29240b4530 2023-10-11T09:52:34.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.387+0000 7f2929f27700 1 --2- 172.21.15.78:0/3237197441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29240b4530 0x7f29240d0fe0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f2920009ea0 tx=0x7f292000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a088359a1d7919 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:34.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.387+0000 7f2928f25700 1 -- 172.21.15.78:0/3237197441 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292000e070 con 0x7f29240b4530 2023-10-11T09:52:34.396 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.387+0000 7f2928f25700 1 -- 172.21.15.78:0/3237197441 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29200091e0 con 0x7f29240b4530 2023-10-11T09:52:34.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.387+0000 7f2928f25700 1 -- 172.21.15.78:0/3237197441 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2920003c80 con 0x7f29240b4530 2023-10-11T09:52:34.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.387+0000 7f292af29700 1 -- 172.21.15.78:0/3237197441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29240b4530 msgr2=0x7f29240d0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:34.397 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.387+0000 7f292af29700 1 --2- 172.21.15.78:0/3237197441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29240b4530 0x7f29240d0fe0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f2920009ea0 tx=0x7f292000b330 comp rx=0 tx=0).stop 2023-10-11T09:52:34.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.387+0000 7f292af29700 1 -- 172.21.15.78:0/3237197441 shutdown_connections 2023-10-11T09:52:34.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.387+0000 7f292af29700 1 --2- 172.21.15.78:0/3237197441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29240b4530 0x7f29240d0fe0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:34.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.387+0000 7f292af29700 1 -- 172.21.15.78:0/3237197441 >> 172.21.15.78:0/3237197441 conn(0x7f292409c3d0 msgr2=0x7f29240d30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:34.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.387+0000 7f292af29700 1 -- 172.21.15.78:0/3237197441 shutdown_connections 2023-10-11T09:52:34.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.387+0000 7f292af29700 1 -- 172.21.15.78:0/3237197441 wait complete. 2023-10-11T09:52:34.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.391+0000 7f292af29700 1 Processor -- start 2023-10-11T09:52:34.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.391+0000 7f292af29700 1 -- start start 2023-10-11T09:52:34.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.395+0000 7f292af29700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29240b4530 0x7f29241ef800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:34.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.395+0000 7f292af29700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29240ce740 0x7f2924153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:34.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.395+0000 7f292af29700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29240d1520 0x7f292406bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:34.402 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.395+0000 7f292af29700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29241f2bf0 con 0x7f29240ce740 2023-10-11T09:52:34.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.395+0000 7f292af29700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29241f25e0 con 0x7f29240b4530 2023-10-11T09:52:34.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.395+0000 7f292af29700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29241546d0 con 0x7f29240d1520 2023-10-11T09:52:34.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.395+0000 7f2929726700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29240ce740 0x7f2924153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:34.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.395+0000 7f2929726700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29240ce740 0x7f2924153d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50604/0 (socket says 172.21.15.78:50604) 2023-10-11T09:52:34.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.395+0000 7f2929726700 1 -- 172.21.15.78:0/223517671 learned_addr learned my addr 172.21.15.78:0/223517671 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:34.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.395+0000 7f2929f27700 1 --2- 172.21.15.78:0/223517671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29240b4530 0x7f29241ef800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:34.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.395+0000 7f292a728700 1 --2- 172.21.15.78:0/223517671 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29240d1520 0x7f292406bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:34.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.395+0000 7f2929726700 1 -- 172.21.15.78:0/223517671 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29240d1520 msgr2=0x7f292406bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:34.405 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.395+0000 7f2929726700 1 --2- 172.21.15.78:0/223517671 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29240d1520 0x7f292406bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:34.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.395+0000 7f2929726700 1 -- 172.21.15.78:0/223517671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29240b4530 msgr2=0x7f29241ef800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:34.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.395+0000 7f2929726700 1 --2- 172.21.15.78:0/223517671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29240b4530 0x7f29241ef800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:34.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.395+0000 7f2929726700 1 -- 172.21.15.78:0/223517671 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2920009b30 con 0x7f29240ce740 2023-10-11T09:52:34.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.395+0000 7f2929726700 1 --2- 172.21.15.78:0/223517671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29240ce740 0x7f2924153d10 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f291400c500 tx=0x7f291400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:34.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.395+0000 7f291a7fc700 1 -- 172.21.15.78:0/223517671 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29140052d0 con 0x7f29240ce740 2023-10-11T09:52:34.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.395+0000 7f292af29700 1 -- 172.21.15.78:0/223517671 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29241f2880 con 0x7f29240ce740 2023-10-11T09:52:34.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.395+0000 7f291a7fc700 1 -- 172.21.15.78:0/223517671 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2914005430 con 0x7f29240ce740 2023-10-11T09:52:34.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.395+0000 7f291a7fc700 1 -- 172.21.15.78:0/223517671 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29140055a0 con 0x7f29240ce740 2023-10-11T09:52:34.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.399+0000 7f292af29700 1 -- 172.21.15.78:0/223517671 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29241efd40 con 0x7f29240ce740 2023-10-11T09:52:34.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.399+0000 7f292af29700 1 -- 172.21.15.78:0/223517671 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29241f2da0 con 0x7f29240ce740 2023-10-11T09:52:34.408 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.399+0000 7f291a7fc700 1 -- 172.21.15.78:0/223517671 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f29140038e0 con 0x7f29240ce740 2023-10-11T09:52:34.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.403+0000 7f291a7fc700 1 --2- 172.21.15.78:0/223517671 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2908065ed0 0x7f2908068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:34.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.403+0000 7f291a7fc700 1 -- 172.21.15.78:0/223517671 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f2914082cc0 con 0x7f29240ce740 2023-10-11T09:52:34.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.403+0000 7f2929f27700 1 --2- 172.21.15.78:0/223517671 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2908065ed0 0x7f2908068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:34.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.403+0000 7f2929f27700 1 --2- 172.21.15.78:0/223517671 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2908065ed0 0x7f2908068380 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f2920015040 tx=0x7f2920009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:34.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.407+0000 7f291a7fc700 1 -- 172.21.15.78:0/223517671 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f29241f2da0 con 0x7f29240ce740 2023-10-11T09:52:34.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.583+0000 7f292af29700 1 -- 172.21.15.78:0/223517671 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f29241f2da0 con 0x7f29240ce740 2023-10-11T09:52:34.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.583+0000 7f291a7fc700 1 -- 172.21.15.78:0/223517671 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v129) v1 ==== 93+0+10 (secure 0 0 0) 0x7f29241f2da0 con 0x7f29240ce740 2023-10-11T09:52:34.590 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:52:34.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.587+0000 7f292af29700 1 -- 172.21.15.78:0/223517671 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2908065ed0 msgr2=0x7f2908068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:34.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.587+0000 7f292af29700 1 --2- 172.21.15.78:0/223517671 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2908065ed0 0x7f2908068380 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f2920015040 tx=0x7f2920009000 comp rx=0 tx=0).stop 2023-10-11T09:52:34.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.587+0000 7f292af29700 1 -- 172.21.15.78:0/223517671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29240ce740 msgr2=0x7f2924153d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:34.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.587+0000 7f292af29700 1 --2- 172.21.15.78:0/223517671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29240ce740 0x7f2924153d10 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f291400c500 tx=0x7f291400c810 comp rx=0 tx=0).stop 2023-10-11T09:52:34.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.587+0000 7f292af29700 1 -- 172.21.15.78:0/223517671 shutdown_connections 2023-10-11T09:52:34.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.587+0000 7f292af29700 1 --2- 172.21.15.78:0/223517671 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2908065ed0 0x7f2908068380 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:34.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.587+0000 7f292af29700 1 --2- 172.21.15.78:0/223517671 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29240d1520 0x7f292406bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:34.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.587+0000 7f292af29700 1 --2- 172.21.15.78:0/223517671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29240ce740 0x7f2924153d10 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:34.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.587+0000 7f292af29700 1 --2- 172.21.15.78:0/223517671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29240b4530 0x7f29241ef800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:34.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.587+0000 7f292af29700 1 -- 172.21.15.78:0/223517671 >> 172.21.15.78:0/223517671 conn(0x7f292409c3d0 msgr2=0x7f29240b6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:34.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.587+0000 7f292af29700 1 -- 172.21.15.78:0/223517671 shutdown_connections 2023-10-11T09:52:34.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.587+0000 7f292af29700 1 -- 172.21.15.78:0/223517671 wait complete. 2023-10-11T09:52:34.616 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:52:34.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.787+0000 7f35ee3a8700 1 Processor -- start 2023-10-11T09:52:34.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.787+0000 7f35ee3a8700 1 -- start start 2023-10-11T09:52:34.797 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.791+0000 7f35ee3a8700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f35e806c8a0 0x7f35e80cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:34.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.791+0000 7f35ee3a8700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35e81648a0 con 0x7f35e80cc230 2023-10-11T09:52:34.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.791+0000 7f35ee3a8700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35e8164b80 con 0x7f35e8160de0 2023-10-11T09:52:34.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.791+0000 7f35ee3a8700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35e80cc6d0 con 0x7f35e806c8a0 2023-10-11T09:52:34.798 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.791+0000 7f35edba7700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f35e80cc230 0x7f35e80cc940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:37976/0 (socket says 172.21.15.78:37976) 2023-10-11T09:52:34.799 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.791+0000 7f35edba7700 1 -- 172.21.15.78:0/2984997207 learned_addr learned my addr 172.21.15.78:0/2984997207 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:34.800 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.791+0000 7f35ecba5700 1 --2- 172.21.15.78:0/2984997207 >> v2:172.21.15.133:3300/0 conn(0x7f35e806c8a0 0x7f35e80cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:34.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.791+0000 7f35dffff700 1 -- 172.21.15.78:0/2984997207 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3775925415 0 0) 0x7f35e81648a0 con 0x7f35e80cc230 2023-10-11T09:52:34.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.791+0000 7f35dffff700 1 -- 172.21.15.78:0/2984997207 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35d40035d0 con 0x7f35e80cc230 2023-10-11T09:52:34.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.791+0000 7f35dffff700 1 -- 172.21.15.78:0/2984997207 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1165963970 0 0) 0x7f35d40035d0 con 0x7f35e80cc230 2023-10-11T09:52:34.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.791+0000 7f35dffff700 1 -- 172.21.15.78:0/2984997207 >> v2:172.21.15.133:3300/0 conn(0x7f35e806c8a0 msgr2=0x7f35e80cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:34.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.791+0000 7f35dffff700 1 --2- 172.21.15.78:0/2984997207 >> v2:172.21.15.133:3300/0 conn(0x7f35e806c8a0 0x7f35e80cbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:34.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.791+0000 7f35dffff700 1 -- 172.21.15.78:0/2984997207 >> v1:172.21.15.133:6789/0 conn(0x7f35e8160de0 legacy=0x7f35e806c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:34.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.791+0000 7f35dffff700 1 -- 172.21.15.78:0/2984997207 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e8164e60 con 0x7f35e80cc230 2023-10-11T09:52:34.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.791+0000 7f35dffff700 1 -- 172.21.15.78:0/2984997207 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f35e00019f0 con 0x7f35e80cc230 2023-10-11T09:52:34.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.791+0000 7f35dffff700 1 -- 172.21.15.78:0/2984997207 >> v1:172.21.15.78:6789/0 conn(0x7f35e80cc230 legacy=0x7f35e80cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:34.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.791+0000 7f35dffff700 1 --2- 172.21.15.78:0/2984997207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35d4003a20 0x7f35d4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:34.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.791+0000 7f35dffff700 1 --2- 172.21.15.78:0/2984997207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35d4004870 0x7f35d4004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:34.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.791+0000 7f35dffff700 1 --2- 172.21.15.78:0/2984997207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35d4005220 0x7f35d400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:34.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.791+0000 7f35dffff700 1 -- 172.21.15.78:0/2984997207 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d400c100 con 0x7f35d4004870 2023-10-11T09:52:34.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.791+0000 7f35dffff700 1 -- 172.21.15.78:0/2984997207 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d400c270 con 0x7f35d4003a20 2023-10-11T09:52:34.803 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.791+0000 7f35dffff700 1 -- 172.21.15.78:0/2984997207 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d400c430 con 0x7f35d4005220 2023-10-11T09:52:34.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.791+0000 7f35ecba5700 1 --2- 172.21.15.78:0/2984997207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35d4004870 0x7f35d4004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:34.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.791+0000 7f35edba7700 1 --2- 172.21.15.78:0/2984997207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35d4005220 0x7f35d400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:34.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.791+0000 7f35ed3a6700 1 --2- 172.21.15.78:0/2984997207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35d4003a20 0x7f35d4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:34.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.791+0000 7f35ecba5700 1 -- 172.21.15.78:0/2984997207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35d4005220 msgr2=0x7f35d400ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:34.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.795+0000 7f35ecba5700 1 --2- 172.21.15.78:0/2984997207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35d4005220 0x7f35d400ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:34.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.795+0000 7f35ecba5700 1 -- 172.21.15.78:0/2984997207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35d4003a20 msgr2=0x7f35d4003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:34.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.795+0000 7f35ecba5700 1 --2- 172.21.15.78:0/2984997207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35d4003a20 0x7f35d4003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:34.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.795+0000 7f35ecba5700 1 -- 172.21.15.78:0/2984997207 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d40035d0 con 0x7f35d4004870 2023-10-11T09:52:34.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.795+0000 7f35edba7700 1 --2- 172.21.15.78:0/2984997207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35d4005220 0x7f35d400ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:34.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.795+0000 7f35edba7700 1 -- 172.21.15.78:0/2984997207 reap_dead start 2023-10-11T09:52:34.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.795+0000 7f35ecba5700 1 --2- 172.21.15.78:0/2984997207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35d4004870 0x7f35d4004ce0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f35e80b4bc0 tx=0x7f35d40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48746a368401e82c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:34.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.795+0000 7f35dffff700 1 -- 172.21.15.78:0/2984997207 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d800e6c0 con 0x7f35d4004870 2023-10-11T09:52:34.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.795+0000 7f35dffff700 1 -- 172.21.15.78:0/2984997207 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d800e820 con 0x7f35d4004870 2023-10-11T09:52:34.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.795+0000 7f35dffff700 1 -- 172.21.15.78:0/2984997207 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d800eb90 con 0x7f35d4004870 2023-10-11T09:52:34.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.795+0000 7f35ee3a8700 1 -- 172.21.15.78:0/2984997207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35d4004870 msgr2=0x7f35d4004ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:34.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.795+0000 7f35ee3a8700 1 --2- 172.21.15.78:0/2984997207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35d4004870 0x7f35d4004ce0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f35e80b4bc0 tx=0x7f35d40058a0 comp rx=0 tx=0).stop 2023-10-11T09:52:34.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.795+0000 7f35ee3a8700 1 -- 172.21.15.78:0/2984997207 shutdown_connections 2023-10-11T09:52:34.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.795+0000 7f35ee3a8700 1 --2- 172.21.15.78:0/2984997207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35d4004870 0x7f35d4004ce0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:34.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.795+0000 7f35ee3a8700 1 -- 172.21.15.78:0/2984997207 >> 172.21.15.78:0/2984997207 conn(0x7f35e8073100 msgr2=0x7f35e80734d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:34.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.795+0000 7f35ee3a8700 1 -- 172.21.15.78:0/2984997207 shutdown_connections 2023-10-11T09:52:34.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.795+0000 7f35ee3a8700 1 -- 172.21.15.78:0/2984997207 wait complete. 2023-10-11T09:52:34.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.799+0000 7f35ee3a8700 1 Processor -- start 2023-10-11T09:52:34.809 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.799+0000 7f35ee3a8700 1 -- start start 2023-10-11T09:52:34.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.799+0000 7f35ee3a8700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35d4004870 0x7f35e8163d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:34.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.799+0000 7f35ee3a8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35e80ae6d0 0x7f35e8152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:34.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.799+0000 7f35ee3a8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35e80cec00 0x7f35e80cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:34.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.799+0000 7f35ee3a8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e80cf5e0 con 0x7f35e80ae6d0 2023-10-11T09:52:34.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.799+0000 7f35ee3a8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e80aebd0 con 0x7f35e80cec00 2023-10-11T09:52:34.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.799+0000 7f35ee3a8700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e80c2400 con 0x7f35d4004870 2023-10-11T09:52:34.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.799+0000 7f35ecba5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35e80ae6d0 0x7f35e8152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:34.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.803+0000 7f35ecba5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35e80ae6d0 0x7f35e8152f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43158/0 (socket says 172.21.15.78:43158) 2023-10-11T09:52:34.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.803+0000 7f35ecba5700 1 -- 172.21.15.78:0/2747158639 learned_addr learned my addr 172.21.15.78:0/2747158639 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:34.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.803+0000 7f35ed3a6700 1 --2- 172.21.15.78:0/2747158639 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35d4004870 0x7f35e8163d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:34.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.803+0000 7f35edba7700 1 --2- 172.21.15.78:0/2747158639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35e80cec00 0x7f35e80cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:34.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.803+0000 7f35ecba5700 1 -- 172.21.15.78:0/2747158639 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35d4004870 msgr2=0x7f35e8163d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:34.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.803+0000 7f35ecba5700 1 --2- 172.21.15.78:0/2747158639 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35d4004870 0x7f35e8163d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:34.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.803+0000 7f35ecba5700 1 -- 172.21.15.78:0/2747158639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35e80cec00 msgr2=0x7f35e80cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:34.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.803+0000 7f35ecba5700 1 --2- 172.21.15.78:0/2747158639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35e80cec00 0x7f35e80cf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:34.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.803+0000 7f35ecba5700 1 -- 172.21.15.78:0/2747158639 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e80c26e0 con 0x7f35e80ae6d0 2023-10-11T09:52:34.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.803+0000 7f35ed3a6700 1 --2- 172.21.15.78:0/2747158639 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35d4004870 0x7f35e8163d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:34.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.803+0000 7f35ecba5700 1 --2- 172.21.15.78:0/2747158639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35e80ae6d0 0x7f35e8152f20 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f35d80090b0 tx=0x7f35d80047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:34.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.803+0000 7f35ddffb700 1 -- 172.21.15.78:0/2747158639 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d800e6c0 con 0x7f35e80ae6d0 2023-10-11T09:52:34.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.803+0000 7f35ddffb700 1 -- 172.21.15.78:0/2747158639 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d80038e0 con 0x7f35e80ae6d0 2023-10-11T09:52:34.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.803+0000 7f35ee3a8700 1 -- 172.21.15.78:0/2747158639 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35e80c29c0 con 0x7f35e80ae6d0 2023-10-11T09:52:34.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.803+0000 7f35ddffb700 1 -- 172.21.15.78:0/2747158639 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d8003c90 con 0x7f35e80ae6d0 2023-10-11T09:52:34.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.803+0000 7f35ee3a8700 1 -- 172.21.15.78:0/2747158639 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35e815d470 con 0x7f35e80ae6d0 2023-10-11T09:52:34.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.803+0000 7f35ee3a8700 1 -- 172.21.15.78:0/2747158639 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35e80685e0 con 0x7f35e80ae6d0 2023-10-11T09:52:34.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.807+0000 7f35ddffb700 1 -- 172.21.15.78:0/2747158639 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f35d8013470 con 0x7f35e80ae6d0 2023-10-11T09:52:34.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.807+0000 7f35ddffb700 1 --2- 172.21.15.78:0/2747158639 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f35cc065f00 0x7f35cc0683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:34.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.807+0000 7f35ddffb700 1 -- 172.21.15.78:0/2747158639 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f35d8084ed0 con 0x7f35e80ae6d0 2023-10-11T09:52:34.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.807+0000 7f35ed3a6700 1 --2- 172.21.15.78:0/2747158639 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f35cc065f00 0x7f35cc0683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:34.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.807+0000 7f35ed3a6700 1 --2- 172.21.15.78:0/2747158639 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f35cc065f00 0x7f35cc0683b0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f35e804fe00 tx=0x7f35e4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:34.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.815+0000 7f35ddffb700 1 -- 172.21.15.78:0/2747158639 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f35d8052620 con 0x7f35e80ae6d0 2023-10-11T09:52:35.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.995+0000 7f35ee3a8700 1 -- 172.21.15.78:0/2747158639 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f35e80744c0 con 0x7f35e80ae6d0 2023-10-11T09:52:35.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.995+0000 7f35ddffb700 1 -- 172.21.15.78:0/2747158639 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v129) v1 ==== 92+0+10 (secure 0 0 0) 0x7f35d8055c40 con 0x7f35e80ae6d0 2023-10-11T09:52:35.002 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:52:35.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.995+0000 7f35ee3a8700 1 -- 172.21.15.78:0/2747158639 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f35cc065f00 msgr2=0x7f35cc0683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:35.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.995+0000 7f35ee3a8700 1 --2- 172.21.15.78:0/2747158639 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f35cc065f00 0x7f35cc0683b0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f35e804fe00 tx=0x7f35e4009040 comp rx=0 tx=0).stop 2023-10-11T09:52:35.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.995+0000 7f35ee3a8700 1 -- 172.21.15.78:0/2747158639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35e80ae6d0 msgr2=0x7f35e8152f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:35.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.995+0000 7f35ee3a8700 1 --2- 172.21.15.78:0/2747158639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35e80ae6d0 0x7f35e8152f20 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f35d80090b0 tx=0x7f35d80047e0 comp rx=0 tx=0).stop 2023-10-11T09:52:35.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.995+0000 7f35ee3a8700 1 -- 172.21.15.78:0/2747158639 shutdown_connections 2023-10-11T09:52:35.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.995+0000 7f35ee3a8700 1 --2- 172.21.15.78:0/2747158639 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f35cc065f00 0x7f35cc0683b0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:35.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.995+0000 7f35ee3a8700 1 --2- 172.21.15.78:0/2747158639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35e80cec00 0x7f35e80cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:35.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.995+0000 7f35ee3a8700 1 --2- 172.21.15.78:0/2747158639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35e80ae6d0 0x7f35e8152f20 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:35.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.995+0000 7f35ee3a8700 1 --2- 172.21.15.78:0/2747158639 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35d4004870 0x7f35e8163d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:35.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.995+0000 7f35ee3a8700 1 -- 172.21.15.78:0/2747158639 >> 172.21.15.78:0/2747158639 conn(0x7f35e8073100 msgr2=0x7f35e80afc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:35.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.999+0000 7f35ee3a8700 1 -- 172.21.15.78:0/2747158639 shutdown_connections 2023-10-11T09:52:35.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:34.999+0000 7f35ee3a8700 1 -- 172.21.15.78:0/2747158639 wait complete. 2023-10-11T09:52:35.027 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T09:52:35.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.199+0000 7f5e3125e700 1 Processor -- start 2023-10-11T09:52:35.208 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.199+0000 7f5e3125e700 1 -- start start 2023-10-11T09:52:35.209 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.203+0000 7f5e3125e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e2c0718d0 0x7f5e2c15a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:35.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.203+0000 7f5e3125e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f5e2c15f090 0x7f5e2c15aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:35.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.203+0000 7f5e3125e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e2c072040 con 0x7f5e2c0caf80 2023-10-11T09:52:35.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.203+0000 7f5e3125e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e2c15ec00 con 0x7f5e2c15f090 2023-10-11T09:52:35.210 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.203+0000 7f5e3125e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c071d60 con 0x7f5e2c0718d0 2023-10-11T09:52:35.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.203+0000 7f5e2b59e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f5e2c15f090 0x7f5e2c15aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:35.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.203+0000 7f5e2a59c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e2c0718d0 0x7f5e2c15a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:35.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.203+0000 7f5e2ad9d700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f5e2c0caf80 0x7f5e2c0711c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:37978/0 (socket says 172.21.15.78:37978) 2023-10-11T09:52:35.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.203+0000 7f5e2ad9d700 1 -- 172.21.15.78:0/3646188636 learned_addr learned my addr 172.21.15.78:0/3646188636 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:35.212 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.203+0000 7f5e2b59e700 1 -- 172.21.15.78:0/3646188636 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e2c0718d0 msgr2=0x7f5e2c15a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:35.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.203+0000 7f5e2b59e700 1 --2- 172.21.15.78:0/3646188636 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e2c0718d0 0x7f5e2c15a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:35.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.203+0000 7f5e2b59e700 1 -- 172.21.15.78:0/3646188636 >> v1:172.21.15.78:6789/0 conn(0x7f5e2c0caf80 legacy=0x7f5e2c0711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:35.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.203+0000 7f5e2b59e700 1 -- 172.21.15.78:0/3646188636 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e2c15e5b0 con 0x7f5e2c15f090 2023-10-11T09:52:35.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.203+0000 7f5e29d9b700 1 -- 172.21.15.78:0/3646188636 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2735262880 0 0) 0x7f5e2c072040 con 0x7f5e2c0caf80 2023-10-11T09:52:35.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.203+0000 7f5e2b59e700 1 --2- 172.21.15.78:0/3646188636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e2c15f090 0x7f5e2c15aba0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f5e24009d90 tx=0x7f5e2400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbdd2fc4fd646bd6 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:35.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.203+0000 7f5e29d9b700 1 -- 172.21.15.78:0/3646188636 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2400e070 con 0x7f5e2c15f090 2023-10-11T09:52:35.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.203+0000 7f5e29d9b700 1 -- 172.21.15.78:0/3646188636 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2400bb20 con 0x7f5e2c15f090 2023-10-11T09:52:35.214 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.203+0000 7f5e29d9b700 1 -- 172.21.15.78:0/3646188636 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e24012430 con 0x7f5e2c15f090 2023-10-11T09:52:35.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.207+0000 7f5e3125e700 1 -- 172.21.15.78:0/3646188636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e2c15f090 msgr2=0x7f5e2c15aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:35.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.207+0000 7f5e3125e700 1 --2- 172.21.15.78:0/3646188636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e2c15f090 0x7f5e2c15aba0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f5e24009d90 tx=0x7f5e2400b330 comp rx=0 tx=0).stop 2023-10-11T09:52:35.215 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.207+0000 7f5e3125e700 1 -- 172.21.15.78:0/3646188636 shutdown_connections 2023-10-11T09:52:35.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.207+0000 7f5e3125e700 1 --2- 172.21.15.78:0/3646188636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e2c15f090 0x7f5e2c15aba0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:35.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.207+0000 7f5e3125e700 1 --2- 172.21.15.78:0/3646188636 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e2c0718d0 0x7f5e2c15a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:35.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.207+0000 7f5e3125e700 1 -- 172.21.15.78:0/3646188636 >> 172.21.15.78:0/3646188636 conn(0x7f5e2c06ae10 msgr2=0x7f5e2c0b0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:35.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.207+0000 7f5e3125e700 1 -- 172.21.15.78:0/3646188636 shutdown_connections 2023-10-11T09:52:35.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.207+0000 7f5e3125e700 1 -- 172.21.15.78:0/3646188636 wait complete. 2023-10-11T09:52:35.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.211+0000 7f5e3125e700 1 Processor -- start 2023-10-11T09:52:35.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.211+0000 7f5e3125e700 1 -- start start 2023-10-11T09:52:35.218 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.211+0000 7f5e3125e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e2c0718d0 0x7f5e2c153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:35.218 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.211+0000 7f5e3125e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e2c0caf80 0x7f5e2c0b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:35.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.211+0000 7f5e3125e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e2c15f090 0x7f5e2c15c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:35.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.211+0000 7f5e3125e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c0b3490 con 0x7f5e2c15f090 2023-10-11T09:52:35.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.211+0000 7f5e3125e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c0b38b0 con 0x7f5e2c0caf80 2023-10-11T09:52:35.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.211+0000 7f5e3125e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c0b2810 con 0x7f5e2c0718d0 2023-10-11T09:52:35.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.211+0000 7f5e2b59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e2c15f090 0x7f5e2c15c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:35.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.211+0000 7f5e2b59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e2c15f090 0x7f5e2c15c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43180/0 (socket says 172.21.15.78:43180) 2023-10-11T09:52:35.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.211+0000 7f5e2b59e700 1 -- 172.21.15.78:0/1143740142 learned_addr learned my addr 172.21.15.78:0/1143740142 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:35.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.211+0000 7f5e2a59c700 1 --2- 172.21.15.78:0/1143740142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e2c0caf80 0x7f5e2c0b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:35.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.211+0000 7f5e2ad9d700 1 --2- 172.21.15.78:0/1143740142 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e2c0718d0 0x7f5e2c153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:35.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.211+0000 7f5e2b59e700 1 -- 172.21.15.78:0/1143740142 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e2c0718d0 msgr2=0x7f5e2c153f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:35.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.211+0000 7f5e2b59e700 1 --2- 172.21.15.78:0/1143740142 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e2c0718d0 0x7f5e2c153f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:35.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.211+0000 7f5e2b59e700 1 -- 172.21.15.78:0/1143740142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e2c0caf80 msgr2=0x7f5e2c0b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:35.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.211+0000 7f5e2b59e700 1 --2- 172.21.15.78:0/1143740142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e2c0caf80 0x7f5e2c0b1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:35.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.211+0000 7f5e2b59e700 1 -- 172.21.15.78:0/1143740142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e24009a70 con 0x7f5e2c15f090 2023-10-11T09:52:35.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.211+0000 7f5e2ad9d700 1 --2- 172.21.15.78:0/1143740142 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e2c0718d0 0x7f5e2c153f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:35.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.211+0000 7f5e2a59c700 1 --2- 172.21.15.78:0/1143740142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e2c0caf80 0x7f5e2c0b1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:35.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.211+0000 7f5e2b59e700 1 --2- 172.21.15.78:0/1143740142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e2c15f090 0x7f5e2c15c330 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f5e24013000 tx=0x7f5e24004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:35.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.211+0000 7f5e137fe700 1 -- 172.21.15.78:0/1143740142 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e24012430 con 0x7f5e2c15f090 2023-10-11T09:52:35.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.215+0000 7f5e137fe700 1 -- 172.21.15.78:0/1143740142 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e24020070 con 0x7f5e2c15f090 2023-10-11T09:52:35.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.215+0000 7f5e3125e700 1 -- 172.21.15.78:0/1143740142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e2c0b2af0 con 0x7f5e2c15f090 2023-10-11T09:52:35.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.215+0000 7f5e137fe700 1 -- 172.21.15.78:0/1143740142 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e24004280 con 0x7f5e2c15f090 2023-10-11T09:52:35.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.215+0000 7f5e3125e700 1 -- 172.21.15.78:0/1143740142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e2c0b30b0 con 0x7f5e2c15f090 2023-10-11T09:52:35.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.215+0000 7f5e3125e700 1 -- 172.21.15.78:0/1143740142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e2c0b35d0 con 0x7f5e2c15f090 2023-10-11T09:52:35.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.215+0000 7f5e137fe700 1 -- 172.21.15.78:0/1143740142 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f5e240043e0 con 0x7f5e2c15f090 2023-10-11T09:52:35.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.219+0000 7f5e137fe700 1 --2- 172.21.15.78:0/1143740142 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5e18065ed0 0x7f5e18068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:35.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.219+0000 7f5e137fe700 1 -- 172.21.15.78:0/1143740142 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f5e24022070 con 0x7f5e2c15f090 2023-10-11T09:52:35.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.219+0000 7f5e2ad9d700 1 --2- 172.21.15.78:0/1143740142 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5e18065ed0 0x7f5e18068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:35.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.219+0000 7f5e2ad9d700 1 --2- 172.21.15.78:0/1143740142 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5e18065ed0 0x7f5e18068380 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c000f60 tx=0x7f5e1c007cc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:35.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.223+0000 7f5e137fe700 1 -- 172.21.15.78:0/1143740142 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f5e2c0b35d0 con 0x7f5e2c15f090 2023-10-11T09:52:35.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.403+0000 7f5e3125e700 1 -- 172.21.15.78:0/1143740142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5e2c0b35d0 con 0x7f5e2c15f090 2023-10-11T09:52:35.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.403+0000 7f5e137fe700 1 -- 172.21.15.78:0/1143740142 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v129) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5e2c0b35d0 con 0x7f5e2c15f090 2023-10-11T09:52:35.412 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:52:35.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.407+0000 7f5e3125e700 1 -- 172.21.15.78:0/1143740142 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5e18065ed0 msgr2=0x7f5e18068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:35.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.407+0000 7f5e3125e700 1 --2- 172.21.15.78:0/1143740142 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5e18065ed0 0x7f5e18068380 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c000f60 tx=0x7f5e1c007cc0 comp rx=0 tx=0).stop 2023-10-11T09:52:35.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.407+0000 7f5e3125e700 1 -- 172.21.15.78:0/1143740142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e2c15f090 msgr2=0x7f5e2c15c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:35.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.407+0000 7f5e3125e700 1 --2- 172.21.15.78:0/1143740142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e2c15f090 0x7f5e2c15c330 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f5e24013000 tx=0x7f5e24004af0 comp rx=0 tx=0).stop 2023-10-11T09:52:35.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.407+0000 7f5e3125e700 1 -- 172.21.15.78:0/1143740142 shutdown_connections 2023-10-11T09:52:35.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.407+0000 7f5e3125e700 1 --2- 172.21.15.78:0/1143740142 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5e18065ed0 0x7f5e18068380 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:35.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.407+0000 7f5e3125e700 1 --2- 172.21.15.78:0/1143740142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e2c15f090 0x7f5e2c15c330 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:35.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.407+0000 7f5e3125e700 1 --2- 172.21.15.78:0/1143740142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e2c0caf80 0x7f5e2c0b1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:35.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.407+0000 7f5e3125e700 1 --2- 172.21.15.78:0/1143740142 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e2c0718d0 0x7f5e2c153f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:35.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.407+0000 7f5e3125e700 1 -- 172.21.15.78:0/1143740142 >> 172.21.15.78:0/1143740142 conn(0x7f5e2c06ae10 msgr2=0x7f5e2c159a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:35.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.407+0000 7f5e3125e700 1 -- 172.21.15.78:0/1143740142 shutdown_connections 2023-10-11T09:52:35.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.407+0000 7f5e3125e700 1 -- 172.21.15.78:0/1143740142 wait complete. 2023-10-11T09:52:35.438 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T09:52:35.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.603+0000 7f70b751f700 1 Processor -- start 2023-10-11T09:52:35.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.603+0000 7f70b751f700 1 -- start start 2023-10-11T09:52:35.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.603+0000 7f70b751f700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f70b015cd50 0x7f70b00b2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:35.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.603+0000 7f70b751f700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f70b00b21b0 0x7f70b00cfc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:35.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.603+0000 7f70b751f700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70b0161640 con 0x7f70b00d0190 2023-10-11T09:52:35.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.603+0000 7f70b751f700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70b00d0610 con 0x7f70b015cd50 2023-10-11T09:52:35.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.603+0000 7f70b751f700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b0160750 con 0x7f70b00b21b0 2023-10-11T09:52:35.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.603+0000 7f70b4aba700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f70b00b21b0 0x7f70b00cfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:35.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.603+0000 7f70b4aba700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f70b00b21b0 0x7f70b00cfc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:35724/0 (socket says 172.21.15.78:35724) 2023-10-11T09:52:35.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.603+0000 7f70b4aba700 1 -- 172.21.15.78:0/418694074 learned_addr learned my addr 172.21.15.78:0/418694074 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:35.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.607+0000 7f70b52bb700 1 --2- 172.21.15.78:0/418694074 >> v2:172.21.15.133:3300/0 conn(0x7f70b015cd50 0x7f70b00b2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:35.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.607+0000 7f70b4aba700 1 -- 172.21.15.78:0/418694074 >> v2:172.21.15.133:3300/0 conn(0x7f70b015cd50 msgr2=0x7f70b00b2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:35.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.607+0000 7f70b4aba700 1 --2- 172.21.15.78:0/418694074 >> v2:172.21.15.133:3300/0 conn(0x7f70b015cd50 0x7f70b00b2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:35.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.607+0000 7f70b4aba700 1 -- 172.21.15.78:0/418694074 >> v1:172.21.15.133:6789/0 conn(0x7f70b00d0190 legacy=0x7f70b00d3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:35.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.607+0000 7f70b4aba700 1 -- 172.21.15.78:0/418694074 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b0160a30 con 0x7f70b00b21b0 2023-10-11T09:52:35.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.607+0000 7f70b4aba700 1 --2- 172.21.15.78:0/418694074 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f70b00b21b0 0x7f70b00cfc50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f70a0009910 tx=0x7f70a0009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2190482488cd919 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:35.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.607+0000 7f70a7fff700 1 -- 172.21.15.78:0/418694074 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a0011830 con 0x7f70b00b21b0 2023-10-11T09:52:35.616 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.607+0000 7f70a7fff700 1 -- 172.21.15.78:0/418694074 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a0011990 con 0x7f70b00b21b0 2023-10-11T09:52:35.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.607+0000 7f70a7fff700 1 -- 172.21.15.78:0/418694074 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a0011d40 con 0x7f70b00b21b0 2023-10-11T09:52:35.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.607+0000 7f70b751f700 1 -- 172.21.15.78:0/418694074 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f70b00b21b0 msgr2=0x7f70b00cfc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:35.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.607+0000 7f70b751f700 1 --2- 172.21.15.78:0/418694074 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f70b00b21b0 0x7f70b00cfc50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f70a0009910 tx=0x7f70a0009c20 comp rx=0 tx=0).stop 2023-10-11T09:52:35.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.607+0000 7f70b751f700 1 -- 172.21.15.78:0/418694074 shutdown_connections 2023-10-11T09:52:35.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.607+0000 7f70b751f700 1 --2- 172.21.15.78:0/418694074 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f70b00b21b0 0x7f70b00cfc50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:35.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.607+0000 7f70b751f700 1 --2- 172.21.15.78:0/418694074 >> v2:172.21.15.133:3300/0 conn(0x7f70b015cd50 0x7f70b00b2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:35.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.607+0000 7f70b751f700 1 -- 172.21.15.78:0/418694074 >> 172.21.15.78:0/418694074 conn(0x7f70b00718b0 msgr2=0x7f70b0071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:35.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.607+0000 7f70b751f700 1 -- 172.21.15.78:0/418694074 shutdown_connections 2023-10-11T09:52:35.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.607+0000 7f70b751f700 1 -- 172.21.15.78:0/418694074 wait complete. 2023-10-11T09:52:35.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.611+0000 7f70b751f700 1 Processor -- start 2023-10-11T09:52:35.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.611+0000 7f70b751f700 1 -- start start 2023-10-11T09:52:35.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.611+0000 7f70b751f700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f70b00b21b0 0x7f70b01584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:35.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.611+0000 7f70b751f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70b00d0190 0x7f70b01f5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:35.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.611+0000 7f70b751f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70b015cd50 0x7f70b01f5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:35.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.611+0000 7f70b751f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b0159f30 con 0x7f70b00d0190 2023-10-11T09:52:35.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.611+0000 7f70b751f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b015a210 con 0x7f70b015cd50 2023-10-11T09:52:35.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.611+0000 7f70b751f700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b015a4f0 con 0x7f70b00b21b0 2023-10-11T09:52:35.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.611+0000 7f70b4aba700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70b00d0190 0x7f70b01f5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:35.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.611+0000 7f70b4aba700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70b00d0190 0x7f70b01f5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43196/0 (socket says 172.21.15.78:43196) 2023-10-11T09:52:35.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.611+0000 7f70b4aba700 1 -- 172.21.15.78:0/1949688536 learned_addr learned my addr 172.21.15.78:0/1949688536 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:35.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.615+0000 7f70b5abc700 1 --2- 172.21.15.78:0/1949688536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70b015cd50 0x7f70b01f5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:35.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.615+0000 7f70b52bb700 1 --2- 172.21.15.78:0/1949688536 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f70b00b21b0 0x7f70b01584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:35.623 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.615+0000 7f70b4aba700 1 -- 172.21.15.78:0/1949688536 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f70b00b21b0 msgr2=0x7f70b01584e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:35.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.615+0000 7f70b4aba700 1 --2- 172.21.15.78:0/1949688536 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f70b00b21b0 0x7f70b01584e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:35.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.615+0000 7f70b4aba700 1 -- 172.21.15.78:0/1949688536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70b015cd50 msgr2=0x7f70b01f5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:35.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.615+0000 7f70b4aba700 1 --2- 172.21.15.78:0/1949688536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70b015cd50 0x7f70b01f5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:35.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.615+0000 7f70b4aba700 1 -- 172.21.15.78:0/1949688536 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a0009570 con 0x7f70b00d0190 2023-10-11T09:52:35.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.615+0000 7f70b52bb700 1 --2- 172.21.15.78:0/1949688536 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f70b00b21b0 0x7f70b01584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:35.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.615+0000 7f70b4aba700 1 --2- 172.21.15.78:0/1949688536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70b00d0190 0x7f70b01f5a10 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f70a0000c00 tx=0x7f70a0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:35.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.615+0000 7f70a5ffb700 1 -- 172.21.15.78:0/1949688536 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a0004590 con 0x7f70b00d0190 2023-10-11T09:52:35.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.615+0000 7f70b751f700 1 -- 172.21.15.78:0/1949688536 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b015a7d0 con 0x7f70b00d0190 2023-10-11T09:52:35.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.615+0000 7f70a5ffb700 1 -- 172.21.15.78:0/1949688536 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a001e070 con 0x7f70b00d0190 2023-10-11T09:52:35.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.615+0000 7f70a5ffb700 1 -- 172.21.15.78:0/1949688536 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a00038e0 con 0x7f70b00d0190 2023-10-11T09:52:35.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.615+0000 7f70b751f700 1 -- 172.21.15.78:0/1949688536 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b00cb1f0 con 0x7f70b00d0190 2023-10-11T09:52:35.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.619+0000 7f70b751f700 1 -- 172.21.15.78:0/1949688536 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b008c200 con 0x7f70b00d0190 2023-10-11T09:52:35.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.619+0000 7f70a5ffb700 1 -- 172.21.15.78:0/1949688536 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f70a0003a40 con 0x7f70b00d0190 2023-10-11T09:52:35.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.619+0000 7f70a5ffb700 1 --2- 172.21.15.78:0/1949688536 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7094065ed0 0x7f7094068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:35.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.619+0000 7f70a5ffb700 1 -- 172.21.15.78:0/1949688536 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f70a0084930 con 0x7f70b00d0190 2023-10-11T09:52:35.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.619+0000 7f70b52bb700 1 --2- 172.21.15.78:0/1949688536 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7094065ed0 0x7f7094068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:35.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.623+0000 7f70b52bb700 1 --2- 172.21.15.78:0/1949688536 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7094065ed0 0x7f7094068380 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f70ac009c40 tx=0x7f70ac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:35.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.627+0000 7f70a5ffb700 1 -- 172.21.15.78:0/1949688536 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f70a0051f00 con 0x7f70b00d0190 2023-10-11T09:52:35.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.799+0000 7f70b751f700 1 -- 172.21.15.78:0/1949688536 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f70b0160fc0 con 0x7f70b00d0190 2023-10-11T09:52:35.808 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.799+0000 7f70a5ffb700 1 -- 172.21.15.78:0/1949688536 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v129) v1 ==== 100+0+11 (secure 0 0 0) 0x7f70a0006da0 con 0x7f70b00d0190 2023-10-11T09:52:35.808 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2023-10-11T09:52:35.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.803+0000 7f70b751f700 1 -- 172.21.15.78:0/1949688536 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7094065ed0 msgr2=0x7f7094068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:35.811 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.803+0000 7f70b751f700 1 --2- 172.21.15.78:0/1949688536 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7094065ed0 0x7f7094068380 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f70ac009c40 tx=0x7f70ac008040 comp rx=0 tx=0).stop 2023-10-11T09:52:35.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.803+0000 7f70b751f700 1 -- 172.21.15.78:0/1949688536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70b00d0190 msgr2=0x7f70b01f5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:35.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.803+0000 7f70b751f700 1 --2- 172.21.15.78:0/1949688536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70b00d0190 0x7f70b01f5a10 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f70a0000c00 tx=0x7f70a0004770 comp rx=0 tx=0).stop 2023-10-11T09:52:35.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.803+0000 7f70b751f700 1 -- 172.21.15.78:0/1949688536 shutdown_connections 2023-10-11T09:52:35.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.803+0000 7f70b751f700 1 --2- 172.21.15.78:0/1949688536 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7094065ed0 0x7f7094068380 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:35.812 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.803+0000 7f70b751f700 1 --2- 172.21.15.78:0/1949688536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70b015cd50 0x7f70b01f5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:35.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.803+0000 7f70b751f700 1 --2- 172.21.15.78:0/1949688536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70b00d0190 0x7f70b01f5a10 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:35.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.803+0000 7f70b751f700 1 --2- 172.21.15.78:0/1949688536 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f70b00b21b0 0x7f70b01584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:35.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.803+0000 7f70b751f700 1 -- 172.21.15.78:0/1949688536 >> 172.21.15.78:0/1949688536 conn(0x7f70b00718b0 msgr2=0x7f70b00b6680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:35.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.803+0000 7f70b751f700 1 -- 172.21.15.78:0/1949688536 shutdown_connections 2023-10-11T09:52:35.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.803+0000 7f70b751f700 1 -- 172.21.15.78:0/1949688536 wait complete. 2023-10-11T09:52:35.833 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T09:52:36.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.999+0000 7f588f1b3700 1 Processor -- start 2023-10-11T09:52:36.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.999+0000 7f588f1b3700 1 -- start start 2023-10-11T09:52:36.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.999+0000 7f588f1b3700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f58880ce7c0 0x7f5888160bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:36.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.999+0000 7f588f1b3700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f58880713b0 0x7f58880c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:36.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.999+0000 7f588f1b3700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58880ceb90 con 0x7f58880c5b40 2023-10-11T09:52:36.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.999+0000 7f588f1b3700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f588815cef0 con 0x7f58880ce7c0 2023-10-11T09:52:36.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.999+0000 7f588f1b3700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5888160770 con 0x7f58880713b0 2023-10-11T09:52:36.010 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.999+0000 7f588d750700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f58880c5b40 0x7f58880b0920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:37988/0 (socket says 172.21.15.78:37988) 2023-10-11T09:52:36.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.999+0000 7f588d750700 1 -- 172.21.15.78:0/1874629063 learned_addr learned my addr 172.21.15.78:0/1874629063 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:36.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.999+0000 7f588cf4f700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f58880ce7c0 0x7f5888160bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:36.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:35.999+0000 7f5887fff700 1 --2- 172.21.15.78:0/1874629063 >> v2:172.21.15.133:3300/0 conn(0x7f58880713b0 0x7f58880c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:36.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.003+0000 7f58877fe700 1 -- 172.21.15.78:0/1874629063 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1201540118 0 0) 0x7f58880ceb90 con 0x7f58880c5b40 2023-10-11T09:52:36.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.003+0000 7f588cf4f700 1 -- 172.21.15.78:0/1874629063 >> v2:172.21.15.133:3300/0 conn(0x7f58880713b0 msgr2=0x7f58880c94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:36.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.003+0000 7f588cf4f700 1 --2- 172.21.15.78:0/1874629063 >> v2:172.21.15.133:3300/0 conn(0x7f58880713b0 0x7f58880c94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:36.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.003+0000 7f588cf4f700 1 -- 172.21.15.78:0/1874629063 >> v1:172.21.15.78:6789/0 conn(0x7f58880c5b40 legacy=0x7f58880b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:36.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.003+0000 7f588cf4f700 1 -- 172.21.15.78:0/1874629063 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5888071820 con 0x7f58880ce7c0 2023-10-11T09:52:36.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.003+0000 7f5887fff700 1 --2- 172.21.15.78:0/1874629063 >> v2:172.21.15.133:3300/0 conn(0x7f58880713b0 0x7f58880c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:52:36.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.003+0000 7f588cf4f700 1 --2- 172.21.15.78:0/1874629063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58880ce7c0 0x7f5888160bc0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f58880caab0 tx=0x7f587c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2edec7bd8c9c68a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:36.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.003+0000 7f58877fe700 1 -- 172.21.15.78:0/1874629063 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587c011830 con 0x7f58880ce7c0 2023-10-11T09:52:36.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.003+0000 7f58877fe700 1 -- 172.21.15.78:0/1874629063 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587c011990 con 0x7f58880ce7c0 2023-10-11T09:52:36.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.003+0000 7f58877fe700 1 -- 172.21.15.78:0/1874629063 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587c011d20 con 0x7f58880ce7c0 2023-10-11T09:52:36.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.003+0000 7f588f1b3700 1 -- 172.21.15.78:0/1874629063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58880ce7c0 msgr2=0x7f5888160bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:36.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.003+0000 7f588f1b3700 1 --2- 172.21.15.78:0/1874629063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58880ce7c0 0x7f5888160bc0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f58880caab0 tx=0x7f587c009b30 comp rx=0 tx=0).stop 2023-10-11T09:52:36.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.003+0000 7f588f1b3700 1 -- 172.21.15.78:0/1874629063 shutdown_connections 2023-10-11T09:52:36.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.003+0000 7f588f1b3700 1 --2- 172.21.15.78:0/1874629063 >> v2:172.21.15.133:3300/0 conn(0x7f58880713b0 0x7f58880c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:36.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.003+0000 7f588f1b3700 1 --2- 172.21.15.78:0/1874629063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58880ce7c0 0x7f5888160bc0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:36.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.003+0000 7f588f1b3700 1 -- 172.21.15.78:0/1874629063 >> 172.21.15.78:0/1874629063 conn(0x7f58880cb920 msgr2=0x7f58880cbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:36.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.003+0000 7f588f1b3700 1 -- 172.21.15.78:0/1874629063 shutdown_connections 2023-10-11T09:52:36.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.003+0000 7f588f1b3700 1 -- 172.21.15.78:0/1874629063 wait complete. 2023-10-11T09:52:36.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.007+0000 7f588f1b3700 1 Processor -- start 2023-10-11T09:52:36.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.007+0000 7f588f1b3700 1 -- start start 2023-10-11T09:52:36.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.007+0000 7f588f1b3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58880713b0 0x7f58880c3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:36.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.007+0000 7f588f1b3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58880c5b40 0x7f58880c4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:36.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.007+0000 7f588f1b3700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58880ce7c0 0x7f58880c4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:36.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.007+0000 7f588f1b3700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58880c4d90 con 0x7f58880713b0 2023-10-11T09:52:36.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.007+0000 7f588f1b3700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58880b18a0 con 0x7f58880c5b40 2023-10-11T09:52:36.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.007+0000 7f588f1b3700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5888099c00 con 0x7f58880ce7c0 2023-10-11T09:52:36.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.007+0000 7f588cf4f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58880713b0 0x7f58880c3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:36.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.007+0000 7f588cf4f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58880713b0 0x7f58880c3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43222/0 (socket says 172.21.15.78:43222) 2023-10-11T09:52:36.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.007+0000 7f588cf4f700 1 -- 172.21.15.78:0/1359523802 learned_addr learned my addr 172.21.15.78:0/1359523802 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:36.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.007+0000 7f5887fff700 1 --2- 172.21.15.78:0/1359523802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58880c5b40 0x7f58880c4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:36.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.007+0000 7f588d750700 1 --2- 172.21.15.78:0/1359523802 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58880ce7c0 0x7f58880c4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:36.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.007+0000 7f588cf4f700 1 -- 172.21.15.78:0/1359523802 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58880ce7c0 msgr2=0x7f58880c4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:36.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.007+0000 7f588cf4f700 1 --2- 172.21.15.78:0/1359523802 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58880ce7c0 0x7f58880c4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:36.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.007+0000 7f588cf4f700 1 -- 172.21.15.78:0/1359523802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58880c5b40 msgr2=0x7f58880c4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:36.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.007+0000 7f588cf4f700 1 --2- 172.21.15.78:0/1359523802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58880c5b40 0x7f58880c4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:36.023 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.007+0000 7f588cf4f700 1 -- 172.21.15.78:0/1359523802 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58780094b0 con 0x7f58880713b0 2023-10-11T09:52:36.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.007+0000 7f5887fff700 1 --2- 172.21.15.78:0/1359523802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58880c5b40 0x7f58880c4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:36.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.011+0000 7f588cf4f700 1 --2- 172.21.15.78:0/1359523802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58880713b0 0x7f58880c3dd0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f587c014040 tx=0x7f587c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:36.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.011+0000 7f58857fa700 1 -- 172.21.15.78:0/1359523802 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587c0036c0 con 0x7f58880713b0 2023-10-11T09:52:36.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.011+0000 7f58857fa700 1 -- 172.21.15.78:0/1359523802 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587c01e070 con 0x7f58880713b0 2023-10-11T09:52:36.024 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.011+0000 7f588f1b3700 1 -- 172.21.15.78:0/1359523802 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f587c0094b0 con 0x7f58880713b0 2023-10-11T09:52:36.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.011+0000 7f58857fa700 1 -- 172.21.15.78:0/1359523802 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587c003a40 con 0x7f58880713b0 2023-10-11T09:52:36.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.011+0000 7f588f1b3700 1 -- 172.21.15.78:0/1359523802 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58880b1000 con 0x7f58880713b0 2023-10-11T09:52:36.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.011+0000 7f588f1b3700 1 -- 172.21.15.78:0/1359523802 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58880999d0 con 0x7f58880713b0 2023-10-11T09:52:36.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.011+0000 7f58857fa700 1 -- 172.21.15.78:0/1359523802 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f587c003ba0 con 0x7f58880713b0 2023-10-11T09:52:36.026 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.015+0000 7f58857fa700 1 --2- 172.21.15.78:0/1359523802 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5874065f00 0x7f58740683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:36.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.015+0000 7f58857fa700 1 -- 172.21.15.78:0/1359523802 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f587c084880 con 0x7f58880713b0 2023-10-11T09:52:36.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.015+0000 7f5887fff700 1 --2- 172.21.15.78:0/1359523802 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5874065f00 0x7f58740683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:36.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.015+0000 7f5887fff700 1 --2- 172.21.15.78:0/1359523802 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5874065f00 0x7f58740683b0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f58881f5640 tx=0x7f5878008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:36.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.023+0000 7f58857fa700 1 -- 172.21.15.78:0/1359523802 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f58880999d0 con 0x7f58880713b0 2023-10-11T09:52:36.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.207+0000 7f588f1b3700 1 -- 172.21.15.78:0/1359523802 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f58880999d0 con 0x7f58880713b0 2023-10-11T09:52:36.213 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.207+0000 7f58857fa700 1 -- 172.21.15.78:0/1359523802 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v129) v1 ==== 80+0+246 (secure 0 0 0) 0x7f58880999d0 con 0x7f58880713b0 2023-10-11T09:52:36.214 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6837/4189411531 2023-10-12T09:52:22.836499+0000 2023-10-11T09:52:36.214 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6835/4189411531 2023-10-12T09:52:22.836499+0000 2023-10-11T09:52:36.214 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.133:6833/1433079498 2023-10-12T09:52:02.877755+0000 2023-10-11T09:52:36.214 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.133:6832/1433079498 2023-10-12T09:52:02.877755+0000 2023-10-11T09:52:36.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.207+0000 7f588f1b3700 1 -- 172.21.15.78:0/1359523802 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5874065f00 msgr2=0x7f58740683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:36.216 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.207+0000 7f588f1b3700 1 --2- 172.21.15.78:0/1359523802 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5874065f00 0x7f58740683b0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f58881f5640 tx=0x7f5878008040 comp rx=0 tx=0).stop 2023-10-11T09:52:36.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.207+0000 7f588f1b3700 1 -- 172.21.15.78:0/1359523802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58880713b0 msgr2=0x7f58880c3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:36.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.207+0000 7f588f1b3700 1 --2- 172.21.15.78:0/1359523802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58880713b0 0x7f58880c3dd0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f587c014040 tx=0x7f587c004770 comp rx=0 tx=0).stop 2023-10-11T09:52:36.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.207+0000 7f588f1b3700 1 -- 172.21.15.78:0/1359523802 shutdown_connections 2023-10-11T09:52:36.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.207+0000 7f588f1b3700 1 --2- 172.21.15.78:0/1359523802 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5874065f00 0x7f58740683b0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:36.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.207+0000 7f588f1b3700 1 --2- 172.21.15.78:0/1359523802 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58880ce7c0 0x7f58880c4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:36.217 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.207+0000 7f588f1b3700 1 --2- 172.21.15.78:0/1359523802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58880c5b40 0x7f58880c4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:36.218 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.207+0000 7f588f1b3700 1 --2- 172.21.15.78:0/1359523802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58880713b0 0x7f58880c3dd0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:36.218 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.207+0000 7f588f1b3700 1 -- 172.21.15.78:0/1359523802 >> 172.21.15.78:0/1359523802 conn(0x7f58880cb920 msgr2=0x7f588815c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:36.218 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.207+0000 7f588f1b3700 1 -- 172.21.15.78:0/1359523802 shutdown_connections 2023-10-11T09:52:36.218 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.207+0000 7f588f1b3700 1 -- 172.21.15.78:0/1359523802 wait complete. 2023-10-11T09:52:36.219 INFO:teuthology.orchestra.run.smithi078.stderr:listed 4 entries 2023-10-11T09:52:36.237 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi078.front.sepia.ceph.com... 2023-10-11T09:52:36.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:36.238 DEBUG:teuthology.orchestra.run.smithi078:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-11T09:52:36.264 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi078.stderr:ceph-fuse[61006]: fuse finished with error 0 and tester_r 0 2023-10-11T09:52:36.269 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2023-10-11T09:52:36.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:36.270 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-11T09:52:36.451 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:52:36.452 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:52:36.472 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352292k,nr_inodes=4088073,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1430 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275376k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-11T09:52:36.472 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2023-10-11T09:52:36.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:36.473 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-11T09:52:36.536 INFO:teuthology.orchestra.run.smithi133.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-10-11T09:52:36.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T09:52:36.538 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:52:36.538 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:52:36.556 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352272k,nr_inodes=4088068,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17487 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-11T09:52:36.557 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2023-10-11T09:52:36.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:36.558 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-11T09:52:36.618 INFO:teuthology.orchestra.run.smithi078.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-11T09:52:36.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T09:52:36.620 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:52:36.620 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:52:36.640 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352292k,nr_inodes=4088073,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1430 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275376k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-11T09:52:36.641 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2023-10-11T09:52:36.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:36.642 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-11T09:52:36.704 INFO:teuthology.orchestra.run.smithi133.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-11T09:52:36.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T09:52:36.707 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:52:36.889 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.883+0000 7f58f23fc700 1 Processor -- start 2023-10-11T09:52:36.890 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.883+0000 7f58f23fc700 1 -- start start 2023-10-11T09:52:36.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.883+0000 7f58f23fc700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f58ec0ce700 0x7f58ec0c20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:36.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.887+0000 7f58f23fc700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58ec0cb220 con 0x7f58ec071720 2023-10-11T09:52:36.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.887+0000 7f58f23fc700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58ec1613f0 con 0x7f58ec161020 2023-10-11T09:52:36.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.887+0000 7f58f23fc700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58ec071c70 con 0x7f58ec0ce700 2023-10-11T09:52:36.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.887+0000 7f58f1bfb700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f58ec071720 0x7f58ec06bc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:38000/0 (socket says 172.21.15.78:38000) 2023-10-11T09:52:36.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.887+0000 7f58f0bf9700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f58ec0ce700 0x7f58ec0c20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:36.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.887+0000 7f58f1bfb700 1 -- 172.21.15.78:0/3707382616 learned_addr learned my addr 172.21.15.78:0/3707382616 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:36.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.887+0000 7f58f0bf9700 1 -- 172.21.15.78:0/3707382616 >> v1:172.21.15.133:6789/0 conn(0x7f58ec161020 legacy=0x7f58ec0cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:36.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.887+0000 7f58f0bf9700 1 -- 172.21.15.78:0/3707382616 >> v1:172.21.15.78:6789/0 conn(0x7f58ec071720 legacy=0x7f58ec06bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:36.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.887+0000 7f58f0bf9700 1 -- 172.21.15.78:0/3707382616 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ec0ca650 con 0x7f58ec0ce700 2023-10-11T09:52:36.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.887+0000 7f58f0bf9700 1 --2- 172.21.15.78:0/3707382616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58ec0ce700 0x7f58ec0c20c0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f58dc009840 tx=0x7f58dc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef02f61b9acefdc0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:36.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.887+0000 7f58e3fff700 1 -- 172.21.15.78:0/3707382616 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc005e50 con 0x7f58ec0ce700 2023-10-11T09:52:36.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.887+0000 7f58e3fff700 1 -- 172.21.15.78:0/3707382616 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58dc006030 con 0x7f58ec0ce700 2023-10-11T09:52:36.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.887+0000 7f58e3fff700 1 -- 172.21.15.78:0/3707382616 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc0063c0 con 0x7f58ec0ce700 2023-10-11T09:52:36.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.887+0000 7f58f23fc700 1 -- 172.21.15.78:0/3707382616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58ec0ce700 msgr2=0x7f58ec0c20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:36.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.887+0000 7f58f23fc700 1 --2- 172.21.15.78:0/3707382616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58ec0ce700 0x7f58ec0c20c0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f58dc009840 tx=0x7f58dc009b50 comp rx=0 tx=0).stop 2023-10-11T09:52:36.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.887+0000 7f58f23fc700 1 -- 172.21.15.78:0/3707382616 shutdown_connections 2023-10-11T09:52:36.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.887+0000 7f58f23fc700 1 --2- 172.21.15.78:0/3707382616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58ec0ce700 0x7f58ec0c20c0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:36.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.887+0000 7f58f23fc700 1 -- 172.21.15.78:0/3707382616 >> 172.21.15.78:0/3707382616 conn(0x7f58ec0cf820 msgr2=0x7f58ec0cb610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:36.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.887+0000 7f58f23fc700 1 -- 172.21.15.78:0/3707382616 shutdown_connections 2023-10-11T09:52:36.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.887+0000 7f58f23fc700 1 -- 172.21.15.78:0/3707382616 wait complete. 2023-10-11T09:52:36.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.891+0000 7f58f23fc700 1 Processor -- start 2023-10-11T09:52:36.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.891+0000 7f58f23fc700 1 -- start start 2023-10-11T09:52:36.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.891+0000 7f58f23fc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58ec071720 0x7f58ec1f5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:36.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.891+0000 7f58f23fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58ec0ce700 0x7f58ec06f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:36.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.891+0000 7f58f23fc700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58ec161020 0x7f58ec06ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:36.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.891+0000 7f58f23fc700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec1fa9f0 con 0x7f58ec0ce700 2023-10-11T09:52:36.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.891+0000 7f58f23fc700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec1645a0 con 0x7f58ec071720 2023-10-11T09:52:36.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.891+0000 7f58f23fc700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec1f9c30 con 0x7f58ec161020 2023-10-11T09:52:36.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.891+0000 7f58f0bf9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58ec0ce700 0x7f58ec06f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:36.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.891+0000 7f58f0bf9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58ec0ce700 0x7f58ec06f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43242/0 (socket says 172.21.15.78:43242) 2023-10-11T09:52:36.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.891+0000 7f58f0bf9700 1 -- 172.21.15.78:0/38874984 learned_addr learned my addr 172.21.15.78:0/38874984 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:36.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.895+0000 7f58f13fa700 1 --2- 172.21.15.78:0/38874984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58ec071720 0x7f58ec1f5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:36.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.895+0000 7f58f1bfb700 1 --2- 172.21.15.78:0/38874984 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58ec161020 0x7f58ec06ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:36.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.895+0000 7f58f0bf9700 1 -- 172.21.15.78:0/38874984 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58ec161020 msgr2=0x7f58ec06ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:36.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.895+0000 7f58f0bf9700 1 --2- 172.21.15.78:0/38874984 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58ec161020 0x7f58ec06ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:36.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.895+0000 7f58f0bf9700 1 -- 172.21.15.78:0/38874984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58ec071720 msgr2=0x7f58ec1f5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:36.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.895+0000 7f58f0bf9700 1 --2- 172.21.15.78:0/38874984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58ec071720 0x7f58ec1f5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:36.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.895+0000 7f58f0bf9700 1 -- 172.21.15.78:0/38874984 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58dc009450 con 0x7f58ec0ce700 2023-10-11T09:52:36.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.895+0000 7f58f0bf9700 1 --2- 172.21.15.78:0/38874984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58ec0ce700 0x7f58ec06f9c0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f58dc009b90 tx=0x7f58dc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:36.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.895+0000 7f58e1ffb700 1 -- 172.21.15.78:0/38874984 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc003ad0 con 0x7f58ec0ce700 2023-10-11T09:52:36.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.895+0000 7f58e1ffb700 1 -- 172.21.15.78:0/38874984 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58dc003c30 con 0x7f58ec0ce700 2023-10-11T09:52:36.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.895+0000 7f58e1ffb700 1 -- 172.21.15.78:0/38874984 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc0061a0 con 0x7f58ec0ce700 2023-10-11T09:52:36.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.895+0000 7f58f23fc700 1 -- 172.21.15.78:0/38874984 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58ec1f9f10 con 0x7f58ec0ce700 2023-10-11T09:52:36.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.895+0000 7f58f23fc700 1 -- 172.21.15.78:0/38874984 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58ec1fa4d0 con 0x7f58ec0ce700 2023-10-11T09:52:36.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.899+0000 7f58f23fc700 1 -- 172.21.15.78:0/38874984 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58ec067d00 con 0x7f58ec0ce700 2023-10-11T09:52:36.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.899+0000 7f58e1ffb700 1 -- 172.21.15.78:0/38874984 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f58dc022070 con 0x7f58ec0ce700 2023-10-11T09:52:36.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.899+0000 7f58e1ffb700 1 --2- 172.21.15.78:0/38874984 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f58d0065ed0 0x7f58d0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:36.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.899+0000 7f58e1ffb700 1 -- 172.21.15.78:0/38874984 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f58dc085260 con 0x7f58ec0ce700 2023-10-11T09:52:36.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.899+0000 7f58f13fa700 1 --2- 172.21.15.78:0/38874984 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f58d0065ed0 0x7f58d0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:36.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.903+0000 7f58f13fa700 1 --2- 172.21.15.78:0/38874984 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f58d0065ed0 0x7f58d0068380 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f58e8000f60 tx=0x7f58e8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:36.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:36.907+0000 7f58e1ffb700 1 -- 172.21.15.78:0/38874984 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f58dc052890 con 0x7f58ec0ce700 2023-10-11T09:52:37.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.115+0000 7f58f23fc700 1 -- 172.21.15.78:0/38874984 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58ec1fa710 con 0x7f58ec0ce700 2023-10-11T09:52:37.122 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.115+0000 7f58e1ffb700 1 -- 172.21.15.78:0/38874984 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4656 (secure 0 0 0) 0x7f58ec1fa710 con 0x7f58ec0ce700 2023-10-11T09:52:37.123 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:52:37.123 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":112,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8452,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/2481257990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2481257990},{"type":"v1","addr":"172.21.15.78:6836","nonce":2481257990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":8464,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1035227282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1035227282},{"type":"v1","addr":"172.21.15.133:6835","nonce":1035227282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":8680,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/36737634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":36737634},{"type":"v1","addr":"172.21.15.133:6833","nonce":36737634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":101}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:51:40.211859+0000","modified":"2023-10-11T09:52:29.509275+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9336},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9336":{"gid":9336,"name":"c","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.78:6837/3884456900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":3884456900},{"type":"v1","addr":"172.21.15.78:6837","nonce":3884456900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-11T09:52:37.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.119+0000 7f58f23fc700 1 -- 172.21.15.78:0/38874984 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f58d0065ed0 msgr2=0x7f58d0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:37.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.119+0000 7f58f23fc700 1 --2- 172.21.15.78:0/38874984 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f58d0065ed0 0x7f58d0068380 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f58e8000f60 tx=0x7f58e8009040 comp rx=0 tx=0).stop 2023-10-11T09:52:37.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.119+0000 7f58f23fc700 1 -- 172.21.15.78:0/38874984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58ec0ce700 msgr2=0x7f58ec06f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:37.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.119+0000 7f58f23fc700 1 --2- 172.21.15.78:0/38874984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58ec0ce700 0x7f58ec06f9c0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f58dc009b90 tx=0x7f58dc004770 comp rx=0 tx=0).stop 2023-10-11T09:52:37.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.119+0000 7f58f23fc700 1 -- 172.21.15.78:0/38874984 shutdown_connections 2023-10-11T09:52:37.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.119+0000 7f58f23fc700 1 --2- 172.21.15.78:0/38874984 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f58d0065ed0 0x7f58d0068380 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:37.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.119+0000 7f58f23fc700 1 --2- 172.21.15.78:0/38874984 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58ec161020 0x7f58ec06ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:37.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.119+0000 7f58f23fc700 1 --2- 172.21.15.78:0/38874984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58ec0ce700 0x7f58ec06f9c0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:37.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.119+0000 7f58f23fc700 1 --2- 172.21.15.78:0/38874984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58ec071720 0x7f58ec1f5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:37.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.119+0000 7f58f23fc700 1 -- 172.21.15.78:0/38874984 >> 172.21.15.78:0/38874984 conn(0x7f58ec0cf820 msgr2=0x7f58ec0aff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:37.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.119+0000 7f58f23fc700 1 -- 172.21.15.78:0/38874984 shutdown_connections 2023-10-11T09:52:37.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.119+0000 7f58f23fc700 1 -- 172.21.15.78:0/38874984 wait complete. 2023-10-11T09:52:37.129 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 112 2023-10-11T09:52:37.146 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:52:37.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.315+0000 7f05188c4700 1 Processor -- start 2023-10-11T09:52:37.323 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.315+0000 7f05188c4700 1 -- start start 2023-10-11T09:52:37.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.319+0000 7f05188c4700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f051415a230 0x7f05140d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:37.326 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.319+0000 7f05188c4700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f051415d760 0x7f05140cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:37.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.319+0000 7f05188c4700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f051415dc40 con 0x7f05140cfdd0 2023-10-11T09:52:37.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.319+0000 7f05188c4700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05140cf1a0 con 0x7f051415d760 2023-10-11T09:52:37.327 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.319+0000 7f05188c4700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05140cf3b0 con 0x7f051415a230 2023-10-11T09:52:37.328 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.319+0000 7f0512ffd700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f051415d760 0x7f05140cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:37.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.319+0000 7f0513fff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f05140cfdd0 0x7f05140b7ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:38008/0 (socket says 172.21.15.78:38008) 2023-10-11T09:52:37.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.319+0000 7f0512ffd700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f051415d760 0x7f05140cf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43258/0 (socket says 172.21.15.78:43258) 2023-10-11T09:52:37.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.319+0000 7f0513fff700 1 -- 172.21.15.78:0/3266925837 learned_addr learned my addr 172.21.15.78:0/3266925837 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:37.329 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.319+0000 7f05137fe700 1 --2- 172.21.15.78:0/3266925837 >> v2:172.21.15.133:3300/0 conn(0x7f051415a230 0x7f05140d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:37.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.319+0000 7f0512ffd700 1 -- 172.21.15.78:0/3266925837 >> v2:172.21.15.133:3300/0 conn(0x7f051415a230 msgr2=0x7f05140d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:37.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.319+0000 7f0512ffd700 1 --2- 172.21.15.78:0/3266925837 >> v2:172.21.15.133:3300/0 conn(0x7f051415a230 0x7f05140d0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:37.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.319+0000 7f0512ffd700 1 -- 172.21.15.78:0/3266925837 >> v1:172.21.15.78:6789/0 conn(0x7f05140cfdd0 legacy=0x7f05140b7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:37.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.319+0000 7f0512ffd700 1 -- 172.21.15.78:0/3266925837 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05140cd6d0 con 0x7f051415d760 2023-10-11T09:52:37.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.319+0000 7f0512ffd700 1 --2- 172.21.15.78:0/3266925837 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f051415d760 0x7f05140cf890 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f051408c220 tx=0x7f0508009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a414ee76242982f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:37.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.319+0000 7f05127fc700 1 -- 172.21.15.78:0/3266925837 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0508011830 con 0x7f051415d760 2023-10-11T09:52:37.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.323+0000 7f05127fc700 1 -- 172.21.15.78:0/3266925837 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0508011990 con 0x7f051415d760 2023-10-11T09:52:37.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.323+0000 7f05127fc700 1 -- 172.21.15.78:0/3266925837 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0508011d40 con 0x7f051415d760 2023-10-11T09:52:37.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.323+0000 7f05188c4700 1 -- 172.21.15.78:0/3266925837 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f051415d760 msgr2=0x7f05140cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:37.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.323+0000 7f05188c4700 1 --2- 172.21.15.78:0/3266925837 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f051415d760 0x7f05140cf890 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f051408c220 tx=0x7f0508009b30 comp rx=0 tx=0).stop 2023-10-11T09:52:37.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.323+0000 7f05188c4700 1 -- 172.21.15.78:0/3266925837 shutdown_connections 2023-10-11T09:52:37.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.323+0000 7f05188c4700 1 --2- 172.21.15.78:0/3266925837 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f051415d760 0x7f05140cf890 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:37.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.323+0000 7f05188c4700 1 --2- 172.21.15.78:0/3266925837 >> v2:172.21.15.133:3300/0 conn(0x7f051415a230 0x7f05140d0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:37.333 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.323+0000 7f05188c4700 1 -- 172.21.15.78:0/3266925837 >> 172.21.15.78:0/3266925837 conn(0x7f051406aa20 msgr2=0x7f051406adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:37.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.323+0000 7f05188c4700 1 -- 172.21.15.78:0/3266925837 shutdown_connections 2023-10-11T09:52:37.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.323+0000 7f05188c4700 1 -- 172.21.15.78:0/3266925837 wait complete. 2023-10-11T09:52:37.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.327+0000 7f05188c4700 1 Processor -- start 2023-10-11T09:52:37.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.327+0000 7f05188c4700 1 -- start start 2023-10-11T09:52:37.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.327+0000 7f05188c4700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f05140cfdd0 0x7f05140b8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:37.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.327+0000 7f05188c4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f051415a230 0x7f05140b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:37.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.327+0000 7f05188c4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f051415d760 0x7f05141f5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:37.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.327+0000 7f05188c4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05140b6440 con 0x7f051415d760 2023-10-11T09:52:37.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.327+0000 7f05188c4700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05140c92c0 con 0x7f051415a230 2023-10-11T09:52:37.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.327+0000 7f05188c4700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05140c95a0 con 0x7f05140cfdd0 2023-10-11T09:52:37.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.327+0000 7f0513fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f051415d760 0x7f05141f5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:37.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.327+0000 7f0513fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f051415d760 0x7f05141f5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43268/0 (socket says 172.21.15.78:43268) 2023-10-11T09:52:37.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.327+0000 7f0513fff700 1 -- 172.21.15.78:0/1307433211 learned_addr learned my addr 172.21.15.78:0/1307433211 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:37.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.327+0000 7f05137fe700 1 --2- 172.21.15.78:0/1307433211 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f05140cfdd0 0x7f05140b8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:37.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.327+0000 7f0512ffd700 1 --2- 172.21.15.78:0/1307433211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f051415a230 0x7f05140b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:37.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.331+0000 7f0513fff700 1 -- 172.21.15.78:0/1307433211 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f05140cfdd0 msgr2=0x7f05140b8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:37.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.331+0000 7f0513fff700 1 --2- 172.21.15.78:0/1307433211 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f05140cfdd0 0x7f05140b8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:37.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.331+0000 7f0513fff700 1 -- 172.21.15.78:0/1307433211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f051415a230 msgr2=0x7f05140b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:37.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.331+0000 7f0513fff700 1 --2- 172.21.15.78:0/1307433211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f051415a230 0x7f05140b3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:37.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.331+0000 7f0513fff700 1 -- 172.21.15.78:0/1307433211 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05080094b0 con 0x7f051415d760 2023-10-11T09:52:37.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.331+0000 7f0513fff700 1 --2- 172.21.15.78:0/1307433211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f051415d760 0x7f05141f5900 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f050c00cbc0 tx=0x7f050c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:37.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.331+0000 7f04fbfff700 1 -- 172.21.15.78:0/1307433211 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c007480 con 0x7f051415d760 2023-10-11T09:52:37.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.331+0000 7f04fbfff700 1 -- 172.21.15.78:0/1307433211 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f050c0075e0 con 0x7f051415d760 2023-10-11T09:52:37.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.331+0000 7f04fbfff700 1 -- 172.21.15.78:0/1307433211 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c007950 con 0x7f051415d760 2023-10-11T09:52:37.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.331+0000 7f05188c4700 1 -- 172.21.15.78:0/1307433211 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05140b5d80 con 0x7f051415d760 2023-10-11T09:52:37.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.331+0000 7f05188c4700 1 -- 172.21.15.78:0/1307433211 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05140b3a90 con 0x7f051415d760 2023-10-11T09:52:37.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.331+0000 7f05188c4700 1 -- 172.21.15.78:0/1307433211 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05140b3c80 con 0x7f051415d760 2023-10-11T09:52:37.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.335+0000 7f04fbfff700 1 -- 172.21.15.78:0/1307433211 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f050c01b3b0 con 0x7f051415d760 2023-10-11T09:52:37.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.335+0000 7f04fbfff700 1 --2- 172.21.15.78:0/1307433211 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0500065ed0 0x7f0500068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:37.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.335+0000 7f04fbfff700 1 -- 172.21.15.78:0/1307433211 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f050c083170 con 0x7f051415d760 2023-10-11T09:52:37.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.335+0000 7f05137fe700 1 --2- 172.21.15.78:0/1307433211 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0500065ed0 0x7f0500068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:37.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.335+0000 7f05137fe700 1 --2- 172.21.15.78:0/1307433211 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0500065ed0 0x7f0500068380 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f0504009a00 tx=0x7f0504008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:37.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.343+0000 7f04fbfff700 1 -- 172.21.15.78:0/1307433211 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f050c050720 con 0x7f051415d760 2023-10-11T09:52:37.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.515+0000 7f05188c4700 1 -- 172.21.15.78:0/1307433211 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05140b6580 con 0x7f051415d760 2023-10-11T09:52:37.524 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.515+0000 7f04fbfff700 1 -- 172.21.15.78:0/1307433211 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+18154 (secure 0 0 0) 0x7f05140b6580 con 0x7f051415d760 2023-10-11T09:52:37.525 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:52:37.525 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":129,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:52:22.836530+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:51:38.592892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-11T09:51:39.204588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6837/4189411531":"2023-10-12T09:52:22.836499+0000","172.21.15.78:6835/4189411531":"2023-10-12T09:52:22.836499+0000","172.21.15.133:6833/1433079498":"2023-10-12T09:52:02.877755+0000","172.21.15.133:6832/1433079498":"2023-10-12T09:52:02.877755+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:52:37.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.519+0000 7f05188c4700 1 -- 172.21.15.78:0/1307433211 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0500065ed0 msgr2=0x7f0500068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:37.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.519+0000 7f05188c4700 1 --2- 172.21.15.78:0/1307433211 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0500065ed0 0x7f0500068380 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f0504009a00 tx=0x7f0504008040 comp rx=0 tx=0).stop 2023-10-11T09:52:37.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.519+0000 7f05188c4700 1 -- 172.21.15.78:0/1307433211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f051415d760 msgr2=0x7f05141f5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:37.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.519+0000 7f05188c4700 1 --2- 172.21.15.78:0/1307433211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f051415d760 0x7f05141f5900 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f050c00cbc0 tx=0x7f050c00cbf0 comp rx=0 tx=0).stop 2023-10-11T09:52:37.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.519+0000 7f05188c4700 1 -- 172.21.15.78:0/1307433211 shutdown_connections 2023-10-11T09:52:37.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.519+0000 7f05188c4700 1 --2- 172.21.15.78:0/1307433211 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0500065ed0 0x7f0500068380 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:37.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.519+0000 7f05188c4700 1 --2- 172.21.15.78:0/1307433211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f051415d760 0x7f05141f5900 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:37.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.519+0000 7f05188c4700 1 --2- 172.21.15.78:0/1307433211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f051415a230 0x7f05140b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:37.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.519+0000 7f05188c4700 1 --2- 172.21.15.78:0/1307433211 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f05140cfdd0 0x7f05140b8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:37.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.519+0000 7f05188c4700 1 -- 172.21.15.78:0/1307433211 >> 172.21.15.78:0/1307433211 conn(0x7f051406aa20 msgr2=0x7f051415a780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:37.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.519+0000 7f05188c4700 1 -- 172.21.15.78:0/1307433211 shutdown_connections 2023-10-11T09:52:37.529 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.519+0000 7f05188c4700 1 -- 172.21.15.78:0/1307433211 wait complete. 2023-10-11T09:52:37.552 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T09:40:28.405906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T09:40:34.657665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T09:51:38.592892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T09:51:39.204588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '127', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T09:52:37.553 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T09:52:37.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.727+0000 7fac98570700 1 Processor -- start 2023-10-11T09:52:37.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.727+0000 7fac98570700 1 -- start start 2023-10-11T09:52:37.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.727+0000 7fac98570700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fac90159770 0x7fac900c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:37.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.727+0000 7fac98570700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac90070080 0x7fac9006cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:37.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.727+0000 7fac98570700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac9006c890 con 0x7fac900704d0 2023-10-11T09:52:37.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.727+0000 7fac98570700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac900ca770 con 0x7fac90159770 2023-10-11T09:52:37.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.727+0000 7fac98570700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac9006bf60 con 0x7fac90070080 2023-10-11T09:52:37.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.727+0000 7fac9630c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fac90159770 0x7fac900c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:37.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.727+0000 7fac9630c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fac90159770 0x7fac900c9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43274/0 (socket says 172.21.15.78:43274) 2023-10-11T09:52:37.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.727+0000 7fac9630c700 1 -- 172.21.15.78:0/516634171 learned_addr learned my addr 172.21.15.78:0/516634171 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:37.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.727+0000 7fac95b0b700 1 --2- 172.21.15.78:0/516634171 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac90070080 0x7fac9006cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:37.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.731+0000 7fac9630c700 1 -- 172.21.15.78:0/516634171 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac90070080 msgr2=0x7fac9006cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:37.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.731+0000 7fac9630c700 1 --2- 172.21.15.78:0/516634171 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac90070080 0x7fac9006cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:37.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.731+0000 7fac9630c700 1 -- 172.21.15.78:0/516634171 >> v1:172.21.15.133:6789/0 conn(0x7fac900704d0 legacy=0x7fac9006d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:37.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.731+0000 7fac9630c700 1 -- 172.21.15.78:0/516634171 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac9006c240 con 0x7fac90159770 2023-10-11T09:52:37.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.731+0000 7fac9630c700 1 --2- 172.21.15.78:0/516634171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac90159770 0x7fac900c9ee0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fac8c009910 tx=0x7fac8c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1acc99c4f1d49f16 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:37.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.731+0000 7fac9530a700 1 -- 172.21.15.78:0/516634171 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c011830 con 0x7fac90159770 2023-10-11T09:52:37.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.731+0000 7fac9530a700 1 -- 172.21.15.78:0/516634171 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8c011990 con 0x7fac90159770 2023-10-11T09:52:37.740 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.731+0000 7fac9530a700 1 -- 172.21.15.78:0/516634171 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c011d40 con 0x7fac90159770 2023-10-11T09:52:37.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.731+0000 7fac98570700 1 -- 172.21.15.78:0/516634171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac90159770 msgr2=0x7fac900c9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:37.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.731+0000 7fac98570700 1 --2- 172.21.15.78:0/516634171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac90159770 0x7fac900c9ee0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fac8c009910 tx=0x7fac8c009c20 comp rx=0 tx=0).stop 2023-10-11T09:52:37.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.731+0000 7fac98570700 1 -- 172.21.15.78:0/516634171 shutdown_connections 2023-10-11T09:52:37.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.731+0000 7fac98570700 1 --2- 172.21.15.78:0/516634171 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac90070080 0x7fac9006cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:37.741 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.731+0000 7fac98570700 1 --2- 172.21.15.78:0/516634171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac90159770 0x7fac900c9ee0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:37.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.731+0000 7fac98570700 1 -- 172.21.15.78:0/516634171 >> 172.21.15.78:0/516634171 conn(0x7fac9006ae10 msgr2=0x7fac900b0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:37.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.731+0000 7fac98570700 1 -- 172.21.15.78:0/516634171 shutdown_connections 2023-10-11T09:52:37.742 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.731+0000 7fac98570700 1 -- 172.21.15.78:0/516634171 wait complete. 2023-10-11T09:52:37.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.735+0000 7fac98570700 1 Processor -- start 2023-10-11T09:52:37.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.735+0000 7fac98570700 1 -- start start 2023-10-11T09:52:37.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.735+0000 7fac98570700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac90070080 0x7fac90154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:37.745 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.735+0000 7fac98570700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac900704d0 0x7fac900c3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:37.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.735+0000 7fac98570700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac90159770 0x7fac90159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:37.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.735+0000 7fac98570700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac900c38c0 con 0x7fac90070080 2023-10-11T09:52:37.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.735+0000 7fac98570700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac901efdc0 con 0x7fac90159770 2023-10-11T09:52:37.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.735+0000 7fac98570700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac900c29d0 con 0x7fac900704d0 2023-10-11T09:52:37.746 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.735+0000 7fac9630c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac90070080 0x7fac90154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:37.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.735+0000 7fac9630c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac90070080 0x7fac90154180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43286/0 (socket says 172.21.15.78:43286) 2023-10-11T09:52:37.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.735+0000 7fac9630c700 1 -- 172.21.15.78:0/1458926672 learned_addr learned my addr 172.21.15.78:0/1458926672 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:37.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.735+0000 7fac95b0b700 1 --2- 172.21.15.78:0/1458926672 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac900704d0 0x7fac900c3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:37.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.735+0000 7fac96b0d700 1 --2- 172.21.15.78:0/1458926672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac90159770 0x7fac90159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:37.747 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.735+0000 7fac9630c700 1 -- 172.21.15.78:0/1458926672 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac900704d0 msgr2=0x7fac900c3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:37.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.735+0000 7fac9630c700 1 --2- 172.21.15.78:0/1458926672 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac900704d0 0x7fac900c3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:37.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.735+0000 7fac9630c700 1 -- 172.21.15.78:0/1458926672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac90159770 msgr2=0x7fac90159de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:37.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.735+0000 7fac9630c700 1 --2- 172.21.15.78:0/1458926672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac90159770 0x7fac90159de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:37.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.735+0000 7fac9630c700 1 -- 172.21.15.78:0/1458926672 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac8c009570 con 0x7fac90070080 2023-10-11T09:52:37.748 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.735+0000 7fac9630c700 1 --2- 172.21.15.78:0/1458926672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac90070080 0x7fac90154180 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fac8c000c00 tx=0x7fac8c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:37.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.735+0000 7fac82ffd700 1 -- 172.21.15.78:0/1458926672 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c004480 con 0x7fac90070080 2023-10-11T09:52:37.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.735+0000 7fac82ffd700 1 -- 172.21.15.78:0/1458926672 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8c01f070 con 0x7fac90070080 2023-10-11T09:52:37.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.735+0000 7fac98570700 1 -- 172.21.15.78:0/1458926672 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac900c2cb0 con 0x7fac90070080 2023-10-11T09:52:37.749 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.735+0000 7fac82ffd700 1 -- 172.21.15.78:0/1458926672 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c003900 con 0x7fac90070080 2023-10-11T09:52:37.750 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.735+0000 7fac98570700 1 -- 172.21.15.78:0/1458926672 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac900c3270 con 0x7fac90070080 2023-10-11T09:52:37.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.739+0000 7fac98570700 1 -- 172.21.15.78:0/1458926672 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac9006f4f0 con 0x7fac90070080 2023-10-11T09:52:37.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.739+0000 7fac82ffd700 1 -- 172.21.15.78:0/1458926672 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fac8c021030 con 0x7fac90070080 2023-10-11T09:52:37.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.743+0000 7fac82ffd700 1 --2- 172.21.15.78:0/1458926672 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fac74065ed0 0x7fac74068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:37.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.743+0000 7fac82ffd700 1 -- 172.21.15.78:0/1458926672 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6234+0+0 (secure 0 0 0) 0x7fac8c084c80 con 0x7fac90070080 2023-10-11T09:52:37.752 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.743+0000 7fac95b0b700 1 --2- 172.21.15.78:0/1458926672 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fac74065ed0 0x7fac74068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:37.753 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.743+0000 7fac95b0b700 1 --2- 172.21.15.78:0/1458926672 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fac74065ed0 0x7fac74068380 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fac900c46e0 tx=0x7fac84008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:37.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.747+0000 7fac82ffd700 1 -- 172.21.15.78:0/1458926672 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fac8c052210 con 0x7fac90070080 2023-10-11T09:52:37.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.919+0000 7fac98570700 1 -- 172.21.15.78:0/1458926672 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fac90074530 con 0x7fac90070080 2023-10-11T09:52:37.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.919+0000 7fac82ffd700 1 -- 172.21.15.78:0/1458926672 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v129) v1 ==== 93+0+10 (secure 0 0 0) 0x7fac8c006da0 con 0x7fac90070080 2023-10-11T09:52:37.927 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2023-10-11T09:52:37.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.923+0000 7fac98570700 1 -- 172.21.15.78:0/1458926672 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fac74065ed0 msgr2=0x7fac74068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:37.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.923+0000 7fac98570700 1 --2- 172.21.15.78:0/1458926672 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fac74065ed0 0x7fac74068380 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fac900c46e0 tx=0x7fac84008040 comp rx=0 tx=0).stop 2023-10-11T09:52:37.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.923+0000 7fac98570700 1 -- 172.21.15.78:0/1458926672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac90070080 msgr2=0x7fac90154180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:37.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.923+0000 7fac98570700 1 --2- 172.21.15.78:0/1458926672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac90070080 0x7fac90154180 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fac8c000c00 tx=0x7fac8c004770 comp rx=0 tx=0).stop 2023-10-11T09:52:37.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.923+0000 7fac98570700 1 -- 172.21.15.78:0/1458926672 shutdown_connections 2023-10-11T09:52:37.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.923+0000 7fac98570700 1 --2- 172.21.15.78:0/1458926672 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fac74065ed0 0x7fac74068380 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:37.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.923+0000 7fac98570700 1 --2- 172.21.15.78:0/1458926672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac90159770 0x7fac90159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:37.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.923+0000 7fac98570700 1 --2- 172.21.15.78:0/1458926672 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac900704d0 0x7fac900c3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:37.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.923+0000 7fac98570700 1 --2- 172.21.15.78:0/1458926672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac90070080 0x7fac90154180 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:37.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.923+0000 7fac98570700 1 -- 172.21.15.78:0/1458926672 >> 172.21.15.78:0/1458926672 conn(0x7fac9006ae10 msgr2=0x7fac900ae8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:37.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.923+0000 7fac98570700 1 -- 172.21.15.78:0/1458926672 shutdown_connections 2023-10-11T09:52:37.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:37.923+0000 7fac98570700 1 -- 172.21.15.78:0/1458926672 wait complete. 2023-10-11T09:52:37.950 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T09:52:38.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.123+0000 7f5d5b40f700 1 Processor -- start 2023-10-11T09:52:38.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.123+0000 7f5d5b40f700 1 -- start start 2023-10-11T09:52:38.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.127+0000 7f5d5b40f700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5d540d16f0 0x7f5d540cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:38.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.127+0000 7f5d5b40f700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f5d540b74c0 0x7f5d540cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:38.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.127+0000 7f5d5b40f700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d5409c960 con 0x7f5d540b4530 2023-10-11T09:52:38.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.127+0000 7f5d5b40f700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d5415d180 con 0x7f5d540b74c0 2023-10-11T09:52:38.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.127+0000 7f5d5b40f700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d540b5fd0 con 0x7f5d540d16f0 2023-10-11T09:52:38.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.127+0000 7f5d5ac0e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f5d540b74c0 0x7f5d540cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:38.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.127+0000 7f5d5ac0e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f5d540b74c0 0x7f5d540cdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43302/0 (socket says 172.21.15.78:43302) 2023-10-11T09:52:38.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.127+0000 7f5d5ac0e700 1 -- 172.21.15.78:0/4262962302 learned_addr learned my addr 172.21.15.78:0/4262962302 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:38.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.127+0000 7f5d59c0c700 1 --2- 172.21.15.78:0/4262962302 >> v2:172.21.15.133:3300/0 conn(0x7f5d540d16f0 0x7f5d540cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:38.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.127+0000 7f5d5ac0e700 1 -- 172.21.15.78:0/4262962302 >> v2:172.21.15.133:3300/0 conn(0x7f5d540d16f0 msgr2=0x7f5d540cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:38.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.127+0000 7f5d5ac0e700 1 --2- 172.21.15.78:0/4262962302 >> v2:172.21.15.133:3300/0 conn(0x7f5d540d16f0 0x7f5d540cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:38.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.127+0000 7f5d5ac0e700 1 -- 172.21.15.78:0/4262962302 >> v1:172.21.15.133:6789/0 conn(0x7f5d540b4530 legacy=0x7f5d540d0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:38.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.127+0000 7f5d5ac0e700 1 -- 172.21.15.78:0/4262962302 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d540cfb70 con 0x7f5d540b74c0 2023-10-11T09:52:38.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.127+0000 7f5d5ac0e700 1 --2- 172.21.15.78:0/4262962302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d540b74c0 0x7f5d540cdc70 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f5d5408c1b0 tx=0x7f5d4c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb1aed32dd697f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:38.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.127+0000 7f5d5940b700 1 -- 172.21.15.78:0/4262962302 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c00e070 con 0x7f5d540b74c0 2023-10-11T09:52:38.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.127+0000 7f5d5940b700 1 -- 172.21.15.78:0/4262962302 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4c00b910 con 0x7f5d540b74c0 2023-10-11T09:52:38.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.127+0000 7f5d5940b700 1 -- 172.21.15.78:0/4262962302 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c00bcc0 con 0x7f5d540b74c0 2023-10-11T09:52:38.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.127+0000 7f5d5b40f700 1 -- 172.21.15.78:0/4262962302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d540b74c0 msgr2=0x7f5d540cdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:38.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.127+0000 7f5d5b40f700 1 --2- 172.21.15.78:0/4262962302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d540b74c0 0x7f5d540cdc70 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f5d5408c1b0 tx=0x7f5d4c00b330 comp rx=0 tx=0).stop 2023-10-11T09:52:38.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.131+0000 7f5d5b40f700 1 -- 172.21.15.78:0/4262962302 shutdown_connections 2023-10-11T09:52:38.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.131+0000 7f5d5b40f700 1 --2- 172.21.15.78:0/4262962302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d540b74c0 0x7f5d540cdc70 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:38.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.131+0000 7f5d5b40f700 1 --2- 172.21.15.78:0/4262962302 >> v2:172.21.15.133:3300/0 conn(0x7f5d540d16f0 0x7f5d540cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:38.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.131+0000 7f5d5b40f700 1 -- 172.21.15.78:0/4262962302 >> 172.21.15.78:0/4262962302 conn(0x7f5d5409c3d0 msgr2=0x7f5d540d30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:38.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.131+0000 7f5d5b40f700 1 -- 172.21.15.78:0/4262962302 shutdown_connections 2023-10-11T09:52:38.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.131+0000 7f5d5b40f700 1 -- 172.21.15.78:0/4262962302 wait complete. 2023-10-11T09:52:38.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.131+0000 7f5d5b40f700 1 Processor -- start 2023-10-11T09:52:38.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.131+0000 7f5d5b40f700 1 -- start start 2023-10-11T09:52:38.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.135+0000 7f5d5b40f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d540b4530 0x7f5d540c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:38.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.135+0000 7f5d5b40f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d540b74c0 0x7f5d5415a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:38.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.135+0000 7f5d5b40f700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5d540d16f0 0x7f5d5409ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:38.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.135+0000 7f5d5b40f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d540c8470 con 0x7f5d540b4530 2023-10-11T09:52:38.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.135+0000 7f5d5b40f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d540c8750 con 0x7f5d540b74c0 2023-10-11T09:52:38.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.135+0000 7f5d5b40f700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d540c8a30 con 0x7f5d540d16f0 2023-10-11T09:52:38.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.135+0000 7f5d5a40d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d540b4530 0x7f5d540c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:38.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.135+0000 7f5d5a40d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d540b4530 0x7f5d540c7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43304/0 (socket says 172.21.15.78:43304) 2023-10-11T09:52:38.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.135+0000 7f5d5a40d700 1 -- 172.21.15.78:0/3523433706 learned_addr learned my addr 172.21.15.78:0/3523433706 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:38.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.135+0000 7f5d59c0c700 1 --2- 172.21.15.78:0/3523433706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d540b74c0 0x7f5d5415a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:38.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.135+0000 7f5d5ac0e700 1 --2- 172.21.15.78:0/3523433706 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5d540d16f0 0x7f5d5409ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:38.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.135+0000 7f5d5a40d700 1 -- 172.21.15.78:0/3523433706 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5d540d16f0 msgr2=0x7f5d5409ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:38.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.135+0000 7f5d5a40d700 1 --2- 172.21.15.78:0/3523433706 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5d540d16f0 0x7f5d5409ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:38.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.135+0000 7f5d5a40d700 1 -- 172.21.15.78:0/3523433706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d540b74c0 msgr2=0x7f5d5415a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:38.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.135+0000 7f5d5a40d700 1 --2- 172.21.15.78:0/3523433706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d540b74c0 0x7f5d5415a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:38.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.135+0000 7f5d5a40d700 1 -- 172.21.15.78:0/3523433706 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d4c009b30 con 0x7f5d540b4530 2023-10-11T09:52:38.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.135+0000 7f5d5ac0e700 1 --2- 172.21.15.78:0/3523433706 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5d540d16f0 0x7f5d5409ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:38.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.135+0000 7f5d5a40d700 1 --2- 172.21.15.78:0/3523433706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d540b4530 0x7f5d540c7c40 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f5d5000c450 tx=0x7f5d5000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:38.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.135+0000 7f5d4affd700 1 -- 172.21.15.78:0/3523433706 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d500052d0 con 0x7f5d540b4530 2023-10-11T09:52:38.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.135+0000 7f5d4affd700 1 -- 172.21.15.78:0/3523433706 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d50005430 con 0x7f5d540b4530 2023-10-11T09:52:38.150 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.135+0000 7f5d5b40f700 1 -- 172.21.15.78:0/3523433706 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d540c8d10 con 0x7f5d540b4530 2023-10-11T09:52:38.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.135+0000 7f5d4affd700 1 -- 172.21.15.78:0/3523433706 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d50003930 con 0x7f5d540b4530 2023-10-11T09:52:38.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.135+0000 7f5d5b40f700 1 -- 172.21.15.78:0/3523433706 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d540c8180 con 0x7f5d540b4530 2023-10-11T09:52:38.151 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.139+0000 7f5d5b40f700 1 -- 172.21.15.78:0/3523433706 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d540744c0 con 0x7f5d540b4530 2023-10-11T09:52:38.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.139+0000 7f5d4affd700 1 -- 172.21.15.78:0/3523433706 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f5d50003a90 con 0x7f5d540b4530 2023-10-11T09:52:38.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.139+0000 7f5d4affd700 1 --2- 172.21.15.78:0/3523433706 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5d38065f00 0x7f5d380683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:38.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.139+0000 7f5d4affd700 1 -- 172.21.15.78:0/3523433706 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f5d50082bb0 con 0x7f5d540b4530 2023-10-11T09:52:38.152 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.143+0000 7f5d59c0c700 1 --2- 172.21.15.78:0/3523433706 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5d38065f00 0x7f5d380683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:38.153 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.143+0000 7f5d59c0c700 1 --2- 172.21.15.78:0/3523433706 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5d38065f00 0x7f5d380683b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f5d44009b50 tx=0x7f5d44009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:38.155 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.147+0000 7f5d4affd700 1 -- 172.21.15.78:0/3523433706 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f5d500501e0 con 0x7f5d540b4530 2023-10-11T09:52:38.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.323+0000 7f5d5b40f700 1 -- 172.21.15.78:0/3523433706 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5d540d3960 con 0x7f5d540b4530 2023-10-11T09:52:38.332 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.323+0000 7f5d4affd700 1 -- 172.21.15.78:0/3523433706 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v129) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5d500170a0 con 0x7f5d540b4530 2023-10-11T09:52:38.332 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 8 2023-10-11T09:52:38.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.331+0000 7f5d5b40f700 1 -- 172.21.15.78:0/3523433706 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5d38065f00 msgr2=0x7f5d380683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:38.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.331+0000 7f5d5b40f700 1 --2- 172.21.15.78:0/3523433706 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5d38065f00 0x7f5d380683b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f5d44009b50 tx=0x7f5d44009240 comp rx=0 tx=0).stop 2023-10-11T09:52:38.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.331+0000 7f5d5b40f700 1 -- 172.21.15.78:0/3523433706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d540b4530 msgr2=0x7f5d540c7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:38.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.331+0000 7f5d5b40f700 1 --2- 172.21.15.78:0/3523433706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d540b4530 0x7f5d540c7c40 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f5d5000c450 tx=0x7f5d5000c760 comp rx=0 tx=0).stop 2023-10-11T09:52:38.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.331+0000 7f5d5b40f700 1 -- 172.21.15.78:0/3523433706 shutdown_connections 2023-10-11T09:52:38.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.331+0000 7f5d5b40f700 1 --2- 172.21.15.78:0/3523433706 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5d38065f00 0x7f5d380683b0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:38.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.331+0000 7f5d5b40f700 1 --2- 172.21.15.78:0/3523433706 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5d540d16f0 0x7f5d5409ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:38.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.331+0000 7f5d5b40f700 1 --2- 172.21.15.78:0/3523433706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d540b74c0 0x7f5d5415a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:38.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.331+0000 7f5d5b40f700 1 --2- 172.21.15.78:0/3523433706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d540b4530 0x7f5d540c7c40 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:38.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.331+0000 7f5d5b40f700 1 -- 172.21.15.78:0/3523433706 >> 172.21.15.78:0/3523433706 conn(0x7f5d5409c3d0 msgr2=0x7f5d5415fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:38.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.331+0000 7f5d5b40f700 1 -- 172.21.15.78:0/3523433706 shutdown_connections 2023-10-11T09:52:38.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.331+0000 7f5d5b40f700 1 -- 172.21.15.78:0/3523433706 wait complete. 2023-10-11T09:52:38.360 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T09:52:38.536 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.527+0000 7f0d56ee2700 1 Processor -- start 2023-10-11T09:52:38.537 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.531+0000 7f0d56ee2700 1 -- start start 2023-10-11T09:52:38.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.531+0000 7f0d56ee2700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0d50160de0 0x7f0d5006c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:38.538 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.531+0000 7f0d56ee2700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0d5006c6d0 0x7f0d500cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:38.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.531+0000 7f0d56ee2700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d500cc660 con 0x7f0d500cc230 2023-10-11T09:52:38.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.531+0000 7f0d56ee2700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d5006cb20 con 0x7f0d5006c6d0 2023-10-11T09:52:38.539 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.531+0000 7f0d56ee2700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d501648a0 con 0x7f0d50160de0 2023-10-11T09:52:38.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.531+0000 7f0d566e1700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f0d500cc230 0x7f0d500cec00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:38020/0 (socket says 172.21.15.78:38020) 2023-10-11T09:52:38.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.531+0000 7f0d566e1700 1 -- 172.21.15.78:0/505686334 learned_addr learned my addr 172.21.15.78:0/505686334 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:38.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.531+0000 7f0d55ee0700 1 --2- 172.21.15.78:0/505686334 >> v2:172.21.15.133:3300/0 conn(0x7f0d50160de0 0x7f0d5006c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:38.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.531+0000 7f0d556df700 1 --2- 172.21.15.78:0/505686334 >> v2:172.21.15.78:3300/0 conn(0x7f0d5006c6d0 0x7f0d500cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:38.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.531+0000 7f0d54ede700 1 -- 172.21.15.78:0/505686334 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 34790751 0 0) 0x7f0d500cc660 con 0x7f0d500cc230 2023-10-11T09:52:38.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.531+0000 7f0d54ede700 1 -- 172.21.15.78:0/505686334 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d3c0035d0 con 0x7f0d500cc230 2023-10-11T09:52:38.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.531+0000 7f0d55ee0700 1 -- 172.21.15.78:0/505686334 >> v2:172.21.15.78:3300/0 conn(0x7f0d5006c6d0 msgr2=0x7f0d500cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:38.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.531+0000 7f0d55ee0700 1 --2- 172.21.15.78:0/505686334 >> v2:172.21.15.78:3300/0 conn(0x7f0d5006c6d0 0x7f0d500cbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:38.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.531+0000 7f0d55ee0700 1 -- 172.21.15.78:0/505686334 >> v1:172.21.15.78:6789/0 conn(0x7f0d500cc230 legacy=0x7f0d500cec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:38.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.531+0000 7f0d55ee0700 1 -- 172.21.15.78:0/505686334 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d50164b80 con 0x7f0d50160de0 2023-10-11T09:52:38.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.535+0000 7f0d55ee0700 1 --2- 172.21.15.78:0/505686334 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d50160de0 0x7f0d5006c190 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f0d50072960 tx=0x7f0d4c009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7acafaccdd052c75 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:38.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.535+0000 7f0d54ede700 1 -- 172.21.15.78:0/505686334 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c011830 con 0x7f0d50160de0 2023-10-11T09:52:38.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.535+0000 7f0d54ede700 1 -- 172.21.15.78:0/505686334 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d4c011990 con 0x7f0d50160de0 2023-10-11T09:52:38.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.535+0000 7f0d54ede700 1 -- 172.21.15.78:0/505686334 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c011d20 con 0x7f0d50160de0 2023-10-11T09:52:38.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.535+0000 7f0d56ee2700 1 -- 172.21.15.78:0/505686334 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d50160de0 msgr2=0x7f0d5006c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:38.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.535+0000 7f0d56ee2700 1 --2- 172.21.15.78:0/505686334 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d50160de0 0x7f0d5006c190 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f0d50072960 tx=0x7f0d4c009b30 comp rx=0 tx=0).stop 2023-10-11T09:52:38.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.535+0000 7f0d56ee2700 1 -- 172.21.15.78:0/505686334 shutdown_connections 2023-10-11T09:52:38.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.535+0000 7f0d56ee2700 1 --2- 172.21.15.78:0/505686334 >> v2:172.21.15.78:3300/0 conn(0x7f0d5006c6d0 0x7f0d500cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:38.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.535+0000 7f0d56ee2700 1 --2- 172.21.15.78:0/505686334 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d50160de0 0x7f0d5006c190 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:38.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.535+0000 7f0d56ee2700 1 -- 172.21.15.78:0/505686334 >> 172.21.15.78:0/505686334 conn(0x7f0d50073100 msgr2=0x7f0d500734d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:38.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.535+0000 7f0d56ee2700 1 -- 172.21.15.78:0/505686334 shutdown_connections 2023-10-11T09:52:38.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.535+0000 7f0d56ee2700 1 -- 172.21.15.78:0/505686334 wait complete. 2023-10-11T09:52:38.546 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.539+0000 7f0d56ee2700 1 Processor -- start 2023-10-11T09:52:38.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.539+0000 7f0d56ee2700 1 -- start start 2023-10-11T09:52:38.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.539+0000 7f0d56ee2700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d5006c6d0 0x7f0d500c51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:38.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.539+0000 7f0d56ee2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d500cc230 0x7f0d500c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:38.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.539+0000 7f0d56ee2700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d50160de0 0x7f0d501f9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:38.547 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.539+0000 7f0d56ee2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d500b4a60 con 0x7f0d500cc230 2023-10-11T09:52:38.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.539+0000 7f0d56ee2700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d500c5730 con 0x7f0d50160de0 2023-10-11T09:52:38.548 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.539+0000 7f0d56ee2700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d500b45d0 con 0x7f0d5006c6d0 2023-10-11T09:52:38.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.539+0000 7f0d556df700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d500cc230 0x7f0d500c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:38.549 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.539+0000 7f0d556df700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d500cc230 0x7f0d500c8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43322/0 (socket says 172.21.15.78:43322) 2023-10-11T09:52:38.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.539+0000 7f0d556df700 1 -- 172.21.15.78:0/3939763048 learned_addr learned my addr 172.21.15.78:0/3939763048 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:38.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.539+0000 7f0d566e1700 1 --2- 172.21.15.78:0/3939763048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d50160de0 0x7f0d501f9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:38.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.539+0000 7f0d55ee0700 1 --2- 172.21.15.78:0/3939763048 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d5006c6d0 0x7f0d500c51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:38.550 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.539+0000 7f0d556df700 1 -- 172.21.15.78:0/3939763048 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d5006c6d0 msgr2=0x7f0d500c51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:38.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.539+0000 7f0d556df700 1 --2- 172.21.15.78:0/3939763048 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d5006c6d0 0x7f0d500c51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:38.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.539+0000 7f0d556df700 1 -- 172.21.15.78:0/3939763048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d50160de0 msgr2=0x7f0d501f9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:38.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.539+0000 7f0d556df700 1 --2- 172.21.15.78:0/3939763048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d50160de0 0x7f0d501f9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:38.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.539+0000 7f0d556df700 1 -- 172.21.15.78:0/3939763048 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d4c0094b0 con 0x7f0d500cc230 2023-10-11T09:52:38.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.543+0000 7f0d556df700 1 --2- 172.21.15.78:0/3939763048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d500cc230 0x7f0d500c8f50 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000ea00 tx=0x7f0d4000ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:38.551 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.543+0000 7f0d467fc700 1 -- 172.21.15.78:0/3939763048 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d40009720 con 0x7f0d500cc230 2023-10-11T09:52:38.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.543+0000 7f0d467fc700 1 -- 172.21.15.78:0/3939763048 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d4000eea0 con 0x7f0d500cc230 2023-10-11T09:52:38.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.543+0000 7f0d467fc700 1 -- 172.21.15.78:0/3939763048 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4000c9f0 con 0x7f0d500cc230 2023-10-11T09:52:38.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.543+0000 7f0d56ee2700 1 -- 172.21.15.78:0/3939763048 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d500b1b40 con 0x7f0d500cc230 2023-10-11T09:52:38.552 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.543+0000 7f0d56ee2700 1 -- 172.21.15.78:0/3939763048 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d500b3f80 con 0x7f0d500cc230 2023-10-11T09:52:38.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.543+0000 7f0d56ee2700 1 -- 172.21.15.78:0/3939763048 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d500744c0 con 0x7f0d500cc230 2023-10-11T09:52:38.553 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.543+0000 7f0d467fc700 1 -- 172.21.15.78:0/3939763048 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f0d4000cb50 con 0x7f0d500cc230 2023-10-11T09:52:38.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.547+0000 7f0d467fc700 1 --2- 172.21.15.78:0/3939763048 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0d34065ed0 0x7f0d34068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:38.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.547+0000 7f0d467fc700 1 -- 172.21.15.78:0/3939763048 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f0d400838a0 con 0x7f0d500cc230 2023-10-11T09:52:38.554 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.547+0000 7f0d55ee0700 1 --2- 172.21.15.78:0/3939763048 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0d34065ed0 0x7f0d34068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:38.556 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.547+0000 7f0d55ee0700 1 --2- 172.21.15.78:0/3939763048 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0d34065ed0 0x7f0d34068380 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c000c00 tx=0x7f0d4c01c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:38.559 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.551+0000 7f0d467fc700 1 -- 172.21.15.78:0/3939763048 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f0d40050da0 con 0x7f0d500cc230 2023-10-11T09:52:38.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.727+0000 7f0d56ee2700 1 -- 172.21.15.78:0/3939763048 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0d500b4710 con 0x7f0d500cc230 2023-10-11T09:52:38.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.727+0000 7f0d467fc700 1 -- 172.21.15.78:0/3939763048 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v129) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0d400543c0 con 0x7f0d500cc230 2023-10-11T09:52:38.734 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2023-10-11T09:52:38.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.727+0000 7f0d56ee2700 1 -- 172.21.15.78:0/3939763048 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0d34065ed0 msgr2=0x7f0d34068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:38.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.727+0000 7f0d56ee2700 1 --2- 172.21.15.78:0/3939763048 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0d34065ed0 0x7f0d34068380 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c000c00 tx=0x7f0d4c01c040 comp rx=0 tx=0).stop 2023-10-11T09:52:38.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.731+0000 7f0d56ee2700 1 -- 172.21.15.78:0/3939763048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d500cc230 msgr2=0x7f0d500c8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:38.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.731+0000 7f0d56ee2700 1 --2- 172.21.15.78:0/3939763048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d500cc230 0x7f0d500c8f50 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000ea00 tx=0x7f0d4000ed10 comp rx=0 tx=0).stop 2023-10-11T09:52:38.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.731+0000 7f0d56ee2700 1 -- 172.21.15.78:0/3939763048 shutdown_connections 2023-10-11T09:52:38.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.731+0000 7f0d56ee2700 1 --2- 172.21.15.78:0/3939763048 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0d34065ed0 0x7f0d34068380 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:38.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.731+0000 7f0d56ee2700 1 --2- 172.21.15.78:0/3939763048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d50160de0 0x7f0d501f9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:38.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.731+0000 7f0d56ee2700 1 --2- 172.21.15.78:0/3939763048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d500cc230 0x7f0d500c8f50 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:38.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.731+0000 7f0d56ee2700 1 --2- 172.21.15.78:0/3939763048 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d5006c6d0 0x7f0d500c51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:38.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.731+0000 7f0d56ee2700 1 -- 172.21.15.78:0/3939763048 >> 172.21.15.78:0/3939763048 conn(0x7f0d50073100 msgr2=0x7f0d500afec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:38.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.731+0000 7f0d56ee2700 1 -- 172.21.15.78:0/3939763048 shutdown_connections 2023-10-11T09:52:38.739 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.731+0000 7f0d56ee2700 1 -- 172.21.15.78:0/3939763048 wait complete. 2023-10-11T09:52:38.758 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T09:52:38.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.923+0000 7f5e8359e700 1 Processor -- start 2023-10-11T09:52:38.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.923+0000 7f5e8359e700 1 -- start start 2023-10-11T09:52:38.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.923+0000 7f5e8359e700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5e840cbcf0 0x7f5e841406b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:38.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.923+0000 7f5e8359e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e8413fe00 0x7f5e840cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:38.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.923+0000 7f5e8359e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f5e840cebe0 0x7f5e840cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:38.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.923+0000 7f5e8359e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e8406bb20 con 0x7f5e840cebe0 2023-10-11T09:52:38.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.923+0000 7f5e8359e700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e84140bf0 con 0x7f5e840cbcf0 2023-10-11T09:52:38.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.923+0000 7f5e8359e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e840cc600 con 0x7f5e8413fe00 2023-10-11T09:52:38.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.923+0000 7f5e82d9d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f5e840cebe0 0x7f5e840cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:38.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.923+0000 7f5e8259c700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5e840cbcf0 0x7f5e841406b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:38.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.927+0000 7f5e81d9b700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e8413fe00 0x7f5e840cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:38.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.927+0000 7f5e81d9b700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e8413fe00 0x7f5e840cc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:35800/0 (socket says 172.21.15.78:35800) 2023-10-11T09:52:38.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.927+0000 7f5e81d9b700 1 -- 172.21.15.78:0/2111911870 learned_addr learned my addr 172.21.15.78:0/2111911870 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:38.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.927+0000 7f5e81d9b700 1 -- 172.21.15.78:0/2111911870 >> v2:172.21.15.133:3300/0 conn(0x7f5e840cbcf0 msgr2=0x7f5e841406b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:38.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.927+0000 7f5e81d9b700 1 --2- 172.21.15.78:0/2111911870 >> v2:172.21.15.133:3300/0 conn(0x7f5e840cbcf0 0x7f5e841406b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:38.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.927+0000 7f5e81d9b700 1 -- 172.21.15.78:0/2111911870 >> v2:172.21.15.78:3300/0 conn(0x7f5e840cebe0 msgr2=0x7f5e840cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:38.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.927+0000 7f5e81d9b700 1 --2- 172.21.15.78:0/2111911870 >> v2:172.21.15.78:3300/0 conn(0x7f5e840cebe0 0x7f5e840cf000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:38.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.927+0000 7f5e81d9b700 1 -- 172.21.15.78:0/2111911870 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e8406bdc0 con 0x7f5e8413fe00 2023-10-11T09:52:38.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.927+0000 7f5e81d9b700 1 --2- 172.21.15.78:0/2111911870 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e8413fe00 0x7f5e840cc0c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5e8406f900 tx=0x7f5e78009bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a6b9a303302c6de server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:38.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.927+0000 7f5e8159a700 1 -- 172.21.15.78:0/2111911870 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e78011830 con 0x7f5e8413fe00 2023-10-11T09:52:38.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.927+0000 7f5e8159a700 1 -- 172.21.15.78:0/2111911870 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e78011990 con 0x7f5e8413fe00 2023-10-11T09:52:38.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.927+0000 7f5e8159a700 1 -- 172.21.15.78:0/2111911870 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e78011d20 con 0x7f5e8413fe00 2023-10-11T09:52:38.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.927+0000 7f5e8359e700 1 -- 172.21.15.78:0/2111911870 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e8413fe00 msgr2=0x7f5e840cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:38.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.927+0000 7f5e8359e700 1 --2- 172.21.15.78:0/2111911870 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e8413fe00 0x7f5e840cc0c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5e8406f900 tx=0x7f5e78009bf0 comp rx=0 tx=0).stop 2023-10-11T09:52:38.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.931+0000 7f5e8359e700 1 -- 172.21.15.78:0/2111911870 shutdown_connections 2023-10-11T09:52:38.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.931+0000 7f5e8359e700 1 --2- 172.21.15.78:0/2111911870 >> v2:172.21.15.78:3300/0 conn(0x7f5e840cebe0 0x7f5e840cf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:38.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.931+0000 7f5e8359e700 1 --2- 172.21.15.78:0/2111911870 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e8413fe00 0x7f5e840cc0c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:38.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.931+0000 7f5e8359e700 1 --2- 172.21.15.78:0/2111911870 >> v2:172.21.15.133:3300/0 conn(0x7f5e840cbcf0 0x7f5e841406b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:38.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.931+0000 7f5e8359e700 1 -- 172.21.15.78:0/2111911870 >> 172.21.15.78:0/2111911870 conn(0x7f5e84073100 msgr2=0x7f5e840734d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:38.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.931+0000 7f5e8359e700 1 -- 172.21.15.78:0/2111911870 shutdown_connections 2023-10-11T09:52:38.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.931+0000 7f5e8359e700 1 -- 172.21.15.78:0/2111911870 wait complete. 2023-10-11T09:52:38.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.931+0000 7f5e8359e700 1 Processor -- start 2023-10-11T09:52:38.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.931+0000 7f5e8359e700 1 -- start start 2023-10-11T09:52:38.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.935+0000 7f5e8359e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e840cbcf0 0x7f5e84150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:38.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.935+0000 7f5e8359e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e840cebe0 0x7f5e8414ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:38.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.935+0000 7f5e8359e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e840b2af0 0x7f5e841efb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:38.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.935+0000 7f5e8359e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8414df00 con 0x7f5e840cebe0 2023-10-11T09:52:38.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.935+0000 7f5e8359e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e841f0410 con 0x7f5e840cbcf0 2023-10-11T09:52:38.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.935+0000 7f5e8359e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e841f06f0 con 0x7f5e840b2af0 2023-10-11T09:52:38.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.935+0000 7f5e81d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e840cebe0 0x7f5e8414ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:38.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.935+0000 7f5e81d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e840cebe0 0x7f5e8414ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43336/0 (socket says 172.21.15.78:43336) 2023-10-11T09:52:38.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.935+0000 7f5e81d9b700 1 -- 172.21.15.78:0/3055465304 learned_addr learned my addr 172.21.15.78:0/3055465304 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:38.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.935+0000 7f5e8259c700 1 --2- 172.21.15.78:0/3055465304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e840cbcf0 0x7f5e84150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:38.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.935+0000 7f5e82d9d700 1 --2- 172.21.15.78:0/3055465304 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e840b2af0 0x7f5e841efb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:38.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.935+0000 7f5e81d9b700 1 -- 172.21.15.78:0/3055465304 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e840b2af0 msgr2=0x7f5e841efb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:38.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.935+0000 7f5e81d9b700 1 --2- 172.21.15.78:0/3055465304 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e840b2af0 0x7f5e841efb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:38.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.935+0000 7f5e81d9b700 1 -- 172.21.15.78:0/3055465304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e840cbcf0 msgr2=0x7f5e84150f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:38.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.935+0000 7f5e81d9b700 1 --2- 172.21.15.78:0/3055465304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e840cbcf0 0x7f5e84150f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:38.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.935+0000 7f5e81d9b700 1 -- 172.21.15.78:0/3055465304 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e78009570 con 0x7f5e840cebe0 2023-10-11T09:52:38.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.935+0000 7f5e8259c700 1 --2- 172.21.15.78:0/3055465304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e840cbcf0 0x7f5e84150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:38.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.935+0000 7f5e81d9b700 1 --2- 172.21.15.78:0/3055465304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e840cebe0 0x7f5e8414ce40 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f5e78000c00 tx=0x7f5e78004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:38.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.935+0000 7f5e72ffd700 1 -- 172.21.15.78:0/3055465304 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e78004590 con 0x7f5e840cebe0 2023-10-11T09:52:38.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.935+0000 7f5e72ffd700 1 -- 172.21.15.78:0/3055465304 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7801e070 con 0x7f5e840cebe0 2023-10-11T09:52:38.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.935+0000 7f5e8359e700 1 -- 172.21.15.78:0/3055465304 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e841f09d0 con 0x7f5e840cebe0 2023-10-11T09:52:38.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.935+0000 7f5e72ffd700 1 -- 172.21.15.78:0/3055465304 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e78003900 con 0x7f5e840cebe0 2023-10-11T09:52:38.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.939+0000 7f5e8359e700 1 -- 172.21.15.78:0/3055465304 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e8414d380 con 0x7f5e840cebe0 2023-10-11T09:52:38.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.939+0000 7f5e8359e700 1 -- 172.21.15.78:0/3055465304 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e840744c0 con 0x7f5e840cebe0 2023-10-11T09:52:38.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.939+0000 7f5e72ffd700 1 -- 172.21.15.78:0/3055465304 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f5e78003a60 con 0x7f5e840cebe0 2023-10-11T09:52:38.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.939+0000 7f5e72ffd700 1 --2- 172.21.15.78:0/3055465304 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5e64065f00 0x7f5e640683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:38.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.939+0000 7f5e72ffd700 1 -- 172.21.15.78:0/3055465304 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f5e78084390 con 0x7f5e840cebe0 2023-10-11T09:52:38.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.943+0000 7f5e8259c700 1 --2- 172.21.15.78:0/3055465304 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5e64065f00 0x7f5e640683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:38.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.943+0000 7f5e8259c700 1 --2- 172.21.15.78:0/3055465304 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5e64065f00 0x7f5e640683b0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f5e74009bc0 tx=0x7f5e74008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:38.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:38.947+0000 7f5e72ffd700 1 -- 172.21.15.78:0/3055465304 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f5e78051a40 con 0x7f5e840cebe0 2023-10-11T09:52:39.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.127+0000 7f5e8359e700 1 -- 172.21.15.78:0/3055465304 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5e841545a0 con 0x7f5e840cebe0 2023-10-11T09:52:39.135 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.127+0000 7f5e72ffd700 1 -- 172.21.15.78:0/3055465304 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v129) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5e78055060 con 0x7f5e840cebe0 2023-10-11T09:52:39.135 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2023-10-11T09:52:39.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.131+0000 7f5e8359e700 1 -- 172.21.15.78:0/3055465304 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5e64065f00 msgr2=0x7f5e640683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:39.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.131+0000 7f5e8359e700 1 --2- 172.21.15.78:0/3055465304 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5e64065f00 0x7f5e640683b0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f5e74009bc0 tx=0x7f5e74008040 comp rx=0 tx=0).stop 2023-10-11T09:52:39.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.131+0000 7f5e8359e700 1 -- 172.21.15.78:0/3055465304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e840cebe0 msgr2=0x7f5e8414ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:39.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.131+0000 7f5e8359e700 1 --2- 172.21.15.78:0/3055465304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e840cebe0 0x7f5e8414ce40 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f5e78000c00 tx=0x7f5e78004770 comp rx=0 tx=0).stop 2023-10-11T09:52:39.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.131+0000 7f5e8359e700 1 -- 172.21.15.78:0/3055465304 shutdown_connections 2023-10-11T09:52:39.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.131+0000 7f5e8359e700 1 --2- 172.21.15.78:0/3055465304 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5e64065f00 0x7f5e640683b0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:39.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.131+0000 7f5e8359e700 1 --2- 172.21.15.78:0/3055465304 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e840b2af0 0x7f5e841efb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:39.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.131+0000 7f5e8359e700 1 --2- 172.21.15.78:0/3055465304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e840cebe0 0x7f5e8414ce40 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:39.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.131+0000 7f5e8359e700 1 --2- 172.21.15.78:0/3055465304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e840cbcf0 0x7f5e84150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:39.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.131+0000 7f5e8359e700 1 -- 172.21.15.78:0/3055465304 >> 172.21.15.78:0/3055465304 conn(0x7f5e84073100 msgr2=0x7f5e840cf000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:39.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.131+0000 7f5e8359e700 1 -- 172.21.15.78:0/3055465304 shutdown_connections 2023-10-11T09:52:39.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.131+0000 7f5e8359e700 1 -- 172.21.15.78:0/3055465304 wait complete. 2023-10-11T09:52:39.160 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:52:39.330 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.323+0000 7f1e66ab0700 1 Processor -- start 2023-10-11T09:52:39.331 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.323+0000 7f1e66ab0700 1 -- start start 2023-10-11T09:52:39.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.327+0000 7f1e66ab0700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1e60161020 0x7f1e600c20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:39.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.327+0000 7f1e66ab0700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e60071720 0x7f1e600cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:39.334 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.327+0000 7f1e66ab0700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e600cb220 con 0x7f1e600ce530 2023-10-11T09:52:39.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.327+0000 7f1e66ab0700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e60071c70 con 0x7f1e60161020 2023-10-11T09:52:39.335 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.327+0000 7f1e66ab0700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e600ce9d0 con 0x7f1e60071720 2023-10-11T09:52:39.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.327+0000 7f1e652ad700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e60071720 0x7f1e600cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:39.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.327+0000 7f1e652ad700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e60071720 0x7f1e600cdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:35810/0 (socket says 172.21.15.78:35810) 2023-10-11T09:52:39.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.327+0000 7f1e652ad700 1 -- 172.21.15.78:0/3653509138 learned_addr learned my addr 172.21.15.78:0/3653509138 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:39.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.327+0000 7f1e65aae700 1 --2- 172.21.15.78:0/3653509138 >> v2:172.21.15.133:3300/0 conn(0x7f1e60161020 0x7f1e600c20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:39.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.327+0000 7f1e652ad700 1 -- 172.21.15.78:0/3653509138 >> v2:172.21.15.133:3300/0 conn(0x7f1e60161020 msgr2=0x7f1e600c20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:39.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.327+0000 7f1e652ad700 1 --2- 172.21.15.78:0/3653509138 >> v2:172.21.15.133:3300/0 conn(0x7f1e60161020 0x7f1e600c20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:39.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.327+0000 7f1e652ad700 1 -- 172.21.15.78:0/3653509138 >> v1:172.21.15.133:6789/0 conn(0x7f1e600ce530 legacy=0x7f1e60164af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:39.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.327+0000 7f1e652ad700 1 -- 172.21.15.78:0/3653509138 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e601613f0 con 0x7f1e60071720 2023-10-11T09:52:39.338 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.327+0000 7f1e652ad700 1 --2- 172.21.15.78:0/3653509138 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e60071720 0x7f1e600cdff0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1e50009910 tx=0x7f1e50009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bfa1d7cf365d842 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:39.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.331+0000 7f1e64aac700 1 -- 172.21.15.78:0/3653509138 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e50005e50 con 0x7f1e60071720 2023-10-11T09:52:39.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.331+0000 7f1e64aac700 1 -- 172.21.15.78:0/3653509138 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e50006030 con 0x7f1e60071720 2023-10-11T09:52:39.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.331+0000 7f1e64aac700 1 -- 172.21.15.78:0/3653509138 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e500063e0 con 0x7f1e60071720 2023-10-11T09:52:39.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.331+0000 7f1e66ab0700 1 -- 172.21.15.78:0/3653509138 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e60071720 msgr2=0x7f1e600cdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:39.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.331+0000 7f1e66ab0700 1 --2- 172.21.15.78:0/3653509138 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e60071720 0x7f1e600cdff0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1e50009910 tx=0x7f1e50009c20 comp rx=0 tx=0).stop 2023-10-11T09:52:39.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.331+0000 7f1e66ab0700 1 -- 172.21.15.78:0/3653509138 shutdown_connections 2023-10-11T09:52:39.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.331+0000 7f1e66ab0700 1 --2- 172.21.15.78:0/3653509138 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e60071720 0x7f1e600cdff0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:39.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.331+0000 7f1e66ab0700 1 --2- 172.21.15.78:0/3653509138 >> v2:172.21.15.133:3300/0 conn(0x7f1e60161020 0x7f1e600c20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:39.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.331+0000 7f1e66ab0700 1 -- 172.21.15.78:0/3653509138 >> 172.21.15.78:0/3653509138 conn(0x7f1e600cf820 msgr2=0x7f1e600cb610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:39.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.331+0000 7f1e66ab0700 1 -- 172.21.15.78:0/3653509138 shutdown_connections 2023-10-11T09:52:39.340 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.331+0000 7f1e66ab0700 1 -- 172.21.15.78:0/3653509138 wait complete. 2023-10-11T09:52:39.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.335+0000 7f1e66ab0700 1 Processor -- start 2023-10-11T09:52:39.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.335+0000 7f1e66ab0700 1 -- start start 2023-10-11T09:52:39.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.335+0000 7f1e66ab0700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e60071720 0x7f1e60162000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:39.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.335+0000 7f1e66ab0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e600ce530 0x7f1e601580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:39.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.335+0000 7f1e66ab0700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e60161020 0x7f1e600b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:39.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.335+0000 7f1e66ab0700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e60162540 con 0x7f1e600ce530 2023-10-11T09:52:39.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.335+0000 7f1e66ab0700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e600b2f50 con 0x7f1e60071720 2023-10-11T09:52:39.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.335+0000 7f1e66ab0700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e600b31f0 con 0x7f1e60161020 2023-10-11T09:52:39.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.335+0000 7f1e652ad700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e600ce530 0x7f1e601580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:39.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.335+0000 7f1e652ad700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e600ce530 0x7f1e601580d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43344/0 (socket says 172.21.15.78:43344) 2023-10-11T09:52:39.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.335+0000 7f1e652ad700 1 -- 172.21.15.78:0/3876068180 learned_addr learned my addr 172.21.15.78:0/3876068180 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:39.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.335+0000 7f1e65aae700 1 --2- 172.21.15.78:0/3876068180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e60071720 0x7f1e60162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:39.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.335+0000 7f1e662af700 1 --2- 172.21.15.78:0/3876068180 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e60161020 0x7f1e600b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:39.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.335+0000 7f1e652ad700 1 -- 172.21.15.78:0/3876068180 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e60161020 msgr2=0x7f1e600b3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:39.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.335+0000 7f1e652ad700 1 --2- 172.21.15.78:0/3876068180 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e60161020 0x7f1e600b3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:39.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.335+0000 7f1e652ad700 1 -- 172.21.15.78:0/3876068180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e60071720 msgr2=0x7f1e60162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:39.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.335+0000 7f1e652ad700 1 --2- 172.21.15.78:0/3876068180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e60071720 0x7f1e60162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:39.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.335+0000 7f1e652ad700 1 -- 172.21.15.78:0/3876068180 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e50009520 con 0x7f1e600ce530 2023-10-11T09:52:39.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.335+0000 7f1e662af700 1 --2- 172.21.15.78:0/3876068180 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e60161020 0x7f1e600b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:39.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.335+0000 7f1e652ad700 1 --2- 172.21.15.78:0/3876068180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e600ce530 0x7f1e601580d0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f1e50012000 tx=0x7f1e50004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:39.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.335+0000 7f1e567fc700 1 -- 172.21.15.78:0/3876068180 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e50006900 con 0x7f1e600ce530 2023-10-11T09:52:39.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.335+0000 7f1e567fc700 1 -- 172.21.15.78:0/3876068180 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e50020070 con 0x7f1e600ce530 2023-10-11T09:52:39.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.339+0000 7f1e66ab0700 1 -- 172.21.15.78:0/3876068180 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e600b34d0 con 0x7f1e600ce530 2023-10-11T09:52:39.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.339+0000 7f1e567fc700 1 -- 172.21.15.78:0/3876068180 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e50025670 con 0x7f1e600ce530 2023-10-11T09:52:39.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.339+0000 7f1e66ab0700 1 -- 172.21.15.78:0/3876068180 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e600b3a90 con 0x7f1e600ce530 2023-10-11T09:52:39.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.339+0000 7f1e66ab0700 1 -- 172.21.15.78:0/3876068180 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e60067d00 con 0x7f1e600ce530 2023-10-11T09:52:39.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.339+0000 7f1e567fc700 1 -- 172.21.15.78:0/3876068180 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f1e50022030 con 0x7f1e600ce530 2023-10-11T09:52:39.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.343+0000 7f1e567fc700 1 --2- 172.21.15.78:0/3876068180 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1e44065ed0 0x7f1e44068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:39.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.343+0000 7f1e567fc700 1 -- 172.21.15.78:0/3876068180 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f1e50084dd0 con 0x7f1e600ce530 2023-10-11T09:52:39.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.343+0000 7f1e65aae700 1 --2- 172.21.15.78:0/3876068180 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1e44065ed0 0x7f1e44068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:39.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.343+0000 7f1e65aae700 1 --2- 172.21.15.78:0/3876068180 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1e44065ed0 0x7f1e44068380 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c009bf0 tx=0x7f1e5c009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:39.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.347+0000 7f1e567fc700 1 -- 172.21.15.78:0/3876068180 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f1e50052360 con 0x7f1e600ce530 2023-10-11T09:52:39.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.555+0000 7f1e66ab0700 1 -- 172.21.15.78:0/3876068180 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1e60065d40 con 0x7f1e600ce530 2023-10-11T09:52:39.562 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.555+0000 7f1e567fc700 1 -- 172.21.15.78:0/3876068180 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4656 (secure 0 0 0) 0x7f1e50055980 con 0x7f1e600ce530 2023-10-11T09:52:39.563 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:52:39.563 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":112,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8452,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/2481257990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2481257990},{"type":"v1","addr":"172.21.15.78:6836","nonce":2481257990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":8464,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1035227282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1035227282},{"type":"v1","addr":"172.21.15.133:6835","nonce":1035227282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":8680,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/36737634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":36737634},{"type":"v1","addr":"172.21.15.133:6833","nonce":36737634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":101}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:51:40.211859+0000","modified":"2023-10-11T09:52:29.509275+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9336},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9336":{"gid":9336,"name":"c","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.78:6837/3884456900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":3884456900},{"type":"v1","addr":"172.21.15.78:6837","nonce":3884456900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-11T09:52:39.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.559+0000 7f1e66ab0700 1 -- 172.21.15.78:0/3876068180 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1e44065ed0 msgr2=0x7f1e44068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:39.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.559+0000 7f1e66ab0700 1 --2- 172.21.15.78:0/3876068180 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1e44065ed0 0x7f1e44068380 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c009bf0 tx=0x7f1e5c009240 comp rx=0 tx=0).stop 2023-10-11T09:52:39.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.559+0000 7f1e66ab0700 1 -- 172.21.15.78:0/3876068180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e600ce530 msgr2=0x7f1e601580d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:39.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.559+0000 7f1e66ab0700 1 --2- 172.21.15.78:0/3876068180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e600ce530 0x7f1e601580d0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f1e50012000 tx=0x7f1e50004770 comp rx=0 tx=0).stop 2023-10-11T09:52:39.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.559+0000 7f1e66ab0700 1 -- 172.21.15.78:0/3876068180 shutdown_connections 2023-10-11T09:52:39.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.559+0000 7f1e66ab0700 1 --2- 172.21.15.78:0/3876068180 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f1e44065ed0 0x7f1e44068380 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:39.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.559+0000 7f1e66ab0700 1 --2- 172.21.15.78:0/3876068180 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e60161020 0x7f1e600b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:39.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.559+0000 7f1e66ab0700 1 --2- 172.21.15.78:0/3876068180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e600ce530 0x7f1e601580d0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:39.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.559+0000 7f1e66ab0700 1 --2- 172.21.15.78:0/3876068180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e60071720 0x7f1e60162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:39.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.559+0000 7f1e66ab0700 1 -- 172.21.15.78:0/3876068180 >> 172.21.15.78:0/3876068180 conn(0x7f1e600cf820 msgr2=0x7f1e600cc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:39.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.559+0000 7f1e66ab0700 1 -- 172.21.15.78:0/3876068180 shutdown_connections 2023-10-11T09:52:39.568 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.559+0000 7f1e66ab0700 1 -- 172.21.15.78:0/3876068180 wait complete. 2023-10-11T09:52:39.569 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 112 2023-10-11T09:52:39.586 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:52:39.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.743+0000 7fc004a17700 1 Processor -- start 2023-10-11T09:52:39.751 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.743+0000 7fc004a17700 1 -- start start 2023-10-11T09:52:39.754 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.747+0000 7fc004a17700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc0000cfbb0 0x7fc0000cff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:39.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.747+0000 7fc004a17700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fc00015cc20 0x7fc0000d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:39.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.747+0000 7fc004a17700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0000d04c0 con 0x7fc0000cd7f0 2023-10-11T09:52:39.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.747+0000 7fc004a17700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0000d1820 con 0x7fc00015cc20 2023-10-11T09:52:39.755 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.747+0000 7fc004a17700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0000d1b00 con 0x7fc0000cfbb0 2023-10-11T09:52:39.756 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.747+0000 7fbffdd9b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fc00015cc20 0x7fc0000d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:39.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.747+0000 7fbffdd9b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fc00015cc20 0x7fc0000d28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43360/0 (socket says 172.21.15.78:43360) 2023-10-11T09:52:39.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.747+0000 7fbffdd9b700 1 -- 172.21.15.78:0/2527469362 learned_addr learned my addr 172.21.15.78:0/2527469362 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:39.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.747+0000 7fbffe59c700 1 --2- 172.21.15.78:0/2527469362 >> v2:172.21.15.133:3300/0 conn(0x7fc0000cfbb0 0x7fc0000cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:39.757 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.747+0000 7fbffdd9b700 1 -- 172.21.15.78:0/2527469362 >> v2:172.21.15.133:3300/0 conn(0x7fc0000cfbb0 msgr2=0x7fc0000cff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:39.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.747+0000 7fbffdd9b700 1 --2- 172.21.15.78:0/2527469362 >> v2:172.21.15.133:3300/0 conn(0x7fc0000cfbb0 0x7fc0000cff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:39.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.747+0000 7fbffdd9b700 1 -- 172.21.15.78:0/2527469362 >> v1:172.21.15.133:6789/0 conn(0x7fc0000cd7f0 legacy=0x7fc0000b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:39.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.747+0000 7fbffdd9b700 1 -- 172.21.15.78:0/2527469362 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0000d1de0 con 0x7fc00015cc20 2023-10-11T09:52:39.758 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.751+0000 7fbffdd9b700 1 --2- 172.21.15.78:0/2527469362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc00015cc20 0x7fc0000d28c0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fc00008c080 tx=0x7fbff4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=205f0f708061ac86 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:39.759 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.751+0000 7fbffd59a700 1 -- 172.21.15.78:0/2527469362 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff4011830 con 0x7fc00015cc20 2023-10-11T09:52:39.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.751+0000 7fbffd59a700 1 -- 172.21.15.78:0/2527469362 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff4011990 con 0x7fc00015cc20 2023-10-11T09:52:39.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.751+0000 7fbffd59a700 1 -- 172.21.15.78:0/2527469362 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff4011d20 con 0x7fc00015cc20 2023-10-11T09:52:39.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.751+0000 7fc004a17700 1 -- 172.21.15.78:0/2527469362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc00015cc20 msgr2=0x7fc0000d28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:39.760 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.751+0000 7fc004a17700 1 --2- 172.21.15.78:0/2527469362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc00015cc20 0x7fc0000d28c0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fc00008c080 tx=0x7fbff4009bf0 comp rx=0 tx=0).stop 2023-10-11T09:52:39.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.751+0000 7fc004a17700 1 -- 172.21.15.78:0/2527469362 shutdown_connections 2023-10-11T09:52:39.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.751+0000 7fc004a17700 1 --2- 172.21.15.78:0/2527469362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc00015cc20 0x7fc0000d28c0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:39.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.751+0000 7fc004a17700 1 --2- 172.21.15.78:0/2527469362 >> v2:172.21.15.133:3300/0 conn(0x7fc0000cfbb0 0x7fc0000cff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:39.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.751+0000 7fc004a17700 1 -- 172.21.15.78:0/2527469362 >> 172.21.15.78:0/2527469362 conn(0x7fc00006abf0 msgr2=0x7fc00006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:39.761 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.751+0000 7fc004a17700 1 -- 172.21.15.78:0/2527469362 shutdown_connections 2023-10-11T09:52:39.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.751+0000 7fc004a17700 1 -- 172.21.15.78:0/2527469362 wait complete. 2023-10-11T09:52:39.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.755+0000 7fc004a17700 1 Processor -- start 2023-10-11T09:52:39.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.755+0000 7fc004a17700 1 -- start start 2023-10-11T09:52:39.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.755+0000 7fc004a17700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc0000cd7f0 0x7fc0001f1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:39.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.755+0000 7fc004a17700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0000cfbb0 0x7fc0000b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:39.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.755+0000 7fc004a17700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc00015cc20 0x7fc000154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:39.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.755+0000 7fc004a17700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0000cbdf0 con 0x7fc0000cfbb0 2023-10-11T09:52:39.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.755+0000 7fc004a17700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0000b36e0 con 0x7fc00015cc20 2023-10-11T09:52:39.764 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.755+0000 7fc004a17700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0000cb830 con 0x7fc0000cd7f0 2023-10-11T09:52:39.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.755+0000 7fbffdd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0000cfbb0 0x7fc0000b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:39.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.755+0000 7fbffdd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0000cfbb0 0x7fc0000b31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43364/0 (socket says 172.21.15.78:43364) 2023-10-11T09:52:39.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.755+0000 7fbffdd9b700 1 -- 172.21.15.78:0/597011706 learned_addr learned my addr 172.21.15.78:0/597011706 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:39.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.755+0000 7fbffe59c700 1 --2- 172.21.15.78:0/597011706 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc0000cd7f0 0x7fc0001f1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:39.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.755+0000 7fbffed9d700 1 --2- 172.21.15.78:0/597011706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc00015cc20 0x7fc000154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:39.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.755+0000 7fbffe59c700 1 -- 172.21.15.78:0/597011706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc00015cc20 msgr2=0x7fc000154170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:39.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.755+0000 7fbffe59c700 1 --2- 172.21.15.78:0/597011706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc00015cc20 0x7fc000154170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:39.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.755+0000 7fbffe59c700 1 -- 172.21.15.78:0/597011706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0000cfbb0 msgr2=0x7fc0000b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:39.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.755+0000 7fbffe59c700 1 --2- 172.21.15.78:0/597011706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0000cfbb0 0x7fc0000b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:39.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.755+0000 7fbffe59c700 1 -- 172.21.15.78:0/597011706 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff4009570 con 0x7fc0000cd7f0 2023-10-11T09:52:39.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.755+0000 7fbffdd9b700 1 --2- 172.21.15.78:0/597011706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0000cfbb0 0x7fc0000b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:52:39.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.755+0000 7fbffe59c700 1 --2- 172.21.15.78:0/597011706 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc0000cd7f0 0x7fc0001f1b00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbff000e900 tx=0x7fbff000ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:39.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.759+0000 7fbfeeffd700 1 -- 172.21.15.78:0/597011706 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0013680 con 0x7fc0000cd7f0 2023-10-11T09:52:39.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.759+0000 7fbfeeffd700 1 -- 172.21.15.78:0/597011706 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff00137e0 con 0x7fc0000cd7f0 2023-10-11T09:52:39.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.759+0000 7fbfeeffd700 1 -- 172.21.15.78:0/597011706 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0013b50 con 0x7fc0000cd7f0 2023-10-11T09:52:39.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.759+0000 7fc004a17700 1 -- 172.21.15.78:0/597011706 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0000cbb10 con 0x7fc0000cd7f0 2023-10-11T09:52:39.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.759+0000 7fc004a17700 1 -- 172.21.15.78:0/597011706 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0000b41f0 con 0x7fc0000cd7f0 2023-10-11T09:52:39.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.759+0000 7fc004a17700 1 -- 172.21.15.78:0/597011706 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc000067d00 con 0x7fc0000cd7f0 2023-10-11T09:52:39.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.763+0000 7fbfeeffd700 1 -- 172.21.15.78:0/597011706 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fbff0013cb0 con 0x7fc0000cd7f0 2023-10-11T09:52:39.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.763+0000 7fbfeeffd700 1 --2- 172.21.15.78:0/597011706 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbfe0065ed0 0x7fbfe0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:39.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.763+0000 7fbfeeffd700 1 -- 172.21.15.78:0/597011706 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6234+0+0 (secure 0 0 0) 0x7fbff0083b30 con 0x7fc0000cd7f0 2023-10-11T09:52:39.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.763+0000 7fbffdd9b700 1 --2- 172.21.15.78:0/597011706 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbfe0065ed0 0x7fbfe0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:39.774 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.763+0000 7fbffdd9b700 1 --2- 172.21.15.78:0/597011706 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbfe0065ed0 0x7fbfe0068380 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fc0001594f0 tx=0x7fbff4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:39.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.771+0000 7fbfeeffd700 1 -- 172.21.15.78:0/597011706 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fbff00510c0 con 0x7fc0000cd7f0 2023-10-11T09:52:39.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.947+0000 7fc004a17700 1 -- 172.21.15.78:0/597011706 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc0000b3d70 con 0x7fc0000cd7f0 2023-10-11T09:52:39.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.951+0000 7fbfeeffd700 1 -- 172.21.15.78:0/597011706 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+18154 (secure 0 0 0) 0x7fc0000b3d70 con 0x7fc0000cd7f0 2023-10-11T09:52:39.958 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:52:39.958 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":129,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:52:22.836530+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:51:38.592892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-11T09:51:39.204588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6837/4189411531":"2023-10-12T09:52:22.836499+0000","172.21.15.78:6835/4189411531":"2023-10-12T09:52:22.836499+0000","172.21.15.133:6833/1433079498":"2023-10-12T09:52:02.877755+0000","172.21.15.133:6832/1433079498":"2023-10-12T09:52:02.877755+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:52:39.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.955+0000 7fc004a17700 1 -- 172.21.15.78:0/597011706 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbfe0065ed0 msgr2=0x7fbfe0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:39.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.955+0000 7fc004a17700 1 --2- 172.21.15.78:0/597011706 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbfe0065ed0 0x7fbfe0068380 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fc0001594f0 tx=0x7fbff4008030 comp rx=0 tx=0).stop 2023-10-11T09:52:39.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.955+0000 7fc004a17700 1 -- 172.21.15.78:0/597011706 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc0000cd7f0 msgr2=0x7fc0001f1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:39.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.955+0000 7fc004a17700 1 --2- 172.21.15.78:0/597011706 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc0000cd7f0 0x7fc0001f1b00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbff000e900 tx=0x7fbff000ec10 comp rx=0 tx=0).stop 2023-10-11T09:52:39.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.955+0000 7fc004a17700 1 -- 172.21.15.78:0/597011706 shutdown_connections 2023-10-11T09:52:39.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.955+0000 7fc004a17700 1 --2- 172.21.15.78:0/597011706 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbfe0065ed0 0x7fbfe0068380 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:39.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.955+0000 7fc004a17700 1 --2- 172.21.15.78:0/597011706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc00015cc20 0x7fc000154170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:39.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.955+0000 7fc004a17700 1 --2- 172.21.15.78:0/597011706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0000cfbb0 0x7fc0000b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:39.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.955+0000 7fc004a17700 1 --2- 172.21.15.78:0/597011706 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc0000cd7f0 0x7fc0001f1b00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:39.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.955+0000 7fc004a17700 1 -- 172.21.15.78:0/597011706 >> 172.21.15.78:0/597011706 conn(0x7fc00006abf0 msgr2=0x7fc0001581d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:39.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.955+0000 7fc004a17700 1 -- 172.21.15.78:0/597011706 shutdown_connections 2023-10-11T09:52:39.965 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:39.955+0000 7fc004a17700 1 -- 172.21.15.78:0/597011706 wait complete. 2023-10-11T09:52:39.985 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-11T09:52:39.986 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:52:40.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.151+0000 7f77a95d2700 1 Processor -- start 2023-10-11T09:52:40.160 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.151+0000 7f77a95d2700 1 -- start start 2023-10-11T09:52:40.161 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.155+0000 7f77a95d2700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f77a415a230 0x7f77a40d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:40.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.155+0000 7f77a95d2700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77a415dc40 con 0x7f77a415d760 2023-10-11T09:52:40.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.155+0000 7f77a95d2700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77a40cf3b0 con 0x7f77a40cffa0 2023-10-11T09:52:40.162 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.155+0000 7f77a95d2700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77a40cf1a0 con 0x7f77a415a230 2023-10-11T09:52:40.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.155+0000 7f77a27fc700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f77a415d760 0x7f77a40cf890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:38024/0 (socket says 172.21.15.78:38024) 2023-10-11T09:52:40.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.155+0000 7f77a27fc700 1 -- 172.21.15.78:0/1277150575 learned_addr learned my addr 172.21.15.78:0/1277150575 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:40.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.155+0000 7f77a2ffd700 1 --2- 172.21.15.78:0/1277150575 >> v2:172.21.15.78:3300/0 conn(0x7f77a415a230 0x7f77a40d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:40.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.155+0000 7f77a1ffb700 1 -- 172.21.15.78:0/1277150575 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2256299082 0 0) 0x7f77a415dc40 con 0x7f77a415d760 2023-10-11T09:52:40.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.155+0000 7f77a1ffb700 1 -- 172.21.15.78:0/1277150575 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f778c0035d0 con 0x7f77a415d760 2023-10-11T09:52:40.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.155+0000 7f77a2ffd700 1 -- 172.21.15.78:0/1277150575 >> v1:172.21.15.133:6789/0 conn(0x7f77a40cffa0 legacy=0x7f77a40b7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:40.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.155+0000 7f77a1ffb700 1 -- 172.21.15.78:0/1277150575 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2029990759 0 0) 0x7f77a40cf3b0 con 0x7f77a40cffa0 2023-10-11T09:52:40.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.155+0000 7f77a2ffd700 1 -- 172.21.15.78:0/1277150575 >> v1:172.21.15.78:6789/0 conn(0x7f77a415d760 legacy=0x7f77a40cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:40.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.155+0000 7f77a2ffd700 1 -- 172.21.15.78:0/1277150575 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a40cd6d0 con 0x7f77a415a230 2023-10-11T09:52:40.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.155+0000 7f77a2ffd700 1 --2- 172.21.15.78:0/1277150575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77a415a230 0x7f77a40d0b20 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f7794009850 tx=0x7f7794009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7040958e7f34e214 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:40.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.155+0000 7f77a1ffb700 1 -- 172.21.15.78:0/1277150575 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7798001670 con 0x7f77a415a230 2023-10-11T09:52:40.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.155+0000 7f77a1ffb700 1 -- 172.21.15.78:0/1277150575 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7794010590 con 0x7f77a415a230 2023-10-11T09:52:40.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.155+0000 7f77a1ffb700 1 -- 172.21.15.78:0/1277150575 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7794010900 con 0x7f77a415a230 2023-10-11T09:52:40.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.159+0000 7f77a95d2700 1 -- 172.21.15.78:0/1277150575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77a415a230 msgr2=0x7f77a40d0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:40.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.159+0000 7f77a95d2700 1 --2- 172.21.15.78:0/1277150575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77a415a230 0x7f77a40d0b20 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f7794009850 tx=0x7f7794009b60 comp rx=0 tx=0).stop 2023-10-11T09:52:40.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.159+0000 7f77a95d2700 1 -- 172.21.15.78:0/1277150575 shutdown_connections 2023-10-11T09:52:40.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.159+0000 7f77a95d2700 1 --2- 172.21.15.78:0/1277150575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77a415a230 0x7f77a40d0b20 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:40.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.159+0000 7f77a95d2700 1 -- 172.21.15.78:0/1277150575 >> 172.21.15.78:0/1277150575 conn(0x7f77a406aa20 msgr2=0x7f77a406adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:40.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.159+0000 7f77a95d2700 1 -- 172.21.15.78:0/1277150575 shutdown_connections 2023-10-11T09:52:40.168 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.159+0000 7f77a95d2700 1 -- 172.21.15.78:0/1277150575 wait complete. 2023-10-11T09:52:40.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.163+0000 7f77a95d2700 1 Processor -- start 2023-10-11T09:52:40.169 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.163+0000 7f77a95d2700 1 -- start start 2023-10-11T09:52:40.170 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.163+0000 7f77a95d2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77a40cffa0 0x7f77a41f11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:40.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.163+0000 7f77a95d2700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77a415a230 0x7f77a41f16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:40.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.163+0000 7f77a95d2700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77a415d760 0x7f77a40c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:40.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.163+0000 7f77a95d2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a40b8c60 con 0x7f77a40cffa0 2023-10-11T09:52:40.171 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.163+0000 7f77a95d2700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a40cc9e0 con 0x7f77a415a230 2023-10-11T09:52:40.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.163+0000 7f77a95d2700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a40cccc0 con 0x7f77a415d760 2023-10-11T09:52:40.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.163+0000 7f77a2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77a40cffa0 0x7f77a41f11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:40.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.163+0000 7f77a2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77a40cffa0 0x7f77a41f11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43374/0 (socket says 172.21.15.78:43374) 2023-10-11T09:52:40.172 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.163+0000 7f77a2ffd700 1 -- 172.21.15.78:0/3060614520 learned_addr learned my addr 172.21.15.78:0/3060614520 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:40.173 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.163+0000 7f77a27fc700 1 --2- 172.21.15.78:0/3060614520 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77a415a230 0x7f77a41f16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:40.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.163+0000 7f77a37fe700 1 --2- 172.21.15.78:0/3060614520 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77a415d760 0x7f77a40c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:40.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.163+0000 7f77a2ffd700 1 -- 172.21.15.78:0/3060614520 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77a415d760 msgr2=0x7f77a40c8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:40.174 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.163+0000 7f77a2ffd700 1 --2- 172.21.15.78:0/3060614520 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77a415d760 0x7f77a40c8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:40.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.163+0000 7f77a2ffd700 1 -- 172.21.15.78:0/3060614520 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77a415a230 msgr2=0x7f77a41f16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:40.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.163+0000 7f77a2ffd700 1 --2- 172.21.15.78:0/3060614520 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77a415a230 0x7f77a41f16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:40.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.163+0000 7f77a2ffd700 1 -- 172.21.15.78:0/3060614520 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77940094b0 con 0x7f77a40cffa0 2023-10-11T09:52:40.175 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.163+0000 7f77a27fc700 1 --2- 172.21.15.78:0/3060614520 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77a415a230 0x7f77a41f16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:40.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.163+0000 7f77a2ffd700 1 --2- 172.21.15.78:0/3060614520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77a40cffa0 0x7f77a41f11a0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f7794013040 tx=0x7f7794004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:40.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.167+0000 7f778b7fe700 1 -- 172.21.15.78:0/3060614520 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7794004230 con 0x7f77a40cffa0 2023-10-11T09:52:40.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.167+0000 7f778b7fe700 1 -- 172.21.15.78:0/3060614520 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f779401e070 con 0x7f77a40cffa0 2023-10-11T09:52:40.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.167+0000 7f778b7fe700 1 -- 172.21.15.78:0/3060614520 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77940045d0 con 0x7f77a40cffa0 2023-10-11T09:52:40.176 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.167+0000 7f77a95d2700 1 -- 172.21.15.78:0/3060614520 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77a40b88f0 con 0x7f77a40cffa0 2023-10-11T09:52:40.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.167+0000 7f77a95d2700 1 -- 172.21.15.78:0/3060614520 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a41f1e00 con 0x7f77a40cffa0 2023-10-11T09:52:40.177 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.167+0000 7f77a95d2700 1 -- 172.21.15.78:0/3060614520 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77a4065d40 con 0x7f77a40cffa0 2023-10-11T09:52:40.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.167+0000 7f778b7fe700 1 -- 172.21.15.78:0/3060614520 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f7794011870 con 0x7f77a40cffa0 2023-10-11T09:52:40.178 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.171+0000 7f778b7fe700 1 --2- 172.21.15.78:0/3060614520 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7790065f00 0x7f77900683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:40.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.171+0000 7f778b7fe700 1 -- 172.21.15.78:0/3060614520 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f7794084800 con 0x7f77a40cffa0 2023-10-11T09:52:40.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.171+0000 7f77a27fc700 1 --2- 172.21.15.78:0/3060614520 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7790065f00 0x7f77900683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:40.179 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.171+0000 7f77a27fc700 1 --2- 172.21.15.78:0/3060614520 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7790065f00 0x7f77900683b0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f77a41f5400 tx=0x7f7798009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:40.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.175+0000 7f778b7fe700 1 -- 172.21.15.78:0/3060614520 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f7794051d90 con 0x7f77a40cffa0 2023-10-11T09:52:40.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.383+0000 7f77a95d2700 1 -- 172.21.15.78:0/3060614520 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77a41f1e00 con 0x7f77a40cffa0 2023-10-11T09:52:40.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.387+0000 7f778b7fe700 1 -- 172.21.15.78:0/3060614520 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4656 (secure 0 0 0) 0x7f77a41f1e00 con 0x7f77a40cffa0 2023-10-11T09:52:40.394 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:52:40.394 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":112,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8452,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/2481257990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2481257990},{"type":"v1","addr":"172.21.15.78:6836","nonce":2481257990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":8464,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1035227282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1035227282},{"type":"v1","addr":"172.21.15.133:6835","nonce":1035227282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":8680,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/36737634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":36737634},{"type":"v1","addr":"172.21.15.133:6833","nonce":36737634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":101}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:51:40.211859+0000","modified":"2023-10-11T09:52:29.509275+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9336},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9336":{"gid":9336,"name":"c","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.78:6837/3884456900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":3884456900},{"type":"v1","addr":"172.21.15.78:6837","nonce":3884456900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-11T09:52:40.398 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.391+0000 7f77a95d2700 1 -- 172.21.15.78:0/3060614520 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7790065f00 msgr2=0x7f77900683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:40.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.391+0000 7f77a95d2700 1 --2- 172.21.15.78:0/3060614520 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7790065f00 0x7f77900683b0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f77a41f5400 tx=0x7f7798009040 comp rx=0 tx=0).stop 2023-10-11T09:52:40.399 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.391+0000 7f77a95d2700 1 -- 172.21.15.78:0/3060614520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77a40cffa0 msgr2=0x7f77a41f11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:40.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.391+0000 7f77a95d2700 1 --2- 172.21.15.78:0/3060614520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77a40cffa0 0x7f77a41f11a0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f7794013040 tx=0x7f7794004770 comp rx=0 tx=0).stop 2023-10-11T09:52:40.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.391+0000 7f77a95d2700 1 -- 172.21.15.78:0/3060614520 shutdown_connections 2023-10-11T09:52:40.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.391+0000 7f77a95d2700 1 --2- 172.21.15.78:0/3060614520 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7790065f00 0x7f77900683b0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:40.400 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.391+0000 7f77a95d2700 1 --2- 172.21.15.78:0/3060614520 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77a415d760 0x7f77a40c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:40.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.391+0000 7f77a95d2700 1 --2- 172.21.15.78:0/3060614520 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77a415a230 0x7f77a41f16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:40.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.391+0000 7f77a95d2700 1 --2- 172.21.15.78:0/3060614520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77a40cffa0 0x7f77a41f11a0 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:40.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.391+0000 7f77a95d2700 1 -- 172.21.15.78:0/3060614520 >> 172.21.15.78:0/3060614520 conn(0x7f77a406aa20 msgr2=0x7f77a415c740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:40.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.391+0000 7f77a95d2700 1 -- 172.21.15.78:0/3060614520 shutdown_connections 2023-10-11T09:52:40.401 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.391+0000 7f77a95d2700 1 -- 172.21.15.78:0/3060614520 wait complete. 2023-10-11T09:52:40.402 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 112 2023-10-11T09:52:40.420 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:52:40.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.587+0000 7f427a637700 1 Processor -- start 2023-10-11T09:52:40.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.587+0000 7f427a637700 1 -- start start 2023-10-11T09:52:40.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.587+0000 7f427a637700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4274158f90 0x7f42740cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:40.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.587+0000 7f427a637700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4274070470 0x7f4274071bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:40.596 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.587+0000 7f427a637700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42740cac90 con 0x7f42740c9f70 2023-10-11T09:52:40.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.587+0000 7f427a637700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42740ca3f0 con 0x7f4274158f90 2023-10-11T09:52:40.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.587+0000 7f427a637700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42740ca6d0 con 0x7f4274070470 2023-10-11T09:52:40.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.591+0000 7f4278bd4700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4274070470 0x7f4274071bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:40.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.591+0000 7f4278bd4700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4274070470 0x7f4274071bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:35860/0 (socket says 172.21.15.78:35860) 2023-10-11T09:52:40.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.591+0000 7f4278bd4700 1 -- 172.21.15.78:0/991376494 learned_addr learned my addr 172.21.15.78:0/991376494 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:40.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.591+0000 7f426b7fe700 1 --1- 172.21.15.78:0/991376494 >> v1:172.21.15.133:6789/0 conn(0x7f42740c9f70 0x7f427406fd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:37690/0 (socket says 172.21.15.78:37690) 2023-10-11T09:52:40.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.591+0000 7f4273fff700 1 --2- 172.21.15.78:0/991376494 >> v2:172.21.15.133:3300/0 conn(0x7f4274158f90 0x7f42740cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:40.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.591+0000 7f4278bd4700 1 -- 172.21.15.78:0/991376494 >> v2:172.21.15.133:3300/0 conn(0x7f4274158f90 msgr2=0x7f42740cb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:40.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.591+0000 7f4278bd4700 1 --2- 172.21.15.78:0/991376494 >> v2:172.21.15.133:3300/0 conn(0x7f4274158f90 0x7f42740cb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:40.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.591+0000 7f4278bd4700 1 -- 172.21.15.78:0/991376494 >> v1:172.21.15.133:6789/0 conn(0x7f42740c9f70 legacy=0x7f427406fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:40.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.591+0000 7f4278bd4700 1 -- 172.21.15.78:0/991376494 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42740ca9b0 con 0x7f4274070470 2023-10-11T09:52:40.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.591+0000 7f4273fff700 1 --2- 172.21.15.78:0/991376494 >> v2:172.21.15.133:3300/0 conn(0x7f4274158f90 0x7f42740cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:52:40.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.591+0000 7f4278bd4700 1 --2- 172.21.15.78:0/991376494 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4274070470 0x7f4274071bd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f426c009e50 tx=0x7f426c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a06c5867d0a4f52 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:40.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.591+0000 7f42737fe700 1 -- 172.21.15.78:0/991376494 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c00e070 con 0x7f4274070470 2023-10-11T09:52:40.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.591+0000 7f42737fe700 1 -- 172.21.15.78:0/991376494 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f426c00b910 con 0x7f4274070470 2023-10-11T09:52:40.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.591+0000 7f42737fe700 1 -- 172.21.15.78:0/991376494 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c00bcc0 con 0x7f4274070470 2023-10-11T09:52:40.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.591+0000 7f427a637700 1 -- 172.21.15.78:0/991376494 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4274070470 msgr2=0x7f4274071bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:40.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.591+0000 7f427a637700 1 --2- 172.21.15.78:0/991376494 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4274070470 0x7f4274071bd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f426c009e50 tx=0x7f426c00b330 comp rx=0 tx=0).stop 2023-10-11T09:52:40.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.591+0000 7f427a637700 1 -- 172.21.15.78:0/991376494 shutdown_connections 2023-10-11T09:52:40.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.591+0000 7f427a637700 1 --2- 172.21.15.78:0/991376494 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4274070470 0x7f4274071bd0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:40.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.591+0000 7f427a637700 1 --2- 172.21.15.78:0/991376494 >> v2:172.21.15.133:3300/0 conn(0x7f4274158f90 0x7f42740cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:40.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.591+0000 7f427a637700 1 -- 172.21.15.78:0/991376494 >> 172.21.15.78:0/991376494 conn(0x7f42740cb920 msgr2=0x7f42740cbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:40.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.591+0000 7f427a637700 1 -- 172.21.15.78:0/991376494 shutdown_connections 2023-10-11T09:52:40.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.591+0000 7f427a637700 1 -- 172.21.15.78:0/991376494 wait complete. 2023-10-11T09:52:40.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.595+0000 7f427a637700 1 Processor -- start 2023-10-11T09:52:40.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.595+0000 7f427a637700 1 -- start start 2023-10-11T09:52:40.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.599+0000 7f427a637700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4274070470 0x7f42740c1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:40.605 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.599+0000 7f427a637700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4274158f90 0x7f427406d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:40.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.599+0000 7f427a637700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f427406db20 0x7f42740c3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:40.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.599+0000 7f427a637700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42740c5640 con 0x7f4274158f90 2023-10-11T09:52:40.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.599+0000 7f427a637700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427406dfa0 con 0x7f4274070470 2023-10-11T09:52:40.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.599+0000 7f427a637700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42740c22f0 con 0x7f427406db20 2023-10-11T09:52:40.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.599+0000 7f426b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4274158f90 0x7f427406d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:40.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.599+0000 7f426b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4274158f90 0x7f427406d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43380/0 (socket says 172.21.15.78:43380) 2023-10-11T09:52:40.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.599+0000 7f426b7fe700 1 -- 172.21.15.78:0/2883175553 learned_addr learned my addr 172.21.15.78:0/2883175553 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:40.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.599+0000 7f4273fff700 1 --2- 172.21.15.78:0/2883175553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4274070470 0x7f42740c1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:40.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.599+0000 7f4278bd4700 1 --2- 172.21.15.78:0/2883175553 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f427406db20 0x7f42740c3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:40.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.599+0000 7f426b7fe700 1 -- 172.21.15.78:0/2883175553 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f427406db20 msgr2=0x7f42740c3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:40.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.599+0000 7f426b7fe700 1 --2- 172.21.15.78:0/2883175553 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f427406db20 0x7f42740c3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:40.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.599+0000 7f426b7fe700 1 -- 172.21.15.78:0/2883175553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4274070470 msgr2=0x7f42740c1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:40.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.599+0000 7f426b7fe700 1 --2- 172.21.15.78:0/2883175553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4274070470 0x7f42740c1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:40.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.599+0000 7f426b7fe700 1 -- 172.21.15.78:0/2883175553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f426400b750 con 0x7f4274158f90 2023-10-11T09:52:40.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.599+0000 7f426b7fe700 1 --2- 172.21.15.78:0/2883175553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4274158f90 0x7f427406d5e0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f425c00c730 tx=0x7f425c00ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:40.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.599+0000 7f42717fa700 1 -- 172.21.15.78:0/2883175553 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c005cd0 con 0x7f4274158f90 2023-10-11T09:52:40.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.599+0000 7f42717fa700 1 -- 172.21.15.78:0/2883175553 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f425c005e30 con 0x7f4274158f90 2023-10-11T09:52:40.610 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.599+0000 7f42717fa700 1 -- 172.21.15.78:0/2883175553 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c0072d0 con 0x7f4274158f90 2023-10-11T09:52:40.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.599+0000 7f427a637700 1 -- 172.21.15.78:0/2883175553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f426c009b30 con 0x7f4274158f90 2023-10-11T09:52:40.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.599+0000 7f427a637700 1 -- 172.21.15.78:0/2883175553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42740c4d10 con 0x7f4274158f90 2023-10-11T09:52:40.611 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.603+0000 7f427a637700 1 -- 172.21.15.78:0/2883175553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42741593f0 con 0x7f4274158f90 2023-10-11T09:52:40.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.603+0000 7f42717fa700 1 -- 172.21.15.78:0/2883175553 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f425c007430 con 0x7f4274158f90 2023-10-11T09:52:40.613 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.607+0000 7f42717fa700 1 --2- 172.21.15.78:0/2883175553 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4254065f00 0x7f42540683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:40.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.607+0000 7f42717fa700 1 -- 172.21.15.78:0/2883175553 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f425c082b20 con 0x7f4274158f90 2023-10-11T09:52:40.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.607+0000 7f4273fff700 1 --2- 172.21.15.78:0/2883175553 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4254065f00 0x7f42540683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:40.615 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.607+0000 7f4273fff700 1 --2- 172.21.15.78:0/2883175553 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4254065f00 0x7f42540683b0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f426400b340 tx=0x7f426400f040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:40.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.611+0000 7f42717fa700 1 -- 172.21.15.78:0/2883175553 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f42741593f0 con 0x7f4274158f90 2023-10-11T09:52:40.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.823+0000 7f427a637700 1 -- 172.21.15.78:0/2883175553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42741593f0 con 0x7f4274158f90 2023-10-11T09:52:40.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.827+0000 7f42717fa700 1 -- 172.21.15.78:0/2883175553 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4656 (secure 0 0 0) 0x7f42741593f0 con 0x7f4274158f90 2023-10-11T09:52:40.833 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:52:40.833 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":112,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8452,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/2481257990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2481257990},{"type":"v1","addr":"172.21.15.78:6836","nonce":2481257990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":8464,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1035227282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1035227282},{"type":"v1","addr":"172.21.15.133:6835","nonce":1035227282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":8680,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/36737634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":36737634},{"type":"v1","addr":"172.21.15.133:6833","nonce":36737634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":101}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T09:51:40.211859+0000","modified":"2023-10-11T09:52:29.509275+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9336},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9336":{"gid":9336,"name":"c","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.78:6837/3884456900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":3884456900},{"type":"v1","addr":"172.21.15.78:6837","nonce":3884456900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-11T09:52:40.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.827+0000 7f427a637700 1 -- 172.21.15.78:0/2883175553 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4254065f00 msgr2=0x7f42540683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:40.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.827+0000 7f427a637700 1 --2- 172.21.15.78:0/2883175553 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4254065f00 0x7f42540683b0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f426400b340 tx=0x7f426400f040 comp rx=0 tx=0).stop 2023-10-11T09:52:40.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.827+0000 7f427a637700 1 -- 172.21.15.78:0/2883175553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4274158f90 msgr2=0x7f427406d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:40.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.827+0000 7f427a637700 1 --2- 172.21.15.78:0/2883175553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4274158f90 0x7f427406d5e0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f425c00c730 tx=0x7f425c00ca40 comp rx=0 tx=0).stop 2023-10-11T09:52:40.836 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.827+0000 7f427a637700 1 -- 172.21.15.78:0/2883175553 shutdown_connections 2023-10-11T09:52:40.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.827+0000 7f427a637700 1 --2- 172.21.15.78:0/2883175553 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4254065f00 0x7f42540683b0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:40.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.827+0000 7f427a637700 1 --2- 172.21.15.78:0/2883175553 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f427406db20 0x7f42740c3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:40.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.827+0000 7f427a637700 1 --2- 172.21.15.78:0/2883175553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4274158f90 0x7f427406d5e0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:40.837 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.827+0000 7f427a637700 1 --2- 172.21.15.78:0/2883175553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4274070470 0x7f42740c1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:40.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.827+0000 7f427a637700 1 -- 172.21.15.78:0/2883175553 >> 172.21.15.78:0/2883175553 conn(0x7f42740cb920 msgr2=0x7f42740cbcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:40.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.827+0000 7f427a637700 1 -- 172.21.15.78:0/2883175553 shutdown_connections 2023-10-11T09:52:40.838 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:40.827+0000 7f427a637700 1 -- 172.21.15.78:0/2883175553 wait complete. 2023-10-11T09:52:40.839 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 112 2023-10-11T09:52:40.856 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:52:41.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.019+0000 7fa5c8e20700 1 Processor -- start 2023-10-11T09:52:41.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.019+0000 7fa5c8e20700 1 -- start start 2023-10-11T09:52:41.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.023+0000 7fa5c8e20700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa5c40b0d20 0x7fa5c4167960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:41.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.023+0000 7fa5c8e20700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5c40cc870 con 0x7fa5c415c9e0 2023-10-11T09:52:41.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.023+0000 7fa5c8e20700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5c40ccb10 con 0x7fa5c4160fb0 2023-10-11T09:52:41.029 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.023+0000 7fa5c8e20700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5c40ade30 con 0x7fa5c40b0d20 2023-10-11T09:52:41.030 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.023+0000 7fa5c37fe700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fa5c415c9e0 0x7fa5c40ce3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:38038/0 (socket says 172.21.15.78:38038) 2023-10-11T09:52:41.030 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.023+0000 7fa5c37fe700 1 -- 172.21.15.78:0/1886141525 learned_addr learned my addr 172.21.15.78:0/1886141525 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:41.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.023+0000 7fa5c3fff700 1 --2- 172.21.15.78:0/1886141525 >> v2:172.21.15.133:3300/0 conn(0x7fa5c40b0d20 0x7fa5c4167960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:41.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.023+0000 7fa5c2ffd700 1 -- 172.21.15.78:0/1886141525 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1331233558 0 0) 0x7fa5c40cc870 con 0x7fa5c415c9e0 2023-10-11T09:52:41.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.023+0000 7fa5c2ffd700 1 -- 172.21.15.78:0/1886141525 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5b00035d0 con 0x7fa5c415c9e0 2023-10-11T09:52:41.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.023+0000 7fa5c3fff700 1 -- 172.21.15.78:0/1886141525 >> v1:172.21.15.133:6789/0 conn(0x7fa5c4160fb0 legacy=0x7fa5c40b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:41.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.023+0000 7fa5c3fff700 1 -- 172.21.15.78:0/1886141525 >> v1:172.21.15.78:6789/0 conn(0x7fa5c415c9e0 legacy=0x7fa5c40ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:41.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.023+0000 7fa5c3fff700 1 -- 172.21.15.78:0/1886141525 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c406bc00 con 0x7fa5c40b0d20 2023-10-11T09:52:41.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.023+0000 7fa5c3fff700 1 --2- 172.21.15.78:0/1886141525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5c40b0d20 0x7fa5c4167960 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc009850 tx=0x7fa5bc009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3346c57c098441bd server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:41.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.023+0000 7fa5c2ffd700 1 -- 172.21.15.78:0/1886141525 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc011830 con 0x7fa5c40b0d20 2023-10-11T09:52:41.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.023+0000 7fa5c2ffd700 1 -- 172.21.15.78:0/1886141525 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5bc011990 con 0x7fa5c40b0d20 2023-10-11T09:52:41.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.023+0000 7fa5c2ffd700 1 -- 172.21.15.78:0/1886141525 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc011d40 con 0x7fa5c40b0d20 2023-10-11T09:52:41.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.023+0000 7fa5c8e20700 1 -- 172.21.15.78:0/1886141525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5c40b0d20 msgr2=0x7fa5c4167960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:41.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.023+0000 7fa5c8e20700 1 --2- 172.21.15.78:0/1886141525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5c40b0d20 0x7fa5c4167960 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc009850 tx=0x7fa5bc009b60 comp rx=0 tx=0).stop 2023-10-11T09:52:41.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.023+0000 7fa5c8e20700 1 -- 172.21.15.78:0/1886141525 shutdown_connections 2023-10-11T09:52:41.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.023+0000 7fa5c8e20700 1 --2- 172.21.15.78:0/1886141525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5c40b0d20 0x7fa5c4167960 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:41.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.023+0000 7fa5c8e20700 1 -- 172.21.15.78:0/1886141525 >> 172.21.15.78:0/1886141525 conn(0x7fa5c40cf820 msgr2=0x7fa5c40cb610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:41.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.023+0000 7fa5c8e20700 1 -- 172.21.15.78:0/1886141525 shutdown_connections 2023-10-11T09:52:41.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.027+0000 7fa5c8e20700 1 -- 172.21.15.78:0/1886141525 wait complete. 2023-10-11T09:52:41.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.027+0000 7fa5c8e20700 1 Processor -- start 2023-10-11T09:52:41.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.027+0000 7fa5c8e20700 1 -- start start 2023-10-11T09:52:41.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.031+0000 7fa5c8e20700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5c40b0d20 0x7fa5c4153390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:41.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.031+0000 7fa5c8e20700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5c415c9e0 0x7fa5c40b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:41.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.031+0000 7fa5c8e20700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa5c4160fb0 0x7fa5c4157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:41.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.031+0000 7fa5c8e20700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c40b3f80 con 0x7fa5c40b0d20 2023-10-11T09:52:41.038 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.031+0000 7fa5c8e20700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c41538d0 con 0x7fa5c415c9e0 2023-10-11T09:52:41.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.031+0000 7fa5c8e20700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c40b3300 con 0x7fa5c4160fb0 2023-10-11T09:52:41.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.031+0000 7fa5c37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5c40b0d20 0x7fa5c4153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:41.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.031+0000 7fa5c37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5c40b0d20 0x7fa5c4153390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43394/0 (socket says 172.21.15.78:43394) 2023-10-11T09:52:41.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.031+0000 7fa5c37fe700 1 -- 172.21.15.78:0/792370978 learned_addr learned my addr 172.21.15.78:0/792370978 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:41.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.031+0000 7fa5c3fff700 1 --2- 172.21.15.78:0/792370978 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa5c4160fb0 0x7fa5c4157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:41.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.031+0000 7fa5baffd700 1 --2- 172.21.15.78:0/792370978 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5c415c9e0 0x7fa5c40b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:41.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.031+0000 7fa5c37fe700 1 -- 172.21.15.78:0/792370978 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa5c4160fb0 msgr2=0x7fa5c4157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:41.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.031+0000 7fa5c37fe700 1 --2- 172.21.15.78:0/792370978 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa5c4160fb0 0x7fa5c4157800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:41.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.031+0000 7fa5c37fe700 1 -- 172.21.15.78:0/792370978 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5c415c9e0 msgr2=0x7fa5c40b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:41.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.031+0000 7fa5c37fe700 1 --2- 172.21.15.78:0/792370978 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5c415c9e0 0x7fa5c40b43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:41.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.031+0000 7fa5c37fe700 1 -- 172.21.15.78:0/792370978 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5bc0094b0 con 0x7fa5c40b0d20 2023-10-11T09:52:41.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.031+0000 7fa5c3fff700 1 --2- 172.21.15.78:0/792370978 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa5c4160fb0 0x7fa5c4157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:52:41.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.031+0000 7fa5c37fe700 1 --2- 172.21.15.78:0/792370978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5c40b0d20 0x7fa5c4153390 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fa5b400cbb0 tx=0x7fa5b400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:41.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.031+0000 7fa5c0ff9700 1 -- 172.21.15.78:0/792370978 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b400a770 con 0x7fa5c40b0d20 2023-10-11T09:52:41.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.031+0000 7fa5c0ff9700 1 -- 172.21.15.78:0/792370978 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b400a8d0 con 0x7fa5c40b0d20 2023-10-11T09:52:41.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.031+0000 7fa5c8e20700 1 -- 172.21.15.78:0/792370978 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c40b35e0 con 0x7fa5c40b0d20 2023-10-11T09:52:41.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.031+0000 7fa5c0ff9700 1 -- 172.21.15.78:0/792370978 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b400ac40 con 0x7fa5c40b0d20 2023-10-11T09:52:41.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.031+0000 7fa5c8e20700 1 -- 172.21.15.78:0/792370978 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c40b3ba0 con 0x7fa5c40b0d20 2023-10-11T09:52:41.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.035+0000 7fa5c8e20700 1 -- 172.21.15.78:0/792370978 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5c4067d00 con 0x7fa5c40b0d20 2023-10-11T09:52:41.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.035+0000 7fa5c0ff9700 1 -- 172.21.15.78:0/792370978 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fa5b401b3b0 con 0x7fa5c40b0d20 2023-10-11T09:52:41.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.039+0000 7fa5c0ff9700 1 --2- 172.21.15.78:0/792370978 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa5a4065ed0 0x7fa5a4068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:41.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.039+0000 7fa5c0ff9700 1 -- 172.21.15.78:0/792370978 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6234+0+0 (secure 0 0 0) 0x7fa5b40831d0 con 0x7fa5c40b0d20 2023-10-11T09:52:41.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.039+0000 7fa5baffd700 1 --2- 172.21.15.78:0/792370978 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa5a4065ed0 0x7fa5a4068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:41.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.039+0000 7fa5baffd700 1 --2- 172.21.15.78:0/792370978 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa5a4065ed0 0x7fa5a4068380 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fa5c415c270 tx=0x7fa5ac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:41.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.043+0000 7fa5c0ff9700 1 -- 172.21.15.78:0/792370978 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fa5b4050780 con 0x7fa5c40b0d20 2023-10-11T09:52:41.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.215+0000 7fa5c8e20700 1 -- 172.21.15.78:0/792370978 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5c40b41f0 con 0x7fa5c40b0d20 2023-10-11T09:52:41.224 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.215+0000 7fa5c0ff9700 1 -- 172.21.15.78:0/792370978 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+18154 (secure 0 0 0) 0x7fa5b4053da0 con 0x7fa5c40b0d20 2023-10-11T09:52:41.225 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:52:41.225 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":129,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:52:22.836530+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-11T09:51:38.592892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-11T09:51:39.204588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6837/4189411531":"2023-10-12T09:52:22.836499+0000","172.21.15.78:6835/4189411531":"2023-10-12T09:52:22.836499+0000","172.21.15.133:6833/1433079498":"2023-10-12T09:52:02.877755+0000","172.21.15.133:6832/1433079498":"2023-10-12T09:52:02.877755+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:52:41.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.219+0000 7fa5c8e20700 1 -- 172.21.15.78:0/792370978 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa5a4065ed0 msgr2=0x7fa5a4068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:41.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.219+0000 7fa5c8e20700 1 --2- 172.21.15.78:0/792370978 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa5a4065ed0 0x7fa5a4068380 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fa5c415c270 tx=0x7fa5ac009040 comp rx=0 tx=0).stop 2023-10-11T09:52:41.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.219+0000 7fa5c8e20700 1 -- 172.21.15.78:0/792370978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5c40b0d20 msgr2=0x7fa5c4153390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:41.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.219+0000 7fa5c8e20700 1 --2- 172.21.15.78:0/792370978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5c40b0d20 0x7fa5c4153390 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fa5b400cbb0 tx=0x7fa5b400cbe0 comp rx=0 tx=0).stop 2023-10-11T09:52:41.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.223+0000 7fa5c8e20700 1 -- 172.21.15.78:0/792370978 shutdown_connections 2023-10-11T09:52:41.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.223+0000 7fa5c8e20700 1 --2- 172.21.15.78:0/792370978 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa5a4065ed0 0x7fa5a4068380 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:41.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.223+0000 7fa5c8e20700 1 --2- 172.21.15.78:0/792370978 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa5c4160fb0 0x7fa5c4157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:41.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.223+0000 7fa5c8e20700 1 --2- 172.21.15.78:0/792370978 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5c415c9e0 0x7fa5c40b43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:41.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.223+0000 7fa5c8e20700 1 --2- 172.21.15.78:0/792370978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5c40b0d20 0x7fa5c4153390 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:41.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.223+0000 7fa5c8e20700 1 -- 172.21.15.78:0/792370978 >> 172.21.15.78:0/792370978 conn(0x7fa5c40cf820 msgr2=0x7fa5c406f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:41.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.223+0000 7fa5c8e20700 1 -- 172.21.15.78:0/792370978 shutdown_connections 2023-10-11T09:52:41.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.223+0000 7fa5c8e20700 1 -- 172.21.15.78:0/792370978 wait complete. 2023-10-11T09:52:41.251 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-11T09:52:41.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.419+0000 7fe7c9bef700 1 Processor -- start 2023-10-11T09:52:41.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.419+0000 7fe7c9bef700 1 -- start start 2023-10-11T09:52:41.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.423+0000 7fe7c9bef700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe7c415a800 0x7fe7c40b4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:41.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.423+0000 7fe7c9bef700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe7c40b0d20 0x7fe7c415e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:41.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.423+0000 7fe7c9bef700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7c4070dc0 con 0x7fe7c406bb60 2023-10-11T09:52:41.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.423+0000 7fe7c9bef700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7c40b46a0 con 0x7fe7c40b0d20 2023-10-11T09:52:41.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.423+0000 7fe7c9bef700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c4070ff0 con 0x7fe7c415a800 2023-10-11T09:52:41.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.423+0000 7fe7c3fff700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe7c406bb60 0x7fe7c40b0610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:37696/0 (socket says 172.21.15.78:37696) 2023-10-11T09:52:41.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.423+0000 7fe7c3fff700 1 -- 172.21.15.78:0/1952740668 learned_addr learned my addr 172.21.15.78:0/1952740668 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:41.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.423+0000 7fe7c93ee700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fe7c40b0d20 0x7fe7c415e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:41.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.423+0000 7fe7c8bed700 1 --2- 172.21.15.78:0/1952740668 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe7c415a800 0x7fe7c40b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:41.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.423+0000 7fe7c37fe700 1 -- 172.21.15.78:0/1952740668 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1454543180 0 0) 0x7fe7c4070dc0 con 0x7fe7c406bb60 2023-10-11T09:52:41.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.423+0000 7fe7c37fe700 1 -- 172.21.15.78:0/1952740668 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7b40035d0 con 0x7fe7c406bb60 2023-10-11T09:52:41.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.423+0000 7fe7c93ee700 1 -- 172.21.15.78:0/1952740668 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe7c415a800 msgr2=0x7fe7c40b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:41.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.423+0000 7fe7c93ee700 1 --2- 172.21.15.78:0/1952740668 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe7c415a800 0x7fe7c40b4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:41.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.423+0000 7fe7c93ee700 1 -- 172.21.15.78:0/1952740668 >> v1:172.21.15.133:6789/0 conn(0x7fe7c406bb60 legacy=0x7fe7c40b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:41.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.423+0000 7fe7c93ee700 1 -- 172.21.15.78:0/1952740668 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c40712d0 con 0x7fe7c40b0d20 2023-10-11T09:52:41.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.423+0000 7fe7c93ee700 1 --2- 172.21.15.78:0/1952740668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7c40b0d20 0x7fe7c415e590 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc009d90 tx=0x7fe7bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b304bd2cac2f392e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:41.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.423+0000 7fe7c37fe700 1 -- 172.21.15.78:0/1952740668 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc00e070 con 0x7fe7c40b0d20 2023-10-11T09:52:41.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.423+0000 7fe7c37fe700 1 -- 172.21.15.78:0/1952740668 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7bc00bb20 con 0x7fe7c40b0d20 2023-10-11T09:52:41.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.423+0000 7fe7c37fe700 1 -- 172.21.15.78:0/1952740668 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc012430 con 0x7fe7c40b0d20 2023-10-11T09:52:41.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.423+0000 7fe7c9bef700 1 -- 172.21.15.78:0/1952740668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7c40b0d20 msgr2=0x7fe7c415e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:41.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.423+0000 7fe7c9bef700 1 --2- 172.21.15.78:0/1952740668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7c40b0d20 0x7fe7c415e590 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc009d90 tx=0x7fe7bc00b330 comp rx=0 tx=0).stop 2023-10-11T09:52:41.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.423+0000 7fe7c9bef700 1 -- 172.21.15.78:0/1952740668 shutdown_connections 2023-10-11T09:52:41.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.423+0000 7fe7c9bef700 1 --2- 172.21.15.78:0/1952740668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7c40b0d20 0x7fe7c415e590 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:41.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.423+0000 7fe7c9bef700 1 --2- 172.21.15.78:0/1952740668 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe7c415a800 0x7fe7c40b4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:41.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.423+0000 7fe7c9bef700 1 -- 172.21.15.78:0/1952740668 >> 172.21.15.78:0/1952740668 conn(0x7fe7c40cf820 msgr2=0x7fe7c40cb610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:41.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.423+0000 7fe7c9bef700 1 -- 172.21.15.78:0/1952740668 shutdown_connections 2023-10-11T09:52:41.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.423+0000 7fe7c9bef700 1 -- 172.21.15.78:0/1952740668 wait complete. 2023-10-11T09:52:41.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.427+0000 7fe7c9bef700 1 Processor -- start 2023-10-11T09:52:41.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.427+0000 7fe7c9bef700 1 -- start start 2023-10-11T09:52:41.437 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.427+0000 7fe7c9bef700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7c406bb60 0x7fe7c4151e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:41.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.427+0000 7fe7c9bef700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7c40b0d20 0x7fe7c415d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:41.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.427+0000 7fe7c9bef700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe7c415a800 0x7fe7c40b2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:41.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.427+0000 7fe7c9bef700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c4153040 con 0x7fe7c40b0d20 2023-10-11T09:52:41.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.427+0000 7fe7c9bef700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c41e44a0 con 0x7fe7c406bb60 2023-10-11T09:52:41.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.427+0000 7fe7c9bef700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c41e4780 con 0x7fe7c415a800 2023-10-11T09:52:41.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.431+0000 7fe7c3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7c40b0d20 0x7fe7c415d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:41.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.431+0000 7fe7c3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7c40b0d20 0x7fe7c415d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43422/0 (socket says 172.21.15.78:43422) 2023-10-11T09:52:41.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.431+0000 7fe7c3fff700 1 -- 172.21.15.78:0/683430563 learned_addr learned my addr 172.21.15.78:0/683430563 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:41.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.431+0000 7fe7c8bed700 1 --2- 172.21.15.78:0/683430563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7c406bb60 0x7fe7c4151e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:41.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.431+0000 7fe7c93ee700 1 --2- 172.21.15.78:0/683430563 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe7c415a800 0x7fe7c40b2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:41.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.431+0000 7fe7c3fff700 1 -- 172.21.15.78:0/683430563 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe7c415a800 msgr2=0x7fe7c40b2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:41.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.431+0000 7fe7c3fff700 1 --2- 172.21.15.78:0/683430563 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe7c415a800 0x7fe7c40b2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:41.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.431+0000 7fe7c3fff700 1 -- 172.21.15.78:0/683430563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7c406bb60 msgr2=0x7fe7c4151e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:41.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.431+0000 7fe7c3fff700 1 --2- 172.21.15.78:0/683430563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7c406bb60 0x7fe7c4151e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:41.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.431+0000 7fe7c3fff700 1 -- 172.21.15.78:0/683430563 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7bc009a70 con 0x7fe7c40b0d20 2023-10-11T09:52:41.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.431+0000 7fe7c3fff700 1 --2- 172.21.15.78:0/683430563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7c40b0d20 0x7fe7c415d380 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0001130 tx=0x7fe7b000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:41.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.431+0000 7fe7c17fa700 1 -- 172.21.15.78:0/683430563 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0005de0 con 0x7fe7c40b0d20 2023-10-11T09:52:41.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.431+0000 7fe7c17fa700 1 -- 172.21.15.78:0/683430563 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b00070d0 con 0x7fe7c40b0d20 2023-10-11T09:52:41.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.431+0000 7fe7c17fa700 1 -- 172.21.15.78:0/683430563 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0007460 con 0x7fe7c40b0d20 2023-10-11T09:52:41.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.431+0000 7fe7c9bef700 1 -- 172.21.15.78:0/683430563 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7c41e4a60 con 0x7fe7c40b0d20 2023-10-11T09:52:41.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.431+0000 7fe7c9bef700 1 -- 172.21.15.78:0/683430563 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7c41e5020 con 0x7fe7c40b0d20 2023-10-11T09:52:41.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.431+0000 7fe7c9bef700 1 -- 172.21.15.78:0/683430563 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7c4067d00 con 0x7fe7c40b0d20 2023-10-11T09:52:41.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.435+0000 7fe7c17fa700 1 -- 172.21.15.78:0/683430563 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fe7b00075c0 con 0x7fe7c40b0d20 2023-10-11T09:52:41.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.435+0000 7fe7c17fa700 1 --2- 172.21.15.78:0/683430563 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe7a8065ed0 0x7fe7a8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:41.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.435+0000 7fe7c17fa700 1 -- 172.21.15.78:0/683430563 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6234+0+0 (secure 0 0 0) 0x7fe7b0082ea0 con 0x7fe7c40b0d20 2023-10-11T09:52:41.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.435+0000 7fe7c8bed700 1 --2- 172.21.15.78:0/683430563 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe7a8065ed0 0x7fe7a8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:41.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.439+0000 7fe7c8bed700 1 --2- 172.21.15.78:0/683430563 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe7a8065ed0 0x7fe7a8068380 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009b80 tx=0x7fe7b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:41.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.443+0000 7fe7c17fa700 1 -- 172.21.15.78:0/683430563 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fe7b00503a0 con 0x7fe7c40b0d20 2023-10-11T09:52:41.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.655+0000 7fe7c9bef700 1 -- 172.21.15.78:0/683430563 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe7c41e5210 con 0x7fe7c40b0d20 2023-10-11T09:52:41.665 INFO:tasks.ceph.mon.a.smithi078.stderr:2023-10-11T09:52:41.659+0000 7f90131f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-11T09:52:41.682 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.675+0000 7fe7c17fa700 1 -- 172.21.15.78:0/683430563 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v113) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe7b0017090 con 0x7fe7c40b0d20 2023-10-11T09:52:41.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.675+0000 7fe7c9bef700 1 -- 172.21.15.78:0/683430563 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe7a8065ed0 msgr2=0x7fe7a8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:41.684 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.675+0000 7fe7c9bef700 1 --2- 172.21.15.78:0/683430563 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe7a8065ed0 0x7fe7a8068380 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009b80 tx=0x7fe7b8008040 comp rx=0 tx=0).stop 2023-10-11T09:52:41.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.675+0000 7fe7c9bef700 1 -- 172.21.15.78:0/683430563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7c40b0d20 msgr2=0x7fe7c415d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:41.685 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.675+0000 7fe7c9bef700 1 --2- 172.21.15.78:0/683430563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7c40b0d20 0x7fe7c415d380 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0001130 tx=0x7fe7b000c880 comp rx=0 tx=0).stop 2023-10-11T09:52:41.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.679+0000 7fe7c9bef700 1 -- 172.21.15.78:0/683430563 shutdown_connections 2023-10-11T09:52:41.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.679+0000 7fe7c9bef700 1 --2- 172.21.15.78:0/683430563 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fe7a8065ed0 0x7fe7a8068380 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:41.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.679+0000 7fe7c9bef700 1 --2- 172.21.15.78:0/683430563 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe7c415a800 0x7fe7c40b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:41.686 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.679+0000 7fe7c9bef700 1 --2- 172.21.15.78:0/683430563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7c40b0d20 0x7fe7c415d380 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:41.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.679+0000 7fe7c9bef700 1 --2- 172.21.15.78:0/683430563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7c406bb60 0x7fe7c4151e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:41.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.679+0000 7fe7c9bef700 1 -- 172.21.15.78:0/683430563 >> 172.21.15.78:0/683430563 conn(0x7fe7c40cf820 msgr2=0x7fe7c40cdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:41.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.679+0000 7fe7c9bef700 1 -- 172.21.15.78:0/683430563 shutdown_connections 2023-10-11T09:52:41.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.679+0000 7fe7c9bef700 1 -- 172.21.15.78:0/683430563 wait complete. 2023-10-11T09:52:41.688 INFO:teuthology.orchestra.run.smithi078.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-11T09:52:41.697 INFO:tasks.ceph.mds.c.smithi078.stderr: -456> 2023-10-11T09:52:33.863+0000 7f7d8d381700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-11T09:52:41.705 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-11T09:52:41.752 INFO:tasks.ceph.mds.c.smithi078.stdout:starting mds.c at 2023-10-11T09:52:41.891 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.883+0000 7f57b2e91700 1 Processor -- start 2023-10-11T09:52:41.892 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.883+0000 7f57b2e91700 1 -- start start 2023-10-11T09:52:41.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.887+0000 7f57b2e91700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f57ac15f090 0x7f57ac165590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:41.894 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.887+0000 7f57b2e91700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57ac072040 con 0x7f57ac0718d0 2023-10-11T09:52:41.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.887+0000 7f57b2e91700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57ac15ec00 con 0x7f57ac0caf80 2023-10-11T09:52:41.895 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.887+0000 7f57b2e91700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57ac15bd80 con 0x7f57ac15f090 2023-10-11T09:52:41.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.887+0000 7f57b142e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f57ac15f090 0x7f57ac165590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:41.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.887+0000 7f57b0c2d700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f57ac0caf80 0x7f57ac0711c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:37720/0 (socket says 172.21.15.78:37720) 2023-10-11T09:52:41.896 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.887+0000 7f57abfff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f57ac0718d0 0x7f57ac15a660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:38046/0 (socket says 172.21.15.78:38046) 2023-10-11T09:52:41.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.887+0000 7f57b142e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f57ac15f090 0x7f57ac165590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:35930/0 (socket says 172.21.15.78:35930) 2023-10-11T09:52:41.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.887+0000 7f57b0c2d700 1 -- 172.21.15.78:0/2625911800 learned_addr learned my addr 172.21.15.78:0/2625911800 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:41.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.887+0000 7f57b142e700 1 -- 172.21.15.78:0/2625911800 >> v1:172.21.15.133:6789/0 conn(0x7f57ac0caf80 legacy=0x7f57ac0711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:41.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.887+0000 7f57b142e700 1 -- 172.21.15.78:0/2625911800 >> v1:172.21.15.78:6789/0 conn(0x7f57ac0718d0 legacy=0x7f57ac15a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:41.897 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.887+0000 7f57b142e700 1 -- 172.21.15.78:0/2625911800 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ac071d60 con 0x7f57ac15f090 2023-10-11T09:52:41.898 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.887+0000 7f57b142e700 1 --2- 172.21.15.78:0/2625911800 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f57ac15f090 0x7f57ac165590 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f57a4009dc0 tx=0x7f57a400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da80705050 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:41.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.891+0000 7f57ab7fe700 1 -- 172.21.15.78:0/2625911800 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a400e070 con 0x7f57ac15f090 2023-10-11T09:52:41.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.891+0000 7f57ab7fe700 1 -- 172.21.15.78:0/2625911800 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57a400bbd0 con 0x7f57ac15f090 2023-10-11T09:52:41.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.891+0000 7f57ab7fe700 1 -- 172.21.15.78:0/2625911800 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a4012430 con 0x7f57ac15f090 2023-10-11T09:52:41.899 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.891+0000 7f57b2e91700 1 -- 172.21.15.78:0/2625911800 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f57ac15f090 msgr2=0x7f57ac165590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:41.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.891+0000 7f57b2e91700 1 --2- 172.21.15.78:0/2625911800 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f57ac15f090 0x7f57ac165590 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f57a4009dc0 tx=0x7f57a400b330 comp rx=0 tx=0).stop 2023-10-11T09:52:41.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.891+0000 7f57b2e91700 1 -- 172.21.15.78:0/2625911800 shutdown_connections 2023-10-11T09:52:41.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.891+0000 7f57b2e91700 1 --2- 172.21.15.78:0/2625911800 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f57ac15f090 0x7f57ac165590 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:41.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.891+0000 7f57b2e91700 1 -- 172.21.15.78:0/2625911800 >> 172.21.15.78:0/2625911800 conn(0x7f57ac06ae10 msgr2=0x7f57ac0b0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:41.900 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.891+0000 7f57b2e91700 1 -- 172.21.15.78:0/2625911800 shutdown_connections 2023-10-11T09:52:41.901 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.891+0000 7f57b2e91700 1 -- 172.21.15.78:0/2625911800 wait complete. 2023-10-11T09:52:41.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.891+0000 7f57b2e91700 1 Processor -- start 2023-10-11T09:52:41.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.891+0000 7f57b2e91700 1 -- start start 2023-10-11T09:52:41.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.895+0000 7f57b2e91700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57ac0718d0 0x7f57ac14b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:41.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.895+0000 7f57b2e91700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f57ac0caf80 0x7f57ac1581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:41.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.895+0000 7f57b2e91700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57ac15f090 0x7f57ac0ce110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:41.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.895+0000 7f57b2e91700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ac0cdc80 con 0x7f57ac15f090 2023-10-11T09:52:41.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.895+0000 7f57b2e91700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ac150bd0 con 0x7f57ac0718d0 2023-10-11T09:52:41.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.895+0000 7f57b2e91700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57ac0cd070 con 0x7f57ac0caf80 2023-10-11T09:52:41.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.895+0000 7f57b142e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57ac15f090 0x7f57ac0ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:41.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.895+0000 7f57b142e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57ac15f090 0x7f57ac0ce110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43454/0 (socket says 172.21.15.78:43454) 2023-10-11T09:52:41.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.895+0000 7f57b142e700 1 -- 172.21.15.78:0/2445310503 learned_addr learned my addr 172.21.15.78:0/2445310503 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:41.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.895+0000 7f57b0c2d700 1 --2- 172.21.15.78:0/2445310503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57ac0718d0 0x7f57ac14b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:41.905 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.895+0000 7f57abfff700 1 --2- 172.21.15.78:0/2445310503 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f57ac0caf80 0x7f57ac1581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:41.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.895+0000 7f57b142e700 1 -- 172.21.15.78:0/2445310503 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f57ac0caf80 msgr2=0x7f57ac1581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:41.906 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.895+0000 7f57b142e700 1 --2- 172.21.15.78:0/2445310503 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f57ac0caf80 0x7f57ac1581f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:41.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.895+0000 7f57b142e700 1 -- 172.21.15.78:0/2445310503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57ac0718d0 msgr2=0x7f57ac14b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:41.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.895+0000 7f57b142e700 1 --2- 172.21.15.78:0/2445310503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57ac0718d0 0x7f57ac14b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:41.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.895+0000 7f57b142e700 1 -- 172.21.15.78:0/2445310503 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a4009a70 con 0x7f57ac15f090 2023-10-11T09:52:41.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.895+0000 7f57b142e700 1 --2- 172.21.15.78:0/2445310503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57ac15f090 0x7f57ac0ce110 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f57a400b370 tx=0x7f57a4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:41.907 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.895+0000 7f57a97fa700 1 -- 172.21.15.78:0/2445310503 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a4012da0 con 0x7f57ac15f090 2023-10-11T09:52:41.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.895+0000 7f57a97fa700 1 -- 172.21.15.78:0/2445310503 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57a4020070 con 0x7f57ac15f090 2023-10-11T09:52:41.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.895+0000 7f57a97fa700 1 -- 172.21.15.78:0/2445310503 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a4004280 con 0x7f57ac15f090 2023-10-11T09:52:41.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.895+0000 7f57b2e91700 1 -- 172.21.15.78:0/2445310503 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57ac0cd350 con 0x7f57ac15f090 2023-10-11T09:52:41.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.895+0000 7f57b2e91700 1 -- 172.21.15.78:0/2445310503 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57ac0cd910 con 0x7f57ac15f090 2023-10-11T09:52:41.908 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.899+0000 7f57b2e91700 1 -- 172.21.15.78:0/2445310503 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57ac0cde30 con 0x7f57ac15f090 2023-10-11T09:52:41.909 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.899+0000 7f57a97fa700 1 -- 172.21.15.78:0/2445310503 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f57a40043e0 con 0x7f57ac15f090 2023-10-11T09:52:41.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.903+0000 7f57a97fa700 1 --2- 172.21.15.78:0/2445310503 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5790065f00 0x7f57900683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:41.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.903+0000 7f57a97fa700 1 -- 172.21.15.78:0/2445310503 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6320+0+0 (secure 0 0 0) 0x7f57a4022070 con 0x7f57ac15f090 2023-10-11T09:52:41.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.903+0000 7f57b0c2d700 1 --2- 172.21.15.78:0/2445310503 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5790065f00 0x7f57900683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:41.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.903+0000 7f57b0c2d700 1 --2- 172.21.15.78:0/2445310503 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5790065f00 0x7f57900683b0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f57a0005930 tx=0x7f57a0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:41.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:41.907+0000 7f57a97fa700 1 -- 172.21.15.78:0/2445310503 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f57ac0cde30 con 0x7f57ac15f090 2023-10-11T09:52:42.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.119+0000 7f57b2e91700 1 -- 172.21.15.78:0/2445310503 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f57ac06f4f0 con 0x7f57ac15f090 2023-10-11T09:52:42.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.683+0000 7f57a97fa700 1 -- 172.21.15.78:0/2445310503 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v114) v1 ==== 104+0+0 (secure 0 0 0) 0x7f57a4052bd0 con 0x7f57ac15f090 2023-10-11T09:52:42.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.683+0000 7f57b2e91700 1 -- 172.21.15.78:0/2445310503 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5790065f00 msgr2=0x7f57900683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:42.692 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.683+0000 7f57b2e91700 1 --2- 172.21.15.78:0/2445310503 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5790065f00 0x7f57900683b0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f57a0005930 tx=0x7f57a0009040 comp rx=0 tx=0).stop 2023-10-11T09:52:42.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.683+0000 7f57b2e91700 1 -- 172.21.15.78:0/2445310503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57ac15f090 msgr2=0x7f57ac0ce110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:42.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.683+0000 7f57b2e91700 1 --2- 172.21.15.78:0/2445310503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57ac15f090 0x7f57ac0ce110 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f57a400b370 tx=0x7f57a4004af0 comp rx=0 tx=0).stop 2023-10-11T09:52:42.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.687+0000 7f57b2e91700 1 -- 172.21.15.78:0/2445310503 shutdown_connections 2023-10-11T09:52:42.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.687+0000 7f57b2e91700 1 --2- 172.21.15.78:0/2445310503 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5790065f00 0x7f57900683b0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:42.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.687+0000 7f57b2e91700 1 --2- 172.21.15.78:0/2445310503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57ac15f090 0x7f57ac0ce110 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:42.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.687+0000 7f57b2e91700 1 --2- 172.21.15.78:0/2445310503 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f57ac0caf80 0x7f57ac1581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:42.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.687+0000 7f57b2e91700 1 --2- 172.21.15.78:0/2445310503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57ac0718d0 0x7f57ac14b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:42.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.687+0000 7f57b2e91700 1 -- 172.21.15.78:0/2445310503 >> 172.21.15.78:0/2445310503 conn(0x7f57ac06ae10 msgr2=0x7f57ac15c070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:42.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.687+0000 7f57b2e91700 1 -- 172.21.15.78:0/2445310503 shutdown_connections 2023-10-11T09:52:42.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.687+0000 7f57b2e91700 1 -- 172.21.15.78:0/2445310503 wait complete. 2023-10-11T09:52:42.716 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-11T09:52:42.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.939+0000 7f0ec3071700 1 Processor -- start 2023-10-11T09:52:42.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.939+0000 7f0ec3071700 1 -- start start 2023-10-11T09:52:42.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.943+0000 7f0ec3071700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ebc162840 0x7f0ebc06df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:42.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.943+0000 7f0ec3071700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0ebc0c2ee0 0x7f0ebc164fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:42.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.943+0000 7f0ec3071700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ebc0cec10 con 0x7f0ebc162470 2023-10-11T09:52:42.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.943+0000 7f0ec3071700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ebc06e5c0 con 0x7f0ebc0c2ee0 2023-10-11T09:52:42.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.943+0000 7f0ec3071700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ebc06e8a0 con 0x7f0ebc162840 2023-10-11T09:52:42.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.943+0000 7f0ec160e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0ebc0c2ee0 0x7f0ebc164fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:42.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.943+0000 7f0ec160e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f0ebc0c2ee0 0x7f0ebc164fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43460/0 (socket says 172.21.15.78:43460) 2023-10-11T09:52:42.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.943+0000 7f0ec160e700 1 -- 172.21.15.78:0/691884229 learned_addr learned my addr 172.21.15.78:0/691884229 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:42.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.943+0000 7f0ec160e700 1 -- 172.21.15.78:0/691884229 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ebc162840 msgr2=0x7f0ebc06df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:42.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.943+0000 7f0ec160e700 1 --2- 172.21.15.78:0/691884229 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ebc162840 0x7f0ebc06df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:42.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.943+0000 7f0ec160e700 1 -- 172.21.15.78:0/691884229 >> v1:172.21.15.133:6789/0 conn(0x7f0ebc162470 legacy=0x7f0ebc0ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:42.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.943+0000 7f0ec160e700 1 -- 172.21.15.78:0/691884229 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ebc06eb80 con 0x7f0ebc0c2ee0 2023-10-11T09:52:42.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.943+0000 7f0ec160e700 1 --2- 172.21.15.78:0/691884229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ebc0c2ee0 0x7f0ebc164fc0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4009910 tx=0x7f0eb4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ff896be732b8c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:42.954 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.943+0000 7f0eb37fe700 1 -- 172.21.15.78:0/691884229 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4004230 con 0x7f0ebc0c2ee0 2023-10-11T09:52:42.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.943+0000 7f0eb37fe700 1 -- 172.21.15.78:0/691884229 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb4004390 con 0x7f0ebc0c2ee0 2023-10-11T09:52:42.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.943+0000 7f0eb37fe700 1 -- 172.21.15.78:0/691884229 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4009e60 con 0x7f0ebc0c2ee0 2023-10-11T09:52:42.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.947+0000 7f0ec3071700 1 -- 172.21.15.78:0/691884229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ebc0c2ee0 msgr2=0x7f0ebc164fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:42.955 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.947+0000 7f0ec3071700 1 --2- 172.21.15.78:0/691884229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ebc0c2ee0 0x7f0ebc164fc0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4009910 tx=0x7f0eb4009c20 comp rx=0 tx=0).stop 2023-10-11T09:52:42.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.947+0000 7f0ec3071700 1 -- 172.21.15.78:0/691884229 shutdown_connections 2023-10-11T09:52:42.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.947+0000 7f0ec3071700 1 --2- 172.21.15.78:0/691884229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ebc0c2ee0 0x7f0ebc164fc0 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:42.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.947+0000 7f0ec3071700 1 --2- 172.21.15.78:0/691884229 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ebc162840 0x7f0ebc06df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:42.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.947+0000 7f0ec3071700 1 -- 172.21.15.78:0/691884229 >> 172.21.15.78:0/691884229 conn(0x7f0ebc0cd650 msgr2=0x7f0ebc0cff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:42.956 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.947+0000 7f0ec3071700 1 -- 172.21.15.78:0/691884229 shutdown_connections 2023-10-11T09:52:42.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.947+0000 7f0ec3071700 1 -- 172.21.15.78:0/691884229 wait complete. 2023-10-11T09:52:42.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.951+0000 7f0ec3071700 1 Processor -- start 2023-10-11T09:52:42.957 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.951+0000 7f0ec3071700 1 -- start start 2023-10-11T09:52:42.958 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.951+0000 7f0ec3071700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ebc0c2ee0 0x7f0ebc15c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:42.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.951+0000 7f0ec3071700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ebc162470 0x7f0ebc165430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:42.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.951+0000 7f0ec3071700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ebc163630 0x7f0ebc0b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:42.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.951+0000 7f0ec3071700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ebc164720 con 0x7f0ebc0c2ee0 2023-10-11T09:52:42.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.951+0000 7f0ec3071700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ebc15c8b0 con 0x7f0ebc163630 2023-10-11T09:52:42.959 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.951+0000 7f0ec3071700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ebc163b10 con 0x7f0ebc162470 2023-10-11T09:52:42.960 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.951+0000 7f0ec0e0d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ebc0c2ee0 0x7f0ebc15c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:42.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.951+0000 7f0ec0e0d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ebc0c2ee0 0x7f0ebc15c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43476/0 (socket says 172.21.15.78:43476) 2023-10-11T09:52:42.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.951+0000 7f0ec0e0d700 1 -- 172.21.15.78:0/2141428920 learned_addr learned my addr 172.21.15.78:0/2141428920 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:42.961 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.951+0000 7f0ec160e700 1 --2- 172.21.15.78:0/2141428920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ebc163630 0x7f0ebc0b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:42.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.951+0000 7f0eb3fff700 1 --2- 172.21.15.78:0/2141428920 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ebc162470 0x7f0ebc165430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:42.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.951+0000 7f0ec0e0d700 1 -- 172.21.15.78:0/2141428920 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ebc162470 msgr2=0x7f0ebc165430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:42.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.951+0000 7f0ec0e0d700 1 --2- 172.21.15.78:0/2141428920 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ebc162470 0x7f0ebc165430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:42.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.951+0000 7f0ec0e0d700 1 -- 172.21.15.78:0/2141428920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ebc163630 msgr2=0x7f0ebc0b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:42.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.951+0000 7f0ec0e0d700 1 --2- 172.21.15.78:0/2141428920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ebc163630 0x7f0ebc0b0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:42.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.951+0000 7f0ec0e0d700 1 -- 172.21.15.78:0/2141428920 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb4009570 con 0x7f0ebc0c2ee0 2023-10-11T09:52:42.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.951+0000 7f0ec0e0d700 1 --2- 172.21.15.78:0/2141428920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ebc0c2ee0 0x7f0ebc15c370 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f0eb800cc50 tx=0x7f0eb800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:42.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.951+0000 7f0eb17fa700 1 -- 172.21.15.78:0/2141428920 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb800aad0 con 0x7f0ebc0c2ee0 2023-10-11T09:52:42.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.951+0000 7f0eb17fa700 1 -- 172.21.15.78:0/2141428920 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb800ac30 con 0x7f0ebc0c2ee0 2023-10-11T09:52:42.963 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.951+0000 7f0ec3071700 1 -- 172.21.15.78:0/2141428920 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ebc163df0 con 0x7f0ebc0c2ee0 2023-10-11T09:52:42.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.951+0000 7f0eb17fa700 1 -- 172.21.15.78:0/2141428920 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb8012420 con 0x7f0ebc0c2ee0 2023-10-11T09:52:42.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.955+0000 7f0ec3071700 1 -- 172.21.15.78:0/2141428920 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ebc1643b0 con 0x7f0ebc0c2ee0 2023-10-11T09:52:42.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.955+0000 7f0ec3071700 1 -- 172.21.15.78:0/2141428920 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ebc065e00 con 0x7f0ebc0c2ee0 2023-10-11T09:52:42.964 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.955+0000 7f0eb17fa700 1 -- 172.21.15.78:0/2141428920 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f0eb8012580 con 0x7f0ebc0c2ee0 2023-10-11T09:52:42.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.959+0000 7f0eb17fa700 1 --2- 172.21.15.78:0/2141428920 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0ea8065f00 0x7f0ea80683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:42.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.959+0000 7f0eb17fa700 1 -- 172.21.15.78:0/2141428920 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6320+0+0 (secure 0 0 0) 0x7f0eb8083ff0 con 0x7f0ebc0c2ee0 2023-10-11T09:52:42.966 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.959+0000 7f0eb3fff700 1 --2- 172.21.15.78:0/2141428920 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0ea8065f00 0x7f0ea80683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:42.967 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.959+0000 7f0eb3fff700 1 --2- 172.21.15.78:0/2141428920 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0ea8065f00 0x7f0ea80683b0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc0cf7f0 tx=0x7f0eac009110 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:42.971 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:42.963+0000 7f0eb17fa700 1 -- 172.21.15.78:0/2141428920 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f0eb80b4fa0 con 0x7f0ebc0c2ee0 2023-10-11T09:52:43.145 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.139+0000 7f0ec3071700 1 -- 172.21.15.78:0/2141428920 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0ebc066b50 con 0x7f0ebc0c2ee0 2023-10-11T09:52:43.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.695+0000 7f0eb17fa700 1 -- 172.21.15.78:0/2141428920 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v131) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0eb8016020 con 0x7f0ebc0c2ee0 2023-10-11T09:52:43.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.699+0000 7f0ec3071700 1 -- 172.21.15.78:0/2141428920 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0ea8065f00 msgr2=0x7f0ea80683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:43.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.699+0000 7f0ec3071700 1 --2- 172.21.15.78:0/2141428920 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0ea8065f00 0x7f0ea80683b0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc0cf7f0 tx=0x7f0eac009110 comp rx=0 tx=0).stop 2023-10-11T09:52:43.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.699+0000 7f0ec3071700 1 -- 172.21.15.78:0/2141428920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ebc0c2ee0 msgr2=0x7f0ebc15c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:43.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.699+0000 7f0ec3071700 1 --2- 172.21.15.78:0/2141428920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ebc0c2ee0 0x7f0ebc15c370 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f0eb800cc50 tx=0x7f0eb800cf60 comp rx=0 tx=0).stop 2023-10-11T09:52:43.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.703+0000 7f0ec3071700 1 -- 172.21.15.78:0/2141428920 shutdown_connections 2023-10-11T09:52:43.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.703+0000 7f0ec3071700 1 --2- 172.21.15.78:0/2141428920 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f0ea8065f00 0x7f0ea80683b0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:43.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.703+0000 7f0ec3071700 1 --2- 172.21.15.78:0/2141428920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ebc163630 0x7f0ebc0b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:43.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.703+0000 7f0ec3071700 1 --2- 172.21.15.78:0/2141428920 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ebc162470 0x7f0ebc165430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:43.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.703+0000 7f0ec3071700 1 --2- 172.21.15.78:0/2141428920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ebc0c2ee0 0x7f0ebc15c370 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:43.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.703+0000 7f0ec3071700 1 -- 172.21.15.78:0/2141428920 >> 172.21.15.78:0/2141428920 conn(0x7f0ebc0cd650 msgr2=0x7f0ebc0cbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:43.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.703+0000 7f0ec3071700 1 -- 172.21.15.78:0/2141428920 shutdown_connections 2023-10-11T09:52:43.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.703+0000 7f0ec3071700 1 -- 172.21.15.78:0/2141428920 wait complete. 2023-10-11T09:52:43.711 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' removed 2023-10-11T09:52:43.730 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-11T09:52:43.910 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.903+0000 7f882605d700 1 Processor -- start 2023-10-11T09:52:43.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.903+0000 7f882605d700 1 -- start start 2023-10-11T09:52:43.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.907+0000 7f882605d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f882015d380 0x7f882015d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:43.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.907+0000 7f882605d700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f882006bed0 0x7f882006c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:43.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.907+0000 7f882605d700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88200ce660 con 0x7f8820071bc0 2023-10-11T09:52:43.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.907+0000 7f882605d700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f882015dd60 con 0x7f882015d380 2023-10-11T09:52:43.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.907+0000 7f882605d700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88200cda50 con 0x7f882006bed0 2023-10-11T09:52:43.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.907+0000 7f882485a700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f882015d380 0x7f882015d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:43.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.907+0000 7f882485a700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f882015d380 0x7f882015d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43482/0 (socket says 172.21.15.78:43482) 2023-10-11T09:52:43.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.907+0000 7f882485a700 1 -- 172.21.15.78:0/2751091801 learned_addr learned my addr 172.21.15.78:0/2751091801 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:43.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.907+0000 7f882585c700 1 --2- 172.21.15.78:0/2751091801 >> v2:172.21.15.133:3300/0 conn(0x7f882006bed0 0x7f882006c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:43.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.907+0000 7f882485a700 1 -- 172.21.15.78:0/2751091801 >> v2:172.21.15.133:3300/0 conn(0x7f882006bed0 msgr2=0x7f882006c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:43.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.907+0000 7f882485a700 1 --2- 172.21.15.78:0/2751091801 >> v2:172.21.15.133:3300/0 conn(0x7f882006bed0 0x7f882006c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:43.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.907+0000 7f882485a700 1 -- 172.21.15.78:0/2751091801 >> v1:172.21.15.133:6789/0 conn(0x7f8820071bc0 legacy=0x7f88200ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:43.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.907+0000 7f882485a700 1 -- 172.21.15.78:0/2751091801 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88200cdd30 con 0x7f882015d380 2023-10-11T09:52:43.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.907+0000 7f882485a700 1 --2- 172.21.15.78:0/2751091801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f882015d380 0x7f882015d820 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f882006fd00 tx=0x7f881000be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f7f73bc1312b504 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:43.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.907+0000 7f8817fff700 1 -- 172.21.15.78:0/2751091801 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881000dd90 con 0x7f882015d380 2023-10-11T09:52:43.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.907+0000 7f8817fff700 1 -- 172.21.15.78:0/2751091801 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8810010470 con 0x7f882015d380 2023-10-11T09:52:43.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.907+0000 7f8817fff700 1 -- 172.21.15.78:0/2751091801 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8810010800 con 0x7f882015d380 2023-10-11T09:52:43.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.907+0000 7f882605d700 1 -- 172.21.15.78:0/2751091801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f882015d380 msgr2=0x7f882015d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:43.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.907+0000 7f882605d700 1 --2- 172.21.15.78:0/2751091801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f882015d380 0x7f882015d820 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f882006fd00 tx=0x7f881000be90 comp rx=0 tx=0).stop 2023-10-11T09:52:43.919 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.907+0000 7f882605d700 1 -- 172.21.15.78:0/2751091801 shutdown_connections 2023-10-11T09:52:43.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.907+0000 7f882605d700 1 --2- 172.21.15.78:0/2751091801 >> v2:172.21.15.133:3300/0 conn(0x7f882006bed0 0x7f882006c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:43.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.907+0000 7f882605d700 1 --2- 172.21.15.78:0/2751091801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f882015d380 0x7f882015d820 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:43.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.907+0000 7f882605d700 1 -- 172.21.15.78:0/2751091801 >> 172.21.15.78:0/2751091801 conn(0x7f88200b13b0 msgr2=0x7f882006a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:43.920 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.911+0000 7f882605d700 1 -- 172.21.15.78:0/2751091801 shutdown_connections 2023-10-11T09:52:43.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.911+0000 7f882605d700 1 -- 172.21.15.78:0/2751091801 wait complete. 2023-10-11T09:52:43.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.915+0000 7f882605d700 1 Processor -- start 2023-10-11T09:52:43.921 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.915+0000 7f882605d700 1 -- start start 2023-10-11T09:52:43.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.915+0000 7f882605d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f882006bed0 0x7f8820154600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:43.922 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.915+0000 7f882605d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8820071bc0 0x7f88200b1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:43.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.915+0000 7f882605d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f882015d380 0x7f88201582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:43.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.915+0000 7f882605d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88201f3330 con 0x7f882015d380 2023-10-11T09:52:43.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.915+0000 7f882605d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88200b2ae0 con 0x7f8820071bc0 2023-10-11T09:52:43.923 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.915+0000 7f882605d700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88200b2520 con 0x7f882006bed0 2023-10-11T09:52:43.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.915+0000 7f882585c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f882015d380 0x7f88201582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:43.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.915+0000 7f882585c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f882015d380 0x7f88201582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43498/0 (socket says 172.21.15.78:43498) 2023-10-11T09:52:43.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.915+0000 7f882585c700 1 -- 172.21.15.78:0/2992479765 learned_addr learned my addr 172.21.15.78:0/2992479765 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:43.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.915+0000 7f882485a700 1 --2- 172.21.15.78:0/2992479765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8820071bc0 0x7f88200b1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:43.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.915+0000 7f882505b700 1 --2- 172.21.15.78:0/2992479765 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f882006bed0 0x7f8820154600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:43.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.915+0000 7f882585c700 1 -- 172.21.15.78:0/2992479765 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f882006bed0 msgr2=0x7f8820154600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:43.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.915+0000 7f882585c700 1 --2- 172.21.15.78:0/2992479765 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f882006bed0 0x7f8820154600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:43.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.915+0000 7f882585c700 1 -- 172.21.15.78:0/2992479765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8820071bc0 msgr2=0x7f88200b1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:43.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.915+0000 7f882585c700 1 --2- 172.21.15.78:0/2992479765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8820071bc0 0x7f88200b1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:43.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.915+0000 7f882585c700 1 -- 172.21.15.78:0/2992479765 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f881000b810 con 0x7f882015d380 2023-10-11T09:52:43.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.915+0000 7f882585c700 1 --2- 172.21.15.78:0/2992479765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f882015d380 0x7f88201582e0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f8818007e60 tx=0x7f881800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:43.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.915+0000 7f8815ffb700 1 -- 172.21.15.78:0/2992479765 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881800ed30 con 0x7f882015d380 2023-10-11T09:52:43.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.915+0000 7f8815ffb700 1 -- 172.21.15.78:0/2992479765 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8818014420 con 0x7f882015d380 2023-10-11T09:52:43.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.915+0000 7f8815ffb700 1 -- 172.21.15.78:0/2992479765 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88180147d0 con 0x7f882015d380 2023-10-11T09:52:43.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.915+0000 7f882605d700 1 -- 172.21.15.78:0/2992479765 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88200b2800 con 0x7f882015d380 2023-10-11T09:52:43.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.919+0000 7f882605d700 1 -- 172.21.15.78:0/2992479765 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88200b32e0 con 0x7f882015d380 2023-10-11T09:52:43.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.919+0000 7f882605d700 1 -- 172.21.15.78:0/2992479765 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88200744c0 con 0x7f882015d380 2023-10-11T09:52:43.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.919+0000 7f8815ffb700 1 -- 172.21.15.78:0/2992479765 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f8818014930 con 0x7f882015d380 2023-10-11T09:52:43.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.923+0000 7f8815ffb700 1 --2- 172.21.15.78:0/2992479765 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8804065ed0 0x7f8804068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:43.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.923+0000 7f8815ffb700 1 -- 172.21.15.78:0/2992479765 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5882+0+0 (secure 0 0 0) 0x7f8818083d00 con 0x7f882015d380 2023-10-11T09:52:43.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.923+0000 7f882505b700 1 --2- 172.21.15.78:0/2992479765 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8804065ed0 0x7f8804068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:43.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.923+0000 7f882505b700 1 --2- 172.21.15.78:0/2992479765 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8804065ed0 0x7f8804068380 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f881c000f60 tx=0x7f881c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:43.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:43.927+0000 7f8815ffb700 1 -- 172.21.15.78:0/2992479765 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f88180513f0 con 0x7f882015d380 2023-10-11T09:52:44.120 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.111+0000 7f882605d700 1 -- 172.21.15.78:0/2992479765 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f88200ccb80 con 0x7f882015d380 2023-10-11T09:52:44.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.707+0000 7f8815ffb700 1 -- 172.21.15.78:0/2992479765 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v132) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8818054a10 con 0x7f882015d380 2023-10-11T09:52:44.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.715+0000 7f882605d700 1 -- 172.21.15.78:0/2992479765 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8804065ed0 msgr2=0x7f8804068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:44.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.715+0000 7f882605d700 1 --2- 172.21.15.78:0/2992479765 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8804065ed0 0x7f8804068380 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f881c000f60 tx=0x7f881c009040 comp rx=0 tx=0).stop 2023-10-11T09:52:44.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.715+0000 7f882605d700 1 -- 172.21.15.78:0/2992479765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f882015d380 msgr2=0x7f88201582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:44.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.715+0000 7f882605d700 1 --2- 172.21.15.78:0/2992479765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f882015d380 0x7f88201582e0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f8818007e60 tx=0x7f881800e3c0 comp rx=0 tx=0).stop 2023-10-11T09:52:44.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.715+0000 7f882605d700 1 -- 172.21.15.78:0/2992479765 shutdown_connections 2023-10-11T09:52:44.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.715+0000 7f882605d700 1 --2- 172.21.15.78:0/2992479765 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8804065ed0 0x7f8804068380 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:44.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.715+0000 7f882605d700 1 --2- 172.21.15.78:0/2992479765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f882015d380 0x7f88201582e0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:44.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.715+0000 7f882605d700 1 --2- 172.21.15.78:0/2992479765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8820071bc0 0x7f88200b1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:44.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.715+0000 7f882605d700 1 --2- 172.21.15.78:0/2992479765 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f882006bed0 0x7f8820154600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:44.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.715+0000 7f882605d700 1 -- 172.21.15.78:0/2992479765 >> 172.21.15.78:0/2992479765 conn(0x7f88200b13b0 msgr2=0x7f8820070b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:44.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.715+0000 7f882605d700 1 -- 172.21.15.78:0/2992479765 shutdown_connections 2023-10-11T09:52:44.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.715+0000 7f882605d700 1 -- 172.21.15.78:0/2992479765 wait complete. 2023-10-11T09:52:44.724 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' removed 2023-10-11T09:52:44.746 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-11T09:52:44.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.915+0000 7fb19149b700 1 Processor -- start 2023-10-11T09:52:44.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.915+0000 7fb19149b700 1 -- start start 2023-10-11T09:52:44.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.919+0000 7fb19149b700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb18c0718d0 0x7fb18c15a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:44.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.919+0000 7fb19149b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fb18c15f090 0x7fb18c15aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:44.925 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.919+0000 7fb19149b700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb18c072040 con 0x7fb18c0caf80 2023-10-11T09:52:44.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.919+0000 7fb19149b700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb18c15ec00 con 0x7fb18c15f090 2023-10-11T09:52:44.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.919+0000 7fb19149b700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c071d60 con 0x7fb18c0718d0 2023-10-11T09:52:44.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.919+0000 7fb18b7fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fb18c15f090 0x7fb18c15aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:44.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.919+0000 7fb18affd700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fb18c0caf80 0x7fb18c0711c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:39806/0 (socket says 172.21.15.78:39806) 2023-10-11T09:52:44.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.919+0000 7fb18affd700 1 -- 172.21.15.78:0/2159145914 learned_addr learned my addr 172.21.15.78:0/2159145914 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:44.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.919+0000 7fb18a7fc700 1 --2- 172.21.15.78:0/2159145914 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb18c0718d0 0x7fb18c15a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:44.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.919+0000 7fb18b7fe700 1 -- 172.21.15.78:0/2159145914 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb18c0718d0 msgr2=0x7fb18c15a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:44.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.919+0000 7fb18b7fe700 1 --2- 172.21.15.78:0/2159145914 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb18c0718d0 0x7fb18c15a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:44.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.919+0000 7fb18b7fe700 1 -- 172.21.15.78:0/2159145914 >> v1:172.21.15.133:6789/0 conn(0x7fb18c0caf80 legacy=0x7fb18c0711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:44.928 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.919+0000 7fb18b7fe700 1 -- 172.21.15.78:0/2159145914 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb18c15e5b0 con 0x7fb18c15f090 2023-10-11T09:52:44.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.919+0000 7fb18b7fe700 1 --2- 172.21.15.78:0/2159145914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb18c15f090 0x7fb18c15aba0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fb184009d90 tx=0x7fb18400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac019df08c80855 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:44.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.919+0000 7fb189ffb700 1 -- 172.21.15.78:0/2159145914 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18400e070 con 0x7fb18c15f090 2023-10-11T09:52:44.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.919+0000 7fb189ffb700 1 -- 172.21.15.78:0/2159145914 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb18400bbd0 con 0x7fb18c15f090 2023-10-11T09:52:44.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.919+0000 7fb189ffb700 1 -- 172.21.15.78:0/2159145914 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb184012430 con 0x7fb18c15f090 2023-10-11T09:52:44.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.919+0000 7fb19149b700 1 -- 172.21.15.78:0/2159145914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb18c15f090 msgr2=0x7fb18c15aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:44.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.919+0000 7fb19149b700 1 --2- 172.21.15.78:0/2159145914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb18c15f090 0x7fb18c15aba0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fb184009d90 tx=0x7fb18400b330 comp rx=0 tx=0).stop 2023-10-11T09:52:44.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.919+0000 7fb19149b700 1 -- 172.21.15.78:0/2159145914 shutdown_connections 2023-10-11T09:52:44.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.919+0000 7fb19149b700 1 --2- 172.21.15.78:0/2159145914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb18c15f090 0x7fb18c15aba0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:44.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.923+0000 7fb19149b700 1 --2- 172.21.15.78:0/2159145914 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb18c0718d0 0x7fb18c15a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:44.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.923+0000 7fb19149b700 1 -- 172.21.15.78:0/2159145914 >> 172.21.15.78:0/2159145914 conn(0x7fb18c06ae10 msgr2=0x7fb18c0b0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:44.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.923+0000 7fb19149b700 1 -- 172.21.15.78:0/2159145914 shutdown_connections 2023-10-11T09:52:44.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.923+0000 7fb19149b700 1 -- 172.21.15.78:0/2159145914 wait complete. 2023-10-11T09:52:44.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.923+0000 7fb19149b700 1 Processor -- start 2023-10-11T09:52:44.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.923+0000 7fb19149b700 1 -- start start 2023-10-11T09:52:44.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.927+0000 7fb19149b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb18c0718d0 0x7fb18c154300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:44.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.927+0000 7fb19149b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb18c0caf80 0x7fb18c159110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:44.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.927+0000 7fb19149b700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb18c15f090 0x7fb18c1581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:44.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.927+0000 7fb19149b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c0cc760 con 0x7fb18c0caf80 2023-10-11T09:52:44.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.927+0000 7fb19149b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c0b1d70 con 0x7fb18c0718d0 2023-10-11T09:52:44.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.927+0000 7fb19149b700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c0ce980 con 0x7fb18c15f090 2023-10-11T09:52:44.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.927+0000 7fb18a7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb18c0caf80 0x7fb18c159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:44.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.927+0000 7fb18a7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb18c0caf80 0x7fb18c159110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48270/0 (socket says 172.21.15.78:48270) 2023-10-11T09:52:44.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.927+0000 7fb18a7fc700 1 -- 172.21.15.78:0/3655878227 learned_addr learned my addr 172.21.15.78:0/3655878227 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:44.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.927+0000 7fb18affd700 1 --2- 172.21.15.78:0/3655878227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb18c0718d0 0x7fb18c154300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:44.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.927+0000 7fb18b7fe700 1 --2- 172.21.15.78:0/3655878227 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb18c15f090 0x7fb18c1581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.927+0000 7fb18a7fc700 1 -- 172.21.15.78:0/3655878227 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb18c15f090 msgr2=0x7fb18c1581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.927+0000 7fb18a7fc700 1 --2- 172.21.15.78:0/3655878227 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb18c15f090 0x7fb18c1581f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:44.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.927+0000 7fb18a7fc700 1 -- 172.21.15.78:0/3655878227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb18c0718d0 msgr2=0x7fb18c154300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:44.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.927+0000 7fb18a7fc700 1 --2- 172.21.15.78:0/3655878227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb18c0718d0 0x7fb18c154300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:44.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.927+0000 7fb18a7fc700 1 -- 172.21.15.78:0/3655878227 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb184009a70 con 0x7fb18c0caf80 2023-10-11T09:52:44.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.927+0000 7fb18b7fe700 1 --2- 172.21.15.78:0/3655878227 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb18c15f090 0x7fb18c1581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:44.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.927+0000 7fb18a7fc700 1 --2- 172.21.15.78:0/3655878227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb18c0caf80 0x7fb18c159110 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fb18000e760 tx=0x7fb18000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:44.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.927+0000 7fb1737fe700 1 -- 172.21.15.78:0/3655878227 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb180013610 con 0x7fb18c0caf80 2023-10-11T09:52:44.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.927+0000 7fb1737fe700 1 -- 172.21.15.78:0/3655878227 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb180013770 con 0x7fb18c0caf80 2023-10-11T09:52:44.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.927+0000 7fb1737fe700 1 -- 172.21.15.78:0/3655878227 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb180013ae0 con 0x7fb18c0caf80 2023-10-11T09:52:44.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.927+0000 7fb19149b700 1 -- 172.21.15.78:0/3655878227 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb18c0b2030 con 0x7fb18c0caf80 2023-10-11T09:52:44.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.927+0000 7fb19149b700 1 -- 172.21.15.78:0/3655878227 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb18c159650 con 0x7fb18c0caf80 2023-10-11T09:52:44.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.931+0000 7fb19149b700 1 -- 172.21.15.78:0/3655878227 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb18c06f4f0 con 0x7fb18c0caf80 2023-10-11T09:52:44.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.931+0000 7fb1737fe700 1 -- 172.21.15.78:0/3655878227 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fb180013c40 con 0x7fb18c0caf80 2023-10-11T09:52:44.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.935+0000 7fb1737fe700 1 --2- 172.21.15.78:0/3655878227 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb178065ed0 0x7fb178068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:44.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.935+0000 7fb1737fe700 1 -- 172.21.15.78:0/3655878227 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fb180083270 con 0x7fb18c0caf80 2023-10-11T09:52:44.941 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.935+0000 7fb18affd700 1 --2- 172.21.15.78:0/3655878227 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb178065ed0 0x7fb178068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:44.942 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.935+0000 7fb18affd700 1 --2- 172.21.15.78:0/3655878227 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb178065ed0 0x7fb178068380 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb17c005930 tx=0x7fb17c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:44.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:44.939+0000 7fb1737fe700 1 -- 172.21.15.78:0/3655878227 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fb180050b00 con 0x7fb18c0caf80 2023-10-11T09:52:45.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.123+0000 7fb19149b700 1 -- 172.21.15.78:0/3655878227 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb18c154900 con 0x7fb18c0caf80 2023-10-11T09:52:45.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.131+0000 7fb1737fe700 1 -- 172.21.15.78:0/3655878227 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb180054120 con 0x7fb18c0caf80 2023-10-11T09:52:45.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.131+0000 7fb19149b700 1 -- 172.21.15.78:0/3655878227 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb178065ed0 msgr2=0x7fb178068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:45.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.131+0000 7fb19149b700 1 --2- 172.21.15.78:0/3655878227 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb178065ed0 0x7fb178068380 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb17c005930 tx=0x7fb17c009040 comp rx=0 tx=0).stop 2023-10-11T09:52:45.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.131+0000 7fb19149b700 1 -- 172.21.15.78:0/3655878227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb18c0caf80 msgr2=0x7fb18c159110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:45.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.131+0000 7fb19149b700 1 --2- 172.21.15.78:0/3655878227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb18c0caf80 0x7fb18c159110 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fb18000e760 tx=0x7fb18000eb20 comp rx=0 tx=0).stop 2023-10-11T09:52:45.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.131+0000 7fb19149b700 1 -- 172.21.15.78:0/3655878227 shutdown_connections 2023-10-11T09:52:45.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.131+0000 7fb19149b700 1 --2- 172.21.15.78:0/3655878227 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb178065ed0 0x7fb178068380 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:45.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.131+0000 7fb19149b700 1 --2- 172.21.15.78:0/3655878227 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb18c15f090 0x7fb18c1581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:45.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.131+0000 7fb19149b700 1 --2- 172.21.15.78:0/3655878227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb18c0caf80 0x7fb18c159110 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:45.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.131+0000 7fb19149b700 1 --2- 172.21.15.78:0/3655878227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb18c0718d0 0x7fb18c154300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:45.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.131+0000 7fb19149b700 1 -- 172.21.15.78:0/3655878227 >> 172.21.15.78:0/3655878227 conn(0x7fb18c06ae10 msgr2=0x7fb18c15c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:45.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.131+0000 7fb19149b700 1 -- 172.21.15.78:0/3655878227 shutdown_connections 2023-10-11T09:52:45.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.135+0000 7fb19149b700 1 -- 172.21.15.78:0/3655878227 wait complete. 2023-10-11T09:52:45.161 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_data_scan.TestDataScan.test_fragmented_injection' 2023-10-11T09:52:45.336 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.327+0000 7fbe42028700 1 Processor -- start 2023-10-11T09:52:45.337 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.331+0000 7fbe42028700 1 -- start start 2023-10-11T09:52:45.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.331+0000 7fbe42028700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe3c0ce700 0x7fbe3c0c20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:45.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.331+0000 7fbe42028700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe3c0cb220 con 0x7fbe3c071720 2023-10-11T09:52:45.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.331+0000 7fbe42028700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe3c1613f0 con 0x7fbe3c161020 2023-10-11T09:52:45.339 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.331+0000 7fbe42028700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c071c70 con 0x7fbe3c0ce700 2023-10-11T09:52:45.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.331+0000 7fbe41827700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fbe3c071720 0x7fbe3c06bc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:49708/0 (socket says 172.21.15.78:49708) 2023-10-11T09:52:45.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.331+0000 7fbe41827700 1 -- 172.21.15.78:0/1942411153 learned_addr learned my addr 172.21.15.78:0/1942411153 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:45.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.331+0000 7fbe40825700 1 --2- 172.21.15.78:0/1942411153 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe3c0ce700 0x7fbe3c0c20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:45.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.331+0000 7fbe33fff700 1 -- 172.21.15.78:0/1942411153 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2848838895 0 0) 0x7fbe3c0cb220 con 0x7fbe3c071720 2023-10-11T09:52:45.341 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.331+0000 7fbe33fff700 1 -- 172.21.15.78:0/1942411153 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe280035d0 con 0x7fbe3c071720 2023-10-11T09:52:45.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.331+0000 7fbe33fff700 1 -- 172.21.15.78:0/1942411153 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 597276967 0 0) 0x7fbe3c1613f0 con 0x7fbe3c161020 2023-10-11T09:52:45.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.331+0000 7fbe33fff700 1 -- 172.21.15.78:0/1942411153 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe3c0cb220 con 0x7fbe3c161020 2023-10-11T09:52:45.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.335+0000 7fbe40825700 1 -- 172.21.15.78:0/1942411153 >> v1:172.21.15.133:6789/0 conn(0x7fbe3c161020 legacy=0x7fbe3c0cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:45.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.335+0000 7fbe40825700 1 -- 172.21.15.78:0/1942411153 >> v1:172.21.15.78:6789/0 conn(0x7fbe3c071720 legacy=0x7fbe3c06bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:45.342 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.335+0000 7fbe33fff700 1 -- 172.21.15.78:0/1942411153 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2754696996 0 0) 0x7fbe280035d0 con 0x7fbe3c071720 2023-10-11T09:52:45.343 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.335+0000 7fbe40825700 1 -- 172.21.15.78:0/1942411153 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3c0ca650 con 0x7fbe3c0ce700 2023-10-11T09:52:45.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.335+0000 7fbe40825700 1 --2- 172.21.15.78:0/1942411153 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe3c0ce700 0x7fbe3c0c20c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c009870 tx=0x7fbe2c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10532fc949803f25 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:45.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.335+0000 7fbe33fff700 1 -- 172.21.15.78:0/1942411153 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe340019f0 con 0x7fbe3c0ce700 2023-10-11T09:52:45.344 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.335+0000 7fbe33fff700 1 -- 172.21.15.78:0/1942411153 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2c004030 con 0x7fbe3c0ce700 2023-10-11T09:52:45.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.335+0000 7fbe33fff700 1 -- 172.21.15.78:0/1942411153 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c0043a0 con 0x7fbe3c0ce700 2023-10-11T09:52:45.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.335+0000 7fbe42028700 1 -- 172.21.15.78:0/1942411153 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe3c0ce700 msgr2=0x7fbe3c0c20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:45.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.335+0000 7fbe42028700 1 --2- 172.21.15.78:0/1942411153 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe3c0ce700 0x7fbe3c0c20c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c009870 tx=0x7fbe2c009b80 comp rx=0 tx=0).stop 2023-10-11T09:52:45.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.335+0000 7fbe42028700 1 -- 172.21.15.78:0/1942411153 shutdown_connections 2023-10-11T09:52:45.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.335+0000 7fbe42028700 1 --2- 172.21.15.78:0/1942411153 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe3c0ce700 0x7fbe3c0c20c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:45.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.335+0000 7fbe42028700 1 -- 172.21.15.78:0/1942411153 >> 172.21.15.78:0/1942411153 conn(0x7fbe3c0cf820 msgr2=0x7fbe3c0cb610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:45.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.335+0000 7fbe42028700 1 -- 172.21.15.78:0/1942411153 shutdown_connections 2023-10-11T09:52:45.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.335+0000 7fbe42028700 1 -- 172.21.15.78:0/1942411153 wait complete. 2023-10-11T09:52:45.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.339+0000 7fbe42028700 1 Processor -- start 2023-10-11T09:52:45.347 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.339+0000 7fbe42028700 1 -- start start 2023-10-11T09:52:45.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.339+0000 7fbe42028700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe3c071720 0x7fbe3c15b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:45.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.339+0000 7fbe42028700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe3c0ce700 0x7fbe3c0c3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:45.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.339+0000 7fbe42028700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe3c161020 0x7fbe3c06f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:45.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.339+0000 7fbe42028700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c15b8e0 con 0x7fbe3c071720 2023-10-11T09:52:45.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.339+0000 7fbe42028700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c0c39b0 con 0x7fbe3c0ce700 2023-10-11T09:52:45.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.339+0000 7fbe42028700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c0c3c90 con 0x7fbe3c161020 2023-10-11T09:52:45.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.339+0000 7fbe41026700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe3c071720 0x7fbe3c15b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:45.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.343+0000 7fbe41026700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe3c071720 0x7fbe3c15b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48272/0 (socket says 172.21.15.78:48272) 2023-10-11T09:52:45.350 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.343+0000 7fbe41026700 1 -- 172.21.15.78:0/3310856635 learned_addr learned my addr 172.21.15.78:0/3310856635 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:45.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.343+0000 7fbe40825700 1 --2- 172.21.15.78:0/3310856635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe3c0ce700 0x7fbe3c0c3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:45.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.343+0000 7fbe41827700 1 --2- 172.21.15.78:0/3310856635 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe3c161020 0x7fbe3c06f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:45.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.343+0000 7fbe41026700 1 -- 172.21.15.78:0/3310856635 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe3c161020 msgr2=0x7fbe3c06f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:45.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.343+0000 7fbe41026700 1 --2- 172.21.15.78:0/3310856635 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe3c161020 0x7fbe3c06f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:45.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.343+0000 7fbe41026700 1 -- 172.21.15.78:0/3310856635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe3c0ce700 msgr2=0x7fbe3c0c3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:45.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.343+0000 7fbe41026700 1 --2- 172.21.15.78:0/3310856635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe3c0ce700 0x7fbe3c0c3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:45.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.343+0000 7fbe41026700 1 -- 172.21.15.78:0/3310856635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe2c009450 con 0x7fbe3c071720 2023-10-11T09:52:45.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.343+0000 7fbe41827700 1 --2- 172.21.15.78:0/3310856635 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe3c161020 0x7fbe3c06f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:52:45.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.343+0000 7fbe40825700 1 --2- 172.21.15.78:0/3310856635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe3c0ce700 0x7fbe3c0c3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:52:45.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.343+0000 7fbe41026700 1 --2- 172.21.15.78:0/3310856635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe3c071720 0x7fbe3c15b3a0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fbe3800c450 tx=0x7fbe3800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:45.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.343+0000 7fbe31ffb700 1 -- 172.21.15.78:0/3310856635 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe380052d0 con 0x7fbe3c071720 2023-10-11T09:52:45.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.343+0000 7fbe31ffb700 1 -- 172.21.15.78:0/3310856635 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe38005450 con 0x7fbe3c071720 2023-10-11T09:52:45.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.343+0000 7fbe31ffb700 1 -- 172.21.15.78:0/3310856635 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe380038e0 con 0x7fbe3c071720 2023-10-11T09:52:45.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.343+0000 7fbe42028700 1 -- 172.21.15.78:0/3310856635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe3c0c3f70 con 0x7fbe3c071720 2023-10-11T09:52:45.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.343+0000 7fbe42028700 1 -- 172.21.15.78:0/3310856635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe3c0c4530 con 0x7fbe3c071720 2023-10-11T09:52:45.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.343+0000 7fbe42028700 1 -- 172.21.15.78:0/3310856635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe3c067d00 con 0x7fbe3c071720 2023-10-11T09:52:45.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.347+0000 7fbe31ffb700 1 -- 172.21.15.78:0/3310856635 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fbe38003a40 con 0x7fbe3c071720 2023-10-11T09:52:45.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.347+0000 7fbe31ffb700 1 --2- 172.21.15.78:0/3310856635 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbe20065ed0 0x7fbe20068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:45.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.347+0000 7fbe31ffb700 1 -- 172.21.15.78:0/3310856635 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fbe38082b10 con 0x7fbe3c071720 2023-10-11T09:52:45.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.347+0000 7fbe40825700 1 --2- 172.21.15.78:0/3310856635 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbe20065ed0 0x7fbe20068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:45.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.351+0000 7fbe40825700 1 --2- 172.21.15.78:0/3310856635 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbe20065ed0 0x7fbe20068380 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c009870 tx=0x7fbe2c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:45.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.355+0000 7fbe31ffb700 1 -- 172.21.15.78:0/3310856635 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fbe38050310 con 0x7fbe3c071720 2023-10-11T09:52:45.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.599+0000 7fbe42028700 1 -- 172.21.15.78:0/3310856635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_data_scan.TestDataScan.test_fragmented_injection"]} v 0) v1 -- 0x7fbe3c0cab80 con 0x7fbe3c071720 2023-10-11T09:52:45.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.715+0000 7fbe31ffb700 1 -- 172.21.15.78:0/3310856635 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_data_scan.TestDataScan.test_fragmented_injection"]}]=0 v545) v1 ==== 145+0+0 (secure 0 0 0) 0x7fbe380170d0 con 0x7fbe3c071720 2023-10-11T09:52:45.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.719+0000 7fbe42028700 1 -- 172.21.15.78:0/3310856635 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbe20065ed0 msgr2=0x7fbe20068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:45.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.719+0000 7fbe42028700 1 --2- 172.21.15.78:0/3310856635 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbe20065ed0 0x7fbe20068380 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c009870 tx=0x7fbe2c008030 comp rx=0 tx=0).stop 2023-10-11T09:52:45.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.719+0000 7fbe42028700 1 -- 172.21.15.78:0/3310856635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe3c071720 msgr2=0x7fbe3c15b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:45.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.719+0000 7fbe42028700 1 --2- 172.21.15.78:0/3310856635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe3c071720 0x7fbe3c15b3a0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fbe3800c450 tx=0x7fbe3800c810 comp rx=0 tx=0).stop 2023-10-11T09:52:45.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.719+0000 7fbe42028700 1 -- 172.21.15.78:0/3310856635 shutdown_connections 2023-10-11T09:52:45.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.719+0000 7fbe42028700 1 --2- 172.21.15.78:0/3310856635 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbe20065ed0 0x7fbe20068380 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:45.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.719+0000 7fbe42028700 1 --2- 172.21.15.78:0/3310856635 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe3c161020 0x7fbe3c06f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:45.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.719+0000 7fbe42028700 1 --2- 172.21.15.78:0/3310856635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe3c0ce700 0x7fbe3c0c3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:45.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.719+0000 7fbe42028700 1 --2- 172.21.15.78:0/3310856635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe3c071720 0x7fbe3c15b3a0 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:45.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.719+0000 7fbe42028700 1 -- 172.21.15.78:0/3310856635 >> 172.21.15.78:0/3310856635 conn(0x7fbe3c0cf820 msgr2=0x7fbe3c0cd520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:45.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.719+0000 7fbe42028700 1 -- 172.21.15.78:0/3310856635 shutdown_connections 2023-10-11T09:52:45.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:45.719+0000 7fbe42028700 1 -- 172.21.15.78:0/3310856635 wait complete. 2023-10-11T09:52:45.748 INFO:tasks.cephfs_test_runner:That when injecting a dentry into a fragmented directory, we put it in the right fragment. ... ERROR 2023-10-11T09:52:45.749 INFO:tasks.cephfs_test_runner: 2023-10-11T09:52:45.749 INFO:tasks.cephfs_test_runner:====================================================================== 2023-10-11T09:52:45.749 INFO:tasks.cephfs_test_runner:ERROR: test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan) 2023-10-11T09:52:45.750 INFO:tasks.cephfs_test_runner:That when injecting a dentry into a fragmented directory, we put it in the right fragment. 2023-10-11T09:52:45.750 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-11T09:52:45.750 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-11T09:52:45.750 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_9a774796f0ecdd11a4b771de396e627aa1a187af/qa/tasks/cephfs/test_data_scan.py", line 560, in test_fragmented_injection 2023-10-11T09:52:45.751 INFO:tasks.cephfs_test_runner: out = self.mount_a.run_shell_payload(f"sudo cat subdir/{victim_dentry}", omit_sudo=False).stdout.getvalue().strip() 2023-10-11T09:52:45.751 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_9a774796f0ecdd11a4b771de396e627aa1a187af/qa/tasks/cephfs/mount.py", line 789, in run_shell_payload 2023-10-11T09:52:45.751 INFO:tasks.cephfs_test_runner: return self.run_shell(**kwargs) 2023-10-11T09:52:45.751 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_9a774796f0ecdd11a4b771de396e627aa1a187af/qa/tasks/cephfs/mount.py", line 780, in run_shell 2023-10-11T09:52:45.752 INFO:tasks.cephfs_test_runner: return self.client_remote.run(args=args, cwd=cwd, timeout=timeout, 2023-10-11T09:52:45.752 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run 2023-10-11T09:52:45.752 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-10-11T09:52:45.752 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run 2023-10-11T09:52:45.753 INFO:tasks.cephfs_test_runner: r.wait() 2023-10-11T09:52:45.753 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait 2023-10-11T09:52:45.753 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-10-11T09:52:45.754 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-10-11T09:52:45.754 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-10-11T09:52:45.754 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi078 with status 1: "(cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo cat subdir/21')" 2023-10-11T09:52:45.754 INFO:tasks.cephfs_test_runner: 2023-10-11T09:52:45.755 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-11T09:52:45.755 INFO:tasks.cephfs_test_runner:Ran 5 tests in 685.625s 2023-10-11T09:52:45.755 INFO:tasks.cephfs_test_runner: 2023-10-11T09:52:45.755 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-10-11T09:52:45.756 INFO:tasks.cephfs_test_runner: 2023-10-11T09:52:45.756 INFO:tasks.cephfs_test_runner:====================================================================== 2023-10-11T09:52:45.756 INFO:tasks.cephfs_test_runner:ERROR: test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan) 2023-10-11T09:52:45.757 INFO:tasks.cephfs_test_runner:That when injecting a dentry into a fragmented directory, we put it in the right fragment. 2023-10-11T09:52:45.757 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-11T09:52:45.757 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-11T09:52:45.758 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_9a774796f0ecdd11a4b771de396e627aa1a187af/qa/tasks/cephfs/test_data_scan.py", line 560, in test_fragmented_injection 2023-10-11T09:52:45.758 INFO:tasks.cephfs_test_runner: out = self.mount_a.run_shell_payload(f"sudo cat subdir/{victim_dentry}", omit_sudo=False).stdout.getvalue().strip() 2023-10-11T09:52:45.758 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_9a774796f0ecdd11a4b771de396e627aa1a187af/qa/tasks/cephfs/mount.py", line 789, in run_shell_payload 2023-10-11T09:52:45.758 INFO:tasks.cephfs_test_runner: return self.run_shell(**kwargs) 2023-10-11T09:52:45.759 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_9a774796f0ecdd11a4b771de396e627aa1a187af/qa/tasks/cephfs/mount.py", line 780, in run_shell 2023-10-11T09:52:45.759 INFO:tasks.cephfs_test_runner: return self.client_remote.run(args=args, cwd=cwd, timeout=timeout, 2023-10-11T09:52:45.759 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run 2023-10-11T09:52:45.759 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-10-11T09:52:45.760 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run 2023-10-11T09:52:45.760 INFO:tasks.cephfs_test_runner: r.wait() 2023-10-11T09:52:45.760 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait 2023-10-11T09:52:45.760 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-10-11T09:52:45.761 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-10-11T09:52:45.761 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-10-11T09:52:45.761 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi078 with status 1: "(cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo cat subdir/21')" 2023-10-11T09:52:45.761 INFO:tasks.cephfs_test_runner: 2023-10-11T09:52:45.762 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_9a774796f0ecdd11a4b771de396e627aa1a187af/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan) 2023-10-11T09:52:47.026 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=4ea49fd8c9cb4ec6a0f0a5c2e48d7150 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_9a774796f0ecdd11a4b771de396e627aa1a187af/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan) 2023-10-11T09:52:47.028 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-11T09:52:47.045 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-10-11T09:52:47.059 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-10-11T09:52:47.060 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:52:47.060 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:52:47.089 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352272k,nr_inodes=4088068,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17487 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-11T09:52:47.089 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-11T09:52:47.090 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:52:47.150 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352272k,nr_inodes=4088068,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17487 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-11T09:52:47.151 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:52:47.151 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:52:47.173 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352292k,nr_inodes=4088073,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1430 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275376k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-11T09:52:47.173 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-10-11T09:52:47.174 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T09:52:47.230 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352292k,nr_inodes=4088073,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1430 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275376k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275380k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-11T09:52:47.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:47.231 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2023-10-11T09:52:47.293 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-11T09:52:47.293 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-11T09:52:47.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:47.295 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-11T09:52:47.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:47.374 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-11T09:52:47.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:47.451 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link delete ceph-brx 2023-10-11T09:52:47.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:47.650 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2023-10-11T09:52:47.670 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-11T09:52:47.671 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-11T09:52:47.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:47.672 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-11T09:52:47.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:47.753 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-11T09:52:47.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T09:52:47.848 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link delete ceph-brx 2023-10-11T09:52:48.036 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-11T09:52:48.046 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-11T09:52:48.046 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-11T09:52:48.221 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.215+0000 7fd55d69b700 1 Processor -- start 2023-10-11T09:52:48.222 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.215+0000 7fd55d69b700 1 -- start start 2023-10-11T09:52:48.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.215+0000 7fd55d69b700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd5580cea30 0x7fd5580ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:48.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.215+0000 7fd55d69b700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5580cb280 con 0x7fd55815a720 2023-10-11T09:52:48.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.215+0000 7fd55d69b700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd55806abd0 con 0x7fd5580cb710 2023-10-11T09:52:48.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.215+0000 7fd55d69b700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5580cf3f0 con 0x7fd5580cea30 2023-10-11T09:52:48.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.215+0000 7fd556ffd700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fd55815a720 0x7fd55806a180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:49724/0 (socket says 172.21.15.78:49724) 2023-10-11T09:52:48.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.215+0000 7fd556ffd700 1 -- 172.21.15.78:0/2832534279 learned_addr learned my addr 172.21.15.78:0/2832534279 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:48.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.215+0000 7fd5567fc700 1 --2- 172.21.15.78:0/2832534279 >> v2:172.21.15.133:3300/0 conn(0x7fd5580cea30 0x7fd5580ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:48.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.215+0000 7fd555ffb700 1 -- 172.21.15.78:0/2832534279 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1673352352 0 0) 0x7fd5580cb280 con 0x7fd55815a720 2023-10-11T09:52:48.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.215+0000 7fd555ffb700 1 -- 172.21.15.78:0/2832534279 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5400035d0 con 0x7fd55815a720 2023-10-11T09:52:48.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.219+0000 7fd555ffb700 1 -- 172.21.15.78:0/2832534279 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3555183782 0 0) 0x7fd5400035d0 con 0x7fd55815a720 2023-10-11T09:52:48.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.219+0000 7fd555ffb700 1 -- 172.21.15.78:0/2832534279 >> v2:172.21.15.133:3300/0 conn(0x7fd5580cea30 msgr2=0x7fd5580ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:48.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.219+0000 7fd555ffb700 1 --2- 172.21.15.78:0/2832534279 >> v2:172.21.15.133:3300/0 conn(0x7fd5580cea30 0x7fd5580ceeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:48.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.219+0000 7fd555ffb700 1 -- 172.21.15.78:0/2832534279 >> v1:172.21.15.133:6789/0 conn(0x7fd5580cb710 legacy=0x7fd5580cbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:48.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.219+0000 7fd555ffb700 1 -- 172.21.15.78:0/2832534279 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd55806aeb0 con 0x7fd55815a720 2023-10-11T09:52:48.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.219+0000 7fd555ffb700 1 -- 172.21.15.78:0/2832534279 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7fd548001670 con 0x7fd55815a720 2023-10-11T09:52:48.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.219+0000 7fd555ffb700 1 -- 172.21.15.78:0/2832534279 >> v1:172.21.15.78:6789/0 conn(0x7fd55815a720 legacy=0x7fd55806a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:48.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.219+0000 7fd555ffb700 1 --2- 172.21.15.78:0/2832534279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd540003a70 0x7fd540003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:48.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.219+0000 7fd555ffb700 1 --2- 172.21.15.78:0/2832534279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd540004870 0x7fd540004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:48.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.219+0000 7fd555ffb700 1 --2- 172.21.15.78:0/2832534279 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd540005220 0x7fd54000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:48.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.219+0000 7fd555ffb700 1 -- 172.21.15.78:0/2832534279 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5580cf3f0 con 0x7fd540004870 2023-10-11T09:52:48.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.219+0000 7fd555ffb700 1 -- 172.21.15.78:0/2832534279 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54000c100 con 0x7fd540003a70 2023-10-11T09:52:48.234 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.219+0000 7fd555ffb700 1 -- 172.21.15.78:0/2832534279 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd54000c2c0 con 0x7fd540005220 2023-10-11T09:52:48.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.219+0000 7fd5567fc700 1 --2- 172.21.15.78:0/2832534279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd540004870 0x7fd540004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:48.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.219+0000 7fd556ffd700 1 --2- 172.21.15.78:0/2832534279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd540003a70 0x7fd540003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:48.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.219+0000 7fd557fff700 1 --2- 172.21.15.78:0/2832534279 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd540005220 0x7fd54000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:48.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.219+0000 7fd5567fc700 1 -- 172.21.15.78:0/2832534279 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd540005220 msgr2=0x7fd54000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:48.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.219+0000 7fd5567fc700 1 --2- 172.21.15.78:0/2832534279 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd540005220 0x7fd54000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:48.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.219+0000 7fd5567fc700 1 -- 172.21.15.78:0/2832534279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd540003a70 msgr2=0x7fd540003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:48.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.219+0000 7fd5567fc700 1 --2- 172.21.15.78:0/2832534279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd540003a70 0x7fd540003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:48.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.219+0000 7fd5567fc700 1 -- 172.21.15.78:0/2832534279 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5400035d0 con 0x7fd540004870 2023-10-11T09:52:48.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.219+0000 7fd557fff700 1 -- 172.21.15.78:0/2832534279 reap_dead start 2023-10-11T09:52:48.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.219+0000 7fd5567fc700 1 --2- 172.21.15.78:0/2832534279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd540004870 0x7fd540004ce0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fd54c009b40 tx=0x7fd5400058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b18f60b7b205574 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:48.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.219+0000 7fd555ffb700 1 -- 172.21.15.78:0/2832534279 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54c00e6c0 con 0x7fd540004870 2023-10-11T09:52:48.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.219+0000 7fd555ffb700 1 -- 172.21.15.78:0/2832534279 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd54c00e820 con 0x7fd540004870 2023-10-11T09:52:48.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.219+0000 7fd555ffb700 1 -- 172.21.15.78:0/2832534279 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54c00eb90 con 0x7fd540004870 2023-10-11T09:52:48.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.219+0000 7fd55d69b700 1 -- 172.21.15.78:0/2832534279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd540004870 msgr2=0x7fd540004ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:48.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.219+0000 7fd55d69b700 1 --2- 172.21.15.78:0/2832534279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd540004870 0x7fd540004ce0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fd54c009b40 tx=0x7fd5400058a0 comp rx=0 tx=0).stop 2023-10-11T09:52:48.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.219+0000 7fd55d69b700 1 -- 172.21.15.78:0/2832534279 shutdown_connections 2023-10-11T09:52:48.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.219+0000 7fd55d69b700 1 --2- 172.21.15.78:0/2832534279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd540004870 0x7fd540004ce0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:48.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.219+0000 7fd55d69b700 1 -- 172.21.15.78:0/2832534279 >> 172.21.15.78:0/2832534279 conn(0x7fd55806c5a0 msgr2=0x7fd5580d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:48.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.219+0000 7fd55d69b700 1 -- 172.21.15.78:0/2832534279 shutdown_connections 2023-10-11T09:52:48.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.219+0000 7fd55d69b700 1 -- 172.21.15.78:0/2832534279 wait complete. 2023-10-11T09:52:48.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.223+0000 7fd55d69b700 1 Processor -- start 2023-10-11T09:52:48.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.223+0000 7fd55d69b700 1 -- start start 2023-10-11T09:52:48.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.227+0000 7fd55d69b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd540004870 0x7fd5580b4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:48.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.227+0000 7fd55d69b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5581514a0 0x7fd5581f3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:48.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.227+0000 7fd55d69b700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd5580ae9e0 0x7fd5580aee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:48.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.227+0000 7fd55d69b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5580af390 con 0x7fd5581514a0 2023-10-11T09:52:48.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.227+0000 7fd55d69b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5580b4790 con 0x7fd540004870 2023-10-11T09:52:48.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.227+0000 7fd55d69b700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5581f43d0 con 0x7fd5580ae9e0 2023-10-11T09:52:48.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.227+0000 7fd5567fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5581514a0 0x7fd5581f3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:48.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.227+0000 7fd5567fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5581514a0 0x7fd5581f3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48292/0 (socket says 172.21.15.78:48292) 2023-10-11T09:52:48.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.227+0000 7fd5567fc700 1 -- 172.21.15.78:0/2053462347 learned_addr learned my addr 172.21.15.78:0/2053462347 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:48.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.227+0000 7fd557fff700 1 --2- 172.21.15.78:0/2053462347 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd5580ae9e0 0x7fd5580aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:48.248 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.227+0000 7fd556ffd700 1 --2- 172.21.15.78:0/2053462347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd540004870 0x7fd5580b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:48.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.227+0000 7fd5567fc700 1 -- 172.21.15.78:0/2053462347 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd5580ae9e0 msgr2=0x7fd5580aee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:48.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.227+0000 7fd5567fc700 1 --2- 172.21.15.78:0/2053462347 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd5580ae9e0 0x7fd5580aee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:48.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.227+0000 7fd5567fc700 1 -- 172.21.15.78:0/2053462347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd540004870 msgr2=0x7fd5580b4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:48.249 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.227+0000 7fd5567fc700 1 --2- 172.21.15.78:0/2053462347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd540004870 0x7fd5580b4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:48.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.227+0000 7fd5567fc700 1 -- 172.21.15.78:0/2053462347 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd55815cab0 con 0x7fd5581514a0 2023-10-11T09:52:48.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.227+0000 7fd5567fc700 1 --2- 172.21.15.78:0/2053462347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5581514a0 0x7fd5581f3e90 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fd54c0090b0 tx=0x7fd54c0047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:48.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.227+0000 7fd53f7fe700 1 -- 172.21.15.78:0/2053462347 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54c00e6c0 con 0x7fd5581514a0 2023-10-11T09:52:48.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.227+0000 7fd53f7fe700 1 -- 172.21.15.78:0/2053462347 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd54c003ad0 con 0x7fd5581514a0 2023-10-11T09:52:48.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.227+0000 7fd55d69b700 1 -- 172.21.15.78:0/2053462347 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd55815cd90 con 0x7fd5581514a0 2023-10-11T09:52:48.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.227+0000 7fd53f7fe700 1 -- 172.21.15.78:0/2053462347 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54c004610 con 0x7fd5581514a0 2023-10-11T09:52:48.251 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.227+0000 7fd55d69b700 1 -- 172.21.15.78:0/2053462347 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd55815d350 con 0x7fd5581514a0 2023-10-11T09:52:48.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.231+0000 7fd55d69b700 1 -- 172.21.15.78:0/2053462347 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd55815d510 con 0x7fd5581514a0 2023-10-11T09:52:48.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.231+0000 7fd53f7fe700 1 -- 172.21.15.78:0/2053462347 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fd54c013470 con 0x7fd5581514a0 2023-10-11T09:52:48.252 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.231+0000 7fd53f7fe700 1 --2- 172.21.15.78:0/2053462347 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd544065f00 0x7fd5440683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:48.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.235+0000 7fd53f7fe700 1 -- 172.21.15.78:0/2053462347 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fd54c0851b0 con 0x7fd5581514a0 2023-10-11T09:52:48.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.235+0000 7fd556ffd700 1 --2- 172.21.15.78:0/2053462347 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd544065f00 0x7fd5440683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:48.253 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.235+0000 7fd556ffd700 1 --2- 172.21.15.78:0/2053462347 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd544065f00 0x7fd5440683b0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd548001eb0 tx=0x7fd548009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:48.254 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.235+0000 7fd53f7fe700 1 -- 172.21.15.78:0/2053462347 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fd54c052b60 con 0x7fd5581514a0 2023-10-11T09:52:48.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.407+0000 7fd55d69b700 1 -- 172.21.15.78:0/2053462347 --> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd55806b470 con 0x7fd544065f00 2023-10-11T09:52:48.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.415+0000 7fd53f7fe700 1 -- 172.21.15.78:0/2053462347 <== mgr.4101 v2:172.21.15.78:6832/35606 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70696 (secure 0 0 0) 0x7fd548001550 con 0x7fd544065f00 2023-10-11T09:52:48.423 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:52:48.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.419+0000 7fd55d69b700 1 -- 172.21.15.78:0/2053462347 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd544065f00 msgr2=0x7fd5440683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:48.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.419+0000 7fd55d69b700 1 --2- 172.21.15.78:0/2053462347 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd544065f00 0x7fd5440683b0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd548001eb0 tx=0x7fd548009040 comp rx=0 tx=0).stop 2023-10-11T09:52:48.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.419+0000 7fd55d69b700 1 -- 172.21.15.78:0/2053462347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5581514a0 msgr2=0x7fd5581f3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:48.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.419+0000 7fd55d69b700 1 --2- 172.21.15.78:0/2053462347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5581514a0 0x7fd5581f3e90 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fd54c0090b0 tx=0x7fd54c0047e0 comp rx=0 tx=0).stop 2023-10-11T09:52:48.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.419+0000 7fd55d69b700 1 -- 172.21.15.78:0/2053462347 shutdown_connections 2023-10-11T09:52:48.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.419+0000 7fd55d69b700 1 --2- 172.21.15.78:0/2053462347 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd544065f00 0x7fd5440683b0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:48.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.419+0000 7fd55d69b700 1 --2- 172.21.15.78:0/2053462347 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd5580ae9e0 0x7fd5580aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:48.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.419+0000 7fd55d69b700 1 --2- 172.21.15.78:0/2053462347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5581514a0 0x7fd5581f3e90 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:48.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.419+0000 7fd55d69b700 1 --2- 172.21.15.78:0/2053462347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd540004870 0x7fd5580b4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:48.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.419+0000 7fd55d69b700 1 -- 172.21.15.78:0/2053462347 >> 172.21.15.78:0/2053462347 conn(0x7fd55806c5a0 msgr2=0x7fd5580cbb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:48.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.419+0000 7fd55d69b700 1 -- 172.21.15.78:0/2053462347 shutdown_connections 2023-10-11T09:52:48.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.419+0000 7fd55d69b700 1 -- 172.21.15.78:0/2053462347 wait complete. 2023-10-11T09:52:48.431 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2023-10-11T09:52:48.449 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":494,"stamp":"2023-10-11T09:52:46.585444+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":263808,"kb_used_data":15424,"kb_used_omap":0,"kb_used_meta":248320,"kb_avail":754710912,"statfs":{"total":773094113280,"available":772823973888,"internally_reserved":0,"allocated":15794176,"data_stored":8297818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":254279680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-407665,"num_objects":-125,"num_object_clones":0,"num_object_copies":-250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-125,"num_whiteouts":0,"num_read":-1808,"num_read_kb":-1515,"num_write":-914,"num_write_kb":-953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001796"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":97,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-10-11T09:52:22.849372+0000","last_change":"2023-10-11T09:40:36.151839+0000","last_active":"2023-10-11T09:52:22.849372+0000","last_peered":"2023-10-11T09:52:22.849372+0000","last_clean":"2023-10-11T09:52:22.849372+0000","last_became_active":"2023-10-11T09:40:36.151179+0000","last_became_peered":"2023-10-11T09:40:36.151179+0000","last_unstale":"2023-10-11T09:52:22.849372+0000","last_undegraded":"2023-10-11T09:52:22.849372+0000","last_fullsized":"2023-10-11T09:52:22.849372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":104,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-10-11T09:52:22.845975+0000","last_change":"2023-10-11T09:40:36.150224+0000","last_active":"2023-10-11T09:52:22.845975+0000","last_peered":"2023-10-11T09:52:22.845975+0000","last_clean":"2023-10-11T09:52:22.845975+0000","last_became_active":"2023-10-11T09:40:36.149919+0000","last_became_peered":"2023-10-11T09:40:36.149919+0000","last_unstale":"2023-10-11T09:52:22.845975+0000","last_undegraded":"2023-10-11T09:52:22.845975+0000","last_fullsized":"2023-10-11T09:52:22.845975+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":126,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-10-11T09:52:22.850757+0000","last_change":"2023-10-11T09:40:36.162174+0000","last_active":"2023-10-11T09:52:22.850757+0000","last_peered":"2023-10-11T09:52:22.850757+0000","last_clean":"2023-10-11T09:52:22.850757+0000","last_became_active":"2023-10-11T09:40:36.161710+0000","last_became_peered":"2023-10-11T09:40:36.161710+0000","last_unstale":"2023-10-11T09:52:22.850757+0000","last_undegraded":"2023-10-11T09:52:22.850757+0000","last_fullsized":"2023-10-11T09:52:22.850757+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:57:03.630878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":104,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-10-11T09:52:22.845772+0000","last_change":"2023-10-11T09:40:36.160620+0000","last_active":"2023-10-11T09:52:22.845772+0000","last_peered":"2023-10-11T09:52:22.845772+0000","last_clean":"2023-10-11T09:52:22.845772+0000","last_became_active":"2023-10-11T09:40:36.160171+0000","last_became_peered":"2023-10-11T09:40:36.160171+0000","last_unstale":"2023-10-11T09:52:22.845772+0000","last_undegraded":"2023-10-11T09:52:22.845772+0000","last_fullsized":"2023-10-11T09:52:22.845772+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":102,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-10-11T09:52:22.973429+0000","last_change":"2023-10-11T09:40:36.151710+0000","last_active":"2023-10-11T09:52:22.973429+0000","last_peered":"2023-10-11T09:52:22.973429+0000","last_clean":"2023-10-11T09:52:22.973429+0000","last_became_active":"2023-10-11T09:40:36.151219+0000","last_became_peered":"2023-10-11T09:40:36.151219+0000","last_unstale":"2023-10-11T09:52:22.973429+0000","last_undegraded":"2023-10-11T09:52:22.973429+0000","last_fullsized":"2023-10-11T09:52:22.973429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":104,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-10-11T09:52:23.000147+0000","last_change":"2023-10-11T09:40:36.216248+0000","last_active":"2023-10-11T09:52:23.000147+0000","last_peered":"2023-10-11T09:52:23.000147+0000","last_clean":"2023-10-11T09:52:23.000147+0000","last_became_active":"2023-10-11T09:40:36.215890+0000","last_became_peered":"2023-10-11T09:40:36.215890+0000","last_unstale":"2023-10-11T09:52:23.000147+0000","last_undegraded":"2023-10-11T09:52:23.000147+0000","last_fullsized":"2023-10-11T09:52:23.000147+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":127,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-10-11T09:52:38.571342+0000","last_change":"2023-10-11T09:40:36.149739+0000","last_active":"2023-10-11T09:52:38.571342+0000","last_peered":"2023-10-11T09:52:38.571342+0000","last_clean":"2023-10-11T09:52:38.571342+0000","last_became_active":"2023-10-11T09:40:36.149419+0000","last_became_peered":"2023-10-11T09:40:36.149419+0000","last_unstale":"2023-10-11T09:52:38.571342+0000","last_undegraded":"2023-10-11T09:52:38.571342+0000","last_fullsized":"2023-10-11T09:52:38.571342+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:54:50.640362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":102,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-10-11T09:52:22.973327+0000","last_change":"2023-10-11T09:40:36.148773+0000","last_active":"2023-10-11T09:52:22.973327+0000","last_peered":"2023-10-11T09:52:22.973327+0000","last_clean":"2023-10-11T09:52:22.973327+0000","last_became_active":"2023-10-11T09:40:36.148489+0000","last_became_peered":"2023-10-11T09:40:36.148489+0000","last_unstale":"2023-10-11T09:52:22.973327+0000","last_undegraded":"2023-10-11T09:52:22.973327+0000","last_fullsized":"2023-10-11T09:52:22.973327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":280,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-10-11T09:52:33.570303+0000","last_change":"2023-10-11T09:40:29.874372+0000","last_active":"2023-10-11T09:52:33.570303+0000","last_peered":"2023-10-11T09:52:33.570303+0000","last_clean":"2023-10-11T09:52:33.570303+0000","last_became_active":"2023-10-11T09:40:29.873621+0000","last_became_peered":"2023-10-11T09:40:29.873621+0000","last_unstale":"2023-10-11T09:52:33.570303+0000","last_undegraded":"2023-10-11T09:52:33.570303+0000","last_fullsized":"2023-10-11T09:52:33.570303+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_clean_scrub_stamp":"2023-10-11T09:40:28.742276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:22:07.985311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574999,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33328,"kb_used_data":2280,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338512,"statfs":{"total":96636764160,"available":96602636288,"internally_reserved":0,"allocated":2334720,"data_stored":1378915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.056,"15min":1.087},"min":{"1min":0.68999999999999995,"5min":0.56499999999999995,"15min":0.48999999999999999},"max":{"1min":1.4730000000000001,"5min":1.7390000000000001,"15min":1.903},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.034,"5min":1.0329999999999999,"15min":1.0860000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.50900000000000001},"max":{"1min":1.4970000000000001,"5min":1.554,"15min":1.696},"last":1.224}]},{"osd":1,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.1140000000000001,"15min":1.123},"min":{"1min":0.91000000000000003,"5min":0.66800000000000004,"15min":0.63100000000000001},"max":{"1min":1.4059999999999999,"5min":1.645,"15min":2.25},"last":1.325},{"interface":"front","average":{"1min":1.167,"5min":1.0489999999999999,"15min":1.0920000000000001},"min":{"1min":0.93300000000000005,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.4950000000000001,"5min":1.587,"15min":1.696},"last":1.167}]},{"osd":2,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.0840000000000001,"15min":1.103},"min":{"1min":0.72899999999999998,"5min":0.59499999999999997,"15min":0.58999999999999997},"max":{"1min":1.411,"5min":1.5049999999999999,"15min":1.873},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.175,"5min":1.1719999999999999,"15min":1.155},"min":{"1min":0.72499999999999998,"5min":0.66400000000000003,"15min":0.65500000000000003},"max":{"1min":1.579,"5min":1.78,"15min":1.78},"last":1.4319999999999999}]},{"osd":3,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.216,"15min":1.22},"min":{"1min":0.89200000000000002,"5min":0.80000000000000004,"15min":0.64900000000000002},"max":{"1min":1.617,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.024},{"interface":"front","average":{"1min":1.302,"5min":1.284,"15min":1.2809999999999999},"min":{"1min":0.94699999999999995,"5min":0.747,"15min":0.61699999999999999},"max":{"1min":1.54,"5min":1.8720000000000001,"15min":2.1579999999999999},"last":1.5860000000000001}]},{"osd":4,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.0609999999999999,"15min":1.1200000000000001},"min":{"1min":0.71999999999999997,"5min":0.64900000000000002,"15min":0.53300000000000003},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":2.2010000000000001},"last":1.365},{"interface":"front","average":{"1min":1.181,"5min":1.208,"15min":1.226},"min":{"1min":0.93100000000000005,"5min":0.67200000000000004,"15min":0.63500000000000001},"max":{"1min":1.4199999999999999,"5min":1.768,"15min":1.9339999999999999},"last":0.86899999999999999}]},{"osd":5,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.2709999999999999,"15min":1.278},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.7350000000000001,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2050000000000001,"15min":1.2749999999999999},"min":{"1min":0.82999999999999996,"5min":0.78300000000000003,"15min":0.65900000000000003},"max":{"1min":1.714,"5min":1.9450000000000001,"15min":3.0750000000000002},"last":0.83899999999999997}]},{"osd":6,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.3300000000000001,"15min":1.3919999999999999},"min":{"1min":0.98599999999999999,"5min":0.77000000000000002,"15min":0.70999999999999996},"max":{"1min":2,"5min":2,"15min":3.137},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3500000000000001,"15min":1.381},"min":{"1min":0.89700000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.897,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.395}]}]},{"osd":6,"up_from":13,"seq":55834574998,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32736,"kb_used_data":1688,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1728512,"data_stored":788353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:51:44 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":1.113,"15min":1.0369999999999999},"min":{"1min":0.64500000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.204,"5min":7.0730000000000004,"15min":7.0730000000000004},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.1819999999999999,"15min":1.0740000000000001},"min":{"1min":0.74299999999999999,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.4570000000000001,"5min":8.8450000000000006,"15min":8.8450000000000006},"last":1.357}]},{"osd":1,"last update":"Wed Oct 11 09:51:44 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.325,"15min":1.23},"min":{"1min":0.78400000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.6439999999999999,"5min":15.523,"15min":15.523},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.167,"5min":1.288,"15min":1.2370000000000001},"min":{"1min":0.73299999999999998,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.5149999999999999,"5min":15.253,"15min":15.253},"last":1.242}]},{"osd":2,"last update":"Wed Oct 11 09:51:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.103,"15min":1.137},"min":{"1min":0.90200000000000002,"5min":0.434,"15min":0.434},"max":{"1min":1.3280000000000001,"5min":2.7919999999999998,"15min":2.7919999999999998},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.196,"5min":1.1639999999999999,"15min":1.196},"min":{"1min":0.65500000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.4950000000000001,"5min":5.0990000000000002,"15min":5.0990000000000002},"last":1.298}]},{"osd":3,"last update":"Wed Oct 11 09:51:44 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.2050000000000001,"15min":1.1619999999999999},"min":{"1min":0.81599999999999995,"5min":0.70599999999999996,"15min":0.627},"max":{"1min":1.363,"5min":11.945,"15min":11.945},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.143,"15min":1.206},"min":{"1min":0.61499999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.536,"5min":1.5980000000000001,"15min":3.0819999999999999},"last":1.3859999999999999}]},{"osd":4,"last update":"Wed Oct 11 09:51:44 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.286,"15min":1.282},"min":{"1min":0.95299999999999996,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.581,"5min":7.5599999999999996,"15min":7.5599999999999996},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.238,"5min":1.327,"15min":1.3169999999999999},"min":{"1min":0.94599999999999995,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.444,"5min":8.7520000000000007,"15min":8.7520000000000007},"last":1.2689999999999999}]},{"osd":5,"last update":"Wed Oct 11 09:51:44 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.1890000000000001,"15min":1.232},"min":{"1min":0.67100000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.5900000000000001,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.202},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.2869999999999999,"15min":1.333},"min":{"1min":0.86199999999999999,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.635,"5min":2.4390000000000001,"15min":2.7290000000000001},"last":0.875}]},{"osd":7,"last update":"Wed Oct 11 09:51:44 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3260000000000001,"15min":1.353},"min":{"1min":0.93100000000000005,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.5549999999999999,"5min":8.7970000000000006,"15min":8.7970000000000006},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.3700000000000001,"15min":1.425},"min":{"1min":0.83699999999999997,"5min":0.625,"15min":0.625},"max":{"1min":1.611,"5min":2.641,"15min":2.641},"last":1.452}]}]},{"osd":5,"up_from":13,"seq":55834574998,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32756,"kb_used_data":1708,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339084,"statfs":{"total":96636764160,"available":96603222016,"internally_reserved":0,"allocated":1748992,"data_stored":787939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:51:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.0189999999999999,"15min":1.01},"min":{"1min":0.755,"5min":0.53400000000000003,"15min":0.45300000000000001},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.0529999999999999,"15min":1.016},"min":{"1min":0.76900000000000002,"5min":0.47699999999999998,"15min":0.46500000000000002},"max":{"1min":1.373,"5min":1.536,"15min":1.7649999999999999},"last":1.3069999999999999}]},{"osd":1,"last update":"Wed Oct 11 09:51:47 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.0960000000000001,"15min":1.022},"min":{"1min":0.79200000000000004,"5min":0.67800000000000005,"15min":0.49399999999999999},"max":{"1min":1.7190000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1200000000000001,"15min":1.085},"min":{"1min":0.69099999999999995,"5min":0.61599999999999999,"15min":0.437},"max":{"1min":1.716,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.3799999999999999}]},{"osd":2,"last update":"Wed Oct 11 09:51:47 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.80200000000000005,"5min":0.746,"15min":0.73899999999999999},"max":{"1min":1.5,"5min":2.133,"15min":2.133},"last":1.254},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.2210000000000001,"15min":1.2190000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.52300000000000002},"max":{"1min":1.665,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.446}]},{"osd":3,"last update":"Wed Oct 11 09:51:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.111,"15min":1.125},"min":{"1min":0.85699999999999998,"5min":0.59799999999999998,"15min":0.56999999999999995},"max":{"1min":1.4550000000000001,"5min":1.8859999999999999,"15min":1.921},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.167,"5min":1.171,"15min":1.163},"min":{"1min":0.71999999999999997,"5min":0.71099999999999997,"15min":0.55300000000000005},"max":{"1min":1.5369999999999999,"5min":1.9690000000000001,"15min":2.0209999999999999},"last":1.3440000000000001}]},{"osd":4,"last update":"Wed Oct 11 09:51:47 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.21,"15min":1.1930000000000001},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.8400000000000001,"5min":2.0169999999999999,"15min":2.117},"last":1.038},{"interface":"front","average":{"1min":1.22,"5min":1.181,"15min":1.1859999999999999},"min":{"1min":0.66600000000000004,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.621,"5min":1.9690000000000001,"15min":2.1749999999999998},"last":0.95199999999999996}]},{"osd":6,"last update":"Wed Oct 11 09:51:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.254,"15min":1.26},"min":{"1min":0.73799999999999999,"5min":0.69499999999999995,"15min":0.59499999999999997},"max":{"1min":1.5940000000000001,"5min":1.9239999999999999,"15min":2.4529999999999998},"last":1.089},{"interface":"front","average":{"1min":1.415,"5min":1.3420000000000001,"15min":1.323},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.044,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.28}]},{"osd":7,"last update":"Wed Oct 11 09:51:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.339,"15min":1.3520000000000001},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.57699999999999996},"max":{"1min":1.728,"5min":2.347,"15min":3.6120000000000001},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.4390000000000001,"15min":1.4370000000000001},"min":{"1min":1.069,"5min":0.89100000000000001,"15min":0.71299999999999997},"max":{"1min":1.8859999999999999,"5min":2.2679999999999998,"15min":3.2109999999999999},"last":1.4179999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574998,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33124,"kb_used_data":2076,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338716,"statfs":{"total":96636764160,"available":96602845184,"internally_reserved":0,"allocated":2125824,"data_stored":1194438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.071,"15min":1.0640000000000001},"min":{"1min":0.74399999999999999,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":2.0369999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.417},{"interface":"front","average":{"1min":1.133,"5min":1.0760000000000001,"15min":1.0880000000000001},"min":{"1min":0.90000000000000002,"5min":0.60199999999999998,"15min":0.54700000000000004},"max":{"1min":1.5129999999999999,"5min":1.669,"15min":2.1160000000000001},"last":1.2769999999999999}]},{"osd":1,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.109,"15min":1.089},"min":{"1min":0.72699999999999998,"5min":0.66500000000000004,"15min":0.60699999999999998},"max":{"1min":1.95,"5min":1.95,"15min":1.95},"last":1.177},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.159,"15min":1.1499999999999999},"min":{"1min":0.77800000000000002,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":2.02,"5min":2.02,"15min":2.02},"last":1.214}]},{"osd":2,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.232,"15min":1.2030000000000001},"min":{"1min":0.72499999999999998,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.5859999999999999,"5min":2.5859999999999999,"15min":2.5859999999999999},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.379,"5min":1.2589999999999999,"15min":1.228},"min":{"1min":0.69699999999999995,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.5350000000000001,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.506}]},{"osd":3,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.1140000000000001,"15min":1.1240000000000001},"min":{"1min":0.79500000000000004,"5min":0.72599999999999998,"15min":0.60199999999999998},"max":{"1min":1.353,"5min":1.6839999999999999,"15min":1.6930000000000001},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.1859999999999999,"15min":1.236},"min":{"1min":0.83799999999999997,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.5030000000000001,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":1.3300000000000001}]},{"osd":5,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.1830000000000001,"15min":1.1879999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.1599999999999999,"15min":1.181},"min":{"1min":0.76200000000000001,"5min":0.63300000000000001,"15min":0.61199999999999999},"max":{"1min":1.6539999999999999,"5min":1.732,"15min":1.8520000000000001},"last":0.83499999999999996}]},{"osd":6,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.2390000000000001,"15min":1.252},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.387},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3149999999999999,"15min":1.3169999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.0830000000000002,"5min":2.0830000000000002,"15min":2.1120000000000001},"last":1.4630000000000001}]},{"osd":7,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.2629999999999999,"15min":1.296},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.60899999999999999},"max":{"1min":1.77,"5min":1.794,"15min":1.9470000000000001},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.371,"15min":1.4079999999999999},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.75900000000000001},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.248}]}]},{"osd":3,"up_from":13,"seq":55834574997,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33156,"kb_used_data":2108,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338684,"statfs":{"total":96636764160,"available":96602812416,"internally_reserved":0,"allocated":2158592,"data_stored":1194488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.76800000000000002,"15min":0.79700000000000004},"min":{"1min":0.67100000000000004,"5min":0.442,"15min":0.33000000000000002},"max":{"1min":1.163,"5min":1.399,"15min":1.462},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.82399999999999995,"15min":0.83299999999999996},"min":{"1min":0.47099999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.3400000000000001,"5min":1.363,"15min":1.4950000000000001},"last":0.91100000000000003}]},{"osd":1,"last update":"Wed Oct 11 09:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.80600000000000005,"15min":0.81200000000000006},"min":{"1min":0.51000000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.2629999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.01,"5min":0.91300000000000003,"15min":0.92500000000000004},"min":{"1min":0.56200000000000006,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.3069999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.70799999999999996}]},{"osd":2,"last update":"Wed Oct 11 09:51:49 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":0.96399999999999997,"15min":0.96899999999999997},"min":{"1min":0.67400000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.4570000000000001,"5min":1.601,"15min":1.601},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.0249999999999999,"15min":1.0549999999999999},"min":{"1min":0.81599999999999995,"5min":0.58499999999999996,"15min":0.432},"max":{"1min":1.4399999999999999,"5min":1.607,"15min":2.5870000000000002},"last":0.88700000000000001}]},{"osd":4,"last update":"Wed Oct 11 09:51:49 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.1060000000000001,"15min":1.1579999999999999},"min":{"1min":0.77900000000000003,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.556,"5min":1.556,"15min":2.395},"last":1.3200000000000001},{"interface":"front","average":{"1min":1.26,"5min":1.1859999999999999,"15min":1.236},"min":{"1min":0.747,"5min":0.747,"15min":0.73599999999999999},"max":{"1min":1.579,"5min":1.9099999999999999,"15min":3.1579999999999999},"last":1.0760000000000001}]},{"osd":5,"last update":"Wed Oct 11 09:51:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.226,"15min":1.268},"min":{"1min":0.91200000000000003,"5min":0.84199999999999997,"15min":0.76500000000000001},"max":{"1min":1.476,"5min":1.724,"15min":3.0009999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.2430000000000001,"15min":1.3240000000000001},"min":{"1min":1.0309999999999999,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.651,"5min":1.651,"15min":3.8050000000000002},"last":1.107}]},{"osd":6,"last update":"Wed Oct 11 09:51:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.216,"15min":1.248},"min":{"1min":0.871,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.514,"5min":1.6890000000000001,"15min":2.1949999999999998},"last":1.161},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.272,"15min":1.3280000000000001},"min":{"1min":0.995,"5min":0.91700000000000004,"15min":0.84499999999999997},"max":{"1min":1.556,"5min":1.782,"15min":3.2549999999999999},"last":1.0389999999999999}]},{"osd":7,"last update":"Wed Oct 11 09:51:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.2849999999999999,"15min":1.3660000000000001},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.76700000000000002},"max":{"1min":1.587,"5min":1.7150000000000001,"15min":3.7490000000000001},"last":1.365},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.2729999999999999,"15min":1.323},"min":{"1min":0.96499999999999997,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.7130000000000001,"5min":1.7969999999999999,"15min":2.625},"last":1.425}]}]},{"osd":2,"up_from":13,"seq":55834574999,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1680,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1720320,"data_stored":787769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:51:48 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.80300000000000005,"15min":0.80800000000000005},"min":{"1min":0.65100000000000002,"5min":0.40600000000000003,"15min":0.38500000000000001},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":2.0600000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.86499999999999999,"15min":0.89500000000000002},"min":{"1min":0.67400000000000004,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":2.2850000000000001,"5min":2.2850000000000001,"15min":2.8919999999999999},"last":0.98799999999999999}]},{"osd":1,"last update":"Wed Oct 11 09:51:48 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.84399999999999997,"15min":0.93500000000000005},"min":{"1min":0.61699999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.621,"5min":1.8069999999999999,"15min":3.7650000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.96599999999999997,"15min":0.96899999999999997},"min":{"1min":0.75600000000000001,"5min":0.45600000000000002,"15min":0.40300000000000002},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":2.5339999999999998},"last":1.157}]},{"osd":3,"last update":"Wed Oct 11 09:51:48 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.91100000000000003,"15min":1.006},"min":{"1min":0.70099999999999996,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.3600000000000001,"5min":1.591,"15min":2.403},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.044,"15min":1.044},"min":{"1min":0.88400000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5660000000000001,"5min":1.754,"15min":2.7589999999999999},"last":1.091}]},{"osd":4,"last update":"Wed Oct 11 09:51:48 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.1679999999999999,"15min":1.1850000000000001},"min":{"1min":0.90500000000000003,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.6040000000000001,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":1.341},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.242,"15min":1.3240000000000001},"min":{"1min":1.0640000000000001,"5min":0.751,"15min":0.73399999999999999},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.6859999999999999},"last":1.4490000000000001}]},{"osd":5,"last update":"Wed Oct 11 09:51:48 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.2509999999999999,"15min":1.2949999999999999},"min":{"1min":0.96299999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":3.044},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.248,"5min":1.1919999999999999,"15min":1.248},"min":{"1min":0.83799999999999997,"5min":0.72799999999999998,"15min":0.69599999999999995},"max":{"1min":2.411,"5min":2.411,"15min":3.5790000000000002},"last":1.218}]},{"osd":6,"last update":"Wed Oct 11 09:51:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.216,"15min":1.3069999999999999},"min":{"1min":1.052,"5min":0.75,"15min":0.75},"max":{"1min":1.774,"5min":1.774,"15min":3.1600000000000001},"last":1.371},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3089999999999999,"15min":1.3280000000000001},"min":{"1min":1.054,"5min":0.83499999999999996,"15min":0.623},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":2.839},"last":1.248}]},{"osd":7,"last update":"Wed Oct 11 09:51:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.286,"15min":1.341},"min":{"1min":0.999,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":2.7130000000000001,"5min":2.7130000000000001,"15min":3.1230000000000002},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.333,"15min":1.421},"min":{"1min":1.2749999999999999,"5min":0.84899999999999998,"15min":0.80000000000000004},"max":{"1min":1.7210000000000001,"5min":1.7909999999999999,"15min":2.988},"last":1.2729999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574998,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32688,"kb_used_data":1640,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339152,"statfs":{"total":96636764160,"available":96603291648,"internally_reserved":0,"allocated":1679360,"data_stored":787768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:51:47 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.79800000000000004,"15min":0.78400000000000003},"min":{"1min":0.51500000000000001,"5min":0.48799999999999999,"15min":0.375},"max":{"1min":1.0900000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.81100000000000005,"15min":0.82099999999999995},"min":{"1min":0.53500000000000003,"5min":0.40100000000000002,"15min":0.36699999999999999},"max":{"1min":1.2210000000000001,"5min":1.593,"15min":1.593},"last":0.85099999999999998}]},{"osd":2,"last update":"Wed Oct 11 09:51:47 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.80300000000000005,"15min":0.83299999999999996},"min":{"1min":0.51700000000000002,"5min":0.437,"15min":0.36199999999999999},"max":{"1min":1.0740000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.92500000000000004,"15min":0.94899999999999995},"min":{"1min":0.72899999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.6619999999999999},"last":0.58799999999999997}]},{"osd":3,"last update":"Wed Oct 11 09:51:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0029999999999999,"15min":1.0089999999999999},"min":{"1min":0.65700000000000003,"5min":0.57799999999999996,"15min":0.46600000000000003},"max":{"1min":1.51,"5min":1.714,"15min":1.714},"last":0.63200000000000001},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.95099999999999996,"15min":0.96999999999999997},"min":{"1min":0.58799999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.4930000000000001,"5min":1.611,"15min":1.611},"last":1.109}]},{"osd":4,"last update":"Wed Oct 11 09:51:47 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.107,"15min":1.1399999999999999},"min":{"1min":0.89700000000000002,"5min":0.77400000000000002,"15min":0.73599999999999999},"max":{"1min":1.5349999999999999,"5min":1.5980000000000001,"15min":2.016},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.2250000000000001,"15min":1.2609999999999999},"min":{"1min":1.002,"5min":0.85699999999999998,"15min":0.80000000000000004},"max":{"1min":1.748,"5min":1.748,"15min":2.323},"last":1.2070000000000001}]},{"osd":5,"last update":"Wed Oct 11 09:51:47 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.1930000000000001,"15min":1.258},"min":{"1min":0.98999999999999999,"5min":0.79300000000000004,"15min":0.65800000000000003},"max":{"1min":1.552,"5min":1.722,"15min":2.4079999999999999},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.177,"15min":1.2210000000000001},"min":{"1min":0.84799999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.5660000000000001,"5min":1.677,"15min":2.1349999999999998},"last":1.262}]},{"osd":6,"last update":"Wed Oct 11 09:51:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.244,"15min":1.304},"min":{"1min":0.95399999999999996,"5min":0.84599999999999997,"15min":0.71499999999999997},"max":{"1min":1.613,"5min":2.4620000000000002,"15min":2.8940000000000001},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2310000000000001,"15min":1.3009999999999999},"min":{"1min":0.877,"5min":0.84799999999999998,"15min":0.79200000000000004},"max":{"1min":1.573,"5min":1.8129999999999999,"15min":3.3719999999999999},"last":1.052}]},{"osd":7,"last update":"Wed Oct 11 09:51:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.3220000000000001,"15min":1.363},"min":{"1min":1.044,"5min":0.86899999999999999,"15min":0.86099999999999999},"max":{"1min":1.7230000000000001,"5min":1.7929999999999999,"15min":2.194},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.381,"5min":1.3169999999999999,"15min":1.377},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.7390000000000001,"5min":1.857,"15min":1.968},"last":1.2410000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574997,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33292,"kb_used_data":2244,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338548,"statfs":{"total":96636764160,"available":96602673152,"internally_reserved":0,"allocated":2297856,"data_stored":1378148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 11 09:51:41 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.81000000000000005,"15min":0.80300000000000005},"min":{"1min":0.60399999999999998,"5min":0.45300000000000001,"15min":0.38500000000000001},"max":{"1min":1.0660000000000001,"5min":1.611,"15min":1.611},"last":1.3160000000000001},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.87,"15min":0.89500000000000002},"min":{"1min":0.56200000000000006,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1220000000000001,"5min":1.5249999999999999,"15min":2.3100000000000001},"last":0.88600000000000001}]},{"osd":2,"last update":"Wed Oct 11 09:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":0.92800000000000005,"15min":0.93799999999999994},"min":{"1min":0.59599999999999997,"5min":0.53100000000000003,"15min":0.50600000000000001},"max":{"1min":1.8919999999999999,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":1.133},{"interface":"front","average":{"1min":0.998,"5min":0.93100000000000005,"15min":0.95599999999999996},"min":{"1min":0.63100000000000001,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.5529999999999999,"5min":1.5820000000000001,"15min":1.9450000000000001},"last":0.99199999999999999}]},{"osd":3,"last update":"Wed Oct 11 09:51:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.79000000000000004,"15min":0.83799999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.45600000000000002},"max":{"1min":1.1759999999999999,"5min":1.22,"15min":2.7639999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":1.002,"5min":0.93100000000000005,"15min":0.98799999999999999},"min":{"1min":0.53300000000000003,"5min":0.51400000000000001,"15min":0.48999999999999999},"max":{"1min":1.482,"5min":1.482,"15min":2.1789999999999998},"last":0.83499999999999996}]},{"osd":4,"last update":"Wed Oct 11 09:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.125,"15min":1.169},"min":{"1min":0.94599999999999995,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.6659999999999999,"5min":1.792,"15min":1.792},"last":1.177},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.125,"15min":1.169},"min":{"1min":0.82399999999999995,"5min":0.755,"15min":0.69799999999999995},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":2.6629999999999998},"last":1.2909999999999999}]},{"osd":5,"last update":"Wed Oct 11 09:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.2210000000000001,"15min":1.292},"min":{"1min":1.1240000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.619,"5min":1.829,"15min":3.0150000000000001},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1599999999999999,"15min":1.256},"min":{"1min":0.81200000000000006,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.399,"5min":1.7070000000000001,"15min":3.601},"last":1.353}]},{"osd":6,"last update":"Wed Oct 11 09:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.1399999999999999,"15min":1.1950000000000001},"min":{"1min":0.98499999999999999,"5min":0.79100000000000004,"15min":0.78000000000000003},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.355,"5min":1.272,"15min":1.3049999999999999},"min":{"1min":1.028,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":1.7430000000000001,"5min":1.7430000000000001,"15min":2.1440000000000001},"last":1.3999999999999999}]},{"osd":7,"last update":"Wed Oct 11 09:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.2410000000000001,"15min":1.2909999999999999},"min":{"1min":0.84899999999999998,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.607,"5min":1.75,"15min":1.8759999999999999},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.24,"5min":1.2290000000000001,"15min":1.2909999999999999},"min":{"1min":0.95099999999999996,"5min":0.88800000000000001,"15min":0.73899999999999999},"max":{"1min":2.016,"5min":2.016,"15min":2.5419999999999998},"last":1.1200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-11T09:52:48.452 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-11T09:52:48.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.611+0000 7f5ca9e43700 1 Processor -- start 2023-10-11T09:52:48.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.611+0000 7f5ca9e43700 1 -- start start 2023-10-11T09:52:48.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.615+0000 7f5ca9e43700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5ca415d690 0x7f5ca40c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:48.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.615+0000 7f5ca9e43700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ca40ca3c0 con 0x7f5ca4158d40 2023-10-11T09:52:48.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.615+0000 7f5ca9e43700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ca415c960 con 0x7f5ca40cc4a0 2023-10-11T09:52:48.622 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.615+0000 7f5ca9e43700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca40ca0e0 con 0x7f5ca415d690 2023-10-11T09:52:48.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.615+0000 7f5ca8e41700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f5ca4158d40 0x7f5ca40cbd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:49736/0 (socket says 172.21.15.78:49736) 2023-10-11T09:52:48.624 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.615+0000 7f5ca8e41700 1 -- 172.21.15.78:0/954609268 learned_addr learned my addr 172.21.15.78:0/954609268 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:48.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.615+0000 7f5ca3fff700 1 --2- 172.21.15.78:0/954609268 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5ca415d690 0x7f5ca40c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:48.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.615+0000 7f5ca37fe700 1 -- 172.21.15.78:0/954609268 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3501012390 0 0) 0x7f5ca40ca3c0 con 0x7f5ca4158d40 2023-10-11T09:52:48.625 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.615+0000 7f5ca37fe700 1 -- 172.21.15.78:0/954609268 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c940034f0 con 0x7f5ca4158d40 2023-10-11T09:52:48.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.615+0000 7f5ca37fe700 1 -- 172.21.15.78:0/954609268 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1319394714 0 0) 0x7f5ca415c960 con 0x7f5ca40cc4a0 2023-10-11T09:52:48.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.615+0000 7f5ca37fe700 1 -- 172.21.15.78:0/954609268 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ca40ca3c0 con 0x7f5ca40cc4a0 2023-10-11T09:52:48.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.615+0000 7f5ca37fe700 1 -- 172.21.15.78:0/954609268 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2470022936 0 0) 0x7f5c940034f0 con 0x7f5ca4158d40 2023-10-11T09:52:48.626 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.615+0000 7f5ca37fe700 1 -- 172.21.15.78:0/954609268 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5ca415d690 msgr2=0x7f5ca40c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:48.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.615+0000 7f5ca37fe700 1 --2- 172.21.15.78:0/954609268 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5ca415d690 0x7f5ca40c9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:48.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.615+0000 7f5ca37fe700 1 -- 172.21.15.78:0/954609268 >> v1:172.21.15.133:6789/0 conn(0x7f5ca40cc4a0 legacy=0x7f5ca40ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:48.627 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.615+0000 7f5ca37fe700 1 -- 172.21.15.78:0/954609268 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca415cbb0 con 0x7f5ca4158d40 2023-10-11T09:52:48.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.615+0000 7f5ca37fe700 1 -- 172.21.15.78:0/954609268 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7f5c98001670 con 0x7f5ca4158d40 2023-10-11T09:52:48.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.615+0000 7f5ca3fff700 1 --2- 172.21.15.78:0/954609268 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5ca415d690 0x7f5ca40c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:52:48.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.615+0000 7f5ca37fe700 1 -- 172.21.15.78:0/954609268 >> v1:172.21.15.78:6789/0 conn(0x7f5ca4158d40 legacy=0x7f5ca40cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:48.628 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.615+0000 7f5ca37fe700 1 --2- 172.21.15.78:0/954609268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c94003cd0 0x7f5c940040c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:48.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.615+0000 7f5ca37fe700 1 --2- 172.21.15.78:0/954609268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c9400ad00 0x7f5c9400d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:48.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.615+0000 7f5ca37fe700 1 --2- 172.21.15.78:0/954609268 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5ca415d690 0x7f5c9400f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:48.629 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.615+0000 7f5ca37fe700 1 -- 172.21.15.78:0/954609268 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca40ca0e0 con 0x7f5c94003cd0 2023-10-11T09:52:48.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.615+0000 7f5ca37fe700 1 -- 172.21.15.78:0/954609268 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9400fed0 con 0x7f5c9400ad00 2023-10-11T09:52:48.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.615+0000 7f5ca37fe700 1 -- 172.21.15.78:0/954609268 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c94010090 con 0x7f5ca415d690 2023-10-11T09:52:48.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.615+0000 7f5ca8e41700 1 --2- 172.21.15.78:0/954609268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c94003cd0 0x7f5c940040c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:48.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.619+0000 7f5ca3fff700 1 --2- 172.21.15.78:0/954609268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c9400ad00 0x7f5c9400d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:48.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.619+0000 7f5ca8e41700 1 -- 172.21.15.78:0/954609268 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5ca415d690 msgr2=0x7f5c9400f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:48.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.619+0000 7f5ca8e41700 1 --2- 172.21.15.78:0/954609268 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5ca415d690 0x7f5c9400f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:48.634 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.619+0000 7f5ca8e41700 1 -- 172.21.15.78:0/954609268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c9400ad00 msgr2=0x7f5c9400d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:48.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.619+0000 7f5ca8e41700 1 --2- 172.21.15.78:0/954609268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c9400ad00 0x7f5c9400d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:48.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.619+0000 7f5ca8e41700 1 -- 172.21.15.78:0/954609268 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c900094b0 con 0x7f5c94003cd0 2023-10-11T09:52:48.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.619+0000 7f5ca8e41700 1 --2- 172.21.15.78:0/954609268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c94003cd0 0x7f5c940040c0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f5c9800c160 tx=0x7f5c940037a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef02eaaf14f32ae3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:48.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.619+0000 7f5ca37fe700 1 -- 172.21.15.78:0/954609268 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9800ccc0 con 0x7f5c94003cd0 2023-10-11T09:52:48.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.619+0000 7f5ca37fe700 1 -- 172.21.15.78:0/954609268 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c9800ce20 con 0x7f5c94003cd0 2023-10-11T09:52:48.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.619+0000 7f5ca37fe700 1 -- 172.21.15.78:0/954609268 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c98005b80 con 0x7f5c94003cd0 2023-10-11T09:52:48.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.619+0000 7f5ca9e43700 1 -- 172.21.15.78:0/954609268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c94003cd0 msgr2=0x7f5c940040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:48.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.619+0000 7f5ca9e43700 1 --2- 172.21.15.78:0/954609268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c94003cd0 0x7f5c940040c0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f5c9800c160 tx=0x7f5c940037a0 comp rx=0 tx=0).stop 2023-10-11T09:52:48.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.619+0000 7f5ca9642700 1 -- 172.21.15.78:0/954609268 reap_dead start 2023-10-11T09:52:48.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.619+0000 7f5ca9e43700 1 -- 172.21.15.78:0/954609268 shutdown_connections 2023-10-11T09:52:48.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.619+0000 7f5ca9e43700 1 -- 172.21.15.78:0/954609268 >> 172.21.15.78:0/954609268 conn(0x7f5ca40b0f40 msgr2=0x7f5ca40b1310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:48.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.619+0000 7f5ca9e43700 1 -- 172.21.15.78:0/954609268 shutdown_connections 2023-10-11T09:52:48.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.619+0000 7f5ca9e43700 1 -- 172.21.15.78:0/954609268 wait complete. 2023-10-11T09:52:48.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.623+0000 7f5ca9e43700 1 Processor -- start 2023-10-11T09:52:48.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.623+0000 7f5ca9e43700 1 -- start start 2023-10-11T09:52:48.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.623+0000 7f5ca9e43700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c94003cd0 0x7f5ca40c4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:48.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.623+0000 7f5ca9e43700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ca4150cf0 0x7f5ca414b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:48.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.623+0000 7f5ca9e43700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5ca4071320 0x7f5ca40717c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:48.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.623+0000 7f5ca9e43700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca4071d00 con 0x7f5ca4150cf0 2023-10-11T09:52:48.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.623+0000 7f5ca9e43700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca41530d0 con 0x7f5c94003cd0 2023-10-11T09:52:48.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.623+0000 7f5ca9e43700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca40c5750 con 0x7f5ca4071320 2023-10-11T09:52:48.640 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.623+0000 7f5ca3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ca4150cf0 0x7f5ca414b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:48.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.623+0000 7f5ca3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ca4150cf0 0x7f5ca414b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48320/0 (socket says 172.21.15.78:48320) 2023-10-11T09:52:48.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.623+0000 7f5ca3fff700 1 -- 172.21.15.78:0/2541385823 learned_addr learned my addr 172.21.15.78:0/2541385823 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:48.641 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.627+0000 7f5ca9642700 1 --2- 172.21.15.78:0/2541385823 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5ca4071320 0x7f5ca40717c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:48.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.627+0000 7f5ca8e41700 1 --2- 172.21.15.78:0/2541385823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c94003cd0 0x7f5ca40c4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:48.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.627+0000 7f5ca3fff700 1 -- 172.21.15.78:0/2541385823 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5ca4071320 msgr2=0x7f5ca40717c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:48.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.627+0000 7f5ca3fff700 1 --2- 172.21.15.78:0/2541385823 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5ca4071320 0x7f5ca40717c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:48.642 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.627+0000 7f5ca3fff700 1 -- 172.21.15.78:0/2541385823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c94003cd0 msgr2=0x7f5ca40c4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:48.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.627+0000 7f5ca3fff700 1 --2- 172.21.15.78:0/2541385823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c94003cd0 0x7f5ca40c4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:48.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.627+0000 7f5ca3fff700 1 -- 172.21.15.78:0/2541385823 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca40c5a30 con 0x7f5ca4150cf0 2023-10-11T09:52:48.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.627+0000 7f5ca9642700 1 --2- 172.21.15.78:0/2541385823 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5ca4071320 0x7f5ca40717c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:48.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.627+0000 7f5ca8e41700 1 --2- 172.21.15.78:0/2541385823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c94003cd0 0x7f5ca40c4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:48.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.627+0000 7f5ca3fff700 1 --2- 172.21.15.78:0/2541385823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ca4150cf0 0x7f5ca414b7c0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f5c900118d0 tx=0x7f5c90011c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:48.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.627+0000 7f5ca17fa700 1 -- 172.21.15.78:0/2541385823 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9000fa80 con 0x7f5ca4150cf0 2023-10-11T09:52:48.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.627+0000 7f5ca17fa700 1 -- 172.21.15.78:0/2541385823 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c9000fbe0 con 0x7f5ca4150cf0 2023-10-11T09:52:48.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.627+0000 7f5ca9e43700 1 -- 172.21.15.78:0/2541385823 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ca4152a80 con 0x7f5ca4150cf0 2023-10-11T09:52:48.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.627+0000 7f5ca17fa700 1 -- 172.21.15.78:0/2541385823 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c90011e70 con 0x7f5ca4150cf0 2023-10-11T09:52:48.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.627+0000 7f5ca9e43700 1 -- 172.21.15.78:0/2541385823 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ca40c5440 con 0x7f5ca4150cf0 2023-10-11T09:52:48.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.627+0000 7f5ca9e43700 1 -- 172.21.15.78:0/2541385823 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ca4153260 con 0x7f5ca4150cf0 2023-10-11T09:52:48.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.631+0000 7f5ca17fa700 1 -- 172.21.15.78:0/2541385823 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f5c90015460 con 0x7f5ca4150cf0 2023-10-11T09:52:48.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.631+0000 7f5ca17fa700 1 --2- 172.21.15.78:0/2541385823 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5c88065f00 0x7f5c880683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:48.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.631+0000 7f5ca17fa700 1 -- 172.21.15.78:0/2541385823 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f5c90083ec0 con 0x7f5ca4150cf0 2023-10-11T09:52:48.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.631+0000 7f5ca8e41700 1 --2- 172.21.15.78:0/2541385823 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5c88065f00 0x7f5c880683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:48.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.631+0000 7f5ca8e41700 1 --2- 172.21.15.78:0/2541385823 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5c88065f00 0x7f5c880683b0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5c98015040 tx=0x7f5c98007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:48.648 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.635+0000 7f5ca17fa700 1 -- 172.21.15.78:0/2541385823 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f5c90051750 con 0x7f5ca4150cf0 2023-10-11T09:52:48.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.807+0000 7f5ca9e43700 1 -- 172.21.15.78:0/2541385823 --> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5ca415d170 con 0x7f5c88065f00 2023-10-11T09:52:48.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.815+0000 7f5ca17fa700 1 -- 172.21.15.78:0/2541385823 <== mgr.4101 v2:172.21.15.78:6832/35606 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70642 (secure 0 0 0) 0x7f5c98022070 con 0x7f5c88065f00 2023-10-11T09:52:48.824 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:52:48.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.819+0000 7f5ca9e43700 1 -- 172.21.15.78:0/2541385823 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5c88065f00 msgr2=0x7f5c880683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:48.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.819+0000 7f5ca9e43700 1 --2- 172.21.15.78:0/2541385823 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5c88065f00 0x7f5c880683b0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5c98015040 tx=0x7f5c98007000 comp rx=0 tx=0).stop 2023-10-11T09:52:48.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.819+0000 7f5ca9e43700 1 -- 172.21.15.78:0/2541385823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ca4150cf0 msgr2=0x7f5ca414b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:48.827 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.819+0000 7f5ca9e43700 1 --2- 172.21.15.78:0/2541385823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ca4150cf0 0x7f5ca414b7c0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f5c900118d0 tx=0x7f5c90011c90 comp rx=0 tx=0).stop 2023-10-11T09:52:48.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.819+0000 7f5ca9e43700 1 -- 172.21.15.78:0/2541385823 shutdown_connections 2023-10-11T09:52:48.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.819+0000 7f5ca9e43700 1 --2- 172.21.15.78:0/2541385823 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5c88065f00 0x7f5c880683b0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:48.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.819+0000 7f5ca9e43700 1 --2- 172.21.15.78:0/2541385823 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5ca4071320 0x7f5ca40717c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:48.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.819+0000 7f5ca9e43700 1 --2- 172.21.15.78:0/2541385823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ca4150cf0 0x7f5ca414b7c0 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:48.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.819+0000 7f5ca9e43700 1 --2- 172.21.15.78:0/2541385823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c94003cd0 0x7f5ca40c4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:48.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.819+0000 7f5ca9e43700 1 -- 172.21.15.78:0/2541385823 >> 172.21.15.78:0/2541385823 conn(0x7f5ca40b0f40 msgr2=0x7f5ca40c9890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:48.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.819+0000 7f5ca9e43700 1 -- 172.21.15.78:0/2541385823 shutdown_connections 2023-10-11T09:52:48.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:48.819+0000 7f5ca9e43700 1 -- 172.21.15.78:0/2541385823 wait complete. 2023-10-11T09:52:48.830 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2023-10-11T09:52:48.848 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":495,"stamp":"2023-10-11T09:52:48.585970+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":31,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":262408,"kb_used_data":14024,"kb_used_omap":0,"kb_used_meta":248320,"kb_avail":754712312,"statfs":{"total":773094113280,"available":772825407488,"internally_reserved":0,"allocated":14360576,"data_stored":7583432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":254279680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-407665,"num_objects":-125,"num_object_clones":0,"num_object_copies":-250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-125,"num_whiteouts":0,"num_read":-1808,"num_read_kb":-1515,"num_write":-914,"num_write_kb":-953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.917857"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":99,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-11T09:52:44.009882+0000","last_change":"2023-10-11T09:40:36.151839+0000","last_active":"2023-10-11T09:52:44.009882+0000","last_peered":"2023-10-11T09:52:44.009882+0000","last_clean":"2023-10-11T09:52:44.009882+0000","last_became_active":"2023-10-11T09:40:36.151179+0000","last_became_peered":"2023-10-11T09:40:36.151179+0000","last_unstale":"2023-10-11T09:52:44.009882+0000","last_undegraded":"2023-10-11T09:52:44.009882+0000","last_fullsized":"2023-10-11T09:52:44.009882+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":106,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-11T09:52:44.008721+0000","last_change":"2023-10-11T09:40:36.150224+0000","last_active":"2023-10-11T09:52:44.008721+0000","last_peered":"2023-10-11T09:52:44.008721+0000","last_clean":"2023-10-11T09:52:44.008721+0000","last_became_active":"2023-10-11T09:40:36.149919+0000","last_became_peered":"2023-10-11T09:40:36.149919+0000","last_unstale":"2023-10-11T09:52:44.008721+0000","last_undegraded":"2023-10-11T09:52:44.008721+0000","last_fullsized":"2023-10-11T09:52:44.008721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":128,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-11T09:52:43.706994+0000","last_change":"2023-10-11T09:40:36.162174+0000","last_active":"2023-10-11T09:52:43.706994+0000","last_peered":"2023-10-11T09:52:43.706994+0000","last_clean":"2023-10-11T09:52:43.706994+0000","last_became_active":"2023-10-11T09:40:36.161710+0000","last_became_peered":"2023-10-11T09:40:36.161710+0000","last_unstale":"2023-10-11T09:52:43.706994+0000","last_undegraded":"2023-10-11T09:52:43.706994+0000","last_fullsized":"2023-10-11T09:52:43.706994+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:57:03.630878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":106,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-11T09:52:44.010145+0000","last_change":"2023-10-11T09:40:36.160620+0000","last_active":"2023-10-11T09:52:44.010145+0000","last_peered":"2023-10-11T09:52:44.010145+0000","last_clean":"2023-10-11T09:52:44.010145+0000","last_became_active":"2023-10-11T09:40:36.160171+0000","last_became_peered":"2023-10-11T09:40:36.160171+0000","last_unstale":"2023-10-11T09:52:44.010145+0000","last_undegraded":"2023-10-11T09:52:44.010145+0000","last_fullsized":"2023-10-11T09:52:44.010145+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":104,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-11T09:52:44.009611+0000","last_change":"2023-10-11T09:40:36.151710+0000","last_active":"2023-10-11T09:52:44.009611+0000","last_peered":"2023-10-11T09:52:44.009611+0000","last_clean":"2023-10-11T09:52:44.009611+0000","last_became_active":"2023-10-11T09:40:36.151219+0000","last_became_peered":"2023-10-11T09:40:36.151219+0000","last_unstale":"2023-10-11T09:52:44.009611+0000","last_undegraded":"2023-10-11T09:52:44.009611+0000","last_fullsized":"2023-10-11T09:52:44.009611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":106,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-11T09:52:44.004783+0000","last_change":"2023-10-11T09:40:36.216248+0000","last_active":"2023-10-11T09:52:44.004783+0000","last_peered":"2023-10-11T09:52:44.004783+0000","last_clean":"2023-10-11T09:52:44.004783+0000","last_became_active":"2023-10-11T09:40:36.215890+0000","last_became_peered":"2023-10-11T09:40:36.215890+0000","last_unstale":"2023-10-11T09:52:44.004783+0000","last_undegraded":"2023-10-11T09:52:44.004783+0000","last_fullsized":"2023-10-11T09:52:44.004783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":129,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-11T09:52:43.703936+0000","last_change":"2023-10-11T09:40:36.149739+0000","last_active":"2023-10-11T09:52:43.703936+0000","last_peered":"2023-10-11T09:52:43.703936+0000","last_clean":"2023-10-11T09:52:43.703936+0000","last_became_active":"2023-10-11T09:40:36.149419+0000","last_became_peered":"2023-10-11T09:40:36.149419+0000","last_unstale":"2023-10-11T09:52:43.703936+0000","last_undegraded":"2023-10-11T09:52:43.703936+0000","last_fullsized":"2023-10-11T09:52:43.703936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:54:50.640362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":104,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-11T09:52:44.010199+0000","last_change":"2023-10-11T09:40:36.148773+0000","last_active":"2023-10-11T09:52:44.010199+0000","last_peered":"2023-10-11T09:52:44.010199+0000","last_clean":"2023-10-11T09:52:44.010199+0000","last_became_active":"2023-10-11T09:40:36.148489+0000","last_became_peered":"2023-10-11T09:40:36.148489+0000","last_unstale":"2023-10-11T09:52:44.010199+0000","last_undegraded":"2023-10-11T09:52:44.010199+0000","last_fullsized":"2023-10-11T09:52:44.010199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":282,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-11T09:52:43.704492+0000","last_change":"2023-10-11T09:40:29.874372+0000","last_active":"2023-10-11T09:52:43.704492+0000","last_peered":"2023-10-11T09:52:43.704492+0000","last_clean":"2023-10-11T09:52:43.704492+0000","last_became_active":"2023-10-11T09:40:29.873621+0000","last_became_peered":"2023-10-11T09:40:29.873621+0000","last_unstale":"2023-10-11T09:52:43.704492+0000","last_undegraded":"2023-10-11T09:52:43.704492+0000","last_fullsized":"2023-10-11T09:52:43.704492+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_clean_scrub_stamp":"2023-10-11T09:40:28.742276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:22:07.985311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575000,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33236,"kb_used_data":2188,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338604,"statfs":{"total":96636764160,"available":96602730496,"internally_reserved":0,"allocated":2240512,"data_stored":1390705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.056,"15min":1.087},"min":{"1min":0.68999999999999995,"5min":0.56499999999999995,"15min":0.48999999999999999},"max":{"1min":1.4730000000000001,"5min":1.7390000000000001,"15min":1.903},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.034,"5min":1.0329999999999999,"15min":1.0860000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.50900000000000001},"max":{"1min":1.4970000000000001,"5min":1.554,"15min":1.696},"last":0.91600000000000004}]},{"osd":1,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.1140000000000001,"15min":1.123},"min":{"1min":0.91000000000000003,"5min":0.66800000000000004,"15min":0.63100000000000001},"max":{"1min":1.4059999999999999,"5min":1.645,"15min":2.25},"last":1.125},{"interface":"front","average":{"1min":1.167,"5min":1.0489999999999999,"15min":1.0920000000000001},"min":{"1min":0.93300000000000005,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.4950000000000001,"5min":1.587,"15min":1.696},"last":1.7090000000000001}]},{"osd":2,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.0840000000000001,"15min":1.103},"min":{"1min":0.72899999999999998,"5min":0.59499999999999997,"15min":0.58999999999999997},"max":{"1min":1.411,"5min":1.5049999999999999,"15min":1.873},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.1719999999999999,"15min":1.155},"min":{"1min":0.72499999999999998,"5min":0.66400000000000003,"15min":0.65500000000000003},"max":{"1min":1.579,"5min":1.78,"15min":1.78},"last":1.196}]},{"osd":3,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.216,"15min":1.22},"min":{"1min":0.89200000000000002,"5min":0.80000000000000004,"15min":0.64900000000000002},"max":{"1min":1.617,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.387},{"interface":"front","average":{"1min":1.302,"5min":1.284,"15min":1.2809999999999999},"min":{"1min":0.94699999999999995,"5min":0.747,"15min":0.61699999999999999},"max":{"1min":1.54,"5min":1.8720000000000001,"15min":2.1579999999999999},"last":1.29}]},{"osd":4,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.0609999999999999,"15min":1.1200000000000001},"min":{"1min":0.71999999999999997,"5min":0.64900000000000002,"15min":0.53300000000000003},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":2.2010000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.181,"5min":1.208,"15min":1.226},"min":{"1min":0.93100000000000005,"5min":0.67200000000000004,"15min":0.63500000000000001},"max":{"1min":1.4199999999999999,"5min":1.768,"15min":1.9339999999999999},"last":1.335}]},{"osd":5,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.2709999999999999,"15min":1.278},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.7350000000000001,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2050000000000001,"15min":1.2749999999999999},"min":{"1min":0.82999999999999996,"5min":0.78300000000000003,"15min":0.65900000000000003},"max":{"1min":1.714,"5min":1.9450000000000001,"15min":3.0750000000000002},"last":1.1559999999999999}]},{"osd":6,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.3300000000000001,"15min":1.3919999999999999},"min":{"1min":0.98599999999999999,"5min":0.77000000000000002,"15min":0.70999999999999996},"max":{"1min":2,"5min":2,"15min":3.137},"last":1.222},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3500000000000001,"15min":1.381},"min":{"1min":0.89700000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.897,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.4319999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574999,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1608,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1646592,"data_stored":800337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.125,"15min":1.036},"min":{"1min":0.626,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.403,"5min":7.0730000000000004,"15min":7.0730000000000004},"last":0.86899999999999999},{"interface":"front","average":{"1min":1.01,"5min":1.179,"15min":1.071},"min":{"1min":0.65900000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.357,"5min":8.8450000000000006,"15min":8.8450000000000006},"last":1.145}]},{"osd":1,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.359,"15min":1.232},"min":{"1min":0.627,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.5149999999999999,"5min":15.523,"15min":15.523},"last":1.45},{"interface":"front","average":{"1min":1.181,"5min":1.3069999999999999,"15min":1.2310000000000001},"min":{"1min":0.80700000000000005,"5min":0.59199999999999997,"15min":0.53400000000000003},"max":{"1min":1.5509999999999999,"5min":15.253,"15min":15.253},"last":1.081}]},{"osd":2,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.1200000000000001,"15min":1.1319999999999999},"min":{"1min":0.745,"5min":0.434,"15min":0.434},"max":{"1min":1.335,"5min":2.7919999999999998,"15min":2.7919999999999998},"last":1.375},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1930000000000001,"15min":1.1890000000000001},"min":{"1min":0.79400000000000004,"5min":0.61599999999999999,"15min":0.44500000000000001},"max":{"1min":1.6060000000000001,"5min":5.0990000000000002,"15min":5.0990000000000002},"last":1.004}]},{"osd":3,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.2150000000000001,"15min":1.155},"min":{"1min":0.72599999999999998,"5min":0.70599999999999996,"15min":0.627},"max":{"1min":1.421,"5min":11.945,"15min":11.945},"last":1.228},{"interface":"front","average":{"1min":1.246,"5min":1.175,"15min":1.2010000000000001},"min":{"1min":0.84299999999999997,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.659,"5min":1.659,"15min":3.0819999999999999},"last":1.629}]},{"osd":4,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.2889999999999999,"15min":1.2769999999999999},"min":{"1min":0.626,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.6359999999999999,"5min":7.5599999999999996,"15min":7.5599999999999996},"last":0.70399999999999996},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.3380000000000001,"15min":1.3069999999999999},"min":{"1min":0.81299999999999994,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.542,"5min":8.7520000000000007,"15min":8.7520000000000007},"last":1.0309999999999999}]},{"osd":5,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.196,"15min":1.2290000000000001},"min":{"1min":0.65900000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.585,"5min":2.1560000000000001,"15min":2.4009999999999998},"last":1.262},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.282,"15min":1.3200000000000001},"min":{"1min":0.68700000000000006,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.978,"5min":2.4390000000000001,"15min":2.7290000000000001},"last":0.93700000000000006}]},{"osd":7,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.335,"15min":1.3440000000000001},"min":{"1min":0.91400000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.706,"5min":8.7970000000000006,"15min":8.7970000000000006},"last":1.302},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.3899999999999999,"15min":1.4199999999999999},"min":{"1min":1.01,"5min":0.625,"15min":0.625},"max":{"1min":1.8620000000000001,"5min":2.641,"15min":2.641},"last":1.54}]}]},{"osd":5,"up_from":13,"seq":55834574999,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1608,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1646592,"data_stored":800337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.044,"15min":1.0129999999999999},"min":{"1min":0.69799999999999995,"5min":0.64100000000000001,"15min":0.45300000000000001},"max":{"1min":1.4770000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.125,"5min":1.087,"15min":1.0269999999999999},"min":{"1min":0.66200000000000003,"5min":0.47699999999999998,"15min":0.46500000000000002},"max":{"1min":1.3460000000000001,"5min":1.536,"15min":1.7649999999999999},"last":1.3089999999999999}]},{"osd":1,"last update":"Wed Oct 11 09:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.1160000000000001,"15min":1.034},"min":{"1min":0.69599999999999995,"5min":0.68100000000000005,"15min":0.49399999999999999},"max":{"1min":1.542,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.874},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.0900000000000001},"min":{"1min":0.71599999999999997,"5min":0.61599999999999999,"15min":0.437},"max":{"1min":1.4299999999999999,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.508}]},{"osd":2,"last update":"Wed Oct 11 09:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1819999999999999,"15min":1.179},"min":{"1min":0.76100000000000001,"5min":0.746,"15min":0.73899999999999999},"max":{"1min":1.5009999999999999,"5min":2.133,"15min":2.133},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2370000000000001,"15min":1.2190000000000001},"min":{"1min":0.94099999999999995,"5min":0.64200000000000002,"15min":0.52300000000000002},"max":{"1min":1.46,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.90100000000000002}]},{"osd":3,"last update":"Wed Oct 11 09:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.80700000000000005,"5min":0.752,"15min":0.56999999999999995},"max":{"1min":1.3580000000000001,"5min":1.8859999999999999,"15min":1.921},"last":1.087},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.1950000000000001,"15min":1.1659999999999999},"min":{"1min":0.92500000000000004,"5min":0.71999999999999997,"15min":0.55300000000000005},"max":{"1min":1.492,"5min":1.9690000000000001,"15min":2.0209999999999999},"last":1.2609999999999999}]},{"osd":4,"last update":"Wed Oct 11 09:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.1919999999999999,"15min":1.1859999999999999},"min":{"1min":0.61399999999999999,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.4139999999999999,"5min":2.0169999999999999,"15min":2.117},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.075,"5min":1.157,"15min":1.1759999999999999},"min":{"1min":0.81999999999999995,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.4159999999999999,"5min":1.9690000000000001,"15min":2.1749999999999998},"last":1.2090000000000001}]},{"osd":6,"last update":"Wed Oct 11 09:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.228,"15min":1.25},"min":{"1min":0.75700000000000001,"5min":0.69499999999999995,"15min":0.59499999999999997},"max":{"1min":1.5449999999999999,"5min":1.9239999999999999,"15min":2.4529999999999998},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.252,"5min":1.331,"15min":1.3200000000000001},"min":{"1min":0.78700000000000003,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.478,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.385}]},{"osd":7,"last update":"Wed Oct 11 09:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.3320000000000001,"15min":1.341},"min":{"1min":1.0069999999999999,"5min":0.58799999999999997,"15min":0.57699999999999996},"max":{"1min":1.502,"5min":2.347,"15min":3.6120000000000001},"last":1.456},{"interface":"front","average":{"1min":1.353,"5min":1.4319999999999999,"15min":1.427},"min":{"1min":0.98499999999999999,"5min":0.89100000000000001,"15min":0.71299999999999997},"max":{"1min":1.589,"5min":2.2679999999999998,"15min":3.2109999999999999},"last":1.4279999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574999,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32656,"kb_used_data":1608,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1646592,"data_stored":800337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.071,"15min":1.0640000000000001},"min":{"1min":0.74399999999999999,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":2.0369999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.133,"5min":1.0760000000000001,"15min":1.0880000000000001},"min":{"1min":0.90000000000000002,"5min":0.60199999999999998,"15min":0.54700000000000004},"max":{"1min":1.5129999999999999,"5min":1.669,"15min":2.1160000000000001},"last":1.089}]},{"osd":1,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.109,"15min":1.089},"min":{"1min":0.72699999999999998,"5min":0.66500000000000004,"15min":0.60699999999999998},"max":{"1min":1.95,"5min":1.95,"15min":1.95},"last":1.111},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.159,"15min":1.1499999999999999},"min":{"1min":0.77800000000000002,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":2.02,"5min":2.02,"15min":2.02},"last":1.2649999999999999}]},{"osd":2,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.232,"15min":1.2030000000000001},"min":{"1min":0.72499999999999998,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.5859999999999999,"5min":2.5859999999999999,"15min":2.5859999999999999},"last":1.163},{"interface":"front","average":{"1min":1.379,"5min":1.2589999999999999,"15min":1.228},"min":{"1min":0.69699999999999995,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.5350000000000001,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.4570000000000001}]},{"osd":3,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.1140000000000001,"15min":1.1240000000000001},"min":{"1min":0.79500000000000004,"5min":0.72599999999999998,"15min":0.60199999999999998},"max":{"1min":1.353,"5min":1.6839999999999999,"15min":1.6930000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.1859999999999999,"15min":1.236},"min":{"1min":0.83799999999999997,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.5030000000000001,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":0.88600000000000001}]},{"osd":5,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.1830000000000001,"15min":1.1879999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":1.361},{"interface":"front","average":{"1min":1.175,"5min":1.1599999999999999,"15min":1.181},"min":{"1min":0.76200000000000001,"5min":0.63300000000000001,"15min":0.61199999999999999},"max":{"1min":1.6539999999999999,"5min":1.732,"15min":1.8520000000000001},"last":1.2929999999999999}]},{"osd":6,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.2390000000000001,"15min":1.252},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3149999999999999,"15min":1.3169999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.0830000000000002,"5min":2.0830000000000002,"15min":2.1120000000000001},"last":1.5489999999999999}]},{"osd":7,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.2629999999999999,"15min":1.296},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.60899999999999999},"max":{"1min":1.77,"5min":1.794,"15min":1.9470000000000001},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.371,"15min":1.4079999999999999},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.75900000000000001},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.4239999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574998,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1608,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1646592,"data_stored":800337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.76800000000000002,"15min":0.79700000000000004},"min":{"1min":0.67100000000000004,"5min":0.442,"15min":0.33000000000000002},"max":{"1min":1.163,"5min":1.399,"15min":1.462},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.82399999999999995,"15min":0.83299999999999996},"min":{"1min":0.47099999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.3400000000000001,"5min":1.363,"15min":1.4950000000000001},"last":0.80800000000000005}]},{"osd":1,"last update":"Wed Oct 11 09:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.80600000000000005,"15min":0.81200000000000006},"min":{"1min":0.51000000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.2629999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.01,"5min":0.91300000000000003,"15min":0.92500000000000004},"min":{"1min":0.56200000000000006,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.3069999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.88200000000000001}]},{"osd":2,"last update":"Wed Oct 11 09:51:49 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":0.96399999999999997,"15min":0.96899999999999997},"min":{"1min":0.67400000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.4570000000000001,"5min":1.601,"15min":1.601},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.0249999999999999,"15min":1.0549999999999999},"min":{"1min":0.81599999999999995,"5min":0.58499999999999996,"15min":0.432},"max":{"1min":1.4399999999999999,"5min":1.607,"15min":2.5870000000000002},"last":0.96099999999999997}]},{"osd":4,"last update":"Wed Oct 11 09:51:49 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.1060000000000001,"15min":1.1579999999999999},"min":{"1min":0.77900000000000003,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.556,"5min":1.556,"15min":2.395},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.26,"5min":1.1859999999999999,"15min":1.236},"min":{"1min":0.747,"5min":0.747,"15min":0.73599999999999999},"max":{"1min":1.579,"5min":1.9099999999999999,"15min":3.1579999999999999},"last":1.133}]},{"osd":5,"last update":"Wed Oct 11 09:51:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.226,"15min":1.268},"min":{"1min":0.91200000000000003,"5min":0.84199999999999997,"15min":0.76500000000000001},"max":{"1min":1.476,"5min":1.724,"15min":3.0009999999999999},"last":1.409},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.2430000000000001,"15min":1.3240000000000001},"min":{"1min":1.0309999999999999,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.651,"5min":1.651,"15min":3.8050000000000002},"last":1.1890000000000001}]},{"osd":6,"last update":"Wed Oct 11 09:51:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.216,"15min":1.248},"min":{"1min":0.871,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.514,"5min":1.6890000000000001,"15min":2.1949999999999998},"last":1.171},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.272,"15min":1.3280000000000001},"min":{"1min":0.995,"5min":0.91700000000000004,"15min":0.84499999999999997},"max":{"1min":1.556,"5min":1.782,"15min":3.2549999999999999},"last":1.492}]},{"osd":7,"last update":"Wed Oct 11 09:51:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.2849999999999999,"15min":1.3660000000000001},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.76700000000000002},"max":{"1min":1.587,"5min":1.7150000000000001,"15min":3.7490000000000001},"last":1.371},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.2729999999999999,"15min":1.323},"min":{"1min":0.96499999999999997,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.7130000000000001,"5min":1.7969999999999999,"15min":2.625},"last":1.3400000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575000,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1608,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1646592,"data_stored":800337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:51:48 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.80300000000000005,"15min":0.80800000000000005},"min":{"1min":0.65100000000000002,"5min":0.40600000000000003,"15min":0.38500000000000001},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":2.0600000000000001},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.86499999999999999,"15min":0.89500000000000002},"min":{"1min":0.67400000000000004,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":2.2850000000000001,"5min":2.2850000000000001,"15min":2.8919999999999999},"last":0.82899999999999996}]},{"osd":1,"last update":"Wed Oct 11 09:51:48 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.84399999999999997,"15min":0.93500000000000005},"min":{"1min":0.61699999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.621,"5min":1.8069999999999999,"15min":3.7650000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.96599999999999997,"15min":0.96899999999999997},"min":{"1min":0.75600000000000001,"5min":0.45600000000000002,"15min":0.40300000000000002},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":2.5339999999999998},"last":1.272}]},{"osd":3,"last update":"Wed Oct 11 09:51:48 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.91100000000000003,"15min":1.006},"min":{"1min":0.70099999999999996,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.3600000000000001,"5min":1.591,"15min":2.403},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.044,"15min":1.044},"min":{"1min":0.88400000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5660000000000001,"5min":1.754,"15min":2.7589999999999999},"last":1.343}]},{"osd":4,"last update":"Wed Oct 11 09:51:48 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.1679999999999999,"15min":1.1850000000000001},"min":{"1min":0.90500000000000003,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.6040000000000001,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.242,"15min":1.3240000000000001},"min":{"1min":1.0640000000000001,"5min":0.751,"15min":0.73399999999999999},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.6859999999999999},"last":1.3009999999999999}]},{"osd":5,"last update":"Wed Oct 11 09:51:48 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.2509999999999999,"15min":1.2949999999999999},"min":{"1min":0.96299999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":3.044},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.248,"5min":1.1919999999999999,"15min":1.248},"min":{"1min":0.83799999999999997,"5min":0.72799999999999998,"15min":0.69599999999999995},"max":{"1min":2.411,"5min":2.411,"15min":3.5790000000000002},"last":1.077}]},{"osd":6,"last update":"Wed Oct 11 09:51:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.216,"15min":1.3069999999999999},"min":{"1min":1.052,"5min":0.75,"15min":0.75},"max":{"1min":1.774,"5min":1.774,"15min":3.1600000000000001},"last":1.224},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3089999999999999,"15min":1.3280000000000001},"min":{"1min":1.054,"5min":0.83499999999999996,"15min":0.623},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":2.839},"last":1.411}]},{"osd":7,"last update":"Wed Oct 11 09:51:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.286,"15min":1.341},"min":{"1min":0.999,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":2.7130000000000001,"5min":2.7130000000000001,"15min":3.1230000000000002},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.333,"15min":1.421},"min":{"1min":1.2749999999999999,"5min":0.84899999999999998,"15min":0.80000000000000004},"max":{"1min":1.7210000000000001,"5min":1.7909999999999999,"15min":2.988},"last":1.367}]}]},{"osd":1,"up_from":13,"seq":55834574999,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1608,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1646592,"data_stored":800337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:51:47 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.79800000000000004,"15min":0.78400000000000003},"min":{"1min":0.51500000000000001,"5min":0.48799999999999999,"15min":0.375},"max":{"1min":1.0900000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.81100000000000005,"15min":0.82099999999999995},"min":{"1min":0.53500000000000003,"5min":0.40100000000000002,"15min":0.36699999999999999},"max":{"1min":1.2210000000000001,"5min":1.593,"15min":1.593},"last":0.88800000000000001}]},{"osd":2,"last update":"Wed Oct 11 09:51:47 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.80300000000000005,"15min":0.83299999999999996},"min":{"1min":0.51700000000000002,"5min":0.437,"15min":0.36199999999999999},"max":{"1min":1.0740000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.92500000000000004,"15min":0.94899999999999995},"min":{"1min":0.72899999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.6619999999999999},"last":1.121}]},{"osd":3,"last update":"Wed Oct 11 09:51:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0029999999999999,"15min":1.0089999999999999},"min":{"1min":0.65700000000000003,"5min":0.57799999999999996,"15min":0.46600000000000003},"max":{"1min":1.51,"5min":1.714,"15min":1.714},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.95099999999999996,"15min":0.96999999999999997},"min":{"1min":0.58799999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.4930000000000001,"5min":1.611,"15min":1.611},"last":1.002}]},{"osd":4,"last update":"Wed Oct 11 09:51:47 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.107,"15min":1.1399999999999999},"min":{"1min":0.89700000000000002,"5min":0.77400000000000002,"15min":0.73599999999999999},"max":{"1min":1.5349999999999999,"5min":1.5980000000000001,"15min":2.016},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.2250000000000001,"15min":1.2609999999999999},"min":{"1min":1.002,"5min":0.85699999999999998,"15min":0.80000000000000004},"max":{"1min":1.748,"5min":1.748,"15min":2.323},"last":1.0589999999999999}]},{"osd":5,"last update":"Wed Oct 11 09:51:47 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.1930000000000001,"15min":1.258},"min":{"1min":0.98999999999999999,"5min":0.79300000000000004,"15min":0.65800000000000003},"max":{"1min":1.552,"5min":1.722,"15min":2.4079999999999999},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.177,"15min":1.2210000000000001},"min":{"1min":0.84799999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.5660000000000001,"5min":1.677,"15min":2.1349999999999998},"last":1.163}]},{"osd":6,"last update":"Wed Oct 11 09:51:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.244,"15min":1.304},"min":{"1min":0.95399999999999996,"5min":0.84599999999999997,"15min":0.71499999999999997},"max":{"1min":1.613,"5min":2.4620000000000002,"15min":2.8940000000000001},"last":1.252},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2310000000000001,"15min":1.3009999999999999},"min":{"1min":0.877,"5min":0.84799999999999998,"15min":0.79200000000000004},"max":{"1min":1.573,"5min":1.8129999999999999,"15min":3.3719999999999999},"last":1.02}]},{"osd":7,"last update":"Wed Oct 11 09:51:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.3220000000000001,"15min":1.363},"min":{"1min":1.044,"5min":0.86899999999999999,"15min":0.86099999999999999},"max":{"1min":1.7230000000000001,"5min":1.7929999999999999,"15min":2.194},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.381,"5min":1.3169999999999999,"15min":1.377},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.7390000000000001,"5min":1.857,"15min":1.968},"last":1.4239999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574998,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33236,"kb_used_data":2188,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338604,"statfs":{"total":96636764160,"available":96602730496,"internally_reserved":0,"allocated":2240512,"data_stored":1390705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.83999999999999997,"15min":0.80800000000000005},"min":{"1min":0.65800000000000003,"5min":0.45300000000000001,"15min":0.38500000000000001},"max":{"1min":1.3160000000000001,"5min":1.611,"15min":1.611},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.88200000000000001,"15min":0.89300000000000002},"min":{"1min":0.60399999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":2.3100000000000001},"last":0.97799999999999998}]},{"osd":2,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.95699999999999996,"15min":0.93600000000000005},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.3160000000000001,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.002,"5min":0.95699999999999996,"15min":0.95799999999999996},"min":{"1min":0.56299999999999994,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.306,"5min":1.569,"15min":1.9450000000000001},"last":1.0569999999999999}]},{"osd":3,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.81100000000000005,"15min":0.83199999999999996},"min":{"1min":0.63900000000000001,"5min":0.48999999999999999,"15min":0.45600000000000002},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":2.7639999999999998},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.95099999999999996,"15min":0.97999999999999998},"min":{"1min":0.73799999999999999,"5min":0.51400000000000001,"15min":0.48999999999999999},"max":{"1min":1.3879999999999999,"5min":1.482,"15min":2.1789999999999998},"last":0.91600000000000004}]},{"osd":4,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.131,"15min":1.163},"min":{"1min":0.79100000000000004,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.4450000000000001,"5min":1.6659999999999999,"15min":1.792},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.149,"5min":1.1379999999999999,"15min":1.161},"min":{"1min":0.94499999999999995,"5min":0.755,"15min":0.69799999999999995},"max":{"1min":1.4039999999999999,"5min":1.9399999999999999,"15min":2.6629999999999998},"last":1.137}]},{"osd":5,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.234,"15min":1.2769999999999999},"min":{"1min":0.99099999999999999,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.4810000000000001,"5min":1.829,"15min":3.0150000000000001},"last":1.024},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.161,"15min":1.2430000000000001},"min":{"1min":0.85599999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.4319999999999999,"5min":1.48,"15min":3.601},"last":1.167}]},{"osd":6,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.151,"15min":1.1870000000000001},"min":{"1min":0.87,"5min":0.79100000000000004,"15min":0.78000000000000003},"max":{"1min":1.46,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.28,"15min":1.2949999999999999},"min":{"1min":0.98699999999999999,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":1.5269999999999999,"5min":1.7430000000000001,"15min":2.1440000000000001},"last":1.095}]},{"osd":7,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2410000000000001,"15min":1.282},"min":{"1min":0.97899999999999998,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.5680000000000001,"5min":1.607,"15min":1.8759999999999999},"last":1.121},{"interface":"front","average":{"1min":1.232,"5min":1.2350000000000001,"15min":1.278},"min":{"1min":1.02,"5min":0.88800000000000001,"15min":0.73899999999999999},"max":{"1min":1.4710000000000001,"5min":2.016,"15min":2.5419999999999998},"last":1.3109999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-11T09:52:48.851 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-11T09:52:48.851 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-11T09:52:49.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.019+0000 7f8a752ee700 1 Processor -- start 2023-10-11T09:52:49.028 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.019+0000 7f8a752ee700 1 -- start start 2023-10-11T09:52:49.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.023+0000 7f8a752ee700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8a7015a800 0x7f8a700b4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:49.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.023+0000 7f8a752ee700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f8a700b0d20 0x7f8a7015e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:49.031 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.023+0000 7f8a752ee700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a70070dc0 con 0x7f8a7006bb60 2023-10-11T09:52:49.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.023+0000 7f8a752ee700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a700b46a0 con 0x7f8a700b0d20 2023-10-11T09:52:49.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.023+0000 7f8a752ee700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a70070ff0 con 0x7f8a7015a800 2023-10-11T09:52:49.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.023+0000 7f8a6f7fe700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f8a700b0d20 0x7f8a7015e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:49.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.023+0000 7f8a6effd700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8a7015a800 0x7f8a700b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:49.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.023+0000 7f8a6effd700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8a7015a800 0x7f8a700b4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.78:39356/0 (socket says 172.21.15.78:39356) 2023-10-11T09:52:49.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.023+0000 7f8a6effd700 1 -- 172.21.15.78:0/1022900744 learned_addr learned my addr 172.21.15.78:0/1022900744 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:49.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.023+0000 7f8a6f7fe700 1 -- 172.21.15.78:0/1022900744 >> v2:172.21.15.133:3300/0 conn(0x7f8a7015a800 msgr2=0x7f8a700b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:49.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.023+0000 7f8a6f7fe700 1 --2- 172.21.15.78:0/1022900744 >> v2:172.21.15.133:3300/0 conn(0x7f8a7015a800 0x7f8a700b4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:49.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.023+0000 7f8a6f7fe700 1 -- 172.21.15.78:0/1022900744 >> v1:172.21.15.133:6789/0 conn(0x7f8a7006bb60 legacy=0x7f8a700b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:49.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.023+0000 7f8a6f7fe700 1 -- 172.21.15.78:0/1022900744 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a700712d0 con 0x7f8a700b0d20 2023-10-11T09:52:49.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.023+0000 7f8a6effd700 1 --2- 172.21.15.78:0/1022900744 >> v2:172.21.15.133:3300/0 conn(0x7f8a7015a800 0x7f8a700b4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:52:49.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.027+0000 7f8a6f7fe700 1 --2- 172.21.15.78:0/1022900744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a700b0d20 0x7f8a7015e590 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f8a700cbea0 tx=0x7f8a6800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c94823d219431c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:49.036 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.027+0000 7f8a6e7fc700 1 -- 172.21.15.78:0/1022900744 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6800e070 con 0x7f8a700b0d20 2023-10-11T09:52:49.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.027+0000 7f8a6e7fc700 1 -- 172.21.15.78:0/1022900744 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6800bb20 con 0x7f8a700b0d20 2023-10-11T09:52:49.037 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.027+0000 7f8a6e7fc700 1 -- 172.21.15.78:0/1022900744 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a68012430 con 0x7f8a700b0d20 2023-10-11T09:52:49.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.027+0000 7f8a752ee700 1 -- 172.21.15.78:0/1022900744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a700b0d20 msgr2=0x7f8a7015e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:49.039 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.027+0000 7f8a752ee700 1 --2- 172.21.15.78:0/1022900744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a700b0d20 0x7f8a7015e590 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f8a700cbea0 tx=0x7f8a6800b330 comp rx=0 tx=0).stop 2023-10-11T09:52:49.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.027+0000 7f8a752ee700 1 -- 172.21.15.78:0/1022900744 shutdown_connections 2023-10-11T09:52:49.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.027+0000 7f8a752ee700 1 --2- 172.21.15.78:0/1022900744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a700b0d20 0x7f8a7015e590 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:49.040 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.027+0000 7f8a752ee700 1 --2- 172.21.15.78:0/1022900744 >> v2:172.21.15.133:3300/0 conn(0x7f8a7015a800 0x7f8a700b4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:49.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.027+0000 7f8a752ee700 1 -- 172.21.15.78:0/1022900744 >> 172.21.15.78:0/1022900744 conn(0x7f8a700cf820 msgr2=0x7f8a700cb610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:49.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.027+0000 7f8a752ee700 1 -- 172.21.15.78:0/1022900744 shutdown_connections 2023-10-11T09:52:49.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.027+0000 7f8a752ee700 1 -- 172.21.15.78:0/1022900744 wait complete. 2023-10-11T09:52:49.041 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.027+0000 7f8a752ee700 1 Processor -- start 2023-10-11T09:52:49.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.031+0000 7f8a752ee700 1 -- start start 2023-10-11T09:52:49.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.031+0000 7f8a752ee700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a7006bb60 0x7f8a701502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:49.042 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.031+0000 7f8a752ee700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a700b0d20 0x7f8a70153430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:49.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.031+0000 7f8a752ee700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a7015a800 0x7f8a70153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:49.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.031+0000 7f8a752ee700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a700c37e0 con 0x7f8a7006bb60 2023-10-11T09:52:49.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.031+0000 7f8a752ee700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a700c2740 con 0x7f8a700b0d20 2023-10-11T09:52:49.043 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.031+0000 7f8a752ee700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a700c2a20 con 0x7f8a7015a800 2023-10-11T09:52:49.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.031+0000 7f8a6effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a7006bb60 0x7f8a701502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:49.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.031+0000 7f8a6effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a7006bb60 0x7f8a701502b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48340/0 (socket says 172.21.15.78:48340) 2023-10-11T09:52:49.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.031+0000 7f8a6effd700 1 -- 172.21.15.78:0/3672255598 learned_addr learned my addr 172.21.15.78:0/3672255598 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:49.044 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.031+0000 7f8a6f7fe700 1 --2- 172.21.15.78:0/3672255598 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a7015a800 0x7f8a70153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:49.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.031+0000 7f8a667fc700 1 --2- 172.21.15.78:0/3672255598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a700b0d20 0x7f8a70153430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:49.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.031+0000 7f8a6effd700 1 -- 172.21.15.78:0/3672255598 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a7015a800 msgr2=0x7f8a70153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:49.045 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.031+0000 7f8a6effd700 1 --2- 172.21.15.78:0/3672255598 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a7015a800 0x7f8a70153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:49.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.031+0000 7f8a6effd700 1 -- 172.21.15.78:0/3672255598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a700b0d20 msgr2=0x7f8a70153430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:49.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.031+0000 7f8a6effd700 1 --2- 172.21.15.78:0/3672255598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a700b0d20 0x7f8a70153430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:49.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.031+0000 7f8a6effd700 1 -- 172.21.15.78:0/3672255598 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a60009570 con 0x7f8a7006bb60 2023-10-11T09:52:49.046 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.031+0000 7f8a6f7fe700 1 --2- 172.21.15.78:0/3672255598 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a7015a800 0x7f8a70153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:52:49.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.031+0000 7f8a667fc700 1 --2- 172.21.15.78:0/3672255598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a700b0d20 0x7f8a70153430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T09:52:49.047 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.031+0000 7f8a6effd700 1 --2- 172.21.15.78:0/3672255598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a7006bb60 0x7f8a701502b0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000ebe0 tx=0x7f8a60009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:49.048 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.031+0000 7f8a67fff700 1 -- 172.21.15.78:0/3672255598 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6000c910 con 0x7f8a7006bb60 2023-10-11T09:52:49.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.031+0000 7f8a67fff700 1 -- 172.21.15.78:0/3672255598 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6000ca70 con 0x7f8a7006bb60 2023-10-11T09:52:49.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.031+0000 7f8a67fff700 1 -- 172.21.15.78:0/3672255598 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6000cde0 con 0x7f8a7006bb60 2023-10-11T09:52:49.049 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.031+0000 7f8a752ee700 1 -- 172.21.15.78:0/3672255598 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a68009a70 con 0x7f8a7006bb60 2023-10-11T09:52:49.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.031+0000 7f8a752ee700 1 -- 172.21.15.78:0/3672255598 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a700c2fe0 con 0x7f8a7006bb60 2023-10-11T09:52:49.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.035+0000 7f8a752ee700 1 -- 172.21.15.78:0/3672255598 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a70067d00 con 0x7f8a7006bb60 2023-10-11T09:52:49.050 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.035+0000 7f8a67fff700 1 -- 172.21.15.78:0/3672255598 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f8a60013470 con 0x7f8a7006bb60 2023-10-11T09:52:49.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.039+0000 7f8a67fff700 1 --2- 172.21.15.78:0/3672255598 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8a50065ed0 0x7f8a50068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:49.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.039+0000 7f8a67fff700 1 -- 172.21.15.78:0/3672255598 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f8a60083480 con 0x7f8a7006bb60 2023-10-11T09:52:49.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.039+0000 7f8a667fc700 1 --2- 172.21.15.78:0/3672255598 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8a50065ed0 0x7f8a50068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:49.051 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.039+0000 7f8a667fc700 1 --2- 172.21.15.78:0/3672255598 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8a50065ed0 0x7f8a50068380 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f8a700c3ad0 tx=0x7f8a58009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:49.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.043+0000 7f8a67fff700 1 -- 172.21.15.78:0/3672255598 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f8a60050d10 con 0x7f8a7006bb60 2023-10-11T09:52:49.219 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.211+0000 7f8a752ee700 1 -- 172.21.15.78:0/3672255598 --> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8a70071890 con 0x7f8a50065ed0 2023-10-11T09:52:49.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.219+0000 7f8a67fff700 1 -- 172.21.15.78:0/3672255598 <== mgr.4101 v2:172.21.15.78:6832/35606 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70642 (secure 0 0 0) 0x7f8a580123d0 con 0x7f8a50065ed0 2023-10-11T09:52:49.227 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:52:49.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.223+0000 7f8a752ee700 1 -- 172.21.15.78:0/3672255598 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8a50065ed0 msgr2=0x7f8a50068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:49.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.223+0000 7f8a752ee700 1 --2- 172.21.15.78:0/3672255598 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8a50065ed0 0x7f8a50068380 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f8a700c3ad0 tx=0x7f8a58009040 comp rx=0 tx=0).stop 2023-10-11T09:52:49.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.223+0000 7f8a752ee700 1 -- 172.21.15.78:0/3672255598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a7006bb60 msgr2=0x7f8a701502b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:49.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.223+0000 7f8a752ee700 1 --2- 172.21.15.78:0/3672255598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a7006bb60 0x7f8a701502b0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000ebe0 tx=0x7f8a60009e90 comp rx=0 tx=0).stop 2023-10-11T09:52:49.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.223+0000 7f8a752ee700 1 -- 172.21.15.78:0/3672255598 shutdown_connections 2023-10-11T09:52:49.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.223+0000 7f8a752ee700 1 --2- 172.21.15.78:0/3672255598 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f8a50065ed0 0x7f8a50068380 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:49.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.223+0000 7f8a752ee700 1 --2- 172.21.15.78:0/3672255598 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a7015a800 0x7f8a70153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:49.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.223+0000 7f8a752ee700 1 --2- 172.21.15.78:0/3672255598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a700b0d20 0x7f8a70153430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:49.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.223+0000 7f8a752ee700 1 --2- 172.21.15.78:0/3672255598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a7006bb60 0x7f8a701502b0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:49.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.223+0000 7f8a752ee700 1 -- 172.21.15.78:0/3672255598 >> 172.21.15.78:0/3672255598 conn(0x7f8a700cf820 msgr2=0x7f8a7006bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:49.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.223+0000 7f8a752ee700 1 -- 172.21.15.78:0/3672255598 shutdown_connections 2023-10-11T09:52:49.235 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.223+0000 7f8a752ee700 1 -- 172.21.15.78:0/3672255598 wait complete. 2023-10-11T09:52:49.236 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2023-10-11T09:52:49.251 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":495,"stamp":"2023-10-11T09:52:48.585970+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":31,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":262408,"kb_used_data":14024,"kb_used_omap":0,"kb_used_meta":248320,"kb_avail":754712312,"statfs":{"total":773094113280,"available":772825407488,"internally_reserved":0,"allocated":14360576,"data_stored":7583432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":254279680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-407665,"num_objects":-125,"num_object_clones":0,"num_object_copies":-250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-125,"num_whiteouts":0,"num_read":-1808,"num_read_kb":-1515,"num_write":-914,"num_write_kb":-953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.917857"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":99,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-11T09:52:44.009882+0000","last_change":"2023-10-11T09:40:36.151839+0000","last_active":"2023-10-11T09:52:44.009882+0000","last_peered":"2023-10-11T09:52:44.009882+0000","last_clean":"2023-10-11T09:52:44.009882+0000","last_became_active":"2023-10-11T09:40:36.151179+0000","last_became_peered":"2023-10-11T09:40:36.151179+0000","last_unstale":"2023-10-11T09:52:44.009882+0000","last_undegraded":"2023-10-11T09:52:44.009882+0000","last_fullsized":"2023-10-11T09:52:44.009882+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":106,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-11T09:52:44.008721+0000","last_change":"2023-10-11T09:40:36.150224+0000","last_active":"2023-10-11T09:52:44.008721+0000","last_peered":"2023-10-11T09:52:44.008721+0000","last_clean":"2023-10-11T09:52:44.008721+0000","last_became_active":"2023-10-11T09:40:36.149919+0000","last_became_peered":"2023-10-11T09:40:36.149919+0000","last_unstale":"2023-10-11T09:52:44.008721+0000","last_undegraded":"2023-10-11T09:52:44.008721+0000","last_fullsized":"2023-10-11T09:52:44.008721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":128,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-11T09:52:43.706994+0000","last_change":"2023-10-11T09:40:36.162174+0000","last_active":"2023-10-11T09:52:43.706994+0000","last_peered":"2023-10-11T09:52:43.706994+0000","last_clean":"2023-10-11T09:52:43.706994+0000","last_became_active":"2023-10-11T09:40:36.161710+0000","last_became_peered":"2023-10-11T09:40:36.161710+0000","last_unstale":"2023-10-11T09:52:43.706994+0000","last_undegraded":"2023-10-11T09:52:43.706994+0000","last_fullsized":"2023-10-11T09:52:43.706994+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:57:03.630878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":106,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-11T09:52:44.010145+0000","last_change":"2023-10-11T09:40:36.160620+0000","last_active":"2023-10-11T09:52:44.010145+0000","last_peered":"2023-10-11T09:52:44.010145+0000","last_clean":"2023-10-11T09:52:44.010145+0000","last_became_active":"2023-10-11T09:40:36.160171+0000","last_became_peered":"2023-10-11T09:40:36.160171+0000","last_unstale":"2023-10-11T09:52:44.010145+0000","last_undegraded":"2023-10-11T09:52:44.010145+0000","last_fullsized":"2023-10-11T09:52:44.010145+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":104,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-11T09:52:44.009611+0000","last_change":"2023-10-11T09:40:36.151710+0000","last_active":"2023-10-11T09:52:44.009611+0000","last_peered":"2023-10-11T09:52:44.009611+0000","last_clean":"2023-10-11T09:52:44.009611+0000","last_became_active":"2023-10-11T09:40:36.151219+0000","last_became_peered":"2023-10-11T09:40:36.151219+0000","last_unstale":"2023-10-11T09:52:44.009611+0000","last_undegraded":"2023-10-11T09:52:44.009611+0000","last_fullsized":"2023-10-11T09:52:44.009611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":106,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-11T09:52:44.004783+0000","last_change":"2023-10-11T09:40:36.216248+0000","last_active":"2023-10-11T09:52:44.004783+0000","last_peered":"2023-10-11T09:52:44.004783+0000","last_clean":"2023-10-11T09:52:44.004783+0000","last_became_active":"2023-10-11T09:40:36.215890+0000","last_became_peered":"2023-10-11T09:40:36.215890+0000","last_unstale":"2023-10-11T09:52:44.004783+0000","last_undegraded":"2023-10-11T09:52:44.004783+0000","last_fullsized":"2023-10-11T09:52:44.004783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":129,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-11T09:52:43.703936+0000","last_change":"2023-10-11T09:40:36.149739+0000","last_active":"2023-10-11T09:52:43.703936+0000","last_peered":"2023-10-11T09:52:43.703936+0000","last_clean":"2023-10-11T09:52:43.703936+0000","last_became_active":"2023-10-11T09:40:36.149419+0000","last_became_peered":"2023-10-11T09:40:36.149419+0000","last_unstale":"2023-10-11T09:52:43.703936+0000","last_undegraded":"2023-10-11T09:52:43.703936+0000","last_fullsized":"2023-10-11T09:52:43.703936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:54:50.640362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":104,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-11T09:52:44.010199+0000","last_change":"2023-10-11T09:40:36.148773+0000","last_active":"2023-10-11T09:52:44.010199+0000","last_peered":"2023-10-11T09:52:44.010199+0000","last_clean":"2023-10-11T09:52:44.010199+0000","last_became_active":"2023-10-11T09:40:36.148489+0000","last_became_peered":"2023-10-11T09:40:36.148489+0000","last_unstale":"2023-10-11T09:52:44.010199+0000","last_undegraded":"2023-10-11T09:52:44.010199+0000","last_fullsized":"2023-10-11T09:52:44.010199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":282,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-11T09:52:43.704492+0000","last_change":"2023-10-11T09:40:29.874372+0000","last_active":"2023-10-11T09:52:43.704492+0000","last_peered":"2023-10-11T09:52:43.704492+0000","last_clean":"2023-10-11T09:52:43.704492+0000","last_became_active":"2023-10-11T09:40:29.873621+0000","last_became_peered":"2023-10-11T09:40:29.873621+0000","last_unstale":"2023-10-11T09:52:43.704492+0000","last_undegraded":"2023-10-11T09:52:43.704492+0000","last_fullsized":"2023-10-11T09:52:43.704492+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_clean_scrub_stamp":"2023-10-11T09:40:28.742276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:22:07.985311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575000,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33236,"kb_used_data":2188,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338604,"statfs":{"total":96636764160,"available":96602730496,"internally_reserved":0,"allocated":2240512,"data_stored":1390705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.056,"15min":1.087},"min":{"1min":0.68999999999999995,"5min":0.56499999999999995,"15min":0.48999999999999999},"max":{"1min":1.4730000000000001,"5min":1.7390000000000001,"15min":1.903},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.034,"5min":1.0329999999999999,"15min":1.0860000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.50900000000000001},"max":{"1min":1.4970000000000001,"5min":1.554,"15min":1.696},"last":0.91600000000000004}]},{"osd":1,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.1140000000000001,"15min":1.123},"min":{"1min":0.91000000000000003,"5min":0.66800000000000004,"15min":0.63100000000000001},"max":{"1min":1.4059999999999999,"5min":1.645,"15min":2.25},"last":1.125},{"interface":"front","average":{"1min":1.167,"5min":1.0489999999999999,"15min":1.0920000000000001},"min":{"1min":0.93300000000000005,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.4950000000000001,"5min":1.587,"15min":1.696},"last":1.7090000000000001}]},{"osd":2,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.0840000000000001,"15min":1.103},"min":{"1min":0.72899999999999998,"5min":0.59499999999999997,"15min":0.58999999999999997},"max":{"1min":1.411,"5min":1.5049999999999999,"15min":1.873},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.1719999999999999,"15min":1.155},"min":{"1min":0.72499999999999998,"5min":0.66400000000000003,"15min":0.65500000000000003},"max":{"1min":1.579,"5min":1.78,"15min":1.78},"last":1.196}]},{"osd":3,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.216,"15min":1.22},"min":{"1min":0.89200000000000002,"5min":0.80000000000000004,"15min":0.64900000000000002},"max":{"1min":1.617,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.387},{"interface":"front","average":{"1min":1.302,"5min":1.284,"15min":1.2809999999999999},"min":{"1min":0.94699999999999995,"5min":0.747,"15min":0.61699999999999999},"max":{"1min":1.54,"5min":1.8720000000000001,"15min":2.1579999999999999},"last":1.29}]},{"osd":4,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.0609999999999999,"15min":1.1200000000000001},"min":{"1min":0.71999999999999997,"5min":0.64900000000000002,"15min":0.53300000000000003},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":2.2010000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.181,"5min":1.208,"15min":1.226},"min":{"1min":0.93100000000000005,"5min":0.67200000000000004,"15min":0.63500000000000001},"max":{"1min":1.4199999999999999,"5min":1.768,"15min":1.9339999999999999},"last":1.335}]},{"osd":5,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.2709999999999999,"15min":1.278},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.7350000000000001,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2050000000000001,"15min":1.2749999999999999},"min":{"1min":0.82999999999999996,"5min":0.78300000000000003,"15min":0.65900000000000003},"max":{"1min":1.714,"5min":1.9450000000000001,"15min":3.0750000000000002},"last":1.1559999999999999}]},{"osd":6,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.3300000000000001,"15min":1.3919999999999999},"min":{"1min":0.98599999999999999,"5min":0.77000000000000002,"15min":0.70999999999999996},"max":{"1min":2,"5min":2,"15min":3.137},"last":1.222},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3500000000000001,"15min":1.381},"min":{"1min":0.89700000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.897,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.4319999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574999,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1608,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1646592,"data_stored":800337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.125,"15min":1.036},"min":{"1min":0.626,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.403,"5min":7.0730000000000004,"15min":7.0730000000000004},"last":0.86899999999999999},{"interface":"front","average":{"1min":1.01,"5min":1.179,"15min":1.071},"min":{"1min":0.65900000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.357,"5min":8.8450000000000006,"15min":8.8450000000000006},"last":1.145}]},{"osd":1,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.359,"15min":1.232},"min":{"1min":0.627,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.5149999999999999,"5min":15.523,"15min":15.523},"last":1.45},{"interface":"front","average":{"1min":1.181,"5min":1.3069999999999999,"15min":1.2310000000000001},"min":{"1min":0.80700000000000005,"5min":0.59199999999999997,"15min":0.53400000000000003},"max":{"1min":1.5509999999999999,"5min":15.253,"15min":15.253},"last":1.081}]},{"osd":2,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.1200000000000001,"15min":1.1319999999999999},"min":{"1min":0.745,"5min":0.434,"15min":0.434},"max":{"1min":1.335,"5min":2.7919999999999998,"15min":2.7919999999999998},"last":1.375},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1930000000000001,"15min":1.1890000000000001},"min":{"1min":0.79400000000000004,"5min":0.61599999999999999,"15min":0.44500000000000001},"max":{"1min":1.6060000000000001,"5min":5.0990000000000002,"15min":5.0990000000000002},"last":1.004}]},{"osd":3,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.2150000000000001,"15min":1.155},"min":{"1min":0.72599999999999998,"5min":0.70599999999999996,"15min":0.627},"max":{"1min":1.421,"5min":11.945,"15min":11.945},"last":1.228},{"interface":"front","average":{"1min":1.246,"5min":1.175,"15min":1.2010000000000001},"min":{"1min":0.84299999999999997,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.659,"5min":1.659,"15min":3.0819999999999999},"last":1.629}]},{"osd":4,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.2889999999999999,"15min":1.2769999999999999},"min":{"1min":0.626,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.6359999999999999,"5min":7.5599999999999996,"15min":7.5599999999999996},"last":0.70399999999999996},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.3380000000000001,"15min":1.3069999999999999},"min":{"1min":0.81299999999999994,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.542,"5min":8.7520000000000007,"15min":8.7520000000000007},"last":1.0309999999999999}]},{"osd":5,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.196,"15min":1.2290000000000001},"min":{"1min":0.65900000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.585,"5min":2.1560000000000001,"15min":2.4009999999999998},"last":1.262},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.282,"15min":1.3200000000000001},"min":{"1min":0.68700000000000006,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.978,"5min":2.4390000000000001,"15min":2.7290000000000001},"last":0.93700000000000006}]},{"osd":7,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.335,"15min":1.3440000000000001},"min":{"1min":0.91400000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.706,"5min":8.7970000000000006,"15min":8.7970000000000006},"last":1.302},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.3899999999999999,"15min":1.4199999999999999},"min":{"1min":1.01,"5min":0.625,"15min":0.625},"max":{"1min":1.8620000000000001,"5min":2.641,"15min":2.641},"last":1.54}]}]},{"osd":5,"up_from":13,"seq":55834574999,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1608,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1646592,"data_stored":800337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.044,"15min":1.0129999999999999},"min":{"1min":0.69799999999999995,"5min":0.64100000000000001,"15min":0.45300000000000001},"max":{"1min":1.4770000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.125,"5min":1.087,"15min":1.0269999999999999},"min":{"1min":0.66200000000000003,"5min":0.47699999999999998,"15min":0.46500000000000002},"max":{"1min":1.3460000000000001,"5min":1.536,"15min":1.7649999999999999},"last":1.3089999999999999}]},{"osd":1,"last update":"Wed Oct 11 09:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.1160000000000001,"15min":1.034},"min":{"1min":0.69599999999999995,"5min":0.68100000000000005,"15min":0.49399999999999999},"max":{"1min":1.542,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.874},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.0900000000000001},"min":{"1min":0.71599999999999997,"5min":0.61599999999999999,"15min":0.437},"max":{"1min":1.4299999999999999,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.508}]},{"osd":2,"last update":"Wed Oct 11 09:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1819999999999999,"15min":1.179},"min":{"1min":0.76100000000000001,"5min":0.746,"15min":0.73899999999999999},"max":{"1min":1.5009999999999999,"5min":2.133,"15min":2.133},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2370000000000001,"15min":1.2190000000000001},"min":{"1min":0.94099999999999995,"5min":0.64200000000000002,"15min":0.52300000000000002},"max":{"1min":1.46,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.90100000000000002}]},{"osd":3,"last update":"Wed Oct 11 09:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.80700000000000005,"5min":0.752,"15min":0.56999999999999995},"max":{"1min":1.3580000000000001,"5min":1.8859999999999999,"15min":1.921},"last":1.087},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.1950000000000001,"15min":1.1659999999999999},"min":{"1min":0.92500000000000004,"5min":0.71999999999999997,"15min":0.55300000000000005},"max":{"1min":1.492,"5min":1.9690000000000001,"15min":2.0209999999999999},"last":1.2609999999999999}]},{"osd":4,"last update":"Wed Oct 11 09:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.1919999999999999,"15min":1.1859999999999999},"min":{"1min":0.61399999999999999,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.4139999999999999,"5min":2.0169999999999999,"15min":2.117},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.075,"5min":1.157,"15min":1.1759999999999999},"min":{"1min":0.81999999999999995,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.4159999999999999,"5min":1.9690000000000001,"15min":2.1749999999999998},"last":1.2090000000000001}]},{"osd":6,"last update":"Wed Oct 11 09:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.228,"15min":1.25},"min":{"1min":0.75700000000000001,"5min":0.69499999999999995,"15min":0.59499999999999997},"max":{"1min":1.5449999999999999,"5min":1.9239999999999999,"15min":2.4529999999999998},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.252,"5min":1.331,"15min":1.3200000000000001},"min":{"1min":0.78700000000000003,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.478,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.385}]},{"osd":7,"last update":"Wed Oct 11 09:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.3320000000000001,"15min":1.341},"min":{"1min":1.0069999999999999,"5min":0.58799999999999997,"15min":0.57699999999999996},"max":{"1min":1.502,"5min":2.347,"15min":3.6120000000000001},"last":1.456},{"interface":"front","average":{"1min":1.353,"5min":1.4319999999999999,"15min":1.427},"min":{"1min":0.98499999999999999,"5min":0.89100000000000001,"15min":0.71299999999999997},"max":{"1min":1.589,"5min":2.2679999999999998,"15min":3.2109999999999999},"last":1.4279999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574999,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32656,"kb_used_data":1608,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1646592,"data_stored":800337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.071,"15min":1.0640000000000001},"min":{"1min":0.74399999999999999,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":2.0369999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.133,"5min":1.0760000000000001,"15min":1.0880000000000001},"min":{"1min":0.90000000000000002,"5min":0.60199999999999998,"15min":0.54700000000000004},"max":{"1min":1.5129999999999999,"5min":1.669,"15min":2.1160000000000001},"last":1.089}]},{"osd":1,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.109,"15min":1.089},"min":{"1min":0.72699999999999998,"5min":0.66500000000000004,"15min":0.60699999999999998},"max":{"1min":1.95,"5min":1.95,"15min":1.95},"last":1.111},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.159,"15min":1.1499999999999999},"min":{"1min":0.77800000000000002,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":2.02,"5min":2.02,"15min":2.02},"last":1.2649999999999999}]},{"osd":2,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.232,"15min":1.2030000000000001},"min":{"1min":0.72499999999999998,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.5859999999999999,"5min":2.5859999999999999,"15min":2.5859999999999999},"last":1.163},{"interface":"front","average":{"1min":1.379,"5min":1.2589999999999999,"15min":1.228},"min":{"1min":0.69699999999999995,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.5350000000000001,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.4570000000000001}]},{"osd":3,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.1140000000000001,"15min":1.1240000000000001},"min":{"1min":0.79500000000000004,"5min":0.72599999999999998,"15min":0.60199999999999998},"max":{"1min":1.353,"5min":1.6839999999999999,"15min":1.6930000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.1859999999999999,"15min":1.236},"min":{"1min":0.83799999999999997,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.5030000000000001,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":0.88600000000000001}]},{"osd":5,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.1830000000000001,"15min":1.1879999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":1.361},{"interface":"front","average":{"1min":1.175,"5min":1.1599999999999999,"15min":1.181},"min":{"1min":0.76200000000000001,"5min":0.63300000000000001,"15min":0.61199999999999999},"max":{"1min":1.6539999999999999,"5min":1.732,"15min":1.8520000000000001},"last":1.2929999999999999}]},{"osd":6,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.2390000000000001,"15min":1.252},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3149999999999999,"15min":1.3169999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.0830000000000002,"5min":2.0830000000000002,"15min":2.1120000000000001},"last":1.5489999999999999}]},{"osd":7,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.2629999999999999,"15min":1.296},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.60899999999999999},"max":{"1min":1.77,"5min":1.794,"15min":1.9470000000000001},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.371,"15min":1.4079999999999999},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.75900000000000001},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.4239999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574998,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1608,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1646592,"data_stored":800337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.76800000000000002,"15min":0.79700000000000004},"min":{"1min":0.67100000000000004,"5min":0.442,"15min":0.33000000000000002},"max":{"1min":1.163,"5min":1.399,"15min":1.462},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.82399999999999995,"15min":0.83299999999999996},"min":{"1min":0.47099999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.3400000000000001,"5min":1.363,"15min":1.4950000000000001},"last":0.80800000000000005}]},{"osd":1,"last update":"Wed Oct 11 09:51:49 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.80600000000000005,"15min":0.81200000000000006},"min":{"1min":0.51000000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.2629999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.01,"5min":0.91300000000000003,"15min":0.92500000000000004},"min":{"1min":0.56200000000000006,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.3069999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.88200000000000001}]},{"osd":2,"last update":"Wed Oct 11 09:51:49 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":0.96399999999999997,"15min":0.96899999999999997},"min":{"1min":0.67400000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.4570000000000001,"5min":1.601,"15min":1.601},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.0249999999999999,"15min":1.0549999999999999},"min":{"1min":0.81599999999999995,"5min":0.58499999999999996,"15min":0.432},"max":{"1min":1.4399999999999999,"5min":1.607,"15min":2.5870000000000002},"last":0.96099999999999997}]},{"osd":4,"last update":"Wed Oct 11 09:51:49 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.1060000000000001,"15min":1.1579999999999999},"min":{"1min":0.77900000000000003,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.556,"5min":1.556,"15min":2.395},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.26,"5min":1.1859999999999999,"15min":1.236},"min":{"1min":0.747,"5min":0.747,"15min":0.73599999999999999},"max":{"1min":1.579,"5min":1.9099999999999999,"15min":3.1579999999999999},"last":1.133}]},{"osd":5,"last update":"Wed Oct 11 09:51:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.226,"15min":1.268},"min":{"1min":0.91200000000000003,"5min":0.84199999999999997,"15min":0.76500000000000001},"max":{"1min":1.476,"5min":1.724,"15min":3.0009999999999999},"last":1.409},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.2430000000000001,"15min":1.3240000000000001},"min":{"1min":1.0309999999999999,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.651,"5min":1.651,"15min":3.8050000000000002},"last":1.1890000000000001}]},{"osd":6,"last update":"Wed Oct 11 09:51:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.216,"15min":1.248},"min":{"1min":0.871,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.514,"5min":1.6890000000000001,"15min":2.1949999999999998},"last":1.171},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.272,"15min":1.3280000000000001},"min":{"1min":0.995,"5min":0.91700000000000004,"15min":0.84499999999999997},"max":{"1min":1.556,"5min":1.782,"15min":3.2549999999999999},"last":1.492}]},{"osd":7,"last update":"Wed Oct 11 09:51:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.2849999999999999,"15min":1.3660000000000001},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.76700000000000002},"max":{"1min":1.587,"5min":1.7150000000000001,"15min":3.7490000000000001},"last":1.371},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.2729999999999999,"15min":1.323},"min":{"1min":0.96499999999999997,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.7130000000000001,"5min":1.7969999999999999,"15min":2.625},"last":1.3400000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575000,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1608,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1646592,"data_stored":800337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:51:48 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.80300000000000005,"15min":0.80800000000000005},"min":{"1min":0.65100000000000002,"5min":0.40600000000000003,"15min":0.38500000000000001},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":2.0600000000000001},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.86499999999999999,"15min":0.89500000000000002},"min":{"1min":0.67400000000000004,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":2.2850000000000001,"5min":2.2850000000000001,"15min":2.8919999999999999},"last":0.82899999999999996}]},{"osd":1,"last update":"Wed Oct 11 09:51:48 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.84399999999999997,"15min":0.93500000000000005},"min":{"1min":0.61699999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.621,"5min":1.8069999999999999,"15min":3.7650000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.96599999999999997,"15min":0.96899999999999997},"min":{"1min":0.75600000000000001,"5min":0.45600000000000002,"15min":0.40300000000000002},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":2.5339999999999998},"last":1.272}]},{"osd":3,"last update":"Wed Oct 11 09:51:48 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.91100000000000003,"15min":1.006},"min":{"1min":0.70099999999999996,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.3600000000000001,"5min":1.591,"15min":2.403},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.044,"15min":1.044},"min":{"1min":0.88400000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5660000000000001,"5min":1.754,"15min":2.7589999999999999},"last":1.343}]},{"osd":4,"last update":"Wed Oct 11 09:51:48 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.1679999999999999,"15min":1.1850000000000001},"min":{"1min":0.90500000000000003,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.6040000000000001,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.242,"15min":1.3240000000000001},"min":{"1min":1.0640000000000001,"5min":0.751,"15min":0.73399999999999999},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.6859999999999999},"last":1.3009999999999999}]},{"osd":5,"last update":"Wed Oct 11 09:51:48 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.2509999999999999,"15min":1.2949999999999999},"min":{"1min":0.96299999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":3.044},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.248,"5min":1.1919999999999999,"15min":1.248},"min":{"1min":0.83799999999999997,"5min":0.72799999999999998,"15min":0.69599999999999995},"max":{"1min":2.411,"5min":2.411,"15min":3.5790000000000002},"last":1.077}]},{"osd":6,"last update":"Wed Oct 11 09:51:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.216,"15min":1.3069999999999999},"min":{"1min":1.052,"5min":0.75,"15min":0.75},"max":{"1min":1.774,"5min":1.774,"15min":3.1600000000000001},"last":1.224},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3089999999999999,"15min":1.3280000000000001},"min":{"1min":1.054,"5min":0.83499999999999996,"15min":0.623},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":2.839},"last":1.411}]},{"osd":7,"last update":"Wed Oct 11 09:51:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.286,"15min":1.341},"min":{"1min":0.999,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":2.7130000000000001,"5min":2.7130000000000001,"15min":3.1230000000000002},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.333,"15min":1.421},"min":{"1min":1.2749999999999999,"5min":0.84899999999999998,"15min":0.80000000000000004},"max":{"1min":1.7210000000000001,"5min":1.7909999999999999,"15min":2.988},"last":1.367}]}]},{"osd":1,"up_from":13,"seq":55834574999,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1608,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1646592,"data_stored":800337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:51:47 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.79800000000000004,"15min":0.78400000000000003},"min":{"1min":0.51500000000000001,"5min":0.48799999999999999,"15min":0.375},"max":{"1min":1.0900000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.81100000000000005,"15min":0.82099999999999995},"min":{"1min":0.53500000000000003,"5min":0.40100000000000002,"15min":0.36699999999999999},"max":{"1min":1.2210000000000001,"5min":1.593,"15min":1.593},"last":0.88800000000000001}]},{"osd":2,"last update":"Wed Oct 11 09:51:47 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.80300000000000005,"15min":0.83299999999999996},"min":{"1min":0.51700000000000002,"5min":0.437,"15min":0.36199999999999999},"max":{"1min":1.0740000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.92500000000000004,"15min":0.94899999999999995},"min":{"1min":0.72899999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.6619999999999999},"last":1.121}]},{"osd":3,"last update":"Wed Oct 11 09:51:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0029999999999999,"15min":1.0089999999999999},"min":{"1min":0.65700000000000003,"5min":0.57799999999999996,"15min":0.46600000000000003},"max":{"1min":1.51,"5min":1.714,"15min":1.714},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.95099999999999996,"15min":0.96999999999999997},"min":{"1min":0.58799999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.4930000000000001,"5min":1.611,"15min":1.611},"last":1.002}]},{"osd":4,"last update":"Wed Oct 11 09:51:47 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.107,"15min":1.1399999999999999},"min":{"1min":0.89700000000000002,"5min":0.77400000000000002,"15min":0.73599999999999999},"max":{"1min":1.5349999999999999,"5min":1.5980000000000001,"15min":2.016},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.2250000000000001,"15min":1.2609999999999999},"min":{"1min":1.002,"5min":0.85699999999999998,"15min":0.80000000000000004},"max":{"1min":1.748,"5min":1.748,"15min":2.323},"last":1.0589999999999999}]},{"osd":5,"last update":"Wed Oct 11 09:51:47 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.1930000000000001,"15min":1.258},"min":{"1min":0.98999999999999999,"5min":0.79300000000000004,"15min":0.65800000000000003},"max":{"1min":1.552,"5min":1.722,"15min":2.4079999999999999},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.177,"15min":1.2210000000000001},"min":{"1min":0.84799999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.5660000000000001,"5min":1.677,"15min":2.1349999999999998},"last":1.163}]},{"osd":6,"last update":"Wed Oct 11 09:51:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.244,"15min":1.304},"min":{"1min":0.95399999999999996,"5min":0.84599999999999997,"15min":0.71499999999999997},"max":{"1min":1.613,"5min":2.4620000000000002,"15min":2.8940000000000001},"last":1.252},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2310000000000001,"15min":1.3009999999999999},"min":{"1min":0.877,"5min":0.84799999999999998,"15min":0.79200000000000004},"max":{"1min":1.573,"5min":1.8129999999999999,"15min":3.3719999999999999},"last":1.02}]},{"osd":7,"last update":"Wed Oct 11 09:51:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.3220000000000001,"15min":1.363},"min":{"1min":1.044,"5min":0.86899999999999999,"15min":0.86099999999999999},"max":{"1min":1.7230000000000001,"5min":1.7929999999999999,"15min":2.194},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.381,"5min":1.3169999999999999,"15min":1.377},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.7390000000000001,"5min":1.857,"15min":1.968},"last":1.4239999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574998,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33236,"kb_used_data":2188,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338604,"statfs":{"total":96636764160,"available":96602730496,"internally_reserved":0,"allocated":2240512,"data_stored":1390705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.83999999999999997,"15min":0.80800000000000005},"min":{"1min":0.65800000000000003,"5min":0.45300000000000001,"15min":0.38500000000000001},"max":{"1min":1.3160000000000001,"5min":1.611,"15min":1.611},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.88200000000000001,"15min":0.89300000000000002},"min":{"1min":0.60399999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":2.3100000000000001},"last":0.97799999999999998}]},{"osd":2,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.95699999999999996,"15min":0.93600000000000005},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.3160000000000001,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.002,"5min":0.95699999999999996,"15min":0.95799999999999996},"min":{"1min":0.56299999999999994,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.306,"5min":1.569,"15min":1.9450000000000001},"last":1.0569999999999999}]},{"osd":3,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.81100000000000005,"15min":0.83199999999999996},"min":{"1min":0.63900000000000001,"5min":0.48999999999999999,"15min":0.45600000000000002},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":2.7639999999999998},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.95099999999999996,"15min":0.97999999999999998},"min":{"1min":0.73799999999999999,"5min":0.51400000000000001,"15min":0.48999999999999999},"max":{"1min":1.3879999999999999,"5min":1.482,"15min":2.1789999999999998},"last":0.91600000000000004}]},{"osd":4,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.131,"15min":1.163},"min":{"1min":0.79100000000000004,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.4450000000000001,"5min":1.6659999999999999,"15min":1.792},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.149,"5min":1.1379999999999999,"15min":1.161},"min":{"1min":0.94499999999999995,"5min":0.755,"15min":0.69799999999999995},"max":{"1min":1.4039999999999999,"5min":1.9399999999999999,"15min":2.6629999999999998},"last":1.137}]},{"osd":5,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.234,"15min":1.2769999999999999},"min":{"1min":0.99099999999999999,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.4810000000000001,"5min":1.829,"15min":3.0150000000000001},"last":1.024},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.161,"15min":1.2430000000000001},"min":{"1min":0.85599999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.4319999999999999,"5min":1.48,"15min":3.601},"last":1.167}]},{"osd":6,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.151,"15min":1.1870000000000001},"min":{"1min":0.87,"5min":0.79100000000000004,"15min":0.78000000000000003},"max":{"1min":1.46,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.28,"15min":1.2949999999999999},"min":{"1min":0.98699999999999999,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":1.5269999999999999,"5min":1.7430000000000001,"15min":2.1440000000000001},"last":1.095}]},{"osd":7,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2410000000000001,"15min":1.282},"min":{"1min":0.97899999999999998,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.5680000000000001,"5min":1.607,"15min":1.8759999999999999},"last":1.121},{"interface":"front","average":{"1min":1.232,"5min":1.2350000000000001,"15min":1.278},"min":{"1min":1.02,"5min":0.88800000000000001,"15min":0.73899999999999999},"max":{"1min":1.4710000000000001,"5min":2.016,"15min":2.5419999999999998},"last":1.3109999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-11T09:52:49.255 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T09:52:49.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.423+0000 7fac1d671700 1 Processor -- start 2023-10-11T09:52:49.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.423+0000 7fac1d671700 1 -- start start 2023-10-11T09:52:49.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.427+0000 7fac1d671700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac180713b0 0x7fac180c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:49.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.427+0000 7fac1d671700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fac180c5b40 0x7fac180c9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:49.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.427+0000 7fac1d671700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac180ceb90 con 0x7fac180ce7c0 2023-10-11T09:52:49.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.427+0000 7fac1d671700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac1815cef0 con 0x7fac180c5b40 2023-10-11T09:52:49.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.427+0000 7fac1d671700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac18160770 con 0x7fac180713b0 2023-10-11T09:52:49.438 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.427+0000 7fac167fc700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac180713b0 0x7fac180c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:49.439 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.427+0000 7fac167fc700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac180713b0 0x7fac180c94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:43790/0 (socket says 172.21.15.78:43790) 2023-10-11T09:52:49.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.427+0000 7fac167fc700 1 -- 172.21.15.78:0/1278737683 learned_addr learned my addr 172.21.15.78:0/1278737683 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:49.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.427+0000 7fac177fe700 1 --2- 172.21.15.78:0/1278737683 >> v2:172.21.15.133:3300/0 conn(0x7fac180c5b40 0x7fac180c9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:49.440 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.427+0000 7fac167fc700 1 -- 172.21.15.78:0/1278737683 >> v2:172.21.15.133:3300/0 conn(0x7fac180c5b40 msgr2=0x7fac180c9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:49.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.427+0000 7fac167fc700 1 --2- 172.21.15.78:0/1278737683 >> v2:172.21.15.133:3300/0 conn(0x7fac180c5b40 0x7fac180c9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:49.441 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.427+0000 7fac167fc700 1 -- 172.21.15.78:0/1278737683 >> v1:172.21.15.133:6789/0 conn(0x7fac180ce7c0 legacy=0x7fac18160bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:49.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.427+0000 7fac167fc700 1 -- 172.21.15.78:0/1278737683 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac18071820 con 0x7fac180713b0 2023-10-11T09:52:49.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.427+0000 7fac167fc700 1 --2- 172.21.15.78:0/1278737683 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac180713b0 0x7fac180c94f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fac0c009910 tx=0x7fac0c009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffe0232628cab53b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:49.442 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.431+0000 7fac15ffb700 1 -- 172.21.15.78:0/1278737683 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0c011830 con 0x7fac180713b0 2023-10-11T09:52:49.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.431+0000 7fac15ffb700 1 -- 172.21.15.78:0/1278737683 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac0c011990 con 0x7fac180713b0 2023-10-11T09:52:49.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.431+0000 7fac15ffb700 1 -- 172.21.15.78:0/1278737683 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0c011d40 con 0x7fac180713b0 2023-10-11T09:52:49.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.431+0000 7fac1d671700 1 -- 172.21.15.78:0/1278737683 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac180713b0 msgr2=0x7fac180c94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:49.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.431+0000 7fac1d671700 1 --2- 172.21.15.78:0/1278737683 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac180713b0 0x7fac180c94f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fac0c009910 tx=0x7fac0c009c20 comp rx=0 tx=0).stop 2023-10-11T09:52:49.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.431+0000 7fac1d671700 1 -- 172.21.15.78:0/1278737683 shutdown_connections 2023-10-11T09:52:49.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.431+0000 7fac1d671700 1 --2- 172.21.15.78:0/1278737683 >> v2:172.21.15.133:3300/0 conn(0x7fac180c5b40 0x7fac180c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:49.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.431+0000 7fac1d671700 1 --2- 172.21.15.78:0/1278737683 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac180713b0 0x7fac180c94f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:49.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.431+0000 7fac1d671700 1 -- 172.21.15.78:0/1278737683 >> 172.21.15.78:0/1278737683 conn(0x7fac180cb920 msgr2=0x7fac180cbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:49.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.431+0000 7fac1d671700 1 -- 172.21.15.78:0/1278737683 shutdown_connections 2023-10-11T09:52:49.445 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.431+0000 7fac1d671700 1 -- 172.21.15.78:0/1278737683 wait complete. 2023-10-11T09:52:49.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.431+0000 7fac1d671700 1 Processor -- start 2023-10-11T09:52:49.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.435+0000 7fac1d671700 1 -- start start 2023-10-11T09:52:49.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.435+0000 7fac1d671700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac180c5b40 0x7fac18149640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:49.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.435+0000 7fac1d671700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac180ce7c0 0x7fac1815f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:49.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.435+0000 7fac1d671700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac1815e800 0x7fac180ac950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:49.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.435+0000 7fac1d671700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac18146d90 con 0x7fac180ce7c0 2023-10-11T09:52:49.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.435+0000 7fac1d671700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac181478d0 con 0x7fac180c5b40 2023-10-11T09:52:49.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.435+0000 7fac1d671700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac18146fa0 con 0x7fac1815e800 2023-10-11T09:52:49.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.435+0000 7fac167fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac180ce7c0 0x7fac1815f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:49.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.435+0000 7fac167fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac180ce7c0 0x7fac1815f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48348/0 (socket says 172.21.15.78:48348) 2023-10-11T09:52:49.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.435+0000 7fac167fc700 1 -- 172.21.15.78:0/1405499540 learned_addr learned my addr 172.21.15.78:0/1405499540 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:49.450 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.435+0000 7fac16ffd700 1 --2- 172.21.15.78:0/1405499540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac180c5b40 0x7fac18149640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:49.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.435+0000 7fac177fe700 1 --2- 172.21.15.78:0/1405499540 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac1815e800 0x7fac180ac950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:49.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.435+0000 7fac167fc700 1 -- 172.21.15.78:0/1405499540 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac1815e800 msgr2=0x7fac180ac950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:49.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.435+0000 7fac167fc700 1 --2- 172.21.15.78:0/1405499540 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac1815e800 0x7fac180ac950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:49.451 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.435+0000 7fac167fc700 1 -- 172.21.15.78:0/1405499540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac180c5b40 msgr2=0x7fac18149640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:49.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.435+0000 7fac167fc700 1 --2- 172.21.15.78:0/1405499540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac180c5b40 0x7fac18149640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:49.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.435+0000 7fac167fc700 1 -- 172.21.15.78:0/1405499540 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac0c009570 con 0x7fac180ce7c0 2023-10-11T09:52:49.452 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.435+0000 7fac167fc700 1 --2- 172.21.15.78:0/1405499540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac180ce7c0 0x7fac1815f640 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fac0c000c00 tx=0x7fac0c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:49.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.439+0000 7fabff7fe700 1 -- 172.21.15.78:0/1405499540 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0c004450 con 0x7fac180ce7c0 2023-10-11T09:52:49.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.439+0000 7fabff7fe700 1 -- 172.21.15.78:0/1405499540 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac0c0045d0 con 0x7fac180ce7c0 2023-10-11T09:52:49.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.439+0000 7fabff7fe700 1 -- 172.21.15.78:0/1405499540 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0c003900 con 0x7fac180ce7c0 2023-10-11T09:52:49.453 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.439+0000 7fac1d671700 1 -- 172.21.15.78:0/1405499540 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac18147280 con 0x7fac180ce7c0 2023-10-11T09:52:49.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.439+0000 7fac1d671700 1 -- 172.21.15.78:0/1405499540 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac18149b80 con 0x7fac180ce7c0 2023-10-11T09:52:49.454 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.439+0000 7fac1d671700 1 -- 172.21.15.78:0/1405499540 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac18147a40 con 0x7fac180ce7c0 2023-10-11T09:52:49.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.443+0000 7fabff7fe700 1 -- 172.21.15.78:0/1405499540 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fac0c003a60 con 0x7fac180ce7c0 2023-10-11T09:52:49.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.443+0000 7fabff7fe700 1 --2- 172.21.15.78:0/1405499540 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fac04065ed0 0x7fac04068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:49.455 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.443+0000 7fabff7fe700 1 -- 172.21.15.78:0/1405499540 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fac0c021030 con 0x7fac180ce7c0 2023-10-11T09:52:49.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.443+0000 7fac16ffd700 1 --2- 172.21.15.78:0/1405499540 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fac04065ed0 0x7fac04068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:49.456 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.443+0000 7fac16ffd700 1 --2- 172.21.15.78:0/1405499540 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fac04065ed0 0x7fac04068380 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fac080012e0 tx=0x7fac08009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:49.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.451+0000 7fabff7fe700 1 -- 172.21.15.78:0/1405499540 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fac0c052260 con 0x7fac180ce7c0 2023-10-11T09:52:49.630 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.623+0000 7fac1d671700 1 -- 172.21.15.78:0/1405499540 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac180744c0 con 0x7fac180ce7c0 2023-10-11T09:52:49.631 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.623+0000 7fabff7fe700 1 -- 172.21.15.78:0/1405499540 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+14572 (secure 0 0 0) 0x7fac0c055880 con 0x7fac180ce7c0 2023-10-11T09:52:49.632 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:52:49.633 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":132,"fsid":"dee99db9-7a72-406f-a1e2-917943f2b13d","created":"2023-10-11T09:40:14.354949+0000","modified":"2023-10-11T09:52:44.694250+0000","last_up_change":"2023-10-11T09:40:27.723971+0000","last_in_change":"2023-10-11T09:40:18.165644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T09:40:28.405906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T09:40:34.657665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7b2760f0-8365-40e0-be27-80ea0140c601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":35735},{"type":"v1","addr":"172.21.15.78:6801","nonce":35735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":35735},{"type":"v1","addr":"172.21.15.78:6803","nonce":35735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":35735},{"type":"v1","addr":"172.21.15.78:6807","nonce":35735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":35735},{"type":"v1","addr":"172.21.15.78:6805","nonce":35735}]},"public_addr":"172.21.15.78:6801/35735","cluster_addr":"172.21.15.78:6803/35735","heartbeat_back_addr":"172.21.15.78:6807/35735","heartbeat_front_addr":"172.21.15.78:6805/35735","state":["exists","up"]},{"osd":1,"uuid":"3de55db3-11d4-44f6-bf0b-41c5f97eb9c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":35734},{"type":"v1","addr":"172.21.15.78:6811","nonce":35734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":35734},{"type":"v1","addr":"172.21.15.78:6816","nonce":35734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6821","nonce":35734},{"type":"v1","addr":"172.21.15.78:6823","nonce":35734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":35734},{"type":"v1","addr":"172.21.15.78:6819","nonce":35734}]},"public_addr":"172.21.15.78:6811/35734","cluster_addr":"172.21.15.78:6816/35734","heartbeat_back_addr":"172.21.15.78:6823/35734","heartbeat_front_addr":"172.21.15.78:6819/35734","state":["exists","up"]},{"osd":2,"uuid":"c1a86a2e-7754-445a-b955-2df5a647ed07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":35732},{"type":"v1","addr":"172.21.15.78:6810","nonce":35732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":35732},{"type":"v1","addr":"172.21.15.78:6813","nonce":35732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":35732},{"type":"v1","addr":"172.21.15.78:6822","nonce":35732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":35732},{"type":"v1","addr":"172.21.15.78:6817","nonce":35732}]},"public_addr":"172.21.15.78:6810/35732","cluster_addr":"172.21.15.78:6813/35732","heartbeat_back_addr":"172.21.15.78:6822/35732","heartbeat_front_addr":"172.21.15.78:6817/35732","state":["exists","up"]},{"osd":3,"uuid":"938b8437-85e7-4833-9e6f-36bcf287c784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":35738},{"type":"v1","addr":"172.21.15.78:6825","nonce":35738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":35738},{"type":"v1","addr":"172.21.15.78:6827","nonce":35738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":35738},{"type":"v1","addr":"172.21.15.78:6831","nonce":35738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":35738},{"type":"v1","addr":"172.21.15.78:6829","nonce":35738}]},"public_addr":"172.21.15.78:6825/35738","cluster_addr":"172.21.15.78:6827/35738","heartbeat_back_addr":"172.21.15.78:6831/35738","heartbeat_front_addr":"172.21.15.78:6829/35738","state":["exists","up"]},{"osd":4,"uuid":"e8f66d61-bee7-4db0-b234-fa865e29d9fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":35898},{"type":"v1","addr":"172.21.15.133:6809","nonce":35898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":35898},{"type":"v1","addr":"172.21.15.133:6811","nonce":35898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":35898},{"type":"v1","addr":"172.21.15.133:6815","nonce":35898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":35898},{"type":"v1","addr":"172.21.15.133:6813","nonce":35898}]},"public_addr":"172.21.15.133:6809/35898","cluster_addr":"172.21.15.133:6811/35898","heartbeat_back_addr":"172.21.15.133:6815/35898","heartbeat_front_addr":"172.21.15.133:6813/35898","state":["exists","up"]},{"osd":5,"uuid":"91314bec-7752-484f-9df5-716e9fe253de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":35899},{"type":"v1","addr":"172.21.15.133:6817","nonce":35899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":35899},{"type":"v1","addr":"172.21.15.133:6819","nonce":35899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":35899},{"type":"v1","addr":"172.21.15.133:6823","nonce":35899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":35899},{"type":"v1","addr":"172.21.15.133:6821","nonce":35899}]},"public_addr":"172.21.15.133:6817/35899","cluster_addr":"172.21.15.133:6819/35899","heartbeat_back_addr":"172.21.15.133:6823/35899","heartbeat_front_addr":"172.21.15.133:6821/35899","state":["exists","up"]},{"osd":6,"uuid":"6cf9c466-c7e1-46a2-8b2e-feea51f253e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":35900},{"type":"v1","addr":"172.21.15.133:6801","nonce":35900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":35900},{"type":"v1","addr":"172.21.15.133:6803","nonce":35900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":35900},{"type":"v1","addr":"172.21.15.133:6807","nonce":35900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":35900},{"type":"v1","addr":"172.21.15.133:6805","nonce":35900}]},"public_addr":"172.21.15.133:6801/35900","cluster_addr":"172.21.15.133:6803/35900","heartbeat_back_addr":"172.21.15.133:6807/35900","heartbeat_front_addr":"172.21.15.133:6805/35900","state":["exists","up"]},{"osd":7,"uuid":"c62a10af-8966-42af-b799-3422ad19ae0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":35901},{"type":"v1","addr":"172.21.15.133:6825","nonce":35901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":35901},{"type":"v1","addr":"172.21.15.133:6827","nonce":35901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":35901},{"type":"v1","addr":"172.21.15.133:6831","nonce":35901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":35901},{"type":"v1","addr":"172.21.15.133:6829","nonce":35901}]},"public_addr":"172.21.15.133:6825/35901","cluster_addr":"172.21.15.133:6827/35901","heartbeat_back_addr":"172.21.15.133:6831/35901","heartbeat_front_addr":"172.21.15.133:6829/35901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.806774+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.983845+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.107849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.268182+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.930188+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.201608+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:23.859133+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T09:40:24.252289+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6837/3884456900":"2023-10-12T09:52:41.660312+0000","172.21.15.78:6837/4189411531":"2023-10-12T09:52:22.836499+0000","172.21.15.78:6835/4189411531":"2023-10-12T09:52:22.836499+0000","172.21.15.78:6835/3884456900":"2023-10-12T09:52:41.660312+0000","172.21.15.133:6833/1433079498":"2023-10-12T09:52:02.877755+0000","172.21.15.133:6832/1433079498":"2023-10-12T09:52:02.877755+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T09:52:49.635 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.627+0000 7fac1d671700 1 -- 172.21.15.78:0/1405499540 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fac04065ed0 msgr2=0x7fac04068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:49.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.627+0000 7fac1d671700 1 --2- 172.21.15.78:0/1405499540 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fac04065ed0 0x7fac04068380 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fac080012e0 tx=0x7fac08009040 comp rx=0 tx=0).stop 2023-10-11T09:52:49.636 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.627+0000 7fac1d671700 1 -- 172.21.15.78:0/1405499540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac180ce7c0 msgr2=0x7fac1815f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:49.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.627+0000 7fac1d671700 1 --2- 172.21.15.78:0/1405499540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac180ce7c0 0x7fac1815f640 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fac0c000c00 tx=0x7fac0c004770 comp rx=0 tx=0).stop 2023-10-11T09:52:49.637 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.627+0000 7fac1d671700 1 -- 172.21.15.78:0/1405499540 shutdown_connections 2023-10-11T09:52:49.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.627+0000 7fac1d671700 1 --2- 172.21.15.78:0/1405499540 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fac04065ed0 0x7fac04068380 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:49.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.627+0000 7fac1d671700 1 --2- 172.21.15.78:0/1405499540 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac1815e800 0x7fac180ac950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:49.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.627+0000 7fac1d671700 1 --2- 172.21.15.78:0/1405499540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac180ce7c0 0x7fac1815f640 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:49.638 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.627+0000 7fac1d671700 1 --2- 172.21.15.78:0/1405499540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac180c5b40 0x7fac18149640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:49.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.627+0000 7fac1d671700 1 -- 172.21.15.78:0/1405499540 >> 172.21.15.78:0/1405499540 conn(0x7fac180cb920 msgr2=0x7fac180b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:49.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.627+0000 7fac1d671700 1 -- 172.21.15.78:0/1405499540 shutdown_connections 2023-10-11T09:52:49.639 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:49.627+0000 7fac1d671700 1 -- 172.21.15.78:0/1405499540 wait complete. 2023-10-11T09:52:50.658 INFO:tasks.ceph:Scrubbing osd.0 2023-10-11T09:52:50.659 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-11T09:52:50.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.835+0000 7f0006ad8700 1 Processor -- start 2023-10-11T09:52:50.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.835+0000 7f0006ad8700 1 -- start start 2023-10-11T09:52:50.846 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.839+0000 7f0006ad8700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f00000d0200 0x7f00000d05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:50.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.839+0000 7f0006ad8700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f000006cf80 0x7f00000c4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:50.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.839+0000 7f0006ad8700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00000d1f60 con 0x7f00000c40f0 2023-10-11T09:52:50.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.839+0000 7f0006ad8700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00000c4ab0 con 0x7f00000d0200 2023-10-11T09:52:50.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.839+0000 7f0006ad8700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00000d0b10 con 0x7f000006cf80 2023-10-11T09:52:50.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.839+0000 7efffffff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f00000c40f0 0x7f000006c870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:49750/0 (socket says 172.21.15.78:49750) 2023-10-11T09:52:50.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.839+0000 7efffffff700 1 -- 172.21.15.78:0/3337018795 learned_addr learned my addr 172.21.15.78:0/3337018795 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:50.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.839+0000 7f0004874700 1 --2- 172.21.15.78:0/3337018795 >> v2:172.21.15.78:3300/0 conn(0x7f00000d0200 0x7f00000d05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:50.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.839+0000 7f0005075700 1 --2- 172.21.15.78:0/3337018795 >> v2:172.21.15.133:3300/0 conn(0x7f000006cf80 0x7f00000c4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:50.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.839+0000 7effff7fe700 1 -- 172.21.15.78:0/3337018795 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3412463701 0 0) 0x7f00000d1f60 con 0x7f00000c40f0 2023-10-11T09:52:50.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.839+0000 7effff7fe700 1 -- 172.21.15.78:0/3337018795 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effe80035d0 con 0x7f00000c40f0 2023-10-11T09:52:50.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.839+0000 7f0004874700 1 -- 172.21.15.78:0/3337018795 >> v2:172.21.15.133:3300/0 conn(0x7f000006cf80 msgr2=0x7f00000c4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:50.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.839+0000 7f0004874700 1 --2- 172.21.15.78:0/3337018795 >> v2:172.21.15.133:3300/0 conn(0x7f000006cf80 0x7f00000c4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:50.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.839+0000 7f0004874700 1 -- 172.21.15.78:0/3337018795 >> v1:172.21.15.78:6789/0 conn(0x7f00000c40f0 legacy=0x7f000006c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:50.852 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.839+0000 7f0004874700 1 -- 172.21.15.78:0/3337018795 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00000d0ec0 con 0x7f00000d0200 2023-10-11T09:52:50.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.843+0000 7f0004874700 1 --2- 172.21.15.78:0/3337018795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00000d0200 0x7f00000d05d0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f0000164c90 tx=0x7efff4009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c0d7f1f83f64938 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:50.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.843+0000 7effff7fe700 1 -- 172.21.15.78:0/3337018795 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff4011830 con 0x7f00000d0200 2023-10-11T09:52:50.853 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.843+0000 7effff7fe700 1 -- 172.21.15.78:0/3337018795 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff4011990 con 0x7f00000d0200 2023-10-11T09:52:50.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.843+0000 7effff7fe700 1 -- 172.21.15.78:0/3337018795 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff4011d20 con 0x7f00000d0200 2023-10-11T09:52:50.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.843+0000 7f0006ad8700 1 -- 172.21.15.78:0/3337018795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00000d0200 msgr2=0x7f00000d05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:50.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.843+0000 7f0006ad8700 1 --2- 172.21.15.78:0/3337018795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00000d0200 0x7f00000d05d0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f0000164c90 tx=0x7efff4009b30 comp rx=0 tx=0).stop 2023-10-11T09:52:50.854 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.843+0000 7f0006ad8700 1 -- 172.21.15.78:0/3337018795 shutdown_connections 2023-10-11T09:52:50.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.843+0000 7f0006ad8700 1 --2- 172.21.15.78:0/3337018795 >> v2:172.21.15.133:3300/0 conn(0x7f000006cf80 0x7f00000c4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:50.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.843+0000 7f0006ad8700 1 --2- 172.21.15.78:0/3337018795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00000d0200 0x7f00000d05d0 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:50.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.843+0000 7f0006ad8700 1 -- 172.21.15.78:0/3337018795 >> 172.21.15.78:0/3337018795 conn(0x7f000006aaa0 msgr2=0x7f000009a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:50.855 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.843+0000 7f0006ad8700 1 -- 172.21.15.78:0/3337018795 shutdown_connections 2023-10-11T09:52:50.856 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.843+0000 7f0006ad8700 1 -- 172.21.15.78:0/3337018795 wait complete. 2023-10-11T09:52:50.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.847+0000 7f0006ad8700 1 Processor -- start 2023-10-11T09:52:50.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.847+0000 7f0006ad8700 1 -- start start 2023-10-11T09:52:50.858 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.847+0000 7f0006ad8700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f000006cf80 0x7f000015cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:50.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.847+0000 7f0006ad8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00000c40f0 0x7f00001f9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:50.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.847+0000 7f0006ad8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00000d0200 0x7f000015a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:50.859 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.847+0000 7f0006ad8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0000148f80 con 0x7f00000c40f0 2023-10-11T09:52:50.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.847+0000 7f0006ad8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00000b4fa0 con 0x7f00000d0200 2023-10-11T09:52:50.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.847+0000 7f0006ad8700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00001f9c80 con 0x7f000006cf80 2023-10-11T09:52:50.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.847+0000 7efffffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00000c40f0 0x7f00001f9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:50.860 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.847+0000 7efffffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00000c40f0 0x7f00001f9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48372/0 (socket says 172.21.15.78:48372) 2023-10-11T09:52:50.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.847+0000 7efffffff700 1 -- 172.21.15.78:0/1073828954 learned_addr learned my addr 172.21.15.78:0/1073828954 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:50.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.851+0000 7f0005075700 1 --2- 172.21.15.78:0/1073828954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00000d0200 0x7f000015a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:50.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.851+0000 7f0004874700 1 --2- 172.21.15.78:0/1073828954 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f000006cf80 0x7f000015cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:50.861 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.851+0000 7efffffff700 1 -- 172.21.15.78:0/1073828954 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f000006cf80 msgr2=0x7f000015cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:50.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.851+0000 7efffffff700 1 --2- 172.21.15.78:0/1073828954 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f000006cf80 0x7f000015cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:50.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.851+0000 7efffffff700 1 -- 172.21.15.78:0/1073828954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00000d0200 msgr2=0x7f000015a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:50.862 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.851+0000 7efffffff700 1 --2- 172.21.15.78:0/1073828954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00000d0200 0x7f000015a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:50.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.851+0000 7efffffff700 1 -- 172.21.15.78:0/1073828954 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff40094b0 con 0x7f00000c40f0 2023-10-11T09:52:50.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.851+0000 7efffffff700 1 --2- 172.21.15.78:0/1073828954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00000c40f0 0x7f00001f9430 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7efff000c700 tx=0x7efff000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:50.865 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.851+0000 7efffd7fa700 1 -- 172.21.15.78:0/1073828954 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff0005d90 con 0x7f00000c40f0 2023-10-11T09:52:50.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.851+0000 7efffd7fa700 1 -- 172.21.15.78:0/1073828954 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff00070d0 con 0x7f00000c40f0 2023-10-11T09:52:50.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.851+0000 7f0006ad8700 1 -- 172.21.15.78:0/1073828954 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00001f9f60 con 0x7f00000c40f0 2023-10-11T09:52:50.866 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.851+0000 7efffd7fa700 1 -- 172.21.15.78:0/1073828954 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff0007480 con 0x7f00000c40f0 2023-10-11T09:52:50.867 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.851+0000 7f0006ad8700 1 -- 172.21.15.78:0/1073828954 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00000b5240 con 0x7f00000c40f0 2023-10-11T09:52:50.868 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.851+0000 7f0006ad8700 1 -- 172.21.15.78:0/1073828954 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f00000d1480 con 0x7f00000c40f0 2023-10-11T09:52:50.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.851+0000 7efffd7fa700 1 -- 172.21.15.78:0/1073828954 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7efff00075e0 con 0x7f00000c40f0 2023-10-11T09:52:50.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.855+0000 7efffd7fa700 1 --2- 172.21.15.78:0/1073828954 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7effec065ed0 0x7effec068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:50.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.855+0000 7efffd7fa700 1 -- 172.21.15.78:0/1073828954 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5468+0+0 (secure 0 0 0) 0x7efff00829a0 con 0x7f00000c40f0 2023-10-11T09:52:50.869 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.855+0000 7f0004874700 1 --2- 172.21.15.78:0/1073828954 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7effec065ed0 0x7effec068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:50.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.855+0000 7efffd7fa700 1 --2- 172.21.15.78:0/1073828954 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x7effec06c480 0x7effec06e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:50.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.855+0000 7efffd7fa700 1 -- 172.21.15.78:0/1073828954 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7effec06ef40 con 0x7effec06c480 2023-10-11T09:52:50.870 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.855+0000 7efffd7fa700 1 -- 172.21.15.78:0/1073828954 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=132) v2 ==== 24+0+0 (secure 0 0 0) 0x7efff00872c0 con 0x7f00000c40f0 2023-10-11T09:52:50.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.855+0000 7f0005075700 1 --2- 172.21.15.78:0/1073828954 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x7effec06c480 0x7effec06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:50.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.855+0000 7f0004874700 1 --2- 172.21.15.78:0/1073828954 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7effec065ed0 0x7effec068380 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7efff4000c00 tx=0x7efff4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:50.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.855+0000 7f0005075700 1 --2- 172.21.15.78:0/1073828954 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x7effec06c480 0x7effec06e890 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:50.871 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.859+0000 7efffd7fa700 1 -- 172.21.15.78:0/1073828954 <== osd.0 v2:172.21.15.78:6800/35735 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7effec06ef40 con 0x7effec06c480 2023-10-11T09:52:50.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.875+0000 7f0006ad8700 1 -- 172.21.15.78:0/1073828954 --> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f00000d11a0 con 0x7effec06c480 2023-10-11T09:52:50.881 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.875+0000 7efffd7fa700 1 -- 172.21.15.78:0/1073828954 <== osd.0 v2:172.21.15.78:6800/35735 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7efff80083a0 con 0x7effec06c480 2023-10-11T09:52:50.882 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:52:50.882 INFO:teuthology.orchestra.run.smithi078.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-11T09:52:50.883 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:52:50.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.875+0000 7f0006ad8700 1 -- 172.21.15.78:0/1073828954 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x7effec06c480 msgr2=0x7effec06e890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:50.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.875+0000 7f0006ad8700 1 --2- 172.21.15.78:0/1073828954 >> [v2:172.21.15.78:6800/35735,v1:172.21.15.78:6801/35735] conn(0x7effec06c480 0x7effec06e890 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:50.884 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.875+0000 7f0006ad8700 1 -- 172.21.15.78:0/1073828954 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7effec065ed0 msgr2=0x7effec068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:50.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.875+0000 7f0006ad8700 1 --2- 172.21.15.78:0/1073828954 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7effec065ed0 0x7effec068380 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7efff4000c00 tx=0x7efff4008030 comp rx=0 tx=0).stop 2023-10-11T09:52:50.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.875+0000 7f0006ad8700 1 -- 172.21.15.78:0/1073828954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00000c40f0 msgr2=0x7f00001f9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:50.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.875+0000 7f0006ad8700 1 --2- 172.21.15.78:0/1073828954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00000c40f0 0x7f00001f9430 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7efff000c700 tx=0x7efff000ca10 comp rx=0 tx=0).stop 2023-10-11T09:52:50.885 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.875+0000 7f0005075700 1 -- 172.21.15.78:0/1073828954 reap_dead start 2023-10-11T09:52:50.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.875+0000 7f0006ad8700 1 -- 172.21.15.78:0/1073828954 shutdown_connections 2023-10-11T09:52:50.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.875+0000 7f0006ad8700 1 -- 172.21.15.78:0/1073828954 >> 172.21.15.78:0/1073828954 conn(0x7f000006aaa0 msgr2=0x7f00000af110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:50.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.875+0000 7f0006ad8700 1 -- 172.21.15.78:0/1073828954 shutdown_connections 2023-10-11T09:52:50.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:50.875+0000 7f0006ad8700 1 -- 172.21.15.78:0/1073828954 wait complete. 2023-10-11T09:52:50.903 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-11T09:52:51.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.071+0000 7f56e6a64700 1 Processor -- start 2023-10-11T09:52:51.079 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.071+0000 7f56e6a64700 1 -- start start 2023-10-11T09:52:51.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.075+0000 7f56e6a64700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56e00ce700 0x7f56e00c20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:51.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.075+0000 7f56e6a64700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f56e0071720 0x7f56e006bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:51.082 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.075+0000 7f56e6a64700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56e00cb220 con 0x7f56e0161020 2023-10-11T09:52:51.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.075+0000 7f56e6a64700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56e0071c70 con 0x7f56e0071720 2023-10-11T09:52:51.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.075+0000 7f56e6a64700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e01613f0 con 0x7f56e00ce700 2023-10-11T09:52:51.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.075+0000 7f56e5a62700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f56e0161020 0x7f56e00cdff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:49762/0 (socket says 172.21.15.78:49762) 2023-10-11T09:52:51.083 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.075+0000 7f56e5a62700 1 -- 172.21.15.78:0/3406981774 learned_addr learned my addr 172.21.15.78:0/3406981774 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:51.085 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.075+0000 7f56e6263700 1 --2- 172.21.15.78:0/3406981774 >> v2:172.21.15.78:3300/0 conn(0x7f56e0071720 0x7f56e006bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:51.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.075+0000 7f56e5261700 1 --2- 172.21.15.78:0/3406981774 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56e00ce700 0x7f56e00c20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:51.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.075+0000 7f56e4a60700 1 -- 172.21.15.78:0/3406981774 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 872341988 0 0) 0x7f56e00cb220 con 0x7f56e0161020 2023-10-11T09:52:51.086 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.075+0000 7f56e4a60700 1 -- 172.21.15.78:0/3406981774 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56cc0035d0 con 0x7f56e0161020 2023-10-11T09:52:51.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.075+0000 7f56e6263700 1 -- 172.21.15.78:0/3406981774 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56e00ce700 msgr2=0x7f56e00c20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:51.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.075+0000 7f56e6263700 1 --2- 172.21.15.78:0/3406981774 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56e00ce700 0x7f56e00c20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:51.087 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.075+0000 7f56e4a60700 1 -- 172.21.15.78:0/3406981774 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1659319339 0 0) 0x7f56cc0035d0 con 0x7f56e0161020 2023-10-11T09:52:51.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.075+0000 7f56e6263700 1 -- 172.21.15.78:0/3406981774 >> v1:172.21.15.78:6789/0 conn(0x7f56e0161020 legacy=0x7f56e00cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:51.088 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.075+0000 7f56e6263700 1 -- 172.21.15.78:0/3406981774 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e00ca650 con 0x7f56e0071720 2023-10-11T09:52:51.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.075+0000 7f56e6263700 1 --2- 172.21.15.78:0/3406981774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56e0071720 0x7f56e006bc60 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f56d8009d90 tx=0x7f56d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2a4154f7a599218 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:51.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.075+0000 7f56e4a60700 1 -- 172.21.15.78:0/3406981774 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d800e070 con 0x7f56e0071720 2023-10-11T09:52:51.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.075+0000 7f56e4a60700 1 -- 172.21.15.78:0/3406981774 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d8003a40 con 0x7f56e0071720 2023-10-11T09:52:51.089 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.075+0000 7f56e4a60700 1 -- 172.21.15.78:0/3406981774 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56dc001670 con 0x7f56e0071720 2023-10-11T09:52:51.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.079+0000 7f56e6a64700 1 -- 172.21.15.78:0/3406981774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56e0071720 msgr2=0x7f56e006bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:51.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.079+0000 7f56e6a64700 1 --2- 172.21.15.78:0/3406981774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56e0071720 0x7f56e006bc60 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f56d8009d90 tx=0x7f56d800b330 comp rx=0 tx=0).stop 2023-10-11T09:52:51.090 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.079+0000 7f56e6a64700 1 -- 172.21.15.78:0/3406981774 shutdown_connections 2023-10-11T09:52:51.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.079+0000 7f56e6a64700 1 --2- 172.21.15.78:0/3406981774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56e0071720 0x7f56e006bc60 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:51.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.079+0000 7f56e6a64700 1 --2- 172.21.15.78:0/3406981774 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56e00ce700 0x7f56e00c20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:51.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.079+0000 7f56e6a64700 1 -- 172.21.15.78:0/3406981774 >> 172.21.15.78:0/3406981774 conn(0x7f56e00cf820 msgr2=0x7f56e00cb610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:51.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.079+0000 7f56e6a64700 1 -- 172.21.15.78:0/3406981774 shutdown_connections 2023-10-11T09:52:51.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.079+0000 7f56e6a64700 1 -- 172.21.15.78:0/3406981774 wait complete. 2023-10-11T09:52:51.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.083+0000 7f56e6a64700 1 Processor -- start 2023-10-11T09:52:51.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.083+0000 7f56e6a64700 1 -- start start 2023-10-11T09:52:51.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.083+0000 7f56e6a64700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56e0071720 0x7f56e00b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:51.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.083+0000 7f56e6a64700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56e00ce700 0x7f56e0162f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:51.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.083+0000 7f56e6a64700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56e0161020 0x7f56e0161900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:51.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.083+0000 7f56e6a64700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e00b34c0 con 0x7f56e00ce700 2023-10-11T09:52:51.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.083+0000 7f56e6a64700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e00c2630 con 0x7f56e0071720 2023-10-11T09:52:51.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.083+0000 7f56e6a64700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e00c2910 con 0x7f56e0161020 2023-10-11T09:52:51.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.083+0000 7f56e5261700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56e00ce700 0x7f56e0162f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:51.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.083+0000 7f56e5261700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56e00ce700 0x7f56e0162f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48388/0 (socket says 172.21.15.78:48388) 2023-10-11T09:52:51.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.083+0000 7f56e5261700 1 -- 172.21.15.78:0/2412302028 learned_addr learned my addr 172.21.15.78:0/2412302028 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:51.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.083+0000 7f56e5a62700 1 --2- 172.21.15.78:0/2412302028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56e0071720 0x7f56e00b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:51.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.083+0000 7f56e6263700 1 --2- 172.21.15.78:0/2412302028 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56e0161020 0x7f56e0161900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:51.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.087+0000 7f56e5261700 1 -- 172.21.15.78:0/2412302028 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56e0161020 msgr2=0x7f56e0161900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:51.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.087+0000 7f56e5261700 1 --2- 172.21.15.78:0/2412302028 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56e0161020 0x7f56e0161900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:51.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.087+0000 7f56e5261700 1 -- 172.21.15.78:0/2412302028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56e0071720 msgr2=0x7f56e00b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:51.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.087+0000 7f56e5261700 1 --2- 172.21.15.78:0/2412302028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56e0071720 0x7f56e00b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:51.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.087+0000 7f56e5261700 1 -- 172.21.15.78:0/2412302028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d8009a70 con 0x7f56e00ce700 2023-10-11T09:52:51.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.087+0000 7f56e5261700 1 --2- 172.21.15.78:0/2412302028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56e00ce700 0x7f56e0162f30 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f56d000e9b0 tx=0x7f56d000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:51.099 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.087+0000 7f56d67fc700 1 -- 172.21.15.78:0/2412302028 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d000ee50 con 0x7f56e00ce700 2023-10-11T09:52:51.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.087+0000 7f56d67fc700 1 -- 172.21.15.78:0/2412302028 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d000c7f0 con 0x7f56e00ce700 2023-10-11T09:52:51.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.087+0000 7f56e6a64700 1 -- 172.21.15.78:0/2412302028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56e00b3760 con 0x7f56e00ce700 2023-10-11T09:52:51.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.087+0000 7f56d67fc700 1 -- 172.21.15.78:0/2412302028 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d000cb60 con 0x7f56e00ce700 2023-10-11T09:52:51.100 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.087+0000 7f56e6a64700 1 -- 172.21.15.78:0/2412302028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e00c2bf0 con 0x7f56e00ce700 2023-10-11T09:52:51.101 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.087+0000 7f56e6a64700 1 -- 172.21.15.78:0/2412302028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56e0067d00 con 0x7f56e00ce700 2023-10-11T09:52:51.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.091+0000 7f56d67fc700 1 -- 172.21.15.78:0/2412302028 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f56d000ccc0 con 0x7f56e00ce700 2023-10-11T09:52:51.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.091+0000 7f56d67fc700 1 --2- 172.21.15.78:0/2412302028 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f56c4065ed0 0x7f56c4068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:51.102 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.091+0000 7f56d67fc700 1 -- 172.21.15.78:0/2412302028 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f56d0083940 con 0x7f56e00ce700 2023-10-11T09:52:51.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.091+0000 7f56e5a62700 1 --2- 172.21.15.78:0/2412302028 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f56c4065ed0 0x7f56c4068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:51.103 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.095+0000 7f56e5a62700 1 --2- 172.21.15.78:0/2412302028 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f56c4065ed0 0x7f56c4068380 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f56e00ccce0 tx=0x7f56dc001730 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:51.105 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.099+0000 7f56d67fc700 1 -- 172.21.15.78:0/2412302028 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f56d0051140 con 0x7f56e00ce700 2023-10-11T09:52:51.278 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.271+0000 7f56e6a64700 1 -- 172.21.15.78:0/2412302028 --> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f56e00cabd0 con 0x7f56c4065ed0 2023-10-11T09:52:51.279 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.271+0000 7f56d67fc700 1 -- 172.21.15.78:0/2412302028 <== mgr.4101 v2:172.21.15.78:6832/35606 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f56dc00f690 con 0x7f56c4065ed0 2023-10-11T09:52:51.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.275+0000 7f56e6a64700 1 -- 172.21.15.78:0/2412302028 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f56c4065ed0 msgr2=0x7f56c4068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:51.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.275+0000 7f56e6a64700 1 --2- 172.21.15.78:0/2412302028 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f56c4065ed0 0x7f56c4068380 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f56e00ccce0 tx=0x7f56dc001730 comp rx=0 tx=0).stop 2023-10-11T09:52:51.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.275+0000 7f56e6a64700 1 -- 172.21.15.78:0/2412302028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56e00ce700 msgr2=0x7f56e0162f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:51.284 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.275+0000 7f56e6a64700 1 --2- 172.21.15.78:0/2412302028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56e00ce700 0x7f56e0162f30 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f56d000e9b0 tx=0x7f56d000ecc0 comp rx=0 tx=0).stop 2023-10-11T09:52:51.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.275+0000 7f56e6a64700 1 -- 172.21.15.78:0/2412302028 shutdown_connections 2023-10-11T09:52:51.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.275+0000 7f56e6a64700 1 --2- 172.21.15.78:0/2412302028 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f56c4065ed0 0x7f56c4068380 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:51.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.275+0000 7f56e6a64700 1 --2- 172.21.15.78:0/2412302028 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56e0161020 0x7f56e0161900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:51.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.275+0000 7f56e6a64700 1 --2- 172.21.15.78:0/2412302028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56e00ce700 0x7f56e0162f30 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:51.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.275+0000 7f56e6a64700 1 --2- 172.21.15.78:0/2412302028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56e0071720 0x7f56e00b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:51.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.275+0000 7f56e6a64700 1 -- 172.21.15.78:0/2412302028 >> 172.21.15.78:0/2412302028 conn(0x7f56e00cf820 msgr2=0x7f56e00c19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:51.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.275+0000 7f56e6a64700 1 -- 172.21.15.78:0/2412302028 shutdown_connections 2023-10-11T09:52:51.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.275+0000 7f56e6a64700 1 -- 172.21.15.78:0/2412302028 wait complete. 2023-10-11T09:52:51.287 INFO:teuthology.orchestra.run.smithi078.stderr:instructed osd(s) 0 to deep-scrub 2023-10-11T09:52:51.304 INFO:tasks.ceph:Scrubbing osd.1 2023-10-11T09:52:51.304 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-11T09:52:51.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.479+0000 7f4df485d700 1 Processor -- start 2023-10-11T09:52:51.487 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.479+0000 7f4df485d700 1 -- start start 2023-10-11T09:52:51.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.483+0000 7f4df485d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f4df015c980 0x7f4df00d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:51.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.483+0000 7f4df485d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4df00d13b0 0x7f4df00b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:51.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.483+0000 7f4df485d700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4df00cf8b0 con 0x7f4df00b5060 2023-10-11T09:52:51.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.483+0000 7f4df485d700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4df00b54e0 con 0x7f4df015c980 2023-10-11T09:52:51.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.483+0000 7f4df485d700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df00d2290 con 0x7f4df00d13b0 2023-10-11T09:52:51.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.483+0000 7f4deed9d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f4df015c980 0x7f4df00d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:51.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.483+0000 7f4def59e700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f4df00b5060 0x7f4df006aa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:49772/0 (socket says 172.21.15.78:49772) 2023-10-11T09:52:51.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.483+0000 7f4def59e700 1 -- 172.21.15.78:0/1570671440 learned_addr learned my addr 172.21.15.78:0/1570671440 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:51.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.483+0000 7f4dee59c700 1 --2- 172.21.15.78:0/1570671440 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4df00d13b0 0x7f4df00b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:51.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.483+0000 7f4deed9d700 1 -- 172.21.15.78:0/1570671440 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4df00d13b0 msgr2=0x7f4df00b4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:51.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.483+0000 7f4deed9d700 1 --2- 172.21.15.78:0/1570671440 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4df00d13b0 0x7f4df00b4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:51.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.483+0000 7f4deed9d700 1 -- 172.21.15.78:0/1570671440 >> v1:172.21.15.78:6789/0 conn(0x7f4df00b5060 legacy=0x7f4df006aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:51.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.483+0000 7f4deed9d700 1 -- 172.21.15.78:0/1570671440 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df00d19f0 con 0x7f4df015c980 2023-10-11T09:52:51.494 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.483+0000 7f4dedd9b700 1 -- 172.21.15.78:0/1570671440 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1199946506 0 0) 0x7f4df00cf8b0 con 0x7f4df00b5060 2023-10-11T09:52:51.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.483+0000 7f4dee59c700 1 --2- 172.21.15.78:0/1570671440 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4df00d13b0 0x7f4df00b4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:51.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.483+0000 7f4deed9d700 1 --2- 172.21.15.78:0/1570671440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4df015c980 0x7f4df00d0e70 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f4de0009850 tx=0x7f4de0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7597e0b2b821f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:51.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.483+0000 7f4dedd9b700 1 -- 172.21.15.78:0/1570671440 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de0011830 con 0x7f4df015c980 2023-10-11T09:52:51.495 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.483+0000 7f4dedd9b700 1 -- 172.21.15.78:0/1570671440 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de0011990 con 0x7f4df015c980 2023-10-11T09:52:51.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.483+0000 7f4dedd9b700 1 -- 172.21.15.78:0/1570671440 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de0011d20 con 0x7f4df015c980 2023-10-11T09:52:51.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.483+0000 7f4df485d700 1 -- 172.21.15.78:0/1570671440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4df015c980 msgr2=0x7f4df00d0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:51.496 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.483+0000 7f4df485d700 1 --2- 172.21.15.78:0/1570671440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4df015c980 0x7f4df00d0e70 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f4de0009850 tx=0x7f4de0009b60 comp rx=0 tx=0).stop 2023-10-11T09:52:51.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.483+0000 7f4df485d700 1 -- 172.21.15.78:0/1570671440 shutdown_connections 2023-10-11T09:52:51.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.483+0000 7f4df485d700 1 --2- 172.21.15.78:0/1570671440 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4df00d13b0 0x7f4df00b4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:51.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.483+0000 7f4df485d700 1 --2- 172.21.15.78:0/1570671440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4df015c980 0x7f4df00d0e70 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:51.497 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.483+0000 7f4df485d700 1 -- 172.21.15.78:0/1570671440 >> 172.21.15.78:0/1570671440 conn(0x7f4df00724b0 msgr2=0x7f4df00d30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:51.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.483+0000 7f4df485d700 1 -- 172.21.15.78:0/1570671440 shutdown_connections 2023-10-11T09:52:51.498 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.483+0000 7f4df485d700 1 -- 172.21.15.78:0/1570671440 wait complete. 2023-10-11T09:52:51.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.487+0000 7f4df485d700 1 Processor -- start 2023-10-11T09:52:51.500 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.491+0000 7f4df485d700 1 -- start start 2023-10-11T09:52:51.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.491+0000 7f4df485d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4df00b5060 0x7f4df01f1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:51.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.491+0000 7f4df485d700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4df00d13b0 0x7f4df00ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:51.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.491+0000 7f4df485d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4df015c980 0x7f4df01f5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:51.501 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.491+0000 7f4df485d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df00cba00 con 0x7f4df00b5060 2023-10-11T09:52:51.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.491+0000 7f4df485d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df01f1d70 con 0x7f4df015c980 2023-10-11T09:52:51.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.491+0000 7f4df485d700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df00cadf0 con 0x7f4df00d13b0 2023-10-11T09:52:51.502 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.491+0000 7f4deed9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4df00b5060 0x7f4df01f1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:51.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.491+0000 7f4deed9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4df00b5060 0x7f4df01f1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48402/0 (socket says 172.21.15.78:48402) 2023-10-11T09:52:51.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.491+0000 7f4deed9d700 1 -- 172.21.15.78:0/1381522036 learned_addr learned my addr 172.21.15.78:0/1381522036 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:51.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.491+0000 7f4def59e700 1 --2- 172.21.15.78:0/1381522036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4df015c980 0x7f4df01f5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:51.503 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.491+0000 7f4dee59c700 1 --2- 172.21.15.78:0/1381522036 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4df00d13b0 0x7f4df00ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:51.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.491+0000 7f4deed9d700 1 -- 172.21.15.78:0/1381522036 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4df00d13b0 msgr2=0x7f4df00ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:51.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.491+0000 7f4deed9d700 1 --2- 172.21.15.78:0/1381522036 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4df00d13b0 0x7f4df00ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:51.504 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.491+0000 7f4deed9d700 1 -- 172.21.15.78:0/1381522036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4df015c980 msgr2=0x7f4df01f5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:51.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.491+0000 7f4deed9d700 1 --2- 172.21.15.78:0/1381522036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4df015c980 0x7f4df01f5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:51.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.491+0000 7f4deed9d700 1 -- 172.21.15.78:0/1381522036 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de00094b0 con 0x7f4df00b5060 2023-10-11T09:52:51.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.491+0000 7f4dee59c700 1 --2- 172.21.15.78:0/1381522036 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4df00d13b0 0x7f4df00ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:51.505 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.491+0000 7f4deed9d700 1 --2- 172.21.15.78:0/1381522036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4df00b5060 0x7f4df01f1830 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f4de0000c00 tx=0x7f4de0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:51.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.491+0000 7f4dd77fe700 1 -- 172.21.15.78:0/1381522036 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de00036c0 con 0x7f4df00b5060 2023-10-11T09:52:51.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.491+0000 7f4df485d700 1 -- 172.21.15.78:0/1381522036 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df00cb0d0 con 0x7f4df00b5060 2023-10-11T09:52:51.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.491+0000 7f4dd77fe700 1 -- 172.21.15.78:0/1381522036 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de0003820 con 0x7f4df00b5060 2023-10-11T09:52:51.506 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.491+0000 7f4dd77fe700 1 -- 172.21.15.78:0/1381522036 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de0003bb0 con 0x7f4df00b5060 2023-10-11T09:52:51.507 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.491+0000 7f4df485d700 1 -- 172.21.15.78:0/1381522036 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df00cb690 con 0x7f4df00b5060 2023-10-11T09:52:51.508 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.495+0000 7f4df485d700 1 -- 172.21.15.78:0/1381522036 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4df00d1fb0 con 0x7f4df00b5060 2023-10-11T09:52:51.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.495+0000 7f4dd77fe700 1 -- 172.21.15.78:0/1381522036 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f4de0003d10 con 0x7f4df00b5060 2023-10-11T09:52:51.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.495+0000 7f4dd77fe700 1 --2- 172.21.15.78:0/1381522036 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4ddc065ed0 0x7f4ddc068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:51.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.495+0000 7f4dd77fe700 1 -- 172.21.15.78:0/1381522036 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f4de0021030 con 0x7f4df00b5060 2023-10-11T09:52:51.509 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.499+0000 7f4dd77fe700 1 --2- 172.21.15.78:0/1381522036 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x7f4ddc06c480 0x7f4ddc06e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:51.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.499+0000 7f4dd77fe700 1 -- 172.21.15.78:0/1381522036 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4ddc06ef40 con 0x7f4ddc06c480 2023-10-11T09:52:51.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.499+0000 7f4dd77fe700 1 -- 172.21.15.78:0/1381522036 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=132) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4de0085570 con 0x7f4df00b5060 2023-10-11T09:52:51.510 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.499+0000 7f4dee59c700 1 --2- 172.21.15.78:0/1381522036 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4ddc065ed0 0x7f4ddc068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:51.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.499+0000 7f4def59e700 1 --2- 172.21.15.78:0/1381522036 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x7f4ddc06c480 0x7f4ddc06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:51.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.499+0000 7f4dee59c700 1 --2- 172.21.15.78:0/1381522036 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4ddc065ed0 0x7f4ddc068380 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4df0142de0 tx=0x7f4de4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:51.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.499+0000 7f4def59e700 1 --2- 172.21.15.78:0/1381522036 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x7f4ddc06c480 0x7f4ddc06e890 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:51.511 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.499+0000 7f4dd77fe700 1 -- 172.21.15.78:0/1381522036 <== osd.1 v2:172.21.15.78:6809/35734 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4ddc06ef40 con 0x7f4ddc06c480 2023-10-11T09:52:51.522 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.515+0000 7f4df485d700 1 -- 172.21.15.78:0/1381522036 --> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4df00d1cd0 con 0x7f4ddc06c480 2023-10-11T09:52:51.523 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.515+0000 7f4dd77fe700 1 -- 172.21.15.78:0/1381522036 <== osd.1 v2:172.21.15.78:6809/35734 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f4de80071c0 con 0x7f4ddc06c480 2023-10-11T09:52:51.524 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:52:51.524 INFO:teuthology.orchestra.run.smithi078.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-11T09:52:51.524 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:52:51.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.515+0000 7f4df485d700 1 -- 172.21.15.78:0/1381522036 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x7f4ddc06c480 msgr2=0x7f4ddc06e890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:51.525 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.515+0000 7f4df485d700 1 --2- 172.21.15.78:0/1381522036 >> [v2:172.21.15.78:6809/35734,v1:172.21.15.78:6811/35734] conn(0x7f4ddc06c480 0x7f4ddc06e890 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:51.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.515+0000 7f4df485d700 1 -- 172.21.15.78:0/1381522036 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4ddc065ed0 msgr2=0x7f4ddc068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:51.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.515+0000 7f4df485d700 1 --2- 172.21.15.78:0/1381522036 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4ddc065ed0 0x7f4ddc068380 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4df0142de0 tx=0x7f4de4008040 comp rx=0 tx=0).stop 2023-10-11T09:52:51.526 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.515+0000 7f4df485d700 1 -- 172.21.15.78:0/1381522036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4df00b5060 msgr2=0x7f4df01f1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:51.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.515+0000 7f4df485d700 1 --2- 172.21.15.78:0/1381522036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4df00b5060 0x7f4df01f1830 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f4de0000c00 tx=0x7f4de0004770 comp rx=0 tx=0).stop 2023-10-11T09:52:51.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.515+0000 7f4def59e700 1 -- 172.21.15.78:0/1381522036 reap_dead start 2023-10-11T09:52:51.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.519+0000 7f4df485d700 1 -- 172.21.15.78:0/1381522036 shutdown_connections 2023-10-11T09:52:51.527 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.519+0000 7f4df485d700 1 -- 172.21.15.78:0/1381522036 >> 172.21.15.78:0/1381522036 conn(0x7f4df00724b0 msgr2=0x7f4df00b8a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:51.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.519+0000 7f4df485d700 1 -- 172.21.15.78:0/1381522036 shutdown_connections 2023-10-11T09:52:51.528 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.519+0000 7f4df485d700 1 -- 172.21.15.78:0/1381522036 wait complete. 2023-10-11T09:52:51.543 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-11T09:52:51.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.707+0000 7f77295dc700 1 Processor -- start 2023-10-11T09:52:51.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.707+0000 7f77295dc700 1 -- start start 2023-10-11T09:52:51.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.707+0000 7f77295dc700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f772415a800 0x7f77240b4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:51.716 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.707+0000 7f77295dc700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f77240b0d20 0x7f772415e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:51.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.707+0000 7f77295dc700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7724070dc0 con 0x7f772406bb60 2023-10-11T09:52:51.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.707+0000 7f77295dc700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77240b46a0 con 0x7f77240b0d20 2023-10-11T09:52:51.717 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.707+0000 7f77295dc700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7724070ff0 con 0x7f772415a800 2023-10-11T09:52:51.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.711+0000 7f7728ddb700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f77240b0d20 0x7f772415e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:51.719 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.711+0000 7f7728ddb700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f77240b0d20 0x7f772415e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48408/0 (socket says 172.21.15.78:48408) 2023-10-11T09:52:51.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.711+0000 7f7728ddb700 1 -- 172.21.15.78:0/335890493 learned_addr learned my addr 172.21.15.78:0/335890493 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:51.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.711+0000 7f7723fff700 1 --2- 172.21.15.78:0/335890493 >> v2:172.21.15.133:3300/0 conn(0x7f772415a800 0x7f77240b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:51.720 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.711+0000 7f7728ddb700 1 -- 172.21.15.78:0/335890493 >> v2:172.21.15.133:3300/0 conn(0x7f772415a800 msgr2=0x7f77240b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:51.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.711+0000 7f7728ddb700 1 --2- 172.21.15.78:0/335890493 >> v2:172.21.15.133:3300/0 conn(0x7f772415a800 0x7f77240b4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:51.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.711+0000 7f7728ddb700 1 -- 172.21.15.78:0/335890493 >> v1:172.21.15.133:6789/0 conn(0x7f772406bb60 legacy=0x7f77240b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:51.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.711+0000 7f7728ddb700 1 -- 172.21.15.78:0/335890493 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77240712d0 con 0x7f77240b0d20 2023-10-11T09:52:51.721 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.711+0000 7f7723fff700 1 --2- 172.21.15.78:0/335890493 >> v2:172.21.15.133:3300/0 conn(0x7f772415a800 0x7f77240b4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:51.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.711+0000 7f7728ddb700 1 --2- 172.21.15.78:0/335890493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77240b0d20 0x7f772415e590 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f77240cbea0 tx=0x7f771c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4bf174d882c2b7e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:51.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.711+0000 7f77237fe700 1 -- 172.21.15.78:0/335890493 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771c00e070 con 0x7f77240b0d20 2023-10-11T09:52:51.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.711+0000 7f77237fe700 1 -- 172.21.15.78:0/335890493 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f771c00b9c0 con 0x7f77240b0d20 2023-10-11T09:52:51.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.711+0000 7f77237fe700 1 -- 172.21.15.78:0/335890493 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771c00bd70 con 0x7f77240b0d20 2023-10-11T09:52:51.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.711+0000 7f77295dc700 1 -- 172.21.15.78:0/335890493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77240b0d20 msgr2=0x7f772415e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:51.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.711+0000 7f77295dc700 1 --2- 172.21.15.78:0/335890493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77240b0d20 0x7f772415e590 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f77240cbea0 tx=0x7f771c00b330 comp rx=0 tx=0).stop 2023-10-11T09:52:51.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.711+0000 7f77295dc700 1 -- 172.21.15.78:0/335890493 shutdown_connections 2023-10-11T09:52:51.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.711+0000 7f77295dc700 1 --2- 172.21.15.78:0/335890493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77240b0d20 0x7f772415e590 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:51.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.711+0000 7f77295dc700 1 --2- 172.21.15.78:0/335890493 >> v2:172.21.15.133:3300/0 conn(0x7f772415a800 0x7f77240b4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:51.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.711+0000 7f77295dc700 1 -- 172.21.15.78:0/335890493 >> 172.21.15.78:0/335890493 conn(0x7f77240cf820 msgr2=0x7f77240cb610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:51.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.711+0000 7f77295dc700 1 -- 172.21.15.78:0/335890493 shutdown_connections 2023-10-11T09:52:51.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.711+0000 7f77295dc700 1 -- 172.21.15.78:0/335890493 wait complete. 2023-10-11T09:52:51.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.715+0000 7f77295dc700 1 Processor -- start 2023-10-11T09:52:51.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.715+0000 7f77295dc700 1 -- start start 2023-10-11T09:52:51.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.715+0000 7f77295dc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f772406bb60 0x7f7724151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:51.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.715+0000 7f77295dc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77240b0d20 0x7f7724152d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:51.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.715+0000 7f77295dc700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f772415a800 0x7f7724153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:51.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.715+0000 7f77295dc700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772414fda0 con 0x7f77240b0d20 2023-10-11T09:52:51.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.715+0000 7f77295dc700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7724151fe0 con 0x7f772406bb60 2023-10-11T09:52:51.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.715+0000 7f77295dc700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77241522c0 con 0x7f772415a800 2023-10-11T09:52:51.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.715+0000 7f771bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77240b0d20 0x7f7724152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:51.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.715+0000 7f771bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77240b0d20 0x7f7724152d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48414/0 (socket says 172.21.15.78:48414) 2023-10-11T09:52:51.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.715+0000 7f771bfff700 1 -- 172.21.15.78:0/2974405846 learned_addr learned my addr 172.21.15.78:0/2974405846 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:51.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.715+0000 7f7723fff700 1 --2- 172.21.15.78:0/2974405846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f772406bb60 0x7f7724151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:51.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.719+0000 7f7728ddb700 1 --2- 172.21.15.78:0/2974405846 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f772415a800 0x7f7724153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:51.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.719+0000 7f771bfff700 1 -- 172.21.15.78:0/2974405846 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f772415a800 msgr2=0x7f7724153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:51.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.719+0000 7f771bfff700 1 --2- 172.21.15.78:0/2974405846 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f772415a800 0x7f7724153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:51.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.719+0000 7f771bfff700 1 -- 172.21.15.78:0/2974405846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f772406bb60 msgr2=0x7f7724151620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:51.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.719+0000 7f771bfff700 1 --2- 172.21.15.78:0/2974405846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f772406bb60 0x7f7724151620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:51.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.719+0000 7f771bfff700 1 -- 172.21.15.78:0/2974405846 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f771c009b30 con 0x7f77240b0d20 2023-10-11T09:52:51.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.719+0000 7f7728ddb700 1 --2- 172.21.15.78:0/2974405846 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f772415a800 0x7f7724153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:51.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.719+0000 7f771bfff700 1 --2- 172.21.15.78:0/2974405846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77240b0d20 0x7f7724152d00 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f770c00c450 tx=0x7f770c00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:51.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.719+0000 7f77217fa700 1 -- 172.21.15.78:0/2974405846 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c0052d0 con 0x7f77240b0d20 2023-10-11T09:52:51.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.719+0000 7f77217fa700 1 -- 172.21.15.78:0/2974405846 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770c005430 con 0x7f77240b0d20 2023-10-11T09:52:51.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.719+0000 7f77295dc700 1 -- 172.21.15.78:0/2974405846 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77241559a0 con 0x7f77240b0d20 2023-10-11T09:52:51.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.719+0000 7f77217fa700 1 -- 172.21.15.78:0/2974405846 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c003930 con 0x7f77240b0d20 2023-10-11T09:52:51.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.719+0000 7f77295dc700 1 -- 172.21.15.78:0/2974405846 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7724155650 con 0x7f77240b0d20 2023-10-11T09:52:51.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.719+0000 7f77295dc700 1 -- 172.21.15.78:0/2974405846 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7724155810 con 0x7f77240b0d20 2023-10-11T09:52:51.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.723+0000 7f77217fa700 1 -- 172.21.15.78:0/2974405846 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f770c003a90 con 0x7f77240b0d20 2023-10-11T09:52:51.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.723+0000 7f77217fa700 1 --2- 172.21.15.78:0/2974405846 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7704065ed0 0x7f7704068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:51.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.723+0000 7f77217fa700 1 -- 172.21.15.78:0/2974405846 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f770c0829c0 con 0x7f77240b0d20 2023-10-11T09:52:51.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.723+0000 7f7723fff700 1 --2- 172.21.15.78:0/2974405846 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7704065ed0 0x7f7704068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:51.737 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.723+0000 7f7723fff700 1 --2- 172.21.15.78:0/2974405846 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7704065ed0 0x7f7704068380 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f77140044b0 tx=0x7f7714008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:51.738 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.731+0000 7f77217fa700 1 -- 172.21.15.78:0/2974405846 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f770c050250 con 0x7f77240b0d20 2023-10-11T09:52:51.911 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.903+0000 7f77295dc700 1 -- 172.21.15.78:0/2974405846 --> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f7724071890 con 0x7f7704065ed0 2023-10-11T09:52:51.912 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.903+0000 7f77217fa700 1 -- 172.21.15.78:0/2974405846 <== mgr.4101 v2:172.21.15.78:6832/35606 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7714004590 con 0x7f7704065ed0 2023-10-11T09:52:51.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.907+0000 7f77295dc700 1 -- 172.21.15.78:0/2974405846 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7704065ed0 msgr2=0x7f7704068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:51.914 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.907+0000 7f77295dc700 1 --2- 172.21.15.78:0/2974405846 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7704065ed0 0x7f7704068380 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f77140044b0 tx=0x7f7714008040 comp rx=0 tx=0).stop 2023-10-11T09:52:51.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.907+0000 7f77295dc700 1 -- 172.21.15.78:0/2974405846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77240b0d20 msgr2=0x7f7724152d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:51.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.907+0000 7f77295dc700 1 --2- 172.21.15.78:0/2974405846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77240b0d20 0x7f7724152d00 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f770c00c450 tx=0x7f770c00c760 comp rx=0 tx=0).stop 2023-10-11T09:52:51.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.907+0000 7f77295dc700 1 -- 172.21.15.78:0/2974405846 shutdown_connections 2023-10-11T09:52:51.915 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.907+0000 7f77295dc700 1 --2- 172.21.15.78:0/2974405846 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f7704065ed0 0x7f7704068380 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:51.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.907+0000 7f77295dc700 1 --2- 172.21.15.78:0/2974405846 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f772415a800 0x7f7724153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:51.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.907+0000 7f77295dc700 1 --2- 172.21.15.78:0/2974405846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77240b0d20 0x7f7724152d00 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:51.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.907+0000 7f77295dc700 1 --2- 172.21.15.78:0/2974405846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f772406bb60 0x7f7724151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:51.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.907+0000 7f77295dc700 1 -- 172.21.15.78:0/2974405846 >> 172.21.15.78:0/2974405846 conn(0x7f77240cf820 msgr2=0x7f772415c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:51.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.907+0000 7f77295dc700 1 -- 172.21.15.78:0/2974405846 shutdown_connections 2023-10-11T09:52:51.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:51.907+0000 7f77295dc700 1 -- 172.21.15.78:0/2974405846 wait complete. 2023-10-11T09:52:51.918 INFO:teuthology.orchestra.run.smithi078.stderr:instructed osd(s) 1 to deep-scrub 2023-10-11T09:52:51.938 INFO:tasks.ceph:Scrubbing osd.2 2023-10-11T09:52:51.939 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-11T09:52:52.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.115+0000 7fdde8ed5700 1 Processor -- start 2023-10-11T09:52:52.121 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.115+0000 7fdde8ed5700 1 -- start start 2023-10-11T09:52:52.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.115+0000 7fdde8ed5700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdde415c980 0x7fdde40d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:52.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.115+0000 7fdde8ed5700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdde40d13b0 0x7fdde40b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:52.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.115+0000 7fdde8ed5700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdde40cf8b0 con 0x7fdde40b5060 2023-10-11T09:52:52.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.115+0000 7fdde8ed5700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdde40b54e0 con 0x7fdde40d13b0 2023-10-11T09:52:52.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.115+0000 7fdde8ed5700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdde40d2290 con 0x7fdde415c980 2023-10-11T09:52:52.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.119+0000 7fdde37fe700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdde415c980 0x7fdde40d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:52.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.119+0000 7fdde37fe700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdde415c980 0x7fdde40d0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.78:43858/0 (socket says 172.21.15.78:43858) 2023-10-11T09:52:52.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.119+0000 7fdde2ffd700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdde40d13b0 0x7fdde40b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:52.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.119+0000 7fdde37fe700 1 -- 172.21.15.78:0/2804439976 learned_addr learned my addr 172.21.15.78:0/2804439976 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:52.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.119+0000 7fdde27fc700 1 -- 172.21.15.78:0/2804439976 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 911965265 0 0) 0x7fdde40cf8b0 con 0x7fdde40b5060 2023-10-11T09:52:52.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.119+0000 7fdde27fc700 1 -- 172.21.15.78:0/2804439976 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddcc0035d0 con 0x7fdde40b5060 2023-10-11T09:52:52.129 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.119+0000 7fdde37fe700 1 -- 172.21.15.78:0/2804439976 >> v2:172.21.15.133:3300/0 conn(0x7fdde40d13b0 msgr2=0x7fdde40b4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:52.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.119+0000 7fdde37fe700 1 --2- 172.21.15.78:0/2804439976 >> v2:172.21.15.133:3300/0 conn(0x7fdde40d13b0 0x7fdde40b4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:52.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.119+0000 7fdde37fe700 1 -- 172.21.15.78:0/2804439976 >> v1:172.21.15.133:6789/0 conn(0x7fdde40b5060 legacy=0x7fdde406aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:52.130 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.119+0000 7fdde37fe700 1 -- 172.21.15.78:0/2804439976 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde40d19f0 con 0x7fdde415c980 2023-10-11T09:52:52.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.119+0000 7fdde2ffd700 1 --2- 172.21.15.78:0/2804439976 >> v2:172.21.15.133:3300/0 conn(0x7fdde40d13b0 0x7fdde40b4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:52:52.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.119+0000 7fdde37fe700 1 --2- 172.21.15.78:0/2804439976 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdde415c980 0x7fdde40d0e70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fddd4009910 tx=0x7fddd4009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ba152d72a06a733 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:52.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.119+0000 7fdde27fc700 1 -- 172.21.15.78:0/2804439976 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd4011830 con 0x7fdde415c980 2023-10-11T09:52:52.131 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.119+0000 7fdde27fc700 1 -- 172.21.15.78:0/2804439976 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd4011990 con 0x7fdde415c980 2023-10-11T09:52:52.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.119+0000 7fdde27fc700 1 -- 172.21.15.78:0/2804439976 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd4011d40 con 0x7fdde415c980 2023-10-11T09:52:52.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.119+0000 7fdde8ed5700 1 -- 172.21.15.78:0/2804439976 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdde415c980 msgr2=0x7fdde40d0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:52.132 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.119+0000 7fdde8ed5700 1 --2- 172.21.15.78:0/2804439976 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdde415c980 0x7fdde40d0e70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fddd4009910 tx=0x7fddd4009c20 comp rx=0 tx=0).stop 2023-10-11T09:52:52.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.119+0000 7fdde8ed5700 1 -- 172.21.15.78:0/2804439976 shutdown_connections 2023-10-11T09:52:52.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.119+0000 7fdde8ed5700 1 --2- 172.21.15.78:0/2804439976 >> v2:172.21.15.133:3300/0 conn(0x7fdde40d13b0 0x7fdde40b4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:52.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.119+0000 7fdde8ed5700 1 --2- 172.21.15.78:0/2804439976 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdde415c980 0x7fdde40d0e70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:52.133 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.119+0000 7fdde8ed5700 1 -- 172.21.15.78:0/2804439976 >> 172.21.15.78:0/2804439976 conn(0x7fdde40724b0 msgr2=0x7fdde40d30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:52.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.119+0000 7fdde8ed5700 1 -- 172.21.15.78:0/2804439976 shutdown_connections 2023-10-11T09:52:52.134 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.119+0000 7fdde8ed5700 1 -- 172.21.15.78:0/2804439976 wait complete. 2023-10-11T09:52:52.137 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.123+0000 7fdde8ed5700 1 Processor -- start 2023-10-11T09:52:52.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.123+0000 7fdde8ed5700 1 -- start start 2023-10-11T09:52:52.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.123+0000 7fdde8ed5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdde40b5060 0x7fdde41f2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:52.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.123+0000 7fdde8ed5700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdde40d13b0 0x7fdde40ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:52.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.123+0000 7fdde8ed5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdde415c980 0x7fdde41529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:52.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.123+0000 7fdde8ed5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde415d7d0 con 0x7fdde40b5060 2023-10-11T09:52:52.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.123+0000 7fdde8ed5700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde41f2c50 con 0x7fdde415c980 2023-10-11T09:52:52.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.123+0000 7fdde8ed5700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdde4150500 con 0x7fdde40d13b0 2023-10-11T09:52:52.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.123+0000 7fdde37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdde40b5060 0x7fdde41f2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:52.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.123+0000 7fdde37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdde40b5060 0x7fdde41f2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48418/0 (socket says 172.21.15.78:48418) 2023-10-11T09:52:52.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.123+0000 7fdde37fe700 1 -- 172.21.15.78:0/2428590562 learned_addr learned my addr 172.21.15.78:0/2428590562 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:52.140 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.127+0000 7fdde2ffd700 1 --2- 172.21.15.78:0/2428590562 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdde40d13b0 0x7fdde40ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:52.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.127+0000 7fdde3fff700 1 --2- 172.21.15.78:0/2428590562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdde415c980 0x7fdde41529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:52.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.127+0000 7fdde37fe700 1 -- 172.21.15.78:0/2428590562 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdde40d13b0 msgr2=0x7fdde40ca6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:52.141 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.127+0000 7fdde37fe700 1 --2- 172.21.15.78:0/2428590562 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdde40d13b0 0x7fdde40ca6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:52.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.127+0000 7fdde37fe700 1 -- 172.21.15.78:0/2428590562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdde415c980 msgr2=0x7fdde41529c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:52.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.127+0000 7fdde37fe700 1 --2- 172.21.15.78:0/2428590562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdde415c980 0x7fdde41529c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:52.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.127+0000 7fdde37fe700 1 -- 172.21.15.78:0/2428590562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd80094b0 con 0x7fdde40b5060 2023-10-11T09:52:52.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.127+0000 7fdde3fff700 1 --2- 172.21.15.78:0/2428590562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdde415c980 0x7fdde41529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:52.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.127+0000 7fdde37fe700 1 --2- 172.21.15.78:0/2428590562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdde40b5060 0x7fdde41f2710 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fddd4000c00 tx=0x7fddd4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:52.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.127+0000 7fddcbfff700 1 -- 172.21.15.78:0/2428590562 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd4004500 con 0x7fdde40b5060 2023-10-11T09:52:52.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.127+0000 7fdde8ed5700 1 -- 172.21.15.78:0/2428590562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddd4009570 con 0x7fdde40b5060 2023-10-11T09:52:52.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.127+0000 7fddcbfff700 1 -- 172.21.15.78:0/2428590562 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd40036c0 con 0x7fdde40b5060 2023-10-11T09:52:52.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.127+0000 7fddcbfff700 1 -- 172.21.15.78:0/2428590562 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd4003a70 con 0x7fdde40b5060 2023-10-11T09:52:52.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.127+0000 7fdde8ed5700 1 -- 172.21.15.78:0/2428590562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdde41521c0 con 0x7fdde40b5060 2023-10-11T09:52:52.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.127+0000 7fdde8ed5700 1 -- 172.21.15.78:0/2428590562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdde40d1fb0 con 0x7fdde40b5060 2023-10-11T09:52:52.145 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.131+0000 7fddcbfff700 1 -- 172.21.15.78:0/2428590562 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fddd4003bd0 con 0x7fdde40b5060 2023-10-11T09:52:52.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.131+0000 7fddcbfff700 1 --2- 172.21.15.78:0/2428590562 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fddd0065ed0 0x7fddd0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:52.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.131+0000 7fddcbfff700 1 -- 172.21.15.78:0/2428590562 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fddd4021030 con 0x7fdde40b5060 2023-10-11T09:52:52.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.131+0000 7fdde2ffd700 1 --2- 172.21.15.78:0/2428590562 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fddd0065ed0 0x7fddd0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:52.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.131+0000 7fddcbfff700 1 --2- 172.21.15.78:0/2428590562 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x7fddd006c480 0x7fddd006e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:52.147 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.131+0000 7fddcbfff700 1 -- 172.21.15.78:0/2428590562 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fddd006ef40 con 0x7fddd006c480 2023-10-11T09:52:52.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.131+0000 7fddcbfff700 1 -- 172.21.15.78:0/2428590562 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=132) v2 ==== 24+0+0 (secure 0 0 0) 0x7fddd4089280 con 0x7fdde40b5060 2023-10-11T09:52:52.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.131+0000 7fdde3fff700 1 --2- 172.21.15.78:0/2428590562 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x7fddd006c480 0x7fddd006e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:52.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.131+0000 7fdde2ffd700 1 --2- 172.21.15.78:0/2428590562 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fddd0065ed0 0x7fddd0068380 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fdde40cc2b0 tx=0x7fddd8009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:52.148 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.135+0000 7fdde3fff700 1 --2- 172.21.15.78:0/2428590562 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x7fddd006c480 0x7fddd006e890 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:52.149 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.135+0000 7fddcbfff700 1 -- 172.21.15.78:0/2428590562 <== osd.2 v2:172.21.15.78:6808/35732 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fddd006ef40 con 0x7fddd006c480 2023-10-11T09:52:52.158 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.151+0000 7fdde8ed5700 1 -- 172.21.15.78:0/2428590562 --> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fdde40d1cd0 con 0x7fddd006c480 2023-10-11T09:52:52.159 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.151+0000 7fddcbfff700 1 -- 172.21.15.78:0/2428590562 <== osd.2 v2:172.21.15.78:6808/35732 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fdddc003a20 con 0x7fddd006c480 2023-10-11T09:52:52.160 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:52:52.162 INFO:teuthology.orchestra.run.smithi078.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-11T09:52:52.162 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:52:52.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.151+0000 7fdde8ed5700 1 -- 172.21.15.78:0/2428590562 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x7fddd006c480 msgr2=0x7fddd006e890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:52.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.151+0000 7fdde8ed5700 1 --2- 172.21.15.78:0/2428590562 >> [v2:172.21.15.78:6808/35732,v1:172.21.15.78:6810/35732] conn(0x7fddd006c480 0x7fddd006e890 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:52.164 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.151+0000 7fdde8ed5700 1 -- 172.21.15.78:0/2428590562 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fddd0065ed0 msgr2=0x7fddd0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:52.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.151+0000 7fdde8ed5700 1 --2- 172.21.15.78:0/2428590562 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fddd0065ed0 0x7fddd0068380 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fdde40cc2b0 tx=0x7fddd8009240 comp rx=0 tx=0).stop 2023-10-11T09:52:52.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.151+0000 7fdde8ed5700 1 -- 172.21.15.78:0/2428590562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdde40b5060 msgr2=0x7fdde41f2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:52.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.151+0000 7fdde8ed5700 1 --2- 172.21.15.78:0/2428590562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdde40b5060 0x7fdde41f2710 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fddd4000c00 tx=0x7fddd4004770 comp rx=0 tx=0).stop 2023-10-11T09:52:52.165 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.151+0000 7fdde3fff700 1 -- 172.21.15.78:0/2428590562 reap_dead start 2023-10-11T09:52:52.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.151+0000 7fdde8ed5700 1 -- 172.21.15.78:0/2428590562 shutdown_connections 2023-10-11T09:52:52.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.151+0000 7fdde8ed5700 1 -- 172.21.15.78:0/2428590562 >> 172.21.15.78:0/2428590562 conn(0x7fdde40724b0 msgr2=0x7fdde40b8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:52.166 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.151+0000 7fdde8ed5700 1 -- 172.21.15.78:0/2428590562 shutdown_connections 2023-10-11T09:52:52.167 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.151+0000 7fdde8ed5700 1 -- 172.21.15.78:0/2428590562 wait complete. 2023-10-11T09:52:52.183 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-11T09:52:52.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.355+0000 7ff79c2ee700 1 Processor -- start 2023-10-11T09:52:52.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.355+0000 7ff79c2ee700 1 -- start start 2023-10-11T09:52:52.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.359+0000 7ff79c2ee700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff7940ce7c0 0x7ff794160bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:52.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.359+0000 7ff79c2ee700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7940ceb90 con 0x7ff7940713b0 2023-10-11T09:52:52.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.359+0000 7ff79c2ee700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff794071820 con 0x7ff7940c5b40 2023-10-11T09:52:52.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.359+0000 7ff79c2ee700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff79415cef0 con 0x7ff7940ce7c0 2023-10-11T09:52:52.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.359+0000 7ff799889700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7ff7940713b0 0x7ff7940c94f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:49776/0 (socket says 172.21.15.78:49776) 2023-10-11T09:52:52.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.359+0000 7ff799889700 1 -- 172.21.15.78:0/1968140993 learned_addr learned my addr 172.21.15.78:0/1968140993 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:52.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.359+0000 7ff79a08a700 1 --2- 172.21.15.78:0/1968140993 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff7940ce7c0 0x7ff794160bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:52.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.359+0000 7ff799088700 1 -- 172.21.15.78:0/1968140993 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 934788740 0 0) 0x7ff7940ceb90 con 0x7ff7940713b0 2023-10-11T09:52:52.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.359+0000 7ff799088700 1 -- 172.21.15.78:0/1968140993 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7800035d0 con 0x7ff7940713b0 2023-10-11T09:52:52.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.359+0000 7ff799088700 1 -- 172.21.15.78:0/1968140993 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3477135827 0 0) 0x7ff7800035d0 con 0x7ff7940713b0 2023-10-11T09:52:52.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.359+0000 7ff799088700 1 -- 172.21.15.78:0/1968140993 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff7940ce7c0 msgr2=0x7ff794160bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:52.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.359+0000 7ff799088700 1 --2- 172.21.15.78:0/1968140993 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff7940ce7c0 0x7ff794160bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:52.370 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.359+0000 7ff799088700 1 -- 172.21.15.78:0/1968140993 >> v1:172.21.15.133:6789/0 conn(0x7ff7940c5b40 legacy=0x7ff7940b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:52.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.359+0000 7ff799088700 1 -- 172.21.15.78:0/1968140993 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff794071b00 con 0x7ff7940713b0 2023-10-11T09:52:52.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.359+0000 7ff799088700 1 -- 172.21.15.78:0/1968140993 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7ff784001670 con 0x7ff7940713b0 2023-10-11T09:52:52.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.359+0000 7ff799088700 1 -- 172.21.15.78:0/1968140993 >> v1:172.21.15.78:6789/0 conn(0x7ff7940713b0 legacy=0x7ff7940c94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:52.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.359+0000 7ff799088700 1 --2- 172.21.15.78:0/1968140993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff780003a50 0x7ff780003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:52.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.359+0000 7ff799088700 1 --2- 172.21.15.78:0/1968140993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff780004870 0x7ff780004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:52.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.359+0000 7ff799088700 1 --2- 172.21.15.78:0/1968140993 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff7940ce7c0 0x7ff7800052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:52.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.359+0000 7ff799088700 1 -- 172.21.15.78:0/1968140993 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78000bbc0 con 0x7ff780004870 2023-10-11T09:52:52.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.359+0000 7ff799088700 1 -- 172.21.15.78:0/1968140993 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78000bd00 con 0x7ff780003a50 2023-10-11T09:52:52.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.359+0000 7ff799088700 1 -- 172.21.15.78:0/1968140993 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff78000be90 con 0x7ff7940ce7c0 2023-10-11T09:52:52.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.359+0000 7ff799889700 1 --2- 172.21.15.78:0/1968140993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff780004870 0x7ff780004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:52.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.359+0000 7ff79a08a700 1 --2- 172.21.15.78:0/1968140993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff780003a50 0x7ff780003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:52.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.359+0000 7ff79a88b700 1 --2- 172.21.15.78:0/1968140993 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff7940ce7c0 0x7ff7800052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:52.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.359+0000 7ff799889700 1 -- 172.21.15.78:0/1968140993 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff7940ce7c0 msgr2=0x7ff7800052d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:52.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.359+0000 7ff799889700 1 --2- 172.21.15.78:0/1968140993 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff7940ce7c0 0x7ff7800052d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:52.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.359+0000 7ff799889700 1 -- 172.21.15.78:0/1968140993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff780003a50 msgr2=0x7ff780003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:52.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.359+0000 7ff799889700 1 --2- 172.21.15.78:0/1968140993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff780003a50 0x7ff780003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:52.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.359+0000 7ff799889700 1 -- 172.21.15.78:0/1968140993 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7800035d0 con 0x7ff780004870 2023-10-11T09:52:52.375 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.359+0000 7ff79a08a700 1 --2- 172.21.15.78:0/1968140993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff780003a50 0x7ff780003e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:52.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.359+0000 7ff799889700 1 --2- 172.21.15.78:0/1968140993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff780004870 0x7ff780004ce0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7ff78400c240 tx=0x7ff7800058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bcb07b24da2aa63 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:52.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.359+0000 7ff799088700 1 -- 172.21.15.78:0/1968140993 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff784007b90 con 0x7ff780004870 2023-10-11T09:52:52.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.359+0000 7ff799088700 1 -- 172.21.15.78:0/1968140993 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78400cd60 con 0x7ff780004870 2023-10-11T09:52:52.376 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.359+0000 7ff799088700 1 -- 172.21.15.78:0/1968140993 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff784005bc0 con 0x7ff780004870 2023-10-11T09:52:52.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.363+0000 7ff79c2ee700 1 -- 172.21.15.78:0/1968140993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff780004870 msgr2=0x7ff780004ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:52.379 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.363+0000 7ff79c2ee700 1 --2- 172.21.15.78:0/1968140993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff780004870 0x7ff780004ce0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7ff78400c240 tx=0x7ff7800058a0 comp rx=0 tx=0).stop 2023-10-11T09:52:52.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.363+0000 7ff79a88b700 1 -- 172.21.15.78:0/1968140993 reap_dead start 2023-10-11T09:52:52.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.363+0000 7ff79c2ee700 1 -- 172.21.15.78:0/1968140993 shutdown_connections 2023-10-11T09:52:52.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.363+0000 7ff79c2ee700 1 -- 172.21.15.78:0/1968140993 >> 172.21.15.78:0/1968140993 conn(0x7ff7940cb920 msgr2=0x7ff7940cbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:52.380 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.363+0000 7ff79c2ee700 1 -- 172.21.15.78:0/1968140993 shutdown_connections 2023-10-11T09:52:52.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.363+0000 7ff79c2ee700 1 -- 172.21.15.78:0/1968140993 wait complete. 2023-10-11T09:52:52.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.367+0000 7ff79c2ee700 1 Processor -- start 2023-10-11T09:52:52.381 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.367+0000 7ff79c2ee700 1 -- start start 2023-10-11T09:52:52.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.367+0000 7ff79c2ee700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff780004870 0x7ff7941585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:52.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.367+0000 7ff79c2ee700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff794154f50 0x7ff794158b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:52.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.367+0000 7ff79c2ee700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff79406d2d0 0x7ff79406d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:52.382 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.367+0000 7ff79c2ee700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79406dcb0 con 0x7ff794154f50 2023-10-11T09:52:52.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.367+0000 7ff79c2ee700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79415f3d0 con 0x7ff780004870 2023-10-11T09:52:52.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.367+0000 7ff79c2ee700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff794156600 con 0x7ff79406d2d0 2023-10-11T09:52:52.383 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.367+0000 7ff799889700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff794154f50 0x7ff794158b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:52.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.367+0000 7ff799889700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff794154f50 0x7ff794158b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48436/0 (socket says 172.21.15.78:48436) 2023-10-11T09:52:52.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.367+0000 7ff799889700 1 -- 172.21.15.78:0/1304350870 learned_addr learned my addr 172.21.15.78:0/1304350870 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:52.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.367+0000 7ff79a88b700 1 --2- 172.21.15.78:0/1304350870 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff79406d2d0 0x7ff79406d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:52.384 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.367+0000 7ff79a08a700 1 --2- 172.21.15.78:0/1304350870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff780004870 0x7ff7941585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:52.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.367+0000 7ff799889700 1 -- 172.21.15.78:0/1304350870 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff79406d2d0 msgr2=0x7ff79406d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:52.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.367+0000 7ff799889700 1 --2- 172.21.15.78:0/1304350870 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff79406d2d0 0x7ff79406d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:52.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.367+0000 7ff799889700 1 -- 172.21.15.78:0/1304350870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff780004870 msgr2=0x7ff7941585c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:52.385 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.367+0000 7ff799889700 1 --2- 172.21.15.78:0/1304350870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff780004870 0x7ff7941585c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:52.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.367+0000 7ff799889700 1 -- 172.21.15.78:0/1304350870 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7941568e0 con 0x7ff794154f50 2023-10-11T09:52:52.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.367+0000 7ff799889700 1 --2- 172.21.15.78:0/1304350870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff794154f50 0x7ff794158b00 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7ff780005810 tx=0x7ff784001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:52.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.371+0000 7ff78affd700 1 -- 172.21.15.78:0/1304350870 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff784004ca0 con 0x7ff794154f50 2023-10-11T09:52:52.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.371+0000 7ff78affd700 1 -- 172.21.15.78:0/1304350870 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff784004e00 con 0x7ff794154f50 2023-10-11T09:52:52.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.371+0000 7ff79c2ee700 1 -- 172.21.15.78:0/1304350870 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff794156bc0 con 0x7ff794154f50 2023-10-11T09:52:52.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.371+0000 7ff78affd700 1 -- 172.21.15.78:0/1304350870 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff784007310 con 0x7ff794154f50 2023-10-11T09:52:52.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.371+0000 7ff79c2ee700 1 -- 172.21.15.78:0/1304350870 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff794155600 con 0x7ff794154f50 2023-10-11T09:52:52.388 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.371+0000 7ff79c2ee700 1 -- 172.21.15.78:0/1304350870 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7940caae0 con 0x7ff794154f50 2023-10-11T09:52:52.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.371+0000 7ff78affd700 1 -- 172.21.15.78:0/1304350870 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7ff784007470 con 0x7ff794154f50 2023-10-11T09:52:52.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.375+0000 7ff78affd700 1 --2- 172.21.15.78:0/1304350870 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff778065f00 0x7ff7780683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:52.389 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.375+0000 7ff78affd700 1 -- 172.21.15.78:0/1304350870 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5468+0+0 (secure 0 0 0) 0x7ff7840849b0 con 0x7ff794154f50 2023-10-11T09:52:52.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.375+0000 7ff79a08a700 1 --2- 172.21.15.78:0/1304350870 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff778065f00 0x7ff7780683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:52.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.375+0000 7ff79a08a700 1 --2- 172.21.15.78:0/1304350870 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff778065f00 0x7ff7780683b0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff790009bd0 tx=0x7ff790008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:52.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.379+0000 7ff78affd700 1 -- 172.21.15.78:0/1304350870 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7ff784055df0 con 0x7ff794154f50 2023-10-11T09:52:52.560 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.551+0000 7ff79c2ee700 1 -- 172.21.15.78:0/1304350870 --> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7ff794071de0 con 0x7ff778065f00 2023-10-11T09:52:52.561 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.555+0000 7ff78affd700 1 -- 172.21.15.78:0/1304350870 <== mgr.4101 v2:172.21.15.78:6832/35606 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff790009820 con 0x7ff778065f00 2023-10-11T09:52:52.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.555+0000 7ff79c2ee700 1 -- 172.21.15.78:0/1304350870 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff778065f00 msgr2=0x7ff7780683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:52.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.555+0000 7ff79c2ee700 1 --2- 172.21.15.78:0/1304350870 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff778065f00 0x7ff7780683b0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff790009bd0 tx=0x7ff790008040 comp rx=0 tx=0).stop 2023-10-11T09:52:52.564 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.555+0000 7ff79c2ee700 1 -- 172.21.15.78:0/1304350870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff794154f50 msgr2=0x7ff794158b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:52.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.555+0000 7ff79c2ee700 1 --2- 172.21.15.78:0/1304350870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff794154f50 0x7ff794158b00 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7ff780005810 tx=0x7ff784001930 comp rx=0 tx=0).stop 2023-10-11T09:52:52.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.555+0000 7ff79c2ee700 1 -- 172.21.15.78:0/1304350870 shutdown_connections 2023-10-11T09:52:52.565 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.555+0000 7ff79c2ee700 1 --2- 172.21.15.78:0/1304350870 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7ff778065f00 0x7ff7780683b0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:52.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.555+0000 7ff79c2ee700 1 --2- 172.21.15.78:0/1304350870 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff79406d2d0 0x7ff79406d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:52.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.555+0000 7ff79c2ee700 1 --2- 172.21.15.78:0/1304350870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff794154f50 0x7ff794158b00 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:52.566 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.555+0000 7ff79c2ee700 1 --2- 172.21.15.78:0/1304350870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff780004870 0x7ff7941585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:52.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.555+0000 7ff79c2ee700 1 -- 172.21.15.78:0/1304350870 >> 172.21.15.78:0/1304350870 conn(0x7ff7940cb920 msgr2=0x7ff79415aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:52.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.555+0000 7ff79c2ee700 1 -- 172.21.15.78:0/1304350870 shutdown_connections 2023-10-11T09:52:52.567 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.555+0000 7ff79c2ee700 1 -- 172.21.15.78:0/1304350870 wait complete. 2023-10-11T09:52:52.568 INFO:teuthology.orchestra.run.smithi078.stderr:instructed osd(s) 2 to deep-scrub 2023-10-11T09:52:52.585 INFO:tasks.ceph:Scrubbing osd.3 2023-10-11T09:52:52.586 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-11T09:52:52.762 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.755+0000 7fb2c4625700 1 Processor -- start 2023-10-11T09:52:52.763 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.755+0000 7fb2c4625700 1 -- start start 2023-10-11T09:52:52.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.759+0000 7fb2c4625700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb2bc164260 0x7fb2bc0caa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:52.766 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.759+0000 7fb2c4625700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2bc0c2180 0x7fb2bc071690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:52.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.759+0000 7fb2c4625700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fb2bc164630 0x7fb2bc071bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:52.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.759+0000 7fb2c4625700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2bc0cb760 con 0x7fb2bc164630 2023-10-11T09:52:52.767 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.759+0000 7fb2c4625700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2bc0cb480 con 0x7fb2bc164260 2023-10-11T09:52:52.768 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.759+0000 7fb2c4625700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc164c40 con 0x7fb2bc0c2180 2023-10-11T09:52:52.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.759+0000 7fb2c2bc2700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fb2bc164630 0x7fb2bc071bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:52.769 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.759+0000 7fb2c2bc2700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fb2bc164630 0x7fb2bc071bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48440/0 (socket says 172.21.15.78:48440) 2023-10-11T09:52:52.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.759+0000 7fb2c2bc2700 1 -- 172.21.15.78:0/3328032676 learned_addr learned my addr 172.21.15.78:0/3328032676 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:52.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.759+0000 7fb2c1bc0700 1 --2- 172.21.15.78:0/3328032676 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2bc0c2180 0x7fb2bc071690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:52.770 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.759+0000 7fb2c23c1700 1 --2- 172.21.15.78:0/3328032676 >> v2:172.21.15.133:3300/0 conn(0x7fb2bc164260 0x7fb2bc0caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:52.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.759+0000 7fb2c2bc2700 1 -- 172.21.15.78:0/3328032676 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2bc0c2180 msgr2=0x7fb2bc071690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:52.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.759+0000 7fb2c2bc2700 1 --2- 172.21.15.78:0/3328032676 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2bc0c2180 0x7fb2bc071690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:52.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.759+0000 7fb2c2bc2700 1 -- 172.21.15.78:0/3328032676 >> v2:172.21.15.133:3300/0 conn(0x7fb2bc164260 msgr2=0x7fb2bc0caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:52.771 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.759+0000 7fb2c2bc2700 1 --2- 172.21.15.78:0/3328032676 >> v2:172.21.15.133:3300/0 conn(0x7fb2bc164260 0x7fb2bc0caa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:52.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.759+0000 7fb2c2bc2700 1 -- 172.21.15.78:0/3328032676 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2bc164f20 con 0x7fb2bc164630 2023-10-11T09:52:52.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.759+0000 7fb2c23c1700 1 --2- 172.21.15.78:0/3328032676 >> v2:172.21.15.133:3300/0 conn(0x7fb2bc164260 0x7fb2bc0caa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:52.772 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.759+0000 7fb2c2bc2700 1 --2- 172.21.15.78:0/3328032676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2bc164630 0x7fb2bc071bd0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc06bae0 tx=0x7fb2b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c574f959ef7848e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:52.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.763+0000 7fb2c13bf700 1 -- 172.21.15.78:0/3328032676 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b400e070 con 0x7fb2bc164630 2023-10-11T09:52:52.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.763+0000 7fb2c13bf700 1 -- 172.21.15.78:0/3328032676 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b400b910 con 0x7fb2bc164630 2023-10-11T09:52:52.773 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.763+0000 7fb2c13bf700 1 -- 172.21.15.78:0/3328032676 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b400bcc0 con 0x7fb2bc164630 2023-10-11T09:52:52.775 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.763+0000 7fb2c4625700 1 -- 172.21.15.78:0/3328032676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2bc164630 msgr2=0x7fb2bc071bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:52.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.763+0000 7fb2c4625700 1 --2- 172.21.15.78:0/3328032676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2bc164630 0x7fb2bc071bd0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc06bae0 tx=0x7fb2b400b330 comp rx=0 tx=0).stop 2023-10-11T09:52:52.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.763+0000 7fb2c4625700 1 -- 172.21.15.78:0/3328032676 shutdown_connections 2023-10-11T09:52:52.776 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.763+0000 7fb2c4625700 1 --2- 172.21.15.78:0/3328032676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2bc164630 0x7fb2bc071bd0 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:52.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.763+0000 7fb2c4625700 1 --2- 172.21.15.78:0/3328032676 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2bc0c2180 0x7fb2bc071690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:52.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.763+0000 7fb2c4625700 1 --2- 172.21.15.78:0/3328032676 >> v2:172.21.15.133:3300/0 conn(0x7fb2bc164260 0x7fb2bc0caa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:52.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.763+0000 7fb2c4625700 1 -- 172.21.15.78:0/3328032676 >> 172.21.15.78:0/3328032676 conn(0x7fb2bc06ae00 msgr2=0x7fb2bc0b0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:52.777 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.763+0000 7fb2c4625700 1 -- 172.21.15.78:0/3328032676 shutdown_connections 2023-10-11T09:52:52.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.763+0000 7fb2c4625700 1 -- 172.21.15.78:0/3328032676 wait complete. 2023-10-11T09:52:52.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.767+0000 7fb2c4625700 1 Processor -- start 2023-10-11T09:52:52.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.767+0000 7fb2c4625700 1 -- start start 2023-10-11T09:52:52.778 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.767+0000 7fb2c4625700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2bc0c2180 0x7fb2bc162590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:52.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.767+0000 7fb2c4625700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2bc164260 0x7fb2bc0c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:52.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.767+0000 7fb2c4625700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2bc164630 0x7fb2bc0c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:52.779 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.767+0000 7fb2c4625700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc162ad0 con 0x7fb2bc164260 2023-10-11T09:52:52.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.767+0000 7fb2c4625700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc0c67c0 con 0x7fb2bc164630 2023-10-11T09:52:52.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.767+0000 7fb2c4625700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc0c6aa0 con 0x7fb2bc0c2180 2023-10-11T09:52:52.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.767+0000 7fb2c1bc0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2bc164260 0x7fb2bc0c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:52.780 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.767+0000 7fb2c1bc0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2bc164260 0x7fb2bc0c5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48444/0 (socket says 172.21.15.78:48444) 2023-10-11T09:52:52.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.767+0000 7fb2c1bc0700 1 -- 172.21.15.78:0/194344682 learned_addr learned my addr 172.21.15.78:0/194344682 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:52.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.767+0000 7fb2c23c1700 1 --2- 172.21.15.78:0/194344682 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2bc0c2180 0x7fb2bc162590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:52.781 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.767+0000 7fb2c2bc2700 1 --2- 172.21.15.78:0/194344682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2bc164630 0x7fb2bc0c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:52.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.767+0000 7fb2c1bc0700 1 -- 172.21.15.78:0/194344682 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2bc0c2180 msgr2=0x7fb2bc162590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:52.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.767+0000 7fb2c1bc0700 1 --2- 172.21.15.78:0/194344682 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2bc0c2180 0x7fb2bc162590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:52.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.767+0000 7fb2c1bc0700 1 -- 172.21.15.78:0/194344682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2bc164630 msgr2=0x7fb2bc0c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:52.782 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.767+0000 7fb2c1bc0700 1 --2- 172.21.15.78:0/194344682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2bc164630 0x7fb2bc0c78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:52.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.767+0000 7fb2c1bc0700 1 -- 172.21.15.78:0/194344682 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b4009b30 con 0x7fb2bc164260 2023-10-11T09:52:52.783 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.767+0000 7fb2c1bc0700 1 --2- 172.21.15.78:0/194344682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2bc164260 0x7fb2bc0c5f70 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac00e960 tx=0x7fb2ac00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:52.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.767+0000 7fb2b2ffd700 1 -- 172.21.15.78:0/194344682 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac0096c0 con 0x7fb2bc164260 2023-10-11T09:52:52.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.767+0000 7fb2b2ffd700 1 -- 172.21.15.78:0/194344682 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2ac013680 con 0x7fb2bc164260 2023-10-11T09:52:52.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.767+0000 7fb2c4625700 1 -- 172.21.15.78:0/194344682 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2bc0c6d80 con 0x7fb2bc164260 2023-10-11T09:52:52.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.767+0000 7fb2b2ffd700 1 -- 172.21.15.78:0/194344682 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac013a10 con 0x7fb2bc164260 2023-10-11T09:52:52.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.767+0000 7fb2c4625700 1 -- 172.21.15.78:0/194344682 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2bc0c64b0 con 0x7fb2bc164260 2023-10-11T09:52:52.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.771+0000 7fb2c4625700 1 -- 172.21.15.78:0/194344682 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb2bc165200 con 0x7fb2bc164260 2023-10-11T09:52:52.786 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.771+0000 7fb2b2ffd700 1 -- 172.21.15.78:0/194344682 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fb2ac013b90 con 0x7fb2bc164260 2023-10-11T09:52:52.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.775+0000 7fb2b2ffd700 1 --2- 172.21.15.78:0/194344682 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb2a0065ed0 0x7fb2a0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:52.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.775+0000 7fb2b2ffd700 1 -- 172.21.15.78:0/194344682 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fb2ac083510 con 0x7fb2bc164260 2023-10-11T09:52:52.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.775+0000 7fb2c23c1700 1 --2- 172.21.15.78:0/194344682 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb2a0065ed0 0x7fb2a0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:52.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.775+0000 7fb2b2ffd700 1 --2- 172.21.15.78:0/194344682 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x7fb2a006c480 0x7fb2a006e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:52.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.775+0000 7fb2b2ffd700 1 -- 172.21.15.78:0/194344682 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb2a006ef40 con 0x7fb2a006c480 2023-10-11T09:52:52.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.775+0000 7fb2b2ffd700 1 -- 172.21.15.78:0/194344682 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=132) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2ac085af0 con 0x7fb2bc164260 2023-10-11T09:52:52.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.775+0000 7fb2c2bc2700 1 --2- 172.21.15.78:0/194344682 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x7fb2a006c480 0x7fb2a006e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:52.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.775+0000 7fb2c23c1700 1 --2- 172.21.15.78:0/194344682 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb2a0065ed0 0x7fb2a0068380 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80099f0 tx=0x7fb2b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:52.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.775+0000 7fb2c2bc2700 1 --2- 172.21.15.78:0/194344682 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x7fb2a006c480 0x7fb2a006e890 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:52.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.775+0000 7fb2b2ffd700 1 -- 172.21.15.78:0/194344682 <== osd.3 v2:172.21.15.78:6824/35738 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb2a006ef40 con 0x7fb2a006c480 2023-10-11T09:52:52.801 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.795+0000 7fb2c4625700 1 -- 172.21.15.78:0/194344682 --> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb2bc0c2800 con 0x7fb2a006c480 2023-10-11T09:52:52.802 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.795+0000 7fb2b2ffd700 1 -- 172.21.15.78:0/194344682 <== osd.3 v2:172.21.15.78:6824/35738 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fb2bc0c2800 con 0x7fb2a006c480 2023-10-11T09:52:52.802 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:52:52.803 INFO:teuthology.orchestra.run.smithi078.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-11T09:52:52.803 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:52:52.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.795+0000 7fb2c4625700 1 -- 172.21.15.78:0/194344682 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x7fb2a006c480 msgr2=0x7fb2a006e890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:52.804 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.795+0000 7fb2c4625700 1 --2- 172.21.15.78:0/194344682 >> [v2:172.21.15.78:6824/35738,v1:172.21.15.78:6825/35738] conn(0x7fb2a006c480 0x7fb2a006e890 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:52.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.795+0000 7fb2c4625700 1 -- 172.21.15.78:0/194344682 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb2a0065ed0 msgr2=0x7fb2a0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:52.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.795+0000 7fb2c4625700 1 --2- 172.21.15.78:0/194344682 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb2a0065ed0 0x7fb2a0068380 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80099f0 tx=0x7fb2b8008040 comp rx=0 tx=0).stop 2023-10-11T09:52:52.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.795+0000 7fb2c4625700 1 -- 172.21.15.78:0/194344682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2bc164260 msgr2=0x7fb2bc0c5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:52.805 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.795+0000 7fb2c4625700 1 --2- 172.21.15.78:0/194344682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2bc164260 0x7fb2bc0c5f70 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac00e960 tx=0x7fb2ac00ec70 comp rx=0 tx=0).stop 2023-10-11T09:52:52.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.795+0000 7fb2c2bc2700 1 -- 172.21.15.78:0/194344682 reap_dead start 2023-10-11T09:52:52.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.795+0000 7fb2c4625700 1 -- 172.21.15.78:0/194344682 shutdown_connections 2023-10-11T09:52:52.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.795+0000 7fb2c4625700 1 -- 172.21.15.78:0/194344682 >> 172.21.15.78:0/194344682 conn(0x7fb2bc06ae00 msgr2=0x7fb2bc0b1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:52.806 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.795+0000 7fb2c4625700 1 -- 172.21.15.78:0/194344682 shutdown_connections 2023-10-11T09:52:52.807 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.795+0000 7fb2c4625700 1 -- 172.21.15.78:0/194344682 wait complete. 2023-10-11T09:52:52.823 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-11T09:52:52.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.987+0000 7f4fee300700 1 Processor -- start 2023-10-11T09:52:52.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.991+0000 7f4fee300700 1 -- start start 2023-10-11T09:52:53.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.991+0000 7f4fee300700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4fe80ce700 0x7f4fe80c20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:53.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.991+0000 7f4fee300700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fe80cb220 con 0x7f4fe8071720 2023-10-11T09:52:53.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.991+0000 7f4fee300700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fe81613f0 con 0x7f4fe8161020 2023-10-11T09:52:53.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.991+0000 7f4fee300700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fe8071c70 con 0x7f4fe80ce700 2023-10-11T09:52:53.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.995+0000 7f4fedaff700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f4fe8071720 0x7f4fe806bc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:49782/0 (socket says 172.21.15.78:49782) 2023-10-11T09:52:53.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.995+0000 7f4fedaff700 1 -- 172.21.15.78:0/3183985145 learned_addr learned my addr 172.21.15.78:0/3183985145 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:53.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.995+0000 7f4fecafd700 1 --2- 172.21.15.78:0/3183985145 >> v2:172.21.15.133:3300/0 conn(0x7f4fe80ce700 0x7f4fe80c20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:53.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.995+0000 7f4fdffff700 1 -- 172.21.15.78:0/3183985145 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4102007143 0 0) 0x7f4fe80cb220 con 0x7f4fe8071720 2023-10-11T09:52:53.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.995+0000 7f4fdffff700 1 -- 172.21.15.78:0/3183985145 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fd40035d0 con 0x7f4fe8071720 2023-10-11T09:52:53.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.995+0000 7f4fdffff700 1 -- 172.21.15.78:0/3183985145 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4273419127 0 0) 0x7f4fe81613f0 con 0x7f4fe8161020 2023-10-11T09:52:53.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.995+0000 7f4fdffff700 1 -- 172.21.15.78:0/3183985145 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fe80cb220 con 0x7f4fe8161020 2023-10-11T09:52:53.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.995+0000 7f4fecafd700 1 -- 172.21.15.78:0/3183985145 >> v1:172.21.15.133:6789/0 conn(0x7f4fe8161020 legacy=0x7f4fe80cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:53.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.995+0000 7f4fdffff700 1 -- 172.21.15.78:0/3183985145 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1886550732 0 0) 0x7f4fd40035d0 con 0x7f4fe8071720 2023-10-11T09:52:53.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.995+0000 7f4fecafd700 1 -- 172.21.15.78:0/3183985145 >> v1:172.21.15.78:6789/0 conn(0x7f4fe8071720 legacy=0x7f4fe806bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:53.005 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.995+0000 7f4fecafd700 1 -- 172.21.15.78:0/3183985145 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe80ca650 con 0x7f4fe80ce700 2023-10-11T09:52:53.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.995+0000 7f4fecafd700 1 --2- 172.21.15.78:0/3183985145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fe80ce700 0x7f4fe80c20c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8009840 tx=0x7f4fd8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b78f1c7ccaf4fc9c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:53.006 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.995+0000 7f4fdffff700 1 -- 172.21.15.78:0/3183985145 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe00019f0 con 0x7f4fe80ce700 2023-10-11T09:52:53.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.995+0000 7f4fdffff700 1 -- 172.21.15.78:0/3183985145 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd8004030 con 0x7f4fe80ce700 2023-10-11T09:52:53.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.995+0000 7f4fdffff700 1 -- 172.21.15.78:0/3183985145 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd80043c0 con 0x7f4fe80ce700 2023-10-11T09:52:53.007 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.995+0000 7f4fee300700 1 -- 172.21.15.78:0/3183985145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fe80ce700 msgr2=0x7f4fe80c20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:53.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.995+0000 7f4fee300700 1 --2- 172.21.15.78:0/3183985145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fe80ce700 0x7f4fe80c20c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8009840 tx=0x7f4fd8009b50 comp rx=0 tx=0).stop 2023-10-11T09:52:53.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.995+0000 7f4fee300700 1 -- 172.21.15.78:0/3183985145 shutdown_connections 2023-10-11T09:52:53.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.995+0000 7f4fee300700 1 --2- 172.21.15.78:0/3183985145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fe80ce700 0x7f4fe80c20c0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:53.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.995+0000 7f4fee300700 1 -- 172.21.15.78:0/3183985145 >> 172.21.15.78:0/3183985145 conn(0x7f4fe80cf820 msgr2=0x7f4fe80cb610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:53.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.999+0000 7f4fee300700 1 -- 172.21.15.78:0/3183985145 shutdown_connections 2023-10-11T09:52:53.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.999+0000 7f4fee300700 1 -- 172.21.15.78:0/3183985145 wait complete. 2023-10-11T09:52:53.011 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.999+0000 7f4fee300700 1 Processor -- start 2023-10-11T09:52:53.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:52.999+0000 7f4fee300700 1 -- start start 2023-10-11T09:52:53.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.003+0000 7f4fee300700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fe8071720 0x7f4fe81f7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:53.012 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.003+0000 7f4fee300700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fe80ce700 0x7f4fe81f9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:53.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.003+0000 7f4fee300700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fe8161020 0x7f4fe80c2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:53.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.003+0000 7f4fee300700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe80b2a10 con 0x7f4fe8161020 2023-10-11T09:52:53.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.003+0000 7f4fee300700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe81f7fe0 con 0x7f4fe80ce700 2023-10-11T09:52:53.013 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.003+0000 7f4fee300700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe81f77c0 con 0x7f4fe8071720 2023-10-11T09:52:53.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.003+0000 7f4fedaff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fe8161020 0x7f4fe80c2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:53.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.003+0000 7f4fedaff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fe8161020 0x7f4fe80c2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48460/0 (socket says 172.21.15.78:48460) 2023-10-11T09:52:53.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.003+0000 7f4fedaff700 1 -- 172.21.15.78:0/2330253034 learned_addr learned my addr 172.21.15.78:0/2330253034 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:53.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.003+0000 7f4fed2fe700 1 --2- 172.21.15.78:0/2330253034 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fe8071720 0x7f4fe81f7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:53.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.003+0000 7f4fecafd700 1 --2- 172.21.15.78:0/2330253034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fe80ce700 0x7f4fe81f9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:53.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.003+0000 7f4fedaff700 1 -- 172.21.15.78:0/2330253034 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fe8071720 msgr2=0x7f4fe81f7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:53.015 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.003+0000 7f4fedaff700 1 --2- 172.21.15.78:0/2330253034 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fe8071720 0x7f4fe81f7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:53.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.003+0000 7f4fedaff700 1 -- 172.21.15.78:0/2330253034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fe80ce700 msgr2=0x7f4fe81f9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:53.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.003+0000 7f4fedaff700 1 --2- 172.21.15.78:0/2330253034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fe80ce700 0x7f4fe81f9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:53.016 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.003+0000 7f4fedaff700 1 -- 172.21.15.78:0/2330253034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd8009450 con 0x7f4fe8161020 2023-10-11T09:52:53.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.003+0000 7f4fedaff700 1 --2- 172.21.15.78:0/2330253034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fe8161020 0x7f4fe80c2630 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f4fe000cc40 tx=0x7f4fe000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:53.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.003+0000 7f4fddffb700 1 -- 172.21.15.78:0/2330253034 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe000aa80 con 0x7f4fe8161020 2023-10-11T09:52:53.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.003+0000 7f4fee300700 1 -- 172.21.15.78:0/2330253034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe80b2c20 con 0x7f4fe8161020 2023-10-11T09:52:53.017 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.003+0000 7f4fddffb700 1 -- 172.21.15.78:0/2330253034 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe000abe0 con 0x7f4fe8161020 2023-10-11T09:52:53.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.003+0000 7f4fddffb700 1 -- 172.21.15.78:0/2330253034 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe00070d0 con 0x7f4fe8161020 2023-10-11T09:52:53.018 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.003+0000 7f4fee300700 1 -- 172.21.15.78:0/2330253034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe80b31e0 con 0x7f4fe8161020 2023-10-11T09:52:53.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.007+0000 7f4fee300700 1 -- 172.21.15.78:0/2330253034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe8067d00 con 0x7f4fe8161020 2023-10-11T09:52:53.019 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.007+0000 7f4fddffb700 1 -- 172.21.15.78:0/2330253034 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f4fe0007230 con 0x7f4fe8161020 2023-10-11T09:52:53.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.007+0000 7f4fddffb700 1 --2- 172.21.15.78:0/2330253034 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4fcc065ed0 0x7f4fcc068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:53.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.007+0000 7f4fddffb700 1 -- 172.21.15.78:0/2330253034 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f4fe0082fe0 con 0x7f4fe8161020 2023-10-11T09:52:53.020 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.007+0000 7f4fed2fe700 1 --2- 172.21.15.78:0/2330253034 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4fcc065ed0 0x7f4fcc068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:53.021 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.011+0000 7f4fed2fe700 1 --2- 172.21.15.78:0/2330253034 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4fcc065ed0 0x7f4fcc068380 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f4fe40011e0 tx=0x7f4fe4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:53.022 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.015+0000 7f4fddffb700 1 -- 172.21.15.78:0/2330253034 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f4fe0050830 con 0x7f4fe8161020 2023-10-11T09:52:53.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.187+0000 7f4fee300700 1 -- 172.21.15.78:0/2330253034 --> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f4fe80cabd0 con 0x7f4fcc065ed0 2023-10-11T09:52:53.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.187+0000 7f4fddffb700 1 -- 172.21.15.78:0/2330253034 <== mgr.4101 v2:172.21.15.78:6832/35606 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4fe4001cd0 con 0x7f4fcc065ed0 2023-10-11T09:52:53.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.191+0000 7f4fee300700 1 -- 172.21.15.78:0/2330253034 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4fcc065ed0 msgr2=0x7f4fcc068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:53.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.191+0000 7f4fee300700 1 --2- 172.21.15.78:0/2330253034 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4fcc065ed0 0x7f4fcc068380 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f4fe40011e0 tx=0x7f4fe4009040 comp rx=0 tx=0).stop 2023-10-11T09:52:53.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.191+0000 7f4fee300700 1 -- 172.21.15.78:0/2330253034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fe8161020 msgr2=0x7f4fe80c2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:53.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.191+0000 7f4fee300700 1 --2- 172.21.15.78:0/2330253034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fe8161020 0x7f4fe80c2630 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f4fe000cc40 tx=0x7f4fe000cf50 comp rx=0 tx=0).stop 2023-10-11T09:52:53.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.191+0000 7f4fee300700 1 -- 172.21.15.78:0/2330253034 shutdown_connections 2023-10-11T09:52:53.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.191+0000 7f4fee300700 1 --2- 172.21.15.78:0/2330253034 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f4fcc065ed0 0x7f4fcc068380 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:53.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.191+0000 7f4fee300700 1 --2- 172.21.15.78:0/2330253034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fe8161020 0x7f4fe80c2630 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:53.201 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.191+0000 7f4fee300700 1 --2- 172.21.15.78:0/2330253034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fe80ce700 0x7f4fe81f9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:53.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.191+0000 7f4fee300700 1 --2- 172.21.15.78:0/2330253034 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fe8071720 0x7f4fe81f7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:53.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.191+0000 7f4fee300700 1 -- 172.21.15.78:0/2330253034 >> 172.21.15.78:0/2330253034 conn(0x7f4fe80cf820 msgr2=0x7f4fe806c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:53.202 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.191+0000 7f4fee300700 1 -- 172.21.15.78:0/2330253034 shutdown_connections 2023-10-11T09:52:53.203 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.191+0000 7f4fee300700 1 -- 172.21.15.78:0/2330253034 wait complete. 2023-10-11T09:52:53.203 INFO:teuthology.orchestra.run.smithi078.stderr:instructed osd(s) 3 to deep-scrub 2023-10-11T09:52:53.221 INFO:tasks.ceph:Scrubbing osd.4 2023-10-11T09:52:53.222 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-11T09:52:53.403 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.395+0000 7fca15417700 1 Processor -- start 2023-10-11T09:52:53.404 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.395+0000 7fca15417700 1 -- start start 2023-10-11T09:52:53.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.399+0000 7fca15417700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fca101598a0 0x7fca100cde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:53.406 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.399+0000 7fca15417700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca100ca130 0x7fca1006ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:53.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.399+0000 7fca15417700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca100704f0 con 0x7fca100ccf30 2023-10-11T09:52:53.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.399+0000 7fca15417700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca10159c70 con 0x7fca101598a0 2023-10-11T09:52:53.407 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.399+0000 7fca15417700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca100ce350 con 0x7fca100ca130 2023-10-11T09:52:53.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.399+0000 7fca0e7fc700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fca100ccf30 0x7fca100c9a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:49794/0 (socket says 172.21.15.78:49794) 2023-10-11T09:52:53.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.399+0000 7fca0e7fc700 1 -- 172.21.15.78:0/1897247582 learned_addr learned my addr 172.21.15.78:0/1897247582 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:53.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.399+0000 7fca0effd700 1 --2- 172.21.15.78:0/1897247582 >> v2:172.21.15.133:3300/0 conn(0x7fca101598a0 0x7fca100cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:53.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.399+0000 7fca0dffb700 1 -- 172.21.15.78:0/1897247582 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1144295533 0 0) 0x7fca100704f0 con 0x7fca100ccf30 2023-10-11T09:52:53.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.399+0000 7fca0dffb700 1 -- 172.21.15.78:0/1897247582 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9f80035d0 con 0x7fca100ccf30 2023-10-11T09:52:53.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.399+0000 7fca0f7fe700 1 --2- 172.21.15.78:0/1897247582 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca100ca130 0x7fca1006ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:53.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.399+0000 7fca0dffb700 1 -- 172.21.15.78:0/1897247582 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4267798155 0 0) 0x7fc9f80035d0 con 0x7fca100ccf30 2023-10-11T09:52:53.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.399+0000 7fca0dffb700 1 -- 172.21.15.78:0/1897247582 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca100ca130 msgr2=0x7fca1006ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:53.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.399+0000 7fca0dffb700 1 --2- 172.21.15.78:0/1897247582 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca100ca130 0x7fca1006ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:53.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.399+0000 7fca0dffb700 1 -- 172.21.15.78:0/1897247582 >> v2:172.21.15.133:3300/0 conn(0x7fca101598a0 msgr2=0x7fca100cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:53.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.399+0000 7fca0dffb700 1 --2- 172.21.15.78:0/1897247582 >> v2:172.21.15.133:3300/0 conn(0x7fca101598a0 0x7fca100cde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:53.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.399+0000 7fca0dffb700 1 -- 172.21.15.78:0/1897247582 --> v1:172.21.15.78:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca100cec90 con 0x7fca100ccf30 2023-10-11T09:52:53.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.399+0000 7fca0dffb700 1 -- 172.21.15.78:0/1897247582 <== mon.0 v1:172.21.15.78:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2795947073 0 0) 0x7fca04001670 con 0x7fca100ccf30 2023-10-11T09:52:53.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.399+0000 7fca0dffb700 1 -- 172.21.15.78:0/1897247582 >> v1:172.21.15.78:6789/0 conn(0x7fca100ccf30 legacy=0x7fca100c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:53.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.399+0000 7fca0dffb700 1 --2- 172.21.15.78:0/1897247582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9f8003a20 0x7fc9f8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:53.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.399+0000 7fca0dffb700 1 --2- 172.21.15.78:0/1897247582 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc9f8004870 0x7fc9f8004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:53.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.399+0000 7fca0dffb700 1 --2- 172.21.15.78:0/1897247582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9f8005220 0x7fc9f800dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:53.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.399+0000 7fca0dffb700 1 -- 172.21.15.78:0/1897247582 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca100ce350 con 0x7fc9f8005220 2023-10-11T09:52:53.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.399+0000 7fca0dffb700 1 -- 172.21.15.78:0/1897247582 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f800e180 con 0x7fc9f8003a20 2023-10-11T09:52:53.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.399+0000 7fca0dffb700 1 -- 172.21.15.78:0/1897247582 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f800e310 con 0x7fc9f8004870 2023-10-11T09:52:53.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.403+0000 7fca0e7fc700 1 --2- 172.21.15.78:0/1897247582 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc9f8004870 0x7fc9f8004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:53.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.403+0000 7fca0f7fe700 1 --2- 172.21.15.78:0/1897247582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9f8005220 0x7fc9f800dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:53.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.403+0000 7fca0effd700 1 --2- 172.21.15.78:0/1897247582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9f8003a20 0x7fc9f8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:53.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.403+0000 7fca0e7fc700 1 -- 172.21.15.78:0/1897247582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9f8003a20 msgr2=0x7fc9f8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:53.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.403+0000 7fca0e7fc700 1 --2- 172.21.15.78:0/1897247582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9f8003a20 0x7fc9f8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:53.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.403+0000 7fca0e7fc700 1 -- 172.21.15.78:0/1897247582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9f8005220 msgr2=0x7fc9f800dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:53.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.403+0000 7fca0e7fc700 1 --2- 172.21.15.78:0/1897247582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9f8005220 0x7fc9f800dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:53.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.403+0000 7fca0e7fc700 1 -- 172.21.15.78:0/1897247582 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f80035d0 con 0x7fc9f8004870 2023-10-11T09:52:53.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.403+0000 7fca0effd700 1 --2- 172.21.15.78:0/1897247582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9f8003a20 0x7fc9f8003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:53.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.403+0000 7fca0e7fc700 1 --2- 172.21.15.78:0/1897247582 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc9f8004870 0x7fc9f8004ce0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fca0400c240 tx=0x7fc9f80058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b5604a70b3f7d06 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:53.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.403+0000 7fca0dffb700 1 -- 172.21.15.78:0/1897247582 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca04007b90 con 0x7fc9f8004870 2023-10-11T09:52:53.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.403+0000 7fca0dffb700 1 -- 172.21.15.78:0/1897247582 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca0400cdc0 con 0x7fc9f8004870 2023-10-11T09:52:53.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.403+0000 7fca0dffb700 1 -- 172.21.15.78:0/1897247582 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca04005bc0 con 0x7fc9f8004870 2023-10-11T09:52:53.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.403+0000 7fca15417700 1 -- 172.21.15.78:0/1897247582 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc9f8004870 msgr2=0x7fc9f8004ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:53.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.403+0000 7fca15417700 1 --2- 172.21.15.78:0/1897247582 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc9f8004870 0x7fc9f8004ce0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fca0400c240 tx=0x7fc9f80058a0 comp rx=0 tx=0).stop 2023-10-11T09:52:53.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.403+0000 7fca0f7fe700 1 -- 172.21.15.78:0/1897247582 reap_dead start 2023-10-11T09:52:53.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.403+0000 7fca15417700 1 -- 172.21.15.78:0/1897247582 shutdown_connections 2023-10-11T09:52:53.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.403+0000 7fca15417700 1 -- 172.21.15.78:0/1897247582 >> 172.21.15.78:0/1897247582 conn(0x7fca1006ae00 msgr2=0x7fca100b0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:53.422 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.403+0000 7fca15417700 1 -- 172.21.15.78:0/1897247582 shutdown_connections 2023-10-11T09:52:53.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.403+0000 7fca15417700 1 -- 172.21.15.78:0/1897247582 wait complete. 2023-10-11T09:52:53.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.407+0000 7fca15417700 1 Processor -- start 2023-10-11T09:52:53.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.407+0000 7fca15417700 1 -- start start 2023-10-11T09:52:53.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.407+0000 7fca15417700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9f8004870 0x7fca100c6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:53.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.407+0000 7fca15417700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca100c2be0 0x7fca101f1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:53.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.407+0000 7fca15417700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca101f1a70 0x7fca100c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:53.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.407+0000 7fca15417700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca101f1110 con 0x7fc9f8004870 2023-10-11T09:52:53.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.407+0000 7fca15417700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca100c3130 con 0x7fca101f1a70 2023-10-11T09:52:53.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.407+0000 7fca15417700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca100c3410 con 0x7fca100c2be0 2023-10-11T09:52:53.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.407+0000 7fca0effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9f8004870 0x7fca100c6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:53.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.407+0000 7fca0effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9f8004870 0x7fca100c6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48476/0 (socket says 172.21.15.78:48476) 2023-10-11T09:52:53.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.407+0000 7fca0effd700 1 -- 172.21.15.78:0/3560503276 learned_addr learned my addr 172.21.15.78:0/3560503276 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:53.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.407+0000 7fca0f7fe700 1 --2- 172.21.15.78:0/3560503276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca101f1a70 0x7fca100c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:53.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.407+0000 7fca0e7fc700 1 --2- 172.21.15.78:0/3560503276 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca100c2be0 0x7fca101f1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:53.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.407+0000 7fca0effd700 1 -- 172.21.15.78:0/3560503276 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca100c2be0 msgr2=0x7fca101f1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:53.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.407+0000 7fca0effd700 1 --2- 172.21.15.78:0/3560503276 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca100c2be0 0x7fca101f1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:53.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.407+0000 7fca0effd700 1 -- 172.21.15.78:0/3560503276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca101f1a70 msgr2=0x7fca100c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:53.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.407+0000 7fca0effd700 1 --2- 172.21.15.78:0/3560503276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca101f1a70 0x7fca100c8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:53.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.407+0000 7fca0effd700 1 -- 172.21.15.78:0/3560503276 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca100c36f0 con 0x7fc9f8004870 2023-10-11T09:52:53.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.407+0000 7fca0e7fc700 1 --2- 172.21.15.78:0/3560503276 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca100c2be0 0x7fca101f1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:53.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.407+0000 7fca0effd700 1 --2- 172.21.15.78:0/3560503276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9f8004870 0x7fca100c6b50 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fca000108e0 tx=0x7fca00010ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:53.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.411+0000 7fc9f77fe700 1 -- 172.21.15.78:0/3560503276 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca00010ee0 con 0x7fc9f8004870 2023-10-11T09:52:53.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.411+0000 7fc9f77fe700 1 -- 172.21.15.78:0/3560503276 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca00004020 con 0x7fc9f8004870 2023-10-11T09:52:53.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.411+0000 7fc9f77fe700 1 -- 172.21.15.78:0/3560503276 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca000043d0 con 0x7fc9f8004870 2023-10-11T09:52:53.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.411+0000 7fca15417700 1 -- 172.21.15.78:0/3560503276 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca100c7090 con 0x7fc9f8004870 2023-10-11T09:52:53.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.411+0000 7fca15417700 1 -- 172.21.15.78:0/3560503276 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca100c8ca0 con 0x7fc9f8004870 2023-10-11T09:52:53.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.411+0000 7fca15417700 1 -- 172.21.15.78:0/3560503276 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fca1006bfd0 con 0x7fc9f8004870 2023-10-11T09:52:53.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.415+0000 7fc9f77fe700 1 -- 172.21.15.78:0/3560503276 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fca0000e850 con 0x7fc9f8004870 2023-10-11T09:52:53.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.415+0000 7fc9f77fe700 1 --2- 172.21.15.78:0/3560503276 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc9fc065f00 0x7fc9fc0683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:53.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.415+0000 7fc9f77fe700 1 -- 172.21.15.78:0/3560503276 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fca000842f0 con 0x7fc9f8004870 2023-10-11T09:52:53.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.415+0000 7fca0e7fc700 1 --2- 172.21.15.78:0/3560503276 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc9fc065f00 0x7fc9fc0683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:53.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.415+0000 7fc9f77fe700 1 --2- 172.21.15.78:0/3560503276 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x7fc9fc06c4b0 0x7fc9fc06e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:53.434 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.415+0000 7fc9f77fe700 1 -- 172.21.15.78:0/3560503276 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc9fc06ef70 con 0x7fc9fc06c4b0 2023-10-11T09:52:53.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.415+0000 7fc9f77fe700 1 -- 172.21.15.78:0/3560503276 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=132) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca00084660 con 0x7fc9f8004870 2023-10-11T09:52:53.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.415+0000 7fca0f7fe700 1 --2- 172.21.15.78:0/3560503276 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x7fc9fc06c4b0 0x7fc9fc06e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:53.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.415+0000 7fca0e7fc700 1 --2- 172.21.15.78:0/3560503276 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc9fc065f00 0x7fc9fc0683b0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fca0400c240 tx=0x7fca0400a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:53.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.415+0000 7fca0f7fe700 1 --2- 172.21.15.78:0/3560503276 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x7fc9fc06c4b0 0x7fc9fc06e8c0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:53.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.419+0000 7fc9f77fe700 1 -- 172.21.15.78:0/3560503276 <== osd.4 v2:172.21.15.133:6808/35898 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fc9fc06ef70 con 0x7fc9fc06c4b0 2023-10-11T09:52:53.443 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.435+0000 7fca15417700 1 -- 172.21.15.78:0/3560503276 --> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fca100ce550 con 0x7fc9fc06c4b0 2023-10-11T09:52:53.444 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.435+0000 7fc9f77fe700 1 -- 172.21.15.78:0/3560503276 <== osd.4 v2:172.21.15.133:6808/35898 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fca08009a20 con 0x7fc9fc06c4b0 2023-10-11T09:52:53.444 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:52:53.445 INFO:teuthology.orchestra.run.smithi078.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-11T09:52:53.445 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:52:53.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.439+0000 7fca15417700 1 -- 172.21.15.78:0/3560503276 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x7fc9fc06c4b0 msgr2=0x7fc9fc06e8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:53.446 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.439+0000 7fca15417700 1 --2- 172.21.15.78:0/3560503276 >> [v2:172.21.15.133:6808/35898,v1:172.21.15.133:6809/35898] conn(0x7fc9fc06c4b0 0x7fc9fc06e8c0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:53.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.439+0000 7fca15417700 1 -- 172.21.15.78:0/3560503276 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc9fc065f00 msgr2=0x7fc9fc0683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:53.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.439+0000 7fca15417700 1 --2- 172.21.15.78:0/3560503276 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fc9fc065f00 0x7fc9fc0683b0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fca0400c240 tx=0x7fca0400a000 comp rx=0 tx=0).stop 2023-10-11T09:52:53.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.439+0000 7fca15417700 1 -- 172.21.15.78:0/3560503276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9f8004870 msgr2=0x7fca100c6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:53.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.439+0000 7fca15417700 1 --2- 172.21.15.78:0/3560503276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9f8004870 0x7fca100c6b50 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fca000108e0 tx=0x7fca00010ca0 comp rx=0 tx=0).stop 2023-10-11T09:52:53.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.439+0000 7fca0f7fe700 1 -- 172.21.15.78:0/3560503276 reap_dead start 2023-10-11T09:52:53.448 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.439+0000 7fca15417700 1 -- 172.21.15.78:0/3560503276 shutdown_connections 2023-10-11T09:52:53.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.439+0000 7fca15417700 1 -- 172.21.15.78:0/3560503276 >> 172.21.15.78:0/3560503276 conn(0x7fca1006ae00 msgr2=0x7fca10154670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:53.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.439+0000 7fca15417700 1 -- 172.21.15.78:0/3560503276 shutdown_connections 2023-10-11T09:52:53.449 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.439+0000 7fca15417700 1 -- 172.21.15.78:0/3560503276 wait complete. 2023-10-11T09:52:53.467 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-11T09:52:53.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.635+0000 7f22ad300700 1 Processor -- start 2023-10-11T09:52:53.643 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.635+0000 7f22ad300700 1 -- start start 2023-10-11T09:52:53.644 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.639+0000 7f22ad300700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f22a81599d0 0x7f22a80c9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:53.645 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.639+0000 7f22ad300700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f22a80ca080 0x7f22a806bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:53.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.639+0000 7f22ad300700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22a80cd180 con 0x7f22a80ce220 2023-10-11T09:52:53.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.639+0000 7f22ad300700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22a80cd460 con 0x7f22a81599d0 2023-10-11T09:52:53.646 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.639+0000 7f22ad300700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22a80cd740 con 0x7f22a80ca080 2023-10-11T09:52:53.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.639+0000 7f22a7fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f22a81599d0 0x7f22a80c9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:53.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.639+0000 7f22a7fff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f22a81599d0 0x7f22a80c9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48480/0 (socket says 172.21.15.78:48480) 2023-10-11T09:52:53.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.639+0000 7f22a7fff700 1 -- 172.21.15.78:0/1583487989 learned_addr learned my addr 172.21.15.78:0/1583487989 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:53.647 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.639+0000 7f22a77fe700 1 --2- 172.21.15.78:0/1583487989 >> v2:172.21.15.133:3300/0 conn(0x7f22a80ca080 0x7f22a806bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:53.649 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.639+0000 7f22a7fff700 1 -- 172.21.15.78:0/1583487989 >> v2:172.21.15.133:3300/0 conn(0x7f22a80ca080 msgr2=0x7f22a806bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:53.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.639+0000 7f22a7fff700 1 --2- 172.21.15.78:0/1583487989 >> v2:172.21.15.133:3300/0 conn(0x7f22a80ca080 0x7f22a806bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:53.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.639+0000 7f22a6ffd700 1 -- 172.21.15.78:0/1583487989 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2245022914 0 0) 0x7f22a80cd180 con 0x7f22a80ce220 2023-10-11T09:52:53.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.639+0000 7f22a7fff700 1 -- 172.21.15.78:0/1583487989 >> v1:172.21.15.133:6789/0 conn(0x7f22a80ce220 legacy=0x7f22a815d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:53.650 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.639+0000 7f22a7fff700 1 -- 172.21.15.78:0/1583487989 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a80cda20 con 0x7f22a81599d0 2023-10-11T09:52:53.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.639+0000 7f22a7fff700 1 --2- 172.21.15.78:0/1583487989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22a81599d0 0x7f22a80c9b40 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f22a80666a0 tx=0x7f2298009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d7d4d70c4de43b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:53.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.639+0000 7f22a6ffd700 1 -- 172.21.15.78:0/1583487989 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2298011830 con 0x7f22a81599d0 2023-10-11T09:52:53.651 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.639+0000 7f22a6ffd700 1 -- 172.21.15.78:0/1583487989 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2298011990 con 0x7f22a81599d0 2023-10-11T09:52:53.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.639+0000 7f22a6ffd700 1 -- 172.21.15.78:0/1583487989 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2298011d20 con 0x7f22a81599d0 2023-10-11T09:52:53.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.643+0000 7f22ad300700 1 -- 172.21.15.78:0/1583487989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22a81599d0 msgr2=0x7f22a80c9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:53.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.643+0000 7f22ad300700 1 --2- 172.21.15.78:0/1583487989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22a81599d0 0x7f22a80c9b40 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f22a80666a0 tx=0x7f2298009bf0 comp rx=0 tx=0).stop 2023-10-11T09:52:53.652 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.643+0000 7f22ad300700 1 -- 172.21.15.78:0/1583487989 shutdown_connections 2023-10-11T09:52:53.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.643+0000 7f22ad300700 1 --2- 172.21.15.78:0/1583487989 >> v2:172.21.15.133:3300/0 conn(0x7f22a80ca080 0x7f22a806bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:53.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.643+0000 7f22ad300700 1 --2- 172.21.15.78:0/1583487989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22a81599d0 0x7f22a80c9b40 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:53.653 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.643+0000 7f22ad300700 1 -- 172.21.15.78:0/1583487989 >> 172.21.15.78:0/1583487989 conn(0x7f22a80cf820 msgr2=0x7f22a80cb610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:53.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.643+0000 7f22ad300700 1 -- 172.21.15.78:0/1583487989 shutdown_connections 2023-10-11T09:52:53.654 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.643+0000 7f22ad300700 1 -- 172.21.15.78:0/1583487989 wait complete. 2023-10-11T09:52:53.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.643+0000 7f22ad300700 1 Processor -- start 2023-10-11T09:52:53.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.643+0000 7f22ad300700 1 -- start start 2023-10-11T09:52:53.656 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.647+0000 7f22ad300700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22a80ca080 0x7f22a80c3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:53.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.647+0000 7f22ad300700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22a80ce220 0x7f22a80c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:53.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.647+0000 7f22ad300700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22a81599d0 0x7f22a80c8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:53.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.647+0000 7f22ad300700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a80c4870 con 0x7f22a81599d0 2023-10-11T09:52:53.657 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.647+0000 7f22ad300700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a80c3c60 con 0x7f22a80ca080 2023-10-11T09:52:53.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.647+0000 7f22ad300700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a80c3f40 con 0x7f22a80ce220 2023-10-11T09:52:53.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.647+0000 7f22acaff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22a81599d0 0x7f22a80c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:53.658 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.647+0000 7f22acaff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22a81599d0 0x7f22a80c8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48490/0 (socket says 172.21.15.78:48490) 2023-10-11T09:52:53.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.647+0000 7f22acaff700 1 -- 172.21.15.78:0/2786233436 learned_addr learned my addr 172.21.15.78:0/2786233436 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:53.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.647+0000 7f22a77fe700 1 --2- 172.21.15.78:0/2786233436 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22a80ce220 0x7f22a80c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:53.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.647+0000 7f22a7fff700 1 --2- 172.21.15.78:0/2786233436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22a80ca080 0x7f22a80c3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:53.659 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.647+0000 7f22acaff700 1 -- 172.21.15.78:0/2786233436 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22a80ce220 msgr2=0x7f22a80c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:53.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.647+0000 7f22acaff700 1 --2- 172.21.15.78:0/2786233436 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22a80ce220 0x7f22a80c7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:53.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.647+0000 7f22acaff700 1 -- 172.21.15.78:0/2786233436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22a80ca080 msgr2=0x7f22a80c3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:53.660 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.647+0000 7f22acaff700 1 --2- 172.21.15.78:0/2786233436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22a80ca080 0x7f22a80c3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:53.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.647+0000 7f22acaff700 1 -- 172.21.15.78:0/2786233436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2298009570 con 0x7f22a81599d0 2023-10-11T09:52:53.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.647+0000 7f22a7fff700 1 --2- 172.21.15.78:0/2786233436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22a80ca080 0x7f22a80c3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:53.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.647+0000 7f22acaff700 1 --2- 172.21.15.78:0/2786233436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22a81599d0 0x7f22a80c8760 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f22a000cc70 tx=0x7f22a000cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:53.661 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.647+0000 7f22a4ff9700 1 -- 172.21.15.78:0/2786233436 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a0003e50 con 0x7f22a81599d0 2023-10-11T09:52:53.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.647+0000 7f22a4ff9700 1 -- 172.21.15.78:0/2786233436 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a000a7a0 con 0x7f22a81599d0 2023-10-11T09:52:53.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.647+0000 7f22ad300700 1 -- 172.21.15.78:0/2786233436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a80c4220 con 0x7f22a81599d0 2023-10-11T09:52:53.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.647+0000 7f22a4ff9700 1 -- 172.21.15.78:0/2786233436 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a000ab50 con 0x7f22a81599d0 2023-10-11T09:52:53.662 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.647+0000 7f22ad300700 1 -- 172.21.15.78:0/2786233436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a80c8ca0 con 0x7f22a81599d0 2023-10-11T09:52:53.663 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.651+0000 7f22ad300700 1 -- 172.21.15.78:0/2786233436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22a8067d00 con 0x7f22a81599d0 2023-10-11T09:52:53.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.651+0000 7f22a4ff9700 1 -- 172.21.15.78:0/2786233436 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f22a000acb0 con 0x7f22a81599d0 2023-10-11T09:52:53.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.651+0000 7f22a4ff9700 1 --2- 172.21.15.78:0/2786233436 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2294065ed0 0x7f2294068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:53.664 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.655+0000 7f22a4ff9700 1 -- 172.21.15.78:0/2786233436 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f22a0083120 con 0x7f22a81599d0 2023-10-11T09:52:53.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.655+0000 7f22a7fff700 1 --2- 172.21.15.78:0/2786233436 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2294065ed0 0x7f2294068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:53.665 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.655+0000 7f22a7fff700 1 --2- 172.21.15.78:0/2786233436 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2294065ed0 0x7f2294068380 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f2298000c00 tx=0x7f2298008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:53.667 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.659+0000 7f22a4ff9700 1 -- 172.21.15.78:0/2786233436 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f22a0050920 con 0x7f22a81599d0 2023-10-11T09:52:53.844 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.835+0000 7f22ad300700 1 -- 172.21.15.78:0/2786233436 --> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f22a80cdd00 con 0x7f2294065ed0 2023-10-11T09:52:53.845 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.839+0000 7f22a4ff9700 1 -- 172.21.15.78:0/2786233436 <== mgr.4101 v2:172.21.15.78:6832/35606 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2298011b80 con 0x7f2294065ed0 2023-10-11T09:52:53.847 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.839+0000 7f22ad300700 1 -- 172.21.15.78:0/2786233436 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2294065ed0 msgr2=0x7f2294068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:53.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.839+0000 7f22ad300700 1 --2- 172.21.15.78:0/2786233436 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2294065ed0 0x7f2294068380 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f2298000c00 tx=0x7f2298008030 comp rx=0 tx=0).stop 2023-10-11T09:52:53.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.839+0000 7f22ad300700 1 -- 172.21.15.78:0/2786233436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22a81599d0 msgr2=0x7f22a80c8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:53.848 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.839+0000 7f22ad300700 1 --2- 172.21.15.78:0/2786233436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22a81599d0 0x7f22a80c8760 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f22a000cc70 tx=0x7f22a000cf80 comp rx=0 tx=0).stop 2023-10-11T09:52:53.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.839+0000 7f22ad300700 1 -- 172.21.15.78:0/2786233436 shutdown_connections 2023-10-11T09:52:53.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.839+0000 7f22ad300700 1 --2- 172.21.15.78:0/2786233436 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f2294065ed0 0x7f2294068380 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:53.849 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.839+0000 7f22ad300700 1 --2- 172.21.15.78:0/2786233436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22a81599d0 0x7f22a80c8760 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:53.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.839+0000 7f22ad300700 1 --2- 172.21.15.78:0/2786233436 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22a80ce220 0x7f22a80c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:53.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.839+0000 7f22ad300700 1 --2- 172.21.15.78:0/2786233436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22a80ca080 0x7f22a80c3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:53.850 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.839+0000 7f22ad300700 1 -- 172.21.15.78:0/2786233436 >> 172.21.15.78:0/2786233436 conn(0x7f22a80cf820 msgr2=0x7f22a806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:53.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.839+0000 7f22ad300700 1 -- 172.21.15.78:0/2786233436 shutdown_connections 2023-10-11T09:52:53.851 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:53.839+0000 7f22ad300700 1 -- 172.21.15.78:0/2786233436 wait complete. 2023-10-11T09:52:53.851 INFO:teuthology.orchestra.run.smithi078.stderr:instructed osd(s) 4 to deep-scrub 2023-10-11T09:52:53.868 INFO:tasks.ceph:Scrubbing osd.5 2023-10-11T09:52:53.869 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-11T09:52:54.052 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.043+0000 7fd91ab58700 1 Processor -- start 2023-10-11T09:52:54.053 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.043+0000 7fd91ab58700 1 -- start start 2023-10-11T09:52:54.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.047+0000 7fd91ab58700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd9140b5230 0x7fd9141674f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:54.055 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.047+0000 7fd91ab58700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9140cf8b0 con 0x7fd9140d1580 2023-10-11T09:52:54.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.047+0000 7fd91ab58700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9140d2290 con 0x7fd91415c980 2023-10-11T09:52:54.056 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.047+0000 7fd91ab58700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9140d19f0 con 0x7fd9140b5230 2023-10-11T09:52:54.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.047+0000 7fd919355700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fd9140d1580 0x7fd9140b4b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:49810/0 (socket says 172.21.15.78:49810) 2023-10-11T09:52:54.057 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.047+0000 7fd919355700 1 -- 172.21.15.78:0/2265610404 learned_addr learned my addr 172.21.15.78:0/2265610404 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:54.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.047+0000 7fd91a357700 1 --2- 172.21.15.78:0/2265610404 >> v2:172.21.15.133:3300/0 conn(0x7fd9140b5230 0x7fd9141674f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:54.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.051+0000 7fd918b54700 1 -- 172.21.15.78:0/2265610404 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 952138230 0 0) 0x7fd9140cf8b0 con 0x7fd9140d1580 2023-10-11T09:52:54.058 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.051+0000 7fd918b54700 1 -- 172.21.15.78:0/2265610404 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9000035d0 con 0x7fd9140d1580 2023-10-11T09:52:54.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.051+0000 7fd91a357700 1 -- 172.21.15.78:0/2265610404 >> v1:172.21.15.133:6789/0 conn(0x7fd91415c980 legacy=0x7fd9140d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:54.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.051+0000 7fd918b54700 1 -- 172.21.15.78:0/2265610404 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1117923122 0 0) 0x7fd9140d2290 con 0x7fd91415c980 2023-10-11T09:52:54.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.051+0000 7fd91a357700 1 -- 172.21.15.78:0/2265610404 >> v1:172.21.15.78:6789/0 conn(0x7fd9140d1580 legacy=0x7fd9140b4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:54.059 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.051+0000 7fd91a357700 1 -- 172.21.15.78:0/2265610404 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9140d1cd0 con 0x7fd9140b5230 2023-10-11T09:52:54.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.051+0000 7fd91a357700 1 --2- 172.21.15.78:0/2265610404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9140b5230 0x7fd9141674f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd90c009d90 tx=0x7fd90c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77e05cd6765e5cb6 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:54.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.051+0000 7fd918b54700 1 -- 172.21.15.78:0/2265610404 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c00e070 con 0x7fd9140b5230 2023-10-11T09:52:54.061 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.051+0000 7fd918b54700 1 -- 172.21.15.78:0/2265610404 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90c003a40 con 0x7fd9140b5230 2023-10-11T09:52:54.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.051+0000 7fd918b54700 1 -- 172.21.15.78:0/2265610404 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd904001690 con 0x7fd9140b5230 2023-10-11T09:52:54.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.051+0000 7fd91ab58700 1 -- 172.21.15.78:0/2265610404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9140b5230 msgr2=0x7fd9141674f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:54.062 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.051+0000 7fd91ab58700 1 --2- 172.21.15.78:0/2265610404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9140b5230 0x7fd9141674f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd90c009d90 tx=0x7fd90c00b330 comp rx=0 tx=0).stop 2023-10-11T09:52:54.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.051+0000 7fd91ab58700 1 -- 172.21.15.78:0/2265610404 shutdown_connections 2023-10-11T09:52:54.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.051+0000 7fd91ab58700 1 --2- 172.21.15.78:0/2265610404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9140b5230 0x7fd9141674f0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:54.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.051+0000 7fd91ab58700 1 -- 172.21.15.78:0/2265610404 >> 172.21.15.78:0/2265610404 conn(0x7fd9140724b0 msgr2=0x7fd9140d30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:54.063 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.051+0000 7fd91ab58700 1 -- 172.21.15.78:0/2265610404 shutdown_connections 2023-10-11T09:52:54.064 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.051+0000 7fd91ab58700 1 -- 172.21.15.78:0/2265610404 wait complete. 2023-10-11T09:52:54.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.055+0000 7fd91ab58700 1 Processor -- start 2023-10-11T09:52:54.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.055+0000 7fd91ab58700 1 -- start start 2023-10-11T09:52:54.066 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.055+0000 7fd91ab58700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9140b5230 0x7fd91415d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:54.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.055+0000 7fd91ab58700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9140d1580 0x7fd914151740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:54.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.055+0000 7fd91ab58700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd91415c980 0x7fd9141f23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:54.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.055+0000 7fd91ab58700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91415dee0 con 0x7fd9140b5230 2023-10-11T09:52:54.067 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.055+0000 7fd91ab58700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91415e1e0 con 0x7fd9140d1580 2023-10-11T09:52:54.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.055+0000 7fd91ab58700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91415e4a0 con 0x7fd91415c980 2023-10-11T09:52:54.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.055+0000 7fd919b56700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9140b5230 0x7fd91415d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:54.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.055+0000 7fd919b56700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9140b5230 0x7fd91415d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48494/0 (socket says 172.21.15.78:48494) 2023-10-11T09:52:54.068 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.055+0000 7fd919b56700 1 -- 172.21.15.78:0/1560761427 learned_addr learned my addr 172.21.15.78:0/1560761427 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:54.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.055+0000 7fd919355700 1 --2- 172.21.15.78:0/1560761427 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9140d1580 0x7fd914151740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:54.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.055+0000 7fd91a357700 1 --2- 172.21.15.78:0/1560761427 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd91415c980 0x7fd9141f23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:54.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.055+0000 7fd919b56700 1 -- 172.21.15.78:0/1560761427 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd91415c980 msgr2=0x7fd9141f23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:54.069 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.055+0000 7fd919b56700 1 --2- 172.21.15.78:0/1560761427 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd91415c980 0x7fd9141f23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:54.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.055+0000 7fd919b56700 1 -- 172.21.15.78:0/1560761427 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9140d1580 msgr2=0x7fd914151740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:54.070 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.055+0000 7fd919b56700 1 --2- 172.21.15.78:0/1560761427 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9140d1580 0x7fd914151740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:54.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.055+0000 7fd919b56700 1 -- 172.21.15.78:0/1560761427 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd90c009a70 con 0x7fd9140b5230 2023-10-11T09:52:54.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.055+0000 7fd919b56700 1 --2- 172.21.15.78:0/1560761427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9140b5230 0x7fd91415d9a0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fd91000c450 tx=0x7fd91000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:54.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.059+0000 7fd90a7fc700 1 -- 172.21.15.78:0/1560761427 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9100054e0 con 0x7fd9140b5230 2023-10-11T09:52:54.071 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.059+0000 7fd90a7fc700 1 -- 172.21.15.78:0/1560761427 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9100036a0 con 0x7fd9140b5230 2023-10-11T09:52:54.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.059+0000 7fd91ab58700 1 -- 172.21.15.78:0/1560761427 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd91415e780 con 0x7fd9140b5230 2023-10-11T09:52:54.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.059+0000 7fd90a7fc700 1 -- 172.21.15.78:0/1560761427 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd910003a50 con 0x7fd9140b5230 2023-10-11T09:52:54.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.059+0000 7fd91ab58700 1 -- 172.21.15.78:0/1560761427 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd91415ed40 con 0x7fd9140b5230 2023-10-11T09:52:54.072 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.059+0000 7fd91ab58700 1 -- 172.21.15.78:0/1560761427 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd9140b5f90 con 0x7fd9140b5230 2023-10-11T09:52:54.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.059+0000 7fd90a7fc700 1 -- 172.21.15.78:0/1560761427 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fd910003bb0 con 0x7fd9140b5230 2023-10-11T09:52:54.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.063+0000 7fd90a7fc700 1 --2- 172.21.15.78:0/1560761427 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd8f8065ed0 0x7fd8f8068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:54.074 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.063+0000 7fd90a7fc700 1 -- 172.21.15.78:0/1560761427 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fd910082af0 con 0x7fd9140b5230 2023-10-11T09:52:54.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.063+0000 7fd90a7fc700 1 --2- 172.21.15.78:0/1560761427 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x7fd8f806c480 0x7fd8f806e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:54.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.063+0000 7fd90a7fc700 1 -- 172.21.15.78:0/1560761427 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd8f806ef40 con 0x7fd8f806c480 2023-10-11T09:52:54.075 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.063+0000 7fd90a7fc700 1 -- 172.21.15.78:0/1560761427 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=132) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9100872c0 con 0x7fd9140b5230 2023-10-11T09:52:54.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.063+0000 7fd919355700 1 --2- 172.21.15.78:0/1560761427 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd8f8065ed0 0x7fd8f8068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:54.076 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.063+0000 7fd91a357700 1 --2- 172.21.15.78:0/1560761427 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x7fd8f806c480 0x7fd8f806e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:54.077 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.063+0000 7fd919355700 1 --2- 172.21.15.78:0/1560761427 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd8f8065ed0 0x7fd8f8068380 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fd9040017c0 tx=0x7fd904001670 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:54.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.063+0000 7fd91a357700 1 --2- 172.21.15.78:0/1560761427 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x7fd8f806c480 0x7fd8f806e890 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:54.078 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.067+0000 7fd90a7fc700 1 -- 172.21.15.78:0/1560761427 <== osd.5 v2:172.21.15.133:6816/35899 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fd8f806ef40 con 0x7fd8f806c480 2023-10-11T09:52:54.091 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.083+0000 7fd91ab58700 1 -- 172.21.15.78:0/1560761427 --> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd9140d1fb0 con 0x7fd8f806c480 2023-10-11T09:52:54.092 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.083+0000 7fd90a7fc700 1 -- 172.21.15.78:0/1560761427 <== osd.5 v2:172.21.15.133:6816/35899 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fd90c003c00 con 0x7fd8f806c480 2023-10-11T09:52:54.092 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:52:54.093 INFO:teuthology.orchestra.run.smithi078.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-11T09:52:54.093 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:52:54.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.083+0000 7fd91ab58700 1 -- 172.21.15.78:0/1560761427 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x7fd8f806c480 msgr2=0x7fd8f806e890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:54.094 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.083+0000 7fd91ab58700 1 --2- 172.21.15.78:0/1560761427 >> [v2:172.21.15.133:6816/35899,v1:172.21.15.133:6817/35899] conn(0x7fd8f806c480 0x7fd8f806e890 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:54.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.087+0000 7fd91ab58700 1 -- 172.21.15.78:0/1560761427 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd8f8065ed0 msgr2=0x7fd8f8068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:54.095 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.087+0000 7fd91ab58700 1 --2- 172.21.15.78:0/1560761427 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fd8f8065ed0 0x7fd8f8068380 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fd9040017c0 tx=0x7fd904001670 comp rx=0 tx=0).stop 2023-10-11T09:52:54.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.087+0000 7fd91ab58700 1 -- 172.21.15.78:0/1560761427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9140b5230 msgr2=0x7fd91415d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:54.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.087+0000 7fd91ab58700 1 --2- 172.21.15.78:0/1560761427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9140b5230 0x7fd91415d9a0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fd91000c450 tx=0x7fd91000c810 comp rx=0 tx=0).stop 2023-10-11T09:52:54.096 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.087+0000 7fd91a357700 1 -- 172.21.15.78:0/1560761427 reap_dead start 2023-10-11T09:52:54.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.087+0000 7fd91ab58700 1 -- 172.21.15.78:0/1560761427 shutdown_connections 2023-10-11T09:52:54.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.087+0000 7fd91ab58700 1 -- 172.21.15.78:0/1560761427 >> 172.21.15.78:0/1560761427 conn(0x7fd9140724b0 msgr2=0x7fd91415b810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:54.097 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.087+0000 7fd91ab58700 1 -- 172.21.15.78:0/1560761427 shutdown_connections 2023-10-11T09:52:54.098 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.087+0000 7fd91ab58700 1 -- 172.21.15.78:0/1560761427 wait complete. 2023-10-11T09:52:54.112 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-11T09:52:54.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.275+0000 7f370b59e700 1 Processor -- start 2023-10-11T09:52:54.283 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.275+0000 7f370b59e700 1 -- start start 2023-10-11T09:52:54.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.279+0000 7f370b59e700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f370c0ce700 0x7f370c0c20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:54.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.279+0000 7f370b59e700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f370c0cb220 con 0x7f370c071720 2023-10-11T09:52:54.285 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.279+0000 7f370b59e700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f370c1613f0 con 0x7f370c161020 2023-10-11T09:52:54.286 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.279+0000 7f370b59e700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f370c071c70 con 0x7f370c0ce700 2023-10-11T09:52:54.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.279+0000 7f3701d9b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f370c0ce700 0x7f370c0c20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:54.287 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.279+0000 7f3701d9b700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f370c0ce700 0x7f370c0c20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48496/0 (socket says 172.21.15.78:48496) 2023-10-11T09:52:54.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.279+0000 7f3701d9b700 1 -- 172.21.15.78:0/2340319116 learned_addr learned my addr 172.21.15.78:0/2340319116 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:54.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.279+0000 7f3701d9b700 1 -- 172.21.15.78:0/2340319116 >> v1:172.21.15.133:6789/0 conn(0x7f370c161020 legacy=0x7f370c0cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:54.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.279+0000 7f3701d9b700 1 -- 172.21.15.78:0/2340319116 >> v1:172.21.15.78:6789/0 conn(0x7f370c071720 legacy=0x7f370c06bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:54.288 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.279+0000 7f3701d9b700 1 -- 172.21.15.78:0/2340319116 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f370c0ca650 con 0x7f370c0ce700 2023-10-11T09:52:54.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.279+0000 7f3709d9b700 1 -- 172.21.15.78:0/2340319116 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1888810911 0 0) 0x7f370c0cb220 con 0x7f370c071720 2023-10-11T09:52:54.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.279+0000 7f3701d9b700 1 --2- 172.21.15.78:0/2340319116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f370c0ce700 0x7f370c0c20c0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f36f4009910 tx=0x7f36f4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08763788f0cd0b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:54.289 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.279+0000 7f3709d9b700 1 -- 172.21.15.78:0/2340319116 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f4005e50 con 0x7f370c0ce700 2023-10-11T09:52:54.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.279+0000 7f3709d9b700 1 -- 172.21.15.78:0/2340319116 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f4009e60 con 0x7f370c0ce700 2023-10-11T09:52:54.290 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.279+0000 7f3709d9b700 1 -- 172.21.15.78:0/2340319116 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f400b870 con 0x7f370c0ce700 2023-10-11T09:52:54.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.279+0000 7f370b59e700 1 -- 172.21.15.78:0/2340319116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f370c0ce700 msgr2=0x7f370c0c20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:54.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.279+0000 7f370b59e700 1 --2- 172.21.15.78:0/2340319116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f370c0ce700 0x7f370c0c20c0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f36f4009910 tx=0x7f36f4009c20 comp rx=0 tx=0).stop 2023-10-11T09:52:54.292 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.279+0000 7f370b59e700 1 -- 172.21.15.78:0/2340319116 shutdown_connections 2023-10-11T09:52:54.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.279+0000 7f370b59e700 1 --2- 172.21.15.78:0/2340319116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f370c0ce700 0x7f370c0c20c0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:54.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.279+0000 7f370b59e700 1 -- 172.21.15.78:0/2340319116 >> 172.21.15.78:0/2340319116 conn(0x7f370c0cf820 msgr2=0x7f370c0cb610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:54.293 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.283+0000 7f370b59e700 1 -- 172.21.15.78:0/2340319116 shutdown_connections 2023-10-11T09:52:54.294 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.283+0000 7f370b59e700 1 -- 172.21.15.78:0/2340319116 wait complete. 2023-10-11T09:52:54.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.287+0000 7f370b59e700 1 Processor -- start 2023-10-11T09:52:54.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.287+0000 7f370b59e700 1 -- start start 2023-10-11T09:52:54.295 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.287+0000 7f370b59e700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f370c071720 0x7f370c161b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:54.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.287+0000 7f370b59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f370c0ce700 0x7f370c163e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:54.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.287+0000 7f370b59e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f370c161020 0x7f370c148d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:54.296 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.287+0000 7f370b59e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370c0c3680 con 0x7f370c0ce700 2023-10-11T09:52:54.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.287+0000 7f370b59e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370c0c3960 con 0x7f370c161020 2023-10-11T09:52:54.297 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.287+0000 7f370b59e700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f370c0c3c40 con 0x7f370c071720 2023-10-11T09:52:54.299 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.287+0000 7f3701d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f370c0ce700 0x7f370c163e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:54.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.287+0000 7f3701d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f370c0ce700 0x7f370c163e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48504/0 (socket says 172.21.15.78:48504) 2023-10-11T09:52:54.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.287+0000 7f3701d9b700 1 -- 172.21.15.78:0/783486514 learned_addr learned my addr 172.21.15.78:0/783486514 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:54.300 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.287+0000 7f370a59c700 1 --2- 172.21.15.78:0/783486514 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f370c071720 0x7f370c161b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:54.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.287+0000 7f370ad9d700 1 --2- 172.21.15.78:0/783486514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f370c161020 0x7f370c148d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:54.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.287+0000 7f3701d9b700 1 -- 172.21.15.78:0/783486514 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f370c071720 msgr2=0x7f370c161b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:54.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.287+0000 7f3701d9b700 1 --2- 172.21.15.78:0/783486514 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f370c071720 0x7f370c161b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:54.301 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.287+0000 7f3701d9b700 1 -- 172.21.15.78:0/783486514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f370c161020 msgr2=0x7f370c148d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:54.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.287+0000 7f3701d9b700 1 --2- 172.21.15.78:0/783486514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f370c161020 0x7f370c148d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:54.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.287+0000 7f3701d9b700 1 -- 172.21.15.78:0/783486514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f4009520 con 0x7f370c0ce700 2023-10-11T09:52:54.302 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.291+0000 7f3701d9b700 1 --2- 172.21.15.78:0/783486514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f370c0ce700 0x7f370c163e90 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f36f4000c00 tx=0x7f36f4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.291+0000 7f37037fe700 1 -- 172.21.15.78:0/783486514 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f400be40 con 0x7f370c0ce700 2023-10-11T09:52:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.291+0000 7f370b59e700 1 -- 172.21.15.78:0/783486514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f370c0c3f20 con 0x7f370c0ce700 2023-10-11T09:52:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.291+0000 7f37037fe700 1 -- 172.21.15.78:0/783486514 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f4020070 con 0x7f370c0ce700 2023-10-11T09:52:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.291+0000 7f37037fe700 1 -- 172.21.15.78:0/783486514 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f4006220 con 0x7f370c0ce700 2023-10-11T09:52:54.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.291+0000 7f370b59e700 1 -- 172.21.15.78:0/783486514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f370c1643d0 con 0x7f370c0ce700 2023-10-11T09:52:54.304 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.291+0000 7f370b59e700 1 -- 172.21.15.78:0/783486514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f370c067d00 con 0x7f370c0ce700 2023-10-11T09:52:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.295+0000 7f37037fe700 1 -- 172.21.15.78:0/783486514 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f36f4022030 con 0x7f370c0ce700 2023-10-11T09:52:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.295+0000 7f37037fe700 1 --2- 172.21.15.78:0/783486514 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f36ec065ed0 0x7f36ec068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.295+0000 7f37037fe700 1 -- 172.21.15.78:0/783486514 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f36f4084cd0 con 0x7f370c0ce700 2023-10-11T09:52:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.295+0000 7f370a59c700 1 --2- 172.21.15.78:0/783486514 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f36ec065ed0 0x7f36ec068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.295+0000 7f370a59c700 1 --2- 172.21.15.78:0/783486514 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f36ec065ed0 0x7f36ec068380 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f36fc000f60 tx=0x7f36fc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:54.311 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.303+0000 7f37037fe700 1 -- 172.21.15.78:0/783486514 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f36f40524e0 con 0x7f370c0ce700 2023-10-11T09:52:54.485 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.479+0000 7f370b59e700 1 -- 172.21.15.78:0/783486514 --> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f370c0c3ad0 con 0x7f36ec065ed0 2023-10-11T09:52:54.486 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.479+0000 7f37037fe700 1 -- 172.21.15.78:0/783486514 <== mgr.4101 v2:172.21.15.78:6832/35606 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f370c0c3ad0 con 0x7f36ec065ed0 2023-10-11T09:52:54.488 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.479+0000 7f370b59e700 1 -- 172.21.15.78:0/783486514 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f36ec065ed0 msgr2=0x7f36ec068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:54.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.479+0000 7f370b59e700 1 --2- 172.21.15.78:0/783486514 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f36ec065ed0 0x7f36ec068380 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f36fc000f60 tx=0x7f36fc009040 comp rx=0 tx=0).stop 2023-10-11T09:52:54.489 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.479+0000 7f370b59e700 1 -- 172.21.15.78:0/783486514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f370c0ce700 msgr2=0x7f370c163e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:54.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.479+0000 7f370b59e700 1 --2- 172.21.15.78:0/783486514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f370c0ce700 0x7f370c163e90 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f36f4000c00 tx=0x7f36f4004770 comp rx=0 tx=0).stop 2023-10-11T09:52:54.490 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.479+0000 7f370b59e700 1 -- 172.21.15.78:0/783486514 shutdown_connections 2023-10-11T09:52:54.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.479+0000 7f370b59e700 1 --2- 172.21.15.78:0/783486514 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f36ec065ed0 0x7f36ec068380 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:54.491 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.479+0000 7f370b59e700 1 --2- 172.21.15.78:0/783486514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f370c161020 0x7f370c148d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:54.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.479+0000 7f370b59e700 1 --2- 172.21.15.78:0/783486514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f370c0ce700 0x7f370c163e90 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:54.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.479+0000 7f370b59e700 1 --2- 172.21.15.78:0/783486514 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f370c071720 0x7f370c161b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:54.492 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.479+0000 7f370b59e700 1 -- 172.21.15.78:0/783486514 >> 172.21.15.78:0/783486514 conn(0x7f370c0cf820 msgr2=0x7f370c0cd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:54.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.479+0000 7f370b59e700 1 -- 172.21.15.78:0/783486514 shutdown_connections 2023-10-11T09:52:54.493 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.479+0000 7f370b59e700 1 -- 172.21.15.78:0/783486514 wait complete. 2023-10-11T09:52:54.495 INFO:teuthology.orchestra.run.smithi078.stderr:instructed osd(s) 5 to deep-scrub 2023-10-11T09:52:54.508 INFO:tasks.ceph:Scrubbing osd.6 2023-10-11T09:52:54.509 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-11T09:52:54.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.679+0000 7f395aee0700 1 Processor -- start 2023-10-11T09:52:54.687 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.679+0000 7f395aee0700 1 -- start start 2023-10-11T09:52:54.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.683+0000 7f395aee0700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f395415a7f0 0x7f39540d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:54.689 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.683+0000 7f395aee0700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f395415dd10 0x7f39540d3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:54.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.683+0000 7f395aee0700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f3954161260 0x7f39540d3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:54.690 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.683+0000 7f395aee0700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f395415d040 con 0x7f3954161260 2023-10-11T09:52:54.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.683+0000 7f395aee0700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f395415d880 con 0x7f395415a7f0 2023-10-11T09:52:54.691 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.683+0000 7f395aee0700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39540d1770 con 0x7f395415dd10 2023-10-11T09:52:54.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.683+0000 7f395947d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f3954161260 0x7f39540d3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:54.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.683+0000 7f3958c7c700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f395415a7f0 0x7f39540d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:54.693 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.683+0000 7f395947d700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f3954161260 0x7f39540d3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45386/0 (socket says 172.21.15.78:45386) 2023-10-11T09:52:54.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.683+0000 7f3953fff700 1 --2- 172.21.15.78:0/3875459228 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f395415dd10 0x7f39540d3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:54.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.683+0000 7f395947d700 1 -- 172.21.15.78:0/3875459228 learned_addr learned my addr 172.21.15.78:0/3875459228 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:54.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.683+0000 7f3958c7c700 1 -- 172.21.15.78:0/3875459228 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f395415dd10 msgr2=0x7f39540d3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:54.694 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.683+0000 7f3958c7c700 1 --2- 172.21.15.78:0/3875459228 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f395415dd10 0x7f39540d3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:54.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.683+0000 7f3958c7c700 1 -- 172.21.15.78:0/3875459228 >> v2:172.21.15.78:3300/0 conn(0x7f3954161260 msgr2=0x7f39540d3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:54.695 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.683+0000 7f3958c7c700 1 --2- 172.21.15.78:0/3875459228 >> v2:172.21.15.78:3300/0 conn(0x7f3954161260 0x7f39540d3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:54.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.683+0000 7f3958c7c700 1 -- 172.21.15.78:0/3875459228 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f395415d230 con 0x7f395415a7f0 2023-10-11T09:52:54.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.683+0000 7f395947d700 1 --2- 172.21.15.78:0/3875459228 >> v2:172.21.15.78:3300/0 conn(0x7f3954161260 0x7f39540d3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:52:54.696 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.683+0000 7f3958c7c700 1 --2- 172.21.15.78:0/3875459228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f395415a7f0 0x7f39540d0eb0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f39540d0e60 tx=0x7f3948009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f17ad9edab0b50fc server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:54.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.683+0000 7f39537fe700 1 -- 172.21.15.78:0/3875459228 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3948005e50 con 0x7f395415a7f0 2023-10-11T09:52:54.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.683+0000 7f39537fe700 1 -- 172.21.15.78:0/3875459228 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3948006030 con 0x7f395415a7f0 2023-10-11T09:52:54.697 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.683+0000 7f39537fe700 1 -- 172.21.15.78:0/3875459228 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39480063c0 con 0x7f395415a7f0 2023-10-11T09:52:54.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.683+0000 7f395aee0700 1 -- 172.21.15.78:0/3875459228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f395415a7f0 msgr2=0x7f39540d0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:54.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.683+0000 7f395aee0700 1 --2- 172.21.15.78:0/3875459228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f395415a7f0 0x7f39540d0eb0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f39540d0e60 tx=0x7f3948009b20 comp rx=0 tx=0).stop 2023-10-11T09:52:54.698 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.687+0000 7f395aee0700 1 -- 172.21.15.78:0/3875459228 shutdown_connections 2023-10-11T09:52:54.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.687+0000 7f395aee0700 1 --2- 172.21.15.78:0/3875459228 >> v2:172.21.15.78:3300/0 conn(0x7f3954161260 0x7f39540d3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:54.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.687+0000 7f395aee0700 1 --2- 172.21.15.78:0/3875459228 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f395415dd10 0x7f39540d3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:54.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.687+0000 7f395aee0700 1 --2- 172.21.15.78:0/3875459228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f395415a7f0 0x7f39540d0eb0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:54.699 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.687+0000 7f395aee0700 1 -- 172.21.15.78:0/3875459228 >> 172.21.15.78:0/3875459228 conn(0x7f395408cc70 msgr2=0x7f395409c590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:54.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.687+0000 7f395aee0700 1 -- 172.21.15.78:0/3875459228 shutdown_connections 2023-10-11T09:52:54.700 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.687+0000 7f395aee0700 1 -- 172.21.15.78:0/3875459228 wait complete. 2023-10-11T09:52:54.702 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.687+0000 7f395aee0700 1 Processor -- start 2023-10-11T09:52:54.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.691+0000 7f395aee0700 1 -- start start 2023-10-11T09:52:54.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.691+0000 7f395aee0700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f395415a7f0 0x7f3954160880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:54.703 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.691+0000 7f395aee0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f395415dd10 0x7f39540cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:54.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.691+0000 7f395aee0700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3954161260 0x7f3954143da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:54.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.691+0000 7f395aee0700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395415e950 con 0x7f395415dd10 2023-10-11T09:52:54.704 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.691+0000 7f395aee0700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3954158120 con 0x7f395415a7f0 2023-10-11T09:52:54.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.691+0000 7f395aee0700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f395415ce90 con 0x7f3954161260 2023-10-11T09:52:54.705 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.691+0000 7f3953fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f395415dd10 0x7f39540cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:54.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.691+0000 7f3953fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f395415dd10 0x7f39540cebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45400/0 (socket says 172.21.15.78:45400) 2023-10-11T09:52:54.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.691+0000 7f3953fff700 1 -- 172.21.15.78:0/676449186 learned_addr learned my addr 172.21.15.78:0/676449186 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:54.706 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.691+0000 7f3958c7c700 1 --2- 172.21.15.78:0/676449186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f395415a7f0 0x7f3954160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:54.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.691+0000 7f395947d700 1 --2- 172.21.15.78:0/676449186 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3954161260 0x7f3954143da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:54.707 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.691+0000 7f3953fff700 1 -- 172.21.15.78:0/676449186 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3954161260 msgr2=0x7f3954143da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:54.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.691+0000 7f3953fff700 1 --2- 172.21.15.78:0/676449186 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3954161260 0x7f3954143da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:54.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.691+0000 7f3953fff700 1 -- 172.21.15.78:0/676449186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f395415a7f0 msgr2=0x7f3954160880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:54.708 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.691+0000 7f3953fff700 1 --2- 172.21.15.78:0/676449186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f395415a7f0 0x7f3954160880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:54.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.691+0000 7f3953fff700 1 -- 172.21.15.78:0/676449186 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f394c009b30 con 0x7f395415dd10 2023-10-11T09:52:54.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.691+0000 7f3953fff700 1 --2- 172.21.15.78:0/676449186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f395415dd10 0x7f39540cebd0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f394000e970 tx=0x7f394000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:54.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.695+0000 7f39517fa700 1 -- 172.21.15.78:0/676449186 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3940013680 con 0x7f395415dd10 2023-10-11T09:52:54.709 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.695+0000 7f39517fa700 1 -- 172.21.15.78:0/676449186 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39400137e0 con 0x7f395415dd10 2023-10-11T09:52:54.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.695+0000 7f395aee0700 1 -- 172.21.15.78:0/676449186 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3948009450 con 0x7f395415dd10 2023-10-11T09:52:54.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.695+0000 7f39517fa700 1 -- 172.21.15.78:0/676449186 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3940013b70 con 0x7f395415dd10 2023-10-11T09:52:54.710 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.695+0000 7f395aee0700 1 -- 172.21.15.78:0/676449186 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f395414e800 con 0x7f395415dd10 2023-10-11T09:52:54.711 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.695+0000 7f395aee0700 1 -- 172.21.15.78:0/676449186 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f39540d13f0 con 0x7f395415dd10 2023-10-11T09:52:54.712 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.695+0000 7f39517fa700 1 -- 172.21.15.78:0/676449186 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f3940013cd0 con 0x7f395415dd10 2023-10-11T09:52:54.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.699+0000 7f39517fa700 1 --2- 172.21.15.78:0/676449186 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3938065f00 0x7f39380683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:54.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.699+0000 7f39517fa700 1 -- 172.21.15.78:0/676449186 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f39400834e0 con 0x7f395415dd10 2023-10-11T09:52:54.713 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.699+0000 7f3958c7c700 1 --2- 172.21.15.78:0/676449186 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3938065f00 0x7f39380683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:54.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.699+0000 7f39517fa700 1 --2- 172.21.15.78:0/676449186 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x7f393806c4b0 0x7f393806e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:54.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.699+0000 7f39517fa700 1 -- 172.21.15.78:0/676449186 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f393806ef70 con 0x7f393806c4b0 2023-10-11T09:52:54.714 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.699+0000 7f39517fa700 1 -- 172.21.15.78:0/676449186 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=132) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3940085ac0 con 0x7f395415dd10 2023-10-11T09:52:54.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.699+0000 7f3958c7c700 1 --2- 172.21.15.78:0/676449186 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3938065f00 0x7f39380683b0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3948000c00 tx=0x7f3948008070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:54.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.699+0000 7f395947d700 1 --2- 172.21.15.78:0/676449186 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x7f393806c4b0 0x7f393806e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:54.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.699+0000 7f395947d700 1 --2- 172.21.15.78:0/676449186 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x7f393806c4b0 0x7f393806e8c0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:54.715 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.703+0000 7f39517fa700 1 -- 172.21.15.78:0/676449186 <== osd.6 v2:172.21.15.133:6800/35900 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f393806ef70 con 0x7f393806c4b0 2023-10-11T09:52:54.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.719+0000 7f395aee0700 1 -- 172.21.15.78:0/676449186 --> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f395415d510 con 0x7f393806c4b0 2023-10-11T09:52:54.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.723+0000 7f39517fa700 1 -- 172.21.15.78:0/676449186 <== osd.6 v2:172.21.15.133:6800/35900 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f394c00f040 con 0x7f393806c4b0 2023-10-11T09:52:54.729 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:52:54.730 INFO:teuthology.orchestra.run.smithi078.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-11T09:52:54.730 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:52:54.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.723+0000 7f395aee0700 1 -- 172.21.15.78:0/676449186 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x7f393806c4b0 msgr2=0x7f393806e8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:54.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.723+0000 7f395aee0700 1 --2- 172.21.15.78:0/676449186 >> [v2:172.21.15.133:6800/35900,v1:172.21.15.133:6801/35900] conn(0x7f393806c4b0 0x7f393806e8c0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:54.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.723+0000 7f395aee0700 1 -- 172.21.15.78:0/676449186 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3938065f00 msgr2=0x7f39380683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:54.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.723+0000 7f395aee0700 1 --2- 172.21.15.78:0/676449186 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3938065f00 0x7f39380683b0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3948000c00 tx=0x7f3948008070 comp rx=0 tx=0).stop 2023-10-11T09:52:54.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.723+0000 7f395aee0700 1 -- 172.21.15.78:0/676449186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f395415dd10 msgr2=0x7f39540cebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:54.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.723+0000 7f395aee0700 1 --2- 172.21.15.78:0/676449186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f395415dd10 0x7f39540cebd0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f394000e970 tx=0x7f394000ec80 comp rx=0 tx=0).stop 2023-10-11T09:52:54.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.723+0000 7f395947d700 1 -- 172.21.15.78:0/676449186 reap_dead start 2023-10-11T09:52:54.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.723+0000 7f395aee0700 1 -- 172.21.15.78:0/676449186 shutdown_connections 2023-10-11T09:52:54.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.723+0000 7f395aee0700 1 -- 172.21.15.78:0/676449186 >> 172.21.15.78:0/676449186 conn(0x7f395408cc70 msgr2=0x7f39540b3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:54.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.723+0000 7f395aee0700 1 -- 172.21.15.78:0/676449186 shutdown_connections 2023-10-11T09:52:54.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.723+0000 7f395aee0700 1 -- 172.21.15.78:0/676449186 wait complete. 2023-10-11T09:52:54.753 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-11T09:52:54.926 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.919+0000 7f328b399700 1 Processor -- start 2023-10-11T09:52:54.927 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.919+0000 7f328b399700 1 -- start start 2023-10-11T09:52:54.929 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.923+0000 7f328b399700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f3284160de0 0x7f32840ca670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:54.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.923+0000 7f328b399700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32840c2580 0x7f3284164ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:54.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.923+0000 7f328b399700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3284162130 con 0x7f32840cca70 2023-10-11T09:52:54.930 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.923+0000 7f328b399700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3284161560 con 0x7f3284160de0 2023-10-11T09:52:54.931 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.923+0000 7f328b399700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3284161800 con 0x7f32840c2580 2023-10-11T09:52:54.932 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.923+0000 7f3289135700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f3284160de0 0x7f32840ca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:54.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.923+0000 7f3288934700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f32840cca70 0x7f32840c1e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:60320/0 (socket says 172.21.15.78:60320) 2023-10-11T09:52:54.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.923+0000 7f3288934700 1 -- 172.21.15.78:0/3928021578 learned_addr learned my addr 172.21.15.78:0/3928021578 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:54.933 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.923+0000 7f3289936700 1 --2- 172.21.15.78:0/3928021578 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32840c2580 0x7f3284164ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:54.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.923+0000 7f3289135700 1 -- 172.21.15.78:0/3928021578 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32840c2580 msgr2=0x7f3284164ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:54.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.923+0000 7f3289135700 1 --2- 172.21.15.78:0/3928021578 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32840c2580 0x7f3284164ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:54.934 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.923+0000 7f3289135700 1 -- 172.21.15.78:0/3928021578 >> v1:172.21.15.78:6789/0 conn(0x7f32840cca70 legacy=0x7f32840c1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:54.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.923+0000 7f327bfff700 1 -- 172.21.15.78:0/3928021578 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1966505635 0 0) 0x7f3284162130 con 0x7f32840cca70 2023-10-11T09:52:54.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.923+0000 7f3289135700 1 -- 172.21.15.78:0/3928021578 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3284161ae0 con 0x7f3284160de0 2023-10-11T09:52:54.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.923+0000 7f3289936700 1 --2- 172.21.15.78:0/3928021578 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32840c2580 0x7f3284164ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:54.935 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.923+0000 7f3289135700 1 --2- 172.21.15.78:0/3928021578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3284160de0 0x7f32840ca670 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f3280009850 tx=0x7f3280009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4adac04993ccd52 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:54.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.923+0000 7f327bfff700 1 -- 172.21.15.78:0/3928021578 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3280011830 con 0x7f3284160de0 2023-10-11T09:52:54.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.923+0000 7f327bfff700 1 -- 172.21.15.78:0/3928021578 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3280011990 con 0x7f3284160de0 2023-10-11T09:52:54.936 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.923+0000 7f327bfff700 1 -- 172.21.15.78:0/3928021578 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3280011d20 con 0x7f3284160de0 2023-10-11T09:52:54.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.927+0000 7f328b399700 1 -- 172.21.15.78:0/3928021578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3284160de0 msgr2=0x7f32840ca670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:54.937 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.927+0000 7f328b399700 1 --2- 172.21.15.78:0/3928021578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3284160de0 0x7f32840ca670 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f3280009850 tx=0x7f3280009b60 comp rx=0 tx=0).stop 2023-10-11T09:52:54.938 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.927+0000 7f328b399700 1 -- 172.21.15.78:0/3928021578 shutdown_connections 2023-10-11T09:52:54.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.927+0000 7f328b399700 1 --2- 172.21.15.78:0/3928021578 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32840c2580 0x7f3284164ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:54.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.927+0000 7f328b399700 1 --2- 172.21.15.78:0/3928021578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3284160de0 0x7f32840ca670 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:54.939 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.927+0000 7f328b399700 1 -- 172.21.15.78:0/3928021578 >> 172.21.15.78:0/3928021578 conn(0x7f32840cb920 msgr2=0x7f32840cbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:54.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.927+0000 7f328b399700 1 -- 172.21.15.78:0/3928021578 shutdown_connections 2023-10-11T09:52:54.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.927+0000 7f328b399700 1 -- 172.21.15.78:0/3928021578 wait complete. 2023-10-11T09:52:54.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.931+0000 7f328b399700 1 Processor -- start 2023-10-11T09:52:54.940 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.931+0000 7f328b399700 1 -- start start 2023-10-11T09:52:54.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.931+0000 7f328b399700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32840c2580 0x7f32840cee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:54.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.931+0000 7f328b399700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32840cca70 0x7f32840cf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:54.943 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.931+0000 7f328b399700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3284160de0 0x7f328406cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:54.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.931+0000 7f328b399700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32840c6700 con 0x7f32840cca70 2023-10-11T09:52:54.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.931+0000 7f328b399700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32840c69a0 con 0x7f3284160de0 2023-10-11T09:52:54.944 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.931+0000 7f328b399700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32840c6c80 con 0x7f32840c2580 2023-10-11T09:52:54.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.931+0000 7f3288934700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32840cca70 0x7f32840cf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:54.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.931+0000 7f3288934700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32840cca70 0x7f32840cf3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45412/0 (socket says 172.21.15.78:45412) 2023-10-11T09:52:54.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.931+0000 7f3288934700 1 -- 172.21.15.78:0/2027326969 learned_addr learned my addr 172.21.15.78:0/2027326969 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:54.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.931+0000 7f3289135700 1 --2- 172.21.15.78:0/2027326969 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32840c2580 0x7f32840cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:54.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.931+0000 7f3289936700 1 --2- 172.21.15.78:0/2027326969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3284160de0 0x7f328406cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:54.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.931+0000 7f3288934700 1 -- 172.21.15.78:0/2027326969 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32840c2580 msgr2=0x7f32840cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:54.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.931+0000 7f3288934700 1 --2- 172.21.15.78:0/2027326969 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32840c2580 0x7f32840cee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:54.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.931+0000 7f3288934700 1 -- 172.21.15.78:0/2027326969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3284160de0 msgr2=0x7f328406cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:54.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.931+0000 7f3288934700 1 --2- 172.21.15.78:0/2027326969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3284160de0 0x7f328406cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:54.947 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.931+0000 7f3288934700 1 -- 172.21.15.78:0/2027326969 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32800094b0 con 0x7f32840cca70 2023-10-11T09:52:54.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.935+0000 7f3288934700 1 --2- 172.21.15.78:0/2027326969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32840cca70 0x7f32840cf3a0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f327400c580 tx=0x7f327400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:54.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.935+0000 7f3279ffb700 1 -- 172.21.15.78:0/2027326969 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3274004ca0 con 0x7f32840cca70 2023-10-11T09:52:54.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.935+0000 7f3279ffb700 1 -- 172.21.15.78:0/2027326969 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3274004e20 con 0x7f32840cca70 2023-10-11T09:52:54.948 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.935+0000 7f328b399700 1 -- 172.21.15.78:0/2027326969 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3284161330 con 0x7f32840cca70 2023-10-11T09:52:54.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.935+0000 7f3279ffb700 1 -- 172.21.15.78:0/2027326969 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3274007310 con 0x7f32840cca70 2023-10-11T09:52:54.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.935+0000 7f328b399700 1 -- 172.21.15.78:0/2027326969 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f328406d960 con 0x7f32840cca70 2023-10-11T09:52:54.950 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.935+0000 7f328b399700 1 -- 172.21.15.78:0/2027326969 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32840cb720 con 0x7f32840cca70 2023-10-11T09:52:54.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.939+0000 7f3279ffb700 1 -- 172.21.15.78:0/2027326969 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f3274007470 con 0x7f32840cca70 2023-10-11T09:52:54.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.939+0000 7f3279ffb700 1 --2- 172.21.15.78:0/2027326969 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3268065ed0 0x7f3268068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:54.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.939+0000 7f3279ffb700 1 -- 172.21.15.78:0/2027326969 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f32740832f0 con 0x7f32840cca70 2023-10-11T09:52:54.951 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.939+0000 7f3289135700 1 --2- 172.21.15.78:0/2027326969 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3268065ed0 0x7f3268068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:54.952 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.943+0000 7f3289135700 1 --2- 172.21.15.78:0/2027326969 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3268065ed0 0x7f3268068380 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f3280014040 tx=0x7f3280008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:54.953 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:54.947+0000 7f3279ffb700 1 -- 172.21.15.78:0/2027326969 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f3274050b80 con 0x7f32840cca70 2023-10-11T09:52:55.138 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.131+0000 7f328b399700 1 -- 172.21.15.78:0/2027326969 --> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f32841611b0 con 0x7f3268065ed0 2023-10-11T09:52:55.139 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.131+0000 7f3279ffb700 1 -- 172.21.15.78:0/2027326969 <== mgr.4101 v2:172.21.15.78:6832/35606 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3280011cb0 con 0x7f3268065ed0 2023-10-11T09:52:55.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.135+0000 7f328b399700 1 -- 172.21.15.78:0/2027326969 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3268065ed0 msgr2=0x7f3268068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:55.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.135+0000 7f328b399700 1 --2- 172.21.15.78:0/2027326969 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3268065ed0 0x7f3268068380 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f3280014040 tx=0x7f3280008030 comp rx=0 tx=0).stop 2023-10-11T09:52:55.142 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.135+0000 7f328b399700 1 -- 172.21.15.78:0/2027326969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32840cca70 msgr2=0x7f32840cf3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:55.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.135+0000 7f328b399700 1 --2- 172.21.15.78:0/2027326969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32840cca70 0x7f32840cf3a0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f327400c580 tx=0x7f327400c890 comp rx=0 tx=0).stop 2023-10-11T09:52:55.143 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.135+0000 7f328b399700 1 -- 172.21.15.78:0/2027326969 shutdown_connections 2023-10-11T09:52:55.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.135+0000 7f328b399700 1 --2- 172.21.15.78:0/2027326969 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f3268065ed0 0x7f3268068380 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:55.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.135+0000 7f328b399700 1 --2- 172.21.15.78:0/2027326969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3284160de0 0x7f328406cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:55.144 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.135+0000 7f328b399700 1 --2- 172.21.15.78:0/2027326969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32840cca70 0x7f32840cf3a0 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:55.145 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.135+0000 7f328b399700 1 --2- 172.21.15.78:0/2027326969 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32840c2580 0x7f32840cee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:55.145 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.135+0000 7f328b399700 1 -- 172.21.15.78:0/2027326969 >> 172.21.15.78:0/2027326969 conn(0x7f32840cb920 msgr2=0x7f3284071770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:55.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.135+0000 7f328b399700 1 -- 172.21.15.78:0/2027326969 shutdown_connections 2023-10-11T09:52:55.146 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.135+0000 7f328b399700 1 -- 172.21.15.78:0/2027326969 wait complete. 2023-10-11T09:52:55.147 INFO:teuthology.orchestra.run.smithi078.stderr:instructed osd(s) 6 to deep-scrub 2023-10-11T09:52:55.163 INFO:tasks.ceph:Scrubbing osd.7 2023-10-11T09:52:55.164 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-11T09:52:55.345 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.339+0000 7fb864ccf700 1 Processor -- start 2023-10-11T09:52:55.346 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.339+0000 7fb864ccf700 1 -- start start 2023-10-11T09:52:55.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.339+0000 7fb864ccf700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb8600cacf0 0x7fb8600cb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:55.348 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.339+0000 7fb864ccf700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb86006c790 con 0x7fb860071d20 2023-10-11T09:52:55.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.339+0000 7fb864ccf700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb86015ae30 con 0x7fb86015aa60 2023-10-11T09:52:55.349 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.339+0000 7fb864ccf700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8600cbfa0 con 0x7fb8600cacf0 2023-10-11T09:52:55.351 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.343+0000 7fb85dd9b700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7fb860071d20 0x7fb86006cc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:60326/0 (socket says 172.21.15.78:60326) 2023-10-11T09:52:55.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.343+0000 7fb85dd9b700 1 -- 172.21.15.78:0/2069793659 learned_addr learned my addr 172.21.15.78:0/2069793659 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:55.352 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.343+0000 7fb85ed9d700 1 --2- 172.21.15.78:0/2069793659 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb8600cacf0 0x7fb8600cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:55.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.343+0000 7fb85d59a700 1 -- 172.21.15.78:0/2069793659 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1388361051 0 0) 0x7fb86006c790 con 0x7fb860071d20 2023-10-11T09:52:55.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.343+0000 7fb85d59a700 1 -- 172.21.15.78:0/2069793659 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8480035d0 con 0x7fb860071d20 2023-10-11T09:52:55.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.343+0000 7fb85d59a700 1 -- 172.21.15.78:0/2069793659 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2235262394 0 0) 0x7fb86015ae30 con 0x7fb86015aa60 2023-10-11T09:52:55.353 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.343+0000 7fb85d59a700 1 -- 172.21.15.78:0/2069793659 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb86006c790 con 0x7fb86015aa60 2023-10-11T09:52:55.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.343+0000 7fb85ed9d700 1 -- 172.21.15.78:0/2069793659 >> v1:172.21.15.133:6789/0 conn(0x7fb86015aa60 legacy=0x7fb8600cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:55.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.343+0000 7fb85ed9d700 1 -- 172.21.15.78:0/2069793659 >> v1:172.21.15.78:6789/0 conn(0x7fb860071d20 legacy=0x7fb86006cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:55.354 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.343+0000 7fb85d59a700 1 -- 172.21.15.78:0/2069793659 <== mon.0 v1:172.21.15.78:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2842533525 0 0) 0x7fb8480035d0 con 0x7fb860071d20 2023-10-11T09:52:55.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.343+0000 7fb85ed9d700 1 -- 172.21.15.78:0/2069793659 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86006be60 con 0x7fb8600cacf0 2023-10-11T09:52:55.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.343+0000 7fb85ed9d700 1 --2- 172.21.15.78:0/2069793659 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb8600cacf0 0x7fb8600cb170 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb858009dc0 tx=0x7fb85800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d05b49e4f15bded server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:55.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.343+0000 7fb85d59a700 1 -- 172.21.15.78:0/2069793659 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85800e070 con 0x7fb8600cacf0 2023-10-11T09:52:55.355 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.343+0000 7fb85d59a700 1 -- 172.21.15.78:0/2069793659 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb858003a40 con 0x7fb8600cacf0 2023-10-11T09:52:55.356 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.343+0000 7fb85d59a700 1 -- 172.21.15.78:0/2069793659 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb854001670 con 0x7fb8600cacf0 2023-10-11T09:52:55.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.347+0000 7fb864ccf700 1 -- 172.21.15.78:0/2069793659 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb8600cacf0 msgr2=0x7fb8600cb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:55.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.347+0000 7fb864ccf700 1 --2- 172.21.15.78:0/2069793659 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb8600cacf0 0x7fb8600cb170 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb858009dc0 tx=0x7fb85800b330 comp rx=0 tx=0).stop 2023-10-11T09:52:55.357 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.347+0000 7fb864ccf700 1 -- 172.21.15.78:0/2069793659 shutdown_connections 2023-10-11T09:52:55.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.347+0000 7fb864ccf700 1 --2- 172.21.15.78:0/2069793659 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb8600cacf0 0x7fb8600cb170 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:55.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.347+0000 7fb864ccf700 1 -- 172.21.15.78:0/2069793659 >> 172.21.15.78:0/2069793659 conn(0x7fb86006ae00 msgr2=0x7fb8600b0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:55.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.347+0000 7fb864ccf700 1 -- 172.21.15.78:0/2069793659 shutdown_connections 2023-10-11T09:52:55.358 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.347+0000 7fb864ccf700 1 -- 172.21.15.78:0/2069793659 wait complete. 2023-10-11T09:52:55.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.347+0000 7fb864ccf700 1 Processor -- start 2023-10-11T09:52:55.359 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.347+0000 7fb864ccf700 1 -- start start 2023-10-11T09:52:55.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.351+0000 7fb864ccf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb860071d20 0x7fb860158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:55.361 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.351+0000 7fb864ccf700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb8600cacf0 0x7fb8600b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:55.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.351+0000 7fb864ccf700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb86015aa60 0x7fb86014b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:55.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.351+0000 7fb864ccf700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8600b4c60 con 0x7fb860071d20 2023-10-11T09:52:55.362 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.351+0000 7fb864ccf700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8600b22b0 con 0x7fb86015aa60 2023-10-11T09:52:55.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.351+0000 7fb864ccf700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb860158b60 con 0x7fb8600cacf0 2023-10-11T09:52:55.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.351+0000 7fb85e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb860071d20 0x7fb860158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:55.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.351+0000 7fb85e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb860071d20 0x7fb860158580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45428/0 (socket says 172.21.15.78:45428) 2023-10-11T09:52:55.363 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.351+0000 7fb85e59c700 1 -- 172.21.15.78:0/2405941281 learned_addr learned my addr 172.21.15.78:0/2405941281 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:55.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.351+0000 7fb85ed9d700 1 --2- 172.21.15.78:0/2405941281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb86015aa60 0x7fb86014b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:55.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.351+0000 7fb85dd9b700 1 --2- 172.21.15.78:0/2405941281 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb8600cacf0 0x7fb8600b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:55.364 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.351+0000 7fb85e59c700 1 -- 172.21.15.78:0/2405941281 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb8600cacf0 msgr2=0x7fb8600b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:55.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.351+0000 7fb85e59c700 1 --2- 172.21.15.78:0/2405941281 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb8600cacf0 0x7fb8600b1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:55.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.351+0000 7fb85e59c700 1 -- 172.21.15.78:0/2405941281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb86015aa60 msgr2=0x7fb86014b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:55.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.351+0000 7fb85e59c700 1 --2- 172.21.15.78:0/2405941281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb86015aa60 0x7fb86014b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:55.365 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.351+0000 7fb85e59c700 1 -- 172.21.15.78:0/2405941281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb858009a70 con 0x7fb860071d20 2023-10-11T09:52:55.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.351+0000 7fb85e59c700 1 --2- 172.21.15.78:0/2405941281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb860071d20 0x7fb860158580 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fb85000c490 tx=0x7fb85000c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:55.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.351+0000 7fb84effd700 1 -- 172.21.15.78:0/2405941281 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8500052d0 con 0x7fb860071d20 2023-10-11T09:52:55.366 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.351+0000 7fb84effd700 1 -- 172.21.15.78:0/2405941281 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb850005450 con 0x7fb860071d20 2023-10-11T09:52:55.367 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.351+0000 7fb864ccf700 1 -- 172.21.15.78:0/2405941281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb860158e40 con 0x7fb860071d20 2023-10-11T09:52:55.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.351+0000 7fb84effd700 1 -- 172.21.15.78:0/2405941281 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8500038e0 con 0x7fb860071d20 2023-10-11T09:52:55.369 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.351+0000 7fb864ccf700 1 -- 172.21.15.78:0/2405941281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb860159400 con 0x7fb860071d20 2023-10-11T09:52:55.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.355+0000 7fb864ccf700 1 -- 172.21.15.78:0/2405941281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb86006c420 con 0x7fb860071d20 2023-10-11T09:52:55.371 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.355+0000 7fb84effd700 1 -- 172.21.15.78:0/2405941281 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fb850003a40 con 0x7fb860071d20 2023-10-11T09:52:55.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.359+0000 7fb84effd700 1 --2- 172.21.15.78:0/2405941281 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb840065ed0 0x7fb840068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:55.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.359+0000 7fb84effd700 1 -- 172.21.15.78:0/2405941281 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fb850082910 con 0x7fb860071d20 2023-10-11T09:52:55.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.359+0000 7fb84effd700 1 --2- 172.21.15.78:0/2405941281 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x7fb84006c480 0x7fb84006e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:55.372 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.359+0000 7fb84effd700 1 -- 172.21.15.78:0/2405941281 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb84006ef40 con 0x7fb84006c480 2023-10-11T09:52:55.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.359+0000 7fb84effd700 1 -- 172.21.15.78:0/2405941281 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=132) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8500872a0 con 0x7fb860071d20 2023-10-11T09:52:55.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.359+0000 7fb85dd9b700 1 --2- 172.21.15.78:0/2405941281 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb840065ed0 0x7fb840068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:55.373 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.359+0000 7fb85ed9d700 1 --2- 172.21.15.78:0/2405941281 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x7fb84006c480 0x7fb84006e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:55.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.359+0000 7fb85dd9b700 1 --2- 172.21.15.78:0/2405941281 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb840065ed0 0x7fb840068380 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb8600c5710 tx=0x7fb854001770 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:55.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.359+0000 7fb85ed9d700 1 --2- 172.21.15.78:0/2405941281 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x7fb84006c480 0x7fb84006e890 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:55.374 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.363+0000 7fb84effd700 1 -- 172.21.15.78:0/2405941281 <== osd.7 v2:172.21.15.133:6824/35901 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb84006ef40 con 0x7fb84006c480 2023-10-11T09:52:55.386 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.379+0000 7fb864ccf700 1 -- 172.21.15.78:0/2405941281 --> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb86006c140 con 0x7fb84006c480 2023-10-11T09:52:55.387 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.379+0000 7fb84effd700 1 -- 172.21.15.78:0/2405941281 <== osd.7 v2:172.21.15.133:6824/35901 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fb858003d40 con 0x7fb84006c480 2023-10-11T09:52:55.388 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-11T09:52:55.388 INFO:teuthology.orchestra.run.smithi078.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-11T09:52:55.388 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-11T09:52:55.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.379+0000 7fb864ccf700 1 -- 172.21.15.78:0/2405941281 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x7fb84006c480 msgr2=0x7fb84006e890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:55.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.379+0000 7fb864ccf700 1 --2- 172.21.15.78:0/2405941281 >> [v2:172.21.15.133:6824/35901,v1:172.21.15.133:6825/35901] conn(0x7fb84006c480 0x7fb84006e890 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:55.390 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.379+0000 7fb864ccf700 1 -- 172.21.15.78:0/2405941281 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb840065ed0 msgr2=0x7fb840068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:55.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.379+0000 7fb864ccf700 1 --2- 172.21.15.78:0/2405941281 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fb840065ed0 0x7fb840068380 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb8600c5710 tx=0x7fb854001770 comp rx=0 tx=0).stop 2023-10-11T09:52:55.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.379+0000 7fb864ccf700 1 -- 172.21.15.78:0/2405941281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb860071d20 msgr2=0x7fb860158580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:55.391 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.379+0000 7fb864ccf700 1 --2- 172.21.15.78:0/2405941281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb860071d20 0x7fb860158580 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fb85000c490 tx=0x7fb85000c7a0 comp rx=0 tx=0).stop 2023-10-11T09:52:55.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.379+0000 7fb85ed9d700 1 -- 172.21.15.78:0/2405941281 reap_dead start 2023-10-11T09:52:55.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.379+0000 7fb864ccf700 1 -- 172.21.15.78:0/2405941281 shutdown_connections 2023-10-11T09:52:55.392 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.383+0000 7fb864ccf700 1 -- 172.21.15.78:0/2405941281 >> 172.21.15.78:0/2405941281 conn(0x7fb86006ae00 msgr2=0x7fb8600c4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:55.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.383+0000 7fb864ccf700 1 -- 172.21.15.78:0/2405941281 shutdown_connections 2023-10-11T09:52:55.393 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.383+0000 7fb864ccf700 1 -- 172.21.15.78:0/2405941281 wait complete. 2023-10-11T09:52:55.409 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-11T09:52:55.583 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.575+0000 7f5932bb7700 1 Processor -- start 2023-10-11T09:52:55.584 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.575+0000 7f5932bb7700 1 -- start start 2023-10-11T09:52:55.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.579+0000 7f5932bb7700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f592c15a800 0x7f592c0b4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:55.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.579+0000 7f5932bb7700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f592c070dc0 con 0x7f592c0b0d20 2023-10-11T09:52:55.586 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.579+0000 7f5932bb7700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f592c070ff0 con 0x7f592c06bb60 2023-10-11T09:52:55.587 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.579+0000 7f5932bb7700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f592c0b46a0 con 0x7f592c15a800 2023-10-11T09:52:55.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.579+0000 7f5931bb5700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f592c15a800 0x7f592c0b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:55.588 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.579+0000 7f59323b6700 1 --1- >> v1:172.21.15.78:6789/0 conn(0x7f592c0b0d20 0x7f592c15e590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:60334/0 (socket says 172.21.15.78:60334) 2023-10-11T09:52:55.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.579+0000 7f59323b6700 1 -- 172.21.15.78:0/3705419060 learned_addr learned my addr 172.21.15.78:0/3705419060 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:55.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.579+0000 7f5930bb3700 1 -- 172.21.15.78:0/3705419060 <== mon.0 v1:172.21.15.78:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2823999868 0 0) 0x7f592c070dc0 con 0x7f592c0b0d20 2023-10-11T09:52:55.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.579+0000 7f5930bb3700 1 -- 172.21.15.78:0/3705419060 --> v1:172.21.15.78:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59180035d0 con 0x7f592c0b0d20 2023-10-11T09:52:55.589 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.579+0000 7f5930bb3700 1 -- 172.21.15.78:0/3705419060 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1553166833 0 0) 0x7f592c070ff0 con 0x7f592c06bb60 2023-10-11T09:52:55.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.579+0000 7f5931bb5700 1 -- 172.21.15.78:0/3705419060 >> v1:172.21.15.133:6789/0 conn(0x7f592c06bb60 legacy=0x7f592c0b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:55.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.579+0000 7f5931bb5700 1 -- 172.21.15.78:0/3705419060 >> v1:172.21.15.78:6789/0 conn(0x7f592c0b0d20 legacy=0x7f592c15e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:55.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.579+0000 7f5931bb5700 1 -- 172.21.15.78:0/3705419060 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592c0712d0 con 0x7f592c15a800 2023-10-11T09:52:55.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.579+0000 7f5931bb5700 1 --2- 172.21.15.78:0/3705419060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f592c15a800 0x7f592c0b4160 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f5928009850 tx=0x7f5928009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46cf1d1a73c44864 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:55.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.583+0000 7f5930bb3700 1 -- 172.21.15.78:0/3705419060 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5928011830 con 0x7f592c15a800 2023-10-11T09:52:55.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.583+0000 7f5930bb3700 1 -- 172.21.15.78:0/3705419060 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5928011990 con 0x7f592c15a800 2023-10-11T09:52:55.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.583+0000 7f5930bb3700 1 -- 172.21.15.78:0/3705419060 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5928011d40 con 0x7f592c15a800 2023-10-11T09:52:55.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.583+0000 7f5932bb7700 1 -- 172.21.15.78:0/3705419060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f592c15a800 msgr2=0x7f592c0b4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:55.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.583+0000 7f5932bb7700 1 --2- 172.21.15.78:0/3705419060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f592c15a800 0x7f592c0b4160 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f5928009850 tx=0x7f5928009b60 comp rx=0 tx=0).stop 2023-10-11T09:52:55.593 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.583+0000 7f5932bb7700 1 -- 172.21.15.78:0/3705419060 shutdown_connections 2023-10-11T09:52:55.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.583+0000 7f5932bb7700 1 --2- 172.21.15.78:0/3705419060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f592c15a800 0x7f592c0b4160 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:55.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.583+0000 7f5932bb7700 1 -- 172.21.15.78:0/3705419060 >> 172.21.15.78:0/3705419060 conn(0x7f592c0cf820 msgr2=0x7f592c0cb610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:55.594 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.583+0000 7f5932bb7700 1 -- 172.21.15.78:0/3705419060 shutdown_connections 2023-10-11T09:52:55.595 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.583+0000 7f5932bb7700 1 -- 172.21.15.78:0/3705419060 wait complete. 2023-10-11T09:52:55.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.587+0000 7f5932bb7700 1 Processor -- start 2023-10-11T09:52:55.597 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.587+0000 7f5932bb7700 1 -- start start 2023-10-11T09:52:55.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.587+0000 7f5932bb7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f592c06bb60 0x7f592c152e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:55.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.587+0000 7f5932bb7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f592c0b0d20 0x7f592c0c5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:55.598 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.587+0000 7f5932bb7700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f592c15a800 0x7f592c158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:55.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.587+0000 7f5932bb7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c1548f0 con 0x7f592c0b0d20 2023-10-11T09:52:55.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.587+0000 7f5932bb7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c153340 con 0x7f592c06bb60 2023-10-11T09:52:55.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.587+0000 7f5932bb7700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f592c153c70 con 0x7f592c15a800 2023-10-11T09:52:55.599 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.587+0000 7f59313b4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f592c0b0d20 0x7f592c0c5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:55.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.587+0000 7f59313b4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f592c0b0d20 0x7f592c0c5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45452/0 (socket says 172.21.15.78:45452) 2023-10-11T09:52:55.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.587+0000 7f59313b4700 1 -- 172.21.15.78:0/2550580324 learned_addr learned my addr 172.21.15.78:0/2550580324 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:55.600 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.587+0000 7f5931bb5700 1 --2- 172.21.15.78:0/2550580324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f592c06bb60 0x7f592c152e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:55.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.587+0000 7f59323b6700 1 --2- 172.21.15.78:0/2550580324 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f592c15a800 0x7f592c158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:55.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.587+0000 7f59313b4700 1 -- 172.21.15.78:0/2550580324 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f592c15a800 msgr2=0x7f592c158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:55.601 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.587+0000 7f59313b4700 1 --2- 172.21.15.78:0/2550580324 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f592c15a800 0x7f592c158240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:55.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.587+0000 7f59313b4700 1 -- 172.21.15.78:0/2550580324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f592c06bb60 msgr2=0x7f592c152e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:55.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.587+0000 7f59313b4700 1 --2- 172.21.15.78:0/2550580324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f592c06bb60 0x7f592c152e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:55.602 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.587+0000 7f59313b4700 1 -- 172.21.15.78:0/2550580324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59280094b0 con 0x7f592c0b0d20 2023-10-11T09:52:55.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.587+0000 7f59323b6700 1 --2- 172.21.15.78:0/2550580324 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f592c15a800 0x7f592c158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:55.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.587+0000 7f59313b4700 1 --2- 172.21.15.78:0/2550580324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f592c0b0d20 0x7f592c0c5090 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f591c00c750 tx=0x7f591c00ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:55.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.591+0000 7f59227fc700 1 -- 172.21.15.78:0/2550580324 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591c0054e0 con 0x7f592c0b0d20 2023-10-11T09:52:55.603 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.591+0000 7f59227fc700 1 -- 172.21.15.78:0/2550580324 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f591c0036f0 con 0x7f592c0b0d20 2023-10-11T09:52:55.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.591+0000 7f5932bb7700 1 -- 172.21.15.78:0/2550580324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592c153f50 con 0x7f592c0b0d20 2023-10-11T09:52:55.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.591+0000 7f59227fc700 1 -- 172.21.15.78:0/2550580324 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591c003aa0 con 0x7f592c0b0d20 2023-10-11T09:52:55.604 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.591+0000 7f5932bb7700 1 -- 172.21.15.78:0/2550580324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f592c154510 con 0x7f592c0b0d20 2023-10-11T09:52:55.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.591+0000 7f5932bb7700 1 -- 172.21.15.78:0/2550580324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f592c067d00 con 0x7f592c0b0d20 2023-10-11T09:52:55.606 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.595+0000 7f59227fc700 1 -- 172.21.15.78:0/2550580324 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f591c003c00 con 0x7f592c0b0d20 2023-10-11T09:52:55.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.595+0000 7f59227fc700 1 --2- 172.21.15.78:0/2550580324 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5910065ed0 0x7f5910068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:55.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.595+0000 7f59227fc700 1 -- 172.21.15.78:0/2550580324 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f591c082a10 con 0x7f592c0b0d20 2023-10-11T09:52:55.607 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.595+0000 7f5931bb5700 1 --2- 172.21.15.78:0/2550580324 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5910065ed0 0x7f5910068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:55.608 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.595+0000 7f5931bb5700 1 --2- 172.21.15.78:0/2550580324 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5910065ed0 0x7f5910068380 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f59280090b0 tx=0x7f592801c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:55.609 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.603+0000 7f59227fc700 1 -- 172.21.15.78:0/2550580324 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f591c0502a0 con 0x7f592c0b0d20 2023-10-11T09:52:55.784 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.779+0000 7f5932bb7700 1 -- 172.21.15.78:0/2550580324 --> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f592c0715b0 con 0x7f5910065ed0 2023-10-11T09:52:55.785 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.779+0000 7f59227fc700 1 -- 172.21.15.78:0/2550580324 <== mgr.4101 v2:172.21.15.78:6832/35606 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f5928011c00 con 0x7f5910065ed0 2023-10-11T09:52:55.787 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.779+0000 7f5932bb7700 1 -- 172.21.15.78:0/2550580324 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5910065ed0 msgr2=0x7f5910068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:55.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.779+0000 7f5932bb7700 1 --2- 172.21.15.78:0/2550580324 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5910065ed0 0x7f5910068380 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f59280090b0 tx=0x7f592801c040 comp rx=0 tx=0).stop 2023-10-11T09:52:55.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.779+0000 7f5932bb7700 1 -- 172.21.15.78:0/2550580324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f592c0b0d20 msgr2=0x7f592c0c5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:55.788 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.779+0000 7f5932bb7700 1 --2- 172.21.15.78:0/2550580324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f592c0b0d20 0x7f592c0c5090 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f591c00c750 tx=0x7f591c00ca60 comp rx=0 tx=0).stop 2023-10-11T09:52:55.789 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.779+0000 7f5932bb7700 1 -- 172.21.15.78:0/2550580324 shutdown_connections 2023-10-11T09:52:55.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.779+0000 7f5932bb7700 1 --2- 172.21.15.78:0/2550580324 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f5910065ed0 0x7f5910068380 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:55.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.779+0000 7f5932bb7700 1 --2- 172.21.15.78:0/2550580324 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f592c15a800 0x7f592c158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:55.790 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.779+0000 7f5932bb7700 1 --2- 172.21.15.78:0/2550580324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f592c0b0d20 0x7f592c0c5090 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:55.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.779+0000 7f5932bb7700 1 --2- 172.21.15.78:0/2550580324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f592c06bb60 0x7f592c152e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:55.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.779+0000 7f5932bb7700 1 -- 172.21.15.78:0/2550580324 >> 172.21.15.78:0/2550580324 conn(0x7f592c0cf820 msgr2=0x7f592c15b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:55.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.779+0000 7f5932bb7700 1 -- 172.21.15.78:0/2550580324 shutdown_connections 2023-10-11T09:52:55.791 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.783+0000 7f5932bb7700 1 -- 172.21.15.78:0/2550580324 wait complete. 2023-10-11T09:52:55.792 INFO:teuthology.orchestra.run.smithi078.stderr:instructed osd(s) 7 to deep-scrub 2023-10-11T09:52:55.808 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-11T09:52:55.981 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.971+0000 7f5870144700 1 Processor -- start 2023-10-11T09:52:55.982 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.975+0000 7f5870144700 1 -- start start 2023-10-11T09:52:55.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.975+0000 7f5870144700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f586815cc20 0x7f58680d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:55.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.975+0000 7f5870144700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58680cd7f0 0x7f58680b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:55.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.975+0000 7f5870144700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58680d1820 con 0x7f58680cfbb0 2023-10-11T09:52:55.984 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.975+0000 7f5870144700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58680d1b00 con 0x7f586815cc20 2023-10-11T09:52:55.985 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.975+0000 7f5870144700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58680d1de0 con 0x7f58680cd7f0 2023-10-11T09:52:55.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.975+0000 7f586dee0700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f58680cfbb0 0x7f58680cff80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:49190/0 (socket says 172.21.15.78:49190) 2023-10-11T09:52:55.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.975+0000 7f586e6e1700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58680cd7f0 0x7f58680b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:55.986 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.979+0000 7f586dee0700 1 -- 172.21.15.78:0/1127466732 learned_addr learned my addr 172.21.15.78:0/1127466732 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:52:55.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.979+0000 7f586d6df700 1 --2- 172.21.15.78:0/1127466732 >> v2:172.21.15.133:3300/0 conn(0x7f586815cc20 0x7f58680d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:55.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.979+0000 7f586e6e1700 1 -- 172.21.15.78:0/1127466732 >> v2:172.21.15.133:3300/0 conn(0x7f586815cc20 msgr2=0x7f58680d28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:52:55.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.979+0000 7f586e6e1700 1 --2- 172.21.15.78:0/1127466732 >> v2:172.21.15.133:3300/0 conn(0x7f586815cc20 0x7f58680d28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:55.987 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.979+0000 7f586e6e1700 1 -- 172.21.15.78:0/1127466732 >> v1:172.21.15.133:6789/0 conn(0x7f58680cfbb0 legacy=0x7f58680cff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:55.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.979+0000 7f586cede700 1 -- 172.21.15.78:0/1127466732 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2610362741 0 0) 0x7f58680d1820 con 0x7f58680cfbb0 2023-10-11T09:52:55.988 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.979+0000 7f586e6e1700 1 -- 172.21.15.78:0/1127466732 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58680d20c0 con 0x7f58680cd7f0 2023-10-11T09:52:55.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.979+0000 7f586e6e1700 1 --2- 172.21.15.78:0/1127466732 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58680cd7f0 0x7f58680b5ff0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5860009d90 tx=0x7f586000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6753f3c1d43b9df server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:55.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.979+0000 7f586cede700 1 -- 172.21.15.78:0/1127466732 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586000e070 con 0x7f58680cd7f0 2023-10-11T09:52:55.990 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.979+0000 7f586cede700 1 -- 172.21.15.78:0/1127466732 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f586000bb20 con 0x7f58680cd7f0 2023-10-11T09:52:55.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.979+0000 7f586cede700 1 -- 172.21.15.78:0/1127466732 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5860012430 con 0x7f58680cd7f0 2023-10-11T09:52:55.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.979+0000 7f5870144700 1 -- 172.21.15.78:0/1127466732 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58680cd7f0 msgr2=0x7f58680b5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:55.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.979+0000 7f5870144700 1 --2- 172.21.15.78:0/1127466732 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58680cd7f0 0x7f58680b5ff0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5860009d90 tx=0x7f586000b330 comp rx=0 tx=0).stop 2023-10-11T09:52:55.991 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.979+0000 7f5870144700 1 -- 172.21.15.78:0/1127466732 shutdown_connections 2023-10-11T09:52:55.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.979+0000 7f5870144700 1 --2- 172.21.15.78:0/1127466732 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58680cd7f0 0x7f58680b5ff0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:55.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.979+0000 7f5870144700 1 --2- 172.21.15.78:0/1127466732 >> v2:172.21.15.133:3300/0 conn(0x7f586815cc20 0x7f58680d28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:55.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.979+0000 7f5870144700 1 -- 172.21.15.78:0/1127466732 >> 172.21.15.78:0/1127466732 conn(0x7f586806abf0 msgr2=0x7f586806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:55.992 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.979+0000 7f5870144700 1 -- 172.21.15.78:0/1127466732 shutdown_connections 2023-10-11T09:52:55.993 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.983+0000 7f5870144700 1 -- 172.21.15.78:0/1127466732 wait complete. 2023-10-11T09:52:55.994 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.983+0000 7f5870144700 1 Processor -- start 2023-10-11T09:52:55.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.983+0000 7f5870144700 1 -- start start 2023-10-11T09:52:55.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.987+0000 7f5870144700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58680cd7f0 0x7f58680b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:55.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.987+0000 7f5870144700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58680cfbb0 0x7f58680b6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:55.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.987+0000 7f5870144700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f586815cc20 0x7f58680b7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:55.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.987+0000 7f5870144700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58680c7d70 con 0x7f58680cd7f0 2023-10-11T09:52:55.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.987+0000 7f5870144700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58680c7a90 con 0x7f586815cc20 2023-10-11T09:52:55.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.987+0000 7f5870144700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58680b27c0 con 0x7f58680cfbb0 2023-10-11T09:52:55.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.987+0000 7f586dee0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58680cd7f0 0x7f58680b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:55.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.987+0000 7f586dee0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58680cd7f0 0x7f58680b32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45458/0 (socket says 172.21.15.78:45458) 2023-10-11T09:52:55.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.987+0000 7f586dee0700 1 -- 172.21.15.78:0/2340673156 learned_addr learned my addr 172.21.15.78:0/2340673156 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:52:55.997 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.987+0000 7f586e6e1700 1 --2- 172.21.15.78:0/2340673156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f586815cc20 0x7f58680b7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:55.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.987+0000 7f586d6df700 1 --2- 172.21.15.78:0/2340673156 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58680cfbb0 0x7f58680b6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:55.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.987+0000 7f586dee0700 1 -- 172.21.15.78:0/2340673156 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58680cfbb0 msgr2=0x7f58680b6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:55.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.987+0000 7f586dee0700 1 --2- 172.21.15.78:0/2340673156 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58680cfbb0 0x7f58680b6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:55.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.987+0000 7f586dee0700 1 -- 172.21.15.78:0/2340673156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f586815cc20 msgr2=0x7f58680b7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:55.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.987+0000 7f586dee0700 1 --2- 172.21.15.78:0/2340673156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f586815cc20 0x7f58680b7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:55.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.987+0000 7f586dee0700 1 -- 172.21.15.78:0/2340673156 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5860009a70 con 0x7f58680cd7f0 2023-10-11T09:52:55.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.987+0000 7f586d6df700 1 --2- 172.21.15.78:0/2340673156 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58680cfbb0 0x7f58680b6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:52:56.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.987+0000 7f586dee0700 1 --2- 172.21.15.78:0/2340673156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58680cd7f0 0x7f58680b32a0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f586400c580 tx=0x7f586400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:56.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.987+0000 7f585e7fc700 1 -- 172.21.15.78:0/2340673156 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5864004ca0 con 0x7f58680cd7f0 2023-10-11T09:52:56.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.987+0000 7f585e7fc700 1 -- 172.21.15.78:0/2340673156 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5864004e20 con 0x7f58680cd7f0 2023-10-11T09:52:56.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.987+0000 7f5870144700 1 -- 172.21.15.78:0/2340673156 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58680d2f80 con 0x7f58680cd7f0 2023-10-11T09:52:56.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.987+0000 7f585e7fc700 1 -- 172.21.15.78:0/2340673156 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5864007310 con 0x7f58680cd7f0 2023-10-11T09:52:56.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.987+0000 7f5870144700 1 -- 172.21.15.78:0/2340673156 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58680b2d80 con 0x7f58680cd7f0 2023-10-11T09:52:56.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.991+0000 7f5870144700 1 -- 172.21.15.78:0/2340673156 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5868067d00 con 0x7f58680cd7f0 2023-10-11T09:52:56.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.991+0000 7f585e7fc700 1 -- 172.21.15.78:0/2340673156 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f5864007470 con 0x7f58680cd7f0 2023-10-11T09:52:56.003 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.995+0000 7f585e7fc700 1 --2- 172.21.15.78:0/2340673156 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f584c065ed0 0x7f584c068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:52:56.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.995+0000 7f585e7fc700 1 -- 172.21.15.78:0/2340673156 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f5864083360 con 0x7f58680cd7f0 2023-10-11T09:52:56.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.995+0000 7f586d6df700 1 --2- 172.21.15.78:0/2340673156 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f584c065ed0 0x7f584c068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:52:56.004 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.995+0000 7f586d6df700 1 --2- 172.21.15.78:0/2340673156 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f584c065ed0 0x7f584c068380 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f5868072780 tx=0x7f5858008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:52:56.008 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:55.999+0000 7f585e7fc700 1 -- 172.21.15.78:0/2340673156 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f5864050b60 con 0x7f58680cd7f0 2023-10-11T09:52:56.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:56.175+0000 7f5870144700 1 -- 172.21.15.78:0/2340673156 --> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f58680d2e00 con 0x7f584c065ed0 2023-10-11T09:52:56.191 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:56.183+0000 7f585e7fc700 1 -- 172.21.15.78:0/2340673156 <== mgr.4101 v2:172.21.15.78:6832/35606 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70586 (secure 0 0 0) 0x7f5858011780 con 0x7f584c065ed0 2023-10-11T09:52:56.192 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:52:56.193 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:56.187+0000 7f5870144700 1 -- 172.21.15.78:0/2340673156 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f584c065ed0 msgr2=0x7f584c068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:56.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:56.187+0000 7f5870144700 1 --2- 172.21.15.78:0/2340673156 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f584c065ed0 0x7f584c068380 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f5868072780 tx=0x7f5858008040 comp rx=0 tx=0).stop 2023-10-11T09:52:56.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:56.187+0000 7f5870144700 1 -- 172.21.15.78:0/2340673156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58680cd7f0 msgr2=0x7f58680b32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:52:56.194 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:56.187+0000 7f5870144700 1 --2- 172.21.15.78:0/2340673156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58680cd7f0 0x7f58680b32a0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f586400c580 tx=0x7f586400c890 comp rx=0 tx=0).stop 2023-10-11T09:52:56.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:56.187+0000 7f5870144700 1 -- 172.21.15.78:0/2340673156 shutdown_connections 2023-10-11T09:52:56.195 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:56.187+0000 7f5870144700 1 --2- 172.21.15.78:0/2340673156 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f584c065ed0 0x7f584c068380 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:56.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:56.187+0000 7f5870144700 1 --2- 172.21.15.78:0/2340673156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f586815cc20 0x7f58680b7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:56.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:56.187+0000 7f5870144700 1 --2- 172.21.15.78:0/2340673156 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58680cfbb0 0x7f58680b6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:56.196 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:56.187+0000 7f5870144700 1 --2- 172.21.15.78:0/2340673156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58680cd7f0 0x7f58680b32a0 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:52:56.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:56.187+0000 7f5870144700 1 -- 172.21.15.78:0/2340673156 >> 172.21.15.78:0/2340673156 conn(0x7f586806abf0 msgr2=0x7f58680b8690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:52:56.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:56.187+0000 7f5870144700 1 -- 172.21.15.78:0/2340673156 shutdown_connections 2023-10-11T09:52:56.197 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:52:56.187+0000 7f5870144700 1 -- 172.21.15.78:0/2340673156 wait complete. 2023-10-11T09:52:56.198 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2023-10-11T09:52:56.219 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":498,"stamp":"2023-10-11T09:52:54.587454+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":262408,"kb_used_data":14024,"kb_used_omap":0,"kb_used_meta":248320,"kb_avail":754712312,"statfs":{"total":773094113280,"available":772825407488,"internally_reserved":0,"allocated":14360576,"data_stored":7583432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":254279680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-190,"num_objects":-100,"num_object_clones":0,"num_object_copies":-200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-100,"num_whiteouts":0,"num_read":-1100,"num_read_kb":-500,"num_write":-600,"num_write_kb":-100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002602"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":99,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-11T09:52:44.009882+0000","last_change":"2023-10-11T09:40:36.151839+0000","last_active":"2023-10-11T09:52:44.009882+0000","last_peered":"2023-10-11T09:52:44.009882+0000","last_clean":"2023-10-11T09:52:44.009882+0000","last_became_active":"2023-10-11T09:40:36.151179+0000","last_became_peered":"2023-10-11T09:40:36.151179+0000","last_unstale":"2023-10-11T09:52:44.009882+0000","last_undegraded":"2023-10-11T09:52:44.009882+0000","last_fullsized":"2023-10-11T09:52:44.009882+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:30:45.525751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":118,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-11T09:52:53.076395+0000","last_change":"2023-10-11T09:52:53.076569+0000","last_active":"2023-10-11T09:52:53.076395+0000","last_peered":"2023-10-11T09:52:53.076395+0000","last_clean":"2023-10-11T09:52:53.076395+0000","last_became_active":"2023-10-11T09:40:36.149919+0000","last_became_peered":"2023-10-11T09:40:36.149919+0000","last_unstale":"2023-10-11T09:52:53.076395+0000","last_undegraded":"2023-10-11T09:52:53.076395+0000","last_fullsized":"2023-10-11T09:52:53.076395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:52:53.076309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:52:53.076309+0000","last_clean_scrub_stamp":"2023-10-11T09:52:53.076309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:42:25.977603+0000","scrub_duration":0.0081017080000000009,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":128,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-11T09:52:43.706994+0000","last_change":"2023-10-11T09:40:36.162174+0000","last_active":"2023-10-11T09:52:43.706994+0000","last_peered":"2023-10-11T09:52:43.706994+0000","last_clean":"2023-10-11T09:52:43.706994+0000","last_became_active":"2023-10-11T09:40:36.161710+0000","last_became_peered":"2023-10-11T09:40:36.161710+0000","last_unstale":"2023-10-11T09:52:43.706994+0000","last_undegraded":"2023-10-11T09:52:43.706994+0000","last_fullsized":"2023-10-11T09:52:43.706994+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:57:03.630878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":107,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-11T09:52:45.509624+0000","last_change":"2023-10-11T09:40:36.160620+0000","last_active":"2023-10-11T09:52:45.509624+0000","last_peered":"2023-10-11T09:52:45.509624+0000","last_clean":"2023-10-11T09:52:45.509624+0000","last_became_active":"2023-10-11T09:40:36.160171+0000","last_became_peered":"2023-10-11T09:40:36.160171+0000","last_unstale":"2023-10-11T09:52:45.509624+0000","last_undegraded":"2023-10-11T09:52:45.509624+0000","last_fullsized":"2023-10-11T09:52:45.509624+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":104,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-11T09:52:44.009611+0000","last_change":"2023-10-11T09:40:36.151710+0000","last_active":"2023-10-11T09:52:44.009611+0000","last_peered":"2023-10-11T09:52:44.009611+0000","last_clean":"2023-10-11T09:52:44.009611+0000","last_became_active":"2023-10-11T09:40:36.151219+0000","last_became_peered":"2023-10-11T09:40:36.151219+0000","last_unstale":"2023-10-11T09:52:44.009611+0000","last_undegraded":"2023-10-11T09:52:44.009611+0000","last_fullsized":"2023-10-11T09:52:44.009611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:12:00.984667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":107,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-11T09:52:45.504655+0000","last_change":"2023-10-11T09:40:36.216248+0000","last_active":"2023-10-11T09:52:45.504655+0000","last_peered":"2023-10-11T09:52:45.504655+0000","last_clean":"2023-10-11T09:52:45.504655+0000","last_became_active":"2023-10-11T09:40:36.215890+0000","last_became_peered":"2023-10-11T09:40:36.215890+0000","last_unstale":"2023-10-11T09:52:45.504655+0000","last_undegraded":"2023-10-11T09:52:45.504655+0000","last_fullsized":"2023-10-11T09:52:45.504655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":129,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-11T09:52:43.703936+0000","last_change":"2023-10-11T09:40:36.149739+0000","last_active":"2023-10-11T09:52:43.703936+0000","last_peered":"2023-10-11T09:52:43.703936+0000","last_clean":"2023-10-11T09:52:43.703936+0000","last_became_active":"2023-10-11T09:40:36.149419+0000","last_became_peered":"2023-10-11T09:40:36.149419+0000","last_unstale":"2023-10-11T09:52:43.703936+0000","last_undegraded":"2023-10-11T09:52:43.703936+0000","last_fullsized":"2023-10-11T09:52:43.703936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:54:50.640362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":104,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-11T09:52:44.010199+0000","last_change":"2023-10-11T09:40:36.148773+0000","last_active":"2023-10-11T09:52:44.010199+0000","last_peered":"2023-10-11T09:52:44.010199+0000","last_clean":"2023-10-11T09:52:44.010199+0000","last_became_active":"2023-10-11T09:40:36.148489+0000","last_became_peered":"2023-10-11T09:40:36.148489+0000","last_unstale":"2023-10-11T09:52:44.010199+0000","last_undegraded":"2023-10-11T09:52:44.010199+0000","last_fullsized":"2023-10-11T09:52:44.010199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:35.118579+0000","last_clean_scrub_stamp":"2023-10-11T09:40:35.118579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:38:34.978388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":282,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-11T09:52:43.704492+0000","last_change":"2023-10-11T09:40:29.874372+0000","last_active":"2023-10-11T09:52:43.704492+0000","last_peered":"2023-10-11T09:52:43.704492+0000","last_clean":"2023-10-11T09:52:43.704492+0000","last_became_active":"2023-10-11T09:40:29.873621+0000","last_became_peered":"2023-10-11T09:40:29.873621+0000","last_unstale":"2023-10-11T09:52:43.704492+0000","last_undegraded":"2023-10-11T09:52:43.704492+0000","last_fullsized":"2023-10-11T09:52:43.704492+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:40:28.742276+0000","last_clean_scrub_stamp":"2023-10-11T09:40:28.742276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:22:07.985311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575001,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33236,"kb_used_data":2188,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338604,"statfs":{"total":96636764160,"available":96602730496,"internally_reserved":0,"allocated":2240512,"data_stored":1390705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.056,"15min":1.087},"min":{"1min":0.68999999999999995,"5min":0.56499999999999995,"15min":0.48999999999999999},"max":{"1min":1.4730000000000001,"5min":1.7390000000000001,"15min":1.903},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.034,"5min":1.0329999999999999,"15min":1.0860000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.50900000000000001},"max":{"1min":1.4970000000000001,"5min":1.554,"15min":1.696},"last":1.381}]},{"osd":1,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.1140000000000001,"15min":1.123},"min":{"1min":0.91000000000000003,"5min":0.66800000000000004,"15min":0.63100000000000001},"max":{"1min":1.4059999999999999,"5min":1.645,"15min":2.25},"last":1.284},{"interface":"front","average":{"1min":1.167,"5min":1.0489999999999999,"15min":1.0920000000000001},"min":{"1min":0.93300000000000005,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.4950000000000001,"5min":1.587,"15min":1.696},"last":1.117}]},{"osd":2,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.0840000000000001,"15min":1.103},"min":{"1min":0.72899999999999998,"5min":0.59499999999999997,"15min":0.58999999999999997},"max":{"1min":1.411,"5min":1.5049999999999999,"15min":1.873},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.175,"5min":1.1719999999999999,"15min":1.155},"min":{"1min":0.72499999999999998,"5min":0.66400000000000003,"15min":0.65500000000000003},"max":{"1min":1.579,"5min":1.78,"15min":1.78},"last":1.1639999999999999}]},{"osd":3,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.216,"15min":1.22},"min":{"1min":0.89200000000000002,"5min":0.80000000000000004,"15min":0.64900000000000002},"max":{"1min":1.617,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.151},{"interface":"front","average":{"1min":1.302,"5min":1.284,"15min":1.2809999999999999},"min":{"1min":0.94699999999999995,"5min":0.747,"15min":0.61699999999999999},"max":{"1min":1.54,"5min":1.8720000000000001,"15min":2.1579999999999999},"last":1.339}]},{"osd":4,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.0609999999999999,"15min":1.1200000000000001},"min":{"1min":0.71999999999999997,"5min":0.64900000000000002,"15min":0.53300000000000003},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":2.2010000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":1.181,"5min":1.208,"15min":1.226},"min":{"1min":0.93100000000000005,"5min":0.67200000000000004,"15min":0.63500000000000001},"max":{"1min":1.4199999999999999,"5min":1.768,"15min":1.9339999999999999},"last":0.95799999999999996}]},{"osd":5,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.2709999999999999,"15min":1.278},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.7350000000000001,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.024},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2050000000000001,"15min":1.2749999999999999},"min":{"1min":0.82999999999999996,"5min":0.78300000000000003,"15min":0.65900000000000003},"max":{"1min":1.714,"5min":1.9450000000000001,"15min":3.0750000000000002},"last":0.75900000000000001}]},{"osd":6,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.3300000000000001,"15min":1.3919999999999999},"min":{"1min":0.98599999999999999,"5min":0.77000000000000002,"15min":0.70999999999999996},"max":{"1min":2,"5min":2,"15min":3.137},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3500000000000001,"15min":1.381},"min":{"1min":0.89700000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.897,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.256}]}]},{"osd":6,"up_from":13,"seq":55834575000,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1608,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1646592,"data_stored":800337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.125,"15min":1.036},"min":{"1min":0.626,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.403,"5min":7.0730000000000004,"15min":7.0730000000000004},"last":1.171},{"interface":"front","average":{"1min":1.01,"5min":1.179,"15min":1.071},"min":{"1min":0.65900000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.357,"5min":8.8450000000000006,"15min":8.8450000000000006},"last":0.65800000000000003}]},{"osd":1,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.359,"15min":1.232},"min":{"1min":0.627,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.5149999999999999,"5min":15.523,"15min":15.523},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.181,"5min":1.3069999999999999,"15min":1.2310000000000001},"min":{"1min":0.80700000000000005,"5min":0.59199999999999997,"15min":0.53400000000000003},"max":{"1min":1.5509999999999999,"5min":15.253,"15min":15.253},"last":1.3240000000000001}]},{"osd":2,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.1200000000000001,"15min":1.1319999999999999},"min":{"1min":0.745,"5min":0.434,"15min":0.434},"max":{"1min":1.335,"5min":2.7919999999999998,"15min":2.7919999999999998},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1930000000000001,"15min":1.1890000000000001},"min":{"1min":0.79400000000000004,"5min":0.61599999999999999,"15min":0.44500000000000001},"max":{"1min":1.6060000000000001,"5min":5.0990000000000002,"15min":5.0990000000000002},"last":1.2729999999999999}]},{"osd":3,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.2150000000000001,"15min":1.155},"min":{"1min":0.72599999999999998,"5min":0.70599999999999996,"15min":0.627},"max":{"1min":1.421,"5min":11.945,"15min":11.945},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.246,"5min":1.175,"15min":1.2010000000000001},"min":{"1min":0.84299999999999997,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.659,"5min":1.659,"15min":3.0819999999999999},"last":1.0109999999999999}]},{"osd":4,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.2889999999999999,"15min":1.2769999999999999},"min":{"1min":0.626,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.6359999999999999,"5min":7.5599999999999996,"15min":7.5599999999999996},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.3380000000000001,"15min":1.3069999999999999},"min":{"1min":0.81299999999999994,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.542,"5min":8.7520000000000007,"15min":8.7520000000000007},"last":1.1399999999999999}]},{"osd":5,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.196,"15min":1.2290000000000001},"min":{"1min":0.65900000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.585,"5min":2.1560000000000001,"15min":2.4009999999999998},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.282,"15min":1.3200000000000001},"min":{"1min":0.68700000000000006,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.978,"5min":2.4390000000000001,"15min":2.7290000000000001},"last":1.3879999999999999}]},{"osd":7,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.335,"15min":1.3440000000000001},"min":{"1min":0.91400000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.706,"5min":8.7970000000000006,"15min":8.7970000000000006},"last":1.29},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.3899999999999999,"15min":1.4199999999999999},"min":{"1min":1.01,"5min":0.625,"15min":0.625},"max":{"1min":1.8620000000000001,"5min":2.641,"15min":2.641},"last":1.347}]}]},{"osd":5,"up_from":13,"seq":55834575000,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1608,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1646592,"data_stored":800337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.044,"15min":1.0129999999999999},"min":{"1min":0.69799999999999995,"5min":0.64100000000000001,"15min":0.45300000000000001},"max":{"1min":1.4770000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.494},{"interface":"front","average":{"1min":1.125,"5min":1.087,"15min":1.0269999999999999},"min":{"1min":0.66200000000000003,"5min":0.47699999999999998,"15min":0.46500000000000002},"max":{"1min":1.3460000000000001,"5min":1.536,"15min":1.7649999999999999},"last":1.1100000000000001}]},{"osd":1,"last update":"Wed Oct 11 09:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.1160000000000001,"15min":1.034},"min":{"1min":0.69599999999999995,"5min":0.68100000000000005,"15min":0.49399999999999999},"max":{"1min":1.542,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.359},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.0900000000000001},"min":{"1min":0.71599999999999997,"5min":0.61599999999999999,"15min":0.437},"max":{"1min":1.4299999999999999,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":0.83099999999999996}]},{"osd":2,"last update":"Wed Oct 11 09:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1819999999999999,"15min":1.179},"min":{"1min":0.76100000000000001,"5min":0.746,"15min":0.73899999999999999},"max":{"1min":1.5009999999999999,"5min":2.133,"15min":2.133},"last":1.234},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2370000000000001,"15min":1.2190000000000001},"min":{"1min":0.94099999999999995,"5min":0.64200000000000002,"15min":0.52300000000000002},"max":{"1min":1.46,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.258}]},{"osd":3,"last update":"Wed Oct 11 09:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.80700000000000005,"5min":0.752,"15min":0.56999999999999995},"max":{"1min":1.3580000000000001,"5min":1.8859999999999999,"15min":1.921},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.1950000000000001,"15min":1.1659999999999999},"min":{"1min":0.92500000000000004,"5min":0.71999999999999997,"15min":0.55300000000000005},"max":{"1min":1.492,"5min":1.9690000000000001,"15min":2.0209999999999999},"last":0.91400000000000003}]},{"osd":4,"last update":"Wed Oct 11 09:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.1919999999999999,"15min":1.1859999999999999},"min":{"1min":0.61399999999999999,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.4139999999999999,"5min":2.0169999999999999,"15min":2.117},"last":1.405},{"interface":"front","average":{"1min":1.075,"5min":1.157,"15min":1.1759999999999999},"min":{"1min":0.81999999999999995,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.4159999999999999,"5min":1.9690000000000001,"15min":2.1749999999999998},"last":1.319}]},{"osd":6,"last update":"Wed Oct 11 09:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.228,"15min":1.25},"min":{"1min":0.75700000000000001,"5min":0.69499999999999995,"15min":0.59499999999999997},"max":{"1min":1.5449999999999999,"5min":1.9239999999999999,"15min":2.4529999999999998},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.252,"5min":1.331,"15min":1.3200000000000001},"min":{"1min":0.78700000000000003,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.478,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.5720000000000001}]},{"osd":7,"last update":"Wed Oct 11 09:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.3320000000000001,"15min":1.341},"min":{"1min":1.0069999999999999,"5min":0.58799999999999997,"15min":0.57699999999999996},"max":{"1min":1.502,"5min":2.347,"15min":3.6120000000000001},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.353,"5min":1.4319999999999999,"15min":1.427},"min":{"1min":0.98499999999999999,"5min":0.89100000000000001,"15min":0.71299999999999997},"max":{"1min":1.589,"5min":2.2679999999999998,"15min":3.2109999999999999},"last":1.54}]}]},{"osd":4,"up_from":13,"seq":55834575000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32656,"kb_used_data":1608,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1646592,"data_stored":800337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.071,"15min":1.0640000000000001},"min":{"1min":0.74399999999999999,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":2.0369999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.133,"5min":1.0760000000000001,"15min":1.0880000000000001},"min":{"1min":0.90000000000000002,"5min":0.60199999999999998,"15min":0.54700000000000004},"max":{"1min":1.5129999999999999,"5min":1.669,"15min":2.1160000000000001},"last":1.1579999999999999}]},{"osd":1,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.109,"15min":1.089},"min":{"1min":0.72699999999999998,"5min":0.66500000000000004,"15min":0.60699999999999998},"max":{"1min":1.95,"5min":1.95,"15min":1.95},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.159,"15min":1.1499999999999999},"min":{"1min":0.77800000000000002,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":2.02,"5min":2.02,"15min":2.02},"last":1.3400000000000001}]},{"osd":2,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.232,"15min":1.2030000000000001},"min":{"1min":0.72499999999999998,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.5859999999999999,"5min":2.5859999999999999,"15min":2.5859999999999999},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.379,"5min":1.2589999999999999,"15min":1.228},"min":{"1min":0.69699999999999995,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.5350000000000001,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.4239999999999999}]},{"osd":3,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.1140000000000001,"15min":1.1240000000000001},"min":{"1min":0.79500000000000004,"5min":0.72599999999999998,"15min":0.60199999999999998},"max":{"1min":1.353,"5min":1.6839999999999999,"15min":1.6930000000000001},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.1859999999999999,"15min":1.236},"min":{"1min":0.83799999999999997,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.5030000000000001,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":1.127}]},{"osd":5,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.1830000000000001,"15min":1.1879999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":1.077},{"interface":"front","average":{"1min":1.175,"5min":1.1599999999999999,"15min":1.181},"min":{"1min":0.76200000000000001,"5min":0.63300000000000001,"15min":0.61199999999999999},"max":{"1min":1.6539999999999999,"5min":1.732,"15min":1.8520000000000001},"last":1.024}]},{"osd":6,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.2390000000000001,"15min":1.252},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.2},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3149999999999999,"15min":1.3169999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.0830000000000002,"5min":2.0830000000000002,"15min":2.1120000000000001},"last":0.92700000000000005}]},{"osd":7,"last update":"Wed Oct 11 09:51:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.2629999999999999,"15min":1.296},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.60899999999999999},"max":{"1min":1.77,"5min":1.794,"15min":1.9470000000000001},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.371,"15min":1.4079999999999999},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.75900000000000001},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.3720000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574999,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1608,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1646592,"data_stored":800337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.79500000000000004,"15min":0.79100000000000004},"min":{"1min":0.63500000000000001,"5min":0.442,"15min":0.33000000000000002},"max":{"1min":1.1220000000000001,"5min":1.399,"15min":1.462},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.84399999999999997,"15min":0.82799999999999996},"min":{"1min":0.51900000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.331,"5min":1.363,"15min":1.4950000000000001},"last":0.59099999999999997}]},{"osd":1,"last update":"Wed Oct 11 09:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.84299999999999997,"15min":0.81499999999999995},"min":{"1min":0.60899999999999999,"5min":0.47799999999999998,"15min":0.42699999999999999},"max":{"1min":1.2250000000000001,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.94299999999999995,"15min":0.92400000000000004},"min":{"1min":0.70799999999999996,"5min":0.53100000000000003,"15min":0.49099999999999999},"max":{"1min":1.4119999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.53800000000000003}]},{"osd":2,"last update":"Wed Oct 11 09:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.98499999999999999,"15min":0.96599999999999997},"min":{"1min":0.63400000000000001,"5min":0.57699999999999996,"15min":0.52900000000000003},"max":{"1min":1.3460000000000001,"5min":1.601,"15min":1.601},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.99299999999999999,"5min":1.042,"15min":1.044},"min":{"1min":0.67400000000000004,"5min":0.63700000000000001,"15min":0.432},"max":{"1min":1.2669999999999999,"5min":1.607,"15min":2.5870000000000002},"last":0.80700000000000005}]},{"osd":4,"last update":"Wed Oct 11 09:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1319999999999999,"15min":1.151},"min":{"1min":0.88100000000000001,"5min":0.77900000000000003,"15min":0.68100000000000005},"max":{"1min":1.538,"5min":1.556,"15min":2.395},"last":1.121},{"interface":"front","average":{"1min":1.226,"5min":1.1919999999999999,"15min":1.2270000000000001},"min":{"1min":0.98799999999999999,"5min":0.747,"15min":0.73599999999999999},"max":{"1min":1.833,"5min":1.9099999999999999,"15min":3.1579999999999999},"last":1.073}]},{"osd":5,"last update":"Wed Oct 11 09:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.2270000000000001,"15min":1.256},"min":{"1min":0.94599999999999995,"5min":0.84699999999999998,"15min":0.76500000000000001},"max":{"1min":1.6990000000000001,"5min":1.724,"15min":3.0009999999999999},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.266,"15min":1.3089999999999999},"min":{"1min":1.107,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":3.8050000000000002},"last":0.94199999999999995}]},{"osd":6,"last update":"Wed Oct 11 09:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.2190000000000001,"15min":1.24},"min":{"1min":0.98499999999999999,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.659,"5min":1.6890000000000001,"15min":2.1949999999999998},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.292,"5min":1.2829999999999999,"15min":1.3169999999999999},"min":{"1min":0.96399999999999997,"5min":0.91700000000000004,"15min":0.84499999999999997},"max":{"1min":1.782,"5min":1.782,"15min":3.2549999999999999},"last":1.02}]},{"osd":7,"last update":"Wed Oct 11 09:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3029999999999999,"15min":1.351},"min":{"1min":1.085,"5min":0.93700000000000006,"15min":0.76700000000000002},"max":{"1min":1.637,"5min":1.7150000000000001,"15min":3.7490000000000001},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.294,"15min":1.3140000000000001},"min":{"1min":0.90600000000000003,"5min":0.83199999999999996,"15min":0.81799999999999995},"max":{"1min":1.619,"5min":1.7969999999999999,"15min":2.625},"last":1.0429999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575001,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1608,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1646592,"data_stored":800337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:52:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.81799999999999995,"15min":0.80600000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.38500000000000001},"max":{"1min":1.1359999999999999,"5min":1.8180000000000001,"15min":2.0600000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.88900000000000001,"15min":0.88400000000000001},"min":{"1min":0.48899999999999999,"5min":0.47299999999999998,"15min":0.33100000000000002},"max":{"1min":1.202,"5min":2.2850000000000001,"15min":2.8919999999999999},"last":0.40400000000000003}]},{"osd":1,"last update":"Wed Oct 11 09:52:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.82899999999999996,"15min":0.90900000000000003},"min":{"1min":0.50800000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.0229999999999999,"5min":1.8069999999999999,"15min":3.7650000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.97499999999999998,"15min":0.96699999999999997},"min":{"1min":0.59399999999999997,"5min":0.45600000000000002,"15min":0.40300000000000002},"max":{"1min":1.3460000000000001,"5min":1.9099999999999999,"15min":2.5339999999999998},"last":0.78600000000000003}]},{"osd":3,"last update":"Wed Oct 11 09:52:48 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.91500000000000004,"15min":0.98599999999999999},"min":{"1min":0.61899999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.167,"5min":1.591,"15min":2.403},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0640000000000001,"15min":1.0389999999999999},"min":{"1min":0.68100000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.351,"5min":1.754,"15min":2.7589999999999999},"last":0.88200000000000001}]},{"osd":4,"last update":"Wed Oct 11 09:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.1639999999999999,"15min":1.1779999999999999},"min":{"1min":0.69999999999999996,"5min":0.68600000000000005,"15min":0.65000000000000002},"max":{"1min":1.5349999999999999,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.242,"15min":1.3069999999999999},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73399999999999999},"max":{"1min":1.4630000000000001,"5min":2.1669999999999998,"15min":2.6859999999999999},"last":1.0489999999999999}]},{"osd":5,"last update":"Wed Oct 11 09:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.26,"15min":1.2829999999999999},"min":{"1min":0.83099999999999996,"5min":0.80500000000000005,"15min":0.77400000000000002},"max":{"1min":1.5800000000000001,"5min":2.0179999999999998,"15min":3.044},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.167,"15min":1.23},"min":{"1min":0.746,"5min":0.746,"15min":0.69599999999999995},"max":{"1min":1.405,"5min":2.411,"15min":3.5790000000000002},"last":0.82199999999999995}]},{"osd":6,"last update":"Wed Oct 11 09:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.2050000000000001,"15min":1.282},"min":{"1min":0.77900000000000003,"5min":0.75,"15min":0.75},"max":{"1min":1.512,"5min":1.774,"15min":3.1600000000000001},"last":1.165},{"interface":"front","average":{"1min":1.29,"5min":1.3220000000000001,"15min":1.321},"min":{"1min":0.88600000000000001,"5min":0.83499999999999996,"15min":0.623},"max":{"1min":1.6819999999999999,"5min":1.9670000000000001,"15min":2.839},"last":1.137}]},{"osd":7,"last update":"Wed Oct 11 09:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.274,"15min":1.325},"min":{"1min":0.879,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.552,"5min":2.7130000000000001,"15min":3.1230000000000002},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.282,"5min":1.3220000000000001,"15min":1.3999999999999999},"min":{"1min":0.877,"5min":0.84899999999999998,"15min":0.80000000000000004},"max":{"1min":1.6200000000000001,"5min":1.7909999999999999,"15min":2.988},"last":1.1140000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575000,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1608,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1646592,"data_stored":800337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:52:50 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.81000000000000005,"15min":0.78500000000000003},"min":{"1min":0.50900000000000001,"5min":0.48799999999999999,"15min":0.375},"max":{"1min":1.2270000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":1.004},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.83299999999999996,"15min":0.82199999999999995},"min":{"1min":0.56699999999999995,"5min":0.40100000000000002,"15min":0.36699999999999999},"max":{"1min":1.181,"5min":1.593,"15min":1.593},"last":0.95199999999999996}]},{"osd":2,"last update":"Wed Oct 11 09:52:50 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.82599999999999996,"15min":0.83399999999999996},"min":{"1min":0.57799999999999996,"5min":0.51700000000000002,"15min":0.36199999999999999},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":1.3640000000000001},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.92900000000000005,"15min":0.94299999999999995},"min":{"1min":0.58799999999999997,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.2190000000000001,"5min":1.4159999999999999,"15min":1.6619999999999999},"last":1.2190000000000001}]},{"osd":3,"last update":"Wed Oct 11 09:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.018,"15min":1.012},"min":{"1min":0.63200000000000001,"5min":0.61599999999999999,"15min":0.46600000000000003},"max":{"1min":1.4890000000000001,"5min":1.714,"15min":1.714},"last":1.403},{"interface":"front","average":{"1min":1.044,"5min":0.97799999999999998,"15min":0.97299999999999998},"min":{"1min":0.68899999999999995,"5min":0.54200000000000004,"15min":0.38900000000000001},"max":{"1min":1.409,"5min":1.611,"15min":1.611},"last":1.1559999999999999}]},{"osd":4,"last update":"Wed Oct 11 09:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.1100000000000001,"15min":1.1319999999999999},"min":{"1min":0.85199999999999998,"5min":0.77400000000000002,"15min":0.73599999999999999},"max":{"1min":1.3460000000000001,"5min":1.5980000000000001,"15min":2.016},"last":1.288},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.2290000000000001,"15min":1.254},"min":{"1min":1.026,"5min":0.85699999999999998,"15min":0.80000000000000004},"max":{"1min":1.601,"5min":1.748,"15min":2.323},"last":1.536}]},{"osd":5,"last update":"Wed Oct 11 09:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.2150000000000001,"15min":1.252},"min":{"1min":1.0089999999999999,"5min":0.79300000000000004,"15min":0.65800000000000003},"max":{"1min":1.524,"5min":1.722,"15min":2.4079999999999999},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.194,"15min":1.2190000000000001},"min":{"1min":1.0600000000000001,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.647,"5min":1.677,"15min":2.1349999999999998},"last":1.4430000000000001}]},{"osd":6,"last update":"Wed Oct 11 09:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.2569999999999999,"15min":1.294},"min":{"1min":1.01,"5min":0.84599999999999997,"15min":0.71499999999999997},"max":{"1min":1.7170000000000001,"5min":2.4620000000000002,"15min":2.8940000000000001},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.2310000000000001,"15min":1.2889999999999999},"min":{"1min":0.997,"5min":0.84799999999999998,"15min":0.79200000000000004},"max":{"1min":1.6100000000000001,"5min":1.673,"15min":3.3719999999999999},"last":1.6100000000000001}]},{"osd":7,"last update":"Wed Oct 11 09:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3260000000000001,"15min":1.357},"min":{"1min":1.0269999999999999,"5min":0.86899999999999999,"15min":0.86099999999999999},"max":{"1min":1.7929999999999999,"5min":1.7929999999999999,"15min":2.194},"last":1.7929999999999999},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.335,"15min":1.369},"min":{"1min":1.014,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.6160000000000001,"5min":1.857,"15min":1.968},"last":1.571}]}]},{"osd":0,"up_from":13,"seq":55834574999,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33236,"kb_used_data":2188,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338604,"statfs":{"total":96636764160,"available":96602730496,"internally_reserved":0,"allocated":2240512,"data_stored":1390705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.83999999999999997,"15min":0.80800000000000005},"min":{"1min":0.65800000000000003,"5min":0.45300000000000001,"15min":0.38500000000000001},"max":{"1min":1.3160000000000001,"5min":1.611,"15min":1.611},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.88200000000000001,"15min":0.89300000000000002},"min":{"1min":0.60399999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":2.3100000000000001},"last":0.64200000000000002}]},{"osd":2,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.95699999999999996,"15min":0.93600000000000005},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.3160000000000001,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":0.995},{"interface":"front","average":{"1min":1.002,"5min":0.95699999999999996,"15min":0.95799999999999996},"min":{"1min":0.56299999999999994,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.306,"5min":1.569,"15min":1.9450000000000001},"last":0.88400000000000001}]},{"osd":3,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.81100000000000005,"15min":0.83199999999999996},"min":{"1min":0.63900000000000001,"5min":0.48999999999999999,"15min":0.45600000000000002},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":2.7639999999999998},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.95099999999999996,"15min":0.97999999999999998},"min":{"1min":0.73799999999999999,"5min":0.51400000000000001,"15min":0.48999999999999999},"max":{"1min":1.3879999999999999,"5min":1.482,"15min":2.1789999999999998},"last":0.92000000000000004}]},{"osd":4,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.131,"15min":1.163},"min":{"1min":0.79100000000000004,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.4450000000000001,"5min":1.6659999999999999,"15min":1.792},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.149,"5min":1.1379999999999999,"15min":1.161},"min":{"1min":0.94499999999999995,"5min":0.755,"15min":0.69799999999999995},"max":{"1min":1.4039999999999999,"5min":1.9399999999999999,"15min":2.6629999999999998},"last":1.175}]},{"osd":5,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.234,"15min":1.2769999999999999},"min":{"1min":0.99099999999999999,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.4810000000000001,"5min":1.829,"15min":3.0150000000000001},"last":1.4079999999999999},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.161,"15min":1.2430000000000001},"min":{"1min":0.85599999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.4319999999999999,"5min":1.48,"15min":3.601},"last":1.016}]},{"osd":6,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.151,"15min":1.1870000000000001},"min":{"1min":0.87,"5min":0.79100000000000004,"15min":0.78000000000000003},"max":{"1min":1.46,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.383},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.28,"15min":1.2949999999999999},"min":{"1min":0.98699999999999999,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":1.5269999999999999,"5min":1.7430000000000001,"15min":2.1440000000000001},"last":1.216}]},{"osd":7,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2410000000000001,"15min":1.282},"min":{"1min":0.97899999999999998,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.5680000000000001,"5min":1.607,"15min":1.8759999999999999},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.232,"5min":1.2350000000000001,"15min":1.278},"min":{"1min":1.02,"5min":0.88800000000000001,"15min":0.73899999999999999},"max":{"1min":1.4710000000000001,"5min":2.016,"15min":2.5419999999999998},"last":1.4690000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-11T09:52:56.223 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-11T09:40:35.118579+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=11, tm_hour=9, tm_min=40, tm_sec=35, tm_wday=2, tm_yday=284, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=11, tm_hour=9, tm_min=52, tm_sec=49, tm_wday=2, tm_yday=284, tm_isdst=0) 2023-10-11T09:52:56.224 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-11T09:40:35.118579+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=11, tm_hour=9, tm_min=40, tm_sec=35, tm_wday=2, tm_yday=284, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=11, tm_hour=9, tm_min=52, tm_sec=49, tm_wday=2, tm_yday=284, tm_isdst=0) 2023-10-11T09:52:56.224 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-11T09:40:35.118579+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=11, tm_hour=9, tm_min=40, tm_sec=35, tm_wday=2, tm_yday=284, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=11, tm_hour=9, tm_min=52, tm_sec=49, tm_wday=2, tm_yday=284, tm_isdst=0) 2023-10-11T09:52:56.224 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-11T09:40:35.118579+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=11, tm_hour=9, tm_min=40, tm_sec=35, tm_wday=2, tm_yday=284, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=11, tm_hour=9, tm_min=52, tm_sec=49, tm_wday=2, tm_yday=284, tm_isdst=0) 2023-10-11T09:52:56.225 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-11T09:40:35.118579+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=11, tm_hour=9, tm_min=40, tm_sec=35, tm_wday=2, tm_yday=284, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=11, tm_hour=9, tm_min=52, tm_sec=49, tm_wday=2, tm_yday=284, tm_isdst=0) 2023-10-11T09:52:56.225 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-11T09:40:35.118579+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=11, tm_hour=9, tm_min=40, tm_sec=35, tm_wday=2, tm_yday=284, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=11, tm_hour=9, tm_min=52, tm_sec=49, tm_wday=2, tm_yday=284, tm_isdst=0) 2023-10-11T09:52:56.226 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-11T09:40:35.118579+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=11, tm_hour=9, tm_min=40, tm_sec=35, tm_wday=2, tm_yday=284, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=11, tm_hour=9, tm_min=52, tm_sec=49, tm_wday=2, tm_yday=284, tm_isdst=0) 2023-10-11T09:52:56.226 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-11T09:40:28.742276+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=11, tm_hour=9, tm_min=40, tm_sec=28, tm_wday=2, tm_yday=284, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=11, tm_hour=9, tm_min=52, tm_sec=49, tm_wday=2, tm_yday=284, tm_isdst=0) 2023-10-11T09:52:56.226 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-11T09:53:01.064 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:53:01.072 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:53:16.227 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-11T09:53:16.409 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.402+0000 7fa171eee700 1 Processor -- start 2023-10-11T09:53:16.410 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.402+0000 7fa171eee700 1 -- start start 2023-10-11T09:53:16.411 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.402+0000 7fa171eee700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa16c160de0 0x7fa16c0ca670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:53:16.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.402+0000 7fa171eee700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa16c0cca70 0x7fa16c0c1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:53:16.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.402+0000 7fa171eee700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa16c162130 con 0x7fa16c0ccec0 2023-10-11T09:53:16.412 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.402+0000 7fa171eee700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa16c161560 con 0x7fa16c0cca70 2023-10-11T09:53:16.413 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.402+0000 7fa171eee700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c161800 con 0x7fa16c160de0 2023-10-11T09:53:16.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.406+0000 7fa16b7fe700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa16c160de0 0x7fa16c0ca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:53:16.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.406+0000 7fa16bfff700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa16c0ccec0 0x7fa16c164ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.78:42022/0 (socket says 172.21.15.78:42022) 2023-10-11T09:53:16.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.406+0000 7fa16bfff700 1 -- 172.21.15.78:0/2603293536 learned_addr learned my addr 172.21.15.78:0/2603293536 (peer_addr_for_me v1:172.21.15.78:0/0) 2023-10-11T09:53:16.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.406+0000 7fa16affd700 1 --2- 172.21.15.78:0/2603293536 >> v2:172.21.15.133:3300/0 conn(0x7fa16c0cca70 0x7fa16c0c1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:53:16.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.406+0000 7fa16a7fc700 1 -- 172.21.15.78:0/2603293536 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1564636287 0 0) 0x7fa16c162130 con 0x7fa16c0ccec0 2023-10-11T09:53:16.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.406+0000 7fa16a7fc700 1 -- 172.21.15.78:0/2603293536 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1540035d0 con 0x7fa16c0ccec0 2023-10-11T09:53:16.415 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.406+0000 7fa16b7fe700 1 -- 172.21.15.78:0/2603293536 >> v2:172.21.15.133:3300/0 conn(0x7fa16c0cca70 msgr2=0x7fa16c0c1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:53:16.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.406+0000 7fa16b7fe700 1 --2- 172.21.15.78:0/2603293536 >> v2:172.21.15.133:3300/0 conn(0x7fa16c0cca70 0x7fa16c0c1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:53:16.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.406+0000 7fa16b7fe700 1 -- 172.21.15.78:0/2603293536 >> v1:172.21.15.133:6789/0 conn(0x7fa16c0ccec0 legacy=0x7fa16c164ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:53:16.416 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.406+0000 7fa16b7fe700 1 -- 172.21.15.78:0/2603293536 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16c161ae0 con 0x7fa16c160de0 2023-10-11T09:53:16.417 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.406+0000 7fa16affd700 1 --2- 172.21.15.78:0/2603293536 >> v2:172.21.15.133:3300/0 conn(0x7fa16c0cca70 0x7fa16c0c1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T09:53:16.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.406+0000 7fa16b7fe700 1 --2- 172.21.15.78:0/2603293536 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa16c160de0 0x7fa16c0ca670 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa15c009850 tx=0x7fa15c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ceea7c9f4d910957 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:53:16.418 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.406+0000 7fa16a7fc700 1 -- 172.21.15.78:0/2603293536 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c011830 con 0x7fa16c160de0 2023-10-11T09:53:16.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.406+0000 7fa16a7fc700 1 -- 172.21.15.78:0/2603293536 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15c011990 con 0x7fa16c160de0 2023-10-11T09:53:16.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.406+0000 7fa16a7fc700 1 -- 172.21.15.78:0/2603293536 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c011d40 con 0x7fa16c160de0 2023-10-11T09:53:16.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.406+0000 7fa171eee700 1 -- 172.21.15.78:0/2603293536 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa16c160de0 msgr2=0x7fa16c0ca670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:53:16.419 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.406+0000 7fa171eee700 1 --2- 172.21.15.78:0/2603293536 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa16c160de0 0x7fa16c0ca670 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa15c009850 tx=0x7fa15c009b60 comp rx=0 tx=0).stop 2023-10-11T09:53:16.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.406+0000 7fa171eee700 1 -- 172.21.15.78:0/2603293536 shutdown_connections 2023-10-11T09:53:16.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.406+0000 7fa171eee700 1 --2- 172.21.15.78:0/2603293536 >> v2:172.21.15.133:3300/0 conn(0x7fa16c0cca70 0x7fa16c0c1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:53:16.420 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.406+0000 7fa171eee700 1 --2- 172.21.15.78:0/2603293536 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa16c160de0 0x7fa16c0ca670 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:53:16.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.406+0000 7fa171eee700 1 -- 172.21.15.78:0/2603293536 >> 172.21.15.78:0/2603293536 conn(0x7fa16c0cb920 msgr2=0x7fa16c0cbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:53:16.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.410+0000 7fa171eee700 1 -- 172.21.15.78:0/2603293536 shutdown_connections 2023-10-11T09:53:16.421 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.410+0000 7fa171eee700 1 -- 172.21.15.78:0/2603293536 wait complete. 2023-10-11T09:53:16.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.410+0000 7fa171eee700 1 Processor -- start 2023-10-11T09:53:16.423 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.410+0000 7fa171eee700 1 -- start start 2023-10-11T09:53:16.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.414+0000 7fa171eee700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa16c0cca70 0x7fa16c0c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:53:16.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.414+0000 7fa171eee700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa16c0ccec0 0x7fa16c0afee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:53:16.424 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.414+0000 7fa171eee700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa16c160de0 0x7fa16c155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:53:16.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.414+0000 7fa171eee700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c0c8010 con 0x7fa16c0cca70 2023-10-11T09:53:16.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.414+0000 7fa171eee700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c0b0420 con 0x7fa16c0ccec0 2023-10-11T09:53:16.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.414+0000 7fa171eee700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c0b0630 con 0x7fa16c160de0 2023-10-11T09:53:16.425 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.414+0000 7fa16b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa16c0cca70 0x7fa16c0c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:53:16.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.414+0000 7fa16b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa16c0cca70 0x7fa16c0c7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39024/0 (socket says 172.21.15.78:39024) 2023-10-11T09:53:16.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.414+0000 7fa16b7fe700 1 -- 172.21.15.78:0/985808436 learned_addr learned my addr 172.21.15.78:0/985808436 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:53:16.426 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.414+0000 7fa16affd700 1 --2- 172.21.15.78:0/985808436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa16c0ccec0 0x7fa16c0afee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:53:16.427 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.414+0000 7fa16bfff700 1 --2- 172.21.15.78:0/985808436 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa16c160de0 0x7fa16c155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:53:16.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.414+0000 7fa16b7fe700 1 -- 172.21.15.78:0/985808436 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa16c160de0 msgr2=0x7fa16c155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:53:16.428 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.414+0000 7fa16b7fe700 1 --2- 172.21.15.78:0/985808436 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa16c160de0 0x7fa16c155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:53:16.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.414+0000 7fa16b7fe700 1 -- 172.21.15.78:0/985808436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa16c0ccec0 msgr2=0x7fa16c0afee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:53:16.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.414+0000 7fa16b7fe700 1 --2- 172.21.15.78:0/985808436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa16c0ccec0 0x7fa16c0afee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:53:16.429 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.414+0000 7fa16b7fe700 1 -- 172.21.15.78:0/985808436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1600094b0 con 0x7fa16c0cca70 2023-10-11T09:53:16.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.414+0000 7fa16b7fe700 1 --2- 172.21.15.78:0/985808436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa16c0cca70 0x7fa16c0c7ad0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fa15c014040 tx=0x7fa15c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:53:16.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.418+0000 7fa153fff700 1 -- 172.21.15.78:0/985808436 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c004510 con 0x7fa16c0cca70 2023-10-11T09:53:16.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.418+0000 7fa153fff700 1 -- 172.21.15.78:0/985808436 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15c01e070 con 0x7fa16c0cca70 2023-10-11T09:53:16.430 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.418+0000 7fa153fff700 1 -- 172.21.15.78:0/985808436 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c003900 con 0x7fa16c0cca70 2023-10-11T09:53:16.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.418+0000 7fa171eee700 1 -- 172.21.15.78:0/985808436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa15c0094b0 con 0x7fa16c0cca70 2023-10-11T09:53:16.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.418+0000 7fa171eee700 1 -- 172.21.15.78:0/985808436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa16c161330 con 0x7fa16c0cca70 2023-10-11T09:53:16.431 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.418+0000 7fa171eee700 1 -- 172.21.15.78:0/985808436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa16c0cb720 con 0x7fa16c0cca70 2023-10-11T09:53:16.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.422+0000 7fa153fff700 1 -- 172.21.15.78:0/985808436 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fa15c003a60 con 0x7fa16c0cca70 2023-10-11T09:53:16.432 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.422+0000 7fa153fff700 1 --2- 172.21.15.78:0/985808436 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa158065ed0 0x7fa158068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:53:16.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.422+0000 7fa153fff700 1 -- 172.21.15.78:0/985808436 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fa15c020070 con 0x7fa16c0cca70 2023-10-11T09:53:16.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.422+0000 7fa16affd700 1 --2- 172.21.15.78:0/985808436 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa158065ed0 0x7fa158068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:53:16.433 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.422+0000 7fa16affd700 1 --2- 172.21.15.78:0/985808436 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa158065ed0 0x7fa158068380 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fa16c1f9990 tx=0x7fa160009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:53:16.436 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.430+0000 7fa153fff700 1 -- 172.21.15.78:0/985808436 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fa15c0521e0 con 0x7fa16c0cca70 2023-10-11T09:53:16.612 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.602+0000 7fa171eee700 1 -- 172.21.15.78:0/985808436 --> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa16c0ca500 con 0x7fa158065ed0 2023-10-11T09:53:16.614 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.606+0000 7fa153fff700 1 -- 172.21.15.78:0/985808436 <== mgr.4101 v2:172.21.15.78:6832/35606 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70720 (secure 0 0 0) 0x7fa16c0ca500 con 0x7fa158065ed0 2023-10-11T09:53:16.616 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:53:16.617 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.610+0000 7fa171eee700 1 -- 172.21.15.78:0/985808436 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa158065ed0 msgr2=0x7fa158068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:53:16.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.610+0000 7fa171eee700 1 --2- 172.21.15.78:0/985808436 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa158065ed0 0x7fa158068380 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fa16c1f9990 tx=0x7fa160009240 comp rx=0 tx=0).stop 2023-10-11T09:53:16.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.610+0000 7fa171eee700 1 -- 172.21.15.78:0/985808436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa16c0cca70 msgr2=0x7fa16c0c7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:53:16.618 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.610+0000 7fa171eee700 1 --2- 172.21.15.78:0/985808436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa16c0cca70 0x7fa16c0c7ad0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fa15c014040 tx=0x7fa15c004770 comp rx=0 tx=0).stop 2023-10-11T09:53:16.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.610+0000 7fa171eee700 1 -- 172.21.15.78:0/985808436 shutdown_connections 2023-10-11T09:53:16.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.610+0000 7fa171eee700 1 --2- 172.21.15.78:0/985808436 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fa158065ed0 0x7fa158068380 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:53:16.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.610+0000 7fa171eee700 1 --2- 172.21.15.78:0/985808436 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa16c160de0 0x7fa16c155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:53:16.619 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.610+0000 7fa171eee700 1 --2- 172.21.15.78:0/985808436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa16c0ccec0 0x7fa16c0afee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:53:16.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.610+0000 7fa171eee700 1 --2- 172.21.15.78:0/985808436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa16c0cca70 0x7fa16c0c7ad0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:53:16.620 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.610+0000 7fa171eee700 1 -- 172.21.15.78:0/985808436 >> 172.21.15.78:0/985808436 conn(0x7fa16c0cb920 msgr2=0x7fa16c160430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:53:16.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.610+0000 7fa171eee700 1 -- 172.21.15.78:0/985808436 shutdown_connections 2023-10-11T09:53:16.621 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.610+0000 7fa171eee700 1 -- 172.21.15.78:0/985808436 wait complete. 2023-10-11T09:53:16.621 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2023-10-11T09:53:16.643 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":509,"stamp":"2023-10-11T09:53:16.594462+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":262408,"kb_used_data":14024,"kb_used_omap":0,"kb_used_meta":248320,"kb_avail":754712312,"statfs":{"total":773094113280,"available":772825407488,"internally_reserved":0,"allocated":14360576,"data_stored":7583432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":254279680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003945"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":111,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-11T09:52:57.562286+0000","last_change":"2023-10-11T09:52:57.562500+0000","last_active":"2023-10-11T09:52:57.562286+0000","last_peered":"2023-10-11T09:52:57.562286+0000","last_clean":"2023-10-11T09:52:57.562286+0000","last_became_active":"2023-10-11T09:40:36.151179+0000","last_became_peered":"2023-10-11T09:40:36.151179+0000","last_unstale":"2023-10-11T09:52:57.562286+0000","last_undegraded":"2023-10-11T09:52:57.562286+0000","last_fullsized":"2023-10-11T09:52:57.562286+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:52:57.562249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:52:57.562249+0000","last_clean_scrub_stamp":"2023-10-11T09:52:57.562249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:12:44.684559+0000","scrub_duration":0.0069995400000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":118,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-11T09:52:53.076395+0000","last_change":"2023-10-11T09:52:53.076569+0000","last_active":"2023-10-11T09:52:53.076395+0000","last_peered":"2023-10-11T09:52:53.076395+0000","last_clean":"2023-10-11T09:52:53.076395+0000","last_became_active":"2023-10-11T09:40:36.149919+0000","last_became_peered":"2023-10-11T09:40:36.149919+0000","last_unstale":"2023-10-11T09:52:53.076395+0000","last_undegraded":"2023-10-11T09:52:53.076395+0000","last_fullsized":"2023-10-11T09:52:53.076395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:52:53.076309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:52:53.076309+0000","last_clean_scrub_stamp":"2023-10-11T09:52:53.076309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:42:25.977603+0000","scrub_duration":0.0081017080000000009,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":141,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-11T09:53:09.356391+0000","last_change":"2023-10-11T09:53:09.356509+0000","last_active":"2023-10-11T09:53:09.356391+0000","last_peered":"2023-10-11T09:53:09.356391+0000","last_clean":"2023-10-11T09:53:09.356391+0000","last_became_active":"2023-10-11T09:40:36.161710+0000","last_became_peered":"2023-10-11T09:40:36.161710+0000","last_unstale":"2023-10-11T09:53:09.356391+0000","last_undegraded":"2023-10-11T09:53:09.356391+0000","last_fullsized":"2023-10-11T09:53:09.356391+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:53:09.356371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:53:09.356371+0000","last_clean_scrub_stamp":"2023-10-11T09:53:09.356371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:35:42.138761+0000","scrub_duration":0.0082302790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":119,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-11T09:52:55.050324+0000","last_change":"2023-10-11T09:52:55.050324+0000","last_active":"2023-10-11T09:52:55.050324+0000","last_peered":"2023-10-11T09:52:55.050324+0000","last_clean":"2023-10-11T09:52:55.050324+0000","last_became_active":"2023-10-11T09:40:36.160171+0000","last_became_peered":"2023-10-11T09:40:36.160171+0000","last_unstale":"2023-10-11T09:52:55.050324+0000","last_undegraded":"2023-10-11T09:52:55.050324+0000","last_fullsized":"2023-10-11T09:52:55.050324+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:52:55.050154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:52:55.050154+0000","last_clean_scrub_stamp":"2023-10-11T09:52:55.050154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:19:21.405813+0000","scrub_duration":0.0060079240000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":117,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-11T09:52:56.935558+0000","last_change":"2023-10-11T09:52:56.935558+0000","last_active":"2023-10-11T09:52:56.935558+0000","last_peered":"2023-10-11T09:52:56.935558+0000","last_clean":"2023-10-11T09:52:56.935558+0000","last_became_active":"2023-10-11T09:40:36.151219+0000","last_became_peered":"2023-10-11T09:40:36.151219+0000","last_unstale":"2023-10-11T09:52:56.935558+0000","last_undegraded":"2023-10-11T09:52:56.935558+0000","last_fullsized":"2023-10-11T09:52:56.935558+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:52:56.935288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:52:56.935288+0000","last_clean_scrub_stamp":"2023-10-11T09:52:56.935288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T21:27:36.244079+0000","scrub_duration":0.0081707749999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":119,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-11T09:52:54.765623+0000","last_change":"2023-10-11T09:52:54.765623+0000","last_active":"2023-10-11T09:52:54.765623+0000","last_peered":"2023-10-11T09:52:54.765623+0000","last_clean":"2023-10-11T09:52:54.765623+0000","last_became_active":"2023-10-11T09:40:36.215890+0000","last_became_peered":"2023-10-11T09:40:36.215890+0000","last_unstale":"2023-10-11T09:52:54.765623+0000","last_undegraded":"2023-10-11T09:52:54.765623+0000","last_fullsized":"2023-10-11T09:52:54.765623+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:52:54.765450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:52:54.765450+0000","last_clean_scrub_stamp":"2023-10-11T09:52:54.765450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:44:53.321731+0000","scrub_duration":0.0072502469999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":141,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-11T09:53:02.203059+0000","last_change":"2023-10-11T09:53:02.203309+0000","last_active":"2023-10-11T09:53:02.203059+0000","last_peered":"2023-10-11T09:53:02.203059+0000","last_clean":"2023-10-11T09:53:02.203059+0000","last_became_active":"2023-10-11T09:40:36.149419+0000","last_became_peered":"2023-10-11T09:40:36.149419+0000","last_unstale":"2023-10-11T09:53:02.203059+0000","last_undegraded":"2023-10-11T09:53:02.203059+0000","last_fullsized":"2023-10-11T09:53:02.203059+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:53:02.203012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:53:02.203012+0000","last_clean_scrub_stamp":"2023-10-11T09:53:02.203012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:15:33.240987+0000","scrub_duration":0.0070770010000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":117,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-11T09:52:58.946246+0000","last_change":"2023-10-11T09:52:58.946246+0000","last_active":"2023-10-11T09:52:58.946246+0000","last_peered":"2023-10-11T09:52:58.946246+0000","last_clean":"2023-10-11T09:52:58.946246+0000","last_became_active":"2023-10-11T09:40:36.148489+0000","last_became_peered":"2023-10-11T09:40:36.148489+0000","last_unstale":"2023-10-11T09:52:58.946246+0000","last_undegraded":"2023-10-11T09:52:58.946246+0000","last_fullsized":"2023-10-11T09:52:58.946246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T09:52:58.946011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T09:52:58.946011+0000","last_clean_scrub_stamp":"2023-10-11T09:52:58.946011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T19:37:19.194925+0000","scrub_duration":0.0059796500000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"132'119","reported_seq":296,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-11T09:52:59.194851+0000","last_change":"2023-10-11T09:52:59.194964+0000","last_active":"2023-10-11T09:52:59.194851+0000","last_peered":"2023-10-11T09:52:59.194851+0000","last_clean":"2023-10-11T09:52:59.194851+0000","last_became_active":"2023-10-11T09:40:29.873621+0000","last_became_peered":"2023-10-11T09:40:29.873621+0000","last_unstale":"2023-10-11T09:52:59.194851+0000","last_undegraded":"2023-10-11T09:52:59.194851+0000","last_fullsized":"2023-10-11T09:52:59.194851+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'119","last_scrub_stamp":"2023-10-11T09:52:59.194826+0000","last_deep_scrub":"132'119","last_deep_scrub_stamp":"2023-10-11T09:52:59.194826+0000","last_clean_scrub_stamp":"2023-10-11T09:52:59.194826+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T20:00:46.133294+0000","scrub_duration":0.013344972,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575005,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33236,"kb_used_data":2188,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338604,"statfs":{"total":96636764160,"available":96602730496,"internally_reserved":0,"allocated":2240512,"data_stored":1390705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:52:57 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.0900000000000001,"15min":1.0780000000000001},"min":{"1min":0.70199999999999996,"5min":0.622,"15min":0.48999999999999999},"max":{"1min":1.409,"5min":1.7390000000000001,"15min":1.903},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.0800000000000001,"15min":1.083},"min":{"1min":0.77700000000000002,"5min":0.54700000000000004,"15min":0.50900000000000001},"max":{"1min":1.595,"5min":1.595,"15min":1.696},"last":1.2410000000000001}]},{"osd":1,"last update":"Wed Oct 11 09:52:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.135,"15min":1.119},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.63100000000000001},"max":{"1min":1.4359999999999999,"5min":1.645,"15min":2.25},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.149,"5min":1.101,"15min":1.0920000000000001},"min":{"1min":0.83899999999999997,"5min":0.68000000000000005,"15min":0.57199999999999995},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.056}]},{"osd":2,"last update":"Wed Oct 11 09:52:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.109,"15min":1.103},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.58999999999999997},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.873},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.157,"5min":1.1910000000000001,"15min":1.1539999999999999},"min":{"1min":0.79700000000000004,"5min":0.66400000000000003,"15min":0.65500000000000003},"max":{"1min":1.5900000000000001,"5min":1.78,"15min":1.78},"last":1.1870000000000001}]},{"osd":3,"last update":"Wed Oct 11 09:52:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2270000000000001,"15min":1.214},"min":{"1min":0.751,"5min":0.751,"15min":0.64900000000000002},"max":{"1min":1.877,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.397},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.3009999999999999,"15min":1.2789999999999999},"min":{"1min":0.76600000000000001,"5min":0.747,"15min":0.61699999999999999},"max":{"1min":1.5860000000000001,"5min":1.8720000000000001,"15min":2.1579999999999999},"last":1.2889999999999999}]},{"osd":4,"last update":"Wed Oct 11 09:52:57 2023","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.0600000000000001,"15min":1.1060000000000001},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.53300000000000003},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":2.2010000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.169,"5min":1.208,"15min":1.218},"min":{"1min":0.81100000000000005,"5min":0.67200000000000004,"15min":0.63500000000000001},"max":{"1min":2.1890000000000001,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":0.81499999999999995}]},{"osd":5,"last update":"Wed Oct 11 09:52:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.256,"15min":1.2709999999999999},"min":{"1min":0.78100000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.5149999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.2050000000000001,"15min":1.256},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.65900000000000003},"max":{"1min":1.474,"5min":1.9450000000000001,"15min":3.0750000000000002},"last":1.206}]},{"osd":6,"last update":"Wed Oct 11 09:52:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.339,"15min":1.3779999999999999},"min":{"1min":0.92900000000000005,"5min":0.77000000000000002,"15min":0.70999999999999996},"max":{"1min":1.5820000000000001,"5min":2,"15min":3.137},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.375,"5min":1.3680000000000001,"15min":1.377},"min":{"1min":0.95499999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.2360000000000002,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.94299999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575004,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1608,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1646592,"data_stored":800337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.125,"15min":1.036},"min":{"1min":0.626,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.403,"5min":7.0730000000000004,"15min":7.0730000000000004},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.01,"5min":1.179,"15min":1.071},"min":{"1min":0.65900000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.357,"5min":8.8450000000000006,"15min":8.8450000000000006},"last":1.478}]},{"osd":1,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.359,"15min":1.232},"min":{"1min":0.627,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.5149999999999999,"5min":15.523,"15min":15.523},"last":1.264},{"interface":"front","average":{"1min":1.181,"5min":1.3069999999999999,"15min":1.2310000000000001},"min":{"1min":0.80700000000000005,"5min":0.59199999999999997,"15min":0.53400000000000003},"max":{"1min":1.5509999999999999,"5min":15.253,"15min":15.253},"last":1.2190000000000001}]},{"osd":2,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.1200000000000001,"15min":1.1319999999999999},"min":{"1min":0.745,"5min":0.434,"15min":0.434},"max":{"1min":1.335,"5min":2.7919999999999998,"15min":2.7919999999999998},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1930000000000001,"15min":1.1890000000000001},"min":{"1min":0.79400000000000004,"5min":0.61599999999999999,"15min":0.44500000000000001},"max":{"1min":1.6060000000000001,"5min":5.0990000000000002,"15min":5.0990000000000002},"last":1.1160000000000001}]},{"osd":3,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.2150000000000001,"15min":1.155},"min":{"1min":0.72599999999999998,"5min":0.70599999999999996,"15min":0.627},"max":{"1min":1.421,"5min":11.945,"15min":11.945},"last":1.5740000000000001},{"interface":"front","average":{"1min":1.246,"5min":1.175,"15min":1.2010000000000001},"min":{"1min":0.84299999999999997,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.659,"5min":1.659,"15min":3.0819999999999999},"last":1.502}]},{"osd":4,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.2889999999999999,"15min":1.2769999999999999},"min":{"1min":0.626,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.6359999999999999,"5min":7.5599999999999996,"15min":7.5599999999999996},"last":1.3200000000000001},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.3380000000000001,"15min":1.3069999999999999},"min":{"1min":0.81299999999999994,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.542,"5min":8.7520000000000007,"15min":8.7520000000000007},"last":1.7210000000000001}]},{"osd":5,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.196,"15min":1.2290000000000001},"min":{"1min":0.65900000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.585,"5min":2.1560000000000001,"15min":2.4009999999999998},"last":1.373},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.282,"15min":1.3200000000000001},"min":{"1min":0.68700000000000006,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.978,"5min":2.4390000000000001,"15min":2.7290000000000001},"last":1.397}]},{"osd":7,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.335,"15min":1.3440000000000001},"min":{"1min":0.91400000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.706,"5min":8.7970000000000006,"15min":8.7970000000000006},"last":1.649},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.3899999999999999,"15min":1.4199999999999999},"min":{"1min":1.01,"5min":0.625,"15min":0.625},"max":{"1min":1.8620000000000001,"5min":2.641,"15min":2.641},"last":1.55}]}]},{"osd":5,"up_from":13,"seq":55834575004,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1608,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1646592,"data_stored":800337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.044,"15min":1.0129999999999999},"min":{"1min":0.69799999999999995,"5min":0.64100000000000001,"15min":0.45300000000000001},"max":{"1min":1.4770000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.012},{"interface":"front","average":{"1min":1.125,"5min":1.087,"15min":1.0269999999999999},"min":{"1min":0.66200000000000003,"5min":0.47699999999999998,"15min":0.46500000000000002},"max":{"1min":1.3460000000000001,"5min":1.536,"15min":1.7649999999999999},"last":1.147}]},{"osd":1,"last update":"Wed Oct 11 09:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.1160000000000001,"15min":1.034},"min":{"1min":0.69599999999999995,"5min":0.68100000000000005,"15min":0.49399999999999999},"max":{"1min":1.542,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.0900000000000001},"min":{"1min":0.71599999999999997,"5min":0.61599999999999999,"15min":0.437},"max":{"1min":1.4299999999999999,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.0960000000000001}]},{"osd":2,"last update":"Wed Oct 11 09:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1819999999999999,"15min":1.179},"min":{"1min":0.76100000000000001,"5min":0.746,"15min":0.73899999999999999},"max":{"1min":1.5009999999999999,"5min":2.133,"15min":2.133},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2370000000000001,"15min":1.2190000000000001},"min":{"1min":0.94099999999999995,"5min":0.64200000000000002,"15min":0.52300000000000002},"max":{"1min":1.46,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.2969999999999999}]},{"osd":3,"last update":"Wed Oct 11 09:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.80700000000000005,"5min":0.752,"15min":0.56999999999999995},"max":{"1min":1.3580000000000001,"5min":1.8859999999999999,"15min":1.921},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.1950000000000001,"15min":1.1659999999999999},"min":{"1min":0.92500000000000004,"5min":0.71999999999999997,"15min":0.55300000000000005},"max":{"1min":1.492,"5min":1.9690000000000001,"15min":2.0209999999999999},"last":1.204}]},{"osd":4,"last update":"Wed Oct 11 09:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.1919999999999999,"15min":1.1859999999999999},"min":{"1min":0.61399999999999999,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.4139999999999999,"5min":2.0169999999999999,"15min":2.117},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.075,"5min":1.157,"15min":1.1759999999999999},"min":{"1min":0.81999999999999995,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.4159999999999999,"5min":1.9690000000000001,"15min":2.1749999999999998},"last":0.92100000000000004}]},{"osd":6,"last update":"Wed Oct 11 09:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.228,"15min":1.25},"min":{"1min":0.75700000000000001,"5min":0.69499999999999995,"15min":0.59499999999999997},"max":{"1min":1.5449999999999999,"5min":1.9239999999999999,"15min":2.4529999999999998},"last":0.879},{"interface":"front","average":{"1min":1.252,"5min":1.331,"15min":1.3200000000000001},"min":{"1min":0.78700000000000003,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.478,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.1659999999999999}]},{"osd":7,"last update":"Wed Oct 11 09:52:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.3320000000000001,"15min":1.341},"min":{"1min":1.0069999999999999,"5min":0.58799999999999997,"15min":0.57699999999999996},"max":{"1min":1.502,"5min":2.347,"15min":3.6120000000000001},"last":1.073},{"interface":"front","average":{"1min":1.353,"5min":1.4319999999999999,"15min":1.427},"min":{"1min":0.98499999999999999,"5min":0.89100000000000001,"15min":0.71299999999999997},"max":{"1min":1.589,"5min":2.2679999999999998,"15min":3.2109999999999999},"last":1.2569999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575004,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32656,"kb_used_data":1608,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1646592,"data_stored":800337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.103,"15min":1.0660000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.54400000000000004},"max":{"1min":1.673,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.107,"15min":1.089},"min":{"1min":0.63800000000000001,"5min":0.60199999999999998,"15min":0.54700000000000004},"max":{"1min":1.6399999999999999,"5min":1.669,"15min":2.1160000000000001},"last":1.397}]},{"osd":1,"last update":"Wed Oct 11 09:52:59 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":1.1140000000000001,"15min":1.0820000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.60699999999999998},"max":{"1min":1.425,"5min":1.95,"15min":1.95},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.133,"5min":1.1679999999999999,"15min":1.1459999999999999},"min":{"1min":0.81000000000000005,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.417,"5min":2.02,"15min":2.02},"last":1.5089999999999999}]},{"osd":2,"last update":"Wed Oct 11 09:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.242,"15min":1.2},"min":{"1min":0.88700000000000001,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.391,"5min":2.5859999999999999,"15min":2.5859999999999999},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.284,"5min":1.3,"15min":1.2310000000000001},"min":{"1min":0.99199999999999999,"5min":0.69699999999999995,"15min":0.63900000000000001},"max":{"1min":1.5920000000000001,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.446}]},{"osd":3,"last update":"Wed Oct 11 09:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.1319999999999999,"15min":1.1240000000000001},"min":{"1min":0.75,"5min":0.72599999999999998,"15min":0.60199999999999998},"max":{"1min":1.548,"5min":1.6839999999999999,"15min":1.6930000000000001},"last":1.262},{"interface":"front","average":{"1min":1.155,"5min":1.202,"15min":1.222},"min":{"1min":0.79200000000000004,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.488,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":1.0309999999999999}]},{"osd":5,"last update":"Wed Oct 11 09:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.181,"15min":1.1839999999999999},"min":{"1min":0.79300000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.7609999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":1.5529999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.161,"15min":1.175},"min":{"1min":0.66000000000000003,"5min":0.63300000000000001,"15min":0.61199999999999999},"max":{"1min":1.7909999999999999,"5min":1.7909999999999999,"15min":1.8520000000000001},"last":1.1559999999999999}]},{"osd":6,"last update":"Wed Oct 11 09:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.2370000000000001,"15min":1.244},"min":{"1min":0.76200000000000001,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.5580000000000001,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.29,"5min":1.3120000000000001,"15min":1.3129999999999999},"min":{"1min":0.84899999999999998,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.839,"5min":2.0830000000000002,"15min":2.1120000000000001},"last":1.6020000000000001}]},{"osd":7,"last update":"Wed Oct 11 09:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.2729999999999999,"15min":1.292},"min":{"1min":0.95599999999999996,"5min":0.69799999999999995,"15min":0.60899999999999999},"max":{"1min":1.982,"5min":1.982,"15min":1.982},"last":1.5820000000000001},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.363,"15min":1.3939999999999999},"min":{"1min":0.84799999999999998,"5min":0.79400000000000004,"15min":0.75900000000000001},"max":{"1min":1.5680000000000001,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.3540000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575003,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1608,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1646592,"data_stored":800337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.79500000000000004,"15min":0.79100000000000004},"min":{"1min":0.63500000000000001,"5min":0.442,"15min":0.33000000000000002},"max":{"1min":1.1220000000000001,"5min":1.399,"15min":1.462},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.84399999999999997,"15min":0.82799999999999996},"min":{"1min":0.51900000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.331,"5min":1.363,"15min":1.4950000000000001},"last":0.98799999999999999}]},{"osd":1,"last update":"Wed Oct 11 09:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.84299999999999997,"15min":0.81499999999999995},"min":{"1min":0.60899999999999999,"5min":0.47799999999999998,"15min":0.42699999999999999},"max":{"1min":1.2250000000000001,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.94299999999999995,"15min":0.92400000000000004},"min":{"1min":0.70799999999999996,"5min":0.53100000000000003,"15min":0.49099999999999999},"max":{"1min":1.4119999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.80600000000000005}]},{"osd":2,"last update":"Wed Oct 11 09:52:49 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.98499999999999999,"15min":0.96599999999999997},"min":{"1min":0.63400000000000001,"5min":0.57699999999999996,"15min":0.52900000000000003},"max":{"1min":1.3460000000000001,"5min":1.601,"15min":1.601},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.99299999999999999,"5min":1.042,"15min":1.044},"min":{"1min":0.67400000000000004,"5min":0.63700000000000001,"15min":0.432},"max":{"1min":1.2669999999999999,"5min":1.607,"15min":2.5870000000000002},"last":0.89200000000000002}]},{"osd":4,"last update":"Wed Oct 11 09:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1319999999999999,"15min":1.151},"min":{"1min":0.88100000000000001,"5min":0.77900000000000003,"15min":0.68100000000000005},"max":{"1min":1.538,"5min":1.556,"15min":2.395},"last":1.198},{"interface":"front","average":{"1min":1.226,"5min":1.1919999999999999,"15min":1.2270000000000001},"min":{"1min":0.98799999999999999,"5min":0.747,"15min":0.73599999999999999},"max":{"1min":1.833,"5min":1.9099999999999999,"15min":3.1579999999999999},"last":1.121}]},{"osd":5,"last update":"Wed Oct 11 09:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.2270000000000001,"15min":1.256},"min":{"1min":0.94599999999999995,"5min":0.84699999999999998,"15min":0.76500000000000001},"max":{"1min":1.6990000000000001,"5min":1.724,"15min":3.0009999999999999},"last":1.339},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.266,"15min":1.3089999999999999},"min":{"1min":1.107,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":3.8050000000000002},"last":1.046}]},{"osd":6,"last update":"Wed Oct 11 09:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.2190000000000001,"15min":1.24},"min":{"1min":0.98499999999999999,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.659,"5min":1.6890000000000001,"15min":2.1949999999999998},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.292,"5min":1.2829999999999999,"15min":1.3169999999999999},"min":{"1min":0.96399999999999997,"5min":0.91700000000000004,"15min":0.84499999999999997},"max":{"1min":1.782,"5min":1.782,"15min":3.2549999999999999},"last":1.0249999999999999}]},{"osd":7,"last update":"Wed Oct 11 09:52:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3029999999999999,"15min":1.351},"min":{"1min":1.085,"5min":0.93700000000000006,"15min":0.76700000000000002},"max":{"1min":1.637,"5min":1.7150000000000001,"15min":3.7490000000000001},"last":1.109},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.294,"15min":1.3140000000000001},"min":{"1min":0.90600000000000003,"5min":0.83199999999999996,"15min":0.81799999999999995},"max":{"1min":1.619,"5min":1.7969999999999999,"15min":2.625},"last":1.145}]}]},{"osd":2,"up_from":13,"seq":55834575005,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1608,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1646592,"data_stored":800337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:52:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.81799999999999995,"15min":0.80600000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.38500000000000001},"max":{"1min":1.1359999999999999,"5min":1.8180000000000001,"15min":2.0600000000000001},"last":1.167},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.88900000000000001,"15min":0.88400000000000001},"min":{"1min":0.48899999999999999,"5min":0.47299999999999998,"15min":0.33100000000000002},"max":{"1min":1.202,"5min":2.2850000000000001,"15min":2.8919999999999999},"last":0.84199999999999997}]},{"osd":1,"last update":"Wed Oct 11 09:52:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.82899999999999996,"15min":0.90900000000000003},"min":{"1min":0.50800000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.0229999999999999,"5min":1.8069999999999999,"15min":3.7650000000000001},"last":1.1100000000000001},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.97499999999999998,"15min":0.96699999999999997},"min":{"1min":0.59399999999999997,"5min":0.45600000000000002,"15min":0.40300000000000002},"max":{"1min":1.3460000000000001,"5min":1.9099999999999999,"15min":2.5339999999999998},"last":1.2450000000000001}]},{"osd":3,"last update":"Wed Oct 11 09:52:48 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.91500000000000004,"15min":0.98599999999999999},"min":{"1min":0.61899999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.167,"5min":1.591,"15min":2.403},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0640000000000001,"15min":1.0389999999999999},"min":{"1min":0.68100000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.351,"5min":1.754,"15min":2.7589999999999999},"last":0.88700000000000001}]},{"osd":4,"last update":"Wed Oct 11 09:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.1639999999999999,"15min":1.1779999999999999},"min":{"1min":0.69999999999999996,"5min":0.68600000000000005,"15min":0.65000000000000002},"max":{"1min":1.5349999999999999,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.242,"15min":1.3069999999999999},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73399999999999999},"max":{"1min":1.4630000000000001,"5min":2.1669999999999998,"15min":2.6859999999999999},"last":1.1990000000000001}]},{"osd":5,"last update":"Wed Oct 11 09:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.26,"15min":1.2829999999999999},"min":{"1min":0.83099999999999996,"5min":0.80500000000000005,"15min":0.77400000000000002},"max":{"1min":1.5800000000000001,"5min":2.0179999999999998,"15min":3.044},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.167,"15min":1.23},"min":{"1min":0.746,"5min":0.746,"15min":0.69599999999999995},"max":{"1min":1.405,"5min":2.411,"15min":3.5790000000000002},"last":1.0289999999999999}]},{"osd":6,"last update":"Wed Oct 11 09:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.2050000000000001,"15min":1.282},"min":{"1min":0.77900000000000003,"5min":0.75,"15min":0.75},"max":{"1min":1.512,"5min":1.774,"15min":3.1600000000000001},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.29,"5min":1.3220000000000001,"15min":1.321},"min":{"1min":0.88600000000000001,"5min":0.83499999999999996,"15min":0.623},"max":{"1min":1.6819999999999999,"5min":1.9670000000000001,"15min":2.839},"last":1.327}]},{"osd":7,"last update":"Wed Oct 11 09:52:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.274,"15min":1.325},"min":{"1min":0.879,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.552,"5min":2.7130000000000001,"15min":3.1230000000000002},"last":1.276},{"interface":"front","average":{"1min":1.282,"5min":1.3220000000000001,"15min":1.3999999999999999},"min":{"1min":0.877,"5min":0.84899999999999998,"15min":0.80000000000000004},"max":{"1min":1.6200000000000001,"5min":1.7909999999999999,"15min":2.988},"last":1.3029999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575004,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1608,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1646592,"data_stored":800337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 09:52:50 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.81000000000000005,"15min":0.78500000000000003},"min":{"1min":0.50900000000000001,"5min":0.48799999999999999,"15min":0.375},"max":{"1min":1.2270000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.83299999999999996,"15min":0.82199999999999995},"min":{"1min":0.56699999999999995,"5min":0.40100000000000002,"15min":0.36699999999999999},"max":{"1min":1.181,"5min":1.593,"15min":1.593},"last":0.996}]},{"osd":2,"last update":"Wed Oct 11 09:52:50 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.82599999999999996,"15min":0.83399999999999996},"min":{"1min":0.57799999999999996,"5min":0.51700000000000002,"15min":0.36199999999999999},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.92900000000000005,"15min":0.94299999999999995},"min":{"1min":0.58799999999999997,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.2190000000000001,"5min":1.4159999999999999,"15min":1.6619999999999999},"last":0.80500000000000005}]},{"osd":3,"last update":"Wed Oct 11 09:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.018,"15min":1.012},"min":{"1min":0.63200000000000001,"5min":0.61599999999999999,"15min":0.46600000000000003},"max":{"1min":1.4890000000000001,"5min":1.714,"15min":1.714},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.044,"5min":0.97799999999999998,"15min":0.97299999999999998},"min":{"1min":0.68899999999999995,"5min":0.54200000000000004,"15min":0.38900000000000001},"max":{"1min":1.409,"5min":1.611,"15min":1.611},"last":0.85899999999999999}]},{"osd":4,"last update":"Wed Oct 11 09:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.1100000000000001,"15min":1.1319999999999999},"min":{"1min":0.85199999999999998,"5min":0.77400000000000002,"15min":0.73599999999999999},"max":{"1min":1.3460000000000001,"5min":1.5980000000000001,"15min":2.016},"last":1.141},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.2290000000000001,"15min":1.254},"min":{"1min":1.026,"5min":0.85699999999999998,"15min":0.80000000000000004},"max":{"1min":1.601,"5min":1.748,"15min":2.323},"last":1.3520000000000001}]},{"osd":5,"last update":"Wed Oct 11 09:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.2150000000000001,"15min":1.252},"min":{"1min":1.0089999999999999,"5min":0.79300000000000004,"15min":0.65800000000000003},"max":{"1min":1.524,"5min":1.722,"15min":2.4079999999999999},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.194,"15min":1.2190000000000001},"min":{"1min":1.0600000000000001,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.647,"5min":1.677,"15min":2.1349999999999998},"last":1.238}]},{"osd":6,"last update":"Wed Oct 11 09:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.2569999999999999,"15min":1.294},"min":{"1min":1.01,"5min":0.84599999999999997,"15min":0.71499999999999997},"max":{"1min":1.7170000000000001,"5min":2.4620000000000002,"15min":2.8940000000000001},"last":1.327},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.2310000000000001,"15min":1.2889999999999999},"min":{"1min":0.997,"5min":0.84799999999999998,"15min":0.79200000000000004},"max":{"1min":1.6100000000000001,"5min":1.673,"15min":3.3719999999999999},"last":1.1879999999999999}]},{"osd":7,"last update":"Wed Oct 11 09:52:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3260000000000001,"15min":1.357},"min":{"1min":1.0269999999999999,"5min":0.86899999999999999,"15min":0.86099999999999999},"max":{"1min":1.7929999999999999,"5min":1.7929999999999999,"15min":2.194},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.335,"15min":1.369},"min":{"1min":1.014,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.6160000000000001,"5min":1.857,"15min":1.968},"last":1.2769999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575003,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33236,"kb_used_data":2188,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338604,"statfs":{"total":96636764160,"available":96602730496,"internally_reserved":0,"allocated":2240512,"data_stored":1390705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.83999999999999997,"15min":0.80800000000000005},"min":{"1min":0.65800000000000003,"5min":0.45300000000000001,"15min":0.38500000000000001},"max":{"1min":1.3160000000000001,"5min":1.611,"15min":1.611},"last":1.1899999999999999},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.88200000000000001,"15min":0.89300000000000002},"min":{"1min":0.60399999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":2.3100000000000001},"last":1.2410000000000001}]},{"osd":2,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.95699999999999996,"15min":0.93600000000000005},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.3160000000000001,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.002,"5min":0.95699999999999996,"15min":0.95799999999999996},"min":{"1min":0.56299999999999994,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.306,"5min":1.569,"15min":1.9450000000000001},"last":1.1399999999999999}]},{"osd":3,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.81100000000000005,"15min":0.83199999999999996},"min":{"1min":0.63900000000000001,"5min":0.48999999999999999,"15min":0.45600000000000002},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":2.7639999999999998},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.95099999999999996,"15min":0.97999999999999998},"min":{"1min":0.73799999999999999,"5min":0.51400000000000001,"15min":0.48999999999999999},"max":{"1min":1.3879999999999999,"5min":1.482,"15min":2.1789999999999998},"last":1.266}]},{"osd":4,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.131,"15min":1.163},"min":{"1min":0.79100000000000004,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.4450000000000001,"5min":1.6659999999999999,"15min":1.792},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.149,"5min":1.1379999999999999,"15min":1.161},"min":{"1min":0.94499999999999995,"5min":0.755,"15min":0.69799999999999995},"max":{"1min":1.4039999999999999,"5min":1.9399999999999999,"15min":2.6629999999999998},"last":0.96099999999999997}]},{"osd":5,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.234,"15min":1.2769999999999999},"min":{"1min":0.99099999999999999,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.4810000000000001,"5min":1.829,"15min":3.0150000000000001},"last":1.413},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.161,"15min":1.2430000000000001},"min":{"1min":0.85599999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.4319999999999999,"5min":1.48,"15min":3.601},"last":1.4370000000000001}]},{"osd":6,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.151,"15min":1.1870000000000001},"min":{"1min":0.87,"5min":0.79100000000000004,"15min":0.78000000000000003},"max":{"1min":1.46,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.28,"15min":1.2949999999999999},"min":{"1min":0.98699999999999999,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":1.5269999999999999,"5min":1.7430000000000001,"15min":2.1440000000000001},"last":1.3600000000000001}]},{"osd":7,"last update":"Wed Oct 11 09:52:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2410000000000001,"15min":1.282},"min":{"1min":0.97899999999999998,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.5680000000000001,"5min":1.607,"15min":1.8759999999999999},"last":1.391},{"interface":"front","average":{"1min":1.232,"5min":1.2350000000000001,"15min":1.278},"min":{"1min":1.02,"5min":0.88800000000000001,"15min":0.73899999999999999},"max":{"1min":1.4710000000000001,"5min":2.016,"15min":2.5419999999999998},"last":1.2969999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-11T09:53:16.646 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-11T09:53:16.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.806+0000 7f6fb990c700 1 Processor -- start 2023-10-11T09:53:16.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.806+0000 7f6fb990c700 1 -- start start 2023-10-11T09:53:16.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.806+0000 7f6fb990c700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f6fb406c8a0 0x7f6fb40cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:53:16.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.806+0000 7f6fb990c700 1 -- --> v1:172.21.15.78:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fb41648a0 con 0x7f6fb4160de0 2023-10-11T09:53:16.816 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.806+0000 7f6fb990c700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fb4164b80 con 0x7f6fb40cc230 2023-10-11T09:53:16.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.806+0000 7f6fb990c700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fb40cc6d0 con 0x7f6fb406c8a0 2023-10-11T09:53:16.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.806+0000 7f6fabfff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f6fb406c8a0 0x7f6fb40cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:53:16.817 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.806+0000 7f6fabfff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7f6fb406c8a0 0x7f6fb40cbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39038/0 (socket says 172.21.15.78:39038) 2023-10-11T09:53:16.818 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.806+0000 7f6fabfff700 1 -- 172.21.15.78:0/2582901768 learned_addr learned my addr 172.21.15.78:0/2582901768 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:53:16.818 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.806+0000 7f6fb2ffd700 1 --1- 172.21.15.78:0/2582901768 >> v1:172.21.15.78:6789/0 conn(0x7f6fb4160de0 0x7f6fb406c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.78:6789/0 says I am v1:172.21.15.78:55028/0 (socket says 172.21.15.78:55028) 2023-10-11T09:53:16.819 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.810+0000 7f6fabfff700 1 -- 172.21.15.78:0/2582901768 >> v1:172.21.15.133:6789/0 conn(0x7f6fb40cc230 legacy=0x7f6fb40cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:53:16.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.810+0000 7f6fabfff700 1 -- 172.21.15.78:0/2582901768 >> v1:172.21.15.78:6789/0 conn(0x7f6fb4160de0 legacy=0x7f6fb406c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:53:16.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.810+0000 7f6fabfff700 1 -- 172.21.15.78:0/2582901768 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb4164e60 con 0x7f6fb406c8a0 2023-10-11T09:53:16.820 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.810+0000 7f6fabfff700 1 --2- 172.21.15.78:0/2582901768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fb406c8a0 0x7f6fb40cbcf0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c009910 tx=0x7f6f9c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=308fcd221f8a66f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:53:16.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.810+0000 7f6fb27fc700 1 -- 172.21.15.78:0/2582901768 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c004230 con 0x7f6fb406c8a0 2023-10-11T09:53:16.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.810+0000 7f6fb27fc700 1 -- 172.21.15.78:0/2582901768 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f9c004390 con 0x7f6fb406c8a0 2023-10-11T09:53:16.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.810+0000 7f6fb27fc700 1 -- 172.21.15.78:0/2582901768 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c009e60 con 0x7f6fb406c8a0 2023-10-11T09:53:16.821 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.810+0000 7f6fb990c700 1 -- 172.21.15.78:0/2582901768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fb406c8a0 msgr2=0x7f6fb40cbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:53:16.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.810+0000 7f6fb990c700 1 --2- 172.21.15.78:0/2582901768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fb406c8a0 0x7f6fb40cbcf0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c009910 tx=0x7f6f9c009c20 comp rx=0 tx=0).stop 2023-10-11T09:53:16.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.810+0000 7f6fb990c700 1 -- 172.21.15.78:0/2582901768 shutdown_connections 2023-10-11T09:53:16.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.810+0000 7f6fb990c700 1 --2- 172.21.15.78:0/2582901768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fb406c8a0 0x7f6fb40cbcf0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:53:16.822 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.810+0000 7f6fb990c700 1 -- 172.21.15.78:0/2582901768 >> 172.21.15.78:0/2582901768 conn(0x7f6fb4073100 msgr2=0x7f6fb40734d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:53:16.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.810+0000 7f6fb990c700 1 -- 172.21.15.78:0/2582901768 shutdown_connections 2023-10-11T09:53:16.823 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.810+0000 7f6fb990c700 1 -- 172.21.15.78:0/2582901768 wait complete. 2023-10-11T09:53:16.824 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.814+0000 7f6fb990c700 1 Processor -- start 2023-10-11T09:53:16.824 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.814+0000 7f6fb990c700 1 -- start start 2023-10-11T09:53:16.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.814+0000 7f6fb990c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fb406c8a0 0x7f6fb40c5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:53:16.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.814+0000 7f6fb990c700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6fb40cc230 0x7f6fb4163580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:53:16.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.814+0000 7f6fb990c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fb4160de0 0x7f6fb40b4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:53:16.825 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.814+0000 7f6fb990c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb4162200 con 0x7f6fb406c8a0 2023-10-11T09:53:16.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.814+0000 7f6fb990c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb41624e0 con 0x7f6fb4160de0 2023-10-11T09:53:16.826 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.814+0000 7f6fb990c700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb41627c0 con 0x7f6fb40cc230 2023-10-11T09:53:16.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.818+0000 7f6fb2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fb406c8a0 0x7f6fb40c5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:53:16.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.818+0000 7f6fb2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fb406c8a0 0x7f6fb40c5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39042/0 (socket says 172.21.15.78:39042) 2023-10-11T09:53:16.828 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.818+0000 7f6fb2ffd700 1 -- 172.21.15.78:0/2735209413 learned_addr learned my addr 172.21.15.78:0/2735209413 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:53:16.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.818+0000 7f6fabfff700 1 --2- 172.21.15.78:0/2735209413 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6fb40cc230 0x7f6fb4163580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:53:16.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.818+0000 7f6fb37fe700 1 --2- 172.21.15.78:0/2735209413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fb4160de0 0x7f6fb40b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:53:16.829 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.818+0000 7f6fb2ffd700 1 -- 172.21.15.78:0/2735209413 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6fb40cc230 msgr2=0x7f6fb4163580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:53:16.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.818+0000 7f6fb2ffd700 1 --2- 172.21.15.78:0/2735209413 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6fb40cc230 0x7f6fb4163580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:53:16.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.818+0000 7f6fb2ffd700 1 -- 172.21.15.78:0/2735209413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fb4160de0 msgr2=0x7f6fb40b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:53:16.830 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.818+0000 7f6fb2ffd700 1 --2- 172.21.15.78:0/2735209413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fb4160de0 0x7f6fb40b4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:53:16.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.818+0000 7f6fb2ffd700 1 -- 172.21.15.78:0/2735209413 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9c009570 con 0x7f6fb406c8a0 2023-10-11T09:53:16.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.818+0000 7f6fb37fe700 1 --2- 172.21.15.78:0/2735209413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fb4160de0 0x7f6fb40b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:53:16.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.818+0000 7f6fabfff700 1 --2- 172.21.15.78:0/2735209413 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6fb40cc230 0x7f6fb4163580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T09:53:16.831 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.818+0000 7f6fb2ffd700 1 --2- 172.21.15.78:0/2735209413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fb406c8a0 0x7f6fb40c5230 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f6fa400c410 tx=0x7f6fa400c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:53:16.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.818+0000 7f6fb0ff9700 1 -- 172.21.15.78:0/2735209413 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4005cd0 con 0x7f6fb406c8a0 2023-10-11T09:53:16.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.818+0000 7f6fb0ff9700 1 -- 172.21.15.78:0/2735209413 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa4005e30 con 0x7f6fb406c8a0 2023-10-11T09:53:16.832 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.818+0000 7f6fb990c700 1 -- 172.21.15.78:0/2735209413 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fb4162aa0 con 0x7f6fb406c8a0 2023-10-11T09:53:16.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.818+0000 7f6fb0ff9700 1 -- 172.21.15.78:0/2735209413 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa40072f0 con 0x7f6fb406c8a0 2023-10-11T09:53:16.833 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.818+0000 7f6fb990c700 1 -- 172.21.15.78:0/2735209413 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb4163060 con 0x7f6fb406c8a0 2023-10-11T09:53:16.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.822+0000 7f6fb990c700 1 -- 172.21.15.78:0/2735209413 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fb40cb3e0 con 0x7f6fb406c8a0 2023-10-11T09:53:16.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.822+0000 7f6fb0ff9700 1 -- 172.21.15.78:0/2735209413 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7f6fa4007450 con 0x7f6fb406c8a0 2023-10-11T09:53:16.834 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.822+0000 7f6fb0ff9700 1 --2- 172.21.15.78:0/2735209413 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6f94065ed0 0x7f6f94068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:53:16.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.822+0000 7f6fb0ff9700 1 -- 172.21.15.78:0/2735209413 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f6fa4082ae0 con 0x7f6fb406c8a0 2023-10-11T09:53:16.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.826+0000 7f6fabfff700 1 --2- 172.21.15.78:0/2735209413 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6f94065ed0 0x7f6f94068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:53:16.835 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.826+0000 7f6fabfff700 1 --2- 172.21.15.78:0/2735209413 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6f94065ed0 0x7f6f94068380 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f6fb40b4bc0 tx=0x7f6f9c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:53:16.839 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:16.830+0000 7f6fb0ff9700 1 -- 172.21.15.78:0/2735209413 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7f6fb40cb3e0 con 0x7f6fb406c8a0 2023-10-11T09:53:17.014 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.006+0000 7f6fb990c700 1 -- 172.21.15.78:0/2735209413 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f6fb40cb3e0 con 0x7f6fb406c8a0 2023-10-11T09:53:17.025 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.018+0000 7f6fb0ff9700 1 -- 172.21.15.78:0/2735209413 <== mon.0 v2:172.21.15.78:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f6fa40502e0 con 0x7f6fb406c8a0 2023-10-11T09:53:17.027 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.018+0000 7f6fb0ff9700 1 -- 172.21.15.78:0/2735209413 <== mon.0 v2:172.21.15.78:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v20)=0 v20) v1 ==== 123+0+0 (secure 0 0 0) 0x7f6fb40cb3e0 con 0x7f6fb406c8a0 2023-10-11T09:53:17.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.022+0000 7f6fb990c700 1 -- 172.21.15.78:0/2735209413 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6f94065ed0 msgr2=0x7f6f94068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:53:17.032 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.022+0000 7f6fb990c700 1 --2- 172.21.15.78:0/2735209413 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6f94065ed0 0x7f6f94068380 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f6fb40b4bc0 tx=0x7f6f9c008030 comp rx=0 tx=0).stop 2023-10-11T09:53:17.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.022+0000 7f6fb990c700 1 -- 172.21.15.78:0/2735209413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fb406c8a0 msgr2=0x7f6fb40c5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:53:17.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.022+0000 7f6fb990c700 1 --2- 172.21.15.78:0/2735209413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fb406c8a0 0x7f6fb40c5230 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f6fa400c410 tx=0x7f6fa400c7d0 comp rx=0 tx=0).stop 2023-10-11T09:53:17.033 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.022+0000 7f6fb990c700 1 -- 172.21.15.78:0/2735209413 shutdown_connections 2023-10-11T09:53:17.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.022+0000 7f6fb990c700 1 --2- 172.21.15.78:0/2735209413 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7f6f94065ed0 0x7f6f94068380 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:53:17.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.022+0000 7f6fb990c700 1 --2- 172.21.15.78:0/2735209413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fb4160de0 0x7f6fb40b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:53:17.034 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.022+0000 7f6fb990c700 1 --2- 172.21.15.78:0/2735209413 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6fb40cc230 0x7f6fb4163580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:53:17.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.022+0000 7f6fb990c700 1 --2- 172.21.15.78:0/2735209413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fb406c8a0 0x7f6fb40c5230 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:53:17.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.022+0000 7f6fb990c700 1 -- 172.21.15.78:0/2735209413 >> 172.21.15.78:0/2735209413 conn(0x7f6fb4073100 msgr2=0x7f6fb406fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:53:17.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.022+0000 7f6fb990c700 1 -- 172.21.15.78:0/2735209413 shutdown_connections 2023-10-11T09:53:17.035 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.022+0000 7f6fb990c700 1 -- 172.21.15.78:0/2735209413 wait complete. 2023-10-11T09:53:17.050 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-11T09:53:17.051 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T09:53:17.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.214+0000 7fbcf4876700 1 Processor -- start 2023-10-11T09:53:17.223 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.214+0000 7fbcf4876700 1 -- start start 2023-10-11T09:53:17.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.214+0000 7fbcf4876700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbcf015a800 0x7fbcf00b4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:53:17.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.214+0000 7fbcf4876700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fbcf00b0d20 0x7fbcf015e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:53:17.225 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.214+0000 7fbcf4876700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcf0070dc0 con 0x7fbcf006bb60 2023-10-11T09:53:17.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.214+0000 7fbcf4876700 1 -- --> v2:172.21.15.78:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcf00b46a0 con 0x7fbcf00b0d20 2023-10-11T09:53:17.226 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.214+0000 7fbcf4876700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcf0070ff0 con 0x7fbcf015a800 2023-10-11T09:53:17.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.214+0000 7fbceffff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fbcf00b0d20 0x7fbcf015e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:53:17.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.218+0000 7fbceffff700 1 --2- >> v2:172.21.15.78:3300/0 conn(0x7fbcf00b0d20 0x7fbcf015e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39052/0 (socket says 172.21.15.78:39052) 2023-10-11T09:53:17.227 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.218+0000 7fbceffff700 1 -- 172.21.15.78:0/3996680465 learned_addr learned my addr 172.21.15.78:0/3996680465 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:53:17.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.218+0000 7fbcef7fe700 1 --2- 172.21.15.78:0/3996680465 >> v2:172.21.15.133:3300/0 conn(0x7fbcf015a800 0x7fbcf00b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:53:17.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.218+0000 7fbceffff700 1 -- 172.21.15.78:0/3996680465 >> v2:172.21.15.133:3300/0 conn(0x7fbcf015a800 msgr2=0x7fbcf00b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T09:53:17.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.218+0000 7fbceffff700 1 --2- 172.21.15.78:0/3996680465 >> v2:172.21.15.133:3300/0 conn(0x7fbcf015a800 0x7fbcf00b4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:53:17.228 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.218+0000 7fbceffff700 1 -- 172.21.15.78:0/3996680465 >> v1:172.21.15.133:6789/0 conn(0x7fbcf006bb60 legacy=0x7fbcf00b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:53:17.229 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.218+0000 7fbceffff700 1 -- 172.21.15.78:0/3996680465 --> v2:172.21.15.78:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf00712d0 con 0x7fbcf00b0d20 2023-10-11T09:53:17.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.218+0000 7fbceffff700 1 --2- 172.21.15.78:0/3996680465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf00b0d20 0x7fbcf015e590 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fbcf00cbea0 tx=0x7fbce800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bdfe6b0a1c7ad99 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:53:17.230 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.218+0000 7fbceeffd700 1 -- 172.21.15.78:0/3996680465 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce800e070 con 0x7fbcf00b0d20 2023-10-11T09:53:17.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.218+0000 7fbceeffd700 1 -- 172.21.15.78:0/3996680465 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fbce800b9c0 con 0x7fbcf00b0d20 2023-10-11T09:53:17.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.218+0000 7fbceeffd700 1 -- 172.21.15.78:0/3996680465 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce800bd90 con 0x7fbcf00b0d20 2023-10-11T09:53:17.231 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.218+0000 7fbcf4876700 1 -- 172.21.15.78:0/3996680465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf00b0d20 msgr2=0x7fbcf015e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:53:17.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.218+0000 7fbcf4876700 1 --2- 172.21.15.78:0/3996680465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf00b0d20 0x7fbcf015e590 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fbcf00cbea0 tx=0x7fbce800b330 comp rx=0 tx=0).stop 2023-10-11T09:53:17.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.218+0000 7fbcf4876700 1 -- 172.21.15.78:0/3996680465 shutdown_connections 2023-10-11T09:53:17.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.218+0000 7fbcf4876700 1 --2- 172.21.15.78:0/3996680465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf00b0d20 0x7fbcf015e590 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:53:17.232 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.218+0000 7fbcf4876700 1 --2- 172.21.15.78:0/3996680465 >> v2:172.21.15.133:3300/0 conn(0x7fbcf015a800 0x7fbcf00b4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:53:17.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.218+0000 7fbcf4876700 1 -- 172.21.15.78:0/3996680465 >> 172.21.15.78:0/3996680465 conn(0x7fbcf00cf820 msgr2=0x7fbcf00cb610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:53:17.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.218+0000 7fbcf4876700 1 -- 172.21.15.78:0/3996680465 shutdown_connections 2023-10-11T09:53:17.233 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.218+0000 7fbcf4876700 1 -- 172.21.15.78:0/3996680465 wait complete. 2023-10-11T09:53:17.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.222+0000 7fbcf4876700 1 Processor -- start 2023-10-11T09:53:17.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.222+0000 7fbcf4876700 1 -- start start 2023-10-11T09:53:17.236 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.226+0000 7fbcf4876700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcf006bb60 0x7fbcf0150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:53:17.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.226+0000 7fbcf4876700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbcf00b0d20 0x7fbcf01efa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:53:17.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.226+0000 7fbcf4876700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf015a800 0x7fbcf0154000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:53:17.237 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.226+0000 7fbcf4876700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf01564e0 con 0x7fbcf015a800 2023-10-11T09:53:17.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.226+0000 7fbcf4876700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf0154540 con 0x7fbcf006bb60 2023-10-11T09:53:17.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.226+0000 7fbcf4876700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf0156730 con 0x7fbcf00b0d20 2023-10-11T09:53:17.238 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.226+0000 7fbceffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf015a800 0x7fbcf0154000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:53:17.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.226+0000 7fbceffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf015a800 0x7fbcf0154000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39060/0 (socket says 172.21.15.78:39060) 2023-10-11T09:53:17.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.226+0000 7fbceffff700 1 -- 172.21.15.78:0/1956071987 learned_addr learned my addr 172.21.15.78:0/1956071987 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-11T09:53:17.239 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.226+0000 7fbcef7fe700 1 --2- 172.21.15.78:0/1956071987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcf006bb60 0x7fbcf0150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:53:17.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.226+0000 7fbce6ffd700 1 --2- 172.21.15.78:0/1956071987 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbcf00b0d20 0x7fbcf01efa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:53:17.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.226+0000 7fbceffff700 1 -- 172.21.15.78:0/1956071987 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbcf00b0d20 msgr2=0x7fbcf01efa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:53:17.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.226+0000 7fbceffff700 1 --2- 172.21.15.78:0/1956071987 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbcf00b0d20 0x7fbcf01efa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:53:17.240 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.226+0000 7fbceffff700 1 -- 172.21.15.78:0/1956071987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcf006bb60 msgr2=0x7fbcf0150f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:53:17.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.226+0000 7fbceffff700 1 --2- 172.21.15.78:0/1956071987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcf006bb60 0x7fbcf0150f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:53:17.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.226+0000 7fbceffff700 1 -- 172.21.15.78:0/1956071987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce8009b30 con 0x7fbcf015a800 2023-10-11T09:53:17.241 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.226+0000 7fbceffff700 1 --2- 172.21.15.78:0/1956071987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf015a800 0x7fbcf0154000 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fbce8012000 tx=0x7fbce8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:53:17.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.226+0000 7fbcecff9700 1 -- 172.21.15.78:0/1956071987 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce8004450 con 0x7fbcf015a800 2023-10-11T09:53:17.243 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.226+0000 7fbcecff9700 1 -- 172.21.15.78:0/1956071987 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fbce80045d0 con 0x7fbcf015a800 2023-10-11T09:53:17.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.226+0000 7fbcecff9700 1 -- 172.21.15.78:0/1956071987 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce80099b0 con 0x7fbcf015a800 2023-10-11T09:53:17.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.230+0000 7fbcf4876700 1 -- 172.21.15.78:0/1956071987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcf0156a10 con 0x7fbcf015a800 2023-10-11T09:53:17.244 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.230+0000 7fbcf4876700 1 -- 172.21.15.78:0/1956071987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcf0156fd0 con 0x7fbcf015a800 2023-10-11T09:53:17.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.230+0000 7fbcf4876700 1 -- 172.21.15.78:0/1956071987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcf0067d00 con 0x7fbcf015a800 2023-10-11T09:53:17.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.230+0000 7fbcecff9700 1 -- 172.21.15.78:0/1956071987 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) v1 ==== 85536+0+0 (secure 0 0 0) 0x7fbce801c440 con 0x7fbcf015a800 2023-10-11T09:53:17.245 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.234+0000 7fbcecff9700 1 --2- 172.21.15.78:0/1956071987 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbcd0065ed0 0x7fbcd0068380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T09:53:17.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.234+0000 7fbcecff9700 1 -- 172.21.15.78:0/1956071987 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fbce8084d50 con 0x7fbcf015a800 2023-10-11T09:53:17.246 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.234+0000 7fbcef7fe700 1 --2- 172.21.15.78:0/1956071987 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbcd0065ed0 0x7fbcd0068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T09:53:17.247 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.234+0000 7fbcef7fe700 1 --2- 172.21.15.78:0/1956071987 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbcd0065ed0 0x7fbcd0068380 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fbce0006b40 tx=0x7fbce0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T09:53:17.250 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.242+0000 7fbcecff9700 1 -- 172.21.15.78:0/1956071987 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+171912 (secure 0 0 0) 0x7fbce80525e0 con 0x7fbcf015a800 2023-10-11T09:53:17.457 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.450+0000 7fbcf4876700 1 -- 172.21.15.78:0/1956071987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbcf01571f0 con 0x7fbcf015a800 2023-10-11T09:53:17.458 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.450+0000 7fbcecff9700 1 -- 172.21.15.78:0/1956071987 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+3532 (secure 0 0 0) 0x7fbce8055c00 con 0x7fbcf015a800 2023-10-11T09:53:17.458 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-11T09:53:17.459 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":114,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8452,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6836/2481257990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2481257990},{"type":"v1","addr":"172.21.15.78:6836","nonce":2481257990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":8464,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1035227282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1035227282},{"type":"v1","addr":"172.21.15.133:6835","nonce":1035227282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":8680,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/36737634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":36737634},{"type":"v1","addr":"172.21.15.133:6833","nonce":36737634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":101},{"gid":9546,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3163288353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6835","nonce":3163288353},{"type":"v1","addr":"172.21.15.78:6837","nonce":3163288353}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114}],"filesystems":[]} 2023-10-11T09:53:17.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.454+0000 7fbcf4876700 1 -- 172.21.15.78:0/1956071987 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbcd0065ed0 msgr2=0x7fbcd0068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:53:17.461 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.454+0000 7fbcf4876700 1 --2- 172.21.15.78:0/1956071987 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbcd0065ed0 0x7fbcd0068380 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fbce0006b40 tx=0x7fbce0008040 comp rx=0 tx=0).stop 2023-10-11T09:53:17.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.454+0000 7fbcf4876700 1 -- 172.21.15.78:0/1956071987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf015a800 msgr2=0x7fbcf0154000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T09:53:17.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.454+0000 7fbcf4876700 1 --2- 172.21.15.78:0/1956071987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf015a800 0x7fbcf0154000 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fbce8012000 tx=0x7fbce8004af0 comp rx=0 tx=0).stop 2023-10-11T09:53:17.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.454+0000 7fbcf4876700 1 -- 172.21.15.78:0/1956071987 shutdown_connections 2023-10-11T09:53:17.462 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.454+0000 7fbcf4876700 1 --2- 172.21.15.78:0/1956071987 >> [v2:172.21.15.78:6832/35606,v1:172.21.15.78:6833/35606] conn(0x7fbcd0065ed0 0x7fbcd0068380 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:53:17.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.454+0000 7fbcf4876700 1 --2- 172.21.15.78:0/1956071987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf015a800 0x7fbcf0154000 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:53:17.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.454+0000 7fbcf4876700 1 --2- 172.21.15.78:0/1956071987 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbcf00b0d20 0x7fbcf01efa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:53:17.463 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.454+0000 7fbcf4876700 1 --2- 172.21.15.78:0/1956071987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcf006bb60 0x7fbcf0150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T09:53:17.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.454+0000 7fbcf4876700 1 -- 172.21.15.78:0/1956071987 >> 172.21.15.78:0/1956071987 conn(0x7fbcf00cf820 msgr2=0x7fbcf015b290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T09:53:17.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.454+0000 7fbcf4876700 1 -- 172.21.15.78:0/1956071987 shutdown_connections 2023-10-11T09:53:17.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-11T09:53:17.454+0000 7fbcf4876700 1 -- 172.21.15.78:0/1956071987 wait complete. 2023-10-11T09:53:17.465 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 114 2023-10-11T09:53:17.480 DEBUG:tasks.cephfs.filesystem:already dead... 2023-10-11T09:53:17.480 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-11T09:53:17.481 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-11T09:53:17.481 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:53:17.502 INFO:tasks.ceph.mds.a:Stopped 2023-10-11T09:53:17.503 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-11T09:53:17.503 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:53:17.562 INFO:tasks.ceph.mds.c:Stopped 2023-10-11T09:53:17.563 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-11T09:53:17.563 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:53:17.586 INFO:tasks.ceph.mds.b:Stopped 2023-10-11T09:53:17.587 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-11T09:53:17.587 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:53:17.655 INFO:tasks.ceph.mds.d:Stopped 2023-10-11T09:53:17.655 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-11T09:53:17.656 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-11T09:53:17.656 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:53:17.732 INFO:tasks.ceph.osd.0:Stopped 2023-10-11T09:53:17.733 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-11T09:53:17.733 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:53:21.344 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2023-10-11T09:53:23.834 INFO:tasks.ceph.osd.1:Stopped 2023-10-11T09:53:23.835 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-11T09:53:23.836 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:53:23.901 INFO:tasks.ceph.osd.2:Stopped 2023-10-11T09:53:23.902 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-11T09:53:23.902 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:53:23.986 INFO:tasks.ceph.osd.3:Stopped 2023-10-11T09:53:23.987 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-11T09:53:23.987 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:53:24.079 INFO:tasks.ceph.osd.4:Stopped 2023-10-11T09:53:24.080 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-11T09:53:24.081 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:53:27.048 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2023-10-11T09:53:27.049 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2023-10-11T09:53:30.183 INFO:tasks.ceph.osd.5:Stopped 2023-10-11T09:53:30.184 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-11T09:53:30.184 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:53:30.270 INFO:tasks.ceph.osd.6:Stopped 2023-10-11T09:53:30.271 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-11T09:53:30.271 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:53:31.152 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:53:31.160 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:53:32.554 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2023-10-11T09:53:32.554 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2023-10-11T09:53:36.374 INFO:tasks.ceph.osd.7:Stopped 2023-10-11T09:53:36.374 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-11T09:53:36.375 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-11T09:53:36.375 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:53:36.434 INFO:tasks.ceph.mgr.y:Stopped 2023-10-11T09:53:36.435 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-11T09:53:36.435 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:53:36.491 INFO:tasks.ceph.mgr.x:Stopped 2023-10-11T09:53:36.491 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-11T09:53:36.492 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-11T09:53:36.492 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:53:36.523 INFO:tasks.ceph.mon.a:Stopped 2023-10-11T09:53:36.524 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-11T09:53:36.524 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:53:36.551 INFO:tasks.ceph.mon.b:Stopped 2023-10-11T09:53:36.552 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-11T09:53:36.552 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T09:53:36.628 INFO:tasks.ceph.mon.c:Stopped 2023-10-11T09:53:36.630 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-11T09:53:36.630 DEBUG:teuthology.orchestra.run.smithi078:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'bad backtrace' | egrep -v 'object missing on disk' | egrep -v 'error reading table object' | egrep -v 'error reading sessionmap' | egrep -v 'unmatched fragstat' | egrep -v 'unmatched rstat' | egrep -v 'was unreadable, recreating it now' | egrep -v 'Scrub error on inode' | egrep -v 'Metadata damage detected' | egrep -v 'inconsistent rstat on inode' | egrep -v 'Error recovering journal' | head -n 1 2023-10-11T09:53:36.682 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi078.front.sepia.ceph.com 2023-10-11T09:53:36.682 DEBUG:teuthology.orchestra.run.smithi078:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-11T09:53:37.210 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi078.front.sepia.ceph.com 2023-10-11T09:53:37.211 DEBUG:teuthology.orchestra.run.smithi078:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-11T09:53:37.555 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2023-10-11T09:53:37.641 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi078.front.sepia.ceph.com 2023-10-11T09:53:37.641 DEBUG:teuthology.orchestra.run.smithi078:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-11T09:53:38.061 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi078.front.sepia.ceph.com 2023-10-11T09:53:38.061 DEBUG:teuthology.orchestra.run.smithi078:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-11T09:53:38.466 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi133.front.sepia.ceph.com 2023-10-11T09:53:38.467 DEBUG:teuthology.orchestra.run.smithi133:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-11T09:53:38.993 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi133.front.sepia.ceph.com 2023-10-11T09:53:38.993 DEBUG:teuthology.orchestra.run.smithi133:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-11T09:53:39.519 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi133.front.sepia.ceph.com 2023-10-11T09:53:39.520 DEBUG:teuthology.orchestra.run.smithi133:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-11T09:53:39.975 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi133.front.sepia.ceph.com 2023-10-11T09:53:39.976 DEBUG:teuthology.orchestra.run.smithi133:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-11T09:53:40.372 INFO:tasks.ceph:Archiving mon data... 2023-10-11T09:53:40.374 DEBUG:teuthology.misc:Transferring archived files from smithi078:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi/7418830/data/mon.a.tgz 2023-10-11T09:53:40.374 DEBUG:teuthology.orchestra.run.smithi078:> mktemp 2023-10-11T09:53:40.394 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/tmp.z6834y9RMN 2023-10-11T09:53:40.394 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.z6834y9RMN 2023-10-11T09:53:40.578 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 0666 /tmp/tmp.z6834y9RMN 2023-10-11T09:53:40.678 DEBUG:teuthology.orchestra.remote:smithi078:/tmp/tmp.z6834y9RMN is 452KB 2023-10-11T09:53:40.759 DEBUG:teuthology.orchestra.run.smithi078:> rm -fr /tmp/tmp.z6834y9RMN 2023-10-11T09:53:40.777 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi/7418830/data/mon.b.tgz 2023-10-11T09:53:40.778 DEBUG:teuthology.orchestra.run.smithi133:> mktemp 2023-10-11T09:53:40.797 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/tmp.svE14eQZJk 2023-10-11T09:53:40.797 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.svE14eQZJk 2023-10-11T09:53:40.986 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0666 /tmp/tmp.svE14eQZJk 2023-10-11T09:53:41.081 DEBUG:teuthology.orchestra.remote:smithi133:/tmp/tmp.svE14eQZJk is 463KB 2023-10-11T09:53:41.157 DEBUG:teuthology.orchestra.run.smithi133:> rm -fr /tmp/tmp.svE14eQZJk 2023-10-11T09:53:41.176 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi/7418830/data/mon.c.tgz 2023-10-11T09:53:41.176 DEBUG:teuthology.orchestra.run.smithi133:> mktemp 2023-10-11T09:53:41.236 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/tmp.75nQlLDC6u 2023-10-11T09:53:41.237 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.75nQlLDC6u 2023-10-11T09:53:41.427 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0666 /tmp/tmp.75nQlLDC6u 2023-10-11T09:53:41.520 DEBUG:teuthology.orchestra.remote:smithi133:/tmp/tmp.75nQlLDC6u is 462KB 2023-10-11T09:53:41.597 DEBUG:teuthology.orchestra.run.smithi133:> rm -fr /tmp/tmp.75nQlLDC6u 2023-10-11T09:53:41.616 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-11T09:53:41.617 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-11T09:53:41.624 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-11T09:53:41.691 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-11T09:53:41.692 DEBUG:teuthology.orchestra.run.smithi078:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-11T09:53:41.706 DEBUG:teuthology.orchestra.run.smithi133:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-11T09:53:41.754 INFO:teuthology.orchestra.run.smithi078.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-11T09:53:41.788 INFO:teuthology.orchestra.run.smithi133.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-11T09:53:41.803 INFO:tasks.ceph:Archiving crash dumps... 2023-10-11T09:53:41.805 DEBUG:teuthology.misc:Transferring archived files from smithi078:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi/7418830/remote/smithi078/crash 2023-10-11T09:53:41.806 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-11T09:53:41.841 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi/7418830/remote/smithi133/crash 2023-10-11T09:53:41.842 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-11T09:53:41.878 INFO:tasks.ceph:Shutting down logrotate 2023-10-11T09:53:41.879 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:53:41.894 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T09:53:41.990 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-11T09:53:42.025 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-11T09:53:42.058 INFO:tasks.ceph:Compressing logs... 2023-10-11T09:53:42.058 DEBUG:teuthology.orchestra.run.smithi078:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-11T09:53:42.075 DEBUG:teuthology.orchestra.run.smithi133:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-11T09:53:46.567 INFO:tasks.ceph:Archiving logs... 2023-10-11T09:53:46.567 DEBUG:teuthology.misc:Transferring archived files from smithi078:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi/7418830/remote/smithi078/log 2023-10-11T09:53:46.569 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-11T09:53:47.722 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi/7418830/remote/smithi133/log 2023-10-11T09:53:47.723 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-11T09:53:48.201 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-11T09:53:48.212 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-11T09:53:48.212 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-11T09:53:48.220 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-11T09:53:48.310 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-11T09:53:48.321 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-11T09:53:48.322 DEBUG:teuthology.orchestra.run.smithi078:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-11T09:53:48.329 DEBUG:teuthology.orchestra.run.smithi133:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-11T09:53:48.364 INFO:teuthology.orchestra.run.smithi078.stdout: remote refid st t when poll reach delay offset jitter 2023-10-11T09:53:48.364 INFO:teuthology.orchestra.run.smithi078.stdout:============================================================================== 2023-10-11T09:53:48.364 INFO:teuthology.orchestra.run.smithi078.stdout:*hv01.front.sepi 44.4.53.2 2 u 21 64 377 0.105 5.016 1.067 2023-10-11T09:53:48.365 INFO:teuthology.orchestra.run.smithi078.stdout:+hv02.front.sepi 63.231.80.2 3 u 30 64 377 0.103 3.027 1.100 2023-10-11T09:53:48.365 INFO:teuthology.orchestra.run.smithi078.stdout:+hv03.front.sepi 74.6.168.72 3 u 36 64 377 0.101 0.828 1.402 2023-10-11T09:53:48.365 INFO:teuthology.orchestra.run.smithi078.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 0.000 0.000 2023-10-11T09:53:48.373 INFO:teuthology.orchestra.run.smithi133.stdout: remote refid st t when poll reach delay offset jitter 2023-10-11T09:53:48.374 INFO:teuthology.orchestra.run.smithi133.stdout:============================================================================== 2023-10-11T09:53:48.374 INFO:teuthology.orchestra.run.smithi133.stdout:*hv01.front.sepi 44.4.53.2 2 u 26 64 377 0.100 2.785 0.961 2023-10-11T09:53:48.374 INFO:teuthology.orchestra.run.smithi133.stdout:+hv02.front.sepi 63.231.80.2 3 u 24 64 377 0.100 2.945 1.054 2023-10-11T09:53:48.375 INFO:teuthology.orchestra.run.smithi133.stdout:+hv03.front.sepi 74.6.168.72 3 u 15 64 377 0.108 0.072 0.861 2023-10-11T09:53:48.375 INFO:teuthology.orchestra.run.smithi133.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 0.000 0.000 2023-10-11T09:53:48.376 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-11T09:53:48.386 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-11T09:53:48.387 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-11T09:53:48.398 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-11T09:53:48.410 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-11T09:53:48.422 INFO:teuthology.task.internal:Duration was 1549.702234 seconds 2023-10-11T09:53:48.422 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-11T09:53:48.434 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-11T09:53:48.434 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-11T09:53:48.441 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-11T09:53:48.595 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-11T09:53:48.595 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi078.front.sepia.ceph.com 2023-10-11T09:53:48.596 DEBUG:teuthology.orchestra.run.smithi078:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-11T09:53:48.632 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi133.front.sepia.ceph.com 2023-10-11T09:53:48.632 DEBUG:teuthology.orchestra.run.smithi133:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-11T09:53:48.669 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-11T09:53:48.670 DEBUG:teuthology.orchestra.run.smithi078:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-11T09:53:48.686 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-11T09:53:48.775 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-11T09:53:48.775 DEBUG:teuthology.orchestra.run.smithi078:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-11T09:53:48.783 DEBUG:teuthology.orchestra.run.smithi133:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-11T09:53:48.892 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-11T09:53:48.903 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-11T09:53:48.903 DEBUG:teuthology.orchestra.run.smithi078:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-11T09:53:48.937 DEBUG:teuthology.orchestra.run.smithi133:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-11T09:53:48.974 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-11T09:53:48.984 DEBUG:teuthology.orchestra.run.smithi078:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-11T09:53:48.992 DEBUG:teuthology.orchestra.run.smithi133:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-11T09:53:49.019 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern = core 2023-10-11T09:53:49.051 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = core 2023-10-11T09:53:49.086 DEBUG:teuthology.orchestra.run.smithi078:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-11T09:53:49.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T09:53:49.115 DEBUG:teuthology.orchestra.run.smithi133:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-11T09:53:49.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T09:53:49.143 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-11T09:53:49.155 INFO:teuthology.task.internal:Transferring archived files... 2023-10-11T09:53:49.156 DEBUG:teuthology.misc:Transferring archived files from smithi078:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi/7418830/remote/smithi078 2023-10-11T09:53:49.156 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-11T09:53:49.202 DEBUG:teuthology.misc:Transferring archived files from smithi133:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi/7418830/remote/smithi133 2023-10-11T09:53:49.202 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-11T09:53:49.243 INFO:teuthology.task.internal:Removing archive directory... 2023-10-11T09:53:49.244 DEBUG:teuthology.orchestra.run.smithi078:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-11T09:53:49.251 DEBUG:teuthology.orchestra.run.smithi133:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-11T09:53:49.303 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-11T09:53:49.314 INFO:teuthology.task.internal:Not uploading archives. 2023-10-11T09:53:49.315 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-11T09:53:49.326 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-11T09:53:49.327 DEBUG:teuthology.orchestra.run.smithi078:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-11T09:53:49.334 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-11T09:53:49.350 INFO:teuthology.orchestra.run.smithi078.stdout: 1048601 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 11 09:53 /home/ubuntu/cephtest 2023-10-11T09:53:49.367 INFO:teuthology.orchestra.run.smithi133.stdout: 1048601 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 11 09:53 /home/ubuntu/cephtest 2023-10-11T09:53:49.374 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-11T09:53:49.390 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-11T09:53:49.413 INFO:teuthology.nuke:Checking targets against current locks 2023-10-11T09:53:49.442 DEBUG:teuthology.nuke:shortname: smithi078 2023-10-11T09:53:49.443 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-11T09:53:49.466 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi078.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi/7418830', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-11 09:16:51.654493', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2023-10-11T09:53:49.467 DEBUG:teuthology.nuke:shortname: smithi133 2023-10-11T09:53:49.468 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-11T09:53:49.492 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi133.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi/7418830', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-11 09:16:51.652908', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2023-10-11T09:53:49.518 INFO:teuthology.orchestra.console.smithi078:Power off 2023-10-11T09:53:49.519 DEBUG:teuthology.orchestra.console.smithi078:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-11T09:53:49.560 INFO:teuthology.orchestra.console.smithi133:Power off 2023-10-11T09:53:49.562 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-11T09:53:49.571 DEBUG:teuthology.orchestra.console.smithi078:power off output: Chassis Power Control: Down/Off 2023-10-11T09:53:49.572 DEBUG:teuthology.orchestra.console.smithi078:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-11T09:53:49.583 DEBUG:teuthology.orchestra.console.smithi133:power off output: Chassis Power Control: Down/Off 2023-10-11T09:53:49.584 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-11T09:53:49.594 DEBUG:teuthology.orchestra.console.smithi078:check power output: Chassis Power is on 2023-10-11T09:53:49.605 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is on 2023-10-11T09:53:53.595 DEBUG:teuthology.orchestra.console.smithi078:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-11T09:53:53.606 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-11T09:53:53.706 DEBUG:teuthology.orchestra.console.smithi078:check power output: Chassis Power is on 2023-10-11T09:53:53.715 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is on 2023-10-11T09:53:57.707 DEBUG:teuthology.orchestra.console.smithi078:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-11T09:53:57.716 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-11T09:53:57.818 DEBUG:teuthology.orchestra.console.smithi078:check power output: Chassis Power is off 2023-10-11T09:53:57.826 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is on 2023-10-11T09:53:57.920 INFO:teuthology.orchestra.console.smithi078:Power off completed 2023-10-11T09:54:01.830 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-11T09:54:01.941 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is off 2023-10-11T09:54:02.042 INFO:teuthology.orchestra.console.smithi133:Power off completed 2023-10-11T09:54:02.147 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/data-scan} duration: 1549.7022337913513 failure_reason: 'Test failure: test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan)' flavor: default mon.a-kernel-sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c mon.b-kernel-sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c owner: scheduled_vshankar@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=4ea49fd8c9cb4ec6a0f0a5c2e48d7150 status: fail success: false 2023-10-11T09:54:02.148 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-11T09:54:02.231 INFO:teuthology.run:FAIL